2023-09-23T02:58:35.007 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-23T02:58:35.007 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_vshankar@teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/data-scan} --name vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi --archive /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401456 -- /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401456/orig.config.yaml 2023-09-23T02:58:35.050 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T02:58:35.135 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401456 branch: wip-vshankar-testing-20230922.144532 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/data-scan} email: vshankar@redhat.com first_in_suite: false job_id: '7401456' kernel: flavor: default kdb: true sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20230922.144532 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - bad backtrace - object missing on disk - error reading table object - error reading sessionmap - unmatched fragstat - unmatched rstat - was unreadable, recreating it now - Scrub error on inode - Metadata damage detected - inconsistent rstat on inode - Error recovering journal sha1: 862663913a74a99e1c52d50e6a7e59dd6705749a ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 862663913a74a99e1c52d50e6a7e59dd6705749a thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20230922.144532 sha1: 862663913a74a99e1c52d50e6a7e59dd6705749a owner: scheduled_vshankar@teuthology priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 8196 sha1: 862663913a74a99e1c52d50e6a7e59dd6705749a sleep_before_teardown: 0 subset: 57/1024 suite: fs suite_branch: wip-vshankar-testing-20230922.144532 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_862663913a74a99e1c52d50e6a7e59dd6705749a/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 862663913a74a99e1c52d50e6a7e59dd6705749a syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi154.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF4iXVXCH7ifGpuKynAenSBvP8YukEyWTkkNqHusRK804qngv2s+hXqAV9tTcZI3im5o9ajvhnPaepqhn47Uq8o= smithi190.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHJ5AsjgG46Z7O7yPa20pbgWscDSfsOFTIUe566npy1eyPjD925g8/ay8Tkm9pywa2BcrKkpubGa3HYMD0SDJXE= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_data_scan teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-09-23_02:50:30 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-09-23T02:58:35.135 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_862663913a74a99e1c52d50e6a7e59dd6705749a/qa; will attempt to use it 2023-09-23T02:58:35.136 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_862663913a74a99e1c52d50e6a7e59dd6705749a/qa/tasks 2023-09-23T02:58:35.136 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-09-23T02:58:35.137 INFO:teuthology.task.internal:Checking packages... 2023-09-23T02:58:35.160 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '862663913a74a99e1c52d50e6a7e59dd6705749a' 2023-09-23T02:58:35.160 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-09-23T02:58:35.160 INFO:teuthology.packaging:ref: None 2023-09-23T02:58:35.160 INFO:teuthology.packaging:tag: None 2023-09-23T02:58:35.160 INFO:teuthology.packaging:branch: wip-vshankar-testing-20230922.144532 2023-09-23T02:58:35.160 INFO:teuthology.packaging:sha1: 862663913a74a99e1c52d50e6a7e59dd6705749a 2023-09-23T02:58:35.160 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20230922.144532 2023-09-23T02:58:35.380 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6336.g86266391 2023-09-23T02:58:35.381 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-09-23T02:58:35.388 INFO:teuthology.task.internal:no buildpackages task found 2023-09-23T02:58:35.388 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-09-23T02:58:35.395 INFO:teuthology.task.internal:Saving configuration 2023-09-23T02:58:35.411 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-09-23T02:58:35.416 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-23T02:58:35.442 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi154.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401456', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-23 02:51:39.834710', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF4iXVXCH7ifGpuKynAenSBvP8YukEyWTkkNqHusRK804qngv2s+hXqAV9tTcZI3im5o9ajvhnPaepqhn47Uq8o='} 2023-09-23T02:58:35.464 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi190.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401456', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-23 02:51:39.833090', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHJ5AsjgG46Z7O7yPa20pbgWscDSfsOFTIUe566npy1eyPjD925g8/ay8Tkm9pywa2BcrKkpubGa3HYMD0SDJXE='} 2023-09-23T02:58:35.465 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-09-23T02:58:35.470 INFO:teuthology.task.internal:roles: ubuntu@smithi154.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-09-23T02:58:35.470 INFO:teuthology.task.internal:roles: ubuntu@smithi190.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-09-23T02:58:35.471 INFO:teuthology.run_tasks:Running task console_log... 2023-09-23T02:58:35.545 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc9fd234b80>, signals=[15]) 2023-09-23T02:58:35.545 INFO:teuthology.run_tasks:Running task internal.connect... 2023-09-23T02:58:35.551 INFO:teuthology.task.internal:Opening connections... 2023-09-23T02:58:35.551 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi154.front.sepia.ceph.com 2023-09-23T02:58:35.552 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T02:58:35.635 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi190.front.sepia.ceph.com 2023-09-23T02:58:35.636 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T02:58:35.718 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-09-23T02:58:35.725 DEBUG:teuthology.orchestra.run.smithi154:> uname -m 2023-09-23T02:58:35.742 INFO:teuthology.orchestra.run.smithi154.stdout:x86_64 2023-09-23T02:58:35.742 DEBUG:teuthology.orchestra.run.smithi154:> cat /etc/os-release 2023-09-23T02:58:35.799 INFO:teuthology.orchestra.run.smithi154.stdout:NAME="CentOS Stream" 2023-09-23T02:58:35.799 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION="9" 2023-09-23T02:58:35.799 INFO:teuthology.orchestra.run.smithi154.stdout:ID="centos" 2023-09-23T02:58:35.799 INFO:teuthology.orchestra.run.smithi154.stdout:ID_LIKE="rhel fedora" 2023-09-23T02:58:35.799 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_ID="9" 2023-09-23T02:58:35.799 INFO:teuthology.orchestra.run.smithi154.stdout:PLATFORM_ID="platform:el9" 2023-09-23T02:58:35.800 INFO:teuthology.orchestra.run.smithi154.stdout:PRETTY_NAME="CentOS Stream 9" 2023-09-23T02:58:35.800 INFO:teuthology.orchestra.run.smithi154.stdout:ANSI_COLOR="0;31" 2023-09-23T02:58:35.800 INFO:teuthology.orchestra.run.smithi154.stdout:LOGO="fedora-logo-icon" 2023-09-23T02:58:35.800 INFO:teuthology.orchestra.run.smithi154.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-09-23T02:58:35.800 INFO:teuthology.orchestra.run.smithi154.stdout:HOME_URL="https://centos.org/" 2023-09-23T02:58:35.800 INFO:teuthology.orchestra.run.smithi154.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-23T02:58:35.800 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-09-23T02:58:35.800 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-09-23T02:58:35.801 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server 2023-09-23T02:58:35.828 DEBUG:teuthology.orchestra.run.smithi190:> uname -m 2023-09-23T02:58:35.845 INFO:teuthology.orchestra.run.smithi190.stdout:x86_64 2023-09-23T02:58:35.846 DEBUG:teuthology.orchestra.run.smithi190:> cat /etc/os-release 2023-09-23T02:58:35.902 INFO:teuthology.orchestra.run.smithi190.stdout:NAME="CentOS Stream" 2023-09-23T02:58:35.902 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION="9" 2023-09-23T02:58:35.902 INFO:teuthology.orchestra.run.smithi190.stdout:ID="centos" 2023-09-23T02:58:35.903 INFO:teuthology.orchestra.run.smithi190.stdout:ID_LIKE="rhel fedora" 2023-09-23T02:58:35.903 INFO:teuthology.orchestra.run.smithi190.stdout:VERSION_ID="9" 2023-09-23T02:58:35.903 INFO:teuthology.orchestra.run.smithi190.stdout:PLATFORM_ID="platform:el9" 2023-09-23T02:58:35.903 INFO:teuthology.orchestra.run.smithi190.stdout:PRETTY_NAME="CentOS Stream 9" 2023-09-23T02:58:35.903 INFO:teuthology.orchestra.run.smithi190.stdout:ANSI_COLOR="0;31" 2023-09-23T02:58:35.903 INFO:teuthology.orchestra.run.smithi190.stdout:LOGO="fedora-logo-icon" 2023-09-23T02:58:35.904 INFO:teuthology.orchestra.run.smithi190.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-09-23T02:58:35.904 INFO:teuthology.orchestra.run.smithi190.stdout:HOME_URL="https://centos.org/" 2023-09-23T02:58:35.904 INFO:teuthology.orchestra.run.smithi190.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-23T02:58:35.904 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-09-23T02:58:35.904 INFO:teuthology.orchestra.run.smithi190.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-09-23T02:58:35.905 INFO:teuthology.lock.ops:Updating smithi190.front.sepia.ceph.com on lock server 2023-09-23T02:58:35.927 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-09-23T02:58:35.937 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-09-23T02:58:35.944 INFO:teuthology.task.internal:Checking for old test directory... 2023-09-23T02:58:35.944 DEBUG:teuthology.orchestra.run.smithi154:> test '!' -e /home/ubuntu/cephtest 2023-09-23T02:58:35.947 DEBUG:teuthology.orchestra.run.smithi190:> test '!' -e /home/ubuntu/cephtest 2023-09-23T02:58:35.962 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-09-23T02:58:35.968 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-09-23T02:58:35.969 DEBUG:teuthology.orchestra.run.smithi154:> test -z $(ls -A /var/lib/ceph) 2023-09-23T02:58:36.003 DEBUG:teuthology.orchestra.run.smithi190:> test -z $(ls -A /var/lib/ceph) 2023-09-23T02:58:36.166 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-09-23T02:58:36.217 INFO:teuthology.run_tasks:Running task kernel... 2023-09-23T02:58:36.231 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': 'fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c'} 2023-09-23T02:58:36.231 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': 'fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': 'fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c'}}, timeout 300 2023-09-23T02:58:36.232 DEBUG:teuthology.orchestra.run.smithi154:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-23T02:58:36.232 DEBUG:teuthology.orchestra.run.smithi190:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-23T02:58:36.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T02:58:36.248 DEBUG:teuthology.task.kernel:sha1 for mon.a is fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-09-23T02:58:36.248 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c"... 2023-09-23T02:58:36.249 DEBUG:teuthology.orchestra.run.smithi154:> uname -r 2023-09-23T02:58:36.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T02:58:36.249 DEBUG:teuthology.task.kernel:sha1 for mon.b is fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-09-23T02:58:36.250 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c"... 2023-09-23T02:58:36.250 DEBUG:teuthology.orchestra.run.smithi190:> uname -r 2023-09-23T02:58:36.305 INFO:teuthology.orchestra.run.smithi154.stdout:5.14.0-361.el9.x86_64 2023-09-23T02:58:36.306 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-09-23T02:58:36.306 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-09-23T02:58:36.306 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-09-23T02:58:36.310 INFO:teuthology.orchestra.run.smithi190.stdout:5.14.0-361.el9.x86_64 2023-09-23T02:58:36.310 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-09-23T02:58:36.310 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-09-23T02:58:36.310 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-09-23T02:58:36.541 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum upgrade -y linux-firmware 2023-09-23T02:58:36.549 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum upgrade -y linux-firmware 2023-09-23T02:58:38.886 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS-9 - AppStream 8.5 MB/s | 6.3 MB 00:00 2023-09-23T02:58:38.986 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS-9 - AppStream 7.6 MB/s | 6.3 MB 00:00 2023-09-23T02:58:40.303 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS-9 - BaseOS 5.1 MB/s | 2.0 MB 00:00 2023-09-23T02:58:40.611 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS-9 - BaseOS 3.3 MB/s | 2.0 MB 00:00 2023-09-23T02:58:41.594 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - BaseOS 8.0 MB/s | 7.8 MB 00:00 2023-09-23T02:58:42.552 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - BaseOS 4.8 MB/s | 7.8 MB 00:01 2023-09-23T02:58:45.136 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - AppStream 11 MB/s | 18 MB 00:01 2023-09-23T02:58:45.978 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - AppStream 5.2 MB/s | 18 MB 00:03 2023-09-23T02:58:50.187 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - CRB 3.2 MB/s | 5.7 MB 00:01 2023-09-23T02:58:52.005 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 14 kB 00:00 2023-09-23T02:58:52.328 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - CRB 1.9 MB/s | 5.7 MB 00:03 2023-09-23T02:58:52.972 INFO:teuthology.orchestra.run.smithi190.stdout:Extra Packages for Enterprise Linux 21 MB/s | 19 MB 00:00 2023-09-23T02:58:53.770 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 14 kB 00:00 2023-09-23T02:58:55.599 INFO:teuthology.orchestra.run.smithi154.stdout:Extra Packages for Enterprise Linux 11 MB/s | 19 MB 00:01 2023-09-23T02:58:57.322 INFO:teuthology.orchestra.run.smithi190.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2023-09-23T02:58:59.338 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2023-09-23T02:58:59.339 INFO:teuthology.orchestra.run.smithi190.stdout:Nothing to do. 2023-09-23T02:58:59.340 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2023-09-23T02:58:59.442 INFO:teuthology.task.kernel:Downloading kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on mon.b... 2023-09-23T02:58:59.442 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-09-23T02:58:59.662 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/9/flavors/default/x86_64/ 2023-09-23T02:58:59.663 DEBUG:teuthology.orchestra.run.smithi190:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/9/flavors/default/x86_64/ --input-file=- 2023-09-23T02:58:59.665 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi190.front.sepia.ceph.com to complete... 2023-09-23T02:59:00.021 INFO:teuthology.orchestra.run.smithi154.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2023-09-23T02:59:02.008 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2023-09-23T02:59:02.009 INFO:teuthology.orchestra.run.smithi154.stdout:Nothing to do. 2023-09-23T02:59:02.009 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2023-09-23T02:59:02.099 INFO:teuthology.task.kernel:Downloading kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on mon.a... 2023-09-23T02:59:02.100 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-09-23T02:59:02.319 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/9/flavors/default/x86_64/ 2023-09-23T02:59:02.812 DEBUG:teuthology.orchestra.run.smithi154:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/9/flavors/default/x86_64/ --input-file=- 2023-09-23T02:59:02.815 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi154.front.sepia.ceph.com to complete... 2023-09-23T02:59:21.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23 02:59:21 URL:https://4.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/9/flavors/default/x86_64/kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm [318991812/318991812] -> "/tmp/kernel.x86_64.rpm" [1] 2023-09-23T02:59:21.212 INFO:teuthology.orchestra.run.smithi154.stderr:FINISHED --2023-09-23 02:59:21-- 2023-09-23T02:59:21.212 INFO:teuthology.orchestra.run.smithi154.stderr:Total wall clock time: 18s 2023-09-23T02:59:21.212 INFO:teuthology.orchestra.run.smithi154.stderr:Downloaded: 1 files, 304M in 18s (16.8 MB/s) 2023-09-23T02:59:21.213 INFO:teuthology.task.kernel:Installing kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on mon.a... 2023-09-23T02:59:21.214 DEBUG:teuthology.orchestra.run.smithi154:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-09-23T02:59:23.108 INFO:teuthology.orchestra.run.smithi154.stdout:Verifying... ######################################## 2023-09-23T02:59:23.273 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing... ######################################## 2023-09-23T02:59:23.401 INFO:teuthology.orchestra.run.smithi154.stdout:Updating / installing... 2023-09-23T02:59:30.437 INFO:teuthology.orchestra.run.smithi154.stdout:kernel-6.5.0_rc7_gfef64f93da2b-1 ######################################## 2023-09-23T02:59:49.656 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23 02:59:49 URL:https://4.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/9/flavors/default/x86_64/kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm [318991812/318991812] -> "/tmp/kernel.x86_64.rpm" [1] 2023-09-23T02:59:49.656 INFO:teuthology.orchestra.run.smithi190.stderr:FINISHED --2023-09-23 02:59:49-- 2023-09-23T02:59:49.656 INFO:teuthology.orchestra.run.smithi190.stderr:Total wall clock time: 50s 2023-09-23T02:59:49.656 INFO:teuthology.orchestra.run.smithi190.stderr:Downloaded: 1 files, 304M in 49s (6.17 MB/s) 2023-09-23T02:59:49.657 INFO:teuthology.task.kernel:Installing kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on mon.b... 2023-09-23T02:59:49.657 DEBUG:teuthology.orchestra.run.smithi190:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-09-23T02:59:51.554 INFO:teuthology.orchestra.run.smithi190.stdout:Verifying... ######################################## 2023-09-23T02:59:51.791 INFO:teuthology.orchestra.run.smithi190.stdout:Preparing... ######################################## 2023-09-23T02:59:51.932 INFO:teuthology.orchestra.run.smithi190.stdout:Updating / installing... 2023-09-23T02:59:59.015 INFO:teuthology.orchestra.run.smithi190.stdout:kernel-6.5.0_rc7_gfef64f93da2b-1 ######################################## 2023-09-23T03:00:06.287 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi154.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-09-23T03:00:06.287 DEBUG:teuthology.orchestra.run.smithi154:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-09-23T03:00:06.367 INFO:teuthology.orchestra.run.smithi154.stdout:/boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-09-23T03:00:06.367 INFO:teuthology.orchestra.run.smithi154.stdout:/boot/config-6.5.0-rc7-gfef64f93da2b 2023-09-23T03:00:06.367 INFO:teuthology.orchestra.run.smithi154.stdout:/boot/vmlinuz-6.5.0-rc7-gfef64f93da2b 2023-09-23T03:00:06.368 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b 2023-09-23T03:00:06.368 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel 2023-09-23T03:00:06.368 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch 2023-09-23T03:00:06.368 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86 2023-09-23T03:00:06.368 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto 2023-09-23T03:00:06.368 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-09-23T03:00:06.368 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-09-23T03:00:06.368 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-09-23T03:00:06.368 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-09-23T03:00:06.368 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-09-23T03:00:06.368 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-09-23T03:00:06.369 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-09-23T03:00:06.369 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32c-intel.ko 2023-09-23T03:00:06.369 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-09-23T03:00:06.369 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-09-23T03:00:06.369 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-09-23T03:00:06.369 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-09-23T03:00:06.369 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-09-23T03:00:06.369 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-09-23T03:00:06.369 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-09-23T03:00:06.369 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-09-23T03:00:06.369 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-09-23T03:00:06.369 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-09-23T03:00:06.370 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-09-23T03:00:06.370 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-09-23T03:00:06.370 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel 2023-09-23T03:00:06.370 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu 2023-09-23T03:00:06.370 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce 2023-09-23T03:00:06.370 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-09-23T03:00:06.370 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm 2023-09-23T03:00:06.370 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-amd.ko 2023-09-23T03:00:06.370 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-intel.ko 2023-09-23T03:00:06.370 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm.ko 2023-09-23T03:00:06.370 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto 2023-09-23T03:00:06.371 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ansi_cprng.ko 2023-09-23T03:00:06.371 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/anubis.ko 2023-09-23T03:00:06.371 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/arc4.ko 2023-09-23T03:00:06.371 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx 2023-09-23T03:00:06.371 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_memcpy.ko 2023-09-23T03:00:06.371 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_pq.ko 2023-09-23T03:00:06.371 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-09-23T03:00:06.371 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_tx.ko 2023-09-23T03:00:06.371 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_xor.ko 2023-09-23T03:00:06.371 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/raid6test.ko 2023-09-23T03:00:06.371 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authenc.ko 2023-09-23T03:00:06.372 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authencesn.ko 2023-09-23T03:00:06.372 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blake2b_generic.ko 2023-09-23T03:00:06.372 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_common.ko 2023-09-23T03:00:06.372 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_generic.ko 2023-09-23T03:00:06.372 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/camellia_generic.ko 2023-09-23T03:00:06.372 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast5_generic.ko 2023-09-23T03:00:06.372 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast6_generic.ko 2023-09-23T03:00:06.372 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast_common.ko 2023-09-23T03:00:06.372 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ccm.ko 2023-09-23T03:00:06.372 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cmac.ko 2023-09-23T03:00:06.372 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crc32_generic.ko 2023-09-23T03:00:06.372 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_engine.ko 2023-09-23T03:00:06.372 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_user.ko 2023-09-23T03:00:06.373 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cts.ko 2023-09-23T03:00:06.373 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/des_generic.ko 2023-09-23T03:00:06.373 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/dh_generic.ko 2023-09-23T03:00:06.373 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/echainiv.ko 2023-09-23T03:00:06.373 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/essiv.ko 2023-09-23T03:00:06.373 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/fcrypt.ko 2023-09-23T03:00:06.373 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/khazad.ko 2023-09-23T03:00:06.373 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4.ko 2023-09-23T03:00:06.373 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4hc.ko 2023-09-23T03:00:06.373 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/md4.ko 2023-09-23T03:00:06.373 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/michael_mic.ko 2023-09-23T03:00:06.373 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcbc.ko 2023-09-23T03:00:06.374 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcrypt.ko 2023-09-23T03:00:06.374 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/rmd160.ko 2023-09-23T03:00:06.374 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/seed.ko 2023-09-23T03:00:06.374 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/serpent_generic.ko 2023-09-23T03:00:06.374 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tcrypt.ko 2023-09-23T03:00:06.374 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tea.ko 2023-09-23T03:00:06.374 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_common.ko 2023-09-23T03:00:06.374 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_generic.ko 2023-09-23T03:00:06.374 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/vmac.ko 2023-09-23T03:00:06.374 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/wp512.ko 2023-09-23T03:00:06.374 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xcbc.ko 2023-09-23T03:00:06.374 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xor.ko 2023-09-23T03:00:06.375 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xxhash_generic.ko 2023-09-23T03:00:06.375 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers 2023-09-23T03:00:06.375 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi 2023-09-23T03:00:06.375 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_ipmi.ko 2023-09-23T03:00:06.375 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_pad.ko 2023-09-23T03:00:06.375 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/custom_method.ko 2023-09-23T03:00:06.375 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/ec_sys.ko 2023-09-23T03:00:06.375 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/platform_profile.ko 2023-09-23T03:00:06.375 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbs.ko 2023-09-23T03:00:06.375 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbshc.ko 2023-09-23T03:00:06.375 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/video.ko 2023-09-23T03:00:06.375 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata 2023-09-23T03:00:06.376 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/acard-ahci.ko 2023-09-23T03:00:06.376 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ahci_platform.ko 2023-09-23T03:00:06.376 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ata_generic.ko 2023-09-23T03:00:06.376 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/libahci_platform.ko 2023-09-23T03:00:06.376 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_acpi.ko 2023-09-23T03:00:06.376 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ali.ko 2023-09-23T03:00:06.376 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_amd.ko 2023-09-23T03:00:06.376 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_artop.ko 2023-09-23T03:00:06.376 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atiixp.ko 2023-09-23T03:00:06.376 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atp867x.ko 2023-09-23T03:00:06.376 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd640.ko 2023-09-23T03:00:06.376 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd64x.ko 2023-09-23T03:00:06.377 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cypress.ko 2023-09-23T03:00:06.377 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_efar.ko 2023-09-23T03:00:06.377 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt366.ko 2023-09-23T03:00:06.377 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt37x.ko 2023-09-23T03:00:06.377 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-09-23T03:00:06.377 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x3.ko 2023-09-23T03:00:06.377 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it8213.ko 2023-09-23T03:00:06.377 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it821x.ko 2023-09-23T03:00:06.377 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_jmicron.ko 2023-09-23T03:00:06.377 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_marvell.ko 2023-09-23T03:00:06.377 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_mpiix.ko 2023-09-23T03:00:06.377 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_netcell.ko 2023-09-23T03:00:06.378 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ninja32.ko 2023-09-23T03:00:06.378 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87410.ko 2023-09-23T03:00:06.378 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87415.ko 2023-09-23T03:00:06.378 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_oldpiix.ko 2023-09-23T03:00:06.378 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_opti.ko 2023-09-23T03:00:06.378 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_optidma.ko 2023-09-23T03:00:06.378 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pcmcia.ko 2023-09-23T03:00:06.378 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc2027x.ko 2023-09-23T03:00:06.378 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-09-23T03:00:06.378 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_piccolo.ko 2023-09-23T03:00:06.378 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_rdc.ko 2023-09-23T03:00:06.379 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sch.ko 2023-09-23T03:00:06.379 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_serverworks.ko 2023-09-23T03:00:06.379 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sil680.ko 2023-09-23T03:00:06.379 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sis.ko 2023-09-23T03:00:06.379 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sl82c105.ko 2023-09-23T03:00:06.379 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_triflex.ko 2023-09-23T03:00:06.379 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_via.ko 2023-09-23T03:00:06.379 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pdc_adma.ko 2023-09-23T03:00:06.379 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_inic162x.ko 2023-09-23T03:00:06.379 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_mv.ko 2023-09-23T03:00:06.379 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_nv.ko 2023-09-23T03:00:06.379 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_promise.ko 2023-09-23T03:00:06.379 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_qstor.ko 2023-09-23T03:00:06.380 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil.ko 2023-09-23T03:00:06.380 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil24.ko 2023-09-23T03:00:06.380 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sis.ko 2023-09-23T03:00:06.380 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_svw.ko 2023-09-23T03:00:06.380 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sx4.ko 2023-09-23T03:00:06.380 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_uli.ko 2023-09-23T03:00:06.380 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_via.ko 2023-09-23T03:00:06.380 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_vsc.ko 2023-09-23T03:00:06.380 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay 2023-09-23T03:00:06.380 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay/ks0108.ko 2023-09-23T03:00:06.380 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base 2023-09-23T03:00:06.380 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap 2023-09-23T03:00:06.381 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-i2c.ko 2023-09-23T03:00:06.381 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-mmio.ko 2023-09-23T03:00:06.381 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma 2023-09-23T03:00:06.381 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma/bcma.ko 2023-09-23T03:00:06.381 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block 2023-09-23T03:00:06.381 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe 2023-09-23T03:00:06.381 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe/aoe.ko 2023-09-23T03:00:06.381 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/brd.ko 2023-09-23T03:00:06.381 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd 2023-09-23T03:00:06.381 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd/drbd.ko 2023-09-23T03:00:06.381 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/floppy.ko 2023-09-23T03:00:06.381 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/loop.ko 2023-09-23T03:00:06.381 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx 2023-09-23T03:00:06.382 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-09-23T03:00:06.382 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/nbd.ko 2023-09-23T03:00:06.382 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk 2023-09-23T03:00:06.382 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk/null_blk.ko 2023-09-23T03:00:06.382 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/pktcdvd.ko 2023-09-23T03:00:06.382 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/rbd.ko 2023-09-23T03:00:06.382 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/virtio_blk.ko 2023-09-23T03:00:06.382 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback 2023-09-23T03:00:06.382 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-09-23T03:00:06.382 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkfront.ko 2023-09-23T03:00:06.382 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram 2023-09-23T03:00:06.383 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram/zram.ko 2023-09-23T03:00:06.383 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char 2023-09-23T03:00:06.383 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hangcheck-timer.ko 2023-09-23T03:00:06.383 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random 2023-09-23T03:00:06.383 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/amd-rng.ko 2023-09-23T03:00:06.383 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/intel-rng.ko 2023-09-23T03:00:06.383 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-09-23T03:00:06.383 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/via-rng.ko 2023-09-23T03:00:06.383 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-09-23T03:00:06.383 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi 2023-09-23T03:00:06.383 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-09-23T03:00:06.383 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-09-23T03:00:06.384 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-09-23T03:00:06.384 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-09-23T03:00:06.384 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-09-23T03:00:06.384 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-09-23T03:00:06.384 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/lp.ko 2023-09-23T03:00:06.384 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave 2023-09-23T03:00:06.384 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave/mwave.ko 2023-09-23T03:00:06.384 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ppdev.ko 2023-09-23T03:00:06.384 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tlclk.ko 2023-09-23T03:00:06.384 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm 2023-09-23T03:00:06.384 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm.ko 2023-09-23T03:00:06.384 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-09-23T03:00:06.385 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-09-23T03:00:06.385 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-09-23T03:00:06.385 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis.ko 2023-09-23T03:00:06.385 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-09-23T03:00:06.385 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/uv_mmtimer.ko 2023-09-23T03:00:06.385 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/virtio_console.ko 2023-09-23T03:00:06.385 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq 2023-09-23T03:00:06.385 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-09-23T03:00:06.385 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-09-23T03:00:06.385 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-09-23T03:00:06.385 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-09-23T03:00:06.386 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/powernow-k8.ko 2023-09-23T03:00:06.386 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-09-23T03:00:06.386 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto 2023-09-23T03:00:06.386 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp 2023-09-23T03:00:06.386 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-09-23T03:00:06.386 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp.ko 2023-09-23T03:00:06.386 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel 2023-09-23T03:00:06.386 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat 2023-09-23T03:00:06.386 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common 2023-09-23T03:00:06.386 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-09-23T03:00:06.386 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-09-23T03:00:06.387 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-09-23T03:00:06.387 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-aes.ko 2023-09-23T03:00:06.387 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-sha.ko 2023-09-23T03:00:06.387 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio 2023-09-23T03:00:06.387 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-09-23T03:00:06.387 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca 2023-09-23T03:00:06.387 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca/dca.ko 2023-09-23T03:00:06.387 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq 2023-09-23T03:00:06.387 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-09-23T03:00:06.387 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma 2023-09-23T03:00:06.387 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw 2023-09-23T03:00:06.387 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw/dw_dmac.ko 2023-09-23T03:00:06.388 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat 2023-09-23T03:00:06.388 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat/ioatdma.ko 2023-09-23T03:00:06.388 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac 2023-09-23T03:00:06.388 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/amd64_edac.ko 2023-09-23T03:00:06.388 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/e752x_edac.ko 2023-09-23T03:00:06.388 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/edac_mce_amd.ko 2023-09-23T03:00:06.388 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3000_edac.ko 2023-09-23T03:00:06.388 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3200_edac.ko 2023-09-23T03:00:06.388 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5100_edac.ko 2023-09-23T03:00:06.388 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5400_edac.ko 2023-09-23T03:00:06.388 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7300_edac.ko 2023-09-23T03:00:06.389 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7core_edac.ko 2023-09-23T03:00:06.389 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i82975x_edac.ko 2023-09-23T03:00:06.389 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/ie31200_edac.ko 2023-09-23T03:00:06.389 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/sb_edac.ko 2023-09-23T03:00:06.389 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/x38_edac.ko 2023-09-23T03:00:06.389 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware 2023-09-23T03:00:06.389 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/edd.ko 2023-09-23T03:00:06.389 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/iscsi_ibft.ko 2023-09-23T03:00:06.389 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio 2023-09-23T03:00:06.389 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-ich.ko 2023-09-23T03:00:06.389 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-viperboard.ko 2023-09-23T03:00:06.389 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid 2023-09-23T03:00:06.389 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-appleir.ko 2023-09-23T03:00:06.390 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-aureal.ko 2023-09-23T03:00:06.390 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-axff.ko 2023-09-23T03:00:06.390 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-dr.ko 2023-09-23T03:00:06.390 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elecom.ko 2023-09-23T03:00:06.390 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elo.ko 2023-09-23T03:00:06.390 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-emsff.ko 2023-09-23T03:00:06.390 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gaff.ko 2023-09-23T03:00:06.390 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gt683r.ko 2023-09-23T03:00:06.390 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gyration.ko 2023-09-23T03:00:06.390 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-09-23T03:00:06.390 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-09-23T03:00:06.390 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtekff.ko 2023-09-23T03:00:06.391 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-hyperv.ko 2023-09-23T03:00:06.391 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-icade.ko 2023-09-23T03:00:06.391 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-keytouch.ko 2023-09-23T03:00:06.391 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-kye.ko 2023-09-23T03:00:06.391 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lcpower.ko 2023-09-23T03:00:06.391 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-led.ko 2023-09-23T03:00:06.391 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lenovo.ko 2023-09-23T03:00:06.391 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-dj.ko 2023-09-23T03:00:06.391 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-09-23T03:00:06.391 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-multitouch.ko 2023-09-23T03:00:06.391 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-ortek.ko 2023-09-23T03:00:06.391 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-penmount.ko 2023-09-23T03:00:06.392 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-petalynx.ko 2023-09-23T03:00:06.392 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-picolcd.ko 2023-09-23T03:00:06.392 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-pl.ko 2023-09-23T03:00:06.392 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-plantronics.ko 2023-09-23T03:00:06.392 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-primax.ko 2023-09-23T03:00:06.392 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-rmi.ko 2023-09-23T03:00:06.392 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-09-23T03:00:06.392 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-common.ko 2023-09-23T03:00:06.392 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-isku.ko 2023-09-23T03:00:06.392 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kone.ko 2023-09-23T03:00:06.392 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-09-23T03:00:06.392 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-09-23T03:00:06.393 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-09-23T03:00:06.393 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-lua.ko 2023-09-23T03:00:06.393 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-09-23T03:00:06.393 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-09-23T03:00:06.393 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-savu.ko 2023-09-23T03:00:06.393 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat.ko 2023-09-23T03:00:06.393 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-saitek.ko 2023-09-23T03:00:06.393 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-samsung.ko 2023-09-23T03:00:06.393 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sensor-hub.ko 2023-09-23T03:00:06.393 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sjoy.ko 2023-09-23T03:00:06.393 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sony.ko 2023-09-23T03:00:06.393 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-speedlink.ko 2023-09-23T03:00:06.394 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-steelseries.ko 2023-09-23T03:00:06.394 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sunplus.ko 2023-09-23T03:00:06.394 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-thrustmaster.ko 2023-09-23T03:00:06.394 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tivo.ko 2023-09-23T03:00:06.394 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tmff.ko 2023-09-23T03:00:06.394 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-topseed.ko 2023-09-23T03:00:06.394 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-twinhan.ko 2023-09-23T03:00:06.394 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-uclogic.ko 2023-09-23T03:00:06.394 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-waltop.ko 2023-09-23T03:00:06.394 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-wiimote.ko 2023-09-23T03:00:06.394 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-xinmo.ko 2023-09-23T03:00:06.394 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zpff.ko 2023-09-23T03:00:06.395 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zydacron.ko 2023-09-23T03:00:06.395 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/uhid.ko 2023-09-23T03:00:06.395 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/wacom.ko 2023-09-23T03:00:06.395 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv 2023-09-23T03:00:06.395 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_balloon.ko 2023-09-23T03:00:06.395 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_utils.ko 2023-09-23T03:00:06.395 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_vmbus.ko 2023-09-23T03:00:06.395 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon 2023-09-23T03:00:06.395 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru.ko 2023-09-23T03:00:06.395 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru3.ko 2023-09-23T03:00:06.395 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-09-23T03:00:06.395 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7414.ko 2023-09-23T03:00:06.395 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7418.ko 2023-09-23T03:00:06.396 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adc128d818.ko 2023-09-23T03:00:06.396 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1025.ko 2023-09-23T03:00:06.396 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1026.ko 2023-09-23T03:00:06.396 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1029.ko 2023-09-23T03:00:06.396 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1031.ko 2023-09-23T03:00:06.396 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm9240.ko 2023-09-23T03:00:06.396 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ads7828.ko 2023-09-23T03:00:06.396 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7410.ko 2023-09-23T03:00:06.396 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7411.ko 2023-09-23T03:00:06.396 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7462.ko 2023-09-23T03:00:06.396 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7470.ko 2023-09-23T03:00:06.397 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7475.ko 2023-09-23T03:00:06.397 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7x10.ko 2023-09-23T03:00:06.397 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/amc6821.ko 2023-09-23T03:00:06.397 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/applesmc.ko 2023-09-23T03:00:06.397 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asb100.ko 2023-09-23T03:00:06.397 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asc7621.ko 2023-09-23T03:00:06.397 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asus_atk0110.ko 2023-09-23T03:00:06.397 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/atxp1.ko 2023-09-23T03:00:06.397 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/coretemp.ko 2023-09-23T03:00:06.397 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-09-23T03:00:06.397 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dme1737.ko 2023-09-23T03:00:06.398 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds1621.ko 2023-09-23T03:00:06.398 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds620.ko 2023-09-23T03:00:06.398 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc1403.ko 2023-09-23T03:00:06.398 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc6w201.ko 2023-09-23T03:00:06.398 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71805f.ko 2023-09-23T03:00:06.398 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71882fg.ko 2023-09-23T03:00:06.398 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f75375s.ko 2023-09-23T03:00:06.398 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fam15h_power.ko 2023-09-23T03:00:06.398 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fschmd.ko 2023-09-23T03:00:06.398 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g760a.ko 2023-09-23T03:00:06.398 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g762.ko 2023-09-23T03:00:06.399 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl518sm.ko 2023-09-23T03:00:06.399 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl520sm.ko 2023-09-23T03:00:06.399 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/hwmon-vid.ko 2023-09-23T03:00:06.399 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5500_temp.ko 2023-09-23T03:00:06.399 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5k_amb.ko 2023-09-23T03:00:06.399 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmaem.ko 2023-09-23T03:00:06.399 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmpex.ko 2023-09-23T03:00:06.399 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina209.ko 2023-09-23T03:00:06.399 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina2xx.ko 2023-09-23T03:00:06.399 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/it87.ko 2023-09-23T03:00:06.399 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k10temp.ko 2023-09-23T03:00:06.399 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k8temp.ko 2023-09-23T03:00:06.400 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lineage-pem.ko 2023-09-23T03:00:06.400 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm63.ko 2023-09-23T03:00:06.400 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm73.ko 2023-09-23T03:00:06.400 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm75.ko 2023-09-23T03:00:06.400 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm77.ko 2023-09-23T03:00:06.400 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm78.ko 2023-09-23T03:00:06.400 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm80.ko 2023-09-23T03:00:06.400 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm83.ko 2023-09-23T03:00:06.400 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm85.ko 2023-09-23T03:00:06.400 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm87.ko 2023-09-23T03:00:06.400 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm90.ko 2023-09-23T03:00:06.400 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm92.ko 2023-09-23T03:00:06.401 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm93.ko 2023-09-23T03:00:06.401 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95234.ko 2023-09-23T03:00:06.401 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95241.ko 2023-09-23T03:00:06.401 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95245.ko 2023-09-23T03:00:06.401 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc2945.ko 2023-09-23T03:00:06.401 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4151.ko 2023-09-23T03:00:06.401 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4215.ko 2023-09-23T03:00:06.401 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4222.ko 2023-09-23T03:00:06.401 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4245.ko 2023-09-23T03:00:06.401 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4260.ko 2023-09-23T03:00:06.401 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4261.ko 2023-09-23T03:00:06.401 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max16065.ko 2023-09-23T03:00:06.401 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1619.ko 2023-09-23T03:00:06.402 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1668.ko 2023-09-23T03:00:06.402 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max197.ko 2023-09-23T03:00:06.402 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6639.ko 2023-09-23T03:00:06.402 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6650.ko 2023-09-23T03:00:06.402 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6697.ko 2023-09-23T03:00:06.402 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/mcp3021.ko 2023-09-23T03:00:06.402 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6683.ko 2023-09-23T03:00:06.402 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775-core.ko 2023-09-23T03:00:06.402 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775.ko 2023-09-23T03:00:06.402 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct7802.ko 2023-09-23T03:00:06.402 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87360.ko 2023-09-23T03:00:06.402 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87427.ko 2023-09-23T03:00:06.403 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pcf8591.ko 2023-09-23T03:00:06.403 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus 2023-09-23T03:00:06.403 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-09-23T03:00:06.403 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-09-23T03:00:06.403 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-09-23T03:00:06.403 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-09-23T03:00:06.403 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-09-23T03:00:06.403 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-09-23T03:00:06.403 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-09-23T03:00:06.403 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-09-23T03:00:06.403 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-09-23T03:00:06.403 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-09-23T03:00:06.404 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-09-23T03:00:06.404 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-09-23T03:00:06.404 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/powr1220.ko 2023-09-23T03:00:06.404 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5627.ko 2023-09-23T03:00:06.404 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5636.ko 2023-09-23T03:00:06.404 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch56xx-common.ko 2023-09-23T03:00:06.404 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht15.ko 2023-09-23T03:00:06.404 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht21.ko 2023-09-23T03:00:06.404 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/shtc1.ko 2023-09-23T03:00:06.404 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sis5595.ko 2023-09-23T03:00:06.404 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47b397.ko 2023-09-23T03:00:06.404 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m1.ko 2023-09-23T03:00:06.405 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m192.ko 2023-09-23T03:00:06.405 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/thmc50.ko 2023-09-23T03:00:06.405 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp102.ko 2023-09-23T03:00:06.405 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp103.ko 2023-09-23T03:00:06.405 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp401.ko 2023-09-23T03:00:06.405 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp421.ko 2023-09-23T03:00:06.405 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via-cputemp.ko 2023-09-23T03:00:06.405 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via686a.ko 2023-09-23T03:00:06.405 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt1211.ko 2023-09-23T03:00:06.405 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt8231.ko 2023-09-23T03:00:06.405 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627ehf.ko 2023-09-23T03:00:06.405 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627hf.ko 2023-09-23T03:00:06.406 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83781d.ko 2023-09-23T03:00:06.406 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83791d.ko 2023-09-23T03:00:06.406 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83792d.ko 2023-09-23T03:00:06.406 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83793.ko 2023-09-23T03:00:06.406 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83795.ko 2023-09-23T03:00:06.406 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l785ts.ko 2023-09-23T03:00:06.406 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l786ng.ko 2023-09-23T03:00:06.406 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c 2023-09-23T03:00:06.406 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos 2023-09-23T03:00:06.406 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-09-23T03:00:06.406 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-09-23T03:00:06.406 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses 2023-09-23T03:00:06.406 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-09-23T03:00:06.407 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-09-23T03:00:06.407 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-09-23T03:00:06.407 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-09-23T03:00:06.407 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-09-23T03:00:06.407 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-09-23T03:00:06.407 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-09-23T03:00:06.407 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-09-23T03:00:06.407 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-09-23T03:00:06.407 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-09-23T03:00:06.407 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-09-23T03:00:06.407 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-09-23T03:00:06.408 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-09-23T03:00:06.408 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-09-23T03:00:06.408 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-09-23T03:00:06.408 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-09-23T03:00:06.408 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-09-23T03:00:06.408 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-09-23T03:00:06.408 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-09-23T03:00:06.408 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-09-23T03:00:06.408 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-via.ko 2023-09-23T03:00:06.408 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-09-23T03:00:06.408 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-09-23T03:00:06.408 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-dev.ko 2023-09-23T03:00:06.409 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-mux.ko 2023-09-23T03:00:06.409 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-09-23T03:00:06.409 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-smbus.ko 2023-09-23T03:00:06.409 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-stub.ko 2023-09-23T03:00:06.409 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband 2023-09-23T03:00:06.409 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core 2023-09-23T03:00:06.409 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_cm.ko 2023-09-23T03:00:06.409 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_core.ko 2023-09-23T03:00:06.409 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_umad.ko 2023-09-23T03:00:06.409 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-09-23T03:00:06.409 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/iw_cm.ko 2023-09-23T03:00:06.409 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-09-23T03:00:06.410 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-09-23T03:00:06.410 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw 2023-09-23T03:00:06.410 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4 2023-09-23T03:00:06.410 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-09-23T03:00:06.410 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4 2023-09-23T03:00:06.410 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-09-23T03:00:06.410 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5 2023-09-23T03:00:06.410 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-09-23T03:00:06.410 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca 2023-09-23T03:00:06.410 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-09-23T03:00:06.410 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp 2023-09-23T03:00:06.410 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib 2023-09-23T03:00:06.411 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-09-23T03:00:06.411 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser 2023-09-23T03:00:06.411 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-09-23T03:00:06.411 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert 2023-09-23T03:00:06.411 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-09-23T03:00:06.411 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp 2023-09-23T03:00:06.411 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-09-23T03:00:06.411 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt 2023-09-23T03:00:06.411 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-09-23T03:00:06.411 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input 2023-09-23T03:00:06.411 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport 2023-09-23T03:00:06.411 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-09-23T03:00:06.411 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/fm801-gp.ko 2023-09-23T03:00:06.412 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/gameport.ko 2023-09-23T03:00:06.412 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/lightning.ko 2023-09-23T03:00:06.412 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/ns558.ko 2023-09-23T03:00:06.412 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/joydev.ko 2023-09-23T03:00:06.412 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard 2023-09-23T03:00:06.412 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-09-23T03:00:06.412 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse 2023-09-23T03:00:06.412 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/appletouch.ko 2023-09-23T03:00:06.412 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/bcm5974.ko 2023-09-23T03:00:06.412 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/cyapatp.ko 2023-09-23T03:00:06.412 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/elan_i2c.ko 2023-09-23T03:00:06.412 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/sermouse.ko 2023-09-23T03:00:06.412 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-09-23T03:00:06.413 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-09-23T03:00:06.413 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-09-23T03:00:06.413 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4 2023-09-23T03:00:06.413 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4/rmi_core.ko 2023-09-23T03:00:06.413 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio 2023-09-23T03:00:06.413 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/altera_ps2.ko 2023-09-23T03:00:06.413 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/arc_ps2.ko 2023-09-23T03:00:06.413 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-09-23T03:00:06.413 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/serio_raw.ko 2023-09-23T03:00:06.413 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/sparse-keymap.ko 2023-09-23T03:00:06.413 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu 2023-09-23T03:00:06.413 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd 2023-09-23T03:00:06.413 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-09-23T03:00:06.414 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds 2023-09-23T03:00:06.414 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-blinkm.ko 2023-09-23T03:00:06.414 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm3530.ko 2023-09-23T03:00:06.414 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lp3944.ko 2023-09-23T03:00:06.414 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-ss4200.ko 2023-09-23T03:00:06.414 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger 2023-09-23T03:00:06.414 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-09-23T03:00:06.414 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-09-23T03:00:06.414 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-09-23T03:00:06.414 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-09-23T03:00:06.414 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-09-23T03:00:06.415 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-09-23T03:00:06.415 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-09-23T03:00:06.415 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-09-23T03:00:06.415 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md 2023-09-23T03:00:06.415 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache 2023-09-23T03:00:06.415 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache/bcache.ko 2023-09-23T03:00:06.415 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-bio-prison.ko 2023-09-23T03:00:06.415 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache-smq.ko 2023-09-23T03:00:06.415 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache.ko 2023-09-23T03:00:06.415 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-crypt.ko 2023-09-23T03:00:06.415 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-delay.ko 2023-09-23T03:00:06.415 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-flakey.ko 2023-09-23T03:00:06.416 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-log-userspace.ko 2023-09-23T03:00:06.416 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-multipath.ko 2023-09-23T03:00:06.416 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-queue-length.ko 2023-09-23T03:00:06.416 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-raid.ko 2023-09-23T03:00:06.416 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-round-robin.ko 2023-09-23T03:00:06.416 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-service-time.ko 2023-09-23T03:00:06.416 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-switch.ko 2023-09-23T03:00:06.416 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-thin-pool.ko 2023-09-23T03:00:06.416 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-verity.ko 2023-09-23T03:00:06.416 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/faulty.ko 2023-09-23T03:00:06.416 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/linear.ko 2023-09-23T03:00:06.417 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/multipath.ko 2023-09-23T03:00:06.417 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data 2023-09-23T03:00:06.417 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-09-23T03:00:06.417 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid0.ko 2023-09-23T03:00:06.417 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid1.ko 2023-09-23T03:00:06.417 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid10.ko 2023-09-23T03:00:06.417 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid456.ko 2023-09-23T03:00:06.417 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message 2023-09-23T03:00:06.417 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion 2023-09-23T03:00:06.417 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptbase.ko 2023-09-23T03:00:06.417 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptctl.ko 2023-09-23T03:00:06.417 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptfc.ko 2023-09-23T03:00:06.418 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptlan.ko 2023-09-23T03:00:06.418 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptsas.ko 2023-09-23T03:00:06.418 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptscsih.ko 2023-09-23T03:00:06.418 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptspi.ko 2023-09-23T03:00:06.418 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd 2023-09-23T03:00:06.418 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_ich.ko 2023-09-23T03:00:06.418 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_sch.ko 2023-09-23T03:00:06.418 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/mfd-core.ko 2023-09-23T03:00:06.418 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/sm501.ko 2023-09-23T03:00:06.418 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/viperboard.ko 2023-09-23T03:00:06.418 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/vx855.ko 2023-09-23T03:00:06.418 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/wl1273-core.ko 2023-09-23T03:00:06.419 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc 2023-09-23T03:00:06.419 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl 2023-09-23T03:00:06.419 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-09-23T03:00:06.419 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds9802als.ko 2023-09-23T03:00:06.419 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds990x.ko 2023-09-23T03:00:06.419 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/bh1770glc.ko 2023-09-23T03:00:06.419 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710 2023-09-23T03:00:06.419 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710/cb710.ko 2023-09-23T03:00:06.419 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom 2023-09-23T03:00:06.419 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/at24.ko 2023-09-23T03:00:06.419 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom.ko 2023-09-23T03:00:06.419 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-09-23T03:00:06.420 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/max6875.ko 2023-09-23T03:00:06.420 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/enclosure.ko 2023-09-23T03:00:06.420 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/hpilo.ko 2023-09-23T03:00:06.420 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29003.ko 2023-09-23T03:00:06.420 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29020.ko 2023-09-23T03:00:06.420 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d 2023-09-23T03:00:06.420 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-09-23T03:00:06.420 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-09-23T03:00:06.420 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei 2023-09-23T03:00:06.420 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-me.ko 2023-09-23T03:00:06.420 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-txe.ko 2023-09-23T03:00:06.420 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei.ko 2023-09-23T03:00:06.420 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru 2023-09-23T03:00:06.421 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru/gru.ko 2023-09-23T03:00:06.421 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp 2023-09-23T03:00:06.421 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xp.ko 2023-09-23T03:00:06.421 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpc.ko 2023-09-23T03:00:06.421 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-09-23T03:00:06.421 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_7xx1.ko 2023-09-23T03:00:06.421 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_core.ko 2023-09-23T03:00:06.421 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tsl2550.ko 2023-09-23T03:00:06.421 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_balloon.ko 2023-09-23T03:00:06.421 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci 2023-09-23T03:00:06.421 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-09-23T03:00:06.421 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net 2023-09-23T03:00:06.422 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding 2023-09-23T03:00:06.422 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding/bonding.ko 2023-09-23T03:00:06.422 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa 2023-09-23T03:00:06.422 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53 2023-09-23T03:00:06.422 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-09-23T03:00:06.422 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-09-23T03:00:06.422 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6060.ko 2023-09-23T03:00:06.422 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx 2023-09-23T03:00:06.422 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-09-23T03:00:06.422 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dummy.ko 2023-09-23T03:00:06.422 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/eql.ko 2023-09-23T03:00:06.422 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet 2023-09-23T03:00:06.422 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com 2023-09-23T03:00:06.423 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-09-23T03:00:06.423 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-09-23T03:00:06.423 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-09-23T03:00:06.423 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-09-23T03:00:06.423 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390 2023-09-23T03:00:06.423 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/8390.ko 2023-09-23T03:00:06.423 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-09-23T03:00:06.423 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-09-23T03:00:06.423 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-09-23T03:00:06.423 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec 2023-09-23T03:00:06.423 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-09-23T03:00:06.424 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere 2023-09-23T03:00:06.424 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-09-23T03:00:06.424 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon 2023-09-23T03:00:06.424 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-09-23T03:00:06.424 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera 2023-09-23T03:00:06.424 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-09-23T03:00:06.424 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd 2023-09-23T03:00:06.424 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-09-23T03:00:06.424 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-09-23T03:00:06.424 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-09-23T03:00:06.424 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros 2023-09-23T03:00:06.424 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx 2023-09-23T03:00:06.425 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-09-23T03:00:06.425 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c 2023-09-23T03:00:06.425 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-09-23T03:00:06.425 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e 2023-09-23T03:00:06.425 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-09-23T03:00:06.425 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx 2023-09-23T03:00:06.425 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-09-23T03:00:06.425 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-09-23T03:00:06.425 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom 2023-09-23T03:00:06.425 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-09-23T03:00:06.425 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-09-23T03:00:06.425 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-09-23T03:00:06.426 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-09-23T03:00:06.426 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-09-23T03:00:06.426 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet 2023-09-23T03:00:06.426 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-09-23T03:00:06.426 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-09-23T03:00:06.426 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade 2023-09-23T03:00:06.426 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna 2023-09-23T03:00:06.426 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-09-23T03:00:06.426 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio 2023-09-23T03:00:06.426 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb 2023-09-23T03:00:06.426 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-09-23T03:00:06.426 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-09-23T03:00:06.427 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-09-23T03:00:06.427 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-09-23T03:00:06.427 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-09-23T03:00:06.427 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-09-23T03:00:06.427 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-09-23T03:00:06.427 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-09-23T03:00:06.427 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-09-23T03:00:06.427 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco 2023-09-23T03:00:06.427 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic 2023-09-23T03:00:06.427 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-09-23T03:00:06.427 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec 2023-09-23T03:00:06.427 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip 2023-09-23T03:00:06.427 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-09-23T03:00:06.427 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-09-23T03:00:06.428 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-09-23T03:00:06.428 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-09-23T03:00:06.428 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-09-23T03:00:06.428 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-09-23T03:00:06.428 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink 2023-09-23T03:00:06.428 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-09-23T03:00:06.428 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-09-23T03:00:06.428 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dnet.ko 2023-09-23T03:00:06.428 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex 2023-09-23T03:00:06.428 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet 2023-09-23T03:00:06.428 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-09-23T03:00:06.429 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ethoc.ko 2023-09-23T03:00:06.429 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/fealnx.ko 2023-09-23T03:00:06.429 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel 2023-09-23T03:00:06.429 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e100.ko 2023-09-23T03:00:06.429 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000 2023-09-23T03:00:06.429 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-09-23T03:00:06.429 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e 2023-09-23T03:00:06.429 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-09-23T03:00:06.429 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k 2023-09-23T03:00:06.429 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-09-23T03:00:06.429 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e 2023-09-23T03:00:06.429 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-09-23T03:00:06.430 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf 2023-09-23T03:00:06.430 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-09-23T03:00:06.430 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb 2023-09-23T03:00:06.430 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-09-23T03:00:06.430 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf 2023-09-23T03:00:06.430 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-09-23T03:00:06.430 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe 2023-09-23T03:00:06.430 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-09-23T03:00:06.430 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf 2023-09-23T03:00:06.430 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-09-23T03:00:06.430 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/jme.ko 2023-09-23T03:00:06.431 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell 2023-09-23T03:00:06.431 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-09-23T03:00:06.431 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-09-23T03:00:06.431 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-09-23T03:00:06.431 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox 2023-09-23T03:00:06.431 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4 2023-09-23T03:00:06.431 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-09-23T03:00:06.431 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-09-23T03:00:06.431 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5 2023-09-23T03:00:06.431 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-09-23T03:00:06.431 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-09-23T03:00:06.431 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel 2023-09-23T03:00:06.431 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-09-23T03:00:06.432 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom 2023-09-23T03:00:06.432 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge 2023-09-23T03:00:06.432 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-09-23T03:00:06.432 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi 2023-09-23T03:00:06.432 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-09-23T03:00:06.432 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-09-23T03:00:06.432 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion 2023-09-23T03:00:06.432 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-09-23T03:00:06.432 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia 2023-09-23T03:00:06.432 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-09-23T03:00:06.432 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines 2023-09-23T03:00:06.432 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-09-23T03:00:06.433 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-09-23T03:00:06.433 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic 2023-09-23T03:00:06.433 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen 2023-09-23T03:00:06.433 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-09-23T03:00:06.433 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-09-23T03:00:06.433 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-09-23T03:00:06.433 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-09-23T03:00:06.433 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc 2023-09-23T03:00:06.433 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-09-23T03:00:06.433 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek 2023-09-23T03:00:06.433 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-09-23T03:00:06.433 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-09-23T03:00:06.433 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-09-23T03:00:06.434 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-09-23T03:00:06.434 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker 2023-09-23T03:00:06.434 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-09-23T03:00:06.434 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc 2023-09-23T03:00:06.434 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-09-23T03:00:06.434 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan 2023-09-23T03:00:06.434 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-09-23T03:00:06.434 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis 2023-09-23T03:00:06.434 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-09-23T03:00:06.434 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-09-23T03:00:06.434 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc 2023-09-23T03:00:06.435 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-09-23T03:00:06.435 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-09-23T03:00:06.435 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-09-23T03:00:06.435 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-09-23T03:00:06.435 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro 2023-09-23T03:00:06.435 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac 2023-09-23T03:00:06.435 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-09-23T03:00:06.435 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-09-23T03:00:06.435 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun 2023-09-23T03:00:06.435 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-09-23T03:00:06.435 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/niu.ko 2023-09-23T03:00:06.435 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-09-23T03:00:06.436 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-09-23T03:00:06.436 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti 2023-09-23T03:00:06.436 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-09-23T03:00:06.436 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti 2023-09-23T03:00:06.436 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-09-23T03:00:06.436 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via 2023-09-23T03:00:06.436 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-09-23T03:00:06.436 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-09-23T03:00:06.436 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet 2023-09-23T03:00:06.436 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-09-23T03:00:06.436 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-09-23T03:00:06.436 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom 2023-09-23T03:00:06.437 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-09-23T03:00:06.437 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/geneve.ko 2023-09-23T03:00:06.437 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv 2023-09-23T03:00:06.437 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-09-23T03:00:06.437 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154 2023-09-23T03:00:06.437 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154/fakelb.ko 2023-09-23T03:00:06.437 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ifb.ko 2023-09-23T03:00:06.437 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan 2023-09-23T03:00:06.437 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-09-23T03:00:06.437 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvlan.ko 2023-09-23T03:00:06.437 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvtap.ko 2023-09-23T03:00:06.437 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio 2023-09-23T03:00:06.437 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio.ko 2023-09-23T03:00:06.438 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-09-23T03:00:06.438 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-09-23T03:00:06.438 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-09-23T03:00:06.438 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mii.ko 2023-09-23T03:00:06.438 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/net_failover.ko 2023-09-23T03:00:06.438 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/netconsole.ko 2023-09-23T03:00:06.438 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/nlmon.ko 2023-09-23T03:00:06.438 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs 2023-09-23T03:00:06.438 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-09-23T03:00:06.438 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-09-23T03:00:06.438 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy 2023-09-23T03:00:06.439 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/amd.ko 2023-09-23T03:00:06.439 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-09-23T03:00:06.439 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-09-23T03:00:06.439 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm7xxx.ko 2023-09-23T03:00:06.439 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm87xx.ko 2023-09-23T03:00:06.439 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/broadcom.ko 2023-09-23T03:00:06.439 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/cicada.ko 2023-09-23T03:00:06.439 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/davicom.ko 2023-09-23T03:00:06.439 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/dp83640.ko 2023-09-23T03:00:06.439 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/et1011c.ko 2023-09-23T03:00:06.439 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/icplus.ko 2023-09-23T03:00:06.440 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/lxt.ko 2023-09-23T03:00:06.440 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/marvell.ko 2023-09-23T03:00:06.440 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/micrel.ko 2023-09-23T03:00:06.440 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/national.ko 2023-09-23T03:00:06.440 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/phylink.ko 2023-09-23T03:00:06.440 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/qsemi.ko 2023-09-23T03:00:06.440 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/realtek.ko 2023-09-23T03:00:06.440 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/smsc.ko 2023-09-23T03:00:06.440 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/ste10Xp.ko 2023-09-23T03:00:06.440 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/vitesse.ko 2023-09-23T03:00:06.440 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp 2023-09-23T03:00:06.440 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/bsd_comp.ko 2023-09-23T03:00:06.440 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_async.ko 2023-09-23T03:00:06.441 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-09-23T03:00:06.441 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_generic.ko 2023-09-23T03:00:06.441 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-09-23T03:00:06.441 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-09-23T03:00:06.441 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppoe.ko 2023-09-23T03:00:06.441 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppox.ko 2023-09-23T03:00:06.441 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pptp.ko 2023-09-23T03:00:06.441 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip 2023-09-23T03:00:06.441 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slhc.ko 2023-09-23T03:00:06.441 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slip.ko 2023-09-23T03:00:06.441 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/sungem_phy.ko 2023-09-23T03:00:06.441 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tap.ko 2023-09-23T03:00:06.442 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team 2023-09-23T03:00:06.442 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team.ko 2023-09-23T03:00:06.442 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-09-23T03:00:06.442 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-09-23T03:00:06.442 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-09-23T03:00:06.442 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_random.ko 2023-09-23T03:00:06.442 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-09-23T03:00:06.442 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tun.ko 2023-09-23T03:00:06.442 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/veth.ko 2023-09-23T03:00:06.442 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/virtio_net.ko 2023-09-23T03:00:06.442 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3 2023-09-23T03:00:06.443 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-09-23T03:00:06.443 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan 2023-09-23T03:00:06.443 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan/vxlan.ko 2023-09-23T03:00:06.443 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback 2023-09-23T03:00:06.443 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-09-23T03:00:06.443 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netfront.ko 2023-09-23T03:00:06.443 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb 2023-09-23T03:00:06.443 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb/ntb.ko 2023-09-23T03:00:06.443 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme 2023-09-23T03:00:06.443 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host 2023-09-23T03:00:06.443 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme-core.ko 2023-09-23T03:00:06.443 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme.ko 2023-09-23T03:00:06.443 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport 2023-09-23T03:00:06.444 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport.ko 2023-09-23T03:00:06.444 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_cs.ko 2023-09-23T03:00:06.444 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_pc.ko 2023-09-23T03:00:06.444 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_serial.ko 2023-09-23T03:00:06.444 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci 2023-09-23T03:00:06.444 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug 2023-09-23T03:00:06.444 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-09-23T03:00:06.444 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie 2023-09-23T03:00:06.444 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie/aer_inject.ko 2023-09-23T03:00:06.444 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/xen-pcifront.ko 2023-09-23T03:00:06.444 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia 2023-09-23T03:00:06.444 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/i82092.ko 2023-09-23T03:00:06.445 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pd6729.ko 2023-09-23T03:00:06.445 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/yenta_socket.ko 2023-09-23T03:00:06.445 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl 2023-09-23T03:00:06.445 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel 2023-09-23T03:00:06.445 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-09-23T03:00:06.445 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform 2023-09-23T03:00:06.445 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86 2023-09-23T03:00:06.445 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acer-wmi.ko 2023-09-23T03:00:06.445 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acerhdf.ko 2023-09-23T03:00:06.445 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/apple-gmux.ko 2023-09-23T03:00:06.445 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-laptop.ko 2023-09-23T03:00:06.445 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-09-23T03:00:06.445 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-wmi.ko 2023-09-23T03:00:06.446 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-09-23T03:00:06.446 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-09-23T03:00:06.446 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-09-23T03:00:06.446 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-09-23T03:00:06.446 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-09-23T03:00:06.446 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/hdaps.ko 2023-09-23T03:00:06.446 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel 2023-09-23T03:00:06.446 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-09-23T03:00:06.446 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel_ips.ko 2023-09-23T03:00:06.446 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/msi-wmi.ko 2023-09-23T03:00:06.446 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-09-23T03:00:06.447 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-09-23T03:00:06.447 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-09-23T03:00:06.447 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-q10.ko 2023-09-23T03:00:06.447 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-09-23T03:00:06.447 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-09-23T03:00:06.447 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-09-23T03:00:06.447 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-09-23T03:00:06.447 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi.ko 2023-09-23T03:00:06.447 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap 2023-09-23T03:00:06.447 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_common.ko 2023-09-23T03:00:06.447 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-09-23T03:00:06.447 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps 2023-09-23T03:00:06.448 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients 2023-09-23T03:00:06.448 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-gpio.ko 2023-09-23T03:00:06.448 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-09-23T03:00:06.448 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps_parport.ko 2023-09-23T03:00:06.448 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/pps_core.ko 2023-09-23T03:00:06.448 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp 2023-09-23T03:00:06.448 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp.ko 2023-09-23T03:00:06.448 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp_kvm.ko 2023-09-23T03:00:06.448 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc 2023-09-23T03:00:06.448 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq32k.ko 2023-09-23T03:00:06.448 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq4802.ko 2023-09-23T03:00:06.448 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1286.ko 2023-09-23T03:00:06.449 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1307.ko 2023-09-23T03:00:06.449 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1374.ko 2023-09-23T03:00:06.449 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1511.ko 2023-09-23T03:00:06.449 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1553.ko 2023-09-23T03:00:06.449 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1672.ko 2023-09-23T03:00:06.449 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1742.ko 2023-09-23T03:00:06.449 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds2404.ko 2023-09-23T03:00:06.449 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds3232.ko 2023-09-23T03:00:06.449 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-em3027.ko 2023-09-23T03:00:06.449 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-fm3130.ko 2023-09-23T03:00:06.449 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl12022.ko 2023-09-23T03:00:06.449 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl1208.ko 2023-09-23T03:00:06.450 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m41t80.ko 2023-09-23T03:00:06.450 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t35.ko 2023-09-23T03:00:06.450 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t59.ko 2023-09-23T03:00:06.450 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max6900.ko 2023-09-23T03:00:06.450 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-msm6242.ko 2023-09-23T03:00:06.450 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-09-23T03:00:06.450 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-09-23T03:00:06.450 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-09-23T03:00:06.450 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-09-23T03:00:06.450 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-09-23T03:00:06.451 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-09-23T03:00:06.451 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-09-23T03:00:06.451 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-09-23T03:00:06.451 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8025.ko 2023-09-23T03:00:06.451 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8581.ko 2023-09-23T03:00:06.451 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-09-23T03:00:06.451 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-x1205.ko 2023-09-23T03:00:06.451 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi 2023-09-23T03:00:06.451 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-9xxx.ko 2023-09-23T03:00:06.451 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-sas.ko 2023-09-23T03:00:06.451 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-xxxx.ko 2023-09-23T03:00:06.451 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/BusLogic.ko 2023-09-23T03:00:06.452 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/a100u2w.ko 2023-09-23T03:00:06.452 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid 2023-09-23T03:00:06.452 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-09-23T03:00:06.452 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/advansys.ko 2023-09-23T03:00:06.452 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx 2023-09-23T03:00:06.452 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-09-23T03:00:06.452 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-09-23T03:00:06.452 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/am53c974.ko 2023-09-23T03:00:06.452 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr 2023-09-23T03:00:06.452 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-09-23T03:00:06.452 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/atp870u.ko 2023-09-23T03:00:06.452 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi 2023-09-23T03:00:06.452 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-09-23T03:00:06.453 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa 2023-09-23T03:00:06.453 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa/bfa.ko 2023-09-23T03:00:06.453 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc 2023-09-23T03:00:06.453 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-09-23T03:00:06.453 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i 2023-09-23T03:00:06.453 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-09-23T03:00:06.453 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ch.ko 2023-09-23T03:00:06.453 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor 2023-09-23T03:00:06.453 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-09-23T03:00:06.453 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi 2023-09-23T03:00:06.453 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i 2023-09-23T03:00:06.453 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-09-23T03:00:06.454 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i 2023-09-23T03:00:06.454 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-09-23T03:00:06.454 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-09-23T03:00:06.454 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dc395x.ko 2023-09-23T03:00:06.454 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler 2023-09-23T03:00:06.454 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-09-23T03:00:06.454 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-09-23T03:00:06.454 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-09-23T03:00:06.454 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-09-23T03:00:06.454 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dmx3191d.ko 2023-09-23T03:00:06.454 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r 2023-09-23T03:00:06.455 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-09-23T03:00:06.455 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esp_scsi.ko 2023-09-23T03:00:06.455 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe 2023-09-23T03:00:06.455 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-09-23T03:00:06.455 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-09-23T03:00:06.455 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic 2023-09-23T03:00:06.455 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic/fnic.ko 2023-09-23T03:00:06.455 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hpsa.ko 2023-09-23T03:00:06.455 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hptiop.ko 2023-09-23T03:00:06.455 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hv_storvsc.ko 2023-09-23T03:00:06.455 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/initio.ko 2023-09-23T03:00:06.455 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ipr.ko 2023-09-23T03:00:06.456 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ips.ko 2023-09-23T03:00:06.456 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci 2023-09-23T03:00:06.456 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci/isci.ko 2023-09-23T03:00:06.456 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-09-23T03:00:06.456 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_tcp.ko 2023-09-23T03:00:06.456 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc 2023-09-23T03:00:06.456 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc/libfc.ko 2023-09-23T03:00:06.456 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi.ko 2023-09-23T03:00:06.456 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-09-23T03:00:06.456 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas 2023-09-23T03:00:06.456 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas/libsas.ko 2023-09-23T03:00:06.456 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc 2023-09-23T03:00:06.457 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-09-23T03:00:06.457 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid 2023-09-23T03:00:06.457 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid.ko 2023-09-23T03:00:06.457 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-09-23T03:00:06.457 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-09-23T03:00:06.457 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-09-23T03:00:06.457 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas 2023-09-23T03:00:06.457 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-09-23T03:00:06.457 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas 2023-09-23T03:00:06.457 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-09-23T03:00:06.457 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvumi.ko 2023-09-23T03:00:06.457 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001 2023-09-23T03:00:06.458 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-09-23T03:00:06.458 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pmcraid.ko 2023-09-23T03:00:06.458 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla1280.ko 2023-09-23T03:00:06.458 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx 2023-09-23T03:00:06.458 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-09-23T03:00:06.458 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-09-23T03:00:06.458 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx 2023-09-23T03:00:06.458 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-09-23T03:00:06.458 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/raid_class.ko 2023-09-23T03:00:06.458 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_debug.ko 2023-09-23T03:00:06.458 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-09-23T03:00:06.458 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-09-23T03:00:06.459 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-09-23T03:00:06.459 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-09-23T03:00:06.459 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-09-23T03:00:06.459 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ses.ko 2023-09-23T03:00:06.459 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/st.ko 2023-09-23T03:00:06.459 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/stex.ko 2023-09-23T03:00:06.459 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2 2023-09-23T03:00:06.459 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-09-23T03:00:06.459 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/virtio_scsi.ko 2023-09-23T03:00:06.459 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-09-23T03:00:06.459 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/wd719x.ko 2023-09-23T03:00:06.459 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb 2023-09-23T03:00:06.460 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb/ssb.ko 2023-09-23T03:00:06.460 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target 2023-09-23T03:00:06.460 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi 2023-09-23T03:00:06.460 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-09-23T03:00:06.460 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback 2023-09-23T03:00:06.460 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback/tcm_loop.ko 2023-09-23T03:00:06.460 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_file.ko 2023-09-23T03:00:06.460 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_iblock.ko 2023-09-23T03:00:06.460 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_mod.ko 2023-09-23T03:00:06.460 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_pscsi.ko 2023-09-23T03:00:06.460 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_user.ko 2023-09-23T03:00:06.460 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc 2023-09-23T03:00:06.460 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-09-23T03:00:06.461 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal 2023-09-23T03:00:06.461 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel 2023-09-23T03:00:06.461 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal 2023-09-23T03:00:06.461 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-09-23T03:00:06.461 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-09-23T03:00:06.461 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-09-23T03:00:06.461 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-09-23T03:00:06.461 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-09-23T03:00:06.461 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-09-23T03:00:06.461 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-09-23T03:00:06.461 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-09-23T03:00:06.461 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-09-23T03:00:06.462 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-09-23T03:00:06.462 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-09-23T03:00:06.462 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-09-23T03:00:06.462 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-09-23T03:00:06.462 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-09-23T03:00:06.462 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-09-23T03:00:06.462 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty 2023-09-23T03:00:06.462 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless 2023-09-23T03:00:06.462 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-09-23T03:00:06.462 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_gsm.ko 2023-09-23T03:00:06.462 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_hdlc.ko 2023-09-23T03:00:06.463 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/nozomi.ko 2023-09-23T03:00:06.463 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial 2023-09-23T03:00:06.463 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250 2023-09-23T03:00:06.463 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-09-23T03:00:06.463 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/arc_uart.ko 2023-09-23T03:00:06.463 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm 2023-09-23T03:00:06.463 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-09-23T03:00:06.463 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/synclink_gt.ko 2023-09-23T03:00:06.463 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs 2023-09-23T03:00:06.463 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core 2023-09-23T03:00:06.463 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-09-23T03:00:06.463 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host 2023-09-23T03:00:06.463 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-09-23T03:00:06.464 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio 2023-09-23T03:00:06.464 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio.ko 2023-09-23T03:00:06.464 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_aec.ko 2023-09-23T03:00:06.464 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_cif.ko 2023-09-23T03:00:06.464 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_pci_generic.ko 2023-09-23T03:00:06.464 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_sercos3.ko 2023-09-23T03:00:06.464 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb 2023-09-23T03:00:06.464 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class 2023-09-23T03:00:06.464 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-acm.ko 2023-09-23T03:00:06.464 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-wdm.ko 2023-09-23T03:00:06.464 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usblp.ko 2023-09-23T03:00:06.464 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usbtmc.ko 2023-09-23T03:00:06.465 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host 2023-09-23T03:00:06.465 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/sl811-hcd.ko 2023-09-23T03:00:06.465 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image 2023-09-23T03:00:06.465 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/mdc800.ko 2023-09-23T03:00:06.465 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/microtek.ko 2023-09-23T03:00:06.465 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc 2023-09-23T03:00:06.465 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/adutux.ko 2023-09-23T03:00:06.465 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/appledisplay.ko 2023-09-23T03:00:06.465 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi26.ko 2023-09-23T03:00:06.465 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi62.ko 2023-09-23T03:00:06.465 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ezusb.ko 2023-09-23T03:00:06.465 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/idmouse.ko 2023-09-23T03:00:06.466 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/iowarrior.ko 2023-09-23T03:00:06.466 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/isight_firmware.ko 2023-09-23T03:00:06.466 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ldusb.ko 2023-09-23T03:00:06.466 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/legousbtower.ko 2023-09-23T03:00:06.466 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga 2023-09-23T03:00:06.466 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-09-23T03:00:06.466 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/trancevibrator.ko 2023-09-23T03:00:06.466 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usb3503.ko 2023-09-23T03:00:06.466 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usblcd.ko 2023-09-23T03:00:06.466 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usbsevseg.ko 2023-09-23T03:00:06.466 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/uss720.ko 2023-09-23T03:00:06.466 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/yurex.ko 2023-09-23T03:00:06.467 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy 2023-09-23T03:00:06.467 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-generic.ko 2023-09-23T03:00:06.467 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial 2023-09-23T03:00:06.467 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/aircable.ko 2023-09-23T03:00:06.467 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ark3116.ko 2023-09-23T03:00:06.467 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/belkin_sa.ko 2023-09-23T03:00:06.467 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ch341.ko 2023-09-23T03:00:06.467 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cp210x.ko 2023-09-23T03:00:06.467 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cyberjack.ko 2023-09-23T03:00:06.467 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cypress_m8.ko 2023-09-23T03:00:06.467 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-09-23T03:00:06.467 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/empeg.ko 2023-09-23T03:00:06.468 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-09-23T03:00:06.468 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/garmin_gps.ko 2023-09-23T03:00:06.468 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_edgeport.ko 2023-09-23T03:00:06.468 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_ti.ko 2023-09-23T03:00:06.468 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipaq.ko 2023-09-23T03:00:06.468 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipw.ko 2023-09-23T03:00:06.468 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ir-usb.ko 2023-09-23T03:00:06.468 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-09-23T03:00:06.468 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan.ko 2023-09-23T03:00:06.468 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-09-23T03:00:06.468 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-09-23T03:00:06.468 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kobil_sct.ko 2023-09-23T03:00:06.468 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mct_u232.ko 2023-09-23T03:00:06.469 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7720.ko 2023-09-23T03:00:06.469 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7840.ko 2023-09-23T03:00:06.469 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/navman.ko 2023-09-23T03:00:06.469 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/omninet.ko 2023-09-23T03:00:06.469 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/opticon.ko 2023-09-23T03:00:06.469 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/option.ko 2023-09-23T03:00:06.469 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/oti6858.ko 2023-09-23T03:00:06.469 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/pl2303.ko 2023-09-23T03:00:06.469 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcaux.ko 2023-09-23T03:00:06.469 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcserial.ko 2023-09-23T03:00:06.469 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/quatech2.ko 2023-09-23T03:00:06.469 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/safe_serial.ko 2023-09-23T03:00:06.470 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/sierra.ko 2023-09-23T03:00:06.470 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/spcp8x5.ko 2023-09-23T03:00:06.470 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ssu100.ko 2023-09-23T03:00:06.470 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/symbolserial.ko 2023-09-23T03:00:06.470 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-09-23T03:00:06.470 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-09-23T03:00:06.470 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_debug.ko 2023-09-23T03:00:06.470 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_wwan.ko 2023-09-23T03:00:06.470 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/visor.ko 2023-09-23T03:00:06.470 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/whiteheat.ko 2023-09-23T03:00:06.470 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/xsens_mt.ko 2023-09-23T03:00:06.470 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage 2023-09-23T03:00:06.470 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/uas.ko 2023-09-23T03:00:06.471 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-alauda.ko 2023-09-23T03:00:06.471 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-cypress.ko 2023-09-23T03:00:06.471 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-datafab.ko 2023-09-23T03:00:06.471 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-09-23T03:00:06.471 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-freecom.ko 2023-09-23T03:00:06.471 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-isd200.ko 2023-09-23T03:00:06.471 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-09-23T03:00:06.471 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-karma.ko 2023-09-23T03:00:06.471 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-09-23T03:00:06.471 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-realtek.ko 2023-09-23T03:00:06.471 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-09-23T03:00:06.472 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-09-23T03:00:06.472 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-usbat.ko 2023-09-23T03:00:06.472 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/usb-storage.ko 2023-09-23T03:00:06.472 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip 2023-09-23T03:00:06.472 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-core.ko 2023-09-23T03:00:06.472 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-host.ko 2023-09-23T03:00:06.472 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-09-23T03:00:06.472 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio 2023-09-23T03:00:06.472 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci 2023-09-23T03:00:06.472 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-09-23T03:00:06.472 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-09-23T03:00:06.472 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio.ko 2023-09-23T03:00:06.472 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-09-23T03:00:06.473 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost 2023-09-23T03:00:06.473 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost.ko 2023-09-23T03:00:06.473 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_iotlb.ko 2023-09-23T03:00:06.473 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_net.ko 2023-09-23T03:00:06.473 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_scsi.ko 2023-09-23T03:00:06.473 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video 2023-09-23T03:00:06.473 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight 2023-09-23T03:00:06.473 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/apple_bl.ko 2023-09-23T03:00:06.473 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lcd.ko 2023-09-23T03:00:06.473 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/platform_lcd.ko 2023-09-23T03:00:06.473 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio 2023-09-23T03:00:06.473 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio.ko 2023-09-23T03:00:06.474 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_balloon.ko 2023-09-23T03:00:06.474 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_mmio.ko 2023-09-23T03:00:06.474 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci.ko 2023-09-23T03:00:06.474 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-09-23T03:00:06.474 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-09-23T03:00:06.474 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_ring.ko 2023-09-23T03:00:06.474 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1 2023-09-23T03:00:06.474 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters 2023-09-23T03:00:06.474 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2482.ko 2023-09-23T03:00:06.474 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2490.ko 2023-09-23T03:00:06.474 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves 2023-09-23T03:00:06.474 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-09-23T03:00:06.475 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-09-23T03:00:06.475 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-09-23T03:00:06.475 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-09-23T03:00:06.475 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-09-23T03:00:06.475 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-09-23T03:00:06.475 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-09-23T03:00:06.475 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-09-23T03:00:06.475 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-09-23T03:00:06.475 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_smem.ko 2023-09-23T03:00:06.475 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_therm.ko 2023-09-23T03:00:06.475 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/wire.ko 2023-09-23T03:00:06.475 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog 2023-09-23T03:00:06.476 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-09-23T03:00:06.476 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-09-23T03:00:06.476 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-09-23T03:00:06.476 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/hpwdt.ko 2023-09-23T03:00:06.476 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/i6300esb.ko 2023-09-23T03:00:06.476 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-09-23T03:00:06.476 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-09-23T03:00:06.476 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ib700wdt.ko 2023-09-23T03:00:06.476 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ibmasr.ko 2023-09-23T03:00:06.476 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-09-23T03:00:06.476 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-09-23T03:00:06.476 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it87_wdt.ko 2023-09-23T03:00:06.477 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/machzwd.ko 2023-09-23T03:00:06.477 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/nv_tco.ko 2023-09-23T03:00:06.477 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_pci.ko 2023-09-23T03:00:06.477 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_usb.ko 2023-09-23T03:00:06.477 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-09-23T03:00:06.477 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-09-23T03:00:06.477 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/softdog.ko 2023-09-23T03:00:06.477 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sp5100_tco.ko 2023-09-23T03:00:06.477 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/via_wdt.ko 2023-09-23T03:00:06.477 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-09-23T03:00:06.477 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-09-23T03:00:06.477 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-09-23T03:00:06.478 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wdt_pci.ko 2023-09-23T03:00:06.478 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/xen_wdt.ko 2023-09-23T03:00:06.478 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen 2023-09-23T03:00:06.478 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-acpi-processor.ko 2023-09-23T03:00:06.478 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-evtchn.ko 2023-09-23T03:00:06.478 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntalloc.ko 2023-09-23T03:00:06.478 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntdev.ko 2023-09-23T03:00:06.478 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback 2023-09-23T03:00:06.478 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-09-23T03:00:06.478 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-privcmd.ko 2023-09-23T03:00:06.478 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs 2023-09-23T03:00:06.478 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs/xenfs.ko 2023-09-23T03:00:06.479 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs 2023-09-23T03:00:06.479 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p 2023-09-23T03:00:06.479 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p/9p.ko 2023-09-23T03:00:06.479 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs 2023-09-23T03:00:06.479 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs/affs.ko 2023-09-23T03:00:06.479 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs 2023-09-23T03:00:06.479 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs/befs.ko 2023-09-23T03:00:06.479 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/binfmt_misc.ko 2023-09-23T03:00:06.479 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs 2023-09-23T03:00:06.479 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs/btrfs.ko 2023-09-23T03:00:06.479 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles 2023-09-23T03:00:06.479 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles/cachefiles.ko 2023-09-23T03:00:06.480 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph 2023-09-23T03:00:06.480 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph/ceph.ko 2023-09-23T03:00:06.480 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda 2023-09-23T03:00:06.480 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda/coda.ko 2023-09-23T03:00:06.480 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs 2023-09-23T03:00:06.480 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs/cramfs.ko 2023-09-23T03:00:06.480 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm 2023-09-23T03:00:06.480 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm/dlm.ko 2023-09-23T03:00:06.480 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs 2023-09-23T03:00:06.480 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs/ecryptfs.ko 2023-09-23T03:00:06.480 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs 2023-09-23T03:00:06.480 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs/f2fs.ko 2023-09-23T03:00:06.480 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat 2023-09-23T03:00:06.480 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/fat.ko 2023-09-23T03:00:06.481 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/msdos.ko 2023-09-23T03:00:06.481 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/vfat.ko 2023-09-23T03:00:06.481 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache 2023-09-23T03:00:06.481 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache/fscache.ko 2023-09-23T03:00:06.481 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse 2023-09-23T03:00:06.481 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/cuse.ko 2023-09-23T03:00:06.481 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/fuse.ko 2023-09-23T03:00:06.481 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2 2023-09-23T03:00:06.481 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2/gfs2.ko 2023-09-23T03:00:06.481 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs 2023-09-23T03:00:06.481 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs/hfs.ko 2023-09-23T03:00:06.481 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus 2023-09-23T03:00:06.481 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus/hfsplus.ko 2023-09-23T03:00:06.482 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs 2023-09-23T03:00:06.482 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs/isofs.ko 2023-09-23T03:00:06.482 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs 2023-09-23T03:00:06.482 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs/jfs.ko 2023-09-23T03:00:06.482 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd 2023-09-23T03:00:06.482 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd/lockd.ko 2023-09-23T03:00:06.482 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix 2023-09-23T03:00:06.482 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix/minix.ko 2023-09-23T03:00:06.482 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs 2023-09-23T03:00:06.482 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs/netfs.ko 2023-09-23T03:00:06.482 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs 2023-09-23T03:00:06.483 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout 2023-09-23T03:00:06.483 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-09-23T03:00:06.483 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout 2023-09-23T03:00:06.483 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-09-23T03:00:06.483 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout 2023-09-23T03:00:06.483 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-09-23T03:00:06.483 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfs.ko 2023-09-23T03:00:06.483 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv3.ko 2023-09-23T03:00:06.483 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv4.ko 2023-09-23T03:00:06.483 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common 2023-09-23T03:00:06.483 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/grace.ko 2023-09-23T03:00:06.483 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/nfs_acl.ko 2023-09-23T03:00:06.484 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd 2023-09-23T03:00:06.484 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd/nfsd.ko 2023-09-23T03:00:06.484 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2 2023-09-23T03:00:06.484 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2/nilfs2.ko 2023-09-23T03:00:06.484 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls 2023-09-23T03:00:06.484 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-celtic.ko 2023-09-23T03:00:06.484 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-centeuro.ko 2023-09-23T03:00:06.484 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-croatian.ko 2023-09-23T03:00:06.484 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-cyrillic.ko 2023-09-23T03:00:06.484 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-gaelic.ko 2023-09-23T03:00:06.484 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-greek.ko 2023-09-23T03:00:06.484 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-iceland.ko 2023-09-23T03:00:06.485 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-inuit.ko 2023-09-23T03:00:06.485 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-roman.ko 2023-09-23T03:00:06.485 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-romanian.ko 2023-09-23T03:00:06.485 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-turkish.ko 2023-09-23T03:00:06.485 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1250.ko 2023-09-23T03:00:06.485 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1251.ko 2023-09-23T03:00:06.485 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1255.ko 2023-09-23T03:00:06.485 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp737.ko 2023-09-23T03:00:06.485 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp775.ko 2023-09-23T03:00:06.485 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp850.ko 2023-09-23T03:00:06.485 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp852.ko 2023-09-23T03:00:06.485 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp855.ko 2023-09-23T03:00:06.486 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp857.ko 2023-09-23T03:00:06.486 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp860.ko 2023-09-23T03:00:06.486 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp861.ko 2023-09-23T03:00:06.486 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp862.ko 2023-09-23T03:00:06.486 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp863.ko 2023-09-23T03:00:06.486 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp864.ko 2023-09-23T03:00:06.486 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp865.ko 2023-09-23T03:00:06.486 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp866.ko 2023-09-23T03:00:06.486 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp869.ko 2023-09-23T03:00:06.486 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp874.ko 2023-09-23T03:00:06.486 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp932.ko 2023-09-23T03:00:06.486 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp936.ko 2023-09-23T03:00:06.487 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp949.ko 2023-09-23T03:00:06.487 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp950.ko 2023-09-23T03:00:06.487 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_euc-jp.ko 2023-09-23T03:00:06.487 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-1.ko 2023-09-23T03:00:06.487 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-13.ko 2023-09-23T03:00:06.487 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-14.ko 2023-09-23T03:00:06.487 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-15.ko 2023-09-23T03:00:06.487 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-2.ko 2023-09-23T03:00:06.487 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-3.ko 2023-09-23T03:00:06.487 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-4.ko 2023-09-23T03:00:06.487 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-5.ko 2023-09-23T03:00:06.487 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-6.ko 2023-09-23T03:00:06.487 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-7.ko 2023-09-23T03:00:06.488 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-9.ko 2023-09-23T03:00:06.488 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-r.ko 2023-09-23T03:00:06.488 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-ru.ko 2023-09-23T03:00:06.488 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-u.ko 2023-09-23T03:00:06.488 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_utf8.ko 2023-09-23T03:00:06.488 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2 2023-09-23T03:00:06.488 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster 2023-09-23T03:00:06.488 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-09-23T03:00:06.488 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm 2023-09-23T03:00:06.488 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-09-23T03:00:06.488 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs 2023-09-23T03:00:06.488 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-09-23T03:00:06.489 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2.ko 2023-09-23T03:00:06.489 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-09-23T03:00:06.489 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-09-23T03:00:06.489 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-09-23T03:00:06.489 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs 2023-09-23T03:00:06.489 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs/overlay.ko 2023-09-23T03:00:06.489 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore 2023-09-23T03:00:06.489 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore/ramoops.ko 2023-09-23T03:00:06.489 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs 2023-09-23T03:00:06.489 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs/reiserfs.ko 2023-09-23T03:00:06.489 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs 2023-09-23T03:00:06.489 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs/romfs.ko 2023-09-23T03:00:06.489 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb 2023-09-23T03:00:06.490 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client 2023-09-23T03:00:06.490 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client/cifs.ko 2023-09-23T03:00:06.490 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common 2023-09-23T03:00:06.490 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_arc4.ko 2023-09-23T03:00:06.490 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_md4.ko 2023-09-23T03:00:06.490 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs 2023-09-23T03:00:06.490 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs/squashfs.ko 2023-09-23T03:00:06.490 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv 2023-09-23T03:00:06.490 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv/sysv.ko 2023-09-23T03:00:06.490 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf 2023-09-23T03:00:06.490 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf/udf.ko 2023-09-23T03:00:06.490 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs 2023-09-23T03:00:06.491 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs/ufs.ko 2023-09-23T03:00:06.491 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs 2023-09-23T03:00:06.491 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs/xfs.ko 2023-09-23T03:00:06.491 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel 2023-09-23T03:00:06.491 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu 2023-09-23T03:00:06.491 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu/rcutorture.ko 2023-09-23T03:00:06.491 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/torture.ko 2023-09-23T03:00:06.491 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace 2023-09-23T03:00:06.491 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-09-23T03:00:06.491 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib 2023-09-23T03:00:06.491 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/asn1_encoder.ko 2023-09-23T03:00:06.491 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc-itu-t.ko 2023-09-23T03:00:06.491 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc8.ko 2023-09-23T03:00:06.492 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto 2023-09-23T03:00:06.492 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libarc4.ko 2023-09-23T03:00:06.492 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libdes.ko 2023-09-23T03:00:06.492 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/libcrc32c.ko 2023-09-23T03:00:06.492 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lru_cache.ko 2023-09-23T03:00:06.492 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4 2023-09-23T03:00:06.492 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4_compress.ko 2023-09-23T03:00:06.492 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4hc_compress.ko 2023-09-23T03:00:06.492 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math 2023-09-23T03:00:06.492 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math/cordic.ko 2023-09-23T03:00:06.492 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6 2023-09-23T03:00:06.492 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6/raid6_pq.ko 2023-09-23T03:00:06.493 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon 2023-09-23T03:00:06.493 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon/reed_solomon.ko 2023-09-23T03:00:06.493 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_bm.ko 2023-09-23T03:00:06.493 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_fsm.ko 2023-09-23T03:00:06.493 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_kmp.ko 2023-09-23T03:00:06.493 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd 2023-09-23T03:00:06.493 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd/zstd_compress.ko 2023-09-23T03:00:06.493 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm 2023-09-23T03:00:06.493 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm/hwpoison-inject.ko 2023-09-23T03:00:06.493 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net 2023-09-23T03:00:06.493 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan 2023-09-23T03:00:06.493 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/6lowpan.ko 2023-09-23T03:00:06.493 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_dest.ko 2023-09-23T03:00:06.494 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_fragment.ko 2023-09-23T03:00:06.494 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_hop.ko 2023-09-23T03:00:06.494 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_ipv6.ko 2023-09-23T03:00:06.494 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_mobility.ko 2023-09-23T03:00:06.494 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_routing.ko 2023-09-23T03:00:06.494 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_udp.ko 2023-09-23T03:00:06.494 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802 2023-09-23T03:00:06.494 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/garp.ko 2023-09-23T03:00:06.494 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/mrp.ko 2023-09-23T03:00:06.494 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/p8022.ko 2023-09-23T03:00:06.494 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/psnap.ko 2023-09-23T03:00:06.494 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/stp.ko 2023-09-23T03:00:06.495 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q 2023-09-23T03:00:06.495 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q/8021q.ko 2023-09-23T03:00:06.495 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p 2023-09-23T03:00:06.495 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet.ko 2023-09-23T03:00:06.495 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_fd.ko 2023-09-23T03:00:06.495 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_rdma.ko 2023-09-23T03:00:06.495 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_virtio.ko 2023-09-23T03:00:06.495 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv 2023-09-23T03:00:06.495 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv/batman-adv.ko 2023-09-23T03:00:06.495 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge 2023-09-23T03:00:06.495 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/br_netfilter.ko 2023-09-23T03:00:06.495 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/bridge.ko 2023-09-23T03:00:06.496 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter 2023-09-23T03:00:06.496 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-09-23T03:00:06.496 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_among.ko 2023-09-23T03:00:06.496 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-09-23T03:00:06.496 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-09-23T03:00:06.496 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-09-23T03:00:06.496 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-09-23T03:00:06.496 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-09-23T03:00:06.496 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-09-23T03:00:06.496 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_log.ko 2023-09-23T03:00:06.496 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-09-23T03:00:06.496 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-09-23T03:00:06.497 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-09-23T03:00:06.497 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-09-23T03:00:06.497 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-09-23T03:00:06.497 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-09-23T03:00:06.497 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-09-23T03:00:06.497 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-09-23T03:00:06.497 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-09-23T03:00:06.497 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-09-23T03:00:06.497 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-09-23T03:00:06.497 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtables.ko 2023-09-23T03:00:06.497 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-09-23T03:00:06.498 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-09-23T03:00:06.498 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-09-23T03:00:06.498 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph 2023-09-23T03:00:06.498 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph/libceph.ko 2023-09-23T03:00:06.498 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core 2023-09-23T03:00:06.498 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/failover.ko 2023-09-23T03:00:06.498 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/pktgen.ko 2023-09-23T03:00:06.498 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp 2023-09-23T03:00:06.498 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp.ko 2023-09-23T03:00:06.498 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_diag.ko 2023-09-23T03:00:06.498 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv4.ko 2023-09-23T03:00:06.498 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv6.ko 2023-09-23T03:00:06.499 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver 2023-09-23T03:00:06.499 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver/dns_resolver.ko 2023-09-23T03:00:06.499 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa 2023-09-23T03:00:06.499 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/dsa_core.ko 2023-09-23T03:00:06.499 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_brcm.ko 2023-09-23T03:00:06.499 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_dsa.ko 2023-09-23T03:00:06.499 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_none.ko 2023-09-23T03:00:06.499 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_trailer.ko 2023-09-23T03:00:06.499 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154 2023-09-23T03:00:06.499 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan 2023-09-23T03:00:06.499 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-09-23T03:00:06.499 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154.ko 2023-09-23T03:00:06.500 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154_socket.ko 2023-09-23T03:00:06.500 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4 2023-09-23T03:00:06.500 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ah4.ko 2023-09-23T03:00:06.500 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/esp4.ko 2023-09-23T03:00:06.500 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/fou.ko 2023-09-23T03:00:06.500 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/gre.ko 2023-09-23T03:00:06.500 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/inet_diag.ko 2023-09-23T03:00:06.500 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_gre.ko 2023-09-23T03:00:06.500 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_tunnel.ko 2023-09-23T03:00:06.500 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_vti.ko 2023-09-23T03:00:06.500 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipcomp.ko 2023-09-23T03:00:06.500 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipip.ko 2023-09-23T03:00:06.500 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter 2023-09-23T03:00:06.501 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-09-23T03:00:06.501 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-09-23T03:00:06.501 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-09-23T03:00:06.501 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-09-23T03:00:06.501 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-09-23T03:00:06.501 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-09-23T03:00:06.501 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-09-23T03:00:06.501 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-09-23T03:00:06.501 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-09-23T03:00:06.501 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-09-23T03:00:06.501 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-09-23T03:00:06.501 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-09-23T03:00:06.502 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-09-23T03:00:06.502 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-09-23T03:00:06.502 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-09-23T03:00:06.502 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-09-23T03:00:06.502 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-09-23T03:00:06.502 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-09-23T03:00:06.502 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-09-23T03:00:06.502 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-09-23T03:00:06.502 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-09-23T03:00:06.502 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_bic.ko 2023-09-23T03:00:06.502 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_dctcp.ko 2023-09-23T03:00:06.502 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_diag.ko 2023-09-23T03:00:06.503 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_highspeed.ko 2023-09-23T03:00:06.503 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_htcp.ko 2023-09-23T03:00:06.503 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_hybla.ko 2023-09-23T03:00:06.503 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_illinois.ko 2023-09-23T03:00:06.503 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_lp.ko 2023-09-23T03:00:06.503 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_scalable.ko 2023-09-23T03:00:06.503 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_vegas.ko 2023-09-23T03:00:06.503 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_veno.ko 2023-09-23T03:00:06.503 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_westwood.ko 2023-09-23T03:00:06.503 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_yeah.ko 2023-09-23T03:00:06.503 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tunnel4.ko 2023-09-23T03:00:06.503 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_diag.ko 2023-09-23T03:00:06.504 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_tunnel.ko 2023-09-23T03:00:06.504 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-09-23T03:00:06.504 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6 2023-09-23T03:00:06.504 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ah6.ko 2023-09-23T03:00:06.504 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/esp6.ko 2023-09-23T03:00:06.504 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/fou6.ko 2023-09-23T03:00:06.504 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_tunnel.ko 2023-09-23T03:00:06.504 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-09-23T03:00:06.504 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_vti.ko 2023-09-23T03:00:06.504 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ipcomp6.ko 2023-09-23T03:00:06.504 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter 2023-09-23T03:00:06.504 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-09-23T03:00:06.505 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-09-23T03:00:06.505 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-09-23T03:00:06.505 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-09-23T03:00:06.505 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-09-23T03:00:06.505 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-09-23T03:00:06.505 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-09-23T03:00:06.505 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-09-23T03:00:06.505 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-09-23T03:00:06.505 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-09-23T03:00:06.505 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-09-23T03:00:06.505 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-09-23T03:00:06.506 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-09-23T03:00:06.506 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-09-23T03:00:06.506 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-09-23T03:00:06.506 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-09-23T03:00:06.506 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-09-23T03:00:06.506 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-09-23T03:00:06.506 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-09-23T03:00:06.506 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-09-23T03:00:06.506 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-09-23T03:00:06.506 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-09-23T03:00:06.506 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-09-23T03:00:06.506 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-09-23T03:00:06.507 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/sit.ko 2023-09-23T03:00:06.507 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/tunnel6.ko 2023-09-23T03:00:06.507 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-09-23T03:00:06.507 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key 2023-09-23T03:00:06.507 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key/af_key.ko 2023-09-23T03:00:06.507 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp 2023-09-23T03:00:06.507 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_core.ko 2023-09-23T03:00:06.507 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_debugfs.ko 2023-09-23T03:00:06.507 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_eth.ko 2023-09-23T03:00:06.507 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip.ko 2023-09-23T03:00:06.507 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip6.ko 2023-09-23T03:00:06.507 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_netlink.ko 2023-09-23T03:00:06.508 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ppp.ko 2023-09-23T03:00:06.508 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc 2023-09-23T03:00:06.508 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc/llc.ko 2023-09-23T03:00:06.508 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154 2023-09-23T03:00:06.508 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154/mac802154.ko 2023-09-23T03:00:06.508 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls 2023-09-23T03:00:06.508 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls/mpls_gso.ko 2023-09-23T03:00:06.508 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter 2023-09-23T03:00:06.508 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset 2023-09-23T03:00:06.508 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set.ko 2023-09-23T03:00:06.508 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-09-23T03:00:06.508 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-09-23T03:00:06.509 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-09-23T03:00:06.509 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-09-23T03:00:06.509 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-09-23T03:00:06.509 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-09-23T03:00:06.509 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-09-23T03:00:06.509 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-09-23T03:00:06.509 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-09-23T03:00:06.509 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-09-23T03:00:06.509 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-09-23T03:00:06.509 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-09-23T03:00:06.509 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-09-23T03:00:06.509 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-09-23T03:00:06.509 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-09-23T03:00:06.510 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs 2023-09-23T03:00:06.510 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-09-23T03:00:06.510 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-09-23T03:00:06.510 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-09-23T03:00:06.510 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-09-23T03:00:06.510 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-09-23T03:00:06.510 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-09-23T03:00:06.510 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-09-23T03:00:06.510 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-09-23T03:00:06.510 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-09-23T03:00:06.510 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-09-23T03:00:06.510 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-09-23T03:00:06.511 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-09-23T03:00:06.511 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-09-23T03:00:06.511 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-09-23T03:00:06.511 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conncount.ko 2023-09-23T03:00:06.511 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack.ko 2023-09-23T03:00:06.511 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-09-23T03:00:06.511 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-09-23T03:00:06.511 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-09-23T03:00:06.511 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-09-23T03:00:06.511 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-09-23T03:00:06.511 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-09-23T03:00:06.511 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-09-23T03:00:06.512 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-09-23T03:00:06.512 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-09-23T03:00:06.512 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-09-23T03:00:06.512 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-09-23T03:00:06.512 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-09-23T03:00:06.512 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_dup_netdev.ko 2023-09-23T03:00:06.512 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table.ko 2023-09-23T03:00:06.512 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-09-23T03:00:06.512 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_log_syslog.ko 2023-09-23T03:00:06.512 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat.ko 2023-09-23T03:00:06.512 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_amanda.ko 2023-09-23T03:00:06.512 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_ftp.ko 2023-09-23T03:00:06.512 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_irc.ko 2023-09-23T03:00:06.513 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_sip.ko 2023-09-23T03:00:06.513 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_tftp.ko 2023-09-23T03:00:06.513 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_synproxy_core.ko 2023-09-23T03:00:06.513 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_tables.ko 2023-09-23T03:00:06.513 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink.ko 2023-09-23T03:00:06.513 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_acct.ko 2023-09-23T03:00:06.513 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_log.ko 2023-09-23T03:00:06.513 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_osf.ko 2023-09-23T03:00:06.513 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_queue.ko 2023-09-23T03:00:06.513 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_chain_nat.ko 2023-09-23T03:00:06.513 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_compat.ko 2023-09-23T03:00:06.513 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_ct.ko 2023-09-23T03:00:06.513 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_dup_netdev.ko 2023-09-23T03:00:06.514 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fib.ko 2023-09-23T03:00:06.514 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-09-23T03:00:06.514 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_hash.ko 2023-09-23T03:00:06.514 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_limit.ko 2023-09-23T03:00:06.514 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_log.ko 2023-09-23T03:00:06.514 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_masq.ko 2023-09-23T03:00:06.514 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_nat.ko 2023-09-23T03:00:06.514 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_numgen.ko 2023-09-23T03:00:06.514 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_queue.ko 2023-09-23T03:00:06.514 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_quota.ko 2023-09-23T03:00:06.514 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_redir.ko 2023-09-23T03:00:06.515 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject.ko 2023-09-23T03:00:06.515 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject_inet.ko 2023-09-23T03:00:06.515 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_socket.ko 2023-09-23T03:00:06.515 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_tunnel.ko 2023-09-23T03:00:06.515 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_AUDIT.ko 2023-09-23T03:00:06.515 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-09-23T03:00:06.515 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-09-23T03:00:06.515 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-09-23T03:00:06.515 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CT.ko 2023-09-23T03:00:06.515 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_DSCP.ko 2023-09-23T03:00:06.515 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HL.ko 2023-09-23T03:00:06.515 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HMARK.ko 2023-09-23T03:00:06.515 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-09-23T03:00:06.516 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LED.ko 2023-09-23T03:00:06.516 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LOG.ko 2023-09-23T03:00:06.516 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-09-23T03:00:06.516 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NETMAP.ko 2023-09-23T03:00:06.516 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFLOG.ko 2023-09-23T03:00:06.516 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-09-23T03:00:06.516 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_RATEEST.ko 2023-09-23T03:00:06.516 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_REDIRECT.ko 2023-09-23T03:00:06.516 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_SECMARK.ko 2023-09-23T03:00:06.516 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPMSS.ko 2023-09-23T03:00:06.516 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-09-23T03:00:06.516 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TEE.ko 2023-09-23T03:00:06.517 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TPROXY.ko 2023-09-23T03:00:06.517 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TRACE.ko 2023-09-23T03:00:06.517 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_addrtype.ko 2023-09-23T03:00:06.517 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_bpf.ko 2023-09-23T03:00:06.517 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cgroup.ko 2023-09-23T03:00:06.517 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cluster.ko 2023-09-23T03:00:06.517 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_comment.ko 2023-09-23T03:00:06.517 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connbytes.ko 2023-09-23T03:00:06.517 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlabel.ko 2023-09-23T03:00:06.517 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlimit.ko 2023-09-23T03:00:06.517 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connmark.ko 2023-09-23T03:00:06.517 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_conntrack.ko 2023-09-23T03:00:06.518 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cpu.ko 2023-09-23T03:00:06.518 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dccp.ko 2023-09-23T03:00:06.518 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_devgroup.ko 2023-09-23T03:00:06.518 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dscp.ko 2023-09-23T03:00:06.518 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ecn.ko 2023-09-23T03:00:06.518 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_esp.ko 2023-09-23T03:00:06.518 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hashlimit.ko 2023-09-23T03:00:06.518 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_helper.ko 2023-09-23T03:00:06.518 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hl.ko 2023-09-23T03:00:06.518 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipcomp.ko 2023-09-23T03:00:06.518 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_iprange.ko 2023-09-23T03:00:06.519 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipvs.ko 2023-09-23T03:00:06.519 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_l2tp.ko 2023-09-23T03:00:06.519 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_length.ko 2023-09-23T03:00:06.519 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_limit.ko 2023-09-23T03:00:06.519 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mac.ko 2023-09-23T03:00:06.519 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mark.ko 2023-09-23T03:00:06.519 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_multiport.ko 2023-09-23T03:00:06.519 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nat.ko 2023-09-23T03:00:06.519 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nfacct.ko 2023-09-23T03:00:06.519 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_osf.ko 2023-09-23T03:00:06.519 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_owner.ko 2023-09-23T03:00:06.519 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_physdev.ko 2023-09-23T03:00:06.519 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_pkttype.ko 2023-09-23T03:00:06.520 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_policy.ko 2023-09-23T03:00:06.520 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_quota.ko 2023-09-23T03:00:06.520 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_rateest.ko 2023-09-23T03:00:06.520 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_realm.ko 2023-09-23T03:00:06.520 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_recent.ko 2023-09-23T03:00:06.520 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_sctp.ko 2023-09-23T03:00:06.520 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_set.ko 2023-09-23T03:00:06.520 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_socket.ko 2023-09-23T03:00:06.520 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_state.ko 2023-09-23T03:00:06.520 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_statistic.ko 2023-09-23T03:00:06.520 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_string.ko 2023-09-23T03:00:06.520 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_tcpmss.ko 2023-09-23T03:00:06.521 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_time.ko 2023-09-23T03:00:06.521 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_u32.ko 2023-09-23T03:00:06.521 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink 2023-09-23T03:00:06.521 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink/netlink_diag.ko 2023-09-23T03:00:06.521 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh 2023-09-23T03:00:06.521 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh/nsh.ko 2023-09-23T03:00:06.521 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch 2023-09-23T03:00:06.521 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/openvswitch.ko 2023-09-23T03:00:06.521 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-geneve.ko 2023-09-23T03:00:06.521 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-gre.ko 2023-09-23T03:00:06.521 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-vxlan.ko 2023-09-23T03:00:06.521 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet 2023-09-23T03:00:06.522 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet/af_packet_diag.ko 2023-09-23T03:00:06.522 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds 2023-09-23T03:00:06.522 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds.ko 2023-09-23T03:00:06.522 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_rdma.ko 2023-09-23T03:00:06.522 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_tcp.ko 2023-09-23T03:00:06.522 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched 2023-09-23T03:00:06.522 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_csum.ko 2023-09-23T03:00:06.522 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_gact.ko 2023-09-23T03:00:06.522 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_ipt.ko 2023-09-23T03:00:06.522 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_mirred.ko 2023-09-23T03:00:06.522 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_nat.ko 2023-09-23T03:00:06.523 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_pedit.ko 2023-09-23T03:00:06.523 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_police.ko 2023-09-23T03:00:06.523 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_simple.ko 2023-09-23T03:00:06.523 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_skbedit.ko 2023-09-23T03:00:06.523 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_vlan.ko 2023-09-23T03:00:06.523 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_basic.ko 2023-09-23T03:00:06.523 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_bpf.ko 2023-09-23T03:00:06.523 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_flow.ko 2023-09-23T03:00:06.523 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_fw.ko 2023-09-23T03:00:06.523 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_route.ko 2023-09-23T03:00:06.523 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_u32.ko 2023-09-23T03:00:06.523 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_cmp.ko 2023-09-23T03:00:06.523 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_ipset.ko 2023-09-23T03:00:06.524 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_meta.ko 2023-09-23T03:00:06.524 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_nbyte.ko 2023-09-23T03:00:06.524 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_text.ko 2023-09-23T03:00:06.524 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_u32.ko 2023-09-23T03:00:06.524 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_choke.ko 2023-09-23T03:00:06.524 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_codel.ko 2023-09-23T03:00:06.524 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_drr.ko 2023-09-23T03:00:06.524 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_fq.ko 2023-09-23T03:00:06.524 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_gred.ko 2023-09-23T03:00:06.524 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hfsc.ko 2023-09-23T03:00:06.524 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hhf.ko 2023-09-23T03:00:06.524 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_htb.ko 2023-09-23T03:00:06.525 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_ingress.ko 2023-09-23T03:00:06.525 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio.ko 2023-09-23T03:00:06.525 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio_lib.ko 2023-09-23T03:00:06.525 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_multiq.ko 2023-09-23T03:00:06.525 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_netem.ko 2023-09-23T03:00:06.525 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_pie.ko 2023-09-23T03:00:06.525 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_plug.ko 2023-09-23T03:00:06.525 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_prio.ko 2023-09-23T03:00:06.525 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_qfq.ko 2023-09-23T03:00:06.525 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_red.ko 2023-09-23T03:00:06.525 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfb.ko 2023-09-23T03:00:06.526 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfq.ko 2023-09-23T03:00:06.526 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_tbf.ko 2023-09-23T03:00:06.526 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_teql.ko 2023-09-23T03:00:06.526 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp 2023-09-23T03:00:06.526 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp.ko 2023-09-23T03:00:06.526 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp_diag.ko 2023-09-23T03:00:06.526 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc 2023-09-23T03:00:06.526 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss 2023-09-23T03:00:06.526 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-09-23T03:00:06.526 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-09-23T03:00:06.526 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/sunrpc.ko 2023-09-23T03:00:06.527 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma 2023-09-23T03:00:06.527 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-09-23T03:00:06.527 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc 2023-09-23T03:00:06.527 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/diag.ko 2023-09-23T03:00:06.527 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/tipc.ko 2023-09-23T03:00:06.527 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix 2023-09-23T03:00:06.527 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix/unix_diag.ko 2023-09-23T03:00:06.527 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock 2023-09-23T03:00:06.527 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-09-23T03:00:06.527 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-09-23T03:00:06.527 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock.ko 2023-09-23T03:00:06.527 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_diag.ko 2023-09-23T03:00:06.528 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-09-23T03:00:06.528 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm 2023-09-23T03:00:06.528 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-09-23T03:00:06.528 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security 2023-09-23T03:00:06.528 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys 2023-09-23T03:00:06.528 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys 2023-09-23T03:00:06.528 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-09-23T03:00:06.528 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys 2023-09-23T03:00:06.528 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys/trusted.ko 2023-09-23T03:00:06.528 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt 2023-09-23T03:00:06.528 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib 2023-09-23T03:00:06.528 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib/irqbypass.ko 2023-09-23T03:00:06.528 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias 2023-09-23T03:00:06.529 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias.bin 2023-09-23T03:00:06.529 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin 2023-09-23T03:00:06.529 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.alias.bin 2023-09-23T03:00:06.529 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.bin 2023-09-23T03:00:06.529 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.modinfo 2023-09-23T03:00:06.529 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep 2023-09-23T03:00:06.529 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep.bin 2023-09-23T03:00:06.529 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.devname 2023-09-23T03:00:06.529 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.order 2023-09-23T03:00:06.529 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.softdep 2023-09-23T03:00:06.529 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols 2023-09-23T03:00:06.529 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols.bin 2023-09-23T03:00:06.530 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc7-gfef64f93da2b 2023-09-23T03:00:06.530 DEBUG:teuthology.orchestra.run.smithi154:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-09-23T03:00:06.552 INFO:teuthology.orchestra.run.smithi154.stdout:postinstall scriptlet (using /bin/sh): 2023-09-23T03:00:06.552 INFO:teuthology.orchestra.run.smithi154.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b -a -r /boot/System.map-6.5.0-rc7-gfef64f93da2b ]; then 2023-09-23T03:00:06.552 INFO:teuthology.orchestra.run.smithi154.stdout:cp /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm 2023-09-23T03:00:06.553 INFO:teuthology.orchestra.run.smithi154.stdout:cp /boot/System.map-6.5.0-rc7-gfef64f93da2b /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-09-23T03:00:06.553 INFO:teuthology.orchestra.run.smithi154.stdout:rm -f /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-09-23T03:00:06.553 INFO:teuthology.orchestra.run.smithi154.stdout:/sbin/installkernel 6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-09-23T03:00:06.553 INFO:teuthology.orchestra.run.smithi154.stdout:rm -f /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-09-23T03:00:06.553 INFO:teuthology.orchestra.run.smithi154.stdout:fi 2023-09-23T03:00:06.553 INFO:teuthology.orchestra.run.smithi154.stdout:preuninstall scriptlet (using /bin/sh): 2023-09-23T03:00:06.553 INFO:teuthology.orchestra.run.smithi154.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-09-23T03:00:06.553 INFO:teuthology.orchestra.run.smithi154.stdout:new-kernel-pkg --remove 6.5.0-rc7-gfef64f93da2b --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc7-gfef64f93da2b.img 2023-09-23T03:00:06.553 INFO:teuthology.orchestra.run.smithi154.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-09-23T03:00:06.553 INFO:teuthology.orchestra.run.smithi154.stdout:kernel-install remove 6.5.0-rc7-gfef64f93da2b 2023-09-23T03:00:06.553 INFO:teuthology.orchestra.run.smithi154.stdout:fi 2023-09-23T03:00:06.554 INFO:teuthology.orchestra.run.smithi154.stdout:postuninstall scriptlet (using /bin/sh): 2023-09-23T03:00:06.554 INFO:teuthology.orchestra.run.smithi154.stdout:if [ -x /sbin/update-bootloader ]; then 2023-09-23T03:00:06.554 INFO:teuthology.orchestra.run.smithi154.stdout:/sbin/update-bootloader --remove 6.5.0-rc7-gfef64f93da2b 2023-09-23T03:00:06.554 INFO:teuthology.orchestra.run.smithi154.stdout:fi 2023-09-23T03:00:06.555 DEBUG:teuthology.orchestra.run.smithi154:> sudo rpm -qi grub2-tools 2023-09-23T03:00:06.637 INFO:teuthology.orchestra.run.smithi154.stdout:Name : grub2-tools 2023-09-23T03:00:06.792 INFO:teuthology.orchestra.run.smithi154.stdout:Epoch : 1 2023-09-23T03:00:06.792 INFO:teuthology.orchestra.run.smithi154.stdout:Version : 2.06 2023-09-23T03:00:06.792 INFO:teuthology.orchestra.run.smithi154.stdout:Release : 68.el9 2023-09-23T03:00:06.792 INFO:teuthology.orchestra.run.smithi154.stdout:Architecture: x86_64 2023-09-23T03:00:06.792 INFO:teuthology.orchestra.run.smithi154.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-09-23T03:00:06.792 INFO:teuthology.orchestra.run.smithi154.stdout:Group : Unspecified 2023-09-23T03:00:06.793 INFO:teuthology.orchestra.run.smithi154.stdout:Size : 8264254 2023-09-23T03:00:06.793 INFO:teuthology.orchestra.run.smithi154.stdout:License : GPLv3+ 2023-09-23T03:00:06.793 INFO:teuthology.orchestra.run.smithi154.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-09-23T03:00:06.793 INFO:teuthology.orchestra.run.smithi154.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-09-23T03:00:06.793 INFO:teuthology.orchestra.run.smithi154.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-09-23T03:00:06.793 INFO:teuthology.orchestra.run.smithi154.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-09-23T03:00:06.793 INFO:teuthology.orchestra.run.smithi154.stdout:Packager : builder@centos.org 2023-09-23T03:00:06.793 INFO:teuthology.orchestra.run.smithi154.stdout:Vendor : CentOS 2023-09-23T03:00:06.793 INFO:teuthology.orchestra.run.smithi154.stdout:URL : http://www.gnu.org/software/grub/ 2023-09-23T03:00:06.793 INFO:teuthology.orchestra.run.smithi154.stdout:Summary : Support tools for GRUB. 2023-09-23T03:00:06.793 INFO:teuthology.orchestra.run.smithi154.stdout:Description : 2023-09-23T03:00:06.793 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:00:06.794 INFO:teuthology.orchestra.run.smithi154.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-09-23T03:00:06.794 INFO:teuthology.orchestra.run.smithi154.stdout:customizable bootloader with modular architecture. It supports a rich 2023-09-23T03:00:06.794 INFO:teuthology.orchestra.run.smithi154.stdout:variety of kernel formats, file systems, computer architectures and 2023-09-23T03:00:06.794 INFO:teuthology.orchestra.run.smithi154.stdout:hardware devices. 2023-09-23T03:00:06.794 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:00:06.794 INFO:teuthology.orchestra.run.smithi154.stdout:This subpackage provides tools for support of all platforms. 2023-09-23T03:00:06.795 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-09-23T03:00:06.795 INFO:teuthology.task.kernel:Updating grub on smithi154 to boot 6.5.0-rc7-gfef64f93da2b 2023-09-23T03:00:06.795 DEBUG:teuthology.orchestra.run.smithi154:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-09-23T03:00:07.401 INFO:teuthology.orchestra.run.smithi154.stderr:Generating grub configuration file ... 2023-09-23T03:00:08.756 INFO:teuthology.orchestra.run.smithi154.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-09-23T03:00:08.795 INFO:teuthology.orchestra.run.smithi154.stderr:done 2023-09-23T03:00:08.798 DEBUG:teuthology.orchestra.run.smithi154:> mktemp 2023-09-23T03:00:08.812 INFO:teuthology.orchestra.run.smithi154.stdout:/tmp/tmp.FogRtJO06i 2023-09-23T03:00:08.812 DEBUG:teuthology.orchestra.run.smithi154:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.FogRtJO06i 2023-09-23T03:00:08.888 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0666 /tmp/tmp.FogRtJO06i 2023-09-23T03:00:09.032 DEBUG:teuthology.orchestra.remote:smithi154:/tmp/tmp.FogRtJO06i is 6KB 2023-09-23T03:00:09.080 DEBUG:teuthology.orchestra.run.smithi154:> rm -fr /tmp/tmp.FogRtJO06i 2023-09-23T03:00:09.094 DEBUG:teuthology.orchestra.run.smithi154:> sudo /bin/ls /boot/loader/entries || true 2023-09-23T03:00:09.165 INFO:teuthology.orchestra.run.smithi154.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-09-23T03:00:09.166 INFO:teuthology.orchestra.run.smithi154.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-09-23T03:00:09.166 INFO:teuthology.orchestra.run.smithi154.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-09-23T03:00:09.166 INFO:teuthology.orchestra.run.smithi154.stdout:15949e3c88704300882482eddfd7baae-6.5.0-rc7-gfef64f93da2b.conf 2023-09-23T03:00:09.167 DEBUG:teuthology.orchestra.run.smithi154:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.5.0-rc7-gfef64f93da2b 2023-09-23T03:00:09.312 DEBUG:teuthology.orchestra.run.smithi154:> sudo shutdown -r now 2023-09-23T03:00:33.257 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi190.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-09-23T03:00:33.257 DEBUG:teuthology.orchestra.run.smithi190:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-09-23T03:00:33.326 INFO:teuthology.orchestra.run.smithi190.stdout:/boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-09-23T03:00:33.326 INFO:teuthology.orchestra.run.smithi190.stdout:/boot/config-6.5.0-rc7-gfef64f93da2b 2023-09-23T03:00:33.326 INFO:teuthology.orchestra.run.smithi190.stdout:/boot/vmlinuz-6.5.0-rc7-gfef64f93da2b 2023-09-23T03:00:33.326 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b 2023-09-23T03:00:33.326 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel 2023-09-23T03:00:33.327 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch 2023-09-23T03:00:33.327 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86 2023-09-23T03:00:33.327 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto 2023-09-23T03:00:33.327 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-09-23T03:00:33.327 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-09-23T03:00:33.327 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-09-23T03:00:33.327 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-09-23T03:00:33.327 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-09-23T03:00:33.327 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-09-23T03:00:33.328 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-09-23T03:00:33.328 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32c-intel.ko 2023-09-23T03:00:33.328 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-09-23T03:00:33.328 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-09-23T03:00:33.328 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-09-23T03:00:33.328 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-09-23T03:00:33.328 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-09-23T03:00:33.328 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-09-23T03:00:33.328 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-09-23T03:00:33.328 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-09-23T03:00:33.328 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-09-23T03:00:33.329 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-09-23T03:00:33.329 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-09-23T03:00:33.329 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-09-23T03:00:33.329 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel 2023-09-23T03:00:33.329 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu 2023-09-23T03:00:33.329 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce 2023-09-23T03:00:33.329 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-09-23T03:00:33.329 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm 2023-09-23T03:00:33.329 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-amd.ko 2023-09-23T03:00:33.329 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-intel.ko 2023-09-23T03:00:33.329 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm.ko 2023-09-23T03:00:33.330 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto 2023-09-23T03:00:33.330 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ansi_cprng.ko 2023-09-23T03:00:33.330 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/anubis.ko 2023-09-23T03:00:33.330 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/arc4.ko 2023-09-23T03:00:33.330 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx 2023-09-23T03:00:33.330 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_memcpy.ko 2023-09-23T03:00:33.330 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_pq.ko 2023-09-23T03:00:33.330 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-09-23T03:00:33.330 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_tx.ko 2023-09-23T03:00:33.330 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_xor.ko 2023-09-23T03:00:33.330 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/raid6test.ko 2023-09-23T03:00:33.330 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authenc.ko 2023-09-23T03:00:33.331 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authencesn.ko 2023-09-23T03:00:33.331 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blake2b_generic.ko 2023-09-23T03:00:33.331 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_common.ko 2023-09-23T03:00:33.331 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_generic.ko 2023-09-23T03:00:33.331 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/camellia_generic.ko 2023-09-23T03:00:33.331 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast5_generic.ko 2023-09-23T03:00:33.331 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast6_generic.ko 2023-09-23T03:00:33.331 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast_common.ko 2023-09-23T03:00:33.331 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ccm.ko 2023-09-23T03:00:33.331 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cmac.ko 2023-09-23T03:00:33.331 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crc32_generic.ko 2023-09-23T03:00:33.331 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_engine.ko 2023-09-23T03:00:33.332 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_user.ko 2023-09-23T03:00:33.332 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cts.ko 2023-09-23T03:00:33.332 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/des_generic.ko 2023-09-23T03:00:33.332 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/dh_generic.ko 2023-09-23T03:00:33.332 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/echainiv.ko 2023-09-23T03:00:33.332 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/essiv.ko 2023-09-23T03:00:33.332 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/fcrypt.ko 2023-09-23T03:00:33.332 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/khazad.ko 2023-09-23T03:00:33.332 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4.ko 2023-09-23T03:00:33.332 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4hc.ko 2023-09-23T03:00:33.332 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/md4.ko 2023-09-23T03:00:33.332 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/michael_mic.ko 2023-09-23T03:00:33.333 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcbc.ko 2023-09-23T03:00:33.333 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcrypt.ko 2023-09-23T03:00:33.333 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/rmd160.ko 2023-09-23T03:00:33.333 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/seed.ko 2023-09-23T03:00:33.333 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/serpent_generic.ko 2023-09-23T03:00:33.333 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tcrypt.ko 2023-09-23T03:00:33.333 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tea.ko 2023-09-23T03:00:33.333 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_common.ko 2023-09-23T03:00:33.333 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_generic.ko 2023-09-23T03:00:33.333 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/vmac.ko 2023-09-23T03:00:33.333 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/wp512.ko 2023-09-23T03:00:33.333 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xcbc.ko 2023-09-23T03:00:33.334 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xor.ko 2023-09-23T03:00:33.334 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xxhash_generic.ko 2023-09-23T03:00:33.334 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers 2023-09-23T03:00:33.334 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi 2023-09-23T03:00:33.334 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_ipmi.ko 2023-09-23T03:00:33.334 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_pad.ko 2023-09-23T03:00:33.334 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/custom_method.ko 2023-09-23T03:00:33.334 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/ec_sys.ko 2023-09-23T03:00:33.334 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/platform_profile.ko 2023-09-23T03:00:33.334 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbs.ko 2023-09-23T03:00:33.334 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbshc.ko 2023-09-23T03:00:33.335 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/video.ko 2023-09-23T03:00:33.335 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata 2023-09-23T03:00:33.335 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/acard-ahci.ko 2023-09-23T03:00:33.335 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ahci_platform.ko 2023-09-23T03:00:33.335 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ata_generic.ko 2023-09-23T03:00:33.335 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/libahci_platform.ko 2023-09-23T03:00:33.335 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_acpi.ko 2023-09-23T03:00:33.335 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ali.ko 2023-09-23T03:00:33.335 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_amd.ko 2023-09-23T03:00:33.335 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_artop.ko 2023-09-23T03:00:33.335 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atiixp.ko 2023-09-23T03:00:33.335 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atp867x.ko 2023-09-23T03:00:33.336 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd640.ko 2023-09-23T03:00:33.336 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd64x.ko 2023-09-23T03:00:33.336 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cypress.ko 2023-09-23T03:00:33.336 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_efar.ko 2023-09-23T03:00:33.336 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt366.ko 2023-09-23T03:00:33.336 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt37x.ko 2023-09-23T03:00:33.336 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-09-23T03:00:33.336 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x3.ko 2023-09-23T03:00:33.336 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it8213.ko 2023-09-23T03:00:33.336 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it821x.ko 2023-09-23T03:00:33.336 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_jmicron.ko 2023-09-23T03:00:33.336 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_marvell.ko 2023-09-23T03:00:33.337 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_mpiix.ko 2023-09-23T03:00:33.337 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_netcell.ko 2023-09-23T03:00:33.337 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ninja32.ko 2023-09-23T03:00:33.337 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87410.ko 2023-09-23T03:00:33.337 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87415.ko 2023-09-23T03:00:33.337 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_oldpiix.ko 2023-09-23T03:00:33.337 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_opti.ko 2023-09-23T03:00:33.337 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_optidma.ko 2023-09-23T03:00:33.337 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pcmcia.ko 2023-09-23T03:00:33.337 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc2027x.ko 2023-09-23T03:00:33.338 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-09-23T03:00:33.338 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_piccolo.ko 2023-09-23T03:00:33.338 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_rdc.ko 2023-09-23T03:00:33.338 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sch.ko 2023-09-23T03:00:33.338 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_serverworks.ko 2023-09-23T03:00:33.338 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sil680.ko 2023-09-23T03:00:33.338 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sis.ko 2023-09-23T03:00:33.338 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sl82c105.ko 2023-09-23T03:00:33.338 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_triflex.ko 2023-09-23T03:00:33.338 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_via.ko 2023-09-23T03:00:33.338 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pdc_adma.ko 2023-09-23T03:00:33.338 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_inic162x.ko 2023-09-23T03:00:33.339 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_mv.ko 2023-09-23T03:00:33.339 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_nv.ko 2023-09-23T03:00:33.339 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_promise.ko 2023-09-23T03:00:33.339 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_qstor.ko 2023-09-23T03:00:33.339 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil.ko 2023-09-23T03:00:33.339 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil24.ko 2023-09-23T03:00:33.339 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sis.ko 2023-09-23T03:00:33.339 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_svw.ko 2023-09-23T03:00:33.339 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sx4.ko 2023-09-23T03:00:33.341 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_uli.ko 2023-09-23T03:00:33.341 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_via.ko 2023-09-23T03:00:33.341 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_vsc.ko 2023-09-23T03:00:33.342 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay 2023-09-23T03:00:33.342 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay/ks0108.ko 2023-09-23T03:00:33.342 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base 2023-09-23T03:00:33.342 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap 2023-09-23T03:00:33.342 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-i2c.ko 2023-09-23T03:00:33.342 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-mmio.ko 2023-09-23T03:00:33.342 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma 2023-09-23T03:00:33.342 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma/bcma.ko 2023-09-23T03:00:33.342 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block 2023-09-23T03:00:33.342 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe 2023-09-23T03:00:33.343 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe/aoe.ko 2023-09-23T03:00:33.343 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/brd.ko 2023-09-23T03:00:33.343 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd 2023-09-23T03:00:33.343 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd/drbd.ko 2023-09-23T03:00:33.343 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/floppy.ko 2023-09-23T03:00:33.343 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/loop.ko 2023-09-23T03:00:33.343 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx 2023-09-23T03:00:33.343 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-09-23T03:00:33.343 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/nbd.ko 2023-09-23T03:00:33.343 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk 2023-09-23T03:00:33.343 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk/null_blk.ko 2023-09-23T03:00:33.344 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/pktcdvd.ko 2023-09-23T03:00:33.344 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/rbd.ko 2023-09-23T03:00:33.344 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/virtio_blk.ko 2023-09-23T03:00:33.344 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback 2023-09-23T03:00:33.344 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-09-23T03:00:33.344 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkfront.ko 2023-09-23T03:00:33.344 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram 2023-09-23T03:00:33.344 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram/zram.ko 2023-09-23T03:00:33.344 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char 2023-09-23T03:00:33.344 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hangcheck-timer.ko 2023-09-23T03:00:33.345 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random 2023-09-23T03:00:33.345 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/amd-rng.ko 2023-09-23T03:00:33.345 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/intel-rng.ko 2023-09-23T03:00:33.345 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-09-23T03:00:33.345 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/via-rng.ko 2023-09-23T03:00:33.345 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-09-23T03:00:33.345 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi 2023-09-23T03:00:33.345 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-09-23T03:00:33.345 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-09-23T03:00:33.345 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-09-23T03:00:33.345 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-09-23T03:00:33.346 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-09-23T03:00:33.346 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-09-23T03:00:33.346 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/lp.ko 2023-09-23T03:00:33.346 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave 2023-09-23T03:00:33.346 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave/mwave.ko 2023-09-23T03:00:33.346 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ppdev.ko 2023-09-23T03:00:33.346 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tlclk.ko 2023-09-23T03:00:33.346 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm 2023-09-23T03:00:33.346 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm.ko 2023-09-23T03:00:33.346 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-09-23T03:00:33.346 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-09-23T03:00:33.346 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-09-23T03:00:33.347 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis.ko 2023-09-23T03:00:33.347 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-09-23T03:00:33.347 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/uv_mmtimer.ko 2023-09-23T03:00:33.347 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/virtio_console.ko 2023-09-23T03:00:33.347 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq 2023-09-23T03:00:33.347 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-09-23T03:00:33.347 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-09-23T03:00:33.347 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-09-23T03:00:33.347 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-09-23T03:00:33.347 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/powernow-k8.ko 2023-09-23T03:00:33.347 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-09-23T03:00:33.347 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto 2023-09-23T03:00:33.348 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp 2023-09-23T03:00:33.348 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-09-23T03:00:33.348 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp.ko 2023-09-23T03:00:33.348 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel 2023-09-23T03:00:33.348 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat 2023-09-23T03:00:33.348 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common 2023-09-23T03:00:33.348 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-09-23T03:00:33.348 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-09-23T03:00:33.348 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-09-23T03:00:33.348 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-aes.ko 2023-09-23T03:00:33.348 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-sha.ko 2023-09-23T03:00:33.349 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio 2023-09-23T03:00:33.349 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-09-23T03:00:33.349 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca 2023-09-23T03:00:33.349 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca/dca.ko 2023-09-23T03:00:33.349 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq 2023-09-23T03:00:33.349 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-09-23T03:00:33.349 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma 2023-09-23T03:00:33.349 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw 2023-09-23T03:00:33.349 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw/dw_dmac.ko 2023-09-23T03:00:33.349 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat 2023-09-23T03:00:33.349 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat/ioatdma.ko 2023-09-23T03:00:33.350 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac 2023-09-23T03:00:33.350 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/amd64_edac.ko 2023-09-23T03:00:33.350 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/e752x_edac.ko 2023-09-23T03:00:33.350 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/edac_mce_amd.ko 2023-09-23T03:00:33.350 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3000_edac.ko 2023-09-23T03:00:33.350 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3200_edac.ko 2023-09-23T03:00:33.350 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5100_edac.ko 2023-09-23T03:00:33.350 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5400_edac.ko 2023-09-23T03:00:33.350 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7300_edac.ko 2023-09-23T03:00:33.350 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7core_edac.ko 2023-09-23T03:00:33.350 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i82975x_edac.ko 2023-09-23T03:00:33.350 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/ie31200_edac.ko 2023-09-23T03:00:33.351 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/sb_edac.ko 2023-09-23T03:00:33.351 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/x38_edac.ko 2023-09-23T03:00:33.351 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware 2023-09-23T03:00:33.351 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/edd.ko 2023-09-23T03:00:33.351 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/iscsi_ibft.ko 2023-09-23T03:00:33.351 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio 2023-09-23T03:00:33.351 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-ich.ko 2023-09-23T03:00:33.351 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-viperboard.ko 2023-09-23T03:00:33.351 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid 2023-09-23T03:00:33.351 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-appleir.ko 2023-09-23T03:00:33.351 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-aureal.ko 2023-09-23T03:00:33.352 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-axff.ko 2023-09-23T03:00:33.352 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-dr.ko 2023-09-23T03:00:33.352 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elecom.ko 2023-09-23T03:00:33.352 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elo.ko 2023-09-23T03:00:33.352 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-emsff.ko 2023-09-23T03:00:33.352 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gaff.ko 2023-09-23T03:00:33.352 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gt683r.ko 2023-09-23T03:00:33.352 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gyration.ko 2023-09-23T03:00:33.352 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-09-23T03:00:33.352 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-09-23T03:00:33.352 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtekff.ko 2023-09-23T03:00:33.352 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-hyperv.ko 2023-09-23T03:00:33.353 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-icade.ko 2023-09-23T03:00:33.353 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-keytouch.ko 2023-09-23T03:00:33.353 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-kye.ko 2023-09-23T03:00:33.353 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lcpower.ko 2023-09-23T03:00:33.353 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-led.ko 2023-09-23T03:00:33.353 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lenovo.ko 2023-09-23T03:00:33.353 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-dj.ko 2023-09-23T03:00:33.353 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-09-23T03:00:33.353 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-multitouch.ko 2023-09-23T03:00:33.353 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-ortek.ko 2023-09-23T03:00:33.353 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-penmount.ko 2023-09-23T03:00:33.354 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-petalynx.ko 2023-09-23T03:00:33.354 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-picolcd.ko 2023-09-23T03:00:33.354 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-pl.ko 2023-09-23T03:00:33.354 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-plantronics.ko 2023-09-23T03:00:33.354 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-primax.ko 2023-09-23T03:00:33.354 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-rmi.ko 2023-09-23T03:00:33.354 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-09-23T03:00:33.354 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-common.ko 2023-09-23T03:00:33.354 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-isku.ko 2023-09-23T03:00:33.354 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kone.ko 2023-09-23T03:00:33.354 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-09-23T03:00:33.354 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-09-23T03:00:33.355 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-09-23T03:00:33.355 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-lua.ko 2023-09-23T03:00:33.355 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-09-23T03:00:33.355 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-09-23T03:00:33.355 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-savu.ko 2023-09-23T03:00:33.355 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat.ko 2023-09-23T03:00:33.355 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-saitek.ko 2023-09-23T03:00:33.355 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-samsung.ko 2023-09-23T03:00:33.355 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sensor-hub.ko 2023-09-23T03:00:33.355 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sjoy.ko 2023-09-23T03:00:33.356 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sony.ko 2023-09-23T03:00:33.356 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-speedlink.ko 2023-09-23T03:00:33.356 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-steelseries.ko 2023-09-23T03:00:33.356 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sunplus.ko 2023-09-23T03:00:33.356 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-thrustmaster.ko 2023-09-23T03:00:33.356 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tivo.ko 2023-09-23T03:00:33.356 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tmff.ko 2023-09-23T03:00:33.356 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-topseed.ko 2023-09-23T03:00:33.356 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-twinhan.ko 2023-09-23T03:00:33.356 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-uclogic.ko 2023-09-23T03:00:33.356 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-waltop.ko 2023-09-23T03:00:33.357 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-wiimote.ko 2023-09-23T03:00:33.357 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-xinmo.ko 2023-09-23T03:00:33.357 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zpff.ko 2023-09-23T03:00:33.357 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zydacron.ko 2023-09-23T03:00:33.357 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/uhid.ko 2023-09-23T03:00:33.357 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/wacom.ko 2023-09-23T03:00:33.357 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv 2023-09-23T03:00:33.357 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_balloon.ko 2023-09-23T03:00:33.357 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_utils.ko 2023-09-23T03:00:33.357 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_vmbus.ko 2023-09-23T03:00:33.357 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon 2023-09-23T03:00:33.358 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru.ko 2023-09-23T03:00:33.358 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru3.ko 2023-09-23T03:00:33.358 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-09-23T03:00:33.358 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7414.ko 2023-09-23T03:00:33.358 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7418.ko 2023-09-23T03:00:33.358 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adc128d818.ko 2023-09-23T03:00:33.358 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1025.ko 2023-09-23T03:00:33.358 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1026.ko 2023-09-23T03:00:33.358 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1029.ko 2023-09-23T03:00:33.358 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1031.ko 2023-09-23T03:00:33.358 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm9240.ko 2023-09-23T03:00:33.359 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ads7828.ko 2023-09-23T03:00:33.359 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7410.ko 2023-09-23T03:00:33.359 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7411.ko 2023-09-23T03:00:33.359 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7462.ko 2023-09-23T03:00:33.359 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7470.ko 2023-09-23T03:00:33.359 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7475.ko 2023-09-23T03:00:33.359 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7x10.ko 2023-09-23T03:00:33.359 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/amc6821.ko 2023-09-23T03:00:33.359 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/applesmc.ko 2023-09-23T03:00:33.359 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asb100.ko 2023-09-23T03:00:33.359 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asc7621.ko 2023-09-23T03:00:33.359 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asus_atk0110.ko 2023-09-23T03:00:33.360 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/atxp1.ko 2023-09-23T03:00:33.360 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/coretemp.ko 2023-09-23T03:00:33.360 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-09-23T03:00:33.360 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dme1737.ko 2023-09-23T03:00:33.360 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds1621.ko 2023-09-23T03:00:33.360 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds620.ko 2023-09-23T03:00:33.360 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc1403.ko 2023-09-23T03:00:33.360 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc6w201.ko 2023-09-23T03:00:33.360 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71805f.ko 2023-09-23T03:00:33.360 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71882fg.ko 2023-09-23T03:00:33.361 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f75375s.ko 2023-09-23T03:00:33.361 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fam15h_power.ko 2023-09-23T03:00:33.361 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fschmd.ko 2023-09-23T03:00:33.361 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g760a.ko 2023-09-23T03:00:33.361 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g762.ko 2023-09-23T03:00:33.361 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl518sm.ko 2023-09-23T03:00:33.361 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl520sm.ko 2023-09-23T03:00:33.361 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/hwmon-vid.ko 2023-09-23T03:00:33.361 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5500_temp.ko 2023-09-23T03:00:33.361 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5k_amb.ko 2023-09-23T03:00:33.362 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmaem.ko 2023-09-23T03:00:33.362 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmpex.ko 2023-09-23T03:00:33.362 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina209.ko 2023-09-23T03:00:33.362 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina2xx.ko 2023-09-23T03:00:33.362 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/it87.ko 2023-09-23T03:00:33.362 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k10temp.ko 2023-09-23T03:00:33.362 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k8temp.ko 2023-09-23T03:00:33.362 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lineage-pem.ko 2023-09-23T03:00:33.362 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm63.ko 2023-09-23T03:00:33.362 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm73.ko 2023-09-23T03:00:33.362 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm75.ko 2023-09-23T03:00:33.362 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm77.ko 2023-09-23T03:00:33.363 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm78.ko 2023-09-23T03:00:33.363 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm80.ko 2023-09-23T03:00:33.363 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm83.ko 2023-09-23T03:00:33.363 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm85.ko 2023-09-23T03:00:33.363 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm87.ko 2023-09-23T03:00:33.363 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm90.ko 2023-09-23T03:00:33.363 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm92.ko 2023-09-23T03:00:33.363 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm93.ko 2023-09-23T03:00:33.363 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95234.ko 2023-09-23T03:00:33.363 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95241.ko 2023-09-23T03:00:33.363 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95245.ko 2023-09-23T03:00:33.363 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc2945.ko 2023-09-23T03:00:33.364 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4151.ko 2023-09-23T03:00:33.364 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4215.ko 2023-09-23T03:00:33.364 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4222.ko 2023-09-23T03:00:33.364 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4245.ko 2023-09-23T03:00:33.364 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4260.ko 2023-09-23T03:00:33.364 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4261.ko 2023-09-23T03:00:33.364 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max16065.ko 2023-09-23T03:00:33.364 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1619.ko 2023-09-23T03:00:33.364 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1668.ko 2023-09-23T03:00:33.364 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max197.ko 2023-09-23T03:00:33.364 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6639.ko 2023-09-23T03:00:33.364 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6650.ko 2023-09-23T03:00:33.365 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6697.ko 2023-09-23T03:00:33.365 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/mcp3021.ko 2023-09-23T03:00:33.365 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6683.ko 2023-09-23T03:00:33.365 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775-core.ko 2023-09-23T03:00:33.365 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775.ko 2023-09-23T03:00:33.365 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct7802.ko 2023-09-23T03:00:33.365 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87360.ko 2023-09-23T03:00:33.365 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87427.ko 2023-09-23T03:00:33.365 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pcf8591.ko 2023-09-23T03:00:33.365 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus 2023-09-23T03:00:33.365 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-09-23T03:00:33.365 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-09-23T03:00:33.366 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-09-23T03:00:33.366 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-09-23T03:00:33.366 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-09-23T03:00:33.366 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-09-23T03:00:33.366 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-09-23T03:00:33.366 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-09-23T03:00:33.366 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-09-23T03:00:33.366 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-09-23T03:00:33.366 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-09-23T03:00:33.366 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-09-23T03:00:33.367 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/powr1220.ko 2023-09-23T03:00:33.367 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5627.ko 2023-09-23T03:00:33.367 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5636.ko 2023-09-23T03:00:33.367 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch56xx-common.ko 2023-09-23T03:00:33.367 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht15.ko 2023-09-23T03:00:33.367 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht21.ko 2023-09-23T03:00:33.367 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/shtc1.ko 2023-09-23T03:00:33.367 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sis5595.ko 2023-09-23T03:00:33.367 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47b397.ko 2023-09-23T03:00:33.367 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m1.ko 2023-09-23T03:00:33.367 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m192.ko 2023-09-23T03:00:33.368 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/thmc50.ko 2023-09-23T03:00:33.368 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp102.ko 2023-09-23T03:00:33.368 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp103.ko 2023-09-23T03:00:33.368 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp401.ko 2023-09-23T03:00:33.368 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp421.ko 2023-09-23T03:00:33.368 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via-cputemp.ko 2023-09-23T03:00:33.368 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via686a.ko 2023-09-23T03:00:33.368 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt1211.ko 2023-09-23T03:00:33.368 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt8231.ko 2023-09-23T03:00:33.368 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627ehf.ko 2023-09-23T03:00:33.368 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627hf.ko 2023-09-23T03:00:33.369 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83781d.ko 2023-09-23T03:00:33.369 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83791d.ko 2023-09-23T03:00:33.369 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83792d.ko 2023-09-23T03:00:33.369 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83793.ko 2023-09-23T03:00:33.369 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83795.ko 2023-09-23T03:00:33.369 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l785ts.ko 2023-09-23T03:00:33.369 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l786ng.ko 2023-09-23T03:00:33.369 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c 2023-09-23T03:00:33.369 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos 2023-09-23T03:00:33.369 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-09-23T03:00:33.369 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-09-23T03:00:33.370 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses 2023-09-23T03:00:33.370 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-09-23T03:00:33.370 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-09-23T03:00:33.370 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-09-23T03:00:33.370 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-09-23T03:00:33.370 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-09-23T03:00:33.370 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-09-23T03:00:33.370 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-09-23T03:00:33.370 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-09-23T03:00:33.371 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-09-23T03:00:33.371 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-09-23T03:00:33.371 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-09-23T03:00:33.371 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-09-23T03:00:33.371 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-09-23T03:00:33.371 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-09-23T03:00:33.371 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-09-23T03:00:33.371 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-09-23T03:00:33.371 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-09-23T03:00:33.371 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-09-23T03:00:33.372 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-09-23T03:00:33.372 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-09-23T03:00:33.372 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-via.ko 2023-09-23T03:00:33.372 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-09-23T03:00:33.372 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-09-23T03:00:33.372 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-dev.ko 2023-09-23T03:00:33.372 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-mux.ko 2023-09-23T03:00:33.372 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-09-23T03:00:33.372 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-smbus.ko 2023-09-23T03:00:33.373 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-stub.ko 2023-09-23T03:00:33.373 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband 2023-09-23T03:00:33.373 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core 2023-09-23T03:00:33.373 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_cm.ko 2023-09-23T03:00:33.373 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_core.ko 2023-09-23T03:00:33.373 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_umad.ko 2023-09-23T03:00:33.373 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-09-23T03:00:33.373 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/iw_cm.ko 2023-09-23T03:00:33.373 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-09-23T03:00:33.374 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-09-23T03:00:33.374 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw 2023-09-23T03:00:33.374 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4 2023-09-23T03:00:33.374 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-09-23T03:00:33.374 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4 2023-09-23T03:00:33.374 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-09-23T03:00:33.374 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5 2023-09-23T03:00:33.375 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-09-23T03:00:33.375 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca 2023-09-23T03:00:33.375 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-09-23T03:00:33.375 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp 2023-09-23T03:00:33.375 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib 2023-09-23T03:00:33.375 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-09-23T03:00:33.375 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser 2023-09-23T03:00:33.375 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-09-23T03:00:33.375 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert 2023-09-23T03:00:33.375 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-09-23T03:00:33.376 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp 2023-09-23T03:00:33.376 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-09-23T03:00:33.376 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt 2023-09-23T03:00:33.376 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-09-23T03:00:33.376 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input 2023-09-23T03:00:33.376 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport 2023-09-23T03:00:33.376 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-09-23T03:00:33.376 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/fm801-gp.ko 2023-09-23T03:00:33.376 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/gameport.ko 2023-09-23T03:00:33.377 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/lightning.ko 2023-09-23T03:00:33.377 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/ns558.ko 2023-09-23T03:00:33.377 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/joydev.ko 2023-09-23T03:00:33.377 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard 2023-09-23T03:00:33.377 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-09-23T03:00:33.377 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse 2023-09-23T03:00:33.377 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/appletouch.ko 2023-09-23T03:00:33.377 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/bcm5974.ko 2023-09-23T03:00:33.377 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/cyapatp.ko 2023-09-23T03:00:33.378 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/elan_i2c.ko 2023-09-23T03:00:33.378 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/sermouse.ko 2023-09-23T03:00:33.378 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-09-23T03:00:33.378 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-09-23T03:00:33.378 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-09-23T03:00:33.378 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4 2023-09-23T03:00:33.378 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4/rmi_core.ko 2023-09-23T03:00:33.378 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio 2023-09-23T03:00:33.378 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/altera_ps2.ko 2023-09-23T03:00:33.378 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/arc_ps2.ko 2023-09-23T03:00:33.378 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-09-23T03:00:33.379 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/serio_raw.ko 2023-09-23T03:00:33.379 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/sparse-keymap.ko 2023-09-23T03:00:33.379 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu 2023-09-23T03:00:33.379 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd 2023-09-23T03:00:33.379 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-09-23T03:00:33.379 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds 2023-09-23T03:00:33.379 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-blinkm.ko 2023-09-23T03:00:33.379 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm3530.ko 2023-09-23T03:00:33.379 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lp3944.ko 2023-09-23T03:00:33.379 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-ss4200.ko 2023-09-23T03:00:33.379 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger 2023-09-23T03:00:33.380 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-09-23T03:00:33.380 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-09-23T03:00:33.380 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-09-23T03:00:33.380 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-09-23T03:00:33.380 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-09-23T03:00:33.380 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-09-23T03:00:33.380 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-09-23T03:00:33.380 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-09-23T03:00:33.380 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md 2023-09-23T03:00:33.380 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache 2023-09-23T03:00:33.380 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache/bcache.ko 2023-09-23T03:00:33.380 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-bio-prison.ko 2023-09-23T03:00:33.381 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache-smq.ko 2023-09-23T03:00:33.381 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache.ko 2023-09-23T03:00:33.381 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-crypt.ko 2023-09-23T03:00:33.381 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-delay.ko 2023-09-23T03:00:33.381 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-flakey.ko 2023-09-23T03:00:33.381 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-log-userspace.ko 2023-09-23T03:00:33.381 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-multipath.ko 2023-09-23T03:00:33.381 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-queue-length.ko 2023-09-23T03:00:33.381 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-raid.ko 2023-09-23T03:00:33.381 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-round-robin.ko 2023-09-23T03:00:33.381 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-service-time.ko 2023-09-23T03:00:33.382 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-switch.ko 2023-09-23T03:00:33.382 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-thin-pool.ko 2023-09-23T03:00:33.382 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-verity.ko 2023-09-23T03:00:33.382 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/faulty.ko 2023-09-23T03:00:33.382 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/linear.ko 2023-09-23T03:00:33.382 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/multipath.ko 2023-09-23T03:00:33.382 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data 2023-09-23T03:00:33.382 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-09-23T03:00:33.382 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid0.ko 2023-09-23T03:00:33.382 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid1.ko 2023-09-23T03:00:33.382 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid10.ko 2023-09-23T03:00:33.382 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid456.ko 2023-09-23T03:00:33.383 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message 2023-09-23T03:00:33.383 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion 2023-09-23T03:00:33.383 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptbase.ko 2023-09-23T03:00:33.383 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptctl.ko 2023-09-23T03:00:33.383 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptfc.ko 2023-09-23T03:00:33.383 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptlan.ko 2023-09-23T03:00:33.383 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptsas.ko 2023-09-23T03:00:33.383 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptscsih.ko 2023-09-23T03:00:33.383 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptspi.ko 2023-09-23T03:00:33.383 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd 2023-09-23T03:00:33.383 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_ich.ko 2023-09-23T03:00:33.384 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_sch.ko 2023-09-23T03:00:33.384 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/mfd-core.ko 2023-09-23T03:00:33.384 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/sm501.ko 2023-09-23T03:00:33.384 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/viperboard.ko 2023-09-23T03:00:33.384 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/vx855.ko 2023-09-23T03:00:33.384 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/wl1273-core.ko 2023-09-23T03:00:33.384 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc 2023-09-23T03:00:33.384 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl 2023-09-23T03:00:33.384 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-09-23T03:00:33.384 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds9802als.ko 2023-09-23T03:00:33.385 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds990x.ko 2023-09-23T03:00:33.385 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/bh1770glc.ko 2023-09-23T03:00:33.385 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710 2023-09-23T03:00:33.385 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710/cb710.ko 2023-09-23T03:00:33.385 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom 2023-09-23T03:00:33.385 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/at24.ko 2023-09-23T03:00:33.385 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom.ko 2023-09-23T03:00:33.385 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-09-23T03:00:33.385 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/max6875.ko 2023-09-23T03:00:33.385 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/enclosure.ko 2023-09-23T03:00:33.386 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/hpilo.ko 2023-09-23T03:00:33.386 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29003.ko 2023-09-23T03:00:33.386 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29020.ko 2023-09-23T03:00:33.386 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d 2023-09-23T03:00:33.386 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-09-23T03:00:33.386 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-09-23T03:00:33.386 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei 2023-09-23T03:00:33.386 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-me.ko 2023-09-23T03:00:33.386 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-txe.ko 2023-09-23T03:00:33.387 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei.ko 2023-09-23T03:00:33.387 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru 2023-09-23T03:00:33.387 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru/gru.ko 2023-09-23T03:00:33.387 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp 2023-09-23T03:00:33.387 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xp.ko 2023-09-23T03:00:33.387 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpc.ko 2023-09-23T03:00:33.387 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-09-23T03:00:33.387 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_7xx1.ko 2023-09-23T03:00:33.387 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_core.ko 2023-09-23T03:00:33.387 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tsl2550.ko 2023-09-23T03:00:33.388 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_balloon.ko 2023-09-23T03:00:33.388 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci 2023-09-23T03:00:33.388 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-09-23T03:00:33.388 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net 2023-09-23T03:00:33.388 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding 2023-09-23T03:00:33.388 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding/bonding.ko 2023-09-23T03:00:33.388 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa 2023-09-23T03:00:33.388 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53 2023-09-23T03:00:33.388 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-09-23T03:00:33.388 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-09-23T03:00:33.389 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6060.ko 2023-09-23T03:00:33.389 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx 2023-09-23T03:00:33.389 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-09-23T03:00:33.389 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dummy.ko 2023-09-23T03:00:33.389 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/eql.ko 2023-09-23T03:00:33.389 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet 2023-09-23T03:00:33.389 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com 2023-09-23T03:00:33.389 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-09-23T03:00:33.389 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-09-23T03:00:33.389 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-09-23T03:00:33.389 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-09-23T03:00:33.390 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390 2023-09-23T03:00:33.390 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/8390.ko 2023-09-23T03:00:33.390 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-09-23T03:00:33.390 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-09-23T03:00:33.390 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-09-23T03:00:33.390 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec 2023-09-23T03:00:33.390 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-09-23T03:00:33.390 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere 2023-09-23T03:00:33.390 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-09-23T03:00:33.390 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon 2023-09-23T03:00:33.390 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-09-23T03:00:33.390 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera 2023-09-23T03:00:33.391 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-09-23T03:00:33.391 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd 2023-09-23T03:00:33.391 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-09-23T03:00:33.391 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-09-23T03:00:33.391 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-09-23T03:00:33.391 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros 2023-09-23T03:00:33.391 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx 2023-09-23T03:00:33.391 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-09-23T03:00:33.391 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c 2023-09-23T03:00:33.391 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-09-23T03:00:33.391 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e 2023-09-23T03:00:33.392 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-09-23T03:00:33.392 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx 2023-09-23T03:00:33.392 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-09-23T03:00:33.392 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-09-23T03:00:33.392 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom 2023-09-23T03:00:33.392 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-09-23T03:00:33.392 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-09-23T03:00:33.392 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-09-23T03:00:33.392 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-09-23T03:00:33.392 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-09-23T03:00:33.392 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet 2023-09-23T03:00:33.393 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-09-23T03:00:33.393 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-09-23T03:00:33.393 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade 2023-09-23T03:00:33.393 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna 2023-09-23T03:00:33.393 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-09-23T03:00:33.393 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio 2023-09-23T03:00:33.393 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb 2023-09-23T03:00:33.393 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-09-23T03:00:33.393 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-09-23T03:00:33.393 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-09-23T03:00:33.393 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-09-23T03:00:33.393 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-09-23T03:00:33.394 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-09-23T03:00:33.394 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-09-23T03:00:33.394 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-09-23T03:00:33.394 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-09-23T03:00:33.394 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco 2023-09-23T03:00:33.394 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic 2023-09-23T03:00:33.394 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-09-23T03:00:33.394 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec 2023-09-23T03:00:33.394 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip 2023-09-23T03:00:33.394 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-09-23T03:00:33.395 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-09-23T03:00:33.395 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-09-23T03:00:33.395 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-09-23T03:00:33.395 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-09-23T03:00:33.395 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-09-23T03:00:33.395 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink 2023-09-23T03:00:33.395 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-09-23T03:00:33.395 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-09-23T03:00:33.395 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dnet.ko 2023-09-23T03:00:33.395 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex 2023-09-23T03:00:33.395 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet 2023-09-23T03:00:33.396 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-09-23T03:00:33.396 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ethoc.ko 2023-09-23T03:00:33.396 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/fealnx.ko 2023-09-23T03:00:33.396 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel 2023-09-23T03:00:33.396 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e100.ko 2023-09-23T03:00:33.396 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000 2023-09-23T03:00:33.396 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-09-23T03:00:33.396 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e 2023-09-23T03:00:33.396 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-09-23T03:00:33.396 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k 2023-09-23T03:00:33.396 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-09-23T03:00:33.397 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e 2023-09-23T03:00:33.397 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-09-23T03:00:33.397 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf 2023-09-23T03:00:33.397 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-09-23T03:00:33.397 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb 2023-09-23T03:00:33.397 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-09-23T03:00:33.397 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf 2023-09-23T03:00:33.397 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-09-23T03:00:33.397 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe 2023-09-23T03:00:33.398 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-09-23T03:00:33.398 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf 2023-09-23T03:00:33.398 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-09-23T03:00:33.398 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/jme.ko 2023-09-23T03:00:33.398 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell 2023-09-23T03:00:33.398 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-09-23T03:00:33.398 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-09-23T03:00:33.398 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-09-23T03:00:33.398 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox 2023-09-23T03:00:33.398 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4 2023-09-23T03:00:33.399 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-09-23T03:00:33.399 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-09-23T03:00:33.399 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5 2023-09-23T03:00:33.399 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-09-23T03:00:33.399 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-09-23T03:00:33.399 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel 2023-09-23T03:00:33.399 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-09-23T03:00:33.399 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom 2023-09-23T03:00:33.399 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge 2023-09-23T03:00:33.399 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-09-23T03:00:33.399 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi 2023-09-23T03:00:33.400 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-09-23T03:00:33.400 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-09-23T03:00:33.400 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion 2023-09-23T03:00:33.400 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-09-23T03:00:33.400 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia 2023-09-23T03:00:33.400 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-09-23T03:00:33.400 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines 2023-09-23T03:00:33.400 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-09-23T03:00:33.400 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-09-23T03:00:33.400 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic 2023-09-23T03:00:33.400 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen 2023-09-23T03:00:33.401 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-09-23T03:00:33.401 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-09-23T03:00:33.401 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-09-23T03:00:33.401 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-09-23T03:00:33.401 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc 2023-09-23T03:00:33.401 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-09-23T03:00:33.401 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek 2023-09-23T03:00:33.401 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-09-23T03:00:33.401 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-09-23T03:00:33.401 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-09-23T03:00:33.401 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-09-23T03:00:33.402 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker 2023-09-23T03:00:33.402 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-09-23T03:00:33.402 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc 2023-09-23T03:00:33.402 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-09-23T03:00:33.402 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan 2023-09-23T03:00:33.402 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-09-23T03:00:33.402 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis 2023-09-23T03:00:33.402 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-09-23T03:00:33.402 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-09-23T03:00:33.402 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc 2023-09-23T03:00:33.402 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-09-23T03:00:33.403 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-09-23T03:00:33.403 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-09-23T03:00:33.403 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-09-23T03:00:33.403 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro 2023-09-23T03:00:33.403 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac 2023-09-23T03:00:33.403 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-09-23T03:00:33.403 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-09-23T03:00:33.403 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun 2023-09-23T03:00:33.403 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-09-23T03:00:33.403 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/niu.ko 2023-09-23T03:00:33.403 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-09-23T03:00:33.404 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-09-23T03:00:33.404 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti 2023-09-23T03:00:33.404 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-09-23T03:00:33.404 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti 2023-09-23T03:00:33.404 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-09-23T03:00:33.404 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via 2023-09-23T03:00:33.404 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-09-23T03:00:33.404 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-09-23T03:00:33.404 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet 2023-09-23T03:00:33.404 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-09-23T03:00:33.405 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-09-23T03:00:33.405 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom 2023-09-23T03:00:33.405 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-09-23T03:00:33.405 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/geneve.ko 2023-09-23T03:00:33.405 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv 2023-09-23T03:00:33.405 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-09-23T03:00:33.405 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154 2023-09-23T03:00:33.405 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154/fakelb.ko 2023-09-23T03:00:33.405 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ifb.ko 2023-09-23T03:00:33.405 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan 2023-09-23T03:00:33.405 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-09-23T03:00:33.406 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvlan.ko 2023-09-23T03:00:33.406 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvtap.ko 2023-09-23T03:00:33.406 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio 2023-09-23T03:00:33.406 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio.ko 2023-09-23T03:00:33.406 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-09-23T03:00:33.406 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-09-23T03:00:33.406 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-09-23T03:00:33.406 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mii.ko 2023-09-23T03:00:33.406 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/net_failover.ko 2023-09-23T03:00:33.407 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/netconsole.ko 2023-09-23T03:00:33.407 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/nlmon.ko 2023-09-23T03:00:33.407 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs 2023-09-23T03:00:33.407 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-09-23T03:00:33.407 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-09-23T03:00:33.407 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy 2023-09-23T03:00:33.407 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/amd.ko 2023-09-23T03:00:33.407 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-09-23T03:00:33.407 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-09-23T03:00:33.407 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm7xxx.ko 2023-09-23T03:00:33.408 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm87xx.ko 2023-09-23T03:00:33.408 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/broadcom.ko 2023-09-23T03:00:33.408 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/cicada.ko 2023-09-23T03:00:33.408 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/davicom.ko 2023-09-23T03:00:33.408 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/dp83640.ko 2023-09-23T03:00:33.408 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/et1011c.ko 2023-09-23T03:00:33.408 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/icplus.ko 2023-09-23T03:00:33.408 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/lxt.ko 2023-09-23T03:00:33.408 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/marvell.ko 2023-09-23T03:00:33.408 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/micrel.ko 2023-09-23T03:00:33.408 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/national.ko 2023-09-23T03:00:33.409 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/phylink.ko 2023-09-23T03:00:33.409 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/qsemi.ko 2023-09-23T03:00:33.409 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/realtek.ko 2023-09-23T03:00:33.409 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/smsc.ko 2023-09-23T03:00:33.409 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/ste10Xp.ko 2023-09-23T03:00:33.409 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/vitesse.ko 2023-09-23T03:00:33.409 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp 2023-09-23T03:00:33.409 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/bsd_comp.ko 2023-09-23T03:00:33.409 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_async.ko 2023-09-23T03:00:33.409 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-09-23T03:00:33.409 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_generic.ko 2023-09-23T03:00:33.409 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-09-23T03:00:33.410 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-09-23T03:00:33.410 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppoe.ko 2023-09-23T03:00:33.410 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppox.ko 2023-09-23T03:00:33.410 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pptp.ko 2023-09-23T03:00:33.410 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip 2023-09-23T03:00:33.410 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slhc.ko 2023-09-23T03:00:33.410 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slip.ko 2023-09-23T03:00:33.410 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/sungem_phy.ko 2023-09-23T03:00:33.410 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tap.ko 2023-09-23T03:00:33.410 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team 2023-09-23T03:00:33.410 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team.ko 2023-09-23T03:00:33.410 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-09-23T03:00:33.411 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-09-23T03:00:33.411 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-09-23T03:00:33.411 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_random.ko 2023-09-23T03:00:33.411 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-09-23T03:00:33.411 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tun.ko 2023-09-23T03:00:33.411 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/veth.ko 2023-09-23T03:00:33.411 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/virtio_net.ko 2023-09-23T03:00:33.411 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3 2023-09-23T03:00:33.411 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-09-23T03:00:33.411 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan 2023-09-23T03:00:33.411 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan/vxlan.ko 2023-09-23T03:00:33.412 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback 2023-09-23T03:00:33.412 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-09-23T03:00:33.412 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netfront.ko 2023-09-23T03:00:33.412 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb 2023-09-23T03:00:33.412 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb/ntb.ko 2023-09-23T03:00:33.412 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme 2023-09-23T03:00:33.412 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host 2023-09-23T03:00:33.412 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme-core.ko 2023-09-23T03:00:33.412 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme.ko 2023-09-23T03:00:33.412 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport 2023-09-23T03:00:33.413 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport.ko 2023-09-23T03:00:33.413 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_cs.ko 2023-09-23T03:00:33.413 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_pc.ko 2023-09-23T03:00:33.413 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_serial.ko 2023-09-23T03:00:33.413 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci 2023-09-23T03:00:33.413 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug 2023-09-23T03:00:33.413 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-09-23T03:00:33.413 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie 2023-09-23T03:00:33.413 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie/aer_inject.ko 2023-09-23T03:00:33.413 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/xen-pcifront.ko 2023-09-23T03:00:33.413 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia 2023-09-23T03:00:33.414 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/i82092.ko 2023-09-23T03:00:33.414 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pd6729.ko 2023-09-23T03:00:33.414 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/yenta_socket.ko 2023-09-23T03:00:33.414 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl 2023-09-23T03:00:33.414 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel 2023-09-23T03:00:33.414 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-09-23T03:00:33.414 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform 2023-09-23T03:00:33.414 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86 2023-09-23T03:00:33.414 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acer-wmi.ko 2023-09-23T03:00:33.414 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acerhdf.ko 2023-09-23T03:00:33.414 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/apple-gmux.ko 2023-09-23T03:00:33.415 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-laptop.ko 2023-09-23T03:00:33.415 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-09-23T03:00:33.415 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-wmi.ko 2023-09-23T03:00:33.415 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-09-23T03:00:33.415 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-09-23T03:00:33.415 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-09-23T03:00:33.415 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-09-23T03:00:33.415 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-09-23T03:00:33.415 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/hdaps.ko 2023-09-23T03:00:33.415 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel 2023-09-23T03:00:33.415 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-09-23T03:00:33.415 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel_ips.ko 2023-09-23T03:00:33.416 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/msi-wmi.ko 2023-09-23T03:00:33.416 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-09-23T03:00:33.416 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-09-23T03:00:33.416 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-09-23T03:00:33.416 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-q10.ko 2023-09-23T03:00:33.416 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-09-23T03:00:33.416 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-09-23T03:00:33.416 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-09-23T03:00:33.416 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-09-23T03:00:33.416 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi.ko 2023-09-23T03:00:33.416 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap 2023-09-23T03:00:33.416 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_common.ko 2023-09-23T03:00:33.417 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-09-23T03:00:33.417 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps 2023-09-23T03:00:33.417 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients 2023-09-23T03:00:33.417 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-gpio.ko 2023-09-23T03:00:33.417 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-09-23T03:00:33.417 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps_parport.ko 2023-09-23T03:00:33.417 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/pps_core.ko 2023-09-23T03:00:33.417 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp 2023-09-23T03:00:33.417 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp.ko 2023-09-23T03:00:33.417 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp_kvm.ko 2023-09-23T03:00:33.417 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc 2023-09-23T03:00:33.418 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq32k.ko 2023-09-23T03:00:33.418 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq4802.ko 2023-09-23T03:00:33.418 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1286.ko 2023-09-23T03:00:33.418 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1307.ko 2023-09-23T03:00:33.418 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1374.ko 2023-09-23T03:00:33.418 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1511.ko 2023-09-23T03:00:33.418 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1553.ko 2023-09-23T03:00:33.418 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1672.ko 2023-09-23T03:00:33.418 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1742.ko 2023-09-23T03:00:33.418 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds2404.ko 2023-09-23T03:00:33.418 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds3232.ko 2023-09-23T03:00:33.419 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-em3027.ko 2023-09-23T03:00:33.419 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-fm3130.ko 2023-09-23T03:00:33.419 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl12022.ko 2023-09-23T03:00:33.419 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl1208.ko 2023-09-23T03:00:33.419 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m41t80.ko 2023-09-23T03:00:33.419 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t35.ko 2023-09-23T03:00:33.419 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t59.ko 2023-09-23T03:00:33.419 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max6900.ko 2023-09-23T03:00:33.419 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-msm6242.ko 2023-09-23T03:00:33.419 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-09-23T03:00:33.419 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-09-23T03:00:33.419 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-09-23T03:00:33.420 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-09-23T03:00:33.420 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-09-23T03:00:33.420 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-09-23T03:00:33.420 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-09-23T03:00:33.420 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-09-23T03:00:33.420 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8025.ko 2023-09-23T03:00:33.420 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8581.ko 2023-09-23T03:00:33.420 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-09-23T03:00:33.420 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-x1205.ko 2023-09-23T03:00:33.420 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi 2023-09-23T03:00:33.420 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-9xxx.ko 2023-09-23T03:00:33.421 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-sas.ko 2023-09-23T03:00:33.421 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-xxxx.ko 2023-09-23T03:00:33.421 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/BusLogic.ko 2023-09-23T03:00:33.421 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/a100u2w.ko 2023-09-23T03:00:33.421 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid 2023-09-23T03:00:33.421 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-09-23T03:00:33.421 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/advansys.ko 2023-09-23T03:00:33.421 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx 2023-09-23T03:00:33.421 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-09-23T03:00:33.421 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-09-23T03:00:33.421 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/am53c974.ko 2023-09-23T03:00:33.422 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr 2023-09-23T03:00:33.422 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-09-23T03:00:33.422 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/atp870u.ko 2023-09-23T03:00:33.422 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi 2023-09-23T03:00:33.422 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-09-23T03:00:33.422 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa 2023-09-23T03:00:33.422 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa/bfa.ko 2023-09-23T03:00:33.422 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc 2023-09-23T03:00:33.422 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-09-23T03:00:33.422 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i 2023-09-23T03:00:33.422 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-09-23T03:00:33.423 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ch.ko 2023-09-23T03:00:33.423 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor 2023-09-23T03:00:33.423 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-09-23T03:00:33.423 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi 2023-09-23T03:00:33.423 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i 2023-09-23T03:00:33.423 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-09-23T03:00:33.423 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i 2023-09-23T03:00:33.423 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-09-23T03:00:33.423 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-09-23T03:00:33.423 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dc395x.ko 2023-09-23T03:00:33.423 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler 2023-09-23T03:00:33.424 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-09-23T03:00:33.424 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-09-23T03:00:33.424 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-09-23T03:00:33.424 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-09-23T03:00:33.424 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dmx3191d.ko 2023-09-23T03:00:33.424 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r 2023-09-23T03:00:33.424 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-09-23T03:00:33.424 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esp_scsi.ko 2023-09-23T03:00:33.424 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe 2023-09-23T03:00:33.424 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-09-23T03:00:33.424 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-09-23T03:00:33.424 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic 2023-09-23T03:00:33.425 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic/fnic.ko 2023-09-23T03:00:33.425 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hpsa.ko 2023-09-23T03:00:33.425 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hptiop.ko 2023-09-23T03:00:33.425 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hv_storvsc.ko 2023-09-23T03:00:33.425 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/initio.ko 2023-09-23T03:00:33.425 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ipr.ko 2023-09-23T03:00:33.425 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ips.ko 2023-09-23T03:00:33.425 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci 2023-09-23T03:00:33.425 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci/isci.ko 2023-09-23T03:00:33.425 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-09-23T03:00:33.425 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_tcp.ko 2023-09-23T03:00:33.426 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc 2023-09-23T03:00:33.426 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc/libfc.ko 2023-09-23T03:00:33.426 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi.ko 2023-09-23T03:00:33.426 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-09-23T03:00:33.426 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas 2023-09-23T03:00:33.426 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas/libsas.ko 2023-09-23T03:00:33.426 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc 2023-09-23T03:00:33.426 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-09-23T03:00:33.426 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid 2023-09-23T03:00:33.426 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid.ko 2023-09-23T03:00:33.426 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-09-23T03:00:33.426 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-09-23T03:00:33.427 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-09-23T03:00:33.427 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas 2023-09-23T03:00:33.427 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-09-23T03:00:33.427 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas 2023-09-23T03:00:33.427 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-09-23T03:00:33.427 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvumi.ko 2023-09-23T03:00:33.427 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001 2023-09-23T03:00:33.427 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-09-23T03:00:33.427 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pmcraid.ko 2023-09-23T03:00:33.427 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla1280.ko 2023-09-23T03:00:33.428 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx 2023-09-23T03:00:33.428 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-09-23T03:00:33.428 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-09-23T03:00:33.428 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx 2023-09-23T03:00:33.428 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-09-23T03:00:33.428 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/raid_class.ko 2023-09-23T03:00:33.428 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_debug.ko 2023-09-23T03:00:33.428 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-09-23T03:00:33.428 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-09-23T03:00:33.428 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-09-23T03:00:33.429 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-09-23T03:00:33.429 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-09-23T03:00:33.429 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ses.ko 2023-09-23T03:00:33.429 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/st.ko 2023-09-23T03:00:33.429 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/stex.ko 2023-09-23T03:00:33.429 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2 2023-09-23T03:00:33.429 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-09-23T03:00:33.429 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/virtio_scsi.ko 2023-09-23T03:00:33.429 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-09-23T03:00:33.429 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/wd719x.ko 2023-09-23T03:00:33.429 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb 2023-09-23T03:00:33.429 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb/ssb.ko 2023-09-23T03:00:33.430 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target 2023-09-23T03:00:33.430 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi 2023-09-23T03:00:33.430 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-09-23T03:00:33.430 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback 2023-09-23T03:00:33.430 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback/tcm_loop.ko 2023-09-23T03:00:33.430 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_file.ko 2023-09-23T03:00:33.430 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_iblock.ko 2023-09-23T03:00:33.430 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_mod.ko 2023-09-23T03:00:33.430 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_pscsi.ko 2023-09-23T03:00:33.430 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_user.ko 2023-09-23T03:00:33.430 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc 2023-09-23T03:00:33.431 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-09-23T03:00:33.431 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal 2023-09-23T03:00:33.431 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel 2023-09-23T03:00:33.431 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal 2023-09-23T03:00:33.431 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-09-23T03:00:33.431 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-09-23T03:00:33.431 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-09-23T03:00:33.431 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-09-23T03:00:33.431 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-09-23T03:00:33.431 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-09-23T03:00:33.431 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-09-23T03:00:33.431 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-09-23T03:00:33.432 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-09-23T03:00:33.432 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-09-23T03:00:33.432 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-09-23T03:00:33.432 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-09-23T03:00:33.432 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-09-23T03:00:33.432 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-09-23T03:00:33.432 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-09-23T03:00:33.432 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty 2023-09-23T03:00:33.432 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless 2023-09-23T03:00:33.432 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-09-23T03:00:33.432 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_gsm.ko 2023-09-23T03:00:33.433 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_hdlc.ko 2023-09-23T03:00:33.433 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/nozomi.ko 2023-09-23T03:00:33.433 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial 2023-09-23T03:00:33.433 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250 2023-09-23T03:00:33.433 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-09-23T03:00:33.433 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/arc_uart.ko 2023-09-23T03:00:33.433 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm 2023-09-23T03:00:33.433 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-09-23T03:00:33.433 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/synclink_gt.ko 2023-09-23T03:00:33.433 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs 2023-09-23T03:00:33.433 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core 2023-09-23T03:00:33.433 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-09-23T03:00:33.434 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host 2023-09-23T03:00:33.434 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-09-23T03:00:33.434 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio 2023-09-23T03:00:33.434 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio.ko 2023-09-23T03:00:33.434 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_aec.ko 2023-09-23T03:00:33.434 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_cif.ko 2023-09-23T03:00:33.434 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_pci_generic.ko 2023-09-23T03:00:33.434 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_sercos3.ko 2023-09-23T03:00:33.435 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb 2023-09-23T03:00:33.435 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class 2023-09-23T03:00:33.435 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-acm.ko 2023-09-23T03:00:33.435 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-wdm.ko 2023-09-23T03:00:33.435 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usblp.ko 2023-09-23T03:00:33.435 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usbtmc.ko 2023-09-23T03:00:33.435 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host 2023-09-23T03:00:33.435 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/sl811-hcd.ko 2023-09-23T03:00:33.435 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image 2023-09-23T03:00:33.435 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/mdc800.ko 2023-09-23T03:00:33.435 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/microtek.ko 2023-09-23T03:00:33.436 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc 2023-09-23T03:00:33.436 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/adutux.ko 2023-09-23T03:00:33.436 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/appledisplay.ko 2023-09-23T03:00:33.436 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi26.ko 2023-09-23T03:00:33.436 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi62.ko 2023-09-23T03:00:33.436 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ezusb.ko 2023-09-23T03:00:33.436 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/idmouse.ko 2023-09-23T03:00:33.436 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/iowarrior.ko 2023-09-23T03:00:33.436 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/isight_firmware.ko 2023-09-23T03:00:33.436 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ldusb.ko 2023-09-23T03:00:33.436 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/legousbtower.ko 2023-09-23T03:00:33.436 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga 2023-09-23T03:00:33.437 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-09-23T03:00:33.437 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/trancevibrator.ko 2023-09-23T03:00:33.437 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usb3503.ko 2023-09-23T03:00:33.437 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usblcd.ko 2023-09-23T03:00:33.437 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usbsevseg.ko 2023-09-23T03:00:33.437 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/uss720.ko 2023-09-23T03:00:33.437 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/yurex.ko 2023-09-23T03:00:33.437 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy 2023-09-23T03:00:33.437 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-generic.ko 2023-09-23T03:00:33.437 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial 2023-09-23T03:00:33.437 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/aircable.ko 2023-09-23T03:00:33.437 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ark3116.ko 2023-09-23T03:00:33.438 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/belkin_sa.ko 2023-09-23T03:00:33.438 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ch341.ko 2023-09-23T03:00:33.438 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cp210x.ko 2023-09-23T03:00:33.438 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cyberjack.ko 2023-09-23T03:00:33.438 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cypress_m8.ko 2023-09-23T03:00:33.438 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-09-23T03:00:33.438 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/empeg.ko 2023-09-23T03:00:33.438 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-09-23T03:00:33.438 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/garmin_gps.ko 2023-09-23T03:00:33.438 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_edgeport.ko 2023-09-23T03:00:33.438 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_ti.ko 2023-09-23T03:00:33.438 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipaq.ko 2023-09-23T03:00:33.439 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipw.ko 2023-09-23T03:00:33.439 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ir-usb.ko 2023-09-23T03:00:33.439 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-09-23T03:00:33.439 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan.ko 2023-09-23T03:00:33.439 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-09-23T03:00:33.439 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-09-23T03:00:33.439 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kobil_sct.ko 2023-09-23T03:00:33.439 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mct_u232.ko 2023-09-23T03:00:33.439 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7720.ko 2023-09-23T03:00:33.439 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7840.ko 2023-09-23T03:00:33.439 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/navman.ko 2023-09-23T03:00:33.440 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/omninet.ko 2023-09-23T03:00:33.440 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/opticon.ko 2023-09-23T03:00:33.440 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/option.ko 2023-09-23T03:00:33.440 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/oti6858.ko 2023-09-23T03:00:33.440 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/pl2303.ko 2023-09-23T03:00:33.440 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcaux.ko 2023-09-23T03:00:33.440 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcserial.ko 2023-09-23T03:00:33.440 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/quatech2.ko 2023-09-23T03:00:33.440 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/safe_serial.ko 2023-09-23T03:00:33.440 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/sierra.ko 2023-09-23T03:00:33.440 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/spcp8x5.ko 2023-09-23T03:00:33.440 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ssu100.ko 2023-09-23T03:00:33.441 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/symbolserial.ko 2023-09-23T03:00:33.441 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-09-23T03:00:33.441 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-09-23T03:00:33.441 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_debug.ko 2023-09-23T03:00:33.441 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_wwan.ko 2023-09-23T03:00:33.441 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/visor.ko 2023-09-23T03:00:33.441 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/whiteheat.ko 2023-09-23T03:00:33.441 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/xsens_mt.ko 2023-09-23T03:00:33.441 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage 2023-09-23T03:00:33.441 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/uas.ko 2023-09-23T03:00:33.441 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-alauda.ko 2023-09-23T03:00:33.441 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-cypress.ko 2023-09-23T03:00:33.442 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-datafab.ko 2023-09-23T03:00:33.442 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-09-23T03:00:33.442 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-freecom.ko 2023-09-23T03:00:33.442 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-isd200.ko 2023-09-23T03:00:33.442 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-09-23T03:00:33.442 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-karma.ko 2023-09-23T03:00:33.443 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-09-23T03:00:33.443 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-realtek.ko 2023-09-23T03:00:33.443 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-09-23T03:00:33.443 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-09-23T03:00:33.443 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-usbat.ko 2023-09-23T03:00:33.443 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/usb-storage.ko 2023-09-23T03:00:33.443 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip 2023-09-23T03:00:33.443 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-core.ko 2023-09-23T03:00:33.444 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-host.ko 2023-09-23T03:00:33.444 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-09-23T03:00:33.444 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio 2023-09-23T03:00:33.444 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci 2023-09-23T03:00:33.444 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-09-23T03:00:33.444 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-09-23T03:00:33.444 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio.ko 2023-09-23T03:00:33.444 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-09-23T03:00:33.444 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost 2023-09-23T03:00:33.444 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost.ko 2023-09-23T03:00:33.445 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_iotlb.ko 2023-09-23T03:00:33.445 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_net.ko 2023-09-23T03:00:33.445 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_scsi.ko 2023-09-23T03:00:33.445 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video 2023-09-23T03:00:33.445 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight 2023-09-23T03:00:33.445 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/apple_bl.ko 2023-09-23T03:00:33.445 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lcd.ko 2023-09-23T03:00:33.445 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/platform_lcd.ko 2023-09-23T03:00:33.445 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio 2023-09-23T03:00:33.445 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio.ko 2023-09-23T03:00:33.446 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_balloon.ko 2023-09-23T03:00:33.446 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_mmio.ko 2023-09-23T03:00:33.446 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci.ko 2023-09-23T03:00:33.446 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-09-23T03:00:33.446 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-09-23T03:00:33.446 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_ring.ko 2023-09-23T03:00:33.446 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1 2023-09-23T03:00:33.446 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters 2023-09-23T03:00:33.446 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2482.ko 2023-09-23T03:00:33.446 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2490.ko 2023-09-23T03:00:33.446 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves 2023-09-23T03:00:33.446 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-09-23T03:00:33.447 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-09-23T03:00:33.447 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-09-23T03:00:33.447 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-09-23T03:00:33.447 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-09-23T03:00:33.447 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-09-23T03:00:33.447 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-09-23T03:00:33.447 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-09-23T03:00:33.447 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-09-23T03:00:33.447 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_smem.ko 2023-09-23T03:00:33.447 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_therm.ko 2023-09-23T03:00:33.447 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/wire.ko 2023-09-23T03:00:33.448 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog 2023-09-23T03:00:33.448 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-09-23T03:00:33.448 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-09-23T03:00:33.448 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-09-23T03:00:33.448 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/hpwdt.ko 2023-09-23T03:00:33.448 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/i6300esb.ko 2023-09-23T03:00:33.448 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-09-23T03:00:33.448 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-09-23T03:00:33.448 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ib700wdt.ko 2023-09-23T03:00:33.448 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ibmasr.ko 2023-09-23T03:00:33.448 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-09-23T03:00:33.449 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-09-23T03:00:33.449 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it87_wdt.ko 2023-09-23T03:00:33.449 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/machzwd.ko 2023-09-23T03:00:33.449 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/nv_tco.ko 2023-09-23T03:00:33.449 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_pci.ko 2023-09-23T03:00:33.449 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_usb.ko 2023-09-23T03:00:33.449 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-09-23T03:00:33.449 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-09-23T03:00:33.449 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/softdog.ko 2023-09-23T03:00:33.449 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sp5100_tco.ko 2023-09-23T03:00:33.449 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/via_wdt.ko 2023-09-23T03:00:33.449 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-09-23T03:00:33.450 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-09-23T03:00:33.450 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-09-23T03:00:33.450 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wdt_pci.ko 2023-09-23T03:00:33.450 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/xen_wdt.ko 2023-09-23T03:00:33.450 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen 2023-09-23T03:00:33.450 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-acpi-processor.ko 2023-09-23T03:00:33.450 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-evtchn.ko 2023-09-23T03:00:33.450 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntalloc.ko 2023-09-23T03:00:33.450 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntdev.ko 2023-09-23T03:00:33.450 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback 2023-09-23T03:00:33.450 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-09-23T03:00:33.451 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-privcmd.ko 2023-09-23T03:00:33.451 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs 2023-09-23T03:00:33.451 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs/xenfs.ko 2023-09-23T03:00:33.451 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs 2023-09-23T03:00:33.451 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p 2023-09-23T03:00:33.451 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p/9p.ko 2023-09-23T03:00:33.451 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs 2023-09-23T03:00:33.451 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs/affs.ko 2023-09-23T03:00:33.451 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs 2023-09-23T03:00:33.451 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs/befs.ko 2023-09-23T03:00:33.451 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/binfmt_misc.ko 2023-09-23T03:00:33.452 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs 2023-09-23T03:00:33.452 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs/btrfs.ko 2023-09-23T03:00:33.452 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles 2023-09-23T03:00:33.452 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles/cachefiles.ko 2023-09-23T03:00:33.452 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph 2023-09-23T03:00:33.452 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph/ceph.ko 2023-09-23T03:00:33.452 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda 2023-09-23T03:00:33.452 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda/coda.ko 2023-09-23T03:00:33.452 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs 2023-09-23T03:00:33.452 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs/cramfs.ko 2023-09-23T03:00:33.452 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm 2023-09-23T03:00:33.452 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm/dlm.ko 2023-09-23T03:00:33.453 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs 2023-09-23T03:00:33.453 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs/ecryptfs.ko 2023-09-23T03:00:33.453 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs 2023-09-23T03:00:33.453 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs/f2fs.ko 2023-09-23T03:00:33.453 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat 2023-09-23T03:00:33.453 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/fat.ko 2023-09-23T03:00:33.453 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/msdos.ko 2023-09-23T03:00:33.453 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/vfat.ko 2023-09-23T03:00:33.453 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache 2023-09-23T03:00:33.453 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache/fscache.ko 2023-09-23T03:00:33.453 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse 2023-09-23T03:00:33.454 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/cuse.ko 2023-09-23T03:00:33.454 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/fuse.ko 2023-09-23T03:00:33.454 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2 2023-09-23T03:00:33.454 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2/gfs2.ko 2023-09-23T03:00:33.454 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs 2023-09-23T03:00:33.454 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs/hfs.ko 2023-09-23T03:00:33.454 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus 2023-09-23T03:00:33.454 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus/hfsplus.ko 2023-09-23T03:00:33.454 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs 2023-09-23T03:00:33.454 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs/isofs.ko 2023-09-23T03:00:33.455 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs 2023-09-23T03:00:33.455 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs/jfs.ko 2023-09-23T03:00:33.455 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd 2023-09-23T03:00:33.455 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd/lockd.ko 2023-09-23T03:00:33.455 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix 2023-09-23T03:00:33.455 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix/minix.ko 2023-09-23T03:00:33.455 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs 2023-09-23T03:00:33.455 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs/netfs.ko 2023-09-23T03:00:33.455 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs 2023-09-23T03:00:33.455 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout 2023-09-23T03:00:33.455 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-09-23T03:00:33.455 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout 2023-09-23T03:00:33.456 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-09-23T03:00:33.456 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout 2023-09-23T03:00:33.456 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-09-23T03:00:33.456 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfs.ko 2023-09-23T03:00:33.456 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv3.ko 2023-09-23T03:00:33.456 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv4.ko 2023-09-23T03:00:33.456 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common 2023-09-23T03:00:33.456 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/grace.ko 2023-09-23T03:00:33.456 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/nfs_acl.ko 2023-09-23T03:00:33.456 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd 2023-09-23T03:00:33.456 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd/nfsd.ko 2023-09-23T03:00:33.456 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2 2023-09-23T03:00:33.457 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2/nilfs2.ko 2023-09-23T03:00:33.457 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls 2023-09-23T03:00:33.457 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-celtic.ko 2023-09-23T03:00:33.457 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-centeuro.ko 2023-09-23T03:00:33.457 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-croatian.ko 2023-09-23T03:00:33.457 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-cyrillic.ko 2023-09-23T03:00:33.457 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-gaelic.ko 2023-09-23T03:00:33.457 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-greek.ko 2023-09-23T03:00:33.457 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-iceland.ko 2023-09-23T03:00:33.457 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-inuit.ko 2023-09-23T03:00:33.457 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-roman.ko 2023-09-23T03:00:33.457 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-romanian.ko 2023-09-23T03:00:33.458 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-turkish.ko 2023-09-23T03:00:33.458 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1250.ko 2023-09-23T03:00:33.458 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1251.ko 2023-09-23T03:00:33.458 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1255.ko 2023-09-23T03:00:33.458 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp737.ko 2023-09-23T03:00:33.458 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp775.ko 2023-09-23T03:00:33.458 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp850.ko 2023-09-23T03:00:33.458 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp852.ko 2023-09-23T03:00:33.458 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp855.ko 2023-09-23T03:00:33.458 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp857.ko 2023-09-23T03:00:33.458 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp860.ko 2023-09-23T03:00:33.458 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp861.ko 2023-09-23T03:00:33.459 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp862.ko 2023-09-23T03:00:33.459 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp863.ko 2023-09-23T03:00:33.459 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp864.ko 2023-09-23T03:00:33.459 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp865.ko 2023-09-23T03:00:33.459 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp866.ko 2023-09-23T03:00:33.459 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp869.ko 2023-09-23T03:00:33.459 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp874.ko 2023-09-23T03:00:33.459 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp932.ko 2023-09-23T03:00:33.459 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp936.ko 2023-09-23T03:00:33.459 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp949.ko 2023-09-23T03:00:33.459 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp950.ko 2023-09-23T03:00:33.460 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_euc-jp.ko 2023-09-23T03:00:33.460 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-1.ko 2023-09-23T03:00:33.460 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-13.ko 2023-09-23T03:00:33.460 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-14.ko 2023-09-23T03:00:33.460 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-15.ko 2023-09-23T03:00:33.460 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-2.ko 2023-09-23T03:00:33.460 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-3.ko 2023-09-23T03:00:33.460 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-4.ko 2023-09-23T03:00:33.460 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-5.ko 2023-09-23T03:00:33.460 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-6.ko 2023-09-23T03:00:33.460 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-7.ko 2023-09-23T03:00:33.460 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-9.ko 2023-09-23T03:00:33.461 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-r.ko 2023-09-23T03:00:33.461 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-ru.ko 2023-09-23T03:00:33.461 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-u.ko 2023-09-23T03:00:33.461 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_utf8.ko 2023-09-23T03:00:33.461 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2 2023-09-23T03:00:33.461 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster 2023-09-23T03:00:33.461 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-09-23T03:00:33.461 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm 2023-09-23T03:00:33.461 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-09-23T03:00:33.461 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs 2023-09-23T03:00:33.461 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-09-23T03:00:33.461 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2.ko 2023-09-23T03:00:33.462 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-09-23T03:00:33.462 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-09-23T03:00:33.462 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-09-23T03:00:33.462 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs 2023-09-23T03:00:33.462 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs/overlay.ko 2023-09-23T03:00:33.462 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore 2023-09-23T03:00:33.462 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore/ramoops.ko 2023-09-23T03:00:33.462 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs 2023-09-23T03:00:33.462 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs/reiserfs.ko 2023-09-23T03:00:33.462 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs 2023-09-23T03:00:33.462 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs/romfs.ko 2023-09-23T03:00:33.462 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb 2023-09-23T03:00:33.463 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client 2023-09-23T03:00:33.463 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client/cifs.ko 2023-09-23T03:00:33.463 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common 2023-09-23T03:00:33.463 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_arc4.ko 2023-09-23T03:00:33.463 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_md4.ko 2023-09-23T03:00:33.463 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs 2023-09-23T03:00:33.463 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs/squashfs.ko 2023-09-23T03:00:33.463 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv 2023-09-23T03:00:33.463 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv/sysv.ko 2023-09-23T03:00:33.463 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf 2023-09-23T03:00:33.463 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf/udf.ko 2023-09-23T03:00:33.464 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs 2023-09-23T03:00:33.464 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs/ufs.ko 2023-09-23T03:00:33.464 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs 2023-09-23T03:00:33.464 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs/xfs.ko 2023-09-23T03:00:33.464 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel 2023-09-23T03:00:33.464 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu 2023-09-23T03:00:33.464 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu/rcutorture.ko 2023-09-23T03:00:33.464 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/torture.ko 2023-09-23T03:00:33.464 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace 2023-09-23T03:00:33.464 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-09-23T03:00:33.464 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib 2023-09-23T03:00:33.464 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/asn1_encoder.ko 2023-09-23T03:00:33.465 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc-itu-t.ko 2023-09-23T03:00:33.465 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc8.ko 2023-09-23T03:00:33.465 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto 2023-09-23T03:00:33.465 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libarc4.ko 2023-09-23T03:00:33.465 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libdes.ko 2023-09-23T03:00:33.465 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/libcrc32c.ko 2023-09-23T03:00:33.465 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lru_cache.ko 2023-09-23T03:00:33.465 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4 2023-09-23T03:00:33.465 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4_compress.ko 2023-09-23T03:00:33.465 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4hc_compress.ko 2023-09-23T03:00:33.465 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math 2023-09-23T03:00:33.466 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math/cordic.ko 2023-09-23T03:00:33.466 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6 2023-09-23T03:00:33.466 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6/raid6_pq.ko 2023-09-23T03:00:33.466 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon 2023-09-23T03:00:33.466 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon/reed_solomon.ko 2023-09-23T03:00:33.466 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_bm.ko 2023-09-23T03:00:33.466 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_fsm.ko 2023-09-23T03:00:33.466 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_kmp.ko 2023-09-23T03:00:33.466 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd 2023-09-23T03:00:33.466 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd/zstd_compress.ko 2023-09-23T03:00:33.466 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm 2023-09-23T03:00:33.467 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm/hwpoison-inject.ko 2023-09-23T03:00:33.467 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net 2023-09-23T03:00:33.467 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan 2023-09-23T03:00:33.467 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/6lowpan.ko 2023-09-23T03:00:33.467 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_dest.ko 2023-09-23T03:00:33.467 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_fragment.ko 2023-09-23T03:00:33.467 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_hop.ko 2023-09-23T03:00:33.467 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_ipv6.ko 2023-09-23T03:00:33.467 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_mobility.ko 2023-09-23T03:00:33.467 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_routing.ko 2023-09-23T03:00:33.467 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_udp.ko 2023-09-23T03:00:33.467 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802 2023-09-23T03:00:33.468 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/garp.ko 2023-09-23T03:00:33.468 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/mrp.ko 2023-09-23T03:00:33.468 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/p8022.ko 2023-09-23T03:00:33.468 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/psnap.ko 2023-09-23T03:00:33.468 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/stp.ko 2023-09-23T03:00:33.468 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q 2023-09-23T03:00:33.468 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q/8021q.ko 2023-09-23T03:00:33.468 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p 2023-09-23T03:00:33.468 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet.ko 2023-09-23T03:00:33.468 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_fd.ko 2023-09-23T03:00:33.468 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_rdma.ko 2023-09-23T03:00:33.468 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_virtio.ko 2023-09-23T03:00:33.469 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv 2023-09-23T03:00:33.469 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv/batman-adv.ko 2023-09-23T03:00:33.469 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge 2023-09-23T03:00:33.469 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/br_netfilter.ko 2023-09-23T03:00:33.469 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/bridge.ko 2023-09-23T03:00:33.469 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter 2023-09-23T03:00:33.469 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-09-23T03:00:33.469 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_among.ko 2023-09-23T03:00:33.469 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-09-23T03:00:33.469 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-09-23T03:00:33.469 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-09-23T03:00:33.469 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-09-23T03:00:33.470 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-09-23T03:00:33.470 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-09-23T03:00:33.470 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_log.ko 2023-09-23T03:00:33.470 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-09-23T03:00:33.470 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-09-23T03:00:33.470 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-09-23T03:00:33.470 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-09-23T03:00:33.470 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-09-23T03:00:33.470 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-09-23T03:00:33.470 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-09-23T03:00:33.470 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-09-23T03:00:33.470 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-09-23T03:00:33.471 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-09-23T03:00:33.471 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-09-23T03:00:33.471 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtables.ko 2023-09-23T03:00:33.471 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-09-23T03:00:33.471 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-09-23T03:00:33.471 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-09-23T03:00:33.471 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph 2023-09-23T03:00:33.471 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph/libceph.ko 2023-09-23T03:00:33.471 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core 2023-09-23T03:00:33.471 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/failover.ko 2023-09-23T03:00:33.471 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/pktgen.ko 2023-09-23T03:00:33.471 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp 2023-09-23T03:00:33.472 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp.ko 2023-09-23T03:00:33.472 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_diag.ko 2023-09-23T03:00:33.472 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv4.ko 2023-09-23T03:00:33.472 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv6.ko 2023-09-23T03:00:33.472 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver 2023-09-23T03:00:33.472 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver/dns_resolver.ko 2023-09-23T03:00:33.472 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa 2023-09-23T03:00:33.472 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/dsa_core.ko 2023-09-23T03:00:33.472 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_brcm.ko 2023-09-23T03:00:33.472 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_dsa.ko 2023-09-23T03:00:33.472 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_none.ko 2023-09-23T03:00:33.472 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_trailer.ko 2023-09-23T03:00:33.472 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154 2023-09-23T03:00:33.473 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan 2023-09-23T03:00:33.473 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-09-23T03:00:33.473 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154.ko 2023-09-23T03:00:33.473 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154_socket.ko 2023-09-23T03:00:33.473 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4 2023-09-23T03:00:33.473 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ah4.ko 2023-09-23T03:00:33.473 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/esp4.ko 2023-09-23T03:00:33.473 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/fou.ko 2023-09-23T03:00:33.473 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/gre.ko 2023-09-23T03:00:33.473 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/inet_diag.ko 2023-09-23T03:00:33.473 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_gre.ko 2023-09-23T03:00:33.474 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_tunnel.ko 2023-09-23T03:00:33.474 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_vti.ko 2023-09-23T03:00:33.474 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipcomp.ko 2023-09-23T03:00:33.474 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipip.ko 2023-09-23T03:00:33.474 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter 2023-09-23T03:00:33.474 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-09-23T03:00:33.474 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-09-23T03:00:33.474 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-09-23T03:00:33.474 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-09-23T03:00:33.474 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-09-23T03:00:33.474 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-09-23T03:00:33.475 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-09-23T03:00:33.475 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-09-23T03:00:33.475 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-09-23T03:00:33.475 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-09-23T03:00:33.475 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-09-23T03:00:33.475 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-09-23T03:00:33.475 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-09-23T03:00:33.475 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-09-23T03:00:33.475 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-09-23T03:00:33.475 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-09-23T03:00:33.475 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-09-23T03:00:33.475 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-09-23T03:00:33.476 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-09-23T03:00:33.476 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-09-23T03:00:33.476 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-09-23T03:00:33.476 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_bic.ko 2023-09-23T03:00:33.476 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_dctcp.ko 2023-09-23T03:00:33.476 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_diag.ko 2023-09-23T03:00:33.476 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_highspeed.ko 2023-09-23T03:00:33.476 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_htcp.ko 2023-09-23T03:00:33.476 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_hybla.ko 2023-09-23T03:00:33.476 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_illinois.ko 2023-09-23T03:00:33.476 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_lp.ko 2023-09-23T03:00:33.476 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_scalable.ko 2023-09-23T03:00:33.477 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_vegas.ko 2023-09-23T03:00:33.477 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_veno.ko 2023-09-23T03:00:33.477 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_westwood.ko 2023-09-23T03:00:33.477 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_yeah.ko 2023-09-23T03:00:33.477 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tunnel4.ko 2023-09-23T03:00:33.477 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_diag.ko 2023-09-23T03:00:33.477 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_tunnel.ko 2023-09-23T03:00:33.477 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-09-23T03:00:33.477 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6 2023-09-23T03:00:33.477 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ah6.ko 2023-09-23T03:00:33.477 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/esp6.ko 2023-09-23T03:00:33.477 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/fou6.ko 2023-09-23T03:00:33.477 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_tunnel.ko 2023-09-23T03:00:33.478 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-09-23T03:00:33.478 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_vti.ko 2023-09-23T03:00:33.478 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ipcomp6.ko 2023-09-23T03:00:33.478 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter 2023-09-23T03:00:33.478 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-09-23T03:00:33.478 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-09-23T03:00:33.478 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-09-23T03:00:33.478 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-09-23T03:00:33.478 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-09-23T03:00:33.478 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-09-23T03:00:33.478 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-09-23T03:00:33.479 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-09-23T03:00:33.479 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-09-23T03:00:33.479 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-09-23T03:00:33.479 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-09-23T03:00:33.479 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-09-23T03:00:33.479 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-09-23T03:00:33.479 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-09-23T03:00:33.479 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-09-23T03:00:33.479 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-09-23T03:00:33.479 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-09-23T03:00:33.479 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-09-23T03:00:33.479 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-09-23T03:00:33.480 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-09-23T03:00:33.480 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-09-23T03:00:33.480 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-09-23T03:00:33.480 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-09-23T03:00:33.480 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-09-23T03:00:33.480 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/sit.ko 2023-09-23T03:00:33.480 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/tunnel6.ko 2023-09-23T03:00:33.480 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-09-23T03:00:33.480 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key 2023-09-23T03:00:33.480 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key/af_key.ko 2023-09-23T03:00:33.480 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp 2023-09-23T03:00:33.480 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_core.ko 2023-09-23T03:00:33.481 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_debugfs.ko 2023-09-23T03:00:33.481 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_eth.ko 2023-09-23T03:00:33.481 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip.ko 2023-09-23T03:00:33.481 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip6.ko 2023-09-23T03:00:33.481 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_netlink.ko 2023-09-23T03:00:33.481 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ppp.ko 2023-09-23T03:00:33.481 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc 2023-09-23T03:00:33.481 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc/llc.ko 2023-09-23T03:00:33.481 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154 2023-09-23T03:00:33.481 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154/mac802154.ko 2023-09-23T03:00:33.481 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls 2023-09-23T03:00:33.481 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls/mpls_gso.ko 2023-09-23T03:00:33.482 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter 2023-09-23T03:00:33.482 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset 2023-09-23T03:00:33.482 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set.ko 2023-09-23T03:00:33.482 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-09-23T03:00:33.482 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-09-23T03:00:33.482 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-09-23T03:00:33.482 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-09-23T03:00:33.482 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-09-23T03:00:33.482 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-09-23T03:00:33.482 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-09-23T03:00:33.482 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-09-23T03:00:33.482 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-09-23T03:00:33.483 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-09-23T03:00:33.483 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-09-23T03:00:33.483 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-09-23T03:00:33.483 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-09-23T03:00:33.483 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-09-23T03:00:33.483 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-09-23T03:00:33.483 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs 2023-09-23T03:00:33.483 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-09-23T03:00:33.483 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-09-23T03:00:33.483 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-09-23T03:00:33.483 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-09-23T03:00:33.484 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-09-23T03:00:33.484 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-09-23T03:00:33.484 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-09-23T03:00:33.484 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-09-23T03:00:33.484 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-09-23T03:00:33.484 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-09-23T03:00:33.484 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-09-23T03:00:33.484 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-09-23T03:00:33.484 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-09-23T03:00:33.484 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-09-23T03:00:33.484 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conncount.ko 2023-09-23T03:00:33.484 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack.ko 2023-09-23T03:00:33.485 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-09-23T03:00:33.485 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-09-23T03:00:33.485 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-09-23T03:00:33.485 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-09-23T03:00:33.485 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-09-23T03:00:33.485 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-09-23T03:00:33.485 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-09-23T03:00:33.485 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-09-23T03:00:33.485 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-09-23T03:00:33.485 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-09-23T03:00:33.485 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-09-23T03:00:33.485 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-09-23T03:00:33.486 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_dup_netdev.ko 2023-09-23T03:00:33.486 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table.ko 2023-09-23T03:00:33.486 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-09-23T03:00:33.486 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_log_syslog.ko 2023-09-23T03:00:33.486 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat.ko 2023-09-23T03:00:33.486 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_amanda.ko 2023-09-23T03:00:33.486 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_ftp.ko 2023-09-23T03:00:33.486 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_irc.ko 2023-09-23T03:00:33.486 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_sip.ko 2023-09-23T03:00:33.486 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_tftp.ko 2023-09-23T03:00:33.486 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_synproxy_core.ko 2023-09-23T03:00:33.486 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_tables.ko 2023-09-23T03:00:33.487 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink.ko 2023-09-23T03:00:33.487 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_acct.ko 2023-09-23T03:00:33.487 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_log.ko 2023-09-23T03:00:33.487 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_osf.ko 2023-09-23T03:00:33.487 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_queue.ko 2023-09-23T03:00:33.487 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_chain_nat.ko 2023-09-23T03:00:33.487 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_compat.ko 2023-09-23T03:00:33.487 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_ct.ko 2023-09-23T03:00:33.487 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_dup_netdev.ko 2023-09-23T03:00:33.487 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fib.ko 2023-09-23T03:00:33.487 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-09-23T03:00:33.487 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_hash.ko 2023-09-23T03:00:33.488 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_limit.ko 2023-09-23T03:00:33.488 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_log.ko 2023-09-23T03:00:33.488 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_masq.ko 2023-09-23T03:00:33.488 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_nat.ko 2023-09-23T03:00:33.488 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_numgen.ko 2023-09-23T03:00:33.488 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_queue.ko 2023-09-23T03:00:33.488 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_quota.ko 2023-09-23T03:00:33.488 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_redir.ko 2023-09-23T03:00:33.488 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject.ko 2023-09-23T03:00:33.488 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject_inet.ko 2023-09-23T03:00:33.488 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_socket.ko 2023-09-23T03:00:33.488 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_tunnel.ko 2023-09-23T03:00:33.489 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_AUDIT.ko 2023-09-23T03:00:33.489 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-09-23T03:00:33.489 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-09-23T03:00:33.489 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-09-23T03:00:33.489 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CT.ko 2023-09-23T03:00:33.489 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_DSCP.ko 2023-09-23T03:00:33.489 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HL.ko 2023-09-23T03:00:33.489 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HMARK.ko 2023-09-23T03:00:33.489 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-09-23T03:00:33.489 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LED.ko 2023-09-23T03:00:33.489 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LOG.ko 2023-09-23T03:00:33.490 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-09-23T03:00:33.490 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NETMAP.ko 2023-09-23T03:00:33.490 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFLOG.ko 2023-09-23T03:00:33.490 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-09-23T03:00:33.490 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_RATEEST.ko 2023-09-23T03:00:33.490 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_REDIRECT.ko 2023-09-23T03:00:33.490 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_SECMARK.ko 2023-09-23T03:00:33.490 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPMSS.ko 2023-09-23T03:00:33.490 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-09-23T03:00:33.490 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TEE.ko 2023-09-23T03:00:33.490 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TPROXY.ko 2023-09-23T03:00:33.490 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TRACE.ko 2023-09-23T03:00:33.491 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_addrtype.ko 2023-09-23T03:00:33.491 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_bpf.ko 2023-09-23T03:00:33.491 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cgroup.ko 2023-09-23T03:00:33.491 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cluster.ko 2023-09-23T03:00:33.491 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_comment.ko 2023-09-23T03:00:33.491 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connbytes.ko 2023-09-23T03:00:33.491 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlabel.ko 2023-09-23T03:00:33.491 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlimit.ko 2023-09-23T03:00:33.491 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connmark.ko 2023-09-23T03:00:33.491 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_conntrack.ko 2023-09-23T03:00:33.491 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cpu.ko 2023-09-23T03:00:33.491 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dccp.ko 2023-09-23T03:00:33.492 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_devgroup.ko 2023-09-23T03:00:33.492 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dscp.ko 2023-09-23T03:00:33.492 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ecn.ko 2023-09-23T03:00:33.492 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_esp.ko 2023-09-23T03:00:33.492 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hashlimit.ko 2023-09-23T03:00:33.492 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_helper.ko 2023-09-23T03:00:33.492 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hl.ko 2023-09-23T03:00:33.492 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipcomp.ko 2023-09-23T03:00:33.492 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_iprange.ko 2023-09-23T03:00:33.492 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipvs.ko 2023-09-23T03:00:33.492 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_l2tp.ko 2023-09-23T03:00:33.493 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_length.ko 2023-09-23T03:00:33.493 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_limit.ko 2023-09-23T03:00:33.493 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mac.ko 2023-09-23T03:00:33.493 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mark.ko 2023-09-23T03:00:33.493 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_multiport.ko 2023-09-23T03:00:33.493 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nat.ko 2023-09-23T03:00:33.493 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nfacct.ko 2023-09-23T03:00:33.493 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_osf.ko 2023-09-23T03:00:33.493 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_owner.ko 2023-09-23T03:00:33.493 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_physdev.ko 2023-09-23T03:00:33.493 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_pkttype.ko 2023-09-23T03:00:33.494 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_policy.ko 2023-09-23T03:00:33.494 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_quota.ko 2023-09-23T03:00:33.494 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_rateest.ko 2023-09-23T03:00:33.494 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_realm.ko 2023-09-23T03:00:33.494 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_recent.ko 2023-09-23T03:00:33.494 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_sctp.ko 2023-09-23T03:00:33.494 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_set.ko 2023-09-23T03:00:33.494 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_socket.ko 2023-09-23T03:00:33.494 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_state.ko 2023-09-23T03:00:33.494 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_statistic.ko 2023-09-23T03:00:33.494 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_string.ko 2023-09-23T03:00:33.495 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_tcpmss.ko 2023-09-23T03:00:33.495 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_time.ko 2023-09-23T03:00:33.495 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_u32.ko 2023-09-23T03:00:33.495 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink 2023-09-23T03:00:33.495 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink/netlink_diag.ko 2023-09-23T03:00:33.495 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh 2023-09-23T03:00:33.495 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh/nsh.ko 2023-09-23T03:00:33.495 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch 2023-09-23T03:00:33.495 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/openvswitch.ko 2023-09-23T03:00:33.495 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-geneve.ko 2023-09-23T03:00:33.495 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-gre.ko 2023-09-23T03:00:33.495 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-vxlan.ko 2023-09-23T03:00:33.496 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet 2023-09-23T03:00:33.496 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet/af_packet_diag.ko 2023-09-23T03:00:33.496 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds 2023-09-23T03:00:33.496 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds.ko 2023-09-23T03:00:33.496 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_rdma.ko 2023-09-23T03:00:33.496 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_tcp.ko 2023-09-23T03:00:33.496 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched 2023-09-23T03:00:33.496 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_csum.ko 2023-09-23T03:00:33.496 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_gact.ko 2023-09-23T03:00:33.496 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_ipt.ko 2023-09-23T03:00:33.496 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_mirred.ko 2023-09-23T03:00:33.496 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_nat.ko 2023-09-23T03:00:33.497 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_pedit.ko 2023-09-23T03:00:33.497 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_police.ko 2023-09-23T03:00:33.497 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_simple.ko 2023-09-23T03:00:33.497 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_skbedit.ko 2023-09-23T03:00:33.497 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_vlan.ko 2023-09-23T03:00:33.497 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_basic.ko 2023-09-23T03:00:33.497 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_bpf.ko 2023-09-23T03:00:33.497 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_flow.ko 2023-09-23T03:00:33.497 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_fw.ko 2023-09-23T03:00:33.497 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_route.ko 2023-09-23T03:00:33.497 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_u32.ko 2023-09-23T03:00:33.497 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_cmp.ko 2023-09-23T03:00:33.498 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_ipset.ko 2023-09-23T03:00:33.498 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_meta.ko 2023-09-23T03:00:33.498 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_nbyte.ko 2023-09-23T03:00:33.498 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_text.ko 2023-09-23T03:00:33.498 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_u32.ko 2023-09-23T03:00:33.498 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_choke.ko 2023-09-23T03:00:33.498 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_codel.ko 2023-09-23T03:00:33.498 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_drr.ko 2023-09-23T03:00:33.498 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_fq.ko 2023-09-23T03:00:33.498 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_gred.ko 2023-09-23T03:00:33.498 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hfsc.ko 2023-09-23T03:00:33.499 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hhf.ko 2023-09-23T03:00:33.499 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_htb.ko 2023-09-23T03:00:33.499 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_ingress.ko 2023-09-23T03:00:33.499 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio.ko 2023-09-23T03:00:33.499 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio_lib.ko 2023-09-23T03:00:33.499 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_multiq.ko 2023-09-23T03:00:33.499 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_netem.ko 2023-09-23T03:00:33.499 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_pie.ko 2023-09-23T03:00:33.499 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_plug.ko 2023-09-23T03:00:33.499 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_prio.ko 2023-09-23T03:00:33.499 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_qfq.ko 2023-09-23T03:00:33.499 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_red.ko 2023-09-23T03:00:33.500 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfb.ko 2023-09-23T03:00:33.500 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfq.ko 2023-09-23T03:00:33.500 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_tbf.ko 2023-09-23T03:00:33.500 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_teql.ko 2023-09-23T03:00:33.500 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp 2023-09-23T03:00:33.500 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp.ko 2023-09-23T03:00:33.500 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp_diag.ko 2023-09-23T03:00:33.500 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc 2023-09-23T03:00:33.500 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss 2023-09-23T03:00:33.500 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-09-23T03:00:33.500 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-09-23T03:00:33.500 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/sunrpc.ko 2023-09-23T03:00:33.501 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma 2023-09-23T03:00:33.501 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-09-23T03:00:33.501 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc 2023-09-23T03:00:33.501 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/diag.ko 2023-09-23T03:00:33.501 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/tipc.ko 2023-09-23T03:00:33.501 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix 2023-09-23T03:00:33.501 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix/unix_diag.ko 2023-09-23T03:00:33.501 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock 2023-09-23T03:00:33.501 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-09-23T03:00:33.501 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-09-23T03:00:33.501 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock.ko 2023-09-23T03:00:33.501 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_diag.ko 2023-09-23T03:00:33.501 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-09-23T03:00:33.502 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm 2023-09-23T03:00:33.502 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-09-23T03:00:33.502 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security 2023-09-23T03:00:33.502 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys 2023-09-23T03:00:33.502 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys 2023-09-23T03:00:33.502 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-09-23T03:00:33.502 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys 2023-09-23T03:00:33.502 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys/trusted.ko 2023-09-23T03:00:33.502 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt 2023-09-23T03:00:33.502 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib 2023-09-23T03:00:33.502 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib/irqbypass.ko 2023-09-23T03:00:33.503 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias 2023-09-23T03:00:33.503 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias.bin 2023-09-23T03:00:33.503 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin 2023-09-23T03:00:33.503 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.alias.bin 2023-09-23T03:00:33.503 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.bin 2023-09-23T03:00:33.503 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.modinfo 2023-09-23T03:00:33.503 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep 2023-09-23T03:00:33.503 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep.bin 2023-09-23T03:00:33.503 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.devname 2023-09-23T03:00:33.503 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.order 2023-09-23T03:00:33.503 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.softdep 2023-09-23T03:00:33.503 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols 2023-09-23T03:00:33.504 INFO:teuthology.orchestra.run.smithi190.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols.bin 2023-09-23T03:00:33.504 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc7-gfef64f93da2b 2023-09-23T03:00:33.504 DEBUG:teuthology.orchestra.run.smithi190:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-09-23T03:00:33.527 INFO:teuthology.orchestra.run.smithi190.stdout:postinstall scriptlet (using /bin/sh): 2023-09-23T03:00:33.528 INFO:teuthology.orchestra.run.smithi190.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b -a -r /boot/System.map-6.5.0-rc7-gfef64f93da2b ]; then 2023-09-23T03:00:33.528 INFO:teuthology.orchestra.run.smithi190.stdout:cp /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm 2023-09-23T03:00:33.528 INFO:teuthology.orchestra.run.smithi190.stdout:cp /boot/System.map-6.5.0-rc7-gfef64f93da2b /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-09-23T03:00:33.528 INFO:teuthology.orchestra.run.smithi190.stdout:rm -f /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-09-23T03:00:33.528 INFO:teuthology.orchestra.run.smithi190.stdout:/sbin/installkernel 6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-09-23T03:00:33.528 INFO:teuthology.orchestra.run.smithi190.stdout:rm -f /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-09-23T03:00:33.528 INFO:teuthology.orchestra.run.smithi190.stdout:fi 2023-09-23T03:00:33.528 INFO:teuthology.orchestra.run.smithi190.stdout:preuninstall scriptlet (using /bin/sh): 2023-09-23T03:00:33.528 INFO:teuthology.orchestra.run.smithi190.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-09-23T03:00:33.529 INFO:teuthology.orchestra.run.smithi190.stdout:new-kernel-pkg --remove 6.5.0-rc7-gfef64f93da2b --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc7-gfef64f93da2b.img 2023-09-23T03:00:33.529 INFO:teuthology.orchestra.run.smithi190.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-09-23T03:00:33.529 INFO:teuthology.orchestra.run.smithi190.stdout:kernel-install remove 6.5.0-rc7-gfef64f93da2b 2023-09-23T03:00:33.529 INFO:teuthology.orchestra.run.smithi190.stdout:fi 2023-09-23T03:00:33.529 INFO:teuthology.orchestra.run.smithi190.stdout:postuninstall scriptlet (using /bin/sh): 2023-09-23T03:00:33.529 INFO:teuthology.orchestra.run.smithi190.stdout:if [ -x /sbin/update-bootloader ]; then 2023-09-23T03:00:33.529 INFO:teuthology.orchestra.run.smithi190.stdout:/sbin/update-bootloader --remove 6.5.0-rc7-gfef64f93da2b 2023-09-23T03:00:33.529 INFO:teuthology.orchestra.run.smithi190.stdout:fi 2023-09-23T03:00:33.530 DEBUG:teuthology.orchestra.run.smithi190:> sudo rpm -qi grub2-tools 2023-09-23T03:00:33.606 INFO:teuthology.orchestra.run.smithi190.stdout:Name : grub2-tools 2023-09-23T03:00:33.606 INFO:teuthology.orchestra.run.smithi190.stdout:Epoch : 1 2023-09-23T03:00:33.606 INFO:teuthology.orchestra.run.smithi190.stdout:Version : 2.06 2023-09-23T03:00:33.607 INFO:teuthology.orchestra.run.smithi190.stdout:Release : 68.el9 2023-09-23T03:00:33.607 INFO:teuthology.orchestra.run.smithi190.stdout:Architecture: x86_64 2023-09-23T03:00:33.607 INFO:teuthology.orchestra.run.smithi190.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-09-23T03:00:33.607 INFO:teuthology.orchestra.run.smithi190.stdout:Group : Unspecified 2023-09-23T03:00:33.607 INFO:teuthology.orchestra.run.smithi190.stdout:Size : 8264254 2023-09-23T03:00:33.607 INFO:teuthology.orchestra.run.smithi190.stdout:License : GPLv3+ 2023-09-23T03:00:33.607 INFO:teuthology.orchestra.run.smithi190.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-09-23T03:00:33.607 INFO:teuthology.orchestra.run.smithi190.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-09-23T03:00:33.607 INFO:teuthology.orchestra.run.smithi190.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-09-23T03:00:33.608 INFO:teuthology.orchestra.run.smithi190.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-09-23T03:00:33.608 INFO:teuthology.orchestra.run.smithi190.stdout:Packager : builder@centos.org 2023-09-23T03:00:33.608 INFO:teuthology.orchestra.run.smithi190.stdout:Vendor : CentOS 2023-09-23T03:00:33.608 INFO:teuthology.orchestra.run.smithi190.stdout:URL : http://www.gnu.org/software/grub/ 2023-09-23T03:00:33.608 INFO:teuthology.orchestra.run.smithi190.stdout:Summary : Support tools for GRUB. 2023-09-23T03:00:33.608 INFO:teuthology.orchestra.run.smithi190.stdout:Description : 2023-09-23T03:00:33.608 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:00:33.608 INFO:teuthology.orchestra.run.smithi190.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-09-23T03:00:33.608 INFO:teuthology.orchestra.run.smithi190.stdout:customizable bootloader with modular architecture. It supports a rich 2023-09-23T03:00:33.608 INFO:teuthology.orchestra.run.smithi190.stdout:variety of kernel formats, file systems, computer architectures and 2023-09-23T03:00:33.609 INFO:teuthology.orchestra.run.smithi190.stdout:hardware devices. 2023-09-23T03:00:33.609 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:00:33.609 INFO:teuthology.orchestra.run.smithi190.stdout:This subpackage provides tools for support of all platforms. 2023-09-23T03:00:33.610 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-09-23T03:00:33.610 INFO:teuthology.task.kernel:Updating grub on smithi190 to boot 6.5.0-rc7-gfef64f93da2b 2023-09-23T03:00:33.610 DEBUG:teuthology.orchestra.run.smithi190:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-09-23T03:00:34.283 INFO:teuthology.orchestra.run.smithi190.stderr:Generating grub configuration file ... 2023-09-23T03:00:35.653 INFO:teuthology.orchestra.run.smithi190.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-09-23T03:00:35.688 INFO:teuthology.orchestra.run.smithi190.stderr:done 2023-09-23T03:00:35.690 DEBUG:teuthology.orchestra.run.smithi190:> mktemp 2023-09-23T03:00:35.705 INFO:teuthology.orchestra.run.smithi190.stdout:/tmp/tmp.AbAvCqsoft 2023-09-23T03:00:35.705 DEBUG:teuthology.orchestra.run.smithi190:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.AbAvCqsoft 2023-09-23T03:00:35.774 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0666 /tmp/tmp.AbAvCqsoft 2023-09-23T03:00:35.917 DEBUG:teuthology.orchestra.remote:smithi190:/tmp/tmp.AbAvCqsoft is 6KB 2023-09-23T03:00:35.966 DEBUG:teuthology.orchestra.run.smithi190:> rm -fr /tmp/tmp.AbAvCqsoft 2023-09-23T03:00:35.981 DEBUG:teuthology.orchestra.run.smithi190:> sudo /bin/ls /boot/loader/entries || true 2023-09-23T03:00:36.046 INFO:teuthology.orchestra.run.smithi190.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-09-23T03:00:36.046 INFO:teuthology.orchestra.run.smithi190.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-09-23T03:00:36.046 INFO:teuthology.orchestra.run.smithi190.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-09-23T03:00:36.046 INFO:teuthology.orchestra.run.smithi190.stdout:15949e3c88704300882482eddfd7baae-6.5.0-rc7-gfef64f93da2b.conf 2023-09-23T03:00:36.048 DEBUG:teuthology.orchestra.run.smithi190:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.5.0-rc7-gfef64f93da2b 2023-09-23T03:00:36.182 DEBUG:teuthology.orchestra.run.smithi190:> sudo shutdown -r now 2023-09-23T03:00:39.318 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-09-23T03:00:39.318 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2023-09-23T03:00:39.318 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T03:01:06.210 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-09-23T03:01:06.210 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2023-09-23T03:01:06.211 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T03:01:13.982 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.154 2023-09-23T03:01:22.990 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2023-09-23T03:01:22.991 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T03:01:41.374 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.154 2023-09-23T03:01:53.386 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2023-09-23T03:01:53.387 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T03:01:56.446 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.154 2023-09-23T03:02:06.222 DEBUG:teuthology.orchestra.remote:timed out 2023-09-23T03:02:11.450 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2023-09-23T03:02:11.451 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T03:02:11.976 DEBUG:teuthology.orchestra.run.smithi154:> true 2023-09-23T03:02:12.686 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi154.front.sepia.ceph.com' 2023-09-23T03:02:12.686 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.5.0-rc7-gfef64f93da2b"... 2023-09-23T03:02:12.686 DEBUG:teuthology.orchestra.run.smithi154:> uname -r 2023-09-23T03:02:12.728 INFO:teuthology.orchestra.run.smithi154.stdout:6.5.0-rc7-gfef64f93da2b 2023-09-23T03:02:12.728 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc7-gfef64f93da2b vs 6.5.0-rc7-gfef64f93da2b 2023-09-23T03:02:12.728 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-09-23T03:02:12.728 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-09-23T03:02:13.729 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-09-23T03:02:14.154 DEBUG:teuthology.orchestra.run.smithi154:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-09-23T03:02:14.287 INFO:teuthology.orchestra.run.smithi154.stdout:ttyS1 2023-09-23T03:02:14.347 DEBUG:teuthology.parallel:result is None 2023-09-23T03:02:15.223 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2023-09-23T03:02:15.224 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T03:02:16.624 DEBUG:teuthology.orchestra.run.smithi190:> true 2023-09-23T03:02:17.256 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com' 2023-09-23T03:02:17.256 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.5.0-rc7-gfef64f93da2b"... 2023-09-23T03:02:17.256 DEBUG:teuthology.orchestra.run.smithi190:> uname -r 2023-09-23T03:02:17.296 INFO:teuthology.orchestra.run.smithi190.stdout:6.5.0-rc7-gfef64f93da2b 2023-09-23T03:02:17.297 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc7-gfef64f93da2b vs 6.5.0-rc7-gfef64f93da2b 2023-09-23T03:02:17.297 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-09-23T03:02:17.297 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-09-23T03:02:18.298 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-09-23T03:02:18.298 DEBUG:teuthology.orchestra.run.smithi190:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-09-23T03:02:18.469 INFO:teuthology.orchestra.run.smithi190.stdout:ttyS1 2023-09-23T03:02:18.510 DEBUG:teuthology.parallel:result is None 2023-09-23T03:02:18.510 INFO:teuthology.run_tasks:Running task internal.base... 2023-09-23T03:02:18.522 INFO:teuthology.task.internal:Creating test directory... 2023-09-23T03:02:18.523 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-23T03:02:18.537 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-23T03:02:18.583 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-09-23T03:02:18.597 INFO:teuthology.run_tasks:Running task internal.archive... 2023-09-23T03:02:18.624 INFO:teuthology.task.internal:Creating archive directory... 2023-09-23T03:02:18.624 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-23T03:02:18.636 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-23T03:02:18.698 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-09-23T03:02:18.708 INFO:teuthology.task.internal:Enabling coredump saving... 2023-09-23T03:02:18.709 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-23T03:02:18.713 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-23T03:02:18.782 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-23T03:02:18.815 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-23T03:02:18.817 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-23T03:02:18.856 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-23T03:02:18.860 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-09-23T03:02:18.870 INFO:teuthology.task.internal:Configuring sudo... 2023-09-23T03:02:18.870 DEBUG:teuthology.orchestra.run.smithi154:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-23T03:02:18.875 DEBUG:teuthology.orchestra.run.smithi190:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-23T03:02:18.961 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-09-23T03:02:18.973 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-09-23T03:02:18.974 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-23T03:02:18.980 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-23T03:02:19.024 DEBUG:teuthology.orchestra.run.smithi154:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-23T03:02:19.148 DEBUG:teuthology.orchestra.run.smithi154:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-23T03:02:19.242 DEBUG:teuthology.orchestra.run.smithi154:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-23T03:02:19.318 DEBUG:teuthology.orchestra.run.smithi154:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-23T03:02:19.405 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:02:19.405 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-23T03:02:19.482 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-23T03:02:19.567 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-23T03:02:19.649 DEBUG:teuthology.orchestra.run.smithi190:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-23T03:02:19.719 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-23T03:02:19.815 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:02:19.815 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-23T03:02:19.885 DEBUG:teuthology.orchestra.run.smithi154:> sudo service rsyslog restart 2023-09-23T03:02:19.892 DEBUG:teuthology.orchestra.run.smithi190:> sudo service rsyslog restart 2023-09-23T03:02:20.019 INFO:teuthology.orchestra.run.smithi154.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-23T03:02:20.049 INFO:teuthology.orchestra.run.smithi190.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-23T03:02:20.319 INFO:teuthology.run_tasks:Running task internal.timer... 2023-09-23T03:02:20.328 INFO:teuthology.task.internal:Starting timer... 2023-09-23T03:02:20.328 INFO:teuthology.run_tasks:Running task pcp... 2023-09-23T03:02:20.344 INFO:teuthology.run_tasks:Running task selinux... 2023-09-23T03:02:20.360 DEBUG:teuthology.orchestra.run.smithi154:> sudo service auditd rotate 2023-09-23T03:02:20.468 INFO:teuthology.orchestra.run.smithi154.stdout:Rotating logs: 2023-09-23T03:02:20.472 DEBUG:teuthology.orchestra.run.smithi190:> sudo service auditd rotate 2023-09-23T03:02:20.595 INFO:teuthology.orchestra.run.smithi190.stdout:Rotating logs: 2023-09-23T03:02:20.599 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-09-23T03:02:20.599 DEBUG:teuthology.orchestra.run.smithi154:> /usr/sbin/getenforce 2023-09-23T03:02:20.701 INFO:teuthology.orchestra.run.smithi154.stdout:Permissive 2023-09-23T03:02:20.701 DEBUG:teuthology.orchestra.run.smithi190:> /usr/sbin/getenforce 2023-09-23T03:02:20.746 INFO:teuthology.orchestra.run.smithi190.stdout:Permissive 2023-09-23T03:02:20.747 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi154.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi190.front.sepia.ceph.com': 'permissive'} 2023-09-23T03:02:20.747 DEBUG:teuthology.orchestra.run.smithi154:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-23T03:02:20.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T03:02:20.868 DEBUG:teuthology.orchestra.run.smithi190:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-23T03:02:20.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T03:02:20.953 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-09-23T03:02:20.953 DEBUG:teuthology.orchestra.run.smithi154:> sudo /usr/sbin/setenforce permissive 2023-09-23T03:02:21.082 DEBUG:teuthology.orchestra.run.smithi190:> sudo /usr/sbin/setenforce permissive 2023-09-23T03:02:21.168 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-09-23T03:02:21.180 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-09-23T03:02:21.180 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-09-23T03:02:21.197 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-09-23T03:02:21.198 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi154.front.sepia.ceph.com,smithi190.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-09-23T03:12:08.161 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi154.front.sepia.ceph.com'), Remote(name='ubuntu@smithi190.front.sepia.ceph.com')] 2023-09-23T03:12:08.161 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2023-09-23T03:12:08.162 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T03:12:08.295 DEBUG:teuthology.orchestra.run.smithi154:> true 2023-09-23T03:12:08.454 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi154.front.sepia.ceph.com' 2023-09-23T03:12:08.455 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi190.front.sepia.ceph.com' 2023-09-23T03:12:08.455 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi190.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-23T03:12:08.592 DEBUG:teuthology.orchestra.run.smithi190:> true 2023-09-23T03:12:08.743 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi190.front.sepia.ceph.com' 2023-09-23T03:12:08.743 INFO:teuthology.run_tasks:Running task clock... 2023-09-23T03:12:08.757 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-09-23T03:12:08.757 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-23T03:12:08.757 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-23T03:12:08.766 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-23T03:12:08.766 DEBUG:teuthology.orchestra.run.smithi190:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-23T03:12:08.863 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-09-23T03:12:08.873 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-09-23T03:12:08.918 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-09-23T03:12:08.928 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-09-23T03:12:09.017 INFO:teuthology.orchestra.run.smithi154.stderr:sudo: ntpd: command not found 2023-09-23T03:12:09.026 INFO:teuthology.orchestra.run.smithi190.stderr:sudo: ntpd: command not found 2023-09-23T03:12:09.060 INFO:teuthology.orchestra.run.smithi154.stdout:506 Cannot talk to daemon 2023-09-23T03:12:09.067 INFO:teuthology.orchestra.run.smithi190.stdout:506 Cannot talk to daemon 2023-09-23T03:12:09.112 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-09-23T03:12:09.117 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-09-23T03:12:09.165 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-09-23T03:12:09.170 INFO:teuthology.orchestra.run.smithi190.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-09-23T03:12:09.295 INFO:teuthology.orchestra.run.smithi190.stderr:bash: line 1: ntpq: command not found 2023-09-23T03:12:09.296 INFO:teuthology.orchestra.run.smithi154.stderr:bash: line 1: ntpq: command not found 2023-09-23T03:12:09.304 INFO:teuthology.orchestra.run.smithi154.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-23T03:12:09.305 INFO:teuthology.orchestra.run.smithi154.stdout:=============================================================================== 2023-09-23T03:12:09.305 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T03:12:09.305 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T03:12:09.305 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T03:12:09.305 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T03:12:09.307 INFO:teuthology.orchestra.run.smithi190.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-23T03:12:09.307 INFO:teuthology.orchestra.run.smithi190.stdout:=============================================================================== 2023-09-23T03:12:09.307 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T03:12:09.307 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T03:12:09.307 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T03:12:09.307 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T03:12:09.308 INFO:teuthology.run_tasks:Running task install... 2023-09-23T03:12:09.317 DEBUG:teuthology.task.install:project ceph 2023-09-23T03:12:09.317 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '862663913a74a99e1c52d50e6a7e59dd6705749a'}} 2023-09-23T03:12:09.317 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '862663913a74a99e1c52d50e6a7e59dd6705749a'} 2023-09-23T03:12:09.317 INFO:teuthology.task.install:Using flavor: default 2023-09-23T03:12:09.323 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-09-23T03:12:09.323 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-09-23T03:12:09.323 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '862663913a74a99e1c52d50e6a7e59dd6705749a', 'tag': None, 'wait_for_package': False} 2023-09-23T03:12:09.323 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=862663913a74a99e1c52d50e6a7e59dd6705749a 2023-09-23T03:12:09.325 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '862663913a74a99e1c52d50e6a7e59dd6705749a', 'tag': None, 'wait_for_package': False} 2023-09-23T03:12:09.325 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=862663913a74a99e1c52d50e6a7e59dd6705749a 2023-09-23T03:12:09.473 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/9/flavors/default/ 2023-09-23T03:12:09.473 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6336.g86266391 2023-09-23T03:12:09.569 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/9/flavors/default/ 2023-09-23T03:12:09.569 INFO:teuthology.task.install.rpm:Package version is 18.0.0-6336.g86266391 2023-09-23T03:12:09.667 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-23T03:12:09.668 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:12:09.668 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-23T03:12:09.773 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-09-23T03:12:09.773 DEBUG:teuthology.orchestra.run.smithi190:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/862663913a74a99e1c52d50e6a7e59dd6705749a/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-23T03:12:09.855 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230922.144532/862663913a74a99e1c52d50e6a7e59dd6705749a/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-23T03:12:09.855 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:12:09.855 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-23T03:12:09.917 DEBUG:teuthology.orchestra.run.smithi190:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-23T03:12:09.960 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-09-23T03:12:09.961 DEBUG:teuthology.orchestra.run.smithi154:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/862663913a74a99e1c52d50e6a7e59dd6705749a/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-23T03:12:10.051 DEBUG:teuthology.orchestra.run.smithi190:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-23T03:12:10.105 DEBUG:teuthology.orchestra.run.smithi154:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-23T03:12:10.143 INFO:teuthology.orchestra.run.smithi190.stdout:check_obsoletes = 1 2023-09-23T03:12:10.147 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum clean all 2023-09-23T03:12:10.233 DEBUG:teuthology.orchestra.run.smithi154:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-23T03:12:10.328 INFO:teuthology.orchestra.run.smithi154.stdout:check_obsoletes = 1 2023-09-23T03:12:10.332 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum clean all 2023-09-23T03:12:10.492 INFO:teuthology.orchestra.run.smithi190.stdout:69 files removed 2023-09-23T03:12:10.530 DEBUG:teuthology.orchestra.run.smithi190:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-09-23T03:12:10.697 INFO:teuthology.orchestra.run.smithi154.stdout:69 files removed 2023-09-23T03:12:10.736 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-09-23T03:12:12.020 INFO:teuthology.orchestra.run.smithi190.stdout:ceph packages for x86_64 79 kB/s | 78 kB 00:00 2023-09-23T03:12:12.294 INFO:teuthology.orchestra.run.smithi154.stdout:ceph packages for x86_64 74 kB/s | 78 kB 00:01 2023-09-23T03:12:13.004 INFO:teuthology.orchestra.run.smithi190.stdout:ceph noarch packages 11 kB/s | 9.5 kB 00:00 2023-09-23T03:12:13.228 INFO:teuthology.orchestra.run.smithi154.stdout:ceph noarch packages 11 kB/s | 9.5 kB 00:00 2023-09-23T03:12:14.063 INFO:teuthology.orchestra.run.smithi190.stdout:ceph source packages 1.9 kB/s | 1.9 kB 00:00 2023-09-23T03:12:14.203 INFO:teuthology.orchestra.run.smithi154.stdout:ceph source packages 2.1 kB/s | 1.9 kB 00:00 2023-09-23T03:12:14.874 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS-9 - AppStream 11 MB/s | 6.3 MB 00:00 2023-09-23T03:12:14.991 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS-9 - AppStream 7.4 MB/s | 6.3 MB 00:00 2023-09-23T03:12:16.566 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS-9 - BaseOS 4.1 MB/s | 2.0 MB 00:00 2023-09-23T03:12:16.600 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS-9 - BaseOS 4.8 MB/s | 2.0 MB 00:00 2023-09-23T03:12:17.854 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - BaseOS 8.6 MB/s | 7.8 MB 00:00 2023-09-23T03:12:18.579 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 7.8 MB 00:01 2023-09-23T03:12:19.688 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - AppStream 25 MB/s | 18 MB 00:00 2023-09-23T03:12:24.802 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 5.7 MB 00:01 2023-09-23T03:12:25.471 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - AppStream 3.1 MB/s | 18 MB 00:05 2023-09-23T03:12:26.406 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 14 kB 00:00 2023-09-23T03:12:27.649 INFO:teuthology.orchestra.run.smithi154.stdout:Extra Packages for Enterprise Linux 17 MB/s | 19 MB 00:01 2023-09-23T03:12:30.539 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 5.7 MB 00:01 2023-09-23T03:12:32.339 INFO:teuthology.orchestra.run.smithi190.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 14 kB 00:00 2023-09-23T03:12:32.733 INFO:teuthology.orchestra.run.smithi154.stdout:lab-extras 23 kB/s | 1.7 kB 00:00 2023-09-23T03:12:33.988 INFO:teuthology.orchestra.run.smithi190.stdout:Extra Packages for Enterprise Linux 12 MB/s | 19 MB 00:01 2023-09-23T03:12:35.157 INFO:teuthology.orchestra.run.smithi154.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-09-23T03:12:35.159 INFO:teuthology.orchestra.run.smithi154.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-09-23T03:12:35.180 INFO:teuthology.orchestra.run.smithi154.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-09-23T03:12:35.182 INFO:teuthology.orchestra.run.smithi154.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-09-23T03:12:35.184 INFO:teuthology.orchestra.run.smithi154.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-09-23T03:12:35.185 INFO:teuthology.orchestra.run.smithi154.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-09-23T03:12:35.188 INFO:teuthology.orchestra.run.smithi154.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-09-23T03:12:35.191 INFO:teuthology.orchestra.run.smithi154.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-09-23T03:12:35.192 INFO:teuthology.orchestra.run.smithi154.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-09-23T03:12:35.268 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2023-09-23T03:12:35.292 INFO:teuthology.orchestra.run.smithi154.stdout:============================================================================================ 2023-09-23T03:12:35.293 INFO:teuthology.orchestra.run.smithi154.stdout: Package Arch Version Repository Size 2023-09-23T03:12:35.293 INFO:teuthology.orchestra.run.smithi154.stdout:============================================================================================ 2023-09-23T03:12:35.293 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2023-09-23T03:12:35.293 INFO:teuthology.orchestra.run.smithi154.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-09-23T03:12:35.293 INFO:teuthology.orchestra.run.smithi154.stdout: ceph x86_64 2:18.0.0-6336.g86266391.el9 ceph 6.5 k 2023-09-23T03:12:35.293 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-base x86_64 2:18.0.0-6336.g86266391.el9 ceph 5.2 M 2023-09-23T03:12:35.293 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-fuse x86_64 2:18.0.0-6336.g86266391.el9 ceph 855 k 2023-09-23T03:12:35.293 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6336.g86266391.el9 ceph 141 k 2023-09-23T03:12:35.294 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr x86_64 2:18.0.0-6336.g86266391.el9 ceph 1.5 M 2023-09-23T03:12:35.294 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6336.g86266391.el9 ceph-noarch 132 k 2023-09-23T03:12:35.294 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6336.g86266391.el9 ceph-noarch 1.7 M 2023-09-23T03:12:35.294 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6336.g86266391.el9 ceph-noarch 7.4 M 2023-09-23T03:12:35.294 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-rook noarch 2:18.0.0-6336.g86266391.el9 ceph-noarch 47 k 2023-09-23T03:12:35.294 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-radosgw x86_64 2:18.0.0-6336.g86266391.el9 ceph 8.6 M 2023-09-23T03:12:35.294 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-test x86_64 2:18.0.0-6336.g86266391.el9 ceph 41 M 2023-09-23T03:12:35.294 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-volume noarch 2:18.0.0-6336.g86266391.el9 ceph-noarch 262 k 2023-09-23T03:12:35.294 INFO:teuthology.orchestra.run.smithi154.stdout: cephadm noarch 2:18.0.0-6336.g86266391.el9 ceph-noarch 248 k 2023-09-23T03:12:35.295 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-mirror x86_64 2:18.0.0-6336.g86266391.el9 ceph 218 k 2023-09-23T03:12:35.295 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-top noarch 2:18.0.0-6336.g86266391.el9 ceph-noarch 18 k 2023-09-23T03:12:35.295 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libelf-devel x86_64 0.189-3.el9 appstream 22 k 2023-09-23T03:12:35.295 INFO:teuthology.orchestra.run.smithi154.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-09-23T03:12:35.295 INFO:teuthology.orchestra.run.smithi154.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-09-23T03:12:35.295 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs-devel x86_64 2:18.0.0-6336.g86266391.el9 ceph 31 k 2023-09-23T03:12:35.295 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs2 x86_64 2:18.0.0-6336.g86266391.el9 ceph 672 k 2023-09-23T03:12:35.295 INFO:teuthology.orchestra.run.smithi154.stdout: librados-devel x86_64 2:18.0.0-6336.g86266391.el9 ceph 126 k 2023-09-23T03:12:35.295 INFO:teuthology.orchestra.run.smithi154.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-09-23T03:12:35.296 INFO:teuthology.orchestra.run.smithi154.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2023-09-23T03:12:35.296 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-devel x86_64 1:3.0.7-24.el9 appstream 4.1 M 2023-09-23T03:12:35.296 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-09-23T03:12:35.296 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-09-23T03:12:35.296 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-09-23T03:12:35.296 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cephfs x86_64 2:18.0.0-6336.g86266391.el9 ceph 162 k 2023-09-23T03:12:35.296 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rados x86_64 2:18.0.0-6336.g86266391.el9 ceph 320 k 2023-09-23T03:12:35.296 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rbd x86_64 2:18.0.0-6336.g86266391.el9 ceph 297 k 2023-09-23T03:12:35.296 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rgw x86_64 2:18.0.0-6336.g86266391.el9 ceph 99 k 2023-09-23T03:12:35.297 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-fuse x86_64 2:18.0.0-6336.g86266391.el9 ceph 85 k 2023-09-23T03:12:35.297 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-mirror x86_64 2:18.0.0-6336.g86266391.el9 ceph 3.0 M 2023-09-23T03:12:35.297 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-nbd x86_64 2:18.0.0-6336.g86266391.el9 ceph 169 k 2023-09-23T03:12:35.297 INFO:teuthology.orchestra.run.smithi154.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 appstream 86 k 2023-09-23T03:12:35.297 INFO:teuthology.orchestra.run.smithi154.stdout:Upgrading: 2023-09-23T03:12:35.297 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager x86_64 1:1.45.2-1.el9 baseos 2.2 M 2023-09-23T03:12:35.297 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-libnm x86_64 1:1.45.2-1.el9 baseos 1.8 M 2023-09-23T03:12:35.297 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-team x86_64 1:1.45.2-1.el9 baseos 42 k 2023-09-23T03:12:35.297 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-tui x86_64 1:1.45.2-1.el9 baseos 247 k 2023-09-23T03:12:35.297 INFO:teuthology.orchestra.run.smithi154.stdout: librados2 x86_64 2:18.0.0-6336.g86266391.el9 ceph 3.3 M 2023-09-23T03:12:35.298 INFO:teuthology.orchestra.run.smithi154.stdout: librbd1 x86_64 2:18.0.0-6336.g86266391.el9 ceph 3.0 M 2023-09-23T03:12:35.298 INFO:teuthology.orchestra.run.smithi154.stdout:Installing dependencies: 2023-09-23T03:12:35.298 INFO:teuthology.orchestra.run.smithi154.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2023-09-23T03:12:35.298 INFO:teuthology.orchestra.run.smithi154.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-09-23T03:12:35.298 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-common x86_64 2:18.0.0-6336.g86266391.el9 ceph 19 M 2023-09-23T03:12:35.298 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6336.g86266391.el9 ceph-noarch 23 k 2023-09-23T03:12:35.298 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mds x86_64 2:18.0.0-6336.g86266391.el9 ceph 2.1 M 2023-09-23T03:12:35.298 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6336.g86266391.el9 ceph-noarch 242 k 2023-09-23T03:12:35.298 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mon x86_64 2:18.0.0-6336.g86266391.el9 ceph 4.4 M 2023-09-23T03:12:35.299 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-osd x86_64 2:18.0.0-6336.g86266391.el9 ceph 18 M 2023-09-23T03:12:35.299 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6336.g86266391.el9 ceph-noarch 15 k 2023-09-23T03:12:35.299 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-selinux x86_64 2:18.0.0-6336.g86266391.el9 ceph 24 k 2023-09-23T03:12:35.299 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-09-23T03:12:35.299 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-09-23T03:12:35.299 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-09-23T03:12:35.299 INFO:teuthology.orchestra.run.smithi154.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-09-23T03:12:35.299 INFO:teuthology.orchestra.run.smithi154.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-09-23T03:12:35.299 INFO:teuthology.orchestra.run.smithi154.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-09-23T03:12:35.300 INFO:teuthology.orchestra.run.smithi154.stdout: jq x86_64 1.6-15.el9 appstream 187 k 2023-09-23T03:12:35.300 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-09-23T03:12:35.300 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-09-23T03:12:35.300 INFO:teuthology.orchestra.run.smithi154.stdout: libcephsqlite x86_64 2:18.0.0-6336.g86266391.el9 ceph 164 k 2023-09-23T03:12:35.300 INFO:teuthology.orchestra.run.smithi154.stdout: libgfortran x86_64 11.4.1-2.1.el9 baseos 809 k 2023-09-23T03:12:35.300 INFO:teuthology.orchestra.run.smithi154.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-09-23T03:12:35.300 INFO:teuthology.orchestra.run.smithi154.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-09-23T03:12:35.300 INFO:teuthology.orchestra.run.smithi154.stdout: libquadmath x86_64 11.4.1-2.1.el9 baseos 198 k 2023-09-23T03:12:35.300 INFO:teuthology.orchestra.run.smithi154.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2023-09-23T03:12:35.300 INFO:teuthology.orchestra.run.smithi154.stdout: libradosstriper1 x86_64 2:18.0.0-6336.g86266391.el9 ceph 473 k 2023-09-23T03:12:35.300 INFO:teuthology.orchestra.run.smithi154.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-09-23T03:12:35.301 INFO:teuthology.orchestra.run.smithi154.stdout: librgw2 x86_64 2:18.0.0-6336.g86266391.el9 ceph 4.8 M 2023-09-23T03:12:35.301 INFO:teuthology.orchestra.run.smithi154.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-09-23T03:12:35.301 INFO:teuthology.orchestra.run.smithi154.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-09-23T03:12:35.301 INFO:teuthology.orchestra.run.smithi154.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-09-23T03:12:35.301 INFO:teuthology.orchestra.run.smithi154.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-09-23T03:12:35.301 INFO:teuthology.orchestra.run.smithi154.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-09-23T03:12:35.301 INFO:teuthology.orchestra.run.smithi154.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-09-23T03:12:35.301 INFO:teuthology.orchestra.run.smithi154.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-09-23T03:12:35.301 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-09-23T03:12:35.301 INFO:teuthology.orchestra.run.smithi154.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-09-23T03:12:35.302 INFO:teuthology.orchestra.run.smithi154.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-09-23T03:12:35.302 INFO:teuthology.orchestra.run.smithi154.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-09-23T03:12:35.302 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-09-23T03:12:35.302 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-argparse x86_64 2:18.0.0-6336.g86266391.el9 ceph 45 k 2023-09-23T03:12:35.302 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-common x86_64 2:18.0.0-6336.g86266391.el9 ceph 127 k 2023-09-23T03:12:35.302 INFO:teuthology.orchestra.run.smithi154.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-09-23T03:12:35.302 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-09-23T03:12:35.302 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-09-23T03:12:35.302 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-09-23T03:12:35.303 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2023-09-23T03:12:35.303 INFO:teuthology.orchestra.run.smithi154.stdout: python3-devel x86_64 3.9.17-2.el9 CentOS-AppStream 205 k 2023-09-23T03:12:35.303 INFO:teuthology.orchestra.run.smithi154.stdout: python3-google-auth noarch 1:2.22.0-1.el9 epel 197 k 2023-09-23T03:12:35.303 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-09-23T03:12:35.303 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-09-23T03:12:35.303 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-09-23T03:12:35.303 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-09-23T03:12:35.303 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-09-23T03:12:35.303 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-09-23T03:12:35.303 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-09-23T03:12:35.304 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-09-23T03:12:35.304 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-09-23T03:12:35.304 INFO:teuthology.orchestra.run.smithi154.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-09-23T03:12:35.304 INFO:teuthology.orchestra.run.smithi154.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-09-23T03:12:35.304 INFO:teuthology.orchestra.run.smithi154.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-09-23T03:12:35.304 INFO:teuthology.orchestra.run.smithi154.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-09-23T03:12:35.304 INFO:teuthology.orchestra.run.smithi154.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-09-23T03:12:35.304 INFO:teuthology.orchestra.run.smithi154.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-09-23T03:12:35.304 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-09-23T03:12:35.304 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-09-23T03:12:35.305 INFO:teuthology.orchestra.run.smithi154.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-09-23T03:12:35.305 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-09-23T03:12:35.305 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-09-23T03:12:35.305 INFO:teuthology.orchestra.run.smithi154.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-09-23T03:12:35.305 INFO:teuthology.orchestra.run.smithi154.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-09-23T03:12:35.305 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-09-23T03:12:35.305 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-09-23T03:12:35.305 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-09-23T03:12:35.305 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-09-23T03:12:35.305 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2023-09-23T03:12:35.306 INFO:teuthology.orchestra.run.smithi154.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-09-23T03:12:35.306 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-09-23T03:12:35.306 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-09-23T03:12:35.306 INFO:teuthology.orchestra.run.smithi154.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-09-23T03:12:35.306 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-09-23T03:12:35.306 INFO:teuthology.orchestra.run.smithi154.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-09-23T03:12:35.306 INFO:teuthology.orchestra.run.smithi154.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-09-23T03:12:35.306 INFO:teuthology.orchestra.run.smithi154.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-09-23T03:12:35.306 INFO:teuthology.orchestra.run.smithi154.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-09-23T03:12:35.307 INFO:teuthology.orchestra.run.smithi154.stdout: python3-urllib3 noarch 1.26.5-3.el9 CentOS-BaseOS 188 k 2023-09-23T03:12:35.307 INFO:teuthology.orchestra.run.smithi154.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-09-23T03:12:35.307 INFO:teuthology.orchestra.run.smithi154.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-09-23T03:12:35.307 INFO:teuthology.orchestra.run.smithi154.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-09-23T03:12:35.307 INFO:teuthology.orchestra.run.smithi154.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-09-23T03:12:35.307 INFO:teuthology.orchestra.run.smithi154.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-09-23T03:12:35.307 INFO:teuthology.orchestra.run.smithi154.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-09-23T03:12:35.307 INFO:teuthology.orchestra.run.smithi154.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-09-23T03:12:35.307 INFO:teuthology.orchestra.run.smithi154.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-09-23T03:12:35.308 INFO:teuthology.orchestra.run.smithi154.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-09-23T03:12:35.308 INFO:teuthology.orchestra.run.smithi154.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2023-09-23T03:12:35.308 INFO:teuthology.orchestra.run.smithi154.stdout:Installing weak dependencies: 2023-09-23T03:12:35.308 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-09-23T03:12:35.308 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:12:35.308 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2023-09-23T03:12:35.308 INFO:teuthology.orchestra.run.smithi154.stdout:============================================================================================ 2023-09-23T03:12:35.308 INFO:teuthology.orchestra.run.smithi154.stdout:Install 133 Packages 2023-09-23T03:12:35.308 INFO:teuthology.orchestra.run.smithi154.stdout:Upgrade 6 Packages 2023-09-23T03:12:35.308 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:12:35.309 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 199 M 2023-09-23T03:12:35.309 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2023-09-23T03:12:36.393 INFO:teuthology.orchestra.run.smithi154.stdout:(1/139): ceph-18.0.0-6336.g86266391.el9.x86_64. 22 kB/s | 6.5 kB 00:00 2023-09-23T03:12:36.651 INFO:teuthology.orchestra.run.smithi154.stdout:(2/139): ceph-fuse-18.0.0-6336.g86266391.el9.x8 3.2 MB/s | 855 kB 00:00 2023-09-23T03:12:36.727 INFO:teuthology.orchestra.run.smithi154.stdout:(3/139): ceph-immutable-object-cache-18.0.0-633 1.9 MB/s | 141 kB 00:00 2023-09-23T03:12:37.060 INFO:teuthology.orchestra.run.smithi154.stdout:(4/139): ceph-mds-18.0.0-6336.g86266391.el9.x86 6.3 MB/s | 2.1 MB 00:00 2023-09-23T03:12:37.352 INFO:teuthology.orchestra.run.smithi154.stdout:(5/139): ceph-base-18.0.0-6336.g86266391.el9.x8 4.2 MB/s | 5.2 MB 00:01 2023-09-23T03:12:37.402 INFO:teuthology.orchestra.run.smithi154.stdout:(6/139): ceph-mgr-18.0.0-6336.g86266391.el9.x86 4.3 MB/s | 1.5 MB 00:00 2023-09-23T03:12:38.097 INFO:teuthology.orchestra.run.smithi154.stdout:(7/139): ceph-common-18.0.0-6336.g86266391.el9. 9.5 MB/s | 19 MB 00:01 2023-09-23T03:12:38.286 INFO:teuthology.orchestra.run.smithi154.stdout:(8/139): ceph-mon-18.0.0-6336.g86266391.el9.x86 4.7 MB/s | 4.4 MB 00:00 2023-09-23T03:12:38.395 INFO:teuthology.orchestra.run.smithi154.stdout:(9/139): ceph-selinux-18.0.0-6336.g86266391.el9 224 kB/s | 24 kB 00:00 2023-09-23T03:12:38.812 INFO:teuthology.orchestra.run.smithi154.stdout:(10/139): ceph-radosgw-18.0.0-6336.g86266391.el 12 MB/s | 8.6 MB 00:00 2023-09-23T03:12:38.887 INFO:teuthology.orchestra.run.smithi154.stdout:(11/139): cephfs-mirror-18.0.0-6336.g86266391.e 2.9 MB/s | 218 kB 00:00 2023-09-23T03:12:38.946 INFO:teuthology.orchestra.run.smithi154.stdout:(12/139): libcephfs-devel-18.0.0-6336.g86266391 542 kB/s | 31 kB 00:00 2023-09-23T03:12:39.032 INFO:teuthology.orchestra.run.smithi190.stdout:lab-extras 24 kB/s | 1.7 kB 00:00 2023-09-23T03:12:39.113 INFO:teuthology.orchestra.run.smithi154.stdout:(13/139): libcephfs2-18.0.0-6336.g86266391.el9. 4.0 MB/s | 672 kB 00:00 2023-09-23T03:12:39.196 INFO:teuthology.orchestra.run.smithi154.stdout:(14/139): libcephsqlite-18.0.0-6336.g86266391.e 1.9 MB/s | 164 kB 00:00 2023-09-23T03:12:39.304 INFO:teuthology.orchestra.run.smithi154.stdout:(15/139): librados-devel-18.0.0-6336.g86266391. 1.2 MB/s | 126 kB 00:00 2023-09-23T03:12:39.447 INFO:teuthology.orchestra.run.smithi154.stdout:(16/139): libradosstriper1-18.0.0-6336.g8626639 3.3 MB/s | 473 kB 00:00 2023-09-23T03:12:39.697 INFO:teuthology.orchestra.run.smithi154.stdout:(17/139): ceph-osd-18.0.0-6336.g86266391.el9.x8 7.7 MB/s | 18 MB 00:02 2023-09-23T03:12:39.807 INFO:teuthology.orchestra.run.smithi154.stdout:(18/139): python3-ceph-argparse-18.0.0-6336.g86 416 kB/s | 45 kB 00:00 2023-09-23T03:12:39.873 INFO:teuthology.orchestra.run.smithi154.stdout:(19/139): python3-ceph-common-18.0.0-6336.g8626 1.9 MB/s | 127 kB 00:00 2023-09-23T03:12:39.956 INFO:teuthology.orchestra.run.smithi154.stdout:(20/139): python3-cephfs-18.0.0-6336.g86266391. 1.9 MB/s | 162 kB 00:00 2023-09-23T03:12:40.107 INFO:teuthology.orchestra.run.smithi154.stdout:(21/139): librgw2-18.0.0-6336.g86266391.el9.x86 7.2 MB/s | 4.8 MB 00:00 2023-09-23T03:12:40.140 INFO:teuthology.orchestra.run.smithi154.stdout:(22/139): python3-rados-18.0.0-6336.g86266391.e 1.7 MB/s | 320 kB 00:00 2023-09-23T03:12:41.338 INFO:teuthology.orchestra.run.smithi190.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-09-23T03:12:41.340 INFO:teuthology.orchestra.run.smithi190.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-09-23T03:12:41.360 INFO:teuthology.orchestra.run.smithi190.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-09-23T03:12:41.362 INFO:teuthology.orchestra.run.smithi190.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-09-23T03:12:41.363 INFO:teuthology.orchestra.run.smithi190.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-09-23T03:12:41.365 INFO:teuthology.orchestra.run.smithi190.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-09-23T03:12:41.367 INFO:teuthology.orchestra.run.smithi190.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-09-23T03:12:41.371 INFO:teuthology.orchestra.run.smithi190.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-09-23T03:12:41.372 INFO:teuthology.orchestra.run.smithi190.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-09-23T03:12:41.446 INFO:teuthology.orchestra.run.smithi190.stdout:Dependencies resolved. 2023-09-23T03:12:41.467 INFO:teuthology.orchestra.run.smithi190.stdout:============================================================================================ 2023-09-23T03:12:41.467 INFO:teuthology.orchestra.run.smithi190.stdout: Package Arch Version Repository Size 2023-09-23T03:12:41.467 INFO:teuthology.orchestra.run.smithi190.stdout:============================================================================================ 2023-09-23T03:12:41.467 INFO:teuthology.orchestra.run.smithi190.stdout:Installing: 2023-09-23T03:12:41.467 INFO:teuthology.orchestra.run.smithi190.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-09-23T03:12:41.467 INFO:teuthology.orchestra.run.smithi190.stdout: ceph x86_64 2:18.0.0-6336.g86266391.el9 ceph 6.5 k 2023-09-23T03:12:41.468 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base x86_64 2:18.0.0-6336.g86266391.el9 ceph 5.2 M 2023-09-23T03:12:41.468 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-fuse x86_64 2:18.0.0-6336.g86266391.el9 ceph 855 k 2023-09-23T03:12:41.468 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-6336.g86266391.el9 ceph 141 k 2023-09-23T03:12:41.468 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr x86_64 2:18.0.0-6336.g86266391.el9 ceph 1.5 M 2023-09-23T03:12:41.468 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-cephadm noarch 2:18.0.0-6336.g86266391.el9 ceph-noarch 132 k 2023-09-23T03:12:41.468 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard noarch 2:18.0.0-6336.g86266391.el9 ceph-noarch 1.7 M 2023-09-23T03:12:41.468 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-6336.g86266391.el9 ceph-noarch 7.4 M 2023-09-23T03:12:41.468 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-rook noarch 2:18.0.0-6336.g86266391.el9 ceph-noarch 47 k 2023-09-23T03:12:41.468 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-radosgw x86_64 2:18.0.0-6336.g86266391.el9 ceph 8.6 M 2023-09-23T03:12:41.468 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-test x86_64 2:18.0.0-6336.g86266391.el9 ceph 41 M 2023-09-23T03:12:41.468 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-volume noarch 2:18.0.0-6336.g86266391.el9 ceph-noarch 262 k 2023-09-23T03:12:41.468 INFO:teuthology.orchestra.run.smithi190.stdout: cephadm noarch 2:18.0.0-6336.g86266391.el9 ceph-noarch 248 k 2023-09-23T03:12:41.469 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-mirror x86_64 2:18.0.0-6336.g86266391.el9 ceph 218 k 2023-09-23T03:12:41.469 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-top noarch 2:18.0.0-6336.g86266391.el9 ceph-noarch 18 k 2023-09-23T03:12:41.469 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libelf-devel x86_64 0.189-3.el9 appstream 22 k 2023-09-23T03:12:41.469 INFO:teuthology.orchestra.run.smithi190.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-09-23T03:12:41.469 INFO:teuthology.orchestra.run.smithi190.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-09-23T03:12:41.469 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs-devel x86_64 2:18.0.0-6336.g86266391.el9 ceph 31 k 2023-09-23T03:12:41.469 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs2 x86_64 2:18.0.0-6336.g86266391.el9 ceph 672 k 2023-09-23T03:12:41.469 INFO:teuthology.orchestra.run.smithi190.stdout: librados-devel x86_64 2:18.0.0-6336.g86266391.el9 ceph 126 k 2023-09-23T03:12:41.469 INFO:teuthology.orchestra.run.smithi190.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-09-23T03:12:41.469 INFO:teuthology.orchestra.run.smithi190.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2023-09-23T03:12:41.469 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-devel x86_64 1:3.0.7-24.el9 appstream 4.1 M 2023-09-23T03:12:41.469 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-09-23T03:12:41.469 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-09-23T03:12:41.470 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-09-23T03:12:41.470 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cephfs x86_64 2:18.0.0-6336.g86266391.el9 ceph 162 k 2023-09-23T03:12:41.470 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rados x86_64 2:18.0.0-6336.g86266391.el9 ceph 320 k 2023-09-23T03:12:41.470 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rbd x86_64 2:18.0.0-6336.g86266391.el9 ceph 297 k 2023-09-23T03:12:41.470 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rgw x86_64 2:18.0.0-6336.g86266391.el9 ceph 99 k 2023-09-23T03:12:41.470 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-fuse x86_64 2:18.0.0-6336.g86266391.el9 ceph 85 k 2023-09-23T03:12:41.470 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-mirror x86_64 2:18.0.0-6336.g86266391.el9 ceph 3.0 M 2023-09-23T03:12:41.470 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-nbd x86_64 2:18.0.0-6336.g86266391.el9 ceph 169 k 2023-09-23T03:12:41.470 INFO:teuthology.orchestra.run.smithi190.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 appstream 86 k 2023-09-23T03:12:41.470 INFO:teuthology.orchestra.run.smithi190.stdout:Upgrading: 2023-09-23T03:12:41.470 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager x86_64 1:1.45.2-1.el9 baseos 2.2 M 2023-09-23T03:12:41.471 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-libnm x86_64 1:1.45.2-1.el9 baseos 1.8 M 2023-09-23T03:12:41.471 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-team x86_64 1:1.45.2-1.el9 baseos 42 k 2023-09-23T03:12:41.471 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-tui x86_64 1:1.45.2-1.el9 baseos 247 k 2023-09-23T03:12:41.471 INFO:teuthology.orchestra.run.smithi190.stdout: librados2 x86_64 2:18.0.0-6336.g86266391.el9 ceph 3.3 M 2023-09-23T03:12:41.471 INFO:teuthology.orchestra.run.smithi190.stdout: librbd1 x86_64 2:18.0.0-6336.g86266391.el9 ceph 3.0 M 2023-09-23T03:12:41.471 INFO:teuthology.orchestra.run.smithi190.stdout:Installing dependencies: 2023-09-23T03:12:41.471 INFO:teuthology.orchestra.run.smithi190.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2023-09-23T03:12:41.471 INFO:teuthology.orchestra.run.smithi190.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-09-23T03:12:41.471 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-common x86_64 2:18.0.0-6336.g86266391.el9 ceph 19 M 2023-09-23T03:12:41.471 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-grafana-dashboards noarch 2:18.0.0-6336.g86266391.el9 ceph-noarch 23 k 2023-09-23T03:12:41.471 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mds x86_64 2:18.0.0-6336.g86266391.el9 ceph 2.1 M 2023-09-23T03:12:41.471 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-modules-core noarch 2:18.0.0-6336.g86266391.el9 ceph-noarch 242 k 2023-09-23T03:12:41.471 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mon x86_64 2:18.0.0-6336.g86266391.el9 ceph 4.4 M 2023-09-23T03:12:41.472 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-osd x86_64 2:18.0.0-6336.g86266391.el9 ceph 18 M 2023-09-23T03:12:41.472 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-prometheus-alerts noarch 2:18.0.0-6336.g86266391.el9 ceph-noarch 15 k 2023-09-23T03:12:41.472 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-selinux x86_64 2:18.0.0-6336.g86266391.el9 ceph 24 k 2023-09-23T03:12:41.472 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-09-23T03:12:41.472 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-09-23T03:12:41.472 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-09-23T03:12:41.472 INFO:teuthology.orchestra.run.smithi190.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-09-23T03:12:41.472 INFO:teuthology.orchestra.run.smithi190.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-09-23T03:12:41.472 INFO:teuthology.orchestra.run.smithi190.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-09-23T03:12:41.472 INFO:teuthology.orchestra.run.smithi190.stdout: jq x86_64 1.6-15.el9 appstream 187 k 2023-09-23T03:12:41.472 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-09-23T03:12:41.472 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-09-23T03:12:41.473 INFO:teuthology.orchestra.run.smithi190.stdout: libcephsqlite x86_64 2:18.0.0-6336.g86266391.el9 ceph 164 k 2023-09-23T03:12:41.473 INFO:teuthology.orchestra.run.smithi190.stdout: libgfortran x86_64 11.4.1-2.1.el9 baseos 809 k 2023-09-23T03:12:41.473 INFO:teuthology.orchestra.run.smithi190.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-09-23T03:12:41.473 INFO:teuthology.orchestra.run.smithi190.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-09-23T03:12:41.473 INFO:teuthology.orchestra.run.smithi190.stdout: libquadmath x86_64 11.4.1-2.1.el9 baseos 198 k 2023-09-23T03:12:41.473 INFO:teuthology.orchestra.run.smithi190.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2023-09-23T03:12:41.473 INFO:teuthology.orchestra.run.smithi190.stdout: libradosstriper1 x86_64 2:18.0.0-6336.g86266391.el9 ceph 473 k 2023-09-23T03:12:41.473 INFO:teuthology.orchestra.run.smithi190.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-09-23T03:12:41.473 INFO:teuthology.orchestra.run.smithi190.stdout: librgw2 x86_64 2:18.0.0-6336.g86266391.el9 ceph 4.8 M 2023-09-23T03:12:41.473 INFO:teuthology.orchestra.run.smithi190.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-09-23T03:12:41.473 INFO:teuthology.orchestra.run.smithi190.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-09-23T03:12:41.473 INFO:teuthology.orchestra.run.smithi190.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-09-23T03:12:41.473 INFO:teuthology.orchestra.run.smithi190.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-09-23T03:12:41.474 INFO:teuthology.orchestra.run.smithi190.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-09-23T03:12:41.474 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-09-23T03:12:41.474 INFO:teuthology.orchestra.run.smithi190.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-09-23T03:12:41.474 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-09-23T03:12:41.474 INFO:teuthology.orchestra.run.smithi190.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-09-23T03:12:41.474 INFO:teuthology.orchestra.run.smithi190.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-09-23T03:12:41.474 INFO:teuthology.orchestra.run.smithi190.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-09-23T03:12:41.474 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-09-23T03:12:41.474 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-argparse x86_64 2:18.0.0-6336.g86266391.el9 ceph 45 k 2023-09-23T03:12:41.474 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-common x86_64 2:18.0.0-6336.g86266391.el9 ceph 127 k 2023-09-23T03:12:41.474 INFO:teuthology.orchestra.run.smithi190.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-09-23T03:12:41.475 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-09-23T03:12:41.475 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-09-23T03:12:41.475 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-09-23T03:12:41.475 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2023-09-23T03:12:41.475 INFO:teuthology.orchestra.run.smithi190.stdout: python3-devel x86_64 3.9.17-2.el9 CentOS-AppStream 205 k 2023-09-23T03:12:41.475 INFO:teuthology.orchestra.run.smithi190.stdout: python3-google-auth noarch 1:2.22.0-1.el9 epel 197 k 2023-09-23T03:12:41.475 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-09-23T03:12:41.475 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-09-23T03:12:41.475 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-09-23T03:12:41.475 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-09-23T03:12:41.475 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-09-23T03:12:41.475 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-09-23T03:12:41.476 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-09-23T03:12:41.476 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-09-23T03:12:41.476 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-09-23T03:12:41.476 INFO:teuthology.orchestra.run.smithi190.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-09-23T03:12:41.476 INFO:teuthology.orchestra.run.smithi190.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-09-23T03:12:41.476 INFO:teuthology.orchestra.run.smithi190.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-09-23T03:12:41.476 INFO:teuthology.orchestra.run.smithi190.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-09-23T03:12:41.476 INFO:teuthology.orchestra.run.smithi190.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-09-23T03:12:41.476 INFO:teuthology.orchestra.run.smithi190.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-09-23T03:12:41.476 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-09-23T03:12:41.476 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-09-23T03:12:41.476 INFO:teuthology.orchestra.run.smithi190.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-09-23T03:12:41.477 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-09-23T03:12:41.477 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-09-23T03:12:41.477 INFO:teuthology.orchestra.run.smithi190.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-09-23T03:12:41.477 INFO:teuthology.orchestra.run.smithi190.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-09-23T03:12:41.477 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-09-23T03:12:41.477 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-09-23T03:12:41.477 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-09-23T03:12:41.477 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-09-23T03:12:41.477 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2023-09-23T03:12:41.477 INFO:teuthology.orchestra.run.smithi190.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-09-23T03:12:41.477 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-09-23T03:12:41.477 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-09-23T03:12:41.477 INFO:teuthology.orchestra.run.smithi190.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-09-23T03:12:41.478 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-09-23T03:12:41.478 INFO:teuthology.orchestra.run.smithi190.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-09-23T03:12:41.478 INFO:teuthology.orchestra.run.smithi190.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-09-23T03:12:41.478 INFO:teuthology.orchestra.run.smithi190.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-09-23T03:12:41.478 INFO:teuthology.orchestra.run.smithi190.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-09-23T03:12:41.478 INFO:teuthology.orchestra.run.smithi190.stdout: python3-urllib3 noarch 1.26.5-3.el9 CentOS-BaseOS 188 k 2023-09-23T03:12:41.478 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-09-23T03:12:41.478 INFO:teuthology.orchestra.run.smithi190.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-09-23T03:12:41.478 INFO:teuthology.orchestra.run.smithi190.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-09-23T03:12:41.478 INFO:teuthology.orchestra.run.smithi190.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-09-23T03:12:41.478 INFO:teuthology.orchestra.run.smithi190.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-09-23T03:12:41.478 INFO:teuthology.orchestra.run.smithi190.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-09-23T03:12:41.479 INFO:teuthology.orchestra.run.smithi190.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-09-23T03:12:41.479 INFO:teuthology.orchestra.run.smithi190.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-09-23T03:12:41.479 INFO:teuthology.orchestra.run.smithi190.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-09-23T03:12:41.479 INFO:teuthology.orchestra.run.smithi190.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2023-09-23T03:12:41.479 INFO:teuthology.orchestra.run.smithi190.stdout:Installing weak dependencies: 2023-09-23T03:12:41.479 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-09-23T03:12:41.479 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:12:41.479 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction Summary 2023-09-23T03:12:41.479 INFO:teuthology.orchestra.run.smithi190.stdout:============================================================================================ 2023-09-23T03:12:41.479 INFO:teuthology.orchestra.run.smithi190.stdout:Install 133 Packages 2023-09-23T03:12:41.479 INFO:teuthology.orchestra.run.smithi190.stdout:Upgrade 6 Packages 2023-09-23T03:12:41.479 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:12:41.480 INFO:teuthology.orchestra.run.smithi190.stdout:Total download size: 199 M 2023-09-23T03:12:41.480 INFO:teuthology.orchestra.run.smithi190.stdout:Downloading Packages: 2023-09-23T03:12:42.409 INFO:teuthology.orchestra.run.smithi190.stdout:(1/139): ceph-18.0.0-6336.g86266391.el9.x86_64. 19 kB/s | 6.5 kB 00:00 2023-09-23T03:12:42.734 INFO:teuthology.orchestra.run.smithi190.stdout:(2/139): ceph-fuse-18.0.0-6336.g86266391.el9.x8 2.6 MB/s | 855 kB 00:00 2023-09-23T03:12:42.901 INFO:teuthology.orchestra.run.smithi190.stdout:(3/139): ceph-base-18.0.0-6336.g86266391.el9.x8 6.2 MB/s | 5.2 MB 00:00 2023-09-23T03:12:42.926 INFO:teuthology.orchestra.run.smithi190.stdout:(4/139): ceph-immutable-object-cache-18.0.0-633 742 kB/s | 141 kB 00:00 2023-09-23T03:12:43.186 INFO:teuthology.orchestra.run.smithi190.stdout:(5/139): ceph-mds-18.0.0-6336.g86266391.el9.x86 7.4 MB/s | 2.1 MB 00:00 2023-09-23T03:12:43.310 INFO:teuthology.orchestra.run.smithi190.stdout:(6/139): ceph-mgr-18.0.0-6336.g86266391.el9.x86 3.8 MB/s | 1.5 MB 00:00 2023-09-23T03:12:43.727 INFO:teuthology.orchestra.run.smithi190.stdout:(7/139): ceph-mon-18.0.0-6336.g86266391.el9.x86 8.1 MB/s | 4.4 MB 00:00 2023-09-23T03:12:43.994 INFO:teuthology.orchestra.run.smithi190.stdout:(8/139): ceph-common-18.0.0-6336.g86266391.el9. 9.9 MB/s | 19 MB 00:01 2023-09-23T03:12:44.036 INFO:teuthology.orchestra.run.smithi190.stdout:(9/139): ceph-selinux-18.0.0-6336.g86266391.el9 598 kB/s | 24 kB 00:00 2023-09-23T03:12:44.444 INFO:teuthology.orchestra.run.smithi190.stdout:(10/139): ceph-radosgw-18.0.0-6336.g86266391.el 12 MB/s | 8.6 MB 00:00 2023-09-23T03:12:44.570 INFO:teuthology.orchestra.run.smithi190.stdout:(11/139): cephfs-mirror-18.0.0-6336.g86266391.e 1.7 MB/s | 218 kB 00:00 2023-09-23T03:12:44.662 INFO:teuthology.orchestra.run.smithi190.stdout:(12/139): libcephfs-devel-18.0.0-6336.g86266391 343 kB/s | 31 kB 00:00 2023-09-23T03:12:44.770 INFO:teuthology.orchestra.run.smithi190.stdout:(13/139): libcephfs2-18.0.0-6336.g86266391.el9. 6.1 MB/s | 672 kB 00:00 2023-09-23T03:12:44.845 INFO:teuthology.orchestra.run.smithi190.stdout:(14/139): libcephsqlite-18.0.0-6336.g86266391.e 2.2 MB/s | 164 kB 00:00 2023-09-23T03:12:44.921 INFO:teuthology.orchestra.run.smithi190.stdout:(15/139): librados-devel-18.0.0-6336.g86266391. 1.7 MB/s | 126 kB 00:00 2023-09-23T03:12:45.013 INFO:teuthology.orchestra.run.smithi190.stdout:(16/139): libradosstriper1-18.0.0-6336.g8626639 5.1 MB/s | 473 kB 00:00 2023-09-23T03:12:46.832 INFO:teuthology.orchestra.run.smithi154.stdout:(23/139): python3-rbd-18.0.0-6336.g86266391.el9 44 kB/s | 297 kB 00:06 2023-09-23T03:12:46.896 INFO:teuthology.orchestra.run.smithi190.stdout:(17/139): librgw2-18.0.0-6336.g86266391.el9.x86 2.5 MB/s | 4.8 MB 00:01 2023-09-23T03:12:46.907 INFO:teuthology.orchestra.run.smithi154.stdout:(24/139): python3-rgw-18.0.0-6336.g86266391.el9 15 kB/s | 99 kB 00:06 2023-09-23T03:12:46.941 INFO:teuthology.orchestra.run.smithi154.stdout:(25/139): rbd-fuse-18.0.0-6336.g86266391.el9.x8 787 kB/s | 85 kB 00:00 2023-09-23T03:12:46.988 INFO:teuthology.orchestra.run.smithi190.stdout:(18/139): python3-ceph-argparse-18.0.0-6336.g86 495 kB/s | 45 kB 00:00 2023-09-23T03:12:47.058 INFO:teuthology.orchestra.run.smithi154.stdout:(26/139): rbd-nbd-18.0.0-6336.g86266391.el9.x86 1.4 MB/s | 169 kB 00:00 2023-09-23T03:12:47.105 INFO:teuthology.orchestra.run.smithi190.stdout:(19/139): python3-ceph-common-18.0.0-6336.g8626 1.1 MB/s | 127 kB 00:00 2023-09-23T03:12:47.175 INFO:teuthology.orchestra.run.smithi154.stdout:(27/139): ceph-grafana-dashboards-18.0.0-6336.g 200 kB/s | 23 kB 00:00 2023-09-23T03:12:47.205 INFO:teuthology.orchestra.run.smithi190.stdout:(20/139): python3-cephfs-18.0.0-6336.g86266391. 1.6 MB/s | 162 kB 00:00 2023-09-23T03:12:47.292 INFO:teuthology.orchestra.run.smithi154.stdout:(28/139): ceph-mgr-cephadm-18.0.0-6336.g8626639 1.1 MB/s | 132 kB 00:00 2023-09-23T03:12:47.314 INFO:teuthology.orchestra.run.smithi190.stdout:(21/139): python3-rados-18.0.0-6336.g86266391.e 2.9 MB/s | 320 kB 00:00 2023-09-23T03:12:47.422 INFO:teuthology.orchestra.run.smithi190.stdout:(22/139): python3-rbd-18.0.0-6336.g86266391.el9 2.7 MB/s | 297 kB 00:00 2023-09-23T03:12:47.514 INFO:teuthology.orchestra.run.smithi190.stdout:(23/139): python3-rgw-18.0.0-6336.g86266391.el9 1.1 MB/s | 99 kB 00:00 2023-09-23T03:12:47.675 INFO:teuthology.orchestra.run.smithi154.stdout:(29/139): rbd-mirror-18.0.0-6336.g86266391.el9. 3.9 MB/s | 3.0 MB 00:00 2023-09-23T03:12:47.734 INFO:teuthology.orchestra.run.smithi154.stdout:(30/139): ceph-mgr-dashboard-18.0.0-6336.g86266 4.0 MB/s | 1.7 MB 00:00 2023-09-23T03:12:47.782 INFO:teuthology.orchestra.run.smithi190.stdout:(24/139): ceph-osd-18.0.0-6336.g86266391.el9.x8 4.0 MB/s | 18 MB 00:04 2023-09-23T03:12:47.815 INFO:teuthology.orchestra.run.smithi190.stdout:(25/139): rbd-fuse-18.0.0-6336.g86266391.el9.x8 284 kB/s | 85 kB 00:00 2023-09-23T03:12:47.834 INFO:teuthology.orchestra.run.smithi154.stdout:(31/139): ceph-mgr-modules-core-18.0.0-6336.g86 2.4 MB/s | 242 kB 00:00 2023-09-23T03:12:47.890 INFO:teuthology.orchestra.run.smithi190.stdout:(26/139): rbd-nbd-18.0.0-6336.g86266391.el9.x86 2.2 MB/s | 169 kB 00:00 2023-09-23T03:12:47.943 INFO:teuthology.orchestra.run.smithi154.stdout:(32/139): ceph-mgr-rook-18.0.0-6336.g86266391.e 443 kB/s | 47 kB 00:00 2023-09-23T03:12:47.982 INFO:teuthology.orchestra.run.smithi190.stdout:(27/139): ceph-grafana-dashboards-18.0.0-6336.g 255 kB/s | 23 kB 00:00 2023-09-23T03:12:48.018 INFO:teuthology.orchestra.run.smithi154.stdout:(33/139): ceph-prometheus-alerts-18.0.0-6336.g8 198 kB/s | 15 kB 00:00 2023-09-23T03:12:48.083 INFO:teuthology.orchestra.run.smithi190.stdout:(28/139): ceph-mgr-cephadm-18.0.0-6336.g8626639 1.3 MB/s | 132 kB 00:00 2023-09-23T03:12:48.135 INFO:teuthology.orchestra.run.smithi154.stdout:(34/139): ceph-volume-18.0.0-6336.g86266391.el9 2.2 MB/s | 262 kB 00:00 2023-09-23T03:12:48.277 INFO:teuthology.orchestra.run.smithi154.stdout:(35/139): cephadm-18.0.0-6336.g86266391.el9.noa 1.7 MB/s | 248 kB 00:00 2023-09-23T03:12:48.369 INFO:teuthology.orchestra.run.smithi154.stdout:(36/139): cephfs-top-18.0.0-6336.g86266391.el9. 201 kB/s | 18 kB 00:00 2023-09-23T03:12:48.416 INFO:teuthology.orchestra.run.smithi190.stdout:(29/139): ceph-mgr-dashboard-18.0.0-6336.g86266 5.3 MB/s | 1.7 MB 00:00 2023-09-23T03:12:48.533 INFO:teuthology.orchestra.run.smithi190.stdout:(30/139): rbd-mirror-18.0.0-6336.g86266391.el9. 4.0 MB/s | 3.0 MB 00:00 2023-09-23T03:12:48.561 INFO:teuthology.orchestra.run.smithi154.stdout:(37/139): bison-3.7.4-5.el9.x86_64.rpm 4.7 MB/s | 920 kB 00:00 2023-09-23T03:12:48.603 INFO:teuthology.orchestra.run.smithi154.stdout:(38/139): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 106 kB 00:00 2023-09-23T03:12:48.634 INFO:teuthology.orchestra.run.smithi190.stdout:(31/139): ceph-mgr-modules-core-18.0.0-6336.g86 2.4 MB/s | 242 kB 00:00 2023-09-23T03:12:48.645 INFO:teuthology.orchestra.run.smithi154.stdout:(39/139): flex-2.6.4-9.el9.x86_64.rpm 7.3 MB/s | 307 kB 00:00 2023-09-23T03:12:48.678 INFO:teuthology.orchestra.run.smithi154.stdout:(40/139): flexiblas-3.0.4-8.el9.x86_64.rpm 989 kB/s | 32 kB 00:00 2023-09-23T03:12:48.700 INFO:teuthology.orchestra.run.smithi190.stdout:(32/139): ceph-mgr-rook-18.0.0-6336.g86266391.e 725 kB/s | 47 kB 00:00 2023-09-23T03:12:48.776 INFO:teuthology.orchestra.run.smithi190.stdout:(33/139): ceph-prometheus-alerts-18.0.0-6336.g8 198 kB/s | 15 kB 00:00 2023-09-23T03:12:48.796 INFO:teuthology.orchestra.run.smithi154.stdout:(41/139): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2023-09-23T03:12:48.859 INFO:teuthology.orchestra.run.smithi190.stdout:(34/139): ceph-volume-18.0.0-6336.g86266391.el9 3.1 MB/s | 262 kB 00:00 2023-09-23T03:12:48.935 INFO:teuthology.orchestra.run.smithi190.stdout:(35/139): cephadm-18.0.0-6336.g86266391.el9.noa 3.3 MB/s | 248 kB 00:00 2023-09-23T03:12:48.963 INFO:teuthology.orchestra.run.smithi154.stdout:(42/139): ceph-mgr-diskprediction-local-18.0.0- 5.8 MB/s | 7.4 MB 00:01 2023-09-23T03:12:48.988 INFO:teuthology.orchestra.run.smithi154.stdout:(43/139): flexiblas-openblas-openmp-3.0.4-8.el9 89 kB/s | 17 kB 00:00 2023-09-23T03:12:49.001 INFO:teuthology.orchestra.run.smithi190.stdout:(36/139): cephfs-top-18.0.0-6336.g86266391.el9. 278 kB/s | 18 kB 00:00 2023-09-23T03:12:49.013 INFO:teuthology.orchestra.run.smithi154.stdout:(44/139): libaio-devel-0.3.111-13.el9.x86_64.rp 205 kB/s | 10 kB 00:00 2023-09-23T03:12:49.047 INFO:teuthology.orchestra.run.smithi154.stdout:(45/139): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 159 kB 00:00 2023-09-23T03:12:49.097 INFO:teuthology.orchestra.run.smithi154.stdout:(46/139): librdkafka-1.6.1-102.el9.x86_64.rpm 8.0 MB/s | 661 kB 00:00 2023-09-23T03:12:49.148 INFO:teuthology.orchestra.run.smithi154.stdout:(47/139): libtool-2.4.6-45.el9.x86_64.rpm 6.1 MB/s | 578 kB 00:00 2023-09-23T03:12:49.175 INFO:teuthology.orchestra.run.smithi154.stdout:(48/139): libzstd-devel-1.5.1-2.el9.x86_64.rpm 657 kB/s | 46 kB 00:00 2023-09-23T03:12:49.193 INFO:teuthology.orchestra.run.smithi190.stdout:(37/139): bison-3.7.4-5.el9.x86_64.rpm 4.7 MB/s | 920 kB 00:00 2023-09-23T03:12:49.208 INFO:teuthology.orchestra.run.smithi154.stdout:(49/139): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.7 MB/s | 282 kB 00:00 2023-09-23T03:12:49.235 INFO:teuthology.orchestra.run.smithi190.stdout:(38/139): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 106 kB 00:00 2023-09-23T03:12:49.242 INFO:teuthology.orchestra.run.smithi154.stdout:(50/139): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.3 MB/s | 217 kB 00:00 2023-09-23T03:12:49.268 INFO:teuthology.orchestra.run.smithi154.stdout:(51/139): openblas-0.3.21-2.el9.x86_64.rpm 612 kB/s | 34 kB 00:00 2023-09-23T03:12:49.440 INFO:teuthology.orchestra.run.smithi154.stdout:(52/139): openblas-openmp-0.3.21-2.el9.x86_64.r 24 MB/s | 4.7 MB 00:00 2023-09-23T03:12:49.502 INFO:teuthology.orchestra.run.smithi154.stdout:(53/139): postgresql-13.11-1.el9.x86_64.rpm 7.4 MB/s | 1.5 MB 00:00 2023-09-23T03:12:49.544 INFO:teuthology.orchestra.run.smithi154.stdout:(54/139): postgresql-contrib-13.11-1.el9.x86_64 7.7 MB/s | 811 kB 00:00 2023-09-23T03:12:49.578 INFO:teuthology.orchestra.run.smithi154.stdout:(55/139): postgresql-private-libs-13.11-1.el9.x 2.0 MB/s | 134 kB 00:00 2023-09-23T03:12:49.652 INFO:teuthology.orchestra.run.smithi190.stdout:(39/139): ceph-test-18.0.0-6336.g86266391.el9.x 7.3 MB/s | 41 MB 00:05 2023-09-23T03:12:49.686 INFO:teuthology.orchestra.run.smithi190.stdout:(40/139): flex-2.6.4-9.el9.x86_64.rpm 684 kB/s | 307 kB 00:00 2023-09-23T03:12:49.711 INFO:teuthology.orchestra.run.smithi190.stdout:(41/139): flexiblas-3.0.4-8.el9.x86_64.rpm 582 kB/s | 32 kB 00:00 2023-09-23T03:12:49.745 INFO:teuthology.orchestra.run.smithi154.stdout:(56/139): postgresql-server-13.11-1.el9.x86_64. 29 MB/s | 5.7 MB 00:00 2023-09-23T03:12:49.812 INFO:teuthology.orchestra.run.smithi190.stdout:(42/139): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2023-09-23T03:12:49.837 INFO:teuthology.orchestra.run.smithi190.stdout:(43/139): flexiblas-openblas-openmp-3.0.4-8.el9 164 kB/s | 17 kB 00:00 2023-09-23T03:12:49.870 INFO:teuthology.orchestra.run.smithi190.stdout:(44/139): libaio-devel-0.3.111-13.el9.x86_64.rp 313 kB/s | 10 kB 00:00 2023-09-23T03:12:49.878 INFO:teuthology.orchestra.run.smithi154.stdout:(57/139): python3-babel-2.9.1-2.el9.noarch.rpm 21 MB/s | 5.8 MB 00:00 2023-09-23T03:12:49.895 INFO:teuthology.orchestra.run.smithi190.stdout:(45/139): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 159 kB 00:00 2023-09-23T03:12:49.903 INFO:teuthology.orchestra.run.smithi154.stdout:(58/139): python3-devel-3.9.17-2.el9.x86_64.rpm 1.4 MB/s | 205 kB 00:00 2023-09-23T03:12:49.937 INFO:teuthology.orchestra.run.smithi190.stdout:(46/139): librdkafka-1.6.1-102.el9.x86_64.rpm 10 MB/s | 661 kB 00:00 2023-09-23T03:12:49.979 INFO:teuthology.orchestra.run.smithi154.stdout:(59/139): python3-jsonpatch-1.21-16.el9.noarch. 332 kB/s | 24 kB 00:00 2023-09-23T03:12:49.988 INFO:teuthology.orchestra.run.smithi190.stdout:(47/139): libtool-2.4.6-45.el9.x86_64.rpm 6.7 MB/s | 578 kB 00:00 2023-09-23T03:12:50.013 INFO:teuthology.orchestra.run.smithi154.stdout:(60/139): python3-jinja2-2.11.3-4.el9.noarch.rp 2.1 MB/s | 228 kB 00:00 2023-09-23T03:12:50.013 INFO:teuthology.orchestra.run.smithi190.stdout:(48/139): libzstd-devel-1.5.1-2.el9.x86_64.rpm 675 kB/s | 46 kB 00:00 2023-09-23T03:12:50.038 INFO:teuthology.orchestra.run.smithi154.stdout:(61/139): python3-jsonpointer-2.0-4.el9.noarch. 324 kB/s | 18 kB 00:00 2023-09-23T03:12:50.046 INFO:teuthology.orchestra.run.smithi190.stdout:(49/139): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 282 kB 00:00 2023-09-23T03:12:50.063 INFO:teuthology.orchestra.run.smithi154.stdout:(62/139): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 144 kB 00:00 2023-09-23T03:12:50.080 INFO:teuthology.orchestra.run.smithi190.stdout:(50/139): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.3 MB/s | 217 kB 00:00 2023-09-23T03:12:50.088 INFO:teuthology.orchestra.run.smithi154.stdout:(63/139): python3-markupsafe-1.1.1-12.el9.x86_6 659 kB/s | 32 kB 00:00 2023-09-23T03:12:50.105 INFO:teuthology.orchestra.run.smithi190.stdout:(51/139): openblas-0.3.21-2.el9.x86_64.rpm 616 kB/s | 34 kB 00:00 2023-09-23T03:12:50.230 INFO:teuthology.orchestra.run.smithi154.stdout:(64/139): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 4.8 MB 00:00 2023-09-23T03:12:50.256 INFO:teuthology.orchestra.run.smithi190.stdout:(52/139): openblas-openmp-0.3.21-2.el9.x86_64.r 27 MB/s | 4.7 MB 00:00 2023-09-23T03:12:50.272 INFO:teuthology.orchestra.run.smithi154.stdout:(65/139): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 217 kB 00:00 2023-09-23T03:12:50.306 INFO:teuthology.orchestra.run.smithi154.stdout:(66/139): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 174 kB 00:00 2023-09-23T03:12:50.306 INFO:teuthology.orchestra.run.smithi190.stdout:(53/139): postgresql-13.11-1.el9.x86_64.rpm 8.6 MB/s | 1.5 MB 00:00 2023-09-23T03:12:50.331 INFO:teuthology.orchestra.run.smithi154.stdout:(67/139): python3-prettytable-0.7.2-27.el9.noar 737 kB/s | 41 kB 00:00 2023-09-23T03:12:50.356 INFO:teuthology.orchestra.run.smithi154.stdout:(68/139): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 132 kB 00:00 2023-09-23T03:12:50.390 INFO:teuthology.orchestra.run.smithi154.stdout:(69/139): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 211 kB 00:00 2023-09-23T03:12:50.415 INFO:teuthology.orchestra.run.smithi154.stdout:(70/139): python3-requests-oauthlib-1.3.0-12.el 765 kB/s | 43 kB 00:00 2023-09-23T03:12:50.431 INFO:teuthology.orchestra.run.smithi190.stdout:(54/139): postgresql-contrib-13.11-1.el9.x86_64 4.6 MB/s | 811 kB 00:00 2023-09-23T03:12:50.457 INFO:teuthology.orchestra.run.smithi190.stdout:(55/139): postgresql-private-libs-13.11-1.el9.x 946 kB/s | 134 kB 00:00 2023-09-23T03:12:50.682 INFO:teuthology.orchestra.run.smithi190.stdout:(56/139): postgresql-server-13.11-1.el9.x86_64. 23 MB/s | 5.7 MB 00:00 2023-09-23T03:12:50.832 INFO:teuthology.orchestra.run.smithi190.stdout:(57/139): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 5.8 MB 00:00 2023-09-23T03:12:50.866 INFO:teuthology.orchestra.run.smithi190.stdout:(58/139): python3-devel-3.9.17-2.el9.x86_64.rpm 1.2 MB/s | 205 kB 00:00 2023-09-23T03:12:50.916 INFO:teuthology.orchestra.run.smithi190.stdout:(59/139): python3-jsonpatch-1.21-16.el9.noarch. 506 kB/s | 24 kB 00:00 2023-09-23T03:12:50.949 INFO:teuthology.orchestra.run.smithi190.stdout:(60/139): python3-jinja2-2.11.3-4.el9.noarch.rp 2.7 MB/s | 228 kB 00:00 2023-09-23T03:12:50.975 INFO:teuthology.orchestra.run.smithi190.stdout:(61/139): python3-jsonpointer-2.0-4.el9.noarch. 324 kB/s | 18 kB 00:00 2023-09-23T03:12:51.009 INFO:teuthology.orchestra.run.smithi190.stdout:(62/139): python3-mako-1.1.4-6.el9.noarch.rpm 2.4 MB/s | 144 kB 00:00 2023-09-23T03:12:51.066 INFO:teuthology.orchestra.run.smithi154.stdout:(71/139): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2023-09-23T03:12:51.091 INFO:teuthology.orchestra.run.smithi154.stdout:(72/139): python3-toml-0.10.2-6.el9.noarch.rpm 59 kB/s | 39 kB 00:00 2023-09-23T03:12:51.125 INFO:teuthology.orchestra.run.smithi154.stdout:(73/139): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 300 kB 00:00 2023-09-23T03:12:51.136 INFO:teuthology.orchestra.run.smithi190.stdout:(63/139): python3-markupsafe-1.1.1-12.el9.x86_6 198 kB/s | 32 kB 00:00 2023-09-23T03:12:51.142 INFO:teuthology.orchestra.run.smithi154.stdout:(74/139): uuid-1.6.2-55.el9.x86_64.rpm 1.2 MB/s | 56 kB 00:00 2023-09-23T03:12:51.167 INFO:teuthology.orchestra.run.smithi154.stdout:(75/139): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 63 kB 00:00 2023-09-23T03:12:51.192 INFO:teuthology.orchestra.run.smithi154.stdout:(76/139): fuse-2.9.9-15.el9.x86_64.rpm 1.6 MB/s | 79 kB 00:00 2023-09-23T03:12:51.234 INFO:teuthology.orchestra.run.smithi154.stdout:(77/139): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.6 MB/s | 241 kB 00:00 2023-09-23T03:12:51.260 INFO:teuthology.orchestra.run.smithi154.stdout:(78/139): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 103 kB 00:00 2023-09-23T03:12:51.268 INFO:teuthology.orchestra.run.smithi190.stdout:(64/139): python3-numpy-1.20.1-5.el9.x86_64.rpm 19 MB/s | 4.8 MB 00:00 2023-09-23T03:12:51.285 INFO:teuthology.orchestra.run.smithi154.stdout:(79/139): python3-pycparser-2.20-6.el9.noarch.r 2.5 MB/s | 124 kB 00:00 2023-09-23T03:12:51.309 INFO:teuthology.orchestra.run.smithi190.stdout:(65/139): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.4 MB/s | 217 kB 00:00 2023-09-23T03:12:51.311 INFO:teuthology.orchestra.run.smithi154.stdout:(80/139): python3-requests-2.25.1-7.el9.noarch. 2.3 MB/s | 113 kB 00:00 2023-09-23T03:12:51.353 INFO:teuthology.orchestra.run.smithi154.stdout:(81/139): python3-urllib3-1.26.5-3.el9.noarch.r 2.8 MB/s | 188 kB 00:00 2023-09-23T03:12:51.376 INFO:teuthology.orchestra.run.smithi190.stdout:(66/139): python3-prettytable-0.7.2-27.el9.noar 632 kB/s | 41 kB 00:00 2023-09-23T03:12:51.410 INFO:teuthology.orchestra.run.smithi190.stdout:(67/139): python3-oauthlib-3.1.1-5.el9.noarch.r 1.7 MB/s | 174 kB 00:00 2023-09-23T03:12:51.436 INFO:teuthology.orchestra.run.smithi190.stdout:(68/139): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 132 kB 00:00 2023-09-23T03:12:51.478 INFO:teuthology.orchestra.run.smithi190.stdout:(69/139): python3-pyasn1-modules-0.4.8-6.el9.no 3.2 MB/s | 211 kB 00:00 2023-09-23T03:12:51.503 INFO:teuthology.orchestra.run.smithi190.stdout:(70/139): python3-requests-oauthlib-1.3.0-12.el 669 kB/s | 43 kB 00:00 2023-09-23T03:12:51.695 INFO:teuthology.orchestra.run.smithi190.stdout:(71/139): ceph-mgr-diskprediction-local-18.0.0- 2.3 MB/s | 7.4 MB 00:03 2023-09-23T03:12:51.795 INFO:teuthology.orchestra.run.smithi190.stdout:(72/139): python3-toml-0.10.2-6.el9.noarch.rpm 144 kB/s | 39 kB 00:00 2023-09-23T03:12:51.803 INFO:teuthology.orchestra.run.smithi154.stdout:(82/139): ceph-test-18.0.0-6336.g86266391.el9.x 3.0 MB/s | 41 MB 00:13 2023-09-23T03:12:51.878 INFO:teuthology.orchestra.run.smithi154.stdout:(83/139): libquadmath-11.4.1-2.1.el9.x86_64.rpm 378 kB/s | 198 kB 00:00 2023-09-23T03:12:51.945 INFO:teuthology.orchestra.run.smithi154.stdout:(84/139): libgfortran-11.4.1-2.1.el9.x86_64.rpm 1.3 MB/s | 809 kB 00:00 2023-09-23T03:12:52.004 INFO:teuthology.orchestra.run.smithi190.stdout:(73/139): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2023-09-23T03:12:52.029 INFO:teuthology.orchestra.run.smithi190.stdout:(74/139): socat-1.7.4.1-5.el9.x86_64.rpm 948 kB/s | 300 kB 00:00 2023-09-23T03:12:52.037 INFO:teuthology.orchestra.run.smithi154.stdout:(85/139): python3-cryptography-36.0.1-4.el9.x86 5.3 MB/s | 1.2 MB 00:00 2023-09-23T03:12:52.054 INFO:teuthology.orchestra.run.smithi190.stdout:(75/139): uuid-1.6.2-55.el9.x86_64.rpm 220 kB/s | 56 kB 00:00 2023-09-23T03:12:52.079 INFO:teuthology.orchestra.run.smithi190.stdout:(76/139): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 63 kB 00:00 2023-09-23T03:12:52.087 INFO:teuthology.orchestra.run.smithi154.stdout:(86/139): elfutils-libelf-devel-0.189-3.el9.x86 158 kB/s | 22 kB 00:00 2023-09-23T03:12:52.105 INFO:teuthology.orchestra.run.smithi190.stdout:(77/139): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2023-09-23T03:12:52.130 INFO:teuthology.orchestra.run.smithi154.stdout:(87/139): automake-1.16.2-8.el9.noarch.rpm 2.7 MB/s | 689 kB 00:00 2023-09-23T03:12:52.147 INFO:teuthology.orchestra.run.smithi190.stdout:(78/139): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.6 MB/s | 241 kB 00:00 2023-09-23T03:12:52.171 INFO:teuthology.orchestra.run.smithi154.stdout:(88/139): jq-1.6-15.el9.x86_64.rpm 1.4 MB/s | 187 kB 00:00 2023-09-23T03:12:52.180 INFO:teuthology.orchestra.run.smithi190.stdout:(79/139): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 103 kB 00:00 2023-09-23T03:12:52.197 INFO:teuthology.orchestra.run.smithi154.stdout:(89/139): libuuid-devel-2.37.4-15.el9.x86_64.rp 386 kB/s | 26 kB 00:00 2023-09-23T03:12:52.205 INFO:teuthology.orchestra.run.smithi190.stdout:(80/139): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 124 kB 00:00 2023-09-23T03:12:52.222 INFO:teuthology.orchestra.run.smithi154.stdout:(90/139): librabbitmq-0.11.0-7.el9.x86_64.rpm 340 kB/s | 45 kB 00:00 2023-09-23T03:12:52.230 INFO:teuthology.orchestra.run.smithi190.stdout:(81/139): python3-requests-2.25.1-7.el9.noarch. 1.3 MB/s | 113 kB 00:00 2023-09-23T03:12:52.247 INFO:teuthology.orchestra.run.smithi154.stdout:(91/139): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2023-09-23T03:12:52.264 INFO:teuthology.orchestra.run.smithi190.stdout:(82/139): python3-urllib3-1.26.5-3.el9.noarch.r 3.3 MB/s | 188 kB 00:00 2023-09-23T03:12:52.289 INFO:teuthology.orchestra.run.smithi154.stdout:(92/139): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2023-09-23T03:12:52.314 INFO:teuthology.orchestra.run.smithi154.stdout:(93/139): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 947 kB/s | 86 kB 00:00 2023-09-23T03:12:52.600 INFO:teuthology.orchestra.run.smithi154.stdout:(94/139): fmt-8.1.1-5.el9.x86_64.rpm 358 kB/s | 111 kB 00:00 2023-09-23T03:12:52.648 INFO:teuthology.orchestra.run.smithi154.stdout:(95/139): gperftools-libs-2.9.1-2.el9.x86_64.rp 930 kB/s | 309 kB 00:00 2023-09-23T03:12:52.740 INFO:teuthology.orchestra.run.smithi154.stdout:(96/139): openssl-devel-3.0.7-24.el9.x86_64.rpm 7.3 MB/s | 4.1 MB 00:00 2023-09-23T03:12:52.769 INFO:teuthology.orchestra.run.smithi154.stdout:(97/139): libarrow-doc-9.0.0-9.el9.noarch.rpm 204 kB/s | 24 kB 00:00 2023-09-23T03:12:52.886 INFO:teuthology.orchestra.run.smithi154.stdout:(98/139): libunwind-1.6.2-1.el9.x86_64.rpm 1.4 MB/s | 67 kB 00:00 2023-09-23T03:12:52.986 INFO:teuthology.orchestra.run.smithi154.stdout:(99/139): parquet-libs-9.0.0-9.el9.x86_64.rpm 8.4 MB/s | 837 kB 00:00 2023-09-23T03:12:53.011 INFO:teuthology.orchestra.run.smithi154.stdout:(100/139): liboath-2.6.7-2.el9.x86_64.rpm 180 kB/s | 49 kB 00:00 2023-09-23T03:12:53.106 INFO:teuthology.orchestra.run.smithi190.stdout:(83/139): libquadmath-11.4.1-2.1.el9.x86_64.rpm 227 kB/s | 198 kB 00:00 2023-09-23T03:12:53.145 INFO:teuthology.orchestra.run.smithi154.stdout:(101/139): libarrow-9.0.0-9.el9.x86_64.rpm 8.1 MB/s | 4.4 MB 00:00 2023-09-23T03:12:53.170 INFO:teuthology.orchestra.run.smithi154.stdout:(102/139): python3-bcrypt-3.2.2-1.el9.x86_64.rp 275 kB/s | 43 kB 00:00 2023-09-23T03:12:53.196 INFO:teuthology.orchestra.run.smithi154.stdout:(103/139): python3-cachetools-4.2.4-1.el9.noarc 651 kB/s | 32 kB 00:00 2023-09-23T03:12:53.221 INFO:teuthology.orchestra.run.smithi154.stdout:(104/139): python3-certifi-2021.10.8-2.el9.noar 302 kB/s | 15 kB 00:00 2023-09-23T03:12:53.263 INFO:teuthology.orchestra.run.smithi154.stdout:(105/139): python3-asyncssh-2.13.2-3.el9.noarch 1.9 MB/s | 548 kB 00:00 2023-09-23T03:12:53.288 INFO:teuthology.orchestra.run.smithi154.stdout:(106/139): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2023-09-23T03:12:53.322 INFO:teuthology.orchestra.run.smithi154.stdout:(107/139): python3-google-auth-2.22.0-1.el9.noa 3.3 MB/s | 197 kB 00:00 2023-09-23T03:12:53.347 INFO:teuthology.orchestra.run.smithi154.stdout:(108/139): python3-jaraco-8.2.1-3.el9.noarch.rp 185 kB/s | 11 kB 00:00 2023-09-23T03:12:53.356 INFO:teuthology.orchestra.run.smithi190.stdout:(84/139): python3-cryptography-36.0.1-4.el9.x86 1.1 MB/s | 1.2 MB 00:01 2023-09-23T03:12:53.372 INFO:teuthology.orchestra.run.smithi154.stdout:(109/139): python3-jaraco-classes-3.2.1-5.el9.n 357 kB/s | 18 kB 00:00 2023-09-23T03:12:53.406 INFO:teuthology.orchestra.run.smithi154.stdout:(110/139): python3-cherrypy-18.6.1-2.el9.noarch 1.9 MB/s | 358 kB 00:00 2023-09-23T03:12:53.426 INFO:teuthology.orchestra.run.smithi190.stdout:(85/139): libgfortran-11.4.1-2.1.el9.x86_64.rpm 664 kB/s | 809 kB 00:01 2023-09-23T03:12:53.431 INFO:teuthology.orchestra.run.smithi154.stdout:(111/139): python3-jaraco-collections-3.0.0-8.e 278 kB/s | 23 kB 00:00 2023-09-23T03:12:53.457 INFO:teuthology.orchestra.run.smithi154.stdout:(112/139): python3-jaraco-functools-3.5.0-2.el9 233 kB/s | 19 kB 00:00 2023-09-23T03:12:53.465 INFO:teuthology.orchestra.run.smithi190.stdout:(86/139): automake-1.16.2-8.el9.noarch.rpm 1.9 MB/s | 689 kB 00:00 2023-09-23T03:12:53.482 INFO:teuthology.orchestra.run.smithi154.stdout:(113/139): python3-jaraco-text-3.2.0-6.el9.noar 263 kB/s | 20 kB 00:00 2023-09-23T03:12:53.507 INFO:teuthology.orchestra.run.smithi154.stdout:(114/139): python3-jwt+crypto-2.4.0-1.el9.noarc 121 kB/s | 9.0 kB 00:00 2023-09-23T03:12:53.515 INFO:teuthology.orchestra.run.smithi190.stdout:(87/139): elfutils-libelf-devel-0.189-3.el9.x86 142 kB/s | 22 kB 00:00 2023-09-23T03:12:53.533 INFO:teuthology.orchestra.run.smithi154.stdout:(115/139): python3-jwt-2.4.0-1.el9.noarch.rpm 547 kB/s | 41 kB 00:00 2023-09-23T03:12:53.540 INFO:teuthology.orchestra.run.smithi190.stdout:(88/139): librabbitmq-0.11.0-7.el9.x86_64.rpm 604 kB/s | 45 kB 00:00 2023-09-23T03:12:53.558 INFO:teuthology.orchestra.run.smithi154.stdout:(116/139): python3-logutils-0.3.5-21.el9.noarch 933 kB/s | 46 kB 00:00 2023-09-23T03:12:53.574 INFO:teuthology.orchestra.run.smithi190.stdout:(89/139): libuuid-devel-2.37.4-15.el9.x86_64.rp 443 kB/s | 26 kB 00:00 2023-09-23T03:12:53.583 INFO:teuthology.orchestra.run.smithi154.stdout:(117/139): python3-more-itertools-8.12.0-2.el9. 1.6 MB/s | 79 kB 00:00 2023-09-23T03:12:53.609 INFO:teuthology.orchestra.run.smithi154.stdout:(118/139): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2023-09-23T03:12:53.633 INFO:teuthology.orchestra.run.smithi190.stdout:(90/139): python3-pytz-2021.1-5.el9.noarch.rpm 887 kB/s | 51 kB 00:00 2023-09-23T03:12:53.666 INFO:teuthology.orchestra.run.smithi190.stdout:(91/139): jq-1.6-15.el9.x86_64.rpm 783 kB/s | 187 kB 00:00 2023-09-23T03:12:53.667 INFO:teuthology.orchestra.run.smithi154.stdout:(119/139): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2023-09-23T03:12:53.701 INFO:teuthology.orchestra.run.smithi154.stdout:(120/139): python3-portend-3.1.0-2.el9.noarch.r 180 kB/s | 16 kB 00:00 2023-09-23T03:12:53.725 INFO:teuthology.orchestra.run.smithi190.stdout:(92/139): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 946 kB/s | 86 kB 00:00 2023-09-23T03:12:53.734 INFO:teuthology.orchestra.run.smithi154.stdout:(121/139): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2023-09-23T03:12:53.742 INFO:teuthology.orchestra.run.smithi190.stdout:(93/139): zlib-devel-1.2.11-40.el9.x86_64.rpm 606 kB/s | 45 kB 00:00 2023-09-23T03:12:53.759 INFO:teuthology.orchestra.run.smithi154.stdout:(122/139): python3-repoze-lru-0.7-16.el9.noarch 532 kB/s | 31 kB 00:00 2023-09-23T03:12:53.810 INFO:teuthology.orchestra.run.smithi154.stdout:(123/139): python3-kubernetes-26.1.0-2.el9.noar 3.1 MB/s | 1.0 MB 00:00 2023-09-23T03:12:53.835 INFO:teuthology.orchestra.run.smithi154.stdout:(124/139): python3-rsa-4.9-2.el9.noarch.rpm 794 kB/s | 59 kB 00:00 2023-09-23T03:12:53.859 INFO:teuthology.orchestra.run.smithi190.stdout:(94/139): openssl-devel-3.0.7-24.el9.x86_64.rpm 13 MB/s | 4.1 MB 00:00 2023-09-23T03:12:53.860 INFO:teuthology.orchestra.run.smithi154.stdout:(125/139): python3-tempora-5.0.0-2.el9.noarch.r 722 kB/s | 36 kB 00:00 2023-09-23T03:12:53.894 INFO:teuthology.orchestra.run.smithi154.stdout:(126/139): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2023-09-23T03:12:53.919 INFO:teuthology.orchestra.run.smithi154.stdout:(127/139): python3-typing-extensions-4.4.0-2.el 609 kB/s | 51 kB 00:00 2023-09-23T03:12:53.961 INFO:teuthology.orchestra.run.smithi154.stdout:(128/139): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2023-09-23T03:12:54.005 INFO:teuthology.orchestra.run.smithi154.stdout:(129/139): python3-webob-1.8.7-6.el9.noarch.rpm 1.6 MB/s | 230 kB 00:00 2023-09-23T03:12:54.031 INFO:teuthology.orchestra.run.smithi154.stdout:(130/139): python3-zc-lockfile-2.0-10.el9.noarc 292 kB/s | 20 kB 00:00 2023-09-23T03:12:54.073 INFO:teuthology.orchestra.run.smithi154.stdout:(131/139): python3-werkzeug-2.0.3-3.el9.1.noarc 2.7 MB/s | 427 kB 00:00 2023-09-23T03:12:54.106 INFO:teuthology.orchestra.run.smithi154.stdout:(132/139): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2023-09-23T03:12:54.207 INFO:teuthology.orchestra.run.smithi154.stdout:(133/139): thrift-0.15.0-2.el9.x86_64.rpm 9.1 MB/s | 1.6 MB 00:00 2023-09-23T03:12:54.415 INFO:teuthology.orchestra.run.smithi154.stdout:(134/139): NetworkManager-1.45.2-1.el9.x86_64.r 11 MB/s | 2.2 MB 00:00 2023-09-23T03:12:54.524 INFO:teuthology.orchestra.run.smithi154.stdout:(135/139): NetworkManager-libnm-1.45.2-1.el9.x8 17 MB/s | 1.8 MB 00:00 2023-09-23T03:12:54.607 INFO:teuthology.orchestra.run.smithi154.stdout:(136/139): librados2-18.0.0-6336.g86266391.el9. 6.2 MB/s | 3.3 MB 00:00 2023-09-23T03:12:54.662 INFO:teuthology.orchestra.run.smithi154.stdout:(137/139): NetworkManager-team-1.45.2-1.el9.x86 307 kB/s | 42 kB 00:00 2023-09-23T03:12:54.758 INFO:teuthology.orchestra.run.smithi154.stdout:(138/139): NetworkManager-tui-1.45.2-1.el9.x86_ 1.6 MB/s | 247 kB 00:00 2023-09-23T03:12:54.899 INFO:teuthology.orchestra.run.smithi154.stdout:(139/139): librbd1-18.0.0-6336.g86266391.el9.x8 3.8 MB/s | 3.0 MB 00:00 2023-09-23T03:12:54.911 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2023-09-23T03:12:54.912 INFO:teuthology.orchestra.run.smithi154.stdout:Total 10 MB/s | 199 MB 00:19 2023-09-23T03:12:55.192 INFO:teuthology.orchestra.run.smithi190.stdout:(95/139): fmt-8.1.1-5.el9.x86_64.rpm 75 kB/s | 111 kB 00:01 2023-09-23T03:12:55.468 INFO:teuthology.orchestra.run.smithi190.stdout:(96/139): libarrow-doc-9.0.0-9.el9.noarch.rpm 89 kB/s | 24 kB 00:00 2023-09-23T03:12:55.613 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2023-09-23T03:12:55.626 INFO:teuthology.orchestra.run.smithi190.stdout:(97/139): gperftools-libs-2.9.1-2.el9.x86_64.rp 164 kB/s | 309 kB 00:01 2023-09-23T03:12:55.776 INFO:teuthology.orchestra.run.smithi190.stdout:(98/139): liboath-2.6.7-2.el9.x86_64.rpm 158 kB/s | 49 kB 00:00 2023-09-23T03:12:55.813 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2023-09-23T03:12:55.813 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2023-09-23T03:12:56.226 INFO:teuthology.orchestra.run.smithi190.stdout:(99/139): libunwind-1.6.2-1.el9.x86_64.rpm 112 kB/s | 67 kB 00:00 2023-09-23T03:12:56.944 INFO:teuthology.orchestra.run.smithi190.stdout:(100/139): parquet-libs-9.0.0-9.el9.x86_64.rpm 718 kB/s | 837 kB 00:01 2023-09-23T03:12:57.002 INFO:teuthology.orchestra.run.smithi190.stdout:(101/139): python3-asyncssh-2.13.2-3.el9.noarch 707 kB/s | 548 kB 00:00 2023-09-23T03:12:57.194 INFO:teuthology.orchestra.run.smithi190.stdout:(102/139): python3-bcrypt-3.2.2-1.el9.x86_64.rp 176 kB/s | 43 kB 00:00 2023-09-23T03:12:57.286 INFO:teuthology.orchestra.run.smithi190.stdout:(103/139): libarrow-9.0.0-9.el9.x86_64.rpm 1.3 MB/s | 4.4 MB 00:03 2023-09-23T03:12:57.311 INFO:teuthology.orchestra.run.smithi190.stdout:(104/139): python3-cachetools-4.2.4-1.el9.noarc 104 kB/s | 32 kB 00:00 2023-09-23T03:12:57.445 INFO:teuthology.orchestra.run.smithi190.stdout:(105/139): python3-certifi-2021.10.8-2.el9.noar 60 kB/s | 15 kB 00:00 2023-09-23T03:12:57.553 INFO:teuthology.orchestra.run.smithi190.stdout:(106/139): python3-cheroot-8.6.0-4.el9.noarch.r 646 kB/s | 172 kB 00:00 2023-09-23T03:12:57.820 INFO:teuthology.orchestra.run.smithi190.stdout:(107/139): python3-cherrypy-18.6.1-2.el9.noarch 705 kB/s | 358 kB 00:00 2023-09-23T03:12:57.845 INFO:teuthology.orchestra.run.smithi190.stdout:(108/139): python3-google-auth-2.22.0-1.el9.noa 492 kB/s | 197 kB 00:00 2023-09-23T03:12:57.871 INFO:teuthology.orchestra.run.smithi190.stdout:(109/139): python3-jaraco-8.2.1-3.el9.noarch.rp 34 kB/s | 11 kB 00:00 2023-09-23T03:12:58.006 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2023-09-23T03:12:58.007 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2023-09-23T03:12:58.113 INFO:teuthology.orchestra.run.smithi190.stdout:(110/139): python3-jaraco-classes-3.2.1-5.el9.n 73 kB/s | 18 kB 00:00 2023-09-23T03:12:58.138 INFO:teuthology.orchestra.run.smithi190.stdout:(111/139): python3-jaraco-collections-3.0.0-8.e 87 kB/s | 23 kB 00:00 2023-09-23T03:12:58.163 INFO:teuthology.orchestra.run.smithi190.stdout:(112/139): python3-jaraco-functools-3.5.0-2.el9 67 kB/s | 19 kB 00:00 2023-09-23T03:12:58.346 INFO:teuthology.orchestra.run.smithi190.stdout:(113/139): python3-jaraco-text-3.2.0-6.el9.noar 85 kB/s | 20 kB 00:00 2023-09-23T03:12:58.405 INFO:teuthology.orchestra.run.smithi190.stdout:(114/139): python3-jwt+crypto-2.4.0-1.el9.noarc 37 kB/s | 9.0 kB 00:00 2023-09-23T03:12:58.430 INFO:teuthology.orchestra.run.smithi190.stdout:(115/139): python3-jwt-2.4.0-1.el9.noarch.rpm 154 kB/s | 41 kB 00:00 2023-09-23T03:12:59.122 INFO:teuthology.orchestra.run.smithi190.stdout:(116/139): python3-kubernetes-26.1.0-2.el9.noar 1.3 MB/s | 1.0 MB 00:00 2023-09-23T03:12:59.147 INFO:teuthology.orchestra.run.smithi190.stdout:(117/139): python3-logutils-0.3.5-21.el9.noarch 62 kB/s | 46 kB 00:00 2023-09-23T03:12:59.172 INFO:teuthology.orchestra.run.smithi190.stdout:(118/139): python3-more-itertools-8.12.0-2.el9. 106 kB/s | 79 kB 00:00 2023-09-23T03:12:59.622 INFO:teuthology.orchestra.run.smithi190.stdout:(119/139): python3-natsort-7.1.1-5.el9.noarch.r 128 kB/s | 58 kB 00:00 2023-09-23T03:12:59.681 INFO:teuthology.orchestra.run.smithi190.stdout:(120/139): python3-pecan-1.4.2-3.el9.noarch.rpm 537 kB/s | 272 kB 00:00 2023-09-23T03:12:59.831 INFO:teuthology.orchestra.run.smithi190.stdout:(121/139): python3-portend-3.1.0-2.el9.noarch.r 25 kB/s | 16 kB 00:00 2023-09-23T03:13:00.073 INFO:teuthology.orchestra.run.smithi190.stdout:(122/139): python3-pyOpenSSL-21.0.0-1.el9.noarc 200 kB/s | 90 kB 00:00 2023-09-23T03:13:00.098 INFO:teuthology.orchestra.run.smithi190.stdout:(123/139): python3-repoze-lru-0.7-16.el9.noarch 74 kB/s | 31 kB 00:00 2023-09-23T03:13:00.298 INFO:teuthology.orchestra.run.smithi190.stdout:(124/139): python3-routes-2.5.1-5.el9.noarch.rp 404 kB/s | 188 kB 00:00 2023-09-23T03:13:00.349 INFO:teuthology.orchestra.run.smithi190.stdout:(125/139): python3-rsa-4.9-2.el9.noarch.rpm 237 kB/s | 59 kB 00:00 2023-09-23T03:13:00.374 INFO:teuthology.orchestra.run.smithi190.stdout:(126/139): python3-tempora-5.0.0-2.el9.noarch.r 130 kB/s | 36 kB 00:00 2023-09-23T03:13:00.580 INFO:teuthology.orchestra.run.smithi190.stdout:(127/139): python3-typing-extensions-4.4.0-2.el 181 kB/s | 51 kB 00:00 2023-09-23T03:13:00.725 INFO:teuthology.orchestra.run.smithi190.stdout:(128/139): python3-webob-1.8.7-6.el9.noarch.rpm 613 kB/s | 230 kB 00:00 2023-09-23T03:13:00.766 INFO:teuthology.orchestra.run.smithi190.stdout:(129/139): python3-websocket-client-1.2.3-2.el9 229 kB/s | 90 kB 00:00 2023-09-23T03:13:01.067 INFO:teuthology.orchestra.run.smithi190.stdout:(130/139): python3-werkzeug-2.0.3-3.el9.1.noarc 880 kB/s | 427 kB 00:00 2023-09-23T03:13:01.092 INFO:teuthology.orchestra.run.smithi190.stdout:(131/139): python3-zc-lockfile-2.0-10.el9.noarc 55 kB/s | 20 kB 00:00 2023-09-23T03:13:01.184 INFO:teuthology.orchestra.run.smithi190.stdout:(132/139): re2-20211101-3.el9.x86_64.rpm 460 kB/s | 192 kB 00:00 2023-09-23T03:13:01.892 INFO:teuthology.orchestra.run.smithi190.stdout:(133/139): librbd1-18.0.0-6336.g86266391.el9.x8 4.3 MB/s | 3.0 MB 00:00 2023-09-23T03:13:01.951 INFO:teuthology.orchestra.run.smithi190.stdout:(134/139): thrift-0.15.0-2.el9.x86_64.rpm 1.8 MB/s | 1.6 MB 00:00 2023-09-23T03:13:02.101 INFO:teuthology.orchestra.run.smithi190.stdout:(135/139): librados2-18.0.0-6336.g86266391.el9. 3.3 MB/s | 3.3 MB 00:01 2023-09-23T03:13:02.951 INFO:teuthology.orchestra.run.smithi190.stdout:(136/139): NetworkManager-team-1.45.2-1.el9.x86 50 kB/s | 42 kB 00:00 2023-09-23T03:13:03.235 INFO:teuthology.orchestra.run.smithi190.stdout:(137/139): NetworkManager-libnm-1.45.2-1.el9.x8 1.4 MB/s | 1.8 MB 00:01 2023-09-23T03:13:03.352 INFO:teuthology.orchestra.run.smithi190.stdout:(138/139): NetworkManager-1.45.2-1.el9.x86_64.r 1.5 MB/s | 2.2 MB 00:01 2023-09-23T03:13:03.885 INFO:teuthology.orchestra.run.smithi190.stdout:(139/139): NetworkManager-tui-1.45.2-1.el9.x86_ 265 kB/s | 247 kB 00:00 2023-09-23T03:13:03.901 INFO:teuthology.orchestra.run.smithi190.stdout:-------------------------------------------------------------------------------- 2023-09-23T03:13:03.901 INFO:teuthology.orchestra.run.smithi190.stdout:Total 8.9 MB/s | 199 MB 00:22 2023-09-23T03:13:03.948 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2023-09-23T03:13:04.034 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/145 2023-09-23T03:13:04.063 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/145 2023-09-23T03:13:04.099 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/145 2023-09-23T03:13:04.205 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/145 2023-09-23T03:13:04.598 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction check 2023-09-23T03:13:04.623 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 5/145 2023-09-23T03:13:04.687 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : librados2-2:18.0.0-6336.g86266391.el9.x86_64 6/145 2023-09-23T03:13:04.800 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction check succeeded. 2023-09-23T03:13:04.800 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction test 2023-09-23T03:13:04.832 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librados2-2:18.0.0-6336.g86266391.el9.x86_64 6/145 2023-09-23T03:13:04.871 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libcephfs2-2:18.0.0-6336.g86266391.el9.x86_64 7/145 2023-09-23T03:13:04.996 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libcephfs2-2:18.0.0-6336.g86266391.el9.x86_64 7/145 2023-09-23T03:13:05.065 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rados-2:18.0.0-6336.g86266391.el9.x86_64 8/145 2023-09-23T03:13:05.178 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 9/145 2023-09-23T03:13:05.241 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-urllib3-1.26.5-3.el9.noarch 10/145 2023-09-23T03:13:05.295 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 11/145 2023-09-23T03:13:05.534 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 12/145 2023-09-23T03:13:05.611 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-13.11-1.el9.x86_64 13/145 2023-09-23T03:13:05.635 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libcephsqlite-2:18.0.0-6336.g86266391.el9.x86_64 14/145 2023-09-23T03:13:05.783 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6336.g86266391.el9.x86_64 14/145 2023-09-23T03:13:05.820 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libradosstriper1-2:18.0.0-6336.g86266391.el9.x86 15/145 2023-09-23T03:13:06.273 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6336.g86266391.el9.x86 15/145 2023-09-23T03:13:06.433 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : NetworkManager-libnm-1:1.45.2-1.el9.x86_64 16/145 2023-09-23T03:13:06.514 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.2-1.el9.x86_64 16/145 2023-09-23T03:13:06.723 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-1:1.45.2-1.el9.x86_64 17/145 2023-09-23T03:13:06.811 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : NetworkManager-1:1.45.2-1.el9.x86_64 17/145 2023-09-23T03:13:06.983 INFO:teuthology.orchestra.run.smithi190.stdout:Transaction test succeeded. 2023-09-23T03:13:06.983 INFO:teuthology.orchestra.run.smithi190.stdout:Running transaction 2023-09-23T03:13:07.261 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-1:1.45.2-1.el9.x86_64 17/145 2023-09-23T03:13:07.317 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/145 2023-09-23T03:13:07.365 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/145 2023-09-23T03:13:07.403 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/145 2023-09-23T03:13:07.489 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libquadmath-11.4.1-2.1.el9.x86_64 21/145 2023-09-23T03:13:07.592 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libgfortran-11.4.1-2.1.el9.x86_64 22/145 2023-09-23T03:13:07.646 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/145 2023-09-23T03:13:07.688 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/145 2023-09-23T03:13:07.809 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/145 2023-09-23T03:13:07.859 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-ceph-common-2:18.0.0-6336.g86266391.el9. 26/145 2023-09-23T03:13:07.911 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-ceph-argparse-2:18.0.0-6336.g86266391.el 27/145 2023-09-23T03:13:08.053 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cephfs-2:18.0.0-6336.g86266391.el9.x86_6 28/145 2023-09-23T03:13:08.235 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/145 2023-09-23T03:13:08.303 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 30/145 2023-09-23T03:13:08.983 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rsa-4.9-2.el9.noarch 31/145 2023-09-23T03:13:09.201 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 32/145 2023-09-23T03:13:09.289 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 33/145 2023-09-23T03:13:09.372 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librados-devel-2:18.0.0-6336.g86266391.el9.x86_6 34/145 2023-09-23T03:13:09.421 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 35/145 2023-09-23T03:13:09.632 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : re2-1:20211101-3.el9.x86_64 36/145 2023-09-23T03:13:09.677 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 37/145 2023-09-23T03:13:09.750 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 38/145 2023-09-23T03:13:09.858 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 39/145 2023-09-23T03:13:09.893 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 40/145 2023-09-23T03:13:09.933 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 41/145 2023-09-23T03:13:10.019 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 42/145 2023-09-23T03:13:10.085 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 43/145 2023-09-23T03:13:10.152 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 44/145 2023-09-23T03:13:10.340 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 45/145 2023-09-23T03:13:10.391 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 46/145 2023-09-23T03:13:10.436 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 47/145 2023-09-23T03:13:10.599 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 48/145 2023-09-23T03:13:10.648 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-google-auth-1:2.22.0-1.el9.noarch 49/145 2023-09-23T03:13:10.723 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 50/145 2023-09-23T03:13:10.758 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 51/145 2023-09-23T03:13:10.798 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 52/145 2023-09-23T03:13:10.841 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : fmt-8.1.1-5.el9.x86_64 53/145 2023-09-23T03:13:10.895 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 54/145 2023-09-23T03:13:11.046 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 55/145 2023-09-23T03:13:11.106 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : automake-1.16.2-8.el9.noarch 56/145 2023-09-23T03:13:11.175 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-ply-3.11-14.el9.noarch 57/145 2023-09-23T03:13:11.257 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 58/145 2023-09-23T03:13:11.545 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 59/145 2023-09-23T03:13:11.610 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 60/145 2023-09-23T03:13:11.784 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 61/145 2023-09-23T03:13:11.862 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 62/145 2023-09-23T03:13:11.912 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 63/145 2023-09-23T03:13:11.961 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 64/145 2023-09-23T03:13:12.038 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 65/145 2023-09-23T03:13:12.093 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : fuse-2.9.9-15.el9.x86_64 66/145 2023-09-23T03:13:12.155 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 67/145 2023-09-23T03:13:12.245 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : uuid-1.6.2-55.el9.x86_64 68/145 2023-09-23T03:13:12.305 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 69/145 2023-09-23T03:13:12.347 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 70/145 2023-09-23T03:13:12.473 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 71/145 2023-09-23T03:13:12.547 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 72/145 2023-09-23T03:13:12.596 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/145 2023-09-23T03:13:12.672 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/145 2023-09-23T03:13:12.717 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/145 2023-09-23T03:13:12.756 INFO:teuthology.orchestra.run.smithi190.stdout: Preparing : 1/1 2023-09-23T03:13:12.866 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/145 2023-09-23T03:13:12.895 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/145 2023-09-23T03:13:12.913 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 77/145 2023-09-23T03:13:12.933 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/145 2023-09-23T03:13:12.971 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/145 2023-09-23T03:13:13.058 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 78/145 2023-09-23T03:13:13.078 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/145 2023-09-23T03:13:13.127 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/145 2023-09-23T03:13:13.490 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 5/145 2023-09-23T03:13:13.539 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : librados2-2:18.0.0-6336.g86266391.el9.x86_64 6/145 2023-09-23T03:13:13.692 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librados2-2:18.0.0-6336.g86266391.el9.x86_64 6/145 2023-09-23T03:13:13.732 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephfs2-2:18.0.0-6336.g86266391.el9.x86_64 7/145 2023-09-23T03:13:13.856 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libcephfs2-2:18.0.0-6336.g86266391.el9.x86_64 7/145 2023-09-23T03:13:13.962 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rados-2:18.0.0-6336.g86266391.el9.x86_64 8/145 2023-09-23T03:13:14.075 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 9/145 2023-09-23T03:13:14.125 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/145 2023-09-23T03:13:14.152 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-urllib3-1.26.5-3.el9.noarch 10/145 2023-09-23T03:13:14.185 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/145 2023-09-23T03:13:14.192 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 11/145 2023-09-23T03:13:14.226 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/145 2023-09-23T03:13:14.354 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/145 2023-09-23T03:13:14.393 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 84/145 2023-09-23T03:13:14.417 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 12/145 2023-09-23T03:13:14.487 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-13.11-1.el9.x86_64 13/145 2023-09-23T03:13:14.513 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephsqlite-2:18.0.0-6336.g86266391.el9.x86_64 14/145 2023-09-23T03:13:14.643 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : openblas-0.3.21-2.el9.x86_64 85/145 2023-09-23T03:13:14.667 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libcephsqlite-2:18.0.0-6336.g86266391.el9.x86_64 14/145 2023-09-23T03:13:14.674 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 86/145 2023-09-23T03:13:14.697 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libradosstriper1-2:18.0.0-6336.g86266391.el9.x86 15/145 2023-09-23T03:13:14.779 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 87/145 2023-09-23T03:13:15.081 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: libradosstriper1-2:18.0.0-6336.g86266391.el9.x86 15/145 2023-09-23T03:13:15.235 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : NetworkManager-libnm-1:1.45.2-1.el9.x86_64 16/145 2023-09-23T03:13:15.325 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.2-1.el9.x86_64 16/145 2023-09-23T03:13:15.570 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-1:1.45.2-1.el9.x86_64 17/145 2023-09-23T03:13:15.602 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 88/145 2023-09-23T03:13:15.631 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : NetworkManager-1:1.45.2-1.el9.x86_64 17/145 2023-09-23T03:13:15.745 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 89/145 2023-09-23T03:13:16.016 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-1:1.45.2-1.el9.x86_64 17/145 2023-09-23T03:13:16.061 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/145 2023-09-23T03:13:16.110 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/145 2023-09-23T03:13:16.150 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/145 2023-09-23T03:13:16.211 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libquadmath-11.4.1-2.1.el9.x86_64 21/145 2023-09-23T03:13:16.325 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libgfortran-11.4.1-2.1.el9.x86_64 22/145 2023-09-23T03:13:16.375 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/145 2023-09-23T03:13:16.409 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/145 2023-09-23T03:13:16.522 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/145 2023-09-23T03:13:16.571 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ceph-common-2:18.0.0-6336.g86266391.el9. 26/145 2023-09-23T03:13:16.633 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ceph-argparse-2:18.0.0-6336.g86266391.el 27/145 2023-09-23T03:13:16.749 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cephfs-2:18.0.0-6336.g86266391.el9.x86_6 28/145 2023-09-23T03:13:16.937 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/145 2023-09-23T03:13:17.017 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 30/145 2023-09-23T03:13:17.348 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 90/145 2023-09-23T03:13:17.445 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 91/145 2023-09-23T03:13:17.482 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 92/145 2023-09-23T03:13:17.520 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : jq-1.6-15.el9.x86_64 93/145 2023-09-23T03:13:17.569 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/145 2023-09-23T03:13:17.748 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rsa-4.9-2.el9.noarch 31/145 2023-09-23T03:13:17.846 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 32/145 2023-09-23T03:13:17.892 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 33/145 2023-09-23T03:13:17.938 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librados-devel-2:18.0.0-6336.g86266391.el9.x86_6 34/145 2023-09-23T03:13:17.978 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 95/145 2023-09-23T03:13:18.018 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : librbd1-2:18.0.0-6336.g86266391.el9.x86_64 96/145 2023-09-23T03:13:18.018 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 35/145 2023-09-23T03:13:18.172 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librbd1-2:18.0.0-6336.g86266391.el9.x86_64 96/145 2023-09-23T03:13:18.195 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : re2-1:20211101-3.el9.x86_64 36/145 2023-09-23T03:13:18.216 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rbd-2:18.0.0-6336.g86266391.el9.x86_64 97/145 2023-09-23T03:13:18.240 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 37/145 2023-09-23T03:13:18.276 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 98/145 2023-09-23T03:13:18.310 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 38/145 2023-09-23T03:13:18.404 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 39/145 2023-09-23T03:13:18.440 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 40/145 2023-09-23T03:13:18.478 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 41/145 2023-09-23T03:13:18.545 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 42/145 2023-09-23T03:13:18.603 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 43/145 2023-09-23T03:13:18.671 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 44/145 2023-09-23T03:13:18.768 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 99/145 2023-09-23T03:13:18.795 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librgw2-2:18.0.0-6336.g86266391.el9.x86_64 100/145 2023-09-23T03:13:18.866 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 45/145 2023-09-23T03:13:18.886 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librgw2-2:18.0.0-6336.g86266391.el9.x86_64 100/145 2023-09-23T03:13:18.914 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rgw-2:18.0.0-6336.g86266391.el9.x86_64 101/145 2023-09-23T03:13:18.931 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 46/145 2023-09-23T03:13:18.976 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 47/145 2023-09-23T03:13:19.140 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 48/145 2023-09-23T03:13:19.181 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-google-auth-1:2.22.0-1.el9.noarch 49/145 2023-09-23T03:13:19.231 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 50/145 2023-09-23T03:13:19.271 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 51/145 2023-09-23T03:13:19.309 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 52/145 2023-09-23T03:13:19.352 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : fmt-8.1.1-5.el9.x86_64 53/145 2023-09-23T03:13:19.397 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 54/145 2023-09-23T03:13:19.552 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 55/145 2023-09-23T03:13:19.613 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : automake-1.16.2-8.el9.noarch 56/145 2023-09-23T03:13:19.695 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-ply-3.11-14.el9.noarch 57/145 2023-09-23T03:13:19.809 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 58/145 2023-09-23T03:13:20.105 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 59/145 2023-09-23T03:13:20.166 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 60/145 2023-09-23T03:13:20.345 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 61/145 2023-09-23T03:13:20.393 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 62/145 2023-09-23T03:13:20.417 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 63/145 2023-09-23T03:13:20.455 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 64/145 2023-09-23T03:13:20.498 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 65/145 2023-09-23T03:13:20.550 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : fuse-2.9.9-15.el9.x86_64 66/145 2023-09-23T03:13:20.609 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 67/145 2023-09-23T03:13:20.698 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : uuid-1.6.2-55.el9.x86_64 68/145 2023-09-23T03:13:20.752 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 69/145 2023-09-23T03:13:20.798 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 70/145 2023-09-23T03:13:20.927 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 71/145 2023-09-23T03:13:20.974 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 72/145 2023-09-23T03:13:21.017 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/145 2023-09-23T03:13:21.070 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/145 2023-09-23T03:13:21.112 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/145 2023-09-23T03:13:21.267 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/145 2023-09-23T03:13:21.311 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 77/145 2023-09-23T03:13:21.455 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 78/145 2023-09-23T03:13:21.534 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/145 2023-09-23T03:13:21.647 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-common-2:18.0.0-6336.g86266391.el9.x86_64 102/145 2023-09-23T03:13:21.694 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-common-2:18.0.0-6336.g86266391.el9.x86_64 102/145 2023-09-23T03:13:22.322 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-common-2:18.0.0-6336.g86266391.el9.x86_64 102/145 2023-09-23T03:13:22.363 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-base-2:18.0.0-6336.g86266391.el9.x86_64 103/145 2023-09-23T03:13:22.549 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/145 2023-09-23T03:13:22.551 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-base-2:18.0.0-6336.g86266391.el9.x86_64 103/145 2023-09-23T03:13:22.551 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-09-23T03:13:22.551 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-09-23T03:13:22.551 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:13:22.599 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/145 2023-09-23T03:13:22.606 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-selinux-2:18.0.0-6336.g86266391.el9.x86_64 104/145 2023-09-23T03:13:22.633 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/145 2023-09-23T03:13:22.758 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/145 2023-09-23T03:13:22.808 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 84/145 2023-09-23T03:13:23.054 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openblas-0.3.21-2.el9.x86_64 85/145 2023-09-23T03:13:23.080 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 86/145 2023-09-23T03:13:23.194 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 87/145 2023-09-23T03:13:24.024 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 88/145 2023-09-23T03:13:24.171 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 89/145 2023-09-23T03:13:25.758 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 90/145 2023-09-23T03:13:25.854 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 91/145 2023-09-23T03:13:25.892 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 92/145 2023-09-23T03:13:25.923 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : jq-1.6-15.el9.x86_64 93/145 2023-09-23T03:13:25.963 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/145 2023-09-23T03:13:26.329 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 95/145 2023-09-23T03:13:26.362 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : librbd1-2:18.0.0-6336.g86266391.el9.x86_64 96/145 2023-09-23T03:13:26.466 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librbd1-2:18.0.0-6336.g86266391.el9.x86_64 96/145 2023-09-23T03:13:26.501 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rbd-2:18.0.0-6336.g86266391.el9.x86_64 97/145 2023-09-23T03:13:26.566 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 98/145 2023-09-23T03:13:27.066 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 99/145 2023-09-23T03:13:27.098 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : librgw2-2:18.0.0-6336.g86266391.el9.x86_64 100/145 2023-09-23T03:13:27.239 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librgw2-2:18.0.0-6336.g86266391.el9.x86_64 100/145 2023-09-23T03:13:27.292 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : python3-rgw-2:18.0.0-6336.g86266391.el9.x86_64 101/145 2023-09-23T03:13:30.122 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-common-2:18.0.0-6336.g86266391.el9.x86_64 102/145 2023-09-23T03:13:30.163 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-common-2:18.0.0-6336.g86266391.el9.x86_64 102/145 2023-09-23T03:13:30.793 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-common-2:18.0.0-6336.g86266391.el9.x86_64 102/145 2023-09-23T03:13:30.834 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-base-2:18.0.0-6336.g86266391.el9.x86_64 103/145 2023-09-23T03:13:31.068 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-base-2:18.0.0-6336.g86266391.el9.x86_64 103/145 2023-09-23T03:13:31.068 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-09-23T03:13:31.069 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-09-23T03:13:31.069 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:13:31.110 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-selinux-2:18.0.0-6336.g86266391.el9.x86_64 104/145 2023-09-23T03:13:35.364 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6336.g86266391.el9.x86_64 104/145 2023-09-23T03:13:35.365 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /sys 2023-09-23T03:13:35.365 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /proc 2023-09-23T03:13:35.365 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /mnt 2023-09-23T03:13:35.365 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /var/tmp 2023-09-23T03:13:35.365 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /home 2023-09-23T03:13:35.365 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /root 2023-09-23T03:13:35.365 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /tmp 2023-09-23T03:13:35.365 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:13:35.585 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-volume-2:18.0.0-6336.g86266391.el9.noarch 105/145 2023-09-23T03:13:35.632 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-volume-2:18.0.0-6336.g86266391.el9.noarch 105/145 2023-09-23T03:13:35.633 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-09-23T03:13:35.633 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-09-23T03:13:35.633 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:13:37.295 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-osd-2:18.0.0-6336.g86266391.el9.x86_64 106/145 2023-09-23T03:13:37.411 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-osd-2:18.0.0-6336.g86266391.el9.x86_64 106/145 2023-09-23T03:13:37.411 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-09-23T03:13:37.411 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-09-23T03:13:37.412 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-09-23T03:13:37.412 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-09-23T03:13:37.412 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:13:37.629 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mds-2:18.0.0-6336.g86266391.el9.x86_64 107/145 2023-09-23T03:13:37.715 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mds-2:18.0.0-6336.g86266391.el9.x86_64 107/145 2023-09-23T03:13:37.715 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-09-23T03:13:37.715 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-09-23T03:13:37.715 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-09-23T03:13:37.715 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-09-23T03:13:37.715 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:13:38.221 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mon-2:18.0.0-6336.g86266391.el9.x86_64 108/145 2023-09-23T03:13:38.293 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mon-2:18.0.0-6336.g86266391.el9.x86_64 108/145 2023-09-23T03:13:38.293 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-09-23T03:13:38.293 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-09-23T03:13:38.293 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-09-23T03:13:38.293 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-09-23T03:13:38.293 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:13:39.004 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: cephadm-2:18.0.0-6336.g86266391.el9.noarch 109/145 2023-09-23T03:13:39.041 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephadm-2:18.0.0-6336.g86266391.el9.noarch 109/145 2023-09-23T03:13:39.104 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6336.g86266391.e 110/145 2023-09-23T03:13:39.220 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6336.g86266391. 111/145 2023-09-23T03:13:39.260 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6336.g86266391.el9.noa 112/145 2023-09-23T03:13:39.583 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6336.g86266391.el9.noa 112/145 2023-09-23T03:13:39.619 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6336.g86266391.el9.n 113/145 2023-09-23T03:13:40.767 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6336.g86266391.el9.n 113/145 2023-09-23T03:13:40.802 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6336.g862 114/145 2023-09-23T03:13:40.944 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6336.g862 114/145 2023-09-23T03:13:41.121 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6336.g86266391.el 115/145 2023-09-23T03:13:41.163 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-2:18.0.0-6336.g86266391.el9.x86_64 116/145 2023-09-23T03:13:41.238 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6336.g86266391.el9.x86_64 116/145 2023-09-23T03:13:41.238 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-09-23T03:13:41.238 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-09-23T03:13:41.238 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-09-23T03:13:41.238 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-09-23T03:13:41.239 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:13:41.297 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-rook-2:18.0.0-6336.g86266391.el9.noarch 117/145 2023-09-23T03:13:41.337 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6336.g86266391.el9.noarch 117/145 2023-09-23T03:13:42.227 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-2:18.0.0-6336.g86266391.el9.x86_64 118/145 2023-09-23T03:13:42.281 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-radosgw-2:18.0.0-6336.g86266391.el9.x86_64 119/145 2023-09-23T03:13:42.357 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6336.g86266391.el9.x86_64 119/145 2023-09-23T03:13:42.357 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-09-23T03:13:42.357 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-09-23T03:13:42.357 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-09-23T03:13:42.357 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-09-23T03:13:42.357 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:13:42.415 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6336.g86266 120/145 2023-09-23T03:13:42.492 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6336.g86266 120/145 2023-09-23T03:13:42.492 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-09-23T03:13:42.492 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-09-23T03:13:42.492 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:13:42.566 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephfs-mirror-2:18.0.0-6336.g86266391.el9.x86_64 121/145 2023-09-23T03:13:42.682 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6336.g86266391.el9.x86_64 121/145 2023-09-23T03:13:42.683 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-09-23T03:13:42.683 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-09-23T03:13:42.683 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-09-23T03:13:42.683 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-09-23T03:13:42.683 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:13:43.075 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : rbd-mirror-2:18.0.0-6336.g86266391.el9.x86_64 122/145 2023-09-23T03:13:43.150 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6336.g86266391.el9.x86_64 122/145 2023-09-23T03:13:43.150 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-09-23T03:13:43.151 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-09-23T03:13:43.151 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-09-23T03:13:43.151 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-09-23T03:13:43.151 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:13:43.624 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-selinux-2:18.0.0-6336.g86266391.el9.x86_64 104/145 2023-09-23T03:13:43.624 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /sys 2023-09-23T03:13:43.624 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /proc 2023-09-23T03:13:43.624 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /mnt 2023-09-23T03:13:43.624 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /var/tmp 2023-09-23T03:13:43.624 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /home 2023-09-23T03:13:43.625 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /root 2023-09-23T03:13:43.625 INFO:teuthology.orchestra.run.smithi190.stdout:skipping the directory /tmp 2023-09-23T03:13:43.625 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:13:43.840 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-volume-2:18.0.0-6336.g86266391.el9.noarch 105/145 2023-09-23T03:13:43.891 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-volume-2:18.0.0-6336.g86266391.el9.noarch 105/145 2023-09-23T03:13:43.891 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-09-23T03:13:43.891 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-09-23T03:13:43.891 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:13:45.585 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-osd-2:18.0.0-6336.g86266391.el9.x86_64 106/145 2023-09-23T03:13:45.665 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-osd-2:18.0.0-6336.g86266391.el9.x86_64 106/145 2023-09-23T03:13:45.665 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-09-23T03:13:45.665 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-09-23T03:13:45.665 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-09-23T03:13:45.665 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-09-23T03:13:45.665 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:13:45.893 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mds-2:18.0.0-6336.g86266391.el9.x86_64 107/145 2023-09-23T03:13:45.979 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mds-2:18.0.0-6336.g86266391.el9.x86_64 107/145 2023-09-23T03:13:45.979 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-09-23T03:13:45.979 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-09-23T03:13:45.979 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-09-23T03:13:45.979 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-09-23T03:13:45.979 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:13:46.477 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mon-2:18.0.0-6336.g86266391.el9.x86_64 108/145 2023-09-23T03:13:46.554 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mon-2:18.0.0-6336.g86266391.el9.x86_64 108/145 2023-09-23T03:13:46.555 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-09-23T03:13:46.555 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-09-23T03:13:46.555 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-09-23T03:13:46.555 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-09-23T03:13:46.555 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:13:47.217 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: cephadm-2:18.0.0-6336.g86266391.el9.noarch 109/145 2023-09-23T03:13:47.264 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephadm-2:18.0.0-6336.g86266391.el9.noarch 109/145 2023-09-23T03:13:47.319 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-6336.g86266391.e 110/145 2023-09-23T03:13:47.381 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-test-2:18.0.0-6336.g86266391.el9.x86_64 123/145 2023-09-23T03:13:47.434 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-6336.g86266391. 111/145 2023-09-23T03:13:47.446 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : rbd-fuse-2:18.0.0-6336.g86266391.el9.x86_64 124/145 2023-09-23T03:13:47.466 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-6336.g86266391.el9.noa 112/145 2023-09-23T03:13:47.495 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : rbd-nbd-2:18.0.0-6336.g86266391.el9.x86_64 125/145 2023-09-23T03:13:47.767 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : elfutils-libelf-devel-0.189-3.el9.x86_64 126/145 2023-09-23T03:13:47.781 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-6336.g86266391.el9.noa 112/145 2023-09-23T03:13:47.817 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-6336.g86266391.el9.n 113/145 2023-09-23T03:13:47.895 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 127/145 2023-09-23T03:13:47.964 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-fuse-2:18.0.0-6336.g86266391.el9.x86_64 128/145 2023-09-23T03:13:48.013 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libtool-2.4.6-45.el9.x86_64 129/145 2023-09-23T03:13:48.051 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 130/145 2023-09-23T03:13:48.081 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libcephfs-devel-2:18.0.0-6336.g86266391.el9.x86_ 131/145 2023-09-23T03:13:48.128 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : NetworkManager-team-1:1.45.2-1.el9.x86_64 132/145 2023-09-23T03:13:48.174 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : NetworkManager-tui-1:1.45.2-1.el9.x86_64 133/145 2023-09-23T03:13:48.939 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-6336.g86266391.el9.n 113/145 2023-09-23T03:13:48.976 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-6336.g862 114/145 2023-09-23T03:13:49.088 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 134/145 2023-09-23T03:13:49.121 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-6336.g862 114/145 2023-09-23T03:13:49.130 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 134/145 2023-09-23T03:13:49.187 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 134/145 2023-09-23T03:13:49.293 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-6336.g86266391.el 115/145 2023-09-23T03:13:49.328 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-2:18.0.0-6336.g86266391.el9.x86_64 116/145 2023-09-23T03:13:49.404 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-2:18.0.0-6336.g86266391.el9.x86_64 116/145 2023-09-23T03:13:49.404 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-09-23T03:13:49.405 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-09-23T03:13:49.405 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-09-23T03:13:49.405 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-09-23T03:13:49.405 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:13:49.462 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-mgr-rook-2:18.0.0-6336.g86266391.el9.noarch 117/145 2023-09-23T03:13:49.501 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-6336.g86266391.el9.noarch 117/145 2023-09-23T03:13:50.427 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-2:18.0.0-6336.g86266391.el9.x86_64 118/145 2023-09-23T03:13:50.463 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-radosgw-2:18.0.0-6336.g86266391.el9.x86_64 119/145 2023-09-23T03:13:50.494 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephfs-top-2:18.0.0-6336.g86266391.el9.noarch 135/145 2023-09-23T03:13:50.540 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-6336.g86266391.el9.x86_64 119/145 2023-09-23T03:13:50.540 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-09-23T03:13:50.540 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-09-23T03:13:50.540 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-09-23T03:13:50.540 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-09-23T03:13:50.541 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:13:50.597 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-6336.g86266 120/145 2023-09-23T03:13:50.601 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : openssl-devel-1:3.0.7-24.el9.x86_64 136/145 2023-09-23T03:13:50.650 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 137/145 2023-09-23T03:13:50.671 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-6336.g86266 120/145 2023-09-23T03:13:50.671 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-09-23T03:13:50.671 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-09-23T03:13:50.671 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:13:50.748 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephfs-mirror-2:18.0.0-6336.g86266391.el9.x86_64 121/145 2023-09-23T03:13:50.841 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-6336.g86266391.el9.x86_64 121/145 2023-09-23T03:13:50.841 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-09-23T03:13:50.841 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-09-23T03:13:50.841 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-09-23T03:13:50.841 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-09-23T03:13:50.842 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:13:50.906 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flex-2.6.4-9.el9.x86_64 138/145 2023-09-23T03:13:51.197 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : bison-3.7.4-5.el9.x86_64 139/145 2023-09-23T03:13:51.199 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-mirror-2:18.0.0-6336.g86266391.el9.x86_64 122/145 2023-09-23T03:13:51.239 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 140/145 2023-09-23T03:13:51.239 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 141/145 2023-09-23T03:13:51.275 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: rbd-mirror-2:18.0.0-6336.g86266391.el9.x86_64 122/145 2023-09-23T03:13:51.275 INFO:teuthology.orchestra.run.smithi190.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-09-23T03:13:51.276 INFO:teuthology.orchestra.run.smithi190.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-09-23T03:13:51.276 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-09-23T03:13:51.276 INFO:teuthology.orchestra.run.smithi190.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-09-23T03:13:51.276 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:13:51.392 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 141/145 2023-09-23T03:13:51.425 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 142/145 2023-09-23T03:13:51.464 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 143/145 2023-09-23T03:13:51.464 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 143/145 2023-09-23T03:13:51.799 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 143/145 2023-09-23T03:13:51.799 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 144/145 2023-09-23T03:13:51.922 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 144/145 2023-09-23T03:13:51.922 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 145/145 2023-09-23T03:13:52.441 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 145/145 2023-09-23T03:13:52.441 INFO:teuthology.orchestra.run.smithi154.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-09-23T03:13:52.441 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:13:55.456 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-test-2:18.0.0-6336.g86266391.el9.x86_64 123/145 2023-09-23T03:13:55.516 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-fuse-2:18.0.0-6336.g86266391.el9.x86_64 124/145 2023-09-23T03:13:55.553 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : rbd-nbd-2:18.0.0-6336.g86266391.el9.x86_64 125/145 2023-09-23T03:13:55.827 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : elfutils-libelf-devel-0.189-3.el9.x86_64 126/145 2023-09-23T03:13:55.945 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 127/145 2023-09-23T03:13:56.013 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : ceph-fuse-2:18.0.0-6336.g86266391.el9.x86_64 128/145 2023-09-23T03:13:56.062 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libtool-2.4.6-45.el9.x86_64 129/145 2023-09-23T03:13:56.100 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 130/145 2023-09-23T03:13:56.131 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libcephfs-devel-2:18.0.0-6336.g86266391.el9.x86_ 131/145 2023-09-23T03:13:56.177 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : NetworkManager-team-1:1.45.2-1.el9.x86_64 132/145 2023-09-23T03:13:56.224 INFO:teuthology.orchestra.run.smithi190.stdout: Upgrading : NetworkManager-tui-1:1.45.2-1.el9.x86_64 133/145 2023-09-23T03:13:56.761 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-2:18.0.0-6336.g86266391.el9.x86_64 1/145 2023-09-23T03:13:56.761 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-base-2:18.0.0-6336.g86266391.el9.x86_64 2/145 2023-09-23T03:13:56.762 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-common-2:18.0.0-6336.g86266391.el9.x86_64 3/145 2023-09-23T03:13:56.762 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-fuse-2:18.0.0-6336.g86266391.el9.x86_64 4/145 2023-09-23T03:13:56.762 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6336.g86266 5/145 2023-09-23T03:13:56.762 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mds-2:18.0.0-6336.g86266391.el9.x86_64 6/145 2023-09-23T03:13:56.762 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-2:18.0.0-6336.g86266391.el9.x86_64 7/145 2023-09-23T03:13:56.762 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mon-2:18.0.0-6336.g86266391.el9.x86_64 8/145 2023-09-23T03:13:56.762 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-osd-2:18.0.0-6336.g86266391.el9.x86_64 9/145 2023-09-23T03:13:56.762 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-radosgw-2:18.0.0-6336.g86266391.el9.x86_64 10/145 2023-09-23T03:13:56.762 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-selinux-2:18.0.0-6336.g86266391.el9.x86_64 11/145 2023-09-23T03:13:56.762 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-test-2:18.0.0-6336.g86266391.el9.x86_64 12/145 2023-09-23T03:13:56.762 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephfs-mirror-2:18.0.0-6336.g86266391.el9.x86_64 13/145 2023-09-23T03:13:56.763 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libcephfs-devel-2:18.0.0-6336.g86266391.el9.x86_ 14/145 2023-09-23T03:13:56.763 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libcephfs2-2:18.0.0-6336.g86266391.el9.x86_64 15/145 2023-09-23T03:13:56.763 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libcephsqlite-2:18.0.0-6336.g86266391.el9.x86_64 16/145 2023-09-23T03:13:56.763 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librados-devel-2:18.0.0-6336.g86266391.el9.x86_6 17/145 2023-09-23T03:13:56.763 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libradosstriper1-2:18.0.0-6336.g86266391.el9.x86 18/145 2023-09-23T03:13:56.763 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librgw2-2:18.0.0-6336.g86266391.el9.x86_64 19/145 2023-09-23T03:13:56.763 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6336.g86266391.el 20/145 2023-09-23T03:13:56.763 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-ceph-common-2:18.0.0-6336.g86266391.el9. 21/145 2023-09-23T03:13:56.763 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cephfs-2:18.0.0-6336.g86266391.el9.x86_6 22/145 2023-09-23T03:13:56.763 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rados-2:18.0.0-6336.g86266391.el9.x86_64 23/145 2023-09-23T03:13:56.763 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rbd-2:18.0.0-6336.g86266391.el9.x86_64 24/145 2023-09-23T03:13:56.763 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rgw-2:18.0.0-6336.g86266391.el9.x86_64 25/145 2023-09-23T03:13:56.764 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : rbd-fuse-2:18.0.0-6336.g86266391.el9.x86_64 26/145 2023-09-23T03:13:56.764 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : rbd-mirror-2:18.0.0-6336.g86266391.el9.x86_64 27/145 2023-09-23T03:13:56.774 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : rbd-nbd-2:18.0.0-6336.g86266391.el9.x86_64 28/145 2023-09-23T03:13:56.775 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6336.g86266391. 29/145 2023-09-23T03:13:56.775 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6336.g86266391.el9.noa 30/145 2023-09-23T03:13:56.775 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6336.g86266391.el9.n 31/145 2023-09-23T03:13:56.775 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6336.g862 32/145 2023-09-23T03:13:56.775 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6336.g86266391.el 33/145 2023-09-23T03:13:56.775 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6336.g86266391.el9.noarch 34/145 2023-09-23T03:13:56.775 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6336.g86266391.e 35/145 2023-09-23T03:13:56.775 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-volume-2:18.0.0-6336.g86266391.el9.noarch 36/145 2023-09-23T03:13:56.775 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephadm-2:18.0.0-6336.g86266391.el9.noarch 37/145 2023-09-23T03:13:56.775 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephfs-top-2:18.0.0-6336.g86266391.el9.noarch 38/145 2023-09-23T03:13:56.776 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : bison-3.7.4-5.el9.x86_64 39/145 2023-09-23T03:13:56.776 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 40/145 2023-09-23T03:13:56.776 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flex-2.6.4-9.el9.x86_64 41/145 2023-09-23T03:13:56.776 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 42/145 2023-09-23T03:13:56.776 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 43/145 2023-09-23T03:13:56.776 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 44/145 2023-09-23T03:13:56.776 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 45/145 2023-09-23T03:13:56.776 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 46/145 2023-09-23T03:13:56.776 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 47/145 2023-09-23T03:13:56.776 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 48/145 2023-09-23T03:13:56.776 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 49/145 2023-09-23T03:13:56.776 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 50/145 2023-09-23T03:13:56.777 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 51/145 2023-09-23T03:13:56.777 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 52/145 2023-09-23T03:13:56.777 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 53/145 2023-09-23T03:13:56.777 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-13.11-1.el9.x86_64 54/145 2023-09-23T03:13:56.777 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 55/145 2023-09-23T03:13:56.777 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 56/145 2023-09-23T03:13:56.777 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 57/145 2023-09-23T03:13:56.777 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 58/145 2023-09-23T03:13:56.777 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 59/145 2023-09-23T03:13:56.777 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 60/145 2023-09-23T03:13:56.777 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/145 2023-09-23T03:13:56.778 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/145 2023-09-23T03:13:56.778 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/145 2023-09-23T03:13:56.778 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/145 2023-09-23T03:13:56.778 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/145 2023-09-23T03:13:56.778 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/145 2023-09-23T03:13:56.778 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/145 2023-09-23T03:13:56.778 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 68/145 2023-09-23T03:13:56.778 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 69/145 2023-09-23T03:13:56.778 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 70/145 2023-09-23T03:13:56.778 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 71/145 2023-09-23T03:13:56.778 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 72/145 2023-09-23T03:13:56.778 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 73/145 2023-09-23T03:13:56.779 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 74/145 2023-09-23T03:13:56.779 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 75/145 2023-09-23T03:13:56.779 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 76/145 2023-09-23T03:13:56.779 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 77/145 2023-09-23T03:13:56.779 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 78/145 2023-09-23T03:13:56.779 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-ply-3.11-14.el9.noarch 79/145 2023-09-23T03:13:56.779 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 80/145 2023-09-23T03:13:56.779 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 81/145 2023-09-23T03:13:56.779 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-urllib3-1.26.5-3.el9.noarch 82/145 2023-09-23T03:13:56.779 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libgfortran-11.4.1-2.1.el9.x86_64 83/145 2023-09-23T03:13:56.779 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libquadmath-11.4.1-2.1.el9.x86_64 84/145 2023-09-23T03:13:56.779 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 85/145 2023-09-23T03:13:56.780 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : automake-1.16.2-8.el9.noarch 86/145 2023-09-23T03:13:56.780 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-libelf-devel-0.189-3.el9.x86_64 87/145 2023-09-23T03:13:56.780 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : jq-1.6-15.el9.x86_64 88/145 2023-09-23T03:13:56.780 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 89/145 2023-09-23T03:13:56.780 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 90/145 2023-09-23T03:13:56.780 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-devel-1:3.0.7-24.el9.x86_64 91/145 2023-09-23T03:13:56.780 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/145 2023-09-23T03:13:56.780 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 93/145 2023-09-23T03:13:56.780 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 94/145 2023-09-23T03:13:56.780 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 95/145 2023-09-23T03:13:56.780 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 96/145 2023-09-23T03:13:56.780 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 97/145 2023-09-23T03:13:56.781 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 98/145 2023-09-23T03:13:56.781 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 99/145 2023-09-23T03:13:56.781 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 100/145 2023-09-23T03:13:56.781 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 101/145 2023-09-23T03:13:56.781 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 102/145 2023-09-23T03:13:56.781 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 103/145 2023-09-23T03:13:56.781 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 104/145 2023-09-23T03:13:56.781 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 105/145 2023-09-23T03:13:56.781 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 106/145 2023-09-23T03:13:56.781 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 107/145 2023-09-23T03:13:56.781 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-google-auth-1:2.22.0-1.el9.noarch 108/145 2023-09-23T03:13:56.781 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 109/145 2023-09-23T03:13:56.782 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 110/145 2023-09-23T03:13:56.782 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 111/145 2023-09-23T03:13:56.782 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 112/145 2023-09-23T03:13:56.782 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 113/145 2023-09-23T03:13:56.782 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 114/145 2023-09-23T03:13:56.782 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 115/145 2023-09-23T03:13:56.782 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 116/145 2023-09-23T03:13:56.782 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 117/145 2023-09-23T03:13:56.782 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 118/145 2023-09-23T03:13:56.782 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 119/145 2023-09-23T03:13:56.782 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 120/145 2023-09-23T03:13:56.782 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 121/145 2023-09-23T03:13:56.783 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/145 2023-09-23T03:13:56.783 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 123/145 2023-09-23T03:13:56.783 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 124/145 2023-09-23T03:13:56.783 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 125/145 2023-09-23T03:13:56.783 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 126/145 2023-09-23T03:13:56.783 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 127/145 2023-09-23T03:13:56.783 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 128/145 2023-09-23T03:13:56.783 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 129/145 2023-09-23T03:13:56.783 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 130/145 2023-09-23T03:13:56.783 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 131/145 2023-09-23T03:13:56.783 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : re2-1:20211101-3.el9.x86_64 132/145 2023-09-23T03:13:56.783 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 133/145 2023-09-23T03:13:56.784 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librados2-2:18.0.0-6336.g86266391.el9.x86_64 134/145 2023-09-23T03:13:56.784 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 135/145 2023-09-23T03:13:56.784 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librbd1-2:18.0.0-6336.g86266391.el9.x86_64 136/145 2023-09-23T03:13:56.784 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 137/145 2023-09-23T03:13:56.784 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-1:1.45.2-1.el9.x86_64 138/145 2023-09-23T03:13:56.784 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 139/145 2023-09-23T03:13:56.784 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-libnm-1:1.45.2-1.el9.x86_64 140/145 2023-09-23T03:13:56.784 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 141/145 2023-09-23T03:13:56.784 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-team-1:1.45.2-1.el9.x86_64 142/145 2023-09-23T03:13:56.784 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 143/145 2023-09-23T03:13:56.784 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-tui-1:1.45.2-1.el9.x86_64 144/145 2023-09-23T03:13:57.137 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 134/145 2023-09-23T03:13:57.180 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 134/145 2023-09-23T03:13:57.241 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 134/145 2023-09-23T03:13:58.593 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : cephfs-top-2:18.0.0-6336.g86266391.el9.noarch 135/145 2023-09-23T03:13:58.702 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : openssl-devel-1:3.0.7-24.el9.x86_64 136/145 2023-09-23T03:13:58.759 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 137/145 2023-09-23T03:13:58.890 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : flex-2.6.4-9.el9.x86_64 138/145 2023-09-23T03:13:58.917 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 145/145 2023-09-23T03:13:58.917 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:13:58.917 INFO:teuthology.orchestra.run.smithi154.stdout:Upgraded: 2023-09-23T03:13:58.917 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-1:1.45.2-1.el9.x86_64 2023-09-23T03:13:58.918 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-libnm-1:1.45.2-1.el9.x86_64 2023-09-23T03:13:58.918 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-team-1:1.45.2-1.el9.x86_64 2023-09-23T03:13:58.918 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-tui-1:1.45.2-1.el9.x86_64 2023-09-23T03:13:58.918 INFO:teuthology.orchestra.run.smithi154.stdout: librados2-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:13:58.918 INFO:teuthology.orchestra.run.smithi154.stdout: librbd1-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:13:58.918 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2023-09-23T03:13:58.918 INFO:teuthology.orchestra.run.smithi154.stdout: automake-1.16.2-8.el9.noarch 2023-09-23T03:13:58.918 INFO:teuthology.orchestra.run.smithi154.stdout: bison-3.7.4-5.el9.x86_64 2023-09-23T03:13:58.918 INFO:teuthology.orchestra.run.smithi154.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-09-23T03:13:58.918 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:13:58.918 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-base-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:13:58.919 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-common-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:13:58.919 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-fuse-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:13:58.919 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-grafana-dashboards-2:18.0.0-6336.g86266391.el9.noarch 2023-09-23T03:13:58.919 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-immutable-object-cache-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:13:58.919 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mds-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:13:58.919 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:13:58.919 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-cephadm-2:18.0.0-6336.g86266391.el9.noarch 2023-09-23T03:13:58.919 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-dashboard-2:18.0.0-6336.g86266391.el9.noarch 2023-09-23T03:13:58.919 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6336.g86266391.el9.noarch 2023-09-23T03:13:58.919 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-modules-core-2:18.0.0-6336.g86266391.el9.noarch 2023-09-23T03:13:58.919 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-rook-2:18.0.0-6336.g86266391.el9.noarch 2023-09-23T03:13:58.920 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mon-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:13:58.920 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-osd-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:13:58.920 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-prometheus-alerts-2:18.0.0-6336.g86266391.el9.noarch 2023-09-23T03:13:58.920 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-radosgw-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:13:58.920 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-selinux-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:13:58.920 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-test-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:13:58.920 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-volume-2:18.0.0-6336.g86266391.el9.noarch 2023-09-23T03:13:58.920 INFO:teuthology.orchestra.run.smithi154.stdout: cephadm-2:18.0.0-6336.g86266391.el9.noarch 2023-09-23T03:13:58.920 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-mirror-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:13:58.920 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-top-2:18.0.0-6336.g86266391.el9.noarch 2023-09-23T03:13:58.920 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libelf-devel-0.189-3.el9.x86_64 2023-09-23T03:13:58.920 INFO:teuthology.orchestra.run.smithi154.stdout: flex-2.6.4-9.el9.x86_64 2023-09-23T03:13:58.921 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-09-23T03:13:58.921 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-09-23T03:13:58.921 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-09-23T03:13:58.921 INFO:teuthology.orchestra.run.smithi154.stdout: fmt-8.1.1-5.el9.x86_64 2023-09-23T03:13:58.921 INFO:teuthology.orchestra.run.smithi154.stdout: fuse-2.9.9-15.el9.x86_64 2023-09-23T03:13:58.921 INFO:teuthology.orchestra.run.smithi154.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-09-23T03:13:58.921 INFO:teuthology.orchestra.run.smithi154.stdout: jq-1.6-15.el9.x86_64 2023-09-23T03:13:58.921 INFO:teuthology.orchestra.run.smithi154.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-09-23T03:13:58.921 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow-9.0.0-9.el9.x86_64 2023-09-23T03:13:58.921 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-09-23T03:13:58.921 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs-devel-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:13:58.921 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs2-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:13:58.922 INFO:teuthology.orchestra.run.smithi154.stdout: libcephsqlite-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:13:58.922 INFO:teuthology.orchestra.run.smithi154.stdout: libgfortran-11.4.1-2.1.el9.x86_64 2023-09-23T03:13:58.922 INFO:teuthology.orchestra.run.smithi154.stdout: liboath-2.6.7-2.el9.x86_64 2023-09-23T03:13:58.922 INFO:teuthology.orchestra.run.smithi154.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-09-23T03:13:58.922 INFO:teuthology.orchestra.run.smithi154.stdout: libquadmath-11.4.1-2.1.el9.x86_64 2023-09-23T03:13:58.922 INFO:teuthology.orchestra.run.smithi154.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-09-23T03:13:58.922 INFO:teuthology.orchestra.run.smithi154.stdout: librados-devel-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:13:58.922 INFO:teuthology.orchestra.run.smithi154.stdout: libradosstriper1-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:13:58.922 INFO:teuthology.orchestra.run.smithi154.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-09-23T03:13:58.922 INFO:teuthology.orchestra.run.smithi154.stdout: librgw2-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:13:58.922 INFO:teuthology.orchestra.run.smithi154.stdout: libtool-2.4.6-45.el9.x86_64 2023-09-23T03:13:58.922 INFO:teuthology.orchestra.run.smithi154.stdout: libunwind-1.6.2-1.el9.x86_64 2023-09-23T03:13:58.923 INFO:teuthology.orchestra.run.smithi154.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-09-23T03:13:58.923 INFO:teuthology.orchestra.run.smithi154.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-09-23T03:13:58.923 INFO:teuthology.orchestra.run.smithi154.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-09-23T03:13:58.923 INFO:teuthology.orchestra.run.smithi154.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-09-23T03:13:58.923 INFO:teuthology.orchestra.run.smithi154.stdout: openblas-0.3.21-2.el9.x86_64 2023-09-23T03:13:58.923 INFO:teuthology.orchestra.run.smithi154.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-09-23T03:13:58.923 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-devel-1:3.0.7-24.el9.x86_64 2023-09-23T03:13:58.923 INFO:teuthology.orchestra.run.smithi154.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-09-23T03:13:58.923 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-13.11-1.el9.x86_64 2023-09-23T03:13:58.923 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-09-23T03:13:58.923 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-09-23T03:13:58.924 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-server-13.11-1.el9.x86_64 2023-09-23T03:13:58.924 INFO:teuthology.orchestra.run.smithi154.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-09-23T03:13:58.924 INFO:teuthology.orchestra.run.smithi154.stdout: python3-babel-2.9.1-2.el9.noarch 2023-09-23T03:13:58.924 INFO:teuthology.orchestra.run.smithi154.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-09-23T03:13:58.924 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-09-23T03:13:58.924 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-argparse-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:13:58.924 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-common-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:13:58.924 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cephfs-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:13:58.924 INFO:teuthology.orchestra.run.smithi154.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-09-23T03:13:58.924 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-09-23T03:13:58.924 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-09-23T03:13:58.924 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-09-23T03:13:58.925 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-09-23T03:13:58.925 INFO:teuthology.orchestra.run.smithi154.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-09-23T03:13:58.925 INFO:teuthology.orchestra.run.smithi154.stdout: python3-google-auth-1:2.22.0-1.el9.noarch 2023-09-23T03:13:58.925 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-09-23T03:13:58.925 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-09-23T03:13:58.925 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-09-23T03:13:58.925 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-09-23T03:13:58.925 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-09-23T03:13:58.925 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-09-23T03:13:58.925 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-09-23T03:13:58.925 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-09-23T03:13:58.925 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-09-23T03:13:58.926 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-09-23T03:13:58.926 INFO:teuthology.orchestra.run.smithi154.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-09-23T03:13:58.926 INFO:teuthology.orchestra.run.smithi154.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-09-23T03:13:58.926 INFO:teuthology.orchestra.run.smithi154.stdout: python3-mako-1.1.4-6.el9.noarch 2023-09-23T03:13:58.926 INFO:teuthology.orchestra.run.smithi154.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-09-23T03:13:58.926 INFO:teuthology.orchestra.run.smithi154.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-09-23T03:13:58.926 INFO:teuthology.orchestra.run.smithi154.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-09-23T03:13:58.926 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-09-23T03:13:58.926 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-09-23T03:13:58.926 INFO:teuthology.orchestra.run.smithi154.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-09-23T03:13:58.926 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-09-23T03:13:58.927 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ply-3.11-14.el9.noarch 2023-09-23T03:13:58.927 INFO:teuthology.orchestra.run.smithi154.stdout: python3-portend-3.1.0-2.el9.noarch 2023-09-23T03:13:58.927 INFO:teuthology.orchestra.run.smithi154.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-09-23T03:13:58.927 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-09-23T03:13:58.927 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-09-23T03:13:58.927 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-09-23T03:13:58.927 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pycparser-2.20-6.el9.noarch 2023-09-23T03:13:58.927 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pytz-2021.1-5.el9.noarch 2023-09-23T03:13:58.927 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rados-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:13:58.927 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rbd-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:13:58.927 INFO:teuthology.orchestra.run.smithi154.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-09-23T03:13:58.927 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-2.25.1-7.el9.noarch 2023-09-23T03:13:58.928 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-09-23T03:13:58.928 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rgw-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:13:58.928 INFO:teuthology.orchestra.run.smithi154.stdout: python3-routes-2.5.1-5.el9.noarch 2023-09-23T03:13:58.928 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rsa-4.9-2.el9.noarch 2023-09-23T03:13:58.928 INFO:teuthology.orchestra.run.smithi154.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-09-23T03:13:58.928 INFO:teuthology.orchestra.run.smithi154.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-09-23T03:13:58.928 INFO:teuthology.orchestra.run.smithi154.stdout: python3-toml-0.10.2-6.el9.noarch 2023-09-23T03:13:58.928 INFO:teuthology.orchestra.run.smithi154.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-09-23T03:13:58.928 INFO:teuthology.orchestra.run.smithi154.stdout: python3-urllib3-1.26.5-3.el9.noarch 2023-09-23T03:13:58.928 INFO:teuthology.orchestra.run.smithi154.stdout: python3-webob-1.8.7-6.el9.noarch 2023-09-23T03:13:58.928 INFO:teuthology.orchestra.run.smithi154.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-09-23T03:13:58.928 INFO:teuthology.orchestra.run.smithi154.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-09-23T03:13:58.928 INFO:teuthology.orchestra.run.smithi154.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-09-23T03:13:58.929 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-fuse-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:13:58.929 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-mirror-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:13:58.929 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-nbd-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:13:58.929 INFO:teuthology.orchestra.run.smithi154.stdout: re2-1:20211101-3.el9.x86_64 2023-09-23T03:13:58.929 INFO:teuthology.orchestra.run.smithi154.stdout: socat-1.7.4.1-5.el9.x86_64 2023-09-23T03:13:58.929 INFO:teuthology.orchestra.run.smithi154.stdout: thrift-0.15.0-2.el9.x86_64 2023-09-23T03:13:58.929 INFO:teuthology.orchestra.run.smithi154.stdout: uuid-1.6.2-55.el9.x86_64 2023-09-23T03:13:58.929 INFO:teuthology.orchestra.run.smithi154.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-09-23T03:13:58.929 INFO:teuthology.orchestra.run.smithi154.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-09-23T03:13:58.930 INFO:teuthology.orchestra.run.smithi190.stdout: Installing : bison-3.7.4-5.el9.x86_64 139/145 2023-09-23T03:13:58.930 INFO:teuthology.orchestra.run.smithi154.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-09-23T03:13:58.930 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:13:58.930 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2023-09-23T03:13:58.966 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 140/145 2023-09-23T03:13:58.966 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 141/145 2023-09-23T03:13:59.093 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 141/145 2023-09-23T03:13:59.137 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 142/145 2023-09-23T03:13:59.175 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 143/145 2023-09-23T03:13:59.175 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 143/145 2023-09-23T03:13:59.409 DEBUG:teuthology.parallel:result is None 2023-09-23T03:13:59.490 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 143/145 2023-09-23T03:13:59.490 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 144/145 2023-09-23T03:14:00.038 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 144/145 2023-09-23T03:14:00.038 INFO:teuthology.orchestra.run.smithi190.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 145/145 2023-09-23T03:14:00.557 INFO:teuthology.orchestra.run.smithi190.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 145/145 2023-09-23T03:14:00.557 INFO:teuthology.orchestra.run.smithi190.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-09-23T03:14:00.557 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:14:04.760 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-2:18.0.0-6336.g86266391.el9.x86_64 1/145 2023-09-23T03:14:04.761 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-base-2:18.0.0-6336.g86266391.el9.x86_64 2/145 2023-09-23T03:14:04.761 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-common-2:18.0.0-6336.g86266391.el9.x86_64 3/145 2023-09-23T03:14:04.761 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-fuse-2:18.0.0-6336.g86266391.el9.x86_64 4/145 2023-09-23T03:14:04.761 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-6336.g86266 5/145 2023-09-23T03:14:04.761 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mds-2:18.0.0-6336.g86266391.el9.x86_64 6/145 2023-09-23T03:14:04.761 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-2:18.0.0-6336.g86266391.el9.x86_64 7/145 2023-09-23T03:14:04.761 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mon-2:18.0.0-6336.g86266391.el9.x86_64 8/145 2023-09-23T03:14:04.761 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-osd-2:18.0.0-6336.g86266391.el9.x86_64 9/145 2023-09-23T03:14:04.761 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-radosgw-2:18.0.0-6336.g86266391.el9.x86_64 10/145 2023-09-23T03:14:04.761 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-selinux-2:18.0.0-6336.g86266391.el9.x86_64 11/145 2023-09-23T03:14:04.761 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-test-2:18.0.0-6336.g86266391.el9.x86_64 12/145 2023-09-23T03:14:04.761 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephfs-mirror-2:18.0.0-6336.g86266391.el9.x86_64 13/145 2023-09-23T03:14:04.762 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephfs-devel-2:18.0.0-6336.g86266391.el9.x86_ 14/145 2023-09-23T03:14:04.762 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephfs2-2:18.0.0-6336.g86266391.el9.x86_64 15/145 2023-09-23T03:14:04.762 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libcephsqlite-2:18.0.0-6336.g86266391.el9.x86_64 16/145 2023-09-23T03:14:04.762 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados-devel-2:18.0.0-6336.g86266391.el9.x86_6 17/145 2023-09-23T03:14:04.762 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libradosstriper1-2:18.0.0-6336.g86266391.el9.x86 18/145 2023-09-23T03:14:04.762 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librgw2-2:18.0.0-6336.g86266391.el9.x86_64 19/145 2023-09-23T03:14:04.762 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ceph-argparse-2:18.0.0-6336.g86266391.el 20/145 2023-09-23T03:14:04.762 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ceph-common-2:18.0.0-6336.g86266391.el9. 21/145 2023-09-23T03:14:04.762 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cephfs-2:18.0.0-6336.g86266391.el9.x86_6 22/145 2023-09-23T03:14:04.762 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rados-2:18.0.0-6336.g86266391.el9.x86_64 23/145 2023-09-23T03:14:04.762 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rbd-2:18.0.0-6336.g86266391.el9.x86_64 24/145 2023-09-23T03:14:04.763 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rgw-2:18.0.0-6336.g86266391.el9.x86_64 25/145 2023-09-23T03:14:04.772 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-fuse-2:18.0.0-6336.g86266391.el9.x86_64 26/145 2023-09-23T03:14:04.772 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-mirror-2:18.0.0-6336.g86266391.el9.x86_64 27/145 2023-09-23T03:14:04.772 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : rbd-nbd-2:18.0.0-6336.g86266391.el9.x86_64 28/145 2023-09-23T03:14:04.772 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-6336.g86266391. 29/145 2023-09-23T03:14:04.772 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-6336.g86266391.el9.noa 30/145 2023-09-23T03:14:04.772 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-6336.g86266391.el9.n 31/145 2023-09-23T03:14:04.773 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-6336.g862 32/145 2023-09-23T03:14:04.773 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-6336.g86266391.el 33/145 2023-09-23T03:14:04.773 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-mgr-rook-2:18.0.0-6336.g86266391.el9.noarch 34/145 2023-09-23T03:14:04.773 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-6336.g86266391.e 35/145 2023-09-23T03:14:04.773 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : ceph-volume-2:18.0.0-6336.g86266391.el9.noarch 36/145 2023-09-23T03:14:04.773 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephadm-2:18.0.0-6336.g86266391.el9.noarch 37/145 2023-09-23T03:14:04.773 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : cephfs-top-2:18.0.0-6336.g86266391.el9.noarch 38/145 2023-09-23T03:14:04.773 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : bison-3.7.4-5.el9.x86_64 39/145 2023-09-23T03:14:04.773 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 40/145 2023-09-23T03:14:04.773 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flex-2.6.4-9.el9.x86_64 41/145 2023-09-23T03:14:04.773 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 42/145 2023-09-23T03:14:04.773 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 43/145 2023-09-23T03:14:04.774 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 44/145 2023-09-23T03:14:04.774 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 45/145 2023-09-23T03:14:04.774 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 46/145 2023-09-23T03:14:04.774 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 47/145 2023-09-23T03:14:04.774 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 48/145 2023-09-23T03:14:04.774 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 49/145 2023-09-23T03:14:04.774 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 50/145 2023-09-23T03:14:04.774 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 51/145 2023-09-23T03:14:04.774 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 52/145 2023-09-23T03:14:04.774 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 53/145 2023-09-23T03:14:04.774 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-13.11-1.el9.x86_64 54/145 2023-09-23T03:14:04.775 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 55/145 2023-09-23T03:14:04.775 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 56/145 2023-09-23T03:14:04.775 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 57/145 2023-09-23T03:14:04.775 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 58/145 2023-09-23T03:14:04.775 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 59/145 2023-09-23T03:14:04.775 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 60/145 2023-09-23T03:14:04.775 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/145 2023-09-23T03:14:04.775 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/145 2023-09-23T03:14:04.775 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/145 2023-09-23T03:14:04.775 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/145 2023-09-23T03:14:04.775 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/145 2023-09-23T03:14:04.776 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/145 2023-09-23T03:14:04.776 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/145 2023-09-23T03:14:04.776 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 68/145 2023-09-23T03:14:04.776 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 69/145 2023-09-23T03:14:04.776 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 70/145 2023-09-23T03:14:04.776 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 71/145 2023-09-23T03:14:04.776 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 72/145 2023-09-23T03:14:04.776 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 73/145 2023-09-23T03:14:04.776 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 74/145 2023-09-23T03:14:04.776 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 75/145 2023-09-23T03:14:04.776 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 76/145 2023-09-23T03:14:04.777 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 77/145 2023-09-23T03:14:04.777 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 78/145 2023-09-23T03:14:04.777 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-ply-3.11-14.el9.noarch 79/145 2023-09-23T03:14:04.777 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 80/145 2023-09-23T03:14:04.777 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 81/145 2023-09-23T03:14:04.777 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-urllib3-1.26.5-3.el9.noarch 82/145 2023-09-23T03:14:04.777 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libgfortran-11.4.1-2.1.el9.x86_64 83/145 2023-09-23T03:14:04.777 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libquadmath-11.4.1-2.1.el9.x86_64 84/145 2023-09-23T03:14:04.777 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 85/145 2023-09-23T03:14:04.777 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : automake-1.16.2-8.el9.noarch 86/145 2023-09-23T03:14:04.777 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : elfutils-libelf-devel-0.189-3.el9.x86_64 87/145 2023-09-23T03:14:04.777 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : jq-1.6-15.el9.x86_64 88/145 2023-09-23T03:14:04.778 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 89/145 2023-09-23T03:14:04.778 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 90/145 2023-09-23T03:14:04.778 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : openssl-devel-1:3.0.7-24.el9.x86_64 91/145 2023-09-23T03:14:04.778 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/145 2023-09-23T03:14:04.778 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 93/145 2023-09-23T03:14:04.778 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 94/145 2023-09-23T03:14:04.778 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 95/145 2023-09-23T03:14:04.778 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 96/145 2023-09-23T03:14:04.778 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 97/145 2023-09-23T03:14:04.778 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 98/145 2023-09-23T03:14:04.778 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 99/145 2023-09-23T03:14:04.779 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 100/145 2023-09-23T03:14:04.779 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 101/145 2023-09-23T03:14:04.779 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 102/145 2023-09-23T03:14:04.779 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 103/145 2023-09-23T03:14:04.779 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 104/145 2023-09-23T03:14:04.779 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 105/145 2023-09-23T03:14:04.779 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 106/145 2023-09-23T03:14:04.779 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 107/145 2023-09-23T03:14:04.779 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-google-auth-1:2.22.0-1.el9.noarch 108/145 2023-09-23T03:14:04.779 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 109/145 2023-09-23T03:14:04.779 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 110/145 2023-09-23T03:14:04.779 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 111/145 2023-09-23T03:14:04.779 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 112/145 2023-09-23T03:14:04.780 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 113/145 2023-09-23T03:14:04.780 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 114/145 2023-09-23T03:14:04.780 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 115/145 2023-09-23T03:14:04.780 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 116/145 2023-09-23T03:14:04.780 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 117/145 2023-09-23T03:14:04.780 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 118/145 2023-09-23T03:14:04.780 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 119/145 2023-09-23T03:14:04.780 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 120/145 2023-09-23T03:14:04.780 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 121/145 2023-09-23T03:14:04.780 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/145 2023-09-23T03:14:04.780 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 123/145 2023-09-23T03:14:04.780 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 124/145 2023-09-23T03:14:04.780 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 125/145 2023-09-23T03:14:04.781 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 126/145 2023-09-23T03:14:04.781 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 127/145 2023-09-23T03:14:04.781 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 128/145 2023-09-23T03:14:04.781 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 129/145 2023-09-23T03:14:04.781 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 130/145 2023-09-23T03:14:04.781 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 131/145 2023-09-23T03:14:04.781 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : re2-1:20211101-3.el9.x86_64 132/145 2023-09-23T03:14:04.781 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 133/145 2023-09-23T03:14:04.781 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados2-2:18.0.0-6336.g86266391.el9.x86_64 134/145 2023-09-23T03:14:04.781 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 135/145 2023-09-23T03:14:04.781 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librbd1-2:18.0.0-6336.g86266391.el9.x86_64 136/145 2023-09-23T03:14:04.781 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 137/145 2023-09-23T03:14:04.781 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-1:1.45.2-1.el9.x86_64 138/145 2023-09-23T03:14:04.782 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 139/145 2023-09-23T03:14:04.782 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-libnm-1:1.45.2-1.el9.x86_64 140/145 2023-09-23T03:14:04.782 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 141/145 2023-09-23T03:14:04.782 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-team-1:1.45.2-1.el9.x86_64 142/145 2023-09-23T03:14:04.782 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 143/145 2023-09-23T03:14:04.782 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-tui-1:1.45.2-1.el9.x86_64 144/145 2023-09-23T03:14:06.787 INFO:teuthology.orchestra.run.smithi190.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 145/145 2023-09-23T03:14:06.787 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:14:06.787 INFO:teuthology.orchestra.run.smithi190.stdout:Upgraded: 2023-09-23T03:14:06.787 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-1:1.45.2-1.el9.x86_64 2023-09-23T03:14:06.787 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-libnm-1:1.45.2-1.el9.x86_64 2023-09-23T03:14:06.788 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-team-1:1.45.2-1.el9.x86_64 2023-09-23T03:14:06.788 INFO:teuthology.orchestra.run.smithi190.stdout: NetworkManager-tui-1:1.45.2-1.el9.x86_64 2023-09-23T03:14:06.788 INFO:teuthology.orchestra.run.smithi190.stdout: librados2-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:14:06.788 INFO:teuthology.orchestra.run.smithi190.stdout: librbd1-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:14:06.788 INFO:teuthology.orchestra.run.smithi190.stdout:Installed: 2023-09-23T03:14:06.788 INFO:teuthology.orchestra.run.smithi190.stdout: automake-1.16.2-8.el9.noarch 2023-09-23T03:14:06.788 INFO:teuthology.orchestra.run.smithi190.stdout: bison-3.7.4-5.el9.x86_64 2023-09-23T03:14:06.788 INFO:teuthology.orchestra.run.smithi190.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-09-23T03:14:06.788 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:14:06.788 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-base-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:14:06.788 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-common-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:14:06.788 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-fuse-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:14:06.789 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-grafana-dashboards-2:18.0.0-6336.g86266391.el9.noarch 2023-09-23T03:14:06.789 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-immutable-object-cache-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:14:06.789 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mds-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:14:06.789 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:14:06.789 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-cephadm-2:18.0.0-6336.g86266391.el9.noarch 2023-09-23T03:14:06.789 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-dashboard-2:18.0.0-6336.g86266391.el9.noarch 2023-09-23T03:14:06.789 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-diskprediction-local-2:18.0.0-6336.g86266391.el9.noarch 2023-09-23T03:14:06.789 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-modules-core-2:18.0.0-6336.g86266391.el9.noarch 2023-09-23T03:14:06.789 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mgr-rook-2:18.0.0-6336.g86266391.el9.noarch 2023-09-23T03:14:06.789 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-mon-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:14:06.789 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-osd-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:14:06.790 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-prometheus-alerts-2:18.0.0-6336.g86266391.el9.noarch 2023-09-23T03:14:06.790 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-radosgw-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:14:06.790 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-selinux-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:14:06.790 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-test-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:14:06.790 INFO:teuthology.orchestra.run.smithi190.stdout: ceph-volume-2:18.0.0-6336.g86266391.el9.noarch 2023-09-23T03:14:06.790 INFO:teuthology.orchestra.run.smithi190.stdout: cephadm-2:18.0.0-6336.g86266391.el9.noarch 2023-09-23T03:14:06.790 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-mirror-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:14:06.790 INFO:teuthology.orchestra.run.smithi190.stdout: cephfs-top-2:18.0.0-6336.g86266391.el9.noarch 2023-09-23T03:14:06.790 INFO:teuthology.orchestra.run.smithi190.stdout: elfutils-libelf-devel-0.189-3.el9.x86_64 2023-09-23T03:14:06.790 INFO:teuthology.orchestra.run.smithi190.stdout: flex-2.6.4-9.el9.x86_64 2023-09-23T03:14:06.790 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-09-23T03:14:06.791 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-09-23T03:14:06.791 INFO:teuthology.orchestra.run.smithi190.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-09-23T03:14:06.791 INFO:teuthology.orchestra.run.smithi190.stdout: fmt-8.1.1-5.el9.x86_64 2023-09-23T03:14:06.791 INFO:teuthology.orchestra.run.smithi190.stdout: fuse-2.9.9-15.el9.x86_64 2023-09-23T03:14:06.791 INFO:teuthology.orchestra.run.smithi190.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-09-23T03:14:06.791 INFO:teuthology.orchestra.run.smithi190.stdout: jq-1.6-15.el9.x86_64 2023-09-23T03:14:06.791 INFO:teuthology.orchestra.run.smithi190.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-09-23T03:14:06.791 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow-9.0.0-9.el9.x86_64 2023-09-23T03:14:06.791 INFO:teuthology.orchestra.run.smithi190.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-09-23T03:14:06.791 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs-devel-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:14:06.791 INFO:teuthology.orchestra.run.smithi190.stdout: libcephfs2-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:14:06.791 INFO:teuthology.orchestra.run.smithi190.stdout: libcephsqlite-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:14:06.792 INFO:teuthology.orchestra.run.smithi190.stdout: libgfortran-11.4.1-2.1.el9.x86_64 2023-09-23T03:14:06.792 INFO:teuthology.orchestra.run.smithi190.stdout: liboath-2.6.7-2.el9.x86_64 2023-09-23T03:14:06.792 INFO:teuthology.orchestra.run.smithi190.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-09-23T03:14:06.792 INFO:teuthology.orchestra.run.smithi190.stdout: libquadmath-11.4.1-2.1.el9.x86_64 2023-09-23T03:14:06.792 INFO:teuthology.orchestra.run.smithi190.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-09-23T03:14:06.792 INFO:teuthology.orchestra.run.smithi190.stdout: librados-devel-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:14:06.792 INFO:teuthology.orchestra.run.smithi190.stdout: libradosstriper1-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:14:06.792 INFO:teuthology.orchestra.run.smithi190.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-09-23T03:14:06.792 INFO:teuthology.orchestra.run.smithi190.stdout: librgw2-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:14:06.792 INFO:teuthology.orchestra.run.smithi190.stdout: libtool-2.4.6-45.el9.x86_64 2023-09-23T03:14:06.792 INFO:teuthology.orchestra.run.smithi190.stdout: libunwind-1.6.2-1.el9.x86_64 2023-09-23T03:14:06.792 INFO:teuthology.orchestra.run.smithi190.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-09-23T03:14:06.793 INFO:teuthology.orchestra.run.smithi190.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-09-23T03:14:06.793 INFO:teuthology.orchestra.run.smithi190.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-09-23T03:14:06.793 INFO:teuthology.orchestra.run.smithi190.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-09-23T03:14:06.793 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-0.3.21-2.el9.x86_64 2023-09-23T03:14:06.793 INFO:teuthology.orchestra.run.smithi190.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-09-23T03:14:06.793 INFO:teuthology.orchestra.run.smithi190.stdout: openssl-devel-1:3.0.7-24.el9.x86_64 2023-09-23T03:14:06.793 INFO:teuthology.orchestra.run.smithi190.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-09-23T03:14:06.793 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-13.11-1.el9.x86_64 2023-09-23T03:14:06.794 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-09-23T03:14:06.794 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-09-23T03:14:06.794 INFO:teuthology.orchestra.run.smithi190.stdout: postgresql-server-13.11-1.el9.x86_64 2023-09-23T03:14:06.794 INFO:teuthology.orchestra.run.smithi190.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-09-23T03:14:06.794 INFO:teuthology.orchestra.run.smithi190.stdout: python3-babel-2.9.1-2.el9.noarch 2023-09-23T03:14:06.794 INFO:teuthology.orchestra.run.smithi190.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-09-23T03:14:06.794 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-09-23T03:14:06.794 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-argparse-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:14:06.794 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ceph-common-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:14:06.794 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cephfs-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:14:06.795 INFO:teuthology.orchestra.run.smithi190.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-09-23T03:14:06.795 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-09-23T03:14:06.795 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-09-23T03:14:06.795 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-09-23T03:14:06.795 INFO:teuthology.orchestra.run.smithi190.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-09-23T03:14:06.795 INFO:teuthology.orchestra.run.smithi190.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-09-23T03:14:06.795 INFO:teuthology.orchestra.run.smithi190.stdout: python3-google-auth-1:2.22.0-1.el9.noarch 2023-09-23T03:14:06.795 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-09-23T03:14:06.795 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-09-23T03:14:06.796 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-09-23T03:14:06.796 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-09-23T03:14:06.796 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-09-23T03:14:06.796 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-09-23T03:14:06.796 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-09-23T03:14:06.796 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-09-23T03:14:06.796 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-09-23T03:14:06.796 INFO:teuthology.orchestra.run.smithi190.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-09-23T03:14:06.796 INFO:teuthology.orchestra.run.smithi190.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-09-23T03:14:06.796 INFO:teuthology.orchestra.run.smithi190.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-09-23T03:14:06.796 INFO:teuthology.orchestra.run.smithi190.stdout: python3-mako-1.1.4-6.el9.noarch 2023-09-23T03:14:06.796 INFO:teuthology.orchestra.run.smithi190.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-09-23T03:14:06.797 INFO:teuthology.orchestra.run.smithi190.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-09-23T03:14:06.797 INFO:teuthology.orchestra.run.smithi190.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-09-23T03:14:06.797 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-09-23T03:14:06.797 INFO:teuthology.orchestra.run.smithi190.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-09-23T03:14:06.797 INFO:teuthology.orchestra.run.smithi190.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-09-23T03:14:06.797 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-09-23T03:14:06.797 INFO:teuthology.orchestra.run.smithi190.stdout: python3-ply-3.11-14.el9.noarch 2023-09-23T03:14:06.797 INFO:teuthology.orchestra.run.smithi190.stdout: python3-portend-3.1.0-2.el9.noarch 2023-09-23T03:14:06.797 INFO:teuthology.orchestra.run.smithi190.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-09-23T03:14:06.797 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-09-23T03:14:06.797 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-09-23T03:14:06.798 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-09-23T03:14:06.798 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pycparser-2.20-6.el9.noarch 2023-09-23T03:14:06.798 INFO:teuthology.orchestra.run.smithi190.stdout: python3-pytz-2021.1-5.el9.noarch 2023-09-23T03:14:06.798 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rados-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:14:06.798 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rbd-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:14:06.798 INFO:teuthology.orchestra.run.smithi190.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-09-23T03:14:06.798 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-2.25.1-7.el9.noarch 2023-09-23T03:14:06.798 INFO:teuthology.orchestra.run.smithi190.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-09-23T03:14:06.799 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rgw-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:14:06.799 INFO:teuthology.orchestra.run.smithi190.stdout: python3-routes-2.5.1-5.el9.noarch 2023-09-23T03:14:06.799 INFO:teuthology.orchestra.run.smithi190.stdout: python3-rsa-4.9-2.el9.noarch 2023-09-23T03:14:06.799 INFO:teuthology.orchestra.run.smithi190.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-09-23T03:14:06.799 INFO:teuthology.orchestra.run.smithi190.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-09-23T03:14:06.799 INFO:teuthology.orchestra.run.smithi190.stdout: python3-toml-0.10.2-6.el9.noarch 2023-09-23T03:14:06.799 INFO:teuthology.orchestra.run.smithi190.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-09-23T03:14:06.799 INFO:teuthology.orchestra.run.smithi190.stdout: python3-urllib3-1.26.5-3.el9.noarch 2023-09-23T03:14:06.799 INFO:teuthology.orchestra.run.smithi190.stdout: python3-webob-1.8.7-6.el9.noarch 2023-09-23T03:14:06.799 INFO:teuthology.orchestra.run.smithi190.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-09-23T03:14:06.799 INFO:teuthology.orchestra.run.smithi190.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-09-23T03:14:06.799 INFO:teuthology.orchestra.run.smithi190.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-09-23T03:14:06.800 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-fuse-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:14:06.800 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-mirror-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:14:06.800 INFO:teuthology.orchestra.run.smithi190.stdout: rbd-nbd-2:18.0.0-6336.g86266391.el9.x86_64 2023-09-23T03:14:06.800 INFO:teuthology.orchestra.run.smithi190.stdout: re2-1:20211101-3.el9.x86_64 2023-09-23T03:14:06.800 INFO:teuthology.orchestra.run.smithi190.stdout: socat-1.7.4.1-5.el9.x86_64 2023-09-23T03:14:06.800 INFO:teuthology.orchestra.run.smithi190.stdout: thrift-0.15.0-2.el9.x86_64 2023-09-23T03:14:06.800 INFO:teuthology.orchestra.run.smithi190.stdout: uuid-1.6.2-55.el9.x86_64 2023-09-23T03:14:06.800 INFO:teuthology.orchestra.run.smithi190.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-09-23T03:14:06.800 INFO:teuthology.orchestra.run.smithi190.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-09-23T03:14:06.801 INFO:teuthology.orchestra.run.smithi190.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-09-23T03:14:06.801 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:14:06.801 INFO:teuthology.orchestra.run.smithi190.stdout:Complete! 2023-09-23T03:14:07.333 DEBUG:teuthology.parallel:result is None 2023-09-23T03:14:07.333 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=862663913a74a99e1c52d50e6a7e59dd6705749a 2023-09-23T03:14:07.479 DEBUG:teuthology.orchestra.run.smithi154:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-23T03:14:07.555 INFO:teuthology.orchestra.run.smithi154.stdout:18.0.0-6336.g86266391.el9 2023-09-23T03:14:07.555 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6336.g86266391.el9 2023-09-23T03:14:07.555 INFO:teuthology.task.install:The correct ceph version 18.0.0-6336.g86266391 is installed. 2023-09-23T03:14:07.556 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=862663913a74a99e1c52d50e6a7e59dd6705749a 2023-09-23T03:14:07.779 DEBUG:teuthology.orchestra.run.smithi190:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-23T03:14:07.838 INFO:teuthology.orchestra.run.smithi190.stdout:18.0.0-6336.g86266391.el9 2023-09-23T03:14:07.839 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6336.g86266391.el9 2023-09-23T03:14:07.839 INFO:teuthology.task.install:The correct ceph version 18.0.0-6336.g86266391 is installed. 2023-09-23T03:14:07.839 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-09-23T03:14:07.839 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:14:07.839 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-23T03:14:07.944 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:14:07.945 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-23T03:14:08.025 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-09-23T03:14:08.025 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:14:08.025 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/daemon-helper 2023-09-23T03:14:08.129 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-23T03:14:08.233 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:14:08.233 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/daemon-helper 2023-09-23T03:14:08.323 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-23T03:14:08.443 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-09-23T03:14:08.444 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:14:08.444 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-23T03:14:08.535 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-23T03:14:08.673 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:14:08.673 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-23T03:14:08.750 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-23T03:14:08.869 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-09-23T03:14:08.869 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:14:08.870 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/stdin-killer 2023-09-23T03:14:08.948 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-23T03:14:09.074 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:14:09.074 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/usr/bin/stdin-killer 2023-09-23T03:14:09.150 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-23T03:14:09.267 INFO:teuthology.run_tasks:Running task ceph... 2023-09-23T03:14:09.330 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-09-23T03:14:09.330 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 777 /var/log/ceph 2023-09-23T03:14:09.339 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 777 /var/log/ceph 2023-09-23T03:14:09.424 INFO:tasks.ceph:Disabling ceph logrotate... 2023-09-23T03:14:09.424 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /etc/logrotate.d/ceph 2023-09-23T03:14:09.429 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /etc/logrotate.d/ceph 2023-09-23T03:14:09.505 INFO:tasks.ceph:Creating extra log directories... 2023-09-23T03:14:09.505 DEBUG:teuthology.orchestra.run.smithi154:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-09-23T03:14:09.513 DEBUG:teuthology.orchestra.run.smithi190:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-09-23T03:14:09.590 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-09-23T03:14:09.590 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-09-23T03:14:09.590 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-09-23T03:14:09.590 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:14:09.591 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-09-23T03:14:09.633 DEBUG:teuthology.orchestra.run.smithi154:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-09-23T03:14:09.869 DEBUG:teuthology.orchestra.run.smithi154:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-09-23T03:14:09.972 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:14:09.972 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-09-23T03:14:10.022 DEBUG:teuthology.orchestra.run.smithi190:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-09-23T03:14:10.238 DEBUG:teuthology.orchestra.run.smithi190:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-09-23T03:14:10.314 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-09-23T03:14:10.315 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'unmatched fragstat', 'unmatched rstat', 'was unreadable, recreating it now', 'Scrub error on inode', 'Metadata damage detected', 'inconsistent rstat on inode', 'Error recovering journal'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-09-23T03:14:10.315 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401456', 'branch': 'wip-vshankar-testing-20230922.144532', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/data-scan}', 'email': 'vshankar@redhat.com', 'first_in_suite': False, 'job_id': '7401456', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': 'fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-vshankar-testing-20230922.144532'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'unmatched fragstat', 'unmatched rstat', 'was unreadable, recreating it now', 'Scrub error on inode', 'Metadata damage detected', 'inconsistent rstat on inode', 'Error recovering journal'], 'sha1': '862663913a74a99e1c52d50e6a7e59dd6705749a'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '862663913a74a99e1c52d50e6a7e59dd6705749a'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-vshankar-testing-20230922.144532', 'sha1': '862663913a74a99e1c52d50e6a7e59dd6705749a'}}, 'owner': 'scheduled_vshankar@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 8196, 'sha1': '862663913a74a99e1c52d50e6a7e59dd6705749a', 'sleep_before_teardown': 0, 'subset': '57/1024', 'suite': 'fs', 'suite_branch': 'wip-vshankar-testing-20230922.144532', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_862663913a74a99e1c52d50e6a7e59dd6705749a/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '862663913a74a99e1c52d50e6a7e59dd6705749a', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi154.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF4iXVXCH7ifGpuKynAenSBvP8YukEyWTkkNqHusRK804qngv2s+hXqAV9tTcZI3im5o9ajvhnPaepqhn47Uq8o=', 'smithi190.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHJ5AsjgG46Z7O7yPa20pbgWscDSfsOFTIUe566npy1eyPjD925g8/ay8Tkm9pywa2BcrKkpubGa3HYMD0SDJXE='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': 'fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '862663913a74a99e1c52d50e6a7e59dd6705749a'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'unmatched fragstat', 'unmatched rstat', 'was unreadable, recreating it now', 'Scrub error on inode', 'Metadata damage detected', 'inconsistent rstat on inode', 'Error recovering journal'], 'sha1': '862663913a74a99e1c52d50e6a7e59dd6705749a', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_data_scan']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-09-23_02:50:30', 'tube': 'smithi', 'user': 'vshankar', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-09-23T03:14:10.315 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-09-23T03:14:10.324 DEBUG:teuthology.orchestra.run.smithi190:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-09-23T03:14:10.377 DEBUG:teuthology.orchestra.run.smithi154:> sudo install -d -m0777 -- /var/run/ceph 2023-09-23T03:14:10.382 DEBUG:teuthology.orchestra.run.smithi190:> sudo install -d -m0777 -- /var/run/ceph 2023-09-23T03:14:10.479 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:14:10.479 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/scratch_devs of=/dev/stdout 2023-09-23T03:14:10.536 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-23T03:14:10.536 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_1 2023-09-23T03:14:10.589 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-23T03:14:10.589 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-23T03:14:10.590 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 633 Links: 1 2023-09-23T03:14:10.590 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-23T03:14:10.590 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2023-09-23T03:14:10.590 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2023-09-23 03:13:52.921012322 +0000 2023-09-23T03:14:10.590 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2023-09-23 03:11:26.041914696 +0000 2023-09-23T03:14:10.590 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2023-09-23 03:11:26.041914696 +0000 2023-09-23T03:14:10.590 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2023-09-23 03:11:26.041914696 +0000 2023-09-23T03:14:10.590 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-23T03:14:10.676 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2023-09-23T03:14:10.676 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2023-09-23T03:14:10.676 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000309662 s, 1.7 MB/s 2023-09-23T03:14:10.679 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-23T03:14:10.733 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_2 2023-09-23T03:14:10.783 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-23T03:14:10.783 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-23T03:14:10.783 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 651 Links: 1 2023-09-23T03:14:10.783 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-23T03:14:10.783 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2023-09-23T03:14:10.783 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2023-09-23 03:13:52.921012322 +0000 2023-09-23T03:14:10.784 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2023-09-23 03:11:27.037888231 +0000 2023-09-23T03:14:10.784 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2023-09-23 03:11:27.037888231 +0000 2023-09-23T03:14:10.784 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2023-09-23 03:11:27.037888231 +0000 2023-09-23T03:14:10.784 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-23T03:14:10.873 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2023-09-23T03:14:10.874 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2023-09-23T03:14:10.874 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000321489 s, 1.6 MB/s 2023-09-23T03:14:10.877 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-23T03:14:10.932 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_3 2023-09-23T03:14:10.982 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-23T03:14:10.982 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-23T03:14:10.982 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 669 Links: 1 2023-09-23T03:14:10.982 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-23T03:14:10.982 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2023-09-23T03:14:10.982 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2023-09-23 03:13:52.922012295 +0000 2023-09-23T03:14:10.983 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2023-09-23 03:11:28.074860677 +0000 2023-09-23T03:14:10.983 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2023-09-23 03:11:28.074860677 +0000 2023-09-23T03:14:10.983 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2023-09-23 03:11:28.074860677 +0000 2023-09-23T03:14:10.983 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-23T03:14:11.071 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2023-09-23T03:14:11.071 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2023-09-23T03:14:11.071 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000383298 s, 1.3 MB/s 2023-09-23T03:14:11.074 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-23T03:14:11.135 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_4 2023-09-23T03:14:11.186 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-23T03:14:11.186 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-23T03:14:11.187 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 689 Links: 1 2023-09-23T03:14:11.187 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-23T03:14:11.187 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2023-09-23T03:14:11.187 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2023-09-23 03:13:52.923012269 +0000 2023-09-23T03:14:11.187 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2023-09-23 03:11:29.101833388 +0000 2023-09-23T03:14:11.187 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2023-09-23 03:11:29.101833388 +0000 2023-09-23T03:14:11.187 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2023-09-23 03:11:29.101833388 +0000 2023-09-23T03:14:11.187 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-23T03:14:11.278 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2023-09-23T03:14:11.278 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2023-09-23T03:14:11.278 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000348099 s, 1.5 MB/s 2023-09-23T03:14:11.282 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-23T03:14:11.339 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-23T03:14:11.339 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:14:11.339 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/scratch_devs of=/dev/stdout 2023-09-23T03:14:11.398 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-23T03:14:11.398 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_1 2023-09-23T03:14:11.449 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-23T03:14:11.449 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-23T03:14:11.449 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 628 Links: 1 2023-09-23T03:14:11.449 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-23T03:14:11.449 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2023-09-23T03:14:11.449 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2023-09-23 03:14:00.990487302 +0000 2023-09-23T03:14:11.449 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2023-09-23 03:11:24.417474031 +0000 2023-09-23T03:14:11.449 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2023-09-23 03:11:24.417474031 +0000 2023-09-23T03:14:11.449 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2023-09-23 03:11:24.417474031 +0000 2023-09-23T03:14:11.450 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-23T03:14:11.534 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2023-09-23T03:14:11.534 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2023-09-23T03:14:11.534 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000406713 s, 1.3 MB/s 2023-09-23T03:14:11.537 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-23T03:14:11.584 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_2 2023-09-23T03:14:11.631 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-23T03:14:11.631 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-23T03:14:11.631 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 647 Links: 1 2023-09-23T03:14:11.631 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-23T03:14:11.631 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2023-09-23T03:14:11.631 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2023-09-23 03:14:00.991487274 +0000 2023-09-23T03:14:11.631 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2023-09-23 03:11:25.413446636 +0000 2023-09-23T03:14:11.631 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2023-09-23 03:11:25.413446636 +0000 2023-09-23T03:14:11.632 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2023-09-23 03:11:25.413446636 +0000 2023-09-23T03:14:11.632 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-23T03:14:11.711 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2023-09-23T03:14:11.711 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2023-09-23T03:14:11.711 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000332303 s, 1.5 MB/s 2023-09-23T03:14:11.714 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-23T03:14:11.765 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_3 2023-09-23T03:14:11.812 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-23T03:14:11.813 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-23T03:14:11.813 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 663 Links: 1 2023-09-23T03:14:11.813 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-23T03:14:11.813 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2023-09-23T03:14:11.813 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2023-09-23 03:14:00.991487274 +0000 2023-09-23T03:14:11.813 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2023-09-23 03:11:26.399419517 +0000 2023-09-23T03:14:11.813 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2023-09-23 03:11:26.399419517 +0000 2023-09-23T03:14:11.813 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2023-09-23 03:11:26.399419517 +0000 2023-09-23T03:14:11.813 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-23T03:14:11.900 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2023-09-23T03:14:11.901 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2023-09-23T03:14:11.901 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000327757 s, 1.6 MB/s 2023-09-23T03:14:11.904 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-23T03:14:11.956 DEBUG:teuthology.orchestra.run.smithi190:> stat /dev/vg_nvme/lv_4 2023-09-23T03:14:12.005 INFO:teuthology.orchestra.run.smithi190.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-23T03:14:12.005 INFO:teuthology.orchestra.run.smithi190.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-23T03:14:12.006 INFO:teuthology.orchestra.run.smithi190.stdout:Device: 5h/5d Inode: 682 Links: 1 2023-09-23T03:14:12.006 INFO:teuthology.orchestra.run.smithi190.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-23T03:14:12.006 INFO:teuthology.orchestra.run.smithi190.stdout:Context: system_u:object_r:device_t:s0 2023-09-23T03:14:12.006 INFO:teuthology.orchestra.run.smithi190.stdout:Access: 2023-09-23 03:14:00.993487219 +0000 2023-09-23T03:14:12.006 INFO:teuthology.orchestra.run.smithi190.stdout:Modify: 2023-09-23 03:11:27.431391132 +0000 2023-09-23T03:14:12.006 INFO:teuthology.orchestra.run.smithi190.stdout:Change: 2023-09-23 03:11:27.431391132 +0000 2023-09-23T03:14:12.006 INFO:teuthology.orchestra.run.smithi190.stdout: Birth: 2023-09-23 03:11:27.431391132 +0000 2023-09-23T03:14:12.006 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-23T03:14:12.096 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records in 2023-09-23T03:14:12.096 INFO:teuthology.orchestra.run.smithi190.stderr:1+0 records out 2023-09-23T03:14:12.096 INFO:teuthology.orchestra.run.smithi190.stderr:512 bytes copied, 0.000350296 s, 1.5 MB/s 2023-09-23T03:14:12.099 DEBUG:teuthology.orchestra.run.smithi190:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-23T03:14:12.157 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-09-23T03:14:12.158 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi154.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi190.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-09-23T03:14:12.158 INFO:tasks.ceph:Generating config... 2023-09-23T03:14:12.159 INFO:tasks.ceph:[client] client mount timeout = 600 2023-09-23T03:14:12.159 INFO:tasks.ceph:[client] debug client = 20 2023-09-23T03:14:12.159 INFO:tasks.ceph:[client] debug ms = 1 2023-09-23T03:14:12.159 INFO:tasks.ceph:[client] pid file = 2023-09-23T03:14:12.159 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-09-23T03:14:12.160 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-09-23T03:14:12.160 INFO:tasks.ceph:[mds] debug mds = 20 2023-09-23T03:14:12.160 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-09-23T03:14:12.160 INFO:tasks.ceph:[mds] debug ms = 1 2023-09-23T03:14:12.160 INFO:tasks.ceph:[mds] mds debug frag = True 2023-09-23T03:14:12.160 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-09-23T03:14:12.160 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-09-23T03:14:12.160 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-09-23T03:14:12.160 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-09-23T03:14:12.160 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-09-23T03:14:12.160 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-09-23T03:14:12.160 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-09-23T03:14:12.161 INFO:tasks.ceph:[mgr] debug ms = 1 2023-09-23T03:14:12.161 INFO:tasks.ceph:[mon] debug mon = 20 2023-09-23T03:14:12.161 INFO:tasks.ceph:[mon] debug ms = 1 2023-09-23T03:14:12.161 INFO:tasks.ceph:[mon] debug paxos = 20 2023-09-23T03:14:12.161 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-09-23T03:14:12.161 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-09-23T03:14:12.161 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-09-23T03:14:12.161 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-09-23T03:14:12.161 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-09-23T03:14:12.161 INFO:tasks.ceph:[osd] debug ms = 1 2023-09-23T03:14:12.161 INFO:tasks.ceph:[osd] debug osd = 20 2023-09-23T03:14:12.161 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-09-23T03:14:12.162 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-09-23T03:14:12.162 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-09-23T03:14:12.162 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-09-23T03:14:12.162 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-09-23T03:14:12.162 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-09-23T03:14:12.162 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-09-23T03:14:12.162 INFO:tasks.ceph:Setting up mon.a... 2023-09-23T03:14:12.162 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-09-23T03:14:12.373 INFO:teuthology.orchestra.run.smithi154.stdout:creating /etc/ceph/ceph.keyring 2023-09-23T03:14:12.382 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-09-23T03:14:12.521 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-09-23T03:14:12.617 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.154'), ('mon.b', '172.21.15.190'), ('mon.c', '[v2:172.21.15.190:3301,v1:172.21.15.190:6790]')] 2023-09-23T03:14:12.617 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.154,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-09-23T03:14:12.618 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:14:12.618 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-09-23T03:14:12.672 DEBUG:teuthology.orchestra.run.smithi154:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.154 --add b 172.21.15.190 --addv c '[v2:172.21.15.190:3301,v1:172.21.15.190:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-09-23T03:14:12.781 INFO:teuthology.orchestra.run.smithi154.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-09-23T03:14:12.781 INFO:teuthology.orchestra.run.smithi154.stdout:monmaptool: generated fsid 8c8039d3-6c84-43db-8808-85e0dbdd47c8 2023-09-23T03:14:12.782 INFO:teuthology.orchestra.run.smithi154.stdout:setting min_mon_release = pacific 2023-09-23T03:14:12.782 INFO:teuthology.orchestra.run.smithi154.stdout:epoch 0 2023-09-23T03:14:12.782 INFO:teuthology.orchestra.run.smithi154.stdout:fsid 8c8039d3-6c84-43db-8808-85e0dbdd47c8 2023-09-23T03:14:12.782 INFO:teuthology.orchestra.run.smithi154.stdout:last_changed 2023-09-23T03:14:12.782019+0000 2023-09-23T03:14:12.782 INFO:teuthology.orchestra.run.smithi154.stdout:created 2023-09-23T03:14:12.782019+0000 2023-09-23T03:14:12.782 INFO:teuthology.orchestra.run.smithi154.stdout:min_mon_release 16 (pacific) 2023-09-23T03:14:12.782 INFO:teuthology.orchestra.run.smithi154.stdout:election_strategy: 1 2023-09-23T03:14:12.782 INFO:teuthology.orchestra.run.smithi154.stdout:0: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.a 2023-09-23T03:14:12.782 INFO:teuthology.orchestra.run.smithi154.stdout:1: [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] mon.b 2023-09-23T03:14:12.782 INFO:teuthology.orchestra.run.smithi154.stdout:2: [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] mon.c 2023-09-23T03:14:12.783 INFO:teuthology.orchestra.run.smithi154.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-09-23T03:14:12.785 DEBUG:teuthology.orchestra.run.smithi154:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-09-23T03:14:12.836 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 8c8039d3-6c84-43db-8808-85e0dbdd47c8... 2023-09-23T03:14:12.837 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-23T03:14:12.842 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-23T03:14:13.013 INFO:teuthology.orchestra.run.smithi154.stdout:[global] 2023-09-23T03:14:13.013 INFO:teuthology.orchestra.run.smithi154.stdout: chdir = "" 2023-09-23T03:14:13.014 INFO:teuthology.orchestra.run.smithi154.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-23T03:14:13.014 INFO:teuthology.orchestra.run.smithi154.stdout: auth supported = cephx 2023-09-23T03:14:13.014 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:14:13.014 INFO:teuthology.orchestra.run.smithi154.stdout: filestore xattr use omap = true 2023-09-23T03:14:13.014 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:14:13.014 INFO:teuthology.orchestra.run.smithi154.stdout: mon clock drift allowed = 1.000 2023-09-23T03:14:13.014 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:14:13.014 INFO:teuthology.orchestra.run.smithi154.stdout: osd crush chooseleaf type = 0 2023-09-23T03:14:13.014 INFO:teuthology.orchestra.run.smithi154.stdout: auth debug = true 2023-09-23T03:14:13.014 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:14:13.014 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on old message = true 2023-09-23T03:14:13.015 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on bug = true 2023-09-23T03:14:13.015 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:14:13.015 INFO:teuthology.orchestra.run.smithi154.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-23T03:14:13.015 INFO:teuthology.orchestra.run.smithi154.stdout: mon pg warn max object skew = 0 2023-09-23T03:14:13.015 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:14:13.015 INFO:teuthology.orchestra.run.smithi154.stdout: # disable pg_autoscaler by default for new pools 2023-09-23T03:14:13.015 INFO:teuthology.orchestra.run.smithi154.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-23T03:14:13.015 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:14:13.015 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default size = 2 2023-09-23T03:14:13.015 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:14:13.015 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow primary affinity = true 2023-09-23T03:14:13.015 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow pg remap = true 2023-09-23T03:14:13.016 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on legacy crush tunables = false 2023-09-23T03:14:13.016 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on crush straw calc version zero = false 2023-09-23T03:14:13.016 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on no sortbitwise = false 2023-09-23T03:14:13.016 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on osd down out interval zero = false 2023-09-23T03:14:13.016 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on too few osds = false 2023-09-23T03:14:13.016 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-23T03:14:13.016 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-23T03:14:13.016 INFO:teuthology.orchestra.run.smithi154.stdout: mon_allow_pool_size_one = true 2023-09-23T03:14:13.016 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:14:13.016 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-23T03:14:13.016 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:14:13.016 INFO:teuthology.orchestra.run.smithi154.stdout: osd default data pool replay window = 5 2023-09-23T03:14:13.017 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:14:13.017 INFO:teuthology.orchestra.run.smithi154.stdout: mon allow pool delete = true 2023-09-23T03:14:13.017 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:14:13.017 INFO:teuthology.orchestra.run.smithi154.stdout: mon cluster log file level = debug 2023-09-23T03:14:13.017 INFO:teuthology.orchestra.run.smithi154.stdout: debug asserts on shutdown = true 2023-09-23T03:14:13.017 INFO:teuthology.orchestra.run.smithi154.stdout: mon health detail to clog = false 2023-09-23T03:14:13.017 INFO:teuthology.orchestra.run.smithi154.stdout: mon host = "172.21.15.154,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]" 2023-09-23T03:14:13.017 INFO:teuthology.orchestra.run.smithi154.stdout: fsid = 8c8039d3-6c84-43db-8808-85e0dbdd47c8 2023-09-23T03:14:13.017 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:14:13.017 INFO:teuthology.orchestra.run.smithi154.stdout:[osd] 2023-09-23T03:14:13.017 INFO:teuthology.orchestra.run.smithi154.stdout: osd journal size = 100 2023-09-23T03:14:13.017 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:14:13.017 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub load threshold = 5.0 2023-09-23T03:14:13.018 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub max interval = 600 2023-09-23T03:14:13.018 INFO:teuthology.orchestra.run.smithi154.stdout: osd mclock profile = high_recovery_ops 2023-09-23T03:14:13.018 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:14:13.018 INFO:teuthology.orchestra.run.smithi154.stdout: osd recover clone overlap = true 2023-09-23T03:14:13.018 INFO:teuthology.orchestra.run.smithi154.stdout: osd recovery max chunk = 1048576 2023-09-23T03:14:13.018 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:14:13.018 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug shutdown = true 2023-09-23T03:14:13.018 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug op order = true 2023-09-23T03:14:13.018 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug verify stray on activate = true 2023-09-23T03:14:13.018 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:14:13.019 INFO:teuthology.orchestra.run.smithi154.stdout: osd open classes on start = true 2023-09-23T03:14:13.019 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug pg log writeout = true 2023-09-23T03:14:13.019 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:14:13.019 INFO:teuthology.orchestra.run.smithi154.stdout: osd deep scrub update digest min age = 30 2023-09-23T03:14:13.019 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:14:13.019 INFO:teuthology.orchestra.run.smithi154.stdout: osd map max advance = 10 2023-09-23T03:14:13.019 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:14:13.019 INFO:teuthology.orchestra.run.smithi154.stdout: journal zero on create = true 2023-09-23T03:14:13.019 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:14:13.020 INFO:teuthology.orchestra.run.smithi154.stdout: filestore ondisk finisher threads = 3 2023-09-23T03:14:13.020 INFO:teuthology.orchestra.run.smithi154.stdout: filestore apply finisher threads = 3 2023-09-23T03:14:13.020 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:14:13.020 INFO:teuthology.orchestra.run.smithi154.stdout: bdev debug aio = true 2023-09-23T03:14:13.020 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug misdirected ops = true 2023-09-23T03:14:13.020 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore block size = 96636764160 2023-09-23T03:14:13.020 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore fsck on mount = True 2023-09-23T03:14:13.020 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluefs = 20 2023-09-23T03:14:13.020 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluestore = 20 2023-09-23T03:14:13.020 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:14:13.020 INFO:teuthology.orchestra.run.smithi154.stdout: debug osd = 20 2023-09-23T03:14:13.021 INFO:teuthology.orchestra.run.smithi154.stdout: debug rocksdb = 10 2023-09-23T03:14:13.021 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd backfillfull_ratio = 0.85 2023-09-23T03:14:13.021 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd full ratio = 0.9 2023-09-23T03:14:13.021 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd nearfull ratio = 0.8 2023-09-23T03:14:13.021 INFO:teuthology.orchestra.run.smithi154.stdout: osd failsafe full ratio = 0.95 2023-09-23T03:14:13.021 INFO:teuthology.orchestra.run.smithi154.stdout: osd objectstore = bluestore 2023-09-23T03:14:13.021 INFO:teuthology.orchestra.run.smithi154.stdout: osd op complaint time = 180 2023-09-23T03:14:13.021 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:14:13.021 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr] 2023-09-23T03:14:13.021 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:14:13.021 INFO:teuthology.orchestra.run.smithi154.stdout: debug mgr = 20 2023-09-23T03:14:13.021 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2023-09-23T03:14:13.022 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2023-09-23T03:14:13.022 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:14:13.022 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:14:13.022 INFO:teuthology.orchestra.run.smithi154.stdout: mgr/telemetry/nag = false 2023-09-23T03:14:13.022 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:14:13.022 INFO:teuthology.orchestra.run.smithi154.stdout:[mon] 2023-09-23T03:14:13.022 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:14:13.022 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2023-09-23T03:14:13.022 INFO:teuthology.orchestra.run.smithi154.stdout: debug paxos = 20 2023-09-23T03:14:13.022 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2023-09-23T03:14:13.023 INFO:teuthology.orchestra.run.smithi154.stdout: mon data avail warn = 5 2023-09-23T03:14:13.023 INFO:teuthology.orchestra.run.smithi154.stdout: mon mgr mkfs grace = 240 2023-09-23T03:14:13.023 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:14:13.023 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd reporter subtree level = osd 2023-09-23T03:14:13.023 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd prime pg temp = true 2023-09-23T03:14:13.023 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:14:13.023 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:14:13.023 INFO:teuthology.orchestra.run.smithi154.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-23T03:14:13.023 INFO:teuthology.orchestra.run.smithi154.stdout: auth mon ticket ttl = 660 # 11m 2023-09-23T03:14:13.023 INFO:teuthology.orchestra.run.smithi154.stdout: auth service ticket ttl = 240 # 4m 2023-09-23T03:14:13.023 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:14:13.023 INFO:teuthology.orchestra.run.smithi154.stdout: # don't complain about insecure global_id in the test suite 2023-09-23T03:14:13.024 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-23T03:14:13.024 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-23T03:14:13.024 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:14:13.024 INFO:teuthology.orchestra.run.smithi154.stdout: # 1m isn't quite enough 2023-09-23T03:14:13.024 INFO:teuthology.orchestra.run.smithi154.stdout: mon_down_mkfs_grace = 2m 2023-09-23T03:14:13.024 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:14:13.024 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_filestore_osds = false 2023-09-23T03:14:13.024 INFO:teuthology.orchestra.run.smithi154.stdout: mon op complaint time = 120 2023-09-23T03:14:13.024 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:14:13.024 INFO:teuthology.orchestra.run.smithi154.stdout:[client] 2023-09-23T03:14:13.024 INFO:teuthology.orchestra.run.smithi154.stdout: rgw cache enabled = true 2023-09-23T03:14:13.025 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable ops log = true 2023-09-23T03:14:13.025 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable usage log = true 2023-09-23T03:14:13.025 INFO:teuthology.orchestra.run.smithi154.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-23T03:14:13.025 INFO:teuthology.orchestra.run.smithi154.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-23T03:14:13.025 INFO:teuthology.orchestra.run.smithi154.stdout: client mount timeout = 600 2023-09-23T03:14:13.025 INFO:teuthology.orchestra.run.smithi154.stdout: debug client = 20 2023-09-23T03:14:13.025 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:14:13.025 INFO:teuthology.orchestra.run.smithi154.stdout: pid file = "" 2023-09-23T03:14:13.025 INFO:teuthology.orchestra.run.smithi154.stdout: rados mon op timeout = 900 2023-09-23T03:14:13.025 INFO:teuthology.orchestra.run.smithi154.stdout: rados osd op timeout = 900 2023-09-23T03:14:13.025 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.a] 2023-09-23T03:14:13.025 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.b] 2023-09-23T03:14:13.025 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.c] 2023-09-23T03:14:13.026 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.a] 2023-09-23T03:14:13.026 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.c] 2023-09-23T03:14:13.026 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.b] 2023-09-23T03:14:13.026 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.d] 2023-09-23T03:14:13.026 INFO:teuthology.orchestra.run.smithi154.stdout:[mds] 2023-09-23T03:14:13.026 INFO:teuthology.orchestra.run.smithi154.stdout: debug mds = 20 2023-09-23T03:14:13.026 INFO:teuthology.orchestra.run.smithi154.stdout: debug mds balancer = 20 2023-09-23T03:14:13.026 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:14:13.026 INFO:teuthology.orchestra.run.smithi154.stdout: mds debug frag = True 2023-09-23T03:14:13.026 INFO:teuthology.orchestra.run.smithi154.stdout: mds debug scatterstat = True 2023-09-23T03:14:13.027 INFO:teuthology.orchestra.run.smithi154.stdout: mds op complaint time = 180 2023-09-23T03:14:13.027 INFO:teuthology.orchestra.run.smithi154.stdout: mds verify scatter = True 2023-09-23T03:14:13.027 INFO:teuthology.orchestra.run.smithi154.stdout: osd op complaint time = 180 2023-09-23T03:14:13.027 INFO:teuthology.orchestra.run.smithi154.stdout: rados mon op timeout = 900 2023-09-23T03:14:13.027 INFO:teuthology.orchestra.run.smithi154.stdout: rados osd op timeout = 900 2023-09-23T03:14:13.028 INFO:teuthology.orchestra.run.smithi190.stdout:[global] 2023-09-23T03:14:13.028 INFO:teuthology.orchestra.run.smithi190.stdout: chdir = "" 2023-09-23T03:14:13.028 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-23T03:14:13.028 INFO:teuthology.orchestra.run.smithi190.stdout: auth supported = cephx 2023-09-23T03:14:13.028 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:14:13.028 INFO:teuthology.orchestra.run.smithi190.stdout: filestore xattr use omap = true 2023-09-23T03:14:13.028 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:14:13.028 INFO:teuthology.orchestra.run.smithi190.stdout: mon clock drift allowed = 1.000 2023-09-23T03:14:13.028 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:14:13.028 INFO:teuthology.orchestra.run.smithi190.stdout: osd crush chooseleaf type = 0 2023-09-23T03:14:13.029 INFO:teuthology.orchestra.run.smithi190.stdout: auth debug = true 2023-09-23T03:14:13.029 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:14:13.029 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on old message = true 2023-09-23T03:14:13.029 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on bug = true 2023-09-23T03:14:13.029 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:14:13.029 INFO:teuthology.orchestra.run.smithi190.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-23T03:14:13.029 INFO:teuthology.orchestra.run.smithi190.stdout: mon pg warn max object skew = 0 2023-09-23T03:14:13.029 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:14:13.029 INFO:teuthology.orchestra.run.smithi190.stdout: # disable pg_autoscaler by default for new pools 2023-09-23T03:14:13.029 INFO:teuthology.orchestra.run.smithi190.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-23T03:14:13.029 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:14:13.029 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default size = 2 2023-09-23T03:14:13.030 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:14:13.030 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow primary affinity = true 2023-09-23T03:14:13.030 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow pg remap = true 2023-09-23T03:14:13.030 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on legacy crush tunables = false 2023-09-23T03:14:13.030 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on crush straw calc version zero = false 2023-09-23T03:14:13.030 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on no sortbitwise = false 2023-09-23T03:14:13.030 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on osd down out interval zero = false 2023-09-23T03:14:13.030 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on too few osds = false 2023-09-23T03:14:13.030 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-23T03:14:13.031 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-23T03:14:13.031 INFO:teuthology.orchestra.run.smithi190.stdout: mon_allow_pool_size_one = true 2023-09-23T03:14:13.031 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:14:13.031 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-23T03:14:13.031 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:14:13.031 INFO:teuthology.orchestra.run.smithi190.stdout: osd default data pool replay window = 5 2023-09-23T03:14:13.031 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:14:13.031 INFO:teuthology.orchestra.run.smithi190.stdout: mon allow pool delete = true 2023-09-23T03:14:13.031 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:14:13.031 INFO:teuthology.orchestra.run.smithi190.stdout: mon cluster log file level = debug 2023-09-23T03:14:13.031 INFO:teuthology.orchestra.run.smithi190.stdout: debug asserts on shutdown = true 2023-09-23T03:14:13.031 INFO:teuthology.orchestra.run.smithi190.stdout: mon health detail to clog = false 2023-09-23T03:14:13.032 INFO:teuthology.orchestra.run.smithi190.stdout: mon host = "172.21.15.154,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]" 2023-09-23T03:14:13.032 INFO:teuthology.orchestra.run.smithi190.stdout: fsid = 8c8039d3-6c84-43db-8808-85e0dbdd47c8 2023-09-23T03:14:13.032 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:14:13.032 INFO:teuthology.orchestra.run.smithi190.stdout:[osd] 2023-09-23T03:14:13.032 INFO:teuthology.orchestra.run.smithi190.stdout: osd journal size = 100 2023-09-23T03:14:13.032 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:14:13.032 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub load threshold = 5.0 2023-09-23T03:14:13.032 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub max interval = 600 2023-09-23T03:14:13.032 INFO:teuthology.orchestra.run.smithi190.stdout: osd mclock profile = high_recovery_ops 2023-09-23T03:14:13.032 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:14:13.032 INFO:teuthology.orchestra.run.smithi190.stdout: osd recover clone overlap = true 2023-09-23T03:14:13.032 INFO:teuthology.orchestra.run.smithi190.stdout: osd recovery max chunk = 1048576 2023-09-23T03:14:13.033 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:14:13.033 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug shutdown = true 2023-09-23T03:14:13.033 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug op order = true 2023-09-23T03:14:13.033 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug verify stray on activate = true 2023-09-23T03:14:13.033 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:14:13.033 INFO:teuthology.orchestra.run.smithi190.stdout: osd open classes on start = true 2023-09-23T03:14:13.033 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug pg log writeout = true 2023-09-23T03:14:13.033 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:14:13.033 INFO:teuthology.orchestra.run.smithi190.stdout: osd deep scrub update digest min age = 30 2023-09-23T03:14:13.033 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:14:13.033 INFO:teuthology.orchestra.run.smithi190.stdout: osd map max advance = 10 2023-09-23T03:14:13.033 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:14:13.034 INFO:teuthology.orchestra.run.smithi190.stdout: journal zero on create = true 2023-09-23T03:14:13.034 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:14:13.034 INFO:teuthology.orchestra.run.smithi190.stdout: filestore ondisk finisher threads = 3 2023-09-23T03:14:13.034 INFO:teuthology.orchestra.run.smithi190.stdout: filestore apply finisher threads = 3 2023-09-23T03:14:13.034 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:14:13.034 INFO:teuthology.orchestra.run.smithi190.stdout: bdev debug aio = true 2023-09-23T03:14:13.034 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug misdirected ops = true 2023-09-23T03:14:13.034 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore block size = 96636764160 2023-09-23T03:14:13.034 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore fsck on mount = True 2023-09-23T03:14:13.034 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluefs = 20 2023-09-23T03:14:13.034 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluestore = 20 2023-09-23T03:14:13.034 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:14:13.035 INFO:teuthology.orchestra.run.smithi190.stdout: debug osd = 20 2023-09-23T03:14:13.035 INFO:teuthology.orchestra.run.smithi190.stdout: debug rocksdb = 10 2023-09-23T03:14:13.035 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd backfillfull_ratio = 0.85 2023-09-23T03:14:13.035 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd full ratio = 0.9 2023-09-23T03:14:13.035 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd nearfull ratio = 0.8 2023-09-23T03:14:13.035 INFO:teuthology.orchestra.run.smithi190.stdout: osd failsafe full ratio = 0.95 2023-09-23T03:14:13.035 INFO:teuthology.orchestra.run.smithi190.stdout: osd objectstore = bluestore 2023-09-23T03:14:13.035 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2023-09-23T03:14:13.035 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:14:13.035 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr] 2023-09-23T03:14:13.035 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:14:13.035 INFO:teuthology.orchestra.run.smithi190.stdout: debug mgr = 20 2023-09-23T03:14:13.036 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2023-09-23T03:14:13.036 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2023-09-23T03:14:13.036 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:14:13.036 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:14:13.036 INFO:teuthology.orchestra.run.smithi190.stdout: mgr/telemetry/nag = false 2023-09-23T03:14:13.036 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:14:13.036 INFO:teuthology.orchestra.run.smithi190.stdout:[mon] 2023-09-23T03:14:13.036 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:14:13.036 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2023-09-23T03:14:13.036 INFO:teuthology.orchestra.run.smithi190.stdout: debug paxos = 20 2023-09-23T03:14:13.036 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2023-09-23T03:14:13.036 INFO:teuthology.orchestra.run.smithi190.stdout: mon data avail warn = 5 2023-09-23T03:14:13.037 INFO:teuthology.orchestra.run.smithi190.stdout: mon mgr mkfs grace = 240 2023-09-23T03:14:13.037 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:14:13.037 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd reporter subtree level = osd 2023-09-23T03:14:13.037 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd prime pg temp = true 2023-09-23T03:14:13.037 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:14:13.037 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:14:13.037 INFO:teuthology.orchestra.run.smithi190.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-23T03:14:13.037 INFO:teuthology.orchestra.run.smithi190.stdout: auth mon ticket ttl = 660 # 11m 2023-09-23T03:14:13.037 INFO:teuthology.orchestra.run.smithi190.stdout: auth service ticket ttl = 240 # 4m 2023-09-23T03:14:13.037 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:14:13.037 INFO:teuthology.orchestra.run.smithi190.stdout: # don't complain about insecure global_id in the test suite 2023-09-23T03:14:13.037 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-23T03:14:13.038 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-23T03:14:13.038 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:14:13.038 INFO:teuthology.orchestra.run.smithi190.stdout: # 1m isn't quite enough 2023-09-23T03:14:13.038 INFO:teuthology.orchestra.run.smithi190.stdout: mon_down_mkfs_grace = 2m 2023-09-23T03:14:13.038 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:14:13.038 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_filestore_osds = false 2023-09-23T03:14:13.038 INFO:teuthology.orchestra.run.smithi190.stdout: mon op complaint time = 120 2023-09-23T03:14:13.038 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:14:13.038 INFO:teuthology.orchestra.run.smithi190.stdout:[client] 2023-09-23T03:14:13.038 INFO:teuthology.orchestra.run.smithi190.stdout: rgw cache enabled = true 2023-09-23T03:14:13.038 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable ops log = true 2023-09-23T03:14:13.038 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable usage log = true 2023-09-23T03:14:13.039 INFO:teuthology.orchestra.run.smithi190.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-23T03:14:13.039 INFO:teuthology.orchestra.run.smithi190.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-23T03:14:13.039 INFO:teuthology.orchestra.run.smithi190.stdout: client mount timeout = 600 2023-09-23T03:14:13.039 INFO:teuthology.orchestra.run.smithi190.stdout: debug client = 20 2023-09-23T03:14:13.039 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:14:13.039 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = "" 2023-09-23T03:14:13.039 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 900 2023-09-23T03:14:13.039 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 900 2023-09-23T03:14:13.039 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.a] 2023-09-23T03:14:13.039 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.b] 2023-09-23T03:14:13.039 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.c] 2023-09-23T03:14:13.039 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.a] 2023-09-23T03:14:13.039 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.c] 2023-09-23T03:14:13.040 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.b] 2023-09-23T03:14:13.040 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.d] 2023-09-23T03:14:13.040 INFO:teuthology.orchestra.run.smithi190.stdout:[mds] 2023-09-23T03:14:13.040 INFO:teuthology.orchestra.run.smithi190.stdout: debug mds = 20 2023-09-23T03:14:13.040 INFO:teuthology.orchestra.run.smithi190.stdout: debug mds balancer = 20 2023-09-23T03:14:13.040 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:14:13.040 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug frag = True 2023-09-23T03:14:13.040 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug scatterstat = True 2023-09-23T03:14:13.040 INFO:teuthology.orchestra.run.smithi190.stdout: mds op complaint time = 180 2023-09-23T03:14:13.040 INFO:teuthology.orchestra.run.smithi190.stdout: mds verify scatter = True 2023-09-23T03:14:13.040 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2023-09-23T03:14:13.040 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 900 2023-09-23T03:14:13.041 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 900 2023-09-23T03:14:13.060 INFO:tasks.ceph:Creating admin key on mon.a... 2023-09-23T03:14:13.060 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-09-23T03:14:13.202 INFO:tasks.ceph:Copying monmap to all nodes... 2023-09-23T03:14:13.203 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:14:13.203 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-09-23T03:14:13.258 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:14:13.258 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-09-23T03:14:13.309 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi154.front.sepia.ceph.com 2023-09-23T03:14:13.309 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:14:13.309 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/ceph/ceph.keyring 2023-09-23T03:14:13.309 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-09-23T03:14:13.463 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:14:13.464 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-09-23T03:14:13.562 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi190.front.sepia.ceph.com 2023-09-23T03:14:13.562 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:14:13.562 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd of=/etc/ceph/ceph.keyring 2023-09-23T03:14:13.562 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-09-23T03:14:13.695 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:14:13.695 DEBUG:teuthology.orchestra.run.smithi190:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-09-23T03:14:13.789 INFO:tasks.ceph:Setting up mon nodes... 2023-09-23T03:14:13.789 INFO:tasks.ceph:Setting up mgr nodes... 2023-09-23T03:14:13.789 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-09-23T03:14:13.982 INFO:teuthology.orchestra.run.smithi154.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-09-23T03:14:13.993 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-09-23T03:14:14.179 INFO:teuthology.orchestra.run.smithi190.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-09-23T03:14:14.186 INFO:tasks.ceph:Setting up mds nodes... 2023-09-23T03:14:14.187 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-09-23T03:14:14.377 INFO:teuthology.orchestra.run.smithi154.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-09-23T03:14:14.383 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-09-23T03:14:14.475 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-09-23T03:14:14.640 INFO:teuthology.orchestra.run.smithi154.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-09-23T03:14:14.648 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-09-23T03:14:14.746 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-09-23T03:14:14.931 INFO:teuthology.orchestra.run.smithi190.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-09-23T03:14:14.940 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-09-23T03:14:15.032 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-09-23T03:14:15.214 INFO:teuthology.orchestra.run.smithi190.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-09-23T03:14:15.223 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-09-23T03:14:15.311 INFO:tasks.ceph_client:Setting up client nodes... 2023-09-23T03:14:15.311 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-09-23T03:14:15.446 INFO:teuthology.orchestra.run.smithi154.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-09-23T03:14:15.494 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-09-23T03:14:15.620 INFO:teuthology.orchestra.run.smithi154.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-09-23T03:14:15.682 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-09-23T03:14:15.817 INFO:teuthology.orchestra.run.smithi190.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-09-23T03:14:15.874 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-09-23T03:14:16.001 INFO:teuthology.orchestra.run.smithi190.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-09-23T03:14:16.067 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-09-23T03:14:16.067 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi154.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi190.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-09-23T03:14:16.067 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-09-23T03:14:16.171 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-23T03:14:16.171 INFO:tasks.ceph:role: osd.0 2023-09-23T03:14:16.171 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi154.front.sepia.ceph.com 2023-09-23T03:14:16.172 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-09-23T03:14:16.275 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-09-23T03:14:16.276 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-23T03:14:16.276 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-23T03:14:16.276 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-09-23T03:14:16.276 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-23T03:14:16.276 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2023-09-23T03:14:16.277 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-23T03:14:16.277 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-09-23T03:14:16.277 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-23T03:14:16.277 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-23T03:14:16.798 INFO:teuthology.orchestra.run.smithi154.stdout:Discarding blocks...Done. 2023-09-23T03:14:16.805 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2023-09-23T03:14:16.805 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-09-23T03:14:16.971 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-09-23T03:14:17.077 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-09-23T03:14:17.159 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-23T03:14:17.159 INFO:tasks.ceph:role: osd.1 2023-09-23T03:14:17.159 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi154.front.sepia.ceph.com 2023-09-23T03:14:17.159 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-09-23T03:14:17.253 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-09-23T03:14:17.254 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-23T03:14:17.254 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-23T03:14:17.254 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-09-23T03:14:17.254 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-23T03:14:17.254 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2023-09-23T03:14:17.254 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-23T03:14:17.254 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-09-23T03:14:17.254 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-23T03:14:17.254 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-23T03:14:17.728 INFO:teuthology.orchestra.run.smithi154.stdout:Discarding blocks...Done. 2023-09-23T03:14:17.734 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2023-09-23T03:14:17.734 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-09-23T03:14:17.881 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-09-23T03:14:17.977 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-09-23T03:14:18.071 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-23T03:14:18.071 INFO:tasks.ceph:role: osd.2 2023-09-23T03:14:18.071 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi154.front.sepia.ceph.com 2023-09-23T03:14:18.072 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-09-23T03:14:18.172 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-09-23T03:14:18.172 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-23T03:14:18.172 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-23T03:14:18.172 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-09-23T03:14:18.173 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-23T03:14:18.173 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2023-09-23T03:14:18.173 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-23T03:14:18.173 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-09-23T03:14:18.173 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-23T03:14:18.173 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-23T03:14:18.689 INFO:teuthology.orchestra.run.smithi154.stdout:Discarding blocks...Done. 2023-09-23T03:14:18.695 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2023-09-23T03:14:18.695 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-09-23T03:14:18.896 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-09-23T03:14:19.002 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-09-23T03:14:19.095 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-23T03:14:19.095 INFO:tasks.ceph:role: osd.3 2023-09-23T03:14:19.096 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi154.front.sepia.ceph.com 2023-09-23T03:14:19.096 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-09-23T03:14:19.199 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-09-23T03:14:19.199 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-23T03:14:19.199 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-23T03:14:19.199 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-09-23T03:14:19.199 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-23T03:14:19.199 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2023-09-23T03:14:19.199 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-23T03:14:19.199 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-09-23T03:14:19.200 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-23T03:14:19.200 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-23T03:14:19.670 INFO:teuthology.orchestra.run.smithi154.stdout:Discarding blocks...Done. 2023-09-23T03:14:19.675 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2023-09-23T03:14:19.676 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-09-23T03:14:19.830 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-09-23T03:14:19.928 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-23T03:14:20.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:20.074+0000 7f03a8825500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-09-23T03:14:20.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:20.079+0000 7f03a8825500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-09-23T03:14:20.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:20.079+0000 7f03a8825500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-09-23T03:14:20.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:20.079+0000 7f03a8825500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-09-23T03:14:20.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:20.079+0000 7f03a8825500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-09-23T03:14:20.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:20.083+0000 7f03a8825500 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-09-23T03:14:23.258 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-09-23T03:14:23.356 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-23T03:14:23.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:23.500+0000 7f8570910500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-09-23T03:14:23.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:23.501+0000 7f8570910500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-09-23T03:14:23.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:23.501+0000 7f8570910500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-09-23T03:14:23.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:23.501+0000 7f8570910500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-09-23T03:14:23.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:23.501+0000 7f8570910500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-09-23T03:14:23.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:23.504+0000 7f8570910500 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-09-23T03:14:26.417 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-09-23T03:14:26.514 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-23T03:14:26.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:26.655+0000 7fedd994a500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-09-23T03:14:26.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:26.656+0000 7fedd994a500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-09-23T03:14:26.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:26.656+0000 7fedd994a500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-09-23T03:14:26.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:26.656+0000 7fedd994a500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-09-23T03:14:26.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:26.656+0000 7fedd994a500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-09-23T03:14:26.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:26.658+0000 7fedd994a500 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-09-23T03:14:29.594 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-09-23T03:14:29.693 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-23T03:14:29.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:29.837+0000 7fcc5e214500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-09-23T03:14:29.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:29.838+0000 7fcc5e214500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-09-23T03:14:29.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:29.838+0000 7fcc5e214500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-09-23T03:14:29.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:29.838+0000 7fcc5e214500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-09-23T03:14:29.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:29.838+0000 7fcc5e214500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-09-23T03:14:29.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:29.840+0000 7fcc5e214500 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-09-23T03:14:32.767 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-09-23T03:14:32.866 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-09-23T03:14:32.968 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-09-23T03:14:32.968 INFO:tasks.ceph:role: osd.4 2023-09-23T03:14:32.968 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi190.front.sepia.ceph.com 2023-09-23T03:14:32.968 DEBUG:teuthology.orchestra.run.smithi190:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-09-23T03:14:33.070 INFO:teuthology.orchestra.run.smithi190.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-09-23T03:14:33.070 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-23T03:14:33.070 INFO:teuthology.orchestra.run.smithi190.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-23T03:14:33.070 INFO:teuthology.orchestra.run.smithi190.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-09-23T03:14:33.070 INFO:teuthology.orchestra.run.smithi190.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-23T03:14:33.071 INFO:teuthology.orchestra.run.smithi190.stdout: = sunit=0 swidth=0 blks 2023-09-23T03:14:33.071 INFO:teuthology.orchestra.run.smithi190.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-23T03:14:33.071 INFO:teuthology.orchestra.run.smithi190.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-09-23T03:14:33.071 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-23T03:14:33.071 INFO:teuthology.orchestra.run.smithi190.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-23T03:14:33.588 INFO:teuthology.orchestra.run.smithi190.stdout:Discarding blocks...Done. 2023-09-23T03:14:33.596 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi190.front.sepia.ceph.com -o noatime 2023-09-23T03:14:33.596 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-09-23T03:14:33.783 DEBUG:teuthology.orchestra.run.smithi190:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-09-23T03:14:33.891 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-09-23T03:14:33.974 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-09-23T03:14:33.974 INFO:tasks.ceph:role: osd.5 2023-09-23T03:14:33.975 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi190.front.sepia.ceph.com 2023-09-23T03:14:33.975 DEBUG:teuthology.orchestra.run.smithi190:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-09-23T03:14:34.061 INFO:teuthology.orchestra.run.smithi190.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-09-23T03:14:34.061 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-23T03:14:34.062 INFO:teuthology.orchestra.run.smithi190.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-23T03:14:34.062 INFO:teuthology.orchestra.run.smithi190.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-09-23T03:14:34.062 INFO:teuthology.orchestra.run.smithi190.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-23T03:14:34.062 INFO:teuthology.orchestra.run.smithi190.stdout: = sunit=0 swidth=0 blks 2023-09-23T03:14:34.062 INFO:teuthology.orchestra.run.smithi190.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-23T03:14:34.062 INFO:teuthology.orchestra.run.smithi190.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-09-23T03:14:34.062 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-23T03:14:34.062 INFO:teuthology.orchestra.run.smithi190.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-23T03:14:34.540 INFO:teuthology.orchestra.run.smithi190.stdout:Discarding blocks...Done. 2023-09-23T03:14:34.546 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi190.front.sepia.ceph.com -o noatime 2023-09-23T03:14:34.546 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-09-23T03:14:34.677 DEBUG:teuthology.orchestra.run.smithi190:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-09-23T03:14:34.767 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-09-23T03:14:34.845 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-09-23T03:14:34.845 INFO:tasks.ceph:role: osd.6 2023-09-23T03:14:34.845 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi190.front.sepia.ceph.com 2023-09-23T03:14:34.845 DEBUG:teuthology.orchestra.run.smithi190:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-09-23T03:14:34.932 INFO:teuthology.orchestra.run.smithi190.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-09-23T03:14:34.933 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-23T03:14:34.933 INFO:teuthology.orchestra.run.smithi190.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-23T03:14:34.933 INFO:teuthology.orchestra.run.smithi190.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-09-23T03:14:34.933 INFO:teuthology.orchestra.run.smithi190.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-23T03:14:34.933 INFO:teuthology.orchestra.run.smithi190.stdout: = sunit=0 swidth=0 blks 2023-09-23T03:14:34.933 INFO:teuthology.orchestra.run.smithi190.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-23T03:14:34.933 INFO:teuthology.orchestra.run.smithi190.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-09-23T03:14:34.933 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-23T03:14:34.933 INFO:teuthology.orchestra.run.smithi190.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-23T03:14:35.449 INFO:teuthology.orchestra.run.smithi190.stdout:Discarding blocks...Done. 2023-09-23T03:14:35.454 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi190.front.sepia.ceph.com -o noatime 2023-09-23T03:14:35.454 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-09-23T03:14:35.644 DEBUG:teuthology.orchestra.run.smithi190:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-09-23T03:14:35.752 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-09-23T03:14:35.850 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-09-23T03:14:35.851 INFO:tasks.ceph:role: osd.7 2023-09-23T03:14:35.851 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi190.front.sepia.ceph.com 2023-09-23T03:14:35.851 DEBUG:teuthology.orchestra.run.smithi190:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-09-23T03:14:35.953 INFO:teuthology.orchestra.run.smithi190.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-09-23T03:14:35.953 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-23T03:14:35.954 INFO:teuthology.orchestra.run.smithi190.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-23T03:14:35.954 INFO:teuthology.orchestra.run.smithi190.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-09-23T03:14:35.954 INFO:teuthology.orchestra.run.smithi190.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-23T03:14:35.954 INFO:teuthology.orchestra.run.smithi190.stdout: = sunit=0 swidth=0 blks 2023-09-23T03:14:35.954 INFO:teuthology.orchestra.run.smithi190.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-23T03:14:35.954 INFO:teuthology.orchestra.run.smithi190.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-09-23T03:14:35.954 INFO:teuthology.orchestra.run.smithi190.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-23T03:14:35.954 INFO:teuthology.orchestra.run.smithi190.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-23T03:14:36.409 INFO:teuthology.orchestra.run.smithi190.stdout:Discarding blocks...Done. 2023-09-23T03:14:36.415 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi190.front.sepia.ceph.com -o noatime 2023-09-23T03:14:36.415 DEBUG:teuthology.orchestra.run.smithi190:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-09-23T03:14:36.567 DEBUG:teuthology.orchestra.run.smithi190:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-09-23T03:14:36.663 DEBUG:teuthology.orchestra.run.smithi190:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-23T03:14:36.816 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:36.816+0000 7ff24582b500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-09-23T03:14:36.822 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:36.821+0000 7ff24582b500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-09-23T03:14:36.822 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:36.821+0000 7ff24582b500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-09-23T03:14:36.822 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:36.821+0000 7ff24582b500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-09-23T03:14:36.822 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:36.821+0000 7ff24582b500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-09-23T03:14:36.825 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:36.825+0000 7ff24582b500 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-09-23T03:14:39.985 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-09-23T03:14:40.086 DEBUG:teuthology.orchestra.run.smithi190:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-23T03:14:40.227 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:40.227+0000 7f60ab012500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-09-23T03:14:40.228 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:40.227+0000 7f60ab012500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-09-23T03:14:40.228 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:40.228+0000 7f60ab012500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-09-23T03:14:40.228 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:40.228+0000 7f60ab012500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-09-23T03:14:40.228 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:40.228+0000 7f60ab012500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-09-23T03:14:40.230 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:40.230+0000 7f60ab012500 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-09-23T03:14:40.316 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:14:40.325 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:14:43.399 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-09-23T03:14:43.491 DEBUG:teuthology.orchestra.run.smithi190:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-23T03:14:43.631 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:43.631+0000 7f2a1b8b6500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-09-23T03:14:43.632 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:43.632+0000 7f2a1b8b6500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-09-23T03:14:43.632 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:43.632+0000 7f2a1b8b6500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-09-23T03:14:43.632 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:43.632+0000 7f2a1b8b6500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-09-23T03:14:43.632 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:43.632+0000 7f2a1b8b6500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-09-23T03:14:43.634 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:43.634+0000 7f2a1b8b6500 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-09-23T03:14:46.817 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-09-23T03:14:46.919 DEBUG:teuthology.orchestra.run.smithi190:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-23T03:14:47.061 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:47.061+0000 7fdeff014500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-09-23T03:14:47.062 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:47.061+0000 7fdeff014500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-09-23T03:14:47.062 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:47.061+0000 7fdeff014500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-09-23T03:14:47.062 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:47.061+0000 7fdeff014500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-09-23T03:14:47.062 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:47.061+0000 7fdeff014500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-09-23T03:14:47.064 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:47.064+0000 7fdeff014500 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-09-23T03:14:50.234 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-09-23T03:14:50.334 INFO:tasks.ceph:Reading keys from all nodes... 2023-09-23T03:14:50.334 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:14:50.334 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-09-23T03:14:50.436 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:14:50.436 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-09-23T03:14:50.531 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:14:50.531 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-09-23T03:14:50.629 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:14:50.630 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-09-23T03:14:50.730 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:14:50.730 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-09-23T03:14:50.827 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:14:50.827 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-09-23T03:14:50.924 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:14:50.924 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-09-23T03:14:51.019 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:14:51.019 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-09-23T03:14:51.122 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:14:51.122 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-09-23T03:14:51.221 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:14:51.221 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-09-23T03:14:51.319 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:14:51.319 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-09-23T03:14:51.418 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:14:51.420 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-09-23T03:14:51.515 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:14:51.515 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-09-23T03:14:51.611 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:14:51.611 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-09-23T03:14:51.707 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:14:51.707 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-09-23T03:14:51.767 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:14:51.767 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-09-23T03:14:51.822 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:14:51.822 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-09-23T03:14:51.881 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:14:51.881 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-09-23T03:14:51.933 INFO:tasks.ceph:Adding keys to all mons... 2023-09-23T03:14:51.933 DEBUG:teuthology.orchestra.run.smithi154:> sudo tee -a /etc/ceph/ceph.keyring 2023-09-23T03:14:51.942 DEBUG:teuthology.orchestra.run.smithi190:> sudo tee -a /etc/ceph/ceph.keyring 2023-09-23T03:14:52.034 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr.y] 2023-09-23T03:14:52.034 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQAFWA5looGaOhAAdxmefaoLuXesHXCeqeCYXQ== 2023-09-23T03:14:52.035 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.a] 2023-09-23T03:14:52.035 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQAGWA5l9biCFhAAhD2xgY9Q/gGqK0pgHRGweQ== 2023-09-23T03:14:52.035 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.c] 2023-09-23T03:14:52.035 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQAGWA5los4vJhAA2/ewjMZcc3pnm9ukhnoNgw== 2023-09-23T03:14:52.035 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.0] 2023-09-23T03:14:52.035 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQAMWA5lT1R4BBAAI+itqub/FadG3aU6LhnmGg== 2023-09-23T03:14:52.035 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.1] 2023-09-23T03:14:52.035 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQAPWA5lSablHRAAmNyr5Zopeihn5ZyoY5pyhA== 2023-09-23T03:14:52.035 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.2] 2023-09-23T03:14:52.035 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQASWA5lSFcdJxAA3UJJ6eSvxcQ31SAm4shV2g== 2023-09-23T03:14:52.035 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.3] 2023-09-23T03:14:52.035 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQAVWA5lmSvzMRAA97YDqX3oplHCB+SENIkLYA== 2023-09-23T03:14:52.036 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr.x] 2023-09-23T03:14:52.036 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQAGWA5ljcy5ChAACUOnrxEGqacSxrB9BkZ+lg== 2023-09-23T03:14:52.036 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.b] 2023-09-23T03:14:52.036 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQAGWA5lsgKCNxAAI+vD2ESh3uNsb5OCO0ydSw== 2023-09-23T03:14:52.036 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.d] 2023-09-23T03:14:52.036 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQAHWA5lml/MDBAAHyXgd5zl/O8CeVE8Da4w5w== 2023-09-23T03:14:52.036 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.4] 2023-09-23T03:14:52.036 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQAcWA5lPZqwMBAAW95bBB0PKS/rwhXvrPf+mQ== 2023-09-23T03:14:52.036 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.5] 2023-09-23T03:14:52.036 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQAgWA5lmZySDRAAsbnUdWq9esKFSqSDeqsVlA== 2023-09-23T03:14:52.036 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.6] 2023-09-23T03:14:52.036 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQAjWA5ll9WqJRAAD9jecVxVJU+yuqANvN9QUQ== 2023-09-23T03:14:52.037 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.7] 2023-09-23T03:14:52.037 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQAnWA5lCvexAxAA/QymygbvOt1HQdCGbO8cyg== 2023-09-23T03:14:52.037 INFO:teuthology.orchestra.run.smithi154.stdout:[client.0] 2023-09-23T03:14:52.037 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQAHWA5lYg+qGhAAjaY0fPsmRTz3ipsKbPugMA== 2023-09-23T03:14:52.037 INFO:teuthology.orchestra.run.smithi154.stdout:[client.2] 2023-09-23T03:14:52.037 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQAHWA5lcLkIJRAAOknGCKwUs78D4atRzLb/Cw== 2023-09-23T03:14:52.037 INFO:teuthology.orchestra.run.smithi154.stdout:[client.1] 2023-09-23T03:14:52.037 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQAHWA5lbrrGMBAAn4kvS/1xGjZ+GtRZFqy31w== 2023-09-23T03:14:52.037 INFO:teuthology.orchestra.run.smithi154.stdout:[client.3] 2023-09-23T03:14:52.037 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQAIWA5lmFUaABAA4EBSJP5IidG2uq4g8lrFKg== 2023-09-23T03:14:52.038 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr.y] 2023-09-23T03:14:52.038 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQAFWA5looGaOhAAdxmefaoLuXesHXCeqeCYXQ== 2023-09-23T03:14:52.038 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.a] 2023-09-23T03:14:52.038 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQAGWA5l9biCFhAAhD2xgY9Q/gGqK0pgHRGweQ== 2023-09-23T03:14:52.038 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.c] 2023-09-23T03:14:52.038 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQAGWA5los4vJhAA2/ewjMZcc3pnm9ukhnoNgw== 2023-09-23T03:14:52.038 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.0] 2023-09-23T03:14:52.038 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQAMWA5lT1R4BBAAI+itqub/FadG3aU6LhnmGg== 2023-09-23T03:14:52.038 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.1] 2023-09-23T03:14:52.038 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQAPWA5lSablHRAAmNyr5Zopeihn5ZyoY5pyhA== 2023-09-23T03:14:52.038 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.2] 2023-09-23T03:14:52.039 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQASWA5lSFcdJxAA3UJJ6eSvxcQ31SAm4shV2g== 2023-09-23T03:14:52.039 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.3] 2023-09-23T03:14:52.039 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQAVWA5lmSvzMRAA97YDqX3oplHCB+SENIkLYA== 2023-09-23T03:14:52.039 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr.x] 2023-09-23T03:14:52.039 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQAGWA5ljcy5ChAACUOnrxEGqacSxrB9BkZ+lg== 2023-09-23T03:14:52.039 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.b] 2023-09-23T03:14:52.039 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQAGWA5lsgKCNxAAI+vD2ESh3uNsb5OCO0ydSw== 2023-09-23T03:14:52.039 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.d] 2023-09-23T03:14:52.039 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQAHWA5lml/MDBAAHyXgd5zl/O8CeVE8Da4w5w== 2023-09-23T03:14:52.039 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.4] 2023-09-23T03:14:52.039 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQAcWA5lPZqwMBAAW95bBB0PKS/rwhXvrPf+mQ== 2023-09-23T03:14:52.039 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.5] 2023-09-23T03:14:52.039 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQAgWA5lmZySDRAAsbnUdWq9esKFSqSDeqsVlA== 2023-09-23T03:14:52.040 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.6] 2023-09-23T03:14:52.040 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQAjWA5ll9WqJRAAD9jecVxVJU+yuqANvN9QUQ== 2023-09-23T03:14:52.040 INFO:teuthology.orchestra.run.smithi190.stdout:[osd.7] 2023-09-23T03:14:52.040 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQAnWA5lCvexAxAA/QymygbvOt1HQdCGbO8cyg== 2023-09-23T03:14:52.040 INFO:teuthology.orchestra.run.smithi190.stdout:[client.0] 2023-09-23T03:14:52.040 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQAHWA5lYg+qGhAAjaY0fPsmRTz3ipsKbPugMA== 2023-09-23T03:14:52.040 INFO:teuthology.orchestra.run.smithi190.stdout:[client.2] 2023-09-23T03:14:52.040 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQAHWA5lcLkIJRAAOknGCKwUs78D4atRzLb/Cw== 2023-09-23T03:14:52.040 INFO:teuthology.orchestra.run.smithi190.stdout:[client.1] 2023-09-23T03:14:52.040 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQAHWA5lbrrGMBAAn4kvS/1xGjZ+GtRZFqy31w== 2023-09-23T03:14:52.040 INFO:teuthology.orchestra.run.smithi190.stdout:[client.3] 2023-09-23T03:14:52.040 INFO:teuthology.orchestra.run.smithi190.stdout: key = AQAIWA5lmFUaABAA4EBSJP5IidG2uq4g8lrFKg== 2023-09-23T03:14:52.042 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-09-23T03:14:52.048 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-09-23T03:14:52.192 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-09-23T03:14:52.200 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-09-23T03:14:52.350 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-09-23T03:14:52.356 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-09-23T03:14:52.505 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-23T03:14:52.511 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-23T03:14:52.660 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-23T03:14:52.667 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-23T03:14:52.807 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-23T03:14:52.815 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-23T03:14:52.961 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-23T03:14:52.966 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-23T03:14:53.113 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-09-23T03:14:53.121 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-09-23T03:14:53.268 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-09-23T03:14:53.275 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-09-23T03:14:53.424 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-09-23T03:14:53.428 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-09-23T03:14:53.572 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-23T03:14:53.577 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-23T03:14:53.717 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-23T03:14:53.723 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-23T03:14:53.865 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-23T03:14:53.870 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-23T03:14:54.014 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-23T03:14:54.020 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-23T03:14:54.166 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-23T03:14:54.172 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-23T03:14:54.320 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-23T03:14:54.326 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-23T03:14:54.473 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-23T03:14:54.481 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-23T03:14:54.632 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-23T03:14:54.638 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-23T03:14:54.781 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-09-23T03:14:54.781 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-09-23T03:14:54.878 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-09-23T03:14:55.040 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-09-23T03:14:55.131 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-09-23T03:14:55.233 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-09-23T03:14:55.392 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-09-23T03:14:55.478 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-09-23T03:14:55.567 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-09-23T03:14:55.722 DEBUG:teuthology.orchestra.run.smithi190:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-09-23T03:14:55.807 DEBUG:teuthology.orchestra.run.smithi154:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-09-23T03:14:55.816 DEBUG:teuthology.orchestra.run.smithi190:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-09-23T03:14:55.867 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-09-23T03:14:55.867 INFO:tasks.ceph.mon.a:Restarting daemon 2023-09-23T03:14:55.868 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-09-23T03:14:55.873 INFO:tasks.ceph.mon.a:Started 2023-09-23T03:14:55.873 INFO:tasks.ceph.mon.b:Restarting daemon 2023-09-23T03:14:55.873 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-09-23T03:14:55.878 INFO:tasks.ceph.mon.b:Started 2023-09-23T03:14:55.879 INFO:tasks.ceph.mon.c:Restarting daemon 2023-09-23T03:14:55.879 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-09-23T03:14:55.884 INFO:tasks.ceph.mon.c:Started 2023-09-23T03:14:55.884 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-09-23T03:14:55.884 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-09-23T03:14:55.884 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-09-23T03:14:55.890 INFO:tasks.ceph.mgr.y:Started 2023-09-23T03:14:55.890 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-09-23T03:14:55.890 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-09-23T03:14:55.894 INFO:tasks.ceph.mgr.x:Started 2023-09-23T03:14:55.895 INFO:tasks.ceph:Setting crush tunables to default 2023-09-23T03:14:55.895 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd crush tunables default 2023-09-23T03:14:56.084 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.084+0000 7fcc52856640 1 Processor -- start 2023-09-23T03:14:56.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.084+0000 7fcc52856640 1 -- start start 2023-09-23T03:14:56.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.086+0000 7fcc52856640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcc4c06a110 0x7fcc4c0ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:56.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.086+0000 7fcc4bfff640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fcc4c15cb90 0x7fcc4c160ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.154:52580/0 (socket says 172.21.15.154:52580) 2023-09-23T03:14:56.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.086+0000 7fcc4bfff640 1 -- 172.21.15.154:0/776310420 learned_addr learned my addr 172.21.15.154:0/776310420 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:14:56.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.086+0000 7fcc50df3640 1 --2- 172.21.15.154:0/776310420 >> v2:172.21.15.154:3300/0 conn(0x7fcc4c06a110 0x7fcc4c0ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:56.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.086+0000 7fcc52856640 1 -- 172.21.15.154:0/776310420 --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc4c1615f0 con 0x7fcc4c069cd0 2023-09-23T03:14:56.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.086+0000 7fcc52856640 1 -- 172.21.15.154:0/776310420 --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc4c15cf70 con 0x7fcc4c15cb90 2023-09-23T03:14:56.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.086+0000 7fcc52856640 1 -- 172.21.15.154:0/776310420 --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc4c0cd320 con 0x7fcc4c06a110 2023-09-23T03:14:56.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.087+0000 7fcc50df3640 1 -- 172.21.15.154:0/776310420 >> v2:172.21.15.154:3300/0 conn(0x7fcc4c06a110 msgr2=0x7fcc4c0ca5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-09-23T03:14:56.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.087+0000 7fcc50df3640 1 -- 172.21.15.154:0/776310420 >> v2:172.21.15.154:3300/0 conn(0x7fcc4c06a110 msgr2=0x7fcc4c0ca5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-23T03:14:56.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.087+0000 7fcc50df3640 1 --2- 172.21.15.154:0/776310420 >> v2:172.21.15.154:3300/0 conn(0x7fcc4c06a110 0x7fcc4c0ca5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-23T03:14:56.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.087+0000 7fcc50df3640 1 --2- 172.21.15.154:0/776310420 >> v2:172.21.15.154:3300/0 conn(0x7fcc4c06a110 0x7fcc4c0ca5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-23T03:14:56.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.094+0000 7fcc4affd640 1 -- 172.21.15.154:0/776310420 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3482215431 0 0) 0x7fcc4c1615f0 con 0x7fcc4c069cd0 2023-09-23T03:14:56.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.094+0000 7fcc4affd640 1 -- 172.21.15.154:0/776310420 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc34003520 con 0x7fcc4c069cd0 2023-09-23T03:14:56.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.095+0000 7fcc4affd640 1 -- 172.21.15.154:0/776310420 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4022115124 0 0) 0x7fcc34003520 con 0x7fcc4c069cd0 2023-09-23T03:14:56.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.095+0000 7fcc4affd640 1 -- 172.21.15.154:0/776310420 >> v2:172.21.15.154:3300/0 conn(0x7fcc4c06a110 msgr2=0x7fcc4c0ca5a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T03:14:56.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.095+0000 7fcc4affd640 1 --2- 172.21.15.154:0/776310420 >> v2:172.21.15.154:3300/0 conn(0x7fcc4c06a110 0x7fcc4c0ca5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:56.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.095+0000 7fcc4affd640 1 -- 172.21.15.154:0/776310420 >> v1:172.21.15.190:6789/0 conn(0x7fcc4c15cb90 legacy=0x7fcc4c160ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:56.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.095+0000 7fcc4affd640 1 -- 172.21.15.154:0/776310420 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc4c0cd600 con 0x7fcc4c069cd0 2023-09-23T03:14:56.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.095+0000 7fcc4affd640 1 -- 172.21.15.154:0/776310420 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fcc40001670 con 0x7fcc4c069cd0 2023-09-23T03:14:56.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.095+0000 7fcc4affd640 1 -- 172.21.15.154:0/776310420 >> v1:172.21.15.154:6789/0 conn(0x7fcc4c069cd0 legacy=0x7fcc4c0c9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:56.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.095+0000 7fcc4affd640 1 --2- 172.21.15.154:0/776310420 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc340039c0 0x7fcc34003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:56.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.095+0000 7fcc4affd640 1 --2- 172.21.15.154:0/776310420 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcc340047c0 0x7fcc34004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:56.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.095+0000 7fcc4affd640 1 --2- 172.21.15.154:0/776310420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc340052a0 0x7fcc3400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:56.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.095+0000 7fcc4affd640 1 -- 172.21.15.154:0/776310420 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c0cd320 con 0x7fcc340039c0 2023-09-23T03:14:56.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.095+0000 7fcc4affd640 1 -- 172.21.15.154:0/776310420 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3400c1c0 con 0x7fcc340052a0 2023-09-23T03:14:56.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.095+0000 7fcc4affd640 1 -- 172.21.15.154:0/776310420 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3400c380 con 0x7fcc340047c0 2023-09-23T03:14:56.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.095+0000 7fcc50df3640 1 --2- 172.21.15.154:0/776310420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc340052a0 0x7fcc3400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:56.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.095+0000 7fcc4b7fe640 1 --2- 172.21.15.154:0/776310420 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcc340047c0 0x7fcc34004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:56.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.096+0000 7fcc4bfff640 1 --2- 172.21.15.154:0/776310420 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc340039c0 0x7fcc34003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:56.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.096+0000 7fcc50df3640 1 -- 172.21.15.154:0/776310420 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcc340047c0 msgr2=0x7fcc34004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:14:56.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.096+0000 7fcc50df3640 1 --2- 172.21.15.154:0/776310420 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcc340047c0 0x7fcc34004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:56.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.096+0000 7fcc50df3640 1 -- 172.21.15.154:0/776310420 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc340039c0 msgr2=0x7fcc34003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:14:56.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.096+0000 7fcc50df3640 1 --2- 172.21.15.154:0/776310420 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc340039c0 0x7fcc34003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:56.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.096+0000 7fcc50df3640 1 -- 172.21.15.154:0/776310420 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc34003520 con 0x7fcc340052a0 2023-09-23T03:14:56.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.096+0000 7fcc4bfff640 1 --2- 172.21.15.154:0/776310420 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc340039c0 0x7fcc34003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:14:56.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.097+0000 7fcc50df3640 1 -- 172.21.15.154:0/776310420 reap_dead start 2023-09-23T03:14:56.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.097+0000 7fcc50df3640 1 --2- 172.21.15.154:0/776310420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc340052a0 0x7fcc3400b980 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcc44009fd0 tx=0x7fcc340057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c299d90e7a8e81cb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:14:56.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.097+0000 7fcc4affd640 1 -- 172.21.15.154:0/776310420 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4402ccb0 con 0x7fcc340052a0 2023-09-23T03:14:56.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.100+0000 7fcc4affd640 1 -- 172.21.15.154:0/776310420 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc4402ce30 con 0x7fcc340052a0 2023-09-23T03:14:56.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.100+0000 7fcc4affd640 1 -- 172.21.15.154:0/776310420 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc44034680 con 0x7fcc340052a0 2023-09-23T03:14:56.101 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.101+0000 7fcc52856640 1 -- 172.21.15.154:0/776310420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc340052a0 msgr2=0x7fcc3400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:56.101 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.101+0000 7fcc52856640 1 --2- 172.21.15.154:0/776310420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc340052a0 0x7fcc3400b980 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcc44009fd0 tx=0x7fcc340057f0 comp rx=0 tx=0).stop 2023-09-23T03:14:56.101 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.101+0000 7fcc52856640 1 -- 172.21.15.154:0/776310420 shutdown_connections 2023-09-23T03:14:56.101 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.101+0000 7fcc52856640 1 --2- 172.21.15.154:0/776310420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc340052a0 0x7fcc3400b980 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:56.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.101+0000 7fcc52856640 1 -- 172.21.15.154:0/776310420 >> 172.21.15.154:0/776310420 conn(0x7fcc4c06d120 msgr2=0x7fcc4c06d500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:14:56.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.101+0000 7fcc52856640 1 -- 172.21.15.154:0/776310420 shutdown_connections 2023-09-23T03:14:56.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.101+0000 7fcc52856640 1 -- 172.21.15.154:0/776310420 wait complete. 2023-09-23T03:14:56.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.103+0000 7fcc52856640 1 Processor -- start 2023-09-23T03:14:56.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.103+0000 7fcc52856640 1 -- start start 2023-09-23T03:14:56.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.104+0000 7fcc52856640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcc340052a0 0x7fcc4c15e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:56.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.104+0000 7fcc52856640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc4c15ba70 0x7fcc4c15bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:56.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.104+0000 7fcc52856640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc4c15d690 0x7fcc4c15db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:56.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.104+0000 7fcc52856640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c15fa30 con 0x7fcc4c15d690 2023-09-23T03:14:56.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.104+0000 7fcc52856640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c15c430 con 0x7fcc4c15ba70 2023-09-23T03:14:56.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.104+0000 7fcc52856640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c15ec90 con 0x7fcc340052a0 2023-09-23T03:14:56.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.105+0000 7fcc50df3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc4c15d690 0x7fcc4c15db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:56.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.105+0000 7fcc4bfff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcc340052a0 0x7fcc4c15e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:56.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.105+0000 7fcc4bfff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcc340052a0 0x7fcc4c15e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:42002/0 (socket says 172.21.15.154:42002) 2023-09-23T03:14:56.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.105+0000 7fcc4bfff640 1 -- 172.21.15.154:0/3256124063 learned_addr learned my addr 172.21.15.154:0/3256124063 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:14:56.106 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.105+0000 7fcc4b7fe640 1 --2- 172.21.15.154:0/3256124063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc4c15ba70 0x7fcc4c15bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:56.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.105+0000 7fcc50df3640 1 -- 172.21.15.154:0/3256124063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcc340052a0 msgr2=0x7fcc4c15e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:56.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.105+0000 7fcc50df3640 1 --2- 172.21.15.154:0/3256124063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcc340052a0 0x7fcc4c15e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:56.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.105+0000 7fcc50df3640 1 -- 172.21.15.154:0/3256124063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc4c15ba70 msgr2=0x7fcc4c15bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:56.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.105+0000 7fcc50df3640 1 --2- 172.21.15.154:0/3256124063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc4c15ba70 0x7fcc4c15bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:56.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.105+0000 7fcc50df3640 1 -- 172.21.15.154:0/3256124063 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc4c15f090 con 0x7fcc4c15d690 2023-09-23T03:14:56.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.105+0000 7fcc50df3640 1 --2- 172.21.15.154:0/3256124063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc4c15d690 0x7fcc4c15db10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c06c4a0 tx=0x7fcc44004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:14:56.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.106+0000 7fcc48ff9640 1 -- 172.21.15.154:0/3256124063 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4403d600 con 0x7fcc4c15d690 2023-09-23T03:14:56.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.106+0000 7fcc48ff9640 1 -- 172.21.15.154:0/3256124063 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc4402ccc0 con 0x7fcc4c15d690 2023-09-23T03:14:56.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.106+0000 7fcc48ff9640 1 -- 172.21.15.154:0/3256124063 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4403d9a0 con 0x7fcc4c15d690 2023-09-23T03:14:56.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.106+0000 7fcc52856640 1 -- 172.21.15.154:0/3256124063 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc4c15f370 con 0x7fcc4c15d690 2023-09-23T03:14:56.108 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.106+0000 7fcc52856640 1 -- 172.21.15.154:0/3256124063 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc4c15e050 con 0x7fcc4c15d690 2023-09-23T03:14:56.108 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.106+0000 7fcc48ff9640 1 -- 172.21.15.154:0/3256124063 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fcc4403b040 con 0x7fcc4c15d690 2023-09-23T03:14:56.108 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.106+0000 7fcc48ff9640 1 -- 172.21.15.154:0/3256124063 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fcc4404abb0 con 0x7fcc4c15d690 2023-09-23T03:14:56.108 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.107+0000 7fcc52856640 1 -- 172.21.15.154:0/3256124063 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc4c0c2ce0 con 0x7fcc4c15d690 2023-09-23T03:14:56.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.110+0000 7fcc48ff9640 1 -- 172.21.15.154:0/3256124063 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fcc44046070 con 0x7fcc4c15d690 2023-09-23T03:14:56.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.180+0000 7fcc52856640 1 -- 172.21.15.154:0/3256124063 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fcc4c15e1e0 con 0x7fcc4c15d690 2023-09-23T03:14:56.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.184+0000 7fcc48ff9640 1 -- 172.21.15.154:0/3256124063 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fcc44039070 con 0x7fcc4c15d690 2023-09-23T03:14:56.184 INFO:teuthology.orchestra.run.smithi154.stderr:adjusted tunables profile to default 2023-09-23T03:14:56.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.186+0000 7fcc52856640 1 -- 172.21.15.154:0/3256124063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc4c15d690 msgr2=0x7fcc4c15db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:56.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.186+0000 7fcc52856640 1 --2- 172.21.15.154:0/3256124063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc4c15d690 0x7fcc4c15db10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c06c4a0 tx=0x7fcc44004ab0 comp rx=0 tx=0).stop 2023-09-23T03:14:56.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.186+0000 7fcc52856640 1 -- 172.21.15.154:0/3256124063 shutdown_connections 2023-09-23T03:14:56.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.186+0000 7fcc52856640 1 --2- 172.21.15.154:0/3256124063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcc4c15d690 0x7fcc4c15db10 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:56.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.186+0000 7fcc52856640 1 --2- 172.21.15.154:0/3256124063 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcc4c15ba70 0x7fcc4c15bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:56.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.186+0000 7fcc52856640 1 --2- 172.21.15.154:0/3256124063 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcc340052a0 0x7fcc4c15e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:56.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.186+0000 7fcc52856640 1 -- 172.21.15.154:0/3256124063 >> 172.21.15.154:0/3256124063 conn(0x7fcc4c06d120 msgr2=0x7fcc4c0cc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:14:56.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.186+0000 7fcc52856640 1 -- 172.21.15.154:0/3256124063 shutdown_connections 2023-09-23T03:14:56.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:56.186+0000 7fcc52856640 1 -- 172.21.15.154:0/3256124063 wait complete. 2023-09-23T03:14:56.203 INFO:tasks.ceph:check_enable_crimson: False 2023-09-23T03:14:56.203 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-09-23T03:14:56.203 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:14:56.203 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-09-23T03:14:56.293 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:14:56.293 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-09-23T03:14:56.426 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:14:56.426 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-09-23T03:14:56.508 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:14:56.509 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-09-23T03:14:56.645 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:14:56.645 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-09-23T03:14:56.741 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:14:56.741 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-09-23T03:14:56.829 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:14:56.830 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-09-23T03:14:56.858 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-09-23T03:14:56.857+0000 7f46d5215040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-23T03:14:56.871 INFO:tasks.ceph.mgr.y.smithi154.stderr:2023-09-23T03:14:56.871+0000 7f0f6863f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-23T03:14:56.924 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-09-23T03:14:56.923+0000 7f46d5215040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-23T03:14:56.935 INFO:tasks.ceph.mgr.y.smithi154.stderr:2023-09-23T03:14:56.935+0000 7f0f6863f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-23T03:14:56.941 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:14:56.941 DEBUG:teuthology.orchestra.run.smithi190:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-09-23T03:14:57.071 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 995eaf65-9eef-4aa4-93e6-39576cf4b85f 0 2023-09-23T03:14:57.304 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.303+0000 7f7f6eeaf640 1 Processor -- start 2023-09-23T03:14:57.304 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.303+0000 7f7f6eeaf640 1 -- start start 2023-09-23T03:14:57.306 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.304+0000 7f7f6eeaf640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7f6815c9f0 0x7f7f6815d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:57.307 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.304+0000 7f7f6eeaf640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7f680cf010 0x7f7f680cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:57.307 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.304+0000 7f7f6eeaf640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f6815cdd0 con 0x7f7f681604e0 2023-09-23T03:14:57.307 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.304+0000 7f7f6eeaf640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f680ce4c0 con 0x7f7f6815c9f0 2023-09-23T03:14:57.307 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.304+0000 7f7f6eeaf640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f680cd8b0 con 0x7f7f680cf010 2023-09-23T03:14:57.307 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.305+0000 7f7f67fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7f680cf010 0x7f7f680cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:57.307 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.305+0000 7f7f67fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7f680cf010 0x7f7f680cf470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:44566/0 (socket says 172.21.15.190:44566) 2023-09-23T03:14:57.307 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.305+0000 7f7f67fff640 1 -- 172.21.15.190:0/3716809370 learned_addr learned my addr 172.21.15.190:0/3716809370 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-09-23T03:14:57.307 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.305+0000 7f7f6cc4b640 1 --2- 172.21.15.190:0/3716809370 >> v2:172.21.15.154:3300/0 conn(0x7f7f6815c9f0 0x7f7f6815d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:57.307 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.305+0000 7f7f67fff640 1 -- 172.21.15.190:0/3716809370 >> v2:172.21.15.154:3300/0 conn(0x7f7f6815c9f0 msgr2=0x7f7f6815d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:14:57.307 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.305+0000 7f7f67fff640 1 --2- 172.21.15.190:0/3716809370 >> v2:172.21.15.154:3300/0 conn(0x7f7f6815c9f0 0x7f7f6815d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:57.308 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.305+0000 7f7f67fff640 1 -- 172.21.15.190:0/3716809370 >> v1:172.21.15.154:6789/0 conn(0x7f7f681604e0 legacy=0x7f7f68160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:14:57.308 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.305+0000 7f7f67fff640 1 -- 172.21.15.190:0/3716809370 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f680cf9b0 con 0x7f7f680cf010 2023-09-23T03:14:57.308 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.305+0000 7f7f6cc4b640 1 --2- 172.21.15.190:0/3716809370 >> v2:172.21.15.154:3300/0 conn(0x7f7f6815c9f0 0x7f7f6815d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:14:57.309 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.306+0000 7f7f67fff640 1 --2- 172.21.15.190:0/3716809370 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f680cf010 0x7f7f680cf470 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7f680b54f0 tx=0x7f7f5402b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9f920be1dc76c76 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:14:57.309 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.306+0000 7f7f677fe640 1 -- 172.21.15.190:0/3716809370 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f54031890 con 0x7f7f680cf010 2023-09-23T03:14:57.309 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.306+0000 7f7f677fe640 1 -- 172.21.15.190:0/3716809370 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f540319f0 con 0x7f7f680cf010 2023-09-23T03:14:57.309 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.306+0000 7f7f677fe640 1 -- 172.21.15.190:0/3716809370 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f54031d80 con 0x7f7f680cf010 2023-09-23T03:14:57.309 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.307+0000 7f7f6eeaf640 1 -- 172.21.15.190:0/3716809370 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f680cf010 msgr2=0x7f7f680cf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:57.309 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.307+0000 7f7f6eeaf640 1 --2- 172.21.15.190:0/3716809370 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f680cf010 0x7f7f680cf470 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7f680b54f0 tx=0x7f7f5402b930 comp rx=0 tx=0).stop 2023-09-23T03:14:57.309 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.307+0000 7f7f6eeaf640 1 -- 172.21.15.190:0/3716809370 shutdown_connections 2023-09-23T03:14:57.309 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.307+0000 7f7f6eeaf640 1 --2- 172.21.15.190:0/3716809370 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f680cf010 0x7f7f680cf470 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:57.310 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.307+0000 7f7f6eeaf640 1 --2- 172.21.15.190:0/3716809370 >> v2:172.21.15.154:3300/0 conn(0x7f7f6815c9f0 0x7f7f6815d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:57.310 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.307+0000 7f7f6eeaf640 1 -- 172.21.15.190:0/3716809370 >> 172.21.15.190:0/3716809370 conn(0x7f7f6809bc10 msgr2=0x7f7f680d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:14:57.310 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.307+0000 7f7f6eeaf640 1 -- 172.21.15.190:0/3716809370 shutdown_connections 2023-09-23T03:14:57.310 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.307+0000 7f7f6eeaf640 1 -- 172.21.15.190:0/3716809370 wait complete. 2023-09-23T03:14:57.311 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.310+0000 7f7f6eeaf640 1 Processor -- start 2023-09-23T03:14:57.311 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.310+0000 7f7f6eeaf640 1 -- start start 2023-09-23T03:14:57.314 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.311+0000 7f7f6eeaf640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f680cf010 0x7f7f681f5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:57.314 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.311+0000 7f7f6eeaf640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f6815c9f0 0x7f7f681f5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:57.314 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.311+0000 7f7f6eeaf640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f681604e0 0x7f7f6809c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:57.314 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.311+0000 7f7f6eeaf640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6809d4e0 con 0x7f7f6815c9f0 2023-09-23T03:14:57.315 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.311+0000 7f7f6eeaf640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6809d7c0 con 0x7f7f680cf010 2023-09-23T03:14:57.315 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.311+0000 7f7f6eeaf640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6809daa0 con 0x7f7f681604e0 2023-09-23T03:14:57.315 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.312+0000 7f7f6cc4b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f680cf010 0x7f7f681f5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:57.315 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.312+0000 7f7f6d44c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f681604e0 0x7f7f6809c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:57.315 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.312+0000 7f7f6cc4b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f680cf010 0x7f7f681f5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:44580/0 (socket says 172.21.15.190:44580) 2023-09-23T03:14:57.315 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.312+0000 7f7f6cc4b640 1 -- 172.21.15.190:0/1146642846 learned_addr learned my addr 172.21.15.190:0/1146642846 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-09-23T03:14:57.315 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.312+0000 7f7f67fff640 1 --2- 172.21.15.190:0/1146642846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f6815c9f0 0x7f7f681f5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:57.315 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.312+0000 7f7f6cc4b640 1 -- 172.21.15.190:0/1146642846 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f681604e0 msgr2=0x7f7f6809c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:57.315 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.312+0000 7f7f6cc4b640 1 --2- 172.21.15.190:0/1146642846 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f681604e0 0x7f7f6809c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:57.315 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.312+0000 7f7f6cc4b640 1 -- 172.21.15.190:0/1146642846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f6815c9f0 msgr2=0x7f7f681f5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:57.315 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.312+0000 7f7f6cc4b640 1 --2- 172.21.15.190:0/1146642846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f6815c9f0 0x7f7f681f5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:57.316 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.312+0000 7f7f6cc4b640 1 -- 172.21.15.190:0/1146642846 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f540094d0 con 0x7f7f680cf010 2023-09-23T03:14:57.316 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.312+0000 7f7f6d44c640 1 --2- 172.21.15.190:0/1146642846 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f681604e0 0x7f7f6809c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:14:57.316 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.312+0000 7f7f6cc4b640 1 --2- 172.21.15.190:0/1146642846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f680cf010 0x7f7f681f5670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00c7f0 tx=0x7f7f5c00cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:14:57.316 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.313+0000 7f7f657fa640 1 -- 172.21.15.190:0/1146642846 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c005b90 con 0x7f7f680cf010 2023-09-23T03:14:57.316 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.313+0000 7f7f657fa640 1 -- 172.21.15.190:0/1146642846 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f5c005cf0 con 0x7f7f680cf010 2023-09-23T03:14:57.316 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.313+0000 7f7f6eeaf640 1 -- 172.21.15.190:0/1146642846 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f6809dd80 con 0x7f7f680cf010 2023-09-23T03:14:57.316 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.313+0000 7f7f657fa640 1 -- 172.21.15.190:0/1146642846 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c007a30 con 0x7f7f680cf010 2023-09-23T03:14:57.316 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.313+0000 7f7f6eeaf640 1 -- 172.21.15.190:0/1146642846 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f6809ccf0 con 0x7f7f680cf010 2023-09-23T03:14:57.316 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.313+0000 7f7f657fa640 1 -- 172.21.15.190:0/1146642846 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7f5c007b90 con 0x7f7f680cf010 2023-09-23T03:14:57.316 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.313+0000 7f7f657fa640 1 -- 172.21.15.190:0/1146642846 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7f5c019730 con 0x7f7f680cf010 2023-09-23T03:14:57.317 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.314+0000 7f7f6eeaf640 1 -- 172.21.15.190:0/1146642846 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f680cde20 con 0x7f7f680cf010 2023-09-23T03:14:57.318 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.317+0000 7f7f657fa640 1 -- 172.21.15.190:0/1146642846 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f7f5c01e060 con 0x7f7f680cf010 2023-09-23T03:14:57.396 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.394+0000 7f7f6eeaf640 1 -- 172.21.15.190:0/1146642846 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd new", "uuid": "995eaf65-9eef-4aa4-93e6-39576cf4b85f", "id": 0} v 0) v1 -- 0x7f7f6809df70 con 0x7f7f680cf010 2023-09-23T03:14:57.402 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.401+0000 7f7f657fa640 1 -- 172.21.15.190:0/1146642846 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "995eaf65-9eef-4aa4-93e6-39576cf4b85f", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7f5c0066c0 con 0x7f7f680cf010 2023-09-23T03:14:57.403 INFO:teuthology.orchestra.run.smithi190.stdout:0 2023-09-23T03:14:57.404 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.402+0000 7f7f6eeaf640 1 -- 172.21.15.190:0/1146642846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f680cf010 msgr2=0x7f7f681f5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:57.404 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.402+0000 7f7f6eeaf640 1 --2- 172.21.15.190:0/1146642846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f680cf010 0x7f7f681f5670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00c7f0 tx=0x7f7f5c00cbf0 comp rx=0 tx=0).stop 2023-09-23T03:14:57.404 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.403+0000 7f7f6eeaf640 1 -- 172.21.15.190:0/1146642846 shutdown_connections 2023-09-23T03:14:57.404 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.403+0000 7f7f6eeaf640 1 --2- 172.21.15.190:0/1146642846 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7f681604e0 0x7f7f6809c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:57.404 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.403+0000 7f7f6eeaf640 1 --2- 172.21.15.190:0/1146642846 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f6815c9f0 0x7f7f681f5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:57.405 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.403+0000 7f7f6eeaf640 1 --2- 172.21.15.190:0/1146642846 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7f680cf010 0x7f7f681f5670 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:57.405 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.403+0000 7f7f6eeaf640 1 -- 172.21.15.190:0/1146642846 >> 172.21.15.190:0/1146642846 conn(0x7f7f6809bc10 msgr2=0x7f7f6806af50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:14:57.405 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.403+0000 7f7f6eeaf640 1 -- 172.21.15.190:0/1146642846 shutdown_connections 2023-09-23T03:14:57.405 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.403+0000 7f7f6eeaf640 1 -- 172.21.15.190:0/1146642846 wait complete. 2023-09-23T03:14:57.421 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 409c115c-d865-492f-85b2-d9bc90f1a296 1 2023-09-23T03:14:57.563 INFO:tasks.ceph.mgr.y.smithi154.stderr:2023-09-23T03:14:57.563+0000 7f0f6863f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-23T03:14:57.577 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-09-23T03:14:57.576+0000 7f46d5215040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-23T03:14:57.615 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.613+0000 7fa10cea3640 1 Processor -- start 2023-09-23T03:14:57.615 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.614+0000 7fa10cea3640 1 -- start start 2023-09-23T03:14:57.616 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.615+0000 7fa10cea3640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa1080d0a10 0x7fa10806c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:57.616 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.615+0000 7fa10cea3640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa1080d0df0 0x7fa10806b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:57.617 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.615+0000 7fa10cea3640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa10806c6a0 con 0x7fa1080b16c0 2023-09-23T03:14:57.617 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.615+0000 7fa10cea3640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1080d1930 con 0x7fa1080d0df0 2023-09-23T03:14:57.617 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.615+0000 7fa10cea3640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1080d38d0 con 0x7fa1080d0a10 2023-09-23T03:14:57.617 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.615+0000 7fa1077fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa1080d0a10 0x7fa10806c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:57.617 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.615+0000 7fa1077fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa1080d0a10 0x7fa10806c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:44586/0 (socket says 172.21.15.190:44586) 2023-09-23T03:14:57.617 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.615+0000 7fa1077fe640 1 -- 172.21.15.190:0/423776314 learned_addr learned my addr 172.21.15.190:0/423776314 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-09-23T03:14:57.618 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.616+0000 7fa107fff640 1 --2- 172.21.15.190:0/423776314 >> v2:172.21.15.154:3300/0 conn(0x7fa1080d0df0 0x7fa10806b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:57.618 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.616+0000 7fa1077fe640 1 -- 172.21.15.190:0/423776314 >> v2:172.21.15.154:3300/0 conn(0x7fa1080d0df0 msgr2=0x7fa10806b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:14:57.618 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.616+0000 7fa1077fe640 1 --2- 172.21.15.190:0/423776314 >> v2:172.21.15.154:3300/0 conn(0x7fa1080d0df0 0x7fa10806b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:57.618 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.616+0000 7fa1077fe640 1 -- 172.21.15.190:0/423776314 >> v1:172.21.15.190:6789/0 conn(0x7fa1080b16c0 legacy=0x7fa10806ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:57.618 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.616+0000 7fa1077fe640 1 -- 172.21.15.190:0/423776314 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1080d1370 con 0x7fa1080d0a10 2023-09-23T03:14:57.618 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.617+0000 7fa1077fe640 1 --2- 172.21.15.190:0/423776314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1080d0a10 0x7fa10806c160 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa1080a0940 tx=0x7fa0f802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=359fa222b01d816c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:14:57.619 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.617+0000 7fa106ffd640 1 -- 172.21.15.190:0/423776314 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8004230 con 0x7fa1080d0a10 2023-09-23T03:14:57.619 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.617+0000 7fa106ffd640 1 -- 172.21.15.190:0/423776314 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f8004390 con 0x7fa1080d0a10 2023-09-23T03:14:57.619 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.617+0000 7fa106ffd640 1 -- 172.21.15.190:0/423776314 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8030840 con 0x7fa1080d0a10 2023-09-23T03:14:57.619 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.618+0000 7fa10cea3640 1 -- 172.21.15.190:0/423776314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1080d0a10 msgr2=0x7fa10806c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:57.620 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.618+0000 7fa10cea3640 1 --2- 172.21.15.190:0/423776314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1080d0a10 0x7fa10806c160 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa1080a0940 tx=0x7fa0f802b980 comp rx=0 tx=0).stop 2023-09-23T03:14:57.620 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.618+0000 7fa10cea3640 1 -- 172.21.15.190:0/423776314 shutdown_connections 2023-09-23T03:14:57.620 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.618+0000 7fa10cea3640 1 --2- 172.21.15.190:0/423776314 >> v2:172.21.15.154:3300/0 conn(0x7fa1080d0df0 0x7fa10806b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:57.620 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.618+0000 7fa10cea3640 1 --2- 172.21.15.190:0/423776314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1080d0a10 0x7fa10806c160 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:57.620 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.618+0000 7fa10cea3640 1 -- 172.21.15.190:0/423776314 >> 172.21.15.190:0/423776314 conn(0x7fa10809bc20 msgr2=0x7fa1080d29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:14:57.620 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.618+0000 7fa10cea3640 1 -- 172.21.15.190:0/423776314 shutdown_connections 2023-09-23T03:14:57.620 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.618+0000 7fa10cea3640 1 -- 172.21.15.190:0/423776314 wait complete. 2023-09-23T03:14:57.621 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.620+0000 7fa10cea3640 1 Processor -- start 2023-09-23T03:14:57.621 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.620+0000 7fa10cea3640 1 -- start start 2023-09-23T03:14:57.623 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.621+0000 7fa10cea3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1080b16c0 0x7fa1080c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:57.623 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.621+0000 7fa10cea3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1080d0a10 0x7fa10815c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:57.623 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.621+0000 7fa10cea3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1080d0df0 0x7fa1080cb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:57.623 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.621+0000 7fa10cea3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1080caed0 con 0x7fa1080d0df0 2023-09-23T03:14:57.623 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.621+0000 7fa10cea3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1080ca2c0 con 0x7fa1080d0a10 2023-09-23T03:14:57.623 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.621+0000 7fa10cea3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1080ca5a0 con 0x7fa1080b16c0 2023-09-23T03:14:57.623 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.621+0000 7fa0fffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1080d0a10 0x7fa10815c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:57.623 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.621+0000 7fa1077fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1080b16c0 0x7fa1080c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:57.623 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.621+0000 7fa1077fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1080b16c0 0x7fa1080c9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:50102/0 (socket says 172.21.15.190:50102) 2023-09-23T03:14:57.623 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.621+0000 7fa1077fe640 1 -- 172.21.15.190:0/1123108719 learned_addr learned my addr 172.21.15.190:0/1123108719 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-09-23T03:14:57.624 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.621+0000 7fa107fff640 1 --2- 172.21.15.190:0/1123108719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1080d0df0 0x7fa1080cb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:57.624 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.621+0000 7fa1077fe640 1 -- 172.21.15.190:0/1123108719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1080d0a10 msgr2=0x7fa10815c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:57.624 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.621+0000 7fa1077fe640 1 --2- 172.21.15.190:0/1123108719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1080d0a10 0x7fa10815c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:57.624 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.621+0000 7fa1077fe640 1 -- 172.21.15.190:0/1123108719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1080d0df0 msgr2=0x7fa1080cb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:57.624 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.622+0000 7fa1077fe640 1 --2- 172.21.15.190:0/1123108719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1080d0df0 0x7fa1080cb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:57.624 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.622+0000 7fa1077fe640 1 -- 172.21.15.190:0/1123108719 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f8009460 con 0x7fa1080b16c0 2023-09-23T03:14:57.624 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.622+0000 7fa1077fe640 1 --2- 172.21.15.190:0/1123108719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1080b16c0 0x7fa1080c9a90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8000c00 tx=0x7fa0f80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:14:57.625 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.622+0000 7fa104ff9640 1 -- 172.21.15.190:0/1123108719 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f803e070 con 0x7fa1080b16c0 2023-09-23T03:14:57.625 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.622+0000 7fa104ff9640 1 -- 172.21.15.190:0/1123108719 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f8004500 con 0x7fa1080b16c0 2023-09-23T03:14:57.625 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.622+0000 7fa104ff9640 1 -- 172.21.15.190:0/1123108719 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8030e40 con 0x7fa1080b16c0 2023-09-23T03:14:57.625 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.622+0000 7fa10cea3640 1 -- 172.21.15.190:0/1123108719 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1080ca880 con 0x7fa1080b16c0 2023-09-23T03:14:57.625 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.622+0000 7fa10cea3640 1 -- 172.21.15.190:0/1123108719 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1080c9fd0 con 0x7fa1080b16c0 2023-09-23T03:14:57.625 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.623+0000 7fa104ff9640 1 -- 172.21.15.190:0/1123108719 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa0f8003610 con 0x7fa1080b16c0 2023-09-23T03:14:57.625 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.623+0000 7fa104ff9640 1 -- 172.21.15.190:0/1123108719 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fa0f8006570 con 0x7fa1080b16c0 2023-09-23T03:14:57.626 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.623+0000 7fa10cea3640 1 -- 172.21.15.190:0/1123108719 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa10806c8b0 con 0x7fa1080b16c0 2023-09-23T03:14:57.626 INFO:tasks.ceph.mgr.y.smithi154.stderr:2023-09-23T03:14:57.625+0000 7f0f6863f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-23T03:14:57.627 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.626+0000 7fa104ff9640 1 -- 172.21.15.190:0/1123108719 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fa0f8048070 con 0x7fa1080b16c0 2023-09-23T03:14:57.643 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-09-23T03:14:57.642+0000 7f46d5215040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-23T03:14:57.676 INFO:tasks.ceph.mgr.y.smithi154.stderr:2023-09-23T03:14:57.676+0000 7f0f6863f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-23T03:14:57.696 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-09-23T03:14:57.695+0000 7f46d5215040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-23T03:14:57.705 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.704+0000 7fa10cea3640 1 -- 172.21.15.190:0/1123108719 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd new", "uuid": "409c115c-d865-492f-85b2-d9bc90f1a296", "id": 1} v 0) v1 -- 0x7fa1080d3520 con 0x7fa1080b16c0 2023-09-23T03:14:57.710 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.709+0000 7fa104ff9640 1 -- 172.21.15.190:0/1123108719 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "409c115c-d865-492f-85b2-d9bc90f1a296", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa0f8006800 con 0x7fa1080b16c0 2023-09-23T03:14:57.710 INFO:teuthology.orchestra.run.smithi190.stdout:1 2023-09-23T03:14:57.712 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.710+0000 7fa10cea3640 1 -- 172.21.15.190:0/1123108719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1080b16c0 msgr2=0x7fa1080c9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:57.712 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.710+0000 7fa10cea3640 1 --2- 172.21.15.190:0/1123108719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1080b16c0 0x7fa1080c9a90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8000c00 tx=0x7fa0f80046c0 comp rx=0 tx=0).stop 2023-09-23T03:14:57.712 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.710+0000 7fa10cea3640 1 -- 172.21.15.190:0/1123108719 shutdown_connections 2023-09-23T03:14:57.712 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.710+0000 7fa10cea3640 1 --2- 172.21.15.190:0/1123108719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1080d0df0 0x7fa1080cb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:57.712 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.710+0000 7fa10cea3640 1 --2- 172.21.15.190:0/1123108719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1080d0a10 0x7fa10815c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:57.712 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.710+0000 7fa10cea3640 1 --2- 172.21.15.190:0/1123108719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1080b16c0 0x7fa1080c9a90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:57.712 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.711+0000 7fa10cea3640 1 -- 172.21.15.190:0/1123108719 >> 172.21.15.190:0/1123108719 conn(0x7fa10809bc20 msgr2=0x7fa1081e5910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:14:57.713 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.711+0000 7fa10cea3640 1 -- 172.21.15.190:0/1123108719 shutdown_connections 2023-09-23T03:14:57.713 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.711+0000 7fa10cea3640 1 -- 172.21.15.190:0/1123108719 wait complete. 2023-09-23T03:14:57.729 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 9e99c1ee-098c-40d9-b017-530bc1140196 2 2023-09-23T03:14:57.739 INFO:tasks.ceph.mgr.y.smithi154.stderr:2023-09-23T03:14:57.739+0000 7f0f6863f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-23T03:14:57.764 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-09-23T03:14:57.763+0000 7f46d5215040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-23T03:14:57.788 INFO:tasks.ceph.mgr.y.smithi154.stderr:2023-09-23T03:14:57.788+0000 7f0f6863f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-23T03:14:57.818 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-09-23T03:14:57.816+0000 7f46d5215040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-23T03:14:57.901 INFO:tasks.ceph.mgr.y.smithi154.stderr:2023-09-23T03:14:57.901+0000 7f0f6863f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-23T03:14:57.912 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.911+0000 7f6afbe4b640 1 Processor -- start 2023-09-23T03:14:57.912 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.911+0000 7f6afbe4b640 1 -- start start 2023-09-23T03:14:57.914 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.913+0000 7f6afbe4b640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6af415cc60 0x7f6af4069cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:57.915 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.913+0000 7f6afbe4b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6af4070780 0x7f6af40cc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:57.915 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.913+0000 7f6afbe4b640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6af40cd2d0 con 0x7f6af40cce70 2023-09-23T03:14:57.915 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.913+0000 7f6afbe4b640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6af406ae00 con 0x7f6af415cc60 2023-09-23T03:14:57.915 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.913+0000 7f6afbe4b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af40c94e0 con 0x7f6af4070780 2023-09-23T03:14:57.915 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.913+0000 7f6af9be7640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6af415cc60 0x7f6af4069cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:57.915 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.913+0000 7f6af9be7640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6af415cc60 0x7f6af4069cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:33458/0 (socket says 172.21.15.190:33458) 2023-09-23T03:14:57.915 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.913+0000 7f6af93e6640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6af4070780 0x7f6af40cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:57.915 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.913+0000 7f6af9be7640 1 -- 172.21.15.190:0/3548379834 learned_addr learned my addr 172.21.15.190:0/3548379834 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-09-23T03:14:57.915 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.913+0000 7f6af9be7640 1 -- 172.21.15.190:0/3548379834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6af4070780 msgr2=0x7f6af40cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:14:57.915 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.913+0000 7f6af9be7640 1 --2- 172.21.15.190:0/3548379834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6af4070780 0x7f6af40cc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:57.915 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.913+0000 7f6af9be7640 1 -- 172.21.15.190:0/3548379834 >> v1:172.21.15.190:6789/0 conn(0x7f6af40cce70 legacy=0x7f6af406c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:57.916 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.913+0000 7f6af9be7640 1 -- 172.21.15.190:0/3548379834 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af415d040 con 0x7f6af415cc60 2023-09-23T03:14:57.916 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.914+0000 7f6af9be7640 1 --2- 172.21.15.190:0/3548379834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6af415cc60 0x7f6af4069cd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6af0009870 tx=0x7f6af002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97313466e20c4167 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:14:57.916 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.915+0000 7f6af8be5640 1 -- 172.21.15.190:0/3548379834 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af0030840 con 0x7f6af415cc60 2023-09-23T03:14:57.916 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.915+0000 7f6af8be5640 1 -- 172.21.15.190:0/3548379834 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af00309a0 con 0x7f6af415cc60 2023-09-23T03:14:57.916 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.915+0000 7f6af8be5640 1 -- 172.21.15.190:0/3548379834 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af0030d30 con 0x7f6af415cc60 2023-09-23T03:14:57.917 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.916+0000 7f6afbe4b640 1 -- 172.21.15.190:0/3548379834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6af415cc60 msgr2=0x7f6af4069cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:57.917 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.916+0000 7f6afbe4b640 1 --2- 172.21.15.190:0/3548379834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6af415cc60 0x7f6af4069cd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6af0009870 tx=0x7f6af002b980 comp rx=0 tx=0).stop 2023-09-23T03:14:57.917 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.916+0000 7f6afbe4b640 1 -- 172.21.15.190:0/3548379834 shutdown_connections 2023-09-23T03:14:57.918 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.916+0000 7f6afbe4b640 1 --2- 172.21.15.190:0/3548379834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6af4070780 0x7f6af40cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:57.918 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.916+0000 7f6afbe4b640 1 --2- 172.21.15.190:0/3548379834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6af415cc60 0x7f6af4069cd0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:57.918 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.916+0000 7f6afbe4b640 1 -- 172.21.15.190:0/3548379834 >> 172.21.15.190:0/3548379834 conn(0x7f6af40ced60 msgr2=0x7f6af40cf170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:14:57.918 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.916+0000 7f6afbe4b640 1 -- 172.21.15.190:0/3548379834 shutdown_connections 2023-09-23T03:14:57.918 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.916+0000 7f6afbe4b640 1 -- 172.21.15.190:0/3548379834 wait complete. 2023-09-23T03:14:57.918 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.917+0000 7f6afbe4b640 1 Processor -- start 2023-09-23T03:14:57.919 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.918+0000 7f6afbe4b640 1 -- start start 2023-09-23T03:14:57.920 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.918+0000 7f6afbe4b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6af4070780 0x7f6af40c2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:57.920 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.918+0000 7f6afbe4b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6af40cce70 0x7f6af4159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:57.921 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.918+0000 7f6afbe4b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6af415cc60 0x7f6af40c26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:57.921 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.918+0000 7f6afbe4b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af40c2c20 con 0x7f6af40cce70 2023-09-23T03:14:57.921 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.919+0000 7f6afbe4b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af40c3430 con 0x7f6af415cc60 2023-09-23T03:14:57.921 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.919+0000 7f6afbe4b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af40c3710 con 0x7f6af4070780 2023-09-23T03:14:57.921 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.919+0000 7f6af9be7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6af4070780 0x7f6af40c2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:57.921 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.919+0000 7f6af9be7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6af4070780 0x7f6af40c2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:49790/0 (socket says 172.21.15.190:49790) 2023-09-23T03:14:57.921 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.919+0000 7f6af9be7640 1 -- 172.21.15.190:0/1016001478 learned_addr learned my addr 172.21.15.190:0/1016001478 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-09-23T03:14:57.921 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.919+0000 7f6afa3e8640 1 --2- 172.21.15.190:0/1016001478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6af415cc60 0x7f6af40c26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:57.921 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.919+0000 7f6af93e6640 1 --2- 172.21.15.190:0/1016001478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6af40cce70 0x7f6af4159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:57.921 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.919+0000 7f6af9be7640 1 -- 172.21.15.190:0/1016001478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6af415cc60 msgr2=0x7f6af40c26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:57.921 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.919+0000 7f6af9be7640 1 --2- 172.21.15.190:0/1016001478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6af415cc60 0x7f6af40c26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:57.922 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.919+0000 7f6af9be7640 1 -- 172.21.15.190:0/1016001478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6af40cce70 msgr2=0x7f6af4159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:57.922 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.919+0000 7f6af9be7640 1 --2- 172.21.15.190:0/1016001478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6af40cce70 0x7f6af4159630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:57.922 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.919+0000 7f6af9be7640 1 -- 172.21.15.190:0/1016001478 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af00094d0 con 0x7f6af4070780 2023-09-23T03:14:57.922 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.919+0000 7f6af9be7640 1 --2- 172.21.15.190:0/1016001478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6af4070780 0x7f6af40c2da0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6af0000c00 tx=0x7f6af0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:14:57.923 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.920+0000 7f6aea7fc640 1 -- 172.21.15.190:0/1016001478 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af0039a90 con 0x7f6af4070780 2023-09-23T03:14:57.923 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.920+0000 7f6aea7fc640 1 -- 172.21.15.190:0/1016001478 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af0039bf0 con 0x7f6af4070780 2023-09-23T03:14:57.923 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.920+0000 7f6aea7fc640 1 -- 172.21.15.190:0/1016001478 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af0043410 con 0x7f6af4070780 2023-09-23T03:14:57.923 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.920+0000 7f6afbe4b640 1 -- 172.21.15.190:0/1016001478 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6af40c39f0 con 0x7f6af4070780 2023-09-23T03:14:57.923 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.920+0000 7f6afbe4b640 1 -- 172.21.15.190:0/1016001478 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af415a640 con 0x7f6af4070780 2023-09-23T03:14:57.923 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.920+0000 7f6aea7fc640 1 -- 172.21.15.190:0/1016001478 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6af0043870 con 0x7f6af4070780 2023-09-23T03:14:57.923 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.920+0000 7f6aea7fc640 1 -- 172.21.15.190:0/1016001478 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f6af002f390 con 0x7f6af4070780 2023-09-23T03:14:57.923 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.921+0000 7f6afbe4b640 1 -- 172.21.15.190:0/1016001478 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6af404e920 con 0x7f6af4070780 2023-09-23T03:14:57.924 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:57.923+0000 7f6aea7fc640 1 -- 172.21.15.190:0/1016001478 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f6af0048070 con 0x7f6af4070780 2023-09-23T03:14:57.941 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-09-23T03:14:57.940+0000 7f46d5215040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-23T03:14:57.981 INFO:tasks.ceph.mgr.y.smithi154.stderr:2023-09-23T03:14:57.980+0000 7f0f6863f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-23T03:14:58.002 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.001+0000 7f6afbe4b640 1 -- 172.21.15.190:0/1016001478 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9e99c1ee-098c-40d9-b017-530bc1140196", "id": 2} v 0) v1 -- 0x7f6af40c3850 con 0x7f6af4070780 2023-09-23T03:14:58.009 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.008+0000 7f6aea7fc640 1 -- 172.21.15.190:0/1016001478 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9e99c1ee-098c-40d9-b017-530bc1140196", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6af40c3850 con 0x7f6af4070780 2023-09-23T03:14:58.009 INFO:teuthology.orchestra.run.smithi190.stdout:2 2023-09-23T03:14:58.010 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.009+0000 7f6afbe4b640 1 -- 172.21.15.190:0/1016001478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6af4070780 msgr2=0x7f6af40c2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:58.010 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.009+0000 7f6afbe4b640 1 --2- 172.21.15.190:0/1016001478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6af4070780 0x7f6af40c2da0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6af0000c00 tx=0x7f6af0004730 comp rx=0 tx=0).stop 2023-09-23T03:14:58.011 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.009+0000 7f6afbe4b640 1 -- 172.21.15.190:0/1016001478 shutdown_connections 2023-09-23T03:14:58.011 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.009+0000 7f6afbe4b640 1 --2- 172.21.15.190:0/1016001478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6af415cc60 0x7f6af40c26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:58.011 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.009+0000 7f6afbe4b640 1 --2- 172.21.15.190:0/1016001478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6af40cce70 0x7f6af4159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:58.011 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.009+0000 7f6afbe4b640 1 --2- 172.21.15.190:0/1016001478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6af4070780 0x7f6af40c2da0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:58.011 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.009+0000 7f6afbe4b640 1 -- 172.21.15.190:0/1016001478 >> 172.21.15.190:0/1016001478 conn(0x7f6af40ced60 msgr2=0x7f6af40ab8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:14:58.011 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.009+0000 7f6afbe4b640 1 -- 172.21.15.190:0/1016001478 shutdown_connections 2023-09-23T03:14:58.011 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.009+0000 7f6afbe4b640 1 -- 172.21.15.190:0/1016001478 wait complete. 2023-09-23T03:14:58.027 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new ac0e7a15-dbce-4197-af2e-396aabbb8de2 3 2023-09-23T03:14:58.035 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-09-23T03:14:58.034+0000 7f46d5215040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-23T03:14:58.036 INFO:tasks.ceph.mgr.y.smithi154.stderr:2023-09-23T03:14:58.035+0000 7f0f6863f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-23T03:14:58.094 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-09-23T03:14:58.092+0000 7f46d5215040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-23T03:14:58.095 INFO:tasks.ceph.mgr.y.smithi154.stderr:2023-09-23T03:14:58.095+0000 7f0f6863f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-23T03:14:58.150 INFO:tasks.ceph.mgr.y.smithi154.stderr:2023-09-23T03:14:58.150+0000 7f0f6863f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-23T03:14:58.158 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-09-23T03:14:58.157+0000 7f46d5215040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-23T03:14:58.216 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.215+0000 7f7b888d3640 1 Processor -- start 2023-09-23T03:14:58.216 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-09-23T03:14:58.215+0000 7f46d5215040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-23T03:14:58.217 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.215+0000 7f7b888d3640 1 -- start start 2023-09-23T03:14:58.218 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.216+0000 7f7b888d3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b840d0df0 0x7f7b8406c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:58.218 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.216+0000 7f7b888d3640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b8406c6a0 con 0x7f7b840b16c0 2023-09-23T03:14:58.218 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.216+0000 7f7b888d3640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b8406bc90 con 0x7f7b840d0a10 2023-09-23T03:14:58.218 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.216+0000 7f7b888d3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b840d1930 con 0x7f7b840d0df0 2023-09-23T03:14:58.218 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.216+0000 7f7b82d9d640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f7b840d0a10 0x7f7b8406ae70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:41904/0 (socket says 172.21.15.190:41904) 2023-09-23T03:14:58.218 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.216+0000 7f7b82d9d640 1 -- 172.21.15.190:0/1816940641 learned_addr learned my addr 172.21.15.190:0/1816940641 (peer_addr_for_me v1:172.21.15.190:0/0) 2023-09-23T03:14:58.218 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.216+0000 7f7b8359e640 1 --2- 172.21.15.190:0/1816940641 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b840d0df0 0x7f7b8406c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:58.219 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.217+0000 7f7b8259c640 1 -- 172.21.15.190:0/1816940641 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3838908090 0 0) 0x7f7b8406bc90 con 0x7f7b840d0a10 2023-09-23T03:14:58.219 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.217+0000 7f7b8259c640 1 -- 172.21.15.190:0/1816940641 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b64003520 con 0x7f7b840d0a10 2023-09-23T03:14:58.219 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.217+0000 7f7b8359e640 1 -- 172.21.15.190:0/1816940641 >> v1:172.21.15.190:6789/0 conn(0x7f7b840d0a10 legacy=0x7f7b8406ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:58.219 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.217+0000 7f7b8359e640 1 -- 172.21.15.190:0/1816940641 >> v1:172.21.15.154:6789/0 conn(0x7f7b840b16c0 legacy=0x7f7b8406b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:14:58.219 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.217+0000 7f7b8359e640 1 -- 172.21.15.190:0/1816940641 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b840d38d0 con 0x7f7b840d0df0 2023-09-23T03:14:58.219 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.218+0000 7f7b8359e640 1 --2- 172.21.15.190:0/1816940641 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b840d0df0 0x7f7b8406c160 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c009dc0 tx=0x7f7b7c02bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea61fa266357c52f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:14:58.220 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.219+0000 7f7b8259c640 1 -- 172.21.15.190:0/1816940641 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c02ccd0 con 0x7f7b840d0df0 2023-09-23T03:14:58.220 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.219+0000 7f7b8259c640 1 -- 172.21.15.190:0/1816940641 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7c02ce30 con 0x7f7b840d0df0 2023-09-23T03:14:58.220 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.219+0000 7f7b8259c640 1 -- 172.21.15.190:0/1816940641 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c034640 con 0x7f7b840d0df0 2023-09-23T03:14:58.221 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.219+0000 7f7b888d3640 1 -- 172.21.15.190:0/1816940641 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b840d0df0 msgr2=0x7f7b8406c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:58.221 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.219+0000 7f7b888d3640 1 --2- 172.21.15.190:0/1816940641 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b840d0df0 0x7f7b8406c160 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c009dc0 tx=0x7f7b7c02bf20 comp rx=0 tx=0).stop 2023-09-23T03:14:58.221 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.219+0000 7f7b888d3640 1 -- 172.21.15.190:0/1816940641 shutdown_connections 2023-09-23T03:14:58.221 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.219+0000 7f7b888d3640 1 --2- 172.21.15.190:0/1816940641 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b840d0df0 0x7f7b8406c160 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:58.221 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.219+0000 7f7b888d3640 1 -- 172.21.15.190:0/1816940641 >> 172.21.15.190:0/1816940641 conn(0x7f7b8409bc20 msgr2=0x7f7b840d29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:14:58.221 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.220+0000 7f7b888d3640 1 -- 172.21.15.190:0/1816940641 shutdown_connections 2023-09-23T03:14:58.221 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.220+0000 7f7b888d3640 1 -- 172.21.15.190:0/1816940641 wait complete. 2023-09-23T03:14:58.222 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.221+0000 7f7b888d3640 1 Processor -- start 2023-09-23T03:14:58.222 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.221+0000 7f7b888d3640 1 -- start start 2023-09-23T03:14:58.225 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.222+0000 7f7b888d3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b840b16c0 0x7f7b840c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:58.225 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.222+0000 7f7b888d3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b840d0a10 0x7f7b840b6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:58.225 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.222+0000 7f7b888d3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b840d0df0 0x7f7b840b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:58.225 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.222+0000 7f7b888d3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b840c8940 con 0x7f7b840d0a10 2023-09-23T03:14:58.225 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.222+0000 7f7b888d3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b840b81c0 con 0x7f7b840d0df0 2023-09-23T03:14:58.226 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.222+0000 7f7b888d3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b840b7640 con 0x7f7b840b16c0 2023-09-23T03:14:58.226 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.223+0000 7f7b82d9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b840b16c0 0x7f7b840c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:58.226 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.223+0000 7f7b8359e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b840d0df0 0x7f7b840b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:58.226 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.223+0000 7f7b82d9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b840b16c0 0x7f7b840c8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:49814/0 (socket says 172.21.15.190:49814) 2023-09-23T03:14:58.226 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.223+0000 7f7b8359e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b840d0df0 0x7f7b840b07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:33472/0 (socket says 172.21.15.190:33472) 2023-09-23T03:14:58.226 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.223+0000 7f7b82d9d640 1 -- 172.21.15.190:0/4167931954 learned_addr learned my addr 172.21.15.190:0/4167931954 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-09-23T03:14:58.226 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.223+0000 7f7b7bfff640 1 --2- 172.21.15.190:0/4167931954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b840d0a10 0x7f7b840b6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:58.226 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.223+0000 7f7b82d9d640 1 -- 172.21.15.190:0/4167931954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b840d0df0 msgr2=0x7f7b840b07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:58.226 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.223+0000 7f7b82d9d640 1 --2- 172.21.15.190:0/4167931954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b840d0df0 0x7f7b840b07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:58.226 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.223+0000 7f7b82d9d640 1 -- 172.21.15.190:0/4167931954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b840d0a10 msgr2=0x7f7b840b6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:58.226 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.223+0000 7f7b82d9d640 1 --2- 172.21.15.190:0/4167931954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b840d0a10 0x7f7b840b6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:58.227 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.223+0000 7f7b82d9d640 1 -- 172.21.15.190:0/4167931954 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b7c009a70 con 0x7f7b840b16c0 2023-09-23T03:14:58.227 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.223+0000 7f7b82d9d640 1 --2- 172.21.15.190:0/4167931954 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b840b16c0 0x7f7b840c8400 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7b74001260 tx=0x7f7b7400c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:14:58.227 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.224+0000 7f7b7b7fe640 1 -- 172.21.15.190:0/4167931954 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b740074e0 con 0x7f7b840b16c0 2023-09-23T03:14:58.227 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.224+0000 7f7b7b7fe640 1 -- 172.21.15.190:0/4167931954 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b74007640 con 0x7f7b840b16c0 2023-09-23T03:14:58.227 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.224+0000 7f7b7b7fe640 1 -- 172.21.15.190:0/4167931954 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b740059a0 con 0x7f7b840b16c0 2023-09-23T03:14:58.227 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.224+0000 7f7b888d3640 1 -- 172.21.15.190:0/4167931954 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b840b7920 con 0x7f7b840b16c0 2023-09-23T03:14:58.227 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.224+0000 7f7b888d3640 1 -- 172.21.15.190:0/4167931954 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b840b7ee0 con 0x7f7b840b16c0 2023-09-23T03:14:58.227 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.224+0000 7f7b7b7fe640 1 -- 172.21.15.190:0/4167931954 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7b7401c440 con 0x7f7b840b16c0 2023-09-23T03:14:58.227 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.224+0000 7f7b7b7fe640 1 -- 172.21.15.190:0/4167931954 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f7b7401cb80 con 0x7f7b840b16c0 2023-09-23T03:14:58.228 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.225+0000 7f7b888d3640 1 -- 172.21.15.190:0/4167931954 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b84067390 con 0x7f7b840b16c0 2023-09-23T03:14:58.228 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.227+0000 7f7b7b7fe640 1 -- 172.21.15.190:0/4167931954 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f7b74021060 con 0x7f7b840b16c0 2023-09-23T03:14:58.306 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.305+0000 7f7b888d3640 1 -- 172.21.15.190:0/4167931954 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ac0e7a15-dbce-4197-af2e-396aabbb8de2", "id": 3} v 0) v1 -- 0x7f7b840b1d40 con 0x7f7b840b16c0 2023-09-23T03:14:58.312 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.311+0000 7f7b7b7fe640 1 -- 172.21.15.190:0/4167931954 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ac0e7a15-dbce-4197-af2e-396aabbb8de2", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7b74017070 con 0x7f7b840b16c0 2023-09-23T03:14:58.312 INFO:teuthology.orchestra.run.smithi190.stdout:3 2023-09-23T03:14:58.313 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.312+0000 7f7b888d3640 1 -- 172.21.15.190:0/4167931954 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b840b16c0 msgr2=0x7f7b840c8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:58.314 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.312+0000 7f7b888d3640 1 --2- 172.21.15.190:0/4167931954 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b840b16c0 0x7f7b840c8400 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7b74001260 tx=0x7f7b7400c960 comp rx=0 tx=0).stop 2023-09-23T03:14:58.314 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.312+0000 7f7b888d3640 1 -- 172.21.15.190:0/4167931954 shutdown_connections 2023-09-23T03:14:58.314 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.312+0000 7f7b888d3640 1 --2- 172.21.15.190:0/4167931954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b840d0df0 0x7f7b840b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:58.314 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.312+0000 7f7b888d3640 1 --2- 172.21.15.190:0/4167931954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b840d0a10 0x7f7b840b6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:58.314 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.312+0000 7f7b888d3640 1 --2- 172.21.15.190:0/4167931954 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b840b16c0 0x7f7b840c8400 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:58.314 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.312+0000 7f7b888d3640 1 -- 172.21.15.190:0/4167931954 >> 172.21.15.190:0/4167931954 conn(0x7f7b8409bc20 msgr2=0x7f7b8415ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:14:58.314 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.312+0000 7f7b888d3640 1 -- 172.21.15.190:0/4167931954 shutdown_connections 2023-09-23T03:14:58.314 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.312+0000 7f7b888d3640 1 -- 172.21.15.190:0/4167931954 wait complete. 2023-09-23T03:14:58.329 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new d183a34d-8694-44b7-a523-785255ba1869 4 2023-09-23T03:14:58.519 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.517+0000 7f45c5458640 1 Processor -- start 2023-09-23T03:14:58.519 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.518+0000 7f45c5458640 1 -- start start 2023-09-23T03:14:58.521 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.519+0000 7f45c5458640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45c015c9c0 0x7f45c00d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:58.521 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.519+0000 7f45c5458640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45c00cd450 con 0x7f45c00d2d30 2023-09-23T03:14:58.521 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.519+0000 7f45c5458640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45c00d0df0 con 0x7f45c015cda0 2023-09-23T03:14:58.521 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.519+0000 7f45c5458640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c00d10d0 con 0x7f45c015c9c0 2023-09-23T03:14:58.521 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.519+0000 7f45beffd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45c015c9c0 0x7f45c00d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:58.521 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.520+0000 7f45b7fff640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f45c015cda0 0x7f45c00d2620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:41906/0 (socket says 172.21.15.190:41906) 2023-09-23T03:14:58.521 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.520+0000 7f45b7fff640 1 -- 172.21.15.190:0/1916757046 learned_addr learned my addr 172.21.15.190:0/1916757046 (peer_addr_for_me v1:172.21.15.190:0/0) 2023-09-23T03:14:58.522 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.520+0000 7f45beffd640 1 -- 172.21.15.190:0/1916757046 >> v1:172.21.15.190:6789/0 conn(0x7f45c015cda0 legacy=0x7f45c00d2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:58.522 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.520+0000 7f45beffd640 1 -- 172.21.15.190:0/1916757046 >> v1:172.21.15.154:6789/0 conn(0x7f45c00d2d30 legacy=0x7f45c00cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:14:58.522 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.520+0000 7f45be7fc640 1 -- 172.21.15.190:0/1916757046 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4057128080 0 0) 0x7f45c00d0df0 con 0x7f45c015cda0 2023-09-23T03:14:58.522 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.520+0000 7f45beffd640 1 -- 172.21.15.190:0/1916757046 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c00d13b0 con 0x7f45c015c9c0 2023-09-23T03:14:58.523 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.521+0000 7f45beffd640 1 --2- 172.21.15.190:0/1916757046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45c015c9c0 0x7f45c00d04a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f45b00097c0 tx=0x7f45b002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7c1b6e1fe97f121 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:14:58.523 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.521+0000 7f45be7fc640 1 -- 172.21.15.190:0/1916757046 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b0031800 con 0x7f45c015c9c0 2023-09-23T03:14:58.523 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.521+0000 7f45be7fc640 1 -- 172.21.15.190:0/1916757046 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b0031960 con 0x7f45c015c9c0 2023-09-23T03:14:58.523 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.521+0000 7f45be7fc640 1 -- 172.21.15.190:0/1916757046 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b0031cf0 con 0x7f45c015c9c0 2023-09-23T03:14:58.524 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.522+0000 7f45c5458640 1 -- 172.21.15.190:0/1916757046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45c015c9c0 msgr2=0x7f45c00d04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:58.524 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.522+0000 7f45c5458640 1 --2- 172.21.15.190:0/1916757046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45c015c9c0 0x7f45c00d04a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f45b00097c0 tx=0x7f45b002b8a0 comp rx=0 tx=0).stop 2023-09-23T03:14:58.524 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.522+0000 7f45c5458640 1 -- 172.21.15.190:0/1916757046 shutdown_connections 2023-09-23T03:14:58.524 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.522+0000 7f45c5458640 1 --2- 172.21.15.190:0/1916757046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45c015c9c0 0x7f45c00d04a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:58.524 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.522+0000 7f45c5458640 1 -- 172.21.15.190:0/1916757046 >> 172.21.15.190:0/1916757046 conn(0x7f45c009bc20 msgr2=0x7f45c0089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:14:58.524 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.522+0000 7f45c5458640 1 -- 172.21.15.190:0/1916757046 shutdown_connections 2023-09-23T03:14:58.524 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.522+0000 7f45c5458640 1 -- 172.21.15.190:0/1916757046 wait complete. 2023-09-23T03:14:58.525 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.524+0000 7f45c5458640 1 Processor -- start 2023-09-23T03:14:58.525 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.524+0000 7f45c5458640 1 -- start start 2023-09-23T03:14:58.527 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.525+0000 7f45c5458640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45c00d2d30 0x7f45c015a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:58.527 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.525+0000 7f45c5458640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45c015c9c0 0x7f45c00ca060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:58.528 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.525+0000 7f45c5458640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45c015cda0 0x7f45c015f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:58.528 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.525+0000 7f45c5458640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c00c9350 con 0x7f45c00d2d30 2023-09-23T03:14:58.528 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.525+0000 7f45c5458640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c00c9c40 con 0x7f45c015cda0 2023-09-23T03:14:58.528 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.525+0000 7f45c5458640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c00c8ab0 con 0x7f45c015c9c0 2023-09-23T03:14:58.528 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.525+0000 7f45b7fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45c015c9c0 0x7f45c00ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:58.528 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.525+0000 7f45b7fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45c015c9c0 0x7f45c00ca060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:49828/0 (socket says 172.21.15.190:49828) 2023-09-23T03:14:58.528 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.525+0000 7f45b7fff640 1 -- 172.21.15.190:0/3868089393 learned_addr learned my addr 172.21.15.190:0/3868089393 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-09-23T03:14:58.528 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.525+0000 7f45bf7fe640 1 --2- 172.21.15.190:0/3868089393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45c015cda0 0x7f45c015f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:58.528 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.525+0000 7f45beffd640 1 --2- 172.21.15.190:0/3868089393 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45c00d2d30 0x7f45c015a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:58.528 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.526+0000 7f45b7fff640 1 -- 172.21.15.190:0/3868089393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45c015cda0 msgr2=0x7f45c015f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:58.529 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.526+0000 7f45b7fff640 1 --2- 172.21.15.190:0/3868089393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45c015cda0 0x7f45c015f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:58.529 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.526+0000 7f45b7fff640 1 -- 172.21.15.190:0/3868089393 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45c00d2d30 msgr2=0x7f45c015a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:58.529 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.526+0000 7f45b7fff640 1 --2- 172.21.15.190:0/3868089393 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45c00d2d30 0x7f45c015a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:58.529 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.526+0000 7f45b7fff640 1 -- 172.21.15.190:0/3868089393 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b00093a0 con 0x7f45c015c9c0 2023-09-23T03:14:58.529 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.526+0000 7f45bf7fe640 1 --2- 172.21.15.190:0/3868089393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45c015cda0 0x7f45c015f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:14:58.529 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.526+0000 7f45b7fff640 1 --2- 172.21.15.190:0/3868089393 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45c015c9c0 0x7f45c00ca060 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f45a800c460 tx=0x7f45a800c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:14:58.529 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.526+0000 7f45b77fe640 1 -- 172.21.15.190:0/3868089393 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a80074e0 con 0x7f45c015c9c0 2023-09-23T03:14:58.529 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.526+0000 7f45b77fe640 1 -- 172.21.15.190:0/3868089393 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a8007660 con 0x7f45c015c9c0 2023-09-23T03:14:58.529 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.526+0000 7f45b77fe640 1 -- 172.21.15.190:0/3868089393 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a8004a20 con 0x7f45c015c9c0 2023-09-23T03:14:58.529 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.526+0000 7f45c5458640 1 -- 172.21.15.190:0/3868089393 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45c00c8d90 con 0x7f45c015c9c0 2023-09-23T03:14:58.529 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.526+0000 7f45c5458640 1 -- 172.21.15.190:0/3868089393 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45c015a850 con 0x7f45c015c9c0 2023-09-23T03:14:58.529 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.526+0000 7f45b77fe640 1 -- 172.21.15.190:0/3868089393 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f45a8012400 con 0x7f45c015c9c0 2023-09-23T03:14:58.530 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.527+0000 7f45b77fe640 1 -- 172.21.15.190:0/3868089393 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f45a8012bc0 con 0x7f45c015c9c0 2023-09-23T03:14:58.530 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.528+0000 7f45c5458640 1 -- 172.21.15.190:0/3868089393 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45c00c9980 con 0x7f45c015c9c0 2023-09-23T03:14:58.531 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.530+0000 7f45b77fe640 1 -- 172.21.15.190:0/3868089393 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f45a8018070 con 0x7f45c015c9c0 2023-09-23T03:14:58.604 INFO:tasks.ceph.mgr.y.smithi154.stderr:2023-09-23T03:14:58.603+0000 7f0f6863f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-23T03:14:58.605 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.604+0000 7f45c5458640 1 -- 172.21.15.190:0/3868089393 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d183a34d-8694-44b7-a523-785255ba1869", "id": 4} v 0) v1 -- 0x7f45c006cd90 con 0x7f45c015c9c0 2023-09-23T03:14:58.612 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.611+0000 7f45b77fe640 1 -- 172.21.15.190:0/3868089393 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d183a34d-8694-44b7-a523-785255ba1869", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f45a80126b0 con 0x7f45c015c9c0 2023-09-23T03:14:58.612 INFO:teuthology.orchestra.run.smithi190.stdout:4 2023-09-23T03:14:58.614 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.612+0000 7f45c5458640 1 -- 172.21.15.190:0/3868089393 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45c015c9c0 msgr2=0x7f45c00ca060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:58.614 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.612+0000 7f45c5458640 1 --2- 172.21.15.190:0/3868089393 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45c015c9c0 0x7f45c00ca060 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f45a800c460 tx=0x7f45a800c920 comp rx=0 tx=0).stop 2023-09-23T03:14:58.614 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.612+0000 7f45c5458640 1 -- 172.21.15.190:0/3868089393 shutdown_connections 2023-09-23T03:14:58.614 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.612+0000 7f45c5458640 1 --2- 172.21.15.190:0/3868089393 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45c015cda0 0x7f45c015f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:58.614 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.612+0000 7f45c5458640 1 --2- 172.21.15.190:0/3868089393 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45c015c9c0 0x7f45c00ca060 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:58.615 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.612+0000 7f45c5458640 1 --2- 172.21.15.190:0/3868089393 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45c00d2d30 0x7f45c015a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:58.615 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.612+0000 7f45c5458640 1 -- 172.21.15.190:0/3868089393 >> 172.21.15.190:0/3868089393 conn(0x7f45c009bc20 msgr2=0x7f45c00cd930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:14:58.615 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.613+0000 7f45c5458640 1 -- 172.21.15.190:0/3868089393 shutdown_connections 2023-09-23T03:14:58.615 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.613+0000 7f45c5458640 1 -- 172.21.15.190:0/3868089393 wait complete. 2023-09-23T03:14:58.635 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 461ae909-2278-4b66-b35c-6a32ea06fbc6 5 2023-09-23T03:14:58.703 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-09-23T03:14:58.702+0000 7f46d5215040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-23T03:14:58.763 INFO:tasks.ceph.mgr.y.smithi154.stderr:2023-09-23T03:14:58.763+0000 7f0f6863f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-23T03:14:58.829 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.828+0000 7f77ca59e640 1 Processor -- start 2023-09-23T03:14:58.829 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.828+0000 7f77ca59e640 1 -- start start 2023-09-23T03:14:58.832 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.830+0000 7f77ca59e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f77c415c9c0 0x7f77c40d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:58.832 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.830+0000 7f77ca59e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f77c415cda0 0x7f77c40d2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:58.832 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.830+0000 7f77ca59e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77c40cd450 con 0x7f77c40d2b60 2023-09-23T03:14:58.832 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.830+0000 7f77ca59e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77c40d0df0 con 0x7f77c415c9c0 2023-09-23T03:14:58.832 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.830+0000 7f77ca59e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77c40d10d0 con 0x7f77c415cda0 2023-09-23T03:14:58.832 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.830+0000 7f77c37fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f77c415cda0 0x7f77c40d2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:58.832 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.830+0000 7f77c37fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f77c415cda0 0x7f77c40d2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:33484/0 (socket says 172.21.15.190:33484) 2023-09-23T03:14:58.832 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.830+0000 7f77c37fe640 1 -- 172.21.15.190:0/3027398103 learned_addr learned my addr 172.21.15.190:0/3027398103 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-09-23T03:14:58.833 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.830+0000 7f77c3fff640 1 --2- 172.21.15.190:0/3027398103 >> v2:172.21.15.154:3300/0 conn(0x7f77c415c9c0 0x7f77c40d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:58.833 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.831+0000 7f77c37fe640 1 -- 172.21.15.190:0/3027398103 >> v2:172.21.15.154:3300/0 conn(0x7f77c415c9c0 msgr2=0x7f77c40d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:14:58.833 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.831+0000 7f77c37fe640 1 --2- 172.21.15.190:0/3027398103 >> v2:172.21.15.154:3300/0 conn(0x7f77c415c9c0 0x7f77c40d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:58.833 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.831+0000 7f77c37fe640 1 -- 172.21.15.190:0/3027398103 >> v1:172.21.15.190:6789/0 conn(0x7f77c40d2b60 legacy=0x7f77c40cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:58.833 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.831+0000 7f77c37fe640 1 -- 172.21.15.190:0/3027398103 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c40d13b0 con 0x7f77c415cda0 2023-09-23T03:14:58.833 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.831+0000 7f77c3fff640 1 --2- 172.21.15.190:0/3027398103 >> v2:172.21.15.154:3300/0 conn(0x7f77c415c9c0 0x7f77c40d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:14:58.833 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.832+0000 7f77c37fe640 1 --2- 172.21.15.190:0/3027398103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77c415cda0 0x7f77c40d2620 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f77c409c330 tx=0x7f77ac02b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=548444931d5468c8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:14:58.834 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.832+0000 7f77c2ffd640 1 -- 172.21.15.190:0/3027398103 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac030860 con 0x7f77c415cda0 2023-09-23T03:14:58.834 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.832+0000 7f77c2ffd640 1 -- 172.21.15.190:0/3027398103 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77ac0309c0 con 0x7f77c415cda0 2023-09-23T03:14:58.834 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.832+0000 7f77c2ffd640 1 -- 172.21.15.190:0/3027398103 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac030d50 con 0x7f77c415cda0 2023-09-23T03:14:58.835 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.833+0000 7f77ca59e640 1 -- 172.21.15.190:0/3027398103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77c415cda0 msgr2=0x7f77c40d2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:58.835 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.833+0000 7f77ca59e640 1 --2- 172.21.15.190:0/3027398103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77c415cda0 0x7f77c40d2620 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f77c409c330 tx=0x7f77ac02b9a0 comp rx=0 tx=0).stop 2023-09-23T03:14:58.835 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.833+0000 7f77ca59e640 1 -- 172.21.15.190:0/3027398103 shutdown_connections 2023-09-23T03:14:58.835 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.833+0000 7f77ca59e640 1 --2- 172.21.15.190:0/3027398103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77c415cda0 0x7f77c40d2620 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:58.835 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.833+0000 7f77ca59e640 1 --2- 172.21.15.190:0/3027398103 >> v2:172.21.15.154:3300/0 conn(0x7f77c415c9c0 0x7f77c40d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:58.835 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.833+0000 7f77ca59e640 1 -- 172.21.15.190:0/3027398103 >> 172.21.15.190:0/3027398103 conn(0x7f77c409bc20 msgr2=0x7f77c4089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:14:58.835 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.833+0000 7f77ca59e640 1 -- 172.21.15.190:0/3027398103 shutdown_connections 2023-09-23T03:14:58.835 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.833+0000 7f77ca59e640 1 -- 172.21.15.190:0/3027398103 wait complete. 2023-09-23T03:14:58.836 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.835+0000 7f77ca59e640 1 Processor -- start 2023-09-23T03:14:58.836 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.835+0000 7f77ca59e640 1 -- start start 2023-09-23T03:14:58.838 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.836+0000 7f77ca59e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77c40d2b60 0x7f77c4153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:58.838 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.836+0000 7f77ca59e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77c415c9c0 0x7f77c415e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:58.838 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.836+0000 7f77ca59e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77c415cda0 0x7f77c40cd830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:58.838 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.836+0000 7f77ca59e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c415ef30 con 0x7f77c415cda0 2023-09-23T03:14:58.838 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.836+0000 7f77ca59e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c415f210 con 0x7f77c415c9c0 2023-09-23T03:14:58.838 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.836+0000 7f77ca59e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77c415f4f0 con 0x7f77c40d2b60 2023-09-23T03:14:58.838 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.836+0000 7f77c3fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77c40d2b60 0x7f77c4153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:58.838 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.836+0000 7f77c37fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77c415c9c0 0x7f77c415e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:58.838 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.836+0000 7f77c3fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77c40d2b60 0x7f77c4153f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:49836/0 (socket says 172.21.15.190:49836) 2023-09-23T03:14:58.839 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.836+0000 7f77c3fff640 1 -- 172.21.15.190:0/2916016367 learned_addr learned my addr 172.21.15.190:0/2916016367 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-09-23T03:14:58.839 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.836+0000 7f77c8b3b640 1 --2- 172.21.15.190:0/2916016367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77c415cda0 0x7f77c40cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:58.839 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.837+0000 7f77c3fff640 1 -- 172.21.15.190:0/2916016367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77c415c9c0 msgr2=0x7f77c415e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:58.839 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.837+0000 7f77c3fff640 1 --2- 172.21.15.190:0/2916016367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77c415c9c0 0x7f77c415e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:58.839 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.837+0000 7f77c3fff640 1 -- 172.21.15.190:0/2916016367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77c415cda0 msgr2=0x7f77c40cd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:58.839 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.837+0000 7f77c3fff640 1 --2- 172.21.15.190:0/2916016367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77c415cda0 0x7f77c40cd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:58.839 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.837+0000 7f77c3fff640 1 -- 172.21.15.190:0/2916016367 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77ac0094d0 con 0x7f77c40d2b60 2023-09-23T03:14:58.839 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.837+0000 7f77c3fff640 1 --2- 172.21.15.190:0/2916016367 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77c40d2b60 0x7f77c4153f80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f77b4007fa0 tx=0x7f77b400d560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:14:58.840 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.837+0000 7f77c0ff9640 1 -- 172.21.15.190:0/2916016367 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b4003b90 con 0x7f77c40d2b60 2023-09-23T03:14:58.840 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.837+0000 7f77c0ff9640 1 -- 172.21.15.190:0/2916016367 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77b400f040 con 0x7f77c40d2b60 2023-09-23T03:14:58.840 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.837+0000 7f77c0ff9640 1 -- 172.21.15.190:0/2916016367 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b4004030 con 0x7f77c40d2b60 2023-09-23T03:14:58.840 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.837+0000 7f77ca59e640 1 -- 172.21.15.190:0/2916016367 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77c415f7d0 con 0x7f77c40d2b60 2023-09-23T03:14:58.840 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.837+0000 7f77ca59e640 1 -- 172.21.15.190:0/2916016367 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77c40c9050 con 0x7f77c40d2b60 2023-09-23T03:14:58.841 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.838+0000 7f77c0ff9640 1 -- 172.21.15.190:0/2916016367 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f77b4004490 con 0x7f77c40d2b60 2023-09-23T03:14:58.841 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.838+0000 7f77c0ff9640 1 -- 172.21.15.190:0/2916016367 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f77b4004740 con 0x7f77c40d2b60 2023-09-23T03:14:58.841 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.838+0000 7f77ca59e640 1 -- 172.21.15.190:0/2916016367 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77c404e920 con 0x7f77c40d2b60 2023-09-23T03:14:58.843 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.842+0000 7f77c0ff9640 1 -- 172.21.15.190:0/2916016367 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f77b4024060 con 0x7f77c40d2b60 2023-09-23T03:14:58.874 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-09-23T03:14:58.873+0000 7f46d5215040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-23T03:14:58.897 INFO:tasks.ceph.mgr.y.smithi154.stderr:2023-09-23T03:14:58.896+0000 7f0f6863f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-23T03:14:58.918 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.917+0000 7f77ca59e640 1 -- 172.21.15.190:0/2916016367 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd new", "uuid": "461ae909-2278-4b66-b35c-6a32ea06fbc6", "id": 5} v 0) v1 -- 0x7f77c40c61b0 con 0x7f77c40d2b60 2023-09-23T03:14:58.925 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.923+0000 7f77c0ff9640 1 -- 172.21.15.190:0/2916016367 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "461ae909-2278-4b66-b35c-6a32ea06fbc6", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f77b401a070 con 0x7f77c40d2b60 2023-09-23T03:14:58.925 INFO:teuthology.orchestra.run.smithi190.stdout:5 2023-09-23T03:14:58.926 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.925+0000 7f77ca59e640 1 -- 172.21.15.190:0/2916016367 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77c40d2b60 msgr2=0x7f77c4153f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:58.927 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.925+0000 7f77ca59e640 1 --2- 172.21.15.190:0/2916016367 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77c40d2b60 0x7f77c4153f80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f77b4007fa0 tx=0x7f77b400d560 comp rx=0 tx=0).stop 2023-09-23T03:14:58.927 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.925+0000 7f77ca59e640 1 -- 172.21.15.190:0/2916016367 shutdown_connections 2023-09-23T03:14:58.927 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.925+0000 7f77ca59e640 1 --2- 172.21.15.190:0/2916016367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f77c415cda0 0x7f77c40cd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:58.927 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.925+0000 7f77ca59e640 1 --2- 172.21.15.190:0/2916016367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f77c415c9c0 0x7f77c415e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:58.927 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.925+0000 7f77ca59e640 1 --2- 172.21.15.190:0/2916016367 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f77c40d2b60 0x7f77c4153f80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:58.927 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.925+0000 7f77ca59e640 1 -- 172.21.15.190:0/2916016367 >> 172.21.15.190:0/2916016367 conn(0x7f77c409bc20 msgr2=0x7f77c406af50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:14:58.927 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.925+0000 7f77ca59e640 1 -- 172.21.15.190:0/2916016367 shutdown_connections 2023-09-23T03:14:58.927 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:58.925+0000 7f77ca59e640 1 -- 172.21.15.190:0/2916016367 wait complete. 2023-09-23T03:14:58.942 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 579b92c0-3741-4058-9173-eb780c598577 6 2023-09-23T03:14:58.950 INFO:tasks.ceph.mgr.y.smithi154.stderr:2023-09-23T03:14:58.949+0000 7f0f6863f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-23T03:14:59.025 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-09-23T03:14:59.024+0000 7f46d5215040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-23T03:14:59.058 INFO:tasks.ceph.mgr.y.smithi154.stderr:2023-09-23T03:14:59.058+0000 7f0f6863f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-23T03:14:59.081 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-09-23T03:14:59.079+0000 7f46d5215040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-23T03:14:59.136 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.135+0000 7f6f8ea83640 1 Processor -- start 2023-09-23T03:14:59.136 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.135+0000 7f6f8ea83640 1 -- start start 2023-09-23T03:14:59.138 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.136+0000 7f6f8ea83640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f8815caa0 0x7f6f880d0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:59.138 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.136+0000 7f6f8ea83640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6f8806bf30 0x7f6f880ceba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:59.138 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.137+0000 7f6f8ea83640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f880cb010 con 0x7f6f8815ce80 2023-09-23T03:14:59.138 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.137+0000 7f6f8ea83640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f8806c490 con 0x7f6f8806bf30 2023-09-23T03:14:59.138 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.137+0000 7f6f8ea83640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8806c770 con 0x7f6f8815caa0 2023-09-23T03:14:59.139 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.137+0000 7f6f8da81640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f8815caa0 0x7f6f880d0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:59.139 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.137+0000 7f6f8da81640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f8815caa0 0x7f6f880d0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:49848/0 (socket says 172.21.15.190:49848) 2023-09-23T03:14:59.139 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.137+0000 7f6f8da81640 1 -- 172.21.15.190:0/3632277931 learned_addr learned my addr 172.21.15.190:0/3632277931 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-09-23T03:14:59.139 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.137+0000 7f6f8d280640 1 --2- 172.21.15.190:0/3632277931 >> v2:172.21.15.154:3300/0 conn(0x7f6f8806bf30 0x7f6f880ceba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:59.139 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.137+0000 7f6f8da81640 1 -- 172.21.15.190:0/3632277931 >> v2:172.21.15.154:3300/0 conn(0x7f6f8806bf30 msgr2=0x7f6f880ceba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:14:59.139 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.137+0000 7f6f8da81640 1 --2- 172.21.15.190:0/3632277931 >> v2:172.21.15.154:3300/0 conn(0x7f6f8806bf30 0x7f6f880ceba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:59.139 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.137+0000 7f6f8da81640 1 -- 172.21.15.190:0/3632277931 >> v1:172.21.15.154:6789/0 conn(0x7f6f8815ce80 legacy=0x7f6f880cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:14:59.139 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.137+0000 7f6f8da81640 1 -- 172.21.15.190:0/3632277931 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f8806ca50 con 0x7f6f8815caa0 2023-09-23T03:14:59.141 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.138+0000 7f6f8da81640 1 --2- 172.21.15.190:0/3632277931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f8815caa0 0x7f6f880d0e80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6f84009870 tx=0x7f6f8402b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8735eb9fad2b10d7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:14:59.141 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.139+0000 7f6f8ca7f640 1 -- 172.21.15.190:0/3632277931 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f840317c0 con 0x7f6f8815caa0 2023-09-23T03:14:59.141 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.139+0000 7f6f8ca7f640 1 -- 172.21.15.190:0/3632277931 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f84031920 con 0x7f6f8815caa0 2023-09-23T03:14:59.141 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.139+0000 7f6f8ca7f640 1 -- 172.21.15.190:0/3632277931 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f84031cb0 con 0x7f6f8815caa0 2023-09-23T03:14:59.141 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.139+0000 7f6f8ea83640 1 -- 172.21.15.190:0/3632277931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f8815caa0 msgr2=0x7f6f880d0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:59.141 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.139+0000 7f6f8ea83640 1 --2- 172.21.15.190:0/3632277931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f8815caa0 0x7f6f880d0e80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6f84009870 tx=0x7f6f8402b860 comp rx=0 tx=0).stop 2023-09-23T03:14:59.141 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.139+0000 7f6f8ea83640 1 -- 172.21.15.190:0/3632277931 shutdown_connections 2023-09-23T03:14:59.141 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.139+0000 7f6f8ea83640 1 --2- 172.21.15.190:0/3632277931 >> v2:172.21.15.154:3300/0 conn(0x7f6f8806bf30 0x7f6f880ceba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:59.141 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.139+0000 7f6f8ea83640 1 --2- 172.21.15.190:0/3632277931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f8815caa0 0x7f6f880d0e80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:59.141 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.139+0000 7f6f8ea83640 1 -- 172.21.15.190:0/3632277931 >> 172.21.15.190:0/3632277931 conn(0x7f6f880998e0 msgr2=0x7f6f880d05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:14:59.141 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.139+0000 7f6f8ea83640 1 -- 172.21.15.190:0/3632277931 shutdown_connections 2023-09-23T03:14:59.142 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.139+0000 7f6f8ea83640 1 -- 172.21.15.190:0/3632277931 wait complete. 2023-09-23T03:14:59.142 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.141+0000 7f6f8ea83640 1 Processor -- start 2023-09-23T03:14:59.142 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.141+0000 7f6f8ea83640 1 -- start start 2023-09-23T03:14:59.144 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.142+0000 7f6f8ea83640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f8806bf30 0x7f6f880ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:59.144 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.142+0000 7f6f8ea83640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f8815caa0 0x7f6f880b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:59.144 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.142+0000 7f6f8ea83640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f8815ce80 0x7f6f880b5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:59.144 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.142+0000 7f6f8ea83640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f880c7bc0 con 0x7f6f8815caa0 2023-09-23T03:14:59.144 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.142+0000 7f6f8ea83640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f880b3980 con 0x7f6f8815ce80 2023-09-23T03:14:59.144 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.142+0000 7f6f8ea83640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f880b6130 con 0x7f6f8806bf30 2023-09-23T03:14:59.144 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.142+0000 7f6f8e282640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f8815ce80 0x7f6f880b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:59.145 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.142+0000 7f6f8da81640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f8806bf30 0x7f6f880ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:59.145 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.142+0000 7f6f8e282640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f8815ce80 0x7f6f880b5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:33492/0 (socket says 172.21.15.190:33492) 2023-09-23T03:14:59.145 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.142+0000 7f6f8e282640 1 -- 172.21.15.190:0/189725248 learned_addr learned my addr 172.21.15.190:0/189725248 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-09-23T03:14:59.145 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.142+0000 7f6f8da81640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f8806bf30 0x7f6f880ca1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.190:49854/0 (socket says 172.21.15.190:49854) 2023-09-23T03:14:59.145 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.143+0000 7f6f8d280640 1 --2- 172.21.15.190:0/189725248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f8815caa0 0x7f6f880b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:59.145 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.143+0000 7f6f8da81640 1 -- 172.21.15.190:0/189725248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f8815ce80 msgr2=0x7f6f880b5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:59.145 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.143+0000 7f6f8da81640 1 --2- 172.21.15.190:0/189725248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f8815ce80 0x7f6f880b5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:59.145 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.143+0000 7f6f8da81640 1 -- 172.21.15.190:0/189725248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f8815caa0 msgr2=0x7f6f880b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:59.145 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.143+0000 7f6f8da81640 1 --2- 172.21.15.190:0/189725248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f8815caa0 0x7f6f880b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:59.145 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.143+0000 7f6f8da81640 1 -- 172.21.15.190:0/189725248 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f840094d0 con 0x7f6f8806bf30 2023-09-23T03:14:59.145 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.143+0000 7f6f8d280640 1 --2- 172.21.15.190:0/189725248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f8815caa0 0x7f6f880b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:14:59.146 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.143+0000 7f6f8da81640 1 --2- 172.21.15.190:0/189725248 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f8806bf30 0x7f6f880ca1b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6f8806c8b0 tx=0x7f6f840046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:14:59.147 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.143+0000 7f6f7e7fc640 1 -- 172.21.15.190:0/189725248 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f84032860 con 0x7f6f8806bf30 2023-09-23T03:14:59.147 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.143+0000 7f6f7e7fc640 1 -- 172.21.15.190:0/189725248 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f840329c0 con 0x7f6f8806bf30 2023-09-23T03:14:59.147 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.143+0000 7f6f8ea83640 1 -- 172.21.15.190:0/189725248 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f880b36e0 con 0x7f6f8806bf30 2023-09-23T03:14:59.147 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.144+0000 7f6f7e7fc640 1 -- 172.21.15.190:0/189725248 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f84032d70 con 0x7f6f8806bf30 2023-09-23T03:14:59.147 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.144+0000 7f6f8ea83640 1 -- 172.21.15.190:0/189725248 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f880b5e50 con 0x7f6f8806bf30 2023-09-23T03:14:59.147 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.144+0000 7f6f7e7fc640 1 -- 172.21.15.190:0/189725248 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6f84045700 con 0x7f6f8806bf30 2023-09-23T03:14:59.147 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.144+0000 7f6f7e7fc640 1 -- 172.21.15.190:0/189725248 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f6f8403b620 con 0x7f6f8806bf30 2023-09-23T03:14:59.147 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.145+0000 7f6f8ea83640 1 -- 172.21.15.190:0/189725248 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f88067390 con 0x7f6f8806bf30 2023-09-23T03:14:59.148 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.147+0000 7f6f7e7fc640 1 -- 172.21.15.190:0/189725248 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f6f84049070 con 0x7f6f8806bf30 2023-09-23T03:14:59.171 INFO:tasks.ceph.mgr.y.smithi154.stderr:2023-09-23T03:14:59.171+0000 7f0f6863f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-23T03:14:59.197 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-09-23T03:14:59.196+0000 7f46d5215040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-23T03:14:59.226 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.224+0000 7f6f8ea83640 1 -- 172.21.15.190:0/189725248 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd new", "uuid": "579b92c0-3741-4058-9173-eb780c598577", "id": 6} v 0) v1 -- 0x7f6f8808a930 con 0x7f6f8806bf30 2023-09-23T03:14:59.232 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.231+0000 7f6f7e7fc640 1 -- 172.21.15.190:0/189725248 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "579b92c0-3741-4058-9173-eb780c598577", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6f840459b0 con 0x7f6f8806bf30 2023-09-23T03:14:59.232 INFO:teuthology.orchestra.run.smithi190.stdout:6 2023-09-23T03:14:59.233 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.232+0000 7f6f8ea83640 1 -- 172.21.15.190:0/189725248 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f8806bf30 msgr2=0x7f6f880ca1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:59.234 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.232+0000 7f6f8ea83640 1 --2- 172.21.15.190:0/189725248 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f8806bf30 0x7f6f880ca1b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6f8806c8b0 tx=0x7f6f840046c0 comp rx=0 tx=0).stop 2023-09-23T03:14:59.234 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.232+0000 7f6f8ea83640 1 -- 172.21.15.190:0/189725248 shutdown_connections 2023-09-23T03:14:59.234 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.232+0000 7f6f8ea83640 1 --2- 172.21.15.190:0/189725248 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6f8815ce80 0x7f6f880b5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:59.234 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.232+0000 7f6f8ea83640 1 --2- 172.21.15.190:0/189725248 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f8815caa0 0x7f6f880b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:59.234 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.232+0000 7f6f8ea83640 1 --2- 172.21.15.190:0/189725248 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6f8806bf30 0x7f6f880ca1b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:59.234 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.232+0000 7f6f8ea83640 1 -- 172.21.15.190:0/189725248 >> 172.21.15.190:0/189725248 conn(0x7f6f880998e0 msgr2=0x7f6f880cce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:14:59.234 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.232+0000 7f6f8ea83640 1 -- 172.21.15.190:0/189725248 shutdown_connections 2023-09-23T03:14:59.234 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.233+0000 7f6f8ea83640 1 -- 172.21.15.190:0/189725248 wait complete. 2023-09-23T03:14:59.250 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --cluster ceph osd new 14ad9957-5ab0-43b2-8a1d-be664cbb9756 7 2023-09-23T03:14:59.317 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-09-23T03:14:59.316+0000 7f46d5215040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-23T03:14:59.330 INFO:tasks.ceph.mgr.y.smithi154.stderr:2023-09-23T03:14:59.330+0000 7f0f6863f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-23T03:14:59.379 INFO:tasks.ceph.mgr.y.smithi154.stderr:2023-09-23T03:14:59.379+0000 7f0f6863f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-23T03:14:59.445 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.443+0000 7f7c47fff640 1 Processor -- start 2023-09-23T03:14:59.445 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.443+0000 7f7c47fff640 1 -- start start 2023-09-23T03:14:59.446 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.445+0000 7f7c47fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7c481534d0 0x7f7c480c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:59.447 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.445+0000 7f7c47fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7c48070630 0x7f7c480afeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:59.447 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.445+0000 7f7c47fff640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c480ca1e0 con 0x7f7c480c8fc0 2023-09-23T03:14:59.447 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.445+0000 7f7c47fff640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c4815d9a0 con 0x7f7c481534d0 2023-09-23T03:14:59.447 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.445+0000 7f7c47fff640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c481538b0 con 0x7f7c48070630 2023-09-23T03:14:59.447 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.445+0000 7f7c477fe640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f7c480c8fc0 0x7f7c4806be60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.190:41918/0 (socket says 172.21.15.190:41918) 2023-09-23T03:14:59.447 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.445+0000 7f7c477fe640 1 -- 172.21.15.190:0/2405884647 learned_addr learned my addr 172.21.15.190:0/2405884647 (peer_addr_for_me v1:172.21.15.190:0/0) 2023-09-23T03:14:59.447 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.445+0000 7f7c467fc640 1 --2- 172.21.15.190:0/2405884647 >> v2:172.21.15.190:3300/0 conn(0x7f7c48070630 0x7f7c480afeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:59.447 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.446+0000 7f7c46ffd640 1 --2- 172.21.15.190:0/2405884647 >> v2:172.21.15.154:3300/0 conn(0x7f7c481534d0 0x7f7c480c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:59.448 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.446+0000 7f7c467fc640 1 -- 172.21.15.190:0/2405884647 >> v2:172.21.15.154:3300/0 conn(0x7f7c481534d0 msgr2=0x7f7c480c8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:14:59.448 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.446+0000 7f7c467fc640 1 --2- 172.21.15.190:0/2405884647 >> v2:172.21.15.154:3300/0 conn(0x7f7c481534d0 0x7f7c480c8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:59.448 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.446+0000 7f7c467fc640 1 -- 172.21.15.190:0/2405884647 >> v1:172.21.15.190:6789/0 conn(0x7f7c480c8fc0 legacy=0x7f7c4806be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:59.448 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.446+0000 7f7c467fc640 1 -- 172.21.15.190:0/2405884647 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c481594b0 con 0x7f7c48070630 2023-09-23T03:14:59.448 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.447+0000 7f7c467fc640 1 --2- 172.21.15.190:0/2405884647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c48070630 0x7f7c480afeb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7c4806dff0 tx=0x7f7c3002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83338ed1cefecf2e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:14:59.449 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.447+0000 7f7c45ffb640 1 -- 172.21.15.190:0/2405884647 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c30031830 con 0x7f7c48070630 2023-09-23T03:14:59.449 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.447+0000 7f7c45ffb640 1 -- 172.21.15.190:0/2405884647 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c30031990 con 0x7f7c48070630 2023-09-23T03:14:59.449 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.447+0000 7f7c45ffb640 1 -- 172.21.15.190:0/2405884647 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c30031d20 con 0x7f7c48070630 2023-09-23T03:14:59.449 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.448+0000 7f7c47fff640 1 -- 172.21.15.190:0/2405884647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c48070630 msgr2=0x7f7c480afeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:59.449 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.448+0000 7f7c47fff640 1 --2- 172.21.15.190:0/2405884647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c48070630 0x7f7c480afeb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7c4806dff0 tx=0x7f7c3002b8d0 comp rx=0 tx=0).stop 2023-09-23T03:14:59.450 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.448+0000 7f7c47fff640 1 -- 172.21.15.190:0/2405884647 shutdown_connections 2023-09-23T03:14:59.450 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.448+0000 7f7c47fff640 1 --2- 172.21.15.190:0/2405884647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c48070630 0x7f7c480afeb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:59.450 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.448+0000 7f7c47fff640 1 --2- 172.21.15.190:0/2405884647 >> v2:172.21.15.154:3300/0 conn(0x7f7c481534d0 0x7f7c480c8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:59.450 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.448+0000 7f7c47fff640 1 -- 172.21.15.190:0/2405884647 >> 172.21.15.190:0/2405884647 conn(0x7f7c4806a300 msgr2=0x7f7c480caeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:14:59.450 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.448+0000 7f7c47fff640 1 -- 172.21.15.190:0/2405884647 shutdown_connections 2023-09-23T03:14:59.450 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.448+0000 7f7c47fff640 1 -- 172.21.15.190:0/2405884647 wait complete. 2023-09-23T03:14:59.451 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.450+0000 7f7c47fff640 1 Processor -- start 2023-09-23T03:14:59.451 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.450+0000 7f7c47fff640 1 -- start start 2023-09-23T03:14:59.453 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.451+0000 7f7c47fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c48070630 0x7f7c48146950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:59.453 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.451+0000 7f7c47fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c480c8fc0 0x7f7c48152250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:59.453 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.451+0000 7f7c47fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c481534d0 0x7f7c48152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:59.453 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.451+0000 7f7c47fff640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4814a8a0 con 0x7f7c48070630 2023-09-23T03:14:59.453 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.451+0000 7f7c47fff640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c48146e90 con 0x7f7c481534d0 2023-09-23T03:14:59.453 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.451+0000 7f7c47fff640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4815b4b0 con 0x7f7c480c8fc0 2023-09-23T03:14:59.453 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.451+0000 7f7c477fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c481534d0 0x7f7c48152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:59.453 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.451+0000 7f7c477fe640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c481534d0 0x7f7c48152790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.190:33516/0 (socket says 172.21.15.190:33516) 2023-09-23T03:14:59.453 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.451+0000 7f7c467fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c480c8fc0 0x7f7c48152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:59.454 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.451+0000 7f7c477fe640 1 -- 172.21.15.190:0/605685460 learned_addr learned my addr 172.21.15.190:0/605685460 (peer_addr_for_me v2:172.21.15.190:0/0) 2023-09-23T03:14:59.454 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.451+0000 7f7c46ffd640 1 --2- 172.21.15.190:0/605685460 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c48070630 0x7f7c48146950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:59.454 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.452+0000 7f7c477fe640 1 -- 172.21.15.190:0/605685460 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c480c8fc0 msgr2=0x7f7c48152250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:59.454 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.452+0000 7f7c477fe640 1 --2- 172.21.15.190:0/605685460 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c480c8fc0 0x7f7c48152250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:59.454 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.452+0000 7f7c477fe640 1 -- 172.21.15.190:0/605685460 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c48070630 msgr2=0x7f7c48146950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:59.454 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.452+0000 7f7c477fe640 1 --2- 172.21.15.190:0/605685460 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c48070630 0x7f7c48146950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:59.454 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.452+0000 7f7c477fe640 1 -- 172.21.15.190:0/605685460 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c30009400 con 0x7f7c481534d0 2023-09-23T03:14:59.454 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.452+0000 7f7c477fe640 1 --2- 172.21.15.190:0/605685460 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c481534d0 0x7f7c48152790 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7c4000c420 tx=0x7f7c4000c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:14:59.455 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.452+0000 7f7c3ffff640 1 -- 172.21.15.190:0/605685460 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c40005ce0 con 0x7f7c481534d0 2023-09-23T03:14:59.455 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.452+0000 7f7c3ffff640 1 -- 172.21.15.190:0/605685460 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c40005e40 con 0x7f7c481534d0 2023-09-23T03:14:59.455 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.452+0000 7f7c3ffff640 1 -- 172.21.15.190:0/605685460 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c400072d0 con 0x7f7c481534d0 2023-09-23T03:14:59.455 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.452+0000 7f7c47fff640 1 -- 172.21.15.190:0/605685460 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c4815b790 con 0x7f7c481534d0 2023-09-23T03:14:59.455 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.452+0000 7f7c47fff640 1 -- 172.21.15.190:0/605685460 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c4815bd50 con 0x7f7c481534d0 2023-09-23T03:14:59.456 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.453+0000 7f7c3ffff640 1 -- 172.21.15.190:0/605685460 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7c4001c440 con 0x7f7c481534d0 2023-09-23T03:14:59.456 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.453+0000 7f7c3ffff640 1 -- 172.21.15.190:0/605685460 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f7c4001cd50 con 0x7f7c481534d0 2023-09-23T03:14:59.456 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.454+0000 7f7c47fff640 1 -- 172.21.15.190:0/605685460 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c4814a9e0 con 0x7f7c481534d0 2023-09-23T03:14:59.457 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.456+0000 7f7c3ffff640 1 -- 172.21.15.190:0/605685460 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f7c4814a9e0 con 0x7f7c481534d0 2023-09-23T03:14:59.486 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-09-23T03:14:59.485+0000 7f46d5215040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-23T03:14:59.534 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.533+0000 7f7c47fff640 1 -- 172.21.15.190:0/605685460 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_command({"prefix": "osd new", "uuid": "14ad9957-5ab0-43b2-8a1d-be664cbb9756", "id": 7} v 0) v1 -- 0x7f7c48067390 con 0x7f7c481534d0 2023-09-23T03:14:59.537 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-09-23T03:14:59.536+0000 7f46d5215040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-23T03:14:59.538 INFO:tasks.ceph.mgr.y.smithi154.stderr:2023-09-23T03:14:59.538+0000 7f0f6863f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-23T03:14:59.541 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.540+0000 7f7c3ffff640 1 -- 172.21.15.190:0/605685460 <== mon.1 v2:172.21.15.190:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "14ad9957-5ab0-43b2-8a1d-be664cbb9756", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7c40021060 con 0x7f7c481534d0 2023-09-23T03:14:59.541 INFO:teuthology.orchestra.run.smithi190.stdout:7 2023-09-23T03:14:59.543 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.541+0000 7f7c47fff640 1 -- 172.21.15.190:0/605685460 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c481534d0 msgr2=0x7f7c48152790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:59.543 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.541+0000 7f7c47fff640 1 --2- 172.21.15.190:0/605685460 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c481534d0 0x7f7c48152790 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7c4000c420 tx=0x7f7c4000c8e0 comp rx=0 tx=0).stop 2023-09-23T03:14:59.543 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.542+0000 7f7c47fff640 1 -- 172.21.15.190:0/605685460 shutdown_connections 2023-09-23T03:14:59.543 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.542+0000 7f7c47fff640 1 --2- 172.21.15.190:0/605685460 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c481534d0 0x7f7c48152790 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:59.543 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.542+0000 7f7c47fff640 1 --2- 172.21.15.190:0/605685460 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c480c8fc0 0x7f7c48152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:59.543 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.542+0000 7f7c47fff640 1 --2- 172.21.15.190:0/605685460 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c48070630 0x7f7c48146950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:59.543 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.542+0000 7f7c47fff640 1 -- 172.21.15.190:0/605685460 >> 172.21.15.190:0/605685460 conn(0x7f7c4806a300 msgr2=0x7f7c480cce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:14:59.543 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.542+0000 7f7c47fff640 1 -- 172.21.15.190:0/605685460 shutdown_connections 2023-09-23T03:14:59.544 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:14:59.542+0000 7f7c47fff640 1 -- 172.21.15.190:0/605685460 wait complete. 2023-09-23T03:14:59.566 INFO:tasks.ceph.osd.0:Restarting daemon 2023-09-23T03:14:59.566 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-09-23T03:14:59.572 INFO:tasks.ceph.osd.0:Started 2023-09-23T03:14:59.573 INFO:tasks.ceph.osd.1:Restarting daemon 2023-09-23T03:14:59.573 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-09-23T03:14:59.579 INFO:tasks.ceph.osd.1:Started 2023-09-23T03:14:59.579 INFO:tasks.ceph.osd.2:Restarting daemon 2023-09-23T03:14:59.579 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-09-23T03:14:59.584 INFO:tasks.ceph.osd.2:Started 2023-09-23T03:14:59.584 INFO:tasks.ceph.osd.3:Restarting daemon 2023-09-23T03:14:59.584 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-09-23T03:14:59.589 INFO:tasks.ceph.osd.3:Started 2023-09-23T03:14:59.589 INFO:tasks.ceph.osd.4:Restarting daemon 2023-09-23T03:14:59.590 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-09-23T03:14:59.598 INFO:tasks.ceph.osd.4:Started 2023-09-23T03:14:59.598 INFO:tasks.ceph.osd.5:Restarting daemon 2023-09-23T03:14:59.598 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-09-23T03:14:59.606 INFO:tasks.ceph.osd.5:Started 2023-09-23T03:14:59.606 INFO:tasks.ceph.osd.6:Restarting daemon 2023-09-23T03:14:59.606 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-09-23T03:14:59.611 INFO:tasks.ceph.osd.6:Started 2023-09-23T03:14:59.612 INFO:tasks.ceph.osd.7:Restarting daemon 2023-09-23T03:14:59.612 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-09-23T03:14:59.618 INFO:tasks.ceph.osd.7:Started 2023-09-23T03:14:59.618 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:14:59.738 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-09-23T03:14:59.737+0000 7f46d5215040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-23T03:14:59.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.864+0000 7fbb01cd1640 1 Processor -- start 2023-09-23T03:14:59.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.864+0000 7fbb01cd1640 1 -- start start 2023-09-23T03:14:59.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.865+0000 7fbb01cd1640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbafc15cc00 0x7fbafc0b1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:59.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.865+0000 7fbb01cd1640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbafc06be10 con 0x7fbafc0cf500 2023-09-23T03:14:59.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.865+0000 7fbb01cd1640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbafc0d17f0 con 0x7fbafc0cf8e0 2023-09-23T03:14:59.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.865+0000 7fbb01cd1640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbafc0d0470 con 0x7fbafc15cc00 2023-09-23T03:14:59.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.865+0000 7fbb014d0640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbafc15cc00 0x7fbafc0b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:59.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.865+0000 7fbb00ccf640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fbafc0cf500 0x7fbafc0ce050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:45858/0 (socket says 172.21.15.154:45858) 2023-09-23T03:14:59.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.865+0000 7fbb00ccf640 1 -- 172.21.15.154:0/46358067 learned_addr learned my addr 172.21.15.154:0/46358067 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:14:59.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.866+0000 7fbb014d0640 1 -- 172.21.15.154:0/46358067 >> v1:172.21.15.190:6789/0 conn(0x7fbafc0cf8e0 legacy=0x7fbafc15c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:59.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.866+0000 7fbb014d0640 1 -- 172.21.15.154:0/46358067 >> v1:172.21.15.154:6789/0 conn(0x7fbafc0cf500 legacy=0x7fbafc0ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:59.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.866+0000 7fbb014d0640 1 -- 172.21.15.154:0/46358067 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbafc0d0750 con 0x7fbafc15cc00 2023-09-23T03:14:59.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.866+0000 7fbafb7fe640 1 -- 172.21.15.154:0/46358067 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2899561289 0 0) 0x7fbafc06be10 con 0x7fbafc0cf500 2023-09-23T03:14:59.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.867+0000 7fbb014d0640 1 --2- 172.21.15.154:0/46358067 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbafc15cc00 0x7fbafc0b1430 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4009dc0 tx=0x7fbaf402be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e64b3ca3137524 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:14:59.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.867+0000 7fbafb7fe640 1 -- 172.21.15.154:0/46358067 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf402cc40 con 0x7fbafc15cc00 2023-09-23T03:14:59.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.867+0000 7fbafb7fe640 1 -- 172.21.15.154:0/46358067 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf402cda0 con 0x7fbafc15cc00 2023-09-23T03:14:59.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.867+0000 7fbafb7fe640 1 -- 172.21.15.154:0/46358067 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4034660 con 0x7fbafc15cc00 2023-09-23T03:14:59.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.868+0000 7fbb01cd1640 1 -- 172.21.15.154:0/46358067 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbafc15cc00 msgr2=0x7fbafc0b1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:59.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.868+0000 7fbb01cd1640 1 --2- 172.21.15.154:0/46358067 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbafc15cc00 0x7fbafc0b1430 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4009dc0 tx=0x7fbaf402be90 comp rx=0 tx=0).stop 2023-09-23T03:14:59.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.868+0000 7fbb01cd1640 1 -- 172.21.15.154:0/46358067 shutdown_connections 2023-09-23T03:14:59.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.868+0000 7fbb01cd1640 1 --2- 172.21.15.154:0/46358067 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbafc15cc00 0x7fbafc0b1430 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:59.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.868+0000 7fbb01cd1640 1 -- 172.21.15.154:0/46358067 >> 172.21.15.154:0/46358067 conn(0x7fbafc09bbb0 msgr2=0x7fbafc0d28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:14:59.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.868+0000 7fbb01cd1640 1 -- 172.21.15.154:0/46358067 shutdown_connections 2023-09-23T03:14:59.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.868+0000 7fbb01cd1640 1 -- 172.21.15.154:0/46358067 wait complete. 2023-09-23T03:14:59.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.870+0000 7fbb01cd1640 1 Processor -- start 2023-09-23T03:14:59.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.870+0000 7fbb01cd1640 1 -- start start 2023-09-23T03:14:59.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.871+0000 7fbb01cd1640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbafc0cf500 0x7fbafc15a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:59.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.871+0000 7fbb01cd1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbafc0cf8e0 0x7fbafc0b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:59.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.871+0000 7fbb01cd1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbafc15cc00 0x7fbafc15e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:14:59.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.871+0000 7fbb01cd1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc09f510 con 0x7fbafc0cf8e0 2023-09-23T03:14:59.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.871+0000 7fbb01cd1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc09f7f0 con 0x7fbafc15cc00 2023-09-23T03:14:59.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.871+0000 7fbb01cd1640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc09fad0 con 0x7fbafc0cf500 2023-09-23T03:14:59.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.871+0000 7fbafbfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbafc0cf8e0 0x7fbafc0b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:59.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.871+0000 7fbafbfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbafc0cf8e0 0x7fbafc0b4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45540/0 (socket says 172.21.15.154:45540) 2023-09-23T03:14:59.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.871+0000 7fbafbfff640 1 -- 172.21.15.154:0/706532324 learned_addr learned my addr 172.21.15.154:0/706532324 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:14:59.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.871+0000 7fbb00ccf640 1 --2- 172.21.15.154:0/706532324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbafc0cf500 0x7fbafc15a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:59.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.871+0000 7fbb014d0640 1 --2- 172.21.15.154:0/706532324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbafc15cc00 0x7fbafc15e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:14:59.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.871+0000 7fbafbfff640 1 -- 172.21.15.154:0/706532324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbafc0cf500 msgr2=0x7fbafc15a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:59.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.871+0000 7fbafbfff640 1 --2- 172.21.15.154:0/706532324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbafc0cf500 0x7fbafc15a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:59.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.871+0000 7fbafbfff640 1 -- 172.21.15.154:0/706532324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbafc15cc00 msgr2=0x7fbafc15e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:59.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.871+0000 7fbafbfff640 1 --2- 172.21.15.154:0/706532324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbafc15cc00 0x7fbafc15e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:59.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.871+0000 7fbafbfff640 1 -- 172.21.15.154:0/706532324 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf4009a70 con 0x7fbafc0cf8e0 2023-09-23T03:14:59.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.871+0000 7fbb00ccf640 1 --2- 172.21.15.154:0/706532324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbafc0cf500 0x7fbafc15a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:14:59.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.872+0000 7fbafbfff640 1 --2- 172.21.15.154:0/706532324 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbafc0cf8e0 0x7fbafc0b4ac0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbae800c450 tx=0x7fbae800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:14:59.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.872+0000 7fbaf97fa640 1 -- 172.21.15.154:0/706532324 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae80052d0 con 0x7fbafc0cf8e0 2023-09-23T03:14:59.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.872+0000 7fbaf97fa640 1 -- 172.21.15.154:0/706532324 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbae8005430 con 0x7fbafc0cf8e0 2023-09-23T03:14:59.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.872+0000 7fbaf97fa640 1 -- 172.21.15.154:0/706532324 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae8012870 con 0x7fbafc0cf8e0 2023-09-23T03:14:59.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.872+0000 7fbb01cd1640 1 -- 172.21.15.154:0/706532324 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbafc09fdb0 con 0x7fbafc0cf8e0 2023-09-23T03:14:59.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.872+0000 7fbb01cd1640 1 -- 172.21.15.154:0/706532324 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbafc15aaf0 con 0x7fbafc0cf8e0 2023-09-23T03:14:59.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.872+0000 7fbaf97fa640 1 -- 172.21.15.154:0/706532324 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fbae8012cd0 con 0x7fbafc0cf8e0 2023-09-23T03:14:59.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.873+0000 7fbaf97fa640 1 -- 172.21.15.154:0/706532324 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fbae801b400 con 0x7fbafc0cf8e0 2023-09-23T03:14:59.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.873+0000 7fbb01cd1640 1 -- 172.21.15.154:0/706532324 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbafc0d0cf0 con 0x7fbafc0cf8e0 2023-09-23T03:14:59.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.876+0000 7fbaf97fa640 1 -- 172.21.15.154:0/706532324 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fbafc0d0cf0 con 0x7fbafc0cf8e0 2023-09-23T03:14:59.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.938+0000 7fbb01cd1640 1 -- 172.21.15.154:0/706532324 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbafc0a02d0 con 0x7fbafc0cf8e0 2023-09-23T03:14:59.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.939+0000 7fbaf97fa640 1 -- 172.21.15.154:0/706532324 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fbafc0a02d0 con 0x7fbafc0cf8e0 2023-09-23T03:14:59.939 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:14:59.939 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":10,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:14:59.536492+0000","last_up_change":"0.000000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:14:59.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.940+0000 7fbb01cd1640 1 -- 172.21.15.154:0/706532324 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbafc0cf8e0 msgr2=0x7fbafc0b4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:14:59.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.940+0000 7fbb01cd1640 1 --2- 172.21.15.154:0/706532324 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbafc0cf8e0 0x7fbafc0b4ac0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbae800c450 tx=0x7fbae800c910 comp rx=0 tx=0).stop 2023-09-23T03:14:59.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.940+0000 7fbb01cd1640 1 -- 172.21.15.154:0/706532324 shutdown_connections 2023-09-23T03:14:59.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.940+0000 7fbb01cd1640 1 --2- 172.21.15.154:0/706532324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbafc15cc00 0x7fbafc15e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:59.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.940+0000 7fbb01cd1640 1 --2- 172.21.15.154:0/706532324 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbafc0cf8e0 0x7fbafc0b4ac0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:59.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.940+0000 7fbb01cd1640 1 --2- 172.21.15.154:0/706532324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbafc0cf500 0x7fbafc15a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:14:59.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.940+0000 7fbb01cd1640 1 -- 172.21.15.154:0/706532324 >> 172.21.15.154:0/706532324 conn(0x7fbafc09bbb0 msgr2=0x7fbafc0b54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:14:59.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.940+0000 7fbb01cd1640 1 -- 172.21.15.154:0/706532324 shutdown_connections 2023-09-23T03:14:59.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:14:59.940+0000 7fbb01cd1640 1 -- 172.21.15.154:0/706532324 wait complete. 2023-09-23T03:14:59.956 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-09-23T03:14:59.956 INFO:tasks.ceph:Waiting for OSDs to come up 2023-09-23T03:15:00.281 INFO:tasks.ceph.osd.5.smithi190.stderr:2023-09-23T03:15:00.280+0000 7f402d811500 -1 Falling back to public interface 2023-09-23T03:15:00.289 INFO:tasks.ceph.osd.4.smithi190.stderr:2023-09-23T03:15:00.288+0000 7f240c411500 -1 Falling back to public interface 2023-09-23T03:15:00.291 INFO:tasks.ceph.osd.0.smithi154.stderr:2023-09-23T03:15:00.291+0000 7fbd1e9de500 -1 Falling back to public interface 2023-09-23T03:15:00.302 INFO:tasks.ceph.osd.3.smithi154.stderr:2023-09-23T03:15:00.302+0000 7f1401c10500 -1 Falling back to public interface 2023-09-23T03:15:00.318 INFO:tasks.ceph.osd.6.smithi190.stderr:2023-09-23T03:15:00.317+0000 7f17e0c0e500 -1 Falling back to public interface 2023-09-23T03:15:00.319 INFO:tasks.ceph.osd.1.smithi154.stderr:2023-09-23T03:15:00.319+0000 7fad6f7e2500 -1 Falling back to public interface 2023-09-23T03:15:00.325 INFO:tasks.ceph.osd.2.smithi154.stderr:2023-09-23T03:15:00.325+0000 7f5967bad500 -1 Falling back to public interface 2023-09-23T03:15:00.329 INFO:tasks.ceph.osd.2.smithi154.stderr:2023-09-23T03:15:00.329+0000 7f5966b56640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6824/0: (98) Address already in use 2023-09-23T03:15:00.340 INFO:tasks.ceph.osd.7.smithi190.stderr:2023-09-23T03:15:00.339+0000 7fc304049500 -1 Falling back to public interface 2023-09-23T03:15:00.646 INFO:tasks.ceph.mgr.y.smithi154.stderr:2023-09-23T03:15:00.646+0000 7f0f6863f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-23T03:15:00.761 DEBUG:teuthology.orchestra.run.smithi154:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-09-23T03:15:00.805 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-09-23T03:15:00.803+0000 7f46d5215040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-23T03:15:00.920 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.920+0000 7f202deec640 1 Processor -- start 2023-09-23T03:15:00.921 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.920+0000 7f202deec640 1 -- start start 2023-09-23T03:15:00.923 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.922+0000 7f202deec640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f202815cee0 0x7f20280cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:00.923 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.922+0000 7f202deec640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f202815d2c0 0x7f20281615b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:00.923 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.922+0000 7f202deec640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2028161af0 con 0x7f20280cb140 2023-09-23T03:15:00.923 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.922+0000 7f202deec640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f202806b8b0 con 0x7f202815d2c0 2023-09-23T03:15:00.923 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.922+0000 7f202deec640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20280cf300 con 0x7f202815cee0 2023-09-23T03:15:00.923 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.922+0000 7f201ffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f202815d2c0 0x7f20281615b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:00.923 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.922+0000 7f201ffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f202815d2c0 0x7f20281615b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45546/0 (socket says 172.21.15.154:45546) 2023-09-23T03:15:00.923 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.922+0000 7f201ffff640 1 -- 172.21.15.154:0/1386591962 learned_addr learned my addr 172.21.15.154:0/1386591962 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:00.923 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.922+0000 7f20277fe640 1 --2- 172.21.15.154:0/1386591962 >> v2:172.21.15.190:3300/0 conn(0x7f202815cee0 0x7f20280cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:00.923 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.922+0000 7f201ffff640 1 -- 172.21.15.154:0/1386591962 >> v2:172.21.15.190:3300/0 conn(0x7f202815cee0 msgr2=0x7f20280cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:00.923 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.922+0000 7f201ffff640 1 --2- 172.21.15.154:0/1386591962 >> v2:172.21.15.190:3300/0 conn(0x7f202815cee0 0x7f20280cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:00.924 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.922+0000 7f201ffff640 1 -- 172.21.15.154:0/1386591962 >> v1:172.21.15.190:6789/0 conn(0x7f20280cb140 legacy=0x7f202806bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:00.924 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.923+0000 7f201ffff640 1 -- 172.21.15.154:0/1386591962 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202806b1f0 con 0x7f202815d2c0 2023-09-23T03:15:00.924 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.924+0000 7f201ffff640 1 --2- 172.21.15.154:0/1386591962 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f202815d2c0 0x7f20281615b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f20280677d0 tx=0x7f201002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56dcdc7620a97bce server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:00.924 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.924+0000 7f2026ffd640 1 -- 172.21.15.154:0/1386591962 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2010031870 con 0x7f202815d2c0 2023-09-23T03:15:00.924 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.924+0000 7f2026ffd640 1 -- 172.21.15.154:0/1386591962 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20100319d0 con 0x7f202815d2c0 2023-09-23T03:15:00.924 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.924+0000 7f2026ffd640 1 -- 172.21.15.154:0/1386591962 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2010031d60 con 0x7f202815d2c0 2023-09-23T03:15:00.925 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.925+0000 7f202deec640 1 -- 172.21.15.154:0/1386591962 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f202815d2c0 msgr2=0x7f20281615b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:00.925 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.925+0000 7f202deec640 1 --2- 172.21.15.154:0/1386591962 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f202815d2c0 0x7f20281615b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f20280677d0 tx=0x7f201002b910 comp rx=0 tx=0).stop 2023-09-23T03:15:00.926 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.925+0000 7f202deec640 1 -- 172.21.15.154:0/1386591962 shutdown_connections 2023-09-23T03:15:00.926 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.925+0000 7f202deec640 1 --2- 172.21.15.154:0/1386591962 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f202815d2c0 0x7f20281615b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:00.926 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.925+0000 7f202deec640 1 --2- 172.21.15.154:0/1386591962 >> v2:172.21.15.190:3300/0 conn(0x7f202815cee0 0x7f20280cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:00.926 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.925+0000 7f202deec640 1 -- 172.21.15.154:0/1386591962 >> 172.21.15.154:0/1386591962 conn(0x7f20280cd1e0 msgr2=0x7f20280cd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:00.926 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.925+0000 7f202deec640 1 -- 172.21.15.154:0/1386591962 shutdown_connections 2023-09-23T03:15:00.926 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.925+0000 7f202deec640 1 -- 172.21.15.154:0/1386591962 wait complete. 2023-09-23T03:15:00.926 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.926+0000 7f202deec640 1 Processor -- start 2023-09-23T03:15:00.927 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.926+0000 7f202deec640 1 -- start start 2023-09-23T03:15:00.928 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.927+0000 7f202deec640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20280cb140 0x7f20280c8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:00.928 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.927+0000 7f202deec640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f202815cee0 0x7f20280c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:00.928 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.927+0000 7f202deec640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f202815d2c0 0x7f20280c6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:00.928 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.927+0000 7f202deec640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20280c9f20 con 0x7f202815d2c0 2023-09-23T03:15:00.928 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.927+0000 7f202deec640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20280c90b0 con 0x7f202815cee0 2023-09-23T03:15:00.928 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.927+0000 7f202deec640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20280c9310 con 0x7f20280cb140 2023-09-23T03:15:00.928 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.928+0000 7f2027fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f202815d2c0 0x7f20280c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:00.928 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.928+0000 7f2027fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f202815d2c0 0x7f20280c6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45550/0 (socket says 172.21.15.154:45550) 2023-09-23T03:15:00.928 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.928+0000 7f2027fff640 1 -- 172.21.15.154:0/4103446154 learned_addr learned my addr 172.21.15.154:0/4103446154 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:00.928 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.928+0000 7f201ffff640 1 --2- 172.21.15.154:0/4103446154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f202815cee0 0x7f20280c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:00.929 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.928+0000 7f20277fe640 1 --2- 172.21.15.154:0/4103446154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20280cb140 0x7f20280c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:00.930 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.928+0000 7f2027fff640 1 -- 172.21.15.154:0/4103446154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20280cb140 msgr2=0x7f20280c8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:00.930 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.928+0000 7f2027fff640 1 --2- 172.21.15.154:0/4103446154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20280cb140 0x7f20280c8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:00.930 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.928+0000 7f2027fff640 1 -- 172.21.15.154:0/4103446154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f202815cee0 msgr2=0x7f20280c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:00.930 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.928+0000 7f2027fff640 1 --2- 172.21.15.154:0/4103446154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f202815cee0 0x7f20280c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:00.930 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.928+0000 7f2027fff640 1 -- 172.21.15.154:0/4103446154 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20100094d0 con 0x7f202815d2c0 2023-09-23T03:15:00.930 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.928+0000 7f2027fff640 1 --2- 172.21.15.154:0/4103446154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f202815d2c0 0x7f20280c6840 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f202000c7a0 tx=0x7f202000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:00.930 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.929+0000 7f2024ff9640 1 -- 172.21.15.154:0/4103446154 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2020003b70 con 0x7f202815d2c0 2023-09-23T03:15:00.931 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.929+0000 7f2024ff9640 1 -- 172.21.15.154:0/4103446154 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2020003cf0 con 0x7f202815d2c0 2023-09-23T03:15:00.931 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.929+0000 7f2024ff9640 1 -- 172.21.15.154:0/4103446154 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2020003ea0 con 0x7f202815d2c0 2023-09-23T03:15:00.931 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.929+0000 7f202deec640 1 -- 172.21.15.154:0/4103446154 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20280c95f0 con 0x7f202815d2c0 2023-09-23T03:15:00.931 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.929+0000 7f202deec640 1 -- 172.21.15.154:0/4103446154 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20280c9bb0 con 0x7f202815d2c0 2023-09-23T03:15:00.931 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.929+0000 7f2024ff9640 1 -- 172.21.15.154:0/4103446154 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f20200050d0 con 0x7f202815d2c0 2023-09-23T03:15:00.931 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.929+0000 7f2024ff9640 1 -- 172.21.15.154:0/4103446154 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f2020012b40 con 0x7f202815d2c0 2023-09-23T03:15:00.931 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.930+0000 7f202deec640 1 -- 172.21.15.154:0/4103446154 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20280c4270 con 0x7f202815d2c0 2023-09-23T03:15:00.932 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.932+0000 7f2024ff9640 1 -- 172.21.15.154:0/4103446154 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f20280c4270 con 0x7f202815d2c0 2023-09-23T03:15:00.960 INFO:tasks.ceph.mgr.y.smithi154.stderr:2023-09-23T03:15:00.960+0000 7f0f6863f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-23T03:15:00.996 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.995+0000 7f202deec640 1 -- 172.21.15.154:0/4103446154 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2028073eb0 con 0x7f202815d2c0 2023-09-23T03:15:00.996 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.996+0000 7f2024ff9640 1 -- 172.21.15.154:0/4103446154 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f2028161af0 con 0x7f202815d2c0 2023-09-23T03:15:00.997 INFO:teuthology.misc.health.smithi154.stdout: 2023-09-23T03:15:00.997 INFO:teuthology.misc.health.smithi154.stdout:{"epoch":10,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:14:59.536492+0000","last_up_change":"0.000000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:15:00.998 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.997+0000 7f202deec640 1 -- 172.21.15.154:0/4103446154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f202815d2c0 msgr2=0x7f20280c6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:00.998 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.998+0000 7f202deec640 1 --2- 172.21.15.154:0/4103446154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f202815d2c0 0x7f20280c6840 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f202000c7a0 tx=0x7f202000cc60 comp rx=0 tx=0).stop 2023-09-23T03:15:00.999 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.998+0000 7f202deec640 1 -- 172.21.15.154:0/4103446154 shutdown_connections 2023-09-23T03:15:00.999 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.998+0000 7f202deec640 1 --2- 172.21.15.154:0/4103446154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f202815d2c0 0x7f20280c6840 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:00.999 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.998+0000 7f202deec640 1 --2- 172.21.15.154:0/4103446154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f202815cee0 0x7f20280c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:00.999 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.998+0000 7f202deec640 1 --2- 172.21.15.154:0/4103446154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20280cb140 0x7f20280c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:00.999 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.998+0000 7f202deec640 1 -- 172.21.15.154:0/4103446154 >> 172.21.15.154:0/4103446154 conn(0x7f20280cd1e0 msgr2=0x7f202806fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:00.999 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.998+0000 7f202deec640 1 -- 172.21.15.154:0/4103446154 shutdown_connections 2023-09-23T03:15:00.999 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:00.998+0000 7f202deec640 1 -- 172.21.15.154:0/4103446154 wait complete. 2023-09-23T03:15:01.010 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-09-23T03:15:01.024 INFO:tasks.ceph.mgr.y.smithi154.stderr:2023-09-23T03:15:01.024+0000 7f0f6863f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-23T03:15:01.114 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-09-23T03:15:01.113+0000 7f46d5215040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-23T03:15:01.139 INFO:tasks.ceph.mgr.y.smithi154.stderr:2023-09-23T03:15:01.139+0000 7f0f6863f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-23T03:15:01.179 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-09-23T03:15:01.178+0000 7f46d5215040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-23T03:15:01.294 INFO:tasks.ceph.mgr.x.smithi190.stderr:2023-09-23T03:15:01.293+0000 7f46d5215040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-23T03:15:02.180 INFO:tasks.ceph.mgr.y.smithi154.stderr:2023-09-23T03:15:02.180+0000 7f0f09a42640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-09-23T03:15:02.182 INFO:tasks.ceph.mgr.y.smithi154.stderr:2023-09-23T03:15:02.182+0000 7f0f09a42640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-09-23T03:15:03.620 INFO:tasks.ceph.osd.7.smithi190.stderr:2023-09-23T03:15:03.619+0000 7fc304049500 -1 osd.7 0 log_to_monitors true 2023-09-23T03:15:03.650 INFO:tasks.ceph.osd.1.smithi154.stderr:2023-09-23T03:15:03.650+0000 7fad6f7e2500 -1 osd.1 0 log_to_monitors true 2023-09-23T03:15:03.818 INFO:tasks.ceph.osd.5.smithi190.stderr:2023-09-23T03:15:03.817+0000 7f402d811500 -1 osd.5 0 log_to_monitors true 2023-09-23T03:15:03.848 INFO:tasks.ceph.osd.4.smithi190.stderr:2023-09-23T03:15:03.846+0000 7f240c411500 -1 osd.4 0 log_to_monitors true 2023-09-23T03:15:03.869 INFO:tasks.ceph.osd.6.smithi190.stderr:2023-09-23T03:15:03.868+0000 7f17e0c0e500 -1 osd.6 0 log_to_monitors true 2023-09-23T03:15:03.870 INFO:tasks.ceph.osd.3.smithi154.stderr:2023-09-23T03:15:03.870+0000 7f1401c10500 -1 osd.3 0 log_to_monitors true 2023-09-23T03:15:03.903 INFO:tasks.ceph.osd.2.smithi154.stderr:2023-09-23T03:15:03.903+0000 7f5967bad500 -1 osd.2 0 log_to_monitors true 2023-09-23T03:15:04.080 INFO:tasks.ceph.osd.0.smithi154.stderr:2023-09-23T03:15:04.079+0000 7fbd1e9de500 -1 osd.0 0 log_to_monitors true 2023-09-23T03:15:07.443 INFO:tasks.ceph.osd.1.smithi154.stderr:2023-09-23T03:15:07.443+0000 7fad6b770640 -1 osd.1 0 waiting for initial osdmap 2023-09-23T03:15:07.467 INFO:tasks.ceph.osd.1.smithi154.stderr:2023-09-23T03:15:07.467+0000 7fad63590640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-23T03:15:07.568 INFO:tasks.ceph.osd.7.smithi190.stderr:2023-09-23T03:15:07.567+0000 7fc3007eb640 -1 osd.7 0 waiting for initial osdmap 2023-09-23T03:15:07.593 INFO:tasks.ceph.osd.7.smithi190.stderr:2023-09-23T03:15:07.591+0000 7fc2f7df5640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-23T03:15:07.593 INFO:tasks.ceph.osd.5.smithi190.stderr:2023-09-23T03:15:07.592+0000 7f40299f2640 -1 osd.5 0 waiting for initial osdmap 2023-09-23T03:15:07.613 INFO:tasks.ceph.osd.5.smithi190.stderr:2023-09-23T03:15:07.612+0000 7f4020ffc640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-23T03:15:07.644 INFO:tasks.ceph.osd.4.smithi190.stderr:2023-09-23T03:15:07.642+0000 7f24084d4640 -1 osd.4 0 waiting for initial osdmap 2023-09-23T03:15:07.656 INFO:tasks.ceph.osd.4.smithi190.stderr:2023-09-23T03:15:07.655+0000 7f23ffade640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-23T03:15:07.660 INFO:tasks.ceph.osd.6.smithi190.stderr:2023-09-23T03:15:07.659+0000 7f17dc532640 -1 osd.6 0 waiting for initial osdmap 2023-09-23T03:15:07.684 INFO:tasks.ceph.osd.6.smithi190.stderr:2023-09-23T03:15:07.683+0000 7f17d4352640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-23T03:15:07.727 INFO:tasks.ceph.osd.2.smithi154.stderr:2023-09-23T03:15:07.725+0000 7f5963b3b640 -1 osd.2 0 waiting for initial osdmap 2023-09-23T03:15:07.745 INFO:tasks.ceph.osd.2.smithi154.stderr:2023-09-23T03:15:07.745+0000 7f595b95b640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-23T03:15:07.816 DEBUG:teuthology.orchestra.run.smithi154:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-09-23T03:15:07.860 INFO:tasks.ceph.osd.0.smithi154.stderr:2023-09-23T03:15:07.860+0000 7fbd1a96a640 -1 osd.0 0 waiting for initial osdmap 2023-09-23T03:15:07.877 INFO:tasks.ceph.osd.0.smithi154.stderr:2023-09-23T03:15:07.877+0000 7fbd1278a640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-23T03:15:07.886 INFO:tasks.ceph.osd.3.smithi154.stderr:2023-09-23T03:15:07.886+0000 7f13fd540640 -1 osd.3 0 waiting for initial osdmap 2023-09-23T03:15:07.903 INFO:tasks.ceph.osd.3.smithi154.stderr:2023-09-23T03:15:07.903+0000 7f13f5360640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-23T03:15:07.969 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.969+0000 7fca62d75640 1 Processor -- start 2023-09-23T03:15:07.969 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.969+0000 7fca62d75640 1 -- start start 2023-09-23T03:15:07.972 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.971+0000 7fca62d75640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fca5c1597d0 0x7fca5c0cb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:07.972 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.971+0000 7fca62d75640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fca5c0714a0 0x7fca5c071900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:07.972 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.971+0000 7fca62d75640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca5c071e40 con 0x7fca5c15d320 2023-09-23T03:15:07.972 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.971+0000 7fca62d75640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca5c15e050 con 0x7fca5c0714a0 2023-09-23T03:15:07.972 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.971+0000 7fca62d75640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca5c0cad80 con 0x7fca5c1597d0 2023-09-23T03:15:07.972 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.971+0000 7fca61572640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fca5c0714a0 0x7fca5c071900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:07.972 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.971+0000 7fca61572640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fca5c0714a0 0x7fca5c071900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:34684/0 (socket says 172.21.15.154:34684) 2023-09-23T03:15:07.972 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.971+0000 7fca61572640 1 -- 172.21.15.154:0/375011243 learned_addr learned my addr 172.21.15.154:0/375011243 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:07.972 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.972+0000 7fca61572640 1 -- 172.21.15.154:0/375011243 >> v2:172.21.15.190:3300/0 conn(0x7fca5c1597d0 msgr2=0x7fca5c0cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:07.973 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.972+0000 7fca61572640 1 --2- 172.21.15.154:0/375011243 >> v2:172.21.15.190:3300/0 conn(0x7fca5c1597d0 0x7fca5c0cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:07.973 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.972+0000 7fca61572640 1 -- 172.21.15.154:0/375011243 >> v1:172.21.15.190:6789/0 conn(0x7fca5c15d320 legacy=0x7fca5c15d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:07.973 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.972+0000 7fca61572640 1 -- 172.21.15.154:0/375011243 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca5c0cb060 con 0x7fca5c0714a0 2023-09-23T03:15:07.974 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.973+0000 7fca61572640 1 --2- 172.21.15.154:0/375011243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca5c0714a0 0x7fca5c071900 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fca5c0734e0 tx=0x7fca4c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b045555a6ed213b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:07.974 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.973+0000 7fca60d71640 1 -- 172.21.15.154:0/375011243 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c0317e0 con 0x7fca5c0714a0 2023-09-23T03:15:07.974 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.973+0000 7fca60d71640 1 -- 172.21.15.154:0/375011243 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca4c031940 con 0x7fca5c0714a0 2023-09-23T03:15:07.974 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.973+0000 7fca60d71640 1 -- 172.21.15.154:0/375011243 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c031cd0 con 0x7fca5c0714a0 2023-09-23T03:15:07.974 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.974+0000 7fca62d75640 1 -- 172.21.15.154:0/375011243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca5c0714a0 msgr2=0x7fca5c071900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:07.974 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.974+0000 7fca62d75640 1 --2- 172.21.15.154:0/375011243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca5c0714a0 0x7fca5c071900 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fca5c0734e0 tx=0x7fca4c02b880 comp rx=0 tx=0).stop 2023-09-23T03:15:07.975 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.974+0000 7fca62d75640 1 -- 172.21.15.154:0/375011243 shutdown_connections 2023-09-23T03:15:07.975 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.974+0000 7fca62d75640 1 --2- 172.21.15.154:0/375011243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca5c0714a0 0x7fca5c071900 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:07.975 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.974+0000 7fca62d75640 1 --2- 172.21.15.154:0/375011243 >> v2:172.21.15.190:3300/0 conn(0x7fca5c1597d0 0x7fca5c0cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:07.975 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.974+0000 7fca62d75640 1 -- 172.21.15.154:0/375011243 >> 172.21.15.154:0/375011243 conn(0x7fca5c0cf440 msgr2=0x7fca5c0cf850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:07.975 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.974+0000 7fca62d75640 1 -- 172.21.15.154:0/375011243 shutdown_connections 2023-09-23T03:15:07.975 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.974+0000 7fca62d75640 1 -- 172.21.15.154:0/375011243 wait complete. 2023-09-23T03:15:07.975 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.975+0000 7fca62d75640 1 Processor -- start 2023-09-23T03:15:07.976 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.976+0000 7fca62d75640 1 -- start start 2023-09-23T03:15:07.978 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.976+0000 7fca62d75640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca5c0714a0 0x7fca5c0aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:07.978 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.976+0000 7fca62d75640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca5c1597d0 0x7fca5c0b1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:07.978 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.976+0000 7fca62d75640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca5c15d320 0x7fca5c0c9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:07.978 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.976+0000 7fca62d75640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c0b1790 con 0x7fca5c15d320 2023-09-23T03:15:07.978 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.976+0000 7fca62d75640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c0af170 con 0x7fca5c1597d0 2023-09-23T03:15:07.978 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.976+0000 7fca62d75640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca5c0af450 con 0x7fca5c0714a0 2023-09-23T03:15:07.978 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.977+0000 7fca62574640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca5c15d320 0x7fca5c0c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:07.979 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.977+0000 7fca62574640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca5c15d320 0x7fca5c0c9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:34686/0 (socket says 172.21.15.154:34686) 2023-09-23T03:15:07.979 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.977+0000 7fca62574640 1 -- 172.21.15.154:0/3254082818 learned_addr learned my addr 172.21.15.154:0/3254082818 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:07.979 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.977+0000 7fca61572640 1 --2- 172.21.15.154:0/3254082818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca5c1597d0 0x7fca5c0b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:07.979 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.977+0000 7fca61d73640 1 --2- 172.21.15.154:0/3254082818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca5c0714a0 0x7fca5c0aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:07.979 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.977+0000 7fca62574640 1 -- 172.21.15.154:0/3254082818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca5c0714a0 msgr2=0x7fca5c0aeb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:07.979 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.977+0000 7fca62574640 1 --2- 172.21.15.154:0/3254082818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca5c0714a0 0x7fca5c0aeb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:07.979 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.977+0000 7fca62574640 1 -- 172.21.15.154:0/3254082818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca5c1597d0 msgr2=0x7fca5c0b1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:07.979 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.977+0000 7fca62574640 1 --2- 172.21.15.154:0/3254082818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca5c1597d0 0x7fca5c0b1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:07.979 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.977+0000 7fca62574640 1 -- 172.21.15.154:0/3254082818 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca4c0094d0 con 0x7fca5c15d320 2023-09-23T03:15:07.980 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.977+0000 7fca61572640 1 --2- 172.21.15.154:0/3254082818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca5c1597d0 0x7fca5c0b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:07.980 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.977+0000 7fca62574640 1 --2- 172.21.15.154:0/3254082818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca5c15d320 0x7fca5c0c9280 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fca5400b990 tx=0x7fca5400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:07.980 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.978+0000 7fca527fc640 1 -- 172.21.15.154:0/3254082818 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5400cba0 con 0x7fca5c15d320 2023-09-23T03:15:07.980 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.978+0000 7fca527fc640 1 -- 172.21.15.154:0/3254082818 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca5400cd00 con 0x7fca5c15d320 2023-09-23T03:15:07.980 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.978+0000 7fca527fc640 1 -- 172.21.15.154:0/3254082818 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca54014680 con 0x7fca5c15d320 2023-09-23T03:15:07.980 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.978+0000 7fca62d75640 1 -- 172.21.15.154:0/3254082818 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca5c0af730 con 0x7fca5c15d320 2023-09-23T03:15:07.980 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.978+0000 7fca62d75640 1 -- 172.21.15.154:0/3254082818 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca5c0afcf0 con 0x7fca5c15d320 2023-09-23T03:15:07.981 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.979+0000 7fca62d75640 1 -- 172.21.15.154:0/3254082818 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca5c073eb0 con 0x7fca5c15d320 2023-09-23T03:15:07.981 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.980+0000 7fca527fc640 1 -- 172.21.15.154:0/3254082818 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 4) v1 ==== 89299+0+0 (secure 0 0 0) 0x7fca540147e0 con 0x7fca5c15d320 2023-09-23T03:15:07.981 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.980+0000 7fca527fc640 1 --2- 172.21.15.154:0/3254082818 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fca4006aea0 0x7fca4006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:07.981 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.981+0000 7fca527fc640 1 -- 172.21.15.154:0/3254082818 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7fca54088820 con 0x7fca5c15d320 2023-09-23T03:15:07.981 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.981+0000 7fca61d73640 1 --2- 172.21.15.154:0/3254082818 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fca4006aea0 0x7fca4006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:07.981 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.981+0000 7fca61d73640 1 --2- 172.21.15.154:0/3254082818 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fca4006aea0 0x7fca4006d360 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fca5800a810 tx=0x7fca58008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:07.986 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:07.986+0000 7fca527fc640 1 -- 172.21.15.154:0/3254082818 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fca540546c0 con 0x7fca5c15d320 2023-09-23T03:15:08.163 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:08.163+0000 7fca62d75640 1 -- 172.21.15.154:0/3254082818 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fca5c06f990 con 0x7fca5c15d320 2023-09-23T03:15:08.164 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:08.164+0000 7fca527fc640 1 -- 172.21.15.154:0/3254082818 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fca54019070 con 0x7fca5c15d320 2023-09-23T03:15:08.164 INFO:teuthology.misc.health.smithi154.stdout: 2023-09-23T03:15:08.164 INFO:teuthology.misc.health.smithi154.stdout:{"epoch":12,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:15:05.192895+0000","last_up_change":"0.000000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:15:08.167 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:08.166+0000 7fca62d75640 1 -- 172.21.15.154:0/3254082818 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fca4006aea0 msgr2=0x7fca4006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:08.167 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:08.166+0000 7fca62d75640 1 --2- 172.21.15.154:0/3254082818 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fca4006aea0 0x7fca4006d360 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fca5800a810 tx=0x7fca58008040 comp rx=0 tx=0).stop 2023-09-23T03:15:08.167 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:08.166+0000 7fca62d75640 1 -- 172.21.15.154:0/3254082818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca5c15d320 msgr2=0x7fca5c0c9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:08.167 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:08.166+0000 7fca62d75640 1 --2- 172.21.15.154:0/3254082818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca5c15d320 0x7fca5c0c9280 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fca5400b990 tx=0x7fca5400be50 comp rx=0 tx=0).stop 2023-09-23T03:15:08.167 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:08.166+0000 7fca62d75640 1 -- 172.21.15.154:0/3254082818 shutdown_connections 2023-09-23T03:15:08.167 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:08.166+0000 7fca62d75640 1 --2- 172.21.15.154:0/3254082818 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fca4006aea0 0x7fca4006d360 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:08.167 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:08.166+0000 7fca62d75640 1 --2- 172.21.15.154:0/3254082818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca5c15d320 0x7fca5c0c9280 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:08.167 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:08.166+0000 7fca62d75640 1 --2- 172.21.15.154:0/3254082818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca5c1597d0 0x7fca5c0b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:08.167 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:08.166+0000 7fca62d75640 1 --2- 172.21.15.154:0/3254082818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca5c0714a0 0x7fca5c0aeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:08.167 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:08.166+0000 7fca62d75640 1 -- 172.21.15.154:0/3254082818 >> 172.21.15.154:0/3254082818 conn(0x7fca5c0cf440 msgr2=0x7fca5c0726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:08.167 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:08.166+0000 7fca62d75640 1 -- 172.21.15.154:0/3254082818 shutdown_connections 2023-09-23T03:15:08.168 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:08.167+0000 7fca62d75640 1 -- 172.21.15.154:0/3254082818 wait complete. 2023-09-23T03:15:08.180 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-09-23T03:15:10.482 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:15:10.492 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:15:14.987 DEBUG:teuthology.orchestra.run.smithi154:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-09-23T03:15:15.176 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.174+0000 7f9920897640 1 Processor -- start 2023-09-23T03:15:15.176 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.174+0000 7f9920897640 1 -- start start 2023-09-23T03:15:15.177 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.176+0000 7f9920897640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f991c0706d0 0x7f991c06a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:15.177 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.176+0000 7f9920897640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f991c06a5d0 con 0x7f991c0cdee0 2023-09-23T03:15:15.177 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.176+0000 7f9920897640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f991c0b14e0 con 0x7f991c069c50 2023-09-23T03:15:15.178 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.176+0000 7f9920897640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991c0cade0 con 0x7f991c0706d0 2023-09-23T03:15:15.178 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.176+0000 7f991a59c640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f991c0cdee0 0x7f991c0b22a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:59048/0 (socket says 172.21.15.154:59048) 2023-09-23T03:15:15.178 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.176+0000 7f991a59c640 1 -- 172.21.15.154:0/3479761053 learned_addr learned my addr 172.21.15.154:0/3479761053 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:15.181 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.176+0000 7f991ad9d640 1 --2- 172.21.15.154:0/3479761053 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f991c0706d0 0x7f991c06a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:15.181 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.177+0000 7f991959a640 1 -- 172.21.15.154:0/3479761053 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 594977579 0 0) 0x7f991c06a5d0 con 0x7f991c0cdee0 2023-09-23T03:15:15.181 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.177+0000 7f991959a640 1 -- 172.21.15.154:0/3479761053 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9904003520 con 0x7f991c0cdee0 2023-09-23T03:15:15.181 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.177+0000 7f991959a640 1 -- 172.21.15.154:0/3479761053 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 867861406 0 0) 0x7f9904003520 con 0x7f991c0cdee0 2023-09-23T03:15:15.181 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.177+0000 7f991959a640 1 -- 172.21.15.154:0/3479761053 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f991c0706d0 msgr2=0x7f991c06a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:15.182 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.177+0000 7f991959a640 1 --2- 172.21.15.154:0/3479761053 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f991c0706d0 0x7f991c06a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:15.182 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.177+0000 7f991959a640 1 -- 172.21.15.154:0/3479761053 >> v1:172.21.15.190:6789/0 conn(0x7f991c069c50 legacy=0x7f991c06ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:15.182 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.177+0000 7f991959a640 1 -- 172.21.15.154:0/3479761053 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991c0b17c0 con 0x7f991c0cdee0 2023-09-23T03:15:15.182 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.177+0000 7f991959a640 1 -- 172.21.15.154:0/3479761053 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f990c001670 con 0x7f991c0cdee0 2023-09-23T03:15:15.182 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.177+0000 7f991959a640 1 -- 172.21.15.154:0/3479761053 >> v1:172.21.15.154:6789/0 conn(0x7f991c0cdee0 legacy=0x7f991c0b22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:15.182 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.177+0000 7f991959a640 1 --2- 172.21.15.154:0/3479761053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99040039a0 0x7f9904003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:15.182 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.177+0000 7f991959a640 1 --2- 172.21.15.154:0/3479761053 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f991c0706d0 0x7f99040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:15.182 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.177+0000 7f991959a640 1 --2- 172.21.15.154:0/3479761053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9904004d00 0x7f9904005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:15.182 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.177+0000 7f991959a640 1 -- 172.21.15.154:0/3479761053 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c0cade0 con 0x7f9904004d00 2023-09-23T03:15:15.183 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.177+0000 7f991959a640 1 -- 172.21.15.154:0/3479761053 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990400ba60 con 0x7f99040039a0 2023-09-23T03:15:15.183 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.177+0000 7f991959a640 1 -- 172.21.15.154:0/3479761053 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f990400bc20 con 0x7f991c0706d0 2023-09-23T03:15:15.183 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.177+0000 7f991ad9d640 1 --2- 172.21.15.154:0/3479761053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9904004d00 0x7f9904005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:15.183 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.178+0000 7f991a59c640 1 --2- 172.21.15.154:0/3479761053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99040039a0 0x7f9904003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:15.183 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.178+0000 7f991ad9d640 1 -- 172.21.15.154:0/3479761053 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f991c0706d0 msgr2=0x7f99040047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:15.183 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.178+0000 7f991ad9d640 1 --2- 172.21.15.154:0/3479761053 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f991c0706d0 0x7f99040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:15.183 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.178+0000 7f991ad9d640 1 -- 172.21.15.154:0/3479761053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99040039a0 msgr2=0x7f9904003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:15.183 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.178+0000 7f991ad9d640 1 --2- 172.21.15.154:0/3479761053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99040039a0 0x7f9904003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:15.183 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.178+0000 7f991ad9d640 1 -- 172.21.15.154:0/3479761053 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9904003520 con 0x7f9904004d00 2023-09-23T03:15:15.183 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.179+0000 7f991ad9d640 1 --2- 172.21.15.154:0/3479761053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9904004d00 0x7f9904005180 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9914009f90 tx=0x7f99040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f9de0e34db9cb1f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:15.183 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.179+0000 7f991959a640 1 -- 172.21.15.154:0/3479761053 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991402cc20 con 0x7f9904004d00 2023-09-23T03:15:15.184 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.179+0000 7f991959a640 1 -- 172.21.15.154:0/3479761053 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991400f040 con 0x7f9904004d00 2023-09-23T03:15:15.184 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.179+0000 7f991959a640 1 -- 172.21.15.154:0/3479761053 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9914034440 con 0x7f9904004d00 2023-09-23T03:15:15.184 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.180+0000 7f9920897640 1 -- 172.21.15.154:0/3479761053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9904004d00 msgr2=0x7f9904005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:15.184 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.180+0000 7f9920897640 1 --2- 172.21.15.154:0/3479761053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9904004d00 0x7f9904005180 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9914009f90 tx=0x7f99040057f0 comp rx=0 tx=0).stop 2023-09-23T03:15:15.184 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.180+0000 7f991ad9d640 1 -- 172.21.15.154:0/3479761053 reap_dead start 2023-09-23T03:15:15.184 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.180+0000 7f9920897640 1 -- 172.21.15.154:0/3479761053 shutdown_connections 2023-09-23T03:15:15.184 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.180+0000 7f9920897640 1 -- 172.21.15.154:0/3479761053 >> 172.21.15.154:0/3479761053 conn(0x7f991c0ceb50 msgr2=0x7f991c0affe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:15.184 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.180+0000 7f9920897640 1 -- 172.21.15.154:0/3479761053 shutdown_connections 2023-09-23T03:15:15.184 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.180+0000 7f9920897640 1 -- 172.21.15.154:0/3479761053 wait complete. 2023-09-23T03:15:15.186 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.182+0000 7f9920897640 1 Processor -- start 2023-09-23T03:15:15.187 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.182+0000 7f9920897640 1 -- start start 2023-09-23T03:15:15.187 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.183+0000 7f9920897640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9904004d00 0x7f991c157a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:15.187 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.183+0000 7f9920897640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f991c0c3ea0 0x7f991c0ac7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:15.187 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.183+0000 7f9920897640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f991c1f1ff0 0x7f991c1f2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:15.187 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.183+0000 7f9920897640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c0b29b0 con 0x7f991c1f1ff0 2023-09-23T03:15:15.187 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.183+0000 7f9920897640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c14a7f0 con 0x7f991c0c3ea0 2023-09-23T03:15:15.187 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.183+0000 7f9920897640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991c1f29b0 con 0x7f9904004d00 2023-09-23T03:15:15.187 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.183+0000 7f991ad9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f991c1f1ff0 0x7f991c1f2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:15.187 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.183+0000 7f991ad9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f991c1f1ff0 0x7f991c1f2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45732/0 (socket says 172.21.15.154:45732) 2023-09-23T03:15:15.187 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.183+0000 7f991ad9d640 1 -- 172.21.15.154:0/2844462010 learned_addr learned my addr 172.21.15.154:0/2844462010 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:15.188 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.184+0000 7f9919d9b640 1 --2- 172.21.15.154:0/2844462010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f991c0c3ea0 0x7f991c0ac7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:15.188 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.184+0000 7f991a59c640 1 --2- 172.21.15.154:0/2844462010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9904004d00 0x7f991c157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:15.188 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.184+0000 7f991ad9d640 1 -- 172.21.15.154:0/2844462010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9904004d00 msgr2=0x7f991c157a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:15.188 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.184+0000 7f991ad9d640 1 --2- 172.21.15.154:0/2844462010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9904004d00 0x7f991c157a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:15.188 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.184+0000 7f991ad9d640 1 -- 172.21.15.154:0/2844462010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f991c0c3ea0 msgr2=0x7f991c0ac7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:15.188 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.184+0000 7f991ad9d640 1 --2- 172.21.15.154:0/2844462010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f991c0c3ea0 0x7f991c0ac7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:15.188 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.184+0000 7f991ad9d640 1 -- 172.21.15.154:0/2844462010 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991c0c4c50 con 0x7f991c1f1ff0 2023-09-23T03:15:15.188 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.184+0000 7f991ad9d640 1 --2- 172.21.15.154:0/2844462010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f991c1f1ff0 0x7f991c1f2470 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9914037040 tx=0x7f9914009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:15.188 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.184+0000 7f990affd640 1 -- 172.21.15.154:0/2844462010 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9914034d50 con 0x7f991c1f1ff0 2023-09-23T03:15:15.188 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.185+0000 7f990affd640 1 -- 172.21.15.154:0/2844462010 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9914003990 con 0x7f991c1f1ff0 2023-09-23T03:15:15.188 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.185+0000 7f990affd640 1 -- 172.21.15.154:0/2844462010 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9914003d20 con 0x7f991c1f1ff0 2023-09-23T03:15:15.189 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.185+0000 7f9920897640 1 -- 172.21.15.154:0/2844462010 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f991c0c4690 con 0x7f991c1f1ff0 2023-09-23T03:15:15.189 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.185+0000 7f9920897640 1 -- 172.21.15.154:0/2844462010 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f991c147eb0 con 0x7f991c1f1ff0 2023-09-23T03:15:15.190 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.186+0000 7f9920897640 1 -- 172.21.15.154:0/2844462010 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f991c15acf0 con 0x7f991c1f1ff0 2023-09-23T03:15:15.190 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.187+0000 7f990affd640 1 -- 172.21.15.154:0/2844462010 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f9914004030 con 0x7f991c1f1ff0 2023-09-23T03:15:15.190 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.188+0000 7f990affd640 1 --2- 172.21.15.154:0/2844462010 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f98fc06af60 0x7f98fc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:15.190 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.188+0000 7f990affd640 1 -- 172.21.15.154:0/2844462010 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f99140aab80 con 0x7f991c1f1ff0 2023-09-23T03:15:15.190 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.188+0000 7f991a59c640 1 --2- 172.21.15.154:0/2844462010 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f98fc06af60 0x7f98fc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:15.190 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.188+0000 7f991a59c640 1 --2- 172.21.15.154:0/2844462010 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f98fc06af60 0x7f98fc06d420 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f990c0017a0 tx=0x7f990c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:15.194 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.193+0000 7f990affd640 1 -- 172.21.15.154:0/2844462010 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f991c15acf0 con 0x7f991c1f1ff0 2023-09-23T03:15:15.387 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.386+0000 7f9920897640 1 -- 172.21.15.154:0/2844462010 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f991c15acf0 con 0x7f991c1f1ff0 2023-09-23T03:15:15.389 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.388+0000 7f990affd640 1 -- 172.21.15.154:0/2844462010 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12886 (secure 0 0 0) 0x7f991c15acf0 con 0x7f991c1f1ff0 2023-09-23T03:15:15.390 INFO:teuthology.misc.health.smithi154.stdout: 2023-09-23T03:15:15.390 INFO:teuthology.misc.health.smithi154.stdout:{"epoch":16,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:15:11.498904+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:15:15.392 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.391+0000 7f9920897640 1 -- 172.21.15.154:0/2844462010 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f98fc06af60 msgr2=0x7f98fc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:15.393 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.391+0000 7f9920897640 1 --2- 172.21.15.154:0/2844462010 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f98fc06af60 0x7f98fc06d420 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f990c0017a0 tx=0x7f990c009040 comp rx=0 tx=0).stop 2023-09-23T03:15:15.393 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.391+0000 7f9920897640 1 -- 172.21.15.154:0/2844462010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f991c1f1ff0 msgr2=0x7f991c1f2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:15.393 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.391+0000 7f9920897640 1 --2- 172.21.15.154:0/2844462010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f991c1f1ff0 0x7f991c1f2470 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9914037040 tx=0x7f9914009750 comp rx=0 tx=0).stop 2023-09-23T03:15:15.393 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.391+0000 7f9920897640 1 -- 172.21.15.154:0/2844462010 shutdown_connections 2023-09-23T03:15:15.393 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.391+0000 7f9920897640 1 --2- 172.21.15.154:0/2844462010 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f98fc06af60 0x7f98fc06d420 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:15.393 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.391+0000 7f9920897640 1 --2- 172.21.15.154:0/2844462010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f991c1f1ff0 0x7f991c1f2470 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:15.393 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.391+0000 7f9920897640 1 --2- 172.21.15.154:0/2844462010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f991c0c3ea0 0x7f991c0ac7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:15.393 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.391+0000 7f9920897640 1 --2- 172.21.15.154:0/2844462010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9904004d00 0x7f991c157a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:15.393 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.391+0000 7f9920897640 1 -- 172.21.15.154:0/2844462010 >> 172.21.15.154:0/2844462010 conn(0x7f991c0ceb50 msgr2=0x7f991c06bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:15.394 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.391+0000 7f9920897640 1 -- 172.21.15.154:0/2844462010 shutdown_connections 2023-09-23T03:15:15.394 INFO:teuthology.misc.health.smithi154.stderr:2023-09-23T03:15:15.391+0000 7f9920897640 1 -- 172.21.15.154:0/2844462010 wait complete. 2023-09-23T03:15:15.407 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-09-23T03:15:15.407 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-09-23T03:15:15.407 INFO:tasks.ceph.mds.a:Restarting daemon 2023-09-23T03:15:15.407 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-09-23T03:15:15.416 INFO:tasks.ceph.mds.a:Started 2023-09-23T03:15:15.416 INFO:tasks.ceph.mds.c:Restarting daemon 2023-09-23T03:15:15.416 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-09-23T03:15:15.424 INFO:tasks.ceph.mds.c:Started 2023-09-23T03:15:15.424 INFO:tasks.ceph.mds.b:Restarting daemon 2023-09-23T03:15:15.424 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-09-23T03:15:15.433 INFO:tasks.ceph.mds.b:Started 2023-09-23T03:15:15.434 INFO:tasks.ceph.mds.d:Restarting daemon 2023-09-23T03:15:15.434 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-09-23T03:15:15.442 INFO:tasks.ceph.mds.d:Started 2023-09-23T03:15:15.443 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-09-23T03:15:15.443 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:15:15.571 INFO:tasks.ceph.mds.a.smithi154.stdout:starting mds.a at 2023-09-23T03:15:15.579 INFO:tasks.ceph.mds.c.smithi154.stdout:starting mds.c at 2023-09-23T03:15:15.603 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-09-23T03:15:15.605 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-09-23T03:15:15.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.634+0000 7f5eb8783640 1 Processor -- start 2023-09-23T03:15:15.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.634+0000 7f5eb8783640 1 -- start start 2023-09-23T03:15:15.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.635+0000 7f5eb8783640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5eb015ca80 0x7f5eb0160680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:15.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.635+0000 7f5eb8783640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5eb00b2620 0x7f5eb00d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:15.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.635+0000 7f5eb8783640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5eb00d25b0 0x7f5eb00d2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:15.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.635+0000 7f5eb8783640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eb00b2ad0 con 0x7f5eb015ca80 2023-09-23T03:15:15.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.635+0000 7f5eb8783640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eb0161500 con 0x7f5eb00b2620 2023-09-23T03:15:15.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.635+0000 7f5eb8783640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb0160c60 con 0x7f5eb00d25b0 2023-09-23T03:15:15.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.635+0000 7f5eb651f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5eb015ca80 0x7f5eb0160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:15.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.635+0000 7f5eb651f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5eb015ca80 0x7f5eb0160680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45770/0 (socket says 172.21.15.154:45770) 2023-09-23T03:15:15.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.635+0000 7f5eb651f640 1 -- 172.21.15.154:0/3549938042 learned_addr learned my addr 172.21.15.154:0/3549938042 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:15.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.636+0000 7f5eb5d1e640 1 --2- 172.21.15.154:0/3549938042 >> v2:172.21.15.190:3300/0 conn(0x7f5eb00b2620 0x7f5eb00d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:15.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.636+0000 7f5eb6d20640 1 --2- 172.21.15.154:0/3549938042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5eb00d25b0 0x7f5eb00d2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:15.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.636+0000 7f5eb651f640 1 -- 172.21.15.154:0/3549938042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5eb00d25b0 msgr2=0x7f5eb00d2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:15.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.636+0000 7f5eb651f640 1 --2- 172.21.15.154:0/3549938042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5eb00d25b0 0x7f5eb00d2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:15.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.636+0000 7f5eb651f640 1 -- 172.21.15.154:0/3549938042 >> v2:172.21.15.190:3300/0 conn(0x7f5eb00b2620 msgr2=0x7f5eb00d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:15.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.636+0000 7f5eb651f640 1 --2- 172.21.15.154:0/3549938042 >> v2:172.21.15.190:3300/0 conn(0x7f5eb00b2620 0x7f5eb00d07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:15.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.636+0000 7f5eb651f640 1 -- 172.21.15.154:0/3549938042 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb0160f40 con 0x7f5eb015ca80 2023-09-23T03:15:15.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.636+0000 7f5eb6d20640 1 --2- 172.21.15.154:0/3549938042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5eb00d25b0 0x7f5eb00d2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:15.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.636+0000 7f5eb5d1e640 1 --2- 172.21.15.154:0/3549938042 >> v2:172.21.15.190:3300/0 conn(0x7f5eb00b2620 0x7f5eb00d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:15.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.637+0000 7f5eb651f640 1 --2- 172.21.15.154:0/3549938042 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5eb015ca80 0x7f5eb0160680 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0067210 tx=0x7f5eac02b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6eff17d8a2dc0f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:15.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.637+0000 7f5eb551d640 1 -- 172.21.15.154:0/3549938042 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eac004030 con 0x7f5eb015ca80 2023-09-23T03:15:15.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.637+0000 7f5eb551d640 1 -- 172.21.15.154:0/3549938042 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eac004190 con 0x7f5eb015ca80 2023-09-23T03:15:15.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.637+0000 7f5eb551d640 1 -- 172.21.15.154:0/3549938042 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eac031640 con 0x7f5eb015ca80 2023-09-23T03:15:15.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.637+0000 7f5eb8783640 1 -- 172.21.15.154:0/3549938042 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5eb015ca80 msgr2=0x7f5eb0160680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:15.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.637+0000 7f5eb8783640 1 --2- 172.21.15.154:0/3549938042 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5eb015ca80 0x7f5eb0160680 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0067210 tx=0x7f5eac02b8e0 comp rx=0 tx=0).stop 2023-09-23T03:15:15.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.638+0000 7f5eb8783640 1 -- 172.21.15.154:0/3549938042 shutdown_connections 2023-09-23T03:15:15.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.638+0000 7f5eb8783640 1 --2- 172.21.15.154:0/3549938042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5eb00d25b0 0x7f5eb00d2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:15.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.638+0000 7f5eb8783640 1 --2- 172.21.15.154:0/3549938042 >> v2:172.21.15.190:3300/0 conn(0x7f5eb00b2620 0x7f5eb00d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:15.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.638+0000 7f5eb8783640 1 --2- 172.21.15.154:0/3549938042 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5eb015ca80 0x7f5eb0160680 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:15.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.638+0000 7f5eb8783640 1 -- 172.21.15.154:0/3549938042 >> 172.21.15.154:0/3549938042 conn(0x7f5eb009bc60 msgr2=0x7f5eb0089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:15.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.638+0000 7f5eb8783640 1 -- 172.21.15.154:0/3549938042 shutdown_connections 2023-09-23T03:15:15.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.638+0000 7f5eb8783640 1 -- 172.21.15.154:0/3549938042 wait complete. 2023-09-23T03:15:15.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.639+0000 7f5eb8783640 1 Processor -- start 2023-09-23T03:15:15.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.640+0000 7f5eb8783640 1 -- start start 2023-09-23T03:15:15.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.640+0000 7f5eb8783640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5eb00b2620 0x7f5eb01f54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:15.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.640+0000 7f5eb8783640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5eb00d25b0 0x7f5eb01f2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:15.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.640+0000 7f5eb8783640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5eb015ca80 0x7f5eb00ce910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:15.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.640+0000 7f5eb8783640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb01f6490 con 0x7f5eb015ca80 2023-09-23T03:15:15.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.640+0000 7f5eb8783640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb01f59e0 con 0x7f5eb00d25b0 2023-09-23T03:15:15.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.640+0000 7f5eb8783640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb01f2e90 con 0x7f5eb00b2620 2023-09-23T03:15:15.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.641+0000 7f5eb6d20640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5eb015ca80 0x7f5eb00ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:15.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.641+0000 7f5eb6d20640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5eb015ca80 0x7f5eb00ce910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45772/0 (socket says 172.21.15.154:45772) 2023-09-23T03:15:15.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.641+0000 7f5eb6d20640 1 -- 172.21.15.154:0/1914126239 learned_addr learned my addr 172.21.15.154:0/1914126239 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:15.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.641+0000 7f5eb651f640 1 --2- 172.21.15.154:0/1914126239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5eb00b2620 0x7f5eb01f54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:15.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.641+0000 7f5eb5d1e640 1 --2- 172.21.15.154:0/1914126239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5eb00d25b0 0x7f5eb01f2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:15.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.641+0000 7f5eb6d20640 1 -- 172.21.15.154:0/1914126239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5eb00b2620 msgr2=0x7f5eb01f54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:15.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.641+0000 7f5eb6d20640 1 --2- 172.21.15.154:0/1914126239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5eb00b2620 0x7f5eb01f54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:15.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.641+0000 7f5eb6d20640 1 -- 172.21.15.154:0/1914126239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5eb00d25b0 msgr2=0x7f5eb01f2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:15.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.641+0000 7f5eb6d20640 1 --2- 172.21.15.154:0/1914126239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5eb00d25b0 0x7f5eb01f2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:15.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.641+0000 7f5eb6d20640 1 -- 172.21.15.154:0/1914126239 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eac009460 con 0x7f5eb015ca80 2023-09-23T03:15:15.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.641+0000 7f5eb651f640 1 --2- 172.21.15.154:0/1914126239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5eb00b2620 0x7f5eb01f54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:15:15.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.641+0000 7f5eb5d1e640 1 --2- 172.21.15.154:0/1914126239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5eb00d25b0 0x7f5eb01f2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:15:15.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.641+0000 7f5eb6d20640 1 --2- 172.21.15.154:0/1914126239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5eb015ca80 0x7f5eb00ce910 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5ea800cef0 tx=0x7f5ea800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:15.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.642+0000 7f5ea6ffd640 1 -- 172.21.15.154:0/1914126239 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea800d840 con 0x7f5eb015ca80 2023-09-23T03:15:15.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.642+0000 7f5ea6ffd640 1 -- 172.21.15.154:0/1914126239 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea800d9c0 con 0x7f5eb015ca80 2023-09-23T03:15:15.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.642+0000 7f5ea6ffd640 1 -- 172.21.15.154:0/1914126239 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea800dd70 con 0x7f5eb015ca80 2023-09-23T03:15:15.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.642+0000 7f5eb8783640 1 -- 172.21.15.154:0/1914126239 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb01f5bf0 con 0x7f5eb015ca80 2023-09-23T03:15:15.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.642+0000 7f5eb8783640 1 -- 172.21.15.154:0/1914126239 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb01f61b0 con 0x7f5eb015ca80 2023-09-23T03:15:15.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.643+0000 7f5eb8783640 1 -- 172.21.15.154:0/1914126239 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eb0073eb0 con 0x7f5eb015ca80 2023-09-23T03:15:15.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.644+0000 7f5ea6ffd640 1 -- 172.21.15.154:0/1914126239 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f5ea801e3b0 con 0x7f5eb015ca80 2023-09-23T03:15:15.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.645+0000 7f5ea6ffd640 1 --2- 172.21.15.154:0/1914126239 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5e9406af30 0x7f5e9406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:15.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.645+0000 7f5ea6ffd640 1 -- 172.21.15.154:0/1914126239 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5ea8089fb0 con 0x7f5eb015ca80 2023-09-23T03:15:15.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.645+0000 7f5eb651f640 1 --2- 172.21.15.154:0/1914126239 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5e9406af30 0x7f5e9406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:15.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.646+0000 7f5eb651f640 1 --2- 172.21.15.154:0/1914126239 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5e9406af30 0x7f5e9406d3f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5eac035000 tx=0x7f5eac008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:15.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.650+0000 7f5ea6ffd640 1 -- 172.21.15.154:0/1914126239 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f5ea8055360 con 0x7f5eb015ca80 2023-09-23T03:15:15.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.835+0000 7f5eb8783640 1 -- 172.21.15.154:0/1914126239 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5eb015b0c0 con 0x7f5eb015ca80 2023-09-23T03:15:15.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.838+0000 7f5ea6ffd640 1 -- 172.21.15.154:0/1914126239 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12886 (secure 0 0 0) 0x7f5ea801a090 con 0x7f5eb015ca80 2023-09-23T03:15:15.839 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:15:15.839 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":16,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:15:11.498904+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:15:15.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.840+0000 7f5eb8783640 1 -- 172.21.15.154:0/1914126239 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5e9406af30 msgr2=0x7f5e9406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:15.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.840+0000 7f5eb8783640 1 --2- 172.21.15.154:0/1914126239 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5e9406af30 0x7f5e9406d3f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5eac035000 tx=0x7f5eac008030 comp rx=0 tx=0).stop 2023-09-23T03:15:15.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.840+0000 7f5eb8783640 1 -- 172.21.15.154:0/1914126239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5eb015ca80 msgr2=0x7f5eb00ce910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:15.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.840+0000 7f5eb8783640 1 --2- 172.21.15.154:0/1914126239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5eb015ca80 0x7f5eb00ce910 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5ea800cef0 tx=0x7f5ea800a710 comp rx=0 tx=0).stop 2023-09-23T03:15:15.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.840+0000 7f5eb8783640 1 -- 172.21.15.154:0/1914126239 shutdown_connections 2023-09-23T03:15:15.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.840+0000 7f5eb8783640 1 --2- 172.21.15.154:0/1914126239 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5e9406af30 0x7f5e9406d3f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:15.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.840+0000 7f5eb8783640 1 --2- 172.21.15.154:0/1914126239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5eb015ca80 0x7f5eb00ce910 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:15.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.840+0000 7f5eb8783640 1 --2- 172.21.15.154:0/1914126239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5eb00d25b0 0x7f5eb01f2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:15.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.840+0000 7f5eb8783640 1 --2- 172.21.15.154:0/1914126239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5eb00b2620 0x7f5eb01f54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:15.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.840+0000 7f5eb8783640 1 -- 172.21.15.154:0/1914126239 >> 172.21.15.154:0/1914126239 conn(0x7f5eb009bc60 msgr2=0x7f5eb0089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:15.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.840+0000 7f5eb8783640 1 -- 172.21.15.154:0/1914126239 shutdown_connections 2023-09-23T03:15:15.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:15.840+0000 7f5eb8783640 1 -- 172.21.15.154:0/1914126239 wait complete. 2023-09-23T03:15:15.859 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T03:15:15.859 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:15:16.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.073+0000 7f7aad0e0640 1 Processor -- start 2023-09-23T03:15:16.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.073+0000 7f7aad0e0640 1 -- start start 2023-09-23T03:15:16.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.074+0000 7f7aad0e0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7aa815bb30 0x7f7aa80caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:16.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.075+0000 7f7aad0e0640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aa815b6a0 con 0x7f7aa815aba0 2023-09-23T03:15:16.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.075+0000 7f7aad0e0640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aa815f0b0 con 0x7f7aa80cb3e0 2023-09-23T03:15:16.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.075+0000 7f7aad0e0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa80cb870 con 0x7f7aa815bb30 2023-09-23T03:15:16.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.075+0000 7f7aa6d9d640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f7aa815aba0 0x7f7aa815e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:59072/0 (socket says 172.21.15.154:59072) 2023-09-23T03:15:16.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.075+0000 7f7aa6d9d640 1 -- 172.21.15.154:0/2143275694 learned_addr learned my addr 172.21.15.154:0/2143275694 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:16.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.075+0000 7f7aa659c640 1 --2- 172.21.15.154:0/2143275694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7aa815bb30 0x7f7aa80caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:16.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.075+0000 7f7aa5d9b640 1 -- 172.21.15.154:0/2143275694 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1774457909 0 0) 0x7f7aa815b6a0 con 0x7f7aa815aba0 2023-09-23T03:15:16.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.075+0000 7f7aa5d9b640 1 -- 172.21.15.154:0/2143275694 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a90003520 con 0x7f7aa815aba0 2023-09-23T03:15:16.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.076+0000 7f7aa5d9b640 1 -- 172.21.15.154:0/2143275694 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3259511924 0 0) 0x7f7a90003520 con 0x7f7aa815aba0 2023-09-23T03:15:16.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.076+0000 7f7aa5d9b640 1 -- 172.21.15.154:0/2143275694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7aa815bb30 msgr2=0x7f7aa80caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:16.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.076+0000 7f7aa5d9b640 1 --2- 172.21.15.154:0/2143275694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7aa815bb30 0x7f7aa80caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:16.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.076+0000 7f7aa5d9b640 1 -- 172.21.15.154:0/2143275694 >> v1:172.21.15.190:6789/0 conn(0x7f7aa80cb3e0 legacy=0x7f7aa80cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:16.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.076+0000 7f7aa5d9b640 1 -- 172.21.15.154:0/2143275694 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa815f390 con 0x7f7aa815aba0 2023-09-23T03:15:16.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.076+0000 7f7aa5d9b640 1 -- 172.21.15.154:0/2143275694 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f7a98001670 con 0x7f7aa815aba0 2023-09-23T03:15:16.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.076+0000 7f7aa5d9b640 1 -- 172.21.15.154:0/2143275694 >> v1:172.21.15.154:6789/0 conn(0x7f7aa815aba0 legacy=0x7f7aa815e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:16.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.076+0000 7f7aa5d9b640 1 --2- 172.21.15.154:0/2143275694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7aa815bb30 0x7f7a900039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:16.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.076+0000 7f7aa5d9b640 1 --2- 172.21.15.154:0/2143275694 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a90003f20 0x7f7a900047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:16.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.076+0000 7f7aa5d9b640 1 --2- 172.21.15.154:0/2143275694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a90004d00 0x7f7a90005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:16.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.076+0000 7f7aa5d9b640 1 -- 172.21.15.154:0/2143275694 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa80cb870 con 0x7f7a90003f20 2023-09-23T03:15:16.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.076+0000 7f7aa5d9b640 1 -- 172.21.15.154:0/2143275694 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9000ba60 con 0x7f7a90004d00 2023-09-23T03:15:16.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.076+0000 7f7aa5d9b640 1 -- 172.21.15.154:0/2143275694 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9000bc20 con 0x7f7aa815bb30 2023-09-23T03:15:16.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.076+0000 7f7aa659c640 1 --2- 172.21.15.154:0/2143275694 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a90003f20 0x7f7a900047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:16.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.076+0000 7f7aa6d9d640 1 --2- 172.21.15.154:0/2143275694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7aa815bb30 0x7f7a900039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:16.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.076+0000 7f7aa659c640 1 -- 172.21.15.154:0/2143275694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7aa815bb30 msgr2=0x7f7a900039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:16.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.076+0000 7f7aa659c640 1 --2- 172.21.15.154:0/2143275694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7aa815bb30 0x7f7a900039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:16.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.076+0000 7f7aa659c640 1 -- 172.21.15.154:0/2143275694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a90004d00 msgr2=0x7f7a90005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:16.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.076+0000 7f7aa659c640 1 --2- 172.21.15.154:0/2143275694 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a90004d00 0x7f7a90005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:16.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.077+0000 7f7aa659c640 1 -- 172.21.15.154:0/2143275694 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a90003520 con 0x7f7a90003f20 2023-09-23T03:15:16.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.077+0000 7f7aa6d9d640 1 --2- 172.21.15.154:0/2143275694 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7aa815bb30 0x7f7a900039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:16.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.078+0000 7f7aa659c640 1 --2- 172.21.15.154:0/2143275694 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a90003f20 0x7f7a900047c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009ac0 tx=0x7f7a900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36b53eca8fafcae server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:16.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.078+0000 7f7aa5d9b640 1 -- 172.21.15.154:0/2143275694 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c02c6b0 con 0x7f7a90003f20 2023-09-23T03:15:16.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.078+0000 7f7aa5d9b640 1 -- 172.21.15.154:0/2143275694 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a9c02c810 con 0x7f7a90003f20 2023-09-23T03:15:16.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.078+0000 7f7aa5d9b640 1 -- 172.21.15.154:0/2143275694 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c02cba0 con 0x7f7a90003f20 2023-09-23T03:15:16.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.078+0000 7f7aad0e0640 1 -- 172.21.15.154:0/2143275694 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a90003f20 msgr2=0x7f7a900047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:16.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.078+0000 7f7aad0e0640 1 --2- 172.21.15.154:0/2143275694 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a90003f20 0x7f7a900047c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009ac0 tx=0x7f7a900057f0 comp rx=0 tx=0).stop 2023-09-23T03:15:16.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.078+0000 7f7aa759e640 1 -- 172.21.15.154:0/2143275694 reap_dead start 2023-09-23T03:15:16.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.078+0000 7f7aad0e0640 1 -- 172.21.15.154:0/2143275694 shutdown_connections 2023-09-23T03:15:16.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.078+0000 7f7aad0e0640 1 -- 172.21.15.154:0/2143275694 >> 172.21.15.154:0/2143275694 conn(0x7f7aa806c010 msgr2=0x7f7aa806c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:16.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.079+0000 7f7aad0e0640 1 -- 172.21.15.154:0/2143275694 shutdown_connections 2023-09-23T03:15:16.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.079+0000 7f7aad0e0640 1 -- 172.21.15.154:0/2143275694 wait complete. 2023-09-23T03:15:16.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.080+0000 7f7aad0e0640 1 Processor -- start 2023-09-23T03:15:16.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.080+0000 7f7aad0e0640 1 -- start start 2023-09-23T03:15:16.084 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.081+0000 7f7aad0e0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a90003f20 0x7f7aa81f6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:16.084 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.081+0000 7f7aad0e0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7aa815c920 0x7f7aa80c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:16.084 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.081+0000 7f7aad0e0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7aa8155910 0x7f7aa80c22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:16.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.081+0000 7f7aad0e0640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa80ca4d0 con 0x7f7aa8155910 2023-09-23T03:15:16.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.081+0000 7f7aad0e0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa81f5d30 con 0x7f7aa815c920 2023-09-23T03:15:16.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.081+0000 7f7aad0e0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa80c9040 con 0x7f7a90003f20 2023-09-23T03:15:16.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.082+0000 7f7aa759e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7aa8155910 0x7f7aa80c22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:16.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.082+0000 7f7aa759e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7aa8155910 0x7f7aa80c22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45790/0 (socket says 172.21.15.154:45790) 2023-09-23T03:15:16.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.082+0000 7f7aa759e640 1 -- 172.21.15.154:0/3831157022 learned_addr learned my addr 172.21.15.154:0/3831157022 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:16.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.082+0000 7f7aa6d9d640 1 --2- 172.21.15.154:0/3831157022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a90003f20 0x7f7aa81f6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:16.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.082+0000 7f7aa759e640 1 -- 172.21.15.154:0/3831157022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a90003f20 msgr2=0x7f7aa81f6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:16.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.082+0000 7f7aa759e640 1 --2- 172.21.15.154:0/3831157022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a90003f20 0x7f7aa81f6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:16.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.082+0000 7f7aa759e640 1 -- 172.21.15.154:0/3831157022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7aa815c920 msgr2=0x7f7aa80c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:16.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.082+0000 7f7aa759e640 1 --2- 172.21.15.154:0/3831157022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7aa815c920 0x7f7aa80c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:16.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.082+0000 7f7aa759e640 1 -- 172.21.15.154:0/3831157022 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa80ca6e0 con 0x7f7aa8155910 2023-09-23T03:15:16.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.082+0000 7f7aa759e640 1 --2- 172.21.15.154:0/3831157022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7aa8155910 0x7f7aa80c22c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7aa80ad7a0 tx=0x7f7aa000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:16.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.083+0000 7f7a8f7fe640 1 -- 172.21.15.154:0/3831157022 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0005d40 con 0x7f7aa8155910 2023-09-23T03:15:16.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.083+0000 7f7a8f7fe640 1 -- 172.21.15.154:0/3831157022 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa000ced0 con 0x7f7aa8155910 2023-09-23T03:15:16.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.083+0000 7f7aad0e0640 1 -- 172.21.15.154:0/3831157022 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aa80ca9c0 con 0x7f7aa8155910 2023-09-23T03:15:16.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.083+0000 7f7a8f7fe640 1 -- 172.21.15.154:0/3831157022 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa000a7f0 con 0x7f7aa8155910 2023-09-23T03:15:16.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.083+0000 7f7aad0e0640 1 -- 172.21.15.154:0/3831157022 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aa81f6260 con 0x7f7aa8155910 2023-09-23T03:15:16.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.084+0000 7f7aad0e0640 1 -- 172.21.15.154:0/3831157022 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aa81f6480 con 0x7f7aa8155910 2023-09-23T03:15:16.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.085+0000 7f7a8f7fe640 1 -- 172.21.15.154:0/3831157022 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f7aa000a950 con 0x7f7aa8155910 2023-09-23T03:15:16.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.086+0000 7f7a8f7fe640 1 --2- 172.21.15.154:0/3831157022 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7a9406af60 0x7f7a9406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:16.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.086+0000 7f7a8f7fe640 1 -- 172.21.15.154:0/3831157022 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7aa00884e0 con 0x7f7aa8155910 2023-09-23T03:15:16.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.086+0000 7f7aa6d9d640 1 --2- 172.21.15.154:0/3831157022 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7a9406af60 0x7f7a9406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:16.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.087+0000 7f7aa6d9d640 1 --2- 172.21.15.154:0/3831157022 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7a9406af60 0x7f7a9406d420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7a98001eb0 tx=0x7f7a98001550 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:16.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.090+0000 7f7a8f7fe640 1 -- 172.21.15.154:0/3831157022 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f7aa0053aa0 con 0x7f7aa8155910 2023-09-23T03:15:16.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.277+0000 7f7aad0e0640 1 -- 172.21.15.154:0/3831157022 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7aa804e920 con 0x7f7aa8155910 2023-09-23T03:15:16.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.278+0000 7f7a8f7fe640 1 -- 172.21.15.154:0/3831157022 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7aa00570c0 con 0x7f7aa8155910 2023-09-23T03:15:16.280 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:15:16.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.281+0000 7f7aad0e0640 1 -- 172.21.15.154:0/3831157022 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7a9406af60 msgr2=0x7f7a9406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:16.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.281+0000 7f7aad0e0640 1 --2- 172.21.15.154:0/3831157022 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7a9406af60 0x7f7a9406d420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7a98001eb0 tx=0x7f7a98001550 comp rx=0 tx=0).stop 2023-09-23T03:15:16.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.281+0000 7f7aad0e0640 1 -- 172.21.15.154:0/3831157022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7aa8155910 msgr2=0x7f7aa80c22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:16.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.281+0000 7f7aad0e0640 1 --2- 172.21.15.154:0/3831157022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7aa8155910 0x7f7aa80c22c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7aa80ad7a0 tx=0x7f7aa000cb90 comp rx=0 tx=0).stop 2023-09-23T03:15:16.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.281+0000 7f7aad0e0640 1 -- 172.21.15.154:0/3831157022 shutdown_connections 2023-09-23T03:15:16.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.281+0000 7f7aad0e0640 1 --2- 172.21.15.154:0/3831157022 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7a9406af60 0x7f7a9406d420 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:16.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.281+0000 7f7aad0e0640 1 --2- 172.21.15.154:0/3831157022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7aa8155910 0x7f7aa80c22c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:16.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.281+0000 7f7aad0e0640 1 --2- 172.21.15.154:0/3831157022 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7aa815c920 0x7f7aa80c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:16.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.281+0000 7f7aad0e0640 1 --2- 172.21.15.154:0/3831157022 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a90003f20 0x7f7aa81f6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:16.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.281+0000 7f7aad0e0640 1 -- 172.21.15.154:0/3831157022 >> 172.21.15.154:0/3831157022 conn(0x7f7aa806c010 msgr2=0x7f7aa8158e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:16.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.281+0000 7f7aad0e0640 1 -- 172.21.15.154:0/3831157022 shutdown_connections 2023-09-23T03:15:16.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.281+0000 7f7aad0e0640 1 -- 172.21.15.154:0/3831157022 wait complete. 2023-09-23T03:15:16.299 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:15:16.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.502+0000 7f33776ec640 1 Processor -- start 2023-09-23T03:15:16.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.502+0000 7f33776ec640 1 -- start start 2023-09-23T03:15:16.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.504+0000 7f33776ec640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33700b1b20 0x7f33700ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:16.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.504+0000 7f33776ec640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f33700cee70 0x7f33700ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:16.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.504+0000 7f33776ec640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f337015ac30 con 0x7f33700acce0 2023-09-23T03:15:16.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.504+0000 7f33776ec640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33700b2e90 con 0x7f33700cee70 2023-09-23T03:15:16.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.504+0000 7f33776ec640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33700cf320 con 0x7f33700b1b20 2023-09-23T03:15:16.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.504+0000 7f3375c89640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f33700cee70 0x7f33700ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:16.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.504+0000 7f3375c89640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f33700cee70 0x7f33700ab140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45800/0 (socket says 172.21.15.154:45800) 2023-09-23T03:15:16.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.504+0000 7f3375c89640 1 -- 172.21.15.154:0/3134653081 learned_addr learned my addr 172.21.15.154:0/3134653081 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:16.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.504+0000 7f3374c87640 1 --2- 172.21.15.154:0/3134653081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33700b1b20 0x7f33700ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:16.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.504+0000 7f3375c89640 1 -- 172.21.15.154:0/3134653081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33700b1b20 msgr2=0x7f33700ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:16.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.504+0000 7f3375c89640 1 --2- 172.21.15.154:0/3134653081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33700b1b20 0x7f33700ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:16.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.504+0000 7f3375c89640 1 -- 172.21.15.154:0/3134653081 >> v1:172.21.15.190:6789/0 conn(0x7f33700acce0 legacy=0x7f337015ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:16.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.504+0000 7f3375c89640 1 -- 172.21.15.154:0/3134653081 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33700b1f70 con 0x7f33700cee70 2023-09-23T03:15:16.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.504+0000 7f3374c87640 1 --2- 172.21.15.154:0/3134653081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33700b1b20 0x7f33700ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:16.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.505+0000 7f3375c89640 1 --2- 172.21.15.154:0/3134653081 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33700cee70 0x7f33700ab140 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3368009e50 tx=0x7f336802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2581f5cc58591abd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:16.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.506+0000 7f3367fff640 1 -- 172.21.15.154:0/3134653081 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336802c9f0 con 0x7f33700cee70 2023-09-23T03:15:16.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.506+0000 7f3367fff640 1 -- 172.21.15.154:0/3134653081 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f336802cb50 con 0x7f33700cee70 2023-09-23T03:15:16.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.506+0000 7f3367fff640 1 -- 172.21.15.154:0/3134653081 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33680343f0 con 0x7f33700cee70 2023-09-23T03:15:16.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.506+0000 7f33776ec640 1 -- 172.21.15.154:0/3134653081 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33700cee70 msgr2=0x7f33700ab140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:16.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.506+0000 7f33776ec640 1 --2- 172.21.15.154:0/3134653081 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33700cee70 0x7f33700ab140 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3368009e50 tx=0x7f336802be50 comp rx=0 tx=0).stop 2023-09-23T03:15:16.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.507+0000 7f33776ec640 1 -- 172.21.15.154:0/3134653081 shutdown_connections 2023-09-23T03:15:16.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.507+0000 7f33776ec640 1 --2- 172.21.15.154:0/3134653081 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33700cee70 0x7f33700ab140 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:16.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.507+0000 7f33776ec640 1 --2- 172.21.15.154:0/3134653081 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33700b1b20 0x7f33700ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:16.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.507+0000 7f33776ec640 1 -- 172.21.15.154:0/3134653081 >> 172.21.15.154:0/3134653081 conn(0x7f337006a0f0 msgr2=0x7f337006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:16.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.507+0000 7f33776ec640 1 -- 172.21.15.154:0/3134653081 shutdown_connections 2023-09-23T03:15:16.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.507+0000 7f33776ec640 1 -- 172.21.15.154:0/3134653081 wait complete. 2023-09-23T03:15:16.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.509+0000 7f33776ec640 1 Processor -- start 2023-09-23T03:15:16.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.509+0000 7f33776ec640 1 -- start start 2023-09-23T03:15:16.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.510+0000 7f33776ec640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33700acce0 0x7f33701548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:16.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.510+0000 7f33776ec640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33700b1b20 0x7f33700c1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:16.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.510+0000 7f33776ec640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33700cee70 0x7f33700c24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:16.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.510+0000 7f33776ec640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33700cc0e0 con 0x7f33700acce0 2023-09-23T03:15:16.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.510+0000 7f33776ec640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33700c3f90 con 0x7f33700cee70 2023-09-23T03:15:16.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.510+0000 7f33776ec640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33700c4270 con 0x7f33700b1b20 2023-09-23T03:15:16.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.510+0000 7f3375488640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33700acce0 0x7f33701548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:16.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.511+0000 7f3375488640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33700acce0 0x7f33701548e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45808/0 (socket says 172.21.15.154:45808) 2023-09-23T03:15:16.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.511+0000 7f3375488640 1 -- 172.21.15.154:0/464366079 learned_addr learned my addr 172.21.15.154:0/464366079 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:16.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.511+0000 7f3374c87640 1 --2- 172.21.15.154:0/464366079 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33700b1b20 0x7f33700c1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:16.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.511+0000 7f3375488640 1 -- 172.21.15.154:0/464366079 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33700b1b20 msgr2=0x7f33700c1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:16.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.511+0000 7f3375c89640 1 --2- 172.21.15.154:0/464366079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33700cee70 0x7f33700c24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:16.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.511+0000 7f3375488640 1 --2- 172.21.15.154:0/464366079 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33700b1b20 0x7f33700c1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:16.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.511+0000 7f3375488640 1 -- 172.21.15.154:0/464366079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33700cee70 msgr2=0x7f33700c24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:16.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.511+0000 7f3375488640 1 --2- 172.21.15.154:0/464366079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33700cee70 0x7f33700c24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:16.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.511+0000 7f3375488640 1 -- 172.21.15.154:0/464366079 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3368009b30 con 0x7f33700acce0 2023-09-23T03:15:16.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.511+0000 7f3374c87640 1 --2- 172.21.15.154:0/464366079 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33700b1b20 0x7f33700c1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:16.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.511+0000 7f3375488640 1 --2- 172.21.15.154:0/464366079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33700acce0 0x7f33701548e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f336c004640 tx=0x7f336c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:16.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.511+0000 7f3365ffb640 1 -- 172.21.15.154:0/464366079 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336c011790 con 0x7f33700acce0 2023-09-23T03:15:16.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.512+0000 7f3365ffb640 1 -- 172.21.15.154:0/464366079 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f336c0118f0 con 0x7f33700acce0 2023-09-23T03:15:16.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.512+0000 7f3365ffb640 1 -- 172.21.15.154:0/464366079 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336c011ce0 con 0x7f33700acce0 2023-09-23T03:15:16.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.512+0000 7f33776ec640 1 -- 172.21.15.154:0/464366079 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33700c4550 con 0x7f33700acce0 2023-09-23T03:15:16.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.512+0000 7f33776ec640 1 -- 172.21.15.154:0/464366079 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33700c3bd0 con 0x7f33700acce0 2023-09-23T03:15:16.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.513+0000 7f33776ec640 1 -- 172.21.15.154:0/464366079 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33700c4a70 con 0x7f33700acce0 2023-09-23T03:15:16.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.514+0000 7f3365ffb640 1 -- 172.21.15.154:0/464366079 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f336c01c460 con 0x7f33700acce0 2023-09-23T03:15:16.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.515+0000 7f3365ffb640 1 --2- 172.21.15.154:0/464366079 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f335406af30 0x7f335406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:16.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.515+0000 7f3365ffb640 1 -- 172.21.15.154:0/464366079 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f336c0892f0 con 0x7f33700acce0 2023-09-23T03:15:16.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.515+0000 7f3374c87640 1 --2- 172.21.15.154:0/464366079 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f335406af30 0x7f335406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:16.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.516+0000 7f3374c87640 1 --2- 172.21.15.154:0/464366079 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f335406af30 0x7f335406d3f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3360006fd0 tx=0x7f3360008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:16.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.520+0000 7f3365ffb640 1 -- 172.21.15.154:0/464366079 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f336c054680 con 0x7f33700acce0 2023-09-23T03:15:16.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.747+0000 7f33776ec640 1 -- 172.21.15.154:0/464366079 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3370073eb0 con 0x7f33700acce0 2023-09-23T03:15:16.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.748+0000 7f3365ffb640 1 -- 172.21.15.154:0/464366079 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3706 (secure 0 0 0) 0x7f336c057ca0 con 0x7f33700acce0 2023-09-23T03:15:16.749 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:15:16.750 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2644365591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2644365591},{"type":"v1","addr":"172.21.15.190:6835","nonce":2644365591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/4145467852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4145467852},{"type":"v1","addr":"172.21.15.154:6835","nonce":4145467852}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1022302774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1022302774},{"type":"v1","addr":"172.21.15.154:6837","nonce":1022302774}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3561509423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3561509423},{"type":"v1","addr":"172.21.15.190:6834","nonce":3561509423}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-09-23T03:15:16.750 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 3 2023-09-23T03:15:16.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.750+0000 7f33776ec640 1 -- 172.21.15.154:0/464366079 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f335406af30 msgr2=0x7f335406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:16.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.750+0000 7f33776ec640 1 --2- 172.21.15.154:0/464366079 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f335406af30 0x7f335406d3f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3360006fd0 tx=0x7f3360008040 comp rx=0 tx=0).stop 2023-09-23T03:15:16.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.750+0000 7f33776ec640 1 -- 172.21.15.154:0/464366079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33700acce0 msgr2=0x7f33701548e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:16.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.750+0000 7f33776ec640 1 --2- 172.21.15.154:0/464366079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33700acce0 0x7f33701548e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f336c004640 tx=0x7f336c00c8e0 comp rx=0 tx=0).stop 2023-09-23T03:15:16.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.751+0000 7f33776ec640 1 -- 172.21.15.154:0/464366079 shutdown_connections 2023-09-23T03:15:16.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.751+0000 7f33776ec640 1 --2- 172.21.15.154:0/464366079 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f335406af30 0x7f335406d3f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:16.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.751+0000 7f33776ec640 1 --2- 172.21.15.154:0/464366079 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33700cee70 0x7f33700c24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:16.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.751+0000 7f33776ec640 1 --2- 172.21.15.154:0/464366079 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33700b1b20 0x7f33700c1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:16.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.751+0000 7f33776ec640 1 --2- 172.21.15.154:0/464366079 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33700acce0 0x7f33701548e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:16.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.751+0000 7f33776ec640 1 -- 172.21.15.154:0/464366079 >> 172.21.15.154:0/464366079 conn(0x7f337006a0f0 msgr2=0x7f33700aecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:16.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.751+0000 7f33776ec640 1 -- 172.21.15.154:0/464366079 shutdown_connections 2023-09-23T03:15:16.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.751+0000 7f33776ec640 1 -- 172.21.15.154:0/464366079 wait complete. 2023-09-23T03:15:16.771 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:15:16.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.976+0000 7fef9b832640 1 Processor -- start 2023-09-23T03:15:16.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.977+0000 7fef9b832640 1 -- start start 2023-09-23T03:15:16.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.978+0000 7fef9b832640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fef940acce0 0x7fef9415ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:16.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.978+0000 7fef9b832640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef940b1b20 0x7fef940ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:16.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.978+0000 7fef9b832640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef9415ac30 con 0x7fef940cee70 2023-09-23T03:15:16.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.978+0000 7fef9b832640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef940b2e90 con 0x7fef940acce0 2023-09-23T03:15:16.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.978+0000 7fef9b832640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef940cf320 con 0x7fef940b1b20 2023-09-23T03:15:16.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.978+0000 7fef995ce640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fef940acce0 0x7fef9415ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:16.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.979+0000 7fef995ce640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fef940acce0 0x7fef9415ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45812/0 (socket says 172.21.15.154:45812) 2023-09-23T03:15:16.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.979+0000 7fef995ce640 1 -- 172.21.15.154:0/4162776581 learned_addr learned my addr 172.21.15.154:0/4162776581 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:16.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.979+0000 7fef98dcd640 1 --2- 172.21.15.154:0/4162776581 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef940b1b20 0x7fef940ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:16.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.979+0000 7fef995ce640 1 -- 172.21.15.154:0/4162776581 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef940b1b20 msgr2=0x7fef940ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:16.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.979+0000 7fef995ce640 1 --2- 172.21.15.154:0/4162776581 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef940b1b20 0x7fef940ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:16.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.979+0000 7fef995ce640 1 -- 172.21.15.154:0/4162776581 >> v1:172.21.15.190:6789/0 conn(0x7fef940cee70 legacy=0x7fef9406cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:16.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.979+0000 7fef995ce640 1 -- 172.21.15.154:0/4162776581 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef940b1f70 con 0x7fef940acce0 2023-09-23T03:15:16.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.981+0000 7fef995ce640 1 --2- 172.21.15.154:0/4162776581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef940acce0 0x7fef9415ed60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fef90009850 tx=0x7fef9002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f2152a3ecee2dc4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:16.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.981+0000 7fef8bfff640 1 -- 172.21.15.154:0/4162776581 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef90004230 con 0x7fef940acce0 2023-09-23T03:15:16.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.981+0000 7fef8bfff640 1 -- 172.21.15.154:0/4162776581 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef90004390 con 0x7fef940acce0 2023-09-23T03:15:16.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.981+0000 7fef8bfff640 1 -- 172.21.15.154:0/4162776581 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef90031830 con 0x7fef940acce0 2023-09-23T03:15:16.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.981+0000 7fef9b832640 1 -- 172.21.15.154:0/4162776581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef940acce0 msgr2=0x7fef9415ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:16.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.981+0000 7fef9b832640 1 --2- 172.21.15.154:0/4162776581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef940acce0 0x7fef9415ed60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fef90009850 tx=0x7fef9002b8d0 comp rx=0 tx=0).stop 2023-09-23T03:15:16.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.982+0000 7fef9b832640 1 -- 172.21.15.154:0/4162776581 shutdown_connections 2023-09-23T03:15:16.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.982+0000 7fef9b832640 1 --2- 172.21.15.154:0/4162776581 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef940b1b20 0x7fef940ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:16.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.982+0000 7fef9b832640 1 --2- 172.21.15.154:0/4162776581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef940acce0 0x7fef9415ed60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:16.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.982+0000 7fef9b832640 1 -- 172.21.15.154:0/4162776581 >> 172.21.15.154:0/4162776581 conn(0x7fef9406a0f0 msgr2=0x7fef9406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:16.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.982+0000 7fef9b832640 1 -- 172.21.15.154:0/4162776581 shutdown_connections 2023-09-23T03:15:16.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.982+0000 7fef9b832640 1 -- 172.21.15.154:0/4162776581 wait complete. 2023-09-23T03:15:16.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.983+0000 7fef9b832640 1 Processor -- start 2023-09-23T03:15:16.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.984+0000 7fef9b832640 1 -- start start 2023-09-23T03:15:16.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.984+0000 7fef9b832640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef940acce0 0x7fef940c6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:16.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.984+0000 7fef9b832640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef940b1b20 0x7fef940c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:16.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.985+0000 7fef9b832640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef940cee70 0x7fef940c7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:16.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.985+0000 7fef9b832640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef940c7f10 con 0x7fef940acce0 2023-09-23T03:15:16.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.985+0000 7fef9b832640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef94097f50 con 0x7fef940b1b20 2023-09-23T03:15:16.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.985+0000 7fef9b832640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef9415b370 con 0x7fef940cee70 2023-09-23T03:15:16.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.985+0000 7fef995ce640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef940acce0 0x7fef940c6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:16.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.985+0000 7fef995ce640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef940acce0 0x7fef940c6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45814/0 (socket says 172.21.15.154:45814) 2023-09-23T03:15:16.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.985+0000 7fef995ce640 1 -- 172.21.15.154:0/908829218 learned_addr learned my addr 172.21.15.154:0/908829218 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:16.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.985+0000 7fef98dcd640 1 --2- 172.21.15.154:0/908829218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef940b1b20 0x7fef940c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:16.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.985+0000 7fef99dcf640 1 --2- 172.21.15.154:0/908829218 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef940cee70 0x7fef940c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:16.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.985+0000 7fef995ce640 1 -- 172.21.15.154:0/908829218 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef940cee70 msgr2=0x7fef940c7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:16.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.985+0000 7fef995ce640 1 --2- 172.21.15.154:0/908829218 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef940cee70 0x7fef940c7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:16.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.985+0000 7fef995ce640 1 -- 172.21.15.154:0/908829218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef940b1b20 msgr2=0x7fef940c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:16.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.985+0000 7fef995ce640 1 --2- 172.21.15.154:0/908829218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef940b1b20 0x7fef940c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:16.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.985+0000 7fef995ce640 1 -- 172.21.15.154:0/908829218 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef90009460 con 0x7fef940acce0 2023-09-23T03:15:16.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.986+0000 7fef995ce640 1 --2- 172.21.15.154:0/908829218 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef940acce0 0x7fef940c6890 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fef90009970 tx=0x7fef900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:16.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.986+0000 7fef89ffb640 1 -- 172.21.15.154:0/908829218 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef90031630 con 0x7fef940acce0 2023-09-23T03:15:16.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.986+0000 7fef89ffb640 1 -- 172.21.15.154:0/908829218 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef9003f070 con 0x7fef940acce0 2023-09-23T03:15:16.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.986+0000 7fef89ffb640 1 -- 172.21.15.154:0/908829218 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef900319b0 con 0x7fef940acce0 2023-09-23T03:15:16.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.986+0000 7fef9b832640 1 -- 172.21.15.154:0/908829218 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef9414c970 con 0x7fef940acce0 2023-09-23T03:15:16.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.986+0000 7fef9b832640 1 -- 172.21.15.154:0/908829218 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef940c78b0 con 0x7fef940acce0 2023-09-23T03:15:16.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.987+0000 7fef9b832640 1 -- 172.21.15.154:0/908829218 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef940c8430 con 0x7fef940acce0 2023-09-23T03:15:16.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.988+0000 7fef89ffb640 1 -- 172.21.15.154:0/908829218 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fef90041030 con 0x7fef940acce0 2023-09-23T03:15:16.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.989+0000 7fef89ffb640 1 --2- 172.21.15.154:0/908829218 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fef7806af30 0x7fef7806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:16.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.989+0000 7fef89ffb640 1 -- 172.21.15.154:0/908829218 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fef900ab5b0 con 0x7fef940acce0 2023-09-23T03:15:16.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.989+0000 7fef98dcd640 1 --2- 172.21.15.154:0/908829218 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fef7806af30 0x7fef7806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:16.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.990+0000 7fef98dcd640 1 --2- 172.21.15.154:0/908829218 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fef7806af30 0x7fef7806d3f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fef940b0ad0 tx=0x7fef84008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:17.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:16.993+0000 7fef89ffb640 1 -- 172.21.15.154:0/908829218 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fef900768a0 con 0x7fef940acce0 2023-09-23T03:15:17.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.184+0000 7fef9b832640 1 -- 172.21.15.154:0/908829218 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef94072d00 con 0x7fef940acce0 2023-09-23T03:15:17.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.185+0000 7fef89ffb640 1 -- 172.21.15.154:0/908829218 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12886 (secure 0 0 0) 0x7fef90079ec0 con 0x7fef940acce0 2023-09-23T03:15:17.186 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:15:17.186 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":16,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:15:11.498904+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:15:17.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.187+0000 7fef9b832640 1 -- 172.21.15.154:0/908829218 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fef7806af30 msgr2=0x7fef7806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:17.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.187+0000 7fef9b832640 1 --2- 172.21.15.154:0/908829218 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fef7806af30 0x7fef7806d3f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fef940b0ad0 tx=0x7fef84008040 comp rx=0 tx=0).stop 2023-09-23T03:15:17.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.187+0000 7fef9b832640 1 -- 172.21.15.154:0/908829218 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef940acce0 msgr2=0x7fef940c6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:17.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.187+0000 7fef9b832640 1 --2- 172.21.15.154:0/908829218 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef940acce0 0x7fef940c6890 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fef90009970 tx=0x7fef900046c0 comp rx=0 tx=0).stop 2023-09-23T03:15:17.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.187+0000 7fef9b832640 1 -- 172.21.15.154:0/908829218 shutdown_connections 2023-09-23T03:15:17.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.187+0000 7fef9b832640 1 --2- 172.21.15.154:0/908829218 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fef7806af30 0x7fef7806d3f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:17.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.187+0000 7fef9b832640 1 --2- 172.21.15.154:0/908829218 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef940cee70 0x7fef940c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:17.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.187+0000 7fef9b832640 1 --2- 172.21.15.154:0/908829218 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef940b1b20 0x7fef940c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:17.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.187+0000 7fef9b832640 1 --2- 172.21.15.154:0/908829218 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef940acce0 0x7fef940c6890 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:17.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.187+0000 7fef9b832640 1 -- 172.21.15.154:0/908829218 >> 172.21.15.154:0/908829218 conn(0x7fef9406a0f0 msgr2=0x7fef940cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:17.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.188+0000 7fef9b832640 1 -- 172.21.15.154:0/908829218 shutdown_connections 2023-09-23T03:15:17.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.188+0000 7fef9b832640 1 -- 172.21.15.154:0/908829218 wait complete. 2023-09-23T03:15:17.207 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T03:15:17.208 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:15:17.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.414+0000 7fe6b6b4c640 1 Processor -- start 2023-09-23T03:15:17.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.415+0000 7fe6b6b4c640 1 -- start start 2023-09-23T03:15:17.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.416+0000 7fe6b6b4c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe6b015bb30 0x7fe6b00caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:17.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.416+0000 7fe6b6b4c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe6b00cb3e0 0x7fe6b00cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:17.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.416+0000 7fe6b6b4c640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b015b6a0 con 0x7fe6b015aba0 2023-09-23T03:15:17.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.416+0000 7fe6b6b4c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6b00cb870 con 0x7fe6b015bb30 2023-09-23T03:15:17.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.416+0000 7fe6b6b4c640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6b015bf70 con 0x7fe6b00cb3e0 2023-09-23T03:15:17.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.416+0000 7fe6affff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe6b015bb30 0x7fe6b00caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:17.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.417+0000 7fe6affff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe6b015bb30 0x7fe6b00caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45822/0 (socket says 172.21.15.154:45822) 2023-09-23T03:15:17.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.417+0000 7fe6affff640 1 -- 172.21.15.154:0/3991001401 learned_addr learned my addr 172.21.15.154:0/3991001401 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:17.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.417+0000 7fe6affff640 1 -- 172.21.15.154:0/3991001401 >> v2:172.21.15.190:3300/0 conn(0x7fe6b00cb3e0 msgr2=0x7fe6b00cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:17.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.417+0000 7fe6b50e9640 1 --2- 172.21.15.154:0/3991001401 >> v2:172.21.15.190:3300/0 conn(0x7fe6b00cb3e0 0x7fe6b00cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:17.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.417+0000 7fe6affff640 1 --2- 172.21.15.154:0/3991001401 >> v2:172.21.15.190:3300/0 conn(0x7fe6b00cb3e0 0x7fe6b00cc5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:17.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.417+0000 7fe6affff640 1 -- 172.21.15.154:0/3991001401 >> v1:172.21.15.190:6789/0 conn(0x7fe6b015aba0 legacy=0x7fe6b015e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:17.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.417+0000 7fe6affff640 1 -- 172.21.15.154:0/3991001401 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b015f0b0 con 0x7fe6b015bb30 2023-09-23T03:15:17.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.418+0000 7fe6affff640 1 --2- 172.21.15.154:0/3991001401 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6b015bb30 0x7fe6b00caea0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe6b00c1f80 tx=0x7fe69c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa990fb5a36f4fa server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:17.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.418+0000 7fe6af7fe640 1 -- 172.21.15.154:0/3991001401 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c031870 con 0x7fe6b015bb30 2023-09-23T03:15:17.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.418+0000 7fe6af7fe640 1 -- 172.21.15.154:0/3991001401 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69c0319d0 con 0x7fe6b015bb30 2023-09-23T03:15:17.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.418+0000 7fe6af7fe640 1 -- 172.21.15.154:0/3991001401 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c031d60 con 0x7fe6b015bb30 2023-09-23T03:15:17.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.419+0000 7fe6b6b4c640 1 -- 172.21.15.154:0/3991001401 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6b015bb30 msgr2=0x7fe6b00caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:17.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.419+0000 7fe6b6b4c640 1 --2- 172.21.15.154:0/3991001401 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6b015bb30 0x7fe6b00caea0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe6b00c1f80 tx=0x7fe69c02b910 comp rx=0 tx=0).stop 2023-09-23T03:15:17.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.419+0000 7fe6b6b4c640 1 -- 172.21.15.154:0/3991001401 shutdown_connections 2023-09-23T03:15:17.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.419+0000 7fe6b6b4c640 1 --2- 172.21.15.154:0/3991001401 >> v2:172.21.15.190:3300/0 conn(0x7fe6b00cb3e0 0x7fe6b00cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:17.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.419+0000 7fe6b6b4c640 1 --2- 172.21.15.154:0/3991001401 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6b015bb30 0x7fe6b00caea0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:17.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.419+0000 7fe6b6b4c640 1 -- 172.21.15.154:0/3991001401 >> 172.21.15.154:0/3991001401 conn(0x7fe6b006c010 msgr2=0x7fe6b006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:17.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.420+0000 7fe6b6b4c640 1 -- 172.21.15.154:0/3991001401 shutdown_connections 2023-09-23T03:15:17.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.420+0000 7fe6b6b4c640 1 -- 172.21.15.154:0/3991001401 wait complete. 2023-09-23T03:15:17.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.422+0000 7fe6b6b4c640 1 Processor -- start 2023-09-23T03:15:17.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.422+0000 7fe6b6b4c640 1 -- start start 2023-09-23T03:15:17.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.423+0000 7fe6b6b4c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6b00cb3e0 0x7fe6b0150a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:17.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.423+0000 7fe6b6b4c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6b015aba0 0x7fe6b00c6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:17.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.423+0000 7fe6b6b4c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6b015bb30 0x7fe6b00ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:17.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.423+0000 7fe6b6b4c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b0157ee0 con 0x7fe6b015bb30 2023-09-23T03:15:17.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.423+0000 7fe6b6b4c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b01572d0 con 0x7fe6b015aba0 2023-09-23T03:15:17.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.423+0000 7fe6b6b4c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b01575b0 con 0x7fe6b00cb3e0 2023-09-23T03:15:17.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.423+0000 7fe6b50e9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6b015bb30 0x7fe6b00ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:17.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.423+0000 7fe6b50e9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6b015bb30 0x7fe6b00ab8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45824/0 (socket says 172.21.15.154:45824) 2023-09-23T03:15:17.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.423+0000 7fe6b50e9640 1 -- 172.21.15.154:0/3980001249 learned_addr learned my addr 172.21.15.154:0/3980001249 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:17.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.424+0000 7fe6b48e8640 1 --2- 172.21.15.154:0/3980001249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6b00cb3e0 0x7fe6b0150a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:17.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.424+0000 7fe6affff640 1 --2- 172.21.15.154:0/3980001249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6b015aba0 0x7fe6b00c6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:17.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.424+0000 7fe6b50e9640 1 -- 172.21.15.154:0/3980001249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6b00cb3e0 msgr2=0x7fe6b0150a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:17.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.424+0000 7fe6b50e9640 1 --2- 172.21.15.154:0/3980001249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6b00cb3e0 0x7fe6b0150a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:17.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.424+0000 7fe6b50e9640 1 -- 172.21.15.154:0/3980001249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6b015aba0 msgr2=0x7fe6b00c6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:17.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.424+0000 7fe6b50e9640 1 --2- 172.21.15.154:0/3980001249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6b015aba0 0x7fe6b00c6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:17.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.424+0000 7fe6b50e9640 1 -- 172.21.15.154:0/3980001249 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe69c0094d0 con 0x7fe6b015bb30 2023-09-23T03:15:17.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.424+0000 7fe6b50e9640 1 --2- 172.21.15.154:0/3980001249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6b015bb30 0x7fe6b00ab8c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe6a800cc30 tx=0x7fe6a800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:17.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.424+0000 7fe6ad7fa640 1 -- 172.21.15.154:0/3980001249 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a800ce40 con 0x7fe6b015bb30 2023-09-23T03:15:17.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.424+0000 7fe6ad7fa640 1 -- 172.21.15.154:0/3980001249 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a8003990 con 0x7fe6b015bb30 2023-09-23T03:15:17.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.424+0000 7fe6ad7fa640 1 -- 172.21.15.154:0/3980001249 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a8003d20 con 0x7fe6b015bb30 2023-09-23T03:15:17.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.424+0000 7fe6b6b4c640 1 -- 172.21.15.154:0/3980001249 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b0157890 con 0x7fe6b015bb30 2023-09-23T03:15:17.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.425+0000 7fe6b6b4c640 1 -- 172.21.15.154:0/3980001249 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b0150fd0 con 0x7fe6b015bb30 2023-09-23T03:15:17.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.426+0000 7fe6b6b4c640 1 -- 172.21.15.154:0/3980001249 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6b0158050 con 0x7fe6b015bb30 2023-09-23T03:15:17.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.427+0000 7fe6ad7fa640 1 -- 172.21.15.154:0/3980001249 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fe6a8003e80 con 0x7fe6b015bb30 2023-09-23T03:15:17.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.428+0000 7fe6ad7fa640 1 --2- 172.21.15.154:0/3980001249 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe69406af30 0x7fe69406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:17.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.428+0000 7fe6ad7fa640 1 -- 172.21.15.154:0/3980001249 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe6a8089890 con 0x7fe6b015bb30 2023-09-23T03:15:17.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.428+0000 7fe6b48e8640 1 --2- 172.21.15.154:0/3980001249 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe69406af30 0x7fe69406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:17.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.429+0000 7fe6b48e8640 1 --2- 172.21.15.154:0/3980001249 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe69406af30 0x7fe69406d3f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4007fd0 tx=0x7fe6a4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:17.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.433+0000 7fe6ad7fa640 1 -- 172.21.15.154:0/3980001249 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fe6a8054b80 con 0x7fe6b015bb30 2023-09-23T03:15:17.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.620+0000 7fe6b6b4c640 1 -- 172.21.15.154:0/3980001249 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe6b004e920 con 0x7fe6b015bb30 2023-09-23T03:15:17.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.620+0000 7fe6ad7fa640 1 -- 172.21.15.154:0/3980001249 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe6a801d070 con 0x7fe6b015bb30 2023-09-23T03:15:17.622 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:15:17.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.623+0000 7fe6b6b4c640 1 -- 172.21.15.154:0/3980001249 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe69406af30 msgr2=0x7fe69406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:17.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.623+0000 7fe6b6b4c640 1 --2- 172.21.15.154:0/3980001249 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe69406af30 0x7fe69406d3f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4007fd0 tx=0x7fe6a4009040 comp rx=0 tx=0).stop 2023-09-23T03:15:17.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.623+0000 7fe6b6b4c640 1 -- 172.21.15.154:0/3980001249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6b015bb30 msgr2=0x7fe6b00ab8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:17.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.623+0000 7fe6b6b4c640 1 --2- 172.21.15.154:0/3980001249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6b015bb30 0x7fe6b00ab8c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe6a800cc30 tx=0x7fe6a800a5c0 comp rx=0 tx=0).stop 2023-09-23T03:15:17.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.623+0000 7fe6b6b4c640 1 -- 172.21.15.154:0/3980001249 shutdown_connections 2023-09-23T03:15:17.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.623+0000 7fe6b6b4c640 1 --2- 172.21.15.154:0/3980001249 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe69406af30 0x7fe69406d3f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:17.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.623+0000 7fe6b6b4c640 1 --2- 172.21.15.154:0/3980001249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe6b015bb30 0x7fe6b00ab8c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:17.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.623+0000 7fe6b6b4c640 1 --2- 172.21.15.154:0/3980001249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe6b015aba0 0x7fe6b00c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:17.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.623+0000 7fe6b6b4c640 1 --2- 172.21.15.154:0/3980001249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe6b00cb3e0 0x7fe6b0150a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:17.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.623+0000 7fe6b6b4c640 1 -- 172.21.15.154:0/3980001249 >> 172.21.15.154:0/3980001249 conn(0x7fe6b006c010 msgr2=0x7fe6b015e360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:17.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.624+0000 7fe6b6b4c640 1 -- 172.21.15.154:0/3980001249 shutdown_connections 2023-09-23T03:15:17.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.624+0000 7fe6b6b4c640 1 -- 172.21.15.154:0/3980001249 wait complete. 2023-09-23T03:15:17.643 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-09-23T03:15:17.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.857+0000 7fd8e50da640 1 Processor -- start 2023-09-23T03:15:17.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.857+0000 7fd8e50da640 1 -- start start 2023-09-23T03:15:17.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.858+0000 7fd8e50da640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd8e015cad0 0x7fd8e00ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:17.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.858+0000 7fd8e50da640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd8e006c7c0 0x7fd8e00b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:17.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.858+0000 7fd8e50da640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8e00b2af0 con 0x7fd8e00b26f0 2023-09-23T03:15:17.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.858+0000 7fd8e50da640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8e00cad80 con 0x7fd8e006c7c0 2023-09-23T03:15:17.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.858+0000 7fd8e50da640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8e006ccc0 con 0x7fd8e015cad0 2023-09-23T03:15:17.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.859+0000 7fd8de59c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd8e006c7c0 0x7fd8e00b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:17.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.859+0000 7fd8de59c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd8e006c7c0 0x7fd8e00b21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45834/0 (socket says 172.21.15.154:45834) 2023-09-23T03:15:17.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.859+0000 7fd8de59c640 1 -- 172.21.15.154:0/1932071717 learned_addr learned my addr 172.21.15.154:0/1932071717 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:17.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.859+0000 7fd8de59c640 1 -- 172.21.15.154:0/1932071717 >> v2:172.21.15.190:3300/0 conn(0x7fd8e015cad0 msgr2=0x7fd8e00ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:17.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.859+0000 7fd8de59c640 1 --2- 172.21.15.154:0/1932071717 >> v2:172.21.15.190:3300/0 conn(0x7fd8e015cad0 0x7fd8e00ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:17.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.859+0000 7fd8de59c640 1 -- 172.21.15.154:0/1932071717 >> v1:172.21.15.190:6789/0 conn(0x7fd8e00b26f0 legacy=0x7fd8e006fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:17.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.859+0000 7fd8de59c640 1 -- 172.21.15.154:0/1932071717 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e006b2b0 con 0x7fd8e006c7c0 2023-09-23T03:15:17.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.860+0000 7fd8de59c640 1 --2- 172.21.15.154:0/1932071717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8e006c7c0 0x7fd8e00b21b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd8e006aba0 tx=0x7fd8c802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=177b77a1f7e8d491 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:17.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.860+0000 7fd8ddd9b640 1 -- 172.21.15.154:0/1932071717 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c8031870 con 0x7fd8e006c7c0 2023-09-23T03:15:17.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.861+0000 7fd8ddd9b640 1 -- 172.21.15.154:0/1932071717 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c80319d0 con 0x7fd8e006c7c0 2023-09-23T03:15:17.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.861+0000 7fd8ddd9b640 1 -- 172.21.15.154:0/1932071717 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c8031d60 con 0x7fd8e006c7c0 2023-09-23T03:15:17.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.861+0000 7fd8e50da640 1 -- 172.21.15.154:0/1932071717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8e006c7c0 msgr2=0x7fd8e00b21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:17.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.861+0000 7fd8e50da640 1 --2- 172.21.15.154:0/1932071717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8e006c7c0 0x7fd8e00b21b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd8e006aba0 tx=0x7fd8c802b910 comp rx=0 tx=0).stop 2023-09-23T03:15:17.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.861+0000 7fd8e50da640 1 -- 172.21.15.154:0/1932071717 shutdown_connections 2023-09-23T03:15:17.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.861+0000 7fd8e50da640 1 --2- 172.21.15.154:0/1932071717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8e006c7c0 0x7fd8e00b21b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:17.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.861+0000 7fd8e50da640 1 --2- 172.21.15.154:0/1932071717 >> v2:172.21.15.190:3300/0 conn(0x7fd8e015cad0 0x7fd8e00ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:17.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.861+0000 7fd8e50da640 1 -- 172.21.15.154:0/1932071717 >> 172.21.15.154:0/1932071717 conn(0x7fd8e00cd1b0 msgr2=0x7fd8e00cd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:17.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.861+0000 7fd8e50da640 1 -- 172.21.15.154:0/1932071717 shutdown_connections 2023-09-23T03:15:17.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.861+0000 7fd8e50da640 1 -- 172.21.15.154:0/1932071717 wait complete. 2023-09-23T03:15:17.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.863+0000 7fd8e50da640 1 Processor -- start 2023-09-23T03:15:17.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.863+0000 7fd8e50da640 1 -- start start 2023-09-23T03:15:17.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.864+0000 7fd8e50da640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8e006c7c0 0x7fd8e00c69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:17.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.864+0000 7fd8e50da640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8e00b26f0 0x7fd8e015e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:17.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.864+0000 7fd8e50da640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8e015cad0 0x7fd8e015ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:17.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.864+0000 7fd8e50da640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e015e910 con 0x7fd8e00b26f0 2023-09-23T03:15:17.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.864+0000 7fd8e50da640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e015ebe0 con 0x7fd8e006c7c0 2023-09-23T03:15:17.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.864+0000 7fd8e50da640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e015eec0 con 0x7fd8e015cad0 2023-09-23T03:15:17.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.864+0000 7fd8de59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8e00b26f0 0x7fd8e015e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:17.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.864+0000 7fd8de59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8e00b26f0 0x7fd8e015e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45836/0 (socket says 172.21.15.154:45836) 2023-09-23T03:15:17.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.864+0000 7fd8de59c640 1 -- 172.21.15.154:0/1919026241 learned_addr learned my addr 172.21.15.154:0/1919026241 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:17.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.865+0000 7fd8de59c640 1 -- 172.21.15.154:0/1919026241 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8e015cad0 msgr2=0x7fd8e015ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:17.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.865+0000 7fd8df59e640 1 --2- 172.21.15.154:0/1919026241 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8e015cad0 0x7fd8e015ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:17.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.865+0000 7fd8ded9d640 1 --2- 172.21.15.154:0/1919026241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8e006c7c0 0x7fd8e00c69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:17.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.865+0000 7fd8de59c640 1 --2- 172.21.15.154:0/1919026241 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8e015cad0 0x7fd8e015ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:17.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.865+0000 7fd8de59c640 1 -- 172.21.15.154:0/1919026241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8e006c7c0 msgr2=0x7fd8e00c69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:17.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.865+0000 7fd8de59c640 1 --2- 172.21.15.154:0/1919026241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8e006c7c0 0x7fd8e00c69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:17.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.865+0000 7fd8de59c640 1 -- 172.21.15.154:0/1919026241 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c80094d0 con 0x7fd8e00b26f0 2023-09-23T03:15:17.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.865+0000 7fd8df59e640 1 --2- 172.21.15.154:0/1919026241 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8e015cad0 0x7fd8e015ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:17.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.865+0000 7fd8de59c640 1 --2- 172.21.15.154:0/1919026241 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8e00b26f0 0x7fd8e015e3d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8000c00 tx=0x7fd8c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:17.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.866+0000 7fd8d77fe640 1 -- 172.21.15.154:0/1919026241 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c803ad10 con 0x7fd8e00b26f0 2023-09-23T03:15:17.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.866+0000 7fd8d77fe640 1 -- 172.21.15.154:0/1919026241 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c803ae70 con 0x7fd8e00b26f0 2023-09-23T03:15:17.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.866+0000 7fd8d77fe640 1 -- 172.21.15.154:0/1919026241 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c8044610 con 0x7fd8e00b26f0 2023-09-23T03:15:17.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.866+0000 7fd8e50da640 1 -- 172.21.15.154:0/1919026241 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8e015f1a0 con 0x7fd8e00b26f0 2023-09-23T03:15:17.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.866+0000 7fd8e50da640 1 -- 172.21.15.154:0/1919026241 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8e015f760 con 0x7fd8e00b26f0 2023-09-23T03:15:17.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.867+0000 7fd8e50da640 1 -- 172.21.15.154:0/1919026241 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8e006b850 con 0x7fd8e00b26f0 2023-09-23T03:15:17.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.868+0000 7fd8d77fe640 1 -- 172.21.15.154:0/1919026241 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fd8c804d450 con 0x7fd8e00b26f0 2023-09-23T03:15:17.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.869+0000 7fd8d77fe640 1 --2- 172.21.15.154:0/1919026241 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd8c006af30 0x7fd8c006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:17.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.870+0000 7fd8d77fe640 1 -- 172.21.15.154:0/1919026241 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd8c802d040 con 0x7fd8e00b26f0 2023-09-23T03:15:17.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.870+0000 7fd8ded9d640 1 --2- 172.21.15.154:0/1919026241 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd8c006af30 0x7fd8c006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:17.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.870+0000 7fd8ded9d640 1 --2- 172.21.15.154:0/1919026241 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd8c006af30 0x7fd8c006d3f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0007dc0 tx=0x7fd8d00072f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:17.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:17.874+0000 7fd8d77fe640 1 -- 172.21.15.154:0/1919026241 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fd8e006b850 con 0x7fd8e00b26f0 2023-09-23T03:15:18.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.067+0000 7fd8e50da640 1 -- 172.21.15.154:0/1919026241 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd8e006b850 con 0x7fd8e00b26f0 2023-09-23T03:15:18.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.068+0000 7fd8d77fe640 1 -- 172.21.15.154:0/1919026241 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd8e006b850 con 0x7fd8e00b26f0 2023-09-23T03:15:18.069 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:15:18.070 INFO:teuthology.orchestra.run.smithi154.stdout:[ 2023-09-23T03:15:18.070 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:15:18.070 INFO:teuthology.orchestra.run.smithi154.stdout: "poolnum": 1, 2023-09-23T03:15:18.070 INFO:teuthology.orchestra.run.smithi154.stdout: "poolname": ".mgr" 2023-09-23T03:15:18.070 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:15:18.070 INFO:teuthology.orchestra.run.smithi154.stdout:] 2023-09-23T03:15:18.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.071+0000 7fd8e50da640 1 -- 172.21.15.154:0/1919026241 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd8c006af30 msgr2=0x7fd8c006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:18.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.071+0000 7fd8e50da640 1 --2- 172.21.15.154:0/1919026241 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd8c006af30 0x7fd8c006d3f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0007dc0 tx=0x7fd8d00072f0 comp rx=0 tx=0).stop 2023-09-23T03:15:18.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.071+0000 7fd8e50da640 1 -- 172.21.15.154:0/1919026241 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8e00b26f0 msgr2=0x7fd8e015e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:18.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.071+0000 7fd8e50da640 1 --2- 172.21.15.154:0/1919026241 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8e00b26f0 0x7fd8e015e3d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8000c00 tx=0x7fd8c80046c0 comp rx=0 tx=0).stop 2023-09-23T03:15:18.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.071+0000 7fd8e50da640 1 -- 172.21.15.154:0/1919026241 shutdown_connections 2023-09-23T03:15:18.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.071+0000 7fd8e50da640 1 --2- 172.21.15.154:0/1919026241 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd8c006af30 0x7fd8c006d3f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:18.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.071+0000 7fd8e50da640 1 --2- 172.21.15.154:0/1919026241 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8e015cad0 0x7fd8e015ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:18.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.071+0000 7fd8e50da640 1 --2- 172.21.15.154:0/1919026241 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8e00b26f0 0x7fd8e015e3d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:18.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.071+0000 7fd8e50da640 1 --2- 172.21.15.154:0/1919026241 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8e006c7c0 0x7fd8e00c69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:18.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.071+0000 7fd8e50da640 1 -- 172.21.15.154:0/1919026241 >> 172.21.15.154:0/1919026241 conn(0x7fd8e00cd1b0 msgr2=0x7fd8e00b0190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:18.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.071+0000 7fd8e50da640 1 -- 172.21.15.154:0/1919026241 shutdown_connections 2023-09-23T03:15:18.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.071+0000 7fd8e50da640 1 -- 172.21.15.154:0/1919026241 wait complete. 2023-09-23T03:15:18.091 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-09-23T03:15:18.091 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-09-23T03:15:18.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.311+0000 7f0a4c47c640 1 Processor -- start 2023-09-23T03:15:18.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.312+0000 7f0a4c47c640 1 -- start start 2023-09-23T03:15:18.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.313+0000 7f0a4c47c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0a440d1520 0x7f0a44167a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:18.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.313+0000 7f0a4c47c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a440cd3f0 con 0x7f0a4415c940 2023-09-23T03:15:18.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.313+0000 7f0a4c47c640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a440d3470 con 0x7f0a4415cd20 2023-09-23T03:15:18.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.313+0000 7f0a4c47c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a440cd690 con 0x7f0a440d1520 2023-09-23T03:15:18.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.313+0000 7f0a4a218640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f0a4415c940 0x7f0a440d0420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:59074/0 (socket says 172.21.15.154:59074) 2023-09-23T03:15:18.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.313+0000 7f0a4a218640 1 -- 172.21.15.154:0/2685889088 learned_addr learned my addr 172.21.15.154:0/2685889088 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:18.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.314+0000 7f0a4aa19640 1 --2- 172.21.15.154:0/2685889088 >> v2:172.21.15.154:3300/0 conn(0x7f0a440d1520 0x7f0a44167a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:18.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.314+0000 7f0a49216640 1 -- 172.21.15.154:0/2685889088 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4001191673 0 0) 0x7f0a440cd3f0 con 0x7f0a4415c940 2023-09-23T03:15:18.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.314+0000 7f0a49216640 1 -- 172.21.15.154:0/2685889088 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a30003520 con 0x7f0a4415c940 2023-09-23T03:15:18.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.314+0000 7f0a4aa19640 1 -- 172.21.15.154:0/2685889088 >> v1:172.21.15.190:6789/0 conn(0x7f0a4415cd20 legacy=0x7f0a440d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:18.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.314+0000 7f0a4aa19640 1 -- 172.21.15.154:0/2685889088 >> v1:172.21.15.154:6789/0 conn(0x7f0a4415c940 legacy=0x7f0a440d0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:18.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.314+0000 7f0a4aa19640 1 -- 172.21.15.154:0/2685889088 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a440cd970 con 0x7f0a440d1520 2023-09-23T03:15:18.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.315+0000 7f0a4aa19640 1 --2- 172.21.15.154:0/2685889088 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a440d1520 0x7f0a44167a00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c009dc0 tx=0x7f0a3c02bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ebf834f63b333c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:18.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.316+0000 7f0a49216640 1 -- 172.21.15.154:0/2685889088 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c02cce0 con 0x7f0a440d1520 2023-09-23T03:15:18.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.316+0000 7f0a49216640 1 -- 172.21.15.154:0/2685889088 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a3c02ce40 con 0x7f0a440d1520 2023-09-23T03:15:18.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.316+0000 7f0a49216640 1 -- 172.21.15.154:0/2685889088 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c034660 con 0x7f0a440d1520 2023-09-23T03:15:18.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.316+0000 7f0a4c47c640 1 -- 172.21.15.154:0/2685889088 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a440d1520 msgr2=0x7f0a44167a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:18.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.316+0000 7f0a4c47c640 1 --2- 172.21.15.154:0/2685889088 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a440d1520 0x7f0a44167a00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c009dc0 tx=0x7f0a3c02bf30 comp rx=0 tx=0).stop 2023-09-23T03:15:18.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.316+0000 7f0a4c47c640 1 -- 172.21.15.154:0/2685889088 shutdown_connections 2023-09-23T03:15:18.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.316+0000 7f0a4c47c640 1 --2- 172.21.15.154:0/2685889088 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a440d1520 0x7f0a44167a00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:18.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.316+0000 7f0a4c47c640 1 -- 172.21.15.154:0/2685889088 >> 172.21.15.154:0/2685889088 conn(0x7f0a44089fd0 msgr2=0x7f0a440d2850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:18.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.316+0000 7f0a4c47c640 1 -- 172.21.15.154:0/2685889088 shutdown_connections 2023-09-23T03:15:18.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.316+0000 7f0a4c47c640 1 -- 172.21.15.154:0/2685889088 wait complete. 2023-09-23T03:15:18.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.318+0000 7f0a4c47c640 1 Processor -- start 2023-09-23T03:15:18.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.318+0000 7f0a4c47c640 1 -- start start 2023-09-23T03:15:18.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.319+0000 7f0a4c47c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a440d1520 0x7f0a441f1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:18.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.319+0000 7f0a4c47c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a4415c940 0x7f0a440b2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:18.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.319+0000 7f0a4c47c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a4415cd20 0x7f0a441f6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:18.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.319+0000 7f0a4c47c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a441f5c30 con 0x7f0a4415c940 2023-09-23T03:15:18.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.319+0000 7f0a4c47c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a440cabf0 con 0x7f0a440d1520 2023-09-23T03:15:18.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.319+0000 7f0a4c47c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a440caed0 con 0x7f0a4415cd20 2023-09-23T03:15:18.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.320+0000 7f0a49a17640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a4415c940 0x7f0a440b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:18.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.320+0000 7f0a49a17640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a4415c940 0x7f0a440b2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45854/0 (socket says 172.21.15.154:45854) 2023-09-23T03:15:18.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.320+0000 7f0a49a17640 1 -- 172.21.15.154:0/444858352 learned_addr learned my addr 172.21.15.154:0/444858352 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:18.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.320+0000 7f0a4aa19640 1 --2- 172.21.15.154:0/444858352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a4415cd20 0x7f0a441f6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:18.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.320+0000 7f0a4a218640 1 --2- 172.21.15.154:0/444858352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a440d1520 0x7f0a441f1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:18.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.320+0000 7f0a49a17640 1 -- 172.21.15.154:0/444858352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a4415cd20 msgr2=0x7f0a441f6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:18.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.320+0000 7f0a49a17640 1 --2- 172.21.15.154:0/444858352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a4415cd20 0x7f0a441f6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:18.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.320+0000 7f0a49a17640 1 -- 172.21.15.154:0/444858352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a440d1520 msgr2=0x7f0a441f1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:18.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.320+0000 7f0a49a17640 1 --2- 172.21.15.154:0/444858352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a440d1520 0x7f0a441f1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:18.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.320+0000 7f0a49a17640 1 -- 172.21.15.154:0/444858352 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3c009a70 con 0x7f0a4415c940 2023-09-23T03:15:18.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.320+0000 7f0a4aa19640 1 --2- 172.21.15.154:0/444858352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a4415cd20 0x7f0a441f6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:18.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.320+0000 7f0a49a17640 1 --2- 172.21.15.154:0/444858352 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a4415c940 0x7f0a440b2180 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400c400 tx=0x7f0a3400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:18.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.321+0000 7f0a3affd640 1 -- 172.21.15.154:0/444858352 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a340052d0 con 0x7f0a4415c940 2023-09-23T03:15:18.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.321+0000 7f0a3affd640 1 -- 172.21.15.154:0/444858352 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a34005430 con 0x7f0a4415c940 2023-09-23T03:15:18.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.321+0000 7f0a4c47c640 1 -- 172.21.15.154:0/444858352 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a441f5850 con 0x7f0a4415c940 2023-09-23T03:15:18.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.321+0000 7f0a3affd640 1 -- 172.21.15.154:0/444858352 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a34003830 con 0x7f0a4415c940 2023-09-23T03:15:18.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.321+0000 7f0a4c47c640 1 -- 172.21.15.154:0/444858352 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a440b26c0 con 0x7f0a4415c940 2023-09-23T03:15:18.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.322+0000 7f0a4c47c640 1 -- 172.21.15.154:0/444858352 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a441f5d70 con 0x7f0a4415c940 2023-09-23T03:15:18.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.323+0000 7f0a3affd640 1 -- 172.21.15.154:0/444858352 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f0a34003990 con 0x7f0a4415c940 2023-09-23T03:15:18.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.324+0000 7f0a3affd640 1 --2- 172.21.15.154:0/444858352 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0a2806af30 0x7f0a2806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:18.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.324+0000 7f0a3affd640 1 -- 172.21.15.154:0/444858352 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0a34088a30 con 0x7f0a4415c940 2023-09-23T03:15:18.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.324+0000 7f0a4a218640 1 --2- 172.21.15.154:0/444858352 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0a2806af30 0x7f0a2806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:18.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.325+0000 7f0a4a218640 1 --2- 172.21.15.154:0/444858352 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0a2806af30 0x7f0a2806d3f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0a400011e0 tx=0x7f0a40009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:18.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.328+0000 7f0a3affd640 1 -- 172.21.15.154:0/444858352 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f0a441f5d70 con 0x7f0a4415c940 2023-09-23T03:15:18.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:18.520+0000 7f0a4c47c640 1 -- 172.21.15.154:0/444858352 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f0a441f5d70 con 0x7f0a4415c940 2023-09-23T03:15:19.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.206+0000 7f0a3affd640 1 -- 172.21.15.154:0/444858352 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0a441f5d70 con 0x7f0a4415c940 2023-09-23T03:15:19.207 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_metadata' created 2023-09-23T03:15:19.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.210+0000 7f0a4c47c640 1 -- 172.21.15.154:0/444858352 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0a2806af30 msgr2=0x7f0a2806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:19.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.210+0000 7f0a4c47c640 1 --2- 172.21.15.154:0/444858352 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0a2806af30 0x7f0a2806d3f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0a400011e0 tx=0x7f0a40009040 comp rx=0 tx=0).stop 2023-09-23T03:15:19.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.210+0000 7f0a4c47c640 1 -- 172.21.15.154:0/444858352 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a4415c940 msgr2=0x7f0a440b2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:19.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.210+0000 7f0a4c47c640 1 --2- 172.21.15.154:0/444858352 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a4415c940 0x7f0a440b2180 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400c400 tx=0x7f0a3400c8c0 comp rx=0 tx=0).stop 2023-09-23T03:15:19.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.210+0000 7f0a4c47c640 1 -- 172.21.15.154:0/444858352 shutdown_connections 2023-09-23T03:15:19.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.210+0000 7f0a4c47c640 1 --2- 172.21.15.154:0/444858352 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0a2806af30 0x7f0a2806d3f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:19.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.210+0000 7f0a4c47c640 1 --2- 172.21.15.154:0/444858352 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a4415cd20 0x7f0a441f6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:19.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.210+0000 7f0a4c47c640 1 --2- 172.21.15.154:0/444858352 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a4415c940 0x7f0a440b2180 secure :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400c400 tx=0x7f0a3400c8c0 comp rx=0 tx=0).stop 2023-09-23T03:15:19.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.210+0000 7f0a4c47c640 1 --2- 172.21.15.154:0/444858352 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a440d1520 0x7f0a441f1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:19.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.210+0000 7f0a4c47c640 1 -- 172.21.15.154:0/444858352 >> 172.21.15.154:0/444858352 conn(0x7f0a44089fd0 msgr2=0x7f0a4415d210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:19.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.212+0000 7f0a4c47c640 1 -- 172.21.15.154:0/444858352 shutdown_connections 2023-09-23T03:15:19.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.213+0000 7f0a4c47c640 1 -- 172.21.15.154:0/444858352 wait complete. 2023-09-23T03:15:19.239 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-09-23T03:15:19.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.448+0000 7fc2fc990640 1 Processor -- start 2023-09-23T03:15:19.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.448+0000 7fc2fc990640 1 -- start start 2023-09-23T03:15:19.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.450+0000 7fc2fc990640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc2f80ce940 0x7fc2f81677f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:19.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.450+0000 7fc2fc990640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2f80d0bb0 con 0x7fc2f815cda0 2023-09-23T03:15:19.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.450+0000 7fc2fc990640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2f80cd450 con 0x7fc2f815c9c0 2023-09-23T03:15:19.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.450+0000 7fc2fc990640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2f80d0df0 con 0x7fc2f80ce940 2023-09-23T03:15:19.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.450+0000 7fc2f6d9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc2f80ce940 0x7fc2f81677f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:19.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.450+0000 7fc2f6d9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc2f80ce940 0x7fc2f81677f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45868/0 (socket says 172.21.15.154:45868) 2023-09-23T03:15:19.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.450+0000 7fc2f6d9d640 1 -- 172.21.15.154:0/2446875020 learned_addr learned my addr 172.21.15.154:0/2446875020 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:19.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.450+0000 7fc2f5d9b640 1 --1- 172.21.15.154:0/2446875020 >> v1:172.21.15.154:6789/0 conn(0x7fc2f815cda0 0x7fc2f80d2620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:59076/0 (socket says 172.21.15.154:59076) 2023-09-23T03:15:19.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.450+0000 7fc2f6d9d640 1 -- 172.21.15.154:0/2446875020 >> v1:172.21.15.190:6789/0 conn(0x7fc2f815c9c0 legacy=0x7fc2f80d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:19.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.450+0000 7fc2f6d9d640 1 -- 172.21.15.154:0/2446875020 >> v1:172.21.15.154:6789/0 conn(0x7fc2f815cda0 legacy=0x7fc2f80d2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:19.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.450+0000 7fc2f6d9d640 1 -- 172.21.15.154:0/2446875020 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f80d10d0 con 0x7fc2f80ce940 2023-09-23T03:15:19.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.452+0000 7fc2f6d9d640 1 --2- 172.21.15.154:0/2446875020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2f80ce940 0x7fc2f81677f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0009870 tx=0x7fc2f002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f6764203c3a3dc server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:19.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.452+0000 7fc2f559a640 1 -- 172.21.15.154:0/2446875020 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0031870 con 0x7fc2f80ce940 2023-09-23T03:15:19.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.452+0000 7fc2f559a640 1 -- 172.21.15.154:0/2446875020 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f00319d0 con 0x7fc2f80ce940 2023-09-23T03:15:19.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.452+0000 7fc2f559a640 1 -- 172.21.15.154:0/2446875020 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0031d60 con 0x7fc2f80ce940 2023-09-23T03:15:19.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.452+0000 7fc2fc990640 1 -- 172.21.15.154:0/2446875020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2f80ce940 msgr2=0x7fc2f81677f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:19.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.452+0000 7fc2fc990640 1 --2- 172.21.15.154:0/2446875020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2f80ce940 0x7fc2f81677f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0009870 tx=0x7fc2f002b910 comp rx=0 tx=0).stop 2023-09-23T03:15:19.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.453+0000 7fc2fc990640 1 -- 172.21.15.154:0/2446875020 shutdown_connections 2023-09-23T03:15:19.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.453+0000 7fc2fc990640 1 --2- 172.21.15.154:0/2446875020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2f80ce940 0x7fc2f81677f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:19.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.453+0000 7fc2fc990640 1 -- 172.21.15.154:0/2446875020 >> 172.21.15.154:0/2446875020 conn(0x7fc2f809bc20 msgr2=0x7fc2f8089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:19.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.453+0000 7fc2fc990640 1 -- 172.21.15.154:0/2446875020 shutdown_connections 2023-09-23T03:15:19.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.453+0000 7fc2fc990640 1 -- 172.21.15.154:0/2446875020 wait complete. 2023-09-23T03:15:19.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.455+0000 7fc2fc990640 1 Processor -- start 2023-09-23T03:15:19.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.455+0000 7fc2fc990640 1 -- start start 2023-09-23T03:15:19.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.456+0000 7fc2fc990640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2f80ce940 0x7fc2f80c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:19.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.456+0000 7fc2fc990640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2f815c9c0 0x7fc2f815db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:19.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.456+0000 7fc2fc990640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2f815cda0 0x7fc2f81526f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:19.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.456+0000 7fc2fc990640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f80d1b40 con 0x7fc2f815c9c0 2023-09-23T03:15:19.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.456+0000 7fc2fc990640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f80c8b80 con 0x7fc2f815cda0 2023-09-23T03:15:19.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.456+0000 7fc2fc990640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f80c8e60 con 0x7fc2f80ce940 2023-09-23T03:15:19.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.457+0000 7fc2f5d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2f815c9c0 0x7fc2f815db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:19.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.457+0000 7fc2f5d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2f815c9c0 0x7fc2f815db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45878/0 (socket says 172.21.15.154:45878) 2023-09-23T03:15:19.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.457+0000 7fc2f5d9b640 1 -- 172.21.15.154:0/648594075 learned_addr learned my addr 172.21.15.154:0/648594075 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:19.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.457+0000 7fc2f659c640 1 --2- 172.21.15.154:0/648594075 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2f80ce940 0x7fc2f80c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:19.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.457+0000 7fc2f6d9d640 1 --2- 172.21.15.154:0/648594075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2f815cda0 0x7fc2f81526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:19.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.457+0000 7fc2f5d9b640 1 -- 172.21.15.154:0/648594075 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2f80ce940 msgr2=0x7fc2f80c84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:19.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.457+0000 7fc2f5d9b640 1 --2- 172.21.15.154:0/648594075 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2f80ce940 0x7fc2f80c84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:19.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.457+0000 7fc2f5d9b640 1 -- 172.21.15.154:0/648594075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2f815cda0 msgr2=0x7fc2f81526f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:19.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.457+0000 7fc2f5d9b640 1 --2- 172.21.15.154:0/648594075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2f815cda0 0x7fc2f81526f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:19.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.457+0000 7fc2f5d9b640 1 -- 172.21.15.154:0/648594075 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f00094d0 con 0x7fc2f815c9c0 2023-09-23T03:15:19.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.457+0000 7fc2f6d9d640 1 --2- 172.21.15.154:0/648594075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2f815cda0 0x7fc2f81526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:19.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.457+0000 7fc2f5d9b640 1 --2- 172.21.15.154:0/648594075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2f815c9c0 0x7fc2f815db60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc2e000c3e0 tx=0x7fc2e000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:19.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.458+0000 7fc2eeffd640 1 -- 172.21.15.154:0/648594075 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e00074e0 con 0x7fc2f815c9c0 2023-09-23T03:15:19.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.458+0000 7fc2eeffd640 1 -- 172.21.15.154:0/648594075 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2e0007640 con 0x7fc2f815c9c0 2023-09-23T03:15:19.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.458+0000 7fc2eeffd640 1 -- 172.21.15.154:0/648594075 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e00049e0 con 0x7fc2f815c9c0 2023-09-23T03:15:19.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.458+0000 7fc2fc990640 1 -- 172.21.15.154:0/648594075 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2f80c9140 con 0x7fc2f815c9c0 2023-09-23T03:15:19.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.458+0000 7fc2fc990640 1 -- 172.21.15.154:0/648594075 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f815e0a0 con 0x7fc2f815c9c0 2023-09-23T03:15:19.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.459+0000 7fc2fc990640 1 -- 172.21.15.154:0/648594075 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2f80d1fd0 con 0x7fc2f815c9c0 2023-09-23T03:15:19.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.460+0000 7fc2eeffd640 1 -- 172.21.15.154:0/648594075 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fc2e0004b40 con 0x7fc2f815c9c0 2023-09-23T03:15:19.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.461+0000 7fc2eeffd640 1 --2- 172.21.15.154:0/648594075 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc2d806af30 0x7fc2d806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:19.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.461+0000 7fc2eeffd640 1 -- 172.21.15.154:0/648594075 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fc2e0088db0 con 0x7fc2f815c9c0 2023-09-23T03:15:19.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.461+0000 7fc2f659c640 1 --2- 172.21.15.154:0/648594075 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc2d806af30 0x7fc2d806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:19.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.462+0000 7fc2f659c640 1 --2- 172.21.15.154:0/648594075 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc2d806af30 0x7fc2d806d3f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8005550 tx=0x7fc2e8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:19.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.466+0000 7fc2eeffd640 1 -- 172.21.15.154:0/648594075 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fc2f80d1fd0 con 0x7fc2f815c9c0 2023-09-23T03:15:19.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:19.657+0000 7fc2fc990640 1 -- 172.21.15.154:0/648594075 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fc2f80d0bb0 con 0x7fc2f815c9c0 2023-09-23T03:15:20.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.213+0000 7fc2eeffd640 1 -- 172.21.15.154:0/648594075 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) v1 ==== 171+0+0 (secure 0 0 0) 0x7fc2f80d0bb0 con 0x7fc2f815c9c0 2023-09-23T03:15:20.215 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_data' created 2023-09-23T03:15:20.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.216+0000 7fc2fc990640 1 -- 172.21.15.154:0/648594075 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc2d806af30 msgr2=0x7fc2d806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:20.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.216+0000 7fc2fc990640 1 --2- 172.21.15.154:0/648594075 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc2d806af30 0x7fc2d806d3f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8005550 tx=0x7fc2e8009040 comp rx=0 tx=0).stop 2023-09-23T03:15:20.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.216+0000 7fc2fc990640 1 -- 172.21.15.154:0/648594075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2f815c9c0 msgr2=0x7fc2f815db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:20.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.216+0000 7fc2fc990640 1 --2- 172.21.15.154:0/648594075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2f815c9c0 0x7fc2f815db60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc2e000c3e0 tx=0x7fc2e000c8a0 comp rx=0 tx=0).stop 2023-09-23T03:15:20.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.217+0000 7fc2fc990640 1 -- 172.21.15.154:0/648594075 shutdown_connections 2023-09-23T03:15:20.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.217+0000 7fc2fc990640 1 --2- 172.21.15.154:0/648594075 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc2d806af30 0x7fc2d806d3f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:20.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.217+0000 7fc2fc990640 1 --2- 172.21.15.154:0/648594075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2f815cda0 0x7fc2f81526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:20.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.217+0000 7fc2fc990640 1 --2- 172.21.15.154:0/648594075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2f815c9c0 0x7fc2f815db60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:20.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.217+0000 7fc2fc990640 1 --2- 172.21.15.154:0/648594075 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2f80ce940 0x7fc2f80c84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:20.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.217+0000 7fc2fc990640 1 -- 172.21.15.154:0/648594075 >> 172.21.15.154:0/648594075 conn(0x7fc2f809bc20 msgr2=0x7fc2f80b0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:20.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.217+0000 7fc2fc990640 1 -- 172.21.15.154:0/648594075 shutdown_connections 2023-09-23T03:15:20.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.217+0000 7fc2fc990640 1 -- 172.21.15.154:0/648594075 wait complete. 2023-09-23T03:15:20.248 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-09-23T03:15:20.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.440+0000 7f3674897640 1 Processor -- start 2023-09-23T03:15:20.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.440+0000 7f3674897640 1 -- start start 2023-09-23T03:15:20.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.441+0000 7f3674897640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f36700c9150 0x7f367015cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:20.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.441+0000 7f3674897640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3670071290 0x7f36700ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:20.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.441+0000 7f3674897640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36700c9d20 con 0x7f36700c8d70 2023-09-23T03:15:20.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.441+0000 7f3674897640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3670071850 con 0x7f36700c9150 2023-09-23T03:15:20.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.441+0000 7f3674897640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3670071b30 con 0x7f3670071290 2023-09-23T03:15:20.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.442+0000 7f366e59c640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f36700c8d70 0x7f367015c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:59086/0 (socket says 172.21.15.154:59086) 2023-09-23T03:15:20.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.442+0000 7f366e59c640 1 -- 172.21.15.154:0/3397668858 learned_addr learned my addr 172.21.15.154:0/3397668858 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:20.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.442+0000 7f3667fff640 1 --2- 172.21.15.154:0/3397668858 >> v2:172.21.15.154:3300/0 conn(0x7f36700c9150 0x7f367015cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:20.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.442+0000 7f366ed9d640 1 --2- 172.21.15.154:0/3397668858 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3670071290 0x7f36700ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:20.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.442+0000 7f366dd9b640 1 -- 172.21.15.154:0/3397668858 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 249452534 0 0) 0x7f36700c9d20 con 0x7f36700c8d70 2023-09-23T03:15:20.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.442+0000 7f366dd9b640 1 -- 172.21.15.154:0/3397668858 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f365c003520 con 0x7f36700c8d70 2023-09-23T03:15:20.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.442+0000 7f3667fff640 1 -- 172.21.15.154:0/3397668858 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3670071290 msgr2=0x7f36700ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:20.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.442+0000 7f3667fff640 1 --2- 172.21.15.154:0/3397668858 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3670071290 0x7f36700ca1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:20.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.442+0000 7f3667fff640 1 -- 172.21.15.154:0/3397668858 >> v1:172.21.15.154:6789/0 conn(0x7f36700c8d70 legacy=0x7f367015c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:20.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.442+0000 7f3667fff640 1 -- 172.21.15.154:0/3397668858 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3670071e10 con 0x7f36700c9150 2023-09-23T03:15:20.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.443+0000 7f3667fff640 1 --2- 172.21.15.154:0/3397668858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36700c9150 0x7f367015cde0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f36580097a0 tx=0x7f365802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab7d66eb3decbe06 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:20.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.444+0000 7f366dd9b640 1 -- 172.21.15.154:0/3397668858 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365802d040 con 0x7f36700c9150 2023-09-23T03:15:20.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.444+0000 7f366dd9b640 1 -- 172.21.15.154:0/3397668858 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3658031810 con 0x7f36700c9150 2023-09-23T03:15:20.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.444+0000 7f366dd9b640 1 -- 172.21.15.154:0/3397668858 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3658031b80 con 0x7f36700c9150 2023-09-23T03:15:20.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.444+0000 7f3674897640 1 -- 172.21.15.154:0/3397668858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36700c9150 msgr2=0x7f367015cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:20.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.444+0000 7f3674897640 1 --2- 172.21.15.154:0/3397668858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36700c9150 0x7f367015cde0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f36580097a0 tx=0x7f365802b8b0 comp rx=0 tx=0).stop 2023-09-23T03:15:20.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.444+0000 7f3674897640 1 -- 172.21.15.154:0/3397668858 shutdown_connections 2023-09-23T03:15:20.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.444+0000 7f3674897640 1 --2- 172.21.15.154:0/3397668858 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3670071290 0x7f36700ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:20.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.444+0000 7f3674897640 1 --2- 172.21.15.154:0/3397668858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36700c9150 0x7f367015cde0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:20.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.444+0000 7f3674897640 1 -- 172.21.15.154:0/3397668858 >> 172.21.15.154:0/3397668858 conn(0x7f367006c010 msgr2=0x7f367006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:20.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.445+0000 7f3674897640 1 -- 172.21.15.154:0/3397668858 shutdown_connections 2023-09-23T03:15:20.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.445+0000 7f3674897640 1 -- 172.21.15.154:0/3397668858 wait complete. 2023-09-23T03:15:20.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.446+0000 7f3674897640 1 Processor -- start 2023-09-23T03:15:20.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.446+0000 7f3674897640 1 -- start start 2023-09-23T03:15:20.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.447+0000 7f3674897640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3670071290 0x7f367015cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:20.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.447+0000 7f3674897640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36700c8d70 0x7f36700b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:20.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.447+0000 7f3674897640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36700c9150 0x7f36700b26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:20.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.447+0000 7f3674897640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36700b2c20 con 0x7f36700c8d70 2023-09-23T03:15:20.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.447+0000 7f3674897640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36700b0d50 con 0x7f3670071290 2023-09-23T03:15:20.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.447+0000 7f3674897640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36700af0b0 con 0x7f36700c9150 2023-09-23T03:15:20.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.448+0000 7f3667fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36700c8d70 0x7f36700b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:20.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.448+0000 7f3667fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36700c8d70 0x7f36700b21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45896/0 (socket says 172.21.15.154:45896) 2023-09-23T03:15:20.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.448+0000 7f3667fff640 1 -- 172.21.15.154:0/112711544 learned_addr learned my addr 172.21.15.154:0/112711544 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:20.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.448+0000 7f366ed9d640 1 --2- 172.21.15.154:0/112711544 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36700c9150 0x7f36700b26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:20.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.448+0000 7f366e59c640 1 --2- 172.21.15.154:0/112711544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3670071290 0x7f367015cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:20.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.448+0000 7f3667fff640 1 -- 172.21.15.154:0/112711544 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36700c9150 msgr2=0x7f36700b26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:20.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.448+0000 7f3667fff640 1 --2- 172.21.15.154:0/112711544 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36700c9150 0x7f36700b26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:20.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.448+0000 7f3667fff640 1 -- 172.21.15.154:0/112711544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3670071290 msgr2=0x7f367015cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:20.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.448+0000 7f3667fff640 1 --2- 172.21.15.154:0/112711544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3670071290 0x7f367015cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:20.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.448+0000 7f3667fff640 1 -- 172.21.15.154:0/112711544 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3658009400 con 0x7f36700c8d70 2023-09-23T03:15:20.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.449+0000 7f3667fff640 1 --2- 172.21.15.154:0/112711544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36700c8d70 0x7f36700b21a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3658000c00 tx=0x7f36580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:20.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.449+0000 7f3666ffd640 1 -- 172.21.15.154:0/112711544 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3658006e10 con 0x7f36700c8d70 2023-09-23T03:15:20.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.449+0000 7f3666ffd640 1 -- 172.21.15.154:0/112711544 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f365803ad60 con 0x7f36700c8d70 2023-09-23T03:15:20.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.449+0000 7f3666ffd640 1 -- 172.21.15.154:0/112711544 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3658044610 con 0x7f36700c8d70 2023-09-23T03:15:20.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.449+0000 7f3674897640 1 -- 172.21.15.154:0/112711544 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36700af390 con 0x7f36700c8d70 2023-09-23T03:15:20.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.449+0000 7f3674897640 1 -- 172.21.15.154:0/112711544 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36700c68b0 con 0x7f36700c8d70 2023-09-23T03:15:20.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.450+0000 7f3674897640 1 -- 172.21.15.154:0/112711544 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f367004e920 con 0x7f36700c8d70 2023-09-23T03:15:20.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.451+0000 7f3666ffd640 1 -- 172.21.15.154:0/112711544 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f365804d450 con 0x7f36700c8d70 2023-09-23T03:15:20.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.452+0000 7f3666ffd640 1 --2- 172.21.15.154:0/112711544 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f365006af60 0x7f365006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:20.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.452+0000 7f3666ffd640 1 -- 172.21.15.154:0/112711544 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f36580abc70 con 0x7f36700c8d70 2023-09-23T03:15:20.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.453+0000 7f366e59c640 1 --2- 172.21.15.154:0/112711544 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f365006af60 0x7f365006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:20.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.453+0000 7f366e59c640 1 --2- 172.21.15.154:0/112711544 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f365006af60 0x7f365006d420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3660007fd0 tx=0x7f3660009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:20.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.456+0000 7f3666ffd640 1 -- 172.21.15.154:0/112711544 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f3658076fa0 con 0x7f36700c8d70 2023-09-23T03:15:20.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:20.682+0000 7f3674897640 1 -- 172.21.15.154:0/112711544 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f367006cbc0 con 0x7f36700c8d70 2023-09-23T03:15:21.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.247+0000 7f3666ffd640 1 -- 172.21.15.154:0/112711544 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f365807a5c0 con 0x7f36700c8d70 2023-09-23T03:15:21.249 INFO:teuthology.orchestra.run.smithi154.stderr:new fs with metadata pool 2 and data pool 3 2023-09-23T03:15:21.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.251+0000 7f3674897640 1 -- 172.21.15.154:0/112711544 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f365006af60 msgr2=0x7f365006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:21.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.251+0000 7f3674897640 1 --2- 172.21.15.154:0/112711544 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f365006af60 0x7f365006d420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3660007fd0 tx=0x7f3660009040 comp rx=0 tx=0).stop 2023-09-23T03:15:21.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.251+0000 7f3674897640 1 -- 172.21.15.154:0/112711544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36700c8d70 msgr2=0x7f36700b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:21.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.251+0000 7f3674897640 1 --2- 172.21.15.154:0/112711544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36700c8d70 0x7f36700b21a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3658000c00 tx=0x7f36580046c0 comp rx=0 tx=0).stop 2023-09-23T03:15:21.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.251+0000 7f3674897640 1 -- 172.21.15.154:0/112711544 shutdown_connections 2023-09-23T03:15:21.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.251+0000 7f3674897640 1 --2- 172.21.15.154:0/112711544 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f365006af60 0x7f365006d420 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:21.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.251+0000 7f3674897640 1 --2- 172.21.15.154:0/112711544 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f36700c9150 0x7f36700b26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:21.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.251+0000 7f3674897640 1 --2- 172.21.15.154:0/112711544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f36700c8d70 0x7f36700b21a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:21.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.251+0000 7f3674897640 1 --2- 172.21.15.154:0/112711544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3670071290 0x7f367015cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:21.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.251+0000 7f3674897640 1 -- 172.21.15.154:0/112711544 >> 172.21.15.154:0/112711544 conn(0x7f367006c010 msgr2=0x7f36700c2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:21.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.251+0000 7f3674897640 1 -- 172.21.15.154:0/112711544 shutdown_connections 2023-09-23T03:15:21.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.251+0000 7f3674897640 1 -- 172.21.15.154:0/112711544 wait complete. 2023-09-23T03:15:21.281 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-09-23T03:15:21.281 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-09-23T03:15:21.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.475+0000 7f99c6490640 1 Processor -- start 2023-09-23T03:15:21.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.476+0000 7f99c6490640 1 -- start start 2023-09-23T03:15:21.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.477+0000 7f99c6490640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99c015abd0 0x7f99c00c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:21.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.477+0000 7f99c6490640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f99c00cb460 0x7f99c00b1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:21.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.477+0000 7f99c6490640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c00c95b0 con 0x7f99c00700f0 2023-09-23T03:15:21.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.477+0000 7f99c6490640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99c00c9890 con 0x7f99c00cb460 2023-09-23T03:15:21.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.477+0000 7f99c6490640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c00c9b70 con 0x7f99c015abd0 2023-09-23T03:15:21.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.478+0000 7f99b7fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99c015abd0 0x7f99c00c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:21.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.478+0000 7f99bffff640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f99c00700f0 0x7f99c00704d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.154:41102/0 (socket says 172.21.15.154:41102) 2023-09-23T03:15:21.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.478+0000 7f99bffff640 1 -- 172.21.15.154:0/798920988 learned_addr learned my addr 172.21.15.154:0/798920988 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:21.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.478+0000 7f99c4a2d640 1 --2- 172.21.15.154:0/798920988 >> v2:172.21.15.190:3300/0 conn(0x7f99c00cb460 0x7f99c00b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:21.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.479+0000 7f99c4a2d640 1 -- 172.21.15.154:0/798920988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99c015abd0 msgr2=0x7f99c00c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:21.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.479+0000 7f99c4a2d640 1 --2- 172.21.15.154:0/798920988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99c015abd0 0x7f99c00c8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:21.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.479+0000 7f99c4a2d640 1 -- 172.21.15.154:0/798920988 >> v1:172.21.15.190:6789/0 conn(0x7f99c00700f0 legacy=0x7f99c00704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:21.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.479+0000 7f99c4a2d640 1 -- 172.21.15.154:0/798920988 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c00c9e50 con 0x7f99c00cb460 2023-09-23T03:15:21.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.479+0000 7f99b7fff640 1 --2- 172.21.15.154:0/798920988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99c015abd0 0x7f99c00c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:15:21.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.480+0000 7f99c4a2d640 1 --2- 172.21.15.154:0/798920988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99c00cb460 0x7f99c00b1420 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f99b8009d90 tx=0x7f99b802beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0215058d5c23b1e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:21.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.481+0000 7f99bf7fe640 1 -- 172.21.15.154:0/798920988 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b802cc60 con 0x7f99c00cb460 2023-09-23T03:15:21.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.481+0000 7f99bf7fe640 1 -- 172.21.15.154:0/798920988 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b802cdc0 con 0x7f99c00cb460 2023-09-23T03:15:21.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.481+0000 7f99bf7fe640 1 -- 172.21.15.154:0/798920988 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b8034660 con 0x7f99c00cb460 2023-09-23T03:15:21.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.482+0000 7f99c6490640 1 -- 172.21.15.154:0/798920988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99c00cb460 msgr2=0x7f99c00b1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:21.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.482+0000 7f99c6490640 1 --2- 172.21.15.154:0/798920988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99c00cb460 0x7f99c00b1420 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f99b8009d90 tx=0x7f99b802beb0 comp rx=0 tx=0).stop 2023-09-23T03:15:21.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.482+0000 7f99c6490640 1 -- 172.21.15.154:0/798920988 shutdown_connections 2023-09-23T03:15:21.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.482+0000 7f99c6490640 1 --2- 172.21.15.154:0/798920988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99c00cb460 0x7f99c00b1420 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:21.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.482+0000 7f99c6490640 1 --2- 172.21.15.154:0/798920988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99c015abd0 0x7f99c00c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:21.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.482+0000 7f99c6490640 1 -- 172.21.15.154:0/798920988 >> 172.21.15.154:0/798920988 conn(0x7f99c006a520 msgr2=0x7f99c006bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:21.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.482+0000 7f99c6490640 1 -- 172.21.15.154:0/798920988 shutdown_connections 2023-09-23T03:15:21.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.482+0000 7f99c6490640 1 -- 172.21.15.154:0/798920988 wait complete. 2023-09-23T03:15:21.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.484+0000 7f99c6490640 1 Processor -- start 2023-09-23T03:15:21.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.484+0000 7f99c6490640 1 -- start start 2023-09-23T03:15:21.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.485+0000 7f99c6490640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99c00700f0 0x7f99c00ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:21.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.485+0000 7f99c6490640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99c00cb460 0x7f99c00c6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:21.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.485+0000 7f99c6490640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99c015abd0 0x7f99c01580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:21.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.485+0000 7f99c6490640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c00cc020 con 0x7f99c00cb460 2023-09-23T03:15:21.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.485+0000 7f99c6490640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c00c6d60 con 0x7f99c015abd0 2023-09-23T03:15:21.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.485+0000 7f99c6490640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c00abf50 con 0x7f99c00700f0 2023-09-23T03:15:21.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.486+0000 7f99b7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99c00cb460 0x7f99c00c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:21.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.486+0000 7f99b7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99c00cb460 0x7f99c00c6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45906/0 (socket says 172.21.15.154:45906) 2023-09-23T03:15:21.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.486+0000 7f99b7fff640 1 -- 172.21.15.154:0/503489854 learned_addr learned my addr 172.21.15.154:0/503489854 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:21.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.486+0000 7f99bffff640 1 --2- 172.21.15.154:0/503489854 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99c00700f0 0x7f99c00ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:21.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.486+0000 7f99c4a2d640 1 --2- 172.21.15.154:0/503489854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99c015abd0 0x7f99c01580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:21.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.486+0000 7f99b7fff640 1 -- 172.21.15.154:0/503489854 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99c00700f0 msgr2=0x7f99c00ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:21.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.486+0000 7f99b7fff640 1 --2- 172.21.15.154:0/503489854 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99c00700f0 0x7f99c00ab8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:21.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.486+0000 7f99b7fff640 1 -- 172.21.15.154:0/503489854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99c015abd0 msgr2=0x7f99c01580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:21.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.486+0000 7f99b7fff640 1 --2- 172.21.15.154:0/503489854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99c015abd0 0x7f99c01580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:21.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.486+0000 7f99b7fff640 1 -- 172.21.15.154:0/503489854 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a8009400 con 0x7f99c00cb460 2023-09-23T03:15:21.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.486+0000 7f99bffff640 1 --2- 172.21.15.154:0/503489854 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99c00700f0 0x7f99c00ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:21.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.486+0000 7f99c4a2d640 1 --2- 172.21.15.154:0/503489854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99c015abd0 0x7f99c01580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:21.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.486+0000 7f99b7fff640 1 --2- 172.21.15.154:0/503489854 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99c00cb460 0x7f99c00c6820 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f99a800e930 tx=0x7f99a800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:21.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.487+0000 7f99bd7fa640 1 -- 172.21.15.154:0/503489854 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a8009660 con 0x7f99c00cb460 2023-09-23T03:15:21.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.487+0000 7f99c6490640 1 -- 172.21.15.154:0/503489854 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99b8009a70 con 0x7f99c00cb460 2023-09-23T03:15:21.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.487+0000 7f99bd7fa640 1 -- 172.21.15.154:0/503489854 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a800ca00 con 0x7f99c00cb460 2023-09-23T03:15:21.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.487+0000 7f99bd7fa640 1 -- 172.21.15.154:0/503489854 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a800cdb0 con 0x7f99c00cb460 2023-09-23T03:15:21.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.487+0000 7f99c6490640 1 -- 172.21.15.154:0/503489854 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c00ac510 con 0x7f99c00cb460 2023-09-23T03:15:21.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.488+0000 7f99c6490640 1 -- 172.21.15.154:0/503489854 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c004e920 con 0x7f99c00cb460 2023-09-23T03:15:21.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.489+0000 7f99bd7fa640 1 -- 172.21.15.154:0/503489854 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f99a8013470 con 0x7f99c00cb460 2023-09-23T03:15:21.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.490+0000 7f99bd7fa640 1 --2- 172.21.15.154:0/503489854 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f99a006af30 0x7f99a006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:21.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.490+0000 7f99bd7fa640 1 -- 172.21.15.154:0/503489854 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f99a80896a0 con 0x7f99c00cb460 2023-09-23T03:15:21.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.490+0000 7f99bffff640 1 --2- 172.21.15.154:0/503489854 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f99a006af30 0x7f99a006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:21.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.491+0000 7f99bffff640 1 --2- 172.21.15.154:0/503489854 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f99a006af30 0x7f99a006d3f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f99b0005880 tx=0x7f99b0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:21.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.495+0000 7f99bd7fa640 1 -- 172.21.15.154:0/503489854 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f99a8054660 con 0x7f99c00cb460 2023-09-23T03:15:21.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:21.692+0000 7f99c6490640 1 -- 172.21.15.154:0/503489854 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f99c00c1ce0 con 0x7f99c00cb460 2023-09-23T03:15:22.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.130+0000 7f99bd7fa640 1 -- 172.21.15.154:0/503489854 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) v1 ==== 157+0+0 (secure 0 0 0) 0x7f99a8057c80 con 0x7f99c00cb460 2023-09-23T03:15:22.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.134+0000 7f99c6490640 1 -- 172.21.15.154:0/503489854 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f99a006af30 msgr2=0x7f99a006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:22.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.134+0000 7f99c6490640 1 --2- 172.21.15.154:0/503489854 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f99a006af30 0x7f99a006d3f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f99b0005880 tx=0x7f99b0009040 comp rx=0 tx=0).stop 2023-09-23T03:15:22.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.134+0000 7f99c6490640 1 -- 172.21.15.154:0/503489854 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99c00cb460 msgr2=0x7f99c00c6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:22.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.134+0000 7f99c6490640 1 --2- 172.21.15.154:0/503489854 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99c00cb460 0x7f99c00c6820 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f99a800e930 tx=0x7f99a800edf0 comp rx=0 tx=0).stop 2023-09-23T03:15:22.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.135+0000 7f99c6490640 1 -- 172.21.15.154:0/503489854 shutdown_connections 2023-09-23T03:15:22.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.135+0000 7f99c6490640 1 --2- 172.21.15.154:0/503489854 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f99a006af30 0x7f99a006d3f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:22.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.135+0000 7f99c6490640 1 --2- 172.21.15.154:0/503489854 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99c015abd0 0x7f99c01580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:22.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.135+0000 7f99c6490640 1 --2- 172.21.15.154:0/503489854 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99c00cb460 0x7f99c00c6820 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:22.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.135+0000 7f99c6490640 1 --2- 172.21.15.154:0/503489854 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99c00700f0 0x7f99c00ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:22.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.135+0000 7f99c6490640 1 -- 172.21.15.154:0/503489854 >> 172.21.15.154:0/503489854 conn(0x7f99c006a520 msgr2=0x7f99c00adeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:22.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.135+0000 7f99c6490640 1 -- 172.21.15.154:0/503489854 shutdown_connections 2023-09-23T03:15:22.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.135+0000 7f99c6490640 1 -- 172.21.15.154:0/503489854 wait complete. 2023-09-23T03:15:22.158 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-09-23T03:15:22.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.382+0000 7fbfbebf5640 1 Processor -- start 2023-09-23T03:15:22.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.382+0000 7fbfbebf5640 1 -- start start 2023-09-23T03:15:22.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.384+0000 7fbfbebf5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbfb80affd0 0x7fbfb80b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:22.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.384+0000 7fbfbebf5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfb80b0970 con 0x7fbfb80b1580 2023-09-23T03:15:22.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.384+0000 7fbfbebf5640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfb8071100 con 0x7fbfb815ac40 2023-09-23T03:15:22.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.384+0000 7fbfbebf5640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfb80caca0 con 0x7fbfb80affd0 2023-09-23T03:15:22.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.384+0000 7fbfb7fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbfb80affd0 0x7fbfb80b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:22.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.384+0000 7fbfbd192640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fbfb80b1580 0x7fbfb80b19d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:59102/0 (socket says 172.21.15.154:59102) 2023-09-23T03:15:22.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.384+0000 7fbfbd192640 1 -- 172.21.15.154:0/1742724607 learned_addr learned my addr 172.21.15.154:0/1742724607 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:22.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.385+0000 7fbfb7fff640 1 -- 172.21.15.154:0/1742724607 >> v1:172.21.15.190:6789/0 conn(0x7fbfb815ac40 legacy=0x7fbfb80709f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:22.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.385+0000 7fbfb7fff640 1 -- 172.21.15.154:0/1742724607 >> v1:172.21.15.154:6789/0 conn(0x7fbfb80b1580 legacy=0x7fbfb80b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:22.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.385+0000 7fbfb7fff640 1 -- 172.21.15.154:0/1742724607 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb80c9c00 con 0x7fbfb80affd0 2023-09-23T03:15:22.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.386+0000 7fbfb7fff640 1 --2- 172.21.15.154:0/1742724607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfb80affd0 0x7fbfb80b0430 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbfa40097a0 tx=0x7fbfa402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70eb579a91dd1e3b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:22.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.386+0000 7fbfb77fe640 1 -- 172.21.15.154:0/1742724607 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4031830 con 0x7fbfb80affd0 2023-09-23T03:15:22.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.386+0000 7fbfb77fe640 1 -- 172.21.15.154:0/1742724607 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa4031990 con 0x7fbfb80affd0 2023-09-23T03:15:22.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.386+0000 7fbfb77fe640 1 -- 172.21.15.154:0/1742724607 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4031d20 con 0x7fbfb80affd0 2023-09-23T03:15:22.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.387+0000 7fbfbebf5640 1 -- 172.21.15.154:0/1742724607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfb80affd0 msgr2=0x7fbfb80b0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:22.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.387+0000 7fbfbebf5640 1 --2- 172.21.15.154:0/1742724607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfb80affd0 0x7fbfb80b0430 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbfa40097a0 tx=0x7fbfa402b8d0 comp rx=0 tx=0).stop 2023-09-23T03:15:22.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.387+0000 7fbfbebf5640 1 -- 172.21.15.154:0/1742724607 shutdown_connections 2023-09-23T03:15:22.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.387+0000 7fbfbebf5640 1 --2- 172.21.15.154:0/1742724607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfb80affd0 0x7fbfb80b0430 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:22.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.387+0000 7fbfbebf5640 1 -- 172.21.15.154:0/1742724607 >> 172.21.15.154:0/1742724607 conn(0x7fbfb806c080 msgr2=0x7fbfb806c460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:22.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.387+0000 7fbfbebf5640 1 -- 172.21.15.154:0/1742724607 shutdown_connections 2023-09-23T03:15:22.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.387+0000 7fbfbebf5640 1 -- 172.21.15.154:0/1742724607 wait complete. 2023-09-23T03:15:22.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.389+0000 7fbfbebf5640 1 Processor -- start 2023-09-23T03:15:22.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.389+0000 7fbfbebf5640 1 -- start start 2023-09-23T03:15:22.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.390+0000 7fbfbebf5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfb80affd0 0x7fbfb814a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:22.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.390+0000 7fbfbebf5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfb80b1580 0x7fbfb80c2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:22.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.390+0000 7fbfbebf5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfb80ce4c0 0x7fbfb80ce940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:22.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.390+0000 7fbfbebf5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb80cee80 con 0x7fbfb80ce4c0 2023-09-23T03:15:22.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.390+0000 7fbfbebf5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb814abc0 con 0x7fbfb80b1580 2023-09-23T03:15:22.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.390+0000 7fbfbebf5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb80c63d0 con 0x7fbfb80affd0 2023-09-23T03:15:22.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.390+0000 7fbfbd192640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfb80ce4c0 0x7fbfb80ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:22.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.390+0000 7fbfbd192640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfb80ce4c0 0x7fbfb80ce940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45924/0 (socket says 172.21.15.154:45924) 2023-09-23T03:15:22.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.390+0000 7fbfbd192640 1 -- 172.21.15.154:0/744271598 learned_addr learned my addr 172.21.15.154:0/744271598 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:22.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.391+0000 7fbfbc991640 1 --2- 172.21.15.154:0/744271598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfb80affd0 0x7fbfb814a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:22.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.391+0000 7fbfb7fff640 1 --2- 172.21.15.154:0/744271598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfb80b1580 0x7fbfb80c2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:22.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.391+0000 7fbfbd192640 1 -- 172.21.15.154:0/744271598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfb80affd0 msgr2=0x7fbfb814a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:22.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.391+0000 7fbfbd192640 1 --2- 172.21.15.154:0/744271598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfb80affd0 0x7fbfb814a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:22.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.391+0000 7fbfbd192640 1 -- 172.21.15.154:0/744271598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfb80b1580 msgr2=0x7fbfb80c2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:22.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.391+0000 7fbfbd192640 1 --2- 172.21.15.154:0/744271598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfb80b1580 0x7fbfb80c2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:22.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.391+0000 7fbfbd192640 1 -- 172.21.15.154:0/744271598 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa4009400 con 0x7fbfb80ce4c0 2023-09-23T03:15:22.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.391+0000 7fbfbd192640 1 --2- 172.21.15.154:0/744271598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfb80ce4c0 0x7fbfb80ce940 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000cbc0 tx=0x7fbfb000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:22.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.391+0000 7fbfb57fa640 1 -- 172.21.15.154:0/744271598 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb000ace0 con 0x7fbfb80ce4c0 2023-09-23T03:15:22.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.391+0000 7fbfb57fa640 1 -- 172.21.15.154:0/744271598 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb000ae40 con 0x7fbfb80ce4c0 2023-09-23T03:15:22.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.391+0000 7fbfb57fa640 1 -- 172.21.15.154:0/744271598 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0012600 con 0x7fbfb80ce4c0 2023-09-23T03:15:22.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.391+0000 7fbfbebf5640 1 -- 172.21.15.154:0/744271598 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfb80c57c0 con 0x7fbfb80ce4c0 2023-09-23T03:15:22.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.392+0000 7fbfbebf5640 1 -- 172.21.15.154:0/744271598 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb80c5d80 con 0x7fbfb80ce4c0 2023-09-23T03:15:22.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.393+0000 7fbfbebf5640 1 -- 172.21.15.154:0/744271598 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfb8073eb0 con 0x7fbfb80ce4c0 2023-09-23T03:15:22.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.394+0000 7fbfb57fa640 1 -- 172.21.15.154:0/744271598 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fbfb0007070 con 0x7fbfb80ce4c0 2023-09-23T03:15:22.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.395+0000 7fbfb57fa640 1 --2- 172.21.15.154:0/744271598 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbf9c06af30 0x7fbf9c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:22.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.395+0000 7fbfb57fa640 1 -- 172.21.15.154:0/744271598 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5935+0+0 (secure 0 0 0) 0x7fbfb00896b0 con 0x7fbfb80ce4c0 2023-09-23T03:15:22.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.395+0000 7fbfbc991640 1 --2- 172.21.15.154:0/744271598 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbf9c06af30 0x7fbf9c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:22.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.396+0000 7fbfbc991640 1 --2- 172.21.15.154:0/744271598 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbf9c06af30 0x7fbf9c06d3f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbfac000f60 tx=0x7fbfac009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:22.400 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.400+0000 7fbfb57fa640 1 -- 172.21.15.154:0/744271598 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fbfb00546c0 con 0x7fbfb80ce4c0 2023-09-23T03:15:22.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:22.592+0000 7fbfbebf5640 1 -- 172.21.15.154:0/744271598 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fbfb80c6510 con 0x7fbfb80ce4c0 2023-09-23T03:15:24.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.284+0000 7fbfb57fa640 1 -- 172.21.15.154:0/744271598 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) v1 ==== 228+0+0 (secure 0 0 0) 0x7fbfb0057ce0 con 0x7fbfb80ce4c0 2023-09-23T03:15:24.285 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_data_ec' created 2023-09-23T03:15:24.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.288+0000 7fbfbebf5640 1 -- 172.21.15.154:0/744271598 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbf9c06af30 msgr2=0x7fbf9c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:24.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.288+0000 7fbfbebf5640 1 --2- 172.21.15.154:0/744271598 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbf9c06af30 0x7fbf9c06d3f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbfac000f60 tx=0x7fbfac009040 comp rx=0 tx=0).stop 2023-09-23T03:15:24.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.288+0000 7fbfbebf5640 1 -- 172.21.15.154:0/744271598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfb80ce4c0 msgr2=0x7fbfb80ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:24.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.288+0000 7fbfbebf5640 1 --2- 172.21.15.154:0/744271598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfb80ce4c0 0x7fbfb80ce940 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbfb000cbc0 tx=0x7fbfb000a430 comp rx=0 tx=0).stop 2023-09-23T03:15:24.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.289+0000 7fbfbebf5640 1 -- 172.21.15.154:0/744271598 shutdown_connections 2023-09-23T03:15:24.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.289+0000 7fbfbebf5640 1 --2- 172.21.15.154:0/744271598 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbf9c06af30 0x7fbf9c06d3f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:24.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.289+0000 7fbfbebf5640 1 --2- 172.21.15.154:0/744271598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbfb80ce4c0 0x7fbfb80ce940 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:24.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.289+0000 7fbfbebf5640 1 --2- 172.21.15.154:0/744271598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbfb80b1580 0x7fbfb80c2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:24.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.289+0000 7fbfbebf5640 1 --2- 172.21.15.154:0/744271598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbfb80affd0 0x7fbfb814a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:24.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.289+0000 7fbfbebf5640 1 -- 172.21.15.154:0/744271598 >> 172.21.15.154:0/744271598 conn(0x7fbfb806c080 msgr2=0x7fbfb80709f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:24.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.289+0000 7fbfbebf5640 1 -- 172.21.15.154:0/744271598 shutdown_connections 2023-09-23T03:15:24.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.289+0000 7fbfbebf5640 1 -- 172.21.15.154:0/744271598 wait complete. 2023-09-23T03:15:24.316 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-09-23T03:15:24.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.504+0000 7fd3d3178640 1 Processor -- start 2023-09-23T03:15:24.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.504+0000 7fd3d3178640 1 -- start start 2023-09-23T03:15:24.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.505+0000 7fd3d3178640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd3cc15c940 0x7fd3cc0d0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:24.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.505+0000 7fd3d3178640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3cc0cd3f0 con 0x7fd3cc0d0960 2023-09-23T03:15:24.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.505+0000 7fd3d3178640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3cc0d3470 con 0x7fd3cc0d1520 2023-09-23T03:15:24.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.505+0000 7fd3d3178640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3cc0cd690 con 0x7fd3cc15c940 2023-09-23T03:15:24.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.506+0000 7fd3cbfff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fd3cc0d0960 0x7fd3cc0d0e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:48088/0 (socket says 172.21.15.154:48088) 2023-09-23T03:15:24.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.506+0000 7fd3cbfff640 1 -- 172.21.15.154:0/3376475606 learned_addr learned my addr 172.21.15.154:0/3376475606 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:24.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.506+0000 7fd3cb7fe640 1 -- 172.21.15.154:0/3376475606 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 970035611 0 0) 0x7fd3cc0cd3f0 con 0x7fd3cc0d0960 2023-09-23T03:15:24.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.506+0000 7fd3cb7fe640 1 -- 172.21.15.154:0/3376475606 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3b8003520 con 0x7fd3cc0d0960 2023-09-23T03:15:24.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.506+0000 7fd3d0f14640 1 --2- 172.21.15.154:0/3376475606 >> v2:172.21.15.190:3300/0 conn(0x7fd3cc15c940 0x7fd3cc0d0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:24.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.506+0000 7fd3cb7fe640 1 -- 172.21.15.154:0/3376475606 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2364882541 0 0) 0x7fd3b8003520 con 0x7fd3cc0d0960 2023-09-23T03:15:24.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.506+0000 7fd3cb7fe640 1 -- 172.21.15.154:0/3376475606 >> v2:172.21.15.190:3300/0 conn(0x7fd3cc15c940 msgr2=0x7fd3cc0d0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:24.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.506+0000 7fd3cb7fe640 1 --2- 172.21.15.154:0/3376475606 >> v2:172.21.15.190:3300/0 conn(0x7fd3cc15c940 0x7fd3cc0d0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:24.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.507+0000 7fd3cb7fe640 1 -- 172.21.15.154:0/3376475606 >> v1:172.21.15.190:6789/0 conn(0x7fd3cc0d1520 legacy=0x7fd3cc1669f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:24.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.507+0000 7fd3cb7fe640 1 -- 172.21.15.154:0/3376475606 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3cc0cd970 con 0x7fd3cc0d0960 2023-09-23T03:15:24.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.507+0000 7fd3cb7fe640 1 -- 172.21.15.154:0/3376475606 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fd3b40019f0 con 0x7fd3cc0d0960 2023-09-23T03:15:24.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.507+0000 7fd3cb7fe640 1 -- 172.21.15.154:0/3376475606 >> v1:172.21.15.154:6789/0 conn(0x7fd3cc0d0960 legacy=0x7fd3cc0d0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:24.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.507+0000 7fd3cb7fe640 1 --2- 172.21.15.154:0/3376475606 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3b80039c0 0x7fd3b8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:24.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.507+0000 7fd3cb7fe640 1 --2- 172.21.15.154:0/3376475606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3b80047c0 0x7fd3b8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:24.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.507+0000 7fd3cb7fe640 1 --2- 172.21.15.154:0/3376475606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3b80051d0 0x7fd3b800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:24.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.507+0000 7fd3cb7fe640 1 -- 172.21.15.154:0/3376475606 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc0cd690 con 0x7fd3b80051d0 2023-09-23T03:15:24.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.507+0000 7fd3cb7fe640 1 -- 172.21.15.154:0/3376475606 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b800c120 con 0x7fd3b80047c0 2023-09-23T03:15:24.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.507+0000 7fd3cb7fe640 1 -- 172.21.15.154:0/3376475606 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b800c2e0 con 0x7fd3b80039c0 2023-09-23T03:15:24.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.507+0000 7fd3d1715640 1 --2- 172.21.15.154:0/3376475606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3b80051d0 0x7fd3b800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:24.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.507+0000 7fd3cbfff640 1 --2- 172.21.15.154:0/3376475606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3b80047c0 0x7fd3b8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:24.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.507+0000 7fd3d0f14640 1 --2- 172.21.15.154:0/3376475606 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3b80039c0 0x7fd3b8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:24.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.507+0000 7fd3d1715640 1 -- 172.21.15.154:0/3376475606 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3b80039c0 msgr2=0x7fd3b8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:24.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.507+0000 7fd3d1715640 1 --2- 172.21.15.154:0/3376475606 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3b80039c0 0x7fd3b8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:24.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.507+0000 7fd3d1715640 1 -- 172.21.15.154:0/3376475606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3b80047c0 msgr2=0x7fd3b8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:24.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.507+0000 7fd3d1715640 1 --2- 172.21.15.154:0/3376475606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3b80047c0 0x7fd3b8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:24.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.507+0000 7fd3d1715640 1 -- 172.21.15.154:0/3376475606 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b8003520 con 0x7fd3b80051d0 2023-09-23T03:15:24.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.508+0000 7fd3d1715640 1 -- 172.21.15.154:0/3376475606 reap_dead start 2023-09-23T03:15:24.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.509+0000 7fd3d1715640 1 --2- 172.21.15.154:0/3376475606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3b80051d0 0x7fd3b800b980 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc0c98d0 tx=0x7fd3b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cbcfb85a357bcdf server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:24.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.509+0000 7fd3cb7fe640 1 -- 172.21.15.154:0/3376475606 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c40052d0 con 0x7fd3b80051d0 2023-09-23T03:15:24.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.509+0000 7fd3cb7fe640 1 -- 172.21.15.154:0/3376475606 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3c402e050 con 0x7fd3b80051d0 2023-09-23T03:15:24.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.509+0000 7fd3cb7fe640 1 -- 172.21.15.154:0/3376475606 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c40035f0 con 0x7fd3b80051d0 2023-09-23T03:15:24.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.509+0000 7fd3d3178640 1 -- 172.21.15.154:0/3376475606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3b80051d0 msgr2=0x7fd3b800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:24.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.509+0000 7fd3d3178640 1 --2- 172.21.15.154:0/3376475606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3b80051d0 0x7fd3b800b980 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc0c98d0 tx=0x7fd3b80057f0 comp rx=0 tx=0).stop 2023-09-23T03:15:24.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.510+0000 7fd3d3178640 1 -- 172.21.15.154:0/3376475606 shutdown_connections 2023-09-23T03:15:24.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.510+0000 7fd3d3178640 1 --2- 172.21.15.154:0/3376475606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3b80051d0 0x7fd3b800b980 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:24.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.510+0000 7fd3d3178640 1 -- 172.21.15.154:0/3376475606 >> 172.21.15.154:0/3376475606 conn(0x7fd3cc089fd0 msgr2=0x7fd3cc0d2850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:24.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.510+0000 7fd3d3178640 1 -- 172.21.15.154:0/3376475606 shutdown_connections 2023-09-23T03:15:24.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.510+0000 7fd3d3178640 1 -- 172.21.15.154:0/3376475606 wait complete. 2023-09-23T03:15:24.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.511+0000 7fd3d3178640 1 Processor -- start 2023-09-23T03:15:24.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.512+0000 7fd3d3178640 1 -- start start 2023-09-23T03:15:24.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.512+0000 7fd3d3178640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3b80051d0 0x7fd3cc1f5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:24.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.512+0000 7fd3d3178640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3cc0cb340 0x7fd3cc0cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:24.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.512+0000 7fd3d3178640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3cc0b2180 0x7fd3cc0b2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:24.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.512+0000 7fd3d3178640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc1f1c70 con 0x7fd3b80051d0 2023-09-23T03:15:24.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.512+0000 7fd3d3178640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc0cbd00 con 0x7fd3cc0b2180 2023-09-23T03:15:24.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.512+0000 7fd3d3178640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc0b2b40 con 0x7fd3cc0cb340 2023-09-23T03:15:24.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.513+0000 7fd3d0f14640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3b80051d0 0x7fd3cc1f5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:24.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.513+0000 7fd3d0f14640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3b80051d0 0x7fd3cc1f5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:38190/0 (socket says 172.21.15.154:38190) 2023-09-23T03:15:24.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.513+0000 7fd3d0f14640 1 -- 172.21.15.154:0/3959190026 learned_addr learned my addr 172.21.15.154:0/3959190026 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:24.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.513+0000 7fd3d1715640 1 --2- 172.21.15.154:0/3959190026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3cc0b2180 0x7fd3cc0b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:24.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.513+0000 7fd3cbfff640 1 --2- 172.21.15.154:0/3959190026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3cc0cb340 0x7fd3cc0cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:24.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.513+0000 7fd3d0f14640 1 -- 172.21.15.154:0/3959190026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3cc0cb340 msgr2=0x7fd3cc0cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:24.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.513+0000 7fd3d0f14640 1 --2- 172.21.15.154:0/3959190026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3cc0cb340 0x7fd3cc0cb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:24.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.513+0000 7fd3d0f14640 1 -- 172.21.15.154:0/3959190026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3cc0b2180 msgr2=0x7fd3cc0b2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:24.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.513+0000 7fd3d0f14640 1 --2- 172.21.15.154:0/3959190026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3cc0b2180 0x7fd3cc0b2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:24.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.513+0000 7fd3d0f14640 1 -- 172.21.15.154:0/3959190026 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3cc1f2c40 con 0x7fd3b80051d0 2023-09-23T03:15:24.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.513+0000 7fd3d0f14640 1 --2- 172.21.15.154:0/3959190026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3b80051d0 0x7fd3cc1f5b50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc00f5b0 tx=0x7fd3bc00fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:24.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.514+0000 7fd3c97fa640 1 -- 172.21.15.154:0/3959190026 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc013810 con 0x7fd3b80051d0 2023-09-23T03:15:24.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.514+0000 7fd3c97fa640 1 -- 172.21.15.154:0/3959190026 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3bc013970 con 0x7fd3b80051d0 2023-09-23T03:15:24.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.514+0000 7fd3c97fa640 1 -- 172.21.15.154:0/3959190026 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc013ce0 con 0x7fd3b80051d0 2023-09-23T03:15:24.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.514+0000 7fd3d3178640 1 -- 172.21.15.154:0/3959190026 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3cc1f1e80 con 0x7fd3b80051d0 2023-09-23T03:15:24.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.514+0000 7fd3d3178640 1 -- 172.21.15.154:0/3959190026 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3cc1f2440 con 0x7fd3b80051d0 2023-09-23T03:15:24.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.515+0000 7fd3d3178640 1 -- 172.21.15.154:0/3959190026 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3cc04e920 con 0x7fd3b80051d0 2023-09-23T03:15:24.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.516+0000 7fd3c97fa640 1 -- 172.21.15.154:0/3959190026 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fd3bc01f460 con 0x7fd3b80051d0 2023-09-23T03:15:24.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.517+0000 7fd3c97fa640 1 --2- 172.21.15.154:0/3959190026 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd3ac06af30 0x7fd3ac06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:24.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.517+0000 7fd3c97fa640 1 -- 172.21.15.154:0/3959190026 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6428+0+0 (secure 0 0 0) 0x7fd3bc089e10 con 0x7fd3b80051d0 2023-09-23T03:15:24.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.517+0000 7fd3cbfff640 1 --2- 172.21.15.154:0/3959190026 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd3ac06af30 0x7fd3ac06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:24.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.518+0000 7fd3cbfff640 1 --2- 172.21.15.154:0/3959190026 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd3ac06af30 0x7fd3ac06d3f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc0a0770 tx=0x7fd3b4001250 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:24.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.522+0000 7fd3c97fa640 1 -- 172.21.15.154:0/3959190026 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fd3bc08d030 con 0x7fd3b80051d0 2023-09-23T03:15:24.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:24.707+0000 7fd3d3178640 1 -- 172.21.15.154:0/3959190026 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fd3cc0cff40 con 0x7fd3b80051d0 2023-09-23T03:15:25.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.288+0000 7fd3c97fa640 1 -- 172.21.15.154:0/3959190026 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd3bc054b70 con 0x7fd3b80051d0 2023-09-23T03:15:25.290 INFO:teuthology.orchestra.run.smithi154.stderr:set pool 4 allow_ec_overwrites to true 2023-09-23T03:15:25.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.293+0000 7fd3d3178640 1 -- 172.21.15.154:0/3959190026 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd3ac06af30 msgr2=0x7fd3ac06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:25.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.293+0000 7fd3d3178640 1 --2- 172.21.15.154:0/3959190026 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd3ac06af30 0x7fd3ac06d3f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc0a0770 tx=0x7fd3b4001250 comp rx=0 tx=0).stop 2023-09-23T03:15:25.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.293+0000 7fd3d3178640 1 -- 172.21.15.154:0/3959190026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3b80051d0 msgr2=0x7fd3cc1f5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:25.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.293+0000 7fd3d3178640 1 --2- 172.21.15.154:0/3959190026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3b80051d0 0x7fd3cc1f5b50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc00f5b0 tx=0x7fd3bc00fa70 comp rx=0 tx=0).stop 2023-09-23T03:15:25.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.296+0000 7fd3d3178640 1 -- 172.21.15.154:0/3959190026 shutdown_connections 2023-09-23T03:15:25.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.296+0000 7fd3d3178640 1 --2- 172.21.15.154:0/3959190026 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd3ac06af30 0x7fd3ac06d3f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:25.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.296+0000 7fd3d3178640 1 --2- 172.21.15.154:0/3959190026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3cc0b2180 0x7fd3cc0b2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:25.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.296+0000 7fd3d3178640 1 --2- 172.21.15.154:0/3959190026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3cc0cb340 0x7fd3cc0cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:25.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.296+0000 7fd3d3178640 1 --2- 172.21.15.154:0/3959190026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3b80051d0 0x7fd3cc1f5b50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:25.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.296+0000 7fd3d3178640 1 -- 172.21.15.154:0/3959190026 >> 172.21.15.154:0/3959190026 conn(0x7fd3cc089fd0 msgr2=0x7fd3cc15cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:25.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.297+0000 7fd3d3178640 1 -- 172.21.15.154:0/3959190026 shutdown_connections 2023-09-23T03:15:25.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.300+0000 7fd3d3178640 1 -- 172.21.15.154:0/3959190026 wait complete. 2023-09-23T03:15:25.340 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-09-23T03:15:25.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.527+0000 7f9742c25640 1 Processor -- start 2023-09-23T03:15:25.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.527+0000 7f9742c25640 1 -- start start 2023-09-23T03:15:25.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.528+0000 7f9742c25640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f973c1582f0 0x7f973c15acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:25.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.528+0000 7f9742c25640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f973c0b08c0 0x7f973c06fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:25.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.528+0000 7f9742c25640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f973c0c9e80 con 0x7f973c0ca790 2023-09-23T03:15:25.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.528+0000 7f9742c25640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f973c1586d0 con 0x7f973c0b08c0 2023-09-23T03:15:25.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.528+0000 7f9742c25640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973c0c9300 con 0x7f973c1582f0 2023-09-23T03:15:25.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.529+0000 7f9741c23640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f973c1582f0 0x7f973c15acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:25.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.529+0000 7f9741c23640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f973c1582f0 0x7f973c15acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:49080/0 (socket says 172.21.15.154:49080) 2023-09-23T03:15:25.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.529+0000 7f9741c23640 1 -- 172.21.15.154:0/571939897 learned_addr learned my addr 172.21.15.154:0/571939897 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:25.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.529+0000 7f9741422640 1 --2- 172.21.15.154:0/571939897 >> v2:172.21.15.190:3300/0 conn(0x7f973c0b08c0 0x7f973c06fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:25.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.530+0000 7f9741c23640 1 -- 172.21.15.154:0/571939897 >> v2:172.21.15.190:3300/0 conn(0x7f973c0b08c0 msgr2=0x7f973c06fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:25.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.530+0000 7f9741c23640 1 --2- 172.21.15.154:0/571939897 >> v2:172.21.15.190:3300/0 conn(0x7f973c0b08c0 0x7f973c06fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:25.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.530+0000 7f9741c23640 1 -- 172.21.15.154:0/571939897 >> v1:172.21.15.190:6789/0 conn(0x7f973c0ca790 legacy=0x7f973c15e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:25.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.530+0000 7f9741c23640 1 -- 172.21.15.154:0/571939897 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f973c0c95e0 con 0x7f973c1582f0 2023-09-23T03:15:25.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.531+0000 7f9741c23640 1 --2- 172.21.15.154:0/571939897 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f973c1582f0 0x7f973c15acd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9738009870 tx=0x7f973802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=224737f7da3a7a80 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:25.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.531+0000 7f9740c21640 1 -- 172.21.15.154:0/571939897 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9738031640 con 0x7f973c1582f0 2023-09-23T03:15:25.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.531+0000 7f9740c21640 1 -- 172.21.15.154:0/571939897 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97380317a0 con 0x7f973c1582f0 2023-09-23T03:15:25.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.531+0000 7f9740c21640 1 -- 172.21.15.154:0/571939897 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9738031b30 con 0x7f973c1582f0 2023-09-23T03:15:25.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.532+0000 7f9742c25640 1 -- 172.21.15.154:0/571939897 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f973c1582f0 msgr2=0x7f973c15acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:25.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.532+0000 7f9742c25640 1 --2- 172.21.15.154:0/571939897 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f973c1582f0 0x7f973c15acd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9738009870 tx=0x7f973802b8f0 comp rx=0 tx=0).stop 2023-09-23T03:15:25.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.532+0000 7f9742c25640 1 -- 172.21.15.154:0/571939897 shutdown_connections 2023-09-23T03:15:25.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.532+0000 7f9742c25640 1 --2- 172.21.15.154:0/571939897 >> v2:172.21.15.190:3300/0 conn(0x7f973c0b08c0 0x7f973c06fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:25.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.532+0000 7f9742c25640 1 --2- 172.21.15.154:0/571939897 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f973c1582f0 0x7f973c15acd0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:25.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.532+0000 7f9742c25640 1 -- 172.21.15.154:0/571939897 >> 172.21.15.154:0/571939897 conn(0x7f973c06a300 msgr2=0x7f973c0caeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:25.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.532+0000 7f9742c25640 1 -- 172.21.15.154:0/571939897 shutdown_connections 2023-09-23T03:15:25.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.532+0000 7f9742c25640 1 -- 172.21.15.154:0/571939897 wait complete. 2023-09-23T03:15:25.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.534+0000 7f9742c25640 1 Processor -- start 2023-09-23T03:15:25.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.534+0000 7f9742c25640 1 -- start start 2023-09-23T03:15:25.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.535+0000 7f9742c25640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f973c0b08c0 0x7f973c148370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:25.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.535+0000 7f9742c25640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f973c0ca790 0x7f973c0c5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:25.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.535+0000 7f9742c25640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f973c1582f0 0x7f973c0c5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:25.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.535+0000 7f9742c25640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c0b3980 con 0x7f973c1582f0 2023-09-23T03:15:25.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.535+0000 7f9742c25640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c145130 con 0x7f973c0ca790 2023-09-23T03:15:25.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.535+0000 7f9742c25640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973c145410 con 0x7f973c0b08c0 2023-09-23T03:15:25.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.535+0000 7f9742424640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f973c1582f0 0x7f973c0c5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:25.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.535+0000 7f9742424640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f973c1582f0 0x7f973c0c5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:38196/0 (socket says 172.21.15.154:38196) 2023-09-23T03:15:25.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.535+0000 7f9742424640 1 -- 172.21.15.154:0/1369382622 learned_addr learned my addr 172.21.15.154:0/1369382622 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:25.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.535+0000 7f9741c23640 1 --2- 172.21.15.154:0/1369382622 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f973c0b08c0 0x7f973c148370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:25.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.535+0000 7f9741422640 1 --2- 172.21.15.154:0/1369382622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f973c0ca790 0x7f973c0c5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:25.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.535+0000 7f9742424640 1 -- 172.21.15.154:0/1369382622 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f973c0b08c0 msgr2=0x7f973c148370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:25.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.535+0000 7f9742424640 1 --2- 172.21.15.154:0/1369382622 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f973c0b08c0 0x7f973c148370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:25.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.535+0000 7f9742424640 1 -- 172.21.15.154:0/1369382622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f973c0ca790 msgr2=0x7f973c0c5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:25.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.535+0000 7f9742424640 1 --2- 172.21.15.154:0/1369382622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f973c0ca790 0x7f973c0c5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:25.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.535+0000 7f9742424640 1 -- 172.21.15.154:0/1369382622 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97380094d0 con 0x7f973c1582f0 2023-09-23T03:15:25.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.535+0000 7f9741422640 1 --2- 172.21.15.154:0/1369382622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f973c0ca790 0x7f973c0c5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:25.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.536+0000 7f9742424640 1 --2- 172.21.15.154:0/1369382622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f973c1582f0 0x7f973c0c5a40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f973400c680 tx=0x7f973400cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:25.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.536+0000 7f972e7fc640 1 -- 172.21.15.154:0/1369382622 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9734003b70 con 0x7f973c1582f0 2023-09-23T03:15:25.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.536+0000 7f972e7fc640 1 -- 172.21.15.154:0/1369382622 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9734003cd0 con 0x7f973c1582f0 2023-09-23T03:15:25.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.536+0000 7f9742c25640 1 -- 172.21.15.154:0/1369382622 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f973c142530 con 0x7f973c1582f0 2023-09-23T03:15:25.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.536+0000 7f972e7fc640 1 -- 172.21.15.154:0/1369382622 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9734005310 con 0x7f973c1582f0 2023-09-23T03:15:25.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.536+0000 7f9742c25640 1 -- 172.21.15.154:0/1369382622 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f973c142af0 con 0x7f973c1582f0 2023-09-23T03:15:25.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.537+0000 7f9742c25640 1 -- 172.21.15.154:0/1369382622 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f973c067390 con 0x7f973c1582f0 2023-09-23T03:15:25.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.538+0000 7f972e7fc640 1 -- 172.21.15.154:0/1369382622 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f9734005470 con 0x7f973c1582f0 2023-09-23T03:15:25.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.539+0000 7f972e7fc640 1 --2- 172.21.15.154:0/1369382622 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f972006af30 0x7f972006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:25.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.539+0000 7f972e7fc640 1 -- 172.21.15.154:0/1369382622 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6428+0+0 (secure 0 0 0) 0x7f97340898c0 con 0x7f973c1582f0 2023-09-23T03:15:25.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.539+0000 7f9741c23640 1 --2- 172.21.15.154:0/1369382622 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f972006af30 0x7f972006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:25.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.540+0000 7f9741c23640 1 --2- 172.21.15.154:0/1369382622 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f972006af30 0x7f972006d3f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9738036000 tx=0x7f973803c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:25.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.543+0000 7f972e7fc640 1 -- 172.21.15.154:0/1369382622 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f9734054500 con 0x7f973c1582f0 2023-09-23T03:15:25.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:25.772+0000 7f9742c25640 1 -- 172.21.15.154:0/1369382622 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f973c0cbc20 con 0x7f973c1582f0 2023-09-23T03:15:26.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.325+0000 7f972e7fc640 1 -- 172.21.15.154:0/1369382622 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7f9734057b20 con 0x7f973c1582f0 2023-09-23T03:15:26.327 INFO:teuthology.orchestra.run.smithi154.stderr:added data pool 4 to fsmap 2023-09-23T03:15:26.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.328+0000 7f9742c25640 1 -- 172.21.15.154:0/1369382622 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f972006af30 msgr2=0x7f972006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:26.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.328+0000 7f9742c25640 1 --2- 172.21.15.154:0/1369382622 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f972006af30 0x7f972006d3f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9738036000 tx=0x7f973803c040 comp rx=0 tx=0).stop 2023-09-23T03:15:26.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.329+0000 7f9742c25640 1 -- 172.21.15.154:0/1369382622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f973c1582f0 msgr2=0x7f973c0c5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:26.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.329+0000 7f9742c25640 1 --2- 172.21.15.154:0/1369382622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f973c1582f0 0x7f973c0c5a40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f973400c680 tx=0x7f973400cb40 comp rx=0 tx=0).stop 2023-09-23T03:15:26.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.329+0000 7f9742c25640 1 -- 172.21.15.154:0/1369382622 shutdown_connections 2023-09-23T03:15:26.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.329+0000 7f9742c25640 1 --2- 172.21.15.154:0/1369382622 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f972006af30 0x7f972006d3f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:26.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.329+0000 7f9742c25640 1 --2- 172.21.15.154:0/1369382622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f973c1582f0 0x7f973c0c5a40 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:26.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.329+0000 7f9742c25640 1 --2- 172.21.15.154:0/1369382622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f973c0ca790 0x7f973c0c5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:26.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.329+0000 7f9742c25640 1 --2- 172.21.15.154:0/1369382622 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f973c0b08c0 0x7f973c148370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:26.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.329+0000 7f9742c25640 1 -- 172.21.15.154:0/1369382622 >> 172.21.15.154:0/1369382622 conn(0x7f973c06a300 msgr2=0x7f973c15c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:26.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.329+0000 7f9742c25640 1 -- 172.21.15.154:0/1369382622 shutdown_connections 2023-09-23T03:15:26.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.329+0000 7f9742c25640 1 -- 172.21.15.154:0/1369382622 wait complete. 2023-09-23T03:15:26.350 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:15:26.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.556+0000 7f5029e10640 1 Processor -- start 2023-09-23T03:15:26.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.556+0000 7f5029e10640 1 -- start start 2023-09-23T03:15:26.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.559+0000 7f5029e10640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f50240b1400 0x7f50240ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:26.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.559+0000 7f5029e10640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f502415cf80 0x7f502406c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:26.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.559+0000 7f5029e10640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50240ce8c0 con 0x7f50240cdfa0 2023-09-23T03:15:26.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.559+0000 7f5029e10640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f502415d550 con 0x7f50240b1400 2023-09-23T03:15:26.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.559+0000 7f5029e10640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f502415d830 con 0x7f502415cf80 2023-09-23T03:15:26.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.559+0000 7f50237fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f50240cdfa0 0x7f50240659f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:48104/0 (socket says 172.21.15.154:48104) 2023-09-23T03:15:26.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.559+0000 7f50237fe640 1 -- 172.21.15.154:0/2578642617 learned_addr learned my addr 172.21.15.154:0/2578642617 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:26.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.559+0000 7f501bfff640 1 --2- 172.21.15.154:0/2578642617 >> v2:172.21.15.154:3300/0 conn(0x7f50240b1400 0x7f50240ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:26.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.559+0000 7f5023fff640 1 --2- 172.21.15.154:0/2578642617 >> v2:172.21.15.190:3300/0 conn(0x7f502415cf80 0x7f502406c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:26.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.559+0000 7f5022ffd640 1 -- 172.21.15.154:0/2578642617 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1115794237 0 0) 0x7f50240ce8c0 con 0x7f50240cdfa0 2023-09-23T03:15:26.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.559+0000 7f5022ffd640 1 -- 172.21.15.154:0/2578642617 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5010003520 con 0x7f50240cdfa0 2023-09-23T03:15:26.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.559+0000 7f501bfff640 1 -- 172.21.15.154:0/2578642617 >> v2:172.21.15.190:3300/0 conn(0x7f502415cf80 msgr2=0x7f502406c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:26.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.559+0000 7f501bfff640 1 --2- 172.21.15.154:0/2578642617 >> v2:172.21.15.190:3300/0 conn(0x7f502415cf80 0x7f502406c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:26.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.559+0000 7f501bfff640 1 -- 172.21.15.154:0/2578642617 >> v1:172.21.15.154:6789/0 conn(0x7f50240cdfa0 legacy=0x7f50240659f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:26.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.559+0000 7f501bfff640 1 -- 172.21.15.154:0/2578642617 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f502415db10 con 0x7f50240b1400 2023-09-23T03:15:26.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.560+0000 7f501bfff640 1 --2- 172.21.15.154:0/2578642617 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50240b1400 0x7f50240ce380 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f50240707c0 tx=0x7f500c02b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51fa756f5c7f5b6b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:26.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.560+0000 7f5022ffd640 1 -- 172.21.15.154:0/2578642617 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500c031710 con 0x7f50240b1400 2023-09-23T03:15:26.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.560+0000 7f5022ffd640 1 -- 172.21.15.154:0/2578642617 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f500c02f050 con 0x7f50240b1400 2023-09-23T03:15:26.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.560+0000 7f5022ffd640 1 -- 172.21.15.154:0/2578642617 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500c031a90 con 0x7f50240b1400 2023-09-23T03:15:26.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.561+0000 7f5029e10640 1 -- 172.21.15.154:0/2578642617 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50240b1400 msgr2=0x7f50240ce380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:26.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.561+0000 7f5029e10640 1 --2- 172.21.15.154:0/2578642617 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50240b1400 0x7f50240ce380 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f50240707c0 tx=0x7f500c02b7b0 comp rx=0 tx=0).stop 2023-09-23T03:15:26.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.561+0000 7f5029e10640 1 -- 172.21.15.154:0/2578642617 shutdown_connections 2023-09-23T03:15:26.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.561+0000 7f5029e10640 1 --2- 172.21.15.154:0/2578642617 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50240b1400 0x7f50240ce380 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:26.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.561+0000 7f5029e10640 1 --2- 172.21.15.154:0/2578642617 >> v2:172.21.15.190:3300/0 conn(0x7f502415cf80 0x7f502406c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:26.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.561+0000 7f5029e10640 1 -- 172.21.15.154:0/2578642617 >> 172.21.15.154:0/2578642617 conn(0x7f50240b08e0 msgr2=0x7f50240b0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:26.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.561+0000 7f5029e10640 1 -- 172.21.15.154:0/2578642617 shutdown_connections 2023-09-23T03:15:26.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.561+0000 7f5029e10640 1 -- 172.21.15.154:0/2578642617 wait complete. 2023-09-23T03:15:26.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.563+0000 7f5029e10640 1 Processor -- start 2023-09-23T03:15:26.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.563+0000 7f5029e10640 1 -- start start 2023-09-23T03:15:26.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.564+0000 7f5029e10640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50240b1400 0x7f502409f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:26.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.564+0000 7f5029e10640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50240cdfa0 0x7f50240c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:26.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.564+0000 7f5029e10640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f502415cf80 0x7f50240c7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:26.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.564+0000 7f5029e10640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502409e270 con 0x7f502415cf80 2023-09-23T03:15:26.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.564+0000 7f5029e10640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50240c7820 con 0x7f50240cdfa0 2023-09-23T03:15:26.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.564+0000 7f5029e10640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50240c7b00 con 0x7f50240b1400 2023-09-23T03:15:26.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.564+0000 7f5023fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f502415cf80 0x7f50240c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:26.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.564+0000 7f5023fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f502415cf80 0x7f50240c7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:38202/0 (socket says 172.21.15.154:38202) 2023-09-23T03:15:26.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.564+0000 7f5023fff640 1 -- 172.21.15.154:0/2879491648 learned_addr learned my addr 172.21.15.154:0/2879491648 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:26.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.564+0000 7f50237fe640 1 --2- 172.21.15.154:0/2879491648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50240b1400 0x7f502409f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:26.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.565+0000 7f501bfff640 1 --2- 172.21.15.154:0/2879491648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50240cdfa0 0x7f50240c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:26.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.565+0000 7f5023fff640 1 -- 172.21.15.154:0/2879491648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50240b1400 msgr2=0x7f502409f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:26.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.565+0000 7f5023fff640 1 --2- 172.21.15.154:0/2879491648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50240b1400 0x7f502409f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:26.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.565+0000 7f5023fff640 1 -- 172.21.15.154:0/2879491648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50240cdfa0 msgr2=0x7f50240c88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:26.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.565+0000 7f5023fff640 1 --2- 172.21.15.154:0/2879491648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50240cdfa0 0x7f50240c88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:26.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.565+0000 7f5023fff640 1 -- 172.21.15.154:0/2879491648 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f500c009400 con 0x7f502415cf80 2023-09-23T03:15:26.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.565+0000 7f50237fe640 1 --2- 172.21.15.154:0/2879491648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50240b1400 0x7f502409f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:26.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.565+0000 7f5023fff640 1 --2- 172.21.15.154:0/2879491648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f502415cf80 0x7f50240c7120 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f501c007e60 tx=0x7f501c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:26.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.565+0000 7f5020ff9640 1 -- 172.21.15.154:0/2879491648 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501c014420 con 0x7f502415cf80 2023-09-23T03:15:26.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.565+0000 7f5020ff9640 1 -- 172.21.15.154:0/2879491648 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f501c0145a0 con 0x7f502415cf80 2023-09-23T03:15:26.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.565+0000 7f5029e10640 1 -- 172.21.15.154:0/2879491648 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50240c7de0 con 0x7f502415cf80 2023-09-23T03:15:26.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.565+0000 7f5020ff9640 1 -- 172.21.15.154:0/2879491648 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501c014950 con 0x7f502415cf80 2023-09-23T03:15:26.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.565+0000 7f5029e10640 1 -- 172.21.15.154:0/2879491648 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50240c83a0 con 0x7f502415cf80 2023-09-23T03:15:26.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.567+0000 7f5029e10640 1 -- 172.21.15.154:0/2879491648 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50240c8590 con 0x7f502415cf80 2023-09-23T03:15:26.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.567+0000 7f5020ff9640 1 -- 172.21.15.154:0/2879491648 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f501c014ab0 con 0x7f502415cf80 2023-09-23T03:15:26.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.568+0000 7f5020ff9640 1 --2- 172.21.15.154:0/2879491648 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f500406af30 0x7f500406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:26.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.569+0000 7f5020ff9640 1 -- 172.21.15.154:0/2879491648 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f501c08a390 con 0x7f502415cf80 2023-09-23T03:15:26.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.569+0000 7f50237fe640 1 --2- 172.21.15.154:0/2879491648 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f500406af30 0x7f500406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:26.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.569+0000 7f50237fe640 1 --2- 172.21.15.154:0/2879491648 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f500406af30 0x7f500406d3f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5014007fd0 tx=0x7f5014009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:26.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.573+0000 7f5020ff9640 1 -- 172.21.15.154:0/2879491648 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f501c0550e0 con 0x7f502415cf80 2023-09-23T03:15:26.601 INFO:tasks.ceph.mds.d.smithi190.stderr:2023-09-23T03:15:26.601+0000 7f8c03fbc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-23T03:15:26.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.803+0000 7f5029e10640 1 -- 172.21.15.154:0/2879491648 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5024073eb0 con 0x7f502415cf80 2023-09-23T03:15:26.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.804+0000 7f5020ff9640 1 -- 172.21.15.154:0/2879491648 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4921 (secure 0 0 0) 0x7f501c058700 con 0x7f502415cf80 2023-09-23T03:15:26.806 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:15:26.806 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2644365591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2644365591},{"type":"v1","addr":"172.21.15.190:6835","nonce":2644365591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/4145467852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4145467852},{"type":"v1","addr":"172.21.15.154:6835","nonce":4145467852}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1022302774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1022302774},{"type":"v1","addr":"172.21.15.154:6837","nonce":1022302774}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:15:20.686035+0000","modified":"2023-09-23T03:15:25.775313+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/3561509423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3561509423},{"type":"v1","addr":"172.21.15.190:6834","nonce":3561509423}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-09-23T03:15:26.806 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 7 2023-09-23T03:15:26.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.807+0000 7f5029e10640 1 -- 172.21.15.154:0/2879491648 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f500406af30 msgr2=0x7f500406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:26.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.807+0000 7f5029e10640 1 --2- 172.21.15.154:0/2879491648 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f500406af30 0x7f500406d3f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5014007fd0 tx=0x7f5014009040 comp rx=0 tx=0).stop 2023-09-23T03:15:26.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.807+0000 7f5029e10640 1 -- 172.21.15.154:0/2879491648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f502415cf80 msgr2=0x7f50240c7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:26.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.807+0000 7f5029e10640 1 --2- 172.21.15.154:0/2879491648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f502415cf80 0x7f50240c7120 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f501c007e60 tx=0x7f501c00e4b0 comp rx=0 tx=0).stop 2023-09-23T03:15:26.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.807+0000 7f5029e10640 1 -- 172.21.15.154:0/2879491648 shutdown_connections 2023-09-23T03:15:26.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.807+0000 7f5029e10640 1 --2- 172.21.15.154:0/2879491648 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f500406af30 0x7f500406d3f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:26.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.807+0000 7f5029e10640 1 --2- 172.21.15.154:0/2879491648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f502415cf80 0x7f50240c7120 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:26.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.807+0000 7f5029e10640 1 --2- 172.21.15.154:0/2879491648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50240cdfa0 0x7f50240c88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:26.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.807+0000 7f5029e10640 1 --2- 172.21.15.154:0/2879491648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50240b1400 0x7f502409f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:26.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.807+0000 7f5029e10640 1 -- 172.21.15.154:0/2879491648 >> 172.21.15.154:0/2879491648 conn(0x7f50240b08e0 msgr2=0x7f50240b0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:26.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.807+0000 7f5029e10640 1 -- 172.21.15.154:0/2879491648 shutdown_connections 2023-09-23T03:15:26.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:26.807+0000 7f5029e10640 1 -- 172.21.15.154:0/2879491648 wait complete. 2023-09-23T03:15:26.827 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:15:27.042 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.041+0000 7f1f2db25640 1 Processor -- start 2023-09-23T03:15:27.042 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.041+0000 7f1f2db25640 1 -- start start 2023-09-23T03:15:27.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.042+0000 7f1f2db25640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1f280cf040 0x7f1f281656e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:27.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.042+0000 7f1f2db25640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f2815ac30 con 0x7f1f280acce0 2023-09-23T03:15:27.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.043+0000 7f1f2db25640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f280b1f70 con 0x7f1f280b1b20 2023-09-23T03:15:27.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.043+0000 7f1f2db25640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f280b2e90 con 0x7f1f280cf040 2023-09-23T03:15:27.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.043+0000 7f1f277fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f1f280acce0 0x7f1f2815ed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:48110/0 (socket says 172.21.15.154:48110) 2023-09-23T03:15:27.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.043+0000 7f1f277fe640 1 -- 172.21.15.154:0/1938493668 learned_addr learned my addr 172.21.15.154:0/1938493668 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:27.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.043+0000 7f1f27fff640 1 --2- 172.21.15.154:0/1938493668 >> v2:172.21.15.190:3300/0 conn(0x7f1f280cf040 0x7f1f281656e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:27.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.043+0000 7f1f26ffd640 1 -- 172.21.15.154:0/1938493668 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2264712706 0 0) 0x7f1f18001290 con 0x7f1f280acce0 2023-09-23T03:15:27.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.043+0000 7f1f26ffd640 1 -- 172.21.15.154:0/1938493668 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f14003520 con 0x7f1f280acce0 2023-09-23T03:15:27.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.044+0000 7f1f26ffd640 1 -- 172.21.15.154:0/1938493668 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1291305475 0 0) 0x7f1f14003520 con 0x7f1f280acce0 2023-09-23T03:15:27.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.044+0000 7f1f26ffd640 1 -- 172.21.15.154:0/1938493668 >> v2:172.21.15.190:3300/0 conn(0x7f1f280cf040 msgr2=0x7f1f281656e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:27.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.044+0000 7f1f26ffd640 1 --2- 172.21.15.154:0/1938493668 >> v2:172.21.15.190:3300/0 conn(0x7f1f280cf040 0x7f1f281656e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:27.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.044+0000 7f1f26ffd640 1 -- 172.21.15.154:0/1938493668 >> v1:172.21.15.190:6789/0 conn(0x7f1f280b1b20 legacy=0x7f1f280ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:27.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.044+0000 7f1f26ffd640 1 -- 172.21.15.154:0/1938493668 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f280b2210 con 0x7f1f280acce0 2023-09-23T03:15:27.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.044+0000 7f1f26ffd640 1 -- 172.21.15.154:0/1938493668 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f1f18001980 con 0x7f1f280acce0 2023-09-23T03:15:27.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.044+0000 7f1f26ffd640 1 -- 172.21.15.154:0/1938493668 >> v1:172.21.15.154:6789/0 conn(0x7f1f280acce0 legacy=0x7f1f2815ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:27.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.044+0000 7f1f26ffd640 1 --2- 172.21.15.154:0/1938493668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f14003970 0x7f1f14003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:27.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.044+0000 7f1f26ffd640 1 --2- 172.21.15.154:0/1938493668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f140047c0 0x7f1f14004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:27.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.044+0000 7f1f26ffd640 1 --2- 172.21.15.154:0/1938493668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f14005180 0x7f1f1400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:27.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.044+0000 7f1f26ffd640 1 -- 172.21.15.154:0/1938493668 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f280b2e90 con 0x7f1f14003970 2023-09-23T03:15:27.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.044+0000 7f1f26ffd640 1 -- 172.21.15.154:0/1938493668 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1400c120 con 0x7f1f140047c0 2023-09-23T03:15:27.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.044+0000 7f1f26ffd640 1 -- 172.21.15.154:0/1938493668 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1400c2e0 con 0x7f1f14005180 2023-09-23T03:15:27.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.044+0000 7f1f277fe640 1 --2- 172.21.15.154:0/1938493668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f14003970 0x7f1f14003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:27.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.044+0000 7f1f1ffff640 1 --2- 172.21.15.154:0/1938493668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f140047c0 0x7f1f14004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:27.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.045+0000 7f1f277fe640 1 -- 172.21.15.154:0/1938493668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f14005180 msgr2=0x7f1f1400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:27.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.045+0000 7f1f27fff640 1 --2- 172.21.15.154:0/1938493668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f14005180 0x7f1f1400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:27.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.045+0000 7f1f277fe640 1 --2- 172.21.15.154:0/1938493668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f14005180 0x7f1f1400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:27.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.045+0000 7f1f277fe640 1 -- 172.21.15.154:0/1938493668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f140047c0 msgr2=0x7f1f14004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:27.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.045+0000 7f1f277fe640 1 --2- 172.21.15.154:0/1938493668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f140047c0 0x7f1f14004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:27.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.045+0000 7f1f277fe640 1 -- 172.21.15.154:0/1938493668 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f14003520 con 0x7f1f14003970 2023-09-23T03:15:27.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.045+0000 7f1f1ffff640 1 --2- 172.21.15.154:0/1938493668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f140047c0 0x7f1f14004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:27.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.045+0000 7f1f27fff640 1 -- 172.21.15.154:0/1938493668 reap_dead start 2023-09-23T03:15:27.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.046+0000 7f1f277fe640 1 --2- 172.21.15.154:0/1938493668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f14003970 0x7f1f14003d90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1f1800c210 tx=0x7f1f140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=182ae0086d097666 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:27.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.046+0000 7f1f26ffd640 1 -- 172.21.15.154:0/1938493668 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f18007270 con 0x7f1f14003970 2023-09-23T03:15:27.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.046+0000 7f1f26ffd640 1 -- 172.21.15.154:0/1938493668 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f180073d0 con 0x7f1f14003970 2023-09-23T03:15:27.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.046+0000 7f1f26ffd640 1 -- 172.21.15.154:0/1938493668 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f180057a0 con 0x7f1f14003970 2023-09-23T03:15:27.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.047+0000 7f1f2db25640 1 -- 172.21.15.154:0/1938493668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f14003970 msgr2=0x7f1f14003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:27.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.047+0000 7f1f2db25640 1 --2- 172.21.15.154:0/1938493668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f14003970 0x7f1f14003d90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1f1800c210 tx=0x7f1f140057f0 comp rx=0 tx=0).stop 2023-09-23T03:15:27.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.047+0000 7f1f2db25640 1 -- 172.21.15.154:0/1938493668 shutdown_connections 2023-09-23T03:15:27.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.047+0000 7f1f2db25640 1 --2- 172.21.15.154:0/1938493668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f14003970 0x7f1f14003d90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:27.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.047+0000 7f1f2db25640 1 -- 172.21.15.154:0/1938493668 >> 172.21.15.154:0/1938493668 conn(0x7f1f2806a0f0 msgr2=0x7f1f2806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:27.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.047+0000 7f1f2db25640 1 -- 172.21.15.154:0/1938493668 shutdown_connections 2023-09-23T03:15:27.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.047+0000 7f1f2db25640 1 -- 172.21.15.154:0/1938493668 wait complete. 2023-09-23T03:15:27.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.049+0000 7f1f2db25640 1 Processor -- start 2023-09-23T03:15:27.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.049+0000 7f1f2db25640 1 -- start start 2023-09-23T03:15:27.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.050+0000 7f1f2db25640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f14003970 0x7f1f2815d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:27.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.050+0000 7f1f2db25640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f28158eb0 0x7f1f281587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:27.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.050+0000 7f1f2db25640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f280aec10 0x7f1f280af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:27.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.050+0000 7f1f2db25640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2815a010 con 0x7f1f14003970 2023-09-23T03:15:27.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.050+0000 7f1f2db25640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f280af600 con 0x7f1f28158eb0 2023-09-23T03:15:27.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.050+0000 7f1f2db25640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f28158d30 con 0x7f1f280aec10 2023-09-23T03:15:27.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.050+0000 7f1f277fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f14003970 0x7f1f2815d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:27.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.050+0000 7f1f277fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f14003970 0x7f1f2815d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:38222/0 (socket says 172.21.15.154:38222) 2023-09-23T03:15:27.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.050+0000 7f1f277fe640 1 -- 172.21.15.154:0/56089169 learned_addr learned my addr 172.21.15.154:0/56089169 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:27.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.051+0000 7f1f277fe640 1 -- 172.21.15.154:0/56089169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f280aec10 msgr2=0x7f1f280af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:27.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.051+0000 7f1f277fe640 1 --2- 172.21.15.154:0/56089169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f280aec10 0x7f1f280af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:27.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.051+0000 7f1f277fe640 1 -- 172.21.15.154:0/56089169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f28158eb0 msgr2=0x7f1f281587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:27.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.051+0000 7f1f277fe640 1 --2- 172.21.15.154:0/56089169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f28158eb0 0x7f1f281587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:27.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.051+0000 7f1f277fe640 1 -- 172.21.15.154:0/56089169 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f28159390 con 0x7f1f14003970 2023-09-23T03:15:27.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.051+0000 7f1f277fe640 1 --2- 172.21.15.154:0/56089169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f14003970 0x7f1f2815d720 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1f18000ef0 tx=0x7f1f18001d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:27.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.051+0000 7f1f24ff9640 1 -- 172.21.15.154:0/56089169 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1803c070 con 0x7f1f14003970 2023-09-23T03:15:27.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.051+0000 7f1f24ff9640 1 -- 172.21.15.154:0/56089169 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f18001290 con 0x7f1f14003970 2023-09-23T03:15:27.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.051+0000 7f1f24ff9640 1 -- 172.21.15.154:0/56089169 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f18000f70 con 0x7f1f14003970 2023-09-23T03:15:27.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.051+0000 7f1f2db25640 1 -- 172.21.15.154:0/56089169 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f28159670 con 0x7f1f14003970 2023-09-23T03:15:27.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.051+0000 7f1f2db25640 1 -- 172.21.15.154:0/56089169 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f28159c30 con 0x7f1f14003970 2023-09-23T03:15:27.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.052+0000 7f1f2db25640 1 -- 172.21.15.154:0/56089169 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f28073eb0 con 0x7f1f14003970 2023-09-23T03:15:27.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.053+0000 7f1f24ff9640 1 -- 172.21.15.154:0/56089169 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f1f18032070 con 0x7f1f14003970 2023-09-23T03:15:27.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.054+0000 7f1f24ff9640 1 --2- 172.21.15.154:0/56089169 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1f0806af30 0x7f1f0806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:27.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.054+0000 7f1f24ff9640 1 -- 172.21.15.154:0/56089169 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1f180aa9c0 con 0x7f1f14003970 2023-09-23T03:15:27.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.055+0000 7f1f1ffff640 1 --2- 172.21.15.154:0/56089169 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1f0806af30 0x7f1f0806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:27.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.055+0000 7f1f1ffff640 1 --2- 172.21.15.154:0/56089169 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1f0806af30 0x7f1f0806d3f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1f10007fd0 tx=0x7f1f10009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:27.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.059+0000 7f1f24ff9640 1 -- 172.21.15.154:0/56089169 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f1f18075700 con 0x7f1f14003970 2023-09-23T03:15:27.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.251+0000 7f1f2db25640 1 -- 172.21.15.154:0/56089169 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f2815ac30 con 0x7f1f14003970 2023-09-23T03:15:27.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.253+0000 7f1f24ff9640 1 -- 172.21.15.154:0/56089169 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18382 (secure 0 0 0) 0x7f1f2815ac30 con 0x7f1f14003970 2023-09-23T03:15:27.254 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:15:27.255 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":24,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:15:25.775060+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:15:18.575849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T03:15:19.707212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T03:15:23.337138+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:15:27.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.255+0000 7f1f2db25640 1 -- 172.21.15.154:0/56089169 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1f0806af30 msgr2=0x7f1f0806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:27.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.255+0000 7f1f2db25640 1 --2- 172.21.15.154:0/56089169 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1f0806af30 0x7f1f0806d3f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1f10007fd0 tx=0x7f1f10009040 comp rx=0 tx=0).stop 2023-09-23T03:15:27.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.255+0000 7f1f2db25640 1 -- 172.21.15.154:0/56089169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f14003970 msgr2=0x7f1f2815d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:27.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.255+0000 7f1f2db25640 1 --2- 172.21.15.154:0/56089169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f14003970 0x7f1f2815d720 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1f18000ef0 tx=0x7f1f18001d10 comp rx=0 tx=0).stop 2023-09-23T03:15:27.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.256+0000 7f1f2db25640 1 -- 172.21.15.154:0/56089169 shutdown_connections 2023-09-23T03:15:27.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.256+0000 7f1f2db25640 1 --2- 172.21.15.154:0/56089169 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1f0806af30 0x7f1f0806d3f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:27.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.256+0000 7f1f2db25640 1 --2- 172.21.15.154:0/56089169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f280aec10 0x7f1f280af0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:27.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.256+0000 7f1f2db25640 1 --2- 172.21.15.154:0/56089169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f28158eb0 0x7f1f281587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:27.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.256+0000 7f1f2db25640 1 --2- 172.21.15.154:0/56089169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f14003970 0x7f1f2815d720 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:27.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.256+0000 7f1f2db25640 1 -- 172.21.15.154:0/56089169 >> 172.21.15.154:0/56089169 conn(0x7f1f2806a0f0 msgr2=0x7f1f280ce900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:27.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.256+0000 7f1f2db25640 1 -- 172.21.15.154:0/56089169 shutdown_connections 2023-09-23T03:15:27.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.256+0000 7f1f2db25640 1 -- 172.21.15.154:0/56089169 wait complete. 2023-09-23T03:15:27.276 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:15:27.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.495+0000 7f76e7f9e640 1 Processor -- start 2023-09-23T03:15:27.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.495+0000 7f76e7f9e640 1 -- start start 2023-09-23T03:15:27.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.496+0000 7f76e7f9e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76e00ada30 0x7f76e00ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:27.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.496+0000 7f76e7f9e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f76e00cf130 0x7f76e015d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:27.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.496+0000 7f76e7f9e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76e006ceb0 con 0x7f76e00b2fe0 2023-09-23T03:15:27.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.496+0000 7f76e7f9e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76e00ade10 con 0x7f76e00cf130 2023-09-23T03:15:27.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.496+0000 7f76e7f9e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e0159c90 con 0x7f76e00ada30 2023-09-23T03:15:27.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.497+0000 7f76e5d3a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76e00ada30 0x7f76e00ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:27.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.497+0000 7f76e5d3a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76e00ada30 0x7f76e00ccaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:49128/0 (socket says 172.21.15.154:49128) 2023-09-23T03:15:27.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.497+0000 7f76e5d3a640 1 -- 172.21.15.154:0/3298250461 learned_addr learned my addr 172.21.15.154:0/3298250461 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:27.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.497+0000 7f76e5539640 1 --1- 172.21.15.154:0/3298250461 >> v1:172.21.15.190:6789/0 conn(0x7f76e00b2fe0 0x7f76e00cea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.154:41480/0 (socket says 172.21.15.154:41480) 2023-09-23T03:15:27.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.497+0000 7f76e653b640 1 --2- 172.21.15.154:0/3298250461 >> v2:172.21.15.190:3300/0 conn(0x7f76e00cf130 0x7f76e015d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:27.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.498+0000 7f76e5d3a640 1 -- 172.21.15.154:0/3298250461 >> v2:172.21.15.190:3300/0 conn(0x7f76e00cf130 msgr2=0x7f76e015d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:27.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.498+0000 7f76e5d3a640 1 --2- 172.21.15.154:0/3298250461 >> v2:172.21.15.190:3300/0 conn(0x7f76e00cf130 0x7f76e015d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:27.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.498+0000 7f76e5d3a640 1 -- 172.21.15.154:0/3298250461 >> v1:172.21.15.190:6789/0 conn(0x7f76e00b2fe0 legacy=0x7f76e00cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:27.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.498+0000 7f76e5d3a640 1 -- 172.21.15.154:0/3298250461 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e006c110 con 0x7f76e00ada30 2023-09-23T03:15:27.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.499+0000 7f76e5d3a640 1 --2- 172.21.15.154:0/3298250461 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76e00ada30 0x7f76e00ccaf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f76dc009850 tx=0x7f76dc02db70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b28c82458363e37 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:27.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.500+0000 7f76e4d38640 1 -- 172.21.15.154:0/3298250461 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76dc004230 con 0x7f76e00ada30 2023-09-23T03:15:27.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.500+0000 7f76e4d38640 1 -- 172.21.15.154:0/3298250461 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76dc004390 con 0x7f76e00ada30 2023-09-23T03:15:27.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.500+0000 7f76e4d38640 1 -- 172.21.15.154:0/3298250461 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76dc02e920 con 0x7f76e00ada30 2023-09-23T03:15:27.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.500+0000 7f76e7f9e640 1 -- 172.21.15.154:0/3298250461 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76e00ada30 msgr2=0x7f76e00ccaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:27.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.500+0000 7f76e7f9e640 1 --2- 172.21.15.154:0/3298250461 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76e00ada30 0x7f76e00ccaf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f76dc009850 tx=0x7f76dc02db70 comp rx=0 tx=0).stop 2023-09-23T03:15:27.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.500+0000 7f76e7f9e640 1 -- 172.21.15.154:0/3298250461 shutdown_connections 2023-09-23T03:15:27.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.500+0000 7f76e7f9e640 1 --2- 172.21.15.154:0/3298250461 >> v2:172.21.15.190:3300/0 conn(0x7f76e00cf130 0x7f76e015d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:27.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.500+0000 7f76e7f9e640 1 --2- 172.21.15.154:0/3298250461 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76e00ada30 0x7f76e00ccaf0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:27.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.500+0000 7f76e7f9e640 1 -- 172.21.15.154:0/3298250461 >> 172.21.15.154:0/3298250461 conn(0x7f76e006a0f0 msgr2=0x7f76e006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:27.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.500+0000 7f76e7f9e640 1 -- 172.21.15.154:0/3298250461 shutdown_connections 2023-09-23T03:15:27.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.500+0000 7f76e7f9e640 1 -- 172.21.15.154:0/3298250461 wait complete. 2023-09-23T03:15:27.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.502+0000 7f76e7f9e640 1 Processor -- start 2023-09-23T03:15:27.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.502+0000 7f76e7f9e640 1 -- start start 2023-09-23T03:15:27.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.503+0000 7f76e7f9e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76e00ada30 0x7f76e0152e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:27.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.503+0000 7f76e7f9e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76e00b2fe0 0x7f76e00c2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:27.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.503+0000 7f76e7f9e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76e00cf130 0x7f76e00c2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:27.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.503+0000 7f76e7f9e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e00c5100 con 0x7f76e00b2fe0 2023-09-23T03:15:27.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.503+0000 7f76e7f9e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e00b2090 con 0x7f76e00cf130 2023-09-23T03:15:27.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.503+0000 7f76e7f9e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e00c4b80 con 0x7f76e00ada30 2023-09-23T03:15:27.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.504+0000 7f76e5539640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76e00b2fe0 0x7f76e00c2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:27.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.504+0000 7f76e5539640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76e00b2fe0 0x7f76e00c2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:38226/0 (socket says 172.21.15.154:38226) 2023-09-23T03:15:27.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.504+0000 7f76e5539640 1 -- 172.21.15.154:0/2569588755 learned_addr learned my addr 172.21.15.154:0/2569588755 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:27.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.504+0000 7f76e5d3a640 1 --2- 172.21.15.154:0/2569588755 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76e00ada30 0x7f76e0152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:27.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.504+0000 7f76e653b640 1 --2- 172.21.15.154:0/2569588755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76e00cf130 0x7f76e00c2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:27.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.504+0000 7f76e5539640 1 -- 172.21.15.154:0/2569588755 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76e00ada30 msgr2=0x7f76e0152e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:27.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.504+0000 7f76e5539640 1 --2- 172.21.15.154:0/2569588755 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76e00ada30 0x7f76e0152e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:27.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.504+0000 7f76e5539640 1 -- 172.21.15.154:0/2569588755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76e00cf130 msgr2=0x7f76e00c2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:27.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.504+0000 7f76e5539640 1 --2- 172.21.15.154:0/2569588755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76e00cf130 0x7f76e00c2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:27.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.504+0000 7f76e5539640 1 -- 172.21.15.154:0/2569588755 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76dc009460 con 0x7f76e00b2fe0 2023-09-23T03:15:27.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.504+0000 7f76e5539640 1 --2- 172.21.15.154:0/2569588755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76e00b2fe0 0x7f76e00c2860 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f76d000c400 tx=0x7f76d000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:27.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.505+0000 7f76d67fc640 1 -- 172.21.15.154:0/2569588755 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d00074e0 con 0x7f76e00b2fe0 2023-09-23T03:15:27.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.505+0000 7f76d67fc640 1 -- 172.21.15.154:0/2569588755 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d0007640 con 0x7f76e00b2fe0 2023-09-23T03:15:27.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.505+0000 7f76d67fc640 1 -- 172.21.15.154:0/2569588755 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d0004a00 con 0x7f76e00b2fe0 2023-09-23T03:15:27.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.505+0000 7f76e7f9e640 1 -- 172.21.15.154:0/2569588755 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76e00c4e20 con 0x7f76e00b2fe0 2023-09-23T03:15:27.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.505+0000 7f76e7f9e640 1 -- 172.21.15.154:0/2569588755 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76e015a9a0 con 0x7f76e00b2fe0 2023-09-23T03:15:27.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.506+0000 7f76e7f9e640 1 -- 172.21.15.154:0/2569588755 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76e00b1c10 con 0x7f76e00b2fe0 2023-09-23T03:15:27.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.507+0000 7f76d67fc640 1 -- 172.21.15.154:0/2569588755 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f76d0004b60 con 0x7f76e00b2fe0 2023-09-23T03:15:27.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.508+0000 7f76d67fc640 1 --2- 172.21.15.154:0/2569588755 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f76c406af30 0x7f76c406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:27.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.508+0000 7f76d67fc640 1 -- 172.21.15.154:0/2569588755 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f76d0089300 con 0x7f76e00b2fe0 2023-09-23T03:15:27.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.508+0000 7f76e5d3a640 1 --2- 172.21.15.154:0/2569588755 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f76c406af30 0x7f76c406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:27.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.509+0000 7f76e5d3a640 1 --2- 172.21.15.154:0/2569588755 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f76c406af30 0x7f76c406d3f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f76dc037000 tx=0x7f76dc032000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:27.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.512+0000 7f76d67fc640 1 -- 172.21.15.154:0/2569588755 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f76e00b1c10 con 0x7f76e00b2fe0 2023-09-23T03:15:27.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.701+0000 7f76e7f9e640 1 -- 172.21.15.154:0/2569588755 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76e006ceb0 con 0x7f76e00b2fe0 2023-09-23T03:15:27.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.704+0000 7f76d67fc640 1 -- 172.21.15.154:0/2569588755 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18382 (secure 0 0 0) 0x7f76e006ceb0 con 0x7f76e00b2fe0 2023-09-23T03:15:27.704 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:15:27.705 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":24,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:15:25.775060+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:15:18.575849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T03:15:19.707212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T03:15:23.337138+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:15:27.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.706+0000 7f76e7f9e640 1 -- 172.21.15.154:0/2569588755 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f76c406af30 msgr2=0x7f76c406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:27.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.706+0000 7f76e7f9e640 1 --2- 172.21.15.154:0/2569588755 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f76c406af30 0x7f76c406d3f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f76dc037000 tx=0x7f76dc032000 comp rx=0 tx=0).stop 2023-09-23T03:15:27.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.706+0000 7f76e7f9e640 1 -- 172.21.15.154:0/2569588755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76e00b2fe0 msgr2=0x7f76e00c2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:27.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.706+0000 7f76e7f9e640 1 --2- 172.21.15.154:0/2569588755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76e00b2fe0 0x7f76e00c2860 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f76d000c400 tx=0x7f76d000c8c0 comp rx=0 tx=0).stop 2023-09-23T03:15:27.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.706+0000 7f76e7f9e640 1 -- 172.21.15.154:0/2569588755 shutdown_connections 2023-09-23T03:15:27.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.706+0000 7f76e7f9e640 1 --2- 172.21.15.154:0/2569588755 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f76c406af30 0x7f76c406d3f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:27.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.706+0000 7f76e7f9e640 1 --2- 172.21.15.154:0/2569588755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76e00cf130 0x7f76e00c2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:27.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.706+0000 7f76e7f9e640 1 --2- 172.21.15.154:0/2569588755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76e00b2fe0 0x7f76e00c2860 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:27.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.706+0000 7f76e7f9e640 1 --2- 172.21.15.154:0/2569588755 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76e00ada30 0x7f76e0152e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:27.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.706+0000 7f76e7f9e640 1 -- 172.21.15.154:0/2569588755 >> 172.21.15.154:0/2569588755 conn(0x7f76e006a0f0 msgr2=0x7f76e00ae8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:27.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.707+0000 7f76e7f9e640 1 -- 172.21.15.154:0/2569588755 shutdown_connections 2023-09-23T03:15:27.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.707+0000 7f76e7f9e640 1 -- 172.21.15.154:0/2569588755 wait complete. 2023-09-23T03:15:27.738 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:15:27.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.955+0000 7f4122b93640 1 Processor -- start 2023-09-23T03:15:27.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.955+0000 7f4122b93640 1 -- start start 2023-09-23T03:15:27.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.956+0000 7f4122b93640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f411c06c7a0 0x7f411c15eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:27.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.956+0000 7f4122b93640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f411c15ab00 0x7f411c15afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:27.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.956+0000 7f4122b93640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f411c06bff0 con 0x7f411c06fdc0 2023-09-23T03:15:27.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.956+0000 7f4122b93640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f411c155070 con 0x7f411c06c7a0 2023-09-23T03:15:27.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.956+0000 7f4122b93640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f411c15e0f0 con 0x7f411c15ab00 2023-09-23T03:15:27.958 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.957+0000 7f4122392640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f411c06fdc0 0x7f411c0701a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:48116/0 (socket says 172.21.15.154:48116) 2023-09-23T03:15:27.958 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.957+0000 7f4122392640 1 -- 172.21.15.154:0/2593121126 learned_addr learned my addr 172.21.15.154:0/2593121126 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:27.958 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.957+0000 7f4121b91640 1 --2- 172.21.15.154:0/2593121126 >> v2:172.21.15.154:3300/0 conn(0x7f411c06c7a0 0x7f411c15eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:27.958 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.957+0000 7f4121390640 1 --2- 172.21.15.154:0/2593121126 >> v2:172.21.15.190:3300/0 conn(0x7f411c15ab00 0x7f411c15afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:27.958 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.957+0000 7f4120b8f640 1 -- 172.21.15.154:0/2593121126 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3283654875 0 0) 0x7f411c06bff0 con 0x7f411c06fdc0 2023-09-23T03:15:27.958 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.957+0000 7f4120b8f640 1 -- 172.21.15.154:0/2593121126 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4108003520 con 0x7f411c06fdc0 2023-09-23T03:15:27.958 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.957+0000 7f4121b91640 1 -- 172.21.15.154:0/2593121126 >> v2:172.21.15.190:3300/0 conn(0x7f411c15ab00 msgr2=0x7f411c15afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:27.958 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.957+0000 7f4121b91640 1 --2- 172.21.15.154:0/2593121126 >> v2:172.21.15.190:3300/0 conn(0x7f411c15ab00 0x7f411c15afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:27.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.957+0000 7f4121b91640 1 -- 172.21.15.154:0/2593121126 >> v1:172.21.15.154:6789/0 conn(0x7f411c06fdc0 legacy=0x7f411c0701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:27.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.957+0000 7f4121b91640 1 -- 172.21.15.154:0/2593121126 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411c06cb80 con 0x7f411c06c7a0 2023-09-23T03:15:27.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.959+0000 7f4121b91640 1 --2- 172.21.15.154:0/2593121126 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f411c06c7a0 0x7f411c15eb00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f411c0cba90 tx=0x7f411802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1bf45c3a22518 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:27.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.959+0000 7f4120b8f640 1 -- 172.21.15.154:0/2593121126 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4118031830 con 0x7f411c06c7a0 2023-09-23T03:15:27.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.959+0000 7f4120b8f640 1 -- 172.21.15.154:0/2593121126 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4118031990 con 0x7f411c06c7a0 2023-09-23T03:15:27.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.959+0000 7f4120b8f640 1 -- 172.21.15.154:0/2593121126 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4118031d20 con 0x7f411c06c7a0 2023-09-23T03:15:27.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.959+0000 7f4122b93640 1 -- 172.21.15.154:0/2593121126 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f411c06c7a0 msgr2=0x7f411c15eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:27.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.959+0000 7f4122b93640 1 --2- 172.21.15.154:0/2593121126 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f411c06c7a0 0x7f411c15eb00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f411c0cba90 tx=0x7f411802b8d0 comp rx=0 tx=0).stop 2023-09-23T03:15:27.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.959+0000 7f4122b93640 1 -- 172.21.15.154:0/2593121126 shutdown_connections 2023-09-23T03:15:27.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.959+0000 7f4122b93640 1 --2- 172.21.15.154:0/2593121126 >> v2:172.21.15.190:3300/0 conn(0x7f411c15ab00 0x7f411c15afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:27.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.959+0000 7f4122b93640 1 --2- 172.21.15.154:0/2593121126 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f411c06c7a0 0x7f411c15eb00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:27.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.959+0000 7f4122b93640 1 -- 172.21.15.154:0/2593121126 >> 172.21.15.154:0/2593121126 conn(0x7f411c06a390 msgr2=0x7f411c0afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:27.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.960+0000 7f4122b93640 1 -- 172.21.15.154:0/2593121126 shutdown_connections 2023-09-23T03:15:27.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.960+0000 7f4122b93640 1 -- 172.21.15.154:0/2593121126 wait complete. 2023-09-23T03:15:27.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.961+0000 7f4122b93640 1 Processor -- start 2023-09-23T03:15:27.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.961+0000 7f4122b93640 1 -- start start 2023-09-23T03:15:27.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.962+0000 7f4122b93640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f411c06c7a0 0x7f411c0b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:27.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.962+0000 7f4122b93640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f411c06fdc0 0x7f411c1543e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:27.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.962+0000 7f4122b93640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f411c15ab00 0x7f411c1f2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:27.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.962+0000 7f4122b93640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c154920 con 0x7f411c06fdc0 2023-09-23T03:15:27.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.962+0000 7f4122b93640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c1f3c00 con 0x7f411c06c7a0 2023-09-23T03:15:27.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.962+0000 7f4122b93640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f411c0b19b0 con 0x7f411c15ab00 2023-09-23T03:15:27.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.963+0000 7f4121390640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f411c06fdc0 0x7f411c1543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:27.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.963+0000 7f4121390640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f411c06fdc0 0x7f411c1543e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:38248/0 (socket says 172.21.15.154:38248) 2023-09-23T03:15:27.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.963+0000 7f4121390640 1 -- 172.21.15.154:0/1219550648 learned_addr learned my addr 172.21.15.154:0/1219550648 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:27.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.963+0000 7f4121b91640 1 --2- 172.21.15.154:0/1219550648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f411c06c7a0 0x7f411c0b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:27.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.963+0000 7f4121390640 1 -- 172.21.15.154:0/1219550648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f411c15ab00 msgr2=0x7f411c1f2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:27.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.963+0000 7f4121390640 1 --2- 172.21.15.154:0/1219550648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f411c15ab00 0x7f411c1f2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:27.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.963+0000 7f4121390640 1 -- 172.21.15.154:0/1219550648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f411c06c7a0 msgr2=0x7f411c0b2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:27.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.963+0000 7f4121390640 1 --2- 172.21.15.154:0/1219550648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f411c06c7a0 0x7f411c0b2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:27.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.963+0000 7f4121390640 1 -- 172.21.15.154:0/1219550648 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4118009400 con 0x7f411c06fdc0 2023-09-23T03:15:27.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.963+0000 7f4121390640 1 --2- 172.21.15.154:0/1219550648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f411c06fdc0 0x7f411c1543e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f410c00e870 tx=0x7f410c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:27.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.964+0000 7f41127fc640 1 -- 172.21.15.154:0/1219550648 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c0094f0 con 0x7f411c06fdc0 2023-09-23T03:15:27.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.964+0000 7f41127fc640 1 -- 172.21.15.154:0/1219550648 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410c00c940 con 0x7f411c06fdc0 2023-09-23T03:15:27.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.964+0000 7f41127fc640 1 -- 172.21.15.154:0/1219550648 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c00ccf0 con 0x7f411c06fdc0 2023-09-23T03:15:27.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.964+0000 7f4122b93640 1 -- 172.21.15.154:0/1219550648 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f411c0b1c90 con 0x7f411c06fdc0 2023-09-23T03:15:27.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.964+0000 7f4122b93640 1 -- 172.21.15.154:0/1219550648 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f411c0b2250 con 0x7f411c06fdc0 2023-09-23T03:15:27.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.965+0000 7f4122b93640 1 -- 172.21.15.154:0/1219550648 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f411c0736b0 con 0x7f411c06fdc0 2023-09-23T03:15:27.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.966+0000 7f41127fc640 1 -- 172.21.15.154:0/1219550648 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f410c013470 con 0x7f411c06fdc0 2023-09-23T03:15:27.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.967+0000 7f41127fc640 1 --2- 172.21.15.154:0/1219550648 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f410006af30 0x7f410006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:27.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.967+0000 7f41127fc640 1 -- 172.21.15.154:0/1219550648 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f410c089b10 con 0x7f411c06fdc0 2023-09-23T03:15:27.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.967+0000 7f4121b91640 1 --2- 172.21.15.154:0/1219550648 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f410006af30 0x7f410006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:27.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.968+0000 7f4121b91640 1 --2- 172.21.15.154:0/1219550648 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f410006af30 0x7f410006d3f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4118036000 tx=0x7f4118008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:27.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:27.972+0000 7f41127fc640 1 -- 172.21.15.154:0/1219550648 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f410c054860 con 0x7f411c06fdc0 2023-09-23T03:15:28.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.197+0000 7f4122b93640 1 -- 172.21.15.154:0/1219550648 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f411c04e920 con 0x7f411c06fdc0 2023-09-23T03:15:28.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.198+0000 7f41127fc640 1 -- 172.21.15.154:0/1219550648 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4921 (secure 0 0 0) 0x7f410c017090 con 0x7f411c06fdc0 2023-09-23T03:15:28.199 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:15:28.199 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2644365591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2644365591},{"type":"v1","addr":"172.21.15.190:6835","nonce":2644365591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/4145467852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4145467852},{"type":"v1","addr":"172.21.15.154:6835","nonce":4145467852}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1022302774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1022302774},{"type":"v1","addr":"172.21.15.154:6837","nonce":1022302774}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:15:20.686035+0000","modified":"2023-09-23T03:15:25.775313+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/3561509423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3561509423},{"type":"v1","addr":"172.21.15.190:6834","nonce":3561509423}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-09-23T03:15:28.199 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 7 2023-09-23T03:15:28.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.201+0000 7f4122b93640 1 -- 172.21.15.154:0/1219550648 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f410006af30 msgr2=0x7f410006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:28.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.201+0000 7f4122b93640 1 --2- 172.21.15.154:0/1219550648 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f410006af30 0x7f410006d3f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4118036000 tx=0x7f4118008030 comp rx=0 tx=0).stop 2023-09-23T03:15:28.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.201+0000 7f4122b93640 1 -- 172.21.15.154:0/1219550648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f411c06fdc0 msgr2=0x7f411c1543e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:28.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.201+0000 7f4122b93640 1 --2- 172.21.15.154:0/1219550648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f411c06fdc0 0x7f411c1543e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f410c00e870 tx=0x7f410c00ed30 comp rx=0 tx=0).stop 2023-09-23T03:15:28.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.201+0000 7f4122b93640 1 -- 172.21.15.154:0/1219550648 shutdown_connections 2023-09-23T03:15:28.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.201+0000 7f4122b93640 1 --2- 172.21.15.154:0/1219550648 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f410006af30 0x7f410006d3f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:28.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.201+0000 7f4122b93640 1 --2- 172.21.15.154:0/1219550648 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f411c15ab00 0x7f411c1f2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:28.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.201+0000 7f4122b93640 1 --2- 172.21.15.154:0/1219550648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f411c06fdc0 0x7f411c1543e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:28.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.201+0000 7f4122b93640 1 --2- 172.21.15.154:0/1219550648 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f411c06c7a0 0x7f411c0b2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:28.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.201+0000 7f4122b93640 1 -- 172.21.15.154:0/1219550648 >> 172.21.15.154:0/1219550648 conn(0x7f411c06a390 msgr2=0x7f411c0b0540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:28.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.201+0000 7f4122b93640 1 -- 172.21.15.154:0/1219550648 shutdown_connections 2023-09-23T03:15:28.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.201+0000 7f4122b93640 1 -- 172.21.15.154:0/1219550648 wait complete. 2023-09-23T03:15:28.220 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:15:28.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.439+0000 7fceb3fff640 1 Processor -- start 2023-09-23T03:15:28.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.440+0000 7fceb3fff640 1 -- start start 2023-09-23T03:15:28.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.441+0000 7fceb3fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fceb40cf500 0x7fceb40ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:28.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.441+0000 7fceb3fff640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fceb406be10 con 0x7fceb415cc00 2023-09-23T03:15:28.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.441+0000 7fceb3fff640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fceb40d17f0 con 0x7fceb40cf8e0 2023-09-23T03:15:28.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.441+0000 7fceb3fff640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fceb40d0470 con 0x7fceb40cf500 2023-09-23T03:15:28.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.442+0000 7fceb37fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fceb415cc00 0x7fceb40b1430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:48130/0 (socket says 172.21.15.154:48130) 2023-09-23T03:15:28.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.442+0000 7fceb37fe640 1 -- 172.21.15.154:0/3322223775 learned_addr learned my addr 172.21.15.154:0/3322223775 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:28.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.442+0000 7fceb2ffd640 1 --2- 172.21.15.154:0/3322223775 >> v2:172.21.15.190:3300/0 conn(0x7fceb40cf500 0x7fceb40ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:28.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.442+0000 7fceb27fc640 1 -- 172.21.15.154:0/3322223775 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 574582764 0 0) 0x7fceb406be10 con 0x7fceb415cc00 2023-09-23T03:15:28.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.442+0000 7fceb27fc640 1 -- 172.21.15.154:0/3322223775 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcea0003520 con 0x7fceb415cc00 2023-09-23T03:15:28.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.443+0000 7fceb27fc640 1 -- 172.21.15.154:0/3322223775 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 432211312 0 0) 0x7fcea0003520 con 0x7fceb415cc00 2023-09-23T03:15:28.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.443+0000 7fceb27fc640 1 -- 172.21.15.154:0/3322223775 >> v2:172.21.15.190:3300/0 conn(0x7fceb40cf500 msgr2=0x7fceb40ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:28.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.443+0000 7fceb27fc640 1 --2- 172.21.15.154:0/3322223775 >> v2:172.21.15.190:3300/0 conn(0x7fceb40cf500 0x7fceb40ce050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:28.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.443+0000 7fceb27fc640 1 -- 172.21.15.154:0/3322223775 >> v1:172.21.15.190:6789/0 conn(0x7fceb40cf8e0 legacy=0x7fceb415c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:28.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.443+0000 7fceb27fc640 1 -- 172.21.15.154:0/3322223775 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb40d0750 con 0x7fceb415cc00 2023-09-23T03:15:28.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.443+0000 7fceb27fc640 1 -- 172.21.15.154:0/3322223775 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fceac001670 con 0x7fceb415cc00 2023-09-23T03:15:28.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.443+0000 7fceb27fc640 1 -- 172.21.15.154:0/3322223775 >> v1:172.21.15.154:6789/0 conn(0x7fceb415cc00 legacy=0x7fceb40b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:28.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.443+0000 7fceb27fc640 1 --2- 172.21.15.154:0/3322223775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcea0003970 0x7fcea0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:28.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.443+0000 7fceb27fc640 1 --2- 172.21.15.154:0/3322223775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcea00047c0 0x7fcea0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:28.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.443+0000 7fceb27fc640 1 --2- 172.21.15.154:0/3322223775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcea0005180 0x7fcea000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:28.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.443+0000 7fceb27fc640 1 -- 172.21.15.154:0/3322223775 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb40d0470 con 0x7fcea0003970 2023-09-23T03:15:28.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.443+0000 7fceb27fc640 1 -- 172.21.15.154:0/3322223775 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea000c120 con 0x7fcea00047c0 2023-09-23T03:15:28.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.443+0000 7fceb27fc640 1 -- 172.21.15.154:0/3322223775 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea000c2e0 con 0x7fcea0005180 2023-09-23T03:15:28.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.443+0000 7fceb2ffd640 1 --2- 172.21.15.154:0/3322223775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcea0003970 0x7fcea0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:28.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.443+0000 7fceb37fe640 1 --2- 172.21.15.154:0/3322223775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcea0005180 0x7fcea000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:28.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.444+0000 7fceaa7fc640 1 --2- 172.21.15.154:0/3322223775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcea00047c0 0x7fcea0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:28.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.444+0000 7fceb37fe640 1 -- 172.21.15.154:0/3322223775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcea00047c0 msgr2=0x7fcea0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:28.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.444+0000 7fceb37fe640 1 --2- 172.21.15.154:0/3322223775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcea00047c0 0x7fcea0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:28.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.444+0000 7fceb37fe640 1 -- 172.21.15.154:0/3322223775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcea0003970 msgr2=0x7fcea0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:28.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.444+0000 7fceb37fe640 1 --2- 172.21.15.154:0/3322223775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcea0003970 0x7fcea0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:28.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.444+0000 7fceb37fe640 1 -- 172.21.15.154:0/3322223775 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea0003520 con 0x7fcea0005180 2023-09-23T03:15:28.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.444+0000 7fceb2ffd640 1 --2- 172.21.15.154:0/3322223775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcea0003970 0x7fcea0003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:15:28.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.445+0000 7fceb37fe640 1 -- 172.21.15.154:0/3322223775 reap_dead start 2023-09-23T03:15:28.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.445+0000 7fceb37fe640 1 --2- 172.21.15.154:0/3322223775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcea0005180 0x7fcea000b980 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fceac00c190 tx=0x7fcea00057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ffb6f1f688ca6a8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:28.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.446+0000 7fceb27fc640 1 -- 172.21.15.154:0/3322223775 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceac005ad0 con 0x7fcea0005180 2023-09-23T03:15:28.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.446+0000 7fceb27fc640 1 -- 172.21.15.154:0/3322223775 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceac005c70 con 0x7fcea0005180 2023-09-23T03:15:28.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.446+0000 7fceb27fc640 1 -- 172.21.15.154:0/3322223775 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceac02c8a0 con 0x7fcea0005180 2023-09-23T03:15:28.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.446+0000 7fceb3fff640 1 -- 172.21.15.154:0/3322223775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcea0005180 msgr2=0x7fcea000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:28.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.446+0000 7fceb3fff640 1 --2- 172.21.15.154:0/3322223775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcea0005180 0x7fcea000b980 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fceac00c190 tx=0x7fcea00057f0 comp rx=0 tx=0).stop 2023-09-23T03:15:28.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.447+0000 7fceb3fff640 1 -- 172.21.15.154:0/3322223775 shutdown_connections 2023-09-23T03:15:28.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.447+0000 7fceb3fff640 1 --2- 172.21.15.154:0/3322223775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcea0005180 0x7fcea000b980 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:28.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.447+0000 7fceb3fff640 1 -- 172.21.15.154:0/3322223775 >> 172.21.15.154:0/3322223775 conn(0x7fceb409bbb0 msgr2=0x7fceb40d28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:28.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.447+0000 7fceb3fff640 1 -- 172.21.15.154:0/3322223775 shutdown_connections 2023-09-23T03:15:28.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.447+0000 7fceb3fff640 1 -- 172.21.15.154:0/3322223775 wait complete. 2023-09-23T03:15:28.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.449+0000 7fceb3fff640 1 Processor -- start 2023-09-23T03:15:28.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.449+0000 7fceb3fff640 1 -- start start 2023-09-23T03:15:28.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.450+0000 7fceb3fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcea0005180 0x7fceb40c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:28.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.450+0000 7fceb3fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fceb4140cb0 0x7fceb40c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:28.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.450+0000 7fceb3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fceb415e570 0x7fceb4152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:28.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.450+0000 7fceb3fff640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb40ca4f0 con 0x7fceb415e570 2023-09-23T03:15:28.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.450+0000 7fceb3fff640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb4154a60 con 0x7fcea0005180 2023-09-23T03:15:28.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.450+0000 7fceb3fff640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fceb4154bf0 con 0x7fceb4140cb0 2023-09-23T03:15:28.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.450+0000 7fceb37fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fceb415e570 0x7fceb4152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:28.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.451+0000 7fceb37fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fceb415e570 0x7fceb4152b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:38270/0 (socket says 172.21.15.154:38270) 2023-09-23T03:15:28.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.451+0000 7fceb37fe640 1 -- 172.21.15.154:0/3390435899 learned_addr learned my addr 172.21.15.154:0/3390435899 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:28.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.451+0000 7fceaa7fc640 1 --2- 172.21.15.154:0/3390435899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fceb4140cb0 0x7fceb40c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:28.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.451+0000 7fceb2ffd640 1 --2- 172.21.15.154:0/3390435899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcea0005180 0x7fceb40c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:28.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.451+0000 7fceb37fe640 1 -- 172.21.15.154:0/3390435899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fceb4140cb0 msgr2=0x7fceb40c9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:28.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.451+0000 7fceb37fe640 1 --2- 172.21.15.154:0/3390435899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fceb4140cb0 0x7fceb40c9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:28.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.451+0000 7fceb37fe640 1 -- 172.21.15.154:0/3390435899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcea0005180 msgr2=0x7fceb40c8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:28.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.451+0000 7fceb37fe640 1 --2- 172.21.15.154:0/3390435899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcea0005180 0x7fceb40c8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:28.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.451+0000 7fceb37fe640 1 -- 172.21.15.154:0/3390435899 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb40c8850 con 0x7fceb415e570 2023-09-23T03:15:28.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.451+0000 7fceaa7fc640 1 --2- 172.21.15.154:0/3390435899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fceb4140cb0 0x7fceb40c9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:28.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.451+0000 7fceb37fe640 1 --2- 172.21.15.154:0/3390435899 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fceb415e570 0x7fceb4152b50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fceac00c190 tx=0x7fceb406bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:28.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.452+0000 7fceb0ff9640 1 -- 172.21.15.154:0/3390435899 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceac0049e0 con 0x7fceb415e570 2023-09-23T03:15:28.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.452+0000 7fceb0ff9640 1 -- 172.21.15.154:0/3390435899 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceac004b40 con 0x7fceb415e570 2023-09-23T03:15:28.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.452+0000 7fceb3fff640 1 -- 172.21.15.154:0/3390435899 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceb40c8b30 con 0x7fceb415e570 2023-09-23T03:15:28.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.452+0000 7fceb0ff9640 1 -- 172.21.15.154:0/3390435899 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceac001640 con 0x7fceb415e570 2023-09-23T03:15:28.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.452+0000 7fceb3fff640 1 -- 172.21.15.154:0/3390435899 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fceb40c90f0 con 0x7fceb415e570 2023-09-23T03:15:28.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.453+0000 7fceb3fff640 1 -- 172.21.15.154:0/3390435899 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fceb4154d60 con 0x7fceb415e570 2023-09-23T03:15:28.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.454+0000 7fceb0ff9640 1 -- 172.21.15.154:0/3390435899 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fceac032070 con 0x7fceb415e570 2023-09-23T03:15:28.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.455+0000 7fceb0ff9640 1 --2- 172.21.15.154:0/3390435899 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fce9406af30 0x7fce9406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:28.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.455+0000 7fceb0ff9640 1 -- 172.21.15.154:0/3390435899 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fceac03c070 con 0x7fceb415e570 2023-09-23T03:15:28.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.455+0000 7fceb2ffd640 1 --2- 172.21.15.154:0/3390435899 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fce9406af30 0x7fce9406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:28.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.456+0000 7fceb2ffd640 1 --2- 172.21.15.154:0/3390435899 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fce9406af30 0x7fce9406d3f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcea4007380 tx=0x7fcea400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:28.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.459+0000 7fceb0ff9640 1 -- 172.21.15.154:0/3390435899 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fceac0751d0 con 0x7fceb415e570 2023-09-23T03:15:28.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.647+0000 7fceb3fff640 1 -- 172.21.15.154:0/3390435899 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fceb404e920 con 0x7fceb415e570 2023-09-23T03:15:28.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.649+0000 7fceb0ff9640 1 -- 172.21.15.154:0/3390435899 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18382 (secure 0 0 0) 0x7fceac0787f0 con 0x7fceb415e570 2023-09-23T03:15:28.650 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:15:28.650 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":24,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:15:25.775060+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:15:18.575849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T03:15:19.707212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T03:15:23.337138+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:15:28.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.652+0000 7fceb3fff640 1 -- 172.21.15.154:0/3390435899 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fce9406af30 msgr2=0x7fce9406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:28.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.652+0000 7fceb3fff640 1 --2- 172.21.15.154:0/3390435899 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fce9406af30 0x7fce9406d3f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcea4007380 tx=0x7fcea400b040 comp rx=0 tx=0).stop 2023-09-23T03:15:28.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.652+0000 7fceb3fff640 1 -- 172.21.15.154:0/3390435899 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fceb415e570 msgr2=0x7fceb4152b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:28.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.652+0000 7fceb3fff640 1 --2- 172.21.15.154:0/3390435899 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fceb415e570 0x7fceb4152b50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fceac00c190 tx=0x7fceb406bfc0 comp rx=0 tx=0).stop 2023-09-23T03:15:28.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.652+0000 7fceb3fff640 1 -- 172.21.15.154:0/3390435899 shutdown_connections 2023-09-23T03:15:28.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.652+0000 7fceb3fff640 1 --2- 172.21.15.154:0/3390435899 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fce9406af30 0x7fce9406d3f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:28.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.652+0000 7fceb3fff640 1 --2- 172.21.15.154:0/3390435899 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fceb415e570 0x7fceb4152b50 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:28.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.652+0000 7fceb3fff640 1 --2- 172.21.15.154:0/3390435899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fceb4140cb0 0x7fceb40c9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:28.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.652+0000 7fceb3fff640 1 --2- 172.21.15.154:0/3390435899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcea0005180 0x7fceb40c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:28.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.653+0000 7fceb3fff640 1 -- 172.21.15.154:0/3390435899 >> 172.21.15.154:0/3390435899 conn(0x7fceb409bbb0 msgr2=0x7fceb40726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:28.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.653+0000 7fceb3fff640 1 -- 172.21.15.154:0/3390435899 shutdown_connections 2023-09-23T03:15:28.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:28.653+0000 7fceb3fff640 1 -- 172.21.15.154:0/3390435899 wait complete. 2023-09-23T03:15:29.072 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-23T03:15:20.686035+0000', 'modified': '2023-09-23T03:15:25.775313+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4285}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4285': {'gid': 4285, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.190:6834/3561509423', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6832', 'nonce': 3561509423}, {'type': 'v1', 'addr': '172.21.15.190:6834', 'nonce': 3561509423}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-09-23T03:15:29.072 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-23T03:15:29.072 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-09-23T03:15:29.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.302+0000 7fb7881d0640 1 Processor -- start 2023-09-23T03:15:29.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.302+0000 7fb7881d0640 1 -- start start 2023-09-23T03:15:29.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.303+0000 7fb7881d0640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb7800c14e0 0x7fb7800c1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:29.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.303+0000 7fb7881d0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb78006c1d0 0x7fb78006c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:29.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.303+0000 7fb7881d0640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7800cb160 con 0x7fb780161010 2023-09-23T03:15:29.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.303+0000 7fb7881d0640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7800c1ea0 con 0x7fb7800c14e0 2023-09-23T03:15:29.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.303+0000 7fb7881d0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb78006cb60 con 0x7fb78006c1d0 2023-09-23T03:15:29.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.304+0000 7fb785f6c640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fb780161010 0x7fb7800cc7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:48142/0 (socket says 172.21.15.154:48142) 2023-09-23T03:15:29.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.304+0000 7fb785f6c640 1 -- 172.21.15.154:0/4199381440 learned_addr learned my addr 172.21.15.154:0/4199381440 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:29.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.304+0000 7fb78576b640 1 --2- 172.21.15.154:0/4199381440 >> v2:172.21.15.190:3300/0 conn(0x7fb7800c14e0 0x7fb7800c1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:29.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.304+0000 7fb784f6a640 1 -- 172.21.15.154:0/4199381440 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1265180893 0 0) 0x7fb7800cb160 con 0x7fb780161010 2023-09-23T03:15:29.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.304+0000 7fb784f6a640 1 -- 172.21.15.154:0/4199381440 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb76c003520 con 0x7fb780161010 2023-09-23T03:15:29.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.304+0000 7fb78676d640 1 --2- 172.21.15.154:0/4199381440 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb78006c1d0 0x7fb78006c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:29.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.304+0000 7fb784f6a640 1 -- 172.21.15.154:0/4199381440 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 86876890 0 0) 0x7fb76c003520 con 0x7fb780161010 2023-09-23T03:15:29.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.304+0000 7fb784f6a640 1 -- 172.21.15.154:0/4199381440 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb78006c1d0 msgr2=0x7fb78006c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:29.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.304+0000 7fb784f6a640 1 --2- 172.21.15.154:0/4199381440 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb78006c1d0 0x7fb78006c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.304+0000 7fb784f6a640 1 -- 172.21.15.154:0/4199381440 >> v2:172.21.15.190:3300/0 conn(0x7fb7800c14e0 msgr2=0x7fb7800c1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:29.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.304+0000 7fb784f6a640 1 --2- 172.21.15.154:0/4199381440 >> v2:172.21.15.190:3300/0 conn(0x7fb7800c14e0 0x7fb7800c1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.304+0000 7fb784f6a640 1 -- 172.21.15.154:0/4199381440 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb78006cde0 con 0x7fb780161010 2023-09-23T03:15:29.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.304+0000 7fb784f6a640 1 -- 172.21.15.154:0/4199381440 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fb77c0017f0 con 0x7fb780161010 2023-09-23T03:15:29.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.304+0000 7fb78676d640 1 --2- 172.21.15.154:0/4199381440 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb78006c1d0 0x7fb78006c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:15:29.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.305+0000 7fb784f6a640 1 -- 172.21.15.154:0/4199381440 >> v1:172.21.15.154:6789/0 conn(0x7fb780161010 legacy=0x7fb7800cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:29.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.305+0000 7fb784f6a640 1 --2- 172.21.15.154:0/4199381440 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb76c003970 0x7fb76c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:29.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.305+0000 7fb784f6a640 1 --2- 172.21.15.154:0/4199381440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb76c0047c0 0x7fb76c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:29.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.305+0000 7fb784f6a640 1 --2- 172.21.15.154:0/4199381440 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb78006c1d0 0x7fb76c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:29.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.305+0000 7fb784f6a640 1 -- 172.21.15.154:0/4199381440 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7800c1ea0 con 0x7fb76c003970 2023-09-23T03:15:29.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.305+0000 7fb784f6a640 1 -- 172.21.15.154:0/4199381440 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78006cb60 con 0x7fb76c0047c0 2023-09-23T03:15:29.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.305+0000 7fb784f6a640 1 -- 172.21.15.154:0/4199381440 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb76c00ba30 con 0x7fb78006c1d0 2023-09-23T03:15:29.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.305+0000 7fb785f6c640 1 --2- 172.21.15.154:0/4199381440 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb76c003970 0x7fb76c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:29.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.305+0000 7fb78676d640 1 --2- 172.21.15.154:0/4199381440 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb78006c1d0 0x7fb76c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:29.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.305+0000 7fb785f6c640 1 -- 172.21.15.154:0/4199381440 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb78006c1d0 msgr2=0x7fb76c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:29.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.305+0000 7fb785f6c640 1 --2- 172.21.15.154:0/4199381440 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb78006c1d0 0x7fb76c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.305+0000 7fb785f6c640 1 -- 172.21.15.154:0/4199381440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb76c0047c0 msgr2=0x7fb76c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:29.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.305+0000 7fb785f6c640 1 --2- 172.21.15.154:0/4199381440 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb76c0047c0 0x7fb76c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.305+0000 7fb785f6c640 1 -- 172.21.15.154:0/4199381440 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb76c003520 con 0x7fb76c003970 2023-09-23T03:15:29.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.306+0000 7fb785f6c640 1 --2- 172.21.15.154:0/4199381440 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb76c003970 0x7fb76c003d90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb77c00c0e0 tx=0x7fb76c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d23da3ced0a7aefb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:29.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.307+0000 7fb784f6a640 1 -- 172.21.15.154:0/4199381440 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c007270 con 0x7fb76c003970 2023-09-23T03:15:29.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.307+0000 7fb784f6a640 1 -- 172.21.15.154:0/4199381440 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77c0073f0 con 0x7fb76c003970 2023-09-23T03:15:29.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.307+0000 7fb784f6a640 1 -- 172.21.15.154:0/4199381440 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c02c7d0 con 0x7fb76c003970 2023-09-23T03:15:29.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.307+0000 7fb7881d0640 1 -- 172.21.15.154:0/4199381440 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb76c003970 msgr2=0x7fb76c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:29.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.307+0000 7fb7881d0640 1 --2- 172.21.15.154:0/4199381440 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb76c003970 0x7fb76c003d90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb77c00c0e0 tx=0x7fb76c0057f0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.307+0000 7fb78676d640 1 -- 172.21.15.154:0/4199381440 reap_dead start 2023-09-23T03:15:29.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.307+0000 7fb7881d0640 1 -- 172.21.15.154:0/4199381440 shutdown_connections 2023-09-23T03:15:29.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.307+0000 7fb7881d0640 1 -- 172.21.15.154:0/4199381440 >> 172.21.15.154:0/4199381440 conn(0x7fb78006a0f0 msgr2=0x7fb78006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:29.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.307+0000 7fb7881d0640 1 -- 172.21.15.154:0/4199381440 shutdown_connections 2023-09-23T03:15:29.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.307+0000 7fb7881d0640 1 -- 172.21.15.154:0/4199381440 wait complete. 2023-09-23T03:15:29.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.309+0000 7fb7881d0640 1 Processor -- start 2023-09-23T03:15:29.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.309+0000 7fb7881d0640 1 -- start start 2023-09-23T03:15:29.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.310+0000 7fb7881d0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb76c003970 0x7fb780155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:29.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.310+0000 7fb7881d0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7800c3aa0 0x7fb7800c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:29.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.310+0000 7fb7881d0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7800c6550 0x7fb7800c69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:29.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.310+0000 7fb7881d0640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7800c6130 con 0x7fb76c003970 2023-09-23T03:15:29.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.310+0000 7fb7881d0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7800c8460 con 0x7fb7800c3aa0 2023-09-23T03:15:29.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.310+0000 7fb7881d0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb780155990 con 0x7fb7800c6550 2023-09-23T03:15:29.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.311+0000 7fb785f6c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb76c003970 0x7fb780155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:29.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.311+0000 7fb785f6c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb76c003970 0x7fb780155450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:38282/0 (socket says 172.21.15.154:38282) 2023-09-23T03:15:29.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.311+0000 7fb785f6c640 1 -- 172.21.15.154:0/3654599147 learned_addr learned my addr 172.21.15.154:0/3654599147 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:29.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.311+0000 7fb785f6c640 1 -- 172.21.15.154:0/3654599147 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7800c6550 msgr2=0x7fb7800c69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:29.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.311+0000 7fb785f6c640 1 --2- 172.21.15.154:0/3654599147 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7800c6550 0x7fb7800c69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.311+0000 7fb785f6c640 1 -- 172.21.15.154:0/3654599147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7800c3aa0 msgr2=0x7fb7800c3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:29.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.311+0000 7fb785f6c640 1 --2- 172.21.15.154:0/3654599147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7800c3aa0 0x7fb7800c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.311+0000 7fb785f6c640 1 -- 172.21.15.154:0/3654599147 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb780071ab0 con 0x7fb76c003970 2023-09-23T03:15:29.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.311+0000 7fb785f6c640 1 --2- 172.21.15.154:0/3654599147 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb76c003970 0x7fb780155450 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb77c000ef0 tx=0x7fb77c001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:29.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.311+0000 7fb7767fc640 1 -- 172.21.15.154:0/3654599147 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c039070 con 0x7fb76c003970 2023-09-23T03:15:29.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.312+0000 7fb7767fc640 1 -- 172.21.15.154:0/3654599147 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77c007280 con 0x7fb76c003970 2023-09-23T03:15:29.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.312+0000 7fb7767fc640 1 -- 172.21.15.154:0/3654599147 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c03c040 con 0x7fb76c003970 2023-09-23T03:15:29.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.312+0000 7fb7881d0640 1 -- 172.21.15.154:0/3654599147 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7800c5790 con 0x7fb76c003970 2023-09-23T03:15:29.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.312+0000 7fb7881d0640 1 -- 172.21.15.154:0/3654599147 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7800c5d50 con 0x7fb76c003970 2023-09-23T03:15:29.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.314+0000 7fb7767fc640 1 -- 172.21.15.154:0/3654599147 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fb77c032070 con 0x7fb76c003970 2023-09-23T03:15:29.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.316+0000 7fb7767fc640 1 --2- 172.21.15.154:0/3654599147 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb76406af30 0x7fb76406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:29.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.316+0000 7fb7767fc640 1 -- 172.21.15.154:0/3654599147 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb77c0aaa80 con 0x7fb76c003970 2023-09-23T03:15:29.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.316+0000 7fb78576b640 1 --2- 172.21.15.154:0/3654599147 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb76406af30 0x7fb76406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:29.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.316+0000 7fb78ad65740 1 Processor -- start 2023-09-23T03:15:29.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.317+0000 7fb78ad65740 1 -- start start 2023-09-23T03:15:29.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.317+0000 7fb78576b640 1 --2- 172.21.15.154:0/3654599147 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb76406af30 0x7fb76406d3f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb7700093d0 tx=0x7fb770009300 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:29.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.317+0000 7fb78ad65740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a55efadc00 0x55a55ef80b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:29.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.317+0000 7fb78ad65740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a55efadfe0 0x55a55efb0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:29.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.318+0000 7fb78ad65740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55ef81040 0x55a55efb0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:29.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.318+0000 7fb78ad65740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a55eef46d0 con 0x55a55ef81040 2023-09-23T03:15:29.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.318+0000 7fb78ad65740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a55eef1dc0 con 0x55a55efadc00 2023-09-23T03:15:29.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.318+0000 7fb78ad65740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a55eeed6b0 con 0x55a55efadfe0 2023-09-23T03:15:29.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.318+0000 7fb78676d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55ef81040 0x55a55efb0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:29.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.318+0000 7fb78676d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55ef81040 0x55a55efb0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:38294/0 (socket says 172.21.15.154:38294) 2023-09-23T03:15:29.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.318+0000 7fb78676d640 1 -- 172.21.15.154:0/2927821477 learned_addr learned my addr 172.21.15.154:0/2927821477 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:29.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.318+0000 7fb785f6c640 1 --2- 172.21.15.154:0/2927821477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a55efadc00 0x55a55ef80b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:29.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.318+0000 7fb78576b640 1 --2- 172.21.15.154:0/2927821477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a55efadfe0 0x55a55efb0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:29.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.318+0000 7fb78676d640 1 -- 172.21.15.154:0/2927821477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a55efadfe0 msgr2=0x55a55efb0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:29.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.318+0000 7fb78676d640 1 --2- 172.21.15.154:0/2927821477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a55efadfe0 0x55a55efb0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.318+0000 7fb78676d640 1 -- 172.21.15.154:0/2927821477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a55efadc00 msgr2=0x55a55ef80b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:29.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.318+0000 7fb78676d640 1 --2- 172.21.15.154:0/2927821477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a55efadc00 0x55a55ef80b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.318+0000 7fb78676d640 1 -- 172.21.15.154:0/2927821477 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a55eeb18e0 con 0x55a55ef81040 2023-09-23T03:15:29.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.319+0000 7fb78676d640 1 --2- 172.21.15.154:0/2927821477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55ef81040 0x55a55efb0ed0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb7801648e0 tx=0x7fb778009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b61ed8586db5d6c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:29.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.319+0000 7fb7637fe640 1 -- 172.21.15.154:0/2927821477 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb778004820 con 0x55a55ef81040 2023-09-23T03:15:29.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.319+0000 7fb7637fe640 1 -- 172.21.15.154:0/2927821477 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb778014400 con 0x55a55ef81040 2023-09-23T03:15:29.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.319+0000 7fb7637fe640 1 -- 172.21.15.154:0/2927821477 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7780147b0 con 0x55a55ef81040 2023-09-23T03:15:29.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.319+0000 7fb78ad65740 1 -- 172.21.15.154:0/2927821477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55ef81040 msgr2=0x55a55efb0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:29.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.319+0000 7fb78ad65740 1 --2- 172.21.15.154:0/2927821477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55ef81040 0x55a55efb0ed0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb7801648e0 tx=0x7fb778009b20 comp rx=0 tx=0).stop 2023-09-23T03:15:29.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.320+0000 7fb78ad65740 1 -- 172.21.15.154:0/2927821477 shutdown_connections 2023-09-23T03:15:29.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.320+0000 7fb78ad65740 1 --2- 172.21.15.154:0/2927821477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55ef81040 0x55a55efb0ed0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.320+0000 7fb78ad65740 1 --2- 172.21.15.154:0/2927821477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a55efadfe0 0x55a55efb0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.320+0000 7fb78ad65740 1 --2- 172.21.15.154:0/2927821477 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a55efadc00 0x55a55ef80b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.320+0000 7fb78ad65740 1 -- 172.21.15.154:0/2927821477 >> 172.21.15.154:0/2927821477 conn(0x55a55ef82e10 msgr2=0x55a55efcb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:29.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.320+0000 7fb78ad65740 1 -- 172.21.15.154:0/2927821477 shutdown_connections 2023-09-23T03:15:29.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.320+0000 7fb78ad65740 1 -- 172.21.15.154:0/2927821477 wait complete. 2023-09-23T03:15:29.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.320+0000 7fb78ad65740 1 -- start start 2023-09-23T03:15:29.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.321+0000 7fb78ad65740 1 Processor -- start 2023-09-23T03:15:29.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.322+0000 7fb78ad65740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55efadc00 0x55a55efc8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:29.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.322+0000 7fb78ad65740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a55efadfe0 0x55a55efb2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:29.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.322+0000 7fb78ad65740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a55efb2db0 0x55a55ef9ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:29.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.322+0000 7fb78ad65740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a55ee8b940 con 0x55a55efadc00 2023-09-23T03:15:29.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.322+0000 7fb78ad65740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a55ee8b5e0 con 0x55a55efb2db0 2023-09-23T03:15:29.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.322+0000 7fb78ad65740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a55ec51f20 con 0x55a55efadfe0 2023-09-23T03:15:29.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.322+0000 7fb78ad65740 10 client.0 _ll_register_callbacks cb 0x55a55eec95e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:15:29.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.322+0000 7fb785f6c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55efadc00 0x55a55efc8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:29.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.322+0000 7fb785f6c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55efadc00 0x55a55efc8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:38310/0 (socket says 172.21.15.154:38310) 2023-09-23T03:15:29.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.322+0000 7fb785f6c640 1 -- 172.21.15.154:0/4132813251 learned_addr learned my addr 172.21.15.154:0/4132813251 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:29.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.322+0000 7fb785f6c640 1 -- 172.21.15.154:0/4132813251 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a55efadfe0 msgr2=0x55a55efb2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:29.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.322+0000 7fb785f6c640 1 --2- 172.21.15.154:0/4132813251 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a55efadfe0 0x55a55efb2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.322+0000 7fb785f6c640 1 -- 172.21.15.154:0/4132813251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a55efb2db0 msgr2=0x55a55ef9ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:29.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.322+0000 7fb785f6c640 1 --2- 172.21.15.154:0/4132813251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a55efb2db0 0x55a55ef9ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.322+0000 7fb785f6c640 1 -- 172.21.15.154:0/4132813251 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb778003ce0 con 0x55a55efadc00 2023-09-23T03:15:29.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.322+0000 7fb785f6c640 1 --2- 172.21.15.154:0/4132813251 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55efadc00 0x55a55efc8fe0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb77c02dfd0 tx=0x7fb77c0795f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6042a5326dce016f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:29.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.322+0000 7fb760ff9640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:15:29.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.323+0000 7fb760ff9640 1 -- 172.21.15.154:0/4132813251 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c090020 con 0x55a55efadc00 2023-09-23T03:15:29.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.323+0000 7fb760ff9640 1 -- 172.21.15.154:0/4132813251 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb77c0a2bd0 con 0x55a55efadc00 2023-09-23T03:15:29.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.323+0000 7fb78ad65740 1 -- 172.21.15.154:0/4132813251 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a55eee6690 con 0x55a55efadc00 2023-09-23T03:15:29.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.323+0000 7fb760ff9640 1 -- 172.21.15.154:0/4132813251 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb77c0a2d90 con 0x55a55efadc00 2023-09-23T03:15:29.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.323+0000 7fb78ad65740 10 client.4401 fetch_fsmap learned FSMap version 7 2023-09-23T03:15:29.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.323+0000 7fb78ad65740 1 -- 172.21.15.154:0/4132813251 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55a55eeebd30 con 0x55a55efadc00 2023-09-23T03:15:29.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.323+0000 7fb760ff9640 1 -- 172.21.15.154:0/4132813251 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a55eee6690 con 0x55a55efadc00 2023-09-23T03:15:29.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.323+0000 7fb760ff9640 1 -- 172.21.15.154:0/4132813251 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fb77c03f070 con 0x55a55efadc00 2023-09-23T03:15:29.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.323+0000 7fb78ad65740 10 client.4401 fetch_fsmap finished waiting for FSMap version 7 2023-09-23T03:15:29.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.323+0000 7fb78ad65740 10 client.4401 resolve_mds: resolved name 'd' to daemon mds.d 2023-09-23T03:15:29.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.323+0000 7fb78ad65740 20 client.4401 populate_metadata read hostname 'smithi154' 2023-09-23T03:15:29.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.323+0000 7fb78ad65740 1 --2- 172.21.15.154:0/4132813251 >> [v2:172.21.15.190:6832/3561509423,v1:172.21.15.190:6834/3561509423] conn(0x55a55efcb5f0 0x55a55efe8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:29.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.323+0000 7fb78ad65740 4 client.4401 mds_command: new command op to 4285 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T03:15:29.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.323+0000 7fb78ad65740 1 -- 172.21.15.154:0/4132813251 --> [v2:172.21.15.190:6832/3561509423,v1:172.21.15.190:6834/3561509423] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a55eee63b0 con 0x55a55efcb5f0 2023-09-23T03:15:29.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.324+0000 7fb78576b640 1 --2- 172.21.15.154:0/4132813251 >> [v2:172.21.15.190:6832/3561509423,v1:172.21.15.190:6834/3561509423] conn(0x55a55efcb5f0 0x55a55efe8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:29.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.325+0000 7fb78576b640 1 --2- 172.21.15.154:0/4132813251 >> [v2:172.21.15.190:6832/3561509423,v1:172.21.15.190:6834/3561509423] conn(0x55a55efcb5f0 0x55a55efe8f30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a141a9bb36d873a7 server_cookie=24189cd707600379 in_seq=0 out_seq=0 2023-09-23T03:15:29.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.325+0000 7fb760ff9640 10 client.4401 ms_handle_connect on v2:172.21.15.190:6832/3561509423 2023-09-23T03:15:29.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.327+0000 7fb760ff9640 1 -- 172.21.15.154:0/4132813251 <== mds.0 v2:172.21.15.190:6832/3561509423 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a55eee63b0 con 0x55a55efcb5f0 2023-09-23T03:15:29.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.327+0000 7fb760ff9640 10 client.4401 handle_command_reply: tid=0 2023-09-23T03:15:29.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.327+0000 7fb78ad65740 1 client.4401 shutdown 2023-09-23T03:15:29.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.327+0000 7fb78ad65740 1 -- 172.21.15.154:0/4132813251 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55efadc00 msgr2=0x55a55efc8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:29.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.327+0000 7fb78ad65740 1 --2- 172.21.15.154:0/4132813251 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55efadc00 0x55a55efc8fe0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb77c02dfd0 tx=0x7fb77c0795f0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.327+0000 7fb78ad65740 1 -- 172.21.15.154:0/4132813251 shutdown_connections 2023-09-23T03:15:29.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.327+0000 7fb78ad65740 1 --2- 172.21.15.154:0/4132813251 >> [v2:172.21.15.190:6832/3561509423,v1:172.21.15.190:6834/3561509423] conn(0x55a55efcb5f0 0x55a55efe8f30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.327+0000 7fb78ad65740 1 --2- 172.21.15.154:0/4132813251 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a55efb2db0 0x55a55ef9ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.327+0000 7fb78ad65740 1 --2- 172.21.15.154:0/4132813251 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a55efadfe0 0x55a55efb2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.327+0000 7fb78ad65740 1 --2- 172.21.15.154:0/4132813251 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55efadc00 0x55a55efc8fe0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.327+0000 7fb760ff9640 0 client.4401 ms_handle_reset on v2:172.21.15.190:6832/3561509423 2023-09-23T03:15:29.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.327+0000 7fb78ad65740 1 -- 172.21.15.154:0/4132813251 >> 172.21.15.154:0/4132813251 conn(0x55a55ef81040 msgr2=0x55a55efc57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:29.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.327+0000 7fb78ad65740 1 -- 172.21.15.154:0/4132813251 shutdown_connections 2023-09-23T03:15:29.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.327+0000 7fb78ad65740 1 -- 172.21.15.154:0/4132813251 wait complete. 2023-09-23T03:15:29.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.327+0000 7fb78ad65740 20 client.4401 trim_cache size 0 max 16384 2023-09-23T03:15:29.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.344+0000 7fb78ad65740 1 Processor -- start 2023-09-23T03:15:29.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.344+0000 7fb78ad65740 1 -- start start 2023-09-23T03:15:29.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.345+0000 7fb78ad65740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55efadc00 0x55a55ef7fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:29.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.345+0000 7fb78ad65740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a55efadfe0 0x55a55efc8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:29.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.345+0000 7fb78ad65740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a55efb2db0 0x55a55ef858a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:29.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.345+0000 7fb78ad65740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a55eeef630 con 0x55a55efadc00 2023-09-23T03:15:29.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.345+0000 7fb78ad65740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a55efce8b0 con 0x55a55efadfe0 2023-09-23T03:15:29.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.345+0000 7fb78ad65740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a55efb0c10 con 0x55a55efb2db0 2023-09-23T03:15:29.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.345+0000 7fb785f6c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55efadc00 0x55a55ef7fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:29.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.345+0000 7fb785f6c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55efadc00 0x55a55ef7fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:38326/0 (socket says 172.21.15.154:38326) 2023-09-23T03:15:29.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.345+0000 7fb785f6c640 1 -- 172.21.15.154:0/8745788 learned_addr learned my addr 172.21.15.154:0/8745788 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:29.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.346+0000 7fb78676d640 1 --2- 172.21.15.154:0/8745788 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a55efb2db0 0x55a55ef858a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:29.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.346+0000 7fb78576b640 1 --2- 172.21.15.154:0/8745788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a55efadfe0 0x55a55efc8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:29.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.346+0000 7fb785f6c640 1 -- 172.21.15.154:0/8745788 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a55efb2db0 msgr2=0x55a55ef858a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:29.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.346+0000 7fb785f6c640 1 --2- 172.21.15.154:0/8745788 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a55efb2db0 0x55a55ef858a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.346+0000 7fb785f6c640 1 -- 172.21.15.154:0/8745788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a55efadfe0 msgr2=0x55a55efc8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:29.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.346+0000 7fb785f6c640 1 --2- 172.21.15.154:0/8745788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a55efadfe0 0x55a55efc8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.346+0000 7fb785f6c640 1 -- 172.21.15.154:0/8745788 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a55ee8ab50 con 0x55a55efadc00 2023-09-23T03:15:29.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.346+0000 7fb785f6c640 1 --2- 172.21.15.154:0/8745788 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55efadc00 0x55a55ef7fd30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb77c0794c0 tx=0x7fb77c07c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=270e81ce369e9ae8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:29.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.346+0000 7fb763fff640 1 -- 172.21.15.154:0/8745788 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c090070 con 0x55a55efadc00 2023-09-23T03:15:29.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.346+0000 7fb763fff640 1 -- 172.21.15.154:0/8745788 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77c07d5d0 con 0x55a55efadc00 2023-09-23T03:15:29.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.346+0000 7fb763fff640 1 -- 172.21.15.154:0/8745788 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77c0812f0 con 0x55a55efadc00 2023-09-23T03:15:29.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.347+0000 7fb78ad65740 1 -- 172.21.15.154:0/8745788 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55efadc00 msgr2=0x55a55ef7fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:29.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.347+0000 7fb78ad65740 1 --2- 172.21.15.154:0/8745788 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55efadc00 0x55a55ef7fd30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb77c0794c0 tx=0x7fb77c07c000 comp rx=0 tx=0).stop 2023-09-23T03:15:29.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.347+0000 7fb78ad65740 1 -- 172.21.15.154:0/8745788 shutdown_connections 2023-09-23T03:15:29.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.347+0000 7fb78ad65740 1 --2- 172.21.15.154:0/8745788 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a55efb2db0 0x55a55ef858a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.347+0000 7fb78ad65740 1 --2- 172.21.15.154:0/8745788 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a55efadfe0 0x55a55efc8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.347+0000 7fb78ad65740 1 --2- 172.21.15.154:0/8745788 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55efadc00 0x55a55ef7fd30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.347+0000 7fb78ad65740 1 -- 172.21.15.154:0/8745788 >> 172.21.15.154:0/8745788 conn(0x55a55ef81040 msgr2=0x55a55efc80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:29.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.347+0000 7fb78ad65740 1 -- 172.21.15.154:0/8745788 shutdown_connections 2023-09-23T03:15:29.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.347+0000 7fb78ad65740 1 -- 172.21.15.154:0/8745788 wait complete. 2023-09-23T03:15:29.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.348+0000 7fb78ad65740 1 -- start start 2023-09-23T03:15:29.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.348+0000 7fb78ad65740 1 Processor -- start 2023-09-23T03:15:29.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.349+0000 7fb78ad65740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a55efadc00 0x55a55f034360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:29.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.349+0000 7fb78ad65740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55efadfe0 0x55a55f023850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:29.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.349+0000 7fb78ad65740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a55efb2db0 0x55a55efb2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:29.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.349+0000 7fb78ad65740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a55f02da40 con 0x55a55efadfe0 2023-09-23T03:15:29.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.349+0000 7fb78ad65740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a55f034bb0 con 0x55a55efb2db0 2023-09-23T03:15:29.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.349+0000 7fb78ad65740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a55efb1a70 con 0x55a55efadc00 2023-09-23T03:15:29.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.349+0000 7fb78ad65740 10 client.0 _ll_register_callbacks cb 0x7fb7780039d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:15:29.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.349+0000 7fb78576b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55efadfe0 0x55a55f023850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:29.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.349+0000 7fb78576b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55efadfe0 0x55a55f023850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:38336/0 (socket says 172.21.15.154:38336) 2023-09-23T03:15:29.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.349+0000 7fb78576b640 1 -- 172.21.15.154:0/2537386888 learned_addr learned my addr 172.21.15.154:0/2537386888 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:29.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.350+0000 7fb78576b640 1 -- 172.21.15.154:0/2537386888 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a55efadc00 msgr2=0x55a55f034360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:29.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.350+0000 7fb78576b640 1 --2- 172.21.15.154:0/2537386888 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a55efadc00 0x55a55f034360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.350+0000 7fb78576b640 1 -- 172.21.15.154:0/2537386888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a55efb2db0 msgr2=0x55a55efb2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:29.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.350+0000 7fb78576b640 1 --2- 172.21.15.154:0/2537386888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a55efb2db0 0x55a55efb2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.350+0000 7fb78576b640 1 -- 172.21.15.154:0/2537386888 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a55eeebd30 con 0x55a55efadfe0 2023-09-23T03:15:29.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.350+0000 7fb78576b640 1 --2- 172.21.15.154:0/2537386888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55efadfe0 0x55a55f023850 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb7800c9e20 tx=0x7fb770009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a243087d377319 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:29.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.350+0000 7fb760ff9640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:15:29.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.350+0000 7fb760ff9640 1 -- 172.21.15.154:0/2537386888 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77001b040 con 0x55a55efadfe0 2023-09-23T03:15:29.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.350+0000 7fb760ff9640 1 -- 172.21.15.154:0/2537386888 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb770024040 con 0x55a55efadfe0 2023-09-23T03:15:29.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.350+0000 7fb78ad65740 1 -- 172.21.15.154:0/2537386888 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a55ebef6c0 con 0x55a55efadfe0 2023-09-23T03:15:29.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.351+0000 7fb760ff9640 1 -- 172.21.15.154:0/2537386888 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb770024200 con 0x55a55efadfe0 2023-09-23T03:15:29.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.351+0000 7fb78ad65740 10 client.4407 fetch_fsmap learned FSMap version 7 2023-09-23T03:15:29.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.351+0000 7fb78ad65740 1 -- 172.21.15.154:0/2537386888 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55a55eeb0140 con 0x55a55efadfe0 2023-09-23T03:15:29.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.351+0000 7fb760ff9640 1 -- 172.21.15.154:0/2537386888 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a55ebef6c0 con 0x55a55efadfe0 2023-09-23T03:15:29.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.351+0000 7fb760ff9640 1 -- 172.21.15.154:0/2537386888 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fb770011070 con 0x55a55efadfe0 2023-09-23T03:15:29.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.351+0000 7fb78ad65740 10 client.4407 fetch_fsmap finished waiting for FSMap version 7 2023-09-23T03:15:29.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.351+0000 7fb78ad65740 10 client.4407 resolve_mds: resolved name 'd' to daemon mds.d 2023-09-23T03:15:29.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.351+0000 7fb78ad65740 20 client.4407 populate_metadata read hostname 'smithi154' 2023-09-23T03:15:29.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.351+0000 7fb78ad65740 1 --2- 172.21.15.154:0/2537386888 >> [v2:172.21.15.190:6832/3561509423,v1:172.21.15.190:6834/3561509423] conn(0x55a55efcc100 0x55a55efcc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:29.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.351+0000 7fb78ad65740 4 client.4407 mds_command: new command op to 4285 tid=0[{"prefix": "status"}] 2023-09-23T03:15:29.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.351+0000 7fb78ad65740 1 -- 172.21.15.154:0/2537386888 --> [v2:172.21.15.190:6832/3561509423,v1:172.21.15.190:6834/3561509423] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a55ec42d10 con 0x55a55efcc100 2023-09-23T03:15:29.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.352+0000 7fb785f6c640 1 --2- 172.21.15.154:0/2537386888 >> [v2:172.21.15.190:6832/3561509423,v1:172.21.15.190:6834/3561509423] conn(0x55a55efcc100 0x55a55efcc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:29.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.352+0000 7fb785f6c640 1 --2- 172.21.15.154:0/2537386888 >> [v2:172.21.15.190:6832/3561509423,v1:172.21.15.190:6834/3561509423] conn(0x55a55efcc100 0x55a55efcc4e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22ff094c9f7b7ccf server_cookie=dbec54a201a2cee7 in_seq=0 out_seq=0 2023-09-23T03:15:29.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.352+0000 7fb760ff9640 10 client.4407 ms_handle_connect on v2:172.21.15.190:6832/3561509423 2023-09-23T03:15:29.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.353+0000 7fb760ff9640 1 -- 172.21.15.154:0/2537386888 <== mds.0 v2:172.21.15.190:6832/3561509423 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55a55ec42d10 con 0x55a55efcc100 2023-09-23T03:15:29.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.353+0000 7fb760ff9640 10 client.4407 handle_command_reply: tid=0 2023-09-23T03:15:29.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.353+0000 7fb78ad65740 1 client.4407 shutdown 2023-09-23T03:15:29.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.353+0000 7fb78ad65740 1 -- 172.21.15.154:0/2537386888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55efadfe0 msgr2=0x55a55f023850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:29.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.353+0000 7fb78ad65740 1 --2- 172.21.15.154:0/2537386888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55efadfe0 0x55a55f023850 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb7800c9e20 tx=0x7fb770009190 comp rx=0 tx=0).stop 2023-09-23T03:15:29.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.353+0000 7fb78ad65740 1 -- 172.21.15.154:0/2537386888 shutdown_connections 2023-09-23T03:15:29.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.353+0000 7fb78ad65740 1 --2- 172.21.15.154:0/2537386888 >> [v2:172.21.15.190:6832/3561509423,v1:172.21.15.190:6834/3561509423] conn(0x55a55efcc100 0x55a55efcc4e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.353+0000 7fb78ad65740 1 --2- 172.21.15.154:0/2537386888 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a55efb2db0 0x55a55efb2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.353+0000 7fb78ad65740 1 --2- 172.21.15.154:0/2537386888 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a55efadfe0 0x55a55f023850 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.353+0000 7fb78ad65740 1 --2- 172.21.15.154:0/2537386888 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a55efadc00 0x55a55f034360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.353+0000 7fb760ff9640 0 client.4407 ms_handle_reset on v2:172.21.15.190:6832/3561509423 2023-09-23T03:15:29.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.353+0000 7fb78ad65740 1 -- 172.21.15.154:0/2537386888 >> 172.21.15.154:0/2537386888 conn(0x55a55ef81040 msgr2=0x55a55efc80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:29.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.353+0000 7fb78ad65740 1 -- 172.21.15.154:0/2537386888 shutdown_connections 2023-09-23T03:15:29.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.353+0000 7fb78ad65740 1 -- 172.21.15.154:0/2537386888 wait complete. 2023-09-23T03:15:29.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.353+0000 7fb78ad65740 20 client.4407 trim_cache size 0 max 16384 2023-09-23T03:15:29.358 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:15:29.358 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_fsid": "8c8039d3-6c84-43db-8808-85e0dbdd47c8", 2023-09-23T03:15:29.358 INFO:teuthology.orchestra.run.smithi154.stdout: "whoami": 0, 2023-09-23T03:15:29.358 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 4285, 2023-09-23T03:15:29.358 INFO:teuthology.orchestra.run.smithi154.stdout: "want_state": "up:active", 2023-09-23T03:15:29.358 INFO:teuthology.orchestra.run.smithi154.stdout: "state": "up:active", 2023-09-23T03:15:29.358 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs", 2023-09-23T03:15:29.358 INFO:teuthology.orchestra.run.smithi154.stdout: "rank_uptime": 8.1034113560000005, 2023-09-23T03:15:29.358 INFO:teuthology.orchestra.run.smithi154.stdout: "mdsmap_epoch": 7, 2023-09-23T03:15:29.358 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_epoch": 24, 2023-09-23T03:15:29.358 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_epoch_barrier": 24, 2023-09-23T03:15:29.359 INFO:teuthology.orchestra.run.smithi154.stdout: "uptime": 13.744270681 2023-09-23T03:15:29.359 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:15:29.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.354+0000 7fb7881d0640 1 -- 172.21.15.154:0/3654599147 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb76406af30 msgr2=0x7fb76406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:29.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.354+0000 7fb7881d0640 1 --2- 172.21.15.154:0/3654599147 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb76406af30 0x7fb76406d3f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb7700093d0 tx=0x7fb770009300 comp rx=0 tx=0).stop 2023-09-23T03:15:29.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.354+0000 7fb7881d0640 1 -- 172.21.15.154:0/3654599147 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb76c003970 msgr2=0x7fb780155450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:29.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.354+0000 7fb7881d0640 1 --2- 172.21.15.154:0/3654599147 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb76c003970 0x7fb780155450 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb77c000ef0 tx=0x7fb77c001ab0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.354+0000 7fb7881d0640 1 -- 172.21.15.154:0/3654599147 shutdown_connections 2023-09-23T03:15:29.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.354+0000 7fb7881d0640 1 --2- 172.21.15.154:0/3654599147 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb76406af30 0x7fb76406d3f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.354+0000 7fb7881d0640 1 --2- 172.21.15.154:0/3654599147 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7800c6550 0x7fb7800c69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.354+0000 7fb7881d0640 1 --2- 172.21.15.154:0/3654599147 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7800c3aa0 0x7fb7800c3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.354+0000 7fb7881d0640 1 --2- 172.21.15.154:0/3654599147 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb76c003970 0x7fb780155450 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.354+0000 7fb7881d0640 1 -- 172.21.15.154:0/3654599147 >> 172.21.15.154:0/3654599147 conn(0x7fb78006a0f0 msgr2=0x7fb78006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:29.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.354+0000 7fb7881d0640 1 -- 172.21.15.154:0/3654599147 shutdown_connections 2023-09-23T03:15:29.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.354+0000 7fb7881d0640 1 -- 172.21.15.154:0/3654599147 wait complete. 2023-09-23T03:15:29.370 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-09-23T03:15:29.370 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-09-23T03:15:29.371 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-23T03:15:29.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.581+0000 7f0e31490640 1 Processor -- start 2023-09-23T03:15:29.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.582+0000 7f0e31490640 1 -- start start 2023-09-23T03:15:29.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.583+0000 7f0e31490640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0e2c15cab0 0x7f0e2c0cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:29.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.583+0000 7f0e31490640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0e2c15ce90 0x7f0e2c0cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:29.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.583+0000 7f0e31490640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e2c0d18e0 con 0x7f0e2c0d0bd0 2023-09-23T03:15:29.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.583+0000 7f0e31490640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e2c0cfc90 con 0x7f0e2c15cab0 2023-09-23T03:15:29.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.583+0000 7f0e31490640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e2c0d1bc0 con 0x7f0e2c15ce90 2023-09-23T03:15:29.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.584+0000 7f0e2b7fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f0e2c0d0bd0 0x7f0e2c0d1080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:48156/0 (socket says 172.21.15.154:48156) 2023-09-23T03:15:29.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.584+0000 7f0e2b7fe640 1 -- 172.21.15.154:0/1042563885 learned_addr learned my addr 172.21.15.154:0/1042563885 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:29.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.584+0000 7f0e2affd640 1 --2- 172.21.15.154:0/1042563885 >> v2:172.21.15.154:3300/0 conn(0x7f0e2c15cab0 0x7f0e2c0cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:29.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.584+0000 7f0e2a7fc640 1 --2- 172.21.15.154:0/1042563885 >> v2:172.21.15.190:3300/0 conn(0x7f0e2c15ce90 0x7f0e2c0cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:29.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.584+0000 7f0e29ffb640 1 -- 172.21.15.154:0/1042563885 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1351965022 0 0) 0x7f0e2c0d18e0 con 0x7f0e2c0d0bd0 2023-09-23T03:15:29.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.584+0000 7f0e29ffb640 1 -- 172.21.15.154:0/1042563885 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e18003520 con 0x7f0e2c0d0bd0 2023-09-23T03:15:29.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.584+0000 7f0e2affd640 1 -- 172.21.15.154:0/1042563885 >> v2:172.21.15.190:3300/0 conn(0x7f0e2c15ce90 msgr2=0x7f0e2c0cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:29.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.584+0000 7f0e2affd640 1 --2- 172.21.15.154:0/1042563885 >> v2:172.21.15.190:3300/0 conn(0x7f0e2c15ce90 0x7f0e2c0cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.584+0000 7f0e2affd640 1 -- 172.21.15.154:0/1042563885 >> v1:172.21.15.154:6789/0 conn(0x7f0e2c0d0bd0 legacy=0x7f0e2c0d1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:29.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.584+0000 7f0e2affd640 1 -- 172.21.15.154:0/1042563885 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e2c0d1ea0 con 0x7f0e2c15cab0 2023-09-23T03:15:29.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.584+0000 7f0e2a7fc640 1 --2- 172.21.15.154:0/1042563885 >> v2:172.21.15.190:3300/0 conn(0x7f0e2c15ce90 0x7f0e2c0cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:15:29.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.586+0000 7f0e2affd640 1 --2- 172.21.15.154:0/1042563885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e2c15cab0 0x7f0e2c0cf210 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c0cef60 tx=0x7f0e1c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6c574991fe96d3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:29.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.586+0000 7f0e29ffb640 1 -- 172.21.15.154:0/1042563885 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c02e070 con 0x7f0e2c15cab0 2023-09-23T03:15:29.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.586+0000 7f0e29ffb640 1 -- 172.21.15.154:0/1042563885 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1c0315a0 con 0x7f0e2c15cab0 2023-09-23T03:15:29.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.586+0000 7f0e29ffb640 1 -- 172.21.15.154:0/1042563885 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c031930 con 0x7f0e2c15cab0 2023-09-23T03:15:29.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.586+0000 7f0e31490640 1 -- 172.21.15.154:0/1042563885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e2c15cab0 msgr2=0x7f0e2c0cf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:29.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.586+0000 7f0e31490640 1 --2- 172.21.15.154:0/1042563885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e2c15cab0 0x7f0e2c0cf210 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c0cef60 tx=0x7f0e1c02b840 comp rx=0 tx=0).stop 2023-09-23T03:15:29.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.587+0000 7f0e31490640 1 -- 172.21.15.154:0/1042563885 shutdown_connections 2023-09-23T03:15:29.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.587+0000 7f0e31490640 1 --2- 172.21.15.154:0/1042563885 >> v2:172.21.15.190:3300/0 conn(0x7f0e2c15ce90 0x7f0e2c0cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.587+0000 7f0e31490640 1 --2- 172.21.15.154:0/1042563885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e2c15cab0 0x7f0e2c0cf210 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.587+0000 7f0e31490640 1 -- 172.21.15.154:0/1042563885 >> 172.21.15.154:0/1042563885 conn(0x7f0e2c071bc0 msgr2=0x7f0e2c0b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:29.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.587+0000 7f0e31490640 1 -- 172.21.15.154:0/1042563885 shutdown_connections 2023-09-23T03:15:29.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.587+0000 7f0e31490640 1 -- 172.21.15.154:0/1042563885 wait complete. 2023-09-23T03:15:29.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.588+0000 7f0e31490640 1 Processor -- start 2023-09-23T03:15:29.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.589+0000 7f0e31490640 1 -- start start 2023-09-23T03:15:29.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.589+0000 7f0e31490640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e2c0d0bd0 0x7f0e2c0cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:29.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.590+0000 7f0e31490640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e2c15cab0 0x7f0e2c1f4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:29.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.590+0000 7f0e31490640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e2c15ce90 0x7f0e2c0c6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:29.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.590+0000 7f0e31490640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c0c7530 con 0x7f0e2c0d0bd0 2023-09-23T03:15:29.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.590+0000 7f0e31490640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c0c7810 con 0x7f0e2c15cab0 2023-09-23T03:15:29.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.590+0000 7f0e31490640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c0c7af0 con 0x7f0e2c15ce90 2023-09-23T03:15:29.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.590+0000 7f0e2affd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e2c0d0bd0 0x7f0e2c0cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:29.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.590+0000 7f0e2affd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e2c0d0bd0 0x7f0e2c0cbeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:38360/0 (socket says 172.21.15.154:38360) 2023-09-23T03:15:29.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.590+0000 7f0e2affd640 1 -- 172.21.15.154:0/3604073174 learned_addr learned my addr 172.21.15.154:0/3604073174 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:29.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.590+0000 7f0e2affd640 1 -- 172.21.15.154:0/3604073174 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e2c15ce90 msgr2=0x7f0e2c0c6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:29.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.590+0000 7f0e2a7fc640 1 --2- 172.21.15.154:0/3604073174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e2c15cab0 0x7f0e2c1f4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:29.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.590+0000 7f0e2affd640 1 --2- 172.21.15.154:0/3604073174 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e2c15ce90 0x7f0e2c0c6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.590+0000 7f0e2affd640 1 -- 172.21.15.154:0/3604073174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e2c15cab0 msgr2=0x7f0e2c1f4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:29.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.590+0000 7f0e2affd640 1 --2- 172.21.15.154:0/3604073174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e2c15cab0 0x7f0e2c1f4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.590+0000 7f0e2affd640 1 -- 172.21.15.154:0/3604073174 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e1c009400 con 0x7f0e2c0d0bd0 2023-09-23T03:15:29.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.590+0000 7f0e2a7fc640 1 --2- 172.21.15.154:0/3604073174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e2c15cab0 0x7f0e2c1f4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:29.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.590+0000 7f0e2affd640 1 --2- 172.21.15.154:0/3604073174 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e2c0d0bd0 0x7f0e2c0cbeb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c000c00 tx=0x7f0e1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:29.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.591+0000 7f0e237fe640 1 -- 172.21.15.154:0/3604073174 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c032630 con 0x7f0e2c0d0bd0 2023-09-23T03:15:29.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.591+0000 7f0e237fe640 1 -- 172.21.15.154:0/3604073174 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1c032790 con 0x7f0e2c0d0bd0 2023-09-23T03:15:29.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.591+0000 7f0e237fe640 1 -- 172.21.15.154:0/3604073174 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c032b20 con 0x7f0e2c0d0bd0 2023-09-23T03:15:29.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.591+0000 7f0e31490640 1 -- 172.21.15.154:0/3604073174 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e2c0c7dd0 con 0x7f0e2c0d0bd0 2023-09-23T03:15:29.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.591+0000 7f0e31490640 1 -- 172.21.15.154:0/3604073174 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e2c1f52b0 con 0x7f0e2c0d0bd0 2023-09-23T03:15:29.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.592+0000 7f0e31490640 1 -- 172.21.15.154:0/3604073174 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e2c0c82f0 con 0x7f0e2c0d0bd0 2023-09-23T03:15:29.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.593+0000 7f0e237fe640 1 -- 172.21.15.154:0/3604073174 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f0e1c032c80 con 0x7f0e2c0d0bd0 2023-09-23T03:15:29.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.594+0000 7f0e237fe640 1 --2- 172.21.15.154:0/3604073174 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0e0c06af30 0x7f0e0c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:29.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.594+0000 7f0e237fe640 1 -- 172.21.15.154:0/3604073174 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0e1c0ab590 con 0x7f0e2c0d0bd0 2023-09-23T03:15:29.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.594+0000 7f0e2a7fc640 1 --2- 172.21.15.154:0/3604073174 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0e0c06af30 0x7f0e0c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:29.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.595+0000 7f0e2a7fc640 1 --2- 172.21.15.154:0/3604073174 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0e0c06af30 0x7f0e0c06d3f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0e14006fd0 tx=0x7f0e14008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:29.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.598+0000 7f0e237fe640 1 -- 172.21.15.154:0/3604073174 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f0e2c0c82f0 con 0x7f0e2c0d0bd0 2023-09-23T03:15:29.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.829+0000 7f0e31490640 1 -- 172.21.15.154:0/3604073174 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0e2c0c82f0 con 0x7f0e2c0d0bd0 2023-09-23T03:15:29.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.829+0000 7f0e237fe640 1 -- 172.21.15.154:0/3604073174 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7f0e2c0c82f0 con 0x7f0e2c0d0bd0 2023-09-23T03:15:29.829 INFO:teuthology.orchestra.run.smithi154.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-09-23T03:15:29.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.832+0000 7f0e31490640 1 -- 172.21.15.154:0/3604073174 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0e0c06af30 msgr2=0x7f0e0c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:29.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.832+0000 7f0e31490640 1 --2- 172.21.15.154:0/3604073174 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0e0c06af30 0x7f0e0c06d3f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0e14006fd0 tx=0x7f0e14008040 comp rx=0 tx=0).stop 2023-09-23T03:15:29.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.832+0000 7f0e31490640 1 -- 172.21.15.154:0/3604073174 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e2c0d0bd0 msgr2=0x7f0e2c0cbeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:29.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.832+0000 7f0e31490640 1 --2- 172.21.15.154:0/3604073174 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e2c0d0bd0 0x7f0e2c0cbeb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c000c00 tx=0x7f0e1c0046c0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.832+0000 7f0e31490640 1 -- 172.21.15.154:0/3604073174 shutdown_connections 2023-09-23T03:15:29.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.832+0000 7f0e31490640 1 --2- 172.21.15.154:0/3604073174 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0e0c06af30 0x7f0e0c06d3f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.832+0000 7f0e31490640 1 --2- 172.21.15.154:0/3604073174 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e2c15ce90 0x7f0e2c0c6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.832+0000 7f0e31490640 1 --2- 172.21.15.154:0/3604073174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e2c15cab0 0x7f0e2c1f4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.832+0000 7f0e31490640 1 --2- 172.21.15.154:0/3604073174 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e2c0d0bd0 0x7f0e2c0cbeb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:29.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.832+0000 7f0e31490640 1 -- 172.21.15.154:0/3604073174 >> 172.21.15.154:0/3604073174 conn(0x7f0e2c071bc0 msgr2=0x7f0e2c0cef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:29.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.832+0000 7f0e31490640 1 -- 172.21.15.154:0/3604073174 shutdown_connections 2023-09-23T03:15:29.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:29.832+0000 7f0e31490640 1 -- 172.21.15.154:0/3604073174 wait complete. 2023-09-23T03:15:29.851 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-09-23T03:15:29.851 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-23T03:15:29.851 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi154.front.sepia.ceph.com 2023-09-23T03:15:29.851 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-09-23T03:15:29.851 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-09-23T03:15:29.851 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-09-23T03:15:29.852 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-23T03:15:29.852 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-23T03:15:29.852 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-09-23T03:15:29.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:15:29.852 DEBUG:teuthology.orchestra.run.smithi154:> ip addr 2023-09-23T03:15:29.905 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-23T03:15:29.905 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-23T03:15:29.905 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2023-09-23T03:15:29.906 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:15:29.906 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2023-09-23T03:15:29.906 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:15:29.906 INFO:teuthology.orchestra.run.smithi154.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:15:29.906 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:54 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:15:29.906 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f0 2023-09-23T03:15:29.906 INFO:teuthology.orchestra.run.smithi154.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:15:29.906 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:55 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:15:29.906 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f1 2023-09-23T03:15:29.906 INFO:teuthology.orchestra.run.smithi154.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:15:29.906 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:60 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:15:29.907 INFO:teuthology.orchestra.run.smithi154.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-23T03:15:29.907 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:61 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:15:29.907 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.21.15.154/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-09-23T03:15:29.907 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft 42395sec preferred_lft 42395sec 2023-09-23T03:15:29.907 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::ec4:7aff:fe88:7761/64 scope link 2023-09-23T03:15:29.907 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:15:29.908 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-09-23T03:15:29.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:15:29.908 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-09-23T03:15:29.908 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-09-23T03:15:29.908 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link add name ceph-brx type bridge 2023-09-23T03:15:29.908 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip addr flush dev ceph-brx 2023-09-23T03:15:29.908 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set ceph-brx up 2023-09-23T03:15:29.908 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-09-23T03:15:29.908 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-09-23T03:15:30.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:15:30.380 DEBUG:teuthology.orchestra.run.smithi154:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-23T03:15:30.462 INFO:teuthology.orchestra.run.smithi154.stdout:1 2023-09-23T03:15:30.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:15:30.464 DEBUG:teuthology.orchestra.run.smithi154:> route 2023-09-23T03:15:30.521 INFO:teuthology.orchestra.run.smithi154.stdout:Kernel IP routing table 2023-09-23T03:15:30.521 INFO:teuthology.orchestra.run.smithi154.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-23T03:15:30.521 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-23T03:15:30.521 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-09-23T03:15:30.521 INFO:teuthology.orchestra.run.smithi154.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-23T03:15:30.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:15:30.521 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-09-23T03:15:30.522 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-09-23T03:15:30.522 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-23T03:15:30.522 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-23T03:15:30.522 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-23T03:15:30.522 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-09-23T03:15:31.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:15:31.313 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2023-09-23T03:15:31.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:15:31.364 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list-id 2023-09-23T03:15:31.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:15:31.410 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-09-23T03:15:31.410 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-09-23T03:15:31.410 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-09-23T03:15:31.410 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-09-23T03:15:31.410 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-09-23T03:15:31.588 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-09-23T03:15:31.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:15:31.589 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-09-23T03:15:31.589 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-09-23T03:15:31.589 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-09-23T03:15:31.589 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-09-23T03:15:31.589 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-09-23T03:15:31.589 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-09-23T03:15:31.589 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-09-23T03:15:31.589 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-09-23T03:15:31.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:15:31.848 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-09-23T03:15:31.848 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-09-23T03:15:31.848 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set brx.0 up 2023-09-23T03:15:31.848 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set dev brx.0 master ceph-brx 2023-09-23T03:15:31.849 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-09-23T03:15:32.005 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-09-23T03:15:32.005 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-23T03:15:32.005 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-09-23T03:15:32.050 INFO:teuthology.orchestra.run.smithi154.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-09-23T03:15:32.051 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-23T03:15:32.051 DEBUG:teuthology.orchestra.run.smithi154:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-09-23T03:15:32.141 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-09-23T03:15:32.272 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2023-09-23T03:15:32.318 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2023-09-23T03:15:32.319 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2023-09-23T03:15:32.319 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2023-09-23T03:15:32.319 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2023-09-23T03:15:32.319 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2023-09-23T03:15:32.319 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2023-09-23T03:15:32.319 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2023-09-23T03:15:32.319 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2023-09-23T03:15:32.319 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2023-09-23T03:15:32.319 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2023-09-23T03:15:32.319 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2023-09-23T03:15:32.319 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T03:15:32.319 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2023-09-23T03:15:32.320 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2023-09-23T03:15:32.320 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2023-09-23T03:15:32.320 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2023-09-23T03:15:32.320 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2023-09-23T03:15:32.320 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2023-09-23T03:15:32.320 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2023-09-23T03:15:32.320 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-09-23T03:15:32.320 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-09-23T03:15:32.320 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2023-09-23T03:15:32.320 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2023-09-23T03:15:32.320 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2023-09-23T03:15:32.320 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-0 2023-09-23T03:15:32.320 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-1 2023-09-23T03:15:32.321 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-2 2023-09-23T03:15:32.321 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-3 2023-09-23T03:15:32.321 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2023-09-23T03:15:32.321 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-09-23T03:15:32.321 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-09-23T03:15:32.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:15:32.322 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-09-23T03:15:32.368 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-23T03:15:32.368 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-09-23T03:15:32.416 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-09-23T03:15:32.496 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2023-09-23T03:15:32.536 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2023-09-23T03:15:32.537 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2023-09-23T03:15:32.537 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2023-09-23T03:15:32.537 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2023-09-23T03:15:32.537 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2023-09-23T03:15:32.537 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2023-09-23T03:15:32.537 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2023-09-23T03:15:32.537 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2023-09-23T03:15:32.537 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2023-09-23T03:15:32.537 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2023-09-23T03:15:32.537 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2023-09-23T03:15:32.537 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T03:15:32.538 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2023-09-23T03:15:32.538 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2023-09-23T03:15:32.538 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2023-09-23T03:15:32.538 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2023-09-23T03:15:32.538 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2023-09-23T03:15:32.538 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2023-09-23T03:15:32.538 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2023-09-23T03:15:32.538 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-09-23T03:15:32.538 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-09-23T03:15:32.538 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2023-09-23T03:15:32.538 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2023-09-23T03:15:32.538 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2023-09-23T03:15:32.539 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-0 2023-09-23T03:15:32.539 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-1 2023-09-23T03:15:32.539 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-2 2023-09-23T03:15:32.539 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-3 2023-09-23T03:15:32.539 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2023-09-23T03:15:32.539 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-09-23T03:15:32.539 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-09-23T03:15:32.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:15:32.540 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-09-23T03:15:32.646 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi154.stderr:2023-09-23T03:15:32.646+0000 7f75b1e43480 -1 init, newargv = 0x55bca308c4c0 newargc=15 2023-09-23T03:15:32.646 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi154.stderr:ceph-fuse[93331]: starting ceph client 2023-09-23T03:15:32.659 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi154.stderr:ceph-fuse[93331]: starting fuse 2023-09-23T03:15:33.730 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-09-23T03:15:33.835 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2023-09-23T03:15:33.895 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2023-09-23T03:15:33.895 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2023-09-23T03:15:33.896 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2023-09-23T03:15:33.896 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2023-09-23T03:15:33.896 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2023-09-23T03:15:33.896 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2023-09-23T03:15:33.896 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2023-09-23T03:15:33.896 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2023-09-23T03:15:33.896 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2023-09-23T03:15:33.896 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2023-09-23T03:15:33.896 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2023-09-23T03:15:33.896 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T03:15:33.896 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2023-09-23T03:15:33.897 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2023-09-23T03:15:33.897 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2023-09-23T03:15:33.897 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2023-09-23T03:15:33.897 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2023-09-23T03:15:33.897 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2023-09-23T03:15:33.897 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2023-09-23T03:15:33.897 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-09-23T03:15:33.897 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-09-23T03:15:33.897 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2023-09-23T03:15:33.897 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2023-09-23T03:15:33.897 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2023-09-23T03:15:33.897 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-0 2023-09-23T03:15:33.897 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-1 2023-09-23T03:15:33.898 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-2 2023-09-23T03:15:33.898 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-3 2023-09-23T03:15:33.898 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2023-09-23T03:15:33.898 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-09-23T03:15:33.898 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-09-23T03:15:33.898 INFO:teuthology.orchestra.run.smithi154.stdout:/home/ubuntu/cephtest/mnt.admin 2023-09-23T03:15:33.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:15:33.899 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-09-23T03:15:33.952 INFO:teuthology.orchestra.run.smithi154.stdout:39 2023-09-23T03:15:33.952 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-09-23T03:15:33.952 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:15:33.952 DEBUG:teuthology.orchestra.run.smithi154:> import glob 2023-09-23T03:15:33.953 DEBUG:teuthology.orchestra.run.smithi154:> import re 2023-09-23T03:15:33.953 DEBUG:teuthology.orchestra.run.smithi154:> import os 2023-09-23T03:15:33.953 DEBUG:teuthology.orchestra.run.smithi154:> import subprocess 2023-09-23T03:15:33.953 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:15:33.953 DEBUG:teuthology.orchestra.run.smithi154:> def _find_admin_socket(client_name): 2023-09-23T03:15:33.953 DEBUG:teuthology.orchestra.run.smithi154:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-09-23T03:15:33.953 DEBUG:teuthology.orchestra.run.smithi154:> files = glob.glob(asok_path) 2023-09-23T03:15:33.953 DEBUG:teuthology.orchestra.run.smithi154:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-09-23T03:15:33.953 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:15:33.953 DEBUG:teuthology.orchestra.run.smithi154:> # Given a non-glob path, it better be there 2023-09-23T03:15:33.953 DEBUG:teuthology.orchestra.run.smithi154:> if "*" not in asok_path: 2023-09-23T03:15:33.953 DEBUG:teuthology.orchestra.run.smithi154:> assert(len(files) == 1) 2023-09-23T03:15:33.953 DEBUG:teuthology.orchestra.run.smithi154:> return files[0] 2023-09-23T03:15:33.954 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:15:33.954 DEBUG:teuthology.orchestra.run.smithi154:> for f in files: 2023-09-23T03:15:33.954 DEBUG:teuthology.orchestra.run.smithi154:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-23T03:15:33.954 DEBUG:teuthology.orchestra.run.smithi154:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-23T03:15:33.954 DEBUG:teuthology.orchestra.run.smithi154:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-23T03:15:33.954 DEBUG:teuthology.orchestra.run.smithi154:> contents = proc_f.read() 2023-09-23T03:15:33.954 DEBUG:teuthology.orchestra.run.smithi154:> if mountpoint in contents: 2023-09-23T03:15:33.954 DEBUG:teuthology.orchestra.run.smithi154:> return f 2023-09-23T03:15:33.954 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-23T03:15:33.954 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:15:33.954 DEBUG:teuthology.orchestra.run.smithi154:> print(_find_admin_socket("client.admin")) 2023-09-23T03:15:33.954 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:15:34.133 INFO:teuthology.orchestra.run.smithi154.stdout:/var/run/ceph/ceph-client.admin.93331.asok 2023-09-23T03:15:34.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:15:34.151 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.93331.asok 2023-09-23T03:15:34.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:15:34.151 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.93331.asok status 2023-09-23T03:15:34.398 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:15:34.398 INFO:teuthology.orchestra.run.smithi154.stdout: "metadata": { 2023-09-23T03:15:34.398 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_sha1": "862663913a74a99e1c52d50e6a7e59dd6705749a", 2023-09-23T03:15:34.398 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_version": "ceph version 18.0.0-6336-g86266391 (862663913a74a99e1c52d50e6a7e59dd6705749a) reef (dev)", 2023-09-23T03:15:34.398 INFO:teuthology.orchestra.run.smithi154.stdout: "entity_id": "admin", 2023-09-23T03:15:34.398 INFO:teuthology.orchestra.run.smithi154.stdout: "hostname": "smithi154", 2023-09-23T03:15:34.398 INFO:teuthology.orchestra.run.smithi154.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-09-23T03:15:34.398 INFO:teuthology.orchestra.run.smithi154.stdout: "pid": "93331", 2023-09-23T03:15:34.398 INFO:teuthology.orchestra.run.smithi154.stdout: "root": "/" 2023-09-23T03:15:34.399 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:15:34.399 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_count": 0, 2023-09-23T03:15:34.399 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_pinned_count": 0, 2023-09-23T03:15:34.399 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 4422, 2023-09-23T03:15:34.399 INFO:teuthology.orchestra.run.smithi154.stdout: "inst": { 2023-09-23T03:15:34.399 INFO:teuthology.orchestra.run.smithi154.stdout: "name": { 2023-09-23T03:15:34.399 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "client", 2023-09-23T03:15:34.399 INFO:teuthology.orchestra.run.smithi154.stdout: "num": 4422 2023-09-23T03:15:34.399 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:15:34.399 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2023-09-23T03:15:34.399 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2023-09-23T03:15:34.400 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.1:0", 2023-09-23T03:15:34.400 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 526158325 2023-09-23T03:15:34.400 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:15:34.400 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:15:34.400 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2023-09-23T03:15:34.400 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2023-09-23T03:15:34.400 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.1:0", 2023-09-23T03:15:34.400 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 526158325 2023-09-23T03:15:34.400 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:15:34.400 INFO:teuthology.orchestra.run.smithi154.stdout: "inst_str": "client.4422 192.168.0.1:0/526158325", 2023-09-23T03:15:34.400 INFO:teuthology.orchestra.run.smithi154.stdout: "addr_str": "192.168.0.1:0/526158325", 2023-09-23T03:15:34.400 INFO:teuthology.orchestra.run.smithi154.stdout: "inode_count": 1, 2023-09-23T03:15:34.401 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_epoch": 7, 2023-09-23T03:15:34.401 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch": 24, 2023-09-23T03:15:34.401 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch_barrier": 0, 2023-09-23T03:15:34.401 INFO:teuthology.orchestra.run.smithi154.stdout: "blocklisted": false, 2023-09-23T03:15:34.401 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs" 2023-09-23T03:15:34.401 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:15:34.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:15:34.413 DEBUG:teuthology.orchestra.run.smithi154:> stat --file-system '--printf=%T 2023-09-23T03:15:34.413 DEBUG:teuthology.orchestra.run.smithi154:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-09-23T03:15:34.463 INFO:teuthology.orchestra.run.smithi154.stdout:fuseblk 2023-09-23T03:15:34.464 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-09-23T03:15:34.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:15:34.464 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-09-23T03:15:34.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:15:34.550 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-09-23T03:15:34.647 INFO:teuthology.orchestra.run.smithi154.stdout:# file: . 2023-09-23T03:15:34.647 INFO:teuthology.orchestra.run.smithi154.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-09-23T03:15:34.647 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:15:34.648 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:15:34.648 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:15:34.705 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:15:34.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.916+0000 7f58bcf16640 1 Processor -- start 2023-09-23T03:15:34.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.916+0000 7f58bcf16640 1 -- start start 2023-09-23T03:15:34.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.918+0000 7f58bcf16640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f58b815caf0 0x7f58b80ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:34.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.918+0000 7f58bcf16640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f58b80cee70 0x7f58b8160880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:34.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.918+0000 7f58bcf16640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58b8160dc0 0x7f58b806cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:34.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.918+0000 7f58bcf16640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58b815ced0 con 0x7f58b80cee70 2023-09-23T03:15:34.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.918+0000 7f58bcf16640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58b80cf2f0 con 0x7f58b815caf0 2023-09-23T03:15:34.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.918+0000 7f58bcf16640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b80ce2f0 con 0x7f58b8160dc0 2023-09-23T03:15:34.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.918+0000 7f58affff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f58b80cee70 0x7f58b8160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:34.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.918+0000 7f58affff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f58b80cee70 0x7f58b8160880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47740/0 (socket says 172.21.15.154:47740) 2023-09-23T03:15:34.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.918+0000 7f58affff640 1 -- 172.21.15.154:0/1463150283 learned_addr learned my addr 172.21.15.154:0/1463150283 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:34.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.918+0000 7f58b6d9d640 1 --2- 172.21.15.154:0/1463150283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58b8160dc0 0x7f58b806cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:34.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.919+0000 7f58b659c640 1 --2- 172.21.15.154:0/1463150283 >> v2:172.21.15.190:3300/0 conn(0x7f58b815caf0 0x7f58b80ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:34.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.919+0000 7f58affff640 1 -- 172.21.15.154:0/1463150283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58b8160dc0 msgr2=0x7f58b806cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:34.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.919+0000 7f58affff640 1 --2- 172.21.15.154:0/1463150283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58b8160dc0 0x7f58b806cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:34.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.919+0000 7f58affff640 1 -- 172.21.15.154:0/1463150283 >> v2:172.21.15.190:3300/0 conn(0x7f58b815caf0 msgr2=0x7f58b80ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:34.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.919+0000 7f58affff640 1 --2- 172.21.15.154:0/1463150283 >> v2:172.21.15.190:3300/0 conn(0x7f58b815caf0 0x7f58b80ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:34.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.919+0000 7f58affff640 1 -- 172.21.15.154:0/1463150283 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b80cd530 con 0x7f58b80cee70 2023-09-23T03:15:34.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.920+0000 7f58affff640 1 --2- 172.21.15.154:0/1463150283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58b80cee70 0x7f58b8160880 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f58b80ce470 tx=0x7f58a002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95b765fc205d103 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:34.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.920+0000 7f58b5d9b640 1 -- 172.21.15.154:0/1463150283 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a0031870 con 0x7f58b80cee70 2023-09-23T03:15:34.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.920+0000 7f58b5d9b640 1 -- 172.21.15.154:0/1463150283 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a00319d0 con 0x7f58b80cee70 2023-09-23T03:15:34.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.920+0000 7f58b5d9b640 1 -- 172.21.15.154:0/1463150283 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a0031d60 con 0x7f58b80cee70 2023-09-23T03:15:34.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.921+0000 7f58bcf16640 1 -- 172.21.15.154:0/1463150283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58b80cee70 msgr2=0x7f58b8160880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:34.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.921+0000 7f58bcf16640 1 --2- 172.21.15.154:0/1463150283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58b80cee70 0x7f58b8160880 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f58b80ce470 tx=0x7f58a002b910 comp rx=0 tx=0).stop 2023-09-23T03:15:34.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.921+0000 7f58bcf16640 1 -- 172.21.15.154:0/1463150283 shutdown_connections 2023-09-23T03:15:34.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.921+0000 7f58bcf16640 1 --2- 172.21.15.154:0/1463150283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58b8160dc0 0x7f58b806cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:34.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.921+0000 7f58bcf16640 1 --2- 172.21.15.154:0/1463150283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58b80cee70 0x7f58b8160880 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:34.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.921+0000 7f58bcf16640 1 --2- 172.21.15.154:0/1463150283 >> v2:172.21.15.190:3300/0 conn(0x7f58b815caf0 0x7f58b80ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:34.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.921+0000 7f58bcf16640 1 -- 172.21.15.154:0/1463150283 >> 172.21.15.154:0/1463150283 conn(0x7f58b806a0f0 msgr2=0x7f58b806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:34.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.921+0000 7f58bcf16640 1 -- 172.21.15.154:0/1463150283 shutdown_connections 2023-09-23T03:15:34.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.921+0000 7f58bcf16640 1 -- 172.21.15.154:0/1463150283 wait complete. 2023-09-23T03:15:34.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.923+0000 7f58bcf16640 1 Processor -- start 2023-09-23T03:15:34.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.923+0000 7f58bcf16640 1 -- start start 2023-09-23T03:15:34.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.924+0000 7f58bcf16640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58b80cee70 0x7f58b8153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:34.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.924+0000 7f58bcf16640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58b815caf0 0x7f58b8158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:34.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.924+0000 7f58bcf16640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58b8160dc0 0x7f58b8158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:34.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.924+0000 7f58bcf16640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b8070110 con 0x7f58b8160dc0 2023-09-23T03:15:34.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.924+0000 7f58bcf16640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b80c3fc0 con 0x7f58b815caf0 2023-09-23T03:15:34.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.924+0000 7f58bcf16640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b80c3720 con 0x7f58b80cee70 2023-09-23T03:15:34.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.924+0000 7f58b6d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58b8160dc0 0x7f58b8158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:34.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.925+0000 7f58b6d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58b8160dc0 0x7f58b8158a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47746/0 (socket says 172.21.15.154:47746) 2023-09-23T03:15:34.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.925+0000 7f58b6d9d640 1 -- 172.21.15.154:0/1014336238 learned_addr learned my addr 172.21.15.154:0/1014336238 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:34.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.925+0000 7f58affff640 1 --2- 172.21.15.154:0/1014336238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58b815caf0 0x7f58b8158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:34.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.925+0000 7f58b659c640 1 --2- 172.21.15.154:0/1014336238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58b80cee70 0x7f58b8153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:34.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.925+0000 7f58b6d9d640 1 -- 172.21.15.154:0/1014336238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58b80cee70 msgr2=0x7f58b8153630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:34.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.925+0000 7f58b6d9d640 1 --2- 172.21.15.154:0/1014336238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58b80cee70 0x7f58b8153630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:34.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.925+0000 7f58b6d9d640 1 -- 172.21.15.154:0/1014336238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58b815caf0 msgr2=0x7f58b8158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:34.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.925+0000 7f58b6d9d640 1 --2- 172.21.15.154:0/1014336238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58b815caf0 0x7f58b8158510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:34.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.925+0000 7f58b6d9d640 1 -- 172.21.15.154:0/1014336238 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a00094d0 con 0x7f58b8160dc0 2023-09-23T03:15:34.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.925+0000 7f58b659c640 1 --2- 172.21.15.154:0/1014336238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58b80cee70 0x7f58b8153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:34.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.925+0000 7f58b6d9d640 1 --2- 172.21.15.154:0/1014336238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58b8160dc0 0x7f58b8158a50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f58b0007e90 tx=0x7f58b000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:34.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.926+0000 7f58aeffd640 1 -- 172.21.15.154:0/1014336238 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b0014420 con 0x7f58b8160dc0 2023-09-23T03:15:34.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.926+0000 7f58aeffd640 1 -- 172.21.15.154:0/1014336238 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b0014580 con 0x7f58b8160dc0 2023-09-23T03:15:34.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.926+0000 7f58bcf16640 1 -- 172.21.15.154:0/1014336238 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58b80c3a00 con 0x7f58b8160dc0 2023-09-23T03:15:34.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.926+0000 7f58aeffd640 1 -- 172.21.15.154:0/1014336238 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b0014930 con 0x7f58b8160dc0 2023-09-23T03:15:34.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.926+0000 7f58bcf16640 1 -- 172.21.15.154:0/1014336238 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b8153b70 con 0x7f58b8160dc0 2023-09-23T03:15:34.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.927+0000 7f58bcf16640 1 -- 172.21.15.154:0/1014336238 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58b806fee0 con 0x7f58b8160dc0 2023-09-23T03:15:34.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.928+0000 7f58aeffd640 1 -- 172.21.15.154:0/1014336238 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f58b0014a90 con 0x7f58b8160dc0 2023-09-23T03:15:34.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.929+0000 7f58aeffd640 1 --2- 172.21.15.154:0/1014336238 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f589806af60 0x7f589806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:34.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.929+0000 7f58aeffd640 1 -- 172.21.15.154:0/1014336238 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f58b008a400 con 0x7f58b8160dc0 2023-09-23T03:15:34.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.929+0000 7f58b659c640 1 --2- 172.21.15.154:0/1014336238 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f589806af60 0x7f589806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:34.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.930+0000 7f58b659c640 1 --2- 172.21.15.154:0/1014336238 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f589806af60 0x7f589806d420 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f58a80097a0 tx=0x7f58a8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:34.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:34.933+0000 7f58aeffd640 1 -- 172.21.15.154:0/1014336238 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f58b806fee0 con 0x7f58b8160dc0 2023-09-23T03:15:35.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.122+0000 7f58bcf16640 1 -- 172.21.15.154:0/1014336238 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58b806fee0 con 0x7f58b8160dc0 2023-09-23T03:15:35.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.124+0000 7f58aeffd640 1 -- 172.21.15.154:0/1014336238 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18382 (secure 0 0 0) 0x7f58b806fee0 con 0x7f58b8160dc0 2023-09-23T03:15:35.125 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:15:35.125 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":24,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:15:25.775060+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:15:18.575849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T03:15:19.707212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T03:15:23.337138+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:15:35.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.126+0000 7f58bcf16640 1 -- 172.21.15.154:0/1014336238 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f589806af60 msgr2=0x7f589806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:35.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.126+0000 7f58bcf16640 1 --2- 172.21.15.154:0/1014336238 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f589806af60 0x7f589806d420 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f58a80097a0 tx=0x7f58a8008040 comp rx=0 tx=0).stop 2023-09-23T03:15:35.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.126+0000 7f58bcf16640 1 -- 172.21.15.154:0/1014336238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58b8160dc0 msgr2=0x7f58b8158a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:35.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.126+0000 7f58bcf16640 1 --2- 172.21.15.154:0/1014336238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58b8160dc0 0x7f58b8158a50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f58b0007e90 tx=0x7f58b000e4b0 comp rx=0 tx=0).stop 2023-09-23T03:15:35.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.127+0000 7f58bcf16640 1 -- 172.21.15.154:0/1014336238 shutdown_connections 2023-09-23T03:15:35.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.127+0000 7f58bcf16640 1 --2- 172.21.15.154:0/1014336238 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f589806af60 0x7f589806d420 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:35.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.127+0000 7f58bcf16640 1 --2- 172.21.15.154:0/1014336238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58b8160dc0 0x7f58b8158a50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:35.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.127+0000 7f58bcf16640 1 --2- 172.21.15.154:0/1014336238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58b815caf0 0x7f58b8158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:35.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.127+0000 7f58bcf16640 1 --2- 172.21.15.154:0/1014336238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58b80cee70 0x7f58b8153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:35.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.127+0000 7f58bcf16640 1 -- 172.21.15.154:0/1014336238 >> 172.21.15.154:0/1014336238 conn(0x7f58b806a0f0 msgr2=0x7f58b80ab140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:35.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.127+0000 7f58bcf16640 1 -- 172.21.15.154:0/1014336238 shutdown_connections 2023-09-23T03:15:35.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.127+0000 7f58bcf16640 1 -- 172.21.15.154:0/1014336238 wait complete. 2023-09-23T03:15:35.144 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:15:18.575849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:15:19.707212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-23T03:15:23.337138+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-23T03:15:35.145 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:15:35.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.367+0000 7f9e4531d640 1 Processor -- start 2023-09-23T03:15:35.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.367+0000 7f9e4531d640 1 -- start start 2023-09-23T03:15:35.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.369+0000 7f9e4531d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9e4015aba0 0x7f9e4015bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:35.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.369+0000 7f9e4531d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9e4015e5f0 0x7f9e4015ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:35.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.369+0000 7f9e4531d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e4015b6a0 con 0x7f9e400caea0 2023-09-23T03:15:35.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.369+0000 7f9e4531d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e4015f0b0 con 0x7f9e4015aba0 2023-09-23T03:15:35.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.369+0000 7f9e4531d640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e4015f390 con 0x7f9e4015e5f0 2023-09-23T03:15:35.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.370+0000 7f9e3effd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9e4015aba0 0x7f9e4015bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:35.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.370+0000 7f9e3effd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9e4015aba0 0x7f9e4015bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47758/0 (socket says 172.21.15.154:47758) 2023-09-23T03:15:35.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.370+0000 7f9e3effd640 1 -- 172.21.15.154:0/2452153034 learned_addr learned my addr 172.21.15.154:0/2452153034 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:35.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.370+0000 7f9e3f7fe640 1 --1- 172.21.15.154:0/2452153034 >> v1:172.21.15.154:6789/0 conn(0x7f9e400caea0 0x7f9e400cc5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:37892/0 (socket says 172.21.15.154:37892) 2023-09-23T03:15:35.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.370+0000 7f9e37fff640 1 --2- 172.21.15.154:0/2452153034 >> v2:172.21.15.190:3300/0 conn(0x7f9e4015e5f0 0x7f9e4015ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:35.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.370+0000 7f9e3effd640 1 -- 172.21.15.154:0/2452153034 >> v2:172.21.15.190:3300/0 conn(0x7f9e4015e5f0 msgr2=0x7f9e4015ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:35.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.370+0000 7f9e3effd640 1 --2- 172.21.15.154:0/2452153034 >> v2:172.21.15.190:3300/0 conn(0x7f9e4015e5f0 0x7f9e4015ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:35.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.370+0000 7f9e3effd640 1 -- 172.21.15.154:0/2452153034 >> v1:172.21.15.154:6789/0 conn(0x7f9e400caea0 legacy=0x7f9e400cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:35.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.370+0000 7f9e3e7fc640 1 -- 172.21.15.154:0/2452153034 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1880661072 0 0) 0x7f9e4015b6a0 con 0x7f9e400caea0 2023-09-23T03:15:35.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.370+0000 7f9e3effd640 1 -- 172.21.15.154:0/2452153034 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e4015b330 con 0x7f9e4015aba0 2023-09-23T03:15:35.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.371+0000 7f9e3effd640 1 --2- 172.21.15.154:0/2452153034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e4015aba0 0x7f9e4015bb30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9e4006cf20 tx=0x7f9e3002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd7b7fbef815805b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:35.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.372+0000 7f9e3e7fc640 1 -- 172.21.15.154:0/2452153034 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e30030860 con 0x7f9e4015aba0 2023-09-23T03:15:35.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.372+0000 7f9e3e7fc640 1 -- 172.21.15.154:0/2452153034 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e300309c0 con 0x7f9e4015aba0 2023-09-23T03:15:35.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.372+0000 7f9e3e7fc640 1 -- 172.21.15.154:0/2452153034 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e30030d50 con 0x7f9e4015aba0 2023-09-23T03:15:35.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.372+0000 7f9e4531d640 1 -- 172.21.15.154:0/2452153034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e4015aba0 msgr2=0x7f9e4015bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:35.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.372+0000 7f9e4531d640 1 --2- 172.21.15.154:0/2452153034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e4015aba0 0x7f9e4015bb30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9e4006cf20 tx=0x7f9e3002b9a0 comp rx=0 tx=0).stop 2023-09-23T03:15:35.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.372+0000 7f9e4531d640 1 -- 172.21.15.154:0/2452153034 shutdown_connections 2023-09-23T03:15:35.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.372+0000 7f9e4531d640 1 --2- 172.21.15.154:0/2452153034 >> v2:172.21.15.190:3300/0 conn(0x7f9e4015e5f0 0x7f9e4015ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:35.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.372+0000 7f9e4531d640 1 --2- 172.21.15.154:0/2452153034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e4015aba0 0x7f9e4015bb30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:35.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.372+0000 7f9e4531d640 1 -- 172.21.15.154:0/2452153034 >> 172.21.15.154:0/2452153034 conn(0x7f9e4006c010 msgr2=0x7f9e4006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:35.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.373+0000 7f9e4531d640 1 -- 172.21.15.154:0/2452153034 shutdown_connections 2023-09-23T03:15:35.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.373+0000 7f9e4531d640 1 -- 172.21.15.154:0/2452153034 wait complete. 2023-09-23T03:15:35.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.374+0000 7f9e4531d640 1 Processor -- start 2023-09-23T03:15:35.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.374+0000 7f9e4531d640 1 -- start start 2023-09-23T03:15:35.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.375+0000 7f9e4531d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e400caea0 0x7f9e401494b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:35.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.375+0000 7f9e4531d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e4015aba0 0x7f9e400c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:35.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.375+0000 7f9e4531d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e4015e5f0 0x7f9e40149d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:35.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.375+0000 7f9e4531d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e400c61f0 con 0x7f9e4015aba0 2023-09-23T03:15:35.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.375+0000 7f9e4531d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e400c5620 con 0x7f9e4015e5f0 2023-09-23T03:15:35.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.375+0000 7f9e4531d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e400c58c0 con 0x7f9e400caea0 2023-09-23T03:15:35.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.376+0000 7f9e37fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e4015aba0 0x7f9e400c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:35.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.376+0000 7f9e37fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e4015aba0 0x7f9e400c9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47772/0 (socket says 172.21.15.154:47772) 2023-09-23T03:15:35.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.376+0000 7f9e37fff640 1 -- 172.21.15.154:0/3091185975 learned_addr learned my addr 172.21.15.154:0/3091185975 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:35.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.376+0000 7f9e3effd640 1 --2- 172.21.15.154:0/3091185975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e400caea0 0x7f9e401494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:35.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.376+0000 7f9e3f7fe640 1 --2- 172.21.15.154:0/3091185975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e4015e5f0 0x7f9e40149d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:35.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.376+0000 7f9e37fff640 1 -- 172.21.15.154:0/3091185975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e400caea0 msgr2=0x7f9e401494b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:35.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.376+0000 7f9e37fff640 1 --2- 172.21.15.154:0/3091185975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e400caea0 0x7f9e401494b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:35.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.376+0000 7f9e37fff640 1 -- 172.21.15.154:0/3091185975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e4015e5f0 msgr2=0x7f9e40149d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:35.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.376+0000 7f9e37fff640 1 --2- 172.21.15.154:0/3091185975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e4015e5f0 0x7f9e40149d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:35.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.376+0000 7f9e37fff640 1 -- 172.21.15.154:0/3091185975 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e300094d0 con 0x7f9e4015aba0 2023-09-23T03:15:35.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.376+0000 7f9e3f7fe640 1 --2- 172.21.15.154:0/3091185975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e4015e5f0 0x7f9e40149d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:35.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.376+0000 7f9e37fff640 1 --2- 172.21.15.154:0/3091185975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e4015aba0 0x7f9e400c9cd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9e2800d5f0 tx=0x7f9e2800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:35.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.377+0000 7f9e377fe640 1 -- 172.21.15.154:0/3091185975 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e280125f0 con 0x7f9e4015aba0 2023-09-23T03:15:35.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.377+0000 7f9e377fe640 1 -- 172.21.15.154:0/3091185975 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e28012750 con 0x7f9e4015aba0 2023-09-23T03:15:35.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.377+0000 7f9e377fe640 1 -- 172.21.15.154:0/3091185975 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e28012b00 con 0x7f9e4015aba0 2023-09-23T03:15:35.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.377+0000 7f9e4531d640 1 -- 172.21.15.154:0/3091185975 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e4015b100 con 0x7f9e4015aba0 2023-09-23T03:15:35.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.377+0000 7f9e4531d640 1 -- 172.21.15.154:0/3091185975 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e400c5e80 con 0x7f9e4015aba0 2023-09-23T03:15:35.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.378+0000 7f9e4531d640 1 -- 172.21.15.154:0/3091185975 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e400c6010 con 0x7f9e4015aba0 2023-09-23T03:15:35.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.379+0000 7f9e377fe640 1 -- 172.21.15.154:0/3091185975 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f9e28012c60 con 0x7f9e4015aba0 2023-09-23T03:15:35.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.380+0000 7f9e377fe640 1 --2- 172.21.15.154:0/3091185975 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9e2006af60 0x7f9e2006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:35.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.380+0000 7f9e377fe640 1 -- 172.21.15.154:0/3091185975 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9e28089fa0 con 0x7f9e4015aba0 2023-09-23T03:15:35.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.380+0000 7f9e3effd640 1 --2- 172.21.15.154:0/3091185975 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9e2006af60 0x7f9e2006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:35.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.381+0000 7f9e3effd640 1 --2- 172.21.15.154:0/3091185975 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9e2006af60 0x7f9e2006d420 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9e30036000 tx=0x7f9e30008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:35.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.385+0000 7f9e377fe640 1 -- 172.21.15.154:0/3091185975 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f9e28054c50 con 0x7f9e4015aba0 2023-09-23T03:15:35.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.577+0000 7f9e4531d640 1 -- 172.21.15.154:0/3091185975 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9e4004e920 con 0x7f9e4015aba0 2023-09-23T03:15:35.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.578+0000 7f9e377fe640 1 -- 172.21.15.154:0/3091185975 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9e28058270 con 0x7f9e4015aba0 2023-09-23T03:15:35.578 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:15:35.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.580+0000 7f9e4531d640 1 -- 172.21.15.154:0/3091185975 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9e2006af60 msgr2=0x7f9e2006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:35.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.580+0000 7f9e4531d640 1 --2- 172.21.15.154:0/3091185975 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9e2006af60 0x7f9e2006d420 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9e30036000 tx=0x7f9e30008030 comp rx=0 tx=0).stop 2023-09-23T03:15:35.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.580+0000 7f9e4531d640 1 -- 172.21.15.154:0/3091185975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e4015aba0 msgr2=0x7f9e400c9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:35.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.580+0000 7f9e4531d640 1 --2- 172.21.15.154:0/3091185975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e4015aba0 0x7f9e400c9cd0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9e2800d5f0 tx=0x7f9e2800dab0 comp rx=0 tx=0).stop 2023-09-23T03:15:35.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.580+0000 7f9e4531d640 1 -- 172.21.15.154:0/3091185975 shutdown_connections 2023-09-23T03:15:35.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.580+0000 7f9e4531d640 1 --2- 172.21.15.154:0/3091185975 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9e2006af60 0x7f9e2006d420 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:35.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.580+0000 7f9e4531d640 1 --2- 172.21.15.154:0/3091185975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e4015e5f0 0x7f9e40149d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:35.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.580+0000 7f9e4531d640 1 --2- 172.21.15.154:0/3091185975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e4015aba0 0x7f9e400c9cd0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:35.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.580+0000 7f9e4531d640 1 --2- 172.21.15.154:0/3091185975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e400caea0 0x7f9e401494b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:35.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.580+0000 7f9e4531d640 1 -- 172.21.15.154:0/3091185975 >> 172.21.15.154:0/3091185975 conn(0x7f9e4006c010 msgr2=0x7f9e4015c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:35.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.580+0000 7f9e4531d640 1 -- 172.21.15.154:0/3091185975 shutdown_connections 2023-09-23T03:15:35.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.581+0000 7f9e4531d640 1 -- 172.21.15.154:0/3091185975 wait complete. 2023-09-23T03:15:35.600 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:15:35.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.814+0000 7fab63c61640 1 Processor -- start 2023-09-23T03:15:35.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.815+0000 7fab63c61640 1 -- start start 2023-09-23T03:15:35.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.816+0000 7fab63c61640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fab5c0c9510 0x7fab5c1636c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:35.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.816+0000 7fab63c61640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab5c0c9980 0x7fab5c163c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:35.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.816+0000 7fab63c61640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab5c0c1a40 con 0x7fab5c0cae90 2023-09-23T03:15:35.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.816+0000 7fab63c61640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab5c1624f0 con 0x7fab5c0c9510 2023-09-23T03:15:35.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.816+0000 7fab63c61640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c162be0 con 0x7fab5c0c9980 2023-09-23T03:15:35.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.817+0000 7fab619fd640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fab5c0cae90 0x7fab5c0cb270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:37894/0 (socket says 172.21.15.154:37894) 2023-09-23T03:15:35.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.817+0000 7fab619fd640 1 -- 172.21.15.154:0/2284216089 learned_addr learned my addr 172.21.15.154:0/2284216089 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:35.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.817+0000 7fab621fe640 1 --2- 172.21.15.154:0/2284216089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab5c0c9980 0x7fab5c163c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:35.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.817+0000 7fab611fc640 1 --2- 172.21.15.154:0/2284216089 >> v2:172.21.15.190:3300/0 conn(0x7fab5c0c9510 0x7fab5c1636c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:35.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.817+0000 7fab609fb640 1 -- 172.21.15.154:0/2284216089 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3491206263 0 0) 0x7fab5c0c1a40 con 0x7fab5c0cae90 2023-09-23T03:15:35.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.817+0000 7fab609fb640 1 -- 172.21.15.154:0/2284216089 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab48003520 con 0x7fab5c0cae90 2023-09-23T03:15:35.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.817+0000 7fab609fb640 1 -- 172.21.15.154:0/2284216089 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4220550744 0 0) 0x7fab48003520 con 0x7fab5c0cae90 2023-09-23T03:15:35.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.817+0000 7fab609fb640 1 -- 172.21.15.154:0/2284216089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab5c0c9980 msgr2=0x7fab5c163c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:35.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.817+0000 7fab609fb640 1 --2- 172.21.15.154:0/2284216089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab5c0c9980 0x7fab5c163c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:35.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.818+0000 7fab609fb640 1 -- 172.21.15.154:0/2284216089 >> v2:172.21.15.190:3300/0 conn(0x7fab5c0c9510 msgr2=0x7fab5c1636c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:35.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.818+0000 7fab609fb640 1 --2- 172.21.15.154:0/2284216089 >> v2:172.21.15.190:3300/0 conn(0x7fab5c0c9510 0x7fab5c1636c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:35.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.818+0000 7fab609fb640 1 -- 172.21.15.154:0/2284216089 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab5c162ec0 con 0x7fab5c0cae90 2023-09-23T03:15:35.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.818+0000 7fab609fb640 1 -- 172.21.15.154:0/2284216089 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fab58001670 con 0x7fab5c0cae90 2023-09-23T03:15:35.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.818+0000 7fab609fb640 1 -- 172.21.15.154:0/2284216089 >> v1:172.21.15.154:6789/0 conn(0x7fab5c0cae90 legacy=0x7fab5c0cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:35.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.818+0000 7fab609fb640 1 --2- 172.21.15.154:0/2284216089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab48003970 0x7fab48003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:35.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.818+0000 7fab609fb640 1 --2- 172.21.15.154:0/2284216089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab5c0c9980 0x7fab480047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:35.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.818+0000 7fab609fb640 1 --2- 172.21.15.154:0/2284216089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab48004d00 0x7fab48005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:35.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.818+0000 7fab609fb640 1 -- 172.21.15.154:0/2284216089 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c1624f0 con 0x7fab48003970 2023-09-23T03:15:35.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.818+0000 7fab609fb640 1 -- 172.21.15.154:0/2284216089 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c162be0 con 0x7fab48004d00 2023-09-23T03:15:35.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.818+0000 7fab609fb640 1 -- 172.21.15.154:0/2284216089 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab4800b980 con 0x7fab5c0c9980 2023-09-23T03:15:35.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.818+0000 7fab619fd640 1 --2- 172.21.15.154:0/2284216089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab48003970 0x7fab48003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:35.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.818+0000 7fab619fd640 1 -- 172.21.15.154:0/2284216089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab5c0c9980 msgr2=0x7fab480047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:35.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.818+0000 7fab619fd640 1 --2- 172.21.15.154:0/2284216089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab5c0c9980 0x7fab480047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:35.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.818+0000 7fab619fd640 1 -- 172.21.15.154:0/2284216089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab48004d00 msgr2=0x7fab48005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:35.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.818+0000 7fab619fd640 1 --2- 172.21.15.154:0/2284216089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab48004d00 0x7fab48005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:35.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.818+0000 7fab619fd640 1 -- 172.21.15.154:0/2284216089 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab48003520 con 0x7fab48003970 2023-09-23T03:15:35.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.819+0000 7fab619fd640 1 --2- 172.21.15.154:0/2284216089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab48003970 0x7fab48003d90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fab5800c240 tx=0x7fab480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8183232b8ae9bd77 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:35.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.820+0000 7fab609fb640 1 -- 172.21.15.154:0/2284216089 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5802cb00 con 0x7fab48003970 2023-09-23T03:15:35.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.820+0000 7fab609fb640 1 -- 172.21.15.154:0/2284216089 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5802cc60 con 0x7fab48003970 2023-09-23T03:15:35.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.820+0000 7fab609fb640 1 -- 172.21.15.154:0/2284216089 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab58005af0 con 0x7fab48003970 2023-09-23T03:15:35.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.820+0000 7fab63c61640 1 -- 172.21.15.154:0/2284216089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab48003970 msgr2=0x7fab48003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:35.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.820+0000 7fab63c61640 1 --2- 172.21.15.154:0/2284216089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab48003970 0x7fab48003d90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fab5800c240 tx=0x7fab480057f0 comp rx=0 tx=0).stop 2023-09-23T03:15:35.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.820+0000 7fab621fe640 1 -- 172.21.15.154:0/2284216089 reap_dead start 2023-09-23T03:15:35.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.820+0000 7fab63c61640 1 -- 172.21.15.154:0/2284216089 shutdown_connections 2023-09-23T03:15:35.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.820+0000 7fab63c61640 1 -- 172.21.15.154:0/2284216089 >> 172.21.15.154:0/2284216089 conn(0x7fab5c06c010 msgr2=0x7fab5c06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:35.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.821+0000 7fab63c61640 1 -- 172.21.15.154:0/2284216089 shutdown_connections 2023-09-23T03:15:35.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.821+0000 7fab63c61640 1 -- 172.21.15.154:0/2284216089 wait complete. 2023-09-23T03:15:35.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.822+0000 7fab63c61640 1 Processor -- start 2023-09-23T03:15:35.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.823+0000 7fab63c61640 1 -- start start 2023-09-23T03:15:35.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.823+0000 7fab63c61640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab48003970 0x7fab5c15f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:35.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.824+0000 7fab63c61640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab5c0b32b0 0x7fab5c0c73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:35.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.824+0000 7fab63c61640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab5c0b3730 0x7fab5c0c7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:35.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.824+0000 7fab63c61640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c0b2e20 con 0x7fab5c0b3730 2023-09-23T03:15:35.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.824+0000 7fab63c61640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c0b2210 con 0x7fab5c0b32b0 2023-09-23T03:15:35.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.824+0000 7fab63c61640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c0b24f0 con 0x7fab48003970 2023-09-23T03:15:35.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.824+0000 7fab621fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab5c0b3730 0x7fab5c0c7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:35.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.824+0000 7fab621fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab5c0b3730 0x7fab5c0c7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47796/0 (socket says 172.21.15.154:47796) 2023-09-23T03:15:35.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.824+0000 7fab621fe640 1 -- 172.21.15.154:0/2965058705 learned_addr learned my addr 172.21.15.154:0/2965058705 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:35.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.824+0000 7fab619fd640 1 --2- 172.21.15.154:0/2965058705 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab48003970 0x7fab5c15f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:35.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.824+0000 7fab611fc640 1 --2- 172.21.15.154:0/2965058705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab5c0b32b0 0x7fab5c0c73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:35.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.824+0000 7fab621fe640 1 -- 172.21.15.154:0/2965058705 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab48003970 msgr2=0x7fab5c15f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:35.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.824+0000 7fab621fe640 1 --2- 172.21.15.154:0/2965058705 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab48003970 0x7fab5c15f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:35.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.824+0000 7fab621fe640 1 -- 172.21.15.154:0/2965058705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab5c0b32b0 msgr2=0x7fab5c0c73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:35.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.824+0000 7fab621fe640 1 --2- 172.21.15.154:0/2965058705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab5c0b32b0 0x7fab5c0c73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:35.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.824+0000 7fab621fe640 1 -- 172.21.15.154:0/2965058705 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab5c0b27d0 con 0x7fab5c0b3730 2023-09-23T03:15:35.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.824+0000 7fab611fc640 1 --2- 172.21.15.154:0/2965058705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab5c0b32b0 0x7fab5c0c73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:35.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.824+0000 7fab619fd640 1 --2- 172.21.15.154:0/2965058705 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab48003970 0x7fab5c15f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:35.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.825+0000 7fab621fe640 1 --2- 172.21.15.154:0/2965058705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab5c0b3730 0x7fab5c0c7920 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fab5c0cbd10 tx=0x7fab5400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:35.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.825+0000 7fab527fc640 1 -- 172.21.15.154:0/2965058705 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab54010040 con 0x7fab5c0b3730 2023-09-23T03:15:35.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.825+0000 7fab527fc640 1 -- 172.21.15.154:0/2965058705 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5400bae0 con 0x7fab5c0b3730 2023-09-23T03:15:35.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.825+0000 7fab527fc640 1 -- 172.21.15.154:0/2965058705 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab54014400 con 0x7fab5c0b3730 2023-09-23T03:15:35.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.825+0000 7fab63c61640 1 -- 172.21.15.154:0/2965058705 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab5c0b2ab0 con 0x7fab5c0b3730 2023-09-23T03:15:35.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.825+0000 7fab63c61640 1 -- 172.21.15.154:0/2965058705 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab5c160510 con 0x7fab5c0b3730 2023-09-23T03:15:35.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.827+0000 7fab63c61640 1 -- 172.21.15.154:0/2965058705 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab5c0b2f90 con 0x7fab5c0b3730 2023-09-23T03:15:35.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.827+0000 7fab527fc640 1 -- 172.21.15.154:0/2965058705 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fab54014560 con 0x7fab5c0b3730 2023-09-23T03:15:35.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.828+0000 7fab527fc640 1 --2- 172.21.15.154:0/2965058705 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fab4006af60 0x7fab4006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:35.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.828+0000 7fab527fc640 1 -- 172.21.15.154:0/2965058705 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fab5408a0e0 con 0x7fab5c0b3730 2023-09-23T03:15:35.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.829+0000 7fab619fd640 1 --2- 172.21.15.154:0/2965058705 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fab4006af60 0x7fab4006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:35.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.829+0000 7fab619fd640 1 --2- 172.21.15.154:0/2965058705 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fab4006af60 0x7fab4006d420 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fab58034000 tx=0x7fab58007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:35.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:35.833+0000 7fab527fc640 1 -- 172.21.15.154:0/2965058705 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fab54054ec0 con 0x7fab5c0b3730 2023-09-23T03:15:36.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.023+0000 7fab63c61640 1 -- 172.21.15.154:0/2965058705 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fab5c04e920 con 0x7fab5c0b3730 2023-09-23T03:15:36.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.024+0000 7fab527fc640 1 -- 172.21.15.154:0/2965058705 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fab540584e0 con 0x7fab5c0b3730 2023-09-23T03:15:36.025 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:15:36.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.027+0000 7fab63c61640 1 -- 172.21.15.154:0/2965058705 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fab4006af60 msgr2=0x7fab4006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:36.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.027+0000 7fab63c61640 1 --2- 172.21.15.154:0/2965058705 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fab4006af60 0x7fab4006d420 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fab58034000 tx=0x7fab58007000 comp rx=0 tx=0).stop 2023-09-23T03:15:36.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.027+0000 7fab63c61640 1 -- 172.21.15.154:0/2965058705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab5c0b3730 msgr2=0x7fab5c0c7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:36.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.027+0000 7fab63c61640 1 --2- 172.21.15.154:0/2965058705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab5c0b3730 0x7fab5c0c7920 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fab5c0cbd10 tx=0x7fab5400b480 comp rx=0 tx=0).stop 2023-09-23T03:15:36.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.027+0000 7fab63c61640 1 -- 172.21.15.154:0/2965058705 shutdown_connections 2023-09-23T03:15:36.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.027+0000 7fab63c61640 1 --2- 172.21.15.154:0/2965058705 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fab4006af60 0x7fab4006d420 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:36.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.027+0000 7fab63c61640 1 --2- 172.21.15.154:0/2965058705 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab5c0b3730 0x7fab5c0c7920 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:36.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.027+0000 7fab63c61640 1 --2- 172.21.15.154:0/2965058705 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fab5c0b32b0 0x7fab5c0c73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:36.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.027+0000 7fab63c61640 1 --2- 172.21.15.154:0/2965058705 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fab48003970 0x7fab5c15f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:36.029 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.027+0000 7fab63c61640 1 -- 172.21.15.154:0/2965058705 >> 172.21.15.154:0/2965058705 conn(0x7fab5c06c010 msgr2=0x7fab5c070110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:36.029 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.028+0000 7fab63c61640 1 -- 172.21.15.154:0/2965058705 shutdown_connections 2023-09-23T03:15:36.029 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.028+0000 7fab63c61640 1 -- 172.21.15.154:0/2965058705 wait complete. 2023-09-23T03:15:36.047 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:15:36.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.267+0000 7fca43fff640 1 Processor -- start 2023-09-23T03:15:36.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.267+0000 7fca43fff640 1 -- start start 2023-09-23T03:15:36.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.269+0000 7fca43fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca44158480 0x7fca440cade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:36.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.269+0000 7fca43fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fca440705a0 0x7fca440cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:36.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.269+0000 7fca43fff640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca440cb6c0 con 0x7fca4415ba50 2023-09-23T03:15:36.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.269+0000 7fca43fff640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca4415ad80 con 0x7fca440705a0 2023-09-23T03:15:36.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.269+0000 7fca43fff640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca4415af70 con 0x7fca44158480 2023-09-23T03:15:36.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.270+0000 7fca42ffd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca44158480 0x7fca440cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:36.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.270+0000 7fca427fc640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fca4415ba50 0x7fca4406fe90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.154:49560/0 (socket says 172.21.15.154:49560) 2023-09-23T03:15:36.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.270+0000 7fca427fc640 1 -- 172.21.15.154:0/19958982 learned_addr learned my addr 172.21.15.154:0/19958982 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:36.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.270+0000 7fca437fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fca440705a0 0x7fca440cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:36.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.270+0000 7fca41ffb640 1 -- 172.21.15.154:0/19958982 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1502076354 0 0) 0x7fca440cb6c0 con 0x7fca4415ba50 2023-09-23T03:15:36.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.270+0000 7fca41ffb640 1 -- 172.21.15.154:0/19958982 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca2c003520 con 0x7fca4415ba50 2023-09-23T03:15:36.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.270+0000 7fca42ffd640 1 -- 172.21.15.154:0/19958982 >> v2:172.21.15.190:3300/0 conn(0x7fca440705a0 msgr2=0x7fca440cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:36.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.270+0000 7fca42ffd640 1 --2- 172.21.15.154:0/19958982 >> v2:172.21.15.190:3300/0 conn(0x7fca440705a0 0x7fca440cc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:36.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.270+0000 7fca42ffd640 1 -- 172.21.15.154:0/19958982 >> v1:172.21.15.190:6789/0 conn(0x7fca4415ba50 legacy=0x7fca4406fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:36.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.270+0000 7fca42ffd640 1 -- 172.21.15.154:0/19958982 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca4415b250 con 0x7fca44158480 2023-09-23T03:15:36.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.270+0000 7fca437fe640 1 --2- 172.21.15.154:0/19958982 >> v2:172.21.15.190:3300/0 conn(0x7fca440705a0 0x7fca440cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:15:36.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.272+0000 7fca42ffd640 1 --2- 172.21.15.154:0/19958982 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca44158480 0x7fca440cade0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fca340097a0 tx=0x7fca3402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f91ccfedc0d7966 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:36.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.272+0000 7fca41ffb640 1 -- 172.21.15.154:0/19958982 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3402d040 con 0x7fca44158480 2023-09-23T03:15:36.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.272+0000 7fca41ffb640 1 -- 172.21.15.154:0/19958982 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca34031810 con 0x7fca44158480 2023-09-23T03:15:36.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.272+0000 7fca41ffb640 1 -- 172.21.15.154:0/19958982 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca34031b80 con 0x7fca44158480 2023-09-23T03:15:36.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.273+0000 7fca43fff640 1 -- 172.21.15.154:0/19958982 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca44158480 msgr2=0x7fca440cade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:36.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.273+0000 7fca43fff640 1 --2- 172.21.15.154:0/19958982 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca44158480 0x7fca440cade0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fca340097a0 tx=0x7fca3402b8b0 comp rx=0 tx=0).stop 2023-09-23T03:15:36.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.273+0000 7fca43fff640 1 -- 172.21.15.154:0/19958982 shutdown_connections 2023-09-23T03:15:36.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.273+0000 7fca43fff640 1 --2- 172.21.15.154:0/19958982 >> v2:172.21.15.190:3300/0 conn(0x7fca440705a0 0x7fca440cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:36.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.273+0000 7fca43fff640 1 --2- 172.21.15.154:0/19958982 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca44158480 0x7fca440cade0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:36.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.273+0000 7fca43fff640 1 -- 172.21.15.154:0/19958982 >> 172.21.15.154:0/19958982 conn(0x7fca440aff20 msgr2=0x7fca440b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:36.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.273+0000 7fca43fff640 1 -- 172.21.15.154:0/19958982 shutdown_connections 2023-09-23T03:15:36.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.273+0000 7fca43fff640 1 -- 172.21.15.154:0/19958982 wait complete. 2023-09-23T03:15:36.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.275+0000 7fca43fff640 1 Processor -- start 2023-09-23T03:15:36.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.275+0000 7fca43fff640 1 -- start start 2023-09-23T03:15:36.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.276+0000 7fca43fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca440705a0 0x7fca440abc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:36.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.276+0000 7fca43fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca44158480 0x7fca44153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:36.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.276+0000 7fca43fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca4415ba50 0x7fca4415cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:36.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.276+0000 7fca43fff640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca440ac140 con 0x7fca44158480 2023-09-23T03:15:36.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.276+0000 7fca43fff640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca44141a60 con 0x7fca4415ba50 2023-09-23T03:15:36.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.276+0000 7fca43fff640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca440c8a20 con 0x7fca440705a0 2023-09-23T03:15:36.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.276+0000 7fca427fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca44158480 0x7fca44153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:36.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.276+0000 7fca427fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca44158480 0x7fca44153060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47798/0 (socket says 172.21.15.154:47798) 2023-09-23T03:15:36.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.276+0000 7fca427fc640 1 -- 172.21.15.154:0/481781456 learned_addr learned my addr 172.21.15.154:0/481781456 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:36.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.276+0000 7fca42ffd640 1 --2- 172.21.15.154:0/481781456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca440705a0 0x7fca440abc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:36.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.276+0000 7fca437fe640 1 --2- 172.21.15.154:0/481781456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca4415ba50 0x7fca4415cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:36.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.277+0000 7fca427fc640 1 -- 172.21.15.154:0/481781456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca440705a0 msgr2=0x7fca440abc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:36.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.277+0000 7fca427fc640 1 --2- 172.21.15.154:0/481781456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca440705a0 0x7fca440abc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:36.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.277+0000 7fca427fc640 1 -- 172.21.15.154:0/481781456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca4415ba50 msgr2=0x7fca4415cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:36.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.277+0000 7fca427fc640 1 --2- 172.21.15.154:0/481781456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca4415ba50 0x7fca4415cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:36.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.277+0000 7fca427fc640 1 -- 172.21.15.154:0/481781456 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3c009a70 con 0x7fca44158480 2023-09-23T03:15:36.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.277+0000 7fca427fc640 1 --2- 172.21.15.154:0/481781456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca44158480 0x7fca44153060 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fca3800c390 tx=0x7fca3800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:36.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.277+0000 7fca2b7fe640 1 -- 172.21.15.154:0/481781456 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca38005ce0 con 0x7fca44158480 2023-09-23T03:15:36.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.277+0000 7fca2b7fe640 1 -- 172.21.15.154:0/481781456 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca38005e40 con 0x7fca44158480 2023-09-23T03:15:36.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.277+0000 7fca2b7fe640 1 -- 172.21.15.154:0/481781456 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca380072f0 con 0x7fca44158480 2023-09-23T03:15:36.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.277+0000 7fca43fff640 1 -- 172.21.15.154:0/481781456 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca34009400 con 0x7fca44158480 2023-09-23T03:15:36.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.277+0000 7fca43fff640 1 -- 172.21.15.154:0/481781456 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca44153820 con 0x7fca44158480 2023-09-23T03:15:36.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.278+0000 7fca43fff640 1 -- 172.21.15.154:0/481781456 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca4404e920 con 0x7fca44158480 2023-09-23T03:15:36.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.280+0000 7fca2b7fe640 1 -- 172.21.15.154:0/481781456 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fca38007450 con 0x7fca44158480 2023-09-23T03:15:36.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.281+0000 7fca2b7fe640 1 --2- 172.21.15.154:0/481781456 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fca3006af30 0x7fca3006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:36.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.281+0000 7fca2b7fe640 1 -- 172.21.15.154:0/481781456 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fca380890d0 con 0x7fca44158480 2023-09-23T03:15:36.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.281+0000 7fca42ffd640 1 --2- 172.21.15.154:0/481781456 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fca3006af30 0x7fca3006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:36.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.282+0000 7fca42ffd640 1 --2- 172.21.15.154:0/481781456 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fca3006af30 0x7fca3006d3f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fca34036000 tx=0x7fca34008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:36.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.286+0000 7fca2b7fe640 1 -- 172.21.15.154:0/481781456 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fca38053e00 con 0x7fca44158480 2023-09-23T03:15:36.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.477+0000 7fca43fff640 1 -- 172.21.15.154:0/481781456 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fca44144390 con 0x7fca44158480 2023-09-23T03:15:36.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.478+0000 7fca2b7fe640 1 -- 172.21.15.154:0/481781456 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fca380170a0 con 0x7fca44158480 2023-09-23T03:15:36.478 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:15:36.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.480+0000 7fca43fff640 1 -- 172.21.15.154:0/481781456 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fca3006af30 msgr2=0x7fca3006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:36.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.480+0000 7fca43fff640 1 --2- 172.21.15.154:0/481781456 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fca3006af30 0x7fca3006d3f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fca34036000 tx=0x7fca34008030 comp rx=0 tx=0).stop 2023-09-23T03:15:36.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.481+0000 7fca43fff640 1 -- 172.21.15.154:0/481781456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca44158480 msgr2=0x7fca44153060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:36.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.481+0000 7fca43fff640 1 --2- 172.21.15.154:0/481781456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca44158480 0x7fca44153060 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fca3800c390 tx=0x7fca3800c850 comp rx=0 tx=0).stop 2023-09-23T03:15:36.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.481+0000 7fca43fff640 1 -- 172.21.15.154:0/481781456 shutdown_connections 2023-09-23T03:15:36.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.481+0000 7fca43fff640 1 --2- 172.21.15.154:0/481781456 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fca3006af30 0x7fca3006d3f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:36.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.481+0000 7fca43fff640 1 --2- 172.21.15.154:0/481781456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca4415ba50 0x7fca4415cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:36.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.481+0000 7fca43fff640 1 --2- 172.21.15.154:0/481781456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca44158480 0x7fca44153060 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:36.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.481+0000 7fca43fff640 1 --2- 172.21.15.154:0/481781456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca440705a0 0x7fca440abc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:36.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.481+0000 7fca43fff640 1 -- 172.21.15.154:0/481781456 >> 172.21.15.154:0/481781456 conn(0x7fca440aff20 msgr2=0x7fca4415a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:36.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.481+0000 7fca43fff640 1 -- 172.21.15.154:0/481781456 shutdown_connections 2023-09-23T03:15:36.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.481+0000 7fca43fff640 1 -- 172.21.15.154:0/481781456 wait complete. 2023-09-23T03:15:36.499 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-23T03:15:36.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.715+0000 7f023a9f5640 1 Processor -- start 2023-09-23T03:15:36.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.716+0000 7f023a9f5640 1 -- start start 2023-09-23T03:15:36.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.717+0000 7f023a9f5640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f023415aba0 0x7f023415bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:36.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.717+0000 7f023a9f5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f02340cb5b0 0x7f023415ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:36.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.717+0000 7f023a9f5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f023415b6a0 con 0x7f023415e5f0 2023-09-23T03:15:36.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.717+0000 7f023a9f5640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f023415f0b0 con 0x7f02340cb5b0 2023-09-23T03:15:36.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.717+0000 7f023a9f5640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f023415f390 con 0x7f023415aba0 2023-09-23T03:15:36.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.717+0000 7f0238f92640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f02340cb5b0 0x7f023415ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:36.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.718+0000 7f0238f92640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f02340cb5b0 0x7f023415ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47814/0 (socket says 172.21.15.154:47814) 2023-09-23T03:15:36.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.718+0000 7f0238f92640 1 -- 172.21.15.154:0/459279039 learned_addr learned my addr 172.21.15.154:0/459279039 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:36.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.718+0000 7f0233fff640 1 --2- 172.21.15.154:0/459279039 >> v2:172.21.15.190:3300/0 conn(0x7f023415aba0 0x7f023415bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:36.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.718+0000 7f0238f92640 1 -- 172.21.15.154:0/459279039 >> v2:172.21.15.190:3300/0 conn(0x7f023415aba0 msgr2=0x7f023415bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:36.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.718+0000 7f0238f92640 1 --2- 172.21.15.154:0/459279039 >> v2:172.21.15.190:3300/0 conn(0x7f023415aba0 0x7f023415bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:36.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.718+0000 7f0238f92640 1 -- 172.21.15.154:0/459279039 >> v1:172.21.15.154:6789/0 conn(0x7f023415e5f0 legacy=0x7f02340caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:36.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.718+0000 7f0238f92640 1 -- 172.21.15.154:0/459279039 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f023415b330 con 0x7f02340cb5b0 2023-09-23T03:15:36.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.718+0000 7f0232ffd640 1 -- 172.21.15.154:0/459279039 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2231890027 0 0) 0x7f023415b6a0 con 0x7f023415e5f0 2023-09-23T03:15:36.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.719+0000 7f0238f92640 1 --2- 172.21.15.154:0/459279039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02340cb5b0 0x7f023415ea30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f023406e1f0 tx=0x7f022c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de6a5a4636f3a083 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:36.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.720+0000 7f0232ffd640 1 -- 172.21.15.154:0/459279039 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022c030860 con 0x7f02340cb5b0 2023-09-23T03:15:36.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.720+0000 7f0232ffd640 1 -- 172.21.15.154:0/459279039 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f022c0309c0 con 0x7f02340cb5b0 2023-09-23T03:15:36.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.720+0000 7f0232ffd640 1 -- 172.21.15.154:0/459279039 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022c030d50 con 0x7f02340cb5b0 2023-09-23T03:15:36.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.720+0000 7f023a9f5640 1 -- 172.21.15.154:0/459279039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02340cb5b0 msgr2=0x7f023415ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:36.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.720+0000 7f023a9f5640 1 --2- 172.21.15.154:0/459279039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02340cb5b0 0x7f023415ea30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f023406e1f0 tx=0x7f022c02b9a0 comp rx=0 tx=0).stop 2023-09-23T03:15:36.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.720+0000 7f023a9f5640 1 -- 172.21.15.154:0/459279039 shutdown_connections 2023-09-23T03:15:36.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.720+0000 7f023a9f5640 1 --2- 172.21.15.154:0/459279039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02340cb5b0 0x7f023415ea30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:36.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.720+0000 7f023a9f5640 1 --2- 172.21.15.154:0/459279039 >> v2:172.21.15.190:3300/0 conn(0x7f023415aba0 0x7f023415bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:36.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.720+0000 7f023a9f5640 1 -- 172.21.15.154:0/459279039 >> 172.21.15.154:0/459279039 conn(0x7f023406c010 msgr2=0x7f023406c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:36.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.720+0000 7f023a9f5640 1 -- 172.21.15.154:0/459279039 shutdown_connections 2023-09-23T03:15:36.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.720+0000 7f023a9f5640 1 -- 172.21.15.154:0/459279039 wait complete. 2023-09-23T03:15:36.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.722+0000 7f023a9f5640 1 Processor -- start 2023-09-23T03:15:36.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.722+0000 7f023a9f5640 1 -- start start 2023-09-23T03:15:36.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.723+0000 7f023a9f5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02340cb5b0 0x7f0234154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:36.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.723+0000 7f023a9f5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f023415aba0 0x7f02340c3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:36.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.723+0000 7f023a9f5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f023415e5f0 0x7f02341512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:36.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.723+0000 7f023a9f5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02340c5a10 con 0x7f02340cb5b0 2023-09-23T03:15:36.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.723+0000 7f023a9f5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02340c5cf0 con 0x7f023415e5f0 2023-09-23T03:15:36.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.723+0000 7f023a9f5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02340c5fd0 con 0x7f023415aba0 2023-09-23T03:15:36.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.723+0000 7f0233fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02340cb5b0 0x7f0234154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:36.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.724+0000 7f0233fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02340cb5b0 0x7f0234154ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47830/0 (socket says 172.21.15.154:47830) 2023-09-23T03:15:36.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.724+0000 7f0233fff640 1 -- 172.21.15.154:0/31011370 learned_addr learned my addr 172.21.15.154:0/31011370 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:36.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.724+0000 7f0238f92640 1 --2- 172.21.15.154:0/31011370 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f023415e5f0 0x7f02341512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:36.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.724+0000 7f0233fff640 1 -- 172.21.15.154:0/31011370 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f023415aba0 msgr2=0x7f02340c3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:36.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.724+0000 7f02337fe640 1 --2- 172.21.15.154:0/31011370 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f023415aba0 0x7f02340c3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:36.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.724+0000 7f0233fff640 1 --2- 172.21.15.154:0/31011370 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f023415aba0 0x7f02340c3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:36.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.724+0000 7f0233fff640 1 -- 172.21.15.154:0/31011370 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f023415e5f0 msgr2=0x7f02341512a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:36.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.724+0000 7f0233fff640 1 --2- 172.21.15.154:0/31011370 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f023415e5f0 0x7f02341512a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:36.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.724+0000 7f0233fff640 1 -- 172.21.15.154:0/31011370 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f022c0094d0 con 0x7f02340cb5b0 2023-09-23T03:15:36.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.724+0000 7f0238f92640 1 --2- 172.21.15.154:0/31011370 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f023415e5f0 0x7f02341512a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:36.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.724+0000 7f0233fff640 1 --2- 172.21.15.154:0/31011370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02340cb5b0 0x7f0234154ec0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0224007ef0 tx=0x7f022400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:36.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.725+0000 7f0230ff9640 1 -- 172.21.15.154:0/31011370 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0224014620 con 0x7f02340cb5b0 2023-09-23T03:15:36.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.725+0000 7f0230ff9640 1 -- 172.21.15.154:0/31011370 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0224014780 con 0x7f02340cb5b0 2023-09-23T03:15:36.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.725+0000 7f0230ff9640 1 -- 172.21.15.154:0/31011370 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0224014b30 con 0x7f02340cb5b0 2023-09-23T03:15:36.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.725+0000 7f023a9f5640 1 -- 172.21.15.154:0/31011370 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f023415b100 con 0x7f02340cb5b0 2023-09-23T03:15:36.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.725+0000 7f023a9f5640 1 -- 172.21.15.154:0/31011370 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02340c39d0 con 0x7f02340cb5b0 2023-09-23T03:15:36.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.726+0000 7f023a9f5640 1 -- 172.21.15.154:0/31011370 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02340c3b90 con 0x7f02340cb5b0 2023-09-23T03:15:36.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.727+0000 7f0230ff9640 1 -- 172.21.15.154:0/31011370 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f0224014c90 con 0x7f02340cb5b0 2023-09-23T03:15:36.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.728+0000 7f0230ff9640 1 --2- 172.21.15.154:0/31011370 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f022006af60 0x7f022006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:36.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.728+0000 7f0230ff9640 1 -- 172.21.15.154:0/31011370 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f022408a460 con 0x7f02340cb5b0 2023-09-23T03:15:36.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.728+0000 7f02337fe640 1 --2- 172.21.15.154:0/31011370 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f022006af60 0x7f022006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:36.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.729+0000 7f02337fe640 1 --2- 172.21.15.154:0/31011370 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f022006af60 0x7f022006d420 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f02340ad750 tx=0x7f0228009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:36.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.732+0000 7f0230ff9640 1 -- 172.21.15.154:0/31011370 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f0224055110 con 0x7f02340cb5b0 2023-09-23T03:15:36.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.923+0000 7f023a9f5640 1 -- 172.21.15.154:0/31011370 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f023404e920 con 0x7f02340cb5b0 2023-09-23T03:15:36.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.924+0000 7f0230ff9640 1 -- 172.21.15.154:0/31011370 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0224058730 con 0x7f02340cb5b0 2023-09-23T03:15:36.924 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:15:36.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.927+0000 7f023a9f5640 1 -- 172.21.15.154:0/31011370 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f022006af60 msgr2=0x7f022006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:36.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.927+0000 7f023a9f5640 1 --2- 172.21.15.154:0/31011370 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f022006af60 0x7f022006d420 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f02340ad750 tx=0x7f0228009040 comp rx=0 tx=0).stop 2023-09-23T03:15:36.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.927+0000 7f023a9f5640 1 -- 172.21.15.154:0/31011370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02340cb5b0 msgr2=0x7f0234154ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:36.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.927+0000 7f023a9f5640 1 --2- 172.21.15.154:0/31011370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02340cb5b0 0x7f0234154ec0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0224007ef0 tx=0x7f022400e4b0 comp rx=0 tx=0).stop 2023-09-23T03:15:36.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.927+0000 7f023a9f5640 1 -- 172.21.15.154:0/31011370 shutdown_connections 2023-09-23T03:15:36.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.927+0000 7f023a9f5640 1 --2- 172.21.15.154:0/31011370 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f022006af60 0x7f022006d420 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:36.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.927+0000 7f023a9f5640 1 --2- 172.21.15.154:0/31011370 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f023415e5f0 0x7f02341512a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:36.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.927+0000 7f023a9f5640 1 --2- 172.21.15.154:0/31011370 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f023415aba0 0x7f02340c3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:36.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.927+0000 7f023a9f5640 1 --2- 172.21.15.154:0/31011370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02340cb5b0 0x7f0234154ec0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:36.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.927+0000 7f023a9f5640 1 -- 172.21.15.154:0/31011370 >> 172.21.15.154:0/31011370 conn(0x7f023406c010 msgr2=0x7f02340701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:36.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.927+0000 7f023a9f5640 1 -- 172.21.15.154:0/31011370 shutdown_connections 2023-09-23T03:15:36.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:36.927+0000 7f023a9f5640 1 -- 172.21.15.154:0/31011370 wait complete. 2023-09-23T03:15:36.945 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-23T03:15:37.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.164+0000 7fa8d256e640 1 Processor -- start 2023-09-23T03:15:37.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.165+0000 7fa8d256e640 1 -- start start 2023-09-23T03:15:37.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.166+0000 7fa8d256e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8cc15c9e0 0x7fa8cc0cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:37.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.167+0000 7fa8d256e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8cc0cd1e0 con 0x7fa8cc0d0c90 2023-09-23T03:15:37.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.167+0000 7fa8d256e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8cc0d0000 con 0x7fa8cc0b1d90 2023-09-23T03:15:37.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.167+0000 7fa8d256e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc0cfd20 con 0x7fa8cc15c9e0 2023-09-23T03:15:37.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.167+0000 7fa8d0d6b640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fa8cc0d0c90 0x7fa8cc0d10b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:37912/0 (socket says 172.21.15.154:37912) 2023-09-23T03:15:37.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.167+0000 7fa8d0d6b640 1 -- 172.21.15.154:0/2374124636 learned_addr learned my addr 172.21.15.154:0/2374124636 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:37.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.167+0000 7fa8d156c640 1 --2- 172.21.15.154:0/2374124636 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8cc15c9e0 0x7fa8cc0cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:37.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.168+0000 7fa8cbfff640 1 -- 172.21.15.154:0/2374124636 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4135230176 0 0) 0x7fa8cc0cd1e0 con 0x7fa8cc0d0c90 2023-09-23T03:15:37.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.168+0000 7fa8cbfff640 1 -- 172.21.15.154:0/2374124636 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8b4003520 con 0x7fa8cc0d0c90 2023-09-23T03:15:37.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.168+0000 7fa8cbfff640 1 -- 172.21.15.154:0/2374124636 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 392179644 0 0) 0x7fa8cc0d0000 con 0x7fa8cc0b1d90 2023-09-23T03:15:37.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.168+0000 7fa8cbfff640 1 -- 172.21.15.154:0/2374124636 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8cc0cd1e0 con 0x7fa8cc0b1d90 2023-09-23T03:15:37.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.168+0000 7fa8d156c640 1 -- 172.21.15.154:0/2374124636 >> v1:172.21.15.190:6789/0 conn(0x7fa8cc0b1d90 legacy=0x7fa8cc0b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:37.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.168+0000 7fa8d156c640 1 -- 172.21.15.154:0/2374124636 >> v1:172.21.15.154:6789/0 conn(0x7fa8cc0d0c90 legacy=0x7fa8cc0d10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:37.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.168+0000 7fa8d156c640 1 -- 172.21.15.154:0/2374124636 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8cc0cd410 con 0x7fa8cc15c9e0 2023-09-23T03:15:37.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.170+0000 7fa8d156c640 1 --2- 172.21.15.154:0/2374124636 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8cc15c9e0 0x7fa8cc0cdef0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc009dc0 tx=0x7fa8bc02bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29c6c7544b76486e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:37.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.170+0000 7fa8cbfff640 1 -- 172.21.15.154:0/2374124636 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc02ccd0 con 0x7fa8cc15c9e0 2023-09-23T03:15:37.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.170+0000 7fa8cbfff640 1 -- 172.21.15.154:0/2374124636 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8bc02ce30 con 0x7fa8cc15c9e0 2023-09-23T03:15:37.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.171+0000 7fa8cbfff640 1 -- 172.21.15.154:0/2374124636 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc034640 con 0x7fa8cc15c9e0 2023-09-23T03:15:37.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.171+0000 7fa8d256e640 1 -- 172.21.15.154:0/2374124636 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8cc15c9e0 msgr2=0x7fa8cc0cdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:37.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.171+0000 7fa8d256e640 1 --2- 172.21.15.154:0/2374124636 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8cc15c9e0 0x7fa8cc0cdef0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc009dc0 tx=0x7fa8bc02bf20 comp rx=0 tx=0).stop 2023-09-23T03:15:37.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.171+0000 7fa8d256e640 1 -- 172.21.15.154:0/2374124636 shutdown_connections 2023-09-23T03:15:37.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.171+0000 7fa8d256e640 1 --2- 172.21.15.154:0/2374124636 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8cc15c9e0 0x7fa8cc0cdef0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:37.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.171+0000 7fa8d256e640 1 -- 172.21.15.154:0/2374124636 >> 172.21.15.154:0/2374124636 conn(0x7fa8cc09ba50 msgr2=0x7fa8cc0d26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:37.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.171+0000 7fa8d256e640 1 -- 172.21.15.154:0/2374124636 shutdown_connections 2023-09-23T03:15:37.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.171+0000 7fa8d256e640 1 -- 172.21.15.154:0/2374124636 wait complete. 2023-09-23T03:15:37.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.173+0000 7fa8d256e640 1 Processor -- start 2023-09-23T03:15:37.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.173+0000 7fa8d256e640 1 -- start start 2023-09-23T03:15:37.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.174+0000 7fa8d256e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8cc0b1d90 0x7fa8cc0b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:37.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.174+0000 7fa8d256e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8cc0d0c90 0x7fa8cc0c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:37.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.174+0000 7fa8d256e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8cc15c9e0 0x7fa8cc0cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:37.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.174+0000 7fa8d256e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc1f2d00 con 0x7fa8cc15c9e0 2023-09-23T03:15:37.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.174+0000 7fa8d256e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc0c98d0 con 0x7fa8cc0d0c90 2023-09-23T03:15:37.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.174+0000 7fa8d256e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc0c9bb0 con 0x7fa8cc0b1d90 2023-09-23T03:15:37.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.175+0000 7fa8d1d6d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8cc15c9e0 0x7fa8cc0cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:37.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.175+0000 7fa8d1d6d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8cc15c9e0 0x7fa8cc0cbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47842/0 (socket says 172.21.15.154:47842) 2023-09-23T03:15:37.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.175+0000 7fa8d1d6d640 1 -- 172.21.15.154:0/2157652716 learned_addr learned my addr 172.21.15.154:0/2157652716 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:37.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.175+0000 7fa8d156c640 1 --2- 172.21.15.154:0/2157652716 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8cc0b1d90 0x7fa8cc0b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:37.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.175+0000 7fa8d0d6b640 1 --2- 172.21.15.154:0/2157652716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8cc0d0c90 0x7fa8cc0c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:37.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.175+0000 7fa8d1d6d640 1 -- 172.21.15.154:0/2157652716 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8cc0b1d90 msgr2=0x7fa8cc0b52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:37.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.175+0000 7fa8d1d6d640 1 --2- 172.21.15.154:0/2157652716 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8cc0b1d90 0x7fa8cc0b52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:37.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.175+0000 7fa8d1d6d640 1 -- 172.21.15.154:0/2157652716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8cc0d0c90 msgr2=0x7fa8cc0c88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:37.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.175+0000 7fa8d1d6d640 1 --2- 172.21.15.154:0/2157652716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8cc0d0c90 0x7fa8cc0c88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:37.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.175+0000 7fa8d1d6d640 1 -- 172.21.15.154:0/2157652716 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8bc009a70 con 0x7fa8cc15c9e0 2023-09-23T03:15:37.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.175+0000 7fa8d0d6b640 1 --2- 172.21.15.154:0/2157652716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8cc0d0c90 0x7fa8cc0c88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:37.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.176+0000 7fa8d1d6d640 1 --2- 172.21.15.154:0/2157652716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8cc15c9e0 0x7fa8cc0cbc20 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4001eb0 tx=0x7fa8c400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:37.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.176+0000 7fa8c9ffb640 1 -- 172.21.15.154:0/2157652716 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c40052d0 con 0x7fa8cc15c9e0 2023-09-23T03:15:37.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.176+0000 7fa8c9ffb640 1 -- 172.21.15.154:0/2157652716 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c4005430 con 0x7fa8cc15c9e0 2023-09-23T03:15:37.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.176+0000 7fa8c9ffb640 1 -- 172.21.15.154:0/2157652716 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c40037f0 con 0x7fa8cc15c9e0 2023-09-23T03:15:37.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.176+0000 7fa8d256e640 1 -- 172.21.15.154:0/2157652716 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8cc0c9e90 con 0x7fa8cc15c9e0 2023-09-23T03:15:37.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.176+0000 7fa8d256e640 1 -- 172.21.15.154:0/2157652716 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8cc0c8e00 con 0x7fa8cc15c9e0 2023-09-23T03:15:37.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.177+0000 7fa8d256e640 1 -- 172.21.15.154:0/2157652716 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8cc0ca3b0 con 0x7fa8cc15c9e0 2023-09-23T03:15:37.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.178+0000 7fa8c9ffb640 1 -- 172.21.15.154:0/2157652716 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fa8c4003950 con 0x7fa8cc15c9e0 2023-09-23T03:15:37.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.179+0000 7fa8c9ffb640 1 --2- 172.21.15.154:0/2157652716 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa8b806af30 0x7fa8b806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:37.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.179+0000 7fa8c9ffb640 1 -- 172.21.15.154:0/2157652716 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa8c4088e50 con 0x7fa8cc15c9e0 2023-09-23T03:15:37.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.179+0000 7fa8d156c640 1 --2- 172.21.15.154:0/2157652716 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa8b806af30 0x7fa8b806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:37.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.180+0000 7fa8d156c640 1 --2- 172.21.15.154:0/2157652716 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa8b806af30 0x7fa8b806d3f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc036000 tx=0x7fa8bc030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:37.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.183+0000 7fa8c9ffb640 1 -- 172.21.15.154:0/2157652716 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fa8c4053c40 con 0x7fa8cc15c9e0 2023-09-23T03:15:37.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.382+0000 7fa8d256e640 1 -- 172.21.15.154:0/2157652716 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa8cc04e920 con 0x7fa8cc15c9e0 2023-09-23T03:15:37.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.383+0000 7fa8c9ffb640 1 -- 172.21.15.154:0/2157652716 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa8c4017080 con 0x7fa8cc15c9e0 2023-09-23T03:15:37.383 INFO:teuthology.orchestra.run.smithi154.stderr:listed 0 entries 2023-09-23T03:15:37.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.386+0000 7fa8d256e640 1 -- 172.21.15.154:0/2157652716 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa8b806af30 msgr2=0x7fa8b806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:37.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.386+0000 7fa8d256e640 1 --2- 172.21.15.154:0/2157652716 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa8b806af30 0x7fa8b806d3f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc036000 tx=0x7fa8bc030000 comp rx=0 tx=0).stop 2023-09-23T03:15:37.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.386+0000 7fa8d256e640 1 -- 172.21.15.154:0/2157652716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8cc15c9e0 msgr2=0x7fa8cc0cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:37.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.386+0000 7fa8d256e640 1 --2- 172.21.15.154:0/2157652716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8cc15c9e0 0x7fa8cc0cbc20 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4001eb0 tx=0x7fa8c400c820 comp rx=0 tx=0).stop 2023-09-23T03:15:37.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.386+0000 7fa8d256e640 1 -- 172.21.15.154:0/2157652716 shutdown_connections 2023-09-23T03:15:37.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.386+0000 7fa8d256e640 1 --2- 172.21.15.154:0/2157652716 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa8b806af30 0x7fa8b806d3f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:37.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.386+0000 7fa8d256e640 1 --2- 172.21.15.154:0/2157652716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8cc15c9e0 0x7fa8cc0cbc20 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:37.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.386+0000 7fa8d256e640 1 --2- 172.21.15.154:0/2157652716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa8cc0d0c90 0x7fa8cc0c88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:37.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.386+0000 7fa8d256e640 1 --2- 172.21.15.154:0/2157652716 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa8cc0b1d90 0x7fa8cc0b52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:37.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.386+0000 7fa8d256e640 1 -- 172.21.15.154:0/2157652716 >> 172.21.15.154:0/2157652716 conn(0x7fa8cc09ba50 msgr2=0x7fa8cc06c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:37.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.386+0000 7fa8d256e640 1 -- 172.21.15.154:0/2157652716 shutdown_connections 2023-09-23T03:15:37.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.386+0000 7fa8d256e640 1 -- 172.21.15.154:0/2157652716 wait complete. 2023-09-23T03:15:37.403 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:15:37.403 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:15:37.453 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:15:37.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.717+0000 7fb767535640 1 Processor -- start 2023-09-23T03:15:37.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.717+0000 7fb767535640 1 -- start start 2023-09-23T03:15:37.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.719+0000 7fb767535640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb7600cf500 0x7fb7600ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:37.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.719+0000 7fb767535640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb7600cf8e0 0x7fb7600ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:37.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.719+0000 7fb767535640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7600d17f0 con 0x7fb76015c4f0 2023-09-23T03:15:37.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.719+0000 7fb767535640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb76006be10 con 0x7fb7600cf500 2023-09-23T03:15:37.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.719+0000 7fb767535640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7600d0470 con 0x7fb7600cf8e0 2023-09-23T03:15:37.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.719+0000 7fb766533640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb7600cf500 0x7fb7600ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:37.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.719+0000 7fb766d34640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fb76015c4f0 0x7fb76015c960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:37918/0 (socket says 172.21.15.154:37918) 2023-09-23T03:15:37.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.719+0000 7fb766d34640 1 -- 172.21.15.154:0/724636806 learned_addr learned my addr 172.21.15.154:0/724636806 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:37.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.720+0000 7fb765d32640 1 --2- 172.21.15.154:0/724636806 >> v2:172.21.15.190:3300/0 conn(0x7fb7600cf8e0 0x7fb7600ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:37.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.720+0000 7fb766533640 1 -- 172.21.15.154:0/724636806 >> v2:172.21.15.190:3300/0 conn(0x7fb7600cf8e0 msgr2=0x7fb7600ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:37.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.720+0000 7fb766533640 1 --2- 172.21.15.154:0/724636806 >> v2:172.21.15.190:3300/0 conn(0x7fb7600cf8e0 0x7fb7600ce590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:37.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.720+0000 7fb766533640 1 -- 172.21.15.154:0/724636806 >> v1:172.21.15.154:6789/0 conn(0x7fb76015c4f0 legacy=0x7fb76015c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:37.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.720+0000 7fb766533640 1 -- 172.21.15.154:0/724636806 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7600d0750 con 0x7fb7600cf500 2023-09-23T03:15:37.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.720+0000 7fb765531640 1 -- 172.21.15.154:0/724636806 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3384510614 0 0) 0x7fb7600d17f0 con 0x7fb76015c4f0 2023-09-23T03:15:37.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.721+0000 7fb766533640 1 --2- 172.21.15.154:0/724636806 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7600cf500 0x7fb7600ce050 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb75c009790 tx=0x7fb75c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b004cb3f556a561 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:37.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.722+0000 7fb765531640 1 -- 172.21.15.154:0/724636806 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75c02e070 con 0x7fb7600cf500 2023-09-23T03:15:37.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.722+0000 7fb765531640 1 -- 172.21.15.154:0/724636806 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb75c0315a0 con 0x7fb7600cf500 2023-09-23T03:15:37.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.722+0000 7fb765531640 1 -- 172.21.15.154:0/724636806 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75c031930 con 0x7fb7600cf500 2023-09-23T03:15:37.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.722+0000 7fb767535640 1 -- 172.21.15.154:0/724636806 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7600cf500 msgr2=0x7fb7600ce050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:37.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.722+0000 7fb767535640 1 --2- 172.21.15.154:0/724636806 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7600cf500 0x7fb7600ce050 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb75c009790 tx=0x7fb75c02b840 comp rx=0 tx=0).stop 2023-09-23T03:15:37.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.723+0000 7fb767535640 1 -- 172.21.15.154:0/724636806 shutdown_connections 2023-09-23T03:15:37.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.723+0000 7fb767535640 1 --2- 172.21.15.154:0/724636806 >> v2:172.21.15.190:3300/0 conn(0x7fb7600cf8e0 0x7fb7600ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:37.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.723+0000 7fb767535640 1 --2- 172.21.15.154:0/724636806 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7600cf500 0x7fb7600ce050 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:37.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.723+0000 7fb767535640 1 -- 172.21.15.154:0/724636806 >> 172.21.15.154:0/724636806 conn(0x7fb76009bbb0 msgr2=0x7fb7600d28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:37.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.723+0000 7fb767535640 1 -- 172.21.15.154:0/724636806 shutdown_connections 2023-09-23T03:15:37.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.723+0000 7fb767535640 1 -- 172.21.15.154:0/724636806 wait complete. 2023-09-23T03:15:37.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.724+0000 7fb767535640 1 Processor -- start 2023-09-23T03:15:37.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.724+0000 7fb767535640 1 -- start start 2023-09-23T03:15:37.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.725+0000 7fb767535640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7600cf500 0x7fb7601549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:37.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.725+0000 7fb767535640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7600cf8e0 0x7fb760156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:37.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.725+0000 7fb767535640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb76015c4f0 0x7fb760151d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:37.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.725+0000 7fb767535640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76015ca00 con 0x7fb7600cf500 2023-09-23T03:15:37.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.725+0000 7fb767535640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76015cce0 con 0x7fb76015c4f0 2023-09-23T03:15:37.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.725+0000 7fb767535640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb76015cfc0 con 0x7fb7600cf8e0 2023-09-23T03:15:37.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.726+0000 7fb766533640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7600cf500 0x7fb7601549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:37.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.726+0000 7fb766533640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7600cf500 0x7fb7601549d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47858/0 (socket says 172.21.15.154:47858) 2023-09-23T03:15:37.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.726+0000 7fb766533640 1 -- 172.21.15.154:0/964522096 learned_addr learned my addr 172.21.15.154:0/964522096 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:37.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.726+0000 7fb765d32640 1 --2- 172.21.15.154:0/964522096 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7600cf8e0 0x7fb760156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:37.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.726+0000 7fb766d34640 1 --2- 172.21.15.154:0/964522096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb76015c4f0 0x7fb760151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:37.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.726+0000 7fb766533640 1 -- 172.21.15.154:0/964522096 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7600cf8e0 msgr2=0x7fb760156210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:37.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.726+0000 7fb766533640 1 --2- 172.21.15.154:0/964522096 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7600cf8e0 0x7fb760156210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:37.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.726+0000 7fb766533640 1 -- 172.21.15.154:0/964522096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb76015c4f0 msgr2=0x7fb760151d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:37.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.726+0000 7fb766533640 1 --2- 172.21.15.154:0/964522096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb76015c4f0 0x7fb760151d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:37.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.726+0000 7fb766533640 1 -- 172.21.15.154:0/964522096 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb75c0093a0 con 0x7fb7600cf500 2023-09-23T03:15:37.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.726+0000 7fb766d34640 1 --2- 172.21.15.154:0/964522096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb76015c4f0 0x7fb760151d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:37.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.727+0000 7fb766533640 1 --2- 172.21.15.154:0/964522096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7600cf500 0x7fb7601549d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb75c000c00 tx=0x7fb75c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:37.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.727+0000 7fb756ffd640 1 -- 172.21.15.154:0/964522096 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75c004230 con 0x7fb7600cf500 2023-09-23T03:15:37.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.727+0000 7fb756ffd640 1 -- 172.21.15.154:0/964522096 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb75c004390 con 0x7fb7600cf500 2023-09-23T03:15:37.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.727+0000 7fb767535640 1 -- 172.21.15.154:0/964522096 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb76015d2a0 con 0x7fb7600cf500 2023-09-23T03:15:37.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.727+0000 7fb756ffd640 1 -- 172.21.15.154:0/964522096 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75c005bd0 con 0x7fb7600cf500 2023-09-23T03:15:37.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.727+0000 7fb767535640 1 -- 172.21.15.154:0/964522096 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb760156750 con 0x7fb7600cf500 2023-09-23T03:15:37.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.728+0000 7fb767535640 1 -- 172.21.15.154:0/964522096 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb76004e920 con 0x7fb7600cf500 2023-09-23T03:15:37.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.729+0000 7fb756ffd640 1 -- 172.21.15.154:0/964522096 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fb75c032640 con 0x7fb7600cf500 2023-09-23T03:15:37.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.730+0000 7fb756ffd640 1 --2- 172.21.15.154:0/964522096 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb74406af60 0x7fb74406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:37.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.730+0000 7fb756ffd640 1 -- 172.21.15.154:0/964522096 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb75c0ab740 con 0x7fb7600cf500 2023-09-23T03:15:37.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.730+0000 7fb765d32640 1 --2- 172.21.15.154:0/964522096 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb74406af60 0x7fb74406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:37.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.731+0000 7fb765d32640 1 --2- 172.21.15.154:0/964522096 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb74406af60 0x7fb74406d420 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb76015f9d0 tx=0x7fb750008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:37.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.735+0000 7fb756ffd640 1 -- 172.21.15.154:0/964522096 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fb75c0765a0 con 0x7fb7600cf500 2023-09-23T03:15:37.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.926+0000 7fb767535640 1 -- 172.21.15.154:0/964522096 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb7600cab90 con 0x7fb7600cf500 2023-09-23T03:15:37.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.928+0000 7fb756ffd640 1 -- 172.21.15.154:0/964522096 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18382 (secure 0 0 0) 0x7fb75c079bc0 con 0x7fb7600cf500 2023-09-23T03:15:37.929 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:15:37.929 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":24,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:15:25.775060+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:15:18.575849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T03:15:19.707212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T03:15:23.337138+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:15:37.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.931+0000 7fb767535640 1 -- 172.21.15.154:0/964522096 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb74406af60 msgr2=0x7fb74406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:37.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.931+0000 7fb767535640 1 --2- 172.21.15.154:0/964522096 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb74406af60 0x7fb74406d420 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb76015f9d0 tx=0x7fb750008040 comp rx=0 tx=0).stop 2023-09-23T03:15:37.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.931+0000 7fb767535640 1 -- 172.21.15.154:0/964522096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7600cf500 msgr2=0x7fb7601549d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:37.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.931+0000 7fb767535640 1 --2- 172.21.15.154:0/964522096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7600cf500 0x7fb7601549d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb75c000c00 tx=0x7fb75c0046c0 comp rx=0 tx=0).stop 2023-09-23T03:15:37.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.931+0000 7fb767535640 1 -- 172.21.15.154:0/964522096 shutdown_connections 2023-09-23T03:15:37.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.931+0000 7fb767535640 1 --2- 172.21.15.154:0/964522096 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb74406af60 0x7fb74406d420 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:37.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.931+0000 7fb767535640 1 --2- 172.21.15.154:0/964522096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb76015c4f0 0x7fb760151d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:37.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.931+0000 7fb767535640 1 --2- 172.21.15.154:0/964522096 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7600cf8e0 0x7fb760156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:37.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.931+0000 7fb767535640 1 --2- 172.21.15.154:0/964522096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7600cf500 0x7fb7601549d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:37.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.931+0000 7fb767535640 1 -- 172.21.15.154:0/964522096 >> 172.21.15.154:0/964522096 conn(0x7fb76009bbb0 msgr2=0x7fb7600cb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:37.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.931+0000 7fb767535640 1 -- 172.21.15.154:0/964522096 shutdown_connections 2023-09-23T03:15:37.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:37.931+0000 7fb767535640 1 -- 172.21.15.154:0/964522096 wait complete. 2023-09-23T03:15:37.949 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:15:18.575849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:15:19.707212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-23T03:15:23.337138+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-23T03:15:37.949 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:15:38.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.164+0000 7ff6dccda640 1 Processor -- start 2023-09-23T03:15:38.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.164+0000 7ff6dccda640 1 -- start start 2023-09-23T03:15:38.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.166+0000 7ff6dccda640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff6d80c9150 0x7ff6d815cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:38.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.166+0000 7ff6dccda640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6d80c9d20 con 0x7ff6d8071290 2023-09-23T03:15:38.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.166+0000 7ff6dccda640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6d8071850 con 0x7ff6d80c8d70 2023-09-23T03:15:38.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.166+0000 7ff6dccda640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6d8071b30 con 0x7ff6d80c9150 2023-09-23T03:15:38.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.166+0000 7ff6d6d9d640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff6d8071290 0x7ff6d80ca1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:37930/0 (socket says 172.21.15.154:37930) 2023-09-23T03:15:38.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.166+0000 7ff6d6d9d640 1 -- 172.21.15.154:0/630203138 learned_addr learned my addr 172.21.15.154:0/630203138 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:38.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.166+0000 7ff6d5d9b640 1 --2- 172.21.15.154:0/630203138 >> v2:172.21.15.154:3300/0 conn(0x7ff6d80c9150 0x7ff6d815cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:38.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.166+0000 7ff6d5d9b640 1 -- 172.21.15.154:0/630203138 >> v1:172.21.15.190:6789/0 conn(0x7ff6d80c8d70 legacy=0x7ff6d815c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:38.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.166+0000 7ff6d5d9b640 1 -- 172.21.15.154:0/630203138 >> v1:172.21.15.154:6789/0 conn(0x7ff6d8071290 legacy=0x7ff6d80ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:38.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.166+0000 7ff6d5d9b640 1 -- 172.21.15.154:0/630203138 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d8071e10 con 0x7ff6d80c9150 2023-09-23T03:15:38.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.167+0000 7ff6d5d9b640 1 --2- 172.21.15.154:0/630203138 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6d80c9150 0x7ff6d815cde0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff6c00097a0 tx=0x7ff6c002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a8ec13ffa5d2f8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:38.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.168+0000 7ff6d559a640 1 -- 172.21.15.154:0/630203138 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c0031830 con 0x7ff6d80c9150 2023-09-23T03:15:38.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.168+0000 7ff6d559a640 1 -- 172.21.15.154:0/630203138 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6c0031990 con 0x7ff6d80c9150 2023-09-23T03:15:38.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.168+0000 7ff6d559a640 1 -- 172.21.15.154:0/630203138 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c0031d20 con 0x7ff6d80c9150 2023-09-23T03:15:38.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.168+0000 7ff6dccda640 1 -- 172.21.15.154:0/630203138 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6d80c9150 msgr2=0x7ff6d815cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:38.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.168+0000 7ff6dccda640 1 --2- 172.21.15.154:0/630203138 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6d80c9150 0x7ff6d815cde0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff6c00097a0 tx=0x7ff6c002b8d0 comp rx=0 tx=0).stop 2023-09-23T03:15:38.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.168+0000 7ff6dccda640 1 -- 172.21.15.154:0/630203138 shutdown_connections 2023-09-23T03:15:38.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.168+0000 7ff6dccda640 1 --2- 172.21.15.154:0/630203138 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6d80c9150 0x7ff6d815cde0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:38.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.168+0000 7ff6dccda640 1 -- 172.21.15.154:0/630203138 >> 172.21.15.154:0/630203138 conn(0x7ff6d806c010 msgr2=0x7ff6d806c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:38.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.168+0000 7ff6dccda640 1 -- 172.21.15.154:0/630203138 shutdown_connections 2023-09-23T03:15:38.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.168+0000 7ff6dccda640 1 -- 172.21.15.154:0/630203138 wait complete. 2023-09-23T03:15:38.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.170+0000 7ff6dccda640 1 Processor -- start 2023-09-23T03:15:38.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.170+0000 7ff6dccda640 1 -- start start 2023-09-23T03:15:38.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.171+0000 7ff6dccda640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6d8071290 0x7ff6d80af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:38.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.171+0000 7ff6dccda640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6d80c8d70 0x7ff6d814b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:38.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.171+0000 7ff6dccda640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6d80c9150 0x7ff6d80c71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:38.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.171+0000 7ff6dccda640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d80b0d30 con 0x7ff6d8071290 2023-09-23T03:15:38.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.171+0000 7ff6dccda640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d80c7a30 con 0x7ff6d80c8d70 2023-09-23T03:15:38.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.171+0000 7ff6dccda640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d80c7d10 con 0x7ff6d80c9150 2023-09-23T03:15:38.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.171+0000 7ff6d659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6d8071290 0x7ff6d80af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:38.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.171+0000 7ff6d659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6d8071290 0x7ff6d80af2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47884/0 (socket says 172.21.15.154:47884) 2023-09-23T03:15:38.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.171+0000 7ff6d659c640 1 -- 172.21.15.154:0/722334233 learned_addr learned my addr 172.21.15.154:0/722334233 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:38.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.172+0000 7ff6d6d9d640 1 --2- 172.21.15.154:0/722334233 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6d80c9150 0x7ff6d80c71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:38.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.172+0000 7ff6d659c640 1 -- 172.21.15.154:0/722334233 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6d80c9150 msgr2=0x7ff6d80c71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:38.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.172+0000 7ff6d659c640 1 --2- 172.21.15.154:0/722334233 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6d80c9150 0x7ff6d80c71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:38.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.172+0000 7ff6d659c640 1 -- 172.21.15.154:0/722334233 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6d80c8d70 msgr2=0x7ff6d814b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:38.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.172+0000 7ff6d659c640 1 --2- 172.21.15.154:0/722334233 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6d80c8d70 0x7ff6d814b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:38.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.172+0000 7ff6d659c640 1 -- 172.21.15.154:0/722334233 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c0009400 con 0x7ff6d8071290 2023-09-23T03:15:38.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.172+0000 7ff6d659c640 1 --2- 172.21.15.154:0/722334233 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6d8071290 0x7ff6d80af2e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff6c800c7c0 tx=0x7ff6c800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:38.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.172+0000 7ff6ceffd640 1 -- 172.21.15.154:0/722334233 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c800aa50 con 0x7ff6d8071290 2023-09-23T03:15:38.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.172+0000 7ff6ceffd640 1 -- 172.21.15.154:0/722334233 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6c800abb0 con 0x7ff6d8071290 2023-09-23T03:15:38.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.172+0000 7ff6ceffd640 1 -- 172.21.15.154:0/722334233 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c8012420 con 0x7ff6d8071290 2023-09-23T03:15:38.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.172+0000 7ff6dccda640 1 -- 172.21.15.154:0/722334233 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d80c7ff0 con 0x7ff6d8071290 2023-09-23T03:15:38.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.172+0000 7ff6dccda640 1 -- 172.21.15.154:0/722334233 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d80c76f0 con 0x7ff6d8071290 2023-09-23T03:15:38.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.174+0000 7ff6dccda640 1 -- 172.21.15.154:0/722334233 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6d804e920 con 0x7ff6d8071290 2023-09-23T03:15:38.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.175+0000 7ff6ceffd640 1 -- 172.21.15.154:0/722334233 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7ff6c8012580 con 0x7ff6d8071290 2023-09-23T03:15:38.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.175+0000 7ff6ceffd640 1 --2- 172.21.15.154:0/722334233 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff6b806af30 0x7ff6b806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:38.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.176+0000 7ff6ceffd640 1 -- 172.21.15.154:0/722334233 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff6c8089990 con 0x7ff6d8071290 2023-09-23T03:15:38.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.176+0000 7ff6d5d9b640 1 --2- 172.21.15.154:0/722334233 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff6b806af30 0x7ff6b806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:38.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.176+0000 7ff6d5d9b640 1 --2- 172.21.15.154:0/722334233 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff6b806af30 0x7ff6b806d3f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0036000 tx=0x7ff6c0030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:38.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.180+0000 7ff6ceffd640 1 -- 172.21.15.154:0/722334233 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7ff6c8054770 con 0x7ff6d8071290 2023-09-23T03:15:38.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.367+0000 7ff6dccda640 1 -- 172.21.15.154:0/722334233 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff6d80b0b00 con 0x7ff6d8071290 2023-09-23T03:15:38.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.368+0000 7ff6ceffd640 1 -- 172.21.15.154:0/722334233 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff6c80160a0 con 0x7ff6d8071290 2023-09-23T03:15:38.368 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:15:38.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.371+0000 7ff6dccda640 1 -- 172.21.15.154:0/722334233 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff6b806af30 msgr2=0x7ff6b806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:38.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.371+0000 7ff6dccda640 1 --2- 172.21.15.154:0/722334233 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff6b806af30 0x7ff6b806d3f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0036000 tx=0x7ff6c0030000 comp rx=0 tx=0).stop 2023-09-23T03:15:38.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.371+0000 7ff6dccda640 1 -- 172.21.15.154:0/722334233 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6d8071290 msgr2=0x7ff6d80af2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:38.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.371+0000 7ff6dccda640 1 --2- 172.21.15.154:0/722334233 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6d8071290 0x7ff6d80af2e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff6c800c7c0 tx=0x7ff6c800cc80 comp rx=0 tx=0).stop 2023-09-23T03:15:38.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.371+0000 7ff6dccda640 1 -- 172.21.15.154:0/722334233 shutdown_connections 2023-09-23T03:15:38.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.371+0000 7ff6dccda640 1 --2- 172.21.15.154:0/722334233 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff6b806af30 0x7ff6b806d3f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:38.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.371+0000 7ff6dccda640 1 --2- 172.21.15.154:0/722334233 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6d80c9150 0x7ff6d80c71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:38.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.371+0000 7ff6dccda640 1 --2- 172.21.15.154:0/722334233 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6d80c8d70 0x7ff6d814b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:38.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.371+0000 7ff6dccda640 1 --2- 172.21.15.154:0/722334233 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6d8071290 0x7ff6d80af2e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:38.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.371+0000 7ff6dccda640 1 -- 172.21.15.154:0/722334233 >> 172.21.15.154:0/722334233 conn(0x7ff6d806c010 msgr2=0x7ff6d815ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:38.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.371+0000 7ff6dccda640 1 -- 172.21.15.154:0/722334233 shutdown_connections 2023-09-23T03:15:38.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.371+0000 7ff6dccda640 1 -- 172.21.15.154:0/722334233 wait complete. 2023-09-23T03:15:38.389 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:15:38.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.603+0000 7fb574ce5640 1 Processor -- start 2023-09-23T03:15:38.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.603+0000 7fb574ce5640 1 -- start start 2023-09-23T03:15:38.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.605+0000 7fb574ce5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb57015bb30 0x7fb5700caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:38.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.605+0000 7fb574ce5640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb5700cb3e0 0x7fb5700cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:38.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.605+0000 7fb574ce5640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb57015b6a0 con 0x7fb57015aba0 2023-09-23T03:15:38.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.605+0000 7fb574ce5640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5700cb870 con 0x7fb5700cb3e0 2023-09-23T03:15:38.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.605+0000 7fb574ce5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb57015bf70 con 0x7fb57015bb30 2023-09-23T03:15:38.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.606+0000 7fb565d9b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb57015bb30 0x7fb5700caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:38.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.606+0000 7fb565d9b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb57015bb30 0x7fb5700caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:44268/0 (socket says 172.21.15.154:44268) 2023-09-23T03:15:38.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.606+0000 7fb565d9b640 1 -- 172.21.15.154:0/1346717535 learned_addr learned my addr 172.21.15.154:0/1346717535 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:38.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.606+0000 7fb56ed9d640 1 --2- 172.21.15.154:0/1346717535 >> v2:172.21.15.190:3300/0 conn(0x7fb5700cb3e0 0x7fb5700cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:38.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.607+0000 7fb56ed9d640 1 -- 172.21.15.154:0/1346717535 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb57015bb30 msgr2=0x7fb5700caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:38.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.607+0000 7fb56ed9d640 1 --2- 172.21.15.154:0/1346717535 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb57015bb30 0x7fb5700caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:38.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.607+0000 7fb56ed9d640 1 -- 172.21.15.154:0/1346717535 >> v1:172.21.15.190:6789/0 conn(0x7fb57015aba0 legacy=0x7fb57015e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:38.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.607+0000 7fb56ed9d640 1 -- 172.21.15.154:0/1346717535 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb57015f0b0 con 0x7fb5700cb3e0 2023-09-23T03:15:38.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.607+0000 7fb565d9b640 1 --2- 172.21.15.154:0/1346717535 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb57015bb30 0x7fb5700caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:15:38.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.608+0000 7fb56ed9d640 1 --2- 172.21.15.154:0/1346717535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5700cb3e0 0x7fb5700cc5c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb5680097a0 tx=0x7fb56802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e8f3404dd056282 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:38.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.609+0000 7fb56dd9b640 1 -- 172.21.15.154:0/1346717535 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56802d040 con 0x7fb5700cb3e0 2023-09-23T03:15:38.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.609+0000 7fb56dd9b640 1 -- 172.21.15.154:0/1346717535 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb568031810 con 0x7fb5700cb3e0 2023-09-23T03:15:38.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.609+0000 7fb56dd9b640 1 -- 172.21.15.154:0/1346717535 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb568031ba0 con 0x7fb5700cb3e0 2023-09-23T03:15:38.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.610+0000 7fb574ce5640 1 -- 172.21.15.154:0/1346717535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5700cb3e0 msgr2=0x7fb5700cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:38.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.610+0000 7fb574ce5640 1 --2- 172.21.15.154:0/1346717535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5700cb3e0 0x7fb5700cc5c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb5680097a0 tx=0x7fb56802b8b0 comp rx=0 tx=0).stop 2023-09-23T03:15:38.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.610+0000 7fb574ce5640 1 -- 172.21.15.154:0/1346717535 shutdown_connections 2023-09-23T03:15:38.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.610+0000 7fb574ce5640 1 --2- 172.21.15.154:0/1346717535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5700cb3e0 0x7fb5700cc5c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:38.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.610+0000 7fb574ce5640 1 --2- 172.21.15.154:0/1346717535 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb57015bb30 0x7fb5700caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:38.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.610+0000 7fb574ce5640 1 -- 172.21.15.154:0/1346717535 >> 172.21.15.154:0/1346717535 conn(0x7fb57006c010 msgr2=0x7fb57006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:38.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.610+0000 7fb574ce5640 1 -- 172.21.15.154:0/1346717535 shutdown_connections 2023-09-23T03:15:38.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.610+0000 7fb574ce5640 1 -- 172.21.15.154:0/1346717535 wait complete. 2023-09-23T03:15:38.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.612+0000 7fb574ce5640 1 Processor -- start 2023-09-23T03:15:38.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.612+0000 7fb574ce5640 1 -- start start 2023-09-23T03:15:38.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.613+0000 7fb574ce5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5700cb3e0 0x7fb5700c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:38.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.613+0000 7fb574ce5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb57015aba0 0x7fb5700c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:38.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.613+0000 7fb574ce5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb57015bb30 0x7fb57015e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:38.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.613+0000 7fb574ce5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5700ca400 con 0x7fb57015bb30 2023-09-23T03:15:38.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.613+0000 7fb574ce5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5700c75a0 con 0x7fb5700cb3e0 2023-09-23T03:15:38.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.613+0000 7fb574ce5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5700c7880 con 0x7fb57015aba0 2023-09-23T03:15:38.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.614+0000 7fb56ed9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb57015bb30 0x7fb57015e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:38.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.614+0000 7fb56ed9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb57015bb30 0x7fb57015e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47886/0 (socket says 172.21.15.154:47886) 2023-09-23T03:15:38.615 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.614+0000 7fb56ed9d640 1 -- 172.21.15.154:0/3066961622 learned_addr learned my addr 172.21.15.154:0/3066961622 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:38.615 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.614+0000 7fb56e59c640 1 --2- 172.21.15.154:0/3066961622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5700cb3e0 0x7fb5700c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:38.615 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.614+0000 7fb565d9b640 1 --2- 172.21.15.154:0/3066961622 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb57015aba0 0x7fb5700c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:38.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.614+0000 7fb56ed9d640 1 -- 172.21.15.154:0/3066961622 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb57015aba0 msgr2=0x7fb5700c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:38.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.614+0000 7fb56ed9d640 1 --2- 172.21.15.154:0/3066961622 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb57015aba0 0x7fb5700c9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:38.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.614+0000 7fb56ed9d640 1 -- 172.21.15.154:0/3066961622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5700cb3e0 msgr2=0x7fb5700c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:38.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.614+0000 7fb56ed9d640 1 --2- 172.21.15.154:0/3066961622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5700cb3e0 0x7fb5700c6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:38.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.614+0000 7fb56ed9d640 1 -- 172.21.15.154:0/3066961622 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5580094d0 con 0x7fb57015bb30 2023-09-23T03:15:38.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.614+0000 7fb56e59c640 1 --2- 172.21.15.154:0/3066961622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5700cb3e0 0x7fb5700c6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:15:38.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.615+0000 7fb56ed9d640 1 --2- 172.21.15.154:0/3066961622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb57015bb30 0x7fb57015e5d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb56802b8f0 tx=0x7fb5680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:38.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.615+0000 7fb5677fe640 1 -- 172.21.15.154:0/3066961622 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb568006e10 con 0x7fb57015bb30 2023-09-23T03:15:38.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.615+0000 7fb5677fe640 1 -- 172.21.15.154:0/3066961622 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb56803ace0 con 0x7fb57015bb30 2023-09-23T03:15:38.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.615+0000 7fb574ce5640 1 -- 172.21.15.154:0/3066961622 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb568009400 con 0x7fb57015bb30 2023-09-23T03:15:38.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.615+0000 7fb5677fe640 1 -- 172.21.15.154:0/3066961622 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb568044610 con 0x7fb57015bb30 2023-09-23T03:15:38.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.615+0000 7fb574ce5640 1 -- 172.21.15.154:0/3066961622 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5700c7e40 con 0x7fb57015bb30 2023-09-23T03:15:38.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.616+0000 7fb574ce5640 1 -- 172.21.15.154:0/3066961622 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb570071710 con 0x7fb57015bb30 2023-09-23T03:15:38.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.617+0000 7fb5677fe640 1 -- 172.21.15.154:0/3066961622 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fb56804d450 con 0x7fb57015bb30 2023-09-23T03:15:38.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.618+0000 7fb5677fe640 1 --2- 172.21.15.154:0/3066961622 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb55006af30 0x7fb55006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:38.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.618+0000 7fb5677fe640 1 -- 172.21.15.154:0/3066961622 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb5680abd10 con 0x7fb57015bb30 2023-09-23T03:15:38.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.618+0000 7fb56e59c640 1 --2- 172.21.15.154:0/3066961622 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb55006af30 0x7fb55006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:38.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.619+0000 7fb56e59c640 1 --2- 172.21.15.154:0/3066961622 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb55006af30 0x7fb55006d3f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb5600012e0 tx=0x7fb560009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:38.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.622+0000 7fb5677fe640 1 -- 172.21.15.154:0/3066961622 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fb5680b1020 con 0x7fb57015bb30 2023-09-23T03:15:38.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.813+0000 7fb574ce5640 1 -- 172.21.15.154:0/3066961622 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb57015b300 con 0x7fb57015bb30 2023-09-23T03:15:38.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.814+0000 7fb5677fe640 1 -- 172.21.15.154:0/3066961622 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb5680769c0 con 0x7fb57015bb30 2023-09-23T03:15:38.814 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:15:38.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.816+0000 7fb574ce5640 1 -- 172.21.15.154:0/3066961622 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb55006af30 msgr2=0x7fb55006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:38.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.816+0000 7fb574ce5640 1 --2- 172.21.15.154:0/3066961622 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb55006af30 0x7fb55006d3f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb5600012e0 tx=0x7fb560009040 comp rx=0 tx=0).stop 2023-09-23T03:15:38.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.816+0000 7fb574ce5640 1 -- 172.21.15.154:0/3066961622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb57015bb30 msgr2=0x7fb57015e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:38.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.816+0000 7fb574ce5640 1 --2- 172.21.15.154:0/3066961622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb57015bb30 0x7fb57015e5d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb56802b8f0 tx=0x7fb5680046c0 comp rx=0 tx=0).stop 2023-09-23T03:15:38.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.816+0000 7fb574ce5640 1 -- 172.21.15.154:0/3066961622 shutdown_connections 2023-09-23T03:15:38.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.816+0000 7fb574ce5640 1 --2- 172.21.15.154:0/3066961622 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb55006af30 0x7fb55006d3f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:38.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.816+0000 7fb574ce5640 1 --2- 172.21.15.154:0/3066961622 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb57015bb30 0x7fb57015e5d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:38.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.816+0000 7fb574ce5640 1 --2- 172.21.15.154:0/3066961622 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb57015aba0 0x7fb5700c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:38.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.816+0000 7fb574ce5640 1 --2- 172.21.15.154:0/3066961622 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb5700cb3e0 0x7fb5700c6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:38.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.816+0000 7fb574ce5640 1 -- 172.21.15.154:0/3066961622 >> 172.21.15.154:0/3066961622 conn(0x7fb57006c010 msgr2=0x7fb570070340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:38.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.817+0000 7fb574ce5640 1 -- 172.21.15.154:0/3066961622 shutdown_connections 2023-09-23T03:15:38.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:38.817+0000 7fb574ce5640 1 -- 172.21.15.154:0/3066961622 wait complete. 2023-09-23T03:15:38.836 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:15:39.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.044+0000 7f2892a90640 1 Processor -- start 2023-09-23T03:15:39.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.044+0000 7f2892a90640 1 -- start start 2023-09-23T03:15:39.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.045+0000 7f2892a90640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f288c0c8d70 0x7f288c15c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:39.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.046+0000 7f2892a90640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f288c071290 0x7f288c0ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:39.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.046+0000 7f2892a90640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f288c0c9d20 con 0x7f288c0c9150 2023-09-23T03:15:39.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.046+0000 7f2892a90640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f288c071850 con 0x7f288c0c8d70 2023-09-23T03:15:39.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.046+0000 7f2892a90640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f288c071b30 con 0x7f288c071290 2023-09-23T03:15:39.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.046+0000 7f288bfff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f288c0c9150 0x7f288c15cc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:37932/0 (socket says 172.21.15.154:37932) 2023-09-23T03:15:39.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.046+0000 7f288bfff640 1 -- 172.21.15.154:0/3467505853 learned_addr learned my addr 172.21.15.154:0/3467505853 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:39.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.046+0000 7f289082c640 1 --2- 172.21.15.154:0/3467505853 >> v2:172.21.15.190:3300/0 conn(0x7f288c0c8d70 0x7f288c15c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:39.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.046+0000 7f289102d640 1 --2- 172.21.15.154:0/3467505853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f288c071290 0x7f288c0ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:39.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.046+0000 7f288b7fe640 1 -- 172.21.15.154:0/3467505853 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2523789108 0 0) 0x7f288c0c9d20 con 0x7f288c0c9150 2023-09-23T03:15:39.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.047+0000 7f288b7fe640 1 -- 172.21.15.154:0/3467505853 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2874005730 con 0x7f288c0c9150 2023-09-23T03:15:39.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.047+0000 7f288b7fe640 1 -- 172.21.15.154:0/3467505853 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3069006232 0 0) 0x7f2874005730 con 0x7f288c0c9150 2023-09-23T03:15:39.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.047+0000 7f289082c640 1 -- 172.21.15.154:0/3467505853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f288c071290 msgr2=0x7f288c0ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:39.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.047+0000 7f289082c640 1 --2- 172.21.15.154:0/3467505853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f288c071290 0x7f288c0ca1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:39.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.047+0000 7f289082c640 1 -- 172.21.15.154:0/3467505853 >> v1:172.21.15.154:6789/0 conn(0x7f288c0c9150 legacy=0x7f288c15cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:39.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.047+0000 7f289082c640 1 -- 172.21.15.154:0/3467505853 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f288c071e10 con 0x7f288c0c8d70 2023-09-23T03:15:39.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.048+0000 7f289082c640 1 --2- 172.21.15.154:0/3467505853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f288c0c8d70 0x7f288c15c6d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f28800097a0 tx=0x7f288002b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f014f6b1a04fec35 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:39.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.048+0000 7f288b7fe640 1 -- 172.21.15.154:0/3467505853 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287c001690 con 0x7f288c0c8d70 2023-09-23T03:15:39.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.048+0000 7f288b7fe640 1 -- 172.21.15.154:0/3467505853 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f288002e050 con 0x7f288c0c8d70 2023-09-23T03:15:39.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.048+0000 7f288b7fe640 1 -- 172.21.15.154:0/3467505853 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2880030730 con 0x7f288c0c8d70 2023-09-23T03:15:39.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.049+0000 7f2892a90640 1 -- 172.21.15.154:0/3467505853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f288c0c8d70 msgr2=0x7f288c15c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:39.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.049+0000 7f2892a90640 1 --2- 172.21.15.154:0/3467505853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f288c0c8d70 0x7f288c15c6d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f28800097a0 tx=0x7f288002b790 comp rx=0 tx=0).stop 2023-09-23T03:15:39.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.049+0000 7f2892a90640 1 -- 172.21.15.154:0/3467505853 shutdown_connections 2023-09-23T03:15:39.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.049+0000 7f2892a90640 1 --2- 172.21.15.154:0/3467505853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f288c071290 0x7f288c0ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:39.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.049+0000 7f2892a90640 1 --2- 172.21.15.154:0/3467505853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f288c0c8d70 0x7f288c15c6d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:39.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.049+0000 7f2892a90640 1 -- 172.21.15.154:0/3467505853 >> 172.21.15.154:0/3467505853 conn(0x7f288c06c010 msgr2=0x7f288c06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:39.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.049+0000 7f2892a90640 1 -- 172.21.15.154:0/3467505853 shutdown_connections 2023-09-23T03:15:39.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.049+0000 7f2892a90640 1 -- 172.21.15.154:0/3467505853 wait complete. 2023-09-23T03:15:39.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.052+0000 7f2892a90640 1 Processor -- start 2023-09-23T03:15:39.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.052+0000 7f2892a90640 1 -- start start 2023-09-23T03:15:39.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.053+0000 7f2892a90640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f288c071290 0x7f288c153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:39.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.053+0000 7f2892a90640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f288c0c8d70 0x7f288c0adf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:39.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.053+0000 7f2892a90640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f288c0c9150 0x7f288c0b00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:39.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.053+0000 7f2892a90640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288c0af4b0 con 0x7f288c0c8d70 2023-09-23T03:15:39.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.053+0000 7f2892a90640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288c0ae930 con 0x7f288c071290 2023-09-23T03:15:39.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.053+0000 7f2892a90640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f288c0aec10 con 0x7f288c0c9150 2023-09-23T03:15:39.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.053+0000 7f288bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f288c0c8d70 0x7f288c0adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:39.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.053+0000 7f288bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f288c0c8d70 0x7f288c0adf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47892/0 (socket says 172.21.15.154:47892) 2023-09-23T03:15:39.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.053+0000 7f288bfff640 1 -- 172.21.15.154:0/3775314655 learned_addr learned my addr 172.21.15.154:0/3775314655 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:39.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.054+0000 7f289082c640 1 --2- 172.21.15.154:0/3775314655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f288c071290 0x7f288c153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:39.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.054+0000 7f289102d640 1 --2- 172.21.15.154:0/3775314655 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f288c0c9150 0x7f288c0b00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:39.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.054+0000 7f288bfff640 1 -- 172.21.15.154:0/3775314655 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f288c0c9150 msgr2=0x7f288c0b00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:39.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.054+0000 7f288bfff640 1 --2- 172.21.15.154:0/3775314655 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f288c0c9150 0x7f288c0b00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:39.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.054+0000 7f288bfff640 1 -- 172.21.15.154:0/3775314655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f288c071290 msgr2=0x7f288c153a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:39.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.054+0000 7f288bfff640 1 --2- 172.21.15.154:0/3775314655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f288c071290 0x7f288c153a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:39.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.054+0000 7f288bfff640 1 -- 172.21.15.154:0/3775314655 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2880009400 con 0x7f288c0c8d70 2023-09-23T03:15:39.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.054+0000 7f289102d640 1 --2- 172.21.15.154:0/3775314655 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f288c0c9150 0x7f288c0b00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:39.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.054+0000 7f288bfff640 1 --2- 172.21.15.154:0/3775314655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f288c0c8d70 0x7f288c0adf70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f287c00c780 tx=0x7f287c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:39.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.054+0000 7f28897fa640 1 -- 172.21.15.154:0/3775314655 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287c0072d0 con 0x7f288c0c8d70 2023-09-23T03:15:39.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.054+0000 7f28897fa640 1 -- 172.21.15.154:0/3775314655 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f287c007450 con 0x7f288c0c8d70 2023-09-23T03:15:39.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.055+0000 7f2892a90640 1 -- 172.21.15.154:0/3775314655 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f288c0aeef0 con 0x7f288c0c8d70 2023-09-23T03:15:39.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.055+0000 7f28897fa640 1 -- 172.21.15.154:0/3775314655 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287c0047e0 con 0x7f288c0c8d70 2023-09-23T03:15:39.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.055+0000 7f2892a90640 1 -- 172.21.15.154:0/3775314655 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f288c153f50 con 0x7f288c0c8d70 2023-09-23T03:15:39.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.056+0000 7f2892a90640 1 -- 172.21.15.154:0/3775314655 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f288c0c6660 con 0x7f288c0c8d70 2023-09-23T03:15:39.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.057+0000 7f28897fa640 1 -- 172.21.15.154:0/3775314655 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f287c004960 con 0x7f288c0c8d70 2023-09-23T03:15:39.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.058+0000 7f28897fa640 1 --2- 172.21.15.154:0/3775314655 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f287806af30 0x7f287806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:39.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.058+0000 7f28897fa640 1 -- 172.21.15.154:0/3775314655 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f287c089080 con 0x7f288c0c8d70 2023-09-23T03:15:39.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.058+0000 7f289082c640 1 --2- 172.21.15.154:0/3775314655 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f287806af30 0x7f287806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:39.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.059+0000 7f289082c640 1 --2- 172.21.15.154:0/3775314655 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f287806af30 0x7f287806d3f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2880034000 tx=0x7f288002f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:39.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.062+0000 7f28897fa640 1 -- 172.21.15.154:0/3775314655 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f288c0c6660 con 0x7f288c0c8d70 2023-09-23T03:15:39.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.255+0000 7f2892a90640 1 -- 172.21.15.154:0/3775314655 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f288c0c6660 con 0x7f288c0c8d70 2023-09-23T03:15:39.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.259+0000 7f28897fa640 1 -- 172.21.15.154:0/3775314655 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f288c0c6660 con 0x7f288c0c8d70 2023-09-23T03:15:39.259 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:15:39.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.261+0000 7f2892a90640 1 -- 172.21.15.154:0/3775314655 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f287806af30 msgr2=0x7f287806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:39.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.262+0000 7f2892a90640 1 --2- 172.21.15.154:0/3775314655 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f287806af30 0x7f287806d3f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2880034000 tx=0x7f288002f000 comp rx=0 tx=0).stop 2023-09-23T03:15:39.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.262+0000 7f2892a90640 1 -- 172.21.15.154:0/3775314655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f288c0c8d70 msgr2=0x7f288c0adf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:39.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.262+0000 7f2892a90640 1 --2- 172.21.15.154:0/3775314655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f288c0c8d70 0x7f288c0adf70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f287c00c780 tx=0x7f287c00cc40 comp rx=0 tx=0).stop 2023-09-23T03:15:39.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.262+0000 7f2892a90640 1 -- 172.21.15.154:0/3775314655 shutdown_connections 2023-09-23T03:15:39.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.262+0000 7f2892a90640 1 --2- 172.21.15.154:0/3775314655 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f287806af30 0x7f287806d3f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:39.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.262+0000 7f2892a90640 1 --2- 172.21.15.154:0/3775314655 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f288c0c9150 0x7f288c0b00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:39.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.262+0000 7f2892a90640 1 --2- 172.21.15.154:0/3775314655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f288c0c8d70 0x7f288c0adf70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:39.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.262+0000 7f2892a90640 1 --2- 172.21.15.154:0/3775314655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f288c071290 0x7f288c153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:39.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.262+0000 7f2892a90640 1 -- 172.21.15.154:0/3775314655 >> 172.21.15.154:0/3775314655 conn(0x7f288c06c010 msgr2=0x7f288c158830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:39.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.262+0000 7f2892a90640 1 -- 172.21.15.154:0/3775314655 shutdown_connections 2023-09-23T03:15:39.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.262+0000 7f2892a90640 1 -- 172.21.15.154:0/3775314655 wait complete. 2023-09-23T03:15:39.281 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-23T03:15:39.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.487+0000 7f74a76a9640 1 Processor -- start 2023-09-23T03:15:39.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.487+0000 7f74a76a9640 1 -- start start 2023-09-23T03:15:39.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.489+0000 7f74a76a9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f74a015aba0 0x7f74a015bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:39.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.489+0000 7f74a76a9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74a015e5f0 0x7f74a015ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:39.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.489+0000 7f74a76a9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f74a00caea0 0x7f74a00cb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:39.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.489+0000 7f74a76a9640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74a00cb870 con 0x7f74a00caea0 2023-09-23T03:15:39.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.489+0000 7f74a76a9640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74a015b6a0 con 0x7f74a015aba0 2023-09-23T03:15:39.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.489+0000 7f74a76a9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a015f0b0 con 0x7f74a015e5f0 2023-09-23T03:15:39.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.489+0000 7f74a5c46640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f74a00caea0 0x7f74a00cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:39.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.489+0000 7f74a5c46640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f74a00caea0 0x7f74a00cb330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47896/0 (socket says 172.21.15.154:47896) 2023-09-23T03:15:39.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.489+0000 7f74a5c46640 1 -- 172.21.15.154:0/2884098579 learned_addr learned my addr 172.21.15.154:0/2884098579 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:39.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.490+0000 7f74a4c44640 1 --2- 172.21.15.154:0/2884098579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74a015e5f0 0x7f74a015ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:39.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.490+0000 7f74a5445640 1 --2- 172.21.15.154:0/2884098579 >> v2:172.21.15.190:3300/0 conn(0x7f74a015aba0 0x7f74a015bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:39.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.490+0000 7f74a5c46640 1 -- 172.21.15.154:0/2884098579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74a015e5f0 msgr2=0x7f74a015ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:39.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.490+0000 7f74a5c46640 1 --2- 172.21.15.154:0/2884098579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74a015e5f0 0x7f74a015ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:39.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.490+0000 7f74a5c46640 1 -- 172.21.15.154:0/2884098579 >> v2:172.21.15.190:3300/0 conn(0x7f74a015aba0 msgr2=0x7f74a015bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:39.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.490+0000 7f74a5c46640 1 --2- 172.21.15.154:0/2884098579 >> v2:172.21.15.190:3300/0 conn(0x7f74a015aba0 0x7f74a015bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:39.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.490+0000 7f74a5c46640 1 -- 172.21.15.154:0/2884098579 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a015f390 con 0x7f74a00caea0 2023-09-23T03:15:39.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.491+0000 7f74a5c46640 1 --2- 172.21.15.154:0/2884098579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74a00caea0 0x7f74a00cb330 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f74a006e1f0 tx=0x7f749802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc7547a17d922e47 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:39.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.491+0000 7f7493fff640 1 -- 172.21.15.154:0/2884098579 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749802ca80 con 0x7f74a00caea0 2023-09-23T03:15:39.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.491+0000 7f7493fff640 1 -- 172.21.15.154:0/2884098579 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749802cbe0 con 0x7f74a00caea0 2023-09-23T03:15:39.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.491+0000 7f7493fff640 1 -- 172.21.15.154:0/2884098579 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74980343f0 con 0x7f74a00caea0 2023-09-23T03:15:39.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.492+0000 7f74a76a9640 1 -- 172.21.15.154:0/2884098579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74a00caea0 msgr2=0x7f74a00cb330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:39.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.492+0000 7f74a76a9640 1 --2- 172.21.15.154:0/2884098579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74a00caea0 0x7f74a00cb330 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f74a006e1f0 tx=0x7f749802bee0 comp rx=0 tx=0).stop 2023-09-23T03:15:39.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.492+0000 7f74a76a9640 1 -- 172.21.15.154:0/2884098579 shutdown_connections 2023-09-23T03:15:39.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.492+0000 7f74a76a9640 1 --2- 172.21.15.154:0/2884098579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74a00caea0 0x7f74a00cb330 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:39.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.492+0000 7f74a76a9640 1 --2- 172.21.15.154:0/2884098579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74a015e5f0 0x7f74a015ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:39.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.492+0000 7f74a76a9640 1 --2- 172.21.15.154:0/2884098579 >> v2:172.21.15.190:3300/0 conn(0x7f74a015aba0 0x7f74a015bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:39.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.492+0000 7f74a76a9640 1 -- 172.21.15.154:0/2884098579 >> 172.21.15.154:0/2884098579 conn(0x7f74a006c010 msgr2=0x7f74a006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:39.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.492+0000 7f74a76a9640 1 -- 172.21.15.154:0/2884098579 shutdown_connections 2023-09-23T03:15:39.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.492+0000 7f74a76a9640 1 -- 172.21.15.154:0/2884098579 wait complete. 2023-09-23T03:15:39.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.494+0000 7f74a76a9640 1 Processor -- start 2023-09-23T03:15:39.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.494+0000 7f74a76a9640 1 -- start start 2023-09-23T03:15:39.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.495+0000 7f74a76a9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74a00caea0 0x7f74a00aca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:39.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.495+0000 7f74a76a9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74a015aba0 0x7f74a01f1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:39.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.495+0000 7f74a76a9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74a015e5f0 0x7f74a00c3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:39.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.495+0000 7f74a76a9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a00ad290 con 0x7f74a015aba0 2023-09-23T03:15:39.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.495+0000 7f74a76a9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a00ac390 con 0x7f74a00caea0 2023-09-23T03:15:39.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.495+0000 7f74a76a9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a00ac820 con 0x7f74a015e5f0 2023-09-23T03:15:39.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.495+0000 7f74a4c44640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74a015aba0 0x7f74a01f1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:39.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.495+0000 7f74a4c44640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74a015aba0 0x7f74a01f1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47912/0 (socket says 172.21.15.154:47912) 2023-09-23T03:15:39.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.495+0000 7f74a4c44640 1 -- 172.21.15.154:0/4161495468 learned_addr learned my addr 172.21.15.154:0/4161495468 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:39.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.496+0000 7f74a5c46640 1 --2- 172.21.15.154:0/4161495468 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74a015e5f0 0x7f74a00c3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:39.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.496+0000 7f74a5445640 1 --2- 172.21.15.154:0/4161495468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74a00caea0 0x7f74a00aca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:39.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.496+0000 7f74a4c44640 1 -- 172.21.15.154:0/4161495468 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74a015e5f0 msgr2=0x7f74a00c3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:39.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.496+0000 7f74a4c44640 1 --2- 172.21.15.154:0/4161495468 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74a015e5f0 0x7f74a00c3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:39.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.496+0000 7f74a4c44640 1 -- 172.21.15.154:0/4161495468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74a00caea0 msgr2=0x7f74a00aca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:39.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.496+0000 7f74a4c44640 1 --2- 172.21.15.154:0/4161495468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74a00caea0 0x7f74a00aca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:39.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.496+0000 7f74a4c44640 1 -- 172.21.15.154:0/4161495468 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7498009b30 con 0x7f74a015aba0 2023-09-23T03:15:39.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.496+0000 7f74a5c46640 1 --2- 172.21.15.154:0/4161495468 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74a015e5f0 0x7f74a00c3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:39.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.496+0000 7f74a4c44640 1 --2- 172.21.15.154:0/4161495468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74a015aba0 0x7f74a01f1b30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f749400d830 tx=0x7f749400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:39.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.496+0000 7f7491ffb640 1 -- 172.21.15.154:0/4161495468 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749400b6c0 con 0x7f74a015aba0 2023-09-23T03:15:39.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.496+0000 7f7491ffb640 1 -- 172.21.15.154:0/4161495468 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749400b820 con 0x7f74a015aba0 2023-09-23T03:15:39.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.496+0000 7f7491ffb640 1 -- 172.21.15.154:0/4161495468 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749400bbb0 con 0x7f74a015aba0 2023-09-23T03:15:39.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.496+0000 7f74a76a9640 1 -- 172.21.15.154:0/4161495468 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74a00aba60 con 0x7f74a015aba0 2023-09-23T03:15:39.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.497+0000 7f74a76a9640 1 -- 172.21.15.154:0/4161495468 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74a00ac020 con 0x7f74a015aba0 2023-09-23T03:15:39.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.498+0000 7f74a76a9640 1 -- 172.21.15.154:0/4161495468 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74a00ac520 con 0x7f74a015aba0 2023-09-23T03:15:39.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.498+0000 7f7491ffb640 1 -- 172.21.15.154:0/4161495468 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f749401e460 con 0x7f74a015aba0 2023-09-23T03:15:39.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.499+0000 7f7491ffb640 1 --2- 172.21.15.154:0/4161495468 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f748406af30 0x7f748406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:39.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.499+0000 7f7491ffb640 1 -- 172.21.15.154:0/4161495468 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f749408a1f0 con 0x7f74a015aba0 2023-09-23T03:15:39.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.500+0000 7f74a5445640 1 --2- 172.21.15.154:0/4161495468 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f748406af30 0x7f748406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:39.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.500+0000 7f74a5445640 1 --2- 172.21.15.154:0/4161495468 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f748406af30 0x7f748406d3f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f749c006fd0 tx=0x7f749c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:39.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.504+0000 7f7491ffb640 1 -- 172.21.15.154:0/4161495468 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f7494054f30 con 0x7f74a015aba0 2023-09-23T03:15:39.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.678+0000 7f74a76a9640 1 -- 172.21.15.154:0/4161495468 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f74a004e920 con 0x7f74a015aba0 2023-09-23T03:15:39.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.679+0000 7f7491ffb640 1 -- 172.21.15.154:0/4161495468 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7494058550 con 0x7f74a015aba0 2023-09-23T03:15:39.679 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:15:39.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.682+0000 7f74a76a9640 1 -- 172.21.15.154:0/4161495468 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f748406af30 msgr2=0x7f748406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:39.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.682+0000 7f74a76a9640 1 --2- 172.21.15.154:0/4161495468 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f748406af30 0x7f748406d3f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f749c006fd0 tx=0x7f749c008040 comp rx=0 tx=0).stop 2023-09-23T03:15:39.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.682+0000 7f74a76a9640 1 -- 172.21.15.154:0/4161495468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74a015aba0 msgr2=0x7f74a01f1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:39.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.682+0000 7f74a76a9640 1 --2- 172.21.15.154:0/4161495468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74a015aba0 0x7f74a01f1b30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f749400d830 tx=0x7f749400dcf0 comp rx=0 tx=0).stop 2023-09-23T03:15:39.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.682+0000 7f74a76a9640 1 -- 172.21.15.154:0/4161495468 shutdown_connections 2023-09-23T03:15:39.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.682+0000 7f74a76a9640 1 --2- 172.21.15.154:0/4161495468 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f748406af30 0x7f748406d3f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:39.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.682+0000 7f74a76a9640 1 --2- 172.21.15.154:0/4161495468 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74a015e5f0 0x7f74a00c3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:39.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.682+0000 7f74a76a9640 1 --2- 172.21.15.154:0/4161495468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74a015aba0 0x7f74a01f1b30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:39.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.682+0000 7f74a76a9640 1 --2- 172.21.15.154:0/4161495468 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74a00caea0 0x7f74a00aca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:39.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.682+0000 7f74a76a9640 1 -- 172.21.15.154:0/4161495468 >> 172.21.15.154:0/4161495468 conn(0x7f74a006c010 msgr2=0x7f74a015d870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:39.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.682+0000 7f74a76a9640 1 -- 172.21.15.154:0/4161495468 shutdown_connections 2023-09-23T03:15:39.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.682+0000 7f74a76a9640 1 -- 172.21.15.154:0/4161495468 wait complete. 2023-09-23T03:15:39.700 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-23T03:15:39.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.912+0000 7f6ef681a640 1 Processor -- start 2023-09-23T03:15:39.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.913+0000 7f6ef681a640 1 -- start start 2023-09-23T03:15:39.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.914+0000 7f6ef681a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6ef00acce0 0x7f6ef015ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:39.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.914+0000 7f6ef681a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ef00cf040 0x7f6ef00ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:39.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.914+0000 7f6ef681a640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ef015ac30 con 0x7f6ef00b1b20 2023-09-23T03:15:39.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.914+0000 7f6ef681a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ef00b2e90 con 0x7f6ef00acce0 2023-09-23T03:15:39.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.914+0000 7f6ef681a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef00b1f70 con 0x7f6ef00cf040 2023-09-23T03:15:39.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.915+0000 7f6eeffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6ef00acce0 0x7f6ef015ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:39.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.915+0000 7f6ee7fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f6ef00b1b20 0x7f6ef00ce930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:37944/0 (socket says 172.21.15.154:37944) 2023-09-23T03:15:39.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.915+0000 7f6ee7fff640 1 -- 172.21.15.154:0/3166288462 learned_addr learned my addr 172.21.15.154:0/3166288462 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:39.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.915+0000 7f6ef4db7640 1 --2- 172.21.15.154:0/3166288462 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ef00cf040 0x7f6ef00ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:39.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.915+0000 7f6eef7fe640 1 -- 172.21.15.154:0/3166288462 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 925311068 0 0) 0x7f6ed8001110 con 0x7f6ef00b1b20 2023-09-23T03:15:39.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.915+0000 7f6eef7fe640 1 -- 172.21.15.154:0/3166288462 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6edc003520 con 0x7f6ef00b1b20 2023-09-23T03:15:39.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.915+0000 7f6eeffff640 1 -- 172.21.15.154:0/3166288462 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ef00cf040 msgr2=0x7f6ef00ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:39.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.915+0000 7f6eeffff640 1 --2- 172.21.15.154:0/3166288462 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ef00cf040 0x7f6ef00ab140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:39.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.915+0000 7f6eeffff640 1 -- 172.21.15.154:0/3166288462 >> v1:172.21.15.154:6789/0 conn(0x7f6ef00b1b20 legacy=0x7f6ef00ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:39.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.915+0000 7f6eeffff640 1 -- 172.21.15.154:0/3166288462 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef015f2a0 con 0x7f6ef00acce0 2023-09-23T03:15:39.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.915+0000 7f6ef4db7640 1 --2- 172.21.15.154:0/3166288462 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ef00cf040 0x7f6ef00ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:15:39.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.916+0000 7f6eeffff640 1 --2- 172.21.15.154:0/3166288462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ef00acce0 0x7f6ef015ed60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0009790 tx=0x7f6ee002b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a930ad08d0a018b1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:39.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.917+0000 7f6eef7fe640 1 -- 172.21.15.154:0/3166288462 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0031770 con 0x7f6ef00acce0 2023-09-23T03:15:39.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.917+0000 7f6eef7fe640 1 -- 172.21.15.154:0/3166288462 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee00318d0 con 0x7f6ef00acce0 2023-09-23T03:15:39.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.917+0000 7f6eef7fe640 1 -- 172.21.15.154:0/3166288462 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0031c60 con 0x7f6ef00acce0 2023-09-23T03:15:39.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.917+0000 7f6ef681a640 1 -- 172.21.15.154:0/3166288462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ef00acce0 msgr2=0x7f6ef015ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:39.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.917+0000 7f6ef681a640 1 --2- 172.21.15.154:0/3166288462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ef00acce0 0x7f6ef015ed60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0009790 tx=0x7f6ee002b810 comp rx=0 tx=0).stop 2023-09-23T03:15:39.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.917+0000 7f6ef681a640 1 -- 172.21.15.154:0/3166288462 shutdown_connections 2023-09-23T03:15:39.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.917+0000 7f6ef681a640 1 --2- 172.21.15.154:0/3166288462 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ef00cf040 0x7f6ef00ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:39.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.917+0000 7f6ef681a640 1 --2- 172.21.15.154:0/3166288462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ef00acce0 0x7f6ef015ed60 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:39.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.917+0000 7f6ef681a640 1 -- 172.21.15.154:0/3166288462 >> 172.21.15.154:0/3166288462 conn(0x7f6ef006a0f0 msgr2=0x7f6ef006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:39.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.918+0000 7f6ef681a640 1 -- 172.21.15.154:0/3166288462 shutdown_connections 2023-09-23T03:15:39.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.918+0000 7f6ef681a640 1 -- 172.21.15.154:0/3166288462 wait complete. 2023-09-23T03:15:39.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.919+0000 7f6ef681a640 1 Processor -- start 2023-09-23T03:15:39.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.920+0000 7f6ef681a640 1 -- start start 2023-09-23T03:15:39.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.920+0000 7f6ef681a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ef00acce0 0x7f6ef0159510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:39.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.920+0000 7f6ef681a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ef00b1b20 0x7f6ef00cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:39.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.921+0000 7f6ef681a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ef00cf040 0x7f6ef00cd630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:39.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.921+0000 7f6ef681a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef01f3390 con 0x7f6ef00acce0 2023-09-23T03:15:39.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.921+0000 7f6ef681a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef01f30b0 con 0x7f6ef00b1b20 2023-09-23T03:15:39.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.921+0000 7f6ef681a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef01f3630 con 0x7f6ef00cf040 2023-09-23T03:15:39.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.921+0000 7f6eeffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ef00acce0 0x7f6ef0159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:39.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.921+0000 7f6eeffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ef00acce0 0x7f6ef0159510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47942/0 (socket says 172.21.15.154:47942) 2023-09-23T03:15:39.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.921+0000 7f6eeffff640 1 -- 172.21.15.154:0/3155737826 learned_addr learned my addr 172.21.15.154:0/3155737826 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:39.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.921+0000 7f6ef4db7640 1 --2- 172.21.15.154:0/3155737826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ef00cf040 0x7f6ef00cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:39.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.921+0000 7f6eeffff640 1 -- 172.21.15.154:0/3155737826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ef00cf040 msgr2=0x7f6ef00cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:39.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.921+0000 7f6eeffff640 1 --2- 172.21.15.154:0/3155737826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ef00cf040 0x7f6ef00cd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:39.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.921+0000 7f6eeffff640 1 -- 172.21.15.154:0/3155737826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ef00b1b20 msgr2=0x7f6ef00cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:39.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.921+0000 7f6eeffff640 1 --2- 172.21.15.154:0/3155737826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ef00b1b20 0x7f6ef00cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:39.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.921+0000 7f6eeffff640 1 -- 172.21.15.154:0/3155737826 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee00093a0 con 0x7f6ef00acce0 2023-09-23T03:15:39.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.922+0000 7f6eeffff640 1 --2- 172.21.15.154:0/3155737826 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ef00acce0 0x7f6ef0159510 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0034040 tx=0x7f6ee00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:39.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.922+0000 7f6eed7fa640 1 -- 172.21.15.154:0/3155737826 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0032810 con 0x7f6ef00acce0 2023-09-23T03:15:39.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.922+0000 7f6eed7fa640 1 -- 172.21.15.154:0/3155737826 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee0032990 con 0x7f6ef00acce0 2023-09-23T03:15:39.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.922+0000 7f6eed7fa640 1 -- 172.21.15.154:0/3155737826 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0032d40 con 0x7f6ef00acce0 2023-09-23T03:15:39.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.922+0000 7f6ef681a640 1 -- 172.21.15.154:0/3155737826 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef01f3910 con 0x7f6ef00acce0 2023-09-23T03:15:39.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.922+0000 7f6ef681a640 1 -- 172.21.15.154:0/3155737826 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef0159a50 con 0x7f6ef00acce0 2023-09-23T03:15:39.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.923+0000 7f6ef681a640 1 -- 172.21.15.154:0/3155737826 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ef0159be0 con 0x7f6ef00acce0 2023-09-23T03:15:39.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.924+0000 7f6eed7fa640 1 -- 172.21.15.154:0/3155737826 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f6ee004e440 con 0x7f6ef00acce0 2023-09-23T03:15:39.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.925+0000 7f6eed7fa640 1 --2- 172.21.15.154:0/3155737826 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6ed006af30 0x7f6ed006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:39.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.925+0000 7f6eed7fa640 1 -- 172.21.15.154:0/3155737826 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6ee00abab0 con 0x7f6ef00acce0 2023-09-23T03:15:39.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.925+0000 7f6ee7fff640 1 --2- 172.21.15.154:0/3155737826 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6ed006af30 0x7f6ed006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:39.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.926+0000 7f6ee7fff640 1 --2- 172.21.15.154:0/3155737826 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6ed006af30 0x7f6ed006d3f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8005880 tx=0x7f6ed8001150 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:39.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:39.930+0000 7f6eed7fa640 1 -- 172.21.15.154:0/3155737826 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f6ee00767e0 con 0x7f6ef00acce0 2023-09-23T03:15:40.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:40.130+0000 7f6ef681a640 1 -- 172.21.15.154:0/3155737826 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6ef0073eb0 con 0x7f6ef00acce0 2023-09-23T03:15:40.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:40.131+0000 7f6eed7fa640 1 -- 172.21.15.154:0/3155737826 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6ee0079e00 con 0x7f6ef00acce0 2023-09-23T03:15:40.133 INFO:teuthology.orchestra.run.smithi154.stderr:listed 0 entries 2023-09-23T03:15:40.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:40.133+0000 7f6ef681a640 1 -- 172.21.15.154:0/3155737826 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6ed006af30 msgr2=0x7f6ed006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:40.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:40.133+0000 7f6ef681a640 1 --2- 172.21.15.154:0/3155737826 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6ed006af30 0x7f6ed006d3f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8005880 tx=0x7f6ed8001150 comp rx=0 tx=0).stop 2023-09-23T03:15:40.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:40.133+0000 7f6ef681a640 1 -- 172.21.15.154:0/3155737826 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ef00acce0 msgr2=0x7f6ef0159510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:40.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:40.134+0000 7f6ef681a640 1 --2- 172.21.15.154:0/3155737826 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ef00acce0 0x7f6ef0159510 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0034040 tx=0x7f6ee00046c0 comp rx=0 tx=0).stop 2023-09-23T03:15:40.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:40.134+0000 7f6ef681a640 1 -- 172.21.15.154:0/3155737826 shutdown_connections 2023-09-23T03:15:40.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:40.134+0000 7f6ef681a640 1 --2- 172.21.15.154:0/3155737826 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6ed006af30 0x7f6ed006d3f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:40.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:40.134+0000 7f6ef681a640 1 --2- 172.21.15.154:0/3155737826 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ef00cf040 0x7f6ef00cd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:40.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:40.134+0000 7f6ef681a640 1 --2- 172.21.15.154:0/3155737826 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ef00b1b20 0x7f6ef00cd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:40.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:40.134+0000 7f6ef681a640 1 --2- 172.21.15.154:0/3155737826 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ef00acce0 0x7f6ef0159510 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:40.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:40.134+0000 7f6ef681a640 1 -- 172.21.15.154:0/3155737826 >> 172.21.15.154:0/3155737826 conn(0x7f6ef006a0f0 msgr2=0x7f6ef00ce900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:40.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:40.134+0000 7f6ef681a640 1 -- 172.21.15.154:0/3155737826 shutdown_connections 2023-09-23T03:15:40.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:40.134+0000 7f6ef681a640 1 -- 172.21.15.154:0/3155737826 wait complete. 2023-09-23T03:15:40.152 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi154.front.sepia.ceph.com... 2023-09-23T03:15:40.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:15:40.153 DEBUG:teuthology.orchestra.run.smithi154:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-09-23T03:15:40.241 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi154.stderr:ceph-fuse[93331]: fuse finished with error 0 and tester_r 0 2023-09-23T03:15:40.246 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:15:40.647 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:15:40.656 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:15:46.348 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2023-09-23T03:15:46.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:15:46.349 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-09-23T03:15:46.420 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:15:46.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.643+0000 7fcb8d5ee640 1 Processor -- start 2023-09-23T03:15:46.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.643+0000 7fcb8d5ee640 1 -- start start 2023-09-23T03:15:46.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.644+0000 7fcb8d5ee640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb880cf130 0x7fcb880ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:46.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.645+0000 7fcb8d5ee640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcb880b2fe0 0x7fcb8815d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:46.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.645+0000 7fcb8d5ee640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb8806ceb0 con 0x7fcb880ada30 2023-09-23T03:15:46.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.645+0000 7fcb8d5ee640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb880ade10 con 0x7fcb880b2fe0 2023-09-23T03:15:46.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.645+0000 7fcb8d5ee640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb88159c90 con 0x7fcb880cf130 2023-09-23T03:15:46.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.645+0000 7fcb867fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb880cf130 0x7fcb880ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:46.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.645+0000 7fcb877fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcb880b2fe0 0x7fcb8815d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:46.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.645+0000 7fcb86ffd640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fcb880ada30 0x7fcb880cea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.154:42294/0 (socket says 172.21.15.154:42294) 2023-09-23T03:15:46.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.645+0000 7fcb86ffd640 1 -- 172.21.15.154:0/2220553693 learned_addr learned my addr 172.21.15.154:0/2220553693 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:46.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.645+0000 7fcb867fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb880cf130 0x7fcb880ccaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:43636/0 (socket says 172.21.15.154:43636) 2023-09-23T03:15:46.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.646+0000 7fcb867fc640 1 -- 172.21.15.154:0/2220553693 >> v2:172.21.15.190:3300/0 conn(0x7fcb880b2fe0 msgr2=0x7fcb8815d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:46.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.646+0000 7fcb867fc640 1 --2- 172.21.15.154:0/2220553693 >> v2:172.21.15.190:3300/0 conn(0x7fcb880b2fe0 0x7fcb8815d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:46.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.646+0000 7fcb867fc640 1 -- 172.21.15.154:0/2220553693 >> v1:172.21.15.190:6789/0 conn(0x7fcb880ada30 legacy=0x7fcb880cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:46.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.646+0000 7fcb867fc640 1 -- 172.21.15.154:0/2220553693 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8806c110 con 0x7fcb880cf130 2023-09-23T03:15:46.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.646+0000 7fcb877fe640 1 --2- 172.21.15.154:0/2220553693 >> v2:172.21.15.190:3300/0 conn(0x7fcb880b2fe0 0x7fcb8815d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:15:46.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.647+0000 7fcb867fc640 1 --2- 172.21.15.154:0/2220553693 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb880cf130 0x7fcb880ccaf0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c0097a0 tx=0x7fcb7c02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20e6949f00b4f802 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:46.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.648+0000 7fcb85ffb640 1 -- 172.21.15.154:0/2220553693 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c02d040 con 0x7fcb880cf130 2023-09-23T03:15:46.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.648+0000 7fcb85ffb640 1 -- 172.21.15.154:0/2220553693 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb7c031810 con 0x7fcb880cf130 2023-09-23T03:15:46.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.648+0000 7fcb85ffb640 1 -- 172.21.15.154:0/2220553693 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c031ba0 con 0x7fcb880cf130 2023-09-23T03:15:46.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.648+0000 7fcb8d5ee640 1 -- 172.21.15.154:0/2220553693 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb880cf130 msgr2=0x7fcb880ccaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:46.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.648+0000 7fcb8d5ee640 1 --2- 172.21.15.154:0/2220553693 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb880cf130 0x7fcb880ccaf0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c0097a0 tx=0x7fcb7c02b8b0 comp rx=0 tx=0).stop 2023-09-23T03:15:46.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.649+0000 7fcb8d5ee640 1 -- 172.21.15.154:0/2220553693 shutdown_connections 2023-09-23T03:15:46.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.649+0000 7fcb8d5ee640 1 --2- 172.21.15.154:0/2220553693 >> v2:172.21.15.190:3300/0 conn(0x7fcb880b2fe0 0x7fcb8815d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:46.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.649+0000 7fcb8d5ee640 1 --2- 172.21.15.154:0/2220553693 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb880cf130 0x7fcb880ccaf0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:46.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.649+0000 7fcb8d5ee640 1 -- 172.21.15.154:0/2220553693 >> 172.21.15.154:0/2220553693 conn(0x7fcb8806a0f0 msgr2=0x7fcb8806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:46.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.649+0000 7fcb8d5ee640 1 -- 172.21.15.154:0/2220553693 shutdown_connections 2023-09-23T03:15:46.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.649+0000 7fcb8d5ee640 1 -- 172.21.15.154:0/2220553693 wait complete. 2023-09-23T03:15:46.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.650+0000 7fcb8d5ee640 1 Processor -- start 2023-09-23T03:15:46.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.650+0000 7fcb8d5ee640 1 -- start start 2023-09-23T03:15:46.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.651+0000 7fcb8d5ee640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb880ada30 0x7fcb88097210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:46.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.651+0000 7fcb8d5ee640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb880b2fe0 0x7fcb88151d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:46.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.651+0000 7fcb8d5ee640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb880cf130 0x7fcb8815bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:46.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.651+0000 7fcb8d5ee640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb880c4b10 con 0x7fcb880b2fe0 2023-09-23T03:15:46.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.651+0000 7fcb8d5ee640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8815b8c0 con 0x7fcb880ada30 2023-09-23T03:15:46.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.651+0000 7fcb8d5ee640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb88097e80 con 0x7fcb880cf130 2023-09-23T03:15:46.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.652+0000 7fcb867fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb880b2fe0 0x7fcb88151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:46.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.652+0000 7fcb867fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb880b2fe0 0x7fcb88151d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44308/0 (socket says 172.21.15.154:44308) 2023-09-23T03:15:46.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.652+0000 7fcb867fc640 1 -- 172.21.15.154:0/299413460 learned_addr learned my addr 172.21.15.154:0/299413460 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:46.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.652+0000 7fcb86ffd640 1 --2- 172.21.15.154:0/299413460 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb880ada30 0x7fcb88097210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:46.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.652+0000 7fcb877fe640 1 --2- 172.21.15.154:0/299413460 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb880cf130 0x7fcb8815bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:46.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.652+0000 7fcb867fc640 1 -- 172.21.15.154:0/299413460 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb880cf130 msgr2=0x7fcb8815bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:46.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.652+0000 7fcb867fc640 1 --2- 172.21.15.154:0/299413460 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb880cf130 0x7fcb8815bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:46.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.652+0000 7fcb867fc640 1 -- 172.21.15.154:0/299413460 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb880ada30 msgr2=0x7fcb88097210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:46.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.652+0000 7fcb867fc640 1 --2- 172.21.15.154:0/299413460 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb880ada30 0x7fcb88097210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:46.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.652+0000 7fcb867fc640 1 -- 172.21.15.154:0/299413460 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb80009a70 con 0x7fcb880b2fe0 2023-09-23T03:15:46.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.652+0000 7fcb86ffd640 1 --2- 172.21.15.154:0/299413460 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb880ada30 0x7fcb88097210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:15:46.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.653+0000 7fcb867fc640 1 --2- 172.21.15.154:0/299413460 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb880b2fe0 0x7fcb88151d10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c000c00 tx=0x7fcb7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:46.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.653+0000 7fcb6f7fe640 1 -- 172.21.15.154:0/299413460 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c006e10 con 0x7fcb880b2fe0 2023-09-23T03:15:46.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.653+0000 7fcb6f7fe640 1 -- 172.21.15.154:0/299413460 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb7c03ad60 con 0x7fcb880b2fe0 2023-09-23T03:15:46.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.653+0000 7fcb6f7fe640 1 -- 172.21.15.154:0/299413460 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c044630 con 0x7fcb880b2fe0 2023-09-23T03:15:46.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.653+0000 7fcb8d5ee640 1 -- 172.21.15.154:0/299413460 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb7c009400 con 0x7fcb880b2fe0 2023-09-23T03:15:46.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.653+0000 7fcb8d5ee640 1 -- 172.21.15.154:0/299413460 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb880b1ef0 con 0x7fcb880b2fe0 2023-09-23T03:15:46.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.654+0000 7fcb8d5ee640 1 -- 172.21.15.154:0/299413460 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb8815ba70 con 0x7fcb880b2fe0 2023-09-23T03:15:46.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.655+0000 7fcb6f7fe640 1 -- 172.21.15.154:0/299413460 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fcb7c04d450 con 0x7fcb880b2fe0 2023-09-23T03:15:46.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.656+0000 7fcb6f7fe640 1 --2- 172.21.15.154:0/299413460 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcb7406af30 0x7fcb7406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:46.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.656+0000 7fcb6f7fe640 1 -- 172.21.15.154:0/299413460 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fcb7c0abc90 con 0x7fcb880b2fe0 2023-09-23T03:15:46.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.656+0000 7fcb86ffd640 1 --2- 172.21.15.154:0/299413460 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcb7406af30 0x7fcb7406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:46.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.657+0000 7fcb86ffd640 1 --2- 172.21.15.154:0/299413460 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcb7406af30 0x7fcb7406d3f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcb780058d0 tx=0x7fcb78009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:46.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.661+0000 7fcb6f7fe640 1 -- 172.21.15.154:0/299413460 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fcb8815ba70 con 0x7fcb880b2fe0 2023-09-23T03:15:46.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.852+0000 7fcb8d5ee640 1 -- 172.21.15.154:0/299413460 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb8815ba30 con 0x7fcb880b2fe0 2023-09-23T03:15:46.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.853+0000 7fcb6f7fe640 1 -- 172.21.15.154:0/299413460 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18382 (secure 0 0 0) 0x7fcb7c076950 con 0x7fcb880b2fe0 2023-09-23T03:15:46.854 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:15:46.854 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":24,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:15:25.775060+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:15:18.575849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T03:15:19.707212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T03:15:23.337138+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:15:46.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.856+0000 7fcb8d5ee640 1 -- 172.21.15.154:0/299413460 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcb7406af30 msgr2=0x7fcb7406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:46.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.856+0000 7fcb8d5ee640 1 --2- 172.21.15.154:0/299413460 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcb7406af30 0x7fcb7406d3f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcb780058d0 tx=0x7fcb78009040 comp rx=0 tx=0).stop 2023-09-23T03:15:46.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.856+0000 7fcb8d5ee640 1 -- 172.21.15.154:0/299413460 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb880b2fe0 msgr2=0x7fcb88151d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:46.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.856+0000 7fcb8d5ee640 1 --2- 172.21.15.154:0/299413460 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb880b2fe0 0x7fcb88151d10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c000c00 tx=0x7fcb7c0046c0 comp rx=0 tx=0).stop 2023-09-23T03:15:46.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.856+0000 7fcb8d5ee640 1 -- 172.21.15.154:0/299413460 shutdown_connections 2023-09-23T03:15:46.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.856+0000 7fcb8d5ee640 1 --2- 172.21.15.154:0/299413460 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcb7406af30 0x7fcb7406d3f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:46.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.856+0000 7fcb8d5ee640 1 --2- 172.21.15.154:0/299413460 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb880cf130 0x7fcb8815bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:46.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.856+0000 7fcb8d5ee640 1 --2- 172.21.15.154:0/299413460 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb880b2fe0 0x7fcb88151d10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:46.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.857+0000 7fcb8d5ee640 1 --2- 172.21.15.154:0/299413460 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb880ada30 0x7fcb88097210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:46.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.857+0000 7fcb8d5ee640 1 -- 172.21.15.154:0/299413460 >> 172.21.15.154:0/299413460 conn(0x7fcb8806a0f0 msgr2=0x7fcb880af170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:46.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.857+0000 7fcb8d5ee640 1 -- 172.21.15.154:0/299413460 shutdown_connections 2023-09-23T03:15:46.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:46.857+0000 7fcb8d5ee640 1 -- 172.21.15.154:0/299413460 wait complete. 2023-09-23T03:15:46.876 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:15:47.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.085+0000 7f2f28010640 1 Processor -- start 2023-09-23T03:15:47.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.085+0000 7f2f28010640 1 -- start start 2023-09-23T03:15:47.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.086+0000 7f2f28010640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f200cf130 0x7f2f200ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:47.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.086+0000 7f2f28010640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2f200b2fe0 0x7f2f2015d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:47.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.086+0000 7f2f28010640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f2006ceb0 con 0x7f2f200ada30 2023-09-23T03:15:47.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.086+0000 7f2f28010640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f200ade10 con 0x7f2f200b2fe0 2023-09-23T03:15:47.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.086+0000 7f2f28010640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f20159c90 con 0x7f2f200cf130 2023-09-23T03:15:47.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.087+0000 7f2f265ad640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2f200b2fe0 0x7f2f2015d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:47.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.087+0000 7f2f25dac640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f2f200ada30 0x7f2f200cea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:42238/0 (socket says 172.21.15.154:42238) 2023-09-23T03:15:47.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.087+0000 7f2f25dac640 1 -- 172.21.15.154:0/2725918546 learned_addr learned my addr 172.21.15.154:0/2725918546 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:47.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.087+0000 7f2f265ad640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2f200b2fe0 0x7f2f2015d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44312/0 (socket says 172.21.15.154:44312) 2023-09-23T03:15:47.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.087+0000 7f2f255ab640 1 --2- 172.21.15.154:0/2725918546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f200cf130 0x7f2f200ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:47.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.087+0000 7f2f265ad640 1 -- 172.21.15.154:0/2725918546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f200cf130 msgr2=0x7f2f200ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:47.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.087+0000 7f2f265ad640 1 --2- 172.21.15.154:0/2725918546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f200cf130 0x7f2f200ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:47.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.087+0000 7f2f265ad640 1 -- 172.21.15.154:0/2725918546 >> v1:172.21.15.154:6789/0 conn(0x7f2f200ada30 legacy=0x7f2f200cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:47.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.087+0000 7f2f265ad640 1 -- 172.21.15.154:0/2725918546 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f2006c110 con 0x7f2f200b2fe0 2023-09-23T03:15:47.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.087+0000 7f2f24daa640 1 -- 172.21.15.154:0/2725918546 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3347384859 0 0) 0x7f2f14001290 con 0x7f2f200ada30 2023-09-23T03:15:47.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.088+0000 7f2f265ad640 1 --2- 172.21.15.154:0/2725918546 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f200b2fe0 0x7f2f2015d380 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2f18009d90 tx=0x7f2f1802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf4d39a07105b3d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:47.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.089+0000 7f2f24daa640 1 -- 172.21.15.154:0/2725918546 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1802cc50 con 0x7f2f200b2fe0 2023-09-23T03:15:47.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.089+0000 7f2f24daa640 1 -- 172.21.15.154:0/2725918546 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f1802cdb0 con 0x7f2f200b2fe0 2023-09-23T03:15:47.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.089+0000 7f2f24daa640 1 -- 172.21.15.154:0/2725918546 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f18034660 con 0x7f2f200b2fe0 2023-09-23T03:15:47.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.089+0000 7f2f28010640 1 -- 172.21.15.154:0/2725918546 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f200b2fe0 msgr2=0x7f2f2015d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:47.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.089+0000 7f2f28010640 1 --2- 172.21.15.154:0/2725918546 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f200b2fe0 0x7f2f2015d380 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2f18009d90 tx=0x7f2f1802bea0 comp rx=0 tx=0).stop 2023-09-23T03:15:47.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.089+0000 7f2f28010640 1 -- 172.21.15.154:0/2725918546 shutdown_connections 2023-09-23T03:15:47.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.089+0000 7f2f28010640 1 --2- 172.21.15.154:0/2725918546 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f200b2fe0 0x7f2f2015d380 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:47.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.089+0000 7f2f28010640 1 --2- 172.21.15.154:0/2725918546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f200cf130 0x7f2f200ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:47.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.089+0000 7f2f28010640 1 -- 172.21.15.154:0/2725918546 >> 172.21.15.154:0/2725918546 conn(0x7f2f2006a0f0 msgr2=0x7f2f2006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:47.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.089+0000 7f2f28010640 1 -- 172.21.15.154:0/2725918546 shutdown_connections 2023-09-23T03:15:47.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.089+0000 7f2f28010640 1 -- 172.21.15.154:0/2725918546 wait complete. 2023-09-23T03:15:47.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.091+0000 7f2f28010640 1 Processor -- start 2023-09-23T03:15:47.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.091+0000 7f2f28010640 1 -- start start 2023-09-23T03:15:47.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.092+0000 7f2f28010640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f200ada30 0x7f2f200c4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:47.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.092+0000 7f2f28010640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f200b2fe0 0x7f2f201f1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:47.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.092+0000 7f2f28010640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f200cf130 0x7f2f2015a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:47.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.092+0000 7f2f28010640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2015b8e0 con 0x7f2f200b2fe0 2023-09-23T03:15:47.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.092+0000 7f2f28010640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f20158590 con 0x7f2f200cf130 2023-09-23T03:15:47.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.092+0000 7f2f28010640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2015b320 con 0x7f2f200ada30 2023-09-23T03:15:47.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.092+0000 7f2f255ab640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f200b2fe0 0x7f2f201f1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:47.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.092+0000 7f2f255ab640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f200b2fe0 0x7f2f201f1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44326/0 (socket says 172.21.15.154:44326) 2023-09-23T03:15:47.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.092+0000 7f2f255ab640 1 -- 172.21.15.154:0/2411303750 learned_addr learned my addr 172.21.15.154:0/2411303750 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:47.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.093+0000 7f2f25dac640 1 --2- 172.21.15.154:0/2411303750 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f200ada30 0x7f2f200c4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:47.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.093+0000 7f2f265ad640 1 --2- 172.21.15.154:0/2411303750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f200cf130 0x7f2f2015a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:47.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.093+0000 7f2f255ab640 1 -- 172.21.15.154:0/2411303750 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f200ada30 msgr2=0x7f2f200c4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:47.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.093+0000 7f2f255ab640 1 --2- 172.21.15.154:0/2411303750 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f200ada30 0x7f2f200c4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:47.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.093+0000 7f2f255ab640 1 -- 172.21.15.154:0/2411303750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f200cf130 msgr2=0x7f2f2015a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:47.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.093+0000 7f2f255ab640 1 --2- 172.21.15.154:0/2411303750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f200cf130 0x7f2f2015a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:47.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.093+0000 7f2f255ab640 1 -- 172.21.15.154:0/2411303750 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f18009a70 con 0x7f2f200b2fe0 2023-09-23T03:15:47.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.093+0000 7f2f265ad640 1 --2- 172.21.15.154:0/2411303750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f200cf130 0x7f2f2015a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:47.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.093+0000 7f2f255ab640 1 --2- 172.21.15.154:0/2411303750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f200b2fe0 0x7f2f201f1f40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c00d5f0 tx=0x7f2f1c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:47.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.094+0000 7f2f127fc640 1 -- 172.21.15.154:0/2411303750 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c0125f0 con 0x7f2f200b2fe0 2023-09-23T03:15:47.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.094+0000 7f2f127fc640 1 -- 172.21.15.154:0/2411303750 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f1c012750 con 0x7f2f200b2fe0 2023-09-23T03:15:47.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.094+0000 7f2f28010640 1 -- 172.21.15.154:0/2411303750 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f2015b600 con 0x7f2f200b2fe0 2023-09-23T03:15:47.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.094+0000 7f2f127fc640 1 -- 172.21.15.154:0/2411303750 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c012b00 con 0x7f2f200b2fe0 2023-09-23T03:15:47.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.094+0000 7f2f28010640 1 -- 172.21.15.154:0/2411303750 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f200aebc0 con 0x7f2f200b2fe0 2023-09-23T03:15:47.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.095+0000 7f2f28010640 1 -- 172.21.15.154:0/2411303750 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f200af6f0 con 0x7f2f200b2fe0 2023-09-23T03:15:47.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.096+0000 7f2f127fc640 1 -- 172.21.15.154:0/2411303750 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f2f1c012c60 con 0x7f2f200b2fe0 2023-09-23T03:15:47.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.097+0000 7f2f127fc640 1 --2- 172.21.15.154:0/2411303750 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2f0406af30 0x7f2f0406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:47.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.097+0000 7f2f127fc640 1 -- 172.21.15.154:0/2411303750 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f2f1c089ff0 con 0x7f2f200b2fe0 2023-09-23T03:15:47.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.097+0000 7f2f25dac640 1 --2- 172.21.15.154:0/2411303750 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2f0406af30 0x7f2f0406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:47.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.098+0000 7f2f25dac640 1 --2- 172.21.15.154:0/2411303750 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2f0406af30 0x7f2f0406d3f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2f140014f0 tx=0x7f2f14009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:47.101 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.101+0000 7f2f127fc640 1 -- 172.21.15.154:0/2411303750 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f2f1c0bca50 con 0x7f2f200b2fe0 2023-09-23T03:15:47.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.294+0000 7f2f28010640 1 -- 172.21.15.154:0/2411303750 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f2006d370 con 0x7f2f200b2fe0 2023-09-23T03:15:47.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.296+0000 7f2f127fc640 1 -- 172.21.15.154:0/2411303750 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18382 (secure 0 0 0) 0x7f2f1c054c60 con 0x7f2f200b2fe0 2023-09-23T03:15:47.297 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:15:47.297 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":24,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:15:25.775060+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:15:18.575849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T03:15:19.707212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T03:15:23.337138+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:15:47.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.297+0000 7f2f28010640 1 -- 172.21.15.154:0/2411303750 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2f0406af30 msgr2=0x7f2f0406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:47.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.297+0000 7f2f28010640 1 --2- 172.21.15.154:0/2411303750 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2f0406af30 0x7f2f0406d3f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2f140014f0 tx=0x7f2f14009040 comp rx=0 tx=0).stop 2023-09-23T03:15:47.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.297+0000 7f2f28010640 1 -- 172.21.15.154:0/2411303750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f200b2fe0 msgr2=0x7f2f201f1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:47.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.297+0000 7f2f28010640 1 --2- 172.21.15.154:0/2411303750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f200b2fe0 0x7f2f201f1f40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c00d5f0 tx=0x7f2f1c00dab0 comp rx=0 tx=0).stop 2023-09-23T03:15:47.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.297+0000 7f2f28010640 1 -- 172.21.15.154:0/2411303750 shutdown_connections 2023-09-23T03:15:47.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.297+0000 7f2f28010640 1 --2- 172.21.15.154:0/2411303750 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2f0406af30 0x7f2f0406d3f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:47.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.297+0000 7f2f28010640 1 --2- 172.21.15.154:0/2411303750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f200cf130 0x7f2f2015a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:47.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.297+0000 7f2f28010640 1 --2- 172.21.15.154:0/2411303750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f200b2fe0 0x7f2f201f1f40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:47.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.297+0000 7f2f28010640 1 --2- 172.21.15.154:0/2411303750 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f200ada30 0x7f2f200c4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:47.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.297+0000 7f2f28010640 1 -- 172.21.15.154:0/2411303750 >> 172.21.15.154:0/2411303750 conn(0x7f2f2006a0f0 msgr2=0x7f2f200adf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:47.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.298+0000 7f2f28010640 1 -- 172.21.15.154:0/2411303750 shutdown_connections 2023-09-23T03:15:47.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.298+0000 7f2f28010640 1 -- 172.21.15.154:0/2411303750 wait complete. 2023-09-23T03:15:47.318 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-09-23T03:15:47.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.529+0000 7f31ea6bb640 1 Processor -- start 2023-09-23T03:15:47.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.529+0000 7f31ea6bb640 1 -- start start 2023-09-23T03:15:47.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.530+0000 7f31ea6bb640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f31e415aba0 0x7f31e415bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:47.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.530+0000 7f31ea6bb640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31e415e5f0 0x7f31e415ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:47.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.530+0000 7f31ea6bb640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31e415b6a0 con 0x7f31e40caea0 2023-09-23T03:15:47.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.530+0000 7f31ea6bb640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31e415f0b0 con 0x7f31e415aba0 2023-09-23T03:15:47.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.530+0000 7f31ea6bb640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31e415f390 con 0x7f31e415e5f0 2023-09-23T03:15:47.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.531+0000 7f31e8c58640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f31e40caea0 0x7f31e40cc5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:42252/0 (socket says 172.21.15.154:42252) 2023-09-23T03:15:47.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.531+0000 7f31e8c58640 1 -- 172.21.15.154:0/232124658 learned_addr learned my addr 172.21.15.154:0/232124658 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:47.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.531+0000 7f31e3fff640 1 --2- 172.21.15.154:0/232124658 >> v2:172.21.15.190:3300/0 conn(0x7f31e415aba0 0x7f31e415bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:47.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.531+0000 7f31dbfff640 1 --2- 172.21.15.154:0/232124658 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31e415e5f0 0x7f31e415ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:47.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.531+0000 7f31e37fe640 1 -- 172.21.15.154:0/232124658 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1755473255 0 0) 0x7f31e415b6a0 con 0x7f31e40caea0 2023-09-23T03:15:47.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.531+0000 7f31e37fe640 1 -- 172.21.15.154:0/232124658 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31d0003520 con 0x7f31e40caea0 2023-09-23T03:15:47.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.532+0000 7f31e37fe640 1 -- 172.21.15.154:0/232124658 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1837927573 0 0) 0x7f31d0003520 con 0x7f31e40caea0 2023-09-23T03:15:47.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.532+0000 7f31e37fe640 1 -- 172.21.15.154:0/232124658 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31e415e5f0 msgr2=0x7f31e415ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:47.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.532+0000 7f31e37fe640 1 --2- 172.21.15.154:0/232124658 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31e415e5f0 0x7f31e415ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:47.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.532+0000 7f31e37fe640 1 -- 172.21.15.154:0/232124658 >> v2:172.21.15.190:3300/0 conn(0x7f31e415aba0 msgr2=0x7f31e415bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:47.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.532+0000 7f31e37fe640 1 --2- 172.21.15.154:0/232124658 >> v2:172.21.15.190:3300/0 conn(0x7f31e415aba0 0x7f31e415bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:47.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.532+0000 7f31e37fe640 1 -- 172.21.15.154:0/232124658 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e415b330 con 0x7f31e40caea0 2023-09-23T03:15:47.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.532+0000 7f31e37fe640 1 -- 172.21.15.154:0/232124658 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f31dc0019f0 con 0x7f31e40caea0 2023-09-23T03:15:47.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.532+0000 7f31e37fe640 1 -- 172.21.15.154:0/232124658 >> v1:172.21.15.154:6789/0 conn(0x7f31e40caea0 legacy=0x7f31e40cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:47.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.532+0000 7f31e37fe640 1 --2- 172.21.15.154:0/232124658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31d0003970 0x7f31d0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:47.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.532+0000 7f31e37fe640 1 --2- 172.21.15.154:0/232124658 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31e415e5f0 0x7f31d00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:47.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.532+0000 7f31e37fe640 1 --2- 172.21.15.154:0/232124658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31d0004d00 0x7f31d0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:47.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.532+0000 7f31e37fe640 1 -- 172.21.15.154:0/232124658 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e415f0b0 con 0x7f31d0004d00 2023-09-23T03:15:47.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.532+0000 7f31e37fe640 1 -- 172.21.15.154:0/232124658 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e415f390 con 0x7f31d0003970 2023-09-23T03:15:47.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.532+0000 7f31e37fe640 1 -- 172.21.15.154:0/232124658 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d000b980 con 0x7f31e415e5f0 2023-09-23T03:15:47.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.532+0000 7f31e8c58640 1 --2- 172.21.15.154:0/232124658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31d0004d00 0x7f31d0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:47.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.532+0000 7f31e3fff640 1 --2- 172.21.15.154:0/232124658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31d0003970 0x7f31d0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:47.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.532+0000 7f31dbfff640 1 --2- 172.21.15.154:0/232124658 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31e415e5f0 0x7f31d00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:47.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.532+0000 7f31e8c58640 1 -- 172.21.15.154:0/232124658 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31e415e5f0 msgr2=0x7f31d00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:47.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.532+0000 7f31e8c58640 1 --2- 172.21.15.154:0/232124658 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31e415e5f0 0x7f31d00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:47.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.532+0000 7f31e8c58640 1 -- 172.21.15.154:0/232124658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31d0003970 msgr2=0x7f31d0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:47.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.532+0000 7f31e8c58640 1 --2- 172.21.15.154:0/232124658 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31d0003970 0x7f31d0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:47.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.532+0000 7f31e8c58640 1 -- 172.21.15.154:0/232124658 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d0003520 con 0x7f31d0004d00 2023-09-23T03:15:47.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.533+0000 7f31e8c58640 1 --2- 172.21.15.154:0/232124658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31d0004d00 0x7f31d0005180 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f31dc00a790 tx=0x7f31d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ded71d9a7f0195 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:47.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.534+0000 7f31e37fe640 1 -- 172.21.15.154:0/232124658 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31dc034630 con 0x7f31d0004d00 2023-09-23T03:15:47.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.534+0000 7f31e37fe640 1 -- 172.21.15.154:0/232124658 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31dc02f070 con 0x7f31d0004d00 2023-09-23T03:15:47.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.534+0000 7f31e37fe640 1 -- 172.21.15.154:0/232124658 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31dc0349f0 con 0x7f31d0004d00 2023-09-23T03:15:47.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.534+0000 7f31ea6bb640 1 -- 172.21.15.154:0/232124658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31d0004d00 msgr2=0x7f31d0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:47.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.534+0000 7f31ea6bb640 1 --2- 172.21.15.154:0/232124658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31d0004d00 0x7f31d0005180 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f31dc00a790 tx=0x7f31d00057f0 comp rx=0 tx=0).stop 2023-09-23T03:15:47.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.534+0000 7f31e8c58640 1 -- 172.21.15.154:0/232124658 reap_dead start 2023-09-23T03:15:47.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.534+0000 7f31ea6bb640 1 -- 172.21.15.154:0/232124658 shutdown_connections 2023-09-23T03:15:47.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.534+0000 7f31ea6bb640 1 -- 172.21.15.154:0/232124658 >> 172.21.15.154:0/232124658 conn(0x7f31e406c010 msgr2=0x7f31e406c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:47.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.535+0000 7f31ea6bb640 1 -- 172.21.15.154:0/232124658 shutdown_connections 2023-09-23T03:15:47.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.535+0000 7f31ea6bb640 1 -- 172.21.15.154:0/232124658 wait complete. 2023-09-23T03:15:47.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.537+0000 7f31ea6bb640 1 Processor -- start 2023-09-23T03:15:47.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.537+0000 7f31ea6bb640 1 -- start start 2023-09-23T03:15:47.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.538+0000 7f31ea6bb640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31d0004d00 0x7f31e40c6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:47.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.538+0000 7f31ea6bb640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31e41f43d0 0x7f31e40c24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:47.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.538+0000 7f31ea6bb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31e40c79b0 0x7f31e40c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:47.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.538+0000 7f31ea6bb640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e40c6910 con 0x7f31e40c79b0 2023-09-23T03:15:47.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.538+0000 7f31ea6bb640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e40c6bf0 con 0x7f31e41f43d0 2023-09-23T03:15:47.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.538+0000 7f31ea6bb640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31e40c6ed0 con 0x7f31d0004d00 2023-09-23T03:15:47.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.538+0000 7f31e8c58640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31e40c79b0 0x7f31e40c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:47.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.538+0000 7f31e8c58640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31e40c79b0 0x7f31e40c7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44340/0 (socket says 172.21.15.154:44340) 2023-09-23T03:15:47.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.538+0000 7f31e8c58640 1 -- 172.21.15.154:0/2570653890 learned_addr learned my addr 172.21.15.154:0/2570653890 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:47.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.539+0000 7f31e3fff640 1 --2- 172.21.15.154:0/2570653890 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31d0004d00 0x7f31e40c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:47.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.539+0000 7f31dbfff640 1 --2- 172.21.15.154:0/2570653890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31e41f43d0 0x7f31e40c24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:47.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.539+0000 7f31e8c58640 1 -- 172.21.15.154:0/2570653890 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31d0004d00 msgr2=0x7f31e40c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:47.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.539+0000 7f31e8c58640 1 --2- 172.21.15.154:0/2570653890 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31d0004d00 0x7f31e40c6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:47.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.539+0000 7f31e8c58640 1 -- 172.21.15.154:0/2570653890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31e41f43d0 msgr2=0x7f31e40c24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:47.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.539+0000 7f31e8c58640 1 --2- 172.21.15.154:0/2570653890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31e41f43d0 0x7f31e40c24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:47.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.539+0000 7f31e8c58640 1 -- 172.21.15.154:0/2570653890 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e415b100 con 0x7f31e40c79b0 2023-09-23T03:15:47.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.539+0000 7f31e8c58640 1 --2- 172.21.15.154:0/2570653890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31e40c79b0 0x7f31e40c7e60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f31dc02cba0 tx=0x7f31dc001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:47.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.539+0000 7f31e17fa640 1 -- 172.21.15.154:0/2570653890 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31dc034840 con 0x7f31e40c79b0 2023-09-23T03:15:47.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.539+0000 7f31e17fa640 1 -- 172.21.15.154:0/2570653890 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31dc001f00 con 0x7f31e40c79b0 2023-09-23T03:15:47.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.539+0000 7f31ea6bb640 1 -- 172.21.15.154:0/2570653890 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31e40c71b0 con 0x7f31e40c79b0 2023-09-23T03:15:47.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.540+0000 7f31e17fa640 1 -- 172.21.15.154:0/2570653890 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31dc034c10 con 0x7f31e40c79b0 2023-09-23T03:15:47.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.540+0000 7f31ea6bb640 1 -- 172.21.15.154:0/2570653890 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31e40c83a0 con 0x7f31e40c79b0 2023-09-23T03:15:47.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.541+0000 7f31ea6bb640 1 -- 172.21.15.154:0/2570653890 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31e40c76d0 con 0x7f31e40c79b0 2023-09-23T03:15:47.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.542+0000 7f31e17fa640 1 -- 172.21.15.154:0/2570653890 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f31dc031030 con 0x7f31e40c79b0 2023-09-23T03:15:47.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.542+0000 7f31e17fa640 1 --2- 172.21.15.154:0/2570653890 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f31c406af60 0x7f31c406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:47.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.543+0000 7f31e17fa640 1 -- 172.21.15.154:0/2570653890 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f31dc0abf00 con 0x7f31e40c79b0 2023-09-23T03:15:47.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.543+0000 7f31e3fff640 1 --2- 172.21.15.154:0/2570653890 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f31c406af60 0x7f31c406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:47.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.544+0000 7f31e3fff640 1 --2- 172.21.15.154:0/2570653890 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f31c406af60 0x7f31c406d420 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f31d4009a10 tx=0x7f31d4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:47.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.547+0000 7f31e17fa640 1 -- 172.21.15.154:0/2570653890 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f31e40c76d0 con 0x7f31e40c79b0 2023-09-23T03:15:47.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:47.757+0000 7f31ea6bb640 1 -- 172.21.15.154:0/2570653890 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f31e40c76d0 con 0x7f31e40c79b0 2023-09-23T03:15:48.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.247+0000 7f31e17fa640 1 -- 172.21.15.154:0/2570653890 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7f31dc07a760 con 0x7f31e40c79b0 2023-09-23T03:15:48.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.251+0000 7f31ea6bb640 1 -- 172.21.15.154:0/2570653890 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f31c406af60 msgr2=0x7f31c406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:48.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.251+0000 7f31ea6bb640 1 --2- 172.21.15.154:0/2570653890 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f31c406af60 0x7f31c406d420 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f31d4009a10 tx=0x7f31d4008040 comp rx=0 tx=0).stop 2023-09-23T03:15:48.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.251+0000 7f31ea6bb640 1 -- 172.21.15.154:0/2570653890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31e40c79b0 msgr2=0x7f31e40c7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:48.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.251+0000 7f31ea6bb640 1 --2- 172.21.15.154:0/2570653890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31e40c79b0 0x7f31e40c7e60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f31dc02cba0 tx=0x7f31dc001cb0 comp rx=0 tx=0).stop 2023-09-23T03:15:48.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.251+0000 7f31ea6bb640 1 -- 172.21.15.154:0/2570653890 shutdown_connections 2023-09-23T03:15:48.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.251+0000 7f31ea6bb640 1 --2- 172.21.15.154:0/2570653890 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f31c406af60 0x7f31c406d420 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:48.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.251+0000 7f31ea6bb640 1 --2- 172.21.15.154:0/2570653890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f31e40c79b0 0x7f31e40c7e60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:48.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.251+0000 7f31ea6bb640 1 --2- 172.21.15.154:0/2570653890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f31e41f43d0 0x7f31e40c24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:48.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.251+0000 7f31ea6bb640 1 --2- 172.21.15.154:0/2570653890 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f31d0004d00 0x7f31e40c6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:48.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.251+0000 7f31ea6bb640 1 -- 172.21.15.154:0/2570653890 >> 172.21.15.154:0/2570653890 conn(0x7f31e406c010 msgr2=0x7f31e415d710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:48.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.251+0000 7f31ea6bb640 1 -- 172.21.15.154:0/2570653890 shutdown_connections 2023-09-23T03:15:48.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.251+0000 7f31ea6bb640 1 -- 172.21.15.154:0/2570653890 wait complete. 2023-09-23T03:15:48.273 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2023-09-23T03:15:48.273 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-09-23T03:15:48.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.493+0000 7f17e9fed640 1 Processor -- start 2023-09-23T03:15:48.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.493+0000 7f17e9fed640 1 -- start start 2023-09-23T03:15:48.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.495+0000 7f17e9fed640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17e415cc60 0x7f17e406fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:48.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.495+0000 7f17e9fed640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f17e41606d0 0x7f17e40cade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:48.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.495+0000 7f17e9fed640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17e4160bd0 con 0x7f17e40c9550 2023-09-23T03:15:48.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.495+0000 7f17e9fed640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17e40cc4e0 con 0x7f17e41606d0 2023-09-23T03:15:48.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.495+0000 7f17e9fed640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17e4160eb0 con 0x7f17e415cc60 2023-09-23T03:15:48.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.496+0000 7f17e97ec640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f17e40c9550 0x7f17e40cc930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:42254/0 (socket says 172.21.15.154:42254) 2023-09-23T03:15:48.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.496+0000 7f17e97ec640 1 -- 172.21.15.154:0/147536225 learned_addr learned my addr 172.21.15.154:0/147536225 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:48.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.496+0000 7f17e8feb640 1 --2- 172.21.15.154:0/147536225 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17e415cc60 0x7f17e406fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:48.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.496+0000 7f17dbfff640 1 --2- 172.21.15.154:0/147536225 >> v2:172.21.15.190:3300/0 conn(0x7f17e41606d0 0x7f17e40cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:48.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.496+0000 7f17db7fe640 1 -- 172.21.15.154:0/147536225 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2573411161 0 0) 0x7f17e4160bd0 con 0x7f17e40c9550 2023-09-23T03:15:48.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.496+0000 7f17db7fe640 1 -- 172.21.15.154:0/147536225 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17cc003520 con 0x7f17e40c9550 2023-09-23T03:15:48.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.496+0000 7f17db7fe640 1 -- 172.21.15.154:0/147536225 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2165775981 0 0) 0x7f17cc003520 con 0x7f17e40c9550 2023-09-23T03:15:48.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.496+0000 7f17db7fe640 1 -- 172.21.15.154:0/147536225 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17e415cc60 msgr2=0x7f17e406fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:48.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.496+0000 7f17db7fe640 1 --2- 172.21.15.154:0/147536225 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17e415cc60 0x7f17e406fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:48.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.496+0000 7f17db7fe640 1 -- 172.21.15.154:0/147536225 >> v2:172.21.15.190:3300/0 conn(0x7f17e41606d0 msgr2=0x7f17e40cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:48.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.496+0000 7f17db7fe640 1 --2- 172.21.15.154:0/147536225 >> v2:172.21.15.190:3300/0 conn(0x7f17e41606d0 0x7f17e40cade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:48.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.496+0000 7f17db7fe640 1 -- 172.21.15.154:0/147536225 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e4161190 con 0x7f17e40c9550 2023-09-23T03:15:48.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.496+0000 7f17db7fe640 1 -- 172.21.15.154:0/147536225 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f17dc0019f0 con 0x7f17e40c9550 2023-09-23T03:15:48.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.496+0000 7f17db7fe640 1 -- 172.21.15.154:0/147536225 >> v1:172.21.15.154:6789/0 conn(0x7f17e40c9550 legacy=0x7f17e40cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:48.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.497+0000 7f17db7fe640 1 --2- 172.21.15.154:0/147536225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17cc003970 0x7f17cc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:48.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.497+0000 7f17db7fe640 1 --2- 172.21.15.154:0/147536225 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17e415cc60 0x7f17cc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:48.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.497+0000 7f17db7fe640 1 --2- 172.21.15.154:0/147536225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17cc004d00 0x7f17cc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:48.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.497+0000 7f17db7fe640 1 -- 172.21.15.154:0/147536225 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e40cc4e0 con 0x7f17cc004d00 2023-09-23T03:15:48.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.497+0000 7f17db7fe640 1 -- 172.21.15.154:0/147536225 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e4160eb0 con 0x7f17cc003970 2023-09-23T03:15:48.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.497+0000 7f17db7fe640 1 -- 172.21.15.154:0/147536225 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc00b980 con 0x7f17e415cc60 2023-09-23T03:15:48.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.497+0000 7f17e97ec640 1 --2- 172.21.15.154:0/147536225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17cc004d00 0x7f17cc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:48.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.497+0000 7f17dbfff640 1 --2- 172.21.15.154:0/147536225 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17e415cc60 0x7f17cc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:48.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.497+0000 7f17e8feb640 1 --2- 172.21.15.154:0/147536225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17cc003970 0x7f17cc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:48.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.497+0000 7f17e97ec640 1 -- 172.21.15.154:0/147536225 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17e415cc60 msgr2=0x7f17cc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:48.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.497+0000 7f17e97ec640 1 --2- 172.21.15.154:0/147536225 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17e415cc60 0x7f17cc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:48.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.497+0000 7f17e97ec640 1 -- 172.21.15.154:0/147536225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17cc003970 msgr2=0x7f17cc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:48.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.497+0000 7f17e97ec640 1 --2- 172.21.15.154:0/147536225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17cc003970 0x7f17cc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:48.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.497+0000 7f17e97ec640 1 -- 172.21.15.154:0/147536225 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17cc003520 con 0x7f17cc004d00 2023-09-23T03:15:48.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.497+0000 7f17e8feb640 1 --2- 172.21.15.154:0/147536225 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17cc003970 0x7f17cc003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:15:48.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.499+0000 7f17e97ec640 1 --2- 172.21.15.154:0/147536225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17cc004d00 0x7f17cc005180 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f17e40c9980 tx=0x7f17cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3834b7d2b94ed8f2 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:48.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.499+0000 7f17db7fe640 1 -- 172.21.15.154:0/147536225 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc034630 con 0x7f17cc004d00 2023-09-23T03:15:48.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.499+0000 7f17db7fe640 1 -- 172.21.15.154:0/147536225 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17dc02f070 con 0x7f17cc004d00 2023-09-23T03:15:48.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.499+0000 7f17db7fe640 1 -- 172.21.15.154:0/147536225 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc0349d0 con 0x7f17cc004d00 2023-09-23T03:15:48.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.499+0000 7f17e9fed640 1 -- 172.21.15.154:0/147536225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17cc004d00 msgr2=0x7f17cc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:48.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.499+0000 7f17e9fed640 1 --2- 172.21.15.154:0/147536225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17cc004d00 0x7f17cc005180 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f17e40c9980 tx=0x7f17cc0057f0 comp rx=0 tx=0).stop 2023-09-23T03:15:48.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.499+0000 7f17e97ec640 1 -- 172.21.15.154:0/147536225 reap_dead start 2023-09-23T03:15:48.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.499+0000 7f17e9fed640 1 -- 172.21.15.154:0/147536225 shutdown_connections 2023-09-23T03:15:48.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.500+0000 7f17e9fed640 1 -- 172.21.15.154:0/147536225 >> 172.21.15.154:0/147536225 conn(0x7f17e40aff20 msgr2=0x7f17e40b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:48.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.500+0000 7f17e9fed640 1 -- 172.21.15.154:0/147536225 shutdown_connections 2023-09-23T03:15:48.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.500+0000 7f17e9fed640 1 -- 172.21.15.154:0/147536225 wait complete. 2023-09-23T03:15:48.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.501+0000 7f17e9fed640 1 Processor -- start 2023-09-23T03:15:48.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.502+0000 7f17e9fed640 1 -- start start 2023-09-23T03:15:48.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.502+0000 7f17e9fed640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17cc004d00 0x7f17e415f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:48.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.503+0000 7f17e9fed640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17e40c2120 0x7f17e415ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:48.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.503+0000 7f17e9fed640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17e40ac910 0x7f17e40acd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:48.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.503+0000 7f17e9fed640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e415d140 con 0x7f17e40c2120 2023-09-23T03:15:48.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.503+0000 7f17e9fed640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e41512e0 con 0x7f17e40ac910 2023-09-23T03:15:48.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.503+0000 7f17e9fed640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17e40ad2d0 con 0x7f17cc004d00 2023-09-23T03:15:48.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.503+0000 7f17dbfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17e40c2120 0x7f17e415ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:48.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.503+0000 7f17dbfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17e40c2120 0x7f17e415ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44360/0 (socket says 172.21.15.154:44360) 2023-09-23T03:15:48.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.503+0000 7f17dbfff640 1 -- 172.21.15.154:0/2155418028 learned_addr learned my addr 172.21.15.154:0/2155418028 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:48.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.503+0000 7f17e8feb640 1 --2- 172.21.15.154:0/2155418028 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17cc004d00 0x7f17e415f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:48.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.503+0000 7f17e97ec640 1 --2- 172.21.15.154:0/2155418028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17e40ac910 0x7f17e40acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:48.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.503+0000 7f17dbfff640 1 -- 172.21.15.154:0/2155418028 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17cc004d00 msgr2=0x7f17e415f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:48.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.503+0000 7f17dbfff640 1 --2- 172.21.15.154:0/2155418028 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17cc004d00 0x7f17e415f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:48.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.503+0000 7f17dbfff640 1 -- 172.21.15.154:0/2155418028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17e40ac910 msgr2=0x7f17e40acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:48.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.503+0000 7f17dbfff640 1 --2- 172.21.15.154:0/2155418028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17e40ac910 0x7f17e40acd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:48.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.503+0000 7f17dbfff640 1 -- 172.21.15.154:0/2155418028 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e40afd10 con 0x7f17e40c2120 2023-09-23T03:15:48.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.503+0000 7f17e97ec640 1 --2- 172.21.15.154:0/2155418028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17e40ac910 0x7f17e40acd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:48.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.504+0000 7f17dbfff640 1 --2- 172.21.15.154:0/2155418028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17e40c2120 0x7f17e415ee40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f17d4011a70 tx=0x7f17d4011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:48.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.504+0000 7f17d97fa640 1 -- 172.21.15.154:0/2155418028 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d4009680 con 0x7f17e40c2120 2023-09-23T03:15:48.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.504+0000 7f17d97fa640 1 -- 172.21.15.154:0/2155418028 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17d400fb50 con 0x7f17e40c2120 2023-09-23T03:15:48.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.504+0000 7f17e9fed640 1 -- 172.21.15.154:0/2155418028 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17e40c4710 con 0x7f17e40c2120 2023-09-23T03:15:48.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.504+0000 7f17d97fa640 1 -- 172.21.15.154:0/2155418028 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d400fee0 con 0x7f17e40c2120 2023-09-23T03:15:48.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.504+0000 7f17e9fed640 1 -- 172.21.15.154:0/2155418028 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17e40c4cd0 con 0x7f17e40c2120 2023-09-23T03:15:48.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.505+0000 7f17e9fed640 1 -- 172.21.15.154:0/2155418028 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17e415d3b0 con 0x7f17e40c2120 2023-09-23T03:15:48.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.506+0000 7f17d97fa640 1 -- 172.21.15.154:0/2155418028 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f17d4014440 con 0x7f17e40c2120 2023-09-23T03:15:48.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.507+0000 7f17d97fa640 1 --2- 172.21.15.154:0/2155418028 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f17d006af30 0x7f17d006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:48.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.507+0000 7f17d97fa640 1 -- 172.21.15.154:0/2155418028 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f17d408a3f0 con 0x7f17e40c2120 2023-09-23T03:15:48.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.507+0000 7f17e8feb640 1 --2- 172.21.15.154:0/2155418028 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f17d006af30 0x7f17d006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:48.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.508+0000 7f17e8feb640 1 --2- 172.21.15.154:0/2155418028 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f17d006af30 0x7f17d006d3f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f17e0009990 tx=0x7f17e0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:48.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.511+0000 7f17d97fa640 1 -- 172.21.15.154:0/2155418028 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f17d4055130 con 0x7f17e40c2120 2023-09-23T03:15:48.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:48.739+0000 7f17e9fed640 1 -- 172.21.15.154:0/2155418028 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f17e404e920 con 0x7f17e40c2120 2023-09-23T03:15:49.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.265+0000 7f17d97fa640 1 -- 172.21.15.154:0/2155418028 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7f17d4058750 con 0x7f17e40c2120 2023-09-23T03:15:49.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.269+0000 7f17e9fed640 1 -- 172.21.15.154:0/2155418028 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f17d006af30 msgr2=0x7f17d006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:49.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.269+0000 7f17e9fed640 1 --2- 172.21.15.154:0/2155418028 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f17d006af30 0x7f17d006d3f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f17e0009990 tx=0x7f17e0008040 comp rx=0 tx=0).stop 2023-09-23T03:15:49.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.269+0000 7f17e9fed640 1 -- 172.21.15.154:0/2155418028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17e40c2120 msgr2=0x7f17e415ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:49.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.269+0000 7f17e9fed640 1 --2- 172.21.15.154:0/2155418028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17e40c2120 0x7f17e415ee40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f17d4011a70 tx=0x7f17d4011f30 comp rx=0 tx=0).stop 2023-09-23T03:15:49.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.269+0000 7f17e9fed640 1 -- 172.21.15.154:0/2155418028 shutdown_connections 2023-09-23T03:15:49.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.269+0000 7f17e9fed640 1 --2- 172.21.15.154:0/2155418028 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f17d006af30 0x7f17d006d3f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:49.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.269+0000 7f17e9fed640 1 --2- 172.21.15.154:0/2155418028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17e40ac910 0x7f17e40acd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:49.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.269+0000 7f17e9fed640 1 --2- 172.21.15.154:0/2155418028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17e40c2120 0x7f17e415ee40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:49.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.269+0000 7f17e9fed640 1 --2- 172.21.15.154:0/2155418028 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17cc004d00 0x7f17e415f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:49.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.269+0000 7f17e9fed640 1 -- 172.21.15.154:0/2155418028 >> 172.21.15.154:0/2155418028 conn(0x7f17e40aff20 msgr2=0x7f17e40add30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:49.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.269+0000 7f17e9fed640 1 -- 172.21.15.154:0/2155418028 shutdown_connections 2023-09-23T03:15:49.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.270+0000 7f17e9fed640 1 -- 172.21.15.154:0/2155418028 wait complete. 2023-09-23T03:15:49.291 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-09-23T03:15:49.291 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:15:49.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.500+0000 7fde12caf640 1 Processor -- start 2023-09-23T03:15:49.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.501+0000 7fde12caf640 1 -- start start 2023-09-23T03:15:49.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.502+0000 7fde12caf640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fde0c0b2420 0x7fde0c1605d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:49.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.502+0000 7fde12caf640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde0c0ceff0 0x7fde0c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:49.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.502+0000 7fde12caf640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde0c06b350 con 0x7fde0c0b2800 2023-09-23T03:15:49.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.502+0000 7fde12caf640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde0c06bfd0 con 0x7fde0c0b2420 2023-09-23T03:15:49.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.502+0000 7fde12caf640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c06b630 con 0x7fde0c0ceff0 2023-09-23T03:15:49.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.502+0000 7fde10a4b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fde0c0b2420 0x7fde0c1605d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:49.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.503+0000 7fde10a4b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fde0c0b2420 0x7fde0c1605d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44372/0 (socket says 172.21.15.154:44372) 2023-09-23T03:15:49.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.503+0000 7fde10a4b640 1 -- 172.21.15.154:0/673012488 learned_addr learned my addr 172.21.15.154:0/673012488 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:49.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.503+0000 7fde0bfff640 1 --2- 172.21.15.154:0/673012488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde0c0ceff0 0x7fde0c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:49.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.503+0000 7fde10a4b640 1 -- 172.21.15.154:0/673012488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde0c0ceff0 msgr2=0x7fde0c06c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:49.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.503+0000 7fde10a4b640 1 --2- 172.21.15.154:0/673012488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde0c0ceff0 0x7fde0c06c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:49.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.503+0000 7fde10a4b640 1 -- 172.21.15.154:0/673012488 >> v1:172.21.15.190:6789/0 conn(0x7fde0c0b2800 legacy=0x7fde0c15d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:49.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.503+0000 7fde10a4b640 1 -- 172.21.15.154:0/673012488 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde0c06b910 con 0x7fde0c0b2420 2023-09-23T03:15:49.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.503+0000 7fde0bfff640 1 --2- 172.21.15.154:0/673012488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde0c0ceff0 0x7fde0c06c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:49.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.504+0000 7fde10a4b640 1 --2- 172.21.15.154:0/673012488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde0c0b2420 0x7fde0c1605d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fddfc009870 tx=0x7fddfc02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7655d2d71fb4b8c6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:49.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.504+0000 7fde0b7fe640 1 -- 172.21.15.154:0/673012488 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc0317c0 con 0x7fde0c0b2420 2023-09-23T03:15:49.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.505+0000 7fde0b7fe640 1 -- 172.21.15.154:0/673012488 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddfc031920 con 0x7fde0c0b2420 2023-09-23T03:15:49.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.505+0000 7fde0b7fe640 1 -- 172.21.15.154:0/673012488 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc031cb0 con 0x7fde0c0b2420 2023-09-23T03:15:49.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.505+0000 7fde12caf640 1 -- 172.21.15.154:0/673012488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde0c0b2420 msgr2=0x7fde0c1605d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:49.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.505+0000 7fde12caf640 1 --2- 172.21.15.154:0/673012488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde0c0b2420 0x7fde0c1605d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fddfc009870 tx=0x7fddfc02b860 comp rx=0 tx=0).stop 2023-09-23T03:15:49.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.505+0000 7fde12caf640 1 -- 172.21.15.154:0/673012488 shutdown_connections 2023-09-23T03:15:49.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.505+0000 7fde12caf640 1 --2- 172.21.15.154:0/673012488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde0c0ceff0 0x7fde0c06c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:49.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.505+0000 7fde12caf640 1 --2- 172.21.15.154:0/673012488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde0c0b2420 0x7fde0c1605d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:49.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.505+0000 7fde12caf640 1 -- 172.21.15.154:0/673012488 >> 172.21.15.154:0/673012488 conn(0x7fde0c0cd1b0 msgr2=0x7fde0c0cd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:49.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.505+0000 7fde12caf640 1 -- 172.21.15.154:0/673012488 shutdown_connections 2023-09-23T03:15:49.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.505+0000 7fde12caf640 1 -- 172.21.15.154:0/673012488 wait complete. 2023-09-23T03:15:49.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.507+0000 7fde12caf640 1 Processor -- start 2023-09-23T03:15:49.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.507+0000 7fde12caf640 1 -- start start 2023-09-23T03:15:49.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.508+0000 7fde12caf640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde0c0b2420 0x7fde0c159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:49.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.508+0000 7fde12caf640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde0c0b2800 0x7fde0c15d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:49.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.508+0000 7fde12caf640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde0c158530 0x7fde0c142a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:49.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.508+0000 7fde12caf640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c158a10 con 0x7fde0c0b2420 2023-09-23T03:15:49.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.508+0000 7fde12caf640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c158cf0 con 0x7fde0c0b2800 2023-09-23T03:15:49.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.508+0000 7fde12caf640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c158fd0 con 0x7fde0c158530 2023-09-23T03:15:49.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.508+0000 7fde10a4b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde0c0b2420 0x7fde0c159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:49.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.508+0000 7fde10a4b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde0c0b2420 0x7fde0c159ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44382/0 (socket says 172.21.15.154:44382) 2023-09-23T03:15:49.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.508+0000 7fde10a4b640 1 -- 172.21.15.154:0/554150482 learned_addr learned my addr 172.21.15.154:0/554150482 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:49.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.509+0000 7fde0bfff640 1 --2- 172.21.15.154:0/554150482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde0c0b2800 0x7fde0c15d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:49.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.509+0000 7fde10a4b640 1 -- 172.21.15.154:0/554150482 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde0c158530 msgr2=0x7fde0c142a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:49.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.509+0000 7fde1124c640 1 --2- 172.21.15.154:0/554150482 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde0c158530 0x7fde0c142a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:49.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.509+0000 7fde10a4b640 1 --2- 172.21.15.154:0/554150482 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde0c158530 0x7fde0c142a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:49.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.509+0000 7fde10a4b640 1 -- 172.21.15.154:0/554150482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde0c0b2800 msgr2=0x7fde0c15d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:49.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.509+0000 7fde10a4b640 1 --2- 172.21.15.154:0/554150482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde0c0b2800 0x7fde0c15d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:49.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.509+0000 7fde10a4b640 1 -- 172.21.15.154:0/554150482 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddfc0094d0 con 0x7fde0c0b2420 2023-09-23T03:15:49.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.509+0000 7fde0bfff640 1 --2- 172.21.15.154:0/554150482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde0c0b2800 0x7fde0c15d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:49.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.509+0000 7fde1124c640 1 --2- 172.21.15.154:0/554150482 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde0c158530 0x7fde0c142a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:49.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.509+0000 7fde10a4b640 1 --2- 172.21.15.154:0/554150482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde0c0b2420 0x7fde0c159ab0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fddfc02b8a0 tx=0x7fddfc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:49.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.510+0000 7fde097fa640 1 -- 172.21.15.154:0/554150482 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc0329a0 con 0x7fde0c0b2420 2023-09-23T03:15:49.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.510+0000 7fde097fa640 1 -- 172.21.15.154:0/554150482 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddfc032b40 con 0x7fde0c0b2420 2023-09-23T03:15:49.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.510+0000 7fde097fa640 1 -- 172.21.15.154:0/554150482 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc0453f0 con 0x7fde0c0b2420 2023-09-23T03:15:49.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.510+0000 7fde12caf640 1 -- 172.21.15.154:0/554150482 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde0c1592b0 con 0x7fde0c0b2420 2023-09-23T03:15:49.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.510+0000 7fde12caf640 1 -- 172.21.15.154:0/554150482 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde0c15d900 con 0x7fde0c0b2420 2023-09-23T03:15:49.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.511+0000 7fde12caf640 1 -- 172.21.15.154:0/554150482 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde0c1597d0 con 0x7fde0c0b2420 2023-09-23T03:15:49.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.512+0000 7fde097fa640 1 -- 172.21.15.154:0/554150482 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fddfc04e450 con 0x7fde0c0b2420 2023-09-23T03:15:49.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.513+0000 7fde097fa640 1 --2- 172.21.15.154:0/554150482 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fddec06af30 0x7fddec06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:49.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.513+0000 7fde097fa640 1 -- 172.21.15.154:0/554150482 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fddfc0abe40 con 0x7fde0c0b2420 2023-09-23T03:15:49.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.513+0000 7fde0bfff640 1 --2- 172.21.15.154:0/554150482 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fddec06af30 0x7fddec06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:49.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.514+0000 7fde0bfff640 1 --2- 172.21.15.154:0/554150482 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fddec06af30 0x7fddec06d3f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fddf4006fd0 tx=0x7fddf4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:49.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.518+0000 7fde097fa640 1 -- 172.21.15.154:0/554150482 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fddfc076af0 con 0x7fde0c0b2420 2023-09-23T03:15:49.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.743+0000 7fde12caf640 1 -- 172.21.15.154:0/554150482 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde0c073eb0 con 0x7fde0c0b2420 2023-09-23T03:15:49.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.744+0000 7fde097fa640 1 -- 172.21.15.154:0/554150482 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4921 (secure 0 0 0) 0x7fddfc07a110 con 0x7fde0c0b2420 2023-09-23T03:15:49.746 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:15:49.746 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2644365591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2644365591},{"type":"v1","addr":"172.21.15.190:6835","nonce":2644365591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/4145467852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4145467852},{"type":"v1","addr":"172.21.15.154:6835","nonce":4145467852}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1022302774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1022302774},{"type":"v1","addr":"172.21.15.154:6837","nonce":1022302774}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:15:20.686035+0000","modified":"2023-09-23T03:15:48.742318+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/3561509423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3561509423},{"type":"v1","addr":"172.21.15.190:6834","nonce":3561509423}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T03:15:49.746 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 9 2023-09-23T03:15:49.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.747+0000 7fde12caf640 1 -- 172.21.15.154:0/554150482 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fddec06af30 msgr2=0x7fddec06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:49.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.747+0000 7fde12caf640 1 --2- 172.21.15.154:0/554150482 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fddec06af30 0x7fddec06d3f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fddf4006fd0 tx=0x7fddf4008040 comp rx=0 tx=0).stop 2023-09-23T03:15:49.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.747+0000 7fde12caf640 1 -- 172.21.15.154:0/554150482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde0c0b2420 msgr2=0x7fde0c159ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:49.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.747+0000 7fde12caf640 1 --2- 172.21.15.154:0/554150482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde0c0b2420 0x7fde0c159ab0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fddfc02b8a0 tx=0x7fddfc0046c0 comp rx=0 tx=0).stop 2023-09-23T03:15:49.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.748+0000 7fde12caf640 1 -- 172.21.15.154:0/554150482 shutdown_connections 2023-09-23T03:15:49.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.748+0000 7fde12caf640 1 --2- 172.21.15.154:0/554150482 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fddec06af30 0x7fddec06d3f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:49.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.748+0000 7fde12caf640 1 --2- 172.21.15.154:0/554150482 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde0c158530 0x7fde0c142a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:49.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.748+0000 7fde12caf640 1 --2- 172.21.15.154:0/554150482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde0c0b2800 0x7fde0c15d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:49.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.748+0000 7fde12caf640 1 --2- 172.21.15.154:0/554150482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde0c0b2420 0x7fde0c159ab0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:49.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.748+0000 7fde12caf640 1 -- 172.21.15.154:0/554150482 >> 172.21.15.154:0/554150482 conn(0x7fde0c0cd1b0 msgr2=0x7fde0c160c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:49.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.748+0000 7fde12caf640 1 -- 172.21.15.154:0/554150482 shutdown_connections 2023-09-23T03:15:49.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.748+0000 7fde12caf640 1 -- 172.21.15.154:0/554150482 wait complete. 2023-09-23T03:15:49.766 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:15:49.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.970+0000 7f97416b4640 1 Processor -- start 2023-09-23T03:15:49.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.971+0000 7f97416b4640 1 -- start start 2023-09-23T03:15:49.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.972+0000 7f97416b4640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f973c0b28e0 0x7f973c1615b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:49.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.972+0000 7f97416b4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f973c06b1e0 0x7f973c0cca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:49.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.972+0000 7f97416b4640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f973c0ccf80 con 0x7f973c15cf40 2023-09-23T03:15:49.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.972+0000 7f97416b4640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f973c0cd1c0 con 0x7f973c0b28e0 2023-09-23T03:15:49.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.972+0000 7f97416b4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973c0cd4a0 con 0x7f973c06b1e0 2023-09-23T03:15:49.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.973+0000 7f973b7fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f973c06b1e0 0x7f973c0cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:49.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.973+0000 7f973affd640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f973c15cf40 0x7f973c0b21d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.154:42316/0 (socket says 172.21.15.154:42316) 2023-09-23T03:15:49.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.973+0000 7f973affd640 1 -- 172.21.15.154:0/3636137320 learned_addr learned my addr 172.21.15.154:0/3636137320 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:49.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.973+0000 7f97327fc640 1 --2- 172.21.15.154:0/3636137320 >> v2:172.21.15.190:3300/0 conn(0x7f973c0b28e0 0x7f973c1615b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:49.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.974+0000 7f97327fc640 1 -- 172.21.15.154:0/3636137320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f973c06b1e0 msgr2=0x7f973c0cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:49.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.974+0000 7f97327fc640 1 --2- 172.21.15.154:0/3636137320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f973c06b1e0 0x7f973c0cca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:49.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.974+0000 7f97327fc640 1 -- 172.21.15.154:0/3636137320 >> v1:172.21.15.190:6789/0 conn(0x7f973c15cf40 legacy=0x7f973c0b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:49.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.974+0000 7f97327fc640 1 -- 172.21.15.154:0/3636137320 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f973c0cd780 con 0x7f973c0b28e0 2023-09-23T03:15:49.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.974+0000 7f973b7fe640 1 --2- 172.21.15.154:0/3636137320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f973c06b1e0 0x7f973c0cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:15:49.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.975+0000 7f97327fc640 1 --2- 172.21.15.154:0/3636137320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f973c0b28e0 0x7f973c1615b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f97240097a0 tx=0x7f972402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d738bebc400a67e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:49.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.975+0000 7f973a7fc640 1 -- 172.21.15.154:0/3636137320 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972402d040 con 0x7f973c0b28e0 2023-09-23T03:15:49.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.975+0000 7f973a7fc640 1 -- 172.21.15.154:0/3636137320 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9724031810 con 0x7f973c0b28e0 2023-09-23T03:15:49.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.975+0000 7f973a7fc640 1 -- 172.21.15.154:0/3636137320 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9724031ba0 con 0x7f973c0b28e0 2023-09-23T03:15:49.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.976+0000 7f97416b4640 1 -- 172.21.15.154:0/3636137320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f973c0b28e0 msgr2=0x7f973c1615b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:49.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.976+0000 7f97416b4640 1 --2- 172.21.15.154:0/3636137320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f973c0b28e0 0x7f973c1615b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f97240097a0 tx=0x7f972402b8b0 comp rx=0 tx=0).stop 2023-09-23T03:15:49.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.976+0000 7f97416b4640 1 -- 172.21.15.154:0/3636137320 shutdown_connections 2023-09-23T03:15:49.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.976+0000 7f97416b4640 1 --2- 172.21.15.154:0/3636137320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f973c06b1e0 0x7f973c0cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:49.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.976+0000 7f97416b4640 1 --2- 172.21.15.154:0/3636137320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f973c0b28e0 0x7f973c1615b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:49.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.976+0000 7f97416b4640 1 -- 172.21.15.154:0/3636137320 >> 172.21.15.154:0/3636137320 conn(0x7f973c06fc60 msgr2=0x7f973c070040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:49.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.976+0000 7f97416b4640 1 -- 172.21.15.154:0/3636137320 shutdown_connections 2023-09-23T03:15:49.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.976+0000 7f97416b4640 1 -- 172.21.15.154:0/3636137320 wait complete. 2023-09-23T03:15:49.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.978+0000 7f97416b4640 1 Processor -- start 2023-09-23T03:15:49.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.978+0000 7f97416b4640 1 -- start start 2023-09-23T03:15:49.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.979+0000 7f97416b4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f973c06b1e0 0x7f973c0b5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:49.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.979+0000 7f97416b4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f973c0b28e0 0x7f973c15b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:49.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.979+0000 7f97416b4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f973c15cf40 0x7f973c15b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:49.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.979+0000 7f97416b4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c0c8780 con 0x7f973c06b1e0 2023-09-23T03:15:49.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.979+0000 7f97416b4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c0c8a60 con 0x7f973c0b28e0 2023-09-23T03:15:49.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.979+0000 7f97416b4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973c0c8d40 con 0x7f973c15cf40 2023-09-23T03:15:49.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.979+0000 7f973affd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f973c06b1e0 0x7f973c0b5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:49.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.979+0000 7f973affd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f973c06b1e0 0x7f973c0b5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44392/0 (socket says 172.21.15.154:44392) 2023-09-23T03:15:49.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.979+0000 7f973affd640 1 -- 172.21.15.154:0/4154838372 learned_addr learned my addr 172.21.15.154:0/4154838372 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:49.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.980+0000 7f973affd640 1 -- 172.21.15.154:0/4154838372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f973c15cf40 msgr2=0x7f973c15b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:49.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.980+0000 7f973b7fe640 1 --2- 172.21.15.154:0/4154838372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f973c15cf40 0x7f973c15b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:49.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.980+0000 7f973affd640 1 --2- 172.21.15.154:0/4154838372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f973c15cf40 0x7f973c15b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:49.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.980+0000 7f973affd640 1 -- 172.21.15.154:0/4154838372 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f973c0b28e0 msgr2=0x7f973c15b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:49.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.980+0000 7f973affd640 1 --2- 172.21.15.154:0/4154838372 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f973c0b28e0 0x7f973c15b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:49.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.980+0000 7f973affd640 1 -- 172.21.15.154:0/4154838372 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9734009a70 con 0x7f973c06b1e0 2023-09-23T03:15:49.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.980+0000 7f973affd640 1 --2- 172.21.15.154:0/4154838372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f973c06b1e0 0x7f973c0b5ba0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f972c00c3c0 tx=0x7f972c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:49.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.980+0000 7f9733fff640 1 -- 172.21.15.154:0/4154838372 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972c005ce0 con 0x7f973c06b1e0 2023-09-23T03:15:49.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.980+0000 7f9733fff640 1 -- 172.21.15.154:0/4154838372 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f972c005e40 con 0x7f973c06b1e0 2023-09-23T03:15:49.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.980+0000 7f9733fff640 1 -- 172.21.15.154:0/4154838372 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972c0072d0 con 0x7f973c06b1e0 2023-09-23T03:15:49.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.980+0000 7f97416b4640 1 -- 172.21.15.154:0/4154838372 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9724009400 con 0x7f973c06b1e0 2023-09-23T03:15:49.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.980+0000 7f97416b4640 1 -- 172.21.15.154:0/4154838372 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f973c0c9300 con 0x7f973c06b1e0 2023-09-23T03:15:49.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.981+0000 7f97416b4640 1 -- 172.21.15.154:0/4154838372 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f973c073eb0 con 0x7f973c06b1e0 2023-09-23T03:15:49.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.982+0000 7f9733fff640 1 -- 172.21.15.154:0/4154838372 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f972c007430 con 0x7f973c06b1e0 2023-09-23T03:15:49.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.983+0000 7f9733fff640 1 --2- 172.21.15.154:0/4154838372 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f971c06af60 0x7f971c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:49.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.983+0000 7f9733fff640 1 -- 172.21.15.154:0/4154838372 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f972c088df0 con 0x7f973c06b1e0 2023-09-23T03:15:49.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.983+0000 7f97327fc640 1 --2- 172.21.15.154:0/4154838372 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f971c06af60 0x7f971c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:49.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.984+0000 7f97327fc640 1 --2- 172.21.15.154:0/4154838372 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f971c06af60 0x7f971c06d420 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9724036000 tx=0x7f9724030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:49.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:49.987+0000 7f9733fff640 1 -- 172.21.15.154:0/4154838372 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f972c053b30 con 0x7f973c06b1e0 2023-09-23T03:15:50.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.177+0000 7f97416b4640 1 -- 172.21.15.154:0/4154838372 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f973c0ccf80 con 0x7f973c06b1e0 2023-09-23T03:15:50.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.178+0000 7f9733fff640 1 -- 172.21.15.154:0/4154838372 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18382 (secure 0 0 0) 0x7f973c0ccf80 con 0x7f973c06b1e0 2023-09-23T03:15:50.180 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:15:50.180 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":24,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:15:25.775060+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:15:18.575849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T03:15:19.707212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T03:15:23.337138+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:15:50.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.181+0000 7f97416b4640 1 -- 172.21.15.154:0/4154838372 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f971c06af60 msgr2=0x7f971c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:50.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.181+0000 7f97416b4640 1 --2- 172.21.15.154:0/4154838372 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f971c06af60 0x7f971c06d420 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9724036000 tx=0x7f9724030000 comp rx=0 tx=0).stop 2023-09-23T03:15:50.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.181+0000 7f97416b4640 1 -- 172.21.15.154:0/4154838372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f973c06b1e0 msgr2=0x7f973c0b5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:50.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.181+0000 7f97416b4640 1 --2- 172.21.15.154:0/4154838372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f973c06b1e0 0x7f973c0b5ba0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f972c00c3c0 tx=0x7f972c00c880 comp rx=0 tx=0).stop 2023-09-23T03:15:50.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.182+0000 7f97416b4640 1 -- 172.21.15.154:0/4154838372 shutdown_connections 2023-09-23T03:15:50.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.182+0000 7f97416b4640 1 --2- 172.21.15.154:0/4154838372 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f971c06af60 0x7f971c06d420 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:50.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.182+0000 7f97416b4640 1 --2- 172.21.15.154:0/4154838372 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f973c15cf40 0x7f973c15b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:50.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.182+0000 7f97416b4640 1 --2- 172.21.15.154:0/4154838372 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f973c0b28e0 0x7f973c15b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:50.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.182+0000 7f97416b4640 1 --2- 172.21.15.154:0/4154838372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f973c06b1e0 0x7f973c0b5ba0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:50.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.182+0000 7f97416b4640 1 -- 172.21.15.154:0/4154838372 >> 172.21.15.154:0/4154838372 conn(0x7f973c06fc60 msgr2=0x7f973c0cbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:50.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.182+0000 7f97416b4640 1 -- 172.21.15.154:0/4154838372 shutdown_connections 2023-09-23T03:15:50.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.182+0000 7f97416b4640 1 -- 172.21.15.154:0/4154838372 wait complete. 2023-09-23T03:15:50.203 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-09-23T03:15:50.203 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-23T03:15:50.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.411+0000 7fe474e10640 1 Processor -- start 2023-09-23T03:15:50.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.411+0000 7fe474e10640 1 -- start start 2023-09-23T03:15:50.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.412+0000 7fe474e10640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe4700ada30 0x7fe4700ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:50.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.412+0000 7fe474e10640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4700b2fe0 0x7fe4700cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:50.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.412+0000 7fe474e10640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe47006ceb0 con 0x7fe4700cef60 2023-09-23T03:15:50.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.412+0000 7fe474e10640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4700ade10 con 0x7fe4700ada30 2023-09-23T03:15:50.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.412+0000 7fe474e10640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4700cf3b0 con 0x7fe4700b2fe0 2023-09-23T03:15:50.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.413+0000 7fe46ed9d640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fe4700cef60 0x7fe4700b18f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:42258/0 (socket says 172.21.15.154:42258) 2023-09-23T03:15:50.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.413+0000 7fe46ed9d640 1 -- 172.21.15.154:0/369333702 learned_addr learned my addr 172.21.15.154:0/369333702 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:50.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.413+0000 7fe467fff640 1 --2- 172.21.15.154:0/369333702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4700b2fe0 0x7fe4700cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:50.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.413+0000 7fe46e59c640 1 --2- 172.21.15.154:0/369333702 >> v2:172.21.15.190:3300/0 conn(0x7fe4700ada30 0x7fe4700ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:50.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.413+0000 7fe46dd9b640 1 -- 172.21.15.154:0/369333702 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1175614847 0 0) 0x7fe468001490 con 0x7fe4700cef60 2023-09-23T03:15:50.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.413+0000 7fe46dd9b640 1 -- 172.21.15.154:0/369333702 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe45c003520 con 0x7fe4700cef60 2023-09-23T03:15:50.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.413+0000 7fe46dd9b640 1 -- 172.21.15.154:0/369333702 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1923644020 0 0) 0x7fe45c003520 con 0x7fe4700cef60 2023-09-23T03:15:50.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.414+0000 7fe46dd9b640 1 -- 172.21.15.154:0/369333702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4700b2fe0 msgr2=0x7fe4700cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:50.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.414+0000 7fe46dd9b640 1 --2- 172.21.15.154:0/369333702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4700b2fe0 0x7fe4700cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:50.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.414+0000 7fe46dd9b640 1 -- 172.21.15.154:0/369333702 >> v2:172.21.15.190:3300/0 conn(0x7fe4700ada30 msgr2=0x7fe4700ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:50.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.414+0000 7fe46dd9b640 1 --2- 172.21.15.154:0/369333702 >> v2:172.21.15.190:3300/0 conn(0x7fe4700ada30 0x7fe4700ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:50.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.414+0000 7fe46dd9b640 1 -- 172.21.15.154:0/369333702 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe470159c90 con 0x7fe4700cef60 2023-09-23T03:15:50.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.414+0000 7fe46dd9b640 1 -- 172.21.15.154:0/369333702 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fe468001b80 con 0x7fe4700cef60 2023-09-23T03:15:50.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.414+0000 7fe46dd9b640 1 -- 172.21.15.154:0/369333702 >> v1:172.21.15.154:6789/0 conn(0x7fe4700cef60 legacy=0x7fe4700b18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:50.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.414+0000 7fe46dd9b640 1 --2- 172.21.15.154:0/369333702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4700b2fe0 0x7fe45c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:50.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.414+0000 7fe46dd9b640 1 --2- 172.21.15.154:0/369333702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe45c003ef0 0x7fe45c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:50.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.414+0000 7fe46dd9b640 1 --2- 172.21.15.154:0/369333702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe45c004d00 0x7fe45c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:50.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.414+0000 7fe46dd9b640 1 -- 172.21.15.154:0/369333702 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4700ade10 con 0x7fe45c003ef0 2023-09-23T03:15:50.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.414+0000 7fe46dd9b640 1 -- 172.21.15.154:0/369333702 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4700cf3b0 con 0x7fe45c004d00 2023-09-23T03:15:50.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.414+0000 7fe46dd9b640 1 -- 172.21.15.154:0/369333702 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c00b980 con 0x7fe4700b2fe0 2023-09-23T03:15:50.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.414+0000 7fe467fff640 1 --2- 172.21.15.154:0/369333702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe45c003ef0 0x7fe45c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:50.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.414+0000 7fe46e59c640 1 --2- 172.21.15.154:0/369333702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4700b2fe0 0x7fe45c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:50.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.414+0000 7fe467fff640 1 -- 172.21.15.154:0/369333702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4700b2fe0 msgr2=0x7fe45c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:50.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.414+0000 7fe467fff640 1 --2- 172.21.15.154:0/369333702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4700b2fe0 0x7fe45c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:50.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.414+0000 7fe467fff640 1 -- 172.21.15.154:0/369333702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe45c004d00 msgr2=0x7fe45c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:50.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.414+0000 7fe46ed9d640 1 --2- 172.21.15.154:0/369333702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe45c004d00 0x7fe45c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:50.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.414+0000 7fe467fff640 1 --2- 172.21.15.154:0/369333702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe45c004d00 0x7fe45c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:50.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.414+0000 7fe467fff640 1 -- 172.21.15.154:0/369333702 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe45c003520 con 0x7fe45c003ef0 2023-09-23T03:15:50.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.416+0000 7fe467fff640 1 --2- 172.21.15.154:0/369333702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe45c003ef0 0x7fe45c0047c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe45800bd30 tx=0x7fe45c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29be9912196d2352 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:50.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.416+0000 7fe46dd9b640 1 -- 172.21.15.154:0/369333702 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe458030690 con 0x7fe45c003ef0 2023-09-23T03:15:50.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.416+0000 7fe46dd9b640 1 -- 172.21.15.154:0/369333702 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4580307f0 con 0x7fe45c003ef0 2023-09-23T03:15:50.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.416+0000 7fe46dd9b640 1 -- 172.21.15.154:0/369333702 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe458030b80 con 0x7fe45c003ef0 2023-09-23T03:15:50.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.416+0000 7fe474e10640 1 -- 172.21.15.154:0/369333702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe45c003ef0 msgr2=0x7fe45c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:50.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.416+0000 7fe474e10640 1 --2- 172.21.15.154:0/369333702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe45c003ef0 0x7fe45c0047c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe45800bd30 tx=0x7fe45c0057f0 comp rx=0 tx=0).stop 2023-09-23T03:15:50.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.416+0000 7fe46ed9d640 1 -- 172.21.15.154:0/369333702 reap_dead start 2023-09-23T03:15:50.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.417+0000 7fe474e10640 1 -- 172.21.15.154:0/369333702 shutdown_connections 2023-09-23T03:15:50.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.417+0000 7fe474e10640 1 -- 172.21.15.154:0/369333702 >> 172.21.15.154:0/369333702 conn(0x7fe47006a0f0 msgr2=0x7fe47006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:50.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.417+0000 7fe474e10640 1 -- 172.21.15.154:0/369333702 shutdown_connections 2023-09-23T03:15:50.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.417+0000 7fe474e10640 1 -- 172.21.15.154:0/369333702 wait complete. 2023-09-23T03:15:50.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.418+0000 7fe474e10640 1 Processor -- start 2023-09-23T03:15:50.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.418+0000 7fe474e10640 1 -- start start 2023-09-23T03:15:50.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.419+0000 7fe474e10640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe45c003ef0 0x7fe470155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:50.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.419+0000 7fe474e10640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4700cbe20 0x7fe4701f19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:50.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.419+0000 7fe474e10640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4700cae90 0x7fe4700cb310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:50.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.419+0000 7fe474e10640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4701f1f10 con 0x7fe4700cae90 2023-09-23T03:15:50.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.419+0000 7fe474e10640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4700cb850 con 0x7fe4700cbe20 2023-09-23T03:15:50.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.419+0000 7fe474e10640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe47015b200 con 0x7fe45c003ef0 2023-09-23T03:15:50.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.420+0000 7fe46ed9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4700cae90 0x7fe4700cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:50.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.420+0000 7fe46ed9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4700cae90 0x7fe4700cb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44420/0 (socket says 172.21.15.154:44420) 2023-09-23T03:15:50.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.420+0000 7fe46ed9d640 1 -- 172.21.15.154:0/1209279312 learned_addr learned my addr 172.21.15.154:0/1209279312 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:50.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.420+0000 7fe46e59c640 1 --2- 172.21.15.154:0/1209279312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe45c003ef0 0x7fe470155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:50.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.420+0000 7fe467fff640 1 --2- 172.21.15.154:0/1209279312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4700cbe20 0x7fe4701f19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:50.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.420+0000 7fe46ed9d640 1 -- 172.21.15.154:0/1209279312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe45c003ef0 msgr2=0x7fe470155f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:50.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.420+0000 7fe46ed9d640 1 --2- 172.21.15.154:0/1209279312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe45c003ef0 0x7fe470155f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:50.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.420+0000 7fe46ed9d640 1 -- 172.21.15.154:0/1209279312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4700cbe20 msgr2=0x7fe4701f19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:50.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.420+0000 7fe46ed9d640 1 --2- 172.21.15.154:0/1209279312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4700cbe20 0x7fe4701f19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:50.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.420+0000 7fe46ed9d640 1 -- 172.21.15.154:0/1209279312 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe470159e80 con 0x7fe4700cae90 2023-09-23T03:15:50.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.421+0000 7fe46ed9d640 1 --2- 172.21.15.154:0/1209279312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4700cae90 0x7fe4700cb310 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe468000f80 tx=0x7fe46800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:50.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.421+0000 7fe466ffd640 1 -- 172.21.15.154:0/1209279312 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46800de90 con 0x7fe4700cae90 2023-09-23T03:15:50.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.421+0000 7fe466ffd640 1 -- 172.21.15.154:0/1209279312 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe46800bb20 con 0x7fe4700cae90 2023-09-23T03:15:50.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.421+0000 7fe466ffd640 1 -- 172.21.15.154:0/1209279312 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46800bed0 con 0x7fe4700cae90 2023-09-23T03:15:50.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.421+0000 7fe474e10640 1 -- 172.21.15.154:0/1209279312 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe47015a160 con 0x7fe4700cae90 2023-09-23T03:15:50.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.421+0000 7fe474e10640 1 -- 172.21.15.154:0/1209279312 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe47015a720 con 0x7fe4700cae90 2023-09-23T03:15:50.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.422+0000 7fe474e10640 1 -- 172.21.15.154:0/1209279312 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe470073eb0 con 0x7fe4700cae90 2023-09-23T03:15:50.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.423+0000 7fe466ffd640 1 -- 172.21.15.154:0/1209279312 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fe468007070 con 0x7fe4700cae90 2023-09-23T03:15:50.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.424+0000 7fe466ffd640 1 --2- 172.21.15.154:0/1209279312 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe45006af30 0x7fe45006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:50.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.425+0000 7fe466ffd640 1 -- 172.21.15.154:0/1209279312 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe468089c80 con 0x7fe4700cae90 2023-09-23T03:15:50.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.425+0000 7fe46e59c640 1 --2- 172.21.15.154:0/1209279312 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe45006af30 0x7fe45006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:50.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.425+0000 7fe46e59c640 1 --2- 172.21.15.154:0/1209279312 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe45006af30 0x7fe45006d3f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe460009a80 tx=0x7fe460008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:50.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.429+0000 7fe466ffd640 1 -- 172.21.15.154:0/1209279312 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fe4680549d0 con 0x7fe4700cae90 2023-09-23T03:15:50.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.600+0000 7fe474e10640 1 -- 172.21.15.154:0/1209279312 --> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe47006c3f0 con 0x7fe45006af30 2023-09-23T03:15:50.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.632+0000 7fe466ffd640 1 -- 172.21.15.154:0/1209279312 <== mgr.4110 v2:172.21.15.154:6832/2422652104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345152 (secure 0 0 0) 0x7fe460013a20 con 0x7fe45006af30 2023-09-23T03:15:50.639 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:15:50.641 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2023-09-23T03:15:50.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.638+0000 7fe474e10640 1 -- 172.21.15.154:0/1209279312 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe45006af30 msgr2=0x7fe45006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:50.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.638+0000 7fe474e10640 1 --2- 172.21.15.154:0/1209279312 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe45006af30 0x7fe45006d3f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe460009a80 tx=0x7fe460008040 comp rx=0 tx=0).stop 2023-09-23T03:15:50.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.638+0000 7fe474e10640 1 -- 172.21.15.154:0/1209279312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4700cae90 msgr2=0x7fe4700cb310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:50.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.638+0000 7fe474e10640 1 --2- 172.21.15.154:0/1209279312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4700cae90 0x7fe4700cb310 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe468000f80 tx=0x7fe46800b4a0 comp rx=0 tx=0).stop 2023-09-23T03:15:50.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.638+0000 7fe474e10640 1 -- 172.21.15.154:0/1209279312 shutdown_connections 2023-09-23T03:15:50.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.638+0000 7fe474e10640 1 --2- 172.21.15.154:0/1209279312 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe45006af30 0x7fe45006d3f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:50.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.638+0000 7fe474e10640 1 --2- 172.21.15.154:0/1209279312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4700cae90 0x7fe4700cb310 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:50.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.638+0000 7fe474e10640 1 --2- 172.21.15.154:0/1209279312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4700cbe20 0x7fe4701f19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:50.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.638+0000 7fe474e10640 1 --2- 172.21.15.154:0/1209279312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe45c003ef0 0x7fe470155f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:50.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.638+0000 7fe474e10640 1 -- 172.21.15.154:0/1209279312 >> 172.21.15.154:0/1209279312 conn(0x7fe47006a0f0 msgr2=0x7fe4700adf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:50.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.639+0000 7fe474e10640 1 -- 172.21.15.154:0/1209279312 shutdown_connections 2023-09-23T03:15:50.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.639+0000 7fe474e10640 1 -- 172.21.15.154:0/1209279312 wait complete. 2023-09-23T03:15:50.659 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2023-09-23T03:15:49.172850+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219416,"kb_used_data":3204,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755304,"statfs":{"total":773094113280,"available":772869431296,"internally_reserved":0,"allocated":3280896,"data_stored":1987235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":221236794},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005051"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.303622+0000","last_change":"2023-09-23T03:15:20.261808+0000","last_active":"2023-09-23T03:15:25.303622+0000","last_peered":"2023-09-23T03:15:25.303622+0000","last_clean":"2023-09-23T03:15:25.303622+0000","last_became_active":"2023-09-23T03:15:20.261512+0000","last_became_peered":"2023-09-23T03:15:20.261512+0000","last_unstale":"2023-09-23T03:15:25.303622+0000","last_undegraded":"2023-09-23T03:15:25.303622+0000","last_fullsized":"2023-09-23T03:15:25.303622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:28:08.970374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.296813+0000","last_change":"2023-09-23T03:15:21.165202+0000","last_active":"2023-09-23T03:15:25.296813+0000","last_peered":"2023-09-23T03:15:25.296813+0000","last_clean":"2023-09-23T03:15:25.296813+0000","last_became_active":"2023-09-23T03:15:21.164933+0000","last_became_peered":"2023-09-23T03:15:21.164933+0000","last_unstale":"2023-09-23T03:15:25.296813+0000","last_undegraded":"2023-09-23T03:15:25.296813+0000","last_fullsized":"2023-09-23T03:15:25.296813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:29:14.943541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.485848+0000","last_change":"2023-09-23T03:15:25.453212+0000","last_active":"2023-09-23T03:15:26.485848+0000","last_peered":"2023-09-23T03:15:26.485848+0000","last_clean":"2023-09-23T03:15:26.485848+0000","last_became_active":"2023-09-23T03:15:25.452429+0000","last_became_peered":"2023-09-23T03:15:25.452429+0000","last_unstale":"2023-09-23T03:15:26.485848+0000","last_undegraded":"2023-09-23T03:15:26.485848+0000","last_fullsized":"2023-09-23T03:15:26.485848+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:28:19.168744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.297194+0000","last_change":"2023-09-23T03:15:20.241249+0000","last_active":"2023-09-23T03:15:25.297194+0000","last_peered":"2023-09-23T03:15:25.297194+0000","last_clean":"2023-09-23T03:15:25.297194+0000","last_became_active":"2023-09-23T03:15:20.240943+0000","last_became_peered":"2023-09-23T03:15:20.240943+0000","last_unstale":"2023-09-23T03:15:25.297194+0000","last_undegraded":"2023-09-23T03:15:25.297194+0000","last_fullsized":"2023-09-23T03:15:25.297194+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:05:48.619034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.303332+0000","last_change":"2023-09-23T03:15:21.172718+0000","last_active":"2023-09-23T03:15:25.303332+0000","last_peered":"2023-09-23T03:15:25.303332+0000","last_clean":"2023-09-23T03:15:25.303332+0000","last_became_active":"2023-09-23T03:15:21.172296+0000","last_became_peered":"2023-09-23T03:15:21.172296+0000","last_unstale":"2023-09-23T03:15:25.303332+0000","last_undegraded":"2023-09-23T03:15:25.303332+0000","last_fullsized":"2023-09-23T03:15:25.303332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:30:31.951544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.137218+0000","last_change":"2023-09-23T03:15:25.330094+0000","last_active":"2023-09-23T03:15:26.137218+0000","last_peered":"2023-09-23T03:15:26.137218+0000","last_clean":"2023-09-23T03:15:26.137218+0000","last_became_active":"2023-09-23T03:15:25.329632+0000","last_became_peered":"2023-09-23T03:15:25.329632+0000","last_unstale":"2023-09-23T03:15:26.137218+0000","last_undegraded":"2023-09-23T03:15:26.137218+0000","last_fullsized":"2023-09-23T03:15:26.137218+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:21:31.656499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.445726+0000","last_change":"2023-09-23T03:15:20.261800+0000","last_active":"2023-09-23T03:15:25.445726+0000","last_peered":"2023-09-23T03:15:25.445726+0000","last_clean":"2023-09-23T03:15:25.445726+0000","last_became_active":"2023-09-23T03:15:20.261459+0000","last_became_peered":"2023-09-23T03:15:20.261459+0000","last_unstale":"2023-09-23T03:15:25.445726+0000","last_undegraded":"2023-09-23T03:15:25.445726+0000","last_fullsized":"2023-09-23T03:15:25.445726+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:30:30.939479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315185+0000","last_change":"2023-09-23T03:15:21.172135+0000","last_active":"2023-09-23T03:15:25.315185+0000","last_peered":"2023-09-23T03:15:25.315185+0000","last_clean":"2023-09-23T03:15:25.315185+0000","last_became_active":"2023-09-23T03:15:21.169373+0000","last_became_peered":"2023-09-23T03:15:21.169373+0000","last_unstale":"2023-09-23T03:15:25.315185+0000","last_undegraded":"2023-09-23T03:15:25.315185+0000","last_fullsized":"2023-09-23T03:15:25.315185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:52:02.570741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.139181+0000","last_change":"2023-09-23T03:15:25.328735+0000","last_active":"2023-09-23T03:15:26.139181+0000","last_peered":"2023-09-23T03:15:26.139181+0000","last_clean":"2023-09-23T03:15:26.139181+0000","last_became_active":"2023-09-23T03:15:25.328423+0000","last_became_peered":"2023-09-23T03:15:25.328423+0000","last_unstale":"2023-09-23T03:15:26.139181+0000","last_undegraded":"2023-09-23T03:15:26.139181+0000","last_fullsized":"2023-09-23T03:15:26.139181+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:25:57.997547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.441936+0000","last_change":"2023-09-23T03:15:20.264630+0000","last_active":"2023-09-23T03:15:25.441936+0000","last_peered":"2023-09-23T03:15:25.441936+0000","last_clean":"2023-09-23T03:15:25.441936+0000","last_became_active":"2023-09-23T03:15:20.264209+0000","last_became_peered":"2023-09-23T03:15:20.264209+0000","last_unstale":"2023-09-23T03:15:25.441936+0000","last_undegraded":"2023-09-23T03:15:25.441936+0000","last_fullsized":"2023-09-23T03:15:25.441936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:41:37.505547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315447+0000","last_change":"2023-09-23T03:15:21.176949+0000","last_active":"2023-09-23T03:15:25.315447+0000","last_peered":"2023-09-23T03:15:25.315447+0000","last_clean":"2023-09-23T03:15:25.315447+0000","last_became_active":"2023-09-23T03:15:21.176593+0000","last_became_peered":"2023-09-23T03:15:21.176593+0000","last_unstale":"2023-09-23T03:15:25.315447+0000","last_undegraded":"2023-09-23T03:15:25.315447+0000","last_fullsized":"2023-09-23T03:15:25.315447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:28:31.810059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.138092+0000","last_change":"2023-09-23T03:15:25.324657+0000","last_active":"2023-09-23T03:15:26.138092+0000","last_peered":"2023-09-23T03:15:26.138092+0000","last_clean":"2023-09-23T03:15:26.138092+0000","last_became_active":"2023-09-23T03:15:25.324107+0000","last_became_peered":"2023-09-23T03:15:25.324107+0000","last_unstale":"2023-09-23T03:15:26.138092+0000","last_undegraded":"2023-09-23T03:15:26.138092+0000","last_fullsized":"2023-09-23T03:15:26.138092+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:30:41.725709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.306037+0000","last_change":"2023-09-23T03:15:21.148790+0000","last_active":"2023-09-23T03:15:25.306037+0000","last_peered":"2023-09-23T03:15:25.306037+0000","last_clean":"2023-09-23T03:15:25.306037+0000","last_became_active":"2023-09-23T03:15:21.148165+0000","last_became_peered":"2023-09-23T03:15:21.148165+0000","last_unstale":"2023-09-23T03:15:25.306037+0000","last_undegraded":"2023-09-23T03:15:25.306037+0000","last_fullsized":"2023-09-23T03:15:25.306037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:22:07.336440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.442912+0000","last_change":"2023-09-23T03:15:20.262080+0000","last_active":"2023-09-23T03:15:25.442912+0000","last_peered":"2023-09-23T03:15:25.442912+0000","last_clean":"2023-09-23T03:15:25.442912+0000","last_became_active":"2023-09-23T03:15:20.261391+0000","last_became_peered":"2023-09-23T03:15:20.261391+0000","last_unstale":"2023-09-23T03:15:25.442912+0000","last_undegraded":"2023-09-23T03:15:25.442912+0000","last_fullsized":"2023-09-23T03:15:25.442912+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:30:36.640364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.140544+0000","last_change":"2023-09-23T03:15:25.324180+0000","last_active":"2023-09-23T03:15:26.140544+0000","last_peered":"2023-09-23T03:15:26.140544+0000","last_clean":"2023-09-23T03:15:26.140544+0000","last_became_active":"2023-09-23T03:15:25.323694+0000","last_became_peered":"2023-09-23T03:15:25.323694+0000","last_unstale":"2023-09-23T03:15:26.140544+0000","last_undegraded":"2023-09-23T03:15:26.140544+0000","last_fullsized":"2023-09-23T03:15:26.140544+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:42:41.516665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.296569+0000","last_change":"2023-09-23T03:15:21.168584+0000","last_active":"2023-09-23T03:15:25.296569+0000","last_peered":"2023-09-23T03:15:25.296569+0000","last_clean":"2023-09-23T03:15:25.296569+0000","last_became_active":"2023-09-23T03:15:21.168238+0000","last_became_peered":"2023-09-23T03:15:21.168238+0000","last_unstale":"2023-09-23T03:15:25.296569+0000","last_undegraded":"2023-09-23T03:15:25.296569+0000","last_fullsized":"2023-09-23T03:15:25.296569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:21:27.583218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.311312+0000","last_change":"2023-09-23T03:15:20.251584+0000","last_active":"2023-09-23T03:15:25.311312+0000","last_peered":"2023-09-23T03:15:25.311312+0000","last_clean":"2023-09-23T03:15:25.311312+0000","last_became_active":"2023-09-23T03:15:20.250687+0000","last_became_peered":"2023-09-23T03:15:20.250687+0000","last_unstale":"2023-09-23T03:15:25.311312+0000","last_undegraded":"2023-09-23T03:15:25.311312+0000","last_fullsized":"2023-09-23T03:15:25.311312+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:01:03.998670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.138838+0000","last_change":"2023-09-23T03:15:25.329231+0000","last_active":"2023-09-23T03:15:26.138838+0000","last_peered":"2023-09-23T03:15:26.138838+0000","last_clean":"2023-09-23T03:15:26.138838+0000","last_became_active":"2023-09-23T03:15:25.328825+0000","last_became_peered":"2023-09-23T03:15:25.328825+0000","last_unstale":"2023-09-23T03:15:26.138838+0000","last_undegraded":"2023-09-23T03:15:26.138838+0000","last_fullsized":"2023-09-23T03:15:26.138838+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:29:19.016822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.295849+0000","last_change":"2023-09-23T03:15:21.162632+0000","last_active":"2023-09-23T03:15:25.295849+0000","last_peered":"2023-09-23T03:15:25.295849+0000","last_clean":"2023-09-23T03:15:25.295849+0000","last_became_active":"2023-09-23T03:15:21.161891+0000","last_became_peered":"2023-09-23T03:15:21.161891+0000","last_unstale":"2023-09-23T03:15:25.295849+0000","last_undegraded":"2023-09-23T03:15:25.295849+0000","last_fullsized":"2023-09-23T03:15:25.295849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:39:10.586582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.311128+0000","last_change":"2023-09-23T03:15:20.265489+0000","last_active":"2023-09-23T03:15:25.311128+0000","last_peered":"2023-09-23T03:15:25.311128+0000","last_clean":"2023-09-23T03:15:25.311128+0000","last_became_active":"2023-09-23T03:15:20.265295+0000","last_became_peered":"2023-09-23T03:15:20.265295+0000","last_unstale":"2023-09-23T03:15:25.311128+0000","last_undegraded":"2023-09-23T03:15:25.311128+0000","last_fullsized":"2023-09-23T03:15:25.311128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:06:15.942585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.619606+0000","last_change":"2023-09-23T03:15:25.450826+0000","last_active":"2023-09-23T03:15:26.619606+0000","last_peered":"2023-09-23T03:15:26.619606+0000","last_clean":"2023-09-23T03:15:26.619606+0000","last_became_active":"2023-09-23T03:15:25.449562+0000","last_became_peered":"2023-09-23T03:15:25.449562+0000","last_unstale":"2023-09-23T03:15:26.619606+0000","last_undegraded":"2023-09-23T03:15:26.619606+0000","last_fullsized":"2023-09-23T03:15:26.619606+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:02:13.107703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.310119+0000","last_change":"2023-09-23T03:15:21.151987+0000","last_active":"2023-09-23T03:15:25.310119+0000","last_peered":"2023-09-23T03:15:25.310119+0000","last_clean":"2023-09-23T03:15:25.310119+0000","last_became_active":"2023-09-23T03:15:21.151421+0000","last_became_peered":"2023-09-23T03:15:21.151421+0000","last_unstale":"2023-09-23T03:15:25.310119+0000","last_undegraded":"2023-09-23T03:15:25.310119+0000","last_fullsized":"2023-09-23T03:15:25.310119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:32:42.587095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.441395+0000","last_change":"2023-09-23T03:15:20.262324+0000","last_active":"2023-09-23T03:15:25.441395+0000","last_peered":"2023-09-23T03:15:25.441395+0000","last_clean":"2023-09-23T03:15:25.441395+0000","last_became_active":"2023-09-23T03:15:20.261456+0000","last_became_peered":"2023-09-23T03:15:20.261456+0000","last_unstale":"2023-09-23T03:15:25.441395+0000","last_undegraded":"2023-09-23T03:15:25.441395+0000","last_fullsized":"2023-09-23T03:15:25.441395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:31:40.244446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.139092+0000","last_change":"2023-09-23T03:15:25.329961+0000","last_active":"2023-09-23T03:15:26.139092+0000","last_peered":"2023-09-23T03:15:26.139092+0000","last_clean":"2023-09-23T03:15:26.139092+0000","last_became_active":"2023-09-23T03:15:25.329525+0000","last_became_peered":"2023-09-23T03:15:25.329525+0000","last_unstale":"2023-09-23T03:15:26.139092+0000","last_undegraded":"2023-09-23T03:15:26.139092+0000","last_fullsized":"2023-09-23T03:15:26.139092+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:39:14.659863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.304127+0000","last_change":"2023-09-23T03:15:21.160274+0000","last_active":"2023-09-23T03:15:25.304127+0000","last_peered":"2023-09-23T03:15:25.304127+0000","last_clean":"2023-09-23T03:15:25.304127+0000","last_became_active":"2023-09-23T03:15:21.160110+0000","last_became_peered":"2023-09-23T03:15:21.160110+0000","last_unstale":"2023-09-23T03:15:25.304127+0000","last_undegraded":"2023-09-23T03:15:25.304127+0000","last_fullsized":"2023-09-23T03:15:25.304127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:15:19.584603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.313804+0000","last_change":"2023-09-23T03:15:20.265791+0000","last_active":"2023-09-23T03:15:25.313804+0000","last_peered":"2023-09-23T03:15:25.313804+0000","last_clean":"2023-09-23T03:15:25.313804+0000","last_became_active":"2023-09-23T03:15:20.265429+0000","last_became_peered":"2023-09-23T03:15:20.265429+0000","last_unstale":"2023-09-23T03:15:25.313804+0000","last_undegraded":"2023-09-23T03:15:25.313804+0000","last_fullsized":"2023-09-23T03:15:25.313804+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:05:48.619034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.138192+0000","last_change":"2023-09-23T03:15:25.451854+0000","last_active":"2023-09-23T03:15:26.138192+0000","last_peered":"2023-09-23T03:15:26.138192+0000","last_clean":"2023-09-23T03:15:26.138192+0000","last_became_active":"2023-09-23T03:15:25.450790+0000","last_became_peered":"2023-09-23T03:15:25.450790+0000","last_unstale":"2023-09-23T03:15:26.138192+0000","last_undegraded":"2023-09-23T03:15:26.138192+0000","last_fullsized":"2023-09-23T03:15:26.138192+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:28:35.883340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.297558+0000","last_change":"2023-09-23T03:15:21.163852+0000","last_active":"2023-09-23T03:15:25.297558+0000","last_peered":"2023-09-23T03:15:25.297558+0000","last_clean":"2023-09-23T03:15:25.297558+0000","last_became_active":"2023-09-23T03:15:21.163332+0000","last_became_peered":"2023-09-23T03:15:21.163332+0000","last_unstale":"2023-09-23T03:15:25.297558+0000","last_undegraded":"2023-09-23T03:15:25.297558+0000","last_fullsized":"2023-09-23T03:15:25.297558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:25:53.924266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.616670+0000","last_change":"2023-09-23T03:15:20.264120+0000","last_active":"2023-09-23T03:15:25.616670+0000","last_peered":"2023-09-23T03:15:25.616670+0000","last_clean":"2023-09-23T03:15:25.616670+0000","last_became_active":"2023-09-23T03:15:20.263598+0000","last_became_peered":"2023-09-23T03:15:20.263598+0000","last_unstale":"2023-09-23T03:15:25.616670+0000","last_undegraded":"2023-09-23T03:15:25.616670+0000","last_fullsized":"2023-09-23T03:15:25.616670+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:31:01.946245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.485429+0000","last_change":"2023-09-23T03:15:25.444557+0000","last_active":"2023-09-23T03:15:26.485429+0000","last_peered":"2023-09-23T03:15:26.485429+0000","last_clean":"2023-09-23T03:15:26.485429+0000","last_became_active":"2023-09-23T03:15:25.444002+0000","last_became_peered":"2023-09-23T03:15:25.444002+0000","last_unstale":"2023-09-23T03:15:26.485429+0000","last_undegraded":"2023-09-23T03:15:26.485429+0000","last_fullsized":"2023-09-23T03:15:26.485429+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:15:29.393607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.311253+0000","last_change":"2023-09-23T03:15:21.155177+0000","last_active":"2023-09-23T03:15:25.311253+0000","last_peered":"2023-09-23T03:15:25.311253+0000","last_clean":"2023-09-23T03:15:25.311253+0000","last_became_active":"2023-09-23T03:15:21.154788+0000","last_became_peered":"2023-09-23T03:15:21.154788+0000","last_unstale":"2023-09-23T03:15:25.311253+0000","last_undegraded":"2023-09-23T03:15:25.311253+0000","last_fullsized":"2023-09-23T03:15:25.311253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:51:00.098552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.446205+0000","last_change":"2023-09-23T03:15:20.261866+0000","last_active":"2023-09-23T03:15:25.446205+0000","last_peered":"2023-09-23T03:15:25.446205+0000","last_clean":"2023-09-23T03:15:25.446205+0000","last_became_active":"2023-09-23T03:15:20.261448+0000","last_became_peered":"2023-09-23T03:15:20.261448+0000","last_unstale":"2023-09-23T03:15:25.446205+0000","last_undegraded":"2023-09-23T03:15:25.446205+0000","last_fullsized":"2023-09-23T03:15:25.446205+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:15:18.572539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.619178+0000","last_change":"2023-09-23T03:15:25.443253+0000","last_active":"2023-09-23T03:15:26.619178+0000","last_peered":"2023-09-23T03:15:26.619178+0000","last_clean":"2023-09-23T03:15:26.619178+0000","last_became_active":"2023-09-23T03:15:25.442252+0000","last_became_peered":"2023-09-23T03:15:25.442252+0000","last_unstale":"2023-09-23T03:15:26.619178+0000","last_undegraded":"2023-09-23T03:15:26.619178+0000","last_fullsized":"2023-09-23T03:15:26.619178+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:53:25.508968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315540+0000","last_change":"2023-09-23T03:15:21.167597+0000","last_active":"2023-09-23T03:15:25.315540+0000","last_peered":"2023-09-23T03:15:25.315540+0000","last_clean":"2023-09-23T03:15:25.315540+0000","last_became_active":"2023-09-23T03:15:21.167306+0000","last_became_peered":"2023-09-23T03:15:21.167306+0000","last_unstale":"2023-09-23T03:15:25.315540+0000","last_undegraded":"2023-09-23T03:15:25.315540+0000","last_fullsized":"2023-09-23T03:15:25.315540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:25:53.924266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.311346+0000","last_change":"2023-09-23T03:15:20.254369+0000","last_active":"2023-09-23T03:15:25.311346+0000","last_peered":"2023-09-23T03:15:25.311346+0000","last_clean":"2023-09-23T03:15:25.311346+0000","last_became_active":"2023-09-23T03:15:20.251723+0000","last_became_peered":"2023-09-23T03:15:20.251723+0000","last_unstale":"2023-09-23T03:15:25.311346+0000","last_undegraded":"2023-09-23T03:15:25.311346+0000","last_fullsized":"2023-09-23T03:15:25.311346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:32:41.575031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.616591+0000","last_change":"2023-09-23T03:15:25.331003+0000","last_active":"2023-09-23T03:15:26.616591+0000","last_peered":"2023-09-23T03:15:26.616591+0000","last_clean":"2023-09-23T03:15:26.616591+0000","last_became_active":"2023-09-23T03:15:25.330351+0000","last_became_peered":"2023-09-23T03:15:25.330351+0000","last_unstale":"2023-09-23T03:15:26.616591+0000","last_undegraded":"2023-09-23T03:15:26.616591+0000","last_fullsized":"2023-09-23T03:15:26.616591+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:42:41.516665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.311073+0000","last_change":"2023-09-23T03:15:21.160153+0000","last_active":"2023-09-23T03:15:25.311073+0000","last_peered":"2023-09-23T03:15:25.311073+0000","last_clean":"2023-09-23T03:15:25.311073+0000","last_became_active":"2023-09-23T03:15:21.159780+0000","last_became_peered":"2023-09-23T03:15:21.159780+0000","last_unstale":"2023-09-23T03:15:25.311073+0000","last_undegraded":"2023-09-23T03:15:25.311073+0000","last_fullsized":"2023-09-23T03:15:25.311073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:31:41.256510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.317480+0000","last_change":"2023-09-23T03:15:20.258357+0000","last_active":"2023-09-23T03:15:25.317480+0000","last_peered":"2023-09-23T03:15:25.317480+0000","last_clean":"2023-09-23T03:15:25.317480+0000","last_became_active":"2023-09-23T03:15:20.258052+0000","last_became_peered":"2023-09-23T03:15:20.258052+0000","last_unstale":"2023-09-23T03:15:25.317480+0000","last_undegraded":"2023-09-23T03:15:25.317480+0000","last_fullsized":"2023-09-23T03:15:25.317480+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:01:03.998670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.618336+0000","last_change":"2023-09-23T03:15:25.326862+0000","last_active":"2023-09-23T03:15:26.618336+0000","last_peered":"2023-09-23T03:15:26.618336+0000","last_clean":"2023-09-23T03:15:26.618336+0000","last_became_active":"2023-09-23T03:15:25.326483+0000","last_became_peered":"2023-09-23T03:15:25.326483+0000","last_unstale":"2023-09-23T03:15:26.618336+0000","last_undegraded":"2023-09-23T03:15:26.618336+0000","last_fullsized":"2023-09-23T03:15:26.618336+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:52:06.644022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.310981+0000","last_change":"2023-09-23T03:15:21.160191+0000","last_active":"2023-09-23T03:15:25.310981+0000","last_peered":"2023-09-23T03:15:25.310981+0000","last_clean":"2023-09-23T03:15:25.310981+0000","last_became_active":"2023-09-23T03:15:21.159841+0000","last_became_peered":"2023-09-23T03:15:21.159841+0000","last_unstale":"2023-09-23T03:15:25.310981+0000","last_undegraded":"2023-09-23T03:15:25.310981+0000","last_fullsized":"2023-09-23T03:15:25.310981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:23:07.611380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.301033+0000","last_change":"2023-09-23T03:15:20.242692+0000","last_active":"2023-09-23T03:15:25.301033+0000","last_peered":"2023-09-23T03:15:25.301033+0000","last_clean":"2023-09-23T03:15:25.301033+0000","last_became_active":"2023-09-23T03:15:20.242383+0000","last_became_peered":"2023-09-23T03:15:20.242383+0000","last_unstale":"2023-09-23T03:15:25.301033+0000","last_undegraded":"2023-09-23T03:15:25.301033+0000","last_fullsized":"2023-09-23T03:15:25.301033+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:35:16.587399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.619952+0000","last_change":"2023-09-23T03:15:25.330531+0000","last_active":"2023-09-23T03:15:26.619952+0000","last_peered":"2023-09-23T03:15:26.619952+0000","last_clean":"2023-09-23T03:15:26.619952+0000","last_became_active":"2023-09-23T03:15:25.330145+0000","last_became_peered":"2023-09-23T03:15:25.330145+0000","last_unstale":"2023-09-23T03:15:26.619952+0000","last_undegraded":"2023-09-23T03:15:26.619952+0000","last_fullsized":"2023-09-23T03:15:26.619952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:44:24.602333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.302012+0000","last_change":"2023-09-23T03:15:21.152726+0000","last_active":"2023-09-23T03:15:25.302012+0000","last_peered":"2023-09-23T03:15:25.302012+0000","last_clean":"2023-09-23T03:15:25.302012+0000","last_became_active":"2023-09-23T03:15:21.151916+0000","last_became_peered":"2023-09-23T03:15:21.151916+0000","last_unstale":"2023-09-23T03:15:25.302012+0000","last_undegraded":"2023-09-23T03:15:25.302012+0000","last_fullsized":"2023-09-23T03:15:25.302012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:05:49.631098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.441189+0000","last_change":"2023-09-23T03:15:20.265672+0000","last_active":"2023-09-23T03:15:25.441189+0000","last_peered":"2023-09-23T03:15:25.441189+0000","last_clean":"2023-09-23T03:15:25.441189+0000","last_became_active":"2023-09-23T03:15:20.265172+0000","last_became_peered":"2023-09-23T03:15:20.265172+0000","last_unstale":"2023-09-23T03:15:25.441189+0000","last_undegraded":"2023-09-23T03:15:25.441189+0000","last_fullsized":"2023-09-23T03:15:25.441189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:28:08.970374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.620019+0000","last_change":"2023-09-23T03:15:25.442450+0000","last_active":"2023-09-23T03:15:26.620019+0000","last_peered":"2023-09-23T03:15:26.620019+0000","last_clean":"2023-09-23T03:15:26.620019+0000","last_became_active":"2023-09-23T03:15:25.441911+0000","last_became_peered":"2023-09-23T03:15:25.441911+0000","last_unstale":"2023-09-23T03:15:26.620019+0000","last_undegraded":"2023-09-23T03:15:26.620019+0000","last_fullsized":"2023-09-23T03:15:26.620019+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:58:07.013813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.295933+0000","last_change":"2023-09-23T03:15:21.171351+0000","last_active":"2023-09-23T03:15:25.295933+0000","last_peered":"2023-09-23T03:15:25.295933+0000","last_clean":"2023-09-23T03:15:25.295933+0000","last_became_active":"2023-09-23T03:15:21.170977+0000","last_became_peered":"2023-09-23T03:15:21.170977+0000","last_unstale":"2023-09-23T03:15:25.295933+0000","last_undegraded":"2023-09-23T03:15:25.295933+0000","last_fullsized":"2023-09-23T03:15:25.295933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:51:00.098552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.314530+0000","last_change":"2023-09-23T03:15:20.258137+0000","last_active":"2023-09-23T03:15:25.314530+0000","last_peered":"2023-09-23T03:15:25.314530+0000","last_clean":"2023-09-23T03:15:25.314530+0000","last_became_active":"2023-09-23T03:15:20.257772+0000","last_became_peered":"2023-09-23T03:15:20.257772+0000","last_unstale":"2023-09-23T03:15:25.314530+0000","last_undegraded":"2023-09-23T03:15:25.314530+0000","last_fullsized":"2023-09-23T03:15:25.314530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:52:01.548454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":169,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.441463+0000","last_change":"2023-09-23T03:15:11.042447+0000","last_active":"2023-09-23T03:15:25.441463+0000","last_peered":"2023-09-23T03:15:25.441463+0000","last_clean":"2023-09-23T03:15:25.441463+0000","last_became_active":"2023-09-23T03:15:11.040322+0000","last_became_peered":"2023-09-23T03:15:11.040322+0000","last_unstale":"2023-09-23T03:15:25.441463+0000","last_undegraded":"2023-09-23T03:15:25.441463+0000","last_fullsized":"2023-09-23T03:15:25.441463+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_clean_scrub_stamp":"2023-09-23T03:15:09.474842+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:18:36.331645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.138753+0000","last_change":"2023-09-23T03:15:25.449566+0000","last_active":"2023-09-23T03:15:26.138753+0000","last_peered":"2023-09-23T03:15:26.138753+0000","last_clean":"2023-09-23T03:15:26.138753+0000","last_became_active":"2023-09-23T03:15:25.448514+0000","last_became_peered":"2023-09-23T03:15:25.448514+0000","last_unstale":"2023-09-23T03:15:26.138753+0000","last_undegraded":"2023-09-23T03:15:26.138753+0000","last_fullsized":"2023-09-23T03:15:26.138753+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:52:06.644022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.442985+0000","last_change":"2023-09-23T03:15:21.180427+0000","last_active":"2023-09-23T03:15:25.442985+0000","last_peered":"2023-09-23T03:15:25.442985+0000","last_clean":"2023-09-23T03:15:25.442985+0000","last_became_active":"2023-09-23T03:15:21.179941+0000","last_became_peered":"2023-09-23T03:15:21.179941+0000","last_unstale":"2023-09-23T03:15:25.442985+0000","last_undegraded":"2023-09-23T03:15:25.442985+0000","last_fullsized":"2023-09-23T03:15:25.442985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:53:21.435687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.302809+0000","last_change":"2023-09-23T03:15:20.259280+0000","last_active":"2023-09-23T03:15:25.302809+0000","last_peered":"2023-09-23T03:15:25.302809+0000","last_clean":"2023-09-23T03:15:25.302809+0000","last_became_active":"2023-09-23T03:15:20.258990+0000","last_became_peered":"2023-09-23T03:15:20.258990+0000","last_unstale":"2023-09-23T03:15:25.302809+0000","last_undegraded":"2023-09-23T03:15:25.302809+0000","last_fullsized":"2023-09-23T03:15:25.302809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:32:41.575031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.137662+0000","last_change":"2023-09-23T03:15:25.448939+0000","last_active":"2023-09-23T03:15:26.137662+0000","last_peered":"2023-09-23T03:15:26.137662+0000","last_clean":"2023-09-23T03:15:26.137662+0000","last_became_active":"2023-09-23T03:15:25.448129+0000","last_became_peered":"2023-09-23T03:15:25.448129+0000","last_unstale":"2023-09-23T03:15:26.137662+0000","last_undegraded":"2023-09-23T03:15:26.137662+0000","last_fullsized":"2023-09-23T03:15:26.137662+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:42:41.516665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315261+0000","last_change":"2023-09-23T03:15:21.172056+0000","last_active":"2023-09-23T03:15:25.315261+0000","last_peered":"2023-09-23T03:15:25.315261+0000","last_clean":"2023-09-23T03:15:25.315261+0000","last_became_active":"2023-09-23T03:15:21.170669+0000","last_became_peered":"2023-09-23T03:15:21.170669+0000","last_unstale":"2023-09-23T03:15:25.315261+0000","last_undegraded":"2023-09-23T03:15:25.315261+0000","last_fullsized":"2023-09-23T03:15:25.315261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:02:09.034422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.312606+0000","last_change":"2023-09-23T03:15:20.254984+0000","last_active":"2023-09-23T03:15:25.312606+0000","last_peered":"2023-09-23T03:15:25.312606+0000","last_clean":"2023-09-23T03:15:25.312606+0000","last_became_active":"2023-09-23T03:15:20.254425+0000","last_became_peered":"2023-09-23T03:15:20.254425+0000","last_unstale":"2023-09-23T03:15:25.312606+0000","last_undegraded":"2023-09-23T03:15:25.312606+0000","last_fullsized":"2023-09-23T03:15:25.312606+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:13:21.067891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.619714+0000","last_change":"2023-09-23T03:15:25.324169+0000","last_active":"2023-09-23T03:15:26.619714+0000","last_peered":"2023-09-23T03:15:26.619714+0000","last_clean":"2023-09-23T03:15:26.619714+0000","last_became_active":"2023-09-23T03:15:25.323647+0000","last_became_peered":"2023-09-23T03:15:25.323647+0000","last_unstale":"2023-09-23T03:15:26.619714+0000","last_undegraded":"2023-09-23T03:15:26.619714+0000","last_fullsized":"2023-09-23T03:15:26.619714+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:44:24.602333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.446178+0000","last_change":"2023-09-23T03:15:21.175250+0000","last_active":"2023-09-23T03:15:25.446178+0000","last_peered":"2023-09-23T03:15:25.446178+0000","last_clean":"2023-09-23T03:15:25.446178+0000","last_became_active":"2023-09-23T03:15:21.174890+0000","last_became_peered":"2023-09-23T03:15:21.174890+0000","last_unstale":"2023-09-23T03:15:25.446178+0000","last_undegraded":"2023-09-23T03:15:25.446178+0000","last_fullsized":"2023-09-23T03:15:25.446178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:03:55.070789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.301471+0000","last_change":"2023-09-23T03:15:20.242773+0000","last_active":"2023-09-23T03:15:25.301471+0000","last_peered":"2023-09-23T03:15:25.301471+0000","last_clean":"2023-09-23T03:15:25.301471+0000","last_became_active":"2023-09-23T03:15:20.242466+0000","last_became_peered":"2023-09-23T03:15:20.242466+0000","last_unstale":"2023-09-23T03:15:25.301471+0000","last_undegraded":"2023-09-23T03:15:25.301471+0000","last_fullsized":"2023-09-23T03:15:25.301471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:52:01.548454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.617142+0000","last_change":"2023-09-23T03:15:25.452683+0000","last_active":"2023-09-23T03:15:26.617142+0000","last_peered":"2023-09-23T03:15:26.617142+0000","last_clean":"2023-09-23T03:15:26.617142+0000","last_became_active":"2023-09-23T03:15:25.451509+0000","last_became_peered":"2023-09-23T03:15:25.451509+0000","last_unstale":"2023-09-23T03:15:26.617142+0000","last_undegraded":"2023-09-23T03:15:26.617142+0000","last_fullsized":"2023-09-23T03:15:26.617142+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:03:59.144070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.443716+0000","last_change":"2023-09-23T03:15:21.177994+0000","last_active":"2023-09-23T03:15:25.443716+0000","last_peered":"2023-09-23T03:15:25.443716+0000","last_clean":"2023-09-23T03:15:25.443716+0000","last_became_active":"2023-09-23T03:15:21.177752+0000","last_became_peered":"2023-09-23T03:15:21.177752+0000","last_unstale":"2023-09-23T03:15:25.443716+0000","last_undegraded":"2023-09-23T03:15:25.443716+0000","last_fullsized":"2023-09-23T03:15:25.443716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:52:02.570741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.309999+0000","last_change":"2023-09-23T03:15:20.245089+0000","last_active":"2023-09-23T03:15:25.309999+0000","last_peered":"2023-09-23T03:15:25.309999+0000","last_clean":"2023-09-23T03:15:25.309999+0000","last_became_active":"2023-09-23T03:15:20.243281+0000","last_became_peered":"2023-09-23T03:15:20.243281+0000","last_unstale":"2023-09-23T03:15:25.309999+0000","last_undegraded":"2023-09-23T03:15:25.309999+0000","last_fullsized":"2023-09-23T03:15:25.309999+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:34:50.786275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.137112+0000","last_change":"2023-09-23T03:15:25.327837+0000","last_active":"2023-09-23T03:15:26.137112+0000","last_peered":"2023-09-23T03:15:26.137112+0000","last_clean":"2023-09-23T03:15:26.137112+0000","last_became_active":"2023-09-23T03:15:25.327245+0000","last_became_peered":"2023-09-23T03:15:25.327245+0000","last_unstale":"2023-09-23T03:15:26.137112+0000","last_undegraded":"2023-09-23T03:15:26.137112+0000","last_fullsized":"2023-09-23T03:15:26.137112+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:15:23.657884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.441989+0000","last_change":"2023-09-23T03:15:21.173588+0000","last_active":"2023-09-23T03:15:25.441989+0000","last_peered":"2023-09-23T03:15:25.441989+0000","last_clean":"2023-09-23T03:15:25.441989+0000","last_became_active":"2023-09-23T03:15:21.173284+0000","last_became_peered":"2023-09-23T03:15:21.173284+0000","last_unstale":"2023-09-23T03:15:25.441989+0000","last_undegraded":"2023-09-23T03:15:25.441989+0000","last_fullsized":"2023-09-23T03:15:25.441989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:43:35.707241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.302930+0000","last_change":"2023-09-23T03:15:20.249033+0000","last_active":"2023-09-23T03:15:25.302930+0000","last_peered":"2023-09-23T03:15:25.302930+0000","last_clean":"2023-09-23T03:15:25.302930+0000","last_became_active":"2023-09-23T03:15:20.248738+0000","last_became_peered":"2023-09-23T03:15:20.248738+0000","last_unstale":"2023-09-23T03:15:25.302930+0000","last_undegraded":"2023-09-23T03:15:25.302930+0000","last_fullsized":"2023-09-23T03:15:25.302930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:34:53.736954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.619712+0000","last_change":"2023-09-23T03:15:25.328800+0000","last_active":"2023-09-23T03:15:26.619712+0000","last_peered":"2023-09-23T03:15:26.619712+0000","last_clean":"2023-09-23T03:15:26.619712+0000","last_became_active":"2023-09-23T03:15:25.327349+0000","last_became_peered":"2023-09-23T03:15:25.327349+0000","last_unstale":"2023-09-23T03:15:26.619712+0000","last_undegraded":"2023-09-23T03:15:26.619712+0000","last_fullsized":"2023-09-23T03:15:26.619712+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:02:13.107703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.306109+0000","last_change":"2023-09-23T03:15:21.156710+0000","last_active":"2023-09-23T03:15:25.306109+0000","last_peered":"2023-09-23T03:15:25.306109+0000","last_clean":"2023-09-23T03:15:25.306109+0000","last_became_active":"2023-09-23T03:15:21.156344+0000","last_became_peered":"2023-09-23T03:15:21.156344+0000","last_unstale":"2023-09-23T03:15:25.306109+0000","last_undegraded":"2023-09-23T03:15:25.306109+0000","last_fullsized":"2023-09-23T03:15:25.306109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:01:05.010735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.307353+0000","last_change":"2023-09-23T03:15:20.254568+0000","last_active":"2023-09-23T03:15:25.307353+0000","last_peered":"2023-09-23T03:15:25.307353+0000","last_clean":"2023-09-23T03:15:25.307353+0000","last_became_active":"2023-09-23T03:15:20.254294+0000","last_became_peered":"2023-09-23T03:15:20.254294+0000","last_unstale":"2023-09-23T03:15:25.307353+0000","last_undegraded":"2023-09-23T03:15:25.307353+0000","last_fullsized":"2023-09-23T03:15:25.307353+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:32:41.575031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.487038+0000","last_change":"2023-09-23T03:15:25.444588+0000","last_active":"2023-09-23T03:15:26.487038+0000","last_peered":"2023-09-23T03:15:26.487038+0000","last_clean":"2023-09-23T03:15:26.487038+0000","last_became_active":"2023-09-23T03:15:25.443847+0000","last_became_peered":"2023-09-23T03:15:25.443847+0000","last_unstale":"2023-09-23T03:15:26.487038+0000","last_undegraded":"2023-09-23T03:15:26.487038+0000","last_fullsized":"2023-09-23T03:15:26.487038+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:54:37.479094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.441909+0000","last_change":"2023-09-23T03:15:21.172873+0000","last_active":"2023-09-23T03:15:25.441909+0000","last_peered":"2023-09-23T03:15:25.441909+0000","last_clean":"2023-09-23T03:15:25.441909+0000","last_became_active":"2023-09-23T03:15:21.172552+0000","last_became_peered":"2023-09-23T03:15:21.172552+0000","last_unstale":"2023-09-23T03:15:25.441909+0000","last_undegraded":"2023-09-23T03:15:25.441909+0000","last_fullsized":"2023-09-23T03:15:25.441909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:42:37.443384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.317383+0000","last_change":"2023-09-23T03:15:20.257041+0000","last_active":"2023-09-23T03:15:25.317383+0000","last_peered":"2023-09-23T03:15:25.317383+0000","last_clean":"2023-09-23T03:15:25.317383+0000","last_became_active":"2023-09-23T03:15:20.256737+0000","last_became_peered":"2023-09-23T03:15:20.256737+0000","last_unstale":"2023-09-23T03:15:25.317383+0000","last_undegraded":"2023-09-23T03:15:25.317383+0000","last_fullsized":"2023-09-23T03:15:25.317383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:10:06.381357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.486900+0000","last_change":"2023-09-23T03:15:25.445742+0000","last_active":"2023-09-23T03:15:26.486900+0000","last_peered":"2023-09-23T03:15:26.486900+0000","last_clean":"2023-09-23T03:15:26.486900+0000","last_became_active":"2023-09-23T03:15:25.445297+0000","last_became_peered":"2023-09-23T03:15:25.445297+0000","last_unstale":"2023-09-23T03:15:26.486900+0000","last_undegraded":"2023-09-23T03:15:26.486900+0000","last_fullsized":"2023-09-23T03:15:26.486900+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:36:02.944327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.301638+0000","last_change":"2023-09-23T03:15:21.166987+0000","last_active":"2023-09-23T03:15:25.301638+0000","last_peered":"2023-09-23T03:15:25.301638+0000","last_clean":"2023-09-23T03:15:25.301638+0000","last_became_active":"2023-09-23T03:15:21.166575+0000","last_became_peered":"2023-09-23T03:15:21.166575+0000","last_unstale":"2023-09-23T03:15:25.301638+0000","last_undegraded":"2023-09-23T03:15:25.301638+0000","last_fullsized":"2023-09-23T03:15:25.301638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:10:07.393421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.305489+0000","last_change":"2023-09-23T03:15:20.253573+0000","last_active":"2023-09-23T03:15:25.305489+0000","last_peered":"2023-09-23T03:15:25.305489+0000","last_clean":"2023-09-23T03:15:25.305489+0000","last_became_active":"2023-09-23T03:15:20.253228+0000","last_became_peered":"2023-09-23T03:15:20.253228+0000","last_unstale":"2023-09-23T03:15:25.305489+0000","last_undegraded":"2023-09-23T03:15:25.305489+0000","last_fullsized":"2023-09-23T03:15:25.305489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:35:16.587399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.619960+0000","last_change":"2023-09-23T03:15:25.453715+0000","last_active":"2023-09-23T03:15:26.619960+0000","last_peered":"2023-09-23T03:15:26.619960+0000","last_clean":"2023-09-23T03:15:26.619960+0000","last_became_active":"2023-09-23T03:15:25.452979+0000","last_became_peered":"2023-09-23T03:15:25.452979+0000","last_unstale":"2023-09-23T03:15:26.619960+0000","last_undegraded":"2023-09-23T03:15:26.619960+0000","last_fullsized":"2023-09-23T03:15:26.619960+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:15:29.393607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.442638+0000","last_change":"2023-09-23T03:15:21.169871+0000","last_active":"2023-09-23T03:15:25.442638+0000","last_peered":"2023-09-23T03:15:25.442638+0000","last_clean":"2023-09-23T03:15:25.442638+0000","last_became_active":"2023-09-23T03:15:21.169539+0000","last_became_peered":"2023-09-23T03:15:21.169539+0000","last_unstale":"2023-09-23T03:15:25.442638+0000","last_undegraded":"2023-09-23T03:15:25.442638+0000","last_fullsized":"2023-09-23T03:15:25.442638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:44:20.529052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:40.246064+0000","last_change":"2023-09-23T03:15:20.252281+0000","last_active":"2023-09-23T03:15:40.246064+0000","last_peered":"2023-09-23T03:15:40.246064+0000","last_clean":"2023-09-23T03:15:40.246064+0000","last_became_active":"2023-09-23T03:15:20.251982+0000","last_became_peered":"2023-09-23T03:15:20.251982+0000","last_unstale":"2023-09-23T03:15:40.246064+0000","last_undegraded":"2023-09-23T03:15:40.246064+0000","last_fullsized":"2023-09-23T03:15:40.246064+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:35:16.587399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.616880+0000","last_change":"2023-09-23T03:15:25.325976+0000","last_active":"2023-09-23T03:15:26.616880+0000","last_peered":"2023-09-23T03:15:26.616880+0000","last_clean":"2023-09-23T03:15:26.616880+0000","last_became_active":"2023-09-23T03:15:25.325613+0000","last_became_peered":"2023-09-23T03:15:25.325613+0000","last_unstale":"2023-09-23T03:15:26.616880+0000","last_undegraded":"2023-09-23T03:15:26.616880+0000","last_fullsized":"2023-09-23T03:15:26.616880+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:53:25.508968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.309427+0000","last_change":"2023-09-23T03:15:21.166420+0000","last_active":"2023-09-23T03:15:25.309427+0000","last_peered":"2023-09-23T03:15:25.309427+0000","last_clean":"2023-09-23T03:15:25.309427+0000","last_became_active":"2023-09-23T03:15:21.166086+0000","last_became_peered":"2023-09-23T03:15:21.166086+0000","last_unstale":"2023-09-23T03:15:25.309427+0000","last_undegraded":"2023-09-23T03:15:25.309427+0000","last_fullsized":"2023-09-23T03:15:25.309427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:13:22.079955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.312418+0000","last_change":"2023-09-23T03:15:20.263920+0000","last_active":"2023-09-23T03:15:25.312418+0000","last_peered":"2023-09-23T03:15:25.312418+0000","last_clean":"2023-09-23T03:15:25.312418+0000","last_became_active":"2023-09-23T03:15:20.263623+0000","last_became_peered":"2023-09-23T03:15:20.263623+0000","last_unstale":"2023-09-23T03:15:25.312418+0000","last_undegraded":"2023-09-23T03:15:25.312418+0000","last_fullsized":"2023-09-23T03:15:25.312418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:28:08.970374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.139553+0000","last_change":"2023-09-23T03:15:25.329301+0000","last_active":"2023-09-23T03:15:26.139553+0000","last_peered":"2023-09-23T03:15:26.139553+0000","last_clean":"2023-09-23T03:15:26.139553+0000","last_became_active":"2023-09-23T03:15:25.328559+0000","last_became_peered":"2023-09-23T03:15:25.328559+0000","last_unstale":"2023-09-23T03:15:26.139553+0000","last_undegraded":"2023-09-23T03:15:26.139553+0000","last_fullsized":"2023-09-23T03:15:26.139553+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:43:39.780522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.303426+0000","last_change":"2023-09-23T03:15:21.172339+0000","last_active":"2023-09-23T03:15:25.303426+0000","last_peered":"2023-09-23T03:15:25.303426+0000","last_clean":"2023-09-23T03:15:25.303426+0000","last_became_active":"2023-09-23T03:15:21.172034+0000","last_became_peered":"2023-09-23T03:15:21.172034+0000","last_unstale":"2023-09-23T03:15:25.303426+0000","last_undegraded":"2023-09-23T03:15:25.303426+0000","last_fullsized":"2023-09-23T03:15:25.303426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:21:27.583218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315333+0000","last_change":"2023-09-23T03:15:20.259342+0000","last_active":"2023-09-23T03:15:25.315333+0000","last_peered":"2023-09-23T03:15:25.315333+0000","last_clean":"2023-09-23T03:15:25.315333+0000","last_became_active":"2023-09-23T03:15:20.258990+0000","last_became_peered":"2023-09-23T03:15:20.258990+0000","last_unstale":"2023-09-23T03:15:25.315333+0000","last_undegraded":"2023-09-23T03:15:25.315333+0000","last_fullsized":"2023-09-23T03:15:25.315333+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:50:59.086488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.485843+0000","last_change":"2023-09-23T03:15:25.322201+0000","last_active":"2023-09-23T03:15:26.485843+0000","last_peered":"2023-09-23T03:15:26.485843+0000","last_clean":"2023-09-23T03:15:26.485843+0000","last_became_active":"2023-09-23T03:15:25.321486+0000","last_became_peered":"2023-09-23T03:15:25.321486+0000","last_unstale":"2023-09-23T03:15:26.485843+0000","last_undegraded":"2023-09-23T03:15:26.485843+0000","last_fullsized":"2023-09-23T03:15:26.485843+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:20:20.193299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315253+0000","last_change":"2023-09-23T03:15:21.175465+0000","last_active":"2023-09-23T03:15:25.315253+0000","last_peered":"2023-09-23T03:15:25.315253+0000","last_clean":"2023-09-23T03:15:25.315253+0000","last_became_active":"2023-09-23T03:15:21.171773+0000","last_became_peered":"2023-09-23T03:15:21.171773+0000","last_unstale":"2023-09-23T03:15:25.315253+0000","last_undegraded":"2023-09-23T03:15:25.315253+0000","last_fullsized":"2023-09-23T03:15:25.315253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:39:10.586582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.313666+0000","last_change":"2023-09-23T03:15:20.259404+0000","last_active":"2023-09-23T03:15:25.313666+0000","last_peered":"2023-09-23T03:15:25.313666+0000","last_clean":"2023-09-23T03:15:25.313666+0000","last_became_active":"2023-09-23T03:15:20.259013+0000","last_became_peered":"2023-09-23T03:15:20.259013+0000","last_unstale":"2023-09-23T03:15:25.313666+0000","last_undegraded":"2023-09-23T03:15:25.313666+0000","last_fullsized":"2023-09-23T03:15:25.313666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:41:37.505547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.485703+0000","last_change":"2023-09-23T03:15:25.452370+0000","last_active":"2023-09-23T03:15:26.485703+0000","last_peered":"2023-09-23T03:15:26.485703+0000","last_clean":"2023-09-23T03:15:26.485703+0000","last_became_active":"2023-09-23T03:15:25.451873+0000","last_became_peered":"2023-09-23T03:15:25.451873+0000","last_unstale":"2023-09-23T03:15:26.485703+0000","last_undegraded":"2023-09-23T03:15:26.485703+0000","last_fullsized":"2023-09-23T03:15:26.485703+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:58:52.853121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.619356+0000","last_change":"2023-09-23T03:15:25.452394+0000","last_active":"2023-09-23T03:15:26.619356+0000","last_peered":"2023-09-23T03:15:26.619356+0000","last_clean":"2023-09-23T03:15:26.619356+0000","last_became_active":"2023-09-23T03:15:25.451251+0000","last_became_peered":"2023-09-23T03:15:25.451251+0000","last_unstale":"2023-09-23T03:15:26.619356+0000","last_undegraded":"2023-09-23T03:15:26.619356+0000","last_fullsized":"2023-09-23T03:15:26.619356+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:52:06.644022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.310954+0000","last_change":"2023-09-23T03:15:21.170461+0000","last_active":"2023-09-23T03:15:25.310954+0000","last_peered":"2023-09-23T03:15:25.310954+0000","last_clean":"2023-09-23T03:15:25.310954+0000","last_became_active":"2023-09-23T03:15:21.170095+0000","last_became_peered":"2023-09-23T03:15:21.170095+0000","last_unstale":"2023-09-23T03:15:25.310954+0000","last_undegraded":"2023-09-23T03:15:25.310954+0000","last_fullsized":"2023-09-23T03:15:25.310954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:30:31.951544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.314895+0000","last_change":"2023-09-23T03:15:20.257662+0000","last_active":"2023-09-23T03:15:25.314895+0000","last_peered":"2023-09-23T03:15:25.314895+0000","last_clean":"2023-09-23T03:15:25.314895+0000","last_became_active":"2023-09-23T03:15:20.257376+0000","last_became_peered":"2023-09-23T03:15:20.257376+0000","last_unstale":"2023-09-23T03:15:25.314895+0000","last_undegraded":"2023-09-23T03:15:25.314895+0000","last_fullsized":"2023-09-23T03:15:25.314895+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:06:15.942585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.138746+0000","last_change":"2023-09-23T03:15:25.326388+0000","last_active":"2023-09-23T03:15:26.138746+0000","last_peered":"2023-09-23T03:15:26.138746+0000","last_clean":"2023-09-23T03:15:26.138746+0000","last_became_active":"2023-09-23T03:15:25.326018+0000","last_became_peered":"2023-09-23T03:15:25.326018+0000","last_unstale":"2023-09-23T03:15:26.138746+0000","last_undegraded":"2023-09-23T03:15:26.138746+0000","last_fullsized":"2023-09-23T03:15:26.138746+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:53:25.508968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.295747+0000","last_change":"2023-09-23T03:15:21.164139+0000","last_active":"2023-09-23T03:15:25.295747+0000","last_peered":"2023-09-23T03:15:25.295747+0000","last_clean":"2023-09-23T03:15:25.295747+0000","last_became_active":"2023-09-23T03:15:21.162300+0000","last_became_peered":"2023-09-23T03:15:21.162300+0000","last_unstale":"2023-09-23T03:15:25.295747+0000","last_undegraded":"2023-09-23T03:15:25.295747+0000","last_fullsized":"2023-09-23T03:15:25.295747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:22:07.336440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.298007+0000","last_change":"2023-09-23T03:15:20.256962+0000","last_active":"2023-09-23T03:15:25.298007+0000","last_peered":"2023-09-23T03:15:25.298007+0000","last_clean":"2023-09-23T03:15:25.298007+0000","last_became_active":"2023-09-23T03:15:20.256686+0000","last_became_peered":"2023-09-23T03:15:20.256686+0000","last_unstale":"2023-09-23T03:15:25.298007+0000","last_undegraded":"2023-09-23T03:15:25.298007+0000","last_fullsized":"2023-09-23T03:15:25.298007+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:06:15.942585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.314864+0000","last_change":"2023-09-23T03:15:20.255630+0000","last_active":"2023-09-23T03:15:25.314864+0000","last_peered":"2023-09-23T03:15:25.314864+0000","last_clean":"2023-09-23T03:15:25.314864+0000","last_became_active":"2023-09-23T03:15:20.255212+0000","last_became_peered":"2023-09-23T03:15:20.255212+0000","last_unstale":"2023-09-23T03:15:25.314864+0000","last_undegraded":"2023-09-23T03:15:25.314864+0000","last_fullsized":"2023-09-23T03:15:25.314864+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:31:01.946245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.305281+0000","last_change":"2023-09-23T03:15:21.156720+0000","last_active":"2023-09-23T03:15:25.305281+0000","last_peered":"2023-09-23T03:15:25.305281+0000","last_clean":"2023-09-23T03:15:25.305281+0000","last_became_active":"2023-09-23T03:15:21.156268+0000","last_became_peered":"2023-09-23T03:15:21.156268+0000","last_unstale":"2023-09-23T03:15:25.305281+0000","last_undegraded":"2023-09-23T03:15:25.305281+0000","last_fullsized":"2023-09-23T03:15:25.305281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:51:00.098552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.134456+0000","last_change":"2023-09-23T03:15:25.319168+0000","last_active":"2023-09-23T03:15:26.134456+0000","last_peered":"2023-09-23T03:15:26.134456+0000","last_clean":"2023-09-23T03:15:26.134456+0000","last_became_active":"2023-09-23T03:15:25.318761+0000","last_became_peered":"2023-09-23T03:15:25.318761+0000","last_unstale":"2023-09-23T03:15:26.134456+0000","last_undegraded":"2023-09-23T03:15:26.134456+0000","last_fullsized":"2023-09-23T03:15:26.134456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:05:53.704379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.296655+0000","last_change":"2023-09-23T03:15:20.261879+0000","last_active":"2023-09-23T03:15:25.296655+0000","last_peered":"2023-09-23T03:15:25.296655+0000","last_clean":"2023-09-23T03:15:25.296655+0000","last_became_active":"2023-09-23T03:15:20.261597+0000","last_became_peered":"2023-09-23T03:15:20.261597+0000","last_unstale":"2023-09-23T03:15:25.296655+0000","last_undegraded":"2023-09-23T03:15:25.296655+0000","last_fullsized":"2023-09-23T03:15:25.296655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:41:37.505547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.311031+0000","last_change":"2023-09-23T03:15:21.161920+0000","last_active":"2023-09-23T03:15:25.311031+0000","last_peered":"2023-09-23T03:15:25.311031+0000","last_clean":"2023-09-23T03:15:25.311031+0000","last_became_active":"2023-09-23T03:15:21.161657+0000","last_became_peered":"2023-09-23T03:15:21.161657+0000","last_unstale":"2023-09-23T03:15:25.311031+0000","last_undegraded":"2023-09-23T03:15:25.311031+0000","last_fullsized":"2023-09-23T03:15:25.311031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:35:17.599463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.486488+0000","last_change":"2023-09-23T03:15:25.319176+0000","last_active":"2023-09-23T03:15:26.486488+0000","last_peered":"2023-09-23T03:15:26.486488+0000","last_clean":"2023-09-23T03:15:26.486488+0000","last_became_active":"2023-09-23T03:15:25.318752+0000","last_became_peered":"2023-09-23T03:15:25.318752+0000","last_unstale":"2023-09-23T03:15:26.486488+0000","last_undegraded":"2023-09-23T03:15:26.486488+0000","last_fullsized":"2023-09-23T03:15:26.486488+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:56:54.725774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.305274+0000","last_change":"2023-09-23T03:15:21.157543+0000","last_active":"2023-09-23T03:15:25.305274+0000","last_peered":"2023-09-23T03:15:25.305274+0000","last_clean":"2023-09-23T03:15:25.305274+0000","last_became_active":"2023-09-23T03:15:21.157240+0000","last_became_peered":"2023-09-23T03:15:21.157240+0000","last_unstale":"2023-09-23T03:15:25.305274+0000","last_undegraded":"2023-09-23T03:15:25.305274+0000","last_fullsized":"2023-09-23T03:15:25.305274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:23:07.611380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.300872+0000","last_change":"2023-09-23T03:15:21.176611+0000","last_active":"2023-09-23T03:15:25.300872+0000","last_peered":"2023-09-23T03:15:25.300872+0000","last_clean":"2023-09-23T03:15:25.300872+0000","last_became_active":"2023-09-23T03:15:21.176316+0000","last_became_peered":"2023-09-23T03:15:21.176316+0000","last_unstale":"2023-09-23T03:15:25.300872+0000","last_undegraded":"2023-09-23T03:15:25.300872+0000","last_fullsized":"2023-09-23T03:15:25.300872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:28:09.982438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.316926+0000","last_change":"2023-09-23T03:15:21.166808+0000","last_active":"2023-09-23T03:15:25.316926+0000","last_peered":"2023-09-23T03:15:25.316926+0000","last_clean":"2023-09-23T03:15:25.316926+0000","last_became_active":"2023-09-23T03:15:21.166488+0000","last_became_peered":"2023-09-23T03:15:21.166488+0000","last_unstale":"2023-09-23T03:15:25.316926+0000","last_undegraded":"2023-09-23T03:15:25.316926+0000","last_fullsized":"2023-09-23T03:15:25.316926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:43:35.707241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.305111+0000","last_change":"2023-09-23T03:15:21.162188+0000","last_active":"2023-09-23T03:15:25.305111+0000","last_peered":"2023-09-23T03:15:25.305111+0000","last_clean":"2023-09-23T03:15:25.305111+0000","last_became_active":"2023-09-23T03:15:21.161886+0000","last_became_peered":"2023-09-23T03:15:21.161886+0000","last_unstale":"2023-09-23T03:15:25.305111+0000","last_undegraded":"2023-09-23T03:15:25.305111+0000","last_fullsized":"2023-09-23T03:15:25.305111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:23:07.611380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.296970+0000","last_change":"2023-09-23T03:15:21.161567+0000","last_active":"2023-09-23T03:15:25.296970+0000","last_peered":"2023-09-23T03:15:25.296970+0000","last_clean":"2023-09-23T03:15:25.296970+0000","last_became_active":"2023-09-23T03:15:21.161168+0000","last_became_peered":"2023-09-23T03:15:21.161168+0000","last_unstale":"2023-09-23T03:15:25.296970+0000","last_undegraded":"2023-09-23T03:15:25.296970+0000","last_fullsized":"2023-09-23T03:15:25.296970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:30:37.652428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.311814+0000","last_change":"2023-09-23T03:15:21.164997+0000","last_active":"2023-09-23T03:15:25.311814+0000","last_peered":"2023-09-23T03:15:25.311814+0000","last_clean":"2023-09-23T03:15:25.311814+0000","last_became_active":"2023-09-23T03:15:21.164680+0000","last_became_peered":"2023-09-23T03:15:21.164680+0000","last_unstale":"2023-09-23T03:15:25.311814+0000","last_undegraded":"2023-09-23T03:15:25.311814+0000","last_fullsized":"2023-09-23T03:15:25.311814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:15:19.584603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.316265+0000","last_change":"2023-09-23T03:15:21.164029+0000","last_active":"2023-09-23T03:15:25.316265+0000","last_peered":"2023-09-23T03:15:25.316265+0000","last_clean":"2023-09-23T03:15:25.316265+0000","last_became_active":"2023-09-23T03:15:21.163673+0000","last_became_peered":"2023-09-23T03:15:21.163673+0000","last_unstale":"2023-09-23T03:15:25.316265+0000","last_undegraded":"2023-09-23T03:15:25.316265+0000","last_fullsized":"2023-09-23T03:15:25.316265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:15:19.584603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.297434+0000","last_change":"2023-09-23T03:15:21.163996+0000","last_active":"2023-09-23T03:15:25.297434+0000","last_peered":"2023-09-23T03:15:25.297434+0000","last_clean":"2023-09-23T03:15:25.297434+0000","last_became_active":"2023-09-23T03:15:21.163674+0000","last_became_peered":"2023-09-23T03:15:21.163674+0000","last_unstale":"2023-09-23T03:15:25.297434+0000","last_undegraded":"2023-09-23T03:15:25.297434+0000","last_fullsized":"2023-09-23T03:15:25.297434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:28:31.810059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.314809+0000","last_change":"2023-09-23T03:15:21.171741+0000","last_active":"2023-09-23T03:15:25.314809+0000","last_peered":"2023-09-23T03:15:25.314809+0000","last_clean":"2023-09-23T03:15:25.314809+0000","last_became_active":"2023-09-23T03:15:21.169357+0000","last_became_peered":"2023-09-23T03:15:21.169357+0000","last_unstale":"2023-09-23T03:15:25.314809+0000","last_undegraded":"2023-09-23T03:15:25.314809+0000","last_fullsized":"2023-09-23T03:15:25.314809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:21:27.583218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.297693+0000","last_change":"2023-09-23T03:15:21.163391+0000","last_active":"2023-09-23T03:15:25.297693+0000","last_peered":"2023-09-23T03:15:25.297693+0000","last_clean":"2023-09-23T03:15:25.297693+0000","last_became_active":"2023-09-23T03:15:21.162413+0000","last_became_peered":"2023-09-23T03:15:21.162413+0000","last_unstale":"2023-09-23T03:15:25.297693+0000","last_undegraded":"2023-09-23T03:15:25.297693+0000","last_fullsized":"2023-09-23T03:15:25.297693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:30:31.951544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315898+0000","last_change":"2023-09-23T03:15:21.171540+0000","last_active":"2023-09-23T03:15:25.315898+0000","last_peered":"2023-09-23T03:15:25.315898+0000","last_clean":"2023-09-23T03:15:25.315898+0000","last_became_active":"2023-09-23T03:15:21.167329+0000","last_became_peered":"2023-09-23T03:15:21.167329+0000","last_unstale":"2023-09-23T03:15:25.315898+0000","last_undegraded":"2023-09-23T03:15:25.315898+0000","last_fullsized":"2023-09-23T03:15:25.315898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:29:14.943541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.304917+0000","last_change":"2023-09-23T03:15:21.167575+0000","last_active":"2023-09-23T03:15:25.304917+0000","last_peered":"2023-09-23T03:15:25.304917+0000","last_clean":"2023-09-23T03:15:25.304917+0000","last_became_active":"2023-09-23T03:15:21.167260+0000","last_became_peered":"2023-09-23T03:15:21.167260+0000","last_unstale":"2023-09-23T03:15:25.304917+0000","last_undegraded":"2023-09-23T03:15:25.304917+0000","last_fullsized":"2023-09-23T03:15:25.304917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:01:05.010735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.314454+0000","last_change":"2023-09-23T03:15:21.151218+0000","last_active":"2023-09-23T03:15:25.314454+0000","last_peered":"2023-09-23T03:15:25.314454+0000","last_clean":"2023-09-23T03:15:25.314454+0000","last_became_active":"2023-09-23T03:15:21.150758+0000","last_became_peered":"2023-09-23T03:15:21.150758+0000","last_unstale":"2023-09-23T03:15:25.314454+0000","last_undegraded":"2023-09-23T03:15:25.314454+0000","last_fullsized":"2023-09-23T03:15:25.314454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:51:00.098552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.445962+0000","last_change":"2023-09-23T03:15:21.175081+0000","last_active":"2023-09-23T03:15:25.445962+0000","last_peered":"2023-09-23T03:15:25.445962+0000","last_clean":"2023-09-23T03:15:25.445962+0000","last_became_active":"2023-09-23T03:15:21.174757+0000","last_became_peered":"2023-09-23T03:15:21.174757+0000","last_unstale":"2023-09-23T03:15:25.445962+0000","last_undegraded":"2023-09-23T03:15:25.445962+0000","last_fullsized":"2023-09-23T03:15:25.445962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:02:09.034422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.301326+0000","last_change":"2023-09-23T03:15:21.165007+0000","last_active":"2023-09-23T03:15:25.301326+0000","last_peered":"2023-09-23T03:15:25.301326+0000","last_clean":"2023-09-23T03:15:25.301326+0000","last_became_active":"2023-09-23T03:15:21.164591+0000","last_became_peered":"2023-09-23T03:15:21.164591+0000","last_unstale":"2023-09-23T03:15:25.301326+0000","last_undegraded":"2023-09-23T03:15:25.301326+0000","last_fullsized":"2023-09-23T03:15:25.301326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:13:22.079955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315446+0000","last_change":"2023-09-23T03:15:21.166029+0000","last_active":"2023-09-23T03:15:25.315446+0000","last_peered":"2023-09-23T03:15:25.315446+0000","last_clean":"2023-09-23T03:15:25.315446+0000","last_became_active":"2023-09-23T03:15:21.165298+0000","last_became_peered":"2023-09-23T03:15:21.165298+0000","last_unstale":"2023-09-23T03:15:25.315446+0000","last_undegraded":"2023-09-23T03:15:25.315446+0000","last_fullsized":"2023-09-23T03:15:25.315446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:42:37.443384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.314713+0000","last_change":"2023-09-23T03:15:21.172091+0000","last_active":"2023-09-23T03:15:25.314713+0000","last_peered":"2023-09-23T03:15:25.314713+0000","last_clean":"2023-09-23T03:15:25.314713+0000","last_became_active":"2023-09-23T03:15:21.170229+0000","last_became_peered":"2023-09-23T03:15:21.170229+0000","last_unstale":"2023-09-23T03:15:25.314713+0000","last_undegraded":"2023-09-23T03:15:25.314713+0000","last_fullsized":"2023-09-23T03:15:25.314713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:15:19.584603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.311046+0000","last_change":"2023-09-23T03:15:21.163298+0000","last_active":"2023-09-23T03:15:25.311046+0000","last_peered":"2023-09-23T03:15:25.311046+0000","last_clean":"2023-09-23T03:15:25.311046+0000","last_became_active":"2023-09-23T03:15:21.163000+0000","last_became_peered":"2023-09-23T03:15:21.163000+0000","last_unstale":"2023-09-23T03:15:25.311046+0000","last_undegraded":"2023-09-23T03:15:25.311046+0000","last_fullsized":"2023-09-23T03:15:25.311046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:22:07.336440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.301412+0000","last_change":"2023-09-23T03:15:21.160829+0000","last_active":"2023-09-23T03:15:25.301412+0000","last_peered":"2023-09-23T03:15:25.301412+0000","last_clean":"2023-09-23T03:15:25.301412+0000","last_became_active":"2023-09-23T03:15:21.160567+0000","last_became_peered":"2023-09-23T03:15:21.160567+0000","last_unstale":"2023-09-23T03:15:25.301412+0000","last_undegraded":"2023-09-23T03:15:25.301412+0000","last_fullsized":"2023-09-23T03:15:25.301412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:06:16.954649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.296315+0000","last_change":"2023-09-23T03:15:21.165912+0000","last_active":"2023-09-23T03:15:25.296315+0000","last_peered":"2023-09-23T03:15:25.296315+0000","last_clean":"2023-09-23T03:15:25.296315+0000","last_became_active":"2023-09-23T03:15:21.165634+0000","last_became_peered":"2023-09-23T03:15:21.165634+0000","last_unstale":"2023-09-23T03:15:25.296315+0000","last_undegraded":"2023-09-23T03:15:25.296315+0000","last_fullsized":"2023-09-23T03:15:25.296315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:15:19.584603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.309291+0000","last_change":"2023-09-23T03:15:21.161315+0000","last_active":"2023-09-23T03:15:25.309291+0000","last_peered":"2023-09-23T03:15:25.309291+0000","last_clean":"2023-09-23T03:15:25.309291+0000","last_became_active":"2023-09-23T03:15:21.160978+0000","last_became_peered":"2023-09-23T03:15:21.160978+0000","last_unstale":"2023-09-23T03:15:25.309291+0000","last_undegraded":"2023-09-23T03:15:25.309291+0000","last_fullsized":"2023-09-23T03:15:25.309291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:28:09.982438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.305734+0000","last_change":"2023-09-23T03:15:21.172253+0000","last_active":"2023-09-23T03:15:25.305734+0000","last_peered":"2023-09-23T03:15:25.305734+0000","last_clean":"2023-09-23T03:15:25.305734+0000","last_became_active":"2023-09-23T03:15:21.171987+0000","last_became_peered":"2023-09-23T03:15:21.171987+0000","last_unstale":"2023-09-23T03:15:25.305734+0000","last_undegraded":"2023-09-23T03:15:25.305734+0000","last_fullsized":"2023-09-23T03:15:25.305734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:30:31.951544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.303805+0000","last_change":"2023-09-23T03:15:21.172883+0000","last_active":"2023-09-23T03:15:25.303805+0000","last_peered":"2023-09-23T03:15:25.303805+0000","last_clean":"2023-09-23T03:15:25.303805+0000","last_became_active":"2023-09-23T03:15:21.172404+0000","last_became_peered":"2023-09-23T03:15:21.172404+0000","last_unstale":"2023-09-23T03:15:25.303805+0000","last_undegraded":"2023-09-23T03:15:25.303805+0000","last_fullsized":"2023-09-23T03:15:25.303805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:21:27.583218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.445963+0000","last_change":"2023-09-23T03:15:21.174812+0000","last_active":"2023-09-23T03:15:25.445963+0000","last_peered":"2023-09-23T03:15:25.445963+0000","last_clean":"2023-09-23T03:15:25.445963+0000","last_became_active":"2023-09-23T03:15:21.174241+0000","last_became_peered":"2023-09-23T03:15:21.174241+0000","last_unstale":"2023-09-23T03:15:25.445963+0000","last_undegraded":"2023-09-23T03:15:25.445963+0000","last_fullsized":"2023-09-23T03:15:25.445963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:58:02.940532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.295571+0000","last_change":"2023-09-23T03:15:21.170905+0000","last_active":"2023-09-23T03:15:25.295571+0000","last_peered":"2023-09-23T03:15:25.295571+0000","last_clean":"2023-09-23T03:15:25.295571+0000","last_became_active":"2023-09-23T03:15:21.170570+0000","last_became_peered":"2023-09-23T03:15:21.170570+0000","last_unstale":"2023-09-23T03:15:25.295571+0000","last_undegraded":"2023-09-23T03:15:25.295571+0000","last_fullsized":"2023-09-23T03:15:25.295571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:23:07.611380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.441959+0000","last_change":"2023-09-23T03:15:21.171879+0000","last_active":"2023-09-23T03:15:25.441959+0000","last_peered":"2023-09-23T03:15:25.441959+0000","last_clean":"2023-09-23T03:15:25.441959+0000","last_became_active":"2023-09-23T03:15:21.171624+0000","last_became_peered":"2023-09-23T03:15:21.171624+0000","last_unstale":"2023-09-23T03:15:25.441959+0000","last_undegraded":"2023-09-23T03:15:25.441959+0000","last_fullsized":"2023-09-23T03:15:25.441959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:39:10.586582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315796+0000","last_change":"2023-09-23T03:15:21.177809+0000","last_active":"2023-09-23T03:15:25.315796+0000","last_peered":"2023-09-23T03:15:25.315796+0000","last_clean":"2023-09-23T03:15:25.315796+0000","last_became_active":"2023-09-23T03:15:21.175320+0000","last_became_peered":"2023-09-23T03:15:21.175320+0000","last_unstale":"2023-09-23T03:15:25.315796+0000","last_undegraded":"2023-09-23T03:15:25.315796+0000","last_fullsized":"2023-09-23T03:15:25.315796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:30:37.652428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.441779+0000","last_change":"2023-09-23T03:15:21.174361+0000","last_active":"2023-09-23T03:15:25.441779+0000","last_peered":"2023-09-23T03:15:25.441779+0000","last_clean":"2023-09-23T03:15:25.441779+0000","last_became_active":"2023-09-23T03:15:21.174101+0000","last_became_peered":"2023-09-23T03:15:21.174101+0000","last_unstale":"2023-09-23T03:15:25.441779+0000","last_undegraded":"2023-09-23T03:15:25.441779+0000","last_fullsized":"2023-09-23T03:15:25.441779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:25:53.924266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315094+0000","last_change":"2023-09-23T03:15:21.171837+0000","last_active":"2023-09-23T03:15:25.315094+0000","last_peered":"2023-09-23T03:15:25.315094+0000","last_clean":"2023-09-23T03:15:25.315094+0000","last_became_active":"2023-09-23T03:15:21.169973+0000","last_became_peered":"2023-09-23T03:15:21.169973+0000","last_unstale":"2023-09-23T03:15:25.315094+0000","last_undegraded":"2023-09-23T03:15:25.315094+0000","last_fullsized":"2023-09-23T03:15:25.315094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:53:21.435687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.303645+0000","last_change":"2023-09-23T03:15:21.157118+0000","last_active":"2023-09-23T03:15:25.303645+0000","last_peered":"2023-09-23T03:15:25.303645+0000","last_clean":"2023-09-23T03:15:25.303645+0000","last_became_active":"2023-09-23T03:15:21.156862+0000","last_became_peered":"2023-09-23T03:15:21.156862+0000","last_unstale":"2023-09-23T03:15:25.303645+0000","last_undegraded":"2023-09-23T03:15:25.303645+0000","last_fullsized":"2023-09-23T03:15:25.303645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:22:07.336440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.311650+0000","last_change":"2023-09-23T03:15:21.167539+0000","last_active":"2023-09-23T03:15:25.311650+0000","last_peered":"2023-09-23T03:15:25.311650+0000","last_clean":"2023-09-23T03:15:25.311650+0000","last_became_active":"2023-09-23T03:15:21.167256+0000","last_became_peered":"2023-09-23T03:15:21.167256+0000","last_unstale":"2023-09-23T03:15:25.311650+0000","last_undegraded":"2023-09-23T03:15:25.311650+0000","last_fullsized":"2023-09-23T03:15:25.311650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:21:27.583218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.445518+0000","last_change":"2023-09-23T03:15:21.171479+0000","last_active":"2023-09-23T03:15:25.445518+0000","last_peered":"2023-09-23T03:15:25.445518+0000","last_clean":"2023-09-23T03:15:25.445518+0000","last_became_active":"2023-09-23T03:15:21.171128+0000","last_became_peered":"2023-09-23T03:15:21.171128+0000","last_unstale":"2023-09-23T03:15:25.445518+0000","last_undegraded":"2023-09-23T03:15:25.445518+0000","last_fullsized":"2023-09-23T03:15:25.445518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:38:51.387638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":851968,"data_stored":690730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1220000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3759999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6930000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5009999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.591}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.429}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1319999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.371}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4119999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6220000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.488}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.51}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":99870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5780000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0069999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5409999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2609999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5109999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3779999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":99906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4179999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2849999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2410000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.716}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.345}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0549999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.298}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1180000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4570000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1559999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":99780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4330000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.51}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4139999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3640000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.387}]}]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":262144,"data_stored":99940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5109999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.405}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.252}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3759999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":689767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.254}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2090000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.296}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3320000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-23T03:15:50.668 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-23T03:15:50.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.917+0000 7f7410e5d640 1 Processor -- start 2023-09-23T03:15:50.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.918+0000 7f7410e5d640 1 -- start start 2023-09-23T03:15:50.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.919+0000 7f7410e5d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f740c0cf130 0x7f740c0ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:50.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.919+0000 7f7410e5d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f740c0b2fe0 0x7f740c15d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:50.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.919+0000 7f7410e5d640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f740c06ceb0 con 0x7f740c0ada30 2023-09-23T03:15:50.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.919+0000 7f7410e5d640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f740c0ade10 con 0x7f740c0cf130 2023-09-23T03:15:50.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.919+0000 7f7410e5d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f740c159c90 con 0x7f740c0b2fe0 2023-09-23T03:15:50.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.920+0000 7f740ad9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f740c0b2fe0 0x7f740c15d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:50.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.920+0000 7f740ad9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f740c0b2fe0 0x7f740c15d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:43798/0 (socket says 172.21.15.154:43798) 2023-09-23T03:15:50.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.920+0000 7f740ad9d640 1 -- 172.21.15.154:0/3487603597 learned_addr learned my addr 172.21.15.154:0/3487603597 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:50.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.920+0000 7f7401d9b640 1 --2- 172.21.15.154:0/3487603597 >> v2:172.21.15.190:3300/0 conn(0x7f740c0cf130 0x7f740c0ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:50.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.921+0000 7f740ad9d640 1 -- 172.21.15.154:0/3487603597 >> v2:172.21.15.190:3300/0 conn(0x7f740c0cf130 msgr2=0x7f740c0ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:50.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.921+0000 7f740ad9d640 1 --2- 172.21.15.154:0/3487603597 >> v2:172.21.15.190:3300/0 conn(0x7f740c0cf130 0x7f740c0ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:50.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.921+0000 7f740ad9d640 1 -- 172.21.15.154:0/3487603597 >> v1:172.21.15.190:6789/0 conn(0x7f740c0ada30 legacy=0x7f740c0cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:50.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.921+0000 7f740ad9d640 1 -- 172.21.15.154:0/3487603597 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f740c06c110 con 0x7f740c0b2fe0 2023-09-23T03:15:50.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.921+0000 7f7401d9b640 1 --2- 172.21.15.154:0/3487603597 >> v2:172.21.15.190:3300/0 conn(0x7f740c0cf130 0x7f740c0ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:15:50.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.922+0000 7f740ad9d640 1 --2- 172.21.15.154:0/3487603597 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f740c0b2fe0 0x7f740c15d380 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7404009e50 tx=0x7f740402e170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac89368416920c53 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:50.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.922+0000 7f7409d9b640 1 -- 172.21.15.154:0/3487603597 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740402ed10 con 0x7f740c0b2fe0 2023-09-23T03:15:50.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.922+0000 7f7409d9b640 1 -- 172.21.15.154:0/3487603597 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f740402ee70 con 0x7f740c0b2fe0 2023-09-23T03:15:50.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.922+0000 7f7409d9b640 1 -- 172.21.15.154:0/3487603597 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74040365f0 con 0x7f740c0b2fe0 2023-09-23T03:15:50.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.923+0000 7f7410e5d640 1 -- 172.21.15.154:0/3487603597 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f740c0b2fe0 msgr2=0x7f740c15d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:50.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.923+0000 7f7410e5d640 1 --2- 172.21.15.154:0/3487603597 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f740c0b2fe0 0x7f740c15d380 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7404009e50 tx=0x7f740402e170 comp rx=0 tx=0).stop 2023-09-23T03:15:50.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.923+0000 7f7410e5d640 1 -- 172.21.15.154:0/3487603597 shutdown_connections 2023-09-23T03:15:50.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.923+0000 7f7410e5d640 1 --2- 172.21.15.154:0/3487603597 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f740c0b2fe0 0x7f740c15d380 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:50.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.923+0000 7f7410e5d640 1 --2- 172.21.15.154:0/3487603597 >> v2:172.21.15.190:3300/0 conn(0x7f740c0cf130 0x7f740c0ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:50.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.923+0000 7f7410e5d640 1 -- 172.21.15.154:0/3487603597 >> 172.21.15.154:0/3487603597 conn(0x7f740c06a0f0 msgr2=0x7f740c06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:50.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.923+0000 7f7410e5d640 1 -- 172.21.15.154:0/3487603597 shutdown_connections 2023-09-23T03:15:50.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.923+0000 7f7410e5d640 1 -- 172.21.15.154:0/3487603597 wait complete. 2023-09-23T03:15:50.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.926+0000 7f7410e5d640 1 Processor -- start 2023-09-23T03:15:50.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.926+0000 7f7410e5d640 1 -- start start 2023-09-23T03:15:50.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.927+0000 7f7410e5d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f740c0ada30 0x7f740c15b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:50.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.927+0000 7f7410e5d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f740c0b2fe0 0x7f740c1ef280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:50.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.927+0000 7f7410e5d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f740c0cf130 0x7f740c1f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:50.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.927+0000 7f7410e5d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c15b9a0 con 0x7f740c0ada30 2023-09-23T03:15:50.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.927+0000 7f7410e5d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c15bc80 con 0x7f740c0cf130 2023-09-23T03:15:50.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.927+0000 7f7410e5d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f740c15bf60 con 0x7f740c0b2fe0 2023-09-23T03:15:50.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.928+0000 7f740a59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f740c0ada30 0x7f740c15b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:50.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.928+0000 7f740a59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f740c0ada30 0x7f740c15b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44426/0 (socket says 172.21.15.154:44426) 2023-09-23T03:15:50.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.928+0000 7f740a59c640 1 -- 172.21.15.154:0/570504771 learned_addr learned my addr 172.21.15.154:0/570504771 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:50.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.928+0000 7f7401d9b640 1 --2- 172.21.15.154:0/570504771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f740c0b2fe0 0x7f740c1ef280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:50.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.928+0000 7f740a59c640 1 -- 172.21.15.154:0/570504771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f740c0b2fe0 msgr2=0x7f740c1ef280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:50.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.928+0000 7f740a59c640 1 --2- 172.21.15.154:0/570504771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f740c0b2fe0 0x7f740c1ef280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:50.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.928+0000 7f740ad9d640 1 --2- 172.21.15.154:0/570504771 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f740c0cf130 0x7f740c1f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:50.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.928+0000 7f740a59c640 1 -- 172.21.15.154:0/570504771 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f740c0cf130 msgr2=0x7f740c1f56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:50.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.928+0000 7f740a59c640 1 --2- 172.21.15.154:0/570504771 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f740c0cf130 0x7f740c1f56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:50.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.928+0000 7f740a59c640 1 -- 172.21.15.154:0/570504771 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f4009400 con 0x7f740c0ada30 2023-09-23T03:15:50.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.928+0000 7f7401d9b640 1 --2- 172.21.15.154:0/570504771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f740c0b2fe0 0x7f740c1ef280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:50.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.928+0000 7f740a59c640 1 --2- 172.21.15.154:0/570504771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f740c0ada30 0x7f740c15b020 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f73fc00c400 tx=0x7f73fc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:50.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.929+0000 7f74037fe640 1 -- 172.21.15.154:0/570504771 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73fc0052d0 con 0x7f740c0ada30 2023-09-23T03:15:50.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.929+0000 7f74037fe640 1 -- 172.21.15.154:0/570504771 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73fc005470 con 0x7f740c0ada30 2023-09-23T03:15:50.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.929+0000 7f74037fe640 1 -- 172.21.15.154:0/570504771 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73fc006a60 con 0x7f740c0ada30 2023-09-23T03:15:50.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.929+0000 7f7410e5d640 1 -- 172.21.15.154:0/570504771 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7404009b30 con 0x7f740c0ada30 2023-09-23T03:15:50.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.929+0000 7f7410e5d640 1 -- 172.21.15.154:0/570504771 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f740c09a0c0 con 0x7f740c0ada30 2023-09-23T03:15:50.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.930+0000 7f7410e5d640 1 -- 172.21.15.154:0/570504771 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f740c09a2e0 con 0x7f740c0ada30 2023-09-23T03:15:50.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.931+0000 7f74037fe640 1 -- 172.21.15.154:0/570504771 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f73fc006bc0 con 0x7f740c0ada30 2023-09-23T03:15:50.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.932+0000 7f74037fe640 1 --2- 172.21.15.154:0/570504771 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f73ec06af30 0x7f73ec06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:50.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.932+0000 7f74037fe640 1 -- 172.21.15.154:0/570504771 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f73fc088cd0 con 0x7f740c0ada30 2023-09-23T03:15:50.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.932+0000 7f7401d9b640 1 --2- 172.21.15.154:0/570504771 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f73ec06af30 0x7f73ec06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:50.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.933+0000 7f7401d9b640 1 --2- 172.21.15.154:0/570504771 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f73ec06af30 0x7f73ec06d3f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f740c0c2000 tx=0x7f73f4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:50.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:50.936+0000 7f74037fe640 1 -- 172.21.15.154:0/570504771 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f740c09a2e0 con 0x7f740c0ada30 2023-09-23T03:15:51.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.120+0000 7f7410e5d640 1 -- 172.21.15.154:0/570504771 --> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f740c06c6d0 con 0x7f73ec06af30 2023-09-23T03:15:51.147 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.146+0000 7f74037fe640 1 -- 172.21.15.154:0/570504771 <== mgr.4110 v2:172.21.15.154:6832/2422652104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345152 (secure 0 0 0) 0x7f73f40109a0 con 0x7f73ec06af30 2023-09-23T03:15:51.153 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:15:51.155 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2023-09-23T03:15:51.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.154+0000 7f7410e5d640 1 -- 172.21.15.154:0/570504771 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f73ec06af30 msgr2=0x7f73ec06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:51.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.154+0000 7f7410e5d640 1 --2- 172.21.15.154:0/570504771 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f73ec06af30 0x7f73ec06d3f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f740c0c2000 tx=0x7f73f4008040 comp rx=0 tx=0).stop 2023-09-23T03:15:51.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.154+0000 7f7410e5d640 1 -- 172.21.15.154:0/570504771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f740c0ada30 msgr2=0x7f740c15b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:51.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.154+0000 7f7410e5d640 1 --2- 172.21.15.154:0/570504771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f740c0ada30 0x7f740c15b020 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f73fc00c400 tx=0x7f73fc00c8c0 comp rx=0 tx=0).stop 2023-09-23T03:15:51.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.154+0000 7f7410e5d640 1 -- 172.21.15.154:0/570504771 shutdown_connections 2023-09-23T03:15:51.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.154+0000 7f7410e5d640 1 --2- 172.21.15.154:0/570504771 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f73ec06af30 0x7f73ec06d3f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:51.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.154+0000 7f7410e5d640 1 --2- 172.21.15.154:0/570504771 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f740c0cf130 0x7f740c1f56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:51.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.154+0000 7f7410e5d640 1 --2- 172.21.15.154:0/570504771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f740c0b2fe0 0x7f740c1ef280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:51.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.154+0000 7f7410e5d640 1 --2- 172.21.15.154:0/570504771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f740c0ada30 0x7f740c15b020 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:51.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.154+0000 7f7410e5d640 1 -- 172.21.15.154:0/570504771 >> 172.21.15.154:0/570504771 conn(0x7f740c06a0f0 msgr2=0x7f740c071d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:51.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.154+0000 7f7410e5d640 1 -- 172.21.15.154:0/570504771 shutdown_connections 2023-09-23T03:15:51.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.154+0000 7f7410e5d640 1 -- 172.21.15.154:0/570504771 wait complete. 2023-09-23T03:15:51.173 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2023-09-23T03:15:49.172850+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219416,"kb_used_data":3204,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755304,"statfs":{"total":773094113280,"available":772869431296,"internally_reserved":0,"allocated":3280896,"data_stored":1987235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":221236794},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005051"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.303622+0000","last_change":"2023-09-23T03:15:20.261808+0000","last_active":"2023-09-23T03:15:25.303622+0000","last_peered":"2023-09-23T03:15:25.303622+0000","last_clean":"2023-09-23T03:15:25.303622+0000","last_became_active":"2023-09-23T03:15:20.261512+0000","last_became_peered":"2023-09-23T03:15:20.261512+0000","last_unstale":"2023-09-23T03:15:25.303622+0000","last_undegraded":"2023-09-23T03:15:25.303622+0000","last_fullsized":"2023-09-23T03:15:25.303622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:28:08.970374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.296813+0000","last_change":"2023-09-23T03:15:21.165202+0000","last_active":"2023-09-23T03:15:25.296813+0000","last_peered":"2023-09-23T03:15:25.296813+0000","last_clean":"2023-09-23T03:15:25.296813+0000","last_became_active":"2023-09-23T03:15:21.164933+0000","last_became_peered":"2023-09-23T03:15:21.164933+0000","last_unstale":"2023-09-23T03:15:25.296813+0000","last_undegraded":"2023-09-23T03:15:25.296813+0000","last_fullsized":"2023-09-23T03:15:25.296813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:29:14.943541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.485848+0000","last_change":"2023-09-23T03:15:25.453212+0000","last_active":"2023-09-23T03:15:26.485848+0000","last_peered":"2023-09-23T03:15:26.485848+0000","last_clean":"2023-09-23T03:15:26.485848+0000","last_became_active":"2023-09-23T03:15:25.452429+0000","last_became_peered":"2023-09-23T03:15:25.452429+0000","last_unstale":"2023-09-23T03:15:26.485848+0000","last_undegraded":"2023-09-23T03:15:26.485848+0000","last_fullsized":"2023-09-23T03:15:26.485848+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:28:19.168744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.297194+0000","last_change":"2023-09-23T03:15:20.241249+0000","last_active":"2023-09-23T03:15:25.297194+0000","last_peered":"2023-09-23T03:15:25.297194+0000","last_clean":"2023-09-23T03:15:25.297194+0000","last_became_active":"2023-09-23T03:15:20.240943+0000","last_became_peered":"2023-09-23T03:15:20.240943+0000","last_unstale":"2023-09-23T03:15:25.297194+0000","last_undegraded":"2023-09-23T03:15:25.297194+0000","last_fullsized":"2023-09-23T03:15:25.297194+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:05:48.619034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.303332+0000","last_change":"2023-09-23T03:15:21.172718+0000","last_active":"2023-09-23T03:15:25.303332+0000","last_peered":"2023-09-23T03:15:25.303332+0000","last_clean":"2023-09-23T03:15:25.303332+0000","last_became_active":"2023-09-23T03:15:21.172296+0000","last_became_peered":"2023-09-23T03:15:21.172296+0000","last_unstale":"2023-09-23T03:15:25.303332+0000","last_undegraded":"2023-09-23T03:15:25.303332+0000","last_fullsized":"2023-09-23T03:15:25.303332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:30:31.951544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.137218+0000","last_change":"2023-09-23T03:15:25.330094+0000","last_active":"2023-09-23T03:15:26.137218+0000","last_peered":"2023-09-23T03:15:26.137218+0000","last_clean":"2023-09-23T03:15:26.137218+0000","last_became_active":"2023-09-23T03:15:25.329632+0000","last_became_peered":"2023-09-23T03:15:25.329632+0000","last_unstale":"2023-09-23T03:15:26.137218+0000","last_undegraded":"2023-09-23T03:15:26.137218+0000","last_fullsized":"2023-09-23T03:15:26.137218+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:21:31.656499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.445726+0000","last_change":"2023-09-23T03:15:20.261800+0000","last_active":"2023-09-23T03:15:25.445726+0000","last_peered":"2023-09-23T03:15:25.445726+0000","last_clean":"2023-09-23T03:15:25.445726+0000","last_became_active":"2023-09-23T03:15:20.261459+0000","last_became_peered":"2023-09-23T03:15:20.261459+0000","last_unstale":"2023-09-23T03:15:25.445726+0000","last_undegraded":"2023-09-23T03:15:25.445726+0000","last_fullsized":"2023-09-23T03:15:25.445726+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:30:30.939479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315185+0000","last_change":"2023-09-23T03:15:21.172135+0000","last_active":"2023-09-23T03:15:25.315185+0000","last_peered":"2023-09-23T03:15:25.315185+0000","last_clean":"2023-09-23T03:15:25.315185+0000","last_became_active":"2023-09-23T03:15:21.169373+0000","last_became_peered":"2023-09-23T03:15:21.169373+0000","last_unstale":"2023-09-23T03:15:25.315185+0000","last_undegraded":"2023-09-23T03:15:25.315185+0000","last_fullsized":"2023-09-23T03:15:25.315185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:52:02.570741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.139181+0000","last_change":"2023-09-23T03:15:25.328735+0000","last_active":"2023-09-23T03:15:26.139181+0000","last_peered":"2023-09-23T03:15:26.139181+0000","last_clean":"2023-09-23T03:15:26.139181+0000","last_became_active":"2023-09-23T03:15:25.328423+0000","last_became_peered":"2023-09-23T03:15:25.328423+0000","last_unstale":"2023-09-23T03:15:26.139181+0000","last_undegraded":"2023-09-23T03:15:26.139181+0000","last_fullsized":"2023-09-23T03:15:26.139181+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:25:57.997547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.441936+0000","last_change":"2023-09-23T03:15:20.264630+0000","last_active":"2023-09-23T03:15:25.441936+0000","last_peered":"2023-09-23T03:15:25.441936+0000","last_clean":"2023-09-23T03:15:25.441936+0000","last_became_active":"2023-09-23T03:15:20.264209+0000","last_became_peered":"2023-09-23T03:15:20.264209+0000","last_unstale":"2023-09-23T03:15:25.441936+0000","last_undegraded":"2023-09-23T03:15:25.441936+0000","last_fullsized":"2023-09-23T03:15:25.441936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:41:37.505547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315447+0000","last_change":"2023-09-23T03:15:21.176949+0000","last_active":"2023-09-23T03:15:25.315447+0000","last_peered":"2023-09-23T03:15:25.315447+0000","last_clean":"2023-09-23T03:15:25.315447+0000","last_became_active":"2023-09-23T03:15:21.176593+0000","last_became_peered":"2023-09-23T03:15:21.176593+0000","last_unstale":"2023-09-23T03:15:25.315447+0000","last_undegraded":"2023-09-23T03:15:25.315447+0000","last_fullsized":"2023-09-23T03:15:25.315447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:28:31.810059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.138092+0000","last_change":"2023-09-23T03:15:25.324657+0000","last_active":"2023-09-23T03:15:26.138092+0000","last_peered":"2023-09-23T03:15:26.138092+0000","last_clean":"2023-09-23T03:15:26.138092+0000","last_became_active":"2023-09-23T03:15:25.324107+0000","last_became_peered":"2023-09-23T03:15:25.324107+0000","last_unstale":"2023-09-23T03:15:26.138092+0000","last_undegraded":"2023-09-23T03:15:26.138092+0000","last_fullsized":"2023-09-23T03:15:26.138092+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:30:41.725709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.306037+0000","last_change":"2023-09-23T03:15:21.148790+0000","last_active":"2023-09-23T03:15:25.306037+0000","last_peered":"2023-09-23T03:15:25.306037+0000","last_clean":"2023-09-23T03:15:25.306037+0000","last_became_active":"2023-09-23T03:15:21.148165+0000","last_became_peered":"2023-09-23T03:15:21.148165+0000","last_unstale":"2023-09-23T03:15:25.306037+0000","last_undegraded":"2023-09-23T03:15:25.306037+0000","last_fullsized":"2023-09-23T03:15:25.306037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:22:07.336440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.442912+0000","last_change":"2023-09-23T03:15:20.262080+0000","last_active":"2023-09-23T03:15:25.442912+0000","last_peered":"2023-09-23T03:15:25.442912+0000","last_clean":"2023-09-23T03:15:25.442912+0000","last_became_active":"2023-09-23T03:15:20.261391+0000","last_became_peered":"2023-09-23T03:15:20.261391+0000","last_unstale":"2023-09-23T03:15:25.442912+0000","last_undegraded":"2023-09-23T03:15:25.442912+0000","last_fullsized":"2023-09-23T03:15:25.442912+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:30:36.640364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.140544+0000","last_change":"2023-09-23T03:15:25.324180+0000","last_active":"2023-09-23T03:15:26.140544+0000","last_peered":"2023-09-23T03:15:26.140544+0000","last_clean":"2023-09-23T03:15:26.140544+0000","last_became_active":"2023-09-23T03:15:25.323694+0000","last_became_peered":"2023-09-23T03:15:25.323694+0000","last_unstale":"2023-09-23T03:15:26.140544+0000","last_undegraded":"2023-09-23T03:15:26.140544+0000","last_fullsized":"2023-09-23T03:15:26.140544+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:42:41.516665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.296569+0000","last_change":"2023-09-23T03:15:21.168584+0000","last_active":"2023-09-23T03:15:25.296569+0000","last_peered":"2023-09-23T03:15:25.296569+0000","last_clean":"2023-09-23T03:15:25.296569+0000","last_became_active":"2023-09-23T03:15:21.168238+0000","last_became_peered":"2023-09-23T03:15:21.168238+0000","last_unstale":"2023-09-23T03:15:25.296569+0000","last_undegraded":"2023-09-23T03:15:25.296569+0000","last_fullsized":"2023-09-23T03:15:25.296569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:21:27.583218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.311312+0000","last_change":"2023-09-23T03:15:20.251584+0000","last_active":"2023-09-23T03:15:25.311312+0000","last_peered":"2023-09-23T03:15:25.311312+0000","last_clean":"2023-09-23T03:15:25.311312+0000","last_became_active":"2023-09-23T03:15:20.250687+0000","last_became_peered":"2023-09-23T03:15:20.250687+0000","last_unstale":"2023-09-23T03:15:25.311312+0000","last_undegraded":"2023-09-23T03:15:25.311312+0000","last_fullsized":"2023-09-23T03:15:25.311312+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:01:03.998670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.138838+0000","last_change":"2023-09-23T03:15:25.329231+0000","last_active":"2023-09-23T03:15:26.138838+0000","last_peered":"2023-09-23T03:15:26.138838+0000","last_clean":"2023-09-23T03:15:26.138838+0000","last_became_active":"2023-09-23T03:15:25.328825+0000","last_became_peered":"2023-09-23T03:15:25.328825+0000","last_unstale":"2023-09-23T03:15:26.138838+0000","last_undegraded":"2023-09-23T03:15:26.138838+0000","last_fullsized":"2023-09-23T03:15:26.138838+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:29:19.016822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.295849+0000","last_change":"2023-09-23T03:15:21.162632+0000","last_active":"2023-09-23T03:15:25.295849+0000","last_peered":"2023-09-23T03:15:25.295849+0000","last_clean":"2023-09-23T03:15:25.295849+0000","last_became_active":"2023-09-23T03:15:21.161891+0000","last_became_peered":"2023-09-23T03:15:21.161891+0000","last_unstale":"2023-09-23T03:15:25.295849+0000","last_undegraded":"2023-09-23T03:15:25.295849+0000","last_fullsized":"2023-09-23T03:15:25.295849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:39:10.586582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.311128+0000","last_change":"2023-09-23T03:15:20.265489+0000","last_active":"2023-09-23T03:15:25.311128+0000","last_peered":"2023-09-23T03:15:25.311128+0000","last_clean":"2023-09-23T03:15:25.311128+0000","last_became_active":"2023-09-23T03:15:20.265295+0000","last_became_peered":"2023-09-23T03:15:20.265295+0000","last_unstale":"2023-09-23T03:15:25.311128+0000","last_undegraded":"2023-09-23T03:15:25.311128+0000","last_fullsized":"2023-09-23T03:15:25.311128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:06:15.942585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.619606+0000","last_change":"2023-09-23T03:15:25.450826+0000","last_active":"2023-09-23T03:15:26.619606+0000","last_peered":"2023-09-23T03:15:26.619606+0000","last_clean":"2023-09-23T03:15:26.619606+0000","last_became_active":"2023-09-23T03:15:25.449562+0000","last_became_peered":"2023-09-23T03:15:25.449562+0000","last_unstale":"2023-09-23T03:15:26.619606+0000","last_undegraded":"2023-09-23T03:15:26.619606+0000","last_fullsized":"2023-09-23T03:15:26.619606+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:02:13.107703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.310119+0000","last_change":"2023-09-23T03:15:21.151987+0000","last_active":"2023-09-23T03:15:25.310119+0000","last_peered":"2023-09-23T03:15:25.310119+0000","last_clean":"2023-09-23T03:15:25.310119+0000","last_became_active":"2023-09-23T03:15:21.151421+0000","last_became_peered":"2023-09-23T03:15:21.151421+0000","last_unstale":"2023-09-23T03:15:25.310119+0000","last_undegraded":"2023-09-23T03:15:25.310119+0000","last_fullsized":"2023-09-23T03:15:25.310119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:32:42.587095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.441395+0000","last_change":"2023-09-23T03:15:20.262324+0000","last_active":"2023-09-23T03:15:25.441395+0000","last_peered":"2023-09-23T03:15:25.441395+0000","last_clean":"2023-09-23T03:15:25.441395+0000","last_became_active":"2023-09-23T03:15:20.261456+0000","last_became_peered":"2023-09-23T03:15:20.261456+0000","last_unstale":"2023-09-23T03:15:25.441395+0000","last_undegraded":"2023-09-23T03:15:25.441395+0000","last_fullsized":"2023-09-23T03:15:25.441395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:31:40.244446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.139092+0000","last_change":"2023-09-23T03:15:25.329961+0000","last_active":"2023-09-23T03:15:26.139092+0000","last_peered":"2023-09-23T03:15:26.139092+0000","last_clean":"2023-09-23T03:15:26.139092+0000","last_became_active":"2023-09-23T03:15:25.329525+0000","last_became_peered":"2023-09-23T03:15:25.329525+0000","last_unstale":"2023-09-23T03:15:26.139092+0000","last_undegraded":"2023-09-23T03:15:26.139092+0000","last_fullsized":"2023-09-23T03:15:26.139092+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:39:14.659863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.304127+0000","last_change":"2023-09-23T03:15:21.160274+0000","last_active":"2023-09-23T03:15:25.304127+0000","last_peered":"2023-09-23T03:15:25.304127+0000","last_clean":"2023-09-23T03:15:25.304127+0000","last_became_active":"2023-09-23T03:15:21.160110+0000","last_became_peered":"2023-09-23T03:15:21.160110+0000","last_unstale":"2023-09-23T03:15:25.304127+0000","last_undegraded":"2023-09-23T03:15:25.304127+0000","last_fullsized":"2023-09-23T03:15:25.304127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:15:19.584603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.313804+0000","last_change":"2023-09-23T03:15:20.265791+0000","last_active":"2023-09-23T03:15:25.313804+0000","last_peered":"2023-09-23T03:15:25.313804+0000","last_clean":"2023-09-23T03:15:25.313804+0000","last_became_active":"2023-09-23T03:15:20.265429+0000","last_became_peered":"2023-09-23T03:15:20.265429+0000","last_unstale":"2023-09-23T03:15:25.313804+0000","last_undegraded":"2023-09-23T03:15:25.313804+0000","last_fullsized":"2023-09-23T03:15:25.313804+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:05:48.619034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.138192+0000","last_change":"2023-09-23T03:15:25.451854+0000","last_active":"2023-09-23T03:15:26.138192+0000","last_peered":"2023-09-23T03:15:26.138192+0000","last_clean":"2023-09-23T03:15:26.138192+0000","last_became_active":"2023-09-23T03:15:25.450790+0000","last_became_peered":"2023-09-23T03:15:25.450790+0000","last_unstale":"2023-09-23T03:15:26.138192+0000","last_undegraded":"2023-09-23T03:15:26.138192+0000","last_fullsized":"2023-09-23T03:15:26.138192+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:28:35.883340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.297558+0000","last_change":"2023-09-23T03:15:21.163852+0000","last_active":"2023-09-23T03:15:25.297558+0000","last_peered":"2023-09-23T03:15:25.297558+0000","last_clean":"2023-09-23T03:15:25.297558+0000","last_became_active":"2023-09-23T03:15:21.163332+0000","last_became_peered":"2023-09-23T03:15:21.163332+0000","last_unstale":"2023-09-23T03:15:25.297558+0000","last_undegraded":"2023-09-23T03:15:25.297558+0000","last_fullsized":"2023-09-23T03:15:25.297558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:25:53.924266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.616670+0000","last_change":"2023-09-23T03:15:20.264120+0000","last_active":"2023-09-23T03:15:25.616670+0000","last_peered":"2023-09-23T03:15:25.616670+0000","last_clean":"2023-09-23T03:15:25.616670+0000","last_became_active":"2023-09-23T03:15:20.263598+0000","last_became_peered":"2023-09-23T03:15:20.263598+0000","last_unstale":"2023-09-23T03:15:25.616670+0000","last_undegraded":"2023-09-23T03:15:25.616670+0000","last_fullsized":"2023-09-23T03:15:25.616670+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:31:01.946245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.485429+0000","last_change":"2023-09-23T03:15:25.444557+0000","last_active":"2023-09-23T03:15:26.485429+0000","last_peered":"2023-09-23T03:15:26.485429+0000","last_clean":"2023-09-23T03:15:26.485429+0000","last_became_active":"2023-09-23T03:15:25.444002+0000","last_became_peered":"2023-09-23T03:15:25.444002+0000","last_unstale":"2023-09-23T03:15:26.485429+0000","last_undegraded":"2023-09-23T03:15:26.485429+0000","last_fullsized":"2023-09-23T03:15:26.485429+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:15:29.393607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.311253+0000","last_change":"2023-09-23T03:15:21.155177+0000","last_active":"2023-09-23T03:15:25.311253+0000","last_peered":"2023-09-23T03:15:25.311253+0000","last_clean":"2023-09-23T03:15:25.311253+0000","last_became_active":"2023-09-23T03:15:21.154788+0000","last_became_peered":"2023-09-23T03:15:21.154788+0000","last_unstale":"2023-09-23T03:15:25.311253+0000","last_undegraded":"2023-09-23T03:15:25.311253+0000","last_fullsized":"2023-09-23T03:15:25.311253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:51:00.098552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.446205+0000","last_change":"2023-09-23T03:15:20.261866+0000","last_active":"2023-09-23T03:15:25.446205+0000","last_peered":"2023-09-23T03:15:25.446205+0000","last_clean":"2023-09-23T03:15:25.446205+0000","last_became_active":"2023-09-23T03:15:20.261448+0000","last_became_peered":"2023-09-23T03:15:20.261448+0000","last_unstale":"2023-09-23T03:15:25.446205+0000","last_undegraded":"2023-09-23T03:15:25.446205+0000","last_fullsized":"2023-09-23T03:15:25.446205+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:15:18.572539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.619178+0000","last_change":"2023-09-23T03:15:25.443253+0000","last_active":"2023-09-23T03:15:26.619178+0000","last_peered":"2023-09-23T03:15:26.619178+0000","last_clean":"2023-09-23T03:15:26.619178+0000","last_became_active":"2023-09-23T03:15:25.442252+0000","last_became_peered":"2023-09-23T03:15:25.442252+0000","last_unstale":"2023-09-23T03:15:26.619178+0000","last_undegraded":"2023-09-23T03:15:26.619178+0000","last_fullsized":"2023-09-23T03:15:26.619178+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:53:25.508968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315540+0000","last_change":"2023-09-23T03:15:21.167597+0000","last_active":"2023-09-23T03:15:25.315540+0000","last_peered":"2023-09-23T03:15:25.315540+0000","last_clean":"2023-09-23T03:15:25.315540+0000","last_became_active":"2023-09-23T03:15:21.167306+0000","last_became_peered":"2023-09-23T03:15:21.167306+0000","last_unstale":"2023-09-23T03:15:25.315540+0000","last_undegraded":"2023-09-23T03:15:25.315540+0000","last_fullsized":"2023-09-23T03:15:25.315540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:25:53.924266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.311346+0000","last_change":"2023-09-23T03:15:20.254369+0000","last_active":"2023-09-23T03:15:25.311346+0000","last_peered":"2023-09-23T03:15:25.311346+0000","last_clean":"2023-09-23T03:15:25.311346+0000","last_became_active":"2023-09-23T03:15:20.251723+0000","last_became_peered":"2023-09-23T03:15:20.251723+0000","last_unstale":"2023-09-23T03:15:25.311346+0000","last_undegraded":"2023-09-23T03:15:25.311346+0000","last_fullsized":"2023-09-23T03:15:25.311346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:32:41.575031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.616591+0000","last_change":"2023-09-23T03:15:25.331003+0000","last_active":"2023-09-23T03:15:26.616591+0000","last_peered":"2023-09-23T03:15:26.616591+0000","last_clean":"2023-09-23T03:15:26.616591+0000","last_became_active":"2023-09-23T03:15:25.330351+0000","last_became_peered":"2023-09-23T03:15:25.330351+0000","last_unstale":"2023-09-23T03:15:26.616591+0000","last_undegraded":"2023-09-23T03:15:26.616591+0000","last_fullsized":"2023-09-23T03:15:26.616591+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:42:41.516665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.311073+0000","last_change":"2023-09-23T03:15:21.160153+0000","last_active":"2023-09-23T03:15:25.311073+0000","last_peered":"2023-09-23T03:15:25.311073+0000","last_clean":"2023-09-23T03:15:25.311073+0000","last_became_active":"2023-09-23T03:15:21.159780+0000","last_became_peered":"2023-09-23T03:15:21.159780+0000","last_unstale":"2023-09-23T03:15:25.311073+0000","last_undegraded":"2023-09-23T03:15:25.311073+0000","last_fullsized":"2023-09-23T03:15:25.311073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:31:41.256510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.317480+0000","last_change":"2023-09-23T03:15:20.258357+0000","last_active":"2023-09-23T03:15:25.317480+0000","last_peered":"2023-09-23T03:15:25.317480+0000","last_clean":"2023-09-23T03:15:25.317480+0000","last_became_active":"2023-09-23T03:15:20.258052+0000","last_became_peered":"2023-09-23T03:15:20.258052+0000","last_unstale":"2023-09-23T03:15:25.317480+0000","last_undegraded":"2023-09-23T03:15:25.317480+0000","last_fullsized":"2023-09-23T03:15:25.317480+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:01:03.998670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.618336+0000","last_change":"2023-09-23T03:15:25.326862+0000","last_active":"2023-09-23T03:15:26.618336+0000","last_peered":"2023-09-23T03:15:26.618336+0000","last_clean":"2023-09-23T03:15:26.618336+0000","last_became_active":"2023-09-23T03:15:25.326483+0000","last_became_peered":"2023-09-23T03:15:25.326483+0000","last_unstale":"2023-09-23T03:15:26.618336+0000","last_undegraded":"2023-09-23T03:15:26.618336+0000","last_fullsized":"2023-09-23T03:15:26.618336+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:52:06.644022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.310981+0000","last_change":"2023-09-23T03:15:21.160191+0000","last_active":"2023-09-23T03:15:25.310981+0000","last_peered":"2023-09-23T03:15:25.310981+0000","last_clean":"2023-09-23T03:15:25.310981+0000","last_became_active":"2023-09-23T03:15:21.159841+0000","last_became_peered":"2023-09-23T03:15:21.159841+0000","last_unstale":"2023-09-23T03:15:25.310981+0000","last_undegraded":"2023-09-23T03:15:25.310981+0000","last_fullsized":"2023-09-23T03:15:25.310981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:23:07.611380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.301033+0000","last_change":"2023-09-23T03:15:20.242692+0000","last_active":"2023-09-23T03:15:25.301033+0000","last_peered":"2023-09-23T03:15:25.301033+0000","last_clean":"2023-09-23T03:15:25.301033+0000","last_became_active":"2023-09-23T03:15:20.242383+0000","last_became_peered":"2023-09-23T03:15:20.242383+0000","last_unstale":"2023-09-23T03:15:25.301033+0000","last_undegraded":"2023-09-23T03:15:25.301033+0000","last_fullsized":"2023-09-23T03:15:25.301033+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:35:16.587399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.619952+0000","last_change":"2023-09-23T03:15:25.330531+0000","last_active":"2023-09-23T03:15:26.619952+0000","last_peered":"2023-09-23T03:15:26.619952+0000","last_clean":"2023-09-23T03:15:26.619952+0000","last_became_active":"2023-09-23T03:15:25.330145+0000","last_became_peered":"2023-09-23T03:15:25.330145+0000","last_unstale":"2023-09-23T03:15:26.619952+0000","last_undegraded":"2023-09-23T03:15:26.619952+0000","last_fullsized":"2023-09-23T03:15:26.619952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:44:24.602333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.302012+0000","last_change":"2023-09-23T03:15:21.152726+0000","last_active":"2023-09-23T03:15:25.302012+0000","last_peered":"2023-09-23T03:15:25.302012+0000","last_clean":"2023-09-23T03:15:25.302012+0000","last_became_active":"2023-09-23T03:15:21.151916+0000","last_became_peered":"2023-09-23T03:15:21.151916+0000","last_unstale":"2023-09-23T03:15:25.302012+0000","last_undegraded":"2023-09-23T03:15:25.302012+0000","last_fullsized":"2023-09-23T03:15:25.302012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:05:49.631098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.441189+0000","last_change":"2023-09-23T03:15:20.265672+0000","last_active":"2023-09-23T03:15:25.441189+0000","last_peered":"2023-09-23T03:15:25.441189+0000","last_clean":"2023-09-23T03:15:25.441189+0000","last_became_active":"2023-09-23T03:15:20.265172+0000","last_became_peered":"2023-09-23T03:15:20.265172+0000","last_unstale":"2023-09-23T03:15:25.441189+0000","last_undegraded":"2023-09-23T03:15:25.441189+0000","last_fullsized":"2023-09-23T03:15:25.441189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:28:08.970374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.620019+0000","last_change":"2023-09-23T03:15:25.442450+0000","last_active":"2023-09-23T03:15:26.620019+0000","last_peered":"2023-09-23T03:15:26.620019+0000","last_clean":"2023-09-23T03:15:26.620019+0000","last_became_active":"2023-09-23T03:15:25.441911+0000","last_became_peered":"2023-09-23T03:15:25.441911+0000","last_unstale":"2023-09-23T03:15:26.620019+0000","last_undegraded":"2023-09-23T03:15:26.620019+0000","last_fullsized":"2023-09-23T03:15:26.620019+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:58:07.013813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.295933+0000","last_change":"2023-09-23T03:15:21.171351+0000","last_active":"2023-09-23T03:15:25.295933+0000","last_peered":"2023-09-23T03:15:25.295933+0000","last_clean":"2023-09-23T03:15:25.295933+0000","last_became_active":"2023-09-23T03:15:21.170977+0000","last_became_peered":"2023-09-23T03:15:21.170977+0000","last_unstale":"2023-09-23T03:15:25.295933+0000","last_undegraded":"2023-09-23T03:15:25.295933+0000","last_fullsized":"2023-09-23T03:15:25.295933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:51:00.098552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.314530+0000","last_change":"2023-09-23T03:15:20.258137+0000","last_active":"2023-09-23T03:15:25.314530+0000","last_peered":"2023-09-23T03:15:25.314530+0000","last_clean":"2023-09-23T03:15:25.314530+0000","last_became_active":"2023-09-23T03:15:20.257772+0000","last_became_peered":"2023-09-23T03:15:20.257772+0000","last_unstale":"2023-09-23T03:15:25.314530+0000","last_undegraded":"2023-09-23T03:15:25.314530+0000","last_fullsized":"2023-09-23T03:15:25.314530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:52:01.548454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":169,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.441463+0000","last_change":"2023-09-23T03:15:11.042447+0000","last_active":"2023-09-23T03:15:25.441463+0000","last_peered":"2023-09-23T03:15:25.441463+0000","last_clean":"2023-09-23T03:15:25.441463+0000","last_became_active":"2023-09-23T03:15:11.040322+0000","last_became_peered":"2023-09-23T03:15:11.040322+0000","last_unstale":"2023-09-23T03:15:25.441463+0000","last_undegraded":"2023-09-23T03:15:25.441463+0000","last_fullsized":"2023-09-23T03:15:25.441463+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_clean_scrub_stamp":"2023-09-23T03:15:09.474842+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:18:36.331645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.138753+0000","last_change":"2023-09-23T03:15:25.449566+0000","last_active":"2023-09-23T03:15:26.138753+0000","last_peered":"2023-09-23T03:15:26.138753+0000","last_clean":"2023-09-23T03:15:26.138753+0000","last_became_active":"2023-09-23T03:15:25.448514+0000","last_became_peered":"2023-09-23T03:15:25.448514+0000","last_unstale":"2023-09-23T03:15:26.138753+0000","last_undegraded":"2023-09-23T03:15:26.138753+0000","last_fullsized":"2023-09-23T03:15:26.138753+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:52:06.644022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.442985+0000","last_change":"2023-09-23T03:15:21.180427+0000","last_active":"2023-09-23T03:15:25.442985+0000","last_peered":"2023-09-23T03:15:25.442985+0000","last_clean":"2023-09-23T03:15:25.442985+0000","last_became_active":"2023-09-23T03:15:21.179941+0000","last_became_peered":"2023-09-23T03:15:21.179941+0000","last_unstale":"2023-09-23T03:15:25.442985+0000","last_undegraded":"2023-09-23T03:15:25.442985+0000","last_fullsized":"2023-09-23T03:15:25.442985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:53:21.435687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.302809+0000","last_change":"2023-09-23T03:15:20.259280+0000","last_active":"2023-09-23T03:15:25.302809+0000","last_peered":"2023-09-23T03:15:25.302809+0000","last_clean":"2023-09-23T03:15:25.302809+0000","last_became_active":"2023-09-23T03:15:20.258990+0000","last_became_peered":"2023-09-23T03:15:20.258990+0000","last_unstale":"2023-09-23T03:15:25.302809+0000","last_undegraded":"2023-09-23T03:15:25.302809+0000","last_fullsized":"2023-09-23T03:15:25.302809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:32:41.575031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.137662+0000","last_change":"2023-09-23T03:15:25.448939+0000","last_active":"2023-09-23T03:15:26.137662+0000","last_peered":"2023-09-23T03:15:26.137662+0000","last_clean":"2023-09-23T03:15:26.137662+0000","last_became_active":"2023-09-23T03:15:25.448129+0000","last_became_peered":"2023-09-23T03:15:25.448129+0000","last_unstale":"2023-09-23T03:15:26.137662+0000","last_undegraded":"2023-09-23T03:15:26.137662+0000","last_fullsized":"2023-09-23T03:15:26.137662+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:42:41.516665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315261+0000","last_change":"2023-09-23T03:15:21.172056+0000","last_active":"2023-09-23T03:15:25.315261+0000","last_peered":"2023-09-23T03:15:25.315261+0000","last_clean":"2023-09-23T03:15:25.315261+0000","last_became_active":"2023-09-23T03:15:21.170669+0000","last_became_peered":"2023-09-23T03:15:21.170669+0000","last_unstale":"2023-09-23T03:15:25.315261+0000","last_undegraded":"2023-09-23T03:15:25.315261+0000","last_fullsized":"2023-09-23T03:15:25.315261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:02:09.034422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.312606+0000","last_change":"2023-09-23T03:15:20.254984+0000","last_active":"2023-09-23T03:15:25.312606+0000","last_peered":"2023-09-23T03:15:25.312606+0000","last_clean":"2023-09-23T03:15:25.312606+0000","last_became_active":"2023-09-23T03:15:20.254425+0000","last_became_peered":"2023-09-23T03:15:20.254425+0000","last_unstale":"2023-09-23T03:15:25.312606+0000","last_undegraded":"2023-09-23T03:15:25.312606+0000","last_fullsized":"2023-09-23T03:15:25.312606+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:13:21.067891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.619714+0000","last_change":"2023-09-23T03:15:25.324169+0000","last_active":"2023-09-23T03:15:26.619714+0000","last_peered":"2023-09-23T03:15:26.619714+0000","last_clean":"2023-09-23T03:15:26.619714+0000","last_became_active":"2023-09-23T03:15:25.323647+0000","last_became_peered":"2023-09-23T03:15:25.323647+0000","last_unstale":"2023-09-23T03:15:26.619714+0000","last_undegraded":"2023-09-23T03:15:26.619714+0000","last_fullsized":"2023-09-23T03:15:26.619714+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:44:24.602333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.446178+0000","last_change":"2023-09-23T03:15:21.175250+0000","last_active":"2023-09-23T03:15:25.446178+0000","last_peered":"2023-09-23T03:15:25.446178+0000","last_clean":"2023-09-23T03:15:25.446178+0000","last_became_active":"2023-09-23T03:15:21.174890+0000","last_became_peered":"2023-09-23T03:15:21.174890+0000","last_unstale":"2023-09-23T03:15:25.446178+0000","last_undegraded":"2023-09-23T03:15:25.446178+0000","last_fullsized":"2023-09-23T03:15:25.446178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:03:55.070789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.301471+0000","last_change":"2023-09-23T03:15:20.242773+0000","last_active":"2023-09-23T03:15:25.301471+0000","last_peered":"2023-09-23T03:15:25.301471+0000","last_clean":"2023-09-23T03:15:25.301471+0000","last_became_active":"2023-09-23T03:15:20.242466+0000","last_became_peered":"2023-09-23T03:15:20.242466+0000","last_unstale":"2023-09-23T03:15:25.301471+0000","last_undegraded":"2023-09-23T03:15:25.301471+0000","last_fullsized":"2023-09-23T03:15:25.301471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:52:01.548454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.617142+0000","last_change":"2023-09-23T03:15:25.452683+0000","last_active":"2023-09-23T03:15:26.617142+0000","last_peered":"2023-09-23T03:15:26.617142+0000","last_clean":"2023-09-23T03:15:26.617142+0000","last_became_active":"2023-09-23T03:15:25.451509+0000","last_became_peered":"2023-09-23T03:15:25.451509+0000","last_unstale":"2023-09-23T03:15:26.617142+0000","last_undegraded":"2023-09-23T03:15:26.617142+0000","last_fullsized":"2023-09-23T03:15:26.617142+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:03:59.144070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.443716+0000","last_change":"2023-09-23T03:15:21.177994+0000","last_active":"2023-09-23T03:15:25.443716+0000","last_peered":"2023-09-23T03:15:25.443716+0000","last_clean":"2023-09-23T03:15:25.443716+0000","last_became_active":"2023-09-23T03:15:21.177752+0000","last_became_peered":"2023-09-23T03:15:21.177752+0000","last_unstale":"2023-09-23T03:15:25.443716+0000","last_undegraded":"2023-09-23T03:15:25.443716+0000","last_fullsized":"2023-09-23T03:15:25.443716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:52:02.570741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.309999+0000","last_change":"2023-09-23T03:15:20.245089+0000","last_active":"2023-09-23T03:15:25.309999+0000","last_peered":"2023-09-23T03:15:25.309999+0000","last_clean":"2023-09-23T03:15:25.309999+0000","last_became_active":"2023-09-23T03:15:20.243281+0000","last_became_peered":"2023-09-23T03:15:20.243281+0000","last_unstale":"2023-09-23T03:15:25.309999+0000","last_undegraded":"2023-09-23T03:15:25.309999+0000","last_fullsized":"2023-09-23T03:15:25.309999+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:34:50.786275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.137112+0000","last_change":"2023-09-23T03:15:25.327837+0000","last_active":"2023-09-23T03:15:26.137112+0000","last_peered":"2023-09-23T03:15:26.137112+0000","last_clean":"2023-09-23T03:15:26.137112+0000","last_became_active":"2023-09-23T03:15:25.327245+0000","last_became_peered":"2023-09-23T03:15:25.327245+0000","last_unstale":"2023-09-23T03:15:26.137112+0000","last_undegraded":"2023-09-23T03:15:26.137112+0000","last_fullsized":"2023-09-23T03:15:26.137112+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:15:23.657884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.441989+0000","last_change":"2023-09-23T03:15:21.173588+0000","last_active":"2023-09-23T03:15:25.441989+0000","last_peered":"2023-09-23T03:15:25.441989+0000","last_clean":"2023-09-23T03:15:25.441989+0000","last_became_active":"2023-09-23T03:15:21.173284+0000","last_became_peered":"2023-09-23T03:15:21.173284+0000","last_unstale":"2023-09-23T03:15:25.441989+0000","last_undegraded":"2023-09-23T03:15:25.441989+0000","last_fullsized":"2023-09-23T03:15:25.441989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:43:35.707241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.302930+0000","last_change":"2023-09-23T03:15:20.249033+0000","last_active":"2023-09-23T03:15:25.302930+0000","last_peered":"2023-09-23T03:15:25.302930+0000","last_clean":"2023-09-23T03:15:25.302930+0000","last_became_active":"2023-09-23T03:15:20.248738+0000","last_became_peered":"2023-09-23T03:15:20.248738+0000","last_unstale":"2023-09-23T03:15:25.302930+0000","last_undegraded":"2023-09-23T03:15:25.302930+0000","last_fullsized":"2023-09-23T03:15:25.302930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:34:53.736954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.619712+0000","last_change":"2023-09-23T03:15:25.328800+0000","last_active":"2023-09-23T03:15:26.619712+0000","last_peered":"2023-09-23T03:15:26.619712+0000","last_clean":"2023-09-23T03:15:26.619712+0000","last_became_active":"2023-09-23T03:15:25.327349+0000","last_became_peered":"2023-09-23T03:15:25.327349+0000","last_unstale":"2023-09-23T03:15:26.619712+0000","last_undegraded":"2023-09-23T03:15:26.619712+0000","last_fullsized":"2023-09-23T03:15:26.619712+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:02:13.107703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.306109+0000","last_change":"2023-09-23T03:15:21.156710+0000","last_active":"2023-09-23T03:15:25.306109+0000","last_peered":"2023-09-23T03:15:25.306109+0000","last_clean":"2023-09-23T03:15:25.306109+0000","last_became_active":"2023-09-23T03:15:21.156344+0000","last_became_peered":"2023-09-23T03:15:21.156344+0000","last_unstale":"2023-09-23T03:15:25.306109+0000","last_undegraded":"2023-09-23T03:15:25.306109+0000","last_fullsized":"2023-09-23T03:15:25.306109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:01:05.010735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.307353+0000","last_change":"2023-09-23T03:15:20.254568+0000","last_active":"2023-09-23T03:15:25.307353+0000","last_peered":"2023-09-23T03:15:25.307353+0000","last_clean":"2023-09-23T03:15:25.307353+0000","last_became_active":"2023-09-23T03:15:20.254294+0000","last_became_peered":"2023-09-23T03:15:20.254294+0000","last_unstale":"2023-09-23T03:15:25.307353+0000","last_undegraded":"2023-09-23T03:15:25.307353+0000","last_fullsized":"2023-09-23T03:15:25.307353+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:32:41.575031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.487038+0000","last_change":"2023-09-23T03:15:25.444588+0000","last_active":"2023-09-23T03:15:26.487038+0000","last_peered":"2023-09-23T03:15:26.487038+0000","last_clean":"2023-09-23T03:15:26.487038+0000","last_became_active":"2023-09-23T03:15:25.443847+0000","last_became_peered":"2023-09-23T03:15:25.443847+0000","last_unstale":"2023-09-23T03:15:26.487038+0000","last_undegraded":"2023-09-23T03:15:26.487038+0000","last_fullsized":"2023-09-23T03:15:26.487038+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:54:37.479094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.441909+0000","last_change":"2023-09-23T03:15:21.172873+0000","last_active":"2023-09-23T03:15:25.441909+0000","last_peered":"2023-09-23T03:15:25.441909+0000","last_clean":"2023-09-23T03:15:25.441909+0000","last_became_active":"2023-09-23T03:15:21.172552+0000","last_became_peered":"2023-09-23T03:15:21.172552+0000","last_unstale":"2023-09-23T03:15:25.441909+0000","last_undegraded":"2023-09-23T03:15:25.441909+0000","last_fullsized":"2023-09-23T03:15:25.441909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:42:37.443384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.317383+0000","last_change":"2023-09-23T03:15:20.257041+0000","last_active":"2023-09-23T03:15:25.317383+0000","last_peered":"2023-09-23T03:15:25.317383+0000","last_clean":"2023-09-23T03:15:25.317383+0000","last_became_active":"2023-09-23T03:15:20.256737+0000","last_became_peered":"2023-09-23T03:15:20.256737+0000","last_unstale":"2023-09-23T03:15:25.317383+0000","last_undegraded":"2023-09-23T03:15:25.317383+0000","last_fullsized":"2023-09-23T03:15:25.317383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:10:06.381357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.486900+0000","last_change":"2023-09-23T03:15:25.445742+0000","last_active":"2023-09-23T03:15:26.486900+0000","last_peered":"2023-09-23T03:15:26.486900+0000","last_clean":"2023-09-23T03:15:26.486900+0000","last_became_active":"2023-09-23T03:15:25.445297+0000","last_became_peered":"2023-09-23T03:15:25.445297+0000","last_unstale":"2023-09-23T03:15:26.486900+0000","last_undegraded":"2023-09-23T03:15:26.486900+0000","last_fullsized":"2023-09-23T03:15:26.486900+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:36:02.944327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.301638+0000","last_change":"2023-09-23T03:15:21.166987+0000","last_active":"2023-09-23T03:15:25.301638+0000","last_peered":"2023-09-23T03:15:25.301638+0000","last_clean":"2023-09-23T03:15:25.301638+0000","last_became_active":"2023-09-23T03:15:21.166575+0000","last_became_peered":"2023-09-23T03:15:21.166575+0000","last_unstale":"2023-09-23T03:15:25.301638+0000","last_undegraded":"2023-09-23T03:15:25.301638+0000","last_fullsized":"2023-09-23T03:15:25.301638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:10:07.393421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.305489+0000","last_change":"2023-09-23T03:15:20.253573+0000","last_active":"2023-09-23T03:15:25.305489+0000","last_peered":"2023-09-23T03:15:25.305489+0000","last_clean":"2023-09-23T03:15:25.305489+0000","last_became_active":"2023-09-23T03:15:20.253228+0000","last_became_peered":"2023-09-23T03:15:20.253228+0000","last_unstale":"2023-09-23T03:15:25.305489+0000","last_undegraded":"2023-09-23T03:15:25.305489+0000","last_fullsized":"2023-09-23T03:15:25.305489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:35:16.587399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.619960+0000","last_change":"2023-09-23T03:15:25.453715+0000","last_active":"2023-09-23T03:15:26.619960+0000","last_peered":"2023-09-23T03:15:26.619960+0000","last_clean":"2023-09-23T03:15:26.619960+0000","last_became_active":"2023-09-23T03:15:25.452979+0000","last_became_peered":"2023-09-23T03:15:25.452979+0000","last_unstale":"2023-09-23T03:15:26.619960+0000","last_undegraded":"2023-09-23T03:15:26.619960+0000","last_fullsized":"2023-09-23T03:15:26.619960+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:15:29.393607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.442638+0000","last_change":"2023-09-23T03:15:21.169871+0000","last_active":"2023-09-23T03:15:25.442638+0000","last_peered":"2023-09-23T03:15:25.442638+0000","last_clean":"2023-09-23T03:15:25.442638+0000","last_became_active":"2023-09-23T03:15:21.169539+0000","last_became_peered":"2023-09-23T03:15:21.169539+0000","last_unstale":"2023-09-23T03:15:25.442638+0000","last_undegraded":"2023-09-23T03:15:25.442638+0000","last_fullsized":"2023-09-23T03:15:25.442638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:44:20.529052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:40.246064+0000","last_change":"2023-09-23T03:15:20.252281+0000","last_active":"2023-09-23T03:15:40.246064+0000","last_peered":"2023-09-23T03:15:40.246064+0000","last_clean":"2023-09-23T03:15:40.246064+0000","last_became_active":"2023-09-23T03:15:20.251982+0000","last_became_peered":"2023-09-23T03:15:20.251982+0000","last_unstale":"2023-09-23T03:15:40.246064+0000","last_undegraded":"2023-09-23T03:15:40.246064+0000","last_fullsized":"2023-09-23T03:15:40.246064+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:35:16.587399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.616880+0000","last_change":"2023-09-23T03:15:25.325976+0000","last_active":"2023-09-23T03:15:26.616880+0000","last_peered":"2023-09-23T03:15:26.616880+0000","last_clean":"2023-09-23T03:15:26.616880+0000","last_became_active":"2023-09-23T03:15:25.325613+0000","last_became_peered":"2023-09-23T03:15:25.325613+0000","last_unstale":"2023-09-23T03:15:26.616880+0000","last_undegraded":"2023-09-23T03:15:26.616880+0000","last_fullsized":"2023-09-23T03:15:26.616880+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:53:25.508968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.309427+0000","last_change":"2023-09-23T03:15:21.166420+0000","last_active":"2023-09-23T03:15:25.309427+0000","last_peered":"2023-09-23T03:15:25.309427+0000","last_clean":"2023-09-23T03:15:25.309427+0000","last_became_active":"2023-09-23T03:15:21.166086+0000","last_became_peered":"2023-09-23T03:15:21.166086+0000","last_unstale":"2023-09-23T03:15:25.309427+0000","last_undegraded":"2023-09-23T03:15:25.309427+0000","last_fullsized":"2023-09-23T03:15:25.309427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:13:22.079955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.312418+0000","last_change":"2023-09-23T03:15:20.263920+0000","last_active":"2023-09-23T03:15:25.312418+0000","last_peered":"2023-09-23T03:15:25.312418+0000","last_clean":"2023-09-23T03:15:25.312418+0000","last_became_active":"2023-09-23T03:15:20.263623+0000","last_became_peered":"2023-09-23T03:15:20.263623+0000","last_unstale":"2023-09-23T03:15:25.312418+0000","last_undegraded":"2023-09-23T03:15:25.312418+0000","last_fullsized":"2023-09-23T03:15:25.312418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:28:08.970374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.139553+0000","last_change":"2023-09-23T03:15:25.329301+0000","last_active":"2023-09-23T03:15:26.139553+0000","last_peered":"2023-09-23T03:15:26.139553+0000","last_clean":"2023-09-23T03:15:26.139553+0000","last_became_active":"2023-09-23T03:15:25.328559+0000","last_became_peered":"2023-09-23T03:15:25.328559+0000","last_unstale":"2023-09-23T03:15:26.139553+0000","last_undegraded":"2023-09-23T03:15:26.139553+0000","last_fullsized":"2023-09-23T03:15:26.139553+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:43:39.780522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.303426+0000","last_change":"2023-09-23T03:15:21.172339+0000","last_active":"2023-09-23T03:15:25.303426+0000","last_peered":"2023-09-23T03:15:25.303426+0000","last_clean":"2023-09-23T03:15:25.303426+0000","last_became_active":"2023-09-23T03:15:21.172034+0000","last_became_peered":"2023-09-23T03:15:21.172034+0000","last_unstale":"2023-09-23T03:15:25.303426+0000","last_undegraded":"2023-09-23T03:15:25.303426+0000","last_fullsized":"2023-09-23T03:15:25.303426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:21:27.583218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315333+0000","last_change":"2023-09-23T03:15:20.259342+0000","last_active":"2023-09-23T03:15:25.315333+0000","last_peered":"2023-09-23T03:15:25.315333+0000","last_clean":"2023-09-23T03:15:25.315333+0000","last_became_active":"2023-09-23T03:15:20.258990+0000","last_became_peered":"2023-09-23T03:15:20.258990+0000","last_unstale":"2023-09-23T03:15:25.315333+0000","last_undegraded":"2023-09-23T03:15:25.315333+0000","last_fullsized":"2023-09-23T03:15:25.315333+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:50:59.086488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.485843+0000","last_change":"2023-09-23T03:15:25.322201+0000","last_active":"2023-09-23T03:15:26.485843+0000","last_peered":"2023-09-23T03:15:26.485843+0000","last_clean":"2023-09-23T03:15:26.485843+0000","last_became_active":"2023-09-23T03:15:25.321486+0000","last_became_peered":"2023-09-23T03:15:25.321486+0000","last_unstale":"2023-09-23T03:15:26.485843+0000","last_undegraded":"2023-09-23T03:15:26.485843+0000","last_fullsized":"2023-09-23T03:15:26.485843+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:20:20.193299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315253+0000","last_change":"2023-09-23T03:15:21.175465+0000","last_active":"2023-09-23T03:15:25.315253+0000","last_peered":"2023-09-23T03:15:25.315253+0000","last_clean":"2023-09-23T03:15:25.315253+0000","last_became_active":"2023-09-23T03:15:21.171773+0000","last_became_peered":"2023-09-23T03:15:21.171773+0000","last_unstale":"2023-09-23T03:15:25.315253+0000","last_undegraded":"2023-09-23T03:15:25.315253+0000","last_fullsized":"2023-09-23T03:15:25.315253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:39:10.586582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.313666+0000","last_change":"2023-09-23T03:15:20.259404+0000","last_active":"2023-09-23T03:15:25.313666+0000","last_peered":"2023-09-23T03:15:25.313666+0000","last_clean":"2023-09-23T03:15:25.313666+0000","last_became_active":"2023-09-23T03:15:20.259013+0000","last_became_peered":"2023-09-23T03:15:20.259013+0000","last_unstale":"2023-09-23T03:15:25.313666+0000","last_undegraded":"2023-09-23T03:15:25.313666+0000","last_fullsized":"2023-09-23T03:15:25.313666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:41:37.505547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.485703+0000","last_change":"2023-09-23T03:15:25.452370+0000","last_active":"2023-09-23T03:15:26.485703+0000","last_peered":"2023-09-23T03:15:26.485703+0000","last_clean":"2023-09-23T03:15:26.485703+0000","last_became_active":"2023-09-23T03:15:25.451873+0000","last_became_peered":"2023-09-23T03:15:25.451873+0000","last_unstale":"2023-09-23T03:15:26.485703+0000","last_undegraded":"2023-09-23T03:15:26.485703+0000","last_fullsized":"2023-09-23T03:15:26.485703+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:58:52.853121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.619356+0000","last_change":"2023-09-23T03:15:25.452394+0000","last_active":"2023-09-23T03:15:26.619356+0000","last_peered":"2023-09-23T03:15:26.619356+0000","last_clean":"2023-09-23T03:15:26.619356+0000","last_became_active":"2023-09-23T03:15:25.451251+0000","last_became_peered":"2023-09-23T03:15:25.451251+0000","last_unstale":"2023-09-23T03:15:26.619356+0000","last_undegraded":"2023-09-23T03:15:26.619356+0000","last_fullsized":"2023-09-23T03:15:26.619356+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:52:06.644022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.310954+0000","last_change":"2023-09-23T03:15:21.170461+0000","last_active":"2023-09-23T03:15:25.310954+0000","last_peered":"2023-09-23T03:15:25.310954+0000","last_clean":"2023-09-23T03:15:25.310954+0000","last_became_active":"2023-09-23T03:15:21.170095+0000","last_became_peered":"2023-09-23T03:15:21.170095+0000","last_unstale":"2023-09-23T03:15:25.310954+0000","last_undegraded":"2023-09-23T03:15:25.310954+0000","last_fullsized":"2023-09-23T03:15:25.310954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:30:31.951544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.314895+0000","last_change":"2023-09-23T03:15:20.257662+0000","last_active":"2023-09-23T03:15:25.314895+0000","last_peered":"2023-09-23T03:15:25.314895+0000","last_clean":"2023-09-23T03:15:25.314895+0000","last_became_active":"2023-09-23T03:15:20.257376+0000","last_became_peered":"2023-09-23T03:15:20.257376+0000","last_unstale":"2023-09-23T03:15:25.314895+0000","last_undegraded":"2023-09-23T03:15:25.314895+0000","last_fullsized":"2023-09-23T03:15:25.314895+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:06:15.942585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.138746+0000","last_change":"2023-09-23T03:15:25.326388+0000","last_active":"2023-09-23T03:15:26.138746+0000","last_peered":"2023-09-23T03:15:26.138746+0000","last_clean":"2023-09-23T03:15:26.138746+0000","last_became_active":"2023-09-23T03:15:25.326018+0000","last_became_peered":"2023-09-23T03:15:25.326018+0000","last_unstale":"2023-09-23T03:15:26.138746+0000","last_undegraded":"2023-09-23T03:15:26.138746+0000","last_fullsized":"2023-09-23T03:15:26.138746+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:53:25.508968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.295747+0000","last_change":"2023-09-23T03:15:21.164139+0000","last_active":"2023-09-23T03:15:25.295747+0000","last_peered":"2023-09-23T03:15:25.295747+0000","last_clean":"2023-09-23T03:15:25.295747+0000","last_became_active":"2023-09-23T03:15:21.162300+0000","last_became_peered":"2023-09-23T03:15:21.162300+0000","last_unstale":"2023-09-23T03:15:25.295747+0000","last_undegraded":"2023-09-23T03:15:25.295747+0000","last_fullsized":"2023-09-23T03:15:25.295747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:22:07.336440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.298007+0000","last_change":"2023-09-23T03:15:20.256962+0000","last_active":"2023-09-23T03:15:25.298007+0000","last_peered":"2023-09-23T03:15:25.298007+0000","last_clean":"2023-09-23T03:15:25.298007+0000","last_became_active":"2023-09-23T03:15:20.256686+0000","last_became_peered":"2023-09-23T03:15:20.256686+0000","last_unstale":"2023-09-23T03:15:25.298007+0000","last_undegraded":"2023-09-23T03:15:25.298007+0000","last_fullsized":"2023-09-23T03:15:25.298007+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:06:15.942585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.314864+0000","last_change":"2023-09-23T03:15:20.255630+0000","last_active":"2023-09-23T03:15:25.314864+0000","last_peered":"2023-09-23T03:15:25.314864+0000","last_clean":"2023-09-23T03:15:25.314864+0000","last_became_active":"2023-09-23T03:15:20.255212+0000","last_became_peered":"2023-09-23T03:15:20.255212+0000","last_unstale":"2023-09-23T03:15:25.314864+0000","last_undegraded":"2023-09-23T03:15:25.314864+0000","last_fullsized":"2023-09-23T03:15:25.314864+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:31:01.946245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.305281+0000","last_change":"2023-09-23T03:15:21.156720+0000","last_active":"2023-09-23T03:15:25.305281+0000","last_peered":"2023-09-23T03:15:25.305281+0000","last_clean":"2023-09-23T03:15:25.305281+0000","last_became_active":"2023-09-23T03:15:21.156268+0000","last_became_peered":"2023-09-23T03:15:21.156268+0000","last_unstale":"2023-09-23T03:15:25.305281+0000","last_undegraded":"2023-09-23T03:15:25.305281+0000","last_fullsized":"2023-09-23T03:15:25.305281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:51:00.098552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.134456+0000","last_change":"2023-09-23T03:15:25.319168+0000","last_active":"2023-09-23T03:15:26.134456+0000","last_peered":"2023-09-23T03:15:26.134456+0000","last_clean":"2023-09-23T03:15:26.134456+0000","last_became_active":"2023-09-23T03:15:25.318761+0000","last_became_peered":"2023-09-23T03:15:25.318761+0000","last_unstale":"2023-09-23T03:15:26.134456+0000","last_undegraded":"2023-09-23T03:15:26.134456+0000","last_fullsized":"2023-09-23T03:15:26.134456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:05:53.704379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.296655+0000","last_change":"2023-09-23T03:15:20.261879+0000","last_active":"2023-09-23T03:15:25.296655+0000","last_peered":"2023-09-23T03:15:25.296655+0000","last_clean":"2023-09-23T03:15:25.296655+0000","last_became_active":"2023-09-23T03:15:20.261597+0000","last_became_peered":"2023-09-23T03:15:20.261597+0000","last_unstale":"2023-09-23T03:15:25.296655+0000","last_undegraded":"2023-09-23T03:15:25.296655+0000","last_fullsized":"2023-09-23T03:15:25.296655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:41:37.505547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.311031+0000","last_change":"2023-09-23T03:15:21.161920+0000","last_active":"2023-09-23T03:15:25.311031+0000","last_peered":"2023-09-23T03:15:25.311031+0000","last_clean":"2023-09-23T03:15:25.311031+0000","last_became_active":"2023-09-23T03:15:21.161657+0000","last_became_peered":"2023-09-23T03:15:21.161657+0000","last_unstale":"2023-09-23T03:15:25.311031+0000","last_undegraded":"2023-09-23T03:15:25.311031+0000","last_fullsized":"2023-09-23T03:15:25.311031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:35:17.599463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.486488+0000","last_change":"2023-09-23T03:15:25.319176+0000","last_active":"2023-09-23T03:15:26.486488+0000","last_peered":"2023-09-23T03:15:26.486488+0000","last_clean":"2023-09-23T03:15:26.486488+0000","last_became_active":"2023-09-23T03:15:25.318752+0000","last_became_peered":"2023-09-23T03:15:25.318752+0000","last_unstale":"2023-09-23T03:15:26.486488+0000","last_undegraded":"2023-09-23T03:15:26.486488+0000","last_fullsized":"2023-09-23T03:15:26.486488+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:56:54.725774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.305274+0000","last_change":"2023-09-23T03:15:21.157543+0000","last_active":"2023-09-23T03:15:25.305274+0000","last_peered":"2023-09-23T03:15:25.305274+0000","last_clean":"2023-09-23T03:15:25.305274+0000","last_became_active":"2023-09-23T03:15:21.157240+0000","last_became_peered":"2023-09-23T03:15:21.157240+0000","last_unstale":"2023-09-23T03:15:25.305274+0000","last_undegraded":"2023-09-23T03:15:25.305274+0000","last_fullsized":"2023-09-23T03:15:25.305274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:23:07.611380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.300872+0000","last_change":"2023-09-23T03:15:21.176611+0000","last_active":"2023-09-23T03:15:25.300872+0000","last_peered":"2023-09-23T03:15:25.300872+0000","last_clean":"2023-09-23T03:15:25.300872+0000","last_became_active":"2023-09-23T03:15:21.176316+0000","last_became_peered":"2023-09-23T03:15:21.176316+0000","last_unstale":"2023-09-23T03:15:25.300872+0000","last_undegraded":"2023-09-23T03:15:25.300872+0000","last_fullsized":"2023-09-23T03:15:25.300872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:28:09.982438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.316926+0000","last_change":"2023-09-23T03:15:21.166808+0000","last_active":"2023-09-23T03:15:25.316926+0000","last_peered":"2023-09-23T03:15:25.316926+0000","last_clean":"2023-09-23T03:15:25.316926+0000","last_became_active":"2023-09-23T03:15:21.166488+0000","last_became_peered":"2023-09-23T03:15:21.166488+0000","last_unstale":"2023-09-23T03:15:25.316926+0000","last_undegraded":"2023-09-23T03:15:25.316926+0000","last_fullsized":"2023-09-23T03:15:25.316926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:43:35.707241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.305111+0000","last_change":"2023-09-23T03:15:21.162188+0000","last_active":"2023-09-23T03:15:25.305111+0000","last_peered":"2023-09-23T03:15:25.305111+0000","last_clean":"2023-09-23T03:15:25.305111+0000","last_became_active":"2023-09-23T03:15:21.161886+0000","last_became_peered":"2023-09-23T03:15:21.161886+0000","last_unstale":"2023-09-23T03:15:25.305111+0000","last_undegraded":"2023-09-23T03:15:25.305111+0000","last_fullsized":"2023-09-23T03:15:25.305111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:23:07.611380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.296970+0000","last_change":"2023-09-23T03:15:21.161567+0000","last_active":"2023-09-23T03:15:25.296970+0000","last_peered":"2023-09-23T03:15:25.296970+0000","last_clean":"2023-09-23T03:15:25.296970+0000","last_became_active":"2023-09-23T03:15:21.161168+0000","last_became_peered":"2023-09-23T03:15:21.161168+0000","last_unstale":"2023-09-23T03:15:25.296970+0000","last_undegraded":"2023-09-23T03:15:25.296970+0000","last_fullsized":"2023-09-23T03:15:25.296970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:30:37.652428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.311814+0000","last_change":"2023-09-23T03:15:21.164997+0000","last_active":"2023-09-23T03:15:25.311814+0000","last_peered":"2023-09-23T03:15:25.311814+0000","last_clean":"2023-09-23T03:15:25.311814+0000","last_became_active":"2023-09-23T03:15:21.164680+0000","last_became_peered":"2023-09-23T03:15:21.164680+0000","last_unstale":"2023-09-23T03:15:25.311814+0000","last_undegraded":"2023-09-23T03:15:25.311814+0000","last_fullsized":"2023-09-23T03:15:25.311814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:15:19.584603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.316265+0000","last_change":"2023-09-23T03:15:21.164029+0000","last_active":"2023-09-23T03:15:25.316265+0000","last_peered":"2023-09-23T03:15:25.316265+0000","last_clean":"2023-09-23T03:15:25.316265+0000","last_became_active":"2023-09-23T03:15:21.163673+0000","last_became_peered":"2023-09-23T03:15:21.163673+0000","last_unstale":"2023-09-23T03:15:25.316265+0000","last_undegraded":"2023-09-23T03:15:25.316265+0000","last_fullsized":"2023-09-23T03:15:25.316265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:15:19.584603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.297434+0000","last_change":"2023-09-23T03:15:21.163996+0000","last_active":"2023-09-23T03:15:25.297434+0000","last_peered":"2023-09-23T03:15:25.297434+0000","last_clean":"2023-09-23T03:15:25.297434+0000","last_became_active":"2023-09-23T03:15:21.163674+0000","last_became_peered":"2023-09-23T03:15:21.163674+0000","last_unstale":"2023-09-23T03:15:25.297434+0000","last_undegraded":"2023-09-23T03:15:25.297434+0000","last_fullsized":"2023-09-23T03:15:25.297434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:28:31.810059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.314809+0000","last_change":"2023-09-23T03:15:21.171741+0000","last_active":"2023-09-23T03:15:25.314809+0000","last_peered":"2023-09-23T03:15:25.314809+0000","last_clean":"2023-09-23T03:15:25.314809+0000","last_became_active":"2023-09-23T03:15:21.169357+0000","last_became_peered":"2023-09-23T03:15:21.169357+0000","last_unstale":"2023-09-23T03:15:25.314809+0000","last_undegraded":"2023-09-23T03:15:25.314809+0000","last_fullsized":"2023-09-23T03:15:25.314809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:21:27.583218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.297693+0000","last_change":"2023-09-23T03:15:21.163391+0000","last_active":"2023-09-23T03:15:25.297693+0000","last_peered":"2023-09-23T03:15:25.297693+0000","last_clean":"2023-09-23T03:15:25.297693+0000","last_became_active":"2023-09-23T03:15:21.162413+0000","last_became_peered":"2023-09-23T03:15:21.162413+0000","last_unstale":"2023-09-23T03:15:25.297693+0000","last_undegraded":"2023-09-23T03:15:25.297693+0000","last_fullsized":"2023-09-23T03:15:25.297693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:30:31.951544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315898+0000","last_change":"2023-09-23T03:15:21.171540+0000","last_active":"2023-09-23T03:15:25.315898+0000","last_peered":"2023-09-23T03:15:25.315898+0000","last_clean":"2023-09-23T03:15:25.315898+0000","last_became_active":"2023-09-23T03:15:21.167329+0000","last_became_peered":"2023-09-23T03:15:21.167329+0000","last_unstale":"2023-09-23T03:15:25.315898+0000","last_undegraded":"2023-09-23T03:15:25.315898+0000","last_fullsized":"2023-09-23T03:15:25.315898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:29:14.943541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.304917+0000","last_change":"2023-09-23T03:15:21.167575+0000","last_active":"2023-09-23T03:15:25.304917+0000","last_peered":"2023-09-23T03:15:25.304917+0000","last_clean":"2023-09-23T03:15:25.304917+0000","last_became_active":"2023-09-23T03:15:21.167260+0000","last_became_peered":"2023-09-23T03:15:21.167260+0000","last_unstale":"2023-09-23T03:15:25.304917+0000","last_undegraded":"2023-09-23T03:15:25.304917+0000","last_fullsized":"2023-09-23T03:15:25.304917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:01:05.010735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.314454+0000","last_change":"2023-09-23T03:15:21.151218+0000","last_active":"2023-09-23T03:15:25.314454+0000","last_peered":"2023-09-23T03:15:25.314454+0000","last_clean":"2023-09-23T03:15:25.314454+0000","last_became_active":"2023-09-23T03:15:21.150758+0000","last_became_peered":"2023-09-23T03:15:21.150758+0000","last_unstale":"2023-09-23T03:15:25.314454+0000","last_undegraded":"2023-09-23T03:15:25.314454+0000","last_fullsized":"2023-09-23T03:15:25.314454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:51:00.098552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.445962+0000","last_change":"2023-09-23T03:15:21.175081+0000","last_active":"2023-09-23T03:15:25.445962+0000","last_peered":"2023-09-23T03:15:25.445962+0000","last_clean":"2023-09-23T03:15:25.445962+0000","last_became_active":"2023-09-23T03:15:21.174757+0000","last_became_peered":"2023-09-23T03:15:21.174757+0000","last_unstale":"2023-09-23T03:15:25.445962+0000","last_undegraded":"2023-09-23T03:15:25.445962+0000","last_fullsized":"2023-09-23T03:15:25.445962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:02:09.034422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.301326+0000","last_change":"2023-09-23T03:15:21.165007+0000","last_active":"2023-09-23T03:15:25.301326+0000","last_peered":"2023-09-23T03:15:25.301326+0000","last_clean":"2023-09-23T03:15:25.301326+0000","last_became_active":"2023-09-23T03:15:21.164591+0000","last_became_peered":"2023-09-23T03:15:21.164591+0000","last_unstale":"2023-09-23T03:15:25.301326+0000","last_undegraded":"2023-09-23T03:15:25.301326+0000","last_fullsized":"2023-09-23T03:15:25.301326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:13:22.079955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315446+0000","last_change":"2023-09-23T03:15:21.166029+0000","last_active":"2023-09-23T03:15:25.315446+0000","last_peered":"2023-09-23T03:15:25.315446+0000","last_clean":"2023-09-23T03:15:25.315446+0000","last_became_active":"2023-09-23T03:15:21.165298+0000","last_became_peered":"2023-09-23T03:15:21.165298+0000","last_unstale":"2023-09-23T03:15:25.315446+0000","last_undegraded":"2023-09-23T03:15:25.315446+0000","last_fullsized":"2023-09-23T03:15:25.315446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:42:37.443384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.314713+0000","last_change":"2023-09-23T03:15:21.172091+0000","last_active":"2023-09-23T03:15:25.314713+0000","last_peered":"2023-09-23T03:15:25.314713+0000","last_clean":"2023-09-23T03:15:25.314713+0000","last_became_active":"2023-09-23T03:15:21.170229+0000","last_became_peered":"2023-09-23T03:15:21.170229+0000","last_unstale":"2023-09-23T03:15:25.314713+0000","last_undegraded":"2023-09-23T03:15:25.314713+0000","last_fullsized":"2023-09-23T03:15:25.314713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:15:19.584603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.311046+0000","last_change":"2023-09-23T03:15:21.163298+0000","last_active":"2023-09-23T03:15:25.311046+0000","last_peered":"2023-09-23T03:15:25.311046+0000","last_clean":"2023-09-23T03:15:25.311046+0000","last_became_active":"2023-09-23T03:15:21.163000+0000","last_became_peered":"2023-09-23T03:15:21.163000+0000","last_unstale":"2023-09-23T03:15:25.311046+0000","last_undegraded":"2023-09-23T03:15:25.311046+0000","last_fullsized":"2023-09-23T03:15:25.311046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:22:07.336440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.301412+0000","last_change":"2023-09-23T03:15:21.160829+0000","last_active":"2023-09-23T03:15:25.301412+0000","last_peered":"2023-09-23T03:15:25.301412+0000","last_clean":"2023-09-23T03:15:25.301412+0000","last_became_active":"2023-09-23T03:15:21.160567+0000","last_became_peered":"2023-09-23T03:15:21.160567+0000","last_unstale":"2023-09-23T03:15:25.301412+0000","last_undegraded":"2023-09-23T03:15:25.301412+0000","last_fullsized":"2023-09-23T03:15:25.301412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:06:16.954649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.296315+0000","last_change":"2023-09-23T03:15:21.165912+0000","last_active":"2023-09-23T03:15:25.296315+0000","last_peered":"2023-09-23T03:15:25.296315+0000","last_clean":"2023-09-23T03:15:25.296315+0000","last_became_active":"2023-09-23T03:15:21.165634+0000","last_became_peered":"2023-09-23T03:15:21.165634+0000","last_unstale":"2023-09-23T03:15:25.296315+0000","last_undegraded":"2023-09-23T03:15:25.296315+0000","last_fullsized":"2023-09-23T03:15:25.296315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:15:19.584603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.309291+0000","last_change":"2023-09-23T03:15:21.161315+0000","last_active":"2023-09-23T03:15:25.309291+0000","last_peered":"2023-09-23T03:15:25.309291+0000","last_clean":"2023-09-23T03:15:25.309291+0000","last_became_active":"2023-09-23T03:15:21.160978+0000","last_became_peered":"2023-09-23T03:15:21.160978+0000","last_unstale":"2023-09-23T03:15:25.309291+0000","last_undegraded":"2023-09-23T03:15:25.309291+0000","last_fullsized":"2023-09-23T03:15:25.309291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:28:09.982438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.305734+0000","last_change":"2023-09-23T03:15:21.172253+0000","last_active":"2023-09-23T03:15:25.305734+0000","last_peered":"2023-09-23T03:15:25.305734+0000","last_clean":"2023-09-23T03:15:25.305734+0000","last_became_active":"2023-09-23T03:15:21.171987+0000","last_became_peered":"2023-09-23T03:15:21.171987+0000","last_unstale":"2023-09-23T03:15:25.305734+0000","last_undegraded":"2023-09-23T03:15:25.305734+0000","last_fullsized":"2023-09-23T03:15:25.305734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:30:31.951544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.303805+0000","last_change":"2023-09-23T03:15:21.172883+0000","last_active":"2023-09-23T03:15:25.303805+0000","last_peered":"2023-09-23T03:15:25.303805+0000","last_clean":"2023-09-23T03:15:25.303805+0000","last_became_active":"2023-09-23T03:15:21.172404+0000","last_became_peered":"2023-09-23T03:15:21.172404+0000","last_unstale":"2023-09-23T03:15:25.303805+0000","last_undegraded":"2023-09-23T03:15:25.303805+0000","last_fullsized":"2023-09-23T03:15:25.303805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:21:27.583218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.445963+0000","last_change":"2023-09-23T03:15:21.174812+0000","last_active":"2023-09-23T03:15:25.445963+0000","last_peered":"2023-09-23T03:15:25.445963+0000","last_clean":"2023-09-23T03:15:25.445963+0000","last_became_active":"2023-09-23T03:15:21.174241+0000","last_became_peered":"2023-09-23T03:15:21.174241+0000","last_unstale":"2023-09-23T03:15:25.445963+0000","last_undegraded":"2023-09-23T03:15:25.445963+0000","last_fullsized":"2023-09-23T03:15:25.445963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:58:02.940532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.295571+0000","last_change":"2023-09-23T03:15:21.170905+0000","last_active":"2023-09-23T03:15:25.295571+0000","last_peered":"2023-09-23T03:15:25.295571+0000","last_clean":"2023-09-23T03:15:25.295571+0000","last_became_active":"2023-09-23T03:15:21.170570+0000","last_became_peered":"2023-09-23T03:15:21.170570+0000","last_unstale":"2023-09-23T03:15:25.295571+0000","last_undegraded":"2023-09-23T03:15:25.295571+0000","last_fullsized":"2023-09-23T03:15:25.295571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:23:07.611380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.441959+0000","last_change":"2023-09-23T03:15:21.171879+0000","last_active":"2023-09-23T03:15:25.441959+0000","last_peered":"2023-09-23T03:15:25.441959+0000","last_clean":"2023-09-23T03:15:25.441959+0000","last_became_active":"2023-09-23T03:15:21.171624+0000","last_became_peered":"2023-09-23T03:15:21.171624+0000","last_unstale":"2023-09-23T03:15:25.441959+0000","last_undegraded":"2023-09-23T03:15:25.441959+0000","last_fullsized":"2023-09-23T03:15:25.441959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:39:10.586582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315796+0000","last_change":"2023-09-23T03:15:21.177809+0000","last_active":"2023-09-23T03:15:25.315796+0000","last_peered":"2023-09-23T03:15:25.315796+0000","last_clean":"2023-09-23T03:15:25.315796+0000","last_became_active":"2023-09-23T03:15:21.175320+0000","last_became_peered":"2023-09-23T03:15:21.175320+0000","last_unstale":"2023-09-23T03:15:25.315796+0000","last_undegraded":"2023-09-23T03:15:25.315796+0000","last_fullsized":"2023-09-23T03:15:25.315796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:30:37.652428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.441779+0000","last_change":"2023-09-23T03:15:21.174361+0000","last_active":"2023-09-23T03:15:25.441779+0000","last_peered":"2023-09-23T03:15:25.441779+0000","last_clean":"2023-09-23T03:15:25.441779+0000","last_became_active":"2023-09-23T03:15:21.174101+0000","last_became_peered":"2023-09-23T03:15:21.174101+0000","last_unstale":"2023-09-23T03:15:25.441779+0000","last_undegraded":"2023-09-23T03:15:25.441779+0000","last_fullsized":"2023-09-23T03:15:25.441779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:25:53.924266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315094+0000","last_change":"2023-09-23T03:15:21.171837+0000","last_active":"2023-09-23T03:15:25.315094+0000","last_peered":"2023-09-23T03:15:25.315094+0000","last_clean":"2023-09-23T03:15:25.315094+0000","last_became_active":"2023-09-23T03:15:21.169973+0000","last_became_peered":"2023-09-23T03:15:21.169973+0000","last_unstale":"2023-09-23T03:15:25.315094+0000","last_undegraded":"2023-09-23T03:15:25.315094+0000","last_fullsized":"2023-09-23T03:15:25.315094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:53:21.435687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.303645+0000","last_change":"2023-09-23T03:15:21.157118+0000","last_active":"2023-09-23T03:15:25.303645+0000","last_peered":"2023-09-23T03:15:25.303645+0000","last_clean":"2023-09-23T03:15:25.303645+0000","last_became_active":"2023-09-23T03:15:21.156862+0000","last_became_peered":"2023-09-23T03:15:21.156862+0000","last_unstale":"2023-09-23T03:15:25.303645+0000","last_undegraded":"2023-09-23T03:15:25.303645+0000","last_fullsized":"2023-09-23T03:15:25.303645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:22:07.336440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.311650+0000","last_change":"2023-09-23T03:15:21.167539+0000","last_active":"2023-09-23T03:15:25.311650+0000","last_peered":"2023-09-23T03:15:25.311650+0000","last_clean":"2023-09-23T03:15:25.311650+0000","last_became_active":"2023-09-23T03:15:21.167256+0000","last_became_peered":"2023-09-23T03:15:21.167256+0000","last_unstale":"2023-09-23T03:15:25.311650+0000","last_undegraded":"2023-09-23T03:15:25.311650+0000","last_fullsized":"2023-09-23T03:15:25.311650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:21:27.583218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.445518+0000","last_change":"2023-09-23T03:15:21.171479+0000","last_active":"2023-09-23T03:15:25.445518+0000","last_peered":"2023-09-23T03:15:25.445518+0000","last_clean":"2023-09-23T03:15:25.445518+0000","last_became_active":"2023-09-23T03:15:21.171128+0000","last_became_peered":"2023-09-23T03:15:21.171128+0000","last_unstale":"2023-09-23T03:15:25.445518+0000","last_undegraded":"2023-09-23T03:15:25.445518+0000","last_fullsized":"2023-09-23T03:15:25.445518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:38:51.387638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":851968,"data_stored":690730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1220000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3759999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6930000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5009999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.591}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.429}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1319999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.371}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4119999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6220000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.488}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.51}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":99870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5780000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0069999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5409999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2609999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5109999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3779999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":99906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0509999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4179999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2849999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2410000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.716}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.345}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0549999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.298}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1180000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4570000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1559999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.218}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":99780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4330000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.51}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4139999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3640000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.387}]}]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":262144,"data_stored":99940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5109999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.405}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.252}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3759999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":689767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.254}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2090000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.296}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3320000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-23T03:15:51.180 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-09-23T03:15:51.180 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-09-23T03:15:51.180 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-09-23T03:15:51.181 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-09-23T03:15:51.181 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-09-23T03:15:51.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.432+0000 7f11f9721640 1 Processor -- start 2023-09-23T03:15:51.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.432+0000 7f11f9721640 1 -- start start 2023-09-23T03:15:51.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.433+0000 7f11f9721640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f11f415eb00 0x7f11f406fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:51.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.433+0000 7f11f9721640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f11f4070300 0x7f11f40ca500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:51.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.433+0000 7f11f9721640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11f406bff0 con 0x7f11f406c7a0 2023-09-23T03:15:51.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.433+0000 7f11f9721640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11f4155070 con 0x7f11f415eb00 2023-09-23T03:15:51.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.433+0000 7f11f9721640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11f415e0f0 con 0x7f11f4070300 2023-09-23T03:15:51.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.433+0000 7f11ebfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f11f415eb00 0x7f11f406fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:51.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.434+0000 7f11ebfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f11f415eb00 0x7f11f406fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44440/0 (socket says 172.21.15.154:44440) 2023-09-23T03:15:51.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.434+0000 7f11ebfff640 1 -- 172.21.15.154:0/2794541028 learned_addr learned my addr 172.21.15.154:0/2794541028 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:51.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.434+0000 7f11f8f20640 1 --2- 172.21.15.154:0/2794541028 >> v2:172.21.15.190:3300/0 conn(0x7f11f4070300 0x7f11f40ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:51.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.434+0000 7f11ebfff640 1 -- 172.21.15.154:0/2794541028 >> v2:172.21.15.190:3300/0 conn(0x7f11f4070300 msgr2=0x7f11f40ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:51.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.434+0000 7f11ebfff640 1 --2- 172.21.15.154:0/2794541028 >> v2:172.21.15.190:3300/0 conn(0x7f11f4070300 0x7f11f40ca500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:51.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.434+0000 7f11ebfff640 1 -- 172.21.15.154:0/2794541028 >> v1:172.21.15.190:6789/0 conn(0x7f11f406c7a0 legacy=0x7f11f415ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:51.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.434+0000 7f11ebfff640 1 -- 172.21.15.154:0/2794541028 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f415efb0 con 0x7f11f415eb00 2023-09-23T03:15:51.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.434+0000 7f11f8f20640 1 --2- 172.21.15.154:0/2794541028 >> v2:172.21.15.190:3300/0 conn(0x7f11f4070300 0x7f11f40ca500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:51.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.435+0000 7f11ebfff640 1 --2- 172.21.15.154:0/2794541028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11f415eb00 0x7f11f406fdc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f11f40cbcf0 tx=0x7f11dc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e02be4c5c2c4e9ab server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:51.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.435+0000 7f11f37fe640 1 -- 172.21.15.154:0/2794541028 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11dc031870 con 0x7f11f415eb00 2023-09-23T03:15:51.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.435+0000 7f11f37fe640 1 -- 172.21.15.154:0/2794541028 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11dc0319d0 con 0x7f11f415eb00 2023-09-23T03:15:51.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.436+0000 7f11f37fe640 1 -- 172.21.15.154:0/2794541028 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11dc031d60 con 0x7f11f415eb00 2023-09-23T03:15:51.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.436+0000 7f11f9721640 1 -- 172.21.15.154:0/2794541028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11f415eb00 msgr2=0x7f11f406fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:51.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.436+0000 7f11f9721640 1 --2- 172.21.15.154:0/2794541028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11f415eb00 0x7f11f406fdc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f11f40cbcf0 tx=0x7f11dc02b910 comp rx=0 tx=0).stop 2023-09-23T03:15:51.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.436+0000 7f11f9721640 1 -- 172.21.15.154:0/2794541028 shutdown_connections 2023-09-23T03:15:51.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.436+0000 7f11f9721640 1 --2- 172.21.15.154:0/2794541028 >> v2:172.21.15.190:3300/0 conn(0x7f11f4070300 0x7f11f40ca500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:51.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.436+0000 7f11f9721640 1 --2- 172.21.15.154:0/2794541028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11f415eb00 0x7f11f406fdc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:51.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.436+0000 7f11f9721640 1 -- 172.21.15.154:0/2794541028 >> 172.21.15.154:0/2794541028 conn(0x7f11f406a390 msgr2=0x7f11f40afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:51.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.436+0000 7f11f9721640 1 -- 172.21.15.154:0/2794541028 shutdown_connections 2023-09-23T03:15:51.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.436+0000 7f11f9721640 1 -- 172.21.15.154:0/2794541028 wait complete. 2023-09-23T03:15:51.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.438+0000 7f11f9721640 1 Processor -- start 2023-09-23T03:15:51.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.438+0000 7f11f9721640 1 -- start start 2023-09-23T03:15:51.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.439+0000 7f11f9721640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11f406c7a0 0x7f11f40c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:51.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.439+0000 7f11f9721640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11f4070300 0x7f11f4151700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:51.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.439+0000 7f11f9721640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11f415eb00 0x7f11f40b24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:51.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.439+0000 7f11f9721640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f41f05b0 con 0x7f11f4070300 2023-09-23T03:15:51.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.439+0000 7f11f9721640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f40c2a40 con 0x7f11f406c7a0 2023-09-23T03:15:51.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.439+0000 7f11f9721640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f40b3d00 con 0x7f11f415eb00 2023-09-23T03:15:51.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.439+0000 7f11ebfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11f4070300 0x7f11f4151700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:51.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.439+0000 7f11ebfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11f4070300 0x7f11f4151700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44454/0 (socket says 172.21.15.154:44454) 2023-09-23T03:15:51.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.439+0000 7f11ebfff640 1 -- 172.21.15.154:0/1631001419 learned_addr learned my addr 172.21.15.154:0/1631001419 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:51.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.440+0000 7f11f8f20640 1 --2- 172.21.15.154:0/1631001419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11f415eb00 0x7f11f40b24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:51.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.440+0000 7f11f3fff640 1 --2- 172.21.15.154:0/1631001419 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11f406c7a0 0x7f11f40c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:51.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.440+0000 7f11ebfff640 1 -- 172.21.15.154:0/1631001419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11f415eb00 msgr2=0x7f11f40b24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:51.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.440+0000 7f11ebfff640 1 --2- 172.21.15.154:0/1631001419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11f415eb00 0x7f11f40b24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:51.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.440+0000 7f11ebfff640 1 -- 172.21.15.154:0/1631001419 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11f406c7a0 msgr2=0x7f11f40c7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:51.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.440+0000 7f11ebfff640 1 --2- 172.21.15.154:0/1631001419 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11f406c7a0 0x7f11f40c7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:51.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.440+0000 7f11ebfff640 1 -- 172.21.15.154:0/1631001419 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11dc0094d0 con 0x7f11f4070300 2023-09-23T03:15:51.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.440+0000 7f11f8f20640 1 --2- 172.21.15.154:0/1631001419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11f415eb00 0x7f11f40b24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:51.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.440+0000 7f11f3fff640 1 --2- 172.21.15.154:0/1631001419 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11f406c7a0 0x7f11f40c7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:51.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.440+0000 7f11ebfff640 1 --2- 172.21.15.154:0/1631001419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11f4070300 0x7f11f4151700 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f11dc02b950 tx=0x7f11dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:51.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.440+0000 7f11f17fa640 1 -- 172.21.15.154:0/1631001419 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11dc031840 con 0x7f11f4070300 2023-09-23T03:15:51.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.440+0000 7f11f17fa640 1 -- 172.21.15.154:0/1631001419 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11dc0443f0 con 0x7f11f4070300 2023-09-23T03:15:51.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.440+0000 7f11f17fa640 1 -- 172.21.15.154:0/1631001419 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11dc0447a0 con 0x7f11f4070300 2023-09-23T03:15:51.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.441+0000 7f11f9721640 1 -- 172.21.15.154:0/1631001419 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f40c1e70 con 0x7f11f4070300 2023-09-23T03:15:51.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.441+0000 7f11f9721640 1 -- 172.21.15.154:0/1631001419 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f40c23f0 con 0x7f11f4070300 2023-09-23T03:15:51.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.442+0000 7f11f9721640 1 -- 172.21.15.154:0/1631001419 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11f40736b0 con 0x7f11f4070300 2023-09-23T03:15:51.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.443+0000 7f11f17fa640 1 -- 172.21.15.154:0/1631001419 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f11dc044900 con 0x7f11f4070300 2023-09-23T03:15:51.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.444+0000 7f11f17fa640 1 --2- 172.21.15.154:0/1631001419 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f11d406af30 0x7f11d406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:51.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.444+0000 7f11f17fa640 1 -- 172.21.15.154:0/1631001419 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f11dc02d040 con 0x7f11f4070300 2023-09-23T03:15:51.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.444+0000 7f11f3fff640 1 --2- 172.21.15.154:0/1631001419 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f11d406af30 0x7f11d406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:51.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.444+0000 7f11f3fff640 1 --2- 172.21.15.154:0/1631001419 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f11d406af30 0x7f11d406d3f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f11e4003eb0 tx=0x7f11e4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:51.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.448+0000 7f11f17fa640 1 -- 172.21.15.154:0/1631001419 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f11dc0dea50 con 0x7f11f4070300 2023-09-23T03:15:51.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.684+0000 7f11f9721640 1 -- 172.21.15.154:0/1631001419 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f11f404e920 con 0x7f11f4070300 2023-09-23T03:15:51.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.693+0000 7f11f17fa640 1 -- 172.21.15.154:0/1631001419 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171162 (secure 0 0 0) 0x7f11dc0dccb0 con 0x7f11f4070300 2023-09-23T03:15:51.697 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:15:51.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.697+0000 7f11f9721640 1 -- 172.21.15.154:0/1631001419 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f11d406af30 msgr2=0x7f11d406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:51.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.697+0000 7f11f9721640 1 --2- 172.21.15.154:0/1631001419 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f11d406af30 0x7f11d406d3f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f11e4003eb0 tx=0x7f11e4009040 comp rx=0 tx=0).stop 2023-09-23T03:15:51.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.697+0000 7f11f9721640 1 -- 172.21.15.154:0/1631001419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11f4070300 msgr2=0x7f11f4151700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:51.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.697+0000 7f11f9721640 1 --2- 172.21.15.154:0/1631001419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11f4070300 0x7f11f4151700 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f11dc02b950 tx=0x7f11dc0046c0 comp rx=0 tx=0).stop 2023-09-23T03:15:51.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.697+0000 7f11f9721640 1 -- 172.21.15.154:0/1631001419 shutdown_connections 2023-09-23T03:15:51.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.697+0000 7f11f9721640 1 --2- 172.21.15.154:0/1631001419 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f11d406af30 0x7f11d406d3f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:51.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.697+0000 7f11f9721640 1 --2- 172.21.15.154:0/1631001419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11f415eb00 0x7f11f40b24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:51.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.697+0000 7f11f9721640 1 --2- 172.21.15.154:0/1631001419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11f4070300 0x7f11f4151700 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:51.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.697+0000 7f11f9721640 1 --2- 172.21.15.154:0/1631001419 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11f406c7a0 0x7f11f40c7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:51.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.697+0000 7f11f9721640 1 -- 172.21.15.154:0/1631001419 >> 172.21.15.154:0/1631001419 conn(0x7f11f406a390 msgr2=0x7f11f40c8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:51.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.698+0000 7f11f9721640 1 -- 172.21.15.154:0/1631001419 shutdown_connections 2023-09-23T03:15:51.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.698+0000 7f11f9721640 1 -- 172.21.15.154:0/1631001419 wait complete. 2023-09-23T03:15:51.718 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":5,"active_gid":4110,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2422652104},{"type":"v1","addr":"172.21.15.154:6833","nonce":2422652104}]},"active_addr":"172.21.15.154:6833/2422652104","active_change":"2023-09-23T03:15:01.143344+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4109,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.154:0","nonce":4118245296}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.154:0","nonce":2498572156}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.154:0","nonce":1828005469}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.154:0","nonce":576281193}]}]} 2023-09-23T03:15:51.722 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-09-23T03:15:51.723 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-09-23T03:15:51.723 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:15:51.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.935+0000 7facd2410640 1 Processor -- start 2023-09-23T03:15:51.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.936+0000 7facd2410640 1 -- start start 2023-09-23T03:15:51.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.937+0000 7facd2410640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faccc0acce0 0x7faccc15ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:51.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.937+0000 7facd2410640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faccc15ac30 con 0x7faccc0cf040 2023-09-23T03:15:51.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.937+0000 7facd2410640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faccc0b1f70 con 0x7faccc0b1b20 2023-09-23T03:15:51.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.937+0000 7facd2410640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faccc0b2e90 con 0x7faccc0acce0 2023-09-23T03:15:51.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.938+0000 7facd09ad640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7faccc0cf040 0x7faccc0ab140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:42272/0 (socket says 172.21.15.154:42272) 2023-09-23T03:15:51.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.938+0000 7facd09ad640 1 -- 172.21.15.154:0/2629550685 learned_addr learned my addr 172.21.15.154:0/2629550685 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:51.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.938+0000 7faccbfff640 1 --2- 172.21.15.154:0/2629550685 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faccc0acce0 0x7faccc15ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:51.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.938+0000 7faccb7fe640 1 -- 172.21.15.154:0/2629550685 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3769354561 0 0) 0x7facc4001490 con 0x7faccc0cf040 2023-09-23T03:15:51.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.938+0000 7faccb7fe640 1 -- 172.21.15.154:0/2629550685 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facb8003520 con 0x7faccc0cf040 2023-09-23T03:15:51.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.939+0000 7faccb7fe640 1 -- 172.21.15.154:0/2629550685 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 459960921 0 0) 0x7facb8003520 con 0x7faccc0cf040 2023-09-23T03:15:51.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.939+0000 7faccbfff640 1 -- 172.21.15.154:0/2629550685 >> v1:172.21.15.190:6789/0 conn(0x7faccc0b1b20 legacy=0x7faccc0ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:51.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.939+0000 7faccbfff640 1 -- 172.21.15.154:0/2629550685 >> v1:172.21.15.154:6789/0 conn(0x7faccc0cf040 legacy=0x7faccc0ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:51.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.939+0000 7faccbfff640 1 -- 172.21.15.154:0/2629550685 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faccc15f2a0 con 0x7faccc0acce0 2023-09-23T03:15:51.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.940+0000 7faccbfff640 1 --2- 172.21.15.154:0/2629550685 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faccc0acce0 0x7faccc15ed60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7facbc0097c0 tx=0x7facbc02b5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=832959d692ef0d96 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:51.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.940+0000 7faccb7fe640 1 -- 172.21.15.154:0/2629550685 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc4001b80 con 0x7faccc0acce0 2023-09-23T03:15:51.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.940+0000 7faccb7fe640 1 -- 172.21.15.154:0/2629550685 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facbc02e050 con 0x7faccc0acce0 2023-09-23T03:15:51.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.940+0000 7faccb7fe640 1 -- 172.21.15.154:0/2629550685 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc030710 con 0x7faccc0acce0 2023-09-23T03:15:51.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.941+0000 7facd2410640 1 -- 172.21.15.154:0/2629550685 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faccc0acce0 msgr2=0x7faccc15ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:51.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.941+0000 7facd2410640 1 --2- 172.21.15.154:0/2629550685 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faccc0acce0 0x7faccc15ed60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7facbc0097c0 tx=0x7facbc02b5d0 comp rx=0 tx=0).stop 2023-09-23T03:15:51.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.941+0000 7facd2410640 1 -- 172.21.15.154:0/2629550685 shutdown_connections 2023-09-23T03:15:51.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.941+0000 7facd2410640 1 --2- 172.21.15.154:0/2629550685 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faccc0acce0 0x7faccc15ed60 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:51.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.941+0000 7facd2410640 1 -- 172.21.15.154:0/2629550685 >> 172.21.15.154:0/2629550685 conn(0x7faccc06a0f0 msgr2=0x7faccc06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:51.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.941+0000 7facd2410640 1 -- 172.21.15.154:0/2629550685 shutdown_connections 2023-09-23T03:15:51.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.941+0000 7facd2410640 1 -- 172.21.15.154:0/2629550685 wait complete. 2023-09-23T03:15:51.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.943+0000 7facd2410640 1 Processor -- start 2023-09-23T03:15:51.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.943+0000 7facd2410640 1 -- start start 2023-09-23T03:15:51.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.944+0000 7facd2410640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faccc0acce0 0x7faccc15c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:51.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.944+0000 7facd2410640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faccc0b1b20 0x7faccc15d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:51.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.944+0000 7facd2410640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faccc0cf040 0x7faccc15d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:51.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.944+0000 7facd2410640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc0c3f50 con 0x7faccc0acce0 2023-09-23T03:15:51.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.944+0000 7facd2410640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc0c4230 con 0x7faccc0b1b20 2023-09-23T03:15:51.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.944+0000 7facd2410640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faccc0c4510 con 0x7faccc0cf040 2023-09-23T03:15:51.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.944+0000 7faccbfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faccc0acce0 0x7faccc15c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:51.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.944+0000 7faccbfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faccc0acce0 0x7faccc15c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44468/0 (socket says 172.21.15.154:44468) 2023-09-23T03:15:51.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.944+0000 7faccbfff640 1 -- 172.21.15.154:0/2268568832 learned_addr learned my addr 172.21.15.154:0/2268568832 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:51.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.945+0000 7facd09ad640 1 --2- 172.21.15.154:0/2268568832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faccc0cf040 0x7faccc15d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:51.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.945+0000 7facc3fff640 1 --2- 172.21.15.154:0/2268568832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faccc0b1b20 0x7faccc15d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:51.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.945+0000 7faccbfff640 1 -- 172.21.15.154:0/2268568832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faccc0cf040 msgr2=0x7faccc15d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:51.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.945+0000 7faccbfff640 1 --2- 172.21.15.154:0/2268568832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faccc0cf040 0x7faccc15d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:51.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.945+0000 7faccbfff640 1 -- 172.21.15.154:0/2268568832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faccc0b1b20 msgr2=0x7faccc15d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:51.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.945+0000 7faccbfff640 1 --2- 172.21.15.154:0/2268568832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faccc0b1b20 0x7faccc15d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:51.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.945+0000 7faccbfff640 1 -- 172.21.15.154:0/2268568832 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facbc0093a0 con 0x7faccc0acce0 2023-09-23T03:15:51.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.945+0000 7faccbfff640 1 --2- 172.21.15.154:0/2268568832 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faccc0acce0 0x7faccc15c5a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7facbc000c00 tx=0x7facbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:51.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.945+0000 7facc97fa640 1 -- 172.21.15.154:0/2268568832 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc03d950 con 0x7faccc0acce0 2023-09-23T03:15:51.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.946+0000 7facc97fa640 1 -- 172.21.15.154:0/2268568832 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facbc03dab0 con 0x7faccc0acce0 2023-09-23T03:15:51.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.946+0000 7facc97fa640 1 -- 172.21.15.154:0/2268568832 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc03de60 con 0x7faccc0acce0 2023-09-23T03:15:51.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.946+0000 7facd2410640 1 -- 172.21.15.154:0/2268568832 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faccc0c47f0 con 0x7faccc0acce0 2023-09-23T03:15:51.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.946+0000 7facd2410640 1 -- 172.21.15.154:0/2268568832 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faccc0c6ca0 con 0x7faccc0acce0 2023-09-23T03:15:51.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.947+0000 7facd2410640 1 -- 172.21.15.154:0/2268568832 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faccc0c4980 con 0x7faccc0acce0 2023-09-23T03:15:51.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.948+0000 7facc97fa640 1 -- 172.21.15.154:0/2268568832 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7facbc039040 con 0x7faccc0acce0 2023-09-23T03:15:51.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.949+0000 7facc97fa640 1 --2- 172.21.15.154:0/2268568832 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7facac06af30 0x7facac06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:51.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.949+0000 7facc97fa640 1 -- 172.21.15.154:0/2268568832 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7facbc0abf00 con 0x7faccc0acce0 2023-09-23T03:15:51.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.949+0000 7facc3fff640 1 --2- 172.21.15.154:0/2268568832 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7facac06af30 0x7facac06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:51.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.950+0000 7facc3fff640 1 --2- 172.21.15.154:0/2268568832 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7facac06af30 0x7facac06d3f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7faccc0c4bf0 tx=0x7facb4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:51.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:51.953+0000 7facc97fa640 1 -- 172.21.15.154:0/2268568832 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7facbc076bb0 con 0x7faccc0acce0 2023-09-23T03:15:52.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.144+0000 7facd2410640 1 -- 172.21.15.154:0/2268568832 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faccc073eb0 con 0x7faccc0acce0 2023-09-23T03:15:52.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.149+0000 7facc97fa640 1 -- 172.21.15.154:0/2268568832 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18382 (secure 0 0 0) 0x7facbc037090 con 0x7faccc0acce0 2023-09-23T03:15:52.151 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:15:52.151 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":24,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:15:25.775060+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:15:18.575849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T03:15:19.707212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T03:15:23.337138+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:15:52.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.151+0000 7facd2410640 1 -- 172.21.15.154:0/2268568832 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7facac06af30 msgr2=0x7facac06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:52.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.151+0000 7facd2410640 1 --2- 172.21.15.154:0/2268568832 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7facac06af30 0x7facac06d3f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7faccc0c4bf0 tx=0x7facb4009040 comp rx=0 tx=0).stop 2023-09-23T03:15:52.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.151+0000 7facd2410640 1 -- 172.21.15.154:0/2268568832 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faccc0acce0 msgr2=0x7faccc15c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:52.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.151+0000 7facd2410640 1 --2- 172.21.15.154:0/2268568832 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faccc0acce0 0x7faccc15c5a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7facbc000c00 tx=0x7facbc0046c0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.152+0000 7facd2410640 1 -- 172.21.15.154:0/2268568832 shutdown_connections 2023-09-23T03:15:52.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.152+0000 7facd2410640 1 --2- 172.21.15.154:0/2268568832 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7facac06af30 0x7facac06d3f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.152+0000 7facd2410640 1 --2- 172.21.15.154:0/2268568832 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faccc0cf040 0x7faccc15d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.152+0000 7facd2410640 1 --2- 172.21.15.154:0/2268568832 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faccc0b1b20 0x7faccc15d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.152+0000 7facd2410640 1 --2- 172.21.15.154:0/2268568832 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faccc0acce0 0x7faccc15c5a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.152+0000 7facd2410640 1 -- 172.21.15.154:0/2268568832 >> 172.21.15.154:0/2268568832 conn(0x7faccc06a0f0 msgr2=0x7faccc0af5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:52.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.152+0000 7facd2410640 1 -- 172.21.15.154:0/2268568832 shutdown_connections 2023-09-23T03:15:52.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.152+0000 7facd2410640 1 -- 172.21.15.154:0/2268568832 wait complete. 2023-09-23T03:15:52.171 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-09-23T03:15:52.172 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:15:52.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.384+0000 7f60c1133640 1 Processor -- start 2023-09-23T03:15:52.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.384+0000 7f60c1133640 1 -- start start 2023-09-23T03:15:52.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.385+0000 7f60c1133640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f60bc0ada30 0x7f60bc0ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.385+0000 7f60c1133640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f60bc0cf130 0x7f60bc15d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.385+0000 7f60c1133640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60bc06ceb0 con 0x7f60bc0b2fe0 2023-09-23T03:15:52.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.385+0000 7f60c1133640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60bc0ade10 con 0x7f60bc0cf130 2023-09-23T03:15:52.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.385+0000 7f60c1133640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60bc159c90 con 0x7f60bc0ada30 2023-09-23T03:15:52.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.386+0000 7f60bb59e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f60bc0cf130 0x7f60bc15d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:52.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.386+0000 7f60b3fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f60bc0b2fe0 0x7f60bc0cea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:42278/0 (socket says 172.21.15.154:42278) 2023-09-23T03:15:52.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.386+0000 7f60b3fff640 1 -- 172.21.15.154:0/2917359145 learned_addr learned my addr 172.21.15.154:0/2917359145 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:52.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.386+0000 7f60bb59e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f60bc0cf130 0x7f60bc15d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44472/0 (socket says 172.21.15.154:44472) 2023-09-23T03:15:52.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.386+0000 7f60bad9d640 1 --2- 172.21.15.154:0/2917359145 >> v2:172.21.15.190:3300/0 conn(0x7f60bc0ada30 0x7f60bc0ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:52.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.386+0000 7f60bb59e640 1 -- 172.21.15.154:0/2917359145 >> v2:172.21.15.190:3300/0 conn(0x7f60bc0ada30 msgr2=0x7f60bc0ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:52.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.386+0000 7f60bb59e640 1 --2- 172.21.15.154:0/2917359145 >> v2:172.21.15.190:3300/0 conn(0x7f60bc0ada30 0x7f60bc0ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.386+0000 7f60bb59e640 1 -- 172.21.15.154:0/2917359145 >> v1:172.21.15.154:6789/0 conn(0x7f60bc0b2fe0 legacy=0x7f60bc0cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:52.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.386+0000 7f60bb59e640 1 -- 172.21.15.154:0/2917359145 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60bc06c110 con 0x7f60bc0cf130 2023-09-23T03:15:52.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.386+0000 7f60bad9d640 1 --2- 172.21.15.154:0/2917359145 >> v2:172.21.15.190:3300/0 conn(0x7f60bc0ada30 0x7f60bc0ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:15:52.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.387+0000 7f60bb59e640 1 --2- 172.21.15.154:0/2917359145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60bc0cf130 0x7f60bc15d380 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f60bc06d990 tx=0x7f60b402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb8ec43045ae20b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:52.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.388+0000 7f60ba59c640 1 -- 172.21.15.154:0/2917359145 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b402cc70 con 0x7f60bc0cf130 2023-09-23T03:15:52.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.388+0000 7f60ba59c640 1 -- 172.21.15.154:0/2917359145 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60b402cdd0 con 0x7f60bc0cf130 2023-09-23T03:15:52.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.388+0000 7f60ba59c640 1 -- 172.21.15.154:0/2917359145 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b4034660 con 0x7f60bc0cf130 2023-09-23T03:15:52.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.388+0000 7f60c1133640 1 -- 172.21.15.154:0/2917359145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60bc0cf130 msgr2=0x7f60bc15d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:52.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.388+0000 7f60c1133640 1 --2- 172.21.15.154:0/2917359145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60bc0cf130 0x7f60bc15d380 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f60bc06d990 tx=0x7f60b402bec0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.388+0000 7f60c1133640 1 -- 172.21.15.154:0/2917359145 shutdown_connections 2023-09-23T03:15:52.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.388+0000 7f60c1133640 1 --2- 172.21.15.154:0/2917359145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60bc0cf130 0x7f60bc15d380 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.388+0000 7f60c1133640 1 --2- 172.21.15.154:0/2917359145 >> v2:172.21.15.190:3300/0 conn(0x7f60bc0ada30 0x7f60bc0ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.388+0000 7f60c1133640 1 -- 172.21.15.154:0/2917359145 >> 172.21.15.154:0/2917359145 conn(0x7f60bc06a0f0 msgr2=0x7f60bc06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:52.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.389+0000 7f60c1133640 1 -- 172.21.15.154:0/2917359145 shutdown_connections 2023-09-23T03:15:52.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.389+0000 7f60c1133640 1 -- 172.21.15.154:0/2917359145 wait complete. 2023-09-23T03:15:52.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.390+0000 7f60c1133640 1 Processor -- start 2023-09-23T03:15:52.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.390+0000 7f60c1133640 1 -- start start 2023-09-23T03:15:52.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.391+0000 7f60c1133640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60bc0ada30 0x7f60bc06fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.391+0000 7f60c1133640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60bc0b2fe0 0x7f60bc095a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.391+0000 7f60c1133640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60bc0cf130 0x7f60bc071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.391+0000 7f60c1133640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60bc154260 con 0x7f60bc0b2fe0 2023-09-23T03:15:52.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.391+0000 7f60c1133640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60bc154500 con 0x7f60bc0ada30 2023-09-23T03:15:52.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.391+0000 7f60c1133640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60bc1547e0 con 0x7f60bc0cf130 2023-09-23T03:15:52.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.392+0000 7f60b3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60bc0b2fe0 0x7f60bc095a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:52.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.392+0000 7f60b3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60bc0b2fe0 0x7f60bc095a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44478/0 (socket says 172.21.15.154:44478) 2023-09-23T03:15:52.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.392+0000 7f60b3fff640 1 -- 172.21.15.154:0/296088930 learned_addr learned my addr 172.21.15.154:0/296088930 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:52.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.392+0000 7f60bb59e640 1 --2- 172.21.15.154:0/296088930 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60bc0cf130 0x7f60bc071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:52.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.392+0000 7f60bad9d640 1 --2- 172.21.15.154:0/296088930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60bc0ada30 0x7f60bc06fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:52.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.392+0000 7f60b3fff640 1 -- 172.21.15.154:0/296088930 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60bc0cf130 msgr2=0x7f60bc071a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:52.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.392+0000 7f60b3fff640 1 --2- 172.21.15.154:0/296088930 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60bc0cf130 0x7f60bc071a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.392+0000 7f60b3fff640 1 -- 172.21.15.154:0/296088930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60bc0ada30 msgr2=0x7f60bc06fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:52.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.392+0000 7f60b3fff640 1 --2- 172.21.15.154:0/296088930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60bc0ada30 0x7f60bc06fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.392+0000 7f60b3fff640 1 -- 172.21.15.154:0/296088930 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b4009a70 con 0x7f60bc0b2fe0 2023-09-23T03:15:52.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.392+0000 7f60bb59e640 1 --2- 172.21.15.154:0/296088930 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60bc0cf130 0x7f60bc071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:52.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.392+0000 7f60b3fff640 1 --2- 172.21.15.154:0/296088930 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60bc0b2fe0 0x7f60bc095a20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f60a400c4d0 tx=0x7f60a400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:52.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.393+0000 7f60b37fe640 1 -- 172.21.15.154:0/296088930 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a4005ce0 con 0x7f60bc0b2fe0 2023-09-23T03:15:52.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.393+0000 7f60b37fe640 1 -- 172.21.15.154:0/296088930 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60a4005e40 con 0x7f60bc0b2fe0 2023-09-23T03:15:52.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.393+0000 7f60b37fe640 1 -- 172.21.15.154:0/296088930 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a4007270 con 0x7f60bc0b2fe0 2023-09-23T03:15:52.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.393+0000 7f60c1133640 1 -- 172.21.15.154:0/296088930 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60bc154ac0 con 0x7f60bc0b2fe0 2023-09-23T03:15:52.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.393+0000 7f60c1133640 1 -- 172.21.15.154:0/296088930 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60bc153f50 con 0x7f60bc0b2fe0 2023-09-23T03:15:52.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.394+0000 7f60c1133640 1 -- 172.21.15.154:0/296088930 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60bc152ff0 con 0x7f60bc0b2fe0 2023-09-23T03:15:52.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.395+0000 7f60b37fe640 1 -- 172.21.15.154:0/296088930 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f60a40073d0 con 0x7f60bc0b2fe0 2023-09-23T03:15:52.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.396+0000 7f60b37fe640 1 --2- 172.21.15.154:0/296088930 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f609c06af60 0x7f609c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.396+0000 7f60b37fe640 1 -- 172.21.15.154:0/296088930 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f60a4088c30 con 0x7f60bc0b2fe0 2023-09-23T03:15:52.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.396+0000 7f60bad9d640 1 --2- 172.21.15.154:0/296088930 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f609c06af60 0x7f609c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:52.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.397+0000 7f60bad9d640 1 --2- 172.21.15.154:0/296088930 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f609c06af60 0x7f609c06d420 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f60ac00c8e0 tx=0x7f60ac008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:52.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.400+0000 7f60b37fe640 1 -- 172.21.15.154:0/296088930 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f60bc152ff0 con 0x7f60bc0b2fe0 2023-09-23T03:15:52.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.592+0000 7f60c1133640 1 -- 172.21.15.154:0/296088930 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f60bc06ceb0 con 0x7f60bc0b2fe0 2023-09-23T03:15:52.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.594+0000 7f60b37fe640 1 -- 172.21.15.154:0/296088930 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18382 (secure 0 0 0) 0x7f60bc06ceb0 con 0x7f60bc0b2fe0 2023-09-23T03:15:52.596 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:15:52.596 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":24,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:15:25.775060+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:15:18.575849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T03:15:19.707212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T03:15:23.337138+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:15:52.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.597+0000 7f60c1133640 1 -- 172.21.15.154:0/296088930 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f609c06af60 msgr2=0x7f609c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:52.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.597+0000 7f60c1133640 1 --2- 172.21.15.154:0/296088930 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f609c06af60 0x7f609c06d420 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f60ac00c8e0 tx=0x7f60ac008040 comp rx=0 tx=0).stop 2023-09-23T03:15:52.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.597+0000 7f60c1133640 1 -- 172.21.15.154:0/296088930 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60bc0b2fe0 msgr2=0x7f60bc095a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:52.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.597+0000 7f60c1133640 1 --2- 172.21.15.154:0/296088930 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60bc0b2fe0 0x7f60bc095a20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f60a400c4d0 tx=0x7f60a400c990 comp rx=0 tx=0).stop 2023-09-23T03:15:52.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.597+0000 7f60c1133640 1 -- 172.21.15.154:0/296088930 shutdown_connections 2023-09-23T03:15:52.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.597+0000 7f60c1133640 1 --2- 172.21.15.154:0/296088930 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f609c06af60 0x7f609c06d420 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.597+0000 7f60c1133640 1 --2- 172.21.15.154:0/296088930 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f60bc0cf130 0x7f60bc071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.597+0000 7f60c1133640 1 --2- 172.21.15.154:0/296088930 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f60bc0b2fe0 0x7f60bc095a20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.597+0000 7f60c1133640 1 --2- 172.21.15.154:0/296088930 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f60bc0ada30 0x7f60bc06fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.597+0000 7f60c1133640 1 -- 172.21.15.154:0/296088930 >> 172.21.15.154:0/296088930 conn(0x7f60bc06a0f0 msgr2=0x7f60bc0af270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:52.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.598+0000 7f60c1133640 1 -- 172.21.15.154:0/296088930 shutdown_connections 2023-09-23T03:15:52.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.598+0000 7f60c1133640 1 -- 172.21.15.154:0/296088930 wait complete. 2023-09-23T03:15:52.618 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-09-23T03:15:52.618 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-09-23T03:15:52.619 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-09-23T03:15:52.619 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-09-23T03:15:52.620 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-09-23T03:15:52.620 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-09-23T03:15:52.620 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-09-23T03:15:52.621 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-09-23T03:15:52.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.925+0000 7fe2ab3e7640 1 Processor -- start 2023-09-23T03:15:52.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.925+0000 7fe2ab3e7640 1 -- start start 2023-09-23T03:15:52.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.926+0000 7fda653c5640 1 Processor -- start 2023-09-23T03:15:52.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.926+0000 7fda653c5640 1 -- start start 2023-09-23T03:15:52.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.927+0000 7fad7d639640 1 Processor -- start 2023-09-23T03:15:52.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.927+0000 7fad7d639640 1 -- start start 2023-09-23T03:15:52.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.927+0000 7fe2ab3e7640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe29c0e1410 0x7fe29c001c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.927+0000 7fe2ab3e7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe29c0358c0 0x7fe29c000e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.927+0000 7fe2ab3e7640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe29c054d40 con 0x7fe29c001360 2023-09-23T03:15:52.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.927+0000 7fe2ab3e7640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe29c001810 con 0x7fe29c0e1410 2023-09-23T03:15:52.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.927+0000 7fe2ab3e7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c03c560 con 0x7fe29c0358c0 2023-09-23T03:15:52.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.928+0000 7fe2a9183640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe29c0e1410 0x7fe29c001c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:52.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.928+0000 7f26a3fff640 1 Processor -- start 2023-09-23T03:15:52.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.928+0000 7fe2a9183640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe29c0e1410 0x7fe29c001c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44494/0 (socket says 172.21.15.154:44494) 2023-09-23T03:15:52.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.928+0000 7fe2a9183640 1 -- 172.21.15.154:0/115775837 learned_addr learned my addr 172.21.15.154:0/115775837 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:52.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.928+0000 7fda653c5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda5800c220 0x7fda58053180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.928+0000 7fda653c5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fda580df9a0 0x7fda580dfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.928+0000 7fda653c5640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda58054420 con 0x7fda58033ff0 2023-09-23T03:15:52.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.928+0000 7fda653c5640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda580343d0 con 0x7fda580df9a0 2023-09-23T03:15:52.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.928+0000 7fda653c5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda58053810 con 0x7fda5800c220 2023-09-23T03:15:52.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.929+0000 7fe2a8982640 1 --2- 172.21.15.154:0/115775837 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe29c0358c0 0x7fe29c000e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:52.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.929+0000 7fad7d639640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fad7000c1f0 0x7fad70033fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.929+0000 7fda5f7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fda580df9a0 0x7fda580dfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:52.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.929+0000 7fda5f7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fda580df9a0 0x7fda580dfe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44506/0 (socket says 172.21.15.154:44506) 2023-09-23T03:15:52.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.929+0000 7fda5f7fe640 1 -- 172.21.15.154:0/1401980209 learned_addr learned my addr 172.21.15.154:0/1401980209 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:52.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.929+0000 7fad7d639640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad70053150 0x7fad700e36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.929+0000 7fad7d639640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad700543f0 con 0x7fad70050230 2023-09-23T03:15:52.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.930+0000 7fad7d639640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad700535d0 con 0x7fad7000c1f0 2023-09-23T03:15:52.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.930+0000 7fad7d639640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad700537e0 con 0x7fad70053150 2023-09-23T03:15:52.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.930+0000 7fad777fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad70053150 0x7fad700e36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:52.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.930+0000 7fad777fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad70053150 0x7fad700e36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:43878/0 (socket says 172.21.15.154:43878) 2023-09-23T03:15:52.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.930+0000 7fad777fe640 1 -- 172.21.15.154:0/2902557866 learned_addr learned my addr 172.21.15.154:0/2902557866 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:52.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.929+0000 7fad77fff640 1 --2- 172.21.15.154:0/2902557866 >> v2:172.21.15.154:3300/0 conn(0x7fad7000c1f0 0x7fad70033fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:52.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.931+0000 7fad77fff640 1 -- 172.21.15.154:0/2902557866 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad70053150 msgr2=0x7fad700e36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:52.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.931+0000 7fad77fff640 1 --2- 172.21.15.154:0/2902557866 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad70053150 0x7fad700e36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.931+0000 7fad76ffd640 1 -- 172.21.15.154:0/2902557866 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 982007438 0 0) 0x7fad700543f0 con 0x7fad70050230 2023-09-23T03:15:52.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.931+0000 7fad77fff640 1 -- 172.21.15.154:0/2902557866 >> v1:172.21.15.190:6789/0 conn(0x7fad70050230 legacy=0x7fad70006190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:52.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.931+0000 7fad77fff640 1 -- 172.21.15.154:0/2902557866 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad70053ac0 con 0x7fad7000c1f0 2023-09-23T03:15:52.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.929+0000 7fda5f7fe640 1 -- 172.21.15.154:0/1401980209 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda5800c220 msgr2=0x7fda58053180 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T03:15:52.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.929+0000 7fda5f7fe640 1 --2- 172.21.15.154:0/1401980209 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda5800c220 0x7fda58053180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.929+0000 7fda5f7fe640 1 -- 172.21.15.154:0/1401980209 >> v1:172.21.15.190:6789/0 conn(0x7fda58033ff0 legacy=0x7fda58050260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:52.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.929+0000 7fda5f7fe640 1 -- 172.21.15.154:0/1401980209 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda58053af0 con 0x7fda580df9a0 2023-09-23T03:15:52.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.930+0000 7fda5f7fe640 1 --2- 172.21.15.154:0/1401980209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda580df9a0 0x7fda580dfe30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fda6004e3a0 tx=0x7fda600a5670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d31bd11645cbb4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:52.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.931+0000 7fda5dffb640 1 -- 172.21.15.154:0/1401980209 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda600ac600 con 0x7fda580df9a0 2023-09-23T03:15:52.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.931+0000 7fda5dffb640 1 -- 172.21.15.154:0/1401980209 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda600a8050 con 0x7fda580df9a0 2023-09-23T03:15:52.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.931+0000 7fda5dffb640 1 -- 172.21.15.154:0/1401980209 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda600aca60 con 0x7fda580df9a0 2023-09-23T03:15:52.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.932+0000 7fda653c5640 1 -- 172.21.15.154:0/1401980209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda580df9a0 msgr2=0x7fda580dfe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:52.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.932+0000 7fda653c5640 1 --2- 172.21.15.154:0/1401980209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda580df9a0 0x7fda580dfe30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fda6004e3a0 tx=0x7fda600a5670 comp rx=0 tx=0).stop 2023-09-23T03:15:52.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.932+0000 7fda653c5640 1 -- 172.21.15.154:0/1401980209 shutdown_connections 2023-09-23T03:15:52.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.932+0000 7fda653c5640 1 --2- 172.21.15.154:0/1401980209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda580df9a0 0x7fda580dfe30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.932+0000 7fda653c5640 1 --2- 172.21.15.154:0/1401980209 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda5800c220 0x7fda58053180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.932+0000 7fda653c5640 1 -- 172.21.15.154:0/1401980209 >> 172.21.15.154:0/1401980209 conn(0x7fda58054c40 msgr2=0x7fda58055020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:52.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.929+0000 7fe2a9183640 1 -- 172.21.15.154:0/115775837 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe29c0358c0 msgr2=0x7fe29c000e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:52.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.929+0000 7fe2a9183640 1 --2- 172.21.15.154:0/115775837 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe29c0358c0 0x7fe29c000e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.929+0000 7fe2a9183640 1 -- 172.21.15.154:0/115775837 >> v1:172.21.15.190:6789/0 conn(0x7fe29c001360 legacy=0x7fe29c007910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:52.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.929+0000 7fe2a9183640 1 -- 172.21.15.154:0/115775837 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29c055020 con 0x7fe29c0e1410 2023-09-23T03:15:52.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.931+0000 7fe2a9183640 1 --2- 172.21.15.154:0/115775837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe29c0e1410 0x7fe29c001c60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe2a00098a0 tx=0x7fe2a002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c419e0b76d7bf389 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:52.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.932+0000 7fe29bfff640 1 -- 172.21.15.154:0/115775837 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0031870 con 0x7fe29c0e1410 2023-09-23T03:15:52.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.932+0000 7fe29bfff640 1 -- 172.21.15.154:0/115775837 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a00319d0 con 0x7fe29c0e1410 2023-09-23T03:15:52.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.933+0000 7fe29bfff640 1 -- 172.21.15.154:0/115775837 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a0031d40 con 0x7fe29c0e1410 2023-09-23T03:15:52.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.931+0000 7f26a3fff640 1 -- start start 2023-09-23T03:15:52.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.933+0000 7fad77fff640 1 --2- 172.21.15.154:0/2902557866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad7000c1f0 0x7fad70033fc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fad6c0097a0 tx=0x7fad6c02dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=becf293a29fdab53 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:52.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.932+0000 7f301a094640 1 Processor -- start 2023-09-23T03:15:52.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.932+0000 7f301a094640 1 -- start start 2023-09-23T03:15:52.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.933+0000 7f26a3fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f269c0e3190 0x7f269c03e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.933+0000 7f26a3fff640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f269c03ee60 con 0x7f269c0544a0 2023-09-23T03:15:52.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.933+0000 7f26a3fff640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f269c006e80 con 0x7f269c038180 2023-09-23T03:15:52.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.933+0000 7f26a3fff640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f269c03f100 con 0x7f269c0e3190 2023-09-23T03:15:52.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.933+0000 7fda653c5640 1 -- 172.21.15.154:0/1401980209 shutdown_connections 2023-09-23T03:15:52.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.933+0000 7fe2ab3e7640 1 -- 172.21.15.154:0/115775837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe29c0e1410 msgr2=0x7fe29c001c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:52.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.933+0000 7fe2ab3e7640 1 --2- 172.21.15.154:0/115775837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe29c0e1410 0x7fe29c001c60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe2a00098a0 tx=0x7fe2a002b910 comp rx=0 tx=0).stop 2023-09-23T03:15:52.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.933+0000 7fda653c5640 1 -- 172.21.15.154:0/1401980209 wait complete. 2023-09-23T03:15:52.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.933+0000 7fad76ffd640 1 -- 172.21.15.154:0/2902557866 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c02f8a0 con 0x7fad7000c1f0 2023-09-23T03:15:52.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.933+0000 7fad76ffd640 1 -- 172.21.15.154:0/2902557866 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad6c02fa00 con 0x7fad7000c1f0 2023-09-23T03:15:52.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.934+0000 7fad76ffd640 1 -- 172.21.15.154:0/2902557866 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c02fd90 con 0x7fad7000c1f0 2023-09-23T03:15:52.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.934+0000 7f26a37fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f269c0544a0 0x7f269c0573e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:42294/0 (socket says 172.21.15.154:42294) 2023-09-23T03:15:52.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.934+0000 7f26a2ffd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f269c0e3190 0x7f269c03e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:52.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.934+0000 7f26a27fc640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f269c038180 0x7f269c053d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.154:42386/0 (socket says 172.21.15.154:42386) 2023-09-23T03:15:52.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.934+0000 7fe2ab3e7640 1 -- 172.21.15.154:0/115775837 shutdown_connections 2023-09-23T03:15:52.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.934+0000 7fe2ab3e7640 1 --2- 172.21.15.154:0/115775837 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe29c0358c0 0x7fe29c000e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.934+0000 7fe2ab3e7640 1 --2- 172.21.15.154:0/115775837 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe29c0e1410 0x7fe29c001c60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.934+0000 7fe2ab3e7640 1 -- 172.21.15.154:0/115775837 >> 172.21.15.154:0/115775837 conn(0x7fe29c00ba30 msgr2=0x7fe29c00be40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:52.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.934+0000 7f26a27fc640 1 -- 172.21.15.154:0/2721102850 learned_addr learned my addr 172.21.15.154:0/2721102850 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:52.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.935+0000 7f301a094640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f300c053180 0x7f300c0061c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.935+0000 7fad7d639640 1 -- 172.21.15.154:0/2902557866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad7000c1f0 msgr2=0x7fad70033fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:52.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.935+0000 7fad7d639640 1 --2- 172.21.15.154:0/2902557866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad7000c1f0 0x7fad70033fc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fad6c0097a0 tx=0x7fad6c02dae0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.935+0000 7fe2ab3e7640 1 -- 172.21.15.154:0/115775837 shutdown_connections 2023-09-23T03:15:52.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.935+0000 7fa01dfd0640 1 Processor -- start 2023-09-23T03:15:52.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.936+0000 7fa01dfd0640 1 -- start start 2023-09-23T03:15:52.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.935+0000 7fad7d639640 1 -- 172.21.15.154:0/2902557866 shutdown_connections 2023-09-23T03:15:52.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.935+0000 7fad7d639640 1 --2- 172.21.15.154:0/2902557866 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad70053150 0x7fad700e36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.935+0000 7fad7d639640 1 --2- 172.21.15.154:0/2902557866 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad7000c1f0 0x7fad70033fc0 secure :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fad6c0097a0 tx=0x7fad6c02dae0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.935+0000 7fad7d639640 1 -- 172.21.15.154:0/2902557866 >> 172.21.15.154:0/2902557866 conn(0x7fad70054c10 msgr2=0x7fad70054ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:52.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.935+0000 7f301a094640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f300c054420 con 0x7f300c006700 2023-09-23T03:15:52.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.935+0000 7f301a094640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f300c0343d0 con 0x7f300c033ff0 2023-09-23T03:15:52.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.935+0000 7f3013fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f300c006700 0x7f300c006b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:42296/0 (socket says 172.21.15.154:42296) 2023-09-23T03:15:52.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.936+0000 7f3013fff640 1 -- 172.21.15.154:0/4026762566 learned_addr learned my addr 172.21.15.154:0/4026762566 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:52.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.936+0000 7f3012ffd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f300c053180 0x7f300c0061c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:52.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.936+0000 7fad7d639640 1 -- 172.21.15.154:0/2902557866 shutdown_connections 2023-09-23T03:15:52.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.936+0000 7fad7d639640 1 -- 172.21.15.154:0/2902557866 wait complete. 2023-09-23T03:15:52.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.936+0000 7f301a094640 1 -- 172.21.15.154:0/4026762566 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f300c053600 con 0x7f300c053180 2023-09-23T03:15:52.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.937+0000 7fda653c5640 1 Processor -- start 2023-09-23T03:15:52.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.937+0000 7fda653c5640 1 -- start start 2023-09-23T03:15:52.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.937+0000 7f30127fc640 1 -- 172.21.15.154:0/4026762566 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3969986541 0 0) 0x7f300c054420 con 0x7f300c006700 2023-09-23T03:15:52.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.937+0000 7f30127fc640 1 -- 172.21.15.154:0/4026762566 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3004003520 con 0x7f300c006700 2023-09-23T03:15:52.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.935+0000 7fe2ab3e7640 1 -- 172.21.15.154:0/115775837 wait complete. 2023-09-23T03:15:52.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.936+0000 7f26a2ffd640 1 -- 172.21.15.154:0/2721102850 >> v1:172.21.15.190:6789/0 conn(0x7f269c038180 legacy=0x7f269c053d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:52.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.936+0000 7f26a2ffd640 1 -- 172.21.15.154:0/2721102850 >> v1:172.21.15.154:6789/0 conn(0x7f269c0544a0 legacy=0x7f269c0573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:52.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.936+0000 7f26a2ffd640 1 -- 172.21.15.154:0/2721102850 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f269c007160 con 0x7f269c0e3190 2023-09-23T03:15:52.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.938+0000 7f26a2ffd640 1 --2- 172.21.15.154:0/2721102850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f269c0e3190 0x7f269c03e920 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2698009750 tx=0x7f269802b7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85ea5ee9a45340e7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:52.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.939+0000 7f26a1ffb640 1 -- 172.21.15.154:0/2721102850 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2698031710 con 0x7f269c0e3190 2023-09-23T03:15:52.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.939+0000 7f26a1ffb640 1 -- 172.21.15.154:0/2721102850 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f269802f050 con 0x7f269c0e3190 2023-09-23T03:15:52.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.939+0000 7f26a1ffb640 1 -- 172.21.15.154:0/2721102850 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2698031a90 con 0x7f269c0e3190 2023-09-23T03:15:52.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.939+0000 7fad7d639640 1 Processor -- start 2023-09-23T03:15:52.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.938+0000 7fe2ab3e7640 1 Processor -- start 2023-09-23T03:15:52.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.938+0000 7fe2ab3e7640 1 -- start start 2023-09-23T03:15:52.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.938+0000 7f3012ffd640 1 -- 172.21.15.154:0/4026762566 >> v1:172.21.15.190:6789/0 conn(0x7f300c033ff0 legacy=0x7f300c050260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:52.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.938+0000 7f3012ffd640 1 -- 172.21.15.154:0/4026762566 >> v1:172.21.15.154:6789/0 conn(0x7f300c006700 legacy=0x7f300c006b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:52.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.938+0000 7f3012ffd640 1 -- 172.21.15.154:0/4026762566 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f300c053810 con 0x7f300c053180 2023-09-23T03:15:52.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.938+0000 7fa0177fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fa01815c930 0x7fa0180d0960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:42306/0 (socket says 172.21.15.154:42306) 2023-09-23T03:15:52.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.938+0000 7fa0177fe640 1 -- 172.21.15.154:0/4232298864 learned_addr learned my addr 172.21.15.154:0/4232298864 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:52.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.939+0000 7fa01dfd0640 1 --2- 172.21.15.154:0/4232298864 >> v2:172.21.15.154:3300/0 conn(0x7fa01806c560 0x7fa018167450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.939+0000 7fa017fff640 1 --2- 172.21.15.154:0/4232298864 >> v2:172.21.15.154:3300/0 conn(0x7fa01806c560 0x7fa018167450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:52.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.940+0000 7fad7d639640 1 -- start start 2023-09-23T03:15:52.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.940+0000 7fa01dfd0640 1 -- 172.21.15.154:0/4232298864 --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa01806b0c0 con 0x7fa01815c930 2023-09-23T03:15:52.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.940+0000 7fa01dfd0640 1 -- 172.21.15.154:0/4232298864 --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0180b4d60 con 0x7fa0180d1070 2023-09-23T03:15:52.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.940+0000 7fa01dfd0640 1 -- 172.21.15.154:0/4232298864 --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa01815cd10 con 0x7fa01806c560 2023-09-23T03:15:52.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.940+0000 7fe2ab3e7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe29c001360 0x7fe29c0df850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.941+0000 7f26a3fff640 1 -- 172.21.15.154:0/2721102850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f269c0e3190 msgr2=0x7f269c03e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:52.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.941+0000 7f26a3fff640 1 --2- 172.21.15.154:0/2721102850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f269c0e3190 0x7f269c03e920 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2698009750 tx=0x7f269802b7b0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.941+0000 7fa0167fc640 1 -- 172.21.15.154:0/4232298864 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1418581676 0 0) 0x7fa01806b0c0 con 0x7fa01815c930 2023-09-23T03:15:52.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.941+0000 7fa0167fc640 1 -- 172.21.15.154:0/4232298864 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa008003520 con 0x7fa01815c930 2023-09-23T03:15:52.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.941+0000 7f26a3fff640 1 -- 172.21.15.154:0/2721102850 shutdown_connections 2023-09-23T03:15:52.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.941+0000 7f26a3fff640 1 --2- 172.21.15.154:0/2721102850 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f269c0e3190 0x7f269c03e920 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2698009750 tx=0x7f269802b7b0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.941+0000 7f26a3fff640 1 -- 172.21.15.154:0/2721102850 >> 172.21.15.154:0/2721102850 conn(0x7f269c0590c0 msgr2=0x7f269c0594d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:52.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.941+0000 7fad7d639640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad70050230 0x7fad700d7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.941+0000 7fad7d639640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad70053150 0x7fad700e1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.941+0000 7fad7d639640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad70032f70 0x7fad700e1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.941+0000 7fa0167fc640 1 -- 172.21.15.154:0/4232298864 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3644779827 0 0) 0x7fa0180b4d60 con 0x7fa0180d1070 2023-09-23T03:15:52.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.941+0000 7fa0167fc640 1 -- 172.21.15.154:0/4232298864 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa01806b0c0 con 0x7fa0180d1070 2023-09-23T03:15:52.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.941+0000 7fad7d639640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad700da550 con 0x7fad70032f70 2023-09-23T03:15:52.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.941+0000 7fad7d639640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad700d7580 con 0x7fad70053150 2023-09-23T03:15:52.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.941+0000 7fad7d639640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad7004c610 con 0x7fad70050230 2023-09-23T03:15:52.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.941+0000 7fad7ce38640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad70032f70 0x7fad700e1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:52.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.941+0000 7fad7ce38640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad70032f70 0x7fad700e1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44534/0 (socket says 172.21.15.154:44534) 2023-09-23T03:15:52.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.941+0000 7fad7ce38640 1 -- 172.21.15.154:0/2709785318 learned_addr learned my addr 172.21.15.154:0/2709785318 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:52.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.941+0000 7fa0167fc640 1 -- 172.21.15.154:0/4232298864 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1199780050 0 0) 0x7fa008003520 con 0x7fa01815c930 2023-09-23T03:15:52.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.941+0000 7fa0167fc640 1 -- 172.21.15.154:0/4232298864 >> v2:172.21.15.154:3300/0 conn(0x7fa01806c560 msgr2=0x7fa018167450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:52.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.941+0000 7fa0167fc640 1 --2- 172.21.15.154:0/4232298864 >> v2:172.21.15.154:3300/0 conn(0x7fa01806c560 0x7fa018167450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.941+0000 7fa0167fc640 1 -- 172.21.15.154:0/4232298864 >> v1:172.21.15.190:6789/0 conn(0x7fa0180d1070 legacy=0x7fa0180cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:52.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.942+0000 7fa0167fc640 1 -- 172.21.15.154:0/4232298864 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0180cede0 con 0x7fa01815c930 2023-09-23T03:15:52.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.942+0000 7fa0167fc640 1 -- 172.21.15.154:0/4232298864 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fa00c0016c0 con 0x7fa01815c930 2023-09-23T03:15:52.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.942+0000 7fda653c5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda5800c220 0x7fda580452d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.942+0000 7f26a3fff640 1 -- 172.21.15.154:0/2721102850 shutdown_connections 2023-09-23T03:15:52.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.942+0000 7fad777fe640 1 --2- 172.21.15.154:0/2709785318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad70053150 0x7fad700e1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:52.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.942+0000 7fad77fff640 1 --2- 172.21.15.154:0/2709785318 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad70050230 0x7fad700d7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:52.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.942+0000 7fa0167fc640 1 -- 172.21.15.154:0/4232298864 >> v1:172.21.15.154:6789/0 conn(0x7fa01815c930 legacy=0x7fa0180d0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:52.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.942+0000 7fa0167fc640 1 --2- 172.21.15.154:0/4232298864 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa008003bc0 0x7fa008003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.942+0000 7fa0167fc640 1 --2- 172.21.15.154:0/4232298864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0080047c0 0x7fa008004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.942+0000 7fa0167fc640 1 --2- 172.21.15.154:0/4232298864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa008005180 0x7fa00800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.942+0000 7fa0167fc640 1 -- 172.21.15.154:0/4232298864 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01815cd10 con 0x7fa008005180 2023-09-23T03:15:52.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.942+0000 7fa0167fc640 1 -- 172.21.15.154:0/4232298864 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa00800c120 con 0x7fa0080047c0 2023-09-23T03:15:52.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.942+0000 7fa0167fc640 1 -- 172.21.15.154:0/4232298864 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa00800c2e0 con 0x7fa008003bc0 2023-09-23T03:15:52.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.942+0000 7fda653c5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda58033ff0 0x7fda580e1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.958 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.942+0000 7fa017fff640 1 --2- 172.21.15.154:0/4232298864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa008005180 0x7fa00800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:52.958 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.942+0000 7fda653c5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda580df9a0 0x7fda58006d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.958 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.942+0000 7fda5effd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda5800c220 0x7fda580452d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:52.958 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.942+0000 7f26a3fff640 1 -- 172.21.15.154:0/2721102850 wait complete. 2023-09-23T03:15:52.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.943+0000 7fda653c5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5804c7c0 con 0x7fda58033ff0 2023-09-23T03:15:52.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.943+0000 7fda653c5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda580e0b20 con 0x7fda5800c220 2023-09-23T03:15:52.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.941+0000 7fe2a9183640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe29c001360 0x7fe29c0df850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:52.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.943+0000 7fda653c5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda580e0e00 con 0x7fda580df9a0 2023-09-23T03:15:52.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.943+0000 7fda5effd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda5800c220 0x7fda580452d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.154:47830/0 (socket says 172.21.15.154:47830) 2023-09-23T03:15:52.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.943+0000 7fda5effd640 1 -- 172.21.15.154:0/2573598503 learned_addr learned my addr 172.21.15.154:0/2573598503 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:52.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.943+0000 7fe2a9183640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe29c001360 0x7fe29c0df850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44530/0 (socket says 172.21.15.154:44530) 2023-09-23T03:15:52.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.943+0000 7fe2a9183640 1 -- 172.21.15.154:0/1957640776 learned_addr learned my addr 172.21.15.154:0/1957640776 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:52.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.943+0000 7fda5f7fe640 1 --2- 172.21.15.154:0/2573598503 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda580df9a0 0x7fda58006d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:52.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.943+0000 7fe2ab3e7640 1 --2- 172.21.15.154:0/1957640776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe29c0358c0 0x7fe29c0e40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.943+0000 7fad7ce38640 1 -- 172.21.15.154:0/2709785318 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad70050230 msgr2=0x7fad700d7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:52.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.943+0000 7fad7ce38640 1 --2- 172.21.15.154:0/2709785318 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad70050230 0x7fad700d7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.943+0000 7fad7ce38640 1 -- 172.21.15.154:0/2709785318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad70053150 msgr2=0x7fad700e1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:52.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.943+0000 7fad7ce38640 1 --2- 172.21.15.154:0/2709785318 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad70053150 0x7fad700e1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.943+0000 7fad7ce38640 1 -- 172.21.15.154:0/2709785318 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad6c009400 con 0x7fad70032f70 2023-09-23T03:15:52.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.943+0000 7fa0177fe640 1 --2- 172.21.15.154:0/4232298864 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa008003bc0 0x7fa008003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:52.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.943+0000 7fa016ffd640 1 --2- 172.21.15.154:0/4232298864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0080047c0 0x7fa008004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:52.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.943+0000 7fa017fff640 1 -- 172.21.15.154:0/4232298864 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa008003bc0 msgr2=0x7fa008003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:52.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.943+0000 7fa017fff640 1 --2- 172.21.15.154:0/4232298864 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa008003bc0 0x7fa008003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.943+0000 7fa017fff640 1 -- 172.21.15.154:0/4232298864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0080047c0 msgr2=0x7fa008004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:52.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.943+0000 7fda5e7fc640 1 --2- 172.21.15.154:0/2573598503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda58033ff0 0x7fda580e1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:52.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.943+0000 7fa017fff640 1 --2- 172.21.15.154:0/4232298864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0080047c0 0x7fa008004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.944+0000 7fe2ab3e7640 1 --2- 172.21.15.154:0/1957640776 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe29c0e1410 0x7fe29c0383e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.943+0000 7fa017fff640 1 -- 172.21.15.154:0/4232298864 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa008003520 con 0x7fa008005180 2023-09-23T03:15:52.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.944+0000 7fe2ab3e7640 1 -- 172.21.15.154:0/1957640776 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c0e4610 con 0x7fe29c001360 2023-09-23T03:15:52.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.944+0000 7fa016ffd640 1 --2- 172.21.15.154:0/4232298864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0080047c0 0x7fa008004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:52.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.944+0000 7f3012ffd640 1 --2- 172.21.15.154:0/4026762566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f300c053180 0x7f300c0061c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f30000097d0 tx=0x7f300002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13548927d89f730b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:52.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.944+0000 7fe2ab3e7640 1 -- 172.21.15.154:0/1957640776 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c0e0080 con 0x7fe29c0358c0 2023-09-23T03:15:52.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.944+0000 7fe2ab3e7640 1 -- 172.21.15.154:0/1957640776 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c04e870 con 0x7fe29c0e1410 2023-09-23T03:15:52.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.944+0000 7fda5f7fe640 1 -- 172.21.15.154:0/2573598503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda5800c220 msgr2=0x7fda580452d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:52.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.944+0000 7fda5f7fe640 1 --2- 172.21.15.154:0/2573598503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda5800c220 0x7fda580452d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.944+0000 7fda5f7fe640 1 -- 172.21.15.154:0/2573598503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda58033ff0 msgr2=0x7fda580e1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:52.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.945+0000 7fda5f7fe640 1 --2- 172.21.15.154:0/2573598503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda58033ff0 0x7fda580e1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.945+0000 7fda5f7fe640 1 -- 172.21.15.154:0/2573598503 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda6006ce70 con 0x7fda580df9a0 2023-09-23T03:15:52.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.945+0000 7fda5e7fc640 1 --2- 172.21.15.154:0/2573598503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda58033ff0 0x7fda580e1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:15:52.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.945+0000 7f26a3fff640 1 Processor -- start 2023-09-23T03:15:52.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.945+0000 7f30127fc640 1 -- 172.21.15.154:0/4026762566 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3000031800 con 0x7f300c053180 2023-09-23T03:15:52.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.945+0000 7f26a3fff640 1 -- start start 2023-09-23T03:15:52.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.945+0000 7fe7ad4a2640 1 Processor -- start 2023-09-23T03:15:52.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.945+0000 7fda5f7fe640 1 --2- 172.21.15.154:0/2573598503 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda580df9a0 0x7fda58006d70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fda6002cb00 tx=0x7fda60069b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:52.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.944+0000 7fad7ce38640 1 --2- 172.21.15.154:0/2709785318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad70032f70 0x7fad700e1580 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fad78067d60 tx=0x7fad7804e770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:52.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.945+0000 7fad74ff9640 1 -- 172.21.15.154:0/2709785318 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad780726a0 con 0x7fad70032f70 2023-09-23T03:15:52.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.945+0000 7fad74ff9640 1 -- 172.21.15.154:0/2709785318 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad78072820 con 0x7fad70032f70 2023-09-23T03:15:52.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.945+0000 7fad74ff9640 1 -- 172.21.15.154:0/2709785318 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad78072bd0 con 0x7fad70032f70 2023-09-23T03:15:52.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.945+0000 7f30127fc640 1 -- 172.21.15.154:0/4026762566 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3000031960 con 0x7f300c053180 2023-09-23T03:15:52.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.945+0000 7fad7d639640 1 -- 172.21.15.154:0/2709785318 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad70033400 con 0x7fad70032f70 2023-09-23T03:15:52.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.944+0000 7fe2a8982640 1 --2- 172.21.15.154:0/1957640776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe29c0358c0 0x7fe29c0e40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:52.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.944+0000 7fe2aa185640 1 --2- 172.21.15.154:0/1957640776 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe29c0e1410 0x7fe29c0383e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:52.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.945+0000 7fe2aa185640 1 -- 172.21.15.154:0/1957640776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe29c0358c0 msgr2=0x7fe29c0e40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:52.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.945+0000 7fe2aa185640 1 --2- 172.21.15.154:0/1957640776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe29c0358c0 0x7fe29c0e40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.946+0000 7fe2aa185640 1 -- 172.21.15.154:0/1957640776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe29c001360 msgr2=0x7fe29c0df850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:52.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.946+0000 7fe2aa185640 1 --2- 172.21.15.154:0/1957640776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe29c001360 0x7fe29c0df850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.946+0000 7fe2aa185640 1 -- 172.21.15.154:0/1957640776 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a00094d0 con 0x7fe29c0e1410 2023-09-23T03:15:52.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.946+0000 7fe2a9183640 1 --2- 172.21.15.154:0/1957640776 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe29c001360 0x7fe29c0df850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:52.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.946+0000 7fda477fe640 1 -- 172.21.15.154:0/2573598503 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda600b6940 con 0x7fda580df9a0 2023-09-23T03:15:52.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.946+0000 7fda477fe640 1 -- 172.21.15.154:0/2573598503 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda600a5e90 con 0x7fda580df9a0 2023-09-23T03:15:52.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.947+0000 7fda477fe640 1 -- 172.21.15.154:0/2573598503 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda600b6cc0 con 0x7fda580df9a0 2023-09-23T03:15:52.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.947+0000 7f26a3fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f269c038180 0x7f269c00bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.947+0000 7f26a3fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f269c0544a0 0x7f269c17e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.947+0000 7f26a3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f269c00d700 0x7f269c16a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.947+0000 7f26a3fff640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269c17e6c0 con 0x7f269c00d700 2023-09-23T03:15:52.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.947+0000 7f26a3fff640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269c1697e0 con 0x7f269c0544a0 2023-09-23T03:15:52.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.947+0000 7f26a3fff640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f269c169ac0 con 0x7f269c038180 2023-09-23T03:15:52.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.946+0000 7fad7d639640 1 -- 172.21.15.154:0/2709785318 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad7004bfc0 con 0x7fad70032f70 2023-09-23T03:15:52.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.946+0000 7fe2aa185640 1 --2- 172.21.15.154:0/1957640776 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe29c0e1410 0x7fe29c0383e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe2a404f340 tx=0x7fe2a40734a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:52.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.946+0000 7f30127fc640 1 -- 172.21.15.154:0/4026762566 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3000031800 con 0x7f300c053180 2023-09-23T03:15:52.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.947+0000 7f301a094640 1 -- 172.21.15.154:0/4026762566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f300c053180 msgr2=0x7f300c0061c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:52.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.947+0000 7f301a094640 1 --2- 172.21.15.154:0/4026762566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f300c053180 0x7f300c0061c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f30000097d0 tx=0x7f300002b8a0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.947+0000 7f301a094640 1 -- 172.21.15.154:0/4026762566 shutdown_connections 2023-09-23T03:15:52.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.947+0000 7f301a094640 1 --2- 172.21.15.154:0/4026762566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f300c053180 0x7f300c0061c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.947+0000 7f301a094640 1 -- 172.21.15.154:0/4026762566 >> 172.21.15.154:0/4026762566 conn(0x7f300c054c40 msgr2=0x7f300c055020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:52.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.946+0000 7fa017fff640 1 --2- 172.21.15.154:0/4232298864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa008005180 0x7fa00800b980 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa01000e490 tx=0x7fa0080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afff320df66463c4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:52.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.946+0000 7fa017fff640 1 -- 172.21.15.154:0/4232298864 reap_dead start 2023-09-23T03:15:52.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.947+0000 7fa0167fc640 1 -- 172.21.15.154:0/4232298864 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01002d7a0 con 0x7fa008005180 2023-09-23T03:15:52.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.947+0000 7fa0167fc640 1 -- 172.21.15.154:0/4232298864 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa01002d900 con 0x7fa008005180 2023-09-23T03:15:52.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.947+0000 7fa0167fc640 1 -- 172.21.15.154:0/4232298864 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01002dc70 con 0x7fa008005180 2023-09-23T03:15:52.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.945+0000 7f806e8b0640 1 Processor -- start 2023-09-23T03:15:52.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.946+0000 7f806e8b0640 1 -- start start 2023-09-23T03:15:52.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.947+0000 7f806e8b0640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8068161010 0x7f80680ad370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.947+0000 7f806e8b0640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f80680afba0 0x7f80680c14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.947+0000 7f806e8b0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80680ca840 0x7f80680c1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.947+0000 7f806e8b0640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80680cc410 con 0x7f8068161010 2023-09-23T03:15:52.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.947+0000 7f806e8b0640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80680ce2b0 con 0x7f80680afba0 2023-09-23T03:15:52.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.947+0000 7f806e8b0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80680cd6a0 con 0x7f80680ca840 2023-09-23T03:15:52.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.948+0000 7fda653c5640 1 -- 172.21.15.154:0/2573598503 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda5804ca60 con 0x7fda580df9a0 2023-09-23T03:15:52.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.948+0000 7fad627fc640 1 -- 172.21.15.154:0/2709785318 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fad70054080 con 0x7fad70032f70 2023-09-23T03:15:52.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.948+0000 7fda653c5640 1 -- 172.21.15.154:0/2573598503 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda580e1a90 con 0x7fda580df9a0 2023-09-23T03:15:52.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.948+0000 7f301a094640 1 -- 172.21.15.154:0/4026762566 shutdown_connections 2023-09-23T03:15:52.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.946+0000 7fe7ad4a2640 1 -- start start 2023-09-23T03:15:52.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.948+0000 7fe299ffb640 1 -- 172.21.15.154:0/1957640776 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4061220 con 0x7fe29c0e1410 2023-09-23T03:15:52.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.948+0000 7f26a37fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f269c00d700 0x7f269c16a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:52.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.948+0000 7f26a37fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f269c00d700 0x7f269c16a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44572/0 (socket says 172.21.15.154:44572) 2023-09-23T03:15:52.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.948+0000 7f26a37fe640 1 -- 172.21.15.154:0/533522302 learned_addr learned my addr 172.21.15.154:0/533522302 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:52.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.948+0000 7fe299ffb640 1 -- 172.21.15.154:0/1957640776 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a40613c0 con 0x7fe29c0e1410 2023-09-23T03:15:52.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.949+0000 7f26a27fc640 1 --2- 172.21.15.154:0/533522302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f269c0544a0 0x7f269c17e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:52.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.948+0000 7f301a094640 1 -- 172.21.15.154:0/4026762566 wait complete. 2023-09-23T03:15:52.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.949+0000 7fa01dfd0640 1 -- 172.21.15.154:0/4232298864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa008005180 msgr2=0x7fa00800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:52.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.949+0000 7fa01dfd0640 1 --2- 172.21.15.154:0/4232298864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa008005180 0x7fa00800b980 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa01000e490 tx=0x7fa0080057f0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.949+0000 7f806ce4d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80680ca840 0x7f80680c1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:52.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.949+0000 7fa01dfd0640 1 -- 172.21.15.154:0/4232298864 shutdown_connections 2023-09-23T03:15:52.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.948+0000 7fe7ad4a2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7a815cac0 0x7fe7a806b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.948+0000 7fe7ad4a2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe7a80cdd80 0x7fe7a80cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.948+0000 7fe7ad4a2640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7a80ce990 con 0x7fe7a80cf2a0 2023-09-23T03:15:52.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.948+0000 7fe7ad4a2640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7a80ce1f0 con 0x7fe7a80cdd80 2023-09-23T03:15:52.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.948+0000 7fe7ad4a2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a80cfc20 con 0x7fe7a815cac0 2023-09-23T03:15:52.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.949+0000 7fa01dfd0640 1 --2- 172.21.15.154:0/4232298864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa008005180 0x7fa00800b980 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.949+0000 7fa01dfd0640 1 -- 172.21.15.154:0/4232298864 >> 172.21.15.154:0/4232298864 conn(0x7fa01809b9b0 msgr2=0x7fa0180d2630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:52.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.949+0000 7fe299ffb640 1 -- 172.21.15.154:0/1957640776 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4060770 con 0x7fe29c0e1410 2023-09-23T03:15:52.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.949+0000 7fe7a6ffd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7a815cac0 0x7fe7a806b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:52.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.949+0000 7fe7a6ffd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7a815cac0 0x7fe7a806b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:43932/0 (socket says 172.21.15.154:43932) 2023-09-23T03:15:52.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.949+0000 7fe7a6ffd640 1 -- 172.21.15.154:0/131857553 learned_addr learned my addr 172.21.15.154:0/131857553 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:52.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.950+0000 7f26a2ffd640 1 --2- 172.21.15.154:0/533522302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f269c038180 0x7f269c00bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:52.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.949+0000 7f8067fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8068161010 0x7f80680ad370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:52.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.949+0000 7f8067fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8068161010 0x7f80680ad370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44576/0 (socket says 172.21.15.154:44576) 2023-09-23T03:15:52.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.949+0000 7f8067fff640 1 -- 172.21.15.154:0/4238702324 learned_addr learned my addr 172.21.15.154:0/4238702324 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:52.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.950+0000 7f80677fe640 1 --2- 172.21.15.154:0/4238702324 >> v2:172.21.15.190:3300/0 conn(0x7f80680afba0 0x7f80680c14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:52.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.950+0000 7f8067fff640 1 -- 172.21.15.154:0/4238702324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80680ca840 msgr2=0x7f80680c1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:52.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.950+0000 7fa01dfd0640 1 -- 172.21.15.154:0/4232298864 shutdown_connections 2023-09-23T03:15:52.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.950+0000 7f26a37fe640 1 -- 172.21.15.154:0/533522302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f269c038180 msgr2=0x7f269c00bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:52.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.950+0000 7fe2ab3e7640 1 -- 172.21.15.154:0/1957640776 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe29c04ebb0 con 0x7fe29c0e1410 2023-09-23T03:15:52.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.950+0000 7fa01dfd0640 1 -- 172.21.15.154:0/4232298864 wait complete. 2023-09-23T03:15:52.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.950+0000 7f8067fff640 1 --2- 172.21.15.154:0/4238702324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80680ca840 0x7f80680c1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.950+0000 7f8067fff640 1 -- 172.21.15.154:0/4238702324 >> v2:172.21.15.190:3300/0 conn(0x7f80680afba0 msgr2=0x7f80680c14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:52.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.950+0000 7f26a37fe640 1 --2- 172.21.15.154:0/533522302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f269c038180 0x7f269c00bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.950+0000 7f8067fff640 1 --2- 172.21.15.154:0/4238702324 >> v2:172.21.15.190:3300/0 conn(0x7f80680afba0 0x7f80680c14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.950+0000 7f26a37fe640 1 -- 172.21.15.154:0/533522302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f269c0544a0 msgr2=0x7f269c17e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:52.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.950+0000 7f26a37fe640 1 --2- 172.21.15.154:0/533522302 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f269c0544a0 0x7f269c17e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.986 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.950+0000 7f8067fff640 1 -- 172.21.15.154:0/4238702324 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80680cd980 con 0x7f8068161010 2023-09-23T03:15:52.986 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.950+0000 7fe2ab3e7640 1 -- 172.21.15.154:0/1957640776 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe29c038920 con 0x7fe29c0e1410 2023-09-23T03:15:52.986 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.950+0000 7f26a37fe640 1 -- 172.21.15.154:0/533522302 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2698009400 con 0x7f269c00d700 2023-09-23T03:15:52.986 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.950+0000 7f301a094640 1 Processor -- start 2023-09-23T03:15:52.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.951+0000 7f80677fe640 1 --2- 172.21.15.154:0/4238702324 >> v2:172.21.15.190:3300/0 conn(0x7f80680afba0 0x7f80680c14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:52.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.950+0000 7fe7a6ffd640 1 -- 172.21.15.154:0/131857553 >> v2:172.21.15.154:3300/0 conn(0x7fe7a80cdd80 msgr2=0x7fe7a80cf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:52.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.950+0000 7fe7a6ffd640 1 --2- 172.21.15.154:0/131857553 >> v2:172.21.15.154:3300/0 conn(0x7fe7a80cdd80 0x7fe7a80cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.950+0000 7fe7a6ffd640 1 -- 172.21.15.154:0/131857553 >> v1:172.21.15.190:6789/0 conn(0x7fe7a80cf2a0 legacy=0x7fe7a80cd670 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T03:15:52.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.951+0000 7fe7a6ffd640 1 -- 172.21.15.154:0/131857553 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a80ce3d0 con 0x7fe7a815cac0 2023-09-23T03:15:52.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.951+0000 7f301a094640 1 -- start start 2023-09-23T03:15:52.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.951+0000 7f26a2ffd640 1 --2- 172.21.15.154:0/533522302 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f269c038180 0x7f269c00bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:52.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.951+0000 7f26a37fe640 1 --2- 172.21.15.154:0/533522302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f269c00d700 0x7f269c16a880 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f26a40657c0 tx=0x7f26a404e770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:52.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.949+0000 7fad74ff9640 1 -- 172.21.15.154:0/2709785318 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fad7808f070 con 0x7fad70032f70 2023-09-23T03:15:52.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.951+0000 7fad74ff9640 1 --2- 172.21.15.154:0/2709785318 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fad5c06afb0 0x7fad5c06d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.951+0000 7fad74ff9640 1 -- 172.21.15.154:0/2709785318 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fad781020f0 con 0x7fad70032f70 2023-09-23T03:15:52.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.951+0000 7fad77fff640 1 --2- 172.21.15.154:0/2709785318 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fad5c06afb0 0x7fad5c06d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:52.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.951+0000 7fad74ff9640 1 --2- 172.21.15.154:0/2709785318 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x7fad5c0745f0 0x7fad5c076a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.951+0000 7fad74ff9640 1 -- 172.21.15.154:0/2709785318 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fad5c077100 con 0x7fad5c0745f0 2023-09-23T03:15:52.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.951+0000 7fad74ff9640 1 -- 172.21.15.154:0/2709785318 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad781023b0 con 0x7fad70032f70 2023-09-23T03:15:52.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.952+0000 7f301a094640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f300c006700 0x7f300c0dd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.952+0000 7f268b7fe640 1 -- 172.21.15.154:0/533522302 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a40726a0 con 0x7f269c00d700 2023-09-23T03:15:52.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.952+0000 7f301a094640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f300c033ff0 0x7f300c02b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.952+0000 7f301a094640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f300c053180 0x7f300c02e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.953+0000 7f301a094640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300c02dd70 con 0x7f300c033ff0 2023-09-23T03:15:52.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.953+0000 7f301a094640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300c02bfd0 con 0x7f300c053180 2023-09-23T03:15:52.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.953+0000 7f30137fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f300c006700 0x7f300c0dd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:52.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.953+0000 7fe7a6ffd640 1 --2- 172.21.15.154:0/131857553 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7a815cac0 0x7fe7a806b280 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe798009850 tx=0x7fe79802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af0ee330b8923e46 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:52.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.953+0000 7f268b7fe640 1 -- 172.21.15.154:0/533522302 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a408d070 con 0x7f269c00d700 2023-09-23T03:15:52.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.953+0000 7f268b7fe640 1 -- 172.21.15.154:0/533522302 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a4072ac0 con 0x7f269c00d700 2023-09-23T03:15:52.992 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.953+0000 7f26a3fff640 1 -- 172.21.15.154:0/533522302 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f269c169da0 con 0x7f269c00d700 2023-09-23T03:15:52.992 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.953+0000 7f301a094640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f300c02c2b0 con 0x7f300c006700 2023-09-23T03:15:52.992 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.953+0000 7f30137fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f300c006700 0x7f300c0dd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:43936/0 (socket says 172.21.15.154:43936) 2023-09-23T03:15:52.992 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.953+0000 7f30137fe640 1 -- 172.21.15.154:0/2422302356 learned_addr learned my addr 172.21.15.154:0/2422302356 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:52.992 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.953+0000 7f3013fff640 1 --2- 172.21.15.154:0/2422302356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f300c053180 0x7f300c02e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:52.992 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.953+0000 7f26a3fff640 1 -- 172.21.15.154:0/533522302 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f269c16a360 con 0x7f269c00d700 2023-09-23T03:15:52.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.953+0000 7fad777fe640 1 --2- 172.21.15.154:0/2709785318 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x7fad5c0745f0 0x7fad5c076a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:52.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.953+0000 7fad77fff640 1 --2- 172.21.15.154:0/2709785318 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fad5c06afb0 0x7fad5c06d470 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fad6c038000 tx=0x7fad6c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:52.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.953+0000 7f3012ffd640 1 --2- 172.21.15.154:0/2422302356 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f300c033ff0 0x7f300c02b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:52.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.953+0000 7fe7a5ffb640 1 -- 172.21.15.154:0/131857553 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe798004230 con 0x7fe7a815cac0 2023-09-23T03:15:52.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.954+0000 7fe7a5ffb640 1 -- 172.21.15.154:0/131857553 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe798004390 con 0x7fe7a815cac0 2023-09-23T03:15:52.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.954+0000 7fe7a5ffb640 1 -- 172.21.15.154:0/131857553 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe798031830 con 0x7fe7a815cac0 2023-09-23T03:15:52.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.951+0000 7fda653c5640 1 -- 172.21.15.154:0/2573598503 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fda580540b0 con 0x7fda580df9a0 2023-09-23T03:15:52.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.954+0000 7f30137fe640 1 -- 172.21.15.154:0/2422302356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f300c053180 msgr2=0x7f300c02e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:52.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.954+0000 7f30137fe640 1 --2- 172.21.15.154:0/2422302356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f300c053180 0x7f300c02e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.954+0000 7f30137fe640 1 -- 172.21.15.154:0/2422302356 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f300c033ff0 msgr2=0x7f300c02b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:52.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.954+0000 7fa01dfd0640 1 Processor -- start 2023-09-23T03:15:52.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.954+0000 7fa01dfd0640 1 -- start start 2023-09-23T03:15:52.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.954+0000 7fe2ab3e7640 1 -- 172.21.15.154:0/1957640776 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe29c0555e0 con 0x7fe29c0e1410 2023-09-23T03:15:52.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.954+0000 7fad777fe640 1 --2- 172.21.15.154:0/2709785318 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x7fad5c0745f0 0x7fad5c076a30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:52.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.955+0000 7f26a3fff640 1 -- 172.21.15.154:0/533522302 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f269c007440 con 0x7f269c00d700 2023-09-23T03:15:52.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.954+0000 7f30137fe640 1 --2- 172.21.15.154:0/2422302356 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f300c033ff0 0x7f300c02b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.955+0000 7f30137fe640 1 -- 172.21.15.154:0/2422302356 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3000009400 con 0x7f300c006700 2023-09-23T03:15:52.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.955+0000 7f3012ffd640 1 --2- 172.21.15.154:0/2422302356 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f300c033ff0 0x7f300c02b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:52.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.955+0000 7f8067fff640 1 --2- 172.21.15.154:0/4238702324 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8068161010 0x7f80680ad370 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f806806f3b0 tx=0x7f806002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b02f8d6d433571a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:52.997 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.955+0000 7fe299ffb640 1 -- 172.21.15.154:0/1957640776 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fe2a408f020 con 0x7fe29c0e1410 2023-09-23T03:15:52.997 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.955+0000 7f30137fe640 1 --2- 172.21.15.154:0/2422302356 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f300c006700 0x7f300c0dd6b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f30140685f0 tx=0x7f3014073870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:52.997 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.956+0000 7fda477fe640 1 -- 172.21.15.154:0/2573598503 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fda600b3040 con 0x7fda580df9a0 2023-09-23T03:15:52.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.956+0000 7fad74ff9640 1 -- 172.21.15.154:0/2709785318 <== osd.2 v2:172.21.15.154:6822/2884407243 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fad5c077100 con 0x7fad5c0745f0 2023-09-23T03:15:52.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.956+0000 7fa01dfd0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa008005180 0x7fa01815a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.956+0000 7f8066ffd640 1 -- 172.21.15.154:0/4238702324 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8060004230 con 0x7f8068161010 2023-09-23T03:15:52.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.956+0000 7f8066ffd640 1 -- 172.21.15.154:0/4238702324 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8060004390 con 0x7f8068161010 2023-09-23T03:15:52.999 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.956+0000 7fe7ad4a2640 1 -- 172.21.15.154:0/131857553 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7a815cac0 msgr2=0x7fe7a806b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:52.999 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.956+0000 7fe7ad4a2640 1 --2- 172.21.15.154:0/131857553 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7a815cac0 0x7fe7a806b280 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe798009850 tx=0x7fe79802b8d0 comp rx=0 tx=0).stop 2023-09-23T03:15:52.999 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.957+0000 7fe299ffb640 1 --2- 172.21.15.154:0/1957640776 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe28806af30 0x7fe28806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:52.999 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.957+0000 7fa01dfd0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa018159010 0x7fa018159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.957+0000 7f2ffffff640 1 -- 172.21.15.154:0/2422302356 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30140728a0 con 0x7f300c006700 2023-09-23T03:15:53.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.957+0000 7f2ffffff640 1 -- 172.21.15.154:0/2422302356 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3014072a40 con 0x7f300c006700 2023-09-23T03:15:53.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.957+0000 7f2ffffff640 1 -- 172.21.15.154:0/2422302356 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3014072e30 con 0x7f300c006700 2023-09-23T03:15:53.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.957+0000 7f301a094640 1 -- 172.21.15.154:0/2422302356 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f300c02d990 con 0x7f300c006700 2023-09-23T03:15:53.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.957+0000 7fa01dfd0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0181599b0 0x7fa0180c74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.957+0000 7f301a094640 1 -- 172.21.15.154:0/2422302356 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f300c02d330 con 0x7f300c006700 2023-09-23T03:15:53.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.957+0000 7fe299ffb640 1 -- 172.21.15.154:0/1957640776 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe2a41029d0 con 0x7fe29c0e1410 2023-09-23T03:15:53.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.957+0000 7fe299ffb640 1 --2- 172.21.15.154:0/1957640776 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x7fe288074380 0x7fe2880767c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.957+0000 7fe299ffb640 1 -- 172.21.15.154:0/1957640776 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe288076e90 con 0x7fe288074380 2023-09-23T03:15:53.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.957+0000 7fe299ffb640 1 -- 172.21.15.154:0/1957640776 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2a41072c0 con 0x7fe29c0e1410 2023-09-23T03:15:53.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.957+0000 7fa0177fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa008005180 0x7fa01815a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.957+0000 7fa01dfd0640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa018155de0 con 0x7fa008005180 2023-09-23T03:15:53.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.957+0000 7fa0177fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa008005180 0x7fa01815a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44608/0 (socket says 172.21.15.154:44608) 2023-09-23T03:15:53.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.957+0000 7fa0177fe640 1 -- 172.21.15.154:0/1580487460 learned_addr learned my addr 172.21.15.154:0/1580487460 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:53.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.958+0000 7fa01dfd0640 1 -- 172.21.15.154:0/1580487460 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0180c61d0 con 0x7fa018159010 2023-09-23T03:15:53.003 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.958+0000 7fe2a8982640 1 --2- 172.21.15.154:0/1957640776 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x7fe288074380 0x7fe2880767c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.003 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.958+0000 7f268b7fe640 1 -- 172.21.15.154:0/533522302 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f26a4072c20 con 0x7f269c00d700 2023-09-23T03:15:53.003 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.958+0000 7fa01dfd0640 1 -- 172.21.15.154:0/1580487460 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa018159e60 con 0x7fa0181599b0 2023-09-23T03:15:53.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.958+0000 7fe2a9183640 1 --2- 172.21.15.154:0/1957640776 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe28806af30 0x7fe28806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.958+0000 7fa017fff640 1 --2- 172.21.15.154:0/1580487460 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0181599b0 0x7fa0180c74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.958+0000 7fa016ffd640 1 --2- 172.21.15.154:0/1580487460 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa018159010 0x7fa018159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.959+0000 7fe2a8982640 1 --2- 172.21.15.154:0/1957640776 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x7fe288074380 0x7fe2880767c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.959+0000 7fa0177fe640 1 -- 172.21.15.154:0/1580487460 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0181599b0 msgr2=0x7fa0180c74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.959+0000 7fa0177fe640 1 --2- 172.21.15.154:0/1580487460 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0181599b0 0x7fa0180c74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.959+0000 7fa0177fe640 1 -- 172.21.15.154:0/1580487460 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa018159010 msgr2=0x7fa018159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.959+0000 7fa0177fe640 1 --2- 172.21.15.154:0/1580487460 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa018159010 0x7fa018159470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.959+0000 7fa0177fe640 1 -- 172.21.15.154:0/1580487460 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0180c6420 con 0x7fa008005180 2023-09-23T03:15:53.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.959+0000 7f268b7fe640 1 --2- 172.21.15.154:0/533522302 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f269006b000 0x7f269006d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.959+0000 7fa0177fe640 1 --2- 172.21.15.154:0/1580487460 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa008005180 0x7fa01815a8a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa00c00e700 tx=0x7fa00c00ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.959+0000 7f301a094640 1 -- 172.21.15.154:0/2422302356 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f300c053dd0 con 0x7f300c006700 2023-09-23T03:15:53.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.959+0000 7f268b7fe640 1 -- 172.21.15.154:0/533522302 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f26a41034d0 con 0x7f269c00d700 2023-09-23T03:15:53.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.960+0000 7fa003fff640 1 -- 172.21.15.154:0/1580487460 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c00c6c0 con 0x7fa008005180 2023-09-23T03:15:53.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.960+0000 7fa003fff640 1 -- 172.21.15.154:0/1580487460 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa00c00c820 con 0x7fa008005180 2023-09-23T03:15:53.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.960+0000 7fa003fff640 1 -- 172.21.15.154:0/1580487460 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c00cbd0 con 0x7fa008005180 2023-09-23T03:15:53.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.957+0000 7f806e8b0640 1 -- 172.21.15.154:0/4238702324 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8068161010 msgr2=0x7f80680ad370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.957+0000 7f806e8b0640 1 --2- 172.21.15.154:0/4238702324 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8068161010 0x7f80680ad370 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f806806f3b0 tx=0x7f806002b8f0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.957+0000 7f806e8b0640 1 -- 172.21.15.154:0/4238702324 shutdown_connections 2023-09-23T03:15:53.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.957+0000 7f806e8b0640 1 --2- 172.21.15.154:0/4238702324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80680ca840 0x7f80680c1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.957+0000 7f806e8b0640 1 --2- 172.21.15.154:0/4238702324 >> v2:172.21.15.190:3300/0 conn(0x7f80680afba0 0x7f80680c14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.957+0000 7f806e8b0640 1 --2- 172.21.15.154:0/4238702324 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8068161010 0x7f80680ad370 secure :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f806806f3b0 tx=0x7f806002b8f0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.957+0000 7f806e8b0640 1 -- 172.21.15.154:0/4238702324 >> 172.21.15.154:0/4238702324 conn(0x7f80680b0000 msgr2=0x7f80680b0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:53.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.960+0000 7fa01dfd0640 1 -- 172.21.15.154:0/1580487460 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0180c6700 con 0x7fa008005180 2023-09-23T03:15:53.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.960+0000 7f268b7fe640 1 --2- 172.21.15.154:0/533522302 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x7f2690074640 0x7f2690076a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.960+0000 7fe299ffb640 1 -- 172.21.15.154:0/1957640776 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fe288076e90 con 0x7fe288074380 2023-09-23T03:15:53.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.961+0000 7fe2a9183640 1 --2- 172.21.15.154:0/1957640776 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe28806af30 0x7fe28806d3f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0034040 tx=0x7fe2a0008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.961+0000 7fe7ad4a2640 1 -- 172.21.15.154:0/131857553 shutdown_connections 2023-09-23T03:15:53.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.961+0000 7fe7ad4a2640 1 --2- 172.21.15.154:0/131857553 >> v2:172.21.15.154:3300/0 conn(0x7fe7a80cdd80 0x7fe7a80cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.961+0000 7fe7ad4a2640 1 --2- 172.21.15.154:0/131857553 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7a815cac0 0x7fe7a806b280 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.961+0000 7fe7ad4a2640 1 -- 172.21.15.154:0/131857553 >> 172.21.15.154:0/131857553 conn(0x7fe7a8089fd0 msgr2=0x7fe7a809bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:53.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.961+0000 7f806e8b0640 1 -- 172.21.15.154:0/4238702324 shutdown_connections 2023-09-23T03:15:53.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.961+0000 7f806e8b0640 1 -- 172.21.15.154:0/4238702324 wait complete. 2023-09-23T03:15:53.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.960+0000 7fa01dfd0640 1 -- 172.21.15.154:0/1580487460 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0180c6cc0 con 0x7fa008005180 2023-09-23T03:15:53.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.961+0000 7fa001ffb640 1 -- 172.21.15.154:0/1580487460 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9ff0000fc0 con 0x7fa008005180 2023-09-23T03:15:53.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.962+0000 7fe7ad4a2640 1 -- 172.21.15.154:0/131857553 shutdown_connections 2023-09-23T03:15:53.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.959+0000 7fda477fe640 1 --2- 172.21.15.154:0/2573598503 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fda4c06b000 0x7fda4c06d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.959+0000 7fda477fe640 1 -- 172.21.15.154:0/2573598503 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fda600bc030 con 0x7fda580df9a0 2023-09-23T03:15:53.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.960+0000 7fda477fe640 1 --2- 172.21.15.154:0/2573598503 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x7fda4c0744d0 0x7fda4c076910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.960+0000 7fda477fe640 1 -- 172.21.15.154:0/2573598503 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fda4c076fe0 con 0x7fda4c0744d0 2023-09-23T03:15:53.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.960+0000 7fda477fe640 1 -- 172.21.15.154:0/2573598503 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda601259e0 con 0x7fda580df9a0 2023-09-23T03:15:53.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.963+0000 7fda5effd640 1 --2- 172.21.15.154:0/2573598503 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fda4c06b000 0x7fda4c06d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.963+0000 7f2ffffff640 1 -- 172.21.15.154:0/2422302356 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f301408f070 con 0x7f300c006700 2023-09-23T03:15:53.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.960+0000 7f268b7fe640 1 -- 172.21.15.154:0/533522302 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2690077150 con 0x7f2690074640 2023-09-23T03:15:53.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.964+0000 7f268b7fe640 1 -- 172.21.15.154:0/533522302 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26a4103880 con 0x7f269c00d700 2023-09-23T03:15:53.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.964+0000 7f26a27fc640 1 --2- 172.21.15.154:0/533522302 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x7f2690074640 0x7f2690076a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.961+0000 7f26a2ffd640 1 --2- 172.21.15.154:0/533522302 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f269006b000 0x7f269006d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.964+0000 7f2ffffff640 1 --2- 172.21.15.154:0/2422302356 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2ff806af80 0x7f2ff806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.964+0000 7f2ffffff640 1 -- 172.21.15.154:0/2422302356 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3014103130 con 0x7f300c006700 2023-09-23T03:15:53.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.962+0000 7fe7ad4a2640 1 -- 172.21.15.154:0/131857553 wait complete. 2023-09-23T03:15:53.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.963+0000 7f806e8b0640 1 Processor -- start 2023-09-23T03:15:53.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.964+0000 7f806e8b0640 1 -- start start 2023-09-23T03:15:53.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.965+0000 7f26a27fc640 1 --2- 172.21.15.154:0/533522302 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x7f2690074640 0x7f2690076a80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.965+0000 7f2ffffff640 1 --2- 172.21.15.154:0/2422302356 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x7f2ff80745c0 0x7f2ff8076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.965+0000 7f2ffffff640 1 -- 172.21.15.154:0/2422302356 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2ff80770d0 con 0x7f2ff80745c0 2023-09-23T03:15:53.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.965+0000 7f2ffffff640 1 -- 172.21.15.154:0/2422302356 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3014103580 con 0x7f300c006700 2023-09-23T03:15:53.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.964+0000 7fda5e7fc640 1 --2- 172.21.15.154:0/2573598503 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x7fda4c0744d0 0x7fda4c076910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.965+0000 7fda5effd640 1 --2- 172.21.15.154:0/2573598503 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fda4c06b000 0x7fda4c06d4c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fda580ed3c0 tx=0x7fda50007ef0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.965+0000 7fe7ad4a2640 1 Processor -- start 2023-09-23T03:15:53.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.965+0000 7fda5e7fc640 1 --2- 172.21.15.154:0/2573598503 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x7fda4c0744d0 0x7fda4c076910 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.965+0000 7f3013fff640 1 --2- 172.21.15.154:0/2422302356 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x7f2ff80745c0 0x7f2ff8076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.963+0000 7fa003fff640 1 -- 172.21.15.154:0/1580487460 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fa00c00cd30 con 0x7fa008005180 2023-09-23T03:15:53.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.966+0000 7f806e8b0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80680afba0 0x7f8068164770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.966+0000 7f806e8b0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80680ca840 0x7f806815eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.966+0000 7f806e8b0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80680ab140 0x7f80680ab5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.966+0000 7f806e8b0640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80680b1e40 con 0x7f80680ab140 2023-09-23T03:15:53.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.966+0000 7f806e8b0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806815f530 con 0x7f80680ca840 2023-09-23T03:15:53.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.966+0000 7f806e8b0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80680ac250 con 0x7f80680afba0 2023-09-23T03:15:53.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.966+0000 7f3012ffd640 1 --2- 172.21.15.154:0/2422302356 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2ff806af80 0x7f2ff806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.966+0000 7f806ce4d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80680ab140 0x7f80680ab5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.966+0000 7fa003fff640 1 --2- 172.21.15.154:0/1580487460 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9ffc06af60 0x7f9ffc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.966+0000 7f806ce4d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80680ab140 0x7f80680ab5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44618/0 (socket says 172.21.15.154:44618) 2023-09-23T03:15:53.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.966+0000 7f806ce4d640 1 -- 172.21.15.154:0/2464435612 learned_addr learned my addr 172.21.15.154:0/2464435612 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:53.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.966+0000 7fa003fff640 1 -- 172.21.15.154:0/1580487460 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa00c089ef0 con 0x7fa008005180 2023-09-23T03:15:53.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.967+0000 7f8067fff640 1 --2- 172.21.15.154:0/2464435612 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80680afba0 0x7f8068164770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.967+0000 7fa003fff640 1 --2- 172.21.15.154:0/1580487460 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x7f9ffc074520 0x7f9ffc076960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.967+0000 7fa003fff640 1 -- 172.21.15.154:0/1580487460 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9ffc077030 con 0x7f9ffc074520 2023-09-23T03:15:53.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.967+0000 7fa003fff640 1 -- 172.21.15.154:0/1580487460 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa00c08a340 con 0x7fa008005180 2023-09-23T03:15:53.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.967+0000 7f26a2ffd640 1 --2- 172.21.15.154:0/533522302 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f269006b000 0x7f269006d4c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2698035000 tx=0x7f2698008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.967+0000 7f3012ffd640 1 --2- 172.21.15.154:0/2422302356 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2ff806af80 0x7f2ff806d440 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3000036000 tx=0x7f3000008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.967+0000 7f80677fe640 1 --2- 172.21.15.154:0/2464435612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80680ca840 0x7f806815eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.967+0000 7f806ce4d640 1 -- 172.21.15.154:0/2464435612 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80680afba0 msgr2=0x7f8068164770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.967+0000 7f806ce4d640 1 --2- 172.21.15.154:0/2464435612 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f80680afba0 0x7f8068164770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.967+0000 7f806ce4d640 1 -- 172.21.15.154:0/2464435612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80680ca840 msgr2=0x7f806815eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.967+0000 7f806ce4d640 1 --2- 172.21.15.154:0/2464435612 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f80680ca840 0x7f806815eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.967+0000 7f806ce4d640 1 -- 172.21.15.154:0/2464435612 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8060009460 con 0x7f80680ab140 2023-09-23T03:15:53.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.967+0000 7fa016ffd640 1 --2- 172.21.15.154:0/1580487460 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9ffc06af60 0x7f9ffc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.968+0000 7fa017fff640 1 --2- 172.21.15.154:0/1580487460 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x7f9ffc074520 0x7f9ffc076960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.968+0000 7f806ce4d640 1 --2- 172.21.15.154:0/2464435612 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80680ab140 0x7f80680ab5f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f805800e830 tx=0x7f805800ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.968+0000 7fda477fe640 1 -- 172.21.15.154:0/2573598503 <== osd.0 v2:172.21.15.154:6800/2920938138 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fda4c076fe0 con 0x7fda4c0744d0 2023-09-23T03:15:53.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.968+0000 7f268b7fe640 1 -- 172.21.15.154:0/533522302 <== osd.1 v2:172.21.15.154:6816/1267710005 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f2690077150 con 0x7f2690074640 2023-09-23T03:15:53.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.966+0000 7fe7ad4a2640 1 -- start start 2023-09-23T03:15:53.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.968+0000 7f8064ff9640 1 -- 172.21.15.154:0/2464435612 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8058009610 con 0x7f80680ab140 2023-09-23T03:15:53.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.968+0000 7f3013fff640 1 --2- 172.21.15.154:0/2422302356 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x7f2ff80745c0 0x7f2ff8076a00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.968+0000 7f8064ff9640 1 -- 172.21.15.154:0/2464435612 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f805800c940 con 0x7f80680ab140 2023-09-23T03:15:53.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.968+0000 7f806e8b0640 1 -- 172.21.15.154:0/2464435612 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80680c41f0 con 0x7f80680ab140 2023-09-23T03:15:53.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.968+0000 7f806e8b0640 1 -- 172.21.15.154:0/2464435612 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80680c47b0 con 0x7f80680ab140 2023-09-23T03:15:53.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.969+0000 7fa016ffd640 1 --2- 172.21.15.154:0/1580487460 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9ffc06af60 0x7f9ffc06d420 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa0180cdff0 tx=0x7fa004009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.970+0000 7f8064ff9640 1 -- 172.21.15.154:0/2464435612 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805800ccf0 con 0x7f80680ab140 2023-09-23T03:15:53.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.970+0000 7f806e8b0640 1 -- 172.21.15.154:0/2464435612 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f80680cdf40 con 0x7f80680ab140 2023-09-23T03:15:53.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.970+0000 7fa017fff640 1 --2- 172.21.15.154:0/1580487460 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x7f9ffc074520 0x7f9ffc076960 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.971+0000 7f8064ff9640 1 -- 172.21.15.154:0/2464435612 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f8058013470 con 0x7f80680ab140 2023-09-23T03:15:53.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.971+0000 7f2ffffff640 1 -- 172.21.15.154:0/2422302356 <== osd.4 v2:172.21.15.190:6808/717302641 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f2ff80770d0 con 0x7f2ff80745c0 2023-09-23T03:15:53.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.972+0000 7fe7ad4a2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7a80cdd80 0x7fe7a80b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.972+0000 7fe7ad4a2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7a80cf2a0 0x7fe7a80b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.972+0000 7fe7ad4a2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7a815cac0 0x7fe7a80b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.972+0000 7fe7ad4a2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a80b7490 con 0x7fe7a80cdd80 2023-09-23T03:15:53.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.972+0000 7fe7ad4a2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a81546a0 con 0x7fe7a815cac0 2023-09-23T03:15:53.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.973+0000 7f8064ff9640 1 --2- 172.21.15.154:0/2464435612 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f804806af80 0x7f804806d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.972+0000 7fe7ad4a2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a80b6810 con 0x7fe7a80cf2a0 2023-09-23T03:15:53.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.973+0000 7f8064ff9640 1 -- 172.21.15.154:0/2464435612 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8058089b00 con 0x7f80680ab140 2023-09-23T03:15:53.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.973+0000 7f8067fff640 1 --2- 172.21.15.154:0/2464435612 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f804806af80 0x7f804806d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.973+0000 7f8064ff9640 1 --2- 172.21.15.154:0/2464435612 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x7f80480745c0 0x7f8048076a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.973+0000 7f8064ff9640 1 -- 172.21.15.154:0/2464435612 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f80480770d0 con 0x7f80480745c0 2023-09-23T03:15:53.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.973+0000 7f8064ff9640 1 -- 172.21.15.154:0/2464435612 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f805808e2a0 con 0x7f80680ab140 2023-09-23T03:15:53.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.974+0000 7fa003fff640 1 -- 172.21.15.154:0/1580487460 <== osd.5 v2:172.21.15.190:6800/882536885 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f9ffc077030 con 0x7f9ffc074520 2023-09-23T03:15:53.029 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.974+0000 7fe7a6ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7a80cdd80 0x7fe7a80b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.029 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.974+0000 7fe7a67fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7a80cf2a0 0x7fe7a80b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.029 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.974+0000 7fe7a67fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7a80cf2a0 0x7fe7a80b2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:43960/0 (socket says 172.21.15.154:43960) 2023-09-23T03:15:53.029 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.974+0000 7f80677fe640 1 --2- 172.21.15.154:0/2464435612 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x7f80480745c0 0x7f8048076a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.029 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.974+0000 7fe7a67fc640 1 -- 172.21.15.154:0/38581211 learned_addr learned my addr 172.21.15.154:0/38581211 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:53.029 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.974+0000 7fe7a77fe640 1 --2- 172.21.15.154:0/38581211 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7a815cac0 0x7fe7a80b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.030 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.975+0000 7f8067fff640 1 --2- 172.21.15.154:0/2464435612 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f804806af80 0x7f804806d440 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8060035000 tx=0x7f8060008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.975+0000 7fe7a77fe640 1 -- 172.21.15.154:0/38581211 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7a80cf2a0 msgr2=0x7fe7a80b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.975+0000 7fe7a77fe640 1 --2- 172.21.15.154:0/38581211 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7a80cf2a0 0x7fe7a80b2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.975+0000 7fe7a77fe640 1 -- 172.21.15.154:0/38581211 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7a80cdd80 msgr2=0x7fe7a80b78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.975+0000 7fe7a77fe640 1 --2- 172.21.15.154:0/38581211 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7a80cdd80 0x7fe7a80b78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.975+0000 7fe7a77fe640 1 -- 172.21.15.154:0/38581211 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe798009460 con 0x7fe7a815cac0 2023-09-23T03:15:53.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.975+0000 7fe7a6ffd640 1 --2- 172.21.15.154:0/38581211 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7a80cdd80 0x7fe7a80b78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:15:53.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.976+0000 7fe7a77fe640 1 --2- 172.21.15.154:0/38581211 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7a815cac0 0x7fe7a80b4b10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe79000b410 tx=0x7fe79000b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.976+0000 7f80677fe640 1 --2- 172.21.15.154:0/2464435612 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x7f80480745c0 0x7f8048076a00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.976+0000 7fe79f7fe640 1 -- 172.21.15.154:0/38581211 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe790011830 con 0x7fe7a815cac0 2023-09-23T03:15:53.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.976+0000 7fe7ad4a2640 1 -- 172.21.15.154:0/38581211 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7a80b6af0 con 0x7fe7a815cac0 2023-09-23T03:15:53.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.977+0000 7fe7ad4a2640 1 -- 172.21.15.154:0/38581211 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a80b70b0 con 0x7fe7a815cac0 2023-09-23T03:15:53.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.977+0000 7fe79f7fe640 1 -- 172.21.15.154:0/38581211 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe790011990 con 0x7fe7a815cac0 2023-09-23T03:15:53.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.977+0000 7fe79f7fe640 1 -- 172.21.15.154:0/38581211 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe790011830 con 0x7fe7a815cac0 2023-09-23T03:15:53.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.978+0000 7f8064ff9640 1 -- 172.21.15.154:0/2464435612 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f80480770d0 con 0x7f80480745c0 2023-09-23T03:15:53.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.979+0000 7fe7ad4a2640 1 -- 172.21.15.154:0/38581211 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe78c000fc0 con 0x7fe7a815cac0 2023-09-23T03:15:53.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.985+0000 7fe79f7fe640 1 -- 172.21.15.154:0/38581211 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fe790011b00 con 0x7fe7a815cac0 2023-09-23T03:15:53.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.986+0000 7fe79f7fe640 1 --2- 172.21.15.154:0/38581211 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe78806aff0 0x7fe78806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.987+0000 7fe79f7fe640 1 -- 172.21.15.154:0/38581211 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe790089d30 con 0x7fe7a815cac0 2023-09-23T03:15:53.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.987+0000 7fe79f7fe640 1 --2- 172.21.15.154:0/38581211 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x7fe7880745b0 0x7fe7880769f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.987+0000 7fe79f7fe640 1 -- 172.21.15.154:0/38581211 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe7880770c0 con 0x7fe7880745b0 2023-09-23T03:15:53.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.987+0000 7fe79f7fe640 1 -- 172.21.15.154:0/38581211 <== mon.1 v2:172.21.15.190:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe790011830 con 0x7fe7a815cac0 2023-09-23T03:15:53.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.988+0000 7fe7a6ffd640 1 --2- 172.21.15.154:0/38581211 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe78806aff0 0x7fe78806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.989+0000 7fe7a6ffd640 1 --2- 172.21.15.154:0/38581211 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe78806aff0 0x7fe78806d4b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe798035000 tx=0x7fe798008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.989+0000 7fe7a67fc640 1 --2- 172.21.15.154:0/38581211 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x7fe7880745b0 0x7fe7880769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.990+0000 7fad627fc640 1 -- 172.21.15.154:0/2709785318 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fad70053da0 con 0x7fad5c0745f0 2023-09-23T03:15:53.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.990+0000 7fad74ff9640 1 -- 172.21.15.154:0/2709785318 <== osd.2 v2:172.21.15.154:6822/2884407243 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fad6400a1d0 con 0x7fad5c0745f0 2023-09-23T03:15:53.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.991+0000 7fe7a67fc640 1 --2- 172.21.15.154:0/38581211 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x7fe7880745b0 0x7fe7880769f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.036 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.991+0000 7fad627fc640 1 -- 172.21.15.154:0/2709785318 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x7fad5c0745f0 msgr2=0x7fad5c076a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.036 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.991+0000 7fad627fc640 1 --2- 172.21.15.154:0/2709785318 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x7fad5c0745f0 0x7fad5c076a30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.036 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.992+0000 7fad627fc640 1 -- 172.21.15.154:0/2709785318 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fad5c06afb0 msgr2=0x7fad5c06d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.036 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.992+0000 7fad627fc640 1 --2- 172.21.15.154:0/2709785318 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fad5c06afb0 0x7fad5c06d470 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fad6c038000 tx=0x7fad6c008030 comp rx=0 tx=0).stop 2023-09-23T03:15:53.036 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.992+0000 7fad627fc640 1 -- 172.21.15.154:0/2709785318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad70032f70 msgr2=0x7fad700e1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.037 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.992+0000 7fad627fc640 1 --2- 172.21.15.154:0/2709785318 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad70032f70 0x7fad700e1580 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fad78067d60 tx=0x7fad7804e770 comp rx=0 tx=0).stop 2023-09-23T03:15:53.037 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.993+0000 7fe79f7fe640 1 -- 172.21.15.154:0/38581211 <== osd.7 v2:172.21.15.190:6824/3460136272 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fe7880770c0 con 0x7fe7880745b0 2023-09-23T03:15:53.037 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.993+0000 7fad7ce38640 1 -- 172.21.15.154:0/2709785318 reap_dead start 2023-09-23T03:15:53.037 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.993+0000 7fad627fc640 1 -- 172.21.15.154:0/2709785318 shutdown_connections 2023-09-23T03:15:53.037 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.993+0000 7fad627fc640 1 -- 172.21.15.154:0/2709785318 >> 172.21.15.154:0/2709785318 conn(0x7fad70054c10 msgr2=0x7fad70054ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:53.038 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.994+0000 7fe2ab3e7640 1 -- 172.21.15.154:0/1957640776 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe29c055300 con 0x7fe288074380 2023-09-23T03:15:53.038 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.994+0000 7fad627fc640 1 -- 172.21.15.154:0/2709785318 shutdown_connections 2023-09-23T03:15:53.038 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.994+0000 7fad627fc640 1 -- 172.21.15.154:0/2709785318 wait complete. 2023-09-23T03:15:53.039 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.995+0000 7fe299ffb640 1 -- 172.21.15.154:0/1957640776 <== osd.3 v2:172.21.15.154:6808/1133688468 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe2940076b0 con 0x7fe288074380 2023-09-23T03:15:53.039 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.996+0000 7fe2877fe640 1 -- 172.21.15.154:0/1957640776 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x7fe288074380 msgr2=0x7fe2880767c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.039 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.996+0000 7fe2877fe640 1 --2- 172.21.15.154:0/1957640776 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x7fe288074380 0x7fe2880767c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.039 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.996+0000 7fe2877fe640 1 -- 172.21.15.154:0/1957640776 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe28806af30 msgr2=0x7fe28806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.039 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.996+0000 7fe2877fe640 1 --2- 172.21.15.154:0/1957640776 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe28806af30 0x7fe28806d3f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0034040 tx=0x7fe2a0008030 comp rx=0 tx=0).stop 2023-09-23T03:15:53.039 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.996+0000 7fe2877fe640 1 -- 172.21.15.154:0/1957640776 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe29c0e1410 msgr2=0x7fe29c0383e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.039 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.996+0000 7fe2877fe640 1 --2- 172.21.15.154:0/1957640776 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe29c0e1410 0x7fe29c0383e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe2a404f340 tx=0x7fe2a40734a0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.996+0000 7fe2aa185640 1 -- 172.21.15.154:0/1957640776 reap_dead start 2023-09-23T03:15:53.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.997+0000 7fe2877fe640 1 -- 172.21.15.154:0/1957640776 shutdown_connections 2023-09-23T03:15:53.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:52.997+0000 7fe2877fe640 1 -- 172.21.15.154:0/1957640776 >> 172.21.15.154:0/1957640776 conn(0x7fe29c00ba30 msgr2=0x7fe29c0e0550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:53.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.000+0000 7fe2877fe640 1 -- 172.21.15.154:0/1957640776 shutdown_connections 2023-09-23T03:15:53.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.000+0000 7fe2877fe640 1 -- 172.21.15.154:0/1957640776 wait complete. 2023-09-23T03:15:53.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.001+0000 7f26a3fff640 1 -- 172.21.15.154:0/533522302 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f269c007c40 con 0x7f2690074640 2023-09-23T03:15:53.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.002+0000 7f268b7fe640 1 -- 172.21.15.154:0/533522302 <== osd.1 v2:172.21.15.154:6816/1267710005 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2694007340 con 0x7f2690074640 2023-09-23T03:15:53.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.002+0000 7fa001ffb640 1 -- 172.21.15.154:0/1580487460 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9ff0002cb0 con 0x7f9ffc074520 2023-09-23T03:15:53.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.002+0000 7fda653c5640 1 -- 172.21.15.154:0/2573598503 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fda58053dd0 con 0x7fda4c0744d0 2023-09-23T03:15:53.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.003+0000 7f26a3fff640 1 -- 172.21.15.154:0/533522302 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x7f2690074640 msgr2=0x7f2690076a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.042 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.003+0000 7f26a3fff640 1 --2- 172.21.15.154:0/533522302 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x7f2690074640 0x7f2690076a80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.042 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.003+0000 7fda477fe640 1 -- 172.21.15.154:0/2573598503 <== osd.0 v2:172.21.15.154:6800/2920938138 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fda54009520 con 0x7fda4c0744d0 2023-09-23T03:15:53.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.003+0000 7f26a3fff640 1 -- 172.21.15.154:0/533522302 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f269006b000 msgr2=0x7f269006d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.003+0000 7f26a3fff640 1 --2- 172.21.15.154:0/533522302 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f269006b000 0x7f269006d4c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2698035000 tx=0x7f2698008030 comp rx=0 tx=0).stop 2023-09-23T03:15:53.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.003+0000 7f26a3fff640 1 -- 172.21.15.154:0/533522302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f269c00d700 msgr2=0x7f269c16a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.003+0000 7f26a3fff640 1 --2- 172.21.15.154:0/533522302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f269c00d700 0x7f269c16a880 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f26a40657c0 tx=0x7f26a404e770 comp rx=0 tx=0).stop 2023-09-23T03:15:53.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.004+0000 7f26a37fe640 1 -- 172.21.15.154:0/533522302 reap_dead start 2023-09-23T03:15:53.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.004+0000 7f26a3fff640 1 -- 172.21.15.154:0/533522302 shutdown_connections 2023-09-23T03:15:53.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.004+0000 7f26a3fff640 1 -- 172.21.15.154:0/533522302 >> 172.21.15.154:0/533522302 conn(0x7f269c0590c0 msgr2=0x7f269c006400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:53.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.004+0000 7fa003fff640 1 -- 172.21.15.154:0/1580487460 <== osd.5 v2:172.21.15.190:6800/882536885 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9ff0002cb0 con 0x7f9ffc074520 2023-09-23T03:15:53.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.004+0000 7fda653c5640 1 -- 172.21.15.154:0/2573598503 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x7fda4c0744d0 msgr2=0x7fda4c076910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.004+0000 7fda653c5640 1 --2- 172.21.15.154:0/2573598503 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x7fda4c0744d0 0x7fda4c076910 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.004+0000 7f26a3fff640 1 -- 172.21.15.154:0/533522302 shutdown_connections 2023-09-23T03:15:53.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.005+0000 7f26a3fff640 1 -- 172.21.15.154:0/533522302 wait complete. 2023-09-23T03:15:53.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.005+0000 7fda653c5640 1 -- 172.21.15.154:0/2573598503 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fda4c06b000 msgr2=0x7fda4c06d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.005+0000 7f301a094640 1 -- 172.21.15.154:0/2422302356 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f300c053af0 con 0x7f2ff80745c0 2023-09-23T03:15:53.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.005+0000 7fda653c5640 1 --2- 172.21.15.154:0/2573598503 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fda4c06b000 0x7fda4c06d4c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fda580ed3c0 tx=0x7fda50007ef0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.005+0000 7fda653c5640 1 -- 172.21.15.154:0/2573598503 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda580df9a0 msgr2=0x7fda58006d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.005+0000 7fda653c5640 1 --2- 172.21.15.154:0/2573598503 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda580df9a0 0x7fda58006d70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fda6002cb00 tx=0x7fda60069b10 comp rx=0 tx=0).stop 2023-09-23T03:15:53.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.005+0000 7fda5f7fe640 1 -- 172.21.15.154:0/2573598503 reap_dead start 2023-09-23T03:15:53.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.006+0000 7fda653c5640 1 -- 172.21.15.154:0/2573598503 shutdown_connections 2023-09-23T03:15:53.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.006+0000 7fda653c5640 1 -- 172.21.15.154:0/2573598503 >> 172.21.15.154:0/2573598503 conn(0x7fda58054c40 msgr2=0x7fda580346a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:53.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.006+0000 7fa01dfd0640 1 -- 172.21.15.154:0/1580487460 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x7f9ffc074520 msgr2=0x7f9ffc076960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.006+0000 7fa01dfd0640 1 --2- 172.21.15.154:0/1580487460 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x7f9ffc074520 0x7f9ffc076960 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.006+0000 7fa01dfd0640 1 -- 172.21.15.154:0/1580487460 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9ffc06af60 msgr2=0x7f9ffc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.006+0000 7fa01dfd0640 1 --2- 172.21.15.154:0/1580487460 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9ffc06af60 0x7f9ffc06d420 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa0180cdff0 tx=0x7fa004009040 comp rx=0 tx=0).stop 2023-09-23T03:15:53.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.006+0000 7fa01dfd0640 1 -- 172.21.15.154:0/1580487460 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa008005180 msgr2=0x7fa01815a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.006+0000 7fa01dfd0640 1 --2- 172.21.15.154:0/1580487460 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa008005180 0x7fa01815a8a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa00c00e700 tx=0x7fa00c00ebc0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.006+0000 7f2ffffff640 1 -- 172.21.15.154:0/2422302356 <== osd.4 v2:172.21.15.190:6808/717302641 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3008001690 con 0x7f2ff80745c0 2023-09-23T03:15:53.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.006+0000 7fa017fff640 1 -- 172.21.15.154:0/1580487460 reap_dead start 2023-09-23T03:15:53.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.007+0000 7fda653c5640 1 -- 172.21.15.154:0/2573598503 shutdown_connections 2023-09-23T03:15:53.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.007+0000 7fda653c5640 1 -- 172.21.15.154:0/2573598503 wait complete. 2023-09-23T03:15:53.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.007+0000 7fa01dfd0640 1 -- 172.21.15.154:0/1580487460 shutdown_connections 2023-09-23T03:15:53.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.007+0000 7fa01dfd0640 1 -- 172.21.15.154:0/1580487460 >> 172.21.15.154:0/1580487460 conn(0x7fa01809b9b0 msgr2=0x7fa01815bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:53.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.007+0000 7f301a094640 1 -- 172.21.15.154:0/2422302356 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x7f2ff80745c0 msgr2=0x7f2ff8076a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.007+0000 7f301a094640 1 --2- 172.21.15.154:0/2422302356 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x7f2ff80745c0 0x7f2ff8076a00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.008+0000 7fa01dfd0640 1 -- 172.21.15.154:0/1580487460 shutdown_connections 2023-09-23T03:15:53.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.008+0000 7fa01dfd0640 1 -- 172.21.15.154:0/1580487460 wait complete. 2023-09-23T03:15:53.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.008+0000 7f301a094640 1 -- 172.21.15.154:0/2422302356 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2ff806af80 msgr2=0x7f2ff806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.008+0000 7f301a094640 1 --2- 172.21.15.154:0/2422302356 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2ff806af80 0x7f2ff806d440 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3000036000 tx=0x7f3000008030 comp rx=0 tx=0).stop 2023-09-23T03:15:53.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.008+0000 7f301a094640 1 -- 172.21.15.154:0/2422302356 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f300c006700 msgr2=0x7f300c0dd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.008+0000 7f301a094640 1 --2- 172.21.15.154:0/2422302356 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f300c006700 0x7f300c0dd6b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f30140685f0 tx=0x7f3014073870 comp rx=0 tx=0).stop 2023-09-23T03:15:53.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.008+0000 7f3013fff640 1 -- 172.21.15.154:0/2422302356 reap_dead start 2023-09-23T03:15:53.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.008+0000 7f806e8b0640 1 -- 172.21.15.154:0/2464435612 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f80680cdc60 con 0x7f80480745c0 2023-09-23T03:15:53.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.009+0000 7f301a094640 1 -- 172.21.15.154:0/2422302356 shutdown_connections 2023-09-23T03:15:53.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.009+0000 7f301a094640 1 -- 172.21.15.154:0/2422302356 >> 172.21.15.154:0/2422302356 conn(0x7f300c054c40 msgr2=0x7f300c035c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:53.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.009+0000 7f301a094640 1 -- 172.21.15.154:0/2422302356 shutdown_connections 2023-09-23T03:15:53.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.009+0000 7f301a094640 1 -- 172.21.15.154:0/2422302356 wait complete. 2023-09-23T03:15:53.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.016+0000 7f8064ff9640 1 -- 172.21.15.154:0/2464435612 <== osd.6 v2:172.21.15.190:6816/1964294001 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8050008b20 con 0x7f80480745c0 2023-09-23T03:15:53.053 INFO:teuthology.orchestra.run.smithi154.stdout:55834574860 2023-09-23T03:15:53.053 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-23T03:15:53.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.019+0000 7f806e8b0640 1 -- 172.21.15.154:0/2464435612 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x7f80480745c0 msgr2=0x7f8048076a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.019+0000 7f806e8b0640 1 --2- 172.21.15.154:0/2464435612 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x7f80480745c0 0x7f8048076a00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.019+0000 7f806e8b0640 1 -- 172.21.15.154:0/2464435612 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f804806af80 msgr2=0x7f804806d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.019+0000 7f806e8b0640 1 --2- 172.21.15.154:0/2464435612 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f804806af80 0x7f804806d440 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8060035000 tx=0x7f8060008030 comp rx=0 tx=0).stop 2023-09-23T03:15:53.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.019+0000 7f806e8b0640 1 -- 172.21.15.154:0/2464435612 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80680ab140 msgr2=0x7f80680ab5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.020+0000 7f806e8b0640 1 --2- 172.21.15.154:0/2464435612 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f80680ab140 0x7f80680ab5f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f805800e830 tx=0x7f805800ecf0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.020+0000 7f806ce4d640 1 -- 172.21.15.154:0/2464435612 reap_dead start 2023-09-23T03:15:53.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.020+0000 7f806e8b0640 1 -- 172.21.15.154:0/2464435612 shutdown_connections 2023-09-23T03:15:53.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.020+0000 7f806e8b0640 1 -- 172.21.15.154:0/2464435612 >> 172.21.15.154:0/2464435612 conn(0x7f80680b0000 msgr2=0x7f8068160500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:53.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.020+0000 7f806e8b0640 1 -- 172.21.15.154:0/2464435612 shutdown_connections 2023-09-23T03:15:53.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.021+0000 7f806e8b0640 1 -- 172.21.15.154:0/2464435612 wait complete. 2023-09-23T03:15:53.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.021+0000 7fe7ad4a2640 1 -- 172.21.15.154:0/38581211 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe78c002ce0 con 0x7fe7880745b0 2023-09-23T03:15:53.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.022+0000 7fe79f7fe640 1 -- 172.21.15.154:0/38581211 <== osd.7 v2:172.21.15.190:6824/3460136272 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe78c002ce0 con 0x7fe7880745b0 2023-09-23T03:15:53.056 INFO:teuthology.orchestra.run.smithi154.stdout:55834574859 2023-09-23T03:15:53.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.023+0000 7fe79d7fa640 1 -- 172.21.15.154:0/38581211 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x7fe7880745b0 msgr2=0x7fe7880769f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.023+0000 7fe79d7fa640 1 --2- 172.21.15.154:0/38581211 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x7fe7880745b0 0x7fe7880769f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.023+0000 7fe79d7fa640 1 -- 172.21.15.154:0/38581211 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe78806aff0 msgr2=0x7fe78806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.023+0000 7fe79d7fa640 1 --2- 172.21.15.154:0/38581211 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe78806aff0 0x7fe78806d4b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe798035000 tx=0x7fe798008030 comp rx=0 tx=0).stop 2023-09-23T03:15:53.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.023+0000 7fe79d7fa640 1 -- 172.21.15.154:0/38581211 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7a815cac0 msgr2=0x7fe7a80b4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.023+0000 7fe79d7fa640 1 --2- 172.21.15.154:0/38581211 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe7a815cac0 0x7fe7a80b4b10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe79000b410 tx=0x7fe79000b8d0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.024+0000 7fe7a77fe640 1 -- 172.21.15.154:0/38581211 reap_dead start 2023-09-23T03:15:53.058 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-23T03:15:53.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.024+0000 7fe79d7fa640 1 -- 172.21.15.154:0/38581211 shutdown_connections 2023-09-23T03:15:53.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.024+0000 7fe79d7fa640 1 -- 172.21.15.154:0/38581211 >> 172.21.15.154:0/38581211 conn(0x7fe7a8089fd0 msgr2=0x7fe7a8160bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:53.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.024+0000 7fe79d7fa640 1 -- 172.21.15.154:0/38581211 shutdown_connections 2023-09-23T03:15:53.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.025+0000 7fe79d7fa640 1 -- 172.21.15.154:0/38581211 wait complete. 2023-09-23T03:15:53.059 INFO:teuthology.orchestra.run.smithi154.stdout:55834574860 2023-09-23T03:15:53.060 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-09-23T03:15:53.060 INFO:teuthology.orchestra.run.smithi154.stdout:55834574861 2023-09-23T03:15:53.061 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-23T03:15:53.062 INFO:teuthology.orchestra.run.smithi154.stdout:55834574859 2023-09-23T03:15:53.062 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-09-23T03:15:53.063 INFO:teuthology.orchestra.run.smithi154.stdout:55834574861 2023-09-23T03:15:53.063 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-23T03:15:53.063 INFO:teuthology.orchestra.run.smithi154.stdout:55834574858 2023-09-23T03:15:53.064 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-09-23T03:15:53.065 INFO:teuthology.orchestra.run.smithi154.stdout:55834574858 2023-09-23T03:15:53.065 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-09-23T03:15:53.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.397+0000 7f189d6c3640 1 Processor -- start 2023-09-23T03:15:53.400 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.398+0000 7f189d6c3640 1 -- start start 2023-09-23T03:15:53.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.400+0000 7f189d6c3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f189003a860 0x7f1890053ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.400+0000 7f189d6c3640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1890038170 0x7f1890038600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.400+0000 7f189d6c3640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1890037d40 con 0x7f1890052fc0 2023-09-23T03:15:53.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.400+0000 7f189d6c3640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18900079c0 con 0x7f1890038170 2023-09-23T03:15:53.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.400+0000 7f189d6c3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1890054340 con 0x7f189003a860 2023-09-23T03:15:53.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.400+0000 7f1897fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f1890052fc0 0x7f18900533a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:42322/0 (socket says 172.21.15.154:42322) 2023-09-23T03:15:53.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.400+0000 7f1897fff640 1 -- 172.21.15.154:0/3527493886 learned_addr learned my addr 172.21.15.154:0/3527493886 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:53.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.401+0000 7f189cec2640 1 --2- 172.21.15.154:0/3527493886 >> v2:172.21.15.190:3300/0 conn(0x7f1890038170 0x7f1890038600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.401+0000 7f1896ffd640 1 -- 172.21.15.154:0/3527493886 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 763264690 0 0) 0x7f1890037d40 con 0x7f1890052fc0 2023-09-23T03:15:53.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.401+0000 7f18977fe640 1 --2- 172.21.15.154:0/3527493886 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f189003a860 0x7f1890053ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.401+0000 7f1896ffd640 1 -- 172.21.15.154:0/3527493886 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1888005730 con 0x7f1890052fc0 2023-09-23T03:15:53.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.402+0000 7f1896ffd640 1 -- 172.21.15.154:0/3527493886 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3701690469 0 0) 0x7f1888005730 con 0x7f1890052fc0 2023-09-23T03:15:53.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.402+0000 7f189cec2640 1 -- 172.21.15.154:0/3527493886 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f189003a860 msgr2=0x7f1890053ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:53.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.402+0000 7f189cec2640 1 --2- 172.21.15.154:0/3527493886 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f189003a860 0x7f1890053ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.402+0000 7f189cec2640 1 -- 172.21.15.154:0/3527493886 >> v1:172.21.15.154:6789/0 conn(0x7f1890052fc0 legacy=0x7f18900533a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.402+0000 7f189cec2640 1 -- 172.21.15.154:0/3527493886 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1890007090 con 0x7f1890038170 2023-09-23T03:15:53.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.404+0000 7f189cec2640 1 --2- 172.21.15.154:0/3527493886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1890038170 0x7f1890038600 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f189804f9d0 tx=0x7f1898094660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53d8bbbda4b999e1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.404+0000 7f1896ffd640 1 -- 172.21.15.154:0/3527493886 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1884004e60 con 0x7f1890038170 2023-09-23T03:15:53.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.404+0000 7f1896ffd640 1 -- 172.21.15.154:0/3527493886 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1898097050 con 0x7f1890038170 2023-09-23T03:15:53.406 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.404+0000 7f1896ffd640 1 -- 172.21.15.154:0/3527493886 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189809b400 con 0x7f1890038170 2023-09-23T03:15:53.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.405+0000 7f189d6c3640 1 -- 172.21.15.154:0/3527493886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1890038170 msgr2=0x7f1890038600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.405+0000 7f189d6c3640 1 --2- 172.21.15.154:0/3527493886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1890038170 0x7f1890038600 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f189804f9d0 tx=0x7f1898094660 comp rx=0 tx=0).stop 2023-09-23T03:15:53.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.406+0000 7f189d6c3640 1 -- 172.21.15.154:0/3527493886 shutdown_connections 2023-09-23T03:15:53.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.406+0000 7f189d6c3640 1 --2- 172.21.15.154:0/3527493886 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f189003a860 0x7f1890053ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.406+0000 7f189d6c3640 1 --2- 172.21.15.154:0/3527493886 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1890038170 0x7f1890038600 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.406+0000 7f189d6c3640 1 -- 172.21.15.154:0/3527493886 >> 172.21.15.154:0/3527493886 conn(0x7f1890054c10 msgr2=0x7f1890055020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:53.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.406+0000 7f189d6c3640 1 -- 172.21.15.154:0/3527493886 shutdown_connections 2023-09-23T03:15:53.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.406+0000 7f97b0960640 1 Processor -- start 2023-09-23T03:15:53.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.406+0000 7f97b0960640 1 -- start start 2023-09-23T03:15:53.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.406+0000 7f189d6c3640 1 -- 172.21.15.154:0/3527493886 wait complete. 2023-09-23T03:15:53.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.409+0000 7f189d6c3640 1 Processor -- start 2023-09-23T03:15:53.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.409+0000 7f189d6c3640 1 -- start start 2023-09-23T03:15:53.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.408+0000 7f97b0960640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97ac15c840 0x7f97ac0cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.408+0000 7f97b0960640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f97ac160fd0 0x7f97ac0c9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.408+0000 7f97b0960640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97ac0cc910 con 0x7f97ac0c99d0 2023-09-23T03:15:53.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.408+0000 7f97b0960640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97ac15cc20 con 0x7f97ac160fd0 2023-09-23T03:15:53.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.408+0000 7f97b0960640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac0c89b0 con 0x7f97ac15c840 2023-09-23T03:15:53.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.409+0000 7f97aad9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97ac15c840 0x7f97ac0cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.409+0000 7f97aad9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97ac15c840 0x7f97ac0cbfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:43986/0 (socket says 172.21.15.154:43986) 2023-09-23T03:15:53.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.409+0000 7f97aad9d640 1 -- 172.21.15.154:0/4140210421 learned_addr learned my addr 172.21.15.154:0/4140210421 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:53.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.410+0000 7ff6f8b25640 1 Processor -- start 2023-09-23T03:15:53.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.410+0000 7f97aad9d640 1 -- 172.21.15.154:0/4140210421 >> v2:172.21.15.154:3300/0 conn(0x7f97ac160fd0 msgr2=0x7f97ac0c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:53.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.410+0000 7f97aad9d640 1 --2- 172.21.15.154:0/4140210421 >> v2:172.21.15.154:3300/0 conn(0x7f97ac160fd0 0x7f97ac0c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.410+0000 7f97aad9d640 1 -- 172.21.15.154:0/4140210421 >> v1:172.21.15.190:6789/0 conn(0x7f97ac0c99d0 legacy=0x7f97ac0cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.410+0000 7f97aad9d640 1 -- 172.21.15.154:0/4140210421 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ac0c8c90 con 0x7f97ac15c840 2023-09-23T03:15:53.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.411+0000 7f189d6c3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1890038170 0x7f189019b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.411+0000 7f189d6c3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f189003a860 0x7f18901990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.411+0000 7f189d6c3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1890052fc0 0x7f18901998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.411+0000 7f189d6c3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1890199630 con 0x7f189003a860 2023-09-23T03:15:53.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.411+0000 7f189d6c3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189019bb90 con 0x7f1890038170 2023-09-23T03:15:53.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.411+0000 7f189d6c3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f189019be70 con 0x7f1890052fc0 2023-09-23T03:15:53.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.411+0000 7f1897fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1890038170 0x7f189019b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.411+0000 7f1897fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1890038170 0x7f189019b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.154:47922/0 (socket says 172.21.15.154:47922) 2023-09-23T03:15:53.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.411+0000 7f1897fff640 1 -- 172.21.15.154:0/4161737363 learned_addr learned my addr 172.21.15.154:0/4161737363 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:53.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.412+0000 7f18977fe640 1 --2- 172.21.15.154:0/4161737363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f189003a860 0x7f18901990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.412+0000 7f18977fe640 1 -- 172.21.15.154:0/4161737363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1890052fc0 msgr2=0x7f18901998a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-23T03:15:53.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.412+0000 7f18977fe640 1 --2- 172.21.15.154:0/4161737363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1890052fc0 0x7f18901998a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.412+0000 7f18977fe640 1 -- 172.21.15.154:0/4161737363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1890038170 msgr2=0x7f189019b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.412+0000 7f18977fe640 1 --2- 172.21.15.154:0/4161737363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1890038170 0x7f189019b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.412+0000 7f18977fe640 1 -- 172.21.15.154:0/4161737363 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f189806ce70 con 0x7f189003a860 2023-09-23T03:15:53.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.410+0000 7ff6f8b25640 1 -- start start 2023-09-23T03:15:53.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.413+0000 7ff6f8b25640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff6ec049f10 0x7ff6ec051960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.413+0000 7ff6f8b25640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6ec0532f0 con 0x7ff6ec055430 2023-09-23T03:15:53.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.413+0000 7ff6f8b25640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6ec0e8e40 con 0x7ff6ec0543f0 2023-09-23T03:15:53.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.413+0000 7ff6f8b25640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6ec039880 con 0x7ff6ec049f10 2023-09-23T03:15:53.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.413+0000 7f18977fe640 1 --2- 172.21.15.154:0/4161737363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f189003a860 0x7f18901990f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f188c00d840 tx=0x7f188c00dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.413+0000 7f1894ff9640 1 -- 172.21.15.154:0/4161737363 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c00b6c0 con 0x7f189003a860 2023-09-23T03:15:53.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.414+0000 7f1894ff9640 1 -- 172.21.15.154:0/4161737363 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f188c00b820 con 0x7f189003a860 2023-09-23T03:15:53.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.413+0000 7f97aad9d640 1 --2- 172.21.15.154:0/4140210421 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97ac15c840 0x7f97ac0cbfe0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f979c009870 tx=0x7f979c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f7a1cf2b61e3030 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.414+0000 7ff6f1d9b640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7ff6ec0543f0 0x7ff6ec039170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.154:42450/0 (socket says 172.21.15.154:42450) 2023-09-23T03:15:53.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.414+0000 7ff6f1d9b640 1 -- 172.21.15.154:0/3613602406 learned_addr learned my addr 172.21.15.154:0/3613602406 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:53.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.414+0000 7f189d6c3640 1 -- 172.21.15.154:0/4161737363 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f189019c150 con 0x7f189003a860 2023-09-23T03:15:53.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.414+0000 7f1dfca90640 1 Processor -- start 2023-09-23T03:15:53.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.414+0000 7f189d6c3640 1 -- 172.21.15.154:0/4161737363 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1890199de0 con 0x7f189003a860 2023-09-23T03:15:53.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.415+0000 7ff6f159a640 1 -- 172.21.15.154:0/3613602406 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2399224782 0 0) 0x7ff6ec0532f0 con 0x7ff6ec055430 2023-09-23T03:15:53.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.415+0000 7ff6f159a640 1 -- 172.21.15.154:0/3613602406 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6dc003520 con 0x7ff6ec055430 2023-09-23T03:15:53.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.416+0000 7ff6f159a640 1 -- 172.21.15.154:0/3613602406 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2548616325 0 0) 0x7ff6dc003520 con 0x7ff6ec055430 2023-09-23T03:15:53.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.416+0000 7ff6f159a640 1 -- 172.21.15.154:0/3613602406 >> v2:172.21.15.190:3300/0 conn(0x7ff6ec049f10 msgr2=0x7ff6ec051960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:53.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.416+0000 7ff6f2d9d640 1 --2- 172.21.15.154:0/3613602406 >> v2:172.21.15.190:3300/0 conn(0x7ff6ec049f10 0x7ff6ec051960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.416+0000 7f97a9d9b640 1 -- 172.21.15.154:0/4140210421 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979c030840 con 0x7f97ac15c840 2023-09-23T03:15:53.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.416+0000 7f97a9d9b640 1 -- 172.21.15.154:0/4140210421 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979c0309a0 con 0x7f97ac15c840 2023-09-23T03:15:53.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.416+0000 7ff6f159a640 1 --2- 172.21.15.154:0/3613602406 >> v2:172.21.15.190:3300/0 conn(0x7ff6ec049f10 0x7ff6ec051960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.416+0000 7ff6f159a640 1 -- 172.21.15.154:0/3613602406 >> v1:172.21.15.190:6789/0 conn(0x7ff6ec0543f0 legacy=0x7ff6ec039170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:53.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.416+0000 7ff6f159a640 1 -- 172.21.15.154:0/3613602406 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ec0e8b60 con 0x7ff6ec055430 2023-09-23T03:15:53.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.416+0000 7ff6f159a640 1 -- 172.21.15.154:0/3613602406 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7ff6e4001670 con 0x7ff6ec055430 2023-09-23T03:15:53.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.416+0000 7ff6f159a640 1 -- 172.21.15.154:0/3613602406 >> v1:172.21.15.154:6789/0 conn(0x7ff6ec055430 legacy=0x7ff6ec055810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.416+0000 7ff6f159a640 1 --2- 172.21.15.154:0/3613602406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6dc003970 0x7ff6dc003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.416+0000 7ff6f159a640 1 --2- 172.21.15.154:0/3613602406 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6dc0047c0 0x7ff6dc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.416+0000 7ff6f159a640 1 --2- 172.21.15.154:0/3613602406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6dc0052a0 0x7ff6dc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.416+0000 7ff6f159a640 1 -- 172.21.15.154:0/3613602406 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec039880 con 0x7ff6dc0047c0 2023-09-23T03:15:53.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.416+0000 7ff6f159a640 1 -- 172.21.15.154:0/3613602406 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc00c190 con 0x7ff6dc0052a0 2023-09-23T03:15:53.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.416+0000 7ff6f159a640 1 -- 172.21.15.154:0/3613602406 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6dc00c350 con 0x7ff6dc003970 2023-09-23T03:15:53.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.416+0000 7ff6f2d9d640 1 --2- 172.21.15.154:0/3613602406 >> v2:172.21.15.190:3300/0 conn(0x7ff6ec049f10 0x7ff6ec051960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:53.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.414+0000 7f1dfca90640 1 -- start start 2023-09-23T03:15:53.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.417+0000 7f97b0960640 1 -- 172.21.15.154:0/4140210421 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97ac15c840 msgr2=0x7f97ac0cbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.417+0000 7f97b0960640 1 --2- 172.21.15.154:0/4140210421 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97ac15c840 0x7f97ac0cbfe0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f979c009870 tx=0x7f979c02b980 comp rx=0 tx=0).stop 2023-09-23T03:15:53.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.417+0000 7f97a9d9b640 1 -- 172.21.15.154:0/4140210421 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979c030d30 con 0x7f97ac15c840 2023-09-23T03:15:53.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.417+0000 7f1dfca90640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1df80c14e0 0x7f1df80c1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.417+0000 7ff6f259c640 1 --2- 172.21.15.154:0/3613602406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6dc003970 0x7ff6dc003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.415+0000 7f1894ff9640 1 -- 172.21.15.154:0/4161737363 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c00bbb0 con 0x7f189003a860 2023-09-23T03:15:53.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.417+0000 7ff6f2d9d640 1 --2- 172.21.15.154:0/3613602406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6dc0052a0 0x7ff6dc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.417+0000 7f1dfca90640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1df806c1d0 0x7f1df806c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.417+0000 7f1dfca90640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1df80cb160 con 0x7f1df8161010 2023-09-23T03:15:53.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.417+0000 7f1dfca90640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1df80c1ea0 con 0x7f1df80c14e0 2023-09-23T03:15:53.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.418+0000 7f1894ff9640 1 -- 172.21.15.154:0/4161737363 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f188c00bd10 con 0x7f189003a860 2023-09-23T03:15:53.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.419+0000 7f97b0960640 1 -- 172.21.15.154:0/4140210421 shutdown_connections 2023-09-23T03:15:53.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.419+0000 7f97b0960640 1 --2- 172.21.15.154:0/4140210421 >> v2:172.21.15.154:3300/0 conn(0x7f97ac160fd0 0x7f97ac0c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.419+0000 7f97b0960640 1 --2- 172.21.15.154:0/4140210421 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97ac15c840 0x7f97ac0cbfe0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.419+0000 7f97b0960640 1 -- 172.21.15.154:0/4140210421 >> 172.21.15.154:0/4140210421 conn(0x7f97ac06a390 msgr2=0x7f97ac0afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:53.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.417+0000 7f1dfca90640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df806cb60 con 0x7f1df806c1d0 2023-09-23T03:15:53.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.418+0000 7f1df659c640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f1df8161010 0x7f1df80cc7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.154:42464/0 (socket says 172.21.15.154:42464) 2023-09-23T03:15:53.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.418+0000 7f1df659c640 1 -- 172.21.15.154:0/642093975 learned_addr learned my addr 172.21.15.154:0/642093975 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:53.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.419+0000 7f1df6d9d640 1 --2- 172.21.15.154:0/642093975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1df806c1d0 0x7f1df806c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.419+0000 7f1df559a640 1 -- 172.21.15.154:0/642093975 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1393353816 0 0) 0x7f1df80cb160 con 0x7f1df8161010 2023-09-23T03:15:53.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.419+0000 7f1df559a640 1 -- 172.21.15.154:0/642093975 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1de0003520 con 0x7f1df8161010 2023-09-23T03:15:53.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.419+0000 7f97b0960640 1 -- 172.21.15.154:0/4140210421 shutdown_connections 2023-09-23T03:15:53.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.419+0000 7f97b0960640 1 -- 172.21.15.154:0/4140210421 wait complete. 2023-09-23T03:15:53.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.418+0000 7ff6f259c640 1 -- 172.21.15.154:0/3613602406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6dc0052a0 msgr2=0x7ff6dc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:53.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.418+0000 7ff6f259c640 1 --2- 172.21.15.154:0/3613602406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6dc0052a0 0x7ff6dc00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.418+0000 7ff6f259c640 1 -- 172.21.15.154:0/3613602406 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6dc0047c0 msgr2=0x7ff6dc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:53.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.418+0000 7ff6f259c640 1 --2- 172.21.15.154:0/3613602406 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6dc0047c0 0x7ff6dc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.418+0000 7ff6f259c640 1 -- 172.21.15.154:0/3613602406 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ec0e8b60 con 0x7ff6dc003970 2023-09-23T03:15:53.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.419+0000 7f1df559a640 1 -- 172.21.15.154:0/642093975 <== mon.1 v1:172.21.15.190:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 739686789 0 0) 0x7f1de0003520 con 0x7f1df8161010 2023-09-23T03:15:53.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.419+0000 7f1df559a640 1 -- 172.21.15.154:0/642093975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1df806c1d0 msgr2=0x7f1df806c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:53.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.419+0000 7f1df559a640 1 --2- 172.21.15.154:0/642093975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1df806c1d0 0x7f1df806c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.420+0000 7f1df559a640 1 -- 172.21.15.154:0/642093975 >> v2:172.21.15.190:3300/0 conn(0x7f1df80c14e0 msgr2=0x7f1df80c1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:53.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.420+0000 7f1df559a640 1 --2- 172.21.15.154:0/642093975 >> v2:172.21.15.190:3300/0 conn(0x7f1df80c14e0 0x7f1df80c1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.420+0000 7f1df559a640 1 -- 172.21.15.154:0/642093975 --> v1:172.21.15.190:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df806cde0 con 0x7f1df8161010 2023-09-23T03:15:53.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.420+0000 7f1df559a640 1 -- 172.21.15.154:0/642093975 <== mon.1 v1:172.21.15.190:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f1de80016c0 con 0x7f1df8161010 2023-09-23T03:15:53.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.419+0000 7f1894ff9640 1 --2- 172.21.15.154:0/4161737363 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f188806b2e0 0x7f188806d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.420+0000 7f1894ff9640 1 -- 172.21.15.154:0/4161737363 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f188c089e40 con 0x7f189003a860 2023-09-23T03:15:53.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.420+0000 7ff6f259c640 1 --2- 172.21.15.154:0/3613602406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6dc003970 0x7ff6dc003d70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff6e400c1e0 tx=0x7ff6dc0057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfa6aaafef88dcc6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.420+0000 7f1897fff640 1 --2- 172.21.15.154:0/4161737363 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f188806b2e0 0x7f188806d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.420+0000 7f1df559a640 1 -- 172.21.15.154:0/642093975 >> v1:172.21.15.190:6789/0 conn(0x7f1df8161010 legacy=0x7f1df80cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.421+0000 7ff6f159a640 1 -- 172.21.15.154:0/3613602406 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e402ca30 con 0x7ff6dc003970 2023-09-23T03:15:53.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.421+0000 7f1df559a640 1 --2- 172.21.15.154:0/642093975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1df806c1d0 0x7f1de0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.421+0000 7f189d6c3640 1 -- 172.21.15.154:0/4161737363 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1878005250 con 0x7f189003a860 2023-09-23T03:15:53.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.421+0000 7ff6f159a640 1 -- 172.21.15.154:0/3613602406 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e402cbb0 con 0x7ff6dc003970 2023-09-23T03:15:53.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.421+0000 7f1df559a640 1 --2- 172.21.15.154:0/642093975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1de0003eb0 0x7f1de00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.421+0000 7ff6f159a640 1 -- 172.21.15.154:0/3613602406 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4034440 con 0x7ff6dc003970 2023-09-23T03:15:53.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.421+0000 7f1df559a640 1 --2- 172.21.15.154:0/642093975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1de0004d00 0x7f1de0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.421+0000 7f1df659c640 1 --2- 172.21.15.154:0/642093975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1df806c1d0 0x7f1de0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.422+0000 7f1df5d9b640 1 --2- 172.21.15.154:0/642093975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1de0003eb0 0x7f1de00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.422+0000 7f1df559a640 1 -- 172.21.15.154:0/642093975 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df80c1ea0 con 0x7f1de0004d00 2023-09-23T03:15:53.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.422+0000 7f1df559a640 1 -- 172.21.15.154:0/642093975 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df806cb60 con 0x7f1de0003eb0 2023-09-23T03:15:53.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.422+0000 7f1df559a640 1 -- 172.21.15.154:0/642093975 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1de000bae0 con 0x7f1df806c1d0 2023-09-23T03:15:53.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.422+0000 7ff6f8b25640 1 -- 172.21.15.154:0/3613602406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6dc003970 msgr2=0x7ff6dc003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.422+0000 7ff6f8b25640 1 --2- 172.21.15.154:0/3613602406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6dc003970 0x7ff6dc003d70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff6e400c1e0 tx=0x7ff6dc0057f0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.423+0000 7f97b0960640 1 Processor -- start 2023-09-23T03:15:53.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.423+0000 7f97b0960640 1 -- start start 2023-09-23T03:15:53.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.423+0000 7f1df659c640 1 -- 172.21.15.154:0/642093975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1de0003eb0 msgr2=0x7f1de00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:53.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.423+0000 7f1df659c640 1 --2- 172.21.15.154:0/642093975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1de0003eb0 0x7f1de00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.423+0000 7f1897fff640 1 --2- 172.21.15.154:0/4161737363 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f188806b2e0 0x7f188806d7a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1884001900 tx=0x7f1884004e50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.423+0000 7f1df659c640 1 -- 172.21.15.154:0/642093975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1de0004d00 msgr2=0x7f1de0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:53.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.423+0000 7f1df659c640 1 --2- 172.21.15.154:0/642093975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1de0004d00 0x7f1de0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.424+0000 7f1df659c640 1 -- 172.21.15.154:0/642093975 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de0003520 con 0x7f1df806c1d0 2023-09-23T03:15:53.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.424+0000 7f1df5d9b640 1 --2- 172.21.15.154:0/642093975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1de0003eb0 0x7f1de00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:15:53.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.424+0000 7f97b0960640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97ac0c99d0 0x7f97ac156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.424+0000 7f97b0960640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97ac15c840 0x7f97ac0c69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.425+0000 7f97b0960640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97ac160fd0 0x7f97ac0aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.425+0000 7f97b0960640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac1566a0 con 0x7f97ac15c840 2023-09-23T03:15:53.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.424+0000 7ff6f2d9d640 1 -- 172.21.15.154:0/3613602406 reap_dead start 2023-09-23T03:15:53.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.425+0000 7f97b0960640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac0b38a0 con 0x7f97ac160fd0 2023-09-23T03:15:53.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.425+0000 7f97b0960640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac0b3b80 con 0x7f97ac0c99d0 2023-09-23T03:15:53.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.425+0000 7ff6f8b25640 1 -- 172.21.15.154:0/3613602406 shutdown_connections 2023-09-23T03:15:53.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.425+0000 7ff6f8b25640 1 -- 172.21.15.154:0/3613602406 >> 172.21.15.154:0/3613602406 conn(0x7ff6ec0538d0 msgr2=0x7ff6ec053ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:53.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.425+0000 7f1df659c640 1 --2- 172.21.15.154:0/642093975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1df806c1d0 0x7f1de0003970 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1de8005040 tx=0x7f1de00057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2af570b3039edc1b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.426+0000 7f97aa59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97ac15c840 0x7f97ac0c69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.426+0000 7f97aa59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97ac15c840 0x7f97ac0c69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44686/0 (socket says 172.21.15.154:44686) 2023-09-23T03:15:53.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.426+0000 7f97aa59c640 1 -- 172.21.15.154:0/3872515652 learned_addr learned my addr 172.21.15.154:0/3872515652 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:53.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.426+0000 7ff6f8b25640 1 -- 172.21.15.154:0/3613602406 shutdown_connections 2023-09-23T03:15:53.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.426+0000 7f97aad9d640 1 --2- 172.21.15.154:0/3872515652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97ac0c99d0 0x7f97ac156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.426+0000 7f1df559a640 1 -- 172.21.15.154:0/642093975 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de8005b20 con 0x7f1df806c1d0 2023-09-23T03:15:53.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.426+0000 7f1df559a640 1 -- 172.21.15.154:0/642093975 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de802f070 con 0x7f1df806c1d0 2023-09-23T03:15:53.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.427+0000 7f97ab59e640 1 --2- 172.21.15.154:0/3872515652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97ac160fd0 0x7f97ac0aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.427+0000 7f1df559a640 1 -- 172.21.15.154:0/642093975 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de80080d0 con 0x7f1df806c1d0 2023-09-23T03:15:53.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.427+0000 7f97aa59c640 1 -- 172.21.15.154:0/3872515652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97ac0c99d0 msgr2=0x7f97ac156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.427+0000 7f97aa59c640 1 --2- 172.21.15.154:0/3872515652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97ac0c99d0 0x7f97ac156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.427+0000 7f97aa59c640 1 -- 172.21.15.154:0/3872515652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97ac160fd0 msgr2=0x7f97ac0aeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.427+0000 7f97aa59c640 1 --2- 172.21.15.154:0/3872515652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97ac160fd0 0x7f97ac0aeb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.427+0000 7f97aa59c640 1 -- 172.21.15.154:0/3872515652 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979c0094d0 con 0x7f97ac15c840 2023-09-23T03:15:53.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.427+0000 7f1dfca90640 1 -- 172.21.15.154:0/642093975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1df806c1d0 msgr2=0x7f1de0003970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.427+0000 7f1dfca90640 1 --2- 172.21.15.154:0/642093975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1df806c1d0 0x7f1de0003970 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1de8005040 tx=0x7f1de00057f0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.427+0000 7ff6f8b25640 1 -- 172.21.15.154:0/3613602406 wait complete. 2023-09-23T03:15:53.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.427+0000 7f1df6d9d640 1 -- 172.21.15.154:0/642093975 reap_dead start 2023-09-23T03:15:53.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.427+0000 7f1dfca90640 1 -- 172.21.15.154:0/642093975 shutdown_connections 2023-09-23T03:15:53.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.427+0000 7f1dfca90640 1 -- 172.21.15.154:0/642093975 >> 172.21.15.154:0/642093975 conn(0x7f1df806a0f0 msgr2=0x7f1df806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:53.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.428+0000 7f1dfca90640 1 -- 172.21.15.154:0/642093975 shutdown_connections 2023-09-23T03:15:53.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.428+0000 7f1dfca90640 1 -- 172.21.15.154:0/642093975 wait complete. 2023-09-23T03:15:53.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.428+0000 7f97aa59c640 1 --2- 172.21.15.154:0/3872515652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97ac15c840 0x7f97ac0c69e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f97a400b9c0 tx=0x7f97a400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.431+0000 7f97937fe640 1 -- 172.21.15.154:0/3872515652 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a400cba0 con 0x7f97ac15c840 2023-09-23T03:15:53.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.431+0000 7f1dfca90640 1 Processor -- start 2023-09-23T03:15:53.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.431+0000 7f97937fe640 1 -- 172.21.15.154:0/3872515652 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a400cd00 con 0x7f97ac15c840 2023-09-23T03:15:53.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.431+0000 7f97937fe640 1 -- 172.21.15.154:0/3872515652 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a4014680 con 0x7f97ac15c840 2023-09-23T03:15:53.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.431+0000 7f1894ff9640 1 -- 172.21.15.154:0/4161737363 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f188c089080 con 0x7f189003a860 2023-09-23T03:15:53.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.431+0000 7ff6f8b25640 1 Processor -- start 2023-09-23T03:15:53.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.432+0000 7ff6f8b25640 1 -- start start 2023-09-23T03:15:53.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.432+0000 7f97b0960640 1 -- 172.21.15.154:0/3872515652 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97ac0b3e60 con 0x7f97ac15c840 2023-09-23T03:15:53.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.431+0000 7f1dfca90640 1 -- start start 2023-09-23T03:15:53.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.432+0000 7f1dfca90640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1df8156900 0x7f1df8163390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.433+0000 7f97b0960640 1 -- 172.21.15.154:0/3872515652 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ac15f3d0 con 0x7f97ac15c840 2023-09-23T03:15:53.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.433+0000 7f1dfca90640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1df80c55e0 0x7f1df80c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.433+0000 7f1dfca90640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1df8161da0 0x7f1df8162220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.433+0000 7f1dfca90640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df8161910 con 0x7f1df80c55e0 2023-09-23T03:15:53.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.433+0000 7f1dfca90640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df80b0e00 con 0x7f1df8161da0 2023-09-23T03:15:53.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.433+0000 7f1dfca90640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df80b0840 con 0x7f1df8156900 2023-09-23T03:15:53.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.433+0000 7ff6f8b25640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6ec0e2d30 0x7ff6ec050110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.433+0000 7f54501e7640 1 Processor -- start 2023-09-23T03:15:53.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.433+0000 7f1df6d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1df8161da0 0x7f1df8162220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.434+0000 7f1df6d9d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1df8161da0 0x7f1df8162220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.154:47972/0 (socket says 172.21.15.154:47972) 2023-09-23T03:15:53.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.434+0000 7f1df6d9d640 1 -- 172.21.15.154:0/2846989260 learned_addr learned my addr 172.21.15.154:0/2846989260 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:53.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.434+0000 7ff6f8b25640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6ec04d250 0x7ff6ec04f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.434+0000 7f97917fa640 1 -- 172.21.15.154:0/3872515652 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97ac0736b0 con 0x7f97ac15c840 2023-09-23T03:15:53.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.434+0000 7f1df5d9b640 1 --2- 172.21.15.154:0/2846989260 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1df80c55e0 0x7f1df80c5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.434+0000 7f1df659c640 1 --2- 172.21.15.154:0/2846989260 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1df8156900 0x7f1df8163390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.434+0000 7ff6f259c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6ec0e2d30 0x7ff6ec050110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.434+0000 7ff6f259c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6ec0e2d30 0x7ff6ec050110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:44070/0 (socket says 172.21.15.154:44070) 2023-09-23T03:15:53.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.434+0000 7f54501e7640 1 -- start start 2023-09-23T03:15:53.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.434+0000 7ff6f1d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6ec04d250 0x7ff6ec04f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.435+0000 7f92b3226640 1 Processor -- start 2023-09-23T03:15:53.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.435+0000 7f92b3226640 1 -- start start 2023-09-23T03:15:53.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.435+0000 7ff6f259c640 1 -- 172.21.15.154:0/3062981940 learned_addr learned my addr 172.21.15.154:0/3062981940 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:53.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.435+0000 7ff6f8b25640 1 --2- 172.21.15.154:0/3062981940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6ec04fc20 0x7ff6ec0e7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.435+0000 7ff6f8b25640 1 -- 172.21.15.154:0/3062981940 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec0eb800 con 0x7ff6ec04d250 2023-09-23T03:15:53.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.435+0000 7ff6f8b25640 1 -- 172.21.15.154:0/3062981940 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec04ef70 con 0x7ff6ec04fc20 2023-09-23T03:15:53.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.435+0000 7ff6f8b25640 1 -- 172.21.15.154:0/3062981940 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec04f250 con 0x7ff6ec0e2d30 2023-09-23T03:15:53.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.436+0000 7f1df659c640 1 -- 172.21.15.154:0/2846989260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1df8161da0 msgr2=0x7f1df8162220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.436+0000 7f1df659c640 1 --2- 172.21.15.154:0/2846989260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1df8161da0 0x7f1df8162220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.436+0000 7f1df659c640 1 -- 172.21.15.154:0/2846989260 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1df80c55e0 msgr2=0x7f1df80c5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.436+0000 7f1df659c640 1 --2- 172.21.15.154:0/2846989260 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1df80c55e0 0x7f1df80c5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.436+0000 7f1df659c640 1 -- 172.21.15.154:0/2846989260 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df80b0b20 con 0x7f1df8156900 2023-09-23T03:15:53.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.436+0000 7ff6f1d9b640 1 -- 172.21.15.154:0/3062981940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6ec0e2d30 msgr2=0x7ff6ec050110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.436+0000 7ff6f1d9b640 1 --2- 172.21.15.154:0/3062981940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6ec0e2d30 0x7ff6ec050110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.436+0000 7ff6f1d9b640 1 -- 172.21.15.154:0/3062981940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6ec04fc20 msgr2=0x7ff6ec0e7ca0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-23T03:15:53.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.436+0000 7ff6f1d9b640 1 --2- 172.21.15.154:0/3062981940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6ec04fc20 0x7ff6ec0e7ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.436+0000 7ff6f1d9b640 1 -- 172.21.15.154:0/3062981940 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ec0506f0 con 0x7ff6ec04d250 2023-09-23T03:15:53.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.436+0000 7f54501e7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5448159370 0x7f54480659f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.436+0000 7f54501e7640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f54480c06f0 0x7f544806ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.436+0000 7f54501e7640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54480b1870 con 0x7f54480c0b50 2023-09-23T03:15:53.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.436+0000 7f54501e7640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f544806a7a0 con 0x7f54480c06f0 2023-09-23T03:15:53.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.436+0000 7f54501e7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f544815ff60 con 0x7f5448159370 2023-09-23T03:15:53.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.436+0000 7ff6f1d9b640 1 --2- 172.21.15.154:0/3062981940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6ec04d250 0x7ff6ec04f6e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec054800 tx=0x7ff6e800c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.436+0000 7f1df659c640 1 --2- 172.21.15.154:0/2846989260 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1df8156900 0x7f1df8163390 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1de8030000 tx=0x7f1de8004a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.437+0000 7f92b3226640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f92ac0cf130 0x7f92ac0ada30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.437+0000 7f92b3226640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92ac0ccaf0 0x7f92ac15d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.437+0000 7f92b3226640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92ac06ceb0 con 0x7f92ac0b2fe0 2023-09-23T03:15:53.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.437+0000 7f92b3226640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92ac0ccfa0 con 0x7f92ac0cf130 2023-09-23T03:15:53.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.437+0000 7f92b3226640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac159c90 con 0x7f92ac0ccaf0 2023-09-23T03:15:53.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.437+0000 7f544d782640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f54480c06f0 0x7f544806ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.437+0000 7f544d782640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f54480c06f0 0x7f544806ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.154:47990/0 (socket says 172.21.15.154:47990) 2023-09-23T03:15:53.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.437+0000 7f544d782640 1 -- 172.21.15.154:0/1984238005 learned_addr learned my addr 172.21.15.154:0/1984238005 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:53.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.437+0000 7f544df83640 1 --2- 172.21.15.154:0/1984238005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5448159370 0x7f54480659f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.438+0000 7f92b0fc2640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f92ac0b2fe0 0x7f92ac0cea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.154:42478/0 (socket says 172.21.15.154:42478) 2023-09-23T03:15:53.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.438+0000 7f92b0fc2640 1 -- 172.21.15.154:0/705885524 learned_addr learned my addr 172.21.15.154:0/705885524 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:53.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.438+0000 7f544d782640 1 -- 172.21.15.154:0/1984238005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5448159370 msgr2=0x7f54480659f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:53.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.438+0000 7f544d782640 1 --2- 172.21.15.154:0/1984238005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5448159370 0x7f54480659f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.438+0000 7f92b17c3640 1 --2- 172.21.15.154:0/705885524 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92ac0ccaf0 0x7f92ac15d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.438+0000 7f544cf81640 1 -- 172.21.15.154:0/1984238005 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2274857563 0 0) 0x7f5438001110 con 0x7f54480c0b50 2023-09-23T03:15:53.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.438+0000 7f544d782640 1 -- 172.21.15.154:0/1984238005 >> v1:172.21.15.154:6789/0 conn(0x7f54480c0b50 legacy=0x7f5448159750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.439+0000 7f544d782640 1 -- 172.21.15.154:0/1984238005 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5448069e70 con 0x7f54480c06f0 2023-09-23T03:15:53.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.439+0000 7f97937fe640 1 -- 172.21.15.154:0/3872515652 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f97a401d3b0 con 0x7f97ac15c840 2023-09-23T03:15:53.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.439+0000 7f544df83640 1 --2- 172.21.15.154:0/1984238005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5448159370 0x7f54480659f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:15:53.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.439+0000 7f92b17c3640 1 -- 172.21.15.154:0/705885524 >> v2:172.21.15.154:3300/0 conn(0x7f92ac0cf130 msgr2=0x7f92ac0ada30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T03:15:53.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.439+0000 7f92b17c3640 1 --2- 172.21.15.154:0/705885524 >> v2:172.21.15.154:3300/0 conn(0x7f92ac0cf130 0x7f92ac0ada30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.439+0000 7f92b17c3640 1 -- 172.21.15.154:0/705885524 >> v1:172.21.15.190:6789/0 conn(0x7f92ac0b2fe0 legacy=0x7f92ac0cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:53.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.439+0000 7f92b17c3640 1 -- 172.21.15.154:0/705885524 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ac06c110 con 0x7f92ac0ccaf0 2023-09-23T03:15:53.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.438+0000 7f1de6ffd640 1 -- 172.21.15.154:0/2846989260 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de8039070 con 0x7f1df8156900 2023-09-23T03:15:53.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.438+0000 7f1de6ffd640 1 -- 172.21.15.154:0/2846989260 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de8005b10 con 0x7f1df8156900 2023-09-23T03:15:53.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.441+0000 7ff6e2ffd640 1 -- 172.21.15.154:0/3062981940 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8005ce0 con 0x7ff6ec04d250 2023-09-23T03:15:53.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.438+0000 7f1dfca90640 1 -- 172.21.15.154:0/2846989260 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df81615a0 con 0x7f1df8156900 2023-09-23T03:15:53.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.439+0000 7f1dfca90640 1 -- 172.21.15.154:0/2846989260 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df80b0510 con 0x7f1df8156900 2023-09-23T03:15:53.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.441+0000 7ff6e2ffd640 1 -- 172.21.15.154:0/3062981940 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e8005e40 con 0x7ff6ec04d250 2023-09-23T03:15:53.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.441+0000 7f544d782640 1 --2- 172.21.15.154:0/1984238005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54480c06f0 0x7f544806ac30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5440009e50 tx=0x7f544002bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eda65156cdb6ce0e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.441+0000 7f97937fe640 1 --2- 172.21.15.154:0/3872515652 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f979806af30 0x7f979806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.441+0000 7ff6f8b25640 1 -- 172.21.15.154:0/3062981940 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6ec0509d0 con 0x7ff6ec04d250 2023-09-23T03:15:53.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.439+0000 7f1de6ffd640 1 -- 172.21.15.154:0/2846989260 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de8005ca0 con 0x7f1df8156900 2023-09-23T03:15:53.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.440+0000 7f1dfca90640 1 -- 172.21.15.154:0/2846989260 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1df80cc280 con 0x7f1df8156900 2023-09-23T03:15:53.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.442+0000 7f97937fe640 1 -- 172.21.15.154:0/3872515652 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f97a40899c0 con 0x7f97ac15c840 2023-09-23T03:15:53.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.442+0000 7f97aad9d640 1 --2- 172.21.15.154:0/3872515652 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f979806af30 0x7f979806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.442+0000 7ff6f8b25640 1 -- 172.21.15.154:0/3062981940 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6ec04d6d0 con 0x7ff6ec04d250 2023-09-23T03:15:53.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.442+0000 7f544cf81640 1 -- 172.21.15.154:0/1984238005 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544002cb00 con 0x7f54480c06f0 2023-09-23T03:15:53.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.442+0000 7f544cf81640 1 -- 172.21.15.154:0/1984238005 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f544002cc60 con 0x7f54480c06f0 2023-09-23T03:15:53.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.442+0000 7f544cf81640 1 -- 172.21.15.154:0/1984238005 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54400343f0 con 0x7f54480c06f0 2023-09-23T03:15:53.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.441+0000 7f92b17c3640 1 --2- 172.21.15.154:0/705885524 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92ac0ccaf0 0x7f92ac15d380 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f92940097a0 tx=0x7f929402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a54f5abf6462aef7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.443+0000 7f97aad9d640 1 --2- 172.21.15.154:0/3872515652 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f979806af30 0x7f979806d3f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f979c036000 tx=0x7f979c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.443+0000 7ff6e2ffd640 1 -- 172.21.15.154:0/3062981940 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e80072d0 con 0x7ff6ec04d250 2023-09-23T03:15:53.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.443+0000 7f1de6ffd640 1 -- 172.21.15.154:0/2846989260 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f1de80080d0 con 0x7f1df8156900 2023-09-23T03:15:53.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.443+0000 7f54501e7640 1 -- 172.21.15.154:0/1984238005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54480c06f0 msgr2=0x7f544806ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.443+0000 7f54501e7640 1 --2- 172.21.15.154:0/1984238005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54480c06f0 0x7f544806ac30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5440009e50 tx=0x7f544002bf60 comp rx=0 tx=0).stop 2023-09-23T03:15:53.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.443+0000 7fd8066b4640 1 Processor -- start 2023-09-23T03:15:53.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.443+0000 7f54501e7640 1 -- 172.21.15.154:0/1984238005 shutdown_connections 2023-09-23T03:15:53.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.443+0000 7f54501e7640 1 --2- 172.21.15.154:0/1984238005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54480c06f0 0x7f544806ac30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.443+0000 7f54501e7640 1 --2- 172.21.15.154:0/1984238005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5448159370 0x7f54480659f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.444+0000 7f54501e7640 1 -- 172.21.15.154:0/1984238005 >> 172.21.15.154:0/1984238005 conn(0x7f54480b08e0 msgr2=0x7f54480b0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:53.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.444+0000 7f54501e7640 1 -- 172.21.15.154:0/1984238005 shutdown_connections 2023-09-23T03:15:53.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.443+0000 7fd8066b4640 1 -- start start 2023-09-23T03:15:53.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.444+0000 7ff6f8b25640 1 -- 172.21.15.154:0/3062981940 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6ec005910 con 0x7ff6ec04d250 2023-09-23T03:15:53.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.444+0000 7f54501e7640 1 -- 172.21.15.154:0/1984238005 wait complete. 2023-09-23T03:15:53.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.445+0000 7f1de6ffd640 1 --2- 172.21.15.154:0/2846989260 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1dd806b000 0x7f1dd806d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.445+0000 7f1de6ffd640 1 -- 172.21.15.154:0/2846989260 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1de80ab2b0 con 0x7f1df8156900 2023-09-23T03:15:53.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.445+0000 7fd8066b4640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd8000cf2a0 0x7fd8000cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.445+0000 7f185b1c9640 1 Processor -- start 2023-09-23T03:15:53.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.445+0000 7f185b1c9640 1 -- start start 2023-09-23T03:15:53.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.446+0000 7f1df5d9b640 1 --2- 172.21.15.154:0/2846989260 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1dd806b000 0x7f1dd806d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.446+0000 7fd8066b4640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8000ce990 con 0x7fd80015cac0 2023-09-23T03:15:53.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.446+0000 7fd7fffff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fd80015cac0 0x7fd80006b280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:42344/0 (socket says 172.21.15.154:42344) 2023-09-23T03:15:53.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.446+0000 7fd7fffff640 1 -- 172.21.15.154:0/1455666776 learned_addr learned my addr 172.21.15.154:0/1455666776 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:53.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.446+0000 7fd8066b4640 1 -- 172.21.15.154:0/1455666776 --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8000ce3d0 con 0x7fd8000cd670 2023-09-23T03:15:53.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.446+0000 7fd8066b4640 1 -- 172.21.15.154:0/1455666776 --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8000ce1d0 con 0x7fd8000cf2a0 2023-09-23T03:15:53.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.446+0000 7fd7ff7fe640 1 --2- 172.21.15.154:0/1455666776 >> v2:172.21.15.190:3300/0 conn(0x7fd8000cf2a0 0x7fd8000cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.447+0000 7f54501e7640 1 Processor -- start 2023-09-23T03:15:53.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.447+0000 7f1df5d9b640 1 --2- 172.21.15.154:0/2846989260 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1dd806b000 0x7f1dd806d4c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1df80c51b0 tx=0x7f1dec009060 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.447+0000 7f54501e7640 1 -- start start 2023-09-23T03:15:53.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.447+0000 7f185b1c9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f185415caf0 0x7f18540ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.447+0000 7f185b1c9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1854160f90 0x7f18540b1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.447+0000 7f185b1c9640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18540cd530 con 0x7f18540cee70 2023-09-23T03:15:53.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.447+0000 7f185b1c9640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f185415ced0 con 0x7f185415caf0 2023-09-23T03:15:53.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.447+0000 7f185b1c9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18540cf2f0 con 0x7f1854160f90 2023-09-23T03:15:53.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.448+0000 7fd7ff7fe640 1 -- 172.21.15.154:0/1455666776 >> v1:172.21.15.190:6789/0 conn(0x7fd8000cd670 legacy=0x7fd8000cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.448+0000 7fd7ff7fe640 1 -- 172.21.15.154:0/1455666776 >> v1:172.21.15.154:6789/0 conn(0x7fd80015cac0 legacy=0x7fd80006b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.448+0000 7f1858f65640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f185415caf0 0x7f18540ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.448+0000 7fd7feffd640 1 -- 172.21.15.154:0/1455666776 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 843181621 0 0) 0x7fd8000ce3d0 con 0x7fd8000cd670 2023-09-23T03:15:53.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.448+0000 7fd7ff7fe640 1 -- 172.21.15.154:0/1455666776 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8000ce6b0 con 0x7fd8000cf2a0 2023-09-23T03:15:53.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.448+0000 7f1859766640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1854160f90 0x7f18540b1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.448+0000 7f1859766640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1854160f90 0x7f18540b1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:44106/0 (socket says 172.21.15.154:44106) 2023-09-23T03:15:53.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.448+0000 7f1859766640 1 -- 172.21.15.154:0/1148446780 learned_addr learned my addr 172.21.15.154:0/1148446780 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:53.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.448+0000 7f54501e7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54480c06f0 0x7f54480b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.448+0000 7f54501e7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54480c0b50 0x7f544814e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.448+0000 7f54501e7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5448159370 0x7f544814ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.448+0000 7f54501e7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54480c5ec0 con 0x7f54480c0b50 2023-09-23T03:15:53.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.448+0000 7f54501e7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54480ad330 con 0x7f54480c06f0 2023-09-23T03:15:53.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.448+0000 7f54501e7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54480b49b0 con 0x7f5448159370 2023-09-23T03:15:53.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.449+0000 7f1859766640 1 -- 172.21.15.154:0/1148446780 >> v2:172.21.15.190:3300/0 conn(0x7f185415caf0 msgr2=0x7f18540ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:53.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.449+0000 7f1859766640 1 --2- 172.21.15.154:0/1148446780 >> v2:172.21.15.190:3300/0 conn(0x7f185415caf0 0x7f18540ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.449+0000 7f1859766640 1 -- 172.21.15.154:0/1148446780 >> v1:172.21.15.190:6789/0 conn(0x7f18540cee70 legacy=0x7f1854160880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.449+0000 7f1859766640 1 -- 172.21.15.154:0/1148446780 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18540cd810 con 0x7f1854160f90 2023-09-23T03:15:53.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.449+0000 7f544d782640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54480c0b50 0x7f544814e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.449+0000 7f544d782640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54480c0b50 0x7f544814e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44704/0 (socket says 172.21.15.154:44704) 2023-09-23T03:15:53.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.449+0000 7f544d782640 1 -- 172.21.15.154:0/3844892727 learned_addr learned my addr 172.21.15.154:0/3844892727 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:53.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.449+0000 7f544df83640 1 --2- 172.21.15.154:0/3844892727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54480c06f0 0x7f54480b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.449+0000 7f544e784640 1 --2- 172.21.15.154:0/3844892727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5448159370 0x7f544814ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.449+0000 7f544d782640 1 -- 172.21.15.154:0/3844892727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5448159370 msgr2=0x7f544814ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.449+0000 7f544d782640 1 --2- 172.21.15.154:0/3844892727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5448159370 0x7f544814ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.449+0000 7f544d782640 1 -- 172.21.15.154:0/3844892727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54480c06f0 msgr2=0x7f54480b41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.449+0000 7f544d782640 1 --2- 172.21.15.154:0/3844892727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54480c06f0 0x7f54480b41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.450+0000 7f544d782640 1 -- 172.21.15.154:0/3844892727 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5444009400 con 0x7f54480c0b50 2023-09-23T03:15:53.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.450+0000 7f97937fe640 1 -- 172.21.15.154:0/3872515652 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f97a4054730 con 0x7f97ac15c840 2023-09-23T03:15:53.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.450+0000 7fd7ff7fe640 1 --2- 172.21.15.154:0/1455666776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8000cf2a0 0x7fd8000cf710 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd7f40097a0 tx=0x7fd7f402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4c5eed90a72dc58 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.450+0000 7f544d782640 1 --2- 172.21.15.154:0/3844892727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54480c0b50 0x7f544814e7e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5440036040 tx=0x7f5440004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.450+0000 7f1859766640 1 --2- 172.21.15.154:0/1148446780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1854160f90 0x7f18540b1360 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f184c009e50 tx=0x7f184c02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cd40560f40ae252 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.451+0000 7fd7feffd640 1 -- 172.21.15.154:0/1455666776 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4030610 con 0x7fd8000cf2a0 2023-09-23T03:15:53.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.451+0000 7fd7feffd640 1 -- 172.21.15.154:0/1455666776 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f4030770 con 0x7fd8000cf2a0 2023-09-23T03:15:53.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.451+0000 7fd7feffd640 1 -- 172.21.15.154:0/1455666776 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4030b00 con 0x7fd8000cf2a0 2023-09-23T03:15:53.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.451+0000 7ff6e2ffd640 1 -- 172.21.15.154:0/3062981940 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7ff6e80074f0 con 0x7ff6ec04d250 2023-09-23T03:15:53.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.452+0000 7fd8066b4640 1 -- 172.21.15.154:0/1455666776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8000cf2a0 msgr2=0x7fd8000cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.452+0000 7fd8066b4640 1 --2- 172.21.15.154:0/1455666776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8000cf2a0 0x7fd8000cf710 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd7f40097a0 tx=0x7fd7f402b960 comp rx=0 tx=0).stop 2023-09-23T03:15:53.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.452+0000 7fd8066b4640 1 -- 172.21.15.154:0/1455666776 shutdown_connections 2023-09-23T03:15:53.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.452+0000 7fd8066b4640 1 --2- 172.21.15.154:0/1455666776 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8000cf2a0 0x7fd8000cf710 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.452+0000 7fd8066b4640 1 -- 172.21.15.154:0/1455666776 >> 172.21.15.154:0/1455666776 conn(0x7fd800089fd0 msgr2=0x7fd80009bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:53.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.452+0000 7fd8066b4640 1 -- 172.21.15.154:0/1455666776 shutdown_connections 2023-09-23T03:15:53.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.452+0000 7fd8066b4640 1 -- 172.21.15.154:0/1455666776 wait complete. 2023-09-23T03:15:53.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.452+0000 7ff6e2ffd640 1 --2- 172.21.15.154:0/3062981940 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff6d406b000 0x7ff6d406d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.452+0000 7ff6e2ffd640 1 -- 172.21.15.154:0/3062981940 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff6e8089470 con 0x7ff6ec04d250 2023-09-23T03:15:53.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.453+0000 7ff6f259c640 1 --2- 172.21.15.154:0/3062981940 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff6d406b000 0x7ff6d406d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.453+0000 7ff6f259c640 1 --2- 172.21.15.154:0/3062981940 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff6d406b000 0x7ff6d406d4c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4000fb0 tx=0x7ff6e4009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.454+0000 7fd8066b4640 1 Processor -- start 2023-09-23T03:15:53.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.454+0000 7fd8066b4640 1 -- start start 2023-09-23T03:15:53.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.455+0000 7fd8066b4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8000cd670 0x7fd8000b3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.455+0000 7fd8066b4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8000cf2a0 0x7fd8000b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.456+0000 7fd8066b4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd80015cac0 0x7fd8000c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.456+0000 7fd8066b4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8000b2f70 con 0x7fd8000cf2a0 2023-09-23T03:15:53.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.456+0000 7fd8066b4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8000b51a0 con 0x7fd80015cac0 2023-09-23T03:15:53.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.456+0000 7fd8066b4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8000b2360 con 0x7fd8000cd670 2023-09-23T03:15:53.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.456+0000 7fd7ff7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8000cf2a0 0x7fd8000b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.456+0000 7fd7ff7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8000cf2a0 0x7fd8000b4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44712/0 (socket says 172.21.15.154:44712) 2023-09-23T03:15:53.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.456+0000 7fd7ff7fe640 1 -- 172.21.15.154:0/2193159124 learned_addr learned my addr 172.21.15.154:0/2193159124 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:53.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.456+0000 7fd7fffff640 1 --2- 172.21.15.154:0/2193159124 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8000cd670 0x7fd8000b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.456+0000 7fd7ff7fe640 1 -- 172.21.15.154:0/2193159124 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8000cd670 msgr2=0x7fd8000b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.456+0000 7fd7ff7fe640 1 --2- 172.21.15.154:0/2193159124 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8000cd670 0x7fd8000b3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.456+0000 7fd7ff7fe640 1 -- 172.21.15.154:0/2193159124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd80015cac0 msgr2=0x7fd8000c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.456+0000 7fd7ff7fe640 1 --2- 172.21.15.154:0/2193159124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd80015cac0 0x7fd8000c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.456+0000 7fd7ff7fe640 1 -- 172.21.15.154:0/2193159124 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f4009400 con 0x7fd8000cf2a0 2023-09-23T03:15:53.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.456+0000 7fd7fffff640 1 --2- 172.21.15.154:0/2193159124 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8000cd670 0x7fd8000b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:15:53.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.457+0000 7fd7ff7fe640 1 --2- 172.21.15.154:0/2193159124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8000cf2a0 0x7fd8000b4c60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd7f40098c0 tx=0x7fd7f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.459+0000 7f92ab7fe640 1 -- 172.21.15.154:0/705885524 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929402e070 con 0x7f92ac0ccaf0 2023-09-23T03:15:53.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.459+0000 7f92ab7fe640 1 -- 172.21.15.154:0/705885524 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92940315a0 con 0x7f92ac0ccaf0 2023-09-23T03:15:53.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.459+0000 7fd7fcff9640 1 -- 172.21.15.154:0/2193159124 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f403e040 con 0x7fd8000cf2a0 2023-09-23T03:15:53.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.459+0000 7fd7fcff9640 1 -- 172.21.15.154:0/2193159124 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f4039d80 con 0x7fd8000cf2a0 2023-09-23T03:15:53.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.459+0000 7fd7fcff9640 1 -- 172.21.15.154:0/2193159124 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4044630 con 0x7fd8000cf2a0 2023-09-23T03:15:53.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.459+0000 7ff6e2ffd640 1 -- 172.21.15.154:0/3062981940 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7ff6e8054250 con 0x7ff6ec04d250 2023-09-23T03:15:53.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.459+0000 7f1de6ffd640 1 -- 172.21.15.154:0/2846989260 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f1de8076090 con 0x7f1df8156900 2023-09-23T03:15:53.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.459+0000 7fd8066b4640 1 -- 172.21.15.154:0/2193159124 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8000c9230 con 0x7fd8000cf2a0 2023-09-23T03:15:53.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.459+0000 7fd8066b4640 1 -- 172.21.15.154:0/2193159124 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8000b28b0 con 0x7fd8000cf2a0 2023-09-23T03:15:53.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.459+0000 7f543e7fc640 1 -- 172.21.15.154:0/3844892727 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544003e450 con 0x7f54480c0b50 2023-09-23T03:15:53.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.459+0000 7f543e7fc640 1 -- 172.21.15.154:0/3844892727 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f544003e5f0 con 0x7f54480c0b50 2023-09-23T03:15:53.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.459+0000 7f543e7fc640 1 -- 172.21.15.154:0/3844892727 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544003e9a0 con 0x7f54480c0b50 2023-09-23T03:15:53.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.459+0000 7f54501e7640 1 -- 172.21.15.154:0/3844892727 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5440009b30 con 0x7f54480c0b50 2023-09-23T03:15:53.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.459+0000 7f54501e7640 1 -- 172.21.15.154:0/3844892727 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54480b2550 con 0x7f54480c0b50 2023-09-23T03:15:53.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.460+0000 7f92b3226640 1 -- 172.21.15.154:0/705885524 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92ac0ccaf0 msgr2=0x7f92ac15d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.460+0000 7f92b3226640 1 --2- 172.21.15.154:0/705885524 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92ac0ccaf0 0x7f92ac15d380 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f92940097a0 tx=0x7f929402b840 comp rx=0 tx=0).stop 2023-09-23T03:15:53.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.460+0000 7f92b3226640 1 -- 172.21.15.154:0/705885524 shutdown_connections 2023-09-23T03:15:53.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.460+0000 7f92b3226640 1 --2- 172.21.15.154:0/705885524 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92ac0ccaf0 0x7f92ac15d380 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.459+0000 7f18537fe640 1 -- 172.21.15.154:0/1148446780 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184c02cb00 con 0x7f1854160f90 2023-09-23T03:15:53.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.459+0000 7f18537fe640 1 -- 172.21.15.154:0/1148446780 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f184c02cc60 con 0x7f1854160f90 2023-09-23T03:15:53.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.459+0000 7f18537fe640 1 -- 172.21.15.154:0/1148446780 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184c0343f0 con 0x7f1854160f90 2023-09-23T03:15:53.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.460+0000 7f185b1c9640 1 -- 172.21.15.154:0/1148446780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1854160f90 msgr2=0x7f18540b1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.460+0000 7f185b1c9640 1 --2- 172.21.15.154:0/1148446780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1854160f90 0x7f18540b1360 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f184c009e50 tx=0x7f184c02bf60 comp rx=0 tx=0).stop 2023-09-23T03:15:53.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.460+0000 7f92b3226640 1 --2- 172.21.15.154:0/705885524 >> v2:172.21.15.154:3300/0 conn(0x7f92ac0cf130 0x7f92ac0ada30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.460+0000 7f92b3226640 1 -- 172.21.15.154:0/705885524 >> 172.21.15.154:0/705885524 conn(0x7f92ac06a0f0 msgr2=0x7f92ac06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:53.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.461+0000 7f185b1c9640 1 -- 172.21.15.154:0/1148446780 shutdown_connections 2023-09-23T03:15:53.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.461+0000 7f185b1c9640 1 --2- 172.21.15.154:0/1148446780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1854160f90 0x7f18540b1360 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.461+0000 7f185b1c9640 1 --2- 172.21.15.154:0/1148446780 >> v2:172.21.15.190:3300/0 conn(0x7f185415caf0 0x7f18540ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.461+0000 7f185b1c9640 1 -- 172.21.15.154:0/1148446780 >> 172.21.15.154:0/1148446780 conn(0x7f185406a0f0 msgr2=0x7f185406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:53.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.461+0000 7f54501e7640 1 -- 172.21.15.154:0/3844892727 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54480cde90 con 0x7f54480c0b50 2023-09-23T03:15:53.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.461+0000 7f92b3226640 1 -- 172.21.15.154:0/705885524 shutdown_connections 2023-09-23T03:15:53.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.462+0000 7f92b3226640 1 -- 172.21.15.154:0/705885524 wait complete. 2023-09-23T03:15:53.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.462+0000 7fd8066b4640 1 -- 172.21.15.154:0/2193159124 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8000b30b0 con 0x7fd8000cf2a0 2023-09-23T03:15:53.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.464+0000 7f185b1c9640 1 -- 172.21.15.154:0/1148446780 shutdown_connections 2023-09-23T03:15:53.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.465+0000 7f92b3226640 1 Processor -- start 2023-09-23T03:15:53.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.466+0000 7f92b3226640 1 -- start start 2023-09-23T03:15:53.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.465+0000 7f543e7fc640 1 -- 172.21.15.154:0/3844892727 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f544003eb00 con 0x7f54480c0b50 2023-09-23T03:15:53.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.466+0000 7f543e7fc640 1 --2- 172.21.15.154:0/3844892727 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f542c06afb0 0x7f542c06d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.464+0000 7f185b1c9640 1 -- 172.21.15.154:0/1148446780 wait complete. 2023-09-23T03:15:53.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.467+0000 7f185b1c9640 1 Processor -- start 2023-09-23T03:15:53.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.467+0000 7f544df83640 1 --2- 172.21.15.154:0/3844892727 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f542c06afb0 0x7f542c06d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.468+0000 7f92b3226640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92ac0b2fe0 0x7f92ac0ada30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.468+0000 7f544df83640 1 --2- 172.21.15.154:0/3844892727 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f542c06afb0 0x7f542c06d470 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5444009c10 tx=0x7f5444008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.467+0000 7f185b1c9640 1 -- start start 2023-09-23T03:15:53.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.468+0000 7f185b1c9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18540cee70 0x7f18541f54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.469+0000 7f92b3226640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92ac0ccaf0 0x7f92ac0adf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.468+0000 7f543e7fc640 1 -- 172.21.15.154:0/3844892727 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f544007bd90 con 0x7f54480c0b50 2023-09-23T03:15:53.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.469+0000 7f92b3226640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92ac0cf130 0x7f92ac0ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.469+0000 7f1858f65640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18540cee70 0x7f18541f54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.469+0000 7f1858f65640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18540cee70 0x7f18541f54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44724/0 (socket says 172.21.15.154:44724) 2023-09-23T03:15:53.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.469+0000 7f1858f65640 1 -- 172.21.15.154:0/3537493835 learned_addr learned my addr 172.21.15.154:0/3537493835 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:53.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.470+0000 7f185b1c9640 1 --2- 172.21.15.154:0/3537493835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f185415caf0 0x7f18540c3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.469+0000 7f92abfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92ac0ccaf0 0x7f92ac0adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.469+0000 7f92abfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92ac0ccaf0 0x7f92ac0adf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44738/0 (socket says 172.21.15.154:44738) 2023-09-23T03:15:53.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.470+0000 7f185b1c9640 1 --2- 172.21.15.154:0/3537493835 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1854160f90 0x7f18540c3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.470+0000 7f92b3226640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac14b570 con 0x7f92ac0ccaf0 2023-09-23T03:15:53.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.470+0000 7f92b3226640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac14b850 con 0x7f92ac0b2fe0 2023-09-23T03:15:53.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.470+0000 7f92b3226640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac14bb30 con 0x7f92ac0cf130 2023-09-23T03:15:53.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.470+0000 7f92abfff640 1 -- 172.21.15.154:0/3770586883 learned_addr learned my addr 172.21.15.154:0/3770586883 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:53.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.466+0000 7fd7fcff9640 1 -- 172.21.15.154:0/2193159124 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fd7f404d3b0 con 0x7fd8000cf2a0 2023-09-23T03:15:53.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.468+0000 7fd7fcff9640 1 --2- 172.21.15.154:0/2193159124 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd7ec06b030 0x7fd7ec06d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.468+0000 7fd7fcff9640 1 -- 172.21.15.154:0/2193159124 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd7f40ac350 con 0x7fd8000cf2a0 2023-09-23T03:15:53.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.469+0000 7fd7fffff640 1 --2- 172.21.15.154:0/2193159124 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd7ec06b030 0x7fd7ec06d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.471+0000 7f185b1c9640 1 -- 172.21.15.154:0/3537493835 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185415dc50 con 0x7f18540cee70 2023-09-23T03:15:53.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.471+0000 7f185b1c9640 1 -- 172.21.15.154:0/3537493835 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18540c4ae0 con 0x7f185415caf0 2023-09-23T03:15:53.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.471+0000 7f185b1c9640 1 -- 172.21.15.154:0/3537493835 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18540c3d90 con 0x7f1854160f90 2023-09-23T03:15:53.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.471+0000 7f1853fff640 1 --2- 172.21.15.154:0/3537493835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f185415caf0 0x7f18540c3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.471+0000 7f1859766640 1 --2- 172.21.15.154:0/3537493835 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1854160f90 0x7f18540c3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.472+0000 7fd7fffff640 1 --2- 172.21.15.154:0/2193159124 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd7ec06b030 0x7fd7ec06d4f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0004980 tx=0x7fd7f0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.471+0000 7f92b17c3640 1 --2- 172.21.15.154:0/3770586883 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92ac0cf130 0x7f92ac0ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.471+0000 7f92b0fc2640 1 --2- 172.21.15.154:0/3770586883 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92ac0b2fe0 0x7f92ac0ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.472+0000 7f92abfff640 1 -- 172.21.15.154:0/3770586883 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92ac0cf130 msgr2=0x7f92ac0ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.472+0000 7f92abfff640 1 --2- 172.21.15.154:0/3770586883 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92ac0cf130 0x7f92ac0ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.472+0000 7f92abfff640 1 -- 172.21.15.154:0/3770586883 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92ac0b2fe0 msgr2=0x7f92ac0ada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.472+0000 7f92abfff640 1 --2- 172.21.15.154:0/3770586883 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92ac0b2fe0 0x7f92ac0ada30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.472+0000 7f92abfff640 1 -- 172.21.15.154:0/3770586883 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9294009400 con 0x7f92ac0ccaf0 2023-09-23T03:15:53.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.472+0000 7f1859766640 1 -- 172.21.15.154:0/3537493835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f185415caf0 msgr2=0x7f18540c3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.472+0000 7f1859766640 1 --2- 172.21.15.154:0/3537493835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f185415caf0 0x7f18540c3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.472+0000 7f1859766640 1 -- 172.21.15.154:0/3537493835 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18540cee70 msgr2=0x7f18541f54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.472+0000 7f543e7fc640 1 -- 172.21.15.154:0/3844892727 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f5440078160 con 0x7f54480c0b50 2023-09-23T03:15:53.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.472+0000 7f1859766640 1 --2- 172.21.15.154:0/3537493835 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18540cee70 0x7f18541f54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.472+0000 7f1859766640 1 -- 172.21.15.154:0/3537493835 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f184c009b30 con 0x7f1854160f90 2023-09-23T03:15:53.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.472+0000 7f1858f65640 1 --2- 172.21.15.154:0/3537493835 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18540cee70 0x7f18541f54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:15:53.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.473+0000 7f92abfff640 1 --2- 172.21.15.154:0/3770586883 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92ac0ccaf0 0x7f92ac0adf70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f92a400b9b0 tx=0x7f92a400be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.473+0000 7f1859766640 1 --2- 172.21.15.154:0/3537493835 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1854160f90 0x7f18540c3850 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f184c032000 tx=0x7f184c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.473+0000 7f18517fa640 1 -- 172.21.15.154:0/3537493835 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184c040670 con 0x7f1854160f90 2023-09-23T03:15:53.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.474+0000 7f18517fa640 1 -- 172.21.15.154:0/3537493835 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f184c0407f0 con 0x7f1854160f90 2023-09-23T03:15:53.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.474+0000 7f185b1c9640 1 -- 172.21.15.154:0/3537493835 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f185415bc20 con 0x7f1854160f90 2023-09-23T03:15:53.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.474+0000 7f18517fa640 1 -- 172.21.15.154:0/3537493835 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184c040ba0 con 0x7f1854160f90 2023-09-23T03:15:53.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.474+0000 7f185b1c9640 1 -- 172.21.15.154:0/3537493835 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f185415d590 con 0x7f1854160f90 2023-09-23T03:15:53.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.476+0000 7f185b1c9640 1 -- 172.21.15.154:0/3537493835 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f185415dd90 con 0x7f1854160f90 2023-09-23T03:15:53.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.478+0000 7f18517fa640 1 -- 172.21.15.154:0/3537493835 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f184c040d00 con 0x7f1854160f90 2023-09-23T03:15:53.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.480+0000 7f18517fa640 1 --2- 172.21.15.154:0/3537493835 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f183806af30 0x7f183806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.480+0000 7f18517fa640 1 -- 172.21.15.154:0/3537493835 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f184c0ac7a0 con 0x7f1854160f90 2023-09-23T03:15:53.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.480+0000 7f92a97fa640 1 -- 172.21.15.154:0/3770586883 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a400cba0 con 0x7f92ac0ccaf0 2023-09-23T03:15:53.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.480+0000 7f92a97fa640 1 -- 172.21.15.154:0/3770586883 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a400cd00 con 0x7f92ac0ccaf0 2023-09-23T03:15:53.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.480+0000 7f92a97fa640 1 -- 172.21.15.154:0/3770586883 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a4014680 con 0x7f92ac0ccaf0 2023-09-23T03:15:53.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.480+0000 7f92b3226640 1 -- 172.21.15.154:0/3770586883 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92ac14be10 con 0x7f92ac0ccaf0 2023-09-23T03:15:53.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.480+0000 7f92b3226640 1 -- 172.21.15.154:0/3770586883 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92ac14dd40 con 0x7f92ac0ccaf0 2023-09-23T03:15:53.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.481+0000 7fd7fcff9640 1 -- 172.21.15.154:0/2193159124 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fd7f4077080 con 0x7fd8000cf2a0 2023-09-23T03:15:53.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.482+0000 7f92b3226640 1 -- 172.21.15.154:0/3770586883 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92ac14c330 con 0x7f92ac0ccaf0 2023-09-23T03:15:53.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.483+0000 7f1858f65640 1 --2- 172.21.15.154:0/3537493835 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f183806af30 0x7f183806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.485+0000 7f1858f65640 1 --2- 172.21.15.154:0/3537493835 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f183806af30 0x7f183806d3f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f18400112c0 tx=0x7f1840009110 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.492+0000 7f92a97fa640 1 -- 172.21.15.154:0/3770586883 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f92a40147e0 con 0x7f92ac0ccaf0 2023-09-23T03:15:53.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.493+0000 7f92a97fa640 1 --2- 172.21.15.154:0/3770586883 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f928c06b000 0x7f928c06d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:53.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.494+0000 7f92b0fc2640 1 --2- 172.21.15.154:0/3770586883 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f928c06b000 0x7f928c06d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:53.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.494+0000 7f18517fa640 1 -- 172.21.15.154:0/3537493835 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f185415dd90 con 0x7f1854160f90 2023-09-23T03:15:53.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.495+0000 7f92a97fa640 1 -- 172.21.15.154:0/3770586883 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f92a4089fa0 con 0x7f92ac0ccaf0 2023-09-23T03:15:53.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.495+0000 7f92a97fa640 1 -- 172.21.15.154:0/3770586883 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f92a408a380 con 0x7f92ac0ccaf0 2023-09-23T03:15:53.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.497+0000 7f92b0fc2640 1 --2- 172.21.15.154:0/3770586883 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f928c06b000 0x7f928c06d4c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f929c007fd0 tx=0x7f929c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:53.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.844+0000 7f189d6c3640 1 -- 172.21.15.154:0/4161737363 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1878005500 con 0x7f189003a860 2023-09-23T03:15:53.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.845+0000 7f1894ff9640 1 -- 172.21.15.154:0/4161737363 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f188c054c00 con 0x7f189003a860 2023-09-23T03:15:53.846 INFO:teuthology.orchestra.run.smithi154.stdout:55834574859 2023-09-23T03:15:53.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.849+0000 7f18827fc640 1 -- 172.21.15.154:0/4161737363 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f188806b2e0 msgr2=0x7f188806d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.849+0000 7f18827fc640 1 --2- 172.21.15.154:0/4161737363 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f188806b2e0 0x7f188806d7a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1884001900 tx=0x7f1884004e50 comp rx=0 tx=0).stop 2023-09-23T03:15:53.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.849+0000 7f18827fc640 1 -- 172.21.15.154:0/4161737363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f189003a860 msgr2=0x7f18901990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.849+0000 7f18827fc640 1 --2- 172.21.15.154:0/4161737363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f189003a860 0x7f18901990f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f188c00d840 tx=0x7f188c00dd00 comp rx=0 tx=0).stop 2023-09-23T03:15:53.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.851+0000 7f18827fc640 1 -- 172.21.15.154:0/4161737363 shutdown_connections 2023-09-23T03:15:53.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.851+0000 7f18827fc640 1 --2- 172.21.15.154:0/4161737363 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f188806b2e0 0x7f188806d7a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.851+0000 7f18827fc640 1 --2- 172.21.15.154:0/4161737363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1890052fc0 0x7f18901998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.851+0000 7f18827fc640 1 --2- 172.21.15.154:0/4161737363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f189003a860 0x7f18901990f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.851+0000 7f18827fc640 1 --2- 172.21.15.154:0/4161737363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1890038170 0x7f189019b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.851+0000 7f18827fc640 1 -- 172.21.15.154:0/4161737363 >> 172.21.15.154:0/4161737363 conn(0x7f1890054c10 msgr2=0x7f1890102540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:53.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.852+0000 7f18827fc640 1 -- 172.21.15.154:0/4161737363 shutdown_connections 2023-09-23T03:15:53.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.852+0000 7f18827fc640 1 -- 172.21.15.154:0/4161737363 wait complete. 2023-09-23T03:15:53.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.853+0000 7f1dfca90640 1 -- 172.21.15.154:0/2846989260 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1df8073eb0 con 0x7f1df8156900 2023-09-23T03:15:53.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.854+0000 7f1de6ffd640 1 -- 172.21.15.154:0/2846989260 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1de8045090 con 0x7f1df8156900 2023-09-23T03:15:53.855 INFO:teuthology.orchestra.run.smithi154.stdout:55834574860 2023-09-23T03:15:53.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.858+0000 7f1de4ff9640 1 -- 172.21.15.154:0/2846989260 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1dd806b000 msgr2=0x7f1dd806d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.858+0000 7f1de4ff9640 1 --2- 172.21.15.154:0/2846989260 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1dd806b000 0x7f1dd806d4c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1df80c51b0 tx=0x7f1dec009060 comp rx=0 tx=0).stop 2023-09-23T03:15:53.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.859+0000 7f1de4ff9640 1 -- 172.21.15.154:0/2846989260 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1df8156900 msgr2=0x7f1df8163390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.859+0000 7f1de4ff9640 1 --2- 172.21.15.154:0/2846989260 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1df8156900 0x7f1df8163390 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1de8030000 tx=0x7f1de8004a10 comp rx=0 tx=0).stop 2023-09-23T03:15:53.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.859+0000 7f1de4ff9640 1 -- 172.21.15.154:0/2846989260 shutdown_connections 2023-09-23T03:15:53.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.859+0000 7f1de4ff9640 1 --2- 172.21.15.154:0/2846989260 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1dd806b000 0x7f1dd806d4c0 secure :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1df80c51b0 tx=0x7f1dec009060 comp rx=0 tx=0).stop 2023-09-23T03:15:53.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.859+0000 7f1de4ff9640 1 --2- 172.21.15.154:0/2846989260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1df8161da0 0x7f1df8162220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.860+0000 7f1de4ff9640 1 --2- 172.21.15.154:0/2846989260 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1df80c55e0 0x7f1df80c5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.860+0000 7f1de4ff9640 1 --2- 172.21.15.154:0/2846989260 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1df8156900 0x7f1df8163390 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.860+0000 7f1de4ff9640 1 -- 172.21.15.154:0/2846989260 >> 172.21.15.154:0/2846989260 conn(0x7f1df806a0f0 msgr2=0x7f1df8071c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:53.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.860+0000 7f1de4ff9640 1 -- 172.21.15.154:0/2846989260 shutdown_connections 2023-09-23T03:15:53.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.861+0000 7f1de4ff9640 1 -- 172.21.15.154:0/2846989260 wait complete. 2023-09-23T03:15:53.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.861+0000 7f97917fa640 1 -- 172.21.15.154:0/3872515652 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f97ac04e920 con 0x7f97ac15c840 2023-09-23T03:15:53.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.862+0000 7f97937fe640 1 -- 172.21.15.154:0/3872515652 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f97a4019020 con 0x7f97ac15c840 2023-09-23T03:15:53.864 INFO:teuthology.orchestra.run.smithi154.stdout:55834574858 2023-09-23T03:15:53.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.866+0000 7f97b0960640 1 -- 172.21.15.154:0/3872515652 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f979806af30 msgr2=0x7f979806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.867+0000 7f97b0960640 1 --2- 172.21.15.154:0/3872515652 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f979806af30 0x7f979806d3f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f979c036000 tx=0x7f979c008030 comp rx=0 tx=0).stop 2023-09-23T03:15:53.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.867+0000 7f97b0960640 1 -- 172.21.15.154:0/3872515652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97ac15c840 msgr2=0x7f97ac0c69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.867+0000 7f97b0960640 1 --2- 172.21.15.154:0/3872515652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97ac15c840 0x7f97ac0c69e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f97a400b9c0 tx=0x7f97a400be80 comp rx=0 tx=0).stop 2023-09-23T03:15:53.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.868+0000 7f97b0960640 1 -- 172.21.15.154:0/3872515652 shutdown_connections 2023-09-23T03:15:53.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.868+0000 7f97b0960640 1 --2- 172.21.15.154:0/3872515652 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f979806af30 0x7f979806d3f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.868+0000 7f97b0960640 1 --2- 172.21.15.154:0/3872515652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97ac160fd0 0x7f97ac0aeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.868+0000 7f97b0960640 1 --2- 172.21.15.154:0/3872515652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97ac15c840 0x7f97ac0c69e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.868+0000 7f97b0960640 1 --2- 172.21.15.154:0/3872515652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97ac0c99d0 0x7f97ac156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.868+0000 7f97b0960640 1 -- 172.21.15.154:0/3872515652 >> 172.21.15.154:0/3872515652 conn(0x7f97ac06a390 msgr2=0x7f97ac0cac00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:53.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.868+0000 7f97b0960640 1 -- 172.21.15.154:0/3872515652 shutdown_connections 2023-09-23T03:15:53.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.868+0000 7f97b0960640 1 -- 172.21.15.154:0/3872515652 wait complete. 2023-09-23T03:15:53.882 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574859 for osd.2 2023-09-23T03:15:53.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.880+0000 7ff6f8b25640 1 -- 172.21.15.154:0/3062981940 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7ff6ec03cdd0 con 0x7ff6ec04d250 2023-09-23T03:15:53.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.881+0000 7ff6e2ffd640 1 -- 172.21.15.154:0/3062981940 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff6e8057870 con 0x7ff6ec04d250 2023-09-23T03:15:53.883 INFO:teuthology.orchestra.run.smithi154.stdout:55834574859 2023-09-23T03:15:53.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.882+0000 7f54501e7640 1 -- 172.21.15.154:0/3844892727 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f54480b0690 con 0x7f54480c0b50 2023-09-23T03:15:53.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.882+0000 7f543e7fc640 1 -- 172.21.15.154:0/3844892727 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5440078160 con 0x7f54480c0b50 2023-09-23T03:15:53.884 INFO:teuthology.orchestra.run.smithi154.stdout:55834574858 2023-09-23T03:15:53.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.884+0000 7ff6f8b25640 1 -- 172.21.15.154:0/3062981940 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff6d406b000 msgr2=0x7ff6d406d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.884+0000 7ff6f8b25640 1 --2- 172.21.15.154:0/3062981940 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff6d406b000 0x7ff6d406d4c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4000fb0 tx=0x7ff6e4009030 comp rx=0 tx=0).stop 2023-09-23T03:15:53.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.884+0000 7ff6f8b25640 1 -- 172.21.15.154:0/3062981940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6ec04d250 msgr2=0x7ff6ec04f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.884+0000 7ff6f8b25640 1 --2- 172.21.15.154:0/3062981940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6ec04d250 0x7ff6ec04f6e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec054800 tx=0x7ff6e800c870 comp rx=0 tx=0).stop 2023-09-23T03:15:53.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.884+0000 7ff6f8b25640 1 -- 172.21.15.154:0/3062981940 shutdown_connections 2023-09-23T03:15:53.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.884+0000 7ff6f8b25640 1 --2- 172.21.15.154:0/3062981940 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff6d406b000 0x7ff6d406d4c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.884+0000 7ff6f8b25640 1 --2- 172.21.15.154:0/3062981940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6ec04fc20 0x7ff6ec0e7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.884+0000 7ff6f8b25640 1 --2- 172.21.15.154:0/3062981940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6ec04d250 0x7ff6ec04f6e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.884+0000 7ff6f8b25640 1 --2- 172.21.15.154:0/3062981940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6ec0e2d30 0x7ff6ec050110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.884+0000 7ff6f8b25640 1 -- 172.21.15.154:0/3062981940 >> 172.21.15.154:0/3062981940 conn(0x7ff6ec0538d0 msgr2=0x7ff6ec0e68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:53.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.885+0000 7ff6f8b25640 1 -- 172.21.15.154:0/3062981940 shutdown_connections 2023-09-23T03:15:53.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.885+0000 7ff6f8b25640 1 -- 172.21.15.154:0/3062981940 wait complete. 2023-09-23T03:15:53.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.886+0000 7f54501e7640 1 -- 172.21.15.154:0/3844892727 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f542c06afb0 msgr2=0x7f542c06d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.886+0000 7f54501e7640 1 --2- 172.21.15.154:0/3844892727 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f542c06afb0 0x7f542c06d470 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5444009c10 tx=0x7f5444008040 comp rx=0 tx=0).stop 2023-09-23T03:15:53.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.886+0000 7f54501e7640 1 -- 172.21.15.154:0/3844892727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54480c0b50 msgr2=0x7f544814e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.886+0000 7f54501e7640 1 --2- 172.21.15.154:0/3844892727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54480c0b50 0x7f544814e7e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5440036040 tx=0x7f5440004a40 comp rx=0 tx=0).stop 2023-09-23T03:15:53.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.886+0000 7f54501e7640 1 -- 172.21.15.154:0/3844892727 shutdown_connections 2023-09-23T03:15:53.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.886+0000 7f54501e7640 1 --2- 172.21.15.154:0/3844892727 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f542c06afb0 0x7f542c06d470 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.886+0000 7f54501e7640 1 --2- 172.21.15.154:0/3844892727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5448159370 0x7f544814ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.886+0000 7f54501e7640 1 --2- 172.21.15.154:0/3844892727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54480c0b50 0x7f544814e7e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.886+0000 7f54501e7640 1 --2- 172.21.15.154:0/3844892727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54480c06f0 0x7f54480b41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.886+0000 7f54501e7640 1 -- 172.21.15.154:0/3844892727 >> 172.21.15.154:0/3844892727 conn(0x7f54480b08e0 msgr2=0x7f544806ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:53.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.887+0000 7f54501e7640 1 -- 172.21.15.154:0/3844892727 shutdown_connections 2023-09-23T03:15:53.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.887+0000 7f54501e7640 1 -- 172.21.15.154:0/3844892727 wait complete. 2023-09-23T03:15:53.893 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574861 got 55834574860 for osd.1 2023-09-23T03:15:53.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.892+0000 7fd8066b4640 1 -- 172.21.15.154:0/2193159124 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd8000cfbd0 con 0x7fd8000cf2a0 2023-09-23T03:15:53.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.893+0000 7fd7fcff9640 1 -- 172.21.15.154:0/2193159124 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd7f407a6a0 con 0x7fd8000cf2a0 2023-09-23T03:15:53.894 INFO:teuthology.orchestra.run.smithi154.stdout:55834574860 2023-09-23T03:15:53.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.894+0000 7f92b3226640 1 -- 172.21.15.154:0/3770586883 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f92ac14c330 con 0x7f92ac0ccaf0 2023-09-23T03:15:53.896 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.3 2023-09-23T03:15:53.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.894+0000 7f92a97fa640 1 -- 172.21.15.154:0/3770586883 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f92ac14c330 con 0x7f92ac0ccaf0 2023-09-23T03:15:53.896 INFO:teuthology.orchestra.run.smithi154.stdout:55834574857 2023-09-23T03:15:53.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.896+0000 7fd8066b4640 1 -- 172.21.15.154:0/2193159124 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd7ec06b030 msgr2=0x7fd7ec06d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.896+0000 7fd8066b4640 1 --2- 172.21.15.154:0/2193159124 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd7ec06b030 0x7fd7ec06d4f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0004980 tx=0x7fd7f0009040 comp rx=0 tx=0).stop 2023-09-23T03:15:53.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.896+0000 7fd8066b4640 1 -- 172.21.15.154:0/2193159124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8000cf2a0 msgr2=0x7fd8000b4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.896+0000 7fd8066b4640 1 --2- 172.21.15.154:0/2193159124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8000cf2a0 0x7fd8000b4c60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd7f40098c0 tx=0x7fd7f4004730 comp rx=0 tx=0).stop 2023-09-23T03:15:53.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.897+0000 7fd8066b4640 1 -- 172.21.15.154:0/2193159124 shutdown_connections 2023-09-23T03:15:53.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.897+0000 7fd8066b4640 1 --2- 172.21.15.154:0/2193159124 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd7ec06b030 0x7fd7ec06d4f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.897+0000 7fd8066b4640 1 --2- 172.21.15.154:0/2193159124 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd80015cac0 0x7fd8000c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.897+0000 7fd8066b4640 1 --2- 172.21.15.154:0/2193159124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8000cf2a0 0x7fd8000b4c60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.897+0000 7fd8066b4640 1 --2- 172.21.15.154:0/2193159124 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8000cd670 0x7fd8000b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.897+0000 7fd8066b4640 1 -- 172.21.15.154:0/2193159124 >> 172.21.15.154:0/2193159124 conn(0x7fd800089fd0 msgr2=0x7fd8000b1120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:53.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.897+0000 7fd8066b4640 1 -- 172.21.15.154:0/2193159124 shutdown_connections 2023-09-23T03:15:53.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.897+0000 7fd8066b4640 1 -- 172.21.15.154:0/2193159124 wait complete. 2023-09-23T03:15:53.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.898+0000 7f92b3226640 1 -- 172.21.15.154:0/3770586883 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f928c06b000 msgr2=0x7f928c06d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.898+0000 7f92b3226640 1 --2- 172.21.15.154:0/3770586883 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f928c06b000 0x7f928c06d4c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f929c007fd0 tx=0x7f929c009040 comp rx=0 tx=0).stop 2023-09-23T03:15:53.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.898+0000 7f92b3226640 1 -- 172.21.15.154:0/3770586883 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92ac0ccaf0 msgr2=0x7f92ac0adf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.898+0000 7f92b3226640 1 --2- 172.21.15.154:0/3770586883 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92ac0ccaf0 0x7f92ac0adf70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f92a400b9b0 tx=0x7f92a400be70 comp rx=0 tx=0).stop 2023-09-23T03:15:53.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.899+0000 7f92b3226640 1 -- 172.21.15.154:0/3770586883 shutdown_connections 2023-09-23T03:15:53.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.899+0000 7f92b3226640 1 --2- 172.21.15.154:0/3770586883 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f928c06b000 0x7f928c06d4c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.899+0000 7f92b3226640 1 --2- 172.21.15.154:0/3770586883 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92ac0cf130 0x7f92ac0ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.899+0000 7f92b3226640 1 --2- 172.21.15.154:0/3770586883 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92ac0ccaf0 0x7f92ac0adf70 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.899+0000 7f92b3226640 1 --2- 172.21.15.154:0/3770586883 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92ac0b2fe0 0x7f92ac0ada30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.899+0000 7f92b3226640 1 -- 172.21.15.154:0/3770586883 >> 172.21.15.154:0/3770586883 conn(0x7f92ac06a0f0 msgr2=0x7f92ac0ab640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:53.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.899+0000 7f92b3226640 1 -- 172.21.15.154:0/3770586883 shutdown_connections 2023-09-23T03:15:53.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.899+0000 7f92b3226640 1 -- 172.21.15.154:0/3770586883 wait complete. 2023-09-23T03:15:53.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.905+0000 7f185b1c9640 1 -- 172.21.15.154:0/3537493835 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f185415dd90 con 0x7f1854160f90 2023-09-23T03:15:53.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.906+0000 7f18517fa640 1 -- 172.21.15.154:0/3537493835 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f185415dd90 con 0x7f1854160f90 2023-09-23T03:15:53.907 INFO:teuthology.orchestra.run.smithi154.stdout:55834574857 2023-09-23T03:15:53.909 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574859 for osd.5 2023-09-23T03:15:53.910 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.4 2023-09-23T03:15:53.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.909+0000 7f185b1c9640 1 -- 172.21.15.154:0/3537493835 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f183806af30 msgr2=0x7f183806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.909+0000 7f185b1c9640 1 --2- 172.21.15.154:0/3537493835 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f183806af30 0x7f183806d3f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f18400112c0 tx=0x7f1840009110 comp rx=0 tx=0).stop 2023-09-23T03:15:53.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.909+0000 7f185b1c9640 1 -- 172.21.15.154:0/3537493835 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1854160f90 msgr2=0x7f18540c3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:53.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.909+0000 7f185b1c9640 1 --2- 172.21.15.154:0/3537493835 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1854160f90 0x7f18540c3850 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f184c032000 tx=0x7f184c004a40 comp rx=0 tx=0).stop 2023-09-23T03:15:53.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.910+0000 7f185b1c9640 1 -- 172.21.15.154:0/3537493835 shutdown_connections 2023-09-23T03:15:53.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.910+0000 7f185b1c9640 1 --2- 172.21.15.154:0/3537493835 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f183806af30 0x7f183806d3f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.910+0000 7f185b1c9640 1 --2- 172.21.15.154:0/3537493835 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1854160f90 0x7f18540c3850 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.910+0000 7f185b1c9640 1 --2- 172.21.15.154:0/3537493835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f185415caf0 0x7f18540c3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.910+0000 7f185b1c9640 1 --2- 172.21.15.154:0/3537493835 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18540cee70 0x7f18541f54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:53.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.910+0000 7f185b1c9640 1 -- 172.21.15.154:0/3537493835 >> 172.21.15.154:0/3537493835 conn(0x7f185406a0f0 msgr2=0x7f18540b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:53.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.910+0000 7f185b1c9640 1 -- 172.21.15.154:0/3537493835 shutdown_connections 2023-09-23T03:15:53.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:53.910+0000 7f185b1c9640 1 -- 172.21.15.154:0/3537493835 wait complete. 2023-09-23T03:15:53.921 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.7 2023-09-23T03:15:53.921 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574861 got 55834574860 for osd.0 2023-09-23T03:15:53.928 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.6 2023-09-23T03:15:54.882 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-23T03:15:54.894 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-23T03:15:54.897 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-23T03:15:54.910 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-09-23T03:15:54.910 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-09-23T03:15:54.922 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-09-23T03:15:54.923 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-23T03:15:54.929 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-09-23T03:15:55.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.186+0000 7fd013abb640 1 Processor -- start 2023-09-23T03:15:55.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.186+0000 7fd013abb640 1 -- start start 2023-09-23T03:15:55.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.188+0000 7fd013abb640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd00c15c850 0x7fd00c0b7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.188+0000 7fd013abb640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd00c0b4050 0x7fd00c0b4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.188+0000 7fd013abb640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd00c0b49d0 con 0x7fd00c0d0710 2023-09-23T03:15:55.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.188+0000 7fd013abb640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd00c15cc30 con 0x7fd00c15c850 2023-09-23T03:15:55.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.188+0000 7fd013abb640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c0b6c50 con 0x7fd00c0b4050 2023-09-23T03:15:55.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.190+0000 7fd011056640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd00c0b4050 0x7fd00c0b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.190+0000 7fd011056640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd00c0b4050 0x7fd00c0b4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:34588/0 (socket says 172.21.15.154:34588) 2023-09-23T03:15:55.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.190+0000 7fd011056640 1 -- 172.21.15.154:0/509172433 learned_addr learned my addr 172.21.15.154:0/509172433 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:55.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.191+0000 7fd011056640 1 -- 172.21.15.154:0/509172433 >> v2:172.21.15.154:3300/0 conn(0x7fd00c15c850 msgr2=0x7fd00c0b7ea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T03:15:55.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.191+0000 7fd011056640 1 --2- 172.21.15.154:0/509172433 >> v2:172.21.15.154:3300/0 conn(0x7fd00c15c850 0x7fd00c0b7ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.191+0000 7fd011056640 1 -- 172.21.15.154:0/509172433 >> v1:172.21.15.154:6789/0 conn(0x7fd00c0d0710 legacy=0x7fd00c0d0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:55.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.192+0000 7fd011056640 1 -- 172.21.15.154:0/509172433 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd00c0b6f30 con 0x7fd00c0b4050 2023-09-23T03:15:55.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.195+0000 7fd011056640 1 --2- 172.21.15.154:0/509172433 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd00c0b4050 0x7fd00c0b4490 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd004009db0 tx=0x7fd00402c250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1219b296620dff9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:55.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.196+0000 7fd010855640 1 -- 172.21.15.154:0/509172433 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd004034440 con 0x7fd00c0b4050 2023-09-23T03:15:55.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.196+0000 7fd010855640 1 -- 172.21.15.154:0/509172433 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0040345a0 con 0x7fd00c0b4050 2023-09-23T03:15:55.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.198+0000 7fd013abb640 1 -- 172.21.15.154:0/509172433 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd00c0b4050 msgr2=0x7fd00c0b4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.198+0000 7fd013abb640 1 --2- 172.21.15.154:0/509172433 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd00c0b4050 0x7fd00c0b4490 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd004009db0 tx=0x7fd00402c250 comp rx=0 tx=0).stop 2023-09-23T03:15:55.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.198+0000 7fd013abb640 1 -- 172.21.15.154:0/509172433 shutdown_connections 2023-09-23T03:15:55.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.198+0000 7fd013abb640 1 --2- 172.21.15.154:0/509172433 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd00c0b4050 0x7fd00c0b4490 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.198+0000 7fd013abb640 1 --2- 172.21.15.154:0/509172433 >> v2:172.21.15.154:3300/0 conn(0x7fd00c15c850 0x7fd00c0b7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.198+0000 7fd013abb640 1 -- 172.21.15.154:0/509172433 >> 172.21.15.154:0/509172433 conn(0x7fd00c06b140 msgr2=0x7fd00c06b550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:55.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.199+0000 7fd013abb640 1 -- 172.21.15.154:0/509172433 shutdown_connections 2023-09-23T03:15:55.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.199+0000 7fd013abb640 1 -- 172.21.15.154:0/509172433 wait complete. 2023-09-23T03:15:55.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.202+0000 7fd013abb640 1 Processor -- start 2023-09-23T03:15:55.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.202+0000 7fd013abb640 1 -- start start 2023-09-23T03:15:55.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.204+0000 7fd013abb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd00c0d0710 0x7fd00c0cb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.205+0000 7fd013abb640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd00c15c850 0x7fd00c1527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.206+0000 7fd013abb640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd00c15e1f0 0x7fd00c0b3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.206+0000 7fd013abb640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c15abe0 con 0x7fd00c0d0710 2023-09-23T03:15:55.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.206+0000 7fd011056640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd00c15c850 0x7fd00c1527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.206+0000 7fd011056640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd00c15c850 0x7fd00c1527d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:34602/0 (socket says 172.21.15.154:34602) 2023-09-23T03:15:55.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.206+0000 7fd011056640 1 -- 172.21.15.154:0/2100529831 learned_addr learned my addr 172.21.15.154:0/2100529831 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:55.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.207+0000 7fd012058640 1 --2- 172.21.15.154:0/2100529831 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd00c15e1f0 0x7fd00c0b3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.207+0000 7fd013abb640 1 -- 172.21.15.154:0/2100529831 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c1f5590 con 0x7fd00c15e1f0 2023-09-23T03:15:55.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.206+0000 7fd011857640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd00c0d0710 0x7fd00c0cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.207+0000 7fd013abb640 1 -- 172.21.15.154:0/2100529831 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c1f5870 con 0x7fd00c15c850 2023-09-23T03:15:55.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.209+0000 7fd011056640 1 -- 172.21.15.154:0/2100529831 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd00c15e1f0 msgr2=0x7fd00c0b3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.209+0000 7fd011056640 1 --2- 172.21.15.154:0/2100529831 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd00c15e1f0 0x7fd00c0b3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.209+0000 7fd011056640 1 -- 172.21.15.154:0/2100529831 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd00c0d0710 msgr2=0x7fd00c0cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.209+0000 7fd011056640 1 --2- 172.21.15.154:0/2100529831 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd00c0d0710 0x7fd00c0cb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.209+0000 7fd011056640 1 -- 172.21.15.154:0/2100529831 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd004009a10 con 0x7fd00c15c850 2023-09-23T03:15:55.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.210+0000 7fd011857640 1 --2- 172.21.15.154:0/2100529831 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd00c0d0710 0x7fd00c0cb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:15:55.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.210+0000 7fd011056640 1 --2- 172.21.15.154:0/2100529831 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd00c15c850 0x7fd00c1527d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd00402c290 tx=0x7fd004002350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:55.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.211+0000 7fd0027fc640 1 -- 172.21.15.154:0/2100529831 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd004032980 con 0x7fd00c15c850 2023-09-23T03:15:55.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.211+0000 7fd013abb640 1 -- 172.21.15.154:0/2100529831 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd00c1f5b50 con 0x7fd00c15c850 2023-09-23T03:15:55.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.212+0000 7fd013abb640 1 -- 172.21.15.154:0/2100529831 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd00c0cb760 con 0x7fd00c15c850 2023-09-23T03:15:55.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.212+0000 7fd0027fc640 1 -- 172.21.15.154:0/2100529831 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd004032ae0 con 0x7fd00c15c850 2023-09-23T03:15:55.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.212+0000 7fd0027fc640 1 -- 172.21.15.154:0/2100529831 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd004032e50 con 0x7fd00c15c850 2023-09-23T03:15:55.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.215+0000 7fd013abb640 1 -- 172.21.15.154:0/2100529831 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcff4005250 con 0x7fd00c15c850 2023-09-23T03:15:55.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.219+0000 7fd0027fc640 1 -- 172.21.15.154:0/2100529831 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fd00403b050 con 0x7fd00c15c850 2023-09-23T03:15:55.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.221+0000 7fd0027fc640 1 --2- 172.21.15.154:0/2100529831 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcff006b030 0x7fcff006d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.221+0000 7fd0027fc640 1 -- 172.21.15.154:0/2100529831 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd0040ac790 con 0x7fd00c15c850 2023-09-23T03:15:55.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.222+0000 7fd011857640 1 --2- 172.21.15.154:0/2100529831 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcff006b030 0x7fcff006d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.223+0000 7fd011857640 1 --2- 172.21.15.154:0/2100529831 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcff006b030 0x7fcff006d4f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcffc009d50 tx=0x7fcffc009060 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:55.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.229+0000 7fd94d3c4640 1 Processor -- start 2023-09-23T03:15:55.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.229+0000 7fd94d3c4640 1 -- start start 2023-09-23T03:15:55.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.230+0000 7fd0027fc640 1 -- 172.21.15.154:0/2100529831 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fd004077600 con 0x7fd00c15c850 2023-09-23T03:15:55.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.231+0000 7fd94d3c4640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd940038180 0x7fd940053d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.231+0000 7fd94d3c4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9400542d0 0x7fd9400573e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.231+0000 7fd94d3c4640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd940006e80 con 0x7fd9400e3190 2023-09-23T03:15:55.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.231+0000 7fd94d3c4640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd94003f100 con 0x7fd940038180 2023-09-23T03:15:55.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.231+0000 7fd94d3c4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd940007c40 con 0x7fd9400542d0 2023-09-23T03:15:55.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.232+0000 7fd94cbc3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9400542d0 0x7fd9400573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.232+0000 7fd94cbc3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9400542d0 0x7fd9400573e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:34614/0 (socket says 172.21.15.154:34614) 2023-09-23T03:15:55.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.232+0000 7fd94cbc3640 1 -- 172.21.15.154:0/2741562486 learned_addr learned my addr 172.21.15.154:0/2741562486 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:55.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.233+0000 7fd94cbc3640 1 -- 172.21.15.154:0/2741562486 >> v2:172.21.15.190:3300/0 conn(0x7fd940038180 msgr2=0x7fd940053d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T03:15:55.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.233+0000 7fd94cbc3640 1 --2- 172.21.15.154:0/2741562486 >> v2:172.21.15.190:3300/0 conn(0x7fd940038180 0x7fd940053d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.233+0000 7fd94cbc3640 1 -- 172.21.15.154:0/2741562486 >> v1:172.21.15.154:6789/0 conn(0x7fd9400e3190 legacy=0x7fd94003e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:55.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.233+0000 7fd94cbc3640 1 -- 172.21.15.154:0/2741562486 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd940007160 con 0x7fd9400542d0 2023-09-23T03:15:55.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.236+0000 7fd94cbc3640 1 --2- 172.21.15.154:0/2741562486 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9400542d0 0x7fd9400573e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd94802cb00 tx=0x7fd9480a5630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=155a5b7ce8efc0b0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:55.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.237+0000 7fd946ffd640 1 -- 172.21.15.154:0/2741562486 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9480ac600 con 0x7fd9400542d0 2023-09-23T03:15:55.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.237+0000 7fd946ffd640 1 -- 172.21.15.154:0/2741562486 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9480a8050 con 0x7fd9400542d0 2023-09-23T03:15:55.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.237+0000 7f8e42f33640 1 Processor -- start 2023-09-23T03:15:55.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.238+0000 7fd94d3c4640 1 -- 172.21.15.154:0/2741562486 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9400542d0 msgr2=0x7fd9400573e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.238+0000 7fd94d3c4640 1 --2- 172.21.15.154:0/2741562486 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9400542d0 0x7fd9400573e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd94802cb00 tx=0x7fd9480a5630 comp rx=0 tx=0).stop 2023-09-23T03:15:55.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.238+0000 7fd94d3c4640 1 -- 172.21.15.154:0/2741562486 shutdown_connections 2023-09-23T03:15:55.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.238+0000 7fd94d3c4640 1 --2- 172.21.15.154:0/2741562486 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9400542d0 0x7fd9400573e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.238+0000 7fd94d3c4640 1 --2- 172.21.15.154:0/2741562486 >> v2:172.21.15.190:3300/0 conn(0x7fd940038180 0x7fd940053d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.238+0000 7f8e42f33640 1 -- start start 2023-09-23T03:15:55.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.238+0000 7fd94d3c4640 1 -- 172.21.15.154:0/2741562486 >> 172.21.15.154:0/2741562486 conn(0x7fd9400590c0 msgr2=0x7fd9400594d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:55.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.239+0000 7fd94d3c4640 1 -- 172.21.15.154:0/2741562486 shutdown_connections 2023-09-23T03:15:55.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.239+0000 7fd94d3c4640 1 -- 172.21.15.154:0/2741562486 wait complete. 2023-09-23T03:15:55.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.240+0000 7f11b982e640 1 Processor -- start 2023-09-23T03:15:55.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.241+0000 7f8e42f33640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e3c15ca80 0x7f8e3c160680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.242+0000 7f8e42f33640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e3c161500 con 0x7f8e3c0d25b0 2023-09-23T03:15:55.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.242+0000 7f8e42f33640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e3c160c60 con 0x7f8e3c0b2620 2023-09-23T03:15:55.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.242+0000 7f8e42f33640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c0b2aa0 con 0x7f8e3c15ca80 2023-09-23T03:15:55.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.242+0000 7f8e40ccf640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e3c15ca80 0x7f8e3c160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.242+0000 7fd94d3c4640 1 Processor -- start 2023-09-23T03:15:55.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.242+0000 7f8e3bfff640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f8e3c0b2620 0x7f8e3c0d07b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.154:46648/0 (socket says 172.21.15.154:46648) 2023-09-23T03:15:55.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.240+0000 7f11b982e640 1 -- start start 2023-09-23T03:15:55.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.242+0000 7f11b982e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f11b40cf500 0x7f11b40ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.242+0000 7f11b982e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11b406be10 con 0x7f11b415cc00 2023-09-23T03:15:55.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.242+0000 7f11b982e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11b40d17f0 con 0x7f11b40cf8e0 2023-09-23T03:15:55.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.242+0000 7f11b982e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11b40d0470 con 0x7f11b40cf500 2023-09-23T03:15:55.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.243+0000 7f11b902d640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f11b415cc00 0x7f11b40b1430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46256/0 (socket says 172.21.15.154:46256) 2023-09-23T03:15:55.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.243+0000 7f11b902d640 1 -- 172.21.15.154:0/3667451426 learned_addr learned my addr 172.21.15.154:0/3667451426 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:55.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.243+0000 7f11b882c640 1 --2- 172.21.15.154:0/3667451426 >> v2:172.21.15.154:3300/0 conn(0x7f11b40cf500 0x7f11b40ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.243+0000 7fd94d3c4640 1 -- start start 2023-09-23T03:15:55.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.244+0000 7fd94d3c4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd940038180 0x7fd94004e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.245+0000 7fd94d3c4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9400542d0 0x7fd940009e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.245+0000 7fd947fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd940038180 0x7fd94004e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.245+0000 7fd947fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd940038180 0x7fd94004e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44516/0 (socket says 172.21.15.154:44516) 2023-09-23T03:15:55.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.245+0000 7fd947fff640 1 -- 172.21.15.154:0/3284415149 learned_addr learned my addr 172.21.15.154:0/3284415149 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:55.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.245+0000 7fd94d3c4640 1 --2- 172.21.15.154:0/3284415149 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9400e3190 0x7fd94000a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.245+0000 7f11ab7fe640 1 -- 172.21.15.154:0/3667451426 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1267366387 0 0) 0x7f11b40d17f0 con 0x7f11b40cf8e0 2023-09-23T03:15:55.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.245+0000 7f11ab7fe640 1 -- 172.21.15.154:0/3667451426 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f119c003520 con 0x7f11b40cf8e0 2023-09-23T03:15:55.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.245+0000 7fd94d3c4640 1 -- 172.21.15.154:0/3284415149 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94000d920 con 0x7fd940038180 2023-09-23T03:15:55.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.245+0000 7fd94d3c4640 1 -- 172.21.15.154:0/3284415149 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94000cd10 con 0x7fd9400542d0 2023-09-23T03:15:55.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.245+0000 7fd94d3c4640 1 -- 172.21.15.154:0/3284415149 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd94000cff0 con 0x7fd9400e3190 2023-09-23T03:15:55.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.245+0000 7fd9477fe640 1 --2- 172.21.15.154:0/3284415149 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9400542d0 0x7fd940009e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.242+0000 7f8e3bfff640 1 -- 172.21.15.154:0/1682557637 learned_addr learned my addr 172.21.15.154:0/1682557637 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:55.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.245+0000 7f11ab7fe640 1 -- 172.21.15.154:0/3667451426 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1262313746 0 0) 0x7f11b406be10 con 0x7f11b415cc00 2023-09-23T03:15:55.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.245+0000 7f8e3b7fe640 1 -- 172.21.15.154:0/1682557637 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1822566215 0 0) 0x7f8e3c160c60 con 0x7f8e3c0b2620 2023-09-23T03:15:55.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.245+0000 7f11b882c640 1 -- 172.21.15.154:0/3667451426 >> v1:172.21.15.190:6789/0 conn(0x7f11b40cf8e0 legacy=0x7f11b415c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.246+0000 7f8e3b7fe640 1 -- 172.21.15.154:0/1682557637 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e2c003520 con 0x7f8e3c0b2620 2023-09-23T03:15:55.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.246+0000 7f11b882c640 1 -- 172.21.15.154:0/3667451426 >> v1:172.21.15.154:6789/0 conn(0x7f11b415cc00 legacy=0x7f11b40b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.246+0000 7f8e3b7fe640 1 -- 172.21.15.154:0/1682557637 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1136260207 0 0) 0x7f8e3c161500 con 0x7f8e3c0d25b0 2023-09-23T03:15:55.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.246+0000 7f8e3b7fe640 1 -- 172.21.15.154:0/1682557637 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e3c160c60 con 0x7f8e3c0d25b0 2023-09-23T03:15:55.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.246+0000 7f11b882c640 1 -- 172.21.15.154:0/3667451426 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b40d0750 con 0x7f11b40cf500 2023-09-23T03:15:55.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.246+0000 7f8e40ccf640 1 -- 172.21.15.154:0/1682557637 >> v1:172.21.15.190:6789/0 conn(0x7f8e3c0b2620 legacy=0x7f8e3c0d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.246+0000 7f8e40ccf640 1 -- 172.21.15.154:0/1682557637 >> v1:172.21.15.154:6789/0 conn(0x7f8e3c0d25b0 legacy=0x7f8e3c0d2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.247+0000 7fd947fff640 1 -- 172.21.15.154:0/3284415149 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9400e3190 msgr2=0x7fd94000a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.247+0000 7fd947fff640 1 --2- 172.21.15.154:0/3284415149 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9400e3190 0x7fd94000a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.247+0000 7fd947fff640 1 -- 172.21.15.154:0/3284415149 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9400542d0 msgr2=0x7fd940009e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.247+0000 7fd947fff640 1 --2- 172.21.15.154:0/3284415149 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9400542d0 0x7fd940009e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.247+0000 7fd947fff640 1 -- 172.21.15.154:0/3284415149 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd94806ce70 con 0x7fd940038180 2023-09-23T03:15:55.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.247+0000 7f8e40ccf640 1 -- 172.21.15.154:0/1682557637 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e3c160f40 con 0x7f8e3c15ca80 2023-09-23T03:15:55.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.247+0000 7fd947fff640 1 --2- 172.21.15.154:0/3284415149 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd940038180 0x7fd94004e050 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd93c00b3e0 tx=0x7fd93c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:55.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.248+0000 7fd944ff9640 1 -- 172.21.15.154:0/3284415149 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c011830 con 0x7fd940038180 2023-09-23T03:15:55.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.248+0000 7f11b882c640 1 --2- 172.21.15.154:0/3667451426 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11b40cf500 0x7f11b40ce050 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f11b00097c0 tx=0x7f11b002b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e093fc90a508abc4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:55.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.248+0000 7f8e40ccf640 1 --2- 172.21.15.154:0/1682557637 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e3c15ca80 0x7f8e3c160680 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8e280097c0 tx=0x7f8e2802b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=100c793b05a48e2b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:55.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.249+0000 7f11ab7fe640 1 -- 172.21.15.154:0/3667451426 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b0031800 con 0x7f11b40cf500 2023-09-23T03:15:55.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.249+0000 7f11ab7fe640 1 -- 172.21.15.154:0/3667451426 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b0031960 con 0x7f11b40cf500 2023-09-23T03:15:55.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.250+0000 7f11ab7fe640 1 -- 172.21.15.154:0/3667451426 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b0031cf0 con 0x7f11b40cf500 2023-09-23T03:15:55.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.250+0000 7f8e3b7fe640 1 -- 172.21.15.154:0/1682557637 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e28031800 con 0x7f8e3c15ca80 2023-09-23T03:15:55.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.250+0000 7f8e3b7fe640 1 -- 172.21.15.154:0/1682557637 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e28031960 con 0x7f8e3c15ca80 2023-09-23T03:15:55.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.250+0000 7f8e3b7fe640 1 -- 172.21.15.154:0/1682557637 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e28031cf0 con 0x7f8e3c15ca80 2023-09-23T03:15:55.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.251+0000 7f8e42f33640 1 -- 172.21.15.154:0/1682557637 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e3c15ca80 msgr2=0x7f8e3c160680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.249+0000 7fd94d3c4640 1 -- 172.21.15.154:0/3284415149 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd94000d2d0 con 0x7fd940038180 2023-09-23T03:15:55.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.249+0000 7fd94d3c4640 1 -- 172.21.15.154:0/3284415149 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd94004e590 con 0x7fd940038180 2023-09-23T03:15:55.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.250+0000 7fd944ff9640 1 -- 172.21.15.154:0/3284415149 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93c0119b0 con 0x7fd940038180 2023-09-23T03:15:55.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.250+0000 7fd944ff9640 1 -- 172.21.15.154:0/3284415149 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c011d60 con 0x7fd940038180 2023-09-23T03:15:55.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.251+0000 7f8e42f33640 1 --2- 172.21.15.154:0/1682557637 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e3c15ca80 0x7f8e3c160680 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8e280097c0 tx=0x7f8e2802b8a0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.252+0000 7f8e42f33640 1 -- 172.21.15.154:0/1682557637 shutdown_connections 2023-09-23T03:15:55.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.252+0000 7f8e42f33640 1 --2- 172.21.15.154:0/1682557637 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e3c15ca80 0x7f8e3c160680 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.252+0000 7f11b982e640 1 -- 172.21.15.154:0/3667451426 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11b40cf500 msgr2=0x7f11b40ce050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.252+0000 7f11b982e640 1 --2- 172.21.15.154:0/3667451426 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11b40cf500 0x7f11b40ce050 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f11b00097c0 tx=0x7f11b002b8a0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.252+0000 7f8e42f33640 1 -- 172.21.15.154:0/1682557637 >> 172.21.15.154:0/1682557637 conn(0x7f8e3c09bc60 msgr2=0x7f8e3c089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:55.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.253+0000 7f8e42f33640 1 -- 172.21.15.154:0/1682557637 shutdown_connections 2023-09-23T03:15:55.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.253+0000 7fd944ff9640 1 -- 172.21.15.154:0/3284415149 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fd93c01a400 con 0x7fd940038180 2023-09-23T03:15:55.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.253+0000 7fd94d3c4640 1 -- 172.21.15.154:0/3284415149 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd928005250 con 0x7fd940038180 2023-09-23T03:15:55.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.253+0000 7f8e42f33640 1 -- 172.21.15.154:0/1682557637 wait complete. 2023-09-23T03:15:55.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.256+0000 7f8e42f33640 1 Processor -- start 2023-09-23T03:15:55.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.257+0000 7f8e42f33640 1 -- start start 2023-09-23T03:15:55.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.257+0000 7f11b982e640 1 -- 172.21.15.154:0/3667451426 shutdown_connections 2023-09-23T03:15:55.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.257+0000 7f11b982e640 1 --2- 172.21.15.154:0/3667451426 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11b40cf500 0x7f11b40ce050 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.257+0000 7f11b982e640 1 -- 172.21.15.154:0/3667451426 >> 172.21.15.154:0/3667451426 conn(0x7f11b409bbb0 msgr2=0x7f11b40d28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:55.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.257+0000 7f8090fd7640 1 Processor -- start 2023-09-23T03:15:55.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.255+0000 7fd944ff9640 1 --2- 172.21.15.154:0/3284415149 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd92c06b000 0x7fd92c06d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.255+0000 7fd944ff9640 1 -- 172.21.15.154:0/3284415149 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd93c089b00 con 0x7fd940038180 2023-09-23T03:15:55.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.256+0000 7fd9477fe640 1 --2- 172.21.15.154:0/3284415149 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd92c06b000 0x7fd92c06d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.257+0000 7fd9477fe640 1 --2- 172.21.15.154:0/3284415149 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd92c06b000 0x7fd92c06d4c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd94017bd40 tx=0x7fd934008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:55.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.257+0000 7f8090fd7640 1 -- start start 2023-09-23T03:15:55.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.258+0000 7f11b982e640 1 -- 172.21.15.154:0/3667451426 shutdown_connections 2023-09-23T03:15:55.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.258+0000 7f11b982e640 1 -- 172.21.15.154:0/3667451426 wait complete. 2023-09-23T03:15:55.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.258+0000 7f8e42f33640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e3c0b2620 0x7f8e3c155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.258+0000 7f8e42f33640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e3c0d25b0 0x7f8e3c1e8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.258+0000 7f8e42f33640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e3c15ca80 0x7f8e3c15a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.259+0000 7f8e414d0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e3c15ca80 0x7f8e3c15a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.259+0000 7f8e414d0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e3c15ca80 0x7f8e3c15a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44520/0 (socket says 172.21.15.154:44520) 2023-09-23T03:15:55.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.259+0000 7f8e414d0640 1 -- 172.21.15.154:0/1800931871 learned_addr learned my addr 172.21.15.154:0/1800931871 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:55.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.259+0000 7f8090fd7640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f808c15ca80 0x7f808c160680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.259+0000 7f8090fd7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f808c0d25b0 0x7f808c0d2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.259+0000 7f8090fd7640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f808c161500 con 0x7f808c0b2620 2023-09-23T03:15:55.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.259+0000 7f8090fd7640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f808c0b2ad0 con 0x7f808c15ca80 2023-09-23T03:15:55.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.259+0000 7f8090fd7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f808c160c60 con 0x7f808c0d25b0 2023-09-23T03:15:55.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.259+0000 7f8e42f33640 1 -- 172.21.15.154:0/1800931871 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c155ad0 con 0x7f8e3c15ca80 2023-09-23T03:15:55.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.259+0000 7f8e42f33640 1 -- 172.21.15.154:0/1800931871 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c0c7f00 con 0x7f8e3c0d25b0 2023-09-23T03:15:55.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.259+0000 7f8e42f33640 1 -- 172.21.15.154:0/1800931871 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c0b5970 con 0x7f8e3c0b2620 2023-09-23T03:15:55.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.260+0000 7f8e414d0640 1 -- 172.21.15.154:0/1800931871 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e3c0b2620 msgr2=0x7f8e3c155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.260+0000 7f8e3bfff640 1 --2- 172.21.15.154:0/1800931871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e3c0d25b0 0x7f8e3c1e8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.260+0000 7f8089d9b640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f808c0b2620 0x7f808c0d07b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46260/0 (socket says 172.21.15.154:46260) 2023-09-23T03:15:55.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.260+0000 7f8089d9b640 1 -- 172.21.15.154:0/2850609295 learned_addr learned my addr 172.21.15.154:0/2850609295 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:55.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.259+0000 7f8e40ccf640 1 --2- 172.21.15.154:0/1800931871 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e3c0b2620 0x7f8e3c155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.260+0000 7f8e414d0640 1 --2- 172.21.15.154:0/1800931871 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e3c0b2620 0x7f8e3c155590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.260+0000 7f8e414d0640 1 -- 172.21.15.154:0/1800931871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e3c0d25b0 msgr2=0x7f8e3c1e8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.260+0000 7f11b982e640 1 Processor -- start 2023-09-23T03:15:55.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.261+0000 7f808ad9d640 1 --2- 172.21.15.154:0/2850609295 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f808c0d25b0 0x7f808c0d2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.260+0000 7f8e414d0640 1 --2- 172.21.15.154:0/1800931871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e3c0d25b0 0x7f8e3c1e8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.260+0000 7f8e414d0640 1 -- 172.21.15.154:0/1800931871 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e280093a0 con 0x7f8e3c15ca80 2023-09-23T03:15:55.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.260+0000 7f8e40ccf640 1 --2- 172.21.15.154:0/1800931871 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e3c0b2620 0x7f8e3c155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:55.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.261+0000 7f8e414d0640 1 --2- 172.21.15.154:0/1800931871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e3c15ca80 0x7f8e3c15a980 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400caf0 tx=0x7f8e34002130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:55.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.261+0000 7f11b982e640 1 -- start start 2023-09-23T03:15:55.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.261+0000 7f808a59c640 1 --2- 172.21.15.154:0/2850609295 >> v2:172.21.15.190:3300/0 conn(0x7f808c15ca80 0x7f808c160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.262+0000 7fb15495e640 1 Processor -- start 2023-09-23T03:15:55.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.262+0000 7f11b982e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11b40cf500 0x7f11b40c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.262+0000 7f808ad9d640 1 -- 172.21.15.154:0/2850609295 >> v2:172.21.15.190:3300/0 conn(0x7f808c15ca80 msgr2=0x7f808c160680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:55.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.262+0000 7f808ad9d640 1 --2- 172.21.15.154:0/2850609295 >> v2:172.21.15.190:3300/0 conn(0x7f808c15ca80 0x7f808c160680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.262+0000 7f808ad9d640 1 -- 172.21.15.154:0/2850609295 >> v1:172.21.15.154:6789/0 conn(0x7f808c0b2620 legacy=0x7f808c0d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.262+0000 7f808ad9d640 1 -- 172.21.15.154:0/2850609295 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f808c160f40 con 0x7f808c0d25b0 2023-09-23T03:15:55.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.263+0000 7f808a59c640 1 --2- 172.21.15.154:0/2850609295 >> v2:172.21.15.190:3300/0 conn(0x7f808c15ca80 0x7f808c160680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:15:55.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.263+0000 7f11b982e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11b40cf8e0 0x7f11b40ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.263+0000 7f11b882c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11b40cf500 0x7f11b40c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.263+0000 7f11b882c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11b40cf500 0x7f11b40c8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:34658/0 (socket says 172.21.15.154:34658) 2023-09-23T03:15:55.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.263+0000 7f11abfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11b40cf8e0 0x7f11b40ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.263+0000 7f11abfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11b40cf8e0 0x7f11b40ca4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44522/0 (socket says 172.21.15.154:44522) 2023-09-23T03:15:55.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.264+0000 7f11b882c640 1 -- 172.21.15.154:0/1811900949 learned_addr learned my addr 172.21.15.154:0/1811900949 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:55.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.264+0000 7f11b982e640 1 --2- 172.21.15.154:0/1811900949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11b415cc00 0x7f11b415ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.264+0000 7f11b982e640 1 -- 172.21.15.154:0/1811900949 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b40c7c30 con 0x7f11b40cf8e0 2023-09-23T03:15:55.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.262+0000 7fb15495e640 1 -- start start 2023-09-23T03:15:55.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.264+0000 7f11b982e640 1 -- 172.21.15.154:0/1811900949 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b40c7f10 con 0x7f11b415cc00 2023-09-23T03:15:55.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.264+0000 7f11b982e640 1 -- 172.21.15.154:0/1811900949 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11b40c81f0 con 0x7f11b40cf500 2023-09-23T03:15:55.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.263+0000 7f944da26640 1 Processor -- start 2023-09-23T03:15:55.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.264+0000 7f944da26640 1 -- start start 2023-09-23T03:15:55.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.264+0000 7fb15495e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1500712b0 0x7fb1500cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.264+0000 7fb15495e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb1500ccb70 0x7fb1500b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.264+0000 7fb15495e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1500c9d00 con 0x7fb150158ba0 2023-09-23T03:15:55.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.264+0000 7fb15495e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1500c9fe0 con 0x7fb1500ccb70 2023-09-23T03:15:55.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.264+0000 7fb15495e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1500ca2c0 con 0x7fb1500712b0 2023-09-23T03:15:55.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.265+0000 7fb14ed9d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb1500ccb70 0x7fb1500b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.265+0000 7fb14ed9d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb1500ccb70 0x7fb1500b11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.154:57792/0 (socket says 172.21.15.154:57792) 2023-09-23T03:15:55.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.265+0000 7fb14ed9d640 1 -- 172.21.15.154:0/1463032401 learned_addr learned my addr 172.21.15.154:0/1463032401 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:55.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.265+0000 7fd944ff9640 1 -- 172.21.15.154:0/3284415149 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fd93c0548c0 con 0x7fd940038180 2023-09-23T03:15:55.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.265+0000 7f944da26640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f944815ea40 0x7f94481620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.266+0000 7fb147fff640 1 --2- 172.21.15.154:0/1463032401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1500712b0 0x7fb1500cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.265+0000 7f8e397fa640 1 -- 172.21.15.154:0/1800931871 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e34003b70 con 0x7f8e3c15ca80 2023-09-23T03:15:55.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.265+0000 7f8e397fa640 1 -- 172.21.15.154:0/1800931871 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e34003cd0 con 0x7f8e3c15ca80 2023-09-23T03:15:55.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.265+0000 7f8e397fa640 1 -- 172.21.15.154:0/1800931871 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3400a980 con 0x7f8e3c15ca80 2023-09-23T03:15:55.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.264+0000 7f11b902d640 1 --2- 172.21.15.154:0/1811900949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11b415cc00 0x7f11b415ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.265+0000 7f808ad9d640 1 --2- 172.21.15.154:0/2850609295 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f808c0d25b0 0x7f808c0d2a10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f80740097a0 tx=0x7f807402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9851851bbeecfc9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:55.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.265+0000 7f11abfff640 1 -- 172.21.15.154:0/1811900949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11b40cf500 msgr2=0x7f11b40c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.266+0000 7f944da26640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94480ccfa0 0x7f94480cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.265+0000 7f11abfff640 1 --2- 172.21.15.154:0/1811900949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11b40cf500 0x7f11b40c8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.266+0000 7f9446ffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f944815ea40 0x7f94481620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.266+0000 7f9446ffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f944815ea40 0x7f94481620d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44524/0 (socket says 172.21.15.154:44524) 2023-09-23T03:15:55.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.266+0000 7f9446ffd640 1 -- 172.21.15.154:0/1824766305 learned_addr learned my addr 172.21.15.154:0/1824766305 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:55.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.266+0000 7fb14ed9d640 1 -- 172.21.15.154:0/1463032401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1500712b0 msgr2=0x7fb1500cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:55.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.266+0000 7fb14ed9d640 1 --2- 172.21.15.154:0/1463032401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1500712b0 0x7fb1500cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.266+0000 7fb14ed9d640 1 -- 172.21.15.154:0/1463032401 >> v1:172.21.15.154:6789/0 conn(0x7fb150158ba0 legacy=0x7fb1500cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:55.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.266+0000 7fb14ed9d640 1 -- 172.21.15.154:0/1463032401 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1500ca5a0 con 0x7fb1500ccb70 2023-09-23T03:15:55.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.267+0000 7f944da26640 1 -- 172.21.15.154:0/1824766305 --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94480cd8f0 con 0x7f9448165810 2023-09-23T03:15:55.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.267+0000 7f944da26640 1 -- 172.21.15.154:0/1824766305 --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94480cf660 con 0x7f944815ea40 2023-09-23T03:15:55.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.267+0000 7f944da26640 1 -- 172.21.15.154:0/1824766305 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94480cf930 con 0x7f94480ccfa0 2023-09-23T03:15:55.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.267+0000 7f8e42f33640 1 -- 172.21.15.154:0/1800931871 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e3c0b5670 con 0x7f8e3c15ca80 2023-09-23T03:15:55.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.267+0000 7f8e42f33640 1 -- 172.21.15.154:0/1800931871 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e3c0c8430 con 0x7f8e3c15ca80 2023-09-23T03:15:55.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.267+0000 7f94477fe640 1 --2- 172.21.15.154:0/1824766305 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94480ccfa0 0x7f94480cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.265+0000 7f11abfff640 1 -- 172.21.15.154:0/1811900949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11b415cc00 msgr2=0x7f11b415ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.265+0000 7f11abfff640 1 --2- 172.21.15.154:0/1811900949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11b415cc00 0x7f11b415ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.265+0000 7f11abfff640 1 -- 172.21.15.154:0/1811900949 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b00093a0 con 0x7f11b40cf8e0 2023-09-23T03:15:55.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.266+0000 7f11abfff640 1 --2- 172.21.15.154:0/1811900949 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11b40cf8e0 0x7f11b40ca4f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f11a400c370 tx=0x7f11a400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:55.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.267+0000 7f11a97fa640 1 -- 172.21.15.154:0/1811900949 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a40052d0 con 0x7f11b40cf8e0 2023-09-23T03:15:55.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.267+0000 7fb147fff640 1 --2- 172.21.15.154:0/1463032401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1500712b0 0x7fb1500cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:55.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.268+0000 7f11b982e640 1 -- 172.21.15.154:0/1811900949 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11b40c84d0 con 0x7f11b40cf8e0 2023-09-23T03:15:55.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.267+0000 7f808959a640 1 -- 172.21.15.154:0/2850609295 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807402d040 con 0x7f808c0d25b0 2023-09-23T03:15:55.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.268+0000 7f808959a640 1 -- 172.21.15.154:0/2850609295 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8074031810 con 0x7f808c0d25b0 2023-09-23T03:15:55.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.268+0000 7f11b982e640 1 -- 172.21.15.154:0/1811900949 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11b40c9210 con 0x7f11b40cf8e0 2023-09-23T03:15:55.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.268+0000 7f8090fd7640 1 -- 172.21.15.154:0/2850609295 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f808c0d25b0 msgr2=0x7f808c0d2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.269+0000 7f8090fd7640 1 --2- 172.21.15.154:0/2850609295 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f808c0d25b0 0x7f808c0d2a10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f80740097a0 tx=0x7f807402b8b0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.269+0000 7fb14ed9d640 1 --2- 172.21.15.154:0/1463032401 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1500ccb70 0x7fb1500b11a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb148009bd0 tx=0x7fb14802e500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2de65396219ed5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:55.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.268+0000 7f94477fe640 1 -- 172.21.15.154:0/1824766305 >> v2:172.21.15.154:3300/0 conn(0x7f944815ea40 msgr2=0x7f94481620d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:55.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.268+0000 7f94477fe640 1 --2- 172.21.15.154:0/1824766305 >> v2:172.21.15.154:3300/0 conn(0x7f944815ea40 0x7f94481620d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.268+0000 7f94477fe640 1 -- 172.21.15.154:0/1824766305 >> v1:172.21.15.190:6789/0 conn(0x7f9448165810 legacy=0x7f94480cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.268+0000 7f94477fe640 1 -- 172.21.15.154:0/1824766305 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94480cfc10 con 0x7f94480ccfa0 2023-09-23T03:15:55.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.269+0000 7f82d79c9640 1 Processor -- start 2023-09-23T03:15:55.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.269+0000 7f11a97fa640 1 -- 172.21.15.154:0/1811900949 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a4005430 con 0x7f11b40cf8e0 2023-09-23T03:15:55.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.269+0000 7f8e42f33640 1 -- 172.21.15.154:0/1800931871 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e3c0b5ae0 con 0x7f8e3c15ca80 2023-09-23T03:15:55.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.270+0000 7f11a97fa640 1 -- 172.21.15.154:0/1811900949 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a4003830 con 0x7f11b40cf8e0 2023-09-23T03:15:55.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.270+0000 7f82d79c9640 1 -- start start 2023-09-23T03:15:55.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.270+0000 7f8090fd7640 1 -- 172.21.15.154:0/2850609295 shutdown_connections 2023-09-23T03:15:55.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.270+0000 7f8090fd7640 1 --2- 172.21.15.154:0/2850609295 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f808c0d25b0 0x7f808c0d2a10 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.270+0000 7f8090fd7640 1 --2- 172.21.15.154:0/2850609295 >> v2:172.21.15.190:3300/0 conn(0x7f808c15ca80 0x7f808c160680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.270+0000 7f8090fd7640 1 -- 172.21.15.154:0/2850609295 >> 172.21.15.154:0/2850609295 conn(0x7f808c09bc60 msgr2=0x7f808c089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:55.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.270+0000 7fb14dd9b640 1 -- 172.21.15.154:0/1463032401 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb148035600 con 0x7fb1500ccb70 2023-09-23T03:15:55.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.270+0000 7fb14dd9b640 1 -- 172.21.15.154:0/1463032401 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb148031050 con 0x7fb1500ccb70 2023-09-23T03:15:55.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.270+0000 7fb14dd9b640 1 -- 172.21.15.154:0/1463032401 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1480359c0 con 0x7fb1500ccb70 2023-09-23T03:15:55.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.270+0000 7f8090fd7640 1 -- 172.21.15.154:0/2850609295 shutdown_connections 2023-09-23T03:15:55.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.270+0000 7f119affd640 1 -- 172.21.15.154:0/1811900949 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11b40c89f0 con 0x7f11b40cf8e0 2023-09-23T03:15:55.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.270+0000 7f94477fe640 1 --2- 172.21.15.154:0/1824766305 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94480ccfa0 0x7f94480cd3b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9440009d90 tx=0x7f944002bec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e282e6cc75f51718 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:55.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.270+0000 7f8090fd7640 1 -- 172.21.15.154:0/2850609295 wait complete. 2023-09-23T03:15:55.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.271+0000 7fb15495e640 1 -- 172.21.15.154:0/1463032401 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1500ccb70 msgr2=0x7fb1500b11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.271+0000 7fb15495e640 1 --2- 172.21.15.154:0/1463032401 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1500ccb70 0x7fb1500b11a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb148009bd0 tx=0x7fb14802e500 comp rx=0 tx=0).stop 2023-09-23T03:15:55.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.272+0000 7fb15495e640 1 -- 172.21.15.154:0/1463032401 shutdown_connections 2023-09-23T03:15:55.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.272+0000 7fb15495e640 1 --2- 172.21.15.154:0/1463032401 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1500712b0 0x7fb1500cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.272+0000 7fb15495e640 1 --2- 172.21.15.154:0/1463032401 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1500ccb70 0x7fb1500b11a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.272+0000 7fb15495e640 1 -- 172.21.15.154:0/1463032401 >> 172.21.15.154:0/1463032401 conn(0x7fb1500afe40 msgr2=0x7fb1500b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:55.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.272+0000 7f82d79c9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f82d00cf2a0 0x7f82d00cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.273+0000 7f8090fd7640 1 Processor -- start 2023-09-23T03:15:55.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.273+0000 7f11a97fa640 1 -- 172.21.15.154:0/1811900949 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f11a4003990 con 0x7f11b40cf8e0 2023-09-23T03:15:55.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.272+0000 7f9445ffb640 1 -- 172.21.15.154:0/1824766305 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944002cc70 con 0x7f94480ccfa0 2023-09-23T03:15:55.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.272+0000 7f9445ffb640 1 -- 172.21.15.154:0/1824766305 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f944002cdd0 con 0x7f94480ccfa0 2023-09-23T03:15:55.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.272+0000 7f9445ffb640 1 -- 172.21.15.154:0/1824766305 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9440034660 con 0x7f94480ccfa0 2023-09-23T03:15:55.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.273+0000 7f944da26640 1 -- 172.21.15.154:0/1824766305 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94480ccfa0 msgr2=0x7f94480cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.273+0000 7f944da26640 1 --2- 172.21.15.154:0/1824766305 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94480ccfa0 0x7f94480cd3b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9440009d90 tx=0x7f944002bec0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.273+0000 7f82d5765640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f82d015cac0 0x7f82d006b280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.154:46668/0 (socket says 172.21.15.154:46668) 2023-09-23T03:15:55.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.272+0000 7f8e397fa640 1 -- 172.21.15.154:0/1800931871 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f8e3400aae0 con 0x7f8e3c15ca80 2023-09-23T03:15:55.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.273+0000 7f8e397fa640 1 --2- 172.21.15.154:0/1800931871 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8e2006b030 0x7f8e2006d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.274+0000 7f8e397fa640 1 -- 172.21.15.154:0/1800931871 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8e340892a0 con 0x7f8e3c15ca80 2023-09-23T03:15:55.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.274+0000 7fb15495e640 1 -- 172.21.15.154:0/1463032401 shutdown_connections 2023-09-23T03:15:55.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.274+0000 7fb15495e640 1 -- 172.21.15.154:0/1463032401 wait complete. 2023-09-23T03:15:55.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.274+0000 7f11a97fa640 1 --2- 172.21.15.154:0/1811900949 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f11a006af30 0x7f11a006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.273+0000 7f8090fd7640 1 -- start start 2023-09-23T03:15:55.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.274+0000 7f82d5765640 1 -- 172.21.15.154:0/472425786 learned_addr learned my addr 172.21.15.154:0/472425786 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:55.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.275+0000 7f944da26640 1 -- 172.21.15.154:0/1824766305 shutdown_connections 2023-09-23T03:15:55.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.274+0000 7f8090fd7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f808c0b2620 0x7f808c0c8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.275+0000 7f8090fd7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f808c0d25b0 0x7f808c0c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.275+0000 7f944da26640 1 --2- 172.21.15.154:0/1824766305 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f94480ccfa0 0x7f94480cd3b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.275+0000 7f944da26640 1 --2- 172.21.15.154:0/1824766305 >> v2:172.21.15.154:3300/0 conn(0x7f944815ea40 0x7f94481620d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.275+0000 7f944da26640 1 -- 172.21.15.154:0/1824766305 >> 172.21.15.154:0/1824766305 conn(0x7f9448071bc0 msgr2=0x7f94480b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:55.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.275+0000 7f11a97fa640 1 -- 172.21.15.154:0/1811900949 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f11a4089000 con 0x7f11b40cf8e0 2023-09-23T03:15:55.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.274+0000 7f82d4f64640 1 --2- 172.21.15.154:0/472425786 >> v2:172.21.15.154:3300/0 conn(0x7f82d00cf2a0 0x7f82d00cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.275+0000 7f82d79c9640 1 -- 172.21.15.154:0/472425786 --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82d00ce990 con 0x7f82d00cd670 2023-09-23T03:15:55.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.275+0000 7f11b882c640 1 --2- 172.21.15.154:0/1811900949 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f11a006af30 0x7f11a006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.276+0000 7f82d79c9640 1 -- 172.21.15.154:0/472425786 --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82d00ce3d0 con 0x7f82d015cac0 2023-09-23T03:15:55.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.276+0000 7f82d79c9640 1 -- 172.21.15.154:0/472425786 --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82d00ce1d0 con 0x7f82d00cf2a0 2023-09-23T03:15:55.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.276+0000 7f8e40ccf640 1 --2- 172.21.15.154:0/1800931871 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8e2006b030 0x7f8e2006d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.276+0000 7f944da26640 1 -- 172.21.15.154:0/1824766305 shutdown_connections 2023-09-23T03:15:55.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.276+0000 7f944da26640 1 -- 172.21.15.154:0/1824766305 wait complete. 2023-09-23T03:15:55.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.276+0000 7f8089d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f808c0d25b0 0x7f808c0c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.276+0000 7f8089d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f808c0d25b0 0x7f808c0c9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44532/0 (socket says 172.21.15.154:44532) 2023-09-23T03:15:55.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.276+0000 7f8089d9b640 1 -- 172.21.15.154:0/1671193752 learned_addr learned my addr 172.21.15.154:0/1671193752 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:55.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.276+0000 7f8090fd7640 1 --2- 172.21.15.154:0/1671193752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f808c15ca80 0x7f808c201a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.276+0000 7f8090fd7640 1 -- 172.21.15.154:0/1671193752 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808c0cae50 con 0x7f808c0d25b0 2023-09-23T03:15:55.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.276+0000 7f8090fd7640 1 -- 172.21.15.154:0/1671193752 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808c0c9fc0 con 0x7f808c0b2620 2023-09-23T03:15:55.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.276+0000 7f8090fd7640 1 -- 172.21.15.154:0/1671193752 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f808c0ca240 con 0x7f808c15ca80 2023-09-23T03:15:55.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.277+0000 7f808ad9d640 1 --2- 172.21.15.154:0/1671193752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f808c15ca80 0x7f808c201a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.278+0000 7f8e40ccf640 1 --2- 172.21.15.154:0/1800931871 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8e2006b030 0x7f8e2006d4f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8e28036000 tx=0x7f8e28008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:55.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.279+0000 7f808a59c640 1 --2- 172.21.15.154:0/1671193752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f808c0b2620 0x7f808c0c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.279+0000 7f944da26640 1 Processor -- start 2023-09-23T03:15:55.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.277+0000 7fb15495e640 1 Processor -- start 2023-09-23T03:15:55.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.277+0000 7fb15495e640 1 -- start start 2023-09-23T03:15:55.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.278+0000 7fb15495e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1500712b0 0x7fb15009be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.278+0000 7fb15495e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb150158ba0 0x7fb1500b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.279+0000 7fb15495e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1501f7a90 0x7fb1501f8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.279+0000 7fb15495e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1501552b0 con 0x7fb1500712b0 2023-09-23T03:15:55.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.279+0000 7fb15495e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15009a220 con 0x7fb1501f7a90 2023-09-23T03:15:55.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.279+0000 7fb15495e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1501f6f20 con 0x7fb150158ba0 2023-09-23T03:15:55.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.279+0000 7fb14e59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1500712b0 0x7fb15009be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.279+0000 7fb14e59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1500712b0 0x7fb15009be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44552/0 (socket says 172.21.15.154:44552) 2023-09-23T03:15:55.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.279+0000 7fb14e59c640 1 -- 172.21.15.154:0/2016337163 learned_addr learned my addr 172.21.15.154:0/2016337163 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:55.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.279+0000 7f808ad9d640 1 -- 172.21.15.154:0/1671193752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f808c0b2620 msgr2=0x7f808c0c8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.279+0000 7f808ad9d640 1 --2- 172.21.15.154:0/1671193752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f808c0b2620 0x7f808c0c8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.279+0000 7f808ad9d640 1 -- 172.21.15.154:0/1671193752 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f808c0d25b0 msgr2=0x7f808c0c9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.279+0000 7f808ad9d640 1 --2- 172.21.15.154:0/1671193752 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f808c0d25b0 0x7f808c0c9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.279+0000 7f808ad9d640 1 -- 172.21.15.154:0/1671193752 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80840093a0 con 0x7f808c15ca80 2023-09-23T03:15:55.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.276+0000 7f82c7fff640 1 -- 172.21.15.154:0/472425786 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1521287079 0 0) 0x7f82d00ce3d0 con 0x7f82d015cac0 2023-09-23T03:15:55.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.276+0000 7f82c7fff640 1 -- 172.21.15.154:0/472425786 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82bc003520 con 0x7f82d015cac0 2023-09-23T03:15:55.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.277+0000 7f82c7fff640 1 -- 172.21.15.154:0/472425786 <== mon.1 v1:172.21.15.190:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1946288620 0 0) 0x7f82bc003520 con 0x7f82d015cac0 2023-09-23T03:15:55.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.277+0000 7f82c7fff640 1 -- 172.21.15.154:0/472425786 >> v2:172.21.15.154:3300/0 conn(0x7f82d00cf2a0 msgr2=0x7f82d00cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:55.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.277+0000 7f82c7fff640 1 --2- 172.21.15.154:0/472425786 >> v2:172.21.15.154:3300/0 conn(0x7f82d00cf2a0 0x7f82d00cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.277+0000 7f82c7fff640 1 -- 172.21.15.154:0/472425786 >> v1:172.21.15.154:6789/0 conn(0x7f82d00cd670 legacy=0x7f82d00cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.277+0000 7f82c7fff640 1 -- 172.21.15.154:0/472425786 --> v1:172.21.15.190:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d00ce6b0 con 0x7f82d015cac0 2023-09-23T03:15:55.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.277+0000 7f82c7fff640 1 -- 172.21.15.154:0/472425786 <== mon.1 v1:172.21.15.190:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f82c8001a40 con 0x7f82d015cac0 2023-09-23T03:15:55.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.277+0000 7f82c7fff640 1 -- 172.21.15.154:0/472425786 >> v1:172.21.15.190:6789/0 conn(0x7f82d015cac0 legacy=0x7f82d006b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.277+0000 7f82c7fff640 1 --2- 172.21.15.154:0/472425786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82bc003a10 0x7f82bc003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.278+0000 7f82c7fff640 1 --2- 172.21.15.154:0/472425786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82bc0047c0 0x7f82bc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.278+0000 7f82c7fff640 1 --2- 172.21.15.154:0/472425786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82bc005180 0x7f82bc009970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.278+0000 7f82c7fff640 1 -- 172.21.15.154:0/472425786 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d00ce1d0 con 0x7f82bc0047c0 2023-09-23T03:15:55.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.278+0000 7f82c7fff640 1 -- 172.21.15.154:0/472425786 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82bc00a110 con 0x7f82bc005180 2023-09-23T03:15:55.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.278+0000 7f82c7fff640 1 -- 172.21.15.154:0/472425786 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82bc00a2d0 con 0x7f82bc003a10 2023-09-23T03:15:55.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.279+0000 7f82d5765640 1 --2- 172.21.15.154:0/472425786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82bc003a10 0x7f82bc003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.279+0000 7f82d4f64640 1 --2- 172.21.15.154:0/472425786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82bc0047c0 0x7f82bc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.279+0000 7f82d5f66640 1 --2- 172.21.15.154:0/472425786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82bc005180 0x7f82bc009970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.280+0000 7f8e397fa640 1 -- 172.21.15.154:0/1800931871 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f8e34053fd0 con 0x7f8e3c15ca80 2023-09-23T03:15:55.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.280+0000 7f944da26640 1 -- start start 2023-09-23T03:15:55.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.280+0000 7fb14e59c640 1 -- 172.21.15.154:0/2016337163 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb150158ba0 msgr2=0x7fb1500b31f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-23T03:15:55.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.281+0000 7f808ad9d640 1 --2- 172.21.15.154:0/1671193752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f808c15ca80 0x7f808c201a80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8074033000 tx=0x7f80740046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:55.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.281+0000 7fb14ed9d640 1 --2- 172.21.15.154:0/2016337163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1501f7a90 0x7fb1501f8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.281+0000 7f11b882c640 1 --2- 172.21.15.154:0/1811900949 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f11a006af30 0x7f11a006d3f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f11b0000c00 tx=0x7f11b003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:55.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.281+0000 7fb147fff640 1 --2- 172.21.15.154:0/2016337163 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb150158ba0 0x7fb1500b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.281+0000 7fb14e59c640 1 --2- 172.21.15.154:0/2016337163 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb150158ba0 0x7fb1500b31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.281+0000 7fb14e59c640 1 -- 172.21.15.154:0/2016337163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1501f7a90 msgr2=0x7fb1501f8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.281+0000 7fb14e59c640 1 --2- 172.21.15.154:0/2016337163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1501f7a90 0x7fb1501f8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.281+0000 7fb14e59c640 1 -- 172.21.15.154:0/2016337163 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb148009830 con 0x7fb1500712b0 2023-09-23T03:15:55.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.281+0000 7f82d4f64640 1 -- 172.21.15.154:0/472425786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82bc003a10 msgr2=0x7f82bc003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:55.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.281+0000 7f82d4f64640 1 --2- 172.21.15.154:0/472425786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82bc003a10 0x7f82bc003e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.281+0000 7f82d4f64640 1 -- 172.21.15.154:0/472425786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82bc005180 msgr2=0x7f82bc009970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:55.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.281+0000 7f82d4f64640 1 --2- 172.21.15.154:0/472425786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82bc005180 0x7f82bc009970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.281+0000 7f82d4f64640 1 -- 172.21.15.154:0/472425786 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82bc003520 con 0x7f82bc0047c0 2023-09-23T03:15:55.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.281+0000 7f82d5f66640 1 --2- 172.21.15.154:0/472425786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82bc005180 0x7f82bc009970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:15:55.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.281+0000 7f8082ffd640 1 -- 172.21.15.154:0/1671193752 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8074044620 con 0x7f808c15ca80 2023-09-23T03:15:55.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.281+0000 7f8082ffd640 1 -- 172.21.15.154:0/1671193752 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8074044780 con 0x7f808c15ca80 2023-09-23T03:15:55.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.281+0000 7f8082ffd640 1 -- 172.21.15.154:0/1671193752 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8074044620 con 0x7f808c15ca80 2023-09-23T03:15:55.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.281+0000 7f82d5f66640 1 -- 172.21.15.154:0/472425786 reap_dead start 2023-09-23T03:15:55.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.283+0000 7f8090fd7640 1 -- 172.21.15.154:0/1671193752 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8074009400 con 0x7f808c15ca80 2023-09-23T03:15:55.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.283+0000 7f8090fd7640 1 -- 172.21.15.154:0/1671193752 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f808c0ca800 con 0x7f808c15ca80 2023-09-23T03:15:55.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.283+0000 7fb14e59c640 1 --2- 172.21.15.154:0/2016337163 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1500712b0 0x7fb15009be40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb14000c430 tx=0x7fb14000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:55.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.283+0000 7f82d4f64640 1 --2- 172.21.15.154:0/472425786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82bc0047c0 0x7f82bc004c40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f82c0009ac0 tx=0x7f82bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46409277599e8a0e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:55.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.284+0000 7f944da26640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94480ccfa0 0x7f94480c8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.285+0000 7f944da26640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f944815ea40 0x7f94480cb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.284+0000 7f8090fd7640 1 -- 172.21.15.154:0/1671193752 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f808c073eb0 con 0x7f808c15ca80 2023-09-23T03:15:55.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.285+0000 7f944da26640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9448165810 0x7f944815d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.285+0000 7f944da26640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94480b56a0 con 0x7f944815ea40 2023-09-23T03:15:55.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.285+0000 7f944da26640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94480cc930 con 0x7f94480ccfa0 2023-09-23T03:15:55.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.285+0000 7f944da26640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94480cc090 con 0x7f9448165810 2023-09-23T03:15:55.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.285+0000 7f94467fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f944815ea40 0x7f94480cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.285+0000 7f9446ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94480ccfa0 0x7f94480c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.285+0000 7f9446ffd640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94480ccfa0 0x7f94480c8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.154:57816/0 (socket says 172.21.15.154:57816) 2023-09-23T03:15:55.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.285+0000 7f9446ffd640 1 -- 172.21.15.154:0/369640610 learned_addr learned my addr 172.21.15.154:0/369640610 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:55.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.286+0000 7f94477fe640 1 --2- 172.21.15.154:0/369640610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9448165810 0x7f944815d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.286+0000 7f94467fc640 1 -- 172.21.15.154:0/369640610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9448165810 msgr2=0x7f944815d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.286+0000 7f94467fc640 1 --2- 172.21.15.154:0/369640610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9448165810 0x7f944815d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.286+0000 7f94467fc640 1 -- 172.21.15.154:0/369640610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94480ccfa0 msgr2=0x7f94480c8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.286+0000 7f94467fc640 1 --2- 172.21.15.154:0/369640610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94480ccfa0 0x7f94480c8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.286+0000 7f94467fc640 1 -- 172.21.15.154:0/369640610 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9440009a70 con 0x7f944815ea40 2023-09-23T03:15:55.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.286+0000 7f94477fe640 1 --2- 172.21.15.154:0/369640610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9448165810 0x7f944815d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:15:55.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.287+0000 7f9446ffd640 1 --2- 172.21.15.154:0/369640610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94480ccfa0 0x7f94480c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:15:55.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.287+0000 7fb146ffd640 1 -- 172.21.15.154:0/2016337163 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb140011790 con 0x7fb1500712b0 2023-09-23T03:15:55.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.288+0000 7fb15495e640 1 -- 172.21.15.154:0/2016337163 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1501f7200 con 0x7fb1500712b0 2023-09-23T03:15:55.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.288+0000 7fb15495e640 1 -- 172.21.15.154:0/2016337163 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1501f77c0 con 0x7fb1500712b0 2023-09-23T03:15:55.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.288+0000 7fb146ffd640 1 -- 172.21.15.154:0/2016337163 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1400118f0 con 0x7fb1500712b0 2023-09-23T03:15:55.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.288+0000 7fb146ffd640 1 -- 172.21.15.154:0/2016337163 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb140011790 con 0x7fb1500712b0 2023-09-23T03:15:55.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.287+0000 7f8082ffd640 1 -- 172.21.15.154:0/1671193752 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f8074042070 con 0x7f808c15ca80 2023-09-23T03:15:55.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.289+0000 7f8082ffd640 1 --2- 172.21.15.154:0/1671193752 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f806c06b020 0x7f806c06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.289+0000 7f11a97fa640 1 -- 172.21.15.154:0/1811900949 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f11a40bba50 con 0x7f11b40cf8e0 2023-09-23T03:15:55.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.290+0000 7f808a59c640 1 --2- 172.21.15.154:0/1671193752 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f806c06b020 0x7f806c06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.290+0000 7f8082ffd640 1 -- 172.21.15.154:0/1671193752 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f807407a1e0 con 0x7f808c15ca80 2023-09-23T03:15:55.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.289+0000 7f82c7fff640 1 -- 172.21.15.154:0/472425786 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c002c6f0 con 0x7f82bc0047c0 2023-09-23T03:15:55.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.289+0000 7f82c7fff640 1 -- 172.21.15.154:0/472425786 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c002c850 con 0x7f82bc0047c0 2023-09-23T03:15:55.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.289+0000 7f82c7fff640 1 -- 172.21.15.154:0/472425786 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c002cbe0 con 0x7f82bc0047c0 2023-09-23T03:15:55.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.290+0000 7f82d79c9640 1 -- 172.21.15.154:0/472425786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82bc0047c0 msgr2=0x7f82bc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.290+0000 7f82d79c9640 1 --2- 172.21.15.154:0/472425786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82bc0047c0 0x7f82bc004c40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f82c0009ac0 tx=0x7f82bc0057f0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.290+0000 7f94467fc640 1 --2- 172.21.15.154:0/369640610 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f944815ea40 0x7f94480cb720 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f943c00c370 tx=0x7f943c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:55.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.291+0000 7f82d79c9640 1 -- 172.21.15.154:0/472425786 shutdown_connections 2023-09-23T03:15:55.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.291+0000 7f82d79c9640 1 --2- 172.21.15.154:0/472425786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82bc0047c0 0x7f82bc004c40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.291+0000 7f82d79c9640 1 -- 172.21.15.154:0/472425786 >> 172.21.15.154:0/472425786 conn(0x7f82d0089fd0 msgr2=0x7f82d009bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:55.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.291+0000 7f808a59c640 1 --2- 172.21.15.154:0/1671193752 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f806c06b020 0x7f806c06d4e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f808c0ccdb0 tx=0x7f8084008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:55.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.291+0000 7f82d79c9640 1 -- 172.21.15.154:0/472425786 shutdown_connections 2023-09-23T03:15:55.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.291+0000 7f82d79c9640 1 -- 172.21.15.154:0/472425786 wait complete. 2023-09-23T03:15:55.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.291+0000 7fb15495e640 1 -- 172.21.15.154:0/2016337163 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb134005250 con 0x7fb1500712b0 2023-09-23T03:15:55.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.293+0000 7f942ffff640 1 -- 172.21.15.154:0/369640610 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c0052d0 con 0x7f944815ea40 2023-09-23T03:15:55.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.293+0000 7f942ffff640 1 -- 172.21.15.154:0/369640610 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f943c005430 con 0x7f944815ea40 2023-09-23T03:15:55.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.293+0000 7f942ffff640 1 -- 172.21.15.154:0/369640610 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c003850 con 0x7f944815ea40 2023-09-23T03:15:55.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.293+0000 7fb146ffd640 1 -- 172.21.15.154:0/2016337163 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fb140011aa0 con 0x7fb1500712b0 2023-09-23T03:15:55.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.294+0000 7fb146ffd640 1 --2- 172.21.15.154:0/2016337163 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb13006aff0 0x7fb13006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.295+0000 7fb147fff640 1 --2- 172.21.15.154:0/2016337163 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb13006aff0 0x7fb13006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.293+0000 7f944da26640 1 -- 172.21.15.154:0/369640610 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94480cc370 con 0x7f944815ea40 2023-09-23T03:15:55.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.293+0000 7f944da26640 1 -- 172.21.15.154:0/369640610 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94480c8dd0 con 0x7f944815ea40 2023-09-23T03:15:55.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.296+0000 7fb147fff640 1 --2- 172.21.15.154:0/2016337163 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb13006aff0 0x7fb13006d4b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb1500af480 tx=0x7fb138008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:55.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.296+0000 7f944da26640 1 -- 172.21.15.154:0/369640610 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9448073eb0 con 0x7f944815ea40 2023-09-23T03:15:55.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.296+0000 7fb146ffd640 1 -- 172.21.15.154:0/2016337163 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb14000e280 con 0x7fb1500712b0 2023-09-23T03:15:55.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.297+0000 7f942ffff640 1 -- 172.21.15.154:0/369640610 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f943c0039b0 con 0x7f944815ea40 2023-09-23T03:15:55.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.298+0000 7f82d79c9640 1 Processor -- start 2023-09-23T03:15:55.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.298+0000 7f82d79c9640 1 -- start start 2023-09-23T03:15:55.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.299+0000 7f942ffff640 1 --2- 172.21.15.154:0/369640610 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f943406b000 0x7f943406d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.299+0000 7f942ffff640 1 -- 172.21.15.154:0/369640610 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f943c088f80 con 0x7f944815ea40 2023-09-23T03:15:55.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.299+0000 7f9446ffd640 1 --2- 172.21.15.154:0/369640610 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f943406b000 0x7f943406d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.300+0000 7f8082ffd640 1 -- 172.21.15.154:0/1671193752 <== mon.2 v2:172.21.15.190:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f8074076cb0 con 0x7f808c15ca80 2023-09-23T03:15:55.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.301+0000 7f82d79c9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82bc0047c0 0x7f82d00c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.301+0000 7f82d79c9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82d01565a0 0x7f82d0156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.301+0000 7f82d79c9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82d0141bf0 0x7f82d0142070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.301+0000 7f82d79c9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d0156f60 con 0x7f82d0141bf0 2023-09-23T03:15:55.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.302+0000 7f9446ffd640 1 --2- 172.21.15.154:0/369640610 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f943406b000 0x7f943406d4c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9438006a90 tx=0x7f9438008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:55.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.301+0000 7f82d79c9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d0150630 con 0x7f82bc0047c0 2023-09-23T03:15:55.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.301+0000 7f82d79c9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d00c90e0 con 0x7f82d01565a0 2023-09-23T03:15:55.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.302+0000 7f82d4f64640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82d01565a0 0x7f82d0156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.303+0000 7f82d5f66640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82d0141bf0 0x7f82d0142070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.303+0000 7f82d5f66640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82d0141bf0 0x7f82d0142070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44560/0 (socket says 172.21.15.154:44560) 2023-09-23T03:15:55.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.303+0000 7f82d5f66640 1 -- 172.21.15.154:0/2350608792 learned_addr learned my addr 172.21.15.154:0/2350608792 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:55.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.303+0000 7f82d5f66640 1 -- 172.21.15.154:0/2350608792 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82d01565a0 msgr2=0x7f82d0156a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.304+0000 7f82d5f66640 1 --2- 172.21.15.154:0/2350608792 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82d01565a0 0x7f82d0156a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.304+0000 7f82d5f66640 1 -- 172.21.15.154:0/2350608792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82bc0047c0 msgr2=0x7f82d00c7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.304+0000 7f82d5f66640 1 --2- 172.21.15.154:0/2350608792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82bc0047c0 0x7f82d00c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.304+0000 7f82d5f66640 1 -- 172.21.15.154:0/2350608792 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d01425b0 con 0x7f82d0141bf0 2023-09-23T03:15:55.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.304+0000 7f82d5f66640 1 --2- 172.21.15.154:0/2350608792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82d0141bf0 0x7f82d0142070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f82d00d2680 tx=0x7f82cc00fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:55.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.306+0000 7fb146ffd640 1 -- 172.21.15.154:0/2016337163 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fb140054a60 con 0x7fb1500712b0 2023-09-23T03:15:55.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.315+0000 7f942ffff640 1 -- 172.21.15.154:0/369640610 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f943c053e20 con 0x7f944815ea40 2023-09-23T03:15:55.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.315+0000 7f82c5ffb640 1 -- 172.21.15.154:0/2350608792 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc005110 con 0x7f82d0141bf0 2023-09-23T03:15:55.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.315+0000 7f82c5ffb640 1 -- 172.21.15.154:0/2350608792 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82cc005270 con 0x7f82d0141bf0 2023-09-23T03:15:55.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.315+0000 7f82c5ffb640 1 -- 172.21.15.154:0/2350608792 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc007030 con 0x7f82d0141bf0 2023-09-23T03:15:55.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.317+0000 7f82d79c9640 1 -- 172.21.15.154:0/2350608792 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82d00c8210 con 0x7f82d0141bf0 2023-09-23T03:15:55.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.317+0000 7f82d79c9640 1 -- 172.21.15.154:0/2350608792 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82d00c87b0 con 0x7f82d0141bf0 2023-09-23T03:15:55.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.321+0000 7f82c5ffb640 1 -- 172.21.15.154:0/2350608792 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f82cc007190 con 0x7f82d0141bf0 2023-09-23T03:15:55.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.323+0000 7f82c5ffb640 1 --2- 172.21.15.154:0/2350608792 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f82b406b000 0x7f82b406d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.323+0000 7f82c5ffb640 1 -- 172.21.15.154:0/2350608792 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f82cc0890f0 con 0x7f82d0141bf0 2023-09-23T03:15:55.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.322+0000 7f82d79c9640 1 -- 172.21.15.154:0/2350608792 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82d0073eb0 con 0x7f82d0141bf0 2023-09-23T03:15:55.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.324+0000 7f82d5765640 1 --2- 172.21.15.154:0/2350608792 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f82b406b000 0x7f82b406d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.329+0000 7f82d5765640 1 --2- 172.21.15.154:0/2350608792 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f82b406b000 0x7f82b406d4c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f82c8007c40 tx=0x7f82c8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:55.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.335+0000 7f82c5ffb640 1 -- 172.21.15.154:0/2350608792 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f82cc01a020 con 0x7f82d0141bf0 2023-09-23T03:15:55.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.635+0000 7fd013abb640 1 -- 172.21.15.154:0/2100529831 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fcff40057d0 con 0x7fd00c15c850 2023-09-23T03:15:55.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.636+0000 7fd0027fc640 1 -- 172.21.15.154:0/2100529831 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd004039090 con 0x7fd00c15c850 2023-09-23T03:15:55.638 INFO:teuthology.orchestra.run.smithi154.stdout:55834574861 2023-09-23T03:15:55.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.640+0000 7fcfeffff640 1 -- 172.21.15.154:0/2100529831 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcff006b030 msgr2=0x7fcff006d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.640+0000 7fcfeffff640 1 --2- 172.21.15.154:0/2100529831 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcff006b030 0x7fcff006d4f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcffc009d50 tx=0x7fcffc009060 comp rx=0 tx=0).stop 2023-09-23T03:15:55.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.641+0000 7fcfeffff640 1 -- 172.21.15.154:0/2100529831 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd00c15c850 msgr2=0x7fd00c1527d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.641+0000 7fcfeffff640 1 --2- 172.21.15.154:0/2100529831 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd00c15c850 0x7fd00c1527d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd00402c290 tx=0x7fd004002350 comp rx=0 tx=0).stop 2023-09-23T03:15:55.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.641+0000 7fcfeffff640 1 -- 172.21.15.154:0/2100529831 shutdown_connections 2023-09-23T03:15:55.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.641+0000 7fcfeffff640 1 --2- 172.21.15.154:0/2100529831 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcff006b030 0x7fcff006d4f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.641+0000 7fcfeffff640 1 --2- 172.21.15.154:0/2100529831 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd00c15e1f0 0x7fd00c0b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.642+0000 7fcfeffff640 1 --2- 172.21.15.154:0/2100529831 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd00c15c850 0x7fd00c1527d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.642+0000 7fcfeffff640 1 --2- 172.21.15.154:0/2100529831 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd00c0d0710 0x7fd00c0cb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.642+0000 7fcfeffff640 1 -- 172.21.15.154:0/2100529831 >> 172.21.15.154:0/2100529831 conn(0x7fd00c06b140 msgr2=0x7fd00c0d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:55.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.642+0000 7fcfeffff640 1 -- 172.21.15.154:0/2100529831 shutdown_connections 2023-09-23T03:15:55.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.642+0000 7fcfeffff640 1 -- 172.21.15.154:0/2100529831 wait complete. 2023-09-23T03:15:55.671 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574861 for osd.2 2023-09-23T03:15:55.671 DEBUG:teuthology.parallel:result is None 2023-09-23T03:15:55.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.673+0000 7fd94d3c4640 1 -- 172.21.15.154:0/3284415149 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd928005500 con 0x7fd940038180 2023-09-23T03:15:55.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.675+0000 7fd944ff9640 1 -- 172.21.15.154:0/3284415149 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd93c015020 con 0x7fd940038180 2023-09-23T03:15:55.676 INFO:teuthology.orchestra.run.smithi154.stdout:55834574862 2023-09-23T03:15:55.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.678+0000 7f8e42f33640 1 -- 172.21.15.154:0/1800931871 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8e3c073eb0 con 0x7f8e3c15ca80 2023-09-23T03:15:55.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.678+0000 7fd94d3c4640 1 -- 172.21.15.154:0/3284415149 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd92c06b000 msgr2=0x7fd92c06d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.678+0000 7fd94d3c4640 1 --2- 172.21.15.154:0/3284415149 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd92c06b000 0x7fd92c06d4c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd94017bd40 tx=0x7fd934008040 comp rx=0 tx=0).stop 2023-09-23T03:15:55.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.678+0000 7fd94d3c4640 1 -- 172.21.15.154:0/3284415149 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd940038180 msgr2=0x7fd94004e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.678+0000 7fd94d3c4640 1 --2- 172.21.15.154:0/3284415149 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd940038180 0x7fd94004e050 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd93c00b3e0 tx=0x7fd93c00b8a0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.678+0000 7fd94d3c4640 1 -- 172.21.15.154:0/3284415149 shutdown_connections 2023-09-23T03:15:55.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.678+0000 7fd94d3c4640 1 --2- 172.21.15.154:0/3284415149 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd92c06b000 0x7fd92c06d4c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.678+0000 7fd94d3c4640 1 --2- 172.21.15.154:0/3284415149 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9400e3190 0x7fd94000a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.678+0000 7fd94d3c4640 1 --2- 172.21.15.154:0/3284415149 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9400542d0 0x7fd940009e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.678+0000 7fd94d3c4640 1 --2- 172.21.15.154:0/3284415149 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd940038180 0x7fd94004e050 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.678+0000 7fd94d3c4640 1 -- 172.21.15.154:0/3284415149 >> 172.21.15.154:0/3284415149 conn(0x7fd9400590c0 msgr2=0x7fd940038940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:55.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.679+0000 7fd94d3c4640 1 -- 172.21.15.154:0/3284415149 shutdown_connections 2023-09-23T03:15:55.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.679+0000 7fd94d3c4640 1 -- 172.21.15.154:0/3284415149 wait complete. 2023-09-23T03:15:55.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.680+0000 7f8e397fa640 1 -- 172.21.15.154:0/1800931871 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8e340575f0 con 0x7f8e3c15ca80 2023-09-23T03:15:55.683 INFO:teuthology.orchestra.run.smithi154.stdout:55834574860 2023-09-23T03:15:55.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.683+0000 7f8e42f33640 1 -- 172.21.15.154:0/1800931871 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8e2006b030 msgr2=0x7f8e2006d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.683+0000 7f8e42f33640 1 --2- 172.21.15.154:0/1800931871 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8e2006b030 0x7f8e2006d4f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8e28036000 tx=0x7f8e28008030 comp rx=0 tx=0).stop 2023-09-23T03:15:55.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.683+0000 7f8e42f33640 1 -- 172.21.15.154:0/1800931871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e3c15ca80 msgr2=0x7f8e3c15a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.683+0000 7f8e42f33640 1 --2- 172.21.15.154:0/1800931871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e3c15ca80 0x7f8e3c15a980 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400caf0 tx=0x7f8e34002130 comp rx=0 tx=0).stop 2023-09-23T03:15:55.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.683+0000 7f8e42f33640 1 -- 172.21.15.154:0/1800931871 shutdown_connections 2023-09-23T03:15:55.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.683+0000 7f8e42f33640 1 --2- 172.21.15.154:0/1800931871 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8e2006b030 0x7f8e2006d4f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.683+0000 7f8e42f33640 1 --2- 172.21.15.154:0/1800931871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e3c15ca80 0x7f8e3c15a980 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.683+0000 7f8e42f33640 1 --2- 172.21.15.154:0/1800931871 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e3c0d25b0 0x7f8e3c1e8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.683+0000 7f8e42f33640 1 --2- 172.21.15.154:0/1800931871 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e3c0b2620 0x7f8e3c155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.683+0000 7f8e42f33640 1 -- 172.21.15.154:0/1800931871 >> 172.21.15.154:0/1800931871 conn(0x7f8e3c09bc60 msgr2=0x7f8e3c15a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:55.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.684+0000 7f8e42f33640 1 -- 172.21.15.154:0/1800931871 shutdown_connections 2023-09-23T03:15:55.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.684+0000 7f8e42f33640 1 -- 172.21.15.154:0/1800931871 wait complete. 2023-09-23T03:15:55.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.690+0000 7f119affd640 1 -- 172.21.15.154:0/1811900949 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f11b404e920 con 0x7f11b40cf8e0 2023-09-23T03:15:55.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.692+0000 7f11a97fa640 1 -- 172.21.15.154:0/1811900949 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f11a4017090 con 0x7f11b40cf8e0 2023-09-23T03:15:55.693 INFO:teuthology.orchestra.run.smithi154.stdout:55834574861 2023-09-23T03:15:55.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.696+0000 7f119affd640 1 -- 172.21.15.154:0/1811900949 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f11a006af30 msgr2=0x7f11a006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.696+0000 7f119affd640 1 --2- 172.21.15.154:0/1811900949 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f11a006af30 0x7f11a006d3f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f11b0000c00 tx=0x7f11b003c040 comp rx=0 tx=0).stop 2023-09-23T03:15:55.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.696+0000 7f119affd640 1 -- 172.21.15.154:0/1811900949 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11b40cf8e0 msgr2=0x7f11b40ca4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.696+0000 7f119affd640 1 --2- 172.21.15.154:0/1811900949 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11b40cf8e0 0x7f11b40ca4f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f11a400c370 tx=0x7f11a400c830 comp rx=0 tx=0).stop 2023-09-23T03:15:55.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.696+0000 7f119affd640 1 -- 172.21.15.154:0/1811900949 shutdown_connections 2023-09-23T03:15:55.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.696+0000 7f119affd640 1 --2- 172.21.15.154:0/1811900949 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f11a006af30 0x7f11a006d3f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.696+0000 7f119affd640 1 --2- 172.21.15.154:0/1811900949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11b415cc00 0x7f11b415ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.697+0000 7f119affd640 1 --2- 172.21.15.154:0/1811900949 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11b40cf8e0 0x7f11b40ca4f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.697+0000 7f119affd640 1 --2- 172.21.15.154:0/1811900949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11b40cf500 0x7f11b40c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.697+0000 7f119affd640 1 -- 172.21.15.154:0/1811900949 >> 172.21.15.154:0/1811900949 conn(0x7f11b409bbb0 msgr2=0x7f11b41e7600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:55.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.699+0000 7f119affd640 1 -- 172.21.15.154:0/1811900949 shutdown_connections 2023-09-23T03:15:55.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.699+0000 7f8090fd7640 1 -- 172.21.15.154:0/1671193752 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f808c15a4e0 con 0x7f808c15ca80 2023-09-23T03:15:55.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.699+0000 7f119affd640 1 -- 172.21.15.154:0/1811900949 wait complete. 2023-09-23T03:15:55.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.700+0000 7f8082ffd640 1 -- 172.21.15.154:0/1671193752 <== mon.2 v2:172.21.15.190:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8074076e50 con 0x7f808c15ca80 2023-09-23T03:15:55.701 INFO:teuthology.orchestra.run.smithi154.stdout:55834574862 2023-09-23T03:15:55.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.702+0000 7f8090fd7640 1 -- 172.21.15.154:0/1671193752 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f806c06b020 msgr2=0x7f806c06d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.702+0000 7f8090fd7640 1 --2- 172.21.15.154:0/1671193752 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f806c06b020 0x7f806c06d4e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f808c0ccdb0 tx=0x7f8084008040 comp rx=0 tx=0).stop 2023-09-23T03:15:55.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.702+0000 7f8090fd7640 1 -- 172.21.15.154:0/1671193752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f808c15ca80 msgr2=0x7f808c201a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.702+0000 7f8090fd7640 1 --2- 172.21.15.154:0/1671193752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f808c15ca80 0x7f808c201a80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8074033000 tx=0x7f80740046c0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.703+0000 7f8090fd7640 1 -- 172.21.15.154:0/1671193752 shutdown_connections 2023-09-23T03:15:55.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.703+0000 7f8090fd7640 1 --2- 172.21.15.154:0/1671193752 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f806c06b020 0x7f806c06d4e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.703+0000 7f8090fd7640 1 --2- 172.21.15.154:0/1671193752 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f808c15ca80 0x7f808c201a80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.703+0000 7f8090fd7640 1 --2- 172.21.15.154:0/1671193752 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f808c0d25b0 0x7f808c0c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.703+0000 7f8090fd7640 1 --2- 172.21.15.154:0/1671193752 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f808c0b2620 0x7f808c0c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.703+0000 7f8090fd7640 1 -- 172.21.15.154:0/1671193752 >> 172.21.15.154:0/1671193752 conn(0x7f808c09bc60 msgr2=0x7f808c0b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:55.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.703+0000 7f8090fd7640 1 -- 172.21.15.154:0/1671193752 shutdown_connections 2023-09-23T03:15:55.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.703+0000 7f8090fd7640 1 -- 172.21.15.154:0/1671193752 wait complete. 2023-09-23T03:15:55.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.707+0000 7f944da26640 1 -- 172.21.15.154:0/369640610 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f94480ce850 con 0x7f944815ea40 2023-09-23T03:15:55.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.708+0000 7f942ffff640 1 -- 172.21.15.154:0/369640610 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f943c017020 con 0x7f944815ea40 2023-09-23T03:15:55.709 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574861 got 55834574862 for osd.1 2023-09-23T03:15:55.710 INFO:teuthology.orchestra.run.smithi154.stdout:55834574860 2023-09-23T03:15:55.710 DEBUG:teuthology.parallel:result is None 2023-09-23T03:15:55.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.710+0000 7f944da26640 1 -- 172.21.15.154:0/369640610 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f943406b000 msgr2=0x7f943406d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.711+0000 7f944da26640 1 --2- 172.21.15.154:0/369640610 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f943406b000 0x7f943406d4c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9438006a90 tx=0x7f9438008040 comp rx=0 tx=0).stop 2023-09-23T03:15:55.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.711+0000 7f944da26640 1 -- 172.21.15.154:0/369640610 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f944815ea40 msgr2=0x7f94480cb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.711+0000 7f944da26640 1 --2- 172.21.15.154:0/369640610 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f944815ea40 0x7f94480cb720 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f943c00c370 tx=0x7f943c00c830 comp rx=0 tx=0).stop 2023-09-23T03:15:55.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.711+0000 7f944da26640 1 -- 172.21.15.154:0/369640610 shutdown_connections 2023-09-23T03:15:55.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.711+0000 7f944da26640 1 --2- 172.21.15.154:0/369640610 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f943406b000 0x7f943406d4c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.711+0000 7f944da26640 1 --2- 172.21.15.154:0/369640610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9448165810 0x7f944815d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.711+0000 7f944da26640 1 --2- 172.21.15.154:0/369640610 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f944815ea40 0x7f94480cb720 secure :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f943c00c370 tx=0x7f943c00c830 comp rx=0 tx=0).stop 2023-09-23T03:15:55.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.711+0000 7f944da26640 1 --2- 172.21.15.154:0/369640610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f94480ccfa0 0x7f94480c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.711+0000 7f944da26640 1 -- 172.21.15.154:0/369640610 >> 172.21.15.154:0/369640610 conn(0x7f9448071bc0 msgr2=0x7f94480d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:55.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.711+0000 7f944da26640 1 -- 172.21.15.154:0/369640610 shutdown_connections 2023-09-23T03:15:55.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.711+0000 7f944da26640 1 -- 172.21.15.154:0/369640610 wait complete. 2023-09-23T03:15:55.714 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574860 for osd.3 2023-09-23T03:15:55.715 DEBUG:teuthology.parallel:result is None 2023-09-23T03:15:55.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.712+0000 7fb15495e640 1 -- 172.21.15.154:0/2016337163 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fb1340057d0 con 0x7fb1500712b0 2023-09-23T03:15:55.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.713+0000 7fb146ffd640 1 -- 172.21.15.154:0/2016337163 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb140054a60 con 0x7fb1500712b0 2023-09-23T03:15:55.715 INFO:teuthology.orchestra.run.smithi154.stdout:55834574859 2023-09-23T03:15:55.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.715+0000 7fb15495e640 1 -- 172.21.15.154:0/2016337163 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb13006aff0 msgr2=0x7fb13006d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.715+0000 7fb15495e640 1 --2- 172.21.15.154:0/2016337163 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb13006aff0 0x7fb13006d4b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb1500af480 tx=0x7fb138008040 comp rx=0 tx=0).stop 2023-09-23T03:15:55.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.715+0000 7fb15495e640 1 -- 172.21.15.154:0/2016337163 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1500712b0 msgr2=0x7fb15009be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.715+0000 7fb15495e640 1 --2- 172.21.15.154:0/2016337163 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1500712b0 0x7fb15009be40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb14000c430 tx=0x7fb14000c8f0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.716+0000 7fb15495e640 1 -- 172.21.15.154:0/2016337163 shutdown_connections 2023-09-23T03:15:55.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.716+0000 7fb15495e640 1 --2- 172.21.15.154:0/2016337163 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb13006aff0 0x7fb13006d4b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.716+0000 7fb15495e640 1 --2- 172.21.15.154:0/2016337163 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1501f7a90 0x7fb1501f8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.716+0000 7fb15495e640 1 --2- 172.21.15.154:0/2016337163 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb150158ba0 0x7fb1500b31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.716+0000 7fb15495e640 1 --2- 172.21.15.154:0/2016337163 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1500712b0 0x7fb15009be40 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.716+0000 7fb15495e640 1 -- 172.21.15.154:0/2016337163 >> 172.21.15.154:0/2016337163 conn(0x7fb1500afe40 msgr2=0x7fb1500b0220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:55.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.716+0000 7fb15495e640 1 -- 172.21.15.154:0/2016337163 shutdown_connections 2023-09-23T03:15:55.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.716+0000 7fb15495e640 1 -- 172.21.15.154:0/2016337163 wait complete. 2023-09-23T03:15:55.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.726+0000 7f82d79c9640 1 -- 172.21.15.154:0/2350608792 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f82d00c9220 con 0x7f82d0141bf0 2023-09-23T03:15:55.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.726+0000 7f82c5ffb640 1 -- 172.21.15.154:0/2350608792 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f82cc053eb0 con 0x7f82d0141bf0 2023-09-23T03:15:55.728 INFO:teuthology.orchestra.run.smithi154.stdout:55834574859 2023-09-23T03:15:55.729 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574861 for osd.5 2023-09-23T03:15:55.730 DEBUG:teuthology.parallel:result is None 2023-09-23T03:15:55.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.729+0000 7f82d79c9640 1 -- 172.21.15.154:0/2350608792 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f82b406b000 msgr2=0x7f82b406d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.729+0000 7f82d79c9640 1 --2- 172.21.15.154:0/2350608792 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f82b406b000 0x7f82b406d4c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f82c8007c40 tx=0x7f82c8009040 comp rx=0 tx=0).stop 2023-09-23T03:15:55.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.729+0000 7f82d79c9640 1 -- 172.21.15.154:0/2350608792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82d0141bf0 msgr2=0x7f82d0142070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:55.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.729+0000 7f82d79c9640 1 --2- 172.21.15.154:0/2350608792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82d0141bf0 0x7f82d0142070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f82d00d2680 tx=0x7f82cc00fd40 comp rx=0 tx=0).stop 2023-09-23T03:15:55.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.729+0000 7f82d79c9640 1 -- 172.21.15.154:0/2350608792 shutdown_connections 2023-09-23T03:15:55.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.729+0000 7f82d79c9640 1 --2- 172.21.15.154:0/2350608792 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f82b406b000 0x7f82b406d4c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.729+0000 7f82d79c9640 1 --2- 172.21.15.154:0/2350608792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82d0141bf0 0x7f82d0142070 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.729+0000 7f82d79c9640 1 --2- 172.21.15.154:0/2350608792 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82d01565a0 0x7f82d0156a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.729+0000 7f82d79c9640 1 --2- 172.21.15.154:0/2350608792 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82bc0047c0 0x7f82d00c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:55.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.729+0000 7f82d79c9640 1 -- 172.21.15.154:0/2350608792 >> 172.21.15.154:0/2350608792 conn(0x7f82d0089fd0 msgr2=0x7f82d0159b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:55.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.729+0000 7f82d79c9640 1 -- 172.21.15.154:0/2350608792 shutdown_connections 2023-09-23T03:15:55.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.729+0000 7f82d79c9640 1 -- 172.21.15.154:0/2350608792 wait complete. 2023-09-23T03:15:55.732 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574860 for osd.4 2023-09-23T03:15:55.733 DEBUG:teuthology.parallel:result is None 2023-09-23T03:15:55.733 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574861 got 55834574862 for osd.0 2023-09-23T03:15:55.733 DEBUG:teuthology.parallel:result is None 2023-09-23T03:15:55.736 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.7 2023-09-23T03:15:55.736 DEBUG:teuthology.parallel:result is None 2023-09-23T03:15:55.747 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574859 for osd.6 2023-09-23T03:15:55.747 DEBUG:teuthology.parallel:result is None 2023-09-23T03:15:55.747 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-09-23T03:15:55.747 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-23T03:15:55.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.994+0000 7f8a6d4af640 1 Processor -- start 2023-09-23T03:15:55.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.995+0000 7f8a6d4af640 1 -- start start 2023-09-23T03:15:55.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.996+0000 7f8a6d4af640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8a6815ca80 0x7f8a68160680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.996+0000 7f8a6d4af640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8a680b2620 0x7f8a680d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:55.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.996+0000 7f8a6d4af640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a68161500 con 0x7f8a680d25b0 2023-09-23T03:15:55.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.996+0000 7f8a6d4af640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a680b2ad0 con 0x7f8a680b2620 2023-09-23T03:15:55.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.996+0000 7f8a6d4af640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a68160c60 con 0x7f8a6815ca80 2023-09-23T03:15:55.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.997+0000 7f8a677fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f8a680d25b0 0x7f8a680d2a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46288/0 (socket says 172.21.15.154:46288) 2023-09-23T03:15:55.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.997+0000 7f8a677fe640 1 -- 172.21.15.154:0/3990589094 learned_addr learned my addr 172.21.15.154:0/3990589094 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:55.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.997+0000 7f8a667fc640 1 --2- 172.21.15.154:0/3990589094 >> v2:172.21.15.154:3300/0 conn(0x7f8a680b2620 0x7f8a680d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.999 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.997+0000 7f8a66ffd640 1 --2- 172.21.15.154:0/3990589094 >> v2:172.21.15.190:3300/0 conn(0x7f8a6815ca80 0x7f8a68160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:55.999 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.997+0000 7f8a65ffb640 1 -- 172.21.15.154:0/3990589094 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2756464875 0 0) 0x7f8a68161500 con 0x7f8a680d25b0 2023-09-23T03:15:56.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.997+0000 7f8a65ffb640 1 -- 172.21.15.154:0/3990589094 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a54003520 con 0x7f8a680d25b0 2023-09-23T03:15:56.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.997+0000 7f8a667fc640 1 -- 172.21.15.154:0/3990589094 >> v2:172.21.15.190:3300/0 conn(0x7f8a6815ca80 msgr2=0x7f8a68160680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:56.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.997+0000 7f8a667fc640 1 --2- 172.21.15.154:0/3990589094 >> v2:172.21.15.190:3300/0 conn(0x7f8a6815ca80 0x7f8a68160680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:56.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.997+0000 7f8a667fc640 1 -- 172.21.15.154:0/3990589094 >> v1:172.21.15.154:6789/0 conn(0x7f8a680d25b0 legacy=0x7f8a680d2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:56.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.997+0000 7f8a667fc640 1 -- 172.21.15.154:0/3990589094 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a68160f40 con 0x7f8a680b2620 2023-09-23T03:15:56.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.998+0000 7f8a667fc640 1 --2- 172.21.15.154:0/3990589094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a680b2620 0x7f8a680d07b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8a6806a970 tx=0x7f8a5002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2078a6ffa055a8f6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:56.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.999+0000 7f8a65ffb640 1 -- 172.21.15.154:0/3990589094 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a50031830 con 0x7f8a680b2620 2023-09-23T03:15:56.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.999+0000 7f8a65ffb640 1 -- 172.21.15.154:0/3990589094 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a50031990 con 0x7f8a680b2620 2023-09-23T03:15:56.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.999+0000 7f8a65ffb640 1 -- 172.21.15.154:0/3990589094 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a50031d20 con 0x7f8a680b2620 2023-09-23T03:15:56.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.999+0000 7f8a6d4af640 1 -- 172.21.15.154:0/3990589094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a680b2620 msgr2=0x7f8a680d07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:56.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.999+0000 7f8a6d4af640 1 --2- 172.21.15.154:0/3990589094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a680b2620 0x7f8a680d07b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8a6806a970 tx=0x7f8a5002b8d0 comp rx=0 tx=0).stop 2023-09-23T03:15:56.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.999+0000 7f8a6d4af640 1 -- 172.21.15.154:0/3990589094 shutdown_connections 2023-09-23T03:15:56.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.999+0000 7f8a6d4af640 1 --2- 172.21.15.154:0/3990589094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a680b2620 0x7f8a680d07b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:56.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.999+0000 7f8a6d4af640 1 --2- 172.21.15.154:0/3990589094 >> v2:172.21.15.190:3300/0 conn(0x7f8a6815ca80 0x7f8a68160680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:56.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.999+0000 7f8a6d4af640 1 -- 172.21.15.154:0/3990589094 >> 172.21.15.154:0/3990589094 conn(0x7f8a6809bc60 msgr2=0x7f8a68089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:56.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.999+0000 7f8a6d4af640 1 -- 172.21.15.154:0/3990589094 shutdown_connections 2023-09-23T03:15:56.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:55.999+0000 7f8a6d4af640 1 -- 172.21.15.154:0/3990589094 wait complete. 2023-09-23T03:15:56.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.001+0000 7f8a6d4af640 1 Processor -- start 2023-09-23T03:15:56.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.001+0000 7f8a6d4af640 1 -- start start 2023-09-23T03:15:56.003 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.002+0000 7f8a6d4af640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a680b2620 0x7f8a681f2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:56.003 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.002+0000 7f8a6d4af640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a680d25b0 0x7f8a680ca330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:56.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.002+0000 7f8a6d4af640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a6815ca80 0x7f8a681f1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:56.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.002+0000 7f8a6d4af640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a680ca870 con 0x7f8a6815ca80 2023-09-23T03:15:56.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.002+0000 7f8a6d4af640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a681f6780 con 0x7f8a680d25b0 2023-09-23T03:15:56.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.002+0000 7f8a6d4af640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a680caaa0 con 0x7f8a680b2620 2023-09-23T03:15:56.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.002+0000 7f8a677fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a6815ca80 0x7f8a681f1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:56.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.002+0000 7f8a677fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a6815ca80 0x7f8a681f1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44578/0 (socket says 172.21.15.154:44578) 2023-09-23T03:15:56.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.002+0000 7f8a677fe640 1 -- 172.21.15.154:0/571236487 learned_addr learned my addr 172.21.15.154:0/571236487 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:56.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.003+0000 7f8a667fc640 1 --2- 172.21.15.154:0/571236487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a680d25b0 0x7f8a680ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:56.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.003+0000 7f8a66ffd640 1 --2- 172.21.15.154:0/571236487 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a680b2620 0x7f8a681f2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:56.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.003+0000 7f8a677fe640 1 -- 172.21.15.154:0/571236487 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a680b2620 msgr2=0x7f8a681f2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:56.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.003+0000 7f8a677fe640 1 --2- 172.21.15.154:0/571236487 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a680b2620 0x7f8a681f2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:56.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.003+0000 7f8a677fe640 1 -- 172.21.15.154:0/571236487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a680d25b0 msgr2=0x7f8a680ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:56.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.003+0000 7f8a677fe640 1 --2- 172.21.15.154:0/571236487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a680d25b0 0x7f8a680ca330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:56.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.003+0000 7f8a677fe640 1 -- 172.21.15.154:0/571236487 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a50009400 con 0x7f8a6815ca80 2023-09-23T03:15:56.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.003+0000 7f8a66ffd640 1 --2- 172.21.15.154:0/571236487 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a680b2620 0x7f8a681f2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:56.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.003+0000 7f8a677fe640 1 --2- 172.21.15.154:0/571236487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a6815ca80 0x7f8a681f1bf0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000c3c0 tx=0x7f8a6000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:56.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.003+0000 7f8a5f7fe640 1 -- 172.21.15.154:0/571236487 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a60005ce0 con 0x7f8a6815ca80 2023-09-23T03:15:56.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.003+0000 7f8a5f7fe640 1 -- 172.21.15.154:0/571236487 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a60005e40 con 0x7f8a6815ca80 2023-09-23T03:15:56.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.003+0000 7f8a5f7fe640 1 -- 172.21.15.154:0/571236487 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a60007310 con 0x7f8a6815ca80 2023-09-23T03:15:56.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.003+0000 7f8a6d4af640 1 -- 172.21.15.154:0/571236487 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a681f62f0 con 0x7f8a6815ca80 2023-09-23T03:15:56.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.004+0000 7f8a6d4af640 1 -- 172.21.15.154:0/571236487 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a681f59c0 con 0x7f8a6815ca80 2023-09-23T03:15:56.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.005+0000 7f8a6d4af640 1 -- 172.21.15.154:0/571236487 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a68073eb0 con 0x7f8a6815ca80 2023-09-23T03:15:56.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.005+0000 7f8a5f7fe640 1 -- 172.21.15.154:0/571236487 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f8a60007470 con 0x7f8a6815ca80 2023-09-23T03:15:56.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.006+0000 7f8a5f7fe640 1 --2- 172.21.15.154:0/571236487 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8a4806af30 0x7f8a4806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:56.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.006+0000 7f8a5f7fe640 1 -- 172.21.15.154:0/571236487 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8a60089060 con 0x7f8a6815ca80 2023-09-23T03:15:56.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.007+0000 7f8a66ffd640 1 --2- 172.21.15.154:0/571236487 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8a4806af30 0x7f8a4806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:56.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.007+0000 7f8a66ffd640 1 --2- 172.21.15.154:0/571236487 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8a4806af30 0x7f8a4806d3f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8a58009fd0 tx=0x7f8a5800b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:56.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.010+0000 7f8a5f7fe640 1 -- 172.21.15.154:0/571236487 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f8a600bba50 con 0x7f8a6815ca80 2023-09-23T03:15:56.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.182+0000 7f8a6d4af640 1 -- 172.21.15.154:0/571236487 --> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8a680cd5d0 con 0x7f8a4806af30 2023-09-23T03:15:56.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.209+0000 7f8a5f7fe640 1 -- 172.21.15.154:0/571236487 <== mgr.4110 v2:172.21.15.154:6832/2422652104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345061 (secure 0 0 0) 0x7f8a58016420 con 0x7f8a4806af30 2023-09-23T03:15:56.214 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:15:56.216 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2023-09-23T03:15:56.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.214+0000 7f8a6d4af640 1 -- 172.21.15.154:0/571236487 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8a4806af30 msgr2=0x7f8a4806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:56.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.214+0000 7f8a6d4af640 1 --2- 172.21.15.154:0/571236487 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8a4806af30 0x7f8a4806d3f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8a58009fd0 tx=0x7f8a5800b040 comp rx=0 tx=0).stop 2023-09-23T03:15:56.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.214+0000 7f8a6d4af640 1 -- 172.21.15.154:0/571236487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a6815ca80 msgr2=0x7f8a681f1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:56.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.214+0000 7f8a6d4af640 1 --2- 172.21.15.154:0/571236487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a6815ca80 0x7f8a681f1bf0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000c3c0 tx=0x7f8a6000c880 comp rx=0 tx=0).stop 2023-09-23T03:15:56.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.214+0000 7f8a6d4af640 1 -- 172.21.15.154:0/571236487 shutdown_connections 2023-09-23T03:15:56.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.214+0000 7f8a6d4af640 1 --2- 172.21.15.154:0/571236487 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8a4806af30 0x7f8a4806d3f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:56.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.214+0000 7f8a6d4af640 1 --2- 172.21.15.154:0/571236487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a6815ca80 0x7f8a681f1bf0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:56.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.214+0000 7f8a6d4af640 1 --2- 172.21.15.154:0/571236487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a680d25b0 0x7f8a680ca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:56.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.214+0000 7f8a6d4af640 1 --2- 172.21.15.154:0/571236487 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a680b2620 0x7f8a681f2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:56.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.214+0000 7f8a6d4af640 1 -- 172.21.15.154:0/571236487 >> 172.21.15.154:0/571236487 conn(0x7f8a6809bc60 msgr2=0x7f8a680cd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:56.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.215+0000 7f8a6d4af640 1 -- 172.21.15.154:0/571236487 shutdown_connections 2023-09-23T03:15:56.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.215+0000 7f8a6d4af640 1 -- 172.21.15.154:0/571236487 wait complete. 2023-09-23T03:15:56.234 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2023-09-23T03:15:55.174768+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219416,"kb_used_data":3204,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755304,"statfs":{"total":773094113280,"available":772869431296,"internally_reserved":0,"allocated":3280896,"data_stored":1987235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":221236794},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004687"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.303622+0000","last_change":"2023-09-23T03:15:20.261808+0000","last_active":"2023-09-23T03:15:25.303622+0000","last_peered":"2023-09-23T03:15:25.303622+0000","last_clean":"2023-09-23T03:15:25.303622+0000","last_became_active":"2023-09-23T03:15:20.261512+0000","last_became_peered":"2023-09-23T03:15:20.261512+0000","last_unstale":"2023-09-23T03:15:25.303622+0000","last_undegraded":"2023-09-23T03:15:25.303622+0000","last_fullsized":"2023-09-23T03:15:25.303622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:28:08.970374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.296813+0000","last_change":"2023-09-23T03:15:21.165202+0000","last_active":"2023-09-23T03:15:25.296813+0000","last_peered":"2023-09-23T03:15:25.296813+0000","last_clean":"2023-09-23T03:15:25.296813+0000","last_became_active":"2023-09-23T03:15:21.164933+0000","last_became_peered":"2023-09-23T03:15:21.164933+0000","last_unstale":"2023-09-23T03:15:25.296813+0000","last_undegraded":"2023-09-23T03:15:25.296813+0000","last_fullsized":"2023-09-23T03:15:25.296813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:29:14.943541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.485848+0000","last_change":"2023-09-23T03:15:25.453212+0000","last_active":"2023-09-23T03:15:26.485848+0000","last_peered":"2023-09-23T03:15:26.485848+0000","last_clean":"2023-09-23T03:15:26.485848+0000","last_became_active":"2023-09-23T03:15:25.452429+0000","last_became_peered":"2023-09-23T03:15:25.452429+0000","last_unstale":"2023-09-23T03:15:26.485848+0000","last_undegraded":"2023-09-23T03:15:26.485848+0000","last_fullsized":"2023-09-23T03:15:26.485848+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:28:19.168744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.297194+0000","last_change":"2023-09-23T03:15:20.241249+0000","last_active":"2023-09-23T03:15:25.297194+0000","last_peered":"2023-09-23T03:15:25.297194+0000","last_clean":"2023-09-23T03:15:25.297194+0000","last_became_active":"2023-09-23T03:15:20.240943+0000","last_became_peered":"2023-09-23T03:15:20.240943+0000","last_unstale":"2023-09-23T03:15:25.297194+0000","last_undegraded":"2023-09-23T03:15:25.297194+0000","last_fullsized":"2023-09-23T03:15:25.297194+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:05:48.619034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.303332+0000","last_change":"2023-09-23T03:15:21.172718+0000","last_active":"2023-09-23T03:15:25.303332+0000","last_peered":"2023-09-23T03:15:25.303332+0000","last_clean":"2023-09-23T03:15:25.303332+0000","last_became_active":"2023-09-23T03:15:21.172296+0000","last_became_peered":"2023-09-23T03:15:21.172296+0000","last_unstale":"2023-09-23T03:15:25.303332+0000","last_undegraded":"2023-09-23T03:15:25.303332+0000","last_fullsized":"2023-09-23T03:15:25.303332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:30:31.951544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.137218+0000","last_change":"2023-09-23T03:15:25.330094+0000","last_active":"2023-09-23T03:15:26.137218+0000","last_peered":"2023-09-23T03:15:26.137218+0000","last_clean":"2023-09-23T03:15:26.137218+0000","last_became_active":"2023-09-23T03:15:25.329632+0000","last_became_peered":"2023-09-23T03:15:25.329632+0000","last_unstale":"2023-09-23T03:15:26.137218+0000","last_undegraded":"2023-09-23T03:15:26.137218+0000","last_fullsized":"2023-09-23T03:15:26.137218+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:21:31.656499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.445726+0000","last_change":"2023-09-23T03:15:20.261800+0000","last_active":"2023-09-23T03:15:25.445726+0000","last_peered":"2023-09-23T03:15:25.445726+0000","last_clean":"2023-09-23T03:15:25.445726+0000","last_became_active":"2023-09-23T03:15:20.261459+0000","last_became_peered":"2023-09-23T03:15:20.261459+0000","last_unstale":"2023-09-23T03:15:25.445726+0000","last_undegraded":"2023-09-23T03:15:25.445726+0000","last_fullsized":"2023-09-23T03:15:25.445726+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:30:30.939479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315185+0000","last_change":"2023-09-23T03:15:21.172135+0000","last_active":"2023-09-23T03:15:25.315185+0000","last_peered":"2023-09-23T03:15:25.315185+0000","last_clean":"2023-09-23T03:15:25.315185+0000","last_became_active":"2023-09-23T03:15:21.169373+0000","last_became_peered":"2023-09-23T03:15:21.169373+0000","last_unstale":"2023-09-23T03:15:25.315185+0000","last_undegraded":"2023-09-23T03:15:25.315185+0000","last_fullsized":"2023-09-23T03:15:25.315185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:52:02.570741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.139181+0000","last_change":"2023-09-23T03:15:25.328735+0000","last_active":"2023-09-23T03:15:26.139181+0000","last_peered":"2023-09-23T03:15:26.139181+0000","last_clean":"2023-09-23T03:15:26.139181+0000","last_became_active":"2023-09-23T03:15:25.328423+0000","last_became_peered":"2023-09-23T03:15:25.328423+0000","last_unstale":"2023-09-23T03:15:26.139181+0000","last_undegraded":"2023-09-23T03:15:26.139181+0000","last_fullsized":"2023-09-23T03:15:26.139181+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:25:57.997547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.441936+0000","last_change":"2023-09-23T03:15:20.264630+0000","last_active":"2023-09-23T03:15:25.441936+0000","last_peered":"2023-09-23T03:15:25.441936+0000","last_clean":"2023-09-23T03:15:25.441936+0000","last_became_active":"2023-09-23T03:15:20.264209+0000","last_became_peered":"2023-09-23T03:15:20.264209+0000","last_unstale":"2023-09-23T03:15:25.441936+0000","last_undegraded":"2023-09-23T03:15:25.441936+0000","last_fullsized":"2023-09-23T03:15:25.441936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:41:37.505547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315447+0000","last_change":"2023-09-23T03:15:21.176949+0000","last_active":"2023-09-23T03:15:25.315447+0000","last_peered":"2023-09-23T03:15:25.315447+0000","last_clean":"2023-09-23T03:15:25.315447+0000","last_became_active":"2023-09-23T03:15:21.176593+0000","last_became_peered":"2023-09-23T03:15:21.176593+0000","last_unstale":"2023-09-23T03:15:25.315447+0000","last_undegraded":"2023-09-23T03:15:25.315447+0000","last_fullsized":"2023-09-23T03:15:25.315447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:28:31.810059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.138092+0000","last_change":"2023-09-23T03:15:25.324657+0000","last_active":"2023-09-23T03:15:26.138092+0000","last_peered":"2023-09-23T03:15:26.138092+0000","last_clean":"2023-09-23T03:15:26.138092+0000","last_became_active":"2023-09-23T03:15:25.324107+0000","last_became_peered":"2023-09-23T03:15:25.324107+0000","last_unstale":"2023-09-23T03:15:26.138092+0000","last_undegraded":"2023-09-23T03:15:26.138092+0000","last_fullsized":"2023-09-23T03:15:26.138092+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:30:41.725709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.306037+0000","last_change":"2023-09-23T03:15:21.148790+0000","last_active":"2023-09-23T03:15:25.306037+0000","last_peered":"2023-09-23T03:15:25.306037+0000","last_clean":"2023-09-23T03:15:25.306037+0000","last_became_active":"2023-09-23T03:15:21.148165+0000","last_became_peered":"2023-09-23T03:15:21.148165+0000","last_unstale":"2023-09-23T03:15:25.306037+0000","last_undegraded":"2023-09-23T03:15:25.306037+0000","last_fullsized":"2023-09-23T03:15:25.306037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:22:07.336440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.442912+0000","last_change":"2023-09-23T03:15:20.262080+0000","last_active":"2023-09-23T03:15:25.442912+0000","last_peered":"2023-09-23T03:15:25.442912+0000","last_clean":"2023-09-23T03:15:25.442912+0000","last_became_active":"2023-09-23T03:15:20.261391+0000","last_became_peered":"2023-09-23T03:15:20.261391+0000","last_unstale":"2023-09-23T03:15:25.442912+0000","last_undegraded":"2023-09-23T03:15:25.442912+0000","last_fullsized":"2023-09-23T03:15:25.442912+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:30:36.640364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.140544+0000","last_change":"2023-09-23T03:15:25.324180+0000","last_active":"2023-09-23T03:15:26.140544+0000","last_peered":"2023-09-23T03:15:26.140544+0000","last_clean":"2023-09-23T03:15:26.140544+0000","last_became_active":"2023-09-23T03:15:25.323694+0000","last_became_peered":"2023-09-23T03:15:25.323694+0000","last_unstale":"2023-09-23T03:15:26.140544+0000","last_undegraded":"2023-09-23T03:15:26.140544+0000","last_fullsized":"2023-09-23T03:15:26.140544+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:42:41.516665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.296569+0000","last_change":"2023-09-23T03:15:21.168584+0000","last_active":"2023-09-23T03:15:25.296569+0000","last_peered":"2023-09-23T03:15:25.296569+0000","last_clean":"2023-09-23T03:15:25.296569+0000","last_became_active":"2023-09-23T03:15:21.168238+0000","last_became_peered":"2023-09-23T03:15:21.168238+0000","last_unstale":"2023-09-23T03:15:25.296569+0000","last_undegraded":"2023-09-23T03:15:25.296569+0000","last_fullsized":"2023-09-23T03:15:25.296569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:21:27.583218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.311312+0000","last_change":"2023-09-23T03:15:20.251584+0000","last_active":"2023-09-23T03:15:25.311312+0000","last_peered":"2023-09-23T03:15:25.311312+0000","last_clean":"2023-09-23T03:15:25.311312+0000","last_became_active":"2023-09-23T03:15:20.250687+0000","last_became_peered":"2023-09-23T03:15:20.250687+0000","last_unstale":"2023-09-23T03:15:25.311312+0000","last_undegraded":"2023-09-23T03:15:25.311312+0000","last_fullsized":"2023-09-23T03:15:25.311312+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:01:03.998670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.138838+0000","last_change":"2023-09-23T03:15:25.329231+0000","last_active":"2023-09-23T03:15:26.138838+0000","last_peered":"2023-09-23T03:15:26.138838+0000","last_clean":"2023-09-23T03:15:26.138838+0000","last_became_active":"2023-09-23T03:15:25.328825+0000","last_became_peered":"2023-09-23T03:15:25.328825+0000","last_unstale":"2023-09-23T03:15:26.138838+0000","last_undegraded":"2023-09-23T03:15:26.138838+0000","last_fullsized":"2023-09-23T03:15:26.138838+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:29:19.016822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.295849+0000","last_change":"2023-09-23T03:15:21.162632+0000","last_active":"2023-09-23T03:15:25.295849+0000","last_peered":"2023-09-23T03:15:25.295849+0000","last_clean":"2023-09-23T03:15:25.295849+0000","last_became_active":"2023-09-23T03:15:21.161891+0000","last_became_peered":"2023-09-23T03:15:21.161891+0000","last_unstale":"2023-09-23T03:15:25.295849+0000","last_undegraded":"2023-09-23T03:15:25.295849+0000","last_fullsized":"2023-09-23T03:15:25.295849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:39:10.586582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.311128+0000","last_change":"2023-09-23T03:15:20.265489+0000","last_active":"2023-09-23T03:15:25.311128+0000","last_peered":"2023-09-23T03:15:25.311128+0000","last_clean":"2023-09-23T03:15:25.311128+0000","last_became_active":"2023-09-23T03:15:20.265295+0000","last_became_peered":"2023-09-23T03:15:20.265295+0000","last_unstale":"2023-09-23T03:15:25.311128+0000","last_undegraded":"2023-09-23T03:15:25.311128+0000","last_fullsized":"2023-09-23T03:15:25.311128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:06:15.942585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.619606+0000","last_change":"2023-09-23T03:15:25.450826+0000","last_active":"2023-09-23T03:15:26.619606+0000","last_peered":"2023-09-23T03:15:26.619606+0000","last_clean":"2023-09-23T03:15:26.619606+0000","last_became_active":"2023-09-23T03:15:25.449562+0000","last_became_peered":"2023-09-23T03:15:25.449562+0000","last_unstale":"2023-09-23T03:15:26.619606+0000","last_undegraded":"2023-09-23T03:15:26.619606+0000","last_fullsized":"2023-09-23T03:15:26.619606+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:02:13.107703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.310119+0000","last_change":"2023-09-23T03:15:21.151987+0000","last_active":"2023-09-23T03:15:25.310119+0000","last_peered":"2023-09-23T03:15:25.310119+0000","last_clean":"2023-09-23T03:15:25.310119+0000","last_became_active":"2023-09-23T03:15:21.151421+0000","last_became_peered":"2023-09-23T03:15:21.151421+0000","last_unstale":"2023-09-23T03:15:25.310119+0000","last_undegraded":"2023-09-23T03:15:25.310119+0000","last_fullsized":"2023-09-23T03:15:25.310119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:32:42.587095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.441395+0000","last_change":"2023-09-23T03:15:20.262324+0000","last_active":"2023-09-23T03:15:25.441395+0000","last_peered":"2023-09-23T03:15:25.441395+0000","last_clean":"2023-09-23T03:15:25.441395+0000","last_became_active":"2023-09-23T03:15:20.261456+0000","last_became_peered":"2023-09-23T03:15:20.261456+0000","last_unstale":"2023-09-23T03:15:25.441395+0000","last_undegraded":"2023-09-23T03:15:25.441395+0000","last_fullsized":"2023-09-23T03:15:25.441395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:31:40.244446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.139092+0000","last_change":"2023-09-23T03:15:25.329961+0000","last_active":"2023-09-23T03:15:26.139092+0000","last_peered":"2023-09-23T03:15:26.139092+0000","last_clean":"2023-09-23T03:15:26.139092+0000","last_became_active":"2023-09-23T03:15:25.329525+0000","last_became_peered":"2023-09-23T03:15:25.329525+0000","last_unstale":"2023-09-23T03:15:26.139092+0000","last_undegraded":"2023-09-23T03:15:26.139092+0000","last_fullsized":"2023-09-23T03:15:26.139092+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:39:14.659863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.304127+0000","last_change":"2023-09-23T03:15:21.160274+0000","last_active":"2023-09-23T03:15:25.304127+0000","last_peered":"2023-09-23T03:15:25.304127+0000","last_clean":"2023-09-23T03:15:25.304127+0000","last_became_active":"2023-09-23T03:15:21.160110+0000","last_became_peered":"2023-09-23T03:15:21.160110+0000","last_unstale":"2023-09-23T03:15:25.304127+0000","last_undegraded":"2023-09-23T03:15:25.304127+0000","last_fullsized":"2023-09-23T03:15:25.304127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:15:19.584603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.313804+0000","last_change":"2023-09-23T03:15:20.265791+0000","last_active":"2023-09-23T03:15:25.313804+0000","last_peered":"2023-09-23T03:15:25.313804+0000","last_clean":"2023-09-23T03:15:25.313804+0000","last_became_active":"2023-09-23T03:15:20.265429+0000","last_became_peered":"2023-09-23T03:15:20.265429+0000","last_unstale":"2023-09-23T03:15:25.313804+0000","last_undegraded":"2023-09-23T03:15:25.313804+0000","last_fullsized":"2023-09-23T03:15:25.313804+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:05:48.619034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.138192+0000","last_change":"2023-09-23T03:15:25.451854+0000","last_active":"2023-09-23T03:15:26.138192+0000","last_peered":"2023-09-23T03:15:26.138192+0000","last_clean":"2023-09-23T03:15:26.138192+0000","last_became_active":"2023-09-23T03:15:25.450790+0000","last_became_peered":"2023-09-23T03:15:25.450790+0000","last_unstale":"2023-09-23T03:15:26.138192+0000","last_undegraded":"2023-09-23T03:15:26.138192+0000","last_fullsized":"2023-09-23T03:15:26.138192+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:28:35.883340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.297558+0000","last_change":"2023-09-23T03:15:21.163852+0000","last_active":"2023-09-23T03:15:25.297558+0000","last_peered":"2023-09-23T03:15:25.297558+0000","last_clean":"2023-09-23T03:15:25.297558+0000","last_became_active":"2023-09-23T03:15:21.163332+0000","last_became_peered":"2023-09-23T03:15:21.163332+0000","last_unstale":"2023-09-23T03:15:25.297558+0000","last_undegraded":"2023-09-23T03:15:25.297558+0000","last_fullsized":"2023-09-23T03:15:25.297558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:25:53.924266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.616670+0000","last_change":"2023-09-23T03:15:20.264120+0000","last_active":"2023-09-23T03:15:25.616670+0000","last_peered":"2023-09-23T03:15:25.616670+0000","last_clean":"2023-09-23T03:15:25.616670+0000","last_became_active":"2023-09-23T03:15:20.263598+0000","last_became_peered":"2023-09-23T03:15:20.263598+0000","last_unstale":"2023-09-23T03:15:25.616670+0000","last_undegraded":"2023-09-23T03:15:25.616670+0000","last_fullsized":"2023-09-23T03:15:25.616670+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:31:01.946245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.485429+0000","last_change":"2023-09-23T03:15:25.444557+0000","last_active":"2023-09-23T03:15:26.485429+0000","last_peered":"2023-09-23T03:15:26.485429+0000","last_clean":"2023-09-23T03:15:26.485429+0000","last_became_active":"2023-09-23T03:15:25.444002+0000","last_became_peered":"2023-09-23T03:15:25.444002+0000","last_unstale":"2023-09-23T03:15:26.485429+0000","last_undegraded":"2023-09-23T03:15:26.485429+0000","last_fullsized":"2023-09-23T03:15:26.485429+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:15:29.393607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.311253+0000","last_change":"2023-09-23T03:15:21.155177+0000","last_active":"2023-09-23T03:15:25.311253+0000","last_peered":"2023-09-23T03:15:25.311253+0000","last_clean":"2023-09-23T03:15:25.311253+0000","last_became_active":"2023-09-23T03:15:21.154788+0000","last_became_peered":"2023-09-23T03:15:21.154788+0000","last_unstale":"2023-09-23T03:15:25.311253+0000","last_undegraded":"2023-09-23T03:15:25.311253+0000","last_fullsized":"2023-09-23T03:15:25.311253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:51:00.098552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.446205+0000","last_change":"2023-09-23T03:15:20.261866+0000","last_active":"2023-09-23T03:15:25.446205+0000","last_peered":"2023-09-23T03:15:25.446205+0000","last_clean":"2023-09-23T03:15:25.446205+0000","last_became_active":"2023-09-23T03:15:20.261448+0000","last_became_peered":"2023-09-23T03:15:20.261448+0000","last_unstale":"2023-09-23T03:15:25.446205+0000","last_undegraded":"2023-09-23T03:15:25.446205+0000","last_fullsized":"2023-09-23T03:15:25.446205+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:15:18.572539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.619178+0000","last_change":"2023-09-23T03:15:25.443253+0000","last_active":"2023-09-23T03:15:26.619178+0000","last_peered":"2023-09-23T03:15:26.619178+0000","last_clean":"2023-09-23T03:15:26.619178+0000","last_became_active":"2023-09-23T03:15:25.442252+0000","last_became_peered":"2023-09-23T03:15:25.442252+0000","last_unstale":"2023-09-23T03:15:26.619178+0000","last_undegraded":"2023-09-23T03:15:26.619178+0000","last_fullsized":"2023-09-23T03:15:26.619178+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:53:25.508968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315540+0000","last_change":"2023-09-23T03:15:21.167597+0000","last_active":"2023-09-23T03:15:25.315540+0000","last_peered":"2023-09-23T03:15:25.315540+0000","last_clean":"2023-09-23T03:15:25.315540+0000","last_became_active":"2023-09-23T03:15:21.167306+0000","last_became_peered":"2023-09-23T03:15:21.167306+0000","last_unstale":"2023-09-23T03:15:25.315540+0000","last_undegraded":"2023-09-23T03:15:25.315540+0000","last_fullsized":"2023-09-23T03:15:25.315540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:25:53.924266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.311346+0000","last_change":"2023-09-23T03:15:20.254369+0000","last_active":"2023-09-23T03:15:25.311346+0000","last_peered":"2023-09-23T03:15:25.311346+0000","last_clean":"2023-09-23T03:15:25.311346+0000","last_became_active":"2023-09-23T03:15:20.251723+0000","last_became_peered":"2023-09-23T03:15:20.251723+0000","last_unstale":"2023-09-23T03:15:25.311346+0000","last_undegraded":"2023-09-23T03:15:25.311346+0000","last_fullsized":"2023-09-23T03:15:25.311346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:32:41.575031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.616591+0000","last_change":"2023-09-23T03:15:25.331003+0000","last_active":"2023-09-23T03:15:26.616591+0000","last_peered":"2023-09-23T03:15:26.616591+0000","last_clean":"2023-09-23T03:15:26.616591+0000","last_became_active":"2023-09-23T03:15:25.330351+0000","last_became_peered":"2023-09-23T03:15:25.330351+0000","last_unstale":"2023-09-23T03:15:26.616591+0000","last_undegraded":"2023-09-23T03:15:26.616591+0000","last_fullsized":"2023-09-23T03:15:26.616591+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:42:41.516665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.311073+0000","last_change":"2023-09-23T03:15:21.160153+0000","last_active":"2023-09-23T03:15:25.311073+0000","last_peered":"2023-09-23T03:15:25.311073+0000","last_clean":"2023-09-23T03:15:25.311073+0000","last_became_active":"2023-09-23T03:15:21.159780+0000","last_became_peered":"2023-09-23T03:15:21.159780+0000","last_unstale":"2023-09-23T03:15:25.311073+0000","last_undegraded":"2023-09-23T03:15:25.311073+0000","last_fullsized":"2023-09-23T03:15:25.311073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:31:41.256510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.317480+0000","last_change":"2023-09-23T03:15:20.258357+0000","last_active":"2023-09-23T03:15:25.317480+0000","last_peered":"2023-09-23T03:15:25.317480+0000","last_clean":"2023-09-23T03:15:25.317480+0000","last_became_active":"2023-09-23T03:15:20.258052+0000","last_became_peered":"2023-09-23T03:15:20.258052+0000","last_unstale":"2023-09-23T03:15:25.317480+0000","last_undegraded":"2023-09-23T03:15:25.317480+0000","last_fullsized":"2023-09-23T03:15:25.317480+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:01:03.998670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.618336+0000","last_change":"2023-09-23T03:15:25.326862+0000","last_active":"2023-09-23T03:15:26.618336+0000","last_peered":"2023-09-23T03:15:26.618336+0000","last_clean":"2023-09-23T03:15:26.618336+0000","last_became_active":"2023-09-23T03:15:25.326483+0000","last_became_peered":"2023-09-23T03:15:25.326483+0000","last_unstale":"2023-09-23T03:15:26.618336+0000","last_undegraded":"2023-09-23T03:15:26.618336+0000","last_fullsized":"2023-09-23T03:15:26.618336+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:52:06.644022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.310981+0000","last_change":"2023-09-23T03:15:21.160191+0000","last_active":"2023-09-23T03:15:25.310981+0000","last_peered":"2023-09-23T03:15:25.310981+0000","last_clean":"2023-09-23T03:15:25.310981+0000","last_became_active":"2023-09-23T03:15:21.159841+0000","last_became_peered":"2023-09-23T03:15:21.159841+0000","last_unstale":"2023-09-23T03:15:25.310981+0000","last_undegraded":"2023-09-23T03:15:25.310981+0000","last_fullsized":"2023-09-23T03:15:25.310981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:23:07.611380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.301033+0000","last_change":"2023-09-23T03:15:20.242692+0000","last_active":"2023-09-23T03:15:25.301033+0000","last_peered":"2023-09-23T03:15:25.301033+0000","last_clean":"2023-09-23T03:15:25.301033+0000","last_became_active":"2023-09-23T03:15:20.242383+0000","last_became_peered":"2023-09-23T03:15:20.242383+0000","last_unstale":"2023-09-23T03:15:25.301033+0000","last_undegraded":"2023-09-23T03:15:25.301033+0000","last_fullsized":"2023-09-23T03:15:25.301033+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:35:16.587399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.619952+0000","last_change":"2023-09-23T03:15:25.330531+0000","last_active":"2023-09-23T03:15:26.619952+0000","last_peered":"2023-09-23T03:15:26.619952+0000","last_clean":"2023-09-23T03:15:26.619952+0000","last_became_active":"2023-09-23T03:15:25.330145+0000","last_became_peered":"2023-09-23T03:15:25.330145+0000","last_unstale":"2023-09-23T03:15:26.619952+0000","last_undegraded":"2023-09-23T03:15:26.619952+0000","last_fullsized":"2023-09-23T03:15:26.619952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:44:24.602333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.302012+0000","last_change":"2023-09-23T03:15:21.152726+0000","last_active":"2023-09-23T03:15:25.302012+0000","last_peered":"2023-09-23T03:15:25.302012+0000","last_clean":"2023-09-23T03:15:25.302012+0000","last_became_active":"2023-09-23T03:15:21.151916+0000","last_became_peered":"2023-09-23T03:15:21.151916+0000","last_unstale":"2023-09-23T03:15:25.302012+0000","last_undegraded":"2023-09-23T03:15:25.302012+0000","last_fullsized":"2023-09-23T03:15:25.302012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:05:49.631098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.441189+0000","last_change":"2023-09-23T03:15:20.265672+0000","last_active":"2023-09-23T03:15:25.441189+0000","last_peered":"2023-09-23T03:15:25.441189+0000","last_clean":"2023-09-23T03:15:25.441189+0000","last_became_active":"2023-09-23T03:15:20.265172+0000","last_became_peered":"2023-09-23T03:15:20.265172+0000","last_unstale":"2023-09-23T03:15:25.441189+0000","last_undegraded":"2023-09-23T03:15:25.441189+0000","last_fullsized":"2023-09-23T03:15:25.441189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:28:08.970374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.620019+0000","last_change":"2023-09-23T03:15:25.442450+0000","last_active":"2023-09-23T03:15:26.620019+0000","last_peered":"2023-09-23T03:15:26.620019+0000","last_clean":"2023-09-23T03:15:26.620019+0000","last_became_active":"2023-09-23T03:15:25.441911+0000","last_became_peered":"2023-09-23T03:15:25.441911+0000","last_unstale":"2023-09-23T03:15:26.620019+0000","last_undegraded":"2023-09-23T03:15:26.620019+0000","last_fullsized":"2023-09-23T03:15:26.620019+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:58:07.013813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.295933+0000","last_change":"2023-09-23T03:15:21.171351+0000","last_active":"2023-09-23T03:15:25.295933+0000","last_peered":"2023-09-23T03:15:25.295933+0000","last_clean":"2023-09-23T03:15:25.295933+0000","last_became_active":"2023-09-23T03:15:21.170977+0000","last_became_peered":"2023-09-23T03:15:21.170977+0000","last_unstale":"2023-09-23T03:15:25.295933+0000","last_undegraded":"2023-09-23T03:15:25.295933+0000","last_fullsized":"2023-09-23T03:15:25.295933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:51:00.098552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.314530+0000","last_change":"2023-09-23T03:15:20.258137+0000","last_active":"2023-09-23T03:15:25.314530+0000","last_peered":"2023-09-23T03:15:25.314530+0000","last_clean":"2023-09-23T03:15:25.314530+0000","last_became_active":"2023-09-23T03:15:20.257772+0000","last_became_peered":"2023-09-23T03:15:20.257772+0000","last_unstale":"2023-09-23T03:15:25.314530+0000","last_undegraded":"2023-09-23T03:15:25.314530+0000","last_fullsized":"2023-09-23T03:15:25.314530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:52:01.548454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":169,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.441463+0000","last_change":"2023-09-23T03:15:11.042447+0000","last_active":"2023-09-23T03:15:25.441463+0000","last_peered":"2023-09-23T03:15:25.441463+0000","last_clean":"2023-09-23T03:15:25.441463+0000","last_became_active":"2023-09-23T03:15:11.040322+0000","last_became_peered":"2023-09-23T03:15:11.040322+0000","last_unstale":"2023-09-23T03:15:25.441463+0000","last_undegraded":"2023-09-23T03:15:25.441463+0000","last_fullsized":"2023-09-23T03:15:25.441463+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_clean_scrub_stamp":"2023-09-23T03:15:09.474842+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:18:36.331645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.138753+0000","last_change":"2023-09-23T03:15:25.449566+0000","last_active":"2023-09-23T03:15:26.138753+0000","last_peered":"2023-09-23T03:15:26.138753+0000","last_clean":"2023-09-23T03:15:26.138753+0000","last_became_active":"2023-09-23T03:15:25.448514+0000","last_became_peered":"2023-09-23T03:15:25.448514+0000","last_unstale":"2023-09-23T03:15:26.138753+0000","last_undegraded":"2023-09-23T03:15:26.138753+0000","last_fullsized":"2023-09-23T03:15:26.138753+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:52:06.644022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.442985+0000","last_change":"2023-09-23T03:15:21.180427+0000","last_active":"2023-09-23T03:15:25.442985+0000","last_peered":"2023-09-23T03:15:25.442985+0000","last_clean":"2023-09-23T03:15:25.442985+0000","last_became_active":"2023-09-23T03:15:21.179941+0000","last_became_peered":"2023-09-23T03:15:21.179941+0000","last_unstale":"2023-09-23T03:15:25.442985+0000","last_undegraded":"2023-09-23T03:15:25.442985+0000","last_fullsized":"2023-09-23T03:15:25.442985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:53:21.435687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.302809+0000","last_change":"2023-09-23T03:15:20.259280+0000","last_active":"2023-09-23T03:15:25.302809+0000","last_peered":"2023-09-23T03:15:25.302809+0000","last_clean":"2023-09-23T03:15:25.302809+0000","last_became_active":"2023-09-23T03:15:20.258990+0000","last_became_peered":"2023-09-23T03:15:20.258990+0000","last_unstale":"2023-09-23T03:15:25.302809+0000","last_undegraded":"2023-09-23T03:15:25.302809+0000","last_fullsized":"2023-09-23T03:15:25.302809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:32:41.575031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.137662+0000","last_change":"2023-09-23T03:15:25.448939+0000","last_active":"2023-09-23T03:15:26.137662+0000","last_peered":"2023-09-23T03:15:26.137662+0000","last_clean":"2023-09-23T03:15:26.137662+0000","last_became_active":"2023-09-23T03:15:25.448129+0000","last_became_peered":"2023-09-23T03:15:25.448129+0000","last_unstale":"2023-09-23T03:15:26.137662+0000","last_undegraded":"2023-09-23T03:15:26.137662+0000","last_fullsized":"2023-09-23T03:15:26.137662+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:42:41.516665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315261+0000","last_change":"2023-09-23T03:15:21.172056+0000","last_active":"2023-09-23T03:15:25.315261+0000","last_peered":"2023-09-23T03:15:25.315261+0000","last_clean":"2023-09-23T03:15:25.315261+0000","last_became_active":"2023-09-23T03:15:21.170669+0000","last_became_peered":"2023-09-23T03:15:21.170669+0000","last_unstale":"2023-09-23T03:15:25.315261+0000","last_undegraded":"2023-09-23T03:15:25.315261+0000","last_fullsized":"2023-09-23T03:15:25.315261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:02:09.034422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.312606+0000","last_change":"2023-09-23T03:15:20.254984+0000","last_active":"2023-09-23T03:15:25.312606+0000","last_peered":"2023-09-23T03:15:25.312606+0000","last_clean":"2023-09-23T03:15:25.312606+0000","last_became_active":"2023-09-23T03:15:20.254425+0000","last_became_peered":"2023-09-23T03:15:20.254425+0000","last_unstale":"2023-09-23T03:15:25.312606+0000","last_undegraded":"2023-09-23T03:15:25.312606+0000","last_fullsized":"2023-09-23T03:15:25.312606+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:13:21.067891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.619714+0000","last_change":"2023-09-23T03:15:25.324169+0000","last_active":"2023-09-23T03:15:26.619714+0000","last_peered":"2023-09-23T03:15:26.619714+0000","last_clean":"2023-09-23T03:15:26.619714+0000","last_became_active":"2023-09-23T03:15:25.323647+0000","last_became_peered":"2023-09-23T03:15:25.323647+0000","last_unstale":"2023-09-23T03:15:26.619714+0000","last_undegraded":"2023-09-23T03:15:26.619714+0000","last_fullsized":"2023-09-23T03:15:26.619714+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:44:24.602333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.446178+0000","last_change":"2023-09-23T03:15:21.175250+0000","last_active":"2023-09-23T03:15:25.446178+0000","last_peered":"2023-09-23T03:15:25.446178+0000","last_clean":"2023-09-23T03:15:25.446178+0000","last_became_active":"2023-09-23T03:15:21.174890+0000","last_became_peered":"2023-09-23T03:15:21.174890+0000","last_unstale":"2023-09-23T03:15:25.446178+0000","last_undegraded":"2023-09-23T03:15:25.446178+0000","last_fullsized":"2023-09-23T03:15:25.446178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:03:55.070789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.301471+0000","last_change":"2023-09-23T03:15:20.242773+0000","last_active":"2023-09-23T03:15:25.301471+0000","last_peered":"2023-09-23T03:15:25.301471+0000","last_clean":"2023-09-23T03:15:25.301471+0000","last_became_active":"2023-09-23T03:15:20.242466+0000","last_became_peered":"2023-09-23T03:15:20.242466+0000","last_unstale":"2023-09-23T03:15:25.301471+0000","last_undegraded":"2023-09-23T03:15:25.301471+0000","last_fullsized":"2023-09-23T03:15:25.301471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:52:01.548454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.617142+0000","last_change":"2023-09-23T03:15:25.452683+0000","last_active":"2023-09-23T03:15:26.617142+0000","last_peered":"2023-09-23T03:15:26.617142+0000","last_clean":"2023-09-23T03:15:26.617142+0000","last_became_active":"2023-09-23T03:15:25.451509+0000","last_became_peered":"2023-09-23T03:15:25.451509+0000","last_unstale":"2023-09-23T03:15:26.617142+0000","last_undegraded":"2023-09-23T03:15:26.617142+0000","last_fullsized":"2023-09-23T03:15:26.617142+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:03:59.144070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.443716+0000","last_change":"2023-09-23T03:15:21.177994+0000","last_active":"2023-09-23T03:15:25.443716+0000","last_peered":"2023-09-23T03:15:25.443716+0000","last_clean":"2023-09-23T03:15:25.443716+0000","last_became_active":"2023-09-23T03:15:21.177752+0000","last_became_peered":"2023-09-23T03:15:21.177752+0000","last_unstale":"2023-09-23T03:15:25.443716+0000","last_undegraded":"2023-09-23T03:15:25.443716+0000","last_fullsized":"2023-09-23T03:15:25.443716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:52:02.570741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.309999+0000","last_change":"2023-09-23T03:15:20.245089+0000","last_active":"2023-09-23T03:15:25.309999+0000","last_peered":"2023-09-23T03:15:25.309999+0000","last_clean":"2023-09-23T03:15:25.309999+0000","last_became_active":"2023-09-23T03:15:20.243281+0000","last_became_peered":"2023-09-23T03:15:20.243281+0000","last_unstale":"2023-09-23T03:15:25.309999+0000","last_undegraded":"2023-09-23T03:15:25.309999+0000","last_fullsized":"2023-09-23T03:15:25.309999+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:34:50.786275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.137112+0000","last_change":"2023-09-23T03:15:25.327837+0000","last_active":"2023-09-23T03:15:26.137112+0000","last_peered":"2023-09-23T03:15:26.137112+0000","last_clean":"2023-09-23T03:15:26.137112+0000","last_became_active":"2023-09-23T03:15:25.327245+0000","last_became_peered":"2023-09-23T03:15:25.327245+0000","last_unstale":"2023-09-23T03:15:26.137112+0000","last_undegraded":"2023-09-23T03:15:26.137112+0000","last_fullsized":"2023-09-23T03:15:26.137112+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:15:23.657884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.441989+0000","last_change":"2023-09-23T03:15:21.173588+0000","last_active":"2023-09-23T03:15:25.441989+0000","last_peered":"2023-09-23T03:15:25.441989+0000","last_clean":"2023-09-23T03:15:25.441989+0000","last_became_active":"2023-09-23T03:15:21.173284+0000","last_became_peered":"2023-09-23T03:15:21.173284+0000","last_unstale":"2023-09-23T03:15:25.441989+0000","last_undegraded":"2023-09-23T03:15:25.441989+0000","last_fullsized":"2023-09-23T03:15:25.441989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:43:35.707241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.302930+0000","last_change":"2023-09-23T03:15:20.249033+0000","last_active":"2023-09-23T03:15:25.302930+0000","last_peered":"2023-09-23T03:15:25.302930+0000","last_clean":"2023-09-23T03:15:25.302930+0000","last_became_active":"2023-09-23T03:15:20.248738+0000","last_became_peered":"2023-09-23T03:15:20.248738+0000","last_unstale":"2023-09-23T03:15:25.302930+0000","last_undegraded":"2023-09-23T03:15:25.302930+0000","last_fullsized":"2023-09-23T03:15:25.302930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:34:53.736954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.619712+0000","last_change":"2023-09-23T03:15:25.328800+0000","last_active":"2023-09-23T03:15:26.619712+0000","last_peered":"2023-09-23T03:15:26.619712+0000","last_clean":"2023-09-23T03:15:26.619712+0000","last_became_active":"2023-09-23T03:15:25.327349+0000","last_became_peered":"2023-09-23T03:15:25.327349+0000","last_unstale":"2023-09-23T03:15:26.619712+0000","last_undegraded":"2023-09-23T03:15:26.619712+0000","last_fullsized":"2023-09-23T03:15:26.619712+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:02:13.107703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.306109+0000","last_change":"2023-09-23T03:15:21.156710+0000","last_active":"2023-09-23T03:15:25.306109+0000","last_peered":"2023-09-23T03:15:25.306109+0000","last_clean":"2023-09-23T03:15:25.306109+0000","last_became_active":"2023-09-23T03:15:21.156344+0000","last_became_peered":"2023-09-23T03:15:21.156344+0000","last_unstale":"2023-09-23T03:15:25.306109+0000","last_undegraded":"2023-09-23T03:15:25.306109+0000","last_fullsized":"2023-09-23T03:15:25.306109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:01:05.010735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.307353+0000","last_change":"2023-09-23T03:15:20.254568+0000","last_active":"2023-09-23T03:15:25.307353+0000","last_peered":"2023-09-23T03:15:25.307353+0000","last_clean":"2023-09-23T03:15:25.307353+0000","last_became_active":"2023-09-23T03:15:20.254294+0000","last_became_peered":"2023-09-23T03:15:20.254294+0000","last_unstale":"2023-09-23T03:15:25.307353+0000","last_undegraded":"2023-09-23T03:15:25.307353+0000","last_fullsized":"2023-09-23T03:15:25.307353+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:32:41.575031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.487038+0000","last_change":"2023-09-23T03:15:25.444588+0000","last_active":"2023-09-23T03:15:26.487038+0000","last_peered":"2023-09-23T03:15:26.487038+0000","last_clean":"2023-09-23T03:15:26.487038+0000","last_became_active":"2023-09-23T03:15:25.443847+0000","last_became_peered":"2023-09-23T03:15:25.443847+0000","last_unstale":"2023-09-23T03:15:26.487038+0000","last_undegraded":"2023-09-23T03:15:26.487038+0000","last_fullsized":"2023-09-23T03:15:26.487038+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:54:37.479094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.441909+0000","last_change":"2023-09-23T03:15:21.172873+0000","last_active":"2023-09-23T03:15:25.441909+0000","last_peered":"2023-09-23T03:15:25.441909+0000","last_clean":"2023-09-23T03:15:25.441909+0000","last_became_active":"2023-09-23T03:15:21.172552+0000","last_became_peered":"2023-09-23T03:15:21.172552+0000","last_unstale":"2023-09-23T03:15:25.441909+0000","last_undegraded":"2023-09-23T03:15:25.441909+0000","last_fullsized":"2023-09-23T03:15:25.441909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:42:37.443384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.317383+0000","last_change":"2023-09-23T03:15:20.257041+0000","last_active":"2023-09-23T03:15:25.317383+0000","last_peered":"2023-09-23T03:15:25.317383+0000","last_clean":"2023-09-23T03:15:25.317383+0000","last_became_active":"2023-09-23T03:15:20.256737+0000","last_became_peered":"2023-09-23T03:15:20.256737+0000","last_unstale":"2023-09-23T03:15:25.317383+0000","last_undegraded":"2023-09-23T03:15:25.317383+0000","last_fullsized":"2023-09-23T03:15:25.317383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:10:06.381357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.486900+0000","last_change":"2023-09-23T03:15:25.445742+0000","last_active":"2023-09-23T03:15:26.486900+0000","last_peered":"2023-09-23T03:15:26.486900+0000","last_clean":"2023-09-23T03:15:26.486900+0000","last_became_active":"2023-09-23T03:15:25.445297+0000","last_became_peered":"2023-09-23T03:15:25.445297+0000","last_unstale":"2023-09-23T03:15:26.486900+0000","last_undegraded":"2023-09-23T03:15:26.486900+0000","last_fullsized":"2023-09-23T03:15:26.486900+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:36:02.944327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.301638+0000","last_change":"2023-09-23T03:15:21.166987+0000","last_active":"2023-09-23T03:15:25.301638+0000","last_peered":"2023-09-23T03:15:25.301638+0000","last_clean":"2023-09-23T03:15:25.301638+0000","last_became_active":"2023-09-23T03:15:21.166575+0000","last_became_peered":"2023-09-23T03:15:21.166575+0000","last_unstale":"2023-09-23T03:15:25.301638+0000","last_undegraded":"2023-09-23T03:15:25.301638+0000","last_fullsized":"2023-09-23T03:15:25.301638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:10:07.393421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.305489+0000","last_change":"2023-09-23T03:15:20.253573+0000","last_active":"2023-09-23T03:15:25.305489+0000","last_peered":"2023-09-23T03:15:25.305489+0000","last_clean":"2023-09-23T03:15:25.305489+0000","last_became_active":"2023-09-23T03:15:20.253228+0000","last_became_peered":"2023-09-23T03:15:20.253228+0000","last_unstale":"2023-09-23T03:15:25.305489+0000","last_undegraded":"2023-09-23T03:15:25.305489+0000","last_fullsized":"2023-09-23T03:15:25.305489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:35:16.587399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.619960+0000","last_change":"2023-09-23T03:15:25.453715+0000","last_active":"2023-09-23T03:15:26.619960+0000","last_peered":"2023-09-23T03:15:26.619960+0000","last_clean":"2023-09-23T03:15:26.619960+0000","last_became_active":"2023-09-23T03:15:25.452979+0000","last_became_peered":"2023-09-23T03:15:25.452979+0000","last_unstale":"2023-09-23T03:15:26.619960+0000","last_undegraded":"2023-09-23T03:15:26.619960+0000","last_fullsized":"2023-09-23T03:15:26.619960+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:15:29.393607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.442638+0000","last_change":"2023-09-23T03:15:21.169871+0000","last_active":"2023-09-23T03:15:25.442638+0000","last_peered":"2023-09-23T03:15:25.442638+0000","last_clean":"2023-09-23T03:15:25.442638+0000","last_became_active":"2023-09-23T03:15:21.169539+0000","last_became_peered":"2023-09-23T03:15:21.169539+0000","last_unstale":"2023-09-23T03:15:25.442638+0000","last_undegraded":"2023-09-23T03:15:25.442638+0000","last_fullsized":"2023-09-23T03:15:25.442638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:44:20.529052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:40.246064+0000","last_change":"2023-09-23T03:15:20.252281+0000","last_active":"2023-09-23T03:15:40.246064+0000","last_peered":"2023-09-23T03:15:40.246064+0000","last_clean":"2023-09-23T03:15:40.246064+0000","last_became_active":"2023-09-23T03:15:20.251982+0000","last_became_peered":"2023-09-23T03:15:20.251982+0000","last_unstale":"2023-09-23T03:15:40.246064+0000","last_undegraded":"2023-09-23T03:15:40.246064+0000","last_fullsized":"2023-09-23T03:15:40.246064+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:35:16.587399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.616880+0000","last_change":"2023-09-23T03:15:25.325976+0000","last_active":"2023-09-23T03:15:26.616880+0000","last_peered":"2023-09-23T03:15:26.616880+0000","last_clean":"2023-09-23T03:15:26.616880+0000","last_became_active":"2023-09-23T03:15:25.325613+0000","last_became_peered":"2023-09-23T03:15:25.325613+0000","last_unstale":"2023-09-23T03:15:26.616880+0000","last_undegraded":"2023-09-23T03:15:26.616880+0000","last_fullsized":"2023-09-23T03:15:26.616880+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:53:25.508968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.309427+0000","last_change":"2023-09-23T03:15:21.166420+0000","last_active":"2023-09-23T03:15:25.309427+0000","last_peered":"2023-09-23T03:15:25.309427+0000","last_clean":"2023-09-23T03:15:25.309427+0000","last_became_active":"2023-09-23T03:15:21.166086+0000","last_became_peered":"2023-09-23T03:15:21.166086+0000","last_unstale":"2023-09-23T03:15:25.309427+0000","last_undegraded":"2023-09-23T03:15:25.309427+0000","last_fullsized":"2023-09-23T03:15:25.309427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:13:22.079955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.312418+0000","last_change":"2023-09-23T03:15:20.263920+0000","last_active":"2023-09-23T03:15:25.312418+0000","last_peered":"2023-09-23T03:15:25.312418+0000","last_clean":"2023-09-23T03:15:25.312418+0000","last_became_active":"2023-09-23T03:15:20.263623+0000","last_became_peered":"2023-09-23T03:15:20.263623+0000","last_unstale":"2023-09-23T03:15:25.312418+0000","last_undegraded":"2023-09-23T03:15:25.312418+0000","last_fullsized":"2023-09-23T03:15:25.312418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:28:08.970374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.139553+0000","last_change":"2023-09-23T03:15:25.329301+0000","last_active":"2023-09-23T03:15:26.139553+0000","last_peered":"2023-09-23T03:15:26.139553+0000","last_clean":"2023-09-23T03:15:26.139553+0000","last_became_active":"2023-09-23T03:15:25.328559+0000","last_became_peered":"2023-09-23T03:15:25.328559+0000","last_unstale":"2023-09-23T03:15:26.139553+0000","last_undegraded":"2023-09-23T03:15:26.139553+0000","last_fullsized":"2023-09-23T03:15:26.139553+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:43:39.780522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.303426+0000","last_change":"2023-09-23T03:15:21.172339+0000","last_active":"2023-09-23T03:15:25.303426+0000","last_peered":"2023-09-23T03:15:25.303426+0000","last_clean":"2023-09-23T03:15:25.303426+0000","last_became_active":"2023-09-23T03:15:21.172034+0000","last_became_peered":"2023-09-23T03:15:21.172034+0000","last_unstale":"2023-09-23T03:15:25.303426+0000","last_undegraded":"2023-09-23T03:15:25.303426+0000","last_fullsized":"2023-09-23T03:15:25.303426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:21:27.583218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315333+0000","last_change":"2023-09-23T03:15:20.259342+0000","last_active":"2023-09-23T03:15:25.315333+0000","last_peered":"2023-09-23T03:15:25.315333+0000","last_clean":"2023-09-23T03:15:25.315333+0000","last_became_active":"2023-09-23T03:15:20.258990+0000","last_became_peered":"2023-09-23T03:15:20.258990+0000","last_unstale":"2023-09-23T03:15:25.315333+0000","last_undegraded":"2023-09-23T03:15:25.315333+0000","last_fullsized":"2023-09-23T03:15:25.315333+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:50:59.086488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.485843+0000","last_change":"2023-09-23T03:15:25.322201+0000","last_active":"2023-09-23T03:15:26.485843+0000","last_peered":"2023-09-23T03:15:26.485843+0000","last_clean":"2023-09-23T03:15:26.485843+0000","last_became_active":"2023-09-23T03:15:25.321486+0000","last_became_peered":"2023-09-23T03:15:25.321486+0000","last_unstale":"2023-09-23T03:15:26.485843+0000","last_undegraded":"2023-09-23T03:15:26.485843+0000","last_fullsized":"2023-09-23T03:15:26.485843+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:20:20.193299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315253+0000","last_change":"2023-09-23T03:15:21.175465+0000","last_active":"2023-09-23T03:15:25.315253+0000","last_peered":"2023-09-23T03:15:25.315253+0000","last_clean":"2023-09-23T03:15:25.315253+0000","last_became_active":"2023-09-23T03:15:21.171773+0000","last_became_peered":"2023-09-23T03:15:21.171773+0000","last_unstale":"2023-09-23T03:15:25.315253+0000","last_undegraded":"2023-09-23T03:15:25.315253+0000","last_fullsized":"2023-09-23T03:15:25.315253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:39:10.586582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.313666+0000","last_change":"2023-09-23T03:15:20.259404+0000","last_active":"2023-09-23T03:15:25.313666+0000","last_peered":"2023-09-23T03:15:25.313666+0000","last_clean":"2023-09-23T03:15:25.313666+0000","last_became_active":"2023-09-23T03:15:20.259013+0000","last_became_peered":"2023-09-23T03:15:20.259013+0000","last_unstale":"2023-09-23T03:15:25.313666+0000","last_undegraded":"2023-09-23T03:15:25.313666+0000","last_fullsized":"2023-09-23T03:15:25.313666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:41:37.505547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.485703+0000","last_change":"2023-09-23T03:15:25.452370+0000","last_active":"2023-09-23T03:15:26.485703+0000","last_peered":"2023-09-23T03:15:26.485703+0000","last_clean":"2023-09-23T03:15:26.485703+0000","last_became_active":"2023-09-23T03:15:25.451873+0000","last_became_peered":"2023-09-23T03:15:25.451873+0000","last_unstale":"2023-09-23T03:15:26.485703+0000","last_undegraded":"2023-09-23T03:15:26.485703+0000","last_fullsized":"2023-09-23T03:15:26.485703+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:58:52.853121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.619356+0000","last_change":"2023-09-23T03:15:25.452394+0000","last_active":"2023-09-23T03:15:26.619356+0000","last_peered":"2023-09-23T03:15:26.619356+0000","last_clean":"2023-09-23T03:15:26.619356+0000","last_became_active":"2023-09-23T03:15:25.451251+0000","last_became_peered":"2023-09-23T03:15:25.451251+0000","last_unstale":"2023-09-23T03:15:26.619356+0000","last_undegraded":"2023-09-23T03:15:26.619356+0000","last_fullsized":"2023-09-23T03:15:26.619356+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:52:06.644022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.310954+0000","last_change":"2023-09-23T03:15:21.170461+0000","last_active":"2023-09-23T03:15:25.310954+0000","last_peered":"2023-09-23T03:15:25.310954+0000","last_clean":"2023-09-23T03:15:25.310954+0000","last_became_active":"2023-09-23T03:15:21.170095+0000","last_became_peered":"2023-09-23T03:15:21.170095+0000","last_unstale":"2023-09-23T03:15:25.310954+0000","last_undegraded":"2023-09-23T03:15:25.310954+0000","last_fullsized":"2023-09-23T03:15:25.310954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:30:31.951544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.314895+0000","last_change":"2023-09-23T03:15:20.257662+0000","last_active":"2023-09-23T03:15:25.314895+0000","last_peered":"2023-09-23T03:15:25.314895+0000","last_clean":"2023-09-23T03:15:25.314895+0000","last_became_active":"2023-09-23T03:15:20.257376+0000","last_became_peered":"2023-09-23T03:15:20.257376+0000","last_unstale":"2023-09-23T03:15:25.314895+0000","last_undegraded":"2023-09-23T03:15:25.314895+0000","last_fullsized":"2023-09-23T03:15:25.314895+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:06:15.942585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.138746+0000","last_change":"2023-09-23T03:15:25.326388+0000","last_active":"2023-09-23T03:15:26.138746+0000","last_peered":"2023-09-23T03:15:26.138746+0000","last_clean":"2023-09-23T03:15:26.138746+0000","last_became_active":"2023-09-23T03:15:25.326018+0000","last_became_peered":"2023-09-23T03:15:25.326018+0000","last_unstale":"2023-09-23T03:15:26.138746+0000","last_undegraded":"2023-09-23T03:15:26.138746+0000","last_fullsized":"2023-09-23T03:15:26.138746+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:53:25.508968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.295747+0000","last_change":"2023-09-23T03:15:21.164139+0000","last_active":"2023-09-23T03:15:25.295747+0000","last_peered":"2023-09-23T03:15:25.295747+0000","last_clean":"2023-09-23T03:15:25.295747+0000","last_became_active":"2023-09-23T03:15:21.162300+0000","last_became_peered":"2023-09-23T03:15:21.162300+0000","last_unstale":"2023-09-23T03:15:25.295747+0000","last_undegraded":"2023-09-23T03:15:25.295747+0000","last_fullsized":"2023-09-23T03:15:25.295747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:22:07.336440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.298007+0000","last_change":"2023-09-23T03:15:20.256962+0000","last_active":"2023-09-23T03:15:25.298007+0000","last_peered":"2023-09-23T03:15:25.298007+0000","last_clean":"2023-09-23T03:15:25.298007+0000","last_became_active":"2023-09-23T03:15:20.256686+0000","last_became_peered":"2023-09-23T03:15:20.256686+0000","last_unstale":"2023-09-23T03:15:25.298007+0000","last_undegraded":"2023-09-23T03:15:25.298007+0000","last_fullsized":"2023-09-23T03:15:25.298007+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:06:15.942585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.314864+0000","last_change":"2023-09-23T03:15:20.255630+0000","last_active":"2023-09-23T03:15:25.314864+0000","last_peered":"2023-09-23T03:15:25.314864+0000","last_clean":"2023-09-23T03:15:25.314864+0000","last_became_active":"2023-09-23T03:15:20.255212+0000","last_became_peered":"2023-09-23T03:15:20.255212+0000","last_unstale":"2023-09-23T03:15:25.314864+0000","last_undegraded":"2023-09-23T03:15:25.314864+0000","last_fullsized":"2023-09-23T03:15:25.314864+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:31:01.946245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.305281+0000","last_change":"2023-09-23T03:15:21.156720+0000","last_active":"2023-09-23T03:15:25.305281+0000","last_peered":"2023-09-23T03:15:25.305281+0000","last_clean":"2023-09-23T03:15:25.305281+0000","last_became_active":"2023-09-23T03:15:21.156268+0000","last_became_peered":"2023-09-23T03:15:21.156268+0000","last_unstale":"2023-09-23T03:15:25.305281+0000","last_undegraded":"2023-09-23T03:15:25.305281+0000","last_fullsized":"2023-09-23T03:15:25.305281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:51:00.098552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.134456+0000","last_change":"2023-09-23T03:15:25.319168+0000","last_active":"2023-09-23T03:15:26.134456+0000","last_peered":"2023-09-23T03:15:26.134456+0000","last_clean":"2023-09-23T03:15:26.134456+0000","last_became_active":"2023-09-23T03:15:25.318761+0000","last_became_peered":"2023-09-23T03:15:25.318761+0000","last_unstale":"2023-09-23T03:15:26.134456+0000","last_undegraded":"2023-09-23T03:15:26.134456+0000","last_fullsized":"2023-09-23T03:15:26.134456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:05:53.704379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.296655+0000","last_change":"2023-09-23T03:15:20.261879+0000","last_active":"2023-09-23T03:15:25.296655+0000","last_peered":"2023-09-23T03:15:25.296655+0000","last_clean":"2023-09-23T03:15:25.296655+0000","last_became_active":"2023-09-23T03:15:20.261597+0000","last_became_peered":"2023-09-23T03:15:20.261597+0000","last_unstale":"2023-09-23T03:15:25.296655+0000","last_undegraded":"2023-09-23T03:15:25.296655+0000","last_fullsized":"2023-09-23T03:15:25.296655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:41:37.505547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.311031+0000","last_change":"2023-09-23T03:15:21.161920+0000","last_active":"2023-09-23T03:15:25.311031+0000","last_peered":"2023-09-23T03:15:25.311031+0000","last_clean":"2023-09-23T03:15:25.311031+0000","last_became_active":"2023-09-23T03:15:21.161657+0000","last_became_peered":"2023-09-23T03:15:21.161657+0000","last_unstale":"2023-09-23T03:15:25.311031+0000","last_undegraded":"2023-09-23T03:15:25.311031+0000","last_fullsized":"2023-09-23T03:15:25.311031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:35:17.599463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.486488+0000","last_change":"2023-09-23T03:15:25.319176+0000","last_active":"2023-09-23T03:15:26.486488+0000","last_peered":"2023-09-23T03:15:26.486488+0000","last_clean":"2023-09-23T03:15:26.486488+0000","last_became_active":"2023-09-23T03:15:25.318752+0000","last_became_peered":"2023-09-23T03:15:25.318752+0000","last_unstale":"2023-09-23T03:15:26.486488+0000","last_undegraded":"2023-09-23T03:15:26.486488+0000","last_fullsized":"2023-09-23T03:15:26.486488+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:56:54.725774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.305274+0000","last_change":"2023-09-23T03:15:21.157543+0000","last_active":"2023-09-23T03:15:25.305274+0000","last_peered":"2023-09-23T03:15:25.305274+0000","last_clean":"2023-09-23T03:15:25.305274+0000","last_became_active":"2023-09-23T03:15:21.157240+0000","last_became_peered":"2023-09-23T03:15:21.157240+0000","last_unstale":"2023-09-23T03:15:25.305274+0000","last_undegraded":"2023-09-23T03:15:25.305274+0000","last_fullsized":"2023-09-23T03:15:25.305274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:23:07.611380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.300872+0000","last_change":"2023-09-23T03:15:21.176611+0000","last_active":"2023-09-23T03:15:25.300872+0000","last_peered":"2023-09-23T03:15:25.300872+0000","last_clean":"2023-09-23T03:15:25.300872+0000","last_became_active":"2023-09-23T03:15:21.176316+0000","last_became_peered":"2023-09-23T03:15:21.176316+0000","last_unstale":"2023-09-23T03:15:25.300872+0000","last_undegraded":"2023-09-23T03:15:25.300872+0000","last_fullsized":"2023-09-23T03:15:25.300872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:28:09.982438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.316926+0000","last_change":"2023-09-23T03:15:21.166808+0000","last_active":"2023-09-23T03:15:25.316926+0000","last_peered":"2023-09-23T03:15:25.316926+0000","last_clean":"2023-09-23T03:15:25.316926+0000","last_became_active":"2023-09-23T03:15:21.166488+0000","last_became_peered":"2023-09-23T03:15:21.166488+0000","last_unstale":"2023-09-23T03:15:25.316926+0000","last_undegraded":"2023-09-23T03:15:25.316926+0000","last_fullsized":"2023-09-23T03:15:25.316926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:43:35.707241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.305111+0000","last_change":"2023-09-23T03:15:21.162188+0000","last_active":"2023-09-23T03:15:25.305111+0000","last_peered":"2023-09-23T03:15:25.305111+0000","last_clean":"2023-09-23T03:15:25.305111+0000","last_became_active":"2023-09-23T03:15:21.161886+0000","last_became_peered":"2023-09-23T03:15:21.161886+0000","last_unstale":"2023-09-23T03:15:25.305111+0000","last_undegraded":"2023-09-23T03:15:25.305111+0000","last_fullsized":"2023-09-23T03:15:25.305111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:23:07.611380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.296970+0000","last_change":"2023-09-23T03:15:21.161567+0000","last_active":"2023-09-23T03:15:25.296970+0000","last_peered":"2023-09-23T03:15:25.296970+0000","last_clean":"2023-09-23T03:15:25.296970+0000","last_became_active":"2023-09-23T03:15:21.161168+0000","last_became_peered":"2023-09-23T03:15:21.161168+0000","last_unstale":"2023-09-23T03:15:25.296970+0000","last_undegraded":"2023-09-23T03:15:25.296970+0000","last_fullsized":"2023-09-23T03:15:25.296970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:30:37.652428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.311814+0000","last_change":"2023-09-23T03:15:21.164997+0000","last_active":"2023-09-23T03:15:25.311814+0000","last_peered":"2023-09-23T03:15:25.311814+0000","last_clean":"2023-09-23T03:15:25.311814+0000","last_became_active":"2023-09-23T03:15:21.164680+0000","last_became_peered":"2023-09-23T03:15:21.164680+0000","last_unstale":"2023-09-23T03:15:25.311814+0000","last_undegraded":"2023-09-23T03:15:25.311814+0000","last_fullsized":"2023-09-23T03:15:25.311814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:15:19.584603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.316265+0000","last_change":"2023-09-23T03:15:21.164029+0000","last_active":"2023-09-23T03:15:25.316265+0000","last_peered":"2023-09-23T03:15:25.316265+0000","last_clean":"2023-09-23T03:15:25.316265+0000","last_became_active":"2023-09-23T03:15:21.163673+0000","last_became_peered":"2023-09-23T03:15:21.163673+0000","last_unstale":"2023-09-23T03:15:25.316265+0000","last_undegraded":"2023-09-23T03:15:25.316265+0000","last_fullsized":"2023-09-23T03:15:25.316265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:15:19.584603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.297434+0000","last_change":"2023-09-23T03:15:21.163996+0000","last_active":"2023-09-23T03:15:25.297434+0000","last_peered":"2023-09-23T03:15:25.297434+0000","last_clean":"2023-09-23T03:15:25.297434+0000","last_became_active":"2023-09-23T03:15:21.163674+0000","last_became_peered":"2023-09-23T03:15:21.163674+0000","last_unstale":"2023-09-23T03:15:25.297434+0000","last_undegraded":"2023-09-23T03:15:25.297434+0000","last_fullsized":"2023-09-23T03:15:25.297434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:28:31.810059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.314809+0000","last_change":"2023-09-23T03:15:21.171741+0000","last_active":"2023-09-23T03:15:25.314809+0000","last_peered":"2023-09-23T03:15:25.314809+0000","last_clean":"2023-09-23T03:15:25.314809+0000","last_became_active":"2023-09-23T03:15:21.169357+0000","last_became_peered":"2023-09-23T03:15:21.169357+0000","last_unstale":"2023-09-23T03:15:25.314809+0000","last_undegraded":"2023-09-23T03:15:25.314809+0000","last_fullsized":"2023-09-23T03:15:25.314809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:21:27.583218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.297693+0000","last_change":"2023-09-23T03:15:21.163391+0000","last_active":"2023-09-23T03:15:25.297693+0000","last_peered":"2023-09-23T03:15:25.297693+0000","last_clean":"2023-09-23T03:15:25.297693+0000","last_became_active":"2023-09-23T03:15:21.162413+0000","last_became_peered":"2023-09-23T03:15:21.162413+0000","last_unstale":"2023-09-23T03:15:25.297693+0000","last_undegraded":"2023-09-23T03:15:25.297693+0000","last_fullsized":"2023-09-23T03:15:25.297693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:30:31.951544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315898+0000","last_change":"2023-09-23T03:15:21.171540+0000","last_active":"2023-09-23T03:15:25.315898+0000","last_peered":"2023-09-23T03:15:25.315898+0000","last_clean":"2023-09-23T03:15:25.315898+0000","last_became_active":"2023-09-23T03:15:21.167329+0000","last_became_peered":"2023-09-23T03:15:21.167329+0000","last_unstale":"2023-09-23T03:15:25.315898+0000","last_undegraded":"2023-09-23T03:15:25.315898+0000","last_fullsized":"2023-09-23T03:15:25.315898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:29:14.943541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.304917+0000","last_change":"2023-09-23T03:15:21.167575+0000","last_active":"2023-09-23T03:15:25.304917+0000","last_peered":"2023-09-23T03:15:25.304917+0000","last_clean":"2023-09-23T03:15:25.304917+0000","last_became_active":"2023-09-23T03:15:21.167260+0000","last_became_peered":"2023-09-23T03:15:21.167260+0000","last_unstale":"2023-09-23T03:15:25.304917+0000","last_undegraded":"2023-09-23T03:15:25.304917+0000","last_fullsized":"2023-09-23T03:15:25.304917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:01:05.010735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.314454+0000","last_change":"2023-09-23T03:15:21.151218+0000","last_active":"2023-09-23T03:15:25.314454+0000","last_peered":"2023-09-23T03:15:25.314454+0000","last_clean":"2023-09-23T03:15:25.314454+0000","last_became_active":"2023-09-23T03:15:21.150758+0000","last_became_peered":"2023-09-23T03:15:21.150758+0000","last_unstale":"2023-09-23T03:15:25.314454+0000","last_undegraded":"2023-09-23T03:15:25.314454+0000","last_fullsized":"2023-09-23T03:15:25.314454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:51:00.098552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.445962+0000","last_change":"2023-09-23T03:15:21.175081+0000","last_active":"2023-09-23T03:15:25.445962+0000","last_peered":"2023-09-23T03:15:25.445962+0000","last_clean":"2023-09-23T03:15:25.445962+0000","last_became_active":"2023-09-23T03:15:21.174757+0000","last_became_peered":"2023-09-23T03:15:21.174757+0000","last_unstale":"2023-09-23T03:15:25.445962+0000","last_undegraded":"2023-09-23T03:15:25.445962+0000","last_fullsized":"2023-09-23T03:15:25.445962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:02:09.034422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.301326+0000","last_change":"2023-09-23T03:15:21.165007+0000","last_active":"2023-09-23T03:15:25.301326+0000","last_peered":"2023-09-23T03:15:25.301326+0000","last_clean":"2023-09-23T03:15:25.301326+0000","last_became_active":"2023-09-23T03:15:21.164591+0000","last_became_peered":"2023-09-23T03:15:21.164591+0000","last_unstale":"2023-09-23T03:15:25.301326+0000","last_undegraded":"2023-09-23T03:15:25.301326+0000","last_fullsized":"2023-09-23T03:15:25.301326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:13:22.079955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315446+0000","last_change":"2023-09-23T03:15:21.166029+0000","last_active":"2023-09-23T03:15:25.315446+0000","last_peered":"2023-09-23T03:15:25.315446+0000","last_clean":"2023-09-23T03:15:25.315446+0000","last_became_active":"2023-09-23T03:15:21.165298+0000","last_became_peered":"2023-09-23T03:15:21.165298+0000","last_unstale":"2023-09-23T03:15:25.315446+0000","last_undegraded":"2023-09-23T03:15:25.315446+0000","last_fullsized":"2023-09-23T03:15:25.315446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:42:37.443384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.314713+0000","last_change":"2023-09-23T03:15:21.172091+0000","last_active":"2023-09-23T03:15:25.314713+0000","last_peered":"2023-09-23T03:15:25.314713+0000","last_clean":"2023-09-23T03:15:25.314713+0000","last_became_active":"2023-09-23T03:15:21.170229+0000","last_became_peered":"2023-09-23T03:15:21.170229+0000","last_unstale":"2023-09-23T03:15:25.314713+0000","last_undegraded":"2023-09-23T03:15:25.314713+0000","last_fullsized":"2023-09-23T03:15:25.314713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:15:19.584603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.311046+0000","last_change":"2023-09-23T03:15:21.163298+0000","last_active":"2023-09-23T03:15:25.311046+0000","last_peered":"2023-09-23T03:15:25.311046+0000","last_clean":"2023-09-23T03:15:25.311046+0000","last_became_active":"2023-09-23T03:15:21.163000+0000","last_became_peered":"2023-09-23T03:15:21.163000+0000","last_unstale":"2023-09-23T03:15:25.311046+0000","last_undegraded":"2023-09-23T03:15:25.311046+0000","last_fullsized":"2023-09-23T03:15:25.311046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:22:07.336440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.301412+0000","last_change":"2023-09-23T03:15:21.160829+0000","last_active":"2023-09-23T03:15:25.301412+0000","last_peered":"2023-09-23T03:15:25.301412+0000","last_clean":"2023-09-23T03:15:25.301412+0000","last_became_active":"2023-09-23T03:15:21.160567+0000","last_became_peered":"2023-09-23T03:15:21.160567+0000","last_unstale":"2023-09-23T03:15:25.301412+0000","last_undegraded":"2023-09-23T03:15:25.301412+0000","last_fullsized":"2023-09-23T03:15:25.301412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:06:16.954649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.296315+0000","last_change":"2023-09-23T03:15:21.165912+0000","last_active":"2023-09-23T03:15:25.296315+0000","last_peered":"2023-09-23T03:15:25.296315+0000","last_clean":"2023-09-23T03:15:25.296315+0000","last_became_active":"2023-09-23T03:15:21.165634+0000","last_became_peered":"2023-09-23T03:15:21.165634+0000","last_unstale":"2023-09-23T03:15:25.296315+0000","last_undegraded":"2023-09-23T03:15:25.296315+0000","last_fullsized":"2023-09-23T03:15:25.296315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:15:19.584603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.309291+0000","last_change":"2023-09-23T03:15:21.161315+0000","last_active":"2023-09-23T03:15:25.309291+0000","last_peered":"2023-09-23T03:15:25.309291+0000","last_clean":"2023-09-23T03:15:25.309291+0000","last_became_active":"2023-09-23T03:15:21.160978+0000","last_became_peered":"2023-09-23T03:15:21.160978+0000","last_unstale":"2023-09-23T03:15:25.309291+0000","last_undegraded":"2023-09-23T03:15:25.309291+0000","last_fullsized":"2023-09-23T03:15:25.309291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:28:09.982438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.305734+0000","last_change":"2023-09-23T03:15:21.172253+0000","last_active":"2023-09-23T03:15:25.305734+0000","last_peered":"2023-09-23T03:15:25.305734+0000","last_clean":"2023-09-23T03:15:25.305734+0000","last_became_active":"2023-09-23T03:15:21.171987+0000","last_became_peered":"2023-09-23T03:15:21.171987+0000","last_unstale":"2023-09-23T03:15:25.305734+0000","last_undegraded":"2023-09-23T03:15:25.305734+0000","last_fullsized":"2023-09-23T03:15:25.305734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:30:31.951544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.303805+0000","last_change":"2023-09-23T03:15:21.172883+0000","last_active":"2023-09-23T03:15:25.303805+0000","last_peered":"2023-09-23T03:15:25.303805+0000","last_clean":"2023-09-23T03:15:25.303805+0000","last_became_active":"2023-09-23T03:15:21.172404+0000","last_became_peered":"2023-09-23T03:15:21.172404+0000","last_unstale":"2023-09-23T03:15:25.303805+0000","last_undegraded":"2023-09-23T03:15:25.303805+0000","last_fullsized":"2023-09-23T03:15:25.303805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:21:27.583218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.445963+0000","last_change":"2023-09-23T03:15:21.174812+0000","last_active":"2023-09-23T03:15:25.445963+0000","last_peered":"2023-09-23T03:15:25.445963+0000","last_clean":"2023-09-23T03:15:25.445963+0000","last_became_active":"2023-09-23T03:15:21.174241+0000","last_became_peered":"2023-09-23T03:15:21.174241+0000","last_unstale":"2023-09-23T03:15:25.445963+0000","last_undegraded":"2023-09-23T03:15:25.445963+0000","last_fullsized":"2023-09-23T03:15:25.445963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:58:02.940532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.295571+0000","last_change":"2023-09-23T03:15:21.170905+0000","last_active":"2023-09-23T03:15:25.295571+0000","last_peered":"2023-09-23T03:15:25.295571+0000","last_clean":"2023-09-23T03:15:25.295571+0000","last_became_active":"2023-09-23T03:15:21.170570+0000","last_became_peered":"2023-09-23T03:15:21.170570+0000","last_unstale":"2023-09-23T03:15:25.295571+0000","last_undegraded":"2023-09-23T03:15:25.295571+0000","last_fullsized":"2023-09-23T03:15:25.295571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:23:07.611380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.441959+0000","last_change":"2023-09-23T03:15:21.171879+0000","last_active":"2023-09-23T03:15:25.441959+0000","last_peered":"2023-09-23T03:15:25.441959+0000","last_clean":"2023-09-23T03:15:25.441959+0000","last_became_active":"2023-09-23T03:15:21.171624+0000","last_became_peered":"2023-09-23T03:15:21.171624+0000","last_unstale":"2023-09-23T03:15:25.441959+0000","last_undegraded":"2023-09-23T03:15:25.441959+0000","last_fullsized":"2023-09-23T03:15:25.441959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:39:10.586582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315796+0000","last_change":"2023-09-23T03:15:21.177809+0000","last_active":"2023-09-23T03:15:25.315796+0000","last_peered":"2023-09-23T03:15:25.315796+0000","last_clean":"2023-09-23T03:15:25.315796+0000","last_became_active":"2023-09-23T03:15:21.175320+0000","last_became_peered":"2023-09-23T03:15:21.175320+0000","last_unstale":"2023-09-23T03:15:25.315796+0000","last_undegraded":"2023-09-23T03:15:25.315796+0000","last_fullsized":"2023-09-23T03:15:25.315796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:30:37.652428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.441779+0000","last_change":"2023-09-23T03:15:21.174361+0000","last_active":"2023-09-23T03:15:25.441779+0000","last_peered":"2023-09-23T03:15:25.441779+0000","last_clean":"2023-09-23T03:15:25.441779+0000","last_became_active":"2023-09-23T03:15:21.174101+0000","last_became_peered":"2023-09-23T03:15:21.174101+0000","last_unstale":"2023-09-23T03:15:25.441779+0000","last_undegraded":"2023-09-23T03:15:25.441779+0000","last_fullsized":"2023-09-23T03:15:25.441779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:25:53.924266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315094+0000","last_change":"2023-09-23T03:15:21.171837+0000","last_active":"2023-09-23T03:15:25.315094+0000","last_peered":"2023-09-23T03:15:25.315094+0000","last_clean":"2023-09-23T03:15:25.315094+0000","last_became_active":"2023-09-23T03:15:21.169973+0000","last_became_peered":"2023-09-23T03:15:21.169973+0000","last_unstale":"2023-09-23T03:15:25.315094+0000","last_undegraded":"2023-09-23T03:15:25.315094+0000","last_fullsized":"2023-09-23T03:15:25.315094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:53:21.435687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.303645+0000","last_change":"2023-09-23T03:15:21.157118+0000","last_active":"2023-09-23T03:15:25.303645+0000","last_peered":"2023-09-23T03:15:25.303645+0000","last_clean":"2023-09-23T03:15:25.303645+0000","last_became_active":"2023-09-23T03:15:21.156862+0000","last_became_peered":"2023-09-23T03:15:21.156862+0000","last_unstale":"2023-09-23T03:15:25.303645+0000","last_undegraded":"2023-09-23T03:15:25.303645+0000","last_fullsized":"2023-09-23T03:15:25.303645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:22:07.336440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.311650+0000","last_change":"2023-09-23T03:15:21.167539+0000","last_active":"2023-09-23T03:15:25.311650+0000","last_peered":"2023-09-23T03:15:25.311650+0000","last_clean":"2023-09-23T03:15:25.311650+0000","last_became_active":"2023-09-23T03:15:21.167256+0000","last_became_peered":"2023-09-23T03:15:21.167256+0000","last_unstale":"2023-09-23T03:15:25.311650+0000","last_undegraded":"2023-09-23T03:15:25.311650+0000","last_fullsized":"2023-09-23T03:15:25.311650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:21:27.583218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.445518+0000","last_change":"2023-09-23T03:15:21.171479+0000","last_active":"2023-09-23T03:15:25.445518+0000","last_peered":"2023-09-23T03:15:25.445518+0000","last_clean":"2023-09-23T03:15:25.445518+0000","last_became_active":"2023-09-23T03:15:21.171128+0000","last_became_peered":"2023-09-23T03:15:21.171128+0000","last_unstale":"2023-09-23T03:15:25.445518+0000","last_undegraded":"2023-09-23T03:15:25.445518+0000","last_fullsized":"2023-09-23T03:15:25.445518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:38:51.387638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":851968,"data_stored":690730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.504}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7130000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8200000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.383}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0489999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.258}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4410000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574861,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":99870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.409}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.204}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4530000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.52}]}]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":99906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0920000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.345}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.149}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2829999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.548}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.524}]}]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0489999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.419}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2809999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574861,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":99780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1870000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.46}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.284}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.486}]}]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":262144,"data_stored":99940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]}]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":689767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.21}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-23T03:15:56.241 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-23T03:15:56.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.485+0000 7f65590da640 1 Processor -- start 2023-09-23T03:15:56.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.485+0000 7f65590da640 1 -- start start 2023-09-23T03:15:56.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.486+0000 7f65590da640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6554159020 0x7f655406d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:56.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.487+0000 7f65590da640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f65540cac90 0x7f65540cb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:56.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.487+0000 7f65590da640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6554159400 con 0x7f65540cc0e0 2023-09-23T03:15:56.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.487+0000 7f65590da640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65540ada00 con 0x7f65540cac90 2023-09-23T03:15:56.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.487+0000 7f65590da640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65540cb660 con 0x7f6554159020 2023-09-23T03:15:56.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.487+0000 7f655359e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f65540cac90 0x7f65540cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:56.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.487+0000 7f655359e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f65540cac90 0x7f65540cb120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44586/0 (socket says 172.21.15.154:44586) 2023-09-23T03:15:56.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.487+0000 7f655359e640 1 -- 172.21.15.154:0/3226242393 learned_addr learned my addr 172.21.15.154:0/3226242393 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:56.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.487+0000 7f6552d9d640 1 --2- 172.21.15.154:0/3226242393 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6554159020 0x7f655406d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:56.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.487+0000 7f655359e640 1 -- 172.21.15.154:0/3226242393 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6554159020 msgr2=0x7f655406d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:56.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.487+0000 7f655359e640 1 --2- 172.21.15.154:0/3226242393 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6554159020 0x7f655406d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:56.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.487+0000 7f655359e640 1 -- 172.21.15.154:0/3226242393 >> v1:172.21.15.190:6789/0 conn(0x7f65540cc0e0 legacy=0x7f65540b1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:56.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.487+0000 7f655359e640 1 -- 172.21.15.154:0/3226242393 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65540cdaf0 con 0x7f65540cac90 2023-09-23T03:15:56.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.488+0000 7f655359e640 1 --2- 172.21.15.154:0/3226242393 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65540cac90 0x7f65540cb120 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f654c00d150 tx=0x7f654c02e0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c93140c45c7f924 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:56.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.489+0000 7f6551d9b640 1 -- 172.21.15.154:0/3226242393 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c02ee80 con 0x7f65540cac90 2023-09-23T03:15:56.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.489+0000 7f6551d9b640 1 -- 172.21.15.154:0/3226242393 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654c032070 con 0x7f65540cac90 2023-09-23T03:15:56.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.489+0000 7f6551d9b640 1 -- 172.21.15.154:0/3226242393 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c038600 con 0x7f65540cac90 2023-09-23T03:15:56.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.489+0000 7f65590da640 1 -- 172.21.15.154:0/3226242393 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65540cac90 msgr2=0x7f65540cb120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:56.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.489+0000 7f65590da640 1 --2- 172.21.15.154:0/3226242393 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65540cac90 0x7f65540cb120 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f654c00d150 tx=0x7f654c02e0d0 comp rx=0 tx=0).stop 2023-09-23T03:15:56.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.489+0000 7f65590da640 1 -- 172.21.15.154:0/3226242393 shutdown_connections 2023-09-23T03:15:56.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.489+0000 7f65590da640 1 --2- 172.21.15.154:0/3226242393 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65540cac90 0x7f65540cb120 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:56.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.489+0000 7f65590da640 1 --2- 172.21.15.154:0/3226242393 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6554159020 0x7f655406d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:56.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.489+0000 7f65590da640 1 -- 172.21.15.154:0/3226242393 >> 172.21.15.154:0/3226242393 conn(0x7f655406a0f0 msgr2=0x7f655406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:56.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.490+0000 7f65590da640 1 -- 172.21.15.154:0/3226242393 shutdown_connections 2023-09-23T03:15:56.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.490+0000 7f65590da640 1 -- 172.21.15.154:0/3226242393 wait complete. 2023-09-23T03:15:56.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.491+0000 7f65590da640 1 Processor -- start 2023-09-23T03:15:56.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.491+0000 7f65590da640 1 -- start start 2023-09-23T03:15:56.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.492+0000 7f65590da640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65540cac90 0x7f65540af100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:56.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.492+0000 7f65590da640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65540cc0e0 0x7f6554159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:56.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.492+0000 7f65590da640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6554159020 0x7f65540b0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:56.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.492+0000 7f65590da640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6554159b40 con 0x7f65540cc0e0 2023-09-23T03:15:56.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.492+0000 7f65590da640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655415c490 con 0x7f65540cac90 2023-09-23T03:15:56.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.492+0000 7f65590da640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65540af6e0 con 0x7f6554159020 2023-09-23T03:15:56.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.492+0000 7f655259c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65540cc0e0 0x7f6554159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:56.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.492+0000 7f655259c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65540cc0e0 0x7f6554159600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44598/0 (socket says 172.21.15.154:44598) 2023-09-23T03:15:56.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.492+0000 7f655259c640 1 -- 172.21.15.154:0/1138985542 learned_addr learned my addr 172.21.15.154:0/1138985542 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:56.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.493+0000 7f6552d9d640 1 --2- 172.21.15.154:0/1138985542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65540cac90 0x7f65540af100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:56.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.493+0000 7f655359e640 1 --2- 172.21.15.154:0/1138985542 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6554159020 0x7f65540b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:56.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.493+0000 7f655259c640 1 -- 172.21.15.154:0/1138985542 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6554159020 msgr2=0x7f65540b0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:56.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.493+0000 7f655259c640 1 --2- 172.21.15.154:0/1138985542 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6554159020 0x7f65540b0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:56.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.493+0000 7f655259c640 1 -- 172.21.15.154:0/1138985542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65540cac90 msgr2=0x7f65540af100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:56.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.493+0000 7f655259c640 1 --2- 172.21.15.154:0/1138985542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65540cac90 0x7f65540af100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:56.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.493+0000 7f655259c640 1 -- 172.21.15.154:0/1138985542 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f654c00bd40 con 0x7f65540cc0e0 2023-09-23T03:15:56.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.493+0000 7f6552d9d640 1 --2- 172.21.15.154:0/1138985542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65540cac90 0x7f65540af100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:56.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.493+0000 7f655259c640 1 --2- 172.21.15.154:0/1138985542 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65540cc0e0 0x7f6554159600 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f654800c430 tx=0x7f654800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:56.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.493+0000 7f653b7fe640 1 -- 172.21.15.154:0/1138985542 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6548011790 con 0x7f65540cc0e0 2023-09-23T03:15:56.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.493+0000 7f653b7fe640 1 -- 172.21.15.154:0/1138985542 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65480118f0 con 0x7f65540cc0e0 2023-09-23T03:15:56.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.493+0000 7f65590da640 1 -- 172.21.15.154:0/1138985542 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65540af9c0 con 0x7f65540cc0e0 2023-09-23T03:15:56.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.493+0000 7f653b7fe640 1 -- 172.21.15.154:0/1138985542 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6548011ce0 con 0x7f65540cc0e0 2023-09-23T03:15:56.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.494+0000 7f65590da640 1 -- 172.21.15.154:0/1138985542 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65540aff80 con 0x7f65540cc0e0 2023-09-23T03:15:56.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.495+0000 7f65590da640 1 -- 172.21.15.154:0/1138985542 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6554073eb0 con 0x7f65540cc0e0 2023-09-23T03:15:56.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.495+0000 7f653b7fe640 1 -- 172.21.15.154:0/1138985542 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f654801c460 con 0x7f65540cc0e0 2023-09-23T03:15:56.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.496+0000 7f653b7fe640 1 --2- 172.21.15.154:0/1138985542 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f654006af30 0x7f654006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:56.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.496+0000 7f653b7fe640 1 -- 172.21.15.154:0/1138985542 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6548089ad0 con 0x7f65540cc0e0 2023-09-23T03:15:56.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.496+0000 7f6552d9d640 1 --2- 172.21.15.154:0/1138985542 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f654006af30 0x7f654006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:56.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.497+0000 7f6552d9d640 1 --2- 172.21.15.154:0/1138985542 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f654006af30 0x7f654006d3f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6544006fd0 tx=0x7f6544008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:56.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.501+0000 7f653b7fe640 1 -- 172.21.15.154:0/1138985542 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f65480546f0 con 0x7f65540cc0e0 2023-09-23T03:15:56.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.672+0000 7f65590da640 1 -- 172.21.15.154:0/1138985542 --> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f65540cc660 con 0x7f654006af30 2023-09-23T03:15:56.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.697+0000 7f653b7fe640 1 -- 172.21.15.154:0/1138985542 <== mgr.4110 v2:172.21.15.154:6832/2422652104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345061 (secure 0 0 0) 0x7f654400c040 con 0x7f654006af30 2023-09-23T03:15:56.701 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:15:56.705 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2023-09-23T03:15:56.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.705+0000 7f65590da640 1 -- 172.21.15.154:0/1138985542 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f654006af30 msgr2=0x7f654006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:56.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.705+0000 7f65590da640 1 --2- 172.21.15.154:0/1138985542 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f654006af30 0x7f654006d3f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6544006fd0 tx=0x7f6544008040 comp rx=0 tx=0).stop 2023-09-23T03:15:56.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.705+0000 7f65590da640 1 -- 172.21.15.154:0/1138985542 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65540cc0e0 msgr2=0x7f6554159600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:56.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.705+0000 7f65590da640 1 --2- 172.21.15.154:0/1138985542 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65540cc0e0 0x7f6554159600 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f654800c430 tx=0x7f654800c8f0 comp rx=0 tx=0).stop 2023-09-23T03:15:56.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.705+0000 7f65590da640 1 -- 172.21.15.154:0/1138985542 shutdown_connections 2023-09-23T03:15:56.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.705+0000 7f65590da640 1 --2- 172.21.15.154:0/1138985542 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f654006af30 0x7f654006d3f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:56.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.705+0000 7f65590da640 1 --2- 172.21.15.154:0/1138985542 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6554159020 0x7f65540b0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:56.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.705+0000 7f65590da640 1 --2- 172.21.15.154:0/1138985542 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65540cc0e0 0x7f6554159600 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:56.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.705+0000 7f65590da640 1 --2- 172.21.15.154:0/1138985542 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65540cac90 0x7f65540af100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:56.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.705+0000 7f65590da640 1 -- 172.21.15.154:0/1138985542 >> 172.21.15.154:0/1138985542 conn(0x7f655406a0f0 msgr2=0x7f6554070760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:56.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.705+0000 7f65590da640 1 -- 172.21.15.154:0/1138985542 shutdown_connections 2023-09-23T03:15:56.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.705+0000 7f65590da640 1 -- 172.21.15.154:0/1138985542 wait complete. 2023-09-23T03:15:56.725 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2023-09-23T03:15:55.174768+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219416,"kb_used_data":3204,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755304,"statfs":{"total":773094113280,"available":772869431296,"internally_reserved":0,"allocated":3280896,"data_stored":1987235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":221236794},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004687"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.303622+0000","last_change":"2023-09-23T03:15:20.261808+0000","last_active":"2023-09-23T03:15:25.303622+0000","last_peered":"2023-09-23T03:15:25.303622+0000","last_clean":"2023-09-23T03:15:25.303622+0000","last_became_active":"2023-09-23T03:15:20.261512+0000","last_became_peered":"2023-09-23T03:15:20.261512+0000","last_unstale":"2023-09-23T03:15:25.303622+0000","last_undegraded":"2023-09-23T03:15:25.303622+0000","last_fullsized":"2023-09-23T03:15:25.303622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:28:08.970374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.296813+0000","last_change":"2023-09-23T03:15:21.165202+0000","last_active":"2023-09-23T03:15:25.296813+0000","last_peered":"2023-09-23T03:15:25.296813+0000","last_clean":"2023-09-23T03:15:25.296813+0000","last_became_active":"2023-09-23T03:15:21.164933+0000","last_became_peered":"2023-09-23T03:15:21.164933+0000","last_unstale":"2023-09-23T03:15:25.296813+0000","last_undegraded":"2023-09-23T03:15:25.296813+0000","last_fullsized":"2023-09-23T03:15:25.296813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:29:14.943541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.485848+0000","last_change":"2023-09-23T03:15:25.453212+0000","last_active":"2023-09-23T03:15:26.485848+0000","last_peered":"2023-09-23T03:15:26.485848+0000","last_clean":"2023-09-23T03:15:26.485848+0000","last_became_active":"2023-09-23T03:15:25.452429+0000","last_became_peered":"2023-09-23T03:15:25.452429+0000","last_unstale":"2023-09-23T03:15:26.485848+0000","last_undegraded":"2023-09-23T03:15:26.485848+0000","last_fullsized":"2023-09-23T03:15:26.485848+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:28:19.168744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.297194+0000","last_change":"2023-09-23T03:15:20.241249+0000","last_active":"2023-09-23T03:15:25.297194+0000","last_peered":"2023-09-23T03:15:25.297194+0000","last_clean":"2023-09-23T03:15:25.297194+0000","last_became_active":"2023-09-23T03:15:20.240943+0000","last_became_peered":"2023-09-23T03:15:20.240943+0000","last_unstale":"2023-09-23T03:15:25.297194+0000","last_undegraded":"2023-09-23T03:15:25.297194+0000","last_fullsized":"2023-09-23T03:15:25.297194+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:05:48.619034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.303332+0000","last_change":"2023-09-23T03:15:21.172718+0000","last_active":"2023-09-23T03:15:25.303332+0000","last_peered":"2023-09-23T03:15:25.303332+0000","last_clean":"2023-09-23T03:15:25.303332+0000","last_became_active":"2023-09-23T03:15:21.172296+0000","last_became_peered":"2023-09-23T03:15:21.172296+0000","last_unstale":"2023-09-23T03:15:25.303332+0000","last_undegraded":"2023-09-23T03:15:25.303332+0000","last_fullsized":"2023-09-23T03:15:25.303332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:30:31.951544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.137218+0000","last_change":"2023-09-23T03:15:25.330094+0000","last_active":"2023-09-23T03:15:26.137218+0000","last_peered":"2023-09-23T03:15:26.137218+0000","last_clean":"2023-09-23T03:15:26.137218+0000","last_became_active":"2023-09-23T03:15:25.329632+0000","last_became_peered":"2023-09-23T03:15:25.329632+0000","last_unstale":"2023-09-23T03:15:26.137218+0000","last_undegraded":"2023-09-23T03:15:26.137218+0000","last_fullsized":"2023-09-23T03:15:26.137218+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:21:31.656499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.445726+0000","last_change":"2023-09-23T03:15:20.261800+0000","last_active":"2023-09-23T03:15:25.445726+0000","last_peered":"2023-09-23T03:15:25.445726+0000","last_clean":"2023-09-23T03:15:25.445726+0000","last_became_active":"2023-09-23T03:15:20.261459+0000","last_became_peered":"2023-09-23T03:15:20.261459+0000","last_unstale":"2023-09-23T03:15:25.445726+0000","last_undegraded":"2023-09-23T03:15:25.445726+0000","last_fullsized":"2023-09-23T03:15:25.445726+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:30:30.939479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315185+0000","last_change":"2023-09-23T03:15:21.172135+0000","last_active":"2023-09-23T03:15:25.315185+0000","last_peered":"2023-09-23T03:15:25.315185+0000","last_clean":"2023-09-23T03:15:25.315185+0000","last_became_active":"2023-09-23T03:15:21.169373+0000","last_became_peered":"2023-09-23T03:15:21.169373+0000","last_unstale":"2023-09-23T03:15:25.315185+0000","last_undegraded":"2023-09-23T03:15:25.315185+0000","last_fullsized":"2023-09-23T03:15:25.315185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:52:02.570741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.139181+0000","last_change":"2023-09-23T03:15:25.328735+0000","last_active":"2023-09-23T03:15:26.139181+0000","last_peered":"2023-09-23T03:15:26.139181+0000","last_clean":"2023-09-23T03:15:26.139181+0000","last_became_active":"2023-09-23T03:15:25.328423+0000","last_became_peered":"2023-09-23T03:15:25.328423+0000","last_unstale":"2023-09-23T03:15:26.139181+0000","last_undegraded":"2023-09-23T03:15:26.139181+0000","last_fullsized":"2023-09-23T03:15:26.139181+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:25:57.997547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.441936+0000","last_change":"2023-09-23T03:15:20.264630+0000","last_active":"2023-09-23T03:15:25.441936+0000","last_peered":"2023-09-23T03:15:25.441936+0000","last_clean":"2023-09-23T03:15:25.441936+0000","last_became_active":"2023-09-23T03:15:20.264209+0000","last_became_peered":"2023-09-23T03:15:20.264209+0000","last_unstale":"2023-09-23T03:15:25.441936+0000","last_undegraded":"2023-09-23T03:15:25.441936+0000","last_fullsized":"2023-09-23T03:15:25.441936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:41:37.505547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315447+0000","last_change":"2023-09-23T03:15:21.176949+0000","last_active":"2023-09-23T03:15:25.315447+0000","last_peered":"2023-09-23T03:15:25.315447+0000","last_clean":"2023-09-23T03:15:25.315447+0000","last_became_active":"2023-09-23T03:15:21.176593+0000","last_became_peered":"2023-09-23T03:15:21.176593+0000","last_unstale":"2023-09-23T03:15:25.315447+0000","last_undegraded":"2023-09-23T03:15:25.315447+0000","last_fullsized":"2023-09-23T03:15:25.315447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:28:31.810059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.138092+0000","last_change":"2023-09-23T03:15:25.324657+0000","last_active":"2023-09-23T03:15:26.138092+0000","last_peered":"2023-09-23T03:15:26.138092+0000","last_clean":"2023-09-23T03:15:26.138092+0000","last_became_active":"2023-09-23T03:15:25.324107+0000","last_became_peered":"2023-09-23T03:15:25.324107+0000","last_unstale":"2023-09-23T03:15:26.138092+0000","last_undegraded":"2023-09-23T03:15:26.138092+0000","last_fullsized":"2023-09-23T03:15:26.138092+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:30:41.725709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.306037+0000","last_change":"2023-09-23T03:15:21.148790+0000","last_active":"2023-09-23T03:15:25.306037+0000","last_peered":"2023-09-23T03:15:25.306037+0000","last_clean":"2023-09-23T03:15:25.306037+0000","last_became_active":"2023-09-23T03:15:21.148165+0000","last_became_peered":"2023-09-23T03:15:21.148165+0000","last_unstale":"2023-09-23T03:15:25.306037+0000","last_undegraded":"2023-09-23T03:15:25.306037+0000","last_fullsized":"2023-09-23T03:15:25.306037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:22:07.336440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.442912+0000","last_change":"2023-09-23T03:15:20.262080+0000","last_active":"2023-09-23T03:15:25.442912+0000","last_peered":"2023-09-23T03:15:25.442912+0000","last_clean":"2023-09-23T03:15:25.442912+0000","last_became_active":"2023-09-23T03:15:20.261391+0000","last_became_peered":"2023-09-23T03:15:20.261391+0000","last_unstale":"2023-09-23T03:15:25.442912+0000","last_undegraded":"2023-09-23T03:15:25.442912+0000","last_fullsized":"2023-09-23T03:15:25.442912+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:30:36.640364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.140544+0000","last_change":"2023-09-23T03:15:25.324180+0000","last_active":"2023-09-23T03:15:26.140544+0000","last_peered":"2023-09-23T03:15:26.140544+0000","last_clean":"2023-09-23T03:15:26.140544+0000","last_became_active":"2023-09-23T03:15:25.323694+0000","last_became_peered":"2023-09-23T03:15:25.323694+0000","last_unstale":"2023-09-23T03:15:26.140544+0000","last_undegraded":"2023-09-23T03:15:26.140544+0000","last_fullsized":"2023-09-23T03:15:26.140544+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:42:41.516665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.296569+0000","last_change":"2023-09-23T03:15:21.168584+0000","last_active":"2023-09-23T03:15:25.296569+0000","last_peered":"2023-09-23T03:15:25.296569+0000","last_clean":"2023-09-23T03:15:25.296569+0000","last_became_active":"2023-09-23T03:15:21.168238+0000","last_became_peered":"2023-09-23T03:15:21.168238+0000","last_unstale":"2023-09-23T03:15:25.296569+0000","last_undegraded":"2023-09-23T03:15:25.296569+0000","last_fullsized":"2023-09-23T03:15:25.296569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:21:27.583218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.311312+0000","last_change":"2023-09-23T03:15:20.251584+0000","last_active":"2023-09-23T03:15:25.311312+0000","last_peered":"2023-09-23T03:15:25.311312+0000","last_clean":"2023-09-23T03:15:25.311312+0000","last_became_active":"2023-09-23T03:15:20.250687+0000","last_became_peered":"2023-09-23T03:15:20.250687+0000","last_unstale":"2023-09-23T03:15:25.311312+0000","last_undegraded":"2023-09-23T03:15:25.311312+0000","last_fullsized":"2023-09-23T03:15:25.311312+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:01:03.998670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.138838+0000","last_change":"2023-09-23T03:15:25.329231+0000","last_active":"2023-09-23T03:15:26.138838+0000","last_peered":"2023-09-23T03:15:26.138838+0000","last_clean":"2023-09-23T03:15:26.138838+0000","last_became_active":"2023-09-23T03:15:25.328825+0000","last_became_peered":"2023-09-23T03:15:25.328825+0000","last_unstale":"2023-09-23T03:15:26.138838+0000","last_undegraded":"2023-09-23T03:15:26.138838+0000","last_fullsized":"2023-09-23T03:15:26.138838+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:29:19.016822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.295849+0000","last_change":"2023-09-23T03:15:21.162632+0000","last_active":"2023-09-23T03:15:25.295849+0000","last_peered":"2023-09-23T03:15:25.295849+0000","last_clean":"2023-09-23T03:15:25.295849+0000","last_became_active":"2023-09-23T03:15:21.161891+0000","last_became_peered":"2023-09-23T03:15:21.161891+0000","last_unstale":"2023-09-23T03:15:25.295849+0000","last_undegraded":"2023-09-23T03:15:25.295849+0000","last_fullsized":"2023-09-23T03:15:25.295849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:39:10.586582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.311128+0000","last_change":"2023-09-23T03:15:20.265489+0000","last_active":"2023-09-23T03:15:25.311128+0000","last_peered":"2023-09-23T03:15:25.311128+0000","last_clean":"2023-09-23T03:15:25.311128+0000","last_became_active":"2023-09-23T03:15:20.265295+0000","last_became_peered":"2023-09-23T03:15:20.265295+0000","last_unstale":"2023-09-23T03:15:25.311128+0000","last_undegraded":"2023-09-23T03:15:25.311128+0000","last_fullsized":"2023-09-23T03:15:25.311128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:06:15.942585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.619606+0000","last_change":"2023-09-23T03:15:25.450826+0000","last_active":"2023-09-23T03:15:26.619606+0000","last_peered":"2023-09-23T03:15:26.619606+0000","last_clean":"2023-09-23T03:15:26.619606+0000","last_became_active":"2023-09-23T03:15:25.449562+0000","last_became_peered":"2023-09-23T03:15:25.449562+0000","last_unstale":"2023-09-23T03:15:26.619606+0000","last_undegraded":"2023-09-23T03:15:26.619606+0000","last_fullsized":"2023-09-23T03:15:26.619606+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:02:13.107703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.310119+0000","last_change":"2023-09-23T03:15:21.151987+0000","last_active":"2023-09-23T03:15:25.310119+0000","last_peered":"2023-09-23T03:15:25.310119+0000","last_clean":"2023-09-23T03:15:25.310119+0000","last_became_active":"2023-09-23T03:15:21.151421+0000","last_became_peered":"2023-09-23T03:15:21.151421+0000","last_unstale":"2023-09-23T03:15:25.310119+0000","last_undegraded":"2023-09-23T03:15:25.310119+0000","last_fullsized":"2023-09-23T03:15:25.310119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:32:42.587095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.441395+0000","last_change":"2023-09-23T03:15:20.262324+0000","last_active":"2023-09-23T03:15:25.441395+0000","last_peered":"2023-09-23T03:15:25.441395+0000","last_clean":"2023-09-23T03:15:25.441395+0000","last_became_active":"2023-09-23T03:15:20.261456+0000","last_became_peered":"2023-09-23T03:15:20.261456+0000","last_unstale":"2023-09-23T03:15:25.441395+0000","last_undegraded":"2023-09-23T03:15:25.441395+0000","last_fullsized":"2023-09-23T03:15:25.441395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:31:40.244446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.139092+0000","last_change":"2023-09-23T03:15:25.329961+0000","last_active":"2023-09-23T03:15:26.139092+0000","last_peered":"2023-09-23T03:15:26.139092+0000","last_clean":"2023-09-23T03:15:26.139092+0000","last_became_active":"2023-09-23T03:15:25.329525+0000","last_became_peered":"2023-09-23T03:15:25.329525+0000","last_unstale":"2023-09-23T03:15:26.139092+0000","last_undegraded":"2023-09-23T03:15:26.139092+0000","last_fullsized":"2023-09-23T03:15:26.139092+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:39:14.659863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.304127+0000","last_change":"2023-09-23T03:15:21.160274+0000","last_active":"2023-09-23T03:15:25.304127+0000","last_peered":"2023-09-23T03:15:25.304127+0000","last_clean":"2023-09-23T03:15:25.304127+0000","last_became_active":"2023-09-23T03:15:21.160110+0000","last_became_peered":"2023-09-23T03:15:21.160110+0000","last_unstale":"2023-09-23T03:15:25.304127+0000","last_undegraded":"2023-09-23T03:15:25.304127+0000","last_fullsized":"2023-09-23T03:15:25.304127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:15:19.584603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.313804+0000","last_change":"2023-09-23T03:15:20.265791+0000","last_active":"2023-09-23T03:15:25.313804+0000","last_peered":"2023-09-23T03:15:25.313804+0000","last_clean":"2023-09-23T03:15:25.313804+0000","last_became_active":"2023-09-23T03:15:20.265429+0000","last_became_peered":"2023-09-23T03:15:20.265429+0000","last_unstale":"2023-09-23T03:15:25.313804+0000","last_undegraded":"2023-09-23T03:15:25.313804+0000","last_fullsized":"2023-09-23T03:15:25.313804+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:05:48.619034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.138192+0000","last_change":"2023-09-23T03:15:25.451854+0000","last_active":"2023-09-23T03:15:26.138192+0000","last_peered":"2023-09-23T03:15:26.138192+0000","last_clean":"2023-09-23T03:15:26.138192+0000","last_became_active":"2023-09-23T03:15:25.450790+0000","last_became_peered":"2023-09-23T03:15:25.450790+0000","last_unstale":"2023-09-23T03:15:26.138192+0000","last_undegraded":"2023-09-23T03:15:26.138192+0000","last_fullsized":"2023-09-23T03:15:26.138192+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:28:35.883340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.297558+0000","last_change":"2023-09-23T03:15:21.163852+0000","last_active":"2023-09-23T03:15:25.297558+0000","last_peered":"2023-09-23T03:15:25.297558+0000","last_clean":"2023-09-23T03:15:25.297558+0000","last_became_active":"2023-09-23T03:15:21.163332+0000","last_became_peered":"2023-09-23T03:15:21.163332+0000","last_unstale":"2023-09-23T03:15:25.297558+0000","last_undegraded":"2023-09-23T03:15:25.297558+0000","last_fullsized":"2023-09-23T03:15:25.297558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:25:53.924266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.616670+0000","last_change":"2023-09-23T03:15:20.264120+0000","last_active":"2023-09-23T03:15:25.616670+0000","last_peered":"2023-09-23T03:15:25.616670+0000","last_clean":"2023-09-23T03:15:25.616670+0000","last_became_active":"2023-09-23T03:15:20.263598+0000","last_became_peered":"2023-09-23T03:15:20.263598+0000","last_unstale":"2023-09-23T03:15:25.616670+0000","last_undegraded":"2023-09-23T03:15:25.616670+0000","last_fullsized":"2023-09-23T03:15:25.616670+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:31:01.946245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.485429+0000","last_change":"2023-09-23T03:15:25.444557+0000","last_active":"2023-09-23T03:15:26.485429+0000","last_peered":"2023-09-23T03:15:26.485429+0000","last_clean":"2023-09-23T03:15:26.485429+0000","last_became_active":"2023-09-23T03:15:25.444002+0000","last_became_peered":"2023-09-23T03:15:25.444002+0000","last_unstale":"2023-09-23T03:15:26.485429+0000","last_undegraded":"2023-09-23T03:15:26.485429+0000","last_fullsized":"2023-09-23T03:15:26.485429+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:15:29.393607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.311253+0000","last_change":"2023-09-23T03:15:21.155177+0000","last_active":"2023-09-23T03:15:25.311253+0000","last_peered":"2023-09-23T03:15:25.311253+0000","last_clean":"2023-09-23T03:15:25.311253+0000","last_became_active":"2023-09-23T03:15:21.154788+0000","last_became_peered":"2023-09-23T03:15:21.154788+0000","last_unstale":"2023-09-23T03:15:25.311253+0000","last_undegraded":"2023-09-23T03:15:25.311253+0000","last_fullsized":"2023-09-23T03:15:25.311253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:51:00.098552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.446205+0000","last_change":"2023-09-23T03:15:20.261866+0000","last_active":"2023-09-23T03:15:25.446205+0000","last_peered":"2023-09-23T03:15:25.446205+0000","last_clean":"2023-09-23T03:15:25.446205+0000","last_became_active":"2023-09-23T03:15:20.261448+0000","last_became_peered":"2023-09-23T03:15:20.261448+0000","last_unstale":"2023-09-23T03:15:25.446205+0000","last_undegraded":"2023-09-23T03:15:25.446205+0000","last_fullsized":"2023-09-23T03:15:25.446205+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:15:18.572539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.619178+0000","last_change":"2023-09-23T03:15:25.443253+0000","last_active":"2023-09-23T03:15:26.619178+0000","last_peered":"2023-09-23T03:15:26.619178+0000","last_clean":"2023-09-23T03:15:26.619178+0000","last_became_active":"2023-09-23T03:15:25.442252+0000","last_became_peered":"2023-09-23T03:15:25.442252+0000","last_unstale":"2023-09-23T03:15:26.619178+0000","last_undegraded":"2023-09-23T03:15:26.619178+0000","last_fullsized":"2023-09-23T03:15:26.619178+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:53:25.508968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315540+0000","last_change":"2023-09-23T03:15:21.167597+0000","last_active":"2023-09-23T03:15:25.315540+0000","last_peered":"2023-09-23T03:15:25.315540+0000","last_clean":"2023-09-23T03:15:25.315540+0000","last_became_active":"2023-09-23T03:15:21.167306+0000","last_became_peered":"2023-09-23T03:15:21.167306+0000","last_unstale":"2023-09-23T03:15:25.315540+0000","last_undegraded":"2023-09-23T03:15:25.315540+0000","last_fullsized":"2023-09-23T03:15:25.315540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:25:53.924266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.311346+0000","last_change":"2023-09-23T03:15:20.254369+0000","last_active":"2023-09-23T03:15:25.311346+0000","last_peered":"2023-09-23T03:15:25.311346+0000","last_clean":"2023-09-23T03:15:25.311346+0000","last_became_active":"2023-09-23T03:15:20.251723+0000","last_became_peered":"2023-09-23T03:15:20.251723+0000","last_unstale":"2023-09-23T03:15:25.311346+0000","last_undegraded":"2023-09-23T03:15:25.311346+0000","last_fullsized":"2023-09-23T03:15:25.311346+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:32:41.575031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.616591+0000","last_change":"2023-09-23T03:15:25.331003+0000","last_active":"2023-09-23T03:15:26.616591+0000","last_peered":"2023-09-23T03:15:26.616591+0000","last_clean":"2023-09-23T03:15:26.616591+0000","last_became_active":"2023-09-23T03:15:25.330351+0000","last_became_peered":"2023-09-23T03:15:25.330351+0000","last_unstale":"2023-09-23T03:15:26.616591+0000","last_undegraded":"2023-09-23T03:15:26.616591+0000","last_fullsized":"2023-09-23T03:15:26.616591+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:42:41.516665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.311073+0000","last_change":"2023-09-23T03:15:21.160153+0000","last_active":"2023-09-23T03:15:25.311073+0000","last_peered":"2023-09-23T03:15:25.311073+0000","last_clean":"2023-09-23T03:15:25.311073+0000","last_became_active":"2023-09-23T03:15:21.159780+0000","last_became_peered":"2023-09-23T03:15:21.159780+0000","last_unstale":"2023-09-23T03:15:25.311073+0000","last_undegraded":"2023-09-23T03:15:25.311073+0000","last_fullsized":"2023-09-23T03:15:25.311073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:31:41.256510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.317480+0000","last_change":"2023-09-23T03:15:20.258357+0000","last_active":"2023-09-23T03:15:25.317480+0000","last_peered":"2023-09-23T03:15:25.317480+0000","last_clean":"2023-09-23T03:15:25.317480+0000","last_became_active":"2023-09-23T03:15:20.258052+0000","last_became_peered":"2023-09-23T03:15:20.258052+0000","last_unstale":"2023-09-23T03:15:25.317480+0000","last_undegraded":"2023-09-23T03:15:25.317480+0000","last_fullsized":"2023-09-23T03:15:25.317480+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:01:03.998670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.618336+0000","last_change":"2023-09-23T03:15:25.326862+0000","last_active":"2023-09-23T03:15:26.618336+0000","last_peered":"2023-09-23T03:15:26.618336+0000","last_clean":"2023-09-23T03:15:26.618336+0000","last_became_active":"2023-09-23T03:15:25.326483+0000","last_became_peered":"2023-09-23T03:15:25.326483+0000","last_unstale":"2023-09-23T03:15:26.618336+0000","last_undegraded":"2023-09-23T03:15:26.618336+0000","last_fullsized":"2023-09-23T03:15:26.618336+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:52:06.644022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.310981+0000","last_change":"2023-09-23T03:15:21.160191+0000","last_active":"2023-09-23T03:15:25.310981+0000","last_peered":"2023-09-23T03:15:25.310981+0000","last_clean":"2023-09-23T03:15:25.310981+0000","last_became_active":"2023-09-23T03:15:21.159841+0000","last_became_peered":"2023-09-23T03:15:21.159841+0000","last_unstale":"2023-09-23T03:15:25.310981+0000","last_undegraded":"2023-09-23T03:15:25.310981+0000","last_fullsized":"2023-09-23T03:15:25.310981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:23:07.611380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.301033+0000","last_change":"2023-09-23T03:15:20.242692+0000","last_active":"2023-09-23T03:15:25.301033+0000","last_peered":"2023-09-23T03:15:25.301033+0000","last_clean":"2023-09-23T03:15:25.301033+0000","last_became_active":"2023-09-23T03:15:20.242383+0000","last_became_peered":"2023-09-23T03:15:20.242383+0000","last_unstale":"2023-09-23T03:15:25.301033+0000","last_undegraded":"2023-09-23T03:15:25.301033+0000","last_fullsized":"2023-09-23T03:15:25.301033+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:35:16.587399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.619952+0000","last_change":"2023-09-23T03:15:25.330531+0000","last_active":"2023-09-23T03:15:26.619952+0000","last_peered":"2023-09-23T03:15:26.619952+0000","last_clean":"2023-09-23T03:15:26.619952+0000","last_became_active":"2023-09-23T03:15:25.330145+0000","last_became_peered":"2023-09-23T03:15:25.330145+0000","last_unstale":"2023-09-23T03:15:26.619952+0000","last_undegraded":"2023-09-23T03:15:26.619952+0000","last_fullsized":"2023-09-23T03:15:26.619952+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:44:24.602333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.302012+0000","last_change":"2023-09-23T03:15:21.152726+0000","last_active":"2023-09-23T03:15:25.302012+0000","last_peered":"2023-09-23T03:15:25.302012+0000","last_clean":"2023-09-23T03:15:25.302012+0000","last_became_active":"2023-09-23T03:15:21.151916+0000","last_became_peered":"2023-09-23T03:15:21.151916+0000","last_unstale":"2023-09-23T03:15:25.302012+0000","last_undegraded":"2023-09-23T03:15:25.302012+0000","last_fullsized":"2023-09-23T03:15:25.302012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:05:49.631098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.441189+0000","last_change":"2023-09-23T03:15:20.265672+0000","last_active":"2023-09-23T03:15:25.441189+0000","last_peered":"2023-09-23T03:15:25.441189+0000","last_clean":"2023-09-23T03:15:25.441189+0000","last_became_active":"2023-09-23T03:15:20.265172+0000","last_became_peered":"2023-09-23T03:15:20.265172+0000","last_unstale":"2023-09-23T03:15:25.441189+0000","last_undegraded":"2023-09-23T03:15:25.441189+0000","last_fullsized":"2023-09-23T03:15:25.441189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:28:08.970374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.620019+0000","last_change":"2023-09-23T03:15:25.442450+0000","last_active":"2023-09-23T03:15:26.620019+0000","last_peered":"2023-09-23T03:15:26.620019+0000","last_clean":"2023-09-23T03:15:26.620019+0000","last_became_active":"2023-09-23T03:15:25.441911+0000","last_became_peered":"2023-09-23T03:15:25.441911+0000","last_unstale":"2023-09-23T03:15:26.620019+0000","last_undegraded":"2023-09-23T03:15:26.620019+0000","last_fullsized":"2023-09-23T03:15:26.620019+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:58:07.013813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.295933+0000","last_change":"2023-09-23T03:15:21.171351+0000","last_active":"2023-09-23T03:15:25.295933+0000","last_peered":"2023-09-23T03:15:25.295933+0000","last_clean":"2023-09-23T03:15:25.295933+0000","last_became_active":"2023-09-23T03:15:21.170977+0000","last_became_peered":"2023-09-23T03:15:21.170977+0000","last_unstale":"2023-09-23T03:15:25.295933+0000","last_undegraded":"2023-09-23T03:15:25.295933+0000","last_fullsized":"2023-09-23T03:15:25.295933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:51:00.098552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.314530+0000","last_change":"2023-09-23T03:15:20.258137+0000","last_active":"2023-09-23T03:15:25.314530+0000","last_peered":"2023-09-23T03:15:25.314530+0000","last_clean":"2023-09-23T03:15:25.314530+0000","last_became_active":"2023-09-23T03:15:20.257772+0000","last_became_peered":"2023-09-23T03:15:20.257772+0000","last_unstale":"2023-09-23T03:15:25.314530+0000","last_undegraded":"2023-09-23T03:15:25.314530+0000","last_fullsized":"2023-09-23T03:15:25.314530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:52:01.548454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":169,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.441463+0000","last_change":"2023-09-23T03:15:11.042447+0000","last_active":"2023-09-23T03:15:25.441463+0000","last_peered":"2023-09-23T03:15:25.441463+0000","last_clean":"2023-09-23T03:15:25.441463+0000","last_became_active":"2023-09-23T03:15:11.040322+0000","last_became_peered":"2023-09-23T03:15:11.040322+0000","last_unstale":"2023-09-23T03:15:25.441463+0000","last_undegraded":"2023-09-23T03:15:25.441463+0000","last_fullsized":"2023-09-23T03:15:25.441463+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_clean_scrub_stamp":"2023-09-23T03:15:09.474842+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:18:36.331645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.138753+0000","last_change":"2023-09-23T03:15:25.449566+0000","last_active":"2023-09-23T03:15:26.138753+0000","last_peered":"2023-09-23T03:15:26.138753+0000","last_clean":"2023-09-23T03:15:26.138753+0000","last_became_active":"2023-09-23T03:15:25.448514+0000","last_became_peered":"2023-09-23T03:15:25.448514+0000","last_unstale":"2023-09-23T03:15:26.138753+0000","last_undegraded":"2023-09-23T03:15:26.138753+0000","last_fullsized":"2023-09-23T03:15:26.138753+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:52:06.644022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.442985+0000","last_change":"2023-09-23T03:15:21.180427+0000","last_active":"2023-09-23T03:15:25.442985+0000","last_peered":"2023-09-23T03:15:25.442985+0000","last_clean":"2023-09-23T03:15:25.442985+0000","last_became_active":"2023-09-23T03:15:21.179941+0000","last_became_peered":"2023-09-23T03:15:21.179941+0000","last_unstale":"2023-09-23T03:15:25.442985+0000","last_undegraded":"2023-09-23T03:15:25.442985+0000","last_fullsized":"2023-09-23T03:15:25.442985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:53:21.435687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.302809+0000","last_change":"2023-09-23T03:15:20.259280+0000","last_active":"2023-09-23T03:15:25.302809+0000","last_peered":"2023-09-23T03:15:25.302809+0000","last_clean":"2023-09-23T03:15:25.302809+0000","last_became_active":"2023-09-23T03:15:20.258990+0000","last_became_peered":"2023-09-23T03:15:20.258990+0000","last_unstale":"2023-09-23T03:15:25.302809+0000","last_undegraded":"2023-09-23T03:15:25.302809+0000","last_fullsized":"2023-09-23T03:15:25.302809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:32:41.575031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.137662+0000","last_change":"2023-09-23T03:15:25.448939+0000","last_active":"2023-09-23T03:15:26.137662+0000","last_peered":"2023-09-23T03:15:26.137662+0000","last_clean":"2023-09-23T03:15:26.137662+0000","last_became_active":"2023-09-23T03:15:25.448129+0000","last_became_peered":"2023-09-23T03:15:25.448129+0000","last_unstale":"2023-09-23T03:15:26.137662+0000","last_undegraded":"2023-09-23T03:15:26.137662+0000","last_fullsized":"2023-09-23T03:15:26.137662+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:42:41.516665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315261+0000","last_change":"2023-09-23T03:15:21.172056+0000","last_active":"2023-09-23T03:15:25.315261+0000","last_peered":"2023-09-23T03:15:25.315261+0000","last_clean":"2023-09-23T03:15:25.315261+0000","last_became_active":"2023-09-23T03:15:21.170669+0000","last_became_peered":"2023-09-23T03:15:21.170669+0000","last_unstale":"2023-09-23T03:15:25.315261+0000","last_undegraded":"2023-09-23T03:15:25.315261+0000","last_fullsized":"2023-09-23T03:15:25.315261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:02:09.034422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.312606+0000","last_change":"2023-09-23T03:15:20.254984+0000","last_active":"2023-09-23T03:15:25.312606+0000","last_peered":"2023-09-23T03:15:25.312606+0000","last_clean":"2023-09-23T03:15:25.312606+0000","last_became_active":"2023-09-23T03:15:20.254425+0000","last_became_peered":"2023-09-23T03:15:20.254425+0000","last_unstale":"2023-09-23T03:15:25.312606+0000","last_undegraded":"2023-09-23T03:15:25.312606+0000","last_fullsized":"2023-09-23T03:15:25.312606+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:13:21.067891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.619714+0000","last_change":"2023-09-23T03:15:25.324169+0000","last_active":"2023-09-23T03:15:26.619714+0000","last_peered":"2023-09-23T03:15:26.619714+0000","last_clean":"2023-09-23T03:15:26.619714+0000","last_became_active":"2023-09-23T03:15:25.323647+0000","last_became_peered":"2023-09-23T03:15:25.323647+0000","last_unstale":"2023-09-23T03:15:26.619714+0000","last_undegraded":"2023-09-23T03:15:26.619714+0000","last_fullsized":"2023-09-23T03:15:26.619714+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:44:24.602333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.446178+0000","last_change":"2023-09-23T03:15:21.175250+0000","last_active":"2023-09-23T03:15:25.446178+0000","last_peered":"2023-09-23T03:15:25.446178+0000","last_clean":"2023-09-23T03:15:25.446178+0000","last_became_active":"2023-09-23T03:15:21.174890+0000","last_became_peered":"2023-09-23T03:15:21.174890+0000","last_unstale":"2023-09-23T03:15:25.446178+0000","last_undegraded":"2023-09-23T03:15:25.446178+0000","last_fullsized":"2023-09-23T03:15:25.446178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:03:55.070789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.301471+0000","last_change":"2023-09-23T03:15:20.242773+0000","last_active":"2023-09-23T03:15:25.301471+0000","last_peered":"2023-09-23T03:15:25.301471+0000","last_clean":"2023-09-23T03:15:25.301471+0000","last_became_active":"2023-09-23T03:15:20.242466+0000","last_became_peered":"2023-09-23T03:15:20.242466+0000","last_unstale":"2023-09-23T03:15:25.301471+0000","last_undegraded":"2023-09-23T03:15:25.301471+0000","last_fullsized":"2023-09-23T03:15:25.301471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:52:01.548454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.617142+0000","last_change":"2023-09-23T03:15:25.452683+0000","last_active":"2023-09-23T03:15:26.617142+0000","last_peered":"2023-09-23T03:15:26.617142+0000","last_clean":"2023-09-23T03:15:26.617142+0000","last_became_active":"2023-09-23T03:15:25.451509+0000","last_became_peered":"2023-09-23T03:15:25.451509+0000","last_unstale":"2023-09-23T03:15:26.617142+0000","last_undegraded":"2023-09-23T03:15:26.617142+0000","last_fullsized":"2023-09-23T03:15:26.617142+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:03:59.144070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.443716+0000","last_change":"2023-09-23T03:15:21.177994+0000","last_active":"2023-09-23T03:15:25.443716+0000","last_peered":"2023-09-23T03:15:25.443716+0000","last_clean":"2023-09-23T03:15:25.443716+0000","last_became_active":"2023-09-23T03:15:21.177752+0000","last_became_peered":"2023-09-23T03:15:21.177752+0000","last_unstale":"2023-09-23T03:15:25.443716+0000","last_undegraded":"2023-09-23T03:15:25.443716+0000","last_fullsized":"2023-09-23T03:15:25.443716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:52:02.570741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.309999+0000","last_change":"2023-09-23T03:15:20.245089+0000","last_active":"2023-09-23T03:15:25.309999+0000","last_peered":"2023-09-23T03:15:25.309999+0000","last_clean":"2023-09-23T03:15:25.309999+0000","last_became_active":"2023-09-23T03:15:20.243281+0000","last_became_peered":"2023-09-23T03:15:20.243281+0000","last_unstale":"2023-09-23T03:15:25.309999+0000","last_undegraded":"2023-09-23T03:15:25.309999+0000","last_fullsized":"2023-09-23T03:15:25.309999+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:34:50.786275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.137112+0000","last_change":"2023-09-23T03:15:25.327837+0000","last_active":"2023-09-23T03:15:26.137112+0000","last_peered":"2023-09-23T03:15:26.137112+0000","last_clean":"2023-09-23T03:15:26.137112+0000","last_became_active":"2023-09-23T03:15:25.327245+0000","last_became_peered":"2023-09-23T03:15:25.327245+0000","last_unstale":"2023-09-23T03:15:26.137112+0000","last_undegraded":"2023-09-23T03:15:26.137112+0000","last_fullsized":"2023-09-23T03:15:26.137112+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:15:23.657884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.441989+0000","last_change":"2023-09-23T03:15:21.173588+0000","last_active":"2023-09-23T03:15:25.441989+0000","last_peered":"2023-09-23T03:15:25.441989+0000","last_clean":"2023-09-23T03:15:25.441989+0000","last_became_active":"2023-09-23T03:15:21.173284+0000","last_became_peered":"2023-09-23T03:15:21.173284+0000","last_unstale":"2023-09-23T03:15:25.441989+0000","last_undegraded":"2023-09-23T03:15:25.441989+0000","last_fullsized":"2023-09-23T03:15:25.441989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:43:35.707241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.302930+0000","last_change":"2023-09-23T03:15:20.249033+0000","last_active":"2023-09-23T03:15:25.302930+0000","last_peered":"2023-09-23T03:15:25.302930+0000","last_clean":"2023-09-23T03:15:25.302930+0000","last_became_active":"2023-09-23T03:15:20.248738+0000","last_became_peered":"2023-09-23T03:15:20.248738+0000","last_unstale":"2023-09-23T03:15:25.302930+0000","last_undegraded":"2023-09-23T03:15:25.302930+0000","last_fullsized":"2023-09-23T03:15:25.302930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:34:53.736954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.619712+0000","last_change":"2023-09-23T03:15:25.328800+0000","last_active":"2023-09-23T03:15:26.619712+0000","last_peered":"2023-09-23T03:15:26.619712+0000","last_clean":"2023-09-23T03:15:26.619712+0000","last_became_active":"2023-09-23T03:15:25.327349+0000","last_became_peered":"2023-09-23T03:15:25.327349+0000","last_unstale":"2023-09-23T03:15:26.619712+0000","last_undegraded":"2023-09-23T03:15:26.619712+0000","last_fullsized":"2023-09-23T03:15:26.619712+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:02:13.107703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.306109+0000","last_change":"2023-09-23T03:15:21.156710+0000","last_active":"2023-09-23T03:15:25.306109+0000","last_peered":"2023-09-23T03:15:25.306109+0000","last_clean":"2023-09-23T03:15:25.306109+0000","last_became_active":"2023-09-23T03:15:21.156344+0000","last_became_peered":"2023-09-23T03:15:21.156344+0000","last_unstale":"2023-09-23T03:15:25.306109+0000","last_undegraded":"2023-09-23T03:15:25.306109+0000","last_fullsized":"2023-09-23T03:15:25.306109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:01:05.010735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.307353+0000","last_change":"2023-09-23T03:15:20.254568+0000","last_active":"2023-09-23T03:15:25.307353+0000","last_peered":"2023-09-23T03:15:25.307353+0000","last_clean":"2023-09-23T03:15:25.307353+0000","last_became_active":"2023-09-23T03:15:20.254294+0000","last_became_peered":"2023-09-23T03:15:20.254294+0000","last_unstale":"2023-09-23T03:15:25.307353+0000","last_undegraded":"2023-09-23T03:15:25.307353+0000","last_fullsized":"2023-09-23T03:15:25.307353+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:32:41.575031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.487038+0000","last_change":"2023-09-23T03:15:25.444588+0000","last_active":"2023-09-23T03:15:26.487038+0000","last_peered":"2023-09-23T03:15:26.487038+0000","last_clean":"2023-09-23T03:15:26.487038+0000","last_became_active":"2023-09-23T03:15:25.443847+0000","last_became_peered":"2023-09-23T03:15:25.443847+0000","last_unstale":"2023-09-23T03:15:26.487038+0000","last_undegraded":"2023-09-23T03:15:26.487038+0000","last_fullsized":"2023-09-23T03:15:26.487038+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:54:37.479094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.441909+0000","last_change":"2023-09-23T03:15:21.172873+0000","last_active":"2023-09-23T03:15:25.441909+0000","last_peered":"2023-09-23T03:15:25.441909+0000","last_clean":"2023-09-23T03:15:25.441909+0000","last_became_active":"2023-09-23T03:15:21.172552+0000","last_became_peered":"2023-09-23T03:15:21.172552+0000","last_unstale":"2023-09-23T03:15:25.441909+0000","last_undegraded":"2023-09-23T03:15:25.441909+0000","last_fullsized":"2023-09-23T03:15:25.441909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:42:37.443384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.317383+0000","last_change":"2023-09-23T03:15:20.257041+0000","last_active":"2023-09-23T03:15:25.317383+0000","last_peered":"2023-09-23T03:15:25.317383+0000","last_clean":"2023-09-23T03:15:25.317383+0000","last_became_active":"2023-09-23T03:15:20.256737+0000","last_became_peered":"2023-09-23T03:15:20.256737+0000","last_unstale":"2023-09-23T03:15:25.317383+0000","last_undegraded":"2023-09-23T03:15:25.317383+0000","last_fullsized":"2023-09-23T03:15:25.317383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:10:06.381357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.486900+0000","last_change":"2023-09-23T03:15:25.445742+0000","last_active":"2023-09-23T03:15:26.486900+0000","last_peered":"2023-09-23T03:15:26.486900+0000","last_clean":"2023-09-23T03:15:26.486900+0000","last_became_active":"2023-09-23T03:15:25.445297+0000","last_became_peered":"2023-09-23T03:15:25.445297+0000","last_unstale":"2023-09-23T03:15:26.486900+0000","last_undegraded":"2023-09-23T03:15:26.486900+0000","last_fullsized":"2023-09-23T03:15:26.486900+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:36:02.944327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.301638+0000","last_change":"2023-09-23T03:15:21.166987+0000","last_active":"2023-09-23T03:15:25.301638+0000","last_peered":"2023-09-23T03:15:25.301638+0000","last_clean":"2023-09-23T03:15:25.301638+0000","last_became_active":"2023-09-23T03:15:21.166575+0000","last_became_peered":"2023-09-23T03:15:21.166575+0000","last_unstale":"2023-09-23T03:15:25.301638+0000","last_undegraded":"2023-09-23T03:15:25.301638+0000","last_fullsized":"2023-09-23T03:15:25.301638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:10:07.393421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.305489+0000","last_change":"2023-09-23T03:15:20.253573+0000","last_active":"2023-09-23T03:15:25.305489+0000","last_peered":"2023-09-23T03:15:25.305489+0000","last_clean":"2023-09-23T03:15:25.305489+0000","last_became_active":"2023-09-23T03:15:20.253228+0000","last_became_peered":"2023-09-23T03:15:20.253228+0000","last_unstale":"2023-09-23T03:15:25.305489+0000","last_undegraded":"2023-09-23T03:15:25.305489+0000","last_fullsized":"2023-09-23T03:15:25.305489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:35:16.587399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.619960+0000","last_change":"2023-09-23T03:15:25.453715+0000","last_active":"2023-09-23T03:15:26.619960+0000","last_peered":"2023-09-23T03:15:26.619960+0000","last_clean":"2023-09-23T03:15:26.619960+0000","last_became_active":"2023-09-23T03:15:25.452979+0000","last_became_peered":"2023-09-23T03:15:25.452979+0000","last_unstale":"2023-09-23T03:15:26.619960+0000","last_undegraded":"2023-09-23T03:15:26.619960+0000","last_fullsized":"2023-09-23T03:15:26.619960+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:15:29.393607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.442638+0000","last_change":"2023-09-23T03:15:21.169871+0000","last_active":"2023-09-23T03:15:25.442638+0000","last_peered":"2023-09-23T03:15:25.442638+0000","last_clean":"2023-09-23T03:15:25.442638+0000","last_became_active":"2023-09-23T03:15:21.169539+0000","last_became_peered":"2023-09-23T03:15:21.169539+0000","last_unstale":"2023-09-23T03:15:25.442638+0000","last_undegraded":"2023-09-23T03:15:25.442638+0000","last_fullsized":"2023-09-23T03:15:25.442638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:44:20.529052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:40.246064+0000","last_change":"2023-09-23T03:15:20.252281+0000","last_active":"2023-09-23T03:15:40.246064+0000","last_peered":"2023-09-23T03:15:40.246064+0000","last_clean":"2023-09-23T03:15:40.246064+0000","last_became_active":"2023-09-23T03:15:20.251982+0000","last_became_peered":"2023-09-23T03:15:20.251982+0000","last_unstale":"2023-09-23T03:15:40.246064+0000","last_undegraded":"2023-09-23T03:15:40.246064+0000","last_fullsized":"2023-09-23T03:15:40.246064+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:35:16.587399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.616880+0000","last_change":"2023-09-23T03:15:25.325976+0000","last_active":"2023-09-23T03:15:26.616880+0000","last_peered":"2023-09-23T03:15:26.616880+0000","last_clean":"2023-09-23T03:15:26.616880+0000","last_became_active":"2023-09-23T03:15:25.325613+0000","last_became_peered":"2023-09-23T03:15:25.325613+0000","last_unstale":"2023-09-23T03:15:26.616880+0000","last_undegraded":"2023-09-23T03:15:26.616880+0000","last_fullsized":"2023-09-23T03:15:26.616880+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:53:25.508968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.309427+0000","last_change":"2023-09-23T03:15:21.166420+0000","last_active":"2023-09-23T03:15:25.309427+0000","last_peered":"2023-09-23T03:15:25.309427+0000","last_clean":"2023-09-23T03:15:25.309427+0000","last_became_active":"2023-09-23T03:15:21.166086+0000","last_became_peered":"2023-09-23T03:15:21.166086+0000","last_unstale":"2023-09-23T03:15:25.309427+0000","last_undegraded":"2023-09-23T03:15:25.309427+0000","last_fullsized":"2023-09-23T03:15:25.309427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:13:22.079955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.312418+0000","last_change":"2023-09-23T03:15:20.263920+0000","last_active":"2023-09-23T03:15:25.312418+0000","last_peered":"2023-09-23T03:15:25.312418+0000","last_clean":"2023-09-23T03:15:25.312418+0000","last_became_active":"2023-09-23T03:15:20.263623+0000","last_became_peered":"2023-09-23T03:15:20.263623+0000","last_unstale":"2023-09-23T03:15:25.312418+0000","last_undegraded":"2023-09-23T03:15:25.312418+0000","last_fullsized":"2023-09-23T03:15:25.312418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:28:08.970374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.139553+0000","last_change":"2023-09-23T03:15:25.329301+0000","last_active":"2023-09-23T03:15:26.139553+0000","last_peered":"2023-09-23T03:15:26.139553+0000","last_clean":"2023-09-23T03:15:26.139553+0000","last_became_active":"2023-09-23T03:15:25.328559+0000","last_became_peered":"2023-09-23T03:15:25.328559+0000","last_unstale":"2023-09-23T03:15:26.139553+0000","last_undegraded":"2023-09-23T03:15:26.139553+0000","last_fullsized":"2023-09-23T03:15:26.139553+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:43:39.780522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.303426+0000","last_change":"2023-09-23T03:15:21.172339+0000","last_active":"2023-09-23T03:15:25.303426+0000","last_peered":"2023-09-23T03:15:25.303426+0000","last_clean":"2023-09-23T03:15:25.303426+0000","last_became_active":"2023-09-23T03:15:21.172034+0000","last_became_peered":"2023-09-23T03:15:21.172034+0000","last_unstale":"2023-09-23T03:15:25.303426+0000","last_undegraded":"2023-09-23T03:15:25.303426+0000","last_fullsized":"2023-09-23T03:15:25.303426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:21:27.583218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315333+0000","last_change":"2023-09-23T03:15:20.259342+0000","last_active":"2023-09-23T03:15:25.315333+0000","last_peered":"2023-09-23T03:15:25.315333+0000","last_clean":"2023-09-23T03:15:25.315333+0000","last_became_active":"2023-09-23T03:15:20.258990+0000","last_became_peered":"2023-09-23T03:15:20.258990+0000","last_unstale":"2023-09-23T03:15:25.315333+0000","last_undegraded":"2023-09-23T03:15:25.315333+0000","last_fullsized":"2023-09-23T03:15:25.315333+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:50:59.086488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.485843+0000","last_change":"2023-09-23T03:15:25.322201+0000","last_active":"2023-09-23T03:15:26.485843+0000","last_peered":"2023-09-23T03:15:26.485843+0000","last_clean":"2023-09-23T03:15:26.485843+0000","last_became_active":"2023-09-23T03:15:25.321486+0000","last_became_peered":"2023-09-23T03:15:25.321486+0000","last_unstale":"2023-09-23T03:15:26.485843+0000","last_undegraded":"2023-09-23T03:15:26.485843+0000","last_fullsized":"2023-09-23T03:15:26.485843+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:20:20.193299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315253+0000","last_change":"2023-09-23T03:15:21.175465+0000","last_active":"2023-09-23T03:15:25.315253+0000","last_peered":"2023-09-23T03:15:25.315253+0000","last_clean":"2023-09-23T03:15:25.315253+0000","last_became_active":"2023-09-23T03:15:21.171773+0000","last_became_peered":"2023-09-23T03:15:21.171773+0000","last_unstale":"2023-09-23T03:15:25.315253+0000","last_undegraded":"2023-09-23T03:15:25.315253+0000","last_fullsized":"2023-09-23T03:15:25.315253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:39:10.586582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.313666+0000","last_change":"2023-09-23T03:15:20.259404+0000","last_active":"2023-09-23T03:15:25.313666+0000","last_peered":"2023-09-23T03:15:25.313666+0000","last_clean":"2023-09-23T03:15:25.313666+0000","last_became_active":"2023-09-23T03:15:20.259013+0000","last_became_peered":"2023-09-23T03:15:20.259013+0000","last_unstale":"2023-09-23T03:15:25.313666+0000","last_undegraded":"2023-09-23T03:15:25.313666+0000","last_fullsized":"2023-09-23T03:15:25.313666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:41:37.505547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.485703+0000","last_change":"2023-09-23T03:15:25.452370+0000","last_active":"2023-09-23T03:15:26.485703+0000","last_peered":"2023-09-23T03:15:26.485703+0000","last_clean":"2023-09-23T03:15:26.485703+0000","last_became_active":"2023-09-23T03:15:25.451873+0000","last_became_peered":"2023-09-23T03:15:25.451873+0000","last_unstale":"2023-09-23T03:15:26.485703+0000","last_undegraded":"2023-09-23T03:15:26.485703+0000","last_fullsized":"2023-09-23T03:15:26.485703+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:58:52.853121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.619356+0000","last_change":"2023-09-23T03:15:25.452394+0000","last_active":"2023-09-23T03:15:26.619356+0000","last_peered":"2023-09-23T03:15:26.619356+0000","last_clean":"2023-09-23T03:15:26.619356+0000","last_became_active":"2023-09-23T03:15:25.451251+0000","last_became_peered":"2023-09-23T03:15:25.451251+0000","last_unstale":"2023-09-23T03:15:26.619356+0000","last_undegraded":"2023-09-23T03:15:26.619356+0000","last_fullsized":"2023-09-23T03:15:26.619356+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:52:06.644022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.310954+0000","last_change":"2023-09-23T03:15:21.170461+0000","last_active":"2023-09-23T03:15:25.310954+0000","last_peered":"2023-09-23T03:15:25.310954+0000","last_clean":"2023-09-23T03:15:25.310954+0000","last_became_active":"2023-09-23T03:15:21.170095+0000","last_became_peered":"2023-09-23T03:15:21.170095+0000","last_unstale":"2023-09-23T03:15:25.310954+0000","last_undegraded":"2023-09-23T03:15:25.310954+0000","last_fullsized":"2023-09-23T03:15:25.310954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:30:31.951544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.314895+0000","last_change":"2023-09-23T03:15:20.257662+0000","last_active":"2023-09-23T03:15:25.314895+0000","last_peered":"2023-09-23T03:15:25.314895+0000","last_clean":"2023-09-23T03:15:25.314895+0000","last_became_active":"2023-09-23T03:15:20.257376+0000","last_became_peered":"2023-09-23T03:15:20.257376+0000","last_unstale":"2023-09-23T03:15:25.314895+0000","last_undegraded":"2023-09-23T03:15:25.314895+0000","last_fullsized":"2023-09-23T03:15:25.314895+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:06:15.942585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.138746+0000","last_change":"2023-09-23T03:15:25.326388+0000","last_active":"2023-09-23T03:15:26.138746+0000","last_peered":"2023-09-23T03:15:26.138746+0000","last_clean":"2023-09-23T03:15:26.138746+0000","last_became_active":"2023-09-23T03:15:25.326018+0000","last_became_peered":"2023-09-23T03:15:25.326018+0000","last_unstale":"2023-09-23T03:15:26.138746+0000","last_undegraded":"2023-09-23T03:15:26.138746+0000","last_fullsized":"2023-09-23T03:15:26.138746+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:53:25.508968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.295747+0000","last_change":"2023-09-23T03:15:21.164139+0000","last_active":"2023-09-23T03:15:25.295747+0000","last_peered":"2023-09-23T03:15:25.295747+0000","last_clean":"2023-09-23T03:15:25.295747+0000","last_became_active":"2023-09-23T03:15:21.162300+0000","last_became_peered":"2023-09-23T03:15:21.162300+0000","last_unstale":"2023-09-23T03:15:25.295747+0000","last_undegraded":"2023-09-23T03:15:25.295747+0000","last_fullsized":"2023-09-23T03:15:25.295747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:22:07.336440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.298007+0000","last_change":"2023-09-23T03:15:20.256962+0000","last_active":"2023-09-23T03:15:25.298007+0000","last_peered":"2023-09-23T03:15:25.298007+0000","last_clean":"2023-09-23T03:15:25.298007+0000","last_became_active":"2023-09-23T03:15:20.256686+0000","last_became_peered":"2023-09-23T03:15:20.256686+0000","last_unstale":"2023-09-23T03:15:25.298007+0000","last_undegraded":"2023-09-23T03:15:25.298007+0000","last_fullsized":"2023-09-23T03:15:25.298007+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:06:15.942585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.314864+0000","last_change":"2023-09-23T03:15:20.255630+0000","last_active":"2023-09-23T03:15:25.314864+0000","last_peered":"2023-09-23T03:15:25.314864+0000","last_clean":"2023-09-23T03:15:25.314864+0000","last_became_active":"2023-09-23T03:15:20.255212+0000","last_became_peered":"2023-09-23T03:15:20.255212+0000","last_unstale":"2023-09-23T03:15:25.314864+0000","last_undegraded":"2023-09-23T03:15:25.314864+0000","last_fullsized":"2023-09-23T03:15:25.314864+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:31:01.946245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.305281+0000","last_change":"2023-09-23T03:15:21.156720+0000","last_active":"2023-09-23T03:15:25.305281+0000","last_peered":"2023-09-23T03:15:25.305281+0000","last_clean":"2023-09-23T03:15:25.305281+0000","last_became_active":"2023-09-23T03:15:21.156268+0000","last_became_peered":"2023-09-23T03:15:21.156268+0000","last_unstale":"2023-09-23T03:15:25.305281+0000","last_undegraded":"2023-09-23T03:15:25.305281+0000","last_fullsized":"2023-09-23T03:15:25.305281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:51:00.098552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.134456+0000","last_change":"2023-09-23T03:15:25.319168+0000","last_active":"2023-09-23T03:15:26.134456+0000","last_peered":"2023-09-23T03:15:26.134456+0000","last_clean":"2023-09-23T03:15:26.134456+0000","last_became_active":"2023-09-23T03:15:25.318761+0000","last_became_peered":"2023-09-23T03:15:25.318761+0000","last_unstale":"2023-09-23T03:15:26.134456+0000","last_undegraded":"2023-09-23T03:15:26.134456+0000","last_fullsized":"2023-09-23T03:15:26.134456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:05:53.704379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.296655+0000","last_change":"2023-09-23T03:15:20.261879+0000","last_active":"2023-09-23T03:15:25.296655+0000","last_peered":"2023-09-23T03:15:25.296655+0000","last_clean":"2023-09-23T03:15:25.296655+0000","last_became_active":"2023-09-23T03:15:20.261597+0000","last_became_peered":"2023-09-23T03:15:20.261597+0000","last_unstale":"2023-09-23T03:15:25.296655+0000","last_undegraded":"2023-09-23T03:15:25.296655+0000","last_fullsized":"2023-09-23T03:15:25.296655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:19.195217+0000","last_clean_scrub_stamp":"2023-09-23T03:15:19.195217+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:41:37.505547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.311031+0000","last_change":"2023-09-23T03:15:21.161920+0000","last_active":"2023-09-23T03:15:25.311031+0000","last_peered":"2023-09-23T03:15:25.311031+0000","last_clean":"2023-09-23T03:15:25.311031+0000","last_became_active":"2023-09-23T03:15:21.161657+0000","last_became_peered":"2023-09-23T03:15:21.161657+0000","last_unstale":"2023-09-23T03:15:25.311031+0000","last_undegraded":"2023-09-23T03:15:25.311031+0000","last_fullsized":"2023-09-23T03:15:25.311031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:35:17.599463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-09-23T03:15:26.486488+0000","last_change":"2023-09-23T03:15:25.319176+0000","last_active":"2023-09-23T03:15:26.486488+0000","last_peered":"2023-09-23T03:15:26.486488+0000","last_clean":"2023-09-23T03:15:26.486488+0000","last_became_active":"2023-09-23T03:15:25.318752+0000","last_became_peered":"2023-09-23T03:15:25.318752+0000","last_unstale":"2023-09-23T03:15:26.486488+0000","last_undegraded":"2023-09-23T03:15:26.486488+0000","last_fullsized":"2023-09-23T03:15:26.486488+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:24.280563+0000","last_clean_scrub_stamp":"2023-09-23T03:15:24.280563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:56:54.725774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.305274+0000","last_change":"2023-09-23T03:15:21.157543+0000","last_active":"2023-09-23T03:15:25.305274+0000","last_peered":"2023-09-23T03:15:25.305274+0000","last_clean":"2023-09-23T03:15:25.305274+0000","last_became_active":"2023-09-23T03:15:21.157240+0000","last_became_peered":"2023-09-23T03:15:21.157240+0000","last_unstale":"2023-09-23T03:15:25.305274+0000","last_undegraded":"2023-09-23T03:15:25.305274+0000","last_fullsized":"2023-09-23T03:15:25.305274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:23:07.611380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.300872+0000","last_change":"2023-09-23T03:15:21.176611+0000","last_active":"2023-09-23T03:15:25.300872+0000","last_peered":"2023-09-23T03:15:25.300872+0000","last_clean":"2023-09-23T03:15:25.300872+0000","last_became_active":"2023-09-23T03:15:21.176316+0000","last_became_peered":"2023-09-23T03:15:21.176316+0000","last_unstale":"2023-09-23T03:15:25.300872+0000","last_undegraded":"2023-09-23T03:15:25.300872+0000","last_fullsized":"2023-09-23T03:15:25.300872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:28:09.982438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.316926+0000","last_change":"2023-09-23T03:15:21.166808+0000","last_active":"2023-09-23T03:15:25.316926+0000","last_peered":"2023-09-23T03:15:25.316926+0000","last_clean":"2023-09-23T03:15:25.316926+0000","last_became_active":"2023-09-23T03:15:21.166488+0000","last_became_peered":"2023-09-23T03:15:21.166488+0000","last_unstale":"2023-09-23T03:15:25.316926+0000","last_undegraded":"2023-09-23T03:15:25.316926+0000","last_fullsized":"2023-09-23T03:15:25.316926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:43:35.707241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.305111+0000","last_change":"2023-09-23T03:15:21.162188+0000","last_active":"2023-09-23T03:15:25.305111+0000","last_peered":"2023-09-23T03:15:25.305111+0000","last_clean":"2023-09-23T03:15:25.305111+0000","last_became_active":"2023-09-23T03:15:21.161886+0000","last_became_peered":"2023-09-23T03:15:21.161886+0000","last_unstale":"2023-09-23T03:15:25.305111+0000","last_undegraded":"2023-09-23T03:15:25.305111+0000","last_fullsized":"2023-09-23T03:15:25.305111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:23:07.611380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.296970+0000","last_change":"2023-09-23T03:15:21.161567+0000","last_active":"2023-09-23T03:15:25.296970+0000","last_peered":"2023-09-23T03:15:25.296970+0000","last_clean":"2023-09-23T03:15:25.296970+0000","last_became_active":"2023-09-23T03:15:21.161168+0000","last_became_peered":"2023-09-23T03:15:21.161168+0000","last_unstale":"2023-09-23T03:15:25.296970+0000","last_undegraded":"2023-09-23T03:15:25.296970+0000","last_fullsized":"2023-09-23T03:15:25.296970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:30:37.652428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.311814+0000","last_change":"2023-09-23T03:15:21.164997+0000","last_active":"2023-09-23T03:15:25.311814+0000","last_peered":"2023-09-23T03:15:25.311814+0000","last_clean":"2023-09-23T03:15:25.311814+0000","last_became_active":"2023-09-23T03:15:21.164680+0000","last_became_peered":"2023-09-23T03:15:21.164680+0000","last_unstale":"2023-09-23T03:15:25.311814+0000","last_undegraded":"2023-09-23T03:15:25.311814+0000","last_fullsized":"2023-09-23T03:15:25.311814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:15:19.584603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.316265+0000","last_change":"2023-09-23T03:15:21.164029+0000","last_active":"2023-09-23T03:15:25.316265+0000","last_peered":"2023-09-23T03:15:25.316265+0000","last_clean":"2023-09-23T03:15:25.316265+0000","last_became_active":"2023-09-23T03:15:21.163673+0000","last_became_peered":"2023-09-23T03:15:21.163673+0000","last_unstale":"2023-09-23T03:15:25.316265+0000","last_undegraded":"2023-09-23T03:15:25.316265+0000","last_fullsized":"2023-09-23T03:15:25.316265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:15:19.584603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.297434+0000","last_change":"2023-09-23T03:15:21.163996+0000","last_active":"2023-09-23T03:15:25.297434+0000","last_peered":"2023-09-23T03:15:25.297434+0000","last_clean":"2023-09-23T03:15:25.297434+0000","last_became_active":"2023-09-23T03:15:21.163674+0000","last_became_peered":"2023-09-23T03:15:21.163674+0000","last_unstale":"2023-09-23T03:15:25.297434+0000","last_undegraded":"2023-09-23T03:15:25.297434+0000","last_fullsized":"2023-09-23T03:15:25.297434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:28:31.810059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.314809+0000","last_change":"2023-09-23T03:15:21.171741+0000","last_active":"2023-09-23T03:15:25.314809+0000","last_peered":"2023-09-23T03:15:25.314809+0000","last_clean":"2023-09-23T03:15:25.314809+0000","last_became_active":"2023-09-23T03:15:21.169357+0000","last_became_peered":"2023-09-23T03:15:21.169357+0000","last_unstale":"2023-09-23T03:15:25.314809+0000","last_undegraded":"2023-09-23T03:15:25.314809+0000","last_fullsized":"2023-09-23T03:15:25.314809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:21:27.583218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.297693+0000","last_change":"2023-09-23T03:15:21.163391+0000","last_active":"2023-09-23T03:15:25.297693+0000","last_peered":"2023-09-23T03:15:25.297693+0000","last_clean":"2023-09-23T03:15:25.297693+0000","last_became_active":"2023-09-23T03:15:21.162413+0000","last_became_peered":"2023-09-23T03:15:21.162413+0000","last_unstale":"2023-09-23T03:15:25.297693+0000","last_undegraded":"2023-09-23T03:15:25.297693+0000","last_fullsized":"2023-09-23T03:15:25.297693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:30:31.951544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315898+0000","last_change":"2023-09-23T03:15:21.171540+0000","last_active":"2023-09-23T03:15:25.315898+0000","last_peered":"2023-09-23T03:15:25.315898+0000","last_clean":"2023-09-23T03:15:25.315898+0000","last_became_active":"2023-09-23T03:15:21.167329+0000","last_became_peered":"2023-09-23T03:15:21.167329+0000","last_unstale":"2023-09-23T03:15:25.315898+0000","last_undegraded":"2023-09-23T03:15:25.315898+0000","last_fullsized":"2023-09-23T03:15:25.315898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:29:14.943541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.304917+0000","last_change":"2023-09-23T03:15:21.167575+0000","last_active":"2023-09-23T03:15:25.304917+0000","last_peered":"2023-09-23T03:15:25.304917+0000","last_clean":"2023-09-23T03:15:25.304917+0000","last_became_active":"2023-09-23T03:15:21.167260+0000","last_became_peered":"2023-09-23T03:15:21.167260+0000","last_unstale":"2023-09-23T03:15:25.304917+0000","last_undegraded":"2023-09-23T03:15:25.304917+0000","last_fullsized":"2023-09-23T03:15:25.304917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:01:05.010735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.314454+0000","last_change":"2023-09-23T03:15:21.151218+0000","last_active":"2023-09-23T03:15:25.314454+0000","last_peered":"2023-09-23T03:15:25.314454+0000","last_clean":"2023-09-23T03:15:25.314454+0000","last_became_active":"2023-09-23T03:15:21.150758+0000","last_became_peered":"2023-09-23T03:15:21.150758+0000","last_unstale":"2023-09-23T03:15:25.314454+0000","last_undegraded":"2023-09-23T03:15:25.314454+0000","last_fullsized":"2023-09-23T03:15:25.314454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:51:00.098552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.445962+0000","last_change":"2023-09-23T03:15:21.175081+0000","last_active":"2023-09-23T03:15:25.445962+0000","last_peered":"2023-09-23T03:15:25.445962+0000","last_clean":"2023-09-23T03:15:25.445962+0000","last_became_active":"2023-09-23T03:15:21.174757+0000","last_became_peered":"2023-09-23T03:15:21.174757+0000","last_unstale":"2023-09-23T03:15:25.445962+0000","last_undegraded":"2023-09-23T03:15:25.445962+0000","last_fullsized":"2023-09-23T03:15:25.445962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:02:09.034422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.301326+0000","last_change":"2023-09-23T03:15:21.165007+0000","last_active":"2023-09-23T03:15:25.301326+0000","last_peered":"2023-09-23T03:15:25.301326+0000","last_clean":"2023-09-23T03:15:25.301326+0000","last_became_active":"2023-09-23T03:15:21.164591+0000","last_became_peered":"2023-09-23T03:15:21.164591+0000","last_unstale":"2023-09-23T03:15:25.301326+0000","last_undegraded":"2023-09-23T03:15:25.301326+0000","last_fullsized":"2023-09-23T03:15:25.301326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:13:22.079955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315446+0000","last_change":"2023-09-23T03:15:21.166029+0000","last_active":"2023-09-23T03:15:25.315446+0000","last_peered":"2023-09-23T03:15:25.315446+0000","last_clean":"2023-09-23T03:15:25.315446+0000","last_became_active":"2023-09-23T03:15:21.165298+0000","last_became_peered":"2023-09-23T03:15:21.165298+0000","last_unstale":"2023-09-23T03:15:25.315446+0000","last_undegraded":"2023-09-23T03:15:25.315446+0000","last_fullsized":"2023-09-23T03:15:25.315446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:42:37.443384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.314713+0000","last_change":"2023-09-23T03:15:21.172091+0000","last_active":"2023-09-23T03:15:25.314713+0000","last_peered":"2023-09-23T03:15:25.314713+0000","last_clean":"2023-09-23T03:15:25.314713+0000","last_became_active":"2023-09-23T03:15:21.170229+0000","last_became_peered":"2023-09-23T03:15:21.170229+0000","last_unstale":"2023-09-23T03:15:25.314713+0000","last_undegraded":"2023-09-23T03:15:25.314713+0000","last_fullsized":"2023-09-23T03:15:25.314713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:15:19.584603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.311046+0000","last_change":"2023-09-23T03:15:21.163298+0000","last_active":"2023-09-23T03:15:25.311046+0000","last_peered":"2023-09-23T03:15:25.311046+0000","last_clean":"2023-09-23T03:15:25.311046+0000","last_became_active":"2023-09-23T03:15:21.163000+0000","last_became_peered":"2023-09-23T03:15:21.163000+0000","last_unstale":"2023-09-23T03:15:25.311046+0000","last_undegraded":"2023-09-23T03:15:25.311046+0000","last_fullsized":"2023-09-23T03:15:25.311046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:22:07.336440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.301412+0000","last_change":"2023-09-23T03:15:21.160829+0000","last_active":"2023-09-23T03:15:25.301412+0000","last_peered":"2023-09-23T03:15:25.301412+0000","last_clean":"2023-09-23T03:15:25.301412+0000","last_became_active":"2023-09-23T03:15:21.160567+0000","last_became_peered":"2023-09-23T03:15:21.160567+0000","last_unstale":"2023-09-23T03:15:25.301412+0000","last_undegraded":"2023-09-23T03:15:25.301412+0000","last_fullsized":"2023-09-23T03:15:25.301412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:06:16.954649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.296315+0000","last_change":"2023-09-23T03:15:21.165912+0000","last_active":"2023-09-23T03:15:25.296315+0000","last_peered":"2023-09-23T03:15:25.296315+0000","last_clean":"2023-09-23T03:15:25.296315+0000","last_became_active":"2023-09-23T03:15:21.165634+0000","last_became_peered":"2023-09-23T03:15:21.165634+0000","last_unstale":"2023-09-23T03:15:25.296315+0000","last_undegraded":"2023-09-23T03:15:25.296315+0000","last_fullsized":"2023-09-23T03:15:25.296315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:15:19.584603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.309291+0000","last_change":"2023-09-23T03:15:21.161315+0000","last_active":"2023-09-23T03:15:25.309291+0000","last_peered":"2023-09-23T03:15:25.309291+0000","last_clean":"2023-09-23T03:15:25.309291+0000","last_became_active":"2023-09-23T03:15:21.160978+0000","last_became_peered":"2023-09-23T03:15:21.160978+0000","last_unstale":"2023-09-23T03:15:25.309291+0000","last_undegraded":"2023-09-23T03:15:25.309291+0000","last_fullsized":"2023-09-23T03:15:25.309291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:28:09.982438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.305734+0000","last_change":"2023-09-23T03:15:21.172253+0000","last_active":"2023-09-23T03:15:25.305734+0000","last_peered":"2023-09-23T03:15:25.305734+0000","last_clean":"2023-09-23T03:15:25.305734+0000","last_became_active":"2023-09-23T03:15:21.171987+0000","last_became_peered":"2023-09-23T03:15:21.171987+0000","last_unstale":"2023-09-23T03:15:25.305734+0000","last_undegraded":"2023-09-23T03:15:25.305734+0000","last_fullsized":"2023-09-23T03:15:25.305734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:30:31.951544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.303805+0000","last_change":"2023-09-23T03:15:21.172883+0000","last_active":"2023-09-23T03:15:25.303805+0000","last_peered":"2023-09-23T03:15:25.303805+0000","last_clean":"2023-09-23T03:15:25.303805+0000","last_became_active":"2023-09-23T03:15:21.172404+0000","last_became_peered":"2023-09-23T03:15:21.172404+0000","last_unstale":"2023-09-23T03:15:25.303805+0000","last_undegraded":"2023-09-23T03:15:25.303805+0000","last_fullsized":"2023-09-23T03:15:25.303805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:21:27.583218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.445963+0000","last_change":"2023-09-23T03:15:21.174812+0000","last_active":"2023-09-23T03:15:25.445963+0000","last_peered":"2023-09-23T03:15:25.445963+0000","last_clean":"2023-09-23T03:15:25.445963+0000","last_became_active":"2023-09-23T03:15:21.174241+0000","last_became_peered":"2023-09-23T03:15:21.174241+0000","last_unstale":"2023-09-23T03:15:25.445963+0000","last_undegraded":"2023-09-23T03:15:25.445963+0000","last_fullsized":"2023-09-23T03:15:25.445963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:58:02.940532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.295571+0000","last_change":"2023-09-23T03:15:21.170905+0000","last_active":"2023-09-23T03:15:25.295571+0000","last_peered":"2023-09-23T03:15:25.295571+0000","last_clean":"2023-09-23T03:15:25.295571+0000","last_became_active":"2023-09-23T03:15:21.170570+0000","last_became_peered":"2023-09-23T03:15:21.170570+0000","last_unstale":"2023-09-23T03:15:25.295571+0000","last_undegraded":"2023-09-23T03:15:25.295571+0000","last_fullsized":"2023-09-23T03:15:25.295571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:23:07.611380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.441959+0000","last_change":"2023-09-23T03:15:21.171879+0000","last_active":"2023-09-23T03:15:25.441959+0000","last_peered":"2023-09-23T03:15:25.441959+0000","last_clean":"2023-09-23T03:15:25.441959+0000","last_became_active":"2023-09-23T03:15:21.171624+0000","last_became_peered":"2023-09-23T03:15:21.171624+0000","last_unstale":"2023-09-23T03:15:25.441959+0000","last_undegraded":"2023-09-23T03:15:25.441959+0000","last_fullsized":"2023-09-23T03:15:25.441959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:39:10.586582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315796+0000","last_change":"2023-09-23T03:15:21.177809+0000","last_active":"2023-09-23T03:15:25.315796+0000","last_peered":"2023-09-23T03:15:25.315796+0000","last_clean":"2023-09-23T03:15:25.315796+0000","last_became_active":"2023-09-23T03:15:21.175320+0000","last_became_peered":"2023-09-23T03:15:21.175320+0000","last_unstale":"2023-09-23T03:15:25.315796+0000","last_undegraded":"2023-09-23T03:15:25.315796+0000","last_fullsized":"2023-09-23T03:15:25.315796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:30:37.652428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.441779+0000","last_change":"2023-09-23T03:15:21.174361+0000","last_active":"2023-09-23T03:15:25.441779+0000","last_peered":"2023-09-23T03:15:25.441779+0000","last_clean":"2023-09-23T03:15:25.441779+0000","last_became_active":"2023-09-23T03:15:21.174101+0000","last_became_peered":"2023-09-23T03:15:21.174101+0000","last_unstale":"2023-09-23T03:15:25.441779+0000","last_undegraded":"2023-09-23T03:15:25.441779+0000","last_fullsized":"2023-09-23T03:15:25.441779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:25:53.924266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.315094+0000","last_change":"2023-09-23T03:15:21.171837+0000","last_active":"2023-09-23T03:15:25.315094+0000","last_peered":"2023-09-23T03:15:25.315094+0000","last_clean":"2023-09-23T03:15:25.315094+0000","last_became_active":"2023-09-23T03:15:21.169973+0000","last_became_peered":"2023-09-23T03:15:21.169973+0000","last_unstale":"2023-09-23T03:15:25.315094+0000","last_undegraded":"2023-09-23T03:15:25.315094+0000","last_fullsized":"2023-09-23T03:15:25.315094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:53:21.435687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.303645+0000","last_change":"2023-09-23T03:15:21.157118+0000","last_active":"2023-09-23T03:15:25.303645+0000","last_peered":"2023-09-23T03:15:25.303645+0000","last_clean":"2023-09-23T03:15:25.303645+0000","last_became_active":"2023-09-23T03:15:21.156862+0000","last_became_peered":"2023-09-23T03:15:21.156862+0000","last_unstale":"2023-09-23T03:15:25.303645+0000","last_undegraded":"2023-09-23T03:15:25.303645+0000","last_fullsized":"2023-09-23T03:15:25.303645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:22:07.336440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.311650+0000","last_change":"2023-09-23T03:15:21.167539+0000","last_active":"2023-09-23T03:15:25.311650+0000","last_peered":"2023-09-23T03:15:25.311650+0000","last_clean":"2023-09-23T03:15:25.311650+0000","last_became_active":"2023-09-23T03:15:21.167256+0000","last_became_peered":"2023-09-23T03:15:21.167256+0000","last_unstale":"2023-09-23T03:15:25.311650+0000","last_undegraded":"2023-09-23T03:15:25.311650+0000","last_fullsized":"2023-09-23T03:15:25.311650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:21:27.583218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-09-23T03:15:25.445518+0000","last_change":"2023-09-23T03:15:21.171479+0000","last_active":"2023-09-23T03:15:25.445518+0000","last_peered":"2023-09-23T03:15:25.445518+0000","last_clean":"2023-09-23T03:15:25.445518+0000","last_became_active":"2023-09-23T03:15:21.171128+0000","last_became_peered":"2023-09-23T03:15:21.171128+0000","last_unstale":"2023-09-23T03:15:25.445518+0000","last_undegraded":"2023-09-23T03:15:25.445518+0000","last_fullsized":"2023-09-23T03:15:25.445518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:20.207282+0000","last_clean_scrub_stamp":"2023-09-23T03:15:20.207282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:38:51.387638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":851968,"data_stored":690730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.504}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7130000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8200000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.383}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0489999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.258}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4410000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574861,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":99870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.409}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.204}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4530000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.52}]}]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":99906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0920000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.345}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.149}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2829999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.548}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.524}]}]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0489999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.419}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2809999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574861,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":99780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1870000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.46}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.284}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.486}]}]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":262144,"data_stored":99940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]}]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":689767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.21}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-23T03:15:56.736 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-09-23T03:15:56.736 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-09-23T03:15:56.736 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-09-23T03:15:56.736 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-23T03:15:56.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.982+0000 7fe9d7389640 1 Processor -- start 2023-09-23T03:15:56.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.982+0000 7fe9d7389640 1 -- start start 2023-09-23T03:15:56.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.983+0000 7fe9d7389640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9d006c7a0 0x7fe9d015eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:56.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.983+0000 7fe9d7389640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe9d015ab00 0x7fe9d015af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:56.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.984+0000 7fe9d7389640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9d006bff0 con 0x7fe9d006fdc0 2023-09-23T03:15:56.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.984+0000 7fe9d7389640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9d0155070 con 0x7fe9d015ab00 2023-09-23T03:15:56.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.984+0000 7fe9d7389640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d015e0f0 con 0x7fe9d006c7a0 2023-09-23T03:15:56.986 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.984+0000 7fe9d5125640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9d006c7a0 0x7fe9d015eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:56.986 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.984+0000 7fe9d5926640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fe9d006fdc0 0x7fe9d00ca500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.154:46678/0 (socket says 172.21.15.154:46678) 2023-09-23T03:15:56.986 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.984+0000 7fe9d5926640 1 -- 172.21.15.154:0/494994168 learned_addr learned my addr 172.21.15.154:0/494994168 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:56.986 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.984+0000 7fe9d4924640 1 --2- 172.21.15.154:0/494994168 >> v2:172.21.15.190:3300/0 conn(0x7fe9d015ab00 0x7fe9d015af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:56.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.985+0000 7fe9c7fff640 1 -- 172.21.15.154:0/494994168 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3957147048 0 0) 0x7fe9d006bff0 con 0x7fe9d006fdc0 2023-09-23T03:15:56.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.985+0000 7fe9c7fff640 1 -- 172.21.15.154:0/494994168 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9bc003520 con 0x7fe9d006fdc0 2023-09-23T03:15:56.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.985+0000 7fe9d4924640 1 -- 172.21.15.154:0/494994168 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9d006c7a0 msgr2=0x7fe9d015eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:56.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.985+0000 7fe9d4924640 1 --2- 172.21.15.154:0/494994168 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9d006c7a0 0x7fe9d015eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:56.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.985+0000 7fe9d4924640 1 -- 172.21.15.154:0/494994168 >> v1:172.21.15.190:6789/0 conn(0x7fe9d006fdc0 legacy=0x7fe9d00ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:56.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.985+0000 7fe9d4924640 1 -- 172.21.15.154:0/494994168 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d006cb80 con 0x7fe9d015ab00 2023-09-23T03:15:56.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.986+0000 7fe9d4924640 1 --2- 172.21.15.154:0/494994168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9d015ab00 0x7fe9d015af80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00097a0 tx=0x7fe9c002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=381fe8548d1d22cb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:56.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.987+0000 7fe9c7fff640 1 -- 172.21.15.154:0/494994168 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c002d040 con 0x7fe9d015ab00 2023-09-23T03:15:56.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.987+0000 7fe9c7fff640 1 -- 172.21.15.154:0/494994168 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c0031810 con 0x7fe9d015ab00 2023-09-23T03:15:56.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.987+0000 7fe9c7fff640 1 -- 172.21.15.154:0/494994168 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0031ba0 con 0x7fe9d015ab00 2023-09-23T03:15:56.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.988+0000 7fe9d7389640 1 -- 172.21.15.154:0/494994168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9d015ab00 msgr2=0x7fe9d015af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:56.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.988+0000 7fe9d7389640 1 --2- 172.21.15.154:0/494994168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9d015ab00 0x7fe9d015af80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe9c00097a0 tx=0x7fe9c002b8b0 comp rx=0 tx=0).stop 2023-09-23T03:15:56.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.988+0000 7fe9d7389640 1 -- 172.21.15.154:0/494994168 shutdown_connections 2023-09-23T03:15:56.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.988+0000 7fe9d7389640 1 --2- 172.21.15.154:0/494994168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9d015ab00 0x7fe9d015af80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:56.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.988+0000 7fe9d7389640 1 --2- 172.21.15.154:0/494994168 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9d006c7a0 0x7fe9d015eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:56.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.988+0000 7fe9d7389640 1 -- 172.21.15.154:0/494994168 >> 172.21.15.154:0/494994168 conn(0x7fe9d006a390 msgr2=0x7fe9d00afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:56.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.988+0000 7fe9d7389640 1 -- 172.21.15.154:0/494994168 shutdown_connections 2023-09-23T03:15:56.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.988+0000 7fe9d7389640 1 -- 172.21.15.154:0/494994168 wait complete. 2023-09-23T03:15:56.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.990+0000 7fe9d7389640 1 Processor -- start 2023-09-23T03:15:56.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.990+0000 7fe9d7389640 1 -- start start 2023-09-23T03:15:56.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.991+0000 7fe9d7389640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9d006c7a0 0x7fe9d0153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:56.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.991+0000 7fe9d7389640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9d006fdc0 0x7fe9d00c6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:56.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.991+0000 7fe9d7389640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9d015ab00 0x7fe9d00c5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:56.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.991+0000 7fe9d7389640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d00c6b10 con 0x7fe9d015ab00 2023-09-23T03:15:56.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.991+0000 7fe9d7389640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d0153e20 con 0x7fe9d006fdc0 2023-09-23T03:15:56.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.991+0000 7fe9d7389640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d00c5f00 con 0x7fe9d006c7a0 2023-09-23T03:15:56.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.992+0000 7fe9d5926640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9d015ab00 0x7fe9d00c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:56.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.992+0000 7fe9d5926640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9d015ab00 0x7fe9d00c5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44612/0 (socket says 172.21.15.154:44612) 2023-09-23T03:15:56.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.992+0000 7fe9d5926640 1 -- 172.21.15.154:0/612229906 learned_addr learned my addr 172.21.15.154:0/612229906 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:56.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.992+0000 7fe9d4924640 1 --2- 172.21.15.154:0/612229906 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9d006fdc0 0x7fe9d00c6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:56.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.992+0000 7fe9d5125640 1 --2- 172.21.15.154:0/612229906 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9d006c7a0 0x7fe9d0153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:56.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.992+0000 7fe9d5926640 1 -- 172.21.15.154:0/612229906 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9d006c7a0 msgr2=0x7fe9d0153360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:56.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.992+0000 7fe9d5926640 1 --2- 172.21.15.154:0/612229906 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9d006c7a0 0x7fe9d0153360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:56.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.992+0000 7fe9d5926640 1 -- 172.21.15.154:0/612229906 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9d006fdc0 msgr2=0x7fe9d00c6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:56.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.992+0000 7fe9d5926640 1 --2- 172.21.15.154:0/612229906 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9d006fdc0 0x7fe9d00c6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:56.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.992+0000 7fe9d5926640 1 -- 172.21.15.154:0/612229906 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c0009400 con 0x7fe9d015ab00 2023-09-23T03:15:56.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.992+0000 7fe9d5926640 1 --2- 172.21.15.154:0/612229906 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9d015ab00 0x7fe9d00c5390 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800cb40 tx=0x7fe9c800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:56.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.993+0000 7fe9c5ffb640 1 -- 172.21.15.154:0/612229906 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c800aca0 con 0x7fe9d015ab00 2023-09-23T03:15:56.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.993+0000 7fe9c5ffb640 1 -- 172.21.15.154:0/612229906 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c800ae00 con 0x7fe9d015ab00 2023-09-23T03:15:56.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.993+0000 7fe9c5ffb640 1 -- 172.21.15.154:0/612229906 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8012620 con 0x7fe9d015ab00 2023-09-23T03:15:56.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.993+0000 7fe9d7389640 1 -- 172.21.15.154:0/612229906 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9d00c61e0 con 0x7fe9d015ab00 2023-09-23T03:15:56.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.993+0000 7fe9d7389640 1 -- 172.21.15.154:0/612229906 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d00c67a0 con 0x7fe9d015ab00 2023-09-23T03:15:56.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.994+0000 7fe9d7389640 1 -- 172.21.15.154:0/612229906 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9d00736b0 con 0x7fe9d015ab00 2023-09-23T03:15:56.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.995+0000 7fe9c5ffb640 1 -- 172.21.15.154:0/612229906 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fe9c8007070 con 0x7fe9d015ab00 2023-09-23T03:15:56.997 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.996+0000 7fe9c5ffb640 1 --2- 172.21.15.154:0/612229906 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe9b406af60 0x7fe9b406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:56.997 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.996+0000 7fe9c5ffb640 1 -- 172.21.15.154:0/612229906 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe9c8089b50 con 0x7fe9d015ab00 2023-09-23T03:15:56.997 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.996+0000 7fe9d5125640 1 --2- 172.21.15.154:0/612229906 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe9b406af60 0x7fe9b406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:56.997 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:56.996+0000 7fe9d5125640 1 --2- 172.21.15.154:0/612229906 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe9b406af60 0x7fe9b406d420 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009ad0 tx=0x7fe9cc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:57.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.000+0000 7fe9c5ffb640 1 -- 172.21.15.154:0/612229906 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fe9c8054800 con 0x7fe9d015ab00 2023-09-23T03:15:57.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.225+0000 7fe9d7389640 1 -- 172.21.15.154:0/612229906 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fe9d004e920 con 0x7fe9d015ab00 2023-09-23T03:15:57.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.225+0000 7fe9c5ffb640 1 -- 172.21.15.154:0/612229906 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fe9c8017080 con 0x7fe9d015ab00 2023-09-23T03:15:57.227 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:15:57.227 INFO:teuthology.orchestra.run.smithi154.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-09-23T03:15:57.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.227+0000 7fe9d7389640 1 -- 172.21.15.154:0/612229906 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe9b406af60 msgr2=0x7fe9b406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:57.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.227+0000 7fe9d7389640 1 --2- 172.21.15.154:0/612229906 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe9b406af60 0x7fe9b406d420 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009ad0 tx=0x7fe9cc008040 comp rx=0 tx=0).stop 2023-09-23T03:15:57.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.227+0000 7fe9d7389640 1 -- 172.21.15.154:0/612229906 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9d015ab00 msgr2=0x7fe9d00c5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:57.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.227+0000 7fe9d7389640 1 --2- 172.21.15.154:0/612229906 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9d015ab00 0x7fe9d00c5390 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800cb40 tx=0x7fe9c800a060 comp rx=0 tx=0).stop 2023-09-23T03:15:57.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.228+0000 7fe9d7389640 1 -- 172.21.15.154:0/612229906 shutdown_connections 2023-09-23T03:15:57.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.228+0000 7fe9d7389640 1 --2- 172.21.15.154:0/612229906 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe9b406af60 0x7fe9b406d420 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:57.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.228+0000 7fe9d7389640 1 --2- 172.21.15.154:0/612229906 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9d015ab00 0x7fe9d00c5390 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:57.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.228+0000 7fe9d7389640 1 --2- 172.21.15.154:0/612229906 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9d006fdc0 0x7fe9d00c6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:57.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.228+0000 7fe9d7389640 1 --2- 172.21.15.154:0/612229906 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9d006c7a0 0x7fe9d0153360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:57.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.228+0000 7fe9d7389640 1 -- 172.21.15.154:0/612229906 >> 172.21.15.154:0/612229906 conn(0x7fe9d006a390 msgr2=0x7fe9d00ce210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:57.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.228+0000 7fe9d7389640 1 -- 172.21.15.154:0/612229906 shutdown_connections 2023-09-23T03:15:57.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.228+0000 7fe9d7389640 1 -- 172.21.15.154:0/612229906 wait complete. 2023-09-23T03:15:57.250 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-09-23T03:15:57.250 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:15:57.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.436+0000 7fde3a6a2640 1 Processor -- start 2023-09-23T03:15:57.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.437+0000 7fde3a6a2640 1 -- start start 2023-09-23T03:15:57.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.438+0000 7fde3a6a2640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fde340ada30 0x7fde340ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:57.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.438+0000 7fde3a6a2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde340cf130 0x7fde3415d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:57.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.438+0000 7fde3a6a2640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde3406ceb0 con 0x7fde340b2fe0 2023-09-23T03:15:57.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.438+0000 7fde3a6a2640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde340ade10 con 0x7fde340ada30 2023-09-23T03:15:57.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.438+0000 7fde3a6a2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde34159c90 con 0x7fde340cf130 2023-09-23T03:15:57.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.439+0000 7fde38c3f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde340cf130 0x7fde3415d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:57.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.439+0000 7fde38c3f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde340cf130 0x7fde3415d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:34760/0 (socket says 172.21.15.154:34760) 2023-09-23T03:15:57.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.439+0000 7fde38c3f640 1 -- 172.21.15.154:0/4125523043 learned_addr learned my addr 172.21.15.154:0/4125523043 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:57.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.439+0000 7fde2bfff640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fde340b2fe0 0x7fde340cea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.154:46690/0 (socket says 172.21.15.154:46690) 2023-09-23T03:15:57.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.439+0000 7fde33fff640 1 --2- 172.21.15.154:0/4125523043 >> v2:172.21.15.190:3300/0 conn(0x7fde340ada30 0x7fde340ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:57.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.439+0000 7fde38c3f640 1 -- 172.21.15.154:0/4125523043 >> v2:172.21.15.190:3300/0 conn(0x7fde340ada30 msgr2=0x7fde340ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:57.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.439+0000 7fde38c3f640 1 --2- 172.21.15.154:0/4125523043 >> v2:172.21.15.190:3300/0 conn(0x7fde340ada30 0x7fde340ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:57.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.440+0000 7fde38c3f640 1 -- 172.21.15.154:0/4125523043 >> v1:172.21.15.190:6789/0 conn(0x7fde340b2fe0 legacy=0x7fde340cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:57.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.440+0000 7fde38c3f640 1 -- 172.21.15.154:0/4125523043 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde3406c110 con 0x7fde340cf130 2023-09-23T03:15:57.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.441+0000 7fde38c3f640 1 --2- 172.21.15.154:0/4125523043 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde340cf130 0x7fde3415d380 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fde2c009e50 tx=0x7fde2c02e170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c81bc0200b5ad0f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:57.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.442+0000 7fde337fe640 1 -- 172.21.15.154:0/4125523043 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2c02ed10 con 0x7fde340cf130 2023-09-23T03:15:57.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.442+0000 7fde337fe640 1 -- 172.21.15.154:0/4125523043 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde2c02ee70 con 0x7fde340cf130 2023-09-23T03:15:57.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.442+0000 7fde337fe640 1 -- 172.21.15.154:0/4125523043 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2c0365f0 con 0x7fde340cf130 2023-09-23T03:15:57.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.442+0000 7fde3a6a2640 1 -- 172.21.15.154:0/4125523043 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde340cf130 msgr2=0x7fde3415d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:57.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.442+0000 7fde3a6a2640 1 --2- 172.21.15.154:0/4125523043 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde340cf130 0x7fde3415d380 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fde2c009e50 tx=0x7fde2c02e170 comp rx=0 tx=0).stop 2023-09-23T03:15:57.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.442+0000 7fde3a6a2640 1 -- 172.21.15.154:0/4125523043 shutdown_connections 2023-09-23T03:15:57.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.442+0000 7fde3a6a2640 1 --2- 172.21.15.154:0/4125523043 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde340cf130 0x7fde3415d380 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:57.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.442+0000 7fde3a6a2640 1 --2- 172.21.15.154:0/4125523043 >> v2:172.21.15.190:3300/0 conn(0x7fde340ada30 0x7fde340ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:57.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.442+0000 7fde3a6a2640 1 -- 172.21.15.154:0/4125523043 >> 172.21.15.154:0/4125523043 conn(0x7fde3406a0f0 msgr2=0x7fde3406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:57.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.442+0000 7fde3a6a2640 1 -- 172.21.15.154:0/4125523043 shutdown_connections 2023-09-23T03:15:57.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.442+0000 7fde3a6a2640 1 -- 172.21.15.154:0/4125523043 wait complete. 2023-09-23T03:15:57.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.444+0000 7fde3a6a2640 1 Processor -- start 2023-09-23T03:15:57.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.444+0000 7fde3a6a2640 1 -- start start 2023-09-23T03:15:57.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.445+0000 7fde3a6a2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde340ada30 0x7fde3415c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:57.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.445+0000 7fde3a6a2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde340b2fe0 0x7fde341f0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:57.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.445+0000 7fde3a6a2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde340cf130 0x7fde340cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:57.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.445+0000 7fde3a6a2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde341f1340 con 0x7fde340ada30 2023-09-23T03:15:57.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.445+0000 7fde3a6a2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde341f1dd0 con 0x7fde340b2fe0 2023-09-23T03:15:57.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.445+0000 7fde3a6a2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde341ed7d0 con 0x7fde340cf130 2023-09-23T03:15:57.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.446+0000 7fde33fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde340ada30 0x7fde3415c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:57.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.446+0000 7fde33fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde340ada30 0x7fde3415c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44628/0 (socket says 172.21.15.154:44628) 2023-09-23T03:15:57.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.446+0000 7fde33fff640 1 -- 172.21.15.154:0/1278648255 learned_addr learned my addr 172.21.15.154:0/1278648255 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:57.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.446+0000 7fde2bfff640 1 --2- 172.21.15.154:0/1278648255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde340b2fe0 0x7fde341f0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:57.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.446+0000 7fde38c3f640 1 --2- 172.21.15.154:0/1278648255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde340cf130 0x7fde340cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:57.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.446+0000 7fde33fff640 1 -- 172.21.15.154:0/1278648255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde340cf130 msgr2=0x7fde340cbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:57.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.446+0000 7fde33fff640 1 --2- 172.21.15.154:0/1278648255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde340cf130 0x7fde340cbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:57.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.446+0000 7fde33fff640 1 -- 172.21.15.154:0/1278648255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde340b2fe0 msgr2=0x7fde341f0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:57.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.446+0000 7fde33fff640 1 --2- 172.21.15.154:0/1278648255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde340b2fe0 0x7fde341f0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:57.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.446+0000 7fde33fff640 1 -- 172.21.15.154:0/1278648255 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde2c009b30 con 0x7fde340ada30 2023-09-23T03:15:57.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.446+0000 7fde2bfff640 1 --2- 172.21.15.154:0/1278648255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde340b2fe0 0x7fde341f0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:15:57.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.447+0000 7fde33fff640 1 --2- 172.21.15.154:0/1278648255 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde340ada30 0x7fde3415c940 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fde2400e870 tx=0x7fde2400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:57.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.447+0000 7fde317fa640 1 -- 172.21.15.154:0/1278648255 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde240095c0 con 0x7fde340ada30 2023-09-23T03:15:57.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.447+0000 7fde317fa640 1 -- 172.21.15.154:0/1278648255 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde2400c940 con 0x7fde340ada30 2023-09-23T03:15:57.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.447+0000 7fde317fa640 1 -- 172.21.15.154:0/1278648255 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2400ccb0 con 0x7fde340ada30 2023-09-23T03:15:57.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.447+0000 7fde3a6a2640 1 -- 172.21.15.154:0/1278648255 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde341f1530 con 0x7fde340ada30 2023-09-23T03:15:57.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.447+0000 7fde3a6a2640 1 -- 172.21.15.154:0/1278648255 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde341f1af0 con 0x7fde340ada30 2023-09-23T03:15:57.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.448+0000 7fde3a6a2640 1 -- 172.21.15.154:0/1278648255 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde34073eb0 con 0x7fde340ada30 2023-09-23T03:15:57.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.449+0000 7fde317fa640 1 -- 172.21.15.154:0/1278648255 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fde24013470 con 0x7fde340ada30 2023-09-23T03:15:57.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.450+0000 7fde317fa640 1 --2- 172.21.15.154:0/1278648255 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fde1406af30 0x7fde1406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:57.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.450+0000 7fde317fa640 1 -- 172.21.15.154:0/1278648255 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fde2408a080 con 0x7fde340ada30 2023-09-23T03:15:57.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.450+0000 7fde2bfff640 1 --2- 172.21.15.154:0/1278648255 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fde1406af30 0x7fde1406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:57.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.451+0000 7fde2bfff640 1 --2- 172.21.15.154:0/1278648255 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fde1406af30 0x7fde1406d3f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fde1c007fd0 tx=0x7fde1c007d40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:57.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.455+0000 7fde317fa640 1 -- 172.21.15.154:0/1278648255 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fde240bca50 con 0x7fde340ada30 2023-09-23T03:15:57.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.631+0000 7fde3a6a2640 1 -- 172.21.15.154:0/1278648255 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fde3406dc60 con 0x7fde340ada30 2023-09-23T03:15:57.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.635+0000 7fde317fa640 1 -- 172.21.15.154:0/1278648255 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18382 (secure 0 0 0) 0x7fde24021090 con 0x7fde340ada30 2023-09-23T03:15:57.636 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:15:57.637 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":24,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:15:25.775060+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:15:18.575849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T03:15:19.707212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T03:15:23.337138+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:15:57.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.637+0000 7fde3a6a2640 1 -- 172.21.15.154:0/1278648255 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fde1406af30 msgr2=0x7fde1406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:57.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.637+0000 7fde3a6a2640 1 --2- 172.21.15.154:0/1278648255 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fde1406af30 0x7fde1406d3f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fde1c007fd0 tx=0x7fde1c007d40 comp rx=0 tx=0).stop 2023-09-23T03:15:57.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.637+0000 7fde3a6a2640 1 -- 172.21.15.154:0/1278648255 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde340ada30 msgr2=0x7fde3415c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:57.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.637+0000 7fde3a6a2640 1 --2- 172.21.15.154:0/1278648255 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde340ada30 0x7fde3415c940 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fde2400e870 tx=0x7fde2400ed30 comp rx=0 tx=0).stop 2023-09-23T03:15:57.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.637+0000 7fde3a6a2640 1 -- 172.21.15.154:0/1278648255 shutdown_connections 2023-09-23T03:15:57.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.637+0000 7fde3a6a2640 1 --2- 172.21.15.154:0/1278648255 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fde1406af30 0x7fde1406d3f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:57.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.637+0000 7fde3a6a2640 1 --2- 172.21.15.154:0/1278648255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde340cf130 0x7fde340cbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:57.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.637+0000 7fde3a6a2640 1 --2- 172.21.15.154:0/1278648255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde340b2fe0 0x7fde341f0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:57.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.637+0000 7fde3a6a2640 1 --2- 172.21.15.154:0/1278648255 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde340ada30 0x7fde3415c940 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:57.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.637+0000 7fde3a6a2640 1 -- 172.21.15.154:0/1278648255 >> 172.21.15.154:0/1278648255 conn(0x7fde3406a0f0 msgr2=0x7fde340713d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:57.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.638+0000 7fde3a6a2640 1 -- 172.21.15.154:0/1278648255 shutdown_connections 2023-09-23T03:15:57.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.638+0000 7fde3a6a2640 1 -- 172.21.15.154:0/1278648255 wait complete. 2023-09-23T03:15:57.658 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:15:18.575849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:15:19.707212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-23T03:15:23.337138+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-23T03:15:57.659 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:15:57.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.856+0000 7fb3d50b4640 1 Processor -- start 2023-09-23T03:15:57.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.857+0000 7fb3d50b4640 1 -- start start 2023-09-23T03:15:57.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.858+0000 7fb3d50b4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3d0158160 0x7fb3d015ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:57.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.858+0000 7fb3d50b4640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb3d015e130 0x7fb3d00c9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:57.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.858+0000 7fb3d50b4640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb3d00c9a80 0x7fb3d006fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:57.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.858+0000 7fb3d50b4640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3d0158540 con 0x7fb3d015e130 2023-09-23T03:15:57.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.858+0000 7fb3d50b4640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3d015b3e0 con 0x7fb3d00c9a80 2023-09-23T03:15:57.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.858+0000 7fb3d50b4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d015b6c0 con 0x7fb3d0158160 2023-09-23T03:15:57.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.858+0000 7fb3c7fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb3d015e130 0x7fb3d00c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:57.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.858+0000 7fb3c7fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb3d015e130 0x7fb3d00c9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44640/0 (socket says 172.21.15.154:44640) 2023-09-23T03:15:57.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.858+0000 7fb3c7fff640 1 -- 172.21.15.154:0/4044725772 learned_addr learned my addr 172.21.15.154:0/4044725772 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:57.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.859+0000 7fb3cffff640 1 --2- 172.21.15.154:0/4044725772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3d0158160 0x7fb3d015ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:57.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.859+0000 7fb3c7fff640 1 -- 172.21.15.154:0/4044725772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3d0158160 msgr2=0x7fb3d015ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:57.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.859+0000 7fb3d48b3640 1 --2- 172.21.15.154:0/4044725772 >> v2:172.21.15.190:3300/0 conn(0x7fb3d00c9a80 0x7fb3d006fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:57.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.859+0000 7fb3c7fff640 1 --2- 172.21.15.154:0/4044725772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3d0158160 0x7fb3d015ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:57.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.859+0000 7fb3c7fff640 1 -- 172.21.15.154:0/4044725772 >> v2:172.21.15.190:3300/0 conn(0x7fb3d00c9a80 msgr2=0x7fb3d006fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:57.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.859+0000 7fb3c7fff640 1 --2- 172.21.15.154:0/4044725772 >> v2:172.21.15.190:3300/0 conn(0x7fb3d00c9a80 0x7fb3d006fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:57.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.859+0000 7fb3c7fff640 1 -- 172.21.15.154:0/4044725772 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d015b9a0 con 0x7fb3d015e130 2023-09-23T03:15:57.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.859+0000 7fb3d48b3640 1 --2- 172.21.15.154:0/4044725772 >> v2:172.21.15.190:3300/0 conn(0x7fb3d00c9a80 0x7fb3d006fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:57.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.860+0000 7fb3c7fff640 1 --2- 172.21.15.154:0/4044725772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3d015e130 0x7fb3d00c9540 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb3d00b0780 tx=0x7fb3b802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed7906fd139341db server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:57.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.860+0000 7fb3cf7fe640 1 -- 172.21.15.154:0/4044725772 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8031860 con 0x7fb3d015e130 2023-09-23T03:15:57.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.860+0000 7fb3cf7fe640 1 -- 172.21.15.154:0/4044725772 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b80319c0 con 0x7fb3d015e130 2023-09-23T03:15:57.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.860+0000 7fb3cf7fe640 1 -- 172.21.15.154:0/4044725772 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8031d50 con 0x7fb3d015e130 2023-09-23T03:15:57.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.861+0000 7fb3d50b4640 1 -- 172.21.15.154:0/4044725772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3d015e130 msgr2=0x7fb3d00c9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:57.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.861+0000 7fb3d50b4640 1 --2- 172.21.15.154:0/4044725772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3d015e130 0x7fb3d00c9540 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb3d00b0780 tx=0x7fb3b802b900 comp rx=0 tx=0).stop 2023-09-23T03:15:57.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.861+0000 7fb3d50b4640 1 -- 172.21.15.154:0/4044725772 shutdown_connections 2023-09-23T03:15:57.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.861+0000 7fb3d50b4640 1 --2- 172.21.15.154:0/4044725772 >> v2:172.21.15.190:3300/0 conn(0x7fb3d00c9a80 0x7fb3d006fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:57.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.861+0000 7fb3d50b4640 1 --2- 172.21.15.154:0/4044725772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3d015e130 0x7fb3d00c9540 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:57.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.861+0000 7fb3d50b4640 1 --2- 172.21.15.154:0/4044725772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3d0158160 0x7fb3d015ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:57.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.861+0000 7fb3d50b4640 1 -- 172.21.15.154:0/4044725772 >> 172.21.15.154:0/4044725772 conn(0x7fb3d006a300 msgr2=0x7fb3d00afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:57.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.861+0000 7fb3d50b4640 1 -- 172.21.15.154:0/4044725772 shutdown_connections 2023-09-23T03:15:57.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.861+0000 7fb3d50b4640 1 -- 172.21.15.154:0/4044725772 wait complete. 2023-09-23T03:15:57.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.863+0000 7fb3d50b4640 1 Processor -- start 2023-09-23T03:15:57.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.863+0000 7fb3d50b4640 1 -- start start 2023-09-23T03:15:57.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.864+0000 7fb3d50b4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3d0158160 0x7fb3d00ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:57.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.864+0000 7fb3d50b4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3d015e130 0x7fb3d00aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:57.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.864+0000 7fb3d50b4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3d00af310 0x7fb3d0147010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:57.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.864+0000 7fb3d50b4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d0146bf0 con 0x7fb3d00af310 2023-09-23T03:15:57.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.864+0000 7fb3d50b4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d00af7a0 con 0x7fb3d0158160 2023-09-23T03:15:57.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.864+0000 7fb3d50b4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d0145f70 con 0x7fb3d015e130 2023-09-23T03:15:57.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.864+0000 7fb3d48b3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3d00af310 0x7fb3d0147010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:57.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.864+0000 7fb3d48b3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3d00af310 0x7fb3d0147010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44644/0 (socket says 172.21.15.154:44644) 2023-09-23T03:15:57.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.864+0000 7fb3d48b3640 1 -- 172.21.15.154:0/1832613689 learned_addr learned my addr 172.21.15.154:0/1832613689 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:57.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.864+0000 7fb3cffff640 1 --2- 172.21.15.154:0/1832613689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3d0158160 0x7fb3d00ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:57.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.864+0000 7fb3d48b3640 1 -- 172.21.15.154:0/1832613689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3d015e130 msgr2=0x7fb3d00aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:57.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.864+0000 7fb3c7fff640 1 --2- 172.21.15.154:0/1832613689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3d015e130 0x7fb3d00aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:57.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.865+0000 7fb3d48b3640 1 --2- 172.21.15.154:0/1832613689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3d015e130 0x7fb3d00aedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:57.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.865+0000 7fb3d48b3640 1 -- 172.21.15.154:0/1832613689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3d0158160 msgr2=0x7fb3d00ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:57.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.865+0000 7fb3d48b3640 1 --2- 172.21.15.154:0/1832613689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3d0158160 0x7fb3d00ab8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:57.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.865+0000 7fb3d48b3640 1 -- 172.21.15.154:0/1832613689 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b80094d0 con 0x7fb3d00af310 2023-09-23T03:15:57.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.865+0000 7fb3cffff640 1 --2- 172.21.15.154:0/1832613689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3d0158160 0x7fb3d00ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:15:57.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.865+0000 7fb3c7fff640 1 --2- 172.21.15.154:0/1832613689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3d015e130 0x7fb3d00aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:57.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.865+0000 7fb3d48b3640 1 --2- 172.21.15.154:0/1832613689 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3d00af310 0x7fb3d0147010 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb3c800cd80 tx=0x7fb3c800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:57.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.865+0000 7fb3cd7fa640 1 -- 172.21.15.154:0/1832613689 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c800d840 con 0x7fb3d00af310 2023-09-23T03:15:57.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.865+0000 7fb3cd7fa640 1 -- 172.21.15.154:0/1832613689 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c800d9a0 con 0x7fb3d00af310 2023-09-23T03:15:57.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.865+0000 7fb3d50b4640 1 -- 172.21.15.154:0/1832613689 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3d0146250 con 0x7fb3d00af310 2023-09-23T03:15:57.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.865+0000 7fb3cd7fa640 1 -- 172.21.15.154:0/1832613689 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c800dd50 con 0x7fb3d00af310 2023-09-23T03:15:57.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.865+0000 7fb3d50b4640 1 -- 172.21.15.154:0/1832613689 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3d0146810 con 0x7fb3d00af310 2023-09-23T03:15:57.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.866+0000 7fb3d50b4640 1 -- 172.21.15.154:0/1832613689 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3d00736b0 con 0x7fb3d00af310 2023-09-23T03:15:57.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.867+0000 7fb3cd7fa640 1 -- 172.21.15.154:0/1832613689 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fb3c801e3b0 con 0x7fb3d00af310 2023-09-23T03:15:57.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.869+0000 7fb3cd7fa640 1 --2- 172.21.15.154:0/1832613689 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb3b006af30 0x7fb3b006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:57.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.869+0000 7fb3cd7fa640 1 -- 172.21.15.154:0/1832613689 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb3c808a3d0 con 0x7fb3d00af310 2023-09-23T03:15:57.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.869+0000 7fb3cffff640 1 --2- 172.21.15.154:0/1832613689 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb3b006af30 0x7fb3b006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:57.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.870+0000 7fb3cffff640 1 --2- 172.21.15.154:0/1832613689 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb3b006af30 0x7fb3b006d3f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb3c000a9d0 tx=0x7fb3c0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:57.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:57.873+0000 7fb3cd7fa640 1 -- 172.21.15.154:0/1832613689 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fb3c8055120 con 0x7fb3d00af310 2023-09-23T03:15:58.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.060+0000 7fb3d50b4640 1 -- 172.21.15.154:0/1832613689 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb3d0146d30 con 0x7fb3d00af310 2023-09-23T03:15:58.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.061+0000 7fb3cd7fa640 1 -- 172.21.15.154:0/1832613689 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb3d0146d30 con 0x7fb3d00af310 2023-09-23T03:15:58.062 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:15:58.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.063+0000 7fb3d50b4640 1 -- 172.21.15.154:0/1832613689 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb3b006af30 msgr2=0x7fb3b006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:58.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.063+0000 7fb3d50b4640 1 --2- 172.21.15.154:0/1832613689 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb3b006af30 0x7fb3b006d3f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb3c000a9d0 tx=0x7fb3c0008040 comp rx=0 tx=0).stop 2023-09-23T03:15:58.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.063+0000 7fb3d50b4640 1 -- 172.21.15.154:0/1832613689 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3d00af310 msgr2=0x7fb3d0147010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:58.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.063+0000 7fb3d50b4640 1 --2- 172.21.15.154:0/1832613689 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3d00af310 0x7fb3d0147010 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb3c800cd80 tx=0x7fb3c800a710 comp rx=0 tx=0).stop 2023-09-23T03:15:58.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.063+0000 7fb3d50b4640 1 -- 172.21.15.154:0/1832613689 shutdown_connections 2023-09-23T03:15:58.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.063+0000 7fb3d50b4640 1 --2- 172.21.15.154:0/1832613689 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb3b006af30 0x7fb3b006d3f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:58.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.063+0000 7fb3d50b4640 1 --2- 172.21.15.154:0/1832613689 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3d00af310 0x7fb3d0147010 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:58.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.063+0000 7fb3d50b4640 1 --2- 172.21.15.154:0/1832613689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3d015e130 0x7fb3d00aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:58.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.063+0000 7fb3d50b4640 1 --2- 172.21.15.154:0/1832613689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3d0158160 0x7fb3d00ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:58.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.063+0000 7fb3d50b4640 1 -- 172.21.15.154:0/1832613689 >> 172.21.15.154:0/1832613689 conn(0x7fb3d006a300 msgr2=0x7fb3d006fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:58.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.064+0000 7fb3d50b4640 1 -- 172.21.15.154:0/1832613689 shutdown_connections 2023-09-23T03:15:58.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.064+0000 7fb3d50b4640 1 -- 172.21.15.154:0/1832613689 wait complete. 2023-09-23T03:15:58.082 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:15:58.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.300+0000 7f7482678640 1 Processor -- start 2023-09-23T03:15:58.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.300+0000 7f7482678640 1 -- start start 2023-09-23T03:15:58.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.302+0000 7f7482678640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f747c15df50 0x7f747c0b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:58.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.302+0000 7f7482678640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f747c0d0980 0x7f747c0d1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:58.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.302+0000 7f7482678640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f747c0b3780 0x7f747c0d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:58.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.302+0000 7f7482678640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f747c0d1c00 con 0x7f747c0d0980 2023-09-23T03:15:58.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.302+0000 7f7482678640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f747c0cd870 con 0x7f747c15df50 2023-09-23T03:15:58.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.302+0000 7f7482678640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f747c15e330 con 0x7f747c0b3780 2023-09-23T03:15:58.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.302+0000 7f7480e75640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f747c0d0980 0x7f747c0d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:58.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.302+0000 7f7480e75640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f747c0d0980 0x7f747c0d1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44650/0 (socket says 172.21.15.154:44650) 2023-09-23T03:15:58.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.302+0000 7f7480e75640 1 -- 172.21.15.154:0/3112661068 learned_addr learned my addr 172.21.15.154:0/3112661068 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:58.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.303+0000 7f7481e77640 1 --2- 172.21.15.154:0/3112661068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f747c0b3780 0x7f747c0d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:58.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.303+0000 7f7481676640 1 --2- 172.21.15.154:0/3112661068 >> v2:172.21.15.190:3300/0 conn(0x7f747c15df50 0x7f747c0b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:58.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.303+0000 7f7480e75640 1 -- 172.21.15.154:0/3112661068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f747c0b3780 msgr2=0x7f747c0d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:58.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.303+0000 7f7480e75640 1 --2- 172.21.15.154:0/3112661068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f747c0b3780 0x7f747c0d16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:58.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.303+0000 7f7480e75640 1 -- 172.21.15.154:0/3112661068 >> v2:172.21.15.190:3300/0 conn(0x7f747c15df50 msgr2=0x7f747c0b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:58.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.303+0000 7f7480e75640 1 --2- 172.21.15.154:0/3112661068 >> v2:172.21.15.190:3300/0 conn(0x7f747c15df50 0x7f747c0b20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:58.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.303+0000 7f7480e75640 1 -- 172.21.15.154:0/3112661068 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f747c0d0e10 con 0x7f747c0d0980 2023-09-23T03:15:58.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.303+0000 7f7481e77640 1 --2- 172.21.15.154:0/3112661068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f747c0b3780 0x7f747c0d16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:58.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.304+0000 7f7480e75640 1 --2- 172.21.15.154:0/3112661068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f747c0d0980 0x7f747c0d1180 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f747c089ff0 tx=0x7f746c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45a3fef70ce1666e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:58.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.304+0000 7f7473fff640 1 -- 172.21.15.154:0/3112661068 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746c0317e0 con 0x7f747c0d0980 2023-09-23T03:15:58.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.304+0000 7f7473fff640 1 -- 172.21.15.154:0/3112661068 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f746c031940 con 0x7f747c0d0980 2023-09-23T03:15:58.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.304+0000 7f7473fff640 1 -- 172.21.15.154:0/3112661068 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746c031cd0 con 0x7f747c0d0980 2023-09-23T03:15:58.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.305+0000 7f7482678640 1 -- 172.21.15.154:0/3112661068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f747c0d0980 msgr2=0x7f747c0d1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:58.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.305+0000 7f7482678640 1 --2- 172.21.15.154:0/3112661068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f747c0d0980 0x7f747c0d1180 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f747c089ff0 tx=0x7f746c02b880 comp rx=0 tx=0).stop 2023-09-23T03:15:58.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.305+0000 7f7482678640 1 -- 172.21.15.154:0/3112661068 shutdown_connections 2023-09-23T03:15:58.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.305+0000 7f7482678640 1 --2- 172.21.15.154:0/3112661068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f747c0b3780 0x7f747c0d16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:58.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.305+0000 7f7482678640 1 --2- 172.21.15.154:0/3112661068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f747c0d0980 0x7f747c0d1180 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:58.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.305+0000 7f7482678640 1 --2- 172.21.15.154:0/3112661068 >> v2:172.21.15.190:3300/0 conn(0x7f747c15df50 0x7f747c0b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:58.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.305+0000 7f7482678640 1 -- 172.21.15.154:0/3112661068 >> 172.21.15.154:0/3112661068 conn(0x7f747c09bb20 msgr2=0x7f747c0d2870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:58.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.305+0000 7f7482678640 1 -- 172.21.15.154:0/3112661068 shutdown_connections 2023-09-23T03:15:58.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.305+0000 7f7482678640 1 -- 172.21.15.154:0/3112661068 wait complete. 2023-09-23T03:15:58.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.307+0000 7f7482678640 1 Processor -- start 2023-09-23T03:15:58.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.307+0000 7f7482678640 1 -- start start 2023-09-23T03:15:58.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.308+0000 7f7482678640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f747c0b3780 0x7f747c15c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:58.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.308+0000 7f7482678640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f747c15df50 0x7f747c1f2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:58.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.308+0000 7f7482678640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f747c0cc190 0x7f747c1f5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:58.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.308+0000 7f7482678640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f747c0cc9e0 con 0x7f747c0b3780 2023-09-23T03:15:58.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.308+0000 7f7482678640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f747c15bc40 con 0x7f747c0cc190 2023-09-23T03:15:58.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.308+0000 7f7482678640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f747c15b390 con 0x7f747c15df50 2023-09-23T03:15:58.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.308+0000 7f7481676640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f747c0b3780 0x7f747c15c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:58.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.308+0000 7f7481676640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f747c0b3780 0x7f747c15c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44664/0 (socket says 172.21.15.154:44664) 2023-09-23T03:15:58.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.308+0000 7f7481676640 1 -- 172.21.15.154:0/3990921604 learned_addr learned my addr 172.21.15.154:0/3990921604 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:58.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.308+0000 7f7481e77640 1 --2- 172.21.15.154:0/3990921604 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f747c0cc190 0x7f747c1f5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:58.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.308+0000 7f7480e75640 1 --2- 172.21.15.154:0/3990921604 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f747c15df50 0x7f747c1f2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:58.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.309+0000 7f7481676640 1 -- 172.21.15.154:0/3990921604 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f747c15df50 msgr2=0x7f747c1f2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:58.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.309+0000 7f7481676640 1 --2- 172.21.15.154:0/3990921604 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f747c15df50 0x7f747c1f2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:58.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.309+0000 7f7481676640 1 -- 172.21.15.154:0/3990921604 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f747c0cc190 msgr2=0x7f747c1f5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:58.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.309+0000 7f7481676640 1 --2- 172.21.15.154:0/3990921604 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f747c0cc190 0x7f747c1f5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:58.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.309+0000 7f7481676640 1 -- 172.21.15.154:0/3990921604 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f746c0094d0 con 0x7f747c0b3780 2023-09-23T03:15:58.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.309+0000 7f7481e77640 1 --2- 172.21.15.154:0/3990921604 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f747c0cc190 0x7f747c1f5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:58.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.309+0000 7f7480e75640 1 --2- 172.21.15.154:0/3990921604 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f747c15df50 0x7f747c1f2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:58.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.309+0000 7f7481676640 1 --2- 172.21.15.154:0/3990921604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f747c0b3780 0x7f747c15c060 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f747800c7f0 tx=0x7f747800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:58.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.309+0000 7f7471ffb640 1 -- 172.21.15.154:0/3990921604 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7478004220 con 0x7f747c0b3780 2023-09-23T03:15:58.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.309+0000 7f7471ffb640 1 -- 172.21.15.154:0/3990921604 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7478004380 con 0x7f747c0b3780 2023-09-23T03:15:58.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.309+0000 7f7471ffb640 1 -- 172.21.15.154:0/3990921604 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7478007840 con 0x7f747c0b3780 2023-09-23T03:15:58.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.309+0000 7f7482678640 1 -- 172.21.15.154:0/3990921604 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f747c0cc700 con 0x7f747c0b3780 2023-09-23T03:15:58.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.309+0000 7f7482678640 1 -- 172.21.15.154:0/3990921604 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f747c15b860 con 0x7f747c0b3780 2023-09-23T03:15:58.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.311+0000 7f7482678640 1 -- 172.21.15.154:0/3990921604 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f747c04e920 con 0x7f747c0b3780 2023-09-23T03:15:58.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.311+0000 7f7471ffb640 1 -- 172.21.15.154:0/3990921604 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f7478007bb0 con 0x7f747c0b3780 2023-09-23T03:15:58.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.312+0000 7f7471ffb640 1 --2- 172.21.15.154:0/3990921604 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f746006af60 0x7f746006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:58.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.312+0000 7f7471ffb640 1 -- 172.21.15.154:0/3990921604 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7478089880 con 0x7f747c0b3780 2023-09-23T03:15:58.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.313+0000 7f7480e75640 1 --2- 172.21.15.154:0/3990921604 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f746006af60 0x7f746006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:58.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.313+0000 7f7480e75640 1 --2- 172.21.15.154:0/3990921604 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f746006af60 0x7f746006d420 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f746c036000 tx=0x7f746c03d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:58.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.316+0000 7f7471ffb640 1 -- 172.21.15.154:0/3990921604 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f74780545d0 con 0x7f747c0b3780 2023-09-23T03:15:58.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.504+0000 7f7482678640 1 -- 172.21.15.154:0/3990921604 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f747c08c8d0 con 0x7f747c0b3780 2023-09-23T03:15:58.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.505+0000 7f7471ffb640 1 -- 172.21.15.154:0/3990921604 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f747801e020 con 0x7f747c0b3780 2023-09-23T03:15:58.506 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:15:58.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.507+0000 7f7482678640 1 -- 172.21.15.154:0/3990921604 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f746006af60 msgr2=0x7f746006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:58.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.507+0000 7f7482678640 1 --2- 172.21.15.154:0/3990921604 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f746006af60 0x7f746006d420 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f746c036000 tx=0x7f746c03d040 comp rx=0 tx=0).stop 2023-09-23T03:15:58.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.507+0000 7f7482678640 1 -- 172.21.15.154:0/3990921604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f747c0b3780 msgr2=0x7f747c15c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:58.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.507+0000 7f7482678640 1 --2- 172.21.15.154:0/3990921604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f747c0b3780 0x7f747c15c060 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f747800c7f0 tx=0x7f747800ccb0 comp rx=0 tx=0).stop 2023-09-23T03:15:58.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.508+0000 7f7482678640 1 -- 172.21.15.154:0/3990921604 shutdown_connections 2023-09-23T03:15:58.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.508+0000 7f7482678640 1 --2- 172.21.15.154:0/3990921604 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f746006af60 0x7f746006d420 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:58.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.508+0000 7f7482678640 1 --2- 172.21.15.154:0/3990921604 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f747c0cc190 0x7f747c1f5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:58.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.508+0000 7f7482678640 1 --2- 172.21.15.154:0/3990921604 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f747c15df50 0x7f747c1f2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:58.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.508+0000 7f7482678640 1 --2- 172.21.15.154:0/3990921604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f747c0b3780 0x7f747c15c060 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:58.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.508+0000 7f7482678640 1 -- 172.21.15.154:0/3990921604 >> 172.21.15.154:0/3990921604 conn(0x7f747c09bb20 msgr2=0x7f747c0d2870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:58.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.508+0000 7f7482678640 1 -- 172.21.15.154:0/3990921604 shutdown_connections 2023-09-23T03:15:58.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.508+0000 7f7482678640 1 -- 172.21.15.154:0/3990921604 wait complete. 2023-09-23T03:15:58.528 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:15:58.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.744+0000 7f82556ca640 1 Processor -- start 2023-09-23T03:15:58.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.745+0000 7f82556ca640 1 -- start start 2023-09-23T03:15:58.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.746+0000 7f82556ca640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f825015df50 0x7f82500b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:58.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.746+0000 7f82556ca640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f82500d0980 0x7f82500d1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:58.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.746+0000 7f82556ca640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f825015e330 con 0x7f82500b3780 2023-09-23T03:15:58.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.746+0000 7f82556ca640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82500cd870 con 0x7f82500d0980 2023-09-23T03:15:58.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.746+0000 7f82556ca640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82500d0e10 con 0x7f825015df50 2023-09-23T03:15:58.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.747+0000 7f824f7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f82500d0980 0x7f82500d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:58.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.747+0000 7f824f7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f82500d0980 0x7f82500d1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44678/0 (socket says 172.21.15.154:44678) 2023-09-23T03:15:58.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.747+0000 7f824f7fe640 1 -- 172.21.15.154:0/2193147126 learned_addr learned my addr 172.21.15.154:0/2193147126 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:58.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.747+0000 7f824f7fe640 1 -- 172.21.15.154:0/2193147126 >> v2:172.21.15.190:3300/0 conn(0x7f825015df50 msgr2=0x7f82500b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:58.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.747+0000 7f824f7fe640 1 --2- 172.21.15.154:0/2193147126 >> v2:172.21.15.190:3300/0 conn(0x7f825015df50 0x7f82500b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:58.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.747+0000 7f824f7fe640 1 -- 172.21.15.154:0/2193147126 >> v1:172.21.15.190:6789/0 conn(0x7f82500b3780 legacy=0x7f82500cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:58.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.747+0000 7f824f7fe640 1 -- 172.21.15.154:0/2193147126 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82500b4c50 con 0x7f82500d0980 2023-09-23T03:15:58.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.748+0000 7f824f7fe640 1 --2- 172.21.15.154:0/2193147126 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82500d0980 0x7f82500d1180 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8250089ff0 tx=0x7f823802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d0f965880727732 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:58.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.749+0000 7f824effd640 1 -- 172.21.15.154:0/2193147126 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8238031870 con 0x7f82500d0980 2023-09-23T03:15:58.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.749+0000 7f824effd640 1 -- 172.21.15.154:0/2193147126 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82380319d0 con 0x7f82500d0980 2023-09-23T03:15:58.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.749+0000 7f824effd640 1 -- 172.21.15.154:0/2193147126 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8238031d60 con 0x7f82500d0980 2023-09-23T03:15:58.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.749+0000 7f82556ca640 1 -- 172.21.15.154:0/2193147126 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82500d0980 msgr2=0x7f82500d1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:58.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.749+0000 7f82556ca640 1 --2- 172.21.15.154:0/2193147126 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82500d0980 0x7f82500d1180 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8250089ff0 tx=0x7f823802b910 comp rx=0 tx=0).stop 2023-09-23T03:15:58.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.750+0000 7f82556ca640 1 -- 172.21.15.154:0/2193147126 shutdown_connections 2023-09-23T03:15:58.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.750+0000 7f82556ca640 1 --2- 172.21.15.154:0/2193147126 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82500d0980 0x7f82500d1180 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:58.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.750+0000 7f82556ca640 1 --2- 172.21.15.154:0/2193147126 >> v2:172.21.15.190:3300/0 conn(0x7f825015df50 0x7f82500b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:58.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.750+0000 7f82556ca640 1 -- 172.21.15.154:0/2193147126 >> 172.21.15.154:0/2193147126 conn(0x7f825009bb20 msgr2=0x7f82500d2870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:58.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.750+0000 7f82556ca640 1 -- 172.21.15.154:0/2193147126 shutdown_connections 2023-09-23T03:15:58.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.750+0000 7f82556ca640 1 -- 172.21.15.154:0/2193147126 wait complete. 2023-09-23T03:15:58.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.751+0000 7f82556ca640 1 Processor -- start 2023-09-23T03:15:58.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.752+0000 7f82556ca640 1 -- start start 2023-09-23T03:15:58.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.752+0000 7f82556ca640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82500b3780 0x7f825014eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:58.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.752+0000 7f82556ca640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f825015df50 0x7f825006b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:58.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.752+0000 7f82556ca640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f825006b910 0x7f82500b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:58.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.753+0000 7f82556ca640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825006bdc0 con 0x7f825006b910 2023-09-23T03:15:58.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.753+0000 7f82556ca640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82500c6290 con 0x7f825015df50 2023-09-23T03:15:58.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.753+0000 7f82556ca640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82500c6570 con 0x7f82500b3780 2023-09-23T03:15:58.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.753+0000 7f8254ec9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f825006b910 0x7f82500b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:58.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.753+0000 7f8254ec9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f825006b910 0x7f82500b07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44692/0 (socket says 172.21.15.154:44692) 2023-09-23T03:15:58.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.753+0000 7f8254ec9640 1 -- 172.21.15.154:0/2590483568 learned_addr learned my addr 172.21.15.154:0/2590483568 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:58.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.753+0000 7f824f7fe640 1 --2- 172.21.15.154:0/2590483568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f825015df50 0x7f825006b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:58.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.753+0000 7f824ffff640 1 --2- 172.21.15.154:0/2590483568 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82500b3780 0x7f825014eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:58.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.753+0000 7f8254ec9640 1 -- 172.21.15.154:0/2590483568 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82500b3780 msgr2=0x7f825014eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:58.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.753+0000 7f8254ec9640 1 --2- 172.21.15.154:0/2590483568 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82500b3780 0x7f825014eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:58.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.753+0000 7f8254ec9640 1 -- 172.21.15.154:0/2590483568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f825015df50 msgr2=0x7f825006b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:58.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.753+0000 7f8254ec9640 1 --2- 172.21.15.154:0/2590483568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f825015df50 0x7f825006b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:58.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.753+0000 7f8254ec9640 1 -- 172.21.15.154:0/2590483568 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82380094d0 con 0x7f825006b910 2023-09-23T03:15:58.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.753+0000 7f824ffff640 1 --2- 172.21.15.154:0/2590483568 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82500b3780 0x7f825014eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:58.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.754+0000 7f8254ec9640 1 --2- 172.21.15.154:0/2590483568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f825006b910 0x7f82500b07f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f824800c410 tx=0x7f824800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:58.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.754+0000 7f824cff9640 1 -- 172.21.15.154:0/2590483568 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8248011790 con 0x7f825006b910 2023-09-23T03:15:58.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.754+0000 7f824cff9640 1 -- 172.21.15.154:0/2590483568 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82480118f0 con 0x7f825006b910 2023-09-23T03:15:58.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.754+0000 7f824cff9640 1 -- 172.21.15.154:0/2590483568 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8248011ce0 con 0x7f825006b910 2023-09-23T03:15:58.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.754+0000 7f82556ca640 1 -- 172.21.15.154:0/2590483568 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82500c6850 con 0x7f825006b910 2023-09-23T03:15:58.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.754+0000 7f82556ca640 1 -- 172.21.15.154:0/2590483568 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82500c6e10 con 0x7f825006b910 2023-09-23T03:15:58.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.755+0000 7f82556ca640 1 -- 172.21.15.154:0/2590483568 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82500c7030 con 0x7f825006b910 2023-09-23T03:15:58.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.756+0000 7f824cff9640 1 -- 172.21.15.154:0/2590483568 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f824801c460 con 0x7f825006b910 2023-09-23T03:15:58.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.757+0000 7f824cff9640 1 --2- 172.21.15.154:0/2590483568 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f823006af30 0x7f823006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:58.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.757+0000 7f824cff9640 1 -- 172.21.15.154:0/2590483568 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f82480897b0 con 0x7f825006b910 2023-09-23T03:15:58.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.757+0000 7f824ffff640 1 --2- 172.21.15.154:0/2590483568 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f823006af30 0x7f823006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:58.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.758+0000 7f824ffff640 1 --2- 172.21.15.154:0/2590483568 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f823006af30 0x7f823006d3f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8240007eb0 tx=0x7f82400073c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:58.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.763+0000 7f824cff9640 1 -- 172.21.15.154:0/2590483568 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f8248054470 con 0x7f825006b910 2023-09-23T03:15:58.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.954+0000 7f82556ca640 1 -- 172.21.15.154:0/2590483568 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f825004e920 con 0x7f825006b910 2023-09-23T03:15:58.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.955+0000 7f824cff9640 1 -- 172.21.15.154:0/2590483568 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8248057a90 con 0x7f825006b910 2023-09-23T03:15:58.957 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:15:58.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.958+0000 7f82556ca640 1 -- 172.21.15.154:0/2590483568 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f823006af30 msgr2=0x7f823006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:58.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.958+0000 7f82556ca640 1 --2- 172.21.15.154:0/2590483568 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f823006af30 0x7f823006d3f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8240007eb0 tx=0x7f82400073c0 comp rx=0 tx=0).stop 2023-09-23T03:15:58.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.958+0000 7f82556ca640 1 -- 172.21.15.154:0/2590483568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f825006b910 msgr2=0x7f82500b07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:58.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.958+0000 7f82556ca640 1 --2- 172.21.15.154:0/2590483568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f825006b910 0x7f82500b07f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f824800c410 tx=0x7f824800c8d0 comp rx=0 tx=0).stop 2023-09-23T03:15:58.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.958+0000 7f82556ca640 1 -- 172.21.15.154:0/2590483568 shutdown_connections 2023-09-23T03:15:58.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.958+0000 7f82556ca640 1 --2- 172.21.15.154:0/2590483568 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f823006af30 0x7f823006d3f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:58.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.958+0000 7f82556ca640 1 --2- 172.21.15.154:0/2590483568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f825006b910 0x7f82500b07f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:58.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.958+0000 7f82556ca640 1 --2- 172.21.15.154:0/2590483568 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f825015df50 0x7f825006b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:58.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.958+0000 7f82556ca640 1 --2- 172.21.15.154:0/2590483568 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82500b3780 0x7f825014eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:58.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.958+0000 7f82556ca640 1 -- 172.21.15.154:0/2590483568 >> 172.21.15.154:0/2590483568 conn(0x7f825009bb20 msgr2=0x7f825006c580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:58.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.958+0000 7f82556ca640 1 -- 172.21.15.154:0/2590483568 shutdown_connections 2023-09-23T03:15:58.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:58.958+0000 7f82556ca640 1 -- 172.21.15.154:0/2590483568 wait complete. 2023-09-23T03:15:58.978 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-23T03:15:59.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.196+0000 7f1e76444640 1 Processor -- start 2023-09-23T03:15:59.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.197+0000 7f1e76444640 1 -- start start 2023-09-23T03:15:59.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.198+0000 7f1e76444640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1e700c8d70 0x7f1e7015c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:59.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.198+0000 7f1e76444640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e700c9d20 con 0x7f1e700c9150 2023-09-23T03:15:59.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.198+0000 7f1e76444640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e70071850 con 0x7f1e70071290 2023-09-23T03:15:59.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.198+0000 7f1e76444640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e70071b30 con 0x7f1e700c8d70 2023-09-23T03:15:59.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.199+0000 7f1e67fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f1e700c9150 0x7f1e7015cc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46296/0 (socket says 172.21.15.154:46296) 2023-09-23T03:15:59.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.199+0000 7f1e67fff640 1 -- 172.21.15.154:0/3843114603 learned_addr learned my addr 172.21.15.154:0/3843114603 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:59.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.199+0000 7f1e6ffff640 1 --2- 172.21.15.154:0/3843114603 >> v2:172.21.15.190:3300/0 conn(0x7f1e700c8d70 0x7f1e7015c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:59.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.199+0000 7f1e6f7fe640 1 -- 172.21.15.154:0/3843114603 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3335032799 0 0) 0x7f1e700c9d20 con 0x7f1e700c9150 2023-09-23T03:15:59.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.199+0000 7f1e6f7fe640 1 -- 172.21.15.154:0/3843114603 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e5c003520 con 0x7f1e700c9150 2023-09-23T03:15:59.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.200+0000 7f1e6f7fe640 1 -- 172.21.15.154:0/3843114603 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2831981903 0 0) 0x7f1e70071850 con 0x7f1e70071290 2023-09-23T03:15:59.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.200+0000 7f1e6f7fe640 1 -- 172.21.15.154:0/3843114603 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e700c9d20 con 0x7f1e70071290 2023-09-23T03:15:59.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.200+0000 7f1e6f7fe640 1 -- 172.21.15.154:0/3843114603 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3127593630 0 0) 0x7f1e5c003520 con 0x7f1e700c9150 2023-09-23T03:15:59.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.200+0000 7f1e6f7fe640 1 -- 172.21.15.154:0/3843114603 >> v2:172.21.15.190:3300/0 conn(0x7f1e700c8d70 msgr2=0x7f1e7015c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:59.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.200+0000 7f1e6f7fe640 1 --2- 172.21.15.154:0/3843114603 >> v2:172.21.15.190:3300/0 conn(0x7f1e700c8d70 0x7f1e7015c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:59.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.200+0000 7f1e6f7fe640 1 -- 172.21.15.154:0/3843114603 >> v1:172.21.15.190:6789/0 conn(0x7f1e70071290 legacy=0x7f1e700ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:59.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.200+0000 7f1e6f7fe640 1 -- 172.21.15.154:0/3843114603 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e70071e10 con 0x7f1e700c9150 2023-09-23T03:15:59.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.200+0000 7f1e6f7fe640 1 -- 172.21.15.154:0/3843114603 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f1e580015f0 con 0x7f1e700c9150 2023-09-23T03:15:59.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.200+0000 7f1e6f7fe640 1 -- 172.21.15.154:0/3843114603 >> v1:172.21.15.154:6789/0 conn(0x7f1e700c9150 legacy=0x7f1e7015cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:59.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.200+0000 7f1e6f7fe640 1 --2- 172.21.15.154:0/3843114603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e5c003b50 0x7f1e5c003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:59.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.200+0000 7f1e6f7fe640 1 --2- 172.21.15.154:0/3843114603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e5c0047c0 0x7f1e5c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:59.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.200+0000 7f1e6f7fe640 1 --2- 172.21.15.154:0/3843114603 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e5c005180 0x7f1e5c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:59.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.200+0000 7f1e6f7fe640 1 -- 172.21.15.154:0/3843114603 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e70071b30 con 0x7f1e5c005180 2023-09-23T03:15:59.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.200+0000 7f1e6f7fe640 1 -- 172.21.15.154:0/3843114603 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c00c050 con 0x7f1e5c0047c0 2023-09-23T03:15:59.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.200+0000 7f1e6f7fe640 1 -- 172.21.15.154:0/3843114603 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c00c210 con 0x7f1e5c003b50 2023-09-23T03:15:59.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.200+0000 7f1e749e1640 1 --2- 172.21.15.154:0/3843114603 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e5c005180 0x7f1e5c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:59.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.200+0000 7f1e6ffff640 1 --2- 172.21.15.154:0/3843114603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e5c003b50 0x7f1e5c003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:59.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.200+0000 7f1e67fff640 1 --2- 172.21.15.154:0/3843114603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e5c0047c0 0x7f1e5c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:59.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.201+0000 7f1e749e1640 1 -- 172.21.15.154:0/3843114603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e5c003b50 msgr2=0x7f1e5c003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:59.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.201+0000 7f1e749e1640 1 --2- 172.21.15.154:0/3843114603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e5c003b50 0x7f1e5c003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:59.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.201+0000 7f1e749e1640 1 -- 172.21.15.154:0/3843114603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e5c0047c0 msgr2=0x7f1e5c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:59.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.201+0000 7f1e749e1640 1 --2- 172.21.15.154:0/3843114603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e5c0047c0 0x7f1e5c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:59.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.201+0000 7f1e749e1640 1 -- 172.21.15.154:0/3843114603 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e5c003520 con 0x7f1e5c005180 2023-09-23T03:15:59.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.201+0000 7f1e6ffff640 1 --2- 172.21.15.154:0/3843114603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e5c003b50 0x7f1e5c003f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:15:59.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.201+0000 7f1e67fff640 1 --2- 172.21.15.154:0/3843114603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e5c0047c0 0x7f1e5c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:15:59.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.202+0000 7f1e749e1640 1 -- 172.21.15.154:0/3843114603 reap_dead start 2023-09-23T03:15:59.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.202+0000 7f1e749e1640 1 --2- 172.21.15.154:0/3843114603 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e5c005180 0x7f1e5c00b980 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1e68001ff0 tx=0x7f1e5c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c8af509bb910cf3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:59.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.202+0000 7f1e6f7fe640 1 -- 172.21.15.154:0/3843114603 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e68003b70 con 0x7f1e5c005180 2023-09-23T03:15:59.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.202+0000 7f1e6f7fe640 1 -- 172.21.15.154:0/3843114603 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e68003cd0 con 0x7f1e5c005180 2023-09-23T03:15:59.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.202+0000 7f1e6f7fe640 1 -- 172.21.15.154:0/3843114603 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e68005310 con 0x7f1e5c005180 2023-09-23T03:15:59.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.203+0000 7f1e76444640 1 -- 172.21.15.154:0/3843114603 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e5c005180 msgr2=0x7f1e5c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:59.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.203+0000 7f1e76444640 1 --2- 172.21.15.154:0/3843114603 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e5c005180 0x7f1e5c00b980 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1e68001ff0 tx=0x7f1e5c0057f0 comp rx=0 tx=0).stop 2023-09-23T03:15:59.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.203+0000 7f1e76444640 1 -- 172.21.15.154:0/3843114603 shutdown_connections 2023-09-23T03:15:59.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.203+0000 7f1e76444640 1 --2- 172.21.15.154:0/3843114603 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e5c005180 0x7f1e5c00b980 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:59.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.203+0000 7f1e76444640 1 -- 172.21.15.154:0/3843114603 >> 172.21.15.154:0/3843114603 conn(0x7f1e7006c010 msgr2=0x7f1e7006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:59.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.203+0000 7f1e76444640 1 -- 172.21.15.154:0/3843114603 shutdown_connections 2023-09-23T03:15:59.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.203+0000 7f1e76444640 1 -- 172.21.15.154:0/3843114603 wait complete. 2023-09-23T03:15:59.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.205+0000 7f1e76444640 1 Processor -- start 2023-09-23T03:15:59.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.205+0000 7f1e76444640 1 -- start start 2023-09-23T03:15:59.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.206+0000 7f1e76444640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e5c005180 0x7f1e700c3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:59.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.206+0000 7f1e76444640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e70152f50 0x7f1e700af270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:59.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.206+0000 7f1e76444640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e700c7750 0x7f1e700c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:59.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.206+0000 7f1e76444640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e700c41e0 con 0x7f1e70152f50 2023-09-23T03:15:59.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.206+0000 7f1e76444640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e700c8110 con 0x7f1e700c7750 2023-09-23T03:15:59.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.206+0000 7f1e76444640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e700af7b0 con 0x7f1e5c005180 2023-09-23T03:15:59.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.206+0000 7f1e67fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e70152f50 0x7f1e700af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:59.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.206+0000 7f1e67fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e70152f50 0x7f1e700af270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44710/0 (socket says 172.21.15.154:44710) 2023-09-23T03:15:59.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.206+0000 7f1e67fff640 1 -- 172.21.15.154:0/3742134269 learned_addr learned my addr 172.21.15.154:0/3742134269 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:59.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.207+0000 7f1e749e1640 1 --2- 172.21.15.154:0/3742134269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e700c7750 0x7f1e700c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:59.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.207+0000 7f1e6ffff640 1 --2- 172.21.15.154:0/3742134269 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e5c005180 0x7f1e700c3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:59.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.207+0000 7f1e67fff640 1 -- 172.21.15.154:0/3742134269 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e5c005180 msgr2=0x7f1e700c3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:59.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.207+0000 7f1e67fff640 1 --2- 172.21.15.154:0/3742134269 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e5c005180 0x7f1e700c3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:59.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.207+0000 7f1e67fff640 1 -- 172.21.15.154:0/3742134269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e700c7750 msgr2=0x7f1e700c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:59.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.207+0000 7f1e67fff640 1 --2- 172.21.15.154:0/3742134269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e700c7750 0x7f1e700c7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:59.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.207+0000 7f1e67fff640 1 -- 172.21.15.154:0/3742134269 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e700c43d0 con 0x7f1e70152f50 2023-09-23T03:15:59.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.207+0000 7f1e749e1640 1 --2- 172.21.15.154:0/3742134269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e700c7750 0x7f1e700c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:15:59.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.207+0000 7f1e67fff640 1 --2- 172.21.15.154:0/3742134269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e70152f50 0x7f1e700af270 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1e58009380 tx=0x7f1e58009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:59.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.207+0000 7f1e6d7fa640 1 -- 172.21.15.154:0/3742134269 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5800e8c0 con 0x7f1e70152f50 2023-09-23T03:15:59.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.207+0000 7f1e6d7fa640 1 -- 172.21.15.154:0/3742134269 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e5800ea20 con 0x7f1e70152f50 2023-09-23T03:15:59.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.207+0000 7f1e6d7fa640 1 -- 172.21.15.154:0/3742134269 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e5800ed90 con 0x7f1e70152f50 2023-09-23T03:15:59.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.208+0000 7f1e76444640 1 -- 172.21.15.154:0/3742134269 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e700c46b0 con 0x7f1e70152f50 2023-09-23T03:15:59.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.208+0000 7f1e76444640 1 -- 172.21.15.154:0/3742134269 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e700c4c70 con 0x7f1e70152f50 2023-09-23T03:15:59.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.209+0000 7f1e76444640 1 -- 172.21.15.154:0/3742134269 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e7004e920 con 0x7f1e70152f50 2023-09-23T03:15:59.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.210+0000 7f1e6d7fa640 1 -- 172.21.15.154:0/3742134269 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f1e58014400 con 0x7f1e70152f50 2023-09-23T03:15:59.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.211+0000 7f1e6d7fa640 1 --2- 172.21.15.154:0/3742134269 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1e5006af30 0x7f1e5006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:59.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.211+0000 7f1e6d7fa640 1 -- 172.21.15.154:0/3742134269 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1e58089070 con 0x7f1e70152f50 2023-09-23T03:15:59.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.211+0000 7f1e6ffff640 1 --2- 172.21.15.154:0/3742134269 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1e5006af30 0x7f1e5006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:59.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.211+0000 7f1e6ffff640 1 --2- 172.21.15.154:0/3742134269 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1e5006af30 0x7f1e5006d3f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1e600098b0 tx=0x7f1e60008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:59.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.215+0000 7f1e6d7fa640 1 -- 172.21.15.154:0/3742134269 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f1e58053f00 con 0x7f1e70152f50 2023-09-23T03:15:59.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.410+0000 7f1e76444640 1 -- 172.21.15.154:0/3742134269 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1e7015b790 con 0x7f1e70152f50 2023-09-23T03:15:59.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.411+0000 7f1e6d7fa640 1 -- 172.21.15.154:0/3742134269 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1e58019020 con 0x7f1e70152f50 2023-09-23T03:15:59.412 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:15:59.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.413+0000 7f1e76444640 1 -- 172.21.15.154:0/3742134269 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1e5006af30 msgr2=0x7f1e5006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:59.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.413+0000 7f1e76444640 1 --2- 172.21.15.154:0/3742134269 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1e5006af30 0x7f1e5006d3f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1e600098b0 tx=0x7f1e60008040 comp rx=0 tx=0).stop 2023-09-23T03:15:59.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.413+0000 7f1e76444640 1 -- 172.21.15.154:0/3742134269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e70152f50 msgr2=0x7f1e700af270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:59.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.413+0000 7f1e76444640 1 --2- 172.21.15.154:0/3742134269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e70152f50 0x7f1e700af270 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1e58009380 tx=0x7f1e58009840 comp rx=0 tx=0).stop 2023-09-23T03:15:59.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.413+0000 7f1e76444640 1 -- 172.21.15.154:0/3742134269 shutdown_connections 2023-09-23T03:15:59.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.413+0000 7f1e76444640 1 --2- 172.21.15.154:0/3742134269 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1e5006af30 0x7f1e5006d3f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:59.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.413+0000 7f1e76444640 1 --2- 172.21.15.154:0/3742134269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1e700c7750 0x7f1e700c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:59.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.413+0000 7f1e76444640 1 --2- 172.21.15.154:0/3742134269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e70152f50 0x7f1e700af270 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:59.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.413+0000 7f1e76444640 1 --2- 172.21.15.154:0/3742134269 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1e5c005180 0x7f1e700c3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:59.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.413+0000 7f1e76444640 1 -- 172.21.15.154:0/3742134269 >> 172.21.15.154:0/3742134269 conn(0x7f1e7006c010 msgr2=0x7f1e7015be30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:59.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.413+0000 7f1e76444640 1 -- 172.21.15.154:0/3742134269 shutdown_connections 2023-09-23T03:15:59.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.414+0000 7f1e76444640 1 -- 172.21.15.154:0/3742134269 wait complete. 2023-09-23T03:15:59.432 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:15:59.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.645+0000 7f34df9c9640 1 Processor -- start 2023-09-23T03:15:59.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.645+0000 7f34df9c9640 1 -- start start 2023-09-23T03:15:59.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.647+0000 7f34df9c9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34d815caf0 0x7f34d80ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:59.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.647+0000 7f34df9c9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f34d80cee70 0x7f34d8160880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:59.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.647+0000 7f34df9c9640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34d80cd530 con 0x7f34d8160dc0 2023-09-23T03:15:59.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.647+0000 7f34df9c9640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34d815ced0 con 0x7f34d80cee70 2023-09-23T03:15:59.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.647+0000 7f34df9c9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34d80cf2f0 con 0x7f34d815caf0 2023-09-23T03:15:59.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.647+0000 7f34dcf64640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f34d80cee70 0x7f34d8160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:59.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.647+0000 7f34ddf66640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f34d8160dc0 0x7f34d806cff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46300/0 (socket says 172.21.15.154:46300) 2023-09-23T03:15:59.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.647+0000 7f34ddf66640 1 -- 172.21.15.154:0/2461899919 learned_addr learned my addr 172.21.15.154:0/2461899919 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:15:59.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.647+0000 7f34dd765640 1 --2- 172.21.15.154:0/2461899919 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34d815caf0 0x7f34d80ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:59.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.648+0000 7f34dcf64640 1 -- 172.21.15.154:0/2461899919 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34d815caf0 msgr2=0x7f34d80ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:15:59.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.648+0000 7f34dcf64640 1 --2- 172.21.15.154:0/2461899919 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34d815caf0 0x7f34d80ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:59.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.648+0000 7f34dcf64640 1 -- 172.21.15.154:0/2461899919 >> v1:172.21.15.154:6789/0 conn(0x7f34d8160dc0 legacy=0x7f34d806cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:59.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.648+0000 7f34dcf64640 1 -- 172.21.15.154:0/2461899919 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d80cd810 con 0x7f34d80cee70 2023-09-23T03:15:59.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.648+0000 7f34cffff640 1 -- 172.21.15.154:0/2461899919 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2345471755 0 0) 0x7f34d80cd530 con 0x7f34d8160dc0 2023-09-23T03:15:59.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.649+0000 7f34dcf64640 1 --2- 172.21.15.154:0/2461899919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34d80cee70 0x7f34d8160880 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f34c80097a0 tx=0x7f34c802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817c002a61efda80 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:59.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.650+0000 7f34cffff640 1 -- 172.21.15.154:0/2461899919 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c802d040 con 0x7f34d80cee70 2023-09-23T03:15:59.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.650+0000 7f34cffff640 1 -- 172.21.15.154:0/2461899919 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34c8031810 con 0x7f34d80cee70 2023-09-23T03:15:59.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.650+0000 7f34cffff640 1 -- 172.21.15.154:0/2461899919 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c8031ba0 con 0x7f34d80cee70 2023-09-23T03:15:59.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.650+0000 7f34df9c9640 1 -- 172.21.15.154:0/2461899919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34d80cee70 msgr2=0x7f34d8160880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:59.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.650+0000 7f34df9c9640 1 --2- 172.21.15.154:0/2461899919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34d80cee70 0x7f34d8160880 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f34c80097a0 tx=0x7f34c802b8b0 comp rx=0 tx=0).stop 2023-09-23T03:15:59.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.650+0000 7f34df9c9640 1 -- 172.21.15.154:0/2461899919 shutdown_connections 2023-09-23T03:15:59.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.650+0000 7f34df9c9640 1 --2- 172.21.15.154:0/2461899919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34d80cee70 0x7f34d8160880 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:59.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.650+0000 7f34df9c9640 1 --2- 172.21.15.154:0/2461899919 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34d815caf0 0x7f34d80ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:59.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.650+0000 7f34df9c9640 1 -- 172.21.15.154:0/2461899919 >> 172.21.15.154:0/2461899919 conn(0x7f34d806a0f0 msgr2=0x7f34d806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:59.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.651+0000 7f34df9c9640 1 -- 172.21.15.154:0/2461899919 shutdown_connections 2023-09-23T03:15:59.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.651+0000 7f34df9c9640 1 -- 172.21.15.154:0/2461899919 wait complete. 2023-09-23T03:15:59.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.653+0000 7f34df9c9640 1 Processor -- start 2023-09-23T03:15:59.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.653+0000 7f34df9c9640 1 -- start start 2023-09-23T03:15:59.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.654+0000 7f34df9c9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34d80cee70 0x7f34d8071fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:59.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.654+0000 7f34df9c9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34d815caf0 0x7f34d80c2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:59.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.654+0000 7f34df9c9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34d8160dc0 0x7f34d80708f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:59.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.654+0000 7f34df9c9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d80710d0 con 0x7f34d8160dc0 2023-09-23T03:15:59.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.654+0000 7f34df9c9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d815da70 con 0x7f34d815caf0 2023-09-23T03:15:59.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.654+0000 7f34df9c9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34d8070e30 con 0x7f34d80cee70 2023-09-23T03:15:59.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.655+0000 7f34ddf66640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34d8160dc0 0x7f34d80708f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:59.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.655+0000 7f34ddf66640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34d8160dc0 0x7f34d80708f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44728/0 (socket says 172.21.15.154:44728) 2023-09-23T03:15:59.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.655+0000 7f34ddf66640 1 -- 172.21.15.154:0/3220805052 learned_addr learned my addr 172.21.15.154:0/3220805052 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:15:59.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.655+0000 7f34dd765640 1 --2- 172.21.15.154:0/3220805052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34d80cee70 0x7f34d8071fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:59.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.655+0000 7f34dcf64640 1 --2- 172.21.15.154:0/3220805052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34d815caf0 0x7f34d80c2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:59.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.655+0000 7f34ddf66640 1 -- 172.21.15.154:0/3220805052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34d80cee70 msgr2=0x7f34d8071fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:59.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.655+0000 7f34ddf66640 1 --2- 172.21.15.154:0/3220805052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34d80cee70 0x7f34d8071fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:59.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.655+0000 7f34ddf66640 1 -- 172.21.15.154:0/3220805052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34d815caf0 msgr2=0x7f34d80c2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:59.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.655+0000 7f34ddf66640 1 --2- 172.21.15.154:0/3220805052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34d815caf0 0x7f34d80c2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:59.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.655+0000 7f34ddf66640 1 -- 172.21.15.154:0/3220805052 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c8009400 con 0x7f34d8160dc0 2023-09-23T03:15:59.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.655+0000 7f34ddf66640 1 --2- 172.21.15.154:0/3220805052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34d8160dc0 0x7f34d80708f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f34d000c7b0 tx=0x7f34d000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:59.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.656+0000 7f34cdffb640 1 -- 172.21.15.154:0/3220805052 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d000a640 con 0x7f34d8160dc0 2023-09-23T03:15:59.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.656+0000 7f34cdffb640 1 -- 172.21.15.154:0/3220805052 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d000a7c0 con 0x7f34d8160dc0 2023-09-23T03:15:59.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.656+0000 7f34cdffb640 1 -- 172.21.15.154:0/3220805052 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d000ab70 con 0x7f34d8160dc0 2023-09-23T03:15:59.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.656+0000 7f34df9c9640 1 -- 172.21.15.154:0/3220805052 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34d815d140 con 0x7f34d8160dc0 2023-09-23T03:15:59.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.656+0000 7f34df9c9640 1 -- 172.21.15.154:0/3220805052 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34d815d700 con 0x7f34d8160dc0 2023-09-23T03:15:59.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.657+0000 7f34df9c9640 1 -- 172.21.15.154:0/3220805052 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34d8073eb0 con 0x7f34d8160dc0 2023-09-23T03:15:59.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.658+0000 7f34cdffb640 1 -- 172.21.15.154:0/3220805052 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f34d000acd0 con 0x7f34d8160dc0 2023-09-23T03:15:59.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.659+0000 7f34cdffb640 1 --2- 172.21.15.154:0/3220805052 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f34bc06af30 0x7f34bc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:15:59.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.659+0000 7f34cdffb640 1 -- 172.21.15.154:0/3220805052 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f34d0088f10 con 0x7f34d8160dc0 2023-09-23T03:15:59.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.659+0000 7f34dd765640 1 --2- 172.21.15.154:0/3220805052 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f34bc06af30 0x7f34bc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:15:59.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.660+0000 7f34dd765640 1 --2- 172.21.15.154:0/3220805052 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f34bc06af30 0x7f34bc06d3f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f34d4009ac0 tx=0x7f34d4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:15:59.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.663+0000 7f34cdffb640 1 -- 172.21.15.154:0/3220805052 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f34d0053bc0 con 0x7f34d8160dc0 2023-09-23T03:15:59.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.887+0000 7f34df9c9640 1 -- 172.21.15.154:0/3220805052 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34d815dbb0 con 0x7f34d8160dc0 2023-09-23T03:15:59.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.888+0000 7f34cdffb640 1 -- 172.21.15.154:0/3220805052 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4921 (secure 0 0 0) 0x7f34d0018020 con 0x7f34d8160dc0 2023-09-23T03:15:59.890 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:15:59.890 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2644365591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2644365591},{"type":"v1","addr":"172.21.15.190:6835","nonce":2644365591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/4145467852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4145467852},{"type":"v1","addr":"172.21.15.154:6835","nonce":4145467852}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1022302774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1022302774},{"type":"v1","addr":"172.21.15.154:6837","nonce":1022302774}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:15:20.686035+0000","modified":"2023-09-23T03:15:48.742318+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/3561509423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3561509423},{"type":"v1","addr":"172.21.15.190:6834","nonce":3561509423}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T03:15:59.890 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 9 2023-09-23T03:15:59.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.891+0000 7f34df9c9640 1 -- 172.21.15.154:0/3220805052 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f34bc06af30 msgr2=0x7f34bc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:59.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.891+0000 7f34df9c9640 1 --2- 172.21.15.154:0/3220805052 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f34bc06af30 0x7f34bc06d3f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f34d4009ac0 tx=0x7f34d4008040 comp rx=0 tx=0).stop 2023-09-23T03:15:59.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.891+0000 7f34df9c9640 1 -- 172.21.15.154:0/3220805052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34d8160dc0 msgr2=0x7f34d80708f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:15:59.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.891+0000 7f34df9c9640 1 --2- 172.21.15.154:0/3220805052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34d8160dc0 0x7f34d80708f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f34d000c7b0 tx=0x7f34d000cc70 comp rx=0 tx=0).stop 2023-09-23T03:15:59.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.891+0000 7f34df9c9640 1 -- 172.21.15.154:0/3220805052 shutdown_connections 2023-09-23T03:15:59.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.891+0000 7f34df9c9640 1 --2- 172.21.15.154:0/3220805052 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f34bc06af30 0x7f34bc06d3f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:59.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.891+0000 7f34df9c9640 1 --2- 172.21.15.154:0/3220805052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34d8160dc0 0x7f34d80708f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:59.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.891+0000 7f34df9c9640 1 --2- 172.21.15.154:0/3220805052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f34d815caf0 0x7f34d80c2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:59.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.891+0000 7f34df9c9640 1 --2- 172.21.15.154:0/3220805052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f34d80cee70 0x7f34d8071fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:15:59.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.891+0000 7f34df9c9640 1 -- 172.21.15.154:0/3220805052 >> 172.21.15.154:0/3220805052 conn(0x7f34d806a0f0 msgr2=0x7f34d80b2180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:15:59.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.892+0000 7f34df9c9640 1 -- 172.21.15.154:0/3220805052 shutdown_connections 2023-09-23T03:15:59.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:15:59.892+0000 7f34df9c9640 1 -- 172.21.15.154:0/3220805052 wait complete. 2023-09-23T03:15:59.911 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:16:00.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.123+0000 7fa0fd53c640 1 Processor -- start 2023-09-23T03:16:00.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.123+0000 7fa0fd53c640 1 -- start start 2023-09-23T03:16:00.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.124+0000 7fa0fd53c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0f815cab0 0x7fa0f80cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:00.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.124+0000 7fa0fd53c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa0f80d12e0 0x7fa0f80cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:00.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.124+0000 7fa0fd53c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0f80d18e0 con 0x7fa0f815ce90 2023-09-23T03:16:00.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.124+0000 7fa0fd53c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0f80cfc90 con 0x7fa0f80d12e0 2023-09-23T03:16:00.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.124+0000 7fa0fd53c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f80d1bc0 con 0x7fa0f815cab0 2023-09-23T03:16:00.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.125+0000 7fa0f77fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa0f80d12e0 0x7fa0f80cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:00.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.125+0000 7fa0f67fc640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fa0f815ce90 0x7fa0f80d0bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46310/0 (socket says 172.21.15.154:46310) 2023-09-23T03:16:00.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.125+0000 7fa0f67fc640 1 -- 172.21.15.154:0/2329911895 learned_addr learned my addr 172.21.15.154:0/2329911895 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:00.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.125+0000 7fa0f6ffd640 1 --2- 172.21.15.154:0/2329911895 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0f815cab0 0x7fa0f80cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:00.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.125+0000 7fa0f77fe640 1 -- 172.21.15.154:0/2329911895 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0f815cab0 msgr2=0x7fa0f80cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:00.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.125+0000 7fa0f77fe640 1 --2- 172.21.15.154:0/2329911895 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0f815cab0 0x7fa0f80cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:00.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.125+0000 7fa0f77fe640 1 -- 172.21.15.154:0/2329911895 >> v1:172.21.15.154:6789/0 conn(0x7fa0f815ce90 legacy=0x7fa0f80d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:00.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.125+0000 7fa0f77fe640 1 -- 172.21.15.154:0/2329911895 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f80d1ea0 con 0x7fa0f80d12e0 2023-09-23T03:16:00.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.125+0000 7fa0f6ffd640 1 --2- 172.21.15.154:0/2329911895 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0f815cab0 0x7fa0f80cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:16:00.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.126+0000 7fa0f77fe640 1 --2- 172.21.15.154:0/2329911895 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0f80d12e0 0x7fa0f80cf750 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa0f00097a0 tx=0x7fa0f002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e431558335087d12 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:00.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.126+0000 7fa0f5ffb640 1 -- 172.21.15.154:0/2329911895 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f002d040 con 0x7fa0f80d12e0 2023-09-23T03:16:00.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.127+0000 7fa0f5ffb640 1 -- 172.21.15.154:0/2329911895 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f0031810 con 0x7fa0f80d12e0 2023-09-23T03:16:00.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.127+0000 7fa0f5ffb640 1 -- 172.21.15.154:0/2329911895 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0031ba0 con 0x7fa0f80d12e0 2023-09-23T03:16:00.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.127+0000 7fa0fd53c640 1 -- 172.21.15.154:0/2329911895 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0f80d12e0 msgr2=0x7fa0f80cf750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:00.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.127+0000 7fa0fd53c640 1 --2- 172.21.15.154:0/2329911895 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0f80d12e0 0x7fa0f80cf750 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa0f00097a0 tx=0x7fa0f002b8b0 comp rx=0 tx=0).stop 2023-09-23T03:16:00.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.127+0000 7fa0fd53c640 1 -- 172.21.15.154:0/2329911895 shutdown_connections 2023-09-23T03:16:00.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.127+0000 7fa0fd53c640 1 --2- 172.21.15.154:0/2329911895 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0f80d12e0 0x7fa0f80cf750 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:00.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.127+0000 7fa0fd53c640 1 --2- 172.21.15.154:0/2329911895 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0f815cab0 0x7fa0f80cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:00.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.127+0000 7fa0fd53c640 1 -- 172.21.15.154:0/2329911895 >> 172.21.15.154:0/2329911895 conn(0x7fa0f8071bc0 msgr2=0x7fa0f80b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:00.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.127+0000 7fa0fd53c640 1 -- 172.21.15.154:0/2329911895 shutdown_connections 2023-09-23T03:16:00.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.127+0000 7fa0fd53c640 1 -- 172.21.15.154:0/2329911895 wait complete. 2023-09-23T03:16:00.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.129+0000 7fa0fd53c640 1 Processor -- start 2023-09-23T03:16:00.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.129+0000 7fa0fd53c640 1 -- start start 2023-09-23T03:16:00.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.130+0000 7fa0fd53c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0f80d12e0 0x7fa0f80c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:00.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.130+0000 7fa0fd53c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0f815cab0 0x7fa0f80b2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:00.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.130+0000 7fa0fd53c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0f815ce90 0x7fa0f814bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:00.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.130+0000 7fa0fd53c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f80c73a0 con 0x7fa0f80d12e0 2023-09-23T03:16:00.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.130+0000 7fa0fd53c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f81f3010 con 0x7fa0f815ce90 2023-09-23T03:16:00.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.130+0000 7fa0fd53c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f80b71e0 con 0x7fa0f815cab0 2023-09-23T03:16:00.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.131+0000 7fa0f6ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0f80d12e0 0x7fa0f80c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:00.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.131+0000 7fa0f6ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0f80d12e0 0x7fa0f80c64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44748/0 (socket says 172.21.15.154:44748) 2023-09-23T03:16:00.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.131+0000 7fa0f6ffd640 1 -- 172.21.15.154:0/3904722641 learned_addr learned my addr 172.21.15.154:0/3904722641 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:00.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.131+0000 7fa0f6ffd640 1 -- 172.21.15.154:0/3904722641 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0f815cab0 msgr2=0x7fa0f80b2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:00.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.131+0000 7fa0f6ffd640 1 --2- 172.21.15.154:0/3904722641 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0f815cab0 0x7fa0f80b2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:00.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.131+0000 7fa0f6ffd640 1 -- 172.21.15.154:0/3904722641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0f815ce90 msgr2=0x7fa0f814bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:00.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.131+0000 7fa0f6ffd640 1 --2- 172.21.15.154:0/3904722641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0f815ce90 0x7fa0f814bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:00.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.131+0000 7fa0f6ffd640 1 -- 172.21.15.154:0/3904722641 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f0009400 con 0x7fa0f80d12e0 2023-09-23T03:16:00.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.131+0000 7fa0f6ffd640 1 --2- 172.21.15.154:0/3904722641 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0f80d12e0 0x7fa0f80c64e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8009fc0 tx=0x7fa0e800d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:00.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.131+0000 7fa0ef7fe640 1 -- 172.21.15.154:0/3904722641 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8003b90 con 0x7fa0f80d12e0 2023-09-23T03:16:00.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.132+0000 7fa0ef7fe640 1 -- 172.21.15.154:0/3904722641 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e800f040 con 0x7fa0f80d12e0 2023-09-23T03:16:00.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.132+0000 7fa0ef7fe640 1 -- 172.21.15.154:0/3904722641 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8004030 con 0x7fa0f80d12e0 2023-09-23T03:16:00.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.132+0000 7fa0fd53c640 1 -- 172.21.15.154:0/3904722641 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0f80b74c0 con 0x7fa0f80d12e0 2023-09-23T03:16:00.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.132+0000 7fa0fd53c640 1 -- 172.21.15.154:0/3904722641 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0f80c6fc0 con 0x7fa0f80d12e0 2023-09-23T03:16:00.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.133+0000 7fa0fd53c640 1 -- 172.21.15.154:0/3904722641 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0f8073eb0 con 0x7fa0f80d12e0 2023-09-23T03:16:00.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.134+0000 7fa0ef7fe640 1 -- 172.21.15.154:0/3904722641 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fa0e8004190 con 0x7fa0f80d12e0 2023-09-23T03:16:00.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.135+0000 7fa0ef7fe640 1 --2- 172.21.15.154:0/3904722641 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa0d806af30 0x7fa0d806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:00.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.135+0000 7fa0ef7fe640 1 -- 172.21.15.154:0/3904722641 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa0e8089ca0 con 0x7fa0f80d12e0 2023-09-23T03:16:00.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.135+0000 7fa0f67fc640 1 --2- 172.21.15.154:0/3904722641 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa0d806af30 0x7fa0d806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:00.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.135+0000 7fa0f67fc640 1 --2- 172.21.15.154:0/3904722641 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa0d806af30 0x7fa0d806d3f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8152c30 tx=0x7fa0e0007a40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:00.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.139+0000 7fa0ef7fe640 1 -- 172.21.15.154:0/3904722641 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fa0e8054b30 con 0x7fa0f80d12e0 2023-09-23T03:16:00.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.330+0000 7fa0fd53c640 1 -- 172.21.15.154:0/3904722641 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0f80c6ac0 con 0x7fa0f80d12e0 2023-09-23T03:16:00.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.332+0000 7fa0ef7fe640 1 -- 172.21.15.154:0/3904722641 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18382 (secure 0 0 0) 0x7fa0e801a090 con 0x7fa0f80d12e0 2023-09-23T03:16:00.333 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:16:00.333 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":24,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:15:25.775060+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:15:18.575849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T03:15:19.707212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T03:15:23.337138+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:16:00.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.334+0000 7fa0fd53c640 1 -- 172.21.15.154:0/3904722641 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa0d806af30 msgr2=0x7fa0d806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:00.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.334+0000 7fa0fd53c640 1 --2- 172.21.15.154:0/3904722641 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa0d806af30 0x7fa0d806d3f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8152c30 tx=0x7fa0e0007a40 comp rx=0 tx=0).stop 2023-09-23T03:16:00.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.334+0000 7fa0fd53c640 1 -- 172.21.15.154:0/3904722641 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0f80d12e0 msgr2=0x7fa0f80c64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:00.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.334+0000 7fa0fd53c640 1 --2- 172.21.15.154:0/3904722641 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0f80d12e0 0x7fa0f80c64e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8009fc0 tx=0x7fa0e800d560 comp rx=0 tx=0).stop 2023-09-23T03:16:00.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.334+0000 7fa0fd53c640 1 -- 172.21.15.154:0/3904722641 shutdown_connections 2023-09-23T03:16:00.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.334+0000 7fa0fd53c640 1 --2- 172.21.15.154:0/3904722641 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa0d806af30 0x7fa0d806d3f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:00.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.334+0000 7fa0fd53c640 1 --2- 172.21.15.154:0/3904722641 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa0f815ce90 0x7fa0f814bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:00.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.334+0000 7fa0fd53c640 1 --2- 172.21.15.154:0/3904722641 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa0f815cab0 0x7fa0f80b2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:00.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.334+0000 7fa0fd53c640 1 --2- 172.21.15.154:0/3904722641 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0f80d12e0 0x7fa0f80c64e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:00.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.334+0000 7fa0fd53c640 1 -- 172.21.15.154:0/3904722641 >> 172.21.15.154:0/3904722641 conn(0x7fa0f8071bc0 msgr2=0x7fa0f80cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:00.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.334+0000 7fa0fd53c640 1 -- 172.21.15.154:0/3904722641 shutdown_connections 2023-09-23T03:16:00.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.334+0000 7fa0fd53c640 1 -- 172.21.15.154:0/3904722641 wait complete. 2023-09-23T03:16:00.353 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:15:18.575849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:15:19.707212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-23T03:15:23.337138+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-23T03:16:00.353 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:16:00.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.568+0000 7f6886eb4640 1 Processor -- start 2023-09-23T03:16:00.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.568+0000 7f6886eb4640 1 -- start start 2023-09-23T03:16:00.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.569+0000 7f6886eb4640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f68800c8d70 0x7f688015c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:00.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.569+0000 7f6886eb4640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68800c9d20 con 0x7f68800c9150 2023-09-23T03:16:00.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.569+0000 7f6886eb4640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6880071850 con 0x7f6880071290 2023-09-23T03:16:00.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.569+0000 7f6886eb4640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6880071b30 con 0x7f68800c8d70 2023-09-23T03:16:00.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.569+0000 7f687ffff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f68800c9150 0x7f688015cc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46314/0 (socket says 172.21.15.154:46314) 2023-09-23T03:16:00.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.569+0000 7f687ffff640 1 -- 172.21.15.154:0/461641462 learned_addr learned my addr 172.21.15.154:0/461641462 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:00.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.570+0000 7f687f7fe640 1 -- 172.21.15.154:0/461641462 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3920575186 0 0) 0x7f68800c9d20 con 0x7f68800c9150 2023-09-23T03:16:00.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.570+0000 7f687f7fe640 1 -- 172.21.15.154:0/461641462 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6868003520 con 0x7f68800c9150 2023-09-23T03:16:00.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.570+0000 7f6884c50640 1 --2- 172.21.15.154:0/461641462 >> v2:172.21.15.190:3300/0 conn(0x7f68800c8d70 0x7f688015c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:00.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.570+0000 7f687f7fe640 1 -- 172.21.15.154:0/461641462 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2942638849 0 0) 0x7f6868003520 con 0x7f68800c9150 2023-09-23T03:16:00.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.570+0000 7f687f7fe640 1 -- 172.21.15.154:0/461641462 >> v2:172.21.15.190:3300/0 conn(0x7f68800c8d70 msgr2=0x7f688015c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:00.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.570+0000 7f687f7fe640 1 --2- 172.21.15.154:0/461641462 >> v2:172.21.15.190:3300/0 conn(0x7f68800c8d70 0x7f688015c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:00.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.570+0000 7f687f7fe640 1 -- 172.21.15.154:0/461641462 >> v1:172.21.15.190:6789/0 conn(0x7f6880071290 legacy=0x7f68800ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:00.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.570+0000 7f687f7fe640 1 -- 172.21.15.154:0/461641462 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6880071e10 con 0x7f68800c9150 2023-09-23T03:16:00.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.570+0000 7f687f7fe640 1 -- 172.21.15.154:0/461641462 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f6874001670 con 0x7f68800c9150 2023-09-23T03:16:00.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.570+0000 7f687f7fe640 1 -- 172.21.15.154:0/461641462 >> v1:172.21.15.154:6789/0 conn(0x7f68800c9150 legacy=0x7f688015cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:00.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.570+0000 7f687f7fe640 1 --2- 172.21.15.154:0/461641462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68680039c0 0x7f6868003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:00.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.570+0000 7f687f7fe640 1 --2- 172.21.15.154:0/461641462 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68680047c0 0x7f6868004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:00.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.570+0000 7f687f7fe640 1 --2- 172.21.15.154:0/461641462 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68680051d0 0x7f686800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:00.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.570+0000 7f687f7fe640 1 -- 172.21.15.154:0/461641462 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6880071b30 con 0x7f68680039c0 2023-09-23T03:16:00.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.570+0000 7f687f7fe640 1 -- 172.21.15.154:0/461641462 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686800c120 con 0x7f68680047c0 2023-09-23T03:16:00.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.570+0000 7f687f7fe640 1 -- 172.21.15.154:0/461641462 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f686800c2e0 con 0x7f68680051d0 2023-09-23T03:16:00.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.571+0000 7f6884c50640 1 --2- 172.21.15.154:0/461641462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68680039c0 0x7f6868003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:00.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.571+0000 7f687ffff640 1 --2- 172.21.15.154:0/461641462 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68680047c0 0x7f6868004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:00.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.571+0000 7f6885451640 1 --2- 172.21.15.154:0/461641462 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68680051d0 0x7f686800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:00.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.571+0000 7f6884c50640 1 -- 172.21.15.154:0/461641462 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68680051d0 msgr2=0x7f686800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:00.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.571+0000 7f6884c50640 1 --2- 172.21.15.154:0/461641462 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68680051d0 0x7f686800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:00.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.571+0000 7f6884c50640 1 -- 172.21.15.154:0/461641462 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68680047c0 msgr2=0x7f6868004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:00.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.571+0000 7f6884c50640 1 --2- 172.21.15.154:0/461641462 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68680047c0 0x7f6868004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:00.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.571+0000 7f6884c50640 1 -- 172.21.15.154:0/461641462 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6868003520 con 0x7f68680039c0 2023-09-23T03:16:00.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.571+0000 7f6885451640 1 -- 172.21.15.154:0/461641462 reap_dead start 2023-09-23T03:16:00.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.572+0000 7f6884c50640 1 --2- 172.21.15.154:0/461641462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68680039c0 0x7f6868003de0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6870007360 tx=0x7f68680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d888bb430f5cd8f7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:00.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.572+0000 7f687f7fe640 1 -- 172.21.15.154:0/461641462 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687002ccd0 con 0x7f68680039c0 2023-09-23T03:16:00.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.572+0000 7f687f7fe640 1 -- 172.21.15.154:0/461641462 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f687002ce30 con 0x7f68680039c0 2023-09-23T03:16:00.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.572+0000 7f687f7fe640 1 -- 172.21.15.154:0/461641462 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6870034660 con 0x7f68680039c0 2023-09-23T03:16:00.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.573+0000 7f6886eb4640 1 -- 172.21.15.154:0/461641462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68680039c0 msgr2=0x7f6868003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:00.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.573+0000 7f6886eb4640 1 --2- 172.21.15.154:0/461641462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68680039c0 0x7f6868003de0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6870007360 tx=0x7f68680057f0 comp rx=0 tx=0).stop 2023-09-23T03:16:00.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.573+0000 7f6886eb4640 1 -- 172.21.15.154:0/461641462 shutdown_connections 2023-09-23T03:16:00.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.573+0000 7f6886eb4640 1 --2- 172.21.15.154:0/461641462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68680039c0 0x7f6868003de0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:00.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.573+0000 7f6886eb4640 1 -- 172.21.15.154:0/461641462 >> 172.21.15.154:0/461641462 conn(0x7f688006c010 msgr2=0x7f688006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:00.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.573+0000 7f6886eb4640 1 -- 172.21.15.154:0/461641462 shutdown_connections 2023-09-23T03:16:00.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.573+0000 7f6886eb4640 1 -- 172.21.15.154:0/461641462 wait complete. 2023-09-23T03:16:00.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.575+0000 7f6886eb4640 1 Processor -- start 2023-09-23T03:16:00.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.575+0000 7f6886eb4640 1 -- start start 2023-09-23T03:16:00.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.576+0000 7f6886eb4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68680039c0 0x7f688014cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:00.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.576+0000 7f6886eb4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68800cb050 0x7f68800cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:00.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.576+0000 7f6886eb4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68801f12e0 0x7f68801f1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:00.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.576+0000 7f6886eb4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68800cba10 con 0x7f68801f12e0 2023-09-23T03:16:00.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.576+0000 7f6886eb4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f688014d090 con 0x7f68800cb050 2023-09-23T03:16:00.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.576+0000 7f6886eb4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68801f2040 con 0x7f68680039c0 2023-09-23T03:16:00.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.576+0000 7f6885451640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68801f12e0 0x7f68801f1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:00.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.576+0000 7f6885451640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68801f12e0 0x7f68801f1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44766/0 (socket says 172.21.15.154:44766) 2023-09-23T03:16:00.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.576+0000 7f6885451640 1 -- 172.21.15.154:0/3887432887 learned_addr learned my addr 172.21.15.154:0/3887432887 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:00.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.576+0000 7f6884c50640 1 --2- 172.21.15.154:0/3887432887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68680039c0 0x7f688014cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:00.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.577+0000 7f687ffff640 1 --2- 172.21.15.154:0/3887432887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68800cb050 0x7f68800cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:00.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.577+0000 7f6885451640 1 -- 172.21.15.154:0/3887432887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68680039c0 msgr2=0x7f688014cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:00.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.577+0000 7f6885451640 1 --2- 172.21.15.154:0/3887432887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68680039c0 0x7f688014cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:00.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.577+0000 7f6885451640 1 -- 172.21.15.154:0/3887432887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68800cb050 msgr2=0x7f68800cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:00.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.577+0000 7f6885451640 1 --2- 172.21.15.154:0/3887432887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68800cb050 0x7f68800cb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:00.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.577+0000 7f6885451640 1 -- 172.21.15.154:0/3887432887 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f688014f2a0 con 0x7f68801f12e0 2023-09-23T03:16:00.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.577+0000 7f6884c50640 1 --2- 172.21.15.154:0/3887432887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68680039c0 0x7f688014cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:00.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.577+0000 7f6885451640 1 --2- 172.21.15.154:0/3887432887 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68801f12e0 0x7f68801f1760 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f68800b4600 tx=0x7f687800d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:00.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.577+0000 7f687d7fa640 1 -- 172.21.15.154:0/3887432887 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687800b070 con 0x7f68801f12e0 2023-09-23T03:16:00.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.577+0000 7f687d7fa640 1 -- 172.21.15.154:0/3887432887 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68780054e0 con 0x7f68801f12e0 2023-09-23T03:16:00.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.577+0000 7f687d7fa640 1 -- 172.21.15.154:0/3887432887 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6878003810 con 0x7f68801f12e0 2023-09-23T03:16:00.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.577+0000 7f6886eb4640 1 -- 172.21.15.154:0/3887432887 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f688014f580 con 0x7f68801f12e0 2023-09-23T03:16:00.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.577+0000 7f6886eb4640 1 -- 172.21.15.154:0/3887432887 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f688014fb40 con 0x7f68801f12e0 2023-09-23T03:16:00.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.579+0000 7f6886eb4640 1 -- 172.21.15.154:0/3887432887 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68800b41b0 con 0x7f68801f12e0 2023-09-23T03:16:00.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.579+0000 7f687d7fa640 1 -- 172.21.15.154:0/3887432887 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f6878003970 con 0x7f68801f12e0 2023-09-23T03:16:00.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.580+0000 7f687d7fa640 1 --2- 172.21.15.154:0/3887432887 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f686c06af30 0x7f686c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:00.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.580+0000 7f687d7fa640 1 -- 172.21.15.154:0/3887432887 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6878089060 con 0x7f68801f12e0 2023-09-23T03:16:00.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.581+0000 7f6884c50640 1 --2- 172.21.15.154:0/3887432887 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f686c06af30 0x7f686c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:00.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.581+0000 7f6884c50640 1 --2- 172.21.15.154:0/3887432887 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f686c06af30 0x7f686c06d3f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6870036000 tx=0x7f6870030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:00.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.585+0000 7f687d7fa640 1 -- 172.21.15.154:0/3887432887 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f6878053db0 con 0x7f68801f12e0 2023-09-23T03:16:00.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.771+0000 7f6886eb4640 1 -- 172.21.15.154:0/3887432887 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f688004e920 con 0x7f68801f12e0 2023-09-23T03:16:00.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.771+0000 7f687d7fa640 1 -- 172.21.15.154:0/3887432887 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f68780180a0 con 0x7f68801f12e0 2023-09-23T03:16:00.773 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:16:00.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.774+0000 7f6886eb4640 1 -- 172.21.15.154:0/3887432887 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f686c06af30 msgr2=0x7f686c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:00.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.774+0000 7f6886eb4640 1 --2- 172.21.15.154:0/3887432887 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f686c06af30 0x7f686c06d3f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6870036000 tx=0x7f6870030000 comp rx=0 tx=0).stop 2023-09-23T03:16:00.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.774+0000 7f6886eb4640 1 -- 172.21.15.154:0/3887432887 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68801f12e0 msgr2=0x7f68801f1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:00.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.774+0000 7f6886eb4640 1 --2- 172.21.15.154:0/3887432887 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68801f12e0 0x7f68801f1760 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f68800b4600 tx=0x7f687800d890 comp rx=0 tx=0).stop 2023-09-23T03:16:00.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.774+0000 7f6886eb4640 1 -- 172.21.15.154:0/3887432887 shutdown_connections 2023-09-23T03:16:00.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.774+0000 7f6886eb4640 1 --2- 172.21.15.154:0/3887432887 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f686c06af30 0x7f686c06d3f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:00.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.774+0000 7f6886eb4640 1 --2- 172.21.15.154:0/3887432887 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68801f12e0 0x7f68801f1760 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:00.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.774+0000 7f6886eb4640 1 --2- 172.21.15.154:0/3887432887 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68800cb050 0x7f68800cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:00.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.774+0000 7f6886eb4640 1 --2- 172.21.15.154:0/3887432887 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68680039c0 0x7f688014cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:00.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.774+0000 7f6886eb4640 1 -- 172.21.15.154:0/3887432887 >> 172.21.15.154:0/3887432887 conn(0x7f688006c010 msgr2=0x7f6880151ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:00.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.774+0000 7f6886eb4640 1 -- 172.21.15.154:0/3887432887 shutdown_connections 2023-09-23T03:16:00.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:00.774+0000 7f6886eb4640 1 -- 172.21.15.154:0/3887432887 wait complete. 2023-09-23T03:16:00.795 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:16:01.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.010+0000 7efc88e6f640 1 Processor -- start 2023-09-23T03:16:01.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.010+0000 7efc88e6f640 1 -- start start 2023-09-23T03:16:01.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.011+0000 7efc88e6f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efc8415cda0 0x7efc840d2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:01.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.011+0000 7efc88e6f640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efc840d2b60 0x7efc840cd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:01.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.011+0000 7efc88e6f640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc840d0bb0 con 0x7efc8415c9c0 2023-09-23T03:16:01.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.012+0000 7efc88e6f640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc840cd450 con 0x7efc8415cda0 2023-09-23T03:16:01.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.012+0000 7efc88e6f640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc840d0df0 con 0x7efc840d2b60 2023-09-23T03:16:01.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.012+0000 7efc7bfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efc8415cda0 0x7efc840d2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:01.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.012+0000 7efc7bfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efc8415cda0 0x7efc840d2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44768/0 (socket says 172.21.15.154:44768) 2023-09-23T03:16:01.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.012+0000 7efc7bfff640 1 -- 172.21.15.154:0/1136039977 learned_addr learned my addr 172.21.15.154:0/1136039977 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:01.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.012+0000 7efc82d9d640 1 --2- 172.21.15.154:0/1136039977 >> v2:172.21.15.190:3300/0 conn(0x7efc840d2b60 0x7efc840cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:01.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.012+0000 7efc7bfff640 1 -- 172.21.15.154:0/1136039977 >> v2:172.21.15.190:3300/0 conn(0x7efc840d2b60 msgr2=0x7efc840cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:01.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.012+0000 7efc7bfff640 1 --2- 172.21.15.154:0/1136039977 >> v2:172.21.15.190:3300/0 conn(0x7efc840d2b60 0x7efc840cd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:01.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.012+0000 7efc7bfff640 1 -- 172.21.15.154:0/1136039977 >> v1:172.21.15.190:6789/0 conn(0x7efc8415c9c0 legacy=0x7efc840d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:01.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.012+0000 7efc7bfff640 1 -- 172.21.15.154:0/1136039977 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc840d10d0 con 0x7efc8415cda0 2023-09-23T03:16:01.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.012+0000 7efc82d9d640 1 --2- 172.21.15.154:0/1136039977 >> v2:172.21.15.190:3300/0 conn(0x7efc840d2b60 0x7efc840cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:01.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.013+0000 7efc7bfff640 1 --2- 172.21.15.154:0/1136039977 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc8415cda0 0x7efc840d2620 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7efc840c9950 tx=0x7efc6c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f59ff355d9daf4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:01.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.014+0000 7efc81d9b640 1 -- 172.21.15.154:0/1136039977 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6c031870 con 0x7efc8415cda0 2023-09-23T03:16:01.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.014+0000 7efc81d9b640 1 -- 172.21.15.154:0/1136039977 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc6c0319d0 con 0x7efc8415cda0 2023-09-23T03:16:01.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.014+0000 7efc81d9b640 1 -- 172.21.15.154:0/1136039977 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6c031d60 con 0x7efc8415cda0 2023-09-23T03:16:01.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.014+0000 7efc88e6f640 1 -- 172.21.15.154:0/1136039977 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc8415cda0 msgr2=0x7efc840d2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:01.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.014+0000 7efc88e6f640 1 --2- 172.21.15.154:0/1136039977 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc8415cda0 0x7efc840d2620 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7efc840c9950 tx=0x7efc6c02b910 comp rx=0 tx=0).stop 2023-09-23T03:16:01.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.014+0000 7efc88e6f640 1 -- 172.21.15.154:0/1136039977 shutdown_connections 2023-09-23T03:16:01.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.014+0000 7efc88e6f640 1 --2- 172.21.15.154:0/1136039977 >> v2:172.21.15.190:3300/0 conn(0x7efc840d2b60 0x7efc840cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:01.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.014+0000 7efc88e6f640 1 --2- 172.21.15.154:0/1136039977 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc8415cda0 0x7efc840d2620 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:01.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.014+0000 7efc88e6f640 1 -- 172.21.15.154:0/1136039977 >> 172.21.15.154:0/1136039977 conn(0x7efc8409bc20 msgr2=0x7efc84089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:01.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.015+0000 7efc88e6f640 1 -- 172.21.15.154:0/1136039977 shutdown_connections 2023-09-23T03:16:01.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.015+0000 7efc88e6f640 1 -- 172.21.15.154:0/1136039977 wait complete. 2023-09-23T03:16:01.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.017+0000 7efc88e6f640 1 Processor -- start 2023-09-23T03:16:01.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.017+0000 7efc88e6f640 1 -- start start 2023-09-23T03:16:01.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.018+0000 7efc88e6f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc840d2b60 0x7efc840ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:01.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.018+0000 7efc88e6f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc8415c9c0 0x7efc840c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:01.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.019+0000 7efc88e6f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc8415cda0 0x7efc84149920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:01.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.019+0000 7efc88e6f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc840cede0 con 0x7efc840d2b60 2023-09-23T03:16:01.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.019+0000 7efc88e6f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc840c86d0 con 0x7efc8415cda0 2023-09-23T03:16:01.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.019+0000 7efc88e6f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc840ca9f0 con 0x7efc8415c9c0 2023-09-23T03:16:01.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.019+0000 7efc8259c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc840d2b60 0x7efc840ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:01.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.019+0000 7efc8259c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc840d2b60 0x7efc840ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44774/0 (socket says 172.21.15.154:44774) 2023-09-23T03:16:01.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.019+0000 7efc8259c640 1 -- 172.21.15.154:0/2200427600 learned_addr learned my addr 172.21.15.154:0/2200427600 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:01.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.019+0000 7efc8259c640 1 -- 172.21.15.154:0/2200427600 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc8415c9c0 msgr2=0x7efc840c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:01.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.019+0000 7efc8259c640 1 --2- 172.21.15.154:0/2200427600 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc8415c9c0 0x7efc840c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:01.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.019+0000 7efc8259c640 1 -- 172.21.15.154:0/2200427600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc8415cda0 msgr2=0x7efc84149920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:01.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.019+0000 7efc8259c640 1 --2- 172.21.15.154:0/2200427600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc8415cda0 0x7efc84149920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:01.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.019+0000 7efc8259c640 1 -- 172.21.15.154:0/2200427600 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc6c0094d0 con 0x7efc840d2b60 2023-09-23T03:16:01.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.019+0000 7efc8259c640 1 --2- 172.21.15.154:0/2200427600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc840d2b60 0x7efc840ce8a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efc7400b3e0 tx=0x7efc7400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:01.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.020+0000 7efc7affd640 1 -- 172.21.15.154:0/2200427600 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc74011830 con 0x7efc840d2b60 2023-09-23T03:16:01.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.020+0000 7efc7affd640 1 -- 172.21.15.154:0/2200427600 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc740119b0 con 0x7efc840d2b60 2023-09-23T03:16:01.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.020+0000 7efc7affd640 1 -- 172.21.15.154:0/2200427600 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc74011da0 con 0x7efc840d2b60 2023-09-23T03:16:01.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.020+0000 7efc88e6f640 1 -- 172.21.15.154:0/2200427600 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc840c88e0 con 0x7efc840d2b60 2023-09-23T03:16:01.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.020+0000 7efc88e6f640 1 -- 172.21.15.154:0/2200427600 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc840c8ea0 con 0x7efc840d2b60 2023-09-23T03:16:01.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.021+0000 7efc88e6f640 1 -- 172.21.15.154:0/2200427600 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc840cab60 con 0x7efc840d2b60 2023-09-23T03:16:01.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.022+0000 7efc7affd640 1 -- 172.21.15.154:0/2200427600 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7efc7401a410 con 0x7efc840d2b60 2023-09-23T03:16:01.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.023+0000 7efc7affd640 1 --2- 172.21.15.154:0/2200427600 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efc6406af30 0x7efc6406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:01.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.023+0000 7efc7affd640 1 -- 172.21.15.154:0/2200427600 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7efc74089aa0 con 0x7efc840d2b60 2023-09-23T03:16:01.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.023+0000 7efc7bfff640 1 --2- 172.21.15.154:0/2200427600 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efc6406af30 0x7efc6406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:01.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.024+0000 7efc7bfff640 1 --2- 172.21.15.154:0/2200427600 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efc6406af30 0x7efc6406d3f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efc6c036000 tx=0x7efc6c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:01.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.027+0000 7efc7affd640 1 -- 172.21.15.154:0/2200427600 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7efc74054750 con 0x7efc840d2b60 2023-09-23T03:16:01.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.217+0000 7efc88e6f640 1 -- 172.21.15.154:0/2200427600 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efc8404e920 con 0x7efc840d2b60 2023-09-23T03:16:01.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.218+0000 7efc7affd640 1 -- 172.21.15.154:0/2200427600 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7efc740150a0 con 0x7efc840d2b60 2023-09-23T03:16:01.220 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:16:01.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.221+0000 7efc88e6f640 1 -- 172.21.15.154:0/2200427600 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efc6406af30 msgr2=0x7efc6406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:01.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.221+0000 7efc88e6f640 1 --2- 172.21.15.154:0/2200427600 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efc6406af30 0x7efc6406d3f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efc6c036000 tx=0x7efc6c030000 comp rx=0 tx=0).stop 2023-09-23T03:16:01.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.221+0000 7efc88e6f640 1 -- 172.21.15.154:0/2200427600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc840d2b60 msgr2=0x7efc840ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:01.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.221+0000 7efc88e6f640 1 --2- 172.21.15.154:0/2200427600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc840d2b60 0x7efc840ce8a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efc7400b3e0 tx=0x7efc7400b8a0 comp rx=0 tx=0).stop 2023-09-23T03:16:01.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.221+0000 7efc88e6f640 1 -- 172.21.15.154:0/2200427600 shutdown_connections 2023-09-23T03:16:01.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.221+0000 7efc88e6f640 1 --2- 172.21.15.154:0/2200427600 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efc6406af30 0x7efc6406d3f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:01.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.221+0000 7efc88e6f640 1 --2- 172.21.15.154:0/2200427600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc8415cda0 0x7efc84149920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:01.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.221+0000 7efc88e6f640 1 --2- 172.21.15.154:0/2200427600 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc8415c9c0 0x7efc840c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:01.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.221+0000 7efc88e6f640 1 --2- 172.21.15.154:0/2200427600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc840d2b60 0x7efc840ce8a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:01.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.221+0000 7efc88e6f640 1 -- 172.21.15.154:0/2200427600 >> 172.21.15.154:0/2200427600 conn(0x7efc8409bc20 msgr2=0x7efc840b4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:01.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.221+0000 7efc88e6f640 1 -- 172.21.15.154:0/2200427600 shutdown_connections 2023-09-23T03:16:01.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.222+0000 7efc88e6f640 1 -- 172.21.15.154:0/2200427600 wait complete. 2023-09-23T03:16:01.239 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:16:01.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.447+0000 7f979cd61640 1 Processor -- start 2023-09-23T03:16:01.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.447+0000 7f979cd61640 1 -- start start 2023-09-23T03:16:01.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.449+0000 7f979cd61640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f97980c8d70 0x7f979815c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:01.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.449+0000 7f979cd61640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97980c9d20 con 0x7f9798071290 2023-09-23T03:16:01.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.449+0000 7f979cd61640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9798071850 con 0x7f97980c9150 2023-09-23T03:16:01.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.449+0000 7f979cd61640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9798071b30 con 0x7f97980c8d70 2023-09-23T03:16:01.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.449+0000 7f9796d9d640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f9798071290 0x7f97980ca1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46318/0 (socket says 172.21.15.154:46318) 2023-09-23T03:16:01.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.449+0000 7f9796d9d640 1 -- 172.21.15.154:0/48826780 learned_addr learned my addr 172.21.15.154:0/48826780 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:01.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.449+0000 7f979659c640 1 --2- 172.21.15.154:0/48826780 >> v2:172.21.15.190:3300/0 conn(0x7f97980c8d70 0x7f979815c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:01.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.450+0000 7f979559a640 1 -- 172.21.15.154:0/48826780 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4280800944 0 0) 0x7f97980c9d20 con 0x7f9798071290 2023-09-23T03:16:01.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.450+0000 7f979559a640 1 -- 172.21.15.154:0/48826780 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9780005730 con 0x7f9798071290 2023-09-23T03:16:01.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.450+0000 7f979559a640 1 -- 172.21.15.154:0/48826780 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3809510573 0 0) 0x7f9780005730 con 0x7f9798071290 2023-09-23T03:16:01.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.450+0000 7f979559a640 1 -- 172.21.15.154:0/48826780 >> v2:172.21.15.190:3300/0 conn(0x7f97980c8d70 msgr2=0x7f979815c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:01.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.450+0000 7f979559a640 1 --2- 172.21.15.154:0/48826780 >> v2:172.21.15.190:3300/0 conn(0x7f97980c8d70 0x7f979815c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:01.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.450+0000 7f979559a640 1 -- 172.21.15.154:0/48826780 >> v1:172.21.15.190:6789/0 conn(0x7f97980c9150 legacy=0x7f979815cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:01.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.450+0000 7f979559a640 1 -- 172.21.15.154:0/48826780 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9798071e10 con 0x7f9798071290 2023-09-23T03:16:01.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.450+0000 7f979559a640 1 -- 172.21.15.154:0/48826780 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f97900019f0 con 0x7f9798071290 2023-09-23T03:16:01.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.450+0000 7f979559a640 1 -- 172.21.15.154:0/48826780 >> v1:172.21.15.154:6789/0 conn(0x7f9798071290 legacy=0x7f97980ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:01.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.450+0000 7f979559a640 1 --2- 172.21.15.154:0/48826780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9780005b80 0x7f9780005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:01.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.450+0000 7f979559a640 1 --2- 172.21.15.154:0/48826780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97800069d0 0x7f9780006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:01.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.450+0000 7f979559a640 1 --2- 172.21.15.154:0/48826780 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9780007390 0x7f978000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:01.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.450+0000 7f979559a640 1 -- 172.21.15.154:0/48826780 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9798071b30 con 0x7f9780007390 2023-09-23T03:16:01.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.450+0000 7f979559a640 1 -- 172.21.15.154:0/48826780 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978000e230 con 0x7f9780005b80 2023-09-23T03:16:01.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.450+0000 7f979559a640 1 -- 172.21.15.154:0/48826780 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f978000e3f0 con 0x7f97800069d0 2023-09-23T03:16:01.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.451+0000 7f9796d9d640 1 --2- 172.21.15.154:0/48826780 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9780007390 0x7f978000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:01.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.451+0000 7f9795d9b640 1 --2- 172.21.15.154:0/48826780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97800069d0 0x7f9780006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:01.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.451+0000 7f979659c640 1 --2- 172.21.15.154:0/48826780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9780005b80 0x7f9780005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:01.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.451+0000 7f9796d9d640 1 -- 172.21.15.154:0/48826780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97800069d0 msgr2=0x7f9780006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:01.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.451+0000 7f9796d9d640 1 --2- 172.21.15.154:0/48826780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97800069d0 0x7f9780006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:01.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.451+0000 7f9796d9d640 1 -- 172.21.15.154:0/48826780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9780005b80 msgr2=0x7f9780005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:01.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.451+0000 7f9796d9d640 1 --2- 172.21.15.154:0/48826780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9780005b80 0x7f9780005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:01.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.451+0000 7f9796d9d640 1 -- 172.21.15.154:0/48826780 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9780005730 con 0x7f9780007390 2023-09-23T03:16:01.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.451+0000 7f979659c640 1 --2- 172.21.15.154:0/48826780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9780005b80 0x7f9780005fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:16:01.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.452+0000 7f9796d9d640 1 -- 172.21.15.154:0/48826780 reap_dead start 2023-09-23T03:16:01.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.452+0000 7f9796d9d640 1 --2- 172.21.15.154:0/48826780 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9780007390 0x7f978000db90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f979000a830 tx=0x7f9780007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=726d9f6f2c4af63 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:01.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.452+0000 7f979559a640 1 -- 172.21.15.154:0/48826780 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97900072d0 con 0x7f9780007390 2023-09-23T03:16:01.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.453+0000 7f979559a640 1 -- 172.21.15.154:0/48826780 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979002f070 con 0x7f9780007390 2023-09-23T03:16:01.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.453+0000 7f979559a640 1 -- 172.21.15.154:0/48826780 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97900076b0 con 0x7f9780007390 2023-09-23T03:16:01.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.453+0000 7f979cd61640 1 -- 172.21.15.154:0/48826780 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9780007390 msgr2=0x7f978000db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:01.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.453+0000 7f979cd61640 1 --2- 172.21.15.154:0/48826780 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9780007390 0x7f978000db90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f979000a830 tx=0x7f9780007a00 comp rx=0 tx=0).stop 2023-09-23T03:16:01.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.453+0000 7f979cd61640 1 -- 172.21.15.154:0/48826780 shutdown_connections 2023-09-23T03:16:01.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.453+0000 7f979cd61640 1 --2- 172.21.15.154:0/48826780 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9780007390 0x7f978000db90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:01.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.453+0000 7f979cd61640 1 -- 172.21.15.154:0/48826780 >> 172.21.15.154:0/48826780 conn(0x7f979806c010 msgr2=0x7f979806c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:01.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.453+0000 7f979cd61640 1 -- 172.21.15.154:0/48826780 shutdown_connections 2023-09-23T03:16:01.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.453+0000 7f979cd61640 1 -- 172.21.15.154:0/48826780 wait complete. 2023-09-23T03:16:01.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.455+0000 7f979cd61640 1 Processor -- start 2023-09-23T03:16:01.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.455+0000 7f979cd61640 1 -- start start 2023-09-23T03:16:01.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.456+0000 7f979cd61640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9780007390 0x7f97981f1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:01.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.456+0000 7f979cd61640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97980c6ec0 0x7f97980c7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:01.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.456+0000 7f979cd61640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97980c3030 0x7f97980c34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:01.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.456+0000 7f979cd61640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9798156d20 con 0x7f9780007390 2023-09-23T03:16:01.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.456+0000 7f979cd61640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97980c39f0 con 0x7f97980c3030 2023-09-23T03:16:01.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.456+0000 7f979cd61640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97981f2030 con 0x7f97980c6ec0 2023-09-23T03:16:01.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.457+0000 7f979659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9780007390 0x7f97981f1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:01.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.457+0000 7f979659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9780007390 0x7f97981f1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44802/0 (socket says 172.21.15.154:44802) 2023-09-23T03:16:01.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.457+0000 7f979659c640 1 -- 172.21.15.154:0/2839908325 learned_addr learned my addr 172.21.15.154:0/2839908325 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:01.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.457+0000 7f9795d9b640 1 --2- 172.21.15.154:0/2839908325 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97980c6ec0 0x7f97980c7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:01.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.457+0000 7f9796d9d640 1 --2- 172.21.15.154:0/2839908325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97980c3030 0x7f97980c34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:01.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.457+0000 7f979659c640 1 -- 172.21.15.154:0/2839908325 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97980c6ec0 msgr2=0x7f97980c7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:01.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.457+0000 7f979659c640 1 --2- 172.21.15.154:0/2839908325 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97980c6ec0 0x7f97980c7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:01.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.457+0000 7f979659c640 1 -- 172.21.15.154:0/2839908325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97980c3030 msgr2=0x7f97980c34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:01.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.457+0000 7f979659c640 1 --2- 172.21.15.154:0/2839908325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97980c3030 0x7f97980c34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:01.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.457+0000 7f979659c640 1 -- 172.21.15.154:0/2839908325 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97980c8500 con 0x7f9780007390 2023-09-23T03:16:01.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.458+0000 7f979659c640 1 --2- 172.21.15.154:0/2839908325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9780007390 0x7f97981f1af0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9788011570 tx=0x7f9788011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:01.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.458+0000 7f9786ffd640 1 -- 172.21.15.154:0/2839908325 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9788015870 con 0x7f9780007390 2023-09-23T03:16:01.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.458+0000 7f9786ffd640 1 -- 172.21.15.154:0/2839908325 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97880159d0 con 0x7f9780007390 2023-09-23T03:16:01.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.458+0000 7f979cd61640 1 -- 172.21.15.154:0/2839908325 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97980c7980 con 0x7f9780007390 2023-09-23T03:16:01.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.458+0000 7f9786ffd640 1 -- 172.21.15.154:0/2839908325 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9788015d40 con 0x7f9780007390 2023-09-23T03:16:01.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.458+0000 7f979cd61640 1 -- 172.21.15.154:0/2839908325 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97980c7f40 con 0x7f9780007390 2023-09-23T03:16:01.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.459+0000 7f979cd61640 1 -- 172.21.15.154:0/2839908325 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f979804e920 con 0x7f9780007390 2023-09-23T03:16:01.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.460+0000 7f9786ffd640 1 -- 172.21.15.154:0/2839908325 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f978801d400 con 0x7f9780007390 2023-09-23T03:16:01.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.461+0000 7f9786ffd640 1 --2- 172.21.15.154:0/2839908325 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f977806af60 0x7f977806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:01.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.461+0000 7f9786ffd640 1 -- 172.21.15.154:0/2839908325 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f978808a2a0 con 0x7f9780007390 2023-09-23T03:16:01.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.461+0000 7f9795d9b640 1 --2- 172.21.15.154:0/2839908325 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f977806af60 0x7f977806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:01.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.462+0000 7f9795d9b640 1 --2- 172.21.15.154:0/2839908325 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f977806af60 0x7f977806d420 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f978c000c00 tx=0x7f978c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:01.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.466+0000 7f9786ffd640 1 -- 172.21.15.154:0/2839908325 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f9788054f50 con 0x7f9780007390 2023-09-23T03:16:01.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.656+0000 7f979cd61640 1 -- 172.21.15.154:0/2839908325 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f979806cbc0 con 0x7f9780007390 2023-09-23T03:16:01.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.657+0000 7f9786ffd640 1 -- 172.21.15.154:0/2839908325 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9788058570 con 0x7f9780007390 2023-09-23T03:16:01.658 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:16:01.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.660+0000 7f979cd61640 1 -- 172.21.15.154:0/2839908325 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f977806af60 msgr2=0x7f977806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:01.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.660+0000 7f979cd61640 1 --2- 172.21.15.154:0/2839908325 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f977806af60 0x7f977806d420 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f978c000c00 tx=0x7f978c009040 comp rx=0 tx=0).stop 2023-09-23T03:16:01.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.660+0000 7f979cd61640 1 -- 172.21.15.154:0/2839908325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9780007390 msgr2=0x7f97981f1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:01.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.660+0000 7f979cd61640 1 --2- 172.21.15.154:0/2839908325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9780007390 0x7f97981f1af0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9788011570 tx=0x7f9788011a30 comp rx=0 tx=0).stop 2023-09-23T03:16:01.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.660+0000 7f979cd61640 1 -- 172.21.15.154:0/2839908325 shutdown_connections 2023-09-23T03:16:01.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.660+0000 7f979cd61640 1 --2- 172.21.15.154:0/2839908325 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f977806af60 0x7f977806d420 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:01.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.660+0000 7f979cd61640 1 --2- 172.21.15.154:0/2839908325 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97980c3030 0x7f97980c34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:01.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.660+0000 7f979cd61640 1 --2- 172.21.15.154:0/2839908325 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97980c6ec0 0x7f97980c7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:01.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.661+0000 7f979cd61640 1 --2- 172.21.15.154:0/2839908325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9780007390 0x7f97981f1af0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:01.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.661+0000 7f979cd61640 1 -- 172.21.15.154:0/2839908325 >> 172.21.15.154:0/2839908325 conn(0x7f979806c010 msgr2=0x7f97980c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:01.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.661+0000 7f979cd61640 1 -- 172.21.15.154:0/2839908325 shutdown_connections 2023-09-23T03:16:01.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.661+0000 7f979cd61640 1 -- 172.21.15.154:0/2839908325 wait complete. 2023-09-23T03:16:01.680 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-23T03:16:01.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.895+0000 7f4daa73c640 1 Processor -- start 2023-09-23T03:16:01.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.895+0000 7f4daa73c640 1 -- start start 2023-09-23T03:16:01.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.897+0000 7f4daa73c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4da40ce940 0x7f4da41677f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:01.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.897+0000 7f4daa73c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da40d0bb0 con 0x7f4da415cda0 2023-09-23T03:16:01.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.897+0000 7f4daa73c640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da40cd450 con 0x7f4da415c9c0 2023-09-23T03:16:01.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.897+0000 7f4daa73c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4da40d0df0 con 0x7f4da40ce940 2023-09-23T03:16:01.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.897+0000 7f4da8cd9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4da40ce940 0x7f4da41677f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:01.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.897+0000 7f4da37fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f4da415cda0 0x7f4da40d2620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46324/0 (socket says 172.21.15.154:46324) 2023-09-23T03:16:01.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.897+0000 7f4da37fe640 1 -- 172.21.15.154:0/1149614495 learned_addr learned my addr 172.21.15.154:0/1149614495 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:01.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.898+0000 7f4da8cd9640 1 -- 172.21.15.154:0/1149614495 >> v1:172.21.15.190:6789/0 conn(0x7f4da415c9c0 legacy=0x7f4da40d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:01.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.898+0000 7f4da8cd9640 1 -- 172.21.15.154:0/1149614495 >> v1:172.21.15.154:6789/0 conn(0x7f4da415cda0 legacy=0x7f4da40d2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:01.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.898+0000 7f4da8cd9640 1 -- 172.21.15.154:0/1149614495 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da40d10d0 con 0x7f4da40ce940 2023-09-23T03:16:01.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.898+0000 7f4da2ffd640 1 -- 172.21.15.154:0/1149614495 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3703609334 0 0) 0x7f4d98001110 con 0x7f4da415cda0 2023-09-23T03:16:01.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.899+0000 7f4da8cd9640 1 --2- 172.21.15.154:0/1149614495 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4da40ce940 0x7f4da41677f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c0097a0 tx=0x7f4d9c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8438c671462b918 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:01.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.899+0000 7f4da2ffd640 1 -- 172.21.15.154:0/1149614495 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c031830 con 0x7f4da40ce940 2023-09-23T03:16:01.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.899+0000 7f4da2ffd640 1 -- 172.21.15.154:0/1149614495 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d9c031990 con 0x7f4da40ce940 2023-09-23T03:16:01.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.899+0000 7f4da2ffd640 1 -- 172.21.15.154:0/1149614495 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c031d20 con 0x7f4da40ce940 2023-09-23T03:16:01.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.900+0000 7f4daa73c640 1 -- 172.21.15.154:0/1149614495 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4da40ce940 msgr2=0x7f4da41677f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:01.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.900+0000 7f4daa73c640 1 --2- 172.21.15.154:0/1149614495 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4da40ce940 0x7f4da41677f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c0097a0 tx=0x7f4d9c02b8d0 comp rx=0 tx=0).stop 2023-09-23T03:16:01.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.900+0000 7f4daa73c640 1 -- 172.21.15.154:0/1149614495 shutdown_connections 2023-09-23T03:16:01.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.900+0000 7f4daa73c640 1 --2- 172.21.15.154:0/1149614495 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4da40ce940 0x7f4da41677f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:01.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.900+0000 7f4daa73c640 1 -- 172.21.15.154:0/1149614495 >> 172.21.15.154:0/1149614495 conn(0x7f4da409bc20 msgr2=0x7f4da4089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:01.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.900+0000 7f4daa73c640 1 -- 172.21.15.154:0/1149614495 shutdown_connections 2023-09-23T03:16:01.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.900+0000 7f4daa73c640 1 -- 172.21.15.154:0/1149614495 wait complete. 2023-09-23T03:16:01.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.901+0000 7f4daa73c640 1 Processor -- start 2023-09-23T03:16:01.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.902+0000 7f4daa73c640 1 -- start start 2023-09-23T03:16:01.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.903+0000 7f4daa73c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4da40ce940 0x7f4da40cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:01.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.903+0000 7f4daa73c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4da415c9c0 0x7f4da415ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:01.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.903+0000 7f4daa73c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4da415cda0 0x7f4da415f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:01.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.903+0000 7f4daa73c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da40cbf30 con 0x7f4da40ce940 2023-09-23T03:16:01.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.903+0000 7f4daa73c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da415e670 con 0x7f4da415cda0 2023-09-23T03:16:01.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.903+0000 7f4daa73c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da415ddd0 con 0x7f4da415c9c0 2023-09-23T03:16:01.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.903+0000 7f4da3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4da40ce940 0x7f4da40cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:01.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.903+0000 7f4da3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4da40ce940 0x7f4da40cb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44826/0 (socket says 172.21.15.154:44826) 2023-09-23T03:16:01.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.903+0000 7f4da3fff640 1 -- 172.21.15.154:0/678778587 learned_addr learned my addr 172.21.15.154:0/678778587 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:01.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.903+0000 7f4da37fe640 1 --2- 172.21.15.154:0/678778587 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4da415c9c0 0x7f4da415ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:01.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.903+0000 7f4da8cd9640 1 --2- 172.21.15.154:0/678778587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4da415cda0 0x7f4da415f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:01.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.903+0000 7f4da3fff640 1 -- 172.21.15.154:0/678778587 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4da415c9c0 msgr2=0x7f4da415ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:01.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.903+0000 7f4da3fff640 1 --2- 172.21.15.154:0/678778587 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4da415c9c0 0x7f4da415ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:01.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.903+0000 7f4da3fff640 1 -- 172.21.15.154:0/678778587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4da415cda0 msgr2=0x7f4da415f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:01.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.903+0000 7f4da3fff640 1 --2- 172.21.15.154:0/678778587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4da415cda0 0x7f4da415f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:01.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.903+0000 7f4da3fff640 1 -- 172.21.15.154:0/678778587 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d9c009400 con 0x7f4da40ce940 2023-09-23T03:16:01.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.904+0000 7f4da3fff640 1 --2- 172.21.15.154:0/678778587 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4da40ce940 0x7f4da40cb9f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4d9400c7c0 tx=0x7f4d9400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:01.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.904+0000 7f4da0ff9640 1 -- 172.21.15.154:0/678778587 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d94003bc0 con 0x7f4da40ce940 2023-09-23T03:16:01.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.904+0000 7f4da0ff9640 1 -- 172.21.15.154:0/678778587 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d94003d40 con 0x7f4da40ce940 2023-09-23T03:16:01.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.904+0000 7f4daa73c640 1 -- 172.21.15.154:0/678778587 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4da415e0b0 con 0x7f4da40ce940 2023-09-23T03:16:01.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.904+0000 7f4da0ff9640 1 -- 172.21.15.154:0/678778587 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9400aa90 con 0x7f4da40ce940 2023-09-23T03:16:01.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.904+0000 7f4daa73c640 1 -- 172.21.15.154:0/678778587 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4da40c9ad0 con 0x7f4da40ce940 2023-09-23T03:16:01.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.905+0000 7f4daa73c640 1 -- 172.21.15.154:0/678778587 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4da404e920 con 0x7f4da40ce940 2023-09-23T03:16:01.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.906+0000 7f4da0ff9640 1 -- 172.21.15.154:0/678778587 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f4d9400abf0 con 0x7f4da40ce940 2023-09-23T03:16:01.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.907+0000 7f4da0ff9640 1 --2- 172.21.15.154:0/678778587 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4d9006af30 0x7f4d9006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:01.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.907+0000 7f4da0ff9640 1 -- 172.21.15.154:0/678778587 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4d940898d0 con 0x7f4da40ce940 2023-09-23T03:16:01.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.907+0000 7f4da37fe640 1 --2- 172.21.15.154:0/678778587 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4d9006af30 0x7f4d9006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:01.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.908+0000 7f4da37fe640 1 --2- 172.21.15.154:0/678778587 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4d9006af30 0x7f4d9006d3f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4da414f870 tx=0x7f4d98007d00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:01.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:01.911+0000 7f4da0ff9640 1 -- 172.21.15.154:0/678778587 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f4d940546b0 con 0x7f4da40ce940 2023-09-23T03:16:02.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.099+0000 7f4daa73c640 1 -- 172.21.15.154:0/678778587 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f4da40cc160 con 0x7f4da40ce940 2023-09-23T03:16:02.101 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.100+0000 7f4da0ff9640 1 -- 172.21.15.154:0/678778587 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4d94057cd0 con 0x7f4da40ce940 2023-09-23T03:16:02.102 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:16:02.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.102+0000 7f4daa73c640 1 -- 172.21.15.154:0/678778587 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4d9006af30 msgr2=0x7f4d9006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:02.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.102+0000 7f4daa73c640 1 --2- 172.21.15.154:0/678778587 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4d9006af30 0x7f4d9006d3f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4da414f870 tx=0x7f4d98007d00 comp rx=0 tx=0).stop 2023-09-23T03:16:02.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.103+0000 7f4daa73c640 1 -- 172.21.15.154:0/678778587 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4da40ce940 msgr2=0x7f4da40cb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:02.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.103+0000 7f4daa73c640 1 --2- 172.21.15.154:0/678778587 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4da40ce940 0x7f4da40cb9f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4d9400c7c0 tx=0x7f4d9400cc80 comp rx=0 tx=0).stop 2023-09-23T03:16:02.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.103+0000 7f4daa73c640 1 -- 172.21.15.154:0/678778587 shutdown_connections 2023-09-23T03:16:02.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.103+0000 7f4daa73c640 1 --2- 172.21.15.154:0/678778587 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4d9006af30 0x7f4d9006d3f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:02.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.103+0000 7f4daa73c640 1 --2- 172.21.15.154:0/678778587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4da415cda0 0x7f4da415f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:02.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.103+0000 7f4daa73c640 1 --2- 172.21.15.154:0/678778587 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4da415c9c0 0x7f4da415ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:02.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.103+0000 7f4daa73c640 1 --2- 172.21.15.154:0/678778587 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4da40ce940 0x7f4da40cb9f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:02.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.103+0000 7f4daa73c640 1 -- 172.21.15.154:0/678778587 >> 172.21.15.154:0/678778587 conn(0x7f4da409bc20 msgr2=0x7f4da415c040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:02.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.103+0000 7f4daa73c640 1 -- 172.21.15.154:0/678778587 shutdown_connections 2023-09-23T03:16:02.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.103+0000 7f4daa73c640 1 -- 172.21.15.154:0/678778587 wait complete. 2023-09-23T03:16:02.122 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:16:02.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.321+0000 7fe45ab97640 1 Processor -- start 2023-09-23T03:16:02.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.321+0000 7fe45ab97640 1 -- start start 2023-09-23T03:16:02.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.322+0000 7fe45ab97640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe4540acce0 0x7fe45415ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:02.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.322+0000 7fe45ab97640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe4540b1b20 0x7fe4540ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:02.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.322+0000 7fe45ab97640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe45415ac30 con 0x7fe4540cee70 2023-09-23T03:16:02.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.322+0000 7fe45ab97640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4540b2e90 con 0x7fe4540acce0 2023-09-23T03:16:02.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.322+0000 7fe45ab97640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4540cf300 con 0x7fe4540b1b20 2023-09-23T03:16:02.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.323+0000 7fe458933640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe4540acce0 0x7fe45415ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:02.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.323+0000 7fe459134640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fe4540cee70 0x7fe45406cb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46338/0 (socket says 172.21.15.154:46338) 2023-09-23T03:16:02.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.323+0000 7fe459134640 1 -- 172.21.15.154:0/1795944792 learned_addr learned my addr 172.21.15.154:0/1795944792 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:02.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.323+0000 7fe453fff640 1 --2- 172.21.15.154:0/1795944792 >> v2:172.21.15.190:3300/0 conn(0x7fe4540b1b20 0x7fe4540ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:02.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.323+0000 7fe458933640 1 -- 172.21.15.154:0/1795944792 >> v2:172.21.15.190:3300/0 conn(0x7fe4540b1b20 msgr2=0x7fe4540ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:02.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.323+0000 7fe458933640 1 --2- 172.21.15.154:0/1795944792 >> v2:172.21.15.190:3300/0 conn(0x7fe4540b1b20 0x7fe4540ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:02.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.323+0000 7fe458933640 1 -- 172.21.15.154:0/1795944792 >> v1:172.21.15.154:6789/0 conn(0x7fe4540cee70 legacy=0x7fe45406cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:02.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.323+0000 7fe458933640 1 -- 172.21.15.154:0/1795944792 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4540b1f70 con 0x7fe4540acce0 2023-09-23T03:16:02.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.323+0000 7fe4537fe640 1 -- 172.21.15.154:0/1795944792 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2468557876 0 0) 0x7fe44c001490 con 0x7fe4540cee70 2023-09-23T03:16:02.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.325+0000 7fe458933640 1 --2- 172.21.15.154:0/1795944792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4540acce0 0x7fe45415ed60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe4540c9430 tx=0x7fe44802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=896eec737d11cc67 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:02.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.325+0000 7fe4537fe640 1 -- 172.21.15.154:0/1795944792 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44802e070 con 0x7fe4540acce0 2023-09-23T03:16:02.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.325+0000 7fe4537fe640 1 -- 172.21.15.154:0/1795944792 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4480315a0 con 0x7fe4540acce0 2023-09-23T03:16:02.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.325+0000 7fe4537fe640 1 -- 172.21.15.154:0/1795944792 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe448031930 con 0x7fe4540acce0 2023-09-23T03:16:02.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.326+0000 7fe45ab97640 1 -- 172.21.15.154:0/1795944792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4540acce0 msgr2=0x7fe45415ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:02.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.326+0000 7fe45ab97640 1 --2- 172.21.15.154:0/1795944792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4540acce0 0x7fe45415ed60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe4540c9430 tx=0x7fe44802b840 comp rx=0 tx=0).stop 2023-09-23T03:16:02.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.326+0000 7fe45ab97640 1 -- 172.21.15.154:0/1795944792 shutdown_connections 2023-09-23T03:16:02.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.326+0000 7fe45ab97640 1 --2- 172.21.15.154:0/1795944792 >> v2:172.21.15.190:3300/0 conn(0x7fe4540b1b20 0x7fe4540ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:02.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.326+0000 7fe45ab97640 1 --2- 172.21.15.154:0/1795944792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4540acce0 0x7fe45415ed60 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:02.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.326+0000 7fe45ab97640 1 -- 172.21.15.154:0/1795944792 >> 172.21.15.154:0/1795944792 conn(0x7fe45406a0f0 msgr2=0x7fe45406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:02.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.326+0000 7fe45ab97640 1 -- 172.21.15.154:0/1795944792 shutdown_connections 2023-09-23T03:16:02.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.326+0000 7fe45ab97640 1 -- 172.21.15.154:0/1795944792 wait complete. 2023-09-23T03:16:02.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.328+0000 7fe45ab97640 1 Processor -- start 2023-09-23T03:16:02.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.328+0000 7fe45ab97640 1 -- start start 2023-09-23T03:16:02.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.329+0000 7fe45ab97640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4540acce0 0x7fe4541564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:02.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.329+0000 7fe45ab97640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4540b1b20 0x7fe4540c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:02.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.329+0000 7fe45ab97640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4540cee70 0x7fe45415b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:02.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.329+0000 7fe45ab97640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45415b5a0 con 0x7fe4540b1b20 2023-09-23T03:16:02.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.329+0000 7fe45ab97640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4541569f0 con 0x7fe4540acce0 2023-09-23T03:16:02.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.329+0000 7fe45ab97640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4540c80d0 con 0x7fe4540cee70 2023-09-23T03:16:02.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.329+0000 7fe453fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4540b1b20 0x7fe4540c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:02.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.329+0000 7fe453fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4540b1b20 0x7fe4540c7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44848/0 (socket says 172.21.15.154:44848) 2023-09-23T03:16:02.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.329+0000 7fe453fff640 1 -- 172.21.15.154:0/1915541222 learned_addr learned my addr 172.21.15.154:0/1915541222 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:02.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.329+0000 7fe458933640 1 --2- 172.21.15.154:0/1915541222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4540acce0 0x7fe4541564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:02.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.329+0000 7fe459134640 1 --2- 172.21.15.154:0/1915541222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4540cee70 0x7fe45415b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:02.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.330+0000 7fe453fff640 1 -- 172.21.15.154:0/1915541222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4540cee70 msgr2=0x7fe45415b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:02.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.330+0000 7fe453fff640 1 --2- 172.21.15.154:0/1915541222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4540cee70 0x7fe45415b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:02.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.330+0000 7fe453fff640 1 -- 172.21.15.154:0/1915541222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4540acce0 msgr2=0x7fe4541564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:02.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.330+0000 7fe453fff640 1 --2- 172.21.15.154:0/1915541222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4540acce0 0x7fe4541564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:02.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.330+0000 7fe453fff640 1 -- 172.21.15.154:0/1915541222 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4480093a0 con 0x7fe4540b1b20 2023-09-23T03:16:02.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.330+0000 7fe453fff640 1 --2- 172.21.15.154:0/1915541222 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4540b1b20 0x7fe4540c7b90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe44000e9e0 tx=0x7fe44000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:02.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.330+0000 7fe4517fa640 1 -- 172.21.15.154:0/1915541222 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe440009670 con 0x7fe4540b1b20 2023-09-23T03:16:02.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.330+0000 7fe4517fa640 1 -- 172.21.15.154:0/1915541222 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe44000c9d0 con 0x7fe4540b1b20 2023-09-23T03:16:02.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.330+0000 7fe4517fa640 1 -- 172.21.15.154:0/1915541222 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44000cd80 con 0x7fe4540b1b20 2023-09-23T03:16:02.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.330+0000 7fe45ab97640 1 -- 172.21.15.154:0/1915541222 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4540c6af0 con 0x7fe4540b1b20 2023-09-23T03:16:02.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.330+0000 7fe45ab97640 1 -- 172.21.15.154:0/1915541222 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4540c70b0 con 0x7fe4540b1b20 2023-09-23T03:16:02.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.332+0000 7fe45ab97640 1 -- 172.21.15.154:0/1915541222 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe454073eb0 con 0x7fe4540b1b20 2023-09-23T03:16:02.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.332+0000 7fe4517fa640 1 -- 172.21.15.154:0/1915541222 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fe440013470 con 0x7fe4540b1b20 2023-09-23T03:16:02.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.333+0000 7fe4517fa640 1 --2- 172.21.15.154:0/1915541222 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe43806af30 0x7fe43806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:02.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.334+0000 7fe4517fa640 1 -- 172.21.15.154:0/1915541222 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6642+0+0 (secure 0 0 0) 0x7fe440089c50 con 0x7fe4540b1b20 2023-09-23T03:16:02.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.334+0000 7fe458933640 1 --2- 172.21.15.154:0/1915541222 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe43806af30 0x7fe43806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:02.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.334+0000 7fe458933640 1 --2- 172.21.15.154:0/1915541222 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe43806af30 0x7fe43806d3f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe448036000 tx=0x7fe44803d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:02.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.338+0000 7fe4517fa640 1 -- 172.21.15.154:0/1915541222 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fe440054980 con 0x7fe4540b1b20 2023-09-23T03:16:02.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.564+0000 7fe45ab97640 1 -- 172.21.15.154:0/1915541222 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4540ac850 con 0x7fe4540b1b20 2023-09-23T03:16:02.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.565+0000 7fe4517fa640 1 -- 172.21.15.154:0/1915541222 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4921 (secure 0 0 0) 0x7fe440017090 con 0x7fe4540b1b20 2023-09-23T03:16:02.567 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:16:02.567 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2644365591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2644365591},{"type":"v1","addr":"172.21.15.190:6835","nonce":2644365591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/4145467852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4145467852},{"type":"v1","addr":"172.21.15.154:6835","nonce":4145467852}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1022302774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1022302774},{"type":"v1","addr":"172.21.15.154:6837","nonce":1022302774}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:15:20.686035+0000","modified":"2023-09-23T03:15:48.742318+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/3561509423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3561509423},{"type":"v1","addr":"172.21.15.190:6834","nonce":3561509423}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T03:16:02.567 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 9 2023-09-23T03:16:02.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.568+0000 7fe45ab97640 1 -- 172.21.15.154:0/1915541222 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe43806af30 msgr2=0x7fe43806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:02.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.568+0000 7fe45ab97640 1 --2- 172.21.15.154:0/1915541222 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe43806af30 0x7fe43806d3f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe448036000 tx=0x7fe44803d040 comp rx=0 tx=0).stop 2023-09-23T03:16:02.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.568+0000 7fe45ab97640 1 -- 172.21.15.154:0/1915541222 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4540b1b20 msgr2=0x7fe4540c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:02.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.568+0000 7fe45ab97640 1 --2- 172.21.15.154:0/1915541222 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4540b1b20 0x7fe4540c7b90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe44000e9e0 tx=0x7fe44000eea0 comp rx=0 tx=0).stop 2023-09-23T03:16:02.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.568+0000 7fe45ab97640 1 -- 172.21.15.154:0/1915541222 shutdown_connections 2023-09-23T03:16:02.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.568+0000 7fe45ab97640 1 --2- 172.21.15.154:0/1915541222 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe43806af30 0x7fe43806d3f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:02.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.568+0000 7fe45ab97640 1 --2- 172.21.15.154:0/1915541222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4540cee70 0x7fe45415b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:02.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.568+0000 7fe45ab97640 1 --2- 172.21.15.154:0/1915541222 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4540b1b20 0x7fe4540c7b90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:02.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.568+0000 7fe45ab97640 1 --2- 172.21.15.154:0/1915541222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4540acce0 0x7fe4541564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:02.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.568+0000 7fe45ab97640 1 -- 172.21.15.154:0/1915541222 >> 172.21.15.154:0/1915541222 conn(0x7fe45406a0f0 msgr2=0x7fe4540cb670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:02.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.568+0000 7fe45ab97640 1 -- 172.21.15.154:0/1915541222 shutdown_connections 2023-09-23T03:16:02.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.568+0000 7fe45ab97640 1 -- 172.21.15.154:0/1915541222 wait complete. 2023-09-23T03:16:02.587 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:16:02.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.800+0000 7f64affff640 1 Processor -- start 2023-09-23T03:16:02.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.800+0000 7f64affff640 1 -- start start 2023-09-23T03:16:02.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.802+0000 7f64affff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f64b00cf500 0x7f64b00ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:02.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.802+0000 7f64affff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f64b015cc00 0x7f64b00ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:02.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.802+0000 7f64affff640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64b00d17f0 con 0x7f64b00cf8e0 2023-09-23T03:16:02.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.802+0000 7f64affff640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64b006be10 con 0x7f64b015cc00 2023-09-23T03:16:02.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.802+0000 7f64affff640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64b00d0470 con 0x7f64b00cf500 2023-09-23T03:16:02.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.802+0000 7f64af7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f64b015cc00 0x7f64b00ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:02.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.802+0000 7f64ae7fc640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f64b00cf8e0 0x7f64b015c4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46342/0 (socket says 172.21.15.154:46342) 2023-09-23T03:16:02.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.803+0000 7f64ae7fc640 1 -- 172.21.15.154:0/665930265 learned_addr learned my addr 172.21.15.154:0/665930265 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:02.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.803+0000 7f64aeffd640 1 --2- 172.21.15.154:0/665930265 >> v2:172.21.15.190:3300/0 conn(0x7f64b00cf500 0x7f64b00ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:02.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.803+0000 7f64af7fe640 1 -- 172.21.15.154:0/665930265 >> v2:172.21.15.190:3300/0 conn(0x7f64b00cf500 msgr2=0x7f64b00ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:02.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.803+0000 7f64af7fe640 1 --2- 172.21.15.154:0/665930265 >> v2:172.21.15.190:3300/0 conn(0x7f64b00cf500 0x7f64b00ce050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:02.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.803+0000 7f64af7fe640 1 -- 172.21.15.154:0/665930265 >> v1:172.21.15.154:6789/0 conn(0x7f64b00cf8e0 legacy=0x7f64b015c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:02.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.803+0000 7f64af7fe640 1 -- 172.21.15.154:0/665930265 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b00d0750 con 0x7f64b015cc00 2023-09-23T03:16:02.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.803+0000 7f64adffb640 1 -- 172.21.15.154:0/665930265 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3469940946 0 0) 0x7f64b00d17f0 con 0x7f64b00cf8e0 2023-09-23T03:16:02.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.803+0000 7f64aeffd640 1 --2- 172.21.15.154:0/665930265 >> v2:172.21.15.190:3300/0 conn(0x7f64b00cf500 0x7f64b00ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:16:02.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.804+0000 7f64af7fe640 1 --2- 172.21.15.154:0/665930265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64b015cc00 0x7f64b00ce590 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f64a8009d90 tx=0x7f64a802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adc4b0b1e148e8e3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:02.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.805+0000 7f64adffb640 1 -- 172.21.15.154:0/665930265 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a802cbd0 con 0x7f64b015cc00 2023-09-23T03:16:02.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.805+0000 7f64adffb640 1 -- 172.21.15.154:0/665930265 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a802cd30 con 0x7f64b015cc00 2023-09-23T03:16:02.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.805+0000 7f64adffb640 1 -- 172.21.15.154:0/665930265 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a802cea0 con 0x7f64b015cc00 2023-09-23T03:16:02.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.805+0000 7f64affff640 1 -- 172.21.15.154:0/665930265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64b015cc00 msgr2=0x7f64b00ce590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:02.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.805+0000 7f64affff640 1 --2- 172.21.15.154:0/665930265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64b015cc00 0x7f64b00ce590 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f64a8009d90 tx=0x7f64a802be20 comp rx=0 tx=0).stop 2023-09-23T03:16:02.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.805+0000 7f64affff640 1 -- 172.21.15.154:0/665930265 shutdown_connections 2023-09-23T03:16:02.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.805+0000 7f64affff640 1 --2- 172.21.15.154:0/665930265 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64b015cc00 0x7f64b00ce590 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:02.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.805+0000 7f64affff640 1 --2- 172.21.15.154:0/665930265 >> v2:172.21.15.190:3300/0 conn(0x7f64b00cf500 0x7f64b00ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:02.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.805+0000 7f64affff640 1 -- 172.21.15.154:0/665930265 >> 172.21.15.154:0/665930265 conn(0x7f64b009bbb0 msgr2=0x7f64b00d28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:02.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.805+0000 7f64affff640 1 -- 172.21.15.154:0/665930265 shutdown_connections 2023-09-23T03:16:02.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.805+0000 7f64affff640 1 -- 172.21.15.154:0/665930265 wait complete. 2023-09-23T03:16:02.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.807+0000 7f64affff640 1 Processor -- start 2023-09-23T03:16:02.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.807+0000 7f64affff640 1 -- start start 2023-09-23T03:16:02.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.808+0000 7f64affff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64b00cf500 0x7f64b00c53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:02.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.808+0000 7f64affff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64b00cf8e0 0x7f64b01f1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:02.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.808+0000 7f64affff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64b015cc00 0x7f64b01474c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:02.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.808+0000 7f64affff640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b00c85c0 con 0x7f64b00cf8e0 2023-09-23T03:16:02.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.808+0000 7f64affff640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b01f2910 con 0x7f64b00cf500 2023-09-23T03:16:02.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.808+0000 7f64affff640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64b01f2bf0 con 0x7f64b015cc00 2023-09-23T03:16:02.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.809+0000 7f64ae7fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64b00cf8e0 0x7f64b01f1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:02.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.809+0000 7f64ae7fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64b00cf8e0 0x7f64b01f1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44860/0 (socket says 172.21.15.154:44860) 2023-09-23T03:16:02.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.809+0000 7f64ae7fc640 1 -- 172.21.15.154:0/1383625787 learned_addr learned my addr 172.21.15.154:0/1383625787 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:02.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.809+0000 7f64aeffd640 1 --2- 172.21.15.154:0/1383625787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64b00cf500 0x7f64b00c53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:02.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.809+0000 7f64af7fe640 1 --2- 172.21.15.154:0/1383625787 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64b015cc00 0x7f64b01474c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:02.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.809+0000 7f64ae7fc640 1 -- 172.21.15.154:0/1383625787 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64b015cc00 msgr2=0x7f64b01474c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:02.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.809+0000 7f64ae7fc640 1 --2- 172.21.15.154:0/1383625787 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64b015cc00 0x7f64b01474c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:02.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.809+0000 7f64ae7fc640 1 -- 172.21.15.154:0/1383625787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64b00cf500 msgr2=0x7f64b00c53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:02.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.809+0000 7f64ae7fc640 1 --2- 172.21.15.154:0/1383625787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64b00cf500 0x7f64b00c53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:02.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.809+0000 7f64ae7fc640 1 -- 172.21.15.154:0/1383625787 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a8009a70 con 0x7f64b00cf8e0 2023-09-23T03:16:02.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.809+0000 7f64af7fe640 1 --2- 172.21.15.154:0/1383625787 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64b015cc00 0x7f64b01474c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:02.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.809+0000 7f64ae7fc640 1 --2- 172.21.15.154:0/1383625787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64b00cf8e0 0x7f64b01f1f60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f64a400c4d0 tx=0x7f64a400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:02.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.810+0000 7f64977fe640 1 -- 172.21.15.154:0/1383625787 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a40074e0 con 0x7f64b00cf8e0 2023-09-23T03:16:02.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.810+0000 7f64977fe640 1 -- 172.21.15.154:0/1383625787 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a4007660 con 0x7f64b00cf8e0 2023-09-23T03:16:02.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.810+0000 7f64977fe640 1 -- 172.21.15.154:0/1383625787 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a4004a20 con 0x7f64b00cf8e0 2023-09-23T03:16:02.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.810+0000 7f64affff640 1 -- 172.21.15.154:0/1383625787 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64b01f2ed0 con 0x7f64b00cf8e0 2023-09-23T03:16:02.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.810+0000 7f64affff640 1 -- 172.21.15.154:0/1383625787 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64b00c5930 con 0x7f64b00cf8e0 2023-09-23T03:16:02.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.811+0000 7f64affff640 1 -- 172.21.15.154:0/1383625787 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64b00c8040 con 0x7f64b00cf8e0 2023-09-23T03:16:02.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.812+0000 7f64977fe640 1 -- 172.21.15.154:0/1383625787 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f64a4004b80 con 0x7f64b00cf8e0 2023-09-23T03:16:02.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.813+0000 7f64977fe640 1 --2- 172.21.15.154:0/1383625787 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f649c06af30 0x7f649c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:02.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.813+0000 7f64977fe640 1 -- 172.21.15.154:0/1383625787 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f64a4088cd0 con 0x7f64b00cf8e0 2023-09-23T03:16:02.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.813+0000 7f64aeffd640 1 --2- 172.21.15.154:0/1383625787 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f649c06af30 0x7f649c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:02.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.814+0000 7f64aeffd640 1 --2- 172.21.15.154:0/1383625787 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f649c06af30 0x7f649c06d3f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f64b015b950 tx=0x7f64a0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:02.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:02.817+0000 7f64977fe640 1 -- 172.21.15.154:0/1383625787 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f64b00c8040 con 0x7f64b00cf8e0 2023-09-23T03:16:03.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.007+0000 7f64affff640 1 -- 172.21.15.154:0/1383625787 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64b00c8700 con 0x7f64b00cf8e0 2023-09-23T03:16:03.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.010+0000 7f64977fe640 1 -- 172.21.15.154:0/1383625787 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18744 (secure 0 0 0) 0x7f64b00c8700 con 0x7f64b00cf8e0 2023-09-23T03:16:03.012 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:16:03.012 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":25,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:16:02.143126+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:15:18.575849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T03:15:19.707212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T03:15:23.337138+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:16:03.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.013+0000 7f64affff640 1 -- 172.21.15.154:0/1383625787 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f649c06af30 msgr2=0x7f649c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:03.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.013+0000 7f64affff640 1 --2- 172.21.15.154:0/1383625787 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f649c06af30 0x7f649c06d3f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f64b015b950 tx=0x7f64a0008040 comp rx=0 tx=0).stop 2023-09-23T03:16:03.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.013+0000 7f64affff640 1 -- 172.21.15.154:0/1383625787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64b00cf8e0 msgr2=0x7f64b01f1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:03.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.013+0000 7f64affff640 1 --2- 172.21.15.154:0/1383625787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64b00cf8e0 0x7f64b01f1f60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f64a400c4d0 tx=0x7f64a400c990 comp rx=0 tx=0).stop 2023-09-23T03:16:03.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.014+0000 7f64affff640 1 -- 172.21.15.154:0/1383625787 shutdown_connections 2023-09-23T03:16:03.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.014+0000 7f64affff640 1 --2- 172.21.15.154:0/1383625787 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f649c06af30 0x7f649c06d3f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:03.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.014+0000 7f64affff640 1 --2- 172.21.15.154:0/1383625787 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64b015cc00 0x7f64b01474c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:03.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.014+0000 7f64affff640 1 --2- 172.21.15.154:0/1383625787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64b00cf8e0 0x7f64b01f1f60 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:03.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.014+0000 7f64affff640 1 --2- 172.21.15.154:0/1383625787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64b00cf500 0x7f64b00c53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:03.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.014+0000 7f64affff640 1 -- 172.21.15.154:0/1383625787 >> 172.21.15.154:0/1383625787 conn(0x7f64b009bbb0 msgr2=0x7f64b00b5550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:03.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.014+0000 7f64affff640 1 -- 172.21.15.154:0/1383625787 shutdown_connections 2023-09-23T03:16:03.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.014+0000 7f64affff640 1 -- 172.21.15.154:0/1383625787 wait complete. 2023-09-23T03:16:03.034 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:16:03.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.252+0000 7fa7960db640 1 Processor -- start 2023-09-23T03:16:03.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.252+0000 7fa7960db640 1 -- start start 2023-09-23T03:16:03.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.253+0000 7fa7960db640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa7900acce0 0x7fa79015ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:03.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.253+0000 7fa7960db640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7900b1b20 0x7fa7900ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:03.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.253+0000 7fa7960db640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa79015ac30 con 0x7fa7900cee70 2023-09-23T03:16:03.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.253+0000 7fa7960db640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7900b2e90 con 0x7fa7900acce0 2023-09-23T03:16:03.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.253+0000 7fa7960db640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7900cf320 con 0x7fa7900b1b20 2023-09-23T03:16:03.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.254+0000 7fa78f7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa7900acce0 0x7fa79015ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:03.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.254+0000 7fa78f7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa7900acce0 0x7fa79015ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44868/0 (socket says 172.21.15.154:44868) 2023-09-23T03:16:03.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.254+0000 7fa78f7fe640 1 -- 172.21.15.154:0/1573099157 learned_addr learned my addr 172.21.15.154:0/1573099157 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:03.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.254+0000 7fa78effd640 1 --2- 172.21.15.154:0/1573099157 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7900b1b20 0x7fa7900ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:03.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.254+0000 7fa78f7fe640 1 -- 172.21.15.154:0/1573099157 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7900b1b20 msgr2=0x7fa7900ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:03.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.254+0000 7fa78f7fe640 1 --2- 172.21.15.154:0/1573099157 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7900b1b20 0x7fa7900ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:03.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.254+0000 7fa78f7fe640 1 -- 172.21.15.154:0/1573099157 >> v1:172.21.15.190:6789/0 conn(0x7fa7900cee70 legacy=0x7fa79006cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:03.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.254+0000 7fa78f7fe640 1 -- 172.21.15.154:0/1573099157 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7900b1f70 con 0x7fa7900acce0 2023-09-23T03:16:03.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.255+0000 7fa78f7fe640 1 --2- 172.21.15.154:0/1573099157 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7900acce0 0x7fa79015ed60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa780009850 tx=0x7fa78002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72f458ba4448e28 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:03.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.256+0000 7fa78e7fc640 1 -- 172.21.15.154:0/1573099157 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa780004230 con 0x7fa7900acce0 2023-09-23T03:16:03.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.256+0000 7fa78e7fc640 1 -- 172.21.15.154:0/1573099157 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa780004390 con 0x7fa7900acce0 2023-09-23T03:16:03.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.256+0000 7fa78e7fc640 1 -- 172.21.15.154:0/1573099157 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa780031830 con 0x7fa7900acce0 2023-09-23T03:16:03.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.256+0000 7fa7960db640 1 -- 172.21.15.154:0/1573099157 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7900acce0 msgr2=0x7fa79015ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:03.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.256+0000 7fa7960db640 1 --2- 172.21.15.154:0/1573099157 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7900acce0 0x7fa79015ed60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa780009850 tx=0x7fa78002b8d0 comp rx=0 tx=0).stop 2023-09-23T03:16:03.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.256+0000 7fa7960db640 1 -- 172.21.15.154:0/1573099157 shutdown_connections 2023-09-23T03:16:03.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.257+0000 7fa7960db640 1 --2- 172.21.15.154:0/1573099157 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7900b1b20 0x7fa7900ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:03.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.257+0000 7fa7960db640 1 --2- 172.21.15.154:0/1573099157 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7900acce0 0x7fa79015ed60 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:03.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.257+0000 7fa7960db640 1 -- 172.21.15.154:0/1573099157 >> 172.21.15.154:0/1573099157 conn(0x7fa79006a0f0 msgr2=0x7fa79006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:03.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.257+0000 7fa7960db640 1 -- 172.21.15.154:0/1573099157 shutdown_connections 2023-09-23T03:16:03.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.257+0000 7fa7960db640 1 -- 172.21.15.154:0/1573099157 wait complete. 2023-09-23T03:16:03.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.258+0000 7fa7960db640 1 Processor -- start 2023-09-23T03:16:03.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.258+0000 7fa7960db640 1 -- start start 2023-09-23T03:16:03.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.259+0000 7fa7960db640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7900acce0 0x7fa7901f6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:03.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.259+0000 7fa7960db640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7900b1b20 0x7fa79015b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:03.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.259+0000 7fa7960db640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7900cee70 0x7fa7901f1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:03.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.259+0000 7fa7960db640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7901f58c0 con 0x7fa7900acce0 2023-09-23T03:16:03.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.259+0000 7fa7960db640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79015c030 con 0x7fa7900cee70 2023-09-23T03:16:03.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.259+0000 7fa7960db640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa79015c310 con 0x7fa7900b1b20 2023-09-23T03:16:03.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.260+0000 7fa78f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7900acce0 0x7fa7901f6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:03.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.260+0000 7fa78f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7900acce0 0x7fa7901f6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44880/0 (socket says 172.21.15.154:44880) 2023-09-23T03:16:03.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.260+0000 7fa78f7fe640 1 -- 172.21.15.154:0/3991118795 learned_addr learned my addr 172.21.15.154:0/3991118795 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:03.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.260+0000 7fa78f7fe640 1 -- 172.21.15.154:0/3991118795 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7900b1b20 msgr2=0x7fa79015b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:03.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.260+0000 7fa78f7fe640 1 --2- 172.21.15.154:0/3991118795 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7900b1b20 0x7fa79015b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:03.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.260+0000 7fa78f7fe640 1 -- 172.21.15.154:0/3991118795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7900cee70 msgr2=0x7fa7901f1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:03.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.260+0000 7fa78f7fe640 1 --2- 172.21.15.154:0/3991118795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7900cee70 0x7fa7901f1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:03.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.260+0000 7fa78f7fe640 1 -- 172.21.15.154:0/3991118795 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa780009460 con 0x7fa7900acce0 2023-09-23T03:16:03.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.260+0000 7fa78f7fe640 1 --2- 172.21.15.154:0/3991118795 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7900acce0 0x7fa7901f6310 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa78002b910 tx=0x7fa7800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:03.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.261+0000 7fa777fff640 1 -- 172.21.15.154:0/3991118795 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa780031830 con 0x7fa7900acce0 2023-09-23T03:16:03.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.261+0000 7fa777fff640 1 -- 172.21.15.154:0/3991118795 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa78003f070 con 0x7fa7900acce0 2023-09-23T03:16:03.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.261+0000 7fa7960db640 1 -- 172.21.15.154:0/3991118795 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7901f5b10 con 0x7fa7900acce0 2023-09-23T03:16:03.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.261+0000 7fa777fff640 1 -- 172.21.15.154:0/3991118795 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa780031bd0 con 0x7fa7900acce0 2023-09-23T03:16:03.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.261+0000 7fa7960db640 1 -- 172.21.15.154:0/3991118795 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa79015bcb0 con 0x7fa7900acce0 2023-09-23T03:16:03.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.262+0000 7fa7960db640 1 -- 172.21.15.154:0/3991118795 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7901f6030 con 0x7fa7900acce0 2023-09-23T03:16:03.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.263+0000 7fa777fff640 1 -- 172.21.15.154:0/3991118795 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fa780041030 con 0x7fa7900acce0 2023-09-23T03:16:03.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.264+0000 7fa777fff640 1 --2- 172.21.15.154:0/3991118795 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa77c06af30 0x7fa77c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:03.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.265+0000 7fa777fff640 1 -- 172.21.15.154:0/3991118795 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7fa7800abd20 con 0x7fa7900acce0 2023-09-23T03:16:03.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.265+0000 7fa78effd640 1 --2- 172.21.15.154:0/3991118795 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa77c06af30 0x7fa77c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:03.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.265+0000 7fa78effd640 1 --2- 172.21.15.154:0/3991118795 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa77c06af30 0x7fa77c06d3f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa7900c1f50 tx=0x7fa784008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:03.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.269+0000 7fa777fff640 1 -- 172.21.15.154:0/3991118795 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fa7901f6030 con 0x7fa7900acce0 2023-09-23T03:16:03.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.503+0000 7fa7960db640 1 -- 172.21.15.154:0/3991118795 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7901f6030 con 0x7fa7900acce0 2023-09-23T03:16:03.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.505+0000 7fa777fff640 1 -- 172.21.15.154:0/3991118795 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4921 (secure 0 0 0) 0x7fa7901f6030 con 0x7fa7900acce0 2023-09-23T03:16:03.506 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:16:03.506 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2644365591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2644365591},{"type":"v1","addr":"172.21.15.190:6835","nonce":2644365591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/4145467852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4145467852},{"type":"v1","addr":"172.21.15.154:6835","nonce":4145467852}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1022302774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1022302774},{"type":"v1","addr":"172.21.15.154:6837","nonce":1022302774}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:15:20.686035+0000","modified":"2023-09-23T03:15:48.742318+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/3561509423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3561509423},{"type":"v1","addr":"172.21.15.190:6834","nonce":3561509423}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T03:16:03.507 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 9 2023-09-23T03:16:03.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.507+0000 7fa7960db640 1 -- 172.21.15.154:0/3991118795 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa77c06af30 msgr2=0x7fa77c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:03.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.507+0000 7fa7960db640 1 --2- 172.21.15.154:0/3991118795 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa77c06af30 0x7fa77c06d3f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa7900c1f50 tx=0x7fa784008040 comp rx=0 tx=0).stop 2023-09-23T03:16:03.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.507+0000 7fa7960db640 1 -- 172.21.15.154:0/3991118795 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7900acce0 msgr2=0x7fa7901f6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:03.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.507+0000 7fa7960db640 1 --2- 172.21.15.154:0/3991118795 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7900acce0 0x7fa7901f6310 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa78002b910 tx=0x7fa7800046c0 comp rx=0 tx=0).stop 2023-09-23T03:16:03.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.507+0000 7fa7960db640 1 -- 172.21.15.154:0/3991118795 shutdown_connections 2023-09-23T03:16:03.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.507+0000 7fa7960db640 1 --2- 172.21.15.154:0/3991118795 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa77c06af30 0x7fa77c06d3f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:03.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.507+0000 7fa7960db640 1 --2- 172.21.15.154:0/3991118795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7900cee70 0x7fa7901f1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:03.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.507+0000 7fa7960db640 1 --2- 172.21.15.154:0/3991118795 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa7900b1b20 0x7fa79015b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:03.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.507+0000 7fa7960db640 1 --2- 172.21.15.154:0/3991118795 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7900acce0 0x7fa7901f6310 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:03.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.508+0000 7fa7960db640 1 -- 172.21.15.154:0/3991118795 >> 172.21.15.154:0/3991118795 conn(0x7fa79006a0f0 msgr2=0x7fa7900726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:03.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.508+0000 7fa7960db640 1 -- 172.21.15.154:0/3991118795 shutdown_connections 2023-09-23T03:16:03.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:03.508+0000 7fa7960db640 1 -- 172.21.15.154:0/3991118795 wait complete. 2023-09-23T03:16:03.927 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-23T03:15:20.686035+0000', 'modified': '2023-09-23T03:15:48.742318+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4285}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4285': {'gid': 4285, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.190:6834/3561509423', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6832', 'nonce': 3561509423}, {'type': 'v1', 'addr': '172.21.15.190:6834', 'nonce': 3561509423}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-23T03:16:03.928 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-23T03:16:03.928 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-09-23T03:16:04.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.154+0000 7f17552a2640 1 Processor -- start 2023-09-23T03:16:04.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.154+0000 7f17552a2640 1 -- start start 2023-09-23T03:16:04.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.155+0000 7f17552a2640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f175015caf0 0x7f17500ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:04.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.156+0000 7f17552a2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17500cee70 0x7f1750160880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:04.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.156+0000 7f17552a2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1750160dc0 0x7f175006cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:04.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.156+0000 7f17552a2640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f175015ced0 con 0x7f1750160dc0 2023-09-23T03:16:04.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.156+0000 7f17552a2640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17500cf2f0 con 0x7f175015caf0 2023-09-23T03:16:04.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.156+0000 7f17552a2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17500ce2f0 con 0x7f17500cee70 2023-09-23T03:16:04.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.156+0000 7f174f7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1750160dc0 0x7f175006cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:04.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.156+0000 7f174f7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1750160dc0 0x7f175006cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59860/0 (socket says 172.21.15.154:59860) 2023-09-23T03:16:04.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.156+0000 7f174f7fe640 1 -- 172.21.15.154:0/2157010105 learned_addr learned my addr 172.21.15.154:0/2157010105 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:04.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.156+0000 7f174effd640 1 --2- 172.21.15.154:0/2157010105 >> v2:172.21.15.190:3300/0 conn(0x7f175015caf0 0x7f17500ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:04.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.156+0000 7f174e7fc640 1 --2- 172.21.15.154:0/2157010105 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17500cee70 0x7f1750160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:04.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.156+0000 7f174f7fe640 1 -- 172.21.15.154:0/2157010105 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17500cee70 msgr2=0x7f1750160880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:04.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.156+0000 7f174f7fe640 1 --2- 172.21.15.154:0/2157010105 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17500cee70 0x7f1750160880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.156+0000 7f174f7fe640 1 -- 172.21.15.154:0/2157010105 >> v2:172.21.15.190:3300/0 conn(0x7f175015caf0 msgr2=0x7f17500ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:04.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.156+0000 7f174f7fe640 1 --2- 172.21.15.154:0/2157010105 >> v2:172.21.15.190:3300/0 conn(0x7f175015caf0 0x7f17500ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.156+0000 7f174f7fe640 1 -- 172.21.15.154:0/2157010105 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17500cd530 con 0x7f1750160dc0 2023-09-23T03:16:04.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.158+0000 7f174f7fe640 1 --2- 172.21.15.154:0/2157010105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1750160dc0 0x7f175006cff0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f17500ce470 tx=0x7f174802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=394dc8a7de2ab6a4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:04.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.158+0000 7f174dffb640 1 -- 172.21.15.154:0/2157010105 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174802ca70 con 0x7f1750160dc0 2023-09-23T03:16:04.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.158+0000 7f174dffb640 1 -- 172.21.15.154:0/2157010105 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f174802cbd0 con 0x7f1750160dc0 2023-09-23T03:16:04.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.158+0000 7f174dffb640 1 -- 172.21.15.154:0/2157010105 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17480343f0 con 0x7f1750160dc0 2023-09-23T03:16:04.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.159+0000 7f17552a2640 1 -- 172.21.15.154:0/2157010105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1750160dc0 msgr2=0x7f175006cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:04.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.159+0000 7f17552a2640 1 --2- 172.21.15.154:0/2157010105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1750160dc0 0x7f175006cff0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f17500ce470 tx=0x7f174802bed0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.159+0000 7f17552a2640 1 -- 172.21.15.154:0/2157010105 shutdown_connections 2023-09-23T03:16:04.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.159+0000 7f17552a2640 1 --2- 172.21.15.154:0/2157010105 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1750160dc0 0x7f175006cff0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.159+0000 7f17552a2640 1 --2- 172.21.15.154:0/2157010105 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17500cee70 0x7f1750160880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.159+0000 7f17552a2640 1 --2- 172.21.15.154:0/2157010105 >> v2:172.21.15.190:3300/0 conn(0x7f175015caf0 0x7f17500ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.159+0000 7f17552a2640 1 -- 172.21.15.154:0/2157010105 >> 172.21.15.154:0/2157010105 conn(0x7f175006a0f0 msgr2=0x7f175006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:04.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.159+0000 7f17552a2640 1 -- 172.21.15.154:0/2157010105 shutdown_connections 2023-09-23T03:16:04.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.159+0000 7f17552a2640 1 -- 172.21.15.154:0/2157010105 wait complete. 2023-09-23T03:16:04.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.161+0000 7f17552a2640 1 Processor -- start 2023-09-23T03:16:04.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.161+0000 7f17552a2640 1 -- start start 2023-09-23T03:16:04.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.162+0000 7f17552a2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17500cee70 0x7f1750098530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:04.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.162+0000 7f17552a2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f175015caf0 0x7f175015b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:04.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.162+0000 7f17552a2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1750160dc0 0x7f175015b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:04.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.162+0000 7f17552a2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175009c350 con 0x7f1750160dc0 2023-09-23T03:16:04.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.162+0000 7f17552a2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175009c070 con 0x7f175015caf0 2023-09-23T03:16:04.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.162+0000 7f17552a2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f175009b040 con 0x7f17500cee70 2023-09-23T03:16:04.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.162+0000 7f174f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1750160dc0 0x7f175015b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:04.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.162+0000 7f174f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1750160dc0 0x7f175015b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59872/0 (socket says 172.21.15.154:59872) 2023-09-23T03:16:04.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.162+0000 7f174f7fe640 1 -- 172.21.15.154:0/1740351309 learned_addr learned my addr 172.21.15.154:0/1740351309 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:04.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.162+0000 7f174effd640 1 --2- 172.21.15.154:0/1740351309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17500cee70 0x7f1750098530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:04.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.162+0000 7f174e7fc640 1 --2- 172.21.15.154:0/1740351309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f175015caf0 0x7f175015b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:04.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.163+0000 7f174f7fe640 1 -- 172.21.15.154:0/1740351309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17500cee70 msgr2=0x7f1750098530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:04.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.163+0000 7f174f7fe640 1 --2- 172.21.15.154:0/1740351309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17500cee70 0x7f1750098530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.163+0000 7f174f7fe640 1 -- 172.21.15.154:0/1740351309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f175015caf0 msgr2=0x7f175015b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:04.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.163+0000 7f174f7fe640 1 --2- 172.21.15.154:0/1740351309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f175015caf0 0x7f175015b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.163+0000 7f174f7fe640 1 -- 172.21.15.154:0/1740351309 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1748009b30 con 0x7f1750160dc0 2023-09-23T03:16:04.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.163+0000 7f174f7fe640 1 --2- 172.21.15.154:0/1740351309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1750160dc0 0x7f175015b7d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1748036040 tx=0x7f1748004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:04.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.163+0000 7f17377fe640 1 -- 172.21.15.154:0/1740351309 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1748009510 con 0x7f1750160dc0 2023-09-23T03:16:04.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.163+0000 7f17377fe640 1 -- 172.21.15.154:0/1740351309 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f174803e450 con 0x7f1750160dc0 2023-09-23T03:16:04.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.163+0000 7f17377fe640 1 -- 172.21.15.154:0/1740351309 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174803e7e0 con 0x7f1750160dc0 2023-09-23T03:16:04.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.163+0000 7f17552a2640 1 -- 172.21.15.154:0/1740351309 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f175009b320 con 0x7f1750160dc0 2023-09-23T03:16:04.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.163+0000 7f17552a2640 1 -- 172.21.15.154:0/1740351309 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1750098c50 con 0x7f1750160dc0 2023-09-23T03:16:04.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.166+0000 7f17377fe640 1 -- 172.21.15.154:0/1740351309 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f174803e940 con 0x7f1750160dc0 2023-09-23T03:16:04.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.167+0000 7f17377fe640 1 --2- 172.21.15.154:0/1740351309 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f173c06af30 0x7f173c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:04.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.167+0000 7f17377fe640 1 -- 172.21.15.154:0/1740351309 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f17480ac880 con 0x7f1750160dc0 2023-09-23T03:16:04.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.168+0000 7f174effd640 1 --2- 172.21.15.154:0/1740351309 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f173c06af30 0x7f173c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:04.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.168+0000 7f1757e35740 1 Processor -- start 2023-09-23T03:16:04.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.168+0000 7f1757e35740 1 -- start start 2023-09-23T03:16:04.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.168+0000 7f174effd640 1 --2- 172.21.15.154:0/1740351309 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f173c06af30 0x7f173c06d3f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f174000b9b0 tx=0x7f1740008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:04.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.169+0000 7f1757e35740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558cee822c00 0x558cee7f5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:04.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.169+0000 7f1757e35740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558cee822fe0 0x558cee825990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:04.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.169+0000 7f1757e35740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558cee7f6040 0x558cee825ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:04.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.169+0000 7f1757e35740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cee7696d0 con 0x558cee822c00 2023-09-23T03:16:04.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.169+0000 7f1757e35740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cee766dc0 con 0x558cee7f6040 2023-09-23T03:16:04.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.169+0000 7f1757e35740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x558cee7626b0 con 0x558cee822fe0 2023-09-23T03:16:04.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.169+0000 7f174effd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558cee822c00 0x558cee7f5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:04.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.169+0000 7f174effd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558cee822c00 0x558cee7f5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59882/0 (socket says 172.21.15.154:59882) 2023-09-23T03:16:04.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.169+0000 7f174effd640 1 -- 172.21.15.154:0/1052900908 learned_addr learned my addr 172.21.15.154:0/1052900908 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:04.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.170+0000 7f174effd640 1 -- 172.21.15.154:0/1052900908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558cee822fe0 msgr2=0x558cee825990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:04.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.170+0000 7f174effd640 1 --2- 172.21.15.154:0/1052900908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558cee822fe0 0x558cee825990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.170+0000 7f174effd640 1 -- 172.21.15.154:0/1052900908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558cee7f6040 msgr2=0x558cee825ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:04.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.170+0000 7f174effd640 1 --2- 172.21.15.154:0/1052900908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558cee7f6040 0x558cee825ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.170+0000 7f174effd640 1 -- 172.21.15.154:0/1052900908 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cee7268e0 con 0x558cee822c00 2023-09-23T03:16:04.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.170+0000 7f174effd640 1 --2- 172.21.15.154:0/1052900908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558cee822c00 0x558cee7f5b00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f174000fa90 tx=0x7f1740012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d91c3984339adcfe server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:04.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.170+0000 7f1734ff9640 1 -- 172.21.15.154:0/1052900908 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1740014be0 con 0x558cee822c00 2023-09-23T03:16:04.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.170+0000 7f1734ff9640 1 -- 172.21.15.154:0/1052900908 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1740019070 con 0x558cee822c00 2023-09-23T03:16:04.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.170+0000 7f1734ff9640 1 -- 172.21.15.154:0/1052900908 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1740020440 con 0x558cee822c00 2023-09-23T03:16:04.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.171+0000 7f1757e35740 1 -- 172.21.15.154:0/1052900908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558cee822c00 msgr2=0x558cee7f5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:04.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.171+0000 7f1757e35740 1 --2- 172.21.15.154:0/1052900908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558cee822c00 0x558cee7f5b00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f174000fa90 tx=0x7f1740012000 comp rx=0 tx=0).stop 2023-09-23T03:16:04.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.171+0000 7f1757e35740 1 -- 172.21.15.154:0/1052900908 shutdown_connections 2023-09-23T03:16:04.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.171+0000 7f1757e35740 1 --2- 172.21.15.154:0/1052900908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558cee7f6040 0x558cee825ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.171+0000 7f1757e35740 1 --2- 172.21.15.154:0/1052900908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558cee822fe0 0x558cee825990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.171+0000 7f1757e35740 1 --2- 172.21.15.154:0/1052900908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558cee822c00 0x558cee7f5b00 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.171+0000 7f1757e35740 1 -- 172.21.15.154:0/1052900908 >> 172.21.15.154:0/1052900908 conn(0x558cee7f7e10 msgr2=0x558cee840990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:04.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.171+0000 7f1757e35740 1 -- 172.21.15.154:0/1052900908 shutdown_connections 2023-09-23T03:16:04.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.171+0000 7f1757e35740 1 -- 172.21.15.154:0/1052900908 wait complete. 2023-09-23T03:16:04.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.172+0000 7f1757e35740 1 -- start start 2023-09-23T03:16:04.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.172+0000 7f1757e35740 1 Processor -- start 2023-09-23T03:16:04.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.173+0000 7f1757e35740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558cee822c00 0x558cee850660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:04.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.173+0000 7f1757e35740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558cee822fe0 0x558cee850ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:04.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.173+0000 7f1757e35740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558cee838340 0x558cee8387c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:04.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.173+0000 7f1757e35740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cee700940 con 0x558cee838340 2023-09-23T03:16:04.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.173+0000 7f1757e35740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cee7005e0 con 0x558cee822fe0 2023-09-23T03:16:04.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.173+0000 7f1757e35740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x558cee4c6f20 con 0x558cee822c00 2023-09-23T03:16:04.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.173+0000 7f1757e35740 10 client.0 _ll_register_callbacks cb 0x558cee73e5e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:16:04.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.173+0000 7f174f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558cee838340 0x558cee8387c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:04.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.173+0000 7f174f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558cee838340 0x558cee8387c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59898/0 (socket says 172.21.15.154:59898) 2023-09-23T03:16:04.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.173+0000 7f174f7fe640 1 -- 172.21.15.154:0/2829465184 learned_addr learned my addr 172.21.15.154:0/2829465184 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:04.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.174+0000 7f174effd640 1 --2- 172.21.15.154:0/2829465184 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558cee822c00 0x558cee850660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:04.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.174+0000 7f174f7fe640 1 -- 172.21.15.154:0/2829465184 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558cee822c00 msgr2=0x558cee850660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:04.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.174+0000 7f174e7fc640 1 --2- 172.21.15.154:0/2829465184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558cee822fe0 0x558cee850ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:04.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.174+0000 7f174f7fe640 1 --2- 172.21.15.154:0/2829465184 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558cee822c00 0x558cee850660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.174+0000 7f174f7fe640 1 -- 172.21.15.154:0/2829465184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558cee822fe0 msgr2=0x558cee850ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:04.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.174+0000 7f174f7fe640 1 --2- 172.21.15.154:0/2829465184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558cee822fe0 0x558cee850ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.174+0000 7f174f7fe640 1 -- 172.21.15.154:0/2829465184 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f174000e040 con 0x558cee838340 2023-09-23T03:16:04.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.174+0000 7f174effd640 1 --2- 172.21.15.154:0/2829465184 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558cee822c00 0x558cee850660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:04.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.174+0000 7f174f7fe640 1 --2- 172.21.15.154:0/2829465184 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558cee838340 0x558cee8387c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1748082f00 tx=0x7f174807ab50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c8aacded68cc673 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:04.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.174+0000 7f172dffb640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:16:04.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.174+0000 7f172dffb640 1 -- 172.21.15.154:0/2829465184 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174807a820 con 0x558cee838340 2023-09-23T03:16:04.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.174+0000 7f1757e35740 1 -- 172.21.15.154:0/2829465184 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558cee75b690 con 0x558cee838340 2023-09-23T03:16:04.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.174+0000 7f172dffb640 1 -- 172.21.15.154:0/2829465184 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f174807f280 con 0x558cee838340 2023-09-23T03:16:04.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.174+0000 7f172dffb640 1 -- 172.21.15.154:0/2829465184 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1748083430 con 0x558cee838340 2023-09-23T03:16:04.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.174+0000 7f1757e35740 10 client.4806 fetch_fsmap learned FSMap version 9 2023-09-23T03:16:04.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.174+0000 7f1757e35740 1 -- 172.21.15.154:0/2829465184 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x558cee760d30 con 0x558cee838340 2023-09-23T03:16:04.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.175+0000 7f172dffb640 1 -- 172.21.15.154:0/2829465184 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558cee75b690 con 0x558cee838340 2023-09-23T03:16:04.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.175+0000 7f172dffb640 1 -- 172.21.15.154:0/2829465184 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f174804f020 con 0x558cee838340 2023-09-23T03:16:04.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.175+0000 7f1757e35740 10 client.4806 fetch_fsmap finished waiting for FSMap version 9 2023-09-23T03:16:04.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.175+0000 7f1757e35740 10 client.4806 resolve_mds: resolved name 'd' to daemon mds.d 2023-09-23T03:16:04.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.175+0000 7f1757e35740 20 client.4806 populate_metadata read hostname 'smithi154' 2023-09-23T03:16:04.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.175+0000 7f1757e35740 1 --2- 172.21.15.154:0/2829465184 >> [v2:172.21.15.190:6832/3561509423,v1:172.21.15.190:6834/3561509423] conn(0x558cee83b230 0x558cee85dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:04.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.175+0000 7f1757e35740 4 client.4806 mds_command: new command op to 4285 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T03:16:04.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.175+0000 7f1757e35740 1 -- 172.21.15.154:0/2829465184 --> [v2:172.21.15.190:6832/3561509423,v1:172.21.15.190:6834/3561509423] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558cee75b3b0 con 0x558cee83b230 2023-09-23T03:16:04.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.176+0000 7f174effd640 1 --2- 172.21.15.154:0/2829465184 >> [v2:172.21.15.190:6832/3561509423,v1:172.21.15.190:6834/3561509423] conn(0x558cee83b230 0x558cee85dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:04.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.176+0000 7f174effd640 1 --2- 172.21.15.154:0/2829465184 >> [v2:172.21.15.190:6832/3561509423,v1:172.21.15.190:6834/3561509423] conn(0x558cee83b230 0x558cee85dd90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54ec5e3b808d0d68 server_cookie=80c9d1f280c8173f in_seq=0 out_seq=0 2023-09-23T03:16:04.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.176+0000 7f172dffb640 10 client.4806 ms_handle_connect on v2:172.21.15.190:6832/3561509423 2023-09-23T03:16:04.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.178+0000 7f172dffb640 1 -- 172.21.15.154:0/2829465184 <== mds.0 v2:172.21.15.190:6832/3561509423 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558cee75b3b0 con 0x558cee83b230 2023-09-23T03:16:04.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.178+0000 7f172dffb640 10 client.4806 handle_command_reply: tid=0 2023-09-23T03:16:04.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.178+0000 7f1757e35740 1 client.4806 shutdown 2023-09-23T03:16:04.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.178+0000 7f1757e35740 1 -- 172.21.15.154:0/2829465184 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558cee838340 msgr2=0x558cee8387c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:04.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.178+0000 7f1757e35740 1 --2- 172.21.15.154:0/2829465184 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558cee838340 0x558cee8387c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1748082f00 tx=0x7f174807ab50 comp rx=0 tx=0).stop 2023-09-23T03:16:04.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.178+0000 7f1757e35740 1 -- 172.21.15.154:0/2829465184 shutdown_connections 2023-09-23T03:16:04.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.178+0000 7f1757e35740 1 --2- 172.21.15.154:0/2829465184 >> [v2:172.21.15.190:6832/3561509423,v1:172.21.15.190:6834/3561509423] conn(0x558cee83b230 0x558cee85dd90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.178+0000 7f1757e35740 1 --2- 172.21.15.154:0/2829465184 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558cee838340 0x558cee8387c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.178+0000 7f1757e35740 1 --2- 172.21.15.154:0/2829465184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558cee822fe0 0x558cee850ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.178+0000 7f1757e35740 1 --2- 172.21.15.154:0/2829465184 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558cee822c00 0x558cee850660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.178+0000 7f1757e35740 1 -- 172.21.15.154:0/2829465184 >> 172.21.15.154:0/2829465184 conn(0x558cee7f6040 msgr2=0x558cee7f9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:04.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.178+0000 7f172dffb640 0 client.4806 ms_handle_reset on v2:172.21.15.190:6832/3561509423 2023-09-23T03:16:04.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.179+0000 7f1757e35740 1 -- 172.21.15.154:0/2829465184 shutdown_connections 2023-09-23T03:16:04.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.179+0000 7f1757e35740 1 -- 172.21.15.154:0/2829465184 wait complete. 2023-09-23T03:16:04.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.179+0000 7f1757e35740 20 client.4806 trim_cache size 0 max 16384 2023-09-23T03:16:04.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.195+0000 7f1757e35740 1 Processor -- start 2023-09-23T03:16:04.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.195+0000 7f1757e35740 1 -- start start 2023-09-23T03:16:04.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.196+0000 7f1757e35740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558cee822c00 0x558cee8223e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:04.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.196+0000 7f1757e35740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558cee822fe0 0x558cee825ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:04.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.196+0000 7f1757e35740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558cee838340 0x558cee837750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:04.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.196+0000 7f1757e35740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cee837c90 con 0x558cee838340 2023-09-23T03:16:04.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.196+0000 7f1757e35740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cee838910 con 0x558cee822c00 2023-09-23T03:16:04.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.196+0000 7f1757e35740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x558cee764590 con 0x558cee822fe0 2023-09-23T03:16:04.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.196+0000 7f174f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558cee838340 0x558cee837750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:04.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.196+0000 7f174f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558cee838340 0x558cee837750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59914/0 (socket says 172.21.15.154:59914) 2023-09-23T03:16:04.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.196+0000 7f174f7fe640 1 -- 172.21.15.154:0/949373544 learned_addr learned my addr 172.21.15.154:0/949373544 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:04.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.197+0000 7f174f7fe640 1 -- 172.21.15.154:0/949373544 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558cee822fe0 msgr2=0x558cee825ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:04.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.197+0000 7f174f7fe640 1 --2- 172.21.15.154:0/949373544 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558cee822fe0 0x558cee825ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.197+0000 7f174f7fe640 1 -- 172.21.15.154:0/949373544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558cee822c00 msgr2=0x558cee8223e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:04.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.197+0000 7f174f7fe640 1 --2- 172.21.15.154:0/949373544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558cee822c00 0x558cee8223e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.197+0000 7f174f7fe640 1 -- 172.21.15.154:0/949373544 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cee6ffb50 con 0x558cee838340 2023-09-23T03:16:04.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.197+0000 7f174f7fe640 1 --2- 172.21.15.154:0/949373544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558cee838340 0x558cee837750 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f174802c4e0 tx=0x7f1748043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa8e876b5c82fab8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:04.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.197+0000 7f17357fa640 1 -- 172.21.15.154:0/949373544 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1748083430 con 0x558cee838340 2023-09-23T03:16:04.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.197+0000 7f17357fa640 1 -- 172.21.15.154:0/949373544 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f174807f280 con 0x558cee838340 2023-09-23T03:16:04.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.197+0000 7f17357fa640 1 -- 172.21.15.154:0/949373544 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174807a620 con 0x558cee838340 2023-09-23T03:16:04.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.198+0000 7f1757e35740 1 -- 172.21.15.154:0/949373544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558cee838340 msgr2=0x558cee837750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:04.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.198+0000 7f1757e35740 1 --2- 172.21.15.154:0/949373544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558cee838340 0x558cee837750 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f174802c4e0 tx=0x7f1748043000 comp rx=0 tx=0).stop 2023-09-23T03:16:04.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.198+0000 7f1757e35740 1 -- 172.21.15.154:0/949373544 shutdown_connections 2023-09-23T03:16:04.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.198+0000 7f1757e35740 1 --2- 172.21.15.154:0/949373544 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558cee838340 0x558cee837750 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.198+0000 7f1757e35740 1 --2- 172.21.15.154:0/949373544 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558cee822fe0 0x558cee825ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.198+0000 7f1757e35740 1 --2- 172.21.15.154:0/949373544 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558cee822c00 0x558cee8223e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.198+0000 7f1757e35740 1 -- 172.21.15.154:0/949373544 >> 172.21.15.154:0/949373544 conn(0x558cee7f6040 msgr2=0x558cee821500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:04.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.198+0000 7f1757e35740 1 -- 172.21.15.154:0/949373544 shutdown_connections 2023-09-23T03:16:04.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.198+0000 7f1757e35740 1 -- 172.21.15.154:0/949373544 wait complete. 2023-09-23T03:16:04.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.199+0000 7f1757e35740 1 -- start start 2023-09-23T03:16:04.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.199+0000 7f1757e35740 1 Processor -- start 2023-09-23T03:16:04.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.200+0000 7f1757e35740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558cee822c00 0x558cee836720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:04.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.200+0000 7f1757e35740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558cee822fe0 0x558cee836c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:04.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.200+0000 7f1757e35740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558cee838340 0x558cee8ac320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:04.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.200+0000 7f1757e35740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cee7fc2e0 con 0x558cee822c00 2023-09-23T03:16:04.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.200+0000 7f1757e35740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cee8442f0 con 0x558cee822fe0 2023-09-23T03:16:04.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.200+0000 7f1757e35740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x558cee844530 con 0x558cee838340 2023-09-23T03:16:04.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.200+0000 7f1757e35740 10 client.0 _ll_register_callbacks cb 0x7f174000b1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:16:04.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.200+0000 7f174effd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558cee822c00 0x558cee836720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:04.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.200+0000 7f174effd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558cee822c00 0x558cee836720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59926/0 (socket says 172.21.15.154:59926) 2023-09-23T03:16:04.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.200+0000 7f174effd640 1 -- 172.21.15.154:0/3955568746 learned_addr learned my addr 172.21.15.154:0/3955568746 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:04.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.200+0000 7f174effd640 1 -- 172.21.15.154:0/3955568746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558cee838340 msgr2=0x558cee8ac320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:04.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.200+0000 7f174f7fe640 1 --2- 172.21.15.154:0/3955568746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558cee838340 0x558cee8ac320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:04.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.201+0000 7f174e7fc640 1 --2- 172.21.15.154:0/3955568746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558cee822fe0 0x558cee836c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:04.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.201+0000 7f174effd640 1 --2- 172.21.15.154:0/3955568746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558cee838340 0x558cee8ac320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.201+0000 7f174effd640 1 -- 172.21.15.154:0/3955568746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558cee822fe0 msgr2=0x558cee836c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:04.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.201+0000 7f174effd640 1 --2- 172.21.15.154:0/3955568746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558cee822fe0 0x558cee836c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.201+0000 7f174effd640 1 -- 172.21.15.154:0/3955568746 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558cee760d30 con 0x558cee822c00 2023-09-23T03:16:04.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.201+0000 7f174effd640 1 --2- 172.21.15.154:0/3955568746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558cee822c00 0x558cee836720 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f174000fa60 tx=0x7f17400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2f33fe01402cb16 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:04.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.201+0000 7f172dffb640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:16:04.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.201+0000 7f172dffb640 1 -- 172.21.15.154:0/3955568746 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174002f040 con 0x558cee822c00 2023-09-23T03:16:04.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.201+0000 7f1757e35740 1 -- 172.21.15.154:0/3955568746 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1740019070 con 0x558cee822c00 2023-09-23T03:16:04.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.201+0000 7f172dffb640 1 -- 172.21.15.154:0/3955568746 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f1740014dc0 con 0x558cee822c00 2023-09-23T03:16:04.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.202+0000 7f172dffb640 1 -- 172.21.15.154:0/3955568746 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17400345c0 con 0x558cee822c00 2023-09-23T03:16:04.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.202+0000 7f1757e35740 10 client.4812 fetch_fsmap learned FSMap version 9 2023-09-23T03:16:04.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.202+0000 7f1757e35740 1 -- 172.21.15.154:0/3955568746 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x558cee725140 con 0x558cee822c00 2023-09-23T03:16:04.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.202+0000 7f172dffb640 1 -- 172.21.15.154:0/3955568746 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1740019070 con 0x558cee822c00 2023-09-23T03:16:04.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.202+0000 7f172dffb640 1 -- 172.21.15.154:0/3955568746 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f1740034900 con 0x558cee822c00 2023-09-23T03:16:04.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.202+0000 7f1757e35740 10 client.4812 fetch_fsmap finished waiting for FSMap version 9 2023-09-23T03:16:04.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.202+0000 7f1757e35740 10 client.4812 resolve_mds: resolved name 'd' to daemon mds.d 2023-09-23T03:16:04.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.202+0000 7f1757e35740 20 client.4812 populate_metadata read hostname 'smithi154' 2023-09-23T03:16:04.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.202+0000 7f1757e35740 1 --2- 172.21.15.154:0/3955568746 >> [v2:172.21.15.190:6832/3561509423,v1:172.21.15.190:6834/3561509423] conn(0x558cee8acc30 0x558cee841be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:04.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.202+0000 7f1757e35740 4 client.4812 mds_command: new command op to 4285 tid=0[{"prefix": "status"}] 2023-09-23T03:16:04.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.202+0000 7f1757e35740 1 -- 172.21.15.154:0/3955568746 --> [v2:172.21.15.190:6832/3561509423,v1:172.21.15.190:6834/3561509423] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558cee4646c0 con 0x558cee8acc30 2023-09-23T03:16:04.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.203+0000 7f174e7fc640 1 --2- 172.21.15.154:0/3955568746 >> [v2:172.21.15.190:6832/3561509423,v1:172.21.15.190:6834/3561509423] conn(0x558cee8acc30 0x558cee841be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:04.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.203+0000 7f174e7fc640 1 --2- 172.21.15.154:0/3955568746 >> [v2:172.21.15.190:6832/3561509423,v1:172.21.15.190:6834/3561509423] conn(0x558cee8acc30 0x558cee841be0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f39e4df26dcf5fa server_cookie=fc4e6d6b10ad7884 in_seq=0 out_seq=0 2023-09-23T03:16:04.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.203+0000 7f172dffb640 10 client.4812 ms_handle_connect on v2:172.21.15.190:6832/3561509423 2023-09-23T03:16:04.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.204+0000 7f172dffb640 1 -- 172.21.15.154:0/3955568746 <== mds.0 v2:172.21.15.190:6832/3561509423 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x558cee4646c0 con 0x558cee8acc30 2023-09-23T03:16:04.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.204+0000 7f172dffb640 10 client.4812 handle_command_reply: tid=0 2023-09-23T03:16:04.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.204+0000 7f1757e35740 1 client.4812 shutdown 2023-09-23T03:16:04.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.204+0000 7f1757e35740 1 -- 172.21.15.154:0/3955568746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558cee822c00 msgr2=0x558cee836720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:04.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.204+0000 7f1757e35740 1 --2- 172.21.15.154:0/3955568746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558cee822c00 0x558cee836720 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f174000fa60 tx=0x7f17400046c0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.204+0000 7f1757e35740 1 -- 172.21.15.154:0/3955568746 shutdown_connections 2023-09-23T03:16:04.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.204+0000 7f1757e35740 1 --2- 172.21.15.154:0/3955568746 >> [v2:172.21.15.190:6832/3561509423,v1:172.21.15.190:6834/3561509423] conn(0x558cee8acc30 0x558cee841be0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.204+0000 7f1757e35740 1 --2- 172.21.15.154:0/3955568746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558cee838340 0x558cee8ac320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.204+0000 7f1757e35740 1 --2- 172.21.15.154:0/3955568746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558cee822fe0 0x558cee836c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.204+0000 7f172dffb640 0 client.4812 ms_handle_reset on v2:172.21.15.190:6832/3561509423 2023-09-23T03:16:04.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.204+0000 7f1757e35740 1 --2- 172.21.15.154:0/3955568746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558cee822c00 0x558cee836720 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.204+0000 7f1757e35740 1 -- 172.21.15.154:0/3955568746 >> 172.21.15.154:0/3955568746 conn(0x558cee7f6040 msgr2=0x558cee7f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:04.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.205+0000 7f1757e35740 1 -- 172.21.15.154:0/3955568746 shutdown_connections 2023-09-23T03:16:04.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.205+0000 7f1757e35740 1 -- 172.21.15.154:0/3955568746 wait complete. 2023-09-23T03:16:04.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.205+0000 7f1757e35740 20 client.4812 trim_cache size 0 max 16384 2023-09-23T03:16:04.209 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:16:04.209 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_fsid": "8c8039d3-6c84-43db-8808-85e0dbdd47c8", 2023-09-23T03:16:04.209 INFO:teuthology.orchestra.run.smithi154.stdout: "whoami": 0, 2023-09-23T03:16:04.209 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 4285, 2023-09-23T03:16:04.209 INFO:teuthology.orchestra.run.smithi154.stdout: "want_state": "up:active", 2023-09-23T03:16:04.209 INFO:teuthology.orchestra.run.smithi154.stdout: "state": "up:active", 2023-09-23T03:16:04.209 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs", 2023-09-23T03:16:04.209 INFO:teuthology.orchestra.run.smithi154.stdout: "rank_uptime": 42.955506264999997, 2023-09-23T03:16:04.209 INFO:teuthology.orchestra.run.smithi154.stdout: "mdsmap_epoch": 9, 2023-09-23T03:16:04.209 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_epoch": 26, 2023-09-23T03:16:04.210 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_epoch_barrier": 24, 2023-09-23T03:16:04.210 INFO:teuthology.orchestra.run.smithi154.stdout: "uptime": 48.596366558 2023-09-23T03:16:04.210 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:16:04.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.205+0000 7f17552a2640 1 -- 172.21.15.154:0/1740351309 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f173c06af30 msgr2=0x7f173c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:04.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.205+0000 7f17552a2640 1 --2- 172.21.15.154:0/1740351309 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f173c06af30 0x7f173c06d3f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f174000b9b0 tx=0x7f1740008040 comp rx=0 tx=0).stop 2023-09-23T03:16:04.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.205+0000 7f17552a2640 1 -- 172.21.15.154:0/1740351309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1750160dc0 msgr2=0x7f175015b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:04.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.205+0000 7f17552a2640 1 --2- 172.21.15.154:0/1740351309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1750160dc0 0x7f175015b7d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1748036040 tx=0x7f1748004a40 comp rx=0 tx=0).stop 2023-09-23T03:16:04.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.205+0000 7f17552a2640 1 -- 172.21.15.154:0/1740351309 shutdown_connections 2023-09-23T03:16:04.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.205+0000 7f17552a2640 1 --2- 172.21.15.154:0/1740351309 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f173c06af30 0x7f173c06d3f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.205+0000 7f17552a2640 1 --2- 172.21.15.154:0/1740351309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1750160dc0 0x7f175015b7d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.205+0000 7f17552a2640 1 --2- 172.21.15.154:0/1740351309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f175015caf0 0x7f175015b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.205+0000 7f17552a2640 1 --2- 172.21.15.154:0/1740351309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17500cee70 0x7f1750098530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.205+0000 7f17552a2640 1 -- 172.21.15.154:0/1740351309 >> 172.21.15.154:0/1740351309 conn(0x7f175006a0f0 msgr2=0x7f17500af3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:04.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.206+0000 7f17552a2640 1 -- 172.21.15.154:0/1740351309 shutdown_connections 2023-09-23T03:16:04.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.206+0000 7f17552a2640 1 -- 172.21.15.154:0/1740351309 wait complete. 2023-09-23T03:16:04.223 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-09-23T03:16:04.233 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-09-23T03:16:04.233 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-09-23T03:16:04.234 INFO:tasks.ceph_fuse:client.0 config is {} 2023-09-23T03:16:04.234 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-09-23T03:16:04.234 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-23T03:16:04.234 INFO:tasks.ceph_fuse:client.2 config is {} 2023-09-23T03:16:04.234 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-09-23T03:16:04.234 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-09-23T03:16:04.234 INFO:tasks.ceph_fuse:client.1 config is {} 2023-09-23T03:16:04.234 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-09-23T03:16:04.234 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-09-23T03:16:04.234 INFO:tasks.ceph_fuse:client.3 config is {} 2023-09-23T03:16:04.235 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-09-23T03:16:04.235 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-09-23T03:16:04.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:04.235 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2023-09-23T03:16:04.289 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-09-23T03:16:04.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:04.289 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-09-23T03:16:04.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:04.383 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link delete ceph-brx 2023-09-23T03:16:04.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:04.514 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2023-09-23T03:16:04.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:04.574 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link delete ceph-brx 2023-09-23T03:16:04.675 INFO:teuthology.orchestra.run.smithi190.stderr:Cannot find device "ceph-brx" 2023-09-23T03:16:04.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T03:16:04.679 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-09-23T03:16:04.679 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-23T03:16:04.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.899+0000 7f8f71fae640 1 Processor -- start 2023-09-23T03:16:04.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.900+0000 7f8f71fae640 1 -- start start 2023-09-23T03:16:04.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.901+0000 7f8f71fae640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8f6c0b2fd0 0x7f8f6c0cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:04.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.901+0000 7f8f71fae640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f6c0cb210 0x7f8f6c0ab030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:04.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.901+0000 7f8f71fae640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f6c15abd0 con 0x7f8f6c15a7f0 2023-09-23T03:16:04.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.901+0000 7f8f71fae640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f6c06c8b0 con 0x7f8f6c0b2fd0 2023-09-23T03:16:04.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.901+0000 7f8f71fae640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c0b2b40 con 0x7f8f6c0cb210 2023-09-23T03:16:04.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.902+0000 7f8f717ad640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f6c0cb210 0x7f8f6c0ab030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:04.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.902+0000 7f8f717ad640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f6c0cb210 0x7f8f6c0ab030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:60776/0 (socket says 172.21.15.154:60776) 2023-09-23T03:16:04.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.902+0000 7f8f717ad640 1 -- 172.21.15.154:0/4023837520 learned_addr learned my addr 172.21.15.154:0/4023837520 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:04.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.902+0000 7f8f6bfff640 1 --2- 172.21.15.154:0/4023837520 >> v2:172.21.15.190:3300/0 conn(0x7f8f6c0b2fd0 0x7f8f6c0cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:04.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.903+0000 7f8f717ad640 1 -- 172.21.15.154:0/4023837520 >> v2:172.21.15.190:3300/0 conn(0x7f8f6c0b2fd0 msgr2=0x7f8f6c0cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:04.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.903+0000 7f8f717ad640 1 --2- 172.21.15.154:0/4023837520 >> v2:172.21.15.190:3300/0 conn(0x7f8f6c0b2fd0 0x7f8f6c0cacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.903+0000 7f8f717ad640 1 -- 172.21.15.154:0/4023837520 >> v1:172.21.15.190:6789/0 conn(0x7f8f6c15a7f0 legacy=0x7f8f6c06bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:04.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.903+0000 7f8f717ad640 1 -- 172.21.15.154:0/4023837520 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f6c0cb640 con 0x7f8f6c0cb210 2023-09-23T03:16:04.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.903+0000 7f8f6b7fe640 1 -- 172.21.15.154:0/4023837520 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 812494684 0 0) 0x7f8f6c15abd0 con 0x7f8f6c15a7f0 2023-09-23T03:16:04.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.903+0000 7f8f6bfff640 1 --2- 172.21.15.154:0/4023837520 >> v2:172.21.15.190:3300/0 conn(0x7f8f6c0b2fd0 0x7f8f6c0cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:16:04.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.905+0000 7f8f717ad640 1 --2- 172.21.15.154:0/4023837520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f6c0cb210 0x7f8f6c0ab030 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c0cb7d0 tx=0x7f8f6402bf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7680c529a61f3df1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:04.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.905+0000 7f8f6b7fe640 1 -- 172.21.15.154:0/4023837520 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6402cae0 con 0x7f8f6c0cb210 2023-09-23T03:16:04.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.905+0000 7f8f6b7fe640 1 -- 172.21.15.154:0/4023837520 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6402cc40 con 0x7f8f6c0cb210 2023-09-23T03:16:04.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.905+0000 7f8f6b7fe640 1 -- 172.21.15.154:0/4023837520 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f640343f0 con 0x7f8f6c0cb210 2023-09-23T03:16:04.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.906+0000 7f8f71fae640 1 -- 172.21.15.154:0/4023837520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f6c0cb210 msgr2=0x7f8f6c0ab030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:04.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.906+0000 7f8f71fae640 1 --2- 172.21.15.154:0/4023837520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f6c0cb210 0x7f8f6c0ab030 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c0cb7d0 tx=0x7f8f6402bf40 comp rx=0 tx=0).stop 2023-09-23T03:16:04.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.906+0000 7f8f71fae640 1 -- 172.21.15.154:0/4023837520 shutdown_connections 2023-09-23T03:16:04.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.906+0000 7f8f71fae640 1 --2- 172.21.15.154:0/4023837520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f6c0cb210 0x7f8f6c0ab030 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.906+0000 7f8f71fae640 1 --2- 172.21.15.154:0/4023837520 >> v2:172.21.15.190:3300/0 conn(0x7f8f6c0b2fd0 0x7f8f6c0cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.906+0000 7f8f71fae640 1 -- 172.21.15.154:0/4023837520 >> 172.21.15.154:0/4023837520 conn(0x7f8f6c069eb0 msgr2=0x7f8f6c06a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:04.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.906+0000 7f8f71fae640 1 -- 172.21.15.154:0/4023837520 shutdown_connections 2023-09-23T03:16:04.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.906+0000 7f8f71fae640 1 -- 172.21.15.154:0/4023837520 wait complete. 2023-09-23T03:16:04.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.908+0000 7f8f71fae640 1 Processor -- start 2023-09-23T03:16:04.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.908+0000 7f8f71fae640 1 -- start start 2023-09-23T03:16:04.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.909+0000 7f8f71fae640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f6c0cb210 0x7f8f6c15d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:04.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.909+0000 7f8f71fae640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f6c15a7f0 0x7f8f6c096e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:04.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.909+0000 7f8f71fae640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f6c1596d0 0x7f8f6c152d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:04.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.909+0000 7f8f71fae640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c0c5d60 con 0x7f8f6c15a7f0 2023-09-23T03:16:04.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.909+0000 7f8f71fae640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c0ae180 con 0x7f8f6c0cb210 2023-09-23T03:16:04.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.909+0000 7f8f71fae640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c0c5430 con 0x7f8f6c1596d0 2023-09-23T03:16:04.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.909+0000 7f8f6bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f6c15a7f0 0x7f8f6c096e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:04.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.910+0000 7f8f6bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f6c15a7f0 0x7f8f6c096e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59932/0 (socket says 172.21.15.154:59932) 2023-09-23T03:16:04.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.910+0000 7f8f6bfff640 1 -- 172.21.15.154:0/914792873 learned_addr learned my addr 172.21.15.154:0/914792873 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:04.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.910+0000 7f8f717ad640 1 --2- 172.21.15.154:0/914792873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f6c1596d0 0x7f8f6c152d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:04.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.910+0000 7f8f6bfff640 1 -- 172.21.15.154:0/914792873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f6c1596d0 msgr2=0x7f8f6c152d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:04.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.910+0000 7f8f6bfff640 1 --2- 172.21.15.154:0/914792873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f6c1596d0 0x7f8f6c152d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.910+0000 7f8f6bfff640 1 -- 172.21.15.154:0/914792873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f6c0cb210 msgr2=0x7f8f6c15d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:04.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.910+0000 7f8f70fac640 1 --2- 172.21.15.154:0/914792873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f6c0cb210 0x7f8f6c15d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:04.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.910+0000 7f8f6bfff640 1 --2- 172.21.15.154:0/914792873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f6c0cb210 0x7f8f6c15d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:04.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.910+0000 7f8f6bfff640 1 -- 172.21.15.154:0/914792873 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f58009400 con 0x7f8f6c15a7f0 2023-09-23T03:16:04.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.910+0000 7f8f6bfff640 1 --2- 172.21.15.154:0/914792873 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f6c15a7f0 0x7f8f6c096e60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8f5800ebe0 tx=0x7f8f5800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:04.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.911+0000 7f8f697fa640 1 -- 172.21.15.154:0/914792873 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f58009760 con 0x7f8f6c15a7f0 2023-09-23T03:16:04.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.911+0000 7f8f697fa640 1 -- 172.21.15.154:0/914792873 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f5800cc30 con 0x7f8f6c15a7f0 2023-09-23T03:16:04.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.911+0000 7f8f71fae640 1 -- 172.21.15.154:0/914792873 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f64009b30 con 0x7f8f6c15a7f0 2023-09-23T03:16:04.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.911+0000 7f8f697fa640 1 -- 172.21.15.154:0/914792873 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5800eee0 con 0x7f8f6c15a7f0 2023-09-23T03:16:04.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.911+0000 7f8f71fae640 1 -- 172.21.15.154:0/914792873 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f6c0c59f0 con 0x7f8f6c15a7f0 2023-09-23T03:16:04.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.912+0000 7f8f71fae640 1 -- 172.21.15.154:0/914792873 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f6c0ada90 con 0x7f8f6c15a7f0 2023-09-23T03:16:04.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.913+0000 7f8f697fa640 1 -- 172.21.15.154:0/914792873 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f8f58013470 con 0x7f8f6c15a7f0 2023-09-23T03:16:04.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.914+0000 7f8f697fa640 1 --2- 172.21.15.154:0/914792873 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8f5006af60 0x7f8f5006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:04.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.914+0000 7f8f697fa640 1 -- 172.21.15.154:0/914792873 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f8f5808a1c0 con 0x7f8f6c15a7f0 2023-09-23T03:16:04.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.914+0000 7f8f70fac640 1 --2- 172.21.15.154:0/914792873 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8f5006af60 0x7f8f5006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:04.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.915+0000 7f8f70fac640 1 --2- 172.21.15.154:0/914792873 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8f5006af60 0x7f8f5006d420 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8f60000f60 tx=0x7f8f60009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:04.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:04.918+0000 7f8f697fa640 1 -- 172.21.15.154:0/914792873 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f8f58054e40 con 0x7f8f6c15a7f0 2023-09-23T03:16:05.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:05.145+0000 7f8f71fae640 1 -- 172.21.15.154:0/914792873 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8f6c04e920 con 0x7f8f6c15a7f0 2023-09-23T03:16:05.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:05.145+0000 7f8f697fa640 1 -- 172.21.15.154:0/914792873 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f8f58058460 con 0x7f8f6c15a7f0 2023-09-23T03:16:05.147 INFO:teuthology.orchestra.run.smithi154.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-09-23T03:16:05.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:05.148+0000 7f8f71fae640 1 -- 172.21.15.154:0/914792873 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8f5006af60 msgr2=0x7f8f5006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:05.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:05.148+0000 7f8f71fae640 1 --2- 172.21.15.154:0/914792873 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8f5006af60 0x7f8f5006d420 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8f60000f60 tx=0x7f8f60009040 comp rx=0 tx=0).stop 2023-09-23T03:16:05.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:05.148+0000 7f8f71fae640 1 -- 172.21.15.154:0/914792873 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f6c15a7f0 msgr2=0x7f8f6c096e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:05.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:05.148+0000 7f8f71fae640 1 --2- 172.21.15.154:0/914792873 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f6c15a7f0 0x7f8f6c096e60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8f5800ebe0 tx=0x7f8f5800c520 comp rx=0 tx=0).stop 2023-09-23T03:16:05.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:05.148+0000 7f8f71fae640 1 -- 172.21.15.154:0/914792873 shutdown_connections 2023-09-23T03:16:05.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:05.148+0000 7f8f71fae640 1 --2- 172.21.15.154:0/914792873 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8f5006af60 0x7f8f5006d420 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:05.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:05.148+0000 7f8f71fae640 1 --2- 172.21.15.154:0/914792873 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f6c1596d0 0x7f8f6c152d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:05.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:05.148+0000 7f8f71fae640 1 --2- 172.21.15.154:0/914792873 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f6c15a7f0 0x7f8f6c096e60 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:05.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:05.148+0000 7f8f71fae640 1 --2- 172.21.15.154:0/914792873 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f6c0cb210 0x7f8f6c15d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:05.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:05.148+0000 7f8f71fae640 1 -- 172.21.15.154:0/914792873 >> 172.21.15.154:0/914792873 conn(0x7f8f6c069eb0 msgr2=0x7f8f6c071660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:05.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:05.148+0000 7f8f71fae640 1 -- 172.21.15.154:0/914792873 shutdown_connections 2023-09-23T03:16:05.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:05.148+0000 7f8f71fae640 1 -- 172.21.15.154:0/914792873 wait complete. 2023-09-23T03:16:05.169 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-23T03:16:05.169 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-23T03:16:05.169 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi154.front.sepia.ceph.com 2023-09-23T03:16:05.169 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-09-23T03:16:05.169 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-23T03:16:05.169 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-23T03:16:05.169 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-23T03:16:05.169 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-23T03:16:05.169 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-23T03:16:05.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:05.170 DEBUG:teuthology.orchestra.run.smithi154:> ip addr 2023-09-23T03:16:05.220 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-23T03:16:05.220 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-23T03:16:05.220 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2023-09-23T03:16:05.220 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:16:05.220 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2023-09-23T03:16:05.220 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:16:05.220 INFO:teuthology.orchestra.run.smithi154.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:16:05.221 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:54 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:16:05.221 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f0 2023-09-23T03:16:05.221 INFO:teuthology.orchestra.run.smithi154.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:16:05.221 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:55 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:16:05.221 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f1 2023-09-23T03:16:05.221 INFO:teuthology.orchestra.run.smithi154.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:16:05.221 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:60 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:16:05.221 INFO:teuthology.orchestra.run.smithi154.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-23T03:16:05.221 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:61 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:16:05.221 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.21.15.154/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-09-23T03:16:05.221 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft 42360sec preferred_lft 42360sec 2023-09-23T03:16:05.221 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::ec4:7aff:fe88:7761/64 scope link 2023-09-23T03:16:05.222 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:16:05.222 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-09-23T03:16:05.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:05.223 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-09-23T03:16:05.223 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-09-23T03:16:05.223 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link add name ceph-brx type bridge 2023-09-23T03:16:05.223 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip addr flush dev ceph-brx 2023-09-23T03:16:05.223 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set ceph-brx up 2023-09-23T03:16:05.223 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-09-23T03:16:05.223 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-09-23T03:16:05.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:05.457 DEBUG:teuthology.orchestra.run.smithi154:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-23T03:16:05.530 INFO:teuthology.orchestra.run.smithi154.stdout:1 2023-09-23T03:16:05.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:05.534 DEBUG:teuthology.orchestra.run.smithi154:> route 2023-09-23T03:16:05.583 INFO:teuthology.orchestra.run.smithi154.stdout:Kernel IP routing table 2023-09-23T03:16:05.583 INFO:teuthology.orchestra.run.smithi154.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-23T03:16:05.583 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-23T03:16:05.583 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-09-23T03:16:05.583 INFO:teuthology.orchestra.run.smithi154.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-23T03:16:05.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:05.584 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-09-23T03:16:05.584 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-09-23T03:16:05.585 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-23T03:16:05.585 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-23T03:16:05.585 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-23T03:16:05.585 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-09-23T03:16:05.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:05.761 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2023-09-23T03:16:05.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:05.809 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list-id 2023-09-23T03:16:05.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:05.859 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-09-23T03:16:05.859 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-09-23T03:16:05.859 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:16:05.859 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-09-23T03:16:05.859 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-09-23T03:16:06.001 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-09-23T03:16:06.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:06.001 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-09-23T03:16:06.001 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-09-23T03:16:06.001 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-09-23T03:16:06.001 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-09-23T03:16:06.001 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-09-23T03:16:06.001 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-09-23T03:16:06.001 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-09-23T03:16:06.002 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-09-23T03:16:06.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:06.265 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-09-23T03:16:06.265 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-09-23T03:16:06.265 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set brx.0 up 2023-09-23T03:16:06.265 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set dev brx.0 master ceph-brx 2023-09-23T03:16:06.265 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-09-23T03:16:06.432 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-09-23T03:16:06.432 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-23T03:16:06.433 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-09-23T03:16:06.481 INFO:teuthology.orchestra.run.smithi154.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-09-23T03:16:06.481 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-23T03:16:06.481 DEBUG:teuthology.orchestra.run.smithi154:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-09-23T03:16:06.532 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-09-23T03:16:06.625 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2023-09-23T03:16:06.683 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2023-09-23T03:16:06.683 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2023-09-23T03:16:06.683 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2023-09-23T03:16:06.683 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2023-09-23T03:16:06.683 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2023-09-23T03:16:06.683 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2023-09-23T03:16:06.683 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2023-09-23T03:16:06.683 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2023-09-23T03:16:06.683 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2023-09-23T03:16:06.684 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2023-09-23T03:16:06.684 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2023-09-23T03:16:06.684 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T03:16:06.684 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2023-09-23T03:16:06.684 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2023-09-23T03:16:06.684 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2023-09-23T03:16:06.684 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2023-09-23T03:16:06.684 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2023-09-23T03:16:06.684 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2023-09-23T03:16:06.684 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2023-09-23T03:16:06.684 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-09-23T03:16:06.684 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-09-23T03:16:06.685 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2023-09-23T03:16:06.685 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2023-09-23T03:16:06.685 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2023-09-23T03:16:06.685 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-0 2023-09-23T03:16:06.685 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-1 2023-09-23T03:16:06.685 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-2 2023-09-23T03:16:06.685 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-3 2023-09-23T03:16:06.685 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2023-09-23T03:16:06.685 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:16:06.685 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:16:06.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:06.686 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-09-23T03:16:06.738 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-23T03:16:06.738 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-09-23T03:16:06.743 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-09-23T03:16:06.819 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2023-09-23T03:16:06.861 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2023-09-23T03:16:06.861 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2023-09-23T03:16:06.861 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2023-09-23T03:16:06.861 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2023-09-23T03:16:06.861 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2023-09-23T03:16:06.861 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2023-09-23T03:16:06.861 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2023-09-23T03:16:06.861 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2023-09-23T03:16:06.861 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2023-09-23T03:16:06.861 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2023-09-23T03:16:06.861 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2023-09-23T03:16:06.862 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T03:16:06.862 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2023-09-23T03:16:06.862 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2023-09-23T03:16:06.862 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2023-09-23T03:16:06.862 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2023-09-23T03:16:06.862 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2023-09-23T03:16:06.862 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2023-09-23T03:16:06.862 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2023-09-23T03:16:06.862 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-09-23T03:16:06.862 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-09-23T03:16:06.862 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2023-09-23T03:16:06.863 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2023-09-23T03:16:06.863 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2023-09-23T03:16:06.863 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-0 2023-09-23T03:16:06.863 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-1 2023-09-23T03:16:06.863 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-2 2023-09-23T03:16:06.863 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-3 2023-09-23T03:16:06.863 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2023-09-23T03:16:06.863 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:16:06.863 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:16:06.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:06.864 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-09-23T03:16:06.934 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:2023-09-23T03:16:06.934+0000 7fd7c2819480 -1 init, newargv = 0x560b2aba5cd0 newargc=15 2023-09-23T03:16:06.934 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:ceph-fuse[96840]: starting ceph client 2023-09-23T03:16:06.946 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:ceph-fuse[96840]: starting fuse 2023-09-23T03:16:06.954 INFO:teuthology.orchestra.run.smithi154.stdout:39 2023-09-23T03:16:06.955 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-09-23T03:16:06.955 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:16:06.955 DEBUG:teuthology.orchestra.run.smithi154:> import glob 2023-09-23T03:16:06.955 DEBUG:teuthology.orchestra.run.smithi154:> import re 2023-09-23T03:16:06.955 DEBUG:teuthology.orchestra.run.smithi154:> import os 2023-09-23T03:16:06.955 DEBUG:teuthology.orchestra.run.smithi154:> import subprocess 2023-09-23T03:16:06.955 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:16:06.955 DEBUG:teuthology.orchestra.run.smithi154:> def _find_admin_socket(client_name): 2023-09-23T03:16:06.956 DEBUG:teuthology.orchestra.run.smithi154:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-09-23T03:16:06.956 DEBUG:teuthology.orchestra.run.smithi154:> files = glob.glob(asok_path) 2023-09-23T03:16:06.956 DEBUG:teuthology.orchestra.run.smithi154:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-09-23T03:16:06.956 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:16:06.956 DEBUG:teuthology.orchestra.run.smithi154:> # Given a non-glob path, it better be there 2023-09-23T03:16:06.956 DEBUG:teuthology.orchestra.run.smithi154:> if "*" not in asok_path: 2023-09-23T03:16:06.956 DEBUG:teuthology.orchestra.run.smithi154:> assert(len(files) == 1) 2023-09-23T03:16:06.956 DEBUG:teuthology.orchestra.run.smithi154:> return files[0] 2023-09-23T03:16:06.956 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:16:06.956 DEBUG:teuthology.orchestra.run.smithi154:> for f in files: 2023-09-23T03:16:06.956 DEBUG:teuthology.orchestra.run.smithi154:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-23T03:16:06.956 DEBUG:teuthology.orchestra.run.smithi154:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-23T03:16:06.956 DEBUG:teuthology.orchestra.run.smithi154:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-23T03:16:06.957 DEBUG:teuthology.orchestra.run.smithi154:> contents = proc_f.read() 2023-09-23T03:16:06.957 DEBUG:teuthology.orchestra.run.smithi154:> if mountpoint in contents: 2023-09-23T03:16:06.957 DEBUG:teuthology.orchestra.run.smithi154:> return f 2023-09-23T03:16:06.957 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-23T03:16:06.957 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:16:06.957 DEBUG:teuthology.orchestra.run.smithi154:> print(_find_admin_socket("client.0")) 2023-09-23T03:16:06.957 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:16:07.162 INFO:teuthology.orchestra.run.smithi154.stdout:/var/run/ceph/ceph-client.0.96840.asok 2023-09-23T03:16:07.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:16:07.178 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.96840.asok 2023-09-23T03:16:07.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:07.178 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.96840.asok status 2023-09-23T03:16:07.422 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:16:07.422 INFO:teuthology.orchestra.run.smithi154.stdout: "metadata": { 2023-09-23T03:16:07.422 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_sha1": "862663913a74a99e1c52d50e6a7e59dd6705749a", 2023-09-23T03:16:07.422 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_version": "ceph version 18.0.0-6336-g86266391 (862663913a74a99e1c52d50e6a7e59dd6705749a) reef (dev)", 2023-09-23T03:16:07.422 INFO:teuthology.orchestra.run.smithi154.stdout: "entity_id": "0", 2023-09-23T03:16:07.423 INFO:teuthology.orchestra.run.smithi154.stdout: "hostname": "smithi154", 2023-09-23T03:16:07.423 INFO:teuthology.orchestra.run.smithi154.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-09-23T03:16:07.423 INFO:teuthology.orchestra.run.smithi154.stdout: "pid": "96840", 2023-09-23T03:16:07.423 INFO:teuthology.orchestra.run.smithi154.stdout: "root": "/" 2023-09-23T03:16:07.423 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:16:07.423 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_count": 0, 2023-09-23T03:16:07.423 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_pinned_count": 0, 2023-09-23T03:16:07.423 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 4821, 2023-09-23T03:16:07.423 INFO:teuthology.orchestra.run.smithi154.stdout: "inst": { 2023-09-23T03:16:07.423 INFO:teuthology.orchestra.run.smithi154.stdout: "name": { 2023-09-23T03:16:07.423 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "client", 2023-09-23T03:16:07.423 INFO:teuthology.orchestra.run.smithi154.stdout: "num": 4821 2023-09-23T03:16:07.424 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:16:07.424 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2023-09-23T03:16:07.424 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2023-09-23T03:16:07.424 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:0", 2023-09-23T03:16:07.424 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 1597875229 2023-09-23T03:16:07.424 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:16:07.424 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:16:07.424 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2023-09-23T03:16:07.424 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2023-09-23T03:16:07.424 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:0", 2023-09-23T03:16:07.424 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 1597875229 2023-09-23T03:16:07.424 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:16:07.424 INFO:teuthology.orchestra.run.smithi154.stdout: "inst_str": "client.4821 172.21.15.154:0/1597875229", 2023-09-23T03:16:07.425 INFO:teuthology.orchestra.run.smithi154.stdout: "addr_str": "172.21.15.154:0/1597875229", 2023-09-23T03:16:07.425 INFO:teuthology.orchestra.run.smithi154.stdout: "inode_count": 1, 2023-09-23T03:16:07.425 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_epoch": 9, 2023-09-23T03:16:07.425 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch": 26, 2023-09-23T03:16:07.425 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch_barrier": 0, 2023-09-23T03:16:07.425 INFO:teuthology.orchestra.run.smithi154.stdout: "blocklisted": false, 2023-09-23T03:16:07.425 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs" 2023-09-23T03:16:07.425 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:16:07.436 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-23T03:16:07.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.652+0000 7fe4493fb640 1 Processor -- start 2023-09-23T03:16:07.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.652+0000 7fe4493fb640 1 -- start start 2023-09-23T03:16:07.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.653+0000 7fe4493fb640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe4440ada30 0x7fe4440ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:07.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.653+0000 7fe4493fb640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4440cf130 0x7fe44415d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:07.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.653+0000 7fe4493fb640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe44406ceb0 con 0x7fe4440b2fe0 2023-09-23T03:16:07.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.653+0000 7fe4493fb640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4440ade10 con 0x7fe4440ada30 2023-09-23T03:16:07.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.653+0000 7fe4493fb640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe444159c90 con 0x7fe4440cf130 2023-09-23T03:16:07.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.653+0000 7fe442ffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe4440ada30 0x7fe4440ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:07.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.653+0000 7fe43a7fc640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fe4440b2fe0 0x7fe4440cea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:58562/0 (socket says 172.21.15.154:58562) 2023-09-23T03:16:07.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.653+0000 7fe43a7fc640 1 -- 172.21.15.154:0/3041833071 learned_addr learned my addr 172.21.15.154:0/3041833071 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:07.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.654+0000 7fe4437fe640 1 --2- 172.21.15.154:0/3041833071 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4440cf130 0x7fe44415d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:07.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.654+0000 7fe442ffd640 1 -- 172.21.15.154:0/3041833071 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4440cf130 msgr2=0x7fe44415d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:07.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.654+0000 7fe442ffd640 1 --2- 172.21.15.154:0/3041833071 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4440cf130 0x7fe44415d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:07.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.654+0000 7fe442ffd640 1 -- 172.21.15.154:0/3041833071 >> v1:172.21.15.154:6789/0 conn(0x7fe4440b2fe0 legacy=0x7fe4440cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:07.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.654+0000 7fe442ffd640 1 -- 172.21.15.154:0/3041833071 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe44406c110 con 0x7fe4440ada30 2023-09-23T03:16:07.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.654+0000 7fe4437fe640 1 --2- 172.21.15.154:0/3041833071 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4440cf130 0x7fe44415d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:07.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.654+0000 7fe4427fc640 1 -- 172.21.15.154:0/3041833071 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2811259368 0 0) 0x7fe42c001110 con 0x7fe4440b2fe0 2023-09-23T03:16:07.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.655+0000 7fe442ffd640 1 --2- 172.21.15.154:0/3041833071 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4440ada30 0x7fe4440ccaf0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe434009790 tx=0x7fe43402b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5136e9ee93960458 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:07.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.655+0000 7fe4427fc640 1 -- 172.21.15.154:0/3041833071 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe434031800 con 0x7fe4440ada30 2023-09-23T03:16:07.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.655+0000 7fe4427fc640 1 -- 172.21.15.154:0/3041833071 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe434031960 con 0x7fe4440ada30 2023-09-23T03:16:07.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.655+0000 7fe4427fc640 1 -- 172.21.15.154:0/3041833071 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe434031cf0 con 0x7fe4440ada30 2023-09-23T03:16:07.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.656+0000 7fe4493fb640 1 -- 172.21.15.154:0/3041833071 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4440ada30 msgr2=0x7fe4440ccaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:07.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.656+0000 7fe4493fb640 1 --2- 172.21.15.154:0/3041833071 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4440ada30 0x7fe4440ccaf0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe434009790 tx=0x7fe43402b8a0 comp rx=0 tx=0).stop 2023-09-23T03:16:07.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.656+0000 7fe4493fb640 1 -- 172.21.15.154:0/3041833071 shutdown_connections 2023-09-23T03:16:07.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.656+0000 7fe4493fb640 1 --2- 172.21.15.154:0/3041833071 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4440cf130 0x7fe44415d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:07.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.656+0000 7fe4493fb640 1 --2- 172.21.15.154:0/3041833071 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4440ada30 0x7fe4440ccaf0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:07.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.656+0000 7fe4493fb640 1 -- 172.21.15.154:0/3041833071 >> 172.21.15.154:0/3041833071 conn(0x7fe44406a0f0 msgr2=0x7fe44406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:07.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.656+0000 7fe4493fb640 1 -- 172.21.15.154:0/3041833071 shutdown_connections 2023-09-23T03:16:07.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.656+0000 7fe4493fb640 1 -- 172.21.15.154:0/3041833071 wait complete. 2023-09-23T03:16:07.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.658+0000 7fe4493fb640 1 Processor -- start 2023-09-23T03:16:07.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.658+0000 7fe4493fb640 1 -- start start 2023-09-23T03:16:07.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.659+0000 7fe4493fb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4440ada30 0x7fe4440c46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:07.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.659+0000 7fe4493fb640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4440b2fe0 0x7fe4440c2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:07.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.659+0000 7fe4493fb640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4440cf130 0x7fe444156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:07.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.659+0000 7fe4493fb640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4440c4c10 con 0x7fe4440ada30 2023-09-23T03:16:07.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.659+0000 7fe4493fb640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4440c4e40 con 0x7fe4440b2fe0 2023-09-23T03:16:07.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.659+0000 7fe4493fb640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4441557d0 con 0x7fe4440cf130 2023-09-23T03:16:07.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.659+0000 7fe442ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4440ada30 0x7fe4440c46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:07.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.659+0000 7fe442ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4440ada30 0x7fe4440c46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59938/0 (socket says 172.21.15.154:59938) 2023-09-23T03:16:07.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.659+0000 7fe442ffd640 1 -- 172.21.15.154:0/2979043285 learned_addr learned my addr 172.21.15.154:0/2979043285 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:07.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.660+0000 7fe442ffd640 1 -- 172.21.15.154:0/2979043285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4440cf130 msgr2=0x7fe444156590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:07.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.660+0000 7fe442ffd640 1 --2- 172.21.15.154:0/2979043285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4440cf130 0x7fe444156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:07.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.660+0000 7fe442ffd640 1 -- 172.21.15.154:0/2979043285 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4440b2fe0 msgr2=0x7fe4440c2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:07.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.660+0000 7fe442ffd640 1 --2- 172.21.15.154:0/2979043285 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4440b2fe0 0x7fe4440c2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:07.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.660+0000 7fe442ffd640 1 -- 172.21.15.154:0/2979043285 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4340093a0 con 0x7fe4440ada30 2023-09-23T03:16:07.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.660+0000 7fe442ffd640 1 --2- 172.21.15.154:0/2979043285 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4440ada30 0x7fe4440c46d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe4340098b0 tx=0x7fe4340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:07.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.660+0000 7fe43bfff640 1 -- 172.21.15.154:0/2979043285 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43403be00 con 0x7fe4440ada30 2023-09-23T03:16:07.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.660+0000 7fe43bfff640 1 -- 172.21.15.154:0/2979043285 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4340453f0 con 0x7fe4440ada30 2023-09-23T03:16:07.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.660+0000 7fe43bfff640 1 -- 172.21.15.154:0/2979043285 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4340457a0 con 0x7fe4440ada30 2023-09-23T03:16:07.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.660+0000 7fe4493fb640 1 -- 172.21.15.154:0/2979043285 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe444155ab0 con 0x7fe4440ada30 2023-09-23T03:16:07.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.661+0000 7fe4493fb640 1 -- 172.21.15.154:0/2979043285 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe444156070 con 0x7fe4440ada30 2023-09-23T03:16:07.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.662+0000 7fe4493fb640 1 -- 172.21.15.154:0/2979043285 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe444073eb0 con 0x7fe4440ada30 2023-09-23T03:16:07.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.663+0000 7fe43bfff640 1 -- 172.21.15.154:0/2979043285 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fe43404e450 con 0x7fe4440ada30 2023-09-23T03:16:07.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.664+0000 7fe43bfff640 1 --2- 172.21.15.154:0/2979043285 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe42406af60 0x7fe42406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:07.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.664+0000 7fe43bfff640 1 -- 172.21.15.154:0/2979043285 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7fe43402e070 con 0x7fe4440ada30 2023-09-23T03:16:07.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.664+0000 7fe43a7fc640 1 --2- 172.21.15.154:0/2979043285 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe42406af60 0x7fe42406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:07.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.665+0000 7fe43a7fc640 1 --2- 172.21.15.154:0/2979043285 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe42406af60 0x7fe42406d420 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe42c007fd0 tx=0x7fe42c001310 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:07.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.668+0000 7fe43bfff640 1 -- 172.21.15.154:0/2979043285 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fe434076d90 con 0x7fe4440ada30 2023-09-23T03:16:07.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.894+0000 7fe4493fb640 1 -- 172.21.15.154:0/2979043285 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe44406c6a0 con 0x7fe4440ada30 2023-09-23T03:16:07.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.895+0000 7fe43bfff640 1 -- 172.21.15.154:0/2979043285 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fe434035020 con 0x7fe4440ada30 2023-09-23T03:16:07.895 INFO:teuthology.orchestra.run.smithi154.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-09-23T03:16:07.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.897+0000 7fe4493fb640 1 -- 172.21.15.154:0/2979043285 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe42406af60 msgr2=0x7fe42406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:07.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.898+0000 7fe4493fb640 1 --2- 172.21.15.154:0/2979043285 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe42406af60 0x7fe42406d420 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe42c007fd0 tx=0x7fe42c001310 comp rx=0 tx=0).stop 2023-09-23T03:16:07.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.898+0000 7fe4493fb640 1 -- 172.21.15.154:0/2979043285 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4440ada30 msgr2=0x7fe4440c46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:07.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.898+0000 7fe4493fb640 1 --2- 172.21.15.154:0/2979043285 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4440ada30 0x7fe4440c46d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe4340098b0 tx=0x7fe4340046c0 comp rx=0 tx=0).stop 2023-09-23T03:16:07.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.898+0000 7fe4493fb640 1 -- 172.21.15.154:0/2979043285 shutdown_connections 2023-09-23T03:16:07.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.898+0000 7fe4493fb640 1 --2- 172.21.15.154:0/2979043285 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe42406af60 0x7fe42406d420 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:07.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.898+0000 7fe4493fb640 1 --2- 172.21.15.154:0/2979043285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4440cf130 0x7fe444156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:07.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.898+0000 7fe4493fb640 1 --2- 172.21.15.154:0/2979043285 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4440b2fe0 0x7fe4440c2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:07.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.898+0000 7fe4493fb640 1 --2- 172.21.15.154:0/2979043285 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4440ada30 0x7fe4440c46d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:07.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.898+0000 7fe4493fb640 1 -- 172.21.15.154:0/2979043285 >> 172.21.15.154:0/2979043285 conn(0x7fe44406a0f0 msgr2=0x7fe4440cc780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:07.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.898+0000 7fe4493fb640 1 -- 172.21.15.154:0/2979043285 shutdown_connections 2023-09-23T03:16:07.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:07.898+0000 7fe4493fb640 1 -- 172.21.15.154:0/2979043285 wait complete. 2023-09-23T03:16:07.915 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-23T03:16:07.915 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-23T03:16:07.915 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi154.front.sepia.ceph.com 2023-09-23T03:16:07.915 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-09-23T03:16:07.915 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-09-23T03:16:07.915 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-23T03:16:07.916 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-23T03:16:07.916 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-23T03:16:07.916 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-09-23T03:16:07.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:07.916 DEBUG:teuthology.orchestra.run.smithi154:> ip addr 2023-09-23T03:16:07.970 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-23T03:16:07.970 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-23T03:16:07.970 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2023-09-23T03:16:07.970 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:16:07.970 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2023-09-23T03:16:07.970 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:16:07.971 INFO:teuthology.orchestra.run.smithi154.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:16:07.971 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:54 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:16:07.971 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f0 2023-09-23T03:16:07.971 INFO:teuthology.orchestra.run.smithi154.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:16:07.971 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:55 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:16:07.971 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f1 2023-09-23T03:16:07.971 INFO:teuthology.orchestra.run.smithi154.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:16:07.971 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:60 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:16:07.971 INFO:teuthology.orchestra.run.smithi154.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-23T03:16:07.971 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:61 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:16:07.971 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.21.15.154/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-09-23T03:16:07.971 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft 42357sec preferred_lft 42357sec 2023-09-23T03:16:07.972 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::ec4:7aff:fe88:7761/64 scope link 2023-09-23T03:16:07.972 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:16:07.972 INFO:teuthology.orchestra.run.smithi154.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-23T03:16:07.972 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 5a:c6:b2:96:fb:dc brd ff:ff:ff:ff:ff:ff 2023-09-23T03:16:07.972 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-09-23T03:16:07.972 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:16:07.972 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::a801:22ff:febf:2833/64 scope link 2023-09-23T03:16:07.972 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:16:07.972 INFO:teuthology.orchestra.run.smithi154.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-23T03:16:07.972 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 5a:c6:b2:96:fb:dc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:16:07.972 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::58c6:b2ff:fe96:fbdc/64 scope link tentative 2023-09-23T03:16:07.972 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:16:07.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:07.973 DEBUG:teuthology.orchestra.run.smithi154:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-23T03:16:08.065 INFO:teuthology.orchestra.run.smithi154.stdout:1 2023-09-23T03:16:08.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:08.069 DEBUG:teuthology.orchestra.run.smithi154:> route 2023-09-23T03:16:08.122 INFO:teuthology.orchestra.run.smithi154.stdout:Kernel IP routing table 2023-09-23T03:16:08.122 INFO:teuthology.orchestra.run.smithi154.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-23T03:16:08.122 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-23T03:16:08.122 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-09-23T03:16:08.122 INFO:teuthology.orchestra.run.smithi154.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-23T03:16:08.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:08.123 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-09-23T03:16:08.123 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-09-23T03:16:08.124 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-23T03:16:08.124 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-23T03:16:08.124 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-23T03:16:08.124 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-09-23T03:16:08.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:08.316 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2023-09-23T03:16:08.370 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-09-23T03:16:08.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:08.371 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list-id 2023-09-23T03:16:08.421 INFO:teuthology.orchestra.run.smithi154.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-09-23T03:16:08.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:08.449 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-09-23T03:16:08.449 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-09-23T03:16:08.449 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:16:08.450 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-09-23T03:16:08.450 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-09-23T03:16:08.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:08.605 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-09-23T03:16:08.691 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-23T03:16:08.691 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-23T03:16:08.691 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2023-09-23T03:16:08.691 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:16:08.691 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2023-09-23T03:16:08.691 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:16:08.691 INFO:teuthology.orchestra.run.smithi154.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-23T03:16:08.691 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether be:fe:d0:a0:79:f3 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-09-23T03:16:08.691 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-09-23T03:16:08.691 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:16:08.692 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::bcfe:d0ff:fea0:79f3/64 scope link 2023-09-23T03:16:08.692 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:16:08.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:08.694 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-09-23T03:16:08.784 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-23T03:16:08.784 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-23T03:16:08.785 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2023-09-23T03:16:08.785 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:16:08.785 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2023-09-23T03:16:08.785 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:16:08.785 INFO:teuthology.orchestra.run.smithi154.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-23T03:16:08.785 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether be:fe:d0:a0:79:f3 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-09-23T03:16:08.785 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-09-23T03:16:08.785 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:16:08.785 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::bcfe:d0ff:fea0:79f3/64 scope link 2023-09-23T03:16:08.786 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:16:08.788 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-09-23T03:16:08.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:08.789 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-09-23T03:16:08.789 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-09-23T03:16:08.789 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-09-23T03:16:08.789 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-09-23T03:16:08.789 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-09-23T03:16:08.789 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-09-23T03:16:08.789 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-09-23T03:16:08.789 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-09-23T03:16:09.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:09.063 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-09-23T03:16:09.063 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-09-23T03:16:09.064 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set brx.1 up 2023-09-23T03:16:09.064 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set dev brx.1 master ceph-brx 2023-09-23T03:16:09.064 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-09-23T03:16:09.236 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-09-23T03:16:09.236 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-23T03:16:09.236 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-09-23T03:16:09.285 INFO:teuthology.orchestra.run.smithi154.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-09-23T03:16:09.285 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-23T03:16:09.285 DEBUG:teuthology.orchestra.run.smithi154:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-09-23T03:16:09.383 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-09-23T03:16:09.480 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2023-09-23T03:16:09.538 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2023-09-23T03:16:09.538 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2023-09-23T03:16:09.539 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2023-09-23T03:16:09.539 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2023-09-23T03:16:09.539 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2023-09-23T03:16:09.539 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2023-09-23T03:16:09.539 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2023-09-23T03:16:09.539 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2023-09-23T03:16:09.539 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2023-09-23T03:16:09.539 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2023-09-23T03:16:09.539 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2023-09-23T03:16:09.539 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T03:16:09.539 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2023-09-23T03:16:09.539 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2023-09-23T03:16:09.540 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2023-09-23T03:16:09.540 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2023-09-23T03:16:09.540 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2023-09-23T03:16:09.540 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2023-09-23T03:16:09.540 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2023-09-23T03:16:09.540 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-09-23T03:16:09.540 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-09-23T03:16:09.540 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2023-09-23T03:16:09.540 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2023-09-23T03:16:09.540 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2023-09-23T03:16:09.540 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-0 2023-09-23T03:16:09.540 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-1 2023-09-23T03:16:09.541 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-2 2023-09-23T03:16:09.541 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-3 2023-09-23T03:16:09.541 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2023-09-23T03:16:09.541 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:16:09.541 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:16:09.541 INFO:teuthology.orchestra.run.smithi154.stdout:/home/ubuntu/cephtest/mnt.0 2023-09-23T03:16:09.541 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:16:09.541 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:16:09.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:09.542 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-09-23T03:16:09.594 INFO:teuthology.orchestra.run.smithi154.stdout:39 2023-09-23T03:16:09.594 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [39] 2023-09-23T03:16:09.594 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-09-23T03:16:09.600 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-09-23T03:16:09.678 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2023-09-23T03:16:09.721 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2023-09-23T03:16:09.721 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2023-09-23T03:16:09.721 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2023-09-23T03:16:09.721 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2023-09-23T03:16:09.721 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2023-09-23T03:16:09.721 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2023-09-23T03:16:09.722 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2023-09-23T03:16:09.722 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2023-09-23T03:16:09.722 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2023-09-23T03:16:09.722 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2023-09-23T03:16:09.722 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2023-09-23T03:16:09.722 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T03:16:09.722 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2023-09-23T03:16:09.722 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2023-09-23T03:16:09.722 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2023-09-23T03:16:09.722 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2023-09-23T03:16:09.722 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2023-09-23T03:16:09.722 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2023-09-23T03:16:09.723 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2023-09-23T03:16:09.723 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-09-23T03:16:09.723 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-09-23T03:16:09.723 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2023-09-23T03:16:09.723 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2023-09-23T03:16:09.723 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2023-09-23T03:16:09.723 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-0 2023-09-23T03:16:09.723 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-1 2023-09-23T03:16:09.723 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-2 2023-09-23T03:16:09.723 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-3 2023-09-23T03:16:09.723 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2023-09-23T03:16:09.723 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:16:09.724 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:16:09.724 INFO:teuthology.orchestra.run.smithi154.stdout:/home/ubuntu/cephtest/mnt.0 2023-09-23T03:16:09.724 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:16:09.724 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:16:09.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:09.725 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-09-23T03:16:09.793 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi154.stderr:2023-09-23T03:16:09.793+0000 7f8d4283a480 -1 init, newargv = 0x5559e54c5820 newargc=15 2023-09-23T03:16:09.793 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi154.stderr:ceph-fuse[97334]: starting ceph client 2023-09-23T03:16:09.804 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi154.stderr:ceph-fuse[97334]: starting fuse 2023-09-23T03:16:09.812 INFO:teuthology.orchestra.run.smithi154.stdout:39 2023-09-23T03:16:09.813 INFO:teuthology.orchestra.run.smithi154.stdout:42 2023-09-23T03:16:09.813 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39, 42] 2023-09-23T03:16:09.813 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:16:09.813 DEBUG:teuthology.orchestra.run.smithi154:> import glob 2023-09-23T03:16:09.813 DEBUG:teuthology.orchestra.run.smithi154:> import re 2023-09-23T03:16:09.813 DEBUG:teuthology.orchestra.run.smithi154:> import os 2023-09-23T03:16:09.813 DEBUG:teuthology.orchestra.run.smithi154:> import subprocess 2023-09-23T03:16:09.813 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:16:09.814 DEBUG:teuthology.orchestra.run.smithi154:> def _find_admin_socket(client_name): 2023-09-23T03:16:09.814 DEBUG:teuthology.orchestra.run.smithi154:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-09-23T03:16:09.814 DEBUG:teuthology.orchestra.run.smithi154:> files = glob.glob(asok_path) 2023-09-23T03:16:09.814 DEBUG:teuthology.orchestra.run.smithi154:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-09-23T03:16:09.814 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:16:09.814 DEBUG:teuthology.orchestra.run.smithi154:> # Given a non-glob path, it better be there 2023-09-23T03:16:09.814 DEBUG:teuthology.orchestra.run.smithi154:> if "*" not in asok_path: 2023-09-23T03:16:09.814 DEBUG:teuthology.orchestra.run.smithi154:> assert(len(files) == 1) 2023-09-23T03:16:09.814 DEBUG:teuthology.orchestra.run.smithi154:> return files[0] 2023-09-23T03:16:09.814 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:16:09.814 DEBUG:teuthology.orchestra.run.smithi154:> for f in files: 2023-09-23T03:16:09.815 DEBUG:teuthology.orchestra.run.smithi154:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-23T03:16:09.815 DEBUG:teuthology.orchestra.run.smithi154:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-23T03:16:09.815 DEBUG:teuthology.orchestra.run.smithi154:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-23T03:16:09.815 DEBUG:teuthology.orchestra.run.smithi154:> contents = proc_f.read() 2023-09-23T03:16:09.815 DEBUG:teuthology.orchestra.run.smithi154:> if mountpoint in contents: 2023-09-23T03:16:09.815 DEBUG:teuthology.orchestra.run.smithi154:> return f 2023-09-23T03:16:09.815 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-23T03:16:09.815 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:16:09.815 DEBUG:teuthology.orchestra.run.smithi154:> print(_find_admin_socket("client.2")) 2023-09-23T03:16:09.815 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:16:10.021 INFO:teuthology.orchestra.run.smithi154.stdout:/var/run/ceph/ceph-client.2.97334.asok 2023-09-23T03:16:10.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:16:10.039 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.97334.asok 2023-09-23T03:16:10.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:10.040 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.97334.asok status 2023-09-23T03:16:10.284 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:16:10.284 INFO:teuthology.orchestra.run.smithi154.stdout: "metadata": { 2023-09-23T03:16:10.284 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_sha1": "862663913a74a99e1c52d50e6a7e59dd6705749a", 2023-09-23T03:16:10.284 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_version": "ceph version 18.0.0-6336-g86266391 (862663913a74a99e1c52d50e6a7e59dd6705749a) reef (dev)", 2023-09-23T03:16:10.284 INFO:teuthology.orchestra.run.smithi154.stdout: "entity_id": "2", 2023-09-23T03:16:10.284 INFO:teuthology.orchestra.run.smithi154.stdout: "hostname": "smithi154", 2023-09-23T03:16:10.284 INFO:teuthology.orchestra.run.smithi154.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-09-23T03:16:10.284 INFO:teuthology.orchestra.run.smithi154.stdout: "pid": "97334", 2023-09-23T03:16:10.285 INFO:teuthology.orchestra.run.smithi154.stdout: "root": "/" 2023-09-23T03:16:10.285 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:16:10.285 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_count": 0, 2023-09-23T03:16:10.285 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_pinned_count": 0, 2023-09-23T03:16:10.285 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 4836, 2023-09-23T03:16:10.285 INFO:teuthology.orchestra.run.smithi154.stdout: "inst": { 2023-09-23T03:16:10.285 INFO:teuthology.orchestra.run.smithi154.stdout: "name": { 2023-09-23T03:16:10.285 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "client", 2023-09-23T03:16:10.285 INFO:teuthology.orchestra.run.smithi154.stdout: "num": 4836 2023-09-23T03:16:10.285 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:16:10.285 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2023-09-23T03:16:10.285 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2023-09-23T03:16:10.285 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.2:0", 2023-09-23T03:16:10.286 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3709313506 2023-09-23T03:16:10.286 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:16:10.286 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:16:10.286 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2023-09-23T03:16:10.286 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2023-09-23T03:16:10.286 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.2:0", 2023-09-23T03:16:10.286 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3709313506 2023-09-23T03:16:10.286 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:16:10.286 INFO:teuthology.orchestra.run.smithi154.stdout: "inst_str": "client.4836 192.168.0.2:0/3709313506", 2023-09-23T03:16:10.286 INFO:teuthology.orchestra.run.smithi154.stdout: "addr_str": "192.168.0.2:0/3709313506", 2023-09-23T03:16:10.287 INFO:teuthology.orchestra.run.smithi154.stdout: "inode_count": 1, 2023-09-23T03:16:10.287 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_epoch": 9, 2023-09-23T03:16:10.287 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch": 26, 2023-09-23T03:16:10.287 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch_barrier": 0, 2023-09-23T03:16:10.287 INFO:teuthology.orchestra.run.smithi154.stdout: "blocklisted": false, 2023-09-23T03:16:10.287 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs" 2023-09-23T03:16:10.287 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:16:10.299 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-23T03:16:10.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.510+0000 7f649de2d640 1 Processor -- start 2023-09-23T03:16:10.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.511+0000 7f649de2d640 1 -- start start 2023-09-23T03:16:10.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.512+0000 7f649de2d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f649815cac0 0x7f649806b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:10.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.512+0000 7f649de2d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64980ce990 con 0x7f64980cf2a0 2023-09-23T03:16:10.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.512+0000 7f649de2d640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64980ce3d0 con 0x7f64980cdd80 2023-09-23T03:16:10.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.512+0000 7f649de2d640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64980ce1d0 con 0x7f649815cac0 2023-09-23T03:16:10.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.512+0000 7f6496ffd640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f64980cf2a0 0x7f64980cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:58572/0 (socket says 172.21.15.154:58572) 2023-09-23T03:16:10.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.512+0000 7f6496ffd640 1 -- 172.21.15.154:0/446321868 learned_addr learned my addr 172.21.15.154:0/446321868 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:10.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.513+0000 7f64967fc640 1 -- 172.21.15.154:0/446321868 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1871455711 0 0) 0x7f64980ce990 con 0x7f64980cf2a0 2023-09-23T03:16:10.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.513+0000 7f64967fc640 1 -- 172.21.15.154:0/446321868 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6484003520 con 0x7f64980cf2a0 2023-09-23T03:16:10.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.513+0000 7f64977fe640 1 --2- 172.21.15.154:0/446321868 >> v2:172.21.15.190:3300/0 conn(0x7f649815cac0 0x7f649806b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:10.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.513+0000 7f64967fc640 1 -- 172.21.15.154:0/446321868 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3776845358 0 0) 0x7f6484003520 con 0x7f64980cf2a0 2023-09-23T03:16:10.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.513+0000 7f64967fc640 1 -- 172.21.15.154:0/446321868 >> v2:172.21.15.190:3300/0 conn(0x7f649815cac0 msgr2=0x7f649806b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:10.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.513+0000 7f64967fc640 1 --2- 172.21.15.154:0/446321868 >> v2:172.21.15.190:3300/0 conn(0x7f649815cac0 0x7f649806b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:10.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.513+0000 7f64967fc640 1 -- 172.21.15.154:0/446321868 >> v1:172.21.15.190:6789/0 conn(0x7f64980cdd80 legacy=0x7f64980d0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:10.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.513+0000 7f64967fc640 1 -- 172.21.15.154:0/446321868 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64980ce6b0 con 0x7f64980cf2a0 2023-09-23T03:16:10.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.513+0000 7f64967fc640 1 -- 172.21.15.154:0/446321868 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f6480001670 con 0x7f64980cf2a0 2023-09-23T03:16:10.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.513+0000 7f64967fc640 1 -- 172.21.15.154:0/446321868 >> v1:172.21.15.154:6789/0 conn(0x7f64980cf2a0 legacy=0x7f64980cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:10.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.513+0000 7f64967fc640 1 --2- 172.21.15.154:0/446321868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6484003970 0x7f6484003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:10.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.513+0000 7f64967fc640 1 --2- 172.21.15.154:0/446321868 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64840047c0 0x7f6484004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:10.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.514+0000 7f64967fc640 1 --2- 172.21.15.154:0/446321868 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6484005180 0x7f648400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:10.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.514+0000 7f64967fc640 1 -- 172.21.15.154:0/446321868 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64980ce1d0 con 0x7f6484005180 2023-09-23T03:16:10.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.514+0000 7f64967fc640 1 -- 172.21.15.154:0/446321868 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648400c020 con 0x7f6484003970 2023-09-23T03:16:10.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.514+0000 7f64967fc640 1 -- 172.21.15.154:0/446321868 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f648400c1e0 con 0x7f64840047c0 2023-09-23T03:16:10.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.514+0000 7f6497fff640 1 --2- 172.21.15.154:0/446321868 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6484005180 0x7f648400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:10.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.514+0000 7f64977fe640 1 --2- 172.21.15.154:0/446321868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6484003970 0x7f6484003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:10.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.514+0000 7f6496ffd640 1 --2- 172.21.15.154:0/446321868 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64840047c0 0x7f6484004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:10.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.514+0000 7f6497fff640 1 -- 172.21.15.154:0/446321868 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64840047c0 msgr2=0x7f6484004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:10.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.514+0000 7f6497fff640 1 --2- 172.21.15.154:0/446321868 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64840047c0 0x7f6484004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:10.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.514+0000 7f6497fff640 1 -- 172.21.15.154:0/446321868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6484003970 msgr2=0x7f6484003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:10.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.514+0000 7f6497fff640 1 --2- 172.21.15.154:0/446321868 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6484003970 0x7f6484003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:10.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.514+0000 7f6497fff640 1 -- 172.21.15.154:0/446321868 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6484003520 con 0x7f6484005180 2023-09-23T03:16:10.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.515+0000 7f6497fff640 1 -- 172.21.15.154:0/446321868 reap_dead start 2023-09-23T03:16:10.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.515+0000 7f6497fff640 1 --2- 172.21.15.154:0/446321868 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6484005180 0x7f648400b980 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6490001750 tx=0x7f64840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=655801b35450c6f0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:10.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.515+0000 7f64967fc640 1 -- 172.21.15.154:0/446321868 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64900052d0 con 0x7f6484005180 2023-09-23T03:16:10.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.516+0000 7f64967fc640 1 -- 172.21.15.154:0/446321868 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f649002e050 con 0x7f6484005180 2023-09-23T03:16:10.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.516+0000 7f64967fc640 1 -- 172.21.15.154:0/446321868 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6490003830 con 0x7f6484005180 2023-09-23T03:16:10.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.516+0000 7f649de2d640 1 -- 172.21.15.154:0/446321868 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6484005180 msgr2=0x7f648400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:10.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.516+0000 7f649de2d640 1 --2- 172.21.15.154:0/446321868 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6484005180 0x7f648400b980 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6490001750 tx=0x7f64840057f0 comp rx=0 tx=0).stop 2023-09-23T03:16:10.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.516+0000 7f649de2d640 1 -- 172.21.15.154:0/446321868 shutdown_connections 2023-09-23T03:16:10.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.516+0000 7f649de2d640 1 --2- 172.21.15.154:0/446321868 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6484005180 0x7f648400b980 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:10.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.516+0000 7f649de2d640 1 -- 172.21.15.154:0/446321868 >> 172.21.15.154:0/446321868 conn(0x7f6498089fd0 msgr2=0x7f649809bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:10.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.516+0000 7f649de2d640 1 -- 172.21.15.154:0/446321868 shutdown_connections 2023-09-23T03:16:10.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.516+0000 7f649de2d640 1 -- 172.21.15.154:0/446321868 wait complete. 2023-09-23T03:16:10.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.518+0000 7f649de2d640 1 Processor -- start 2023-09-23T03:16:10.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.518+0000 7f649de2d640 1 -- start start 2023-09-23T03:16:10.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.519+0000 7f649de2d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6484005180 0x7f64980cc470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:10.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.519+0000 7f649de2d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64980b3310 0x7f6498159a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:10.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.519+0000 7f649de2d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64980b3790 0x7f6498159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:10.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.519+0000 7f649de2d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64980ca0c0 con 0x7f64980b3310 2023-09-23T03:16:10.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.519+0000 7f649de2d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64980ca3a0 con 0x7f64980b3790 2023-09-23T03:16:10.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.519+0000 7f649de2d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64980ca680 con 0x7f6484005180 2023-09-23T03:16:10.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.519+0000 7f6496ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64980b3310 0x7f6498159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:10.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.519+0000 7f6496ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64980b3310 0x7f6498159a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59954/0 (socket says 172.21.15.154:59954) 2023-09-23T03:16:10.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.519+0000 7f6496ffd640 1 -- 172.21.15.154:0/728420353 learned_addr learned my addr 172.21.15.154:0/728420353 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:10.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.520+0000 7f64977fe640 1 --2- 172.21.15.154:0/728420353 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6484005180 0x7f64980cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:10.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.520+0000 7f6497fff640 1 --2- 172.21.15.154:0/728420353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64980b3790 0x7f6498159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:10.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.520+0000 7f6496ffd640 1 -- 172.21.15.154:0/728420353 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6484005180 msgr2=0x7f64980cc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:10.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.520+0000 7f6496ffd640 1 --2- 172.21.15.154:0/728420353 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6484005180 0x7f64980cc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:10.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.520+0000 7f6496ffd640 1 -- 172.21.15.154:0/728420353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64980b3790 msgr2=0x7f6498159fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:10.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.520+0000 7f6496ffd640 1 --2- 172.21.15.154:0/728420353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64980b3790 0x7f6498159fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:10.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.520+0000 7f6496ffd640 1 -- 172.21.15.154:0/728420353 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64980ca960 con 0x7f64980b3310 2023-09-23T03:16:10.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.520+0000 7f6497fff640 1 --2- 172.21.15.154:0/728420353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64980b3790 0x7f6498159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:10.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.520+0000 7f64977fe640 1 --2- 172.21.15.154:0/728420353 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6484005180 0x7f64980cc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:10.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.520+0000 7f6496ffd640 1 --2- 172.21.15.154:0/728420353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64980b3310 0x7f6498159a90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f648000d6f0 tx=0x7f648000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:10.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.520+0000 7f648ffff640 1 -- 172.21.15.154:0/728420353 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64800072e0 con 0x7f64980b3310 2023-09-23T03:16:10.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.520+0000 7f648ffff640 1 -- 172.21.15.154:0/728420353 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6480007440 con 0x7f64980b3310 2023-09-23T03:16:10.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.520+0000 7f648ffff640 1 -- 172.21.15.154:0/728420353 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64800047e0 con 0x7f64980b3310 2023-09-23T03:16:10.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.520+0000 7f649de2d640 1 -- 172.21.15.154:0/728420353 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64980cc9b0 con 0x7f64980b3310 2023-09-23T03:16:10.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.521+0000 7f649de2d640 1 -- 172.21.15.154:0/728420353 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64980cb060 con 0x7f64980b3310 2023-09-23T03:16:10.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.522+0000 7f649de2d640 1 -- 172.21.15.154:0/728420353 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6498073eb0 con 0x7f64980b3310 2023-09-23T03:16:10.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.523+0000 7f648ffff640 1 -- 172.21.15.154:0/728420353 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f6480004940 con 0x7f64980b3310 2023-09-23T03:16:10.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.524+0000 7f648ffff640 1 --2- 172.21.15.154:0/728420353 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f647806af30 0x7f647806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:10.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.524+0000 7f648ffff640 1 -- 172.21.15.154:0/728420353 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f6480089380 con 0x7f64980b3310 2023-09-23T03:16:10.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.524+0000 7f64977fe640 1 --2- 172.21.15.154:0/728420353 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f647806af30 0x7f647806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:10.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.525+0000 7f64977fe640 1 --2- 172.21.15.154:0/728420353 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f647806af30 0x7f647806d3f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f64880073f0 tx=0x7f648800b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:10.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.528+0000 7f648ffff640 1 -- 172.21.15.154:0/728420353 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f6480057b30 con 0x7f64980b3310 2023-09-23T03:16:10.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.757+0000 7f649de2d640 1 -- 172.21.15.154:0/728420353 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f64980ccba0 con 0x7f64980b3310 2023-09-23T03:16:10.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.758+0000 7f648ffff640 1 -- 172.21.15.154:0/728420353 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f64980ccba0 con 0x7f64980b3310 2023-09-23T03:16:10.758 INFO:teuthology.orchestra.run.smithi154.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-09-23T03:16:10.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.760+0000 7f649de2d640 1 -- 172.21.15.154:0/728420353 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f647806af30 msgr2=0x7f647806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:10.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.760+0000 7f649de2d640 1 --2- 172.21.15.154:0/728420353 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f647806af30 0x7f647806d3f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f64880073f0 tx=0x7f648800b040 comp rx=0 tx=0).stop 2023-09-23T03:16:10.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.760+0000 7f649de2d640 1 -- 172.21.15.154:0/728420353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64980b3310 msgr2=0x7f6498159a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:10.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.760+0000 7f649de2d640 1 --2- 172.21.15.154:0/728420353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64980b3310 0x7f6498159a90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f648000d6f0 tx=0x7f648000dbb0 comp rx=0 tx=0).stop 2023-09-23T03:16:10.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.761+0000 7f649de2d640 1 -- 172.21.15.154:0/728420353 shutdown_connections 2023-09-23T03:16:10.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.761+0000 7f649de2d640 1 --2- 172.21.15.154:0/728420353 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f647806af30 0x7f647806d3f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:10.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.761+0000 7f649de2d640 1 --2- 172.21.15.154:0/728420353 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64980b3790 0x7f6498159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:10.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.761+0000 7f649de2d640 1 --2- 172.21.15.154:0/728420353 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64980b3310 0x7f6498159a90 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:10.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.761+0000 7f649de2d640 1 --2- 172.21.15.154:0/728420353 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6484005180 0x7f64980cc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:10.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.761+0000 7f649de2d640 1 -- 172.21.15.154:0/728420353 >> 172.21.15.154:0/728420353 conn(0x7f6498089fd0 msgr2=0x7f649815bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:10.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.761+0000 7f649de2d640 1 -- 172.21.15.154:0/728420353 shutdown_connections 2023-09-23T03:16:10.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:10.761+0000 7f649de2d640 1 -- 172.21.15.154:0/728420353 wait complete. 2023-09-23T03:16:10.779 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-23T03:16:10.779 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-23T03:16:10.779 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi190.front.sepia.ceph.com 2023-09-23T03:16:10.779 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-09-23T03:16:10.779 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-09-23T03:16:10.779 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-23T03:16:10.779 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-23T03:16:10.779 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-23T03:16:10.779 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-09-23T03:16:10.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:10.780 DEBUG:teuthology.orchestra.run.smithi190:> ip addr 2023-09-23T03:16:10.797 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:16:10.804 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:16:10.839 INFO:teuthology.orchestra.run.smithi190.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-23T03:16:10.839 INFO:teuthology.orchestra.run.smithi190.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-23T03:16:10.840 INFO:teuthology.orchestra.run.smithi190.stdout: inet 127.0.0.1/8 scope host lo 2023-09-23T03:16:10.840 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:16:10.840 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 ::1/128 scope host 2023-09-23T03:16:10.840 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:16:10.840 INFO:teuthology.orchestra.run.smithi190.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:16:10.840 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d0 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:16:10.840 INFO:teuthology.orchestra.run.smithi190.stdout: altname enp7s0f0 2023-09-23T03:16:10.840 INFO:teuthology.orchestra.run.smithi190.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:16:10.841 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d1 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:16:10.841 INFO:teuthology.orchestra.run.smithi190.stdout: altname enp7s0f1 2023-09-23T03:16:10.841 INFO:teuthology.orchestra.run.smithi190.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:16:10.841 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:62 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:16:10.841 INFO:teuthology.orchestra.run.smithi190.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-23T03:16:10.841 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:63 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:16:10.841 INFO:teuthology.orchestra.run.smithi190.stdout: inet 172.21.15.190/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-09-23T03:16:10.841 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft 42365sec preferred_lft 42365sec 2023-09-23T03:16:10.842 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::ec4:7aff:fe88:7763/64 scope link 2023-09-23T03:16:10.842 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:16:10.843 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-09-23T03:16:10.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:10.843 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec bash -c ' 2023-09-23T03:16:10.843 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2023-09-23T03:16:10.843 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link add name ceph-brx type bridge 2023-09-23T03:16:10.844 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip addr flush dev ceph-brx 2023-09-23T03:16:10.844 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set ceph-brx up 2023-09-23T03:16:10.844 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-09-23T03:16:10.844 DEBUG:teuthology.orchestra.run.smithi190:> ') 2023-09-23T03:16:11.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:11.342 DEBUG:teuthology.orchestra.run.smithi190:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-23T03:16:11.424 INFO:teuthology.orchestra.run.smithi190.stdout:1 2023-09-23T03:16:11.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:11.427 DEBUG:teuthology.orchestra.run.smithi190:> route 2023-09-23T03:16:11.479 INFO:teuthology.orchestra.run.smithi190.stdout:Kernel IP routing table 2023-09-23T03:16:11.479 INFO:teuthology.orchestra.run.smithi190.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-23T03:16:11.479 INFO:teuthology.orchestra.run.smithi190.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-23T03:16:11.480 INFO:teuthology.orchestra.run.smithi190.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-09-23T03:16:11.480 INFO:teuthology.orchestra.run.smithi190.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-23T03:16:11.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:11.480 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec bash -c ' 2023-09-23T03:16:11.480 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2023-09-23T03:16:11.480 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-23T03:16:11.480 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-23T03:16:11.480 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-23T03:16:11.481 DEBUG:teuthology.orchestra.run.smithi190:> ') 2023-09-23T03:16:12.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:12.172 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2023-09-23T03:16:12.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:12.228 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list-id 2023-09-23T03:16:12.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:12.281 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec bash -c ' 2023-09-23T03:16:12.281 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2023-09-23T03:16:12.282 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-09-23T03:16:12.282 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-09-23T03:16:12.282 DEBUG:teuthology.orchestra.run.smithi190:> ') 2023-09-23T03:16:12.425 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-09-23T03:16:12.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:12.425 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec bash -c ' 2023-09-23T03:16:12.425 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2023-09-23T03:16:12.425 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-09-23T03:16:12.426 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-09-23T03:16:12.426 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-09-23T03:16:12.426 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-09-23T03:16:12.426 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-09-23T03:16:12.426 DEBUG:teuthology.orchestra.run.smithi190:> ') 2023-09-23T03:16:12.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:12.707 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec bash -c ' 2023-09-23T03:16:12.707 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2023-09-23T03:16:12.707 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set brx.0 up 2023-09-23T03:16:12.707 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set dev brx.0 master ceph-brx 2023-09-23T03:16:12.708 DEBUG:teuthology.orchestra.run.smithi190:> ') 2023-09-23T03:16:12.872 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-09-23T03:16:12.872 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-23T03:16:12.872 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-09-23T03:16:12.921 INFO:teuthology.orchestra.run.smithi190.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-09-23T03:16:12.921 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-23T03:16:12.921 DEBUG:teuthology.orchestra.run.smithi190:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-09-23T03:16:12.968 DEBUG:teuthology.orchestra.run.smithi190:> sudo modprobe fuse 2023-09-23T03:16:13.057 DEBUG:teuthology.orchestra.run.smithi190:> cat /proc/self/mounts | awk '{print $2}' 2023-09-23T03:16:13.109 INFO:teuthology.orchestra.run.smithi190.stdout:/proc 2023-09-23T03:16:13.110 INFO:teuthology.orchestra.run.smithi190.stdout:/sys 2023-09-23T03:16:13.110 INFO:teuthology.orchestra.run.smithi190.stdout:/dev 2023-09-23T03:16:13.110 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/security 2023-09-23T03:16:13.110 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/shm 2023-09-23T03:16:13.110 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/pts 2023-09-23T03:16:13.110 INFO:teuthology.orchestra.run.smithi190.stdout:/run 2023-09-23T03:16:13.110 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup 2023-09-23T03:16:13.110 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/pstore 2023-09-23T03:16:13.110 INFO:teuthology.orchestra.run.smithi190.stdout:/ 2023-09-23T03:16:13.110 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/selinux 2023-09-23T03:16:13.111 INFO:teuthology.orchestra.run.smithi190.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T03:16:13.111 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/hugepages 2023-09-23T03:16:13.111 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/mqueue 2023-09-23T03:16:13.111 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/debug 2023-09-23T03:16:13.111 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/tracing 2023-09-23T03:16:13.111 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/fuse/connections 2023-09-23T03:16:13.111 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/config 2023-09-23T03:16:13.111 INFO:teuthology.orchestra.run.smithi190.stdout:/run/credentials/systemd-sysctl.service 2023-09-23T03:16:13.112 INFO:teuthology.orchestra.run.smithi190.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-09-23T03:16:13.112 INFO:teuthology.orchestra.run.smithi190.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-09-23T03:16:13.112 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/nfs/rpc_pipefs 2023-09-23T03:16:13.112 INFO:teuthology.orchestra.run.smithi190.stdout:/run/user/1000 2023-09-23T03:16:13.112 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/ceph 2023-09-23T03:16:13.112 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/ceph/osd/ceph-4 2023-09-23T03:16:13.112 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/ceph/osd/ceph-5 2023-09-23T03:16:13.112 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/ceph/osd/ceph-6 2023-09-23T03:16:13.112 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/ceph/osd/ceph-7 2023-09-23T03:16:13.112 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns 2023-09-23T03:16:13.112 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-09-23T03:16:13.112 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-09-23T03:16:13.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:13.114 DEBUG:teuthology.orchestra.run.smithi190:> ls /sys/fs/fuse/connections 2023-09-23T03:16:13.158 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-23T03:16:13.158 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-09-23T03:16:13.207 DEBUG:teuthology.orchestra.run.smithi190:> sudo modprobe fuse 2023-09-23T03:16:13.287 DEBUG:teuthology.orchestra.run.smithi190:> cat /proc/self/mounts | awk '{print $2}' 2023-09-23T03:16:13.332 INFO:teuthology.orchestra.run.smithi190.stdout:/proc 2023-09-23T03:16:13.332 INFO:teuthology.orchestra.run.smithi190.stdout:/sys 2023-09-23T03:16:13.332 INFO:teuthology.orchestra.run.smithi190.stdout:/dev 2023-09-23T03:16:13.332 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/security 2023-09-23T03:16:13.333 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/shm 2023-09-23T03:16:13.333 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/pts 2023-09-23T03:16:13.333 INFO:teuthology.orchestra.run.smithi190.stdout:/run 2023-09-23T03:16:13.333 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup 2023-09-23T03:16:13.333 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/pstore 2023-09-23T03:16:13.333 INFO:teuthology.orchestra.run.smithi190.stdout:/ 2023-09-23T03:16:13.333 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/selinux 2023-09-23T03:16:13.333 INFO:teuthology.orchestra.run.smithi190.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T03:16:13.333 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/hugepages 2023-09-23T03:16:13.333 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/mqueue 2023-09-23T03:16:13.333 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/debug 2023-09-23T03:16:13.333 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/tracing 2023-09-23T03:16:13.334 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/fuse/connections 2023-09-23T03:16:13.334 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/config 2023-09-23T03:16:13.334 INFO:teuthology.orchestra.run.smithi190.stdout:/run/credentials/systemd-sysctl.service 2023-09-23T03:16:13.334 INFO:teuthology.orchestra.run.smithi190.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-09-23T03:16:13.334 INFO:teuthology.orchestra.run.smithi190.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-09-23T03:16:13.334 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/nfs/rpc_pipefs 2023-09-23T03:16:13.334 INFO:teuthology.orchestra.run.smithi190.stdout:/run/user/1000 2023-09-23T03:16:13.334 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/ceph 2023-09-23T03:16:13.334 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/ceph/osd/ceph-4 2023-09-23T03:16:13.334 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/ceph/osd/ceph-5 2023-09-23T03:16:13.334 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/ceph/osd/ceph-6 2023-09-23T03:16:13.335 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/ceph/osd/ceph-7 2023-09-23T03:16:13.335 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns 2023-09-23T03:16:13.335 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-09-23T03:16:13.335 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-09-23T03:16:13.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:13.336 DEBUG:teuthology.orchestra.run.smithi190:> ls /sys/fs/fuse/connections 2023-09-23T03:16:13.407 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:2023-09-23T03:16:13.406+0000 7f111d63c480 -1 init, newargv = 0x56491d810d60 newargc=15 2023-09-23T03:16:13.408 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:ceph-fuse[92450]: starting ceph client 2023-09-23T03:16:13.419 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi190.stderr:ceph-fuse[92450]: starting fuse 2023-09-23T03:16:13.424 INFO:teuthology.orchestra.run.smithi190.stdout:39 2023-09-23T03:16:13.424 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-09-23T03:16:13.424 DEBUG:teuthology.orchestra.run.smithi190:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:16:13.424 DEBUG:teuthology.orchestra.run.smithi190:> import glob 2023-09-23T03:16:13.424 DEBUG:teuthology.orchestra.run.smithi190:> import re 2023-09-23T03:16:13.424 DEBUG:teuthology.orchestra.run.smithi190:> import os 2023-09-23T03:16:13.424 DEBUG:teuthology.orchestra.run.smithi190:> import subprocess 2023-09-23T03:16:13.425 DEBUG:teuthology.orchestra.run.smithi190:> 2023-09-23T03:16:13.425 DEBUG:teuthology.orchestra.run.smithi190:> def _find_admin_socket(client_name): 2023-09-23T03:16:13.425 DEBUG:teuthology.orchestra.run.smithi190:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-09-23T03:16:13.425 DEBUG:teuthology.orchestra.run.smithi190:> files = glob.glob(asok_path) 2023-09-23T03:16:13.425 DEBUG:teuthology.orchestra.run.smithi190:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-09-23T03:16:13.425 DEBUG:teuthology.orchestra.run.smithi190:> 2023-09-23T03:16:13.425 DEBUG:teuthology.orchestra.run.smithi190:> # Given a non-glob path, it better be there 2023-09-23T03:16:13.425 DEBUG:teuthology.orchestra.run.smithi190:> if "*" not in asok_path: 2023-09-23T03:16:13.425 DEBUG:teuthology.orchestra.run.smithi190:> assert(len(files) == 1) 2023-09-23T03:16:13.425 DEBUG:teuthology.orchestra.run.smithi190:> return files[0] 2023-09-23T03:16:13.425 DEBUG:teuthology.orchestra.run.smithi190:> 2023-09-23T03:16:13.425 DEBUG:teuthology.orchestra.run.smithi190:> for f in files: 2023-09-23T03:16:13.426 DEBUG:teuthology.orchestra.run.smithi190:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-23T03:16:13.426 DEBUG:teuthology.orchestra.run.smithi190:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-23T03:16:13.426 DEBUG:teuthology.orchestra.run.smithi190:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-23T03:16:13.426 DEBUG:teuthology.orchestra.run.smithi190:> contents = proc_f.read() 2023-09-23T03:16:13.426 DEBUG:teuthology.orchestra.run.smithi190:> if mountpoint in contents: 2023-09-23T03:16:13.426 DEBUG:teuthology.orchestra.run.smithi190:> return f 2023-09-23T03:16:13.426 DEBUG:teuthology.orchestra.run.smithi190:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-23T03:16:13.426 DEBUG:teuthology.orchestra.run.smithi190:> 2023-09-23T03:16:13.426 DEBUG:teuthology.orchestra.run.smithi190:> print(_find_admin_socket("client.1")) 2023-09-23T03:16:13.426 DEBUG:teuthology.orchestra.run.smithi190:> ' 2023-09-23T03:16:13.615 INFO:teuthology.orchestra.run.smithi190.stdout:/var/run/ceph/ceph-client.1.92450.asok 2023-09-23T03:16:13.620 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:16:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:16:13.632 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.92450.asok 2023-09-23T03:16:13.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:13.632 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.92450.asok status 2023-09-23T03:16:13.871 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2023-09-23T03:16:13.871 INFO:teuthology.orchestra.run.smithi190.stdout: "metadata": { 2023-09-23T03:16:13.871 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_sha1": "862663913a74a99e1c52d50e6a7e59dd6705749a", 2023-09-23T03:16:13.871 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_version": "ceph version 18.0.0-6336-g86266391 (862663913a74a99e1c52d50e6a7e59dd6705749a) reef (dev)", 2023-09-23T03:16:13.871 INFO:teuthology.orchestra.run.smithi190.stdout: "entity_id": "1", 2023-09-23T03:16:13.871 INFO:teuthology.orchestra.run.smithi190.stdout: "hostname": "smithi190", 2023-09-23T03:16:13.871 INFO:teuthology.orchestra.run.smithi190.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-09-23T03:16:13.871 INFO:teuthology.orchestra.run.smithi190.stdout: "pid": "92450", 2023-09-23T03:16:13.871 INFO:teuthology.orchestra.run.smithi190.stdout: "root": "/" 2023-09-23T03:16:13.872 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-09-23T03:16:13.872 INFO:teuthology.orchestra.run.smithi190.stdout: "dentry_count": 0, 2023-09-23T03:16:13.872 INFO:teuthology.orchestra.run.smithi190.stdout: "dentry_pinned_count": 0, 2023-09-23T03:16:13.872 INFO:teuthology.orchestra.run.smithi190.stdout: "id": 4669, 2023-09-23T03:16:13.872 INFO:teuthology.orchestra.run.smithi190.stdout: "inst": { 2023-09-23T03:16:13.872 INFO:teuthology.orchestra.run.smithi190.stdout: "name": { 2023-09-23T03:16:13.872 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "client", 2023-09-23T03:16:13.872 INFO:teuthology.orchestra.run.smithi190.stdout: "num": 4669 2023-09-23T03:16:13.872 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-09-23T03:16:13.872 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": { 2023-09-23T03:16:13.872 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "v1", 2023-09-23T03:16:13.872 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": "192.168.0.1:0", 2023-09-23T03:16:13.873 INFO:teuthology.orchestra.run.smithi190.stdout: "nonce": 44628323 2023-09-23T03:16:13.873 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-09-23T03:16:13.873 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-09-23T03:16:13.873 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": { 2023-09-23T03:16:13.873 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "v1", 2023-09-23T03:16:13.873 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": "192.168.0.1:0", 2023-09-23T03:16:13.873 INFO:teuthology.orchestra.run.smithi190.stdout: "nonce": 44628323 2023-09-23T03:16:13.873 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-09-23T03:16:13.873 INFO:teuthology.orchestra.run.smithi190.stdout: "inst_str": "client.4669 192.168.0.1:0/44628323", 2023-09-23T03:16:13.873 INFO:teuthology.orchestra.run.smithi190.stdout: "addr_str": "192.168.0.1:0/44628323", 2023-09-23T03:16:13.873 INFO:teuthology.orchestra.run.smithi190.stdout: "inode_count": 1, 2023-09-23T03:16:13.873 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_epoch": 9, 2023-09-23T03:16:13.874 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_epoch": 26, 2023-09-23T03:16:13.874 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_epoch_barrier": 0, 2023-09-23T03:16:13.874 INFO:teuthology.orchestra.run.smithi190.stdout: "blocklisted": false, 2023-09-23T03:16:13.874 INFO:teuthology.orchestra.run.smithi190.stdout: "fs_name": "cephfs" 2023-09-23T03:16:13.874 INFO:teuthology.orchestra.run.smithi190.stdout:} 2023-09-23T03:16:13.886 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-23T03:16:14.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.117+0000 7f5fb56f1640 1 Processor -- start 2023-09-23T03:16:14.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.117+0000 7f5fb56f1640 1 -- start start 2023-09-23T03:16:14.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.118+0000 7f5fb56f1640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5fb015a7f0 0x7f5fb006bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:14.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.118+0000 7f5fb56f1640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5fb00b2fd0 0x7f5fb006c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:14.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.118+0000 7f5fb56f1640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fb015abd0 con 0x7f5fb00cacd0 2023-09-23T03:16:14.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.118+0000 7f5fb56f1640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fb006c8b0 con 0x7f5fb015a7f0 2023-09-23T03:16:14.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.118+0000 7f5fb56f1640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb00b2b40 con 0x7f5fb00b2fd0 2023-09-23T03:16:14.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.119+0000 7f5faffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5fb015a7f0 0x7f5fb006bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:14.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.119+0000 7f5fb4ef0640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f5fb00cacd0 0x7f5fb015e300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:47372/0 (socket says 172.21.15.154:47372) 2023-09-23T03:16:14.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.119+0000 7f5fb4ef0640 1 -- 172.21.15.154:0/1014706929 learned_addr learned my addr 172.21.15.154:0/1014706929 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:14.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.119+0000 7f5faf7fe640 1 --2- 172.21.15.154:0/1014706929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5fb00b2fd0 0x7f5fb006c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:14.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.119+0000 7f5faffff640 1 -- 172.21.15.154:0/1014706929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5fb00b2fd0 msgr2=0x7f5fb006c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:14.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.119+0000 7f5faffff640 1 --2- 172.21.15.154:0/1014706929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5fb00b2fd0 0x7f5fb006c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:14.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.119+0000 7f5faffff640 1 -- 172.21.15.154:0/1014706929 >> v1:172.21.15.154:6789/0 conn(0x7f5fb00cacd0 legacy=0x7f5fb015e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:14.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.119+0000 7f5faffff640 1 -- 172.21.15.154:0/1014706929 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb00b1f70 con 0x7f5fb015a7f0 2023-09-23T03:16:14.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.119+0000 7f5faeffd640 1 -- 172.21.15.154:0/1014706929 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2453470363 0 0) 0x7f5fb015abd0 con 0x7f5fb00cacd0 2023-09-23T03:16:14.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.119+0000 7f5faf7fe640 1 --2- 172.21.15.154:0/1014706929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5fb00b2fd0 0x7f5fb006c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:14.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.120+0000 7f5faffff640 1 --2- 172.21.15.154:0/1014706929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fb015a7f0 0x7f5fb006bd60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5fa00097a0 tx=0x7f5fa002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75135a66bf06e72 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:14.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.120+0000 7f5faeffd640 1 -- 172.21.15.154:0/1014706929 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa002d040 con 0x7f5fb015a7f0 2023-09-23T03:16:14.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.121+0000 7f5faeffd640 1 -- 172.21.15.154:0/1014706929 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa0031810 con 0x7f5fb015a7f0 2023-09-23T03:16:14.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.121+0000 7f5faeffd640 1 -- 172.21.15.154:0/1014706929 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0031ba0 con 0x7f5fb015a7f0 2023-09-23T03:16:14.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.121+0000 7f5fb56f1640 1 -- 172.21.15.154:0/1014706929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fb015a7f0 msgr2=0x7f5fb006bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:14.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.121+0000 7f5fb56f1640 1 --2- 172.21.15.154:0/1014706929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fb015a7f0 0x7f5fb006bd60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5fa00097a0 tx=0x7f5fa002b8b0 comp rx=0 tx=0).stop 2023-09-23T03:16:14.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.121+0000 7f5fb56f1640 1 -- 172.21.15.154:0/1014706929 shutdown_connections 2023-09-23T03:16:14.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.121+0000 7f5fb56f1640 1 --2- 172.21.15.154:0/1014706929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5fb00b2fd0 0x7f5fb006c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:14.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.121+0000 7f5fb56f1640 1 --2- 172.21.15.154:0/1014706929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fb015a7f0 0x7f5fb006bd60 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:14.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.121+0000 7f5fb56f1640 1 -- 172.21.15.154:0/1014706929 >> 172.21.15.154:0/1014706929 conn(0x7f5fb0069eb0 msgr2=0x7f5fb006a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:14.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.121+0000 7f5fb56f1640 1 -- 172.21.15.154:0/1014706929 shutdown_connections 2023-09-23T03:16:14.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.121+0000 7f5fb56f1640 1 -- 172.21.15.154:0/1014706929 wait complete. 2023-09-23T03:16:14.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.123+0000 7f5fb56f1640 1 Processor -- start 2023-09-23T03:16:14.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.123+0000 7f5fb56f1640 1 -- start start 2023-09-23T03:16:14.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.125+0000 7f5fb56f1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fb00cacd0 0x7f5fb00c3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:14.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.125+0000 7f5fb56f1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fb015a7f0 0x7f5fb01487a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:14.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.125+0000 7f5fb56f1640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5fb00c4130 0x7f5fb0148ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:14.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.125+0000 7f5fb56f1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb00c6270 con 0x7f5fb015a7f0 2023-09-23T03:16:14.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.125+0000 7f5fb56f1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb00c5f20 con 0x7f5fb00cacd0 2023-09-23T03:16:14.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.125+0000 7f5fb56f1640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb00c7940 con 0x7f5fb00c4130 2023-09-23T03:16:14.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.125+0000 7f5faf7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fb015a7f0 0x7f5fb01487a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:14.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.125+0000 7f5faf7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fb015a7f0 0x7f5fb01487a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52998/0 (socket says 172.21.15.154:52998) 2023-09-23T03:16:14.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.125+0000 7f5faf7fe640 1 -- 172.21.15.154:0/424575813 learned_addr learned my addr 172.21.15.154:0/424575813 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:14.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.125+0000 7f5faffff640 1 --2- 172.21.15.154:0/424575813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fb00cacd0 0x7f5fb00c3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:14.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.125+0000 7f5fb4ef0640 1 --2- 172.21.15.154:0/424575813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5fb00c4130 0x7f5fb0148ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:14.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.125+0000 7f5faf7fe640 1 -- 172.21.15.154:0/424575813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5fb00c4130 msgr2=0x7f5fb0148ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:14.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.125+0000 7f5faf7fe640 1 --2- 172.21.15.154:0/424575813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5fb00c4130 0x7f5fb0148ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:14.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.125+0000 7f5faf7fe640 1 -- 172.21.15.154:0/424575813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fb00cacd0 msgr2=0x7f5fb00c3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:14.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.125+0000 7f5faf7fe640 1 --2- 172.21.15.154:0/424575813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fb00cacd0 0x7f5fb00c3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:14.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.125+0000 7f5faf7fe640 1 -- 172.21.15.154:0/424575813 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa0009400 con 0x7f5fb015a7f0 2023-09-23T03:16:14.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.125+0000 7f5faffff640 1 --2- 172.21.15.154:0/424575813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fb00cacd0 0x7f5fb00c3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:14.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.125+0000 7f5fb4ef0640 1 --2- 172.21.15.154:0/424575813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5fb00c4130 0x7f5fb0148ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:14.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.126+0000 7f5faf7fe640 1 --2- 172.21.15.154:0/424575813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fb015a7f0 0x7f5fb01487a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5fa400c7f0 tx=0x7f5fa400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:14.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.126+0000 7f5facff9640 1 -- 172.21.15.154:0/424575813 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa400cea0 con 0x7f5fb015a7f0 2023-09-23T03:16:14.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.126+0000 7f5facff9640 1 -- 172.21.15.154:0/424575813 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa4004230 con 0x7f5fb015a7f0 2023-09-23T03:16:14.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.126+0000 7f5facff9640 1 -- 172.21.15.154:0/424575813 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4007830 con 0x7f5fb015a7f0 2023-09-23T03:16:14.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.126+0000 7f5fb56f1640 1 -- 172.21.15.154:0/424575813 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fb00c45f0 con 0x7f5fb015a7f0 2023-09-23T03:16:14.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.126+0000 7f5fb56f1640 1 -- 172.21.15.154:0/424575813 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fb00c4bb0 con 0x7f5fb015a7f0 2023-09-23T03:16:14.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.127+0000 7f5fb56f1640 1 -- 172.21.15.154:0/424575813 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fb00c64e0 con 0x7f5fb015a7f0 2023-09-23T03:16:14.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.128+0000 7f5facff9640 1 -- 172.21.15.154:0/424575813 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f5fa4007990 con 0x7f5fb015a7f0 2023-09-23T03:16:14.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.129+0000 7f5facff9640 1 --2- 172.21.15.154:0/424575813 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5f9c06af30 0x7f5f9c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:14.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.129+0000 7f5facff9640 1 -- 172.21.15.154:0/424575813 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f5fa4089a40 con 0x7f5fb015a7f0 2023-09-23T03:16:14.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.129+0000 7f5faffff640 1 --2- 172.21.15.154:0/424575813 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5f9c06af30 0x7f5f9c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:14.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.130+0000 7f5faffff640 1 --2- 172.21.15.154:0/424575813 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5f9c06af30 0x7f5f9c06d3f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0036000 tx=0x7f5fa003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:14.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.134+0000 7f5facff9640 1 -- 172.21.15.154:0/424575813 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f5fa40546c0 con 0x7f5fb015a7f0 2023-09-23T03:16:14.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.365+0000 7f5fb56f1640 1 -- 172.21.15.154:0/424575813 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5fb004e920 con 0x7f5fb015a7f0 2023-09-23T03:16:14.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.366+0000 7f5facff9640 1 -- 172.21.15.154:0/424575813 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f5fa401e070 con 0x7f5fb015a7f0 2023-09-23T03:16:14.366 INFO:teuthology.orchestra.run.smithi154.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-09-23T03:16:14.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.368+0000 7f5fb56f1640 1 -- 172.21.15.154:0/424575813 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5f9c06af30 msgr2=0x7f5f9c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:14.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.368+0000 7f5fb56f1640 1 --2- 172.21.15.154:0/424575813 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5f9c06af30 0x7f5f9c06d3f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0036000 tx=0x7f5fa003c040 comp rx=0 tx=0).stop 2023-09-23T03:16:14.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.368+0000 7f5fb56f1640 1 -- 172.21.15.154:0/424575813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fb015a7f0 msgr2=0x7f5fb01487a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:14.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.368+0000 7f5fb56f1640 1 --2- 172.21.15.154:0/424575813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fb015a7f0 0x7f5fb01487a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5fa400c7f0 tx=0x7f5fa400ccb0 comp rx=0 tx=0).stop 2023-09-23T03:16:14.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.368+0000 7f5fb56f1640 1 -- 172.21.15.154:0/424575813 shutdown_connections 2023-09-23T03:16:14.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.368+0000 7f5fb56f1640 1 --2- 172.21.15.154:0/424575813 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5f9c06af30 0x7f5f9c06d3f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:14.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.368+0000 7f5fb56f1640 1 --2- 172.21.15.154:0/424575813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5fb00c4130 0x7f5fb0148ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:14.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.368+0000 7f5fb56f1640 1 --2- 172.21.15.154:0/424575813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fb015a7f0 0x7f5fb01487a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:14.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.368+0000 7f5fb56f1640 1 --2- 172.21.15.154:0/424575813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5fb00cacd0 0x7f5fb00c3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:14.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.368+0000 7f5fb56f1640 1 -- 172.21.15.154:0/424575813 >> 172.21.15.154:0/424575813 conn(0x7f5fb0069eb0 msgr2=0x7f5fb00ccf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:14.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.369+0000 7f5fb56f1640 1 -- 172.21.15.154:0/424575813 shutdown_connections 2023-09-23T03:16:14.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:14.369+0000 7f5fb56f1640 1 -- 172.21.15.154:0/424575813 wait complete. 2023-09-23T03:16:14.385 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-23T03:16:14.386 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-23T03:16:14.386 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi190.front.sepia.ceph.com 2023-09-23T03:16:14.386 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-09-23T03:16:14.386 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-09-23T03:16:14.386 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-23T03:16:14.386 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-23T03:16:14.386 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-23T03:16:14.386 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-09-23T03:16:14.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:14.387 DEBUG:teuthology.orchestra.run.smithi190:> ip addr 2023-09-23T03:16:14.448 INFO:teuthology.orchestra.run.smithi190.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-23T03:16:14.449 INFO:teuthology.orchestra.run.smithi190.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-23T03:16:14.449 INFO:teuthology.orchestra.run.smithi190.stdout: inet 127.0.0.1/8 scope host lo 2023-09-23T03:16:14.449 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:16:14.449 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 ::1/128 scope host 2023-09-23T03:16:14.449 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:16:14.450 INFO:teuthology.orchestra.run.smithi190.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:16:14.450 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d0 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:16:14.450 INFO:teuthology.orchestra.run.smithi190.stdout: altname enp7s0f0 2023-09-23T03:16:14.450 INFO:teuthology.orchestra.run.smithi190.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:16:14.450 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:d9:36:d1 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:16:14.451 INFO:teuthology.orchestra.run.smithi190.stdout: altname enp7s0f1 2023-09-23T03:16:14.451 INFO:teuthology.orchestra.run.smithi190.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:16:14.451 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:62 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:16:14.451 INFO:teuthology.orchestra.run.smithi190.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-23T03:16:14.451 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 0c:c4:7a:88:77:63 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:16:14.452 INFO:teuthology.orchestra.run.smithi190.stdout: inet 172.21.15.190/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-09-23T03:16:14.452 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft 42362sec preferred_lft 42362sec 2023-09-23T03:16:14.452 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::ec4:7aff:fe88:7763/64 scope link 2023-09-23T03:16:14.452 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:16:14.452 INFO:teuthology.orchestra.run.smithi190.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-23T03:16:14.452 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 06:66:43:a3:49:84 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:16:14.453 INFO:teuthology.orchestra.run.smithi190.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-09-23T03:16:14.453 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:16:14.453 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::80e6:8dff:feb0:8b24/64 scope link 2023-09-23T03:16:14.453 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:16:14.453 INFO:teuthology.orchestra.run.smithi190.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-23T03:16:14.454 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 06:66:43:a3:49:84 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-09-23T03:16:14.454 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::466:43ff:fea3:4984/64 scope link tentative 2023-09-23T03:16:14.454 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:16:14.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:14.454 DEBUG:teuthology.orchestra.run.smithi190:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-23T03:16:14.546 INFO:teuthology.orchestra.run.smithi190.stdout:1 2023-09-23T03:16:14.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:14.550 DEBUG:teuthology.orchestra.run.smithi190:> route 2023-09-23T03:16:14.603 INFO:teuthology.orchestra.run.smithi190.stdout:Kernel IP routing table 2023-09-23T03:16:14.603 INFO:teuthology.orchestra.run.smithi190.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-23T03:16:14.603 INFO:teuthology.orchestra.run.smithi190.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-23T03:16:14.603 INFO:teuthology.orchestra.run.smithi190.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-09-23T03:16:14.603 INFO:teuthology.orchestra.run.smithi190.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-23T03:16:14.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:14.604 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec bash -c ' 2023-09-23T03:16:14.604 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2023-09-23T03:16:14.604 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-23T03:16:14.604 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-23T03:16:14.604 DEBUG:teuthology.orchestra.run.smithi190:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-23T03:16:14.604 DEBUG:teuthology.orchestra.run.smithi190:> ') 2023-09-23T03:16:14.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:14.787 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2023-09-23T03:16:14.837 INFO:teuthology.orchestra.run.smithi190.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-09-23T03:16:14.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:14.839 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list-id 2023-09-23T03:16:14.890 INFO:teuthology.orchestra.run.smithi190.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-09-23T03:16:14.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:14.891 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec bash -c ' 2023-09-23T03:16:14.891 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2023-09-23T03:16:14.892 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-09-23T03:16:14.892 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-09-23T03:16:14.892 DEBUG:teuthology.orchestra.run.smithi190:> ') 2023-09-23T03:16:15.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:15.033 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-09-23T03:16:15.124 INFO:teuthology.orchestra.run.smithi190.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-23T03:16:15.125 INFO:teuthology.orchestra.run.smithi190.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-23T03:16:15.125 INFO:teuthology.orchestra.run.smithi190.stdout: inet 127.0.0.1/8 scope host lo 2023-09-23T03:16:15.125 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:16:15.125 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 ::1/128 scope host 2023-09-23T03:16:15.125 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:16:15.125 INFO:teuthology.orchestra.run.smithi190.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-23T03:16:15.125 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 02:4f:47:11:57:63 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-09-23T03:16:15.125 INFO:teuthology.orchestra.run.smithi190.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-09-23T03:16:15.125 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:16:15.125 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::4f:47ff:fe11:5763/64 scope link 2023-09-23T03:16:15.126 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:16:15.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:15.128 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-09-23T03:16:15.222 INFO:teuthology.orchestra.run.smithi190.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-23T03:16:15.223 INFO:teuthology.orchestra.run.smithi190.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-23T03:16:15.223 INFO:teuthology.orchestra.run.smithi190.stdout: inet 127.0.0.1/8 scope host lo 2023-09-23T03:16:15.223 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:16:15.223 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 ::1/128 scope host 2023-09-23T03:16:15.223 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:16:15.223 INFO:teuthology.orchestra.run.smithi190.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-23T03:16:15.223 INFO:teuthology.orchestra.run.smithi190.stdout: link/ether 02:4f:47:11:57:63 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-09-23T03:16:15.223 INFO:teuthology.orchestra.run.smithi190.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-09-23T03:16:15.223 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:16:15.224 INFO:teuthology.orchestra.run.smithi190.stdout: inet6 fe80::4f:47ff:fe11:5763/64 scope link 2023-09-23T03:16:15.224 INFO:teuthology.orchestra.run.smithi190.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:16:15.226 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-09-23T03:16:15.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:15.226 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec bash -c ' 2023-09-23T03:16:15.227 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2023-09-23T03:16:15.227 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-09-23T03:16:15.227 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-09-23T03:16:15.227 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-09-23T03:16:15.227 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-09-23T03:16:15.227 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-09-23T03:16:15.227 DEBUG:teuthology.orchestra.run.smithi190:> ') 2023-09-23T03:16:15.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:15.490 DEBUG:teuthology.orchestra.run.smithi190:> (cd / && exec bash -c ' 2023-09-23T03:16:15.490 DEBUG:teuthology.orchestra.run.smithi190:> set -e 2023-09-23T03:16:15.490 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set brx.1 up 2023-09-23T03:16:15.490 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link set dev brx.1 master ceph-brx 2023-09-23T03:16:15.490 DEBUG:teuthology.orchestra.run.smithi190:> ') 2023-09-23T03:16:15.651 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-09-23T03:16:15.651 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-23T03:16:15.651 DEBUG:teuthology.orchestra.run.smithi190:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-09-23T03:16:15.699 INFO:teuthology.orchestra.run.smithi190.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-09-23T03:16:15.699 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-23T03:16:15.699 DEBUG:teuthology.orchestra.run.smithi190:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-09-23T03:16:15.747 DEBUG:teuthology.orchestra.run.smithi190:> sudo modprobe fuse 2023-09-23T03:16:15.845 DEBUG:teuthology.orchestra.run.smithi190:> cat /proc/self/mounts | awk '{print $2}' 2023-09-23T03:16:15.906 INFO:teuthology.orchestra.run.smithi190.stdout:/proc 2023-09-23T03:16:15.906 INFO:teuthology.orchestra.run.smithi190.stdout:/sys 2023-09-23T03:16:15.907 INFO:teuthology.orchestra.run.smithi190.stdout:/dev 2023-09-23T03:16:15.907 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/security 2023-09-23T03:16:15.907 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/shm 2023-09-23T03:16:15.907 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/pts 2023-09-23T03:16:15.907 INFO:teuthology.orchestra.run.smithi190.stdout:/run 2023-09-23T03:16:15.907 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup 2023-09-23T03:16:15.907 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/pstore 2023-09-23T03:16:15.907 INFO:teuthology.orchestra.run.smithi190.stdout:/ 2023-09-23T03:16:15.907 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/selinux 2023-09-23T03:16:15.907 INFO:teuthology.orchestra.run.smithi190.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T03:16:15.907 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/hugepages 2023-09-23T03:16:15.908 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/mqueue 2023-09-23T03:16:15.908 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/debug 2023-09-23T03:16:15.908 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/tracing 2023-09-23T03:16:15.908 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/fuse/connections 2023-09-23T03:16:15.908 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/config 2023-09-23T03:16:15.908 INFO:teuthology.orchestra.run.smithi190.stdout:/run/credentials/systemd-sysctl.service 2023-09-23T03:16:15.908 INFO:teuthology.orchestra.run.smithi190.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-09-23T03:16:15.908 INFO:teuthology.orchestra.run.smithi190.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-09-23T03:16:15.908 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/nfs/rpc_pipefs 2023-09-23T03:16:15.908 INFO:teuthology.orchestra.run.smithi190.stdout:/run/user/1000 2023-09-23T03:16:15.908 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/ceph 2023-09-23T03:16:15.909 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/ceph/osd/ceph-4 2023-09-23T03:16:15.909 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/ceph/osd/ceph-5 2023-09-23T03:16:15.909 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/ceph/osd/ceph-6 2023-09-23T03:16:15.909 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/ceph/osd/ceph-7 2023-09-23T03:16:15.909 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns 2023-09-23T03:16:15.909 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-09-23T03:16:15.909 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-09-23T03:16:15.909 INFO:teuthology.orchestra.run.smithi190.stdout:/home/ubuntu/cephtest/mnt.1 2023-09-23T03:16:15.909 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-09-23T03:16:15.909 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-09-23T03:16:15.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:15.910 DEBUG:teuthology.orchestra.run.smithi190:> ls /sys/fs/fuse/connections 2023-09-23T03:16:15.966 INFO:teuthology.orchestra.run.smithi190.stdout:39 2023-09-23T03:16:15.966 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [39] 2023-09-23T03:16:15.967 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-09-23T03:16:15.972 DEBUG:teuthology.orchestra.run.smithi190:> sudo modprobe fuse 2023-09-23T03:16:16.052 DEBUG:teuthology.orchestra.run.smithi190:> cat /proc/self/mounts | awk '{print $2}' 2023-09-23T03:16:16.095 INFO:teuthology.orchestra.run.smithi190.stdout:/proc 2023-09-23T03:16:16.096 INFO:teuthology.orchestra.run.smithi190.stdout:/sys 2023-09-23T03:16:16.096 INFO:teuthology.orchestra.run.smithi190.stdout:/dev 2023-09-23T03:16:16.096 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/security 2023-09-23T03:16:16.096 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/shm 2023-09-23T03:16:16.096 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/pts 2023-09-23T03:16:16.096 INFO:teuthology.orchestra.run.smithi190.stdout:/run 2023-09-23T03:16:16.096 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/cgroup 2023-09-23T03:16:16.096 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/pstore 2023-09-23T03:16:16.096 INFO:teuthology.orchestra.run.smithi190.stdout:/ 2023-09-23T03:16:16.096 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/selinux 2023-09-23T03:16:16.097 INFO:teuthology.orchestra.run.smithi190.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T03:16:16.097 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/hugepages 2023-09-23T03:16:16.097 INFO:teuthology.orchestra.run.smithi190.stdout:/dev/mqueue 2023-09-23T03:16:16.097 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/debug 2023-09-23T03:16:16.097 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/tracing 2023-09-23T03:16:16.097 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/fs/fuse/connections 2023-09-23T03:16:16.097 INFO:teuthology.orchestra.run.smithi190.stdout:/sys/kernel/config 2023-09-23T03:16:16.097 INFO:teuthology.orchestra.run.smithi190.stdout:/run/credentials/systemd-sysctl.service 2023-09-23T03:16:16.097 INFO:teuthology.orchestra.run.smithi190.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-09-23T03:16:16.097 INFO:teuthology.orchestra.run.smithi190.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-09-23T03:16:16.097 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/nfs/rpc_pipefs 2023-09-23T03:16:16.098 INFO:teuthology.orchestra.run.smithi190.stdout:/run/user/1000 2023-09-23T03:16:16.098 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/ceph 2023-09-23T03:16:16.098 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/ceph/osd/ceph-4 2023-09-23T03:16:16.098 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/ceph/osd/ceph-5 2023-09-23T03:16:16.098 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/ceph/osd/ceph-6 2023-09-23T03:16:16.098 INFO:teuthology.orchestra.run.smithi190.stdout:/var/lib/ceph/osd/ceph-7 2023-09-23T03:16:16.098 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns 2023-09-23T03:16:16.098 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-09-23T03:16:16.098 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-09-23T03:16:16.098 INFO:teuthology.orchestra.run.smithi190.stdout:/home/ubuntu/cephtest/mnt.1 2023-09-23T03:16:16.098 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-09-23T03:16:16.098 INFO:teuthology.orchestra.run.smithi190.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-09-23T03:16:16.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:16.100 DEBUG:teuthology.orchestra.run.smithi190:> ls /sys/fs/fuse/connections 2023-09-23T03:16:16.166 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi190.stderr:2023-09-23T03:16:16.165+0000 7f48803f6480 -1 init, newargv = 0x555a02a4dcb0 newargc=15 2023-09-23T03:16:16.166 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi190.stderr:ceph-fuse[92902]: starting ceph client 2023-09-23T03:16:16.175 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi190.stderr:ceph-fuse[92902]: starting fuse 2023-09-23T03:16:16.192 INFO:teuthology.orchestra.run.smithi190.stdout:39 2023-09-23T03:16:16.192 INFO:teuthology.orchestra.run.smithi190.stdout:42 2023-09-23T03:16:16.192 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39, 42] 2023-09-23T03:16:16.193 DEBUG:teuthology.orchestra.run.smithi190:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:16:16.193 DEBUG:teuthology.orchestra.run.smithi190:> import glob 2023-09-23T03:16:16.193 DEBUG:teuthology.orchestra.run.smithi190:> import re 2023-09-23T03:16:16.193 DEBUG:teuthology.orchestra.run.smithi190:> import os 2023-09-23T03:16:16.193 DEBUG:teuthology.orchestra.run.smithi190:> import subprocess 2023-09-23T03:16:16.193 DEBUG:teuthology.orchestra.run.smithi190:> 2023-09-23T03:16:16.193 DEBUG:teuthology.orchestra.run.smithi190:> def _find_admin_socket(client_name): 2023-09-23T03:16:16.193 DEBUG:teuthology.orchestra.run.smithi190:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-09-23T03:16:16.193 DEBUG:teuthology.orchestra.run.smithi190:> files = glob.glob(asok_path) 2023-09-23T03:16:16.193 DEBUG:teuthology.orchestra.run.smithi190:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-09-23T03:16:16.193 DEBUG:teuthology.orchestra.run.smithi190:> 2023-09-23T03:16:16.194 DEBUG:teuthology.orchestra.run.smithi190:> # Given a non-glob path, it better be there 2023-09-23T03:16:16.194 DEBUG:teuthology.orchestra.run.smithi190:> if "*" not in asok_path: 2023-09-23T03:16:16.194 DEBUG:teuthology.orchestra.run.smithi190:> assert(len(files) == 1) 2023-09-23T03:16:16.194 DEBUG:teuthology.orchestra.run.smithi190:> return files[0] 2023-09-23T03:16:16.194 DEBUG:teuthology.orchestra.run.smithi190:> 2023-09-23T03:16:16.194 DEBUG:teuthology.orchestra.run.smithi190:> for f in files: 2023-09-23T03:16:16.194 DEBUG:teuthology.orchestra.run.smithi190:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-23T03:16:16.194 DEBUG:teuthology.orchestra.run.smithi190:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-23T03:16:16.194 DEBUG:teuthology.orchestra.run.smithi190:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-23T03:16:16.194 DEBUG:teuthology.orchestra.run.smithi190:> contents = proc_f.read() 2023-09-23T03:16:16.194 DEBUG:teuthology.orchestra.run.smithi190:> if mountpoint in contents: 2023-09-23T03:16:16.194 DEBUG:teuthology.orchestra.run.smithi190:> return f 2023-09-23T03:16:16.194 DEBUG:teuthology.orchestra.run.smithi190:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-23T03:16:16.195 DEBUG:teuthology.orchestra.run.smithi190:> 2023-09-23T03:16:16.195 DEBUG:teuthology.orchestra.run.smithi190:> print(_find_admin_socket("client.3")) 2023-09-23T03:16:16.195 DEBUG:teuthology.orchestra.run.smithi190:> ' 2023-09-23T03:16:16.397 INFO:teuthology.orchestra.run.smithi190.stdout:/var/run/ceph/ceph-client.3.92902.asok 2023-09-23T03:16:16.401 INFO:teuthology.orchestra.run.smithi190.stderr:2023-09-23T03:16:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:16:16.414 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.92902.asok 2023-09-23T03:16:16.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:16.414 DEBUG:teuthology.orchestra.run.smithi190:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.92902.asok status 2023-09-23T03:16:16.654 INFO:teuthology.orchestra.run.smithi190.stdout:{ 2023-09-23T03:16:16.654 INFO:teuthology.orchestra.run.smithi190.stdout: "metadata": { 2023-09-23T03:16:16.654 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_sha1": "862663913a74a99e1c52d50e6a7e59dd6705749a", 2023-09-23T03:16:16.655 INFO:teuthology.orchestra.run.smithi190.stdout: "ceph_version": "ceph version 18.0.0-6336-g86266391 (862663913a74a99e1c52d50e6a7e59dd6705749a) reef (dev)", 2023-09-23T03:16:16.655 INFO:teuthology.orchestra.run.smithi190.stdout: "entity_id": "3", 2023-09-23T03:16:16.655 INFO:teuthology.orchestra.run.smithi190.stdout: "hostname": "smithi190", 2023-09-23T03:16:16.655 INFO:teuthology.orchestra.run.smithi190.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-09-23T03:16:16.655 INFO:teuthology.orchestra.run.smithi190.stdout: "pid": "92902", 2023-09-23T03:16:16.655 INFO:teuthology.orchestra.run.smithi190.stdout: "root": "/" 2023-09-23T03:16:16.655 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-09-23T03:16:16.655 INFO:teuthology.orchestra.run.smithi190.stdout: "dentry_count": 0, 2023-09-23T03:16:16.655 INFO:teuthology.orchestra.run.smithi190.stdout: "dentry_pinned_count": 0, 2023-09-23T03:16:16.655 INFO:teuthology.orchestra.run.smithi190.stdout: "id": 4529, 2023-09-23T03:16:16.656 INFO:teuthology.orchestra.run.smithi190.stdout: "inst": { 2023-09-23T03:16:16.656 INFO:teuthology.orchestra.run.smithi190.stdout: "name": { 2023-09-23T03:16:16.656 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "client", 2023-09-23T03:16:16.656 INFO:teuthology.orchestra.run.smithi190.stdout: "num": 4529 2023-09-23T03:16:16.656 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-09-23T03:16:16.656 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": { 2023-09-23T03:16:16.656 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "v1", 2023-09-23T03:16:16.656 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": "192.168.0.2:0", 2023-09-23T03:16:16.656 INFO:teuthology.orchestra.run.smithi190.stdout: "nonce": 3398726499 2023-09-23T03:16:16.656 INFO:teuthology.orchestra.run.smithi190.stdout: } 2023-09-23T03:16:16.657 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-09-23T03:16:16.657 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": { 2023-09-23T03:16:16.657 INFO:teuthology.orchestra.run.smithi190.stdout: "type": "v1", 2023-09-23T03:16:16.657 INFO:teuthology.orchestra.run.smithi190.stdout: "addr": "192.168.0.2:0", 2023-09-23T03:16:16.657 INFO:teuthology.orchestra.run.smithi190.stdout: "nonce": 3398726499 2023-09-23T03:16:16.657 INFO:teuthology.orchestra.run.smithi190.stdout: }, 2023-09-23T03:16:16.657 INFO:teuthology.orchestra.run.smithi190.stdout: "inst_str": "client.4529 192.168.0.2:0/3398726499", 2023-09-23T03:16:16.657 INFO:teuthology.orchestra.run.smithi190.stdout: "addr_str": "192.168.0.2:0/3398726499", 2023-09-23T03:16:16.657 INFO:teuthology.orchestra.run.smithi190.stdout: "inode_count": 1, 2023-09-23T03:16:16.657 INFO:teuthology.orchestra.run.smithi190.stdout: "mds_epoch": 9, 2023-09-23T03:16:16.657 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_epoch": 26, 2023-09-23T03:16:16.658 INFO:teuthology.orchestra.run.smithi190.stdout: "osd_epoch_barrier": 0, 2023-09-23T03:16:16.658 INFO:teuthology.orchestra.run.smithi190.stdout: "blocklisted": false, 2023-09-23T03:16:16.658 INFO:teuthology.orchestra.run.smithi190.stdout: "fs_name": "cephfs" 2023-09-23T03:16:16.658 INFO:teuthology.orchestra.run.smithi190.stdout:} 2023-09-23T03:16:16.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:16.669 DEBUG:teuthology.orchestra.run.smithi154:> stat --file-system '--printf=%T 2023-09-23T03:16:16.669 DEBUG:teuthology.orchestra.run.smithi154:> ' -- /home/ubuntu/cephtest/mnt.0 2023-09-23T03:16:16.729 INFO:teuthology.orchestra.run.smithi154.stdout:fuseblk 2023-09-23T03:16:16.729 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-09-23T03:16:16.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:16.729 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-09-23T03:16:16.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:16.830 DEBUG:teuthology.orchestra.run.smithi154:> stat --file-system '--printf=%T 2023-09-23T03:16:16.831 DEBUG:teuthology.orchestra.run.smithi154:> ' -- /home/ubuntu/cephtest/mnt.2 2023-09-23T03:16:16.889 INFO:teuthology.orchestra.run.smithi154.stdout:fuseblk 2023-09-23T03:16:16.889 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-09-23T03:16:16.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:16.890 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-09-23T03:16:16.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:16.990 DEBUG:teuthology.orchestra.run.smithi190:> stat --file-system '--printf=%T 2023-09-23T03:16:16.990 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.1 2023-09-23T03:16:17.050 INFO:teuthology.orchestra.run.smithi190.stdout:fuseblk 2023-09-23T03:16:17.050 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-09-23T03:16:17.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:17.050 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-09-23T03:16:17.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:17.150 DEBUG:teuthology.orchestra.run.smithi190:> stat --file-system '--printf=%T 2023-09-23T03:16:17.150 DEBUG:teuthology.orchestra.run.smithi190:> ' -- /home/ubuntu/cephtest/mnt.3 2023-09-23T03:16:17.204 INFO:teuthology.orchestra.run.smithi190.stdout:fuseblk 2023-09-23T03:16:17.205 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-09-23T03:16:17.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:17.205 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-09-23T03:16:17.298 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-09-23T03:16:17.312 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:16:17.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.539+0000 7f40a937c640 1 Processor -- start 2023-09-23T03:16:17.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.539+0000 7f40a937c640 1 -- start start 2023-09-23T03:16:17.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.541+0000 7f40a937c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f40a40acce0 0x7f40a415ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:17.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.541+0000 7f40a937c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40a40b1b20 0x7f40a40ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:17.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.541+0000 7f40a937c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f40a40cee70 0x7f40a406cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:17.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.541+0000 7f40a937c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40a40b2e90 con 0x7f40a40acce0 2023-09-23T03:16:17.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.541+0000 7f40a937c640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40a40cf320 con 0x7f40a40cee70 2023-09-23T03:16:17.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.541+0000 7f40a937c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a415ac30 con 0x7f40a40b1b20 2023-09-23T03:16:17.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.541+0000 7f40a2ffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f40a40acce0 0x7f40a415ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:17.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.541+0000 7f40a2ffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f40a40acce0 0x7f40a415ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53000/0 (socket says 172.21.15.154:53000) 2023-09-23T03:16:17.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.541+0000 7f40a2ffd640 1 -- 172.21.15.154:0/3978451309 learned_addr learned my addr 172.21.15.154:0/3978451309 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:17.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.541+0000 7f40a37fe640 1 --2- 172.21.15.154:0/3978451309 >> v2:172.21.15.190:3300/0 conn(0x7f40a40cee70 0x7f40a406cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:17.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.541+0000 7f40a27fc640 1 --2- 172.21.15.154:0/3978451309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40a40b1b20 0x7f40a40ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:17.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.542+0000 7f40a2ffd640 1 -- 172.21.15.154:0/3978451309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40a40b1b20 msgr2=0x7f40a40ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:17.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.542+0000 7f40a2ffd640 1 --2- 172.21.15.154:0/3978451309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40a40b1b20 0x7f40a40ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:17.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.542+0000 7f40a2ffd640 1 -- 172.21.15.154:0/3978451309 >> v2:172.21.15.190:3300/0 conn(0x7f40a40cee70 msgr2=0x7f40a406cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:17.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.542+0000 7f40a2ffd640 1 --2- 172.21.15.154:0/3978451309 >> v2:172.21.15.190:3300/0 conn(0x7f40a40cee70 0x7f40a406cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:17.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.542+0000 7f40a2ffd640 1 -- 172.21.15.154:0/3978451309 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a40b1f70 con 0x7f40a40acce0 2023-09-23T03:16:17.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.543+0000 7f40a2ffd640 1 --2- 172.21.15.154:0/3978451309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40a40acce0 0x7f40a415ed60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f40a40c9150 tx=0x7f409402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2066aa9de72e91 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:17.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.543+0000 7f40a1ffb640 1 -- 172.21.15.154:0/3978451309 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409402e070 con 0x7f40a40acce0 2023-09-23T03:16:17.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.543+0000 7f40a1ffb640 1 -- 172.21.15.154:0/3978451309 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4094004030 con 0x7f40a40acce0 2023-09-23T03:16:17.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.543+0000 7f40a1ffb640 1 -- 172.21.15.154:0/3978451309 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4094004420 con 0x7f40a40acce0 2023-09-23T03:16:17.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.544+0000 7f40a937c640 1 -- 172.21.15.154:0/3978451309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40a40acce0 msgr2=0x7f40a415ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:17.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.544+0000 7f40a937c640 1 --2- 172.21.15.154:0/3978451309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40a40acce0 0x7f40a415ed60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f40a40c9150 tx=0x7f409402b840 comp rx=0 tx=0).stop 2023-09-23T03:16:17.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.544+0000 7f40a937c640 1 -- 172.21.15.154:0/3978451309 shutdown_connections 2023-09-23T03:16:17.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.544+0000 7f40a937c640 1 --2- 172.21.15.154:0/3978451309 >> v2:172.21.15.190:3300/0 conn(0x7f40a40cee70 0x7f40a406cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:17.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.544+0000 7f40a937c640 1 --2- 172.21.15.154:0/3978451309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40a40b1b20 0x7f40a40ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:17.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.544+0000 7f40a937c640 1 --2- 172.21.15.154:0/3978451309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40a40acce0 0x7f40a415ed60 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:17.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.544+0000 7f40a937c640 1 -- 172.21.15.154:0/3978451309 >> 172.21.15.154:0/3978451309 conn(0x7f40a406a0f0 msgr2=0x7f40a406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:17.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.544+0000 7f40a937c640 1 -- 172.21.15.154:0/3978451309 shutdown_connections 2023-09-23T03:16:17.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.544+0000 7f40a937c640 1 -- 172.21.15.154:0/3978451309 wait complete. 2023-09-23T03:16:17.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.546+0000 7f40a937c640 1 Processor -- start 2023-09-23T03:16:17.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.546+0000 7f40a937c640 1 -- start start 2023-09-23T03:16:17.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.547+0000 7f40a937c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40a40acce0 0x7f40a40cd360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:17.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.547+0000 7f40a937c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40a40b1b20 0x7f40a415ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:17.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.547+0000 7f40a937c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40a40cee70 0x7f40a415b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:17.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.547+0000 7f40a937c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a40c2d50 con 0x7f40a40cee70 2023-09-23T03:16:17.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.547+0000 7f40a937c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a415b8c0 con 0x7f40a40b1b20 2023-09-23T03:16:17.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.547+0000 7f40a937c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a40c20d0 con 0x7f40a40acce0 2023-09-23T03:16:17.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.547+0000 7f40a37fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40a40cee70 0x7f40a415b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:17.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.547+0000 7f40a37fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40a40cee70 0x7f40a415b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53016/0 (socket says 172.21.15.154:53016) 2023-09-23T03:16:17.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.547+0000 7f40a37fe640 1 -- 172.21.15.154:0/3999770231 learned_addr learned my addr 172.21.15.154:0/3999770231 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:17.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.547+0000 7f40a2ffd640 1 --2- 172.21.15.154:0/3999770231 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40a40acce0 0x7f40a40cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:17.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.547+0000 7f40a27fc640 1 --2- 172.21.15.154:0/3999770231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40a40b1b20 0x7f40a415ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:17.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.548+0000 7f40a37fe640 1 -- 172.21.15.154:0/3999770231 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40a40acce0 msgr2=0x7f40a40cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:17.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.548+0000 7f40a37fe640 1 --2- 172.21.15.154:0/3999770231 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40a40acce0 0x7f40a40cd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:17.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.548+0000 7f40a37fe640 1 -- 172.21.15.154:0/3999770231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40a40b1b20 msgr2=0x7f40a415ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:17.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.548+0000 7f40a37fe640 1 --2- 172.21.15.154:0/3999770231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40a40b1b20 0x7f40a415ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:17.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.548+0000 7f40a37fe640 1 -- 172.21.15.154:0/3999770231 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4094009460 con 0x7f40a40cee70 2023-09-23T03:16:17.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.548+0000 7f40a37fe640 1 --2- 172.21.15.154:0/3999770231 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40a40cee70 0x7f40a415b380 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f409c007e60 tx=0x7f409c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:17.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.548+0000 7f408b7fe640 1 -- 172.21.15.154:0/3999770231 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c014420 con 0x7f40a40cee70 2023-09-23T03:16:17.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.548+0000 7f408b7fe640 1 -- 172.21.15.154:0/3999770231 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409c0145a0 con 0x7f40a40cee70 2023-09-23T03:16:17.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.548+0000 7f408b7fe640 1 -- 172.21.15.154:0/3999770231 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c014950 con 0x7f40a40cee70 2023-09-23T03:16:17.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.548+0000 7f40a937c640 1 -- 172.21.15.154:0/3999770231 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a40c23b0 con 0x7f40a40cee70 2023-09-23T03:16:17.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.548+0000 7f40a937c640 1 -- 172.21.15.154:0/3999770231 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a40c2970 con 0x7f40a40cee70 2023-09-23T03:16:17.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.550+0000 7f40a937c640 1 -- 172.21.15.154:0/3999770231 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40a4073eb0 con 0x7f40a40cee70 2023-09-23T03:16:17.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.550+0000 7f408b7fe640 1 -- 172.21.15.154:0/3999770231 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f409c014ab0 con 0x7f40a40cee70 2023-09-23T03:16:17.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.551+0000 7f408b7fe640 1 --2- 172.21.15.154:0/3999770231 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f409006af30 0x7f409006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:17.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.551+0000 7f408b7fe640 1 -- 172.21.15.154:0/3999770231 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f409c08a310 con 0x7f40a40cee70 2023-09-23T03:16:17.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.552+0000 7f40a2ffd640 1 --2- 172.21.15.154:0/3999770231 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f409006af30 0x7f409006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:17.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.552+0000 7f40a2ffd640 1 --2- 172.21.15.154:0/3999770231 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f409006af30 0x7f409006d3f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4094034000 tx=0x7f4094008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:17.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.556+0000 7f408b7fe640 1 -- 172.21.15.154:0/3999770231 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f409c054fa0 con 0x7f40a40cee70 2023-09-23T03:16:17.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.749+0000 7f40a937c640 1 -- 172.21.15.154:0/3999770231 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40a40b2aa0 con 0x7f40a40cee70 2023-09-23T03:16:17.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.751+0000 7f408b7fe640 1 -- 172.21.15.154:0/3999770231 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18744 (secure 0 0 0) 0x7f409c0585c0 con 0x7f40a40cee70 2023-09-23T03:16:17.752 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:16:17.752 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":26,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:16:03.151371+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:15:18.575849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T03:15:19.707212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T03:15:23.337138+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:16:17.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.753+0000 7f40a937c640 1 -- 172.21.15.154:0/3999770231 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f409006af30 msgr2=0x7f409006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:17.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.754+0000 7f40a937c640 1 --2- 172.21.15.154:0/3999770231 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f409006af30 0x7f409006d3f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4094034000 tx=0x7f4094008030 comp rx=0 tx=0).stop 2023-09-23T03:16:17.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.754+0000 7f40a937c640 1 -- 172.21.15.154:0/3999770231 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40a40cee70 msgr2=0x7f40a415b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:17.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.754+0000 7f40a937c640 1 --2- 172.21.15.154:0/3999770231 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40a40cee70 0x7f40a415b380 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f409c007e60 tx=0x7f409c00e4b0 comp rx=0 tx=0).stop 2023-09-23T03:16:17.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.754+0000 7f40a937c640 1 -- 172.21.15.154:0/3999770231 shutdown_connections 2023-09-23T03:16:17.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.754+0000 7f40a937c640 1 --2- 172.21.15.154:0/3999770231 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f409006af30 0x7f409006d3f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:17.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.754+0000 7f40a937c640 1 --2- 172.21.15.154:0/3999770231 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40a40cee70 0x7f40a415b380 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:17.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.754+0000 7f40a937c640 1 --2- 172.21.15.154:0/3999770231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40a40b1b20 0x7f40a415ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:17.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.754+0000 7f40a937c640 1 --2- 172.21.15.154:0/3999770231 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40a40acce0 0x7f40a40cd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:17.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.754+0000 7f40a937c640 1 -- 172.21.15.154:0/3999770231 >> 172.21.15.154:0/3999770231 conn(0x7f40a406a0f0 msgr2=0x7f40a40affe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:17.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.754+0000 7f40a937c640 1 -- 172.21.15.154:0/3999770231 shutdown_connections 2023-09-23T03:16:17.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.754+0000 7f40a937c640 1 -- 172.21.15.154:0/3999770231 wait complete. 2023-09-23T03:16:17.774 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:15:18.575849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:15:19.707212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-23T03:15:23.337138+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-23T03:16:17.774 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:16:17.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.996+0000 7fc174ff5640 1 Processor -- start 2023-09-23T03:16:17.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.996+0000 7fc174ff5640 1 -- start start 2023-09-23T03:16:17.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.998+0000 7fc174ff5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc17015aba0 0x7fc17015bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:17.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.998+0000 7fc174ff5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc17015e5f0 0x7fc17015ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:17.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.998+0000 7fc174ff5640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc17015b6a0 con 0x7fc1700caea0 2023-09-23T03:16:17.999 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.998+0000 7fc174ff5640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc17015f0b0 con 0x7fc17015e5f0 2023-09-23T03:16:17.999 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.998+0000 7fc174ff5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc17015f390 con 0x7fc17015aba0 2023-09-23T03:16:17.999 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.998+0000 7fc167fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc17015e5f0 0x7fc17015ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:17.999 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.998+0000 7fc167fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc17015e5f0 0x7fc17015ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53022/0 (socket says 172.21.15.154:53022) 2023-09-23T03:16:17.999 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.998+0000 7fc167fff640 1 -- 172.21.15.154:0/2207437430 learned_addr learned my addr 172.21.15.154:0/2207437430 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:17.999 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.998+0000 7fc16e59c640 1 --2- 172.21.15.154:0/2207437430 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc17015aba0 0x7fc17015bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:18.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.999+0000 7fc167fff640 1 -- 172.21.15.154:0/2207437430 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc17015aba0 msgr2=0x7fc17015bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:18.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.999+0000 7fc167fff640 1 --2- 172.21.15.154:0/2207437430 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc17015aba0 0x7fc17015bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:18.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.999+0000 7fc167fff640 1 -- 172.21.15.154:0/2207437430 >> v1:172.21.15.190:6789/0 conn(0x7fc1700caea0 legacy=0x7fc1700cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:18.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:17.999+0000 7fc167fff640 1 -- 172.21.15.154:0/2207437430 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17015b330 con 0x7fc17015e5f0 2023-09-23T03:16:18.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.000+0000 7fc167fff640 1 --2- 172.21.15.154:0/2207437430 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc17015e5f0 0x7fc17015ea30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc158009870 tx=0x7fc15802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a09dfe57c6d4fdbc server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:18.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.000+0000 7fc16dd9b640 1 -- 172.21.15.154:0/2207437430 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc158030840 con 0x7fc17015e5f0 2023-09-23T03:16:18.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.000+0000 7fc16dd9b640 1 -- 172.21.15.154:0/2207437430 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1580309a0 con 0x7fc17015e5f0 2023-09-23T03:16:18.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.000+0000 7fc16dd9b640 1 -- 172.21.15.154:0/2207437430 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc158030d30 con 0x7fc17015e5f0 2023-09-23T03:16:18.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.001+0000 7fc174ff5640 1 -- 172.21.15.154:0/2207437430 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc17015e5f0 msgr2=0x7fc17015ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:18.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.001+0000 7fc174ff5640 1 --2- 172.21.15.154:0/2207437430 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc17015e5f0 0x7fc17015ea30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc158009870 tx=0x7fc15802b980 comp rx=0 tx=0).stop 2023-09-23T03:16:18.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.001+0000 7fc174ff5640 1 -- 172.21.15.154:0/2207437430 shutdown_connections 2023-09-23T03:16:18.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.001+0000 7fc174ff5640 1 --2- 172.21.15.154:0/2207437430 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc17015e5f0 0x7fc17015ea30 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:18.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.001+0000 7fc174ff5640 1 --2- 172.21.15.154:0/2207437430 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc17015aba0 0x7fc17015bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:18.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.001+0000 7fc174ff5640 1 -- 172.21.15.154:0/2207437430 >> 172.21.15.154:0/2207437430 conn(0x7fc17006c010 msgr2=0x7fc17006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:18.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.001+0000 7fc174ff5640 1 -- 172.21.15.154:0/2207437430 shutdown_connections 2023-09-23T03:16:18.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.001+0000 7fc174ff5640 1 -- 172.21.15.154:0/2207437430 wait complete. 2023-09-23T03:16:18.003 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.003+0000 7fc174ff5640 1 Processor -- start 2023-09-23T03:16:18.003 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.003+0000 7fc174ff5640 1 -- start start 2023-09-23T03:16:18.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.004+0000 7fc174ff5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1700caea0 0x7fc170153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:18.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.004+0000 7fc174ff5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc17015aba0 0x7fc1701f3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:18.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.004+0000 7fc174ff5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc17015e5f0 0x7fc1701f2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:18.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.004+0000 7fc174ff5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc170155560 con 0x7fc1700caea0 2023-09-23T03:16:18.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.004+0000 7fc174ff5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc170155800 con 0x7fc17015aba0 2023-09-23T03:16:18.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.004+0000 7fc174ff5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1701f29d0 con 0x7fc17015e5f0 2023-09-23T03:16:18.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.004+0000 7fc16e59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1700caea0 0x7fc170153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:18.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.004+0000 7fc16e59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1700caea0 0x7fc170153cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53032/0 (socket says 172.21.15.154:53032) 2023-09-23T03:16:18.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.004+0000 7fc16e59c640 1 -- 172.21.15.154:0/448539675 learned_addr learned my addr 172.21.15.154:0/448539675 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:18.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.005+0000 7fc16e59c640 1 -- 172.21.15.154:0/448539675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc17015e5f0 msgr2=0x7fc1701f2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:18.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.005+0000 7fc16e59c640 1 --2- 172.21.15.154:0/448539675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc17015e5f0 0x7fc1701f2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:18.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.005+0000 7fc16e59c640 1 -- 172.21.15.154:0/448539675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc17015aba0 msgr2=0x7fc1701f3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:18.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.005+0000 7fc16e59c640 1 --2- 172.21.15.154:0/448539675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc17015aba0 0x7fc1701f3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:18.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.005+0000 7fc16e59c640 1 -- 172.21.15.154:0/448539675 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1580094d0 con 0x7fc1700caea0 2023-09-23T03:16:18.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.005+0000 7fc16e59c640 1 --2- 172.21.15.154:0/448539675 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1700caea0 0x7fc170153cc0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc16000d800 tx=0x7fc16000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:18.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.005+0000 7fc166ffd640 1 -- 172.21.15.154:0/448539675 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16000b6c0 con 0x7fc1700caea0 2023-09-23T03:16:18.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.005+0000 7fc166ffd640 1 -- 172.21.15.154:0/448539675 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16000b820 con 0x7fc1700caea0 2023-09-23T03:16:18.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.005+0000 7fc166ffd640 1 -- 172.21.15.154:0/448539675 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16000bbb0 con 0x7fc1700caea0 2023-09-23T03:16:18.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.005+0000 7fc174ff5640 1 -- 172.21.15.154:0/448539675 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc17015b100 con 0x7fc1700caea0 2023-09-23T03:16:18.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.006+0000 7fc174ff5640 1 -- 172.21.15.154:0/448539675 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1701f2f90 con 0x7fc1700caea0 2023-09-23T03:16:18.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.007+0000 7fc174ff5640 1 -- 172.21.15.154:0/448539675 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1701f3120 con 0x7fc1700caea0 2023-09-23T03:16:18.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.008+0000 7fc166ffd640 1 -- 172.21.15.154:0/448539675 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fc16000bd10 con 0x7fc1700caea0 2023-09-23T03:16:18.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.009+0000 7fc166ffd640 1 --2- 172.21.15.154:0/448539675 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc15006af60 0x7fc15006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:18.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.009+0000 7fc166ffd640 1 -- 172.21.15.154:0/448539675 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7fc16008a100 con 0x7fc1700caea0 2023-09-23T03:16:18.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.009+0000 7fc167fff640 1 --2- 172.21.15.154:0/448539675 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc15006af60 0x7fc15006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:18.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.010+0000 7fc167fff640 1 --2- 172.21.15.154:0/448539675 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc15006af60 0x7fc15006d420 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc158036000 tx=0x7fc15803b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:18.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.013+0000 7fc166ffd640 1 -- 172.21.15.154:0/448539675 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fc160054cf0 con 0x7fc1700caea0 2023-09-23T03:16:18.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.201+0000 7fc174ff5640 1 -- 172.21.15.154:0/448539675 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc17004e920 con 0x7fc1700caea0 2023-09-23T03:16:18.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.202+0000 7fc166ffd640 1 -- 172.21.15.154:0/448539675 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc160058310 con 0x7fc1700caea0 2023-09-23T03:16:18.202 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:16:18.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.204+0000 7fc174ff5640 1 -- 172.21.15.154:0/448539675 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc15006af60 msgr2=0x7fc15006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:18.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.204+0000 7fc174ff5640 1 --2- 172.21.15.154:0/448539675 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc15006af60 0x7fc15006d420 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc158036000 tx=0x7fc15803b040 comp rx=0 tx=0).stop 2023-09-23T03:16:18.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.204+0000 7fc174ff5640 1 -- 172.21.15.154:0/448539675 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1700caea0 msgr2=0x7fc170153cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:18.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.204+0000 7fc174ff5640 1 --2- 172.21.15.154:0/448539675 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1700caea0 0x7fc170153cc0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc16000d800 tx=0x7fc16000dcc0 comp rx=0 tx=0).stop 2023-09-23T03:16:18.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.204+0000 7fc174ff5640 1 -- 172.21.15.154:0/448539675 shutdown_connections 2023-09-23T03:16:18.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.204+0000 7fc174ff5640 1 --2- 172.21.15.154:0/448539675 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc15006af60 0x7fc15006d420 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:18.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.204+0000 7fc174ff5640 1 --2- 172.21.15.154:0/448539675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc17015e5f0 0x7fc1701f2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:18.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.204+0000 7fc174ff5640 1 --2- 172.21.15.154:0/448539675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc17015aba0 0x7fc1701f3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:18.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.204+0000 7fc174ff5640 1 --2- 172.21.15.154:0/448539675 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1700caea0 0x7fc170153cc0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:18.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.204+0000 7fc174ff5640 1 -- 172.21.15.154:0/448539675 >> 172.21.15.154:0/448539675 conn(0x7fc17006c010 msgr2=0x7fc170159e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:18.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.204+0000 7fc174ff5640 1 -- 172.21.15.154:0/448539675 shutdown_connections 2023-09-23T03:16:18.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.204+0000 7fc174ff5640 1 -- 172.21.15.154:0/448539675 wait complete. 2023-09-23T03:16:18.223 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:16:18.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.443+0000 7fef23797640 1 Processor -- start 2023-09-23T03:16:18.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.444+0000 7fef23797640 1 -- start start 2023-09-23T03:16:18.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.445+0000 7fef23797640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef1c0ce940 0x7fef1c1677f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:18.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.445+0000 7fef23797640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef1c0d0bb0 con 0x7fef1c15cda0 2023-09-23T03:16:18.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.445+0000 7fef23797640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef1c0cd450 con 0x7fef1c15c9c0 2023-09-23T03:16:18.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.445+0000 7fef23797640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c0d0df0 con 0x7fef1c0ce940 2023-09-23T03:16:18.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.445+0000 7fef20d32640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fef1c15cda0 0x7fef1c0d2620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:47376/0 (socket says 172.21.15.154:47376) 2023-09-23T03:16:18.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.446+0000 7fef20d32640 1 -- 172.21.15.154:0/2262433796 learned_addr learned my addr 172.21.15.154:0/2262433796 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:18.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.446+0000 7fef1bfff640 1 -- 172.21.15.154:0/2262433796 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2578731698 0 0) 0x7fef08001110 con 0x7fef1c15cda0 2023-09-23T03:16:18.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.446+0000 7fef1bfff640 1 -- 172.21.15.154:0/2262433796 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef0c003520 con 0x7fef1c15cda0 2023-09-23T03:16:18.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.446+0000 7fef21d34640 1 --2- 172.21.15.154:0/2262433796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef1c0ce940 0x7fef1c1677f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:18.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.446+0000 7fef1bfff640 1 -- 172.21.15.154:0/2262433796 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3371838712 0 0) 0x7fef0c003520 con 0x7fef1c15cda0 2023-09-23T03:16:18.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.446+0000 7fef1bfff640 1 -- 172.21.15.154:0/2262433796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef1c0ce940 msgr2=0x7fef1c1677f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:18.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.446+0000 7fef1bfff640 1 --2- 172.21.15.154:0/2262433796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef1c0ce940 0x7fef1c1677f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:18.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.446+0000 7fef1bfff640 1 -- 172.21.15.154:0/2262433796 >> v1:172.21.15.190:6789/0 conn(0x7fef1c15c9c0 legacy=0x7fef1c0d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:18.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.446+0000 7fef1bfff640 1 -- 172.21.15.154:0/2262433796 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef1c0d10d0 con 0x7fef1c15cda0 2023-09-23T03:16:18.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.446+0000 7fef1bfff640 1 -- 172.21.15.154:0/2262433796 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fef08001820 con 0x7fef1c15cda0 2023-09-23T03:16:18.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.446+0000 7fef21d34640 1 --2- 172.21.15.154:0/2262433796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef1c0ce940 0x7fef1c1677f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:16:18.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.446+0000 7fef1bfff640 1 -- 172.21.15.154:0/2262433796 >> v1:172.21.15.154:6789/0 conn(0x7fef1c15cda0 legacy=0x7fef1c0d2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:18.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.447+0000 7fef1bfff640 1 --2- 172.21.15.154:0/2262433796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef1c0ce940 0x7fef0c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:18.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.447+0000 7fef1bfff640 1 --2- 172.21.15.154:0/2262433796 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef0c003f90 0x7fef0c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:18.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.447+0000 7fef1bfff640 1 --2- 172.21.15.154:0/2262433796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef0c004d00 0x7fef0c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:18.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.447+0000 7fef1bfff640 1 -- 172.21.15.154:0/2262433796 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c0d0df0 con 0x7fef0c003f90 2023-09-23T03:16:18.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.447+0000 7fef1bfff640 1 -- 172.21.15.154:0/2262433796 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c00bb70 con 0x7fef0c004d00 2023-09-23T03:16:18.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.447+0000 7fef1bfff640 1 -- 172.21.15.154:0/2262433796 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c00bd30 con 0x7fef1c0ce940 2023-09-23T03:16:18.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.447+0000 7fef20d32640 1 --2- 172.21.15.154:0/2262433796 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef0c003f90 0x7fef0c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:18.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.447+0000 7fef21d34640 1 --2- 172.21.15.154:0/2262433796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef0c004d00 0x7fef0c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:18.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.447+0000 7fef21533640 1 --2- 172.21.15.154:0/2262433796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef1c0ce940 0x7fef0c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:18.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.447+0000 7fef20d32640 1 -- 172.21.15.154:0/2262433796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef1c0ce940 msgr2=0x7fef0c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:18.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.447+0000 7fef20d32640 1 --2- 172.21.15.154:0/2262433796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef1c0ce940 0x7fef0c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:18.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.447+0000 7fef20d32640 1 -- 172.21.15.154:0/2262433796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef0c004d00 msgr2=0x7fef0c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:18.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.447+0000 7fef20d32640 1 --2- 172.21.15.154:0/2262433796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef0c004d00 0x7fef0c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:18.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.447+0000 7fef20d32640 1 -- 172.21.15.154:0/2262433796 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef0c003520 con 0x7fef0c003f90 2023-09-23T03:16:18.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.448+0000 7fef20d32640 1 --2- 172.21.15.154:0/2262433796 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef0c003f90 0x7fef0c0047c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fef0800c210 tx=0x7fef0c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec2ead78285802e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:18.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.449+0000 7fef1bfff640 1 -- 172.21.15.154:0/2262433796 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef08005ad0 con 0x7fef0c003f90 2023-09-23T03:16:18.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.449+0000 7fef1bfff640 1 -- 172.21.15.154:0/2262433796 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef08005c30 con 0x7fef0c003f90 2023-09-23T03:16:18.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.449+0000 7fef1bfff640 1 -- 172.21.15.154:0/2262433796 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0802c870 con 0x7fef0c003f90 2023-09-23T03:16:18.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.449+0000 7fef23797640 1 -- 172.21.15.154:0/2262433796 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef0c003f90 msgr2=0x7fef0c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:18.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.449+0000 7fef23797640 1 --2- 172.21.15.154:0/2262433796 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef0c003f90 0x7fef0c0047c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fef0800c210 tx=0x7fef0c0057f0 comp rx=0 tx=0).stop 2023-09-23T03:16:18.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.449+0000 7fef21d34640 1 -- 172.21.15.154:0/2262433796 reap_dead start 2023-09-23T03:16:18.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.449+0000 7fef23797640 1 -- 172.21.15.154:0/2262433796 shutdown_connections 2023-09-23T03:16:18.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.449+0000 7fef23797640 1 -- 172.21.15.154:0/2262433796 >> 172.21.15.154:0/2262433796 conn(0x7fef1c09bc20 msgr2=0x7fef1c089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:18.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.450+0000 7fef23797640 1 -- 172.21.15.154:0/2262433796 shutdown_connections 2023-09-23T03:16:18.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.450+0000 7fef23797640 1 -- 172.21.15.154:0/2262433796 wait complete. 2023-09-23T03:16:18.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.451+0000 7fef23797640 1 Processor -- start 2023-09-23T03:16:18.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.451+0000 7fef23797640 1 -- start start 2023-09-23T03:16:18.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.452+0000 7fef23797640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef0c003f90 0x7fef1c15fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:18.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.452+0000 7fef23797640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef1c1f5df0 0x7fef1c1519c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:18.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.452+0000 7fef23797640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef1c1f6270 0x7fef1c151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:18.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.452+0000 7fef23797640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c15dca0 con 0x7fef1c1f6270 2023-09-23T03:16:18.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.452+0000 7fef23797640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c152440 con 0x7fef0c003f90 2023-09-23T03:16:18.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.452+0000 7fef23797640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c1600c0 con 0x7fef1c1f5df0 2023-09-23T03:16:18.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.453+0000 7fef21d34640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef1c1f6270 0x7fef1c151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:18.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.453+0000 7fef21d34640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef1c1f6270 0x7fef1c151f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53048/0 (socket says 172.21.15.154:53048) 2023-09-23T03:16:18.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.453+0000 7fef21d34640 1 -- 172.21.15.154:0/1847483264 learned_addr learned my addr 172.21.15.154:0/1847483264 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:18.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.453+0000 7fef21533640 1 --2- 172.21.15.154:0/1847483264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef0c003f90 0x7fef1c15fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:18.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.453+0000 7fef20d32640 1 --2- 172.21.15.154:0/1847483264 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef1c1f5df0 0x7fef1c1519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:18.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.453+0000 7fef21d34640 1 -- 172.21.15.154:0/1847483264 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef1c1f5df0 msgr2=0x7fef1c1519c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:18.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.453+0000 7fef21d34640 1 --2- 172.21.15.154:0/1847483264 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef1c1f5df0 0x7fef1c1519c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:18.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.453+0000 7fef21d34640 1 -- 172.21.15.154:0/1847483264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef0c003f90 msgr2=0x7fef1c15fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:18.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.453+0000 7fef21d34640 1 --2- 172.21.15.154:0/1847483264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef0c003f90 0x7fef1c15fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:18.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.453+0000 7fef21d34640 1 -- 172.21.15.154:0/1847483264 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef1c15def0 con 0x7fef1c1f6270 2023-09-23T03:16:18.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.453+0000 7fef20d32640 1 --2- 172.21.15.154:0/1847483264 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef1c1f5df0 0x7fef1c1519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:18.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.453+0000 7fef21533640 1 --2- 172.21.15.154:0/1847483264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef0c003f90 0x7fef1c15fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:18.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.454+0000 7fef21d34640 1 --2- 172.21.15.154:0/1847483264 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef1c1f6270 0x7fef1c151f00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fef1c0d39d0 tx=0x7fef14010930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:18.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.454+0000 7fef19ffb640 1 -- 172.21.15.154:0/1847483264 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef14004440 con 0x7fef1c1f6270 2023-09-23T03:16:18.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.454+0000 7fef19ffb640 1 -- 172.21.15.154:0/1847483264 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef14003610 con 0x7fef1c1f6270 2023-09-23T03:16:18.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.454+0000 7fef19ffb640 1 -- 172.21.15.154:0/1847483264 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef140039c0 con 0x7fef1c1f6270 2023-09-23T03:16:18.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.454+0000 7fef23797640 1 -- 172.21.15.154:0/1847483264 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef1c15e1d0 con 0x7fef1c1f6270 2023-09-23T03:16:18.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.454+0000 7fef23797640 1 -- 172.21.15.154:0/1847483264 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef1c15e790 con 0x7fef1c1f6270 2023-09-23T03:16:18.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.455+0000 7fef23797640 1 -- 172.21.15.154:0/1847483264 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef1c06cdc0 con 0x7fef1c1f6270 2023-09-23T03:16:18.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.456+0000 7fef19ffb640 1 -- 172.21.15.154:0/1847483264 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fef14003b20 con 0x7fef1c1f6270 2023-09-23T03:16:18.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.457+0000 7fef19ffb640 1 --2- 172.21.15.154:0/1847483264 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fef0006af30 0x7fef0006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:18.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.457+0000 7fef19ffb640 1 -- 172.21.15.154:0/1847483264 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7fef14089b50 con 0x7fef1c1f6270 2023-09-23T03:16:18.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.457+0000 7fef21533640 1 --2- 172.21.15.154:0/1847483264 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fef0006af30 0x7fef0006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:18.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.458+0000 7fef21533640 1 --2- 172.21.15.154:0/1847483264 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fef0006af30 0x7fef0006d3f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fef1c15b710 tx=0x7fef10009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:18.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.461+0000 7fef19ffb640 1 -- 172.21.15.154:0/1847483264 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fef140547d0 con 0x7fef1c1f6270 2023-09-23T03:16:18.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.650+0000 7fef23797640 1 -- 172.21.15.154:0/1847483264 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fef1c04e920 con 0x7fef1c1f6270 2023-09-23T03:16:18.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.651+0000 7fef19ffb640 1 -- 172.21.15.154:0/1847483264 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fef14000e60 con 0x7fef1c1f6270 2023-09-23T03:16:18.652 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:16:18.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.654+0000 7fef23797640 1 -- 172.21.15.154:0/1847483264 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fef0006af30 msgr2=0x7fef0006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:18.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.654+0000 7fef23797640 1 --2- 172.21.15.154:0/1847483264 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fef0006af30 0x7fef0006d3f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fef1c15b710 tx=0x7fef10009040 comp rx=0 tx=0).stop 2023-09-23T03:16:18.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.654+0000 7fef23797640 1 -- 172.21.15.154:0/1847483264 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef1c1f6270 msgr2=0x7fef1c151f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:18.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.654+0000 7fef23797640 1 --2- 172.21.15.154:0/1847483264 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef1c1f6270 0x7fef1c151f00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fef1c0d39d0 tx=0x7fef14010930 comp rx=0 tx=0).stop 2023-09-23T03:16:18.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.654+0000 7fef23797640 1 -- 172.21.15.154:0/1847483264 shutdown_connections 2023-09-23T03:16:18.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.654+0000 7fef23797640 1 --2- 172.21.15.154:0/1847483264 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fef0006af30 0x7fef0006d3f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:18.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.654+0000 7fef23797640 1 --2- 172.21.15.154:0/1847483264 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef1c1f6270 0x7fef1c151f00 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:18.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.654+0000 7fef23797640 1 --2- 172.21.15.154:0/1847483264 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef1c1f5df0 0x7fef1c1519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:18.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.654+0000 7fef23797640 1 --2- 172.21.15.154:0/1847483264 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef0c003f90 0x7fef1c15fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:18.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.654+0000 7fef23797640 1 -- 172.21.15.154:0/1847483264 >> 172.21.15.154:0/1847483264 conn(0x7fef1c09bc20 msgr2=0x7fef1c06af50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:18.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.654+0000 7fef23797640 1 -- 172.21.15.154:0/1847483264 shutdown_connections 2023-09-23T03:16:18.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.654+0000 7fef23797640 1 -- 172.21.15.154:0/1847483264 wait complete. 2023-09-23T03:16:18.674 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:16:18.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.894+0000 7f85b4d27640 1 Processor -- start 2023-09-23T03:16:18.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.894+0000 7f85b4d27640 1 -- start start 2023-09-23T03:16:18.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.896+0000 7f85b4d27640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f85b015df50 0x7f85b00b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:18.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.896+0000 7f85b4d27640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85b00d0980 0x7f85b00d1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:18.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.896+0000 7f85b4d27640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f85b00b3780 0x7f85b00d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:18.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.896+0000 7f85b4d27640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85b00d1c00 con 0x7f85b015df50 2023-09-23T03:16:18.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.896+0000 7f85b4d27640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85b00cd870 con 0x7f85b00b3780 2023-09-23T03:16:18.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.896+0000 7f85b4d27640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b015e330 con 0x7f85b00d0980 2023-09-23T03:16:18.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.896+0000 7f85af7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f85b015df50 0x7f85b00b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:18.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.896+0000 7f85af7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f85b015df50 0x7f85b00b20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53052/0 (socket says 172.21.15.154:53052) 2023-09-23T03:16:18.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.896+0000 7f85af7fe640 1 -- 172.21.15.154:0/616636238 learned_addr learned my addr 172.21.15.154:0/616636238 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:18.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.897+0000 7f85affff640 1 --2- 172.21.15.154:0/616636238 >> v2:172.21.15.190:3300/0 conn(0x7f85b00b3780 0x7f85b00d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:18.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.897+0000 7f85af7fe640 1 -- 172.21.15.154:0/616636238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85b00d0980 msgr2=0x7f85b00d1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:18.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.897+0000 7f85af7fe640 1 --2- 172.21.15.154:0/616636238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85b00d0980 0x7f85b00d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:18.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.897+0000 7f85af7fe640 1 -- 172.21.15.154:0/616636238 >> v2:172.21.15.190:3300/0 conn(0x7f85b00b3780 msgr2=0x7f85b00d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:18.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.897+0000 7f85af7fe640 1 --2- 172.21.15.154:0/616636238 >> v2:172.21.15.190:3300/0 conn(0x7f85b00b3780 0x7f85b00d16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:18.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.897+0000 7f85af7fe640 1 -- 172.21.15.154:0/616636238 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b00d0e10 con 0x7f85b015df50 2023-09-23T03:16:18.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.897+0000 7f85affff640 1 --2- 172.21.15.154:0/616636238 >> v2:172.21.15.190:3300/0 conn(0x7f85b00b3780 0x7f85b00d16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:18.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.898+0000 7f85af7fe640 1 --2- 172.21.15.154:0/616636238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85b015df50 0x7f85b00b20f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f85b00c6140 tx=0x7f85a002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53005d852e1ef11 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:18.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.898+0000 7f85aeffd640 1 -- 172.21.15.154:0/616636238 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a0031860 con 0x7f85b015df50 2023-09-23T03:16:18.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.898+0000 7f85aeffd640 1 -- 172.21.15.154:0/616636238 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a00319c0 con 0x7f85b015df50 2023-09-23T03:16:18.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.898+0000 7f85aeffd640 1 -- 172.21.15.154:0/616636238 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a0031d50 con 0x7f85b015df50 2023-09-23T03:16:18.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.899+0000 7f85b4d27640 1 -- 172.21.15.154:0/616636238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85b015df50 msgr2=0x7f85b00b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:18.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.899+0000 7f85b4d27640 1 --2- 172.21.15.154:0/616636238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85b015df50 0x7f85b00b20f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f85b00c6140 tx=0x7f85a002b900 comp rx=0 tx=0).stop 2023-09-23T03:16:18.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.899+0000 7f85b4d27640 1 -- 172.21.15.154:0/616636238 shutdown_connections 2023-09-23T03:16:18.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.899+0000 7f85b4d27640 1 --2- 172.21.15.154:0/616636238 >> v2:172.21.15.190:3300/0 conn(0x7f85b00b3780 0x7f85b00d16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:18.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.899+0000 7f85b4d27640 1 --2- 172.21.15.154:0/616636238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85b00d0980 0x7f85b00d1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:18.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.899+0000 7f85b4d27640 1 --2- 172.21.15.154:0/616636238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85b015df50 0x7f85b00b20f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:18.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.899+0000 7f85b4d27640 1 -- 172.21.15.154:0/616636238 >> 172.21.15.154:0/616636238 conn(0x7f85b009bb20 msgr2=0x7f85b00d2870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:18.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.899+0000 7f85b4d27640 1 -- 172.21.15.154:0/616636238 shutdown_connections 2023-09-23T03:16:18.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.899+0000 7f85b4d27640 1 -- 172.21.15.154:0/616636238 wait complete. 2023-09-23T03:16:18.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.901+0000 7f85b4d27640 1 Processor -- start 2023-09-23T03:16:18.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.901+0000 7f85b4d27640 1 -- start start 2023-09-23T03:16:18.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.902+0000 7f85b4d27640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85b00b3780 0x7f85b00c6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:18.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.902+0000 7f85b4d27640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85b015df50 0x7f85b006b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:18.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.902+0000 7f85b4d27640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85b00c64b0 0x7f85b00b6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:18.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.902+0000 7f85b4d27640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b00c7b10 con 0x7f85b00c64b0 2023-09-23T03:16:18.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.902+0000 7f85b4d27640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b00c7270 con 0x7f85b00b3780 2023-09-23T03:16:18.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.902+0000 7f85b4d27640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b00c7550 con 0x7f85b015df50 2023-09-23T03:16:18.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.902+0000 7f85affff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85b00c64b0 0x7f85b00b6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:18.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.902+0000 7f85affff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85b00c64b0 0x7f85b00b6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53064/0 (socket says 172.21.15.154:53064) 2023-09-23T03:16:18.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.902+0000 7f85affff640 1 -- 172.21.15.154:0/2151533675 learned_addr learned my addr 172.21.15.154:0/2151533675 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:18.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.902+0000 7f85af7fe640 1 --2- 172.21.15.154:0/2151533675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85b00b3780 0x7f85b00c6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:18.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.903+0000 7f85a7fff640 1 --2- 172.21.15.154:0/2151533675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85b015df50 0x7f85b006b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:18.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.903+0000 7f85affff640 1 -- 172.21.15.154:0/2151533675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85b015df50 msgr2=0x7f85b006b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:18.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.903+0000 7f85affff640 1 --2- 172.21.15.154:0/2151533675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85b015df50 0x7f85b006b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:18.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.903+0000 7f85affff640 1 -- 172.21.15.154:0/2151533675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85b00b3780 msgr2=0x7f85b00c6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:18.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.903+0000 7f85affff640 1 --2- 172.21.15.154:0/2151533675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85b00b3780 0x7f85b00c6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:18.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.903+0000 7f85affff640 1 -- 172.21.15.154:0/2151533675 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a00094d0 con 0x7f85b00c64b0 2023-09-23T03:16:18.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.903+0000 7f85affff640 1 --2- 172.21.15.154:0/2151533675 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85b00c64b0 0x7f85b00b6f10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f85a800cef0 tx=0x7f85a800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:18.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.903+0000 7f85acff9640 1 -- 172.21.15.154:0/2151533675 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a800d840 con 0x7f85b00c64b0 2023-09-23T03:16:18.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.903+0000 7f85acff9640 1 -- 172.21.15.154:0/2151533675 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a800d9c0 con 0x7f85b00c64b0 2023-09-23T03:16:18.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.903+0000 7f85b4d27640 1 -- 172.21.15.154:0/2151533675 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85b00c7830 con 0x7f85b00c64b0 2023-09-23T03:16:18.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.903+0000 7f85acff9640 1 -- 172.21.15.154:0/2151533675 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a800dd70 con 0x7f85b00c64b0 2023-09-23T03:16:18.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.904+0000 7f85b4d27640 1 -- 172.21.15.154:0/2151533675 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b00c6f40 con 0x7f85b00c64b0 2023-09-23T03:16:18.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.905+0000 7f85b4d27640 1 -- 172.21.15.154:0/2151533675 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85b015bed0 con 0x7f85b00c64b0 2023-09-23T03:16:18.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.906+0000 7f85acff9640 1 -- 172.21.15.154:0/2151533675 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f85a801e3b0 con 0x7f85b00c64b0 2023-09-23T03:16:18.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.907+0000 7f85acff9640 1 --2- 172.21.15.154:0/2151533675 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f859006af30 0x7f859006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:18.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.907+0000 7f85acff9640 1 -- 172.21.15.154:0/2151533675 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f85a808a540 con 0x7f85b00c64b0 2023-09-23T03:16:18.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.907+0000 7f85af7fe640 1 --2- 172.21.15.154:0/2151533675 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f859006af30 0x7f859006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:18.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.908+0000 7f85af7fe640 1 --2- 172.21.15.154:0/2151533675 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f859006af30 0x7f859006d3f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f85a0036000 tx=0x7f85a003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:18.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:18.911+0000 7f85acff9640 1 -- 172.21.15.154:0/2151533675 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f85a8055320 con 0x7f85b00c64b0 2023-09-23T03:16:19.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.100+0000 7f85b4d27640 1 -- 172.21.15.154:0/2151533675 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f85b004e920 con 0x7f85b00c64b0 2023-09-23T03:16:19.101 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.101+0000 7f85acff9640 1 -- 172.21.15.154:0/2151533675 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f85a8058940 con 0x7f85b00c64b0 2023-09-23T03:16:19.101 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:16:19.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.104+0000 7f85b4d27640 1 -- 172.21.15.154:0/2151533675 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f859006af30 msgr2=0x7f859006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:19.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.104+0000 7f85b4d27640 1 --2- 172.21.15.154:0/2151533675 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f859006af30 0x7f859006d3f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f85a0036000 tx=0x7f85a003c040 comp rx=0 tx=0).stop 2023-09-23T03:16:19.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.104+0000 7f85b4d27640 1 -- 172.21.15.154:0/2151533675 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85b00c64b0 msgr2=0x7f85b00b6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:19.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.104+0000 7f85b4d27640 1 --2- 172.21.15.154:0/2151533675 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85b00c64b0 0x7f85b00b6f10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f85a800cef0 tx=0x7f85a800a710 comp rx=0 tx=0).stop 2023-09-23T03:16:19.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.104+0000 7f85b4d27640 1 -- 172.21.15.154:0/2151533675 shutdown_connections 2023-09-23T03:16:19.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.104+0000 7f85b4d27640 1 --2- 172.21.15.154:0/2151533675 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f859006af30 0x7f859006d3f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:19.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.104+0000 7f85b4d27640 1 --2- 172.21.15.154:0/2151533675 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85b00c64b0 0x7f85b00b6f10 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:19.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.104+0000 7f85b4d27640 1 --2- 172.21.15.154:0/2151533675 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85b015df50 0x7f85b006b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:19.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.104+0000 7f85b4d27640 1 --2- 172.21.15.154:0/2151533675 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85b00b3780 0x7f85b00c6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:19.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.104+0000 7f85b4d27640 1 -- 172.21.15.154:0/2151533675 >> 172.21.15.154:0/2151533675 conn(0x7f85b009bb20 msgr2=0x7f85b00b19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:19.106 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.104+0000 7f85b4d27640 1 -- 172.21.15.154:0/2151533675 shutdown_connections 2023-09-23T03:16:19.106 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.104+0000 7f85b4d27640 1 -- 172.21.15.154:0/2151533675 wait complete. 2023-09-23T03:16:19.123 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-23T03:16:19.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.334+0000 7f7e41826640 1 Processor -- start 2023-09-23T03:16:19.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.334+0000 7f7e41826640 1 -- start start 2023-09-23T03:16:19.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.336+0000 7f7e41826640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7e3c15aba0 0x7f7e3c15bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:19.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.336+0000 7f7e41826640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7e3c0cb5b0 0x7f7e3c15ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:19.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.336+0000 7f7e41826640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e3c15b6a0 con 0x7f7e3c15e5f0 2023-09-23T03:16:19.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.336+0000 7f7e41826640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e3c15f0b0 con 0x7f7e3c15aba0 2023-09-23T03:16:19.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.336+0000 7f7e41826640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e3c15f390 con 0x7f7e3c0cb5b0 2023-09-23T03:16:19.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.336+0000 7f7e3affd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7e3c15aba0 0x7f7e3c15bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:19.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.336+0000 7f7e3affd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7e3c15aba0 0x7f7e3c15bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53074/0 (socket says 172.21.15.154:53074) 2023-09-23T03:16:19.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.336+0000 7f7e3b7fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7e3c0cb5b0 0x7f7e3c15ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:19.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.336+0000 7f7e3affd640 1 -- 172.21.15.154:0/3257783903 learned_addr learned my addr 172.21.15.154:0/3257783903 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:19.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.337+0000 7f7e3affd640 1 -- 172.21.15.154:0/3257783903 >> v2:172.21.15.190:3300/0 conn(0x7f7e3c0cb5b0 msgr2=0x7f7e3c15ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:19.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.337+0000 7f7e3affd640 1 --2- 172.21.15.154:0/3257783903 >> v2:172.21.15.190:3300/0 conn(0x7f7e3c0cb5b0 0x7f7e3c15ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:19.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.337+0000 7f7e3affd640 1 -- 172.21.15.154:0/3257783903 >> v1:172.21.15.190:6789/0 conn(0x7f7e3c15e5f0 legacy=0x7f7e3c0caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:19.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.337+0000 7f7e3affd640 1 -- 172.21.15.154:0/3257783903 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3c15b330 con 0x7f7e3c15aba0 2023-09-23T03:16:19.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.338+0000 7f7e3affd640 1 --2- 172.21.15.154:0/3257783903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e3c15aba0 0x7f7e3c15bb30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c0cbfa0 tx=0x7f7e2c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78f46dec0b21779 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:19.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.339+0000 7f7e39ffb640 1 -- 172.21.15.154:0/3257783903 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c030860 con 0x7f7e3c15aba0 2023-09-23T03:16:19.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.339+0000 7f7e39ffb640 1 -- 172.21.15.154:0/3257783903 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2c0309c0 con 0x7f7e3c15aba0 2023-09-23T03:16:19.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.339+0000 7f7e39ffb640 1 -- 172.21.15.154:0/3257783903 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c030d50 con 0x7f7e3c15aba0 2023-09-23T03:16:19.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.339+0000 7f7e41826640 1 -- 172.21.15.154:0/3257783903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e3c15aba0 msgr2=0x7f7e3c15bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:19.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.339+0000 7f7e41826640 1 --2- 172.21.15.154:0/3257783903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e3c15aba0 0x7f7e3c15bb30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c0cbfa0 tx=0x7f7e2c02b9a0 comp rx=0 tx=0).stop 2023-09-23T03:16:19.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.340+0000 7f7e41826640 1 -- 172.21.15.154:0/3257783903 shutdown_connections 2023-09-23T03:16:19.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.340+0000 7f7e41826640 1 --2- 172.21.15.154:0/3257783903 >> v2:172.21.15.190:3300/0 conn(0x7f7e3c0cb5b0 0x7f7e3c15ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:19.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.340+0000 7f7e41826640 1 --2- 172.21.15.154:0/3257783903 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e3c15aba0 0x7f7e3c15bb30 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:19.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.340+0000 7f7e41826640 1 -- 172.21.15.154:0/3257783903 >> 172.21.15.154:0/3257783903 conn(0x7f7e3c06c010 msgr2=0x7f7e3c06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:19.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.340+0000 7f7e41826640 1 -- 172.21.15.154:0/3257783903 shutdown_connections 2023-09-23T03:16:19.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.340+0000 7f7e41826640 1 -- 172.21.15.154:0/3257783903 wait complete. 2023-09-23T03:16:19.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.342+0000 7f7e41826640 1 Processor -- start 2023-09-23T03:16:19.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.342+0000 7f7e41826640 1 -- start start 2023-09-23T03:16:19.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.343+0000 7f7e41826640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e3c0cb5b0 0x7f7e3c1522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:19.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.343+0000 7f7e41826640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e3c15aba0 0x7f7e3c0c4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:19.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.343+0000 7f7e41826640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e3c15e5f0 0x7f7e3c14e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:19.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.343+0000 7f7e41826640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c0c6720 con 0x7f7e3c0cb5b0 2023-09-23T03:16:19.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.343+0000 7f7e41826640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c0c5b40 con 0x7f7e3c15aba0 2023-09-23T03:16:19.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.343+0000 7f7e41826640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c0c5e20 con 0x7f7e3c15e5f0 2023-09-23T03:16:19.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.344+0000 7f7e3affd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e3c0cb5b0 0x7f7e3c1522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:19.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.344+0000 7f7e3affd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e3c0cb5b0 0x7f7e3c1522a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53076/0 (socket says 172.21.15.154:53076) 2023-09-23T03:16:19.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.344+0000 7f7e3affd640 1 -- 172.21.15.154:0/3437786343 learned_addr learned my addr 172.21.15.154:0/3437786343 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:19.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.344+0000 7f7e3a7fc640 1 --2- 172.21.15.154:0/3437786343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e3c15aba0 0x7f7e3c0c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:19.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.344+0000 7f7e3b7fe640 1 --2- 172.21.15.154:0/3437786343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e3c15e5f0 0x7f7e3c14e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:19.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.344+0000 7f7e3affd640 1 -- 172.21.15.154:0/3437786343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e3c15e5f0 msgr2=0x7f7e3c14e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:19.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.344+0000 7f7e3affd640 1 --2- 172.21.15.154:0/3437786343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e3c15e5f0 0x7f7e3c14e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:19.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.344+0000 7f7e3affd640 1 -- 172.21.15.154:0/3437786343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e3c15aba0 msgr2=0x7f7e3c0c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:19.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.344+0000 7f7e3affd640 1 --2- 172.21.15.154:0/3437786343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e3c15aba0 0x7f7e3c0c4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:19.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.344+0000 7f7e3affd640 1 -- 172.21.15.154:0/3437786343 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2c0094d0 con 0x7f7e3c0cb5b0 2023-09-23T03:16:19.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.344+0000 7f7e3a7fc640 1 --2- 172.21.15.154:0/3437786343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e3c15aba0 0x7f7e3c0c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:16:19.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.345+0000 7f7e3affd640 1 --2- 172.21.15.154:0/3437786343 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e3c0cb5b0 0x7f7e3c1522a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c02b9e0 tx=0x7f7e2c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:19.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.345+0000 7f7e23fff640 1 -- 172.21.15.154:0/3437786343 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c039c90 con 0x7f7e3c0cb5b0 2023-09-23T03:16:19.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.345+0000 7f7e23fff640 1 -- 172.21.15.154:0/3437786343 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2c039e10 con 0x7f7e3c0cb5b0 2023-09-23T03:16:19.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.345+0000 7f7e23fff640 1 -- 172.21.15.154:0/3437786343 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c043640 con 0x7f7e3c0cb5b0 2023-09-23T03:16:19.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.345+0000 7f7e41826640 1 -- 172.21.15.154:0/3437786343 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e3c15b100 con 0x7f7e3c0cb5b0 2023-09-23T03:16:19.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.345+0000 7f7e41826640 1 -- 172.21.15.154:0/3437786343 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e3c0c63e0 con 0x7f7e3c0cb5b0 2023-09-23T03:16:19.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.346+0000 7f7e41826640 1 -- 172.21.15.154:0/3437786343 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e3c04e920 con 0x7f7e3c0cb5b0 2023-09-23T03:16:19.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.347+0000 7f7e23fff640 1 -- 172.21.15.154:0/3437786343 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f7e2c0437a0 con 0x7f7e3c0cb5b0 2023-09-23T03:16:19.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.348+0000 7f7e23fff640 1 --2- 172.21.15.154:0/3437786343 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7e2806af60 0x7f7e2806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:19.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.348+0000 7f7e23fff640 1 -- 172.21.15.154:0/3437786343 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f7e2c03e070 con 0x7f7e3c0cb5b0 2023-09-23T03:16:19.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.348+0000 7f7e3a7fc640 1 --2- 172.21.15.154:0/3437786343 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7e2806af60 0x7f7e2806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:19.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.349+0000 7f7e3a7fc640 1 --2- 172.21.15.154:0/3437786343 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7e2806af60 0x7f7e2806d420 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7e30007fd0 tx=0x7f7e30009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:19.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.353+0000 7f7e23fff640 1 -- 172.21.15.154:0/3437786343 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f7e2c077850 con 0x7f7e3c0cb5b0 2023-09-23T03:16:19.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.541+0000 7f7e41826640 1 -- 172.21.15.154:0/3437786343 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7e3c0ad580 con 0x7f7e3c0cb5b0 2023-09-23T03:16:19.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.542+0000 7f7e23fff640 1 -- 172.21.15.154:0/3437786343 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7e2c0340a0 con 0x7f7e3c0cb5b0 2023-09-23T03:16:19.543 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:16:19.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.545+0000 7f7e41826640 1 -- 172.21.15.154:0/3437786343 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7e2806af60 msgr2=0x7f7e2806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:19.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.545+0000 7f7e41826640 1 --2- 172.21.15.154:0/3437786343 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7e2806af60 0x7f7e2806d420 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7e30007fd0 tx=0x7f7e30009040 comp rx=0 tx=0).stop 2023-09-23T03:16:19.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.545+0000 7f7e41826640 1 -- 172.21.15.154:0/3437786343 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e3c0cb5b0 msgr2=0x7f7e3c1522a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:19.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.545+0000 7f7e41826640 1 --2- 172.21.15.154:0/3437786343 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e3c0cb5b0 0x7f7e3c1522a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c02b9e0 tx=0x7f7e2c004730 comp rx=0 tx=0).stop 2023-09-23T03:16:19.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.545+0000 7f7e41826640 1 -- 172.21.15.154:0/3437786343 shutdown_connections 2023-09-23T03:16:19.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.545+0000 7f7e41826640 1 --2- 172.21.15.154:0/3437786343 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7e2806af60 0x7f7e2806d420 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:19.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.545+0000 7f7e41826640 1 --2- 172.21.15.154:0/3437786343 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e3c15e5f0 0x7f7e3c14e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:19.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.545+0000 7f7e41826640 1 --2- 172.21.15.154:0/3437786343 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e3c15aba0 0x7f7e3c0c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:19.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.545+0000 7f7e41826640 1 --2- 172.21.15.154:0/3437786343 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e3c0cb5b0 0x7f7e3c1522a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:19.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.545+0000 7f7e41826640 1 -- 172.21.15.154:0/3437786343 >> 172.21.15.154:0/3437786343 conn(0x7f7e3c06c010 msgr2=0x7f7e3c0701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:19.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.545+0000 7f7e41826640 1 -- 172.21.15.154:0/3437786343 shutdown_connections 2023-09-23T03:16:19.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.545+0000 7f7e41826640 1 -- 172.21.15.154:0/3437786343 wait complete. 2023-09-23T03:16:19.563 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:16:19.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.766+0000 7f360bfff640 1 Processor -- start 2023-09-23T03:16:19.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.767+0000 7f360bfff640 1 -- start start 2023-09-23T03:16:19.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.768+0000 7f360bfff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f360c0cacd0 0x7f360c0cb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:19.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.768+0000 7f360bfff640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f360c0cc910 con 0x7f360c160fd0 2023-09-23T03:16:19.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.768+0000 7f360bfff640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f360c0c89b0 con 0x7f360c15c840 2023-09-23T03:16:19.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.768+0000 7f360bfff640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f360c0cb6c0 con 0x7f360c0cacd0 2023-09-23T03:16:19.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.769+0000 7f3603fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f360c160fd0 0x7f360c0c9490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:47378/0 (socket says 172.21.15.154:47378) 2023-09-23T03:16:19.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.769+0000 7f3603fff640 1 -- 172.21.15.154:0/2052887947 learned_addr learned my addr 172.21.15.154:0/2052887947 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:19.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.769+0000 7f360b7fe640 1 --2- 172.21.15.154:0/2052887947 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f360c0cacd0 0x7f360c0cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:19.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.769+0000 7f360a7fc640 1 -- 172.21.15.154:0/2052887947 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3243766725 0 0) 0x7f360c0cc910 con 0x7f360c160fd0 2023-09-23T03:16:19.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.769+0000 7f360a7fc640 1 -- 172.21.15.154:0/2052887947 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35f8003520 con 0x7f360c160fd0 2023-09-23T03:16:19.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.769+0000 7f360a7fc640 1 -- 172.21.15.154:0/2052887947 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2241154328 0 0) 0x7f35f8003520 con 0x7f360c160fd0 2023-09-23T03:16:19.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.769+0000 7f360a7fc640 1 -- 172.21.15.154:0/2052887947 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f360c0cacd0 msgr2=0x7f360c0cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:19.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.769+0000 7f360a7fc640 1 --2- 172.21.15.154:0/2052887947 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f360c0cacd0 0x7f360c0cb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:19.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.769+0000 7f360a7fc640 1 -- 172.21.15.154:0/2052887947 >> v1:172.21.15.190:6789/0 conn(0x7f360c15c840 legacy=0x7f360c0cbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:19.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.769+0000 7f360a7fc640 1 -- 172.21.15.154:0/2052887947 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360c0c8c90 con 0x7f360c160fd0 2023-09-23T03:16:19.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.769+0000 7f360a7fc640 1 -- 172.21.15.154:0/2052887947 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f35f4001670 con 0x7f360c160fd0 2023-09-23T03:16:19.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.770+0000 7f360a7fc640 1 -- 172.21.15.154:0/2052887947 >> v1:172.21.15.154:6789/0 conn(0x7f360c160fd0 legacy=0x7f360c0c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:19.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.770+0000 7f360a7fc640 1 --2- 172.21.15.154:0/2052887947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35f80039a0 0x7f35f8003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:19.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.770+0000 7f360a7fc640 1 --2- 172.21.15.154:0/2052887947 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f360c0cacd0 0x7f35f80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:19.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.770+0000 7f360a7fc640 1 --2- 172.21.15.154:0/2052887947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35f8004d00 0x7f35f8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:19.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.770+0000 7f360a7fc640 1 -- 172.21.15.154:0/2052887947 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360c0cb6c0 con 0x7f35f80039a0 2023-09-23T03:16:19.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.770+0000 7f360a7fc640 1 -- 172.21.15.154:0/2052887947 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f800ba60 con 0x7f35f8004d00 2023-09-23T03:16:19.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.770+0000 7f360a7fc640 1 -- 172.21.15.154:0/2052887947 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35f800bc20 con 0x7f360c0cacd0 2023-09-23T03:16:19.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.770+0000 7f360affd640 1 --2- 172.21.15.154:0/2052887947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35f80039a0 0x7f35f8003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:19.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.770+0000 7f360b7fe640 1 --2- 172.21.15.154:0/2052887947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35f8004d00 0x7f35f8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:19.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.770+0000 7f3603fff640 1 --2- 172.21.15.154:0/2052887947 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f360c0cacd0 0x7f35f80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:19.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.770+0000 7f360affd640 1 -- 172.21.15.154:0/2052887947 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f360c0cacd0 msgr2=0x7f35f80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:19.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.770+0000 7f360affd640 1 --2- 172.21.15.154:0/2052887947 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f360c0cacd0 0x7f35f80047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:19.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.770+0000 7f360affd640 1 -- 172.21.15.154:0/2052887947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35f8004d00 msgr2=0x7f35f8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:19.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.770+0000 7f360affd640 1 --2- 172.21.15.154:0/2052887947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35f8004d00 0x7f35f8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:19.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.770+0000 7f360affd640 1 -- 172.21.15.154:0/2052887947 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f8003520 con 0x7f35f80039a0 2023-09-23T03:16:19.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.770+0000 7f360b7fe640 1 --2- 172.21.15.154:0/2052887947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35f8004d00 0x7f35f8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:19.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.771+0000 7f360affd640 1 --2- 172.21.15.154:0/2052887947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35f80039a0 0x7f35f8003d80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f35fc001830 tx=0x7f35f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15903aa009b12a05 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:19.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.772+0000 7f360a7fc640 1 -- 172.21.15.154:0/2052887947 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35fc033600 con 0x7f35f80039a0 2023-09-23T03:16:19.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.772+0000 7f360a7fc640 1 -- 172.21.15.154:0/2052887947 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35fc02f050 con 0x7f35f80039a0 2023-09-23T03:16:19.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.772+0000 7f360a7fc640 1 -- 172.21.15.154:0/2052887947 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35fc0339c0 con 0x7f35f80039a0 2023-09-23T03:16:19.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.772+0000 7f360bfff640 1 -- 172.21.15.154:0/2052887947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35f80039a0 msgr2=0x7f35f8003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:19.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.772+0000 7f360bfff640 1 --2- 172.21.15.154:0/2052887947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35f80039a0 0x7f35f8003d80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f35fc001830 tx=0x7f35f80057f0 comp rx=0 tx=0).stop 2023-09-23T03:16:19.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.772+0000 7f360b7fe640 1 -- 172.21.15.154:0/2052887947 reap_dead start 2023-09-23T03:16:19.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.772+0000 7f360bfff640 1 -- 172.21.15.154:0/2052887947 shutdown_connections 2023-09-23T03:16:19.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.772+0000 7f360bfff640 1 -- 172.21.15.154:0/2052887947 >> 172.21.15.154:0/2052887947 conn(0x7f360c06a390 msgr2=0x7f360c0afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:19.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.772+0000 7f360bfff640 1 -- 172.21.15.154:0/2052887947 shutdown_connections 2023-09-23T03:16:19.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.772+0000 7f360bfff640 1 -- 172.21.15.154:0/2052887947 wait complete. 2023-09-23T03:16:19.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.774+0000 7f360bfff640 1 Processor -- start 2023-09-23T03:16:19.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.774+0000 7f360bfff640 1 -- start start 2023-09-23T03:16:19.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.775+0000 7f360bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35f80039a0 0x7f360c0b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:19.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.775+0000 7f360bfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f360c0b0e70 0x7f360c1f1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:19.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.775+0000 7f360bfff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f360c0af370 0x7f360c0af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:19.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.775+0000 7f360bfff640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360c1f63d0 con 0x7f35f80039a0 2023-09-23T03:16:19.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.775+0000 7f360bfff640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360c0afd30 con 0x7f360c0b0e70 2023-09-23T03:16:19.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.775+0000 7f360bfff640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f360c0b3df0 con 0x7f360c0af370 2023-09-23T03:16:19.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.775+0000 7f360affd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35f80039a0 0x7f360c0b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:19.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.775+0000 7f360affd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35f80039a0 0x7f360c0b38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53090/0 (socket says 172.21.15.154:53090) 2023-09-23T03:16:19.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.775+0000 7f360affd640 1 -- 172.21.15.154:0/144272414 learned_addr learned my addr 172.21.15.154:0/144272414 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:19.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.776+0000 7f3603fff640 1 --2- 172.21.15.154:0/144272414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f360c0b0e70 0x7f360c1f1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:19.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.776+0000 7f360affd640 1 -- 172.21.15.154:0/144272414 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f360c0af370 msgr2=0x7f360c0af7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:19.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.776+0000 7f360affd640 1 --2- 172.21.15.154:0/144272414 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f360c0af370 0x7f360c0af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:19.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.776+0000 7f360affd640 1 -- 172.21.15.154:0/144272414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f360c0b0e70 msgr2=0x7f360c1f1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:19.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.776+0000 7f360affd640 1 --2- 172.21.15.154:0/144272414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f360c0b0e70 0x7f360c1f1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:19.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.776+0000 7f360affd640 1 -- 172.21.15.154:0/144272414 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360c1f5750 con 0x7f35f80039a0 2023-09-23T03:16:19.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.776+0000 7f360affd640 1 --2- 172.21.15.154:0/144272414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35f80039a0 0x7f360c0b38b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f35fc02bdc0 tx=0x7f35fc001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:19.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.776+0000 7f3608ff9640 1 -- 172.21.15.154:0/144272414 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35fc0054e0 con 0x7f35f80039a0 2023-09-23T03:16:19.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.776+0000 7f3608ff9640 1 -- 172.21.15.154:0/144272414 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35fc0035f0 con 0x7f35f80039a0 2023-09-23T03:16:19.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.776+0000 7f3608ff9640 1 -- 172.21.15.154:0/144272414 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35fc003960 con 0x7f35f80039a0 2023-09-23T03:16:19.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.776+0000 7f360bfff640 1 -- 172.21.15.154:0/144272414 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f360c1f5a30 con 0x7f35f80039a0 2023-09-23T03:16:19.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.776+0000 7f360bfff640 1 -- 172.21.15.154:0/144272414 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f360c1f5ff0 con 0x7f35f80039a0 2023-09-23T03:16:19.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.777+0000 7f360bfff640 1 -- 172.21.15.154:0/144272414 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f360c0736b0 con 0x7f35f80039a0 2023-09-23T03:16:19.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.778+0000 7f3608ff9640 1 -- 172.21.15.154:0/144272414 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f35fc003ac0 con 0x7f35f80039a0 2023-09-23T03:16:19.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.779+0000 7f3608ff9640 1 --2- 172.21.15.154:0/144272414 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f35ec06af60 0x7f35ec06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:19.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.779+0000 7f3608ff9640 1 -- 172.21.15.154:0/144272414 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f35fc0abb60 con 0x7f35f80039a0 2023-09-23T03:16:19.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.779+0000 7f3603fff640 1 --2- 172.21.15.154:0/144272414 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f35ec06af60 0x7f35ec06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:19.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.780+0000 7f3603fff640 1 --2- 172.21.15.154:0/144272414 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f35ec06af60 0x7f35ec06d420 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f360c0ce910 tx=0x7f35f4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:19.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.783+0000 7f3608ff9640 1 -- 172.21.15.154:0/144272414 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f35fc07a3a0 con 0x7f35f80039a0 2023-09-23T03:16:19.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.960+0000 7f360bfff640 1 -- 172.21.15.154:0/144272414 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f360c0cc670 con 0x7f35f80039a0 2023-09-23T03:16:19.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.962+0000 7f3608ff9640 1 -- 172.21.15.154:0/144272414 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18744 (secure 0 0 0) 0x7f35fc079f30 con 0x7f35f80039a0 2023-09-23T03:16:19.962 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:16:19.963 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":26,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:16:03.151371+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:15:18.575849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T03:15:19.707212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T03:15:23.337138+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:16:19.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.964+0000 7f360bfff640 1 -- 172.21.15.154:0/144272414 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f35ec06af60 msgr2=0x7f35ec06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:19.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.964+0000 7f360bfff640 1 --2- 172.21.15.154:0/144272414 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f35ec06af60 0x7f35ec06d420 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f360c0ce910 tx=0x7f35f4009040 comp rx=0 tx=0).stop 2023-09-23T03:16:19.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.964+0000 7f360bfff640 1 -- 172.21.15.154:0/144272414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35f80039a0 msgr2=0x7f360c0b38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:19.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.964+0000 7f360bfff640 1 --2- 172.21.15.154:0/144272414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35f80039a0 0x7f360c0b38b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f35fc02bdc0 tx=0x7f35fc001230 comp rx=0 tx=0).stop 2023-09-23T03:16:19.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.964+0000 7f360bfff640 1 -- 172.21.15.154:0/144272414 shutdown_connections 2023-09-23T03:16:19.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.964+0000 7f360bfff640 1 --2- 172.21.15.154:0/144272414 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f35ec06af60 0x7f35ec06d420 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:19.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.964+0000 7f360bfff640 1 --2- 172.21.15.154:0/144272414 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f360c0af370 0x7f360c0af7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:19.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.964+0000 7f360bfff640 1 --2- 172.21.15.154:0/144272414 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f360c0b0e70 0x7f360c1f1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:19.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.964+0000 7f360bfff640 1 --2- 172.21.15.154:0/144272414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35f80039a0 0x7f360c0b38b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:19.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.964+0000 7f360bfff640 1 -- 172.21.15.154:0/144272414 >> 172.21.15.154:0/144272414 conn(0x7f360c06a390 msgr2=0x7f360c15b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:19.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.965+0000 7f360bfff640 1 -- 172.21.15.154:0/144272414 shutdown_connections 2023-09-23T03:16:19.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:19.965+0000 7f360bfff640 1 -- 172.21.15.154:0/144272414 wait complete. 2023-09-23T03:16:19.983 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:15:18.575849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:15:19.707212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-23T03:15:23.337138+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-23T03:16:19.983 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:16:20.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.199+0000 7f61af2a9640 1 Processor -- start 2023-09-23T03:16:20.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.200+0000 7f61af2a9640 1 -- start start 2023-09-23T03:16:20.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.201+0000 7f61af2a9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61a80c9150 0x7f61a815cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:20.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.201+0000 7f61af2a9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f61a8071290 0x7f61a80ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:20.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.201+0000 7f61af2a9640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61a80c9d20 con 0x7f61a80c8d70 2023-09-23T03:16:20.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.201+0000 7f61af2a9640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61a8071850 con 0x7f61a8071290 2023-09-23T03:16:20.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.201+0000 7f61af2a9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a8071b30 con 0x7f61a80c9150 2023-09-23T03:16:20.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.202+0000 7f61ad045640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f61a80c8d70 0x7f61a815c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:47384/0 (socket says 172.21.15.154:47384) 2023-09-23T03:16:20.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.202+0000 7f61ad045640 1 -- 172.21.15.154:0/3989209758 learned_addr learned my addr 172.21.15.154:0/3989209758 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:20.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.202+0000 7f61ad846640 1 --2- 172.21.15.154:0/3989209758 >> v2:172.21.15.154:3300/0 conn(0x7f61a8071290 0x7f61a80ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:20.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.202+0000 7f61ac844640 1 --2- 172.21.15.154:0/3989209758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61a80c9150 0x7f61a815cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:20.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.202+0000 7f619ffff640 1 -- 172.21.15.154:0/3989209758 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2768619880 0 0) 0x7f61a80c9d20 con 0x7f61a80c8d70 2023-09-23T03:16:20.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.202+0000 7f619ffff640 1 -- 172.21.15.154:0/3989209758 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6194003520 con 0x7f61a80c8d70 2023-09-23T03:16:20.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.202+0000 7f61ad846640 1 -- 172.21.15.154:0/3989209758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61a80c9150 msgr2=0x7f61a815cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:20.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.202+0000 7f61ad846640 1 --2- 172.21.15.154:0/3989209758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61a80c9150 0x7f61a815cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:20.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.202+0000 7f61ad846640 1 -- 172.21.15.154:0/3989209758 >> v1:172.21.15.154:6789/0 conn(0x7f61a80c8d70 legacy=0x7f61a815c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:20.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.202+0000 7f61ad846640 1 -- 172.21.15.154:0/3989209758 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a8071e10 con 0x7f61a8071290 2023-09-23T03:16:20.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.203+0000 7f61ad846640 1 --2- 172.21.15.154:0/3989209758 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61a8071290 0x7f61a80ca1b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f61a00097a0 tx=0x7f61a002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e00c4b338e0ccf57 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:20.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.204+0000 7f619ffff640 1 -- 172.21.15.154:0/3989209758 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a0031780 con 0x7f61a8071290 2023-09-23T03:16:20.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.204+0000 7f619ffff640 1 -- 172.21.15.154:0/3989209758 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a00318e0 con 0x7f61a8071290 2023-09-23T03:16:20.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.204+0000 7f619ffff640 1 -- 172.21.15.154:0/3989209758 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a0031c70 con 0x7f61a8071290 2023-09-23T03:16:20.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.204+0000 7f61af2a9640 1 -- 172.21.15.154:0/3989209758 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61a8071290 msgr2=0x7f61a80ca1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:20.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.204+0000 7f61af2a9640 1 --2- 172.21.15.154:0/3989209758 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61a8071290 0x7f61a80ca1b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f61a00097a0 tx=0x7f61a002b820 comp rx=0 tx=0).stop 2023-09-23T03:16:20.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.204+0000 7f61af2a9640 1 -- 172.21.15.154:0/3989209758 shutdown_connections 2023-09-23T03:16:20.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.204+0000 7f61af2a9640 1 --2- 172.21.15.154:0/3989209758 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61a8071290 0x7f61a80ca1b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:20.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.204+0000 7f61af2a9640 1 --2- 172.21.15.154:0/3989209758 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61a80c9150 0x7f61a815cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:20.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.204+0000 7f61af2a9640 1 -- 172.21.15.154:0/3989209758 >> 172.21.15.154:0/3989209758 conn(0x7f61a806c010 msgr2=0x7f61a806c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:20.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.204+0000 7f61af2a9640 1 -- 172.21.15.154:0/3989209758 shutdown_connections 2023-09-23T03:16:20.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.205+0000 7f61af2a9640 1 -- 172.21.15.154:0/3989209758 wait complete. 2023-09-23T03:16:20.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.206+0000 7f61af2a9640 1 Processor -- start 2023-09-23T03:16:20.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.206+0000 7f61af2a9640 1 -- start start 2023-09-23T03:16:20.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.207+0000 7f61af2a9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61a8071290 0x7f61a815d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:20.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.207+0000 7f61af2a9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61a80c8d70 0x7f61a815d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:20.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.207+0000 7f61af2a9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61a80c9150 0x7f61a815dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:20.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.207+0000 7f61af2a9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a80c4020 con 0x7f61a8071290 2023-09-23T03:16:20.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.207+0000 7f61af2a9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a80c3410 con 0x7f61a80c9150 2023-09-23T03:16:20.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.207+0000 7f61af2a9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a80c36f0 con 0x7f61a80c8d70 2023-09-23T03:16:20.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.208+0000 7f61ad045640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61a8071290 0x7f61a815d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:20.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.208+0000 7f61ad045640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61a8071290 0x7f61a815d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53118/0 (socket says 172.21.15.154:53118) 2023-09-23T03:16:20.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.208+0000 7f61ad045640 1 -- 172.21.15.154:0/875046711 learned_addr learned my addr 172.21.15.154:0/875046711 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:20.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.208+0000 7f61ad846640 1 --2- 172.21.15.154:0/875046711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61a80c9150 0x7f61a815dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:20.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.208+0000 7f61ac844640 1 --2- 172.21.15.154:0/875046711 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61a80c8d70 0x7f61a815d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:20.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.208+0000 7f61ad045640 1 -- 172.21.15.154:0/875046711 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61a80c8d70 msgr2=0x7f61a815d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:20.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.208+0000 7f61ad045640 1 --2- 172.21.15.154:0/875046711 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61a80c8d70 0x7f61a815d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:20.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.208+0000 7f61ad045640 1 -- 172.21.15.154:0/875046711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61a80c9150 msgr2=0x7f61a815dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:20.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.208+0000 7f61ad045640 1 --2- 172.21.15.154:0/875046711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61a80c9150 0x7f61a815dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:20.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.208+0000 7f61ad045640 1 -- 172.21.15.154:0/875046711 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a0009400 con 0x7f61a8071290 2023-09-23T03:16:20.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.208+0000 7f61ad045640 1 --2- 172.21.15.154:0/875046711 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61a8071290 0x7f61a815d020 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f61a400cbc0 tx=0x7f61a400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:20.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.209+0000 7f619dffb640 1 -- 172.21.15.154:0/875046711 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a400ad40 con 0x7f61a8071290 2023-09-23T03:16:20.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.209+0000 7f619dffb640 1 -- 172.21.15.154:0/875046711 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a4012400 con 0x7f61a8071290 2023-09-23T03:16:20.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.209+0000 7f61af2a9640 1 -- 172.21.15.154:0/875046711 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61a80c39d0 con 0x7f61a8071290 2023-09-23T03:16:20.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.209+0000 7f619dffb640 1 -- 172.21.15.154:0/875046711 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a400aea0 con 0x7f61a8071290 2023-09-23T03:16:20.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.209+0000 7f61af2a9640 1 -- 172.21.15.154:0/875046711 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61a81546b0 con 0x7f61a8071290 2023-09-23T03:16:20.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.210+0000 7f61af2a9640 1 -- 172.21.15.154:0/875046711 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61a804e920 con 0x7f61a8071290 2023-09-23T03:16:20.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.211+0000 7f619dffb640 1 -- 172.21.15.154:0/875046711 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f61a4007070 con 0x7f61a8071290 2023-09-23T03:16:20.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.212+0000 7f619dffb640 1 --2- 172.21.15.154:0/875046711 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f618c06af30 0x7f618c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:20.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.212+0000 7f619dffb640 1 -- 172.21.15.154:0/875046711 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f61a4089a90 con 0x7f61a8071290 2023-09-23T03:16:20.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.212+0000 7f61ac844640 1 --2- 172.21.15.154:0/875046711 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f618c06af30 0x7f618c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:20.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.213+0000 7f61ac844640 1 --2- 172.21.15.154:0/875046711 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f618c06af30 0x7f618c06d3f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6198006fd0 tx=0x7f6198008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:20.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.216+0000 7f619dffb640 1 -- 172.21.15.154:0/875046711 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f61a40547d0 con 0x7f61a8071290 2023-09-23T03:16:20.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.391+0000 7f61af2a9640 1 -- 172.21.15.154:0/875046711 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f61a80720c0 con 0x7f61a8071290 2023-09-23T03:16:20.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.392+0000 7f619dffb640 1 -- 172.21.15.154:0/875046711 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f61a4017020 con 0x7f61a8071290 2023-09-23T03:16:20.393 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:16:20.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.395+0000 7f61af2a9640 1 -- 172.21.15.154:0/875046711 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f618c06af30 msgr2=0x7f618c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:20.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.395+0000 7f61af2a9640 1 --2- 172.21.15.154:0/875046711 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f618c06af30 0x7f618c06d3f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6198006fd0 tx=0x7f6198008040 comp rx=0 tx=0).stop 2023-09-23T03:16:20.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.395+0000 7f61af2a9640 1 -- 172.21.15.154:0/875046711 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61a8071290 msgr2=0x7f61a815d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:20.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.395+0000 7f61af2a9640 1 --2- 172.21.15.154:0/875046711 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61a8071290 0x7f61a815d020 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f61a400cbc0 tx=0x7f61a400a430 comp rx=0 tx=0).stop 2023-09-23T03:16:20.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.395+0000 7f61af2a9640 1 -- 172.21.15.154:0/875046711 shutdown_connections 2023-09-23T03:16:20.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.395+0000 7f61af2a9640 1 --2- 172.21.15.154:0/875046711 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f618c06af30 0x7f618c06d3f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:20.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.395+0000 7f61af2a9640 1 --2- 172.21.15.154:0/875046711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61a80c9150 0x7f61a815dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:20.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.395+0000 7f61af2a9640 1 --2- 172.21.15.154:0/875046711 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61a80c8d70 0x7f61a815d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:20.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.395+0000 7f61af2a9640 1 --2- 172.21.15.154:0/875046711 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61a8071290 0x7f61a815d020 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:20.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.395+0000 7f61af2a9640 1 -- 172.21.15.154:0/875046711 >> 172.21.15.154:0/875046711 conn(0x7f61a806c010 msgr2=0x7f61a80b2380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:20.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.395+0000 7f61af2a9640 1 -- 172.21.15.154:0/875046711 shutdown_connections 2023-09-23T03:16:20.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.395+0000 7f61af2a9640 1 -- 172.21.15.154:0/875046711 wait complete. 2023-09-23T03:16:20.411 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:16:20.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.626+0000 7f041b426640 1 Processor -- start 2023-09-23T03:16:20.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.626+0000 7f041b426640 1 -- start start 2023-09-23T03:16:20.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.628+0000 7f041b426640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f04140c8d70 0x7f041415c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:20.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.628+0000 7f041b426640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f04140c9150 0x7f041415cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:20.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.628+0000 7f041b426640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04140c9d20 con 0x7f0414071290 2023-09-23T03:16:20.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.628+0000 7f041b426640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0414071850 con 0x7f04140c9150 2023-09-23T03:16:20.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.628+0000 7f041b426640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0414071b30 con 0x7f04140c8d70 2023-09-23T03:16:20.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.628+0000 7f04189c1640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f04140c9150 0x7f041415cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:20.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.628+0000 7f04199c3640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f0414071290 0x7f04140ca1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:47388/0 (socket says 172.21.15.154:47388) 2023-09-23T03:16:20.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.628+0000 7f04199c3640 1 -- 172.21.15.154:0/124046193 learned_addr learned my addr 172.21.15.154:0/124046193 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:20.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.628+0000 7f04191c2640 1 --2- 172.21.15.154:0/124046193 >> v2:172.21.15.190:3300/0 conn(0x7f04140c8d70 0x7f041415c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:20.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.628+0000 7f040bfff640 1 -- 172.21.15.154:0/124046193 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 275428820 0 0) 0x7f04140c9d20 con 0x7f0414071290 2023-09-23T03:16:20.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.628+0000 7f040bfff640 1 -- 172.21.15.154:0/124046193 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0400003520 con 0x7f0414071290 2023-09-23T03:16:20.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.628+0000 7f04189c1640 1 -- 172.21.15.154:0/124046193 >> v2:172.21.15.190:3300/0 conn(0x7f04140c8d70 msgr2=0x7f041415c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:20.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.628+0000 7f04189c1640 1 --2- 172.21.15.154:0/124046193 >> v2:172.21.15.190:3300/0 conn(0x7f04140c8d70 0x7f041415c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:20.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.629+0000 7f04189c1640 1 -- 172.21.15.154:0/124046193 >> v1:172.21.15.154:6789/0 conn(0x7f0414071290 legacy=0x7f04140ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:20.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.629+0000 7f04189c1640 1 -- 172.21.15.154:0/124046193 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0414071e10 con 0x7f04140c9150 2023-09-23T03:16:20.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.630+0000 7f04189c1640 1 --2- 172.21.15.154:0/124046193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04140c9150 0x7f041415cc10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f041406b790 tx=0x7f040402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd29f2f9c5794b8e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:20.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.630+0000 7f040bfff640 1 -- 172.21.15.154:0/124046193 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0404031830 con 0x7f04140c9150 2023-09-23T03:16:20.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.630+0000 7f040bfff640 1 -- 172.21.15.154:0/124046193 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0404031990 con 0x7f04140c9150 2023-09-23T03:16:20.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.630+0000 7f040bfff640 1 -- 172.21.15.154:0/124046193 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0404031d20 con 0x7f04140c9150 2023-09-23T03:16:20.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.630+0000 7f041b426640 1 -- 172.21.15.154:0/124046193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04140c9150 msgr2=0x7f041415cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:20.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.630+0000 7f041b426640 1 --2- 172.21.15.154:0/124046193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04140c9150 0x7f041415cc10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f041406b790 tx=0x7f040402b8d0 comp rx=0 tx=0).stop 2023-09-23T03:16:20.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.631+0000 7f041b426640 1 -- 172.21.15.154:0/124046193 shutdown_connections 2023-09-23T03:16:20.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.631+0000 7f041b426640 1 --2- 172.21.15.154:0/124046193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04140c9150 0x7f041415cc10 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:20.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.631+0000 7f041b426640 1 --2- 172.21.15.154:0/124046193 >> v2:172.21.15.190:3300/0 conn(0x7f04140c8d70 0x7f041415c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:20.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.631+0000 7f041b426640 1 -- 172.21.15.154:0/124046193 >> 172.21.15.154:0/124046193 conn(0x7f041406c010 msgr2=0x7f041406c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:20.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.631+0000 7f041b426640 1 -- 172.21.15.154:0/124046193 shutdown_connections 2023-09-23T03:16:20.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.631+0000 7f041b426640 1 -- 172.21.15.154:0/124046193 wait complete. 2023-09-23T03:16:20.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.632+0000 7f041b426640 1 Processor -- start 2023-09-23T03:16:20.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.632+0000 7f041b426640 1 -- start start 2023-09-23T03:16:20.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.633+0000 7f041b426640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0414071290 0x7f04140cd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:20.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.633+0000 7f041b426640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04140c8d70 0x7f04140c6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:20.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.633+0000 7f041b426640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04140c9150 0x7f04140c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:20.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.633+0000 7f041b426640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04140cd3c0 con 0x7f04140c9150 2023-09-23T03:16:20.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.633+0000 7f041b426640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04140cc560 con 0x7f0414071290 2023-09-23T03:16:20.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.633+0000 7f041b426640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04140cc7b0 con 0x7f04140c8d70 2023-09-23T03:16:20.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.633+0000 7f04199c3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04140c9150 0x7f04140c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:20.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.634+0000 7f04199c3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04140c9150 0x7f04140c7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53126/0 (socket says 172.21.15.154:53126) 2023-09-23T03:16:20.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.634+0000 7f04199c3640 1 -- 172.21.15.154:0/2303775480 learned_addr learned my addr 172.21.15.154:0/2303775480 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:20.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.634+0000 7f04191c2640 1 --2- 172.21.15.154:0/2303775480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0414071290 0x7f04140cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:20.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.634+0000 7f04199c3640 1 -- 172.21.15.154:0/2303775480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04140c8d70 msgr2=0x7f04140c6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:20.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.634+0000 7f04199c3640 1 --2- 172.21.15.154:0/2303775480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04140c8d70 0x7f04140c6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:20.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.634+0000 7f04199c3640 1 -- 172.21.15.154:0/2303775480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0414071290 msgr2=0x7f04140cd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:20.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.634+0000 7f04199c3640 1 --2- 172.21.15.154:0/2303775480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0414071290 0x7f04140cd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:20.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.634+0000 7f04199c3640 1 -- 172.21.15.154:0/2303775480 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0404009400 con 0x7f04140c9150 2023-09-23T03:16:20.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.634+0000 7f04191c2640 1 --2- 172.21.15.154:0/2303775480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0414071290 0x7f04140cd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:20.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.634+0000 7f04199c3640 1 --2- 172.21.15.154:0/2303775480 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04140c9150 0x7f04140c7c90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f040c00cb40 tx=0x7f040c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:20.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.634+0000 7f0409ffb640 1 -- 172.21.15.154:0/2303775480 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c00aca0 con 0x7f04140c9150 2023-09-23T03:16:20.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.634+0000 7f0409ffb640 1 -- 172.21.15.154:0/2303775480 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f040c00ae00 con 0x7f04140c9150 2023-09-23T03:16:20.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.634+0000 7f0409ffb640 1 -- 172.21.15.154:0/2303775480 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c012620 con 0x7f04140c9150 2023-09-23T03:16:20.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.634+0000 7f041b426640 1 -- 172.21.15.154:0/2303775480 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04140cca90 con 0x7f04140c9150 2023-09-23T03:16:20.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.635+0000 7f041b426640 1 -- 172.21.15.154:0/2303775480 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04140cd050 con 0x7f04140c9150 2023-09-23T03:16:20.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.636+0000 7f041b426640 1 -- 172.21.15.154:0/2303775480 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04140cd570 con 0x7f04140c9150 2023-09-23T03:16:20.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.636+0000 7f0409ffb640 1 -- 172.21.15.154:0/2303775480 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f040c007070 con 0x7f04140c9150 2023-09-23T03:16:20.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.637+0000 7f0409ffb640 1 --2- 172.21.15.154:0/2303775480 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f03f806af60 0x7f03f806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:20.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.637+0000 7f0409ffb640 1 -- 172.21.15.154:0/2303775480 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f040c089c00 con 0x7f04140c9150 2023-09-23T03:16:20.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.637+0000 7f04191c2640 1 --2- 172.21.15.154:0/2303775480 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f03f806af60 0x7f03f806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:20.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.638+0000 7f04191c2640 1 --2- 172.21.15.154:0/2303775480 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f03f806af60 0x7f03f806d420 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0410006fd0 tx=0x7f0410008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:20.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.641+0000 7f0409ffb640 1 -- 172.21.15.154:0/2303775480 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f04140cd570 con 0x7f04140c9150 2023-09-23T03:16:20.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.827+0000 7f041b426640 1 -- 172.21.15.154:0/2303775480 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f04140cd570 con 0x7f04140c9150 2023-09-23T03:16:20.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.828+0000 7f0409ffb640 1 -- 172.21.15.154:0/2303775480 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f04140cd570 con 0x7f04140c9150 2023-09-23T03:16:20.830 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:16:20.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.831+0000 7f041b426640 1 -- 172.21.15.154:0/2303775480 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f03f806af60 msgr2=0x7f03f806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:20.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.831+0000 7f041b426640 1 --2- 172.21.15.154:0/2303775480 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f03f806af60 0x7f03f806d420 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0410006fd0 tx=0x7f0410008040 comp rx=0 tx=0).stop 2023-09-23T03:16:20.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.831+0000 7f041b426640 1 -- 172.21.15.154:0/2303775480 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04140c9150 msgr2=0x7f04140c7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:20.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.831+0000 7f041b426640 1 --2- 172.21.15.154:0/2303775480 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04140c9150 0x7f04140c7c90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f040c00cb40 tx=0x7f040c00a060 comp rx=0 tx=0).stop 2023-09-23T03:16:20.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.831+0000 7f041b426640 1 -- 172.21.15.154:0/2303775480 shutdown_connections 2023-09-23T03:16:20.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.831+0000 7f041b426640 1 --2- 172.21.15.154:0/2303775480 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f03f806af60 0x7f03f806d420 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:20.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.831+0000 7f041b426640 1 --2- 172.21.15.154:0/2303775480 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04140c9150 0x7f04140c7c90 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:20.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.831+0000 7f041b426640 1 --2- 172.21.15.154:0/2303775480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f04140c8d70 0x7f04140c6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:20.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.831+0000 7f041b426640 1 --2- 172.21.15.154:0/2303775480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0414071290 0x7f04140cd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:20.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.831+0000 7f041b426640 1 -- 172.21.15.154:0/2303775480 >> 172.21.15.154:0/2303775480 conn(0x7f041406c010 msgr2=0x7f041415b430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:20.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.831+0000 7f041b426640 1 -- 172.21.15.154:0/2303775480 shutdown_connections 2023-09-23T03:16:20.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:20.831+0000 7f041b426640 1 -- 172.21.15.154:0/2303775480 wait complete. 2023-09-23T03:16:20.851 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:16:21.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.067+0000 7fc996978640 1 Processor -- start 2023-09-23T03:16:21.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.067+0000 7fc996978640 1 -- start start 2023-09-23T03:16:21.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.069+0000 7fc996978640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc9900c9150 0x7fc99015cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:21.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.069+0000 7fc996978640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc990071290 0x7fc9900ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:21.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.069+0000 7fc996978640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9900c9d20 con 0x7fc9900c8d70 2023-09-23T03:16:21.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.069+0000 7fc996978640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc990071850 con 0x7fc9900c9150 2023-09-23T03:16:21.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.069+0000 7fc996978640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc990071b30 con 0x7fc990071290 2023-09-23T03:16:21.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.069+0000 7fc98ffff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fc9900c8d70 0x7fc99015c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:47396/0 (socket says 172.21.15.154:47396) 2023-09-23T03:16:21.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.069+0000 7fc98ffff640 1 -- 172.21.15.154:0/3701128789 learned_addr learned my addr 172.21.15.154:0/3701128789 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:21.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.069+0000 7fc98f7fe640 1 --2- 172.21.15.154:0/3701128789 >> v2:172.21.15.154:3300/0 conn(0x7fc9900c9150 0x7fc99015cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:21.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.069+0000 7fc994f15640 1 --2- 172.21.15.154:0/3701128789 >> v2:172.21.15.190:3300/0 conn(0x7fc990071290 0x7fc9900ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:21.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.069+0000 7fc98effd640 1 -- 172.21.15.154:0/3701128789 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 847284779 0 0) 0x7fc9900c9d20 con 0x7fc9900c8d70 2023-09-23T03:16:21.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.069+0000 7fc98effd640 1 -- 172.21.15.154:0/3701128789 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc978003520 con 0x7fc9900c8d70 2023-09-23T03:16:21.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.070+0000 7fc98f7fe640 1 -- 172.21.15.154:0/3701128789 >> v2:172.21.15.190:3300/0 conn(0x7fc990071290 msgr2=0x7fc9900ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:21.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.070+0000 7fc98f7fe640 1 --2- 172.21.15.154:0/3701128789 >> v2:172.21.15.190:3300/0 conn(0x7fc990071290 0x7fc9900ca1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:21.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.070+0000 7fc98f7fe640 1 -- 172.21.15.154:0/3701128789 >> v1:172.21.15.154:6789/0 conn(0x7fc9900c8d70 legacy=0x7fc99015c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:21.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.070+0000 7fc98f7fe640 1 -- 172.21.15.154:0/3701128789 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc990071e10 con 0x7fc9900c9150 2023-09-23T03:16:21.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.070+0000 7fc994f15640 1 --2- 172.21.15.154:0/3701128789 >> v2:172.21.15.190:3300/0 conn(0x7fc990071290 0x7fc9900ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:16:21.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.071+0000 7fc98f7fe640 1 --2- 172.21.15.154:0/3701128789 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9900c9150 0x7fc99015cde0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc9900b3820 tx=0x7fc98402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66f23171b1511126 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:21.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.071+0000 7fc98effd640 1 -- 172.21.15.154:0/3701128789 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc984031830 con 0x7fc9900c9150 2023-09-23T03:16:21.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.071+0000 7fc98effd640 1 -- 172.21.15.154:0/3701128789 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc984031990 con 0x7fc9900c9150 2023-09-23T03:16:21.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.071+0000 7fc98effd640 1 -- 172.21.15.154:0/3701128789 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc984031d20 con 0x7fc9900c9150 2023-09-23T03:16:21.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.072+0000 7fc996978640 1 -- 172.21.15.154:0/3701128789 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9900c9150 msgr2=0x7fc99015cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:21.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.072+0000 7fc996978640 1 --2- 172.21.15.154:0/3701128789 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9900c9150 0x7fc99015cde0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc9900b3820 tx=0x7fc98402b8d0 comp rx=0 tx=0).stop 2023-09-23T03:16:21.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.072+0000 7fc996978640 1 -- 172.21.15.154:0/3701128789 shutdown_connections 2023-09-23T03:16:21.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.072+0000 7fc996978640 1 --2- 172.21.15.154:0/3701128789 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9900c9150 0x7fc99015cde0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:21.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.072+0000 7fc996978640 1 --2- 172.21.15.154:0/3701128789 >> v2:172.21.15.190:3300/0 conn(0x7fc990071290 0x7fc9900ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:21.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.072+0000 7fc996978640 1 -- 172.21.15.154:0/3701128789 >> 172.21.15.154:0/3701128789 conn(0x7fc99006c010 msgr2=0x7fc99006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:21.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.072+0000 7fc996978640 1 -- 172.21.15.154:0/3701128789 shutdown_connections 2023-09-23T03:16:21.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.072+0000 7fc996978640 1 -- 172.21.15.154:0/3701128789 wait complete. 2023-09-23T03:16:21.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.074+0000 7fc996978640 1 Processor -- start 2023-09-23T03:16:21.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.074+0000 7fc996978640 1 -- start start 2023-09-23T03:16:21.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.075+0000 7fc996978640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc990071290 0x7fc9900c38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:21.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.075+0000 7fc996978640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9900c8d70 0x7fc9900b2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:21.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.075+0000 7fc996978640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9900c9150 0x7fc9900b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:21.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.075+0000 7fc996978640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99015b530 con 0x7fc9900c9150 2023-09-23T03:16:21.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.075+0000 7fc996978640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99015a920 con 0x7fc990071290 2023-09-23T03:16:21.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.075+0000 7fc996978640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc99015ac00 con 0x7fc9900c8d70 2023-09-23T03:16:21.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.075+0000 7fc994f15640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9900c9150 0x7fc9900b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:21.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.075+0000 7fc994f15640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9900c9150 0x7fc9900b28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53148/0 (socket says 172.21.15.154:53148) 2023-09-23T03:16:21.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.075+0000 7fc994f15640 1 -- 172.21.15.154:0/1639771960 learned_addr learned my addr 172.21.15.154:0/1639771960 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:21.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.075+0000 7fc98ffff640 1 --2- 172.21.15.154:0/1639771960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc990071290 0x7fc9900c38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:21.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.076+0000 7fc994f15640 1 -- 172.21.15.154:0/1639771960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9900c8d70 msgr2=0x7fc9900b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:21.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.076+0000 7fc994f15640 1 --2- 172.21.15.154:0/1639771960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9900c8d70 0x7fc9900b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:21.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.076+0000 7fc994f15640 1 -- 172.21.15.154:0/1639771960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc990071290 msgr2=0x7fc9900c38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:21.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.076+0000 7fc994f15640 1 --2- 172.21.15.154:0/1639771960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc990071290 0x7fc9900c38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:21.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.076+0000 7fc994f15640 1 -- 172.21.15.154:0/1639771960 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc984009400 con 0x7fc9900c9150 2023-09-23T03:16:21.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.076+0000 7fc994f15640 1 --2- 172.21.15.154:0/1639771960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9900c9150 0x7fc9900b28c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc988007ef0 tx=0x7fc98800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:21.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.076+0000 7fc98cff9640 1 -- 172.21.15.154:0/1639771960 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc988003b90 con 0x7fc9900c9150 2023-09-23T03:16:21.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.076+0000 7fc98cff9640 1 -- 172.21.15.154:0/1639771960 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc98800f040 con 0x7fc9900c9150 2023-09-23T03:16:21.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.076+0000 7fc98cff9640 1 -- 172.21.15.154:0/1639771960 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98800de80 con 0x7fc9900c9150 2023-09-23T03:16:21.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.076+0000 7fc996978640 1 -- 172.21.15.154:0/1639771960 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc99015aee0 con 0x7fc9900c9150 2023-09-23T03:16:21.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.076+0000 7fc996978640 1 -- 172.21.15.154:0/1639771960 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc99015a560 con 0x7fc9900c9150 2023-09-23T03:16:21.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.077+0000 7fc996978640 1 -- 172.21.15.154:0/1639771960 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc99004e920 con 0x7fc9900c9150 2023-09-23T03:16:21.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.078+0000 7fc98cff9640 1 -- 172.21.15.154:0/1639771960 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fc988004030 con 0x7fc9900c9150 2023-09-23T03:16:21.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.079+0000 7fc98cff9640 1 --2- 172.21.15.154:0/1639771960 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc97c06af30 0x7fc97c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:21.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.079+0000 7fc98cff9640 1 -- 172.21.15.154:0/1639771960 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7fc988089da0 con 0x7fc9900c9150 2023-09-23T03:16:21.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.079+0000 7fc98ffff640 1 --2- 172.21.15.154:0/1639771960 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc97c06af30 0x7fc97c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:21.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.080+0000 7fc98ffff640 1 --2- 172.21.15.154:0/1639771960 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc97c06af30 0x7fc97c06d3f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc9800011e0 tx=0x7fc980009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:21.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.083+0000 7fc98cff9640 1 -- 172.21.15.154:0/1639771960 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fc988054ae0 con 0x7fc9900c9150 2023-09-23T03:16:21.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.273+0000 7fc996978640 1 -- 172.21.15.154:0/1639771960 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc99015b820 con 0x7fc9900c9150 2023-09-23T03:16:21.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.273+0000 7fc98cff9640 1 -- 172.21.15.154:0/1639771960 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc98801a0a0 con 0x7fc9900c9150 2023-09-23T03:16:21.274 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:16:21.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.276+0000 7fc996978640 1 -- 172.21.15.154:0/1639771960 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc97c06af30 msgr2=0x7fc97c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:21.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.276+0000 7fc996978640 1 --2- 172.21.15.154:0/1639771960 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc97c06af30 0x7fc97c06d3f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc9800011e0 tx=0x7fc980009040 comp rx=0 tx=0).stop 2023-09-23T03:16:21.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.276+0000 7fc996978640 1 -- 172.21.15.154:0/1639771960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9900c9150 msgr2=0x7fc9900b28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:21.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.276+0000 7fc996978640 1 --2- 172.21.15.154:0/1639771960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9900c9150 0x7fc9900b28c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc988007ef0 tx=0x7fc98800d4a0 comp rx=0 tx=0).stop 2023-09-23T03:16:21.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.276+0000 7fc996978640 1 -- 172.21.15.154:0/1639771960 shutdown_connections 2023-09-23T03:16:21.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.276+0000 7fc996978640 1 --2- 172.21.15.154:0/1639771960 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc97c06af30 0x7fc97c06d3f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:21.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.276+0000 7fc996978640 1 --2- 172.21.15.154:0/1639771960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9900c9150 0x7fc9900b28c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:21.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.276+0000 7fc996978640 1 --2- 172.21.15.154:0/1639771960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9900c8d70 0x7fc9900b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:21.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.276+0000 7fc996978640 1 --2- 172.21.15.154:0/1639771960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc990071290 0x7fc9900c38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:21.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.276+0000 7fc996978640 1 -- 172.21.15.154:0/1639771960 >> 172.21.15.154:0/1639771960 conn(0x7fc99006c010 msgr2=0x7fc9900caf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:21.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.276+0000 7fc996978640 1 -- 172.21.15.154:0/1639771960 shutdown_connections 2023-09-23T03:16:21.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.276+0000 7fc996978640 1 -- 172.21.15.154:0/1639771960 wait complete. 2023-09-23T03:16:21.295 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-23T03:16:21.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.509+0000 7f63da4c3640 1 Processor -- start 2023-09-23T03:16:21.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.509+0000 7f63da4c3640 1 -- start start 2023-09-23T03:16:21.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.511+0000 7f63da4c3640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f63d40c9410 0x7f63d4165520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:21.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.511+0000 7f63da4c3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63d40b0a80 0x7f63d40cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:21.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.511+0000 7f63da4c3640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f63d40cade0 0x7f63d40cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:21.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.511+0000 7f63da4c3640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63d40cd660 con 0x7f63d40c9410 2023-09-23T03:16:21.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.511+0000 7f63da4c3640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63d40cb7b0 con 0x7f63d40cade0 2023-09-23T03:16:21.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.511+0000 7f63da4c3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d40c97f0 con 0x7f63d40b0a80 2023-09-23T03:16:21.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.511+0000 7f63d94c1640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f63d40c9410 0x7f63d4165520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:21.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.511+0000 7f63d94c1640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f63d40c9410 0x7f63d4165520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53164/0 (socket says 172.21.15.154:53164) 2023-09-23T03:16:21.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.511+0000 7f63d94c1640 1 -- 172.21.15.154:0/3763365413 learned_addr learned my addr 172.21.15.154:0/3763365413 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:21.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.511+0000 7f63d8cc0640 1 --2- 172.21.15.154:0/3763365413 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63d40b0a80 0x7f63d40cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:21.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.511+0000 7f63d94c1640 1 -- 172.21.15.154:0/3763365413 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63d40b0a80 msgr2=0x7f63d40cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:21.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.511+0000 7f63d94c1640 1 --2- 172.21.15.154:0/3763365413 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63d40b0a80 0x7f63d40cc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:21.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.511+0000 7f63d94c1640 1 -- 172.21.15.154:0/3763365413 >> v2:172.21.15.190:3300/0 conn(0x7f63d40cade0 msgr2=0x7f63d40cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:21.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.511+0000 7f63d94c1640 1 --2- 172.21.15.154:0/3763365413 >> v2:172.21.15.190:3300/0 conn(0x7f63d40cade0 0x7f63d40cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:21.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.511+0000 7f63d94c1640 1 -- 172.21.15.154:0/3763365413 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d4161450 con 0x7f63d40c9410 2023-09-23T03:16:21.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.511+0000 7f63d8cc0640 1 --2- 172.21.15.154:0/3763365413 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63d40b0a80 0x7f63d40cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:21.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.513+0000 7f63d94c1640 1 --2- 172.21.15.154:0/3763365413 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63d40c9410 0x7f63d4165520 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f63d40c1e30 tx=0x7f63d002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc432c2acb449c1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:21.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.513+0000 7f63cbfff640 1 -- 172.21.15.154:0/3763365413 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d0031870 con 0x7f63d40c9410 2023-09-23T03:16:21.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.513+0000 7f63cbfff640 1 -- 172.21.15.154:0/3763365413 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d00319d0 con 0x7f63d40c9410 2023-09-23T03:16:21.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.513+0000 7f63cbfff640 1 -- 172.21.15.154:0/3763365413 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d0031d60 con 0x7f63d40c9410 2023-09-23T03:16:21.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.513+0000 7f63da4c3640 1 -- 172.21.15.154:0/3763365413 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63d40c9410 msgr2=0x7f63d4165520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:21.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.513+0000 7f63da4c3640 1 --2- 172.21.15.154:0/3763365413 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63d40c9410 0x7f63d4165520 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f63d40c1e30 tx=0x7f63d002b910 comp rx=0 tx=0).stop 2023-09-23T03:16:21.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.513+0000 7f63da4c3640 1 -- 172.21.15.154:0/3763365413 shutdown_connections 2023-09-23T03:16:21.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.513+0000 7f63da4c3640 1 --2- 172.21.15.154:0/3763365413 >> v2:172.21.15.190:3300/0 conn(0x7f63d40cade0 0x7f63d40cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:21.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.514+0000 7f63da4c3640 1 --2- 172.21.15.154:0/3763365413 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63d40b0a80 0x7f63d40cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:21.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.514+0000 7f63da4c3640 1 --2- 172.21.15.154:0/3763365413 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63d40c9410 0x7f63d4165520 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:21.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.514+0000 7f63da4c3640 1 -- 172.21.15.154:0/3763365413 >> 172.21.15.154:0/3763365413 conn(0x7f63d40aff20 msgr2=0x7f63d40b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:21.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.514+0000 7f63da4c3640 1 -- 172.21.15.154:0/3763365413 shutdown_connections 2023-09-23T03:16:21.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.514+0000 7f63da4c3640 1 -- 172.21.15.154:0/3763365413 wait complete. 2023-09-23T03:16:21.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.515+0000 7f63da4c3640 1 Processor -- start 2023-09-23T03:16:21.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.515+0000 7f63da4c3640 1 -- start start 2023-09-23T03:16:21.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.516+0000 7f63da4c3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63d40b0a80 0x7f63d40c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:21.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.516+0000 7f63da4c3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63d40c9410 0x7f63d41637e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:21.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.516+0000 7f63da4c3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63d40cade0 0x7f63d4161780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:21.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.517+0000 7f63da4c3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d40c8010 con 0x7f63d40c9410 2023-09-23T03:16:21.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.517+0000 7f63da4c3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d41645a0 con 0x7f63d40b0a80 2023-09-23T03:16:21.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.517+0000 7f63da4c3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d415b060 con 0x7f63d40cade0 2023-09-23T03:16:21.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.517+0000 7f63d8cc0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63d40c9410 0x7f63d41637e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:21.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.517+0000 7f63d8cc0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63d40c9410 0x7f63d41637e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53172/0 (socket says 172.21.15.154:53172) 2023-09-23T03:16:21.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.517+0000 7f63d8cc0640 1 -- 172.21.15.154:0/296936938 learned_addr learned my addr 172.21.15.154:0/296936938 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:21.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.517+0000 7f63d9cc2640 1 --2- 172.21.15.154:0/296936938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63d40cade0 0x7f63d4161780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:21.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.517+0000 7f63d94c1640 1 --2- 172.21.15.154:0/296936938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63d40b0a80 0x7f63d40c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:21.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.517+0000 7f63d8cc0640 1 -- 172.21.15.154:0/296936938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63d40cade0 msgr2=0x7f63d4161780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:21.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.517+0000 7f63d8cc0640 1 --2- 172.21.15.154:0/296936938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63d40cade0 0x7f63d4161780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:21.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.517+0000 7f63d8cc0640 1 -- 172.21.15.154:0/296936938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63d40b0a80 msgr2=0x7f63d40c7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:21.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.517+0000 7f63d8cc0640 1 --2- 172.21.15.154:0/296936938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63d40b0a80 0x7f63d40c7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:21.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.517+0000 7f63d8cc0640 1 -- 172.21.15.154:0/296936938 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d00094d0 con 0x7f63d40c9410 2023-09-23T03:16:21.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.517+0000 7f63d9cc2640 1 --2- 172.21.15.154:0/296936938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63d40cade0 0x7f63d4161780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:21.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.518+0000 7f63d8cc0640 1 --2- 172.21.15.154:0/296936938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63d40c9410 0x7f63d41637e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f63c400c7f0 tx=0x7f63c400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:21.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.518+0000 7f63c9ffb640 1 -- 172.21.15.154:0/296936938 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c400cea0 con 0x7f63d40c9410 2023-09-23T03:16:21.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.518+0000 7f63c9ffb640 1 -- 172.21.15.154:0/296936938 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c4007a30 con 0x7f63d40c9410 2023-09-23T03:16:21.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.518+0000 7f63da4c3640 1 -- 172.21.15.154:0/296936938 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d41ea360 con 0x7f63d40c9410 2023-09-23T03:16:21.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.518+0000 7f63c9ffb640 1 -- 172.21.15.154:0/296936938 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c4007dc0 con 0x7f63d40c9410 2023-09-23T03:16:21.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.518+0000 7f63da4c3640 1 -- 172.21.15.154:0/296936938 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d4163fe0 con 0x7f63d40c9410 2023-09-23T03:16:21.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.519+0000 7f63da4c3640 1 -- 172.21.15.154:0/296936938 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63d404e920 con 0x7f63d40c9410 2023-09-23T03:16:21.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.520+0000 7f63c9ffb640 1 -- 172.21.15.154:0/296936938 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f63c40113f0 con 0x7f63d40c9410 2023-09-23T03:16:21.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.521+0000 7f63c9ffb640 1 --2- 172.21.15.154:0/296936938 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f63b806af30 0x7f63b806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:21.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.521+0000 7f63c9ffb640 1 -- 172.21.15.154:0/296936938 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f63c4089c70 con 0x7f63d40c9410 2023-09-23T03:16:21.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.521+0000 7f63d94c1640 1 --2- 172.21.15.154:0/296936938 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f63b806af30 0x7f63b806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:21.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.522+0000 7f63d94c1640 1 --2- 172.21.15.154:0/296936938 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f63b806af30 0x7f63b806d3f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f63d0036000 tx=0x7f63d003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:21.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.526+0000 7f63c9ffb640 1 -- 172.21.15.154:0/296936938 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f63c40548f0 con 0x7f63d40c9410 2023-09-23T03:16:21.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.714+0000 7f63da4c3640 1 -- 172.21.15.154:0/296936938 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f63d40b4350 con 0x7f63d40c9410 2023-09-23T03:16:21.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.715+0000 7f63c9ffb640 1 -- 172.21.15.154:0/296936938 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f63c401f0a0 con 0x7f63d40c9410 2023-09-23T03:16:21.715 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:16:21.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.717+0000 7f63da4c3640 1 -- 172.21.15.154:0/296936938 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f63b806af30 msgr2=0x7f63b806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:21.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.717+0000 7f63da4c3640 1 --2- 172.21.15.154:0/296936938 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f63b806af30 0x7f63b806d3f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f63d0036000 tx=0x7f63d003c040 comp rx=0 tx=0).stop 2023-09-23T03:16:21.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.717+0000 7f63da4c3640 1 -- 172.21.15.154:0/296936938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63d40c9410 msgr2=0x7f63d41637e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:21.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.717+0000 7f63da4c3640 1 --2- 172.21.15.154:0/296936938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63d40c9410 0x7f63d41637e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f63c400c7f0 tx=0x7f63c400ccb0 comp rx=0 tx=0).stop 2023-09-23T03:16:21.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.718+0000 7f63da4c3640 1 -- 172.21.15.154:0/296936938 shutdown_connections 2023-09-23T03:16:21.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.718+0000 7f63da4c3640 1 --2- 172.21.15.154:0/296936938 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f63b806af30 0x7f63b806d3f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:21.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.718+0000 7f63da4c3640 1 --2- 172.21.15.154:0/296936938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63d40cade0 0x7f63d4161780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:21.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.718+0000 7f63da4c3640 1 --2- 172.21.15.154:0/296936938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63d40c9410 0x7f63d41637e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:21.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.718+0000 7f63da4c3640 1 --2- 172.21.15.154:0/296936938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63d40b0a80 0x7f63d40c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:21.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.718+0000 7f63da4c3640 1 -- 172.21.15.154:0/296936938 >> 172.21.15.154:0/296936938 conn(0x7f63d40aff20 msgr2=0x7f63d40c2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:21.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.718+0000 7f63da4c3640 1 -- 172.21.15.154:0/296936938 shutdown_connections 2023-09-23T03:16:21.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.718+0000 7f63da4c3640 1 -- 172.21.15.154:0/296936938 wait complete. 2023-09-23T03:16:21.737 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:16:21.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.950+0000 7fa575990640 1 Processor -- start 2023-09-23T03:16:21.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.951+0000 7fa575990640 1 -- start start 2023-09-23T03:16:21.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.952+0000 7fa575990640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa5700ada30 0x7fa5700ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:21.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.952+0000 7fa575990640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5700cf130 0x7fa57015d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:21.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.952+0000 7fa575990640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa57006ceb0 con 0x7fa5700b2fe0 2023-09-23T03:16:21.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.952+0000 7fa575990640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5700ade10 con 0x7fa5700ada30 2023-09-23T03:16:21.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.952+0000 7fa575990640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa570159c90 con 0x7fa5700cf130 2023-09-23T03:16:21.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.952+0000 7fa5667fc640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fa5700b2fe0 0x7fa5700cea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:47402/0 (socket says 172.21.15.154:47402) 2023-09-23T03:16:21.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.952+0000 7fa56effd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa5700ada30 0x7fa5700ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:21.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.952+0000 7fa5667fc640 1 -- 172.21.15.154:0/3129277896 learned_addr learned my addr 172.21.15.154:0/3129277896 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:21.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.953+0000 7fa56effd640 1 -- 172.21.15.154:0/3129277896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5700cf130 msgr2=0x7fa57015d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:21.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.953+0000 7fa56f7fe640 1 --2- 172.21.15.154:0/3129277896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5700cf130 0x7fa57015d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:21.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.953+0000 7fa56effd640 1 --2- 172.21.15.154:0/3129277896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5700cf130 0x7fa57015d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:21.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.953+0000 7fa56effd640 1 -- 172.21.15.154:0/3129277896 >> v1:172.21.15.154:6789/0 conn(0x7fa5700b2fe0 legacy=0x7fa5700cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:21.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.953+0000 7fa56e7fc640 1 -- 172.21.15.154:0/3129277896 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 48001468 0 0) 0x7fa558001490 con 0x7fa5700b2fe0 2023-09-23T03:16:21.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.953+0000 7fa56effd640 1 -- 172.21.15.154:0/3129277896 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa57006c110 con 0x7fa5700ada30 2023-09-23T03:16:21.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.953+0000 7fa56f7fe640 1 --2- 172.21.15.154:0/3129277896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5700cf130 0x7fa57015d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:21.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.954+0000 7fa56effd640 1 --2- 172.21.15.154:0/3129277896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5700ada30 0x7fa5700ccaf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa560009790 tx=0x7fa56002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a6829e68f8dbeed server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:21.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.954+0000 7fa56e7fc640 1 -- 172.21.15.154:0/3129277896 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56002d040 con 0x7fa5700ada30 2023-09-23T03:16:21.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.954+0000 7fa56e7fc640 1 -- 172.21.15.154:0/3129277896 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa560031810 con 0x7fa5700ada30 2023-09-23T03:16:21.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.954+0000 7fa56e7fc640 1 -- 172.21.15.154:0/3129277896 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa560031ba0 con 0x7fa5700ada30 2023-09-23T03:16:21.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.955+0000 7fa575990640 1 -- 172.21.15.154:0/3129277896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5700ada30 msgr2=0x7fa5700ccaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:21.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.955+0000 7fa575990640 1 --2- 172.21.15.154:0/3129277896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5700ada30 0x7fa5700ccaf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa560009790 tx=0x7fa56002b8b0 comp rx=0 tx=0).stop 2023-09-23T03:16:21.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.955+0000 7fa575990640 1 -- 172.21.15.154:0/3129277896 shutdown_connections 2023-09-23T03:16:21.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.955+0000 7fa575990640 1 --2- 172.21.15.154:0/3129277896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5700cf130 0x7fa57015d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:21.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.955+0000 7fa575990640 1 --2- 172.21.15.154:0/3129277896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5700ada30 0x7fa5700ccaf0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:21.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.955+0000 7fa575990640 1 -- 172.21.15.154:0/3129277896 >> 172.21.15.154:0/3129277896 conn(0x7fa57006a0f0 msgr2=0x7fa57006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:21.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.955+0000 7fa575990640 1 -- 172.21.15.154:0/3129277896 shutdown_connections 2023-09-23T03:16:21.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.955+0000 7fa575990640 1 -- 172.21.15.154:0/3129277896 wait complete. 2023-09-23T03:16:21.958 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.957+0000 7fa575990640 1 Processor -- start 2023-09-23T03:16:21.958 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.957+0000 7fa575990640 1 -- start start 2023-09-23T03:16:21.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.958+0000 7fa575990640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5700ada30 0x7fa5700977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:21.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.958+0000 7fa575990640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5700b2fe0 0x7fa570153140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:21.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.958+0000 7fa575990640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5700cf130 0x7fa570153680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:21.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.958+0000 7fa575990640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5700c3280 con 0x7fa5700cf130 2023-09-23T03:16:21.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.958+0000 7fa575990640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5700c21c0 con 0x7fa5700b2fe0 2023-09-23T03:16:21.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.958+0000 7fa575990640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa57015c5a0 con 0x7fa5700ada30 2023-09-23T03:16:21.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.958+0000 7fa56f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5700cf130 0x7fa570153680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:21.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.958+0000 7fa56f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5700cf130 0x7fa570153680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53196/0 (socket says 172.21.15.154:53196) 2023-09-23T03:16:21.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.958+0000 7fa56f7fe640 1 -- 172.21.15.154:0/3769242961 learned_addr learned my addr 172.21.15.154:0/3769242961 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:21.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.959+0000 7fa56effd640 1 --2- 172.21.15.154:0/3769242961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5700ada30 0x7fa5700977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:21.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.959+0000 7fa56f7fe640 1 -- 172.21.15.154:0/3769242961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5700ada30 msgr2=0x7fa5700977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:21.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.959+0000 7fa56f7fe640 1 --2- 172.21.15.154:0/3769242961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5700ada30 0x7fa5700977b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:21.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.959+0000 7fa5667fc640 1 --2- 172.21.15.154:0/3769242961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5700b2fe0 0x7fa570153140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:21.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.959+0000 7fa56f7fe640 1 -- 172.21.15.154:0/3769242961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5700b2fe0 msgr2=0x7fa570153140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:21.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.959+0000 7fa56f7fe640 1 --2- 172.21.15.154:0/3769242961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5700b2fe0 0x7fa570153140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:21.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.959+0000 7fa56f7fe640 1 -- 172.21.15.154:0/3769242961 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5600093a0 con 0x7fa5700cf130 2023-09-23T03:16:21.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.959+0000 7fa5667fc640 1 --2- 172.21.15.154:0/3769242961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5700b2fe0 0x7fa570153140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:21.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.959+0000 7fa56f7fe640 1 --2- 172.21.15.154:0/3769242961 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5700cf130 0x7fa570153680 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa56800c5e0 tx=0x7fa56800caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:21.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.959+0000 7fa56cff9640 1 -- 172.21.15.154:0/3769242961 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa568004220 con 0x7fa5700cf130 2023-09-23T03:16:21.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.959+0000 7fa56cff9640 1 -- 172.21.15.154:0/3769242961 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5680043a0 con 0x7fa5700cf130 2023-09-23T03:16:21.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.959+0000 7fa56cff9640 1 -- 172.21.15.154:0/3769242961 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa568003850 con 0x7fa5700cf130 2023-09-23T03:16:21.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.959+0000 7fa575990640 1 -- 172.21.15.154:0/3769242961 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5700c1ee0 con 0x7fa5700cf130 2023-09-23T03:16:21.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.960+0000 7fa575990640 1 -- 172.21.15.154:0/3769242961 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa570070e40 con 0x7fa5700cf130 2023-09-23T03:16:21.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.961+0000 7fa575990640 1 -- 172.21.15.154:0/3769242961 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5700c33c0 con 0x7fa5700cf130 2023-09-23T03:16:21.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.962+0000 7fa56cff9640 1 -- 172.21.15.154:0/3769242961 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fa5680039b0 con 0x7fa5700cf130 2023-09-23T03:16:21.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.963+0000 7fa56cff9640 1 --2- 172.21.15.154:0/3769242961 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa55006af30 0x7fa55006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:21.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.963+0000 7fa56cff9640 1 -- 172.21.15.154:0/3769242961 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7fa568089840 con 0x7fa5700cf130 2023-09-23T03:16:21.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.963+0000 7fa56effd640 1 --2- 172.21.15.154:0/3769242961 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa55006af30 0x7fa55006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:21.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.964+0000 7fa56effd640 1 --2- 172.21.15.154:0/3769242961 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa55006af30 0x7fa55006d3f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa560036000 tx=0x7fa56003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:21.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:21.967+0000 7fa56cff9640 1 -- 172.21.15.154:0/3769242961 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fa5700c33c0 con 0x7fa5700cf130 2023-09-23T03:16:22.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.158+0000 7fa575990640 1 -- 172.21.15.154:0/3769242961 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5700c33c0 con 0x7fa5700cf130 2023-09-23T03:16:22.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.160+0000 7fa56cff9640 1 -- 172.21.15.154:0/3769242961 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18744 (secure 0 0 0) 0x7fa5700c33c0 con 0x7fa5700cf130 2023-09-23T03:16:22.161 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:16:22.161 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":26,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:16:03.151371+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:15:18.575849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T03:15:19.707212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T03:15:23.337138+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:16:22.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.163+0000 7fa575990640 1 -- 172.21.15.154:0/3769242961 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa55006af30 msgr2=0x7fa55006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:22.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.163+0000 7fa575990640 1 --2- 172.21.15.154:0/3769242961 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa55006af30 0x7fa55006d3f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa560036000 tx=0x7fa56003c040 comp rx=0 tx=0).stop 2023-09-23T03:16:22.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.163+0000 7fa575990640 1 -- 172.21.15.154:0/3769242961 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5700cf130 msgr2=0x7fa570153680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:22.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.163+0000 7fa575990640 1 --2- 172.21.15.154:0/3769242961 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5700cf130 0x7fa570153680 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa56800c5e0 tx=0x7fa56800caa0 comp rx=0 tx=0).stop 2023-09-23T03:16:22.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.163+0000 7fa575990640 1 -- 172.21.15.154:0/3769242961 shutdown_connections 2023-09-23T03:16:22.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.163+0000 7fa575990640 1 --2- 172.21.15.154:0/3769242961 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa55006af30 0x7fa55006d3f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:22.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.163+0000 7fa575990640 1 --2- 172.21.15.154:0/3769242961 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5700cf130 0x7fa570153680 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:22.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.163+0000 7fa575990640 1 --2- 172.21.15.154:0/3769242961 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa5700b2fe0 0x7fa570153140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:22.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.163+0000 7fa575990640 1 --2- 172.21.15.154:0/3769242961 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa5700ada30 0x7fa5700977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:22.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.163+0000 7fa575990640 1 -- 172.21.15.154:0/3769242961 >> 172.21.15.154:0/3769242961 conn(0x7fa57006a0f0 msgr2=0x7fa5700b1670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:22.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.163+0000 7fa575990640 1 -- 172.21.15.154:0/3769242961 shutdown_connections 2023-09-23T03:16:22.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.163+0000 7fa575990640 1 -- 172.21.15.154:0/3769242961 wait complete. 2023-09-23T03:16:22.183 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:15:18.575849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:15:19.707212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-23T03:15:23.337138+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-23T03:16:22.183 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:16:22.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.401+0000 7fc964b1a640 1 Processor -- start 2023-09-23T03:16:22.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.401+0000 7fc964b1a640 1 -- start start 2023-09-23T03:16:22.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.403+0000 7fc964b1a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc96015caa0 0x7fc9600cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:22.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.403+0000 7fc964b1a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9600cc3b0 0x7fc960070720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:22.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.403+0000 7fc964b1a640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc960070290 con 0x7fc96015c6c0 2023-09-23T03:16:22.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.403+0000 7fc964b1a640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9600cc860 con 0x7fc96015caa0 2023-09-23T03:16:22.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.403+0000 7fc964b1a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc96006ff40 con 0x7fc9600cc3b0 2023-09-23T03:16:22.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.403+0000 7fc95f7fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fc96015c6c0 0x7fc960160410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:47410/0 (socket says 172.21.15.154:47410) 2023-09-23T03:16:22.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.403+0000 7fc95f7fe640 1 -- 172.21.15.154:0/1324019704 learned_addr learned my addr 172.21.15.154:0/1324019704 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:22.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.404+0000 7fc95effd640 1 --2- 172.21.15.154:0/1324019704 >> v2:172.21.15.190:3300/0 conn(0x7fc96015caa0 0x7fc9600cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:22.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.404+0000 7fc95ffff640 1 --2- 172.21.15.154:0/1324019704 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9600cc3b0 0x7fc960070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:22.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.404+0000 7fc95e7fc640 1 -- 172.21.15.154:0/1324019704 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1389091545 0 0) 0x7fc960070290 con 0x7fc96015c6c0 2023-09-23T03:16:22.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.404+0000 7fc95e7fc640 1 -- 172.21.15.154:0/1324019704 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc948003520 con 0x7fc96015c6c0 2023-09-23T03:16:22.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.404+0000 7fc95e7fc640 1 -- 172.21.15.154:0/1324019704 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1277047070 0 0) 0x7fc948003520 con 0x7fc96015c6c0 2023-09-23T03:16:22.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.404+0000 7fc95e7fc640 1 -- 172.21.15.154:0/1324019704 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9600cc3b0 msgr2=0x7fc960070720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:22.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.404+0000 7fc95e7fc640 1 --2- 172.21.15.154:0/1324019704 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9600cc3b0 0x7fc960070720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:22.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.404+0000 7fc95e7fc640 1 -- 172.21.15.154:0/1324019704 >> v2:172.21.15.190:3300/0 conn(0x7fc96015caa0 msgr2=0x7fc9600cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:22.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.404+0000 7fc95e7fc640 1 --2- 172.21.15.154:0/1324019704 >> v2:172.21.15.190:3300/0 conn(0x7fc96015caa0 0x7fc9600cbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:22.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.404+0000 7fc95e7fc640 1 -- 172.21.15.154:0/1324019704 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc960160b20 con 0x7fc96015c6c0 2023-09-23T03:16:22.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.404+0000 7fc95e7fc640 1 -- 172.21.15.154:0/1324019704 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fc950001670 con 0x7fc96015c6c0 2023-09-23T03:16:22.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.404+0000 7fc95e7fc640 1 -- 172.21.15.154:0/1324019704 >> v1:172.21.15.154:6789/0 conn(0x7fc96015c6c0 legacy=0x7fc960160410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:22.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.405+0000 7fc95e7fc640 1 --2- 172.21.15.154:0/1324019704 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9600cc3b0 0x7fc948003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:22.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.405+0000 7fc95e7fc640 1 --2- 172.21.15.154:0/1324019704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc948003ef0 0x7fc9480047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:22.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.405+0000 7fc95e7fc640 1 --2- 172.21.15.154:0/1324019704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc948004d00 0x7fc948005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:22.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.405+0000 7fc95e7fc640 1 -- 172.21.15.154:0/1324019704 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9600cc860 con 0x7fc948004d00 2023-09-23T03:16:22.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.405+0000 7fc95e7fc640 1 -- 172.21.15.154:0/1324019704 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96006ff40 con 0x7fc948003ef0 2023-09-23T03:16:22.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.405+0000 7fc95e7fc640 1 -- 172.21.15.154:0/1324019704 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc94800b980 con 0x7fc9600cc3b0 2023-09-23T03:16:22.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.405+0000 7fc95ffff640 1 --2- 172.21.15.154:0/1324019704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc948004d00 0x7fc948005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:22.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.405+0000 7fc95f7fe640 1 --2- 172.21.15.154:0/1324019704 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9600cc3b0 0x7fc948003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:22.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.405+0000 7fc95effd640 1 --2- 172.21.15.154:0/1324019704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc948003ef0 0x7fc9480047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:22.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.405+0000 7fc95ffff640 1 -- 172.21.15.154:0/1324019704 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9600cc3b0 msgr2=0x7fc948003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:22.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.405+0000 7fc95ffff640 1 --2- 172.21.15.154:0/1324019704 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9600cc3b0 0x7fc948003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:22.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.405+0000 7fc95ffff640 1 -- 172.21.15.154:0/1324019704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc948003ef0 msgr2=0x7fc9480047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:22.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.405+0000 7fc95ffff640 1 --2- 172.21.15.154:0/1324019704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc948003ef0 0x7fc9480047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:22.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.405+0000 7fc95ffff640 1 -- 172.21.15.154:0/1324019704 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc948003520 con 0x7fc948004d00 2023-09-23T03:16:22.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.406+0000 7fc95ffff640 1 --2- 172.21.15.154:0/1324019704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc948004d00 0x7fc948005180 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc958009fc0 tx=0x7fc9480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72b960be0618eed8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:22.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.407+0000 7fc95e7fc640 1 -- 172.21.15.154:0/1324019704 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95802cc00 con 0x7fc948004d00 2023-09-23T03:16:22.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.407+0000 7fc95e7fc640 1 -- 172.21.15.154:0/1324019704 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc95800f040 con 0x7fc948004d00 2023-09-23T03:16:22.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.407+0000 7fc95e7fc640 1 -- 172.21.15.154:0/1324019704 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc958034440 con 0x7fc948004d00 2023-09-23T03:16:22.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.407+0000 7fc964b1a640 1 -- 172.21.15.154:0/1324019704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc948004d00 msgr2=0x7fc948005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:22.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.407+0000 7fc964b1a640 1 --2- 172.21.15.154:0/1324019704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc948004d00 0x7fc948005180 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc958009fc0 tx=0x7fc9480057f0 comp rx=0 tx=0).stop 2023-09-23T03:16:22.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.407+0000 7fc95ffff640 1 -- 172.21.15.154:0/1324019704 reap_dead start 2023-09-23T03:16:22.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.407+0000 7fc964b1a640 1 -- 172.21.15.154:0/1324019704 shutdown_connections 2023-09-23T03:16:22.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.407+0000 7fc964b1a640 1 -- 172.21.15.154:0/1324019704 >> 172.21.15.154:0/1324019704 conn(0x7fc96006a300 msgr2=0x7fc9600afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:22.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.408+0000 7fc964b1a640 1 -- 172.21.15.154:0/1324019704 shutdown_connections 2023-09-23T03:16:22.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.408+0000 7fc964b1a640 1 -- 172.21.15.154:0/1324019704 wait complete. 2023-09-23T03:16:22.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.409+0000 7fc964b1a640 1 Processor -- start 2023-09-23T03:16:22.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.409+0000 7fc964b1a640 1 -- start start 2023-09-23T03:16:22.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.410+0000 7fc964b1a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc948004d00 0x7fc9601585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:22.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.410+0000 7fc964b1a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9600cac70 0x7fc9600cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:22.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.410+0000 7fc964b1a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9600c7180 0x7fc9600c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:22.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.410+0000 7fc964b1a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9600cb630 con 0x7fc948004d00 2023-09-23T03:16:22.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.410+0000 7fc964b1a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9600c9410 con 0x7fc9600c7180 2023-09-23T03:16:22.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.410+0000 7fc964b1a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9600c96b0 con 0x7fc9600cac70 2023-09-23T03:16:22.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.411+0000 7fc95f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc948004d00 0x7fc9601585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:22.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.411+0000 7fc95f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc948004d00 0x7fc9601585b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53226/0 (socket says 172.21.15.154:53226) 2023-09-23T03:16:22.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.411+0000 7fc95f7fe640 1 -- 172.21.15.154:0/3793909028 learned_addr learned my addr 172.21.15.154:0/3793909028 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:22.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.411+0000 7fc95f7fe640 1 -- 172.21.15.154:0/3793909028 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9600cac70 msgr2=0x7fc9600cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:22.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.411+0000 7fc95f7fe640 1 --2- 172.21.15.154:0/3793909028 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9600cac70 0x7fc9600cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:22.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.411+0000 7fc95f7fe640 1 -- 172.21.15.154:0/3793909028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9600c7180 msgr2=0x7fc9600c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:22.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.411+0000 7fc95f7fe640 1 --2- 172.21.15.154:0/3793909028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9600c7180 0x7fc9600c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:22.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.411+0000 7fc95f7fe640 1 -- 172.21.15.154:0/3793909028 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9600c9990 con 0x7fc948004d00 2023-09-23T03:16:22.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.411+0000 7fc95f7fe640 1 --2- 172.21.15.154:0/3793909028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc948004d00 0x7fc9601585b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc95000e8c0 tx=0x7fc95000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:22.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.411+0000 7fc947fff640 1 -- 172.21.15.154:0/3793909028 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc950005320 con 0x7fc948004d00 2023-09-23T03:16:22.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.412+0000 7fc947fff640 1 -- 172.21.15.154:0/3793909028 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9500054a0 con 0x7fc948004d00 2023-09-23T03:16:22.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.412+0000 7fc947fff640 1 -- 172.21.15.154:0/3793909028 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9500047e0 con 0x7fc948004d00 2023-09-23T03:16:22.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.412+0000 7fc964b1a640 1 -- 172.21.15.154:0/3793909028 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9600c9c70 con 0x7fc948004d00 2023-09-23T03:16:22.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.412+0000 7fc964b1a640 1 -- 172.21.15.154:0/3793909028 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc960158af0 con 0x7fc948004d00 2023-09-23T03:16:22.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.413+0000 7fc964b1a640 1 -- 172.21.15.154:0/3793909028 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9600736b0 con 0x7fc948004d00 2023-09-23T03:16:22.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.414+0000 7fc947fff640 1 -- 172.21.15.154:0/3793909028 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fc950004960 con 0x7fc948004d00 2023-09-23T03:16:22.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.415+0000 7fc947fff640 1 --2- 172.21.15.154:0/3793909028 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc94c06af30 0x7fc94c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:22.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.415+0000 7fc947fff640 1 -- 172.21.15.154:0/3793909028 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7fc9500896d0 con 0x7fc948004d00 2023-09-23T03:16:22.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.415+0000 7fc95effd640 1 --2- 172.21.15.154:0/3793909028 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc94c06af30 0x7fc94c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:22.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.416+0000 7fc95effd640 1 --2- 172.21.15.154:0/3793909028 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc94c06af30 0x7fc94c06d3f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc9600c70c0 tx=0x7fc954008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:22.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.419+0000 7fc947fff640 1 -- 172.21.15.154:0/3793909028 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fc950054350 con 0x7fc948004d00 2023-09-23T03:16:22.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.608+0000 7fc964b1a640 1 -- 172.21.15.154:0/3793909028 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc9600ca190 con 0x7fc948004d00 2023-09-23T03:16:22.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.609+0000 7fc947fff640 1 -- 172.21.15.154:0/3793909028 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc950057970 con 0x7fc948004d00 2023-09-23T03:16:22.609 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:16:22.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.611+0000 7fc964b1a640 1 -- 172.21.15.154:0/3793909028 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc94c06af30 msgr2=0x7fc94c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:22.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.611+0000 7fc964b1a640 1 --2- 172.21.15.154:0/3793909028 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc94c06af30 0x7fc94c06d3f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc9600c70c0 tx=0x7fc954008040 comp rx=0 tx=0).stop 2023-09-23T03:16:22.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.611+0000 7fc964b1a640 1 -- 172.21.15.154:0/3793909028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc948004d00 msgr2=0x7fc9601585b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:22.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.611+0000 7fc964b1a640 1 --2- 172.21.15.154:0/3793909028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc948004d00 0x7fc9601585b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc95000e8c0 tx=0x7fc95000ed80 comp rx=0 tx=0).stop 2023-09-23T03:16:22.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.611+0000 7fc964b1a640 1 -- 172.21.15.154:0/3793909028 shutdown_connections 2023-09-23T03:16:22.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.611+0000 7fc964b1a640 1 --2- 172.21.15.154:0/3793909028 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc94c06af30 0x7fc94c06d3f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:22.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.611+0000 7fc964b1a640 1 --2- 172.21.15.154:0/3793909028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9600c7180 0x7fc9600c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:22.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.611+0000 7fc964b1a640 1 --2- 172.21.15.154:0/3793909028 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9600cac70 0x7fc9600cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:22.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.611+0000 7fc964b1a640 1 --2- 172.21.15.154:0/3793909028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc948004d00 0x7fc9601585b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:22.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.611+0000 7fc964b1a640 1 -- 172.21.15.154:0/3793909028 >> 172.21.15.154:0/3793909028 conn(0x7fc96006a300 msgr2=0x7fc9600af5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:22.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.612+0000 7fc964b1a640 1 -- 172.21.15.154:0/3793909028 shutdown_connections 2023-09-23T03:16:22.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.612+0000 7fc964b1a640 1 -- 172.21.15.154:0/3793909028 wait complete. 2023-09-23T03:16:22.629 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:16:22.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.842+0000 7ff1f2c61640 1 Processor -- start 2023-09-23T03:16:22.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.842+0000 7ff1f2c61640 1 -- start start 2023-09-23T03:16:22.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.844+0000 7ff1f2c61640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff1ec15e5c0 0x7ff1ec15ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:22.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.844+0000 7ff1f2c61640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff1ec0d0390 0x7ff1ec0d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:22.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.844+0000 7ff1f2c61640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1ec15f480 con 0x7ff1ec15ab20 2023-09-23T03:16:22.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.844+0000 7ff1f2c61640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1ec15efb0 con 0x7ff1ec0d0390 2023-09-23T03:16:22.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.844+0000 7ff1f2c61640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1ec15af00 con 0x7ff1ec15e5c0 2023-09-23T03:16:22.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.844+0000 7ff1f09fd640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff1ec15ab20 0x7ff1ec0cee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:47416/0 (socket says 172.21.15.154:47416) 2023-09-23T03:16:22.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.844+0000 7ff1f09fd640 1 -- 172.21.15.154:0/433127179 learned_addr learned my addr 172.21.15.154:0/433127179 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:22.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.844+0000 7ff1f11fe640 1 --2- 172.21.15.154:0/433127179 >> v2:172.21.15.154:3300/0 conn(0x7ff1ec0d0390 0x7ff1ec0d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:22.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.844+0000 7ff1ebfff640 1 --2- 172.21.15.154:0/433127179 >> v2:172.21.15.190:3300/0 conn(0x7ff1ec15e5c0 0x7ff1ec15ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:22.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.844+0000 7ff1eb7fe640 1 -- 172.21.15.154:0/433127179 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 497054639 0 0) 0x7ff1ec15f480 con 0x7ff1ec15ab20 2023-09-23T03:16:22.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.844+0000 7ff1eb7fe640 1 -- 172.21.15.154:0/433127179 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1dc003520 con 0x7ff1ec15ab20 2023-09-23T03:16:22.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.845+0000 7ff1f11fe640 1 -- 172.21.15.154:0/433127179 >> v2:172.21.15.190:3300/0 conn(0x7ff1ec15e5c0 msgr2=0x7ff1ec15ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:22.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.845+0000 7ff1f11fe640 1 --2- 172.21.15.154:0/433127179 >> v2:172.21.15.190:3300/0 conn(0x7ff1ec15e5c0 0x7ff1ec15ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:22.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.845+0000 7ff1f11fe640 1 -- 172.21.15.154:0/433127179 >> v1:172.21.15.154:6789/0 conn(0x7ff1ec15ab20 legacy=0x7ff1ec0cee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:22.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.845+0000 7ff1f11fe640 1 -- 172.21.15.154:0/433127179 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ec0cd3f0 con 0x7ff1ec0d0390 2023-09-23T03:16:22.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.846+0000 7ff1f11fe640 1 --2- 172.21.15.154:0/433127179 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ec0d0390 0x7ff1ec0d07c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0ce220 tx=0x7ff1e402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=287d5cf4dacea901 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:22.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.846+0000 7ff1eb7fe640 1 -- 172.21.15.154:0/433127179 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e402cc70 con 0x7ff1ec0d0390 2023-09-23T03:16:22.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.846+0000 7ff1eb7fe640 1 -- 172.21.15.154:0/433127179 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e402cdd0 con 0x7ff1ec0d0390 2023-09-23T03:16:22.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.846+0000 7ff1eb7fe640 1 -- 172.21.15.154:0/433127179 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e4034660 con 0x7ff1ec0d0390 2023-09-23T03:16:22.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.847+0000 7ff1f2c61640 1 -- 172.21.15.154:0/433127179 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ec0d0390 msgr2=0x7ff1ec0d07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:22.848 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.847+0000 7ff1f2c61640 1 --2- 172.21.15.154:0/433127179 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ec0d0390 0x7ff1ec0d07c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0ce220 tx=0x7ff1e402bec0 comp rx=0 tx=0).stop 2023-09-23T03:16:22.848 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.847+0000 7ff1f2c61640 1 -- 172.21.15.154:0/433127179 shutdown_connections 2023-09-23T03:16:22.848 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.847+0000 7ff1f2c61640 1 --2- 172.21.15.154:0/433127179 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ec0d0390 0x7ff1ec0d07c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:22.848 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.847+0000 7ff1f2c61640 1 --2- 172.21.15.154:0/433127179 >> v2:172.21.15.190:3300/0 conn(0x7ff1ec15e5c0 0x7ff1ec15ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:22.848 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.847+0000 7ff1f2c61640 1 -- 172.21.15.154:0/433127179 >> 172.21.15.154:0/433127179 conn(0x7ff1ec089fd0 msgr2=0x7ff1ec08a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:22.848 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.847+0000 7ff1f2c61640 1 -- 172.21.15.154:0/433127179 shutdown_connections 2023-09-23T03:16:22.848 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.847+0000 7ff1f2c61640 1 -- 172.21.15.154:0/433127179 wait complete. 2023-09-23T03:16:22.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.849+0000 7ff1f2c61640 1 Processor -- start 2023-09-23T03:16:22.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.849+0000 7ff1f2c61640 1 -- start start 2023-09-23T03:16:22.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.850+0000 7ff1f2c61640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1ec0d0390 0x7ff1ec0c4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:22.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.850+0000 7ff1f2c61640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ec15ab20 0x7ff1ec15db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:22.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.850+0000 7ff1f2c61640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1ec15e5c0 0x7ff1ec0b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:22.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.850+0000 7ff1f2c61640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec0c5f30 con 0x7ff1ec15ab20 2023-09-23T03:16:22.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.850+0000 7ff1f2c61640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec0c52b0 con 0x7ff1ec0d0390 2023-09-23T03:16:22.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.850+0000 7ff1f2c61640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec0c5590 con 0x7ff1ec15e5c0 2023-09-23T03:16:22.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.850+0000 7ff1ebfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ec15ab20 0x7ff1ec15db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:22.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.850+0000 7ff1ebfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ec15ab20 0x7ff1ec15db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53248/0 (socket says 172.21.15.154:53248) 2023-09-23T03:16:22.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.850+0000 7ff1ebfff640 1 -- 172.21.15.154:0/1802324898 learned_addr learned my addr 172.21.15.154:0/1802324898 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:22.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.851+0000 7ff1ebfff640 1 -- 172.21.15.154:0/1802324898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1ec15e5c0 msgr2=0x7ff1ec0b5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:22.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.851+0000 7ff1ebfff640 1 --2- 172.21.15.154:0/1802324898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1ec15e5c0 0x7ff1ec0b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:22.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.851+0000 7ff1ebfff640 1 -- 172.21.15.154:0/1802324898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1ec0d0390 msgr2=0x7ff1ec0c4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:22.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.851+0000 7ff1ebfff640 1 --2- 172.21.15.154:0/1802324898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1ec0d0390 0x7ff1ec0c4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:22.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.851+0000 7ff1ebfff640 1 -- 172.21.15.154:0/1802324898 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e4009a70 con 0x7ff1ec15ab20 2023-09-23T03:16:22.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.851+0000 7ff1ebfff640 1 --2- 172.21.15.154:0/1802324898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ec15ab20 0x7ff1ec15db70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff1d800d5f0 tx=0x7ff1d800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:22.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.851+0000 7ff1e97fa640 1 -- 172.21.15.154:0/1802324898 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d80125f0 con 0x7ff1ec15ab20 2023-09-23T03:16:22.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.851+0000 7ff1e97fa640 1 -- 172.21.15.154:0/1802324898 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d8012750 con 0x7ff1ec15ab20 2023-09-23T03:16:22.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.851+0000 7ff1e97fa640 1 -- 172.21.15.154:0/1802324898 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d8012b00 con 0x7ff1ec15ab20 2023-09-23T03:16:22.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.851+0000 7ff1f2c61640 1 -- 172.21.15.154:0/1802324898 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1ec0c5870 con 0x7ff1ec15ab20 2023-09-23T03:16:22.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.851+0000 7ff1f2c61640 1 -- 172.21.15.154:0/1802324898 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1ec0b5fe0 con 0x7ff1ec15ab20 2023-09-23T03:16:22.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.853+0000 7ff1f2c61640 1 -- 172.21.15.154:0/1802324898 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1ec04e920 con 0x7ff1ec15ab20 2023-09-23T03:16:22.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.854+0000 7ff1e97fa640 1 -- 172.21.15.154:0/1802324898 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7ff1d8012c60 con 0x7ff1ec15ab20 2023-09-23T03:16:22.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.855+0000 7ff1e97fa640 1 --2- 172.21.15.154:0/1802324898 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff1d006af30 0x7ff1d006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:22.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.855+0000 7ff1e97fa640 1 -- 172.21.15.154:0/1802324898 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7ff1d808a000 con 0x7ff1ec15ab20 2023-09-23T03:16:22.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.855+0000 7ff1f09fd640 1 --2- 172.21.15.154:0/1802324898 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff1d006af30 0x7ff1d006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:22.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.856+0000 7ff1f09fd640 1 --2- 172.21.15.154:0/1802324898 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff1d006af30 0x7ff1d006d3f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0001360 tx=0x7ff1e00077c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:22.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:22.859+0000 7ff1e97fa640 1 -- 172.21.15.154:0/1802324898 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7ff1d8054c90 con 0x7ff1ec15ab20 2023-09-23T03:16:23.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.047+0000 7ff1f2c61640 1 -- 172.21.15.154:0/1802324898 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff1ec0b0320 con 0x7ff1ec15ab20 2023-09-23T03:16:23.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.048+0000 7ff1e97fa640 1 -- 172.21.15.154:0/1802324898 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff1d80582b0 con 0x7ff1ec15ab20 2023-09-23T03:16:23.048 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:16:23.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.051+0000 7ff1f2c61640 1 -- 172.21.15.154:0/1802324898 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff1d006af30 msgr2=0x7ff1d006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:23.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.051+0000 7ff1f2c61640 1 --2- 172.21.15.154:0/1802324898 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff1d006af30 0x7ff1d006d3f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0001360 tx=0x7ff1e00077c0 comp rx=0 tx=0).stop 2023-09-23T03:16:23.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.051+0000 7ff1f2c61640 1 -- 172.21.15.154:0/1802324898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ec15ab20 msgr2=0x7ff1ec15db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:23.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.051+0000 7ff1f2c61640 1 --2- 172.21.15.154:0/1802324898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ec15ab20 0x7ff1ec15db70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff1d800d5f0 tx=0x7ff1d800dab0 comp rx=0 tx=0).stop 2023-09-23T03:16:23.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.051+0000 7ff1f2c61640 1 -- 172.21.15.154:0/1802324898 shutdown_connections 2023-09-23T03:16:23.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.051+0000 7ff1f2c61640 1 --2- 172.21.15.154:0/1802324898 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff1d006af30 0x7ff1d006d3f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:23.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.051+0000 7ff1f2c61640 1 --2- 172.21.15.154:0/1802324898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1ec15e5c0 0x7ff1ec0b5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:23.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.051+0000 7ff1f2c61640 1 --2- 172.21.15.154:0/1802324898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ec15ab20 0x7ff1ec15db70 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:23.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.051+0000 7ff1f2c61640 1 --2- 172.21.15.154:0/1802324898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1ec0d0390 0x7ff1ec0c4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:23.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.051+0000 7ff1f2c61640 1 -- 172.21.15.154:0/1802324898 >> 172.21.15.154:0/1802324898 conn(0x7ff1ec089fd0 msgr2=0x7ff1ec0b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:23.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.051+0000 7ff1f2c61640 1 -- 172.21.15.154:0/1802324898 shutdown_connections 2023-09-23T03:16:23.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.051+0000 7ff1f2c61640 1 -- 172.21.15.154:0/1802324898 wait complete. 2023-09-23T03:16:23.069 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:16:23.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.286+0000 7f586691d640 1 Processor -- start 2023-09-23T03:16:23.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.286+0000 7f586691d640 1 -- start start 2023-09-23T03:16:23.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.288+0000 7f586691d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f58600c9150 0x7f586015cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:23.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.288+0000 7f586691d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5860071290 0x7f58600ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:23.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.288+0000 7f586691d640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58600c9d20 con 0x7f58600c8d70 2023-09-23T03:16:23.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.288+0000 7f586691d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5860071850 con 0x7f5860071290 2023-09-23T03:16:23.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.288+0000 7f586691d640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5860071b30 con 0x7f58600c9150 2023-09-23T03:16:23.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.288+0000 7f5864eba640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5860071290 0x7f58600ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:23.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.289+0000 7f5864eba640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5860071290 0x7f58600ca1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53262/0 (socket says 172.21.15.154:53262) 2023-09-23T03:16:23.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.289+0000 7f5864eba640 1 -- 172.21.15.154:0/1529629048 learned_addr learned my addr 172.21.15.154:0/1529629048 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:23.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.289+0000 7f5857fff640 1 --2- 172.21.15.154:0/1529629048 >> v2:172.21.15.190:3300/0 conn(0x7f58600c9150 0x7f586015cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:23.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.289+0000 7f5864eba640 1 -- 172.21.15.154:0/1529629048 >> v2:172.21.15.190:3300/0 conn(0x7f58600c9150 msgr2=0x7f586015cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:23.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.289+0000 7f5864eba640 1 --2- 172.21.15.154:0/1529629048 >> v2:172.21.15.190:3300/0 conn(0x7f58600c9150 0x7f586015cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:23.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.289+0000 7f5864eba640 1 -- 172.21.15.154:0/1529629048 >> v1:172.21.15.190:6789/0 conn(0x7f58600c8d70 legacy=0x7f586015c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:23.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.289+0000 7f5864eba640 1 -- 172.21.15.154:0/1529629048 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5860071e10 con 0x7f5860071290 2023-09-23T03:16:23.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.290+0000 7f5864eba640 1 --2- 172.21.15.154:0/1529629048 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5860071290 0x7f58600ca1b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f58600b3ce0 tx=0x7f585802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d86950487289bcee server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:23.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.290+0000 7f585f7fe640 1 -- 172.21.15.154:0/1529629048 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585802ca80 con 0x7f5860071290 2023-09-23T03:16:23.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.290+0000 7f585f7fe640 1 -- 172.21.15.154:0/1529629048 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f585802cbe0 con 0x7f5860071290 2023-09-23T03:16:23.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.290+0000 7f585f7fe640 1 -- 172.21.15.154:0/1529629048 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58580343f0 con 0x7f5860071290 2023-09-23T03:16:23.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.291+0000 7f586691d640 1 -- 172.21.15.154:0/1529629048 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5860071290 msgr2=0x7f58600ca1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:23.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.291+0000 7f586691d640 1 --2- 172.21.15.154:0/1529629048 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5860071290 0x7f58600ca1b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f58600b3ce0 tx=0x7f585802bee0 comp rx=0 tx=0).stop 2023-09-23T03:16:23.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.291+0000 7f586691d640 1 -- 172.21.15.154:0/1529629048 shutdown_connections 2023-09-23T03:16:23.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.291+0000 7f586691d640 1 --2- 172.21.15.154:0/1529629048 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5860071290 0x7f58600ca1b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:23.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.291+0000 7f586691d640 1 --2- 172.21.15.154:0/1529629048 >> v2:172.21.15.190:3300/0 conn(0x7f58600c9150 0x7f586015cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:23.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.291+0000 7f586691d640 1 -- 172.21.15.154:0/1529629048 >> 172.21.15.154:0/1529629048 conn(0x7f586006c010 msgr2=0x7f586006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:23.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.291+0000 7f586691d640 1 -- 172.21.15.154:0/1529629048 shutdown_connections 2023-09-23T03:16:23.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.291+0000 7f586691d640 1 -- 172.21.15.154:0/1529629048 wait complete. 2023-09-23T03:16:23.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.294+0000 7f586691d640 1 Processor -- start 2023-09-23T03:16:23.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.294+0000 7f586691d640 1 -- start start 2023-09-23T03:16:23.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.295+0000 7f586691d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5860071290 0x7f586014c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:23.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.295+0000 7f586691d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58600c8d70 0x7f586014b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:23.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.295+0000 7f586691d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58600c9150 0x7f5860167420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:23.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.295+0000 7f586691d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58600c0110 con 0x7f58600c8d70 2023-09-23T03:16:23.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.295+0000 7f586691d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586014b7e0 con 0x7f5860071290 2023-09-23T03:16:23.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.295+0000 7f586691d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58600c6530 con 0x7f58600c9150 2023-09-23T03:16:23.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.296+0000 7f5857fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58600c8d70 0x7f586014b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:23.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.296+0000 7f5857fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58600c8d70 0x7f586014b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53274/0 (socket says 172.21.15.154:53274) 2023-09-23T03:16:23.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.296+0000 7f5857fff640 1 -- 172.21.15.154:0/2947541420 learned_addr learned my addr 172.21.15.154:0/2947541420 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:23.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.296+0000 7f585ffff640 1 --2- 172.21.15.154:0/2947541420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5860071290 0x7f586014c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:23.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.296+0000 7f5857fff640 1 -- 172.21.15.154:0/2947541420 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58600c9150 msgr2=0x7f5860167420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:23.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.296+0000 7f5857fff640 1 --2- 172.21.15.154:0/2947541420 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58600c9150 0x7f5860167420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:23.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.296+0000 7f5857fff640 1 -- 172.21.15.154:0/2947541420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5860071290 msgr2=0x7f586014c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:23.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.296+0000 7f5857fff640 1 --2- 172.21.15.154:0/2947541420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5860071290 0x7f586014c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:23.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.296+0000 7f5857fff640 1 -- 172.21.15.154:0/2947541420 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5858009b30 con 0x7f58600c8d70 2023-09-23T03:16:23.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.297+0000 7f5857fff640 1 --2- 172.21.15.154:0/2947541420 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58600c8d70 0x7f586014b2a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f584800d5f0 tx=0x7f584800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:23.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.297+0000 7f585d7fa640 1 -- 172.21.15.154:0/2947541420 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58480125f0 con 0x7f58600c8d70 2023-09-23T03:16:23.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.297+0000 7f585d7fa640 1 -- 172.21.15.154:0/2947541420 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5848012750 con 0x7f58600c8d70 2023-09-23T03:16:23.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.297+0000 7f585d7fa640 1 -- 172.21.15.154:0/2947541420 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5848012b00 con 0x7f58600c8d70 2023-09-23T03:16:23.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.297+0000 7f586691d640 1 -- 172.21.15.154:0/2947541420 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58600c5920 con 0x7f58600c8d70 2023-09-23T03:16:23.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.297+0000 7f586691d640 1 -- 172.21.15.154:0/2947541420 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58600c5ee0 con 0x7f58600c8d70 2023-09-23T03:16:23.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.299+0000 7f586691d640 1 -- 172.21.15.154:0/2947541420 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f586004e920 con 0x7f58600c8d70 2023-09-23T03:16:23.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.299+0000 7f585d7fa640 1 -- 172.21.15.154:0/2947541420 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f5848012c60 con 0x7f58600c8d70 2023-09-23T03:16:23.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.300+0000 7f585d7fa640 1 --2- 172.21.15.154:0/2947541420 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f584006af30 0x7f584006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:23.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.300+0000 7f585d7fa640 1 -- 172.21.15.154:0/2947541420 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f5848089fa0 con 0x7f58600c8d70 2023-09-23T03:16:23.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.301+0000 7f585ffff640 1 --2- 172.21.15.154:0/2947541420 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f584006af30 0x7f584006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:23.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.301+0000 7f585ffff640 1 --2- 172.21.15.154:0/2947541420 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f584006af30 0x7f584006d3f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5850004ea0 tx=0x7f5850009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:23.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.305+0000 7f585d7fa640 1 -- 172.21.15.154:0/2947541420 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f58480bca50 con 0x7f58600c8d70 2023-09-23T03:16:23.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.494+0000 7f586691d640 1 -- 172.21.15.154:0/2947541420 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f58600c6670 con 0x7f58600c8d70 2023-09-23T03:16:23.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.495+0000 7f585d7fa640 1 -- 172.21.15.154:0/2947541420 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5848054cc0 con 0x7f58600c8d70 2023-09-23T03:16:23.495 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:16:23.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.498+0000 7f586691d640 1 -- 172.21.15.154:0/2947541420 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f584006af30 msgr2=0x7f584006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:23.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.498+0000 7f586691d640 1 --2- 172.21.15.154:0/2947541420 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f584006af30 0x7f584006d3f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5850004ea0 tx=0x7f5850009040 comp rx=0 tx=0).stop 2023-09-23T03:16:23.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.498+0000 7f586691d640 1 -- 172.21.15.154:0/2947541420 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58600c8d70 msgr2=0x7f586014b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:23.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.498+0000 7f586691d640 1 --2- 172.21.15.154:0/2947541420 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58600c8d70 0x7f586014b2a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f584800d5f0 tx=0x7f584800dab0 comp rx=0 tx=0).stop 2023-09-23T03:16:23.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.498+0000 7f586691d640 1 -- 172.21.15.154:0/2947541420 shutdown_connections 2023-09-23T03:16:23.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.498+0000 7f586691d640 1 --2- 172.21.15.154:0/2947541420 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f584006af30 0x7f584006d3f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:23.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.498+0000 7f586691d640 1 --2- 172.21.15.154:0/2947541420 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58600c9150 0x7f5860167420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:23.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.498+0000 7f586691d640 1 --2- 172.21.15.154:0/2947541420 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58600c8d70 0x7f586014b2a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:23.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.498+0000 7f586691d640 1 --2- 172.21.15.154:0/2947541420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5860071290 0x7f586014c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:23.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.498+0000 7f586691d640 1 -- 172.21.15.154:0/2947541420 >> 172.21.15.154:0/2947541420 conn(0x7f586006c010 msgr2=0x7f586015af00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:23.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.498+0000 7f586691d640 1 -- 172.21.15.154:0/2947541420 shutdown_connections 2023-09-23T03:16:23.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.498+0000 7f586691d640 1 -- 172.21.15.154:0/2947541420 wait complete. 2023-09-23T03:16:23.516 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-23T03:16:23.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.736+0000 7f7d1f2fc640 1 Processor -- start 2023-09-23T03:16:23.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.736+0000 7f7d1f2fc640 1 -- start start 2023-09-23T03:16:23.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.737+0000 7f7d1f2fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d18159570 0x7f7d18071100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:23.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.738+0000 7f7d1f2fc640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d18071e70 con 0x7f7d1815a060 2023-09-23T03:16:23.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.738+0000 7f7d1f2fc640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d180c8b50 con 0x7f7d18071640 2023-09-23T03:16:23.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.738+0000 7f7d1f2fc640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d18072150 con 0x7f7d18159570 2023-09-23T03:16:23.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.738+0000 7f7d1eafb640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f7d1815a060 0x7f7d1815a4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:47418/0 (socket says 172.21.15.154:47418) 2023-09-23T03:16:23.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.738+0000 7f7d1eafb640 1 -- 172.21.15.154:0/423846359 learned_addr learned my addr 172.21.15.154:0/423846359 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:23.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.738+0000 7f7d1e2fa640 1 --2- 172.21.15.154:0/423846359 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d18159570 0x7f7d18071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:23.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.738+0000 7f7d1d2f8640 1 -- 172.21.15.154:0/423846359 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1806923409 0 0) 0x7f7d18071e70 con 0x7f7d1815a060 2023-09-23T03:16:23.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.739+0000 7f7d1d2f8640 1 -- 172.21.15.154:0/423846359 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d04003520 con 0x7f7d1815a060 2023-09-23T03:16:23.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.739+0000 7f7d1d2f8640 1 -- 172.21.15.154:0/423846359 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1553146162 0 0) 0x7f7d04003520 con 0x7f7d1815a060 2023-09-23T03:16:23.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.739+0000 7f7d1d2f8640 1 -- 172.21.15.154:0/423846359 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d18159570 msgr2=0x7f7d18071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:23.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.739+0000 7f7d1d2f8640 1 --2- 172.21.15.154:0/423846359 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d18159570 0x7f7d18071100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:23.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.739+0000 7f7d1d2f8640 1 -- 172.21.15.154:0/423846359 >> v1:172.21.15.190:6789/0 conn(0x7f7d18071640 legacy=0x7f7d18159950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:23.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.739+0000 7f7d1d2f8640 1 -- 172.21.15.154:0/423846359 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d180c8e30 con 0x7f7d1815a060 2023-09-23T03:16:23.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.739+0000 7f7d1d2f8640 1 -- 172.21.15.154:0/423846359 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f7d100019f0 con 0x7f7d1815a060 2023-09-23T03:16:23.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.739+0000 7f7d1d2f8640 1 -- 172.21.15.154:0/423846359 >> v1:172.21.15.154:6789/0 conn(0x7f7d1815a060 legacy=0x7f7d1815a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:23.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.739+0000 7f7d1d2f8640 1 --2- 172.21.15.154:0/423846359 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d18159570 0x7f7d040039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:23.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.739+0000 7f7d1d2f8640 1 --2- 172.21.15.154:0/423846359 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d04003f70 0x7f7d040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:23.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.739+0000 7f7d1d2f8640 1 --2- 172.21.15.154:0/423846359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d04004d00 0x7f7d04005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:23.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.739+0000 7f7d1d2f8640 1 -- 172.21.15.154:0/423846359 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d18072150 con 0x7f7d04003f70 2023-09-23T03:16:23.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.739+0000 7f7d1d2f8640 1 -- 172.21.15.154:0/423846359 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0400dd80 con 0x7f7d04004d00 2023-09-23T03:16:23.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.739+0000 7f7d1d2f8640 1 -- 172.21.15.154:0/423846359 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0400df40 con 0x7f7d18159570 2023-09-23T03:16:23.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.739+0000 7f7d1daf9640 1 --2- 172.21.15.154:0/423846359 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d04003f70 0x7f7d040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:23.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.740+0000 7f7d1e2fa640 1 --2- 172.21.15.154:0/423846359 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d18159570 0x7f7d040039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:23.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.740+0000 7f7d1eafb640 1 --2- 172.21.15.154:0/423846359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d04004d00 0x7f7d04005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:23.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.740+0000 7f7d1daf9640 1 -- 172.21.15.154:0/423846359 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d18159570 msgr2=0x7f7d040039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:23.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.740+0000 7f7d1daf9640 1 --2- 172.21.15.154:0/423846359 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d18159570 0x7f7d040039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:23.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.740+0000 7f7d1daf9640 1 -- 172.21.15.154:0/423846359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d04004d00 msgr2=0x7f7d04005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:23.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.740+0000 7f7d1daf9640 1 --2- 172.21.15.154:0/423846359 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d04004d00 0x7f7d04005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:23.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.740+0000 7f7d1daf9640 1 -- 172.21.15.154:0/423846359 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d04003520 con 0x7f7d04003f70 2023-09-23T03:16:23.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.741+0000 7f7d1daf9640 1 --2- 172.21.15.154:0/423846359 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d04003f70 0x7f7d040047c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7d08001750 tx=0x7f7d040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf5dd722f660725 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:23.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.741+0000 7f7d1d2f8640 1 -- 172.21.15.154:0/423846359 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d080052d0 con 0x7f7d04003f70 2023-09-23T03:16:23.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.741+0000 7f7d1d2f8640 1 -- 172.21.15.154:0/423846359 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0802e050 con 0x7f7d04003f70 2023-09-23T03:16:23.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.741+0000 7f7d1d2f8640 1 -- 172.21.15.154:0/423846359 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d08003830 con 0x7f7d04003f70 2023-09-23T03:16:23.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.742+0000 7f7d1f2fc640 1 -- 172.21.15.154:0/423846359 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d04003f70 msgr2=0x7f7d040047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:23.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.742+0000 7f7d1f2fc640 1 --2- 172.21.15.154:0/423846359 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d04003f70 0x7f7d040047c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7d08001750 tx=0x7f7d040057f0 comp rx=0 tx=0).stop 2023-09-23T03:16:23.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.742+0000 7f7d1eafb640 1 -- 172.21.15.154:0/423846359 reap_dead start 2023-09-23T03:16:23.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.742+0000 7f7d1f2fc640 1 -- 172.21.15.154:0/423846359 shutdown_connections 2023-09-23T03:16:23.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.742+0000 7f7d1f2fc640 1 -- 172.21.15.154:0/423846359 >> 172.21.15.154:0/423846359 conn(0x7f7d180aff20 msgr2=0x7f7d180b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:23.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.742+0000 7f7d1f2fc640 1 -- 172.21.15.154:0/423846359 shutdown_connections 2023-09-23T03:16:23.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.742+0000 7f7d1f2fc640 1 -- 172.21.15.154:0/423846359 wait complete. 2023-09-23T03:16:23.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.744+0000 7f7d1f2fc640 1 Processor -- start 2023-09-23T03:16:23.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.744+0000 7f7d1f2fc640 1 -- start start 2023-09-23T03:16:23.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.745+0000 7f7d1f2fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d04003f70 0x7f7d18152200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:23.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.745+0000 7f7d1f2fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d180cd6c0 0x7f7d180ce790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:23.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.745+0000 7f7d1f2fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d180cecd0 0x7f7d180c6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:23.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.745+0000 7f7d1f2fc640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d180cd2a0 con 0x7f7d180cd6c0 2023-09-23T03:16:23.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.745+0000 7f7d1f2fc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d180cf180 con 0x7f7d04003f70 2023-09-23T03:16:23.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.745+0000 7f7d1f2fc640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d18152740 con 0x7f7d180cecd0 2023-09-23T03:16:23.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.745+0000 7f7d1daf9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d180cd6c0 0x7f7d180ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:23.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.745+0000 7f7d1daf9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d180cd6c0 0x7f7d180ce790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53286/0 (socket says 172.21.15.154:53286) 2023-09-23T03:16:23.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.745+0000 7f7d1daf9640 1 -- 172.21.15.154:0/3979155821 learned_addr learned my addr 172.21.15.154:0/3979155821 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:23.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.745+0000 7f7d1e2fa640 1 --2- 172.21.15.154:0/3979155821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d04003f70 0x7f7d18152200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:23.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.745+0000 7f7d1eafb640 1 --2- 172.21.15.154:0/3979155821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d180cecd0 0x7f7d180c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:23.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.745+0000 7f7d1daf9640 1 -- 172.21.15.154:0/3979155821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d180cecd0 msgr2=0x7f7d180c6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:23.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.745+0000 7f7d1daf9640 1 --2- 172.21.15.154:0/3979155821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d180cecd0 0x7f7d180c6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:23.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.745+0000 7f7d1daf9640 1 -- 172.21.15.154:0/3979155821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d04003f70 msgr2=0x7f7d18152200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:23.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.745+0000 7f7d1daf9640 1 --2- 172.21.15.154:0/3979155821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d04003f70 0x7f7d18152200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:23.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.745+0000 7f7d1daf9640 1 -- 172.21.15.154:0/3979155821 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d180cc620 con 0x7f7d180cd6c0 2023-09-23T03:16:23.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.746+0000 7f7d1daf9640 1 --2- 172.21.15.154:0/3979155821 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d180cd6c0 0x7f7d180ce790 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7d08008fd0 tx=0x7f7d08001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:23.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.746+0000 7f7d0effd640 1 -- 172.21.15.154:0/3979155821 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d080076c0 con 0x7f7d180cd6c0 2023-09-23T03:16:23.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.746+0000 7f7d0effd640 1 -- 172.21.15.154:0/3979155821 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d08007a30 con 0x7f7d180cd6c0 2023-09-23T03:16:23.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.746+0000 7f7d1f2fc640 1 -- 172.21.15.154:0/3979155821 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d180cc900 con 0x7f7d180cd6c0 2023-09-23T03:16:23.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.746+0000 7f7d0effd640 1 -- 172.21.15.154:0/3979155821 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d08006c00 con 0x7f7d180cd6c0 2023-09-23T03:16:23.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.746+0000 7f7d1f2fc640 1 -- 172.21.15.154:0/3979155821 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d180ccec0 con 0x7f7d180cd6c0 2023-09-23T03:16:23.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.748+0000 7f7d1f2fc640 1 -- 172.21.15.154:0/3979155821 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d1804e920 con 0x7f7d180cd6c0 2023-09-23T03:16:23.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.748+0000 7f7d0effd640 1 -- 172.21.15.154:0/3979155821 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f7d080035f0 con 0x7f7d180cd6c0 2023-09-23T03:16:23.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.749+0000 7f7d0effd640 1 --2- 172.21.15.154:0/3979155821 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7cfc06af60 0x7f7cfc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:23.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.749+0000 7f7d0effd640 1 -- 172.21.15.154:0/3979155821 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f7d080abb40 con 0x7f7d180cd6c0 2023-09-23T03:16:23.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.750+0000 7f7d1e2fa640 1 --2- 172.21.15.154:0/3979155821 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7cfc06af60 0x7f7cfc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:23.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.750+0000 7f7d1e2fa640 1 --2- 172.21.15.154:0/3979155821 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7cfc06af60 0x7f7cfc06d420 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7d180acc20 tx=0x7f7d14008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:23.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.754+0000 7f7d0effd640 1 -- 172.21.15.154:0/3979155821 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f7d080768e0 con 0x7f7d180cd6c0 2023-09-23T03:16:23.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.943+0000 7f7d1f2fc640 1 -- 172.21.15.154:0/3979155821 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7d180b3e30 con 0x7f7d180cd6c0 2023-09-23T03:16:23.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.943+0000 7f7d0effd640 1 -- 172.21.15.154:0/3979155821 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7d0804b0a0 con 0x7f7d180cd6c0 2023-09-23T03:16:23.944 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:16:23.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.946+0000 7f7d1f2fc640 1 -- 172.21.15.154:0/3979155821 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7cfc06af60 msgr2=0x7f7cfc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:23.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.946+0000 7f7d1f2fc640 1 --2- 172.21.15.154:0/3979155821 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7cfc06af60 0x7f7cfc06d420 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7d180acc20 tx=0x7f7d14008040 comp rx=0 tx=0).stop 2023-09-23T03:16:23.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.946+0000 7f7d1f2fc640 1 -- 172.21.15.154:0/3979155821 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d180cd6c0 msgr2=0x7f7d180ce790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:23.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.946+0000 7f7d1f2fc640 1 --2- 172.21.15.154:0/3979155821 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d180cd6c0 0x7f7d180ce790 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7d08008fd0 tx=0x7f7d08001120 comp rx=0 tx=0).stop 2023-09-23T03:16:23.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.946+0000 7f7d1f2fc640 1 -- 172.21.15.154:0/3979155821 shutdown_connections 2023-09-23T03:16:23.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.946+0000 7f7d1f2fc640 1 --2- 172.21.15.154:0/3979155821 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7cfc06af60 0x7f7cfc06d420 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:23.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.946+0000 7f7d1f2fc640 1 --2- 172.21.15.154:0/3979155821 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d180cecd0 0x7f7d180c6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:23.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.946+0000 7f7d1f2fc640 1 --2- 172.21.15.154:0/3979155821 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d180cd6c0 0x7f7d180ce790 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:23.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.946+0000 7f7d1f2fc640 1 --2- 172.21.15.154:0/3979155821 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d04003f70 0x7f7d18152200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:23.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.946+0000 7f7d1f2fc640 1 -- 172.21.15.154:0/3979155821 >> 172.21.15.154:0/3979155821 conn(0x7f7d180aff20 msgr2=0x7f7d1815a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:23.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.947+0000 7f7d1f2fc640 1 -- 172.21.15.154:0/3979155821 shutdown_connections 2023-09-23T03:16:23.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:23.947+0000 7f7d1f2fc640 1 -- 172.21.15.154:0/3979155821 wait complete. 2023-09-23T03:16:23.966 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:16:24.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.178+0000 7f7fa7c75640 1 Processor -- start 2023-09-23T03:16:24.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.178+0000 7f7fa7c75640 1 -- start start 2023-09-23T03:16:24.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.180+0000 7f7fa7c75640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7fa00cf040 0x7f7fa0160880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:24.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.180+0000 7f7fa7c75640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7fa0160dc0 0x7f7fa00b1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:24.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.180+0000 7f7fa7c75640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fa00cd530 con 0x7f7fa015caf0 2023-09-23T03:16:24.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.180+0000 7f7fa7c75640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fa015ced0 con 0x7f7fa00cf040 2023-09-23T03:16:24.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.180+0000 7f7fa7c75640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fa00ce2f0 con 0x7f7fa0160dc0 2023-09-23T03:16:24.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.180+0000 7f7fa5a11640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f7fa015caf0 0x7f7fa00ce930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:52780/0 (socket says 172.21.15.154:52780) 2023-09-23T03:16:24.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.180+0000 7f7fa5210640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7fa00cf040 0x7f7fa0160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:24.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.180+0000 7f7fa5a11640 1 -- 172.21.15.154:0/3171775428 learned_addr learned my addr 172.21.15.154:0/3171775428 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:24.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.181+0000 7f7fa6212640 1 --2- 172.21.15.154:0/3171775428 >> v2:172.21.15.190:3300/0 conn(0x7f7fa0160dc0 0x7f7fa00b1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:24.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.181+0000 7f7fa5210640 1 -- 172.21.15.154:0/3171775428 >> v2:172.21.15.190:3300/0 conn(0x7f7fa0160dc0 msgr2=0x7f7fa00b1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:24.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.181+0000 7f7fa5210640 1 --2- 172.21.15.154:0/3171775428 >> v2:172.21.15.190:3300/0 conn(0x7f7fa0160dc0 0x7f7fa00b1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:24.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.181+0000 7f7fa5210640 1 -- 172.21.15.154:0/3171775428 >> v1:172.21.15.154:6789/0 conn(0x7f7fa015caf0 legacy=0x7f7fa00ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:24.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.181+0000 7f7fa5210640 1 -- 172.21.15.154:0/3171775428 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa00cd810 con 0x7f7fa00cf040 2023-09-23T03:16:24.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.181+0000 7f7fa4a0f640 1 -- 172.21.15.154:0/3171775428 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4054277223 0 0) 0x7f7fa00cd530 con 0x7f7fa015caf0 2023-09-23T03:16:24.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.182+0000 7f7fa5210640 1 --2- 172.21.15.154:0/3171775428 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fa00cf040 0x7f7fa0160880 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7fa00ad620 tx=0x7f7f9002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1469f5e01f76e22 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:24.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.182+0000 7f7fa4a0f640 1 -- 172.21.15.154:0/3171775428 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f90031830 con 0x7f7fa00cf040 2023-09-23T03:16:24.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.182+0000 7f7fa4a0f640 1 -- 172.21.15.154:0/3171775428 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f90031990 con 0x7f7fa00cf040 2023-09-23T03:16:24.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.182+0000 7f7fa4a0f640 1 -- 172.21.15.154:0/3171775428 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f90031d20 con 0x7f7fa00cf040 2023-09-23T03:16:24.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.183+0000 7f7fa7c75640 1 -- 172.21.15.154:0/3171775428 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fa00cf040 msgr2=0x7f7fa0160880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:24.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.183+0000 7f7fa7c75640 1 --2- 172.21.15.154:0/3171775428 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fa00cf040 0x7f7fa0160880 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7fa00ad620 tx=0x7f7f9002b8d0 comp rx=0 tx=0).stop 2023-09-23T03:16:24.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.183+0000 7f7fa7c75640 1 -- 172.21.15.154:0/3171775428 shutdown_connections 2023-09-23T03:16:24.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.183+0000 7f7fa7c75640 1 --2- 172.21.15.154:0/3171775428 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fa00cf040 0x7f7fa0160880 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:24.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.183+0000 7f7fa7c75640 1 --2- 172.21.15.154:0/3171775428 >> v2:172.21.15.190:3300/0 conn(0x7f7fa0160dc0 0x7f7fa00b1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:24.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.183+0000 7f7fa7c75640 1 -- 172.21.15.154:0/3171775428 >> 172.21.15.154:0/3171775428 conn(0x7f7fa006a0f0 msgr2=0x7f7fa006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:24.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.183+0000 7f7fa7c75640 1 -- 172.21.15.154:0/3171775428 shutdown_connections 2023-09-23T03:16:24.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.183+0000 7f7fa7c75640 1 -- 172.21.15.154:0/3171775428 wait complete. 2023-09-23T03:16:24.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.185+0000 7f7fa7c75640 1 Processor -- start 2023-09-23T03:16:24.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.185+0000 7f7fa7c75640 1 -- start start 2023-09-23T03:16:24.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.186+0000 7f7fa7c75640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7fa00cf040 0x7f7fa0097a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:24.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.186+0000 7f7fa7c75640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7fa015caf0 0x7f7fa0158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:24.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.186+0000 7f7fa7c75640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fa0160dc0 0x7f7fa01553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:24.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.186+0000 7f7fa7c75640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa0097f40 con 0x7f7fa0160dc0 2023-09-23T03:16:24.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.186+0000 7f7fa7c75640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa014cd70 con 0x7f7fa015caf0 2023-09-23T03:16:24.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.186+0000 7f7fa7c75640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa014d050 con 0x7f7fa00cf040 2023-09-23T03:16:24.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.186+0000 7f7fa6212640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fa0160dc0 0x7f7fa01553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:24.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.186+0000 7f7fa6212640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fa0160dc0 0x7f7fa01553c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46330/0 (socket says 172.21.15.154:46330) 2023-09-23T03:16:24.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.186+0000 7f7fa6212640 1 -- 172.21.15.154:0/2013080055 learned_addr learned my addr 172.21.15.154:0/2013080055 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:24.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.186+0000 7f7fa5a11640 1 --2- 172.21.15.154:0/2013080055 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7fa00cf040 0x7f7fa0097a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:24.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.186+0000 7f7fa5210640 1 --2- 172.21.15.154:0/2013080055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7fa015caf0 0x7f7fa0158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:24.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.186+0000 7f7fa6212640 1 -- 172.21.15.154:0/2013080055 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7fa00cf040 msgr2=0x7f7fa0097a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:24.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.186+0000 7f7fa6212640 1 --2- 172.21.15.154:0/2013080055 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7fa00cf040 0x7f7fa0097a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:24.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.186+0000 7f7fa6212640 1 -- 172.21.15.154:0/2013080055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7fa015caf0 msgr2=0x7f7fa0158050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:24.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.186+0000 7f7fa6212640 1 --2- 172.21.15.154:0/2013080055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7fa015caf0 0x7f7fa0158050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:24.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.186+0000 7f7fa6212640 1 -- 172.21.15.154:0/2013080055 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f90009400 con 0x7f7fa0160dc0 2023-09-23T03:16:24.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.187+0000 7f7fa5a11640 1 --2- 172.21.15.154:0/2013080055 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7fa00cf040 0x7f7fa0097a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:24.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.187+0000 7f7fa6212640 1 --2- 172.21.15.154:0/2013080055 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fa0160dc0 0x7f7fa01553c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7f98007e60 tx=0x7f7f9800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:24.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.187+0000 7f7f967fc640 1 -- 172.21.15.154:0/2013080055 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f98014420 con 0x7f7fa0160dc0 2023-09-23T03:16:24.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.187+0000 7f7f967fc640 1 -- 172.21.15.154:0/2013080055 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f980145a0 con 0x7f7fa0160dc0 2023-09-23T03:16:24.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.187+0000 7f7f967fc640 1 -- 172.21.15.154:0/2013080055 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f98014950 con 0x7f7fa0160dc0 2023-09-23T03:16:24.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.187+0000 7f7fa7c75640 1 -- 172.21.15.154:0/2013080055 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa0157570 con 0x7f7fa0160dc0 2023-09-23T03:16:24.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.187+0000 7f7fa7c75640 1 -- 172.21.15.154:0/2013080055 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa0157b30 con 0x7f7fa0160dc0 2023-09-23T03:16:24.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.188+0000 7f7fa7c75640 1 -- 172.21.15.154:0/2013080055 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fa0073eb0 con 0x7f7fa0160dc0 2023-09-23T03:16:24.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.189+0000 7f7f967fc640 1 -- 172.21.15.154:0/2013080055 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f7f98014ab0 con 0x7f7fa0160dc0 2023-09-23T03:16:24.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.190+0000 7f7f967fc640 1 --2- 172.21.15.154:0/2013080055 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7f8406af60 0x7f7f8406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:24.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.190+0000 7f7f967fc640 1 -- 172.21.15.154:0/2013080055 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f7f9808a3c0 con 0x7f7fa0160dc0 2023-09-23T03:16:24.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.191+0000 7f7fa5a11640 1 --2- 172.21.15.154:0/2013080055 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7f8406af60 0x7f7f8406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:24.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.191+0000 7f7fa5a11640 1 --2- 172.21.15.154:0/2013080055 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7f8406af60 0x7f7f8406d420 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c004870 tx=0x7f7f9c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:24.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.195+0000 7f7f967fc640 1 -- 172.21.15.154:0/2013080055 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f7f980550e0 con 0x7f7fa0160dc0 2023-09-23T03:16:24.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.388+0000 7f7fa7c75640 1 -- 172.21.15.154:0/2013080055 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7fa0156920 con 0x7f7fa0160dc0 2023-09-23T03:16:24.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.389+0000 7f7f967fc640 1 -- 172.21.15.154:0/2013080055 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18744 (secure 0 0 0) 0x7f7f98058700 con 0x7f7fa0160dc0 2023-09-23T03:16:24.390 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:16:24.390 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":26,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:16:03.151371+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:15:18.575849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T03:15:19.707212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T03:15:23.337138+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:16:24.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.392+0000 7f7fa7c75640 1 -- 172.21.15.154:0/2013080055 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7f8406af60 msgr2=0x7f7f8406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:24.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.392+0000 7f7fa7c75640 1 --2- 172.21.15.154:0/2013080055 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7f8406af60 0x7f7f8406d420 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c004870 tx=0x7f7f9c008040 comp rx=0 tx=0).stop 2023-09-23T03:16:24.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.392+0000 7f7fa7c75640 1 -- 172.21.15.154:0/2013080055 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fa0160dc0 msgr2=0x7f7fa01553c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:24.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.392+0000 7f7fa7c75640 1 --2- 172.21.15.154:0/2013080055 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fa0160dc0 0x7f7fa01553c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7f98007e60 tx=0x7f7f9800e4b0 comp rx=0 tx=0).stop 2023-09-23T03:16:24.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.393+0000 7f7fa7c75640 1 -- 172.21.15.154:0/2013080055 shutdown_connections 2023-09-23T03:16:24.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.393+0000 7f7fa7c75640 1 --2- 172.21.15.154:0/2013080055 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7f8406af60 0x7f7f8406d420 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:24.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.393+0000 7f7fa7c75640 1 --2- 172.21.15.154:0/2013080055 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7fa0160dc0 0x7f7fa01553c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:24.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.393+0000 7f7fa7c75640 1 --2- 172.21.15.154:0/2013080055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7fa015caf0 0x7f7fa0158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:24.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.393+0000 7f7fa7c75640 1 --2- 172.21.15.154:0/2013080055 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7fa00cf040 0x7f7fa0097a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:24.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.393+0000 7f7fa7c75640 1 -- 172.21.15.154:0/2013080055 >> 172.21.15.154:0/2013080055 conn(0x7f7fa006a0f0 msgr2=0x7f7fa00aea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:24.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.393+0000 7f7fa7c75640 1 -- 172.21.15.154:0/2013080055 shutdown_connections 2023-09-23T03:16:24.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.393+0000 7f7fa7c75640 1 -- 172.21.15.154:0/2013080055 wait complete. 2023-09-23T03:16:24.411 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:15:18.575849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:15:19.707212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-23T03:15:23.337138+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-23T03:16:24.411 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:16:24.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.619+0000 7efcdcc10640 1 Processor -- start 2023-09-23T03:16:24.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.619+0000 7efcdcc10640 1 -- start start 2023-09-23T03:16:24.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.621+0000 7efcdcc10640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efcd80c8d70 0x7efcd815c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:24.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.621+0000 7efcdcc10640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efcd80c9150 0x7efcd815cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:24.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.621+0000 7efcdcc10640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcd80c9d20 con 0x7efcd8071290 2023-09-23T03:16:24.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.621+0000 7efcdcc10640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcd8071850 con 0x7efcd80c8d70 2023-09-23T03:16:24.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.621+0000 7efcdcc10640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcd8071b30 con 0x7efcd80c9150 2023-09-23T03:16:24.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.621+0000 7efcd659c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efcd80c8d70 0x7efcd815c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:24.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.621+0000 7efcd6d9d640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7efcd8071290 0x7efcd80ca1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:52786/0 (socket says 172.21.15.154:52786) 2023-09-23T03:16:24.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.621+0000 7efcd6d9d640 1 -- 172.21.15.154:0/2234097176 learned_addr learned my addr 172.21.15.154:0/2234097176 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:24.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.622+0000 7efccffff640 1 --2- 172.21.15.154:0/2234097176 >> v2:172.21.15.190:3300/0 conn(0x7efcd80c9150 0x7efcd815cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:24.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.622+0000 7efcd659c640 1 -- 172.21.15.154:0/2234097176 >> v2:172.21.15.190:3300/0 conn(0x7efcd80c9150 msgr2=0x7efcd815cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:24.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.622+0000 7efcd659c640 1 --2- 172.21.15.154:0/2234097176 >> v2:172.21.15.190:3300/0 conn(0x7efcd80c9150 0x7efcd815cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:24.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.622+0000 7efcd659c640 1 -- 172.21.15.154:0/2234097176 >> v1:172.21.15.154:6789/0 conn(0x7efcd8071290 legacy=0x7efcd80ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:24.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.622+0000 7efcd659c640 1 -- 172.21.15.154:0/2234097176 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd8071e10 con 0x7efcd80c8d70 2023-09-23T03:16:24.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.622+0000 7efcd5d9b640 1 -- 172.21.15.154:0/2234097176 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3985192182 0 0) 0x7efcd80c9d20 con 0x7efcd8071290 2023-09-23T03:16:24.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.622+0000 7efccffff640 1 --2- 172.21.15.154:0/2234097176 >> v2:172.21.15.190:3300/0 conn(0x7efcd80c9150 0x7efcd815cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:24.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.623+0000 7efcd659c640 1 --2- 172.21.15.154:0/2234097176 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcd80c8d70 0x7efcd815c6d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7efcd80b3ce0 tx=0x7efcc802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f73155adb341bfe server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:24.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.623+0000 7efcd5d9b640 1 -- 172.21.15.154:0/2234097176 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc802cc80 con 0x7efcd80c8d70 2023-09-23T03:16:24.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.623+0000 7efcd5d9b640 1 -- 172.21.15.154:0/2234097176 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcc802cde0 con 0x7efcd80c8d70 2023-09-23T03:16:24.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.623+0000 7efcd5d9b640 1 -- 172.21.15.154:0/2234097176 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc8034660 con 0x7efcd80c8d70 2023-09-23T03:16:24.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.624+0000 7efcdcc10640 1 -- 172.21.15.154:0/2234097176 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcd80c8d70 msgr2=0x7efcd815c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:24.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.624+0000 7efcdcc10640 1 --2- 172.21.15.154:0/2234097176 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcd80c8d70 0x7efcd815c6d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7efcd80b3ce0 tx=0x7efcc802bed0 comp rx=0 tx=0).stop 2023-09-23T03:16:24.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.624+0000 7efcdcc10640 1 -- 172.21.15.154:0/2234097176 shutdown_connections 2023-09-23T03:16:24.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.624+0000 7efcdcc10640 1 --2- 172.21.15.154:0/2234097176 >> v2:172.21.15.190:3300/0 conn(0x7efcd80c9150 0x7efcd815cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:24.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.624+0000 7efcdcc10640 1 --2- 172.21.15.154:0/2234097176 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcd80c8d70 0x7efcd815c6d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:24.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.624+0000 7efcdcc10640 1 -- 172.21.15.154:0/2234097176 >> 172.21.15.154:0/2234097176 conn(0x7efcd806c010 msgr2=0x7efcd806c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:24.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.624+0000 7efcdcc10640 1 -- 172.21.15.154:0/2234097176 shutdown_connections 2023-09-23T03:16:24.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.624+0000 7efcdcc10640 1 -- 172.21.15.154:0/2234097176 wait complete. 2023-09-23T03:16:24.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.626+0000 7efcdcc10640 1 Processor -- start 2023-09-23T03:16:24.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.626+0000 7efcdcc10640 1 -- start start 2023-09-23T03:16:24.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.627+0000 7efcdcc10640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efcd8071290 0x7efcd80c4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:24.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.627+0000 7efcdcc10640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcd80c8d70 0x7efcd8153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:24.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.627+0000 7efcdcc10640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcd80c9150 0x7efcd80c5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:24.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.627+0000 7efcdcc10640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd80c7750 con 0x7efcd80c9150 2023-09-23T03:16:24.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.627+0000 7efcdcc10640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd80c7a30 con 0x7efcd80c8d70 2023-09-23T03:16:24.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.627+0000 7efcdcc10640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd80c7d10 con 0x7efcd8071290 2023-09-23T03:16:24.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.627+0000 7efcd6d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcd80c9150 0x7efcd80c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:24.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.627+0000 7efcd6d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcd80c9150 0x7efcd80c5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46342/0 (socket says 172.21.15.154:46342) 2023-09-23T03:16:24.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.627+0000 7efcd6d9d640 1 -- 172.21.15.154:0/471754681 learned_addr learned my addr 172.21.15.154:0/471754681 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:24.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.627+0000 7efcd659c640 1 --2- 172.21.15.154:0/471754681 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efcd8071290 0x7efcd80c4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:24.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.627+0000 7efccffff640 1 --2- 172.21.15.154:0/471754681 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcd80c8d70 0x7efcd8153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:24.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.627+0000 7efcd6d9d640 1 -- 172.21.15.154:0/471754681 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efcd8071290 msgr2=0x7efcd80c4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:24.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.627+0000 7efcd6d9d640 1 --2- 172.21.15.154:0/471754681 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efcd8071290 0x7efcd80c4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:24.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.627+0000 7efcd6d9d640 1 -- 172.21.15.154:0/471754681 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcd80c8d70 msgr2=0x7efcd8153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:24.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.627+0000 7efcd6d9d640 1 --2- 172.21.15.154:0/471754681 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcd80c8d70 0x7efcd8153a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:24.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.627+0000 7efcd6d9d640 1 -- 172.21.15.154:0/471754681 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc8009a70 con 0x7efcd80c9150 2023-09-23T03:16:24.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.627+0000 7efcd659c640 1 --2- 172.21.15.154:0/471754681 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efcd8071290 0x7efcd80c4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:24.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.627+0000 7efccffff640 1 --2- 172.21.15.154:0/471754681 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcd80c8d70 0x7efcd8153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:24.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.628+0000 7efcd6d9d640 1 --2- 172.21.15.154:0/471754681 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcd80c9150 0x7efcd80c5580 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efcd000c450 tx=0x7efcd000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:24.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.628+0000 7efcceffd640 1 -- 172.21.15.154:0/471754681 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd00074e0 con 0x7efcd80c9150 2023-09-23T03:16:24.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.628+0000 7efcceffd640 1 -- 172.21.15.154:0/471754681 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd0007660 con 0x7efcd80c9150 2023-09-23T03:16:24.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.628+0000 7efcceffd640 1 -- 172.21.15.154:0/471754681 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0004a20 con 0x7efcd80c9150 2023-09-23T03:16:24.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.628+0000 7efcdcc10640 1 -- 172.21.15.154:0/471754681 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcd80c7ff0 con 0x7efcd80c9150 2023-09-23T03:16:24.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.628+0000 7efcdcc10640 1 -- 172.21.15.154:0/471754681 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd8152440 con 0x7efcd80c9150 2023-09-23T03:16:24.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.629+0000 7efcdcc10640 1 -- 172.21.15.154:0/471754681 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcd80c8510 con 0x7efcd80c9150 2023-09-23T03:16:24.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.630+0000 7efcceffd640 1 -- 172.21.15.154:0/471754681 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7efcd0004b80 con 0x7efcd80c9150 2023-09-23T03:16:24.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.631+0000 7efcceffd640 1 --2- 172.21.15.154:0/471754681 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efcb806af30 0x7efcb806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:24.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.631+0000 7efcceffd640 1 -- 172.21.15.154:0/471754681 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7efcd0088d30 con 0x7efcd80c9150 2023-09-23T03:16:24.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.631+0000 7efcd659c640 1 --2- 172.21.15.154:0/471754681 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efcb806af30 0x7efcb806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:24.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.632+0000 7efcd659c640 1 --2- 172.21.15.154:0/471754681 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efcb806af30 0x7efcb806d3f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efcc8036000 tx=0x7efcc8030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:24.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.635+0000 7efcceffd640 1 -- 172.21.15.154:0/471754681 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7efcd80c8510 con 0x7efcd80c9150 2023-09-23T03:16:24.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.822+0000 7efcdcc10640 1 -- 172.21.15.154:0/471754681 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efcd80c8510 con 0x7efcd80c9150 2023-09-23T03:16:24.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.823+0000 7efcceffd640 1 -- 172.21.15.154:0/471754681 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7efcd80c8510 con 0x7efcd80c9150 2023-09-23T03:16:24.823 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:16:24.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.825+0000 7efcdcc10640 1 -- 172.21.15.154:0/471754681 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efcb806af30 msgr2=0x7efcb806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:24.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.825+0000 7efcdcc10640 1 --2- 172.21.15.154:0/471754681 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efcb806af30 0x7efcb806d3f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efcc8036000 tx=0x7efcc8030000 comp rx=0 tx=0).stop 2023-09-23T03:16:24.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.825+0000 7efcdcc10640 1 -- 172.21.15.154:0/471754681 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcd80c9150 msgr2=0x7efcd80c5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:24.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.825+0000 7efcdcc10640 1 --2- 172.21.15.154:0/471754681 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcd80c9150 0x7efcd80c5580 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7efcd000c450 tx=0x7efcd000c910 comp rx=0 tx=0).stop 2023-09-23T03:16:24.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.825+0000 7efcdcc10640 1 -- 172.21.15.154:0/471754681 shutdown_connections 2023-09-23T03:16:24.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.825+0000 7efcdcc10640 1 --2- 172.21.15.154:0/471754681 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efcb806af30 0x7efcb806d3f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:24.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.825+0000 7efcdcc10640 1 --2- 172.21.15.154:0/471754681 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcd80c9150 0x7efcd80c5580 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:24.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.825+0000 7efcdcc10640 1 --2- 172.21.15.154:0/471754681 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcd80c8d70 0x7efcd8153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:24.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.825+0000 7efcdcc10640 1 --2- 172.21.15.154:0/471754681 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efcd8071290 0x7efcd80c4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:24.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.825+0000 7efcdcc10640 1 -- 172.21.15.154:0/471754681 >> 172.21.15.154:0/471754681 conn(0x7efcd806c010 msgr2=0x7efcd8158850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:24.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.825+0000 7efcdcc10640 1 -- 172.21.15.154:0/471754681 shutdown_connections 2023-09-23T03:16:24.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:24.825+0000 7efcdcc10640 1 -- 172.21.15.154:0/471754681 wait complete. 2023-09-23T03:16:24.842 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:16:25.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.052+0000 7f4b54cfc640 1 Processor -- start 2023-09-23T03:16:25.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.052+0000 7f4b54cfc640 1 -- start start 2023-09-23T03:16:25.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.053+0000 7f4b54cfc640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4b5015aba0 0x7f4b5015bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:25.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.053+0000 7f4b54cfc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b500cb5b0 0x7f4b5015ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:25.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.053+0000 7f4b54cfc640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b5015b6a0 con 0x7f4b5015e5f0 2023-09-23T03:16:25.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.053+0000 7f4b54cfc640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b5015f0b0 con 0x7f4b5015aba0 2023-09-23T03:16:25.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.053+0000 7f4b54cfc640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5015f390 con 0x7f4b500cb5b0 2023-09-23T03:16:25.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.053+0000 7f4b47fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f4b5015e5f0 0x7f4b500caea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:52794/0 (socket says 172.21.15.154:52794) 2023-09-23T03:16:25.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.053+0000 7f4b47fff640 1 -- 172.21.15.154:0/3706267724 learned_addr learned my addr 172.21.15.154:0/3706267724 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:25.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.053+0000 7f4b4e59c640 1 --2- 172.21.15.154:0/3706267724 >> v2:172.21.15.154:3300/0 conn(0x7f4b5015aba0 0x7f4b5015bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:25.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.054+0000 7f4b4ed9d640 1 --2- 172.21.15.154:0/3706267724 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b500cb5b0 0x7f4b5015ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:25.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.054+0000 7f4b4e59c640 1 -- 172.21.15.154:0/3706267724 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b500cb5b0 msgr2=0x7f4b5015ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:25.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.054+0000 7f4b4e59c640 1 --2- 172.21.15.154:0/3706267724 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b500cb5b0 0x7f4b5015ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:25.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.054+0000 7f4b4e59c640 1 -- 172.21.15.154:0/3706267724 >> v1:172.21.15.154:6789/0 conn(0x7f4b5015e5f0 legacy=0x7f4b500caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:25.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.054+0000 7f4b4e59c640 1 -- 172.21.15.154:0/3706267724 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b5015b330 con 0x7f4b5015aba0 2023-09-23T03:16:25.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.054+0000 7f4b4ed9d640 1 --2- 172.21.15.154:0/3706267724 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b500cb5b0 0x7f4b5015ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:25.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.054+0000 7f4b4dd9b640 1 -- 172.21.15.154:0/3706267724 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3583413168 0 0) 0x7f4b5015b6a0 con 0x7f4b5015e5f0 2023-09-23T03:16:25.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.055+0000 7f4b4e59c640 1 --2- 172.21.15.154:0/3706267724 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b5015aba0 0x7f4b5015bb30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4b400097a0 tx=0x7f4b4002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985c528a48ef235d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:25.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.055+0000 7f4b4dd9b640 1 -- 172.21.15.154:0/3706267724 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4002d040 con 0x7f4b5015aba0 2023-09-23T03:16:25.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.055+0000 7f4b4dd9b640 1 -- 172.21.15.154:0/3706267724 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b40031810 con 0x7f4b5015aba0 2023-09-23T03:16:25.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.055+0000 7f4b4dd9b640 1 -- 172.21.15.154:0/3706267724 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b40031ba0 con 0x7f4b5015aba0 2023-09-23T03:16:25.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.056+0000 7f4b54cfc640 1 -- 172.21.15.154:0/3706267724 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b5015aba0 msgr2=0x7f4b5015bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:25.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.056+0000 7f4b54cfc640 1 --2- 172.21.15.154:0/3706267724 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b5015aba0 0x7f4b5015bb30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4b400097a0 tx=0x7f4b4002b8b0 comp rx=0 tx=0).stop 2023-09-23T03:16:25.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.056+0000 7f4b54cfc640 1 -- 172.21.15.154:0/3706267724 shutdown_connections 2023-09-23T03:16:25.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.056+0000 7f4b54cfc640 1 --2- 172.21.15.154:0/3706267724 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b500cb5b0 0x7f4b5015ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:25.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.056+0000 7f4b54cfc640 1 --2- 172.21.15.154:0/3706267724 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b5015aba0 0x7f4b5015bb30 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:25.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.056+0000 7f4b54cfc640 1 -- 172.21.15.154:0/3706267724 >> 172.21.15.154:0/3706267724 conn(0x7f4b5006c010 msgr2=0x7f4b5006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:25.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.056+0000 7f4b54cfc640 1 -- 172.21.15.154:0/3706267724 shutdown_connections 2023-09-23T03:16:25.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.056+0000 7f4b54cfc640 1 -- 172.21.15.154:0/3706267724 wait complete. 2023-09-23T03:16:25.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.058+0000 7f4b54cfc640 1 Processor -- start 2023-09-23T03:16:25.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.058+0000 7f4b54cfc640 1 -- start start 2023-09-23T03:16:25.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.059+0000 7f4b54cfc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b500cb5b0 0x7f4b500c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:25.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.059+0000 7f4b54cfc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b5015aba0 0x7f4b50154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:25.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.059+0000 7f4b54cfc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b5015e5f0 0x7f4b500ae9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:25.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.059+0000 7f4b54cfc640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b500c7bb0 con 0x7f4b5015aba0 2023-09-23T03:16:25.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.059+0000 7f4b54cfc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b50154bc0 con 0x7f4b500cb5b0 2023-09-23T03:16:25.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.059+0000 7f4b54cfc640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b500c6fa0 con 0x7f4b5015e5f0 2023-09-23T03:16:25.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.059+0000 7f4b47fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b5015aba0 0x7f4b50154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:25.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.059+0000 7f4b47fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b5015aba0 0x7f4b50154680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46364/0 (socket says 172.21.15.154:46364) 2023-09-23T03:16:25.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.059+0000 7f4b47fff640 1 -- 172.21.15.154:0/1776111479 learned_addr learned my addr 172.21.15.154:0/1776111479 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:25.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.059+0000 7f4b4ed9d640 1 --2- 172.21.15.154:0/1776111479 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b5015e5f0 0x7f4b500ae9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:25.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.059+0000 7f4b4e59c640 1 --2- 172.21.15.154:0/1776111479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b500cb5b0 0x7f4b500c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:25.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.059+0000 7f4b47fff640 1 -- 172.21.15.154:0/1776111479 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b5015e5f0 msgr2=0x7f4b500ae9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:25.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.059+0000 7f4b47fff640 1 --2- 172.21.15.154:0/1776111479 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b5015e5f0 0x7f4b500ae9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:25.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.059+0000 7f4b47fff640 1 -- 172.21.15.154:0/1776111479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b500cb5b0 msgr2=0x7f4b500c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:25.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.059+0000 7f4b47fff640 1 --2- 172.21.15.154:0/1776111479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b500cb5b0 0x7f4b500c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:25.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.060+0000 7f4b47fff640 1 -- 172.21.15.154:0/1776111479 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b40009400 con 0x7f4b5015aba0 2023-09-23T03:16:25.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.060+0000 7f4b4ed9d640 1 --2- 172.21.15.154:0/1776111479 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b5015e5f0 0x7f4b500ae9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:25.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.060+0000 7f4b4e59c640 1 --2- 172.21.15.154:0/1776111479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b500cb5b0 0x7f4b500c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:25.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.060+0000 7f4b47fff640 1 --2- 172.21.15.154:0/1776111479 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b5015aba0 0x7f4b50154680 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4b3800c430 tx=0x7f4b3800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:25.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.060+0000 7f4b46ffd640 1 -- 172.21.15.154:0/1776111479 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b380074e0 con 0x7f4b5015aba0 2023-09-23T03:16:25.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.060+0000 7f4b46ffd640 1 -- 172.21.15.154:0/1776111479 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b38007660 con 0x7f4b5015aba0 2023-09-23T03:16:25.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.060+0000 7f4b46ffd640 1 -- 172.21.15.154:0/1776111479 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b38004a20 con 0x7f4b5015aba0 2023-09-23T03:16:25.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.060+0000 7f4b54cfc640 1 -- 172.21.15.154:0/1776111479 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b5015b100 con 0x7f4b5015aba0 2023-09-23T03:16:25.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.060+0000 7f4b54cfc640 1 -- 172.21.15.154:0/1776111479 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b500c7560 con 0x7f4b5015aba0 2023-09-23T03:16:25.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.061+0000 7f4b54cfc640 1 -- 172.21.15.154:0/1776111479 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b500c7d20 con 0x7f4b5015aba0 2023-09-23T03:16:25.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.062+0000 7f4b46ffd640 1 -- 172.21.15.154:0/1776111479 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f4b38004b80 con 0x7f4b5015aba0 2023-09-23T03:16:25.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.063+0000 7f4b46ffd640 1 --2- 172.21.15.154:0/1776111479 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4b3006af60 0x7f4b3006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:25.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.063+0000 7f4b46ffd640 1 -- 172.21.15.154:0/1776111479 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f4b38089900 con 0x7f4b5015aba0 2023-09-23T03:16:25.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.064+0000 7f4b4e59c640 1 --2- 172.21.15.154:0/1776111479 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4b3006af60 0x7f4b3006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:25.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.064+0000 7f4b4e59c640 1 --2- 172.21.15.154:0/1776111479 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4b3006af60 0x7f4b3006d420 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4b40036000 tx=0x7f4b4003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:25.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.068+0000 7f4b46ffd640 1 -- 172.21.15.154:0/1776111479 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f4b38054620 con 0x7f4b5015aba0 2023-09-23T03:16:25.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.250+0000 7f4b54cfc640 1 -- 172.21.15.154:0/1776111479 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4b5004e920 con 0x7f4b5015aba0 2023-09-23T03:16:25.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.250+0000 7f4b46ffd640 1 -- 172.21.15.154:0/1776111479 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4b3805e030 con 0x7f4b5015aba0 2023-09-23T03:16:25.251 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:16:25.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.253+0000 7f4b54cfc640 1 -- 172.21.15.154:0/1776111479 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4b3006af60 msgr2=0x7f4b3006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:25.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.253+0000 7f4b54cfc640 1 --2- 172.21.15.154:0/1776111479 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4b3006af60 0x7f4b3006d420 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4b40036000 tx=0x7f4b4003c040 comp rx=0 tx=0).stop 2023-09-23T03:16:25.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.253+0000 7f4b54cfc640 1 -- 172.21.15.154:0/1776111479 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b5015aba0 msgr2=0x7f4b50154680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:25.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.253+0000 7f4b54cfc640 1 --2- 172.21.15.154:0/1776111479 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b5015aba0 0x7f4b50154680 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4b3800c430 tx=0x7f4b3800c8f0 comp rx=0 tx=0).stop 2023-09-23T03:16:25.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.253+0000 7f4b54cfc640 1 -- 172.21.15.154:0/1776111479 shutdown_connections 2023-09-23T03:16:25.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.253+0000 7f4b54cfc640 1 --2- 172.21.15.154:0/1776111479 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4b3006af60 0x7f4b3006d420 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:25.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.253+0000 7f4b54cfc640 1 --2- 172.21.15.154:0/1776111479 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4b5015e5f0 0x7f4b500ae9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:25.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.253+0000 7f4b54cfc640 1 --2- 172.21.15.154:0/1776111479 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4b5015aba0 0x7f4b50154680 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:25.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.253+0000 7f4b54cfc640 1 --2- 172.21.15.154:0/1776111479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4b500cb5b0 0x7f4b500c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:25.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.253+0000 7f4b54cfc640 1 -- 172.21.15.154:0/1776111479 >> 172.21.15.154:0/1776111479 conn(0x7f4b5006c010 msgr2=0x7f4b500701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:25.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.253+0000 7f4b54cfc640 1 -- 172.21.15.154:0/1776111479 shutdown_connections 2023-09-23T03:16:25.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.253+0000 7f4b54cfc640 1 -- 172.21.15.154:0/1776111479 wait complete. 2023-09-23T03:16:25.270 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:16:25.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.468+0000 7f2bdac83640 1 Processor -- start 2023-09-23T03:16:25.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.469+0000 7f2bdac83640 1 -- start start 2023-09-23T03:16:25.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.470+0000 7f2bdac83640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2bd40c9980 0x7f2bd4163dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:25.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.470+0000 7f2bdac83640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bd40c1a40 con 0x7f2bd40cae90 2023-09-23T03:16:25.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.470+0000 7f2bdac83640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bd41624f0 con 0x7f2bd40c9510 2023-09-23T03:16:25.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.470+0000 7f2bdac83640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bd4162be0 con 0x7f2bd40c9980 2023-09-23T03:16:25.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.471+0000 7f2bd8a1f640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f2bd40cae90 0x7f2bd40cb270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:52810/0 (socket says 172.21.15.154:52810) 2023-09-23T03:16:25.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.471+0000 7f2bd8a1f640 1 -- 172.21.15.154:0/3674352180 learned_addr learned my addr 172.21.15.154:0/3674352180 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:25.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.471+0000 7f2bd9220640 1 --2- 172.21.15.154:0/3674352180 >> v2:172.21.15.190:3300/0 conn(0x7f2bd40c9980 0x7f2bd4163dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:25.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.471+0000 7f2bd37fe640 1 -- 172.21.15.154:0/3674352180 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 467464016 0 0) 0x7f2bd40c1a40 con 0x7f2bd40cae90 2023-09-23T03:16:25.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.471+0000 7f2bd37fe640 1 -- 172.21.15.154:0/3674352180 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bc0003520 con 0x7f2bd40cae90 2023-09-23T03:16:25.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.471+0000 7f2bd37fe640 1 -- 172.21.15.154:0/3674352180 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2350338320 0 0) 0x7f2bc0003520 con 0x7f2bd40cae90 2023-09-23T03:16:25.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.471+0000 7f2bd37fe640 1 -- 172.21.15.154:0/3674352180 >> v2:172.21.15.190:3300/0 conn(0x7f2bd40c9980 msgr2=0x7f2bd4163dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:25.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.471+0000 7f2bd37fe640 1 --2- 172.21.15.154:0/3674352180 >> v2:172.21.15.190:3300/0 conn(0x7f2bd40c9980 0x7f2bd4163dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:25.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.471+0000 7f2bd37fe640 1 -- 172.21.15.154:0/3674352180 >> v1:172.21.15.190:6789/0 conn(0x7f2bd40c9510 legacy=0x7f2bd41636c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:25.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.472+0000 7f2bd37fe640 1 -- 172.21.15.154:0/3674352180 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd4162ec0 con 0x7f2bd40cae90 2023-09-23T03:16:25.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.472+0000 7f2bd37fe640 1 -- 172.21.15.154:0/3674352180 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f2bc4001a10 con 0x7f2bd40cae90 2023-09-23T03:16:25.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.472+0000 7f2bd37fe640 1 -- 172.21.15.154:0/3674352180 >> v1:172.21.15.154:6789/0 conn(0x7f2bd40cae90 legacy=0x7f2bd40cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:25.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.472+0000 7f2bd37fe640 1 --2- 172.21.15.154:0/3674352180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bc00039c0 0x7f2bc0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:25.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.472+0000 7f2bd37fe640 1 --2- 172.21.15.154:0/3674352180 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bc00047c0 0x7f2bc0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:25.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.472+0000 7f2bd37fe640 1 --2- 172.21.15.154:0/3674352180 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bc0005180 0x7f2bc000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:25.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.472+0000 7f2bd37fe640 1 -- 172.21.15.154:0/3674352180 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd4162be0 con 0x7f2bc00047c0 2023-09-23T03:16:25.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.472+0000 7f2bd37fe640 1 -- 172.21.15.154:0/3674352180 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc000c060 con 0x7f2bc00039c0 2023-09-23T03:16:25.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.472+0000 7f2bd37fe640 1 -- 172.21.15.154:0/3674352180 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc000c220 con 0x7f2bc0005180 2023-09-23T03:16:25.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.472+0000 7f2bd3fff640 1 --2- 172.21.15.154:0/3674352180 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bc00047c0 0x7f2bc0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:25.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.472+0000 7f2bd3fff640 1 -- 172.21.15.154:0/3674352180 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bc0005180 msgr2=0x7f2bc000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:25.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.472+0000 7f2bd3fff640 1 --2- 172.21.15.154:0/3674352180 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bc0005180 0x7f2bc000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:25.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.472+0000 7f2bd3fff640 1 -- 172.21.15.154:0/3674352180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bc00039c0 msgr2=0x7f2bc0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:25.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.472+0000 7f2bd8a1f640 1 --2- 172.21.15.154:0/3674352180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bc00039c0 0x7f2bc0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:25.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.472+0000 7f2bd3fff640 1 --2- 172.21.15.154:0/3674352180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bc00039c0 0x7f2bc0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:25.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.472+0000 7f2bd9220640 1 -- 172.21.15.154:0/3674352180 reap_dead start 2023-09-23T03:16:25.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.472+0000 7f2bd3fff640 1 -- 172.21.15.154:0/3674352180 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc0003520 con 0x7f2bc00047c0 2023-09-23T03:16:25.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.473+0000 7f2bd8a1f640 1 --2- 172.21.15.154:0/3674352180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bc00039c0 0x7f2bc0003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:25.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.474+0000 7f2bd3fff640 1 --2- 172.21.15.154:0/3674352180 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bc00047c0 0x7f2bc0004c40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc001750 tx=0x7f2bc00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c8dc44c0c6f69f9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:25.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.474+0000 7f2bd37fe640 1 -- 172.21.15.154:0/3674352180 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc0052d0 con 0x7f2bc00047c0 2023-09-23T03:16:25.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.474+0000 7f2bd37fe640 1 -- 172.21.15.154:0/3674352180 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bbc02e050 con 0x7f2bc00047c0 2023-09-23T03:16:25.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.474+0000 7f2bd37fe640 1 -- 172.21.15.154:0/3674352180 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc003830 con 0x7f2bc00047c0 2023-09-23T03:16:25.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.474+0000 7f2bdac83640 1 -- 172.21.15.154:0/3674352180 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bc00047c0 msgr2=0x7f2bc0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:25.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.474+0000 7f2bdac83640 1 --2- 172.21.15.154:0/3674352180 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bc00047c0 0x7f2bc0004c40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc001750 tx=0x7f2bc00057f0 comp rx=0 tx=0).stop 2023-09-23T03:16:25.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.475+0000 7f2bdac83640 1 -- 172.21.15.154:0/3674352180 shutdown_connections 2023-09-23T03:16:25.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.475+0000 7f2bdac83640 1 --2- 172.21.15.154:0/3674352180 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bc00047c0 0x7f2bc0004c40 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:25.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.475+0000 7f2bdac83640 1 -- 172.21.15.154:0/3674352180 >> 172.21.15.154:0/3674352180 conn(0x7f2bd406c010 msgr2=0x7f2bd406c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:25.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.475+0000 7f2bdac83640 1 -- 172.21.15.154:0/3674352180 shutdown_connections 2023-09-23T03:16:25.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.475+0000 7f2bdac83640 1 -- 172.21.15.154:0/3674352180 wait complete. 2023-09-23T03:16:25.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.476+0000 7f2bdac83640 1 Processor -- start 2023-09-23T03:16:25.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.477+0000 7f2bdac83640 1 -- start start 2023-09-23T03:16:25.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.477+0000 7f2bdac83640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bc00047c0 0x7f2bd415ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:25.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.477+0000 7f2bdac83640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bd40b3b70 0x7f2bd414f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:25.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.478+0000 7f2bdac83640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bd415f0c0 0x7f2bd415f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:25.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.478+0000 7f2bdac83640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd415fab0 con 0x7f2bc00047c0 2023-09-23T03:16:25.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.478+0000 7f2bdac83640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd414ff10 con 0x7f2bd415f0c0 2023-09-23T03:16:25.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.478+0000 7f2bdac83640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd40b3ff0 con 0x7f2bd40b3b70 2023-09-23T03:16:25.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.478+0000 7f2bd8a1f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bc00047c0 0x7f2bd415ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:25.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.478+0000 7f2bd8a1f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bc00047c0 0x7f2bd415ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46376/0 (socket says 172.21.15.154:46376) 2023-09-23T03:16:25.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.478+0000 7f2bd8a1f640 1 -- 172.21.15.154:0/1058372054 learned_addr learned my addr 172.21.15.154:0/1058372054 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:25.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.478+0000 7f2bd3fff640 1 --2- 172.21.15.154:0/1058372054 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bd40b3b70 0x7f2bd414f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:25.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.478+0000 7f2bd9220640 1 --2- 172.21.15.154:0/1058372054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bd415f0c0 0x7f2bd415f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:25.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.478+0000 7f2bd8a1f640 1 -- 172.21.15.154:0/1058372054 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bd40b3b70 msgr2=0x7f2bd414f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:25.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.478+0000 7f2bd8a1f640 1 --2- 172.21.15.154:0/1058372054 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bd40b3b70 0x7f2bd414f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:25.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.478+0000 7f2bd8a1f640 1 -- 172.21.15.154:0/1058372054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bd415f0c0 msgr2=0x7f2bd415f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:25.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.478+0000 7f2bd8a1f640 1 --2- 172.21.15.154:0/1058372054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bd415f0c0 0x7f2bd415f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:25.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.478+0000 7f2bd8a1f640 1 -- 172.21.15.154:0/1058372054 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd40b4270 con 0x7f2bc00047c0 2023-09-23T03:16:25.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.478+0000 7f2bd3fff640 1 --2- 172.21.15.154:0/1058372054 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bd40b3b70 0x7f2bd414f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:25.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.479+0000 7f2bd8a1f640 1 --2- 172.21.15.154:0/1058372054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bc00047c0 0x7f2bd415ce10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2bc400dcc0 tx=0x7f2bc400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:25.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.479+0000 7f2bd17fa640 1 -- 172.21.15.154:0/1058372054 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4004b60 con 0x7f2bc00047c0 2023-09-23T03:16:25.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.479+0000 7f2bd17fa640 1 -- 172.21.15.154:0/1058372054 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc4004cc0 con 0x7f2bc00047c0 2023-09-23T03:16:25.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.479+0000 7f2bd17fa640 1 -- 172.21.15.154:0/1058372054 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc40072f0 con 0x7f2bc00047c0 2023-09-23T03:16:25.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.479+0000 7f2bdac83640 1 -- 172.21.15.154:0/1058372054 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd40b1b40 con 0x7f2bc00047c0 2023-09-23T03:16:25.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.479+0000 7f2bdac83640 1 -- 172.21.15.154:0/1058372054 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd40b2100 con 0x7f2bc00047c0 2023-09-23T03:16:25.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.480+0000 7f2bdac83640 1 -- 172.21.15.154:0/1058372054 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bd404e920 con 0x7f2bc00047c0 2023-09-23T03:16:25.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.481+0000 7f2bd17fa640 1 -- 172.21.15.154:0/1058372054 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f2bc4007450 con 0x7f2bc00047c0 2023-09-23T03:16:25.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.483+0000 7f2bd17fa640 1 --2- 172.21.15.154:0/1058372054 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2bb406af60 0x7f2bb406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:25.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.483+0000 7f2bd17fa640 1 -- 172.21.15.154:0/1058372054 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f2bc4089990 con 0x7f2bc00047c0 2023-09-23T03:16:25.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.483+0000 7f2bd3fff640 1 --2- 172.21.15.154:0/1058372054 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2bb406af60 0x7f2bb406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:25.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.484+0000 7f2bd3fff640 1 --2- 172.21.15.154:0/1058372054 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2bb406af60 0x7f2bb406d420 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc033000 tx=0x7f2bbc009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:25.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.487+0000 7f2bd17fa640 1 -- 172.21.15.154:0/1058372054 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f2bc4054620 con 0x7f2bc00047c0 2023-09-23T03:16:25.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.676+0000 7f2bdac83640 1 -- 172.21.15.154:0/1058372054 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2bd40cbdc0 con 0x7f2bc00047c0 2023-09-23T03:16:25.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.677+0000 7f2bd17fa640 1 -- 172.21.15.154:0/1058372054 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2bc40190a0 con 0x7f2bc00047c0 2023-09-23T03:16:25.678 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:16:25.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.679+0000 7f2bdac83640 1 -- 172.21.15.154:0/1058372054 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2bb406af60 msgr2=0x7f2bb406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:25.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.679+0000 7f2bdac83640 1 --2- 172.21.15.154:0/1058372054 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2bb406af60 0x7f2bb406d420 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc033000 tx=0x7f2bbc009030 comp rx=0 tx=0).stop 2023-09-23T03:16:25.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.679+0000 7f2bdac83640 1 -- 172.21.15.154:0/1058372054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bc00047c0 msgr2=0x7f2bd415ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:25.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.679+0000 7f2bdac83640 1 --2- 172.21.15.154:0/1058372054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bc00047c0 0x7f2bd415ce10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2bc400dcc0 tx=0x7f2bc400b4a0 comp rx=0 tx=0).stop 2023-09-23T03:16:25.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.680+0000 7f2bdac83640 1 -- 172.21.15.154:0/1058372054 shutdown_connections 2023-09-23T03:16:25.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.680+0000 7f2bdac83640 1 --2- 172.21.15.154:0/1058372054 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2bb406af60 0x7f2bb406d420 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:25.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.680+0000 7f2bdac83640 1 --2- 172.21.15.154:0/1058372054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2bd415f0c0 0x7f2bd415f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:25.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.680+0000 7f2bdac83640 1 --2- 172.21.15.154:0/1058372054 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bd40b3b70 0x7f2bd414f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:25.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.680+0000 7f2bdac83640 1 --2- 172.21.15.154:0/1058372054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bc00047c0 0x7f2bd415ce10 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:25.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.680+0000 7f2bdac83640 1 -- 172.21.15.154:0/1058372054 >> 172.21.15.154:0/1058372054 conn(0x7f2bd406c010 msgr2=0x7f2bd406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:25.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.680+0000 7f2bdac83640 1 -- 172.21.15.154:0/1058372054 shutdown_connections 2023-09-23T03:16:25.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.680+0000 7f2bdac83640 1 -- 172.21.15.154:0/1058372054 wait complete. 2023-09-23T03:16:25.698 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-23T03:16:25.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.906+0000 7f6566d97640 1 Processor -- start 2023-09-23T03:16:25.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.906+0000 7f6566d97640 1 -- start start 2023-09-23T03:16:25.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.907+0000 7f6566d97640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f656015bb30 0x7f65600caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:25.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.907+0000 7f6566d97640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f656015b6a0 con 0x7f656015aba0 2023-09-23T03:16:25.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.907+0000 7f6566d97640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f656015f0b0 con 0x7f65600cb3e0 2023-09-23T03:16:25.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.907+0000 7f6566d97640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65600cb870 con 0x7f656015bb30 2023-09-23T03:16:25.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.908+0000 7f6564b33640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f656015aba0 0x7f656015e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:52818/0 (socket says 172.21.15.154:52818) 2023-09-23T03:16:25.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.908+0000 7f655ffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f656015bb30 0x7f65600caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:25.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.908+0000 7f6564b33640 1 -- 172.21.15.154:0/1074408831 learned_addr learned my addr 172.21.15.154:0/1074408831 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:25.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.908+0000 7f655f7fe640 1 -- 172.21.15.154:0/1074408831 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3569560432 0 0) 0x7f656015b6a0 con 0x7f656015aba0 2023-09-23T03:16:25.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.908+0000 7f655f7fe640 1 -- 172.21.15.154:0/1074408831 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6548003520 con 0x7f656015aba0 2023-09-23T03:16:25.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.908+0000 7f655ffff640 1 -- 172.21.15.154:0/1074408831 >> v1:172.21.15.190:6789/0 conn(0x7f65600cb3e0 legacy=0x7f65600cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:25.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.908+0000 7f655ffff640 1 -- 172.21.15.154:0/1074408831 >> v1:172.21.15.154:6789/0 conn(0x7f656015aba0 legacy=0x7f656015e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:25.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.908+0000 7f655ffff640 1 -- 172.21.15.154:0/1074408831 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f656015f390 con 0x7f656015bb30 2023-09-23T03:16:25.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.910+0000 7f655ffff640 1 --2- 172.21.15.154:0/1074408831 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f656015bb30 0x7f65600caea0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6554009d90 tx=0x7f655402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d7b29932be0ef71 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:25.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.910+0000 7f655f7fe640 1 -- 172.21.15.154:0/1074408831 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655402cc60 con 0x7f656015bb30 2023-09-23T03:16:25.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.910+0000 7f655f7fe640 1 -- 172.21.15.154:0/1074408831 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f655402cdc0 con 0x7f656015bb30 2023-09-23T03:16:25.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.910+0000 7f655f7fe640 1 -- 172.21.15.154:0/1074408831 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6554034660 con 0x7f656015bb30 2023-09-23T03:16:25.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.910+0000 7f6566d97640 1 -- 172.21.15.154:0/1074408831 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f656015bb30 msgr2=0x7f65600caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:25.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.910+0000 7f6566d97640 1 --2- 172.21.15.154:0/1074408831 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f656015bb30 0x7f65600caea0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6554009d90 tx=0x7f655402beb0 comp rx=0 tx=0).stop 2023-09-23T03:16:25.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.911+0000 7f6566d97640 1 -- 172.21.15.154:0/1074408831 shutdown_connections 2023-09-23T03:16:25.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.911+0000 7f6566d97640 1 --2- 172.21.15.154:0/1074408831 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f656015bb30 0x7f65600caea0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:25.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.911+0000 7f6566d97640 1 -- 172.21.15.154:0/1074408831 >> 172.21.15.154:0/1074408831 conn(0x7f656006c010 msgr2=0x7f656006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:25.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.911+0000 7f6566d97640 1 -- 172.21.15.154:0/1074408831 shutdown_connections 2023-09-23T03:16:25.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.911+0000 7f6566d97640 1 -- 172.21.15.154:0/1074408831 wait complete. 2023-09-23T03:16:25.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.912+0000 7f6566d97640 1 Processor -- start 2023-09-23T03:16:25.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.913+0000 7f6566d97640 1 -- start start 2023-09-23T03:16:25.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.913+0000 7f6566d97640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65600cb3e0 0x7f65600b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:25.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.913+0000 7f6566d97640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f656015aba0 0x7f656015d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:25.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.914+0000 7f6566d97640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f656015bb30 0x7f656015d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:25.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.914+0000 7f6566d97640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65600c9c20 con 0x7f656015bb30 2023-09-23T03:16:25.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.914+0000 7f6566d97640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65600c9f00 con 0x7f65600cb3e0 2023-09-23T03:16:25.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.914+0000 7f6566d97640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65600ca1e0 con 0x7f656015aba0 2023-09-23T03:16:25.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.914+0000 7f6565334640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f656015bb30 0x7f656015d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:25.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.914+0000 7f6565334640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f656015bb30 0x7f656015d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46408/0 (socket says 172.21.15.154:46408) 2023-09-23T03:16:25.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.914+0000 7f6565334640 1 -- 172.21.15.154:0/250173484 learned_addr learned my addr 172.21.15.154:0/250173484 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:25.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.914+0000 7f6564b33640 1 --2- 172.21.15.154:0/250173484 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65600cb3e0 0x7f65600b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:25.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.914+0000 7f6565334640 1 -- 172.21.15.154:0/250173484 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f656015aba0 msgr2=0x7f656015d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:25.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.914+0000 7f655ffff640 1 --2- 172.21.15.154:0/250173484 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f656015aba0 0x7f656015d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:25.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.914+0000 7f6565334640 1 --2- 172.21.15.154:0/250173484 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f656015aba0 0x7f656015d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:25.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.914+0000 7f6565334640 1 -- 172.21.15.154:0/250173484 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65600cb3e0 msgr2=0x7f65600b36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:25.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.914+0000 7f6565334640 1 --2- 172.21.15.154:0/250173484 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65600cb3e0 0x7f65600b36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:25.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.914+0000 7f6565334640 1 -- 172.21.15.154:0/250173484 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6554009a70 con 0x7f656015bb30 2023-09-23T03:16:25.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.914+0000 7f6564b33640 1 --2- 172.21.15.154:0/250173484 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65600cb3e0 0x7f65600b36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:25.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.915+0000 7f6565334640 1 --2- 172.21.15.154:0/250173484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f656015bb30 0x7f656015d740 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f655800c410 tx=0x7f655800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:25.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.915+0000 7f655d7fa640 1 -- 172.21.15.154:0/250173484 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6558011790 con 0x7f656015bb30 2023-09-23T03:16:25.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.915+0000 7f655d7fa640 1 -- 172.21.15.154:0/250173484 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65580118f0 con 0x7f656015bb30 2023-09-23T03:16:25.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.915+0000 7f655d7fa640 1 -- 172.21.15.154:0/250173484 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6558011ce0 con 0x7f656015bb30 2023-09-23T03:16:25.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.915+0000 7f6566d97640 1 -- 172.21.15.154:0/250173484 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65600ca4c0 con 0x7f656015bb30 2023-09-23T03:16:25.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.915+0000 7f6566d97640 1 -- 172.21.15.154:0/250173484 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6560152f00 con 0x7f656015bb30 2023-09-23T03:16:25.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.916+0000 7f6566d97640 1 -- 172.21.15.154:0/250173484 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65600ca9e0 con 0x7f656015bb30 2023-09-23T03:16:25.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.917+0000 7f655d7fa640 1 -- 172.21.15.154:0/250173484 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f655801c460 con 0x7f656015bb30 2023-09-23T03:16:25.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.918+0000 7f655d7fa640 1 --2- 172.21.15.154:0/250173484 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f654c06af30 0x7f654c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:25.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.918+0000 7f655d7fa640 1 -- 172.21.15.154:0/250173484 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f6558089720 con 0x7f656015bb30 2023-09-23T03:16:25.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.918+0000 7f6564b33640 1 --2- 172.21.15.154:0/250173484 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f654c06af30 0x7f654c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:25.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.919+0000 7f6564b33640 1 --2- 172.21.15.154:0/250173484 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f654c06af30 0x7f654c06d3f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f65500011e0 tx=0x7f6550009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:25.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:25.922+0000 7f655d7fa640 1 -- 172.21.15.154:0/250173484 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f65600ca9e0 con 0x7f656015bb30 2023-09-23T03:16:26.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.109+0000 7f6566d97640 1 -- 172.21.15.154:0/250173484 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f65600ca9e0 con 0x7f656015bb30 2023-09-23T03:16:26.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.110+0000 7f655d7fa640 1 -- 172.21.15.154:0/250173484 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f65600ca9e0 con 0x7f656015bb30 2023-09-23T03:16:26.111 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:16:26.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.113+0000 7f6566d97640 1 -- 172.21.15.154:0/250173484 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f654c06af30 msgr2=0x7f654c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:26.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.113+0000 7f6566d97640 1 --2- 172.21.15.154:0/250173484 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f654c06af30 0x7f654c06d3f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f65500011e0 tx=0x7f6550009040 comp rx=0 tx=0).stop 2023-09-23T03:16:26.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.113+0000 7f6566d97640 1 -- 172.21.15.154:0/250173484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f656015bb30 msgr2=0x7f656015d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:26.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.113+0000 7f6566d97640 1 --2- 172.21.15.154:0/250173484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f656015bb30 0x7f656015d740 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f655800c410 tx=0x7f655800c8d0 comp rx=0 tx=0).stop 2023-09-23T03:16:26.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.113+0000 7f6566d97640 1 -- 172.21.15.154:0/250173484 shutdown_connections 2023-09-23T03:16:26.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.113+0000 7f6566d97640 1 --2- 172.21.15.154:0/250173484 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f654c06af30 0x7f654c06d3f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:26.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.113+0000 7f6566d97640 1 --2- 172.21.15.154:0/250173484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f656015bb30 0x7f656015d740 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:26.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.113+0000 7f6566d97640 1 --2- 172.21.15.154:0/250173484 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f656015aba0 0x7f656015d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:26.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.113+0000 7f6566d97640 1 --2- 172.21.15.154:0/250173484 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f65600cb3e0 0x7f65600b36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:26.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.113+0000 7f6566d97640 1 -- 172.21.15.154:0/250173484 >> 172.21.15.154:0/250173484 conn(0x7f656006c010 msgr2=0x7f6560159d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:26.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.113+0000 7f6566d97640 1 -- 172.21.15.154:0/250173484 shutdown_connections 2023-09-23T03:16:26.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.113+0000 7f6566d97640 1 -- 172.21.15.154:0/250173484 wait complete. 2023-09-23T03:16:26.146 INFO:tasks.cephfs_test_runner:Starting test: test_extra_data_pool_rebuild_few_files (tasks.cephfs.test_data_scan.TestDataScan) 2023-09-23T03:16:26.147 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:16:26.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.345+0000 7fbbc6821640 1 Processor -- start 2023-09-23T03:16:26.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.345+0000 7fbbc6821640 1 -- start start 2023-09-23T03:16:26.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.347+0000 7fbbc6821640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbc015caf0 0x7fbbc00ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:26.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.347+0000 7fbbc6821640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbbc0160f90 0x7fbbc00b1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:26.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.347+0000 7fbbc6821640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbc00cd530 con 0x7fbbc00cee70 2023-09-23T03:16:26.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.347+0000 7fbbc6821640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbc015ced0 con 0x7fbbc0160f90 2023-09-23T03:16:26.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.347+0000 7fbbc6821640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc00cf2f0 con 0x7fbbc015caf0 2023-09-23T03:16:26.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.347+0000 7fbbbffff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbc015caf0 0x7fbbc00ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:26.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.347+0000 7fbbc4dbe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbbc0160f90 0x7fbbc00b1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:26.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.347+0000 7fbbbffff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbc015caf0 0x7fbbc00ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:36240/0 (socket says 172.21.15.154:36240) 2023-09-23T03:16:26.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.347+0000 7fbbbffff640 1 -- 172.21.15.154:0/3637273408 learned_addr learned my addr 172.21.15.154:0/3637273408 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:26.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.348+0000 7fbbbffff640 1 -- 172.21.15.154:0/3637273408 >> v2:172.21.15.190:3300/0 conn(0x7fbbc0160f90 msgr2=0x7fbbc00b1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:26.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.348+0000 7fbbbffff640 1 --2- 172.21.15.154:0/3637273408 >> v2:172.21.15.190:3300/0 conn(0x7fbbc0160f90 0x7fbbc00b1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:26.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.348+0000 7fbbbffff640 1 -- 172.21.15.154:0/3637273408 >> v1:172.21.15.190:6789/0 conn(0x7fbbc00cee70 legacy=0x7fbbc0160880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:26.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.348+0000 7fbbbffff640 1 -- 172.21.15.154:0/3637273408 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc00cd810 con 0x7fbbc015caf0 2023-09-23T03:16:26.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.349+0000 7fbbbffff640 1 --2- 172.21.15.154:0/3637273408 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbc015caf0 0x7fbbc00ce930 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0009850 tx=0x7fbbb002b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5e60e0dadb3afc3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:26.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.350+0000 7fbbbf7fe640 1 -- 172.21.15.154:0/3637273408 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb0004230 con 0x7fbbc015caf0 2023-09-23T03:16:26.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.350+0000 7fbbbf7fe640 1 -- 172.21.15.154:0/3637273408 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb0004390 con 0x7fbbc015caf0 2023-09-23T03:16:26.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.350+0000 7fbbbf7fe640 1 -- 172.21.15.154:0/3637273408 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb0003810 con 0x7fbbc015caf0 2023-09-23T03:16:26.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.350+0000 7fbbc6821640 1 -- 172.21.15.154:0/3637273408 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbc015caf0 msgr2=0x7fbbc00ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:26.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.350+0000 7fbbc6821640 1 --2- 172.21.15.154:0/3637273408 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbc015caf0 0x7fbbc00ce930 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0009850 tx=0x7fbbb002b960 comp rx=0 tx=0).stop 2023-09-23T03:16:26.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.351+0000 7fbbc6821640 1 -- 172.21.15.154:0/3637273408 shutdown_connections 2023-09-23T03:16:26.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.351+0000 7fbbc6821640 1 --2- 172.21.15.154:0/3637273408 >> v2:172.21.15.190:3300/0 conn(0x7fbbc0160f90 0x7fbbc00b1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:26.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.351+0000 7fbbc6821640 1 --2- 172.21.15.154:0/3637273408 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbc015caf0 0x7fbbc00ce930 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:26.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.351+0000 7fbbc6821640 1 -- 172.21.15.154:0/3637273408 >> 172.21.15.154:0/3637273408 conn(0x7fbbc006a0f0 msgr2=0x7fbbc006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:26.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.351+0000 7fbbc6821640 1 -- 172.21.15.154:0/3637273408 shutdown_connections 2023-09-23T03:16:26.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.351+0000 7fbbc6821640 1 -- 172.21.15.154:0/3637273408 wait complete. 2023-09-23T03:16:26.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.353+0000 7fbbc6821640 1 Processor -- start 2023-09-23T03:16:26.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.353+0000 7fbbc6821640 1 -- start start 2023-09-23T03:16:26.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.354+0000 7fbbc6821640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbbc00cee70 0x7fbbc0157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:26.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.354+0000 7fbbc6821640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbbc015caf0 0x7fbbc01f1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:26.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.354+0000 7fbbc6821640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbc0160f90 0x7fbbc01f2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:26.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.354+0000 7fbbc6821640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc015ee10 con 0x7fbbc015caf0 2023-09-23T03:16:26.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.354+0000 7fbbc6821640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc015f0f0 con 0x7fbbc00cee70 2023-09-23T03:16:26.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.354+0000 7fbbc6821640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc015f3d0 con 0x7fbbc0160f90 2023-09-23T03:16:26.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.354+0000 7fbbb7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbbc015caf0 0x7fbbc01f1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:26.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.354+0000 7fbbb7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbbc015caf0 0x7fbbc01f1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46420/0 (socket says 172.21.15.154:46420) 2023-09-23T03:16:26.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.354+0000 7fbbb7fff640 1 -- 172.21.15.154:0/1733477425 learned_addr learned my addr 172.21.15.154:0/1733477425 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:26.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.355+0000 7fbbc4dbe640 1 --2- 172.21.15.154:0/1733477425 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbc0160f90 0x7fbbc01f2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:26.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.355+0000 7fbbbffff640 1 --2- 172.21.15.154:0/1733477425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbbc00cee70 0x7fbbc0157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:26.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.355+0000 7fbbb7fff640 1 -- 172.21.15.154:0/1733477425 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbc0160f90 msgr2=0x7fbbc01f2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:26.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.355+0000 7fbbb7fff640 1 --2- 172.21.15.154:0/1733477425 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbc0160f90 0x7fbbc01f2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:26.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.355+0000 7fbbb7fff640 1 -- 172.21.15.154:0/1733477425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbbc00cee70 msgr2=0x7fbbc0157eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:26.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.355+0000 7fbbb7fff640 1 --2- 172.21.15.154:0/1733477425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbbc00cee70 0x7fbbc0157eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:26.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.355+0000 7fbbb7fff640 1 -- 172.21.15.154:0/1733477425 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb0009460 con 0x7fbbc015caf0 2023-09-23T03:16:26.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.355+0000 7fbbc4dbe640 1 --2- 172.21.15.154:0/1733477425 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbc0160f90 0x7fbbc01f2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:26.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.355+0000 7fbbb7fff640 1 --2- 172.21.15.154:0/1733477425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbbc015caf0 0x7fbbc01f1c50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbba800c450 tx=0x7fbba800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:26.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.355+0000 7fbbbd7fa640 1 -- 172.21.15.154:0/1733477425 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba80052d0 con 0x7fbbc015caf0 2023-09-23T03:16:26.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.355+0000 7fbbbd7fa640 1 -- 172.21.15.154:0/1733477425 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba8005430 con 0x7fbbc015caf0 2023-09-23T03:16:26.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.355+0000 7fbbbd7fa640 1 -- 172.21.15.154:0/1733477425 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba8012870 con 0x7fbbc015caf0 2023-09-23T03:16:26.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.355+0000 7fbbc6821640 1 -- 172.21.15.154:0/1733477425 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbc015f6b0 con 0x7fbbc015caf0 2023-09-23T03:16:26.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.356+0000 7fbbc6821640 1 -- 172.21.15.154:0/1733477425 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbc015eaa0 con 0x7fbbc015caf0 2023-09-23T03:16:26.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.357+0000 7fbbc6821640 1 -- 172.21.15.154:0/1733477425 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbc015fbd0 con 0x7fbbc015caf0 2023-09-23T03:16:26.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.358+0000 7fbbbd7fa640 1 -- 172.21.15.154:0/1733477425 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fbba80129d0 con 0x7fbbc015caf0 2023-09-23T03:16:26.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.358+0000 7fbbbd7fa640 1 --2- 172.21.15.154:0/1733477425 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbba006af30 0x7fbba006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:26.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.359+0000 7fbbbd7fa640 1 -- 172.21.15.154:0/1733477425 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7fbba8089680 con 0x7fbbc015caf0 2023-09-23T03:16:26.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.359+0000 7fbbbffff640 1 --2- 172.21.15.154:0/1733477425 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbba006af30 0x7fbba006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:26.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.359+0000 7fbbbffff640 1 --2- 172.21.15.154:0/1733477425 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbba006af30 0x7fbba006d3f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0035000 tx=0x7fbbb0008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:26.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.364+0000 7fbbbd7fa640 1 -- 172.21.15.154:0/1733477425 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fbbc015fbd0 con 0x7fbbc015caf0 2023-09-23T03:16:26.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.541+0000 7fbbc6821640 1 -- 172.21.15.154:0/1733477425 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbbc015fbd0 con 0x7fbbc015caf0 2023-09-23T03:16:26.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.543+0000 7fbbbd7fa640 1 -- 172.21.15.154:0/1733477425 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18744 (secure 0 0 0) 0x7fbbc015fbd0 con 0x7fbbc015caf0 2023-09-23T03:16:26.545 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:16:26.545 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":26,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:16:03.151371+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:15:18.575849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T03:15:19.707212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T03:15:23.337138+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:16:26.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.545+0000 7fbbc6821640 1 -- 172.21.15.154:0/1733477425 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbba006af30 msgr2=0x7fbba006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:26.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.545+0000 7fbbc6821640 1 --2- 172.21.15.154:0/1733477425 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbba006af30 0x7fbba006d3f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0035000 tx=0x7fbbb0008030 comp rx=0 tx=0).stop 2023-09-23T03:16:26.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.545+0000 7fbbc6821640 1 -- 172.21.15.154:0/1733477425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbbc015caf0 msgr2=0x7fbbc01f1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:26.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.545+0000 7fbbc6821640 1 --2- 172.21.15.154:0/1733477425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbbc015caf0 0x7fbbc01f1c50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbba800c450 tx=0x7fbba800c910 comp rx=0 tx=0).stop 2023-09-23T03:16:26.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.545+0000 7fbbc6821640 1 -- 172.21.15.154:0/1733477425 shutdown_connections 2023-09-23T03:16:26.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.545+0000 7fbbc6821640 1 --2- 172.21.15.154:0/1733477425 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbba006af30 0x7fbba006d3f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:26.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.546+0000 7fbbc6821640 1 --2- 172.21.15.154:0/1733477425 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbbc0160f90 0x7fbbc01f2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:26.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.546+0000 7fbbc6821640 1 --2- 172.21.15.154:0/1733477425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbbc015caf0 0x7fbbc01f1c50 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:26.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.546+0000 7fbbc6821640 1 --2- 172.21.15.154:0/1733477425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbbc00cee70 0x7fbbc0157eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:26.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.546+0000 7fbbc6821640 1 -- 172.21.15.154:0/1733477425 >> 172.21.15.154:0/1733477425 conn(0x7fbbc006a0f0 msgr2=0x7fbbc00ac640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:26.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.546+0000 7fbbc6821640 1 -- 172.21.15.154:0/1733477425 shutdown_connections 2023-09-23T03:16:26.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.546+0000 7fbbc6821640 1 -- 172.21.15.154:0/1733477425 wait complete. 2023-09-23T03:16:26.566 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:15:18.575849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:15:19.707212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-23T03:15:23.337138+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-23T03:16:26.566 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:16:26.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.783+0000 7ffb44021640 1 Processor -- start 2023-09-23T03:16:26.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.783+0000 7ffb44021640 1 -- start start 2023-09-23T03:16:26.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.784+0000 7ffb44021640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ffb3c0cb650 0x7ffb3c15c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:26.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.784+0000 7ffb44021640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ffb3c071770 0x7ffb3c0c97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:26.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.784+0000 7ffb44021640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb3c0ca140 con 0x7ffb3c0c9cf0 2023-09-23T03:16:26.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.785+0000 7ffb44021640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb3c0709f0 con 0x7ffb3c0cb650 2023-09-23T03:16:26.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.785+0000 7ffb44021640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb3c0c8ee0 con 0x7ffb3c071770 2023-09-23T03:16:26.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.785+0000 7ffb41dbd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ffb3c0cb650 0x7ffb3c15c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:26.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.785+0000 7ffb41dbd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ffb3c0cb650 0x7ffb3c15c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46432/0 (socket says 172.21.15.154:46432) 2023-09-23T03:16:26.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.785+0000 7ffb41dbd640 1 -- 172.21.15.154:0/1950492035 learned_addr learned my addr 172.21.15.154:0/1950492035 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:26.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.785+0000 7ffb415bc640 1 --2- 172.21.15.154:0/1950492035 >> v2:172.21.15.190:3300/0 conn(0x7ffb3c071770 0x7ffb3c0c97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:26.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.785+0000 7ffb41dbd640 1 -- 172.21.15.154:0/1950492035 >> v2:172.21.15.190:3300/0 conn(0x7ffb3c071770 msgr2=0x7ffb3c0c97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:26.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.785+0000 7ffb41dbd640 1 --2- 172.21.15.154:0/1950492035 >> v2:172.21.15.190:3300/0 conn(0x7ffb3c071770 0x7ffb3c0c97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:26.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.785+0000 7ffb41dbd640 1 -- 172.21.15.154:0/1950492035 >> v1:172.21.15.190:6789/0 conn(0x7ffb3c0c9cf0 legacy=0x7ffb3c0cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:26.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.785+0000 7ffb41dbd640 1 -- 172.21.15.154:0/1950492035 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb3c070c90 con 0x7ffb3c0cb650 2023-09-23T03:16:26.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.786+0000 7ffb41dbd640 1 --2- 172.21.15.154:0/1950492035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb3c0cb650 0x7ffb3c15c5c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009870 tx=0x7ffb3802db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0ca9cb328c396dd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:26.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.787+0000 7ffb40dbb640 1 -- 172.21.15.154:0/1950492035 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3802f8e0 con 0x7ffb3c0cb650 2023-09-23T03:16:26.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.787+0000 7ffb40dbb640 1 -- 172.21.15.154:0/1950492035 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb3802fa40 con 0x7ffb3c0cb650 2023-09-23T03:16:26.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.787+0000 7ffb40dbb640 1 -- 172.21.15.154:0/1950492035 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3802fdd0 con 0x7ffb3c0cb650 2023-09-23T03:16:26.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.787+0000 7ffb44021640 1 -- 172.21.15.154:0/1950492035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb3c0cb650 msgr2=0x7ffb3c15c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:26.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.787+0000 7ffb44021640 1 --2- 172.21.15.154:0/1950492035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb3c0cb650 0x7ffb3c15c5c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009870 tx=0x7ffb3802db20 comp rx=0 tx=0).stop 2023-09-23T03:16:26.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.788+0000 7ffb44021640 1 -- 172.21.15.154:0/1950492035 shutdown_connections 2023-09-23T03:16:26.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.788+0000 7ffb44021640 1 --2- 172.21.15.154:0/1950492035 >> v2:172.21.15.190:3300/0 conn(0x7ffb3c071770 0x7ffb3c0c97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:26.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.788+0000 7ffb44021640 1 --2- 172.21.15.154:0/1950492035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb3c0cb650 0x7ffb3c15c5c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:26.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.788+0000 7ffb44021640 1 -- 172.21.15.154:0/1950492035 >> 172.21.15.154:0/1950492035 conn(0x7ffb3c06c010 msgr2=0x7ffb3c06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:26.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.788+0000 7ffb44021640 1 -- 172.21.15.154:0/1950492035 shutdown_connections 2023-09-23T03:16:26.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.788+0000 7ffb44021640 1 -- 172.21.15.154:0/1950492035 wait complete. 2023-09-23T03:16:26.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.790+0000 7ffb44021640 1 Processor -- start 2023-09-23T03:16:26.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.790+0000 7ffb44021640 1 -- start start 2023-09-23T03:16:26.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.791+0000 7ffb44021640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb3c071770 0x7ffb3c14c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:26.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.792+0000 7ffb44021640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb3c0c9cf0 0x7ffb3c14d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:26.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.792+0000 7ffb44021640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb3c0cb650 0x7ffb3c0cea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:26.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.792+0000 7ffb44021640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3c14fe30 con 0x7ffb3c071770 2023-09-23T03:16:26.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.792+0000 7ffb44021640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3c150110 con 0x7ffb3c0c9cf0 2023-09-23T03:16:26.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.792+0000 7ffb44021640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3c1503f0 con 0x7ffb3c0cb650 2023-09-23T03:16:26.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.792+0000 7ffb41dbd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb3c071770 0x7ffb3c14c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:26.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.792+0000 7ffb41dbd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb3c071770 0x7ffb3c14c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46448/0 (socket says 172.21.15.154:46448) 2023-09-23T03:16:26.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.792+0000 7ffb41dbd640 1 -- 172.21.15.154:0/597440829 learned_addr learned my addr 172.21.15.154:0/597440829 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:26.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.792+0000 7ffb41dbd640 1 -- 172.21.15.154:0/597440829 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb3c0cb650 msgr2=0x7ffb3c0cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:26.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.792+0000 7ffb41dbd640 1 --2- 172.21.15.154:0/597440829 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb3c0cb650 0x7ffb3c0cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:26.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.792+0000 7ffb41dbd640 1 -- 172.21.15.154:0/597440829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb3c0c9cf0 msgr2=0x7ffb3c14d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:26.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.792+0000 7ffb41dbd640 1 --2- 172.21.15.154:0/597440829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb3c0c9cf0 0x7ffb3c14d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:26.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.792+0000 7ffb41dbd640 1 -- 172.21.15.154:0/597440829 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb380094d0 con 0x7ffb3c071770 2023-09-23T03:16:26.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.793+0000 7ffb41dbd640 1 --2- 172.21.15.154:0/597440829 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb3c071770 0x7ffb3c14c800 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ffb38000c00 tx=0x7ffb380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:26.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.793+0000 7ffb327fc640 1 -- 172.21.15.154:0/597440829 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb38037df0 con 0x7ffb3c071770 2023-09-23T03:16:26.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.793+0000 7ffb327fc640 1 -- 172.21.15.154:0/597440829 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb38042470 con 0x7ffb3c071770 2023-09-23T03:16:26.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.793+0000 7ffb327fc640 1 -- 172.21.15.154:0/597440829 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb38042820 con 0x7ffb3c071770 2023-09-23T03:16:26.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.793+0000 7ffb44021640 1 -- 172.21.15.154:0/597440829 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb3c1506d0 con 0x7ffb3c071770 2023-09-23T03:16:26.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.793+0000 7ffb44021640 1 -- 172.21.15.154:0/597440829 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb3c14d150 con 0x7ffb3c071770 2023-09-23T03:16:26.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.794+0000 7ffb44021640 1 -- 172.21.15.154:0/597440829 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb3c14d2e0 con 0x7ffb3c071770 2023-09-23T03:16:26.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.795+0000 7ffb327fc640 1 -- 172.21.15.154:0/597440829 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7ffb38042980 con 0x7ffb3c071770 2023-09-23T03:16:26.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.796+0000 7ffb327fc640 1 --2- 172.21.15.154:0/597440829 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffb2006af30 0x7ffb2006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:26.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.796+0000 7ffb327fc640 1 -- 172.21.15.154:0/597440829 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7ffb38033070 con 0x7ffb3c071770 2023-09-23T03:16:26.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.796+0000 7ffb415bc640 1 --2- 172.21.15.154:0/597440829 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffb2006af30 0x7ffb2006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:26.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.797+0000 7ffb415bc640 1 --2- 172.21.15.154:0/597440829 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffb2006af30 0x7ffb2006d3f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c0ce270 tx=0x7ffb2c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:26.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.801+0000 7ffb327fc640 1 -- 172.21.15.154:0/597440829 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7ffb380778e0 con 0x7ffb3c071770 2023-09-23T03:16:26.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.990+0000 7ffb44021640 1 -- 172.21.15.154:0/597440829 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffb3c0ca140 con 0x7ffb3c071770 2023-09-23T03:16:26.992 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.991+0000 7ffb327fc640 1 -- 172.21.15.154:0/597440829 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffb3c0ca140 con 0x7ffb3c071770 2023-09-23T03:16:26.992 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:16:26.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.993+0000 7ffb44021640 1 -- 172.21.15.154:0/597440829 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffb2006af30 msgr2=0x7ffb2006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:26.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.993+0000 7ffb44021640 1 --2- 172.21.15.154:0/597440829 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffb2006af30 0x7ffb2006d3f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c0ce270 tx=0x7ffb2c008040 comp rx=0 tx=0).stop 2023-09-23T03:16:26.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.993+0000 7ffb44021640 1 -- 172.21.15.154:0/597440829 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb3c071770 msgr2=0x7ffb3c14c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:26.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.993+0000 7ffb44021640 1 --2- 172.21.15.154:0/597440829 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb3c071770 0x7ffb3c14c800 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ffb38000c00 tx=0x7ffb380046c0 comp rx=0 tx=0).stop 2023-09-23T03:16:26.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.993+0000 7ffb44021640 1 -- 172.21.15.154:0/597440829 shutdown_connections 2023-09-23T03:16:26.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.993+0000 7ffb44021640 1 --2- 172.21.15.154:0/597440829 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffb2006af30 0x7ffb2006d3f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:26.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.993+0000 7ffb44021640 1 --2- 172.21.15.154:0/597440829 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb3c0cb650 0x7ffb3c0cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:26.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.993+0000 7ffb44021640 1 --2- 172.21.15.154:0/597440829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb3c0c9cf0 0x7ffb3c14d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:26.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.993+0000 7ffb44021640 1 --2- 172.21.15.154:0/597440829 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb3c071770 0x7ffb3c14c800 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:26.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.993+0000 7ffb44021640 1 -- 172.21.15.154:0/597440829 >> 172.21.15.154:0/597440829 conn(0x7ffb3c06c010 msgr2=0x7ffb3c0cac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:26.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.994+0000 7ffb44021640 1 -- 172.21.15.154:0/597440829 shutdown_connections 2023-09-23T03:16:26.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:26.994+0000 7ffb44021640 1 -- 172.21.15.154:0/597440829 wait complete. 2023-09-23T03:16:27.012 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:16:27.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.232+0000 7f0700eda640 1 Processor -- start 2023-09-23T03:16:27.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.232+0000 7f0700eda640 1 -- start start 2023-09-23T03:16:27.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.233+0000 7f0700eda640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06fc070c60 0x7f06fc0caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:27.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.233+0000 7f0700eda640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f06fc0cb400 0x7f06fc0aba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:27.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.233+0000 7f0700eda640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06fc0710c0 con 0x7f06fc0ad830 2023-09-23T03:16:27.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.233+0000 7f0700eda640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06fc0cb890 con 0x7f06fc0cb400 2023-09-23T03:16:27.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.233+0000 7f0700eda640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc0712f0 con 0x7f06fc070c60 2023-09-23T03:16:27.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.234+0000 7f06fad9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f06fc0cb400 0x7f06fc0aba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:27.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.234+0000 7f06fad9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f06fc0cb400 0x7f06fc0aba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46462/0 (socket says 172.21.15.154:46462) 2023-09-23T03:16:27.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.234+0000 7f06fad9d640 1 -- 172.21.15.154:0/1314749712 learned_addr learned my addr 172.21.15.154:0/1314749712 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:27.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.234+0000 7f06f3fff640 1 --2- 172.21.15.154:0/1314749712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06fc070c60 0x7f06fc0caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:27.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.234+0000 7f06fad9d640 1 -- 172.21.15.154:0/1314749712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06fc070c60 msgr2=0x7f06fc0caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:27.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.234+0000 7f06fad9d640 1 --2- 172.21.15.154:0/1314749712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06fc070c60 0x7f06fc0caec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:27.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.234+0000 7f06fad9d640 1 -- 172.21.15.154:0/1314749712 >> v1:172.21.15.190:6789/0 conn(0x7f06fc0ad830 legacy=0x7f06fc0c9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:27.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.234+0000 7f06fad9d640 1 -- 172.21.15.154:0/1314749712 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06fc0715d0 con 0x7f06fc0cb400 2023-09-23T03:16:27.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.235+0000 7f06fad9d640 1 --2- 172.21.15.154:0/1314749712 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06fc0cb400 0x7f06fc0aba70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009e50 tx=0x7f06f402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3665148cf2bcd2da server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:27.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.236+0000 7f06f9d9b640 1 -- 172.21.15.154:0/1314749712 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f402ca70 con 0x7f06fc0cb400 2023-09-23T03:16:27.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.236+0000 7f06f9d9b640 1 -- 172.21.15.154:0/1314749712 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f402cbd0 con 0x7f06fc0cb400 2023-09-23T03:16:27.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.236+0000 7f06f9d9b640 1 -- 172.21.15.154:0/1314749712 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f40343f0 con 0x7f06fc0cb400 2023-09-23T03:16:27.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.236+0000 7f0700eda640 1 -- 172.21.15.154:0/1314749712 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06fc0cb400 msgr2=0x7f06fc0aba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:27.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.236+0000 7f0700eda640 1 --2- 172.21.15.154:0/1314749712 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06fc0cb400 0x7f06fc0aba70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009e50 tx=0x7f06f402bed0 comp rx=0 tx=0).stop 2023-09-23T03:16:27.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.236+0000 7f0700eda640 1 -- 172.21.15.154:0/1314749712 shutdown_connections 2023-09-23T03:16:27.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.236+0000 7f0700eda640 1 --2- 172.21.15.154:0/1314749712 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06fc0cb400 0x7f06fc0aba70 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:27.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.236+0000 7f0700eda640 1 --2- 172.21.15.154:0/1314749712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06fc070c60 0x7f06fc0caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:27.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.236+0000 7f0700eda640 1 -- 172.21.15.154:0/1314749712 >> 172.21.15.154:0/1314749712 conn(0x7f06fc06c010 msgr2=0x7f06fc06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:27.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.236+0000 7f0700eda640 1 -- 172.21.15.154:0/1314749712 shutdown_connections 2023-09-23T03:16:27.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.237+0000 7f0700eda640 1 -- 172.21.15.154:0/1314749712 wait complete. 2023-09-23T03:16:27.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.238+0000 7f0700eda640 1 Processor -- start 2023-09-23T03:16:27.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.238+0000 7f0700eda640 1 -- start start 2023-09-23T03:16:27.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.239+0000 7f0700eda640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06fc070c60 0x7f06fc14d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:27.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.239+0000 7f0700eda640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06fc0ad830 0x7f06fc0b3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:27.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.239+0000 7f0700eda640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06fc0cb400 0x7f06fc0b1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:27.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.239+0000 7f0700eda640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc14ded0 con 0x7f06fc0ad830 2023-09-23T03:16:27.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.239+0000 7f0700eda640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc158370 con 0x7f06fc070c60 2023-09-23T03:16:27.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.239+0000 7f0700eda640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc0c2080 con 0x7f06fc0cb400 2023-09-23T03:16:27.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.240+0000 7f06f3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06fc0ad830 0x7f06fc0b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:27.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.240+0000 7f06f3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06fc0ad830 0x7f06fc0b3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46464/0 (socket says 172.21.15.154:46464) 2023-09-23T03:16:27.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.240+0000 7f06f3fff640 1 -- 172.21.15.154:0/3070748297 learned_addr learned my addr 172.21.15.154:0/3070748297 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:27.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.240+0000 7f06fa59c640 1 --2- 172.21.15.154:0/3070748297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06fc070c60 0x7f06fc14d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:27.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.240+0000 7f06fad9d640 1 --2- 172.21.15.154:0/3070748297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06fc0cb400 0x7f06fc0b1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:27.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.240+0000 7f06f3fff640 1 -- 172.21.15.154:0/3070748297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06fc0cb400 msgr2=0x7f06fc0b1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:27.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.240+0000 7f06f3fff640 1 --2- 172.21.15.154:0/3070748297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06fc0cb400 0x7f06fc0b1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:27.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.240+0000 7f06f3fff640 1 -- 172.21.15.154:0/3070748297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06fc070c60 msgr2=0x7f06fc14d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:27.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.240+0000 7f06f3fff640 1 --2- 172.21.15.154:0/3070748297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06fc070c60 0x7f06fc14d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:27.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.240+0000 7f06f3fff640 1 -- 172.21.15.154:0/3070748297 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f4009b30 con 0x7f06fc0ad830 2023-09-23T03:16:27.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.240+0000 7f06f3fff640 1 --2- 172.21.15.154:0/3070748297 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06fc0ad830 0x7f06fc0b3880 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f06e400c7f0 tx=0x7f06e400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:27.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.241+0000 7f06f2ffd640 1 -- 172.21.15.154:0/3070748297 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e4004220 con 0x7f06fc0ad830 2023-09-23T03:16:27.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.241+0000 7f06f2ffd640 1 -- 172.21.15.154:0/3070748297 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06e4004380 con 0x7f06fc0ad830 2023-09-23T03:16:27.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.241+0000 7f0700eda640 1 -- 172.21.15.154:0/3070748297 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06fc0c2360 con 0x7f06fc0ad830 2023-09-23T03:16:27.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.241+0000 7f06f2ffd640 1 -- 172.21.15.154:0/3070748297 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e4007840 con 0x7f06fc0ad830 2023-09-23T03:16:27.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.241+0000 7f0700eda640 1 -- 172.21.15.154:0/3070748297 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06fc0c2920 con 0x7f06fc0ad830 2023-09-23T03:16:27.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.242+0000 7f0700eda640 1 -- 172.21.15.154:0/3070748297 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06fc04e920 con 0x7f06fc0ad830 2023-09-23T03:16:27.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.243+0000 7f06f2ffd640 1 -- 172.21.15.154:0/3070748297 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f06e4007bb0 con 0x7f06fc0ad830 2023-09-23T03:16:27.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.244+0000 7f06f2ffd640 1 --2- 172.21.15.154:0/3070748297 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f06dc06af30 0x7f06dc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:27.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.244+0000 7f06f2ffd640 1 -- 172.21.15.154:0/3070748297 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f06e4089a70 con 0x7f06fc0ad830 2023-09-23T03:16:27.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.244+0000 7f06fa59c640 1 --2- 172.21.15.154:0/3070748297 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f06dc06af30 0x7f06dc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:27.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.245+0000 7f06fa59c640 1 --2- 172.21.15.154:0/3070748297 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f06dc06af30 0x7f06dc06d3f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f06ec006fd0 tx=0x7f06ec008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:27.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.248+0000 7f06f2ffd640 1 -- 172.21.15.154:0/3070748297 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f06e40546f0 con 0x7f06fc0ad830 2023-09-23T03:16:27.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.442+0000 7f0700eda640 1 -- 172.21.15.154:0/3070748297 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f06fc0c8870 con 0x7f06fc0ad830 2023-09-23T03:16:27.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.443+0000 7f06f2ffd640 1 -- 172.21.15.154:0/3070748297 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f06e401e020 con 0x7f06fc0ad830 2023-09-23T03:16:27.444 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:16:27.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.446+0000 7f0700eda640 1 -- 172.21.15.154:0/3070748297 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f06dc06af30 msgr2=0x7f06dc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:27.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.446+0000 7f0700eda640 1 --2- 172.21.15.154:0/3070748297 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f06dc06af30 0x7f06dc06d3f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f06ec006fd0 tx=0x7f06ec008040 comp rx=0 tx=0).stop 2023-09-23T03:16:27.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.446+0000 7f0700eda640 1 -- 172.21.15.154:0/3070748297 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06fc0ad830 msgr2=0x7f06fc0b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:27.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.446+0000 7f0700eda640 1 --2- 172.21.15.154:0/3070748297 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06fc0ad830 0x7f06fc0b3880 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f06e400c7f0 tx=0x7f06e400ccb0 comp rx=0 tx=0).stop 2023-09-23T03:16:27.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.446+0000 7f0700eda640 1 -- 172.21.15.154:0/3070748297 shutdown_connections 2023-09-23T03:16:27.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.446+0000 7f0700eda640 1 --2- 172.21.15.154:0/3070748297 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f06dc06af30 0x7f06dc06d3f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:27.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.446+0000 7f0700eda640 1 --2- 172.21.15.154:0/3070748297 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06fc0cb400 0x7f06fc0b1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:27.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.446+0000 7f0700eda640 1 --2- 172.21.15.154:0/3070748297 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06fc0ad830 0x7f06fc0b3880 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:27.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.446+0000 7f0700eda640 1 --2- 172.21.15.154:0/3070748297 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06fc070c60 0x7f06fc14d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:27.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.446+0000 7f0700eda640 1 -- 172.21.15.154:0/3070748297 >> 172.21.15.154:0/3070748297 conn(0x7f06fc06c010 msgr2=0x7f06fc0af230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:27.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.446+0000 7f0700eda640 1 -- 172.21.15.154:0/3070748297 shutdown_connections 2023-09-23T03:16:27.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.446+0000 7f0700eda640 1 -- 172.21.15.154:0/3070748297 wait complete. 2023-09-23T03:16:27.467 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:16:27.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.681+0000 7f2666aad640 1 Processor -- start 2023-09-23T03:16:27.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.682+0000 7f2666aad640 1 -- start start 2023-09-23T03:16:27.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.683+0000 7f2666aad640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f266015ba50 0x7f266006fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:27.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.683+0000 7f2666aad640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f266015eef0 0x7f26600703d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:27.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.683+0000 7f2666aad640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26600cb6c0 con 0x7f2660158480 2023-09-23T03:16:27.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.683+0000 7f2666aad640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f266015ad80 con 0x7f266015ba50 2023-09-23T03:16:27.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.683+0000 7f2666aad640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f266015af70 con 0x7f266015eef0 2023-09-23T03:16:27.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.683+0000 7f2665aab640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f2660158480 0x7f26600cade0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:52826/0 (socket says 172.21.15.154:52826) 2023-09-23T03:16:27.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.683+0000 7f2665aab640 1 -- 172.21.15.154:0/940793803 learned_addr learned my addr 172.21.15.154:0/940793803 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:27.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.684+0000 7f26652aa640 1 --2- 172.21.15.154:0/940793803 >> v2:172.21.15.190:3300/0 conn(0x7f266015ba50 0x7f266006fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:27.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.684+0000 7f2664aa9640 1 -- 172.21.15.154:0/940793803 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1353687122 0 0) 0x7f26600cb6c0 con 0x7f2660158480 2023-09-23T03:16:27.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.684+0000 7f2664aa9640 1 -- 172.21.15.154:0/940793803 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f264c003520 con 0x7f2660158480 2023-09-23T03:16:27.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.684+0000 7f26662ac640 1 --2- 172.21.15.154:0/940793803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f266015eef0 0x7f26600703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:27.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.684+0000 7f2664aa9640 1 -- 172.21.15.154:0/940793803 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1542121783 0 0) 0x7f264c003520 con 0x7f2660158480 2023-09-23T03:16:27.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.684+0000 7f2664aa9640 1 -- 172.21.15.154:0/940793803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f266015eef0 msgr2=0x7f26600703d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:27.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.684+0000 7f2664aa9640 1 --2- 172.21.15.154:0/940793803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f266015eef0 0x7f26600703d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:27.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.684+0000 7f2664aa9640 1 -- 172.21.15.154:0/940793803 >> v2:172.21.15.190:3300/0 conn(0x7f266015ba50 msgr2=0x7f266006fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:27.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.684+0000 7f2664aa9640 1 --2- 172.21.15.154:0/940793803 >> v2:172.21.15.190:3300/0 conn(0x7f266015ba50 0x7f266006fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:27.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.684+0000 7f2664aa9640 1 -- 172.21.15.154:0/940793803 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f266015b250 con 0x7f2660158480 2023-09-23T03:16:27.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.684+0000 7f2664aa9640 1 -- 172.21.15.154:0/940793803 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f265c001690 con 0x7f2660158480 2023-09-23T03:16:27.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.684+0000 7f2664aa9640 1 -- 172.21.15.154:0/940793803 >> v1:172.21.15.154:6789/0 conn(0x7f2660158480 legacy=0x7f26600cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:27.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.684+0000 7f2664aa9640 1 --2- 172.21.15.154:0/940793803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f264c003970 0x7f264c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:27.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.684+0000 7f2664aa9640 1 --2- 172.21.15.154:0/940793803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f266015eef0 0x7f264c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:27.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.684+0000 7f2664aa9640 1 --2- 172.21.15.154:0/940793803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f264c004d00 0x7f264c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:27.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.685+0000 7f2664aa9640 1 -- 172.21.15.154:0/940793803 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266015af70 con 0x7f264c004d00 2023-09-23T03:16:27.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.685+0000 7f2664aa9640 1 -- 172.21.15.154:0/940793803 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264c00b980 con 0x7f264c003970 2023-09-23T03:16:27.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.685+0000 7f2664aa9640 1 -- 172.21.15.154:0/940793803 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f264c00bac0 con 0x7f266015eef0 2023-09-23T03:16:27.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.685+0000 7f26662ac640 1 --2- 172.21.15.154:0/940793803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f264c004d00 0x7f264c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:27.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.685+0000 7f2665aab640 1 --2- 172.21.15.154:0/940793803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f264c003970 0x7f264c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:27.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.685+0000 7f26662ac640 1 -- 172.21.15.154:0/940793803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f266015eef0 msgr2=0x7f264c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:27.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.685+0000 7f26652aa640 1 --2- 172.21.15.154:0/940793803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f266015eef0 0x7f264c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:27.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.685+0000 7f26662ac640 1 --2- 172.21.15.154:0/940793803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f266015eef0 0x7f264c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:27.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.685+0000 7f26662ac640 1 -- 172.21.15.154:0/940793803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f264c003970 msgr2=0x7f264c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:27.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.685+0000 7f26662ac640 1 --2- 172.21.15.154:0/940793803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f264c003970 0x7f264c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:27.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.685+0000 7f26662ac640 1 -- 172.21.15.154:0/940793803 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f264c003520 con 0x7f264c004d00 2023-09-23T03:16:27.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.685+0000 7f2665aab640 1 --2- 172.21.15.154:0/940793803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f264c003970 0x7f264c003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:27.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.686+0000 7f26662ac640 1 --2- 172.21.15.154:0/940793803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f264c004d00 0x7f264c005180 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2658007390 tx=0x7f264c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88be20cad1bf71d7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:27.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.687+0000 7f2664aa9640 1 -- 172.21.15.154:0/940793803 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265802ccb0 con 0x7f264c004d00 2023-09-23T03:16:27.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.687+0000 7f2664aa9640 1 -- 172.21.15.154:0/940793803 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f265802ce10 con 0x7f264c004d00 2023-09-23T03:16:27.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.687+0000 7f2664aa9640 1 -- 172.21.15.154:0/940793803 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2658034660 con 0x7f264c004d00 2023-09-23T03:16:27.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.687+0000 7f2666aad640 1 -- 172.21.15.154:0/940793803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f264c004d00 msgr2=0x7f264c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:27.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.687+0000 7f2666aad640 1 --2- 172.21.15.154:0/940793803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f264c004d00 0x7f264c005180 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2658007390 tx=0x7f264c0057f0 comp rx=0 tx=0).stop 2023-09-23T03:16:27.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.687+0000 7f26662ac640 1 -- 172.21.15.154:0/940793803 reap_dead start 2023-09-23T03:16:27.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.687+0000 7f2666aad640 1 -- 172.21.15.154:0/940793803 shutdown_connections 2023-09-23T03:16:27.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.687+0000 7f2666aad640 1 -- 172.21.15.154:0/940793803 >> 172.21.15.154:0/940793803 conn(0x7f26600aff20 msgr2=0x7f26600b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:27.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.687+0000 7f2666aad640 1 -- 172.21.15.154:0/940793803 shutdown_connections 2023-09-23T03:16:27.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.687+0000 7f2666aad640 1 -- 172.21.15.154:0/940793803 wait complete. 2023-09-23T03:16:27.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.689+0000 7f2666aad640 1 Processor -- start 2023-09-23T03:16:27.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.689+0000 7f2666aad640 1 -- start start 2023-09-23T03:16:27.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.690+0000 7f2666aad640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f264c004d00 0x7f26600c3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:27.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.690+0000 7f2666aad640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2660157280 0x7f26600c4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:27.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.690+0000 7f2666aad640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2660156d30 0x7f26600c48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:27.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.690+0000 7f2666aad640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26600c8310 con 0x7f264c004d00 2023-09-23T03:16:27.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.690+0000 7f2666aad640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266015a6a0 con 0x7f2660157280 2023-09-23T03:16:27.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.690+0000 7f2666aad640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f266015a980 con 0x7f2660156d30 2023-09-23T03:16:27.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.691+0000 7f2665aab640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f264c004d00 0x7f26600c3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:27.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.691+0000 7f2665aab640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f264c004d00 0x7f26600c3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46486/0 (socket says 172.21.15.154:46486) 2023-09-23T03:16:27.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.691+0000 7f2665aab640 1 -- 172.21.15.154:0/2413116590 learned_addr learned my addr 172.21.15.154:0/2413116590 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:27.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.691+0000 7f2665aab640 1 -- 172.21.15.154:0/2413116590 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2660156d30 msgr2=0x7f26600c48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:27.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.691+0000 7f2665aab640 1 --2- 172.21.15.154:0/2413116590 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2660156d30 0x7f26600c48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:27.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.691+0000 7f2665aab640 1 -- 172.21.15.154:0/2413116590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2660157280 msgr2=0x7f26600c4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:27.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.691+0000 7f2665aab640 1 --2- 172.21.15.154:0/2413116590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2660157280 0x7f26600c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:27.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.691+0000 7f2665aab640 1 -- 172.21.15.154:0/2413116590 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26600c5ed0 con 0x7f264c004d00 2023-09-23T03:16:27.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.691+0000 7f2665aab640 1 --2- 172.21.15.154:0/2413116590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f264c004d00 0x7f26600c3940 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f265c00d8e0 tx=0x7f265c00dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:27.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.692+0000 7f26567fc640 1 -- 172.21.15.154:0/2413116590 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c0074e0 con 0x7f264c004d00 2023-09-23T03:16:27.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.692+0000 7f26567fc640 1 -- 172.21.15.154:0/2413116590 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f265c007640 con 0x7f264c004d00 2023-09-23T03:16:27.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.692+0000 7f26567fc640 1 -- 172.21.15.154:0/2413116590 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265c004a00 con 0x7f264c004d00 2023-09-23T03:16:27.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.692+0000 7f2666aad640 1 -- 172.21.15.154:0/2413116590 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26600c61b0 con 0x7f264c004d00 2023-09-23T03:16:27.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.692+0000 7f2666aad640 1 -- 172.21.15.154:0/2413116590 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26600c69b0 con 0x7f264c004d00 2023-09-23T03:16:27.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.693+0000 7f2666aad640 1 -- 172.21.15.154:0/2413116590 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f266004e920 con 0x7f264c004d00 2023-09-23T03:16:27.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.694+0000 7f26567fc640 1 -- 172.21.15.154:0/2413116590 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f265c004b60 con 0x7f264c004d00 2023-09-23T03:16:27.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.695+0000 7f26567fc640 1 --2- 172.21.15.154:0/2413116590 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f264406af30 0x7f264406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:27.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.695+0000 7f26567fc640 1 -- 172.21.15.154:0/2413116590 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f265c0895a0 con 0x7f264c004d00 2023-09-23T03:16:27.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.695+0000 7f26652aa640 1 --2- 172.21.15.154:0/2413116590 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f264406af30 0x7f264406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:27.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.696+0000 7f26652aa640 1 --2- 172.21.15.154:0/2413116590 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f264406af30 0x7f264406d3f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f266015e520 tx=0x7f2650008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:27.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.699+0000 7f26567fc640 1 -- 172.21.15.154:0/2413116590 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f265c054250 con 0x7f264c004d00 2023-09-23T03:16:27.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.887+0000 7f2666aad640 1 -- 172.21.15.154:0/2413116590 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f26600b0900 con 0x7f264c004d00 2023-09-23T03:16:27.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.887+0000 7f26567fc640 1 -- 172.21.15.154:0/2413116590 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f265c057870 con 0x7f264c004d00 2023-09-23T03:16:27.889 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:16:27.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.890+0000 7f2666aad640 1 -- 172.21.15.154:0/2413116590 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f264406af30 msgr2=0x7f264406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:27.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.890+0000 7f2666aad640 1 --2- 172.21.15.154:0/2413116590 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f264406af30 0x7f264406d3f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f266015e520 tx=0x7f2650008040 comp rx=0 tx=0).stop 2023-09-23T03:16:27.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.890+0000 7f2666aad640 1 -- 172.21.15.154:0/2413116590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f264c004d00 msgr2=0x7f26600c3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:27.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.890+0000 7f2666aad640 1 --2- 172.21.15.154:0/2413116590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f264c004d00 0x7f26600c3940 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f265c00d8e0 tx=0x7f265c00dda0 comp rx=0 tx=0).stop 2023-09-23T03:16:27.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.890+0000 7f2666aad640 1 -- 172.21.15.154:0/2413116590 shutdown_connections 2023-09-23T03:16:27.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.890+0000 7f2666aad640 1 --2- 172.21.15.154:0/2413116590 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f264406af30 0x7f264406d3f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:27.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.890+0000 7f2666aad640 1 --2- 172.21.15.154:0/2413116590 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2660156d30 0x7f26600c48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:27.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.890+0000 7f2666aad640 1 --2- 172.21.15.154:0/2413116590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2660157280 0x7f26600c4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:27.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.890+0000 7f2666aad640 1 --2- 172.21.15.154:0/2413116590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f264c004d00 0x7f26600c3940 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:27.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.890+0000 7f2666aad640 1 -- 172.21.15.154:0/2413116590 >> 172.21.15.154:0/2413116590 conn(0x7f26600aff20 msgr2=0x7f26600c8f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:27.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.890+0000 7f2666aad640 1 -- 172.21.15.154:0/2413116590 shutdown_connections 2023-09-23T03:16:27.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:27.890+0000 7f2666aad640 1 -- 172.21.15.154:0/2413116590 wait complete. 2023-09-23T03:16:27.910 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-23T03:16:28.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.126+0000 7f4ad7064640 1 Processor -- start 2023-09-23T03:16:28.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.126+0000 7f4ad7064640 1 -- start start 2023-09-23T03:16:28.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.128+0000 7f4ad7064640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4ad00c8d70 0x7f4ad015c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:28.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.128+0000 7f4ad7064640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ad00c9150 0x7f4ad015cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:28.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.128+0000 7f4ad7064640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4ad0071290 0x7f4ad00ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:28.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.128+0000 7f4ad7064640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ad00c9d20 con 0x7f4ad0071290 2023-09-23T03:16:28.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.128+0000 7f4ad7064640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ad0071850 con 0x7f4ad00c8d70 2023-09-23T03:16:28.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.128+0000 7f4ad7064640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad0071b30 con 0x7f4ad00c9150 2023-09-23T03:16:28.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.128+0000 7f4ad5601640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4ad0071290 0x7f4ad00ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:28.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.128+0000 7f4ad5601640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4ad0071290 0x7f4ad00ca1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46494/0 (socket says 172.21.15.154:46494) 2023-09-23T03:16:28.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.128+0000 7f4ad5601640 1 -- 172.21.15.154:0/1712661947 learned_addr learned my addr 172.21.15.154:0/1712661947 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:28.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.129+0000 7f4ad5601640 1 -- 172.21.15.154:0/1712661947 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ad00c9150 msgr2=0x7f4ad015cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:28.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.129+0000 7f4acffff640 1 --2- 172.21.15.154:0/1712661947 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ad00c9150 0x7f4ad015cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:28.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.129+0000 7f4ad4e00640 1 --2- 172.21.15.154:0/1712661947 >> v2:172.21.15.190:3300/0 conn(0x7f4ad00c8d70 0x7f4ad015c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:28.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.129+0000 7f4ad5601640 1 --2- 172.21.15.154:0/1712661947 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ad00c9150 0x7f4ad015cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:28.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.129+0000 7f4ad5601640 1 -- 172.21.15.154:0/1712661947 >> v2:172.21.15.190:3300/0 conn(0x7f4ad00c8d70 msgr2=0x7f4ad015c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:28.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.129+0000 7f4ad5601640 1 --2- 172.21.15.154:0/1712661947 >> v2:172.21.15.190:3300/0 conn(0x7f4ad00c8d70 0x7f4ad015c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:28.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.129+0000 7f4ad5601640 1 -- 172.21.15.154:0/1712661947 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad0071e10 con 0x7f4ad0071290 2023-09-23T03:16:28.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.129+0000 7f4acffff640 1 --2- 172.21.15.154:0/1712661947 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ad00c9150 0x7f4ad015cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:28.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.130+0000 7f4ad5601640 1 --2- 172.21.15.154:0/1712661947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ad0071290 0x7f4ad00ca1b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4ad006b790 tx=0x7f4ac802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d777f6b4ac6761 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:28.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.130+0000 7f4acf7fe640 1 -- 172.21.15.154:0/1712661947 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac802c9f0 con 0x7f4ad0071290 2023-09-23T03:16:28.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.130+0000 7f4acf7fe640 1 -- 172.21.15.154:0/1712661947 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac802cb50 con 0x7f4ad0071290 2023-09-23T03:16:28.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.130+0000 7f4acf7fe640 1 -- 172.21.15.154:0/1712661947 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac80343f0 con 0x7f4ad0071290 2023-09-23T03:16:28.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.131+0000 7f4ad7064640 1 -- 172.21.15.154:0/1712661947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ad0071290 msgr2=0x7f4ad00ca1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:28.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.131+0000 7f4ad7064640 1 --2- 172.21.15.154:0/1712661947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ad0071290 0x7f4ad00ca1b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4ad006b790 tx=0x7f4ac802be50 comp rx=0 tx=0).stop 2023-09-23T03:16:28.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.131+0000 7f4ad7064640 1 -- 172.21.15.154:0/1712661947 shutdown_connections 2023-09-23T03:16:28.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.131+0000 7f4ad7064640 1 --2- 172.21.15.154:0/1712661947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ad0071290 0x7f4ad00ca1b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:28.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.131+0000 7f4ad7064640 1 --2- 172.21.15.154:0/1712661947 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ad00c9150 0x7f4ad015cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:28.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.131+0000 7f4ad7064640 1 --2- 172.21.15.154:0/1712661947 >> v2:172.21.15.190:3300/0 conn(0x7f4ad00c8d70 0x7f4ad015c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:28.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.131+0000 7f4ad7064640 1 -- 172.21.15.154:0/1712661947 >> 172.21.15.154:0/1712661947 conn(0x7f4ad006c010 msgr2=0x7f4ad006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:28.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.131+0000 7f4ad7064640 1 -- 172.21.15.154:0/1712661947 shutdown_connections 2023-09-23T03:16:28.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.131+0000 7f4ad7064640 1 -- 172.21.15.154:0/1712661947 wait complete. 2023-09-23T03:16:28.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.133+0000 7f4ad7064640 1 Processor -- start 2023-09-23T03:16:28.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.133+0000 7f4ad7064640 1 -- start start 2023-09-23T03:16:28.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.134+0000 7f4ad7064640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ad0071290 0x7f4ad00c5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:28.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.134+0000 7f4ad7064640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ad00c8d70 0x7f4ad00c6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:28.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.134+0000 7f4ad7064640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ad00c9150 0x7f4ad014e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:28.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.134+0000 7f4ad7064640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad00c2630 con 0x7f4ad0071290 2023-09-23T03:16:28.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.134+0000 7f4ad7064640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad0152b10 con 0x7f4ad00c8d70 2023-09-23T03:16:28.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.134+0000 7f4ad7064640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad00c6690 con 0x7f4ad00c9150 2023-09-23T03:16:28.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.134+0000 7f4ad4e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ad0071290 0x7f4ad00c5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:28.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.134+0000 7f4ad4e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ad0071290 0x7f4ad00c5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46506/0 (socket says 172.21.15.154:46506) 2023-09-23T03:16:28.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.134+0000 7f4ad4e00640 1 -- 172.21.15.154:0/3641345035 learned_addr learned my addr 172.21.15.154:0/3641345035 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:28.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.135+0000 7f4acffff640 1 --2- 172.21.15.154:0/3641345035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ad00c8d70 0x7f4ad00c6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:28.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.135+0000 7f4ad4e00640 1 -- 172.21.15.154:0/3641345035 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ad00c9150 msgr2=0x7f4ad014e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:28.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.135+0000 7f4ad5601640 1 --2- 172.21.15.154:0/3641345035 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ad00c9150 0x7f4ad014e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:28.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.135+0000 7f4ad4e00640 1 --2- 172.21.15.154:0/3641345035 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ad00c9150 0x7f4ad014e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:28.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.135+0000 7f4ad4e00640 1 -- 172.21.15.154:0/3641345035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ad00c8d70 msgr2=0x7f4ad00c6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:28.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.135+0000 7f4ad4e00640 1 --2- 172.21.15.154:0/3641345035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ad00c8d70 0x7f4ad00c6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:28.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.135+0000 7f4ad4e00640 1 -- 172.21.15.154:0/3641345035 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac8009b30 con 0x7f4ad0071290 2023-09-23T03:16:28.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.135+0000 7f4acffff640 1 --2- 172.21.15.154:0/3641345035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ad00c8d70 0x7f4ad00c6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:28.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.135+0000 7f4ad4e00640 1 --2- 172.21.15.154:0/3641345035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ad0071290 0x7f4ad00c5f90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4ac400c570 tx=0x7f4ac400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:28.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.135+0000 7f4acd7fa640 1 -- 172.21.15.154:0/3641345035 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac4011800 con 0x7f4ad0071290 2023-09-23T03:16:28.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.135+0000 7f4acd7fa640 1 -- 172.21.15.154:0/3641345035 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac4011960 con 0x7f4ad0071290 2023-09-23T03:16:28.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.135+0000 7f4ad7064640 1 -- 172.21.15.154:0/3641345035 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ad00c6970 con 0x7f4ad0071290 2023-09-23T03:16:28.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.135+0000 7f4acd7fa640 1 -- 172.21.15.154:0/3641345035 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac4011d10 con 0x7f4ad0071290 2023-09-23T03:16:28.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.136+0000 7f4ad7064640 1 -- 172.21.15.154:0/3641345035 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ad00c22c0 con 0x7f4ad0071290 2023-09-23T03:16:28.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.137+0000 7f4ad7064640 1 -- 172.21.15.154:0/3641345035 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ad004e920 con 0x7f4ad0071290 2023-09-23T03:16:28.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.138+0000 7f4acd7fa640 1 -- 172.21.15.154:0/3641345035 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f4ac4018360 con 0x7f4ad0071290 2023-09-23T03:16:28.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.139+0000 7f4acd7fa640 1 --2- 172.21.15.154:0/3641345035 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4abc06af30 0x7f4abc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:28.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.139+0000 7f4acd7fa640 1 -- 172.21.15.154:0/3641345035 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f4ac4089b80 con 0x7f4ad0071290 2023-09-23T03:16:28.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.139+0000 7f4acffff640 1 --2- 172.21.15.154:0/3641345035 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4abc06af30 0x7f4abc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:28.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.140+0000 7f4acffff640 1 --2- 172.21.15.154:0/3641345035 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4abc06af30 0x7f4abc06d3f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0006fd0 tx=0x7f4ac0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:28.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.144+0000 7f4acd7fa640 1 -- 172.21.15.154:0/3641345035 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f4ac4054770 con 0x7f4ad0071290 2023-09-23T03:16:28.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.333+0000 7f4ad7064640 1 -- 172.21.15.154:0/3641345035 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f4ad006cbc0 con 0x7f4ad0071290 2023-09-23T03:16:28.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.335+0000 7f4acd7fa640 1 -- 172.21.15.154:0/3641345035 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4ac4089e60 con 0x7f4ad0071290 2023-09-23T03:16:28.336 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:16:28.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.337+0000 7f4ad7064640 1 -- 172.21.15.154:0/3641345035 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4abc06af30 msgr2=0x7f4abc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:28.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.337+0000 7f4ad7064640 1 --2- 172.21.15.154:0/3641345035 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4abc06af30 0x7f4abc06d3f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0006fd0 tx=0x7f4ac0008040 comp rx=0 tx=0).stop 2023-09-23T03:16:28.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.337+0000 7f4ad7064640 1 -- 172.21.15.154:0/3641345035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ad0071290 msgr2=0x7f4ad00c5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:28.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.337+0000 7f4ad7064640 1 --2- 172.21.15.154:0/3641345035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ad0071290 0x7f4ad00c5f90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4ac400c570 tx=0x7f4ac400ca30 comp rx=0 tx=0).stop 2023-09-23T03:16:28.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.337+0000 7f4ad7064640 1 -- 172.21.15.154:0/3641345035 shutdown_connections 2023-09-23T03:16:28.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.337+0000 7f4ad7064640 1 --2- 172.21.15.154:0/3641345035 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4abc06af30 0x7f4abc06d3f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:28.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.337+0000 7f4ad7064640 1 --2- 172.21.15.154:0/3641345035 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ad00c9150 0x7f4ad014e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:28.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.337+0000 7f4ad7064640 1 --2- 172.21.15.154:0/3641345035 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ad00c8d70 0x7f4ad00c6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:28.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.337+0000 7f4ad7064640 1 --2- 172.21.15.154:0/3641345035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ad0071290 0x7f4ad00c5f90 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:28.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.337+0000 7f4ad7064640 1 -- 172.21.15.154:0/3641345035 >> 172.21.15.154:0/3641345035 conn(0x7f4ad006c010 msgr2=0x7f4ad00ce4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:28.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.337+0000 7f4ad7064640 1 -- 172.21.15.154:0/3641345035 shutdown_connections 2023-09-23T03:16:28.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.337+0000 7f4ad7064640 1 -- 172.21.15.154:0/3641345035 wait complete. 2023-09-23T03:16:28.358 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_few_files' 2023-09-23T03:16:28.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.570+0000 7f385d4e5640 1 Processor -- start 2023-09-23T03:16:28.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.571+0000 7f385d4e5640 1 -- start start 2023-09-23T03:16:28.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.572+0000 7f385d4e5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3858070760 0x7f38580b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:28.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.572+0000 7f385d4e5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38580b1700 con 0x7f38580c2060 2023-09-23T03:16:28.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.572+0000 7f385d4e5640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38580b19e0 con 0x7f385815a840 2023-09-23T03:16:28.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.572+0000 7f385d4e5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38580b1cc0 con 0x7f3858070760 2023-09-23T03:16:28.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.572+0000 7f38577fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f38580c2060 0x7f38580b2ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:52832/0 (socket says 172.21.15.154:52832) 2023-09-23T03:16:28.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.572+0000 7f38577fe640 1 -- 172.21.15.154:0/1830193635 learned_addr learned my addr 172.21.15.154:0/1830193635 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:28.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.573+0000 7f384ffff640 1 --2- 172.21.15.154:0/1830193635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3858070760 0x7f38580b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:28.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.573+0000 7f38567fc640 1 -- 172.21.15.154:0/1830193635 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3880124206 0 0) 0x7f38580b1700 con 0x7f38580c2060 2023-09-23T03:16:28.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.573+0000 7f38567fc640 1 -- 172.21.15.154:0/1830193635 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3844003520 con 0x7f38580c2060 2023-09-23T03:16:28.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.573+0000 7f38567fc640 1 -- 172.21.15.154:0/1830193635 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2219988985 0 0) 0x7f3844003520 con 0x7f38580c2060 2023-09-23T03:16:28.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.573+0000 7f38567fc640 1 -- 172.21.15.154:0/1830193635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3858070760 msgr2=0x7f38580b27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:28.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.573+0000 7f38567fc640 1 --2- 172.21.15.154:0/1830193635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3858070760 0x7f38580b27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:28.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.573+0000 7f38567fc640 1 -- 172.21.15.154:0/1830193635 >> v1:172.21.15.190:6789/0 conn(0x7f385815a840 legacy=0x7f3858070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:28.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.573+0000 7f38567fc640 1 -- 172.21.15.154:0/1830193635 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38580b1fa0 con 0x7f38580c2060 2023-09-23T03:16:28.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.573+0000 7f38567fc640 1 -- 172.21.15.154:0/1830193635 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f38500019f0 con 0x7f38580c2060 2023-09-23T03:16:28.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.573+0000 7f38567fc640 1 -- 172.21.15.154:0/1830193635 >> v1:172.21.15.154:6789/0 conn(0x7f38580c2060 legacy=0x7f38580b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:28.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.573+0000 7f38567fc640 1 --2- 172.21.15.154:0/1830193635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3858070760 0x7f38440039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:28.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.573+0000 7f38567fc640 1 --2- 172.21.15.154:0/1830193635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3844003f70 0x7f38440047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:28.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.573+0000 7f38567fc640 1 --2- 172.21.15.154:0/1830193635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3844004d00 0x7f3844005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:28.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.573+0000 7f38567fc640 1 -- 172.21.15.154:0/1830193635 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38580b1cc0 con 0x7f3844004d00 2023-09-23T03:16:28.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.573+0000 7f38567fc640 1 -- 172.21.15.154:0/1830193635 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384400bac0 con 0x7f3844003f70 2023-09-23T03:16:28.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.573+0000 7f38567fc640 1 -- 172.21.15.154:0/1830193635 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f384400bc80 con 0x7f3858070760 2023-09-23T03:16:28.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.574+0000 7f38577fe640 1 --2- 172.21.15.154:0/1830193635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3844004d00 0x7f3844005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:28.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.574+0000 7f384ffff640 1 --2- 172.21.15.154:0/1830193635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3844003f70 0x7f38440047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:28.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.574+0000 7f38577fe640 1 -- 172.21.15.154:0/1830193635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3858070760 msgr2=0x7f38440039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:28.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.574+0000 7f38577fe640 1 --2- 172.21.15.154:0/1830193635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3858070760 0x7f38440039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:28.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.574+0000 7f38577fe640 1 -- 172.21.15.154:0/1830193635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3844003f70 msgr2=0x7f38440047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:28.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.574+0000 7f38577fe640 1 --2- 172.21.15.154:0/1830193635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3844003f70 0x7f38440047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:28.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.574+0000 7f38577fe640 1 -- 172.21.15.154:0/1830193635 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3844003520 con 0x7f3844004d00 2023-09-23T03:16:28.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.574+0000 7f384ffff640 1 --2- 172.21.15.154:0/1830193635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3844003f70 0x7f38440047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:28.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.575+0000 7f38577fe640 1 --2- 172.21.15.154:0/1830193635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3844004d00 0x7f3844005180 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f385000a760 tx=0x7f38440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=226df165b64f5e74 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:28.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.575+0000 7f38567fc640 1 -- 172.21.15.154:0/1830193635 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3850034630 con 0x7f3844004d00 2023-09-23T03:16:28.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.575+0000 7f38567fc640 1 -- 172.21.15.154:0/1830193635 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385002f070 con 0x7f3844004d00 2023-09-23T03:16:28.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.575+0000 7f38567fc640 1 -- 172.21.15.154:0/1830193635 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3850034a50 con 0x7f3844004d00 2023-09-23T03:16:28.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.576+0000 7f385d4e5640 1 -- 172.21.15.154:0/1830193635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3844004d00 msgr2=0x7f3844005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:28.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.576+0000 7f385d4e5640 1 --2- 172.21.15.154:0/1830193635 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3844004d00 0x7f3844005180 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f385000a760 tx=0x7f38440057f0 comp rx=0 tx=0).stop 2023-09-23T03:16:28.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.576+0000 7f38577fe640 1 -- 172.21.15.154:0/1830193635 reap_dead start 2023-09-23T03:16:28.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.576+0000 7f385d4e5640 1 -- 172.21.15.154:0/1830193635 shutdown_connections 2023-09-23T03:16:28.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.576+0000 7f385d4e5640 1 -- 172.21.15.154:0/1830193635 >> 172.21.15.154:0/1830193635 conn(0x7f385806bfd0 msgr2=0x7f385806c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:28.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.576+0000 7f385d4e5640 1 -- 172.21.15.154:0/1830193635 shutdown_connections 2023-09-23T03:16:28.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.576+0000 7f385d4e5640 1 -- 172.21.15.154:0/1830193635 wait complete. 2023-09-23T03:16:28.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.578+0000 7f385d4e5640 1 Processor -- start 2023-09-23T03:16:28.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.578+0000 7f385d4e5640 1 -- start start 2023-09-23T03:16:28.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.579+0000 7f385d4e5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3844004d00 0x7f38580c38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:28.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.579+0000 7f385d4e5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38580c3e20 0x7f38580c55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:28.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.579+0000 7f385d4e5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38580c5b20 0x7f38580c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:28.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.579+0000 7f385d4e5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38580c42a0 con 0x7f3844004d00 2023-09-23T03:16:28.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.579+0000 7f385d4e5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38581f1080 con 0x7f38580c3e20 2023-09-23T03:16:28.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.579+0000 7f385d4e5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38581f0750 con 0x7f38580c5b20 2023-09-23T03:16:28.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.579+0000 7f3856ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3844004d00 0x7f38580c38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:28.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.579+0000 7f3856ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3844004d00 0x7f38580c38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46536/0 (socket says 172.21.15.154:46536) 2023-09-23T03:16:28.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.579+0000 7f3856ffd640 1 -- 172.21.15.154:0/4121867678 learned_addr learned my addr 172.21.15.154:0/4121867678 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:28.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.580+0000 7f3856ffd640 1 -- 172.21.15.154:0/4121867678 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38580c5b20 msgr2=0x7f38580c5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:28.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.580+0000 7f3856ffd640 1 --2- 172.21.15.154:0/4121867678 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38580c5b20 0x7f38580c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:28.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.580+0000 7f3856ffd640 1 -- 172.21.15.154:0/4121867678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38580c3e20 msgr2=0x7f38580c55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:28.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.580+0000 7f3856ffd640 1 --2- 172.21.15.154:0/4121867678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38580c3e20 0x7f38580c55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:28.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.580+0000 7f3856ffd640 1 -- 172.21.15.154:0/4121867678 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38581f0a30 con 0x7f3844004d00 2023-09-23T03:16:28.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.580+0000 7f3856ffd640 1 --2- 172.21.15.154:0/4121867678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3844004d00 0x7f38580c38e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f384800c480 tx=0x7f384800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:28.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.580+0000 7f384f7fe640 1 -- 172.21.15.154:0/4121867678 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38480052d0 con 0x7f3844004d00 2023-09-23T03:16:28.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.580+0000 7f384f7fe640 1 -- 172.21.15.154:0/4121867678 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3848005470 con 0x7f3844004d00 2023-09-23T03:16:28.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.580+0000 7f385d4e5640 1 -- 172.21.15.154:0/4121867678 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38581f0d10 con 0x7f3844004d00 2023-09-23T03:16:28.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.580+0000 7f384f7fe640 1 -- 172.21.15.154:0/4121867678 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3848003830 con 0x7f3844004d00 2023-09-23T03:16:28.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.580+0000 7f385d4e5640 1 -- 172.21.15.154:0/4121867678 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f385815b690 con 0x7f3844004d00 2023-09-23T03:16:28.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.582+0000 7f385d4e5640 1 -- 172.21.15.154:0/4121867678 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f385804e920 con 0x7f3844004d00 2023-09-23T03:16:28.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.582+0000 7f384f7fe640 1 -- 172.21.15.154:0/4121867678 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f3848003990 con 0x7f3844004d00 2023-09-23T03:16:28.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.584+0000 7f384f7fe640 1 --2- 172.21.15.154:0/4121867678 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f383806af60 0x7f383806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:28.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.584+0000 7f384f7fe640 1 -- 172.21.15.154:0/4121867678 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f3848088f00 con 0x7f3844004d00 2023-09-23T03:16:28.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.584+0000 7f384ffff640 1 --2- 172.21.15.154:0/4121867678 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f383806af60 0x7f383806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:28.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.585+0000 7f384ffff640 1 --2- 172.21.15.154:0/4121867678 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f383806af60 0x7f383806d420 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f38580afd30 tx=0x7f3840008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:28.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.588+0000 7f384f7fe640 1 -- 172.21.15.154:0/4121867678 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f3848053c40 con 0x7f3844004d00 2023-09-23T03:16:28.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:28.850+0000 7f385d4e5640 1 -- 172.21.15.154:0/4121867678 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_few_files"]} v 0) v1 -- 0x7f38580b1330 con 0x7f3844004d00 2023-09-23T03:16:29.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.338+0000 7f384f7fe640 1 -- 172.21.15.154:0/4121867678 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_few_files"]}]=0 v82) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3848017020 con 0x7f3844004d00 2023-09-23T03:16:29.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.341+0000 7f385d4e5640 1 -- 172.21.15.154:0/4121867678 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f383806af60 msgr2=0x7f383806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:29.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.341+0000 7f385d4e5640 1 --2- 172.21.15.154:0/4121867678 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f383806af60 0x7f383806d420 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f38580afd30 tx=0x7f3840008040 comp rx=0 tx=0).stop 2023-09-23T03:16:29.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.341+0000 7f385d4e5640 1 -- 172.21.15.154:0/4121867678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3844004d00 msgr2=0x7f38580c38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:29.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.341+0000 7f385d4e5640 1 --2- 172.21.15.154:0/4121867678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3844004d00 0x7f38580c38e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f384800c480 tx=0x7f384800c940 comp rx=0 tx=0).stop 2023-09-23T03:16:29.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.342+0000 7f385d4e5640 1 -- 172.21.15.154:0/4121867678 shutdown_connections 2023-09-23T03:16:29.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.342+0000 7f385d4e5640 1 --2- 172.21.15.154:0/4121867678 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f383806af60 0x7f383806d420 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:29.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.342+0000 7f385d4e5640 1 --2- 172.21.15.154:0/4121867678 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38580c5b20 0x7f38580c5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:29.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.342+0000 7f385d4e5640 1 --2- 172.21.15.154:0/4121867678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38580c3e20 0x7f38580c55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:29.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.342+0000 7f385d4e5640 1 --2- 172.21.15.154:0/4121867678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3844004d00 0x7f38580c38e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:29.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.342+0000 7f385d4e5640 1 -- 172.21.15.154:0/4121867678 >> 172.21.15.154:0/4121867678 conn(0x7f385806bfd0 msgr2=0x7f38580ade40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:29.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.342+0000 7f385d4e5640 1 -- 172.21.15.154:0/4121867678 shutdown_connections 2023-09-23T03:16:29.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.342+0000 7f385d4e5640 1 -- 172.21.15.154:0/4121867678 wait complete. 2023-09-23T03:16:29.364 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-09-23T03:16:29.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.575+0000 7f9643fff640 1 Processor -- start 2023-09-23T03:16:29.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.575+0000 7f9643fff640 1 -- start start 2023-09-23T03:16:29.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.577+0000 7f9643fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9644071810 0x7f96441605e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:29.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.577+0000 7f9643fff640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9644071e70 con 0x7f9644159570 2023-09-23T03:16:29.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.577+0000 7f9643fff640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96440c8b50 con 0x7f9644160b20 2023-09-23T03:16:29.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.577+0000 7f9643fff640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9644072150 con 0x7f9644071810 2023-09-23T03:16:29.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.577+0000 7f963bfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9644071810 0x7f96441605e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:29.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.577+0000 7f9642ffd640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f9644159570 0x7f9644071100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:52836/0 (socket says 172.21.15.154:52836) 2023-09-23T03:16:29.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.577+0000 7f9642ffd640 1 -- 172.21.15.154:0/1747823921 learned_addr learned my addr 172.21.15.154:0/1747823921 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:29.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.578+0000 7f963bfff640 1 -- 172.21.15.154:0/1747823921 >> v1:172.21.15.190:6789/0 conn(0x7f9644160b20 legacy=0x7f9644162f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:29.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.578+0000 7f963bfff640 1 -- 172.21.15.154:0/1747823921 >> v1:172.21.15.154:6789/0 conn(0x7f9644159570 legacy=0x7f9644071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:29.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.578+0000 7f96427fc640 1 -- 172.21.15.154:0/1747823921 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4242499574 0 0) 0x7f9644071e70 con 0x7f9644159570 2023-09-23T03:16:29.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.578+0000 7f963bfff640 1 -- 172.21.15.154:0/1747823921 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96440c8e30 con 0x7f9644071810 2023-09-23T03:16:29.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.579+0000 7f963bfff640 1 --2- 172.21.15.154:0/1747823921 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9644071810 0x7f96441605e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f962c0097a0 tx=0x7f962c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a8f4cb881b100ea server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:29.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.579+0000 7f96427fc640 1 -- 172.21.15.154:0/1747823921 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962c02e070 con 0x7f9644071810 2023-09-23T03:16:29.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.579+0000 7f96427fc640 1 -- 172.21.15.154:0/1747823921 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f962c0315a0 con 0x7f9644071810 2023-09-23T03:16:29.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.579+0000 7f96427fc640 1 -- 172.21.15.154:0/1747823921 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962c031930 con 0x7f9644071810 2023-09-23T03:16:29.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.580+0000 7f9643fff640 1 -- 172.21.15.154:0/1747823921 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9644071810 msgr2=0x7f96441605e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:29.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.580+0000 7f9643fff640 1 --2- 172.21.15.154:0/1747823921 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9644071810 0x7f96441605e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f962c0097a0 tx=0x7f962c02b840 comp rx=0 tx=0).stop 2023-09-23T03:16:29.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.580+0000 7f9643fff640 1 -- 172.21.15.154:0/1747823921 shutdown_connections 2023-09-23T03:16:29.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.580+0000 7f9643fff640 1 --2- 172.21.15.154:0/1747823921 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9644071810 0x7f96441605e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:29.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.580+0000 7f9643fff640 1 -- 172.21.15.154:0/1747823921 >> 172.21.15.154:0/1747823921 conn(0x7f96440aff20 msgr2=0x7f96440b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:29.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.580+0000 7f9643fff640 1 -- 172.21.15.154:0/1747823921 shutdown_connections 2023-09-23T03:16:29.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.580+0000 7f9643fff640 1 -- 172.21.15.154:0/1747823921 wait complete. 2023-09-23T03:16:29.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.582+0000 7f9643fff640 1 Processor -- start 2023-09-23T03:16:29.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.582+0000 7f9643fff640 1 -- start start 2023-09-23T03:16:29.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.583+0000 7f9643fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9644071810 0x7f96440ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:29.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.583+0000 7f9643fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9644159570 0x7f9644149260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:29.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.583+0000 7f9643fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9644160b20 0x7f96440cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:29.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.583+0000 7f9643fff640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96440ce8f0 con 0x7f9644160b20 2023-09-23T03:16:29.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.583+0000 7f9643fff640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9644149bb0 con 0x7f9644071810 2023-09-23T03:16:29.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.583+0000 7f9643fff640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9644149e90 con 0x7f9644159570 2023-09-23T03:16:29.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.583+0000 7f96437fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9644160b20 0x7f96440cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:29.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.583+0000 7f96437fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9644160b20 0x7f96440cc2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46556/0 (socket says 172.21.15.154:46556) 2023-09-23T03:16:29.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.583+0000 7f96437fe640 1 -- 172.21.15.154:0/2176766502 learned_addr learned my addr 172.21.15.154:0/2176766502 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:29.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.583+0000 7f96437fe640 1 -- 172.21.15.154:0/2176766502 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9644159570 msgr2=0x7f9644149260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:29.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.583+0000 7f96437fe640 1 --2- 172.21.15.154:0/2176766502 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9644159570 0x7f9644149260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:29.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.583+0000 7f9642ffd640 1 --2- 172.21.15.154:0/2176766502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9644071810 0x7f96440ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:29.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.583+0000 7f96437fe640 1 -- 172.21.15.154:0/2176766502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9644071810 msgr2=0x7f96440ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:29.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.583+0000 7f96437fe640 1 --2- 172.21.15.154:0/2176766502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9644071810 0x7f96440ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:29.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.583+0000 7f96437fe640 1 -- 172.21.15.154:0/2176766502 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f962c009400 con 0x7f9644160b20 2023-09-23T03:16:29.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.583+0000 7f9642ffd640 1 --2- 172.21.15.154:0/2176766502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9644071810 0x7f96440ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:29.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.584+0000 7f96437fe640 1 --2- 172.21.15.154:0/2176766502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9644160b20 0x7f96440cc2d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f963c00b990 tx=0x7f963c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:29.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.584+0000 7f963b7fe640 1 -- 172.21.15.154:0/2176766502 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963c00cba0 con 0x7f9644160b20 2023-09-23T03:16:29.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.584+0000 7f963b7fe640 1 -- 172.21.15.154:0/2176766502 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f963c00cd00 con 0x7f9644160b20 2023-09-23T03:16:29.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.584+0000 7f963b7fe640 1 -- 172.21.15.154:0/2176766502 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963c014680 con 0x7f9644160b20 2023-09-23T03:16:29.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.584+0000 7f9643fff640 1 -- 172.21.15.154:0/2176766502 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f964414a170 con 0x7f9644160b20 2023-09-23T03:16:29.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.584+0000 7f9643fff640 1 -- 172.21.15.154:0/2176766502 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f964414a730 con 0x7f9644160b20 2023-09-23T03:16:29.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.585+0000 7f9643fff640 1 -- 172.21.15.154:0/2176766502 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96440c4da0 con 0x7f9644160b20 2023-09-23T03:16:29.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.586+0000 7f963b7fe640 1 -- 172.21.15.154:0/2176766502 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f963c01d3b0 con 0x7f9644160b20 2023-09-23T03:16:29.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.587+0000 7f963b7fe640 1 --2- 172.21.15.154:0/2176766502 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f962406af60 0x7f962406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:29.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.587+0000 7f963b7fe640 1 -- 172.21.15.154:0/2176766502 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f963c089b10 con 0x7f9644160b20 2023-09-23T03:16:29.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.588+0000 7f9642ffd640 1 --2- 172.21.15.154:0/2176766502 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f962406af60 0x7f962406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:29.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.588+0000 7f9642ffd640 1 --2- 172.21.15.154:0/2176766502 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f962406af60 0x7f962406d420 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9634000f60 tx=0x7f9634009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:29.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.591+0000 7f963b7fe640 1 -- 172.21.15.154:0/2176766502 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f963c054700 con 0x7f9644160b20 2023-09-23T03:16:29.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.783+0000 7f9643fff640 1 -- 172.21.15.154:0/2176766502 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f964414a920 con 0x7f9644160b20 2023-09-23T03:16:29.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.789+0000 7f963b7fe640 1 -- 172.21.15.154:0/2176766502 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f963c019020 con 0x7f9644160b20 2023-09-23T03:16:29.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.792+0000 7f9643fff640 1 -- 172.21.15.154:0/2176766502 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f962406af60 msgr2=0x7f962406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:29.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.792+0000 7f9643fff640 1 --2- 172.21.15.154:0/2176766502 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f962406af60 0x7f962406d420 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9634000f60 tx=0x7f9634009040 comp rx=0 tx=0).stop 2023-09-23T03:16:29.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.792+0000 7f9643fff640 1 -- 172.21.15.154:0/2176766502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9644160b20 msgr2=0x7f96440cc2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:29.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.792+0000 7f9643fff640 1 --2- 172.21.15.154:0/2176766502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9644160b20 0x7f96440cc2d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f963c00b990 tx=0x7f963c00be50 comp rx=0 tx=0).stop 2023-09-23T03:16:29.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.792+0000 7f9643fff640 1 -- 172.21.15.154:0/2176766502 shutdown_connections 2023-09-23T03:16:29.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.792+0000 7f9643fff640 1 --2- 172.21.15.154:0/2176766502 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f962406af60 0x7f962406d420 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:29.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.792+0000 7f9643fff640 1 --2- 172.21.15.154:0/2176766502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9644160b20 0x7f96440cc2d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:29.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.792+0000 7f9643fff640 1 --2- 172.21.15.154:0/2176766502 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9644159570 0x7f9644149260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:29.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.792+0000 7f9643fff640 1 --2- 172.21.15.154:0/2176766502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9644071810 0x7f96440ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:29.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.792+0000 7f9643fff640 1 -- 172.21.15.154:0/2176766502 >> 172.21.15.154:0/2176766502 conn(0x7f96440aff20 msgr2=0x7f964415d170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:29.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.792+0000 7f9643fff640 1 -- 172.21.15.154:0/2176766502 shutdown_connections 2023-09-23T03:16:29.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:29.792+0000 7f9643fff640 1 -- 172.21.15.154:0/2176766502 wait complete. 2023-09-23T03:16:29.812 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-09-23T03:16:29.812 DEBUG:teuthology.orchestra.run.smithi154:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-23T03:16:29.929 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-23T03:16:30.054 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-09-23T03:16:30.054 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:16:30.054 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:16:30.114 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:16:30.114 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:16:30.169 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:16:30.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.378+0000 7f793af89640 1 Processor -- start 2023-09-23T03:16:30.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.378+0000 7f793af89640 1 -- start start 2023-09-23T03:16:30.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.379+0000 7f793af89640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f79340acce0 0x7f793415ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:30.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.380+0000 7f793af89640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f79340b1b20 0x7f79340ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:30.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.380+0000 7f793af89640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79340cee70 0x7f793406cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:30.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.380+0000 7f793af89640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79340b2e90 con 0x7f79340b1b20 2023-09-23T03:16:30.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.380+0000 7f793af89640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79340cf300 con 0x7f79340acce0 2023-09-23T03:16:30.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.380+0000 7f793af89640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f793415ac30 con 0x7f79340cee70 2023-09-23T03:16:30.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.380+0000 7f7933fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f79340b1b20 0x7f79340ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:30.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.380+0000 7f7933fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f79340b1b20 0x7f79340ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46564/0 (socket says 172.21.15.154:46564) 2023-09-23T03:16:30.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.380+0000 7f7933fff640 1 -- 172.21.15.154:0/3179964589 learned_addr learned my addr 172.21.15.154:0/3179964589 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:30.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.380+0000 7f7938d25640 1 --2- 172.21.15.154:0/3179964589 >> v2:172.21.15.190:3300/0 conn(0x7f79340acce0 0x7f793415ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:30.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.380+0000 7f7939526640 1 --2- 172.21.15.154:0/3179964589 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79340cee70 0x7f793406cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:30.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.381+0000 7f7933fff640 1 -- 172.21.15.154:0/3179964589 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79340cee70 msgr2=0x7f793406cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:30.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.381+0000 7f7933fff640 1 --2- 172.21.15.154:0/3179964589 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79340cee70 0x7f793406cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:30.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.381+0000 7f7933fff640 1 -- 172.21.15.154:0/3179964589 >> v2:172.21.15.190:3300/0 conn(0x7f79340acce0 msgr2=0x7f793415ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:30.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.381+0000 7f7933fff640 1 --2- 172.21.15.154:0/3179964589 >> v2:172.21.15.190:3300/0 conn(0x7f79340acce0 0x7f793415ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:30.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.381+0000 7f7933fff640 1 -- 172.21.15.154:0/3179964589 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79340b1f70 con 0x7f79340b1b20 2023-09-23T03:16:30.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.382+0000 7f7933fff640 1 --2- 172.21.15.154:0/3179964589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79340b1b20 0x7f79340ce930 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f79340cf570 tx=0x7f792002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd54ce972057f43 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:30.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.382+0000 7f79337fe640 1 -- 172.21.15.154:0/3179964589 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79200317c0 con 0x7f79340b1b20 2023-09-23T03:16:30.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.382+0000 7f79337fe640 1 -- 172.21.15.154:0/3179964589 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7920031920 con 0x7f79340b1b20 2023-09-23T03:16:30.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.382+0000 7f79337fe640 1 -- 172.21.15.154:0/3179964589 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7920031cb0 con 0x7f79340b1b20 2023-09-23T03:16:30.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.383+0000 7f793af89640 1 -- 172.21.15.154:0/3179964589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79340b1b20 msgr2=0x7f79340ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:30.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.383+0000 7f793af89640 1 --2- 172.21.15.154:0/3179964589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79340b1b20 0x7f79340ce930 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f79340cf570 tx=0x7f792002b860 comp rx=0 tx=0).stop 2023-09-23T03:16:30.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.383+0000 7f793af89640 1 -- 172.21.15.154:0/3179964589 shutdown_connections 2023-09-23T03:16:30.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.383+0000 7f793af89640 1 --2- 172.21.15.154:0/3179964589 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79340cee70 0x7f793406cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:30.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.383+0000 7f793af89640 1 --2- 172.21.15.154:0/3179964589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79340b1b20 0x7f79340ce930 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:30.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.383+0000 7f793af89640 1 --2- 172.21.15.154:0/3179964589 >> v2:172.21.15.190:3300/0 conn(0x7f79340acce0 0x7f793415ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:30.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.383+0000 7f793af89640 1 -- 172.21.15.154:0/3179964589 >> 172.21.15.154:0/3179964589 conn(0x7f793406a0f0 msgr2=0x7f793406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:30.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.383+0000 7f793af89640 1 -- 172.21.15.154:0/3179964589 shutdown_connections 2023-09-23T03:16:30.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.383+0000 7f793af89640 1 -- 172.21.15.154:0/3179964589 wait complete. 2023-09-23T03:16:30.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.384+0000 7f793af89640 1 Processor -- start 2023-09-23T03:16:30.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.385+0000 7f793af89640 1 -- start start 2023-09-23T03:16:30.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.385+0000 7f793af89640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79340acce0 0x7f7934157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:30.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.385+0000 7f793af89640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79340b1b20 0x7f79340c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:30.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.385+0000 7f793af89640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79340cee70 0x7f79340c8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:30.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.386+0000 7f793af89640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793415baf0 con 0x7f79340acce0 2023-09-23T03:16:30.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.386+0000 7f793af89640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793415aee0 con 0x7f79340cee70 2023-09-23T03:16:30.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.386+0000 7f793af89640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f793415b1c0 con 0x7f79340b1b20 2023-09-23T03:16:30.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.386+0000 7f7938d25640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79340acce0 0x7f7934157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:30.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.386+0000 7f7938d25640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79340acce0 0x7f7934157550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46572/0 (socket says 172.21.15.154:46572) 2023-09-23T03:16:30.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.386+0000 7f7938d25640 1 -- 172.21.15.154:0/3234242628 learned_addr learned my addr 172.21.15.154:0/3234242628 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:30.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.386+0000 7f7939526640 1 --2- 172.21.15.154:0/3234242628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79340cee70 0x7f79340c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:30.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.386+0000 7f7933fff640 1 --2- 172.21.15.154:0/3234242628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79340b1b20 0x7f79340c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:30.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.386+0000 7f7938d25640 1 -- 172.21.15.154:0/3234242628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79340b1b20 msgr2=0x7f79340c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:30.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.386+0000 7f7938d25640 1 --2- 172.21.15.154:0/3234242628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79340b1b20 0x7f79340c8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:30.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.386+0000 7f7938d25640 1 -- 172.21.15.154:0/3234242628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79340cee70 msgr2=0x7f79340c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:30.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.386+0000 7f7938d25640 1 --2- 172.21.15.154:0/3234242628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79340cee70 0x7f79340c8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:30.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.386+0000 7f7938d25640 1 -- 172.21.15.154:0/3234242628 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79200094d0 con 0x7f79340acce0 2023-09-23T03:16:30.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.386+0000 7f7939526640 1 --2- 172.21.15.154:0/3234242628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79340cee70 0x7f79340c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:30.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.386+0000 7f7933fff640 1 --2- 172.21.15.154:0/3234242628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79340b1b20 0x7f79340c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:30.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.387+0000 7f7938d25640 1 --2- 172.21.15.154:0/3234242628 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79340acce0 0x7f7934157550 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f792800d800 tx=0x7f792800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:30.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.387+0000 7f79317fa640 1 -- 172.21.15.154:0/3234242628 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792800b6c0 con 0x7f79340acce0 2023-09-23T03:16:30.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.387+0000 7f79317fa640 1 -- 172.21.15.154:0/3234242628 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f792800b820 con 0x7f79340acce0 2023-09-23T03:16:30.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.387+0000 7f79317fa640 1 -- 172.21.15.154:0/3234242628 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792800bbd0 con 0x7f79340acce0 2023-09-23T03:16:30.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.387+0000 7f793af89640 1 -- 172.21.15.154:0/3234242628 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f793415b4a0 con 0x7f79340acce0 2023-09-23T03:16:30.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.387+0000 7f793af89640 1 -- 172.21.15.154:0/3234242628 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79340c4db0 con 0x7f79340acce0 2023-09-23T03:16:30.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.388+0000 7f793af89640 1 -- 172.21.15.154:0/3234242628 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7934073eb0 con 0x7f79340acce0 2023-09-23T03:16:30.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.389+0000 7f79317fa640 1 -- 172.21.15.154:0/3234242628 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f792800bd30 con 0x7f79340acce0 2023-09-23T03:16:30.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.390+0000 7f79317fa640 1 --2- 172.21.15.154:0/3234242628 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f791806af60 0x7f791806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:30.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.390+0000 7f79317fa640 1 -- 172.21.15.154:0/3234242628 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f792808a0c0 con 0x7f79340acce0 2023-09-23T03:16:30.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.390+0000 7f7933fff640 1 --2- 172.21.15.154:0/3234242628 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f791806af60 0x7f791806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:30.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.391+0000 7f7933fff640 1 --2- 172.21.15.154:0/3234242628 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f791806af60 0x7f791806d420 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7920036000 tx=0x7f792003d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:30.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.394+0000 7f79317fa640 1 -- 172.21.15.154:0/3234242628 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f7928054d50 con 0x7f79340acce0 2023-09-23T03:16:30.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.588+0000 7f793af89640 1 -- 172.21.15.154:0/3234242628 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f793406c930 con 0x7f79340acce0 2023-09-23T03:16:30.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.591+0000 7f79317fa640 1 -- 172.21.15.154:0/3234242628 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18744 (secure 0 0 0) 0x7f7928058370 con 0x7f79340acce0 2023-09-23T03:16:30.592 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:16:30.593 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":26,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:16:03.151371+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:15:18.575849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T03:15:19.707212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T03:15:23.337138+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:16:30.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.594+0000 7f793af89640 1 -- 172.21.15.154:0/3234242628 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f791806af60 msgr2=0x7f791806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:30.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.594+0000 7f793af89640 1 --2- 172.21.15.154:0/3234242628 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f791806af60 0x7f791806d420 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7920036000 tx=0x7f792003d040 comp rx=0 tx=0).stop 2023-09-23T03:16:30.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.594+0000 7f793af89640 1 -- 172.21.15.154:0/3234242628 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79340acce0 msgr2=0x7f7934157550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:30.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.594+0000 7f793af89640 1 --2- 172.21.15.154:0/3234242628 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79340acce0 0x7f7934157550 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f792800d800 tx=0x7f792800dcc0 comp rx=0 tx=0).stop 2023-09-23T03:16:30.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.594+0000 7f793af89640 1 -- 172.21.15.154:0/3234242628 shutdown_connections 2023-09-23T03:16:30.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.594+0000 7f793af89640 1 --2- 172.21.15.154:0/3234242628 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f791806af60 0x7f791806d420 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:30.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.594+0000 7f793af89640 1 --2- 172.21.15.154:0/3234242628 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79340cee70 0x7f79340c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:30.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.594+0000 7f793af89640 1 --2- 172.21.15.154:0/3234242628 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79340b1b20 0x7f79340c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:30.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.594+0000 7f793af89640 1 --2- 172.21.15.154:0/3234242628 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79340acce0 0x7f7934157550 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:30.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.594+0000 7f793af89640 1 -- 172.21.15.154:0/3234242628 >> 172.21.15.154:0/3234242628 conn(0x7f793406a0f0 msgr2=0x7f79340b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:30.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.594+0000 7f793af89640 1 -- 172.21.15.154:0/3234242628 shutdown_connections 2023-09-23T03:16:30.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.594+0000 7f793af89640 1 -- 172.21.15.154:0/3234242628 wait complete. 2023-09-23T03:16:30.614 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:15:18.575849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:15:19.707212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-23T03:15:23.337138+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-23T03:16:30.615 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:16:30.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.831+0000 7f4325021640 1 Processor -- start 2023-09-23T03:16:30.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.831+0000 7f4325021640 1 -- start start 2023-09-23T03:16:30.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.832+0000 7f4325021640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f432015aba0 0x7f432015bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:30.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.832+0000 7f4325021640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43200cb5b0 0x7f432015ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:30.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.832+0000 7f4325021640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f432015b6a0 con 0x7f432015e5f0 2023-09-23T03:16:30.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.832+0000 7f4325021640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f432015f0b0 con 0x7f432015aba0 2023-09-23T03:16:30.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.832+0000 7f4325021640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f432015f390 con 0x7f43200cb5b0 2023-09-23T03:16:30.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.832+0000 7f431ed9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f432015aba0 0x7f432015bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:30.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.833+0000 7f431ed9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f432015aba0 0x7f432015bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46574/0 (socket says 172.21.15.154:46574) 2023-09-23T03:16:30.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.833+0000 7f431ed9d640 1 -- 172.21.15.154:0/2236965588 learned_addr learned my addr 172.21.15.154:0/2236965588 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:30.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.833+0000 7f431f59e640 1 --2- 172.21.15.154:0/2236965588 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43200cb5b0 0x7f432015ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:30.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.833+0000 7f431ed9d640 1 -- 172.21.15.154:0/2236965588 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43200cb5b0 msgr2=0x7f432015ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:30.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.833+0000 7f431ed9d640 1 --2- 172.21.15.154:0/2236965588 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43200cb5b0 0x7f432015ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:30.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.833+0000 7f431ed9d640 1 -- 172.21.15.154:0/2236965588 >> v1:172.21.15.154:6789/0 conn(0x7f432015e5f0 legacy=0x7f43200caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:30.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.833+0000 7f431ed9d640 1 -- 172.21.15.154:0/2236965588 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432015b330 con 0x7f432015aba0 2023-09-23T03:16:30.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.834+0000 7f431ed9d640 1 --2- 172.21.15.154:0/2236965588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f432015aba0 0x7f432015bb30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4310009870 tx=0x7f431002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1442e03a143da88 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:30.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.834+0000 7f431e59c640 1 -- 172.21.15.154:0/2236965588 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4310030840 con 0x7f432015aba0 2023-09-23T03:16:30.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.834+0000 7f431e59c640 1 -- 172.21.15.154:0/2236965588 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43100309a0 con 0x7f432015aba0 2023-09-23T03:16:30.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.834+0000 7f431e59c640 1 -- 172.21.15.154:0/2236965588 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4310030d30 con 0x7f432015aba0 2023-09-23T03:16:30.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.835+0000 7f4325021640 1 -- 172.21.15.154:0/2236965588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f432015aba0 msgr2=0x7f432015bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:30.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.835+0000 7f4325021640 1 --2- 172.21.15.154:0/2236965588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f432015aba0 0x7f432015bb30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4310009870 tx=0x7f431002b980 comp rx=0 tx=0).stop 2023-09-23T03:16:30.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.835+0000 7f4325021640 1 -- 172.21.15.154:0/2236965588 shutdown_connections 2023-09-23T03:16:30.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.835+0000 7f4325021640 1 --2- 172.21.15.154:0/2236965588 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43200cb5b0 0x7f432015ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:30.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.835+0000 7f4325021640 1 --2- 172.21.15.154:0/2236965588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f432015aba0 0x7f432015bb30 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:30.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.835+0000 7f4325021640 1 -- 172.21.15.154:0/2236965588 >> 172.21.15.154:0/2236965588 conn(0x7f432006c010 msgr2=0x7f432006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:30.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.835+0000 7f4325021640 1 -- 172.21.15.154:0/2236965588 shutdown_connections 2023-09-23T03:16:30.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.835+0000 7f4325021640 1 -- 172.21.15.154:0/2236965588 wait complete. 2023-09-23T03:16:30.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.837+0000 7f4325021640 1 Processor -- start 2023-09-23T03:16:30.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.837+0000 7f4325021640 1 -- start start 2023-09-23T03:16:30.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.838+0000 7f4325021640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43200cb5b0 0x7f43201f1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:30.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.838+0000 7f4325021640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f432015aba0 0x7f43200c5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:30.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.838+0000 7f4325021640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f432015e5f0 0x7f43200ac910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:30.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.838+0000 7f4325021640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4320142340 con 0x7f432015aba0 2023-09-23T03:16:30.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.838+0000 7f4325021640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43201431a0 con 0x7f432015e5f0 2023-09-23T03:16:30.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.838+0000 7f4325021640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4320142590 con 0x7f43200cb5b0 2023-09-23T03:16:30.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.838+0000 7f4317fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f432015aba0 0x7f43200c5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:30.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.838+0000 7f4317fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f432015aba0 0x7f43200c5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46578/0 (socket says 172.21.15.154:46578) 2023-09-23T03:16:30.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.838+0000 7f4317fff640 1 -- 172.21.15.154:0/2848229094 learned_addr learned my addr 172.21.15.154:0/2848229094 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:30.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.839+0000 7f4317fff640 1 -- 172.21.15.154:0/2848229094 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43200cb5b0 msgr2=0x7f43201f1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:30.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.839+0000 7f4317fff640 1 --2- 172.21.15.154:0/2848229094 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43200cb5b0 0x7f43201f1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:30.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.839+0000 7f4317fff640 1 -- 172.21.15.154:0/2848229094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f432015e5f0 msgr2=0x7f43200ac910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:30.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.839+0000 7f4317fff640 1 --2- 172.21.15.154:0/2848229094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f432015e5f0 0x7f43200ac910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:30.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.839+0000 7f4317fff640 1 -- 172.21.15.154:0/2848229094 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43100094d0 con 0x7f432015aba0 2023-09-23T03:16:30.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.839+0000 7f4317fff640 1 --2- 172.21.15.154:0/2848229094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f432015aba0 0x7f43200c5b60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f430800c400 tx=0x7f430800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:30.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.839+0000 7f43177fe640 1 -- 172.21.15.154:0/2848229094 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43080052d0 con 0x7f432015aba0 2023-09-23T03:16:30.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.839+0000 7f43177fe640 1 -- 172.21.15.154:0/2848229094 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4308005430 con 0x7f432015aba0 2023-09-23T03:16:30.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.839+0000 7f43177fe640 1 -- 172.21.15.154:0/2848229094 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4308003830 con 0x7f432015aba0 2023-09-23T03:16:30.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.839+0000 7f4325021640 1 -- 172.21.15.154:0/2848229094 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f432015b100 con 0x7f432015aba0 2023-09-23T03:16:30.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.839+0000 7f4325021640 1 -- 172.21.15.154:0/2848229094 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4320142b50 con 0x7f432015aba0 2023-09-23T03:16:30.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.840+0000 7f4325021640 1 -- 172.21.15.154:0/2848229094 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f432004e920 con 0x7f432015aba0 2023-09-23T03:16:30.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.841+0000 7f43177fe640 1 -- 172.21.15.154:0/2848229094 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f4308003990 con 0x7f432015aba0 2023-09-23T03:16:30.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.842+0000 7f43177fe640 1 --2- 172.21.15.154:0/2848229094 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f430006af30 0x7f430006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:30.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.842+0000 7f43177fe640 1 -- 172.21.15.154:0/2848229094 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f43080890d0 con 0x7f432015aba0 2023-09-23T03:16:30.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.843+0000 7f431ed9d640 1 --2- 172.21.15.154:0/2848229094 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f430006af30 0x7f430006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:30.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.843+0000 7f431ed9d640 1 --2- 172.21.15.154:0/2848229094 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f430006af30 0x7f430006d3f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4310036000 tx=0x7f431003b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:30.848 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:30.847+0000 7f43177fe640 1 -- 172.21.15.154:0/2848229094 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f4308053d50 con 0x7f432015aba0 2023-09-23T03:16:31.036 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.035+0000 7f4325021640 1 -- 172.21.15.154:0/2848229094 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f43200cbc10 con 0x7f432015aba0 2023-09-23T03:16:31.037 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.036+0000 7f43177fe640 1 -- 172.21.15.154:0/2848229094 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4308057370 con 0x7f432015aba0 2023-09-23T03:16:31.037 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:16:31.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.039+0000 7f4325021640 1 -- 172.21.15.154:0/2848229094 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f430006af30 msgr2=0x7f430006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:31.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.039+0000 7f4325021640 1 --2- 172.21.15.154:0/2848229094 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f430006af30 0x7f430006d3f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4310036000 tx=0x7f431003b040 comp rx=0 tx=0).stop 2023-09-23T03:16:31.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.039+0000 7f4325021640 1 -- 172.21.15.154:0/2848229094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f432015aba0 msgr2=0x7f43200c5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:31.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.039+0000 7f4325021640 1 --2- 172.21.15.154:0/2848229094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f432015aba0 0x7f43200c5b60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f430800c400 tx=0x7f430800c8c0 comp rx=0 tx=0).stop 2023-09-23T03:16:31.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.039+0000 7f4325021640 1 -- 172.21.15.154:0/2848229094 shutdown_connections 2023-09-23T03:16:31.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.039+0000 7f4325021640 1 --2- 172.21.15.154:0/2848229094 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f430006af30 0x7f430006d3f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:31.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.039+0000 7f4325021640 1 --2- 172.21.15.154:0/2848229094 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f432015e5f0 0x7f43200ac910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:31.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.039+0000 7f4325021640 1 --2- 172.21.15.154:0/2848229094 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f432015aba0 0x7f43200c5b60 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:31.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.039+0000 7f4325021640 1 --2- 172.21.15.154:0/2848229094 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43200cb5b0 0x7f43201f1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:31.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.039+0000 7f4325021640 1 -- 172.21.15.154:0/2848229094 >> 172.21.15.154:0/2848229094 conn(0x7f432006c010 msgr2=0x7f43200c20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:31.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.039+0000 7f4325021640 1 -- 172.21.15.154:0/2848229094 shutdown_connections 2023-09-23T03:16:31.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.039+0000 7f4325021640 1 -- 172.21.15.154:0/2848229094 wait complete. 2023-09-23T03:16:31.059 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:16:31.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.280+0000 7f8e156c3640 1 Processor -- start 2023-09-23T03:16:31.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.281+0000 7f8e156c3640 1 -- start start 2023-09-23T03:16:31.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.282+0000 7f8e156c3640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8e1015df50 0x7f8e100b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:31.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.282+0000 7f8e156c3640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8e100d1890 0x7f8e100cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:31.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.282+0000 7f8e156c3640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e1015e330 con 0x7f8e100d0980 2023-09-23T03:16:31.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.282+0000 7f8e156c3640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e100cd870 con 0x7f8e1015df50 2023-09-23T03:16:31.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.282+0000 7f8e156c3640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e100d0e10 con 0x7f8e100d1890 2023-09-23T03:16:31.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.282+0000 7f8e0ffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8e1015df50 0x7f8e100b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:31.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.283+0000 7f8e0ffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8e1015df50 0x7f8e100b20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46588/0 (socket says 172.21.15.154:46588) 2023-09-23T03:16:31.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.283+0000 7f8e0ffff640 1 -- 172.21.15.154:0/963042564 learned_addr learned my addr 172.21.15.154:0/963042564 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:31.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.283+0000 7f8e0ffff640 1 -- 172.21.15.154:0/963042564 >> v2:172.21.15.190:3300/0 conn(0x7f8e100d1890 msgr2=0x7f8e100cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:31.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.283+0000 7f8e14ec2640 1 --2- 172.21.15.154:0/963042564 >> v2:172.21.15.190:3300/0 conn(0x7f8e100d1890 0x7f8e100cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:31.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.283+0000 7f8e0ffff640 1 --2- 172.21.15.154:0/963042564 >> v2:172.21.15.190:3300/0 conn(0x7f8e100d1890 0x7f8e100cdcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:31.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.283+0000 7f8e0ffff640 1 -- 172.21.15.154:0/963042564 >> v1:172.21.15.190:6789/0 conn(0x7f8e100d0980 legacy=0x7f8e100d1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:31.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.283+0000 7f8e0ffff640 1 -- 172.21.15.154:0/963042564 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e100b4c50 con 0x7f8e1015df50 2023-09-23T03:16:31.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.284+0000 7f8e0ffff640 1 --2- 172.21.15.154:0/963042564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e1015df50 0x7f8e100b20f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8e101607c0 tx=0x7f8e0002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b95939d63c6646d1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:31.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.284+0000 7f8e0f7fe640 1 -- 172.21.15.154:0/963042564 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e000317e0 con 0x7f8e1015df50 2023-09-23T03:16:31.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.284+0000 7f8e0f7fe640 1 -- 172.21.15.154:0/963042564 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e00031940 con 0x7f8e1015df50 2023-09-23T03:16:31.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.284+0000 7f8e0f7fe640 1 -- 172.21.15.154:0/963042564 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e00031cd0 con 0x7f8e1015df50 2023-09-23T03:16:31.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.285+0000 7f8e156c3640 1 -- 172.21.15.154:0/963042564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e1015df50 msgr2=0x7f8e100b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:31.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.285+0000 7f8e156c3640 1 --2- 172.21.15.154:0/963042564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e1015df50 0x7f8e100b20f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8e101607c0 tx=0x7f8e0002b880 comp rx=0 tx=0).stop 2023-09-23T03:16:31.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.285+0000 7f8e156c3640 1 -- 172.21.15.154:0/963042564 shutdown_connections 2023-09-23T03:16:31.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.285+0000 7f8e156c3640 1 --2- 172.21.15.154:0/963042564 >> v2:172.21.15.190:3300/0 conn(0x7f8e100d1890 0x7f8e100cdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:31.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.285+0000 7f8e156c3640 1 --2- 172.21.15.154:0/963042564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e1015df50 0x7f8e100b20f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:31.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.285+0000 7f8e156c3640 1 -- 172.21.15.154:0/963042564 >> 172.21.15.154:0/963042564 conn(0x7f8e1009bb20 msgr2=0x7f8e100d2870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:31.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.285+0000 7f8e156c3640 1 -- 172.21.15.154:0/963042564 shutdown_connections 2023-09-23T03:16:31.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.285+0000 7f8e156c3640 1 -- 172.21.15.154:0/963042564 wait complete. 2023-09-23T03:16:31.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.287+0000 7f8e156c3640 1 Processor -- start 2023-09-23T03:16:31.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.287+0000 7f8e156c3640 1 -- start start 2023-09-23T03:16:31.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.288+0000 7f8e156c3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e100d1890 0x7f8e100c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:31.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.288+0000 7f8e156c3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e1015df50 0x7f8e100c6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:31.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.288+0000 7f8e156c3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e100cc6c0 0x7f8e1015f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:31.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.288+0000 7f8e156c3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e100ccb70 con 0x7f8e100cc6c0 2023-09-23T03:16:31.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.288+0000 7f8e156c3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e101f3520 con 0x7f8e100d1890 2023-09-23T03:16:31.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.288+0000 7f8e156c3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e100c8740 con 0x7f8e1015df50 2023-09-23T03:16:31.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.288+0000 7f8e14ec2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e100cc6c0 0x7f8e1015f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:31.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.288+0000 7f8e14ec2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e100cc6c0 0x7f8e1015f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46594/0 (socket says 172.21.15.154:46594) 2023-09-23T03:16:31.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.288+0000 7f8e14ec2640 1 -- 172.21.15.154:0/2368987120 learned_addr learned my addr 172.21.15.154:0/2368987120 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:31.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.289+0000 7f8e0ffff640 1 --2- 172.21.15.154:0/2368987120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e100d1890 0x7f8e100c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.289+0000 7f8e14ec2640 1 -- 172.21.15.154:0/2368987120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e1015df50 msgr2=0x7f8e100c6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.289+0000 7f8e14ec2640 1 --2- 172.21.15.154:0/2368987120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e1015df50 0x7f8e100c6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.289+0000 7f8e14ec2640 1 -- 172.21.15.154:0/2368987120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e100d1890 msgr2=0x7f8e100c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.289+0000 7f8e14ec2640 1 --2- 172.21.15.154:0/2368987120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e100d1890 0x7f8e100c80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.289+0000 7f8e14ec2640 1 -- 172.21.15.154:0/2368987120 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e000094d0 con 0x7f8e100cc6c0 2023-09-23T03:16:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.289+0000 7f8e14ec2640 1 --2- 172.21.15.154:0/2368987120 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e100cc6c0 0x7f8e1015f530 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8e0800cda0 tx=0x7f8e0800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.289+0000 7f8e0d7fa640 1 -- 172.21.15.154:0/2368987120 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e08003b90 con 0x7f8e100cc6c0 2023-09-23T03:16:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.289+0000 7f8e0d7fa640 1 -- 172.21.15.154:0/2368987120 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e08003cf0 con 0x7f8e100cc6c0 2023-09-23T03:16:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.290+0000 7f8e0d7fa640 1 -- 172.21.15.154:0/2368987120 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e08004240 con 0x7f8e100cc6c0 2023-09-23T03:16:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.290+0000 7f8e156c3640 1 -- 172.21.15.154:0/2368987120 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e100c8a20 con 0x7f8e100cc6c0 2023-09-23T03:16:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.290+0000 7f8e156c3640 1 -- 172.21.15.154:0/2368987120 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e100c8fe0 con 0x7f8e100cc6c0 2023-09-23T03:16:31.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.291+0000 7f8e156c3640 1 -- 172.21.15.154:0/2368987120 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e101f36f0 con 0x7f8e100cc6c0 2023-09-23T03:16:31.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.292+0000 7f8e0d7fa640 1 -- 172.21.15.154:0/2368987120 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f8e080043a0 con 0x7f8e100cc6c0 2023-09-23T03:16:31.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.293+0000 7f8e0d7fa640 1 --2- 172.21.15.154:0/2368987120 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8df006af30 0x7f8df006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:31.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.293+0000 7f8e0d7fa640 1 -- 172.21.15.154:0/2368987120 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f8e08012070 con 0x7f8e100cc6c0 2023-09-23T03:16:31.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.293+0000 7f8e0ffff640 1 --2- 172.21.15.154:0/2368987120 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8df006af30 0x7f8df006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:31.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.294+0000 7f8e0ffff640 1 --2- 172.21.15.154:0/2368987120 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8df006af30 0x7f8df006d3f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8e00036000 tx=0x7f8e0003d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:31.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.298+0000 7f8e0d7fa640 1 -- 172.21.15.154:0/2368987120 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f8e101f36f0 con 0x7f8e100cc6c0 2023-09-23T03:16:31.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.489+0000 7f8e156c3640 1 -- 172.21.15.154:0/2368987120 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8e101f36f0 con 0x7f8e100cc6c0 2023-09-23T03:16:31.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.490+0000 7f8e0d7fa640 1 -- 172.21.15.154:0/2368987120 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8e101f36f0 con 0x7f8e100cc6c0 2023-09-23T03:16:31.492 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:16:31.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.493+0000 7f8e156c3640 1 -- 172.21.15.154:0/2368987120 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8df006af30 msgr2=0x7f8df006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:31.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.493+0000 7f8e156c3640 1 --2- 172.21.15.154:0/2368987120 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8df006af30 0x7f8df006d3f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8e00036000 tx=0x7f8e0003d040 comp rx=0 tx=0).stop 2023-09-23T03:16:31.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.493+0000 7f8e156c3640 1 -- 172.21.15.154:0/2368987120 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e100cc6c0 msgr2=0x7f8e1015f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:31.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.493+0000 7f8e156c3640 1 --2- 172.21.15.154:0/2368987120 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e100cc6c0 0x7f8e1015f530 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8e0800cda0 tx=0x7f8e0800a660 comp rx=0 tx=0).stop 2023-09-23T03:16:31.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.493+0000 7f8e156c3640 1 -- 172.21.15.154:0/2368987120 shutdown_connections 2023-09-23T03:16:31.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.493+0000 7f8e156c3640 1 --2- 172.21.15.154:0/2368987120 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8df006af30 0x7f8df006d3f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:31.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.493+0000 7f8e156c3640 1 --2- 172.21.15.154:0/2368987120 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e100cc6c0 0x7f8e1015f530 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:31.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.493+0000 7f8e156c3640 1 --2- 172.21.15.154:0/2368987120 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e1015df50 0x7f8e100c6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:31.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.493+0000 7f8e156c3640 1 --2- 172.21.15.154:0/2368987120 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e100d1890 0x7f8e100c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:31.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.493+0000 7f8e156c3640 1 -- 172.21.15.154:0/2368987120 >> 172.21.15.154:0/2368987120 conn(0x7f8e1009bb20 msgr2=0x7f8e1006c580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:31.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.493+0000 7f8e156c3640 1 -- 172.21.15.154:0/2368987120 shutdown_connections 2023-09-23T03:16:31.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.493+0000 7f8e156c3640 1 -- 172.21.15.154:0/2368987120 wait complete. 2023-09-23T03:16:31.514 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:16:31.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.729+0000 7f8b19524640 1 Processor -- start 2023-09-23T03:16:31.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.729+0000 7f8b19524640 1 -- start start 2023-09-23T03:16:31.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.731+0000 7f8b19524640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8b140c8d70 0x7f8b1415c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:31.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.731+0000 7f8b19524640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b140c9150 0x7f8b1415cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:31.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.731+0000 7f8b19524640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8b14071290 0x7f8b140ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:31.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.731+0000 7f8b19524640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b140c9d20 con 0x7f8b140c8d70 2023-09-23T03:16:31.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.731+0000 7f8b19524640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b14071850 con 0x7f8b14071290 2023-09-23T03:16:31.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.731+0000 7f8b19524640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b14071b30 con 0x7f8b140c9150 2023-09-23T03:16:31.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.731+0000 7f8b12ffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8b140c8d70 0x7f8b1415c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:31.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.731+0000 7f8b12ffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8b140c8d70 0x7f8b1415c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46604/0 (socket says 172.21.15.154:46604) 2023-09-23T03:16:31.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.731+0000 7f8b12ffd640 1 -- 172.21.15.154:0/2273762032 learned_addr learned my addr 172.21.15.154:0/2273762032 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:31.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.731+0000 7f8b12ffd640 1 -- 172.21.15.154:0/2273762032 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b140c9150 msgr2=0x7f8b1415cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:31.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.731+0000 7f8b127fc640 1 --2- 172.21.15.154:0/2273762032 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b140c9150 0x7f8b1415cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:31.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.731+0000 7f8b12ffd640 1 --2- 172.21.15.154:0/2273762032 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b140c9150 0x7f8b1415cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:31.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.731+0000 7f8b137fe640 1 --2- 172.21.15.154:0/2273762032 >> v2:172.21.15.190:3300/0 conn(0x7f8b14071290 0x7f8b140ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:31.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.731+0000 7f8b12ffd640 1 -- 172.21.15.154:0/2273762032 >> v2:172.21.15.190:3300/0 conn(0x7f8b14071290 msgr2=0x7f8b140ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:31.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.732+0000 7f8b12ffd640 1 --2- 172.21.15.154:0/2273762032 >> v2:172.21.15.190:3300/0 conn(0x7f8b14071290 0x7f8b140ca1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:31.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.732+0000 7f8b12ffd640 1 -- 172.21.15.154:0/2273762032 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b14071e10 con 0x7f8b140c8d70 2023-09-23T03:16:31.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.732+0000 7f8b127fc640 1 --2- 172.21.15.154:0/2273762032 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b140c9150 0x7f8b1415cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:31.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.732+0000 7f8b137fe640 1 --2- 172.21.15.154:0/2273762032 >> v2:172.21.15.190:3300/0 conn(0x7f8b14071290 0x7f8b140ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:31.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.733+0000 7f8b12ffd640 1 --2- 172.21.15.154:0/2273762032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b140c8d70 0x7f8b1415c6d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8b140c95d0 tx=0x7f8b0402b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a823c1cee69ecf server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:31.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.733+0000 7f8b11ffb640 1 -- 172.21.15.154:0/2273762032 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b040317f0 con 0x7f8b140c8d70 2023-09-23T03:16:31.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.733+0000 7f8b11ffb640 1 -- 172.21.15.154:0/2273762032 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b04031950 con 0x7f8b140c8d70 2023-09-23T03:16:31.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.733+0000 7f8b11ffb640 1 -- 172.21.15.154:0/2273762032 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b04031ce0 con 0x7f8b140c8d70 2023-09-23T03:16:31.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.734+0000 7f8b19524640 1 -- 172.21.15.154:0/2273762032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b140c8d70 msgr2=0x7f8b1415c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:31.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.734+0000 7f8b19524640 1 --2- 172.21.15.154:0/2273762032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b140c8d70 0x7f8b1415c6d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8b140c95d0 tx=0x7f8b0402b890 comp rx=0 tx=0).stop 2023-09-23T03:16:31.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.734+0000 7f8b19524640 1 -- 172.21.15.154:0/2273762032 shutdown_connections 2023-09-23T03:16:31.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.734+0000 7f8b19524640 1 --2- 172.21.15.154:0/2273762032 >> v2:172.21.15.190:3300/0 conn(0x7f8b14071290 0x7f8b140ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:31.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.734+0000 7f8b19524640 1 --2- 172.21.15.154:0/2273762032 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b140c9150 0x7f8b1415cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:31.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.734+0000 7f8b19524640 1 --2- 172.21.15.154:0/2273762032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b140c8d70 0x7f8b1415c6d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:31.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.734+0000 7f8b19524640 1 -- 172.21.15.154:0/2273762032 >> 172.21.15.154:0/2273762032 conn(0x7f8b1406c010 msgr2=0x7f8b1406c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:31.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.734+0000 7f8b19524640 1 -- 172.21.15.154:0/2273762032 shutdown_connections 2023-09-23T03:16:31.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.734+0000 7f8b19524640 1 -- 172.21.15.154:0/2273762032 wait complete. 2023-09-23T03:16:31.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.735+0000 7f8b19524640 1 Processor -- start 2023-09-23T03:16:31.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.736+0000 7f8b19524640 1 -- start start 2023-09-23T03:16:31.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.737+0000 7f8b19524640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b14071290 0x7f8b140ad160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:31.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.737+0000 7f8b19524640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b140c8d70 0x7f8b141f1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:31.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.737+0000 7f8b19524640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b140c9150 0x7f8b140cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:31.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.737+0000 7f8b19524640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b140ce070 con 0x7f8b14071290 2023-09-23T03:16:31.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.737+0000 7f8b19524640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b140ad6a0 con 0x7f8b140c8d70 2023-09-23T03:16:31.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.737+0000 7f8b19524640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b140c2a50 con 0x7f8b140c9150 2023-09-23T03:16:31.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.737+0000 7f8b12ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b14071290 0x7f8b140ad160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:31.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.737+0000 7f8b12ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b14071290 0x7f8b140ad160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46618/0 (socket says 172.21.15.154:46618) 2023-09-23T03:16:31.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.737+0000 7f8b12ffd640 1 -- 172.21.15.154:0/2685560415 learned_addr learned my addr 172.21.15.154:0/2685560415 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:31.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.737+0000 7f8b137fe640 1 --2- 172.21.15.154:0/2685560415 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b140c9150 0x7f8b140cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:31.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.737+0000 7f8b127fc640 1 --2- 172.21.15.154:0/2685560415 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b140c8d70 0x7f8b141f1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:31.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.737+0000 7f8b12ffd640 1 -- 172.21.15.154:0/2685560415 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b140c9150 msgr2=0x7f8b140cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:31.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.737+0000 7f8b12ffd640 1 --2- 172.21.15.154:0/2685560415 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b140c9150 0x7f8b140cd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:31.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.737+0000 7f8b12ffd640 1 -- 172.21.15.154:0/2685560415 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b140c8d70 msgr2=0x7f8b141f1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:31.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.737+0000 7f8b12ffd640 1 --2- 172.21.15.154:0/2685560415 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b140c8d70 0x7f8b141f1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:31.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.737+0000 7f8b12ffd640 1 -- 172.21.15.154:0/2685560415 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b040094d0 con 0x7f8b14071290 2023-09-23T03:16:31.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.737+0000 7f8b137fe640 1 --2- 172.21.15.154:0/2685560415 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b140c9150 0x7f8b140cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:31.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.737+0000 7f8b127fc640 1 --2- 172.21.15.154:0/2685560415 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b140c8d70 0x7f8b141f1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:31.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.738+0000 7f8b12ffd640 1 --2- 172.21.15.154:0/2685560415 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b14071290 0x7f8b140ad160 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8b04000c00 tx=0x7f8b040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:31.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.738+0000 7f8afb7fe640 1 -- 172.21.15.154:0/2685560415 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b04032910 con 0x7f8b14071290 2023-09-23T03:16:31.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.738+0000 7f8afb7fe640 1 -- 172.21.15.154:0/2685560415 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b04032a70 con 0x7f8b14071290 2023-09-23T03:16:31.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.738+0000 7f8b19524640 1 -- 172.21.15.154:0/2685560415 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b140c2d30 con 0x7f8b14071290 2023-09-23T03:16:31.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.738+0000 7f8afb7fe640 1 -- 172.21.15.154:0/2685560415 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b04032e20 con 0x7f8b14071290 2023-09-23T03:16:31.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.738+0000 7f8b19524640 1 -- 172.21.15.154:0/2685560415 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b140c32f0 con 0x7f8b14071290 2023-09-23T03:16:31.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.739+0000 7f8b19524640 1 -- 172.21.15.154:0/2685560415 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b1404e920 con 0x7f8b14071290 2023-09-23T03:16:31.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.740+0000 7f8afb7fe640 1 -- 172.21.15.154:0/2685560415 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f8b0404e450 con 0x7f8b14071290 2023-09-23T03:16:31.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.741+0000 7f8afb7fe640 1 --2- 172.21.15.154:0/2685560415 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8b0006af30 0x7f8b0006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:31.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.741+0000 7f8afb7fe640 1 -- 172.21.15.154:0/2685560415 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f8b040abfe0 con 0x7f8b14071290 2023-09-23T03:16:31.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.742+0000 7f8b127fc640 1 --2- 172.21.15.154:0/2685560415 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8b0006af30 0x7f8b0006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:31.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.742+0000 7f8b127fc640 1 --2- 172.21.15.154:0/2685560415 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8b0006af30 0x7f8b0006d3f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8b08006fd0 tx=0x7f8b08008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:31.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.746+0000 7f8afb7fe640 1 -- 172.21.15.154:0/2685560415 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f8b04076de0 con 0x7f8b14071290 2023-09-23T03:16:31.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.934+0000 7f8b19524640 1 -- 172.21.15.154:0/2685560415 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8b1406cbc0 con 0x7f8b14071290 2023-09-23T03:16:31.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.935+0000 7f8afb7fe640 1 -- 172.21.15.154:0/2685560415 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8b0407a400 con 0x7f8b14071290 2023-09-23T03:16:31.936 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:16:31.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.938+0000 7f8b19524640 1 -- 172.21.15.154:0/2685560415 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8b0006af30 msgr2=0x7f8b0006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:31.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.938+0000 7f8b19524640 1 --2- 172.21.15.154:0/2685560415 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8b0006af30 0x7f8b0006d3f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8b08006fd0 tx=0x7f8b08008040 comp rx=0 tx=0).stop 2023-09-23T03:16:31.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.938+0000 7f8b19524640 1 -- 172.21.15.154:0/2685560415 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b14071290 msgr2=0x7f8b140ad160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:31.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.938+0000 7f8b19524640 1 --2- 172.21.15.154:0/2685560415 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b14071290 0x7f8b140ad160 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8b04000c00 tx=0x7f8b040046c0 comp rx=0 tx=0).stop 2023-09-23T03:16:31.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.938+0000 7f8b19524640 1 -- 172.21.15.154:0/2685560415 shutdown_connections 2023-09-23T03:16:31.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.938+0000 7f8b19524640 1 --2- 172.21.15.154:0/2685560415 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8b0006af30 0x7f8b0006d3f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:31.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.938+0000 7f8b19524640 1 --2- 172.21.15.154:0/2685560415 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b140c9150 0x7f8b140cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:31.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.938+0000 7f8b19524640 1 --2- 172.21.15.154:0/2685560415 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b140c8d70 0x7f8b141f1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:31.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.938+0000 7f8b19524640 1 --2- 172.21.15.154:0/2685560415 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b14071290 0x7f8b140ad160 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:31.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.938+0000 7f8b19524640 1 -- 172.21.15.154:0/2685560415 >> 172.21.15.154:0/2685560415 conn(0x7f8b1406c010 msgr2=0x7f8b140cb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:31.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.938+0000 7f8b19524640 1 -- 172.21.15.154:0/2685560415 shutdown_connections 2023-09-23T03:16:31.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:31.938+0000 7f8b19524640 1 -- 172.21.15.154:0/2685560415 wait complete. 2023-09-23T03:16:31.959 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-23T03:16:32.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.183+0000 7f95579ee640 1 Processor -- start 2023-09-23T03:16:32.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.183+0000 7f95579ee640 1 -- start start 2023-09-23T03:16:32.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.185+0000 7f95579ee640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f955015bb30 0x7f95500caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:32.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.185+0000 7f95579ee640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f955015b6a0 con 0x7f95500cb3e0 2023-09-23T03:16:32.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.185+0000 7f95579ee640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f955015f0b0 con 0x7f955015aba0 2023-09-23T03:16:32.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.185+0000 7f95579ee640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95500cb870 con 0x7f955015bb30 2023-09-23T03:16:32.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.185+0000 7f9554f89640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f955015bb30 0x7f95500caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:32.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.185+0000 7f9554f89640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f955015bb30 0x7f95500caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46632/0 (socket says 172.21.15.154:46632) 2023-09-23T03:16:32.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.185+0000 7f9554f89640 1 -- 172.21.15.154:0/2821285229 learned_addr learned my addr 172.21.15.154:0/2821285229 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:32.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.186+0000 7f9554f89640 1 -- 172.21.15.154:0/2821285229 >> v1:172.21.15.190:6789/0 conn(0x7f955015aba0 legacy=0x7f955015e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:32.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.186+0000 7f9554f89640 1 -- 172.21.15.154:0/2821285229 >> v1:172.21.15.154:6789/0 conn(0x7f95500cb3e0 legacy=0x7f95500cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:32.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.186+0000 7f9554f89640 1 -- 172.21.15.154:0/2821285229 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f955015f390 con 0x7f955015bb30 2023-09-23T03:16:32.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.187+0000 7f9554f89640 1 --2- 172.21.15.154:0/2821285229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f955015bb30 0x7f95500caea0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9540009870 tx=0x7f954002ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15416bec9f619e4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:32.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.187+0000 7f9547fff640 1 -- 172.21.15.154:0/2821285229 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954002f850 con 0x7f955015bb30 2023-09-23T03:16:32.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.187+0000 7f9547fff640 1 -- 172.21.15.154:0/2821285229 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954002f9b0 con 0x7f955015bb30 2023-09-23T03:16:32.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.187+0000 7f9547fff640 1 -- 172.21.15.154:0/2821285229 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954002fd40 con 0x7f955015bb30 2023-09-23T03:16:32.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.188+0000 7f95579ee640 1 -- 172.21.15.154:0/2821285229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f955015bb30 msgr2=0x7f95500caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:32.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.188+0000 7f95579ee640 1 --2- 172.21.15.154:0/2821285229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f955015bb30 0x7f95500caea0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9540009870 tx=0x7f954002ba30 comp rx=0 tx=0).stop 2023-09-23T03:16:32.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.188+0000 7f95579ee640 1 -- 172.21.15.154:0/2821285229 shutdown_connections 2023-09-23T03:16:32.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.188+0000 7f95579ee640 1 --2- 172.21.15.154:0/2821285229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f955015bb30 0x7f95500caea0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:32.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.188+0000 7f95579ee640 1 -- 172.21.15.154:0/2821285229 >> 172.21.15.154:0/2821285229 conn(0x7f955006c010 msgr2=0x7f955006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:32.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.188+0000 7f95579ee640 1 -- 172.21.15.154:0/2821285229 shutdown_connections 2023-09-23T03:16:32.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.188+0000 7f95579ee640 1 -- 172.21.15.154:0/2821285229 wait complete. 2023-09-23T03:16:32.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.190+0000 7f95579ee640 1 Processor -- start 2023-09-23T03:16:32.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.190+0000 7f95579ee640 1 -- start start 2023-09-23T03:16:32.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.191+0000 7f95579ee640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95500cb3e0 0x7f95500c8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:32.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.191+0000 7f95579ee640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f955015aba0 0x7f95500c4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:32.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.191+0000 7f95579ee640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f955015bb30 0x7f9550148630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:32.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.191+0000 7f95579ee640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9550149390 con 0x7f95500cb3e0 2023-09-23T03:16:32.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.191+0000 7f95579ee640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95500c5690 con 0x7f955015bb30 2023-09-23T03:16:32.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.191+0000 7f95579ee640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95500c5970 con 0x7f955015aba0 2023-09-23T03:16:32.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.191+0000 7f955578a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95500cb3e0 0x7f95500c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:32.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.191+0000 7f955578a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95500cb3e0 0x7f95500c8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46644/0 (socket says 172.21.15.154:46644) 2023-09-23T03:16:32.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.191+0000 7f955578a640 1 -- 172.21.15.154:0/800899625 learned_addr learned my addr 172.21.15.154:0/800899625 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:32.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.192+0000 7f955578a640 1 -- 172.21.15.154:0/800899625 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f955015aba0 msgr2=0x7f95500c4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:32.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.192+0000 7f955578a640 1 --2- 172.21.15.154:0/800899625 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f955015aba0 0x7f95500c4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:32.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.192+0000 7f9555f8b640 1 --2- 172.21.15.154:0/800899625 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f955015bb30 0x7f9550148630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:32.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.192+0000 7f955578a640 1 -- 172.21.15.154:0/800899625 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f955015bb30 msgr2=0x7f9550148630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:32.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.192+0000 7f955578a640 1 --2- 172.21.15.154:0/800899625 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f955015bb30 0x7f9550148630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:32.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.192+0000 7f955578a640 1 -- 172.21.15.154:0/800899625 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95400094d0 con 0x7f95500cb3e0 2023-09-23T03:16:32.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.192+0000 7f9555f8b640 1 --2- 172.21.15.154:0/800899625 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f955015bb30 0x7f9550148630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:32.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.192+0000 7f955578a640 1 --2- 172.21.15.154:0/800899625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95500cb3e0 0x7f95500c8740 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f954c00c450 tx=0x7f954c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:32.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.192+0000 7f9545ffb640 1 -- 172.21.15.154:0/800899625 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c0052d0 con 0x7f95500cb3e0 2023-09-23T03:16:32.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.192+0000 7f9545ffb640 1 -- 172.21.15.154:0/800899625 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954c005430 con 0x7f95500cb3e0 2023-09-23T03:16:32.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.192+0000 7f95579ee640 1 -- 172.21.15.154:0/800899625 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95500c5c50 con 0x7f95500cb3e0 2023-09-23T03:16:32.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.193+0000 7f9545ffb640 1 -- 172.21.15.154:0/800899625 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954c012870 con 0x7f95500cb3e0 2023-09-23T03:16:32.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.193+0000 7f95579ee640 1 -- 172.21.15.154:0/800899625 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95500c4770 con 0x7f95500cb3e0 2023-09-23T03:16:32.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.194+0000 7f95579ee640 1 -- 172.21.15.154:0/800899625 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95500c6170 con 0x7f95500cb3e0 2023-09-23T03:16:32.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.195+0000 7f9545ffb640 1 -- 172.21.15.154:0/800899625 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f954c0129d0 con 0x7f95500cb3e0 2023-09-23T03:16:32.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.196+0000 7f9545ffb640 1 --2- 172.21.15.154:0/800899625 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f953406af30 0x7f953406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:32.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.196+0000 7f9545ffb640 1 -- 172.21.15.154:0/800899625 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f954c089680 con 0x7f95500cb3e0 2023-09-23T03:16:32.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.197+0000 7f9554f89640 1 --2- 172.21.15.154:0/800899625 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f953406af30 0x7f953406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:32.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.197+0000 7f9554f89640 1 --2- 172.21.15.154:0/800899625 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f953406af30 0x7f953406d3f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9540036000 tx=0x7f9540039040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:32.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.201+0000 7f9545ffb640 1 -- 172.21.15.154:0/800899625 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f95500c6170 con 0x7f95500cb3e0 2023-09-23T03:16:32.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.392+0000 7f95579ee640 1 -- 172.21.15.154:0/800899625 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f95500c6170 con 0x7f95500cb3e0 2023-09-23T03:16:32.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.392+0000 7f9545ffb640 1 -- 172.21.15.154:0/800899625 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f95500c6170 con 0x7f95500cb3e0 2023-09-23T03:16:32.394 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:16:32.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.395+0000 7f95579ee640 1 -- 172.21.15.154:0/800899625 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f953406af30 msgr2=0x7f953406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:32.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.395+0000 7f95579ee640 1 --2- 172.21.15.154:0/800899625 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f953406af30 0x7f953406d3f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9540036000 tx=0x7f9540039040 comp rx=0 tx=0).stop 2023-09-23T03:16:32.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.395+0000 7f95579ee640 1 -- 172.21.15.154:0/800899625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95500cb3e0 msgr2=0x7f95500c8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:32.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.395+0000 7f95579ee640 1 --2- 172.21.15.154:0/800899625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95500cb3e0 0x7f95500c8740 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f954c00c450 tx=0x7f954c00c910 comp rx=0 tx=0).stop 2023-09-23T03:16:32.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.395+0000 7f95579ee640 1 -- 172.21.15.154:0/800899625 shutdown_connections 2023-09-23T03:16:32.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.395+0000 7f95579ee640 1 --2- 172.21.15.154:0/800899625 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f953406af30 0x7f953406d3f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:32.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.395+0000 7f95579ee640 1 --2- 172.21.15.154:0/800899625 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f955015bb30 0x7f9550148630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:32.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.395+0000 7f95579ee640 1 --2- 172.21.15.154:0/800899625 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f955015aba0 0x7f95500c4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:32.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.395+0000 7f95579ee640 1 --2- 172.21.15.154:0/800899625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95500cb3e0 0x7f95500c8740 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:32.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.395+0000 7f95579ee640 1 -- 172.21.15.154:0/800899625 >> 172.21.15.154:0/800899625 conn(0x7f955006c010 msgr2=0x7f955015d230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:32.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.395+0000 7f95579ee640 1 -- 172.21.15.154:0/800899625 shutdown_connections 2023-09-23T03:16:32.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.395+0000 7f95579ee640 1 -- 172.21.15.154:0/800899625 wait complete. 2023-09-23T03:16:32.416 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-23T03:16:32.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.630+0000 7fc499410640 1 Processor -- start 2023-09-23T03:16:32.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.630+0000 7fc499410640 1 -- start start 2023-09-23T03:16:32.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.631+0000 7fc499410640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc4940ada30 0x7fc4940ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:32.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.631+0000 7fc499410640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4940b2fe0 0x7fc4940cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:32.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.631+0000 7fc499410640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc49406ceb0 con 0x7fc4940cef60 2023-09-23T03:16:32.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.631+0000 7fc499410640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4940ade10 con 0x7fc4940ada30 2023-09-23T03:16:32.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.631+0000 7fc499410640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4940cf3b0 con 0x7fc4940b2fe0 2023-09-23T03:16:32.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.632+0000 7fc492ffd640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc4940ada30 0x7fc4940ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:32.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.632+0000 7fc4927fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4940b2fe0 0x7fc4940cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:32.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.632+0000 7fc492ffd640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc4940ada30 0x7fc4940ccaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.154:58834/0 (socket says 172.21.15.154:58834) 2023-09-23T03:16:32.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.632+0000 7fc492ffd640 1 -- 172.21.15.154:0/1219965747 learned_addr learned my addr 172.21.15.154:0/1219965747 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:32.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.633+0000 7fc4927fc640 1 -- 172.21.15.154:0/1219965747 >> v2:172.21.15.190:3300/0 conn(0x7fc4940ada30 msgr2=0x7fc4940ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:32.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.633+0000 7fc4927fc640 1 --2- 172.21.15.154:0/1219965747 >> v2:172.21.15.190:3300/0 conn(0x7fc4940ada30 0x7fc4940ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:32.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.633+0000 7fc4927fc640 1 -- 172.21.15.154:0/1219965747 >> v1:172.21.15.190:6789/0 conn(0x7fc4940cef60 legacy=0x7fc4940b18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:32.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.633+0000 7fc4927fc640 1 -- 172.21.15.154:0/1219965747 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc494159c90 con 0x7fc4940b2fe0 2023-09-23T03:16:32.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.633+0000 7fc491ffb640 1 -- 172.21.15.154:0/1219965747 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4274571451 0 0) 0x7fc48c001110 con 0x7fc4940cef60 2023-09-23T03:16:32.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.633+0000 7fc492ffd640 1 --2- 172.21.15.154:0/1219965747 >> v2:172.21.15.190:3300/0 conn(0x7fc4940ada30 0x7fc4940ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:16:32.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.634+0000 7fc4927fc640 1 --2- 172.21.15.154:0/1219965747 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4940b2fe0 0x7fc4940cea20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc47c0097a0 tx=0x7fc47c02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f057f08cf987786d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:32.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.635+0000 7fc491ffb640 1 -- 172.21.15.154:0/1219965747 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47c02d040 con 0x7fc4940b2fe0 2023-09-23T03:16:32.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.635+0000 7fc491ffb640 1 -- 172.21.15.154:0/1219965747 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc47c031810 con 0x7fc4940b2fe0 2023-09-23T03:16:32.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.635+0000 7fc491ffb640 1 -- 172.21.15.154:0/1219965747 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47c031ba0 con 0x7fc4940b2fe0 2023-09-23T03:16:32.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.635+0000 7fc499410640 1 -- 172.21.15.154:0/1219965747 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4940b2fe0 msgr2=0x7fc4940cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:32.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.635+0000 7fc499410640 1 --2- 172.21.15.154:0/1219965747 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4940b2fe0 0x7fc4940cea20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc47c0097a0 tx=0x7fc47c02b8b0 comp rx=0 tx=0).stop 2023-09-23T03:16:32.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.636+0000 7fc499410640 1 -- 172.21.15.154:0/1219965747 shutdown_connections 2023-09-23T03:16:32.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.636+0000 7fc499410640 1 --2- 172.21.15.154:0/1219965747 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4940b2fe0 0x7fc4940cea20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:32.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.636+0000 7fc499410640 1 --2- 172.21.15.154:0/1219965747 >> v2:172.21.15.190:3300/0 conn(0x7fc4940ada30 0x7fc4940ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:32.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.636+0000 7fc499410640 1 -- 172.21.15.154:0/1219965747 >> 172.21.15.154:0/1219965747 conn(0x7fc49406a0f0 msgr2=0x7fc49406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:32.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.636+0000 7fc499410640 1 -- 172.21.15.154:0/1219965747 shutdown_connections 2023-09-23T03:16:32.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.636+0000 7fc499410640 1 -- 172.21.15.154:0/1219965747 wait complete. 2023-09-23T03:16:32.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.637+0000 7fc499410640 1 Processor -- start 2023-09-23T03:16:32.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.638+0000 7fc499410640 1 -- start start 2023-09-23T03:16:32.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.639+0000 7fc499410640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4940ada30 0x7fc494155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:32.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.639+0000 7fc499410640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4940b2fe0 0x7fc49415b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:32.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.639+0000 7fc499410640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4940cef60 0x7fc49415b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:32.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.639+0000 7fc499410640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49415a230 con 0x7fc4940cef60 2023-09-23T03:16:32.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.639+0000 7fc499410640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4940c2fd0 con 0x7fc4940b2fe0 2023-09-23T03:16:32.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.639+0000 7fc499410640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4940c32b0 con 0x7fc4940ada30 2023-09-23T03:16:32.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.639+0000 7fc4937fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4940cef60 0x7fc49415b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:32.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.639+0000 7fc4937fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4940cef60 0x7fc49415b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46648/0 (socket says 172.21.15.154:46648) 2023-09-23T03:16:32.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.639+0000 7fc4937fe640 1 -- 172.21.15.154:0/942725987 learned_addr learned my addr 172.21.15.154:0/942725987 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:32.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.639+0000 7fc4927fc640 1 --2- 172.21.15.154:0/942725987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4940b2fe0 0x7fc49415b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:32.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.639+0000 7fc492ffd640 1 --2- 172.21.15.154:0/942725987 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4940ada30 0x7fc494155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:32.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.639+0000 7fc4937fe640 1 -- 172.21.15.154:0/942725987 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4940ada30 msgr2=0x7fc494155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:32.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.640+0000 7fc4937fe640 1 --2- 172.21.15.154:0/942725987 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4940ada30 0x7fc494155ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:32.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.640+0000 7fc4937fe640 1 -- 172.21.15.154:0/942725987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4940b2fe0 msgr2=0x7fc49415b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:32.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.640+0000 7fc4937fe640 1 --2- 172.21.15.154:0/942725987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4940b2fe0 0x7fc49415b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:32.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.640+0000 7fc4937fe640 1 -- 172.21.15.154:0/942725987 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc484009b30 con 0x7fc4940cef60 2023-09-23T03:16:32.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.640+0000 7fc4927fc640 1 --2- 172.21.15.154:0/942725987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4940b2fe0 0x7fc49415b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:16:32.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.640+0000 7fc4937fe640 1 --2- 172.21.15.154:0/942725987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4940cef60 0x7fc49415b830 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc48c00c390 tx=0x7fc48c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:32.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.640+0000 7fc48b7fe640 1 -- 172.21.15.154:0/942725987 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48c0035f0 con 0x7fc4940cef60 2023-09-23T03:16:32.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.640+0000 7fc48b7fe640 1 -- 172.21.15.154:0/942725987 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc48c003750 con 0x7fc4940cef60 2023-09-23T03:16:32.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.640+0000 7fc499410640 1 -- 172.21.15.154:0/942725987 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc47c009400 con 0x7fc4940cef60 2023-09-23T03:16:32.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.640+0000 7fc48b7fe640 1 -- 172.21.15.154:0/942725987 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48c003b00 con 0x7fc4940cef60 2023-09-23T03:16:32.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.640+0000 7fc499410640 1 -- 172.21.15.154:0/942725987 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4940c3870 con 0x7fc4940cef60 2023-09-23T03:16:32.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.641+0000 7fc499410640 1 -- 172.21.15.154:0/942725987 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc494073eb0 con 0x7fc4940cef60 2023-09-23T03:16:32.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.642+0000 7fc48b7fe640 1 -- 172.21.15.154:0/942725987 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fc48c003c60 con 0x7fc4940cef60 2023-09-23T03:16:32.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.643+0000 7fc48b7fe640 1 --2- 172.21.15.154:0/942725987 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc47406af30 0x7fc47406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:32.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.643+0000 7fc48b7fe640 1 -- 172.21.15.154:0/942725987 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7fc48c088fb0 con 0x7fc4940cef60 2023-09-23T03:16:32.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.644+0000 7fc492ffd640 1 --2- 172.21.15.154:0/942725987 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc47406af30 0x7fc47406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:32.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.644+0000 7fc492ffd640 1 --2- 172.21.15.154:0/942725987 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc47406af30 0x7fc47406d3f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc484009fd0 tx=0x7fc48400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:32.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.648+0000 7fc48b7fe640 1 -- 172.21.15.154:0/942725987 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fc48c053ba0 con 0x7fc4940cef60 2023-09-23T03:16:32.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.849+0000 7fc499410640 1 -- 172.21.15.154:0/942725987 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc49406ceb0 con 0x7fc4940cef60 2023-09-23T03:16:32.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.850+0000 7fc48b7fe640 1 -- 172.21.15.154:0/942725987 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc49406ceb0 con 0x7fc4940cef60 2023-09-23T03:16:32.852 INFO:teuthology.orchestra.run.smithi154.stderr:listed 0 entries 2023-09-23T03:16:32.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.853+0000 7fc499410640 1 -- 172.21.15.154:0/942725987 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc47406af30 msgr2=0x7fc47406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:32.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.853+0000 7fc499410640 1 --2- 172.21.15.154:0/942725987 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc47406af30 0x7fc47406d3f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc484009fd0 tx=0x7fc48400b040 comp rx=0 tx=0).stop 2023-09-23T03:16:32.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.853+0000 7fc499410640 1 -- 172.21.15.154:0/942725987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4940cef60 msgr2=0x7fc49415b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:32.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.853+0000 7fc499410640 1 --2- 172.21.15.154:0/942725987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4940cef60 0x7fc49415b830 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc48c00c390 tx=0x7fc48c00c850 comp rx=0 tx=0).stop 2023-09-23T03:16:32.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.853+0000 7fc499410640 1 -- 172.21.15.154:0/942725987 shutdown_connections 2023-09-23T03:16:32.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.853+0000 7fc499410640 1 --2- 172.21.15.154:0/942725987 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc47406af30 0x7fc47406d3f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:32.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.853+0000 7fc499410640 1 --2- 172.21.15.154:0/942725987 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4940cef60 0x7fc49415b830 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:32.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.853+0000 7fc499410640 1 --2- 172.21.15.154:0/942725987 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4940b2fe0 0x7fc49415b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:32.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.853+0000 7fc499410640 1 --2- 172.21.15.154:0/942725987 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4940ada30 0x7fc494155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:32.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.853+0000 7fc499410640 1 -- 172.21.15.154:0/942725987 >> 172.21.15.154:0/942725987 conn(0x7fc49406a0f0 msgr2=0x7fc4940b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:32.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.853+0000 7fc499410640 1 -- 172.21.15.154:0/942725987 shutdown_connections 2023-09-23T03:16:32.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:32.853+0000 7fc499410640 1 -- 172.21.15.154:0/942725987 wait complete. 2023-09-23T03:16:32.871 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:16:32.872 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:16:32.923 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:16:33.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.133+0000 7f53d4bf5640 1 Processor -- start 2023-09-23T03:16:33.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.133+0000 7f53d4bf5640 1 -- start start 2023-09-23T03:16:33.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.134+0000 7f53d4bf5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53d015cf40 0x7f53d01615b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:33.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.134+0000 7f53d4bf5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f53d006b1e0 0x7f53d00cca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:33.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.134+0000 7f53d4bf5640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f53d00b21d0 0x7f53d00b2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:33.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.134+0000 7f53d4bf5640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53d00b2b80 con 0x7f53d006b1e0 2023-09-23T03:16:33.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.134+0000 7f53d4bf5640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53d00ccf80 con 0x7f53d00b21d0 2023-09-23T03:16:33.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.134+0000 7f53d4bf5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d00cd1c0 con 0x7f53d015cf40 2023-09-23T03:16:33.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.135+0000 7f53c7fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f53d006b1e0 0x7f53d00cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:33.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.135+0000 7f53c7fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f53d006b1e0 0x7f53d00cca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46654/0 (socket says 172.21.15.154:46654) 2023-09-23T03:16:33.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.135+0000 7f53c7fff640 1 -- 172.21.15.154:0/952607061 learned_addr learned my addr 172.21.15.154:0/952607061 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:33.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.135+0000 7f53ced9d640 1 --2- 172.21.15.154:0/952607061 >> v2:172.21.15.190:3300/0 conn(0x7f53d00b21d0 0x7f53d00b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:33.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.135+0000 7f53ce59c640 1 --2- 172.21.15.154:0/952607061 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53d015cf40 0x7f53d01615b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:33.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.135+0000 7f53c7fff640 1 -- 172.21.15.154:0/952607061 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53d015cf40 msgr2=0x7f53d01615b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:33.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.135+0000 7f53c7fff640 1 --2- 172.21.15.154:0/952607061 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53d015cf40 0x7f53d01615b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:33.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.135+0000 7f53c7fff640 1 -- 172.21.15.154:0/952607061 >> v2:172.21.15.190:3300/0 conn(0x7f53d00b21d0 msgr2=0x7f53d00b2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:33.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.135+0000 7f53c7fff640 1 --2- 172.21.15.154:0/952607061 >> v2:172.21.15.190:3300/0 conn(0x7f53d00b21d0 0x7f53d00b2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:33.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.135+0000 7f53c7fff640 1 -- 172.21.15.154:0/952607061 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d00cd4a0 con 0x7f53d006b1e0 2023-09-23T03:16:33.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.135+0000 7f53ce59c640 1 --2- 172.21.15.154:0/952607061 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53d015cf40 0x7f53d01615b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:33.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.136+0000 7f53c7fff640 1 --2- 172.21.15.154:0/952607061 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53d006b1e0 0x7f53d00cca40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f53d00c77b0 tx=0x7f53b802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b890bbc56c7dd3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:33.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.137+0000 7f53cdd9b640 1 -- 172.21.15.154:0/952607061 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b8031870 con 0x7f53d006b1e0 2023-09-23T03:16:33.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.137+0000 7f53cdd9b640 1 -- 172.21.15.154:0/952607061 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b80319d0 con 0x7f53d006b1e0 2023-09-23T03:16:33.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.137+0000 7f53cdd9b640 1 -- 172.21.15.154:0/952607061 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b8031d60 con 0x7f53d006b1e0 2023-09-23T03:16:33.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.137+0000 7f53d4bf5640 1 -- 172.21.15.154:0/952607061 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53d006b1e0 msgr2=0x7f53d00cca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:33.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.137+0000 7f53d4bf5640 1 --2- 172.21.15.154:0/952607061 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53d006b1e0 0x7f53d00cca40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f53d00c77b0 tx=0x7f53b802b910 comp rx=0 tx=0).stop 2023-09-23T03:16:33.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.137+0000 7f53d4bf5640 1 -- 172.21.15.154:0/952607061 shutdown_connections 2023-09-23T03:16:33.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.137+0000 7f53d4bf5640 1 --2- 172.21.15.154:0/952607061 >> v2:172.21.15.190:3300/0 conn(0x7f53d00b21d0 0x7f53d00b2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:33.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.137+0000 7f53d4bf5640 1 --2- 172.21.15.154:0/952607061 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53d006b1e0 0x7f53d00cca40 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:33.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.137+0000 7f53d4bf5640 1 --2- 172.21.15.154:0/952607061 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53d015cf40 0x7f53d01615b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:33.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.137+0000 7f53d4bf5640 1 -- 172.21.15.154:0/952607061 >> 172.21.15.154:0/952607061 conn(0x7f53d006fc60 msgr2=0x7f53d0070040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:33.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.137+0000 7f53d4bf5640 1 -- 172.21.15.154:0/952607061 shutdown_connections 2023-09-23T03:16:33.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.137+0000 7f53d4bf5640 1 -- 172.21.15.154:0/952607061 wait complete. 2023-09-23T03:16:33.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.139+0000 7f53d4bf5640 1 Processor -- start 2023-09-23T03:16:33.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.139+0000 7f53d4bf5640 1 -- start start 2023-09-23T03:16:33.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.140+0000 7f53d4bf5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53d006b1e0 0x7f53d00b3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:33.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.140+0000 7f53d4bf5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53d00b21d0 0x7f53d0148be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:33.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.140+0000 7f53d4bf5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53d015cf40 0x7f53d00c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:33.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.140+0000 7f53d4bf5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d0149120 con 0x7f53d00b21d0 2023-09-23T03:16:33.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.140+0000 7f53d4bf5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d00ca0d0 con 0x7f53d015cf40 2023-09-23T03:16:33.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.140+0000 7f53d4bf5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d0156470 con 0x7f53d006b1e0 2023-09-23T03:16:33.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.140+0000 7f53c7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53d00b21d0 0x7f53d0148be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:33.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.141+0000 7f53c7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53d00b21d0 0x7f53d0148be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46670/0 (socket says 172.21.15.154:46670) 2023-09-23T03:16:33.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.141+0000 7f53c7fff640 1 -- 172.21.15.154:0/1971580780 learned_addr learned my addr 172.21.15.154:0/1971580780 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:33.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.141+0000 7f53ced9d640 1 --2- 172.21.15.154:0/1971580780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53d015cf40 0x7f53d00c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:33.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.141+0000 7f53ce59c640 1 --2- 172.21.15.154:0/1971580780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53d006b1e0 0x7f53d00b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:33.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.141+0000 7f53c7fff640 1 -- 172.21.15.154:0/1971580780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53d006b1e0 msgr2=0x7f53d00b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:33.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.141+0000 7f53c7fff640 1 --2- 172.21.15.154:0/1971580780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53d006b1e0 0x7f53d00b3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:33.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.141+0000 7f53c7fff640 1 -- 172.21.15.154:0/1971580780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53d015cf40 msgr2=0x7f53d00c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:33.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.141+0000 7f53c7fff640 1 --2- 172.21.15.154:0/1971580780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53d015cf40 0x7f53d00c8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:33.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.141+0000 7f53c7fff640 1 -- 172.21.15.154:0/1971580780 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b80094d0 con 0x7f53d00b21d0 2023-09-23T03:16:33.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.141+0000 7f53ce59c640 1 --2- 172.21.15.154:0/1971580780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53d006b1e0 0x7f53d00b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:33.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.141+0000 7f53c7fff640 1 --2- 172.21.15.154:0/1971580780 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53d00b21d0 0x7f53d0148be0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f53b8000c00 tx=0x7f53b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:33.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.142+0000 7f53c6ffd640 1 -- 172.21.15.154:0/1971580780 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b803ad30 con 0x7f53d00b21d0 2023-09-23T03:16:33.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.142+0000 7f53c6ffd640 1 -- 172.21.15.154:0/1971580780 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b80443f0 con 0x7f53d00b21d0 2023-09-23T03:16:33.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.142+0000 7f53c6ffd640 1 -- 172.21.15.154:0/1971580780 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b80447a0 con 0x7f53d00b21d0 2023-09-23T03:16:33.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.142+0000 7f53d4bf5640 1 -- 172.21.15.154:0/1971580780 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53d015fc40 con 0x7f53d00b21d0 2023-09-23T03:16:33.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.142+0000 7f53d4bf5640 1 -- 172.21.15.154:0/1971580780 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53d00c9d60 con 0x7f53d00b21d0 2023-09-23T03:16:33.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.143+0000 7f53d4bf5640 1 -- 172.21.15.154:0/1971580780 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53d006ce60 con 0x7f53d00b21d0 2023-09-23T03:16:33.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.144+0000 7f53c6ffd640 1 -- 172.21.15.154:0/1971580780 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f53b8044900 con 0x7f53d00b21d0 2023-09-23T03:16:33.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.145+0000 7f53c6ffd640 1 --2- 172.21.15.154:0/1971580780 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f53b006af30 0x7f53b006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:33.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.145+0000 7f53c6ffd640 1 -- 172.21.15.154:0/1971580780 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f53b802d040 con 0x7f53d00b21d0 2023-09-23T03:16:33.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.145+0000 7f53ce59c640 1 --2- 172.21.15.154:0/1971580780 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f53b006af30 0x7f53b006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:33.147 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.146+0000 7f53ce59c640 1 --2- 172.21.15.154:0/1971580780 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f53b006af30 0x7f53b006d3f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f53c0006fd0 tx=0x7f53c0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:33.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.149+0000 7f53c6ffd640 1 -- 172.21.15.154:0/1971580780 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f53b8077070 con 0x7f53d00b21d0 2023-09-23T03:16:33.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.346+0000 7f53d4bf5640 1 -- 172.21.15.154:0/1971580780 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53d00cd750 con 0x7f53d00b21d0 2023-09-23T03:16:33.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.349+0000 7f53c6ffd640 1 -- 172.21.15.154:0/1971580780 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18744 (secure 0 0 0) 0x7f53b807a690 con 0x7f53d00b21d0 2023-09-23T03:16:33.351 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:16:33.351 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":26,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:16:03.151371+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:15:18.575849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T03:15:19.707212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T03:15:23.337138+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:16:33.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.352+0000 7f53d4bf5640 1 -- 172.21.15.154:0/1971580780 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f53b006af30 msgr2=0x7f53b006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:33.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.352+0000 7f53d4bf5640 1 --2- 172.21.15.154:0/1971580780 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f53b006af30 0x7f53b006d3f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f53c0006fd0 tx=0x7f53c0008040 comp rx=0 tx=0).stop 2023-09-23T03:16:33.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.352+0000 7f53d4bf5640 1 -- 172.21.15.154:0/1971580780 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53d00b21d0 msgr2=0x7f53d0148be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:33.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.352+0000 7f53d4bf5640 1 --2- 172.21.15.154:0/1971580780 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53d00b21d0 0x7f53d0148be0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f53b8000c00 tx=0x7f53b80046c0 comp rx=0 tx=0).stop 2023-09-23T03:16:33.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.352+0000 7f53d4bf5640 1 -- 172.21.15.154:0/1971580780 shutdown_connections 2023-09-23T03:16:33.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.352+0000 7f53d4bf5640 1 --2- 172.21.15.154:0/1971580780 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f53b006af30 0x7f53b006d3f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:33.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.352+0000 7f53d4bf5640 1 --2- 172.21.15.154:0/1971580780 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53d015cf40 0x7f53d00c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:33.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.352+0000 7f53d4bf5640 1 --2- 172.21.15.154:0/1971580780 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53d00b21d0 0x7f53d0148be0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:33.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.352+0000 7f53d4bf5640 1 --2- 172.21.15.154:0/1971580780 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53d006b1e0 0x7f53d00b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:33.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.352+0000 7f53d4bf5640 1 -- 172.21.15.154:0/1971580780 >> 172.21.15.154:0/1971580780 conn(0x7f53d006fc60 msgr2=0x7f53d00cc510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:33.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.352+0000 7f53d4bf5640 1 -- 172.21.15.154:0/1971580780 shutdown_connections 2023-09-23T03:16:33.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.353+0000 7f53d4bf5640 1 -- 172.21.15.154:0/1971580780 wait complete. 2023-09-23T03:16:33.375 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:15:18.575849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:15:19.707212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-23T03:15:23.337138+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-23T03:16:33.375 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:16:33.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.587+0000 7fc7d359e640 1 Processor -- start 2023-09-23T03:16:33.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.587+0000 7fc7d359e640 1 -- start start 2023-09-23T03:16:33.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.588+0000 7fc7d359e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc7d406fdc0 0x7fc7d4070200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:33.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.589+0000 7fc7d359e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7d4070740 con 0x7fc7d415e130 2023-09-23T03:16:33.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.589+0000 7fc7d359e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7d4158540 con 0x7fc7d4158160 2023-09-23T03:16:33.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.589+0000 7fc7d359e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7d40c9c50 con 0x7fc7d406fdc0 2023-09-23T03:16:33.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.589+0000 7fc7cbfff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fc7d415e130 0x7fc7d40c9540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:52866/0 (socket says 172.21.15.154:52866) 2023-09-23T03:16:33.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.589+0000 7fc7cbfff640 1 -- 172.21.15.154:0/3187123269 learned_addr learned my addr 172.21.15.154:0/3187123269 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:33.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.589+0000 7fc7d2d9d640 1 --2- 172.21.15.154:0/3187123269 >> v2:172.21.15.154:3300/0 conn(0x7fc7d406fdc0 0x7fc7d4070200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:33.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.589+0000 7fc7d1d9b640 1 -- 172.21.15.154:0/3187123269 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1104171710 0 0) 0x7fc7d4070740 con 0x7fc7d415e130 2023-09-23T03:16:33.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.590+0000 7fc7d1d9b640 1 -- 172.21.15.154:0/3187123269 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7c0003520 con 0x7fc7d415e130 2023-09-23T03:16:33.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.590+0000 7fc7d2d9d640 1 -- 172.21.15.154:0/3187123269 >> v1:172.21.15.190:6789/0 conn(0x7fc7d4158160 legacy=0x7fc7d415ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:33.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.590+0000 7fc7d2d9d640 1 -- 172.21.15.154:0/3187123269 >> v1:172.21.15.154:6789/0 conn(0x7fc7d415e130 legacy=0x7fc7d40c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:33.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.590+0000 7fc7d2d9d640 1 -- 172.21.15.154:0/3187123269 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d415b3e0 con 0x7fc7d406fdc0 2023-09-23T03:16:33.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.591+0000 7fc7d2d9d640 1 --2- 172.21.15.154:0/3187123269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7d406fdc0 0x7fc7d4070200 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc009d90 tx=0x7fc7cc02bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d857e93b8950d0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:33.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.591+0000 7fc7d1d9b640 1 -- 172.21.15.154:0/3187123269 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc02ccf0 con 0x7fc7d406fdc0 2023-09-23T03:16:33.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.591+0000 7fc7d1d9b640 1 -- 172.21.15.154:0/3187123269 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7cc02ce50 con 0x7fc7d406fdc0 2023-09-23T03:16:33.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.591+0000 7fc7d1d9b640 1 -- 172.21.15.154:0/3187123269 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc034640 con 0x7fc7d406fdc0 2023-09-23T03:16:33.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.592+0000 7fc7d359e640 1 -- 172.21.15.154:0/3187123269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7d406fdc0 msgr2=0x7fc7d4070200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:33.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.592+0000 7fc7d359e640 1 --2- 172.21.15.154:0/3187123269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7d406fdc0 0x7fc7d4070200 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc009d90 tx=0x7fc7cc02bf40 comp rx=0 tx=0).stop 2023-09-23T03:16:33.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.592+0000 7fc7d359e640 1 -- 172.21.15.154:0/3187123269 shutdown_connections 2023-09-23T03:16:33.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.592+0000 7fc7d359e640 1 --2- 172.21.15.154:0/3187123269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7d406fdc0 0x7fc7d4070200 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:33.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.592+0000 7fc7d359e640 1 -- 172.21.15.154:0/3187123269 >> 172.21.15.154:0/3187123269 conn(0x7fc7d406a300 msgr2=0x7fc7d40afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:33.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.592+0000 7fc7d359e640 1 -- 172.21.15.154:0/3187123269 shutdown_connections 2023-09-23T03:16:33.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.592+0000 7fc7d359e640 1 -- 172.21.15.154:0/3187123269 wait complete. 2023-09-23T03:16:33.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.594+0000 7fc7d359e640 1 Processor -- start 2023-09-23T03:16:33.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.594+0000 7fc7d359e640 1 -- start start 2023-09-23T03:16:33.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.595+0000 7fc7d359e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7d406fdc0 0x7fc7d414a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:33.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.595+0000 7fc7d359e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7d4158160 0x7fc7d40b3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:33.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.595+0000 7fc7d359e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7d415e130 0x7fc7d40ce230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:33.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.595+0000 7fc7d359e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d40c5b00 con 0x7fc7d406fdc0 2023-09-23T03:16:33.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.595+0000 7fc7d359e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d40c5de0 con 0x7fc7d4158160 2023-09-23T03:16:33.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.595+0000 7fc7d359e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d414ac40 con 0x7fc7d415e130 2023-09-23T03:16:33.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.595+0000 7fc7d259c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7d406fdc0 0x7fc7d414a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:33.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.595+0000 7fc7d259c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7d406fdc0 0x7fc7d414a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46676/0 (socket says 172.21.15.154:46676) 2023-09-23T03:16:33.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.595+0000 7fc7d259c640 1 -- 172.21.15.154:0/2703242331 learned_addr learned my addr 172.21.15.154:0/2703242331 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:33.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.595+0000 7fc7d259c640 1 -- 172.21.15.154:0/2703242331 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7d415e130 msgr2=0x7fc7d40ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:33.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.595+0000 7fc7d259c640 1 --2- 172.21.15.154:0/2703242331 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7d415e130 0x7fc7d40ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:33.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.595+0000 7fc7d259c640 1 -- 172.21.15.154:0/2703242331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7d4158160 msgr2=0x7fc7d40b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:33.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.595+0000 7fc7d259c640 1 --2- 172.21.15.154:0/2703242331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7d4158160 0x7fc7d40b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:33.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.595+0000 7fc7d259c640 1 -- 172.21.15.154:0/2703242331 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7cc009a70 con 0x7fc7d406fdc0 2023-09-23T03:16:33.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.596+0000 7fc7d259c640 1 --2- 172.21.15.154:0/2703242331 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7d406fdc0 0x7fc7d414a700 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc7c400c430 tx=0x7fc7c400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:33.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.596+0000 7fc7caffd640 1 -- 172.21.15.154:0/2703242331 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c40052d0 con 0x7fc7d406fdc0 2023-09-23T03:16:33.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.596+0000 7fc7caffd640 1 -- 172.21.15.154:0/2703242331 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c4005450 con 0x7fc7d406fdc0 2023-09-23T03:16:33.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.596+0000 7fc7caffd640 1 -- 172.21.15.154:0/2703242331 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4007260 con 0x7fc7d406fdc0 2023-09-23T03:16:33.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.596+0000 7fc7d359e640 1 -- 172.21.15.154:0/2703242331 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7d40b3960 con 0x7fc7d406fdc0 2023-09-23T03:16:33.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.596+0000 7fc7d359e640 1 -- 172.21.15.154:0/2703242331 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7d40b3f20 con 0x7fc7d406fdc0 2023-09-23T03:16:33.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.597+0000 7fc7d359e640 1 -- 172.21.15.154:0/2703242331 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7d40736b0 con 0x7fc7d406fdc0 2023-09-23T03:16:33.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.598+0000 7fc7caffd640 1 -- 172.21.15.154:0/2703242331 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fc7c40073c0 con 0x7fc7d406fdc0 2023-09-23T03:16:33.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.599+0000 7fc7caffd640 1 --2- 172.21.15.154:0/2703242331 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc7b406af30 0x7fc7b406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:33.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.599+0000 7fc7caffd640 1 -- 172.21.15.154:0/2703242331 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7fc7c4088fc0 con 0x7fc7d406fdc0 2023-09-23T03:16:33.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.599+0000 7fc7cbfff640 1 --2- 172.21.15.154:0/2703242331 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc7b406af30 0x7fc7b406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:33.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.600+0000 7fc7cbfff640 1 --2- 172.21.15.154:0/2703242331 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc7b406af30 0x7fc7b406d3f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc7d415cbf0 tx=0x7fc7bc007930 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:33.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.603+0000 7fc7caffd640 1 -- 172.21.15.154:0/2703242331 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fc7c4057640 con 0x7fc7d406fdc0 2023-09-23T03:16:33.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.790+0000 7fc7d359e640 1 -- 172.21.15.154:0/2703242331 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc7d4067390 con 0x7fc7d406fdc0 2023-09-23T03:16:33.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.791+0000 7fc7caffd640 1 -- 172.21.15.154:0/2703242331 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc7c40571d0 con 0x7fc7d406fdc0 2023-09-23T03:16:33.792 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:16:33.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.793+0000 7fc7d359e640 1 -- 172.21.15.154:0/2703242331 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc7b406af30 msgr2=0x7fc7b406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:33.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.793+0000 7fc7d359e640 1 --2- 172.21.15.154:0/2703242331 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc7b406af30 0x7fc7b406d3f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc7d415cbf0 tx=0x7fc7bc007930 comp rx=0 tx=0).stop 2023-09-23T03:16:33.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.793+0000 7fc7d359e640 1 -- 172.21.15.154:0/2703242331 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7d406fdc0 msgr2=0x7fc7d414a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:33.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.793+0000 7fc7d359e640 1 --2- 172.21.15.154:0/2703242331 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7d406fdc0 0x7fc7d414a700 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc7c400c430 tx=0x7fc7c400c8f0 comp rx=0 tx=0).stop 2023-09-23T03:16:33.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.793+0000 7fc7d359e640 1 -- 172.21.15.154:0/2703242331 shutdown_connections 2023-09-23T03:16:33.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.793+0000 7fc7d359e640 1 --2- 172.21.15.154:0/2703242331 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc7b406af30 0x7fc7b406d3f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:33.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.793+0000 7fc7d359e640 1 --2- 172.21.15.154:0/2703242331 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7d415e130 0x7fc7d40ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:33.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.793+0000 7fc7d359e640 1 --2- 172.21.15.154:0/2703242331 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7d4158160 0x7fc7d40b3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:33.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.793+0000 7fc7d359e640 1 --2- 172.21.15.154:0/2703242331 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7d406fdc0 0x7fc7d414a700 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:33.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.793+0000 7fc7d359e640 1 -- 172.21.15.154:0/2703242331 >> 172.21.15.154:0/2703242331 conn(0x7fc7d406a300 msgr2=0x7fc7d4072730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:33.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.793+0000 7fc7d359e640 1 -- 172.21.15.154:0/2703242331 shutdown_connections 2023-09-23T03:16:33.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:33.793+0000 7fc7d359e640 1 -- 172.21.15.154:0/2703242331 wait complete. 2023-09-23T03:16:33.814 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:16:34.038 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.036+0000 7f8c9112c640 1 Processor -- start 2023-09-23T03:16:34.038 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.037+0000 7f8c9112c640 1 -- start start 2023-09-23T03:16:34.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.038+0000 7f8c9112c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8c8c0c8d70 0x7f8c8c15c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:34.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.038+0000 7f8c9112c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c8c0c9150 0x7f8c8c15cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:34.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.038+0000 7f8c9112c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8c8c071290 0x7f8c8c0ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:34.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.038+0000 7f8c9112c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c8c0c9d20 con 0x7f8c8c0c8d70 2023-09-23T03:16:34.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.038+0000 7f8c9112c640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c8c071850 con 0x7f8c8c071290 2023-09-23T03:16:34.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.038+0000 7f8c9112c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8c071b30 con 0x7f8c8c0c9150 2023-09-23T03:16:34.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.038+0000 7f8c8ad9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8c8c0c8d70 0x7f8c8c15c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:34.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.039+0000 7f8c8ad9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8c8c0c8d70 0x7f8c8c15c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52468/0 (socket says 172.21.15.154:52468) 2023-09-23T03:16:34.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.039+0000 7f8c8ad9d640 1 -- 172.21.15.154:0/754791834 learned_addr learned my addr 172.21.15.154:0/754791834 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:34.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.039+0000 7f8c8b59e640 1 --2- 172.21.15.154:0/754791834 >> v2:172.21.15.190:3300/0 conn(0x7f8c8c071290 0x7f8c8c0ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:34.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.039+0000 7f8c83fff640 1 --2- 172.21.15.154:0/754791834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c8c0c9150 0x7f8c8c15cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:34.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.039+0000 7f8c8ad9d640 1 -- 172.21.15.154:0/754791834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c8c0c9150 msgr2=0x7f8c8c15cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:34.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.039+0000 7f8c8ad9d640 1 --2- 172.21.15.154:0/754791834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c8c0c9150 0x7f8c8c15cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:34.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.039+0000 7f8c8ad9d640 1 -- 172.21.15.154:0/754791834 >> v2:172.21.15.190:3300/0 conn(0x7f8c8c071290 msgr2=0x7f8c8c0ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:34.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.039+0000 7f8c8ad9d640 1 --2- 172.21.15.154:0/754791834 >> v2:172.21.15.190:3300/0 conn(0x7f8c8c071290 0x7f8c8c0ca1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:34.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.039+0000 7f8c8ad9d640 1 -- 172.21.15.154:0/754791834 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8c071e10 con 0x7f8c8c0c8d70 2023-09-23T03:16:34.042 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.040+0000 7f8c8ad9d640 1 --2- 172.21.15.154:0/754791834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c8c0c8d70 0x7f8c8c15c6d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c0c95d0 tx=0x7f8c7c02b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d8837773dc7b00 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:34.042 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.041+0000 7f8c8a59c640 1 -- 172.21.15.154:0/754791834 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c0317d0 con 0x7f8c8c0c8d70 2023-09-23T03:16:34.042 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.041+0000 7f8c8a59c640 1 -- 172.21.15.154:0/754791834 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7c031930 con 0x7f8c8c0c8d70 2023-09-23T03:16:34.042 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.041+0000 7f8c8a59c640 1 -- 172.21.15.154:0/754791834 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c031cc0 con 0x7f8c8c0c8d70 2023-09-23T03:16:34.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.042+0000 7f8c9112c640 1 -- 172.21.15.154:0/754791834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c8c0c8d70 msgr2=0x7f8c8c15c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:34.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.042+0000 7f8c9112c640 1 --2- 172.21.15.154:0/754791834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c8c0c8d70 0x7f8c8c15c6d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c0c95d0 tx=0x7f8c7c02b870 comp rx=0 tx=0).stop 2023-09-23T03:16:34.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.042+0000 7f8c9112c640 1 -- 172.21.15.154:0/754791834 shutdown_connections 2023-09-23T03:16:34.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.042+0000 7f8c9112c640 1 --2- 172.21.15.154:0/754791834 >> v2:172.21.15.190:3300/0 conn(0x7f8c8c071290 0x7f8c8c0ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:34.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.042+0000 7f8c9112c640 1 --2- 172.21.15.154:0/754791834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c8c0c9150 0x7f8c8c15cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:34.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.042+0000 7f8c9112c640 1 --2- 172.21.15.154:0/754791834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c8c0c8d70 0x7f8c8c15c6d0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:34.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.042+0000 7f8c9112c640 1 -- 172.21.15.154:0/754791834 >> 172.21.15.154:0/754791834 conn(0x7f8c8c06c010 msgr2=0x7f8c8c06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:34.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.042+0000 7f8c9112c640 1 -- 172.21.15.154:0/754791834 shutdown_connections 2023-09-23T03:16:34.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.042+0000 7f8c9112c640 1 -- 172.21.15.154:0/754791834 wait complete. 2023-09-23T03:16:34.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.043+0000 7f8c9112c640 1 Processor -- start 2023-09-23T03:16:34.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.044+0000 7f8c9112c640 1 -- start start 2023-09-23T03:16:34.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.045+0000 7f8c9112c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c8c071290 0x7f8c8c0c1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:34.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.045+0000 7f8c9112c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c8c0c8d70 0x7f8c8c14aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:34.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.045+0000 7f8c9112c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c8c0c9150 0x7f8c8c1f2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:34.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.045+0000 7f8c9112c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8c14af50 con 0x7f8c8c071290 2023-09-23T03:16:34.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.045+0000 7f8c9112c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8c1f2c60 con 0x7f8c8c0c8d70 2023-09-23T03:16:34.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.045+0000 7f8c9112c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8c0c38b0 con 0x7f8c8c0c9150 2023-09-23T03:16:34.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.045+0000 7f8c8ad9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c8c071290 0x7f8c8c0c1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:34.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.045+0000 7f8c8ad9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c8c071290 0x7f8c8c0c1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52484/0 (socket says 172.21.15.154:52484) 2023-09-23T03:16:34.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.045+0000 7f8c8ad9d640 1 -- 172.21.15.154:0/3024815316 learned_addr learned my addr 172.21.15.154:0/3024815316 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:34.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.045+0000 7f8c83fff640 1 --2- 172.21.15.154:0/3024815316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c8c0c8d70 0x7f8c8c14aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:34.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.045+0000 7f8c8b59e640 1 --2- 172.21.15.154:0/3024815316 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c8c0c9150 0x7f8c8c1f2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:34.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.045+0000 7f8c8ad9d640 1 -- 172.21.15.154:0/3024815316 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c8c0c9150 msgr2=0x7f8c8c1f2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:34.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.045+0000 7f8c8ad9d640 1 --2- 172.21.15.154:0/3024815316 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c8c0c9150 0x7f8c8c1f2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:34.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.045+0000 7f8c8ad9d640 1 -- 172.21.15.154:0/3024815316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c8c0c8d70 msgr2=0x7f8c8c14aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:34.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.045+0000 7f8c8ad9d640 1 --2- 172.21.15.154:0/3024815316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c8c0c8d70 0x7f8c8c14aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:34.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.045+0000 7f8c8ad9d640 1 -- 172.21.15.154:0/3024815316 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c7c0094d0 con 0x7f8c8c071290 2023-09-23T03:16:34.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.046+0000 7f8c8ad9d640 1 --2- 172.21.15.154:0/3024815316 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c8c071290 0x7f8c8c0c1fe0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c000c00 tx=0x7f8c7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:34.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.046+0000 7f8c837fe640 1 -- 172.21.15.154:0/3024815316 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c032870 con 0x7f8c8c071290 2023-09-23T03:16:34.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.046+0000 7f8c837fe640 1 -- 172.21.15.154:0/3024815316 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7c032a10 con 0x7f8c8c071290 2023-09-23T03:16:34.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.046+0000 7f8c837fe640 1 -- 172.21.15.154:0/3024815316 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c032dc0 con 0x7f8c8c071290 2023-09-23T03:16:34.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.046+0000 7f8c9112c640 1 -- 172.21.15.154:0/3024815316 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c8c1f2e20 con 0x7f8c8c071290 2023-09-23T03:16:34.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.046+0000 7f8c9112c640 1 -- 172.21.15.154:0/3024815316 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c8c0c28a0 con 0x7f8c8c071290 2023-09-23T03:16:34.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.047+0000 7f8c9112c640 1 -- 172.21.15.154:0/3024815316 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c8c04e920 con 0x7f8c8c071290 2023-09-23T03:16:34.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.048+0000 7f8c837fe640 1 -- 172.21.15.154:0/3024815316 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f8c7c030070 con 0x7f8c8c071290 2023-09-23T03:16:34.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.050+0000 7f8c837fe640 1 --2- 172.21.15.154:0/3024815316 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8c6c06af30 0x7f8c6c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:34.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.050+0000 7f8c837fe640 1 -- 172.21.15.154:0/3024815316 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f8c7c0ab970 con 0x7f8c8c071290 2023-09-23T03:16:34.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.050+0000 7f8c83fff640 1 --2- 172.21.15.154:0/3024815316 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8c6c06af30 0x7f8c6c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:34.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.051+0000 7f8c83fff640 1 --2- 172.21.15.154:0/3024815316 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8c6c06af30 0x7f8c6c06d3f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c159210 tx=0x7f8c74008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:34.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.054+0000 7f8c837fe640 1 -- 172.21.15.154:0/3024815316 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f8c7c076690 con 0x7f8c8c071290 2023-09-23T03:16:34.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.245+0000 7f8c9112c640 1 -- 172.21.15.154:0/3024815316 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8c8c06cbc0 con 0x7f8c8c071290 2023-09-23T03:16:34.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.246+0000 7f8c837fe640 1 -- 172.21.15.154:0/3024815316 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8c7c079cb0 con 0x7f8c8c071290 2023-09-23T03:16:34.247 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:16:34.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.248+0000 7f8c9112c640 1 -- 172.21.15.154:0/3024815316 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8c6c06af30 msgr2=0x7f8c6c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:34.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.248+0000 7f8c9112c640 1 --2- 172.21.15.154:0/3024815316 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8c6c06af30 0x7f8c6c06d3f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c159210 tx=0x7f8c74008040 comp rx=0 tx=0).stop 2023-09-23T03:16:34.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.248+0000 7f8c9112c640 1 -- 172.21.15.154:0/3024815316 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c8c071290 msgr2=0x7f8c8c0c1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:34.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.248+0000 7f8c9112c640 1 --2- 172.21.15.154:0/3024815316 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c8c071290 0x7f8c8c0c1fe0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c000c00 tx=0x7f8c7c0046c0 comp rx=0 tx=0).stop 2023-09-23T03:16:34.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.248+0000 7f8c9112c640 1 -- 172.21.15.154:0/3024815316 shutdown_connections 2023-09-23T03:16:34.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.248+0000 7f8c9112c640 1 --2- 172.21.15.154:0/3024815316 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8c6c06af30 0x7f8c6c06d3f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:34.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.248+0000 7f8c9112c640 1 --2- 172.21.15.154:0/3024815316 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c8c0c9150 0x7f8c8c1f2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:34.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.248+0000 7f8c9112c640 1 --2- 172.21.15.154:0/3024815316 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c8c0c8d70 0x7f8c8c14aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:34.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.248+0000 7f8c9112c640 1 --2- 172.21.15.154:0/3024815316 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c8c071290 0x7f8c8c0c1fe0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:34.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.248+0000 7f8c9112c640 1 -- 172.21.15.154:0/3024815316 >> 172.21.15.154:0/3024815316 conn(0x7f8c8c06c010 msgr2=0x7f8c8c0b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:34.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.248+0000 7f8c9112c640 1 -- 172.21.15.154:0/3024815316 shutdown_connections 2023-09-23T03:16:34.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.248+0000 7f8c9112c640 1 -- 172.21.15.154:0/3024815316 wait complete. 2023-09-23T03:16:34.268 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:16:34.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.485+0000 7f5602b2c640 1 Processor -- start 2023-09-23T03:16:34.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.485+0000 7f5602b2c640 1 -- start start 2023-09-23T03:16:34.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.487+0000 7f5602b2c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55fc15bb30 0x7f55fc0caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:34.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.487+0000 7f5602b2c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f55fc0cb3e0 0x7f55fc0cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:34.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.487+0000 7f5602b2c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55fc15b6a0 con 0x7f55fc15aba0 2023-09-23T03:16:34.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.487+0000 7f5602b2c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55fc0cb870 con 0x7f55fc0cb3e0 2023-09-23T03:16:34.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.487+0000 7f5602b2c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55fc15bf70 con 0x7f55fc15bb30 2023-09-23T03:16:34.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.487+0000 7f56008c8640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f55fc15aba0 0x7f55fc15e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:42016/0 (socket says 172.21.15.154:42016) 2023-09-23T03:16:34.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.487+0000 7f56008c8640 1 -- 172.21.15.154:0/4094888692 learned_addr learned my addr 172.21.15.154:0/4094888692 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:34.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.487+0000 7f56010c9640 1 --2- 172.21.15.154:0/4094888692 >> v2:172.21.15.154:3300/0 conn(0x7f55fc0cb3e0 0x7f55fc0cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:34.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.487+0000 7f55fbfff640 1 --2- 172.21.15.154:0/4094888692 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55fc15bb30 0x7f55fc0caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:34.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.488+0000 7f56010c9640 1 -- 172.21.15.154:0/4094888692 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55fc15bb30 msgr2=0x7f55fc0caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:34.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.488+0000 7f56010c9640 1 --2- 172.21.15.154:0/4094888692 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55fc15bb30 0x7f55fc0caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:34.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.488+0000 7f56010c9640 1 -- 172.21.15.154:0/4094888692 >> v1:172.21.15.154:6789/0 conn(0x7f55fc15aba0 legacy=0x7f55fc15e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:34.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.488+0000 7f56010c9640 1 -- 172.21.15.154:0/4094888692 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55fc15f0b0 con 0x7f55fc0cb3e0 2023-09-23T03:16:34.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.489+0000 7f56010c9640 1 --2- 172.21.15.154:0/4094888692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55fc0cb3e0 0x7f55fc0cc5c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f55f40097a0 tx=0x7f55f402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a7d3c177df42496 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:34.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.489+0000 7f55fb7fe640 1 -- 172.21.15.154:0/4094888692 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f402d040 con 0x7f55fc0cb3e0 2023-09-23T03:16:34.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.489+0000 7f55fb7fe640 1 -- 172.21.15.154:0/4094888692 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55f4031810 con 0x7f55fc0cb3e0 2023-09-23T03:16:34.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.489+0000 7f55fb7fe640 1 -- 172.21.15.154:0/4094888692 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f4031ba0 con 0x7f55fc0cb3e0 2023-09-23T03:16:34.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.489+0000 7f5602b2c640 1 -- 172.21.15.154:0/4094888692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55fc0cb3e0 msgr2=0x7f55fc0cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:34.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.489+0000 7f5602b2c640 1 --2- 172.21.15.154:0/4094888692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55fc0cb3e0 0x7f55fc0cc5c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f55f40097a0 tx=0x7f55f402b8b0 comp rx=0 tx=0).stop 2023-09-23T03:16:34.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.490+0000 7f5602b2c640 1 -- 172.21.15.154:0/4094888692 shutdown_connections 2023-09-23T03:16:34.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.490+0000 7f5602b2c640 1 --2- 172.21.15.154:0/4094888692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55fc0cb3e0 0x7f55fc0cc5c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:34.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.490+0000 7f5602b2c640 1 --2- 172.21.15.154:0/4094888692 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55fc15bb30 0x7f55fc0caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:34.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.490+0000 7f5602b2c640 1 -- 172.21.15.154:0/4094888692 >> 172.21.15.154:0/4094888692 conn(0x7f55fc06c010 msgr2=0x7f55fc06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:34.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.490+0000 7f5602b2c640 1 -- 172.21.15.154:0/4094888692 shutdown_connections 2023-09-23T03:16:34.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.490+0000 7f5602b2c640 1 -- 172.21.15.154:0/4094888692 wait complete. 2023-09-23T03:16:34.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.491+0000 7f5602b2c640 1 Processor -- start 2023-09-23T03:16:34.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.492+0000 7f5602b2c640 1 -- start start 2023-09-23T03:16:34.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.493+0000 7f5602b2c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55fc0cb3e0 0x7f55fc155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:34.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.493+0000 7f5602b2c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55fc15aba0 0x7f55fc0c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:34.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.493+0000 7f5602b2c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55fc15bb30 0x7f55fc0c7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:34.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.493+0000 7f5602b2c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55fc0ae5f0 con 0x7f55fc15aba0 2023-09-23T03:16:34.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.493+0000 7f5602b2c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55fc0ae8d0 con 0x7f55fc15bb30 2023-09-23T03:16:34.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.493+0000 7f5602b2c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55fc0aebb0 con 0x7f55fc0cb3e0 2023-09-23T03:16:34.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.493+0000 7f55fbfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55fc15aba0 0x7f55fc0c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:34.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.493+0000 7f55fbfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55fc15aba0 0x7f55fc0c71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52498/0 (socket says 172.21.15.154:52498) 2023-09-23T03:16:34.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.493+0000 7f55fbfff640 1 -- 172.21.15.154:0/3063633686 learned_addr learned my addr 172.21.15.154:0/3063633686 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:34.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.493+0000 7f56008c8640 1 --2- 172.21.15.154:0/3063633686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55fc0cb3e0 0x7f55fc155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:34.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.493+0000 7f56010c9640 1 --2- 172.21.15.154:0/3063633686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55fc15bb30 0x7f55fc0c7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:34.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.493+0000 7f55fbfff640 1 -- 172.21.15.154:0/3063633686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55fc0cb3e0 msgr2=0x7f55fc155b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:34.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.493+0000 7f55fbfff640 1 --2- 172.21.15.154:0/3063633686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55fc0cb3e0 0x7f55fc155b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:34.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.493+0000 7f55fbfff640 1 -- 172.21.15.154:0/3063633686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55fc15bb30 msgr2=0x7f55fc0c7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:34.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.493+0000 7f55fbfff640 1 --2- 172.21.15.154:0/3063633686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55fc15bb30 0x7f55fc0c7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:34.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.493+0000 7f55fbfff640 1 -- 172.21.15.154:0/3063633686 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f4009400 con 0x7f55fc15aba0 2023-09-23T03:16:34.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.493+0000 7f56008c8640 1 --2- 172.21.15.154:0/3063633686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55fc0cb3e0 0x7f55fc155b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:34.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.493+0000 7f56010c9640 1 --2- 172.21.15.154:0/3063633686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55fc15bb30 0x7f55fc0c7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:34.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.494+0000 7f55fbfff640 1 --2- 172.21.15.154:0/3063633686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55fc15aba0 0x7f55fc0c71e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f55f0007ef0 tx=0x7f55f000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:34.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.494+0000 7f55f97fa640 1 -- 172.21.15.154:0/3063633686 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f0014620 con 0x7f55fc15aba0 2023-09-23T03:16:34.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.494+0000 7f55f97fa640 1 -- 172.21.15.154:0/3063633686 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55f0014780 con 0x7f55fc15aba0 2023-09-23T03:16:34.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.494+0000 7f55f97fa640 1 -- 172.21.15.154:0/3063633686 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f0014b30 con 0x7f55fc15aba0 2023-09-23T03:16:34.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.494+0000 7f5602b2c640 1 -- 172.21.15.154:0/3063633686 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55fc0aee90 con 0x7f55fc15aba0 2023-09-23T03:16:34.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.494+0000 7f5602b2c640 1 -- 172.21.15.154:0/3063633686 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55fc0b35f0 con 0x7f55fc15aba0 2023-09-23T03:16:34.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.495+0000 7f5602b2c640 1 -- 172.21.15.154:0/3063633686 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55fc04e920 con 0x7f55fc15aba0 2023-09-23T03:16:34.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.496+0000 7f55f97fa640 1 -- 172.21.15.154:0/3063633686 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f55f0014c90 con 0x7f55fc15aba0 2023-09-23T03:16:34.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.497+0000 7f55f97fa640 1 --2- 172.21.15.154:0/3063633686 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f55e806af30 0x7f55e806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:34.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.497+0000 7f55f97fa640 1 -- 172.21.15.154:0/3063633686 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f55f008a500 con 0x7f55fc15aba0 2023-09-23T03:16:34.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.497+0000 7f56008c8640 1 --2- 172.21.15.154:0/3063633686 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f55e806af30 0x7f55e806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:34.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.498+0000 7f56008c8640 1 --2- 172.21.15.154:0/3063633686 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f55e806af30 0x7f55e806d3f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f55ec005880 tx=0x7f55ec009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:34.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.502+0000 7f55f97fa640 1 -- 172.21.15.154:0/3063633686 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f55f0055180 con 0x7f55fc15aba0 2023-09-23T03:16:34.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.691+0000 7f5602b2c640 1 -- 172.21.15.154:0/3063633686 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f55fc0cbc10 con 0x7f55fc15aba0 2023-09-23T03:16:34.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.692+0000 7f55f97fa640 1 -- 172.21.15.154:0/3063633686 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f55f00587a0 con 0x7f55fc15aba0 2023-09-23T03:16:34.693 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:16:34.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.694+0000 7f5602b2c640 1 -- 172.21.15.154:0/3063633686 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f55e806af30 msgr2=0x7f55e806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:34.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.694+0000 7f5602b2c640 1 --2- 172.21.15.154:0/3063633686 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f55e806af30 0x7f55e806d3f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f55ec005880 tx=0x7f55ec009040 comp rx=0 tx=0).stop 2023-09-23T03:16:34.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.694+0000 7f5602b2c640 1 -- 172.21.15.154:0/3063633686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55fc15aba0 msgr2=0x7f55fc0c71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:34.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.694+0000 7f5602b2c640 1 --2- 172.21.15.154:0/3063633686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55fc15aba0 0x7f55fc0c71e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f55f0007ef0 tx=0x7f55f000e4b0 comp rx=0 tx=0).stop 2023-09-23T03:16:34.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.695+0000 7f5602b2c640 1 -- 172.21.15.154:0/3063633686 shutdown_connections 2023-09-23T03:16:34.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.695+0000 7f5602b2c640 1 --2- 172.21.15.154:0/3063633686 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f55e806af30 0x7f55e806d3f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:34.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.695+0000 7f5602b2c640 1 --2- 172.21.15.154:0/3063633686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f55fc15bb30 0x7f55fc0c7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:34.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.695+0000 7f5602b2c640 1 --2- 172.21.15.154:0/3063633686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55fc15aba0 0x7f55fc0c71e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:34.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.695+0000 7f5602b2c640 1 --2- 172.21.15.154:0/3063633686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55fc0cb3e0 0x7f55fc155b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:34.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.695+0000 7f5602b2c640 1 -- 172.21.15.154:0/3063633686 >> 172.21.15.154:0/3063633686 conn(0x7f55fc06c010 msgr2=0x7f55fc15e360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:34.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.695+0000 7f5602b2c640 1 -- 172.21.15.154:0/3063633686 shutdown_connections 2023-09-23T03:16:34.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.695+0000 7f5602b2c640 1 -- 172.21.15.154:0/3063633686 wait complete. 2023-09-23T03:16:34.715 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-23T03:16:34.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.925+0000 7f4ac5c6f640 1 Processor -- start 2023-09-23T03:16:34.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.925+0000 7f4ac5c6f640 1 -- start start 2023-09-23T03:16:34.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.927+0000 7f4ac5c6f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4ac00c8d70 0x7f4ac015c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:34.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.927+0000 7f4ac5c6f640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4ac00c9150 0x7f4ac015cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:34.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.927+0000 7f4ac5c6f640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ac00c9d20 con 0x7f4ac0071290 2023-09-23T03:16:34.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.927+0000 7f4ac5c6f640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ac0071850 con 0x7f4ac00c8d70 2023-09-23T03:16:34.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.927+0000 7f4ac5c6f640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ac0071b30 con 0x7f4ac00c9150 2023-09-23T03:16:34.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.927+0000 7f4abf7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4ac00c8d70 0x7f4ac015c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:34.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.927+0000 7f4abf7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4ac00c8d70 0x7f4ac015c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52502/0 (socket says 172.21.15.154:52502) 2023-09-23T03:16:34.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.927+0000 7f4abf7fe640 1 -- 172.21.15.154:0/1798190080 learned_addr learned my addr 172.21.15.154:0/1798190080 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:34.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.927+0000 7f4abffff640 1 --1- 172.21.15.154:0/1798190080 >> v1:172.21.15.154:6789/0 conn(0x7f4ac0071290 0x7f4ac00ca1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:42032/0 (socket says 172.21.15.154:42032) 2023-09-23T03:16:34.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.928+0000 7f4ab7fff640 1 --2- 172.21.15.154:0/1798190080 >> v2:172.21.15.190:3300/0 conn(0x7f4ac00c9150 0x7f4ac015cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:34.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.928+0000 7f4abf7fe640 1 -- 172.21.15.154:0/1798190080 >> v2:172.21.15.190:3300/0 conn(0x7f4ac00c9150 msgr2=0x7f4ac015cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:34.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.928+0000 7f4abf7fe640 1 --2- 172.21.15.154:0/1798190080 >> v2:172.21.15.190:3300/0 conn(0x7f4ac00c9150 0x7f4ac015cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:34.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.928+0000 7f4abf7fe640 1 -- 172.21.15.154:0/1798190080 >> v1:172.21.15.154:6789/0 conn(0x7f4ac0071290 legacy=0x7f4ac00ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:34.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.928+0000 7f4abf7fe640 1 -- 172.21.15.154:0/1798190080 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac0071e10 con 0x7f4ac00c8d70 2023-09-23T03:16:34.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.929+0000 7f4abf7fe640 1 --2- 172.21.15.154:0/1798190080 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ac00c8d70 0x7f4ac015c6d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4ac00b3ce0 tx=0x7f4ab002e170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa270f52cc25b9b9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:34.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.929+0000 7f4abeffd640 1 -- 172.21.15.154:0/1798190080 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab0036440 con 0x7f4ac00c8d70 2023-09-23T03:16:34.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.929+0000 7f4abeffd640 1 -- 172.21.15.154:0/1798190080 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab00365a0 con 0x7f4ac00c8d70 2023-09-23T03:16:34.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.929+0000 7f4abeffd640 1 -- 172.21.15.154:0/1798190080 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab0036930 con 0x7f4ac00c8d70 2023-09-23T03:16:34.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.930+0000 7f4ac5c6f640 1 -- 172.21.15.154:0/1798190080 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ac00c8d70 msgr2=0x7f4ac015c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:34.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.930+0000 7f4ac5c6f640 1 --2- 172.21.15.154:0/1798190080 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ac00c8d70 0x7f4ac015c6d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4ac00b3ce0 tx=0x7f4ab002e170 comp rx=0 tx=0).stop 2023-09-23T03:16:34.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.930+0000 7f4ac5c6f640 1 -- 172.21.15.154:0/1798190080 shutdown_connections 2023-09-23T03:16:34.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.930+0000 7f4ac5c6f640 1 --2- 172.21.15.154:0/1798190080 >> v2:172.21.15.190:3300/0 conn(0x7f4ac00c9150 0x7f4ac015cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:34.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.930+0000 7f4ac5c6f640 1 --2- 172.21.15.154:0/1798190080 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ac00c8d70 0x7f4ac015c6d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:34.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.930+0000 7f4ac5c6f640 1 -- 172.21.15.154:0/1798190080 >> 172.21.15.154:0/1798190080 conn(0x7f4ac006c010 msgr2=0x7f4ac006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:34.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.930+0000 7f4ac5c6f640 1 -- 172.21.15.154:0/1798190080 shutdown_connections 2023-09-23T03:16:34.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.930+0000 7f4ac5c6f640 1 -- 172.21.15.154:0/1798190080 wait complete. 2023-09-23T03:16:34.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.932+0000 7f4ac5c6f640 1 Processor -- start 2023-09-23T03:16:34.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.932+0000 7f4ac5c6f640 1 -- start start 2023-09-23T03:16:34.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.933+0000 7f4ac5c6f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ac0071290 0x7f4ac00c47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:34.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.933+0000 7f4ac5c6f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ac00c8d70 0x7f4ac014e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:34.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.933+0000 7f4ac5c6f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ac00c9150 0x7f4ac01f1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:34.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.933+0000 7f4ac5c6f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac00c6230 con 0x7f4ac00c8d70 2023-09-23T03:16:34.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.933+0000 7f4ac5c6f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac014ef00 con 0x7f4ac0071290 2023-09-23T03:16:34.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.933+0000 7f4ac5c6f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac01f1f70 con 0x7f4ac00c9150 2023-09-23T03:16:34.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.933+0000 7f4ab7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ac00c8d70 0x7f4ac014e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:34.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.933+0000 7f4ab7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ac00c8d70 0x7f4ac014e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52514/0 (socket says 172.21.15.154:52514) 2023-09-23T03:16:34.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.933+0000 7f4ab7fff640 1 -- 172.21.15.154:0/3294662773 learned_addr learned my addr 172.21.15.154:0/3294662773 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:34.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.934+0000 7f4ab7fff640 1 -- 172.21.15.154:0/3294662773 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ac00c9150 msgr2=0x7f4ac01f1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:34.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.934+0000 7f4ab7fff640 1 --2- 172.21.15.154:0/3294662773 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ac00c9150 0x7f4ac01f1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:34.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.934+0000 7f4ab7fff640 1 -- 172.21.15.154:0/3294662773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ac0071290 msgr2=0x7f4ac00c47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:34.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.934+0000 7f4ab7fff640 1 --2- 172.21.15.154:0/3294662773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ac0071290 0x7f4ac00c47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:34.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.934+0000 7f4ab7fff640 1 -- 172.21.15.154:0/3294662773 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab000b770 con 0x7f4ac00c8d70 2023-09-23T03:16:34.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.934+0000 7f4ab7fff640 1 --2- 172.21.15.154:0/3294662773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ac00c8d70 0x7f4ac014e9c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4aa800d5f0 tx=0x7f4aa800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:34.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.934+0000 7f4abcff9640 1 -- 172.21.15.154:0/3294662773 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa80125f0 con 0x7f4ac00c8d70 2023-09-23T03:16:34.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.934+0000 7f4abcff9640 1 -- 172.21.15.154:0/3294662773 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa8012750 con 0x7f4ac00c8d70 2023-09-23T03:16:34.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.934+0000 7f4abcff9640 1 -- 172.21.15.154:0/3294662773 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa8012b00 con 0x7f4ac00c8d70 2023-09-23T03:16:34.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.934+0000 7f4ac5c6f640 1 -- 172.21.15.154:0/3294662773 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ac00c5620 con 0x7f4ac00c8d70 2023-09-23T03:16:34.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.935+0000 7f4ac5c6f640 1 -- 172.21.15.154:0/3294662773 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac00c5be0 con 0x7f4ac00c8d70 2023-09-23T03:16:34.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.936+0000 7f4ac5c6f640 1 -- 172.21.15.154:0/3294662773 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ac004e920 con 0x7f4ac00c8d70 2023-09-23T03:16:34.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.937+0000 7f4abcff9640 1 -- 172.21.15.154:0/3294662773 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f4aa8012c60 con 0x7f4ac00c8d70 2023-09-23T03:16:34.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.937+0000 7f4abcff9640 1 --2- 172.21.15.154:0/3294662773 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4aa006af60 0x7f4aa006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:34.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.938+0000 7f4abcff9640 1 -- 172.21.15.154:0/3294662773 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f4aa8089fe0 con 0x7f4ac00c8d70 2023-09-23T03:16:34.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.938+0000 7f4abf7fe640 1 --2- 172.21.15.154:0/3294662773 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4aa006af60 0x7f4aa006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:34.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.938+0000 7f4abf7fe640 1 --2- 172.21.15.154:0/3294662773 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4aa006af60 0x7f4aa006d420 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0038000 tx=0x7f4ab0030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:34.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:34.942+0000 7f4abcff9640 1 -- 172.21.15.154:0/3294662773 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f4aa8054bf0 con 0x7f4ac00c8d70 2023-09-23T03:16:35.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.128+0000 7f4ac5c6f640 1 -- 172.21.15.154:0/3294662773 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f4ac00c6370 con 0x7f4ac00c8d70 2023-09-23T03:16:35.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.129+0000 7f4abcff9640 1 -- 172.21.15.154:0/3294662773 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4aa8058210 con 0x7f4ac00c8d70 2023-09-23T03:16:35.130 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:16:35.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.131+0000 7f4ac5c6f640 1 -- 172.21.15.154:0/3294662773 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4aa006af60 msgr2=0x7f4aa006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:35.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.131+0000 7f4ac5c6f640 1 --2- 172.21.15.154:0/3294662773 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4aa006af60 0x7f4aa006d420 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0038000 tx=0x7f4ab0030000 comp rx=0 tx=0).stop 2023-09-23T03:16:35.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.131+0000 7f4ac5c6f640 1 -- 172.21.15.154:0/3294662773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ac00c8d70 msgr2=0x7f4ac014e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:35.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.131+0000 7f4ac5c6f640 1 --2- 172.21.15.154:0/3294662773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ac00c8d70 0x7f4ac014e9c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4aa800d5f0 tx=0x7f4aa800dab0 comp rx=0 tx=0).stop 2023-09-23T03:16:35.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.131+0000 7f4ac5c6f640 1 -- 172.21.15.154:0/3294662773 shutdown_connections 2023-09-23T03:16:35.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.131+0000 7f4ac5c6f640 1 --2- 172.21.15.154:0/3294662773 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4aa006af60 0x7f4aa006d420 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:35.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.131+0000 7f4ac5c6f640 1 --2- 172.21.15.154:0/3294662773 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ac00c9150 0x7f4ac01f1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:35.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.131+0000 7f4ac5c6f640 1 --2- 172.21.15.154:0/3294662773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ac00c8d70 0x7f4ac014e9c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:35.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.131+0000 7f4ac5c6f640 1 --2- 172.21.15.154:0/3294662773 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ac0071290 0x7f4ac00c47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:35.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.131+0000 7f4ac5c6f640 1 -- 172.21.15.154:0/3294662773 >> 172.21.15.154:0/3294662773 conn(0x7f4ac006c010 msgr2=0x7f4ac0152ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:35.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.131+0000 7f4ac5c6f640 1 -- 172.21.15.154:0/3294662773 shutdown_connections 2023-09-23T03:16:35.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.131+0000 7f4ac5c6f640 1 -- 172.21.15.154:0/3294662773 wait complete. 2023-09-23T03:16:35.149 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-23T03:16:35.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.345+0000 7f4e8b475640 1 Processor -- start 2023-09-23T03:16:35.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.346+0000 7f4e8b475640 1 -- start start 2023-09-23T03:16:35.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.347+0000 7f4e8b475640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e840b2fe0 0x7f4e840ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:35.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.347+0000 7f4e8b475640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e8406ceb0 con 0x7f4e840ada30 2023-09-23T03:16:35.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.347+0000 7f4e8b475640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e84159c90 con 0x7f4e840cf130 2023-09-23T03:16:35.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.347+0000 7f4e8b475640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e840ade10 con 0x7f4e840b2fe0 2023-09-23T03:16:35.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.347+0000 7f4e89211640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f4e840ada30 0x7f4e840cea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:42042/0 (socket says 172.21.15.154:42042) 2023-09-23T03:16:35.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.347+0000 7f4e89211640 1 -- 172.21.15.154:0/3728236361 learned_addr learned my addr 172.21.15.154:0/3728236361 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:35.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.348+0000 7f4e89a12640 1 --2- 172.21.15.154:0/3728236361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e840b2fe0 0x7f4e840ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:35.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.348+0000 7f4e83fff640 1 -- 172.21.15.154:0/3728236361 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 81226731 0 0) 0x7f4e74001610 con 0x7f4e840ada30 2023-09-23T03:16:35.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.348+0000 7f4e83fff640 1 -- 172.21.15.154:0/3728236361 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e6c003520 con 0x7f4e840ada30 2023-09-23T03:16:35.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.348+0000 7f4e83fff640 1 -- 172.21.15.154:0/3728236361 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1351384213 0 0) 0x7f4e84159c90 con 0x7f4e840cf130 2023-09-23T03:16:35.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.348+0000 7f4e83fff640 1 -- 172.21.15.154:0/3728236361 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e74001610 con 0x7f4e840cf130 2023-09-23T03:16:35.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.348+0000 7f4e89a12640 1 -- 172.21.15.154:0/3728236361 >> v1:172.21.15.190:6789/0 conn(0x7f4e840cf130 legacy=0x7f4e8415d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:35.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.348+0000 7f4e83fff640 1 -- 172.21.15.154:0/3728236361 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2448357645 0 0) 0x7f4e6c003520 con 0x7f4e840ada30 2023-09-23T03:16:35.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.348+0000 7f4e89a12640 1 -- 172.21.15.154:0/3728236361 >> v1:172.21.15.154:6789/0 conn(0x7f4e840ada30 legacy=0x7f4e840cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:35.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.349+0000 7f4e89a12640 1 -- 172.21.15.154:0/3728236361 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e8406c110 con 0x7f4e840b2fe0 2023-09-23T03:16:35.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.350+0000 7f4e89a12640 1 --2- 172.21.15.154:0/3728236361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e840b2fe0 0x7f4e840ccaf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c0097d0 tx=0x7f4e7c02b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=500e64b681b3d8f8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:35.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.350+0000 7f4e83fff640 1 -- 172.21.15.154:0/3728236361 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e74001d00 con 0x7f4e840b2fe0 2023-09-23T03:16:35.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.350+0000 7f4e83fff640 1 -- 172.21.15.154:0/3728236361 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7c02e050 con 0x7f4e840b2fe0 2023-09-23T03:16:35.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.351+0000 7f4e83fff640 1 -- 172.21.15.154:0/3728236361 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c030710 con 0x7f4e840b2fe0 2023-09-23T03:16:35.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.351+0000 7f4e8b475640 1 -- 172.21.15.154:0/3728236361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e840b2fe0 msgr2=0x7f4e840ccaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:35.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.351+0000 7f4e8b475640 1 --2- 172.21.15.154:0/3728236361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e840b2fe0 0x7f4e840ccaf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c0097d0 tx=0x7f4e7c02b800 comp rx=0 tx=0).stop 2023-09-23T03:16:35.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.351+0000 7f4e8b475640 1 -- 172.21.15.154:0/3728236361 shutdown_connections 2023-09-23T03:16:35.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.351+0000 7f4e8b475640 1 --2- 172.21.15.154:0/3728236361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e840b2fe0 0x7f4e840ccaf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:35.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.351+0000 7f4e8b475640 1 -- 172.21.15.154:0/3728236361 >> 172.21.15.154:0/3728236361 conn(0x7f4e8406a0f0 msgr2=0x7f4e8406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:35.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.351+0000 7f4e8b475640 1 -- 172.21.15.154:0/3728236361 shutdown_connections 2023-09-23T03:16:35.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.351+0000 7f4e8b475640 1 -- 172.21.15.154:0/3728236361 wait complete. 2023-09-23T03:16:35.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.353+0000 7f4e8b475640 1 Processor -- start 2023-09-23T03:16:35.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.353+0000 7f4e8b475640 1 -- start start 2023-09-23T03:16:35.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.354+0000 7f4e8b475640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e840ada30 0x7f4e84097a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:35.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.354+0000 7f4e8b475640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e840b2fe0 0x7f4e8415c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:35.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.354+0000 7f4e8b475640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e840cf130 0x7f4e84153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:35.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.354+0000 7f4e8b475640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e84155b70 con 0x7f4e840ada30 2023-09-23T03:16:35.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.354+0000 7f4e8b475640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8415cd50 con 0x7f4e840cf130 2023-09-23T03:16:35.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.354+0000 7f4e8b475640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8415c530 con 0x7f4e840b2fe0 2023-09-23T03:16:35.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.355+0000 7f4e89211640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e840ada30 0x7f4e84097a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:35.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.355+0000 7f4e89211640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e840ada30 0x7f4e84097a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52528/0 (socket says 172.21.15.154:52528) 2023-09-23T03:16:35.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.355+0000 7f4e89211640 1 -- 172.21.15.154:0/3662274267 learned_addr learned my addr 172.21.15.154:0/3662274267 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:35.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.355+0000 7f4e88a10640 1 --2- 172.21.15.154:0/3662274267 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e840b2fe0 0x7f4e8415c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:35.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.355+0000 7f4e89a12640 1 --2- 172.21.15.154:0/3662274267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e840cf130 0x7f4e84153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:35.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.355+0000 7f4e89211640 1 -- 172.21.15.154:0/3662274267 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e840b2fe0 msgr2=0x7f4e8415c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:35.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.355+0000 7f4e89211640 1 --2- 172.21.15.154:0/3662274267 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e840b2fe0 0x7f4e8415c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:35.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.355+0000 7f4e89211640 1 -- 172.21.15.154:0/3662274267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e840cf130 msgr2=0x7f4e84153300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:35.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.355+0000 7f4e89211640 1 --2- 172.21.15.154:0/3662274267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e840cf130 0x7f4e84153300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:35.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.355+0000 7f4e89211640 1 -- 172.21.15.154:0/3662274267 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e7c009400 con 0x7f4e840ada30 2023-09-23T03:16:35.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.356+0000 7f4e89211640 1 --2- 172.21.15.154:0/3662274267 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e840ada30 0x7f4e84097a90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4e740014d0 tx=0x7f4e7400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:35.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.356+0000 7f4e81ffb640 1 -- 172.21.15.154:0/3662274267 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7400a810 con 0x7f4e840ada30 2023-09-23T03:16:35.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.356+0000 7f4e81ffb640 1 -- 172.21.15.154:0/3662274267 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7400a970 con 0x7f4e840ada30 2023-09-23T03:16:35.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.356+0000 7f4e81ffb640 1 -- 172.21.15.154:0/3662274267 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7400ada0 con 0x7f4e840ada30 2023-09-23T03:16:35.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.356+0000 7f4e8b475640 1 -- 172.21.15.154:0/3662274267 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e84155520 con 0x7f4e840ada30 2023-09-23T03:16:35.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.356+0000 7f4e8b475640 1 -- 172.21.15.154:0/3662274267 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e841596e0 con 0x7f4e840ada30 2023-09-23T03:16:35.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.357+0000 7f4e8b475640 1 -- 172.21.15.154:0/3662274267 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e84073eb0 con 0x7f4e840ada30 2023-09-23T03:16:35.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.358+0000 7f4e81ffb640 1 -- 172.21.15.154:0/3662274267 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f4e74007070 con 0x7f4e840ada30 2023-09-23T03:16:35.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.359+0000 7f4e81ffb640 1 --2- 172.21.15.154:0/3662274267 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4e7006af30 0x7f4e7006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:35.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.359+0000 7f4e81ffb640 1 -- 172.21.15.154:0/3662274267 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f4e74089690 con 0x7f4e840ada30 2023-09-23T03:16:35.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.359+0000 7f4e88a10640 1 --2- 172.21.15.154:0/3662274267 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4e7006af30 0x7f4e7006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:35.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.360+0000 7f4e88a10640 1 --2- 172.21.15.154:0/3662274267 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4e7006af30 0x7f4e7006d3f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4e841f3240 tx=0x7f4e78009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:35.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.364+0000 7f4e81ffb640 1 -- 172.21.15.154:0/3662274267 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f4e74054280 con 0x7f4e840ada30 2023-09-23T03:16:35.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.565+0000 7f4e8b475640 1 -- 172.21.15.154:0/3662274267 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4e8406ceb0 con 0x7f4e840ada30 2023-09-23T03:16:35.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.566+0000 7f4e81ffb640 1 -- 172.21.15.154:0/3662274267 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4e8406ceb0 con 0x7f4e840ada30 2023-09-23T03:16:35.567 INFO:teuthology.orchestra.run.smithi154.stderr:listed 0 entries 2023-09-23T03:16:35.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.568+0000 7f4e8b475640 1 -- 172.21.15.154:0/3662274267 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4e7006af30 msgr2=0x7f4e7006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:35.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.568+0000 7f4e8b475640 1 --2- 172.21.15.154:0/3662274267 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4e7006af30 0x7f4e7006d3f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4e841f3240 tx=0x7f4e78009040 comp rx=0 tx=0).stop 2023-09-23T03:16:35.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.568+0000 7f4e8b475640 1 -- 172.21.15.154:0/3662274267 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e840ada30 msgr2=0x7f4e84097a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:35.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.568+0000 7f4e8b475640 1 --2- 172.21.15.154:0/3662274267 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e840ada30 0x7f4e84097a90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4e740014d0 tx=0x7f4e7400a060 comp rx=0 tx=0).stop 2023-09-23T03:16:35.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.568+0000 7f4e8b475640 1 -- 172.21.15.154:0/3662274267 shutdown_connections 2023-09-23T03:16:35.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.568+0000 7f4e8b475640 1 --2- 172.21.15.154:0/3662274267 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4e7006af30 0x7f4e7006d3f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:35.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.568+0000 7f4e8b475640 1 --2- 172.21.15.154:0/3662274267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e840cf130 0x7f4e84153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:35.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.568+0000 7f4e8b475640 1 --2- 172.21.15.154:0/3662274267 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e840b2fe0 0x7f4e8415c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:35.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.568+0000 7f4e8b475640 1 --2- 172.21.15.154:0/3662274267 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e840ada30 0x7f4e84097a90 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:35.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.568+0000 7f4e8b475640 1 -- 172.21.15.154:0/3662274267 >> 172.21.15.154:0/3662274267 conn(0x7f4e8406a0f0 msgr2=0x7f4e84070040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:35.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.568+0000 7f4e8b475640 1 -- 172.21.15.154:0/3662274267 shutdown_connections 2023-09-23T03:16:35.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:35.568+0000 7f4e8b475640 1 -- 172.21.15.154:0/3662274267 wait complete. 2023-09-23T03:16:35.586 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi154.front.sepia.ceph.com... 2023-09-23T03:16:35.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:35.586 DEBUG:teuthology.orchestra.run.smithi154:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-09-23T03:16:35.682 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:16:35.682 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2023-09-23T03:16:35.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:35.682 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-09-23T03:16:35.729 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:16:35.729 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:16:35.791 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:16:35.791 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:16:35.845 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:16:36.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.064+0000 7ff0f9dae640 1 Processor -- start 2023-09-23T03:16:36.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.064+0000 7ff0f9dae640 1 -- start start 2023-09-23T03:16:36.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.066+0000 7ff0f9dae640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff0f406c470 0x7ff0f4164110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:36.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.066+0000 7ff0f9dae640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0f4159a00 con 0x7ff0f40c4970 2023-09-23T03:16:36.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.066+0000 7ff0f9dae640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0f4071cc0 con 0x7ff0f4071870 2023-09-23T03:16:36.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.066+0000 7ff0f9dae640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0f40cbe90 con 0x7ff0f406c470 2023-09-23T03:16:36.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.066+0000 7ff0f8dac640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff0f40c4970 0x7ff0f40ca260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:42046/0 (socket says 172.21.15.154:42046) 2023-09-23T03:16:36.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.066+0000 7ff0f8dac640 1 -- 172.21.15.154:0/1575486294 learned_addr learned my addr 172.21.15.154:0/1575486294 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:36.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.066+0000 7ff0f37fe640 1 -- 172.21.15.154:0/1575486294 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1335025493 0 0) 0x7ff0f4159a00 con 0x7ff0f40c4970 2023-09-23T03:16:36.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.066+0000 7ff0f37fe640 1 -- 172.21.15.154:0/1575486294 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0e4005730 con 0x7ff0f40c4970 2023-09-23T03:16:36.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.067+0000 7ff0f95ad640 1 --2- 172.21.15.154:0/1575486294 >> v2:172.21.15.190:3300/0 conn(0x7ff0f406c470 0x7ff0f4164110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:36.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.067+0000 7ff0f37fe640 1 -- 172.21.15.154:0/1575486294 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 409572815 0 0) 0x7ff0e4005730 con 0x7ff0f40c4970 2023-09-23T03:16:36.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.067+0000 7ff0f37fe640 1 -- 172.21.15.154:0/1575486294 >> v2:172.21.15.190:3300/0 conn(0x7ff0f406c470 msgr2=0x7ff0f4164110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:36.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.067+0000 7ff0f37fe640 1 --2- 172.21.15.154:0/1575486294 >> v2:172.21.15.190:3300/0 conn(0x7ff0f406c470 0x7ff0f4164110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:36.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.067+0000 7ff0f37fe640 1 -- 172.21.15.154:0/1575486294 >> v1:172.21.15.190:6789/0 conn(0x7ff0f4071870 legacy=0x7ff0f406bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:36.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.067+0000 7ff0f37fe640 1 -- 172.21.15.154:0/1575486294 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f4071f70 con 0x7ff0f40c4970 2023-09-23T03:16:36.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.067+0000 7ff0f37fe640 1 -- 172.21.15.154:0/1575486294 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7ff0e8001670 con 0x7ff0f40c4970 2023-09-23T03:16:36.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.067+0000 7ff0f37fe640 1 -- 172.21.15.154:0/1575486294 >> v1:172.21.15.154:6789/0 conn(0x7ff0f40c4970 legacy=0x7ff0f40ca260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:36.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.067+0000 7ff0f37fe640 1 --2- 172.21.15.154:0/1575486294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0e4005bd0 0x7ff0e4005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:36.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.067+0000 7ff0f37fe640 1 --2- 172.21.15.154:0/1575486294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0e40069d0 0x7ff0e4006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:36.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.067+0000 7ff0f37fe640 1 --2- 172.21.15.154:0/1575486294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0e40074b0 0x7ff0e400db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:36.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.067+0000 7ff0f37fe640 1 -- 172.21.15.154:0/1575486294 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f40cbe90 con 0x7ff0e40069d0 2023-09-23T03:16:36.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.067+0000 7ff0f37fe640 1 -- 172.21.15.154:0/1575486294 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e400e400 con 0x7ff0e4005bd0 2023-09-23T03:16:36.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.067+0000 7ff0f37fe640 1 -- 172.21.15.154:0/1575486294 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e400e5c0 con 0x7ff0e40074b0 2023-09-23T03:16:36.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.067+0000 7ff0f3fff640 1 --2- 172.21.15.154:0/1575486294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0e40069d0 0x7ff0e4006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:36.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.067+0000 7ff0f8dac640 1 --2- 172.21.15.154:0/1575486294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0e4005bd0 0x7ff0e4005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:36.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.068+0000 7ff0f95ad640 1 --2- 172.21.15.154:0/1575486294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0e40074b0 0x7ff0e400db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:36.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.068+0000 7ff0f3fff640 1 -- 172.21.15.154:0/1575486294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0e40074b0 msgr2=0x7ff0e400db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:36.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.068+0000 7ff0f3fff640 1 --2- 172.21.15.154:0/1575486294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0e40074b0 0x7ff0e400db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:36.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.068+0000 7ff0f3fff640 1 -- 172.21.15.154:0/1575486294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0e4005bd0 msgr2=0x7ff0e4005fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:36.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.068+0000 7ff0f3fff640 1 --2- 172.21.15.154:0/1575486294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0e4005bd0 0x7ff0e4005fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:36.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.068+0000 7ff0f3fff640 1 -- 172.21.15.154:0/1575486294 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e4005730 con 0x7ff0e40069d0 2023-09-23T03:16:36.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.068+0000 7ff0f95ad640 1 -- 172.21.15.154:0/1575486294 reap_dead start 2023-09-23T03:16:36.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.069+0000 7ff0f3fff640 1 --2- 172.21.15.154:0/1575486294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0e40069d0 0x7ff0e4006e50 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0003d80 tx=0x7ff0e4007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985b45933efefef6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:36.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.069+0000 7ff0f37fe640 1 -- 172.21.15.154:0/1575486294 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0032600 con 0x7ff0e40069d0 2023-09-23T03:16:36.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.069+0000 7ff0f37fe640 1 -- 172.21.15.154:0/1575486294 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e002e050 con 0x7ff0e40069d0 2023-09-23T03:16:36.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.069+0000 7ff0f37fe640 1 -- 172.21.15.154:0/1575486294 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e00329e0 con 0x7ff0e40069d0 2023-09-23T03:16:36.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.070+0000 7ff0f9dae640 1 -- 172.21.15.154:0/1575486294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0e40069d0 msgr2=0x7ff0e4006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:36.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.070+0000 7ff0f9dae640 1 --2- 172.21.15.154:0/1575486294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0e40069d0 0x7ff0e4006e50 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0003d80 tx=0x7ff0e4007a00 comp rx=0 tx=0).stop 2023-09-23T03:16:36.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.070+0000 7ff0f9dae640 1 -- 172.21.15.154:0/1575486294 shutdown_connections 2023-09-23T03:16:36.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.070+0000 7ff0f9dae640 1 --2- 172.21.15.154:0/1575486294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0e40069d0 0x7ff0e4006e50 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:36.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.070+0000 7ff0f9dae640 1 -- 172.21.15.154:0/1575486294 >> 172.21.15.154:0/1575486294 conn(0x7ff0f406a390 msgr2=0x7ff0f40afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:36.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.070+0000 7ff0f9dae640 1 -- 172.21.15.154:0/1575486294 shutdown_connections 2023-09-23T03:16:36.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.070+0000 7ff0f9dae640 1 -- 172.21.15.154:0/1575486294 wait complete. 2023-09-23T03:16:36.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.072+0000 7ff0f9dae640 1 Processor -- start 2023-09-23T03:16:36.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.072+0000 7ff0f9dae640 1 -- start start 2023-09-23T03:16:36.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.073+0000 7ff0f9dae640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0e40069d0 0x7ff0f40b2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:36.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.073+0000 7ff0f9dae640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0f415b5d0 0x7ff0f40c7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:36.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.073+0000 7ff0f9dae640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0f40c8060 0x7ff0f40b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:36.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.073+0000 7ff0f9dae640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f40b1280 con 0x7ff0f40c8060 2023-09-23T03:16:36.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.073+0000 7ff0f9dae640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f40b2ab0 con 0x7ff0e40069d0 2023-09-23T03:16:36.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.073+0000 7ff0f9dae640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f40b14d0 con 0x7ff0f415b5d0 2023-09-23T03:16:36.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.073+0000 7ff0f95ad640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0f40c8060 0x7ff0f40b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:36.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.073+0000 7ff0f95ad640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0f40c8060 0x7ff0f40b35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52544/0 (socket says 172.21.15.154:52544) 2023-09-23T03:16:36.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.073+0000 7ff0f95ad640 1 -- 172.21.15.154:0/3274725475 learned_addr learned my addr 172.21.15.154:0/3274725475 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:36.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.073+0000 7ff0f8dac640 1 --2- 172.21.15.154:0/3274725475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0e40069d0 0x7ff0f40b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:36.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.074+0000 7ff0f3fff640 1 --2- 172.21.15.154:0/3274725475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0f415b5d0 0x7ff0f40c7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:36.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.074+0000 7ff0f95ad640 1 -- 172.21.15.154:0/3274725475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0f415b5d0 msgr2=0x7ff0f40c7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:36.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.074+0000 7ff0f95ad640 1 --2- 172.21.15.154:0/3274725475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0f415b5d0 0x7ff0f40c7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:36.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.074+0000 7ff0f95ad640 1 -- 172.21.15.154:0/3274725475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0e40069d0 msgr2=0x7ff0f40b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:36.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.074+0000 7ff0f95ad640 1 --2- 172.21.15.154:0/3274725475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0e40069d0 0x7ff0f40b2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:36.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.074+0000 7ff0f95ad640 1 -- 172.21.15.154:0/3274725475 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f40b17b0 con 0x7ff0f40c8060 2023-09-23T03:16:36.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.074+0000 7ff0f95ad640 1 --2- 172.21.15.154:0/3274725475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0f40c8060 0x7ff0f40b35c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff0f40705f0 tx=0x7ff0ec00de30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:36.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.074+0000 7ff0f17fa640 1 -- 172.21.15.154:0/3274725475 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec00bcf0 con 0x7ff0f40c8060 2023-09-23T03:16:36.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.074+0000 7ff0f17fa640 1 -- 172.21.15.154:0/3274725475 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0ec00be50 con 0x7ff0f40c8060 2023-09-23T03:16:36.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.074+0000 7ff0f9dae640 1 -- 172.21.15.154:0/3274725475 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0f40b1a90 con 0x7ff0f40c8060 2023-09-23T03:16:36.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.074+0000 7ff0f17fa640 1 -- 172.21.15.154:0/3274725475 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec016680 con 0x7ff0f40c8060 2023-09-23T03:16:36.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.074+0000 7ff0f9dae640 1 -- 172.21.15.154:0/3274725475 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0f40b2050 con 0x7ff0f40c8060 2023-09-23T03:16:36.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.075+0000 7ff0f9dae640 1 -- 172.21.15.154:0/3274725475 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0f40736b0 con 0x7ff0f40c8060 2023-09-23T03:16:36.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.076+0000 7ff0f17fa640 1 -- 172.21.15.154:0/3274725475 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7ff0ec00e3f0 con 0x7ff0f40c8060 2023-09-23T03:16:36.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.077+0000 7ff0f17fa640 1 --2- 172.21.15.154:0/3274725475 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff0d806af60 0x7ff0d806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:36.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.077+0000 7ff0f17fa640 1 -- 172.21.15.154:0/3274725475 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7ff0ec089ea0 con 0x7ff0f40c8060 2023-09-23T03:16:36.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.077+0000 7ff0f8dac640 1 --2- 172.21.15.154:0/3274725475 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff0d806af60 0x7ff0d806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:36.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.078+0000 7ff0f8dac640 1 --2- 172.21.15.154:0/3274725475 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff0d806af60 0x7ff0d806d420 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8001eb0 tx=0x7ff0e8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:36.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.081+0000 7ff0f17fa640 1 -- 172.21.15.154:0/3274725475 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7ff0ec054a90 con 0x7ff0f40c8060 2023-09-23T03:16:36.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.275+0000 7ff0f9dae640 1 -- 172.21.15.154:0/3274725475 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff0f40b2270 con 0x7ff0f40c8060 2023-09-23T03:16:36.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.276+0000 7ff0f17fa640 1 -- 172.21.15.154:0/3274725475 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18744 (secure 0 0 0) 0x7ff0ec01b020 con 0x7ff0f40c8060 2023-09-23T03:16:36.278 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:16:36.278 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":26,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:16:03.151371+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:15:18.575849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T03:15:19.707212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T03:15:23.337138+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:16:36.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.279+0000 7ff0f9dae640 1 -- 172.21.15.154:0/3274725475 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff0d806af60 msgr2=0x7ff0d806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:36.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.279+0000 7ff0f9dae640 1 --2- 172.21.15.154:0/3274725475 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff0d806af60 0x7ff0d806d420 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8001eb0 tx=0x7ff0e8009040 comp rx=0 tx=0).stop 2023-09-23T03:16:36.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.279+0000 7ff0f9dae640 1 -- 172.21.15.154:0/3274725475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0f40c8060 msgr2=0x7ff0f40b35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:36.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.279+0000 7ff0f9dae640 1 --2- 172.21.15.154:0/3274725475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0f40c8060 0x7ff0f40b35c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff0f40705f0 tx=0x7ff0ec00de30 comp rx=0 tx=0).stop 2023-09-23T03:16:36.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.279+0000 7ff0f9dae640 1 -- 172.21.15.154:0/3274725475 shutdown_connections 2023-09-23T03:16:36.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.279+0000 7ff0f9dae640 1 --2- 172.21.15.154:0/3274725475 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff0d806af60 0x7ff0d806d420 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:36.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.279+0000 7ff0f9dae640 1 --2- 172.21.15.154:0/3274725475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0f40c8060 0x7ff0f40b35c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:36.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.279+0000 7ff0f9dae640 1 --2- 172.21.15.154:0/3274725475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0f415b5d0 0x7ff0f40c7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:36.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.279+0000 7ff0f9dae640 1 --2- 172.21.15.154:0/3274725475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0e40069d0 0x7ff0f40b2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:36.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.279+0000 7ff0f9dae640 1 -- 172.21.15.154:0/3274725475 >> 172.21.15.154:0/3274725475 conn(0x7ff0f406a390 msgr2=0x7ff0f40cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:36.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.279+0000 7ff0f9dae640 1 -- 172.21.15.154:0/3274725475 shutdown_connections 2023-09-23T03:16:36.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.279+0000 7ff0f9dae640 1 -- 172.21.15.154:0/3274725475 wait complete. 2023-09-23T03:16:36.298 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:15:18.575849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:15:19.707212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-23T03:15:23.337138+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-23T03:16:36.299 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:16:36.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.515+0000 7f6174a44640 1 Processor -- start 2023-09-23T03:16:36.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.515+0000 7f6174a44640 1 -- start start 2023-09-23T03:16:36.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.517+0000 7f6174a44640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f61700ad830 0x7f6170070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:36.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.517+0000 7f6174a44640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61700c9680 0x7f61700caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:36.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.517+0000 7f6174a44640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f61700cb400 0x7f6170160720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:36.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.517+0000 7f6174a44640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61700cb890 con 0x7f61700cb400 2023-09-23T03:16:36.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.517+0000 7f6174a44640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61700712f0 con 0x7f61700ad830 2023-09-23T03:16:36.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.517+0000 7f6174a44640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61700715d0 con 0x7f61700c9680 2023-09-23T03:16:36.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.517+0000 7f616ed9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f61700cb400 0x7f6170160720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:36.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.517+0000 7f616e59c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f61700ad830 0x7f6170070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:36.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.517+0000 7f616ed9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f61700cb400 0x7f6170160720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52550/0 (socket says 172.21.15.154:52550) 2023-09-23T03:16:36.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.517+0000 7f616ed9d640 1 -- 172.21.15.154:0/4182285428 learned_addr learned my addr 172.21.15.154:0/4182285428 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:36.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.517+0000 7f616dd9b640 1 --2- 172.21.15.154:0/4182285428 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61700c9680 0x7f61700caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:36.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.518+0000 7f616ed9d640 1 -- 172.21.15.154:0/4182285428 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61700c9680 msgr2=0x7f61700caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:36.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.518+0000 7f616ed9d640 1 --2- 172.21.15.154:0/4182285428 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61700c9680 0x7f61700caec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:36.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.518+0000 7f616ed9d640 1 -- 172.21.15.154:0/4182285428 >> v2:172.21.15.190:3300/0 conn(0x7f61700ad830 msgr2=0x7f6170070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:36.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.518+0000 7f616ed9d640 1 --2- 172.21.15.154:0/4182285428 >> v2:172.21.15.190:3300/0 conn(0x7f61700ad830 0x7f6170070c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:36.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.518+0000 7f616ed9d640 1 -- 172.21.15.154:0/4182285428 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61700718b0 con 0x7f61700cb400 2023-09-23T03:16:36.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.518+0000 7f616e59c640 1 --2- 172.21.15.154:0/4182285428 >> v2:172.21.15.190:3300/0 conn(0x7f61700ad830 0x7f6170070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:16:36.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.519+0000 7f616ed9d640 1 --2- 172.21.15.154:0/4182285428 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61700cb400 0x7f6170160720 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f617006d9c0 tx=0x7f616802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc9e17b22bd6a3e3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:36.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.520+0000 7f616d59a640 1 -- 172.21.15.154:0/4182285428 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616802ca80 con 0x7f61700cb400 2023-09-23T03:16:36.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.520+0000 7f616d59a640 1 -- 172.21.15.154:0/4182285428 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f616802cbe0 con 0x7f61700cb400 2023-09-23T03:16:36.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.520+0000 7f616d59a640 1 -- 172.21.15.154:0/4182285428 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61680343f0 con 0x7f61700cb400 2023-09-23T03:16:36.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.520+0000 7f6174a44640 1 -- 172.21.15.154:0/4182285428 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61700cb400 msgr2=0x7f6170160720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:36.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.520+0000 7f6174a44640 1 --2- 172.21.15.154:0/4182285428 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61700cb400 0x7f6170160720 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f617006d9c0 tx=0x7f616802bee0 comp rx=0 tx=0).stop 2023-09-23T03:16:36.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.521+0000 7f6174a44640 1 -- 172.21.15.154:0/4182285428 shutdown_connections 2023-09-23T03:16:36.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.521+0000 7f6174a44640 1 --2- 172.21.15.154:0/4182285428 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61700cb400 0x7f6170160720 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:36.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.521+0000 7f6174a44640 1 --2- 172.21.15.154:0/4182285428 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61700c9680 0x7f61700caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:36.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.521+0000 7f6174a44640 1 --2- 172.21.15.154:0/4182285428 >> v2:172.21.15.190:3300/0 conn(0x7f61700ad830 0x7f6170070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:36.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.521+0000 7f6174a44640 1 -- 172.21.15.154:0/4182285428 >> 172.21.15.154:0/4182285428 conn(0x7f617006c010 msgr2=0x7f617006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:36.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.521+0000 7f6174a44640 1 -- 172.21.15.154:0/4182285428 shutdown_connections 2023-09-23T03:16:36.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.521+0000 7f6174a44640 1 -- 172.21.15.154:0/4182285428 wait complete. 2023-09-23T03:16:36.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.523+0000 7f6174a44640 1 Processor -- start 2023-09-23T03:16:36.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.523+0000 7f6174a44640 1 -- start start 2023-09-23T03:16:36.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.524+0000 7f6174a44640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61700ad830 0x7f61701f6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:36.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.524+0000 7f6174a44640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61700c9680 0x7f61700b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:36.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.524+0000 7f6174a44640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61700cb400 0x7f61700cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:36.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.524+0000 7f6174a44640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61700b1cd0 con 0x7f61700cb400 2023-09-23T03:16:36.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.524+0000 7f6174a44640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61701f3380 con 0x7f61700ad830 2023-09-23T03:16:36.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.524+0000 7f6174a44640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61701f30a0 con 0x7f61700c9680 2023-09-23T03:16:36.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.524+0000 7f616ed9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61700cb400 0x7f61700cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:36.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.525+0000 7f616ed9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61700cb400 0x7f61700cd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52566/0 (socket says 172.21.15.154:52566) 2023-09-23T03:16:36.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.525+0000 7f616ed9d640 1 -- 172.21.15.154:0/3119463 learned_addr learned my addr 172.21.15.154:0/3119463 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:36.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.525+0000 7f616e59c640 1 --2- 172.21.15.154:0/3119463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61700ad830 0x7f61701f6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:36.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.525+0000 7f616dd9b640 1 --2- 172.21.15.154:0/3119463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61700c9680 0x7f61700b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:36.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.525+0000 7f616ed9d640 1 -- 172.21.15.154:0/3119463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61700c9680 msgr2=0x7f61700b24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:36.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.525+0000 7f616ed9d640 1 --2- 172.21.15.154:0/3119463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61700c9680 0x7f61700b24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:36.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.525+0000 7f616ed9d640 1 -- 172.21.15.154:0/3119463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61700ad830 msgr2=0x7f61701f6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:36.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.525+0000 7f616ed9d640 1 --2- 172.21.15.154:0/3119463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61700ad830 0x7f61701f6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:36.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.525+0000 7f616ed9d640 1 -- 172.21.15.154:0/3119463 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6160009400 con 0x7f61700cb400 2023-09-23T03:16:36.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.525+0000 7f616dd9b640 1 --2- 172.21.15.154:0/3119463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61700c9680 0x7f61700b24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:36.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.525+0000 7f616ed9d640 1 --2- 172.21.15.154:0/3119463 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61700cb400 0x7f61700cd8a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6168036040 tx=0x7f6168004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:36.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.526+0000 7f615effd640 1 -- 172.21.15.154:0/3119463 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6168009510 con 0x7f61700cb400 2023-09-23T03:16:36.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.526+0000 7f615effd640 1 -- 172.21.15.154:0/3119463 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f616803e450 con 0x7f61700cb400 2023-09-23T03:16:36.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.526+0000 7f6174a44640 1 -- 172.21.15.154:0/3119463 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6168009b30 con 0x7f61700cb400 2023-09-23T03:16:36.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.526+0000 7f615effd640 1 -- 172.21.15.154:0/3119463 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616803e7e0 con 0x7f61700cb400 2023-09-23T03:16:36.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.526+0000 7f6174a44640 1 -- 172.21.15.154:0/3119463 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61701f5e30 con 0x7f61700cb400 2023-09-23T03:16:36.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.527+0000 7f6174a44640 1 -- 172.21.15.154:0/3119463 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f617004e920 con 0x7f61700cb400 2023-09-23T03:16:36.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.528+0000 7f615effd640 1 -- 172.21.15.154:0/3119463 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f616802cd50 con 0x7f61700cb400 2023-09-23T03:16:36.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.530+0000 7f615effd640 1 --2- 172.21.15.154:0/3119463 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f615006af30 0x7f615006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:36.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.530+0000 7f615effd640 1 -- 172.21.15.154:0/3119463 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f61680ac560 con 0x7f61700cb400 2023-09-23T03:16:36.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.530+0000 7f616e59c640 1 --2- 172.21.15.154:0/3119463 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f615006af30 0x7f615006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:36.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.530+0000 7f616e59c640 1 --2- 172.21.15.154:0/3119463 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f615006af30 0x7f615006d3f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6160009b40 tx=0x7f6160008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:36.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.534+0000 7f615effd640 1 -- 172.21.15.154:0/3119463 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f61680771e0 con 0x7f61700cb400 2023-09-23T03:16:36.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.721+0000 7f6174a44640 1 -- 172.21.15.154:0/3119463 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6170072140 con 0x7f61700cb400 2023-09-23T03:16:36.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.722+0000 7f615effd640 1 -- 172.21.15.154:0/3119463 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f616807a800 con 0x7f61700cb400 2023-09-23T03:16:36.723 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:16:36.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.725+0000 7f6174a44640 1 -- 172.21.15.154:0/3119463 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f615006af30 msgr2=0x7f615006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:36.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.725+0000 7f6174a44640 1 --2- 172.21.15.154:0/3119463 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f615006af30 0x7f615006d3f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6160009b40 tx=0x7f6160008040 comp rx=0 tx=0).stop 2023-09-23T03:16:36.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.725+0000 7f6174a44640 1 -- 172.21.15.154:0/3119463 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61700cb400 msgr2=0x7f61700cd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:36.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.725+0000 7f6174a44640 1 --2- 172.21.15.154:0/3119463 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61700cb400 0x7f61700cd8a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6168036040 tx=0x7f6168004a40 comp rx=0 tx=0).stop 2023-09-23T03:16:36.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.725+0000 7f6174a44640 1 -- 172.21.15.154:0/3119463 shutdown_connections 2023-09-23T03:16:36.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.725+0000 7f6174a44640 1 --2- 172.21.15.154:0/3119463 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f615006af30 0x7f615006d3f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:36.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.725+0000 7f6174a44640 1 --2- 172.21.15.154:0/3119463 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61700cb400 0x7f61700cd8a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:36.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.725+0000 7f6174a44640 1 --2- 172.21.15.154:0/3119463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61700c9680 0x7f61700b24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:36.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.725+0000 7f6174a44640 1 --2- 172.21.15.154:0/3119463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61700ad830 0x7f61701f6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:36.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.725+0000 7f6174a44640 1 -- 172.21.15.154:0/3119463 >> 172.21.15.154:0/3119463 conn(0x7f617006c010 msgr2=0x7f61700af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:36.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.725+0000 7f6174a44640 1 -- 172.21.15.154:0/3119463 shutdown_connections 2023-09-23T03:16:36.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.725+0000 7f6174a44640 1 -- 172.21.15.154:0/3119463 wait complete. 2023-09-23T03:16:36.745 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:16:36.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.962+0000 7eff4e2f2640 1 Processor -- start 2023-09-23T03:16:36.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.963+0000 7eff4e2f2640 1 -- start start 2023-09-23T03:16:36.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.964+0000 7eff4e2f2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff4815abe0 0x7eff480cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:36.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.964+0000 7eff4e2f2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7eff480cc9b0 0x7eff480b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:36.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.964+0000 7eff4e2f2640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff480ce510 con 0x7eff480cce10 2023-09-23T03:16:36.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.964+0000 7eff4e2f2640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff480cb710 con 0x7eff480cc9b0 2023-09-23T03:16:36.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.964+0000 7eff4e2f2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff480cd390 con 0x7eff4815abe0 2023-09-23T03:16:36.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.965+0000 7eff3f7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7eff480cc9b0 0x7eff480b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:36.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.965+0000 7eff3f7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7eff480cc9b0 0x7eff480b21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52578/0 (socket says 172.21.15.154:52578) 2023-09-23T03:16:36.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.965+0000 7eff3f7fe640 1 -- 172.21.15.154:0/2110445370 learned_addr learned my addr 172.21.15.154:0/2110445370 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:36.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.965+0000 7eff47fff640 1 --2- 172.21.15.154:0/2110445370 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff4815abe0 0x7eff480cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:36.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.965+0000 7eff3f7fe640 1 -- 172.21.15.154:0/2110445370 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff4815abe0 msgr2=0x7eff480cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:36.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.965+0000 7eff3f7fe640 1 --2- 172.21.15.154:0/2110445370 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff4815abe0 0x7eff480cb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:36.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.965+0000 7eff3f7fe640 1 -- 172.21.15.154:0/2110445370 >> v1:172.21.15.154:6789/0 conn(0x7eff480cce10 legacy=0x7eff480ce9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:36.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.965+0000 7eff3f7fe640 1 -- 172.21.15.154:0/2110445370 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff480cd670 con 0x7eff480cc9b0 2023-09-23T03:16:36.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.966+0000 7eff3f7fe640 1 --2- 172.21.15.154:0/2110445370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff480cc9b0 0x7eff480b21b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7eff30009e50 tx=0x7eff3002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f12f51c8ee8c0da server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:36.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.967+0000 7eff477fe640 1 -- 172.21.15.154:0/2110445370 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3002ca70 con 0x7eff480cc9b0 2023-09-23T03:16:36.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.967+0000 7eff477fe640 1 -- 172.21.15.154:0/2110445370 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff3002cbd0 con 0x7eff480cc9b0 2023-09-23T03:16:36.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.967+0000 7eff477fe640 1 -- 172.21.15.154:0/2110445370 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff300343f0 con 0x7eff480cc9b0 2023-09-23T03:16:36.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.967+0000 7eff4e2f2640 1 -- 172.21.15.154:0/2110445370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff480cc9b0 msgr2=0x7eff480b21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:36.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.967+0000 7eff4e2f2640 1 --2- 172.21.15.154:0/2110445370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff480cc9b0 0x7eff480b21b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7eff30009e50 tx=0x7eff3002bed0 comp rx=0 tx=0).stop 2023-09-23T03:16:36.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.967+0000 7eff4e2f2640 1 -- 172.21.15.154:0/2110445370 shutdown_connections 2023-09-23T03:16:36.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.967+0000 7eff4e2f2640 1 --2- 172.21.15.154:0/2110445370 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff480cc9b0 0x7eff480b21b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:36.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.967+0000 7eff4e2f2640 1 --2- 172.21.15.154:0/2110445370 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff4815abe0 0x7eff480cb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:36.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.967+0000 7eff4e2f2640 1 -- 172.21.15.154:0/2110445370 >> 172.21.15.154:0/2110445370 conn(0x7eff4806fc40 msgr2=0x7eff48070020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:36.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.967+0000 7eff4e2f2640 1 -- 172.21.15.154:0/2110445370 shutdown_connections 2023-09-23T03:16:36.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.967+0000 7eff4e2f2640 1 -- 172.21.15.154:0/2110445370 wait complete. 2023-09-23T03:16:36.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.969+0000 7eff4e2f2640 1 Processor -- start 2023-09-23T03:16:36.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.969+0000 7eff4e2f2640 1 -- start start 2023-09-23T03:16:36.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.970+0000 7eff4e2f2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff480cc9b0 0x7eff480c7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:36.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.970+0000 7eff4e2f2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff480cce10 0x7eff480c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:36.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.970+0000 7eff4e2f2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff4815abe0 0x7eff48149bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:36.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.970+0000 7eff4e2f2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4814a110 con 0x7eff4815abe0 2023-09-23T03:16:36.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.970+0000 7eff4e2f2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff480c4de0 con 0x7eff480cce10 2023-09-23T03:16:36.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.970+0000 7eff4e2f2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff480c8960 con 0x7eff480cc9b0 2023-09-23T03:16:36.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.971+0000 7eff4c88f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff4815abe0 0x7eff48149bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:36.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.971+0000 7eff4c88f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff4815abe0 0x7eff48149bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52580/0 (socket says 172.21.15.154:52580) 2023-09-23T03:16:36.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.971+0000 7eff4c88f640 1 -- 172.21.15.154:0/2567083929 learned_addr learned my addr 172.21.15.154:0/2567083929 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:36.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.971+0000 7eff47fff640 1 --2- 172.21.15.154:0/2567083929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff480cc9b0 0x7eff480c7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:36.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.971+0000 7eff4c88f640 1 -- 172.21.15.154:0/2567083929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff480cc9b0 msgr2=0x7eff480c7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:36.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.971+0000 7eff4c88f640 1 --2- 172.21.15.154:0/2567083929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff480cc9b0 0x7eff480c7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:36.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.971+0000 7eff4c88f640 1 -- 172.21.15.154:0/2567083929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff480cce10 msgr2=0x7eff480c9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:36.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.971+0000 7eff3f7fe640 1 --2- 172.21.15.154:0/2567083929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff480cce10 0x7eff480c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:36.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.971+0000 7eff4c88f640 1 --2- 172.21.15.154:0/2567083929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff480cce10 0x7eff480c9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:36.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.971+0000 7eff4c88f640 1 -- 172.21.15.154:0/2567083929 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff30009b30 con 0x7eff4815abe0 2023-09-23T03:16:36.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.971+0000 7eff47fff640 1 --2- 172.21.15.154:0/2567083929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff480cc9b0 0x7eff480c7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:36.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.971+0000 7eff4c88f640 1 --2- 172.21.15.154:0/2567083929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff4815abe0 0x7eff48149bd0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7eff4000c3c0 tx=0x7eff4000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:36.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.972+0000 7eff457fa640 1 -- 172.21.15.154:0/2567083929 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff400052d0 con 0x7eff4815abe0 2023-09-23T03:16:36.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.972+0000 7eff457fa640 1 -- 172.21.15.154:0/2567083929 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff40005430 con 0x7eff4815abe0 2023-09-23T03:16:36.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.972+0000 7eff457fa640 1 -- 172.21.15.154:0/2567083929 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff40003810 con 0x7eff4815abe0 2023-09-23T03:16:36.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.972+0000 7eff4e2f2640 1 -- 172.21.15.154:0/2567083929 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff480b1ef0 con 0x7eff4815abe0 2023-09-23T03:16:36.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.972+0000 7eff4e2f2640 1 -- 172.21.15.154:0/2567083929 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff480c80c0 con 0x7eff4815abe0 2023-09-23T03:16:36.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.973+0000 7eff4e2f2640 1 -- 172.21.15.154:0/2567083929 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff480c4f70 con 0x7eff4815abe0 2023-09-23T03:16:36.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.974+0000 7eff457fa640 1 -- 172.21.15.154:0/2567083929 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7eff40003970 con 0x7eff4815abe0 2023-09-23T03:16:36.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.975+0000 7eff457fa640 1 --2- 172.21.15.154:0/2567083929 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7eff2806af30 0x7eff2806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:36.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.975+0000 7eff457fa640 1 -- 172.21.15.154:0/2567083929 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7eff40089070 con 0x7eff4815abe0 2023-09-23T03:16:36.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.975+0000 7eff47fff640 1 --2- 172.21.15.154:0/2567083929 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7eff2806af30 0x7eff2806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:36.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.976+0000 7eff47fff640 1 --2- 172.21.15.154:0/2567083929 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7eff2806af30 0x7eff2806d3f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7eff38009780 tx=0x7eff38008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:36.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:36.979+0000 7eff457fa640 1 -- 172.21.15.154:0/2567083929 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7eff400bba50 con 0x7eff4815abe0 2023-09-23T03:16:37.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.171+0000 7eff4e2f2640 1 -- 172.21.15.154:0/2567083929 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7eff4804e920 con 0x7eff4815abe0 2023-09-23T03:16:37.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.172+0000 7eff457fa640 1 -- 172.21.15.154:0/2567083929 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7eff40017020 con 0x7eff4815abe0 2023-09-23T03:16:37.174 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:16:37.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.175+0000 7eff4e2f2640 1 -- 172.21.15.154:0/2567083929 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7eff2806af30 msgr2=0x7eff2806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:37.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.175+0000 7eff4e2f2640 1 --2- 172.21.15.154:0/2567083929 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7eff2806af30 0x7eff2806d3f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7eff38009780 tx=0x7eff38008040 comp rx=0 tx=0).stop 2023-09-23T03:16:37.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.175+0000 7eff4e2f2640 1 -- 172.21.15.154:0/2567083929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff4815abe0 msgr2=0x7eff48149bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:37.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.175+0000 7eff4e2f2640 1 --2- 172.21.15.154:0/2567083929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff4815abe0 0x7eff48149bd0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7eff4000c3c0 tx=0x7eff4000c880 comp rx=0 tx=0).stop 2023-09-23T03:16:37.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.175+0000 7eff4e2f2640 1 -- 172.21.15.154:0/2567083929 shutdown_connections 2023-09-23T03:16:37.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.175+0000 7eff4e2f2640 1 --2- 172.21.15.154:0/2567083929 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7eff2806af30 0x7eff2806d3f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:37.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.175+0000 7eff4e2f2640 1 --2- 172.21.15.154:0/2567083929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7eff4815abe0 0x7eff48149bd0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:37.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.175+0000 7eff4e2f2640 1 --2- 172.21.15.154:0/2567083929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7eff480cce10 0x7eff480c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:37.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.175+0000 7eff4e2f2640 1 --2- 172.21.15.154:0/2567083929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7eff480cc9b0 0x7eff480c7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:37.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.175+0000 7eff4e2f2640 1 -- 172.21.15.154:0/2567083929 >> 172.21.15.154:0/2567083929 conn(0x7eff4806fc40 msgr2=0x7eff4815d540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:37.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.175+0000 7eff4e2f2640 1 -- 172.21.15.154:0/2567083929 shutdown_connections 2023-09-23T03:16:37.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.175+0000 7eff4e2f2640 1 -- 172.21.15.154:0/2567083929 wait complete. 2023-09-23T03:16:37.195 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:16:37.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.418+0000 7f50f32e1640 1 Processor -- start 2023-09-23T03:16:37.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.418+0000 7f50f32e1640 1 -- start start 2023-09-23T03:16:37.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.419+0000 7f50f32e1640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f50ec0d0aa0 0x7f50ec15ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:37.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.419+0000 7f50f32e1640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50ec15f480 con 0x7f50ec15e5c0 2023-09-23T03:16:37.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.420+0000 7f50f32e1640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50ec15af00 con 0x7f50ec15ab20 2023-09-23T03:16:37.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.420+0000 7f50f32e1640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50ec15efb0 con 0x7f50ec0d0aa0 2023-09-23T03:16:37.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.420+0000 7f50f187e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f50ec0d0aa0 0x7f50ec15ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:37.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.420+0000 7f50f187e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f50ec0d0aa0 0x7f50ec15ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52584/0 (socket says 172.21.15.154:52584) 2023-09-23T03:16:37.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.420+0000 7f50f187e640 1 -- 172.21.15.154:0/246664689 learned_addr learned my addr 172.21.15.154:0/246664689 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:37.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.420+0000 7f50f187e640 1 -- 172.21.15.154:0/246664689 >> v1:172.21.15.190:6789/0 conn(0x7f50ec15ab20 legacy=0x7f50ec0cee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:37.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.420+0000 7f50f187e640 1 -- 172.21.15.154:0/246664689 >> v1:172.21.15.154:6789/0 conn(0x7f50ec15e5c0 legacy=0x7f50ec0d0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:37.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.420+0000 7f50f187e640 1 -- 172.21.15.154:0/246664689 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ec0cd3f0 con 0x7f50ec0d0aa0 2023-09-23T03:16:37.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.422+0000 7f50f187e640 1 --2- 172.21.15.154:0/246664689 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50ec0d0aa0 0x7f50ec15ea70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f50e4009870 tx=0x7f50e402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea6d1bb798bcac24 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:37.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.422+0000 7f50ebfff640 1 -- 172.21.15.154:0/246664689 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e4030860 con 0x7f50ec0d0aa0 2023-09-23T03:16:37.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.422+0000 7f50ebfff640 1 -- 172.21.15.154:0/246664689 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e40309c0 con 0x7f50ec0d0aa0 2023-09-23T03:16:37.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.422+0000 7f50ebfff640 1 -- 172.21.15.154:0/246664689 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e4030d50 con 0x7f50ec0d0aa0 2023-09-23T03:16:37.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.422+0000 7f50f32e1640 1 -- 172.21.15.154:0/246664689 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50ec0d0aa0 msgr2=0x7f50ec15ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:37.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.422+0000 7f50f32e1640 1 --2- 172.21.15.154:0/246664689 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50ec0d0aa0 0x7f50ec15ea70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f50e4009870 tx=0x7f50e402b9a0 comp rx=0 tx=0).stop 2023-09-23T03:16:37.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.423+0000 7f50f32e1640 1 -- 172.21.15.154:0/246664689 shutdown_connections 2023-09-23T03:16:37.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.423+0000 7f50f32e1640 1 --2- 172.21.15.154:0/246664689 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50ec0d0aa0 0x7f50ec15ea70 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:37.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.423+0000 7f50f32e1640 1 -- 172.21.15.154:0/246664689 >> 172.21.15.154:0/246664689 conn(0x7f50ec089fd0 msgr2=0x7f50ec08a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:37.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.423+0000 7f50f32e1640 1 -- 172.21.15.154:0/246664689 shutdown_connections 2023-09-23T03:16:37.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.423+0000 7f50f32e1640 1 -- 172.21.15.154:0/246664689 wait complete. 2023-09-23T03:16:37.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.424+0000 7f50f32e1640 1 Processor -- start 2023-09-23T03:16:37.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.424+0000 7f50f32e1640 1 -- start start 2023-09-23T03:16:37.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.425+0000 7f50f32e1640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50ec0d0aa0 0x7f50ec156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:37.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.425+0000 7f50f32e1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50ec15ab20 0x7f50ec15d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:37.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.425+0000 7f50f32e1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50ec15e5c0 0x7f50ec0c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:37.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.425+0000 7f50f32e1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec15cb70 con 0x7f50ec15ab20 2023-09-23T03:16:37.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.425+0000 7f50f32e1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec157220 con 0x7f50ec15e5c0 2023-09-23T03:16:37.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.425+0000 7f50f32e1640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec15bf60 con 0x7f50ec0d0aa0 2023-09-23T03:16:37.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.426+0000 7f50f087c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50ec15ab20 0x7f50ec15d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:37.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.426+0000 7f50f087c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50ec15ab20 0x7f50ec15d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52588/0 (socket says 172.21.15.154:52588) 2023-09-23T03:16:37.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.426+0000 7f50f087c640 1 -- 172.21.15.154:0/2636053706 learned_addr learned my addr 172.21.15.154:0/2636053706 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:37.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.426+0000 7f50f107d640 1 --2- 172.21.15.154:0/2636053706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50ec0d0aa0 0x7f50ec156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:37.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.426+0000 7f50f187e640 1 --2- 172.21.15.154:0/2636053706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50ec15e5c0 0x7f50ec0c60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:37.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.426+0000 7f50f087c640 1 -- 172.21.15.154:0/2636053706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50ec0d0aa0 msgr2=0x7f50ec156ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:37.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.426+0000 7f50f087c640 1 --2- 172.21.15.154:0/2636053706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50ec0d0aa0 0x7f50ec156ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:37.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.426+0000 7f50f087c640 1 -- 172.21.15.154:0/2636053706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50ec15e5c0 msgr2=0x7f50ec0c60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:37.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.426+0000 7f50f087c640 1 --2- 172.21.15.154:0/2636053706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50ec15e5c0 0x7f50ec0c60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:37.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.426+0000 7f50f087c640 1 -- 172.21.15.154:0/2636053706 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e40094d0 con 0x7f50ec15ab20 2023-09-23T03:16:37.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.426+0000 7f50f187e640 1 --2- 172.21.15.154:0/2636053706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50ec15e5c0 0x7f50ec0c60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:37.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.427+0000 7f50f087c640 1 --2- 172.21.15.154:0/2636053706 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50ec15ab20 0x7f50ec15d970 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f50e000c400 tx=0x7f50e000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:37.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.427+0000 7f50e9ffb640 1 -- 172.21.15.154:0/2636053706 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e00052d0 con 0x7f50ec15ab20 2023-09-23T03:16:37.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.427+0000 7f50e9ffb640 1 -- 172.21.15.154:0/2636053706 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e0005430 con 0x7f50ec15ab20 2023-09-23T03:16:37.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.427+0000 7f50e9ffb640 1 -- 172.21.15.154:0/2636053706 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e0003830 con 0x7f50ec15ab20 2023-09-23T03:16:37.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.427+0000 7f50f32e1640 1 -- 172.21.15.154:0/2636053706 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50ec15c240 con 0x7f50ec15ab20 2023-09-23T03:16:37.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.427+0000 7f50f32e1640 1 -- 172.21.15.154:0/2636053706 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50ec15c800 con 0x7f50ec15ab20 2023-09-23T03:16:37.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.428+0000 7f50f32e1640 1 -- 172.21.15.154:0/2636053706 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50ec15c9c0 con 0x7f50ec15ab20 2023-09-23T03:16:37.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.429+0000 7f50e9ffb640 1 -- 172.21.15.154:0/2636053706 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f50e0003990 con 0x7f50ec15ab20 2023-09-23T03:16:37.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.430+0000 7f50e9ffb640 1 --2- 172.21.15.154:0/2636053706 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f50d806af60 0x7f50d806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:37.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.431+0000 7f50e9ffb640 1 -- 172.21.15.154:0/2636053706 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f50e0089060 con 0x7f50ec15ab20 2023-09-23T03:16:37.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.431+0000 7f50f107d640 1 --2- 172.21.15.154:0/2636053706 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f50d806af60 0x7f50d806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:37.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.431+0000 7f50f107d640 1 --2- 172.21.15.154:0/2636053706 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f50d806af60 0x7f50d806d420 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f50ec0af830 tx=0x7f50dc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:37.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.435+0000 7f50e9ffb640 1 -- 172.21.15.154:0/2636053706 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f50e0053cf0 con 0x7f50ec15ab20 2023-09-23T03:16:37.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.626+0000 7f50f32e1640 1 -- 172.21.15.154:0/2636053706 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f50ec0ce320 con 0x7f50ec15ab20 2023-09-23T03:16:37.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.627+0000 7f50e9ffb640 1 -- 172.21.15.154:0/2636053706 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f50e0017020 con 0x7f50ec15ab20 2023-09-23T03:16:37.628 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:16:37.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.629+0000 7f50f32e1640 1 -- 172.21.15.154:0/2636053706 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f50d806af60 msgr2=0x7f50d806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:37.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.629+0000 7f50f32e1640 1 --2- 172.21.15.154:0/2636053706 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f50d806af60 0x7f50d806d420 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f50ec0af830 tx=0x7f50dc009040 comp rx=0 tx=0).stop 2023-09-23T03:16:37.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.630+0000 7f50f32e1640 1 -- 172.21.15.154:0/2636053706 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50ec15ab20 msgr2=0x7f50ec15d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:37.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.630+0000 7f50f32e1640 1 --2- 172.21.15.154:0/2636053706 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50ec15ab20 0x7f50ec15d970 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f50e000c400 tx=0x7f50e000c8c0 comp rx=0 tx=0).stop 2023-09-23T03:16:37.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.630+0000 7f50f32e1640 1 -- 172.21.15.154:0/2636053706 shutdown_connections 2023-09-23T03:16:37.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.630+0000 7f50f32e1640 1 --2- 172.21.15.154:0/2636053706 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f50d806af60 0x7f50d806d420 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:37.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.630+0000 7f50f32e1640 1 --2- 172.21.15.154:0/2636053706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50ec15e5c0 0x7f50ec0c60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:37.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.630+0000 7f50f32e1640 1 --2- 172.21.15.154:0/2636053706 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50ec15ab20 0x7f50ec15d970 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:37.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.630+0000 7f50f32e1640 1 --2- 172.21.15.154:0/2636053706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50ec0d0aa0 0x7f50ec156ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:37.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.630+0000 7f50f32e1640 1 -- 172.21.15.154:0/2636053706 >> 172.21.15.154:0/2636053706 conn(0x7f50ec089fd0 msgr2=0x7f50ec06e290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:37.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.630+0000 7f50f32e1640 1 -- 172.21.15.154:0/2636053706 shutdown_connections 2023-09-23T03:16:37.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.630+0000 7f50f32e1640 1 -- 172.21.15.154:0/2636053706 wait complete. 2023-09-23T03:16:37.649 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-23T03:16:37.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.864+0000 7f5bc827c640 1 Processor -- start 2023-09-23T03:16:37.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.865+0000 7f5bc827c640 1 -- start start 2023-09-23T03:16:37.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.866+0000 7f5bc827c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5bc0159980 0x7f5bc00cc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:37.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.866+0000 7f5bc827c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bc006ce90 0x7f5bc00cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:37.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.866+0000 7f5bc827c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bc015d2a0 con 0x7f5bc01595a0 2023-09-23T03:16:37.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.866+0000 7f5bc827c640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bc00cd3b0 con 0x7f5bc0159980 2023-09-23T03:16:37.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.866+0000 7f5bc827c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc00cb450 con 0x7f5bc006ce90 2023-09-23T03:16:37.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.866+0000 7f5bc6018640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f5bc01595a0 0x7f5bc006c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:42060/0 (socket says 172.21.15.154:42060) 2023-09-23T03:16:37.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.866+0000 7f5bc6018640 1 -- 172.21.15.154:0/4190143522 learned_addr learned my addr 172.21.15.154:0/4190143522 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:37.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.867+0000 7f5bc5016640 1 -- 172.21.15.154:0/4190143522 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1283140941 0 0) 0x7f5bc015d2a0 con 0x7f5bc01595a0 2023-09-23T03:16:37.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.867+0000 7f5bc6819640 1 --2- 172.21.15.154:0/4190143522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bc006ce90 0x7f5bc00cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:37.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.867+0000 7f5bc5016640 1 -- 172.21.15.154:0/4190143522 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ba8003520 con 0x7f5bc01595a0 2023-09-23T03:16:37.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.867+0000 7f5bc5817640 1 --2- 172.21.15.154:0/4190143522 >> v2:172.21.15.190:3300/0 conn(0x7f5bc0159980 0x7f5bc00cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:37.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.867+0000 7f5bc5016640 1 -- 172.21.15.154:0/4190143522 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 430519525 0 0) 0x7f5ba8003520 con 0x7f5bc01595a0 2023-09-23T03:16:37.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.867+0000 7f5bc5016640 1 -- 172.21.15.154:0/4190143522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bc006ce90 msgr2=0x7f5bc00cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:37.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.867+0000 7f5bc5016640 1 --2- 172.21.15.154:0/4190143522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bc006ce90 0x7f5bc00cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:37.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.867+0000 7f5bc5016640 1 -- 172.21.15.154:0/4190143522 >> v2:172.21.15.190:3300/0 conn(0x7f5bc0159980 msgr2=0x7f5bc00cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:37.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.867+0000 7f5bc5016640 1 --2- 172.21.15.154:0/4190143522 >> v2:172.21.15.190:3300/0 conn(0x7f5bc0159980 0x7f5bc00cc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:37.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.867+0000 7f5bc5016640 1 -- 172.21.15.154:0/4190143522 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc00cd660 con 0x7f5bc01595a0 2023-09-23T03:16:37.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.867+0000 7f5bc5016640 1 -- 172.21.15.154:0/4190143522 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f5bb00019f0 con 0x7f5bc01595a0 2023-09-23T03:16:37.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.867+0000 7f5bc5016640 1 -- 172.21.15.154:0/4190143522 >> v1:172.21.15.154:6789/0 conn(0x7f5bc01595a0 legacy=0x7f5bc006c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:37.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.867+0000 7f5bc5016640 1 --2- 172.21.15.154:0/4190143522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ba8003970 0x7f5ba8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:37.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.867+0000 7f5bc5016640 1 --2- 172.21.15.154:0/4190143522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ba80047c0 0x7f5ba8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:37.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.867+0000 7f5bc5016640 1 --2- 172.21.15.154:0/4190143522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bc006ce90 0x7f5ba8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:37.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.867+0000 7f5bc5016640 1 -- 172.21.15.154:0/4190143522 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc00cd3b0 con 0x7f5ba8003970 2023-09-23T03:16:37.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.867+0000 7f5bc5016640 1 -- 172.21.15.154:0/4190143522 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc00cb450 con 0x7f5ba80047c0 2023-09-23T03:16:37.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.867+0000 7f5bc5016640 1 -- 172.21.15.154:0/4190143522 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba800dc90 con 0x7f5bc006ce90 2023-09-23T03:16:37.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.867+0000 7f5bc6018640 1 --2- 172.21.15.154:0/4190143522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ba8003970 0x7f5ba8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:37.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.868+0000 7f5bc5817640 1 --2- 172.21.15.154:0/4190143522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ba80047c0 0x7f5ba8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:37.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.868+0000 7f5bc6018640 1 -- 172.21.15.154:0/4190143522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bc006ce90 msgr2=0x7f5ba8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:37.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.868+0000 7f5bc6018640 1 --2- 172.21.15.154:0/4190143522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bc006ce90 0x7f5ba8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:37.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.868+0000 7f5bc6018640 1 -- 172.21.15.154:0/4190143522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ba80047c0 msgr2=0x7f5ba8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:37.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.868+0000 7f5bc6018640 1 --2- 172.21.15.154:0/4190143522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ba80047c0 0x7f5ba8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:37.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.868+0000 7f5bc6018640 1 -- 172.21.15.154:0/4190143522 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba8003520 con 0x7f5ba8003970 2023-09-23T03:16:37.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.868+0000 7f5bc5817640 1 --2- 172.21.15.154:0/4190143522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ba80047c0 0x7f5ba8004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:37.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.869+0000 7f5bc6018640 1 --2- 172.21.15.154:0/4190143522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ba8003970 0x7f5ba8003d90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0002230 tx=0x7f5ba80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4814ea4d8a5c9d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:37.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.869+0000 7f5bc5016640 1 -- 172.21.15.154:0/4190143522 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0034630 con 0x7f5ba8003970 2023-09-23T03:16:37.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.869+0000 7f5bc5016640 1 -- 172.21.15.154:0/4190143522 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb002f070 con 0x7f5ba8003970 2023-09-23T03:16:37.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.869+0000 7f5bc5016640 1 -- 172.21.15.154:0/4190143522 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb00349b0 con 0x7f5ba8003970 2023-09-23T03:16:37.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.870+0000 7f5bc827c640 1 -- 172.21.15.154:0/4190143522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ba8003970 msgr2=0x7f5ba8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:37.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.870+0000 7f5bc827c640 1 --2- 172.21.15.154:0/4190143522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ba8003970 0x7f5ba8003d90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0002230 tx=0x7f5ba80057f0 comp rx=0 tx=0).stop 2023-09-23T03:16:37.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.870+0000 7f5bc6819640 1 -- 172.21.15.154:0/4190143522 reap_dead start 2023-09-23T03:16:37.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.870+0000 7f5bc827c640 1 -- 172.21.15.154:0/4190143522 shutdown_connections 2023-09-23T03:16:37.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.870+0000 7f5bc827c640 1 -- 172.21.15.154:0/4190143522 >> 172.21.15.154:0/4190143522 conn(0x7f5bc00ced60 msgr2=0x7f5bc00cf170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:37.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.870+0000 7f5bc827c640 1 -- 172.21.15.154:0/4190143522 shutdown_connections 2023-09-23T03:16:37.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.870+0000 7f5bc827c640 1 -- 172.21.15.154:0/4190143522 wait complete. 2023-09-23T03:16:37.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.872+0000 7f5bc827c640 1 Processor -- start 2023-09-23T03:16:37.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.872+0000 7f5bc827c640 1 -- start start 2023-09-23T03:16:37.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.873+0000 7f5bc827c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ba8003970 0x7f5bc00b3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:37.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.873+0000 7f5bc827c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bc00afb10 0x7f5bc00c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:37.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.873+0000 7f5bc827c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bc00aff70 0x7f5bc00c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:37.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.873+0000 7f5bc827c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc00c8dd0 con 0x7f5bc00aff70 2023-09-23T03:16:37.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.873+0000 7f5bc827c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc00b3cc0 con 0x7f5ba8003970 2023-09-23T03:16:37.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.873+0000 7f5bc827c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc00c8af0 con 0x7f5bc00afb10 2023-09-23T03:16:37.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.873+0000 7f5bc6819640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bc00aff70 0x7f5bc00c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:37.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.873+0000 7f5bc6819640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bc00aff70 0x7f5bc00c83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52598/0 (socket says 172.21.15.154:52598) 2023-09-23T03:16:37.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.873+0000 7f5bc6819640 1 -- 172.21.15.154:0/1969194045 learned_addr learned my addr 172.21.15.154:0/1969194045 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:37.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.873+0000 7f5bc5817640 1 --2- 172.21.15.154:0/1969194045 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bc00afb10 0x7f5bc00c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:37.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.873+0000 7f5bc6819640 1 -- 172.21.15.154:0/1969194045 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bc00afb10 msgr2=0x7f5bc00c7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:37.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.873+0000 7f5bc6819640 1 --2- 172.21.15.154:0/1969194045 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bc00afb10 0x7f5bc00c7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:37.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.873+0000 7f5bc6819640 1 -- 172.21.15.154:0/1969194045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ba8003970 msgr2=0x7f5bc00b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:37.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.873+0000 7f5bc6819640 1 --2- 172.21.15.154:0/1969194045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ba8003970 0x7f5bc00b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:37.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.874+0000 7f5bc6819640 1 -- 172.21.15.154:0/1969194045 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc00b2ca0 con 0x7f5bc00aff70 2023-09-23T03:16:37.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.874+0000 7f5bc6819640 1 --2- 172.21.15.154:0/1969194045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bc00aff70 0x7f5bc00c83e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5bc00c92e0 tx=0x7f5bb800e9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:37.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.874+0000 7f5bbeffd640 1 -- 172.21.15.154:0/1969194045 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8013680 con 0x7f5bc00aff70 2023-09-23T03:16:37.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.874+0000 7f5bbeffd640 1 -- 172.21.15.154:0/1969194045 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb80137e0 con 0x7f5bc00aff70 2023-09-23T03:16:37.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.874+0000 7f5bbeffd640 1 -- 172.21.15.154:0/1969194045 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8013b90 con 0x7f5bc00aff70 2023-09-23T03:16:37.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.874+0000 7f5bc827c640 1 -- 172.21.15.154:0/1969194045 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bc00b2f80 con 0x7f5bc00aff70 2023-09-23T03:16:37.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.874+0000 7f5bc827c640 1 -- 172.21.15.154:0/1969194045 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bc01ee0d0 con 0x7f5bc00aff70 2023-09-23T03:16:37.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.875+0000 7f5bc827c640 1 -- 172.21.15.154:0/1969194045 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bc00b34a0 con 0x7f5bc00aff70 2023-09-23T03:16:37.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.876+0000 7f5bbeffd640 1 -- 172.21.15.154:0/1969194045 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f5bb8013cf0 con 0x7f5bc00aff70 2023-09-23T03:16:37.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.877+0000 7f5bbeffd640 1 --2- 172.21.15.154:0/1969194045 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5bac06af60 0x7f5bac06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:37.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.877+0000 7f5bbeffd640 1 -- 172.21.15.154:0/1969194045 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f5bb8089370 con 0x7f5bc00aff70 2023-09-23T03:16:37.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.877+0000 7f5bc6018640 1 --2- 172.21.15.154:0/1969194045 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5bac06af60 0x7f5bac06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:37.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.878+0000 7f5bc6018640 1 --2- 172.21.15.154:0/1969194045 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5bac06af60 0x7f5bac06d420 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0034000 tx=0x7f5bb0007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:37.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:37.881+0000 7f5bbeffd640 1 -- 172.21.15.154:0/1969194045 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f5bc00b34a0 con 0x7f5bc00aff70 2023-09-23T03:16:38.075 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.074+0000 7f5bc827c640 1 -- 172.21.15.154:0/1969194045 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5bc00b34a0 con 0x7f5bc00aff70 2023-09-23T03:16:38.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.075+0000 7f5bbeffd640 1 -- 172.21.15.154:0/1969194045 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5bc00b34a0 con 0x7f5bc00aff70 2023-09-23T03:16:38.076 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:16:38.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.077+0000 7f5bc827c640 1 -- 172.21.15.154:0/1969194045 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5bac06af60 msgr2=0x7f5bac06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:38.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.077+0000 7f5bc827c640 1 --2- 172.21.15.154:0/1969194045 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5bac06af60 0x7f5bac06d420 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0034000 tx=0x7f5bb0007000 comp rx=0 tx=0).stop 2023-09-23T03:16:38.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.077+0000 7f5bc827c640 1 -- 172.21.15.154:0/1969194045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bc00aff70 msgr2=0x7f5bc00c83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:38.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.077+0000 7f5bc827c640 1 --2- 172.21.15.154:0/1969194045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bc00aff70 0x7f5bc00c83e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5bc00c92e0 tx=0x7f5bb800e9f0 comp rx=0 tx=0).stop 2023-09-23T03:16:38.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.077+0000 7f5bc827c640 1 -- 172.21.15.154:0/1969194045 shutdown_connections 2023-09-23T03:16:38.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.077+0000 7f5bc827c640 1 --2- 172.21.15.154:0/1969194045 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5bac06af60 0x7f5bac06d420 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:38.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.077+0000 7f5bc827c640 1 --2- 172.21.15.154:0/1969194045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5bc00aff70 0x7f5bc00c83e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:38.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.077+0000 7f5bc827c640 1 --2- 172.21.15.154:0/1969194045 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5bc00afb10 0x7f5bc00c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:38.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.077+0000 7f5bc827c640 1 --2- 172.21.15.154:0/1969194045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ba8003970 0x7f5bc00b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:38.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.077+0000 7f5bc827c640 1 -- 172.21.15.154:0/1969194045 >> 172.21.15.154:0/1969194045 conn(0x7f5bc00ced60 msgr2=0x7f5bc006ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:38.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.078+0000 7f5bc827c640 1 -- 172.21.15.154:0/1969194045 shutdown_connections 2023-09-23T03:16:38.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.078+0000 7f5bc827c640 1 -- 172.21.15.154:0/1969194045 wait complete. 2023-09-23T03:16:38.098 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-23T03:16:38.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.314+0000 7fcd51c44640 1 Processor -- start 2023-09-23T03:16:38.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.315+0000 7fcd51c44640 1 -- start start 2023-09-23T03:16:38.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.316+0000 7fcd51c44640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd4c0acce0 0x7fcd4c15ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:38.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.316+0000 7fcd51c44640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd4c15ac30 con 0x7fcd4c0cf040 2023-09-23T03:16:38.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.316+0000 7fcd51c44640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd4c0b1f70 con 0x7fcd4c0b1b20 2023-09-23T03:16:38.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.316+0000 7fcd51c44640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4c0b2e90 con 0x7fcd4c0acce0 2023-09-23T03:16:38.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.316+0000 7fcd4bfff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fcd4c0cf040 0x7fcd4c0ab140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:42064/0 (socket says 172.21.15.154:42064) 2023-09-23T03:16:38.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.316+0000 7fcd4bfff640 1 -- 172.21.15.154:0/559798659 learned_addr learned my addr 172.21.15.154:0/559798659 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:38.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.317+0000 7fcd4b7fe640 1 --2- 172.21.15.154:0/559798659 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd4c0acce0 0x7fcd4c15ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:38.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.317+0000 7fcd4a7fc640 1 -- 172.21.15.154:0/559798659 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2335196779 0 0) 0x7fcd44001110 con 0x7fcd4c0cf040 2023-09-23T03:16:38.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.317+0000 7fcd4a7fc640 1 -- 172.21.15.154:0/559798659 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd38003520 con 0x7fcd4c0cf040 2023-09-23T03:16:38.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.317+0000 7fcd4a7fc640 1 -- 172.21.15.154:0/559798659 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 626188933 0 0) 0x7fcd38003520 con 0x7fcd4c0cf040 2023-09-23T03:16:38.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.317+0000 7fcd4a7fc640 1 -- 172.21.15.154:0/559798659 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd4c0acce0 msgr2=0x7fcd4c15ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:38.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.317+0000 7fcd4a7fc640 1 --2- 172.21.15.154:0/559798659 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd4c0acce0 0x7fcd4c15ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:38.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.317+0000 7fcd4a7fc640 1 -- 172.21.15.154:0/559798659 >> v1:172.21.15.190:6789/0 conn(0x7fcd4c0b1b20 legacy=0x7fcd4c0ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:38.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.317+0000 7fcd4a7fc640 1 -- 172.21.15.154:0/559798659 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd4c15f2a0 con 0x7fcd4c0cf040 2023-09-23T03:16:38.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.317+0000 7fcd4b7fe640 1 --2- 172.21.15.154:0/559798659 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd4c0acce0 0x7fcd4c15ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:16:38.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.317+0000 7fcd4a7fc640 1 -- 172.21.15.154:0/559798659 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fcd44001800 con 0x7fcd4c0cf040 2023-09-23T03:16:38.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.317+0000 7fcd4a7fc640 1 -- 172.21.15.154:0/559798659 >> v1:172.21.15.154:6789/0 conn(0x7fcd4c0cf040 legacy=0x7fcd4c0ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:38.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.317+0000 7fcd4a7fc640 1 --2- 172.21.15.154:0/559798659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd380039a0 0x7fcd38003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:38.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.317+0000 7fcd4a7fc640 1 --2- 172.21.15.154:0/559798659 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd4c0acce0 0x7fcd380047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:38.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.317+0000 7fcd4a7fc640 1 --2- 172.21.15.154:0/559798659 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd38004d00 0x7fcd38005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:38.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.317+0000 7fcd4a7fc640 1 -- 172.21.15.154:0/559798659 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4c0b2e90 con 0x7fcd380039a0 2023-09-23T03:16:38.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.317+0000 7fcd4a7fc640 1 -- 172.21.15.154:0/559798659 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3800bb10 con 0x7fcd38004d00 2023-09-23T03:16:38.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.317+0000 7fcd4a7fc640 1 -- 172.21.15.154:0/559798659 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3800bcd0 con 0x7fcd4c0acce0 2023-09-23T03:16:38.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.318+0000 7fcd4b7fe640 1 --2- 172.21.15.154:0/559798659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd380039a0 0x7fcd38003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:38.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.318+0000 7fcd4b7fe640 1 -- 172.21.15.154:0/559798659 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd4c0acce0 msgr2=0x7fcd380047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:38.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.318+0000 7fcd4b7fe640 1 --2- 172.21.15.154:0/559798659 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd4c0acce0 0x7fcd380047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:38.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.318+0000 7fcd4b7fe640 1 -- 172.21.15.154:0/559798659 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd38004d00 msgr2=0x7fcd38005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:38.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.318+0000 7fcd4b7fe640 1 --2- 172.21.15.154:0/559798659 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd38004d00 0x7fcd38005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:38.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.318+0000 7fcd4b7fe640 1 -- 172.21.15.154:0/559798659 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd38003520 con 0x7fcd380039a0 2023-09-23T03:16:38.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.319+0000 7fcd4b7fe640 1 --2- 172.21.15.154:0/559798659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd380039a0 0x7fcd38003dc0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c007430 tx=0x7fcd380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b52c5959908d6766 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:38.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.319+0000 7fcd4a7fc640 1 -- 172.21.15.154:0/559798659 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c02cdf0 con 0x7fcd380039a0 2023-09-23T03:16:38.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.319+0000 7fcd4a7fc640 1 -- 172.21.15.154:0/559798659 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd3c034440 con 0x7fcd380039a0 2023-09-23T03:16:38.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.319+0000 7fcd4a7fc640 1 -- 172.21.15.154:0/559798659 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c0347b0 con 0x7fcd380039a0 2023-09-23T03:16:38.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.320+0000 7fcd51c44640 1 -- 172.21.15.154:0/559798659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd380039a0 msgr2=0x7fcd38003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:38.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.320+0000 7fcd51c44640 1 --2- 172.21.15.154:0/559798659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd380039a0 0x7fcd38003dc0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c007430 tx=0x7fcd380057f0 comp rx=0 tx=0).stop 2023-09-23T03:16:38.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.320+0000 7fcd4bfff640 1 -- 172.21.15.154:0/559798659 reap_dead start 2023-09-23T03:16:38.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.320+0000 7fcd51c44640 1 -- 172.21.15.154:0/559798659 shutdown_connections 2023-09-23T03:16:38.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.320+0000 7fcd51c44640 1 -- 172.21.15.154:0/559798659 >> 172.21.15.154:0/559798659 conn(0x7fcd4c06a0f0 msgr2=0x7fcd4c06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:38.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.320+0000 7fcd51c44640 1 -- 172.21.15.154:0/559798659 shutdown_connections 2023-09-23T03:16:38.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.320+0000 7fcd51c44640 1 -- 172.21.15.154:0/559798659 wait complete. 2023-09-23T03:16:38.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.322+0000 7fcd51c44640 1 Processor -- start 2023-09-23T03:16:38.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.322+0000 7fcd51c44640 1 -- start start 2023-09-23T03:16:38.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.323+0000 7fcd51c44640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd380039a0 0x7fcd4c09bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:38.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.323+0000 7fcd51c44640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd4c15d450 0x7fcd4c09a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:38.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.323+0000 7fcd51c44640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd4c14cb30 0x7fcd4c14cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:38.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.323+0000 7fcd51c44640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4c09c2c0 con 0x7fcd4c14cb30 2023-09-23T03:16:38.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.323+0000 7fcd51c44640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4c09ace0 con 0x7fcd4c15d450 2023-09-23T03:16:38.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.323+0000 7fcd51c44640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4c09afc0 con 0x7fcd380039a0 2023-09-23T03:16:38.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.323+0000 7fcd4bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd4c14cb30 0x7fcd4c14cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:38.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.323+0000 7fcd4bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd4c14cb30 0x7fcd4c14cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52612/0 (socket says 172.21.15.154:52612) 2023-09-23T03:16:38.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.323+0000 7fcd4bfff640 1 -- 172.21.15.154:0/2786563279 learned_addr learned my addr 172.21.15.154:0/2786563279 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:38.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.324+0000 7fcd4affd640 1 --2- 172.21.15.154:0/2786563279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd4c15d450 0x7fcd4c09a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:38.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.324+0000 7fcd4bfff640 1 -- 172.21.15.154:0/2786563279 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd380039a0 msgr2=0x7fcd4c09bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:38.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.324+0000 7fcd4b7fe640 1 --2- 172.21.15.154:0/2786563279 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd380039a0 0x7fcd4c09bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:38.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.324+0000 7fcd4bfff640 1 --2- 172.21.15.154:0/2786563279 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd380039a0 0x7fcd4c09bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:38.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.324+0000 7fcd4bfff640 1 -- 172.21.15.154:0/2786563279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd4c15d450 msgr2=0x7fcd4c09a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:38.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.324+0000 7fcd4bfff640 1 --2- 172.21.15.154:0/2786563279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd4c15d450 0x7fcd4c09a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:38.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.324+0000 7fcd4bfff640 1 -- 172.21.15.154:0/2786563279 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd4c09b2a0 con 0x7fcd4c14cb30 2023-09-23T03:16:38.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.324+0000 7fcd4affd640 1 --2- 172.21.15.154:0/2786563279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd4c15d450 0x7fcd4c09a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:38.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.324+0000 7fcd4bfff640 1 --2- 172.21.15.154:0/2786563279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd4c14cb30 0x7fcd4c14cfe0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c0c8e40 tx=0x7fcd4400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:38.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.325+0000 7fcd43fff640 1 -- 172.21.15.154:0/2786563279 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd440075a0 con 0x7fcd4c14cb30 2023-09-23T03:16:38.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.325+0000 7fcd43fff640 1 -- 172.21.15.154:0/2786563279 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd440047e0 con 0x7fcd4c14cb30 2023-09-23T03:16:38.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.325+0000 7fcd43fff640 1 -- 172.21.15.154:0/2786563279 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd44004b90 con 0x7fcd4c14cb30 2023-09-23T03:16:38.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.325+0000 7fcd51c44640 1 -- 172.21.15.154:0/2786563279 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd4c09b580 con 0x7fcd4c14cb30 2023-09-23T03:16:38.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.325+0000 7fcd51c44640 1 -- 172.21.15.154:0/2786563279 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd4c09a9f0 con 0x7fcd4c14cb30 2023-09-23T03:16:38.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.326+0000 7fcd51c44640 1 -- 172.21.15.154:0/2786563279 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd4c09baa0 con 0x7fcd4c14cb30 2023-09-23T03:16:38.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.327+0000 7fcd43fff640 1 -- 172.21.15.154:0/2786563279 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fcd44004cf0 con 0x7fcd4c14cb30 2023-09-23T03:16:38.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.328+0000 7fcd43fff640 1 --2- 172.21.15.154:0/2786563279 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcd2c06f370 0x7fcd2c071830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:38.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.328+0000 7fcd43fff640 1 -- 172.21.15.154:0/2786563279 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7fcd44088b80 con 0x7fcd4c14cb30 2023-09-23T03:16:38.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.328+0000 7fcd4b7fe640 1 --2- 172.21.15.154:0/2786563279 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcd2c06f370 0x7fcd2c071830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:38.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.329+0000 7fcd4b7fe640 1 --2- 172.21.15.154:0/2786563279 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcd2c06f370 0x7fcd2c071830 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c036000 tx=0x7fcd3c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:38.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.332+0000 7fcd43fff640 1 -- 172.21.15.154:0/2786563279 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fcd4c09baa0 con 0x7fcd4c14cb30 2023-09-23T03:16:38.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.534+0000 7fcd51c44640 1 -- 172.21.15.154:0/2786563279 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcd4c15ac30 con 0x7fcd4c14cb30 2023-09-23T03:16:38.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.535+0000 7fcd43fff640 1 -- 172.21.15.154:0/2786563279 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcd4c15ac30 con 0x7fcd4c14cb30 2023-09-23T03:16:38.536 INFO:teuthology.orchestra.run.smithi154.stderr:listed 0 entries 2023-09-23T03:16:38.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.538+0000 7fcd51c44640 1 -- 172.21.15.154:0/2786563279 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcd2c06f370 msgr2=0x7fcd2c071830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:38.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.538+0000 7fcd51c44640 1 --2- 172.21.15.154:0/2786563279 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcd2c06f370 0x7fcd2c071830 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c036000 tx=0x7fcd3c030000 comp rx=0 tx=0).stop 2023-09-23T03:16:38.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.538+0000 7fcd51c44640 1 -- 172.21.15.154:0/2786563279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd4c14cb30 msgr2=0x7fcd4c14cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:38.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.538+0000 7fcd51c44640 1 --2- 172.21.15.154:0/2786563279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd4c14cb30 0x7fcd4c14cfe0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c0c8e40 tx=0x7fcd4400c960 comp rx=0 tx=0).stop 2023-09-23T03:16:38.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.538+0000 7fcd51c44640 1 -- 172.21.15.154:0/2786563279 shutdown_connections 2023-09-23T03:16:38.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.538+0000 7fcd51c44640 1 --2- 172.21.15.154:0/2786563279 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcd2c06f370 0x7fcd2c071830 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:38.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.538+0000 7fcd51c44640 1 --2- 172.21.15.154:0/2786563279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd4c14cb30 0x7fcd4c14cfe0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:38.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.538+0000 7fcd51c44640 1 --2- 172.21.15.154:0/2786563279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd4c15d450 0x7fcd4c09a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:38.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.538+0000 7fcd51c44640 1 --2- 172.21.15.154:0/2786563279 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd380039a0 0x7fcd4c09bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:38.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.538+0000 7fcd51c44640 1 -- 172.21.15.154:0/2786563279 >> 172.21.15.154:0/2786563279 conn(0x7fcd4c06a0f0 msgr2=0x7fcd4c0af5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:38.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.538+0000 7fcd51c44640 1 -- 172.21.15.154:0/2786563279 shutdown_connections 2023-09-23T03:16:38.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.538+0000 7fcd51c44640 1 -- 172.21.15.154:0/2786563279 wait complete. 2023-09-23T03:16:38.557 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:16:38.557 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:16:38.617 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:16:38.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.837+0000 7fb076607640 1 Processor -- start 2023-09-23T03:16:38.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.838+0000 7fb076607640 1 -- start start 2023-09-23T03:16:38.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.839+0000 7fb076607640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb0700acce0 0x7fb07015ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:38.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.839+0000 7fb076607640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0700cf040 0x7fb0700ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:38.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.839+0000 7fb076607640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb07015ac30 con 0x7fb0700b1b20 2023-09-23T03:16:38.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.839+0000 7fb076607640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0700b2e90 con 0x7fb0700acce0 2023-09-23T03:16:38.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.839+0000 7fb076607640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0700b1f70 con 0x7fb0700cf040 2023-09-23T03:16:38.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.839+0000 7fb067fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fb0700b1b20 0x7fb0700ce930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:42080/0 (socket says 172.21.15.154:42080) 2023-09-23T03:16:38.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.839+0000 7fb067fff640 1 -- 172.21.15.154:0/1381261249 learned_addr learned my addr 172.21.15.154:0/1381261249 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:38.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.840+0000 7fb06ffff640 1 --2- 172.21.15.154:0/1381261249 >> v2:172.21.15.190:3300/0 conn(0x7fb0700acce0 0x7fb07015ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:38.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.840+0000 7fb06f7fe640 1 -- 172.21.15.154:0/1381261249 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2242887179 0 0) 0x7fb058001110 con 0x7fb0700b1b20 2023-09-23T03:16:38.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.840+0000 7fb06f7fe640 1 -- 172.21.15.154:0/1381261249 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb05c003520 con 0x7fb0700b1b20 2023-09-23T03:16:38.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.840+0000 7fb074ba4640 1 --2- 172.21.15.154:0/1381261249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0700cf040 0x7fb0700ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:38.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.840+0000 7fb06f7fe640 1 -- 172.21.15.154:0/1381261249 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 173896169 0 0) 0x7fb05c003520 con 0x7fb0700b1b20 2023-09-23T03:16:38.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.840+0000 7fb06f7fe640 1 -- 172.21.15.154:0/1381261249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0700cf040 msgr2=0x7fb0700ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:38.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.840+0000 7fb06f7fe640 1 --2- 172.21.15.154:0/1381261249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0700cf040 0x7fb0700ab140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:38.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.840+0000 7fb06f7fe640 1 -- 172.21.15.154:0/1381261249 >> v2:172.21.15.190:3300/0 conn(0x7fb0700acce0 msgr2=0x7fb07015ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:38.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.840+0000 7fb06f7fe640 1 --2- 172.21.15.154:0/1381261249 >> v2:172.21.15.190:3300/0 conn(0x7fb0700acce0 0x7fb07015ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:38.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.840+0000 7fb06f7fe640 1 -- 172.21.15.154:0/1381261249 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb07015f2a0 con 0x7fb0700b1b20 2023-09-23T03:16:38.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.840+0000 7fb06f7fe640 1 -- 172.21.15.154:0/1381261249 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fb058001800 con 0x7fb0700b1b20 2023-09-23T03:16:38.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.840+0000 7fb06f7fe640 1 -- 172.21.15.154:0/1381261249 >> v1:172.21.15.154:6789/0 conn(0x7fb0700b1b20 legacy=0x7fb0700ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:38.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.840+0000 7fb06f7fe640 1 --2- 172.21.15.154:0/1381261249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0700cf040 0x7fb05c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:38.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.840+0000 7fb06f7fe640 1 --2- 172.21.15.154:0/1381261249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb05c003ef0 0x7fb05c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:38.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.840+0000 7fb06f7fe640 1 --2- 172.21.15.154:0/1381261249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb05c004d00 0x7fb05c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:38.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.840+0000 7fb06f7fe640 1 -- 172.21.15.154:0/1381261249 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0700b2e90 con 0x7fb05c004d00 2023-09-23T03:16:38.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.840+0000 7fb06f7fe640 1 -- 172.21.15.154:0/1381261249 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0700b1f70 con 0x7fb05c003ef0 2023-09-23T03:16:38.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.840+0000 7fb06f7fe640 1 -- 172.21.15.154:0/1381261249 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb05c00ba80 con 0x7fb0700cf040 2023-09-23T03:16:38.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.841+0000 7fb074ba4640 1 --2- 172.21.15.154:0/1381261249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb05c004d00 0x7fb05c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:38.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.841+0000 7fb067fff640 1 --2- 172.21.15.154:0/1381261249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb05c003ef0 0x7fb05c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:38.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.841+0000 7fb074ba4640 1 -- 172.21.15.154:0/1381261249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0700cf040 msgr2=0x7fb05c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:38.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.841+0000 7fb06ffff640 1 --2- 172.21.15.154:0/1381261249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0700cf040 0x7fb05c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:38.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.841+0000 7fb074ba4640 1 --2- 172.21.15.154:0/1381261249 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0700cf040 0x7fb05c003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:38.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.841+0000 7fb074ba4640 1 -- 172.21.15.154:0/1381261249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb05c003ef0 msgr2=0x7fb05c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:38.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.841+0000 7fb074ba4640 1 --2- 172.21.15.154:0/1381261249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb05c003ef0 0x7fb05c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:38.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.841+0000 7fb074ba4640 1 -- 172.21.15.154:0/1381261249 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb05c003520 con 0x7fb05c004d00 2023-09-23T03:16:38.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.841+0000 7fb067fff640 1 --2- 172.21.15.154:0/1381261249 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb05c003ef0 0x7fb05c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:38.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.842+0000 7fb074ba4640 1 --2- 172.21.15.154:0/1381261249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb05c004d00 0x7fb05c005180 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb068007390 tx=0x7fb05c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9f4400afd9af85 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:38.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.842+0000 7fb06f7fe640 1 -- 172.21.15.154:0/1381261249 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06802ccb0 con 0x7fb05c004d00 2023-09-23T03:16:38.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.842+0000 7fb06f7fe640 1 -- 172.21.15.154:0/1381261249 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb06802ce10 con 0x7fb05c004d00 2023-09-23T03:16:38.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.842+0000 7fb06f7fe640 1 -- 172.21.15.154:0/1381261249 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb068034660 con 0x7fb05c004d00 2023-09-23T03:16:38.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.843+0000 7fb076607640 1 -- 172.21.15.154:0/1381261249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb05c004d00 msgr2=0x7fb05c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:38.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.843+0000 7fb076607640 1 --2- 172.21.15.154:0/1381261249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb05c004d00 0x7fb05c005180 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb068007390 tx=0x7fb05c0057f0 comp rx=0 tx=0).stop 2023-09-23T03:16:38.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.843+0000 7fb074ba4640 1 -- 172.21.15.154:0/1381261249 reap_dead start 2023-09-23T03:16:38.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.843+0000 7fb076607640 1 -- 172.21.15.154:0/1381261249 shutdown_connections 2023-09-23T03:16:38.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.843+0000 7fb076607640 1 -- 172.21.15.154:0/1381261249 >> 172.21.15.154:0/1381261249 conn(0x7fb07006a0f0 msgr2=0x7fb07006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:38.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.843+0000 7fb076607640 1 -- 172.21.15.154:0/1381261249 shutdown_connections 2023-09-23T03:16:38.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.843+0000 7fb076607640 1 -- 172.21.15.154:0/1381261249 wait complete. 2023-09-23T03:16:38.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.845+0000 7fb076607640 1 Processor -- start 2023-09-23T03:16:38.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.845+0000 7fb076607640 1 -- start start 2023-09-23T03:16:38.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.846+0000 7fb076607640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb05c004d00 0x7fb0700aea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:38.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.846+0000 7fb076607640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb070158760 0x7fb0700c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:38.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.846+0000 7fb076607640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0700c9ff0 0x7fb0700ca470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:38.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.846+0000 7fb076607640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0700ca9b0 con 0x7fb070158760 2023-09-23T03:16:38.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.846+0000 7fb076607640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0700c6aa0 con 0x7fb05c004d00 2023-09-23T03:16:38.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.846+0000 7fb076607640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0700c6f30 con 0x7fb0700c9ff0 2023-09-23T03:16:38.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.847+0000 7fb067fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb070158760 0x7fb0700c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:38.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.847+0000 7fb067fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb070158760 0x7fb0700c4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52642/0 (socket says 172.21.15.154:52642) 2023-09-23T03:16:38.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.847+0000 7fb067fff640 1 -- 172.21.15.154:0/3858860109 learned_addr learned my addr 172.21.15.154:0/3858860109 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:38.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.847+0000 7fb06ffff640 1 --2- 172.21.15.154:0/3858860109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb05c004d00 0x7fb0700aea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:38.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.847+0000 7fb074ba4640 1 --2- 172.21.15.154:0/3858860109 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0700c9ff0 0x7fb0700ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:38.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.847+0000 7fb067fff640 1 -- 172.21.15.154:0/3858860109 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0700c9ff0 msgr2=0x7fb0700ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:38.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.847+0000 7fb067fff640 1 --2- 172.21.15.154:0/3858860109 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0700c9ff0 0x7fb0700ca470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:38.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.847+0000 7fb067fff640 1 -- 172.21.15.154:0/3858860109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb05c004d00 msgr2=0x7fb0700aea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:38.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.847+0000 7fb067fff640 1 --2- 172.21.15.154:0/3858860109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb05c004d00 0x7fb0700aea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:38.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.847+0000 7fb067fff640 1 -- 172.21.15.154:0/3858860109 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0700af020 con 0x7fb070158760 2023-09-23T03:16:38.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.847+0000 7fb067fff640 1 --2- 172.21.15.154:0/3858860109 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb070158760 0x7fb0700c4a00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb05800dae0 tx=0x7fb058007e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:38.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.848+0000 7fb06d7fa640 1 -- 172.21.15.154:0/3858860109 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb058007560 con 0x7fb070158760 2023-09-23T03:16:38.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.848+0000 7fb06d7fa640 1 -- 172.21.15.154:0/3858860109 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0580076c0 con 0x7fb070158760 2023-09-23T03:16:38.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.848+0000 7fb06d7fa640 1 -- 172.21.15.154:0/3858860109 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb058004a00 con 0x7fb070158760 2023-09-23T03:16:38.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.848+0000 7fb076607640 1 -- 172.21.15.154:0/3858860109 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0700c6450 con 0x7fb070158760 2023-09-23T03:16:38.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.848+0000 7fb076607640 1 -- 172.21.15.154:0/3858860109 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0700c4f40 con 0x7fb070158760 2023-09-23T03:16:38.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.849+0000 7fb076607640 1 -- 172.21.15.154:0/3858860109 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb07006e940 con 0x7fb070158760 2023-09-23T03:16:38.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.850+0000 7fb06d7fa640 1 -- 172.21.15.154:0/3858860109 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fb058004b60 con 0x7fb070158760 2023-09-23T03:16:38.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.851+0000 7fb06d7fa640 1 --2- 172.21.15.154:0/3858860109 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb05006af30 0x7fb05006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:38.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.851+0000 7fb06d7fa640 1 -- 172.21.15.154:0/3858860109 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7fb058089430 con 0x7fb070158760 2023-09-23T03:16:38.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.851+0000 7fb06ffff640 1 --2- 172.21.15.154:0/3858860109 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb05006af30 0x7fb05006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:38.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.852+0000 7fb06ffff640 1 --2- 172.21.15.154:0/3858860109 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb05006af30 0x7fb05006d3f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb060009a80 tx=0x7fb060008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:38.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:38.855+0000 7fb06d7fa640 1 -- 172.21.15.154:0/3858860109 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fb058057b40 con 0x7fb070158760 2023-09-23T03:16:39.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.047+0000 7fb076607640 1 -- 172.21.15.154:0/3858860109 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb07015ac30 con 0x7fb070158760 2023-09-23T03:16:39.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.050+0000 7fb06d7fa640 1 -- 172.21.15.154:0/3858860109 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18744 (secure 0 0 0) 0x7fb07015ac30 con 0x7fb070158760 2023-09-23T03:16:39.051 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:16:39.052 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":26,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:16:03.151371+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:15:18.575849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T03:15:19.707212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T03:15:23.337138+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:16:39.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.053+0000 7fb076607640 1 -- 172.21.15.154:0/3858860109 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb05006af30 msgr2=0x7fb05006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:39.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.053+0000 7fb076607640 1 --2- 172.21.15.154:0/3858860109 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb05006af30 0x7fb05006d3f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb060009a80 tx=0x7fb060008040 comp rx=0 tx=0).stop 2023-09-23T03:16:39.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.053+0000 7fb076607640 1 -- 172.21.15.154:0/3858860109 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb070158760 msgr2=0x7fb0700c4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:39.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.053+0000 7fb076607640 1 --2- 172.21.15.154:0/3858860109 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb070158760 0x7fb0700c4a00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb05800dae0 tx=0x7fb058007e50 comp rx=0 tx=0).stop 2023-09-23T03:16:39.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.053+0000 7fb076607640 1 -- 172.21.15.154:0/3858860109 shutdown_connections 2023-09-23T03:16:39.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.053+0000 7fb076607640 1 --2- 172.21.15.154:0/3858860109 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb05006af30 0x7fb05006d3f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:39.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.053+0000 7fb076607640 1 --2- 172.21.15.154:0/3858860109 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0700c9ff0 0x7fb0700ca470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:39.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.053+0000 7fb076607640 1 --2- 172.21.15.154:0/3858860109 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb070158760 0x7fb0700c4a00 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:39.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.053+0000 7fb076607640 1 --2- 172.21.15.154:0/3858860109 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb05c004d00 0x7fb0700aea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:39.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.053+0000 7fb076607640 1 -- 172.21.15.154:0/3858860109 >> 172.21.15.154:0/3858860109 conn(0x7fb07006a0f0 msgr2=0x7fb0700ccbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:39.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.053+0000 7fb076607640 1 -- 172.21.15.154:0/3858860109 shutdown_connections 2023-09-23T03:16:39.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.053+0000 7fb076607640 1 -- 172.21.15.154:0/3858860109 wait complete. 2023-09-23T03:16:39.072 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:15:18.575849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:15:19.707212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-23T03:15:23.337138+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-23T03:16:39.072 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:16:39.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.296+0000 7f4e773fc640 1 Processor -- start 2023-09-23T03:16:39.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.296+0000 7f4e773fc640 1 -- start start 2023-09-23T03:16:39.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.298+0000 7f4e773fc640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4e700c9980 0x7f4e70163dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:39.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.298+0000 7f4e773fc640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e700c1a40 con 0x7f4e700c9510 2023-09-23T03:16:39.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.298+0000 7f4e773fc640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e701624f0 con 0x7f4e700cae90 2023-09-23T03:16:39.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.298+0000 7f4e773fc640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e70162be0 con 0x7f4e700c9980 2023-09-23T03:16:39.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.298+0000 7f4e74997640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f4e700c9510 0x7f4e701636c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:42090/0 (socket says 172.21.15.154:42090) 2023-09-23T03:16:39.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.298+0000 7f4e74997640 1 -- 172.21.15.154:0/775814317 learned_addr learned my addr 172.21.15.154:0/775814317 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:39.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.299+0000 7f4e75999640 1 --2- 172.21.15.154:0/775814317 >> v2:172.21.15.190:3300/0 conn(0x7f4e700c9980 0x7f4e70163dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:39.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.299+0000 7f4e67fff640 1 -- 172.21.15.154:0/775814317 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 531484893 0 0) 0x7f4e700c1a40 con 0x7f4e700c9510 2023-09-23T03:16:39.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.299+0000 7f4e67fff640 1 -- 172.21.15.154:0/775814317 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e5c003520 con 0x7f4e700c9510 2023-09-23T03:16:39.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.299+0000 7f4e67fff640 1 -- 172.21.15.154:0/775814317 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1129220565 0 0) 0x7f4e5c003520 con 0x7f4e700c9510 2023-09-23T03:16:39.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.299+0000 7f4e67fff640 1 -- 172.21.15.154:0/775814317 >> v2:172.21.15.190:3300/0 conn(0x7f4e700c9980 msgr2=0x7f4e70163dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:39.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.299+0000 7f4e67fff640 1 --2- 172.21.15.154:0/775814317 >> v2:172.21.15.190:3300/0 conn(0x7f4e700c9980 0x7f4e70163dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:39.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.299+0000 7f4e67fff640 1 -- 172.21.15.154:0/775814317 >> v1:172.21.15.190:6789/0 conn(0x7f4e700cae90 legacy=0x7f4e700cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:39.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.299+0000 7f4e67fff640 1 -- 172.21.15.154:0/775814317 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e70162ec0 con 0x7f4e700c9510 2023-09-23T03:16:39.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.299+0000 7f4e67fff640 1 -- 172.21.15.154:0/775814317 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f4e60001690 con 0x7f4e700c9510 2023-09-23T03:16:39.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.299+0000 7f4e67fff640 1 -- 172.21.15.154:0/775814317 >> v1:172.21.15.154:6789/0 conn(0x7f4e700c9510 legacy=0x7f4e701636c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:39.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.299+0000 7f4e67fff640 1 --2- 172.21.15.154:0/775814317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e5c003970 0x7f4e5c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:39.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.299+0000 7f4e67fff640 1 --2- 172.21.15.154:0/775814317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e5c0047c0 0x7f4e5c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:39.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.299+0000 7f4e67fff640 1 --2- 172.21.15.154:0/775814317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e5c005180 0x7f4e5c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:39.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.299+0000 7f4e67fff640 1 -- 172.21.15.154:0/775814317 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e70162be0 con 0x7f4e5c0047c0 2023-09-23T03:16:39.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.299+0000 7f4e67fff640 1 -- 172.21.15.154:0/775814317 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c00c030 con 0x7f4e5c005180 2023-09-23T03:16:39.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.299+0000 7f4e67fff640 1 -- 172.21.15.154:0/775814317 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c00c1f0 con 0x7f4e5c003970 2023-09-23T03:16:39.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.300+0000 7f4e74997640 1 --2- 172.21.15.154:0/775814317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e5c0047c0 0x7f4e5c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:39.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.300+0000 7f4e74997640 1 -- 172.21.15.154:0/775814317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e5c003970 msgr2=0x7f4e5c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:39.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.300+0000 7f4e75198640 1 --2- 172.21.15.154:0/775814317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e5c003970 0x7f4e5c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:39.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.300+0000 7f4e74997640 1 --2- 172.21.15.154:0/775814317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e5c003970 0x7f4e5c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:39.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.300+0000 7f4e75999640 1 --2- 172.21.15.154:0/775814317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e5c005180 0x7f4e5c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:39.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.300+0000 7f4e74997640 1 -- 172.21.15.154:0/775814317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e5c005180 msgr2=0x7f4e5c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:39.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.300+0000 7f4e74997640 1 --2- 172.21.15.154:0/775814317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e5c005180 0x7f4e5c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:39.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.300+0000 7f4e74997640 1 -- 172.21.15.154:0/775814317 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e5c003520 con 0x7f4e5c0047c0 2023-09-23T03:16:39.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.300+0000 7f4e75999640 1 -- 172.21.15.154:0/775814317 reap_dead start 2023-09-23T03:16:39.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.300+0000 7f4e75198640 1 --2- 172.21.15.154:0/775814317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e5c003970 0x7f4e5c003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:39.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.301+0000 7f4e74997640 1 --2- 172.21.15.154:0/775814317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e5c0047c0 0x7f4e5c004c40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4e700c9dc0 tx=0x7f4e5c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c505aed2dc3eafa server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:39.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.301+0000 7f4e67fff640 1 -- 172.21.15.154:0/775814317 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e60005ad0 con 0x7f4e5c0047c0 2023-09-23T03:16:39.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.301+0000 7f4e67fff640 1 -- 172.21.15.154:0/775814317 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e60005c30 con 0x7f4e5c0047c0 2023-09-23T03:16:39.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.301+0000 7f4e67fff640 1 -- 172.21.15.154:0/775814317 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6002c820 con 0x7f4e5c0047c0 2023-09-23T03:16:39.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.302+0000 7f4e773fc640 1 -- 172.21.15.154:0/775814317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e5c0047c0 msgr2=0x7f4e5c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:39.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.302+0000 7f4e773fc640 1 --2- 172.21.15.154:0/775814317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e5c0047c0 0x7f4e5c004c40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4e700c9dc0 tx=0x7f4e5c0057f0 comp rx=0 tx=0).stop 2023-09-23T03:16:39.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.302+0000 7f4e773fc640 1 -- 172.21.15.154:0/775814317 shutdown_connections 2023-09-23T03:16:39.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.302+0000 7f4e773fc640 1 --2- 172.21.15.154:0/775814317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e5c0047c0 0x7f4e5c004c40 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:39.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.302+0000 7f4e773fc640 1 -- 172.21.15.154:0/775814317 >> 172.21.15.154:0/775814317 conn(0x7f4e7006c010 msgr2=0x7f4e7006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:39.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.302+0000 7f4e773fc640 1 -- 172.21.15.154:0/775814317 shutdown_connections 2023-09-23T03:16:39.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.302+0000 7f4e773fc640 1 -- 172.21.15.154:0/775814317 wait complete. 2023-09-23T03:16:39.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.304+0000 7f4e773fc640 1 Processor -- start 2023-09-23T03:16:39.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.304+0000 7f4e773fc640 1 -- start start 2023-09-23T03:16:39.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.305+0000 7f4e773fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e5c0047c0 0x7f4e701f6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:39.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.305+0000 7f4e773fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e701f72f0 0x7f4e700c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:39.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.305+0000 7f4e773fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e700c8630 0x7f4e7015f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:39.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.305+0000 7f4e773fc640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e701f9d50 con 0x7f4e700c8630 2023-09-23T03:16:39.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.305+0000 7f4e773fc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e701fa030 con 0x7f4e701f72f0 2023-09-23T03:16:39.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.305+0000 7f4e773fc640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e701fa310 con 0x7f4e5c0047c0 2023-09-23T03:16:39.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.306+0000 7f4e75999640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e700c8630 0x7f4e7015f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:39.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.306+0000 7f4e75198640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e5c0047c0 0x7f4e701f6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:39.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.306+0000 7f4e75999640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e700c8630 0x7f4e7015f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52650/0 (socket says 172.21.15.154:52650) 2023-09-23T03:16:39.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.306+0000 7f4e75999640 1 -- 172.21.15.154:0/3636739244 learned_addr learned my addr 172.21.15.154:0/3636739244 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:39.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.306+0000 7f4e74997640 1 --2- 172.21.15.154:0/3636739244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e701f72f0 0x7f4e700c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:39.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.306+0000 7f4e75999640 1 -- 172.21.15.154:0/3636739244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e5c0047c0 msgr2=0x7f4e701f6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:39.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.306+0000 7f4e75999640 1 --2- 172.21.15.154:0/3636739244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e5c0047c0 0x7f4e701f6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:39.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.306+0000 7f4e75999640 1 -- 172.21.15.154:0/3636739244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e701f72f0 msgr2=0x7f4e700c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:39.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.306+0000 7f4e75999640 1 --2- 172.21.15.154:0/3636739244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e701f72f0 0x7f4e700c80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:39.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.306+0000 7f4e75999640 1 -- 172.21.15.154:0/3636739244 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e701fa5f0 con 0x7f4e700c8630 2023-09-23T03:16:39.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.307+0000 7f4e75999640 1 --2- 172.21.15.154:0/3636739244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e700c8630 0x7f4e7015f210 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4e700c9d90 tx=0x7f4e6800f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:39.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.307+0000 7f4e65ffb640 1 -- 172.21.15.154:0/3636739244 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e68016420 con 0x7f4e700c8630 2023-09-23T03:16:39.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.307+0000 7f4e65ffb640 1 -- 172.21.15.154:0/3636739244 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e680165a0 con 0x7f4e700c8630 2023-09-23T03:16:39.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.307+0000 7f4e773fc640 1 -- 172.21.15.154:0/3636739244 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e701fa8d0 con 0x7f4e700c8630 2023-09-23T03:16:39.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.307+0000 7f4e65ffb640 1 -- 172.21.15.154:0/3636739244 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e68016950 con 0x7f4e700c8630 2023-09-23T03:16:39.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.307+0000 7f4e773fc640 1 -- 172.21.15.154:0/3636739244 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e7015f930 con 0x7f4e700c8630 2023-09-23T03:16:39.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.308+0000 7f4e773fc640 1 -- 172.21.15.154:0/3636739244 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e701faa60 con 0x7f4e700c8630 2023-09-23T03:16:39.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.309+0000 7f4e65ffb640 1 -- 172.21.15.154:0/3636739244 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f4e68016ab0 con 0x7f4e700c8630 2023-09-23T03:16:39.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.310+0000 7f4e65ffb640 1 --2- 172.21.15.154:0/3636739244 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4e5406af30 0x7f4e5406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:39.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.310+0000 7f4e65ffb640 1 -- 172.21.15.154:0/3636739244 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f4e6808aaa0 con 0x7f4e700c8630 2023-09-23T03:16:39.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.310+0000 7f4e75198640 1 --2- 172.21.15.154:0/3636739244 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4e5406af30 0x7f4e5406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:39.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.311+0000 7f4e75198640 1 --2- 172.21.15.154:0/3636739244 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4e5406af30 0x7f4e5406d3f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4e70070740 tx=0x7f4e6c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:39.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.315+0000 7f4e65ffb640 1 -- 172.21.15.154:0/3636739244 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f4e68055690 con 0x7f4e700c8630 2023-09-23T03:16:39.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.505+0000 7f4e773fc640 1 -- 172.21.15.154:0/3636739244 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4e7004e920 con 0x7f4e700c8630 2023-09-23T03:16:39.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.505+0000 7f4e65ffb640 1 -- 172.21.15.154:0/3636739244 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4e68058cb0 con 0x7f4e700c8630 2023-09-23T03:16:39.507 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:16:39.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.508+0000 7f4e773fc640 1 -- 172.21.15.154:0/3636739244 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4e5406af30 msgr2=0x7f4e5406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:39.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.508+0000 7f4e773fc640 1 --2- 172.21.15.154:0/3636739244 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4e5406af30 0x7f4e5406d3f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4e70070740 tx=0x7f4e6c009040 comp rx=0 tx=0).stop 2023-09-23T03:16:39.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.508+0000 7f4e773fc640 1 -- 172.21.15.154:0/3636739244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e700c8630 msgr2=0x7f4e7015f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:39.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.508+0000 7f4e773fc640 1 --2- 172.21.15.154:0/3636739244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e700c8630 0x7f4e7015f210 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4e700c9d90 tx=0x7f4e6800f520 comp rx=0 tx=0).stop 2023-09-23T03:16:39.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.508+0000 7f4e773fc640 1 -- 172.21.15.154:0/3636739244 shutdown_connections 2023-09-23T03:16:39.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.508+0000 7f4e773fc640 1 --2- 172.21.15.154:0/3636739244 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4e5406af30 0x7f4e5406d3f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:39.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.508+0000 7f4e773fc640 1 --2- 172.21.15.154:0/3636739244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e700c8630 0x7f4e7015f210 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:39.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.508+0000 7f4e773fc640 1 --2- 172.21.15.154:0/3636739244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4e701f72f0 0x7f4e700c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:39.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.508+0000 7f4e773fc640 1 --2- 172.21.15.154:0/3636739244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4e5c0047c0 0x7f4e701f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:39.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.508+0000 7f4e773fc640 1 -- 172.21.15.154:0/3636739244 >> 172.21.15.154:0/3636739244 conn(0x7f4e7006c010 msgr2=0x7f4e70070930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:39.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.508+0000 7f4e773fc640 1 -- 172.21.15.154:0/3636739244 shutdown_connections 2023-09-23T03:16:39.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.508+0000 7f4e773fc640 1 -- 172.21.15.154:0/3636739244 wait complete. 2023-09-23T03:16:39.528 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:16:39.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.741+0000 7f7ae57e2640 1 Processor -- start 2023-09-23T03:16:39.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.741+0000 7f7ae57e2640 1 -- start start 2023-09-23T03:16:39.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.742+0000 7f7ae57e2640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7ae015cc40 0x7f7ae00afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:39.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.742+0000 7f7ae57e2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7ae006b6d0 0x7f7ae006bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:39.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.742+0000 7f7ae57e2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ae01607a0 0x7f7ae0160c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:39.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.742+0000 7f7ae57e2640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ae006c0a0 con 0x7f7ae006b6d0 2023-09-23T03:16:39.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.742+0000 7f7ae57e2640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ae006b440 con 0x7f7ae015cc40 2023-09-23T03:16:39.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.742+0000 7f7ae57e2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae00cd000 con 0x7f7ae01607a0 2023-09-23T03:16:39.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.743+0000 7f7ade7fc640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7ae006b6d0 0x7f7ae006bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:39.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.743+0000 7f7ade7fc640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7ae006b6d0 0x7f7ae006bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52666/0 (socket says 172.21.15.154:52666) 2023-09-23T03:16:39.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.743+0000 7f7ade7fc640 1 -- 172.21.15.154:0/3089335414 learned_addr learned my addr 172.21.15.154:0/3089335414 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:39.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.743+0000 7f7adeffd640 1 --2- 172.21.15.154:0/3089335414 >> v2:172.21.15.190:3300/0 conn(0x7f7ae015cc40 0x7f7ae00afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:39.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.743+0000 7f7adf7fe640 1 --2- 172.21.15.154:0/3089335414 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ae01607a0 0x7f7ae0160c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:39.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.743+0000 7f7ade7fc640 1 -- 172.21.15.154:0/3089335414 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ae01607a0 msgr2=0x7f7ae0160c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:39.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.743+0000 7f7ade7fc640 1 --2- 172.21.15.154:0/3089335414 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ae01607a0 0x7f7ae0160c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:39.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.743+0000 7f7ade7fc640 1 -- 172.21.15.154:0/3089335414 >> v2:172.21.15.190:3300/0 conn(0x7f7ae015cc40 msgr2=0x7f7ae00afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:39.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.743+0000 7f7ade7fc640 1 --2- 172.21.15.154:0/3089335414 >> v2:172.21.15.190:3300/0 conn(0x7f7ae015cc40 0x7f7ae00afcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:39.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.743+0000 7f7ade7fc640 1 -- 172.21.15.154:0/3089335414 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae00cd2e0 con 0x7f7ae006b6d0 2023-09-23T03:16:39.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.745+0000 7f7ade7fc640 1 --2- 172.21.15.154:0/3089335414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ae006b6d0 0x7f7ae006bb60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8009870 tx=0x7f7ac802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ffc96c4c063939 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:39.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.745+0000 7f7addffb640 1 -- 172.21.15.154:0/3089335414 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac80317c0 con 0x7f7ae006b6d0 2023-09-23T03:16:39.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.745+0000 7f7addffb640 1 -- 172.21.15.154:0/3089335414 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ac8031920 con 0x7f7ae006b6d0 2023-09-23T03:16:39.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.745+0000 7f7addffb640 1 -- 172.21.15.154:0/3089335414 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac8031cb0 con 0x7f7ae006b6d0 2023-09-23T03:16:39.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.746+0000 7f7ae57e2640 1 -- 172.21.15.154:0/3089335414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ae006b6d0 msgr2=0x7f7ae006bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:39.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.746+0000 7f7ae57e2640 1 --2- 172.21.15.154:0/3089335414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ae006b6d0 0x7f7ae006bb60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8009870 tx=0x7f7ac802b860 comp rx=0 tx=0).stop 2023-09-23T03:16:39.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.746+0000 7f7ae57e2640 1 -- 172.21.15.154:0/3089335414 shutdown_connections 2023-09-23T03:16:39.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.746+0000 7f7ae57e2640 1 --2- 172.21.15.154:0/3089335414 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ae01607a0 0x7f7ae0160c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:39.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.746+0000 7f7ae57e2640 1 --2- 172.21.15.154:0/3089335414 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ae006b6d0 0x7f7ae006bb60 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:39.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.746+0000 7f7ae57e2640 1 --2- 172.21.15.154:0/3089335414 >> v2:172.21.15.190:3300/0 conn(0x7f7ae015cc40 0x7f7ae00afcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:39.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.746+0000 7f7ae57e2640 1 -- 172.21.15.154:0/3089335414 >> 172.21.15.154:0/3089335414 conn(0x7f7ae0089fd0 msgr2=0x7f7ae008a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:39.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.746+0000 7f7ae57e2640 1 -- 172.21.15.154:0/3089335414 shutdown_connections 2023-09-23T03:16:39.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.746+0000 7f7ae57e2640 1 -- 172.21.15.154:0/3089335414 wait complete. 2023-09-23T03:16:39.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.748+0000 7f7ae57e2640 1 Processor -- start 2023-09-23T03:16:39.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.748+0000 7f7ae57e2640 1 -- start start 2023-09-23T03:16:39.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.750+0000 7f7ae57e2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ae006b6d0 0x7f7ae0159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:39.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.750+0000 7f7ae57e2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ae015cc40 0x7f7ae00cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:39.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.750+0000 7f7ae57e2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ae01607a0 0x7f7ae015a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:39.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.750+0000 7f7ae57e2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae00cc3d0 con 0x7f7ae01607a0 2023-09-23T03:16:39.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.750+0000 7f7ae57e2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae00cc6b0 con 0x7f7ae006b6d0 2023-09-23T03:16:39.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.750+0000 7f7ae57e2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae00cc990 con 0x7f7ae015cc40 2023-09-23T03:16:39.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.750+0000 7f7adf7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ae01607a0 0x7f7ae015a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:39.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.750+0000 7f7adf7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ae01607a0 0x7f7ae015a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52676/0 (socket says 172.21.15.154:52676) 2023-09-23T03:16:39.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.750+0000 7f7adf7fe640 1 -- 172.21.15.154:0/3381769443 learned_addr learned my addr 172.21.15.154:0/3381769443 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:39.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.750+0000 7f7adeffd640 1 --2- 172.21.15.154:0/3381769443 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ae006b6d0 0x7f7ae0159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:39.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.750+0000 7f7ade7fc640 1 --2- 172.21.15.154:0/3381769443 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ae015cc40 0x7f7ae00cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:39.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.751+0000 7f7adf7fe640 1 -- 172.21.15.154:0/3381769443 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ae015cc40 msgr2=0x7f7ae00cbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:39.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.751+0000 7f7adf7fe640 1 --2- 172.21.15.154:0/3381769443 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ae015cc40 0x7f7ae00cbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:39.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.751+0000 7f7adf7fe640 1 -- 172.21.15.154:0/3381769443 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ae006b6d0 msgr2=0x7f7ae0159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:39.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.751+0000 7f7adf7fe640 1 --2- 172.21.15.154:0/3381769443 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ae006b6d0 0x7f7ae0159690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:39.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.751+0000 7f7adf7fe640 1 -- 172.21.15.154:0/3381769443 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac80094d0 con 0x7f7ae01607a0 2023-09-23T03:16:39.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.751+0000 7f7adf7fe640 1 --2- 172.21.15.154:0/3381769443 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ae01607a0 0x7f7ae015a7f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7ad800d800 tx=0x7f7ad800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:39.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.751+0000 7f7ad77fe640 1 -- 172.21.15.154:0/3381769443 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad800b6c0 con 0x7f7ae01607a0 2023-09-23T03:16:39.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.751+0000 7f7ad77fe640 1 -- 172.21.15.154:0/3381769443 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad800b840 con 0x7f7ae01607a0 2023-09-23T03:16:39.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.751+0000 7f7ae57e2640 1 -- 172.21.15.154:0/3381769443 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ae00ccc70 con 0x7f7ae01607a0 2023-09-23T03:16:39.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.751+0000 7f7ad77fe640 1 -- 172.21.15.154:0/3381769443 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad800bbf0 con 0x7f7ae01607a0 2023-09-23T03:16:39.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.751+0000 7f7ae57e2640 1 -- 172.21.15.154:0/3381769443 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae00c4970 con 0x7f7ae01607a0 2023-09-23T03:16:39.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.753+0000 7f7ae57e2640 1 -- 172.21.15.154:0/3381769443 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ae006bec0 con 0x7f7ae01607a0 2023-09-23T03:16:39.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.753+0000 7f7ad77fe640 1 -- 172.21.15.154:0/3381769443 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f7ad800bd50 con 0x7f7ae01607a0 2023-09-23T03:16:39.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.754+0000 7f7ad77fe640 1 --2- 172.21.15.154:0/3381769443 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7ac006af30 0x7f7ac006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:39.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.754+0000 7f7ad77fe640 1 -- 172.21.15.154:0/3381769443 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f7ad808a080 con 0x7f7ae01607a0 2023-09-23T03:16:39.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.755+0000 7f7adeffd640 1 --2- 172.21.15.154:0/3381769443 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7ac006af30 0x7f7ac006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:39.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.755+0000 7f7adeffd640 1 --2- 172.21.15.154:0/3381769443 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7ac006af30 0x7f7ac006d3f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0009ab0 tx=0x7f7ad0003990 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:39.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.759+0000 7f7ad77fe640 1 -- 172.21.15.154:0/3381769443 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f7ad80bca50 con 0x7f7ae01607a0 2023-09-23T03:16:39.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.945+0000 7f7ae57e2640 1 -- 172.21.15.154:0/3381769443 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7ae004e920 con 0x7f7ae01607a0 2023-09-23T03:16:39.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.946+0000 7f7ad77fe640 1 -- 172.21.15.154:0/3381769443 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7ad8054c70 con 0x7f7ae01607a0 2023-09-23T03:16:39.948 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:16:39.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.949+0000 7f7ae57e2640 1 -- 172.21.15.154:0/3381769443 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7ac006af30 msgr2=0x7f7ac006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:39.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.949+0000 7f7ae57e2640 1 --2- 172.21.15.154:0/3381769443 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7ac006af30 0x7f7ac006d3f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0009ab0 tx=0x7f7ad0003990 comp rx=0 tx=0).stop 2023-09-23T03:16:39.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.949+0000 7f7ae57e2640 1 -- 172.21.15.154:0/3381769443 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ae01607a0 msgr2=0x7f7ae015a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:39.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.949+0000 7f7ae57e2640 1 --2- 172.21.15.154:0/3381769443 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ae01607a0 0x7f7ae015a7f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7ad800d800 tx=0x7f7ad800dcc0 comp rx=0 tx=0).stop 2023-09-23T03:16:39.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.949+0000 7f7ae57e2640 1 -- 172.21.15.154:0/3381769443 shutdown_connections 2023-09-23T03:16:39.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.949+0000 7f7ae57e2640 1 --2- 172.21.15.154:0/3381769443 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7ac006af30 0x7f7ac006d3f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:39.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.949+0000 7f7ae57e2640 1 --2- 172.21.15.154:0/3381769443 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ae01607a0 0x7f7ae015a7f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:39.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.949+0000 7f7ae57e2640 1 --2- 172.21.15.154:0/3381769443 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ae015cc40 0x7f7ae00cbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:39.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.949+0000 7f7ae57e2640 1 --2- 172.21.15.154:0/3381769443 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ae006b6d0 0x7f7ae0159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:39.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.949+0000 7f7ae57e2640 1 -- 172.21.15.154:0/3381769443 >> 172.21.15.154:0/3381769443 conn(0x7f7ae0089fd0 msgr2=0x7f7ae00ceb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:39.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.949+0000 7f7ae57e2640 1 -- 172.21.15.154:0/3381769443 shutdown_connections 2023-09-23T03:16:39.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:39.949+0000 7f7ae57e2640 1 -- 172.21.15.154:0/3381769443 wait complete. 2023-09-23T03:16:39.969 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:16:40.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.169+0000 7f82a9e40640 1 Processor -- start 2023-09-23T03:16:40.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.169+0000 7f82a9e40640 1 -- start start 2023-09-23T03:16:40.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.171+0000 7f82a9e40640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82a41606d0 0x7f82a406fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:40.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.171+0000 7f82a9e40640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82a4160bd0 con 0x7f82a40cb4f0 2023-09-23T03:16:40.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.171+0000 7f82a9e40640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82a4160eb0 con 0x7f82a415cc60 2023-09-23T03:16:40.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.171+0000 7f82a9e40640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82a40cc4e0 con 0x7f82a41606d0 2023-09-23T03:16:40.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.171+0000 7f82a3fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f82a40cb4f0 0x7f82a40cc930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:42106/0 (socket says 172.21.15.154:42106) 2023-09-23T03:16:40.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.171+0000 7f82a3fff640 1 -- 172.21.15.154:0/3289100569 learned_addr learned my addr 172.21.15.154:0/3289100569 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:40.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.171+0000 7f82a963f640 1 --2- 172.21.15.154:0/3289100569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82a41606d0 0x7f82a406fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:40.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.171+0000 7f82a37fe640 1 -- 172.21.15.154:0/3289100569 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1077829142 0 0) 0x7f82a4160bd0 con 0x7f82a40cb4f0 2023-09-23T03:16:40.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.171+0000 7f82a37fe640 1 -- 172.21.15.154:0/3289100569 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8294003520 con 0x7f82a40cb4f0 2023-09-23T03:16:40.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.172+0000 7f82a37fe640 1 -- 172.21.15.154:0/3289100569 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3674114564 0 0) 0x7f8294003520 con 0x7f82a40cb4f0 2023-09-23T03:16:40.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.172+0000 7f82a37fe640 1 -- 172.21.15.154:0/3289100569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82a41606d0 msgr2=0x7f82a406fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:40.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.172+0000 7f82a37fe640 1 --2- 172.21.15.154:0/3289100569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82a41606d0 0x7f82a406fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:40.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.172+0000 7f82a37fe640 1 -- 172.21.15.154:0/3289100569 >> v1:172.21.15.190:6789/0 conn(0x7f82a415cc60 legacy=0x7f82a40cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:40.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.172+0000 7f82a37fe640 1 -- 172.21.15.154:0/3289100569 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a4161190 con 0x7f82a40cb4f0 2023-09-23T03:16:40.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.172+0000 7f82a37fe640 1 -- 172.21.15.154:0/3289100569 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f8290001670 con 0x7f82a40cb4f0 2023-09-23T03:16:40.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.172+0000 7f82a37fe640 1 -- 172.21.15.154:0/3289100569 >> v1:172.21.15.154:6789/0 conn(0x7f82a40cb4f0 legacy=0x7f82a40cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:40.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.172+0000 7f82a37fe640 1 --2- 172.21.15.154:0/3289100569 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82940039a0 0x7f8294003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:40.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.172+0000 7f82a37fe640 1 --2- 172.21.15.154:0/3289100569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82a41606d0 0x7f82940047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:40.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.172+0000 7f82a37fe640 1 --2- 172.21.15.154:0/3289100569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8294004d00 0x7f8294005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:40.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.172+0000 7f82a37fe640 1 -- 172.21.15.154:0/3289100569 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a40cc4e0 con 0x7f82940039a0 2023-09-23T03:16:40.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.172+0000 7f82a37fe640 1 -- 172.21.15.154:0/3289100569 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829400bb10 con 0x7f8294004d00 2023-09-23T03:16:40.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.172+0000 7f82a37fe640 1 -- 172.21.15.154:0/3289100569 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f829400bca0 con 0x7f82a41606d0 2023-09-23T03:16:40.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.172+0000 7f82a8e3e640 1 --2- 172.21.15.154:0/3289100569 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82940039a0 0x7f8294003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:40.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.173+0000 7f82a3fff640 1 --2- 172.21.15.154:0/3289100569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82a41606d0 0x7f82940047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:40.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.173+0000 7f82a8e3e640 1 -- 172.21.15.154:0/3289100569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82a41606d0 msgr2=0x7f82940047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:40.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.173+0000 7f82a8e3e640 1 --2- 172.21.15.154:0/3289100569 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82a41606d0 0x7f82940047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:40.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.173+0000 7f82a8e3e640 1 -- 172.21.15.154:0/3289100569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8294004d00 msgr2=0x7f8294005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:40.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.173+0000 7f82a8e3e640 1 --2- 172.21.15.154:0/3289100569 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8294004d00 0x7f8294005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:40.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.173+0000 7f82a8e3e640 1 -- 172.21.15.154:0/3289100569 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8294003520 con 0x7f82940039a0 2023-09-23T03:16:40.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.174+0000 7f82a8e3e640 1 --2- 172.21.15.154:0/3289100569 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82940039a0 0x7f8294003dc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8298001780 tx=0x7f82940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=865683921a58138b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:40.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.174+0000 7f82a37fe640 1 -- 172.21.15.154:0/3289100569 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82980052d0 con 0x7f82940039a0 2023-09-23T03:16:40.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.174+0000 7f82a37fe640 1 -- 172.21.15.154:0/3289100569 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f829802e050 con 0x7f82940039a0 2023-09-23T03:16:40.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.174+0000 7f82a37fe640 1 -- 172.21.15.154:0/3289100569 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82980035f0 con 0x7f82940039a0 2023-09-23T03:16:40.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.174+0000 7f82a9e40640 1 -- 172.21.15.154:0/3289100569 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82940039a0 msgr2=0x7f8294003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:40.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.174+0000 7f82a9e40640 1 --2- 172.21.15.154:0/3289100569 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82940039a0 0x7f8294003dc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8298001780 tx=0x7f82940057f0 comp rx=0 tx=0).stop 2023-09-23T03:16:40.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.174+0000 7f82a963f640 1 -- 172.21.15.154:0/3289100569 reap_dead start 2023-09-23T03:16:40.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.175+0000 7f82a9e40640 1 -- 172.21.15.154:0/3289100569 shutdown_connections 2023-09-23T03:16:40.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.175+0000 7f82a9e40640 1 -- 172.21.15.154:0/3289100569 >> 172.21.15.154:0/3289100569 conn(0x7f82a40aff20 msgr2=0x7f82a40b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:40.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.175+0000 7f82a9e40640 1 -- 172.21.15.154:0/3289100569 shutdown_connections 2023-09-23T03:16:40.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.175+0000 7f82a9e40640 1 -- 172.21.15.154:0/3289100569 wait complete. 2023-09-23T03:16:40.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.176+0000 7f82a9e40640 1 Processor -- start 2023-09-23T03:16:40.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.177+0000 7f82a9e40640 1 -- start start 2023-09-23T03:16:40.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.177+0000 7f82a9e40640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82940039a0 0x7f82a4155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:40.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.177+0000 7f82a9e40640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82a40b1d30 0x7f82a40b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:40.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.177+0000 7f82a9e40640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82a415f480 0x7f82a415f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:40.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.177+0000 7f82a9e40640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a415d040 con 0x7f82a40b1d30 2023-09-23T03:16:40.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.178+0000 7f82a9e40640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a415fe40 con 0x7f82940039a0 2023-09-23T03:16:40.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.178+0000 7f82a9e40640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82a40b3150 con 0x7f82a415f480 2023-09-23T03:16:40.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.178+0000 7f82a3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82a40b1d30 0x7f82a40b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:40.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.178+0000 7f82a3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82a40b1d30 0x7f82a40b21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52702/0 (socket says 172.21.15.154:52702) 2023-09-23T03:16:40.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.178+0000 7f82a3fff640 1 -- 172.21.15.154:0/3795483949 learned_addr learned my addr 172.21.15.154:0/3795483949 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:40.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.178+0000 7f82a3fff640 1 -- 172.21.15.154:0/3795483949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82a415f480 msgr2=0x7f82a415f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:40.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.178+0000 7f82a3fff640 1 --2- 172.21.15.154:0/3795483949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82a415f480 0x7f82a415f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:40.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.178+0000 7f82a3fff640 1 -- 172.21.15.154:0/3795483949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82940039a0 msgr2=0x7f82a4155f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:40.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.178+0000 7f82a3fff640 1 --2- 172.21.15.154:0/3795483949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82940039a0 0x7f82a4155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:40.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.178+0000 7f82a3fff640 1 -- 172.21.15.154:0/3795483949 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a40b28b0 con 0x7f82a40b1d30 2023-09-23T03:16:40.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.178+0000 7f82a3fff640 1 --2- 172.21.15.154:0/3795483949 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82a40b1d30 0x7f82a40b21b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f829000d840 tx=0x7f829000dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:40.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.179+0000 7f82a17fa640 1 -- 172.21.15.154:0/3795483949 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82900074e0 con 0x7f82a40b1d30 2023-09-23T03:16:40.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.179+0000 7f82a17fa640 1 -- 172.21.15.154:0/3795483949 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8290007640 con 0x7f82a40b1d30 2023-09-23T03:16:40.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.179+0000 7f82a17fa640 1 -- 172.21.15.154:0/3795483949 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82900049e0 con 0x7f82a40b1d30 2023-09-23T03:16:40.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.179+0000 7f82a9e40640 1 -- 172.21.15.154:0/3795483949 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82a40b2b90 con 0x7f82a40b1d30 2023-09-23T03:16:40.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.179+0000 7f82a9e40640 1 -- 172.21.15.154:0/3795483949 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82a40c2030 con 0x7f82a40b1d30 2023-09-23T03:16:40.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.180+0000 7f82a9e40640 1 -- 172.21.15.154:0/3795483949 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82a40c2250 con 0x7f82a40b1d30 2023-09-23T03:16:40.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.181+0000 7f82a17fa640 1 -- 172.21.15.154:0/3795483949 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f8290004b40 con 0x7f82a40b1d30 2023-09-23T03:16:40.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.182+0000 7f82a17fa640 1 --2- 172.21.15.154:0/3795483949 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f828806af60 0x7f828806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:40.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.182+0000 7f82a17fa640 1 -- 172.21.15.154:0/3795483949 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f8290089440 con 0x7f82a40b1d30 2023-09-23T03:16:40.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.183+0000 7f82a8e3e640 1 --2- 172.21.15.154:0/3795483949 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f828806af60 0x7f828806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:40.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.183+0000 7f82a8e3e640 1 --2- 172.21.15.154:0/3795483949 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f828806af60 0x7f828806d420 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8298033000 tx=0x7f829802f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:40.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.187+0000 7f82a17fa640 1 -- 172.21.15.154:0/3795483949 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f8290054160 con 0x7f82a40b1d30 2023-09-23T03:16:40.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.380+0000 7f82a9e40640 1 -- 172.21.15.154:0/3795483949 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f82a404e920 con 0x7f82a40b1d30 2023-09-23T03:16:40.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.381+0000 7f82a17fa640 1 -- 172.21.15.154:0/3795483949 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8290057780 con 0x7f82a40b1d30 2023-09-23T03:16:40.382 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:16:40.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.384+0000 7f82a9e40640 1 -- 172.21.15.154:0/3795483949 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f828806af60 msgr2=0x7f828806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:40.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.384+0000 7f82a9e40640 1 --2- 172.21.15.154:0/3795483949 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f828806af60 0x7f828806d420 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8298033000 tx=0x7f829802f000 comp rx=0 tx=0).stop 2023-09-23T03:16:40.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.384+0000 7f82a9e40640 1 -- 172.21.15.154:0/3795483949 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82a40b1d30 msgr2=0x7f82a40b21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:40.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.384+0000 7f82a9e40640 1 --2- 172.21.15.154:0/3795483949 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82a40b1d30 0x7f82a40b21b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f829000d840 tx=0x7f829000dd00 comp rx=0 tx=0).stop 2023-09-23T03:16:40.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.384+0000 7f82a9e40640 1 -- 172.21.15.154:0/3795483949 shutdown_connections 2023-09-23T03:16:40.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.384+0000 7f82a9e40640 1 --2- 172.21.15.154:0/3795483949 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f828806af60 0x7f828806d420 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:40.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.384+0000 7f82a9e40640 1 --2- 172.21.15.154:0/3795483949 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82a415f480 0x7f82a415f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:40.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.384+0000 7f82a9e40640 1 --2- 172.21.15.154:0/3795483949 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82a40b1d30 0x7f82a40b21b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:40.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.384+0000 7f82a9e40640 1 --2- 172.21.15.154:0/3795483949 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82940039a0 0x7f82a4155f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:40.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.384+0000 7f82a9e40640 1 -- 172.21.15.154:0/3795483949 >> 172.21.15.154:0/3795483949 conn(0x7f82a40aff20 msgr2=0x7f82a40aeba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:40.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.384+0000 7f82a9e40640 1 -- 172.21.15.154:0/3795483949 shutdown_connections 2023-09-23T03:16:40.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.384+0000 7f82a9e40640 1 -- 172.21.15.154:0/3795483949 wait complete. 2023-09-23T03:16:40.404 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-23T03:16:40.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.622+0000 7f239d9f4640 1 Processor -- start 2023-09-23T03:16:40.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.622+0000 7f239d9f4640 1 -- start start 2023-09-23T03:16:40.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.624+0000 7f239d9f4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2398158480 0x7f23980cade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:40.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.624+0000 7f239d9f4640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f239815ba50 0x7f239806fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:40.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.624+0000 7f239d9f4640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f239815eef0 0x7f23980703d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:40.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.624+0000 7f239d9f4640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f239815ad80 con 0x7f239815eef0 2023-09-23T03:16:40.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.624+0000 7f239d9f4640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23980cb6c0 con 0x7f239815ba50 2023-09-23T03:16:40.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.624+0000 7f239d9f4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f239815af70 con 0x7f2398158480 2023-09-23T03:16:40.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.624+0000 7f239d1f3640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f239815eef0 0x7f23980703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:40.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.624+0000 7f239d1f3640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f239815eef0 0x7f23980703d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52716/0 (socket says 172.21.15.154:52716) 2023-09-23T03:16:40.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.624+0000 7f239d1f3640 1 -- 172.21.15.154:0/4159701330 learned_addr learned my addr 172.21.15.154:0/4159701330 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:40.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.624+0000 7f2397fff640 1 --2- 172.21.15.154:0/4159701330 >> v2:172.21.15.190:3300/0 conn(0x7f239815ba50 0x7f239806fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:40.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.624+0000 7f239c9f2640 1 --2- 172.21.15.154:0/4159701330 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2398158480 0x7f23980cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:40.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.625+0000 7f239d1f3640 1 -- 172.21.15.154:0/4159701330 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2398158480 msgr2=0x7f23980cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:40.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.625+0000 7f239d1f3640 1 --2- 172.21.15.154:0/4159701330 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2398158480 0x7f23980cade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:40.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.625+0000 7f239d1f3640 1 -- 172.21.15.154:0/4159701330 >> v2:172.21.15.190:3300/0 conn(0x7f239815ba50 msgr2=0x7f239806fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:40.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.625+0000 7f239d1f3640 1 --2- 172.21.15.154:0/4159701330 >> v2:172.21.15.190:3300/0 conn(0x7f239815ba50 0x7f239806fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:40.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.625+0000 7f239d1f3640 1 -- 172.21.15.154:0/4159701330 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f239815b250 con 0x7f239815eef0 2023-09-23T03:16:40.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.626+0000 7f239d1f3640 1 --2- 172.21.15.154:0/4159701330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f239815eef0 0x7f23980703d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f239806e3a0 tx=0x7f239002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf1c11b69b30951f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:40.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.626+0000 7f23977fe640 1 -- 172.21.15.154:0/4159701330 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239002c9f0 con 0x7f239815eef0 2023-09-23T03:16:40.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.626+0000 7f23977fe640 1 -- 172.21.15.154:0/4159701330 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f239002cb50 con 0x7f239815eef0 2023-09-23T03:16:40.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.626+0000 7f23977fe640 1 -- 172.21.15.154:0/4159701330 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23900343f0 con 0x7f239815eef0 2023-09-23T03:16:40.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.627+0000 7f239d9f4640 1 -- 172.21.15.154:0/4159701330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f239815eef0 msgr2=0x7f23980703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:40.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.627+0000 7f239d9f4640 1 --2- 172.21.15.154:0/4159701330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f239815eef0 0x7f23980703d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f239806e3a0 tx=0x7f239002be50 comp rx=0 tx=0).stop 2023-09-23T03:16:40.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.627+0000 7f239d9f4640 1 -- 172.21.15.154:0/4159701330 shutdown_connections 2023-09-23T03:16:40.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.627+0000 7f239d9f4640 1 --2- 172.21.15.154:0/4159701330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f239815eef0 0x7f23980703d0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:40.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.627+0000 7f239d9f4640 1 --2- 172.21.15.154:0/4159701330 >> v2:172.21.15.190:3300/0 conn(0x7f239815ba50 0x7f239806fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:40.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.627+0000 7f239d9f4640 1 --2- 172.21.15.154:0/4159701330 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2398158480 0x7f23980cade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:40.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.627+0000 7f239d9f4640 1 -- 172.21.15.154:0/4159701330 >> 172.21.15.154:0/4159701330 conn(0x7f23980aff20 msgr2=0x7f23980b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:40.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.627+0000 7f239d9f4640 1 -- 172.21.15.154:0/4159701330 shutdown_connections 2023-09-23T03:16:40.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.627+0000 7f239d9f4640 1 -- 172.21.15.154:0/4159701330 wait complete. 2023-09-23T03:16:40.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.629+0000 7f239d9f4640 1 Processor -- start 2023-09-23T03:16:40.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.629+0000 7f239d9f4640 1 -- start start 2023-09-23T03:16:40.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.630+0000 7f239d9f4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2398158480 0x7f2398152710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:40.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.630+0000 7f239d9f4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f239815ba50 0x7f239815d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:40.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.630+0000 7f239d9f4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f239815eef0 0x7f239815d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:40.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.630+0000 7f239d9f4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23980c1f00 con 0x7f239815ba50 2023-09-23T03:16:40.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.630+0000 7f239d9f4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2398152ef0 con 0x7f2398158480 2023-09-23T03:16:40.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.630+0000 7f239d9f4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23980c8a20 con 0x7f239815eef0 2023-09-23T03:16:40.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.630+0000 7f2397fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f239815ba50 0x7f239815d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:40.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.630+0000 7f2397fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f239815ba50 0x7f239815d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52724/0 (socket says 172.21.15.154:52724) 2023-09-23T03:16:40.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.630+0000 7f2397fff640 1 -- 172.21.15.154:0/1288647127 learned_addr learned my addr 172.21.15.154:0/1288647127 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:40.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.631+0000 7f2397fff640 1 -- 172.21.15.154:0/1288647127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f239815eef0 msgr2=0x7f239815d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:40.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.631+0000 7f2397fff640 1 --2- 172.21.15.154:0/1288647127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f239815eef0 0x7f239815d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:40.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.631+0000 7f2397fff640 1 -- 172.21.15.154:0/1288647127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2398158480 msgr2=0x7f2398152710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:40.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.631+0000 7f2397fff640 1 --2- 172.21.15.154:0/1288647127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2398158480 0x7f2398152710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:40.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.631+0000 7f2397fff640 1 -- 172.21.15.154:0/1288647127 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2390009b30 con 0x7f239815ba50 2023-09-23T03:16:40.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.631+0000 7f2397fff640 1 --2- 172.21.15.154:0/1288647127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f239815ba50 0x7f239815d430 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f238400d5f0 tx=0x7f238400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:40.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.631+0000 7f23957fa640 1 -- 172.21.15.154:0/1288647127 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23840125f0 con 0x7f239815ba50 2023-09-23T03:16:40.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.631+0000 7f23957fa640 1 -- 172.21.15.154:0/1288647127 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2384012750 con 0x7f239815ba50 2023-09-23T03:16:40.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.631+0000 7f239d9f4640 1 -- 172.21.15.154:0/1288647127 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f239815d110 con 0x7f239815ba50 2023-09-23T03:16:40.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.631+0000 7f23957fa640 1 -- 172.21.15.154:0/1288647127 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2384012b00 con 0x7f239815ba50 2023-09-23T03:16:40.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.631+0000 7f239d9f4640 1 -- 172.21.15.154:0/1288647127 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2398153100 con 0x7f239815ba50 2023-09-23T03:16:40.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.632+0000 7f239d9f4640 1 -- 172.21.15.154:0/1288647127 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f239804e920 con 0x7f239815ba50 2023-09-23T03:16:40.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.633+0000 7f23957fa640 1 -- 172.21.15.154:0/1288647127 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f2384012c60 con 0x7f239815ba50 2023-09-23T03:16:40.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.634+0000 7f23957fa640 1 --2- 172.21.15.154:0/1288647127 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f237c06af30 0x7f237c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:40.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.634+0000 7f23957fa640 1 -- 172.21.15.154:0/1288647127 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f2384089f70 con 0x7f239815ba50 2023-09-23T03:16:40.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.634+0000 7f239c9f2640 1 --2- 172.21.15.154:0/1288647127 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f237c06af30 0x7f237c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:40.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.635+0000 7f239c9f2640 1 --2- 172.21.15.154:0/1288647127 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f237c06af30 0x7f237c06d3f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f238c006fd0 tx=0x7f238c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:40.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.638+0000 7f23957fa640 1 -- 172.21.15.154:0/1288647127 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f23840bca50 con 0x7f239815ba50 2023-09-23T03:16:40.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.812+0000 7f239d9f4640 1 -- 172.21.15.154:0/1288647127 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f23981594a0 con 0x7f239815ba50 2023-09-23T03:16:40.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.813+0000 7f23957fa640 1 -- 172.21.15.154:0/1288647127 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2384054c00 con 0x7f239815ba50 2023-09-23T03:16:40.814 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:16:40.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.815+0000 7f239d9f4640 1 -- 172.21.15.154:0/1288647127 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f237c06af30 msgr2=0x7f237c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:40.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.815+0000 7f239d9f4640 1 --2- 172.21.15.154:0/1288647127 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f237c06af30 0x7f237c06d3f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f238c006fd0 tx=0x7f238c008040 comp rx=0 tx=0).stop 2023-09-23T03:16:40.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.815+0000 7f239d9f4640 1 -- 172.21.15.154:0/1288647127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f239815ba50 msgr2=0x7f239815d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:40.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.815+0000 7f239d9f4640 1 --2- 172.21.15.154:0/1288647127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f239815ba50 0x7f239815d430 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f238400d5f0 tx=0x7f238400dab0 comp rx=0 tx=0).stop 2023-09-23T03:16:40.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.816+0000 7f239d9f4640 1 -- 172.21.15.154:0/1288647127 shutdown_connections 2023-09-23T03:16:40.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.816+0000 7f239d9f4640 1 --2- 172.21.15.154:0/1288647127 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f237c06af30 0x7f237c06d3f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:40.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.816+0000 7f239d9f4640 1 --2- 172.21.15.154:0/1288647127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f239815eef0 0x7f239815d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:40.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.816+0000 7f239d9f4640 1 --2- 172.21.15.154:0/1288647127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f239815ba50 0x7f239815d430 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:40.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.816+0000 7f239d9f4640 1 --2- 172.21.15.154:0/1288647127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2398158480 0x7f2398152710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:40.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.816+0000 7f239d9f4640 1 -- 172.21.15.154:0/1288647127 >> 172.21.15.154:0/1288647127 conn(0x7f23980aff20 msgr2=0x7f239806fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:40.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.816+0000 7f239d9f4640 1 -- 172.21.15.154:0/1288647127 shutdown_connections 2023-09-23T03:16:40.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:40.816+0000 7f239d9f4640 1 -- 172.21.15.154:0/1288647127 wait complete. 2023-09-23T03:16:40.835 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-23T03:16:40.951 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:16:40.956 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:16:41.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.047+0000 7ffb1c3b6640 1 Processor -- start 2023-09-23T03:16:41.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.047+0000 7ffb1c3b6640 1 -- start start 2023-09-23T03:16:41.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.049+0000 7ffb1c3b6640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb1415cab0 0x7ffb140cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:41.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.049+0000 7ffb1c3b6640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ffb1415ce90 0x7ffb140cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:41.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.049+0000 7ffb1c3b6640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb140d18e0 con 0x7ffb140d0bd0 2023-09-23T03:16:41.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.049+0000 7ffb1c3b6640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb140cfc90 con 0x7ffb1415ce90 2023-09-23T03:16:41.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.049+0000 7ffb1c3b6640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb140d1bc0 con 0x7ffb1415cab0 2023-09-23T03:16:41.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.049+0000 7ffb1a953640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ffb140d0bd0 0x7ffb140d1080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:42116/0 (socket says 172.21.15.154:42116) 2023-09-23T03:16:41.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.049+0000 7ffb1a953640 1 -- 172.21.15.154:0/4294864160 learned_addr learned my addr 172.21.15.154:0/4294864160 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:41.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.049+0000 7ffb1a152640 1 --2- 172.21.15.154:0/4294864160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb1415cab0 0x7ffb140cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:41.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.049+0000 7ffb19951640 1 --2- 172.21.15.154:0/4294864160 >> v2:172.21.15.190:3300/0 conn(0x7ffb1415ce90 0x7ffb140cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:41.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.050+0000 7ffb19150640 1 -- 172.21.15.154:0/4294864160 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3084752076 0 0) 0x7ffb140d18e0 con 0x7ffb140d0bd0 2023-09-23T03:16:41.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.050+0000 7ffb19150640 1 -- 172.21.15.154:0/4294864160 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb00003520 con 0x7ffb140d0bd0 2023-09-23T03:16:41.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.050+0000 7ffb19150640 1 -- 172.21.15.154:0/4294864160 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 690167153 0 0) 0x7ffb00003520 con 0x7ffb140d0bd0 2023-09-23T03:16:41.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.050+0000 7ffb19150640 1 -- 172.21.15.154:0/4294864160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb1415cab0 msgr2=0x7ffb140cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:41.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.050+0000 7ffb19150640 1 --2- 172.21.15.154:0/4294864160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb1415cab0 0x7ffb140cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:41.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.050+0000 7ffb19150640 1 -- 172.21.15.154:0/4294864160 >> v2:172.21.15.190:3300/0 conn(0x7ffb1415ce90 msgr2=0x7ffb140cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:41.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.050+0000 7ffb19150640 1 --2- 172.21.15.154:0/4294864160 >> v2:172.21.15.190:3300/0 conn(0x7ffb1415ce90 0x7ffb140cf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:41.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.050+0000 7ffb19150640 1 -- 172.21.15.154:0/4294864160 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb140d1ea0 con 0x7ffb140d0bd0 2023-09-23T03:16:41.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.050+0000 7ffb19150640 1 -- 172.21.15.154:0/4294864160 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7ffb0c0019f0 con 0x7ffb140d0bd0 2023-09-23T03:16:41.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.050+0000 7ffb19150640 1 -- 172.21.15.154:0/4294864160 >> v1:172.21.15.154:6789/0 conn(0x7ffb140d0bd0 legacy=0x7ffb140d1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:41.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.050+0000 7ffb19150640 1 --2- 172.21.15.154:0/4294864160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb1415cab0 0x7ffb00003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:41.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.050+0000 7ffb19150640 1 --2- 172.21.15.154:0/4294864160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb00003ef0 0x7ffb000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:41.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.050+0000 7ffb19150640 1 --2- 172.21.15.154:0/4294864160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb00004d00 0x7ffb00005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:41.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.050+0000 7ffb19150640 1 -- 172.21.15.154:0/4294864160 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb140d1bc0 con 0x7ffb00004d00 2023-09-23T03:16:41.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.050+0000 7ffb19150640 1 -- 172.21.15.154:0/4294864160 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0000b980 con 0x7ffb00003ef0 2023-09-23T03:16:41.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.050+0000 7ffb19150640 1 -- 172.21.15.154:0/4294864160 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0000bac0 con 0x7ffb1415cab0 2023-09-23T03:16:41.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.050+0000 7ffb1a953640 1 --2- 172.21.15.154:0/4294864160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb00004d00 0x7ffb00005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:41.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.051+0000 7ffb1a152640 1 --2- 172.21.15.154:0/4294864160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb1415cab0 0x7ffb00003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:41.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.051+0000 7ffb19951640 1 --2- 172.21.15.154:0/4294864160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb00003ef0 0x7ffb000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:41.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.051+0000 7ffb1a953640 1 -- 172.21.15.154:0/4294864160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb1415cab0 msgr2=0x7ffb00003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:41.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.051+0000 7ffb1a953640 1 --2- 172.21.15.154:0/4294864160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb1415cab0 0x7ffb00003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:41.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.051+0000 7ffb1a953640 1 -- 172.21.15.154:0/4294864160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb00003ef0 msgr2=0x7ffb000047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:41.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.051+0000 7ffb1a953640 1 --2- 172.21.15.154:0/4294864160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb00003ef0 0x7ffb000047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:41.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.051+0000 7ffb1a953640 1 -- 172.21.15.154:0/4294864160 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb00003520 con 0x7ffb00004d00 2023-09-23T03:16:41.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.051+0000 7ffb19951640 1 --2- 172.21.15.154:0/4294864160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb00003ef0 0x7ffb000047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:41.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.052+0000 7ffb1a953640 1 --2- 172.21.15.154:0/4294864160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb00004d00 0x7ffb00005180 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00a860 tx=0x7ffb000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf128cd3b9700277 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:41.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.052+0000 7ffb19150640 1 -- 172.21.15.154:0/4294864160 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c034630 con 0x7ffb00004d00 2023-09-23T03:16:41.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.052+0000 7ffb19150640 1 -- 172.21.15.154:0/4294864160 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb0c02f070 con 0x7ffb00004d00 2023-09-23T03:16:41.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.052+0000 7ffb19150640 1 -- 172.21.15.154:0/4294864160 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c034a30 con 0x7ffb00004d00 2023-09-23T03:16:41.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.053+0000 7ffb1c3b6640 1 -- 172.21.15.154:0/4294864160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb00004d00 msgr2=0x7ffb00005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:41.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.053+0000 7ffb1c3b6640 1 --2- 172.21.15.154:0/4294864160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb00004d00 0x7ffb00005180 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00a860 tx=0x7ffb000057f0 comp rx=0 tx=0).stop 2023-09-23T03:16:41.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.053+0000 7ffb1a953640 1 -- 172.21.15.154:0/4294864160 reap_dead start 2023-09-23T03:16:41.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.053+0000 7ffb1c3b6640 1 -- 172.21.15.154:0/4294864160 shutdown_connections 2023-09-23T03:16:41.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.053+0000 7ffb1c3b6640 1 -- 172.21.15.154:0/4294864160 >> 172.21.15.154:0/4294864160 conn(0x7ffb14071bc0 msgr2=0x7ffb140b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:41.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.053+0000 7ffb1c3b6640 1 -- 172.21.15.154:0/4294864160 shutdown_connections 2023-09-23T03:16:41.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.053+0000 7ffb1c3b6640 1 -- 172.21.15.154:0/4294864160 wait complete. 2023-09-23T03:16:41.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.055+0000 7ffb1c3b6640 1 Processor -- start 2023-09-23T03:16:41.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.055+0000 7ffb1c3b6640 1 -- start start 2023-09-23T03:16:41.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.056+0000 7ffb1c3b6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb00004d00 0x7ffb140b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:41.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.056+0000 7ffb1c3b6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb141f2df0 0x7ffb1415f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:41.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.056+0000 7ffb1c3b6640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb1415f9d0 0x7ffb141f5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:41.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.056+0000 7ffb1c3b6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1415fe50 con 0x7ffb00004d00 2023-09-23T03:16:41.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.056+0000 7ffb1c3b6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb140c9dc0 con 0x7ffb141f2df0 2023-09-23T03:16:41.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.056+0000 7ffb1c3b6640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb140ca0a0 con 0x7ffb1415f9d0 2023-09-23T03:16:41.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.056+0000 7ffb1a152640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb00004d00 0x7ffb140b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:41.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.056+0000 7ffb1a152640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb00004d00 0x7ffb140b1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52746/0 (socket says 172.21.15.154:52746) 2023-09-23T03:16:41.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.056+0000 7ffb1a152640 1 -- 172.21.15.154:0/2290550964 learned_addr learned my addr 172.21.15.154:0/2290550964 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:41.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.056+0000 7ffb1a953640 1 --2- 172.21.15.154:0/2290550964 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb1415f9d0 0x7ffb141f5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:41.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.057+0000 7ffb19951640 1 --2- 172.21.15.154:0/2290550964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb141f2df0 0x7ffb1415f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:41.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.057+0000 7ffb1a152640 1 -- 172.21.15.154:0/2290550964 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb1415f9d0 msgr2=0x7ffb141f5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:41.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.057+0000 7ffb1a152640 1 --2- 172.21.15.154:0/2290550964 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb1415f9d0 0x7ffb141f5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:41.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.057+0000 7ffb1a152640 1 -- 172.21.15.154:0/2290550964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb141f2df0 msgr2=0x7ffb1415f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:41.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.057+0000 7ffb1a152640 1 --2- 172.21.15.154:0/2290550964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb141f2df0 0x7ffb1415f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:41.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.057+0000 7ffb1a152640 1 -- 172.21.15.154:0/2290550964 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb140ca380 con 0x7ffb00004d00 2023-09-23T03:16:41.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.057+0000 7ffb1a152640 1 --2- 172.21.15.154:0/2290550964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb00004d00 0x7ffb140b1d60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ffb10011600 tx=0x7ffb10011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:41.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.057+0000 7ffb0affd640 1 -- 172.21.15.154:0/2290550964 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1000f920 con 0x7ffb00004d00 2023-09-23T03:16:41.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.057+0000 7ffb0affd640 1 -- 172.21.15.154:0/2290550964 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb1000fa80 con 0x7ffb00004d00 2023-09-23T03:16:41.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.057+0000 7ffb1c3b6640 1 -- 172.21.15.154:0/2290550964 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb140ca660 con 0x7ffb00004d00 2023-09-23T03:16:41.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.057+0000 7ffb0affd640 1 -- 172.21.15.154:0/2290550964 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1000fdf0 con 0x7ffb00004d00 2023-09-23T03:16:41.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.057+0000 7ffb1c3b6640 1 -- 172.21.15.154:0/2290550964 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb141f3270 con 0x7ffb00004d00 2023-09-23T03:16:41.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.059+0000 7ffb1c3b6640 1 -- 172.21.15.154:0/2290550964 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb14073eb0 con 0x7ffb00004d00 2023-09-23T03:16:41.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.060+0000 7ffb0affd640 1 -- 172.21.15.154:0/2290550964 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7ffb10015460 con 0x7ffb00004d00 2023-09-23T03:16:41.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.061+0000 7ffb0affd640 1 --2- 172.21.15.154:0/2290550964 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffaf806af60 0x7ffaf806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:41.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.061+0000 7ffb0affd640 1 -- 172.21.15.154:0/2290550964 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7ffb1008a480 con 0x7ffb00004d00 2023-09-23T03:16:41.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.061+0000 7ffb19951640 1 --2- 172.21.15.154:0/2290550964 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffaf806af60 0x7ffaf806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:41.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.062+0000 7ffb19951640 1 --2- 172.21.15.154:0/2290550964 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffaf806af60 0x7ffaf806d420 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ffb141602c0 tx=0x7ffb04008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:41.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.065+0000 7ffb0affd640 1 -- 172.21.15.154:0/2290550964 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7ffb10055070 con 0x7ffb00004d00 2023-09-23T03:16:41.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.275+0000 7ffb1c3b6640 1 -- 172.21.15.154:0/2290550964 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ffb140cba70 con 0x7ffb00004d00 2023-09-23T03:16:41.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.276+0000 7ffb0affd640 1 -- 172.21.15.154:0/2290550964 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7ffb10058690 con 0x7ffb00004d00 2023-09-23T03:16:41.277 INFO:teuthology.orchestra.run.smithi154.stderr:listed 0 entries 2023-09-23T03:16:41.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.279+0000 7ffb1c3b6640 1 -- 172.21.15.154:0/2290550964 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffaf806af60 msgr2=0x7ffaf806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:41.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.279+0000 7ffb1c3b6640 1 --2- 172.21.15.154:0/2290550964 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffaf806af60 0x7ffaf806d420 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ffb141602c0 tx=0x7ffb04008040 comp rx=0 tx=0).stop 2023-09-23T03:16:41.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.279+0000 7ffb1c3b6640 1 -- 172.21.15.154:0/2290550964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb00004d00 msgr2=0x7ffb140b1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:41.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.279+0000 7ffb1c3b6640 1 --2- 172.21.15.154:0/2290550964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb00004d00 0x7ffb140b1d60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ffb10011600 tx=0x7ffb10011ac0 comp rx=0 tx=0).stop 2023-09-23T03:16:41.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.279+0000 7ffb1c3b6640 1 -- 172.21.15.154:0/2290550964 shutdown_connections 2023-09-23T03:16:41.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.279+0000 7ffb1c3b6640 1 --2- 172.21.15.154:0/2290550964 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffaf806af60 0x7ffaf806d420 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:41.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.279+0000 7ffb1c3b6640 1 --2- 172.21.15.154:0/2290550964 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb1415f9d0 0x7ffb141f5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:41.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.279+0000 7ffb1c3b6640 1 --2- 172.21.15.154:0/2290550964 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb141f2df0 0x7ffb1415f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:41.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.279+0000 7ffb1c3b6640 1 --2- 172.21.15.154:0/2290550964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb00004d00 0x7ffb140b1d60 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:41.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.279+0000 7ffb1c3b6640 1 -- 172.21.15.154:0/2290550964 >> 172.21.15.154:0/2290550964 conn(0x7ffb14071bc0 msgr2=0x7ffb140728b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:41.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.279+0000 7ffb1c3b6640 1 -- 172.21.15.154:0/2290550964 shutdown_connections 2023-09-23T03:16:41.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.279+0000 7ffb1c3b6640 1 -- 172.21.15.154:0/2290550964 wait complete. 2023-09-23T03:16:41.299 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi190.front.sepia.ceph.com... 2023-09-23T03:16:41.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:41.300 DEBUG:teuthology.orchestra.run.smithi190:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-09-23T03:16:41.408 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:16:41.408 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi190.front.sepia.ceph.com 2023-09-23T03:16:41.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:41.409 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-09-23T03:16:41.465 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:16:41.465 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:16:41.524 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:16:41.525 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:16:41.578 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:16:41.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.793+0000 7fb1b759e640 1 Processor -- start 2023-09-23T03:16:41.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.793+0000 7fb1b759e640 1 -- start start 2023-09-23T03:16:41.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.795+0000 7fb1b759e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1b80cacd0 0x7fb1b80cb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:41.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.795+0000 7fb1b759e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1b80cc910 con 0x7fb1b815c840 2023-09-23T03:16:41.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.795+0000 7fb1b759e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1b80c89b0 con 0x7fb1b8160fd0 2023-09-23T03:16:41.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.795+0000 7fb1b759e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b80cb6c0 con 0x7fb1b80cacd0 2023-09-23T03:16:41.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.795+0000 7fb1b659c640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fb1b815c840 0x7fb1b80cbfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:42132/0 (socket says 172.21.15.154:42132) 2023-09-23T03:16:41.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.795+0000 7fb1b659c640 1 -- 172.21.15.154:0/1370616742 learned_addr learned my addr 172.21.15.154:0/1370616742 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:41.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.796+0000 7fb1b6d9d640 1 --2- 172.21.15.154:0/1370616742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1b80cacd0 0x7fb1b80cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:41.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.796+0000 7fb1b559a640 1 -- 172.21.15.154:0/1370616742 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3051828728 0 0) 0x7fb1b80cc910 con 0x7fb1b815c840 2023-09-23T03:16:41.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.796+0000 7fb1b559a640 1 -- 172.21.15.154:0/1370616742 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1a4003520 con 0x7fb1b815c840 2023-09-23T03:16:41.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.796+0000 7fb1b559a640 1 -- 172.21.15.154:0/1370616742 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 733183912 0 0) 0x7fb1a4003520 con 0x7fb1b815c840 2023-09-23T03:16:41.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.796+0000 7fb1b559a640 1 -- 172.21.15.154:0/1370616742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1b80cacd0 msgr2=0x7fb1b80cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:41.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.796+0000 7fb1b559a640 1 --2- 172.21.15.154:0/1370616742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1b80cacd0 0x7fb1b80cb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:41.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.796+0000 7fb1b559a640 1 -- 172.21.15.154:0/1370616742 >> v1:172.21.15.190:6789/0 conn(0x7fb1b8160fd0 legacy=0x7fb1b80c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:41.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.796+0000 7fb1b559a640 1 -- 172.21.15.154:0/1370616742 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b80c8c90 con 0x7fb1b815c840 2023-09-23T03:16:41.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.796+0000 7fb1b559a640 1 -- 172.21.15.154:0/1370616742 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fb1a80019f0 con 0x7fb1b815c840 2023-09-23T03:16:41.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.796+0000 7fb1b559a640 1 -- 172.21.15.154:0/1370616742 >> v1:172.21.15.154:6789/0 conn(0x7fb1b815c840 legacy=0x7fb1b80cbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:41.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.796+0000 7fb1b559a640 1 --2- 172.21.15.154:0/1370616742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1a40039a0 0x7fb1a4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:41.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.796+0000 7fb1b559a640 1 --2- 172.21.15.154:0/1370616742 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1a40047c0 0x7fb1a4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:41.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.796+0000 7fb1b559a640 1 --2- 172.21.15.154:0/1370616742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1b80cacd0 0x7fb1a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:41.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.796+0000 7fb1b559a640 1 -- 172.21.15.154:0/1370616742 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b80cb6c0 con 0x7fb1a40047c0 2023-09-23T03:16:41.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.796+0000 7fb1b559a640 1 -- 172.21.15.154:0/1370616742 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a400bae0 con 0x7fb1a40039a0 2023-09-23T03:16:41.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.796+0000 7fb1b559a640 1 -- 172.21.15.154:0/1370616742 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a400bca0 con 0x7fb1b80cacd0 2023-09-23T03:16:41.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.797+0000 7fb1b5d9b640 1 --2- 172.21.15.154:0/1370616742 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1a40047c0 0x7fb1a4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:41.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.797+0000 7fb1b659c640 1 --2- 172.21.15.154:0/1370616742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1a40039a0 0x7fb1a4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:41.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.797+0000 7fb1b6d9d640 1 --2- 172.21.15.154:0/1370616742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1b80cacd0 0x7fb1a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:41.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.797+0000 7fb1b5d9b640 1 -- 172.21.15.154:0/1370616742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1b80cacd0 msgr2=0x7fb1a40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:41.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.797+0000 7fb1b5d9b640 1 --2- 172.21.15.154:0/1370616742 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1b80cacd0 0x7fb1a40051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:41.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.797+0000 7fb1b5d9b640 1 -- 172.21.15.154:0/1370616742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1a40039a0 msgr2=0x7fb1a4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:41.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.797+0000 7fb1b5d9b640 1 --2- 172.21.15.154:0/1370616742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1a40039a0 0x7fb1a4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:41.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.797+0000 7fb1b5d9b640 1 -- 172.21.15.154:0/1370616742 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a4003520 con 0x7fb1a40047c0 2023-09-23T03:16:41.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.797+0000 7fb1b659c640 1 --2- 172.21.15.154:0/1370616742 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1a40039a0 0x7fb1a4003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:41.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.798+0000 7fb1b5d9b640 1 --2- 172.21.15.154:0/1370616742 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1a40047c0 0x7fb1a4004c40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0001720 tx=0x7fb1a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff439d1362f03a1c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:41.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.799+0000 7fb1b559a640 1 -- 172.21.15.154:0/1370616742 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a00052d0 con 0x7fb1a40047c0 2023-09-23T03:16:41.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.799+0000 7fb1b559a640 1 -- 172.21.15.154:0/1370616742 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a002e050 con 0x7fb1a40047c0 2023-09-23T03:16:41.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.799+0000 7fb1b559a640 1 -- 172.21.15.154:0/1370616742 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a00035f0 con 0x7fb1a40047c0 2023-09-23T03:16:41.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.799+0000 7fb1b759e640 1 -- 172.21.15.154:0/1370616742 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1a40047c0 msgr2=0x7fb1a4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:41.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.799+0000 7fb1b759e640 1 --2- 172.21.15.154:0/1370616742 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1a40047c0 0x7fb1a4004c40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0001720 tx=0x7fb1a40057f0 comp rx=0 tx=0).stop 2023-09-23T03:16:41.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.799+0000 7fb1b6d9d640 1 -- 172.21.15.154:0/1370616742 reap_dead start 2023-09-23T03:16:41.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.800+0000 7fb1b759e640 1 -- 172.21.15.154:0/1370616742 shutdown_connections 2023-09-23T03:16:41.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.800+0000 7fb1b759e640 1 -- 172.21.15.154:0/1370616742 >> 172.21.15.154:0/1370616742 conn(0x7fb1b806a390 msgr2=0x7fb1b80afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:41.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.800+0000 7fb1b759e640 1 -- 172.21.15.154:0/1370616742 shutdown_connections 2023-09-23T03:16:41.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.800+0000 7fb1b759e640 1 -- 172.21.15.154:0/1370616742 wait complete. 2023-09-23T03:16:41.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.801+0000 7fb1b759e640 1 Processor -- start 2023-09-23T03:16:41.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.802+0000 7fb1b759e640 1 -- start start 2023-09-23T03:16:41.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.802+0000 7fb1b759e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1a40047c0 0x7fb1b8157530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:41.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.803+0000 7fb1b759e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1b80b10f0 0x7fb1b80b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:41.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.803+0000 7fb1b759e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1b80aedb0 0x7fb1b80af230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:41.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.803+0000 7fb1b759e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b80af770 con 0x7fb1b80aedb0 2023-09-23T03:16:41.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.803+0000 7fb1b759e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b8157a70 con 0x7fb1b80b10f0 2023-09-23T03:16:41.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.803+0000 7fb1b759e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b815cda0 con 0x7fb1a40047c0 2023-09-23T03:16:41.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.803+0000 7fb1b6d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1b80aedb0 0x7fb1b80af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:41.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.803+0000 7fb1b6d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1b80aedb0 0x7fb1b80af230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52762/0 (socket says 172.21.15.154:52762) 2023-09-23T03:16:41.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.803+0000 7fb1b6d9d640 1 -- 172.21.15.154:0/2409364014 learned_addr learned my addr 172.21.15.154:0/2409364014 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:41.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.803+0000 7fb1b659c640 1 --2- 172.21.15.154:0/2409364014 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1a40047c0 0x7fb1b8157530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:41.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.803+0000 7fb1b5d9b640 1 --2- 172.21.15.154:0/2409364014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1b80b10f0 0x7fb1b80b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:41.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.803+0000 7fb1b6d9d640 1 -- 172.21.15.154:0/2409364014 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1a40047c0 msgr2=0x7fb1b8157530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:41.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.803+0000 7fb1b6d9d640 1 --2- 172.21.15.154:0/2409364014 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1a40047c0 0x7fb1b8157530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:41.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.803+0000 7fb1b6d9d640 1 -- 172.21.15.154:0/2409364014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1b80b10f0 msgr2=0x7fb1b80b39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:41.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.803+0000 7fb1b6d9d640 1 --2- 172.21.15.154:0/2409364014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1b80b10f0 0x7fb1b80b39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:41.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.803+0000 7fb1b6d9d640 1 -- 172.21.15.154:0/2409364014 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b815d080 con 0x7fb1b80aedb0 2023-09-23T03:16:41.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.803+0000 7fb1b659c640 1 --2- 172.21.15.154:0/2409364014 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1a40047c0 0x7fb1b8157530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:41.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.803+0000 7fb1b5d9b640 1 --2- 172.21.15.154:0/2409364014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1b80b10f0 0x7fb1b80b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:41.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.804+0000 7fb1b6d9d640 1 --2- 172.21.15.154:0/2409364014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1b80aedb0 0x7fb1b80af230 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb1b80cbf40 tx=0x7fb1b000e8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:41.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.804+0000 7fb1aeffd640 1 -- 172.21.15.154:0/2409364014 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b000c070 con 0x7fb1b80aedb0 2023-09-23T03:16:41.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.804+0000 7fb1aeffd640 1 -- 172.21.15.154:0/2409364014 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b0013610 con 0x7fb1b80aedb0 2023-09-23T03:16:41.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.804+0000 7fb1aeffd640 1 -- 172.21.15.154:0/2409364014 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b00139c0 con 0x7fb1b80aedb0 2023-09-23T03:16:41.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.804+0000 7fb1b759e640 1 -- 172.21.15.154:0/2409364014 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1b815d360 con 0x7fb1b80aedb0 2023-09-23T03:16:41.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.804+0000 7fb1b759e640 1 -- 172.21.15.154:0/2409364014 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1b815db60 con 0x7fb1b80aedb0 2023-09-23T03:16:41.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.805+0000 7fb1b759e640 1 -- 172.21.15.154:0/2409364014 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1b80736b0 con 0x7fb1b80aedb0 2023-09-23T03:16:41.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.806+0000 7fb1aeffd640 1 -- 172.21.15.154:0/2409364014 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fb1b0013b20 con 0x7fb1b80aedb0 2023-09-23T03:16:41.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.808+0000 7fb1aeffd640 1 --2- 172.21.15.154:0/2409364014 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb19806af30 0x7fb19806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:41.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.808+0000 7fb1aeffd640 1 -- 172.21.15.154:0/2409364014 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7fb1b00892a0 con 0x7fb1b80aedb0 2023-09-23T03:16:41.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.808+0000 7fb1b659c640 1 --2- 172.21.15.154:0/2409364014 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb19806af30 0x7fb19806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:41.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.809+0000 7fb1b659c640 1 --2- 172.21.15.154:0/2409364014 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb19806af30 0x7fb19806d3f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8001580 tx=0x7fb1a8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:41.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:41.812+0000 7fb1aeffd640 1 -- 172.21.15.154:0/2409364014 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fb1b0053f30 con 0x7fb1b80aedb0 2023-09-23T03:16:42.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.007+0000 7fb1b759e640 1 -- 172.21.15.154:0/2409364014 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1b80cc670 con 0x7fb1b80aedb0 2023-09-23T03:16:42.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.009+0000 7fb1aeffd640 1 -- 172.21.15.154:0/2409364014 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18744 (secure 0 0 0) 0x7fb1b0057550 con 0x7fb1b80aedb0 2023-09-23T03:16:42.011 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:16:42.011 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":26,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:16:03.151371+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:15:18.575849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T03:15:19.707212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T03:15:23.337138+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:16:42.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.012+0000 7fb1b759e640 1 -- 172.21.15.154:0/2409364014 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb19806af30 msgr2=0x7fb19806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:42.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.012+0000 7fb1b759e640 1 --2- 172.21.15.154:0/2409364014 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb19806af30 0x7fb19806d3f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8001580 tx=0x7fb1a8009040 comp rx=0 tx=0).stop 2023-09-23T03:16:42.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.012+0000 7fb1b759e640 1 -- 172.21.15.154:0/2409364014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1b80aedb0 msgr2=0x7fb1b80af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:42.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.012+0000 7fb1b759e640 1 --2- 172.21.15.154:0/2409364014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1b80aedb0 0x7fb1b80af230 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb1b80cbf40 tx=0x7fb1b000e8b0 comp rx=0 tx=0).stop 2023-09-23T03:16:42.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.012+0000 7fb1b759e640 1 -- 172.21.15.154:0/2409364014 shutdown_connections 2023-09-23T03:16:42.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.012+0000 7fb1b759e640 1 --2- 172.21.15.154:0/2409364014 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb19806af30 0x7fb19806d3f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:42.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.012+0000 7fb1b759e640 1 --2- 172.21.15.154:0/2409364014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1b80aedb0 0x7fb1b80af230 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:42.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.012+0000 7fb1b759e640 1 --2- 172.21.15.154:0/2409364014 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1b80b10f0 0x7fb1b80b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:42.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.012+0000 7fb1b759e640 1 --2- 172.21.15.154:0/2409364014 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1a40047c0 0x7fb1b8157530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:42.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.012+0000 7fb1b759e640 1 -- 172.21.15.154:0/2409364014 >> 172.21.15.154:0/2409364014 conn(0x7fb1b806a390 msgr2=0x7fb1b80afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:42.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.012+0000 7fb1b759e640 1 -- 172.21.15.154:0/2409364014 shutdown_connections 2023-09-23T03:16:42.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.012+0000 7fb1b759e640 1 -- 172.21.15.154:0/2409364014 wait complete. 2023-09-23T03:16:42.033 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:15:18.575849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:15:19.707212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-23T03:15:23.337138+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-23T03:16:42.033 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:16:42.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.255+0000 7f4c9794c640 1 Processor -- start 2023-09-23T03:16:42.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.255+0000 7f4c9794c640 1 -- start start 2023-09-23T03:16:42.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.257+0000 7f4c9794c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4c9015abe0 0x7f4c900cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:42.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.257+0000 7f4c9794c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c900ce510 con 0x7f4c900cc9b0 2023-09-23T03:16:42.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.257+0000 7f4c9794c640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c900cb710 con 0x7f4c900b28c0 2023-09-23T03:16:42.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.257+0000 7f4c9794c640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c900cd390 con 0x7f4c9015abe0 2023-09-23T03:16:42.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.258+0000 7f4c94ee7640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f4c900cc9b0 0x7f4c900b21b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:42146/0 (socket says 172.21.15.154:42146) 2023-09-23T03:16:42.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.258+0000 7f4c94ee7640 1 -- 172.21.15.154:0/268760724 learned_addr learned my addr 172.21.15.154:0/268760724 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:42.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.258+0000 7f4c956e8640 1 --2- 172.21.15.154:0/268760724 >> v2:172.21.15.190:3300/0 conn(0x7f4c9015abe0 0x7f4c900cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:42.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.258+0000 7f4c87fff640 1 -- 172.21.15.154:0/268760724 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2006121690 0 0) 0x7f4c900ce510 con 0x7f4c900cc9b0 2023-09-23T03:16:42.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.258+0000 7f4c87fff640 1 -- 172.21.15.154:0/268760724 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c7c003520 con 0x7f4c900cc9b0 2023-09-23T03:16:42.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.259+0000 7f4c87fff640 1 -- 172.21.15.154:0/268760724 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3964754388 0 0) 0x7f4c7c003520 con 0x7f4c900cc9b0 2023-09-23T03:16:42.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.259+0000 7f4c87fff640 1 -- 172.21.15.154:0/268760724 >> v2:172.21.15.190:3300/0 conn(0x7f4c9015abe0 msgr2=0x7f4c900cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:42.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.259+0000 7f4c87fff640 1 --2- 172.21.15.154:0/268760724 >> v2:172.21.15.190:3300/0 conn(0x7f4c9015abe0 0x7f4c900cb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:42.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.259+0000 7f4c87fff640 1 -- 172.21.15.154:0/268760724 >> v1:172.21.15.190:6789/0 conn(0x7f4c900b28c0 legacy=0x7f4c900ce9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:42.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.259+0000 7f4c87fff640 1 -- 172.21.15.154:0/268760724 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c900cd670 con 0x7f4c900cc9b0 2023-09-23T03:16:42.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.259+0000 7f4c87fff640 1 -- 172.21.15.154:0/268760724 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f4c80001670 con 0x7f4c900cc9b0 2023-09-23T03:16:42.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.259+0000 7f4c87fff640 1 -- 172.21.15.154:0/268760724 >> v1:172.21.15.154:6789/0 conn(0x7f4c900cc9b0 legacy=0x7f4c900b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:42.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.259+0000 7f4c87fff640 1 --2- 172.21.15.154:0/268760724 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c7c003970 0x7f4c7c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:42.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.259+0000 7f4c87fff640 1 --2- 172.21.15.154:0/268760724 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c7c0047c0 0x7f4c7c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:42.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.259+0000 7f4c87fff640 1 --2- 172.21.15.154:0/268760724 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c7c005180 0x7f4c7c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:42.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.259+0000 7f4c87fff640 1 -- 172.21.15.154:0/268760724 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c900cd390 con 0x7f4c7c003970 2023-09-23T03:16:42.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.259+0000 7f4c87fff640 1 -- 172.21.15.154:0/268760724 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c00c120 con 0x7f4c7c0047c0 2023-09-23T03:16:42.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.259+0000 7f4c87fff640 1 -- 172.21.15.154:0/268760724 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c00c2e0 con 0x7f4c7c005180 2023-09-23T03:16:42.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.259+0000 7f4c956e8640 1 --2- 172.21.15.154:0/268760724 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c7c003970 0x7f4c7c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:42.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.259+0000 7f4c94ee7640 1 --2- 172.21.15.154:0/268760724 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c7c0047c0 0x7f4c7c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:42.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.259+0000 7f4c956e8640 1 -- 172.21.15.154:0/268760724 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c7c005180 msgr2=0x7f4c7c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:42.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.259+0000 7f4c956e8640 1 --2- 172.21.15.154:0/268760724 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c7c005180 0x7f4c7c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:42.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.259+0000 7f4c956e8640 1 -- 172.21.15.154:0/268760724 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c7c0047c0 msgr2=0x7f4c7c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:42.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.259+0000 7f4c956e8640 1 --2- 172.21.15.154:0/268760724 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c7c0047c0 0x7f4c7c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:42.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.259+0000 7f4c956e8640 1 -- 172.21.15.154:0/268760724 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c7c003520 con 0x7f4c7c003970 2023-09-23T03:16:42.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.260+0000 7f4c94ee7640 1 --2- 172.21.15.154:0/268760724 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c7c0047c0 0x7f4c7c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:42.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.260+0000 7f4c95ee9640 1 -- 172.21.15.154:0/268760724 reap_dead start 2023-09-23T03:16:42.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.261+0000 7f4c956e8640 1 --2- 172.21.15.154:0/268760724 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c7c003970 0x7f4c7c003d90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c009ab0 tx=0x7f4c7c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57fa22d76e23b9a2 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:42.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.261+0000 7f4c87fff640 1 -- 172.21.15.154:0/268760724 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c005990 con 0x7f4c7c003970 2023-09-23T03:16:42.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.261+0000 7f4c87fff640 1 -- 172.21.15.154:0/268760724 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8c005af0 con 0x7f4c7c003970 2023-09-23T03:16:42.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.261+0000 7f4c87fff640 1 -- 172.21.15.154:0/268760724 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c005ea0 con 0x7f4c7c003970 2023-09-23T03:16:42.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.261+0000 7f4c9794c640 1 -- 172.21.15.154:0/268760724 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c7c003970 msgr2=0x7f4c7c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:42.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.261+0000 7f4c9794c640 1 --2- 172.21.15.154:0/268760724 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c7c003970 0x7f4c7c003d90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c009ab0 tx=0x7f4c7c0057f0 comp rx=0 tx=0).stop 2023-09-23T03:16:42.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.261+0000 7f4c9794c640 1 -- 172.21.15.154:0/268760724 shutdown_connections 2023-09-23T03:16:42.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.261+0000 7f4c9794c640 1 --2- 172.21.15.154:0/268760724 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c7c003970 0x7f4c7c003d90 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:42.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.261+0000 7f4c9794c640 1 -- 172.21.15.154:0/268760724 >> 172.21.15.154:0/268760724 conn(0x7f4c9006fc40 msgr2=0x7f4c90070020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:42.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.261+0000 7f4c9794c640 1 -- 172.21.15.154:0/268760724 shutdown_connections 2023-09-23T03:16:42.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.262+0000 7f4c9794c640 1 -- 172.21.15.154:0/268760724 wait complete. 2023-09-23T03:16:42.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.263+0000 7f4c9794c640 1 Processor -- start 2023-09-23T03:16:42.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.264+0000 7f4c9794c640 1 -- start start 2023-09-23T03:16:42.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.264+0000 7f4c9794c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c7c003970 0x7f4c900b0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:42.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.264+0000 7f4c9794c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c900c7800 0x7f4c901498d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:42.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.265+0000 7f4c9794c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c900c7c80 0x7f4c901f19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:42.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.265+0000 7f4c9794c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c900b1b90 con 0x7f4c7c003970 2023-09-23T03:16:42.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.265+0000 7f4c9794c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c900b1010 con 0x7f4c900c7c80 2023-09-23T03:16:42.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.265+0000 7f4c9794c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c900b12f0 con 0x7f4c900c7800 2023-09-23T03:16:42.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.265+0000 7f4c956e8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c7c003970 0x7f4c900b0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:42.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.265+0000 7f4c956e8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c7c003970 0x7f4c900b0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52782/0 (socket says 172.21.15.154:52782) 2023-09-23T03:16:42.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.265+0000 7f4c956e8640 1 -- 172.21.15.154:0/3545372027 learned_addr learned my addr 172.21.15.154:0/3545372027 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:42.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.265+0000 7f4c95ee9640 1 --2- 172.21.15.154:0/3545372027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c900c7c80 0x7f4c901f19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:42.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.265+0000 7f4c94ee7640 1 --2- 172.21.15.154:0/3545372027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c900c7800 0x7f4c901498d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:42.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.265+0000 7f4c956e8640 1 -- 172.21.15.154:0/3545372027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c900c7800 msgr2=0x7f4c901498d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:42.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.266+0000 7f4c956e8640 1 --2- 172.21.15.154:0/3545372027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c900c7800 0x7f4c901498d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:42.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.266+0000 7f4c956e8640 1 -- 172.21.15.154:0/3545372027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c900c7c80 msgr2=0x7f4c901f19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:42.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.266+0000 7f4c956e8640 1 --2- 172.21.15.154:0/3545372027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c900c7c80 0x7f4c901f19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:42.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.266+0000 7f4c956e8640 1 -- 172.21.15.154:0/3545372027 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c900b15d0 con 0x7f4c7c003970 2023-09-23T03:16:42.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.266+0000 7f4c956e8640 1 --2- 172.21.15.154:0/3545372027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c7c003970 0x7f4c900b0a30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c036040 tx=0x7f4c8c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:42.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.266+0000 7f4c85ffb640 1 -- 172.21.15.154:0/3545372027 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c005e70 con 0x7f4c7c003970 2023-09-23T03:16:42.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.266+0000 7f4c85ffb640 1 -- 172.21.15.154:0/3545372027 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8c004020 con 0x7f4c7c003970 2023-09-23T03:16:42.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.266+0000 7f4c85ffb640 1 -- 172.21.15.154:0/3545372027 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c0043d0 con 0x7f4c7c003970 2023-09-23T03:16:42.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.266+0000 7f4c9794c640 1 -- 172.21.15.154:0/3545372027 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c900b18b0 con 0x7f4c7c003970 2023-09-23T03:16:42.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.266+0000 7f4c9794c640 1 -- 172.21.15.154:0/3545372027 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c90149ff0 con 0x7f4c7c003970 2023-09-23T03:16:42.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.267+0000 7f4c9794c640 1 -- 172.21.15.154:0/3545372027 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c9004e920 con 0x7f4c7c003970 2023-09-23T03:16:42.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.268+0000 7f4c85ffb640 1 -- 172.21.15.154:0/3545372027 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f4c8c003610 con 0x7f4c7c003970 2023-09-23T03:16:42.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.270+0000 7f4c85ffb640 1 --2- 172.21.15.154:0/3545372027 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4c7406af30 0x7f4c7406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:42.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.270+0000 7f4c85ffb640 1 -- 172.21.15.154:0/3545372027 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f4c8c0ab9d0 con 0x7f4c7c003970 2023-09-23T03:16:42.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.270+0000 7f4c94ee7640 1 --2- 172.21.15.154:0/3545372027 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4c7406af30 0x7f4c7406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:42.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.271+0000 7f4c94ee7640 1 --2- 172.21.15.154:0/3545372027 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4c7406af30 0x7f4c7406d3f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4c80001eb0 tx=0x7f4c80009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:42.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.274+0000 7f4c85ffb640 1 -- 172.21.15.154:0/3545372027 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f4c8c076710 con 0x7f4c7c003970 2023-09-23T03:16:42.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.463+0000 7f4c9794c640 1 -- 172.21.15.154:0/3545372027 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4c90147610 con 0x7f4c7c003970 2023-09-23T03:16:42.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.464+0000 7f4c85ffb640 1 -- 172.21.15.154:0/3545372027 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4c8c04c0a0 con 0x7f4c7c003970 2023-09-23T03:16:42.465 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:16:42.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.467+0000 7f4c9794c640 1 -- 172.21.15.154:0/3545372027 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4c7406af30 msgr2=0x7f4c7406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:42.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.467+0000 7f4c9794c640 1 --2- 172.21.15.154:0/3545372027 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4c7406af30 0x7f4c7406d3f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4c80001eb0 tx=0x7f4c80009040 comp rx=0 tx=0).stop 2023-09-23T03:16:42.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.467+0000 7f4c9794c640 1 -- 172.21.15.154:0/3545372027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c7c003970 msgr2=0x7f4c900b0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:42.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.467+0000 7f4c9794c640 1 --2- 172.21.15.154:0/3545372027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c7c003970 0x7f4c900b0a30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c036040 tx=0x7f4c8c004730 comp rx=0 tx=0).stop 2023-09-23T03:16:42.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.467+0000 7f4c9794c640 1 -- 172.21.15.154:0/3545372027 shutdown_connections 2023-09-23T03:16:42.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.467+0000 7f4c9794c640 1 --2- 172.21.15.154:0/3545372027 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4c7406af30 0x7f4c7406d3f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:42.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.467+0000 7f4c9794c640 1 --2- 172.21.15.154:0/3545372027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c900c7c80 0x7f4c901f19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:42.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.467+0000 7f4c9794c640 1 --2- 172.21.15.154:0/3545372027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c900c7800 0x7f4c901498d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:42.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.467+0000 7f4c9794c640 1 --2- 172.21.15.154:0/3545372027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c7c003970 0x7f4c900b0a30 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:42.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.467+0000 7f4c9794c640 1 -- 172.21.15.154:0/3545372027 >> 172.21.15.154:0/3545372027 conn(0x7f4c9006fc40 msgr2=0x7f4c900cf270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:42.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.467+0000 7f4c9794c640 1 -- 172.21.15.154:0/3545372027 shutdown_connections 2023-09-23T03:16:42.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.467+0000 7f4c9794c640 1 -- 172.21.15.154:0/3545372027 wait complete. 2023-09-23T03:16:42.487 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:16:42.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.706+0000 7f69c94b4640 1 Processor -- start 2023-09-23T03:16:42.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.706+0000 7f69c94b4640 1 -- start start 2023-09-23T03:16:42.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.708+0000 7f69c94b4640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f69c40c8d70 0x7f69c415c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:42.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.708+0000 7f69c94b4640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f69c40c9150 0x7f69c415cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:42.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.708+0000 7f69c94b4640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69c40c9d20 con 0x7f69c4071290 2023-09-23T03:16:42.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.708+0000 7f69c94b4640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69c4071850 con 0x7f69c40c8d70 2023-09-23T03:16:42.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.708+0000 7f69c94b4640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69c4071b30 con 0x7f69c40c9150 2023-09-23T03:16:42.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.708+0000 7f69c2ffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f69c40c8d70 0x7f69c415c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:42.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.708+0000 7f69c2ffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f69c40c8d70 0x7f69c415c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52798/0 (socket says 172.21.15.154:52798) 2023-09-23T03:16:42.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.708+0000 7f69c2ffd640 1 -- 172.21.15.154:0/4253129293 learned_addr learned my addr 172.21.15.154:0/4253129293 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:42.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.708+0000 7f69c2ffd640 1 -- 172.21.15.154:0/4253129293 >> v2:172.21.15.190:3300/0 conn(0x7f69c40c9150 msgr2=0x7f69c415cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:42.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.708+0000 7f69c2ffd640 1 --2- 172.21.15.154:0/4253129293 >> v2:172.21.15.190:3300/0 conn(0x7f69c40c9150 0x7f69c415cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:42.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.708+0000 7f69c2ffd640 1 -- 172.21.15.154:0/4253129293 >> v1:172.21.15.190:6789/0 conn(0x7f69c4071290 legacy=0x7f69c40ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:42.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.708+0000 7f69c2ffd640 1 -- 172.21.15.154:0/4253129293 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c4071e10 con 0x7f69c40c8d70 2023-09-23T03:16:42.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.709+0000 7f69c2ffd640 1 --2- 172.21.15.154:0/4253129293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69c40c8d70 0x7f69c415c6d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f69c40b3ce0 tx=0x7f69b402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b688704294f150a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:42.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.709+0000 7f69c1ffb640 1 -- 172.21.15.154:0/4253129293 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b4031870 con 0x7f69c40c8d70 2023-09-23T03:16:42.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.709+0000 7f69c1ffb640 1 -- 172.21.15.154:0/4253129293 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b40319d0 con 0x7f69c40c8d70 2023-09-23T03:16:42.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.709+0000 7f69c1ffb640 1 -- 172.21.15.154:0/4253129293 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b4031d60 con 0x7f69c40c8d70 2023-09-23T03:16:42.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.710+0000 7f69c94b4640 1 -- 172.21.15.154:0/4253129293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69c40c8d70 msgr2=0x7f69c415c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:42.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.710+0000 7f69c94b4640 1 --2- 172.21.15.154:0/4253129293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69c40c8d70 0x7f69c415c6d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f69c40b3ce0 tx=0x7f69b402b910 comp rx=0 tx=0).stop 2023-09-23T03:16:42.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.710+0000 7f69c94b4640 1 -- 172.21.15.154:0/4253129293 shutdown_connections 2023-09-23T03:16:42.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.710+0000 7f69c94b4640 1 --2- 172.21.15.154:0/4253129293 >> v2:172.21.15.190:3300/0 conn(0x7f69c40c9150 0x7f69c415cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:42.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.710+0000 7f69c94b4640 1 --2- 172.21.15.154:0/4253129293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69c40c8d70 0x7f69c415c6d0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:42.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.710+0000 7f69c94b4640 1 -- 172.21.15.154:0/4253129293 >> 172.21.15.154:0/4253129293 conn(0x7f69c406c010 msgr2=0x7f69c406c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:42.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.710+0000 7f69c94b4640 1 -- 172.21.15.154:0/4253129293 shutdown_connections 2023-09-23T03:16:42.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.710+0000 7f69c94b4640 1 -- 172.21.15.154:0/4253129293 wait complete. 2023-09-23T03:16:42.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.712+0000 7f69c94b4640 1 Processor -- start 2023-09-23T03:16:42.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.712+0000 7f69c94b4640 1 -- start start 2023-09-23T03:16:42.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.713+0000 7f69c94b4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69c4071290 0x7f69c40afa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:42.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.713+0000 7f69c94b4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69c40c8d70 0x7f69c40b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:42.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.713+0000 7f69c94b4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69c40c9150 0x7f69c40b1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:42.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.713+0000 7f69c94b4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c40c43b0 con 0x7f69c40c8d70 2023-09-23T03:16:42.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.713+0000 7f69c94b4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c40b2630 con 0x7f69c40c9150 2023-09-23T03:16:42.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.713+0000 7f69c94b4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69c40b2910 con 0x7f69c4071290 2023-09-23T03:16:42.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.713+0000 7f69c27fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69c40c8d70 0x7f69c40b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:42.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.713+0000 7f69c27fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69c40c8d70 0x7f69c40b19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52804/0 (socket says 172.21.15.154:52804) 2023-09-23T03:16:42.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.713+0000 7f69c27fc640 1 -- 172.21.15.154:0/3617895176 learned_addr learned my addr 172.21.15.154:0/3617895176 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:42.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.713+0000 7f69c2ffd640 1 --2- 172.21.15.154:0/3617895176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69c4071290 0x7f69c40afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:42.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.713+0000 7f69c37fe640 1 --2- 172.21.15.154:0/3617895176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69c40c9150 0x7f69c40b1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:42.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.714+0000 7f69c27fc640 1 -- 172.21.15.154:0/3617895176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69c4071290 msgr2=0x7f69c40afa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:42.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.714+0000 7f69c27fc640 1 --2- 172.21.15.154:0/3617895176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69c4071290 0x7f69c40afa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:42.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.714+0000 7f69c27fc640 1 -- 172.21.15.154:0/3617895176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69c40c9150 msgr2=0x7f69c40b1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:42.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.714+0000 7f69c27fc640 1 --2- 172.21.15.154:0/3617895176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69c40c9150 0x7f69c40b1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:42.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.714+0000 7f69c27fc640 1 -- 172.21.15.154:0/3617895176 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b40094d0 con 0x7f69c40c8d70 2023-09-23T03:16:42.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.714+0000 7f69c27fc640 1 --2- 172.21.15.154:0/3617895176 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69c40c8d70 0x7f69c40b19e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f69b800b410 tx=0x7f69b800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:42.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.714+0000 7f69ab7fe640 1 -- 172.21.15.154:0/3617895176 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b8011830 con 0x7f69c40c8d70 2023-09-23T03:16:42.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.714+0000 7f69ab7fe640 1 -- 172.21.15.154:0/3617895176 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b80119b0 con 0x7f69c40c8d70 2023-09-23T03:16:42.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.714+0000 7f69ab7fe640 1 -- 172.21.15.154:0/3617895176 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b8011d60 con 0x7f69c40c8d70 2023-09-23T03:16:42.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.714+0000 7f69c94b4640 1 -- 172.21.15.154:0/3617895176 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69c40b2bf0 con 0x7f69c40c8d70 2023-09-23T03:16:42.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.715+0000 7f69c94b4640 1 -- 172.21.15.154:0/3617895176 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69c40aff90 con 0x7f69c40c8d70 2023-09-23T03:16:42.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.716+0000 7f69c94b4640 1 -- 172.21.15.154:0/3617895176 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69c40b0180 con 0x7f69c40c8d70 2023-09-23T03:16:42.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.717+0000 7f69ab7fe640 1 -- 172.21.15.154:0/3617895176 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f69b801a400 con 0x7f69c40c8d70 2023-09-23T03:16:42.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.717+0000 7f69ab7fe640 1 --2- 172.21.15.154:0/3617895176 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f69b006af30 0x7f69b006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:42.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.718+0000 7f69ab7fe640 1 -- 172.21.15.154:0/3617895176 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f69b8089d10 con 0x7f69c40c8d70 2023-09-23T03:16:42.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.718+0000 7f69c2ffd640 1 --2- 172.21.15.154:0/3617895176 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f69b006af30 0x7f69b006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:42.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.719+0000 7f69c2ffd640 1 --2- 172.21.15.154:0/3617895176 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f69b006af30 0x7f69b006d3f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f69b4036000 tx=0x7f69b403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:42.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.723+0000 7f69ab7fe640 1 -- 172.21.15.154:0/3617895176 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f69b808f020 con 0x7f69c40c8d70 2023-09-23T03:16:42.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.911+0000 7f69c94b4640 1 -- 172.21.15.154:0/3617895176 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f69c40c23c0 con 0x7f69c40c8d70 2023-09-23T03:16:42.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.912+0000 7f69ab7fe640 1 -- 172.21.15.154:0/3617895176 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f69c40c23c0 con 0x7f69c40c8d70 2023-09-23T03:16:42.913 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:16:42.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.915+0000 7f69c94b4640 1 -- 172.21.15.154:0/3617895176 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f69b006af30 msgr2=0x7f69b006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:42.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.915+0000 7f69c94b4640 1 --2- 172.21.15.154:0/3617895176 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f69b006af30 0x7f69b006d3f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f69b4036000 tx=0x7f69b403c040 comp rx=0 tx=0).stop 2023-09-23T03:16:42.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.915+0000 7f69c94b4640 1 -- 172.21.15.154:0/3617895176 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69c40c8d70 msgr2=0x7f69c40b19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:42.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.915+0000 7f69c94b4640 1 --2- 172.21.15.154:0/3617895176 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69c40c8d70 0x7f69c40b19e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f69b800b410 tx=0x7f69b800b8d0 comp rx=0 tx=0).stop 2023-09-23T03:16:42.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.915+0000 7f69c94b4640 1 -- 172.21.15.154:0/3617895176 shutdown_connections 2023-09-23T03:16:42.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.915+0000 7f69c94b4640 1 --2- 172.21.15.154:0/3617895176 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f69b006af30 0x7f69b006d3f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:42.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.915+0000 7f69c94b4640 1 --2- 172.21.15.154:0/3617895176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69c40c9150 0x7f69c40b1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:42.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.915+0000 7f69c94b4640 1 --2- 172.21.15.154:0/3617895176 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69c40c8d70 0x7f69c40b19e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:42.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.915+0000 7f69c94b4640 1 --2- 172.21.15.154:0/3617895176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69c4071290 0x7f69c40afa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:42.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.915+0000 7f69c94b4640 1 -- 172.21.15.154:0/3617895176 >> 172.21.15.154:0/3617895176 conn(0x7f69c406c010 msgr2=0x7f69c4158850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:42.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.915+0000 7f69c94b4640 1 -- 172.21.15.154:0/3617895176 shutdown_connections 2023-09-23T03:16:42.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:42.915+0000 7f69c94b4640 1 -- 172.21.15.154:0/3617895176 wait complete. 2023-09-23T03:16:42.934 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:16:43.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.149+0000 7faeb2d1d640 1 Processor -- start 2023-09-23T03:16:43.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.150+0000 7faeb2d1d640 1 -- start start 2023-09-23T03:16:43.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.151+0000 7faeb2d1d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7faeac15aba0 0x7faeac15bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:43.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.151+0000 7faeb2d1d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faeac0cb5b0 0x7faeac15ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:43.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.151+0000 7faeb2d1d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeac15b6a0 con 0x7faeac15e5f0 2023-09-23T03:16:43.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.151+0000 7faeb2d1d640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faeac15f0b0 con 0x7faeac15aba0 2023-09-23T03:16:43.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.151+0000 7faeb2d1d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeac15f390 con 0x7faeac0cb5b0 2023-09-23T03:16:43.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.151+0000 7faeabfff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7faeac15e5f0 0x7faeac0caea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:42156/0 (socket says 172.21.15.154:42156) 2023-09-23T03:16:43.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.152+0000 7faeabfff640 1 -- 172.21.15.154:0/3323726492 learned_addr learned my addr 172.21.15.154:0/3323726492 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:43.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.152+0000 7faeb0ab9640 1 --2- 172.21.15.154:0/3323726492 >> v2:172.21.15.190:3300/0 conn(0x7faeac15aba0 0x7faeac15bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:43.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.152+0000 7faeb12ba640 1 --2- 172.21.15.154:0/3323726492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faeac0cb5b0 0x7faeac15ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:43.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.152+0000 7faeab7fe640 1 -- 172.21.15.154:0/3323726492 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1050453510 0 0) 0x7faeac15b6a0 con 0x7faeac15e5f0 2023-09-23T03:16:43.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.152+0000 7faeab7fe640 1 -- 172.21.15.154:0/3323726492 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae94003520 con 0x7faeac15e5f0 2023-09-23T03:16:43.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.153+0000 7faeab7fe640 1 -- 172.21.15.154:0/3323726492 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 629675772 0 0) 0x7fae94003520 con 0x7faeac15e5f0 2023-09-23T03:16:43.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.153+0000 7faeab7fe640 1 -- 172.21.15.154:0/3323726492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faeac0cb5b0 msgr2=0x7faeac15ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:43.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.153+0000 7faeab7fe640 1 --2- 172.21.15.154:0/3323726492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faeac0cb5b0 0x7faeac15ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:43.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.153+0000 7faeab7fe640 1 -- 172.21.15.154:0/3323726492 >> v2:172.21.15.190:3300/0 conn(0x7faeac15aba0 msgr2=0x7faeac15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:43.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.153+0000 7faeab7fe640 1 --2- 172.21.15.154:0/3323726492 >> v2:172.21.15.190:3300/0 conn(0x7faeac15aba0 0x7faeac15bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:43.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.153+0000 7faeab7fe640 1 -- 172.21.15.154:0/3323726492 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeac15b330 con 0x7faeac15e5f0 2023-09-23T03:16:43.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.153+0000 7faeab7fe640 1 -- 172.21.15.154:0/3323726492 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7faea0001670 con 0x7faeac15e5f0 2023-09-23T03:16:43.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.153+0000 7faeab7fe640 1 -- 172.21.15.154:0/3323726492 >> v1:172.21.15.154:6789/0 conn(0x7faeac15e5f0 legacy=0x7faeac0caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:43.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.153+0000 7faeab7fe640 1 --2- 172.21.15.154:0/3323726492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae94003970 0x7fae94003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:43.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.153+0000 7faeab7fe640 1 --2- 172.21.15.154:0/3323726492 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae940047c0 0x7fae94004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:43.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.153+0000 7faeab7fe640 1 --2- 172.21.15.154:0/3323726492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faeac0cb5b0 0x7fae94005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:43.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.153+0000 7faeab7fe640 1 -- 172.21.15.154:0/3323726492 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeac15f0b0 con 0x7fae940047c0 2023-09-23T03:16:43.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.153+0000 7faeab7fe640 1 -- 172.21.15.154:0/3323726492 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeac15f390 con 0x7fae94003970 2023-09-23T03:16:43.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.153+0000 7faeab7fe640 1 -- 172.21.15.154:0/3323726492 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9400b980 con 0x7faeac0cb5b0 2023-09-23T03:16:43.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.153+0000 7faeabfff640 1 --2- 172.21.15.154:0/3323726492 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae940047c0 0x7fae94004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:43.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.153+0000 7faeabfff640 1 -- 172.21.15.154:0/3323726492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faeac0cb5b0 msgr2=0x7fae94005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:43.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.153+0000 7faeabfff640 1 --2- 172.21.15.154:0/3323726492 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faeac0cb5b0 0x7fae94005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:43.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.153+0000 7faeabfff640 1 -- 172.21.15.154:0/3323726492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae94003970 msgr2=0x7fae94003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:43.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.153+0000 7faeabfff640 1 --2- 172.21.15.154:0/3323726492 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae94003970 0x7fae94003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:43.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.153+0000 7faeabfff640 1 -- 172.21.15.154:0/3323726492 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae94003520 con 0x7fae940047c0 2023-09-23T03:16:43.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.154+0000 7faeabfff640 1 --2- 172.21.15.154:0/3323726492 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae940047c0 0x7fae94004c40 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7faea000c110 tx=0x7fae940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23dac202ce84044c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:43.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.155+0000 7faeab7fe640 1 -- 172.21.15.154:0/3323726492 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea0005ad0 con 0x7fae940047c0 2023-09-23T03:16:43.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.155+0000 7faeab7fe640 1 -- 172.21.15.154:0/3323726492 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea0005c50 con 0x7fae940047c0 2023-09-23T03:16:43.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.155+0000 7faeab7fe640 1 -- 172.21.15.154:0/3323726492 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea002c830 con 0x7fae940047c0 2023-09-23T03:16:43.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.155+0000 7faeb2d1d640 1 -- 172.21.15.154:0/3323726492 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae940047c0 msgr2=0x7fae94004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:43.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.155+0000 7faeb2d1d640 1 --2- 172.21.15.154:0/3323726492 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae940047c0 0x7fae94004c40 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7faea000c110 tx=0x7fae940057f0 comp rx=0 tx=0).stop 2023-09-23T03:16:43.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.155+0000 7faeb12ba640 1 -- 172.21.15.154:0/3323726492 reap_dead start 2023-09-23T03:16:43.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.155+0000 7faeb2d1d640 1 -- 172.21.15.154:0/3323726492 shutdown_connections 2023-09-23T03:16:43.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.155+0000 7faeb2d1d640 1 -- 172.21.15.154:0/3323726492 >> 172.21.15.154:0/3323726492 conn(0x7faeac06c010 msgr2=0x7faeac06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:43.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.155+0000 7faeb2d1d640 1 -- 172.21.15.154:0/3323726492 shutdown_connections 2023-09-23T03:16:43.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.156+0000 7faeb2d1d640 1 -- 172.21.15.154:0/3323726492 wait complete. 2023-09-23T03:16:43.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.157+0000 7faeb2d1d640 1 Processor -- start 2023-09-23T03:16:43.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.157+0000 7faeb2d1d640 1 -- start start 2023-09-23T03:16:43.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.158+0000 7faeb2d1d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae940047c0 0x7faeac1f1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:43.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.158+0000 7faeb2d1d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faeac157590 0x7faeac0b0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:43.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.158+0000 7faeb2d1d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faeac0b2f90 0x7faeac0b3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:43.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.158+0000 7faeb2d1d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeac0ad1c0 con 0x7faeac0b2f90 2023-09-23T03:16:43.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.158+0000 7faeb2d1d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeac0b0c70 con 0x7faeac157590 2023-09-23T03:16:43.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.158+0000 7faeb2d1d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeac0b0f40 con 0x7fae940047c0 2023-09-23T03:16:43.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.159+0000 7faeb12ba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faeac0b2f90 0x7faeac0b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:43.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.159+0000 7faeb12ba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faeac0b2f90 0x7faeac0b3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52832/0 (socket says 172.21.15.154:52832) 2023-09-23T03:16:43.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.159+0000 7faeb12ba640 1 -- 172.21.15.154:0/638560743 learned_addr learned my addr 172.21.15.154:0/638560743 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:43.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.159+0000 7faeb12ba640 1 -- 172.21.15.154:0/638560743 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae940047c0 msgr2=0x7faeac1f1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:43.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.159+0000 7faeb12ba640 1 --2- 172.21.15.154:0/638560743 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae940047c0 0x7faeac1f1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:43.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.159+0000 7faeb12ba640 1 -- 172.21.15.154:0/638560743 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faeac157590 msgr2=0x7faeac0b0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:43.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.159+0000 7faeb12ba640 1 --2- 172.21.15.154:0/638560743 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faeac157590 0x7faeac0b0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:43.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.159+0000 7faeb12ba640 1 -- 172.21.15.154:0/638560743 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeac15b100 con 0x7faeac0b2f90 2023-09-23T03:16:43.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.159+0000 7faeb12ba640 1 --2- 172.21.15.154:0/638560743 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faeac0b2f90 0x7faeac0b3440 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7faeac0adf40 tx=0x7faea400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:43.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.159+0000 7faea97fa640 1 -- 172.21.15.154:0/638560743 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea400c680 con 0x7faeac0b2f90 2023-09-23T03:16:43.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.160+0000 7faea97fa640 1 -- 172.21.15.154:0/638560743 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea400c7e0 con 0x7faeac0b2f90 2023-09-23T03:16:43.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.160+0000 7faea97fa640 1 -- 172.21.15.154:0/638560743 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea400cb90 con 0x7faeac0b2f90 2023-09-23T03:16:43.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.160+0000 7faeb2d1d640 1 -- 172.21.15.154:0/638560743 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeac0b1220 con 0x7faeac0b2f90 2023-09-23T03:16:43.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.160+0000 7faeb2d1d640 1 -- 172.21.15.154:0/638560743 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeac0b17e0 con 0x7faeac0b2f90 2023-09-23T03:16:43.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.161+0000 7faeb2d1d640 1 -- 172.21.15.154:0/638560743 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeac04e920 con 0x7faeac0b2f90 2023-09-23T03:16:43.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.162+0000 7faea97fa640 1 -- 172.21.15.154:0/638560743 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7faea400ccf0 con 0x7faeac0b2f90 2023-09-23T03:16:43.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.163+0000 7faea97fa640 1 --2- 172.21.15.154:0/638560743 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fae9806af30 0x7fae9806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:43.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.163+0000 7faea97fa640 1 -- 172.21.15.154:0/638560743 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7faea40897e0 con 0x7faeac0b2f90 2023-09-23T03:16:43.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.163+0000 7faeb0ab9640 1 --2- 172.21.15.154:0/638560743 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fae9806af30 0x7fae9806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:43.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.164+0000 7faeb0ab9640 1 --2- 172.21.15.154:0/638560743 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fae9806af30 0x7fae9806d3f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fae9c004b30 tx=0x7fae9c00b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:43.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.167+0000 7faea97fa640 1 -- 172.21.15.154:0/638560743 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7faea4054460 con 0x7faeac0b2f90 2023-09-23T03:16:43.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.358+0000 7faeb2d1d640 1 -- 172.21.15.154:0/638560743 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faeac15edf0 con 0x7faeac0b2f90 2023-09-23T03:16:43.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.359+0000 7faea97fa640 1 -- 172.21.15.154:0/638560743 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7faea4057a80 con 0x7faeac0b2f90 2023-09-23T03:16:43.360 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:16:43.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.362+0000 7faeb2d1d640 1 -- 172.21.15.154:0/638560743 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fae9806af30 msgr2=0x7fae9806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:43.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.362+0000 7faeb2d1d640 1 --2- 172.21.15.154:0/638560743 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fae9806af30 0x7fae9806d3f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fae9c004b30 tx=0x7fae9c00b040 comp rx=0 tx=0).stop 2023-09-23T03:16:43.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.362+0000 7faeb2d1d640 1 -- 172.21.15.154:0/638560743 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faeac0b2f90 msgr2=0x7faeac0b3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:43.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.362+0000 7faeb2d1d640 1 --2- 172.21.15.154:0/638560743 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faeac0b2f90 0x7faeac0b3440 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7faeac0adf40 tx=0x7faea400ec00 comp rx=0 tx=0).stop 2023-09-23T03:16:43.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.362+0000 7faeb2d1d640 1 -- 172.21.15.154:0/638560743 shutdown_connections 2023-09-23T03:16:43.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.362+0000 7faeb2d1d640 1 --2- 172.21.15.154:0/638560743 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fae9806af30 0x7fae9806d3f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:43.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.362+0000 7faeb2d1d640 1 --2- 172.21.15.154:0/638560743 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faeac0b2f90 0x7faeac0b3440 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:43.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.362+0000 7faeb2d1d640 1 --2- 172.21.15.154:0/638560743 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faeac157590 0x7faeac0b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:43.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.362+0000 7faeb2d1d640 1 --2- 172.21.15.154:0/638560743 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae940047c0 0x7faeac1f1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:43.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.362+0000 7faeb2d1d640 1 -- 172.21.15.154:0/638560743 >> 172.21.15.154:0/638560743 conn(0x7faeac06c010 msgr2=0x7faeac159dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:43.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.362+0000 7faeb2d1d640 1 -- 172.21.15.154:0/638560743 shutdown_connections 2023-09-23T03:16:43.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.362+0000 7faeb2d1d640 1 -- 172.21.15.154:0/638560743 wait complete. 2023-09-23T03:16:43.382 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-23T03:16:43.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.592+0000 7fde4b64b640 1 Processor -- start 2023-09-23T03:16:43.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.592+0000 7fde4b64b640 1 -- start start 2023-09-23T03:16:43.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.593+0000 7fde4b64b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde4415aba0 0x7fde4415bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:43.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.594+0000 7fde4b64b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fde440cb5b0 0x7fde4415ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:43.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.594+0000 7fde4b64b640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde4415b6a0 con 0x7fde4415e5f0 2023-09-23T03:16:43.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.594+0000 7fde4b64b640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde4415f0b0 con 0x7fde440cb5b0 2023-09-23T03:16:43.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.594+0000 7fde4b64b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde4415f390 con 0x7fde4415aba0 2023-09-23T03:16:43.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.594+0000 7fde49be8640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fde440cb5b0 0x7fde4415ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:43.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.594+0000 7fde49be8640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fde440cb5b0 0x7fde4415ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52848/0 (socket says 172.21.15.154:52848) 2023-09-23T03:16:43.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.594+0000 7fde49be8640 1 -- 172.21.15.154:0/465999141 learned_addr learned my addr 172.21.15.154:0/465999141 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:43.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.594+0000 7fde493e7640 1 --2- 172.21.15.154:0/465999141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde4415aba0 0x7fde4415bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:43.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.594+0000 7fde49be8640 1 -- 172.21.15.154:0/465999141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde4415aba0 msgr2=0x7fde4415bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:43.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.594+0000 7fde49be8640 1 --2- 172.21.15.154:0/465999141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde4415aba0 0x7fde4415bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:43.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.594+0000 7fde49be8640 1 -- 172.21.15.154:0/465999141 >> v1:172.21.15.190:6789/0 conn(0x7fde4415e5f0 legacy=0x7fde440caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:43.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.594+0000 7fde49be8640 1 -- 172.21.15.154:0/465999141 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde4415b330 con 0x7fde440cb5b0 2023-09-23T03:16:43.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.595+0000 7fde49be8640 1 --2- 172.21.15.154:0/465999141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde440cb5b0 0x7fde4415ea30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fde3c009e50 tx=0x7fde3c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=499634f422857c9c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:43.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.596+0000 7fde3bfff640 1 -- 172.21.15.154:0/465999141 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c02cb00 con 0x7fde440cb5b0 2023-09-23T03:16:43.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.596+0000 7fde3bfff640 1 -- 172.21.15.154:0/465999141 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3c02cc60 con 0x7fde440cb5b0 2023-09-23T03:16:43.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.596+0000 7fde3bfff640 1 -- 172.21.15.154:0/465999141 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c0343f0 con 0x7fde440cb5b0 2023-09-23T03:16:43.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.596+0000 7fde4b64b640 1 -- 172.21.15.154:0/465999141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde440cb5b0 msgr2=0x7fde4415ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:43.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.596+0000 7fde4b64b640 1 --2- 172.21.15.154:0/465999141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde440cb5b0 0x7fde4415ea30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fde3c009e50 tx=0x7fde3c02bf60 comp rx=0 tx=0).stop 2023-09-23T03:16:43.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.596+0000 7fde4b64b640 1 -- 172.21.15.154:0/465999141 shutdown_connections 2023-09-23T03:16:43.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.596+0000 7fde4b64b640 1 --2- 172.21.15.154:0/465999141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde440cb5b0 0x7fde4415ea30 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:43.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.596+0000 7fde4b64b640 1 --2- 172.21.15.154:0/465999141 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde4415aba0 0x7fde4415bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:43.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.596+0000 7fde4b64b640 1 -- 172.21.15.154:0/465999141 >> 172.21.15.154:0/465999141 conn(0x7fde4406c010 msgr2=0x7fde4406c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:43.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.596+0000 7fde4b64b640 1 -- 172.21.15.154:0/465999141 shutdown_connections 2023-09-23T03:16:43.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.596+0000 7fde4b64b640 1 -- 172.21.15.154:0/465999141 wait complete. 2023-09-23T03:16:43.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.598+0000 7fde4b64b640 1 Processor -- start 2023-09-23T03:16:43.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.598+0000 7fde4b64b640 1 -- start start 2023-09-23T03:16:43.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.599+0000 7fde4b64b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde440cb5b0 0x7fde441f2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:43.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.599+0000 7fde4b64b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde4415aba0 0x7fde441450d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:43.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.599+0000 7fde4b64b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde4415e5f0 0x7fde441f5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:43.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.599+0000 7fde4b64b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde441f2850 con 0x7fde4415e5f0 2023-09-23T03:16:43.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.599+0000 7fde4b64b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde441f2b30 con 0x7fde440cb5b0 2023-09-23T03:16:43.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.599+0000 7fde4b64b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde441f2e10 con 0x7fde4415aba0 2023-09-23T03:16:43.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.599+0000 7fde49be8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde4415e5f0 0x7fde441f5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:43.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.599+0000 7fde49be8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde4415e5f0 0x7fde441f5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52858/0 (socket says 172.21.15.154:52858) 2023-09-23T03:16:43.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.600+0000 7fde49be8640 1 -- 172.21.15.154:0/3889302807 learned_addr learned my addr 172.21.15.154:0/3889302807 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:43.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.600+0000 7fde48be6640 1 --2- 172.21.15.154:0/3889302807 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde4415aba0 0x7fde441450d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:43.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.600+0000 7fde49be8640 1 -- 172.21.15.154:0/3889302807 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde4415aba0 msgr2=0x7fde441450d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:43.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.600+0000 7fde493e7640 1 --2- 172.21.15.154:0/3889302807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde440cb5b0 0x7fde441f2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:43.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.600+0000 7fde49be8640 1 --2- 172.21.15.154:0/3889302807 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde4415aba0 0x7fde441450d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:43.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.600+0000 7fde49be8640 1 -- 172.21.15.154:0/3889302807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde440cb5b0 msgr2=0x7fde441f2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:43.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.600+0000 7fde49be8640 1 --2- 172.21.15.154:0/3889302807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde440cb5b0 0x7fde441f2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:43.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.600+0000 7fde49be8640 1 -- 172.21.15.154:0/3889302807 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde3c009b30 con 0x7fde4415e5f0 2023-09-23T03:16:43.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.600+0000 7fde48be6640 1 --2- 172.21.15.154:0/3889302807 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde4415aba0 0x7fde441450d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:43.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.600+0000 7fde49be8640 1 --2- 172.21.15.154:0/3889302807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde4415e5f0 0x7fde441f5f60 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fde3c036040 tx=0x7fde3c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:43.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.600+0000 7fde39ffb640 1 -- 172.21.15.154:0/3889302807 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c009510 con 0x7fde4415e5f0 2023-09-23T03:16:43.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.601+0000 7fde39ffb640 1 -- 172.21.15.154:0/3889302807 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3c03e450 con 0x7fde4415e5f0 2023-09-23T03:16:43.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.601+0000 7fde4b64b640 1 -- 172.21.15.154:0/3889302807 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde4415b100 con 0x7fde4415e5f0 2023-09-23T03:16:43.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.601+0000 7fde39ffb640 1 -- 172.21.15.154:0/3889302807 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c03e7e0 con 0x7fde4415e5f0 2023-09-23T03:16:43.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.601+0000 7fde4b64b640 1 -- 172.21.15.154:0/3889302807 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde440c5580 con 0x7fde4415e5f0 2023-09-23T03:16:43.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.602+0000 7fde4b64b640 1 -- 172.21.15.154:0/3889302807 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde4404e920 con 0x7fde4415e5f0 2023-09-23T03:16:43.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.603+0000 7fde39ffb640 1 -- 172.21.15.154:0/3889302807 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fde3c03e940 con 0x7fde4415e5f0 2023-09-23T03:16:43.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.604+0000 7fde39ffb640 1 --2- 172.21.15.154:0/3889302807 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fde2806af30 0x7fde2806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:43.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.604+0000 7fde39ffb640 1 -- 172.21.15.154:0/3889302807 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7fde3c0ac870 con 0x7fde4415e5f0 2023-09-23T03:16:43.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.604+0000 7fde493e7640 1 --2- 172.21.15.154:0/3889302807 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fde2806af30 0x7fde2806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:43.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.604+0000 7fde493e7640 1 --2- 172.21.15.154:0/3889302807 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fde2806af30 0x7fde2806d3f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fde40006fd0 tx=0x7fde40008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:43.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.608+0000 7fde39ffb640 1 -- 172.21.15.154:0/3889302807 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fde3c0776b0 con 0x7fde4415e5f0 2023-09-23T03:16:43.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.790+0000 7fde4b64b640 1 -- 172.21.15.154:0/3889302807 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fde441f2990 con 0x7fde4415e5f0 2023-09-23T03:16:43.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.791+0000 7fde39ffb640 1 -- 172.21.15.154:0/3889302807 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fde441f2990 con 0x7fde4415e5f0 2023-09-23T03:16:43.791 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:16:43.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.793+0000 7fde4b64b640 1 -- 172.21.15.154:0/3889302807 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fde2806af30 msgr2=0x7fde2806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:43.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.793+0000 7fde4b64b640 1 --2- 172.21.15.154:0/3889302807 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fde2806af30 0x7fde2806d3f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fde40006fd0 tx=0x7fde40008040 comp rx=0 tx=0).stop 2023-09-23T03:16:43.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.793+0000 7fde4b64b640 1 -- 172.21.15.154:0/3889302807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde4415e5f0 msgr2=0x7fde441f5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:43.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.793+0000 7fde4b64b640 1 --2- 172.21.15.154:0/3889302807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde4415e5f0 0x7fde441f5f60 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fde3c036040 tx=0x7fde3c004a40 comp rx=0 tx=0).stop 2023-09-23T03:16:43.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.793+0000 7fde4b64b640 1 -- 172.21.15.154:0/3889302807 shutdown_connections 2023-09-23T03:16:43.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.793+0000 7fde4b64b640 1 --2- 172.21.15.154:0/3889302807 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fde2806af30 0x7fde2806d3f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:43.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.793+0000 7fde4b64b640 1 --2- 172.21.15.154:0/3889302807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde4415e5f0 0x7fde441f5f60 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:43.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.793+0000 7fde4b64b640 1 --2- 172.21.15.154:0/3889302807 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde4415aba0 0x7fde441450d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:43.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.793+0000 7fde4b64b640 1 --2- 172.21.15.154:0/3889302807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde440cb5b0 0x7fde441f2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:43.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.793+0000 7fde4b64b640 1 -- 172.21.15.154:0/3889302807 >> 172.21.15.154:0/3889302807 conn(0x7fde4406c010 msgr2=0x7fde4415cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:43.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.794+0000 7fde4b64b640 1 -- 172.21.15.154:0/3889302807 shutdown_connections 2023-09-23T03:16:43.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:43.794+0000 7fde4b64b640 1 -- 172.21.15.154:0/3889302807 wait complete. 2023-09-23T03:16:43.811 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-23T03:16:44.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.022+0000 7f41cf01a640 1 Processor -- start 2023-09-23T03:16:44.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.023+0000 7f41cf01a640 1 -- start start 2023-09-23T03:16:44.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.024+0000 7f41cf01a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f41c80ada30 0x7f41c80ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:44.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.024+0000 7f41cf01a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41c80cf130 0x7f41c815d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:44.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.024+0000 7f41cf01a640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41c806ceb0 con 0x7f41c80b2fe0 2023-09-23T03:16:44.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.024+0000 7f41cf01a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41c80ade10 con 0x7f41c80ada30 2023-09-23T03:16:44.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.024+0000 7f41cf01a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41c8159c90 con 0x7f41c80cf130 2023-09-23T03:16:44.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.024+0000 7f41ccdb6640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f41c80ada30 0x7f41c80ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:44.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.024+0000 7f41ccdb6640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f41c80ada30 0x7f41c80ccaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:41018/0 (socket says 172.21.15.154:41018) 2023-09-23T03:16:44.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.024+0000 7f41ccdb6640 1 -- 172.21.15.154:0/3721501113 learned_addr learned my addr 172.21.15.154:0/3721501113 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:44.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.025+0000 7f41cd5b7640 1 --2- 172.21.15.154:0/3721501113 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41c80cf130 0x7f41c815d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:44.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.025+0000 7f41ccdb6640 1 -- 172.21.15.154:0/3721501113 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41c80cf130 msgr2=0x7f41c815d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:44.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.025+0000 7f41ccdb6640 1 --2- 172.21.15.154:0/3721501113 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41c80cf130 0x7f41c815d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:44.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.025+0000 7f41ccdb6640 1 -- 172.21.15.154:0/3721501113 >> v1:172.21.15.190:6789/0 conn(0x7f41c80b2fe0 legacy=0x7f41c80cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:44.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.025+0000 7f41ccdb6640 1 -- 172.21.15.154:0/3721501113 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41c806c110 con 0x7f41c80ada30 2023-09-23T03:16:44.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.025+0000 7f41cd5b7640 1 --2- 172.21.15.154:0/3721501113 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41c80cf130 0x7f41c815d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:44.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.026+0000 7f41ccdb6640 1 --2- 172.21.15.154:0/3721501113 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41c80ada30 0x7f41c80ccaf0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f41bc009850 tx=0x7f41bc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb0c18a14244d31 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:44.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.026+0000 7f41c77fe640 1 -- 172.21.15.154:0/3721501113 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc004230 con 0x7f41c80ada30 2023-09-23T03:16:44.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.026+0000 7f41c77fe640 1 -- 172.21.15.154:0/3721501113 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41bc004390 con 0x7f41c80ada30 2023-09-23T03:16:44.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.026+0000 7f41c77fe640 1 -- 172.21.15.154:0/3721501113 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc031830 con 0x7f41c80ada30 2023-09-23T03:16:44.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.027+0000 7f41cf01a640 1 -- 172.21.15.154:0/3721501113 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41c80ada30 msgr2=0x7f41c80ccaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:44.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.027+0000 7f41cf01a640 1 --2- 172.21.15.154:0/3721501113 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41c80ada30 0x7f41c80ccaf0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f41bc009850 tx=0x7f41bc02b8d0 comp rx=0 tx=0).stop 2023-09-23T03:16:44.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.027+0000 7f41cf01a640 1 -- 172.21.15.154:0/3721501113 shutdown_connections 2023-09-23T03:16:44.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.027+0000 7f41cf01a640 1 --2- 172.21.15.154:0/3721501113 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41c80cf130 0x7f41c815d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:44.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.027+0000 7f41cf01a640 1 --2- 172.21.15.154:0/3721501113 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41c80ada30 0x7f41c80ccaf0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:44.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.027+0000 7f41cf01a640 1 -- 172.21.15.154:0/3721501113 >> 172.21.15.154:0/3721501113 conn(0x7f41c806a0f0 msgr2=0x7f41c806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:44.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.027+0000 7f41cf01a640 1 -- 172.21.15.154:0/3721501113 shutdown_connections 2023-09-23T03:16:44.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.027+0000 7f41cf01a640 1 -- 172.21.15.154:0/3721501113 wait complete. 2023-09-23T03:16:44.029 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.029+0000 7f41cf01a640 1 Processor -- start 2023-09-23T03:16:44.029 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.029+0000 7f41cf01a640 1 -- start start 2023-09-23T03:16:44.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.030+0000 7f41cf01a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41c80ada30 0x7f41c815c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:44.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.030+0000 7f41cf01a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41c80b2fe0 0x7f41c809b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:44.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.030+0000 7f41cf01a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41c80cf130 0x7f41c8156d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:44.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.030+0000 7f41cf01a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c8072c50 con 0x7f41c80b2fe0 2023-09-23T03:16:44.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.030+0000 7f41cf01a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c809a2b0 con 0x7f41c80cf130 2023-09-23T03:16:44.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.030+0000 7f41cf01a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41c809adf0 con 0x7f41c80ada30 2023-09-23T03:16:44.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.030+0000 7f41c7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41c80b2fe0 0x7f41c809b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:44.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.030+0000 7f41c7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41c80b2fe0 0x7f41c809b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:41028/0 (socket says 172.21.15.154:41028) 2023-09-23T03:16:44.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.030+0000 7f41c7fff640 1 -- 172.21.15.154:0/3109244085 learned_addr learned my addr 172.21.15.154:0/3109244085 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:44.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.031+0000 7f41ccdb6640 1 --2- 172.21.15.154:0/3109244085 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41c80ada30 0x7f41c815c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:44.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.031+0000 7f41cd5b7640 1 --2- 172.21.15.154:0/3109244085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41c80cf130 0x7f41c8156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:44.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.031+0000 7f41c7fff640 1 -- 172.21.15.154:0/3109244085 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41c80ada30 msgr2=0x7f41c815c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:44.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.031+0000 7f41c7fff640 1 --2- 172.21.15.154:0/3109244085 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41c80ada30 0x7f41c815c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:44.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.031+0000 7f41c7fff640 1 -- 172.21.15.154:0/3109244085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41c80cf130 msgr2=0x7f41c8156d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:44.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.031+0000 7f41c7fff640 1 --2- 172.21.15.154:0/3109244085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41c80cf130 0x7f41c8156d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:44.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.031+0000 7f41c7fff640 1 -- 172.21.15.154:0/3109244085 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41bc009460 con 0x7f41c80b2fe0 2023-09-23T03:16:44.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.031+0000 7f41cd5b7640 1 --2- 172.21.15.154:0/3109244085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41c80cf130 0x7f41c8156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:44.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.031+0000 7f41c7fff640 1 --2- 172.21.15.154:0/3109244085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41c80b2fe0 0x7f41c809b280 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f41b400c3f0 tx=0x7f41b400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:44.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.031+0000 7f41c57fa640 1 -- 172.21.15.154:0/3109244085 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b4016070 con 0x7f41c80b2fe0 2023-09-23T03:16:44.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.031+0000 7f41c57fa640 1 -- 172.21.15.154:0/3109244085 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41b4007020 con 0x7f41c80b2fe0 2023-09-23T03:16:44.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.031+0000 7f41c57fa640 1 -- 172.21.15.154:0/3109244085 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b40073b0 con 0x7f41c80b2fe0 2023-09-23T03:16:44.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.031+0000 7f41cf01a640 1 -- 172.21.15.154:0/3109244085 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41c815cd40 con 0x7f41c80b2fe0 2023-09-23T03:16:44.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.032+0000 7f41cf01a640 1 -- 172.21.15.154:0/3109244085 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41c809a7a0 con 0x7f41c80b2fe0 2023-09-23T03:16:44.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.033+0000 7f41cf01a640 1 -- 172.21.15.154:0/3109244085 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41c8073eb0 con 0x7f41c80b2fe0 2023-09-23T03:16:44.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.034+0000 7f41c57fa640 1 -- 172.21.15.154:0/3109244085 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f41b4007510 con 0x7f41c80b2fe0 2023-09-23T03:16:44.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.035+0000 7f41c57fa640 1 --2- 172.21.15.154:0/3109244085 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f41ac06af30 0x7f41ac06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:44.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.035+0000 7f41c57fa640 1 -- 172.21.15.154:0/3109244085 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f41b4089390 con 0x7f41c80b2fe0 2023-09-23T03:16:44.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.035+0000 7f41ccdb6640 1 --2- 172.21.15.154:0/3109244085 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f41ac06af30 0x7f41ac06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:44.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.036+0000 7f41ccdb6640 1 --2- 172.21.15.154:0/3109244085 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f41ac06af30 0x7f41ac06d3f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f41bc035000 tx=0x7f41bc008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:44.039 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.039+0000 7f41c57fa640 1 -- 172.21.15.154:0/3109244085 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f41b40540d0 con 0x7f41c80b2fe0 2023-09-23T03:16:44.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.243+0000 7f41cf01a640 1 -- 172.21.15.154:0/3109244085 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f41c806dc60 con 0x7f41c80b2fe0 2023-09-23T03:16:44.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.243+0000 7f41c57fa640 1 -- 172.21.15.154:0/3109244085 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f41b40576f0 con 0x7f41c80b2fe0 2023-09-23T03:16:44.244 INFO:teuthology.orchestra.run.smithi154.stderr:listed 0 entries 2023-09-23T03:16:44.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.246+0000 7f41cf01a640 1 -- 172.21.15.154:0/3109244085 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f41ac06af30 msgr2=0x7f41ac06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:44.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.246+0000 7f41cf01a640 1 --2- 172.21.15.154:0/3109244085 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f41ac06af30 0x7f41ac06d3f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f41bc035000 tx=0x7f41bc008030 comp rx=0 tx=0).stop 2023-09-23T03:16:44.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.246+0000 7f41cf01a640 1 -- 172.21.15.154:0/3109244085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41c80b2fe0 msgr2=0x7f41c809b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:44.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.246+0000 7f41cf01a640 1 --2- 172.21.15.154:0/3109244085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41c80b2fe0 0x7f41c809b280 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f41b400c3f0 tx=0x7f41b400c8b0 comp rx=0 tx=0).stop 2023-09-23T03:16:44.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.246+0000 7f41cf01a640 1 -- 172.21.15.154:0/3109244085 shutdown_connections 2023-09-23T03:16:44.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.246+0000 7f41cf01a640 1 --2- 172.21.15.154:0/3109244085 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f41ac06af30 0x7f41ac06d3f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:44.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.246+0000 7f41cf01a640 1 --2- 172.21.15.154:0/3109244085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f41c80cf130 0x7f41c8156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:44.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.246+0000 7f41cf01a640 1 --2- 172.21.15.154:0/3109244085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f41c80b2fe0 0x7f41c809b280 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:44.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.246+0000 7f41cf01a640 1 --2- 172.21.15.154:0/3109244085 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f41c80ada30 0x7f41c815c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:44.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.246+0000 7f41cf01a640 1 -- 172.21.15.154:0/3109244085 >> 172.21.15.154:0/3109244085 conn(0x7f41c806a0f0 msgr2=0x7f41c80caf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:44.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.247+0000 7f41cf01a640 1 -- 172.21.15.154:0/3109244085 shutdown_connections 2023-09-23T03:16:44.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.247+0000 7f41cf01a640 1 -- 172.21.15.154:0/3109244085 wait complete. 2023-09-23T03:16:44.264 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:16:44.264 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:16:44.314 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:16:44.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.510+0000 7f14d8893640 1 Processor -- start 2023-09-23T03:16:44.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.511+0000 7f14d8893640 1 -- start start 2023-09-23T03:16:44.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.512+0000 7f14d8893640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f14d40c4970 0x7f14d4071870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:44.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.512+0000 7f14d8893640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14d406c470 0x7f14d40ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:44.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.512+0000 7f14d8893640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14d4159a00 con 0x7f14d40ca260 2023-09-23T03:16:44.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.512+0000 7f14d8893640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14d40cbe90 con 0x7f14d40c4970 2023-09-23T03:16:44.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.512+0000 7f14d8893640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d4071f70 con 0x7f14d406c470 2023-09-23T03:16:44.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.513+0000 7f14cbfff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f14d40ca260 0x7f14d406bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:36284/0 (socket says 172.21.15.154:36284) 2023-09-23T03:16:44.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.513+0000 7f14cbfff640 1 -- 172.21.15.154:0/126652883 learned_addr learned my addr 172.21.15.154:0/126652883 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:44.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.513+0000 7f14d2d9d640 1 --2- 172.21.15.154:0/126652883 >> v2:172.21.15.190:3300/0 conn(0x7f14d40c4970 0x7f14d4071870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:44.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.513+0000 7f14d359e640 1 --2- 172.21.15.154:0/126652883 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14d406c470 0x7f14d40ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:44.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.513+0000 7f14d259c640 1 -- 172.21.15.154:0/126652883 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2997518247 0 0) 0x7f14d4159a00 con 0x7f14d40ca260 2023-09-23T03:16:44.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.513+0000 7f14d259c640 1 -- 172.21.15.154:0/126652883 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14c0003520 con 0x7f14d40ca260 2023-09-23T03:16:44.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.513+0000 7f14d259c640 1 -- 172.21.15.154:0/126652883 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1708946080 0 0) 0x7f14c0003520 con 0x7f14d40ca260 2023-09-23T03:16:44.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.513+0000 7f14d259c640 1 -- 172.21.15.154:0/126652883 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14d406c470 msgr2=0x7f14d40ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:44.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.513+0000 7f14d259c640 1 --2- 172.21.15.154:0/126652883 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14d406c470 0x7f14d40ccc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:44.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.513+0000 7f14d259c640 1 -- 172.21.15.154:0/126652883 >> v2:172.21.15.190:3300/0 conn(0x7f14d40c4970 msgr2=0x7f14d4071870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:44.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.513+0000 7f14d259c640 1 --2- 172.21.15.154:0/126652883 >> v2:172.21.15.190:3300/0 conn(0x7f14d40c4970 0x7f14d4071870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:44.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.513+0000 7f14d259c640 1 -- 172.21.15.154:0/126652883 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d40cc150 con 0x7f14d40ca260 2023-09-23T03:16:44.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.513+0000 7f14d259c640 1 -- 172.21.15.154:0/126652883 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f14bc001670 con 0x7f14d40ca260 2023-09-23T03:16:44.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.513+0000 7f14d259c640 1 -- 172.21.15.154:0/126652883 >> v1:172.21.15.154:6789/0 conn(0x7f14d40ca260 legacy=0x7f14d406bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:44.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.514+0000 7f14d259c640 1 --2- 172.21.15.154:0/126652883 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14c0003970 0x7f14c0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:44.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.514+0000 7f14d259c640 1 --2- 172.21.15.154:0/126652883 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14d406c470 0x7f14c00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:44.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.514+0000 7f14d259c640 1 --2- 172.21.15.154:0/126652883 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14c0004d00 0x7f14c0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:44.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.514+0000 7f14d259c640 1 -- 172.21.15.154:0/126652883 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d40cbe90 con 0x7f14c0004d00 2023-09-23T03:16:44.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.514+0000 7f14d259c640 1 -- 172.21.15.154:0/126652883 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d4071f70 con 0x7f14c0003970 2023-09-23T03:16:44.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.514+0000 7f14d259c640 1 -- 172.21.15.154:0/126652883 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14c000b980 con 0x7f14d406c470 2023-09-23T03:16:44.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.514+0000 7f14d359e640 1 --2- 172.21.15.154:0/126652883 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14c0004d00 0x7f14c0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:44.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.514+0000 7f14d2d9d640 1 --2- 172.21.15.154:0/126652883 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14c0003970 0x7f14c0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:44.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.514+0000 7f14cbfff640 1 --2- 172.21.15.154:0/126652883 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14d406c470 0x7f14c00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:44.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.514+0000 7f14d359e640 1 -- 172.21.15.154:0/126652883 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14d406c470 msgr2=0x7f14c00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:44.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.514+0000 7f14d359e640 1 --2- 172.21.15.154:0/126652883 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14d406c470 0x7f14c00047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:44.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.514+0000 7f14d359e640 1 -- 172.21.15.154:0/126652883 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14c0003970 msgr2=0x7f14c0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:44.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.514+0000 7f14d359e640 1 --2- 172.21.15.154:0/126652883 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14c0003970 0x7f14c0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:44.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.514+0000 7f14d359e640 1 -- 172.21.15.154:0/126652883 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c0003520 con 0x7f14c0004d00 2023-09-23T03:16:44.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.515+0000 7f14d359e640 1 --2- 172.21.15.154:0/126652883 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14c0004d00 0x7f14c0005180 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f14cc009fc0 tx=0x7f14c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f345a88af8b9f1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:44.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.516+0000 7f14d259c640 1 -- 172.21.15.154:0/126652883 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14cc02cc20 con 0x7f14c0004d00 2023-09-23T03:16:44.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.516+0000 7f14d259c640 1 -- 172.21.15.154:0/126652883 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14cc00f040 con 0x7f14c0004d00 2023-09-23T03:16:44.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.516+0000 7f14d259c640 1 -- 172.21.15.154:0/126652883 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14cc034440 con 0x7f14c0004d00 2023-09-23T03:16:44.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.516+0000 7f14d8893640 1 -- 172.21.15.154:0/126652883 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14c0004d00 msgr2=0x7f14c0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:44.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.516+0000 7f14d8893640 1 --2- 172.21.15.154:0/126652883 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14c0004d00 0x7f14c0005180 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f14cc009fc0 tx=0x7f14c00057f0 comp rx=0 tx=0).stop 2023-09-23T03:16:44.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.516+0000 7f14d359e640 1 -- 172.21.15.154:0/126652883 reap_dead start 2023-09-23T03:16:44.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.516+0000 7f14d8893640 1 -- 172.21.15.154:0/126652883 shutdown_connections 2023-09-23T03:16:44.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.516+0000 7f14d8893640 1 -- 172.21.15.154:0/126652883 >> 172.21.15.154:0/126652883 conn(0x7f14d406a390 msgr2=0x7f14d40afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:44.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.516+0000 7f14d8893640 1 -- 172.21.15.154:0/126652883 shutdown_connections 2023-09-23T03:16:44.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.516+0000 7f14d8893640 1 -- 172.21.15.154:0/126652883 wait complete. 2023-09-23T03:16:44.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.518+0000 7f14d8893640 1 Processor -- start 2023-09-23T03:16:44.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.518+0000 7f14d8893640 1 -- start start 2023-09-23T03:16:44.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.519+0000 7f14d8893640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14c0004d00 0x7f14d414fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:44.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.519+0000 7f14d8893640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14d414b300 0x7f14d414d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:44.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.519+0000 7f14d8893640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14d414f1f0 0x7f14d414d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:44.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.519+0000 7f14d8893640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d414c410 con 0x7f14c0004d00 2023-09-23T03:16:44.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.519+0000 7f14d8893640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d414b790 con 0x7f14d414f1f0 2023-09-23T03:16:44.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.519+0000 7f14d8893640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d414ba70 con 0x7f14d414b300 2023-09-23T03:16:44.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.519+0000 7f14d2d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14c0004d00 0x7f14d414fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:44.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.519+0000 7f14d2d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14c0004d00 0x7f14d414fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:41038/0 (socket says 172.21.15.154:41038) 2023-09-23T03:16:44.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.519+0000 7f14d2d9d640 1 -- 172.21.15.154:0/2262222532 learned_addr learned my addr 172.21.15.154:0/2262222532 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:44.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.520+0000 7f14d2d9d640 1 -- 172.21.15.154:0/2262222532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14d414b300 msgr2=0x7f14d414d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:44.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.520+0000 7f14d2d9d640 1 --2- 172.21.15.154:0/2262222532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14d414b300 0x7f14d414d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:44.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.520+0000 7f14d2d9d640 1 -- 172.21.15.154:0/2262222532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14d414f1f0 msgr2=0x7f14d414d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:44.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.520+0000 7f14d2d9d640 1 --2- 172.21.15.154:0/2262222532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14d414f1f0 0x7f14d414d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:44.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.520+0000 7f14d2d9d640 1 -- 172.21.15.154:0/2262222532 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d414bd50 con 0x7f14c0004d00 2023-09-23T03:16:44.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.520+0000 7f14d2d9d640 1 --2- 172.21.15.154:0/2262222532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14c0004d00 0x7f14d414fe80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f14c40116d0 tx=0x7f14c4011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:44.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.520+0000 7f14cb7fe640 1 -- 172.21.15.154:0/2262222532 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c400f940 con 0x7f14c0004d00 2023-09-23T03:16:44.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.520+0000 7f14cb7fe640 1 -- 172.21.15.154:0/2262222532 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c400faa0 con 0x7f14c0004d00 2023-09-23T03:16:44.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.520+0000 7f14cb7fe640 1 -- 172.21.15.154:0/2262222532 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c400fe10 con 0x7f14c0004d00 2023-09-23T03:16:44.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.520+0000 7f14d8893640 1 -- 172.21.15.154:0/2262222532 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14d414c030 con 0x7f14c0004d00 2023-09-23T03:16:44.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.520+0000 7f14d8893640 1 -- 172.21.15.154:0/2262222532 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d41505a0 con 0x7f14c0004d00 2023-09-23T03:16:44.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.522+0000 7f14d8893640 1 -- 172.21.15.154:0/2262222532 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14d40736b0 con 0x7f14c0004d00 2023-09-23T03:16:44.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.522+0000 7f14cb7fe640 1 -- 172.21.15.154:0/2262222532 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f14c4015460 con 0x7f14c0004d00 2023-09-23T03:16:44.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.523+0000 7f14cb7fe640 1 --2- 172.21.15.154:0/2262222532 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f14b406af30 0x7f14b406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:44.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.523+0000 7f14cb7fe640 1 -- 172.21.15.154:0/2262222532 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f14c4089e00 con 0x7f14c0004d00 2023-09-23T03:16:44.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.524+0000 7f14cbfff640 1 --2- 172.21.15.154:0/2262222532 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f14b406af30 0x7f14b406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:44.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.524+0000 7f14cbfff640 1 --2- 172.21.15.154:0/2262222532 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f14b406af30 0x7f14b406d3f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f14d415ccd0 tx=0x7f14bc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:44.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.528+0000 7f14cb7fe640 1 -- 172.21.15.154:0/2262222532 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f14c4054a80 con 0x7f14c0004d00 2023-09-23T03:16:44.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.717+0000 7f14d8893640 1 -- 172.21.15.154:0/2262222532 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14d40cbd00 con 0x7f14c0004d00 2023-09-23T03:16:44.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.720+0000 7f14cb7fe640 1 -- 172.21.15.154:0/2262222532 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18744 (secure 0 0 0) 0x7f14c4022090 con 0x7f14c0004d00 2023-09-23T03:16:44.721 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:16:44.721 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":26,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:16:03.151371+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:15:18.575849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T03:15:19.707212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T03:15:23.337138+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:16:44.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.723+0000 7f14d8893640 1 -- 172.21.15.154:0/2262222532 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f14b406af30 msgr2=0x7f14b406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:44.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.723+0000 7f14d8893640 1 --2- 172.21.15.154:0/2262222532 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f14b406af30 0x7f14b406d3f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f14d415ccd0 tx=0x7f14bc009040 comp rx=0 tx=0).stop 2023-09-23T03:16:44.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.723+0000 7f14d8893640 1 -- 172.21.15.154:0/2262222532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14c0004d00 msgr2=0x7f14d414fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:44.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.723+0000 7f14d8893640 1 --2- 172.21.15.154:0/2262222532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14c0004d00 0x7f14d414fe80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f14c40116d0 tx=0x7f14c4011b90 comp rx=0 tx=0).stop 2023-09-23T03:16:44.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.723+0000 7f14d8893640 1 -- 172.21.15.154:0/2262222532 shutdown_connections 2023-09-23T03:16:44.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.723+0000 7f14d8893640 1 --2- 172.21.15.154:0/2262222532 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f14b406af30 0x7f14b406d3f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:44.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.723+0000 7f14d8893640 1 --2- 172.21.15.154:0/2262222532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14d414f1f0 0x7f14d414d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:44.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.723+0000 7f14d8893640 1 --2- 172.21.15.154:0/2262222532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14d414b300 0x7f14d414d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:44.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.723+0000 7f14d8893640 1 --2- 172.21.15.154:0/2262222532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14c0004d00 0x7f14d414fe80 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:44.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.723+0000 7f14d8893640 1 -- 172.21.15.154:0/2262222532 >> 172.21.15.154:0/2262222532 conn(0x7f14d406a390 msgr2=0x7f14d40ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:44.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.723+0000 7f14d8893640 1 -- 172.21.15.154:0/2262222532 shutdown_connections 2023-09-23T03:16:44.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.723+0000 7f14d8893640 1 -- 172.21.15.154:0/2262222532 wait complete. 2023-09-23T03:16:44.742 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:15:18.575849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:15:19.707212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-23T03:15:23.337138+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-23T03:16:44.742 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:16:44.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.959+0000 7fec7a561640 1 Processor -- start 2023-09-23T03:16:44.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.960+0000 7fec7a561640 1 -- start start 2023-09-23T03:16:44.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.961+0000 7fec7a561640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec740ce940 0x7fec741677f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:44.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.961+0000 7fec7a561640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec740d0bb0 con 0x7fec7415c9c0 2023-09-23T03:16:44.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.961+0000 7fec7a561640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec740cd450 con 0x7fec7415cda0 2023-09-23T03:16:44.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.962+0000 7fec7a561640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec740d0df0 con 0x7fec740ce940 2023-09-23T03:16:44.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.962+0000 7fec73fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fec7415c9c0 0x7fec740d04a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:36290/0 (socket says 172.21.15.154:36290) 2023-09-23T03:16:44.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.962+0000 7fec73fff640 1 -- 172.21.15.154:0/2041138242 learned_addr learned my addr 172.21.15.154:0/2041138242 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:44.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.962+0000 7fec78afe640 1 --2- 172.21.15.154:0/2041138242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec740ce940 0x7fec741677f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:44.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.962+0000 7fec72ffd640 1 -- 172.21.15.154:0/2041138242 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2051674394 0 0) 0x7fec64001610 con 0x7fec7415c9c0 2023-09-23T03:16:44.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.962+0000 7fec72ffd640 1 -- 172.21.15.154:0/2041138242 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec60003520 con 0x7fec7415c9c0 2023-09-23T03:16:44.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.963+0000 7fec72ffd640 1 -- 172.21.15.154:0/2041138242 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1186961282 0 0) 0x7fec60003520 con 0x7fec7415c9c0 2023-09-23T03:16:44.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.963+0000 7fec72ffd640 1 -- 172.21.15.154:0/2041138242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec740ce940 msgr2=0x7fec741677f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:44.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.963+0000 7fec72ffd640 1 --2- 172.21.15.154:0/2041138242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec740ce940 0x7fec741677f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:44.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.963+0000 7fec72ffd640 1 -- 172.21.15.154:0/2041138242 >> v1:172.21.15.190:6789/0 conn(0x7fec7415cda0 legacy=0x7fec740d2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:44.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.963+0000 7fec72ffd640 1 -- 172.21.15.154:0/2041138242 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec740d10d0 con 0x7fec7415c9c0 2023-09-23T03:16:44.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.963+0000 7fec72ffd640 1 -- 172.21.15.154:0/2041138242 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fec64001d20 con 0x7fec7415c9c0 2023-09-23T03:16:44.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.963+0000 7fec78afe640 1 --2- 172.21.15.154:0/2041138242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec740ce940 0x7fec741677f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:16:44.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.963+0000 7fec72ffd640 1 -- 172.21.15.154:0/2041138242 >> v1:172.21.15.154:6789/0 conn(0x7fec7415c9c0 legacy=0x7fec740d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:44.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.963+0000 7fec72ffd640 1 --2- 172.21.15.154:0/2041138242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec740ce940 0x7fec600039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:44.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.963+0000 7fec72ffd640 1 --2- 172.21.15.154:0/2041138242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec60003fe0 0x7fec600047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:44.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.963+0000 7fec72ffd640 1 --2- 172.21.15.154:0/2041138242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec60004d00 0x7fec60005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:44.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.963+0000 7fec72ffd640 1 -- 172.21.15.154:0/2041138242 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec740d0df0 con 0x7fec60004d00 2023-09-23T03:16:44.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.963+0000 7fec72ffd640 1 -- 172.21.15.154:0/2041138242 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6000bbc0 con 0x7fec60003fe0 2023-09-23T03:16:44.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.963+0000 7fec72ffd640 1 -- 172.21.15.154:0/2041138242 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec6000bd80 con 0x7fec740ce940 2023-09-23T03:16:44.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.963+0000 7fec78afe640 1 --2- 172.21.15.154:0/2041138242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec60004d00 0x7fec60005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:44.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.963+0000 7fec737fe640 1 --2- 172.21.15.154:0/2041138242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec60003fe0 0x7fec600047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:44.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.963+0000 7fec78afe640 1 -- 172.21.15.154:0/2041138242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec740ce940 msgr2=0x7fec600039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:44.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.963+0000 7fec78afe640 1 --2- 172.21.15.154:0/2041138242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec740ce940 0x7fec600039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:44.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.963+0000 7fec78afe640 1 -- 172.21.15.154:0/2041138242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec60003fe0 msgr2=0x7fec600047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:44.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.963+0000 7fec78afe640 1 --2- 172.21.15.154:0/2041138242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec60003fe0 0x7fec600047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:44.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.963+0000 7fec78afe640 1 -- 172.21.15.154:0/2041138242 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec60003520 con 0x7fec60004d00 2023-09-23T03:16:44.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.963+0000 7fec737fe640 1 --2- 172.21.15.154:0/2041138242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec60003fe0 0x7fec600047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:44.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.964+0000 7fec78afe640 1 --2- 172.21.15.154:0/2041138242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec60004d00 0x7fec60005180 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0098a0 tx=0x7fec600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f89c90fffcbe28 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:44.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.965+0000 7fec72ffd640 1 -- 172.21.15.154:0/2041138242 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c004230 con 0x7fec60004d00 2023-09-23T03:16:44.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.965+0000 7fec72ffd640 1 -- 172.21.15.154:0/2041138242 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec6c004390 con 0x7fec60004d00 2023-09-23T03:16:44.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.965+0000 7fec72ffd640 1 -- 172.21.15.154:0/2041138242 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c003830 con 0x7fec60004d00 2023-09-23T03:16:44.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.965+0000 7fec7a561640 1 -- 172.21.15.154:0/2041138242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec60004d00 msgr2=0x7fec60005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:44.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.965+0000 7fec7a561640 1 --2- 172.21.15.154:0/2041138242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec60004d00 0x7fec60005180 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0098a0 tx=0x7fec600057f0 comp rx=0 tx=0).stop 2023-09-23T03:16:44.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.965+0000 7fec78afe640 1 -- 172.21.15.154:0/2041138242 reap_dead start 2023-09-23T03:16:44.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.965+0000 7fec7a561640 1 -- 172.21.15.154:0/2041138242 shutdown_connections 2023-09-23T03:16:44.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.965+0000 7fec7a561640 1 -- 172.21.15.154:0/2041138242 >> 172.21.15.154:0/2041138242 conn(0x7fec7409bc20 msgr2=0x7fec74089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:44.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.965+0000 7fec7a561640 1 -- 172.21.15.154:0/2041138242 shutdown_connections 2023-09-23T03:16:44.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.965+0000 7fec7a561640 1 -- 172.21.15.154:0/2041138242 wait complete. 2023-09-23T03:16:44.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.967+0000 7fec7a561640 1 Processor -- start 2023-09-23T03:16:44.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.967+0000 7fec7a561640 1 -- start start 2023-09-23T03:16:44.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.968+0000 7fec7a561640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec60004d00 0x7fec7415fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:44.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.968+0000 7fec7a561640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec741f1cb0 0x7fec741f2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:44.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.968+0000 7fec7a561640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec740b44b0 0x7fec740b4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:44.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.968+0000 7fec7a561640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec741f3260 con 0x7fec60004d00 2023-09-23T03:16:44.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.968+0000 7fec7a561640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec740b4e70 con 0x7fec741f1cb0 2023-09-23T03:16:44.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.968+0000 7fec7a561640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec741f6760 con 0x7fec740b44b0 2023-09-23T03:16:44.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.969+0000 7fec73fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec60004d00 0x7fec7415fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:44.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.969+0000 7fec73fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec60004d00 0x7fec7415fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:41054/0 (socket says 172.21.15.154:41054) 2023-09-23T03:16:44.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.969+0000 7fec73fff640 1 -- 172.21.15.154:0/3558238396 learned_addr learned my addr 172.21.15.154:0/3558238396 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:44.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.969+0000 7fec73fff640 1 -- 172.21.15.154:0/3558238396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec740b44b0 msgr2=0x7fec740b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:44.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.969+0000 7fec73fff640 1 --2- 172.21.15.154:0/3558238396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec740b44b0 0x7fec740b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:44.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.969+0000 7fec737fe640 1 --2- 172.21.15.154:0/3558238396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec741f1cb0 0x7fec741f2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:44.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.969+0000 7fec73fff640 1 -- 172.21.15.154:0/3558238396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec741f1cb0 msgr2=0x7fec741f2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:44.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.969+0000 7fec73fff640 1 --2- 172.21.15.154:0/3558238396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec741f1cb0 0x7fec741f2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:44.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.969+0000 7fec73fff640 1 -- 172.21.15.154:0/3558238396 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec741600e0 con 0x7fec60004d00 2023-09-23T03:16:44.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.969+0000 7fec737fe640 1 --2- 172.21.15.154:0/3558238396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec741f1cb0 0x7fec741f2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:44.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.969+0000 7fec73fff640 1 --2- 172.21.15.154:0/3558238396 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec60004d00 0x7fec7415fad0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fec6400ca50 tx=0x7fec6400cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:44.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.969+0000 7fec70ff9640 1 -- 172.21.15.154:0/3558238396 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6400a720 con 0x7fec60004d00 2023-09-23T03:16:44.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.970+0000 7fec70ff9640 1 -- 172.21.15.154:0/3558238396 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec6400a880 con 0x7fec60004d00 2023-09-23T03:16:44.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.970+0000 7fec70ff9640 1 -- 172.21.15.154:0/3558238396 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6400ac30 con 0x7fec60004d00 2023-09-23T03:16:44.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.970+0000 7fec7a561640 1 -- 172.21.15.154:0/3558238396 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec741f2130 con 0x7fec60004d00 2023-09-23T03:16:44.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.970+0000 7fec7a561640 1 -- 172.21.15.154:0/3558238396 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec741f5c80 con 0x7fec60004d00 2023-09-23T03:16:44.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.971+0000 7fec7a561640 1 -- 172.21.15.154:0/3558238396 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec7404e920 con 0x7fec60004d00 2023-09-23T03:16:44.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.972+0000 7fec70ff9640 1 -- 172.21.15.154:0/3558238396 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fec64007070 con 0x7fec60004d00 2023-09-23T03:16:44.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.973+0000 7fec70ff9640 1 --2- 172.21.15.154:0/3558238396 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fec5406af30 0x7fec5406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:44.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.973+0000 7fec70ff9640 1 -- 172.21.15.154:0/3558238396 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7fec64089490 con 0x7fec60004d00 2023-09-23T03:16:44.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.974+0000 7fec737fe640 1 --2- 172.21.15.154:0/3558238396 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fec5406af30 0x7fec5406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:44.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.974+0000 7fec737fe640 1 --2- 172.21.15.154:0/3558238396 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fec5406af30 0x7fec5406d3f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fec74159980 tx=0x7fec5c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:44.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:44.978+0000 7fec70ff9640 1 -- 172.21.15.154:0/3558238396 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fec64054110 con 0x7fec60004d00 2023-09-23T03:16:45.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.164+0000 7fec7a561640 1 -- 172.21.15.154:0/3558238396 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fec740b2810 con 0x7fec60004d00 2023-09-23T03:16:45.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.165+0000 7fec70ff9640 1 -- 172.21.15.154:0/3558238396 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fec64017020 con 0x7fec60004d00 2023-09-23T03:16:45.165 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:16:45.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.169+0000 7fec7a561640 1 -- 172.21.15.154:0/3558238396 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fec5406af30 msgr2=0x7fec5406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:45.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.169+0000 7fec7a561640 1 --2- 172.21.15.154:0/3558238396 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fec5406af30 0x7fec5406d3f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fec74159980 tx=0x7fec5c009040 comp rx=0 tx=0).stop 2023-09-23T03:16:45.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.169+0000 7fec7a561640 1 -- 172.21.15.154:0/3558238396 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec60004d00 msgr2=0x7fec7415fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:45.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.169+0000 7fec7a561640 1 --2- 172.21.15.154:0/3558238396 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec60004d00 0x7fec7415fad0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fec6400ca50 tx=0x7fec6400cf10 comp rx=0 tx=0).stop 2023-09-23T03:16:45.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.169+0000 7fec7a561640 1 -- 172.21.15.154:0/3558238396 shutdown_connections 2023-09-23T03:16:45.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.169+0000 7fec7a561640 1 --2- 172.21.15.154:0/3558238396 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fec5406af30 0x7fec5406d3f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:45.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.169+0000 7fec7a561640 1 --2- 172.21.15.154:0/3558238396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec740b44b0 0x7fec740b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:45.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.169+0000 7fec7a561640 1 --2- 172.21.15.154:0/3558238396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec741f1cb0 0x7fec741f2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:45.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.169+0000 7fec7a561640 1 --2- 172.21.15.154:0/3558238396 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec60004d00 0x7fec7415fad0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:45.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.169+0000 7fec7a561640 1 -- 172.21.15.154:0/3558238396 >> 172.21.15.154:0/3558238396 conn(0x7fec7409bc20 msgr2=0x7fec740d2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:45.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.169+0000 7fec7a561640 1 -- 172.21.15.154:0/3558238396 shutdown_connections 2023-09-23T03:16:45.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.169+0000 7fec7a561640 1 -- 172.21.15.154:0/3558238396 wait complete. 2023-09-23T03:16:45.187 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:16:45.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.407+0000 7f967ffff640 1 Processor -- start 2023-09-23T03:16:45.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.407+0000 7f967ffff640 1 -- start start 2023-09-23T03:16:45.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.408+0000 7f967ffff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9680158480 0x7f96800cade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:45.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.409+0000 7f967ffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f968015ba50 0x7f968006fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:45.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.409+0000 7f967ffff640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96800cb6c0 con 0x7f968015eef0 2023-09-23T03:16:45.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.409+0000 7f967ffff640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f968015ad80 con 0x7f968015ba50 2023-09-23T03:16:45.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.409+0000 7f967ffff640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f968015af70 con 0x7f9680158480 2023-09-23T03:16:45.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.409+0000 7f9677fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f968015ba50 0x7f968006fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:45.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.409+0000 7f9677fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f968015ba50 0x7f968006fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:41068/0 (socket says 172.21.15.154:41068) 2023-09-23T03:16:45.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.409+0000 7f9677fff640 1 -- 172.21.15.154:0/1447157593 learned_addr learned my addr 172.21.15.154:0/1447157593 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:45.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.409+0000 7f967effd640 1 --2- 172.21.15.154:0/1447157593 >> v2:172.21.15.190:3300/0 conn(0x7f9680158480 0x7f96800cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:45.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.409+0000 7f9677fff640 1 -- 172.21.15.154:0/1447157593 >> v2:172.21.15.190:3300/0 conn(0x7f9680158480 msgr2=0x7f96800cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:45.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.410+0000 7f9677fff640 1 --2- 172.21.15.154:0/1447157593 >> v2:172.21.15.190:3300/0 conn(0x7f9680158480 0x7f96800cade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:45.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.410+0000 7f9677fff640 1 -- 172.21.15.154:0/1447157593 >> v1:172.21.15.190:6789/0 conn(0x7f968015eef0 legacy=0x7f968006bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:45.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.410+0000 7f9677fff640 1 -- 172.21.15.154:0/1447157593 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f968015b250 con 0x7f968015ba50 2023-09-23T03:16:45.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.410+0000 7f967effd640 1 --2- 172.21.15.154:0/1447157593 >> v2:172.21.15.190:3300/0 conn(0x7f9680158480 0x7f96800cade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:45.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.411+0000 7f9677fff640 1 --2- 172.21.15.154:0/1447157593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f968015ba50 0x7f968006fe90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f968006d1d0 tx=0x7f966802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=261c42f3fa6ee0ec server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:45.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.411+0000 7f967e7fc640 1 -- 172.21.15.154:0/1447157593 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9668031870 con 0x7f968015ba50 2023-09-23T03:16:45.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.411+0000 7f967e7fc640 1 -- 172.21.15.154:0/1447157593 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96680319d0 con 0x7f968015ba50 2023-09-23T03:16:45.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.411+0000 7f967e7fc640 1 -- 172.21.15.154:0/1447157593 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9668031d60 con 0x7f968015ba50 2023-09-23T03:16:45.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.412+0000 7f967ffff640 1 -- 172.21.15.154:0/1447157593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f968015ba50 msgr2=0x7f968006fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:45.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.412+0000 7f967ffff640 1 --2- 172.21.15.154:0/1447157593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f968015ba50 0x7f968006fe90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f968006d1d0 tx=0x7f966802b910 comp rx=0 tx=0).stop 2023-09-23T03:16:45.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.412+0000 7f967ffff640 1 -- 172.21.15.154:0/1447157593 shutdown_connections 2023-09-23T03:16:45.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.412+0000 7f967ffff640 1 --2- 172.21.15.154:0/1447157593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f968015ba50 0x7f968006fe90 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:45.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.412+0000 7f967ffff640 1 --2- 172.21.15.154:0/1447157593 >> v2:172.21.15.190:3300/0 conn(0x7f9680158480 0x7f96800cade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:45.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.412+0000 7f967ffff640 1 -- 172.21.15.154:0/1447157593 >> 172.21.15.154:0/1447157593 conn(0x7f96800aff20 msgr2=0x7f96800b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:45.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.412+0000 7f967ffff640 1 -- 172.21.15.154:0/1447157593 shutdown_connections 2023-09-23T03:16:45.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.412+0000 7f967ffff640 1 -- 172.21.15.154:0/1447157593 wait complete. 2023-09-23T03:16:45.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.414+0000 7f967ffff640 1 Processor -- start 2023-09-23T03:16:45.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.414+0000 7f967ffff640 1 -- start start 2023-09-23T03:16:45.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.415+0000 7f967ffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9680158480 0x7f96801541a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:45.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.415+0000 7f967ffff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f968015ba50 0x7f96800c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:45.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.415+0000 7f967ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f968015eef0 0x7f96800ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:45.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.415+0000 7f967ffff640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96801577a0 con 0x7f968015eef0 2023-09-23T03:16:45.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.415+0000 7f967ffff640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968015db50 con 0x7f9680158480 2023-09-23T03:16:45.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.415+0000 7f967ffff640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96801546e0 con 0x7f968015ba50 2023-09-23T03:16:45.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.415+0000 7f967f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f968015eef0 0x7f96800ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:45.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.415+0000 7f967f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f968015eef0 0x7f96800ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:41074/0 (socket says 172.21.15.154:41074) 2023-09-23T03:16:45.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.415+0000 7f967f7fe640 1 -- 172.21.15.154:0/2129603559 learned_addr learned my addr 172.21.15.154:0/2129603559 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:45.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.416+0000 7f9677fff640 1 --2- 172.21.15.154:0/2129603559 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f968015ba50 0x7f96800c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:45.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.416+0000 7f967f7fe640 1 -- 172.21.15.154:0/2129603559 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f968015ba50 msgr2=0x7f96800c9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:45.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.416+0000 7f967f7fe640 1 --2- 172.21.15.154:0/2129603559 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f968015ba50 0x7f96800c9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:45.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.416+0000 7f967f7fe640 1 -- 172.21.15.154:0/2129603559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9680158480 msgr2=0x7f96801541a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:45.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.416+0000 7f967f7fe640 1 --2- 172.21.15.154:0/2129603559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9680158480 0x7f96801541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:45.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.416+0000 7f967f7fe640 1 -- 172.21.15.154:0/2129603559 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96680094d0 con 0x7f968015eef0 2023-09-23T03:16:45.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.416+0000 7f967f7fe640 1 --2- 172.21.15.154:0/2129603559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f968015eef0 0x7f96800ca1e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f967800b970 tx=0x7f967800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:45.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.416+0000 7f96777fe640 1 -- 172.21.15.154:0/2129603559 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967800cb40 con 0x7f968015eef0 2023-09-23T03:16:45.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.416+0000 7f96777fe640 1 -- 172.21.15.154:0/2129603559 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f967800cca0 con 0x7f968015eef0 2023-09-23T03:16:45.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.416+0000 7f967ffff640 1 -- 172.21.15.154:0/2129603559 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f968015cf40 con 0x7f968015eef0 2023-09-23T03:16:45.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.416+0000 7f96777fe640 1 -- 172.21.15.154:0/2129603559 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9678014680 con 0x7f968015eef0 2023-09-23T03:16:45.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.417+0000 7f967ffff640 1 -- 172.21.15.154:0/2129603559 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f968015d500 con 0x7f968015eef0 2023-09-23T03:16:45.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.418+0000 7f967ffff640 1 -- 172.21.15.154:0/2129603559 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f968004e920 con 0x7f968015eef0 2023-09-23T03:16:45.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.419+0000 7f96777fe640 1 -- 172.21.15.154:0/2129603559 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f967801d3b0 con 0x7f968015eef0 2023-09-23T03:16:45.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.419+0000 7f96777fe640 1 --2- 172.21.15.154:0/2129603559 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f966406af60 0x7f966406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:45.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.420+0000 7f96777fe640 1 -- 172.21.15.154:0/2129603559 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f9678089980 con 0x7f968015eef0 2023-09-23T03:16:45.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.420+0000 7f967effd640 1 --2- 172.21.15.154:0/2129603559 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f966406af60 0x7f966406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:45.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.421+0000 7f967effd640 1 --2- 172.21.15.154:0/2129603559 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f966406af60 0x7f966406d420 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9670006fd0 tx=0x7f9670008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:45.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.424+0000 7f96777fe640 1 -- 172.21.15.154:0/2129603559 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f9678054570 con 0x7f968015eef0 2023-09-23T03:16:45.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.618+0000 7f967ffff640 1 -- 172.21.15.154:0/2129603559 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f968015dc90 con 0x7f968015eef0 2023-09-23T03:16:45.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.619+0000 7f96777fe640 1 -- 172.21.15.154:0/2129603559 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f968015dc90 con 0x7f968015eef0 2023-09-23T03:16:45.619 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:16:45.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.622+0000 7f967ffff640 1 -- 172.21.15.154:0/2129603559 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f966406af60 msgr2=0x7f966406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:45.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.622+0000 7f967ffff640 1 --2- 172.21.15.154:0/2129603559 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f966406af60 0x7f966406d420 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9670006fd0 tx=0x7f9670008040 comp rx=0 tx=0).stop 2023-09-23T03:16:45.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.622+0000 7f967ffff640 1 -- 172.21.15.154:0/2129603559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f968015eef0 msgr2=0x7f96800ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:45.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.622+0000 7f967ffff640 1 --2- 172.21.15.154:0/2129603559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f968015eef0 0x7f96800ca1e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f967800b970 tx=0x7f967800be30 comp rx=0 tx=0).stop 2023-09-23T03:16:45.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.622+0000 7f967ffff640 1 -- 172.21.15.154:0/2129603559 shutdown_connections 2023-09-23T03:16:45.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.622+0000 7f967ffff640 1 --2- 172.21.15.154:0/2129603559 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f966406af60 0x7f966406d420 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:45.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.622+0000 7f967ffff640 1 --2- 172.21.15.154:0/2129603559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f968015eef0 0x7f96800ca1e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:45.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.622+0000 7f967ffff640 1 --2- 172.21.15.154:0/2129603559 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f968015ba50 0x7f96800c9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:45.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.622+0000 7f967ffff640 1 --2- 172.21.15.154:0/2129603559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9680158480 0x7f96801541a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:45.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.622+0000 7f967ffff640 1 -- 172.21.15.154:0/2129603559 >> 172.21.15.154:0/2129603559 conn(0x7f96800aff20 msgr2=0x7f96800ce3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:45.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.622+0000 7f967ffff640 1 -- 172.21.15.154:0/2129603559 shutdown_connections 2023-09-23T03:16:45.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.622+0000 7f967ffff640 1 -- 172.21.15.154:0/2129603559 wait complete. 2023-09-23T03:16:45.640 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:16:45.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.853+0000 7fc4de376640 1 Processor -- start 2023-09-23T03:16:45.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.854+0000 7fc4de376640 1 -- start start 2023-09-23T03:16:45.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.855+0000 7fc4de376640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc4d815ab20 0x7fc4d80cee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:45.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.855+0000 7fc4de376640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4d815f480 con 0x7fc4d80cf370 2023-09-23T03:16:45.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.855+0000 7fc4de376640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4d815af00 con 0x7fc4d815ecd0 2023-09-23T03:16:45.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.855+0000 7fc4de376640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4d80cd3f0 con 0x7fc4d815ab20 2023-09-23T03:16:45.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.855+0000 7fc4d77fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fc4d80cf370 0x7fc4d815e5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:36304/0 (socket says 172.21.15.154:36304) 2023-09-23T03:16:45.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.855+0000 7fc4d7fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc4d815ab20 0x7fc4d80cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:45.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.855+0000 7fc4d77fe640 1 -- 172.21.15.154:0/225775337 learned_addr learned my addr 172.21.15.154:0/225775337 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:45.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.856+0000 7fc4d6ffd640 1 -- 172.21.15.154:0/225775337 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 402362815 0 0) 0x7fc4d815f480 con 0x7fc4d80cf370 2023-09-23T03:16:45.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.856+0000 7fc4d6ffd640 1 -- 172.21.15.154:0/225775337 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4c4003520 con 0x7fc4d80cf370 2023-09-23T03:16:45.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.856+0000 7fc4d7fff640 1 -- 172.21.15.154:0/225775337 >> v1:172.21.15.190:6789/0 conn(0x7fc4d815ecd0 legacy=0x7fc4d80d0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:45.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.856+0000 7fc4d7fff640 1 -- 172.21.15.154:0/225775337 >> v1:172.21.15.154:6789/0 conn(0x7fc4d80cf370 legacy=0x7fc4d815e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:45.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.856+0000 7fc4d7fff640 1 -- 172.21.15.154:0/225775337 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d815f1a0 con 0x7fc4d815ab20 2023-09-23T03:16:45.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.857+0000 7fc4d7fff640 1 --2- 172.21.15.154:0/225775337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4d815ab20 0x7fc4d80cee30 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8009de0 tx=0x7fc4c802bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b1be3b31b961071 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:45.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.857+0000 7fc4d6ffd640 1 -- 172.21.15.154:0/225775337 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c802ccb0 con 0x7fc4d815ab20 2023-09-23T03:16:45.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.857+0000 7fc4d6ffd640 1 -- 172.21.15.154:0/225775337 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c802ce10 con 0x7fc4d815ab20 2023-09-23T03:16:45.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.857+0000 7fc4d6ffd640 1 -- 172.21.15.154:0/225775337 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c8034660 con 0x7fc4d815ab20 2023-09-23T03:16:45.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.858+0000 7fc4de376640 1 -- 172.21.15.154:0/225775337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4d815ab20 msgr2=0x7fc4d80cee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:45.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.858+0000 7fc4de376640 1 --2- 172.21.15.154:0/225775337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4d815ab20 0x7fc4d80cee30 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8009de0 tx=0x7fc4c802bf00 comp rx=0 tx=0).stop 2023-09-23T03:16:45.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.858+0000 7fc4de376640 1 -- 172.21.15.154:0/225775337 shutdown_connections 2023-09-23T03:16:45.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.858+0000 7fc4de376640 1 --2- 172.21.15.154:0/225775337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4d815ab20 0x7fc4d80cee30 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:45.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.858+0000 7fc4de376640 1 -- 172.21.15.154:0/225775337 >> 172.21.15.154:0/225775337 conn(0x7fc4d8089fd0 msgr2=0x7fc4d808a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:45.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.858+0000 7fc4de376640 1 -- 172.21.15.154:0/225775337 shutdown_connections 2023-09-23T03:16:45.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.858+0000 7fc4de376640 1 -- 172.21.15.154:0/225775337 wait complete. 2023-09-23T03:16:45.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.860+0000 7fc4de376640 1 Processor -- start 2023-09-23T03:16:45.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.860+0000 7fc4de376640 1 -- start start 2023-09-23T03:16:45.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.861+0000 7fc4de376640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4d815ab20 0x7fc4d815da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:45.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.861+0000 7fc4de376640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4d815ecd0 0x7fc4d80c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:45.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.861+0000 7fc4de376640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4d814eef0 0x7fc4d80c4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:45.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.861+0000 7fc4de376640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d815df60 con 0x7fc4d814eef0 2023-09-23T03:16:45.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.861+0000 7fc4de376640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d80ca370 con 0x7fc4d815ecd0 2023-09-23T03:16:45.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.861+0000 7fc4de376640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d80c8260 con 0x7fc4d815ab20 2023-09-23T03:16:45.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.861+0000 7fc4dc913640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4d814eef0 0x7fc4d80c4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:45.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.861+0000 7fc4dc913640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4d814eef0 0x7fc4d80c4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:41094/0 (socket says 172.21.15.154:41094) 2023-09-23T03:16:45.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.861+0000 7fc4dc913640 1 -- 172.21.15.154:0/3225460071 learned_addr learned my addr 172.21.15.154:0/3225460071 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:45.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.862+0000 7fc4d7fff640 1 --2- 172.21.15.154:0/3225460071 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4d815ab20 0x7fc4d815da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:45.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.862+0000 7fc4dc913640 1 -- 172.21.15.154:0/3225460071 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4d815ab20 msgr2=0x7fc4d815da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:45.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.862+0000 7fc4dc913640 1 --2- 172.21.15.154:0/3225460071 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4d815ab20 0x7fc4d815da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:45.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.862+0000 7fc4dc913640 1 -- 172.21.15.154:0/3225460071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4d815ecd0 msgr2=0x7fc4d80c9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:45.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.862+0000 7fc4dc913640 1 --2- 172.21.15.154:0/3225460071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4d815ecd0 0x7fc4d80c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:45.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.862+0000 7fc4dc913640 1 -- 172.21.15.154:0/3225460071 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c8009a70 con 0x7fc4d814eef0 2023-09-23T03:16:45.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.862+0000 7fc4dc913640 1 --2- 172.21.15.154:0/3225460071 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4d814eef0 0x7fc4d80c4210 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc4d000b410 tx=0x7fc4d000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:45.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.862+0000 7fc4d4ff9640 1 -- 172.21.15.154:0/3225460071 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d0004020 con 0x7fc4d814eef0 2023-09-23T03:16:45.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.862+0000 7fc4d4ff9640 1 -- 172.21.15.154:0/3225460071 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d0004180 con 0x7fc4d814eef0 2023-09-23T03:16:45.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.862+0000 7fc4d4ff9640 1 -- 172.21.15.154:0/3225460071 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d0011630 con 0x7fc4d814eef0 2023-09-23T03:16:45.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.862+0000 7fc4de376640 1 -- 172.21.15.154:0/3225460071 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4d815c780 con 0x7fc4d814eef0 2023-09-23T03:16:45.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.863+0000 7fc4de376640 1 -- 172.21.15.154:0/3225460071 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4d80c87b0 con 0x7fc4d814eef0 2023-09-23T03:16:45.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.864+0000 7fc4de376640 1 -- 172.21.15.154:0/3225460071 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4d80c89d0 con 0x7fc4d814eef0 2023-09-23T03:16:45.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.865+0000 7fc4d4ff9640 1 -- 172.21.15.154:0/3225460071 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fc4d0011790 con 0x7fc4d814eef0 2023-09-23T03:16:45.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.866+0000 7fc4d4ff9640 1 --2- 172.21.15.154:0/3225460071 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc4b806af30 0x7fc4b806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:45.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.866+0000 7fc4d4ff9640 1 -- 172.21.15.154:0/3225460071 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7fc4d00898c0 con 0x7fc4d814eef0 2023-09-23T03:16:45.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.867+0000 7fc4d7fff640 1 --2- 172.21.15.154:0/3225460071 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc4b806af30 0x7fc4b806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:45.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.867+0000 7fc4d7fff640 1 --2- 172.21.15.154:0/3225460071 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc4b806af30 0x7fc4b806d3f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8036000 tx=0x7fc4c8030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:45.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:45.871+0000 7fc4d4ff9640 1 -- 172.21.15.154:0/3225460071 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fc4d00544b0 con 0x7fc4d814eef0 2023-09-23T03:16:46.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.059+0000 7fc4de376640 1 -- 172.21.15.154:0/3225460071 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc4d804e920 con 0x7fc4d814eef0 2023-09-23T03:16:46.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.060+0000 7fc4d4ff9640 1 -- 172.21.15.154:0/3225460071 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc4d0057ad0 con 0x7fc4d814eef0 2023-09-23T03:16:46.060 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:16:46.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.062+0000 7fc4de376640 1 -- 172.21.15.154:0/3225460071 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc4b806af30 msgr2=0x7fc4b806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:46.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.062+0000 7fc4de376640 1 --2- 172.21.15.154:0/3225460071 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc4b806af30 0x7fc4b806d3f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8036000 tx=0x7fc4c8030000 comp rx=0 tx=0).stop 2023-09-23T03:16:46.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.062+0000 7fc4de376640 1 -- 172.21.15.154:0/3225460071 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4d814eef0 msgr2=0x7fc4d80c4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:46.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.062+0000 7fc4de376640 1 --2- 172.21.15.154:0/3225460071 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4d814eef0 0x7fc4d80c4210 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc4d000b410 tx=0x7fc4d000b8d0 comp rx=0 tx=0).stop 2023-09-23T03:16:46.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.063+0000 7fc4de376640 1 -- 172.21.15.154:0/3225460071 shutdown_connections 2023-09-23T03:16:46.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.063+0000 7fc4de376640 1 --2- 172.21.15.154:0/3225460071 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc4b806af30 0x7fc4b806d3f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:46.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.063+0000 7fc4de376640 1 --2- 172.21.15.154:0/3225460071 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4d814eef0 0x7fc4d80c4210 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:46.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.063+0000 7fc4de376640 1 --2- 172.21.15.154:0/3225460071 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4d815ecd0 0x7fc4d80c9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:46.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.063+0000 7fc4de376640 1 --2- 172.21.15.154:0/3225460071 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc4d815ab20 0x7fc4d815da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:46.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.063+0000 7fc4de376640 1 -- 172.21.15.154:0/3225460071 >> 172.21.15.154:0/3225460071 conn(0x7fc4d8089fd0 msgr2=0x7fc4d80cd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:46.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.063+0000 7fc4de376640 1 -- 172.21.15.154:0/3225460071 shutdown_connections 2023-09-23T03:16:46.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.063+0000 7fc4de376640 1 -- 172.21.15.154:0/3225460071 wait complete. 2023-09-23T03:16:46.081 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-23T03:16:46.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.299+0000 7ff2a359e640 1 Processor -- start 2023-09-23T03:16:46.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.299+0000 7ff2a359e640 1 -- start start 2023-09-23T03:16:46.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.301+0000 7ff2a359e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff2a4158480 0x7ff2a40cade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:46.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.301+0000 7ff2a359e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff2a415ba50 0x7ff2a406fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:46.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.301+0000 7ff2a359e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a40cb6c0 con 0x7ff2a415eef0 2023-09-23T03:16:46.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.301+0000 7ff2a359e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a415ad80 con 0x7ff2a4158480 2023-09-23T03:16:46.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.301+0000 7ff2a359e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a415af70 con 0x7ff2a415ba50 2023-09-23T03:16:46.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.301+0000 7ff2a259c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff2a4158480 0x7ff2a40cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:46.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.301+0000 7ff2a259c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff2a4158480 0x7ff2a40cade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:41102/0 (socket says 172.21.15.154:41102) 2023-09-23T03:16:46.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.301+0000 7ff2a259c640 1 -- 172.21.15.154:0/3979147739 learned_addr learned my addr 172.21.15.154:0/3979147739 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:46.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.302+0000 7ff29bfff640 1 --2- 172.21.15.154:0/3979147739 >> v2:172.21.15.190:3300/0 conn(0x7ff2a415ba50 0x7ff2a406fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:46.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.302+0000 7ff2a259c640 1 -- 172.21.15.154:0/3979147739 >> v2:172.21.15.190:3300/0 conn(0x7ff2a415ba50 msgr2=0x7ff2a406fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:46.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.302+0000 7ff2a259c640 1 --2- 172.21.15.154:0/3979147739 >> v2:172.21.15.190:3300/0 conn(0x7ff2a415ba50 0x7ff2a406fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:46.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.302+0000 7ff2a259c640 1 -- 172.21.15.154:0/3979147739 >> v1:172.21.15.190:6789/0 conn(0x7ff2a415eef0 legacy=0x7ff2a406bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:46.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.302+0000 7ff2a259c640 1 -- 172.21.15.154:0/3979147739 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a415b250 con 0x7ff2a4158480 2023-09-23T03:16:46.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.302+0000 7ff29bfff640 1 --2- 172.21.15.154:0/3979147739 >> v2:172.21.15.190:3300/0 conn(0x7ff2a415ba50 0x7ff2a406fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:46.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.304+0000 7ff2a259c640 1 --2- 172.21.15.154:0/3979147739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2a4158480 0x7ff2a40cade0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff2a406f970 tx=0x7ff29402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c400bb2f080595f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:46.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.304+0000 7ff2a1d9b640 1 -- 172.21.15.154:0/3979147739 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff294031870 con 0x7ff2a4158480 2023-09-23T03:16:46.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.304+0000 7ff2a1d9b640 1 -- 172.21.15.154:0/3979147739 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2940319d0 con 0x7ff2a4158480 2023-09-23T03:16:46.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.304+0000 7ff2a1d9b640 1 -- 172.21.15.154:0/3979147739 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff294031d60 con 0x7ff2a4158480 2023-09-23T03:16:46.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.304+0000 7ff2a359e640 1 -- 172.21.15.154:0/3979147739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2a4158480 msgr2=0x7ff2a40cade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:46.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.304+0000 7ff2a359e640 1 --2- 172.21.15.154:0/3979147739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2a4158480 0x7ff2a40cade0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff2a406f970 tx=0x7ff29402b910 comp rx=0 tx=0).stop 2023-09-23T03:16:46.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.305+0000 7ff2a359e640 1 -- 172.21.15.154:0/3979147739 shutdown_connections 2023-09-23T03:16:46.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.305+0000 7ff2a359e640 1 --2- 172.21.15.154:0/3979147739 >> v2:172.21.15.190:3300/0 conn(0x7ff2a415ba50 0x7ff2a406fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:46.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.305+0000 7ff2a359e640 1 --2- 172.21.15.154:0/3979147739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2a4158480 0x7ff2a40cade0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:46.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.305+0000 7ff2a359e640 1 -- 172.21.15.154:0/3979147739 >> 172.21.15.154:0/3979147739 conn(0x7ff2a40aff20 msgr2=0x7ff2a40b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:46.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.305+0000 7ff2a359e640 1 -- 172.21.15.154:0/3979147739 shutdown_connections 2023-09-23T03:16:46.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.305+0000 7ff2a359e640 1 -- 172.21.15.154:0/3979147739 wait complete. 2023-09-23T03:16:46.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.306+0000 7ff2a359e640 1 Processor -- start 2023-09-23T03:16:46.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.307+0000 7ff2a359e640 1 -- start start 2023-09-23T03:16:46.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.308+0000 7ff2a359e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2a4158480 0x7ff2a4143c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:46.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.308+0000 7ff2a359e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2a415ba50 0x7ff2a415cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:46.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.308+0000 7ff2a359e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2a415eef0 0x7ff2a415d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:46.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.308+0000 7ff2a359e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a4156700 con 0x7ff2a415ba50 2023-09-23T03:16:46.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.308+0000 7ff2a359e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a40c28b0 con 0x7ff2a4158480 2023-09-23T03:16:46.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.308+0000 7ff2a359e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a40c2b50 con 0x7ff2a415eef0 2023-09-23T03:16:46.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.308+0000 7ff29bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2a415ba50 0x7ff2a415cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:46.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.308+0000 7ff29bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2a415ba50 0x7ff2a415cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:41110/0 (socket says 172.21.15.154:41110) 2023-09-23T03:16:46.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.308+0000 7ff29bfff640 1 -- 172.21.15.154:0/2837833320 learned_addr learned my addr 172.21.15.154:0/2837833320 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:46.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.308+0000 7ff2a259c640 1 --2- 172.21.15.154:0/2837833320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2a4158480 0x7ff2a4143c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:46.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.308+0000 7ff2a2d9d640 1 --2- 172.21.15.154:0/2837833320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2a415eef0 0x7ff2a415d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:46.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.308+0000 7ff29bfff640 1 -- 172.21.15.154:0/2837833320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2a415eef0 msgr2=0x7ff2a415d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:46.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.308+0000 7ff29bfff640 1 --2- 172.21.15.154:0/2837833320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2a415eef0 0x7ff2a415d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:46.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.308+0000 7ff29bfff640 1 -- 172.21.15.154:0/2837833320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2a4158480 msgr2=0x7ff2a4143c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:46.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.308+0000 7ff29bfff640 1 --2- 172.21.15.154:0/2837833320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2a4158480 0x7ff2a4143c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:46.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.308+0000 7ff29bfff640 1 -- 172.21.15.154:0/2837833320 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2940094d0 con 0x7ff2a415ba50 2023-09-23T03:16:46.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.309+0000 7ff29bfff640 1 --2- 172.21.15.154:0/2837833320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2a415ba50 0x7ff2a415cff0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff28c00c7f0 tx=0x7ff28c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:46.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.309+0000 7ff29affd640 1 -- 172.21.15.154:0/2837833320 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c007a30 con 0x7ff2a415ba50 2023-09-23T03:16:46.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.309+0000 7ff29affd640 1 -- 172.21.15.154:0/2837833320 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28c007b90 con 0x7ff2a415ba50 2023-09-23T03:16:46.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.309+0000 7ff29affd640 1 -- 172.21.15.154:0/2837833320 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c00ce40 con 0x7ff2a415ba50 2023-09-23T03:16:46.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.309+0000 7ff2a359e640 1 -- 172.21.15.154:0/2837833320 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a40c2e30 con 0x7ff2a415ba50 2023-09-23T03:16:46.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.309+0000 7ff2a359e640 1 -- 172.21.15.154:0/2837833320 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a40cd5b0 con 0x7ff2a415ba50 2023-09-23T03:16:46.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.310+0000 7ff2a359e640 1 -- 172.21.15.154:0/2837833320 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a404e920 con 0x7ff2a415ba50 2023-09-23T03:16:46.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.311+0000 7ff29affd640 1 -- 172.21.15.154:0/2837833320 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7ff28c0113f0 con 0x7ff2a415ba50 2023-09-23T03:16:46.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.312+0000 7ff29affd640 1 --2- 172.21.15.154:0/2837833320 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff28406af30 0x7ff28406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:46.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.312+0000 7ff29affd640 1 -- 172.21.15.154:0/2837833320 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7ff28c089c90 con 0x7ff2a415ba50 2023-09-23T03:16:46.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.313+0000 7ff2a259c640 1 --2- 172.21.15.154:0/2837833320 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff28406af30 0x7ff28406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:46.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.313+0000 7ff2a259c640 1 --2- 172.21.15.154:0/2837833320 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff28406af30 0x7ff28406d3f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff294036000 tx=0x7ff29403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:46.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.317+0000 7ff29affd640 1 -- 172.21.15.154:0/2837833320 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7ff28c054930 con 0x7ff2a415ba50 2023-09-23T03:16:46.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.505+0000 7ff2a359e640 1 -- 172.21.15.154:0/2837833320 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff2a406aee0 con 0x7ff2a415ba50 2023-09-23T03:16:46.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.506+0000 7ff29affd640 1 -- 172.21.15.154:0/2837833320 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff28c057f50 con 0x7ff2a415ba50 2023-09-23T03:16:46.506 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:16:46.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.508+0000 7ff2a359e640 1 -- 172.21.15.154:0/2837833320 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff28406af30 msgr2=0x7ff28406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:46.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.508+0000 7ff2a359e640 1 --2- 172.21.15.154:0/2837833320 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff28406af30 0x7ff28406d3f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff294036000 tx=0x7ff29403c040 comp rx=0 tx=0).stop 2023-09-23T03:16:46.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.508+0000 7ff2a359e640 1 -- 172.21.15.154:0/2837833320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2a415ba50 msgr2=0x7ff2a415cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:46.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.508+0000 7ff2a359e640 1 --2- 172.21.15.154:0/2837833320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2a415ba50 0x7ff2a415cff0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff28c00c7f0 tx=0x7ff28c00cbf0 comp rx=0 tx=0).stop 2023-09-23T03:16:46.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.509+0000 7ff2a359e640 1 -- 172.21.15.154:0/2837833320 shutdown_connections 2023-09-23T03:16:46.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.509+0000 7ff2a359e640 1 --2- 172.21.15.154:0/2837833320 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff28406af30 0x7ff28406d3f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:46.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.509+0000 7ff2a359e640 1 --2- 172.21.15.154:0/2837833320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff2a415eef0 0x7ff2a415d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:46.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.509+0000 7ff2a359e640 1 --2- 172.21.15.154:0/2837833320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff2a415ba50 0x7ff2a415cff0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:46.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.509+0000 7ff2a359e640 1 --2- 172.21.15.154:0/2837833320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2a4158480 0x7ff2a4143c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:46.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.509+0000 7ff2a359e640 1 -- 172.21.15.154:0/2837833320 >> 172.21.15.154:0/2837833320 conn(0x7ff2a40aff20 msgr2=0x7ff2a40c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:46.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.509+0000 7ff2a359e640 1 -- 172.21.15.154:0/2837833320 shutdown_connections 2023-09-23T03:16:46.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.509+0000 7ff2a359e640 1 -- 172.21.15.154:0/2837833320 wait complete. 2023-09-23T03:16:46.527 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-23T03:16:46.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.735+0000 7f66313ee640 1 Processor -- start 2023-09-23T03:16:46.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.736+0000 7f66313ee640 1 -- start start 2023-09-23T03:16:46.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.737+0000 7f66313ee640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f662c0acce0 0x7f662c15ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:46.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.737+0000 7f66313ee640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f662c0cf040 0x7f662c0ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:46.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.737+0000 7f66313ee640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f662c15ac30 con 0x7f662c0b1b20 2023-09-23T03:16:46.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.737+0000 7f66313ee640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f662c0b2e90 con 0x7f662c0cf040 2023-09-23T03:16:46.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.737+0000 7f66313ee640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f662c0b1f70 con 0x7f662c0acce0 2023-09-23T03:16:46.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.738+0000 7f662b7fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f662c0cf040 0x7f662c0ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:46.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.738+0000 7f662affd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f662c0acce0 0x7f662c15ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:46.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.738+0000 7f662b7fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f662c0cf040 0x7f662c0ab140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.154:50468/0 (socket says 172.21.15.154:50468) 2023-09-23T03:16:46.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.738+0000 7f662a7fc640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f662c0b1b20 0x7f662c0ce930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.154:38266/0 (socket says 172.21.15.154:38266) 2023-09-23T03:16:46.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.738+0000 7f662affd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f662c0acce0 0x7f662c15ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:45626/0 (socket says 172.21.15.154:45626) 2023-09-23T03:16:46.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.738+0000 7f662b7fe640 1 -- 172.21.15.154:0/3485438766 learned_addr learned my addr 172.21.15.154:0/3485438766 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:46.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.739+0000 7f662affd640 1 -- 172.21.15.154:0/3485438766 >> v2:172.21.15.190:3300/0 conn(0x7f662c0cf040 msgr2=0x7f662c0ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:46.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.739+0000 7f662affd640 1 --2- 172.21.15.154:0/3485438766 >> v2:172.21.15.190:3300/0 conn(0x7f662c0cf040 0x7f662c0ab140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:46.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.739+0000 7f662affd640 1 -- 172.21.15.154:0/3485438766 >> v1:172.21.15.190:6789/0 conn(0x7f662c0b1b20 legacy=0x7f662c0ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:46.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.739+0000 7f662affd640 1 -- 172.21.15.154:0/3485438766 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f662c15f2a0 con 0x7f662c0acce0 2023-09-23T03:16:46.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.739+0000 7f662b7fe640 1 --2- 172.21.15.154:0/3485438766 >> v2:172.21.15.190:3300/0 conn(0x7f662c0cf040 0x7f662c0ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:16:46.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.740+0000 7f662affd640 1 --2- 172.21.15.154:0/3485438766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f662c0acce0 0x7f662c15ed60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f661c009de0 tx=0x7f661c02be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58b5f4f4c0f38f6a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:46.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.741+0000 7f6629ffb640 1 -- 172.21.15.154:0/3485438766 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c02cc00 con 0x7f662c0acce0 2023-09-23T03:16:46.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.741+0000 7f6629ffb640 1 -- 172.21.15.154:0/3485438766 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661c02cd60 con 0x7f662c0acce0 2023-09-23T03:16:46.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.741+0000 7f6629ffb640 1 -- 172.21.15.154:0/3485438766 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c034660 con 0x7f662c0acce0 2023-09-23T03:16:46.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.742+0000 7f66313ee640 1 -- 172.21.15.154:0/3485438766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f662c0acce0 msgr2=0x7f662c15ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:46.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.742+0000 7f66313ee640 1 --2- 172.21.15.154:0/3485438766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f662c0acce0 0x7f662c15ed60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f661c009de0 tx=0x7f661c02be50 comp rx=0 tx=0).stop 2023-09-23T03:16:46.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.742+0000 7f66313ee640 1 -- 172.21.15.154:0/3485438766 shutdown_connections 2023-09-23T03:16:46.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.742+0000 7f66313ee640 1 --2- 172.21.15.154:0/3485438766 >> v2:172.21.15.190:3300/0 conn(0x7f662c0cf040 0x7f662c0ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:46.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.742+0000 7f66313ee640 1 --2- 172.21.15.154:0/3485438766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f662c0acce0 0x7f662c15ed60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:46.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.742+0000 7f66313ee640 1 -- 172.21.15.154:0/3485438766 >> 172.21.15.154:0/3485438766 conn(0x7f662c06a0f0 msgr2=0x7f662c06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:46.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.742+0000 7f66313ee640 1 -- 172.21.15.154:0/3485438766 shutdown_connections 2023-09-23T03:16:46.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.742+0000 7f66313ee640 1 -- 172.21.15.154:0/3485438766 wait complete. 2023-09-23T03:16:46.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.744+0000 7f66313ee640 1 Processor -- start 2023-09-23T03:16:46.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.744+0000 7f66313ee640 1 -- start start 2023-09-23T03:16:46.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.746+0000 7f66313ee640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f662c0acce0 0x7f662c0978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:46.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.746+0000 7f66313ee640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f662c0b1b20 0x7f662c0c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:46.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.746+0000 7f66313ee640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f662c0cf040 0x7f662c0c1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:46.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.746+0000 7f66313ee640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662c15add0 con 0x7f662c0b1b20 2023-09-23T03:16:46.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.746+0000 7f66313ee640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662c15bc30 con 0x7f662c0acce0 2023-09-23T03:16:46.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.746+0000 7f66313ee640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f662c15dcd0 con 0x7f662c0cf040 2023-09-23T03:16:46.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.746+0000 7f662a7fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f662c0b1b20 0x7f662c0c7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:46.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.746+0000 7f662a7fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f662c0b1b20 0x7f662c0c7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:41114/0 (socket says 172.21.15.154:41114) 2023-09-23T03:16:46.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.746+0000 7f662a7fc640 1 -- 172.21.15.154:0/945771534 learned_addr learned my addr 172.21.15.154:0/945771534 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:46.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.746+0000 7f662affd640 1 --2- 172.21.15.154:0/945771534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f662c0acce0 0x7f662c0978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:46.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.746+0000 7f662b7fe640 1 --2- 172.21.15.154:0/945771534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f662c0cf040 0x7f662c0c1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:46.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.746+0000 7f662a7fc640 1 -- 172.21.15.154:0/945771534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f662c0cf040 msgr2=0x7f662c0c1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:46.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.746+0000 7f662a7fc640 1 --2- 172.21.15.154:0/945771534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f662c0cf040 0x7f662c0c1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:46.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.746+0000 7f662a7fc640 1 -- 172.21.15.154:0/945771534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f662c0acce0 msgr2=0x7f662c0978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:46.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.746+0000 7f662a7fc640 1 --2- 172.21.15.154:0/945771534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f662c0acce0 0x7f662c0978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:46.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.746+0000 7f662a7fc640 1 -- 172.21.15.154:0/945771534 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66240094d0 con 0x7f662c0b1b20 2023-09-23T03:16:46.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.747+0000 7f662affd640 1 --2- 172.21.15.154:0/945771534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f662c0acce0 0x7f662c0978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:16:46.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.747+0000 7f662a7fc640 1 --2- 172.21.15.154:0/945771534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f662c0b1b20 0x7f662c0c7dd0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f661400c480 tx=0x7f661400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:46.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.747+0000 7f66237fe640 1 -- 172.21.15.154:0/945771534 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6614007480 con 0x7f662c0b1b20 2023-09-23T03:16:46.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.747+0000 7f66237fe640 1 -- 172.21.15.154:0/945771534 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66140075e0 con 0x7f662c0b1b20 2023-09-23T03:16:46.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.747+0000 7f66313ee640 1 -- 172.21.15.154:0/945771534 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f661c009a70 con 0x7f662c0b1b20 2023-09-23T03:16:46.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.747+0000 7f66237fe640 1 -- 172.21.15.154:0/945771534 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6614004a00 con 0x7f662c0b1b20 2023-09-23T03:16:46.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.747+0000 7f66313ee640 1 -- 172.21.15.154:0/945771534 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f662c15b300 con 0x7f662c0b1b20 2023-09-23T03:16:46.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.748+0000 7f66313ee640 1 -- 172.21.15.154:0/945771534 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f662c073eb0 con 0x7f662c0b1b20 2023-09-23T03:16:46.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.749+0000 7f66237fe640 1 -- 172.21.15.154:0/945771534 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f6614004b60 con 0x7f662c0b1b20 2023-09-23T03:16:46.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.750+0000 7f66237fe640 1 --2- 172.21.15.154:0/945771534 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f660c06af30 0x7f660c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:46.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.750+0000 7f66237fe640 1 -- 172.21.15.154:0/945771534 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f6614088c20 con 0x7f662c0b1b20 2023-09-23T03:16:46.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.751+0000 7f662affd640 1 --2- 172.21.15.154:0/945771534 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f660c06af30 0x7f660c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:46.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.751+0000 7f662affd640 1 --2- 172.21.15.154:0/945771534 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f660c06af30 0x7f660c06d3f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f661c036000 tx=0x7f661c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:46.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.755+0000 7f66237fe640 1 -- 172.21.15.154:0/945771534 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f6614053820 con 0x7f662c0b1b20 2023-09-23T03:16:46.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.962+0000 7f66313ee640 1 -- 172.21.15.154:0/945771534 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f662c15ac10 con 0x7f662c0b1b20 2023-09-23T03:16:46.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.963+0000 7f66237fe640 1 -- 172.21.15.154:0/945771534 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6614056e40 con 0x7f662c0b1b20 2023-09-23T03:16:46.963 INFO:teuthology.orchestra.run.smithi154.stderr:listed 0 entries 2023-09-23T03:16:46.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.965+0000 7f66313ee640 1 -- 172.21.15.154:0/945771534 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f660c06af30 msgr2=0x7f660c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:46.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.965+0000 7f66313ee640 1 --2- 172.21.15.154:0/945771534 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f660c06af30 0x7f660c06d3f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f661c036000 tx=0x7f661c030000 comp rx=0 tx=0).stop 2023-09-23T03:16:46.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.965+0000 7f66313ee640 1 -- 172.21.15.154:0/945771534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f662c0b1b20 msgr2=0x7f662c0c7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:46.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.966+0000 7f66313ee640 1 --2- 172.21.15.154:0/945771534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f662c0b1b20 0x7f662c0c7dd0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f661400c480 tx=0x7f661400c940 comp rx=0 tx=0).stop 2023-09-23T03:16:46.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.966+0000 7f66313ee640 1 -- 172.21.15.154:0/945771534 shutdown_connections 2023-09-23T03:16:46.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.966+0000 7f66313ee640 1 --2- 172.21.15.154:0/945771534 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f660c06af30 0x7f660c06d3f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:46.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.966+0000 7f66313ee640 1 --2- 172.21.15.154:0/945771534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f662c0cf040 0x7f662c0c1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:46.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.966+0000 7f66313ee640 1 --2- 172.21.15.154:0/945771534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f662c0b1b20 0x7f662c0c7dd0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:46.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.966+0000 7f66313ee640 1 --2- 172.21.15.154:0/945771534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f662c0acce0 0x7f662c0978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:46.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.966+0000 7f66313ee640 1 -- 172.21.15.154:0/945771534 >> 172.21.15.154:0/945771534 conn(0x7f662c06a0f0 msgr2=0x7f662c0cb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:46.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.966+0000 7f66313ee640 1 -- 172.21.15.154:0/945771534 shutdown_connections 2023-09-23T03:16:46.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:46.966+0000 7f66313ee640 1 -- 172.21.15.154:0/945771534 wait complete. 2023-09-23T03:16:46.985 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi154.front.sepia.ceph.com... 2023-09-23T03:16:46.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:46.985 DEBUG:teuthology.orchestra.run.smithi154:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-09-23T03:16:47.070 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:16:47.070 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2023-09-23T03:16:47.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:47.070 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-09-23T03:16:47.126 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:16:47.126 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:16:47.185 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:16:47.185 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:16:47.239 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:16:47.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.457+0000 7ff804ce1640 1 Processor -- start 2023-09-23T03:16:47.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.457+0000 7ff804ce1640 1 -- start start 2023-09-23T03:16:47.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.459+0000 7ff804ce1640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff8000cf040 0x7ff8001656e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:47.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.459+0000 7ff804ce1640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff80015ac30 con 0x7ff8000acce0 2023-09-23T03:16:47.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.459+0000 7ff804ce1640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8000b1f70 con 0x7ff8000b1b20 2023-09-23T03:16:47.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.459+0000 7ff804ce1640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8000b2e90 con 0x7ff8000cf040 2023-09-23T03:16:47.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.459+0000 7ff7fe59c640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff8000acce0 0x7ff80015ed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:36316/0 (socket says 172.21.15.154:36316) 2023-09-23T03:16:47.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.459+0000 7ff7fe59c640 1 -- 172.21.15.154:0/1168789233 learned_addr learned my addr 172.21.15.154:0/1168789233 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:47.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.459+0000 7ff7fed9d640 1 --2- 172.21.15.154:0/1168789233 >> v2:172.21.15.154:3300/0 conn(0x7ff8000cf040 0x7ff8001656e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:47.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.459+0000 7ff7fdd9b640 1 -- 172.21.15.154:0/1168789233 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2505488451 0 0) 0x7ff7f0001290 con 0x7ff8000acce0 2023-09-23T03:16:47.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.459+0000 7ff7fdd9b640 1 -- 172.21.15.154:0/1168789233 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7ec003520 con 0x7ff8000acce0 2023-09-23T03:16:47.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.459+0000 7ff7fed9d640 1 -- 172.21.15.154:0/1168789233 >> v1:172.21.15.190:6789/0 conn(0x7ff8000b1b20 legacy=0x7ff8000ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:47.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.459+0000 7ff7fed9d640 1 -- 172.21.15.154:0/1168789233 >> v1:172.21.15.154:6789/0 conn(0x7ff8000acce0 legacy=0x7ff80015ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:47.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.459+0000 7ff7fed9d640 1 -- 172.21.15.154:0/1168789233 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8000b2210 con 0x7ff8000cf040 2023-09-23T03:16:47.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.461+0000 7ff7fed9d640 1 --2- 172.21.15.154:0/1168789233 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8000cf040 0x7ff8001656e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8009d90 tx=0x7ff7f802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9827fc61f1b781f2 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:47.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.461+0000 7ff7fdd9b640 1 -- 172.21.15.154:0/1168789233 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f802cbd0 con 0x7ff8000cf040 2023-09-23T03:16:47.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.461+0000 7ff7fdd9b640 1 -- 172.21.15.154:0/1168789233 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f802cd30 con 0x7ff8000cf040 2023-09-23T03:16:47.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.461+0000 7ff7fdd9b640 1 -- 172.21.15.154:0/1168789233 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f802cea0 con 0x7ff8000cf040 2023-09-23T03:16:47.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.461+0000 7ff804ce1640 1 -- 172.21.15.154:0/1168789233 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8000cf040 msgr2=0x7ff8001656e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:47.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.461+0000 7ff804ce1640 1 --2- 172.21.15.154:0/1168789233 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8000cf040 0x7ff8001656e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8009d90 tx=0x7ff7f802be20 comp rx=0 tx=0).stop 2023-09-23T03:16:47.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.462+0000 7ff804ce1640 1 -- 172.21.15.154:0/1168789233 shutdown_connections 2023-09-23T03:16:47.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.462+0000 7ff804ce1640 1 --2- 172.21.15.154:0/1168789233 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8000cf040 0x7ff8001656e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:47.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.462+0000 7ff804ce1640 1 -- 172.21.15.154:0/1168789233 >> 172.21.15.154:0/1168789233 conn(0x7ff80006a0f0 msgr2=0x7ff80006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:47.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.462+0000 7ff804ce1640 1 -- 172.21.15.154:0/1168789233 shutdown_connections 2023-09-23T03:16:47.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.462+0000 7ff804ce1640 1 -- 172.21.15.154:0/1168789233 wait complete. 2023-09-23T03:16:47.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.464+0000 7ff804ce1640 1 Processor -- start 2023-09-23T03:16:47.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.464+0000 7ff804ce1640 1 -- start start 2023-09-23T03:16:47.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.465+0000 7ff804ce1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8000acce0 0x7ff80015add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:47.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.465+0000 7ff804ce1640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8000b1b20 0x7ff80015d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:47.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.465+0000 7ff804ce1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8000cf040 0x7ff800158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:47.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.465+0000 7ff804ce1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8001f0d50 con 0x7ff8000acce0 2023-09-23T03:16:47.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.465+0000 7ff804ce1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8001f03b0 con 0x7ff8000cf040 2023-09-23T03:16:47.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.465+0000 7ff804ce1640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8001f0690 con 0x7ff8000b1b20 2023-09-23T03:16:47.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.465+0000 7ff7fe59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8000acce0 0x7ff80015add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:47.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.465+0000 7ff7fe59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8000acce0 0x7ff80015add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:41128/0 (socket says 172.21.15.154:41128) 2023-09-23T03:16:47.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.465+0000 7ff7fe59c640 1 -- 172.21.15.154:0/1433213885 learned_addr learned my addr 172.21.15.154:0/1433213885 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:47.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.465+0000 7ff7fed9d640 1 --2- 172.21.15.154:0/1433213885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8000cf040 0x7ff800158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:47.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.466+0000 7ff7f7fff640 1 --2- 172.21.15.154:0/1433213885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8000b1b20 0x7ff80015d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:47.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.466+0000 7ff7fe59c640 1 -- 172.21.15.154:0/1433213885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8000b1b20 msgr2=0x7ff80015d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:47.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.466+0000 7ff7fe59c640 1 --2- 172.21.15.154:0/1433213885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8000b1b20 0x7ff80015d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:47.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.466+0000 7ff7fe59c640 1 -- 172.21.15.154:0/1433213885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8000cf040 msgr2=0x7ff800158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:47.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.466+0000 7ff7fe59c640 1 --2- 172.21.15.154:0/1433213885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8000cf040 0x7ff800158d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:47.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.466+0000 7ff7fe59c640 1 -- 172.21.15.154:0/1433213885 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f8009a70 con 0x7ff8000acce0 2023-09-23T03:16:47.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.466+0000 7ff7fe59c640 1 --2- 172.21.15.154:0/1433213885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8000acce0 0x7ff80015add0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff7f000c4d0 tx=0x7ff7f000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:47.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.466+0000 7ff7f6ffd640 1 -- 172.21.15.154:0/1433213885 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f0007480 con 0x7ff8000acce0 2023-09-23T03:16:47.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.466+0000 7ff7f6ffd640 1 -- 172.21.15.154:0/1433213885 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f00075e0 con 0x7ff8000acce0 2023-09-23T03:16:47.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.466+0000 7ff804ce1640 1 -- 172.21.15.154:0/1433213885 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8001f0970 con 0x7ff8000acce0 2023-09-23T03:16:47.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.466+0000 7ff7f6ffd640 1 -- 172.21.15.154:0/1433213885 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f00059e0 con 0x7ff8000acce0 2023-09-23T03:16:47.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.467+0000 7ff804ce1640 1 -- 172.21.15.154:0/1433213885 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8001f0080 con 0x7ff8000acce0 2023-09-23T03:16:47.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.468+0000 7ff804ce1640 1 -- 172.21.15.154:0/1433213885 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8000b2a90 con 0x7ff8000acce0 2023-09-23T03:16:47.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.469+0000 7ff7f6ffd640 1 -- 172.21.15.154:0/1433213885 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7ff7f0005b40 con 0x7ff8000acce0 2023-09-23T03:16:47.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.470+0000 7ff7f6ffd640 1 --2- 172.21.15.154:0/1433213885 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff7e006af30 0x7ff7e006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:47.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.470+0000 7ff7f6ffd640 1 -- 172.21.15.154:0/1433213885 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7ff7f0088fe0 con 0x7ff8000acce0 2023-09-23T03:16:47.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.470+0000 7ff7f7fff640 1 --2- 172.21.15.154:0/1433213885 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff7e006af30 0x7ff7e006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:47.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.471+0000 7ff7f7fff640 1 --2- 172.21.15.154:0/1433213885 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff7e006af30 0x7ff7e006d3f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff800070e50 tx=0x7ff7e8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:47.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.474+0000 7ff7f6ffd640 1 -- 172.21.15.154:0/1433213885 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7ff8000b2a90 con 0x7ff8000acce0 2023-09-23T03:16:47.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.664+0000 7ff804ce1640 1 -- 172.21.15.154:0/1433213885 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8000b2a90 con 0x7ff8000acce0 2023-09-23T03:16:47.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.667+0000 7ff7f6ffd640 1 -- 172.21.15.154:0/1433213885 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18744 (secure 0 0 0) 0x7ff8000b2a90 con 0x7ff8000acce0 2023-09-23T03:16:47.668 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:16:47.668 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":26,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:16:03.151371+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:15:18.575849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T03:15:19.707212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T03:15:23.337138+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:16:47.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.670+0000 7ff804ce1640 1 -- 172.21.15.154:0/1433213885 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff7e006af30 msgr2=0x7ff7e006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:47.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.670+0000 7ff804ce1640 1 --2- 172.21.15.154:0/1433213885 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff7e006af30 0x7ff7e006d3f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff800070e50 tx=0x7ff7e8009040 comp rx=0 tx=0).stop 2023-09-23T03:16:47.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.670+0000 7ff804ce1640 1 -- 172.21.15.154:0/1433213885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8000acce0 msgr2=0x7ff80015add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:47.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.670+0000 7ff804ce1640 1 --2- 172.21.15.154:0/1433213885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8000acce0 0x7ff80015add0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff7f000c4d0 tx=0x7ff7f000c990 comp rx=0 tx=0).stop 2023-09-23T03:16:47.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.670+0000 7ff804ce1640 1 -- 172.21.15.154:0/1433213885 shutdown_connections 2023-09-23T03:16:47.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.670+0000 7ff804ce1640 1 --2- 172.21.15.154:0/1433213885 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff7e006af30 0x7ff7e006d3f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:47.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.670+0000 7ff804ce1640 1 --2- 172.21.15.154:0/1433213885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8000cf040 0x7ff800158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:47.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.670+0000 7ff804ce1640 1 --2- 172.21.15.154:0/1433213885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8000b1b20 0x7ff80015d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:47.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.670+0000 7ff804ce1640 1 --2- 172.21.15.154:0/1433213885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8000acce0 0x7ff80015add0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:47.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.670+0000 7ff804ce1640 1 -- 172.21.15.154:0/1433213885 >> 172.21.15.154:0/1433213885 conn(0x7ff80006a0f0 msgr2=0x7ff80006a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:47.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.670+0000 7ff804ce1640 1 -- 172.21.15.154:0/1433213885 shutdown_connections 2023-09-23T03:16:47.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.671+0000 7ff804ce1640 1 -- 172.21.15.154:0/1433213885 wait complete. 2023-09-23T03:16:47.689 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:15:18.575849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:15:19.707212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-23T03:15:23.337138+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-23T03:16:47.689 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:16:47.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.909+0000 7f8a2f59e640 1 Processor -- start 2023-09-23T03:16:47.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.909+0000 7f8a2f59e640 1 -- start start 2023-09-23T03:16:47.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.911+0000 7f8a2f59e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8a3015e130 0x7f8a300c9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:47.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.911+0000 7f8a2f59e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a300c9a80 0x7f8a3006fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:47.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.911+0000 7f8a2f59e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a30158540 con 0x7f8a30158160 2023-09-23T03:16:47.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.911+0000 7f8a2f59e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a3015b3e0 con 0x7f8a3015e130 2023-09-23T03:16:47.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.911+0000 7f8a2f59e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3015b6c0 con 0x7f8a300c9a80 2023-09-23T03:16:47.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.911+0000 7f8a2e59c640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f8a30158160 0x7f8a3015ab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:36318/0 (socket says 172.21.15.154:36318) 2023-09-23T03:16:47.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.911+0000 7f8a2e59c640 1 -- 172.21.15.154:0/855293661 learned_addr learned my addr 172.21.15.154:0/855293661 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:47.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.912+0000 7f8a27fff640 1 --2- 172.21.15.154:0/855293661 >> v2:172.21.15.190:3300/0 conn(0x7f8a3015e130 0x7f8a300c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:47.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.912+0000 7f8a2ed9d640 1 --2- 172.21.15.154:0/855293661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a300c9a80 0x7f8a3006fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:47.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.912+0000 7f8a2dd9b640 1 -- 172.21.15.154:0/855293661 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 124276578 0 0) 0x7f8a30158540 con 0x7f8a30158160 2023-09-23T03:16:47.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.912+0000 7f8a2dd9b640 1 -- 172.21.15.154:0/855293661 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a1c003520 con 0x7f8a30158160 2023-09-23T03:16:47.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.912+0000 7f8a2dd9b640 1 -- 172.21.15.154:0/855293661 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1560315992 0 0) 0x7f8a1c003520 con 0x7f8a30158160 2023-09-23T03:16:47.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.912+0000 7f8a2dd9b640 1 -- 172.21.15.154:0/855293661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a300c9a80 msgr2=0x7f8a3006fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:47.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.912+0000 7f8a2dd9b640 1 --2- 172.21.15.154:0/855293661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a300c9a80 0x7f8a3006fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:47.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.912+0000 7f8a2dd9b640 1 -- 172.21.15.154:0/855293661 >> v2:172.21.15.190:3300/0 conn(0x7f8a3015e130 msgr2=0x7f8a300c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:47.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.912+0000 7f8a2dd9b640 1 --2- 172.21.15.154:0/855293661 >> v2:172.21.15.190:3300/0 conn(0x7f8a3015e130 0x7f8a300c9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:47.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.912+0000 7f8a2dd9b640 1 -- 172.21.15.154:0/855293661 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a3015b9a0 con 0x7f8a30158160 2023-09-23T03:16:47.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.912+0000 7f8a2dd9b640 1 -- 172.21.15.154:0/855293661 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f8a20001670 con 0x7f8a30158160 2023-09-23T03:16:47.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.912+0000 7f8a2dd9b640 1 -- 172.21.15.154:0/855293661 >> v1:172.21.15.154:6789/0 conn(0x7f8a30158160 legacy=0x7f8a3015ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:47.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.912+0000 7f8a2dd9b640 1 --2- 172.21.15.154:0/855293661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a1c003970 0x7f8a1c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:47.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.912+0000 7f8a2dd9b640 1 --2- 172.21.15.154:0/855293661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a1c0047c0 0x7f8a1c004c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:47.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.912+0000 7f8a2dd9b640 1 --2- 172.21.15.154:0/855293661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a1c005140 0x7f8a1c00fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:47.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.912+0000 7f8a2dd9b640 1 -- 172.21.15.154:0/855293661 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3015b3e0 con 0x7f8a1c005140 2023-09-23T03:16:47.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.912+0000 7f8a2dd9b640 1 -- 172.21.15.154:0/855293661 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3015b6c0 con 0x7f8a1c003970 2023-09-23T03:16:47.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.913+0000 7f8a2dd9b640 1 -- 172.21.15.154:0/855293661 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1c0045f0 con 0x7f8a1c0047c0 2023-09-23T03:16:47.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.913+0000 7f8a2ed9d640 1 --2- 172.21.15.154:0/855293661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a1c005140 0x7f8a1c00fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:47.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.913+0000 7f8a2ed9d640 1 -- 172.21.15.154:0/855293661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a1c0047c0 msgr2=0x7f8a1c004c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:47.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.913+0000 7f8a2ed9d640 1 --2- 172.21.15.154:0/855293661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a1c0047c0 0x7f8a1c004c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:47.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.913+0000 7f8a2ed9d640 1 -- 172.21.15.154:0/855293661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a1c003970 msgr2=0x7f8a1c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:47.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.913+0000 7f8a2ed9d640 1 --2- 172.21.15.154:0/855293661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a1c003970 0x7f8a1c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:47.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.913+0000 7f8a2ed9d640 1 -- 172.21.15.154:0/855293661 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a1c003520 con 0x7f8a1c005140 2023-09-23T03:16:47.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.914+0000 7f8a2ed9d640 1 --2- 172.21.15.154:0/855293661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a1c005140 0x7f8a1c00fd90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8a2800e170 tx=0x7f8a1c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9002041b28b16010 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:47.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.915+0000 7f8a2dd9b640 1 -- 172.21.15.154:0/855293661 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2802cc30 con 0x7f8a1c005140 2023-09-23T03:16:47.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.915+0000 7f8a2dd9b640 1 -- 172.21.15.154:0/855293661 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a2800f040 con 0x7f8a1c005140 2023-09-23T03:16:47.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.915+0000 7f8a2dd9b640 1 -- 172.21.15.154:0/855293661 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a28034440 con 0x7f8a1c005140 2023-09-23T03:16:47.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.915+0000 7f8a2f59e640 1 -- 172.21.15.154:0/855293661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a1c005140 msgr2=0x7f8a1c00fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:47.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.915+0000 7f8a2f59e640 1 --2- 172.21.15.154:0/855293661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a1c005140 0x7f8a1c00fd90 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8a2800e170 tx=0x7f8a1c0057f0 comp rx=0 tx=0).stop 2023-09-23T03:16:47.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.915+0000 7f8a2ed9d640 1 -- 172.21.15.154:0/855293661 reap_dead start 2023-09-23T03:16:47.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.915+0000 7f8a2f59e640 1 -- 172.21.15.154:0/855293661 shutdown_connections 2023-09-23T03:16:47.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.915+0000 7f8a2f59e640 1 -- 172.21.15.154:0/855293661 >> 172.21.15.154:0/855293661 conn(0x7f8a3006a300 msgr2=0x7f8a300afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:47.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.915+0000 7f8a2f59e640 1 -- 172.21.15.154:0/855293661 shutdown_connections 2023-09-23T03:16:47.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.915+0000 7f8a2f59e640 1 -- 172.21.15.154:0/855293661 wait complete. 2023-09-23T03:16:47.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.917+0000 7f8a2f59e640 1 Processor -- start 2023-09-23T03:16:47.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.917+0000 7f8a2f59e640 1 -- start start 2023-09-23T03:16:47.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.918+0000 7f8a2f59e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a1c005140 0x7f8a30153820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:47.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.918+0000 7f8a2f59e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a300c2530 0x7f8a300c29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:47.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.918+0000 7f8a2f59e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a3015cec0 0x7f8a3015d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:47.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.918+0000 7f8a2f59e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a300c4c00 con 0x7f8a300c2530 2023-09-23T03:16:47.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.918+0000 7f8a2f59e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3015d880 con 0x7f8a1c005140 2023-09-23T03:16:47.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.918+0000 7f8a2f59e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a30153d60 con 0x7f8a3015cec0 2023-09-23T03:16:47.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.918+0000 7f8a27fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a300c2530 0x7f8a300c29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:47.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.919+0000 7f8a27fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a300c2530 0x7f8a300c29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:41146/0 (socket says 172.21.15.154:41146) 2023-09-23T03:16:47.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.919+0000 7f8a27fff640 1 -- 172.21.15.154:0/1578875727 learned_addr learned my addr 172.21.15.154:0/1578875727 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:47.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.919+0000 7f8a2ed9d640 1 --2- 172.21.15.154:0/1578875727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a3015cec0 0x7f8a3015d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:47.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.919+0000 7f8a2e59c640 1 --2- 172.21.15.154:0/1578875727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a1c005140 0x7f8a30153820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:47.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.919+0000 7f8a27fff640 1 -- 172.21.15.154:0/1578875727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a3015cec0 msgr2=0x7f8a3015d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:47.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.919+0000 7f8a27fff640 1 --2- 172.21.15.154:0/1578875727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a3015cec0 0x7f8a3015d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:47.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.919+0000 7f8a27fff640 1 -- 172.21.15.154:0/1578875727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a1c005140 msgr2=0x7f8a30153820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:47.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.919+0000 7f8a27fff640 1 --2- 172.21.15.154:0/1578875727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a1c005140 0x7f8a30153820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:47.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.919+0000 7f8a27fff640 1 -- 172.21.15.154:0/1578875727 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a301f0ff0 con 0x7f8a300c2530 2023-09-23T03:16:47.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.919+0000 7f8a27fff640 1 --2- 172.21.15.154:0/1578875727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a300c2530 0x7f8a300c29b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8a1800e830 tx=0x7f8a1800ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:47.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.919+0000 7f8a26ffd640 1 -- 172.21.15.154:0/1578875727 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1800eee0 con 0x7f8a300c2530 2023-09-23T03:16:47.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.920+0000 7f8a26ffd640 1 -- 172.21.15.154:0/1578875727 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a1800c860 con 0x7f8a300c2530 2023-09-23T03:16:47.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.920+0000 7f8a2f59e640 1 -- 172.21.15.154:0/1578875727 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a300c3060 con 0x7f8a300c2530 2023-09-23T03:16:47.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.920+0000 7f8a26ffd640 1 -- 172.21.15.154:0/1578875727 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1800cbf0 con 0x7f8a300c2530 2023-09-23T03:16:47.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.920+0000 7f8a2f59e640 1 -- 172.21.15.154:0/1578875727 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a300c9b40 con 0x7f8a300c2530 2023-09-23T03:16:47.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.921+0000 7f8a2f59e640 1 -- 172.21.15.154:0/1578875727 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a300736b0 con 0x7f8a300c2530 2023-09-23T03:16:47.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.922+0000 7f8a26ffd640 1 -- 172.21.15.154:0/1578875727 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f8a1800cd50 con 0x7f8a300c2530 2023-09-23T03:16:47.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.923+0000 7f8a26ffd640 1 --2- 172.21.15.154:0/1578875727 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8a1006af60 0x7f8a1006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:47.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.923+0000 7f8a26ffd640 1 -- 172.21.15.154:0/1578875727 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f8a18089c10 con 0x7f8a300c2530 2023-09-23T03:16:47.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.923+0000 7f8a2e59c640 1 --2- 172.21.15.154:0/1578875727 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8a1006af60 0x7f8a1006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:47.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.924+0000 7f8a2e59c640 1 --2- 172.21.15.154:0/1578875727 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8a1006af60 0x7f8a1006d420 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8a20001eb0 tx=0x7f8a20009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:47.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:47.927+0000 7f8a26ffd640 1 -- 172.21.15.154:0/1578875727 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f8a18054800 con 0x7f8a300c2530 2023-09-23T03:16:48.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.117+0000 7f8a2f59e640 1 -- 172.21.15.154:0/1578875727 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8a30067390 con 0x7f8a300c2530 2023-09-23T03:16:48.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.118+0000 7f8a26ffd640 1 -- 172.21.15.154:0/1578875727 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8a180210a0 con 0x7f8a300c2530 2023-09-23T03:16:48.118 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:16:48.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.121+0000 7f8a2f59e640 1 -- 172.21.15.154:0/1578875727 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8a1006af60 msgr2=0x7f8a1006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:48.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.121+0000 7f8a2f59e640 1 --2- 172.21.15.154:0/1578875727 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8a1006af60 0x7f8a1006d420 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8a20001eb0 tx=0x7f8a20009040 comp rx=0 tx=0).stop 2023-09-23T03:16:48.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.121+0000 7f8a2f59e640 1 -- 172.21.15.154:0/1578875727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a300c2530 msgr2=0x7f8a300c29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:48.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.121+0000 7f8a2f59e640 1 --2- 172.21.15.154:0/1578875727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a300c2530 0x7f8a300c29b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8a1800e830 tx=0x7f8a1800ecf0 comp rx=0 tx=0).stop 2023-09-23T03:16:48.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.121+0000 7f8a2f59e640 1 -- 172.21.15.154:0/1578875727 shutdown_connections 2023-09-23T03:16:48.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.121+0000 7f8a2f59e640 1 --2- 172.21.15.154:0/1578875727 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8a1006af60 0x7f8a1006d420 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:48.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.121+0000 7f8a2f59e640 1 --2- 172.21.15.154:0/1578875727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8a3015cec0 0x7f8a3015d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:48.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.121+0000 7f8a2f59e640 1 --2- 172.21.15.154:0/1578875727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a300c2530 0x7f8a300c29b0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:48.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.121+0000 7f8a2f59e640 1 --2- 172.21.15.154:0/1578875727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a1c005140 0x7f8a30153820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:48.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.121+0000 7f8a2f59e640 1 -- 172.21.15.154:0/1578875727 >> 172.21.15.154:0/1578875727 conn(0x7f8a3006a300 msgr2=0x7f8a30156970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:48.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.121+0000 7f8a2f59e640 1 -- 172.21.15.154:0/1578875727 shutdown_connections 2023-09-23T03:16:48.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.121+0000 7f8a2f59e640 1 -- 172.21.15.154:0/1578875727 wait complete. 2023-09-23T03:16:48.138 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:16:48.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.356+0000 7fd3c6f4c640 1 Processor -- start 2023-09-23T03:16:48.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.356+0000 7fd3c6f4c640 1 -- start start 2023-09-23T03:16:48.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.357+0000 7fd3c6f4c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd3c015bb30 0x7fd3c00caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:48.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.357+0000 7fd3c6f4c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd3c00cb3e0 0x7fd3c00cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:48.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.357+0000 7fd3c6f4c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3c015b6a0 con 0x7fd3c015aba0 2023-09-23T03:16:48.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.357+0000 7fd3c6f4c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3c00cb870 con 0x7fd3c015bb30 2023-09-23T03:16:48.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.357+0000 7fd3c6f4c640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3c015bf70 con 0x7fd3c00cb3e0 2023-09-23T03:16:48.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.358+0000 7fd3bffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd3c015bb30 0x7fd3c00caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:48.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.358+0000 7fd3c4ce8640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fd3c015aba0 0x7fd3c015e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:36324/0 (socket says 172.21.15.154:36324) 2023-09-23T03:16:48.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.358+0000 7fd3c4ce8640 1 -- 172.21.15.154:0/912427230 learned_addr learned my addr 172.21.15.154:0/912427230 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:48.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.358+0000 7fd3c54e9640 1 --2- 172.21.15.154:0/912427230 >> v2:172.21.15.190:3300/0 conn(0x7fd3c00cb3e0 0x7fd3c00cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:48.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.358+0000 7fd3bffff640 1 -- 172.21.15.154:0/912427230 >> v2:172.21.15.190:3300/0 conn(0x7fd3c00cb3e0 msgr2=0x7fd3c00cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:48.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.358+0000 7fd3bffff640 1 --2- 172.21.15.154:0/912427230 >> v2:172.21.15.190:3300/0 conn(0x7fd3c00cb3e0 0x7fd3c00cc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:48.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.358+0000 7fd3bffff640 1 -- 172.21.15.154:0/912427230 >> v1:172.21.15.154:6789/0 conn(0x7fd3c015aba0 legacy=0x7fd3c015e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:48.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.358+0000 7fd3bffff640 1 -- 172.21.15.154:0/912427230 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c015f0b0 con 0x7fd3c015bb30 2023-09-23T03:16:48.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.358+0000 7fd3bf7fe640 1 -- 172.21.15.154:0/912427230 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4183807770 0 0) 0x7fd3c015b6a0 con 0x7fd3c015aba0 2023-09-23T03:16:48.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.360+0000 7fd3bffff640 1 --2- 172.21.15.154:0/912427230 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3c015bb30 0x7fd3c00caea0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd3c00c1f80 tx=0x7fd3a802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f2277c3837ece1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:48.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.360+0000 7fd3bf7fe640 1 -- 172.21.15.154:0/912427230 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8031830 con 0x7fd3c015bb30 2023-09-23T03:16:48.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.360+0000 7fd3bf7fe640 1 -- 172.21.15.154:0/912427230 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3a8031990 con 0x7fd3c015bb30 2023-09-23T03:16:48.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.360+0000 7fd3bf7fe640 1 -- 172.21.15.154:0/912427230 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8031d20 con 0x7fd3c015bb30 2023-09-23T03:16:48.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.361+0000 7fd3c6f4c640 1 -- 172.21.15.154:0/912427230 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3c015bb30 msgr2=0x7fd3c00caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:48.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.361+0000 7fd3c6f4c640 1 --2- 172.21.15.154:0/912427230 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3c015bb30 0x7fd3c00caea0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd3c00c1f80 tx=0x7fd3a802b8d0 comp rx=0 tx=0).stop 2023-09-23T03:16:48.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.361+0000 7fd3c6f4c640 1 -- 172.21.15.154:0/912427230 shutdown_connections 2023-09-23T03:16:48.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.361+0000 7fd3c6f4c640 1 --2- 172.21.15.154:0/912427230 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3c015bb30 0x7fd3c00caea0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:48.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.361+0000 7fd3c6f4c640 1 --2- 172.21.15.154:0/912427230 >> v2:172.21.15.190:3300/0 conn(0x7fd3c00cb3e0 0x7fd3c00cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:48.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.361+0000 7fd3c6f4c640 1 -- 172.21.15.154:0/912427230 >> 172.21.15.154:0/912427230 conn(0x7fd3c006c010 msgr2=0x7fd3c006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:48.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.361+0000 7fd3c6f4c640 1 -- 172.21.15.154:0/912427230 shutdown_connections 2023-09-23T03:16:48.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.361+0000 7fd3c6f4c640 1 -- 172.21.15.154:0/912427230 wait complete. 2023-09-23T03:16:48.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.363+0000 7fd3c6f4c640 1 Processor -- start 2023-09-23T03:16:48.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.363+0000 7fd3c6f4c640 1 -- start start 2023-09-23T03:16:48.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.365+0000 7fd3c6f4c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3c00cb3e0 0x7fd3c01f2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:48.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.365+0000 7fd3c6f4c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3c015aba0 0x7fd3c0154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:48.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.365+0000 7fd3c6f4c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3c015bb30 0x7fd3c015e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:48.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.365+0000 7fd3c6f4c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c00c8090 con 0x7fd3c015aba0 2023-09-23T03:16:48.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.365+0000 7fd3c6f4c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c00c84b0 con 0x7fd3c00cb3e0 2023-09-23T03:16:48.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.365+0000 7fd3c6f4c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c01f3180 con 0x7fd3c015bb30 2023-09-23T03:16:48.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.365+0000 7fd3bffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3c015aba0 0x7fd3c0154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:48.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.365+0000 7fd3bffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3c015aba0 0x7fd3c0154cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:41168/0 (socket says 172.21.15.154:41168) 2023-09-23T03:16:48.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.365+0000 7fd3bffff640 1 -- 172.21.15.154:0/1797133980 learned_addr learned my addr 172.21.15.154:0/1797133980 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:48.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.365+0000 7fd3c4ce8640 1 --2- 172.21.15.154:0/1797133980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3c00cb3e0 0x7fd3c01f2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:48.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.365+0000 7fd3c54e9640 1 --2- 172.21.15.154:0/1797133980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3c015bb30 0x7fd3c015e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:48.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.366+0000 7fd3bffff640 1 -- 172.21.15.154:0/1797133980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3c015bb30 msgr2=0x7fd3c015e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:48.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.366+0000 7fd3bffff640 1 --2- 172.21.15.154:0/1797133980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3c015bb30 0x7fd3c015e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:48.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.366+0000 7fd3bffff640 1 -- 172.21.15.154:0/1797133980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3c00cb3e0 msgr2=0x7fd3c01f2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:48.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.366+0000 7fd3bffff640 1 --2- 172.21.15.154:0/1797133980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3c00cb3e0 0x7fd3c01f2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:48.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.366+0000 7fd3bffff640 1 -- 172.21.15.154:0/1797133980 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a8009400 con 0x7fd3c015aba0 2023-09-23T03:16:48.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.366+0000 7fd3bffff640 1 --2- 172.21.15.154:0/1797133980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3c015aba0 0x7fd3c0154cd0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd3a802bee0 tx=0x7fd3a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:48.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.366+0000 7fd3bd7fa640 1 -- 172.21.15.154:0/1797133980 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a80317d0 con 0x7fd3c015aba0 2023-09-23T03:16:48.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.366+0000 7fd3bd7fa640 1 -- 172.21.15.154:0/1797133980 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3a803ba70 con 0x7fd3c015aba0 2023-09-23T03:16:48.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.366+0000 7fd3bd7fa640 1 -- 172.21.15.154:0/1797133980 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a803be20 con 0x7fd3c015aba0 2023-09-23T03:16:48.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.366+0000 7fd3c6f4c640 1 -- 172.21.15.154:0/1797133980 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3c00c7410 con 0x7fd3c015aba0 2023-09-23T03:16:48.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.366+0000 7fd3c6f4c640 1 -- 172.21.15.154:0/1797133980 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3c00c79d0 con 0x7fd3c015aba0 2023-09-23T03:16:48.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.368+0000 7fd3c6f4c640 1 -- 172.21.15.154:0/1797133980 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3c00c81d0 con 0x7fd3c015aba0 2023-09-23T03:16:48.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.369+0000 7fd3bd7fa640 1 -- 172.21.15.154:0/1797133980 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fd3a804c3b0 con 0x7fd3c015aba0 2023-09-23T03:16:48.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.369+0000 7fd3bd7fa640 1 --2- 172.21.15.154:0/1797133980 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd3a006af30 0x7fd3a006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:48.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.370+0000 7fd3bd7fa640 1 -- 172.21.15.154:0/1797133980 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7fd3a80abad0 con 0x7fd3c015aba0 2023-09-23T03:16:48.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.370+0000 7fd3c4ce8640 1 --2- 172.21.15.154:0/1797133980 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd3a006af30 0x7fd3a006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:48.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.370+0000 7fd3c4ce8640 1 --2- 172.21.15.154:0/1797133980 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd3a006af30 0x7fd3a006d3f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd3b00012e0 tx=0x7fd3b0007ef0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:48.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.374+0000 7fd3bd7fa640 1 -- 172.21.15.154:0/1797133980 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fd3c00c81d0 con 0x7fd3c015aba0 2023-09-23T03:16:48.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.562+0000 7fd3c6f4c640 1 -- 172.21.15.154:0/1797133980 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd3c00c81d0 con 0x7fd3c015aba0 2023-09-23T03:16:48.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.563+0000 7fd3bd7fa640 1 -- 172.21.15.154:0/1797133980 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd3a8035020 con 0x7fd3c015aba0 2023-09-23T03:16:48.563 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:16:48.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.565+0000 7fd3c6f4c640 1 -- 172.21.15.154:0/1797133980 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd3a006af30 msgr2=0x7fd3a006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:48.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.565+0000 7fd3c6f4c640 1 --2- 172.21.15.154:0/1797133980 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd3a006af30 0x7fd3a006d3f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd3b00012e0 tx=0x7fd3b0007ef0 comp rx=0 tx=0).stop 2023-09-23T03:16:48.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.565+0000 7fd3c6f4c640 1 -- 172.21.15.154:0/1797133980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3c015aba0 msgr2=0x7fd3c0154cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:48.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.565+0000 7fd3c6f4c640 1 --2- 172.21.15.154:0/1797133980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3c015aba0 0x7fd3c0154cd0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd3a802bee0 tx=0x7fd3a80046c0 comp rx=0 tx=0).stop 2023-09-23T03:16:48.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.565+0000 7fd3c6f4c640 1 -- 172.21.15.154:0/1797133980 shutdown_connections 2023-09-23T03:16:48.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.565+0000 7fd3c6f4c640 1 --2- 172.21.15.154:0/1797133980 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd3a006af30 0x7fd3a006d3f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:48.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.565+0000 7fd3c6f4c640 1 --2- 172.21.15.154:0/1797133980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3c015bb30 0x7fd3c015e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:48.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.565+0000 7fd3c6f4c640 1 --2- 172.21.15.154:0/1797133980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3c015aba0 0x7fd3c0154cd0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:48.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.565+0000 7fd3c6f4c640 1 --2- 172.21.15.154:0/1797133980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3c00cb3e0 0x7fd3c01f2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:48.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.565+0000 7fd3c6f4c640 1 -- 172.21.15.154:0/1797133980 >> 172.21.15.154:0/1797133980 conn(0x7fd3c006c010 msgr2=0x7fd3c00c5580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:48.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.566+0000 7fd3c6f4c640 1 -- 172.21.15.154:0/1797133980 shutdown_connections 2023-09-23T03:16:48.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.566+0000 7fd3c6f4c640 1 -- 172.21.15.154:0/1797133980 wait complete. 2023-09-23T03:16:48.585 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:16:48.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.804+0000 7f3c04589640 1 Processor -- start 2023-09-23T03:16:48.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.805+0000 7f3c04589640 1 -- start start 2023-09-23T03:16:48.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.806+0000 7f3c04589640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3bfc070c60 0x7f3bfc0caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:48.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.806+0000 7f3c04589640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bfc0710c0 con 0x7f3bfc0cb400 2023-09-23T03:16:48.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.806+0000 7f3c04589640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bfc0712f0 con 0x7f3bfc0ad830 2023-09-23T03:16:48.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.806+0000 7f3c04589640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bfc0cb890 con 0x7f3bfc070c60 2023-09-23T03:16:48.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.807+0000 7f3c02b26640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f3bfc0cb400 0x7f3bfc0aba70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:36334/0 (socket says 172.21.15.154:36334) 2023-09-23T03:16:48.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.807+0000 7f3c02b26640 1 -- 172.21.15.154:0/189326439 learned_addr learned my addr 172.21.15.154:0/189326439 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:48.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.807+0000 7f3c01b24640 1 --2- 172.21.15.154:0/189326439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3bfc070c60 0x7f3bfc0caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:48.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.807+0000 7f3c01323640 1 -- 172.21.15.154:0/189326439 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 795397157 0 0) 0x7f3bfc0710c0 con 0x7f3bfc0cb400 2023-09-23T03:16:48.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.807+0000 7f3c01323640 1 -- 172.21.15.154:0/189326439 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3be8003520 con 0x7f3bfc0cb400 2023-09-23T03:16:48.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.807+0000 7f3c01323640 1 -- 172.21.15.154:0/189326439 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 952271887 0 0) 0x7f3be8003520 con 0x7f3bfc0cb400 2023-09-23T03:16:48.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.807+0000 7f3c01323640 1 -- 172.21.15.154:0/189326439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3bfc070c60 msgr2=0x7f3bfc0caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:48.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.807+0000 7f3c01323640 1 --2- 172.21.15.154:0/189326439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3bfc070c60 0x7f3bfc0caec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:48.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.807+0000 7f3c01323640 1 -- 172.21.15.154:0/189326439 >> v1:172.21.15.190:6789/0 conn(0x7f3bfc0ad830 legacy=0x7f3bfc0c9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:48.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.808+0000 7f3c01323640 1 -- 172.21.15.154:0/189326439 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bfc0715d0 con 0x7f3bfc0cb400 2023-09-23T03:16:48.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.808+0000 7f3c01323640 1 -- 172.21.15.154:0/189326439 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f3bf40019f0 con 0x7f3bfc0cb400 2023-09-23T03:16:48.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.808+0000 7f3c01323640 1 -- 172.21.15.154:0/189326439 >> v1:172.21.15.154:6789/0 conn(0x7f3bfc0cb400 legacy=0x7f3bfc0aba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:48.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.808+0000 7f3c01323640 1 --2- 172.21.15.154:0/189326439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3be80039a0 0x7f3be8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:48.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.808+0000 7f3c01323640 1 --2- 172.21.15.154:0/189326439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3bfc070c60 0x7f3be80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:48.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.808+0000 7f3c01323640 1 --2- 172.21.15.154:0/189326439 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3be8004d00 0x7f3be8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:48.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.808+0000 7f3c01323640 1 -- 172.21.15.154:0/189326439 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bfc0cb890 con 0x7f3be8004d00 2023-09-23T03:16:48.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.808+0000 7f3c01323640 1 -- 172.21.15.154:0/189326439 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be800ba30 con 0x7f3be80039a0 2023-09-23T03:16:48.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.808+0000 7f3c01323640 1 -- 172.21.15.154:0/189326439 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3be800bbf0 con 0x7f3bfc070c60 2023-09-23T03:16:48.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.808+0000 7f3c02b26640 1 --2- 172.21.15.154:0/189326439 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3be8004d00 0x7f3be8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:48.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.808+0000 7f3c02325640 1 --2- 172.21.15.154:0/189326439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3be80039a0 0x7f3be8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:48.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.808+0000 7f3c01b24640 1 --2- 172.21.15.154:0/189326439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3bfc070c60 0x7f3be80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:48.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.808+0000 7f3c02b26640 1 -- 172.21.15.154:0/189326439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3bfc070c60 msgr2=0x7f3be80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:48.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.808+0000 7f3c02b26640 1 --2- 172.21.15.154:0/189326439 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3bfc070c60 0x7f3be80047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:48.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.808+0000 7f3c02b26640 1 -- 172.21.15.154:0/189326439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3be80039a0 msgr2=0x7f3be8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:48.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.808+0000 7f3c02b26640 1 --2- 172.21.15.154:0/189326439 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3be80039a0 0x7f3be8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:48.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.808+0000 7f3c02b26640 1 -- 172.21.15.154:0/189326439 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be8003520 con 0x7f3be8004d00 2023-09-23T03:16:48.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.810+0000 7f3c02b26640 1 --2- 172.21.15.154:0/189326439 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3be8004d00 0x7f3be8005180 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3bf400a800 tx=0x7f3be80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac7d98762189829 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:48.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.810+0000 7f3c01323640 1 -- 172.21.15.154:0/189326439 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf40072d0 con 0x7f3be8004d00 2023-09-23T03:16:48.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.810+0000 7f3c01323640 1 -- 172.21.15.154:0/189326439 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bf402f070 con 0x7f3be8004d00 2023-09-23T03:16:48.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.810+0000 7f3c01323640 1 -- 172.21.15.154:0/189326439 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf40076b0 con 0x7f3be8004d00 2023-09-23T03:16:48.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.810+0000 7f3c04589640 1 -- 172.21.15.154:0/189326439 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3be8004d00 msgr2=0x7f3be8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:48.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.810+0000 7f3c04589640 1 --2- 172.21.15.154:0/189326439 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3be8004d00 0x7f3be8005180 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3bf400a800 tx=0x7f3be80057f0 comp rx=0 tx=0).stop 2023-09-23T03:16:48.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.810+0000 7f3c02b26640 1 -- 172.21.15.154:0/189326439 reap_dead start 2023-09-23T03:16:48.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.811+0000 7f3c04589640 1 -- 172.21.15.154:0/189326439 shutdown_connections 2023-09-23T03:16:48.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.811+0000 7f3c04589640 1 -- 172.21.15.154:0/189326439 >> 172.21.15.154:0/189326439 conn(0x7f3bfc06c010 msgr2=0x7f3bfc06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:48.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.811+0000 7f3c04589640 1 -- 172.21.15.154:0/189326439 shutdown_connections 2023-09-23T03:16:48.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.811+0000 7f3c04589640 1 -- 172.21.15.154:0/189326439 wait complete. 2023-09-23T03:16:48.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.812+0000 7f3c04589640 1 Processor -- start 2023-09-23T03:16:48.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.813+0000 7f3c04589640 1 -- start start 2023-09-23T03:16:48.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.813+0000 7f3c04589640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3be8004d00 0x7f3bfc0b3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:48.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.814+0000 7f3c04589640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bfc0af8a0 0x7f3bfc0afd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:48.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.814+0000 7f3c04589640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3bfc0b1780 0x7f3bfc0b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:48.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.814+0000 7f3c04589640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bfc0c5580 con 0x7f3bfc0af8a0 2023-09-23T03:16:48.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.814+0000 7f3c04589640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bfc0c8330 con 0x7f3be8004d00 2023-09-23T03:16:48.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.814+0000 7f3c04589640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bfc0c57d0 con 0x7f3bfc0b1780 2023-09-23T03:16:48.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.814+0000 7f3c01b24640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bfc0af8a0 0x7f3bfc0afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:48.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.814+0000 7f3c01b24640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bfc0af8a0 0x7f3bfc0afd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:41186/0 (socket says 172.21.15.154:41186) 2023-09-23T03:16:48.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.814+0000 7f3c01b24640 1 -- 172.21.15.154:0/2973363099 learned_addr learned my addr 172.21.15.154:0/2973363099 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:48.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.814+0000 7f3c02325640 1 --2- 172.21.15.154:0/2973363099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3be8004d00 0x7f3bfc0b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:48.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.814+0000 7f3c02b26640 1 --2- 172.21.15.154:0/2973363099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3bfc0b1780 0x7f3bfc0b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:48.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.814+0000 7f3c01b24640 1 -- 172.21.15.154:0/2973363099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3bfc0b1780 msgr2=0x7f3bfc0b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:48.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.814+0000 7f3c01b24640 1 --2- 172.21.15.154:0/2973363099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3bfc0b1780 0x7f3bfc0b1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:48.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.814+0000 7f3c01b24640 1 -- 172.21.15.154:0/2973363099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3be8004d00 msgr2=0x7f3bfc0b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:48.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.814+0000 7f3c01b24640 1 --2- 172.21.15.154:0/2973363099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3be8004d00 0x7f3bfc0b3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:48.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.814+0000 7f3c01b24640 1 -- 172.21.15.154:0/2973363099 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bfc0c5ab0 con 0x7f3bfc0af8a0 2023-09-23T03:16:48.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.815+0000 7f3c01b24640 1 --2- 172.21.15.154:0/2973363099 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bfc0af8a0 0x7f3bfc0afd20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3bec010860 tx=0x7f3bec010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:48.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.815+0000 7f3bf2ffd640 1 -- 172.21.15.154:0/2973363099 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bec009610 con 0x7f3bfc0af8a0 2023-09-23T03:16:48.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.815+0000 7f3bf2ffd640 1 -- 172.21.15.154:0/2973363099 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bec00eb60 con 0x7f3bfc0af8a0 2023-09-23T03:16:48.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.815+0000 7f3bf2ffd640 1 -- 172.21.15.154:0/2973363099 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bec01c400 con 0x7f3bfc0af8a0 2023-09-23T03:16:48.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.815+0000 7f3c04589640 1 -- 172.21.15.154:0/2973363099 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bfc0c5d90 con 0x7f3bfc0af8a0 2023-09-23T03:16:48.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.815+0000 7f3c04589640 1 -- 172.21.15.154:0/2973363099 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bfc0b2140 con 0x7f3bfc0af8a0 2023-09-23T03:16:48.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.816+0000 7f3c04589640 1 -- 172.21.15.154:0/2973363099 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bfc04e920 con 0x7f3bfc0af8a0 2023-09-23T03:16:48.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.817+0000 7f3bf2ffd640 1 -- 172.21.15.154:0/2973363099 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f3bec01c560 con 0x7f3bfc0af8a0 2023-09-23T03:16:48.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.818+0000 7f3bf2ffd640 1 --2- 172.21.15.154:0/2973363099 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3be006af30 0x7f3be006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:48.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.818+0000 7f3bf2ffd640 1 -- 172.21.15.154:0/2973363099 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f3bec08a120 con 0x7f3bfc0af8a0 2023-09-23T03:16:48.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.818+0000 7f3c02325640 1 --2- 172.21.15.154:0/2973363099 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3be006af30 0x7f3be006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:48.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.819+0000 7f3c02325640 1 --2- 172.21.15.154:0/2973363099 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3be006af30 0x7f3be006d3f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8007fd0 tx=0x7f3bf8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:48.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:48.822+0000 7f3bf2ffd640 1 -- 172.21.15.154:0/2973363099 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f3bec054dc0 con 0x7f3bfc0af8a0 2023-09-23T03:16:49.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.017+0000 7f3c04589640 1 -- 172.21.15.154:0/2973363099 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3bfc0cbdc0 con 0x7f3bfc0af8a0 2023-09-23T03:16:49.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.018+0000 7f3bf2ffd640 1 -- 172.21.15.154:0/2973363099 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3bec0583e0 con 0x7f3bfc0af8a0 2023-09-23T03:16:49.018 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:16:49.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.020+0000 7f3c04589640 1 -- 172.21.15.154:0/2973363099 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3be006af30 msgr2=0x7f3be006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:49.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.020+0000 7f3c04589640 1 --2- 172.21.15.154:0/2973363099 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3be006af30 0x7f3be006d3f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8007fd0 tx=0x7f3bf8009040 comp rx=0 tx=0).stop 2023-09-23T03:16:49.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.020+0000 7f3c04589640 1 -- 172.21.15.154:0/2973363099 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bfc0af8a0 msgr2=0x7f3bfc0afd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:49.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.020+0000 7f3c04589640 1 --2- 172.21.15.154:0/2973363099 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bfc0af8a0 0x7f3bfc0afd20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3bec010860 tx=0x7f3bec010d20 comp rx=0 tx=0).stop 2023-09-23T03:16:49.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.020+0000 7f3c04589640 1 -- 172.21.15.154:0/2973363099 shutdown_connections 2023-09-23T03:16:49.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.021+0000 7f3c04589640 1 --2- 172.21.15.154:0/2973363099 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3be006af30 0x7f3be006d3f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:49.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.021+0000 7f3c04589640 1 --2- 172.21.15.154:0/2973363099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3bfc0b1780 0x7f3bfc0b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:49.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.021+0000 7f3c04589640 1 --2- 172.21.15.154:0/2973363099 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3bfc0af8a0 0x7f3bfc0afd20 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:49.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.021+0000 7f3c04589640 1 --2- 172.21.15.154:0/2973363099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3be8004d00 0x7f3bfc0b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:49.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.021+0000 7f3c04589640 1 -- 172.21.15.154:0/2973363099 >> 172.21.15.154:0/2973363099 conn(0x7f3bfc06c010 msgr2=0x7f3bfc070150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:49.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.021+0000 7f3c04589640 1 -- 172.21.15.154:0/2973363099 shutdown_connections 2023-09-23T03:16:49.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.021+0000 7f3c04589640 1 -- 172.21.15.154:0/2973363099 wait complete. 2023-09-23T03:16:49.040 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-23T03:16:49.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.261+0000 7f53d1664640 1 Processor -- start 2023-09-23T03:16:49.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.262+0000 7f53d1664640 1 -- start start 2023-09-23T03:16:49.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.263+0000 7f53d1664640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f53cc15c9f0 0x7f53cc15d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:49.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.263+0000 7f53d1664640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53cc0cf010 0x7f53cc0cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:49.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.263+0000 7f53d1664640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53cc15cdd0 con 0x7f53cc1604e0 2023-09-23T03:16:49.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.263+0000 7f53d1664640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53cc0ce4c0 con 0x7f53cc15c9f0 2023-09-23T03:16:49.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.263+0000 7f53d1664640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53cc0cd8b0 con 0x7f53cc0cf010 2023-09-23T03:16:49.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.263+0000 7f53caffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f53cc15c9f0 0x7f53cc15d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:49.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.263+0000 7f53caffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f53cc15c9f0 0x7f53cc15d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:41196/0 (socket says 172.21.15.154:41196) 2023-09-23T03:16:49.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.263+0000 7f53caffd640 1 -- 172.21.15.154:0/1575222945 learned_addr learned my addr 172.21.15.154:0/1575222945 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:49.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.263+0000 7f53ca7fc640 1 --2- 172.21.15.154:0/1575222945 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53cc0cf010 0x7f53cc0cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:49.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.264+0000 7f53caffd640 1 -- 172.21.15.154:0/1575222945 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53cc0cf010 msgr2=0x7f53cc0cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:49.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.264+0000 7f53caffd640 1 --2- 172.21.15.154:0/1575222945 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53cc0cf010 0x7f53cc0cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:49.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.264+0000 7f53caffd640 1 -- 172.21.15.154:0/1575222945 >> v1:172.21.15.190:6789/0 conn(0x7f53cc1604e0 legacy=0x7f53cc160970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:49.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.264+0000 7f53caffd640 1 -- 172.21.15.154:0/1575222945 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53cc0cf9b0 con 0x7f53cc15c9f0 2023-09-23T03:16:49.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.265+0000 7f53caffd640 1 --2- 172.21.15.154:0/1575222945 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53cc15c9f0 0x7f53cc15d7d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f53bc009ea0 tx=0x7f53bc02bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33bb790ae322568b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:49.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.265+0000 7f53c9ffb640 1 -- 172.21.15.154:0/1575222945 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc02cab0 con 0x7f53cc15c9f0 2023-09-23T03:16:49.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.265+0000 7f53c9ffb640 1 -- 172.21.15.154:0/1575222945 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53bc02cc10 con 0x7f53cc15c9f0 2023-09-23T03:16:49.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.265+0000 7f53c9ffb640 1 -- 172.21.15.154:0/1575222945 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc0343f0 con 0x7f53cc15c9f0 2023-09-23T03:16:49.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.266+0000 7f53d1664640 1 -- 172.21.15.154:0/1575222945 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53cc15c9f0 msgr2=0x7f53cc15d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:49.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.266+0000 7f53d1664640 1 --2- 172.21.15.154:0/1575222945 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53cc15c9f0 0x7f53cc15d7d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f53bc009ea0 tx=0x7f53bc02bf10 comp rx=0 tx=0).stop 2023-09-23T03:16:49.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.266+0000 7f53d1664640 1 -- 172.21.15.154:0/1575222945 shutdown_connections 2023-09-23T03:16:49.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.266+0000 7f53d1664640 1 --2- 172.21.15.154:0/1575222945 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53cc0cf010 0x7f53cc0cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:49.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.266+0000 7f53d1664640 1 --2- 172.21.15.154:0/1575222945 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53cc15c9f0 0x7f53cc15d7d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:49.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.266+0000 7f53d1664640 1 -- 172.21.15.154:0/1575222945 >> 172.21.15.154:0/1575222945 conn(0x7f53cc09bc10 msgr2=0x7f53cc0d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:49.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.266+0000 7f53d1664640 1 -- 172.21.15.154:0/1575222945 shutdown_connections 2023-09-23T03:16:49.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.266+0000 7f53d1664640 1 -- 172.21.15.154:0/1575222945 wait complete. 2023-09-23T03:16:49.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.267+0000 7f53d1664640 1 Processor -- start 2023-09-23T03:16:49.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.268+0000 7f53d1664640 1 -- start start 2023-09-23T03:16:49.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.269+0000 7f53d1664640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53cc0cf010 0x7f53cc0cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:49.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.269+0000 7f53d1664640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53cc15c9f0 0x7f53cc1f1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:49.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.269+0000 7f53d1664640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53cc1604e0 0x7f53cc1f1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:49.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.269+0000 7f53d1664640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53cc0b32a0 con 0x7f53cc1604e0 2023-09-23T03:16:49.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.269+0000 7f53d1664640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53cc0b2a00 con 0x7f53cc15c9f0 2023-09-23T03:16:49.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.269+0000 7f53d1664640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53cc0b2ce0 con 0x7f53cc0cf010 2023-09-23T03:16:49.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.269+0000 7f53cb7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53cc1604e0 0x7f53cc1f1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:49.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.269+0000 7f53cb7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53cc1604e0 0x7f53cc1f1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:41208/0 (socket says 172.21.15.154:41208) 2023-09-23T03:16:49.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.269+0000 7f53cb7fe640 1 -- 172.21.15.154:0/3332152176 learned_addr learned my addr 172.21.15.154:0/3332152176 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:49.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.269+0000 7f53caffd640 1 --2- 172.21.15.154:0/3332152176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53cc0cf010 0x7f53cc0cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:49.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.269+0000 7f53ca7fc640 1 --2- 172.21.15.154:0/3332152176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53cc15c9f0 0x7f53cc1f1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:49.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.269+0000 7f53cb7fe640 1 -- 172.21.15.154:0/3332152176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53cc0cf010 msgr2=0x7f53cc0cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:49.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.269+0000 7f53cb7fe640 1 --2- 172.21.15.154:0/3332152176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53cc0cf010 0x7f53cc0cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:49.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.269+0000 7f53cb7fe640 1 -- 172.21.15.154:0/3332152176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53cc15c9f0 msgr2=0x7f53cc1f1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:49.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.269+0000 7f53cb7fe640 1 --2- 172.21.15.154:0/3332152176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53cc15c9f0 0x7f53cc1f1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:49.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.269+0000 7f53cb7fe640 1 -- 172.21.15.154:0/3332152176 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53bc009b30 con 0x7f53cc1604e0 2023-09-23T03:16:49.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.270+0000 7f53cb7fe640 1 --2- 172.21.15.154:0/3332152176 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53cc1604e0 0x7f53cc1f1bb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f53c400b410 tx=0x7f53c400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:49.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.270+0000 7f53c37fe640 1 -- 172.21.15.154:0/3332152176 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c4004020 con 0x7f53cc1604e0 2023-09-23T03:16:49.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.270+0000 7f53c37fe640 1 -- 172.21.15.154:0/3332152176 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c4004180 con 0x7f53cc1604e0 2023-09-23T03:16:49.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.270+0000 7f53d1664640 1 -- 172.21.15.154:0/3332152176 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53cc0b2fc0 con 0x7f53cc1604e0 2023-09-23T03:16:49.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.270+0000 7f53c37fe640 1 -- 172.21.15.154:0/3332152176 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c4011630 con 0x7f53cc1604e0 2023-09-23T03:16:49.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.270+0000 7f53d1664640 1 -- 172.21.15.154:0/3332152176 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53cc0cc350 con 0x7f53cc1604e0 2023-09-23T03:16:49.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.271+0000 7f53d1664640 1 -- 172.21.15.154:0/3332152176 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53cc157bf0 con 0x7f53cc1604e0 2023-09-23T03:16:49.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.272+0000 7f53c37fe640 1 -- 172.21.15.154:0/3332152176 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f53c4011790 con 0x7f53cc1604e0 2023-09-23T03:16:49.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.273+0000 7f53c37fe640 1 --2- 172.21.15.154:0/3332152176 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f53ac06af30 0x7f53ac06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:49.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.273+0000 7f53c37fe640 1 -- 172.21.15.154:0/3332152176 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f53c4089750 con 0x7f53cc1604e0 2023-09-23T03:16:49.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.273+0000 7f53caffd640 1 --2- 172.21.15.154:0/3332152176 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f53ac06af30 0x7f53ac06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:49.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.274+0000 7f53caffd640 1 --2- 172.21.15.154:0/3332152176 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f53ac06af30 0x7f53ac06d3f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f53bc035000 tx=0x7f53bc030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:49.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.277+0000 7f53c37fe640 1 -- 172.21.15.154:0/3332152176 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f53cc157bf0 con 0x7f53cc1604e0 2023-09-23T03:16:49.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.467+0000 7f53d1664640 1 -- 172.21.15.154:0/3332152176 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f53cc157bf0 con 0x7f53cc1604e0 2023-09-23T03:16:49.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.469+0000 7f53c37fe640 1 -- 172.21.15.154:0/3332152176 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f53cc157bf0 con 0x7f53cc1604e0 2023-09-23T03:16:49.469 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:16:49.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.471+0000 7f53d1664640 1 -- 172.21.15.154:0/3332152176 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f53ac06af30 msgr2=0x7f53ac06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:49.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.471+0000 7f53d1664640 1 --2- 172.21.15.154:0/3332152176 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f53ac06af30 0x7f53ac06d3f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f53bc035000 tx=0x7f53bc030000 comp rx=0 tx=0).stop 2023-09-23T03:16:49.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.471+0000 7f53d1664640 1 -- 172.21.15.154:0/3332152176 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53cc1604e0 msgr2=0x7f53cc1f1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:49.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.471+0000 7f53d1664640 1 --2- 172.21.15.154:0/3332152176 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53cc1604e0 0x7f53cc1f1bb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f53c400b410 tx=0x7f53c400b8d0 comp rx=0 tx=0).stop 2023-09-23T03:16:49.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.471+0000 7f53d1664640 1 -- 172.21.15.154:0/3332152176 shutdown_connections 2023-09-23T03:16:49.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.471+0000 7f53d1664640 1 --2- 172.21.15.154:0/3332152176 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f53ac06af30 0x7f53ac06d3f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:49.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.471+0000 7f53d1664640 1 --2- 172.21.15.154:0/3332152176 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53cc1604e0 0x7f53cc1f1bb0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:49.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.471+0000 7f53d1664640 1 --2- 172.21.15.154:0/3332152176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53cc15c9f0 0x7f53cc1f1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:49.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.471+0000 7f53d1664640 1 --2- 172.21.15.154:0/3332152176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53cc0cf010 0x7f53cc0cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:49.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.471+0000 7f53d1664640 1 -- 172.21.15.154:0/3332152176 >> 172.21.15.154:0/3332152176 conn(0x7f53cc09bc10 msgr2=0x7f53cc06af50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:49.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.472+0000 7f53d1664640 1 -- 172.21.15.154:0/3332152176 shutdown_connections 2023-09-23T03:16:49.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.472+0000 7f53d1664640 1 -- 172.21.15.154:0/3332152176 wait complete. 2023-09-23T03:16:49.492 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-23T03:16:49.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.699+0000 7f4fe2dc4640 1 Processor -- start 2023-09-23T03:16:49.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.700+0000 7f4fe2dc4640 1 -- start start 2023-09-23T03:16:49.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.701+0000 7f4fe2dc4640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4fdc06c7a0 0x7f4fdc15eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:49.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.701+0000 7f4fe2dc4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fdc15ab00 0x7f4fdc15af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:49.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.701+0000 7f4fe2dc4640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fdc06bff0 con 0x7f4fdc06fdc0 2023-09-23T03:16:49.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.701+0000 7f4fe2dc4640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fdc155070 con 0x7f4fdc06c7a0 2023-09-23T03:16:49.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.701+0000 7f4fe2dc4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fdc15e0f0 con 0x7f4fdc15ab00 2023-09-23T03:16:49.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.702+0000 7f4fe1dc2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4fdc06c7a0 0x7f4fdc15eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:49.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.702+0000 7f4fe1dc2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4fdc06c7a0 0x7f4fdc15eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:41224/0 (socket says 172.21.15.154:41224) 2023-09-23T03:16:49.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.702+0000 7f4fe1dc2640 1 -- 172.21.15.154:0/1063308361 learned_addr learned my addr 172.21.15.154:0/1063308361 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:49.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.702+0000 7f4fe15c1640 1 --2- 172.21.15.154:0/1063308361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fdc15ab00 0x7f4fdc15af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:49.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.702+0000 7f4fe1dc2640 1 -- 172.21.15.154:0/1063308361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fdc15ab00 msgr2=0x7f4fdc15af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:49.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.702+0000 7f4fe1dc2640 1 --2- 172.21.15.154:0/1063308361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fdc15ab00 0x7f4fdc15af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:49.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.702+0000 7f4fe1dc2640 1 -- 172.21.15.154:0/1063308361 >> v1:172.21.15.190:6789/0 conn(0x7f4fdc06fdc0 legacy=0x7f4fdc0ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:49.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.702+0000 7f4fe1dc2640 1 -- 172.21.15.154:0/1063308361 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fdc06cb80 con 0x7f4fdc06c7a0 2023-09-23T03:16:49.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.702+0000 7f4fe15c1640 1 --2- 172.21.15.154:0/1063308361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fdc15ab00 0x7f4fdc15af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:16:49.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.703+0000 7f4fe1dc2640 1 --2- 172.21.15.154:0/1063308361 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fdc06c7a0 0x7f4fdc15eb00 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc009870 tx=0x7f4fcc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17988ca2e21069f9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:49.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.703+0000 7f4fe0dc0640 1 -- 172.21.15.154:0/1063308361 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc030840 con 0x7f4fdc06c7a0 2023-09-23T03:16:49.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.704+0000 7f4fe0dc0640 1 -- 172.21.15.154:0/1063308361 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fcc0309a0 con 0x7f4fdc06c7a0 2023-09-23T03:16:49.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.704+0000 7f4fe0dc0640 1 -- 172.21.15.154:0/1063308361 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc030d30 con 0x7f4fdc06c7a0 2023-09-23T03:16:49.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.704+0000 7f4fe2dc4640 1 -- 172.21.15.154:0/1063308361 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fdc06c7a0 msgr2=0x7f4fdc15eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:49.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.704+0000 7f4fe2dc4640 1 --2- 172.21.15.154:0/1063308361 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fdc06c7a0 0x7f4fdc15eb00 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc009870 tx=0x7f4fcc02b980 comp rx=0 tx=0).stop 2023-09-23T03:16:49.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.704+0000 7f4fe2dc4640 1 -- 172.21.15.154:0/1063308361 shutdown_connections 2023-09-23T03:16:49.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.704+0000 7f4fe2dc4640 1 --2- 172.21.15.154:0/1063308361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fdc15ab00 0x7f4fdc15af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:49.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.704+0000 7f4fe2dc4640 1 --2- 172.21.15.154:0/1063308361 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fdc06c7a0 0x7f4fdc15eb00 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:49.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.704+0000 7f4fe2dc4640 1 -- 172.21.15.154:0/1063308361 >> 172.21.15.154:0/1063308361 conn(0x7f4fdc06a390 msgr2=0x7f4fdc0afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:49.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.704+0000 7f4fe2dc4640 1 -- 172.21.15.154:0/1063308361 shutdown_connections 2023-09-23T03:16:49.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.704+0000 7f4fe2dc4640 1 -- 172.21.15.154:0/1063308361 wait complete. 2023-09-23T03:16:49.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.706+0000 7f4fe2dc4640 1 Processor -- start 2023-09-23T03:16:49.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.706+0000 7f4fe2dc4640 1 -- start start 2023-09-23T03:16:49.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.707+0000 7f4fe2dc4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fdc06c7a0 0x7f4fdc0c3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:49.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.707+0000 7f4fe2dc4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fdc06fdc0 0x7f4fdc0b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:49.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.707+0000 7f4fe2dc4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fdc15ab00 0x7f4fdc14b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:49.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.707+0000 7f4fe2dc4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fdc14bcc0 con 0x7f4fdc06c7a0 2023-09-23T03:16:49.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.707+0000 7f4fe2dc4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fdc0c3c70 con 0x7f4fdc15ab00 2023-09-23T03:16:49.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.707+0000 7f4fe2dc4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fdc0c3f50 con 0x7f4fdc06fdc0 2023-09-23T03:16:49.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.707+0000 7f4fe1dc2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fdc06c7a0 0x7f4fdc0c3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:49.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.708+0000 7f4fe1dc2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fdc06c7a0 0x7f4fdc0c3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:41238/0 (socket says 172.21.15.154:41238) 2023-09-23T03:16:49.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.708+0000 7f4fe1dc2640 1 -- 172.21.15.154:0/2469658770 learned_addr learned my addr 172.21.15.154:0/2469658770 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:49.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.708+0000 7f4fe15c1640 1 --2- 172.21.15.154:0/2469658770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fdc06fdc0 0x7f4fdc0b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:49.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.708+0000 7f4fe25c3640 1 --2- 172.21.15.154:0/2469658770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fdc15ab00 0x7f4fdc14b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:49.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.708+0000 7f4fe1dc2640 1 -- 172.21.15.154:0/2469658770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fdc06fdc0 msgr2=0x7f4fdc0b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:49.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.708+0000 7f4fe1dc2640 1 --2- 172.21.15.154:0/2469658770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fdc06fdc0 0x7f4fdc0b21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:49.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.708+0000 7f4fe1dc2640 1 -- 172.21.15.154:0/2469658770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fdc15ab00 msgr2=0x7f4fdc14b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:49.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.708+0000 7f4fe1dc2640 1 --2- 172.21.15.154:0/2469658770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fdc15ab00 0x7f4fdc14b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:49.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.708+0000 7f4fe1dc2640 1 -- 172.21.15.154:0/2469658770 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fcc0094d0 con 0x7f4fdc06c7a0 2023-09-23T03:16:49.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.708+0000 7f4fe25c3640 1 --2- 172.21.15.154:0/2469658770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fdc15ab00 0x7f4fdc14b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:49.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.708+0000 7f4fe1dc2640 1 --2- 172.21.15.154:0/2469658770 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fdc06c7a0 0x7f4fdc0c3690 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc000c00 tx=0x7f4fcc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:49.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.709+0000 7f4fda7fc640 1 -- 172.21.15.154:0/2469658770 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc039c90 con 0x7f4fdc06c7a0 2023-09-23T03:16:49.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.709+0000 7f4fda7fc640 1 -- 172.21.15.154:0/2469658770 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fcc039df0 con 0x7f4fdc06c7a0 2023-09-23T03:16:49.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.709+0000 7f4fda7fc640 1 -- 172.21.15.154:0/2469658770 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc043640 con 0x7f4fdc06c7a0 2023-09-23T03:16:49.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.709+0000 7f4fe2dc4640 1 -- 172.21.15.154:0/2469658770 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fdc0c4230 con 0x7f4fdc06c7a0 2023-09-23T03:16:49.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.709+0000 7f4fe2dc4640 1 -- 172.21.15.154:0/2469658770 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fdc0c47f0 con 0x7f4fdc06c7a0 2023-09-23T03:16:49.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.710+0000 7f4fe2dc4640 1 -- 172.21.15.154:0/2469658770 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fdc0736b0 con 0x7f4fdc06c7a0 2023-09-23T03:16:49.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.711+0000 7f4fda7fc640 1 -- 172.21.15.154:0/2469658770 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f4fcc0437a0 con 0x7f4fdc06c7a0 2023-09-23T03:16:49.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.712+0000 7f4fda7fc640 1 --2- 172.21.15.154:0/2469658770 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4fc806af30 0x7f4fc806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:49.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.712+0000 7f4fda7fc640 1 -- 172.21.15.154:0/2469658770 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f4fcc03e070 con 0x7f4fdc06c7a0 2023-09-23T03:16:49.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.712+0000 7f4fe15c1640 1 --2- 172.21.15.154:0/2469658770 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4fc806af30 0x7f4fc806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:49.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.713+0000 7f4fe15c1640 1 --2- 172.21.15.154:0/2469658770 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4fc806af30 0x7f4fc806d3f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0006fd0 tx=0x7f4fd0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:49.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.716+0000 7f4fda7fc640 1 -- 172.21.15.154:0/2469658770 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f4fcc077880 con 0x7f4fdc06c7a0 2023-09-23T03:16:49.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.917+0000 7f4fe2dc4640 1 -- 172.21.15.154:0/2469658770 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4fdc04e920 con 0x7f4fdc06c7a0 2023-09-23T03:16:49.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.918+0000 7f4fda7fc640 1 -- 172.21.15.154:0/2469658770 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4fcc07aea0 con 0x7f4fdc06c7a0 2023-09-23T03:16:49.919 INFO:teuthology.orchestra.run.smithi154.stderr:listed 0 entries 2023-09-23T03:16:49.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.920+0000 7f4fe2dc4640 1 -- 172.21.15.154:0/2469658770 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4fc806af30 msgr2=0x7f4fc806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:49.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.920+0000 7f4fe2dc4640 1 --2- 172.21.15.154:0/2469658770 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4fc806af30 0x7f4fc806d3f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0006fd0 tx=0x7f4fd0008040 comp rx=0 tx=0).stop 2023-09-23T03:16:49.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.920+0000 7f4fe2dc4640 1 -- 172.21.15.154:0/2469658770 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fdc06c7a0 msgr2=0x7f4fdc0c3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:49.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.920+0000 7f4fe2dc4640 1 --2- 172.21.15.154:0/2469658770 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fdc06c7a0 0x7f4fdc0c3690 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc000c00 tx=0x7f4fcc004730 comp rx=0 tx=0).stop 2023-09-23T03:16:49.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.921+0000 7f4fe2dc4640 1 -- 172.21.15.154:0/2469658770 shutdown_connections 2023-09-23T03:16:49.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.921+0000 7f4fe2dc4640 1 --2- 172.21.15.154:0/2469658770 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4fc806af30 0x7f4fc806d3f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:49.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.921+0000 7f4fe2dc4640 1 --2- 172.21.15.154:0/2469658770 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fdc15ab00 0x7f4fdc14b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:49.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.921+0000 7f4fe2dc4640 1 --2- 172.21.15.154:0/2469658770 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fdc06fdc0 0x7f4fdc0b21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:49.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.921+0000 7f4fe2dc4640 1 --2- 172.21.15.154:0/2469658770 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fdc06c7a0 0x7f4fdc0c3690 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:49.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.921+0000 7f4fe2dc4640 1 -- 172.21.15.154:0/2469658770 >> 172.21.15.154:0/2469658770 conn(0x7f4fdc06a390 msgr2=0x7f4fdc0c8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:49.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.921+0000 7f4fe2dc4640 1 -- 172.21.15.154:0/2469658770 shutdown_connections 2023-09-23T03:16:49.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:49.921+0000 7f4fe2dc4640 1 -- 172.21.15.154:0/2469658770 wait complete. 2023-09-23T03:16:49.940 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:16:49.940 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:16:49.994 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:16:50.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.215+0000 7f450a67c640 1 Processor -- start 2023-09-23T03:16:50.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.215+0000 7f450a67c640 1 -- start start 2023-09-23T03:16:50.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.217+0000 7f450a67c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f45040acce0 0x7f450415ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:50.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.217+0000 7f450a67c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f45040b1b20 0x7f45040ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:50.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.217+0000 7f450a67c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45040cee70 0x7f450406cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:50.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.217+0000 7f450a67c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45040b2e90 con 0x7f45040acce0 2023-09-23T03:16:50.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.217+0000 7f450a67c640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45040cf300 con 0x7f45040b1b20 2023-09-23T03:16:50.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.217+0000 7f450a67c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f450415ac30 con 0x7f45040cee70 2023-09-23T03:16:50.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.217+0000 7f4503fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f45040acce0 0x7f450415ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:50.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.217+0000 7f4503fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f45040acce0 0x7f450415ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:41250/0 (socket says 172.21.15.154:41250) 2023-09-23T03:16:50.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.217+0000 7f4503fff640 1 -- 172.21.15.154:0/1699598366 learned_addr learned my addr 172.21.15.154:0/1699598366 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:50.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.217+0000 7f4508c19640 1 --2- 172.21.15.154:0/1699598366 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45040cee70 0x7f450406cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:50.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.217+0000 7f44fbfff640 1 --2- 172.21.15.154:0/1699598366 >> v2:172.21.15.190:3300/0 conn(0x7f45040b1b20 0x7f45040ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:50.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.218+0000 7f4503fff640 1 -- 172.21.15.154:0/1699598366 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45040cee70 msgr2=0x7f450406cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:50.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.218+0000 7f4503fff640 1 --2- 172.21.15.154:0/1699598366 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45040cee70 0x7f450406cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:50.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.218+0000 7f4503fff640 1 -- 172.21.15.154:0/1699598366 >> v2:172.21.15.190:3300/0 conn(0x7f45040b1b20 msgr2=0x7f45040ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:50.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.218+0000 7f4503fff640 1 --2- 172.21.15.154:0/1699598366 >> v2:172.21.15.190:3300/0 conn(0x7f45040b1b20 0x7f45040ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:50.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.218+0000 7f4503fff640 1 -- 172.21.15.154:0/1699598366 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45040b1f70 con 0x7f45040acce0 2023-09-23T03:16:50.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.218+0000 7f44fbfff640 1 --2- 172.21.15.154:0/1699598366 >> v2:172.21.15.190:3300/0 conn(0x7f45040b1b20 0x7f45040ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:50.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.219+0000 7f4503fff640 1 --2- 172.21.15.154:0/1699598366 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45040acce0 0x7f450415ed60 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f45040c9430 tx=0x7f44f402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ec60343b8eb2c10 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:50.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.219+0000 7f45037fe640 1 -- 172.21.15.154:0/1699598366 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f402e070 con 0x7f45040acce0 2023-09-23T03:16:50.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.219+0000 7f45037fe640 1 -- 172.21.15.154:0/1699598366 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f4004030 con 0x7f45040acce0 2023-09-23T03:16:50.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.219+0000 7f45037fe640 1 -- 172.21.15.154:0/1699598366 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f4004420 con 0x7f45040acce0 2023-09-23T03:16:50.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.219+0000 7f450a67c640 1 -- 172.21.15.154:0/1699598366 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45040acce0 msgr2=0x7f450415ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:50.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.220+0000 7f450a67c640 1 --2- 172.21.15.154:0/1699598366 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45040acce0 0x7f450415ed60 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f45040c9430 tx=0x7f44f402b840 comp rx=0 tx=0).stop 2023-09-23T03:16:50.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.220+0000 7f450a67c640 1 -- 172.21.15.154:0/1699598366 shutdown_connections 2023-09-23T03:16:50.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.220+0000 7f450a67c640 1 --2- 172.21.15.154:0/1699598366 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45040cee70 0x7f450406cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:50.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.220+0000 7f450a67c640 1 --2- 172.21.15.154:0/1699598366 >> v2:172.21.15.190:3300/0 conn(0x7f45040b1b20 0x7f45040ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:50.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.220+0000 7f450a67c640 1 --2- 172.21.15.154:0/1699598366 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45040acce0 0x7f450415ed60 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:50.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.220+0000 7f450a67c640 1 -- 172.21.15.154:0/1699598366 >> 172.21.15.154:0/1699598366 conn(0x7f450406a0f0 msgr2=0x7f450406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:50.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.220+0000 7f450a67c640 1 -- 172.21.15.154:0/1699598366 shutdown_connections 2023-09-23T03:16:50.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.220+0000 7f450a67c640 1 -- 172.21.15.154:0/1699598366 wait complete. 2023-09-23T03:16:50.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.221+0000 7f450a67c640 1 Processor -- start 2023-09-23T03:16:50.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.222+0000 7f450a67c640 1 -- start start 2023-09-23T03:16:50.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.223+0000 7f450a67c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45040acce0 0x7f45041f6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:50.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.223+0000 7f450a67c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45040b1b20 0x7f4504159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:50.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.223+0000 7f450a67c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45040cee70 0x7f450415add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:50.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.223+0000 7f450a67c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4504159b40 con 0x7f45040b1b20 2023-09-23T03:16:50.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.223+0000 7f450a67c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450415b820 con 0x7f45040cee70 2023-09-23T03:16:50.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.223+0000 7f450a67c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45041f5b60 con 0x7f45040acce0 2023-09-23T03:16:50.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.223+0000 7f44fbfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45040b1b20 0x7f4504159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:50.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.223+0000 7f44fbfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45040b1b20 0x7f4504159600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:41260/0 (socket says 172.21.15.154:41260) 2023-09-23T03:16:50.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.223+0000 7f44fbfff640 1 -- 172.21.15.154:0/1213263902 learned_addr learned my addr 172.21.15.154:0/1213263902 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:50.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.223+0000 7f4503fff640 1 --2- 172.21.15.154:0/1213263902 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45040acce0 0x7f45041f6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:50.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.223+0000 7f44fbfff640 1 -- 172.21.15.154:0/1213263902 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45040acce0 msgr2=0x7f45041f6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:50.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.223+0000 7f44fbfff640 1 --2- 172.21.15.154:0/1213263902 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45040acce0 0x7f45041f6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:50.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.223+0000 7f44fbfff640 1 -- 172.21.15.154:0/1213263902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45040cee70 msgr2=0x7f450415add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:50.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.223+0000 7f44fbfff640 1 --2- 172.21.15.154:0/1213263902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45040cee70 0x7f450415add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:50.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.223+0000 7f44fbfff640 1 -- 172.21.15.154:0/1213263902 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f4009460 con 0x7f45040b1b20 2023-09-23T03:16:50.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.223+0000 7f4503fff640 1 --2- 172.21.15.154:0/1213263902 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45040acce0 0x7f45041f6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:50.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.224+0000 7f44fbfff640 1 --2- 172.21.15.154:0/1213263902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45040b1b20 0x7f4504159600 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f44ec00c7f0 tx=0x7f44ec00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:50.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.224+0000 7f45017fa640 1 -- 172.21.15.154:0/1213263902 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ec007a30 con 0x7f45040b1b20 2023-09-23T03:16:50.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.224+0000 7f45017fa640 1 -- 172.21.15.154:0/1213263902 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44ec007b90 con 0x7f45040b1b20 2023-09-23T03:16:50.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.224+0000 7f450a67c640 1 -- 172.21.15.154:0/1213263902 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45041f5e40 con 0x7f45040b1b20 2023-09-23T03:16:50.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.224+0000 7f45017fa640 1 -- 172.21.15.154:0/1213263902 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ec00ce40 con 0x7f45040b1b20 2023-09-23T03:16:50.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.224+0000 7f450a67c640 1 -- 172.21.15.154:0/1213263902 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f450415bb00 con 0x7f45040b1b20 2023-09-23T03:16:50.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.225+0000 7f450a67c640 1 -- 172.21.15.154:0/1213263902 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45041f6360 con 0x7f45040b1b20 2023-09-23T03:16:50.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.226+0000 7f45017fa640 1 -- 172.21.15.154:0/1213263902 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f44ec0113f0 con 0x7f45040b1b20 2023-09-23T03:16:50.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.227+0000 7f45017fa640 1 --2- 172.21.15.154:0/1213263902 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f44e406af30 0x7f44e406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:50.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.227+0000 7f45017fa640 1 -- 172.21.15.154:0/1213263902 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f44ec089c70 con 0x7f45040b1b20 2023-09-23T03:16:50.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.227+0000 7f4503fff640 1 --2- 172.21.15.154:0/1213263902 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f44e406af30 0x7f44e406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:50.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.228+0000 7f4503fff640 1 --2- 172.21.15.154:0/1213263902 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f44e406af30 0x7f44e406d3f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f44f4034000 tx=0x7f44f4008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:50.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.232+0000 7f45017fa640 1 -- 172.21.15.154:0/1213263902 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f45041f6360 con 0x7f45040b1b20 2023-09-23T03:16:50.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.421+0000 7f450a67c640 1 -- 172.21.15.154:0/1213263902 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45041f6360 con 0x7f45040b1b20 2023-09-23T03:16:50.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.425+0000 7f45017fa640 1 -- 172.21.15.154:0/1213263902 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18744 (secure 0 0 0) 0x7f45041f6360 con 0x7f45040b1b20 2023-09-23T03:16:50.425 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:16:50.426 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":26,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:16:03.151371+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:15:18.575849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T03:15:19.707212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T03:15:23.337138+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:16:50.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.427+0000 7f450a67c640 1 -- 172.21.15.154:0/1213263902 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f44e406af30 msgr2=0x7f44e406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:50.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.427+0000 7f450a67c640 1 --2- 172.21.15.154:0/1213263902 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f44e406af30 0x7f44e406d3f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f44f4034000 tx=0x7f44f4008030 comp rx=0 tx=0).stop 2023-09-23T03:16:50.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.427+0000 7f450a67c640 1 -- 172.21.15.154:0/1213263902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45040b1b20 msgr2=0x7f4504159600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:50.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.427+0000 7f450a67c640 1 --2- 172.21.15.154:0/1213263902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45040b1b20 0x7f4504159600 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f44ec00c7f0 tx=0x7f44ec00cbf0 comp rx=0 tx=0).stop 2023-09-23T03:16:50.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.427+0000 7f450a67c640 1 -- 172.21.15.154:0/1213263902 shutdown_connections 2023-09-23T03:16:50.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.427+0000 7f450a67c640 1 --2- 172.21.15.154:0/1213263902 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f44e406af30 0x7f44e406d3f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:50.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.427+0000 7f450a67c640 1 --2- 172.21.15.154:0/1213263902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45040cee70 0x7f450415add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:50.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.427+0000 7f450a67c640 1 --2- 172.21.15.154:0/1213263902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45040b1b20 0x7f4504159600 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:50.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.427+0000 7f450a67c640 1 --2- 172.21.15.154:0/1213263902 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45040acce0 0x7f45041f6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:50.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.427+0000 7f450a67c640 1 -- 172.21.15.154:0/1213263902 >> 172.21.15.154:0/1213263902 conn(0x7f450406a0f0 msgr2=0x7f45040b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:50.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.428+0000 7f450a67c640 1 -- 172.21.15.154:0/1213263902 shutdown_connections 2023-09-23T03:16:50.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.428+0000 7f450a67c640 1 -- 172.21.15.154:0/1213263902 wait complete. 2023-09-23T03:16:50.445 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:15:18.575849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:15:19.707212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-23T03:15:23.337138+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-23T03:16:50.445 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:16:50.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.662+0000 7f71a1061640 1 Processor -- start 2023-09-23T03:16:50.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.663+0000 7f71a1061640 1 -- start start 2023-09-23T03:16:50.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.664+0000 7f71a1061640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f719c15abe0 0x7f719c0cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:50.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.664+0000 7f71a1061640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f719c0ce510 con 0x7f719c0cc9b0 2023-09-23T03:16:50.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.664+0000 7f71a1061640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f719c0cb710 con 0x7f719c0b28c0 2023-09-23T03:16:50.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.664+0000 7f71a1061640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f719c0cd390 con 0x7f719c15abe0 2023-09-23T03:16:50.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.664+0000 7f719a59c640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f719c0cc9b0 0x7f719c0b21b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:36348/0 (socket says 172.21.15.154:36348) 2023-09-23T03:16:50.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.664+0000 7f719ad9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f719c15abe0 0x7f719c0cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:50.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.664+0000 7f719a59c640 1 -- 172.21.15.154:0/2154399373 learned_addr learned my addr 172.21.15.154:0/2154399373 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:50.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.665+0000 7f719ad9d640 1 -- 172.21.15.154:0/2154399373 >> v1:172.21.15.190:6789/0 conn(0x7f719c0b28c0 legacy=0x7f719c0ce9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:50.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.665+0000 7f719ad9d640 1 -- 172.21.15.154:0/2154399373 >> v1:172.21.15.154:6789/0 conn(0x7f719c0cc9b0 legacy=0x7f719c0b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:50.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.665+0000 7f719ad9d640 1 -- 172.21.15.154:0/2154399373 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f719c0cd670 con 0x7f719c15abe0 2023-09-23T03:16:50.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.665+0000 7f7199d9b640 1 -- 172.21.15.154:0/2154399373 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3220818838 0 0) 0x7f719c0ce510 con 0x7f719c0cc9b0 2023-09-23T03:16:50.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.666+0000 7f719ad9d640 1 --2- 172.21.15.154:0/2154399373 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f719c15abe0 0x7f719c0cb1d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f718c009de0 tx=0x7f718c02be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efaf63d188cad70d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:50.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.666+0000 7f7199d9b640 1 -- 172.21.15.154:0/2154399373 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c02cc20 con 0x7f719c15abe0 2023-09-23T03:16:50.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.666+0000 7f7199d9b640 1 -- 172.21.15.154:0/2154399373 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f718c02cd80 con 0x7f719c15abe0 2023-09-23T03:16:50.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.666+0000 7f7199d9b640 1 -- 172.21.15.154:0/2154399373 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c034660 con 0x7f719c15abe0 2023-09-23T03:16:50.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.667+0000 7f71a1061640 1 -- 172.21.15.154:0/2154399373 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f719c15abe0 msgr2=0x7f719c0cb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:50.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.667+0000 7f71a1061640 1 --2- 172.21.15.154:0/2154399373 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f719c15abe0 0x7f719c0cb1d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f718c009de0 tx=0x7f718c02be70 comp rx=0 tx=0).stop 2023-09-23T03:16:50.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.667+0000 7f71a1061640 1 -- 172.21.15.154:0/2154399373 shutdown_connections 2023-09-23T03:16:50.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.667+0000 7f71a1061640 1 --2- 172.21.15.154:0/2154399373 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f719c15abe0 0x7f719c0cb1d0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:50.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.667+0000 7f71a1061640 1 -- 172.21.15.154:0/2154399373 >> 172.21.15.154:0/2154399373 conn(0x7f719c06fc40 msgr2=0x7f719c070020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:50.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.667+0000 7f71a1061640 1 -- 172.21.15.154:0/2154399373 shutdown_connections 2023-09-23T03:16:50.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.667+0000 7f71a1061640 1 -- 172.21.15.154:0/2154399373 wait complete. 2023-09-23T03:16:50.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.669+0000 7f71a1061640 1 Processor -- start 2023-09-23T03:16:50.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.669+0000 7f71a1061640 1 -- start start 2023-09-23T03:16:50.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.670+0000 7f71a1061640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f719c0b28c0 0x7f719c0c6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:50.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.670+0000 7f71a1061640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f719c0cc9b0 0x7f719c0c7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:50.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.670+0000 7f71a1061640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f719c15abe0 0x7f719c1f2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:50.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.670+0000 7f71a1061640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c0c7270 con 0x7f719c15abe0 2023-09-23T03:16:50.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.670+0000 7f71a1061640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c1473e0 con 0x7f719c0b28c0 2023-09-23T03:16:50.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.670+0000 7f71a1061640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f719c0c9470 con 0x7f719c0cc9b0 2023-09-23T03:16:50.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.670+0000 7f719b59e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f719c15abe0 0x7f719c1f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:50.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.670+0000 7f719b59e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f719c15abe0 0x7f719c1f2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:41264/0 (socket says 172.21.15.154:41264) 2023-09-23T03:16:50.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.670+0000 7f719b59e640 1 -- 172.21.15.154:0/365469593 learned_addr learned my addr 172.21.15.154:0/365469593 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:50.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.670+0000 7f719ad9d640 1 --2- 172.21.15.154:0/365469593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f719c0b28c0 0x7f719c0c6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:50.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.670+0000 7f719a59c640 1 --2- 172.21.15.154:0/365469593 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f719c0cc9b0 0x7f719c0c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:50.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.670+0000 7f719b59e640 1 -- 172.21.15.154:0/365469593 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f719c0cc9b0 msgr2=0x7f719c0c7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:50.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.670+0000 7f719b59e640 1 --2- 172.21.15.154:0/365469593 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f719c0cc9b0 0x7f719c0c7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:50.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.670+0000 7f719b59e640 1 -- 172.21.15.154:0/365469593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f719c0b28c0 msgr2=0x7f719c0c6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:50.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.670+0000 7f719b59e640 1 --2- 172.21.15.154:0/365469593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f719c0b28c0 0x7f719c0c6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:50.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.671+0000 7f719b59e640 1 -- 172.21.15.154:0/365469593 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f718c009a70 con 0x7f719c15abe0 2023-09-23T03:16:50.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.671+0000 7f719a59c640 1 --2- 172.21.15.154:0/365469593 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f719c0cc9b0 0x7f719c0c7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:50.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.671+0000 7f719b59e640 1 --2- 172.21.15.154:0/365469593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f719c15abe0 0x7f719c1f2680 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f719400c410 tx=0x7f719400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:50.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.671+0000 7f71937fe640 1 -- 172.21.15.154:0/365469593 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7194011790 con 0x7f719c15abe0 2023-09-23T03:16:50.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.671+0000 7f71937fe640 1 -- 172.21.15.154:0/365469593 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71940118f0 con 0x7f719c15abe0 2023-09-23T03:16:50.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.671+0000 7f71937fe640 1 -- 172.21.15.154:0/365469593 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7194011ce0 con 0x7f719c15abe0 2023-09-23T03:16:50.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.671+0000 7f71a1061640 1 -- 172.21.15.154:0/365469593 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f719c0c96f0 con 0x7f719c15abe0 2023-09-23T03:16:50.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.671+0000 7f71a1061640 1 -- 172.21.15.154:0/365469593 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f719c0c9f90 con 0x7f719c15abe0 2023-09-23T03:16:50.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.672+0000 7f71a1061640 1 -- 172.21.15.154:0/365469593 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f719c04e920 con 0x7f719c15abe0 2023-09-23T03:16:50.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.673+0000 7f71937fe640 1 -- 172.21.15.154:0/365469593 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f719401c460 con 0x7f719c15abe0 2023-09-23T03:16:50.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.674+0000 7f71937fe640 1 --2- 172.21.15.154:0/365469593 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f717c06af30 0x7f717c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:50.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.675+0000 7f71937fe640 1 -- 172.21.15.154:0/365469593 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f7194089850 con 0x7f719c15abe0 2023-09-23T03:16:50.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.675+0000 7f719ad9d640 1 --2- 172.21.15.154:0/365469593 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f717c06af30 0x7f717c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:50.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.675+0000 7f719ad9d640 1 --2- 172.21.15.154:0/365469593 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f717c06af30 0x7f717c06d3f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f718c036000 tx=0x7f718c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:50.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.678+0000 7f71937fe640 1 -- 172.21.15.154:0/365469593 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f7194054440 con 0x7f719c15abe0 2023-09-23T03:16:50.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.865+0000 7f71a1061640 1 -- 172.21.15.154:0/365469593 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f719c06af90 con 0x7f719c15abe0 2023-09-23T03:16:50.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.866+0000 7f71937fe640 1 -- 172.21.15.154:0/365469593 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7194057a60 con 0x7f719c15abe0 2023-09-23T03:16:50.867 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:16:50.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.869+0000 7f71a1061640 1 -- 172.21.15.154:0/365469593 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f717c06af30 msgr2=0x7f717c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:50.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.869+0000 7f71a1061640 1 --2- 172.21.15.154:0/365469593 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f717c06af30 0x7f717c06d3f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f718c036000 tx=0x7f718c030000 comp rx=0 tx=0).stop 2023-09-23T03:16:50.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.869+0000 7f71a1061640 1 -- 172.21.15.154:0/365469593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f719c15abe0 msgr2=0x7f719c1f2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:50.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.869+0000 7f71a1061640 1 --2- 172.21.15.154:0/365469593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f719c15abe0 0x7f719c1f2680 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f719400c410 tx=0x7f719400c8d0 comp rx=0 tx=0).stop 2023-09-23T03:16:50.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.869+0000 7f71a1061640 1 -- 172.21.15.154:0/365469593 shutdown_connections 2023-09-23T03:16:50.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.869+0000 7f71a1061640 1 --2- 172.21.15.154:0/365469593 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f717c06af30 0x7f717c06d3f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:50.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.869+0000 7f71a1061640 1 --2- 172.21.15.154:0/365469593 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f719c15abe0 0x7f719c1f2680 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:50.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.869+0000 7f71a1061640 1 --2- 172.21.15.154:0/365469593 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f719c0cc9b0 0x7f719c0c7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:50.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.869+0000 7f71a1061640 1 --2- 172.21.15.154:0/365469593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f719c0b28c0 0x7f719c0c6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:50.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.869+0000 7f71a1061640 1 -- 172.21.15.154:0/365469593 >> 172.21.15.154:0/365469593 conn(0x7f719c06fc40 msgr2=0x7f719c0d0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:50.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.869+0000 7f71a1061640 1 -- 172.21.15.154:0/365469593 shutdown_connections 2023-09-23T03:16:50.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:50.869+0000 7f71a1061640 1 -- 172.21.15.154:0/365469593 wait complete. 2023-09-23T03:16:50.887 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:16:51.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.104+0000 7ff54524b640 1 Processor -- start 2023-09-23T03:16:51.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.105+0000 7ff54524b640 1 -- start start 2023-09-23T03:16:51.106 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.106+0000 7ff54524b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff54015c9c0 0x7ff5400d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:51.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.106+0000 7ff54524b640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff5400d2d30 0x7ff5400cd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:51.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.106+0000 7ff54524b640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5400cd450 con 0x7ff54015cda0 2023-09-23T03:16:51.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.106+0000 7ff54524b640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5400d0df0 con 0x7ff5400d2d30 2023-09-23T03:16:51.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.106+0000 7ff54524b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5400d10d0 con 0x7ff54015c9c0 2023-09-23T03:16:51.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.107+0000 7ff53e59c640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff54015cda0 0x7ff5400d2620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:36352/0 (socket says 172.21.15.154:36352) 2023-09-23T03:16:51.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.107+0000 7ff53e59c640 1 -- 172.21.15.154:0/2574229898 learned_addr learned my addr 172.21.15.154:0/2574229898 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:51.109 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.107+0000 7ff53ed9d640 1 --2- 172.21.15.154:0/2574229898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff54015c9c0 0x7ff5400d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:51.109 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.107+0000 7ff53dd9b640 1 -- 172.21.15.154:0/2574229898 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3150784617 0 0) 0x7ff5400cd450 con 0x7ff54015cda0 2023-09-23T03:16:51.109 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.107+0000 7ff53dd9b640 1 -- 172.21.15.154:0/2574229898 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff52c003520 con 0x7ff54015cda0 2023-09-23T03:16:51.109 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.107+0000 7ff53f59e640 1 --2- 172.21.15.154:0/2574229898 >> v2:172.21.15.190:3300/0 conn(0x7ff5400d2d30 0x7ff5400cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:51.109 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.107+0000 7ff53dd9b640 1 -- 172.21.15.154:0/2574229898 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 675588380 0 0) 0x7ff52c003520 con 0x7ff54015cda0 2023-09-23T03:16:51.109 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.108+0000 7ff53dd9b640 1 -- 172.21.15.154:0/2574229898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff54015c9c0 msgr2=0x7ff5400d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:51.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.108+0000 7ff53dd9b640 1 --2- 172.21.15.154:0/2574229898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff54015c9c0 0x7ff5400d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:51.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.108+0000 7ff53dd9b640 1 -- 172.21.15.154:0/2574229898 >> v2:172.21.15.190:3300/0 conn(0x7ff5400d2d30 msgr2=0x7ff5400cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:51.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.108+0000 7ff53dd9b640 1 --2- 172.21.15.154:0/2574229898 >> v2:172.21.15.190:3300/0 conn(0x7ff5400d2d30 0x7ff5400cd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:51.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.108+0000 7ff53dd9b640 1 -- 172.21.15.154:0/2574229898 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5400d13b0 con 0x7ff54015cda0 2023-09-23T03:16:51.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.108+0000 7ff53dd9b640 1 -- 172.21.15.154:0/2574229898 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7ff528001670 con 0x7ff54015cda0 2023-09-23T03:16:51.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.108+0000 7ff53dd9b640 1 -- 172.21.15.154:0/2574229898 >> v1:172.21.15.154:6789/0 conn(0x7ff54015cda0 legacy=0x7ff5400d2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:51.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.108+0000 7ff53dd9b640 1 --2- 172.21.15.154:0/2574229898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff52c003970 0x7ff52c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:51.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.108+0000 7ff53dd9b640 1 --2- 172.21.15.154:0/2574229898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff52c0047c0 0x7ff52c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:51.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.108+0000 7ff53dd9b640 1 --2- 172.21.15.154:0/2574229898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff54015c9c0 0x7ff52c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:51.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.108+0000 7ff53dd9b640 1 -- 172.21.15.154:0/2574229898 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5400d0df0 con 0x7ff52c0047c0 2023-09-23T03:16:51.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.108+0000 7ff53dd9b640 1 -- 172.21.15.154:0/2574229898 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5400d10d0 con 0x7ff52c003970 2023-09-23T03:16:51.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.108+0000 7ff53dd9b640 1 -- 172.21.15.154:0/2574229898 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c00b980 con 0x7ff54015c9c0 2023-09-23T03:16:51.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.108+0000 7ff53e59c640 1 --2- 172.21.15.154:0/2574229898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff52c0047c0 0x7ff52c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:51.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.108+0000 7ff53f59e640 1 --2- 172.21.15.154:0/2574229898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff54015c9c0 0x7ff52c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:51.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.108+0000 7ff53ed9d640 1 --2- 172.21.15.154:0/2574229898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff52c003970 0x7ff52c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:51.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.108+0000 7ff53e59c640 1 -- 172.21.15.154:0/2574229898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff54015c9c0 msgr2=0x7ff52c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:51.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.108+0000 7ff53e59c640 1 --2- 172.21.15.154:0/2574229898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff54015c9c0 0x7ff52c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:51.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.109+0000 7ff53e59c640 1 -- 172.21.15.154:0/2574229898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff52c003970 msgr2=0x7ff52c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:51.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.109+0000 7ff53e59c640 1 --2- 172.21.15.154:0/2574229898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff52c003970 0x7ff52c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:51.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.109+0000 7ff53e59c640 1 -- 172.21.15.154:0/2574229898 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff52c003520 con 0x7ff52c0047c0 2023-09-23T03:16:51.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.109+0000 7ff53ed9d640 1 --2- 172.21.15.154:0/2574229898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff52c003970 0x7ff52c003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:51.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.110+0000 7ff53e59c640 1 --2- 172.21.15.154:0/2574229898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff52c0047c0 0x7ff52c004c40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff52800c240 tx=0x7ff52c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1753241cc983a0f6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:51.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.110+0000 7ff53dd9b640 1 -- 172.21.15.154:0/2574229898 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52802cb00 con 0x7ff52c0047c0 2023-09-23T03:16:51.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.110+0000 7ff53dd9b640 1 -- 172.21.15.154:0/2574229898 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52802cc60 con 0x7ff52c0047c0 2023-09-23T03:16:51.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.110+0000 7ff53dd9b640 1 -- 172.21.15.154:0/2574229898 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff528005af0 con 0x7ff52c0047c0 2023-09-23T03:16:51.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.110+0000 7ff54524b640 1 -- 172.21.15.154:0/2574229898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff52c0047c0 msgr2=0x7ff52c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:51.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.110+0000 7ff54524b640 1 --2- 172.21.15.154:0/2574229898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff52c0047c0 0x7ff52c004c40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff52800c240 tx=0x7ff52c0057f0 comp rx=0 tx=0).stop 2023-09-23T03:16:51.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.110+0000 7ff53f59e640 1 -- 172.21.15.154:0/2574229898 reap_dead start 2023-09-23T03:16:51.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.111+0000 7ff54524b640 1 -- 172.21.15.154:0/2574229898 shutdown_connections 2023-09-23T03:16:51.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.111+0000 7ff54524b640 1 -- 172.21.15.154:0/2574229898 >> 172.21.15.154:0/2574229898 conn(0x7ff54009bc20 msgr2=0x7ff540089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:51.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.111+0000 7ff54524b640 1 -- 172.21.15.154:0/2574229898 shutdown_connections 2023-09-23T03:16:51.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.111+0000 7ff54524b640 1 -- 172.21.15.154:0/2574229898 wait complete. 2023-09-23T03:16:51.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.112+0000 7ff54524b640 1 Processor -- start 2023-09-23T03:16:51.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.113+0000 7ff54524b640 1 -- start start 2023-09-23T03:16:51.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.113+0000 7ff54524b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff52c0047c0 0x7ff5400c6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:51.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.113+0000 7ff54524b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5400cb400 0x7ff5400c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:51.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.114+0000 7ff54524b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5400c8540 0x7ff5400c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:51.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.114+0000 7ff54524b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54015d510 con 0x7ff5400c8540 2023-09-23T03:16:51.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.114+0000 7ff54524b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5400c8f30 con 0x7ff52c0047c0 2023-09-23T03:16:51.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.114+0000 7ff54524b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5400cb900 con 0x7ff5400cb400 2023-09-23T03:16:51.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.114+0000 7ff53f59e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5400c8540 0x7ff5400c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:51.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.114+0000 7ff53f59e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5400c8540 0x7ff5400c89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:41292/0 (socket says 172.21.15.154:41292) 2023-09-23T03:16:51.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.114+0000 7ff53f59e640 1 -- 172.21.15.154:0/3087082784 learned_addr learned my addr 172.21.15.154:0/3087082784 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:51.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.114+0000 7ff53e59c640 1 --2- 172.21.15.154:0/3087082784 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5400cb400 0x7ff5400c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:51.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.114+0000 7ff53ed9d640 1 --2- 172.21.15.154:0/3087082784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff52c0047c0 0x7ff5400c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:51.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.114+0000 7ff53f59e640 1 -- 172.21.15.154:0/3087082784 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5400cb400 msgr2=0x7ff5400c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:51.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.114+0000 7ff53f59e640 1 --2- 172.21.15.154:0/3087082784 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5400cb400 0x7ff5400c8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:51.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.114+0000 7ff53f59e640 1 -- 172.21.15.154:0/3087082784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff52c0047c0 msgr2=0x7ff5400c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:51.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.114+0000 7ff53f59e640 1 --2- 172.21.15.154:0/3087082784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff52c0047c0 0x7ff5400c6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:51.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.114+0000 7ff53f59e640 1 -- 172.21.15.154:0/3087082784 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5400cbbe0 con 0x7ff5400c8540 2023-09-23T03:16:51.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.115+0000 7ff53f59e640 1 --2- 172.21.15.154:0/3087082784 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5400c8540 0x7ff5400c89f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff538006fd0 tx=0x7ff53800f8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:51.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.115+0000 7ff5377fe640 1 -- 172.21.15.154:0/3087082784 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff538004440 con 0x7ff5400c8540 2023-09-23T03:16:51.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.115+0000 7ff5377fe640 1 -- 172.21.15.154:0/3087082784 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff538003610 con 0x7ff5400c8540 2023-09-23T03:16:51.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.115+0000 7ff54524b640 1 -- 172.21.15.154:0/3087082784 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5400cbec0 con 0x7ff5400c8540 2023-09-23T03:16:51.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.115+0000 7ff5377fe640 1 -- 172.21.15.154:0/3087082784 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5380039c0 con 0x7ff5400c8540 2023-09-23T03:16:51.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.115+0000 7ff54524b640 1 -- 172.21.15.154:0/3087082784 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5400c69b0 con 0x7ff5400c8540 2023-09-23T03:16:51.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.116+0000 7ff54524b640 1 -- 172.21.15.154:0/3087082784 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff54015d680 con 0x7ff5400c8540 2023-09-23T03:16:51.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.117+0000 7ff5377fe640 1 -- 172.21.15.154:0/3087082784 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7ff538003b20 con 0x7ff5400c8540 2023-09-23T03:16:51.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.118+0000 7ff5377fe640 1 --2- 172.21.15.154:0/3087082784 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff52006af60 0x7ff52006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:51.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.118+0000 7ff5377fe640 1 -- 172.21.15.154:0/3087082784 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7ff538089ad0 con 0x7ff5400c8540 2023-09-23T03:16:51.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.119+0000 7ff53ed9d640 1 --2- 172.21.15.154:0/3087082784 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff52006af60 0x7ff52006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:51.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.119+0000 7ff53ed9d640 1 --2- 172.21.15.154:0/3087082784 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff52006af60 0x7ff52006d420 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff5300074a0 tx=0x7ff53000d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:51.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.123+0000 7ff5377fe640 1 -- 172.21.15.154:0/3087082784 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7ff54015d680 con 0x7ff5400c8540 2023-09-23T03:16:51.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.316+0000 7ff54524b640 1 -- 172.21.15.154:0/3087082784 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff54015d650 con 0x7ff5400c8540 2023-09-23T03:16:51.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.317+0000 7ff5377fe640 1 -- 172.21.15.154:0/3087082784 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff5380581e0 con 0x7ff5400c8540 2023-09-23T03:16:51.317 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:16:51.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.320+0000 7ff54524b640 1 -- 172.21.15.154:0/3087082784 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff52006af60 msgr2=0x7ff52006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:51.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.320+0000 7ff54524b640 1 --2- 172.21.15.154:0/3087082784 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff52006af60 0x7ff52006d420 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff5300074a0 tx=0x7ff53000d040 comp rx=0 tx=0).stop 2023-09-23T03:16:51.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.320+0000 7ff54524b640 1 -- 172.21.15.154:0/3087082784 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5400c8540 msgr2=0x7ff5400c89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:51.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.320+0000 7ff54524b640 1 --2- 172.21.15.154:0/3087082784 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5400c8540 0x7ff5400c89f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff538006fd0 tx=0x7ff53800f8a0 comp rx=0 tx=0).stop 2023-09-23T03:16:51.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.320+0000 7ff54524b640 1 -- 172.21.15.154:0/3087082784 shutdown_connections 2023-09-23T03:16:51.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.320+0000 7ff54524b640 1 --2- 172.21.15.154:0/3087082784 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff52006af60 0x7ff52006d420 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:51.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.320+0000 7ff54524b640 1 --2- 172.21.15.154:0/3087082784 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5400c8540 0x7ff5400c89f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:51.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.320+0000 7ff54524b640 1 --2- 172.21.15.154:0/3087082784 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5400cb400 0x7ff5400c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:51.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.320+0000 7ff54524b640 1 --2- 172.21.15.154:0/3087082784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff52c0047c0 0x7ff5400c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:51.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.320+0000 7ff54524b640 1 -- 172.21.15.154:0/3087082784 >> 172.21.15.154:0/3087082784 conn(0x7ff54009bc20 msgr2=0x7ff5400cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:51.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.320+0000 7ff54524b640 1 -- 172.21.15.154:0/3087082784 shutdown_connections 2023-09-23T03:16:51.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.320+0000 7ff54524b640 1 -- 172.21.15.154:0/3087082784 wait complete. 2023-09-23T03:16:51.340 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:16:51.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.555+0000 7f4d22b8c640 1 Processor -- start 2023-09-23T03:16:51.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.555+0000 7f4d22b8c640 1 -- start start 2023-09-23T03:16:51.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.557+0000 7f4d22b8c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d1c159570 0x7f4d1c071100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:51.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.557+0000 7f4d22b8c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4d1c071640 0x7f4d1c159950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:51.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.557+0000 7f4d22b8c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d1c071e70 con 0x7f4d1c159e90 2023-09-23T03:16:51.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.557+0000 7f4d22b8c640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d1c072150 con 0x7f4d1c071640 2023-09-23T03:16:51.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.557+0000 7f4d22b8c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c0c8b50 con 0x7f4d1c159570 2023-09-23T03:16:51.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.557+0000 7f4d2238b640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f4d1c159e90 0x7f4d1c15a320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:36364/0 (socket says 172.21.15.154:36364) 2023-09-23T03:16:51.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.557+0000 7f4d2238b640 1 -- 172.21.15.154:0/274235061 learned_addr learned my addr 172.21.15.154:0/274235061 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:51.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.557+0000 7f4d21389640 1 --2- 172.21.15.154:0/274235061 >> v2:172.21.15.190:3300/0 conn(0x7f4d1c071640 0x7f4d1c159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:51.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.557+0000 7f4d21b8a640 1 --2- 172.21.15.154:0/274235061 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d1c159570 0x7f4d1c071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:51.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.557+0000 7f4d20b88640 1 -- 172.21.15.154:0/274235061 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1706529189 0 0) 0x7f4d1c071e70 con 0x7f4d1c159e90 2023-09-23T03:16:51.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.558+0000 7f4d20b88640 1 -- 172.21.15.154:0/274235061 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d08003520 con 0x7f4d1c159e90 2023-09-23T03:16:51.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.558+0000 7f4d20b88640 1 -- 172.21.15.154:0/274235061 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3729067696 0 0) 0x7f4d08003520 con 0x7f4d1c159e90 2023-09-23T03:16:51.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.558+0000 7f4d20b88640 1 -- 172.21.15.154:0/274235061 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d1c159570 msgr2=0x7f4d1c071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:51.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.558+0000 7f4d20b88640 1 --2- 172.21.15.154:0/274235061 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d1c159570 0x7f4d1c071100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:51.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.558+0000 7f4d20b88640 1 -- 172.21.15.154:0/274235061 >> v2:172.21.15.190:3300/0 conn(0x7f4d1c071640 msgr2=0x7f4d1c159950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:51.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.558+0000 7f4d20b88640 1 --2- 172.21.15.154:0/274235061 >> v2:172.21.15.190:3300/0 conn(0x7f4d1c071640 0x7f4d1c159950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:51.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.558+0000 7f4d20b88640 1 -- 172.21.15.154:0/274235061 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1c0c8e30 con 0x7f4d1c159e90 2023-09-23T03:16:51.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.558+0000 7f4d20b88640 1 -- 172.21.15.154:0/274235061 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f4d140019f0 con 0x7f4d1c159e90 2023-09-23T03:16:51.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.558+0000 7f4d20b88640 1 -- 172.21.15.154:0/274235061 >> v1:172.21.15.154:6789/0 conn(0x7f4d1c159e90 legacy=0x7f4d1c15a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:51.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.558+0000 7f4d20b88640 1 --2- 172.21.15.154:0/274235061 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d1c159570 0x7f4d08003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:51.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.558+0000 7f4d20b88640 1 --2- 172.21.15.154:0/274235061 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d08003ef0 0x7f4d080047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:51.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.558+0000 7f4d20b88640 1 --2- 172.21.15.154:0/274235061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d08004d00 0x7f4d08005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:51.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.558+0000 7f4d20b88640 1 -- 172.21.15.154:0/274235061 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c072150 con 0x7f4d08003ef0 2023-09-23T03:16:51.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.558+0000 7f4d20b88640 1 -- 172.21.15.154:0/274235061 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c0c8b50 con 0x7f4d08004d00 2023-09-23T03:16:51.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.558+0000 7f4d20b88640 1 -- 172.21.15.154:0/274235061 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0800b9d0 con 0x7f4d1c159570 2023-09-23T03:16:51.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.558+0000 7f4d21389640 1 --2- 172.21.15.154:0/274235061 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d08003ef0 0x7f4d080047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:51.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.559+0000 7f4d21b8a640 1 --2- 172.21.15.154:0/274235061 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d1c159570 0x7f4d08003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:51.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.559+0000 7f4d21389640 1 -- 172.21.15.154:0/274235061 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d1c159570 msgr2=0x7f4d08003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:51.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.559+0000 7f4d21389640 1 --2- 172.21.15.154:0/274235061 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d1c159570 0x7f4d08003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:51.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.559+0000 7f4d21389640 1 -- 172.21.15.154:0/274235061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d08004d00 msgr2=0x7f4d08005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:51.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.559+0000 7f4d21389640 1 --2- 172.21.15.154:0/274235061 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d08004d00 0x7f4d08005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:51.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.559+0000 7f4d21389640 1 -- 172.21.15.154:0/274235061 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d08003520 con 0x7f4d08003ef0 2023-09-23T03:16:51.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.559+0000 7f4d21b8a640 1 --2- 172.21.15.154:0/274235061 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d1c159570 0x7f4d08003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:51.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.560+0000 7f4d21389640 1 --2- 172.21.15.154:0/274235061 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d08003ef0 0x7f4d080047c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c009af0 tx=0x7f4d080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5441a970f4b12ab0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:51.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.560+0000 7f4d20b88640 1 -- 172.21.15.154:0/274235061 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c02c710 con 0x7f4d08003ef0 2023-09-23T03:16:51.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.560+0000 7f4d20b88640 1 -- 172.21.15.154:0/274235061 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0c02c870 con 0x7f4d08003ef0 2023-09-23T03:16:51.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.560+0000 7f4d20b88640 1 -- 172.21.15.154:0/274235061 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c02cc00 con 0x7f4d08003ef0 2023-09-23T03:16:51.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.561+0000 7f4d22b8c640 1 -- 172.21.15.154:0/274235061 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d08003ef0 msgr2=0x7f4d080047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:51.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.561+0000 7f4d22b8c640 1 --2- 172.21.15.154:0/274235061 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d08003ef0 0x7f4d080047c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c009af0 tx=0x7f4d080057f0 comp rx=0 tx=0).stop 2023-09-23T03:16:51.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.561+0000 7f4d2238b640 1 -- 172.21.15.154:0/274235061 reap_dead start 2023-09-23T03:16:51.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.561+0000 7f4d22b8c640 1 -- 172.21.15.154:0/274235061 shutdown_connections 2023-09-23T03:16:51.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.561+0000 7f4d22b8c640 1 -- 172.21.15.154:0/274235061 >> 172.21.15.154:0/274235061 conn(0x7f4d1c0aff20 msgr2=0x7f4d1c0b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:51.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.561+0000 7f4d22b8c640 1 -- 172.21.15.154:0/274235061 shutdown_connections 2023-09-23T03:16:51.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.561+0000 7f4d22b8c640 1 -- 172.21.15.154:0/274235061 wait complete. 2023-09-23T03:16:51.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.563+0000 7f4d22b8c640 1 Processor -- start 2023-09-23T03:16:51.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.563+0000 7f4d22b8c640 1 -- start start 2023-09-23T03:16:51.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.564+0000 7f4d22b8c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d08003ef0 0x7f4d1c0ac360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:51.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.564+0000 7f4d22b8c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1c14eda0 0x7f4d1c0ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:51.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.564+0000 7f4d22b8c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d1c1e5530 0x7f4d1c1e59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:51.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.564+0000 7f4d22b8c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c1e6890 con 0x7f4d1c1e5530 2023-09-23T03:16:51.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.564+0000 7f4d22b8c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c1e62d0 con 0x7f4d1c14eda0 2023-09-23T03:16:51.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.564+0000 7f4d22b8c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c1e65b0 con 0x7f4d08003ef0 2023-09-23T03:16:51.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.564+0000 7f4d2238b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d1c1e5530 0x7f4d1c1e59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:51.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.564+0000 7f4d2238b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d1c1e5530 0x7f4d1c1e59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:41320/0 (socket says 172.21.15.154:41320) 2023-09-23T03:16:51.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.564+0000 7f4d2238b640 1 -- 172.21.15.154:0/1768735590 learned_addr learned my addr 172.21.15.154:0/1768735590 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:51.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.565+0000 7f4d21b8a640 1 --2- 172.21.15.154:0/1768735590 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d08003ef0 0x7f4d1c0ac360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:51.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.565+0000 7f4d21389640 1 --2- 172.21.15.154:0/1768735590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1c14eda0 0x7f4d1c0ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:51.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.565+0000 7f4d2238b640 1 -- 172.21.15.154:0/1768735590 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d08003ef0 msgr2=0x7f4d1c0ac360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:51.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.565+0000 7f4d2238b640 1 --2- 172.21.15.154:0/1768735590 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d08003ef0 0x7f4d1c0ac360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:51.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.565+0000 7f4d2238b640 1 -- 172.21.15.154:0/1768735590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1c14eda0 msgr2=0x7f4d1c0ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:51.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.565+0000 7f4d2238b640 1 --2- 172.21.15.154:0/1768735590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1c14eda0 0x7f4d1c0ac8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:51.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.565+0000 7f4d2238b640 1 -- 172.21.15.154:0/1768735590 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1c14f2d0 con 0x7f4d1c1e5530 2023-09-23T03:16:51.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.565+0000 7f4d2238b640 1 --2- 172.21.15.154:0/1768735590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d1c1e5530 0x7f4d1c1e59e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4d140013d0 tx=0x7f4d1400cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:51.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.565+0000 7f4d127fc640 1 -- 172.21.15.154:0/1768735590 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1400a610 con 0x7f4d1c1e5530 2023-09-23T03:16:51.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.565+0000 7f4d127fc640 1 -- 172.21.15.154:0/1768735590 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d1400a770 con 0x7f4d1c1e5530 2023-09-23T03:16:51.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.565+0000 7f4d127fc640 1 -- 172.21.15.154:0/1768735590 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1400ab60 con 0x7f4d1c1e5530 2023-09-23T03:16:51.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.565+0000 7f4d22b8c640 1 -- 172.21.15.154:0/1768735590 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d1c14f5b0 con 0x7f4d1c1e5530 2023-09-23T03:16:51.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.566+0000 7f4d22b8c640 1 -- 172.21.15.154:0/1768735590 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d1c1545b0 con 0x7f4d1c1e5530 2023-09-23T03:16:51.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.567+0000 7f4d22b8c640 1 -- 172.21.15.154:0/1768735590 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d1c15a9b0 con 0x7f4d1c1e5530 2023-09-23T03:16:51.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.568+0000 7f4d127fc640 1 -- 172.21.15.154:0/1768735590 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f4d1400acc0 con 0x7f4d1c1e5530 2023-09-23T03:16:51.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.568+0000 7f4d127fc640 1 --2- 172.21.15.154:0/1768735590 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4d0006af60 0x7f4d0006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:51.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.569+0000 7f4d127fc640 1 -- 172.21.15.154:0/1768735590 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f4d14089440 con 0x7f4d1c1e5530 2023-09-23T03:16:51.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.569+0000 7f4d21b8a640 1 --2- 172.21.15.154:0/1768735590 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4d0006af60 0x7f4d0006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:51.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.569+0000 7f4d21b8a640 1 --2- 172.21.15.154:0/1768735590 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4d0006af60 0x7f4d0006d420 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4d180099c0 tx=0x7f4d18008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:51.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.573+0000 7f4d127fc640 1 -- 172.21.15.154:0/1768735590 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f4d140540c0 con 0x7f4d1c1e5530 2023-09-23T03:16:51.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.766+0000 7f4d22b8c640 1 -- 172.21.15.154:0/1768735590 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4d1c04e920 con 0x7f4d1c1e5530 2023-09-23T03:16:51.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.767+0000 7f4d127fc640 1 -- 172.21.15.154:0/1768735590 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4d140576e0 con 0x7f4d1c1e5530 2023-09-23T03:16:51.767 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:16:51.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.769+0000 7f4d22b8c640 1 -- 172.21.15.154:0/1768735590 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4d0006af60 msgr2=0x7f4d0006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:51.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.769+0000 7f4d22b8c640 1 --2- 172.21.15.154:0/1768735590 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4d0006af60 0x7f4d0006d420 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4d180099c0 tx=0x7f4d18008040 comp rx=0 tx=0).stop 2023-09-23T03:16:51.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.769+0000 7f4d22b8c640 1 -- 172.21.15.154:0/1768735590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d1c1e5530 msgr2=0x7f4d1c1e59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:51.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.769+0000 7f4d22b8c640 1 --2- 172.21.15.154:0/1768735590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d1c1e5530 0x7f4d1c1e59e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4d140013d0 tx=0x7f4d1400cf00 comp rx=0 tx=0).stop 2023-09-23T03:16:51.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.770+0000 7f4d22b8c640 1 -- 172.21.15.154:0/1768735590 shutdown_connections 2023-09-23T03:16:51.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.770+0000 7f4d22b8c640 1 --2- 172.21.15.154:0/1768735590 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4d0006af60 0x7f4d0006d420 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:51.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.770+0000 7f4d22b8c640 1 --2- 172.21.15.154:0/1768735590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4d1c1e5530 0x7f4d1c1e59e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:51.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.770+0000 7f4d22b8c640 1 --2- 172.21.15.154:0/1768735590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4d1c14eda0 0x7f4d1c0ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:51.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.770+0000 7f4d22b8c640 1 --2- 172.21.15.154:0/1768735590 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4d08003ef0 0x7f4d1c0ac360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:51.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.770+0000 7f4d22b8c640 1 -- 172.21.15.154:0/1768735590 >> 172.21.15.154:0/1768735590 conn(0x7f4d1c0aff20 msgr2=0x7f4d1c15bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:51.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.770+0000 7f4d22b8c640 1 -- 172.21.15.154:0/1768735590 shutdown_connections 2023-09-23T03:16:51.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:51.770+0000 7f4d22b8c640 1 -- 172.21.15.154:0/1768735590 wait complete. 2023-09-23T03:16:51.789 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-23T03:16:52.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.006+0000 7fa590b90640 1 Processor -- start 2023-09-23T03:16:52.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.006+0000 7fa590b90640 1 -- start start 2023-09-23T03:16:52.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.008+0000 7fa590b90640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa58c0ad830 0x7fa58c070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:52.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.008+0000 7fa590b90640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa58c0cb5d0 0x7fa58c0aba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:52.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.008+0000 7fa590b90640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa58c0712f0 con 0x7fa58c0c9680 2023-09-23T03:16:52.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.008+0000 7fa590b90640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa58c0715d0 con 0x7fa58c0cb5d0 2023-09-23T03:16:52.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.008+0000 7fa590b90640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c0718b0 con 0x7fa58c0ad830 2023-09-23T03:16:52.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.009+0000 7fa58a59c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa58c0ad830 0x7fa58c070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:52.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.009+0000 7fa58ad9d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa58c0cb5d0 0x7fa58c0aba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:52.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.009+0000 7fa58ad9d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa58c0cb5d0 0x7fa58c0aba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.154:50704/0 (socket says 172.21.15.154:50704) 2023-09-23T03:16:52.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.009+0000 7fa58ad9d640 1 -- 172.21.15.154:0/98607918 learned_addr learned my addr 172.21.15.154:0/98607918 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:52.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.009+0000 7fa583fff640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fa58c0c9680 0x7fa58c0caec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.154:38308/0 (socket says 172.21.15.154:38308) 2023-09-23T03:16:52.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.009+0000 7fa58a59c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa58c0ad830 0x7fa58c070c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:45838/0 (socket says 172.21.15.154:45838) 2023-09-23T03:16:52.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.009+0000 7fa58ad9d640 1 -- 172.21.15.154:0/98607918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa58c0ad830 msgr2=0x7fa58c070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:52.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.009+0000 7fa58ad9d640 1 --2- 172.21.15.154:0/98607918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa58c0ad830 0x7fa58c070c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:52.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.009+0000 7fa58ad9d640 1 -- 172.21.15.154:0/98607918 >> v1:172.21.15.190:6789/0 conn(0x7fa58c0c9680 legacy=0x7fa58c0caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:52.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.009+0000 7fa58ad9d640 1 -- 172.21.15.154:0/98607918 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa58c071b90 con 0x7fa58c0cb5d0 2023-09-23T03:16:52.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.009+0000 7fa58a59c640 1 --2- 172.21.15.154:0/98607918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa58c0ad830 0x7fa58c070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:16:52.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.011+0000 7fa58ad9d640 1 --2- 172.21.15.154:0/98607918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa58c0cb5d0 0x7fa58c0aba70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa584009e50 tx=0x7fa58402bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23862ca96da40b96 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:52.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.011+0000 7fa589d9b640 1 -- 172.21.15.154:0/98607918 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58402cb00 con 0x7fa58c0cb5d0 2023-09-23T03:16:52.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.011+0000 7fa589d9b640 1 -- 172.21.15.154:0/98607918 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58402cc60 con 0x7fa58c0cb5d0 2023-09-23T03:16:52.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.011+0000 7fa589d9b640 1 -- 172.21.15.154:0/98607918 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5840343f0 con 0x7fa58c0cb5d0 2023-09-23T03:16:52.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.012+0000 7fa590b90640 1 -- 172.21.15.154:0/98607918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa58c0cb5d0 msgr2=0x7fa58c0aba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:52.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.012+0000 7fa590b90640 1 --2- 172.21.15.154:0/98607918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa58c0cb5d0 0x7fa58c0aba70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa584009e50 tx=0x7fa58402bf60 comp rx=0 tx=0).stop 2023-09-23T03:16:52.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.012+0000 7fa590b90640 1 -- 172.21.15.154:0/98607918 shutdown_connections 2023-09-23T03:16:52.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.012+0000 7fa590b90640 1 --2- 172.21.15.154:0/98607918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa58c0cb5d0 0x7fa58c0aba70 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:52.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.012+0000 7fa590b90640 1 --2- 172.21.15.154:0/98607918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa58c0ad830 0x7fa58c070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:52.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.012+0000 7fa590b90640 1 -- 172.21.15.154:0/98607918 >> 172.21.15.154:0/98607918 conn(0x7fa58c06c010 msgr2=0x7fa58c06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:52.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.012+0000 7fa590b90640 1 -- 172.21.15.154:0/98607918 shutdown_connections 2023-09-23T03:16:52.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.012+0000 7fa590b90640 1 -- 172.21.15.154:0/98607918 wait complete. 2023-09-23T03:16:52.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.014+0000 7fa590b90640 1 Processor -- start 2023-09-23T03:16:52.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.015+0000 7fa590b90640 1 -- start start 2023-09-23T03:16:52.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.016+0000 7fa590b90640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa58c0ad830 0x7fa58c0c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:52.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.016+0000 7fa590b90640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa58c0c9680 0x7fa58c0b2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:52.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.016+0000 7fa590b90640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa58c0cb5d0 0x7fa58c154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:52.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.016+0000 7fa590b90640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c0b3540 con 0x7fa58c0cb5d0 2023-09-23T03:16:52.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.016+0000 7fa590b90640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c0b2690 con 0x7fa58c0c9680 2023-09-23T03:16:52.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.016+0000 7fa590b90640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c0b2930 con 0x7fa58c0ad830 2023-09-23T03:16:52.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.016+0000 7fa58ad9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa58c0cb5d0 0x7fa58c154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:52.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.016+0000 7fa58ad9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa58c0cb5d0 0x7fa58c154a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:41324/0 (socket says 172.21.15.154:41324) 2023-09-23T03:16:52.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.016+0000 7fa58ad9d640 1 -- 172.21.15.154:0/4022977347 learned_addr learned my addr 172.21.15.154:0/4022977347 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:52.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.017+0000 7fa583fff640 1 --2- 172.21.15.154:0/4022977347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa58c0c9680 0x7fa58c0b2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:52.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.017+0000 7fa58a59c640 1 --2- 172.21.15.154:0/4022977347 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa58c0ad830 0x7fa58c0c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:52.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.017+0000 7fa58ad9d640 1 -- 172.21.15.154:0/4022977347 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa58c0ad830 msgr2=0x7fa58c0c65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:52.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.017+0000 7fa58ad9d640 1 --2- 172.21.15.154:0/4022977347 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa58c0ad830 0x7fa58c0c65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:52.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.017+0000 7fa58ad9d640 1 -- 172.21.15.154:0/4022977347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa58c0c9680 msgr2=0x7fa58c0b2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:52.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.017+0000 7fa58ad9d640 1 --2- 172.21.15.154:0/4022977347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa58c0c9680 0x7fa58c0b2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:52.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.017+0000 7fa58ad9d640 1 -- 172.21.15.154:0/4022977347 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa57c009400 con 0x7fa58c0cb5d0 2023-09-23T03:16:52.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.017+0000 7fa583fff640 1 --2- 172.21.15.154:0/4022977347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa58c0c9680 0x7fa58c0b2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:52.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.017+0000 7fa58ad9d640 1 --2- 172.21.15.154:0/4022977347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa58c0cb5d0 0x7fa58c154a20 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa584009e50 tx=0x7fa584004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:52.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.018+0000 7fa582ffd640 1 -- 172.21.15.154:0/4022977347 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa584040670 con 0x7fa58c0cb5d0 2023-09-23T03:16:52.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.018+0000 7fa582ffd640 1 -- 172.21.15.154:0/4022977347 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa584040810 con 0x7fa58c0cb5d0 2023-09-23T03:16:52.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.018+0000 7fa590b90640 1 -- 172.21.15.154:0/4022977347 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa584009b30 con 0x7fa58c0cb5d0 2023-09-23T03:16:52.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.018+0000 7fa582ffd640 1 -- 172.21.15.154:0/4022977347 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa584040bc0 con 0x7fa58c0cb5d0 2023-09-23T03:16:52.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.018+0000 7fa590b90640 1 -- 172.21.15.154:0/4022977347 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa58c0c9c60 con 0x7fa58c0cb5d0 2023-09-23T03:16:52.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.019+0000 7fa590b90640 1 -- 172.21.15.154:0/4022977347 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa58c0b36f0 con 0x7fa58c0cb5d0 2023-09-23T03:16:52.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.020+0000 7fa582ffd640 1 -- 172.21.15.154:0/4022977347 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fa584040d20 con 0x7fa58c0cb5d0 2023-09-23T03:16:52.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.021+0000 7fa582ffd640 1 --2- 172.21.15.154:0/4022977347 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa56c06af30 0x7fa56c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:52.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.021+0000 7fa582ffd640 1 -- 172.21.15.154:0/4022977347 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7fa5840acb50 con 0x7fa58c0cb5d0 2023-09-23T03:16:52.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.021+0000 7fa58a59c640 1 --2- 172.21.15.154:0/4022977347 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa56c06af30 0x7fa56c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:52.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.022+0000 7fa58a59c640 1 --2- 172.21.15.154:0/4022977347 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa56c06af30 0x7fa56c06d3f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa57c009bc0 tx=0x7fa57c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:52.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.025+0000 7fa582ffd640 1 -- 172.21.15.154:0/4022977347 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fa58c0b36f0 con 0x7fa58c0cb5d0 2023-09-23T03:16:52.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.217+0000 7fa590b90640 1 -- 172.21.15.154:0/4022977347 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa58c0b36f0 con 0x7fa58c0cb5d0 2023-09-23T03:16:52.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.218+0000 7fa582ffd640 1 -- 172.21.15.154:0/4022977347 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa58c0b36f0 con 0x7fa58c0cb5d0 2023-09-23T03:16:52.218 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:16:52.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.221+0000 7fa590b90640 1 -- 172.21.15.154:0/4022977347 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa56c06af30 msgr2=0x7fa56c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:52.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.221+0000 7fa590b90640 1 --2- 172.21.15.154:0/4022977347 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa56c06af30 0x7fa56c06d3f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa57c009bc0 tx=0x7fa57c008040 comp rx=0 tx=0).stop 2023-09-23T03:16:52.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.221+0000 7fa590b90640 1 -- 172.21.15.154:0/4022977347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa58c0cb5d0 msgr2=0x7fa58c154a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:52.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.221+0000 7fa590b90640 1 --2- 172.21.15.154:0/4022977347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa58c0cb5d0 0x7fa58c154a20 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa584009e50 tx=0x7fa584004a40 comp rx=0 tx=0).stop 2023-09-23T03:16:52.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.221+0000 7fa590b90640 1 -- 172.21.15.154:0/4022977347 shutdown_connections 2023-09-23T03:16:52.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.221+0000 7fa590b90640 1 --2- 172.21.15.154:0/4022977347 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa56c06af30 0x7fa56c06d3f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:52.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.221+0000 7fa590b90640 1 --2- 172.21.15.154:0/4022977347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa58c0cb5d0 0x7fa58c154a20 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:52.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.221+0000 7fa590b90640 1 --2- 172.21.15.154:0/4022977347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa58c0c9680 0x7fa58c0b2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:52.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.221+0000 7fa590b90640 1 --2- 172.21.15.154:0/4022977347 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa58c0ad830 0x7fa58c0c65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:52.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.221+0000 7fa590b90640 1 -- 172.21.15.154:0/4022977347 >> 172.21.15.154:0/4022977347 conn(0x7fa58c06c010 msgr2=0x7fa58c15ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:52.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.221+0000 7fa590b90640 1 -- 172.21.15.154:0/4022977347 shutdown_connections 2023-09-23T03:16:52.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.221+0000 7fa590b90640 1 -- 172.21.15.154:0/4022977347 wait complete. 2023-09-23T03:16:52.240 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-23T03:16:52.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.458+0000 7ff80ffff640 1 Processor -- start 2023-09-23T03:16:52.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.458+0000 7ff80ffff640 1 -- start start 2023-09-23T03:16:52.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.460+0000 7ff80ffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff81015eb00 0x7ff81006fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:52.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.460+0000 7ff80ffff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff810070300 0x7ff8100ca500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:52.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.460+0000 7ff80ffff640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff81006bff0 con 0x7ff81006c7a0 2023-09-23T03:16:52.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.460+0000 7ff80ffff640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff810155070 con 0x7ff81015eb00 2023-09-23T03:16:52.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.460+0000 7ff80ffff640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff81015e0f0 con 0x7ff810070300 2023-09-23T03:16:52.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.460+0000 7ff80e7fc640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff81015eb00 0x7ff81006fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:52.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.460+0000 7ff80e7fc640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff81015eb00 0x7ff81006fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:41338/0 (socket says 172.21.15.154:41338) 2023-09-23T03:16:52.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.460+0000 7ff80e7fc640 1 -- 172.21.15.154:0/924027384 learned_addr learned my addr 172.21.15.154:0/924027384 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:52.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.460+0000 7ff80f7fe640 1 --2- 172.21.15.154:0/924027384 >> v2:172.21.15.190:3300/0 conn(0x7ff810070300 0x7ff8100ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:52.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.461+0000 7ff80e7fc640 1 -- 172.21.15.154:0/924027384 >> v2:172.21.15.190:3300/0 conn(0x7ff810070300 msgr2=0x7ff8100ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:52.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.461+0000 7ff80e7fc640 1 --2- 172.21.15.154:0/924027384 >> v2:172.21.15.190:3300/0 conn(0x7ff810070300 0x7ff8100ca500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:52.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.461+0000 7ff80e7fc640 1 -- 172.21.15.154:0/924027384 >> v1:172.21.15.190:6789/0 conn(0x7ff81006c7a0 legacy=0x7ff81015ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:52.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.461+0000 7ff80e7fc640 1 -- 172.21.15.154:0/924027384 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff81015efb0 con 0x7ff81015eb00 2023-09-23T03:16:52.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.462+0000 7ff80e7fc640 1 --2- 172.21.15.154:0/924027384 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff81015eb00 0x7ff81006fdc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff8100cbcf0 tx=0x7ff80402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e6436571102de6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:52.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.462+0000 7ff80dffb640 1 -- 172.21.15.154:0/924027384 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff804031870 con 0x7ff81015eb00 2023-09-23T03:16:52.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.462+0000 7ff80dffb640 1 -- 172.21.15.154:0/924027384 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8040319d0 con 0x7ff81015eb00 2023-09-23T03:16:52.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.462+0000 7ff80dffb640 1 -- 172.21.15.154:0/924027384 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff804031d60 con 0x7ff81015eb00 2023-09-23T03:16:52.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.463+0000 7ff80ffff640 1 -- 172.21.15.154:0/924027384 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff81015eb00 msgr2=0x7ff81006fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:52.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.463+0000 7ff80ffff640 1 --2- 172.21.15.154:0/924027384 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff81015eb00 0x7ff81006fdc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff8100cbcf0 tx=0x7ff80402b910 comp rx=0 tx=0).stop 2023-09-23T03:16:52.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.463+0000 7ff80ffff640 1 -- 172.21.15.154:0/924027384 shutdown_connections 2023-09-23T03:16:52.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.463+0000 7ff80ffff640 1 --2- 172.21.15.154:0/924027384 >> v2:172.21.15.190:3300/0 conn(0x7ff810070300 0x7ff8100ca500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:52.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.463+0000 7ff80ffff640 1 --2- 172.21.15.154:0/924027384 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff81015eb00 0x7ff81006fdc0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:52.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.463+0000 7ff80ffff640 1 -- 172.21.15.154:0/924027384 >> 172.21.15.154:0/924027384 conn(0x7ff81006a390 msgr2=0x7ff8100afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:52.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.463+0000 7ff80ffff640 1 -- 172.21.15.154:0/924027384 shutdown_connections 2023-09-23T03:16:52.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.463+0000 7ff80ffff640 1 -- 172.21.15.154:0/924027384 wait complete. 2023-09-23T03:16:52.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.465+0000 7ff80ffff640 1 Processor -- start 2023-09-23T03:16:52.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.465+0000 7ff80ffff640 1 -- start start 2023-09-23T03:16:52.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.466+0000 7ff80ffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff81006c7a0 0x7ff8100c4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:52.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.466+0000 7ff80ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff810070300 0x7ff8100c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:52.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.466+0000 7ff80ffff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff81015eb00 0x7ff8100aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:52.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.466+0000 7ff80ffff640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8101f67f0 con 0x7ff810070300 2023-09-23T03:16:52.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.466+0000 7ff80ffff640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8101f2700 con 0x7ff81006c7a0 2023-09-23T03:16:52.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.466+0000 7ff80ffff640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8101f6510 con 0x7ff81015eb00 2023-09-23T03:16:52.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.466+0000 7ff80e7fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff810070300 0x7ff8100c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:52.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.466+0000 7ff80e7fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff810070300 0x7ff8100c7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:41354/0 (socket says 172.21.15.154:41354) 2023-09-23T03:16:52.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.466+0000 7ff80e7fc640 1 -- 172.21.15.154:0/1868898885 learned_addr learned my addr 172.21.15.154:0/1868898885 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:52.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.467+0000 7ff80effd640 1 --2- 172.21.15.154:0/1868898885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff81006c7a0 0x7ff8100c4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:52.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.467+0000 7ff80f7fe640 1 --2- 172.21.15.154:0/1868898885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff81015eb00 0x7ff8100aeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:52.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.467+0000 7ff80e7fc640 1 -- 172.21.15.154:0/1868898885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff81015eb00 msgr2=0x7ff8100aeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:52.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.467+0000 7ff80e7fc640 1 --2- 172.21.15.154:0/1868898885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff81015eb00 0x7ff8100aeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:52.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.467+0000 7ff80e7fc640 1 -- 172.21.15.154:0/1868898885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff81006c7a0 msgr2=0x7ff8100c4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:52.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.467+0000 7ff80e7fc640 1 --2- 172.21.15.154:0/1868898885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff81006c7a0 0x7ff8100c4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:52.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.467+0000 7ff80e7fc640 1 -- 172.21.15.154:0/1868898885 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8040094d0 con 0x7ff810070300 2023-09-23T03:16:52.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.467+0000 7ff80e7fc640 1 --2- 172.21.15.154:0/1868898885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff810070300 0x7ff8100c7b00 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff804000c00 tx=0x7ff8040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:52.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.467+0000 7ff7f77fe640 1 -- 172.21.15.154:0/1868898885 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80403abe0 con 0x7ff810070300 2023-09-23T03:16:52.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.467+0000 7ff7f77fe640 1 -- 172.21.15.154:0/1868898885 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff80403ad40 con 0x7ff810070300 2023-09-23T03:16:52.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.468+0000 7ff80ffff640 1 -- 172.21.15.154:0/1868898885 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8100c7020 con 0x7ff810070300 2023-09-23T03:16:52.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.468+0000 7ff7f77fe640 1 -- 172.21.15.154:0/1868898885 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff804044610 con 0x7ff810070300 2023-09-23T03:16:52.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.468+0000 7ff80ffff640 1 -- 172.21.15.154:0/1868898885 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8100c75e0 con 0x7ff810070300 2023-09-23T03:16:52.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.469+0000 7ff80ffff640 1 -- 172.21.15.154:0/1868898885 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8100736b0 con 0x7ff810070300 2023-09-23T03:16:52.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.472+0000 7ff7f77fe640 1 -- 172.21.15.154:0/1868898885 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7ff80404d450 con 0x7ff810070300 2023-09-23T03:16:52.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.473+0000 7ff7f77fe640 1 --2- 172.21.15.154:0/1868898885 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff7fc06af60 0x7ff7fc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:52.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.473+0000 7ff7f77fe640 1 -- 172.21.15.154:0/1868898885 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7ff80402d040 con 0x7ff810070300 2023-09-23T03:16:52.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.473+0000 7ff80effd640 1 --2- 172.21.15.154:0/1868898885 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff7fc06af60 0x7ff7fc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:52.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.474+0000 7ff80effd640 1 --2- 172.21.15.154:0/1868898885 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff7fc06af60 0x7ff7fc06d420 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff8100c6cb0 tx=0x7ff800009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:52.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.476+0000 7ff7f77fe640 1 -- 172.21.15.154:0/1868898885 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7ff8040771a0 con 0x7ff810070300 2023-09-23T03:16:52.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.676+0000 7ff80ffff640 1 -- 172.21.15.154:0/1868898885 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff810071530 con 0x7ff810070300 2023-09-23T03:16:52.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.677+0000 7ff7f77fe640 1 -- 172.21.15.154:0/1868898885 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff80407a7c0 con 0x7ff810070300 2023-09-23T03:16:52.677 INFO:teuthology.orchestra.run.smithi154.stderr:listed 0 entries 2023-09-23T03:16:52.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.680+0000 7ff80ffff640 1 -- 172.21.15.154:0/1868898885 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff7fc06af60 msgr2=0x7ff7fc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:52.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.680+0000 7ff80ffff640 1 --2- 172.21.15.154:0/1868898885 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff7fc06af60 0x7ff7fc06d420 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff8100c6cb0 tx=0x7ff800009040 comp rx=0 tx=0).stop 2023-09-23T03:16:52.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.680+0000 7ff80ffff640 1 -- 172.21.15.154:0/1868898885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff810070300 msgr2=0x7ff8100c7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:52.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.680+0000 7ff80ffff640 1 --2- 172.21.15.154:0/1868898885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff810070300 0x7ff8100c7b00 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff804000c00 tx=0x7ff8040046c0 comp rx=0 tx=0).stop 2023-09-23T03:16:52.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.680+0000 7ff80ffff640 1 -- 172.21.15.154:0/1868898885 shutdown_connections 2023-09-23T03:16:52.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.680+0000 7ff80ffff640 1 --2- 172.21.15.154:0/1868898885 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff7fc06af60 0x7ff7fc06d420 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:52.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.680+0000 7ff80ffff640 1 --2- 172.21.15.154:0/1868898885 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff81015eb00 0x7ff8100aeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:52.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.680+0000 7ff80ffff640 1 --2- 172.21.15.154:0/1868898885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff810070300 0x7ff8100c7b00 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:52.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.680+0000 7ff80ffff640 1 --2- 172.21.15.154:0/1868898885 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff81006c7a0 0x7ff8100c4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:52.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.680+0000 7ff80ffff640 1 -- 172.21.15.154:0/1868898885 >> 172.21.15.154:0/1868898885 conn(0x7ff81006a390 msgr2=0x7ff8100c96b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:52.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.680+0000 7ff80ffff640 1 -- 172.21.15.154:0/1868898885 shutdown_connections 2023-09-23T03:16:52.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:52.680+0000 7ff80ffff640 1 -- 172.21.15.154:0/1868898885 wait complete. 2023-09-23T03:16:52.698 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi190.front.sepia.ceph.com... 2023-09-23T03:16:52.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:52.698 DEBUG:teuthology.orchestra.run.smithi190:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-09-23T03:16:52.805 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:16:52.805 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi190.front.sepia.ceph.com 2023-09-23T03:16:52.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:16:52.805 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-09-23T03:16:52.858 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-09-23T03:16:52.858 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:16:53.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.081+0000 7f0c21d2c640 1 Processor -- start 2023-09-23T03:16:53.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.081+0000 7f0c21d2c640 1 -- start start 2023-09-23T03:16:53.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.083+0000 7f0c21d2c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0c1c0acce0 0x7f0c1c15ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:53.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.083+0000 7f0c21d2c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c1c15ac30 con 0x7f0c1c0cf040 2023-09-23T03:16:53.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.083+0000 7f0c21d2c640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c1c0b1f70 con 0x7f0c1c0b1b20 2023-09-23T03:16:53.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.083+0000 7f0c21d2c640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c1c0b2e90 con 0x7f0c1c0acce0 2023-09-23T03:16:53.084 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.083+0000 7f0c1bfff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f0c1c0cf040 0x7f0c1c0ab140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:36366/0 (socket says 172.21.15.154:36366) 2023-09-23T03:16:53.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.083+0000 7f0c1bfff640 1 -- 172.21.15.154:0/1903119759 learned_addr learned my addr 172.21.15.154:0/1903119759 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:53.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.084+0000 7f0c1affd640 1 -- 172.21.15.154:0/1903119759 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3052199783 0 0) 0x7f0c14001490 con 0x7f0c1c0cf040 2023-09-23T03:16:53.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.084+0000 7f0c1affd640 1 -- 172.21.15.154:0/1903119759 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c08003520 con 0x7f0c1c0cf040 2023-09-23T03:16:53.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.084+0000 7f0c1b7fe640 1 --2- 172.21.15.154:0/1903119759 >> v2:172.21.15.190:3300/0 conn(0x7f0c1c0acce0 0x7f0c1c15ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:53.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.084+0000 7f0c1affd640 1 -- 172.21.15.154:0/1903119759 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2174912915 0 0) 0x7f0c08003520 con 0x7f0c1c0cf040 2023-09-23T03:16:53.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.084+0000 7f0c1affd640 1 -- 172.21.15.154:0/1903119759 >> v2:172.21.15.190:3300/0 conn(0x7f0c1c0acce0 msgr2=0x7f0c1c15ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:53.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.084+0000 7f0c1affd640 1 --2- 172.21.15.154:0/1903119759 >> v2:172.21.15.190:3300/0 conn(0x7f0c1c0acce0 0x7f0c1c15ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:53.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.084+0000 7f0c1affd640 1 -- 172.21.15.154:0/1903119759 >> v1:172.21.15.190:6789/0 conn(0x7f0c1c0b1b20 legacy=0x7f0c1c0ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:53.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.084+0000 7f0c1affd640 1 -- 172.21.15.154:0/1903119759 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c1c15f2a0 con 0x7f0c1c0cf040 2023-09-23T03:16:53.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.084+0000 7f0c1affd640 1 -- 172.21.15.154:0/1903119759 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f0c14001b80 con 0x7f0c1c0cf040 2023-09-23T03:16:53.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.084+0000 7f0c1affd640 1 -- 172.21.15.154:0/1903119759 >> v1:172.21.15.154:6789/0 conn(0x7f0c1c0cf040 legacy=0x7f0c1c0ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:53.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.084+0000 7f0c1affd640 1 --2- 172.21.15.154:0/1903119759 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c08003970 0x7f0c08003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:53.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.084+0000 7f0c1affd640 1 --2- 172.21.15.154:0/1903119759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c080047c0 0x7f0c08004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:53.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.084+0000 7f0c1affd640 1 --2- 172.21.15.154:0/1903119759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c080052a0 0x7f0c0800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:53.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.084+0000 7f0c1affd640 1 -- 172.21.15.154:0/1903119759 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1c0b2e90 con 0x7f0c08003970 2023-09-23T03:16:53.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.084+0000 7f0c1affd640 1 -- 172.21.15.154:0/1903119759 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0800c170 con 0x7f0c080047c0 2023-09-23T03:16:53.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.084+0000 7f0c1affd640 1 -- 172.21.15.154:0/1903119759 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0800c330 con 0x7f0c080052a0 2023-09-23T03:16:53.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.085+0000 7f0c1b7fe640 1 --2- 172.21.15.154:0/1903119759 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c08003970 0x7f0c08003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:53.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.085+0000 7f0c13fff640 1 --2- 172.21.15.154:0/1903119759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c080047c0 0x7f0c08004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:53.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.085+0000 7f0c1b7fe640 1 -- 172.21.15.154:0/1903119759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c080052a0 msgr2=0x7f0c0800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:53.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.085+0000 7f0c1b7fe640 1 --2- 172.21.15.154:0/1903119759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c080052a0 0x7f0c0800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:53.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.085+0000 7f0c1b7fe640 1 -- 172.21.15.154:0/1903119759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c080047c0 msgr2=0x7f0c08004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:53.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.085+0000 7f0c1b7fe640 1 --2- 172.21.15.154:0/1903119759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c080047c0 0x7f0c08004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:53.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.085+0000 7f0c1b7fe640 1 -- 172.21.15.154:0/1903119759 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c08003520 con 0x7f0c08003970 2023-09-23T03:16:53.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.085+0000 7f0c1bfff640 1 -- 172.21.15.154:0/1903119759 reap_dead start 2023-09-23T03:16:53.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.086+0000 7f0c1b7fe640 1 --2- 172.21.15.154:0/1903119759 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c08003970 0x7f0c08003d90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c00b9e0 tx=0x7f0c080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd446f13fad8a57 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:53.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.086+0000 7f0c1affd640 1 -- 172.21.15.154:0/1903119759 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c005b90 con 0x7f0c08003970 2023-09-23T03:16:53.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.086+0000 7f0c1affd640 1 -- 172.21.15.154:0/1903119759 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c0c005cf0 con 0x7f0c08003970 2023-09-23T03:16:53.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.086+0000 7f0c1affd640 1 -- 172.21.15.154:0/1903119759 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c006270 con 0x7f0c08003970 2023-09-23T03:16:53.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.087+0000 7f0c21d2c640 1 -- 172.21.15.154:0/1903119759 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c08003970 msgr2=0x7f0c08003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:53.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.087+0000 7f0c21d2c640 1 --2- 172.21.15.154:0/1903119759 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c08003970 0x7f0c08003d90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c00b9e0 tx=0x7f0c080057f0 comp rx=0 tx=0).stop 2023-09-23T03:16:53.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.087+0000 7f0c21d2c640 1 -- 172.21.15.154:0/1903119759 shutdown_connections 2023-09-23T03:16:53.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.087+0000 7f0c21d2c640 1 --2- 172.21.15.154:0/1903119759 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c08003970 0x7f0c08003d90 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:53.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.087+0000 7f0c21d2c640 1 -- 172.21.15.154:0/1903119759 >> 172.21.15.154:0/1903119759 conn(0x7f0c1c06a0f0 msgr2=0x7f0c1c06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:53.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.087+0000 7f0c21d2c640 1 -- 172.21.15.154:0/1903119759 shutdown_connections 2023-09-23T03:16:53.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.087+0000 7f0c21d2c640 1 -- 172.21.15.154:0/1903119759 wait complete. 2023-09-23T03:16:53.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.089+0000 7f0c21d2c640 1 Processor -- start 2023-09-23T03:16:53.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.089+0000 7f0c21d2c640 1 -- start start 2023-09-23T03:16:53.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.090+0000 7f0c21d2c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c08003970 0x7f0c1c155000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:53.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.090+0000 7f0c21d2c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c1c1f0c90 0x7f0c1c154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:53.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.090+0000 7f0c21d2c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c1c0ae600 0x7f0c1c1573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:53.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.090+0000 7f0c21d2c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1c1f10f0 con 0x7f0c1c0ae600 2023-09-23T03:16:53.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.090+0000 7f0c21d2c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1c0aee50 con 0x7f0c08003970 2023-09-23T03:16:53.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.090+0000 7f0c21d2c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1c0af130 con 0x7f0c1c1f0c90 2023-09-23T03:16:53.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.091+0000 7f0c1bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c1c0ae600 0x7f0c1c1573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:53.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.091+0000 7f0c1bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c1c0ae600 0x7f0c1c1573e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:41378/0 (socket says 172.21.15.154:41378) 2023-09-23T03:16:53.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.091+0000 7f0c1bfff640 1 -- 172.21.15.154:0/2354618070 learned_addr learned my addr 172.21.15.154:0/2354618070 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:53.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.091+0000 7f0c1b7fe640 1 --2- 172.21.15.154:0/2354618070 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c08003970 0x7f0c1c155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:53.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.091+0000 7f0c13fff640 1 --2- 172.21.15.154:0/2354618070 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c1c1f0c90 0x7f0c1c154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:53.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.091+0000 7f0c1bfff640 1 -- 172.21.15.154:0/2354618070 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c1c1f0c90 msgr2=0x7f0c1c154490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:53.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.091+0000 7f0c1bfff640 1 --2- 172.21.15.154:0/2354618070 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c1c1f0c90 0x7f0c1c154490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:53.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.091+0000 7f0c1bfff640 1 -- 172.21.15.154:0/2354618070 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c08003970 msgr2=0x7f0c1c155000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:53.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.091+0000 7f0c1bfff640 1 --2- 172.21.15.154:0/2354618070 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c08003970 0x7f0c1c155000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:53.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.091+0000 7f0c1bfff640 1 -- 172.21.15.154:0/2354618070 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c1c1f0490 con 0x7f0c1c0ae600 2023-09-23T03:16:53.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.091+0000 7f0c1b7fe640 1 --2- 172.21.15.154:0/2354618070 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c08003970 0x7f0c1c155000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:53.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.091+0000 7f0c1bfff640 1 --2- 172.21.15.154:0/2354618070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c1c0ae600 0x7f0c1c1573e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0c14001370 tx=0x7f0c1400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:53.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.091+0000 7f0c18ff9640 1 -- 172.21.15.154:0/2354618070 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1400ce70 con 0x7f0c1c0ae600 2023-09-23T03:16:53.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.092+0000 7f0c18ff9640 1 -- 172.21.15.154:0/2354618070 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1400a610 con 0x7f0c1c0ae600 2023-09-23T03:16:53.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.092+0000 7f0c18ff9640 1 -- 172.21.15.154:0/2354618070 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1400a9c0 con 0x7f0c1c0ae600 2023-09-23T03:16:53.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.092+0000 7f0c21d2c640 1 -- 172.21.15.154:0/2354618070 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c1c1f0770 con 0x7f0c1c0ae600 2023-09-23T03:16:53.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.092+0000 7f0c21d2c640 1 -- 172.21.15.154:0/2354618070 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c1c1f00e0 con 0x7f0c1c0ae600 2023-09-23T03:16:53.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.093+0000 7f0c21d2c640 1 -- 172.21.15.154:0/2354618070 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c1c073eb0 con 0x7f0c1c0ae600 2023-09-23T03:16:53.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.094+0000 7f0c18ff9640 1 -- 172.21.15.154:0/2354618070 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f0c1400ab20 con 0x7f0c1c0ae600 2023-09-23T03:16:53.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.095+0000 7f0c18ff9640 1 --2- 172.21.15.154:0/2354618070 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0bfc06af60 0x7f0bfc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:53.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.095+0000 7f0c18ff9640 1 -- 172.21.15.154:0/2354618070 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f0c14089000 con 0x7f0c1c0ae600 2023-09-23T03:16:53.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.095+0000 7f0c1b7fe640 1 --2- 172.21.15.154:0/2354618070 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0bfc06af60 0x7f0bfc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:53.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.096+0000 7f0c1b7fe640 1 --2- 172.21.15.154:0/2354618070 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0bfc06af60 0x7f0bfc06d420 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c036000 tx=0x7f0c0c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:53.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.099+0000 7f0c18ff9640 1 -- 172.21.15.154:0/2354618070 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f0c14053c90 con 0x7f0c1c0ae600 2023-09-23T03:16:53.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.333+0000 7f0c21d2c640 1 -- 172.21.15.154:0/2354618070 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c1c15ac30 con 0x7f0c1c0ae600 2023-09-23T03:16:53.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.334+0000 7f0c18ff9640 1 -- 172.21.15.154:0/2354618070 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4921 (secure 0 0 0) 0x7f0c1c15ac30 con 0x7f0c1c0ae600 2023-09-23T03:16:53.335 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:16:53.335 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2644365591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2644365591},{"type":"v1","addr":"172.21.15.190:6835","nonce":2644365591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/4145467852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4145467852},{"type":"v1","addr":"172.21.15.154:6835","nonce":4145467852}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1022302774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1022302774},{"type":"v1","addr":"172.21.15.154:6837","nonce":1022302774}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:15:20.686035+0000","modified":"2023-09-23T03:15:48.742318+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/3561509423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3561509423},{"type":"v1","addr":"172.21.15.190:6834","nonce":3561509423}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T03:16:53.335 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 9 2023-09-23T03:16:53.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.337+0000 7f0c21d2c640 1 -- 172.21.15.154:0/2354618070 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0bfc06af60 msgr2=0x7f0bfc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:53.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.337+0000 7f0c21d2c640 1 --2- 172.21.15.154:0/2354618070 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0bfc06af60 0x7f0bfc06d420 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c036000 tx=0x7f0c0c030000 comp rx=0 tx=0).stop 2023-09-23T03:16:53.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.337+0000 7f0c21d2c640 1 -- 172.21.15.154:0/2354618070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c1c0ae600 msgr2=0x7f0c1c1573e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:53.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.337+0000 7f0c21d2c640 1 --2- 172.21.15.154:0/2354618070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c1c0ae600 0x7f0c1c1573e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0c14001370 tx=0x7f0c1400cc20 comp rx=0 tx=0).stop 2023-09-23T03:16:53.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.337+0000 7f0c21d2c640 1 -- 172.21.15.154:0/2354618070 shutdown_connections 2023-09-23T03:16:53.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.337+0000 7f0c21d2c640 1 --2- 172.21.15.154:0/2354618070 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0bfc06af60 0x7f0bfc06d420 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:53.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.337+0000 7f0c21d2c640 1 --2- 172.21.15.154:0/2354618070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c1c0ae600 0x7f0c1c1573e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:53.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.337+0000 7f0c21d2c640 1 --2- 172.21.15.154:0/2354618070 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c1c1f0c90 0x7f0c1c154490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:53.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.337+0000 7f0c21d2c640 1 --2- 172.21.15.154:0/2354618070 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c08003970 0x7f0c1c155000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:53.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.337+0000 7f0c21d2c640 1 -- 172.21.15.154:0/2354618070 >> 172.21.15.154:0/2354618070 conn(0x7f0c1c06a0f0 msgr2=0x7f0c1c0cb480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:53.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.337+0000 7f0c21d2c640 1 -- 172.21.15.154:0/2354618070 shutdown_connections 2023-09-23T03:16:53.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.337+0000 7f0c21d2c640 1 -- 172.21.15.154:0/2354618070 wait complete. 2023-09-23T03:16:53.354 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:16:53.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.572+0000 7ff9d9444640 1 Processor -- start 2023-09-23T03:16:53.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.572+0000 7ff9d9444640 1 -- start start 2023-09-23T03:16:53.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.573+0000 7ff9d9444640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff9d40b1b20 0x7ff9d40ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:53.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.573+0000 7ff9d9444640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9d40cee70 0x7ff9d40ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:53.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.573+0000 7ff9d9444640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9d415ac30 con 0x7ff9d40acce0 2023-09-23T03:16:53.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.573+0000 7ff9d9444640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9d40b2e90 con 0x7ff9d40b1b20 2023-09-23T03:16:53.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.573+0000 7ff9d9444640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d40cf320 con 0x7ff9d40cee70 2023-09-23T03:16:53.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.574+0000 7ff9d27fc640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff9d40b1b20 0x7ff9d40ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:53.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.574+0000 7ff9d37fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9d40cee70 0x7ff9d40ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:53.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.574+0000 7ff9d2ffd640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7ff9d40acce0 0x7ff9d415ed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.154:38340/0 (socket says 172.21.15.154:38340) 2023-09-23T03:16:53.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.574+0000 7ff9d2ffd640 1 -- 172.21.15.154:0/2822328462 learned_addr learned my addr 172.21.15.154:0/2822328462 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:53.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.575+0000 7ff9d27fc640 1 -- 172.21.15.154:0/2822328462 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9d40cee70 msgr2=0x7ff9d40ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:53.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.575+0000 7ff9d27fc640 1 --2- 172.21.15.154:0/2822328462 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9d40cee70 0x7ff9d40ab140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:53.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.575+0000 7ff9d27fc640 1 -- 172.21.15.154:0/2822328462 >> v1:172.21.15.190:6789/0 conn(0x7ff9d40acce0 legacy=0x7ff9d415ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:53.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.575+0000 7ff9d27fc640 1 -- 172.21.15.154:0/2822328462 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d40b1f70 con 0x7ff9d40b1b20 2023-09-23T03:16:53.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.576+0000 7ff9d27fc640 1 --2- 172.21.15.154:0/2822328462 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9d40b1b20 0x7ff9d40ce930 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff9c80097a0 tx=0x7ff9c802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0d58456c0743d43 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:53.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.577+0000 7ff9d1ffb640 1 -- 172.21.15.154:0/2822328462 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c802e070 con 0x7ff9d40b1b20 2023-09-23T03:16:53.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.577+0000 7ff9d1ffb640 1 -- 172.21.15.154:0/2822328462 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c80315a0 con 0x7ff9d40b1b20 2023-09-23T03:16:53.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.577+0000 7ff9d1ffb640 1 -- 172.21.15.154:0/2822328462 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8031930 con 0x7ff9d40b1b20 2023-09-23T03:16:53.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.577+0000 7ff9d9444640 1 -- 172.21.15.154:0/2822328462 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9d40b1b20 msgr2=0x7ff9d40ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:53.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.577+0000 7ff9d9444640 1 --2- 172.21.15.154:0/2822328462 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9d40b1b20 0x7ff9d40ce930 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff9c80097a0 tx=0x7ff9c802b840 comp rx=0 tx=0).stop 2023-09-23T03:16:53.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.578+0000 7ff9d9444640 1 -- 172.21.15.154:0/2822328462 shutdown_connections 2023-09-23T03:16:53.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.578+0000 7ff9d9444640 1 --2- 172.21.15.154:0/2822328462 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9d40cee70 0x7ff9d40ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:53.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.578+0000 7ff9d9444640 1 --2- 172.21.15.154:0/2822328462 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9d40b1b20 0x7ff9d40ce930 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:53.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.578+0000 7ff9d9444640 1 -- 172.21.15.154:0/2822328462 >> 172.21.15.154:0/2822328462 conn(0x7ff9d406a0f0 msgr2=0x7ff9d406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:53.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.578+0000 7ff9d9444640 1 -- 172.21.15.154:0/2822328462 shutdown_connections 2023-09-23T03:16:53.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.578+0000 7ff9d9444640 1 -- 172.21.15.154:0/2822328462 wait complete. 2023-09-23T03:16:53.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.579+0000 7ff9d9444640 1 Processor -- start 2023-09-23T03:16:53.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.580+0000 7ff9d9444640 1 -- start start 2023-09-23T03:16:53.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.581+0000 7ff9d9444640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9d40acce0 0x7ff9d414c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:53.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.581+0000 7ff9d9444640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9d40b1b20 0x7ff9d40c6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:53.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.581+0000 7ff9d9444640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9d40cee70 0x7ff9d40affe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:53.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.581+0000 7ff9d9444640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d40c8250 con 0x7ff9d40acce0 2023-09-23T03:16:53.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.581+0000 7ff9d9444640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d40c7640 con 0x7ff9d40b1b20 2023-09-23T03:16:53.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.581+0000 7ff9d9444640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d40c7920 con 0x7ff9d40cee70 2023-09-23T03:16:53.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.581+0000 7ff9d2ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9d40acce0 0x7ff9d414c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:53.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.581+0000 7ff9d2ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9d40acce0 0x7ff9d414c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:41388/0 (socket says 172.21.15.154:41388) 2023-09-23T03:16:53.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.581+0000 7ff9d2ffd640 1 -- 172.21.15.154:0/3297767724 learned_addr learned my addr 172.21.15.154:0/3297767724 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:53.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.581+0000 7ff9d27fc640 1 --2- 172.21.15.154:0/3297767724 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9d40b1b20 0x7ff9d40c6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:53.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.581+0000 7ff9d37fe640 1 --2- 172.21.15.154:0/3297767724 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9d40cee70 0x7ff9d40affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:53.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.581+0000 7ff9d2ffd640 1 -- 172.21.15.154:0/3297767724 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9d40cee70 msgr2=0x7ff9d40affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:53.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.581+0000 7ff9d2ffd640 1 --2- 172.21.15.154:0/3297767724 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9d40cee70 0x7ff9d40affe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:53.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.582+0000 7ff9d2ffd640 1 -- 172.21.15.154:0/3297767724 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9d40b1b20 msgr2=0x7ff9d40c6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:53.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.582+0000 7ff9d2ffd640 1 --2- 172.21.15.154:0/3297767724 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9d40b1b20 0x7ff9d40c6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:53.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.582+0000 7ff9d2ffd640 1 -- 172.21.15.154:0/3297767724 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c8009400 con 0x7ff9d40acce0 2023-09-23T03:16:53.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.582+0000 7ff9d2ffd640 1 --2- 172.21.15.154:0/3297767724 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9d40acce0 0x7ff9d414c170 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff9c400c480 tx=0x7ff9c400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:53.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.582+0000 7ff9bb7fe640 1 -- 172.21.15.154:0/3297767724 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4005d30 con 0x7ff9d40acce0 2023-09-23T03:16:53.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.582+0000 7ff9bb7fe640 1 -- 172.21.15.154:0/3297767724 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c4005e90 con 0x7ff9d40acce0 2023-09-23T03:16:53.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.582+0000 7ff9d9444640 1 -- 172.21.15.154:0/3297767724 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9d40c7c00 con 0x7ff9d40acce0 2023-09-23T03:16:53.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.582+0000 7ff9bb7fe640 1 -- 172.21.15.154:0/3297767724 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4007270 con 0x7ff9d40acce0 2023-09-23T03:16:53.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.582+0000 7ff9d9444640 1 -- 172.21.15.154:0/3297767724 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9d414c6b0 con 0x7ff9d40acce0 2023-09-23T03:16:53.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.583+0000 7ff9d9444640 1 -- 172.21.15.154:0/3297767724 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9d4073eb0 con 0x7ff9d40acce0 2023-09-23T03:16:53.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.584+0000 7ff9bb7fe640 1 -- 172.21.15.154:0/3297767724 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7ff9c40073d0 con 0x7ff9d40acce0 2023-09-23T03:16:53.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.585+0000 7ff9bb7fe640 1 --2- 172.21.15.154:0/3297767724 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff9c006af30 0x7ff9c006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:53.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.585+0000 7ff9bb7fe640 1 -- 172.21.15.154:0/3297767724 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7ff9c4088fa0 con 0x7ff9d40acce0 2023-09-23T03:16:53.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.586+0000 7ff9d27fc640 1 --2- 172.21.15.154:0/3297767724 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff9c006af30 0x7ff9c006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:53.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.586+0000 7ff9d27fc640 1 --2- 172.21.15.154:0/3297767724 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff9c006af30 0x7ff9c006d3f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8036000 tx=0x7ff9c803d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:53.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.590+0000 7ff9bb7fe640 1 -- 172.21.15.154:0/3297767724 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7ff9c40576b0 con 0x7ff9d40acce0 2023-09-23T03:16:53.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.778+0000 7ff9d9444640 1 -- 172.21.15.154:0/3297767724 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9d415ac30 con 0x7ff9d40acce0 2023-09-23T03:16:53.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.781+0000 7ff9bb7fe640 1 -- 172.21.15.154:0/3297767724 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18744 (secure 0 0 0) 0x7ff9d415ac30 con 0x7ff9d40acce0 2023-09-23T03:16:53.782 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:16:53.782 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":26,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:16:03.151371+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:15:18.575849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T03:15:19.707212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T03:15:23.337138+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:16:53.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.784+0000 7ff9d9444640 1 -- 172.21.15.154:0/3297767724 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff9c006af30 msgr2=0x7ff9c006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:53.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.784+0000 7ff9d9444640 1 --2- 172.21.15.154:0/3297767724 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff9c006af30 0x7ff9c006d3f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8036000 tx=0x7ff9c803d040 comp rx=0 tx=0).stop 2023-09-23T03:16:53.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.784+0000 7ff9d9444640 1 -- 172.21.15.154:0/3297767724 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9d40acce0 msgr2=0x7ff9d414c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:53.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.784+0000 7ff9d9444640 1 --2- 172.21.15.154:0/3297767724 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9d40acce0 0x7ff9d414c170 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff9c400c480 tx=0x7ff9c400c940 comp rx=0 tx=0).stop 2023-09-23T03:16:53.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.784+0000 7ff9d9444640 1 -- 172.21.15.154:0/3297767724 shutdown_connections 2023-09-23T03:16:53.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.784+0000 7ff9d9444640 1 --2- 172.21.15.154:0/3297767724 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff9c006af30 0x7ff9c006d3f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:53.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.784+0000 7ff9d9444640 1 --2- 172.21.15.154:0/3297767724 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9d40cee70 0x7ff9d40affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:53.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.784+0000 7ff9d9444640 1 --2- 172.21.15.154:0/3297767724 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9d40b1b20 0x7ff9d40c6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:53.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.784+0000 7ff9d9444640 1 --2- 172.21.15.154:0/3297767724 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9d40acce0 0x7ff9d414c170 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:53.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.784+0000 7ff9d9444640 1 -- 172.21.15.154:0/3297767724 >> 172.21.15.154:0/3297767724 conn(0x7ff9d406a0f0 msgr2=0x7ff9d40ca050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:53.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.784+0000 7ff9d9444640 1 -- 172.21.15.154:0/3297767724 shutdown_connections 2023-09-23T03:16:53.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:53.784+0000 7ff9d9444640 1 -- 172.21.15.154:0/3297767724 wait complete. 2023-09-23T03:16:53.802 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:15:18.575849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:15:19.707212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-09-23T03:15:23.337138+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-23T03:16:53.802 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:16:54.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.021+0000 7fda4c1a9640 1 Processor -- start 2023-09-23T03:16:54.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.021+0000 7fda4c1a9640 1 -- start start 2023-09-23T03:16:54.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.022+0000 7fda4c1a9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fda440c8d70 0x7fda4415c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:54.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.022+0000 7fda4c1a9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fda44071290 0x7fda440ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:54.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.022+0000 7fda4c1a9640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda440c9d20 con 0x7fda440c9150 2023-09-23T03:16:54.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.022+0000 7fda4c1a9640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda44071850 con 0x7fda44071290 2023-09-23T03:16:54.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.022+0000 7fda4c1a9640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda44071b30 con 0x7fda440c8d70 2023-09-23T03:16:54.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.023+0000 7fda4a746640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fda44071290 0x7fda440ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:54.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.023+0000 7fda49744640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fda440c9150 0x7fda4415cc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:54808/0 (socket says 172.21.15.154:54808) 2023-09-23T03:16:54.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.023+0000 7fda49744640 1 -- 172.21.15.154:0/3187533840 learned_addr learned my addr 172.21.15.154:0/3187533840 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:54.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.023+0000 7fda49f45640 1 --2- 172.21.15.154:0/3187533840 >> v2:172.21.15.190:3300/0 conn(0x7fda440c8d70 0x7fda4415c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:54.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.023+0000 7fda4a746640 1 -- 172.21.15.154:0/3187533840 >> v2:172.21.15.190:3300/0 conn(0x7fda440c8d70 msgr2=0x7fda4415c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:54.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.023+0000 7fda4a746640 1 --2- 172.21.15.154:0/3187533840 >> v2:172.21.15.190:3300/0 conn(0x7fda440c8d70 0x7fda4415c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:54.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.023+0000 7fda4a746640 1 -- 172.21.15.154:0/3187533840 >> v1:172.21.15.154:6789/0 conn(0x7fda440c9150 legacy=0x7fda4415cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:54.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.023+0000 7fda4a746640 1 -- 172.21.15.154:0/3187533840 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda44071e10 con 0x7fda44071290 2023-09-23T03:16:54.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.023+0000 7fda48f43640 1 -- 172.21.15.154:0/3187533840 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 902552876 0 0) 0x7fda440c9d20 con 0x7fda440c9150 2023-09-23T03:16:54.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.024+0000 7fda4a746640 1 --2- 172.21.15.154:0/3187533840 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda44071290 0x7fda440ca1b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fda4406b790 tx=0x7fda3c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c39adaaec8b7581 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:54.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.025+0000 7fda48f43640 1 -- 172.21.15.154:0/3187533840 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c031830 con 0x7fda44071290 2023-09-23T03:16:54.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.025+0000 7fda48f43640 1 -- 172.21.15.154:0/3187533840 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda3c031990 con 0x7fda44071290 2023-09-23T03:16:54.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.025+0000 7fda48f43640 1 -- 172.21.15.154:0/3187533840 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c031d20 con 0x7fda44071290 2023-09-23T03:16:54.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.025+0000 7fda4c1a9640 1 -- 172.21.15.154:0/3187533840 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda44071290 msgr2=0x7fda440ca1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:54.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.025+0000 7fda4c1a9640 1 --2- 172.21.15.154:0/3187533840 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda44071290 0x7fda440ca1b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fda4406b790 tx=0x7fda3c02b8d0 comp rx=0 tx=0).stop 2023-09-23T03:16:54.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.025+0000 7fda4c1a9640 1 -- 172.21.15.154:0/3187533840 shutdown_connections 2023-09-23T03:16:54.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.025+0000 7fda4c1a9640 1 --2- 172.21.15.154:0/3187533840 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda44071290 0x7fda440ca1b0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:54.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.025+0000 7fda4c1a9640 1 --2- 172.21.15.154:0/3187533840 >> v2:172.21.15.190:3300/0 conn(0x7fda440c8d70 0x7fda4415c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:54.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.026+0000 7fda4c1a9640 1 -- 172.21.15.154:0/3187533840 >> 172.21.15.154:0/3187533840 conn(0x7fda4406c010 msgr2=0x7fda4406c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:54.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.026+0000 7fda4c1a9640 1 -- 172.21.15.154:0/3187533840 shutdown_connections 2023-09-23T03:16:54.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.026+0000 7fda4c1a9640 1 -- 172.21.15.154:0/3187533840 wait complete. 2023-09-23T03:16:54.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.027+0000 7fda4c1a9640 1 Processor -- start 2023-09-23T03:16:54.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.027+0000 7fda4c1a9640 1 -- start start 2023-09-23T03:16:54.030 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.028+0000 7fda4c1a9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda44071290 0x7fda44150d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:54.030 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.028+0000 7fda4c1a9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda440c8d70 0x7fda440ce390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:54.030 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.028+0000 7fda4c1a9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda440c9150 0x7fda440ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:54.030 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.028+0000 7fda4c1a9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda441512b0 con 0x7fda440c9150 2023-09-23T03:16:54.030 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.028+0000 7fda4c1a9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda440c65d0 con 0x7fda44071290 2023-09-23T03:16:54.030 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.028+0000 7fda4c1a9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda44158d40 con 0x7fda440c8d70 2023-09-23T03:16:54.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.029+0000 7fda4a746640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda440c9150 0x7fda440ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:54.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.029+0000 7fda4a746640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda440c9150 0x7fda440ce8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:55844/0 (socket says 172.21.15.154:55844) 2023-09-23T03:16:54.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.029+0000 7fda4a746640 1 -- 172.21.15.154:0/1437870975 learned_addr learned my addr 172.21.15.154:0/1437870975 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:54.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.029+0000 7fda49f45640 1 --2- 172.21.15.154:0/1437870975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda44071290 0x7fda44150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:54.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.029+0000 7fda49744640 1 --2- 172.21.15.154:0/1437870975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda440c8d70 0x7fda440ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:54.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.029+0000 7fda4a746640 1 -- 172.21.15.154:0/1437870975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda440c8d70 msgr2=0x7fda440ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:54.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.029+0000 7fda4a746640 1 --2- 172.21.15.154:0/1437870975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda440c8d70 0x7fda440ce390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:54.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.029+0000 7fda4a746640 1 -- 172.21.15.154:0/1437870975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda44071290 msgr2=0x7fda44150d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:54.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.029+0000 7fda4a746640 1 --2- 172.21.15.154:0/1437870975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda44071290 0x7fda44150d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:54.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.029+0000 7fda4a746640 1 -- 172.21.15.154:0/1437870975 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda3c009400 con 0x7fda440c9150 2023-09-23T03:16:54.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.029+0000 7fda49f45640 1 --2- 172.21.15.154:0/1437870975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda44071290 0x7fda44150d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:54.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.030+0000 7fda4a746640 1 --2- 172.21.15.154:0/1437870975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda440c9150 0x7fda440ce8d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fda3c02bee0 tx=0x7fda3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:54.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.030+0000 7fda3a7fc640 1 -- 172.21.15.154:0/1437870975 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c0317d0 con 0x7fda440c9150 2023-09-23T03:16:54.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.030+0000 7fda3a7fc640 1 -- 172.21.15.154:0/1437870975 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda3c03b9f0 con 0x7fda440c9150 2023-09-23T03:16:54.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.030+0000 7fda4c1a9640 1 -- 172.21.15.154:0/1437870975 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda44159020 con 0x7fda440c9150 2023-09-23T03:16:54.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.030+0000 7fda3a7fc640 1 -- 172.21.15.154:0/1437870975 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c03bda0 con 0x7fda440c9150 2023-09-23T03:16:54.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.030+0000 7fda4c1a9640 1 -- 172.21.15.154:0/1437870975 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda440c6870 con 0x7fda440c9150 2023-09-23T03:16:54.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.031+0000 7fda4c1a9640 1 -- 172.21.15.154:0/1437870975 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda4404e920 con 0x7fda440c9150 2023-09-23T03:16:54.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.032+0000 7fda3a7fc640 1 -- 172.21.15.154:0/1437870975 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fda3c04e440 con 0x7fda440c9150 2023-09-23T03:16:54.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.034+0000 7fda3a7fc640 1 --2- 172.21.15.154:0/1437870975 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fda2806af30 0x7fda2806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:54.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.034+0000 7fda3a7fc640 1 -- 172.21.15.154:0/1437870975 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7fda3c0abd60 con 0x7fda440c9150 2023-09-23T03:16:54.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.034+0000 7fda49f45640 1 --2- 172.21.15.154:0/1437870975 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fda2806af30 0x7fda2806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:54.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.035+0000 7fda49f45640 1 --2- 172.21.15.154:0/1437870975 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fda2806af30 0x7fda2806d3f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fda40007330 tx=0x7fda4000b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:54.038 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.038+0000 7fda3a7fc640 1 -- 172.21.15.154:0/1437870975 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fda3c0769e0 con 0x7fda440c9150 2023-09-23T03:16:54.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.227+0000 7fda4c1a9640 1 -- 172.21.15.154:0/1437870975 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fda4406cbc0 con 0x7fda440c9150 2023-09-23T03:16:54.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.227+0000 7fda3a7fc640 1 -- 172.21.15.154:0/1437870975 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fda3c0350a0 con 0x7fda440c9150 2023-09-23T03:16:54.228 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:16:54.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.230+0000 7fda4c1a9640 1 -- 172.21.15.154:0/1437870975 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fda2806af30 msgr2=0x7fda2806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:54.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.230+0000 7fda4c1a9640 1 --2- 172.21.15.154:0/1437870975 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fda2806af30 0x7fda2806d3f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fda40007330 tx=0x7fda4000b040 comp rx=0 tx=0).stop 2023-09-23T03:16:54.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.230+0000 7fda4c1a9640 1 -- 172.21.15.154:0/1437870975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda440c9150 msgr2=0x7fda440ce8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:54.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.230+0000 7fda4c1a9640 1 --2- 172.21.15.154:0/1437870975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda440c9150 0x7fda440ce8d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fda3c02bee0 tx=0x7fda3c0046c0 comp rx=0 tx=0).stop 2023-09-23T03:16:54.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.230+0000 7fda4c1a9640 1 -- 172.21.15.154:0/1437870975 shutdown_connections 2023-09-23T03:16:54.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.230+0000 7fda4c1a9640 1 --2- 172.21.15.154:0/1437870975 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fda2806af30 0x7fda2806d3f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:54.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.230+0000 7fda4c1a9640 1 --2- 172.21.15.154:0/1437870975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda440c9150 0x7fda440ce8d0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:54.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.230+0000 7fda4c1a9640 1 --2- 172.21.15.154:0/1437870975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fda440c8d70 0x7fda440ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:54.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.230+0000 7fda4c1a9640 1 --2- 172.21.15.154:0/1437870975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fda44071290 0x7fda44150d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:54.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.230+0000 7fda4c1a9640 1 -- 172.21.15.154:0/1437870975 >> 172.21.15.154:0/1437870975 conn(0x7fda4406c010 msgr2=0x7fda440c47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:54.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.230+0000 7fda4c1a9640 1 -- 172.21.15.154:0/1437870975 shutdown_connections 2023-09-23T03:16:54.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.230+0000 7fda4c1a9640 1 -- 172.21.15.154:0/1437870975 wait complete. 2023-09-23T03:16:54.249 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:16:54.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.471+0000 7f9df9a56640 1 Processor -- start 2023-09-23T03:16:54.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.472+0000 7f9df9a56640 1 -- start start 2023-09-23T03:16:54.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.473+0000 7f9df9a56640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9df40c9150 0x7f9df415cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:54.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.473+0000 7f9df9a56640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9df4071290 0x7f9df40ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:54.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.473+0000 7f9df9a56640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9df40c9d20 con 0x7f9df40c8d70 2023-09-23T03:16:54.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.473+0000 7f9df9a56640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9df4071850 con 0x7f9df40c9150 2023-09-23T03:16:54.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.473+0000 7f9df9a56640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df4071b30 con 0x7f9df4071290 2023-09-23T03:16:54.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.474+0000 7f9df2ffd640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f9df40c8d70 0x7f9df415c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:54818/0 (socket says 172.21.15.154:54818) 2023-09-23T03:16:54.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.474+0000 7f9df2ffd640 1 -- 172.21.15.154:0/1966594145 learned_addr learned my addr 172.21.15.154:0/1966594145 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:54.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.474+0000 7f9debfff640 1 --2- 172.21.15.154:0/1966594145 >> v2:172.21.15.190:3300/0 conn(0x7f9df40c9150 0x7f9df415cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:54.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.474+0000 7f9df37fe640 1 --2- 172.21.15.154:0/1966594145 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9df4071290 0x7f9df40ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:54.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.474+0000 7f9df27fc640 1 -- 172.21.15.154:0/1966594145 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3326012115 0 0) 0x7f9df40c9d20 con 0x7f9df40c8d70 2023-09-23T03:16:54.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.474+0000 7f9df27fc640 1 -- 172.21.15.154:0/1966594145 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9de0005730 con 0x7f9df40c8d70 2023-09-23T03:16:54.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.475+0000 7f9df27fc640 1 -- 172.21.15.154:0/1966594145 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2446270919 0 0) 0x7f9de0005730 con 0x7f9df40c8d70 2023-09-23T03:16:54.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.475+0000 7f9df27fc640 1 -- 172.21.15.154:0/1966594145 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9df4071290 msgr2=0x7f9df40ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:54.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.475+0000 7f9df27fc640 1 --2- 172.21.15.154:0/1966594145 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9df4071290 0x7f9df40ca1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:54.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.475+0000 7f9df27fc640 1 -- 172.21.15.154:0/1966594145 >> v2:172.21.15.190:3300/0 conn(0x7f9df40c9150 msgr2=0x7f9df415cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:54.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.475+0000 7f9df27fc640 1 --2- 172.21.15.154:0/1966594145 >> v2:172.21.15.190:3300/0 conn(0x7f9df40c9150 0x7f9df415cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:54.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.475+0000 7f9df27fc640 1 -- 172.21.15.154:0/1966594145 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df4071e10 con 0x7f9df40c8d70 2023-09-23T03:16:54.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.475+0000 7f9df27fc640 1 -- 172.21.15.154:0/1966594145 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f9de4001690 con 0x7f9df40c8d70 2023-09-23T03:16:54.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.475+0000 7f9df27fc640 1 -- 172.21.15.154:0/1966594145 >> v1:172.21.15.154:6789/0 conn(0x7f9df40c8d70 legacy=0x7f9df415c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:54.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.475+0000 7f9df27fc640 1 --2- 172.21.15.154:0/1966594145 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9df4071290 0x7f9de0005b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:54.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.475+0000 7f9df27fc640 1 --2- 172.21.15.154:0/1966594145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9de0006100 0x7f9de00069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:54.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.475+0000 7f9df27fc640 1 --2- 172.21.15.154:0/1966594145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9de0006f10 0x7f9de0007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:54.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.475+0000 7f9df27fc640 1 -- 172.21.15.154:0/1966594145 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df4071850 con 0x7f9de0006100 2023-09-23T03:16:54.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.475+0000 7f9df27fc640 1 -- 172.21.15.154:0/1966594145 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df4071b30 con 0x7f9de0006f10 2023-09-23T03:16:54.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.475+0000 7f9df27fc640 1 -- 172.21.15.154:0/1966594145 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de000db90 con 0x7f9df4071290 2023-09-23T03:16:54.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.475+0000 7f9debfff640 1 --2- 172.21.15.154:0/1966594145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9de0006100 0x7f9de00069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:54.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.475+0000 7f9debfff640 1 -- 172.21.15.154:0/1966594145 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9df4071290 msgr2=0x7f9de0005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:54.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.475+0000 7f9debfff640 1 --2- 172.21.15.154:0/1966594145 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9df4071290 0x7f9de0005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:54.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.475+0000 7f9debfff640 1 -- 172.21.15.154:0/1966594145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9de0006f10 msgr2=0x7f9de0007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:54.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.475+0000 7f9debfff640 1 --2- 172.21.15.154:0/1966594145 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9de0006f10 0x7f9de0007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:54.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.475+0000 7f9debfff640 1 -- 172.21.15.154:0/1966594145 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de0005730 con 0x7f9de0006100 2023-09-23T03:16:54.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.477+0000 7f9debfff640 1 --2- 172.21.15.154:0/1966594145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9de0006100 0x7f9de00069d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc009ac0 tx=0x7f9de0007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e230e0d46546ff9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:54.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.477+0000 7f9df27fc640 1 -- 172.21.15.154:0/1966594145 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc02c6d0 con 0x7f9de0006100 2023-09-23T03:16:54.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.477+0000 7f9df27fc640 1 -- 172.21.15.154:0/1966594145 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ddc02c830 con 0x7f9de0006100 2023-09-23T03:16:54.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.477+0000 7f9df27fc640 1 -- 172.21.15.154:0/1966594145 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc02cbc0 con 0x7f9de0006100 2023-09-23T03:16:54.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.477+0000 7f9df9a56640 1 -- 172.21.15.154:0/1966594145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9de0006100 msgr2=0x7f9de00069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:54.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.477+0000 7f9df9a56640 1 --2- 172.21.15.154:0/1966594145 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9de0006100 0x7f9de00069d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc009ac0 tx=0x7f9de0007a00 comp rx=0 tx=0).stop 2023-09-23T03:16:54.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.477+0000 7f9df37fe640 1 -- 172.21.15.154:0/1966594145 reap_dead start 2023-09-23T03:16:54.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.477+0000 7f9df9a56640 1 -- 172.21.15.154:0/1966594145 shutdown_connections 2023-09-23T03:16:54.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.477+0000 7f9df9a56640 1 -- 172.21.15.154:0/1966594145 >> 172.21.15.154:0/1966594145 conn(0x7f9df406c010 msgr2=0x7f9df406c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:54.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.478+0000 7f9df9a56640 1 -- 172.21.15.154:0/1966594145 shutdown_connections 2023-09-23T03:16:54.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.478+0000 7f9df9a56640 1 -- 172.21.15.154:0/1966594145 wait complete. 2023-09-23T03:16:54.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.479+0000 7f9df9a56640 1 Processor -- start 2023-09-23T03:16:54.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.480+0000 7f9df9a56640 1 -- start start 2023-09-23T03:16:54.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.480+0000 7f9df9a56640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9de0006100 0x7f9df4152e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:54.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.481+0000 7f9df9a56640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9df4158850 0x7f9df4158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:54.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.481+0000 7f9df9a56640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9df415c800 0x7f9df415ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:54.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.481+0000 7f9df9a56640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df4153370 con 0x7f9df415c800 2023-09-23T03:16:54.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.481+0000 7f9df9a56640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df4159210 con 0x7f9df4158850 2023-09-23T03:16:54.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.481+0000 7f9df9a56640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df40aa350 con 0x7f9de0006100 2023-09-23T03:16:54.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.481+0000 7f9df37fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9df415c800 0x7f9df415ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:54.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.481+0000 7f9df37fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9df415c800 0x7f9df415ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:55852/0 (socket says 172.21.15.154:55852) 2023-09-23T03:16:54.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.481+0000 7f9df37fe640 1 -- 172.21.15.154:0/3564923000 learned_addr learned my addr 172.21.15.154:0/3564923000 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:54.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.481+0000 7f9df2ffd640 1 --2- 172.21.15.154:0/3564923000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9de0006100 0x7f9df4152e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:54.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.481+0000 7f9debfff640 1 --2- 172.21.15.154:0/3564923000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9df4158850 0x7f9df4158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:54.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.481+0000 7f9df37fe640 1 -- 172.21.15.154:0/3564923000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9de0006100 msgr2=0x7f9df4152e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:54.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.481+0000 7f9df37fe640 1 --2- 172.21.15.154:0/3564923000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9de0006100 0x7f9df4152e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:54.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.481+0000 7f9df37fe640 1 -- 172.21.15.154:0/3564923000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9df4158850 msgr2=0x7f9df4158cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:54.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.481+0000 7f9df37fe640 1 --2- 172.21.15.154:0/3564923000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9df4158850 0x7f9df4158cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:54.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.481+0000 7f9df37fe640 1 -- 172.21.15.154:0/3564923000 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df415d5a0 con 0x7f9df415c800 2023-09-23T03:16:54.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.481+0000 7f9debfff640 1 --2- 172.21.15.154:0/3564923000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9df4158850 0x7f9df4158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:54.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.482+0000 7f9df37fe640 1 --2- 172.21.15.154:0/3564923000 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9df415c800 0x7f9df415ccb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9df406b8b0 tx=0x7f9dec00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:54.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.483+0000 7f9df0ff9640 1 -- 172.21.15.154:0/3564923000 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec010040 con 0x7f9df415c800 2023-09-23T03:16:54.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.483+0000 7f9df0ff9640 1 -- 172.21.15.154:0/3564923000 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dec00bb10 con 0x7f9df415c800 2023-09-23T03:16:54.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.483+0000 7f9df0ff9640 1 -- 172.21.15.154:0/3564923000 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec014400 con 0x7f9df415c800 2023-09-23T03:16:54.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.483+0000 7f9df9a56640 1 -- 172.21.15.154:0/3564923000 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9df415d880 con 0x7f9df415c800 2023-09-23T03:16:54.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.483+0000 7f9df9a56640 1 -- 172.21.15.154:0/3564923000 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9df415de40 con 0x7f9df415c800 2023-09-23T03:16:54.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.484+0000 7f9df9a56640 1 -- 172.21.15.154:0/3564923000 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9df40aa520 con 0x7f9df415c800 2023-09-23T03:16:54.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.485+0000 7f9df0ff9640 1 -- 172.21.15.154:0/3564923000 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f9dec014560 con 0x7f9df415c800 2023-09-23T03:16:54.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.486+0000 7f9df0ff9640 1 --2- 172.21.15.154:0/3564923000 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9dd406af60 0x7f9dd406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:54.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.486+0000 7f9df0ff9640 1 -- 172.21.15.154:0/3564923000 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f9dec08a140 con 0x7f9df415c800 2023-09-23T03:16:54.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.486+0000 7f9df2ffd640 1 --2- 172.21.15.154:0/3564923000 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9dd406af60 0x7f9dd406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:54.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.487+0000 7f9df2ffd640 1 --2- 172.21.15.154:0/3564923000 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9dd406af60 0x7f9dd406d420 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9de4001840 tx=0x7f9de40015f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:54.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.491+0000 7f9df0ff9640 1 -- 172.21.15.154:0/3564923000 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f9df40aa520 con 0x7f9df415c800 2023-09-23T03:16:54.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.680+0000 7f9df9a56640 1 -- 172.21.15.154:0/3564923000 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9df40aa520 con 0x7f9df415c800 2023-09-23T03:16:54.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.681+0000 7f9df0ff9640 1 -- 172.21.15.154:0/3564923000 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9df40aa520 con 0x7f9df415c800 2023-09-23T03:16:54.681 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:16:54.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.684+0000 7f9df9a56640 1 -- 172.21.15.154:0/3564923000 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9dd406af60 msgr2=0x7f9dd406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:54.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.684+0000 7f9df9a56640 1 --2- 172.21.15.154:0/3564923000 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9dd406af60 0x7f9dd406d420 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9de4001840 tx=0x7f9de40015f0 comp rx=0 tx=0).stop 2023-09-23T03:16:54.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.684+0000 7f9df9a56640 1 -- 172.21.15.154:0/3564923000 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9df415c800 msgr2=0x7f9df415ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:54.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.684+0000 7f9df9a56640 1 --2- 172.21.15.154:0/3564923000 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9df415c800 0x7f9df415ccb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9df406b8b0 tx=0x7f9dec00b480 comp rx=0 tx=0).stop 2023-09-23T03:16:54.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.684+0000 7f9df9a56640 1 -- 172.21.15.154:0/3564923000 shutdown_connections 2023-09-23T03:16:54.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.684+0000 7f9df9a56640 1 --2- 172.21.15.154:0/3564923000 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9dd406af60 0x7f9dd406d420 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:54.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.684+0000 7f9df9a56640 1 --2- 172.21.15.154:0/3564923000 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9df415c800 0x7f9df415ccb0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:54.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.684+0000 7f9df9a56640 1 --2- 172.21.15.154:0/3564923000 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9df4158850 0x7f9df4158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:54.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.684+0000 7f9df9a56640 1 --2- 172.21.15.154:0/3564923000 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9de0006100 0x7f9df4152e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:54.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.684+0000 7f9df9a56640 1 -- 172.21.15.154:0/3564923000 >> 172.21.15.154:0/3564923000 conn(0x7f9df406c010 msgr2=0x7f9df40c38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:54.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.684+0000 7f9df9a56640 1 -- 172.21.15.154:0/3564923000 shutdown_connections 2023-09-23T03:16:54.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.684+0000 7f9df9a56640 1 -- 172.21.15.154:0/3564923000 wait complete. 2023-09-23T03:16:54.704 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:16:54.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.919+0000 7fab045e7640 1 Processor -- start 2023-09-23T03:16:54.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.919+0000 7fab045e7640 1 -- start start 2023-09-23T03:16:54.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.921+0000 7fab045e7640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7faafc0cb5b0 0x7faafc0cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:54.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.921+0000 7fab045e7640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faafc15b6a0 con 0x7faafc15bb30 2023-09-23T03:16:54.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.921+0000 7fab045e7640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faafc15f0b0 con 0x7faafc15aba0 2023-09-23T03:16:54.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.921+0000 7fab045e7640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faafc15bf70 con 0x7faafc0cb5b0 2023-09-23T03:16:54.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.921+0000 7fab01b82640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7faafc15bb30 0x7faafc0caea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:54822/0 (socket says 172.21.15.154:54822) 2023-09-23T03:16:54.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.921+0000 7fab01b82640 1 -- 172.21.15.154:0/4195167733 learned_addr learned my addr 172.21.15.154:0/4195167733 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:54.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.922+0000 7fab02b84640 1 --2- 172.21.15.154:0/4195167733 >> v2:172.21.15.190:3300/0 conn(0x7faafc0cb5b0 0x7faafc0cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:54.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.922+0000 7fab01381640 1 -- 172.21.15.154:0/4195167733 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 211881730 0 0) 0x7faafc15b6a0 con 0x7faafc15bb30 2023-09-23T03:16:54.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.922+0000 7fab01381640 1 -- 172.21.15.154:0/4195167733 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faae8003520 con 0x7faafc15bb30 2023-09-23T03:16:54.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.922+0000 7fab01381640 1 -- 172.21.15.154:0/4195167733 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3861567283 0 0) 0x7faae8003520 con 0x7faafc15bb30 2023-09-23T03:16:54.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.922+0000 7fab01381640 1 -- 172.21.15.154:0/4195167733 >> v2:172.21.15.190:3300/0 conn(0x7faafc0cb5b0 msgr2=0x7faafc0cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:54.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.922+0000 7fab01381640 1 --2- 172.21.15.154:0/4195167733 >> v2:172.21.15.190:3300/0 conn(0x7faafc0cb5b0 0x7faafc0cc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:54.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.922+0000 7fab01381640 1 -- 172.21.15.154:0/4195167733 >> v1:172.21.15.190:6789/0 conn(0x7faafc15aba0 legacy=0x7faafc15e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:54.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.922+0000 7fab01381640 1 -- 172.21.15.154:0/4195167733 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faafc15f390 con 0x7faafc15bb30 2023-09-23T03:16:54.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.922+0000 7fab01381640 1 -- 172.21.15.154:0/4195167733 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7faaec001670 con 0x7faafc15bb30 2023-09-23T03:16:54.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.922+0000 7fab01381640 1 -- 172.21.15.154:0/4195167733 >> v1:172.21.15.154:6789/0 conn(0x7faafc15bb30 legacy=0x7faafc0caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:54.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.922+0000 7fab01381640 1 --2- 172.21.15.154:0/4195167733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faae80039c0 0x7faae8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:54.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.922+0000 7fab01381640 1 --2- 172.21.15.154:0/4195167733 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faae80047c0 0x7faae8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:54.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.922+0000 7fab01381640 1 --2- 172.21.15.154:0/4195167733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faae8005180 0x7faae800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:54.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.922+0000 7fab01381640 1 -- 172.21.15.154:0/4195167733 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faafc15bf70 con 0x7faae80039c0 2023-09-23T03:16:54.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.922+0000 7fab01381640 1 -- 172.21.15.154:0/4195167733 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae800c080 con 0x7faae8005180 2023-09-23T03:16:54.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.922+0000 7fab01381640 1 -- 172.21.15.154:0/4195167733 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faae800c240 con 0x7faae80047c0 2023-09-23T03:16:54.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.923+0000 7fab02383640 1 --2- 172.21.15.154:0/4195167733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faae80039c0 0x7faae8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:54.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.923+0000 7fab02b84640 1 --2- 172.21.15.154:0/4195167733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faae8005180 0x7faae800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:54.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.923+0000 7fab01b82640 1 --2- 172.21.15.154:0/4195167733 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faae80047c0 0x7faae8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:54.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.924+0000 7fab02383640 1 -- 172.21.15.154:0/4195167733 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faae80047c0 msgr2=0x7faae8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:54.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.924+0000 7fab02383640 1 --2- 172.21.15.154:0/4195167733 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faae80047c0 0x7faae8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:54.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.924+0000 7fab02383640 1 -- 172.21.15.154:0/4195167733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faae8005180 msgr2=0x7faae800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:54.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.924+0000 7fab02383640 1 --2- 172.21.15.154:0/4195167733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faae8005180 0x7faae800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:54.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.924+0000 7fab02383640 1 -- 172.21.15.154:0/4195167733 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae8003520 con 0x7faae80039c0 2023-09-23T03:16:54.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.924+0000 7fab02b84640 1 -- 172.21.15.154:0/4195167733 reap_dead start 2023-09-23T03:16:54.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.925+0000 7fab02383640 1 --2- 172.21.15.154:0/4195167733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faae80039c0 0x7faae8003de0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7faaf8001750 tx=0x7faae80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=801e308c7b93fb6d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:54.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.926+0000 7fab01381640 1 -- 172.21.15.154:0/4195167733 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf80052d0 con 0x7faae80039c0 2023-09-23T03:16:54.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.926+0000 7fab01381640 1 -- 172.21.15.154:0/4195167733 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaf802e050 con 0x7faae80039c0 2023-09-23T03:16:54.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.926+0000 7fab01381640 1 -- 172.21.15.154:0/4195167733 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf8003830 con 0x7faae80039c0 2023-09-23T03:16:54.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.926+0000 7fab045e7640 1 -- 172.21.15.154:0/4195167733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faae80039c0 msgr2=0x7faae8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:54.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.926+0000 7fab045e7640 1 --2- 172.21.15.154:0/4195167733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faae80039c0 0x7faae8003de0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7faaf8001750 tx=0x7faae80057f0 comp rx=0 tx=0).stop 2023-09-23T03:16:54.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.926+0000 7fab045e7640 1 -- 172.21.15.154:0/4195167733 shutdown_connections 2023-09-23T03:16:54.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.926+0000 7fab045e7640 1 --2- 172.21.15.154:0/4195167733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faae80039c0 0x7faae8003de0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:54.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.926+0000 7fab045e7640 1 -- 172.21.15.154:0/4195167733 >> 172.21.15.154:0/4195167733 conn(0x7faafc06c010 msgr2=0x7faafc06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:54.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.926+0000 7fab045e7640 1 -- 172.21.15.154:0/4195167733 shutdown_connections 2023-09-23T03:16:54.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.926+0000 7fab045e7640 1 -- 172.21.15.154:0/4195167733 wait complete. 2023-09-23T03:16:54.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.928+0000 7fab045e7640 1 Processor -- start 2023-09-23T03:16:54.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.928+0000 7fab045e7640 1 -- start start 2023-09-23T03:16:54.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.929+0000 7fab045e7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faae80039c0 0x7faafc0c4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:54.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.929+0000 7fab045e7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faafc154810 0x7faafc154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:54.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.929+0000 7fab045e7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faafc0c57b0 0x7faafc0c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:54.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.929+0000 7fab045e7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faafc1551d0 con 0x7faafc0c57b0 2023-09-23T03:16:54.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.929+0000 7fab045e7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faafc0c3810 con 0x7faafc154810 2023-09-23T03:16:54.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.929+0000 7fab045e7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faafc0c6520 con 0x7faae80039c0 2023-09-23T03:16:54.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.930+0000 7fab02b84640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faafc0c57b0 0x7faafc0c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:54.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.930+0000 7fab02b84640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faafc0c57b0 0x7faafc0c5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:55876/0 (socket says 172.21.15.154:55876) 2023-09-23T03:16:54.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.930+0000 7fab02b84640 1 -- 172.21.15.154:0/3292631113 learned_addr learned my addr 172.21.15.154:0/3292631113 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:54.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.930+0000 7fab01b82640 1 --2- 172.21.15.154:0/3292631113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faafc154810 0x7faafc154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:54.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.930+0000 7fab02383640 1 --2- 172.21.15.154:0/3292631113 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faae80039c0 0x7faafc0c4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:54.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.930+0000 7fab02b84640 1 -- 172.21.15.154:0/3292631113 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faae80039c0 msgr2=0x7faafc0c4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:54.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.930+0000 7fab02b84640 1 --2- 172.21.15.154:0/3292631113 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faae80039c0 0x7faafc0c4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:54.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.930+0000 7fab02b84640 1 -- 172.21.15.154:0/3292631113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faafc154810 msgr2=0x7faafc154c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:54.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.930+0000 7fab02b84640 1 --2- 172.21.15.154:0/3292631113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faafc154810 0x7faafc154c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:54.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.930+0000 7fab02b84640 1 -- 172.21.15.154:0/3292631113 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faafc0c2ee0 con 0x7faafc0c57b0 2023-09-23T03:16:54.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.931+0000 7fab02b84640 1 --2- 172.21.15.154:0/3292631113 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faafc0c57b0 0x7faafc0c5c30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7faafc0adf60 tx=0x7faaf400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:54.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.931+0000 7faaf2ffd640 1 -- 172.21.15.154:0/3292631113 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf400e040 con 0x7faafc0c57b0 2023-09-23T03:16:54.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.931+0000 7faaf2ffd640 1 -- 172.21.15.154:0/3292631113 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaf400f040 con 0x7faafc0c57b0 2023-09-23T03:16:54.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.931+0000 7faaf2ffd640 1 -- 172.21.15.154:0/3292631113 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf4015430 con 0x7faafc0c57b0 2023-09-23T03:16:54.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.931+0000 7fab045e7640 1 -- 172.21.15.154:0/3292631113 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faafc0c31c0 con 0x7faafc0c57b0 2023-09-23T03:16:54.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.931+0000 7fab045e7640 1 -- 172.21.15.154:0/3292631113 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faafc0c5250 con 0x7faafc0c57b0 2023-09-23T03:16:54.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.932+0000 7fab045e7640 1 -- 172.21.15.154:0/3292631113 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faafc0c39c0 con 0x7faafc0c57b0 2023-09-23T03:16:54.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.933+0000 7faaf2ffd640 1 -- 172.21.15.154:0/3292631113 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7faaf4015590 con 0x7faafc0c57b0 2023-09-23T03:16:54.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.934+0000 7faaf2ffd640 1 --2- 172.21.15.154:0/3292631113 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7faae006af60 0x7faae006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:54.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.934+0000 7faaf2ffd640 1 -- 172.21.15.154:0/3292631113 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7faaf408a330 con 0x7faafc0c57b0 2023-09-23T03:16:54.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.934+0000 7fab02383640 1 --2- 172.21.15.154:0/3292631113 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7faae006af60 0x7faae006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:54.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.935+0000 7fab02383640 1 --2- 172.21.15.154:0/3292631113 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7faae006af60 0x7faae006d420 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faaf8033000 tx=0x7faaf8009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:54.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:54.938+0000 7faaf2ffd640 1 -- 172.21.15.154:0/3292631113 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7faafc0c39c0 con 0x7faafc0c57b0 2023-09-23T03:16:55.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.126+0000 7fab045e7640 1 -- 172.21.15.154:0/3292631113 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faafc0c3980 con 0x7faafc0c57b0 2023-09-23T03:16:55.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.126+0000 7faaf2ffd640 1 -- 172.21.15.154:0/3292631113 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7faaf4054f20 con 0x7faafc0c57b0 2023-09-23T03:16:55.127 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:16:55.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.129+0000 7fab045e7640 1 -- 172.21.15.154:0/3292631113 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7faae006af60 msgr2=0x7faae006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:55.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.129+0000 7fab045e7640 1 --2- 172.21.15.154:0/3292631113 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7faae006af60 0x7faae006d420 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faaf8033000 tx=0x7faaf8009030 comp rx=0 tx=0).stop 2023-09-23T03:16:55.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.129+0000 7fab045e7640 1 -- 172.21.15.154:0/3292631113 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faafc0c57b0 msgr2=0x7faafc0c5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:55.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.129+0000 7fab045e7640 1 --2- 172.21.15.154:0/3292631113 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faafc0c57b0 0x7faafc0c5c30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7faafc0adf60 tx=0x7faaf400f580 comp rx=0 tx=0).stop 2023-09-23T03:16:55.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.129+0000 7fab045e7640 1 -- 172.21.15.154:0/3292631113 shutdown_connections 2023-09-23T03:16:55.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.129+0000 7fab045e7640 1 --2- 172.21.15.154:0/3292631113 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7faae006af60 0x7faae006d420 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:55.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.129+0000 7fab045e7640 1 --2- 172.21.15.154:0/3292631113 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faafc0c57b0 0x7faafc0c5c30 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:55.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.129+0000 7fab045e7640 1 --2- 172.21.15.154:0/3292631113 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faafc154810 0x7faafc154c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:55.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.129+0000 7fab045e7640 1 --2- 172.21.15.154:0/3292631113 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faae80039c0 0x7faafc0c4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:55.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.129+0000 7fab045e7640 1 -- 172.21.15.154:0/3292631113 >> 172.21.15.154:0/3292631113 conn(0x7faafc06c010 msgr2=0x7faafc15cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:55.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.129+0000 7fab045e7640 1 -- 172.21.15.154:0/3292631113 shutdown_connections 2023-09-23T03:16:55.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.129+0000 7fab045e7640 1 -- 172.21.15.154:0/3292631113 wait complete. 2023-09-23T03:16:55.147 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-09-23T03:16:55.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.364+0000 7f89e27e2640 1 Processor -- start 2023-09-23T03:16:55.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.364+0000 7f89e27e2640 1 -- start start 2023-09-23T03:16:55.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.365+0000 7f89e27e2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f89dc15ab20 0x7f89dc0cee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:55.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.365+0000 7f89e27e2640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f89dc0cf370 0x7f89dc15e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:55.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.365+0000 7f89e27e2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89dc15eb00 0x7f89dc0d0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:55.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.365+0000 7f89e27e2640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89dc15f480 con 0x7f89dc15ab20 2023-09-23T03:16:55.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.365+0000 7f89e27e2640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89dc15af00 con 0x7f89dc0cf370 2023-09-23T03:16:55.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.365+0000 7f89e27e2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89dc0cd3f0 con 0x7f89dc15eb00 2023-09-23T03:16:55.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.366+0000 7f89dbfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f89dc15ab20 0x7f89dc0cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:55.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.366+0000 7f89dbfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f89dc15ab20 0x7f89dc0cee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:55882/0 (socket says 172.21.15.154:55882) 2023-09-23T03:16:55.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.366+0000 7f89dbfff640 1 -- 172.21.15.154:0/3602404639 learned_addr learned my addr 172.21.15.154:0/3602404639 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:55.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.366+0000 7f89dbfff640 1 -- 172.21.15.154:0/3602404639 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89dc15eb00 msgr2=0x7f89dc0d0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:55.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.366+0000 7f89db7fe640 1 --2- 172.21.15.154:0/3602404639 >> v2:172.21.15.190:3300/0 conn(0x7f89dc0cf370 0x7f89dc15e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:55.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.366+0000 7f89e0d7f640 1 --2- 172.21.15.154:0/3602404639 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89dc15eb00 0x7f89dc0d0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:55.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.366+0000 7f89dbfff640 1 --2- 172.21.15.154:0/3602404639 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89dc15eb00 0x7f89dc0d0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:55.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.366+0000 7f89dbfff640 1 -- 172.21.15.154:0/3602404639 >> v2:172.21.15.190:3300/0 conn(0x7f89dc0cf370 msgr2=0x7f89dc15e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:55.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.366+0000 7f89dbfff640 1 --2- 172.21.15.154:0/3602404639 >> v2:172.21.15.190:3300/0 conn(0x7f89dc0cf370 0x7f89dc15e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:55.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.366+0000 7f89dbfff640 1 -- 172.21.15.154:0/3602404639 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89dc15ef30 con 0x7f89dc15ab20 2023-09-23T03:16:55.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.366+0000 7f89db7fe640 1 --2- 172.21.15.154:0/3602404639 >> v2:172.21.15.190:3300/0 conn(0x7f89dc0cf370 0x7f89dc15e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:55.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.366+0000 7f89e0d7f640 1 --2- 172.21.15.154:0/3602404639 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89dc15eb00 0x7f89dc0d0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:55.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.367+0000 7f89dbfff640 1 --2- 172.21.15.154:0/3602404639 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89dc15ab20 0x7f89dc0cee30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f89dc06e090 tx=0x7f89cc02bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d36fdcd85c5c0974 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:55.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.368+0000 7f89daffd640 1 -- 172.21.15.154:0/3602404639 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89cc02cac0 con 0x7f89dc15ab20 2023-09-23T03:16:55.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.368+0000 7f89daffd640 1 -- 172.21.15.154:0/3602404639 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89cc02cc20 con 0x7f89dc15ab20 2023-09-23T03:16:55.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.368+0000 7f89daffd640 1 -- 172.21.15.154:0/3602404639 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89cc0343f0 con 0x7f89dc15ab20 2023-09-23T03:16:55.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.368+0000 7f89e27e2640 1 -- 172.21.15.154:0/3602404639 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89dc15ab20 msgr2=0x7f89dc0cee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:55.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.368+0000 7f89e27e2640 1 --2- 172.21.15.154:0/3602404639 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89dc15ab20 0x7f89dc0cee30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f89dc06e090 tx=0x7f89cc02bf20 comp rx=0 tx=0).stop 2023-09-23T03:16:55.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.368+0000 7f89e27e2640 1 -- 172.21.15.154:0/3602404639 shutdown_connections 2023-09-23T03:16:55.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.368+0000 7f89e27e2640 1 --2- 172.21.15.154:0/3602404639 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89dc15eb00 0x7f89dc0d0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:55.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.368+0000 7f89e27e2640 1 --2- 172.21.15.154:0/3602404639 >> v2:172.21.15.190:3300/0 conn(0x7f89dc0cf370 0x7f89dc15e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:55.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.368+0000 7f89e27e2640 1 --2- 172.21.15.154:0/3602404639 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89dc15ab20 0x7f89dc0cee30 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:55.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.368+0000 7f89e27e2640 1 -- 172.21.15.154:0/3602404639 >> 172.21.15.154:0/3602404639 conn(0x7f89dc089fd0 msgr2=0x7f89dc08a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:55.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.369+0000 7f89e27e2640 1 -- 172.21.15.154:0/3602404639 shutdown_connections 2023-09-23T03:16:55.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.369+0000 7f89e27e2640 1 -- 172.21.15.154:0/3602404639 wait complete. 2023-09-23T03:16:55.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.370+0000 7f89e27e2640 1 Processor -- start 2023-09-23T03:16:55.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.370+0000 7f89e27e2640 1 -- start start 2023-09-23T03:16:55.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.371+0000 7f89e27e2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89dc15ab20 0x7f89dc15d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:55.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.371+0000 7f89e27e2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89dc15eb00 0x7f89dc14ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:55.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.371+0000 7f89e27e2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89dc06ead0 0x7f89dc15b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:55.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.371+0000 7f89e27e2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89dc0c4210 con 0x7f89dc06ead0 2023-09-23T03:16:55.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.371+0000 7f89e27e2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89dc0c44f0 con 0x7f89dc15ab20 2023-09-23T03:16:55.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.371+0000 7f89e27e2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89dc15d960 con 0x7f89dc15eb00 2023-09-23T03:16:55.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.372+0000 7f89e0d7f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89dc06ead0 0x7f89dc15b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:55.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.372+0000 7f89e0d7f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89dc06ead0 0x7f89dc15b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:55896/0 (socket says 172.21.15.154:55896) 2023-09-23T03:16:55.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.372+0000 7f89e0d7f640 1 -- 172.21.15.154:0/2947258396 learned_addr learned my addr 172.21.15.154:0/2947258396 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:55.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.372+0000 7f89db7fe640 1 --2- 172.21.15.154:0/2947258396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89dc15eb00 0x7f89dc14ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:55.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.372+0000 7f89e0d7f640 1 -- 172.21.15.154:0/2947258396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89dc15eb00 msgr2=0x7f89dc14ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:55.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.372+0000 7f89dbfff640 1 --2- 172.21.15.154:0/2947258396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89dc15ab20 0x7f89dc15d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:55.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.372+0000 7f89e0d7f640 1 --2- 172.21.15.154:0/2947258396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89dc15eb00 0x7f89dc14ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:55.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.372+0000 7f89e0d7f640 1 -- 172.21.15.154:0/2947258396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89dc15ab20 msgr2=0x7f89dc15d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:55.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.372+0000 7f89e0d7f640 1 --2- 172.21.15.154:0/2947258396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89dc15ab20 0x7f89dc15d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:55.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.372+0000 7f89e0d7f640 1 -- 172.21.15.154:0/2947258396 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89cc009b30 con 0x7f89dc06ead0 2023-09-23T03:16:55.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.372+0000 7f89e0d7f640 1 --2- 172.21.15.154:0/2947258396 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89dc06ead0 0x7f89dc15b0a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f89d400c7f0 tx=0x7f89d400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:55.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.373+0000 7f89d8ff9640 1 -- 172.21.15.154:0/2947258396 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d4007820 con 0x7f89dc06ead0 2023-09-23T03:16:55.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.373+0000 7f89d8ff9640 1 -- 172.21.15.154:0/2947258396 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d4007980 con 0x7f89dc06ead0 2023-09-23T03:16:55.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.373+0000 7f89d8ff9640 1 -- 172.21.15.154:0/2947258396 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d4007d70 con 0x7f89dc06ead0 2023-09-23T03:16:55.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.373+0000 7f89e27e2640 1 -- 172.21.15.154:0/2947258396 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89dc15dc40 con 0x7f89dc06ead0 2023-09-23T03:16:55.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.373+0000 7f89e27e2640 1 -- 172.21.15.154:0/2947258396 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89dc15d670 con 0x7f89dc06ead0 2023-09-23T03:16:55.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.374+0000 7f89e27e2640 1 -- 172.21.15.154:0/2947258396 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89dc0c4660 con 0x7f89dc06ead0 2023-09-23T03:16:55.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.375+0000 7f89d8ff9640 1 -- 172.21.15.154:0/2947258396 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f89d40113f0 con 0x7f89dc06ead0 2023-09-23T03:16:55.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.376+0000 7f89d8ff9640 1 --2- 172.21.15.154:0/2947258396 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f89bc06af30 0x7f89bc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:55.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.376+0000 7f89d8ff9640 1 -- 172.21.15.154:0/2947258396 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f89d4089e80 con 0x7f89dc06ead0 2023-09-23T03:16:55.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.376+0000 7f89dbfff640 1 --2- 172.21.15.154:0/2947258396 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f89bc06af30 0x7f89bc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:55.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.377+0000 7f89dbfff640 1 --2- 172.21.15.154:0/2947258396 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f89bc06af30 0x7f89bc06d3f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f89cc035000 tx=0x7f89cc030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:55.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.380+0000 7f89d8ff9640 1 -- 172.21.15.154:0/2947258396 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f89d4054a70 con 0x7f89dc06ead0 2023-09-23T03:16:55.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.572+0000 7f89e27e2640 1 -- 172.21.15.154:0/2947258396 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f89dc04e920 con 0x7f89dc06ead0 2023-09-23T03:16:55.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.573+0000 7f89d8ff9640 1 -- 172.21.15.154:0/2947258396 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f89d401f0a0 con 0x7f89dc06ead0 2023-09-23T03:16:55.573 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:16:55.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.576+0000 7f89e27e2640 1 -- 172.21.15.154:0/2947258396 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f89bc06af30 msgr2=0x7f89bc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:55.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.576+0000 7f89e27e2640 1 --2- 172.21.15.154:0/2947258396 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f89bc06af30 0x7f89bc06d3f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f89cc035000 tx=0x7f89cc030000 comp rx=0 tx=0).stop 2023-09-23T03:16:55.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.576+0000 7f89e27e2640 1 -- 172.21.15.154:0/2947258396 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89dc06ead0 msgr2=0x7f89dc15b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:55.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.576+0000 7f89e27e2640 1 --2- 172.21.15.154:0/2947258396 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89dc06ead0 0x7f89dc15b0a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f89d400c7f0 tx=0x7f89d400ccb0 comp rx=0 tx=0).stop 2023-09-23T03:16:55.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.576+0000 7f89e27e2640 1 -- 172.21.15.154:0/2947258396 shutdown_connections 2023-09-23T03:16:55.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.576+0000 7f89e27e2640 1 --2- 172.21.15.154:0/2947258396 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f89bc06af30 0x7f89bc06d3f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:55.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.576+0000 7f89e27e2640 1 --2- 172.21.15.154:0/2947258396 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89dc06ead0 0x7f89dc15b0a0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:55.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.576+0000 7f89e27e2640 1 --2- 172.21.15.154:0/2947258396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89dc15eb00 0x7f89dc14ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:55.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.576+0000 7f89e27e2640 1 --2- 172.21.15.154:0/2947258396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89dc15ab20 0x7f89dc15d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:55.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.576+0000 7f89e27e2640 1 -- 172.21.15.154:0/2947258396 >> 172.21.15.154:0/2947258396 conn(0x7f89dc089fd0 msgr2=0x7f89dc0b1740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:55.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.576+0000 7f89e27e2640 1 -- 172.21.15.154:0/2947258396 shutdown_connections 2023-09-23T03:16:55.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.576+0000 7f89e27e2640 1 -- 172.21.15.154:0/2947258396 wait complete. 2023-09-23T03:16:55.596 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:16:55.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.809+0000 7fca79256640 1 Processor -- start 2023-09-23T03:16:55.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.809+0000 7fca79256640 1 -- start start 2023-09-23T03:16:55.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.810+0000 7fca79256640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca74161010 0x7fca740cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:55.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.811+0000 7fca79256640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca740cb160 con 0x7fca7406c8e0 2023-09-23T03:16:55.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.811+0000 7fca79256640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca740ccd20 con 0x7fca740c14e0 2023-09-23T03:16:55.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.811+0000 7fca79256640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca7406cde0 con 0x7fca74161010 2023-09-23T03:16:55.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.811+0000 7fca7359e640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fca7406c8e0 0x7fca740b1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:54824/0 (socket says 172.21.15.154:54824) 2023-09-23T03:16:55.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.811+0000 7fca7359e640 1 -- 172.21.15.154:0/1325260280 learned_addr learned my addr 172.21.15.154:0/1325260280 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:55.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.811+0000 7fca72d9d640 1 --2- 172.21.15.154:0/1325260280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca74161010 0x7fca740cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:55.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.811+0000 7fca7259c640 1 -- 172.21.15.154:0/1325260280 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3817501753 0 0) 0x7fca740cb160 con 0x7fca7406c8e0 2023-09-23T03:16:55.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.811+0000 7fca7259c640 1 -- 172.21.15.154:0/1325260280 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca60003520 con 0x7fca7406c8e0 2023-09-23T03:16:55.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.812+0000 7fca7259c640 1 -- 172.21.15.154:0/1325260280 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 535523771 0 0) 0x7fca60003520 con 0x7fca7406c8e0 2023-09-23T03:16:55.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.812+0000 7fca7259c640 1 -- 172.21.15.154:0/1325260280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca74161010 msgr2=0x7fca740cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:55.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.812+0000 7fca7259c640 1 --2- 172.21.15.154:0/1325260280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca74161010 0x7fca740cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:55.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.812+0000 7fca7259c640 1 -- 172.21.15.154:0/1325260280 >> v1:172.21.15.190:6789/0 conn(0x7fca740c14e0 legacy=0x7fca7406c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:55.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.812+0000 7fca7259c640 1 -- 172.21.15.154:0/1325260280 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca7406d0c0 con 0x7fca7406c8e0 2023-09-23T03:16:55.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.812+0000 7fca7259c640 1 -- 172.21.15.154:0/1325260280 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fca6c0019f0 con 0x7fca7406c8e0 2023-09-23T03:16:55.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.812+0000 7fca7259c640 1 -- 172.21.15.154:0/1325260280 >> v1:172.21.15.154:6789/0 conn(0x7fca7406c8e0 legacy=0x7fca740b1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:55.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.812+0000 7fca7259c640 1 --2- 172.21.15.154:0/1325260280 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca600039a0 0x7fca60003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:55.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.812+0000 7fca7259c640 1 --2- 172.21.15.154:0/1325260280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca74161010 0x7fca600047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:55.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.812+0000 7fca7259c640 1 --2- 172.21.15.154:0/1325260280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca60004d00 0x7fca60005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:55.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.812+0000 7fca7259c640 1 -- 172.21.15.154:0/1325260280 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7406cde0 con 0x7fca600039a0 2023-09-23T03:16:55.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.812+0000 7fca7259c640 1 -- 172.21.15.154:0/1325260280 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca6000bb10 con 0x7fca60004d00 2023-09-23T03:16:55.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.812+0000 7fca7259c640 1 -- 172.21.15.154:0/1325260280 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca6000bcd0 con 0x7fca74161010 2023-09-23T03:16:55.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.812+0000 7fca72d9d640 1 --2- 172.21.15.154:0/1325260280 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca600039a0 0x7fca60003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:55.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.813+0000 7fca72d9d640 1 -- 172.21.15.154:0/1325260280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca74161010 msgr2=0x7fca600047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:55.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.813+0000 7fca72d9d640 1 --2- 172.21.15.154:0/1325260280 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca74161010 0x7fca600047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:55.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.813+0000 7fca72d9d640 1 -- 172.21.15.154:0/1325260280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca60004d00 msgr2=0x7fca60005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:55.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.813+0000 7fca72d9d640 1 --2- 172.21.15.154:0/1325260280 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca60004d00 0x7fca60005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:55.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.813+0000 7fca72d9d640 1 -- 172.21.15.154:0/1325260280 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca60003520 con 0x7fca600039a0 2023-09-23T03:16:55.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.814+0000 7fca72d9d640 1 --2- 172.21.15.154:0/1325260280 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca600039a0 0x7fca60003dc0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fca64009ab0 tx=0x7fca600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ee092bed8028989 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:55.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.814+0000 7fca7259c640 1 -- 172.21.15.154:0/1325260280 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca64005990 con 0x7fca600039a0 2023-09-23T03:16:55.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.814+0000 7fca7259c640 1 -- 172.21.15.154:0/1325260280 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca64005af0 con 0x7fca600039a0 2023-09-23T03:16:55.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.814+0000 7fca7259c640 1 -- 172.21.15.154:0/1325260280 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca64005ea0 con 0x7fca600039a0 2023-09-23T03:16:55.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.814+0000 7fca79256640 1 -- 172.21.15.154:0/1325260280 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca600039a0 msgr2=0x7fca60003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:55.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.814+0000 7fca79256640 1 --2- 172.21.15.154:0/1325260280 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca600039a0 0x7fca60003dc0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fca64009ab0 tx=0x7fca600057f0 comp rx=0 tx=0).stop 2023-09-23T03:16:55.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.814+0000 7fca7359e640 1 -- 172.21.15.154:0/1325260280 reap_dead start 2023-09-23T03:16:55.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.815+0000 7fca79256640 1 -- 172.21.15.154:0/1325260280 shutdown_connections 2023-09-23T03:16:55.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.815+0000 7fca79256640 1 -- 172.21.15.154:0/1325260280 >> 172.21.15.154:0/1325260280 conn(0x7fca7406a0f0 msgr2=0x7fca7406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:55.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.815+0000 7fca79256640 1 -- 172.21.15.154:0/1325260280 shutdown_connections 2023-09-23T03:16:55.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.815+0000 7fca79256640 1 -- 172.21.15.154:0/1325260280 wait complete. 2023-09-23T03:16:55.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.816+0000 7fca79256640 1 Processor -- start 2023-09-23T03:16:55.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.817+0000 7fca79256640 1 -- start start 2023-09-23T03:16:55.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.817+0000 7fca79256640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca600039a0 0x7fca740adf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:55.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.817+0000 7fca79256640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca740acb20 0x7fca740b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:55.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.818+0000 7fca79256640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca740c7ba0 0x7fca740c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:55.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.818+0000 7fca79256640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca740acfa0 con 0x7fca740c7ba0 2023-09-23T03:16:55.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.818+0000 7fca79256640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca740c8560 con 0x7fca740acb20 2023-09-23T03:16:55.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.818+0000 7fca79256640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca740aecb0 con 0x7fca600039a0 2023-09-23T03:16:55.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.818+0000 7fca7359e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca740c7ba0 0x7fca740c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:55.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.818+0000 7fca7359e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca740c7ba0 0x7fca740c8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:55912/0 (socket says 172.21.15.154:55912) 2023-09-23T03:16:55.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.818+0000 7fca7359e640 1 -- 172.21.15.154:0/1133012623 learned_addr learned my addr 172.21.15.154:0/1133012623 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:55.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.818+0000 7fca6bfff640 1 --2- 172.21.15.154:0/1133012623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca740acb20 0x7fca740b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:55.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.818+0000 7fca72d9d640 1 --2- 172.21.15.154:0/1133012623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca600039a0 0x7fca740adf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:55.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.818+0000 7fca7359e640 1 -- 172.21.15.154:0/1133012623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca600039a0 msgr2=0x7fca740adf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:55.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.818+0000 7fca7359e640 1 --2- 172.21.15.154:0/1133012623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca600039a0 0x7fca740adf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:55.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.818+0000 7fca7359e640 1 -- 172.21.15.154:0/1133012623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca740acb20 msgr2=0x7fca740b08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:55.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.818+0000 7fca7359e640 1 --2- 172.21.15.154:0/1133012623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca740acb20 0x7fca740b08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:55.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.818+0000 7fca7359e640 1 -- 172.21.15.154:0/1133012623 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca740aef90 con 0x7fca740c7ba0 2023-09-23T03:16:55.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.818+0000 7fca72d9d640 1 --2- 172.21.15.154:0/1133012623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca600039a0 0x7fca740adf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:55.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.818+0000 7fca6bfff640 1 --2- 172.21.15.154:0/1133012623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca740acb20 0x7fca740b08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:55.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.819+0000 7fca7359e640 1 --2- 172.21.15.154:0/1133012623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca740c7ba0 0x7fca740c8020 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fca740cecc0 tx=0x7fca6c00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:55.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.819+0000 7fca6b7fe640 1 -- 172.21.15.154:0/1133012623 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6c00a610 con 0x7fca740c7ba0 2023-09-23T03:16:55.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.819+0000 7fca6b7fe640 1 -- 172.21.15.154:0/1133012623 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca6c00a790 con 0x7fca740c7ba0 2023-09-23T03:16:55.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.819+0000 7fca6b7fe640 1 -- 172.21.15.154:0/1133012623 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6c00ab40 con 0x7fca740c7ba0 2023-09-23T03:16:55.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.819+0000 7fca79256640 1 -- 172.21.15.154:0/1133012623 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca740ac040 con 0x7fca740c7ba0 2023-09-23T03:16:55.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.819+0000 7fca79256640 1 -- 172.21.15.154:0/1133012623 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca740ac600 con 0x7fca740c7ba0 2023-09-23T03:16:55.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.820+0000 7fca79256640 1 -- 172.21.15.154:0/1133012623 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca740ac820 con 0x7fca740c7ba0 2023-09-23T03:16:55.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.821+0000 7fca6b7fe640 1 -- 172.21.15.154:0/1133012623 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fca6c00aca0 con 0x7fca740c7ba0 2023-09-23T03:16:55.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.822+0000 7fca6b7fe640 1 --2- 172.21.15.154:0/1133012623 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fca5406af30 0x7fca5406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:55.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.822+0000 7fca6b7fe640 1 -- 172.21.15.154:0/1133012623 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7fca6c089430 con 0x7fca740c7ba0 2023-09-23T03:16:55.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.822+0000 7fca72d9d640 1 --2- 172.21.15.154:0/1133012623 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fca5406af30 0x7fca5406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:55.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.823+0000 7fca72d9d640 1 --2- 172.21.15.154:0/1133012623 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fca5406af30 0x7fca5406d3f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fca64035000 tx=0x7fca64030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:55.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:55.826+0000 7fca6b7fe640 1 -- 172.21.15.154:0/1133012623 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fca6c054020 con 0x7fca740c7ba0 2023-09-23T03:16:56.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.052+0000 7fca79256640 1 -- 172.21.15.154:0/1133012623 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca74073eb0 con 0x7fca740c7ba0 2023-09-23T03:16:56.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.053+0000 7fca6b7fe640 1 -- 172.21.15.154:0/1133012623 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4921 (secure 0 0 0) 0x7fca6c057640 con 0x7fca740c7ba0 2023-09-23T03:16:56.053 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:16:56.053 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2644365591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2644365591},{"type":"v1","addr":"172.21.15.190:6835","nonce":2644365591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/4145467852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4145467852},{"type":"v1","addr":"172.21.15.154:6835","nonce":4145467852}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1022302774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1022302774},{"type":"v1","addr":"172.21.15.154:6837","nonce":1022302774}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:15:20.686035+0000","modified":"2023-09-23T03:15:48.742318+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/3561509423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3561509423},{"type":"v1","addr":"172.21.15.190:6834","nonce":3561509423}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T03:16:56.053 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 9 2023-09-23T03:16:56.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.056+0000 7fca79256640 1 -- 172.21.15.154:0/1133012623 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fca5406af30 msgr2=0x7fca5406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:56.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.056+0000 7fca79256640 1 --2- 172.21.15.154:0/1133012623 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fca5406af30 0x7fca5406d3f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fca64035000 tx=0x7fca64030000 comp rx=0 tx=0).stop 2023-09-23T03:16:56.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.056+0000 7fca79256640 1 -- 172.21.15.154:0/1133012623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca740c7ba0 msgr2=0x7fca740c8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:56.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.056+0000 7fca79256640 1 --2- 172.21.15.154:0/1133012623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca740c7ba0 0x7fca740c8020 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fca740cecc0 tx=0x7fca6c00ccd0 comp rx=0 tx=0).stop 2023-09-23T03:16:56.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.056+0000 7fca79256640 1 -- 172.21.15.154:0/1133012623 shutdown_connections 2023-09-23T03:16:56.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.056+0000 7fca79256640 1 --2- 172.21.15.154:0/1133012623 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fca5406af30 0x7fca5406d3f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:56.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.056+0000 7fca79256640 1 --2- 172.21.15.154:0/1133012623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca740c7ba0 0x7fca740c8020 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:56.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.056+0000 7fca79256640 1 --2- 172.21.15.154:0/1133012623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fca740acb20 0x7fca740b08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:56.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.056+0000 7fca79256640 1 --2- 172.21.15.154:0/1133012623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fca600039a0 0x7fca740adf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:56.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.056+0000 7fca79256640 1 -- 172.21.15.154:0/1133012623 >> 172.21.15.154:0/1133012623 conn(0x7fca7406a0f0 msgr2=0x7fca7406d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:56.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.056+0000 7fca79256640 1 -- 172.21.15.154:0/1133012623 shutdown_connections 2023-09-23T03:16:56.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.056+0000 7fca79256640 1 -- 172.21.15.154:0/1133012623 wait complete. 2023-09-23T03:16:56.075 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:16:56.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.298+0000 7f6d08e90640 1 Processor -- start 2023-09-23T03:16:56.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.298+0000 7f6d08e90640 1 -- start start 2023-09-23T03:16:56.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.300+0000 7f6d08e90640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6d040b1400 0x7f6d040ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:56.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.300+0000 7f6d08e90640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6d0415cf80 0x7f6d0406c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:56.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.300+0000 7f6d08e90640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d040ce8c0 con 0x7f6d040cdfa0 2023-09-23T03:16:56.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.300+0000 7f6d08e90640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d0415d550 con 0x7f6d040b1400 2023-09-23T03:16:56.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.300+0000 7f6d08e90640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d0415d830 con 0x7f6d0415cf80 2023-09-23T03:16:56.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.300+0000 7f6d01d9b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6d040b1400 0x7f6d040ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:56.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.300+0000 7f6d01d9b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6d040b1400 0x7f6d040ce380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:55928/0 (socket says 172.21.15.154:55928) 2023-09-23T03:16:56.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.300+0000 7f6d01d9b640 1 -- 172.21.15.154:0/3056492333 learned_addr learned my addr 172.21.15.154:0/3056492333 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:56.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.301+0000 7f6d01d9b640 1 -- 172.21.15.154:0/3056492333 >> v2:172.21.15.190:3300/0 conn(0x7f6d0415cf80 msgr2=0x7f6d0406c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:56.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.301+0000 7f6d01d9b640 1 --2- 172.21.15.154:0/3056492333 >> v2:172.21.15.190:3300/0 conn(0x7f6d0415cf80 0x7f6d0406c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:56.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.301+0000 7f6d01d9b640 1 -- 172.21.15.154:0/3056492333 >> v1:172.21.15.190:6789/0 conn(0x7f6d040cdfa0 legacy=0x7f6d040659f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:56.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.301+0000 7f6d01d9b640 1 -- 172.21.15.154:0/3056492333 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d0415db10 con 0x7f6d040b1400 2023-09-23T03:16:56.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.302+0000 7f6d01d9b640 1 --2- 172.21.15.154:0/3056492333 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d040b1400 0x7f6d040ce380 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6d040707c0 tx=0x7f6cec02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=680ee7a7b4db5fe6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:56.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.302+0000 7f6d0159a640 1 -- 172.21.15.154:0/3056492333 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cec031750 con 0x7f6d040b1400 2023-09-23T03:16:56.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.302+0000 7f6d0159a640 1 -- 172.21.15.154:0/3056492333 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cec0318b0 con 0x7f6d040b1400 2023-09-23T03:16:56.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.302+0000 7f6d0159a640 1 -- 172.21.15.154:0/3056492333 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cec031c40 con 0x7f6d040b1400 2023-09-23T03:16:56.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.303+0000 7f6d08e90640 1 -- 172.21.15.154:0/3056492333 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d040b1400 msgr2=0x7f6d040ce380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:56.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.303+0000 7f6d08e90640 1 --2- 172.21.15.154:0/3056492333 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d040b1400 0x7f6d040ce380 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6d040707c0 tx=0x7f6cec02b7f0 comp rx=0 tx=0).stop 2023-09-23T03:16:56.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.303+0000 7f6d08e90640 1 -- 172.21.15.154:0/3056492333 shutdown_connections 2023-09-23T03:16:56.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.303+0000 7f6d08e90640 1 --2- 172.21.15.154:0/3056492333 >> v2:172.21.15.190:3300/0 conn(0x7f6d0415cf80 0x7f6d0406c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:56.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.303+0000 7f6d08e90640 1 --2- 172.21.15.154:0/3056492333 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d040b1400 0x7f6d040ce380 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:56.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.303+0000 7f6d08e90640 1 -- 172.21.15.154:0/3056492333 >> 172.21.15.154:0/3056492333 conn(0x7f6d040b08e0 msgr2=0x7f6d040b0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:56.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.303+0000 7f6d08e90640 1 -- 172.21.15.154:0/3056492333 shutdown_connections 2023-09-23T03:16:56.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.303+0000 7f6d08e90640 1 -- 172.21.15.154:0/3056492333 wait complete. 2023-09-23T03:16:56.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.305+0000 7f6d08e90640 1 Processor -- start 2023-09-23T03:16:56.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.305+0000 7f6d08e90640 1 -- start start 2023-09-23T03:16:56.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.306+0000 7f6d08e90640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d040b1400 0x7f6d040c4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:56.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.306+0000 7f6d08e90640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d040cdfa0 0x7f6d04152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:56.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.306+0000 7f6d08e90640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d0415cf80 0x7f6d041f5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:56.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.306+0000 7f6d08e90640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d040c4920 con 0x7f6d040b1400 2023-09-23T03:16:56.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.306+0000 7f6d08e90640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d040c4c00 con 0x7f6d040cdfa0 2023-09-23T03:16:56.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.306+0000 7f6d08e90640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d040c4ee0 con 0x7f6d0415cf80 2023-09-23T03:16:56.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.306+0000 7f6d0259c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d040b1400 0x7f6d040c4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:56.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.306+0000 7f6d0259c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d040b1400 0x7f6d040c4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:55944/0 (socket says 172.21.15.154:55944) 2023-09-23T03:16:56.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.306+0000 7f6d0259c640 1 -- 172.21.15.154:0/1661887204 learned_addr learned my addr 172.21.15.154:0/1661887204 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:56.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.307+0000 7f6d01d9b640 1 --2- 172.21.15.154:0/1661887204 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d040cdfa0 0x7f6d04152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:56.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.307+0000 7f6d0259c640 1 -- 172.21.15.154:0/1661887204 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d0415cf80 msgr2=0x7f6d041f5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:56.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.307+0000 7f6d0259c640 1 --2- 172.21.15.154:0/1661887204 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d0415cf80 0x7f6d041f5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:56.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.307+0000 7f6d0259c640 1 -- 172.21.15.154:0/1661887204 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d040cdfa0 msgr2=0x7f6d04152640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:56.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.307+0000 7f6d0259c640 1 --2- 172.21.15.154:0/1661887204 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d040cdfa0 0x7f6d04152640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:56.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.307+0000 7f6d0259c640 1 -- 172.21.15.154:0/1661887204 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cec0094d0 con 0x7f6d040b1400 2023-09-23T03:16:56.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.307+0000 7f6d01d9b640 1 --2- 172.21.15.154:0/1661887204 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d040cdfa0 0x7f6d04152640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:56.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.307+0000 7f6d0259c640 1 --2- 172.21.15.154:0/1661887204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d040b1400 0x7f6d040c4340 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6cf400c7a0 tx=0x7f6cf400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:56.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.307+0000 7f6cfaffd640 1 -- 172.21.15.154:0/1661887204 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4003d80 con 0x7f6d040b1400 2023-09-23T03:16:56.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.307+0000 7f6cfaffd640 1 -- 172.21.15.154:0/1661887204 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cf4005070 con 0x7f6d040b1400 2023-09-23T03:16:56.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.307+0000 7f6cfaffd640 1 -- 172.21.15.154:0/1661887204 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4005460 con 0x7f6d040b1400 2023-09-23T03:16:56.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.307+0000 7f6d08e90640 1 -- 172.21.15.154:0/1661887204 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d040c51c0 con 0x7f6d040b1400 2023-09-23T03:16:56.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.307+0000 7f6d08e90640 1 -- 172.21.15.154:0/1661887204 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d04152b80 con 0x7f6d040b1400 2023-09-23T03:16:56.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.308+0000 7f6d08e90640 1 -- 172.21.15.154:0/1661887204 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d04152dd0 con 0x7f6d040b1400 2023-09-23T03:16:56.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.309+0000 7f6cfaffd640 1 -- 172.21.15.154:0/1661887204 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f6cf40055c0 con 0x7f6d040b1400 2023-09-23T03:16:56.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.310+0000 7f6cfaffd640 1 --2- 172.21.15.154:0/1661887204 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6ce406af30 0x7f6ce406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:56.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.311+0000 7f6cfaffd640 1 -- 172.21.15.154:0/1661887204 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f6cf4088fd0 con 0x7f6d040b1400 2023-09-23T03:16:56.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.311+0000 7f6d01d9b640 1 --2- 172.21.15.154:0/1661887204 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6ce406af30 0x7f6ce406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:56.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.312+0000 7f6d01d9b640 1 --2- 172.21.15.154:0/1661887204 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6ce406af30 0x7f6ce406d3f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6cec035000 tx=0x7f6cec008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:56.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.315+0000 7f6cfaffd640 1 -- 172.21.15.154:0/1661887204 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f6cf4053c50 con 0x7f6d040b1400 2023-09-23T03:16:56.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.506+0000 7f6d08e90640 1 -- 172.21.15.154:0/1661887204 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6d04073eb0 con 0x7f6d040b1400 2023-09-23T03:16:56.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.508+0000 7f6cfaffd640 1 -- 172.21.15.154:0/1661887204 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18744 (secure 0 0 0) 0x7f6cf4057270 con 0x7f6d040b1400 2023-09-23T03:16:56.509 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:16:56.509 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":26,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:16:03.151371+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:15:18.575849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T03:15:19.707212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T03:15:23.337138+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:16:56.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.511+0000 7f6d08e90640 1 -- 172.21.15.154:0/1661887204 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6ce406af30 msgr2=0x7f6ce406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:56.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.511+0000 7f6d08e90640 1 --2- 172.21.15.154:0/1661887204 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6ce406af30 0x7f6ce406d3f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6cec035000 tx=0x7f6cec008030 comp rx=0 tx=0).stop 2023-09-23T03:16:56.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.511+0000 7f6d08e90640 1 -- 172.21.15.154:0/1661887204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d040b1400 msgr2=0x7f6d040c4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:56.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.511+0000 7f6d08e90640 1 --2- 172.21.15.154:0/1661887204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d040b1400 0x7f6d040c4340 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6cf400c7a0 tx=0x7f6cf400cc60 comp rx=0 tx=0).stop 2023-09-23T03:16:56.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.511+0000 7f6d08e90640 1 -- 172.21.15.154:0/1661887204 shutdown_connections 2023-09-23T03:16:56.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.511+0000 7f6d08e90640 1 --2- 172.21.15.154:0/1661887204 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6ce406af30 0x7f6ce406d3f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:56.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.511+0000 7f6d08e90640 1 --2- 172.21.15.154:0/1661887204 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6d0415cf80 0x7f6d041f5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:56.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.511+0000 7f6d08e90640 1 --2- 172.21.15.154:0/1661887204 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6d040cdfa0 0x7f6d04152640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:56.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.511+0000 7f6d08e90640 1 --2- 172.21.15.154:0/1661887204 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6d040b1400 0x7f6d040c4340 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:56.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.511+0000 7f6d08e90640 1 -- 172.21.15.154:0/1661887204 >> 172.21.15.154:0/1661887204 conn(0x7f6d040b08e0 msgr2=0x7f6d04070810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:56.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.511+0000 7f6d08e90640 1 -- 172.21.15.154:0/1661887204 shutdown_connections 2023-09-23T03:16:56.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.511+0000 7f6d08e90640 1 -- 172.21.15.154:0/1661887204 wait complete. 2023-09-23T03:16:56.529 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-09-23T03:16:56.529 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:16:56.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.746+0000 7f810ffff640 1 Processor -- start 2023-09-23T03:16:56.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.746+0000 7f810ffff640 1 -- start start 2023-09-23T03:16:56.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.747+0000 7f810ffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f811006c7a0 0x7f811015eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:56.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.747+0000 7f810ffff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f811015ab00 0x7f811015afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:56.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.747+0000 7f810ffff640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f811006bff0 con 0x7f811006fdc0 2023-09-23T03:16:56.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.747+0000 7f810ffff640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8110155070 con 0x7f811006c7a0 2023-09-23T03:16:56.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.747+0000 7f810ffff640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f811015e0f0 con 0x7f811015ab00 2023-09-23T03:16:56.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.748+0000 7f810effd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f811006c7a0 0x7f811015eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:56.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.748+0000 7f810effd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f811006c7a0 0x7f811015eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:55960/0 (socket says 172.21.15.154:55960) 2023-09-23T03:16:56.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.748+0000 7f810effd640 1 -- 172.21.15.154:0/2564348359 learned_addr learned my addr 172.21.15.154:0/2564348359 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:56.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.748+0000 7f810effd640 1 -- 172.21.15.154:0/2564348359 >> v2:172.21.15.190:3300/0 conn(0x7f811015ab00 msgr2=0x7f811015afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:56.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.748+0000 7f810effd640 1 --2- 172.21.15.154:0/2564348359 >> v2:172.21.15.190:3300/0 conn(0x7f811015ab00 0x7f811015afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:56.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.748+0000 7f810effd640 1 -- 172.21.15.154:0/2564348359 >> v1:172.21.15.190:6789/0 conn(0x7f811006fdc0 legacy=0x7f81100701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:56.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.748+0000 7f810effd640 1 -- 172.21.15.154:0/2564348359 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f811006cb80 con 0x7f811006c7a0 2023-09-23T03:16:56.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.749+0000 7f810effd640 1 --2- 172.21.15.154:0/2564348359 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f811006c7a0 0x7f811015eb00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f81100cba90 tx=0x7f810002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b5d8576848bd8c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:56.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.749+0000 7f810dffb640 1 -- 172.21.15.154:0/2564348359 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810002ca80 con 0x7f811006c7a0 2023-09-23T03:16:56.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.750+0000 7f810dffb640 1 -- 172.21.15.154:0/2564348359 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f810002cbe0 con 0x7f811006c7a0 2023-09-23T03:16:56.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.750+0000 7f810dffb640 1 -- 172.21.15.154:0/2564348359 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81000343f0 con 0x7f811006c7a0 2023-09-23T03:16:56.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.750+0000 7f810ffff640 1 -- 172.21.15.154:0/2564348359 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f811006c7a0 msgr2=0x7f811015eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:56.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.750+0000 7f810ffff640 1 --2- 172.21.15.154:0/2564348359 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f811006c7a0 0x7f811015eb00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f81100cba90 tx=0x7f810002bee0 comp rx=0 tx=0).stop 2023-09-23T03:16:56.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.750+0000 7f810ffff640 1 -- 172.21.15.154:0/2564348359 shutdown_connections 2023-09-23T03:16:56.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.750+0000 7f810ffff640 1 --2- 172.21.15.154:0/2564348359 >> v2:172.21.15.190:3300/0 conn(0x7f811015ab00 0x7f811015afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:56.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.750+0000 7f810ffff640 1 --2- 172.21.15.154:0/2564348359 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f811006c7a0 0x7f811015eb00 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:56.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.750+0000 7f810ffff640 1 -- 172.21.15.154:0/2564348359 >> 172.21.15.154:0/2564348359 conn(0x7f811006a390 msgr2=0x7f81100afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:56.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.750+0000 7f810ffff640 1 -- 172.21.15.154:0/2564348359 shutdown_connections 2023-09-23T03:16:56.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.750+0000 7f810ffff640 1 -- 172.21.15.154:0/2564348359 wait complete. 2023-09-23T03:16:56.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.752+0000 7f810ffff640 1 Processor -- start 2023-09-23T03:16:56.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.752+0000 7f810ffff640 1 -- start start 2023-09-23T03:16:56.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.753+0000 7f810ffff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f811006c7a0 0x7f81101517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:56.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.753+0000 7f810ffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f811006fdc0 0x7f81100b23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:56.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.753+0000 7f810ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f811015ab00 0x7f811014e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:56.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.753+0000 7f810ffff640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81100b2e70 con 0x7f811015ab00 2023-09-23T03:16:56.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.753+0000 7f810ffff640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81100b3150 con 0x7f811006fdc0 2023-09-23T03:16:56.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.753+0000 7f810ffff640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81100b3430 con 0x7f811006c7a0 2023-09-23T03:16:56.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.753+0000 7f810f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f811015ab00 0x7f811014e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:56.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.754+0000 7f810f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f811015ab00 0x7f811014e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:55976/0 (socket says 172.21.15.154:55976) 2023-09-23T03:16:56.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.754+0000 7f810f7fe640 1 -- 172.21.15.154:0/1001686271 learned_addr learned my addr 172.21.15.154:0/1001686271 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:56.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.754+0000 7f810effd640 1 --2- 172.21.15.154:0/1001686271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f811006c7a0 0x7f81101517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:56.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.754+0000 7f810f7fe640 1 -- 172.21.15.154:0/1001686271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f811006c7a0 msgr2=0x7f81101517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:56.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.754+0000 7f810f7fe640 1 --2- 172.21.15.154:0/1001686271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f811006c7a0 0x7f81101517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:56.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.754+0000 7f810f7fe640 1 -- 172.21.15.154:0/1001686271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f811006fdc0 msgr2=0x7f81100b23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:56.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.754+0000 7f810f7fe640 1 --2- 172.21.15.154:0/1001686271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f811006fdc0 0x7f81100b23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:56.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.754+0000 7f810f7fe640 1 -- 172.21.15.154:0/1001686271 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8100009b30 con 0x7f811015ab00 2023-09-23T03:16:56.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.754+0000 7f810effd640 1 --2- 172.21.15.154:0/1001686271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f811006c7a0 0x7f81101517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:56.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.754+0000 7f810f7fe640 1 --2- 172.21.15.154:0/1001686271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f811015ab00 0x7f811014e480 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f810800b490 tx=0x7f810800b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:56.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.754+0000 7f81077fe640 1 -- 172.21.15.154:0/1001686271 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8108011830 con 0x7f811015ab00 2023-09-23T03:16:56.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.755+0000 7f81077fe640 1 -- 172.21.15.154:0/1001686271 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81080119b0 con 0x7f811015ab00 2023-09-23T03:16:56.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.755+0000 7f81077fe640 1 -- 172.21.15.154:0/1001686271 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8108011d60 con 0x7f811015ab00 2023-09-23T03:16:56.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.755+0000 7f810ffff640 1 -- 172.21.15.154:0/1001686271 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81100b3710 con 0x7f811015ab00 2023-09-23T03:16:56.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.755+0000 7f810ffff640 1 -- 172.21.15.154:0/1001686271 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81100c4f50 con 0x7f811015ab00 2023-09-23T03:16:56.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.756+0000 7f810ffff640 1 -- 172.21.15.154:0/1001686271 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81100736b0 con 0x7f811015ab00 2023-09-23T03:16:56.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.757+0000 7f81077fe640 1 -- 172.21.15.154:0/1001686271 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f810801a400 con 0x7f811015ab00 2023-09-23T03:16:56.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.758+0000 7f81077fe640 1 --2- 172.21.15.154:0/1001686271 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f80f006af60 0x7f80f006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:56.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.758+0000 7f81077fe640 1 -- 172.21.15.154:0/1001686271 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f8108089b60 con 0x7f811015ab00 2023-09-23T03:16:56.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.758+0000 7f810effd640 1 --2- 172.21.15.154:0/1001686271 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f80f006af60 0x7f80f006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:56.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.759+0000 7f810effd640 1 --2- 172.21.15.154:0/1001686271 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f80f006af60 0x7f80f006d420 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8100035000 tx=0x7f8100030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:56.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.762+0000 7f81077fe640 1 -- 172.21.15.154:0/1001686271 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f81080547e0 con 0x7f811015ab00 2023-09-23T03:16:56.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.988+0000 7f810ffff640 1 -- 172.21.15.154:0/1001686271 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81100b3c30 con 0x7f811015ab00 2023-09-23T03:16:56.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.989+0000 7f81077fe640 1 -- 172.21.15.154:0/1001686271 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4921 (secure 0 0 0) 0x7f81100b3c30 con 0x7f811015ab00 2023-09-23T03:16:56.990 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:16:56.990 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2644365591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2644365591},{"type":"v1","addr":"172.21.15.190:6835","nonce":2644365591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/4145467852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4145467852},{"type":"v1","addr":"172.21.15.154:6835","nonce":4145467852}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1022302774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1022302774},{"type":"v1","addr":"172.21.15.154:6837","nonce":1022302774}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:15:20.686035+0000","modified":"2023-09-23T03:15:48.742318+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/3561509423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3561509423},{"type":"v1","addr":"172.21.15.190:6834","nonce":3561509423}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T03:16:56.990 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 9 2023-09-23T03:16:56.992 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.992+0000 7f810ffff640 1 -- 172.21.15.154:0/1001686271 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f80f006af60 msgr2=0x7f80f006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:56.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.992+0000 7f810ffff640 1 --2- 172.21.15.154:0/1001686271 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f80f006af60 0x7f80f006d420 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8100035000 tx=0x7f8100030000 comp rx=0 tx=0).stop 2023-09-23T03:16:56.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.992+0000 7f810ffff640 1 -- 172.21.15.154:0/1001686271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f811015ab00 msgr2=0x7f811014e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:56.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.992+0000 7f810ffff640 1 --2- 172.21.15.154:0/1001686271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f811015ab00 0x7f811014e480 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f810800b490 tx=0x7f810800b950 comp rx=0 tx=0).stop 2023-09-23T03:16:56.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.992+0000 7f810ffff640 1 -- 172.21.15.154:0/1001686271 shutdown_connections 2023-09-23T03:16:56.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.992+0000 7f810ffff640 1 --2- 172.21.15.154:0/1001686271 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f80f006af60 0x7f80f006d420 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:56.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.992+0000 7f810ffff640 1 --2- 172.21.15.154:0/1001686271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f811015ab00 0x7f811014e480 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:56.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.992+0000 7f810ffff640 1 --2- 172.21.15.154:0/1001686271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f811006fdc0 0x7f81100b23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:56.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.992+0000 7f810ffff640 1 --2- 172.21.15.154:0/1001686271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f811006c7a0 0x7f81101517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:56.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.992+0000 7f810ffff640 1 -- 172.21.15.154:0/1001686271 >> 172.21.15.154:0/1001686271 conn(0x7f811006a390 msgr2=0x7f81100af260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:56.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.993+0000 7f810ffff640 1 -- 172.21.15.154:0/1001686271 shutdown_connections 2023-09-23T03:16:56.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:56.993+0000 7f810ffff640 1 -- 172.21.15.154:0/1001686271 wait complete. 2023-09-23T03:16:57.012 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:16:57.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.225+0000 7f51f2c26640 1 Processor -- start 2023-09-23T03:16:57.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.226+0000 7f51f2c26640 1 -- start start 2023-09-23T03:16:57.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.227+0000 7f51f2c26640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51ec0b1b20 0x7f51ec0ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:57.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.227+0000 7f51f2c26640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51ec15ac30 con 0x7f51ec0acce0 2023-09-23T03:16:57.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.227+0000 7f51f2c26640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51ec0b1f70 con 0x7f51ec0cee70 2023-09-23T03:16:57.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.227+0000 7f51f2c26640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec0b2e90 con 0x7f51ec0b1b20 2023-09-23T03:16:57.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.227+0000 7f51f09c2640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f51ec0acce0 0x7f51ec15ed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:54830/0 (socket says 172.21.15.154:54830) 2023-09-23T03:16:57.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.227+0000 7f51f09c2640 1 -- 172.21.15.154:0/3314287618 learned_addr learned my addr 172.21.15.154:0/3314287618 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:57.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.228+0000 7f51eb7fe640 1 -- 172.21.15.154:0/3314287618 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3455125570 0 0) 0x7f51e0001290 con 0x7f51ec0acce0 2023-09-23T03:16:57.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.228+0000 7f51eb7fe640 1 -- 172.21.15.154:0/3314287618 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51dc003520 con 0x7f51ec0acce0 2023-09-23T03:16:57.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.228+0000 7f51ebfff640 1 --2- 172.21.15.154:0/3314287618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51ec0b1b20 0x7f51ec0ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:57.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.228+0000 7f51eb7fe640 1 -- 172.21.15.154:0/3314287618 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1683274292 0 0) 0x7f51dc003520 con 0x7f51ec0acce0 2023-09-23T03:16:57.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.228+0000 7f51eb7fe640 1 -- 172.21.15.154:0/3314287618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51ec0b1b20 msgr2=0x7f51ec0ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:57.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.228+0000 7f51eb7fe640 1 --2- 172.21.15.154:0/3314287618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51ec0b1b20 0x7f51ec0ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:57.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.228+0000 7f51eb7fe640 1 -- 172.21.15.154:0/3314287618 >> v1:172.21.15.190:6789/0 conn(0x7f51ec0cee70 legacy=0x7f51ec0ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:57.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.228+0000 7f51eb7fe640 1 -- 172.21.15.154:0/3314287618 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ec0b2210 con 0x7f51ec0acce0 2023-09-23T03:16:57.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.228+0000 7f51eb7fe640 1 -- 172.21.15.154:0/3314287618 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f51e0001980 con 0x7f51ec0acce0 2023-09-23T03:16:57.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.228+0000 7f51eb7fe640 1 -- 172.21.15.154:0/3314287618 >> v1:172.21.15.154:6789/0 conn(0x7f51ec0acce0 legacy=0x7f51ec15ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:57.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.228+0000 7f51eb7fe640 1 --2- 172.21.15.154:0/3314287618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51dc0039a0 0x7f51dc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:57.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.228+0000 7f51eb7fe640 1 --2- 172.21.15.154:0/3314287618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51ec0b1b20 0x7f51dc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:57.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.228+0000 7f51eb7fe640 1 --2- 172.21.15.154:0/3314287618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51dc004d00 0x7f51dc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:57.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.228+0000 7f51eb7fe640 1 -- 172.21.15.154:0/3314287618 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec0b2e90 con 0x7f51dc0039a0 2023-09-23T03:16:57.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.228+0000 7f51eb7fe640 1 -- 172.21.15.154:0/3314287618 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc00bb60 con 0x7f51dc004d00 2023-09-23T03:16:57.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.228+0000 7f51eb7fe640 1 -- 172.21.15.154:0/3314287618 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc00bd20 con 0x7f51ec0b1b20 2023-09-23T03:16:57.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.228+0000 7f51f09c2640 1 --2- 172.21.15.154:0/3314287618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51dc0039a0 0x7f51dc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:57.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.229+0000 7f51ebfff640 1 --2- 172.21.15.154:0/3314287618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51ec0b1b20 0x7f51dc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:57.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.229+0000 7f51f11c3640 1 --2- 172.21.15.154:0/3314287618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51dc004d00 0x7f51dc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:57.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.229+0000 7f51f09c2640 1 -- 172.21.15.154:0/3314287618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51ec0b1b20 msgr2=0x7f51dc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:57.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.229+0000 7f51f09c2640 1 --2- 172.21.15.154:0/3314287618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51ec0b1b20 0x7f51dc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:57.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.229+0000 7f51f09c2640 1 -- 172.21.15.154:0/3314287618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51dc004d00 msgr2=0x7f51dc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:57.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.229+0000 7f51f09c2640 1 --2- 172.21.15.154:0/3314287618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51dc004d00 0x7f51dc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:57.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.229+0000 7f51f09c2640 1 -- 172.21.15.154:0/3314287618 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc003520 con 0x7f51dc0039a0 2023-09-23T03:16:57.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.229+0000 7f51f11c3640 1 --2- 172.21.15.154:0/3314287618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51dc004d00 0x7f51dc005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:57.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.230+0000 7f51f09c2640 1 --2- 172.21.15.154:0/3314287618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51dc0039a0 0x7f51dc003dc0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f51e000c210 tx=0x7f51dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f87f8ac51a6b4c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:57.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.230+0000 7f51eb7fe640 1 -- 172.21.15.154:0/3314287618 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e0007270 con 0x7f51dc0039a0 2023-09-23T03:16:57.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.230+0000 7f51eb7fe640 1 -- 172.21.15.154:0/3314287618 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e00073f0 con 0x7f51dc0039a0 2023-09-23T03:16:57.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.230+0000 7f51eb7fe640 1 -- 172.21.15.154:0/3314287618 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e002c890 con 0x7f51dc0039a0 2023-09-23T03:16:57.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.231+0000 7f51f2c26640 1 -- 172.21.15.154:0/3314287618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51dc0039a0 msgr2=0x7f51dc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:57.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.231+0000 7f51f2c26640 1 --2- 172.21.15.154:0/3314287618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51dc0039a0 0x7f51dc003dc0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f51e000c210 tx=0x7f51dc0057f0 comp rx=0 tx=0).stop 2023-09-23T03:16:57.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.231+0000 7f51f11c3640 1 -- 172.21.15.154:0/3314287618 reap_dead start 2023-09-23T03:16:57.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.231+0000 7f51f2c26640 1 -- 172.21.15.154:0/3314287618 shutdown_connections 2023-09-23T03:16:57.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.231+0000 7f51f2c26640 1 -- 172.21.15.154:0/3314287618 >> 172.21.15.154:0/3314287618 conn(0x7f51ec06a0f0 msgr2=0x7f51ec06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:57.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.231+0000 7f51f2c26640 1 -- 172.21.15.154:0/3314287618 shutdown_connections 2023-09-23T03:16:57.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.231+0000 7f51f2c26640 1 -- 172.21.15.154:0/3314287618 wait complete. 2023-09-23T03:16:57.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.233+0000 7f51f2c26640 1 Processor -- start 2023-09-23T03:16:57.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.233+0000 7f51f2c26640 1 -- start start 2023-09-23T03:16:57.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.234+0000 7f51f2c26640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51dc0039a0 0x7f51ec0af6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:57.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.234+0000 7f51f2c26640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51ec0c4920 0x7f51ec15add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:57.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.234+0000 7f51f2c26640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51ec15b310 0x7f51ec0c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:57.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.234+0000 7f51f2c26640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec0cf320 con 0x7f51ec15b310 2023-09-23T03:16:57.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.234+0000 7f51f2c26640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec0c4da0 con 0x7f51ec0c4920 2023-09-23T03:16:57.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.234+0000 7f51f2c26640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec0c3b60 con 0x7f51dc0039a0 2023-09-23T03:16:57.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.234+0000 7f51f11c3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51ec15b310 0x7f51ec0c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:57.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.234+0000 7f51f11c3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51ec15b310 0x7f51ec0c5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:55996/0 (socket says 172.21.15.154:55996) 2023-09-23T03:16:57.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.235+0000 7f51f11c3640 1 -- 172.21.15.154:0/3225194354 learned_addr learned my addr 172.21.15.154:0/3225194354 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:57.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.235+0000 7f51ebfff640 1 --2- 172.21.15.154:0/3225194354 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51ec0c4920 0x7f51ec15add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:57.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.235+0000 7f51f09c2640 1 --2- 172.21.15.154:0/3225194354 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51dc0039a0 0x7f51ec0af6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:57.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.235+0000 7f51f11c3640 1 -- 172.21.15.154:0/3225194354 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51dc0039a0 msgr2=0x7f51ec0af6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:57.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.235+0000 7f51f11c3640 1 --2- 172.21.15.154:0/3225194354 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51dc0039a0 0x7f51ec0af6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:57.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.235+0000 7f51f11c3640 1 -- 172.21.15.154:0/3225194354 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51ec0c4920 msgr2=0x7f51ec15add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:57.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.235+0000 7f51f11c3640 1 --2- 172.21.15.154:0/3225194354 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51ec0c4920 0x7f51ec15add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:57.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.235+0000 7f51f11c3640 1 -- 172.21.15.154:0/3225194354 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ec0c3e40 con 0x7f51ec15b310 2023-09-23T03:16:57.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.235+0000 7f51f09c2640 1 --2- 172.21.15.154:0/3225194354 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51dc0039a0 0x7f51ec0af6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:57.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.235+0000 7f51f11c3640 1 --2- 172.21.15.154:0/3225194354 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51ec15b310 0x7f51ec0c5fb0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f51ec0c9430 tx=0x7f51e400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:57.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.236+0000 7f51e97fa640 1 -- 172.21.15.154:0/3225194354 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e4006d50 con 0x7f51ec15b310 2023-09-23T03:16:57.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.236+0000 7f51e97fa640 1 -- 172.21.15.154:0/3225194354 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e4003970 con 0x7f51ec15b310 2023-09-23T03:16:57.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.236+0000 7f51e97fa640 1 -- 172.21.15.154:0/3225194354 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e4003d20 con 0x7f51ec15b310 2023-09-23T03:16:57.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.236+0000 7f51f2c26640 1 -- 172.21.15.154:0/3225194354 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51ec0c4120 con 0x7f51ec15b310 2023-09-23T03:16:57.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.236+0000 7f51f2c26640 1 -- 172.21.15.154:0/3225194354 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51ec15b7a0 con 0x7f51ec15b310 2023-09-23T03:16:57.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.237+0000 7f51f2c26640 1 -- 172.21.15.154:0/3225194354 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51ec0c4640 con 0x7f51ec15b310 2023-09-23T03:16:57.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.238+0000 7f51e97fa640 1 -- 172.21.15.154:0/3225194354 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f51e4005070 con 0x7f51ec15b310 2023-09-23T03:16:57.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.239+0000 7f51e97fa640 1 --2- 172.21.15.154:0/3225194354 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f51d006af30 0x7f51d006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:57.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.239+0000 7f51e97fa640 1 -- 172.21.15.154:0/3225194354 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7f51e4088d00 con 0x7f51ec15b310 2023-09-23T03:16:57.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.239+0000 7f51f09c2640 1 --2- 172.21.15.154:0/3225194354 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f51d006af30 0x7f51d006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:57.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.240+0000 7f51f09c2640 1 --2- 172.21.15.154:0/3225194354 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f51d006af30 0x7f51d006d3f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f51e0034000 tx=0x7f51e0009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:57.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.243+0000 7f51e97fa640 1 -- 172.21.15.154:0/3225194354 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f51ec0c4640 con 0x7f51ec15b310 2023-09-23T03:16:57.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.466+0000 7f51f2c26640 1 -- 172.21.15.154:0/3225194354 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51ec0c4640 con 0x7f51ec15b310 2023-09-23T03:16:57.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.467+0000 7f51e97fa640 1 -- 172.21.15.154:0/3225194354 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4921 (secure 0 0 0) 0x7f51ec0c4640 con 0x7f51ec15b310 2023-09-23T03:16:57.467 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:16:57.468 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2644365591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2644365591},{"type":"v1","addr":"172.21.15.190:6835","nonce":2644365591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/4145467852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4145467852},{"type":"v1","addr":"172.21.15.154:6835","nonce":4145467852}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1022302774","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1022302774},{"type":"v1","addr":"172.21.15.154:6837","nonce":1022302774}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:15:20.686035+0000","modified":"2023-09-23T03:15:48.742318+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6834/3561509423","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3561509423},{"type":"v1","addr":"172.21.15.190:6834","nonce":3561509423}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-23T03:16:57.468 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 9 2023-09-23T03:16:57.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.469+0000 7f51f2c26640 1 -- 172.21.15.154:0/3225194354 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f51d006af30 msgr2=0x7f51d006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:57.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.469+0000 7f51f2c26640 1 --2- 172.21.15.154:0/3225194354 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f51d006af30 0x7f51d006d3f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f51e0034000 tx=0x7f51e0009030 comp rx=0 tx=0).stop 2023-09-23T03:16:57.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.469+0000 7f51f2c26640 1 -- 172.21.15.154:0/3225194354 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51ec15b310 msgr2=0x7f51ec0c5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:57.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.469+0000 7f51f2c26640 1 --2- 172.21.15.154:0/3225194354 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51ec15b310 0x7f51ec0c5fb0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f51ec0c9430 tx=0x7f51e400dc00 comp rx=0 tx=0).stop 2023-09-23T03:16:57.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.470+0000 7f51f2c26640 1 -- 172.21.15.154:0/3225194354 shutdown_connections 2023-09-23T03:16:57.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.470+0000 7f51f2c26640 1 --2- 172.21.15.154:0/3225194354 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f51d006af30 0x7f51d006d3f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:57.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.470+0000 7f51f2c26640 1 --2- 172.21.15.154:0/3225194354 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51ec15b310 0x7f51ec0c5fb0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:57.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.470+0000 7f51f2c26640 1 --2- 172.21.15.154:0/3225194354 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51ec0c4920 0x7f51ec15add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:57.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.470+0000 7f51f2c26640 1 --2- 172.21.15.154:0/3225194354 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51dc0039a0 0x7f51ec0af6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:57.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.470+0000 7f51f2c26640 1 -- 172.21.15.154:0/3225194354 >> 172.21.15.154:0/3225194354 conn(0x7f51ec06a0f0 msgr2=0x7f51ec0b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:57.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.470+0000 7f51f2c26640 1 -- 172.21.15.154:0/3225194354 shutdown_connections 2023-09-23T03:16:57.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.470+0000 7f51f2c26640 1 -- 172.21.15.154:0/3225194354 wait complete. 2023-09-23T03:16:57.489 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:16:57.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.699+0000 7fd404a7c640 1 Processor -- start 2023-09-23T03:16:57.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.699+0000 7fd404a7c640 1 -- start start 2023-09-23T03:16:57.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.700+0000 7fd404a7c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd4000b2fe0 0x7fd4000ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:57.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.700+0000 7fd404a7c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd40006ceb0 con 0x7fd4000ada30 2023-09-23T03:16:57.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.700+0000 7fd404a7c640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd400159c90 con 0x7fd4000cf130 2023-09-23T03:16:57.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.700+0000 7fd404a7c640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4000ade10 con 0x7fd4000b2fe0 2023-09-23T03:16:57.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.701+0000 7fd3fe59c640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fd4000ada30 0x7fd4000cea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:54836/0 (socket says 172.21.15.154:54836) 2023-09-23T03:16:57.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.701+0000 7fd3fe59c640 1 -- 172.21.15.154:0/868733607 learned_addr learned my addr 172.21.15.154:0/868733607 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:57.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.701+0000 7fd3fdd9b640 1 -- 172.21.15.154:0/868733607 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 540470564 0 0) 0x7fd3f0001290 con 0x7fd4000ada30 2023-09-23T03:16:57.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.701+0000 7fd3fed9d640 1 --2- 172.21.15.154:0/868733607 >> v2:172.21.15.190:3300/0 conn(0x7fd4000b2fe0 0x7fd4000ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:57.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.701+0000 7fd3fdd9b640 1 -- 172.21.15.154:0/868733607 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3ec003520 con 0x7fd4000ada30 2023-09-23T03:16:57.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.701+0000 7fd3fdd9b640 1 -- 172.21.15.154:0/868733607 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2426728306 0 0) 0x7fd3ec003520 con 0x7fd4000ada30 2023-09-23T03:16:57.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.701+0000 7fd3fdd9b640 1 -- 172.21.15.154:0/868733607 >> v2:172.21.15.190:3300/0 conn(0x7fd4000b2fe0 msgr2=0x7fd4000ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:57.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.701+0000 7fd3fdd9b640 1 --2- 172.21.15.154:0/868733607 >> v2:172.21.15.190:3300/0 conn(0x7fd4000b2fe0 0x7fd4000ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:57.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.702+0000 7fd3fdd9b640 1 -- 172.21.15.154:0/868733607 >> v1:172.21.15.190:6789/0 conn(0x7fd4000cf130 legacy=0x7fd40015d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:57.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.702+0000 7fd3fdd9b640 1 -- 172.21.15.154:0/868733607 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40006c110 con 0x7fd4000ada30 2023-09-23T03:16:57.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.702+0000 7fd3fdd9b640 1 -- 172.21.15.154:0/868733607 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fd3f0001980 con 0x7fd4000ada30 2023-09-23T03:16:57.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.702+0000 7fd3fdd9b640 1 -- 172.21.15.154:0/868733607 >> v1:172.21.15.154:6789/0 conn(0x7fd4000ada30 legacy=0x7fd4000cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:57.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.702+0000 7fd3fdd9b640 1 --2- 172.21.15.154:0/868733607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3ec0039c0 0x7fd3ec003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:57.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.702+0000 7fd3fdd9b640 1 --2- 172.21.15.154:0/868733607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3ec0047c0 0x7fd3ec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:57.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.702+0000 7fd3fdd9b640 1 --2- 172.21.15.154:0/868733607 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3ec0051d0 0x7fd3ec00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:57.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.702+0000 7fd3fdd9b640 1 -- 172.21.15.154:0/868733607 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4000ade10 con 0x7fd3ec0047c0 2023-09-23T03:16:57.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.702+0000 7fd3fdd9b640 1 -- 172.21.15.154:0/868733607 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ec00c120 con 0x7fd3ec0039c0 2023-09-23T03:16:57.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.702+0000 7fd3fdd9b640 1 -- 172.21.15.154:0/868733607 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ec00c2e0 con 0x7fd3ec0051d0 2023-09-23T03:16:57.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.702+0000 7fd3f7fff640 1 --2- 172.21.15.154:0/868733607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3ec0047c0 0x7fd3ec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:57.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.702+0000 7fd3fe59c640 1 --2- 172.21.15.154:0/868733607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3ec0039c0 0x7fd3ec003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:57.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.702+0000 7fd3fed9d640 1 --2- 172.21.15.154:0/868733607 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3ec0051d0 0x7fd3ec00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:57.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.703+0000 7fd3f7fff640 1 -- 172.21.15.154:0/868733607 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3ec0051d0 msgr2=0x7fd3ec00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:57.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.703+0000 7fd3f7fff640 1 --2- 172.21.15.154:0/868733607 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3ec0051d0 0x7fd3ec00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:57.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.703+0000 7fd3f7fff640 1 -- 172.21.15.154:0/868733607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3ec0039c0 msgr2=0x7fd3ec003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:57.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.703+0000 7fd3f7fff640 1 --2- 172.21.15.154:0/868733607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3ec0039c0 0x7fd3ec003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:57.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.703+0000 7fd3f7fff640 1 -- 172.21.15.154:0/868733607 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ec003520 con 0x7fd3ec0047c0 2023-09-23T03:16:57.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.703+0000 7fd3fed9d640 1 --2- 172.21.15.154:0/868733607 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3ec0051d0 0x7fd3ec00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:57.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.703+0000 7fd3fe59c640 1 --2- 172.21.15.154:0/868733607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3ec0039c0 0x7fd3ec003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:16:57.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.703+0000 7fd3fed9d640 1 -- 172.21.15.154:0/868733607 reap_dead start 2023-09-23T03:16:57.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.704+0000 7fd3f7fff640 1 --2- 172.21.15.154:0/868733607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3ec0047c0 0x7fd3ec004c40 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd40006e070 tx=0x7fd3ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6e5f58d2b5df62 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:57.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.704+0000 7fd3fdd9b640 1 -- 172.21.15.154:0/868733607 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e80052d0 con 0x7fd3ec0047c0 2023-09-23T03:16:57.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.704+0000 7fd3fdd9b640 1 -- 172.21.15.154:0/868733607 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e802e050 con 0x7fd3ec0047c0 2023-09-23T03:16:57.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.704+0000 7fd3fdd9b640 1 -- 172.21.15.154:0/868733607 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e8003830 con 0x7fd3ec0047c0 2023-09-23T03:16:57.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.704+0000 7fd404a7c640 1 -- 172.21.15.154:0/868733607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3ec0047c0 msgr2=0x7fd3ec004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:57.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.704+0000 7fd404a7c640 1 --2- 172.21.15.154:0/868733607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3ec0047c0 0x7fd3ec004c40 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd40006e070 tx=0x7fd3ec0057f0 comp rx=0 tx=0).stop 2023-09-23T03:16:57.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.705+0000 7fd404a7c640 1 -- 172.21.15.154:0/868733607 shutdown_connections 2023-09-23T03:16:57.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.705+0000 7fd404a7c640 1 --2- 172.21.15.154:0/868733607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3ec0047c0 0x7fd3ec004c40 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:57.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.705+0000 7fd404a7c640 1 -- 172.21.15.154:0/868733607 >> 172.21.15.154:0/868733607 conn(0x7fd40006a0f0 msgr2=0x7fd40006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:57.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.705+0000 7fd404a7c640 1 -- 172.21.15.154:0/868733607 shutdown_connections 2023-09-23T03:16:57.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.705+0000 7fd404a7c640 1 -- 172.21.15.154:0/868733607 wait complete. 2023-09-23T03:16:57.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.706+0000 7fd404a7c640 1 Processor -- start 2023-09-23T03:16:57.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.707+0000 7fd404a7c640 1 -- start start 2023-09-23T03:16:57.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.707+0000 7fd404a7c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3ec0047c0 0x7fd40015bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:57.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.707+0000 7fd404a7c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd400070f30 0x7fd400096ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:57.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.707+0000 7fd404a7c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4000713b0 0x7fd4000973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:57.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.708+0000 7fd404a7c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40015c980 con 0x7fd3ec0047c0 2023-09-23T03:16:57.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.708+0000 7fd404a7c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4000964a0 con 0x7fd4000713b0 2023-09-23T03:16:57.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.708+0000 7fd404a7c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4000ae480 con 0x7fd400070f30 2023-09-23T03:16:57.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.708+0000 7fd3fe59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3ec0047c0 0x7fd40015bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:57.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.708+0000 7fd3fe59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3ec0047c0 0x7fd40015bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56010/0 (socket says 172.21.15.154:56010) 2023-09-23T03:16:57.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.708+0000 7fd3fe59c640 1 -- 172.21.15.154:0/396560614 learned_addr learned my addr 172.21.15.154:0/396560614 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:57.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.708+0000 7fd3fed9d640 1 --2- 172.21.15.154:0/396560614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4000713b0 0x7fd4000973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:57.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.708+0000 7fd3f7fff640 1 --2- 172.21.15.154:0/396560614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd400070f30 0x7fd400096ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:57.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.708+0000 7fd3fe59c640 1 -- 172.21.15.154:0/396560614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd400070f30 msgr2=0x7fd400096ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:57.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.708+0000 7fd3fe59c640 1 --2- 172.21.15.154:0/396560614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd400070f30 0x7fd400096ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:57.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.708+0000 7fd3fe59c640 1 -- 172.21.15.154:0/396560614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4000713b0 msgr2=0x7fd4000973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:57.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.708+0000 7fd3fe59c640 1 --2- 172.21.15.154:0/396560614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4000713b0 0x7fd4000973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:57.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.708+0000 7fd3fe59c640 1 -- 172.21.15.154:0/396560614 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd400096a10 con 0x7fd3ec0047c0 2023-09-23T03:16:57.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.708+0000 7fd3fed9d640 1 --2- 172.21.15.154:0/396560614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4000713b0 0x7fd4000973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:57.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.708+0000 7fd3fe59c640 1 --2- 172.21.15.154:0/396560614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3ec0047c0 0x7fd40015bf70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0001510 tx=0x7fd3f000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:57.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.709+0000 7fd3f6ffd640 1 -- 172.21.15.154:0/396560614 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f000ee00 con 0x7fd3ec0047c0 2023-09-23T03:16:57.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.709+0000 7fd3f6ffd640 1 -- 172.21.15.154:0/396560614 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3f00072e0 con 0x7fd3ec0047c0 2023-09-23T03:16:57.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.709+0000 7fd404a7c640 1 -- 172.21.15.154:0/396560614 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd40015b1d0 con 0x7fd3ec0047c0 2023-09-23T03:16:57.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.709+0000 7fd3f6ffd640 1 -- 172.21.15.154:0/396560614 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0007650 con 0x7fd3ec0047c0 2023-09-23T03:16:57.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.709+0000 7fd404a7c640 1 -- 172.21.15.154:0/396560614 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4000ae130 con 0x7fd3ec0047c0 2023-09-23T03:16:57.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.710+0000 7fd404a7c640 1 -- 172.21.15.154:0/396560614 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd40015b3c0 con 0x7fd3ec0047c0 2023-09-23T03:16:57.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.711+0000 7fd3f6ffd640 1 -- 172.21.15.154:0/396560614 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fd3f00057a0 con 0x7fd3ec0047c0 2023-09-23T03:16:57.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.712+0000 7fd3f6ffd640 1 --2- 172.21.15.154:0/396560614 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd3e006af60 0x7fd3e006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:57.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.712+0000 7fd3f6ffd640 1 -- 172.21.15.154:0/396560614 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7fd3f0089940 con 0x7fd3ec0047c0 2023-09-23T03:16:57.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.712+0000 7fd3f7fff640 1 --2- 172.21.15.154:0/396560614 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd3e006af60 0x7fd3e006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:57.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.713+0000 7fd3f7fff640 1 --2- 172.21.15.154:0/396560614 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd3e006af60 0x7fd3e006d420 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8033000 tx=0x7fd3e8009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:57.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.716+0000 7fd3f6ffd640 1 -- 172.21.15.154:0/396560614 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fd3f00545c0 con 0x7fd3ec0047c0 2023-09-23T03:16:57.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.906+0000 7fd404a7c640 1 -- 172.21.15.154:0/396560614 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd40006ceb0 con 0x7fd3ec0047c0 2023-09-23T03:16:57.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.909+0000 7fd3f6ffd640 1 -- 172.21.15.154:0/396560614 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18744 (secure 0 0 0) 0x7fd40006ceb0 con 0x7fd3ec0047c0 2023-09-23T03:16:57.909 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:16:57.909 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":26,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:16:03.151371+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:15:18.575849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-09-23T03:15:19.707212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-09-23T03:15:23.337138+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.30","mappings":[{"from":5,"to":2}]},{"pgid":"3.39","mappings":[{"from":7,"to":2}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1c","mappings":[{"from":1,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:16:57.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.911+0000 7fd404a7c640 1 -- 172.21.15.154:0/396560614 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd3e006af60 msgr2=0x7fd3e006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:57.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.912+0000 7fd404a7c640 1 --2- 172.21.15.154:0/396560614 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd3e006af60 0x7fd3e006d420 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8033000 tx=0x7fd3e8009030 comp rx=0 tx=0).stop 2023-09-23T03:16:57.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.912+0000 7fd404a7c640 1 -- 172.21.15.154:0/396560614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3ec0047c0 msgr2=0x7fd40015bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:57.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.912+0000 7fd404a7c640 1 --2- 172.21.15.154:0/396560614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3ec0047c0 0x7fd40015bf70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0001510 tx=0x7fd3f000ec10 comp rx=0 tx=0).stop 2023-09-23T03:16:57.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.912+0000 7fd404a7c640 1 -- 172.21.15.154:0/396560614 shutdown_connections 2023-09-23T03:16:57.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.912+0000 7fd404a7c640 1 --2- 172.21.15.154:0/396560614 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd3e006af60 0x7fd3e006d420 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:57.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.912+0000 7fd404a7c640 1 --2- 172.21.15.154:0/396560614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4000713b0 0x7fd4000973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:57.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.912+0000 7fd404a7c640 1 --2- 172.21.15.154:0/396560614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd400070f30 0x7fd400096ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:57.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.912+0000 7fd404a7c640 1 --2- 172.21.15.154:0/396560614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3ec0047c0 0x7fd40015bf70 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:57.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.912+0000 7fd404a7c640 1 -- 172.21.15.154:0/396560614 >> 172.21.15.154:0/396560614 conn(0x7fd40006a0f0 msgr2=0x7fd4000affe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:57.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.912+0000 7fd404a7c640 1 -- 172.21.15.154:0/396560614 shutdown_connections 2023-09-23T03:16:57.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:57.912+0000 7fd404a7c640 1 -- 172.21.15.154:0/396560614 wait complete. 2023-09-23T03:16:57.932 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-09-23T03:16:58.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.138+0000 7fc5a1fa9640 1 Processor -- start 2023-09-23T03:16:58.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.139+0000 7fc5a1fa9640 1 -- start start 2023-09-23T03:16:58.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.140+0000 7fc5a1fa9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc59c15caf0 0x7fc59c0ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:58.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.140+0000 7fc5a1fa9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc59c0cee70 0x7fc59c160880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:58.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.140+0000 7fc5a1fa9640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc59c0cd530 con 0x7fc59c160dc0 2023-09-23T03:16:58.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.140+0000 7fc5a1fa9640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc59c15ced0 con 0x7fc59c0cee70 2023-09-23T03:16:58.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.140+0000 7fc5a1fa9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc59c0cf2f0 con 0x7fc59c15caf0 2023-09-23T03:16:58.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.140+0000 7fc593fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc59c0cee70 0x7fc59c160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:58.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.140+0000 7fc593fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc59c0cee70 0x7fc59c160880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56012/0 (socket says 172.21.15.154:56012) 2023-09-23T03:16:58.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.140+0000 7fc593fff640 1 -- 172.21.15.154:0/1663616798 learned_addr learned my addr 172.21.15.154:0/1663616798 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:58.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.141+0000 7fc59b7fe640 1 --2- 172.21.15.154:0/1663616798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc59c15caf0 0x7fc59c0ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:58.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.141+0000 7fc593fff640 1 -- 172.21.15.154:0/1663616798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc59c15caf0 msgr2=0x7fc59c0ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:58.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.141+0000 7fc593fff640 1 --2- 172.21.15.154:0/1663616798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc59c15caf0 0x7fc59c0ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:58.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.141+0000 7fc593fff640 1 -- 172.21.15.154:0/1663616798 >> v1:172.21.15.190:6789/0 conn(0x7fc59c160dc0 legacy=0x7fc59c06cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:58.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.141+0000 7fc593fff640 1 -- 172.21.15.154:0/1663616798 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc59c0cd810 con 0x7fc59c0cee70 2023-09-23T03:16:58.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.142+0000 7fc593fff640 1 --2- 172.21.15.154:0/1663616798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc59c0cee70 0x7fc59c160880 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc584009870 tx=0x7fc58402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f87da25a1e6e2fb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:58.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.142+0000 7fc59affd640 1 -- 172.21.15.154:0/1663616798 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc584031640 con 0x7fc59c0cee70 2023-09-23T03:16:58.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.142+0000 7fc59affd640 1 -- 172.21.15.154:0/1663616798 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5840317a0 con 0x7fc59c0cee70 2023-09-23T03:16:58.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.142+0000 7fc59affd640 1 -- 172.21.15.154:0/1663616798 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc584031b30 con 0x7fc59c0cee70 2023-09-23T03:16:58.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.143+0000 7fc5a1fa9640 1 -- 172.21.15.154:0/1663616798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc59c0cee70 msgr2=0x7fc59c160880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:58.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.143+0000 7fc5a1fa9640 1 --2- 172.21.15.154:0/1663616798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc59c0cee70 0x7fc59c160880 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc584009870 tx=0x7fc58402b8f0 comp rx=0 tx=0).stop 2023-09-23T03:16:58.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.143+0000 7fc5a1fa9640 1 -- 172.21.15.154:0/1663616798 shutdown_connections 2023-09-23T03:16:58.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.143+0000 7fc5a1fa9640 1 --2- 172.21.15.154:0/1663616798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc59c0cee70 0x7fc59c160880 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:58.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.143+0000 7fc5a1fa9640 1 --2- 172.21.15.154:0/1663616798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc59c15caf0 0x7fc59c0ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:58.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.143+0000 7fc5a1fa9640 1 -- 172.21.15.154:0/1663616798 >> 172.21.15.154:0/1663616798 conn(0x7fc59c06a0f0 msgr2=0x7fc59c06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:58.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.143+0000 7fc5a1fa9640 1 -- 172.21.15.154:0/1663616798 shutdown_connections 2023-09-23T03:16:58.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.143+0000 7fc5a1fa9640 1 -- 172.21.15.154:0/1663616798 wait complete. 2023-09-23T03:16:58.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.145+0000 7fc5a1fa9640 1 Processor -- start 2023-09-23T03:16:58.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.145+0000 7fc5a1fa9640 1 -- start start 2023-09-23T03:16:58.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.146+0000 7fc5a1fa9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc59c0cee70 0x7fc59c153610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:58.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.146+0000 7fc5a1fa9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc59c15caf0 0x7fc59c15b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:58.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.146+0000 7fc5a1fa9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc59c160dc0 0x7fc59c15bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:58.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.146+0000 7fc5a1fa9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59c0c1ff0 con 0x7fc59c160dc0 2023-09-23T03:16:58.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.146+0000 7fc5a1fa9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59c153b50 con 0x7fc59c0cee70 2023-09-23T03:16:58.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.146+0000 7fc5a1fa9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc59c0c2240 con 0x7fc59c15caf0 2023-09-23T03:16:58.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.147+0000 7fc59bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc59c160dc0 0x7fc59c15bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:58.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.147+0000 7fc59bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc59c160dc0 0x7fc59c15bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56024/0 (socket says 172.21.15.154:56024) 2023-09-23T03:16:58.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.147+0000 7fc59bfff640 1 -- 172.21.15.154:0/2470999623 learned_addr learned my addr 172.21.15.154:0/2470999623 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:58.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.147+0000 7fc593fff640 1 --2- 172.21.15.154:0/2470999623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc59c15caf0 0x7fc59c15b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:58.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.147+0000 7fc59b7fe640 1 --2- 172.21.15.154:0/2470999623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc59c0cee70 0x7fc59c153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:58.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.147+0000 7fc59bfff640 1 -- 172.21.15.154:0/2470999623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc59c15caf0 msgr2=0x7fc59c15b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:58.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.147+0000 7fc59bfff640 1 --2- 172.21.15.154:0/2470999623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc59c15caf0 0x7fc59c15b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:58.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.147+0000 7fc59bfff640 1 -- 172.21.15.154:0/2470999623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc59c0cee70 msgr2=0x7fc59c153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:58.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.147+0000 7fc59bfff640 1 --2- 172.21.15.154:0/2470999623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc59c0cee70 0x7fc59c153610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:58.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.147+0000 7fc59bfff640 1 -- 172.21.15.154:0/2470999623 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5840094d0 con 0x7fc59c160dc0 2023-09-23T03:16:58.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.147+0000 7fc59bfff640 1 --2- 172.21.15.154:0/2470999623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc59c160dc0 0x7fc59c15bd40 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc59400b970 tx=0x7fc59400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:58.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.148+0000 7fc598ff9640 1 -- 172.21.15.154:0/2470999623 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59400cb40 con 0x7fc59c160dc0 2023-09-23T03:16:58.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.148+0000 7fc598ff9640 1 -- 172.21.15.154:0/2470999623 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59400cca0 con 0x7fc59c160dc0 2023-09-23T03:16:58.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.148+0000 7fc598ff9640 1 -- 172.21.15.154:0/2470999623 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc594014680 con 0x7fc59c160dc0 2023-09-23T03:16:58.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.148+0000 7fc5a1fa9640 1 -- 172.21.15.154:0/2470999623 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc59c0c2520 con 0x7fc59c160dc0 2023-09-23T03:16:58.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.148+0000 7fc5a1fa9640 1 -- 172.21.15.154:0/2470999623 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc59c0c2ae0 con 0x7fc59c160dc0 2023-09-23T03:16:58.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.149+0000 7fc5a1fa9640 1 -- 172.21.15.154:0/2470999623 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc59c0c2cd0 con 0x7fc59c160dc0 2023-09-23T03:16:58.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.150+0000 7fc598ff9640 1 -- 172.21.15.154:0/2470999623 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fc59401d3b0 con 0x7fc59c160dc0 2023-09-23T03:16:58.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.151+0000 7fc598ff9640 1 --2- 172.21.15.154:0/2470999623 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc57c06af30 0x7fc57c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:58.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.151+0000 7fc598ff9640 1 -- 172.21.15.154:0/2470999623 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6642+0+0 (secure 0 0 0) 0x7fc594089a10 con 0x7fc59c160dc0 2023-09-23T03:16:58.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.151+0000 7fc59b7fe640 1 --2- 172.21.15.154:0/2470999623 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc57c06af30 0x7fc57c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:58.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.152+0000 7fc59b7fe640 1 --2- 172.21.15.154:0/2470999623 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc57c06af30 0x7fc57c06d3f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc58c00b9b0 tx=0x7fc58c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:58.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.155+0000 7fc598ff9640 1 -- 172.21.15.154:0/2470999623 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fc594054600 con 0x7fc59c160dc0 2023-09-23T03:16:58.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.376+0000 7fc5a1fa9640 1 -- 172.21.15.154:0/2470999623 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc59c073eb0 con 0x7fc59c160dc0 2023-09-23T03:16:58.377 INFO:tasks.ceph.mon.a.smithi154.stderr:2023-09-23T03:16:58.377+0000 7f3df35a6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-23T03:16:58.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.381+0000 7fc598ff9640 1 -- 172.21.15.154:0/2470999623 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc594057c20 con 0x7fc59c160dc0 2023-09-23T03:16:58.381 INFO:teuthology.orchestra.run.smithi154.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-09-23T03:16:58.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.384+0000 7fc5a1fa9640 1 -- 172.21.15.154:0/2470999623 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc57c06af30 msgr2=0x7fc57c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:58.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.384+0000 7fc5a1fa9640 1 --2- 172.21.15.154:0/2470999623 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc57c06af30 0x7fc57c06d3f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc58c00b9b0 tx=0x7fc58c008040 comp rx=0 tx=0).stop 2023-09-23T03:16:58.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.384+0000 7fc5a1fa9640 1 -- 172.21.15.154:0/2470999623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc59c160dc0 msgr2=0x7fc59c15bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:58.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.384+0000 7fc5a1fa9640 1 --2- 172.21.15.154:0/2470999623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc59c160dc0 0x7fc59c15bd40 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc59400b970 tx=0x7fc59400be30 comp rx=0 tx=0).stop 2023-09-23T03:16:58.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.384+0000 7fc5a1fa9640 1 -- 172.21.15.154:0/2470999623 shutdown_connections 2023-09-23T03:16:58.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.384+0000 7fc5a1fa9640 1 --2- 172.21.15.154:0/2470999623 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc57c06af30 0x7fc57c06d3f0 secure :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc58c00b9b0 tx=0x7fc58c008040 comp rx=0 tx=0).stop 2023-09-23T03:16:58.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.384+0000 7fc5a1fa9640 1 --2- 172.21.15.154:0/2470999623 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc59c160dc0 0x7fc59c15bd40 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:58.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.384+0000 7fc5a1fa9640 1 --2- 172.21.15.154:0/2470999623 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc59c15caf0 0x7fc59c15b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:58.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.384+0000 7fc5a1fa9640 1 --2- 172.21.15.154:0/2470999623 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc59c0cee70 0x7fc59c153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:58.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.384+0000 7fc5a1fa9640 1 -- 172.21.15.154:0/2470999623 >> 172.21.15.154:0/2470999623 conn(0x7fc59c06a0f0 msgr2=0x7fc59c0b0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:58.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.385+0000 7fc5a1fa9640 1 -- 172.21.15.154:0/2470999623 shutdown_connections 2023-09-23T03:16:58.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.385+0000 7fc5a1fa9640 1 -- 172.21.15.154:0/2470999623 wait complete. 2023-09-23T03:16:58.403 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-09-23T03:16:58.416 INFO:tasks.ceph.mds.d.smithi190.stderr: -5242> 2023-09-23T03:15:26.601+0000 7f8c03fbc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-23T03:16:58.512 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-09-23T03:16:58.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.613+0000 7f17e45e2640 1 Processor -- start 2023-09-23T03:16:58.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.614+0000 7f17e45e2640 1 -- start start 2023-09-23T03:16:58.615 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.615+0000 7f17e45e2640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f17dc0c9150 0x7f17dc15cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:58.615 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.615+0000 7f17e45e2640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17dc0c9d20 con 0x7f17dc0c8d70 2023-09-23T03:16:58.615 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.615+0000 7f17e45e2640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17dc071850 con 0x7f17dc071290 2023-09-23T03:16:58.615 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.615+0000 7f17e45e2640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17dc071b30 con 0x7f17dc0c9150 2023-09-23T03:16:58.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.615+0000 7f17e237e640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f17dc0c8d70 0x7f17dc15c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:54848/0 (socket says 172.21.15.154:54848) 2023-09-23T03:16:58.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.615+0000 7f17e237e640 1 -- 172.21.15.154:0/1210198764 learned_addr learned my addr 172.21.15.154:0/1210198764 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:58.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.615+0000 7f17e137c640 1 -- 172.21.15.154:0/1210198764 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2924674865 0 0) 0x7f17dc0c9d20 con 0x7f17dc0c8d70 2023-09-23T03:16:58.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.616+0000 7f17e137c640 1 -- 172.21.15.154:0/1210198764 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17c8003520 con 0x7f17dc0c8d70 2023-09-23T03:16:58.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.616+0000 7f17e1b7d640 1 --2- 172.21.15.154:0/1210198764 >> v2:172.21.15.190:3300/0 conn(0x7f17dc0c9150 0x7f17dc15cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:58.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.616+0000 7f17e137c640 1 -- 172.21.15.154:0/1210198764 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1318985320 0 0) 0x7f17c8003520 con 0x7f17dc0c8d70 2023-09-23T03:16:58.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.616+0000 7f17e137c640 1 -- 172.21.15.154:0/1210198764 >> v2:172.21.15.190:3300/0 conn(0x7f17dc0c9150 msgr2=0x7f17dc15cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:58.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.616+0000 7f17e137c640 1 --2- 172.21.15.154:0/1210198764 >> v2:172.21.15.190:3300/0 conn(0x7f17dc0c9150 0x7f17dc15cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:58.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.616+0000 7f17e137c640 1 -- 172.21.15.154:0/1210198764 >> v1:172.21.15.190:6789/0 conn(0x7f17dc071290 legacy=0x7f17dc0ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:58.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.616+0000 7f17e137c640 1 -- 172.21.15.154:0/1210198764 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17dc071e10 con 0x7f17dc0c8d70 2023-09-23T03:16:58.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.616+0000 7f17e137c640 1 -- 172.21.15.154:0/1210198764 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f17d8001670 con 0x7f17dc0c8d70 2023-09-23T03:16:58.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.616+0000 7f17e137c640 1 -- 172.21.15.154:0/1210198764 >> v1:172.21.15.154:6789/0 conn(0x7f17dc0c8d70 legacy=0x7f17dc15c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:58.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.616+0000 7f17e137c640 1 --2- 172.21.15.154:0/1210198764 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17c8003970 0x7f17c8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:58.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.616+0000 7f17e137c640 1 --2- 172.21.15.154:0/1210198764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17c80047c0 0x7f17c8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:58.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.616+0000 7f17e137c640 1 --2- 172.21.15.154:0/1210198764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17c8005180 0x7f17c800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:58.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.616+0000 7f17e137c640 1 -- 172.21.15.154:0/1210198764 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17dc071b30 con 0x7f17c80047c0 2023-09-23T03:16:58.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.616+0000 7f17e137c640 1 -- 172.21.15.154:0/1210198764 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c800c0d0 con 0x7f17c8005180 2023-09-23T03:16:58.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.616+0000 7f17e137c640 1 -- 172.21.15.154:0/1210198764 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c800c290 con 0x7f17c8003970 2023-09-23T03:16:58.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.616+0000 7f17e1b7d640 1 --2- 172.21.15.154:0/1210198764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17c80047c0 0x7f17c8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:58.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.617+0000 7f17e237e640 1 --2- 172.21.15.154:0/1210198764 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17c8003970 0x7f17c8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:58.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.617+0000 7f17e1b7d640 1 -- 172.21.15.154:0/1210198764 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17c8003970 msgr2=0x7f17c8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:58.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.617+0000 7f17e1b7d640 1 --2- 172.21.15.154:0/1210198764 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17c8003970 0x7f17c8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:58.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.617+0000 7f17e1b7d640 1 -- 172.21.15.154:0/1210198764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17c8005180 msgr2=0x7f17c800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:58.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.617+0000 7f17e2b7f640 1 --2- 172.21.15.154:0/1210198764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17c8005180 0x7f17c800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:58.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.617+0000 7f17e1b7d640 1 --2- 172.21.15.154:0/1210198764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17c8005180 0x7f17c800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:58.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.617+0000 7f17e1b7d640 1 -- 172.21.15.154:0/1210198764 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c8003520 con 0x7f17c80047c0 2023-09-23T03:16:58.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.617+0000 7f17e2b7f640 1 -- 172.21.15.154:0/1210198764 reap_dead start 2023-09-23T03:16:58.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.617+0000 7f17e237e640 1 --2- 172.21.15.154:0/1210198764 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17c8003970 0x7f17c8003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:58.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.618+0000 7f17e1b7d640 1 --2- 172.21.15.154:0/1210198764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17c80047c0 0x7f17c8004c40 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f17cc00b5c0 tx=0x7f17c80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1893dc18445e8372 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:58.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.618+0000 7f17e137c640 1 -- 172.21.15.154:0/1210198764 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17cc02e750 con 0x7f17c80047c0 2023-09-23T03:16:58.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.618+0000 7f17e137c640 1 -- 172.21.15.154:0/1210198764 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17cc032070 con 0x7f17c80047c0 2023-09-23T03:16:58.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.618+0000 7f17e137c640 1 -- 172.21.15.154:0/1210198764 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17cc02ead0 con 0x7f17c80047c0 2023-09-23T03:16:58.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.619+0000 7f17e45e2640 1 -- 172.21.15.154:0/1210198764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17c80047c0 msgr2=0x7f17c8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:58.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.619+0000 7f17e45e2640 1 --2- 172.21.15.154:0/1210198764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17c80047c0 0x7f17c8004c40 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f17cc00b5c0 tx=0x7f17c80057f0 comp rx=0 tx=0).stop 2023-09-23T03:16:58.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.619+0000 7f17e45e2640 1 -- 172.21.15.154:0/1210198764 shutdown_connections 2023-09-23T03:16:58.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.619+0000 7f17e45e2640 1 --2- 172.21.15.154:0/1210198764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17c80047c0 0x7f17c8004c40 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:58.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.619+0000 7f17e45e2640 1 -- 172.21.15.154:0/1210198764 >> 172.21.15.154:0/1210198764 conn(0x7f17dc06c010 msgr2=0x7f17dc06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:58.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.619+0000 7f17e45e2640 1 -- 172.21.15.154:0/1210198764 shutdown_connections 2023-09-23T03:16:58.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.619+0000 7f17e45e2640 1 -- 172.21.15.154:0/1210198764 wait complete. 2023-09-23T03:16:58.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.621+0000 7f17e45e2640 1 Processor -- start 2023-09-23T03:16:58.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.621+0000 7f17e45e2640 1 -- start start 2023-09-23T03:16:58.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.622+0000 7f17e45e2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17c80047c0 0x7f17dc14e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:58.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.622+0000 7f17e45e2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17dc0b2380 0x7f17dc0b2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:58.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.622+0000 7f17e45e2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17dc0c56d0 0x7f17dc0c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:58.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.622+0000 7f17e45e2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17dc0b2d40 con 0x7f17dc0b2380 2023-09-23T03:16:58.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.622+0000 7f17e45e2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17dc0c8970 con 0x7f17dc0c56d0 2023-09-23T03:16:58.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.622+0000 7f17e45e2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17dc0c78d0 con 0x7f17c80047c0 2023-09-23T03:16:58.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.622+0000 7f17e1b7d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17dc0b2380 0x7f17dc0b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:58.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.622+0000 7f17e1b7d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17dc0b2380 0x7f17dc0b2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56054/0 (socket says 172.21.15.154:56054) 2023-09-23T03:16:58.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.622+0000 7f17e1b7d640 1 -- 172.21.15.154:0/3084211199 learned_addr learned my addr 172.21.15.154:0/3084211199 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:58.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.622+0000 7f17e237e640 1 --2- 172.21.15.154:0/3084211199 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17c80047c0 0x7f17dc14e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:58.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.622+0000 7f17e2b7f640 1 --2- 172.21.15.154:0/3084211199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17dc0c56d0 0x7f17dc0c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:58.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.622+0000 7f17e1b7d640 1 -- 172.21.15.154:0/3084211199 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17c80047c0 msgr2=0x7f17dc14e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:58.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.622+0000 7f17e1b7d640 1 --2- 172.21.15.154:0/3084211199 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17c80047c0 0x7f17dc14e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:58.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.622+0000 7f17e1b7d640 1 -- 172.21.15.154:0/3084211199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17dc0c56d0 msgr2=0x7f17dc0c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:58.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.622+0000 7f17e1b7d640 1 --2- 172.21.15.154:0/3084211199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17dc0c56d0 0x7f17dc0c5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:58.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.622+0000 7f17e1b7d640 1 -- 172.21.15.154:0/3084211199 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17dc0c7bb0 con 0x7f17dc0b2380 2023-09-23T03:16:58.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.622+0000 7f17e2b7f640 1 --2- 172.21.15.154:0/3084211199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17dc0c56d0 0x7f17dc0c5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:16:58.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.623+0000 7f17e1b7d640 1 --2- 172.21.15.154:0/3084211199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17dc0b2380 0x7f17dc0b2800 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f17cc00baf0 tx=0x7f17cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:58.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.623+0000 7f17d2ffd640 1 -- 172.21.15.154:0/3084211199 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17cc046070 con 0x7f17dc0b2380 2023-09-23T03:16:58.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.623+0000 7f17d2ffd640 1 -- 172.21.15.154:0/3084211199 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17cc004020 con 0x7f17dc0b2380 2023-09-23T03:16:58.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.623+0000 7f17d2ffd640 1 -- 172.21.15.154:0/3084211199 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17cc004190 con 0x7f17dc0b2380 2023-09-23T03:16:58.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.623+0000 7f17e45e2640 1 -- 172.21.15.154:0/3084211199 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17dc0c7e90 con 0x7f17dc0b2380 2023-09-23T03:16:58.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.623+0000 7f17e45e2640 1 -- 172.21.15.154:0/3084211199 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17dc0c8450 con 0x7f17dc0b2380 2023-09-23T03:16:58.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.624+0000 7f17e45e2640 1 -- 172.21.15.154:0/3084211199 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17dc04e920 con 0x7f17dc0b2380 2023-09-23T03:16:58.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.625+0000 7f17d2ffd640 1 -- 172.21.15.154:0/3084211199 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f17cc0042f0 con 0x7f17dc0b2380 2023-09-23T03:16:58.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.626+0000 7f17d2ffd640 1 --2- 172.21.15.154:0/3084211199 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f17c006af60 0x7f17c006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:58.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.626+0000 7f17d2ffd640 1 -- 172.21.15.154:0/3084211199 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6728+0+0 (secure 0 0 0) 0x7f17cc0ab770 con 0x7f17dc0b2380 2023-09-23T03:16:58.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.626+0000 7f17e237e640 1 --2- 172.21.15.154:0/3084211199 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f17c006af60 0x7f17c006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:58.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.627+0000 7f17e237e640 1 --2- 172.21.15.154:0/3084211199 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f17c006af60 0x7f17c006d420 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f17dc074b80 tx=0x7f17d8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:58.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.630+0000 7f17d2ffd640 1 -- 172.21.15.154:0/3084211199 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f17cc0763a0 con 0x7f17dc0b2380 2023-09-23T03:16:58.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:58.861+0000 7f17e45e2640 1 -- 172.21.15.154:0/3084211199 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f17dc06cbc0 con 0x7f17dc0b2380 2023-09-23T03:16:59.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.516+0000 7f17d2ffd640 1 -- 172.21.15.154:0/3084211199 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f17cc0799c0 con 0x7f17dc0b2380 2023-09-23T03:16:59.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.520+0000 7f17e45e2640 1 -- 172.21.15.154:0/3084211199 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f17c006af60 msgr2=0x7f17c006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:59.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.520+0000 7f17e45e2640 1 --2- 172.21.15.154:0/3084211199 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f17c006af60 0x7f17c006d420 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f17dc074b80 tx=0x7f17d8009040 comp rx=0 tx=0).stop 2023-09-23T03:16:59.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.520+0000 7f17e45e2640 1 -- 172.21.15.154:0/3084211199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17dc0b2380 msgr2=0x7f17dc0b2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:59.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.520+0000 7f17e45e2640 1 --2- 172.21.15.154:0/3084211199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17dc0b2380 0x7f17dc0b2800 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f17cc00baf0 tx=0x7f17cc004730 comp rx=0 tx=0).stop 2023-09-23T03:16:59.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.520+0000 7f17e45e2640 1 -- 172.21.15.154:0/3084211199 shutdown_connections 2023-09-23T03:16:59.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.520+0000 7f17e45e2640 1 --2- 172.21.15.154:0/3084211199 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f17c006af60 0x7f17c006d420 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:59.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.520+0000 7f17e45e2640 1 --2- 172.21.15.154:0/3084211199 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f17dc0c56d0 0x7f17dc0c5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:59.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.520+0000 7f17e45e2640 1 --2- 172.21.15.154:0/3084211199 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17dc0b2380 0x7f17dc0b2800 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:59.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.520+0000 7f17e45e2640 1 --2- 172.21.15.154:0/3084211199 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f17c80047c0 0x7f17dc14e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:59.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.520+0000 7f17e45e2640 1 -- 172.21.15.154:0/3084211199 >> 172.21.15.154:0/3084211199 conn(0x7f17dc06c010 msgr2=0x7f17dc152050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:59.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.520+0000 7f17e45e2640 1 -- 172.21.15.154:0/3084211199 shutdown_connections 2023-09-23T03:16:59.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.520+0000 7f17e45e2640 1 -- 172.21.15.154:0/3084211199 wait complete. 2023-09-23T03:16:59.542 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-09-23T03:16:59.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.764+0000 7fed824da640 1 Processor -- start 2023-09-23T03:16:59.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.765+0000 7fed824da640 1 -- start start 2023-09-23T03:16:59.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.766+0000 7fed824da640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed7c0700f0 0x7fed7c0c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:59.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.766+0000 7fed824da640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fed7c15abd0 0x7fed7c0cb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:59.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.766+0000 7fed824da640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed7c0cb9a0 con 0x7fed7c0704d0 2023-09-23T03:16:59.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.766+0000 7fed824da640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed7c0b2430 con 0x7fed7c15abd0 2023-09-23T03:16:59.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.766+0000 7fed824da640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c0c95b0 con 0x7fed7c0700f0 2023-09-23T03:16:59.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.766+0000 7fed73fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fed7c15abd0 0x7fed7c0cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:59.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.767+0000 7fed80a77640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fed7c0704d0 0x7fed7c0ce3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.154:51858/0 (socket says 172.21.15.154:51858) 2023-09-23T03:16:59.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.767+0000 7fed80a77640 1 -- 172.21.15.154:0/450504686 learned_addr learned my addr 172.21.15.154:0/450504686 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:16:59.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.767+0000 7fed7bfff640 1 --2- 172.21.15.154:0/450504686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed7c0700f0 0x7fed7c0c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:59.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.767+0000 7fed73fff640 1 -- 172.21.15.154:0/450504686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed7c0700f0 msgr2=0x7fed7c0c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:16:59.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.767+0000 7fed73fff640 1 --2- 172.21.15.154:0/450504686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed7c0700f0 0x7fed7c0c8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:59.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.767+0000 7fed7b7fe640 1 -- 172.21.15.154:0/450504686 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1564054008 0 0) 0x7fed7c0cb9a0 con 0x7fed7c0704d0 2023-09-23T03:16:59.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.767+0000 7fed73fff640 1 -- 172.21.15.154:0/450504686 >> v1:172.21.15.190:6789/0 conn(0x7fed7c0704d0 legacy=0x7fed7c0ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:59.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.767+0000 7fed73fff640 1 -- 172.21.15.154:0/450504686 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed7c0c9890 con 0x7fed7c15abd0 2023-09-23T03:16:59.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.769+0000 7fed73fff640 1 --2- 172.21.15.154:0/450504686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed7c15abd0 0x7fed7c0cb460 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fed640097a0 tx=0x7fed6402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aec52b7d6a4c084 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:59.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.769+0000 7fed7b7fe640 1 -- 172.21.15.154:0/450504686 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6402d040 con 0x7fed7c15abd0 2023-09-23T03:16:59.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.769+0000 7fed7b7fe640 1 -- 172.21.15.154:0/450504686 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed64031810 con 0x7fed7c15abd0 2023-09-23T03:16:59.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.769+0000 7fed7b7fe640 1 -- 172.21.15.154:0/450504686 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed64031ba0 con 0x7fed7c15abd0 2023-09-23T03:16:59.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.770+0000 7fed824da640 1 -- 172.21.15.154:0/450504686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed7c15abd0 msgr2=0x7fed7c0cb460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:59.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.770+0000 7fed824da640 1 --2- 172.21.15.154:0/450504686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed7c15abd0 0x7fed7c0cb460 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fed640097a0 tx=0x7fed6402b8b0 comp rx=0 tx=0).stop 2023-09-23T03:16:59.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.770+0000 7fed824da640 1 -- 172.21.15.154:0/450504686 shutdown_connections 2023-09-23T03:16:59.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.770+0000 7fed824da640 1 --2- 172.21.15.154:0/450504686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed7c15abd0 0x7fed7c0cb460 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:59.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.770+0000 7fed824da640 1 --2- 172.21.15.154:0/450504686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed7c0700f0 0x7fed7c0c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:59.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.770+0000 7fed824da640 1 -- 172.21.15.154:0/450504686 >> 172.21.15.154:0/450504686 conn(0x7fed7c06a520 msgr2=0x7fed7c06bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:16:59.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.770+0000 7fed824da640 1 -- 172.21.15.154:0/450504686 shutdown_connections 2023-09-23T03:16:59.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.770+0000 7fed824da640 1 -- 172.21.15.154:0/450504686 wait complete. 2023-09-23T03:16:59.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.772+0000 7fed824da640 1 Processor -- start 2023-09-23T03:16:59.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.772+0000 7fed824da640 1 -- start start 2023-09-23T03:16:59.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.773+0000 7fed824da640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed7c0700f0 0x7fed7c156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:59.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.773+0000 7fed824da640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed7c0704d0 0x7fed7c0ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:59.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.773+0000 7fed824da640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed7c15abd0 0x7fed7c0ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:59.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.773+0000 7fed824da640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c0ae4b0 con 0x7fed7c15abd0 2023-09-23T03:16:59.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.773+0000 7fed824da640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c0ae790 con 0x7fed7c0704d0 2023-09-23T03:16:59.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.773+0000 7fed824da640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c0aea70 con 0x7fed7c0700f0 2023-09-23T03:16:59.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.773+0000 7fed80a77640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed7c15abd0 0x7fed7c0ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:59.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.773+0000 7fed80a77640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed7c15abd0 0x7fed7c0ad5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56074/0 (socket says 172.21.15.154:56074) 2023-09-23T03:16:59.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.773+0000 7fed80a77640 1 -- 172.21.15.154:0/3516237473 learned_addr learned my addr 172.21.15.154:0/3516237473 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:16:59.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.773+0000 7fed73fff640 1 --2- 172.21.15.154:0/3516237473 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed7c0704d0 0x7fed7c0ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:59.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.773+0000 7fed7bfff640 1 --2- 172.21.15.154:0/3516237473 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed7c0700f0 0x7fed7c156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:59.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.774+0000 7fed80a77640 1 -- 172.21.15.154:0/3516237473 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed7c0700f0 msgr2=0x7fed7c156490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:59.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.774+0000 7fed80a77640 1 --2- 172.21.15.154:0/3516237473 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed7c0700f0 0x7fed7c156490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:59.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.774+0000 7fed80a77640 1 -- 172.21.15.154:0/3516237473 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed7c0704d0 msgr2=0x7fed7c0ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:16:59.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.774+0000 7fed80a77640 1 --2- 172.21.15.154:0/3516237473 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed7c0704d0 0x7fed7c0ab8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:16:59.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.774+0000 7fed80a77640 1 -- 172.21.15.154:0/3516237473 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed64009400 con 0x7fed7c15abd0 2023-09-23T03:16:59.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.774+0000 7fed80a77640 1 --2- 172.21.15.154:0/3516237473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed7c15abd0 0x7fed7c0ad5a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fed7400c7a0 tx=0x7fed7400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:59.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.774+0000 7fed797fa640 1 -- 172.21.15.154:0/3516237473 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7400a640 con 0x7fed7c15abd0 2023-09-23T03:16:59.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.774+0000 7fed797fa640 1 -- 172.21.15.154:0/3516237473 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed7400a7a0 con 0x7fed7c15abd0 2023-09-23T03:16:59.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.774+0000 7fed797fa640 1 -- 172.21.15.154:0/3516237473 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7400abd0 con 0x7fed7c15abd0 2023-09-23T03:16:59.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.774+0000 7fed824da640 1 -- 172.21.15.154:0/3516237473 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed7c0abe00 con 0x7fed7c15abd0 2023-09-23T03:16:59.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.774+0000 7fed824da640 1 -- 172.21.15.154:0/3516237473 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed7c156cd0 con 0x7fed7c15abd0 2023-09-23T03:16:59.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.775+0000 7fed824da640 1 -- 172.21.15.154:0/3516237473 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed7c0ae5f0 con 0x7fed7c15abd0 2023-09-23T03:16:59.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.776+0000 7fed797fa640 1 -- 172.21.15.154:0/3516237473 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fed7400ad30 con 0x7fed7c15abd0 2023-09-23T03:16:59.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.777+0000 7fed797fa640 1 --2- 172.21.15.154:0/3516237473 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fed5c06af30 0x7fed5c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:16:59.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.778+0000 7fed797fa640 1 -- 172.21.15.154:0/3516237473 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6728+0+0 (secure 0 0 0) 0x7fed74089080 con 0x7fed7c15abd0 2023-09-23T03:16:59.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.778+0000 7fed7bfff640 1 --2- 172.21.15.154:0/3516237473 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fed5c06af30 0x7fed5c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:16:59.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.779+0000 7fed7bfff640 1 --2- 172.21.15.154:0/3516237473 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fed5c06af30 0x7fed5c06d3f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0cd890 tx=0x7fed6c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:16:59.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.783+0000 7fed797fa640 1 -- 172.21.15.154:0/3516237473 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fed740bba50 con 0x7fed7c15abd0 2023-09-23T03:16:59.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:16:59.976+0000 7fed824da640 1 -- 172.21.15.154:0/3516237473 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fed7c15a350 con 0x7fed7c15abd0 2023-09-23T03:17:00.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.520+0000 7fed797fa640 1 -- 172.21.15.154:0/3516237473 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) v1 ==== 181+0+0 (secure 0 0 0) 0x7fed74018070 con 0x7fed7c15abd0 2023-09-23T03:17:00.521 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_metadata' removed 2023-09-23T03:17:00.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.524+0000 7fed824da640 1 -- 172.21.15.154:0/3516237473 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fed5c06af30 msgr2=0x7fed5c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:00.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.524+0000 7fed824da640 1 --2- 172.21.15.154:0/3516237473 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fed5c06af30 0x7fed5c06d3f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0cd890 tx=0x7fed6c008040 comp rx=0 tx=0).stop 2023-09-23T03:17:00.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.524+0000 7fed824da640 1 -- 172.21.15.154:0/3516237473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed7c15abd0 msgr2=0x7fed7c0ad5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:00.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.524+0000 7fed824da640 1 --2- 172.21.15.154:0/3516237473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed7c15abd0 0x7fed7c0ad5a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fed7400c7a0 tx=0x7fed7400cc60 comp rx=0 tx=0).stop 2023-09-23T03:17:00.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.525+0000 7fed824da640 1 -- 172.21.15.154:0/3516237473 shutdown_connections 2023-09-23T03:17:00.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.525+0000 7fed824da640 1 --2- 172.21.15.154:0/3516237473 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fed5c06af30 0x7fed5c06d3f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:00.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.525+0000 7fed824da640 1 --2- 172.21.15.154:0/3516237473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed7c15abd0 0x7fed7c0ad5a0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:00.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.525+0000 7fed824da640 1 --2- 172.21.15.154:0/3516237473 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed7c0704d0 0x7fed7c0ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:00.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.525+0000 7fed824da640 1 --2- 172.21.15.154:0/3516237473 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed7c0700f0 0x7fed7c156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:00.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.525+0000 7fed824da640 1 -- 172.21.15.154:0/3516237473 >> 172.21.15.154:0/3516237473 conn(0x7fed7c06a520 msgr2=0x7fed7c0b0750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:00.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.525+0000 7fed824da640 1 -- 172.21.15.154:0/3516237473 shutdown_connections 2023-09-23T03:17:00.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.525+0000 7fed824da640 1 -- 172.21.15.154:0/3516237473 wait complete. 2023-09-23T03:17:00.550 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-09-23T03:17:00.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.760+0000 7ff3a5694640 1 Processor -- start 2023-09-23T03:17:00.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.760+0000 7ff3a5694640 1 -- start start 2023-09-23T03:17:00.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.762+0000 7ff3a5694640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff3a00700f0 0x7ff3a00c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:00.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.762+0000 7ff3a5694640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff3a00cb460 0x7ff3a00b1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:00.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.762+0000 7ff3a5694640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3a00c95b0 con 0x7ff3a015abd0 2023-09-23T03:17:00.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.762+0000 7ff3a5694640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3a00c9890 con 0x7ff3a00cb460 2023-09-23T03:17:00.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.762+0000 7ff3a5694640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3a00c9b70 con 0x7ff3a00700f0 2023-09-23T03:17:00.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.762+0000 7ff39f7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff3a00cb460 0x7ff3a00b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:00.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.762+0000 7ff39e7fc640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff3a015abd0 0x7ff3a00704d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:54854/0 (socket says 172.21.15.154:54854) 2023-09-23T03:17:00.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.762+0000 7ff39e7fc640 1 -- 172.21.15.154:0/306405773 learned_addr learned my addr 172.21.15.154:0/306405773 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:17:00.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.762+0000 7ff39f7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff3a00cb460 0x7ff3a00b1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56076/0 (socket says 172.21.15.154:56076) 2023-09-23T03:17:00.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.763+0000 7ff39effd640 1 --2- 172.21.15.154:0/306405773 >> v2:172.21.15.190:3300/0 conn(0x7ff3a00700f0 0x7ff3a00c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:00.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.763+0000 7ff39f7fe640 1 -- 172.21.15.154:0/306405773 >> v2:172.21.15.190:3300/0 conn(0x7ff3a00700f0 msgr2=0x7ff3a00c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:00.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.763+0000 7ff39f7fe640 1 --2- 172.21.15.154:0/306405773 >> v2:172.21.15.190:3300/0 conn(0x7ff3a00700f0 0x7ff3a00c8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:00.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.763+0000 7ff39dffb640 1 -- 172.21.15.154:0/306405773 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1498997990 0 0) 0x7ff3a00c95b0 con 0x7ff3a015abd0 2023-09-23T03:17:00.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.763+0000 7ff39f7fe640 1 -- 172.21.15.154:0/306405773 >> v1:172.21.15.154:6789/0 conn(0x7ff3a015abd0 legacy=0x7ff3a00704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:00.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.763+0000 7ff39f7fe640 1 -- 172.21.15.154:0/306405773 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a00c9e50 con 0x7ff3a00cb460 2023-09-23T03:17:00.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.764+0000 7ff39f7fe640 1 --2- 172.21.15.154:0/306405773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a00cb460 0x7ff3a00b1420 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff3a00cbc40 tx=0x7ff39802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ae5de39475654 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:00.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.765+0000 7ff39dffb640 1 -- 172.21.15.154:0/306405773 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39802cc80 con 0x7ff3a00cb460 2023-09-23T03:17:00.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.765+0000 7ff39dffb640 1 -- 172.21.15.154:0/306405773 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff39802cde0 con 0x7ff3a00cb460 2023-09-23T03:17:00.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.765+0000 7ff39dffb640 1 -- 172.21.15.154:0/306405773 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff398034660 con 0x7ff3a00cb460 2023-09-23T03:17:00.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.765+0000 7ff3a5694640 1 -- 172.21.15.154:0/306405773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a00cb460 msgr2=0x7ff3a00b1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:00.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.765+0000 7ff3a5694640 1 --2- 172.21.15.154:0/306405773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a00cb460 0x7ff3a00b1420 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff3a00cbc40 tx=0x7ff39802bed0 comp rx=0 tx=0).stop 2023-09-23T03:17:00.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.766+0000 7ff3a5694640 1 -- 172.21.15.154:0/306405773 shutdown_connections 2023-09-23T03:17:00.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.766+0000 7ff3a5694640 1 --2- 172.21.15.154:0/306405773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a00cb460 0x7ff3a00b1420 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:00.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.766+0000 7ff3a5694640 1 --2- 172.21.15.154:0/306405773 >> v2:172.21.15.190:3300/0 conn(0x7ff3a00700f0 0x7ff3a00c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:00.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.766+0000 7ff3a5694640 1 -- 172.21.15.154:0/306405773 >> 172.21.15.154:0/306405773 conn(0x7ff3a006a520 msgr2=0x7ff3a006bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:00.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.766+0000 7ff3a5694640 1 -- 172.21.15.154:0/306405773 shutdown_connections 2023-09-23T03:17:00.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.766+0000 7ff3a5694640 1 -- 172.21.15.154:0/306405773 wait complete. 2023-09-23T03:17:00.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.767+0000 7ff3a5694640 1 Processor -- start 2023-09-23T03:17:00.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.768+0000 7ff3a5694640 1 -- start start 2023-09-23T03:17:00.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.769+0000 7ff3a5694640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3a00700f0 0x7ff3a01efa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:00.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.769+0000 7ff3a5694640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a00cb460 0x7ff3a0145ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:00.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.769+0000 7ff3a5694640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3a015abd0 0x7ff3a0146530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:00.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.769+0000 7ff3a5694640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a01f03c0 con 0x7ff3a00cb460 2023-09-23T03:17:00.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.769+0000 7ff3a5694640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a01f06a0 con 0x7ff3a00700f0 2023-09-23T03:17:00.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.769+0000 7ff3a5694640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a01f0980 con 0x7ff3a015abd0 2023-09-23T03:17:00.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.769+0000 7ff39e7fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a00cb460 0x7ff3a0145ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:00.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.769+0000 7ff39e7fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a00cb460 0x7ff3a0145ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56088/0 (socket says 172.21.15.154:56088) 2023-09-23T03:17:00.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.769+0000 7ff39e7fc640 1 -- 172.21.15.154:0/1209607207 learned_addr learned my addr 172.21.15.154:0/1209607207 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:00.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.769+0000 7ff39effd640 1 --2- 172.21.15.154:0/1209607207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3a00700f0 0x7ff3a01efa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:00.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.769+0000 7ff39e7fc640 1 -- 172.21.15.154:0/1209607207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3a015abd0 msgr2=0x7ff3a0146530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:00.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.769+0000 7ff39e7fc640 1 --2- 172.21.15.154:0/1209607207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3a015abd0 0x7ff3a0146530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:00.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.769+0000 7ff39e7fc640 1 -- 172.21.15.154:0/1209607207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3a00700f0 msgr2=0x7ff3a01efa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:00.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.769+0000 7ff39e7fc640 1 --2- 172.21.15.154:0/1209607207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3a00700f0 0x7ff3a01efa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:00.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.769+0000 7ff39e7fc640 1 -- 172.21.15.154:0/1209607207 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff398009a70 con 0x7ff3a00cb460 2023-09-23T03:17:00.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.770+0000 7ff39e7fc640 1 --2- 172.21.15.154:0/1209607207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a00cb460 0x7ff3a0145ff0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff38800c410 tx=0x7ff38800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:00.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.770+0000 7ff3977fe640 1 -- 172.21.15.154:0/1209607207 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3880074e0 con 0x7ff3a00cb460 2023-09-23T03:17:00.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.770+0000 7ff3977fe640 1 -- 172.21.15.154:0/1209607207 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff388007660 con 0x7ff3a00cb460 2023-09-23T03:17:00.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.770+0000 7ff3977fe640 1 -- 172.21.15.154:0/1209607207 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff388004a20 con 0x7ff3a00cb460 2023-09-23T03:17:00.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.770+0000 7ff3a5694640 1 -- 172.21.15.154:0/1209607207 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3a00c3070 con 0x7ff3a00cb460 2023-09-23T03:17:00.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.770+0000 7ff3a5694640 1 -- 172.21.15.154:0/1209607207 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a01e10c0 con 0x7ff3a00cb460 2023-09-23T03:17:00.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.771+0000 7ff3a5694640 1 -- 172.21.15.154:0/1209607207 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3a00c3590 con 0x7ff3a00cb460 2023-09-23T03:17:00.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.772+0000 7ff3977fe640 1 -- 172.21.15.154:0/1209607207 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7ff388004b80 con 0x7ff3a00cb460 2023-09-23T03:17:00.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.773+0000 7ff3977fe640 1 --2- 172.21.15.154:0/1209607207 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff38006af30 0x7ff38006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:00.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.773+0000 7ff3977fe640 1 -- 172.21.15.154:0/1209607207 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6290+0+0 (secure 0 0 0) 0x7ff388089870 con 0x7ff3a00cb460 2023-09-23T03:17:00.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.774+0000 7ff39effd640 1 --2- 172.21.15.154:0/1209607207 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff38006af30 0x7ff38006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:00.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.774+0000 7ff39effd640 1 --2- 172.21.15.154:0/1209607207 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff38006af30 0x7ff38006d3f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff390009ad0 tx=0x7ff390008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:00.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.778+0000 7ff3977fe640 1 -- 172.21.15.154:0/1209607207 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7ff3a00c3590 con 0x7ff3a00cb460 2023-09-23T03:17:00.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:00.957+0000 7ff3a5694640 1 -- 172.21.15.154:0/1209607207 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff3a00c3590 con 0x7ff3a00cb460 2023-09-23T03:17:01.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.551+0000 7ff3977fe640 1 -- 172.21.15.154:0/1209607207 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff3a00c3590 con 0x7ff3a00cb460 2023-09-23T03:17:01.552 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_data' removed 2023-09-23T03:17:01.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.554+0000 7ff3a5694640 1 -- 172.21.15.154:0/1209607207 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff38006af30 msgr2=0x7ff38006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:01.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.554+0000 7ff3a5694640 1 --2- 172.21.15.154:0/1209607207 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff38006af30 0x7ff38006d3f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff390009ad0 tx=0x7ff390008040 comp rx=0 tx=0).stop 2023-09-23T03:17:01.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.554+0000 7ff3a5694640 1 -- 172.21.15.154:0/1209607207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a00cb460 msgr2=0x7ff3a0145ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:01.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.554+0000 7ff3a5694640 1 --2- 172.21.15.154:0/1209607207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a00cb460 0x7ff3a0145ff0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff38800c410 tx=0x7ff38800c8d0 comp rx=0 tx=0).stop 2023-09-23T03:17:01.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.555+0000 7ff3a5694640 1 -- 172.21.15.154:0/1209607207 shutdown_connections 2023-09-23T03:17:01.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.555+0000 7ff3a5694640 1 --2- 172.21.15.154:0/1209607207 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff38006af30 0x7ff38006d3f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:01.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.555+0000 7ff3a5694640 1 --2- 172.21.15.154:0/1209607207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff3a015abd0 0x7ff3a0146530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:01.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.555+0000 7ff3a5694640 1 --2- 172.21.15.154:0/1209607207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff3a00cb460 0x7ff3a0145ff0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:01.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.555+0000 7ff3a5694640 1 --2- 172.21.15.154:0/1209607207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff3a00700f0 0x7ff3a01efa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:01.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.555+0000 7ff3a5694640 1 -- 172.21.15.154:0/1209607207 >> 172.21.15.154:0/1209607207 conn(0x7ff3a006a520 msgr2=0x7ff3a00ce340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:01.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.555+0000 7ff3a5694640 1 -- 172.21.15.154:0/1209607207 shutdown_connections 2023-09-23T03:17:01.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.555+0000 7ff3a5694640 1 -- 172.21.15.154:0/1209607207 wait complete. 2023-09-23T03:17:01.579 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-09-23T03:17:01.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.788+0000 7fbe5b083640 1 Processor -- start 2023-09-23T03:17:01.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.788+0000 7fbe5b083640 1 -- start start 2023-09-23T03:17:01.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.790+0000 7fbe5b083640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe540c8fd0 0x7fbe540cb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:01.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.790+0000 7fbe5b083640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe540cb9a0 con 0x7fbe540700f0 2023-09-23T03:17:01.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.790+0000 7fbe5b083640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe540c95b0 con 0x7fbe5415abd0 2023-09-23T03:17:01.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.790+0000 7fbe5b083640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe540c9890 con 0x7fbe540c8fd0 2023-09-23T03:17:01.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.790+0000 7fbe58e1f640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fbe540700f0 0x7fbe540704d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:54864/0 (socket says 172.21.15.154:54864) 2023-09-23T03:17:01.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.790+0000 7fbe58e1f640 1 -- 172.21.15.154:0/1172315332 learned_addr learned my addr 172.21.15.154:0/1172315332 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:17:01.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.790+0000 7fbe537fe640 1 -- 172.21.15.154:0/1172315332 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3852684027 0 0) 0x7fbe540cb9a0 con 0x7fbe540700f0 2023-09-23T03:17:01.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.790+0000 7fbe537fe640 1 -- 172.21.15.154:0/1172315332 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe44003520 con 0x7fbe540700f0 2023-09-23T03:17:01.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.790+0000 7fbe59620640 1 --2- 172.21.15.154:0/1172315332 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe540c8fd0 0x7fbe540cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:01.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.791+0000 7fbe537fe640 1 -- 172.21.15.154:0/1172315332 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 773538043 0 0) 0x7fbe44003520 con 0x7fbe540700f0 2023-09-23T03:17:01.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.791+0000 7fbe537fe640 1 -- 172.21.15.154:0/1172315332 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe540c8fd0 msgr2=0x7fbe540cb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:01.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.791+0000 7fbe537fe640 1 --2- 172.21.15.154:0/1172315332 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe540c8fd0 0x7fbe540cb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:01.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.791+0000 7fbe537fe640 1 -- 172.21.15.154:0/1172315332 >> v1:172.21.15.190:6789/0 conn(0x7fbe5415abd0 legacy=0x7fbe540b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:01.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.791+0000 7fbe537fe640 1 -- 172.21.15.154:0/1172315332 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe540c9b70 con 0x7fbe540700f0 2023-09-23T03:17:01.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.791+0000 7fbe537fe640 1 -- 172.21.15.154:0/1172315332 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fbe48001670 con 0x7fbe540700f0 2023-09-23T03:17:01.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.791+0000 7fbe537fe640 1 -- 172.21.15.154:0/1172315332 >> v1:172.21.15.154:6789/0 conn(0x7fbe540700f0 legacy=0x7fbe540704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:01.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.791+0000 7fbe537fe640 1 --2- 172.21.15.154:0/1172315332 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe540c8fd0 0x7fbe440039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:01.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.791+0000 7fbe537fe640 1 --2- 172.21.15.154:0/1172315332 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe44003f70 0x7fbe440047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:01.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.791+0000 7fbe537fe640 1 --2- 172.21.15.154:0/1172315332 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe44004d00 0x7fbe44005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:01.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.791+0000 7fbe537fe640 1 -- 172.21.15.154:0/1172315332 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe540c9890 con 0x7fbe44003f70 2023-09-23T03:17:01.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.791+0000 7fbe537fe640 1 -- 172.21.15.154:0/1172315332 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4400bac0 con 0x7fbe44004d00 2023-09-23T03:17:01.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.791+0000 7fbe537fe640 1 -- 172.21.15.154:0/1172315332 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4400bc80 con 0x7fbe540c8fd0 2023-09-23T03:17:01.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.791+0000 7fbe53fff640 1 --2- 172.21.15.154:0/1172315332 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe44003f70 0x7fbe440047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:01.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.792+0000 7fbe53fff640 1 -- 172.21.15.154:0/1172315332 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe540c8fd0 msgr2=0x7fbe440039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:01.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.792+0000 7fbe58e1f640 1 --2- 172.21.15.154:0/1172315332 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe540c8fd0 0x7fbe440039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:01.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.792+0000 7fbe53fff640 1 --2- 172.21.15.154:0/1172315332 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe540c8fd0 0x7fbe440039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:01.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.792+0000 7fbe59620640 1 --2- 172.21.15.154:0/1172315332 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe44004d00 0x7fbe44005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:01.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.792+0000 7fbe53fff640 1 -- 172.21.15.154:0/1172315332 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe44004d00 msgr2=0x7fbe44005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:01.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.792+0000 7fbe53fff640 1 --2- 172.21.15.154:0/1172315332 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe44004d00 0x7fbe44005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:01.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.792+0000 7fbe53fff640 1 -- 172.21.15.154:0/1172315332 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe44003520 con 0x7fbe44003f70 2023-09-23T03:17:01.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.793+0000 7fbe53fff640 1 --2- 172.21.15.154:0/1172315332 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe44003f70 0x7fbe440047c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fbe40001720 tx=0x7fbe440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa977d9cd534b25 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:01.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.793+0000 7fbe537fe640 1 -- 172.21.15.154:0/1172315332 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe400052d0 con 0x7fbe44003f70 2023-09-23T03:17:01.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.793+0000 7fbe537fe640 1 -- 172.21.15.154:0/1172315332 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe4002e050 con 0x7fbe44003f70 2023-09-23T03:17:01.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.793+0000 7fbe537fe640 1 -- 172.21.15.154:0/1172315332 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe40003830 con 0x7fbe44003f70 2023-09-23T03:17:01.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.794+0000 7fbe5b083640 1 -- 172.21.15.154:0/1172315332 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe44003f70 msgr2=0x7fbe440047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:01.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.794+0000 7fbe5b083640 1 --2- 172.21.15.154:0/1172315332 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe44003f70 0x7fbe440047c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fbe40001720 tx=0x7fbe440057f0 comp rx=0 tx=0).stop 2023-09-23T03:17:01.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.794+0000 7fbe59620640 1 -- 172.21.15.154:0/1172315332 reap_dead start 2023-09-23T03:17:01.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.794+0000 7fbe5b083640 1 -- 172.21.15.154:0/1172315332 shutdown_connections 2023-09-23T03:17:01.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.794+0000 7fbe5b083640 1 -- 172.21.15.154:0/1172315332 >> 172.21.15.154:0/1172315332 conn(0x7fbe5406a520 msgr2=0x7fbe5406bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:01.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.794+0000 7fbe5b083640 1 -- 172.21.15.154:0/1172315332 shutdown_connections 2023-09-23T03:17:01.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.794+0000 7fbe5b083640 1 -- 172.21.15.154:0/1172315332 wait complete. 2023-09-23T03:17:01.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.796+0000 7fbe5b083640 1 Processor -- start 2023-09-23T03:17:01.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.796+0000 7fbe5b083640 1 -- start start 2023-09-23T03:17:01.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.797+0000 7fbe5b083640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe44003f70 0x7fbe541e0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:01.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.797+0000 7fbe5b083640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe54154650 0x7fbe541e5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:01.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.797+0000 7fbe5b083640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe54155f40 0x7fbe540c5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:01.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.797+0000 7fbe5b083640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe54154ea0 con 0x7fbe54154650 2023-09-23T03:17:01.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.797+0000 7fbe5b083640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe54155180 con 0x7fbe54155f40 2023-09-23T03:17:01.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.797+0000 7fbe5b083640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe54155460 con 0x7fbe44003f70 2023-09-23T03:17:01.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.797+0000 7fbe53fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe54154650 0x7fbe541e5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:01.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.797+0000 7fbe53fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe54154650 0x7fbe541e5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56100/0 (socket says 172.21.15.154:56100) 2023-09-23T03:17:01.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.797+0000 7fbe53fff640 1 -- 172.21.15.154:0/3374905505 learned_addr learned my addr 172.21.15.154:0/3374905505 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:01.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.797+0000 7fbe58e1f640 1 --2- 172.21.15.154:0/3374905505 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe44003f70 0x7fbe541e0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:01.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.797+0000 7fbe59620640 1 --2- 172.21.15.154:0/3374905505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe54155f40 0x7fbe540c5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:01.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.797+0000 7fbe53fff640 1 -- 172.21.15.154:0/3374905505 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe44003f70 msgr2=0x7fbe541e0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:01.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.797+0000 7fbe53fff640 1 --2- 172.21.15.154:0/3374905505 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe44003f70 0x7fbe541e0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:01.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.797+0000 7fbe53fff640 1 -- 172.21.15.154:0/3374905505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe54155f40 msgr2=0x7fbe540c5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:01.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.797+0000 7fbe53fff640 1 --2- 172.21.15.154:0/3374905505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe54155f40 0x7fbe540c5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:01.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.797+0000 7fbe53fff640 1 -- 172.21.15.154:0/3374905505 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe54155740 con 0x7fbe54154650 2023-09-23T03:17:01.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.797+0000 7fbe59620640 1 --2- 172.21.15.154:0/3374905505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe54155f40 0x7fbe540c5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:01.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.798+0000 7fbe53fff640 1 --2- 172.21.15.154:0/3374905505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe54154650 0x7fbe541e5e30 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fbe40008fd0 tx=0x7fbe40001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:01.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.798+0000 7fbe517fa640 1 -- 172.21.15.154:0/3374905505 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe40007a30 con 0x7fbe54154650 2023-09-23T03:17:01.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.798+0000 7fbe517fa640 1 -- 172.21.15.154:0/3374905505 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe400069e0 con 0x7fbe54154650 2023-09-23T03:17:01.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.798+0000 7fbe5b083640 1 -- 172.21.15.154:0/3374905505 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe54155a20 con 0x7fbe54154650 2023-09-23T03:17:01.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.798+0000 7fbe517fa640 1 -- 172.21.15.154:0/3374905505 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe40006d50 con 0x7fbe54154650 2023-09-23T03:17:01.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.798+0000 7fbe5b083640 1 -- 172.21.15.154:0/3374905505 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe541e6370 con 0x7fbe54154650 2023-09-23T03:17:01.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.799+0000 7fbe5b083640 1 -- 172.21.15.154:0/3374905505 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe5404e920 con 0x7fbe54154650 2023-09-23T03:17:01.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.800+0000 7fbe517fa640 1 -- 172.21.15.154:0/3374905505 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fbe400035f0 con 0x7fbe54154650 2023-09-23T03:17:01.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.801+0000 7fbe517fa640 1 --2- 172.21.15.154:0/3374905505 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbe3806af60 0x7fbe3806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:01.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.801+0000 7fbe517fa640 1 -- 172.21.15.154:0/3374905505 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5781+0+0 (secure 0 0 0) 0x7fbe400ab9c0 con 0x7fbe54154650 2023-09-23T03:17:01.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.801+0000 7fbe58e1f640 1 --2- 172.21.15.154:0/3374905505 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbe3806af60 0x7fbe3806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:01.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.802+0000 7fbe58e1f640 1 --2- 172.21.15.154:0/3374905505 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbe3806af60 0x7fbe3806d420 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbe480059d0 tx=0x7fbe48009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:01.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:01.806+0000 7fbe517fa640 1 -- 172.21.15.154:0/3374905505 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fbe40076910 con 0x7fbe54154650 2023-09-23T03:17:02.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.009+0000 7fbe5b083640 1 -- 172.21.15.154:0/3374905505 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbe541555a0 con 0x7fbe54154650 2023-09-23T03:17:02.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.554+0000 7fbe517fa640 1 -- 172.21.15.154:0/3374905505 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v30) v1 ==== 178+0+0 (secure 0 0 0) 0x7fbe541555a0 con 0x7fbe54154650 2023-09-23T03:17:02.555 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_data_ec' removed 2023-09-23T03:17:02.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.558+0000 7fbe5b083640 1 -- 172.21.15.154:0/3374905505 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbe3806af60 msgr2=0x7fbe3806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:02.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.558+0000 7fbe5b083640 1 --2- 172.21.15.154:0/3374905505 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbe3806af60 0x7fbe3806d420 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbe480059d0 tx=0x7fbe48009040 comp rx=0 tx=0).stop 2023-09-23T03:17:02.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.558+0000 7fbe5b083640 1 -- 172.21.15.154:0/3374905505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe54154650 msgr2=0x7fbe541e5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:02.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.558+0000 7fbe5b083640 1 --2- 172.21.15.154:0/3374905505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe54154650 0x7fbe541e5e30 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fbe40008fd0 tx=0x7fbe40001120 comp rx=0 tx=0).stop 2023-09-23T03:17:02.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.558+0000 7fbe5b083640 1 -- 172.21.15.154:0/3374905505 shutdown_connections 2023-09-23T03:17:02.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.558+0000 7fbe5b083640 1 --2- 172.21.15.154:0/3374905505 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbe3806af60 0x7fbe3806d420 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:02.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.558+0000 7fbe5b083640 1 --2- 172.21.15.154:0/3374905505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe54155f40 0x7fbe540c5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:02.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.558+0000 7fbe5b083640 1 --2- 172.21.15.154:0/3374905505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe54154650 0x7fbe541e5e30 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:02.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.558+0000 7fbe5b083640 1 --2- 172.21.15.154:0/3374905505 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe44003f70 0x7fbe541e0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:02.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.559+0000 7fbe5b083640 1 -- 172.21.15.154:0/3374905505 >> 172.21.15.154:0/3374905505 conn(0x7fbe5406a520 msgr2=0x7fbe5406bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:02.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.559+0000 7fbe5b083640 1 -- 172.21.15.154:0/3374905505 shutdown_connections 2023-09-23T03:17:02.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.559+0000 7fbe5b083640 1 -- 172.21.15.154:0/3374905505 wait complete. 2023-09-23T03:17:02.584 INFO:tasks.ceph.mds.a:Restarting daemon 2023-09-23T03:17:02.584 INFO:tasks.ceph.mds.a:Stopping old one... 2023-09-23T03:17:02.584 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-09-23T03:17:02.584 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:17:02.584 INFO:tasks.ceph.mds.c:Restarting daemon 2023-09-23T03:17:02.584 INFO:tasks.ceph.mds.c:Stopping old one... 2023-09-23T03:17:02.585 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-09-23T03:17:02.585 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:17:02.585 INFO:tasks.ceph.mds.b:Restarting daemon 2023-09-23T03:17:02.585 INFO:tasks.ceph.mds.b:Stopping old one... 2023-09-23T03:17:02.585 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-09-23T03:17:02.585 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:17:02.585 INFO:tasks.ceph.mds.d:Restarting daemon 2023-09-23T03:17:02.585 INFO:tasks.ceph.mds.d:Stopping old one... 2023-09-23T03:17:02.586 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-09-23T03:17:02.586 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:17:02.602 INFO:tasks.ceph.mds.c:Stopped 2023-09-23T03:17:02.602 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-09-23T03:17:02.605 INFO:tasks.ceph.mds.a:Stopped 2023-09-23T03:17:02.606 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-09-23T03:17:02.608 INFO:tasks.ceph.mds.d:Stopped 2023-09-23T03:17:02.608 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-09-23T03:17:02.612 INFO:tasks.ceph.mds.b:Stopped 2023-09-23T03:17:02.612 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-09-23T03:17:02.654 INFO:tasks.ceph.mds.c:Started 2023-09-23T03:17:02.654 DEBUG:teuthology.parallel:result is None 2023-09-23T03:17:02.660 INFO:tasks.ceph.mds.d:Started 2023-09-23T03:17:02.660 DEBUG:teuthology.parallel:result is None 2023-09-23T03:17:02.662 INFO:tasks.ceph.mds.a:Started 2023-09-23T03:17:02.662 DEBUG:teuthology.parallel:result is None 2023-09-23T03:17:02.668 INFO:tasks.ceph.mds.b:Started 2023-09-23T03:17:02.668 DEBUG:teuthology.parallel:result is None 2023-09-23T03:17:02.668 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-09-23T03:17:02.815 INFO:tasks.ceph.mds.a.smithi154.stdout:starting mds.a at 2023-09-23T03:17:02.815 INFO:tasks.ceph.mds.c.smithi154.stdout:starting mds.c at 2023-09-23T03:17:02.824 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-09-23T03:17:02.824 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-09-23T03:17:02.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.858+0000 7f1f50b16640 1 Processor -- start 2023-09-23T03:17:02.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.858+0000 7f1f50b16640 1 -- start start 2023-09-23T03:17:02.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.859+0000 7f1f50b16640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1f4c0b1b20 0x7f1f4c0ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:02.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.859+0000 7f1f50b16640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f4c0cee70 0x7f1f4c0ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:02.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.860+0000 7f1f50b16640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f4c15ac30 con 0x7f1f4c0acce0 2023-09-23T03:17:02.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.860+0000 7f1f50b16640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f4c0b2e90 con 0x7f1f4c0b1b20 2023-09-23T03:17:02.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.860+0000 7f1f50b16640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c0cf320 con 0x7f1f4c0cee70 2023-09-23T03:17:02.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.860+0000 7f1f43fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1f4c0b1b20 0x7f1f4c0ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:02.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.860+0000 7f1f43fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1f4c0b1b20 0x7f1f4c0ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56172/0 (socket says 172.21.15.154:56172) 2023-09-23T03:17:02.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.860+0000 7f1f43fff640 1 -- 172.21.15.154:0/593546112 learned_addr learned my addr 172.21.15.154:0/593546112 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:02.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.860+0000 7f1f4ad9d640 1 --2- 172.21.15.154:0/593546112 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f4c0cee70 0x7f1f4c0ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:02.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.860+0000 7f1f43fff640 1 -- 172.21.15.154:0/593546112 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f4c0cee70 msgr2=0x7f1f4c0ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:02.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.860+0000 7f1f43fff640 1 --2- 172.21.15.154:0/593546112 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f4c0cee70 0x7f1f4c0ab140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:02.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.860+0000 7f1f43fff640 1 -- 172.21.15.154:0/593546112 >> v1:172.21.15.190:6789/0 conn(0x7f1f4c0acce0 legacy=0x7f1f4c15ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:02.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.860+0000 7f1f43fff640 1 -- 172.21.15.154:0/593546112 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4c0b1f70 con 0x7f1f4c0b1b20 2023-09-23T03:17:02.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.860+0000 7f1f4ad9d640 1 --2- 172.21.15.154:0/593546112 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f4c0cee70 0x7f1f4c0ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:17:02.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.860+0000 7f1f49d9b640 1 -- 172.21.15.154:0/593546112 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3352989152 0 0) 0x7f1f3c001160 con 0x7f1f4c0acce0 2023-09-23T03:17:02.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.861+0000 7f1f43fff640 1 --2- 172.21.15.154:0/593546112 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f4c0b1b20 0x7f1f4c0ce930 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1f34009870 tx=0x7f1f3402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0666af8a65cc3e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:02.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.862+0000 7f1f49d9b640 1 -- 172.21.15.154:0/593546112 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f34031640 con 0x7f1f4c0b1b20 2023-09-23T03:17:02.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.862+0000 7f1f49d9b640 1 -- 172.21.15.154:0/593546112 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f340317a0 con 0x7f1f4c0b1b20 2023-09-23T03:17:02.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.862+0000 7f1f49d9b640 1 -- 172.21.15.154:0/593546112 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f34031b30 con 0x7f1f4c0b1b20 2023-09-23T03:17:02.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.862+0000 7f1f50b16640 1 -- 172.21.15.154:0/593546112 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f4c0b1b20 msgr2=0x7f1f4c0ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:02.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.862+0000 7f1f50b16640 1 --2- 172.21.15.154:0/593546112 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f4c0b1b20 0x7f1f4c0ce930 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1f34009870 tx=0x7f1f3402b8f0 comp rx=0 tx=0).stop 2023-09-23T03:17:02.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.862+0000 7f1f50b16640 1 -- 172.21.15.154:0/593546112 shutdown_connections 2023-09-23T03:17:02.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.862+0000 7f1f50b16640 1 --2- 172.21.15.154:0/593546112 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f4c0cee70 0x7f1f4c0ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:02.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.862+0000 7f1f50b16640 1 --2- 172.21.15.154:0/593546112 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f4c0b1b20 0x7f1f4c0ce930 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:02.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.862+0000 7f1f50b16640 1 -- 172.21.15.154:0/593546112 >> 172.21.15.154:0/593546112 conn(0x7f1f4c06a0f0 msgr2=0x7f1f4c06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:02.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.863+0000 7f1f50b16640 1 -- 172.21.15.154:0/593546112 shutdown_connections 2023-09-23T03:17:02.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.863+0000 7f1f50b16640 1 -- 172.21.15.154:0/593546112 wait complete. 2023-09-23T03:17:02.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.864+0000 7f1f50b16640 1 Processor -- start 2023-09-23T03:17:02.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.864+0000 7f1f50b16640 1 -- start start 2023-09-23T03:17:02.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.865+0000 7f1f50b16640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f4c0acce0 0x7f1f4c157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:02.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.865+0000 7f1f50b16640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f4c0b1b20 0x7f1f4c15cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:02.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.865+0000 7f1f50b16640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f4c0cee70 0x7f1f4c0c7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:02.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.865+0000 7f1f50b16640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c0c6c00 con 0x7f1f4c0cee70 2023-09-23T03:17:02.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.865+0000 7f1f50b16640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c0c5ff0 con 0x7f1f4c0b1b20 2023-09-23T03:17:02.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.865+0000 7f1f50b16640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c0c62d0 con 0x7f1f4c0acce0 2023-09-23T03:17:02.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.866+0000 7f1f4ad9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f4c0cee70 0x7f1f4c0c7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:02.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.866+0000 7f1f4ad9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f4c0cee70 0x7f1f4c0c7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56184/0 (socket says 172.21.15.154:56184) 2023-09-23T03:17:02.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.866+0000 7f1f4ad9d640 1 -- 172.21.15.154:0/1372920782 learned_addr learned my addr 172.21.15.154:0/1372920782 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:02.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.866+0000 7f1f43fff640 1 --2- 172.21.15.154:0/1372920782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f4c0b1b20 0x7f1f4c15cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:02.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.866+0000 7f1f4a59c640 1 --2- 172.21.15.154:0/1372920782 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f4c0acce0 0x7f1f4c157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:02.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.866+0000 7f1f4ad9d640 1 -- 172.21.15.154:0/1372920782 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f4c0acce0 msgr2=0x7f1f4c157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:02.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.866+0000 7f1f4ad9d640 1 --2- 172.21.15.154:0/1372920782 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f4c0acce0 0x7f1f4c157910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:02.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.866+0000 7f1f4ad9d640 1 -- 172.21.15.154:0/1372920782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f4c0b1b20 msgr2=0x7f1f4c15cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:02.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.866+0000 7f1f4ad9d640 1 --2- 172.21.15.154:0/1372920782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f4c0b1b20 0x7f1f4c15cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:02.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.866+0000 7f1f4ad9d640 1 -- 172.21.15.154:0/1372920782 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f340094d0 con 0x7f1f4c0cee70 2023-09-23T03:17:02.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.866+0000 7f1f43fff640 1 --2- 172.21.15.154:0/1372920782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f4c0b1b20 0x7f1f4c15cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:17:02.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.866+0000 7f1f4ad9d640 1 --2- 172.21.15.154:0/1372920782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f4c0cee70 0x7f1f4c0c7090 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1f44007e70 tx=0x7f1f4400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:02.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.867+0000 7f1f42ffd640 1 -- 172.21.15.154:0/1372920782 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f44003b90 con 0x7f1f4c0cee70 2023-09-23T03:17:02.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.867+0000 7f1f42ffd640 1 -- 172.21.15.154:0/1372920782 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f4400f040 con 0x7f1f4c0cee70 2023-09-23T03:17:02.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.867+0000 7f1f50b16640 1 -- 172.21.15.154:0/1372920782 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f4c0c65b0 con 0x7f1f4c0cee70 2023-09-23T03:17:02.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.867+0000 7f1f42ffd640 1 -- 172.21.15.154:0/1372920782 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4400de80 con 0x7f1f4c0cee70 2023-09-23T03:17:02.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.867+0000 7f1f50b16640 1 -- 172.21.15.154:0/1372920782 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f4c15d090 con 0x7f1f4c0cee70 2023-09-23T03:17:02.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.868+0000 7f1f50b16640 1 -- 172.21.15.154:0/1372920782 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f4c15d280 con 0x7f1f4c0cee70 2023-09-23T03:17:02.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.869+0000 7f1f42ffd640 1 -- 172.21.15.154:0/1372920782 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f1f44004030 con 0x7f1f4c0cee70 2023-09-23T03:17:02.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.870+0000 7f1f42ffd640 1 --2- 172.21.15.154:0/1372920782 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1f2c06af30 0x7f1f2c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:02.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.871+0000 7f1f42ffd640 1 -- 172.21.15.154:0/1372920782 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f1f44089930 con 0x7f1f4c0cee70 2023-09-23T03:17:02.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.871+0000 7f1f4a59c640 1 --2- 172.21.15.154:0/1372920782 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1f2c06af30 0x7f1f2c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:02.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.871+0000 7f1f4a59c640 1 --2- 172.21.15.154:0/1372920782 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1f2c06af30 0x7f1f2c06d3f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c000fb0 tx=0x7f1f3c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:02.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:02.875+0000 7f1f42ffd640 1 -- 172.21.15.154:0/1372920782 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f1f4c15d280 con 0x7f1f4c0cee70 2023-09-23T03:17:03.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.082+0000 7f1f50b16640 1 -- 172.21.15.154:0/1372920782 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f1f4c15d280 con 0x7f1f4c0cee70 2023-09-23T03:17:03.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.601+0000 7f1f42ffd640 1 -- 172.21.15.154:0/1372920782 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v32) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1f4c15d280 con 0x7f1f4c0cee70 2023-09-23T03:17:03.602 INFO:teuthology.orchestra.run.smithi154.stderr: removed all blocklist entries 2023-09-23T03:17:03.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.604+0000 7f1f50b16640 1 -- 172.21.15.154:0/1372920782 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1f2c06af30 msgr2=0x7f1f2c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:03.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.604+0000 7f1f50b16640 1 --2- 172.21.15.154:0/1372920782 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1f2c06af30 0x7f1f2c06d3f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c000fb0 tx=0x7f1f3c009040 comp rx=0 tx=0).stop 2023-09-23T03:17:03.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.604+0000 7f1f50b16640 1 -- 172.21.15.154:0/1372920782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f4c0cee70 msgr2=0x7f1f4c0c7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:03.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.604+0000 7f1f50b16640 1 --2- 172.21.15.154:0/1372920782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f4c0cee70 0x7f1f4c0c7090 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1f44007e70 tx=0x7f1f4400d4a0 comp rx=0 tx=0).stop 2023-09-23T03:17:03.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.605+0000 7f1f50b16640 1 -- 172.21.15.154:0/1372920782 shutdown_connections 2023-09-23T03:17:03.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.605+0000 7f1f50b16640 1 --2- 172.21.15.154:0/1372920782 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1f2c06af30 0x7f1f2c06d3f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:03.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.605+0000 7f1f50b16640 1 --2- 172.21.15.154:0/1372920782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f4c0cee70 0x7f1f4c0c7090 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:03.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.605+0000 7f1f50b16640 1 --2- 172.21.15.154:0/1372920782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f4c0b1b20 0x7f1f4c15cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:03.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.605+0000 7f1f50b16640 1 --2- 172.21.15.154:0/1372920782 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f4c0acce0 0x7f1f4c157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:03.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.605+0000 7f1f50b16640 1 -- 172.21.15.154:0/1372920782 >> 172.21.15.154:0/1372920782 conn(0x7f1f4c06a0f0 msgr2=0x7f1f4c0c9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:03.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.605+0000 7f1f50b16640 1 -- 172.21.15.154:0/1372920782 shutdown_connections 2023-09-23T03:17:03.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.605+0000 7f1f50b16640 1 -- 172.21.15.154:0/1372920782 wait complete. 2023-09-23T03:17:03.631 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-09-23T03:17:03.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.819+0000 7f7116f78640 1 Processor -- start 2023-09-23T03:17:03.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.819+0000 7f7116f78640 1 -- start start 2023-09-23T03:17:03.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.820+0000 7f7116f78640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f71100acce0 0x7f711015ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:03.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.820+0000 7f7116f78640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f71100b1b20 0x7f71100ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:03.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.820+0000 7f7116f78640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f711015ac30 con 0x7f71100cee70 2023-09-23T03:17:03.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.820+0000 7f7116f78640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71100b2e90 con 0x7f71100b1b20 2023-09-23T03:17:03.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.820+0000 7f7116f78640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71100cf300 con 0x7f71100acce0 2023-09-23T03:17:03.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.821+0000 7f7115515640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f71100cee70 0x7f711006cb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:54896/0 (socket says 172.21.15.154:54896) 2023-09-23T03:17:03.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.821+0000 7f7115515640 1 -- 172.21.15.154:0/2676453514 learned_addr learned my addr 172.21.15.154:0/2676453514 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:17:03.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.821+0000 7f710ffff640 1 --2- 172.21.15.154:0/2676453514 >> v2:172.21.15.154:3300/0 conn(0x7f71100b1b20 0x7f71100ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:03.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.821+0000 7f7114d14640 1 --2- 172.21.15.154:0/2676453514 >> v2:172.21.15.190:3300/0 conn(0x7f71100acce0 0x7f711015ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:03.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.821+0000 7f710ffff640 1 -- 172.21.15.154:0/2676453514 >> v2:172.21.15.190:3300/0 conn(0x7f71100acce0 msgr2=0x7f711015ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:03.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.821+0000 7f710ffff640 1 --2- 172.21.15.154:0/2676453514 >> v2:172.21.15.190:3300/0 conn(0x7f71100acce0 0x7f711015ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:03.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.821+0000 7f710ffff640 1 -- 172.21.15.154:0/2676453514 >> v1:172.21.15.154:6789/0 conn(0x7f71100cee70 legacy=0x7f711006cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:03.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.821+0000 7f710ffff640 1 -- 172.21.15.154:0/2676453514 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71100b1f70 con 0x7f71100b1b20 2023-09-23T03:17:03.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.821+0000 7f710f7fe640 1 -- 172.21.15.154:0/2676453514 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 269702957 0 0) 0x7f7108001490 con 0x7f71100cee70 2023-09-23T03:17:03.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.823+0000 7f710ffff640 1 --2- 172.21.15.154:0/2676453514 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71100b1b20 0x7f71100ce930 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f71100cf570 tx=0x7f70fc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a69a2daf0e0ea1d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:03.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.823+0000 7f710f7fe640 1 -- 172.21.15.154:0/2676453514 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc02e070 con 0x7f71100b1b20 2023-09-23T03:17:03.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.823+0000 7f710f7fe640 1 -- 172.21.15.154:0/2676453514 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70fc0315a0 con 0x7f71100b1b20 2023-09-23T03:17:03.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.823+0000 7f710f7fe640 1 -- 172.21.15.154:0/2676453514 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc031930 con 0x7f71100b1b20 2023-09-23T03:17:03.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.823+0000 7f7116f78640 1 -- 172.21.15.154:0/2676453514 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71100b1b20 msgr2=0x7f71100ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:03.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.823+0000 7f7116f78640 1 --2- 172.21.15.154:0/2676453514 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71100b1b20 0x7f71100ce930 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f71100cf570 tx=0x7f70fc02b840 comp rx=0 tx=0).stop 2023-09-23T03:17:03.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.824+0000 7f7116f78640 1 -- 172.21.15.154:0/2676453514 shutdown_connections 2023-09-23T03:17:03.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.824+0000 7f7116f78640 1 --2- 172.21.15.154:0/2676453514 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71100b1b20 0x7f71100ce930 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:03.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.824+0000 7f7116f78640 1 --2- 172.21.15.154:0/2676453514 >> v2:172.21.15.190:3300/0 conn(0x7f71100acce0 0x7f711015ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:03.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.824+0000 7f7116f78640 1 -- 172.21.15.154:0/2676453514 >> 172.21.15.154:0/2676453514 conn(0x7f711006a0f0 msgr2=0x7f711006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:03.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.824+0000 7f7116f78640 1 -- 172.21.15.154:0/2676453514 shutdown_connections 2023-09-23T03:17:03.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.824+0000 7f7116f78640 1 -- 172.21.15.154:0/2676453514 wait complete. 2023-09-23T03:17:03.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.825+0000 7f7116f78640 1 Processor -- start 2023-09-23T03:17:03.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.825+0000 7f7116f78640 1 -- start start 2023-09-23T03:17:03.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.826+0000 7f7116f78640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71100acce0 0x7f71100cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:03.828 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.826+0000 7f7116f78640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71100b1b20 0x7f71100c2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:03.828 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.826+0000 7f7116f78640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71100cee70 0x7f71100c7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:03.828 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.826+0000 7f7116f78640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71100cc3b0 con 0x7f71100b1b20 2023-09-23T03:17:03.828 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.826+0000 7f7116f78640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71101f6010 con 0x7f71100cee70 2023-09-23T03:17:03.828 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.826+0000 7f7116f78640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71101f62b0 con 0x7f71100acce0 2023-09-23T03:17:03.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.827+0000 7f710ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71100b1b20 0x7f71100c2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:03.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.827+0000 7f710ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71100b1b20 0x7f71100c2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56206/0 (socket says 172.21.15.154:56206) 2023-09-23T03:17:03.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.827+0000 7f710ffff640 1 -- 172.21.15.154:0/493804534 learned_addr learned my addr 172.21.15.154:0/493804534 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:03.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.827+0000 7f7115515640 1 --2- 172.21.15.154:0/493804534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71100cee70 0x7f71100c7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:03.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.827+0000 7f7114d14640 1 --2- 172.21.15.154:0/493804534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71100acce0 0x7f71100cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:03.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.827+0000 7f710ffff640 1 -- 172.21.15.154:0/493804534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71100acce0 msgr2=0x7f71100cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:03.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.827+0000 7f710ffff640 1 --2- 172.21.15.154:0/493804534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71100acce0 0x7f71100cbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:03.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.827+0000 7f710ffff640 1 -- 172.21.15.154:0/493804534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71100cee70 msgr2=0x7f71100c7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:03.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.827+0000 7f710ffff640 1 --2- 172.21.15.154:0/493804534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71100cee70 0x7f71100c7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:03.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.827+0000 7f710ffff640 1 -- 172.21.15.154:0/493804534 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70fc009400 con 0x7f71100b1b20 2023-09-23T03:17:03.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.828+0000 7f710ffff640 1 --2- 172.21.15.154:0/493804534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71100b1b20 0x7f71100c2d30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f70fc000c00 tx=0x7f70fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:03.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.828+0000 7f710d7fa640 1 -- 172.21.15.154:0/493804534 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc032900 con 0x7f71100b1b20 2023-09-23T03:17:03.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.828+0000 7f710d7fa640 1 -- 172.21.15.154:0/493804534 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70fc032a80 con 0x7f71100b1b20 2023-09-23T03:17:03.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.828+0000 7f710d7fa640 1 -- 172.21.15.154:0/493804534 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc032e30 con 0x7f71100b1b20 2023-09-23T03:17:03.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.828+0000 7f7116f78640 1 -- 172.21.15.154:0/493804534 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71100c7470 con 0x7f71100b1b20 2023-09-23T03:17:03.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.828+0000 7f7116f78640 1 -- 172.21.15.154:0/493804534 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71100c7140 con 0x7f71100b1b20 2023-09-23T03:17:03.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.829+0000 7f7116f78640 1 -- 172.21.15.154:0/493804534 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71100c7990 con 0x7f71100b1b20 2023-09-23T03:17:03.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.831+0000 7f710d7fa640 1 -- 172.21.15.154:0/493804534 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f70fc04d3f0 con 0x7f71100b1b20 2023-09-23T03:17:03.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.831+0000 7f710d7fa640 1 --2- 172.21.15.154:0/493804534 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f70f406af30 0x7f70f406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:03.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.832+0000 7f710d7fa640 1 -- 172.21.15.154:0/493804534 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f70fc0ab0b0 con 0x7f71100b1b20 2023-09-23T03:17:03.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.832+0000 7f7114d14640 1 --2- 172.21.15.154:0/493804534 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f70f406af30 0x7f70f406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:03.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.832+0000 7f7114d14640 1 --2- 172.21.15.154:0/493804534 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f70f406af30 0x7f70f406d3f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f710400b9b0 tx=0x7f7104008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:03.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:03.836+0000 7f710d7fa640 1 -- 172.21.15.154:0/493804534 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f71100c7990 con 0x7f71100b1b20 2023-09-23T03:17:04.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.092+0000 7f7116f78640 1 -- 172.21.15.154:0/493804534 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f71100c7990 con 0x7f71100b1b20 2023-09-23T03:17:04.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.093+0000 7f710d7fa640 1 -- 172.21.15.154:0/493804534 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v3) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f71100c7990 con 0x7f71100b1b20 2023-09-23T03:17:04.095 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:04.095 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:17:04.095 INFO:teuthology.orchestra.run.smithi154.stdout: "auth_dump": [ 2023-09-23T03:17:04.095 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:17:04.095 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mds.a", 2023-09-23T03:17:04.095 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAGWA5l9biCFhAAhD2xgY9Q/gGqK0pgHRGweQ==", 2023-09-23T03:17:04.096 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:17:04.096 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:17:04.096 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2023-09-23T03:17:04.096 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2023-09-23T03:17:04.096 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:17:04.096 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:17:04.096 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:17:04.096 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:17:04.096 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mds.b", 2023-09-23T03:17:04.096 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAGWA5lsgKCNxAAI+vD2ESh3uNsb5OCO0ydSw==", 2023-09-23T03:17:04.096 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:17:04.097 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:17:04.097 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2023-09-23T03:17:04.097 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2023-09-23T03:17:04.097 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:17:04.097 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:17:04.097 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:17:04.097 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:17:04.097 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mds.c", 2023-09-23T03:17:04.097 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAGWA5los4vJhAA2/ewjMZcc3pnm9ukhnoNgw==", 2023-09-23T03:17:04.097 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:17:04.097 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:17:04.098 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2023-09-23T03:17:04.098 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2023-09-23T03:17:04.098 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:17:04.098 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:17:04.098 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:17:04.098 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:17:04.098 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mds.d", 2023-09-23T03:17:04.098 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAHWA5lml/MDBAAHyXgd5zl/O8CeVE8Da4w5w==", 2023-09-23T03:17:04.098 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:17:04.098 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:17:04.099 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2023-09-23T03:17:04.099 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2023-09-23T03:17:04.099 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:17:04.099 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:17:04.099 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:17:04.099 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:17:04.099 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.0", 2023-09-23T03:17:04.099 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAMWA5lT1R4BBAAI+itqub/FadG3aU6LhnmGg==", 2023-09-23T03:17:04.099 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:17:04.099 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:17:04.099 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:17:04.100 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:17:04.100 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:17:04.100 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:17:04.100 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:17:04.100 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.1", 2023-09-23T03:17:04.100 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAPWA5lSablHRAAmNyr5Zopeihn5ZyoY5pyhA==", 2023-09-23T03:17:04.100 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:17:04.100 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:17:04.100 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:17:04.100 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:17:04.100 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:17:04.101 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:17:04.101 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:17:04.101 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.2", 2023-09-23T03:17:04.101 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQASWA5lSFcdJxAA3UJJ6eSvxcQ31SAm4shV2g==", 2023-09-23T03:17:04.101 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:17:04.101 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:17:04.101 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:17:04.101 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:17:04.101 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:17:04.101 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:17:04.101 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:17:04.101 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.3", 2023-09-23T03:17:04.102 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAVWA5lmSvzMRAA97YDqX3oplHCB+SENIkLYA==", 2023-09-23T03:17:04.102 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:17:04.102 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:17:04.102 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:17:04.102 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:17:04.102 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:17:04.102 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:17:04.102 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:17:04.102 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.4", 2023-09-23T03:17:04.102 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAcWA5lPZqwMBAAW95bBB0PKS/rwhXvrPf+mQ==", 2023-09-23T03:17:04.102 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:17:04.102 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:17:04.103 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:17:04.103 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:17:04.103 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:17:04.103 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:17:04.103 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:17:04.103 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.5", 2023-09-23T03:17:04.103 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAgWA5lmZySDRAAsbnUdWq9esKFSqSDeqsVlA==", 2023-09-23T03:17:04.103 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:17:04.103 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:17:04.103 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:17:04.103 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:17:04.103 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:17:04.104 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:17:04.104 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:17:04.104 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.6", 2023-09-23T03:17:04.104 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAjWA5ll9WqJRAAD9jecVxVJU+yuqANvN9QUQ==", 2023-09-23T03:17:04.104 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:17:04.104 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:17:04.104 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:17:04.104 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:17:04.104 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:17:04.105 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:17:04.105 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:17:04.105 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.7", 2023-09-23T03:17:04.105 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAnWA5lCvexAxAA/QymygbvOt1HQdCGbO8cyg==", 2023-09-23T03:17:04.105 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:17:04.105 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:17:04.105 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:17:04.105 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:17:04.105 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:17:04.105 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:17:04.105 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:17:04.106 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.0", 2023-09-23T03:17:04.106 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAHWA5lYg+qGhAAjaY0fPsmRTz3ipsKbPugMA==", 2023-09-23T03:17:04.106 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:17:04.106 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:17:04.106 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow r", 2023-09-23T03:17:04.106 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow rw", 2023-09-23T03:17:04.106 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow rwx" 2023-09-23T03:17:04.106 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:17:04.106 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:17:04.106 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:17:04.106 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.1", 2023-09-23T03:17:04.107 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAHWA5lbrrGMBAAn4kvS/1xGjZ+GtRZFqy31w==", 2023-09-23T03:17:04.107 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:17:04.107 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:17:04.107 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow r", 2023-09-23T03:17:04.107 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow rw", 2023-09-23T03:17:04.107 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow rwx" 2023-09-23T03:17:04.107 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:17:04.107 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:17:04.107 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:17:04.107 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.2", 2023-09-23T03:17:04.107 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAHWA5lcLkIJRAAOknGCKwUs78D4atRzLb/Cw==", 2023-09-23T03:17:04.107 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:17:04.108 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:17:04.108 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow r", 2023-09-23T03:17:04.108 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow rw", 2023-09-23T03:17:04.108 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow rwx" 2023-09-23T03:17:04.108 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:17:04.108 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:17:04.108 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:17:04.108 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.3", 2023-09-23T03:17:04.108 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAIWA5lmFUaABAA4EBSJP5IidG2uq4g8lrFKg==", 2023-09-23T03:17:04.108 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:17:04.108 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:17:04.108 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow r", 2023-09-23T03:17:04.109 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow rw", 2023-09-23T03:17:04.109 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow rwx" 2023-09-23T03:17:04.109 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:17:04.109 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:17:04.109 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:17:04.109 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.admin", 2023-09-23T03:17:04.109 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAFWA5l0x+tCxAAGEYW1egctWzycEhM6FjJZw==", 2023-09-23T03:17:04.109 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:17:04.109 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow *", 2023-09-23T03:17:04.109 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2023-09-23T03:17:04.109 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2023-09-23T03:17:04.109 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:17:04.110 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:17:04.110 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:17:04.110 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:17:04.110 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.bootstrap-mds", 2023-09-23T03:17:04.110 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAwWA5lLg7hBBAAEP7f9UrmKrsTzfS9X+Il/A==", 2023-09-23T03:17:04.110 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:17:04.110 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile bootstrap-mds" 2023-09-23T03:17:04.110 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:17:04.110 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:17:04.110 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:17:04.110 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.bootstrap-mgr", 2023-09-23T03:17:04.111 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAwWA5lpiThBBAA68Jb81GSyjKMZygr6LHWTg==", 2023-09-23T03:17:04.111 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:17:04.111 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile bootstrap-mgr" 2023-09-23T03:17:04.111 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:17:04.111 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:17:04.111 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:17:04.111 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.bootstrap-osd", 2023-09-23T03:17:04.111 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAwWA5lQzfhBBAAcNwFRpHTKwxsr32UqGJWjw==", 2023-09-23T03:17:04.111 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:17:04.111 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile bootstrap-osd" 2023-09-23T03:17:04.111 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:17:04.111 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:17:04.112 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:17:04.112 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.bootstrap-rbd", 2023-09-23T03:17:04.112 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAwWA5lNknhBBAAZcIN7y5EarOPpljAA38H8Q==", 2023-09-23T03:17:04.112 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:17:04.112 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile bootstrap-rbd" 2023-09-23T03:17:04.112 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:17:04.112 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:17:04.112 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:17:04.112 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-09-23T03:17:04.112 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAwWA5lAVzhBBAAHu2FYcymKrs15iBUo7K+tw==", 2023-09-23T03:17:04.112 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:17:04.113 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-09-23T03:17:04.113 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:17:04.113 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:17:04.113 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:17:04.113 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.bootstrap-rgw", 2023-09-23T03:17:04.113 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAwWA5lIW/hBBAAOGyhuDmkvqZr+Cx8nlot4Q==", 2023-09-23T03:17:04.113 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:17:04.113 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile bootstrap-rgw" 2023-09-23T03:17:04.113 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:17:04.113 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:17:04.113 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:17:04.113 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mgr.x", 2023-09-23T03:17:04.114 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAGWA5ljcy5ChAACUOnrxEGqacSxrB9BkZ+lg==", 2023-09-23T03:17:04.114 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:17:04.114 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow *", 2023-09-23T03:17:04.114 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile mgr", 2023-09-23T03:17:04.114 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:17:04.114 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:17:04.114 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:17:04.114 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:17:04.114 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mgr.y", 2023-09-23T03:17:04.114 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAFWA5looGaOhAAdxmefaoLuXesHXCeqeCYXQ==", 2023-09-23T03:17:04.114 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:17:04.114 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow *", 2023-09-23T03:17:04.115 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile mgr", 2023-09-23T03:17:04.115 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:17:04.115 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:17:04.115 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:17:04.115 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2023-09-23T03:17:04.115 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:17:04.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.096+0000 7f7116f78640 1 -- 172.21.15.154:0/493804534 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f70f406af30 msgr2=0x7f70f406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:04.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.096+0000 7f7116f78640 1 --2- 172.21.15.154:0/493804534 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f70f406af30 0x7f70f406d3f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f710400b9b0 tx=0x7f7104008040 comp rx=0 tx=0).stop 2023-09-23T03:17:04.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.096+0000 7f7116f78640 1 -- 172.21.15.154:0/493804534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71100b1b20 msgr2=0x7f71100c2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:04.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.096+0000 7f7116f78640 1 --2- 172.21.15.154:0/493804534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71100b1b20 0x7f71100c2d30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f70fc000c00 tx=0x7f70fc0046c0 comp rx=0 tx=0).stop 2023-09-23T03:17:04.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.096+0000 7f7116f78640 1 -- 172.21.15.154:0/493804534 shutdown_connections 2023-09-23T03:17:04.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.096+0000 7f7116f78640 1 --2- 172.21.15.154:0/493804534 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f70f406af30 0x7f70f406d3f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:04.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.096+0000 7f7116f78640 1 --2- 172.21.15.154:0/493804534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71100cee70 0x7f71100c7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:04.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.096+0000 7f7116f78640 1 --2- 172.21.15.154:0/493804534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71100b1b20 0x7f71100c2d30 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:04.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.096+0000 7f7116f78640 1 --2- 172.21.15.154:0/493804534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71100acce0 0x7f71100cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:04.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.097+0000 7f7116f78640 1 -- 172.21.15.154:0/493804534 >> 172.21.15.154:0/493804534 conn(0x7f711006a0f0 msgr2=0x7f71100b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:04.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.097+0000 7f7116f78640 1 -- 172.21.15.154:0/493804534 shutdown_connections 2023-09-23T03:17:04.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.097+0000 7f7116f78640 1 -- 172.21.15.154:0/493804534 wait complete. 2023-09-23T03:17:04.118 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-09-23T03:17:04.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.336+0000 7fc845e4f640 1 Processor -- start 2023-09-23T03:17:04.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.337+0000 7fc845e4f640 1 -- start start 2023-09-23T03:17:04.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.338+0000 7fc845e4f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc84015cf40 0x7fc8401615b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:04.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.338+0000 7fc845e4f640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc84006b1e0 0x7fc8400cca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:04.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.338+0000 7fc845e4f640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8400ccf80 con 0x7fc8400b21d0 2023-09-23T03:17:04.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.338+0000 7fc845e4f640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8400cd1c0 con 0x7fc84015cf40 2023-09-23T03:17:04.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.338+0000 7fc845e4f640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8400cd4a0 con 0x7fc84006b1e0 2023-09-23T03:17:04.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.338+0000 7fc83f7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc84015cf40 0x7fc8401615b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:04.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.338+0000 7fc83f7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc84015cf40 0x7fc8401615b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47040/0 (socket says 172.21.15.154:47040) 2023-09-23T03:17:04.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.338+0000 7fc83f7fe640 1 -- 172.21.15.154:0/3550406942 learned_addr learned my addr 172.21.15.154:0/3550406942 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:04.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.339+0000 7fc83f7fe640 1 -- 172.21.15.154:0/3550406942 >> v2:172.21.15.190:3300/0 conn(0x7fc84006b1e0 msgr2=0x7fc8400cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:04.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.339+0000 7fc83f7fe640 1 --2- 172.21.15.154:0/3550406942 >> v2:172.21.15.190:3300/0 conn(0x7fc84006b1e0 0x7fc8400cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:04.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.339+0000 7fc83f7fe640 1 -- 172.21.15.154:0/3550406942 >> v1:172.21.15.190:6789/0 conn(0x7fc8400b21d0 legacy=0x7fc8400cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:04.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.339+0000 7fc83f7fe640 1 -- 172.21.15.154:0/3550406942 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8400cd780 con 0x7fc84015cf40 2023-09-23T03:17:04.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.340+0000 7fc83f7fe640 1 --2- 172.21.15.154:0/3550406942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc84015cf40 0x7fc8401615b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fc8400ce210 tx=0x7fc83002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0de177cc63a83bf server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:04.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.340+0000 7fc83e7fc640 1 -- 172.21.15.154:0/3550406942 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83002ca80 con 0x7fc84015cf40 2023-09-23T03:17:04.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.340+0000 7fc83e7fc640 1 -- 172.21.15.154:0/3550406942 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc83002cbe0 con 0x7fc84015cf40 2023-09-23T03:17:04.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.340+0000 7fc83e7fc640 1 -- 172.21.15.154:0/3550406942 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8300343f0 con 0x7fc84015cf40 2023-09-23T03:17:04.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.341+0000 7fc845e4f640 1 -- 172.21.15.154:0/3550406942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc84015cf40 msgr2=0x7fc8401615b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:04.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.341+0000 7fc845e4f640 1 --2- 172.21.15.154:0/3550406942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc84015cf40 0x7fc8401615b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fc8400ce210 tx=0x7fc83002bee0 comp rx=0 tx=0).stop 2023-09-23T03:17:04.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.341+0000 7fc845e4f640 1 -- 172.21.15.154:0/3550406942 shutdown_connections 2023-09-23T03:17:04.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.341+0000 7fc845e4f640 1 --2- 172.21.15.154:0/3550406942 >> v2:172.21.15.190:3300/0 conn(0x7fc84006b1e0 0x7fc8400cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:04.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.341+0000 7fc845e4f640 1 --2- 172.21.15.154:0/3550406942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc84015cf40 0x7fc8401615b0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:04.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.341+0000 7fc845e4f640 1 -- 172.21.15.154:0/3550406942 >> 172.21.15.154:0/3550406942 conn(0x7fc84006fc60 msgr2=0x7fc840070040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:04.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.341+0000 7fc845e4f640 1 -- 172.21.15.154:0/3550406942 shutdown_connections 2023-09-23T03:17:04.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.341+0000 7fc845e4f640 1 -- 172.21.15.154:0/3550406942 wait complete. 2023-09-23T03:17:04.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.343+0000 7fc845e4f640 1 Processor -- start 2023-09-23T03:17:04.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.343+0000 7fc845e4f640 1 -- start start 2023-09-23T03:17:04.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.344+0000 7fc845e4f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc84006b1e0 0x7fc8400c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:04.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.344+0000 7fc845e4f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8400b21d0 0x7fc8401496e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:04.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.344+0000 7fc845e4f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc84015cf40 0x7fc840149c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:04.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.344+0000 7fc845e4f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84014a160 con 0x7fc84006b1e0 2023-09-23T03:17:04.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.344+0000 7fc845e4f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8400ca7d0 con 0x7fc84015cf40 2023-09-23T03:17:04.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.344+0000 7fc845e4f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8400c9f30 con 0x7fc8400b21d0 2023-09-23T03:17:04.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.344+0000 7fc83f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc84006b1e0 0x7fc8400c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:04.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.344+0000 7fc83f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc84006b1e0 0x7fc8400c6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47056/0 (socket says 172.21.15.154:47056) 2023-09-23T03:17:04.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.344+0000 7fc83f7fe640 1 -- 172.21.15.154:0/2242306645 learned_addr learned my addr 172.21.15.154:0/2242306645 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:04.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.344+0000 7fc83ffff640 1 --2- 172.21.15.154:0/2242306645 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc84015cf40 0x7fc840149c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:04.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.344+0000 7fc83effd640 1 --2- 172.21.15.154:0/2242306645 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8400b21d0 0x7fc8401496e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:04.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.344+0000 7fc83f7fe640 1 -- 172.21.15.154:0/2242306645 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8400b21d0 msgr2=0x7fc8401496e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:04.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.344+0000 7fc83f7fe640 1 --2- 172.21.15.154:0/2242306645 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8400b21d0 0x7fc8401496e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:04.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.344+0000 7fc83f7fe640 1 -- 172.21.15.154:0/2242306645 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc84015cf40 msgr2=0x7fc840149c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:04.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.344+0000 7fc83f7fe640 1 --2- 172.21.15.154:0/2242306645 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc84015cf40 0x7fc840149c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:04.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.344+0000 7fc83f7fe640 1 -- 172.21.15.154:0/2242306645 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc830009b30 con 0x7fc84006b1e0 2023-09-23T03:17:04.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.345+0000 7fc83f7fe640 1 --2- 172.21.15.154:0/2242306645 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc84006b1e0 0x7fc8400c6a20 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc830036040 tx=0x7fc830004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:04.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.345+0000 7fc837fff640 1 -- 172.21.15.154:0/2242306645 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83003e450 con 0x7fc84006b1e0 2023-09-23T03:17:04.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.345+0000 7fc837fff640 1 -- 172.21.15.154:0/2242306645 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc83003e5b0 con 0x7fc84006b1e0 2023-09-23T03:17:04.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.345+0000 7fc837fff640 1 -- 172.21.15.154:0/2242306645 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83003e960 con 0x7fc84006b1e0 2023-09-23T03:17:04.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.345+0000 7fc845e4f640 1 -- 172.21.15.154:0/2242306645 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8400ca210 con 0x7fc84006b1e0 2023-09-23T03:17:04.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.345+0000 7fc845e4f640 1 -- 172.21.15.154:0/2242306645 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8400c6f60 con 0x7fc84006b1e0 2023-09-23T03:17:04.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.346+0000 7fc845e4f640 1 -- 172.21.15.154:0/2242306645 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8400c4280 con 0x7fc84006b1e0 2023-09-23T03:17:04.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.347+0000 7fc837fff640 1 -- 172.21.15.154:0/2242306645 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fc83002cd50 con 0x7fc84006b1e0 2023-09-23T03:17:04.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.348+0000 7fc837fff640 1 --2- 172.21.15.154:0/2242306645 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc82006af30 0x7fc82006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:04.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.348+0000 7fc837fff640 1 -- 172.21.15.154:0/2242306645 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fc8300ac120 con 0x7fc84006b1e0 2023-09-23T03:17:04.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.349+0000 7fc83effd640 1 --2- 172.21.15.154:0/2242306645 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc82006af30 0x7fc82006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:04.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.349+0000 7fc83effd640 1 --2- 172.21.15.154:0/2242306645 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc82006af30 0x7fc82006d3f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc8400c4210 tx=0x7fc828008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:04.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.353+0000 7fc837fff640 1 -- 172.21.15.154:0/2242306645 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fc8400c4280 con 0x7fc84006b1e0 2023-09-23T03:17:04.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.612+0000 7fc845e4f640 1 -- 172.21.15.154:0/2242306645 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fc8400ccf80 con 0x7fc84006b1e0 2023-09-23T03:17:04.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.617+0000 7fc837fff640 1 -- 172.21.15.154:0/2242306645 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v4) v1 ==== 90+0+0 (secure 0 0 0) 0x7fc8400ccf80 con 0x7fc84006b1e0 2023-09-23T03:17:04.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.620+0000 7fc845e4f640 1 -- 172.21.15.154:0/2242306645 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc82006af30 msgr2=0x7fc82006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:04.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.620+0000 7fc845e4f640 1 --2- 172.21.15.154:0/2242306645 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc82006af30 0x7fc82006d3f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc8400c4210 tx=0x7fc828008040 comp rx=0 tx=0).stop 2023-09-23T03:17:04.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.620+0000 7fc845e4f640 1 -- 172.21.15.154:0/2242306645 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc84006b1e0 msgr2=0x7fc8400c6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:04.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.620+0000 7fc845e4f640 1 --2- 172.21.15.154:0/2242306645 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc84006b1e0 0x7fc8400c6a20 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc830036040 tx=0x7fc830004a40 comp rx=0 tx=0).stop 2023-09-23T03:17:04.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.620+0000 7fc845e4f640 1 -- 172.21.15.154:0/2242306645 shutdown_connections 2023-09-23T03:17:04.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.620+0000 7fc845e4f640 1 --2- 172.21.15.154:0/2242306645 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc82006af30 0x7fc82006d3f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:04.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.620+0000 7fc845e4f640 1 --2- 172.21.15.154:0/2242306645 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc84015cf40 0x7fc840149c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:04.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.620+0000 7fc845e4f640 1 --2- 172.21.15.154:0/2242306645 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc8400b21d0 0x7fc8401496e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:04.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.620+0000 7fc845e4f640 1 --2- 172.21.15.154:0/2242306645 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc84006b1e0 0x7fc8400c6a20 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:04.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.620+0000 7fc845e4f640 1 -- 172.21.15.154:0/2242306645 >> 172.21.15.154:0/2242306645 conn(0x7fc84006fc60 msgr2=0x7fc840070040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:04.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.620+0000 7fc845e4f640 1 -- 172.21.15.154:0/2242306645 shutdown_connections 2023-09-23T03:17:04.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.620+0000 7fc845e4f640 1 -- 172.21.15.154:0/2242306645 wait complete. 2023-09-23T03:17:04.640 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-09-23T03:17:04.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.851+0000 7fd61ffff640 1 Processor -- start 2023-09-23T03:17:04.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.851+0000 7fd61ffff640 1 -- start start 2023-09-23T03:17:04.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.853+0000 7fd61ffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd62006c7a0 0x7fd62015eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:04.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.853+0000 7fd61ffff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6200704d0 0x7fd62015afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:04.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.853+0000 7fd61ffff640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd62006bff0 con 0x7fd62015ab00 2023-09-23T03:17:04.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.853+0000 7fd61ffff640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd620155070 con 0x7fd62006c7a0 2023-09-23T03:17:04.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.853+0000 7fd61ffff640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd62015e0f0 con 0x7fd6200704d0 2023-09-23T03:17:04.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.853+0000 7fd61effd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd62006c7a0 0x7fd62015eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:04.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.853+0000 7fd61effd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd62006c7a0 0x7fd62015eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47058/0 (socket says 172.21.15.154:47058) 2023-09-23T03:17:04.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.853+0000 7fd61effd640 1 -- 172.21.15.154:0/1341796946 learned_addr learned my addr 172.21.15.154:0/1341796946 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:04.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.854+0000 7fd61f7fe640 1 --2- 172.21.15.154:0/1341796946 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6200704d0 0x7fd62015afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:04.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.854+0000 7fd61effd640 1 -- 172.21.15.154:0/1341796946 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6200704d0 msgr2=0x7fd62015afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:04.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.854+0000 7fd61effd640 1 --2- 172.21.15.154:0/1341796946 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6200704d0 0x7fd62015afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:04.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.854+0000 7fd61effd640 1 -- 172.21.15.154:0/1341796946 >> v1:172.21.15.190:6789/0 conn(0x7fd62015ab00 legacy=0x7fd62006fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:04.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.854+0000 7fd61effd640 1 -- 172.21.15.154:0/1341796946 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62006cb80 con 0x7fd62006c7a0 2023-09-23T03:17:04.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.856+0000 7fd61effd640 1 --2- 172.21.15.154:0/1341796946 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd62006c7a0 0x7fd62015eb00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd610009870 tx=0x7fd61002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e8cffc67ba4e3de server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:04.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.856+0000 7fd61e7fc640 1 -- 172.21.15.154:0/1341796946 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd610030840 con 0x7fd62006c7a0 2023-09-23T03:17:04.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.856+0000 7fd61e7fc640 1 -- 172.21.15.154:0/1341796946 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6100309a0 con 0x7fd62006c7a0 2023-09-23T03:17:04.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.856+0000 7fd61e7fc640 1 -- 172.21.15.154:0/1341796946 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd610030d30 con 0x7fd62006c7a0 2023-09-23T03:17:04.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.856+0000 7fd61ffff640 1 -- 172.21.15.154:0/1341796946 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd62006c7a0 msgr2=0x7fd62015eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:04.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.856+0000 7fd61ffff640 1 --2- 172.21.15.154:0/1341796946 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd62006c7a0 0x7fd62015eb00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd610009870 tx=0x7fd61002b980 comp rx=0 tx=0).stop 2023-09-23T03:17:04.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.857+0000 7fd61ffff640 1 -- 172.21.15.154:0/1341796946 shutdown_connections 2023-09-23T03:17:04.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.857+0000 7fd61ffff640 1 --2- 172.21.15.154:0/1341796946 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6200704d0 0x7fd62015afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:04.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.857+0000 7fd61ffff640 1 --2- 172.21.15.154:0/1341796946 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd62006c7a0 0x7fd62015eb00 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:04.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.857+0000 7fd61ffff640 1 -- 172.21.15.154:0/1341796946 >> 172.21.15.154:0/1341796946 conn(0x7fd62006a390 msgr2=0x7fd6200afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:04.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.857+0000 7fd61ffff640 1 -- 172.21.15.154:0/1341796946 shutdown_connections 2023-09-23T03:17:04.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.857+0000 7fd61ffff640 1 -- 172.21.15.154:0/1341796946 wait complete. 2023-09-23T03:17:04.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.858+0000 7fd61ffff640 1 Processor -- start 2023-09-23T03:17:04.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.859+0000 7fd61ffff640 1 -- start start 2023-09-23T03:17:04.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.860+0000 7fd61ffff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd62006c7a0 0x7fd6201f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:04.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.860+0000 7fd61ffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6200704d0 0x7fd6201f5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:04.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.860+0000 7fd61ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd62015ab00 0x7fd6200ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:04.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.860+0000 7fd61ffff640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6200c7cc0 con 0x7fd62015ab00 2023-09-23T03:17:04.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.860+0000 7fd61ffff640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6201f2d20 con 0x7fd6200704d0 2023-09-23T03:17:04.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.860+0000 7fd61ffff640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6201f50e0 con 0x7fd62006c7a0 2023-09-23T03:17:04.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.860+0000 7fd61f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd62015ab00 0x7fd6200ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:04.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.860+0000 7fd61f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd62015ab00 0x7fd6200ccee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47060/0 (socket says 172.21.15.154:47060) 2023-09-23T03:17:04.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.860+0000 7fd61f7fe640 1 -- 172.21.15.154:0/4036780517 learned_addr learned my addr 172.21.15.154:0/4036780517 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:04.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.860+0000 7fd61effd640 1 --2- 172.21.15.154:0/4036780517 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd62006c7a0 0x7fd6201f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:04.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.860+0000 7fd617fff640 1 --2- 172.21.15.154:0/4036780517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6200704d0 0x7fd6201f5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:04.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.860+0000 7fd61f7fe640 1 -- 172.21.15.154:0/4036780517 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd62006c7a0 msgr2=0x7fd6201f4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:04.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.860+0000 7fd61f7fe640 1 --2- 172.21.15.154:0/4036780517 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd62006c7a0 0x7fd6201f4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:04.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.860+0000 7fd61f7fe640 1 -- 172.21.15.154:0/4036780517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6200704d0 msgr2=0x7fd6201f5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:04.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.860+0000 7fd61f7fe640 1 --2- 172.21.15.154:0/4036780517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6200704d0 0x7fd6201f5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:04.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.860+0000 7fd61f7fe640 1 -- 172.21.15.154:0/4036780517 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6100094d0 con 0x7fd62015ab00 2023-09-23T03:17:04.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.861+0000 7fd61f7fe640 1 --2- 172.21.15.154:0/4036780517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd62015ab00 0x7fd6200ccee0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd618007d80 tx=0x7fd61800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:04.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.861+0000 7fd61cff9640 1 -- 172.21.15.154:0/4036780517 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd618014420 con 0x7fd62015ab00 2023-09-23T03:17:04.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.861+0000 7fd61cff9640 1 -- 172.21.15.154:0/4036780517 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd618014580 con 0x7fd62015ab00 2023-09-23T03:17:04.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.861+0000 7fd61cff9640 1 -- 172.21.15.154:0/4036780517 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd618014930 con 0x7fd62015ab00 2023-09-23T03:17:04.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.861+0000 7fd61ffff640 1 -- 172.21.15.154:0/4036780517 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6201f61f0 con 0x7fd62015ab00 2023-09-23T03:17:04.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.861+0000 7fd61ffff640 1 -- 172.21.15.154:0/4036780517 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6201f23f0 con 0x7fd62015ab00 2023-09-23T03:17:04.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.862+0000 7fd61ffff640 1 -- 172.21.15.154:0/4036780517 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6200736b0 con 0x7fd62015ab00 2023-09-23T03:17:04.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.863+0000 7fd61cff9640 1 -- 172.21.15.154:0/4036780517 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fd618014a90 con 0x7fd62015ab00 2023-09-23T03:17:04.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.864+0000 7fd61cff9640 1 --2- 172.21.15.154:0/4036780517 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd60006af30 0x7fd60006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:04.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.864+0000 7fd61cff9640 1 -- 172.21.15.154:0/4036780517 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fd618089e50 con 0x7fd62015ab00 2023-09-23T03:17:04.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.864+0000 7fd61effd640 1 --2- 172.21.15.154:0/4036780517 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd60006af30 0x7fd60006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:04.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.865+0000 7fd61effd640 1 --2- 172.21.15.154:0/4036780517 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd60006af30 0x7fd60006d3f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd610036000 tx=0x7fd61003b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:04.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:04.868+0000 7fd61cff9640 1 -- 172.21.15.154:0/4036780517 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fd618055090 con 0x7fd62015ab00 2023-09-23T03:17:05.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.113+0000 7fd61ffff640 1 -- 172.21.15.154:0/4036780517 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fd6201f2f50 con 0x7fd62015ab00 2023-09-23T03:17:05.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.118+0000 7fd61cff9640 1 -- 172.21.15.154:0/4036780517 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd6180586b0 con 0x7fd62015ab00 2023-09-23T03:17:05.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.122+0000 7fd61ffff640 1 -- 172.21.15.154:0/4036780517 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd60006af30 msgr2=0x7fd60006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:05.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.122+0000 7fd61ffff640 1 --2- 172.21.15.154:0/4036780517 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd60006af30 0x7fd60006d3f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd610036000 tx=0x7fd61003b040 comp rx=0 tx=0).stop 2023-09-23T03:17:05.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.122+0000 7fd61ffff640 1 -- 172.21.15.154:0/4036780517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd62015ab00 msgr2=0x7fd6200ccee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:05.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.122+0000 7fd61ffff640 1 --2- 172.21.15.154:0/4036780517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd62015ab00 0x7fd6200ccee0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd618007d80 tx=0x7fd61800e4b0 comp rx=0 tx=0).stop 2023-09-23T03:17:05.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.122+0000 7fd61ffff640 1 -- 172.21.15.154:0/4036780517 shutdown_connections 2023-09-23T03:17:05.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.122+0000 7fd61ffff640 1 --2- 172.21.15.154:0/4036780517 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd60006af30 0x7fd60006d3f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:05.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.122+0000 7fd61ffff640 1 --2- 172.21.15.154:0/4036780517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd62015ab00 0x7fd6200ccee0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:05.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.122+0000 7fd61ffff640 1 --2- 172.21.15.154:0/4036780517 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6200704d0 0x7fd6201f5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:05.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.122+0000 7fd61ffff640 1 --2- 172.21.15.154:0/4036780517 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd62006c7a0 0x7fd6201f4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:05.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.122+0000 7fd61ffff640 1 -- 172.21.15.154:0/4036780517 >> 172.21.15.154:0/4036780517 conn(0x7fd62006a390 msgr2=0x7fd6200ce210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:05.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.122+0000 7fd61ffff640 1 -- 172.21.15.154:0/4036780517 shutdown_connections 2023-09-23T03:17:05.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.122+0000 7fd61ffff640 1 -- 172.21.15.154:0/4036780517 wait complete. 2023-09-23T03:17:05.139 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-09-23T03:17:05.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.342+0000 7fc9fb59e640 1 Processor -- start 2023-09-23T03:17:05.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.342+0000 7fc9fb59e640 1 -- start start 2023-09-23T03:17:05.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.344+0000 7fc9fb59e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9fc0cb3e0 0x7fc9fc0cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:05.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.344+0000 7fc9fb59e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc9fc15c8c0 0x7fc9fc15cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:05.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.344+0000 7fc9fb59e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9fc15d630 con 0x7fc9fc1568e0 2023-09-23T03:17:05.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.344+0000 7fc9fb59e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9fc0cd4d0 con 0x7fc9fc15c8c0 2023-09-23T03:17:05.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.344+0000 7fc9fb59e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9fc0ccba0 con 0x7fc9fc0cb3e0 2023-09-23T03:17:05.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.344+0000 7fc9fa59c640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fc9fc1568e0 0x7fc9fc0cacd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:42242/0 (socket says 172.21.15.154:42242) 2023-09-23T03:17:05.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.344+0000 7fc9fa59c640 1 -- 172.21.15.154:0/911189997 learned_addr learned my addr 172.21.15.154:0/911189997 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:17:05.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.345+0000 7fc9fad9d640 1 --2- 172.21.15.154:0/911189997 >> v2:172.21.15.190:3300/0 conn(0x7fc9fc15c8c0 0x7fc9fc15cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:05.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.345+0000 7fc9f9d9b640 1 -- 172.21.15.154:0/911189997 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3960392784 0 0) 0x7fc9fc15d630 con 0x7fc9fc1568e0 2023-09-23T03:17:05.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.345+0000 7fc9f1d9b640 1 --2- 172.21.15.154:0/911189997 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9fc0cb3e0 0x7fc9fc0cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:05.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.345+0000 7fc9f9d9b640 1 -- 172.21.15.154:0/911189997 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9e8003520 con 0x7fc9fc1568e0 2023-09-23T03:17:05.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.345+0000 7fc9f9d9b640 1 -- 172.21.15.154:0/911189997 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4033191918 0 0) 0x7fc9e8003520 con 0x7fc9fc1568e0 2023-09-23T03:17:05.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.345+0000 7fc9f9d9b640 1 -- 172.21.15.154:0/911189997 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9fc0cb3e0 msgr2=0x7fc9fc0cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:05.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.345+0000 7fc9f9d9b640 1 --2- 172.21.15.154:0/911189997 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9fc0cb3e0 0x7fc9fc0cd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:05.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.345+0000 7fc9f9d9b640 1 -- 172.21.15.154:0/911189997 >> v2:172.21.15.190:3300/0 conn(0x7fc9fc15c8c0 msgr2=0x7fc9fc15cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:05.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.345+0000 7fc9f9d9b640 1 --2- 172.21.15.154:0/911189997 >> v2:172.21.15.190:3300/0 conn(0x7fc9fc15c8c0 0x7fc9fc15cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:05.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.345+0000 7fc9f9d9b640 1 -- 172.21.15.154:0/911189997 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9fc0cce80 con 0x7fc9fc1568e0 2023-09-23T03:17:05.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.345+0000 7fc9f9d9b640 1 -- 172.21.15.154:0/911189997 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fc9ec001670 con 0x7fc9fc1568e0 2023-09-23T03:17:05.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.345+0000 7fc9f9d9b640 1 -- 172.21.15.154:0/911189997 >> v1:172.21.15.154:6789/0 conn(0x7fc9fc1568e0 legacy=0x7fc9fc0cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:05.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.345+0000 7fc9f9d9b640 1 --2- 172.21.15.154:0/911189997 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9fc0cb3e0 0x7fc9e8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:05.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.345+0000 7fc9f9d9b640 1 --2- 172.21.15.154:0/911189997 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9e8003ff0 0x7fc9e80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:05.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.345+0000 7fc9f9d9b640 1 --2- 172.21.15.154:0/911189997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9e8004d00 0x7fc9e8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:05.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.345+0000 7fc9f9d9b640 1 -- 172.21.15.154:0/911189997 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9fc0cd4d0 con 0x7fc9e8003ff0 2023-09-23T03:17:05.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.345+0000 7fc9f9d9b640 1 -- 172.21.15.154:0/911189997 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9fc0ccba0 con 0x7fc9e8004d00 2023-09-23T03:17:05.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.345+0000 7fc9f9d9b640 1 -- 172.21.15.154:0/911189997 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e800dda0 con 0x7fc9fc0cb3e0 2023-09-23T03:17:05.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.345+0000 7fc9f1d9b640 1 --2- 172.21.15.154:0/911189997 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9e8003ff0 0x7fc9e80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:05.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.346+0000 7fc9fa59c640 1 --2- 172.21.15.154:0/911189997 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9fc0cb3e0 0x7fc9e8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:05.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.346+0000 7fc9fad9d640 1 --2- 172.21.15.154:0/911189997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9e8004d00 0x7fc9e8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:05.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.346+0000 7fc9f1d9b640 1 -- 172.21.15.154:0/911189997 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9fc0cb3e0 msgr2=0x7fc9e8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:05.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.346+0000 7fc9f1d9b640 1 --2- 172.21.15.154:0/911189997 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9fc0cb3e0 0x7fc9e8003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:05.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.346+0000 7fc9f1d9b640 1 -- 172.21.15.154:0/911189997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9e8004d00 msgr2=0x7fc9e8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:05.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.346+0000 7fc9f1d9b640 1 --2- 172.21.15.154:0/911189997 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9e8004d00 0x7fc9e8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:05.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.346+0000 7fc9f1d9b640 1 -- 172.21.15.154:0/911189997 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e8003520 con 0x7fc9e8003ff0 2023-09-23T03:17:05.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.347+0000 7fc9f1d9b640 1 --2- 172.21.15.154:0/911189997 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9e8003ff0 0x7fc9e80047c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc9e400ba30 tx=0x7fc9e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7069fca13aeb2517 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:05.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.347+0000 7fc9f9d9b640 1 -- 172.21.15.154:0/911189997 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4032640 con 0x7fc9e8003ff0 2023-09-23T03:17:05.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.347+0000 7fc9f9d9b640 1 -- 172.21.15.154:0/911189997 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e40327a0 con 0x7fc9e8003ff0 2023-09-23T03:17:05.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.347+0000 7fc9f9d9b640 1 -- 172.21.15.154:0/911189997 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4032b30 con 0x7fc9e8003ff0 2023-09-23T03:17:05.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.348+0000 7fc9fb59e640 1 -- 172.21.15.154:0/911189997 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9e8003ff0 msgr2=0x7fc9e80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:05.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.348+0000 7fc9fb59e640 1 --2- 172.21.15.154:0/911189997 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9e8003ff0 0x7fc9e80047c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc9e400ba30 tx=0x7fc9e80057f0 comp rx=0 tx=0).stop 2023-09-23T03:17:05.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.348+0000 7fc9fad9d640 1 -- 172.21.15.154:0/911189997 reap_dead start 2023-09-23T03:17:05.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.348+0000 7fc9fb59e640 1 -- 172.21.15.154:0/911189997 shutdown_connections 2023-09-23T03:17:05.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.348+0000 7fc9fb59e640 1 -- 172.21.15.154:0/911189997 >> 172.21.15.154:0/911189997 conn(0x7fc9fc06a390 msgr2=0x7fc9fc0afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:05.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.348+0000 7fc9fb59e640 1 -- 172.21.15.154:0/911189997 shutdown_connections 2023-09-23T03:17:05.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.348+0000 7fc9fb59e640 1 -- 172.21.15.154:0/911189997 wait complete. 2023-09-23T03:17:05.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.350+0000 7fc9fb59e640 1 Processor -- start 2023-09-23T03:17:05.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.350+0000 7fc9fb59e640 1 -- start start 2023-09-23T03:17:05.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.351+0000 7fc9fb59e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9e8003ff0 0x7fc9fc0c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:05.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.351+0000 7fc9fb59e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9fc0b0f00 0x7fc9fc0b1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:05.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.351+0000 7fc9fb59e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9fc0c46f0 0x7fc9fc0c4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:05.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.351+0000 7fc9fb59e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9fc0c50b0 con 0x7fc9fc0c46f0 2023-09-23T03:17:05.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.351+0000 7fc9fb59e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9fc0b2680 con 0x7fc9fc0b0f00 2023-09-23T03:17:05.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.351+0000 7fc9fb59e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9fc157db0 con 0x7fc9e8003ff0 2023-09-23T03:17:05.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.351+0000 7fc9fad9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9fc0c46f0 0x7fc9fc0c4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:05.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.352+0000 7fc9fad9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9fc0c46f0 0x7fc9fc0c4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47076/0 (socket says 172.21.15.154:47076) 2023-09-23T03:17:05.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.352+0000 7fc9fad9d640 1 -- 172.21.15.154:0/4144635984 learned_addr learned my addr 172.21.15.154:0/4144635984 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:05.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.352+0000 7fc9f1d9b640 1 --2- 172.21.15.154:0/4144635984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9fc0b0f00 0x7fc9fc0b1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:05.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.352+0000 7fc9fad9d640 1 -- 172.21.15.154:0/4144635984 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9e8003ff0 msgr2=0x7fc9fc0c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:05.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.352+0000 7fc9fad9d640 1 --2- 172.21.15.154:0/4144635984 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9e8003ff0 0x7fc9fc0c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:05.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.352+0000 7fc9fad9d640 1 -- 172.21.15.154:0/4144635984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9fc0b0f00 msgr2=0x7fc9fc0b1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:05.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.352+0000 7fc9fad9d640 1 --2- 172.21.15.154:0/4144635984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9fc0b0f00 0x7fc9fc0b1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:05.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.352+0000 7fc9fad9d640 1 -- 172.21.15.154:0/4144635984 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9fc0b1a70 con 0x7fc9fc0c46f0 2023-09-23T03:17:05.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.352+0000 7fc9fad9d640 1 --2- 172.21.15.154:0/4144635984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9fc0c46f0 0x7fc9fc0c4b70 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc06e2b0 tx=0x7fc9f4007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:05.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.352+0000 7fc9f37fe640 1 -- 172.21.15.154:0/4144635984 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f4009e70 con 0x7fc9fc0c46f0 2023-09-23T03:17:05.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.352+0000 7fc9f37fe640 1 -- 172.21.15.154:0/4144635984 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f400f050 con 0x7fc9fc0c46f0 2023-09-23T03:17:05.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.352+0000 7fc9f37fe640 1 -- 172.21.15.154:0/4144635984 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f4014670 con 0x7fc9fc0c46f0 2023-09-23T03:17:05.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.352+0000 7fc9fb59e640 1 -- 172.21.15.154:0/4144635984 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9fc0b1d50 con 0x7fc9fc0c46f0 2023-09-23T03:17:05.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.353+0000 7fc9fb59e640 1 -- 172.21.15.154:0/4144635984 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9fc0b2310 con 0x7fc9fc0c46f0 2023-09-23T03:17:05.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.354+0000 7fc9fb59e640 1 -- 172.21.15.154:0/4144635984 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9fc0b24d0 con 0x7fc9fc0c46f0 2023-09-23T03:17:05.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.355+0000 7fc9f37fe640 1 -- 172.21.15.154:0/4144635984 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fc9f40147d0 con 0x7fc9fc0c46f0 2023-09-23T03:17:05.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.356+0000 7fc9f37fe640 1 --2- 172.21.15.154:0/4144635984 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc9dc06af30 0x7fc9dc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:05.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.356+0000 7fc9f37fe640 1 -- 172.21.15.154:0/4144635984 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fc9f4089b80 con 0x7fc9fc0c46f0 2023-09-23T03:17:05.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.356+0000 7fc9fa59c640 1 --2- 172.21.15.154:0/4144635984 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc9dc06af30 0x7fc9dc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:05.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.356+0000 7fc9fa59c640 1 --2- 172.21.15.154:0/4144635984 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc9dc06af30 0x7fc9dc06d3f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec005990 tx=0x7fc9ec000f50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:05.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.360+0000 7fc9f37fe640 1 -- 172.21.15.154:0/4144635984 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fc9f4054d30 con 0x7fc9fc0c46f0 2023-09-23T03:17:05.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.620+0000 7fc9fb59e640 1 -- 172.21.15.154:0/4144635984 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fc9fc0736b0 con 0x7fc9fc0c46f0 2023-09-23T03:17:05.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.624+0000 7fc9f37fe640 1 -- 172.21.15.154:0/4144635984 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7fc9f4058350 con 0x7fc9fc0c46f0 2023-09-23T03:17:05.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.627+0000 7fc9fb59e640 1 -- 172.21.15.154:0/4144635984 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc9dc06af30 msgr2=0x7fc9dc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:05.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.627+0000 7fc9fb59e640 1 --2- 172.21.15.154:0/4144635984 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc9dc06af30 0x7fc9dc06d3f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec005990 tx=0x7fc9ec000f50 comp rx=0 tx=0).stop 2023-09-23T03:17:05.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.627+0000 7fc9fb59e640 1 -- 172.21.15.154:0/4144635984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9fc0c46f0 msgr2=0x7fc9fc0c4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:05.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.627+0000 7fc9fb59e640 1 --2- 172.21.15.154:0/4144635984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9fc0c46f0 0x7fc9fc0c4b70 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc06e2b0 tx=0x7fc9f4007700 comp rx=0 tx=0).stop 2023-09-23T03:17:05.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.628+0000 7fc9fb59e640 1 -- 172.21.15.154:0/4144635984 shutdown_connections 2023-09-23T03:17:05.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.628+0000 7fc9fb59e640 1 --2- 172.21.15.154:0/4144635984 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc9dc06af30 0x7fc9dc06d3f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:05.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.628+0000 7fc9fb59e640 1 --2- 172.21.15.154:0/4144635984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9fc0c46f0 0x7fc9fc0c4b70 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:05.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.628+0000 7fc9fb59e640 1 --2- 172.21.15.154:0/4144635984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc9fc0b0f00 0x7fc9fc0b1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:05.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.628+0000 7fc9fb59e640 1 --2- 172.21.15.154:0/4144635984 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9e8003ff0 0x7fc9fc0c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:05.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.628+0000 7fc9fb59e640 1 -- 172.21.15.154:0/4144635984 >> 172.21.15.154:0/4144635984 conn(0x7fc9fc06a390 msgr2=0x7fc9fc155aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:05.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.628+0000 7fc9fb59e640 1 -- 172.21.15.154:0/4144635984 shutdown_connections 2023-09-23T03:17:05.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.628+0000 7fc9fb59e640 1 -- 172.21.15.154:0/4144635984 wait complete. 2023-09-23T03:17:05.648 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-09-23T03:17:05.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.856+0000 7fc1d48ba640 1 Processor -- start 2023-09-23T03:17:05.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.856+0000 7fc1d48ba640 1 -- start start 2023-09-23T03:17:05.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.857+0000 7fc1d48ba640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc1d00cf8e0 0x7fc1d015c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:05.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.857+0000 7fc1d48ba640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1d006be10 con 0x7fc1d00cf500 2023-09-23T03:17:05.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.857+0000 7fc1d48ba640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1d00d17f0 con 0x7fc1d015ca30 2023-09-23T03:17:05.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.857+0000 7fc1d48ba640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1d00d0470 con 0x7fc1d00cf8e0 2023-09-23T03:17:05.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.857+0000 7fc1ced9d640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fc1d00cf500 0x7fc1d00ce050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:42244/0 (socket says 172.21.15.154:42244) 2023-09-23T03:17:05.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.857+0000 7fc1ced9d640 1 -- 172.21.15.154:0/4163879579 learned_addr learned my addr 172.21.15.154:0/4163879579 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:17:05.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.858+0000 7fc1cdd9b640 1 -- 172.21.15.154:0/4163879579 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3313883602 0 0) 0x7fc1d006be10 con 0x7fc1d00cf500 2023-09-23T03:17:05.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.858+0000 7fc1cdd9b640 1 -- 172.21.15.154:0/4163879579 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1bc003520 con 0x7fc1d00cf500 2023-09-23T03:17:05.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.858+0000 7fc1ce59c640 1 --2- 172.21.15.154:0/4163879579 >> v2:172.21.15.190:3300/0 conn(0x7fc1d00cf8e0 0x7fc1d015c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:05.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.858+0000 7fc1cdd9b640 1 -- 172.21.15.154:0/4163879579 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1525604331 0 0) 0x7fc1bc003520 con 0x7fc1d00cf500 2023-09-23T03:17:05.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.858+0000 7fc1cdd9b640 1 -- 172.21.15.154:0/4163879579 >> v2:172.21.15.190:3300/0 conn(0x7fc1d00cf8e0 msgr2=0x7fc1d015c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:05.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.858+0000 7fc1cdd9b640 1 --2- 172.21.15.154:0/4163879579 >> v2:172.21.15.190:3300/0 conn(0x7fc1d00cf8e0 0x7fc1d015c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:05.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.858+0000 7fc1cdd9b640 1 -- 172.21.15.154:0/4163879579 >> v1:172.21.15.190:6789/0 conn(0x7fc1d015ca30 legacy=0x7fc1d00b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:05.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.858+0000 7fc1cdd9b640 1 -- 172.21.15.154:0/4163879579 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d00d0750 con 0x7fc1d00cf500 2023-09-23T03:17:05.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.858+0000 7fc1cdd9b640 1 -- 172.21.15.154:0/4163879579 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fc1c0001b70 con 0x7fc1d00cf500 2023-09-23T03:17:05.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.858+0000 7fc1cdd9b640 1 -- 172.21.15.154:0/4163879579 >> v1:172.21.15.154:6789/0 conn(0x7fc1d00cf500 legacy=0x7fc1d00ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:05.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.858+0000 7fc1cdd9b640 1 --2- 172.21.15.154:0/4163879579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1bc0039a0 0x7fc1bc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:05.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.858+0000 7fc1cdd9b640 1 --2- 172.21.15.154:0/4163879579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1bc0047c0 0x7fc1bc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:05.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.858+0000 7fc1cdd9b640 1 --2- 172.21.15.154:0/4163879579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1bc005380 0x7fc1bc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:05.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.858+0000 7fc1cdd9b640 1 -- 172.21.15.154:0/4163879579 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d00d0470 con 0x7fc1bc0047c0 2023-09-23T03:17:05.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.858+0000 7fc1cdd9b640 1 -- 172.21.15.154:0/4163879579 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc00c260 con 0x7fc1bc005380 2023-09-23T03:17:05.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.858+0000 7fc1cdd9b640 1 -- 172.21.15.154:0/4163879579 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc00c420 con 0x7fc1bc0039a0 2023-09-23T03:17:05.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.859+0000 7fc1ce59c640 1 --2- 172.21.15.154:0/4163879579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1bc0047c0 0x7fc1bc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:05.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.859+0000 7fc1ced9d640 1 --2- 172.21.15.154:0/4163879579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1bc0039a0 0x7fc1bc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:05.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.859+0000 7fc1ce59c640 1 -- 172.21.15.154:0/4163879579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1bc0039a0 msgr2=0x7fc1bc003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:05.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.859+0000 7fc1ce59c640 1 --2- 172.21.15.154:0/4163879579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1bc0039a0 0x7fc1bc003da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:05.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.859+0000 7fc1ce59c640 1 -- 172.21.15.154:0/4163879579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1bc005380 msgr2=0x7fc1bc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:05.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.859+0000 7fc1ce59c640 1 --2- 172.21.15.154:0/4163879579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1bc005380 0x7fc1bc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:05.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.859+0000 7fc1ce59c640 1 -- 172.21.15.154:0/4163879579 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1bc003520 con 0x7fc1bc0047c0 2023-09-23T03:17:05.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.859+0000 7fc1cf59e640 1 -- 172.21.15.154:0/4163879579 reap_dead start 2023-09-23T03:17:05.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.860+0000 7fc1ce59c640 1 --2- 172.21.15.154:0/4163879579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1bc0047c0 0x7fc1bc004c40 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800a9a0 tx=0x7fc1bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb2dba7201a40ebc server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:05.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.860+0000 7fc1cdd9b640 1 -- 172.21.15.154:0/4163879579 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8030660 con 0x7fc1bc0047c0 2023-09-23T03:17:05.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.860+0000 7fc1cdd9b640 1 -- 172.21.15.154:0/4163879579 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b80307c0 con 0x7fc1bc0047c0 2023-09-23T03:17:05.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.860+0000 7fc1cdd9b640 1 -- 172.21.15.154:0/4163879579 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8030bb0 con 0x7fc1bc0047c0 2023-09-23T03:17:05.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.861+0000 7fc1d48ba640 1 -- 172.21.15.154:0/4163879579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1bc0047c0 msgr2=0x7fc1bc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:05.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.861+0000 7fc1d48ba640 1 --2- 172.21.15.154:0/4163879579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1bc0047c0 0x7fc1bc004c40 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800a9a0 tx=0x7fc1bc0057f0 comp rx=0 tx=0).stop 2023-09-23T03:17:05.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.861+0000 7fc1d48ba640 1 -- 172.21.15.154:0/4163879579 shutdown_connections 2023-09-23T03:17:05.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.861+0000 7fc1d48ba640 1 --2- 172.21.15.154:0/4163879579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1bc0047c0 0x7fc1bc004c40 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:05.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.861+0000 7fc1d48ba640 1 -- 172.21.15.154:0/4163879579 >> 172.21.15.154:0/4163879579 conn(0x7fc1d009bbb0 msgr2=0x7fc1d00d28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:05.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.861+0000 7fc1d48ba640 1 -- 172.21.15.154:0/4163879579 shutdown_connections 2023-09-23T03:17:05.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.861+0000 7fc1d48ba640 1 -- 172.21.15.154:0/4163879579 wait complete. 2023-09-23T03:17:05.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.863+0000 7fc1d48ba640 1 Processor -- start 2023-09-23T03:17:05.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.863+0000 7fc1d48ba640 1 -- start start 2023-09-23T03:17:05.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.864+0000 7fc1d48ba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1bc0047c0 0x7fc1d01f2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:05.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.864+0000 7fc1d48ba640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1d00c80a0 0x7fc1d00cbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:05.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.864+0000 7fc1d48ba640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1d015e540 0x7fc1d015e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:05.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.864+0000 7fc1d48ba640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d01f6180 con 0x7fc1bc0047c0 2023-09-23T03:17:05.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.864+0000 7fc1d48ba640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d01f5570 con 0x7fc1d00c80a0 2023-09-23T03:17:05.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.864+0000 7fc1d48ba640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d01f5850 con 0x7fc1d015e540 2023-09-23T03:17:05.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.864+0000 7fc1ced9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1bc0047c0 0x7fc1d01f2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:05.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.864+0000 7fc1ced9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1bc0047c0 0x7fc1d01f2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47090/0 (socket says 172.21.15.154:47090) 2023-09-23T03:17:05.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.864+0000 7fc1ced9d640 1 -- 172.21.15.154:0/3446278202 learned_addr learned my addr 172.21.15.154:0/3446278202 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:05.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.865+0000 7fc1ce59c640 1 --2- 172.21.15.154:0/3446278202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1d00c80a0 0x7fc1d00cbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:05.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.865+0000 7fc1cf59e640 1 --2- 172.21.15.154:0/3446278202 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1d015e540 0x7fc1d015e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:05.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.865+0000 7fc1ced9d640 1 -- 172.21.15.154:0/3446278202 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1d015e540 msgr2=0x7fc1d015e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:05.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.865+0000 7fc1ced9d640 1 --2- 172.21.15.154:0/3446278202 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1d015e540 0x7fc1d015e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:05.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.865+0000 7fc1ced9d640 1 -- 172.21.15.154:0/3446278202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1d00c80a0 msgr2=0x7fc1d00cbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:05.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.865+0000 7fc1ced9d640 1 --2- 172.21.15.154:0/3446278202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1d00c80a0 0x7fc1d00cbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:05.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.865+0000 7fc1ced9d640 1 -- 172.21.15.154:0/3446278202 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d01f5b30 con 0x7fc1bc0047c0 2023-09-23T03:17:05.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.865+0000 7fc1cf59e640 1 --2- 172.21.15.154:0/3446278202 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1d015e540 0x7fc1d015e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:05.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.865+0000 7fc1ced9d640 1 --2- 172.21.15.154:0/3446278202 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1bc0047c0 0x7fc1d01f2ab0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc1c000d9b0 tx=0x7fc1c000de70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:05.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.865+0000 7fc1c77fe640 1 -- 172.21.15.154:0/3446278202 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c000b770 con 0x7fc1bc0047c0 2023-09-23T03:17:05.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.865+0000 7fc1c77fe640 1 -- 172.21.15.154:0/3446278202 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1c000b8d0 con 0x7fc1bc0047c0 2023-09-23T03:17:05.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.865+0000 7fc1c77fe640 1 -- 172.21.15.154:0/3446278202 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c000bc40 con 0x7fc1bc0047c0 2023-09-23T03:17:05.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.865+0000 7fc1d48ba640 1 -- 172.21.15.154:0/3446278202 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1d01f5e10 con 0x7fc1bc0047c0 2023-09-23T03:17:05.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.866+0000 7fc1d48ba640 1 -- 172.21.15.154:0/3446278202 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1d00c8520 con 0x7fc1bc0047c0 2023-09-23T03:17:05.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.867+0000 7fc1d48ba640 1 -- 172.21.15.154:0/3446278202 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1d004e920 con 0x7fc1bc0047c0 2023-09-23T03:17:05.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.868+0000 7fc1c77fe640 1 -- 172.21.15.154:0/3446278202 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fc1c0007070 con 0x7fc1bc0047c0 2023-09-23T03:17:05.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.868+0000 7fc1c77fe640 1 --2- 172.21.15.154:0/3446278202 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc1b006af60 0x7fc1b006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:05.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.869+0000 7fc1c77fe640 1 -- 172.21.15.154:0/3446278202 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fc1c00899c0 con 0x7fc1bc0047c0 2023-09-23T03:17:05.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.869+0000 7fc1ce59c640 1 --2- 172.21.15.154:0/3446278202 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc1b006af60 0x7fc1b006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:05.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.869+0000 7fc1ce59c640 1 --2- 172.21.15.154:0/3446278202 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc1b006af60 0x7fc1b006d420 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8036000 tx=0x7fc1b8031000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:05.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:05.873+0000 7fc1c77fe640 1 -- 172.21.15.154:0/3446278202 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fc1c0054b70 con 0x7fc1bc0047c0 2023-09-23T03:17:06.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.112+0000 7fc1d48ba640 1 -- 172.21.15.154:0/3446278202 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fc1d00d12c0 con 0x7fc1bc0047c0 2023-09-23T03:17:06.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.117+0000 7fc1c77fe640 1 -- 172.21.15.154:0/3446278202 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7fc1c0058190 con 0x7fc1bc0047c0 2023-09-23T03:17:06.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.119+0000 7fc1d48ba640 1 -- 172.21.15.154:0/3446278202 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc1b006af60 msgr2=0x7fc1b006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:06.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.119+0000 7fc1d48ba640 1 --2- 172.21.15.154:0/3446278202 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc1b006af60 0x7fc1b006d420 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8036000 tx=0x7fc1b8031000 comp rx=0 tx=0).stop 2023-09-23T03:17:06.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.119+0000 7fc1d48ba640 1 -- 172.21.15.154:0/3446278202 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1bc0047c0 msgr2=0x7fc1d01f2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:06.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.119+0000 7fc1d48ba640 1 --2- 172.21.15.154:0/3446278202 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1bc0047c0 0x7fc1d01f2ab0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc1c000d9b0 tx=0x7fc1c000de70 comp rx=0 tx=0).stop 2023-09-23T03:17:06.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.119+0000 7fc1d48ba640 1 -- 172.21.15.154:0/3446278202 shutdown_connections 2023-09-23T03:17:06.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.119+0000 7fc1d48ba640 1 --2- 172.21.15.154:0/3446278202 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc1b006af60 0x7fc1b006d420 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:06.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.119+0000 7fc1d48ba640 1 --2- 172.21.15.154:0/3446278202 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc1d015e540 0x7fc1d015e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:06.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.119+0000 7fc1d48ba640 1 --2- 172.21.15.154:0/3446278202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc1d00c80a0 0x7fc1d00cbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:06.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.119+0000 7fc1d48ba640 1 --2- 172.21.15.154:0/3446278202 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1bc0047c0 0x7fc1d01f2ab0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:06.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.120+0000 7fc1d48ba640 1 -- 172.21.15.154:0/3446278202 >> 172.21.15.154:0/3446278202 conn(0x7fc1d009bbb0 msgr2=0x7fc1d015b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:06.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.120+0000 7fc1d48ba640 1 -- 172.21.15.154:0/3446278202 shutdown_connections 2023-09-23T03:17:06.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.120+0000 7fc1d48ba640 1 -- 172.21.15.154:0/3446278202 wait complete. 2023-09-23T03:17:06.139 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-09-23T03:17:06.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.349+0000 7f1519b39640 1 Processor -- start 2023-09-23T03:17:06.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.349+0000 7f1519b39640 1 -- start start 2023-09-23T03:17:06.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.350+0000 7f1519b39640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f151415ca80 0x7f1514160680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:06.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.350+0000 7f1519b39640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f15140b2620 0x7f15140d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:06.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.350+0000 7f1519b39640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1514161500 con 0x7f15140d25b0 2023-09-23T03:17:06.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.351+0000 7f1519b39640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15140b2aa0 con 0x7f15140b2620 2023-09-23T03:17:06.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.351+0000 7f1519b39640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1514160c60 con 0x7f151415ca80 2023-09-23T03:17:06.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.351+0000 7f1513fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f15140d25b0 0x7f15140d2a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:42256/0 (socket says 172.21.15.154:42256) 2023-09-23T03:17:06.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.351+0000 7f1513fff640 1 -- 172.21.15.154:0/2236657558 learned_addr learned my addr 172.21.15.154:0/2236657558 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:17:06.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.351+0000 7f15137fe640 1 --2- 172.21.15.154:0/2236657558 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f151415ca80 0x7f1514160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:06.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.351+0000 7f150bfff640 1 --2- 172.21.15.154:0/2236657558 >> v2:172.21.15.190:3300/0 conn(0x7f15140b2620 0x7f15140d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:06.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.351+0000 7f1512ffd640 1 -- 172.21.15.154:0/2236657558 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2586055251 0 0) 0x7f1514161500 con 0x7f15140d25b0 2023-09-23T03:17:06.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.351+0000 7f1512ffd640 1 -- 172.21.15.154:0/2236657558 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1500003520 con 0x7f15140d25b0 2023-09-23T03:17:06.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.352+0000 7f1512ffd640 1 -- 172.21.15.154:0/2236657558 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1193168780 0 0) 0x7f1500003520 con 0x7f15140d25b0 2023-09-23T03:17:06.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.352+0000 7f1512ffd640 1 -- 172.21.15.154:0/2236657558 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f151415ca80 msgr2=0x7f1514160680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:06.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.352+0000 7f1512ffd640 1 --2- 172.21.15.154:0/2236657558 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f151415ca80 0x7f1514160680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:06.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.352+0000 7f1512ffd640 1 -- 172.21.15.154:0/2236657558 >> v2:172.21.15.190:3300/0 conn(0x7f15140b2620 msgr2=0x7f15140d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:06.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.352+0000 7f1512ffd640 1 --2- 172.21.15.154:0/2236657558 >> v2:172.21.15.190:3300/0 conn(0x7f15140b2620 0x7f15140d07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:06.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.352+0000 7f1512ffd640 1 -- 172.21.15.154:0/2236657558 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1514160f40 con 0x7f15140d25b0 2023-09-23T03:17:06.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.352+0000 7f1512ffd640 1 -- 172.21.15.154:0/2236657558 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f150c0019f0 con 0x7f15140d25b0 2023-09-23T03:17:06.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.352+0000 7f1512ffd640 1 -- 172.21.15.154:0/2236657558 >> v1:172.21.15.154:6789/0 conn(0x7f15140d25b0 legacy=0x7f15140d2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:06.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.352+0000 7f1512ffd640 1 --2- 172.21.15.154:0/2236657558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1500003970 0x7f1500003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:06.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.352+0000 7f1512ffd640 1 --2- 172.21.15.154:0/2236657558 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f151415ca80 0x7f15000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:06.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.352+0000 7f1512ffd640 1 --2- 172.21.15.154:0/2236657558 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1500004d00 0x7f1500005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:06.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.352+0000 7f1512ffd640 1 -- 172.21.15.154:0/2236657558 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15140b2aa0 con 0x7f1500004d00 2023-09-23T03:17:06.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.352+0000 7f1512ffd640 1 -- 172.21.15.154:0/2236657558 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1514160c60 con 0x7f1500003970 2023-09-23T03:17:06.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.352+0000 7f1512ffd640 1 -- 172.21.15.154:0/2236657558 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f150000b980 con 0x7f151415ca80 2023-09-23T03:17:06.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.352+0000 7f1513fff640 1 --2- 172.21.15.154:0/2236657558 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1500004d00 0x7f1500005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:06.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.353+0000 7f15137fe640 1 --2- 172.21.15.154:0/2236657558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1500003970 0x7f1500003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:06.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.353+0000 7f150bfff640 1 --2- 172.21.15.154:0/2236657558 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f151415ca80 0x7f15000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:06.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.353+0000 7f1513fff640 1 -- 172.21.15.154:0/2236657558 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f151415ca80 msgr2=0x7f15000047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:06.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.353+0000 7f1513fff640 1 --2- 172.21.15.154:0/2236657558 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f151415ca80 0x7f15000047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:06.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.353+0000 7f1513fff640 1 -- 172.21.15.154:0/2236657558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1500003970 msgr2=0x7f1500003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:06.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.353+0000 7f1513fff640 1 --2- 172.21.15.154:0/2236657558 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1500003970 0x7f1500003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:06.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.353+0000 7f1513fff640 1 -- 172.21.15.154:0/2236657558 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1500003520 con 0x7f1500004d00 2023-09-23T03:17:06.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.354+0000 7f1513fff640 1 --2- 172.21.15.154:0/2236657558 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1500004d00 0x7f1500005180 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f150c00a8d0 tx=0x7f15000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686a6fa3a5e6a31c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:06.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.354+0000 7f1512ffd640 1 -- 172.21.15.154:0/2236657558 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150c034630 con 0x7f1500004d00 2023-09-23T03:17:06.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.354+0000 7f1512ffd640 1 -- 172.21.15.154:0/2236657558 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f150c02f070 con 0x7f1500004d00 2023-09-23T03:17:06.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.354+0000 7f1512ffd640 1 -- 172.21.15.154:0/2236657558 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150c0349d0 con 0x7f1500004d00 2023-09-23T03:17:06.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.355+0000 7f1519b39640 1 -- 172.21.15.154:0/2236657558 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1500004d00 msgr2=0x7f1500005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:06.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.355+0000 7f1519b39640 1 --2- 172.21.15.154:0/2236657558 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1500004d00 0x7f1500005180 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f150c00a8d0 tx=0x7f15000057f0 comp rx=0 tx=0).stop 2023-09-23T03:17:06.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.355+0000 7f1513fff640 1 -- 172.21.15.154:0/2236657558 reap_dead start 2023-09-23T03:17:06.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.355+0000 7f1519b39640 1 -- 172.21.15.154:0/2236657558 shutdown_connections 2023-09-23T03:17:06.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.355+0000 7f1519b39640 1 -- 172.21.15.154:0/2236657558 >> 172.21.15.154:0/2236657558 conn(0x7f151409bc60 msgr2=0x7f1514089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:06.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.355+0000 7f1519b39640 1 -- 172.21.15.154:0/2236657558 shutdown_connections 2023-09-23T03:17:06.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.355+0000 7f1519b39640 1 -- 172.21.15.154:0/2236657558 wait complete. 2023-09-23T03:17:06.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.357+0000 7f1519b39640 1 Processor -- start 2023-09-23T03:17:06.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.357+0000 7f1519b39640 1 -- start start 2023-09-23T03:17:06.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.358+0000 7f1519b39640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1500004d00 0x7f1514159840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:06.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.358+0000 7f1519b39640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15140cc190 0x7f15140cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:06.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.358+0000 7f1519b39640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f151415f3c0 0x7f151406af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:06.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.358+0000 7f1519b39640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15140cc950 con 0x7f151415f3c0 2023-09-23T03:17:06.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.358+0000 7f1519b39640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15141f2020 con 0x7f1500004d00 2023-09-23T03:17:06.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.358+0000 7f1519b39640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f151415ef30 con 0x7f15140cc190 2023-09-23T03:17:06.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.358+0000 7f1513fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f151415f3c0 0x7f151406af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:06.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.358+0000 7f1513fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f151415f3c0 0x7f151406af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47120/0 (socket says 172.21.15.154:47120) 2023-09-23T03:17:06.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.358+0000 7f1513fff640 1 -- 172.21.15.154:0/470114853 learned_addr learned my addr 172.21.15.154:0/470114853 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:06.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.359+0000 7f150bfff640 1 --2- 172.21.15.154:0/470114853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15140cc190 0x7f15140cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:06.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.359+0000 7f15137fe640 1 --2- 172.21.15.154:0/470114853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1500004d00 0x7f1514159840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:06.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.359+0000 7f1513fff640 1 -- 172.21.15.154:0/470114853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15140cc190 msgr2=0x7f15140cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:06.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.359+0000 7f1513fff640 1 --2- 172.21.15.154:0/470114853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15140cc190 0x7f15140cb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:06.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.359+0000 7f1513fff640 1 -- 172.21.15.154:0/470114853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1500004d00 msgr2=0x7f1514159840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:06.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.359+0000 7f1513fff640 1 --2- 172.21.15.154:0/470114853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1500004d00 0x7f1514159840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:06.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.359+0000 7f1513fff640 1 -- 172.21.15.154:0/470114853 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1514159d80 con 0x7f151415f3c0 2023-09-23T03:17:06.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.359+0000 7f150bfff640 1 --2- 172.21.15.154:0/470114853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15140cc190 0x7f15140cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:06.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.359+0000 7f15137fe640 1 --2- 172.21.15.154:0/470114853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1500004d00 0x7f1514159840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:06.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.359+0000 7f1513fff640 1 --2- 172.21.15.154:0/470114853 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f151415f3c0 0x7f151406af50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f150c001c20 tx=0x7f150c001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:06.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.360+0000 7f1510ff9640 1 -- 172.21.15.154:0/470114853 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150c034840 con 0x7f151415f3c0 2023-09-23T03:17:06.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.360+0000 7f1510ff9640 1 -- 172.21.15.154:0/470114853 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f150c0349c0 con 0x7f151415f3c0 2023-09-23T03:17:06.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.360+0000 7f1510ff9640 1 -- 172.21.15.154:0/470114853 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150c034d70 con 0x7f151415f3c0 2023-09-23T03:17:06.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.360+0000 7f1519b39640 1 -- 172.21.15.154:0/470114853 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15140cc670 con 0x7f151415f3c0 2023-09-23T03:17:06.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.360+0000 7f1519b39640 1 -- 172.21.15.154:0/470114853 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f151415ebc0 con 0x7f151415f3c0 2023-09-23T03:17:06.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.361+0000 7f1519b39640 1 -- 172.21.15.154:0/470114853 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f151415e740 con 0x7f151415f3c0 2023-09-23T03:17:06.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.362+0000 7f1510ff9640 1 -- 172.21.15.154:0/470114853 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f150c031030 con 0x7f151415f3c0 2023-09-23T03:17:06.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.363+0000 7f1510ff9640 1 --2- 172.21.15.154:0/470114853 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f14f406af60 0x7f14f406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:06.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.363+0000 7f1510ff9640 1 -- 172.21.15.154:0/470114853 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f150c0ab8f0 con 0x7f151415f3c0 2023-09-23T03:17:06.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.363+0000 7f15137fe640 1 --2- 172.21.15.154:0/470114853 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f14f406af60 0x7f14f406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:06.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.364+0000 7f15137fe640 1 --2- 172.21.15.154:0/470114853 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f14f406af60 0x7f14f406d420 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f15040099f0 tx=0x7f1504008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:06.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.368+0000 7f1510ff9640 1 -- 172.21.15.154:0/470114853 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f151415e740 con 0x7f151415f3c0 2023-09-23T03:17:06.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.624+0000 7f1519b39640 1 -- 172.21.15.154:0/470114853 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f151415e740 con 0x7f151415f3c0 2023-09-23T03:17:06.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.629+0000 7f1510ff9640 1 -- 172.21.15.154:0/470114853 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v8) v1 ==== 97+0+0 (secure 0 0 0) 0x7f151415e740 con 0x7f151415f3c0 2023-09-23T03:17:06.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.631+0000 7f1519b39640 1 -- 172.21.15.154:0/470114853 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f14f406af60 msgr2=0x7f14f406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:06.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.631+0000 7f1519b39640 1 --2- 172.21.15.154:0/470114853 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f14f406af60 0x7f14f406d420 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f15040099f0 tx=0x7f1504008040 comp rx=0 tx=0).stop 2023-09-23T03:17:06.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.631+0000 7f1519b39640 1 -- 172.21.15.154:0/470114853 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f151415f3c0 msgr2=0x7f151406af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:06.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.631+0000 7f1519b39640 1 --2- 172.21.15.154:0/470114853 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f151415f3c0 0x7f151406af50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f150c001c20 tx=0x7f150c001360 comp rx=0 tx=0).stop 2023-09-23T03:17:06.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.631+0000 7f1519b39640 1 -- 172.21.15.154:0/470114853 shutdown_connections 2023-09-23T03:17:06.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.631+0000 7f1519b39640 1 --2- 172.21.15.154:0/470114853 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f14f406af60 0x7f14f406d420 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:06.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.631+0000 7f1519b39640 1 --2- 172.21.15.154:0/470114853 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f151415f3c0 0x7f151406af50 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:06.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.631+0000 7f1519b39640 1 --2- 172.21.15.154:0/470114853 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15140cc190 0x7f15140cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:06.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.631+0000 7f1519b39640 1 --2- 172.21.15.154:0/470114853 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1500004d00 0x7f1514159840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:06.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.631+0000 7f1519b39640 1 -- 172.21.15.154:0/470114853 >> 172.21.15.154:0/470114853 conn(0x7f151409bc60 msgr2=0x7f15140b3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:06.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.632+0000 7f1519b39640 1 -- 172.21.15.154:0/470114853 shutdown_connections 2023-09-23T03:17:06.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.632+0000 7f1519b39640 1 -- 172.21.15.154:0/470114853 wait complete. 2023-09-23T03:17:06.651 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-09-23T03:17:06.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.860+0000 7fe10d34c640 1 Processor -- start 2023-09-23T03:17:06.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.860+0000 7fe10d34c640 1 -- start start 2023-09-23T03:17:06.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.861+0000 7fe10d34c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1080cf130 0x7fe1080ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:06.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.861+0000 7fe10d34c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe1080b2fe0 0x7fe10815d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:06.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.861+0000 7fe10d34c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe10806ceb0 con 0x7fe1080ada30 2023-09-23T03:17:06.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.861+0000 7fe10d34c640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1080ade10 con 0x7fe1080b2fe0 2023-09-23T03:17:06.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.861+0000 7fe10d34c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe108159c90 con 0x7fe1080cf130 2023-09-23T03:17:06.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.862+0000 7fe106ffd640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fe1080ada30 0x7fe1080cea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:42258/0 (socket says 172.21.15.154:42258) 2023-09-23T03:17:06.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.862+0000 7fe106ffd640 1 -- 172.21.15.154:0/3465727369 learned_addr learned my addr 172.21.15.154:0/3465727369 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:17:06.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.862+0000 7fe1077fe640 1 --2- 172.21.15.154:0/3465727369 >> v2:172.21.15.190:3300/0 conn(0x7fe1080b2fe0 0x7fe10815d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:06.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.862+0000 7fe1067fc640 1 --2- 172.21.15.154:0/3465727369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1080cf130 0x7fe1080ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:06.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.862+0000 7fe105ffb640 1 -- 172.21.15.154:0/3465727369 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 429596626 0 0) 0x7fe0f8001290 con 0x7fe1080ada30 2023-09-23T03:17:06.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.862+0000 7fe105ffb640 1 -- 172.21.15.154:0/3465727369 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0f0003520 con 0x7fe1080ada30 2023-09-23T03:17:06.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.862+0000 7fe105ffb640 1 -- 172.21.15.154:0/3465727369 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3512591877 0 0) 0x7fe0f0003520 con 0x7fe1080ada30 2023-09-23T03:17:06.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.862+0000 7fe105ffb640 1 -- 172.21.15.154:0/3465727369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1080cf130 msgr2=0x7fe1080ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:06.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.862+0000 7fe105ffb640 1 --2- 172.21.15.154:0/3465727369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1080cf130 0x7fe1080ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:06.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.862+0000 7fe105ffb640 1 -- 172.21.15.154:0/3465727369 >> v2:172.21.15.190:3300/0 conn(0x7fe1080b2fe0 msgr2=0x7fe10815d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:06.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.862+0000 7fe105ffb640 1 --2- 172.21.15.154:0/3465727369 >> v2:172.21.15.190:3300/0 conn(0x7fe1080b2fe0 0x7fe10815d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:06.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.862+0000 7fe105ffb640 1 -- 172.21.15.154:0/3465727369 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10806c110 con 0x7fe1080ada30 2023-09-23T03:17:06.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.863+0000 7fe105ffb640 1 -- 172.21.15.154:0/3465727369 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fe0f8001980 con 0x7fe1080ada30 2023-09-23T03:17:06.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.863+0000 7fe105ffb640 1 -- 172.21.15.154:0/3465727369 >> v1:172.21.15.154:6789/0 conn(0x7fe1080ada30 legacy=0x7fe1080cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:06.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.863+0000 7fe105ffb640 1 --2- 172.21.15.154:0/3465727369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0f0003970 0x7fe0f0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:06.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.863+0000 7fe105ffb640 1 --2- 172.21.15.154:0/3465727369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0f0004360 0x7fe0f0004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:06.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.863+0000 7fe105ffb640 1 --2- 172.21.15.154:0/3465727369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0f0004f60 0x7fe0f000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:06.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.863+0000 7fe105ffb640 1 -- 172.21.15.154:0/3465727369 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1080ade10 con 0x7fe0f0004f60 2023-09-23T03:17:06.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.863+0000 7fe105ffb640 1 -- 172.21.15.154:0/3465727369 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe108159c90 con 0x7fe0f0004360 2023-09-23T03:17:06.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.863+0000 7fe105ffb640 1 -- 172.21.15.154:0/3465727369 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f000e100 con 0x7fe0f0003970 2023-09-23T03:17:06.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.863+0000 7fe1067fc640 1 --2- 172.21.15.154:0/3465727369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe1080cf130 0x7fe1080ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:17:06.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.863+0000 7fe1077fe640 1 --2- 172.21.15.154:0/3465727369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0f0004f60 0x7fe0f000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:06.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.863+0000 7fe1077fe640 1 -- 172.21.15.154:0/3465727369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0f0003970 msgr2=0x7fe0f0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:06.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.863+0000 7fe1077fe640 1 --2- 172.21.15.154:0/3465727369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0f0003970 0x7fe0f0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:06.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.863+0000 7fe1077fe640 1 -- 172.21.15.154:0/3465727369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0f0004360 msgr2=0x7fe0f0004840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:06.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.863+0000 7fe1077fe640 1 --2- 172.21.15.154:0/3465727369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe0f0004360 0x7fe0f0004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:06.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.863+0000 7fe1077fe640 1 -- 172.21.15.154:0/3465727369 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f0003520 con 0x7fe0f0004f60 2023-09-23T03:17:06.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.865+0000 7fe1077fe640 1 --2- 172.21.15.154:0/3465727369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0f0004f60 0x7fe0f000b980 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe100009f90 tx=0x7fe0f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e3f777458b1bf5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:06.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.865+0000 7fe105ffb640 1 -- 172.21.15.154:0/3465727369 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10002cc20 con 0x7fe0f0004f60 2023-09-23T03:17:06.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.865+0000 7fe105ffb640 1 -- 172.21.15.154:0/3465727369 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe10000f040 con 0x7fe0f0004f60 2023-09-23T03:17:06.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.865+0000 7fe105ffb640 1 -- 172.21.15.154:0/3465727369 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe100034440 con 0x7fe0f0004f60 2023-09-23T03:17:06.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.865+0000 7fe10d34c640 1 -- 172.21.15.154:0/3465727369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0f0004f60 msgr2=0x7fe0f000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:06.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.865+0000 7fe10d34c640 1 --2- 172.21.15.154:0/3465727369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe0f0004f60 0x7fe0f000b980 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe100009f90 tx=0x7fe0f00057f0 comp rx=0 tx=0).stop 2023-09-23T03:17:06.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.865+0000 7fe1077fe640 1 -- 172.21.15.154:0/3465727369 reap_dead start 2023-09-23T03:17:06.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.865+0000 7fe10d34c640 1 -- 172.21.15.154:0/3465727369 shutdown_connections 2023-09-23T03:17:06.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.865+0000 7fe10d34c640 1 -- 172.21.15.154:0/3465727369 >> 172.21.15.154:0/3465727369 conn(0x7fe10806a0f0 msgr2=0x7fe10806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:06.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.866+0000 7fe10d34c640 1 -- 172.21.15.154:0/3465727369 shutdown_connections 2023-09-23T03:17:06.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.866+0000 7fe10d34c640 1 -- 172.21.15.154:0/3465727369 wait complete. 2023-09-23T03:17:06.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.867+0000 7fe10d34c640 1 Processor -- start 2023-09-23T03:17:06.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.867+0000 7fe10d34c640 1 -- start start 2023-09-23T03:17:06.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.868+0000 7fe10d34c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0f0004f60 0x7fe1081f4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:06.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.868+0000 7fe10d34c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe10815a850 0x7fe10815acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:06.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.868+0000 7fe10d34c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1080cbb10 0x7fe1080cbf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:06.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.868+0000 7fe10d34c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1080cc4d0 con 0x7fe10815a850 2023-09-23T03:17:06.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.868+0000 7fe10d34c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe108153d50 con 0x7fe1080cbb10 2023-09-23T03:17:06.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.868+0000 7fe10d34c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1080c2c60 con 0x7fe0f0004f60 2023-09-23T03:17:06.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.869+0000 7fe1067fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe10815a850 0x7fe10815acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:06.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.869+0000 7fe1067fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe10815a850 0x7fe10815acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47144/0 (socket says 172.21.15.154:47144) 2023-09-23T03:17:06.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.869+0000 7fe1067fc640 1 -- 172.21.15.154:0/4207586149 learned_addr learned my addr 172.21.15.154:0/4207586149 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:06.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.869+0000 7fe106ffd640 1 --2- 172.21.15.154:0/4207586149 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0f0004f60 0x7fe1081f4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:06.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.869+0000 7fe1067fc640 1 -- 172.21.15.154:0/4207586149 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0f0004f60 msgr2=0x7fe1081f4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:06.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.869+0000 7fe1067fc640 1 --2- 172.21.15.154:0/4207586149 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0f0004f60 0x7fe1081f4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:06.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.869+0000 7fe1067fc640 1 -- 172.21.15.154:0/4207586149 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1080cbb10 msgr2=0x7fe1080cbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:06.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.869+0000 7fe1067fc640 1 --2- 172.21.15.154:0/4207586149 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1080cbb10 0x7fe1080cbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:06.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.869+0000 7fe1067fc640 1 -- 172.21.15.154:0/4207586149 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1080c2f40 con 0x7fe10815a850 2023-09-23T03:17:06.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.869+0000 7fe1067fc640 1 --2- 172.21.15.154:0/4207586149 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe10815a850 0x7fe10815acd0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc00e560 tx=0x7fe0fc00ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:06.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.870+0000 7fe0ef7fe640 1 -- 172.21.15.154:0/4207586149 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc004230 con 0x7fe10815a850 2023-09-23T03:17:06.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.870+0000 7fe0ef7fe640 1 -- 172.21.15.154:0/4207586149 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0fc0043b0 con 0x7fe10815a850 2023-09-23T03:17:06.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.870+0000 7fe10d34c640 1 -- 172.21.15.154:0/4207586149 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1080c3220 con 0x7fe10815a850 2023-09-23T03:17:06.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.870+0000 7fe0ef7fe640 1 -- 172.21.15.154:0/4207586149 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc004520 con 0x7fe10815a850 2023-09-23T03:17:06.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.870+0000 7fe10d34c640 1 -- 172.21.15.154:0/4207586149 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1080c37e0 con 0x7fe10815a850 2023-09-23T03:17:06.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.871+0000 7fe10d34c640 1 -- 172.21.15.154:0/4207586149 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe108073eb0 con 0x7fe10815a850 2023-09-23T03:17:06.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.872+0000 7fe0ef7fe640 1 -- 172.21.15.154:0/4207586149 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fe0fc003850 con 0x7fe10815a850 2023-09-23T03:17:06.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.873+0000 7fe0ef7fe640 1 --2- 172.21.15.154:0/4207586149 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe0f406af60 0x7fe0f406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:06.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.873+0000 7fe0ef7fe640 1 -- 172.21.15.154:0/4207586149 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fe0fc088fe0 con 0x7fe10815a850 2023-09-23T03:17:06.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.873+0000 7fe106ffd640 1 --2- 172.21.15.154:0/4207586149 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe0f406af60 0x7fe0f406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:06.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.873+0000 7fe106ffd640 1 --2- 172.21.15.154:0/4207586149 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe0f406af60 0x7fe0f406d420 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8001ab0 tx=0x7fe0f8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:06.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:06.877+0000 7fe0ef7fe640 1 -- 172.21.15.154:0/4207586149 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fe0fc054280 con 0x7fe10815a850 2023-09-23T03:17:07.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.135+0000 7fe10d34c640 1 -- 172.21.15.154:0/4207586149 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fe1080cc610 con 0x7fe10815a850 2023-09-23T03:17:07.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.138+0000 7fe0ef7fe640 1 -- 172.21.15.154:0/4207586149 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe1080cc610 con 0x7fe10815a850 2023-09-23T03:17:07.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.141+0000 7fe10d34c640 1 -- 172.21.15.154:0/4207586149 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe0f406af60 msgr2=0x7fe0f406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:07.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.141+0000 7fe10d34c640 1 --2- 172.21.15.154:0/4207586149 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe0f406af60 0x7fe0f406d420 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8001ab0 tx=0x7fe0f8009040 comp rx=0 tx=0).stop 2023-09-23T03:17:07.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.141+0000 7fe10d34c640 1 -- 172.21.15.154:0/4207586149 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe10815a850 msgr2=0x7fe10815acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:07.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.141+0000 7fe10d34c640 1 --2- 172.21.15.154:0/4207586149 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe10815a850 0x7fe10815acd0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc00e560 tx=0x7fe0fc00ea20 comp rx=0 tx=0).stop 2023-09-23T03:17:07.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.142+0000 7fe10d34c640 1 -- 172.21.15.154:0/4207586149 shutdown_connections 2023-09-23T03:17:07.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.142+0000 7fe10d34c640 1 --2- 172.21.15.154:0/4207586149 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe0f406af60 0x7fe0f406d420 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:07.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.142+0000 7fe10d34c640 1 --2- 172.21.15.154:0/4207586149 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe1080cbb10 0x7fe1080cbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:07.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.142+0000 7fe10d34c640 1 --2- 172.21.15.154:0/4207586149 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe10815a850 0x7fe10815acd0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:07.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.142+0000 7fe10d34c640 1 --2- 172.21.15.154:0/4207586149 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe0f0004f60 0x7fe1081f4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:07.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.142+0000 7fe10d34c640 1 -- 172.21.15.154:0/4207586149 >> 172.21.15.154:0/4207586149 conn(0x7fe10806a0f0 msgr2=0x7fe1080cae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:07.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.142+0000 7fe10d34c640 1 -- 172.21.15.154:0/4207586149 shutdown_connections 2023-09-23T03:17:07.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.142+0000 7fe10d34c640 1 -- 172.21.15.154:0/4207586149 wait complete. 2023-09-23T03:17:07.162 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:17:07.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.371+0000 7f177659e640 1 Processor -- start 2023-09-23T03:17:07.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.371+0000 7f177659e640 1 -- start start 2023-09-23T03:17:07.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.373+0000 7f177659e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f177006c7a0 0x7f177015eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:07.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.373+0000 7f177659e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f177015ab00 0x7f177015afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:07.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.373+0000 7f177659e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f177006fdc0 0x7f17700701f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:07.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.373+0000 7f177659e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1770155070 con 0x7f177006c7a0 2023-09-23T03:17:07.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.373+0000 7f177659e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f177015e0f0 con 0x7f177015ab00 2023-09-23T03:17:07.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.373+0000 7f177659e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f177015f040 con 0x7f177006fdc0 2023-09-23T03:17:07.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.373+0000 7f177559c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f177006c7a0 0x7f177015eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:07.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.373+0000 7f177559c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f177006c7a0 0x7f177015eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47156/0 (socket says 172.21.15.154:47156) 2023-09-23T03:17:07.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.373+0000 7f177559c640 1 -- 172.21.15.154:0/1495573929 learned_addr learned my addr 172.21.15.154:0/1495573929 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:07.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.374+0000 7f1775d9d640 1 --2- 172.21.15.154:0/1495573929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f177006fdc0 0x7f17700701f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:07.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.374+0000 7f1774d9b640 1 --2- 172.21.15.154:0/1495573929 >> v2:172.21.15.190:3300/0 conn(0x7f177015ab00 0x7f177015afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:07.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.374+0000 7f177559c640 1 -- 172.21.15.154:0/1495573929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f177006fdc0 msgr2=0x7f17700701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:07.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.374+0000 7f177559c640 1 --2- 172.21.15.154:0/1495573929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f177006fdc0 0x7f17700701f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:07.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.374+0000 7f177559c640 1 -- 172.21.15.154:0/1495573929 >> v2:172.21.15.190:3300/0 conn(0x7f177015ab00 msgr2=0x7f177015afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:07.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.374+0000 7f177559c640 1 --2- 172.21.15.154:0/1495573929 >> v2:172.21.15.190:3300/0 conn(0x7f177015ab00 0x7f177015afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:07.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.374+0000 7f177559c640 1 -- 172.21.15.154:0/1495573929 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1770070730 con 0x7f177006c7a0 2023-09-23T03:17:07.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.375+0000 7f177559c640 1 --2- 172.21.15.154:0/1495573929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f177006c7a0 0x7f177015eb00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f17700708c0 tx=0x7f176c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa93a3cb7b824e7d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:07.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.375+0000 7f1767fff640 1 -- 172.21.15.154:0/1495573929 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176c031640 con 0x7f177006c7a0 2023-09-23T03:17:07.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.375+0000 7f1767fff640 1 -- 172.21.15.154:0/1495573929 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f176c0317a0 con 0x7f177006c7a0 2023-09-23T03:17:07.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.375+0000 7f1767fff640 1 -- 172.21.15.154:0/1495573929 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176c031b30 con 0x7f177006c7a0 2023-09-23T03:17:07.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.376+0000 7f177659e640 1 -- 172.21.15.154:0/1495573929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f177006c7a0 msgr2=0x7f177015eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:07.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.376+0000 7f177659e640 1 --2- 172.21.15.154:0/1495573929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f177006c7a0 0x7f177015eb00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f17700708c0 tx=0x7f176c02b8f0 comp rx=0 tx=0).stop 2023-09-23T03:17:07.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.376+0000 7f177659e640 1 -- 172.21.15.154:0/1495573929 shutdown_connections 2023-09-23T03:17:07.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.376+0000 7f177659e640 1 --2- 172.21.15.154:0/1495573929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f177006fdc0 0x7f17700701f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:07.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.376+0000 7f177659e640 1 --2- 172.21.15.154:0/1495573929 >> v2:172.21.15.190:3300/0 conn(0x7f177015ab00 0x7f177015afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:07.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.376+0000 7f177659e640 1 --2- 172.21.15.154:0/1495573929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f177006c7a0 0x7f177015eb00 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:07.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.376+0000 7f177659e640 1 -- 172.21.15.154:0/1495573929 >> 172.21.15.154:0/1495573929 conn(0x7f177006a390 msgr2=0x7f17700afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:07.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.376+0000 7f177659e640 1 -- 172.21.15.154:0/1495573929 shutdown_connections 2023-09-23T03:17:07.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.376+0000 7f177659e640 1 -- 172.21.15.154:0/1495573929 wait complete. 2023-09-23T03:17:07.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.378+0000 7f177659e640 1 Processor -- start 2023-09-23T03:17:07.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.378+0000 7f177659e640 1 -- start start 2023-09-23T03:17:07.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.379+0000 7f177659e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f177006c7a0 0x7f17701ee8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:07.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.379+0000 7f177659e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f177006fdc0 0x7f17701eef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:07.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.379+0000 7f177659e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f177015ab00 0x7f17700ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:07.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.379+0000 7f177659e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17701540f0 con 0x7f177015ab00 2023-09-23T03:17:07.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.379+0000 7f177659e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1770155ef0 con 0x7f177006c7a0 2023-09-23T03:17:07.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.379+0000 7f177659e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17700c6280 con 0x7f177006fdc0 2023-09-23T03:17:07.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.379+0000 7f1775d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f177015ab00 0x7f17700ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:07.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.379+0000 7f1775d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f177015ab00 0x7f17700ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47158/0 (socket says 172.21.15.154:47158) 2023-09-23T03:17:07.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.379+0000 7f1775d9d640 1 -- 172.21.15.154:0/4246275488 learned_addr learned my addr 172.21.15.154:0/4246275488 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:07.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.379+0000 7f177559c640 1 --2- 172.21.15.154:0/4246275488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f177006c7a0 0x7f17701ee8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:07.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.379+0000 7f1774d9b640 1 --2- 172.21.15.154:0/4246275488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f177006fdc0 0x7f17701eef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:07.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.380+0000 7f1775d9d640 1 -- 172.21.15.154:0/4246275488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f177006fdc0 msgr2=0x7f17701eef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:07.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.380+0000 7f1775d9d640 1 --2- 172.21.15.154:0/4246275488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f177006fdc0 0x7f17701eef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:07.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.380+0000 7f1775d9d640 1 -- 172.21.15.154:0/4246275488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f177006c7a0 msgr2=0x7f17701ee8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:07.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.380+0000 7f1775d9d640 1 --2- 172.21.15.154:0/4246275488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f177006c7a0 0x7f17701ee8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:07.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.380+0000 7f1775d9d640 1 -- 172.21.15.154:0/4246275488 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f176c0094d0 con 0x7f177015ab00 2023-09-23T03:17:07.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.380+0000 7f1775d9d640 1 --2- 172.21.15.154:0/4246275488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f177015ab00 0x7f17700ae8e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f176800cda0 tx=0x7f176800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:07.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.380+0000 7f1765ffb640 1 -- 172.21.15.154:0/4246275488 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1768003b90 con 0x7f177015ab00 2023-09-23T03:17:07.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.380+0000 7f1765ffb640 1 -- 172.21.15.154:0/4246275488 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1768003cf0 con 0x7f177015ab00 2023-09-23T03:17:07.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.380+0000 7f1765ffb640 1 -- 172.21.15.154:0/4246275488 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1768004240 con 0x7f177015ab00 2023-09-23T03:17:07.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.380+0000 7f177659e640 1 -- 172.21.15.154:0/4246275488 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17700c64d0 con 0x7f177015ab00 2023-09-23T03:17:07.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.380+0000 7f177659e640 1 -- 172.21.15.154:0/4246275488 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17700c6a90 con 0x7f177015ab00 2023-09-23T03:17:07.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.381+0000 7f177659e640 1 -- 172.21.15.154:0/4246275488 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17700736b0 con 0x7f177015ab00 2023-09-23T03:17:07.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.382+0000 7f1765ffb640 1 -- 172.21.15.154:0/4246275488 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f17680043a0 con 0x7f177015ab00 2023-09-23T03:17:07.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.384+0000 7f1765ffb640 1 --2- 172.21.15.154:0/4246275488 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f175406af30 0x7f175406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:07.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.384+0000 7f1765ffb640 1 -- 172.21.15.154:0/4246275488 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f1768012070 con 0x7f177015ab00 2023-09-23T03:17:07.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.384+0000 7f177559c640 1 --2- 172.21.15.154:0/4246275488 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f175406af30 0x7f175406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:07.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.385+0000 7f177559c640 1 --2- 172.21.15.154:0/4246275488 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f175406af30 0x7f175406d3f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f176c036000 tx=0x7f176c03c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:07.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.388+0000 7f1765ffb640 1 -- 172.21.15.154:0/4246275488 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f1768055190 con 0x7f177015ab00 2023-09-23T03:17:07.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.565+0000 7f177659e640 1 -- 172.21.15.154:0/4246275488 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f177004e920 con 0x7f177015ab00 2023-09-23T03:17:07.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.567+0000 7f1765ffb640 1 -- 172.21.15.154:0/4246275488 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+13098 (secure 0 0 0) 0x7f17680587b0 con 0x7f177015ab00 2023-09-23T03:17:07.569 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:07.569 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":34,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:17:03.616664+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:17:07.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.570+0000 7f177659e640 1 -- 172.21.15.154:0/4246275488 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f175406af30 msgr2=0x7f175406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:07.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.570+0000 7f177659e640 1 --2- 172.21.15.154:0/4246275488 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f175406af30 0x7f175406d3f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f176c036000 tx=0x7f176c03c040 comp rx=0 tx=0).stop 2023-09-23T03:17:07.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.570+0000 7f177659e640 1 -- 172.21.15.154:0/4246275488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f177015ab00 msgr2=0x7f17700ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:07.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.570+0000 7f177659e640 1 --2- 172.21.15.154:0/4246275488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f177015ab00 0x7f17700ae8e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f176800cda0 tx=0x7f176800a660 comp rx=0 tx=0).stop 2023-09-23T03:17:07.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.570+0000 7f177659e640 1 -- 172.21.15.154:0/4246275488 shutdown_connections 2023-09-23T03:17:07.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.570+0000 7f177659e640 1 --2- 172.21.15.154:0/4246275488 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f175406af30 0x7f175406d3f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:07.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.570+0000 7f177659e640 1 --2- 172.21.15.154:0/4246275488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f177015ab00 0x7f17700ae8e0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:07.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.570+0000 7f177659e640 1 --2- 172.21.15.154:0/4246275488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f177006fdc0 0x7f17701eef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:07.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.570+0000 7f177659e640 1 --2- 172.21.15.154:0/4246275488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f177006c7a0 0x7f17701ee8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:07.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.570+0000 7f177659e640 1 -- 172.21.15.154:0/4246275488 >> 172.21.15.154:0/4246275488 conn(0x7f177006a390 msgr2=0x7f17700af5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:07.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.570+0000 7f177659e640 1 -- 172.21.15.154:0/4246275488 shutdown_connections 2023-09-23T03:17:07.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.570+0000 7f177659e640 1 -- 172.21.15.154:0/4246275488 wait complete. 2023-09-23T03:17:07.589 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T03:17:07.590 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:17:07.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.795+0000 7f1b96aa3640 1 Processor -- start 2023-09-23T03:17:07.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.795+0000 7f1b96aa3640 1 -- start start 2023-09-23T03:17:07.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.797+0000 7f1b96aa3640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1b901603c0 0x7f1b900d0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:07.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.797+0000 7f1b96aa3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b900cd270 0x7f1b900cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:07.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.797+0000 7f1b96aa3640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b900d0870 con 0x7f1b900cf3d0 2023-09-23T03:17:07.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.797+0000 7f1b96aa3640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b900d0aa0 con 0x7f1b901603c0 2023-09-23T03:17:07.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.797+0000 7f1b96aa3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b900d3010 con 0x7f1b900cd270 2023-09-23T03:17:07.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.797+0000 7f1b95aa1640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1b901603c0 0x7f1b900d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:07.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.797+0000 7f1b95aa1640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1b901603c0 0x7f1b900d0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47174/0 (socket says 172.21.15.154:47174) 2023-09-23T03:17:07.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.797+0000 7f1b95aa1640 1 -- 172.21.15.154:0/3755130154 learned_addr learned my addr 172.21.15.154:0/3755130154 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:07.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.797+0000 7f1b952a0640 1 --2- 172.21.15.154:0/3755130154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b900cd270 0x7f1b900cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:07.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.798+0000 7f1b95aa1640 1 -- 172.21.15.154:0/3755130154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b900cd270 msgr2=0x7f1b900cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:07.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.798+0000 7f1b95aa1640 1 --2- 172.21.15.154:0/3755130154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b900cd270 0x7f1b900cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:07.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.798+0000 7f1b95aa1640 1 -- 172.21.15.154:0/3755130154 >> v1:172.21.15.190:6789/0 conn(0x7f1b900cf3d0 legacy=0x7f1b901607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:07.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.798+0000 7f1b95aa1640 1 -- 172.21.15.154:0/3755130154 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b900d32f0 con 0x7f1b901603c0 2023-09-23T03:17:07.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.799+0000 7f1b95aa1640 1 --2- 172.21.15.154:0/3755130154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b901603c0 0x7f1b900d0330 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c009850 tx=0x7f1b8c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e97ff2e9784d75a5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:07.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.800+0000 7f1b94a9f640 1 -- 172.21.15.154:0/3755130154 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c004230 con 0x7f1b901603c0 2023-09-23T03:17:07.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.800+0000 7f1b94a9f640 1 -- 172.21.15.154:0/3755130154 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8c004390 con 0x7f1b901603c0 2023-09-23T03:17:07.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.800+0000 7f1b94a9f640 1 -- 172.21.15.154:0/3755130154 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c031830 con 0x7f1b901603c0 2023-09-23T03:17:07.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.800+0000 7f1b96aa3640 1 -- 172.21.15.154:0/3755130154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b901603c0 msgr2=0x7f1b900d0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:07.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.800+0000 7f1b96aa3640 1 --2- 172.21.15.154:0/3755130154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b901603c0 0x7f1b900d0330 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c009850 tx=0x7f1b8c02b8d0 comp rx=0 tx=0).stop 2023-09-23T03:17:07.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.800+0000 7f1b96aa3640 1 -- 172.21.15.154:0/3755130154 shutdown_connections 2023-09-23T03:17:07.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.800+0000 7f1b96aa3640 1 --2- 172.21.15.154:0/3755130154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b900cd270 0x7f1b900cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:07.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.800+0000 7f1b96aa3640 1 --2- 172.21.15.154:0/3755130154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b901603c0 0x7f1b900d0330 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:07.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.800+0000 7f1b96aa3640 1 -- 172.21.15.154:0/3755130154 >> 172.21.15.154:0/3755130154 conn(0x7f1b9009ba90 msgr2=0x7f1b900d2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:07.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.800+0000 7f1b96aa3640 1 -- 172.21.15.154:0/3755130154 shutdown_connections 2023-09-23T03:17:07.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.801+0000 7f1b96aa3640 1 -- 172.21.15.154:0/3755130154 wait complete. 2023-09-23T03:17:07.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.802+0000 7f1b96aa3640 1 Processor -- start 2023-09-23T03:17:07.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.802+0000 7f1b96aa3640 1 -- start start 2023-09-23T03:17:07.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.803+0000 7f1b96aa3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b900cd270 0x7f1b90157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:07.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.803+0000 7f1b96aa3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b900cf3d0 0x7f1b9015b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:07.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.803+0000 7f1b96aa3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b901603c0 0x7f1b9015bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:07.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.803+0000 7f1b96aa3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b900caa20 con 0x7f1b900cd270 2023-09-23T03:17:07.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.803+0000 7f1b96aa3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b900ca740 con 0x7f1b900cf3d0 2023-09-23T03:17:07.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.803+0000 7f1b96aa3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b900c7080 con 0x7f1b901603c0 2023-09-23T03:17:07.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.804+0000 7f1b95aa1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b900cd270 0x7f1b90157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:07.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.804+0000 7f1b95aa1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b900cd270 0x7f1b90157d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47176/0 (socket says 172.21.15.154:47176) 2023-09-23T03:17:07.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.804+0000 7f1b95aa1640 1 -- 172.21.15.154:0/3501770117 learned_addr learned my addr 172.21.15.154:0/3501770117 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:07.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.804+0000 7f1b95aa1640 1 -- 172.21.15.154:0/3501770117 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b901603c0 msgr2=0x7f1b9015bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:07.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.804+0000 7f1b95aa1640 1 --2- 172.21.15.154:0/3501770117 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b901603c0 0x7f1b9015bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:07.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.804+0000 7f1b95aa1640 1 -- 172.21.15.154:0/3501770117 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b900cf3d0 msgr2=0x7f1b9015b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:07.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.804+0000 7f1b95aa1640 1 --2- 172.21.15.154:0/3501770117 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b900cf3d0 0x7f1b9015b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:07.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.804+0000 7f1b95aa1640 1 -- 172.21.15.154:0/3501770117 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b8c009460 con 0x7f1b900cd270 2023-09-23T03:17:07.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.804+0000 7f1b95aa1640 1 --2- 172.21.15.154:0/3501770117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b900cd270 0x7f1b90157d30 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c02b910 tx=0x7f1b8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:07.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.805+0000 7f1b867fc640 1 -- 172.21.15.154:0/3501770117 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c031830 con 0x7f1b900cd270 2023-09-23T03:17:07.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.805+0000 7f1b867fc640 1 -- 172.21.15.154:0/3501770117 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8c031990 con 0x7f1b900cd270 2023-09-23T03:17:07.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.805+0000 7f1b867fc640 1 -- 172.21.15.154:0/3501770117 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c031d20 con 0x7f1b900cd270 2023-09-23T03:17:07.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.805+0000 7f1b96aa3640 1 -- 172.21.15.154:0/3501770117 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b900c7360 con 0x7f1b900cd270 2023-09-23T03:17:07.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.805+0000 7f1b96aa3640 1 -- 172.21.15.154:0/3501770117 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b901f2800 con 0x7f1b900cd270 2023-09-23T03:17:07.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.806+0000 7f1b96aa3640 1 -- 172.21.15.154:0/3501770117 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b900c6ee0 con 0x7f1b900cd270 2023-09-23T03:17:07.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.807+0000 7f1b867fc640 1 -- 172.21.15.154:0/3501770117 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f1b8c003610 con 0x7f1b900cd270 2023-09-23T03:17:07.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.808+0000 7f1b867fc640 1 --2- 172.21.15.154:0/3501770117 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1b7406af30 0x7f1b7406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:07.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.808+0000 7f1b867fc640 1 -- 172.21.15.154:0/3501770117 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f1b8c0ab9f0 con 0x7f1b900cd270 2023-09-23T03:17:07.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.808+0000 7f1b952a0640 1 --2- 172.21.15.154:0/3501770117 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1b7406af30 0x7f1b7406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:07.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.809+0000 7f1b952a0640 1 --2- 172.21.15.154:0/3501770117 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1b7406af30 0x7f1b7406d3f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1b901f5cd0 tx=0x7f1b80008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:07.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:07.812+0000 7f1b867fc640 1 -- 172.21.15.154:0/3501770117 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f1b900c6ee0 con 0x7f1b900cd270 2023-09-23T03:17:08.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.001+0000 7f1b96aa3640 1 -- 172.21.15.154:0/3501770117 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1b900c7880 con 0x7f1b900cd270 2023-09-23T03:17:08.003 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.002+0000 7f1b867fc640 1 -- 172.21.15.154:0/3501770117 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1b900c7880 con 0x7f1b900cd270 2023-09-23T03:17:08.003 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:17:08.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.004+0000 7f1b96aa3640 1 -- 172.21.15.154:0/3501770117 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1b7406af30 msgr2=0x7f1b7406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:08.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.004+0000 7f1b96aa3640 1 --2- 172.21.15.154:0/3501770117 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1b7406af30 0x7f1b7406d3f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1b901f5cd0 tx=0x7f1b80008040 comp rx=0 tx=0).stop 2023-09-23T03:17:08.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.004+0000 7f1b96aa3640 1 -- 172.21.15.154:0/3501770117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b900cd270 msgr2=0x7f1b90157d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:08.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.004+0000 7f1b96aa3640 1 --2- 172.21.15.154:0/3501770117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b900cd270 0x7f1b90157d30 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c02b910 tx=0x7f1b8c0046c0 comp rx=0 tx=0).stop 2023-09-23T03:17:08.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.005+0000 7f1b96aa3640 1 -- 172.21.15.154:0/3501770117 shutdown_connections 2023-09-23T03:17:08.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.005+0000 7f1b96aa3640 1 --2- 172.21.15.154:0/3501770117 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1b7406af30 0x7f1b7406d3f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:08.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.005+0000 7f1b96aa3640 1 --2- 172.21.15.154:0/3501770117 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b901603c0 0x7f1b9015bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:08.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.005+0000 7f1b96aa3640 1 --2- 172.21.15.154:0/3501770117 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b900cf3d0 0x7f1b9015b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:08.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.005+0000 7f1b96aa3640 1 --2- 172.21.15.154:0/3501770117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b900cd270 0x7f1b90157d30 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:08.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.005+0000 7f1b96aa3640 1 -- 172.21.15.154:0/3501770117 >> 172.21.15.154:0/3501770117 conn(0x7f1b9009ba90 msgr2=0x7f1b900cd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:08.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.005+0000 7f1b96aa3640 1 -- 172.21.15.154:0/3501770117 shutdown_connections 2023-09-23T03:17:08.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.005+0000 7f1b96aa3640 1 -- 172.21.15.154:0/3501770117 wait complete. 2023-09-23T03:17:08.025 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-09-23T03:17:08.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.230+0000 7f2747116640 1 Processor -- start 2023-09-23T03:17:08.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.230+0000 7f2747116640 1 -- start start 2023-09-23T03:17:08.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.231+0000 7f2747116640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f27400ada30 0x7f27400ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:08.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.231+0000 7f2747116640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f27400cf130 0x7f274015d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:08.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.231+0000 7f2747116640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f274006ceb0 con 0x7f27400b2fe0 2023-09-23T03:17:08.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.231+0000 7f2747116640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27400ade10 con 0x7f27400cf130 2023-09-23T03:17:08.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.231+0000 7f2747116640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2740159c90 con 0x7f27400ada30 2023-09-23T03:17:08.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.232+0000 7f27456b3640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f27400cf130 0x7f274015d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:08.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.232+0000 7f27456b3640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f27400cf130 0x7f274015d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47190/0 (socket says 172.21.15.154:47190) 2023-09-23T03:17:08.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.232+0000 7f27456b3640 1 -- 172.21.15.154:0/3476434472 learned_addr learned my addr 172.21.15.154:0/3476434472 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:08.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.232+0000 7f27456b3640 1 -- 172.21.15.154:0/3476434472 >> v2:172.21.15.190:3300/0 conn(0x7f27400ada30 msgr2=0x7f27400ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:08.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.232+0000 7f27456b3640 1 --2- 172.21.15.154:0/3476434472 >> v2:172.21.15.190:3300/0 conn(0x7f27400ada30 0x7f27400ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:08.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.232+0000 7f27456b3640 1 -- 172.21.15.154:0/3476434472 >> v1:172.21.15.190:6789/0 conn(0x7f27400b2fe0 legacy=0x7f27400cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:08.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.232+0000 7f27456b3640 1 -- 172.21.15.154:0/3476434472 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f274006c110 con 0x7f27400cf130 2023-09-23T03:17:08.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.233+0000 7f27456b3640 1 --2- 172.21.15.154:0/3476434472 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27400cf130 0x7f274015d380 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f274006d990 tx=0x7f273802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c12f47bd89640bed server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:08.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.233+0000 7f273f7fe640 1 -- 172.21.15.154:0/3476434472 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273802ca80 con 0x7f27400cf130 2023-09-23T03:17:08.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.233+0000 7f273f7fe640 1 -- 172.21.15.154:0/3476434472 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f273802cbe0 con 0x7f27400cf130 2023-09-23T03:17:08.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.233+0000 7f273f7fe640 1 -- 172.21.15.154:0/3476434472 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27380343f0 con 0x7f27400cf130 2023-09-23T03:17:08.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.234+0000 7f2747116640 1 -- 172.21.15.154:0/3476434472 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27400cf130 msgr2=0x7f274015d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:08.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.234+0000 7f2747116640 1 --2- 172.21.15.154:0/3476434472 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27400cf130 0x7f274015d380 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f274006d990 tx=0x7f273802bee0 comp rx=0 tx=0).stop 2023-09-23T03:17:08.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.234+0000 7f2747116640 1 -- 172.21.15.154:0/3476434472 shutdown_connections 2023-09-23T03:17:08.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.234+0000 7f2747116640 1 --2- 172.21.15.154:0/3476434472 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27400cf130 0x7f274015d380 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:08.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.234+0000 7f2747116640 1 --2- 172.21.15.154:0/3476434472 >> v2:172.21.15.190:3300/0 conn(0x7f27400ada30 0x7f27400ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:08.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.234+0000 7f2747116640 1 -- 172.21.15.154:0/3476434472 >> 172.21.15.154:0/3476434472 conn(0x7f274006a0f0 msgr2=0x7f274006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:08.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.234+0000 7f2747116640 1 -- 172.21.15.154:0/3476434472 shutdown_connections 2023-09-23T03:17:08.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.234+0000 7f2747116640 1 -- 172.21.15.154:0/3476434472 wait complete. 2023-09-23T03:17:08.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.236+0000 7f2747116640 1 Processor -- start 2023-09-23T03:17:08.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.236+0000 7f2747116640 1 -- start start 2023-09-23T03:17:08.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.237+0000 7f2747116640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27400ada30 0x7f27401f0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:08.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.237+0000 7f2747116640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27400b2fe0 0x7f274009a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:08.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.237+0000 7f2747116640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27400cf130 0x7f2740152470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:08.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.237+0000 7f2747116640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27401f0f90 con 0x7f27400cf130 2023-09-23T03:17:08.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.237+0000 7f2747116640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274009ae80 con 0x7f27400b2fe0 2023-09-23T03:17:08.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.237+0000 7f2747116640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27401563f0 con 0x7f27400ada30 2023-09-23T03:17:08.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.237+0000 7f27456b3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27400cf130 0x7f2740152470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:08.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.237+0000 7f27456b3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27400cf130 0x7f2740152470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47202/0 (socket says 172.21.15.154:47202) 2023-09-23T03:17:08.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.237+0000 7f27456b3640 1 -- 172.21.15.154:0/2755348073 learned_addr learned my addr 172.21.15.154:0/2755348073 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:08.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.238+0000 7f27456b3640 1 -- 172.21.15.154:0/2755348073 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27400ada30 msgr2=0x7f27401f0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:08.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.238+0000 7f27456b3640 1 --2- 172.21.15.154:0/2755348073 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27400ada30 0x7f27401f0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:08.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.238+0000 7f27456b3640 1 -- 172.21.15.154:0/2755348073 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27400b2fe0 msgr2=0x7f274009a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:08.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.238+0000 7f27456b3640 1 --2- 172.21.15.154:0/2755348073 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27400b2fe0 0x7f274009a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:08.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.238+0000 7f27456b3640 1 -- 172.21.15.154:0/2755348073 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2738009b30 con 0x7f27400cf130 2023-09-23T03:17:08.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.238+0000 7f27456b3640 1 --2- 172.21.15.154:0/2755348073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27400cf130 0x7f2740152470 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f2738036040 tx=0x7f2738004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:08.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.238+0000 7f273d7fa640 1 -- 172.21.15.154:0/2755348073 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273803e450 con 0x7f27400cf130 2023-09-23T03:17:08.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.238+0000 7f273d7fa640 1 -- 172.21.15.154:0/2755348073 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f273803e5d0 con 0x7f27400cf130 2023-09-23T03:17:08.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.238+0000 7f273d7fa640 1 -- 172.21.15.154:0/2755348073 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273803e980 con 0x7f27400cf130 2023-09-23T03:17:08.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.238+0000 7f2747116640 1 -- 172.21.15.154:0/2755348073 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2740098dd0 con 0x7f27400cf130 2023-09-23T03:17:08.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.239+0000 7f2747116640 1 -- 172.21.15.154:0/2755348073 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2740099390 con 0x7f27400cf130 2023-09-23T03:17:08.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.240+0000 7f2747116640 1 -- 172.21.15.154:0/2755348073 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2740073eb0 con 0x7f27400cf130 2023-09-23T03:17:08.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.240+0000 7f273d7fa640 1 -- 172.21.15.154:0/2755348073 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f273802cd50 con 0x7f27400cf130 2023-09-23T03:17:08.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.242+0000 7f273d7fa640 1 --2- 172.21.15.154:0/2755348073 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f272406af30 0x7f272406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:08.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.242+0000 7f273d7fa640 1 -- 172.21.15.154:0/2755348073 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f27380abf30 con 0x7f27400cf130 2023-09-23T03:17:08.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.242+0000 7f2744eb2640 1 --2- 172.21.15.154:0/2755348073 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f272406af30 0x7f272406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:08.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.243+0000 7f2744eb2640 1 --2- 172.21.15.154:0/2755348073 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f272406af30 0x7f272406d3f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2740159410 tx=0x7f2734006be0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:08.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.246+0000 7f273d7fa640 1 -- 172.21.15.154:0/2755348073 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f27380772c0 con 0x7f27400cf130 2023-09-23T03:17:08.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.436+0000 7f2747116640 1 -- 172.21.15.154:0/2755348073 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f274006ceb0 con 0x7f27400cf130 2023-09-23T03:17:08.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.437+0000 7f273d7fa640 1 -- 172.21.15.154:0/2755348073 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v34) v1 ==== 84+0+65 (secure 0 0 0) 0x7f2738043060 con 0x7f27400cf130 2023-09-23T03:17:08.438 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:08.438 INFO:teuthology.orchestra.run.smithi154.stdout:[ 2023-09-23T03:17:08.438 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:17:08.438 INFO:teuthology.orchestra.run.smithi154.stdout: "poolnum": 1, 2023-09-23T03:17:08.439 INFO:teuthology.orchestra.run.smithi154.stdout: "poolname": ".mgr" 2023-09-23T03:17:08.439 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:17:08.439 INFO:teuthology.orchestra.run.smithi154.stdout:] 2023-09-23T03:17:08.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.439+0000 7f2747116640 1 -- 172.21.15.154:0/2755348073 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f272406af30 msgr2=0x7f272406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:08.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.439+0000 7f2747116640 1 --2- 172.21.15.154:0/2755348073 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f272406af30 0x7f272406d3f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2740159410 tx=0x7f2734006be0 comp rx=0 tx=0).stop 2023-09-23T03:17:08.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.439+0000 7f2747116640 1 -- 172.21.15.154:0/2755348073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27400cf130 msgr2=0x7f2740152470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:08.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.439+0000 7f2747116640 1 --2- 172.21.15.154:0/2755348073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27400cf130 0x7f2740152470 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f2738036040 tx=0x7f2738004a40 comp rx=0 tx=0).stop 2023-09-23T03:17:08.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.439+0000 7f2747116640 1 -- 172.21.15.154:0/2755348073 shutdown_connections 2023-09-23T03:17:08.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.439+0000 7f2747116640 1 --2- 172.21.15.154:0/2755348073 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f272406af30 0x7f272406d3f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:08.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.439+0000 7f2747116640 1 --2- 172.21.15.154:0/2755348073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27400cf130 0x7f2740152470 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:08.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.439+0000 7f2747116640 1 --2- 172.21.15.154:0/2755348073 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27400b2fe0 0x7f274009a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:08.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.439+0000 7f2747116640 1 --2- 172.21.15.154:0/2755348073 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27400ada30 0x7f27401f0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:08.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.439+0000 7f2747116640 1 -- 172.21.15.154:0/2755348073 >> 172.21.15.154:0/2755348073 conn(0x7f274006a0f0 msgr2=0x7f27400adf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:08.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.439+0000 7f2747116640 1 -- 172.21.15.154:0/2755348073 shutdown_connections 2023-09-23T03:17:08.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.440+0000 7f2747116640 1 -- 172.21.15.154:0/2755348073 wait complete. 2023-09-23T03:17:08.458 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-09-23T03:17:08.459 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-09-23T03:17:08.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.673+0000 7fb353d6c640 1 Processor -- start 2023-09-23T03:17:08.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.674+0000 7fb353d6c640 1 -- start start 2023-09-23T03:17:08.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.675+0000 7fb353d6c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb34c0700f0 0x7fb34c0c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:08.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.675+0000 7fb353d6c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb34c0cb460 0x7fb34c0b1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:08.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.675+0000 7fb353d6c640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb34c0c95b0 con 0x7fb34c15abd0 2023-09-23T03:17:08.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.675+0000 7fb353d6c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb34c0c9890 con 0x7fb34c0700f0 2023-09-23T03:17:08.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.675+0000 7fb353d6c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb34c0c9b70 con 0x7fb34c0cb460 2023-09-23T03:17:08.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.675+0000 7fb351b08640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb34c0700f0 0x7fb34c0c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:08.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.675+0000 7fb351b08640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb34c0700f0 0x7fb34c0c8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47216/0 (socket says 172.21.15.154:47216) 2023-09-23T03:17:08.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.676+0000 7fb351b08640 1 -- 172.21.15.154:0/4229411771 learned_addr learned my addr 172.21.15.154:0/4229411771 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:08.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.676+0000 7fb352309640 1 --2- 172.21.15.154:0/4229411771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb34c0cb460 0x7fb34c0b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:08.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.676+0000 7fb351b08640 1 -- 172.21.15.154:0/4229411771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb34c0cb460 msgr2=0x7fb34c0b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:08.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.676+0000 7fb351b08640 1 --2- 172.21.15.154:0/4229411771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb34c0cb460 0x7fb34c0b1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:08.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.676+0000 7fb351b08640 1 -- 172.21.15.154:0/4229411771 >> v1:172.21.15.190:6789/0 conn(0x7fb34c15abd0 legacy=0x7fb34c0704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:08.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.676+0000 7fb351b08640 1 -- 172.21.15.154:0/4229411771 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb34c0c9e50 con 0x7fb34c0700f0 2023-09-23T03:17:08.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.676+0000 7fb352309640 1 --2- 172.21.15.154:0/4229411771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb34c0cb460 0x7fb34c0b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:08.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.677+0000 7fb351b08640 1 --2- 172.21.15.154:0/4229411771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb34c0700f0 0x7fb34c0c8fd0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb34c0cbc40 tx=0x7fb33c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c088dcfdb0cc94f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:08.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.677+0000 7fb350b06640 1 -- 172.21.15.154:0/4229411771 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c02ca50 con 0x7fb34c0700f0 2023-09-23T03:17:08.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.677+0000 7fb350b06640 1 -- 172.21.15.154:0/4229411771 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33c02cbb0 con 0x7fb34c0700f0 2023-09-23T03:17:08.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.678+0000 7fb350b06640 1 -- 172.21.15.154:0/4229411771 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c0343f0 con 0x7fb34c0700f0 2023-09-23T03:17:08.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.678+0000 7fb353d6c640 1 -- 172.21.15.154:0/4229411771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb34c0700f0 msgr2=0x7fb34c0c8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:08.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.678+0000 7fb353d6c640 1 --2- 172.21.15.154:0/4229411771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb34c0700f0 0x7fb34c0c8fd0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb34c0cbc40 tx=0x7fb33c02beb0 comp rx=0 tx=0).stop 2023-09-23T03:17:08.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.678+0000 7fb353d6c640 1 -- 172.21.15.154:0/4229411771 shutdown_connections 2023-09-23T03:17:08.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.678+0000 7fb353d6c640 1 --2- 172.21.15.154:0/4229411771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb34c0cb460 0x7fb34c0b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:08.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.678+0000 7fb353d6c640 1 --2- 172.21.15.154:0/4229411771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb34c0700f0 0x7fb34c0c8fd0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:08.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.678+0000 7fb353d6c640 1 -- 172.21.15.154:0/4229411771 >> 172.21.15.154:0/4229411771 conn(0x7fb34c06a520 msgr2=0x7fb34c06bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:08.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.678+0000 7fb353d6c640 1 -- 172.21.15.154:0/4229411771 shutdown_connections 2023-09-23T03:17:08.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.678+0000 7fb353d6c640 1 -- 172.21.15.154:0/4229411771 wait complete. 2023-09-23T03:17:08.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.680+0000 7fb353d6c640 1 Processor -- start 2023-09-23T03:17:08.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.680+0000 7fb353d6c640 1 -- start start 2023-09-23T03:17:08.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.681+0000 7fb353d6c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb34c0700f0 0x7fb34c0c37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:08.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.681+0000 7fb353d6c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb34c0cb460 0x7fb34c0ad960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:08.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.681+0000 7fb353d6c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb34c15abd0 0x7fb34c0adea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:08.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.681+0000 7fb353d6c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34c1f2280 con 0x7fb34c0cb460 2023-09-23T03:17:08.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.681+0000 7fb353d6c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34c156b40 con 0x7fb34c0700f0 2023-09-23T03:17:08.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.681+0000 7fb353d6c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb34c0ae3e0 con 0x7fb34c15abd0 2023-09-23T03:17:08.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.681+0000 7fb351307640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb34c0cb460 0x7fb34c0ad960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:08.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.681+0000 7fb351307640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb34c0cb460 0x7fb34c0ad960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47232/0 (socket says 172.21.15.154:47232) 2023-09-23T03:17:08.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.681+0000 7fb351307640 1 -- 172.21.15.154:0/1787213073 learned_addr learned my addr 172.21.15.154:0/1787213073 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:08.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.682+0000 7fb351b08640 1 --2- 172.21.15.154:0/1787213073 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb34c0700f0 0x7fb34c0c37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:08.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.682+0000 7fb351307640 1 -- 172.21.15.154:0/1787213073 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb34c15abd0 msgr2=0x7fb34c0adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:08.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.682+0000 7fb351307640 1 --2- 172.21.15.154:0/1787213073 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb34c15abd0 0x7fb34c0adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:08.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.682+0000 7fb351307640 1 -- 172.21.15.154:0/1787213073 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb34c0700f0 msgr2=0x7fb34c0c37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:08.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.682+0000 7fb351307640 1 --2- 172.21.15.154:0/1787213073 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb34c0700f0 0x7fb34c0c37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:08.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.682+0000 7fb351307640 1 -- 172.21.15.154:0/1787213073 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33c009b30 con 0x7fb34c0cb460 2023-09-23T03:17:08.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.682+0000 7fb351307640 1 --2- 172.21.15.154:0/1787213073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb34c0cb460 0x7fb34c0ad960 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb34000c400 tx=0x7fb34000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:08.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.682+0000 7fb34a7fc640 1 -- 172.21.15.154:0/1787213073 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb340011790 con 0x7fb34c0cb460 2023-09-23T03:17:08.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.682+0000 7fb34a7fc640 1 -- 172.21.15.154:0/1787213073 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb340011930 con 0x7fb34c0cb460 2023-09-23T03:17:08.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.682+0000 7fb34a7fc640 1 -- 172.21.15.154:0/1787213073 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb340011ce0 con 0x7fb34c0cb460 2023-09-23T03:17:08.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.682+0000 7fb353d6c640 1 -- 172.21.15.154:0/1787213073 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb34c0ae920 con 0x7fb34c0cb460 2023-09-23T03:17:08.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.682+0000 7fb353d6c640 1 -- 172.21.15.154:0/1787213073 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb34c0aeee0 con 0x7fb34c0cb460 2023-09-23T03:17:08.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.683+0000 7fb353d6c640 1 -- 172.21.15.154:0/1787213073 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb34c04e920 con 0x7fb34c0cb460 2023-09-23T03:17:08.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.685+0000 7fb34a7fc640 1 -- 172.21.15.154:0/1787213073 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fb34001c460 con 0x7fb34c0cb460 2023-09-23T03:17:08.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.685+0000 7fb34a7fc640 1 --2- 172.21.15.154:0/1787213073 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb33806af30 0x7fb33806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:08.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.686+0000 7fb34a7fc640 1 -- 172.21.15.154:0/1787213073 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fb340089570 con 0x7fb34c0cb460 2023-09-23T03:17:08.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.686+0000 7fb351b08640 1 --2- 172.21.15.154:0/1787213073 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb33806af30 0x7fb33806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:08.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.687+0000 7fb351b08640 1 --2- 172.21.15.154:0/1787213073 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb33806af30 0x7fb33806d3f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb33c035000 tx=0x7fb33c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:08.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.690+0000 7fb34a7fc640 1 -- 172.21.15.154:0/1787213073 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fb340054720 con 0x7fb34c0cb460 2023-09-23T03:17:08.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:08.879+0000 7fb353d6c640 1 -- 172.21.15.154:0/1787213073 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb34c0b43f0 con 0x7fb34c0cb460 2023-09-23T03:17:09.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.626+0000 7fb34a7fc640 1 -- 172.21.15.154:0/1787213073 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v35) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb340057d40 con 0x7fb34c0cb460 2023-09-23T03:17:09.628 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_metadata' created 2023-09-23T03:17:09.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.631+0000 7fb353d6c640 1 -- 172.21.15.154:0/1787213073 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb33806af30 msgr2=0x7fb33806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:09.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.631+0000 7fb353d6c640 1 --2- 172.21.15.154:0/1787213073 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb33806af30 0x7fb33806d3f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb33c035000 tx=0x7fb33c030000 comp rx=0 tx=0).stop 2023-09-23T03:17:09.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.631+0000 7fb353d6c640 1 -- 172.21.15.154:0/1787213073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb34c0cb460 msgr2=0x7fb34c0ad960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:09.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.631+0000 7fb353d6c640 1 --2- 172.21.15.154:0/1787213073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb34c0cb460 0x7fb34c0ad960 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb34000c400 tx=0x7fb34000c8c0 comp rx=0 tx=0).stop 2023-09-23T03:17:09.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.631+0000 7fb353d6c640 1 -- 172.21.15.154:0/1787213073 shutdown_connections 2023-09-23T03:17:09.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.631+0000 7fb353d6c640 1 --2- 172.21.15.154:0/1787213073 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb33806af30 0x7fb33806d3f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:09.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.631+0000 7fb353d6c640 1 --2- 172.21.15.154:0/1787213073 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb34c15abd0 0x7fb34c0adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:09.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.631+0000 7fb353d6c640 1 --2- 172.21.15.154:0/1787213073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb34c0cb460 0x7fb34c0ad960 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:09.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.631+0000 7fb353d6c640 1 --2- 172.21.15.154:0/1787213073 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb34c0700f0 0x7fb34c0c37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:09.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.631+0000 7fb353d6c640 1 -- 172.21.15.154:0/1787213073 >> 172.21.15.154:0/1787213073 conn(0x7fb34c06a520 msgr2=0x7fb34c0b33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:09.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.632+0000 7fb353d6c640 1 -- 172.21.15.154:0/1787213073 shutdown_connections 2023-09-23T03:17:09.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.632+0000 7fb353d6c640 1 -- 172.21.15.154:0/1787213073 wait complete. 2023-09-23T03:17:09.661 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-09-23T03:17:09.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.878+0000 7fbe2e91d640 1 Processor -- start 2023-09-23T03:17:09.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.878+0000 7fbe2e91d640 1 -- start start 2023-09-23T03:17:09.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.879+0000 7fbe2e91d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbe2815c9c0 0x7fbe280d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:09.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.879+0000 7fbe2e91d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe280cd450 con 0x7fbe2815cda0 2023-09-23T03:17:09.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.879+0000 7fbe2e91d640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe280d0df0 con 0x7fbe280d2d30 2023-09-23T03:17:09.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.879+0000 7fbe2e91d640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe280d10d0 con 0x7fbe2815c9c0 2023-09-23T03:17:09.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.880+0000 7fbe277fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fbe2815cda0 0x7fbe280d2620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:42264/0 (socket says 172.21.15.154:42264) 2023-09-23T03:17:09.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.880+0000 7fbe277fe640 1 -- 172.21.15.154:0/1804513021 learned_addr learned my addr 172.21.15.154:0/1804513021 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:17:09.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.880+0000 7fbe27fff640 1 --2- 172.21.15.154:0/1804513021 >> v2:172.21.15.190:3300/0 conn(0x7fbe2815c9c0 0x7fbe280d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:09.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.880+0000 7fbe26ffd640 1 -- 172.21.15.154:0/1804513021 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1767143398 0 0) 0x7fbe280cd450 con 0x7fbe2815cda0 2023-09-23T03:17:09.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.881+0000 7fbe26ffd640 1 -- 172.21.15.154:0/1804513021 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe14003520 con 0x7fbe2815cda0 2023-09-23T03:17:09.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.881+0000 7fbe26ffd640 1 -- 172.21.15.154:0/1804513021 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1864544070 0 0) 0x7fbe14003520 con 0x7fbe2815cda0 2023-09-23T03:17:09.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.881+0000 7fbe26ffd640 1 -- 172.21.15.154:0/1804513021 >> v2:172.21.15.190:3300/0 conn(0x7fbe2815c9c0 msgr2=0x7fbe280d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:09.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.881+0000 7fbe26ffd640 1 --2- 172.21.15.154:0/1804513021 >> v2:172.21.15.190:3300/0 conn(0x7fbe2815c9c0 0x7fbe280d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:09.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.881+0000 7fbe26ffd640 1 -- 172.21.15.154:0/1804513021 >> v1:172.21.15.190:6789/0 conn(0x7fbe280d2d30 legacy=0x7fbe280cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:09.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.881+0000 7fbe26ffd640 1 -- 172.21.15.154:0/1804513021 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe280d13b0 con 0x7fbe2815cda0 2023-09-23T03:17:09.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.881+0000 7fbe26ffd640 1 -- 172.21.15.154:0/1804513021 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fbe10001670 con 0x7fbe2815cda0 2023-09-23T03:17:09.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.881+0000 7fbe26ffd640 1 -- 172.21.15.154:0/1804513021 >> v1:172.21.15.154:6789/0 conn(0x7fbe2815cda0 legacy=0x7fbe280d2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:09.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.881+0000 7fbe26ffd640 1 --2- 172.21.15.154:0/1804513021 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe14003970 0x7fbe14003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:09.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.881+0000 7fbe26ffd640 1 --2- 172.21.15.154:0/1804513021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe140047c0 0x7fbe14004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:09.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.881+0000 7fbe26ffd640 1 --2- 172.21.15.154:0/1804513021 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe14005180 0x7fbe1400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:09.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.881+0000 7fbe26ffd640 1 -- 172.21.15.154:0/1804513021 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe280d10d0 con 0x7fbe140047c0 2023-09-23T03:17:09.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.881+0000 7fbe26ffd640 1 -- 172.21.15.154:0/1804513021 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1400c120 con 0x7fbe14003970 2023-09-23T03:17:09.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.881+0000 7fbe26ffd640 1 -- 172.21.15.154:0/1804513021 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1400c2e0 con 0x7fbe14005180 2023-09-23T03:17:09.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.881+0000 7fbe277fe640 1 --2- 172.21.15.154:0/1804513021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe140047c0 0x7fbe14004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:09.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.882+0000 7fbe27fff640 1 --2- 172.21.15.154:0/1804513021 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe14003970 0x7fbe14003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:09.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.882+0000 7fbe277fe640 1 -- 172.21.15.154:0/1804513021 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe14005180 msgr2=0x7fbe1400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:09.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.882+0000 7fbe277fe640 1 --2- 172.21.15.154:0/1804513021 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe14005180 0x7fbe1400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:09.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.882+0000 7fbe277fe640 1 -- 172.21.15.154:0/1804513021 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe14003970 msgr2=0x7fbe14003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:09.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.882+0000 7fbe277fe640 1 --2- 172.21.15.154:0/1804513021 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe14003970 0x7fbe14003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:09.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.882+0000 7fbe277fe640 1 -- 172.21.15.154:0/1804513021 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe14003520 con 0x7fbe140047c0 2023-09-23T03:17:09.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.882+0000 7fbe2ceba640 1 -- 172.21.15.154:0/1804513021 reap_dead start 2023-09-23T03:17:09.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.883+0000 7fbe277fe640 1 --2- 172.21.15.154:0/1804513021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe140047c0 0x7fbe14004c40 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fbe1000c190 tx=0x7fbe140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c687ab02cfd2de8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:09.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.883+0000 7fbe26ffd640 1 -- 172.21.15.154:0/1804513021 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe10005ad0 con 0x7fbe140047c0 2023-09-23T03:17:09.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.883+0000 7fbe26ffd640 1 -- 172.21.15.154:0/1804513021 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe10005c50 con 0x7fbe140047c0 2023-09-23T03:17:09.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.883+0000 7fbe26ffd640 1 -- 172.21.15.154:0/1804513021 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1002c8a0 con 0x7fbe140047c0 2023-09-23T03:17:09.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.884+0000 7fbe2e91d640 1 -- 172.21.15.154:0/1804513021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe140047c0 msgr2=0x7fbe14004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:09.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.884+0000 7fbe2e91d640 1 --2- 172.21.15.154:0/1804513021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe140047c0 0x7fbe14004c40 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fbe1000c190 tx=0x7fbe140057f0 comp rx=0 tx=0).stop 2023-09-23T03:17:09.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.884+0000 7fbe2e91d640 1 -- 172.21.15.154:0/1804513021 shutdown_connections 2023-09-23T03:17:09.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.884+0000 7fbe2e91d640 1 --2- 172.21.15.154:0/1804513021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe140047c0 0x7fbe14004c40 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:09.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.884+0000 7fbe2e91d640 1 -- 172.21.15.154:0/1804513021 >> 172.21.15.154:0/1804513021 conn(0x7fbe2809bc20 msgr2=0x7fbe28089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:09.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.884+0000 7fbe2e91d640 1 -- 172.21.15.154:0/1804513021 shutdown_connections 2023-09-23T03:17:09.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.884+0000 7fbe2e91d640 1 -- 172.21.15.154:0/1804513021 wait complete. 2023-09-23T03:17:09.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.886+0000 7fbe2e91d640 1 Processor -- start 2023-09-23T03:17:09.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.886+0000 7fbe2e91d640 1 -- start start 2023-09-23T03:17:09.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.887+0000 7fbe2e91d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe140047c0 0x7fbe281ef840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:09.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.887+0000 7fbe2e91d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe2815e2e0 0x7fbe281f22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:09.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.887+0000 7fbe2e91d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe281f2820 0x7fbe280b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:09.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.887+0000 7fbe2e91d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe281efd80 con 0x7fbe281f2820 2023-09-23T03:17:09.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.887+0000 7fbe2e91d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe281f2da0 con 0x7fbe2815e2e0 2023-09-23T03:17:09.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.887+0000 7fbe2e91d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe281f3080 con 0x7fbe140047c0 2023-09-23T03:17:09.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.887+0000 7fbe2ceba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe281f2820 0x7fbe280b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:09.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.887+0000 7fbe2ceba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe281f2820 0x7fbe280b77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47238/0 (socket says 172.21.15.154:47238) 2023-09-23T03:17:09.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.887+0000 7fbe2ceba640 1 -- 172.21.15.154:0/2274965268 learned_addr learned my addr 172.21.15.154:0/2274965268 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:09.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.888+0000 7fbe27fff640 1 --2- 172.21.15.154:0/2274965268 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe140047c0 0x7fbe281ef840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:09.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.888+0000 7fbe277fe640 1 --2- 172.21.15.154:0/2274965268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe2815e2e0 0x7fbe281f22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:09.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.888+0000 7fbe2ceba640 1 -- 172.21.15.154:0/2274965268 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe140047c0 msgr2=0x7fbe281ef840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:09.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.888+0000 7fbe2ceba640 1 --2- 172.21.15.154:0/2274965268 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe140047c0 0x7fbe281ef840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:09.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.888+0000 7fbe2ceba640 1 -- 172.21.15.154:0/2274965268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe2815e2e0 msgr2=0x7fbe281f22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:09.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.888+0000 7fbe2ceba640 1 --2- 172.21.15.154:0/2274965268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe2815e2e0 0x7fbe281f22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:09.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.888+0000 7fbe2ceba640 1 -- 172.21.15.154:0/2274965268 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe280c7190 con 0x7fbe281f2820 2023-09-23T03:17:09.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.888+0000 7fbe277fe640 1 --2- 172.21.15.154:0/2274965268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe2815e2e0 0x7fbe281f22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:09.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.888+0000 7fbe2ceba640 1 --2- 172.21.15.154:0/2274965268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe281f2820 0x7fbe280b77f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fbe280d39d0 tx=0x7fbe2000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:09.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.889+0000 7fbe24ff9640 1 -- 172.21.15.154:0/2274965268 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe20004220 con 0x7fbe281f2820 2023-09-23T03:17:09.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.889+0000 7fbe24ff9640 1 -- 172.21.15.154:0/2274965268 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe200043a0 con 0x7fbe281f2820 2023-09-23T03:17:09.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.889+0000 7fbe24ff9640 1 -- 172.21.15.154:0/2274965268 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe20007310 con 0x7fbe281f2820 2023-09-23T03:17:09.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.889+0000 7fbe2e91d640 1 -- 172.21.15.154:0/2274965268 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe280c7470 con 0x7fbe281f2820 2023-09-23T03:17:09.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.889+0000 7fbe2e91d640 1 -- 172.21.15.154:0/2274965268 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe280b7d30 con 0x7fbe281f2820 2023-09-23T03:17:09.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.890+0000 7fbe2e91d640 1 -- 172.21.15.154:0/2274965268 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe280c7660 con 0x7fbe281f2820 2023-09-23T03:17:09.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.891+0000 7fbe24ff9640 1 -- 172.21.15.154:0/2274965268 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fbe20007470 con 0x7fbe281f2820 2023-09-23T03:17:09.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.892+0000 7fbe24ff9640 1 --2- 172.21.15.154:0/2274965268 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbe0806af30 0x7fbe0806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:09.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.892+0000 7fbe24ff9640 1 -- 172.21.15.154:0/2274965268 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5547+0+0 (secure 0 0 0) 0x7fbe20088610 con 0x7fbe281f2820 2023-09-23T03:17:09.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.892+0000 7fbe27fff640 1 --2- 172.21.15.154:0/2274965268 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbe0806af30 0x7fbe0806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:09.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.893+0000 7fbe27fff640 1 --2- 172.21.15.154:0/2274965268 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbe0806af30 0x7fbe0806d3f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbe281f5830 tx=0x7fbe1800b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:09.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:09.897+0000 7fbe24ff9640 1 -- 172.21.15.154:0/2274965268 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fbe20053780 con 0x7fbe281f2820 2023-09-23T03:17:10.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.087+0000 7fbe2e91d640 1 -- 172.21.15.154:0/2274965268 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fbe2804e920 con 0x7fbe281f2820 2023-09-23T03:17:10.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.632+0000 7fbe24ff9640 1 -- 172.21.15.154:0/2274965268 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v36) v1 ==== 170+0+0 (secure 0 0 0) 0x7fbe20056da0 con 0x7fbe281f2820 2023-09-23T03:17:10.634 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_data' created 2023-09-23T03:17:10.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.637+0000 7fbe2e91d640 1 -- 172.21.15.154:0/2274965268 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbe0806af30 msgr2=0x7fbe0806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:10.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.637+0000 7fbe2e91d640 1 --2- 172.21.15.154:0/2274965268 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbe0806af30 0x7fbe0806d3f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbe281f5830 tx=0x7fbe1800b040 comp rx=0 tx=0).stop 2023-09-23T03:17:10.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.637+0000 7fbe2e91d640 1 -- 172.21.15.154:0/2274965268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe281f2820 msgr2=0x7fbe280b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:10.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.637+0000 7fbe2e91d640 1 --2- 172.21.15.154:0/2274965268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe281f2820 0x7fbe280b77f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fbe280d39d0 tx=0x7fbe2000c820 comp rx=0 tx=0).stop 2023-09-23T03:17:10.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.637+0000 7fbe2e91d640 1 -- 172.21.15.154:0/2274965268 shutdown_connections 2023-09-23T03:17:10.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.637+0000 7fbe2e91d640 1 --2- 172.21.15.154:0/2274965268 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbe0806af30 0x7fbe0806d3f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:10.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.637+0000 7fbe2e91d640 1 --2- 172.21.15.154:0/2274965268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe281f2820 0x7fbe280b77f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:10.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.637+0000 7fbe2e91d640 1 --2- 172.21.15.154:0/2274965268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbe2815e2e0 0x7fbe281f22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:10.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.637+0000 7fbe2e91d640 1 --2- 172.21.15.154:0/2274965268 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe140047c0 0x7fbe281ef840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:10.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.637+0000 7fbe2e91d640 1 -- 172.21.15.154:0/2274965268 >> 172.21.15.154:0/2274965268 conn(0x7fbe2809bc20 msgr2=0x7fbe280b1740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:10.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.637+0000 7fbe2e91d640 1 -- 172.21.15.154:0/2274965268 shutdown_connections 2023-09-23T03:17:10.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.637+0000 7fbe2e91d640 1 -- 172.21.15.154:0/2274965268 wait complete. 2023-09-23T03:17:10.670 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-09-23T03:17:10.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.859+0000 7f621f7a9640 1 Processor -- start 2023-09-23T03:17:10.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.860+0000 7f621f7a9640 1 -- start start 2023-09-23T03:17:10.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.861+0000 7f621f7a9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62180c8d70 0x7f621815c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:10.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.861+0000 7f621f7a9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f62180c9150 0x7f621815cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:10.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.861+0000 7f621f7a9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6218071290 0x7f62180ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:10.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.861+0000 7f621f7a9640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62180c9d20 con 0x7f62180c9150 2023-09-23T03:17:10.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.861+0000 7f621f7a9640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6218071850 con 0x7f6218071290 2023-09-23T03:17:10.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.861+0000 7f621f7a9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6218071b30 con 0x7f62180c8d70 2023-09-23T03:17:10.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.861+0000 7f621cd44640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f62180c9150 0x7f621815cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:10.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.862+0000 7f621cd44640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f62180c9150 0x7f621815cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47254/0 (socket says 172.21.15.154:47254) 2023-09-23T03:17:10.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.862+0000 7f621cd44640 1 -- 172.21.15.154:0/1755911531 learned_addr learned my addr 172.21.15.154:0/1755911531 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:10.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.862+0000 7f621dd46640 1 --2- 172.21.15.154:0/1755911531 >> v2:172.21.15.190:3300/0 conn(0x7f6218071290 0x7f62180ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:10.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.862+0000 7f621d545640 1 --2- 172.21.15.154:0/1755911531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62180c8d70 0x7f621815c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:10.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.862+0000 7f621cd44640 1 -- 172.21.15.154:0/1755911531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62180c8d70 msgr2=0x7f621815c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:10.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.862+0000 7f621cd44640 1 --2- 172.21.15.154:0/1755911531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62180c8d70 0x7f621815c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:10.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.862+0000 7f621cd44640 1 -- 172.21.15.154:0/1755911531 >> v2:172.21.15.190:3300/0 conn(0x7f6218071290 msgr2=0x7f62180ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:10.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.862+0000 7f621cd44640 1 --2- 172.21.15.154:0/1755911531 >> v2:172.21.15.190:3300/0 conn(0x7f6218071290 0x7f62180ca1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:10.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.862+0000 7f621cd44640 1 -- 172.21.15.154:0/1755911531 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6218071e10 con 0x7f62180c9150 2023-09-23T03:17:10.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.862+0000 7f621dd46640 1 --2- 172.21.15.154:0/1755911531 >> v2:172.21.15.190:3300/0 conn(0x7f6218071290 0x7f62180ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:10.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.862+0000 7f621d545640 1 --2- 172.21.15.154:0/1755911531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62180c8d70 0x7f621815c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:10.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.863+0000 7f621cd44640 1 --2- 172.21.15.154:0/1755911531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62180c9150 0x7f621815cc10 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f62180c95d0 tx=0x7f620802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d397168efa3b1bdd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:10.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.864+0000 7f620ffff640 1 -- 172.21.15.154:0/1755911531 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62080317e0 con 0x7f62180c9150 2023-09-23T03:17:10.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.864+0000 7f620ffff640 1 -- 172.21.15.154:0/1755911531 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6208031940 con 0x7f62180c9150 2023-09-23T03:17:10.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.864+0000 7f620ffff640 1 -- 172.21.15.154:0/1755911531 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6208031cd0 con 0x7f62180c9150 2023-09-23T03:17:10.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.864+0000 7f621f7a9640 1 -- 172.21.15.154:0/1755911531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62180c9150 msgr2=0x7f621815cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:10.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.864+0000 7f621f7a9640 1 --2- 172.21.15.154:0/1755911531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62180c9150 0x7f621815cc10 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f62180c95d0 tx=0x7f620802b880 comp rx=0 tx=0).stop 2023-09-23T03:17:10.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.864+0000 7f621f7a9640 1 -- 172.21.15.154:0/1755911531 shutdown_connections 2023-09-23T03:17:10.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.864+0000 7f621f7a9640 1 --2- 172.21.15.154:0/1755911531 >> v2:172.21.15.190:3300/0 conn(0x7f6218071290 0x7f62180ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:10.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.864+0000 7f621f7a9640 1 --2- 172.21.15.154:0/1755911531 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62180c9150 0x7f621815cc10 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:10.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.864+0000 7f621f7a9640 1 --2- 172.21.15.154:0/1755911531 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62180c8d70 0x7f621815c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:10.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.864+0000 7f621f7a9640 1 -- 172.21.15.154:0/1755911531 >> 172.21.15.154:0/1755911531 conn(0x7f621806c010 msgr2=0x7f621806c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:10.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.864+0000 7f621f7a9640 1 -- 172.21.15.154:0/1755911531 shutdown_connections 2023-09-23T03:17:10.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.865+0000 7f621f7a9640 1 -- 172.21.15.154:0/1755911531 wait complete. 2023-09-23T03:17:10.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.866+0000 7f621f7a9640 1 Processor -- start 2023-09-23T03:17:10.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.866+0000 7f621f7a9640 1 -- start start 2023-09-23T03:17:10.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.867+0000 7f621f7a9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6218071290 0x7f621814c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:10.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.867+0000 7f621f7a9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62180c8d70 0x7f62181505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:10.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.867+0000 7f621f7a9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62180c9150 0x7f6218145420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:10.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.867+0000 7f621f7a9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62180c8510 con 0x7f6218071290 2023-09-23T03:17:10.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.867+0000 7f621f7a9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62180c41c0 con 0x7f62180c9150 2023-09-23T03:17:10.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.867+0000 7f621f7a9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62180c44a0 con 0x7f62180c8d70 2023-09-23T03:17:10.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.868+0000 7f621d545640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6218071290 0x7f621814c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:10.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.868+0000 7f621d545640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6218071290 0x7f621814c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47270/0 (socket says 172.21.15.154:47270) 2023-09-23T03:17:10.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.868+0000 7f621d545640 1 -- 172.21.15.154:0/1751424383 learned_addr learned my addr 172.21.15.154:0/1751424383 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:10.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.868+0000 7f621cd44640 1 --2- 172.21.15.154:0/1751424383 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62180c8d70 0x7f62181505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:10.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.868+0000 7f621dd46640 1 --2- 172.21.15.154:0/1751424383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62180c9150 0x7f6218145420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:10.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.868+0000 7f621d545640 1 -- 172.21.15.154:0/1751424383 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62180c8d70 msgr2=0x7f62181505e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:10.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.868+0000 7f621d545640 1 --2- 172.21.15.154:0/1751424383 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62180c8d70 0x7f62181505e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:10.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.868+0000 7f621d545640 1 -- 172.21.15.154:0/1751424383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62180c9150 msgr2=0x7f6218145420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:10.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.868+0000 7f621d545640 1 --2- 172.21.15.154:0/1751424383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62180c9150 0x7f6218145420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:10.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.868+0000 7f621d545640 1 -- 172.21.15.154:0/1751424383 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62080094d0 con 0x7f6218071290 2023-09-23T03:17:10.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.868+0000 7f621dd46640 1 --2- 172.21.15.154:0/1751424383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62180c9150 0x7f6218145420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:10.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.868+0000 7f621d545640 1 --2- 172.21.15.154:0/1751424383 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6218071290 0x7f621814c290 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f621400c7f0 tx=0x7f621400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:10.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.869+0000 7f620dffb640 1 -- 172.21.15.154:0/1751424383 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6214007a30 con 0x7f6218071290 2023-09-23T03:17:10.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.869+0000 7f620dffb640 1 -- 172.21.15.154:0/1751424383 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6214007b90 con 0x7f6218071290 2023-09-23T03:17:10.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.869+0000 7f620dffb640 1 -- 172.21.15.154:0/1751424383 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621400ce40 con 0x7f6218071290 2023-09-23T03:17:10.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.869+0000 7f621f7a9640 1 -- 172.21.15.154:0/1751424383 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62180c4780 con 0x7f6218071290 2023-09-23T03:17:10.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.869+0000 7f621f7a9640 1 -- 172.21.15.154:0/1751424383 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62180c4ca0 con 0x7f6218071290 2023-09-23T03:17:10.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.870+0000 7f621f7a9640 1 -- 172.21.15.154:0/1751424383 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62180c4970 con 0x7f6218071290 2023-09-23T03:17:10.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.871+0000 7f620dffb640 1 -- 172.21.15.154:0/1751424383 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f6214024480 con 0x7f6218071290 2023-09-23T03:17:10.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.872+0000 7f620dffb640 1 --2- 172.21.15.154:0/1751424383 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f61fc06af30 0x7f61fc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:10.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.872+0000 7f620dffb640 1 -- 172.21.15.154:0/1751424383 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f6214089ed0 con 0x7f6218071290 2023-09-23T03:17:10.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.873+0000 7f621cd44640 1 --2- 172.21.15.154:0/1751424383 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f61fc06af30 0x7f61fc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:10.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.873+0000 7f621cd44640 1 --2- 172.21.15.154:0/1751424383 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f61fc06af30 0x7f61fc06d3f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6208036000 tx=0x7f620803d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:10.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:10.876+0000 7f620dffb640 1 -- 172.21.15.154:0/1751424383 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f6214054e10 con 0x7f6218071290 2023-09-23T03:17:11.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.085+0000 7f621f7a9640 1 -- 172.21.15.154:0/1751424383 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f621804e920 con 0x7f6218071290 2023-09-23T03:17:11.097 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:17:11.106 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:17:11.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.654+0000 7f620dffb640 1 -- 172.21.15.154:0/1751424383 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f621401f0c0 con 0x7f6218071290 2023-09-23T03:17:11.655 INFO:teuthology.orchestra.run.smithi154.stderr:new fs with metadata pool 5 and data pool 6 2023-09-23T03:17:11.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.657+0000 7f621f7a9640 1 -- 172.21.15.154:0/1751424383 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f61fc06af30 msgr2=0x7f61fc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:11.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.657+0000 7f621f7a9640 1 --2- 172.21.15.154:0/1751424383 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f61fc06af30 0x7f61fc06d3f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6208036000 tx=0x7f620803d040 comp rx=0 tx=0).stop 2023-09-23T03:17:11.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.657+0000 7f621f7a9640 1 -- 172.21.15.154:0/1751424383 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6218071290 msgr2=0x7f621814c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:11.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.657+0000 7f621f7a9640 1 --2- 172.21.15.154:0/1751424383 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6218071290 0x7f621814c290 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f621400c7f0 tx=0x7f621400cbf0 comp rx=0 tx=0).stop 2023-09-23T03:17:11.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.658+0000 7f621f7a9640 1 -- 172.21.15.154:0/1751424383 shutdown_connections 2023-09-23T03:17:11.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.658+0000 7f621f7a9640 1 --2- 172.21.15.154:0/1751424383 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f61fc06af30 0x7f61fc06d3f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:11.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.658+0000 7f621f7a9640 1 --2- 172.21.15.154:0/1751424383 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62180c9150 0x7f6218145420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:11.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.658+0000 7f621f7a9640 1 --2- 172.21.15.154:0/1751424383 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62180c8d70 0x7f62181505e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:11.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.658+0000 7f621f7a9640 1 --2- 172.21.15.154:0/1751424383 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6218071290 0x7f621814c290 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:11.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.658+0000 7f621f7a9640 1 -- 172.21.15.154:0/1751424383 >> 172.21.15.154:0/1751424383 conn(0x7f621806c010 msgr2=0x7f621815b100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:11.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.658+0000 7f621f7a9640 1 -- 172.21.15.154:0/1751424383 shutdown_connections 2023-09-23T03:17:11.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.658+0000 7f621f7a9640 1 -- 172.21.15.154:0/1751424383 wait complete. 2023-09-23T03:17:11.684 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:17:11.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.881+0000 7f2fa3fff640 1 Processor -- start 2023-09-23T03:17:11.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.881+0000 7f2fa3fff640 1 -- start start 2023-09-23T03:17:11.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.883+0000 7f2fa3fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2fa415a7d0 0x7f2fa40cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:11.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.883+0000 7f2fa3fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2fa40d01b0 0x7f2fa40caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:11.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.883+0000 7f2fa3fff640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fa415f240 con 0x7f2fa40cb1e0 2023-09-23T03:17:11.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.883+0000 7f2fa3fff640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fa40cb610 con 0x7f2fa40d01b0 2023-09-23T03:17:11.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.883+0000 7f2fa3fff640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fa415e6c0 con 0x7f2fa415a7d0 2023-09-23T03:17:11.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.883+0000 7f2f9a7fc640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2fa40d01b0 0x7f2fa40caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:11.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.883+0000 7f2fa37fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f2fa40cb1e0 0x7f2fa40b20d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:42270/0 (socket says 172.21.15.154:42270) 2023-09-23T03:17:11.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.883+0000 7f2fa37fe640 1 -- 172.21.15.154:0/2706911087 learned_addr learned my addr 172.21.15.154:0/2706911087 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:17:11.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.884+0000 7f2fa2ffd640 1 --2- 172.21.15.154:0/2706911087 >> v2:172.21.15.190:3300/0 conn(0x7f2fa415a7d0 0x7f2fa40cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:11.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.884+0000 7f2f9a7fc640 1 -- 172.21.15.154:0/2706911087 >> v2:172.21.15.190:3300/0 conn(0x7f2fa415a7d0 msgr2=0x7f2fa40cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:11.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.884+0000 7f2f9a7fc640 1 --2- 172.21.15.154:0/2706911087 >> v2:172.21.15.190:3300/0 conn(0x7f2fa415a7d0 0x7f2fa40cfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:11.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.884+0000 7f2f9a7fc640 1 -- 172.21.15.154:0/2706911087 >> v1:172.21.15.154:6789/0 conn(0x7f2fa40cb1e0 legacy=0x7f2fa40b20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:11.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.884+0000 7f2f9a7fc640 1 -- 172.21.15.154:0/2706911087 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa415e9a0 con 0x7f2fa40d01b0 2023-09-23T03:17:11.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.884+0000 7f2fa27fc640 1 -- 172.21.15.154:0/2706911087 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 340038980 0 0) 0x7f2fa415f240 con 0x7f2fa40cb1e0 2023-09-23T03:17:11.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.884+0000 7f2fa2ffd640 1 --2- 172.21.15.154:0/2706911087 >> v2:172.21.15.190:3300/0 conn(0x7f2fa415a7d0 0x7f2fa40cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:11.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.885+0000 7f2f9a7fc640 1 --2- 172.21.15.154:0/2706911087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa40d01b0 0x7f2fa40caca0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4067080 tx=0x7f2f8c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b103e3c028c04f0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:11.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.885+0000 7f2fa27fc640 1 -- 172.21.15.154:0/2706911087 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c02e070 con 0x7f2fa40d01b0 2023-09-23T03:17:11.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.885+0000 7f2fa27fc640 1 -- 172.21.15.154:0/2706911087 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f8c0315a0 con 0x7f2fa40d01b0 2023-09-23T03:17:11.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.885+0000 7f2fa27fc640 1 -- 172.21.15.154:0/2706911087 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c031930 con 0x7f2fa40d01b0 2023-09-23T03:17:11.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.886+0000 7f2fa3fff640 1 -- 172.21.15.154:0/2706911087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa40d01b0 msgr2=0x7f2fa40caca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:11.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.886+0000 7f2fa3fff640 1 --2- 172.21.15.154:0/2706911087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa40d01b0 0x7f2fa40caca0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4067080 tx=0x7f2f8c02b840 comp rx=0 tx=0).stop 2023-09-23T03:17:11.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.886+0000 7f2fa3fff640 1 -- 172.21.15.154:0/2706911087 shutdown_connections 2023-09-23T03:17:11.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.886+0000 7f2fa3fff640 1 --2- 172.21.15.154:0/2706911087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa40d01b0 0x7f2fa40caca0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:11.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.886+0000 7f2fa3fff640 1 --2- 172.21.15.154:0/2706911087 >> v2:172.21.15.190:3300/0 conn(0x7f2fa415a7d0 0x7f2fa40cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:11.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.886+0000 7f2fa3fff640 1 -- 172.21.15.154:0/2706911087 >> 172.21.15.154:0/2706911087 conn(0x7f2fa406e0d0 msgr2=0x7f2fa406e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:11.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.886+0000 7f2fa3fff640 1 -- 172.21.15.154:0/2706911087 shutdown_connections 2023-09-23T03:17:11.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.886+0000 7f2fa3fff640 1 -- 172.21.15.154:0/2706911087 wait complete. 2023-09-23T03:17:11.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.888+0000 7f2fa3fff640 1 Processor -- start 2023-09-23T03:17:11.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.888+0000 7f2fa3fff640 1 -- start start 2023-09-23T03:17:11.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.889+0000 7f2fa3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa40cb1e0 0x7f2fa40b2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:11.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.889+0000 7f2fa3fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fa40d01b0 0x7f2fa41efde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:11.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.889+0000 7f2fa3fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fa415a7d0 0x7f2fa41f2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:11.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.889+0000 7f2fa3fff640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa40b0150 con 0x7f2fa40cb1e0 2023-09-23T03:17:11.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.889+0000 7f2fa3fff640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa41f0320 con 0x7f2fa415a7d0 2023-09-23T03:17:11.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.889+0000 7f2fa3fff640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa41f3130 con 0x7f2fa40d01b0 2023-09-23T03:17:11.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.889+0000 7f2fa2ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa40cb1e0 0x7f2fa40b2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:11.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.889+0000 7f2fa2ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa40cb1e0 0x7f2fa40b2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47278/0 (socket says 172.21.15.154:47278) 2023-09-23T03:17:11.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.889+0000 7f2fa2ffd640 1 -- 172.21.15.154:0/3772496049 learned_addr learned my addr 172.21.15.154:0/3772496049 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:11.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.889+0000 7f2fa37fe640 1 --2- 172.21.15.154:0/3772496049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fa415a7d0 0x7f2fa41f2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:11.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.889+0000 7f2f9a7fc640 1 --2- 172.21.15.154:0/3772496049 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fa40d01b0 0x7f2fa41efde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:11.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.889+0000 7f2fa2ffd640 1 -- 172.21.15.154:0/3772496049 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fa40d01b0 msgr2=0x7f2fa41efde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:11.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.889+0000 7f2fa2ffd640 1 --2- 172.21.15.154:0/3772496049 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fa40d01b0 0x7f2fa41efde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:11.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.889+0000 7f2fa2ffd640 1 -- 172.21.15.154:0/3772496049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fa415a7d0 msgr2=0x7f2fa41f2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:11.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.890+0000 7f2fa2ffd640 1 --2- 172.21.15.154:0/3772496049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fa415a7d0 0x7f2fa41f2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:11.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.890+0000 7f2fa2ffd640 1 -- 172.21.15.154:0/3772496049 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f8c009400 con 0x7f2fa40cb1e0 2023-09-23T03:17:11.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.890+0000 7f2fa37fe640 1 --2- 172.21.15.154:0/3772496049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fa415a7d0 0x7f2fa41f2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:11.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.890+0000 7f2fa2ffd640 1 --2- 172.21.15.154:0/3772496049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa40cb1e0 0x7f2fa40b2bb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2f9400c7f0 tx=0x7f2f9400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:11.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.890+0000 7f2fa0ff9640 1 -- 172.21.15.154:0/3772496049 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f94007820 con 0x7f2fa40cb1e0 2023-09-23T03:17:11.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.890+0000 7f2fa0ff9640 1 -- 172.21.15.154:0/3772496049 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f94007980 con 0x7f2fa40cb1e0 2023-09-23T03:17:11.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.890+0000 7f2fa0ff9640 1 -- 172.21.15.154:0/3772496049 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f94007cf0 con 0x7f2fa40cb1e0 2023-09-23T03:17:11.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.890+0000 7f2fa3fff640 1 -- 172.21.15.154:0/3772496049 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa40b03c0 con 0x7f2fa40cb1e0 2023-09-23T03:17:11.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.890+0000 7f2fa3fff640 1 -- 172.21.15.154:0/3772496049 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa40b0980 con 0x7f2fa40cb1e0 2023-09-23T03:17:11.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.892+0000 7f2fa3fff640 1 -- 172.21.15.154:0/3772496049 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fa404e920 con 0x7f2fa40cb1e0 2023-09-23T03:17:11.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.892+0000 7f2fa0ff9640 1 -- 172.21.15.154:0/3772496049 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f2f940113f0 con 0x7f2fa40cb1e0 2023-09-23T03:17:11.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.893+0000 7f2fa0ff9640 1 --2- 172.21.15.154:0/3772496049 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2f8406af60 0x7f2f8406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:11.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.894+0000 7f2fa0ff9640 1 -- 172.21.15.154:0/3772496049 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2f94089af0 con 0x7f2fa40cb1e0 2023-09-23T03:17:11.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.894+0000 7f2f9a7fc640 1 --2- 172.21.15.154:0/3772496049 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2f8406af60 0x7f2f8406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:11.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.894+0000 7f2f9a7fc640 1 --2- 172.21.15.154:0/3772496049 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2f8406af60 0x7f2f8406d420 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c036000 tx=0x7f2f8c03d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:11.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:11.898+0000 7f2fa0ff9640 1 -- 172.21.15.154:0/3772496049 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f2f94054a80 con 0x7f2fa40cb1e0 2023-09-23T03:17:12.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.088+0000 7f2fa3fff640 1 -- 172.21.15.154:0/3772496049 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2fa40cafc0 con 0x7f2fa40cb1e0 2023-09-23T03:17:12.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.090+0000 7f2fa0ff9640 1 -- 172.21.15.154:0/3772496049 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+16823 (secure 0 0 0) 0x7f2f9401f090 con 0x7f2fa40cb1e0 2023-09-23T03:17:12.091 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:12.092 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":37,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:17:11.088176+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:17:08.933690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-09-23T03:17:10.141711+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:17:12.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.092+0000 7f2fa3fff640 1 -- 172.21.15.154:0/3772496049 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2f8406af60 msgr2=0x7f2f8406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:12.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.092+0000 7f2fa3fff640 1 --2- 172.21.15.154:0/3772496049 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2f8406af60 0x7f2f8406d420 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c036000 tx=0x7f2f8c03d040 comp rx=0 tx=0).stop 2023-09-23T03:17:12.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.092+0000 7f2fa3fff640 1 -- 172.21.15.154:0/3772496049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa40cb1e0 msgr2=0x7f2fa40b2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:12.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.092+0000 7f2fa3fff640 1 --2- 172.21.15.154:0/3772496049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa40cb1e0 0x7f2fa40b2bb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2f9400c7f0 tx=0x7f2f9400ccb0 comp rx=0 tx=0).stop 2023-09-23T03:17:12.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.093+0000 7f2fa3fff640 1 -- 172.21.15.154:0/3772496049 shutdown_connections 2023-09-23T03:17:12.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.093+0000 7f2fa3fff640 1 --2- 172.21.15.154:0/3772496049 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2f8406af60 0x7f2f8406d420 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:12.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.093+0000 7f2fa3fff640 1 --2- 172.21.15.154:0/3772496049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2fa415a7d0 0x7f2fa41f2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:12.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.093+0000 7f2fa3fff640 1 --2- 172.21.15.154:0/3772496049 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2fa40d01b0 0x7f2fa41efde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:12.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.093+0000 7f2fa3fff640 1 --2- 172.21.15.154:0/3772496049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa40cb1e0 0x7f2fa40b2bb0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:12.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.093+0000 7f2fa3fff640 1 -- 172.21.15.154:0/3772496049 >> 172.21.15.154:0/3772496049 conn(0x7f2fa406e0d0 msgr2=0x7f2fa415ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:12.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.093+0000 7f2fa3fff640 1 -- 172.21.15.154:0/3772496049 shutdown_connections 2023-09-23T03:17:12.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.093+0000 7f2fa3fff640 1 -- 172.21.15.154:0/3772496049 wait complete. 2023-09-23T03:17:12.113 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:17:12.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.324+0000 7fcaf9955640 1 Processor -- start 2023-09-23T03:17:12.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.324+0000 7fcaf9955640 1 -- start start 2023-09-23T03:17:12.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.325+0000 7fcaf9955640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaf40ada30 0x7fcaf40ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:12.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.325+0000 7fcaf9955640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcaf40cf130 0x7fcaf415d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:12.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.325+0000 7fcaf9955640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf406ceb0 con 0x7fcaf40b2fe0 2023-09-23T03:17:12.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.325+0000 7fcaf9955640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaf40ade10 con 0x7fcaf40cf130 2023-09-23T03:17:12.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.325+0000 7fcaf9955640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf4159c90 con 0x7fcaf40ada30 2023-09-23T03:17:12.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.326+0000 7fcaf2ffd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaf40ada30 0x7fcaf40ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:12.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.326+0000 7fcaf2ffd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaf40ada30 0x7fcaf40ccaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:39286/0 (socket says 172.21.15.154:39286) 2023-09-23T03:17:12.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.326+0000 7fcaf2ffd640 1 -- 172.21.15.154:0/1116797056 learned_addr learned my addr 172.21.15.154:0/1116797056 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:12.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.326+0000 7fcaf37fe640 1 --2- 172.21.15.154:0/1116797056 >> v2:172.21.15.190:3300/0 conn(0x7fcaf40cf130 0x7fcaf415d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:12.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.327+0000 7fcaf1ffb640 1 -- 172.21.15.154:0/1116797056 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2989277070 0 0) 0x7fcae8001110 con 0x7fcaf40b2fe0 2023-09-23T03:17:12.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.327+0000 7fcaf1ffb640 1 -- 172.21.15.154:0/1116797056 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcadc003520 con 0x7fcaf40b2fe0 2023-09-23T03:17:12.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.327+0000 7fcaf2ffd640 1 -- 172.21.15.154:0/1116797056 >> v2:172.21.15.190:3300/0 conn(0x7fcaf40cf130 msgr2=0x7fcaf415d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:12.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.327+0000 7fcaf2ffd640 1 --2- 172.21.15.154:0/1116797056 >> v2:172.21.15.190:3300/0 conn(0x7fcaf40cf130 0x7fcaf415d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:12.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.327+0000 7fcaf2ffd640 1 -- 172.21.15.154:0/1116797056 >> v1:172.21.15.190:6789/0 conn(0x7fcaf40b2fe0 legacy=0x7fcaf40cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:12.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.327+0000 7fcaf2ffd640 1 -- 172.21.15.154:0/1116797056 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf406c110 con 0x7fcaf40ada30 2023-09-23T03:17:12.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.329+0000 7fcaf2ffd640 1 --2- 172.21.15.154:0/1116797056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaf40ada30 0x7fcaf40ccaf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcae400baf0 tx=0x7fcae402e130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80afec3f38c0d6c1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:12.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.329+0000 7fcaf1ffb640 1 -- 172.21.15.154:0/1116797056 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae4004030 con 0x7fcaf40ada30 2023-09-23T03:17:12.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.329+0000 7fcaf1ffb640 1 -- 172.21.15.154:0/1116797056 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae4004190 con 0x7fcaf40ada30 2023-09-23T03:17:12.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.329+0000 7fcaf1ffb640 1 -- 172.21.15.154:0/1116797056 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae4003610 con 0x7fcaf40ada30 2023-09-23T03:17:12.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.330+0000 7fcaf9955640 1 -- 172.21.15.154:0/1116797056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaf40ada30 msgr2=0x7fcaf40ccaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:12.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.330+0000 7fcaf9955640 1 --2- 172.21.15.154:0/1116797056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaf40ada30 0x7fcaf40ccaf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcae400baf0 tx=0x7fcae402e130 comp rx=0 tx=0).stop 2023-09-23T03:17:12.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.330+0000 7fcaf9955640 1 -- 172.21.15.154:0/1116797056 shutdown_connections 2023-09-23T03:17:12.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.330+0000 7fcaf9955640 1 --2- 172.21.15.154:0/1116797056 >> v2:172.21.15.190:3300/0 conn(0x7fcaf40cf130 0x7fcaf415d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:12.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.330+0000 7fcaf9955640 1 --2- 172.21.15.154:0/1116797056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaf40ada30 0x7fcaf40ccaf0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:12.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.330+0000 7fcaf9955640 1 -- 172.21.15.154:0/1116797056 >> 172.21.15.154:0/1116797056 conn(0x7fcaf406a0f0 msgr2=0x7fcaf406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:12.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.330+0000 7fcaf9955640 1 -- 172.21.15.154:0/1116797056 shutdown_connections 2023-09-23T03:17:12.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.330+0000 7fcaf9955640 1 -- 172.21.15.154:0/1116797056 wait complete. 2023-09-23T03:17:12.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.331+0000 7fcaf9955640 1 Processor -- start 2023-09-23T03:17:12.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.332+0000 7fcaf9955640 1 -- start start 2023-09-23T03:17:12.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.333+0000 7fcaf9955640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaf40ada30 0x7fcaf40c4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:12.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.333+0000 7fcaf9955640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcaf40b2fe0 0x7fcaf4159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:12.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.333+0000 7fcaf9955640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaf40cf130 0x7fcaf415a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:12.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.333+0000 7fcaf9955640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf40ae810 con 0x7fcaf40ada30 2023-09-23T03:17:12.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.333+0000 7fcaf9955640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf40c4db0 con 0x7fcaf40b2fe0 2023-09-23T03:17:12.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.333+0000 7fcaf9955640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf415cd50 con 0x7fcaf40cf130 2023-09-23T03:17:12.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.333+0000 7fcaf2ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaf40ada30 0x7fcaf40c4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:12.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.333+0000 7fcaf2ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaf40ada30 0x7fcaf40c4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47288/0 (socket says 172.21.15.154:47288) 2023-09-23T03:17:12.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.333+0000 7fcaf2ffd640 1 -- 172.21.15.154:0/2128724814 learned_addr learned my addr 172.21.15.154:0/2128724814 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:12.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.333+0000 7fcaf27fc640 1 --2- 172.21.15.154:0/2128724814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcaf40b2fe0 0x7fcaf4159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:12.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.333+0000 7fcaf37fe640 1 --2- 172.21.15.154:0/2128724814 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaf40cf130 0x7fcaf415a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:12.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.333+0000 7fcaf2ffd640 1 -- 172.21.15.154:0/2128724814 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaf40cf130 msgr2=0x7fcaf415a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:12.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.333+0000 7fcaf2ffd640 1 --2- 172.21.15.154:0/2128724814 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaf40cf130 0x7fcaf415a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:12.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.333+0000 7fcaf2ffd640 1 -- 172.21.15.154:0/2128724814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcaf40b2fe0 msgr2=0x7fcaf4159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:12.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.333+0000 7fcaf2ffd640 1 --2- 172.21.15.154:0/2128724814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcaf40b2fe0 0x7fcaf4159e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:12.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.333+0000 7fcaf2ffd640 1 -- 172.21.15.154:0/2128724814 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae400b700 con 0x7fcaf40ada30 2023-09-23T03:17:12.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.333+0000 7fcaf37fe640 1 --2- 172.21.15.154:0/2128724814 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaf40cf130 0x7fcaf415a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:12.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.334+0000 7fcaf2ffd640 1 --2- 172.21.15.154:0/2128724814 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaf40ada30 0x7fcaf40c4870 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fcae4038040 tx=0x7fcae40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:12.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.334+0000 7fcadbfff640 1 -- 172.21.15.154:0/2128724814 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae4044070 con 0x7fcaf40ada30 2023-09-23T03:17:12.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.334+0000 7fcadbfff640 1 -- 172.21.15.154:0/2128724814 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae4004510 con 0x7fcaf40ada30 2023-09-23T03:17:12.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.334+0000 7fcadbfff640 1 -- 172.21.15.154:0/2128724814 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae4003870 con 0x7fcaf40ada30 2023-09-23T03:17:12.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.334+0000 7fcaf9955640 1 -- 172.21.15.154:0/2128724814 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf415cab0 con 0x7fcaf40ada30 2023-09-23T03:17:12.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.334+0000 7fcaf9955640 1 -- 172.21.15.154:0/2128724814 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf40c22b0 con 0x7fcaf40ada30 2023-09-23T03:17:12.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.336+0000 7fcaf9955640 1 -- 172.21.15.154:0/2128724814 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaf4073eb0 con 0x7fcaf40ada30 2023-09-23T03:17:12.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.336+0000 7fcadbfff640 1 -- 172.21.15.154:0/2128724814 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fcae40039d0 con 0x7fcaf40ada30 2023-09-23T03:17:12.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.337+0000 7fcadbfff640 1 --2- 172.21.15.154:0/2128724814 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcae006af30 0x7fcae006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:12.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.337+0000 7fcadbfff640 1 -- 172.21.15.154:0/2128724814 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fcae40abb00 con 0x7fcaf40ada30 2023-09-23T03:17:12.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.338+0000 7fcaf27fc640 1 --2- 172.21.15.154:0/2128724814 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcae006af30 0x7fcae006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:12.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.338+0000 7fcaf27fc640 1 --2- 172.21.15.154:0/2128724814 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcae006af30 0x7fcae006d3f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcaf40b26e0 tx=0x7fcae8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:12.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.342+0000 7fcadbfff640 1 -- 172.21.15.154:0/2128724814 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fcae4076a90 con 0x7fcaf40ada30 2023-09-23T03:17:12.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.536+0000 7fcaf9955640 1 -- 172.21.15.154:0/2128724814 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcaf406dc60 con 0x7fcaf40ada30 2023-09-23T03:17:12.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.538+0000 7fcadbfff640 1 -- 172.21.15.154:0/2128724814 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16810 (secure 0 0 0) 0x7fcae407a0b0 con 0x7fcaf40ada30 2023-09-23T03:17:12.540 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:12.540 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":38,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:17:12.168442+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:17:08.933690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-09-23T03:17:10.141711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:17:12.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.541+0000 7fcaf9955640 1 -- 172.21.15.154:0/2128724814 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcae006af30 msgr2=0x7fcae006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:12.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.541+0000 7fcaf9955640 1 --2- 172.21.15.154:0/2128724814 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcae006af30 0x7fcae006d3f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcaf40b26e0 tx=0x7fcae8009040 comp rx=0 tx=0).stop 2023-09-23T03:17:12.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.541+0000 7fcaf9955640 1 -- 172.21.15.154:0/2128724814 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaf40ada30 msgr2=0x7fcaf40c4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:12.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.541+0000 7fcaf9955640 1 --2- 172.21.15.154:0/2128724814 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaf40ada30 0x7fcaf40c4870 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fcae4038040 tx=0x7fcae40046c0 comp rx=0 tx=0).stop 2023-09-23T03:17:12.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.541+0000 7fcaf9955640 1 -- 172.21.15.154:0/2128724814 shutdown_connections 2023-09-23T03:17:12.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.541+0000 7fcaf9955640 1 --2- 172.21.15.154:0/2128724814 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcae006af30 0x7fcae006d3f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:12.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.541+0000 7fcaf9955640 1 --2- 172.21.15.154:0/2128724814 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaf40cf130 0x7fcaf415a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:12.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.541+0000 7fcaf9955640 1 --2- 172.21.15.154:0/2128724814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcaf40b2fe0 0x7fcaf4159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:12.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.541+0000 7fcaf9955640 1 --2- 172.21.15.154:0/2128724814 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaf40ada30 0x7fcaf40c4870 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:12.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.541+0000 7fcaf9955640 1 -- 172.21.15.154:0/2128724814 >> 172.21.15.154:0/2128724814 conn(0x7fcaf406a0f0 msgr2=0x7fcaf40adf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:12.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.541+0000 7fcaf9955640 1 -- 172.21.15.154:0/2128724814 shutdown_connections 2023-09-23T03:17:12.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.541+0000 7fcaf9955640 1 -- 172.21.15.154:0/2128724814 wait complete. 2023-09-23T03:17:12.561 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-09-23T03:17:12.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.780+0000 7ff1b0fb6640 1 Processor -- start 2023-09-23T03:17:12.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.780+0000 7ff1b0fb6640 1 -- start start 2023-09-23T03:17:12.782 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.781+0000 7ff1b0fb6640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff1ac15cc40 0x7ff1ac0afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:12.782 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.781+0000 7ff1b0fb6640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff1ac06b6d0 0x7ff1ac06bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:12.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.781+0000 7ff1b0fb6640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1ac06b440 con 0x7ff1ac1607a0 2023-09-23T03:17:12.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.781+0000 7ff1b0fb6640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1ac06c0a0 con 0x7ff1ac15cc40 2023-09-23T03:17:12.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.781+0000 7ff1b0fb6640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1ac0cd000 con 0x7ff1ac06b6d0 2023-09-23T03:17:12.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.782+0000 7ff1aad9d640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff1ac1607a0 0x7ff1ac160c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:42278/0 (socket says 172.21.15.154:42278) 2023-09-23T03:17:12.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.782+0000 7ff1aad9d640 1 -- 172.21.15.154:0/1296678485 learned_addr learned my addr 172.21.15.154:0/1296678485 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:17:12.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.782+0000 7ff1aa59c640 1 --2- 172.21.15.154:0/1296678485 >> v2:172.21.15.154:3300/0 conn(0x7ff1ac15cc40 0x7ff1ac0afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:12.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.782+0000 7ff1a9d9b640 1 --2- 172.21.15.154:0/1296678485 >> v2:172.21.15.190:3300/0 conn(0x7ff1ac06b6d0 0x7ff1ac06bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:12.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.782+0000 7ff1aa59c640 1 -- 172.21.15.154:0/1296678485 >> v2:172.21.15.190:3300/0 conn(0x7ff1ac06b6d0 msgr2=0x7ff1ac06bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:12.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.782+0000 7ff1aa59c640 1 --2- 172.21.15.154:0/1296678485 >> v2:172.21.15.190:3300/0 conn(0x7ff1ac06b6d0 0x7ff1ac06bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:12.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.782+0000 7ff1aa59c640 1 -- 172.21.15.154:0/1296678485 >> v1:172.21.15.154:6789/0 conn(0x7ff1ac1607a0 legacy=0x7ff1ac160c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:12.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.782+0000 7ff1aa59c640 1 -- 172.21.15.154:0/1296678485 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ac0cd2e0 con 0x7ff1ac15cc40 2023-09-23T03:17:12.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.782+0000 7ff1a959a640 1 -- 172.21.15.154:0/1296678485 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2295866322 0 0) 0x7ff1ac06b440 con 0x7ff1ac1607a0 2023-09-23T03:17:12.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.782+0000 7ff1a9d9b640 1 --2- 172.21.15.154:0/1296678485 >> v2:172.21.15.190:3300/0 conn(0x7ff1ac06b6d0 0x7ff1ac06bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:12.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.783+0000 7ff1aa59c640 1 --2- 172.21.15.154:0/1296678485 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ac15cc40 0x7ff1ac0afcb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff19c009790 tx=0x7ff19c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=705495edc12ddc44 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:12.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.784+0000 7ff1a959a640 1 -- 172.21.15.154:0/1296678485 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c031780 con 0x7ff1ac15cc40 2023-09-23T03:17:12.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.784+0000 7ff1a959a640 1 -- 172.21.15.154:0/1296678485 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff19c0318e0 con 0x7ff1ac15cc40 2023-09-23T03:17:12.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.784+0000 7ff1a959a640 1 -- 172.21.15.154:0/1296678485 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19c031c70 con 0x7ff1ac15cc40 2023-09-23T03:17:12.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.784+0000 7ff1b0fb6640 1 -- 172.21.15.154:0/1296678485 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ac15cc40 msgr2=0x7ff1ac0afcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:12.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.784+0000 7ff1b0fb6640 1 --2- 172.21.15.154:0/1296678485 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ac15cc40 0x7ff1ac0afcb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff19c009790 tx=0x7ff19c02b820 comp rx=0 tx=0).stop 2023-09-23T03:17:12.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.784+0000 7ff1b0fb6640 1 -- 172.21.15.154:0/1296678485 shutdown_connections 2023-09-23T03:17:12.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.784+0000 7ff1b0fb6640 1 --2- 172.21.15.154:0/1296678485 >> v2:172.21.15.190:3300/0 conn(0x7ff1ac06b6d0 0x7ff1ac06bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:12.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.784+0000 7ff1b0fb6640 1 --2- 172.21.15.154:0/1296678485 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ac15cc40 0x7ff1ac0afcb0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:12.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.784+0000 7ff1b0fb6640 1 -- 172.21.15.154:0/1296678485 >> 172.21.15.154:0/1296678485 conn(0x7ff1ac089fd0 msgr2=0x7ff1ac08a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:12.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.784+0000 7ff1b0fb6640 1 -- 172.21.15.154:0/1296678485 shutdown_connections 2023-09-23T03:17:12.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.784+0000 7ff1b0fb6640 1 -- 172.21.15.154:0/1296678485 wait complete. 2023-09-23T03:17:12.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.786+0000 7ff1b0fb6640 1 Processor -- start 2023-09-23T03:17:12.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.786+0000 7ff1b0fb6640 1 -- start start 2023-09-23T03:17:12.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.787+0000 7ff1b0fb6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1ac06b6d0 0x7ff1ac0b4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:12.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.787+0000 7ff1b0fb6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ac15cc40 0x7ff1ac159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:12.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.787+0000 7ff1b0fb6640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1ac1607a0 0x7ff1ac0c61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:12.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.787+0000 7ff1b0fb6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ac1f27f0 con 0x7ff1ac15cc40 2023-09-23T03:17:12.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.787+0000 7ff1b0fb6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ac0c7000 con 0x7ff1ac06b6d0 2023-09-23T03:17:12.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.787+0000 7ff1b0fb6640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ac15eb40 con 0x7ff1ac1607a0 2023-09-23T03:17:12.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.787+0000 7ff1a9d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ac15cc40 0x7ff1ac159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:12.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.788+0000 7ff1a9d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ac15cc40 0x7ff1ac159bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47308/0 (socket says 172.21.15.154:47308) 2023-09-23T03:17:12.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.788+0000 7ff1a9d9b640 1 -- 172.21.15.154:0/1853961402 learned_addr learned my addr 172.21.15.154:0/1853961402 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:12.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.788+0000 7ff1a9d9b640 1 -- 172.21.15.154:0/1853961402 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1ac1607a0 msgr2=0x7ff1ac0c61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:12.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.788+0000 7ff1a9d9b640 1 --2- 172.21.15.154:0/1853961402 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1ac1607a0 0x7ff1ac0c61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:12.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.788+0000 7ff1a9d9b640 1 -- 172.21.15.154:0/1853961402 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1ac06b6d0 msgr2=0x7ff1ac0b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:12.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.788+0000 7ff1a9d9b640 1 --2- 172.21.15.154:0/1853961402 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1ac06b6d0 0x7ff1ac0b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:12.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.788+0000 7ff1a9d9b640 1 -- 172.21.15.154:0/1853961402 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19c0093a0 con 0x7ff1ac15cc40 2023-09-23T03:17:12.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.788+0000 7ff1a9d9b640 1 --2- 172.21.15.154:0/1853961402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ac15cc40 0x7ff1ac159bd0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff19400c7f0 tx=0x7ff19400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:12.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.789+0000 7ff1a2ffd640 1 -- 172.21.15.154:0/1853961402 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff194007820 con 0x7ff1ac15cc40 2023-09-23T03:17:12.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.789+0000 7ff1a2ffd640 1 -- 172.21.15.154:0/1853961402 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff194007980 con 0x7ff1ac15cc40 2023-09-23T03:17:12.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.789+0000 7ff1a2ffd640 1 -- 172.21.15.154:0/1853961402 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff194007cf0 con 0x7ff1ac15cc40 2023-09-23T03:17:12.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.789+0000 7ff1b0fb6640 1 -- 172.21.15.154:0/1853961402 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1ac15a110 con 0x7ff1ac15cc40 2023-09-23T03:17:12.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.789+0000 7ff1b0fb6640 1 -- 172.21.15.154:0/1853961402 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1ac0c6d20 con 0x7ff1ac15cc40 2023-09-23T03:17:12.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.790+0000 7ff1b0fb6640 1 -- 172.21.15.154:0/1853961402 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1ac04e920 con 0x7ff1ac15cc40 2023-09-23T03:17:12.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.791+0000 7ff1a2ffd640 1 -- 172.21.15.154:0/1853961402 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7ff1940113f0 con 0x7ff1ac15cc40 2023-09-23T03:17:12.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.792+0000 7ff1a2ffd640 1 --2- 172.21.15.154:0/1853961402 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff18c06af60 0x7ff18c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:12.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.792+0000 7ff1a2ffd640 1 -- 172.21.15.154:0/1853961402 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff194089b30 con 0x7ff1ac15cc40 2023-09-23T03:17:12.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.792+0000 7ff1aa59c640 1 --2- 172.21.15.154:0/1853961402 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff18c06af60 0x7ff18c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:12.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.793+0000 7ff1aa59c640 1 --2- 172.21.15.154:0/1853961402 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff18c06af60 0x7ff18c06d420 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff19c036000 tx=0x7ff19c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:12.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:12.796+0000 7ff1a2ffd640 1 -- 172.21.15.154:0/1853961402 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7ff194054ac0 con 0x7ff1ac15cc40 2023-09-23T03:17:13.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.026+0000 7ff1b0fb6640 1 -- 172.21.15.154:0/1853961402 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff1ac15ec80 con 0x7ff1ac15cc40 2023-09-23T03:17:13.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.685+0000 7ff1a2ffd640 1 -- 172.21.15.154:0/1853961402 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff19401f0b0 con 0x7ff1ac15cc40 2023-09-23T03:17:13.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.689+0000 7ff1b0fb6640 1 -- 172.21.15.154:0/1853961402 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff18c06af60 msgr2=0x7ff18c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:13.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.689+0000 7ff1b0fb6640 1 --2- 172.21.15.154:0/1853961402 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff18c06af60 0x7ff18c06d420 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff19c036000 tx=0x7ff19c030000 comp rx=0 tx=0).stop 2023-09-23T03:17:13.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.689+0000 7ff1b0fb6640 1 -- 172.21.15.154:0/1853961402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ac15cc40 msgr2=0x7ff1ac159bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:13.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.689+0000 7ff1b0fb6640 1 --2- 172.21.15.154:0/1853961402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ac15cc40 0x7ff1ac159bd0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff19400c7f0 tx=0x7ff19400ccb0 comp rx=0 tx=0).stop 2023-09-23T03:17:13.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.689+0000 7ff1b0fb6640 1 -- 172.21.15.154:0/1853961402 shutdown_connections 2023-09-23T03:17:13.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.689+0000 7ff1b0fb6640 1 --2- 172.21.15.154:0/1853961402 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff18c06af60 0x7ff18c06d420 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:13.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.690+0000 7ff1b0fb6640 1 --2- 172.21.15.154:0/1853961402 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1ac1607a0 0x7ff1ac0c61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:13.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.690+0000 7ff1b0fb6640 1 --2- 172.21.15.154:0/1853961402 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1ac15cc40 0x7ff1ac159bd0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:13.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.690+0000 7ff1b0fb6640 1 --2- 172.21.15.154:0/1853961402 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1ac06b6d0 0x7ff1ac0b4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:13.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.690+0000 7ff1b0fb6640 1 -- 172.21.15.154:0/1853961402 >> 172.21.15.154:0/1853961402 conn(0x7ff1ac089fd0 msgr2=0x7ff1ac0af9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:13.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.690+0000 7ff1b0fb6640 1 -- 172.21.15.154:0/1853961402 shutdown_connections 2023-09-23T03:17:13.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.690+0000 7ff1b0fb6640 1 -- 172.21.15.154:0/1853961402 wait complete. 2023-09-23T03:17:13.712 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-09-23T03:17:13.712 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-09-23T03:17:13.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.934+0000 7f4fc3fff640 1 Processor -- start 2023-09-23T03:17:13.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.934+0000 7f4fc3fff640 1 -- start start 2023-09-23T03:17:13.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.936+0000 7f4fc3fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4fc4158480 0x7f4fc40cade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:13.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.936+0000 7f4fc3fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4fc40705a0 0x7f4fc40cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:13.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.936+0000 7f4fc3fff640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fc40cb6c0 con 0x7f4fc415ba50 2023-09-23T03:17:13.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.936+0000 7f4fc3fff640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fc415ad80 con 0x7f4fc4158480 2023-09-23T03:17:13.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.936+0000 7f4fc3fff640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fc415af70 con 0x7f4fc40705a0 2023-09-23T03:17:13.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.936+0000 7f4fc2ffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4fc4158480 0x7f4fc40cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:13.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.936+0000 7f4fbbfff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f4fc415ba50 0x7f4fc406fe90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40694/0 (socket says 172.21.15.154:40694) 2023-09-23T03:17:13.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.936+0000 7f4fbbfff640 1 -- 172.21.15.154:0/1173930897 learned_addr learned my addr 172.21.15.154:0/1173930897 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:17:13.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.936+0000 7f4fc37fe640 1 --2- 172.21.15.154:0/1173930897 >> v2:172.21.15.190:3300/0 conn(0x7f4fc40705a0 0x7f4fc40cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:13.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.937+0000 7f4fc2ffd640 1 -- 172.21.15.154:0/1173930897 >> v2:172.21.15.190:3300/0 conn(0x7f4fc40705a0 msgr2=0x7f4fc40cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:13.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.937+0000 7f4fc2ffd640 1 --2- 172.21.15.154:0/1173930897 >> v2:172.21.15.190:3300/0 conn(0x7f4fc40705a0 0x7f4fc40cc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:13.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.937+0000 7f4fc2ffd640 1 -- 172.21.15.154:0/1173930897 >> v1:172.21.15.154:6789/0 conn(0x7f4fc415ba50 legacy=0x7f4fc406fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:13.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.937+0000 7f4fc2ffd640 1 -- 172.21.15.154:0/1173930897 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc415b250 con 0x7f4fc4158480 2023-09-23T03:17:13.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.937+0000 7f4fc37fe640 1 --2- 172.21.15.154:0/1173930897 >> v2:172.21.15.190:3300/0 conn(0x7f4fc40705a0 0x7f4fc40cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:13.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.937+0000 7f4fc27fc640 1 -- 172.21.15.154:0/1173930897 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2463462915 0 0) 0x7f4fc40cb6c0 con 0x7f4fc415ba50 2023-09-23T03:17:13.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.938+0000 7f4fc2ffd640 1 --2- 172.21.15.154:0/1173930897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fc4158480 0x7f4fc40cade0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4fc406f9c0 tx=0x7f4fb402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deeaab5f858b1ee6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:13.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.938+0000 7f4fc27fc640 1 -- 172.21.15.154:0/1173930897 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4031830 con 0x7f4fc4158480 2023-09-23T03:17:13.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.938+0000 7f4fc27fc640 1 -- 172.21.15.154:0/1173930897 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb4031990 con 0x7f4fc4158480 2023-09-23T03:17:13.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.938+0000 7f4fc27fc640 1 -- 172.21.15.154:0/1173930897 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4031d20 con 0x7f4fc4158480 2023-09-23T03:17:13.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.939+0000 7f4fc3fff640 1 -- 172.21.15.154:0/1173930897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fc4158480 msgr2=0x7f4fc40cade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:13.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.939+0000 7f4fc3fff640 1 --2- 172.21.15.154:0/1173930897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fc4158480 0x7f4fc40cade0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4fc406f9c0 tx=0x7f4fb402b8d0 comp rx=0 tx=0).stop 2023-09-23T03:17:13.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.939+0000 7f4fc3fff640 1 -- 172.21.15.154:0/1173930897 shutdown_connections 2023-09-23T03:17:13.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.939+0000 7f4fc3fff640 1 --2- 172.21.15.154:0/1173930897 >> v2:172.21.15.190:3300/0 conn(0x7f4fc40705a0 0x7f4fc40cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:13.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.939+0000 7f4fc3fff640 1 --2- 172.21.15.154:0/1173930897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fc4158480 0x7f4fc40cade0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:13.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.939+0000 7f4fc3fff640 1 -- 172.21.15.154:0/1173930897 >> 172.21.15.154:0/1173930897 conn(0x7f4fc40aff20 msgr2=0x7f4fc40b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:13.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.939+0000 7f4fc3fff640 1 -- 172.21.15.154:0/1173930897 shutdown_connections 2023-09-23T03:17:13.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.939+0000 7f4fc3fff640 1 -- 172.21.15.154:0/1173930897 wait complete. 2023-09-23T03:17:13.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.941+0000 7f4fc3fff640 1 Processor -- start 2023-09-23T03:17:13.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.941+0000 7f4fc3fff640 1 -- start start 2023-09-23T03:17:13.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.942+0000 7f4fc3fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fc40705a0 0x7f4fc40ad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:13.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.943+0000 7f4fc3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fc4158480 0x7f4fc4143810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:13.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.943+0000 7f4fc3fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fc415ba50 0x7f4fc414a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:13.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.943+0000 7f4fc3fff640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc40b3450 con 0x7f4fc4158480 2023-09-23T03:17:13.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.943+0000 7f4fc3fff640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc40b3730 con 0x7f4fc415ba50 2023-09-23T03:17:13.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.943+0000 7f4fc3fff640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc40b3a10 con 0x7f4fc40705a0 2023-09-23T03:17:13.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.943+0000 7f4fbbfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fc4158480 0x7f4fc4143810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:13.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.943+0000 7f4fbbfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fc4158480 0x7f4fc4143810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46384/0 (socket says 172.21.15.154:46384) 2023-09-23T03:17:13.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.943+0000 7f4fbbfff640 1 -- 172.21.15.154:0/2375838786 learned_addr learned my addr 172.21.15.154:0/2375838786 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:13.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.943+0000 7f4fc2ffd640 1 --2- 172.21.15.154:0/2375838786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fc40705a0 0x7f4fc40ad6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:13.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.943+0000 7f4fbbfff640 1 -- 172.21.15.154:0/2375838786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fc40705a0 msgr2=0x7f4fc40ad6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:13.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.943+0000 7f4fc37fe640 1 --2- 172.21.15.154:0/2375838786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fc415ba50 0x7f4fc414a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:13.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.943+0000 7f4fbbfff640 1 --2- 172.21.15.154:0/2375838786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fc40705a0 0x7f4fc40ad6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:13.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.943+0000 7f4fbbfff640 1 -- 172.21.15.154:0/2375838786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fc415ba50 msgr2=0x7f4fc414a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:13.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.943+0000 7f4fbbfff640 1 --2- 172.21.15.154:0/2375838786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fc415ba50 0x7f4fc414a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:13.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.943+0000 7f4fbbfff640 1 -- 172.21.15.154:0/2375838786 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb4009400 con 0x7f4fc4158480 2023-09-23T03:17:13.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.943+0000 7f4fc37fe640 1 --2- 172.21.15.154:0/2375838786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fc415ba50 0x7f4fc414a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:13.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.944+0000 7f4fbbfff640 1 --2- 172.21.15.154:0/2375838786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fc4158480 0x7f4fc4143810 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4fac00c430 tx=0x7f4fac00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:13.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.944+0000 7f4fbb7fe640 1 -- 172.21.15.154:0/2375838786 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fac0074e0 con 0x7f4fc4158480 2023-09-23T03:17:13.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.944+0000 7f4fbb7fe640 1 -- 172.21.15.154:0/2375838786 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fac007660 con 0x7f4fc4158480 2023-09-23T03:17:13.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.944+0000 7f4fbb7fe640 1 -- 172.21.15.154:0/2375838786 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fac004a20 con 0x7f4fc4158480 2023-09-23T03:17:13.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.944+0000 7f4fc3fff640 1 -- 172.21.15.154:0/2375838786 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc40c8a20 con 0x7f4fc4158480 2023-09-23T03:17:13.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.944+0000 7f4fc3fff640 1 -- 172.21.15.154:0/2375838786 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc40adc00 con 0x7f4fc4158480 2023-09-23T03:17:13.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.945+0000 7f4fc3fff640 1 -- 172.21.15.154:0/2375838786 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc404e920 con 0x7f4fc4158480 2023-09-23T03:17:13.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.946+0000 7f4fbb7fe640 1 -- 172.21.15.154:0/2375838786 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f4fac004b80 con 0x7f4fc4158480 2023-09-23T03:17:13.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.947+0000 7f4fbb7fe640 1 --2- 172.21.15.154:0/2375838786 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4fa406af30 0x7f4fa406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:13.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.947+0000 7f4fbb7fe640 1 -- 172.21.15.154:0/2375838786 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4fac0896f0 con 0x7f4fc4158480 2023-09-23T03:17:13.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.947+0000 7f4fc2ffd640 1 --2- 172.21.15.154:0/2375838786 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4fa406af30 0x7f4fa406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:13.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.948+0000 7f4fc2ffd640 1 --2- 172.21.15.154:0/2375838786 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4fa406af30 0x7f4fa406d3f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4036000 tx=0x7f4fb403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:13.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:13.952+0000 7f4fbb7fe640 1 -- 172.21.15.154:0/2375838786 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f4fac0545e0 con 0x7f4fc4158480 2023-09-23T03:17:14.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.179+0000 7f4fc3fff640 1 -- 172.21.15.154:0/2375838786 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4fc40c8f40 con 0x7f4fc4158480 2023-09-23T03:17:14.617 INFO:tasks.ceph.mds.d.smithi190.stderr:2023-09-23T03:17:14.616+0000 7f4ba2e71640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-23T03:17:14.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.696+0000 7f4fbb7fe640 1 -- 172.21.15.154:0/2375838786 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4fac087e60 con 0x7f4fc4158480 2023-09-23T03:17:14.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.700+0000 7f4fc3fff640 1 -- 172.21.15.154:0/2375838786 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4fa406af30 msgr2=0x7f4fa406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:14.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.700+0000 7f4fc3fff640 1 --2- 172.21.15.154:0/2375838786 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4fa406af30 0x7f4fa406d3f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4036000 tx=0x7f4fb403c040 comp rx=0 tx=0).stop 2023-09-23T03:17:14.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.700+0000 7f4fc3fff640 1 -- 172.21.15.154:0/2375838786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fc4158480 msgr2=0x7f4fc4143810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:14.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.700+0000 7f4fc3fff640 1 --2- 172.21.15.154:0/2375838786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fc4158480 0x7f4fc4143810 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4fac00c430 tx=0x7f4fac00c8f0 comp rx=0 tx=0).stop 2023-09-23T03:17:14.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.700+0000 7f4fc3fff640 1 -- 172.21.15.154:0/2375838786 shutdown_connections 2023-09-23T03:17:14.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.700+0000 7f4fc3fff640 1 --2- 172.21.15.154:0/2375838786 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4fa406af30 0x7f4fa406d3f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:14.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.700+0000 7f4fc3fff640 1 --2- 172.21.15.154:0/2375838786 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fc415ba50 0x7f4fc414a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:14.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.700+0000 7f4fc3fff640 1 --2- 172.21.15.154:0/2375838786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fc4158480 0x7f4fc4143810 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:14.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.700+0000 7f4fc3fff640 1 --2- 172.21.15.154:0/2375838786 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fc40705a0 0x7f4fc40ad6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:14.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.700+0000 7f4fc3fff640 1 -- 172.21.15.154:0/2375838786 >> 172.21.15.154:0/2375838786 conn(0x7f4fc40aff20 msgr2=0x7f4fc415a570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:14.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.700+0000 7f4fc3fff640 1 -- 172.21.15.154:0/2375838786 shutdown_connections 2023-09-23T03:17:14.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.700+0000 7f4fc3fff640 1 -- 172.21.15.154:0/2375838786 wait complete. 2023-09-23T03:17:14.721 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-09-23T03:17:14.721 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:17:14.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.937+0000 7efcb0f33640 1 Processor -- start 2023-09-23T03:17:14.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.937+0000 7efcb0f33640 1 -- start start 2023-09-23T03:17:14.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.939+0000 7efcb0f33640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efcac0acce0 0x7efcac15ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:14.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.939+0000 7efcb0f33640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcac15ac30 con 0x7efcac0cf040 2023-09-23T03:17:14.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.939+0000 7efcb0f33640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcac0b1f70 con 0x7efcac0b1b20 2023-09-23T03:17:14.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.939+0000 7efcb0f33640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcac0b2e90 con 0x7efcac0acce0 2023-09-23T03:17:14.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.939+0000 7efcaa59c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efcac0acce0 0x7efcac15ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:14.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.939+0000 7efcaad9d640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7efcac0cf040 0x7efcac0ab140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40696/0 (socket says 172.21.15.154:40696) 2023-09-23T03:17:14.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.939+0000 7efcaa59c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efcac0acce0 0x7efcac15ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46396/0 (socket says 172.21.15.154:46396) 2023-09-23T03:17:14.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.939+0000 7efcaad9d640 1 -- 172.21.15.154:0/3442683087 learned_addr learned my addr 172.21.15.154:0/3442683087 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:17:14.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.940+0000 7efcaa59c640 1 -- 172.21.15.154:0/3442683087 >> v1:172.21.15.190:6789/0 conn(0x7efcac0b1b20 legacy=0x7efcac0ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:14.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.940+0000 7efcaa59c640 1 -- 172.21.15.154:0/3442683087 >> v1:172.21.15.154:6789/0 conn(0x7efcac0cf040 legacy=0x7efcac0ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:14.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.940+0000 7efcaa59c640 1 -- 172.21.15.154:0/3442683087 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcac15f2a0 con 0x7efcac0acce0 2023-09-23T03:17:14.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.940+0000 7efca959a640 1 -- 172.21.15.154:0/3442683087 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 948308563 0 0) 0x7efca4001110 con 0x7efcac0cf040 2023-09-23T03:17:14.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.941+0000 7efcaa59c640 1 --2- 172.21.15.154:0/3442683087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcac0acce0 0x7efcac15ed60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7efc9c009790 tx=0x7efc9c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2627b0b956e86ae9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:14.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.941+0000 7efca959a640 1 -- 172.21.15.154:0/3442683087 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9c031780 con 0x7efcac0acce0 2023-09-23T03:17:14.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.941+0000 7efca959a640 1 -- 172.21.15.154:0/3442683087 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc9c0318e0 con 0x7efcac0acce0 2023-09-23T03:17:14.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.941+0000 7efca959a640 1 -- 172.21.15.154:0/3442683087 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9c031c70 con 0x7efcac0acce0 2023-09-23T03:17:14.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.942+0000 7efcb0f33640 1 -- 172.21.15.154:0/3442683087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcac0acce0 msgr2=0x7efcac15ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:14.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.942+0000 7efcb0f33640 1 --2- 172.21.15.154:0/3442683087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcac0acce0 0x7efcac15ed60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7efc9c009790 tx=0x7efc9c02b820 comp rx=0 tx=0).stop 2023-09-23T03:17:14.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.942+0000 7efcb0f33640 1 -- 172.21.15.154:0/3442683087 shutdown_connections 2023-09-23T03:17:14.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.942+0000 7efcb0f33640 1 --2- 172.21.15.154:0/3442683087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcac0acce0 0x7efcac15ed60 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:14.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.942+0000 7efcb0f33640 1 -- 172.21.15.154:0/3442683087 >> 172.21.15.154:0/3442683087 conn(0x7efcac06a0f0 msgr2=0x7efcac06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:14.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.942+0000 7efcb0f33640 1 -- 172.21.15.154:0/3442683087 shutdown_connections 2023-09-23T03:17:14.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.942+0000 7efcb0f33640 1 -- 172.21.15.154:0/3442683087 wait complete. 2023-09-23T03:17:14.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.944+0000 7efcb0f33640 1 Processor -- start 2023-09-23T03:17:14.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.944+0000 7efcb0f33640 1 -- start start 2023-09-23T03:17:14.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.945+0000 7efcb0f33640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efcac0acce0 0x7efcac0c55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:14.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.945+0000 7efcb0f33640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcac0b1b20 0x7efcac1f4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:14.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.945+0000 7efcb0f33640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcac0cf040 0x7efcac0cd640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:14.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.945+0000 7efcb0f33640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcac0c3180 con 0x7efcac0b1b20 2023-09-23T03:17:14.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.945+0000 7efcb0f33640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcac0c33d0 con 0x7efcac0cf040 2023-09-23T03:17:14.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.945+0000 7efcb0f33640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcac0c36b0 con 0x7efcac0acce0 2023-09-23T03:17:14.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.945+0000 7efca9d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcac0b1b20 0x7efcac1f4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:14.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.945+0000 7efca9d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcac0b1b20 0x7efcac1f4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46410/0 (socket says 172.21.15.154:46410) 2023-09-23T03:17:14.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.945+0000 7efca9d9b640 1 -- 172.21.15.154:0/4082818774 learned_addr learned my addr 172.21.15.154:0/4082818774 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:14.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.945+0000 7efcaad9d640 1 --2- 172.21.15.154:0/4082818774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcac0cf040 0x7efcac0cd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:14.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.945+0000 7efcaa59c640 1 --2- 172.21.15.154:0/4082818774 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efcac0acce0 0x7efcac0c55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:14.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.945+0000 7efca9d9b640 1 -- 172.21.15.154:0/4082818774 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efcac0acce0 msgr2=0x7efcac0c55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:14.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.945+0000 7efca9d9b640 1 --2- 172.21.15.154:0/4082818774 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efcac0acce0 0x7efcac0c55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:14.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.945+0000 7efca9d9b640 1 -- 172.21.15.154:0/4082818774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcac0cf040 msgr2=0x7efcac0cd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:14.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.945+0000 7efca9d9b640 1 --2- 172.21.15.154:0/4082818774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcac0cf040 0x7efcac0cd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:14.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.945+0000 7efca9d9b640 1 -- 172.21.15.154:0/4082818774 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc9c0093a0 con 0x7efcac0b1b20 2023-09-23T03:17:14.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.945+0000 7efcaa59c640 1 --2- 172.21.15.154:0/4082818774 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efcac0acce0 0x7efcac0c55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:14.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.946+0000 7efca9d9b640 1 --2- 172.21.15.154:0/4082818774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcac0b1b20 0x7efcac1f4ff0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7efc9400c7c0 tx=0x7efc9400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:14.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.946+0000 7efca2ffd640 1 -- 172.21.15.154:0/4082818774 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9400aa50 con 0x7efcac0b1b20 2023-09-23T03:17:14.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.946+0000 7efca2ffd640 1 -- 172.21.15.154:0/4082818774 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc9400abb0 con 0x7efcac0b1b20 2023-09-23T03:17:14.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.946+0000 7efca2ffd640 1 -- 172.21.15.154:0/4082818774 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc94012420 con 0x7efcac0b1b20 2023-09-23T03:17:14.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.946+0000 7efcb0f33640 1 -- 172.21.15.154:0/4082818774 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcac0c3990 con 0x7efcac0b1b20 2023-09-23T03:17:14.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.946+0000 7efcb0f33640 1 -- 172.21.15.154:0/4082818774 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcac0c5b20 con 0x7efcac0b1b20 2023-09-23T03:17:14.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.947+0000 7efcb0f33640 1 -- 172.21.15.154:0/4082818774 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcac0c3eb0 con 0x7efcac0b1b20 2023-09-23T03:17:14.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.948+0000 7efca2ffd640 1 -- 172.21.15.154:0/4082818774 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7efc94012580 con 0x7efcac0b1b20 2023-09-23T03:17:14.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.949+0000 7efca2ffd640 1 --2- 172.21.15.154:0/4082818774 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efc8c06af30 0x7efc8c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:14.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.949+0000 7efca2ffd640 1 -- 172.21.15.154:0/4082818774 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7efc94089b50 con 0x7efcac0b1b20 2023-09-23T03:17:14.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.949+0000 7efcaa59c640 1 --2- 172.21.15.154:0/4082818774 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efc8c06af30 0x7efc8c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:14.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.950+0000 7efcaa59c640 1 --2- 172.21.15.154:0/4082818774 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efc8c06af30 0x7efc8c06d3f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efc9c036000 tx=0x7efc9c03d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:14.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:14.953+0000 7efca2ffd640 1 -- 172.21.15.154:0/4082818774 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7efcac0c3eb0 con 0x7efcac0b1b20 2023-09-23T03:17:15.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.172+0000 7efcb0f33640 1 -- 172.21.15.154:0/4082818774 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efcac0c3eb0 con 0x7efcac0b1b20 2023-09-23T03:17:15.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.173+0000 7efca2ffd640 1 -- 172.21.15.154:0/4082818774 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4922 (secure 0 0 0) 0x7efcac0c3eb0 con 0x7efcac0b1b20 2023-09-23T03:17:15.174 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:15.175 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4760,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/4034412019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4034412019},{"type":"v1","addr":"172.21.15.154:6836","nonce":4034412019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3736308401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3736308401},{"type":"v1","addr":"172.21.15.190:6835","nonce":3736308401}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4999,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/617287200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":617287200},{"type":"v1","addr":"172.21.15.154:6837","nonce":617287200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:17:11.088637+0000","modified":"2023-09-23T03:17:14.181716+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5002},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5002":{"gid":5002,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.190:6834/2131436414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2131436414},{"type":"v1","addr":"172.21.15.190:6834","nonce":2131436414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-09-23T03:17:15.175 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 18 2023-09-23T03:17:15.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.175+0000 7efcb0f33640 1 -- 172.21.15.154:0/4082818774 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efc8c06af30 msgr2=0x7efc8c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:15.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.175+0000 7efcb0f33640 1 --2- 172.21.15.154:0/4082818774 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efc8c06af30 0x7efc8c06d3f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efc9c036000 tx=0x7efc9c03d040 comp rx=0 tx=0).stop 2023-09-23T03:17:15.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.175+0000 7efcb0f33640 1 -- 172.21.15.154:0/4082818774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcac0b1b20 msgr2=0x7efcac1f4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:15.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.175+0000 7efcb0f33640 1 --2- 172.21.15.154:0/4082818774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcac0b1b20 0x7efcac1f4ff0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7efc9400c7c0 tx=0x7efc9400cc80 comp rx=0 tx=0).stop 2023-09-23T03:17:15.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.176+0000 7efcb0f33640 1 -- 172.21.15.154:0/4082818774 shutdown_connections 2023-09-23T03:17:15.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.176+0000 7efcb0f33640 1 --2- 172.21.15.154:0/4082818774 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efc8c06af30 0x7efc8c06d3f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:15.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.176+0000 7efcb0f33640 1 --2- 172.21.15.154:0/4082818774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcac0cf040 0x7efcac0cd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:15.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.176+0000 7efcb0f33640 1 --2- 172.21.15.154:0/4082818774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcac0b1b20 0x7efcac1f4ff0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:15.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.176+0000 7efcb0f33640 1 --2- 172.21.15.154:0/4082818774 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efcac0acce0 0x7efcac0c55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:15.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.176+0000 7efcb0f33640 1 -- 172.21.15.154:0/4082818774 >> 172.21.15.154:0/4082818774 conn(0x7efcac06a0f0 msgr2=0x7efcac0ab140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:15.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.176+0000 7efcb0f33640 1 -- 172.21.15.154:0/4082818774 shutdown_connections 2023-09-23T03:17:15.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.176+0000 7efcb0f33640 1 -- 172.21.15.154:0/4082818774 wait complete. 2023-09-23T03:17:15.195 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:17:15.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.407+0000 7f9d22c1a640 1 Processor -- start 2023-09-23T03:17:15.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.408+0000 7f9d22c1a640 1 -- start start 2023-09-23T03:17:15.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.409+0000 7f9d22c1a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9d1c0cf040 0x7f9d1c1656e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:15.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.409+0000 7f9d22c1a640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d1c15ac30 con 0x7f9d1c0acce0 2023-09-23T03:17:15.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.409+0000 7f9d22c1a640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d1c0b1f70 con 0x7f9d1c0b1b20 2023-09-23T03:17:15.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.409+0000 7f9d22c1a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d1c0b2e90 con 0x7f9d1c0cf040 2023-09-23T03:17:15.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.409+0000 7f9d209b6640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f9d1c0acce0 0x7f9d1c15ed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40710/0 (socket says 172.21.15.154:40710) 2023-09-23T03:17:15.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.409+0000 7f9d209b6640 1 -- 172.21.15.154:0/3924688055 learned_addr learned my addr 172.21.15.154:0/3924688055 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:17:15.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.409+0000 7f9d211b7640 1 --2- 172.21.15.154:0/3924688055 >> v2:172.21.15.154:3300/0 conn(0x7f9d1c0cf040 0x7f9d1c1656e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:15.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.410+0000 7f9d1b7fe640 1 -- 172.21.15.154:0/3924688055 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4147960005 0 0) 0x7f9d10001290 con 0x7f9d1c0acce0 2023-09-23T03:17:15.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.410+0000 7f9d1b7fe640 1 -- 172.21.15.154:0/3924688055 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d0c003520 con 0x7f9d1c0acce0 2023-09-23T03:17:15.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.410+0000 7f9d211b7640 1 -- 172.21.15.154:0/3924688055 >> v1:172.21.15.190:6789/0 conn(0x7f9d1c0b1b20 legacy=0x7f9d1c0ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:15.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.410+0000 7f9d211b7640 1 -- 172.21.15.154:0/3924688055 >> v1:172.21.15.154:6789/0 conn(0x7f9d1c0acce0 legacy=0x7f9d1c15ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:15.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.410+0000 7f9d211b7640 1 -- 172.21.15.154:0/3924688055 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d1c0b2210 con 0x7f9d1c0cf040 2023-09-23T03:17:15.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.411+0000 7f9d211b7640 1 --2- 172.21.15.154:0/3924688055 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d1c0cf040 0x7f9d1c1656e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9d14009d90 tx=0x7f9d1402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94da1c5ca391ec77 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:15.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.411+0000 7f9d1b7fe640 1 -- 172.21.15.154:0/3924688055 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1402cbd0 con 0x7f9d1c0cf040 2023-09-23T03:17:15.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.411+0000 7f9d1b7fe640 1 -- 172.21.15.154:0/3924688055 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1402cd30 con 0x7f9d1c0cf040 2023-09-23T03:17:15.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.411+0000 7f9d1b7fe640 1 -- 172.21.15.154:0/3924688055 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1402cea0 con 0x7f9d1c0cf040 2023-09-23T03:17:15.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.412+0000 7f9d22c1a640 1 -- 172.21.15.154:0/3924688055 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d1c0cf040 msgr2=0x7f9d1c1656e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:15.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.412+0000 7f9d22c1a640 1 --2- 172.21.15.154:0/3924688055 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d1c0cf040 0x7f9d1c1656e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9d14009d90 tx=0x7f9d1402be20 comp rx=0 tx=0).stop 2023-09-23T03:17:15.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.412+0000 7f9d22c1a640 1 -- 172.21.15.154:0/3924688055 shutdown_connections 2023-09-23T03:17:15.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.412+0000 7f9d22c1a640 1 --2- 172.21.15.154:0/3924688055 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d1c0cf040 0x7f9d1c1656e0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:15.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.412+0000 7f9d22c1a640 1 -- 172.21.15.154:0/3924688055 >> 172.21.15.154:0/3924688055 conn(0x7f9d1c06a0f0 msgr2=0x7f9d1c06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:15.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.412+0000 7f9d22c1a640 1 -- 172.21.15.154:0/3924688055 shutdown_connections 2023-09-23T03:17:15.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.412+0000 7f9d22c1a640 1 -- 172.21.15.154:0/3924688055 wait complete. 2023-09-23T03:17:15.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.414+0000 7f9d22c1a640 1 Processor -- start 2023-09-23T03:17:15.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.414+0000 7f9d22c1a640 1 -- start start 2023-09-23T03:17:15.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.415+0000 7f9d22c1a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d1c0acce0 0x7f9d1c1f1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:15.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.415+0000 7f9d22c1a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d1c0b1b20 0x7f9d1c15d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:15.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.415+0000 7f9d22c1a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d1c0cf040 0x7f9d1c15b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:15.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.415+0000 7f9d22c1a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c0c71a0 con 0x7f9d1c0cf040 2023-09-23T03:17:15.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.415+0000 7f9d22c1a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c0c6800 con 0x7f9d1c0b1b20 2023-09-23T03:17:15.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.415+0000 7f9d22c1a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c0c6ae0 con 0x7f9d1c0acce0 2023-09-23T03:17:15.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.415+0000 7f9d211b7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d1c0cf040 0x7f9d1c15b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:15.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.415+0000 7f9d211b7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d1c0cf040 0x7f9d1c15b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46428/0 (socket says 172.21.15.154:46428) 2023-09-23T03:17:15.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.415+0000 7f9d211b7640 1 -- 172.21.15.154:0/2150951296 learned_addr learned my addr 172.21.15.154:0/2150951296 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:15.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.416+0000 7f9d209b6640 1 --2- 172.21.15.154:0/2150951296 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d1c0acce0 0x7f9d1c1f1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:15.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.416+0000 7f9d211b7640 1 -- 172.21.15.154:0/2150951296 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d1c0acce0 msgr2=0x7f9d1c1f1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:15.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.416+0000 7f9d211b7640 1 --2- 172.21.15.154:0/2150951296 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d1c0acce0 0x7f9d1c1f1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:15.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.416+0000 7f9d1bfff640 1 --2- 172.21.15.154:0/2150951296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d1c0b1b20 0x7f9d1c15d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:15.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.416+0000 7f9d211b7640 1 -- 172.21.15.154:0/2150951296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d1c0b1b20 msgr2=0x7f9d1c15d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:15.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.416+0000 7f9d211b7640 1 --2- 172.21.15.154:0/2150951296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d1c0b1b20 0x7f9d1c15d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:15.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.416+0000 7f9d211b7640 1 -- 172.21.15.154:0/2150951296 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d14009a70 con 0x7f9d1c0cf040 2023-09-23T03:17:15.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.416+0000 7f9d211b7640 1 --2- 172.21.15.154:0/2150951296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d1c0cf040 0x7f9d1c15b650 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f9d14009d90 tx=0x7f9d14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:15.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.416+0000 7f9d197fa640 1 -- 172.21.15.154:0/2150951296 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1402cea0 con 0x7f9d1c0cf040 2023-09-23T03:17:15.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.416+0000 7f9d197fa640 1 -- 172.21.15.154:0/2150951296 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1403e610 con 0x7f9d1c0cf040 2023-09-23T03:17:15.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.416+0000 7f9d197fa640 1 -- 172.21.15.154:0/2150951296 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1403e9c0 con 0x7f9d1c0cf040 2023-09-23T03:17:15.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.416+0000 7f9d22c1a640 1 -- 172.21.15.154:0/2150951296 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d1c0c6dc0 con 0x7f9d1c0cf040 2023-09-23T03:17:15.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.417+0000 7f9d22c1a640 1 -- 172.21.15.154:0/2150951296 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d1c15da40 con 0x7f9d1c0cf040 2023-09-23T03:17:15.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.418+0000 7f9d22c1a640 1 -- 172.21.15.154:0/2150951296 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d1c06aa80 con 0x7f9d1c0cf040 2023-09-23T03:17:15.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.419+0000 7f9d197fa640 1 -- 172.21.15.154:0/2150951296 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f9d1403b050 con 0x7f9d1c0cf040 2023-09-23T03:17:15.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.419+0000 7f9d197fa640 1 --2- 172.21.15.154:0/2150951296 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9d0006af30 0x7f9d0006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:15.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.420+0000 7f9d197fa640 1 -- 172.21.15.154:0/2150951296 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9d140ac0e0 con 0x7f9d1c0cf040 2023-09-23T03:17:15.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.420+0000 7f9d209b6640 1 --2- 172.21.15.154:0/2150951296 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9d0006af30 0x7f9d0006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:15.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.420+0000 7f9d209b6640 1 --2- 172.21.15.154:0/2150951296 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9d0006af30 0x7f9d0006d3f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9d100013f0 tx=0x7f9d10009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:15.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.423+0000 7f9d197fa640 1 -- 172.21.15.154:0/2150951296 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f9d14077120 con 0x7f9d1c0cf040 2023-09-23T03:17:15.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.615+0000 7f9d22c1a640 1 -- 172.21.15.154:0/2150951296 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d1c0c72e0 con 0x7f9d1c0cf040 2023-09-23T03:17:15.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.618+0000 7f9d197fa640 1 -- 172.21.15.154:0/2150951296 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16810 (secure 0 0 0) 0x7f9d1c0c72e0 con 0x7f9d1c0cf040 2023-09-23T03:17:15.620 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:15.620 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":38,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:17:12.168442+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:17:08.933690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-09-23T03:17:10.141711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:17:15.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.621+0000 7f9d22c1a640 1 -- 172.21.15.154:0/2150951296 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9d0006af30 msgr2=0x7f9d0006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:15.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.621+0000 7f9d22c1a640 1 --2- 172.21.15.154:0/2150951296 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9d0006af30 0x7f9d0006d3f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9d100013f0 tx=0x7f9d10009040 comp rx=0 tx=0).stop 2023-09-23T03:17:15.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.621+0000 7f9d22c1a640 1 -- 172.21.15.154:0/2150951296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d1c0cf040 msgr2=0x7f9d1c15b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:15.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.621+0000 7f9d22c1a640 1 --2- 172.21.15.154:0/2150951296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d1c0cf040 0x7f9d1c15b650 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f9d14009d90 tx=0x7f9d14004a40 comp rx=0 tx=0).stop 2023-09-23T03:17:15.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.621+0000 7f9d22c1a640 1 -- 172.21.15.154:0/2150951296 shutdown_connections 2023-09-23T03:17:15.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.621+0000 7f9d22c1a640 1 --2- 172.21.15.154:0/2150951296 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9d0006af30 0x7f9d0006d3f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:15.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.621+0000 7f9d22c1a640 1 --2- 172.21.15.154:0/2150951296 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9d1c0cf040 0x7f9d1c15b650 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:15.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.621+0000 7f9d22c1a640 1 --2- 172.21.15.154:0/2150951296 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9d1c0b1b20 0x7f9d1c15d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:15.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.621+0000 7f9d22c1a640 1 --2- 172.21.15.154:0/2150951296 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9d1c0acce0 0x7f9d1c1f1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:15.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.621+0000 7f9d22c1a640 1 -- 172.21.15.154:0/2150951296 >> 172.21.15.154:0/2150951296 conn(0x7f9d1c06a0f0 msgr2=0x7f9d1c0ca6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:15.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.621+0000 7f9d22c1a640 1 -- 172.21.15.154:0/2150951296 shutdown_connections 2023-09-23T03:17:15.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.621+0000 7f9d22c1a640 1 -- 172.21.15.154:0/2150951296 wait complete. 2023-09-23T03:17:15.640 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-09-23T03:17:15.640 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-23T03:17:15.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.848+0000 7f13aa501640 1 Processor -- start 2023-09-23T03:17:15.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.849+0000 7f13aa501640 1 -- start start 2023-09-23T03:17:15.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.850+0000 7f13aa501640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13a40cf040 0x7f13a4160880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:15.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.850+0000 7f13aa501640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13a40cd530 con 0x7f13a415caf0 2023-09-23T03:17:15.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.850+0000 7f13aa501640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13a40cd810 con 0x7f13a4160dc0 2023-09-23T03:17:15.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.850+0000 7f13aa501640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a415ced0 con 0x7f13a40cf040 2023-09-23T03:17:15.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.850+0000 7f13a3fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f13a415caf0 0x7f13a40ce930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40712/0 (socket says 172.21.15.154:40712) 2023-09-23T03:17:15.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.850+0000 7f13a3fff640 1 -- 172.21.15.154:0/563929479 learned_addr learned my addr 172.21.15.154:0/563929479 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:17:15.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.851+0000 7f13a37fe640 1 -- 172.21.15.154:0/563929479 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4059296809 0 0) 0x7f13a40cd530 con 0x7f13a415caf0 2023-09-23T03:17:15.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.851+0000 7f13a37fe640 1 -- 172.21.15.154:0/563929479 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1390003520 con 0x7f13a415caf0 2023-09-23T03:17:15.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.851+0000 7f139bfff640 1 --2- 172.21.15.154:0/563929479 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13a40cf040 0x7f13a4160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:15.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.851+0000 7f13a37fe640 1 -- 172.21.15.154:0/563929479 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 533671456 0 0) 0x7f1390003520 con 0x7f13a415caf0 2023-09-23T03:17:15.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.851+0000 7f13a37fe640 1 -- 172.21.15.154:0/563929479 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13a40cf040 msgr2=0x7f13a4160880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:15.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.851+0000 7f13a37fe640 1 --2- 172.21.15.154:0/563929479 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13a40cf040 0x7f13a4160880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:15.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.851+0000 7f13a37fe640 1 -- 172.21.15.154:0/563929479 >> v1:172.21.15.190:6789/0 conn(0x7f13a4160dc0 legacy=0x7f13a40b1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:15.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.851+0000 7f13a37fe640 1 -- 172.21.15.154:0/563929479 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a40cdaf0 con 0x7f13a415caf0 2023-09-23T03:17:15.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.851+0000 7f13a37fe640 1 -- 172.21.15.154:0/563929479 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f13940017f0 con 0x7f13a415caf0 2023-09-23T03:17:15.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.851+0000 7f13a37fe640 1 -- 172.21.15.154:0/563929479 >> v1:172.21.15.154:6789/0 conn(0x7f13a415caf0 legacy=0x7f13a40ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:15.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.851+0000 7f13a37fe640 1 --2- 172.21.15.154:0/563929479 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13900039a0 0x7f1390003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:15.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.851+0000 7f13a37fe640 1 --2- 172.21.15.154:0/563929479 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13a40cf040 0x7f13900047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:15.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.851+0000 7f13a37fe640 1 --2- 172.21.15.154:0/563929479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1390004d00 0x7f1390005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:15.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.851+0000 7f13a37fe640 1 -- 172.21.15.154:0/563929479 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a415ced0 con 0x7f13900039a0 2023-09-23T03:17:15.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.851+0000 7f13a37fe640 1 -- 172.21.15.154:0/563929479 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f139000bb60 con 0x7f1390004d00 2023-09-23T03:17:15.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.851+0000 7f13a37fe640 1 -- 172.21.15.154:0/563929479 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f139000bd20 con 0x7f13a40cf040 2023-09-23T03:17:15.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.852+0000 7f13a3fff640 1 --2- 172.21.15.154:0/563929479 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13900039a0 0x7f1390003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:15.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.852+0000 7f13a3fff640 1 -- 172.21.15.154:0/563929479 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13a40cf040 msgr2=0x7f13900047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:15.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.852+0000 7f139bfff640 1 --2- 172.21.15.154:0/563929479 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13a40cf040 0x7f13900047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:15.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.852+0000 7f13a3fff640 1 --2- 172.21.15.154:0/563929479 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13a40cf040 0x7f13900047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:15.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.852+0000 7f13a3fff640 1 -- 172.21.15.154:0/563929479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1390004d00 msgr2=0x7f1390005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:15.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.852+0000 7f13a8a9e640 1 --2- 172.21.15.154:0/563929479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1390004d00 0x7f1390005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:15.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.852+0000 7f13a3fff640 1 --2- 172.21.15.154:0/563929479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1390004d00 0x7f1390005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:15.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.852+0000 7f13a3fff640 1 -- 172.21.15.154:0/563929479 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1390003520 con 0x7f13900039a0 2023-09-23T03:17:15.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.853+0000 7f13a3fff640 1 --2- 172.21.15.154:0/563929479 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13900039a0 0x7f1390003dc0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f139400c0e0 tx=0x7f13900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e440cf1624b7daa0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:15.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.853+0000 7f13a37fe640 1 -- 172.21.15.154:0/563929479 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1394007270 con 0x7f13900039a0 2023-09-23T03:17:15.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.853+0000 7f13a37fe640 1 -- 172.21.15.154:0/563929479 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13940073d0 con 0x7f13900039a0 2023-09-23T03:17:15.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.853+0000 7f13a37fe640 1 -- 172.21.15.154:0/563929479 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139402c7d0 con 0x7f13900039a0 2023-09-23T03:17:15.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.854+0000 7f13aa501640 1 -- 172.21.15.154:0/563929479 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13900039a0 msgr2=0x7f1390003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:15.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.854+0000 7f13aa501640 1 --2- 172.21.15.154:0/563929479 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13900039a0 0x7f1390003dc0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f139400c0e0 tx=0x7f13900057f0 comp rx=0 tx=0).stop 2023-09-23T03:17:15.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.854+0000 7f13a8a9e640 1 -- 172.21.15.154:0/563929479 reap_dead start 2023-09-23T03:17:15.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.854+0000 7f13aa501640 1 -- 172.21.15.154:0/563929479 shutdown_connections 2023-09-23T03:17:15.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.854+0000 7f13aa501640 1 -- 172.21.15.154:0/563929479 >> 172.21.15.154:0/563929479 conn(0x7f13a406a0f0 msgr2=0x7f13a406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:15.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.854+0000 7f13aa501640 1 -- 172.21.15.154:0/563929479 shutdown_connections 2023-09-23T03:17:15.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.854+0000 7f13aa501640 1 -- 172.21.15.154:0/563929479 wait complete. 2023-09-23T03:17:15.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.856+0000 7f13aa501640 1 Processor -- start 2023-09-23T03:17:15.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.856+0000 7f13aa501640 1 -- start start 2023-09-23T03:17:15.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.857+0000 7f13aa501640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13900039a0 0x7f13a41f2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:15.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.857+0000 7f13aa501640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13a40c2a00 0x7f13a41f5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:15.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.857+0000 7f13aa501640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13a40abea0 0x7f13a415dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:15.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.857+0000 7f13aa501640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a40c2f70 con 0x7f13a40c2a00 2023-09-23T03:17:15.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.857+0000 7f13aa501640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a40c3250 con 0x7f13900039a0 2023-09-23T03:17:15.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.857+0000 7f13aa501640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a40c3530 con 0x7f13a40abea0 2023-09-23T03:17:15.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.857+0000 7f139bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13a40c2a00 0x7f13a41f5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:15.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.857+0000 7f139bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13a40c2a00 0x7f13a41f5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46458/0 (socket says 172.21.15.154:46458) 2023-09-23T03:17:15.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.857+0000 7f139bfff640 1 -- 172.21.15.154:0/1974540138 learned_addr learned my addr 172.21.15.154:0/1974540138 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:15.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.858+0000 7f13a8a9e640 1 --2- 172.21.15.154:0/1974540138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13a40abea0 0x7f13a415dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:15.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.858+0000 7f13a3fff640 1 --2- 172.21.15.154:0/1974540138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13900039a0 0x7f13a41f2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:15.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.858+0000 7f139bfff640 1 -- 172.21.15.154:0/1974540138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13a40abea0 msgr2=0x7f13a415dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:15.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.858+0000 7f139bfff640 1 --2- 172.21.15.154:0/1974540138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13a40abea0 0x7f13a415dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:15.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.858+0000 7f139bfff640 1 -- 172.21.15.154:0/1974540138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13900039a0 msgr2=0x7f13a41f2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:15.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.858+0000 7f139bfff640 1 --2- 172.21.15.154:0/1974540138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13900039a0 0x7f13a41f2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:15.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.858+0000 7f139bfff640 1 -- 172.21.15.154:0/1974540138 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a40c3810 con 0x7f13a40c2a00 2023-09-23T03:17:15.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.858+0000 7f139bfff640 1 --2- 172.21.15.154:0/1974540138 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13a40c2a00 0x7f13a41f5160 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f138c00e7e0 tx=0x7f138c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:15.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.858+0000 7f13a17fa640 1 -- 172.21.15.154:0/1974540138 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138c00c6d0 con 0x7f13a40c2a00 2023-09-23T03:17:15.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.858+0000 7f13a17fa640 1 -- 172.21.15.154:0/1974540138 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f138c00c830 con 0x7f13a40c2a00 2023-09-23T03:17:15.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.858+0000 7f13a17fa640 1 -- 172.21.15.154:0/1974540138 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138c00cba0 con 0x7f13a40c2a00 2023-09-23T03:17:15.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.858+0000 7f13aa501640 1 -- 172.21.15.154:0/1974540138 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13a40c3af0 con 0x7f13a40c2a00 2023-09-23T03:17:15.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.858+0000 7f13aa501640 1 -- 172.21.15.154:0/1974540138 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13a415e230 con 0x7f13a40c2a00 2023-09-23T03:17:15.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.859+0000 7f13aa501640 1 -- 172.21.15.154:0/1974540138 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13a4073eb0 con 0x7f13a40c2a00 2023-09-23T03:17:15.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.861+0000 7f13a17fa640 1 -- 172.21.15.154:0/1974540138 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f138c00cd00 con 0x7f13a40c2a00 2023-09-23T03:17:15.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.862+0000 7f13a17fa640 1 --2- 172.21.15.154:0/1974540138 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f138406af60 0x7f138406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:15.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.862+0000 7f13a17fa640 1 -- 172.21.15.154:0/1974540138 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f138c0895f0 con 0x7f13a40c2a00 2023-09-23T03:17:15.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.862+0000 7f13a3fff640 1 --2- 172.21.15.154:0/1974540138 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f138406af60 0x7f138406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:15.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.863+0000 7f13a3fff640 1 --2- 172.21.15.154:0/1974540138 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f138406af60 0x7f138406d420 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1394034000 tx=0x7f1394009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:15.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:15.866+0000 7f13a17fa640 1 -- 172.21.15.154:0/1974540138 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f138c0544e0 con 0x7f13a40c2a00 2023-09-23T03:17:16.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.053+0000 7f13aa501640 1 -- 172.21.15.154:0/1974540138 --> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f13a40c33c0 con 0x7f138406af60 2023-09-23T03:17:16.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.064+0000 7f13a17fa640 1 -- 172.21.15.154:0/1974540138 <== mgr.4110 v2:172.21.15.154:6832/2422652104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286593 (secure 0 0 0) 0x7f13a40c33c0 con 0x7f138406af60 2023-09-23T03:17:16.069 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:16.070 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2023-09-23T03:17:16.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.069+0000 7f13aa501640 1 -- 172.21.15.154:0/1974540138 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f138406af60 msgr2=0x7f138406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:16.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.069+0000 7f13aa501640 1 --2- 172.21.15.154:0/1974540138 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f138406af60 0x7f138406d420 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1394034000 tx=0x7f1394009030 comp rx=0 tx=0).stop 2023-09-23T03:17:16.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.069+0000 7f13aa501640 1 -- 172.21.15.154:0/1974540138 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13a40c2a00 msgr2=0x7f13a41f5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:16.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.069+0000 7f13aa501640 1 --2- 172.21.15.154:0/1974540138 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13a40c2a00 0x7f13a41f5160 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f138c00e7e0 tx=0x7f138c00eca0 comp rx=0 tx=0).stop 2023-09-23T03:17:16.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.069+0000 7f13aa501640 1 -- 172.21.15.154:0/1974540138 shutdown_connections 2023-09-23T03:17:16.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.069+0000 7f13aa501640 1 --2- 172.21.15.154:0/1974540138 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f138406af60 0x7f138406d420 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:16.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.069+0000 7f13aa501640 1 --2- 172.21.15.154:0/1974540138 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13a40abea0 0x7f13a415dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:16.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.069+0000 7f13aa501640 1 --2- 172.21.15.154:0/1974540138 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13a40c2a00 0x7f13a41f5160 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:16.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.069+0000 7f13aa501640 1 --2- 172.21.15.154:0/1974540138 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13900039a0 0x7f13a41f2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:16.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.069+0000 7f13aa501640 1 -- 172.21.15.154:0/1974540138 >> 172.21.15.154:0/1974540138 conn(0x7f13a406a0f0 msgr2=0x7f13a4071630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:16.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.070+0000 7f13aa501640 1 -- 172.21.15.154:0/1974540138 shutdown_connections 2023-09-23T03:17:16.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.070+0000 7f13aa501640 1 -- 172.21.15.154:0/1974540138 wait complete. 2023-09-23T03:17:16.089 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":94,"stamp":"2023-09-23T03:17:15.200584+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":169,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220828,"kb_used_data":4644,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753892,"statfs":{"total":773094113280,"available":772867985408,"internally_reserved":0,"allocated":4755456,"data_stored":2696337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":221236794},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4],"upper_bound":8},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.570627"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.471356+0000","last_change":"2023-09-23T03:17:11.190009+0000","last_active":"2023-09-23T03:17:12.471356+0000","last_peered":"2023-09-23T03:17:12.471356+0000","last_clean":"2023-09-23T03:17:12.471356+0000","last_became_active":"2023-09-23T03:17:11.189506+0000","last_became_peered":"2023-09-23T03:17:11.189506+0000","last_unstale":"2023-09-23T03:17:12.471356+0000","last_undegraded":"2023-09-23T03:17:12.471356+0000","last_fullsized":"2023-09-23T03:17:12.471356+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:09:12.010638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.470746+0000","last_change":"2023-09-23T03:17:11.193653+0000","last_active":"2023-09-23T03:17:12.470746+0000","last_peered":"2023-09-23T03:17:12.470746+0000","last_clean":"2023-09-23T03:17:12.470746+0000","last_became_active":"2023-09-23T03:17:11.193374+0000","last_became_peered":"2023-09-23T03:17:11.193374+0000","last_unstale":"2023-09-23T03:17:12.470746+0000","last_undegraded":"2023-09-23T03:17:12.470746+0000","last_fullsized":"2023-09-23T03:17:12.470746+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:43:28.809579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.470092+0000","last_change":"2023-09-23T03:17:11.208246+0000","last_active":"2023-09-23T03:17:12.470092+0000","last_peered":"2023-09-23T03:17:12.470092+0000","last_clean":"2023-09-23T03:17:12.470092+0000","last_became_active":"2023-09-23T03:17:11.207949+0000","last_became_peered":"2023-09-23T03:17:11.207949+0000","last_unstale":"2023-09-23T03:17:12.470092+0000","last_undegraded":"2023-09-23T03:17:12.470092+0000","last_fullsized":"2023-09-23T03:17:12.470092+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:53:25.443853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.182322+0000","last_change":"2023-09-23T03:17:11.195297+0000","last_active":"2023-09-23T03:17:12.182322+0000","last_peered":"2023-09-23T03:17:12.182322+0000","last_clean":"2023-09-23T03:17:12.182322+0000","last_became_active":"2023-09-23T03:17:11.194760+0000","last_became_peered":"2023-09-23T03:17:11.194760+0000","last_unstale":"2023-09-23T03:17:12.182322+0000","last_undegraded":"2023-09-23T03:17:12.182322+0000","last_fullsized":"2023-09-23T03:17:12.182322+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:42:53.828556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.182024+0000","last_change":"2023-09-23T03:17:11.194362+0000","last_active":"2023-09-23T03:17:12.182024+0000","last_peered":"2023-09-23T03:17:12.182024+0000","last_clean":"2023-09-23T03:17:12.182024+0000","last_became_active":"2023-09-23T03:17:11.193300+0000","last_became_peered":"2023-09-23T03:17:11.193300+0000","last_unstale":"2023-09-23T03:17:12.182024+0000","last_undegraded":"2023-09-23T03:17:12.182024+0000","last_fullsized":"2023-09-23T03:17:12.182024+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:10:49.754292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.472275+0000","last_change":"2023-09-23T03:17:11.195516+0000","last_active":"2023-09-23T03:17:12.472275+0000","last_peered":"2023-09-23T03:17:12.472275+0000","last_clean":"2023-09-23T03:17:12.472275+0000","last_became_active":"2023-09-23T03:17:11.195245+0000","last_became_peered":"2023-09-23T03:17:11.195245+0000","last_unstale":"2023-09-23T03:17:12.472275+0000","last_undegraded":"2023-09-23T03:17:12.472275+0000","last_fullsized":"2023-09-23T03:17:12.472275+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:45:37.742495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.268925+0000","last_change":"2023-09-23T03:17:11.206803+0000","last_active":"2023-09-23T03:17:12.268925+0000","last_peered":"2023-09-23T03:17:12.268925+0000","last_clean":"2023-09-23T03:17:12.268925+0000","last_became_active":"2023-09-23T03:17:11.206498+0000","last_became_peered":"2023-09-23T03:17:11.206498+0000","last_unstale":"2023-09-23T03:17:12.268925+0000","last_undegraded":"2023-09-23T03:17:12.268925+0000","last_fullsized":"2023-09-23T03:17:12.268925+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:48:30.246329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.469171+0000","last_change":"2023-09-23T03:17:11.201040+0000","last_active":"2023-09-23T03:17:12.469171+0000","last_peered":"2023-09-23T03:17:12.469171+0000","last_clean":"2023-09-23T03:17:12.469171+0000","last_became_active":"2023-09-23T03:17:11.200707+0000","last_became_peered":"2023-09-23T03:17:11.200707+0000","last_unstale":"2023-09-23T03:17:12.469171+0000","last_undegraded":"2023-09-23T03:17:12.469171+0000","last_fullsized":"2023-09-23T03:17:12.469171+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:38:15.647687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.470011+0000","last_change":"2023-09-23T03:17:11.212538+0000","last_active":"2023-09-23T03:17:12.470011+0000","last_peered":"2023-09-23T03:17:12.470011+0000","last_clean":"2023-09-23T03:17:12.470011+0000","last_became_active":"2023-09-23T03:17:11.212235+0000","last_became_peered":"2023-09-23T03:17:11.212235+0000","last_unstale":"2023-09-23T03:17:12.470011+0000","last_undegraded":"2023-09-23T03:17:12.470011+0000","last_fullsized":"2023-09-23T03:17:12.470011+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:19:14.912945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.269655+0000","last_change":"2023-09-23T03:17:11.202206+0000","last_active":"2023-09-23T03:17:12.269655+0000","last_peered":"2023-09-23T03:17:12.269655+0000","last_clean":"2023-09-23T03:17:12.269655+0000","last_became_active":"2023-09-23T03:17:11.201915+0000","last_became_peered":"2023-09-23T03:17:11.201915+0000","last_unstale":"2023-09-23T03:17:12.269655+0000","last_undegraded":"2023-09-23T03:17:12.269655+0000","last_fullsized":"2023-09-23T03:17:12.269655+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:27:00.718764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.269733+0000","last_change":"2023-09-23T03:17:11.206765+0000","last_active":"2023-09-23T03:17:12.269733+0000","last_peered":"2023-09-23T03:17:12.269733+0000","last_clean":"2023-09-23T03:17:12.269733+0000","last_became_active":"2023-09-23T03:17:11.205402+0000","last_became_peered":"2023-09-23T03:17:11.205402+0000","last_unstale":"2023-09-23T03:17:12.269733+0000","last_undegraded":"2023-09-23T03:17:12.269733+0000","last_fullsized":"2023-09-23T03:17:12.269733+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:09:12.010638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.470030+0000","last_change":"2023-09-23T03:17:11.195198+0000","last_active":"2023-09-23T03:17:12.470030+0000","last_peered":"2023-09-23T03:17:12.470030+0000","last_clean":"2023-09-23T03:17:12.470030+0000","last_became_active":"2023-09-23T03:17:11.194866+0000","last_became_peered":"2023-09-23T03:17:11.194866+0000","last_unstale":"2023-09-23T03:17:12.470030+0000","last_undegraded":"2023-09-23T03:17:12.470030+0000","last_fullsized":"2023-09-23T03:17:12.470030+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:39:36.236424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.268977+0000","last_change":"2023-09-23T03:17:11.209599+0000","last_active":"2023-09-23T03:17:12.268977+0000","last_peered":"2023-09-23T03:17:12.268977+0000","last_clean":"2023-09-23T03:17:12.268977+0000","last_became_active":"2023-09-23T03:17:11.209183+0000","last_became_peered":"2023-09-23T03:17:11.209183+0000","last_unstale":"2023-09-23T03:17:12.268977+0000","last_undegraded":"2023-09-23T03:17:12.268977+0000","last_fullsized":"2023-09-23T03:17:12.268977+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:54:57.579461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.470110+0000","last_change":"2023-09-23T03:17:11.193566+0000","last_active":"2023-09-23T03:17:12.470110+0000","last_peered":"2023-09-23T03:17:12.470110+0000","last_clean":"2023-09-23T03:17:12.470110+0000","last_became_active":"2023-09-23T03:17:11.193279+0000","last_became_peered":"2023-09-23T03:17:11.193279+0000","last_unstale":"2023-09-23T03:17:12.470110+0000","last_undegraded":"2023-09-23T03:17:12.470110+0000","last_fullsized":"2023-09-23T03:17:12.470110+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:13:52.054858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.269035+0000","last_change":"2023-09-23T03:17:11.202546+0000","last_active":"2023-09-23T03:17:12.269035+0000","last_peered":"2023-09-23T03:17:12.269035+0000","last_clean":"2023-09-23T03:17:12.269035+0000","last_became_active":"2023-09-23T03:17:11.202088+0000","last_became_peered":"2023-09-23T03:17:11.202088+0000","last_unstale":"2023-09-23T03:17:12.269035+0000","last_undegraded":"2023-09-23T03:17:12.269035+0000","last_fullsized":"2023-09-23T03:17:12.269035+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:38:24.272265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.184013+0000","last_change":"2023-09-23T03:17:11.200054+0000","last_active":"2023-09-23T03:17:12.184013+0000","last_peered":"2023-09-23T03:17:12.184013+0000","last_clean":"2023-09-23T03:17:12.184013+0000","last_became_active":"2023-09-23T03:17:11.199793+0000","last_became_peered":"2023-09-23T03:17:11.199793+0000","last_unstale":"2023-09-23T03:17:12.184013+0000","last_undegraded":"2023-09-23T03:17:12.184013+0000","last_fullsized":"2023-09-23T03:17:12.184013+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:09:54.728444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.185955+0000","last_change":"2023-09-23T03:17:11.211184+0000","last_active":"2023-09-23T03:17:12.185955+0000","last_peered":"2023-09-23T03:17:12.185955+0000","last_clean":"2023-09-23T03:17:12.185955+0000","last_became_active":"2023-09-23T03:17:11.210919+0000","last_became_peered":"2023-09-23T03:17:11.210919+0000","last_unstale":"2023-09-23T03:17:12.185955+0000","last_undegraded":"2023-09-23T03:17:12.185955+0000","last_fullsized":"2023-09-23T03:17:12.185955+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:48:30.246329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.184947+0000","last_change":"2023-09-23T03:17:11.197077+0000","last_active":"2023-09-23T03:17:12.184947+0000","last_peered":"2023-09-23T03:17:12.184947+0000","last_clean":"2023-09-23T03:17:12.184947+0000","last_became_active":"2023-09-23T03:17:11.196792+0000","last_became_peered":"2023-09-23T03:17:11.196792+0000","last_unstale":"2023-09-23T03:17:12.184947+0000","last_undegraded":"2023-09-23T03:17:12.184947+0000","last_fullsized":"2023-09-23T03:17:12.184947+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:43:41.961245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.182161+0000","last_change":"2023-09-23T03:17:11.198829+0000","last_active":"2023-09-23T03:17:12.182161+0000","last_peered":"2023-09-23T03:17:12.182161+0000","last_clean":"2023-09-23T03:17:12.182161+0000","last_became_active":"2023-09-23T03:17:11.198557+0000","last_became_peered":"2023-09-23T03:17:11.198557+0000","last_unstale":"2023-09-23T03:17:12.182161+0000","last_undegraded":"2023-09-23T03:17:12.182161+0000","last_fullsized":"2023-09-23T03:17:12.182161+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:53:57.419735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.184342+0000","last_change":"2023-09-23T03:17:11.195620+0000","last_active":"2023-09-23T03:17:12.184342+0000","last_peered":"2023-09-23T03:17:12.184342+0000","last_clean":"2023-09-23T03:17:12.184342+0000","last_became_active":"2023-09-23T03:17:11.195337+0000","last_became_peered":"2023-09-23T03:17:11.195337+0000","last_unstale":"2023-09-23T03:17:12.184342+0000","last_undegraded":"2023-09-23T03:17:12.184342+0000","last_fullsized":"2023-09-23T03:17:12.184342+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:24:03.337494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.269238+0000","last_change":"2023-09-23T03:17:11.209429+0000","last_active":"2023-09-23T03:17:12.269238+0000","last_peered":"2023-09-23T03:17:12.269238+0000","last_clean":"2023-09-23T03:17:12.269238+0000","last_became_active":"2023-09-23T03:17:11.208872+0000","last_became_peered":"2023-09-23T03:17:11.208872+0000","last_unstale":"2023-09-23T03:17:12.269238+0000","last_undegraded":"2023-09-23T03:17:12.269238+0000","last_fullsized":"2023-09-23T03:17:12.269238+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:29:14.484941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.470374+0000","last_change":"2023-09-23T03:17:11.197506+0000","last_active":"2023-09-23T03:17:12.470374+0000","last_peered":"2023-09-23T03:17:12.470374+0000","last_clean":"2023-09-23T03:17:12.470374+0000","last_became_active":"2023-09-23T03:17:11.197241+0000","last_became_peered":"2023-09-23T03:17:11.197241+0000","last_unstale":"2023-09-23T03:17:12.470374+0000","last_undegraded":"2023-09-23T03:17:12.470374+0000","last_fullsized":"2023-09-23T03:17:12.470374+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:04:50.851324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.184022+0000","last_change":"2023-09-23T03:17:11.202976+0000","last_active":"2023-09-23T03:17:12.184022+0000","last_peered":"2023-09-23T03:17:12.184022+0000","last_clean":"2023-09-23T03:17:12.184022+0000","last_became_active":"2023-09-23T03:17:11.200288+0000","last_became_peered":"2023-09-23T03:17:11.200288+0000","last_unstale":"2023-09-23T03:17:12.184022+0000","last_undegraded":"2023-09-23T03:17:12.184022+0000","last_fullsized":"2023-09-23T03:17:12.184022+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:54:53.210843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.181919+0000","last_change":"2023-09-23T03:17:11.195305+0000","last_active":"2023-09-23T03:17:12.181919+0000","last_peered":"2023-09-23T03:17:12.181919+0000","last_clean":"2023-09-23T03:17:12.181919+0000","last_became_active":"2023-09-23T03:17:11.195053+0000","last_became_peered":"2023-09-23T03:17:11.195053+0000","last_unstale":"2023-09-23T03:17:12.181919+0000","last_undegraded":"2023-09-23T03:17:12.181919+0000","last_fullsized":"2023-09-23T03:17:12.181919+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:22:15.164792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.185344+0000","last_change":"2023-09-23T03:17:11.187479+0000","last_active":"2023-09-23T03:17:12.185344+0000","last_peered":"2023-09-23T03:17:12.185344+0000","last_clean":"2023-09-23T03:17:12.185344+0000","last_became_active":"2023-09-23T03:17:11.187101+0000","last_became_peered":"2023-09-23T03:17:11.187101+0000","last_unstale":"2023-09-23T03:17:12.185344+0000","last_undegraded":"2023-09-23T03:17:12.185344+0000","last_fullsized":"2023-09-23T03:17:12.185344+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:16:38.377968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.472781+0000","last_change":"2023-09-23T03:17:11.212271+0000","last_active":"2023-09-23T03:17:12.472781+0000","last_peered":"2023-09-23T03:17:12.472781+0000","last_clean":"2023-09-23T03:17:12.472781+0000","last_became_active":"2023-09-23T03:17:11.211986+0000","last_became_peered":"2023-09-23T03:17:11.211986+0000","last_unstale":"2023-09-23T03:17:12.472781+0000","last_undegraded":"2023-09-23T03:17:12.472781+0000","last_fullsized":"2023-09-23T03:17:12.472781+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:39:36.236424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.185292+0000","last_change":"2023-09-23T03:17:11.184328+0000","last_active":"2023-09-23T03:17:12.185292+0000","last_peered":"2023-09-23T03:17:12.185292+0000","last_clean":"2023-09-23T03:17:12.185292+0000","last_became_active":"2023-09-23T03:17:11.183957+0000","last_became_peered":"2023-09-23T03:17:11.183957+0000","last_unstale":"2023-09-23T03:17:12.185292+0000","last_undegraded":"2023-09-23T03:17:12.185292+0000","last_fullsized":"2023-09-23T03:17:12.185292+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:04:50.851324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.269834+0000","last_change":"2023-09-23T03:17:11.205816+0000","last_active":"2023-09-23T03:17:12.269834+0000","last_peered":"2023-09-23T03:17:12.269834+0000","last_clean":"2023-09-23T03:17:12.269834+0000","last_became_active":"2023-09-23T03:17:11.205507+0000","last_became_peered":"2023-09-23T03:17:11.205507+0000","last_unstale":"2023-09-23T03:17:12.269834+0000","last_undegraded":"2023-09-23T03:17:12.269834+0000","last_fullsized":"2023-09-23T03:17:12.269834+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:35:47.380029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.185011+0000","last_change":"2023-09-23T03:17:11.198594+0000","last_active":"2023-09-23T03:17:12.185011+0000","last_peered":"2023-09-23T03:17:12.185011+0000","last_clean":"2023-09-23T03:17:12.185011+0000","last_became_active":"2023-09-23T03:17:11.198296+0000","last_became_peered":"2023-09-23T03:17:11.198296+0000","last_unstale":"2023-09-23T03:17:12.185011+0000","last_undegraded":"2023-09-23T03:17:12.185011+0000","last_fullsized":"2023-09-23T03:17:12.185011+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:13:52.054858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.185394+0000","last_change":"2023-09-23T03:17:11.191073+0000","last_active":"2023-09-23T03:17:12.185394+0000","last_peered":"2023-09-23T03:17:12.185394+0000","last_clean":"2023-09-23T03:17:12.185394+0000","last_became_active":"2023-09-23T03:17:11.190811+0000","last_became_peered":"2023-09-23T03:17:11.190811+0000","last_unstale":"2023-09-23T03:17:12.185394+0000","last_undegraded":"2023-09-23T03:17:12.185394+0000","last_fullsized":"2023-09-23T03:17:12.185394+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:08:54.761338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.185023+0000","last_change":"2023-09-23T03:17:11.204108+0000","last_active":"2023-09-23T03:17:12.185023+0000","last_peered":"2023-09-23T03:17:12.185023+0000","last_clean":"2023-09-23T03:17:12.185023+0000","last_became_active":"2023-09-23T03:17:11.200467+0000","last_became_peered":"2023-09-23T03:17:11.200467+0000","last_unstale":"2023-09-23T03:17:12.185023+0000","last_undegraded":"2023-09-23T03:17:12.185023+0000","last_fullsized":"2023-09-23T03:17:12.185023+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:07:51.120761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.472080+0000","last_change":"2023-09-23T03:17:11.199169+0000","last_active":"2023-09-23T03:17:12.472080+0000","last_peered":"2023-09-23T03:17:12.472080+0000","last_clean":"2023-09-23T03:17:12.472080+0000","last_became_active":"2023-09-23T03:17:11.198876+0000","last_became_peered":"2023-09-23T03:17:11.198876+0000","last_unstale":"2023-09-23T03:17:12.472080+0000","last_undegraded":"2023-09-23T03:17:12.472080+0000","last_fullsized":"2023-09-23T03:17:12.472080+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:08:54.761338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.185653+0000","last_change":"2023-09-23T03:17:11.204328+0000","last_active":"2023-09-23T03:17:12.185653+0000","last_peered":"2023-09-23T03:17:12.185653+0000","last_clean":"2023-09-23T03:17:12.185653+0000","last_became_active":"2023-09-23T03:17:11.204080+0000","last_became_peered":"2023-09-23T03:17:11.204080+0000","last_unstale":"2023-09-23T03:17:12.185653+0000","last_undegraded":"2023-09-23T03:17:12.185653+0000","last_fullsized":"2023-09-23T03:17:12.185653+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:38:15.647687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.185420+0000","last_change":"2023-09-23T03:17:10.675152+0000","last_active":"2023-09-23T03:17:12.185420+0000","last_peered":"2023-09-23T03:17:12.185420+0000","last_clean":"2023-09-23T03:17:12.185420+0000","last_became_active":"2023-09-23T03:17:10.674843+0000","last_became_peered":"2023-09-23T03:17:10.674843+0000","last_unstale":"2023-09-23T03:17:12.185420+0000","last_undegraded":"2023-09-23T03:17:12.185420+0000","last_fullsized":"2023-09-23T03:17:12.185420+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:12:25.960972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.473086+0000","last_change":"2023-09-23T03:17:11.212311+0000","last_active":"2023-09-23T03:17:12.473086+0000","last_peered":"2023-09-23T03:17:12.473086+0000","last_clean":"2023-09-23T03:17:12.473086+0000","last_became_active":"2023-09-23T03:17:11.212038+0000","last_became_peered":"2023-09-23T03:17:11.212038+0000","last_unstale":"2023-09-23T03:17:12.473086+0000","last_undegraded":"2023-09-23T03:17:12.473086+0000","last_fullsized":"2023-09-23T03:17:12.473086+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:48:40.699566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.469980+0000","last_change":"2023-09-23T03:17:10.669913+0000","last_active":"2023-09-23T03:17:12.469980+0000","last_peered":"2023-09-23T03:17:12.469980+0000","last_clean":"2023-09-23T03:17:12.469980+0000","last_became_active":"2023-09-23T03:17:10.669425+0000","last_became_peered":"2023-09-23T03:17:10.669425+0000","last_unstale":"2023-09-23T03:17:12.469980+0000","last_undegraded":"2023-09-23T03:17:12.469980+0000","last_fullsized":"2023-09-23T03:17:12.469980+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:04:49.837772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.471036+0000","last_change":"2023-09-23T03:17:11.191988+0000","last_active":"2023-09-23T03:17:12.471036+0000","last_peered":"2023-09-23T03:17:12.471036+0000","last_clean":"2023-09-23T03:17:12.471036+0000","last_became_active":"2023-09-23T03:17:11.191721+0000","last_became_peered":"2023-09-23T03:17:11.191721+0000","last_unstale":"2023-09-23T03:17:12.471036+0000","last_undegraded":"2023-09-23T03:17:12.471036+0000","last_fullsized":"2023-09-23T03:17:12.471036+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:55:42.170648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.473257+0000","last_change":"2023-09-23T03:17:10.677903+0000","last_active":"2023-09-23T03:17:12.473257+0000","last_peered":"2023-09-23T03:17:12.473257+0000","last_clean":"2023-09-23T03:17:12.473257+0000","last_became_active":"2023-09-23T03:17:10.677703+0000","last_became_peered":"2023-09-23T03:17:10.677703+0000","last_unstale":"2023-09-23T03:17:12.473257+0000","last_undegraded":"2023-09-23T03:17:12.473257+0000","last_fullsized":"2023-09-23T03:17:12.473257+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:04:49.837772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.471084+0000","last_change":"2023-09-23T03:17:11.187515+0000","last_active":"2023-09-23T03:17:12.471084+0000","last_peered":"2023-09-23T03:17:12.471084+0000","last_clean":"2023-09-23T03:17:12.471084+0000","last_became_active":"2023-09-23T03:17:11.187200+0000","last_became_peered":"2023-09-23T03:17:11.187200+0000","last_unstale":"2023-09-23T03:17:12.471084+0000","last_undegraded":"2023-09-23T03:17:12.471084+0000","last_fullsized":"2023-09-23T03:17:12.471084+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:22:53.611756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.470754+0000","last_change":"2023-09-23T03:17:10.675330+0000","last_active":"2023-09-23T03:17:12.470754+0000","last_peered":"2023-09-23T03:17:12.470754+0000","last_clean":"2023-09-23T03:17:12.470754+0000","last_became_active":"2023-09-23T03:17:10.675098+0000","last_became_peered":"2023-09-23T03:17:10.675098+0000","last_unstale":"2023-09-23T03:17:12.470754+0000","last_undegraded":"2023-09-23T03:17:12.470754+0000","last_fullsized":"2023-09-23T03:17:12.470754+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:54:28.226116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.468826+0000","last_change":"2023-09-23T03:17:11.196664+0000","last_active":"2023-09-23T03:17:12.468826+0000","last_peered":"2023-09-23T03:17:12.468826+0000","last_clean":"2023-09-23T03:17:12.468826+0000","last_became_active":"2023-09-23T03:17:11.196391+0000","last_became_peered":"2023-09-23T03:17:11.196391+0000","last_unstale":"2023-09-23T03:17:12.468826+0000","last_undegraded":"2023-09-23T03:17:12.468826+0000","last_fullsized":"2023-09-23T03:17:12.468826+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:45:37.742495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.269254+0000","last_change":"2023-09-23T03:17:10.670864+0000","last_active":"2023-09-23T03:17:12.269254+0000","last_peered":"2023-09-23T03:17:12.269254+0000","last_clean":"2023-09-23T03:17:12.269254+0000","last_became_active":"2023-09-23T03:17:10.670596+0000","last_became_peered":"2023-09-23T03:17:10.670596+0000","last_unstale":"2023-09-23T03:17:12.269254+0000","last_undegraded":"2023-09-23T03:17:12.269254+0000","last_fullsized":"2023-09-23T03:17:12.269254+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:26:40.873065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.269322+0000","last_change":"2023-09-23T03:17:11.199748+0000","last_active":"2023-09-23T03:17:12.269322+0000","last_peered":"2023-09-23T03:17:12.269322+0000","last_clean":"2023-09-23T03:17:12.269322+0000","last_became_active":"2023-09-23T03:17:11.199443+0000","last_became_peered":"2023-09-23T03:17:11.199443+0000","last_unstale":"2023-09-23T03:17:12.269322+0000","last_undegraded":"2023-09-23T03:17:12.269322+0000","last_fullsized":"2023-09-23T03:17:12.269322+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:50:13.926507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.185527+0000","last_change":"2023-09-23T03:17:10.674112+0000","last_active":"2023-09-23T03:17:12.185527+0000","last_peered":"2023-09-23T03:17:12.185527+0000","last_clean":"2023-09-23T03:17:12.185527+0000","last_became_active":"2023-09-23T03:17:10.673784+0000","last_became_peered":"2023-09-23T03:17:10.673784+0000","last_unstale":"2023-09-23T03:17:12.185527+0000","last_undegraded":"2023-09-23T03:17:12.185527+0000","last_fullsized":"2023-09-23T03:17:12.185527+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:48:39.686014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.470798+0000","last_change":"2023-09-23T03:17:11.210521+0000","last_active":"2023-09-23T03:17:12.470798+0000","last_peered":"2023-09-23T03:17:12.470798+0000","last_clean":"2023-09-23T03:17:12.470798+0000","last_became_active":"2023-09-23T03:17:11.210263+0000","last_became_peered":"2023-09-23T03:17:11.210263+0000","last_unstale":"2023-09-23T03:17:12.470798+0000","last_undegraded":"2023-09-23T03:17:12.470798+0000","last_fullsized":"2023-09-23T03:17:12.470798+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:08:54.761338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.472436+0000","last_change":"2023-09-23T03:17:10.671113+0000","last_active":"2023-09-23T03:17:12.472436+0000","last_peered":"2023-09-23T03:17:12.472436+0000","last_clean":"2023-09-23T03:17:12.472436+0000","last_became_active":"2023-09-23T03:17:10.670893+0000","last_became_peered":"2023-09-23T03:17:10.670893+0000","last_unstale":"2023-09-23T03:17:12.472436+0000","last_undegraded":"2023-09-23T03:17:12.472436+0000","last_fullsized":"2023-09-23T03:17:12.472436+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:24:02.323942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.469549+0000","last_change":"2023-09-23T03:17:11.200984+0000","last_active":"2023-09-23T03:17:12.469549+0000","last_peered":"2023-09-23T03:17:12.469549+0000","last_clean":"2023-09-23T03:17:12.469549+0000","last_became_active":"2023-09-23T03:17:11.200723+0000","last_became_peered":"2023-09-23T03:17:11.200723+0000","last_unstale":"2023-09-23T03:17:12.469549+0000","last_undegraded":"2023-09-23T03:17:12.469549+0000","last_fullsized":"2023-09-23T03:17:12.469549+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:48:30.246329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.469855+0000","last_change":"2023-09-23T03:17:10.672597+0000","last_active":"2023-09-23T03:17:12.469855+0000","last_peered":"2023-09-23T03:17:12.469855+0000","last_clean":"2023-09-23T03:17:12.469855+0000","last_became_active":"2023-09-23T03:17:10.672281+0000","last_became_peered":"2023-09-23T03:17:10.672281+0000","last_unstale":"2023-09-23T03:17:12.469855+0000","last_undegraded":"2023-09-23T03:17:12.469855+0000","last_fullsized":"2023-09-23T03:17:12.469855+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:25:56.275212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.182603+0000","last_change":"2023-09-23T03:17:11.198209+0000","last_active":"2023-09-23T03:17:12.182603+0000","last_peered":"2023-09-23T03:17:12.182603+0000","last_clean":"2023-09-23T03:17:12.182603+0000","last_became_active":"2023-09-23T03:17:11.197878+0000","last_became_peered":"2023-09-23T03:17:11.197878+0000","last_unstale":"2023-09-23T03:17:12.182603+0000","last_undegraded":"2023-09-23T03:17:12.182603+0000","last_fullsized":"2023-09-23T03:17:12.182603+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:25:57.288764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.471282+0000","last_change":"2023-09-23T03:17:10.664834+0000","last_active":"2023-09-23T03:17:12.471282+0000","last_peered":"2023-09-23T03:17:12.471282+0000","last_clean":"2023-09-23T03:17:12.471282+0000","last_became_active":"2023-09-23T03:17:10.664543+0000","last_became_peered":"2023-09-23T03:17:10.664543+0000","last_unstale":"2023-09-23T03:17:12.471282+0000","last_undegraded":"2023-09-23T03:17:12.471282+0000","last_fullsized":"2023-09-23T03:17:12.471282+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:43:08.791421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.186099+0000","last_change":"2023-09-23T03:17:11.204643+0000","last_active":"2023-09-23T03:17:12.186099+0000","last_peered":"2023-09-23T03:17:12.186099+0000","last_clean":"2023-09-23T03:17:12.186099+0000","last_became_active":"2023-09-23T03:17:11.203859+0000","last_became_peered":"2023-09-23T03:17:11.203859+0000","last_unstale":"2023-09-23T03:17:12.186099+0000","last_undegraded":"2023-09-23T03:17:12.186099+0000","last_fullsized":"2023-09-23T03:17:12.186099+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:38:24.272265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.270105+0000","last_change":"2023-09-23T03:17:10.662881+0000","last_active":"2023-09-23T03:17:12.270105+0000","last_peered":"2023-09-23T03:17:12.270105+0000","last_clean":"2023-09-23T03:17:12.270105+0000","last_became_active":"2023-09-23T03:17:10.662647+0000","last_became_peered":"2023-09-23T03:17:10.662647+0000","last_unstale":"2023-09-23T03:17:12.270105+0000","last_undegraded":"2023-09-23T03:17:12.270105+0000","last_fullsized":"2023-09-23T03:17:12.270105+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:45:36.728943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.470176+0000","last_change":"2023-09-23T03:17:11.203051+0000","last_active":"2023-09-23T03:17:12.470176+0000","last_peered":"2023-09-23T03:17:12.470176+0000","last_clean":"2023-09-23T03:17:12.470176+0000","last_became_active":"2023-09-23T03:17:11.202760+0000","last_became_peered":"2023-09-23T03:17:11.202760+0000","last_unstale":"2023-09-23T03:17:12.470176+0000","last_undegraded":"2023-09-23T03:17:12.470176+0000","last_fullsized":"2023-09-23T03:17:12.470176+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:44:08.447318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.269963+0000","last_change":"2023-09-23T03:17:10.669252+0000","last_active":"2023-09-23T03:17:12.269963+0000","last_peered":"2023-09-23T03:17:12.269963+0000","last_clean":"2023-09-23T03:17:12.269963+0000","last_became_active":"2023-09-23T03:17:10.668985+0000","last_became_peered":"2023-09-23T03:17:10.668985+0000","last_unstale":"2023-09-23T03:17:12.269963+0000","last_undegraded":"2023-09-23T03:17:12.269963+0000","last_fullsized":"2023-09-23T03:17:12.269963+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:09:53.714892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.470545+0000","last_change":"2023-09-23T03:17:11.196157+0000","last_active":"2023-09-23T03:17:12.470545+0000","last_peered":"2023-09-23T03:17:12.470545+0000","last_clean":"2023-09-23T03:17:12.470545+0000","last_became_active":"2023-09-23T03:17:11.195906+0000","last_became_peered":"2023-09-23T03:17:11.195906+0000","last_unstale":"2023-09-23T03:17:12.470545+0000","last_undegraded":"2023-09-23T03:17:12.470545+0000","last_fullsized":"2023-09-23T03:17:12.470545+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:16:38.377968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.471447+0000","last_change":"2023-09-23T03:17:10.668386+0000","last_active":"2023-09-23T03:17:12.471447+0000","last_peered":"2023-09-23T03:17:12.471447+0000","last_clean":"2023-09-23T03:17:12.471447+0000","last_became_active":"2023-09-23T03:17:10.667806+0000","last_became_peered":"2023-09-23T03:17:10.667806+0000","last_unstale":"2023-09-23T03:17:12.471447+0000","last_undegraded":"2023-09-23T03:17:12.471447+0000","last_fullsized":"2023-09-23T03:17:12.471447+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:44:07.433766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.470190+0000","last_change":"2023-09-23T03:17:11.201450+0000","last_active":"2023-09-23T03:17:12.470190+0000","last_peered":"2023-09-23T03:17:12.470190+0000","last_clean":"2023-09-23T03:17:12.470190+0000","last_became_active":"2023-09-23T03:17:11.201037+0000","last_became_peered":"2023-09-23T03:17:11.201037+0000","last_unstale":"2023-09-23T03:17:12.470190+0000","last_undegraded":"2023-09-23T03:17:12.470190+0000","last_fullsized":"2023-09-23T03:17:12.470190+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:09:54.728444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.470312+0000","last_change":"2023-09-23T03:17:10.660419+0000","last_active":"2023-09-23T03:17:12.470312+0000","last_peered":"2023-09-23T03:17:12.470312+0000","last_clean":"2023-09-23T03:17:12.470312+0000","last_became_active":"2023-09-23T03:17:10.660152+0000","last_became_peered":"2023-09-23T03:17:10.660152+0000","last_unstale":"2023-09-23T03:17:12.470312+0000","last_undegraded":"2023-09-23T03:17:12.470312+0000","last_fullsized":"2023-09-23T03:17:12.470312+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:09:53.714892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.470732+0000","last_change":"2023-09-23T03:17:11.193191+0000","last_active":"2023-09-23T03:17:12.470732+0000","last_peered":"2023-09-23T03:17:12.470732+0000","last_clean":"2023-09-23T03:17:12.470732+0000","last_became_active":"2023-09-23T03:17:11.189869+0000","last_became_peered":"2023-09-23T03:17:11.189869+0000","last_unstale":"2023-09-23T03:17:12.470732+0000","last_undegraded":"2023-09-23T03:17:12.470732+0000","last_fullsized":"2023-09-23T03:17:12.470732+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:29:14.484941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.472434+0000","last_change":"2023-09-23T03:17:10.674149+0000","last_active":"2023-09-23T03:17:12.472434+0000","last_peered":"2023-09-23T03:17:12.472434+0000","last_clean":"2023-09-23T03:17:12.472434+0000","last_became_active":"2023-09-23T03:17:10.673894+0000","last_became_peered":"2023-09-23T03:17:10.673894+0000","last_unstale":"2023-09-23T03:17:12.472434+0000","last_undegraded":"2023-09-23T03:17:12.472434+0000","last_fullsized":"2023-09-23T03:17:12.472434+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:43:40.947693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.185250+0000","last_change":"2023-09-23T03:17:11.203167+0000","last_active":"2023-09-23T03:17:12.185250+0000","last_peered":"2023-09-23T03:17:12.185250+0000","last_clean":"2023-09-23T03:17:12.185250+0000","last_became_active":"2023-09-23T03:17:11.202873+0000","last_became_peered":"2023-09-23T03:17:11.202873+0000","last_unstale":"2023-09-23T03:17:12.185250+0000","last_undegraded":"2023-09-23T03:17:12.185250+0000","last_fullsized":"2023-09-23T03:17:12.185250+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:43:09.804974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.183910+0000","last_change":"2023-09-23T03:17:10.675461+0000","last_active":"2023-09-23T03:17:12.183910+0000","last_peered":"2023-09-23T03:17:12.183910+0000","last_clean":"2023-09-23T03:17:12.183910+0000","last_became_active":"2023-09-23T03:17:10.672873+0000","last_became_peered":"2023-09-23T03:17:10.672873+0000","last_unstale":"2023-09-23T03:17:12.183910+0000","last_undegraded":"2023-09-23T03:17:12.183910+0000","last_fullsized":"2023-09-23T03:17:12.183910+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:10:48.740740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.469869+0000","last_change":"2023-09-23T03:17:11.190739+0000","last_active":"2023-09-23T03:17:12.469869+0000","last_peered":"2023-09-23T03:17:12.469869+0000","last_clean":"2023-09-23T03:17:12.469869+0000","last_became_active":"2023-09-23T03:17:11.190415+0000","last_became_peered":"2023-09-23T03:17:11.190415+0000","last_unstale":"2023-09-23T03:17:12.469869+0000","last_undegraded":"2023-09-23T03:17:12.469869+0000","last_fullsized":"2023-09-23T03:17:12.469869+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:54:57.579461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.184254+0000","last_change":"2023-09-23T03:17:10.674832+0000","last_active":"2023-09-23T03:17:12.184254+0000","last_peered":"2023-09-23T03:17:12.184254+0000","last_clean":"2023-09-23T03:17:12.184254+0000","last_became_active":"2023-09-23T03:17:10.674508+0000","last_became_peered":"2023-09-23T03:17:10.674508+0000","last_unstale":"2023-09-23T03:17:12.184254+0000","last_undegraded":"2023-09-23T03:17:12.184254+0000","last_fullsized":"2023-09-23T03:17:12.184254+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:13:51.041305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.471209+0000","last_change":"2023-09-23T03:17:11.184116+0000","last_active":"2023-09-23T03:17:12.471209+0000","last_peered":"2023-09-23T03:17:12.471209+0000","last_clean":"2023-09-23T03:17:12.471209+0000","last_became_active":"2023-09-23T03:17:11.183770+0000","last_became_peered":"2023-09-23T03:17:11.183770+0000","last_unstale":"2023-09-23T03:17:12.471209+0000","last_undegraded":"2023-09-23T03:17:12.471209+0000","last_fullsized":"2023-09-23T03:17:12.471209+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:27:00.718764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.269895+0000","last_change":"2023-09-23T03:17:10.673938+0000","last_active":"2023-09-23T03:17:12.269895+0000","last_peered":"2023-09-23T03:17:12.269895+0000","last_clean":"2023-09-23T03:17:12.269895+0000","last_became_active":"2023-09-23T03:17:10.673635+0000","last_became_peered":"2023-09-23T03:17:10.673635+0000","last_unstale":"2023-09-23T03:17:12.269895+0000","last_undegraded":"2023-09-23T03:17:12.269895+0000","last_fullsized":"2023-09-23T03:17:12.269895+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:07:50.107209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.269785+0000","last_change":"2023-09-23T03:17:11.205131+0000","last_active":"2023-09-23T03:17:12.269785+0000","last_peered":"2023-09-23T03:17:12.269785+0000","last_clean":"2023-09-23T03:17:12.269785+0000","last_became_active":"2023-09-23T03:17:11.204850+0000","last_became_peered":"2023-09-23T03:17:11.204850+0000","last_unstale":"2023-09-23T03:17:12.269785+0000","last_undegraded":"2023-09-23T03:17:12.269785+0000","last_fullsized":"2023-09-23T03:17:12.269785+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:55:45.937101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.471135+0000","last_change":"2023-09-23T03:17:10.676267+0000","last_active":"2023-09-23T03:17:12.471135+0000","last_peered":"2023-09-23T03:17:12.471135+0000","last_clean":"2023-09-23T03:17:12.471135+0000","last_became_active":"2023-09-23T03:17:10.675883+0000","last_became_peered":"2023-09-23T03:17:10.675883+0000","last_unstale":"2023-09-23T03:17:12.471135+0000","last_undegraded":"2023-09-23T03:17:12.471135+0000","last_fullsized":"2023-09-23T03:17:12.471135+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:24:02.323942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.470925+0000","last_change":"2023-09-23T03:17:11.196964+0000","last_active":"2023-09-23T03:17:12.470925+0000","last_peered":"2023-09-23T03:17:12.470925+0000","last_clean":"2023-09-23T03:17:12.470925+0000","last_became_active":"2023-09-23T03:17:11.196468+0000","last_became_peered":"2023-09-23T03:17:11.196468+0000","last_unstale":"2023-09-23T03:17:12.470925+0000","last_undegraded":"2023-09-23T03:17:12.470925+0000","last_fullsized":"2023-09-23T03:17:12.470925+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:45:37.742495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.182213+0000","last_change":"2023-09-23T03:17:10.666770+0000","last_active":"2023-09-23T03:17:12.182213+0000","last_peered":"2023-09-23T03:17:12.182213+0000","last_clean":"2023-09-23T03:17:12.182213+0000","last_became_active":"2023-09-23T03:17:10.666474+0000","last_became_peered":"2023-09-23T03:17:10.666474+0000","last_unstale":"2023-09-23T03:17:12.182213+0000","last_undegraded":"2023-09-23T03:17:12.182213+0000","last_fullsized":"2023-09-23T03:17:12.182213+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:12:25.960972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.182473+0000","last_change":"2023-09-23T03:17:11.187777+0000","last_active":"2023-09-23T03:17:12.182473+0000","last_peered":"2023-09-23T03:17:12.182473+0000","last_clean":"2023-09-23T03:17:12.182473+0000","last_became_active":"2023-09-23T03:17:11.187477+0000","last_became_peered":"2023-09-23T03:17:11.187477+0000","last_unstale":"2023-09-23T03:17:12.182473+0000","last_undegraded":"2023-09-23T03:17:12.182473+0000","last_fullsized":"2023-09-23T03:17:12.182473+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:50:02.952823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.472891+0000","last_change":"2023-09-23T03:17:10.678937+0000","last_active":"2023-09-23T03:17:12.472891+0000","last_peered":"2023-09-23T03:17:12.472891+0000","last_clean":"2023-09-23T03:17:12.472891+0000","last_became_active":"2023-09-23T03:17:10.678694+0000","last_became_peered":"2023-09-23T03:17:10.678694+0000","last_unstale":"2023-09-23T03:17:12.472891+0000","last_undegraded":"2023-09-23T03:17:12.472891+0000","last_fullsized":"2023-09-23T03:17:12.472891+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:13:51.041305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.184614+0000","last_change":"2023-09-23T03:17:11.200899+0000","last_active":"2023-09-23T03:17:12.184614+0000","last_peered":"2023-09-23T03:17:12.184614+0000","last_clean":"2023-09-23T03:17:12.184614+0000","last_became_active":"2023-09-23T03:17:11.200669+0000","last_became_peered":"2023-09-23T03:17:11.200669+0000","last_unstale":"2023-09-23T03:17:12.184614+0000","last_undegraded":"2023-09-23T03:17:12.184614+0000","last_fullsized":"2023-09-23T03:17:12.184614+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:26:41.886617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.182000+0000","last_change":"2023-09-23T03:17:10.662742+0000","last_active":"2023-09-23T03:17:12.182000+0000","last_peered":"2023-09-23T03:17:12.182000+0000","last_clean":"2023-09-23T03:17:12.182000+0000","last_became_active":"2023-09-23T03:17:10.662441+0000","last_became_peered":"2023-09-23T03:17:10.662441+0000","last_unstale":"2023-09-23T03:17:12.182000+0000","last_undegraded":"2023-09-23T03:17:12.182000+0000","last_fullsized":"2023-09-23T03:17:12.182000+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:28:57.388433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.470086+0000","last_change":"2023-09-23T03:17:11.203572+0000","last_active":"2023-09-23T03:17:12.470086+0000","last_peered":"2023-09-23T03:17:12.470086+0000","last_clean":"2023-09-23T03:17:12.470086+0000","last_became_active":"2023-09-23T03:17:11.203326+0000","last_became_peered":"2023-09-23T03:17:11.203326+0000","last_unstale":"2023-09-23T03:17:12.470086+0000","last_undegraded":"2023-09-23T03:17:12.470086+0000","last_fullsized":"2023-09-23T03:17:12.470086+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:48:40.699566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.182022+0000","last_change":"2023-09-23T03:17:10.674511+0000","last_active":"2023-09-23T03:17:12.182022+0000","last_peered":"2023-09-23T03:17:12.182022+0000","last_clean":"2023-09-23T03:17:12.182022+0000","last_became_active":"2023-09-23T03:17:10.674212+0000","last_became_peered":"2023-09-23T03:17:10.674212+0000","last_unstale":"2023-09-23T03:17:12.182022+0000","last_undegraded":"2023-09-23T03:17:12.182022+0000","last_fullsized":"2023-09-23T03:17:12.182022+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:28:43.374996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.269376+0000","last_change":"2023-09-23T03:17:11.205308+0000","last_active":"2023-09-23T03:17:12.269376+0000","last_peered":"2023-09-23T03:17:12.269376+0000","last_clean":"2023-09-23T03:17:12.269376+0000","last_became_active":"2023-09-23T03:17:11.205000+0000","last_became_peered":"2023-09-23T03:17:11.205000+0000","last_unstale":"2023-09-23T03:17:12.269376+0000","last_undegraded":"2023-09-23T03:17:12.269376+0000","last_fullsized":"2023-09-23T03:17:12.269376+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:22:53.611756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.185266+0000","last_change":"2023-09-23T03:17:10.664662+0000","last_active":"2023-09-23T03:17:12.185266+0000","last_peered":"2023-09-23T03:17:12.185266+0000","last_clean":"2023-09-23T03:17:12.185266+0000","last_became_active":"2023-09-23T03:17:10.663348+0000","last_became_peered":"2023-09-23T03:17:10.663348+0000","last_unstale":"2023-09-23T03:17:12.185266+0000","last_undegraded":"2023-09-23T03:17:12.185266+0000","last_fullsized":"2023-09-23T03:17:12.185266+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:45:36.728943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.470168+0000","last_change":"2023-09-23T03:17:11.201753+0000","last_active":"2023-09-23T03:17:12.470168+0000","last_peered":"2023-09-23T03:17:12.470168+0000","last_clean":"2023-09-23T03:17:12.470168+0000","last_became_active":"2023-09-23T03:17:11.201135+0000","last_became_peered":"2023-09-23T03:17:11.201135+0000","last_unstale":"2023-09-23T03:17:12.470168+0000","last_undegraded":"2023-09-23T03:17:12.470168+0000","last_fullsized":"2023-09-23T03:17:12.470168+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:38:24.272265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.181851+0000","last_change":"2023-09-23T03:17:10.661978+0000","last_active":"2023-09-23T03:17:12.181851+0000","last_peered":"2023-09-23T03:17:12.181851+0000","last_clean":"2023-09-23T03:17:12.181851+0000","last_became_active":"2023-09-23T03:17:10.661687+0000","last_became_peered":"2023-09-23T03:17:10.661687+0000","last_unstale":"2023-09-23T03:17:12.181851+0000","last_undegraded":"2023-09-23T03:17:12.181851+0000","last_fullsized":"2023-09-23T03:17:12.181851+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:15:06.006263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.472688+0000","last_change":"2023-09-23T03:17:11.207631+0000","last_active":"2023-09-23T03:17:12.472688+0000","last_peered":"2023-09-23T03:17:12.472688+0000","last_clean":"2023-09-23T03:17:12.472688+0000","last_became_active":"2023-09-23T03:17:11.207309+0000","last_became_peered":"2023-09-23T03:17:11.207309+0000","last_unstale":"2023-09-23T03:17:12.472688+0000","last_undegraded":"2023-09-23T03:17:12.472688+0000","last_fullsized":"2023-09-23T03:17:12.472688+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:19:14.912945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.270144+0000","last_change":"2023-09-23T03:17:10.672697+0000","last_active":"2023-09-23T03:17:12.270144+0000","last_peered":"2023-09-23T03:17:12.270144+0000","last_clean":"2023-09-23T03:17:12.270144+0000","last_became_active":"2023-09-23T03:17:10.672369+0000","last_became_peered":"2023-09-23T03:17:10.672369+0000","last_unstale":"2023-09-23T03:17:12.270144+0000","last_undegraded":"2023-09-23T03:17:12.270144+0000","last_fullsized":"2023-09-23T03:17:12.270144+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:43:08.791421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.186351+0000","last_change":"2023-09-23T03:17:11.204819+0000","last_active":"2023-09-23T03:17:12.186351+0000","last_peered":"2023-09-23T03:17:12.186351+0000","last_clean":"2023-09-23T03:17:12.186351+0000","last_became_active":"2023-09-23T03:17:11.204351+0000","last_became_peered":"2023-09-23T03:17:11.204351+0000","last_unstale":"2023-09-23T03:17:12.186351+0000","last_undegraded":"2023-09-23T03:17:12.186351+0000","last_fullsized":"2023-09-23T03:17:12.186351+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:29:14.484941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.836150+0000","last_change":"2023-09-23T03:17:10.662393+0000","last_active":"2023-09-23T03:17:12.836150+0000","last_peered":"2023-09-23T03:17:12.836150+0000","last_clean":"2023-09-23T03:17:12.836150+0000","last_became_active":"2023-09-23T03:17:10.662131+0000","last_became_peered":"2023-09-23T03:17:10.662131+0000","last_unstale":"2023-09-23T03:17:12.836150+0000","last_undegraded":"2023-09-23T03:17:12.836150+0000","last_fullsized":"2023-09-23T03:17:12.836150+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:38:23.642804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.470735+0000","last_change":"2023-09-23T03:17:11.194286+0000","last_active":"2023-09-23T03:17:12.470735+0000","last_peered":"2023-09-23T03:17:12.470735+0000","last_clean":"2023-09-23T03:17:12.470735+0000","last_became_active":"2023-09-23T03:17:11.194000+0000","last_became_peered":"2023-09-23T03:17:11.194000+0000","last_unstale":"2023-09-23T03:17:12.470735+0000","last_undegraded":"2023-09-23T03:17:12.470735+0000","last_fullsized":"2023-09-23T03:17:12.470735+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:50:13.926507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.469690+0000","last_change":"2023-09-23T03:17:10.665420+0000","last_active":"2023-09-23T03:17:12.469690+0000","last_peered":"2023-09-23T03:17:12.469690+0000","last_clean":"2023-09-23T03:17:12.469690+0000","last_became_active":"2023-09-23T03:17:10.665158+0000","last_became_peered":"2023-09-23T03:17:10.665158+0000","last_unstale":"2023-09-23T03:17:12.469690+0000","last_undegraded":"2023-09-23T03:17:12.469690+0000","last_fullsized":"2023-09-23T03:17:12.469690+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:13:51.041305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.468967+0000","last_change":"2023-09-23T03:17:11.191126+0000","last_active":"2023-09-23T03:17:12.468967+0000","last_peered":"2023-09-23T03:17:12.468967+0000","last_clean":"2023-09-23T03:17:12.468967+0000","last_became_active":"2023-09-23T03:17:11.190749+0000","last_became_peered":"2023-09-23T03:17:11.190749+0000","last_unstale":"2023-09-23T03:17:12.468967+0000","last_undegraded":"2023-09-23T03:17:12.468967+0000","last_fullsized":"2023-09-23T03:17:12.468967+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:55:42.170648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.470316+0000","last_change":"2023-09-23T03:17:10.664455+0000","last_active":"2023-09-23T03:17:12.470316+0000","last_peered":"2023-09-23T03:17:12.470316+0000","last_clean":"2023-09-23T03:17:12.470316+0000","last_became_active":"2023-09-23T03:17:10.663712+0000","last_became_peered":"2023-09-23T03:17:10.663712+0000","last_unstale":"2023-09-23T03:17:12.470316+0000","last_undegraded":"2023-09-23T03:17:12.470316+0000","last_fullsized":"2023-09-23T03:17:12.470316+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:16:37.364416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.472382+0000","last_change":"2023-09-23T03:17:11.202707+0000","last_active":"2023-09-23T03:17:12.472382+0000","last_peered":"2023-09-23T03:17:12.472382+0000","last_clean":"2023-09-23T03:17:12.472382+0000","last_became_active":"2023-09-23T03:17:11.202457+0000","last_became_peered":"2023-09-23T03:17:11.202457+0000","last_unstale":"2023-09-23T03:17:12.472382+0000","last_undegraded":"2023-09-23T03:17:12.472382+0000","last_fullsized":"2023-09-23T03:17:12.472382+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:55:42.170648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.185522+0000","last_change":"2023-09-23T03:17:10.661244+0000","last_active":"2023-09-23T03:17:12.185522+0000","last_peered":"2023-09-23T03:17:12.185522+0000","last_clean":"2023-09-23T03:17:12.185522+0000","last_became_active":"2023-09-23T03:17:10.660916+0000","last_became_peered":"2023-09-23T03:17:10.660916+0000","last_unstale":"2023-09-23T03:17:12.185522+0000","last_undegraded":"2023-09-23T03:17:12.185522+0000","last_fullsized":"2023-09-23T03:17:12.185522+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:25:56.275212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.470002+0000","last_change":"2023-09-23T03:17:11.193935+0000","last_active":"2023-09-23T03:17:12.470002+0000","last_peered":"2023-09-23T03:17:12.470002+0000","last_clean":"2023-09-23T03:17:12.470002+0000","last_became_active":"2023-09-23T03:17:11.193672+0000","last_became_peered":"2023-09-23T03:17:11.193672+0000","last_unstale":"2023-09-23T03:17:12.470002+0000","last_undegraded":"2023-09-23T03:17:12.470002+0000","last_fullsized":"2023-09-23T03:17:12.470002+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:43:41.961245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.469855+0000","last_change":"2023-09-23T03:17:10.674754+0000","last_active":"2023-09-23T03:17:12.469855+0000","last_peered":"2023-09-23T03:17:12.469855+0000","last_clean":"2023-09-23T03:17:12.469855+0000","last_became_active":"2023-09-23T03:17:10.674470+0000","last_became_peered":"2023-09-23T03:17:10.674470+0000","last_unstale":"2023-09-23T03:17:12.469855+0000","last_undegraded":"2023-09-23T03:17:12.469855+0000","last_fullsized":"2023-09-23T03:17:12.469855+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:50:01.939271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.185987+0000","last_change":"2023-09-23T03:17:11.204835+0000","last_active":"2023-09-23T03:17:12.185987+0000","last_peered":"2023-09-23T03:17:12.185987+0000","last_clean":"2023-09-23T03:17:12.185987+0000","last_became_active":"2023-09-23T03:17:11.204240+0000","last_became_peered":"2023-09-23T03:17:11.204240+0000","last_unstale":"2023-09-23T03:17:12.185987+0000","last_undegraded":"2023-09-23T03:17:12.185987+0000","last_fullsized":"2023-09-23T03:17:12.185987+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:54:57.579461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.469274+0000","last_change":"2023-09-23T03:17:10.664308+0000","last_active":"2023-09-23T03:17:12.469274+0000","last_peered":"2023-09-23T03:17:12.469274+0000","last_clean":"2023-09-23T03:17:12.469274+0000","last_became_active":"2023-09-23T03:17:10.663824+0000","last_became_peered":"2023-09-23T03:17:10.663824+0000","last_unstale":"2023-09-23T03:17:12.469274+0000","last_undegraded":"2023-09-23T03:17:12.469274+0000","last_fullsized":"2023-09-23T03:17:12.469274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:45:36.728943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.269326+0000","last_change":"2023-09-23T03:17:11.212740+0000","last_active":"2023-09-23T03:17:12.269326+0000","last_peered":"2023-09-23T03:17:12.269326+0000","last_clean":"2023-09-23T03:17:12.269326+0000","last_became_active":"2023-09-23T03:17:11.211922+0000","last_became_peered":"2023-09-23T03:17:11.211922+0000","last_unstale":"2023-09-23T03:17:12.269326+0000","last_undegraded":"2023-09-23T03:17:12.269326+0000","last_fullsized":"2023-09-23T03:17:12.269326+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:43:28.809579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.185887+0000","last_change":"2023-09-23T03:17:10.668744+0000","last_active":"2023-09-23T03:17:12.185887+0000","last_peered":"2023-09-23T03:17:12.185887+0000","last_clean":"2023-09-23T03:17:12.185887+0000","last_became_active":"2023-09-23T03:17:10.668277+0000","last_became_peered":"2023-09-23T03:17:10.668277+0000","last_unstale":"2023-09-23T03:17:12.185887+0000","last_undegraded":"2023-09-23T03:17:12.185887+0000","last_fullsized":"2023-09-23T03:17:12.185887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:19:13.899393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":179,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.184858+0000","last_change":"2023-09-23T03:15:11.042447+0000","last_active":"2023-09-23T03:17:12.184858+0000","last_peered":"2023-09-23T03:17:12.184858+0000","last_clean":"2023-09-23T03:17:12.184858+0000","last_became_active":"2023-09-23T03:15:11.040322+0000","last_became_peered":"2023-09-23T03:15:11.040322+0000","last_unstale":"2023-09-23T03:17:12.184858+0000","last_undegraded":"2023-09-23T03:17:12.184858+0000","last_fullsized":"2023-09-23T03:17:12.184858+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_clean_scrub_stamp":"2023-09-23T03:15:09.474842+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:18:36.331645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343796,"statfs":{"total":96636764160,"available":96608047104,"internally_reserved":0,"allocated":1048576,"data_stored":781025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.65600000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.8049999999999999,"5min":2.149,"15min":2.149},"last":1.008},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.048,"15min":1.044},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.526,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":1.24}]},{"osd":1,"last update":"Sat Sep 23 03:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.224,"15min":1.2290000000000001},"min":{"1min":0.747,"5min":0.504,"15min":0.504},"max":{"1min":1.6950000000000001,"5min":3.1859999999999999,"15min":3.1859999999999999},"last":1.163},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.3740000000000001,"15min":1.403},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.591,"5min":2.613,"15min":2.613},"last":1.095}]},{"osd":2,"last update":"Sat Sep 23 03:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.151,"15min":1.1639999999999999},"min":{"1min":0.69099999999999995,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.393,"5min":2.331,"15min":2.331},"last":1.504},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.2949999999999999,"15min":1.3140000000000001},"min":{"1min":0.72399999999999998,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.0609999999999999,"5min":3.093,"15min":3.093},"last":1.4099999999999999}]},{"osd":3,"last update":"Sat Sep 23 03:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.298,"15min":1.3160000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.641,"5min":2.532,"15min":2.532},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.4350000000000001,"15min":1.4610000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.9570000000000001,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":0.80000000000000004}]},{"osd":4,"last update":"Sat Sep 23 03:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.514,"15min":1.5609999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":2,"5min":2.7210000000000001,"15min":2.7210000000000001},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.395,"5min":1.5569999999999999,"15min":1.5840000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.8740000000000001,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":1.0429999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.4710000000000001,"15min":1.4990000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.835,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":1.276},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.5469999999999999,"15min":1.5800000000000001},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.8,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":1.3500000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.6459999999999999,"15min":1.6759999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.9039999999999999,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":1.55},{"interface":"front","average":{"1min":1.389,"5min":1.6020000000000001,"15min":1.6379999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":2.1429999999999998,"5min":3.6739999999999999,"15min":3.6739999999999999},"last":1.625}]}]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":442368,"data_stored":189557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.016,"15min":1.018},"min":{"1min":0.63200000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.444,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.03},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.046,"15min":1.0549999999999999},"min":{"1min":0.67400000000000004,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.403,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.68899999999999995}]},{"osd":1,"last update":"Sat Sep 23 03:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.085,"15min":1.0900000000000001},"min":{"1min":0.63500000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":1.867},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.24,"15min":1.25},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":1.1579999999999999}]},{"osd":2,"last update":"Sat Sep 23 03:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.0900000000000001,"15min":1.0960000000000001},"min":{"1min":0.79900000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.51,"5min":1.597,"15min":1.597},"last":0.84599999999999997},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2749999999999999,"15min":1.2809999999999999},"min":{"1min":0.79600000000000004,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.8080000000000001,"5min":1.903,"15min":1.903},"last":1.8080000000000001}]},{"osd":3,"last update":"Sat Sep 23 03:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.357,"15min":1.367},"min":{"1min":0.96099999999999997,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.7410000000000001,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":1.7410000000000001},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.4550000000000001,"15min":1.4770000000000001},"min":{"1min":0.93200000000000005,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.8069999999999999,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.202}]},{"osd":4,"last update":"Sat Sep 23 03:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"min":{"1min":0.94499999999999995,"5min":0.498,"15min":0.498},"max":{"1min":1.804,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.121},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.2749999999999999,"15min":1.2909999999999999},"min":{"1min":0.68700000000000006,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.512,"5min":2.2109999999999999,"15min":2.2109999999999999},"last":0.88700000000000001}]},{"osd":5,"last update":"Sat Sep 23 03:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.3859999999999999,"15min":1.393},"min":{"1min":0.75800000000000001,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.722,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.5029999999999999,"15min":1.4930000000000001},"min":{"1min":0.83599999999999997,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":4.758,"5min":4.758,"15min":4.758},"last":4.758}]},{"osd":7,"last update":"Sat Sep 23 03:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":1.5509999999999999,"15min":1.569},"min":{"1min":0.86899999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.859,"5min":2.476,"15min":2.476},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.431,"5min":1.5509999999999999,"15min":1.571},"min":{"1min":0.90700000000000003,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.8360000000000001,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":0.90700000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":189087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0309999999999999,"15min":1.026},"min":{"1min":0.68600000000000005,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0189999999999999,"15min":1.018},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.4830000000000001,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.464}]},{"osd":1,"last update":"Sat Sep 23 03:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.2070000000000001,"15min":1.2230000000000001},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.7010000000000001,"5min":1.849,"15min":1.849},"last":1.282},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.304,"15min":1.3280000000000001},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.625,"5min":1.893,"15min":1.893},"last":1.3480000000000001}]},{"osd":2,"last update":"Sat Sep 23 03:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.1950000000000001,"15min":1.2090000000000001},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.6319999999999999,"5min":1.681,"15min":1.681},"last":1.085},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.288,"15min":1.3120000000000001},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.6200000000000001,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.996}]},{"osd":3,"last update":"Sat Sep 23 03:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.3540000000000001,"15min":1.395},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.6719999999999999,"5min":1.905,"15min":1.905},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.254,"5min":1.4610000000000001,"15min":1.4950000000000001},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":2.024,"5min":3.6110000000000002,"15min":3.6110000000000002},"last":0.86099999999999999}]},{"osd":4,"last update":"Sat Sep 23 03:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.4359999999999999,"15min":1.458},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.9199999999999999,"5min":2.069,"15min":2.069},"last":1.752},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.456,"15min":1.472},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.847,"5min":2.319,"15min":2.319},"last":1.9179999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.5249999999999999,"15min":1.546},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.1440000000000001,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":1.649},{"interface":"front","average":{"1min":1.444,"5min":1.6120000000000001,"15min":1.6399999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.8799999999999999,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.7989999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.532,"15min":1.5629999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.8400000000000001,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.21},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.645,"15min":1.663},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":2.1019999999999999,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.5840000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":189579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.1259999999999999,"15min":1.1379999999999999},"min":{"1min":0.74199999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.5029999999999999,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.1499999999999999,"15min":1.133},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.6579999999999999,"5min":1.9179999999999999,"15min":1.9179999999999999},"last":1.381}]},{"osd":1,"last update":"Sat Sep 23 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.2030000000000001,"15min":1.212},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.7030000000000001,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.228,"15min":1.234},"min":{"1min":0.83899999999999997,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.6000000000000001,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":1.268}]},{"osd":2,"last update":"Sat Sep 23 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.2390000000000001,"15min":1.2629999999999999},"min":{"1min":0.72499999999999998,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.2989999999999999,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.22,"15min":1.2150000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.6080000000000001,"5min":1.663,"15min":1.663},"last":1.137}]},{"osd":3,"last update":"Sat Sep 23 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.1679999999999999,"15min":1.1779999999999999},"min":{"1min":0.76700000000000002,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3400000000000001,"15min":1.3480000000000001},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.651,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.4970000000000001}]},{"osd":5,"last update":"Sat Sep 23 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.7390000000000001,"5min":2.097,"15min":2.097},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.327,"5min":1.325,"15min":1.325},"min":{"1min":0.874,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.6599999999999999,"5min":1.857,"15min":1.857},"last":1.423}]},{"osd":6,"last update":"Sat Sep 23 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.361,"15min":1.379},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.5700000000000001,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.452,"5min":1.462,"15min":1.464},"min":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":1.7230000000000001,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.2330000000000001}]},{"osd":7,"last update":"Sat Sep 23 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.4570000000000001,"15min":1.4710000000000001},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.665,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.099},{"interface":"front","average":{"1min":1.458,"5min":1.546,"15min":1.5609999999999999},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.7470000000000001,"5min":2.137,"15min":2.137},"last":1.323}]}]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":190621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.1240000000000001}]},{"osd":1,"last update":"Sat Sep 23 03:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":1.2310000000000001},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.82499999999999996}]},{"osd":2,"last update":"Sat Sep 23 03:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":2.2240000000000002,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":0.99299999999999999}]},{"osd":4,"last update":"Sat Sep 23 03:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.698,"5min":2.698,"15min":2.698},"last":1.734},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"min":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"max":{"1min":2.419,"5min":2.419,"15min":2.419},"last":1.599}]},{"osd":5,"last update":"Sat Sep 23 03:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.373,"15min":1.373},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.391,"5min":1.391,"15min":1.391},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.0070000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.4390000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":2.6440000000000001,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.502,"5min":1.502,"15min":1.502},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.0600000000000001,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":1.2869999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.544,"15min":1.544},"min":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":2.7879999999999998,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.552},{"interface":"front","average":{"1min":1.569,"5min":1.569,"15min":1.569},"min":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"max":{"1min":2.5379999999999998,"5min":2.5379999999999998,"15min":2.5379999999999998},"last":1.476}]}]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":442368,"data_stored":189045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.81100000000000005,"15min":0.82799999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1719999999999999,"5min":1.232,"15min":1.232},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.81799999999999995,"15min":0.82799999999999996},"min":{"1min":0.38900000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.71099999999999997}]},{"osd":1,"last update":"Sat Sep 23 03:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.83999999999999997,"15min":0.85799999999999998},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.23,"5min":1.454,"15min":1.454},"last":0.751},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.92500000000000004,"15min":0.93899999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.3620000000000001,"5min":1.452,"15min":1.452},"last":0.45400000000000001}]},{"osd":3,"last update":"Sat Sep 23 03:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.92600000000000005,"15min":0.93799999999999994},"min":{"1min":0.45200000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.462,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.98499999999999999,"15min":0.997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.3520000000000001,"5min":1.78,"15min":1.78},"last":0.91500000000000004}]},{"osd":4,"last update":"Sat Sep 23 03:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.3049999999999999,"15min":1.335},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.4990000000000001,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.127,"5min":1.3160000000000001,"15min":1.3480000000000001},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.8660000000000001,"5min":2.234,"15min":2.234},"last":1.3899999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.387,"15min":1.425},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.5229999999999999,"5min":3.9409999999999998,"15min":3.9409999999999998},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.214,"5min":1.514,"15min":1.5640000000000001},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.649,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":1.1279999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.387,"15min":1.427},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.724,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":1.085},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.4630000000000001,"15min":1.5009999999999999},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":1.647,"5min":4.5800000000000001,"15min":4.5800000000000001},"last":1.0289999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.3839999999999999,"15min":1.4099999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.5600000000000001,"5min":2.3149999999999999,"15min":2.3149999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.246,"5min":1.4299999999999999,"15min":1.4610000000000001},"min":{"1min":0.85399999999999998,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.575,"5min":2.621,"15min":2.621},"last":1.262}]}]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":189065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3],"upper_bound":8},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:17:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.80400000000000005,"15min":0.81000000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":2.8279999999999998,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":2.8279999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.84299999999999997,"15min":0.85099999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.6379999999999999,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":1.6379999999999999}]},{"osd":2,"last update":"Sat Sep 23 03:17:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.88500000000000001,"15min":0.90500000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.482,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":1.482},{"interface":"front","average":{"1min":0.89700000000000002,"5min":1.022,"15min":1.0429999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5109999999999999,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.5109999999999999}]},{"osd":3,"last update":"Sat Sep 23 03:17:11 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":1.069,"15min":1.101},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.6819999999999999,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.92700000000000005,"5min":1.141,"15min":1.177},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.548,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":0.45900000000000002}]},{"osd":4,"last update":"Sat Sep 23 03:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.268,"15min":1.294},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.4630000000000001,"5min":3.2629999999999999,"15min":3.2629999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.228,"5min":1.3009999999999999,"15min":1.3129999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.4689999999999999,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":2.4689999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.294,"15min":1.3120000000000001},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.593,"5min":3.161,"15min":3.161},"last":1.593},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.3300000000000001,"15min":1.3560000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.6040000000000001,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":1.532}]},{"osd":6,"last update":"Sat Sep 23 03:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.323,"15min":1.349},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.631,"5min":3.0179999999999998,"15min":3.0179999999999998},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.389,"15min":1.411},"min":{"1min":0.95299999999999996,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.5660000000000001,"5min":3.3580000000000001,"15min":3.3580000000000001},"last":1.419}]},{"osd":7,"last update":"Sat Sep 23 03:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.409,"15min":1.4279999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":2.0369999999999999,"5min":3.1099999999999999,"15min":3.1099999999999999},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.286,"5min":1.468,"15min":1.498},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":1.8899999999999999,"5min":3.306,"15min":3.306},"last":1.117}]}]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343812,"statfs":{"total":96636764160,"available":96608063488,"internally_reserved":0,"allocated":1032192,"data_stored":778358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 23 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.44900000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.2749999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.89300000000000002,"15min":0.90200000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.2330000000000001,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":1.1519999999999999}]},{"osd":2,"last update":"Sat Sep 23 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.79200000000000004,"15min":0.79600000000000004},"min":{"1min":0.54500000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.747},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.98099999999999998,"15min":0.98799999999999999},"min":{"1min":0.51000000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.4359999999999999,"5min":1.544,"15min":1.544},"last":0.79300000000000004}]},{"osd":3,"last update":"Sat Sep 23 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":0.93400000000000005,"15min":0.91600000000000004},"min":{"1min":0.60199999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.999},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.98699999999999999,"15min":1.0029999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.504,"5min":1.607,"15min":1.607},"last":1.0960000000000001}]},{"osd":4,"last update":"Sat Sep 23 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.2010000000000001,"15min":1.2030000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.619,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.526},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.278,"15min":1.286},"min":{"1min":0.88400000000000001,"5min":0.751,"15min":0.751},"max":{"1min":1.591,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.052}]},{"osd":5,"last update":"Sat Sep 23 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.2190000000000001,"15min":1.2270000000000001},"min":{"1min":0.78800000000000003,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.6060000000000001,"5min":1.867,"15min":1.867},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.2649999999999999,"15min":1.262},"min":{"1min":0.995,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.7430000000000001,"5min":1.776,"15min":1.776},"last":1.3380000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.2829999999999999,"15min":1.2969999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.651,"5min":1.887,"15min":1.887},"last":1.21},{"interface":"front","average":{"1min":1.26,"5min":1.2849999999999999,"15min":1.2889999999999999},"min":{"1min":0.85599999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.9019999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":1.254}]},{"osd":7,"last update":"Sat Sep 23 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.3240000000000001,"15min":1.3140000000000001},"min":{"1min":0.83499999999999996,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.9430000000000001,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":1.387},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.415,"15min":1.4119999999999999},"min":{"1min":1.0189999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":1.4850000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-23T03:17:16.097 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-23T03:17:16.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.350+0000 7fb5428a3640 1 Processor -- start 2023-09-23T03:17:16.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.350+0000 7fb5428a3640 1 -- start start 2023-09-23T03:17:16.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.352+0000 7fb5428a3640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb53c0c4970 0x7fb53c071870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:16.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.352+0000 7fb5428a3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb53c06c470 0x7fb53c0ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:16.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.352+0000 7fb5428a3640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb53c159a00 con 0x7fb53c0ca260 2023-09-23T03:17:16.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.352+0000 7fb5428a3640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb53c0cbe90 con 0x7fb53c0c4970 2023-09-23T03:17:16.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.352+0000 7fb5428a3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c071f70 con 0x7fb53c06c470 2023-09-23T03:17:16.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.352+0000 7fb5418a1640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb53c0c4970 0x7fb53c071870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:16.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.352+0000 7fb5418a1640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb53c0c4970 0x7fb53c071870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46472/0 (socket says 172.21.15.154:46472) 2023-09-23T03:17:16.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.352+0000 7fb5418a1640 1 -- 172.21.15.154:0/583842258 learned_addr learned my addr 172.21.15.154:0/583842258 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:16.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.353+0000 7fb5420a2640 1 --2- 172.21.15.154:0/583842258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb53c06c470 0x7fb53c0ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:16.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.353+0000 7fb5418a1640 1 -- 172.21.15.154:0/583842258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb53c06c470 msgr2=0x7fb53c0ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:16.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.353+0000 7fb5418a1640 1 --2- 172.21.15.154:0/583842258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb53c06c470 0x7fb53c0ccc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:16.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.353+0000 7fb5418a1640 1 -- 172.21.15.154:0/583842258 >> v1:172.21.15.190:6789/0 conn(0x7fb53c0ca260 legacy=0x7fb53c06bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:16.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.353+0000 7fb5418a1640 1 -- 172.21.15.154:0/583842258 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb53c0cc150 con 0x7fb53c0c4970 2023-09-23T03:17:16.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.354+0000 7fb5418a1640 1 --2- 172.21.15.154:0/583842258 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb53c0c4970 0x7fb53c071870 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb538009870 tx=0x7fb53802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8884281596b8ee4a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:16.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.355+0000 7fb54089f640 1 -- 172.21.15.154:0/583842258 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb538030840 con 0x7fb53c0c4970 2023-09-23T03:17:16.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.355+0000 7fb54089f640 1 -- 172.21.15.154:0/583842258 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5380309a0 con 0x7fb53c0c4970 2023-09-23T03:17:16.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.355+0000 7fb54089f640 1 -- 172.21.15.154:0/583842258 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb538030d30 con 0x7fb53c0c4970 2023-09-23T03:17:16.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.355+0000 7fb5428a3640 1 -- 172.21.15.154:0/583842258 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb53c0c4970 msgr2=0x7fb53c071870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:16.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.355+0000 7fb5428a3640 1 --2- 172.21.15.154:0/583842258 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb53c0c4970 0x7fb53c071870 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb538009870 tx=0x7fb53802b980 comp rx=0 tx=0).stop 2023-09-23T03:17:16.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.356+0000 7fb5428a3640 1 -- 172.21.15.154:0/583842258 shutdown_connections 2023-09-23T03:17:16.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.356+0000 7fb5428a3640 1 --2- 172.21.15.154:0/583842258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb53c06c470 0x7fb53c0ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:16.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.356+0000 7fb5428a3640 1 --2- 172.21.15.154:0/583842258 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb53c0c4970 0x7fb53c071870 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:16.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.356+0000 7fb5428a3640 1 -- 172.21.15.154:0/583842258 >> 172.21.15.154:0/583842258 conn(0x7fb53c06a390 msgr2=0x7fb53c0afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:16.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.356+0000 7fb5428a3640 1 -- 172.21.15.154:0/583842258 shutdown_connections 2023-09-23T03:17:16.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.356+0000 7fb5428a3640 1 -- 172.21.15.154:0/583842258 wait complete. 2023-09-23T03:17:16.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.357+0000 7fb5428a3640 1 Processor -- start 2023-09-23T03:17:16.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.357+0000 7fb5428a3640 1 -- start start 2023-09-23T03:17:16.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.358+0000 7fb5428a3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb53c06c470 0x7fb53c0b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:16.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.358+0000 7fb5428a3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb53c0c4970 0x7fb53c150040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:16.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.358+0000 7fb5428a3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb53c0ca260 0x7fb53c0b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:16.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.358+0000 7fb5428a3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c0b2e10 con 0x7fb53c0ca260 2023-09-23T03:17:16.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.358+0000 7fb5428a3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c0c3de0 con 0x7fb53c0c4970 2023-09-23T03:17:16.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.358+0000 7fb5428a3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb53c0c40c0 con 0x7fb53c06c470 2023-09-23T03:17:16.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.359+0000 7fb5420a2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb53c0ca260 0x7fb53c0b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:16.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.359+0000 7fb5420a2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb53c0ca260 0x7fb53c0b0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46476/0 (socket says 172.21.15.154:46476) 2023-09-23T03:17:16.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.359+0000 7fb5420a2640 1 -- 172.21.15.154:0/3675547967 learned_addr learned my addr 172.21.15.154:0/3675547967 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:16.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.359+0000 7fb5418a1640 1 --2- 172.21.15.154:0/3675547967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb53c06c470 0x7fb53c0b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:16.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.359+0000 7fb5420a2640 1 -- 172.21.15.154:0/3675547967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb53c06c470 msgr2=0x7fb53c0b35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:16.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.359+0000 7fb5420a2640 1 --2- 172.21.15.154:0/3675547967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb53c06c470 0x7fb53c0b35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:16.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.359+0000 7fb5420a2640 1 -- 172.21.15.154:0/3675547967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb53c0c4970 msgr2=0x7fb53c150040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:16.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.359+0000 7fb5420a2640 1 --2- 172.21.15.154:0/3675547967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb53c0c4970 0x7fb53c150040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:16.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.359+0000 7fb5420a2640 1 -- 172.21.15.154:0/3675547967 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5380094d0 con 0x7fb53c0ca260 2023-09-23T03:17:16.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.359+0000 7fb5418a1640 1 --2- 172.21.15.154:0/3675547967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb53c06c470 0x7fb53c0b35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:16.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.359+0000 7fb5420a2640 1 --2- 172.21.15.154:0/3675547967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb53c0ca260 0x7fb53c0b0c80 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb534007e40 tx=0x7fb53400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:16.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.360+0000 7fb5327fc640 1 -- 172.21.15.154:0/3675547967 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb534014620 con 0x7fb53c0ca260 2023-09-23T03:17:16.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.360+0000 7fb5327fc640 1 -- 172.21.15.154:0/3675547967 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb53400ee90 con 0x7fb53c0ca260 2023-09-23T03:17:16.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.360+0000 7fb5327fc640 1 -- 172.21.15.154:0/3675547967 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5340149e0 con 0x7fb53c0ca260 2023-09-23T03:17:16.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.360+0000 7fb5428a3640 1 -- 172.21.15.154:0/3675547967 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb53c0b30b0 con 0x7fb53c0ca260 2023-09-23T03:17:16.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.360+0000 7fb5428a3640 1 -- 172.21.15.154:0/3675547967 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb53c06bef0 con 0x7fb53c0ca260 2023-09-23T03:17:16.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.361+0000 7fb5428a3640 1 -- 172.21.15.154:0/3675547967 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb53c0736b0 con 0x7fb53c0ca260 2023-09-23T03:17:16.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.362+0000 7fb5327fc640 1 -- 172.21.15.154:0/3675547967 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fb534014b40 con 0x7fb53c0ca260 2023-09-23T03:17:16.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.363+0000 7fb5327fc640 1 --2- 172.21.15.154:0/3675547967 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb52006af60 0x7fb52006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:16.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.363+0000 7fb5327fc640 1 -- 172.21.15.154:0/3675547967 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb53408a250 con 0x7fb53c0ca260 2023-09-23T03:17:16.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.363+0000 7fb5418a1640 1 --2- 172.21.15.154:0/3675547967 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb52006af60 0x7fb52006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:16.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.364+0000 7fb5418a1640 1 --2- 172.21.15.154:0/3675547967 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb52006af60 0x7fb52006d420 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb538036000 tx=0x7fb53803b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:16.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.368+0000 7fb5327fc640 1 -- 172.21.15.154:0/3675547967 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fb534055140 con 0x7fb53c0ca260 2023-09-23T03:17:16.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.554+0000 7fb5428a3640 1 -- 172.21.15.154:0/3675547967 --> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb53c0c4230 con 0x7fb52006af60 2023-09-23T03:17:16.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.565+0000 7fb5327fc640 1 -- 172.21.15.154:0/3675547967 <== mgr.4110 v2:172.21.15.154:6832/2422652104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286593 (secure 0 0 0) 0x7fb53c0c4230 con 0x7fb52006af60 2023-09-23T03:17:16.570 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:16.572 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2023-09-23T03:17:16.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.571+0000 7fb5428a3640 1 -- 172.21.15.154:0/3675547967 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb52006af60 msgr2=0x7fb52006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:16.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.571+0000 7fb5428a3640 1 --2- 172.21.15.154:0/3675547967 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb52006af60 0x7fb52006d420 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb538036000 tx=0x7fb53803b040 comp rx=0 tx=0).stop 2023-09-23T03:17:16.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.571+0000 7fb5428a3640 1 -- 172.21.15.154:0/3675547967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb53c0ca260 msgr2=0x7fb53c0b0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:16.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.571+0000 7fb5428a3640 1 --2- 172.21.15.154:0/3675547967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb53c0ca260 0x7fb53c0b0c80 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb534007e40 tx=0x7fb53400e4b0 comp rx=0 tx=0).stop 2023-09-23T03:17:16.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.571+0000 7fb5428a3640 1 -- 172.21.15.154:0/3675547967 shutdown_connections 2023-09-23T03:17:16.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.571+0000 7fb5428a3640 1 --2- 172.21.15.154:0/3675547967 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb52006af60 0x7fb52006d420 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:16.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.571+0000 7fb5428a3640 1 --2- 172.21.15.154:0/3675547967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb53c0ca260 0x7fb53c0b0c80 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:16.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.571+0000 7fb5428a3640 1 --2- 172.21.15.154:0/3675547967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb53c0c4970 0x7fb53c150040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:16.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.571+0000 7fb5428a3640 1 --2- 172.21.15.154:0/3675547967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb53c06c470 0x7fb53c0b35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:16.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.571+0000 7fb5428a3640 1 -- 172.21.15.154:0/3675547967 >> 172.21.15.154:0/3675547967 conn(0x7fb53c06a390 msgr2=0x7fb53c151160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:16.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.571+0000 7fb5428a3640 1 -- 172.21.15.154:0/3675547967 shutdown_connections 2023-09-23T03:17:16.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.571+0000 7fb5428a3640 1 -- 172.21.15.154:0/3675547967 wait complete. 2023-09-23T03:17:16.591 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":94,"stamp":"2023-09-23T03:17:15.200584+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":169,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220828,"kb_used_data":4644,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753892,"statfs":{"total":773094113280,"available":772867985408,"internally_reserved":0,"allocated":4755456,"data_stored":2696337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":221236794},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4],"upper_bound":8},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.570627"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.471356+0000","last_change":"2023-09-23T03:17:11.190009+0000","last_active":"2023-09-23T03:17:12.471356+0000","last_peered":"2023-09-23T03:17:12.471356+0000","last_clean":"2023-09-23T03:17:12.471356+0000","last_became_active":"2023-09-23T03:17:11.189506+0000","last_became_peered":"2023-09-23T03:17:11.189506+0000","last_unstale":"2023-09-23T03:17:12.471356+0000","last_undegraded":"2023-09-23T03:17:12.471356+0000","last_fullsized":"2023-09-23T03:17:12.471356+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:09:12.010638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.470746+0000","last_change":"2023-09-23T03:17:11.193653+0000","last_active":"2023-09-23T03:17:12.470746+0000","last_peered":"2023-09-23T03:17:12.470746+0000","last_clean":"2023-09-23T03:17:12.470746+0000","last_became_active":"2023-09-23T03:17:11.193374+0000","last_became_peered":"2023-09-23T03:17:11.193374+0000","last_unstale":"2023-09-23T03:17:12.470746+0000","last_undegraded":"2023-09-23T03:17:12.470746+0000","last_fullsized":"2023-09-23T03:17:12.470746+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:43:28.809579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.470092+0000","last_change":"2023-09-23T03:17:11.208246+0000","last_active":"2023-09-23T03:17:12.470092+0000","last_peered":"2023-09-23T03:17:12.470092+0000","last_clean":"2023-09-23T03:17:12.470092+0000","last_became_active":"2023-09-23T03:17:11.207949+0000","last_became_peered":"2023-09-23T03:17:11.207949+0000","last_unstale":"2023-09-23T03:17:12.470092+0000","last_undegraded":"2023-09-23T03:17:12.470092+0000","last_fullsized":"2023-09-23T03:17:12.470092+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:53:25.443853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.182322+0000","last_change":"2023-09-23T03:17:11.195297+0000","last_active":"2023-09-23T03:17:12.182322+0000","last_peered":"2023-09-23T03:17:12.182322+0000","last_clean":"2023-09-23T03:17:12.182322+0000","last_became_active":"2023-09-23T03:17:11.194760+0000","last_became_peered":"2023-09-23T03:17:11.194760+0000","last_unstale":"2023-09-23T03:17:12.182322+0000","last_undegraded":"2023-09-23T03:17:12.182322+0000","last_fullsized":"2023-09-23T03:17:12.182322+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:42:53.828556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.182024+0000","last_change":"2023-09-23T03:17:11.194362+0000","last_active":"2023-09-23T03:17:12.182024+0000","last_peered":"2023-09-23T03:17:12.182024+0000","last_clean":"2023-09-23T03:17:12.182024+0000","last_became_active":"2023-09-23T03:17:11.193300+0000","last_became_peered":"2023-09-23T03:17:11.193300+0000","last_unstale":"2023-09-23T03:17:12.182024+0000","last_undegraded":"2023-09-23T03:17:12.182024+0000","last_fullsized":"2023-09-23T03:17:12.182024+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:10:49.754292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.472275+0000","last_change":"2023-09-23T03:17:11.195516+0000","last_active":"2023-09-23T03:17:12.472275+0000","last_peered":"2023-09-23T03:17:12.472275+0000","last_clean":"2023-09-23T03:17:12.472275+0000","last_became_active":"2023-09-23T03:17:11.195245+0000","last_became_peered":"2023-09-23T03:17:11.195245+0000","last_unstale":"2023-09-23T03:17:12.472275+0000","last_undegraded":"2023-09-23T03:17:12.472275+0000","last_fullsized":"2023-09-23T03:17:12.472275+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:45:37.742495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.268925+0000","last_change":"2023-09-23T03:17:11.206803+0000","last_active":"2023-09-23T03:17:12.268925+0000","last_peered":"2023-09-23T03:17:12.268925+0000","last_clean":"2023-09-23T03:17:12.268925+0000","last_became_active":"2023-09-23T03:17:11.206498+0000","last_became_peered":"2023-09-23T03:17:11.206498+0000","last_unstale":"2023-09-23T03:17:12.268925+0000","last_undegraded":"2023-09-23T03:17:12.268925+0000","last_fullsized":"2023-09-23T03:17:12.268925+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:48:30.246329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.469171+0000","last_change":"2023-09-23T03:17:11.201040+0000","last_active":"2023-09-23T03:17:12.469171+0000","last_peered":"2023-09-23T03:17:12.469171+0000","last_clean":"2023-09-23T03:17:12.469171+0000","last_became_active":"2023-09-23T03:17:11.200707+0000","last_became_peered":"2023-09-23T03:17:11.200707+0000","last_unstale":"2023-09-23T03:17:12.469171+0000","last_undegraded":"2023-09-23T03:17:12.469171+0000","last_fullsized":"2023-09-23T03:17:12.469171+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:38:15.647687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.470011+0000","last_change":"2023-09-23T03:17:11.212538+0000","last_active":"2023-09-23T03:17:12.470011+0000","last_peered":"2023-09-23T03:17:12.470011+0000","last_clean":"2023-09-23T03:17:12.470011+0000","last_became_active":"2023-09-23T03:17:11.212235+0000","last_became_peered":"2023-09-23T03:17:11.212235+0000","last_unstale":"2023-09-23T03:17:12.470011+0000","last_undegraded":"2023-09-23T03:17:12.470011+0000","last_fullsized":"2023-09-23T03:17:12.470011+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:19:14.912945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.269655+0000","last_change":"2023-09-23T03:17:11.202206+0000","last_active":"2023-09-23T03:17:12.269655+0000","last_peered":"2023-09-23T03:17:12.269655+0000","last_clean":"2023-09-23T03:17:12.269655+0000","last_became_active":"2023-09-23T03:17:11.201915+0000","last_became_peered":"2023-09-23T03:17:11.201915+0000","last_unstale":"2023-09-23T03:17:12.269655+0000","last_undegraded":"2023-09-23T03:17:12.269655+0000","last_fullsized":"2023-09-23T03:17:12.269655+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:27:00.718764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.269733+0000","last_change":"2023-09-23T03:17:11.206765+0000","last_active":"2023-09-23T03:17:12.269733+0000","last_peered":"2023-09-23T03:17:12.269733+0000","last_clean":"2023-09-23T03:17:12.269733+0000","last_became_active":"2023-09-23T03:17:11.205402+0000","last_became_peered":"2023-09-23T03:17:11.205402+0000","last_unstale":"2023-09-23T03:17:12.269733+0000","last_undegraded":"2023-09-23T03:17:12.269733+0000","last_fullsized":"2023-09-23T03:17:12.269733+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:09:12.010638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.470030+0000","last_change":"2023-09-23T03:17:11.195198+0000","last_active":"2023-09-23T03:17:12.470030+0000","last_peered":"2023-09-23T03:17:12.470030+0000","last_clean":"2023-09-23T03:17:12.470030+0000","last_became_active":"2023-09-23T03:17:11.194866+0000","last_became_peered":"2023-09-23T03:17:11.194866+0000","last_unstale":"2023-09-23T03:17:12.470030+0000","last_undegraded":"2023-09-23T03:17:12.470030+0000","last_fullsized":"2023-09-23T03:17:12.470030+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:39:36.236424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.268977+0000","last_change":"2023-09-23T03:17:11.209599+0000","last_active":"2023-09-23T03:17:12.268977+0000","last_peered":"2023-09-23T03:17:12.268977+0000","last_clean":"2023-09-23T03:17:12.268977+0000","last_became_active":"2023-09-23T03:17:11.209183+0000","last_became_peered":"2023-09-23T03:17:11.209183+0000","last_unstale":"2023-09-23T03:17:12.268977+0000","last_undegraded":"2023-09-23T03:17:12.268977+0000","last_fullsized":"2023-09-23T03:17:12.268977+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:54:57.579461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.470110+0000","last_change":"2023-09-23T03:17:11.193566+0000","last_active":"2023-09-23T03:17:12.470110+0000","last_peered":"2023-09-23T03:17:12.470110+0000","last_clean":"2023-09-23T03:17:12.470110+0000","last_became_active":"2023-09-23T03:17:11.193279+0000","last_became_peered":"2023-09-23T03:17:11.193279+0000","last_unstale":"2023-09-23T03:17:12.470110+0000","last_undegraded":"2023-09-23T03:17:12.470110+0000","last_fullsized":"2023-09-23T03:17:12.470110+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:13:52.054858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.269035+0000","last_change":"2023-09-23T03:17:11.202546+0000","last_active":"2023-09-23T03:17:12.269035+0000","last_peered":"2023-09-23T03:17:12.269035+0000","last_clean":"2023-09-23T03:17:12.269035+0000","last_became_active":"2023-09-23T03:17:11.202088+0000","last_became_peered":"2023-09-23T03:17:11.202088+0000","last_unstale":"2023-09-23T03:17:12.269035+0000","last_undegraded":"2023-09-23T03:17:12.269035+0000","last_fullsized":"2023-09-23T03:17:12.269035+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:38:24.272265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.184013+0000","last_change":"2023-09-23T03:17:11.200054+0000","last_active":"2023-09-23T03:17:12.184013+0000","last_peered":"2023-09-23T03:17:12.184013+0000","last_clean":"2023-09-23T03:17:12.184013+0000","last_became_active":"2023-09-23T03:17:11.199793+0000","last_became_peered":"2023-09-23T03:17:11.199793+0000","last_unstale":"2023-09-23T03:17:12.184013+0000","last_undegraded":"2023-09-23T03:17:12.184013+0000","last_fullsized":"2023-09-23T03:17:12.184013+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:09:54.728444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.185955+0000","last_change":"2023-09-23T03:17:11.211184+0000","last_active":"2023-09-23T03:17:12.185955+0000","last_peered":"2023-09-23T03:17:12.185955+0000","last_clean":"2023-09-23T03:17:12.185955+0000","last_became_active":"2023-09-23T03:17:11.210919+0000","last_became_peered":"2023-09-23T03:17:11.210919+0000","last_unstale":"2023-09-23T03:17:12.185955+0000","last_undegraded":"2023-09-23T03:17:12.185955+0000","last_fullsized":"2023-09-23T03:17:12.185955+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:48:30.246329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.184947+0000","last_change":"2023-09-23T03:17:11.197077+0000","last_active":"2023-09-23T03:17:12.184947+0000","last_peered":"2023-09-23T03:17:12.184947+0000","last_clean":"2023-09-23T03:17:12.184947+0000","last_became_active":"2023-09-23T03:17:11.196792+0000","last_became_peered":"2023-09-23T03:17:11.196792+0000","last_unstale":"2023-09-23T03:17:12.184947+0000","last_undegraded":"2023-09-23T03:17:12.184947+0000","last_fullsized":"2023-09-23T03:17:12.184947+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:43:41.961245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.182161+0000","last_change":"2023-09-23T03:17:11.198829+0000","last_active":"2023-09-23T03:17:12.182161+0000","last_peered":"2023-09-23T03:17:12.182161+0000","last_clean":"2023-09-23T03:17:12.182161+0000","last_became_active":"2023-09-23T03:17:11.198557+0000","last_became_peered":"2023-09-23T03:17:11.198557+0000","last_unstale":"2023-09-23T03:17:12.182161+0000","last_undegraded":"2023-09-23T03:17:12.182161+0000","last_fullsized":"2023-09-23T03:17:12.182161+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:53:57.419735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.184342+0000","last_change":"2023-09-23T03:17:11.195620+0000","last_active":"2023-09-23T03:17:12.184342+0000","last_peered":"2023-09-23T03:17:12.184342+0000","last_clean":"2023-09-23T03:17:12.184342+0000","last_became_active":"2023-09-23T03:17:11.195337+0000","last_became_peered":"2023-09-23T03:17:11.195337+0000","last_unstale":"2023-09-23T03:17:12.184342+0000","last_undegraded":"2023-09-23T03:17:12.184342+0000","last_fullsized":"2023-09-23T03:17:12.184342+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:24:03.337494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.269238+0000","last_change":"2023-09-23T03:17:11.209429+0000","last_active":"2023-09-23T03:17:12.269238+0000","last_peered":"2023-09-23T03:17:12.269238+0000","last_clean":"2023-09-23T03:17:12.269238+0000","last_became_active":"2023-09-23T03:17:11.208872+0000","last_became_peered":"2023-09-23T03:17:11.208872+0000","last_unstale":"2023-09-23T03:17:12.269238+0000","last_undegraded":"2023-09-23T03:17:12.269238+0000","last_fullsized":"2023-09-23T03:17:12.269238+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:29:14.484941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.470374+0000","last_change":"2023-09-23T03:17:11.197506+0000","last_active":"2023-09-23T03:17:12.470374+0000","last_peered":"2023-09-23T03:17:12.470374+0000","last_clean":"2023-09-23T03:17:12.470374+0000","last_became_active":"2023-09-23T03:17:11.197241+0000","last_became_peered":"2023-09-23T03:17:11.197241+0000","last_unstale":"2023-09-23T03:17:12.470374+0000","last_undegraded":"2023-09-23T03:17:12.470374+0000","last_fullsized":"2023-09-23T03:17:12.470374+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:04:50.851324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.184022+0000","last_change":"2023-09-23T03:17:11.202976+0000","last_active":"2023-09-23T03:17:12.184022+0000","last_peered":"2023-09-23T03:17:12.184022+0000","last_clean":"2023-09-23T03:17:12.184022+0000","last_became_active":"2023-09-23T03:17:11.200288+0000","last_became_peered":"2023-09-23T03:17:11.200288+0000","last_unstale":"2023-09-23T03:17:12.184022+0000","last_undegraded":"2023-09-23T03:17:12.184022+0000","last_fullsized":"2023-09-23T03:17:12.184022+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:54:53.210843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.181919+0000","last_change":"2023-09-23T03:17:11.195305+0000","last_active":"2023-09-23T03:17:12.181919+0000","last_peered":"2023-09-23T03:17:12.181919+0000","last_clean":"2023-09-23T03:17:12.181919+0000","last_became_active":"2023-09-23T03:17:11.195053+0000","last_became_peered":"2023-09-23T03:17:11.195053+0000","last_unstale":"2023-09-23T03:17:12.181919+0000","last_undegraded":"2023-09-23T03:17:12.181919+0000","last_fullsized":"2023-09-23T03:17:12.181919+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:22:15.164792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.185344+0000","last_change":"2023-09-23T03:17:11.187479+0000","last_active":"2023-09-23T03:17:12.185344+0000","last_peered":"2023-09-23T03:17:12.185344+0000","last_clean":"2023-09-23T03:17:12.185344+0000","last_became_active":"2023-09-23T03:17:11.187101+0000","last_became_peered":"2023-09-23T03:17:11.187101+0000","last_unstale":"2023-09-23T03:17:12.185344+0000","last_undegraded":"2023-09-23T03:17:12.185344+0000","last_fullsized":"2023-09-23T03:17:12.185344+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:16:38.377968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.472781+0000","last_change":"2023-09-23T03:17:11.212271+0000","last_active":"2023-09-23T03:17:12.472781+0000","last_peered":"2023-09-23T03:17:12.472781+0000","last_clean":"2023-09-23T03:17:12.472781+0000","last_became_active":"2023-09-23T03:17:11.211986+0000","last_became_peered":"2023-09-23T03:17:11.211986+0000","last_unstale":"2023-09-23T03:17:12.472781+0000","last_undegraded":"2023-09-23T03:17:12.472781+0000","last_fullsized":"2023-09-23T03:17:12.472781+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:39:36.236424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.185292+0000","last_change":"2023-09-23T03:17:11.184328+0000","last_active":"2023-09-23T03:17:12.185292+0000","last_peered":"2023-09-23T03:17:12.185292+0000","last_clean":"2023-09-23T03:17:12.185292+0000","last_became_active":"2023-09-23T03:17:11.183957+0000","last_became_peered":"2023-09-23T03:17:11.183957+0000","last_unstale":"2023-09-23T03:17:12.185292+0000","last_undegraded":"2023-09-23T03:17:12.185292+0000","last_fullsized":"2023-09-23T03:17:12.185292+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:04:50.851324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.269834+0000","last_change":"2023-09-23T03:17:11.205816+0000","last_active":"2023-09-23T03:17:12.269834+0000","last_peered":"2023-09-23T03:17:12.269834+0000","last_clean":"2023-09-23T03:17:12.269834+0000","last_became_active":"2023-09-23T03:17:11.205507+0000","last_became_peered":"2023-09-23T03:17:11.205507+0000","last_unstale":"2023-09-23T03:17:12.269834+0000","last_undegraded":"2023-09-23T03:17:12.269834+0000","last_fullsized":"2023-09-23T03:17:12.269834+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:35:47.380029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.185011+0000","last_change":"2023-09-23T03:17:11.198594+0000","last_active":"2023-09-23T03:17:12.185011+0000","last_peered":"2023-09-23T03:17:12.185011+0000","last_clean":"2023-09-23T03:17:12.185011+0000","last_became_active":"2023-09-23T03:17:11.198296+0000","last_became_peered":"2023-09-23T03:17:11.198296+0000","last_unstale":"2023-09-23T03:17:12.185011+0000","last_undegraded":"2023-09-23T03:17:12.185011+0000","last_fullsized":"2023-09-23T03:17:12.185011+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:13:52.054858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.185394+0000","last_change":"2023-09-23T03:17:11.191073+0000","last_active":"2023-09-23T03:17:12.185394+0000","last_peered":"2023-09-23T03:17:12.185394+0000","last_clean":"2023-09-23T03:17:12.185394+0000","last_became_active":"2023-09-23T03:17:11.190811+0000","last_became_peered":"2023-09-23T03:17:11.190811+0000","last_unstale":"2023-09-23T03:17:12.185394+0000","last_undegraded":"2023-09-23T03:17:12.185394+0000","last_fullsized":"2023-09-23T03:17:12.185394+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:08:54.761338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.185023+0000","last_change":"2023-09-23T03:17:11.204108+0000","last_active":"2023-09-23T03:17:12.185023+0000","last_peered":"2023-09-23T03:17:12.185023+0000","last_clean":"2023-09-23T03:17:12.185023+0000","last_became_active":"2023-09-23T03:17:11.200467+0000","last_became_peered":"2023-09-23T03:17:11.200467+0000","last_unstale":"2023-09-23T03:17:12.185023+0000","last_undegraded":"2023-09-23T03:17:12.185023+0000","last_fullsized":"2023-09-23T03:17:12.185023+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:07:51.120761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.472080+0000","last_change":"2023-09-23T03:17:11.199169+0000","last_active":"2023-09-23T03:17:12.472080+0000","last_peered":"2023-09-23T03:17:12.472080+0000","last_clean":"2023-09-23T03:17:12.472080+0000","last_became_active":"2023-09-23T03:17:11.198876+0000","last_became_peered":"2023-09-23T03:17:11.198876+0000","last_unstale":"2023-09-23T03:17:12.472080+0000","last_undegraded":"2023-09-23T03:17:12.472080+0000","last_fullsized":"2023-09-23T03:17:12.472080+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:08:54.761338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.185653+0000","last_change":"2023-09-23T03:17:11.204328+0000","last_active":"2023-09-23T03:17:12.185653+0000","last_peered":"2023-09-23T03:17:12.185653+0000","last_clean":"2023-09-23T03:17:12.185653+0000","last_became_active":"2023-09-23T03:17:11.204080+0000","last_became_peered":"2023-09-23T03:17:11.204080+0000","last_unstale":"2023-09-23T03:17:12.185653+0000","last_undegraded":"2023-09-23T03:17:12.185653+0000","last_fullsized":"2023-09-23T03:17:12.185653+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:38:15.647687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.185420+0000","last_change":"2023-09-23T03:17:10.675152+0000","last_active":"2023-09-23T03:17:12.185420+0000","last_peered":"2023-09-23T03:17:12.185420+0000","last_clean":"2023-09-23T03:17:12.185420+0000","last_became_active":"2023-09-23T03:17:10.674843+0000","last_became_peered":"2023-09-23T03:17:10.674843+0000","last_unstale":"2023-09-23T03:17:12.185420+0000","last_undegraded":"2023-09-23T03:17:12.185420+0000","last_fullsized":"2023-09-23T03:17:12.185420+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:12:25.960972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.473086+0000","last_change":"2023-09-23T03:17:11.212311+0000","last_active":"2023-09-23T03:17:12.473086+0000","last_peered":"2023-09-23T03:17:12.473086+0000","last_clean":"2023-09-23T03:17:12.473086+0000","last_became_active":"2023-09-23T03:17:11.212038+0000","last_became_peered":"2023-09-23T03:17:11.212038+0000","last_unstale":"2023-09-23T03:17:12.473086+0000","last_undegraded":"2023-09-23T03:17:12.473086+0000","last_fullsized":"2023-09-23T03:17:12.473086+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:48:40.699566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.469980+0000","last_change":"2023-09-23T03:17:10.669913+0000","last_active":"2023-09-23T03:17:12.469980+0000","last_peered":"2023-09-23T03:17:12.469980+0000","last_clean":"2023-09-23T03:17:12.469980+0000","last_became_active":"2023-09-23T03:17:10.669425+0000","last_became_peered":"2023-09-23T03:17:10.669425+0000","last_unstale":"2023-09-23T03:17:12.469980+0000","last_undegraded":"2023-09-23T03:17:12.469980+0000","last_fullsized":"2023-09-23T03:17:12.469980+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:04:49.837772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.471036+0000","last_change":"2023-09-23T03:17:11.191988+0000","last_active":"2023-09-23T03:17:12.471036+0000","last_peered":"2023-09-23T03:17:12.471036+0000","last_clean":"2023-09-23T03:17:12.471036+0000","last_became_active":"2023-09-23T03:17:11.191721+0000","last_became_peered":"2023-09-23T03:17:11.191721+0000","last_unstale":"2023-09-23T03:17:12.471036+0000","last_undegraded":"2023-09-23T03:17:12.471036+0000","last_fullsized":"2023-09-23T03:17:12.471036+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:55:42.170648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.473257+0000","last_change":"2023-09-23T03:17:10.677903+0000","last_active":"2023-09-23T03:17:12.473257+0000","last_peered":"2023-09-23T03:17:12.473257+0000","last_clean":"2023-09-23T03:17:12.473257+0000","last_became_active":"2023-09-23T03:17:10.677703+0000","last_became_peered":"2023-09-23T03:17:10.677703+0000","last_unstale":"2023-09-23T03:17:12.473257+0000","last_undegraded":"2023-09-23T03:17:12.473257+0000","last_fullsized":"2023-09-23T03:17:12.473257+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:04:49.837772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.471084+0000","last_change":"2023-09-23T03:17:11.187515+0000","last_active":"2023-09-23T03:17:12.471084+0000","last_peered":"2023-09-23T03:17:12.471084+0000","last_clean":"2023-09-23T03:17:12.471084+0000","last_became_active":"2023-09-23T03:17:11.187200+0000","last_became_peered":"2023-09-23T03:17:11.187200+0000","last_unstale":"2023-09-23T03:17:12.471084+0000","last_undegraded":"2023-09-23T03:17:12.471084+0000","last_fullsized":"2023-09-23T03:17:12.471084+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:22:53.611756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.470754+0000","last_change":"2023-09-23T03:17:10.675330+0000","last_active":"2023-09-23T03:17:12.470754+0000","last_peered":"2023-09-23T03:17:12.470754+0000","last_clean":"2023-09-23T03:17:12.470754+0000","last_became_active":"2023-09-23T03:17:10.675098+0000","last_became_peered":"2023-09-23T03:17:10.675098+0000","last_unstale":"2023-09-23T03:17:12.470754+0000","last_undegraded":"2023-09-23T03:17:12.470754+0000","last_fullsized":"2023-09-23T03:17:12.470754+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:54:28.226116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.468826+0000","last_change":"2023-09-23T03:17:11.196664+0000","last_active":"2023-09-23T03:17:12.468826+0000","last_peered":"2023-09-23T03:17:12.468826+0000","last_clean":"2023-09-23T03:17:12.468826+0000","last_became_active":"2023-09-23T03:17:11.196391+0000","last_became_peered":"2023-09-23T03:17:11.196391+0000","last_unstale":"2023-09-23T03:17:12.468826+0000","last_undegraded":"2023-09-23T03:17:12.468826+0000","last_fullsized":"2023-09-23T03:17:12.468826+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:45:37.742495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.269254+0000","last_change":"2023-09-23T03:17:10.670864+0000","last_active":"2023-09-23T03:17:12.269254+0000","last_peered":"2023-09-23T03:17:12.269254+0000","last_clean":"2023-09-23T03:17:12.269254+0000","last_became_active":"2023-09-23T03:17:10.670596+0000","last_became_peered":"2023-09-23T03:17:10.670596+0000","last_unstale":"2023-09-23T03:17:12.269254+0000","last_undegraded":"2023-09-23T03:17:12.269254+0000","last_fullsized":"2023-09-23T03:17:12.269254+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:26:40.873065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.269322+0000","last_change":"2023-09-23T03:17:11.199748+0000","last_active":"2023-09-23T03:17:12.269322+0000","last_peered":"2023-09-23T03:17:12.269322+0000","last_clean":"2023-09-23T03:17:12.269322+0000","last_became_active":"2023-09-23T03:17:11.199443+0000","last_became_peered":"2023-09-23T03:17:11.199443+0000","last_unstale":"2023-09-23T03:17:12.269322+0000","last_undegraded":"2023-09-23T03:17:12.269322+0000","last_fullsized":"2023-09-23T03:17:12.269322+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:50:13.926507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.185527+0000","last_change":"2023-09-23T03:17:10.674112+0000","last_active":"2023-09-23T03:17:12.185527+0000","last_peered":"2023-09-23T03:17:12.185527+0000","last_clean":"2023-09-23T03:17:12.185527+0000","last_became_active":"2023-09-23T03:17:10.673784+0000","last_became_peered":"2023-09-23T03:17:10.673784+0000","last_unstale":"2023-09-23T03:17:12.185527+0000","last_undegraded":"2023-09-23T03:17:12.185527+0000","last_fullsized":"2023-09-23T03:17:12.185527+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:48:39.686014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.470798+0000","last_change":"2023-09-23T03:17:11.210521+0000","last_active":"2023-09-23T03:17:12.470798+0000","last_peered":"2023-09-23T03:17:12.470798+0000","last_clean":"2023-09-23T03:17:12.470798+0000","last_became_active":"2023-09-23T03:17:11.210263+0000","last_became_peered":"2023-09-23T03:17:11.210263+0000","last_unstale":"2023-09-23T03:17:12.470798+0000","last_undegraded":"2023-09-23T03:17:12.470798+0000","last_fullsized":"2023-09-23T03:17:12.470798+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:08:54.761338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.472436+0000","last_change":"2023-09-23T03:17:10.671113+0000","last_active":"2023-09-23T03:17:12.472436+0000","last_peered":"2023-09-23T03:17:12.472436+0000","last_clean":"2023-09-23T03:17:12.472436+0000","last_became_active":"2023-09-23T03:17:10.670893+0000","last_became_peered":"2023-09-23T03:17:10.670893+0000","last_unstale":"2023-09-23T03:17:12.472436+0000","last_undegraded":"2023-09-23T03:17:12.472436+0000","last_fullsized":"2023-09-23T03:17:12.472436+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:24:02.323942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.469549+0000","last_change":"2023-09-23T03:17:11.200984+0000","last_active":"2023-09-23T03:17:12.469549+0000","last_peered":"2023-09-23T03:17:12.469549+0000","last_clean":"2023-09-23T03:17:12.469549+0000","last_became_active":"2023-09-23T03:17:11.200723+0000","last_became_peered":"2023-09-23T03:17:11.200723+0000","last_unstale":"2023-09-23T03:17:12.469549+0000","last_undegraded":"2023-09-23T03:17:12.469549+0000","last_fullsized":"2023-09-23T03:17:12.469549+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:48:30.246329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.469855+0000","last_change":"2023-09-23T03:17:10.672597+0000","last_active":"2023-09-23T03:17:12.469855+0000","last_peered":"2023-09-23T03:17:12.469855+0000","last_clean":"2023-09-23T03:17:12.469855+0000","last_became_active":"2023-09-23T03:17:10.672281+0000","last_became_peered":"2023-09-23T03:17:10.672281+0000","last_unstale":"2023-09-23T03:17:12.469855+0000","last_undegraded":"2023-09-23T03:17:12.469855+0000","last_fullsized":"2023-09-23T03:17:12.469855+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:25:56.275212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.182603+0000","last_change":"2023-09-23T03:17:11.198209+0000","last_active":"2023-09-23T03:17:12.182603+0000","last_peered":"2023-09-23T03:17:12.182603+0000","last_clean":"2023-09-23T03:17:12.182603+0000","last_became_active":"2023-09-23T03:17:11.197878+0000","last_became_peered":"2023-09-23T03:17:11.197878+0000","last_unstale":"2023-09-23T03:17:12.182603+0000","last_undegraded":"2023-09-23T03:17:12.182603+0000","last_fullsized":"2023-09-23T03:17:12.182603+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:25:57.288764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.471282+0000","last_change":"2023-09-23T03:17:10.664834+0000","last_active":"2023-09-23T03:17:12.471282+0000","last_peered":"2023-09-23T03:17:12.471282+0000","last_clean":"2023-09-23T03:17:12.471282+0000","last_became_active":"2023-09-23T03:17:10.664543+0000","last_became_peered":"2023-09-23T03:17:10.664543+0000","last_unstale":"2023-09-23T03:17:12.471282+0000","last_undegraded":"2023-09-23T03:17:12.471282+0000","last_fullsized":"2023-09-23T03:17:12.471282+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:43:08.791421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.186099+0000","last_change":"2023-09-23T03:17:11.204643+0000","last_active":"2023-09-23T03:17:12.186099+0000","last_peered":"2023-09-23T03:17:12.186099+0000","last_clean":"2023-09-23T03:17:12.186099+0000","last_became_active":"2023-09-23T03:17:11.203859+0000","last_became_peered":"2023-09-23T03:17:11.203859+0000","last_unstale":"2023-09-23T03:17:12.186099+0000","last_undegraded":"2023-09-23T03:17:12.186099+0000","last_fullsized":"2023-09-23T03:17:12.186099+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:38:24.272265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.270105+0000","last_change":"2023-09-23T03:17:10.662881+0000","last_active":"2023-09-23T03:17:12.270105+0000","last_peered":"2023-09-23T03:17:12.270105+0000","last_clean":"2023-09-23T03:17:12.270105+0000","last_became_active":"2023-09-23T03:17:10.662647+0000","last_became_peered":"2023-09-23T03:17:10.662647+0000","last_unstale":"2023-09-23T03:17:12.270105+0000","last_undegraded":"2023-09-23T03:17:12.270105+0000","last_fullsized":"2023-09-23T03:17:12.270105+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:45:36.728943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.470176+0000","last_change":"2023-09-23T03:17:11.203051+0000","last_active":"2023-09-23T03:17:12.470176+0000","last_peered":"2023-09-23T03:17:12.470176+0000","last_clean":"2023-09-23T03:17:12.470176+0000","last_became_active":"2023-09-23T03:17:11.202760+0000","last_became_peered":"2023-09-23T03:17:11.202760+0000","last_unstale":"2023-09-23T03:17:12.470176+0000","last_undegraded":"2023-09-23T03:17:12.470176+0000","last_fullsized":"2023-09-23T03:17:12.470176+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:44:08.447318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.269963+0000","last_change":"2023-09-23T03:17:10.669252+0000","last_active":"2023-09-23T03:17:12.269963+0000","last_peered":"2023-09-23T03:17:12.269963+0000","last_clean":"2023-09-23T03:17:12.269963+0000","last_became_active":"2023-09-23T03:17:10.668985+0000","last_became_peered":"2023-09-23T03:17:10.668985+0000","last_unstale":"2023-09-23T03:17:12.269963+0000","last_undegraded":"2023-09-23T03:17:12.269963+0000","last_fullsized":"2023-09-23T03:17:12.269963+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:09:53.714892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.470545+0000","last_change":"2023-09-23T03:17:11.196157+0000","last_active":"2023-09-23T03:17:12.470545+0000","last_peered":"2023-09-23T03:17:12.470545+0000","last_clean":"2023-09-23T03:17:12.470545+0000","last_became_active":"2023-09-23T03:17:11.195906+0000","last_became_peered":"2023-09-23T03:17:11.195906+0000","last_unstale":"2023-09-23T03:17:12.470545+0000","last_undegraded":"2023-09-23T03:17:12.470545+0000","last_fullsized":"2023-09-23T03:17:12.470545+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:16:38.377968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.471447+0000","last_change":"2023-09-23T03:17:10.668386+0000","last_active":"2023-09-23T03:17:12.471447+0000","last_peered":"2023-09-23T03:17:12.471447+0000","last_clean":"2023-09-23T03:17:12.471447+0000","last_became_active":"2023-09-23T03:17:10.667806+0000","last_became_peered":"2023-09-23T03:17:10.667806+0000","last_unstale":"2023-09-23T03:17:12.471447+0000","last_undegraded":"2023-09-23T03:17:12.471447+0000","last_fullsized":"2023-09-23T03:17:12.471447+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:44:07.433766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.470190+0000","last_change":"2023-09-23T03:17:11.201450+0000","last_active":"2023-09-23T03:17:12.470190+0000","last_peered":"2023-09-23T03:17:12.470190+0000","last_clean":"2023-09-23T03:17:12.470190+0000","last_became_active":"2023-09-23T03:17:11.201037+0000","last_became_peered":"2023-09-23T03:17:11.201037+0000","last_unstale":"2023-09-23T03:17:12.470190+0000","last_undegraded":"2023-09-23T03:17:12.470190+0000","last_fullsized":"2023-09-23T03:17:12.470190+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:09:54.728444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.470312+0000","last_change":"2023-09-23T03:17:10.660419+0000","last_active":"2023-09-23T03:17:12.470312+0000","last_peered":"2023-09-23T03:17:12.470312+0000","last_clean":"2023-09-23T03:17:12.470312+0000","last_became_active":"2023-09-23T03:17:10.660152+0000","last_became_peered":"2023-09-23T03:17:10.660152+0000","last_unstale":"2023-09-23T03:17:12.470312+0000","last_undegraded":"2023-09-23T03:17:12.470312+0000","last_fullsized":"2023-09-23T03:17:12.470312+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:09:53.714892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.470732+0000","last_change":"2023-09-23T03:17:11.193191+0000","last_active":"2023-09-23T03:17:12.470732+0000","last_peered":"2023-09-23T03:17:12.470732+0000","last_clean":"2023-09-23T03:17:12.470732+0000","last_became_active":"2023-09-23T03:17:11.189869+0000","last_became_peered":"2023-09-23T03:17:11.189869+0000","last_unstale":"2023-09-23T03:17:12.470732+0000","last_undegraded":"2023-09-23T03:17:12.470732+0000","last_fullsized":"2023-09-23T03:17:12.470732+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:29:14.484941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.472434+0000","last_change":"2023-09-23T03:17:10.674149+0000","last_active":"2023-09-23T03:17:12.472434+0000","last_peered":"2023-09-23T03:17:12.472434+0000","last_clean":"2023-09-23T03:17:12.472434+0000","last_became_active":"2023-09-23T03:17:10.673894+0000","last_became_peered":"2023-09-23T03:17:10.673894+0000","last_unstale":"2023-09-23T03:17:12.472434+0000","last_undegraded":"2023-09-23T03:17:12.472434+0000","last_fullsized":"2023-09-23T03:17:12.472434+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:43:40.947693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.185250+0000","last_change":"2023-09-23T03:17:11.203167+0000","last_active":"2023-09-23T03:17:12.185250+0000","last_peered":"2023-09-23T03:17:12.185250+0000","last_clean":"2023-09-23T03:17:12.185250+0000","last_became_active":"2023-09-23T03:17:11.202873+0000","last_became_peered":"2023-09-23T03:17:11.202873+0000","last_unstale":"2023-09-23T03:17:12.185250+0000","last_undegraded":"2023-09-23T03:17:12.185250+0000","last_fullsized":"2023-09-23T03:17:12.185250+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:43:09.804974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.183910+0000","last_change":"2023-09-23T03:17:10.675461+0000","last_active":"2023-09-23T03:17:12.183910+0000","last_peered":"2023-09-23T03:17:12.183910+0000","last_clean":"2023-09-23T03:17:12.183910+0000","last_became_active":"2023-09-23T03:17:10.672873+0000","last_became_peered":"2023-09-23T03:17:10.672873+0000","last_unstale":"2023-09-23T03:17:12.183910+0000","last_undegraded":"2023-09-23T03:17:12.183910+0000","last_fullsized":"2023-09-23T03:17:12.183910+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:10:48.740740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.469869+0000","last_change":"2023-09-23T03:17:11.190739+0000","last_active":"2023-09-23T03:17:12.469869+0000","last_peered":"2023-09-23T03:17:12.469869+0000","last_clean":"2023-09-23T03:17:12.469869+0000","last_became_active":"2023-09-23T03:17:11.190415+0000","last_became_peered":"2023-09-23T03:17:11.190415+0000","last_unstale":"2023-09-23T03:17:12.469869+0000","last_undegraded":"2023-09-23T03:17:12.469869+0000","last_fullsized":"2023-09-23T03:17:12.469869+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:54:57.579461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.184254+0000","last_change":"2023-09-23T03:17:10.674832+0000","last_active":"2023-09-23T03:17:12.184254+0000","last_peered":"2023-09-23T03:17:12.184254+0000","last_clean":"2023-09-23T03:17:12.184254+0000","last_became_active":"2023-09-23T03:17:10.674508+0000","last_became_peered":"2023-09-23T03:17:10.674508+0000","last_unstale":"2023-09-23T03:17:12.184254+0000","last_undegraded":"2023-09-23T03:17:12.184254+0000","last_fullsized":"2023-09-23T03:17:12.184254+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:13:51.041305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.471209+0000","last_change":"2023-09-23T03:17:11.184116+0000","last_active":"2023-09-23T03:17:12.471209+0000","last_peered":"2023-09-23T03:17:12.471209+0000","last_clean":"2023-09-23T03:17:12.471209+0000","last_became_active":"2023-09-23T03:17:11.183770+0000","last_became_peered":"2023-09-23T03:17:11.183770+0000","last_unstale":"2023-09-23T03:17:12.471209+0000","last_undegraded":"2023-09-23T03:17:12.471209+0000","last_fullsized":"2023-09-23T03:17:12.471209+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:27:00.718764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.269895+0000","last_change":"2023-09-23T03:17:10.673938+0000","last_active":"2023-09-23T03:17:12.269895+0000","last_peered":"2023-09-23T03:17:12.269895+0000","last_clean":"2023-09-23T03:17:12.269895+0000","last_became_active":"2023-09-23T03:17:10.673635+0000","last_became_peered":"2023-09-23T03:17:10.673635+0000","last_unstale":"2023-09-23T03:17:12.269895+0000","last_undegraded":"2023-09-23T03:17:12.269895+0000","last_fullsized":"2023-09-23T03:17:12.269895+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:07:50.107209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.269785+0000","last_change":"2023-09-23T03:17:11.205131+0000","last_active":"2023-09-23T03:17:12.269785+0000","last_peered":"2023-09-23T03:17:12.269785+0000","last_clean":"2023-09-23T03:17:12.269785+0000","last_became_active":"2023-09-23T03:17:11.204850+0000","last_became_peered":"2023-09-23T03:17:11.204850+0000","last_unstale":"2023-09-23T03:17:12.269785+0000","last_undegraded":"2023-09-23T03:17:12.269785+0000","last_fullsized":"2023-09-23T03:17:12.269785+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:55:45.937101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.471135+0000","last_change":"2023-09-23T03:17:10.676267+0000","last_active":"2023-09-23T03:17:12.471135+0000","last_peered":"2023-09-23T03:17:12.471135+0000","last_clean":"2023-09-23T03:17:12.471135+0000","last_became_active":"2023-09-23T03:17:10.675883+0000","last_became_peered":"2023-09-23T03:17:10.675883+0000","last_unstale":"2023-09-23T03:17:12.471135+0000","last_undegraded":"2023-09-23T03:17:12.471135+0000","last_fullsized":"2023-09-23T03:17:12.471135+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:24:02.323942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.470925+0000","last_change":"2023-09-23T03:17:11.196964+0000","last_active":"2023-09-23T03:17:12.470925+0000","last_peered":"2023-09-23T03:17:12.470925+0000","last_clean":"2023-09-23T03:17:12.470925+0000","last_became_active":"2023-09-23T03:17:11.196468+0000","last_became_peered":"2023-09-23T03:17:11.196468+0000","last_unstale":"2023-09-23T03:17:12.470925+0000","last_undegraded":"2023-09-23T03:17:12.470925+0000","last_fullsized":"2023-09-23T03:17:12.470925+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:45:37.742495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.182213+0000","last_change":"2023-09-23T03:17:10.666770+0000","last_active":"2023-09-23T03:17:12.182213+0000","last_peered":"2023-09-23T03:17:12.182213+0000","last_clean":"2023-09-23T03:17:12.182213+0000","last_became_active":"2023-09-23T03:17:10.666474+0000","last_became_peered":"2023-09-23T03:17:10.666474+0000","last_unstale":"2023-09-23T03:17:12.182213+0000","last_undegraded":"2023-09-23T03:17:12.182213+0000","last_fullsized":"2023-09-23T03:17:12.182213+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:12:25.960972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.182473+0000","last_change":"2023-09-23T03:17:11.187777+0000","last_active":"2023-09-23T03:17:12.182473+0000","last_peered":"2023-09-23T03:17:12.182473+0000","last_clean":"2023-09-23T03:17:12.182473+0000","last_became_active":"2023-09-23T03:17:11.187477+0000","last_became_peered":"2023-09-23T03:17:11.187477+0000","last_unstale":"2023-09-23T03:17:12.182473+0000","last_undegraded":"2023-09-23T03:17:12.182473+0000","last_fullsized":"2023-09-23T03:17:12.182473+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:50:02.952823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.472891+0000","last_change":"2023-09-23T03:17:10.678937+0000","last_active":"2023-09-23T03:17:12.472891+0000","last_peered":"2023-09-23T03:17:12.472891+0000","last_clean":"2023-09-23T03:17:12.472891+0000","last_became_active":"2023-09-23T03:17:10.678694+0000","last_became_peered":"2023-09-23T03:17:10.678694+0000","last_unstale":"2023-09-23T03:17:12.472891+0000","last_undegraded":"2023-09-23T03:17:12.472891+0000","last_fullsized":"2023-09-23T03:17:12.472891+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:13:51.041305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.184614+0000","last_change":"2023-09-23T03:17:11.200899+0000","last_active":"2023-09-23T03:17:12.184614+0000","last_peered":"2023-09-23T03:17:12.184614+0000","last_clean":"2023-09-23T03:17:12.184614+0000","last_became_active":"2023-09-23T03:17:11.200669+0000","last_became_peered":"2023-09-23T03:17:11.200669+0000","last_unstale":"2023-09-23T03:17:12.184614+0000","last_undegraded":"2023-09-23T03:17:12.184614+0000","last_fullsized":"2023-09-23T03:17:12.184614+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:26:41.886617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.182000+0000","last_change":"2023-09-23T03:17:10.662742+0000","last_active":"2023-09-23T03:17:12.182000+0000","last_peered":"2023-09-23T03:17:12.182000+0000","last_clean":"2023-09-23T03:17:12.182000+0000","last_became_active":"2023-09-23T03:17:10.662441+0000","last_became_peered":"2023-09-23T03:17:10.662441+0000","last_unstale":"2023-09-23T03:17:12.182000+0000","last_undegraded":"2023-09-23T03:17:12.182000+0000","last_fullsized":"2023-09-23T03:17:12.182000+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:28:57.388433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.470086+0000","last_change":"2023-09-23T03:17:11.203572+0000","last_active":"2023-09-23T03:17:12.470086+0000","last_peered":"2023-09-23T03:17:12.470086+0000","last_clean":"2023-09-23T03:17:12.470086+0000","last_became_active":"2023-09-23T03:17:11.203326+0000","last_became_peered":"2023-09-23T03:17:11.203326+0000","last_unstale":"2023-09-23T03:17:12.470086+0000","last_undegraded":"2023-09-23T03:17:12.470086+0000","last_fullsized":"2023-09-23T03:17:12.470086+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:48:40.699566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.182022+0000","last_change":"2023-09-23T03:17:10.674511+0000","last_active":"2023-09-23T03:17:12.182022+0000","last_peered":"2023-09-23T03:17:12.182022+0000","last_clean":"2023-09-23T03:17:12.182022+0000","last_became_active":"2023-09-23T03:17:10.674212+0000","last_became_peered":"2023-09-23T03:17:10.674212+0000","last_unstale":"2023-09-23T03:17:12.182022+0000","last_undegraded":"2023-09-23T03:17:12.182022+0000","last_fullsized":"2023-09-23T03:17:12.182022+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:28:43.374996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.269376+0000","last_change":"2023-09-23T03:17:11.205308+0000","last_active":"2023-09-23T03:17:12.269376+0000","last_peered":"2023-09-23T03:17:12.269376+0000","last_clean":"2023-09-23T03:17:12.269376+0000","last_became_active":"2023-09-23T03:17:11.205000+0000","last_became_peered":"2023-09-23T03:17:11.205000+0000","last_unstale":"2023-09-23T03:17:12.269376+0000","last_undegraded":"2023-09-23T03:17:12.269376+0000","last_fullsized":"2023-09-23T03:17:12.269376+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:22:53.611756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.185266+0000","last_change":"2023-09-23T03:17:10.664662+0000","last_active":"2023-09-23T03:17:12.185266+0000","last_peered":"2023-09-23T03:17:12.185266+0000","last_clean":"2023-09-23T03:17:12.185266+0000","last_became_active":"2023-09-23T03:17:10.663348+0000","last_became_peered":"2023-09-23T03:17:10.663348+0000","last_unstale":"2023-09-23T03:17:12.185266+0000","last_undegraded":"2023-09-23T03:17:12.185266+0000","last_fullsized":"2023-09-23T03:17:12.185266+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:45:36.728943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.470168+0000","last_change":"2023-09-23T03:17:11.201753+0000","last_active":"2023-09-23T03:17:12.470168+0000","last_peered":"2023-09-23T03:17:12.470168+0000","last_clean":"2023-09-23T03:17:12.470168+0000","last_became_active":"2023-09-23T03:17:11.201135+0000","last_became_peered":"2023-09-23T03:17:11.201135+0000","last_unstale":"2023-09-23T03:17:12.470168+0000","last_undegraded":"2023-09-23T03:17:12.470168+0000","last_fullsized":"2023-09-23T03:17:12.470168+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:38:24.272265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.181851+0000","last_change":"2023-09-23T03:17:10.661978+0000","last_active":"2023-09-23T03:17:12.181851+0000","last_peered":"2023-09-23T03:17:12.181851+0000","last_clean":"2023-09-23T03:17:12.181851+0000","last_became_active":"2023-09-23T03:17:10.661687+0000","last_became_peered":"2023-09-23T03:17:10.661687+0000","last_unstale":"2023-09-23T03:17:12.181851+0000","last_undegraded":"2023-09-23T03:17:12.181851+0000","last_fullsized":"2023-09-23T03:17:12.181851+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:15:06.006263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.472688+0000","last_change":"2023-09-23T03:17:11.207631+0000","last_active":"2023-09-23T03:17:12.472688+0000","last_peered":"2023-09-23T03:17:12.472688+0000","last_clean":"2023-09-23T03:17:12.472688+0000","last_became_active":"2023-09-23T03:17:11.207309+0000","last_became_peered":"2023-09-23T03:17:11.207309+0000","last_unstale":"2023-09-23T03:17:12.472688+0000","last_undegraded":"2023-09-23T03:17:12.472688+0000","last_fullsized":"2023-09-23T03:17:12.472688+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:19:14.912945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.270144+0000","last_change":"2023-09-23T03:17:10.672697+0000","last_active":"2023-09-23T03:17:12.270144+0000","last_peered":"2023-09-23T03:17:12.270144+0000","last_clean":"2023-09-23T03:17:12.270144+0000","last_became_active":"2023-09-23T03:17:10.672369+0000","last_became_peered":"2023-09-23T03:17:10.672369+0000","last_unstale":"2023-09-23T03:17:12.270144+0000","last_undegraded":"2023-09-23T03:17:12.270144+0000","last_fullsized":"2023-09-23T03:17:12.270144+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:43:08.791421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.186351+0000","last_change":"2023-09-23T03:17:11.204819+0000","last_active":"2023-09-23T03:17:12.186351+0000","last_peered":"2023-09-23T03:17:12.186351+0000","last_clean":"2023-09-23T03:17:12.186351+0000","last_became_active":"2023-09-23T03:17:11.204351+0000","last_became_peered":"2023-09-23T03:17:11.204351+0000","last_unstale":"2023-09-23T03:17:12.186351+0000","last_undegraded":"2023-09-23T03:17:12.186351+0000","last_fullsized":"2023-09-23T03:17:12.186351+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:29:14.484941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.836150+0000","last_change":"2023-09-23T03:17:10.662393+0000","last_active":"2023-09-23T03:17:12.836150+0000","last_peered":"2023-09-23T03:17:12.836150+0000","last_clean":"2023-09-23T03:17:12.836150+0000","last_became_active":"2023-09-23T03:17:10.662131+0000","last_became_peered":"2023-09-23T03:17:10.662131+0000","last_unstale":"2023-09-23T03:17:12.836150+0000","last_undegraded":"2023-09-23T03:17:12.836150+0000","last_fullsized":"2023-09-23T03:17:12.836150+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:38:23.642804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.470735+0000","last_change":"2023-09-23T03:17:11.194286+0000","last_active":"2023-09-23T03:17:12.470735+0000","last_peered":"2023-09-23T03:17:12.470735+0000","last_clean":"2023-09-23T03:17:12.470735+0000","last_became_active":"2023-09-23T03:17:11.194000+0000","last_became_peered":"2023-09-23T03:17:11.194000+0000","last_unstale":"2023-09-23T03:17:12.470735+0000","last_undegraded":"2023-09-23T03:17:12.470735+0000","last_fullsized":"2023-09-23T03:17:12.470735+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:50:13.926507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.469690+0000","last_change":"2023-09-23T03:17:10.665420+0000","last_active":"2023-09-23T03:17:12.469690+0000","last_peered":"2023-09-23T03:17:12.469690+0000","last_clean":"2023-09-23T03:17:12.469690+0000","last_became_active":"2023-09-23T03:17:10.665158+0000","last_became_peered":"2023-09-23T03:17:10.665158+0000","last_unstale":"2023-09-23T03:17:12.469690+0000","last_undegraded":"2023-09-23T03:17:12.469690+0000","last_fullsized":"2023-09-23T03:17:12.469690+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:13:51.041305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.468967+0000","last_change":"2023-09-23T03:17:11.191126+0000","last_active":"2023-09-23T03:17:12.468967+0000","last_peered":"2023-09-23T03:17:12.468967+0000","last_clean":"2023-09-23T03:17:12.468967+0000","last_became_active":"2023-09-23T03:17:11.190749+0000","last_became_peered":"2023-09-23T03:17:11.190749+0000","last_unstale":"2023-09-23T03:17:12.468967+0000","last_undegraded":"2023-09-23T03:17:12.468967+0000","last_fullsized":"2023-09-23T03:17:12.468967+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:55:42.170648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.470316+0000","last_change":"2023-09-23T03:17:10.664455+0000","last_active":"2023-09-23T03:17:12.470316+0000","last_peered":"2023-09-23T03:17:12.470316+0000","last_clean":"2023-09-23T03:17:12.470316+0000","last_became_active":"2023-09-23T03:17:10.663712+0000","last_became_peered":"2023-09-23T03:17:10.663712+0000","last_unstale":"2023-09-23T03:17:12.470316+0000","last_undegraded":"2023-09-23T03:17:12.470316+0000","last_fullsized":"2023-09-23T03:17:12.470316+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:16:37.364416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.472382+0000","last_change":"2023-09-23T03:17:11.202707+0000","last_active":"2023-09-23T03:17:12.472382+0000","last_peered":"2023-09-23T03:17:12.472382+0000","last_clean":"2023-09-23T03:17:12.472382+0000","last_became_active":"2023-09-23T03:17:11.202457+0000","last_became_peered":"2023-09-23T03:17:11.202457+0000","last_unstale":"2023-09-23T03:17:12.472382+0000","last_undegraded":"2023-09-23T03:17:12.472382+0000","last_fullsized":"2023-09-23T03:17:12.472382+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:55:42.170648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.185522+0000","last_change":"2023-09-23T03:17:10.661244+0000","last_active":"2023-09-23T03:17:12.185522+0000","last_peered":"2023-09-23T03:17:12.185522+0000","last_clean":"2023-09-23T03:17:12.185522+0000","last_became_active":"2023-09-23T03:17:10.660916+0000","last_became_peered":"2023-09-23T03:17:10.660916+0000","last_unstale":"2023-09-23T03:17:12.185522+0000","last_undegraded":"2023-09-23T03:17:12.185522+0000","last_fullsized":"2023-09-23T03:17:12.185522+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:25:56.275212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.470002+0000","last_change":"2023-09-23T03:17:11.193935+0000","last_active":"2023-09-23T03:17:12.470002+0000","last_peered":"2023-09-23T03:17:12.470002+0000","last_clean":"2023-09-23T03:17:12.470002+0000","last_became_active":"2023-09-23T03:17:11.193672+0000","last_became_peered":"2023-09-23T03:17:11.193672+0000","last_unstale":"2023-09-23T03:17:12.470002+0000","last_undegraded":"2023-09-23T03:17:12.470002+0000","last_fullsized":"2023-09-23T03:17:12.470002+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:43:41.961245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.469855+0000","last_change":"2023-09-23T03:17:10.674754+0000","last_active":"2023-09-23T03:17:12.469855+0000","last_peered":"2023-09-23T03:17:12.469855+0000","last_clean":"2023-09-23T03:17:12.469855+0000","last_became_active":"2023-09-23T03:17:10.674470+0000","last_became_peered":"2023-09-23T03:17:10.674470+0000","last_unstale":"2023-09-23T03:17:12.469855+0000","last_undegraded":"2023-09-23T03:17:12.469855+0000","last_fullsized":"2023-09-23T03:17:12.469855+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:50:01.939271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.185987+0000","last_change":"2023-09-23T03:17:11.204835+0000","last_active":"2023-09-23T03:17:12.185987+0000","last_peered":"2023-09-23T03:17:12.185987+0000","last_clean":"2023-09-23T03:17:12.185987+0000","last_became_active":"2023-09-23T03:17:11.204240+0000","last_became_peered":"2023-09-23T03:17:11.204240+0000","last_unstale":"2023-09-23T03:17:12.185987+0000","last_undegraded":"2023-09-23T03:17:12.185987+0000","last_fullsized":"2023-09-23T03:17:12.185987+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:54:57.579461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.469274+0000","last_change":"2023-09-23T03:17:10.664308+0000","last_active":"2023-09-23T03:17:12.469274+0000","last_peered":"2023-09-23T03:17:12.469274+0000","last_clean":"2023-09-23T03:17:12.469274+0000","last_became_active":"2023-09-23T03:17:10.663824+0000","last_became_peered":"2023-09-23T03:17:10.663824+0000","last_unstale":"2023-09-23T03:17:12.469274+0000","last_undegraded":"2023-09-23T03:17:12.469274+0000","last_fullsized":"2023-09-23T03:17:12.469274+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:45:36.728943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.269326+0000","last_change":"2023-09-23T03:17:11.212740+0000","last_active":"2023-09-23T03:17:12.269326+0000","last_peered":"2023-09-23T03:17:12.269326+0000","last_clean":"2023-09-23T03:17:12.269326+0000","last_became_active":"2023-09-23T03:17:11.211922+0000","last_became_peered":"2023-09-23T03:17:11.211922+0000","last_unstale":"2023-09-23T03:17:12.269326+0000","last_undegraded":"2023-09-23T03:17:12.269326+0000","last_fullsized":"2023-09-23T03:17:12.269326+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:10.627478+0000","last_clean_scrub_stamp":"2023-09-23T03:17:10.627478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:43:28.809579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.185887+0000","last_change":"2023-09-23T03:17:10.668744+0000","last_active":"2023-09-23T03:17:12.185887+0000","last_peered":"2023-09-23T03:17:12.185887+0000","last_clean":"2023-09-23T03:17:12.185887+0000","last_became_active":"2023-09-23T03:17:10.668277+0000","last_became_peered":"2023-09-23T03:17:10.668277+0000","last_unstale":"2023-09-23T03:17:12.185887+0000","last_undegraded":"2023-09-23T03:17:12.185887+0000","last_fullsized":"2023-09-23T03:17:12.185887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:17:09.613926+0000","last_clean_scrub_stamp":"2023-09-23T03:17:09.613926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:19:13.899393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":179,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-09-23T03:17:12.184858+0000","last_change":"2023-09-23T03:15:11.042447+0000","last_active":"2023-09-23T03:17:12.184858+0000","last_peered":"2023-09-23T03:17:12.184858+0000","last_clean":"2023-09-23T03:17:12.184858+0000","last_became_active":"2023-09-23T03:15:11.040322+0000","last_became_peered":"2023-09-23T03:15:11.040322+0000","last_unstale":"2023-09-23T03:17:12.184858+0000","last_undegraded":"2023-09-23T03:17:12.184858+0000","last_fullsized":"2023-09-23T03:17:12.184858+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_clean_scrub_stamp":"2023-09-23T03:15:09.474842+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:18:36.331645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343796,"statfs":{"total":96636764160,"available":96608047104,"internally_reserved":0,"allocated":1048576,"data_stored":781025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.65600000000000003,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.8049999999999999,"5min":2.149,"15min":2.149},"last":1.008},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.048,"15min":1.044},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.526,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":1.24}]},{"osd":1,"last update":"Sat Sep 23 03:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.224,"15min":1.2290000000000001},"min":{"1min":0.747,"5min":0.504,"15min":0.504},"max":{"1min":1.6950000000000001,"5min":3.1859999999999999,"15min":3.1859999999999999},"last":1.163},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.3740000000000001,"15min":1.403},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.591,"5min":2.613,"15min":2.613},"last":1.095}]},{"osd":2,"last update":"Sat Sep 23 03:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.151,"15min":1.1639999999999999},"min":{"1min":0.69099999999999995,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.393,"5min":2.331,"15min":2.331},"last":1.504},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.2949999999999999,"15min":1.3140000000000001},"min":{"1min":0.72399999999999998,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.0609999999999999,"5min":3.093,"15min":3.093},"last":1.4099999999999999}]},{"osd":3,"last update":"Sat Sep 23 03:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.298,"15min":1.3160000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.641,"5min":2.532,"15min":2.532},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.4350000000000001,"15min":1.4610000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.9570000000000001,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":0.80000000000000004}]},{"osd":4,"last update":"Sat Sep 23 03:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.514,"15min":1.5609999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":2,"5min":2.7210000000000001,"15min":2.7210000000000001},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.395,"5min":1.5569999999999999,"15min":1.5840000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.8740000000000001,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":1.0429999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.4710000000000001,"15min":1.4990000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.835,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":1.276},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.5469999999999999,"15min":1.5800000000000001},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.8,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":1.3500000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.6459999999999999,"15min":1.6759999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.9039999999999999,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":1.55},{"interface":"front","average":{"1min":1.389,"5min":1.6020000000000001,"15min":1.6379999999999999},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":2.1429999999999998,"5min":3.6739999999999999,"15min":3.6739999999999999},"last":1.625}]}]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":442368,"data_stored":189557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.016,"15min":1.018},"min":{"1min":0.63200000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.444,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.03},{"interface":"front","average":{"1min":0.98999999999999999,"5min":1.046,"15min":1.0549999999999999},"min":{"1min":0.67400000000000004,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.403,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.68899999999999995}]},{"osd":1,"last update":"Sat Sep 23 03:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.085,"15min":1.0900000000000001},"min":{"1min":0.63500000000000001,"5min":0.438,"15min":0.438},"max":{"1min":1.867,"5min":1.867,"15min":1.867},"last":1.867},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.24,"15min":1.25},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":1.1579999999999999}]},{"osd":2,"last update":"Sat Sep 23 03:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.0900000000000001,"15min":1.0960000000000001},"min":{"1min":0.79900000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.51,"5min":1.597,"15min":1.597},"last":0.84599999999999997},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2749999999999999,"15min":1.2809999999999999},"min":{"1min":0.79600000000000004,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.8080000000000001,"5min":1.903,"15min":1.903},"last":1.8080000000000001}]},{"osd":3,"last update":"Sat Sep 23 03:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.357,"15min":1.367},"min":{"1min":0.96099999999999997,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.7410000000000001,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":1.7410000000000001},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.4550000000000001,"15min":1.4770000000000001},"min":{"1min":0.93200000000000005,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.8069999999999999,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.202}]},{"osd":4,"last update":"Sat Sep 23 03:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"min":{"1min":0.94499999999999995,"5min":0.498,"15min":0.498},"max":{"1min":1.804,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.121},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.2749999999999999,"15min":1.2909999999999999},"min":{"1min":0.68700000000000006,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.512,"5min":2.2109999999999999,"15min":2.2109999999999999},"last":0.88700000000000001}]},{"osd":5,"last update":"Sat Sep 23 03:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.3859999999999999,"15min":1.393},"min":{"1min":0.75800000000000001,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.722,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.5600000000000001,"5min":1.5029999999999999,"15min":1.4930000000000001},"min":{"1min":0.83599999999999997,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":4.758,"5min":4.758,"15min":4.758},"last":4.758}]},{"osd":7,"last update":"Sat Sep 23 03:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":1.5509999999999999,"15min":1.569},"min":{"1min":0.86899999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.859,"5min":2.476,"15min":2.476},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.431,"5min":1.5509999999999999,"15min":1.571},"min":{"1min":0.90700000000000003,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.8360000000000001,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":0.90700000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":189087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0309999999999999,"15min":1.026},"min":{"1min":0.68600000000000005,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.0189999999999999,"15min":1.018},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.4830000000000001,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.464}]},{"osd":1,"last update":"Sat Sep 23 03:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.2070000000000001,"15min":1.2230000000000001},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.7010000000000001,"5min":1.849,"15min":1.849},"last":1.282},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.304,"15min":1.3280000000000001},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.625,"5min":1.893,"15min":1.893},"last":1.3480000000000001}]},{"osd":2,"last update":"Sat Sep 23 03:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.1950000000000001,"15min":1.2090000000000001},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.6319999999999999,"5min":1.681,"15min":1.681},"last":1.085},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.288,"15min":1.3120000000000001},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.6200000000000001,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.996}]},{"osd":3,"last update":"Sat Sep 23 03:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.3540000000000001,"15min":1.395},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.6719999999999999,"5min":1.905,"15min":1.905},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.254,"5min":1.4610000000000001,"15min":1.4950000000000001},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":2.024,"5min":3.6110000000000002,"15min":3.6110000000000002},"last":0.86099999999999999}]},{"osd":4,"last update":"Sat Sep 23 03:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.4359999999999999,"15min":1.458},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.9199999999999999,"5min":2.069,"15min":2.069},"last":1.752},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.456,"15min":1.472},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.847,"5min":2.319,"15min":2.319},"last":1.9179999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.5249999999999999,"15min":1.546},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.1440000000000001,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":1.649},{"interface":"front","average":{"1min":1.444,"5min":1.6120000000000001,"15min":1.6399999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.8799999999999999,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.7989999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:17:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.532,"15min":1.5629999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.8400000000000001,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.21},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.645,"15min":1.663},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":2.1019999999999999,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.5840000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":189579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.1259999999999999,"15min":1.1379999999999999},"min":{"1min":0.74199999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.5029999999999999,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.1499999999999999,"15min":1.133},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.6579999999999999,"5min":1.9179999999999999,"15min":1.9179999999999999},"last":1.381}]},{"osd":1,"last update":"Sat Sep 23 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.2030000000000001,"15min":1.212},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.7030000000000001,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.228,"15min":1.234},"min":{"1min":0.83899999999999997,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.6000000000000001,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":1.268}]},{"osd":2,"last update":"Sat Sep 23 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.2390000000000001,"15min":1.2629999999999999},"min":{"1min":0.72499999999999998,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.2989999999999999,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.22,"15min":1.2150000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.6080000000000001,"5min":1.663,"15min":1.663},"last":1.137}]},{"osd":3,"last update":"Sat Sep 23 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.1679999999999999,"15min":1.1779999999999999},"min":{"1min":0.76700000000000002,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3400000000000001,"15min":1.3480000000000001},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.651,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.4970000000000001}]},{"osd":5,"last update":"Sat Sep 23 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.7390000000000001,"5min":2.097,"15min":2.097},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.327,"5min":1.325,"15min":1.325},"min":{"1min":0.874,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.6599999999999999,"5min":1.857,"15min":1.857},"last":1.423}]},{"osd":6,"last update":"Sat Sep 23 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.361,"15min":1.379},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.5700000000000001,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.452,"5min":1.462,"15min":1.464},"min":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":1.7230000000000001,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.2330000000000001}]},{"osd":7,"last update":"Sat Sep 23 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.4570000000000001,"15min":1.4710000000000001},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.665,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.099},{"interface":"front","average":{"1min":1.458,"5min":1.546,"15min":1.5609999999999999},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.7470000000000001,"5min":2.137,"15min":2.137},"last":1.323}]}]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":190621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.1240000000000001}]},{"osd":1,"last update":"Sat Sep 23 03:16:10 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":1.2310000000000001},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.82499999999999996}]},{"osd":2,"last update":"Sat Sep 23 03:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":2.2240000000000002,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.145,"5min":1.145,"15min":1.145},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.635,"5min":1.635,"15min":1.635},"last":0.99299999999999999}]},{"osd":4,"last update":"Sat Sep 23 03:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.698,"5min":2.698,"15min":2.698},"last":1.734},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"min":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"max":{"1min":2.419,"5min":2.419,"15min":2.419},"last":1.599}]},{"osd":5,"last update":"Sat Sep 23 03:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.373,"15min":1.373},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.391,"5min":1.391,"15min":1.391},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.0070000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.4390000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":2.6440000000000001,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.502,"5min":1.502,"15min":1.502},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.0600000000000001,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":1.2869999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:16:10 2023","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.544,"15min":1.544},"min":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"max":{"1min":2.7879999999999998,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.552},{"interface":"front","average":{"1min":1.569,"5min":1.569,"15min":1.569},"min":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"max":{"1min":2.5379999999999998,"5min":2.5379999999999998,"15min":2.5379999999999998},"last":1.476}]}]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":442368,"data_stored":189045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.81100000000000005,"15min":0.82799999999999996},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1719999999999999,"5min":1.232,"15min":1.232},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.81799999999999995,"15min":0.82799999999999996},"min":{"1min":0.38900000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.71099999999999997}]},{"osd":1,"last update":"Sat Sep 23 03:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.83999999999999997,"15min":0.85799999999999998},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.23,"5min":1.454,"15min":1.454},"last":0.751},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.92500000000000004,"15min":0.93899999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.3620000000000001,"5min":1.452,"15min":1.452},"last":0.45400000000000001}]},{"osd":3,"last update":"Sat Sep 23 03:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.92600000000000005,"15min":0.93799999999999994},"min":{"1min":0.45200000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.462,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.98499999999999999,"15min":0.997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.3520000000000001,"5min":1.78,"15min":1.78},"last":0.91500000000000004}]},{"osd":4,"last update":"Sat Sep 23 03:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.3049999999999999,"15min":1.335},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.4990000000000001,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.127,"5min":1.3160000000000001,"15min":1.3480000000000001},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.8660000000000001,"5min":2.234,"15min":2.234},"last":1.3899999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.387,"15min":1.425},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.5229999999999999,"5min":3.9409999999999998,"15min":3.9409999999999998},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.214,"5min":1.514,"15min":1.5640000000000001},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.649,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":1.1279999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.387,"15min":1.427},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.724,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":1.085},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.4630000000000001,"15min":1.5009999999999999},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"max":{"1min":1.647,"5min":4.5800000000000001,"15min":4.5800000000000001},"last":1.0289999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:17:10 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.3839999999999999,"15min":1.4099999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.5600000000000001,"5min":2.3149999999999999,"15min":2.3149999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.246,"5min":1.4299999999999999,"15min":1.4610000000000001},"min":{"1min":0.85399999999999998,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.575,"5min":2.621,"15min":2.621},"last":1.262}]}]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":189065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3],"upper_bound":8},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:17:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.80400000000000005,"15min":0.81000000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":2.8279999999999998,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":2.8279999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.84299999999999997,"15min":0.85099999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.6379999999999999,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":1.6379999999999999}]},{"osd":2,"last update":"Sat Sep 23 03:17:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.88500000000000001,"15min":0.90500000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.482,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":1.482},{"interface":"front","average":{"1min":0.89700000000000002,"5min":1.022,"15min":1.0429999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5109999999999999,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.5109999999999999}]},{"osd":3,"last update":"Sat Sep 23 03:17:11 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":1.069,"15min":1.101},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.6819999999999999,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.92700000000000005,"5min":1.141,"15min":1.177},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.548,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":0.45900000000000002}]},{"osd":4,"last update":"Sat Sep 23 03:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.268,"15min":1.294},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.4630000000000001,"5min":3.2629999999999999,"15min":3.2629999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.228,"5min":1.3009999999999999,"15min":1.3129999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.4689999999999999,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":2.4689999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.294,"15min":1.3120000000000001},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.593,"5min":3.161,"15min":3.161},"last":1.593},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.3300000000000001,"15min":1.3560000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.6040000000000001,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":1.532}]},{"osd":6,"last update":"Sat Sep 23 03:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.323,"15min":1.349},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.631,"5min":3.0179999999999998,"15min":3.0179999999999998},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.389,"15min":1.411},"min":{"1min":0.95299999999999996,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.5660000000000001,"5min":3.3580000000000001,"15min":3.3580000000000001},"last":1.419}]},{"osd":7,"last update":"Sat Sep 23 03:17:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.409,"15min":1.4279999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":2.0369999999999999,"5min":3.1099999999999999,"15min":3.1099999999999999},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.286,"5min":1.468,"15min":1.498},"min":{"1min":0.876,"5min":0.876,"15min":0.876},"max":{"1min":1.8899999999999999,"5min":3.306,"15min":3.306},"last":1.117}]}]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343812,"statfs":{"total":96636764160,"available":96608063488,"internally_reserved":0,"allocated":1032192,"data_stored":778358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 23 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.44900000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.2749999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.89300000000000002,"15min":0.90200000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.2330000000000001,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":1.1519999999999999}]},{"osd":2,"last update":"Sat Sep 23 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.79200000000000004,"15min":0.79600000000000004},"min":{"1min":0.54500000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.747},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.98099999999999998,"15min":0.98799999999999999},"min":{"1min":0.51000000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.4359999999999999,"5min":1.544,"15min":1.544},"last":0.79300000000000004}]},{"osd":3,"last update":"Sat Sep 23 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":0.93400000000000005,"15min":0.91600000000000004},"min":{"1min":0.60199999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.999},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.98699999999999999,"15min":1.0029999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.504,"5min":1.607,"15min":1.607},"last":1.0960000000000001}]},{"osd":4,"last update":"Sat Sep 23 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.2010000000000001,"15min":1.2030000000000001},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":1.619,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.526},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.278,"15min":1.286},"min":{"1min":0.88400000000000001,"5min":0.751,"15min":0.751},"max":{"1min":1.591,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.052}]},{"osd":5,"last update":"Sat Sep 23 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.2190000000000001,"15min":1.2270000000000001},"min":{"1min":0.78800000000000003,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.6060000000000001,"5min":1.867,"15min":1.867},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.2649999999999999,"15min":1.262},"min":{"1min":0.995,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.7430000000000001,"5min":1.776,"15min":1.776},"last":1.3380000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.2829999999999999,"15min":1.2969999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.651,"5min":1.887,"15min":1.887},"last":1.21},{"interface":"front","average":{"1min":1.26,"5min":1.2849999999999999,"15min":1.2889999999999999},"min":{"1min":0.85599999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.9019999999999999,"5min":1.9019999999999999,"15min":1.9019999999999999},"last":1.254}]},{"osd":7,"last update":"Sat Sep 23 03:17:12 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.3240000000000001,"15min":1.3140000000000001},"min":{"1min":0.83499999999999996,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.9430000000000001,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":1.387},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.415,"15min":1.4119999999999999},"min":{"1min":1.0189999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":1.4850000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-23T03:17:16.598 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-09-23T03:17:16.598 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-09-23T03:17:16.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.844+0000 7ff6a2056640 1 Processor -- start 2023-09-23T03:17:16.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.844+0000 7ff6a2056640 1 -- start start 2023-09-23T03:17:16.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.846+0000 7ff6a2056640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff69c15cc60 0x7ff69c069cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:16.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.846+0000 7ff6a2056640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff69c0cd040 0x7ff69c0c9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:16.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.846+0000 7ff6a2056640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff69c06ae00 con 0x7ff69c070780 2023-09-23T03:17:16.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.846+0000 7ff6a2056640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff69c0c94e0 con 0x7ff69c15cc60 2023-09-23T03:17:16.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.846+0000 7ff6a2056640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c15d040 con 0x7ff69c0cd040 2023-09-23T03:17:16.848 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.846+0000 7ff69bfff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff69c0cd040 0x7ff69c0c9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:16.848 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.847+0000 7ff692ffd640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7ff69c070780 0x7ff69c0cc930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.154:48156/0 (socket says 172.21.15.154:48156) 2023-09-23T03:17:16.848 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.847+0000 7ff692ffd640 1 -- 172.21.15.154:0/18075310 learned_addr learned my addr 172.21.15.154:0/18075310 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:17:16.848 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.847+0000 7ff69b7fe640 1 --2- 172.21.15.154:0/18075310 >> v2:172.21.15.190:3300/0 conn(0x7ff69c15cc60 0x7ff69c069cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:16.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.847+0000 7ff69b7fe640 1 -- 172.21.15.154:0/18075310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff69c0cd040 msgr2=0x7ff69c0c9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:16.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.847+0000 7ff69b7fe640 1 --2- 172.21.15.154:0/18075310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff69c0cd040 0x7ff69c0c9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:16.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.847+0000 7ff69b7fe640 1 -- 172.21.15.154:0/18075310 >> v1:172.21.15.190:6789/0 conn(0x7ff69c070780 legacy=0x7ff69c0cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:16.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.847+0000 7ff69b7fe640 1 -- 172.21.15.154:0/18075310 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff69c06a560 con 0x7ff69c15cc60 2023-09-23T03:17:16.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.849+0000 7ff69b7fe640 1 --2- 172.21.15.154:0/18075310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff69c15cc60 0x7ff69c069cd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff68c0097a0 tx=0x7ff68c02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40b45cab7f8fd50b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:16.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.849+0000 7ff69affd640 1 -- 172.21.15.154:0/18075310 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68c02d040 con 0x7ff69c15cc60 2023-09-23T03:17:16.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.849+0000 7ff69affd640 1 -- 172.21.15.154:0/18075310 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff68c031810 con 0x7ff69c15cc60 2023-09-23T03:17:16.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.849+0000 7ff69affd640 1 -- 172.21.15.154:0/18075310 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68c031ba0 con 0x7ff69c15cc60 2023-09-23T03:17:16.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.850+0000 7ff6a2056640 1 -- 172.21.15.154:0/18075310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff69c15cc60 msgr2=0x7ff69c069cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:16.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.850+0000 7ff6a2056640 1 --2- 172.21.15.154:0/18075310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff69c15cc60 0x7ff69c069cd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff68c0097a0 tx=0x7ff68c02b8b0 comp rx=0 tx=0).stop 2023-09-23T03:17:16.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.850+0000 7ff6a2056640 1 -- 172.21.15.154:0/18075310 shutdown_connections 2023-09-23T03:17:16.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.850+0000 7ff6a2056640 1 --2- 172.21.15.154:0/18075310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff69c0cd040 0x7ff69c0c9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:16.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.850+0000 7ff6a2056640 1 --2- 172.21.15.154:0/18075310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff69c15cc60 0x7ff69c069cd0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:16.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.850+0000 7ff6a2056640 1 -- 172.21.15.154:0/18075310 >> 172.21.15.154:0/18075310 conn(0x7ff69c0ced60 msgr2=0x7ff69c0cf170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:16.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.850+0000 7ff6a2056640 1 -- 172.21.15.154:0/18075310 shutdown_connections 2023-09-23T03:17:16.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.850+0000 7ff6a2056640 1 -- 172.21.15.154:0/18075310 wait complete. 2023-09-23T03:17:16.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.852+0000 7ff6a2056640 1 Processor -- start 2023-09-23T03:17:16.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.852+0000 7ff6a2056640 1 -- start start 2023-09-23T03:17:16.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.853+0000 7ff6a2056640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff69c070780 0x7ff69c1510f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:16.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.853+0000 7ff6a2056640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff69c0cd040 0x7ff69c0af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:16.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.853+0000 7ff6a2056640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff69c15cc60 0x7ff69c0af8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:16.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.853+0000 7ff6a2056640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c0c8ca0 con 0x7ff69c0cd040 2023-09-23T03:17:16.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.853+0000 7ff6a2056640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c0c7c00 con 0x7ff69c15cc60 2023-09-23T03:17:16.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.853+0000 7ff6a2056640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c0c7ee0 con 0x7ff69c070780 2023-09-23T03:17:16.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.854+0000 7ff692ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff69c0cd040 0x7ff69c0af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:16.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.854+0000 7ff692ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff69c0cd040 0x7ff69c0af3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46480/0 (socket says 172.21.15.154:46480) 2023-09-23T03:17:16.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.854+0000 7ff692ffd640 1 -- 172.21.15.154:0/2155461396 learned_addr learned my addr 172.21.15.154:0/2155461396 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:16.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.854+0000 7ff69bfff640 1 --2- 172.21.15.154:0/2155461396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff69c15cc60 0x7ff69c0af8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:16.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.854+0000 7ff69b7fe640 1 --2- 172.21.15.154:0/2155461396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff69c070780 0x7ff69c1510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:16.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.854+0000 7ff692ffd640 1 -- 172.21.15.154:0/2155461396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff69c070780 msgr2=0x7ff69c1510f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:16.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.854+0000 7ff692ffd640 1 --2- 172.21.15.154:0/2155461396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff69c070780 0x7ff69c1510f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:16.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.854+0000 7ff692ffd640 1 -- 172.21.15.154:0/2155461396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff69c15cc60 msgr2=0x7ff69c0af8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:16.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.854+0000 7ff692ffd640 1 --2- 172.21.15.154:0/2155461396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff69c15cc60 0x7ff69c0af8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:16.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.854+0000 7ff692ffd640 1 -- 172.21.15.154:0/2155461396 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68c009400 con 0x7ff69c0cd040 2023-09-23T03:17:16.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.854+0000 7ff69bfff640 1 --2- 172.21.15.154:0/2155461396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff69c15cc60 0x7ff69c0af8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:17:16.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.855+0000 7ff692ffd640 1 --2- 172.21.15.154:0/2155461396 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff69c0cd040 0x7ff69c0af3b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff68400c4a0 tx=0x7ff68400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:16.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.855+0000 7ff698ff9640 1 -- 172.21.15.154:0/2155461396 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff684005ce0 con 0x7ff69c0cd040 2023-09-23T03:17:16.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.855+0000 7ff698ff9640 1 -- 172.21.15.154:0/2155461396 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff684005e40 con 0x7ff69c0cd040 2023-09-23T03:17:16.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.855+0000 7ff698ff9640 1 -- 172.21.15.154:0/2155461396 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff684007270 con 0x7ff69c0cd040 2023-09-23T03:17:16.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.855+0000 7ff6a2056640 1 -- 172.21.15.154:0/2155461396 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff69c0c81c0 con 0x7ff69c0cd040 2023-09-23T03:17:16.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.855+0000 7ff6a2056640 1 -- 172.21.15.154:0/2155461396 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff69c0c8780 con 0x7ff69c0cd040 2023-09-23T03:17:16.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.856+0000 7ff6a2056640 1 -- 172.21.15.154:0/2155461396 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff69c04e920 con 0x7ff69c0cd040 2023-09-23T03:17:16.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.857+0000 7ff698ff9640 1 -- 172.21.15.154:0/2155461396 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7ff684007490 con 0x7ff69c0cd040 2023-09-23T03:17:16.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.858+0000 7ff698ff9640 1 --2- 172.21.15.154:0/2155461396 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff67c06af30 0x7ff67c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:16.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.858+0000 7ff698ff9640 1 -- 172.21.15.154:0/2155461396 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff684088d70 con 0x7ff69c0cd040 2023-09-23T03:17:16.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.858+0000 7ff69b7fe640 1 --2- 172.21.15.154:0/2155461396 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff67c06af30 0x7ff67c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:16.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.859+0000 7ff69b7fe640 1 --2- 172.21.15.154:0/2155461396 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff67c06af30 0x7ff67c06d3f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff68c036000 tx=0x7ff68c03c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:16.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:16.864+0000 7ff698ff9640 1 -- 172.21.15.154:0/2155461396 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7ff684053d20 con 0x7ff69c0cd040 2023-09-23T03:17:17.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.104+0000 7ff6a2056640 1 -- 172.21.15.154:0/2155461396 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff69c0b04d0 con 0x7ff69c0cd040 2023-09-23T03:17:17.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.109+0000 7ff698ff9640 1 -- 172.21.15.154:0/2155461396 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) v1 ==== 190+0+147 (secure 0 0 0) 0x7ff684017020 con 0x7ff69c0cd040 2023-09-23T03:17:17.111 INFO:teuthology.orchestra.run.smithi154.stdout:[client.0] 2023-09-23T03:17:17.111 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQAHWA5lYg+qGhAAjaY0fPsmRTz3ipsKbPugMA== 2023-09-23T03:17:17.111 INFO:teuthology.orchestra.run.smithi154.stdout: caps mds = "allow" 2023-09-23T03:17:17.111 INFO:teuthology.orchestra.run.smithi154.stdout: caps mon = "allow r" 2023-09-23T03:17:17.111 INFO:teuthology.orchestra.run.smithi154.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-09-23T03:17:17.111 INFO:teuthology.orchestra.run.smithi154.stderr:updated caps for client.0 2023-09-23T03:17:17.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.112+0000 7ff6a2056640 1 -- 172.21.15.154:0/2155461396 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff67c06af30 msgr2=0x7ff67c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:17.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.112+0000 7ff6a2056640 1 --2- 172.21.15.154:0/2155461396 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff67c06af30 0x7ff67c06d3f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff68c036000 tx=0x7ff68c03c040 comp rx=0 tx=0).stop 2023-09-23T03:17:17.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.112+0000 7ff6a2056640 1 -- 172.21.15.154:0/2155461396 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff69c0cd040 msgr2=0x7ff69c0af3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:17.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.112+0000 7ff6a2056640 1 --2- 172.21.15.154:0/2155461396 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff69c0cd040 0x7ff69c0af3b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff68400c4a0 tx=0x7ff68400c960 comp rx=0 tx=0).stop 2023-09-23T03:17:17.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.113+0000 7ff6a2056640 1 -- 172.21.15.154:0/2155461396 shutdown_connections 2023-09-23T03:17:17.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.113+0000 7ff6a2056640 1 --2- 172.21.15.154:0/2155461396 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff67c06af30 0x7ff67c06d3f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:17.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.113+0000 7ff6a2056640 1 --2- 172.21.15.154:0/2155461396 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff69c15cc60 0x7ff69c0af8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:17.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.113+0000 7ff6a2056640 1 --2- 172.21.15.154:0/2155461396 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff69c0cd040 0x7ff69c0af3b0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:17.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.113+0000 7ff6a2056640 1 --2- 172.21.15.154:0/2155461396 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff69c070780 0x7ff69c1510f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:17.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.113+0000 7ff6a2056640 1 -- 172.21.15.154:0/2155461396 >> 172.21.15.154:0/2155461396 conn(0x7ff69c0ced60 msgr2=0x7ff69c0cf140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:17.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.113+0000 7ff6a2056640 1 -- 172.21.15.154:0/2155461396 shutdown_connections 2023-09-23T03:17:17.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.113+0000 7ff6a2056640 1 -- 172.21.15.154:0/2155461396 wait complete. 2023-09-23T03:17:17.131 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:17:17.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.337+0000 7f8c3bfff640 1 Processor -- start 2023-09-23T03:17:17.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.337+0000 7f8c3bfff640 1 -- start start 2023-09-23T03:17:17.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.338+0000 7f8c3bfff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c3c0c4970 0x7f8c3c071870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:17.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.338+0000 7f8c3bfff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8c3c06c470 0x7f8c3c0ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:17.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.338+0000 7f8c3bfff640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c3c159a00 con 0x7f8c3c0ca260 2023-09-23T03:17:17.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.338+0000 7f8c3bfff640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c3c0cbe90 con 0x7f8c3c06c470 2023-09-23T03:17:17.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.338+0000 7f8c3bfff640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c071f70 con 0x7f8c3c0c4970 2023-09-23T03:17:17.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.339+0000 7f8c327fc640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f8c3c0ca260 0x7f8c3c06bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40716/0 (socket says 172.21.15.154:40716) 2023-09-23T03:17:17.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.339+0000 7f8c327fc640 1 -- 172.21.15.154:0/179548944 learned_addr learned my addr 172.21.15.154:0/179548944 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:17:17.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.339+0000 7f8c3b7fe640 1 --2- 172.21.15.154:0/179548944 >> v2:172.21.15.190:3300/0 conn(0x7f8c3c06c470 0x7f8c3c0ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:17.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.339+0000 7f8c3affd640 1 --2- 172.21.15.154:0/179548944 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c3c0c4970 0x7f8c3c071870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:17.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.339+0000 7f8c3a7fc640 1 -- 172.21.15.154:0/179548944 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4261187817 0 0) 0x7f8c3c159a00 con 0x7f8c3c0ca260 2023-09-23T03:17:17.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.339+0000 7f8c3a7fc640 1 -- 172.21.15.154:0/179548944 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c28003520 con 0x7f8c3c0ca260 2023-09-23T03:17:17.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.339+0000 7f8c3b7fe640 1 -- 172.21.15.154:0/179548944 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c3c0c4970 msgr2=0x7f8c3c071870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:17.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.339+0000 7f8c3b7fe640 1 --2- 172.21.15.154:0/179548944 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c3c0c4970 0x7f8c3c071870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:17.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.339+0000 7f8c3b7fe640 1 -- 172.21.15.154:0/179548944 >> v1:172.21.15.154:6789/0 conn(0x7f8c3c0ca260 legacy=0x7f8c3c06bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:17.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.340+0000 7f8c3b7fe640 1 -- 172.21.15.154:0/179548944 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c3c0cc150 con 0x7f8c3c06c470 2023-09-23T03:17:17.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.341+0000 7f8c3b7fe640 1 --2- 172.21.15.154:0/179548944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c3c06c470 0x7f8c3c0ccc30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8c34009d90 tx=0x7f8c3402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c553a7add8ff9b8c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:17.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.341+0000 7f8c3a7fc640 1 -- 172.21.15.154:0/179548944 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3402cc50 con 0x7f8c3c06c470 2023-09-23T03:17:17.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.341+0000 7f8c3a7fc640 1 -- 172.21.15.154:0/179548944 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c3402cdb0 con 0x7f8c3c06c470 2023-09-23T03:17:17.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.342+0000 7f8c3a7fc640 1 -- 172.21.15.154:0/179548944 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c34034640 con 0x7f8c3c06c470 2023-09-23T03:17:17.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.342+0000 7f8c3bfff640 1 -- 172.21.15.154:0/179548944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c3c06c470 msgr2=0x7f8c3c0ccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:17.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.342+0000 7f8c3bfff640 1 --2- 172.21.15.154:0/179548944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c3c06c470 0x7f8c3c0ccc30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8c34009d90 tx=0x7f8c3402bea0 comp rx=0 tx=0).stop 2023-09-23T03:17:17.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.342+0000 7f8c3bfff640 1 -- 172.21.15.154:0/179548944 shutdown_connections 2023-09-23T03:17:17.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.342+0000 7f8c3bfff640 1 --2- 172.21.15.154:0/179548944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c3c06c470 0x7f8c3c0ccc30 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:17.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.342+0000 7f8c3bfff640 1 --2- 172.21.15.154:0/179548944 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c3c0c4970 0x7f8c3c071870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:17.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.342+0000 7f8c3bfff640 1 -- 172.21.15.154:0/179548944 >> 172.21.15.154:0/179548944 conn(0x7f8c3c06a390 msgr2=0x7f8c3c0afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:17.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.342+0000 7f8c3bfff640 1 -- 172.21.15.154:0/179548944 shutdown_connections 2023-09-23T03:17:17.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.342+0000 7f8c3bfff640 1 -- 172.21.15.154:0/179548944 wait complete. 2023-09-23T03:17:17.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.344+0000 7f8c3bfff640 1 Processor -- start 2023-09-23T03:17:17.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.344+0000 7f8c3bfff640 1 -- start start 2023-09-23T03:17:17.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.345+0000 7f8c3bfff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c3c06c470 0x7f8c3c0ad090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:17.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.345+0000 7f8c3bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c3c0c4970 0x7f8c3c0c7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:17.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.345+0000 7f8c3bfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c3c0ca260 0x7f8c3c0c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:17.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.345+0000 7f8c3bfff640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c15c530 con 0x7f8c3c0c4970 2023-09-23T03:17:17.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.345+0000 7f8c3bfff640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c0ad5d0 con 0x7f8c3c0ca260 2023-09-23T03:17:17.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.345+0000 7f8c3bfff640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3c14ded0 con 0x7f8c3c06c470 2023-09-23T03:17:17.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.345+0000 7f8c327fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c3c0c4970 0x7f8c3c0c7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:17.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.345+0000 7f8c327fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c3c0c4970 0x7f8c3c0c7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46488/0 (socket says 172.21.15.154:46488) 2023-09-23T03:17:17.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.345+0000 7f8c327fc640 1 -- 172.21.15.154:0/4134534506 learned_addr learned my addr 172.21.15.154:0/4134534506 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:17.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.346+0000 7f8c327fc640 1 -- 172.21.15.154:0/4134534506 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c3c06c470 msgr2=0x7f8c3c0ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:17.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.346+0000 7f8c327fc640 1 --2- 172.21.15.154:0/4134534506 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c3c06c470 0x7f8c3c0ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:17.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.346+0000 7f8c327fc640 1 -- 172.21.15.154:0/4134534506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c3c0ca260 msgr2=0x7f8c3c0c7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:17.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.346+0000 7f8c327fc640 1 --2- 172.21.15.154:0/4134534506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c3c0ca260 0x7f8c3c0c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:17.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.346+0000 7f8c327fc640 1 -- 172.21.15.154:0/4134534506 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c34009a70 con 0x7f8c3c0c4970 2023-09-23T03:17:17.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.346+0000 7f8c327fc640 1 --2- 172.21.15.154:0/4134534506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c3c0c4970 0x7f8c3c0c7560 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f8c2400c390 tx=0x7f8c2400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:17.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.346+0000 7f8c33fff640 1 -- 172.21.15.154:0/4134534506 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c24005ce0 con 0x7f8c3c0c4970 2023-09-23T03:17:17.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.346+0000 7f8c33fff640 1 -- 172.21.15.154:0/4134534506 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c24005e40 con 0x7f8c3c0c4970 2023-09-23T03:17:17.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.346+0000 7f8c33fff640 1 -- 172.21.15.154:0/4134534506 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c240072f0 con 0x7f8c3c0c4970 2023-09-23T03:17:17.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.346+0000 7f8c3bfff640 1 -- 172.21.15.154:0/4134534506 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c3c14e1b0 con 0x7f8c3c0c4970 2023-09-23T03:17:17.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.346+0000 7f8c3bfff640 1 -- 172.21.15.154:0/4134534506 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c3c14e770 con 0x7f8c3c0c4970 2023-09-23T03:17:17.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.348+0000 7f8c3bfff640 1 -- 172.21.15.154:0/4134534506 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c3c0736b0 con 0x7f8c3c0c4970 2023-09-23T03:17:17.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.349+0000 7f8c33fff640 1 -- 172.21.15.154:0/4134534506 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f8c24007450 con 0x7f8c3c0c4970 2023-09-23T03:17:17.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.349+0000 7f8c33fff640 1 --2- 172.21.15.154:0/4134534506 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8c1c06af30 0x7f8c1c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:17.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.350+0000 7f8c33fff640 1 -- 172.21.15.154:0/4134534506 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8c24088e70 con 0x7f8c3c0c4970 2023-09-23T03:17:17.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.350+0000 7f8c3affd640 1 --2- 172.21.15.154:0/4134534506 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8c1c06af30 0x7f8c1c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:17.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.350+0000 7f8c3affd640 1 --2- 172.21.15.154:0/4134534506 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8c1c06af30 0x7f8c1c06d3f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c006a90 tx=0x7f8c2c006a20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:17.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.354+0000 7f8c33fff640 1 -- 172.21.15.154:0/4134534506 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f8c24053e00 con 0x7f8c3c0c4970 2023-09-23T03:17:17.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.583+0000 7f8c3bfff640 1 -- 172.21.15.154:0/4134534506 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c3c070150 con 0x7f8c3c0c4970 2023-09-23T03:17:17.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.584+0000 7f8c33fff640 1 -- 172.21.15.154:0/4134534506 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f8c24017020 con 0x7f8c3c0c4970 2023-09-23T03:17:17.586 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:17.586 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4760,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/4034412019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4034412019},{"type":"v1","addr":"172.21.15.154:6836","nonce":4034412019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3736308401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3736308401},{"type":"v1","addr":"172.21.15.190:6835","nonce":3736308401}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4999,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/617287200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":617287200},{"type":"v1","addr":"172.21.15.154:6837","nonce":617287200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:17:11.088637+0000","modified":"2023-09-23T03:17:14.181716+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5002},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5002":{"gid":5002,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.190:6834/2131436414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2131436414},{"type":"v1","addr":"172.21.15.190:6834","nonce":2131436414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-09-23T03:17:17.586 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 18 2023-09-23T03:17:17.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.587+0000 7f8c3bfff640 1 -- 172.21.15.154:0/4134534506 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8c1c06af30 msgr2=0x7f8c1c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:17.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.587+0000 7f8c3bfff640 1 --2- 172.21.15.154:0/4134534506 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8c1c06af30 0x7f8c1c06d3f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c006a90 tx=0x7f8c2c006a20 comp rx=0 tx=0).stop 2023-09-23T03:17:17.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.587+0000 7f8c3bfff640 1 -- 172.21.15.154:0/4134534506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c3c0c4970 msgr2=0x7f8c3c0c7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:17.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.587+0000 7f8c3bfff640 1 --2- 172.21.15.154:0/4134534506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c3c0c4970 0x7f8c3c0c7560 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f8c2400c390 tx=0x7f8c2400c850 comp rx=0 tx=0).stop 2023-09-23T03:17:17.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.587+0000 7f8c3bfff640 1 -- 172.21.15.154:0/4134534506 shutdown_connections 2023-09-23T03:17:17.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.587+0000 7f8c3bfff640 1 --2- 172.21.15.154:0/4134534506 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8c1c06af30 0x7f8c1c06d3f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:17.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.587+0000 7f8c3bfff640 1 --2- 172.21.15.154:0/4134534506 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8c3c0ca260 0x7f8c3c0c7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:17.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.587+0000 7f8c3bfff640 1 --2- 172.21.15.154:0/4134534506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c3c0c4970 0x7f8c3c0c7560 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:17.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.587+0000 7f8c3bfff640 1 --2- 172.21.15.154:0/4134534506 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8c3c06c470 0x7f8c3c0ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:17.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.587+0000 7f8c3bfff640 1 -- 172.21.15.154:0/4134534506 >> 172.21.15.154:0/4134534506 conn(0x7f8c3c06a390 msgr2=0x7f8c3c0c2080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:17.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.587+0000 7f8c3bfff640 1 -- 172.21.15.154:0/4134534506 shutdown_connections 2023-09-23T03:17:17.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:17.587+0000 7f8c3bfff640 1 -- 172.21.15.154:0/4134534506 wait complete. 2023-09-23T03:17:18.009 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-23T03:17:11.088637+0000', 'modified': '2023-09-23T03:17:14.181716+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5002}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5002': {'gid': 5002, 'name': 'd', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.190:6834/2131436414', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6832', 'nonce': 2131436414}, {'type': 'v1', 'addr': '172.21.15.190:6834', 'nonce': 2131436414}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-23T03:17:18.009 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-23T03:17:18.009 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-09-23T03:17:18.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.236+0000 7f0abe639640 1 Processor -- start 2023-09-23T03:17:18.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.237+0000 7f0abe639640 1 -- start start 2023-09-23T03:17:18.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.238+0000 7f0abe639640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0ab80b1b20 0x7f0ab80ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:18.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.238+0000 7f0abe639640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0ab80cee70 0x7f0ab80ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:18.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.238+0000 7f0abe639640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab815ac30 con 0x7f0ab80acce0 2023-09-23T03:17:18.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.238+0000 7f0abe639640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab80b2e90 con 0x7f0ab80b1b20 2023-09-23T03:17:18.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.238+0000 7f0abe639640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab80cf320 con 0x7f0ab80cee70 2023-09-23T03:17:18.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.238+0000 7f0ab7fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f0ab80acce0 0x7f0ab815ed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40732/0 (socket says 172.21.15.154:40732) 2023-09-23T03:17:18.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.238+0000 7f0ab7fff640 1 -- 172.21.15.154:0/2261310762 learned_addr learned my addr 172.21.15.154:0/2261310762 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:17:18.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.238+0000 7f0ab77fe640 1 --2- 172.21.15.154:0/2261310762 >> v2:172.21.15.154:3300/0 conn(0x7f0ab80b1b20 0x7f0ab80ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:18.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.239+0000 7f0abcbd6640 1 --2- 172.21.15.154:0/2261310762 >> v2:172.21.15.190:3300/0 conn(0x7f0ab80cee70 0x7f0ab80ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:18.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.239+0000 7f0ab6ffd640 1 -- 172.21.15.154:0/2261310762 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2918726016 0 0) 0x7f0aa8001290 con 0x7f0ab80acce0 2023-09-23T03:17:18.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.239+0000 7f0ab77fe640 1 -- 172.21.15.154:0/2261310762 >> v2:172.21.15.190:3300/0 conn(0x7f0ab80cee70 msgr2=0x7f0ab80ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:18.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.239+0000 7f0ab77fe640 1 --2- 172.21.15.154:0/2261310762 >> v2:172.21.15.190:3300/0 conn(0x7f0ab80cee70 0x7f0ab80ab140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.239+0000 7f0ab77fe640 1 -- 172.21.15.154:0/2261310762 >> v1:172.21.15.154:6789/0 conn(0x7f0ab80acce0 legacy=0x7f0ab815ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:18.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.239+0000 7f0ab77fe640 1 -- 172.21.15.154:0/2261310762 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab80b1f70 con 0x7f0ab80b1b20 2023-09-23T03:17:18.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.239+0000 7f0abcbd6640 1 --2- 172.21.15.154:0/2261310762 >> v2:172.21.15.190:3300/0 conn(0x7f0ab80cee70 0x7f0ab80ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:18.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.240+0000 7f0ab77fe640 1 --2- 172.21.15.154:0/2261310762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ab80b1b20 0x7f0ab80ce930 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0ab80c9430 tx=0x7f0aa002be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a88c0ebe6424e27 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:18.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.240+0000 7f0ab6ffd640 1 -- 172.21.15.154:0/2261310762 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa002cbe0 con 0x7f0ab80b1b20 2023-09-23T03:17:18.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.241+0000 7f0ab6ffd640 1 -- 172.21.15.154:0/2261310762 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa002cd40 con 0x7f0ab80b1b20 2023-09-23T03:17:18.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.241+0000 7f0ab6ffd640 1 -- 172.21.15.154:0/2261310762 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa0034660 con 0x7f0ab80b1b20 2023-09-23T03:17:18.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.241+0000 7f0abe639640 1 -- 172.21.15.154:0/2261310762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ab80b1b20 msgr2=0x7f0ab80ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:18.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.241+0000 7f0abe639640 1 --2- 172.21.15.154:0/2261310762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ab80b1b20 0x7f0ab80ce930 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0ab80c9430 tx=0x7f0aa002be30 comp rx=0 tx=0).stop 2023-09-23T03:17:18.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.241+0000 7f0abe639640 1 -- 172.21.15.154:0/2261310762 shutdown_connections 2023-09-23T03:17:18.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.241+0000 7f0abe639640 1 --2- 172.21.15.154:0/2261310762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ab80b1b20 0x7f0ab80ce930 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.241+0000 7f0abe639640 1 --2- 172.21.15.154:0/2261310762 >> v2:172.21.15.190:3300/0 conn(0x7f0ab80cee70 0x7f0ab80ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.241+0000 7f0abe639640 1 -- 172.21.15.154:0/2261310762 >> 172.21.15.154:0/2261310762 conn(0x7f0ab806a0f0 msgr2=0x7f0ab806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:18.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.241+0000 7f0abe639640 1 -- 172.21.15.154:0/2261310762 shutdown_connections 2023-09-23T03:17:18.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.241+0000 7f0abe639640 1 -- 172.21.15.154:0/2261310762 wait complete. 2023-09-23T03:17:18.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.243+0000 7f0abe639640 1 Processor -- start 2023-09-23T03:17:18.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.243+0000 7f0abe639640 1 -- start start 2023-09-23T03:17:18.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.244+0000 7f0abe639640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ab80acce0 0x7f0ab80c1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:18.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.244+0000 7f0abe639640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ab80b1b20 0x7f0ab81595e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:18.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.244+0000 7f0abe639640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ab80cee70 0x7f0ab815b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:18.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.244+0000 7f0abe639640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab8159b20 con 0x7f0ab80cee70 2023-09-23T03:17:18.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.244+0000 7f0abe639640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab815bd50 con 0x7f0ab80b1b20 2023-09-23T03:17:18.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.244+0000 7f0abe639640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab80c25c0 con 0x7f0ab80acce0 2023-09-23T03:17:18.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.244+0000 7f0abcbd6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ab80cee70 0x7f0ab815b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:18.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.245+0000 7f0abcbd6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ab80cee70 0x7f0ab815b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46512/0 (socket says 172.21.15.154:46512) 2023-09-23T03:17:18.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.245+0000 7f0abcbd6640 1 -- 172.21.15.154:0/868751337 learned_addr learned my addr 172.21.15.154:0/868751337 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:18.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.245+0000 7f0ab7fff640 1 --2- 172.21.15.154:0/868751337 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ab80acce0 0x7f0ab80c1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:18.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.245+0000 7f0abcbd6640 1 -- 172.21.15.154:0/868751337 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ab80acce0 msgr2=0x7f0ab80c1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:18.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.245+0000 7f0abcbd6640 1 --2- 172.21.15.154:0/868751337 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ab80acce0 0x7f0ab80c1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.245+0000 7f0ab77fe640 1 --2- 172.21.15.154:0/868751337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ab80b1b20 0x7f0ab81595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:18.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.245+0000 7f0abcbd6640 1 -- 172.21.15.154:0/868751337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ab80b1b20 msgr2=0x7f0ab81595e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:18.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.245+0000 7f0abcbd6640 1 --2- 172.21.15.154:0/868751337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ab80b1b20 0x7f0ab81595e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.245+0000 7f0abcbd6640 1 -- 172.21.15.154:0/868751337 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa0009a70 con 0x7f0ab80cee70 2023-09-23T03:17:18.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.245+0000 7f0abcbd6640 1 --2- 172.21.15.154:0/868751337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ab80cee70 0x7f0ab815b810 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000c7c0 tx=0x7f0ab000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:18.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.246+0000 7f0ab4ff9640 1 -- 172.21.15.154:0/868751337 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0004230 con 0x7f0ab80cee70 2023-09-23T03:17:18.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.246+0000 7f0ab4ff9640 1 -- 172.21.15.154:0/868751337 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab0004390 con 0x7f0ab80cee70 2023-09-23T03:17:18.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.246+0000 7f0abe639640 1 -- 172.21.15.154:0/868751337 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab80c28a0 con 0x7f0ab80cee70 2023-09-23T03:17:18.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.246+0000 7f0ab4ff9640 1 -- 172.21.15.154:0/868751337 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0007860 con 0x7f0ab80cee70 2023-09-23T03:17:18.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.246+0000 7f0abe639640 1 -- 172.21.15.154:0/868751337 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab80c2e60 con 0x7f0ab80cee70 2023-09-23T03:17:18.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.249+0000 7f0ab4ff9640 1 -- 172.21.15.154:0/868751337 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f0ab00079c0 con 0x7f0ab80cee70 2023-09-23T03:17:18.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.250+0000 7f0ab4ff9640 1 --2- 172.21.15.154:0/868751337 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0a9806af30 0x7f0a9806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:18.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.250+0000 7f0ab4ff9640 1 -- 172.21.15.154:0/868751337 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0ab0089b60 con 0x7f0ab80cee70 2023-09-23T03:17:18.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.250+0000 7f0ab7fff640 1 --2- 172.21.15.154:0/868751337 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0a9806af30 0x7f0a9806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:18.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.250+0000 7f0ac11cb740 1 Processor -- start 2023-09-23T03:17:18.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.251+0000 7f0ac11cb740 1 -- start start 2023-09-23T03:17:18.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.251+0000 7f0ab7fff640 1 --2- 172.21.15.154:0/868751337 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0a9806af30 0x7f0a9806d3f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80014f0 tx=0x7f0aa8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:18.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.251+0000 7f0ac11cb740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c38f6ac00 0x564c38f3db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:18.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.251+0000 7f0ac11cb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c38f6afe0 0x564c38f6d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:18.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.251+0000 7f0ac11cb740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c38f3e040 0x564c38f6ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:18.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.251+0000 7f0ac11cb740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c38eb16d0 con 0x564c38f6ac00 2023-09-23T03:17:18.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.251+0000 7f0ac11cb740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c38eaedc0 con 0x564c38f6afe0 2023-09-23T03:17:18.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.251+0000 7f0ac11cb740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c38eaa6b0 con 0x564c38f3e040 2023-09-23T03:17:18.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.252+0000 7f0ab7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c38f6ac00 0x564c38f3db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:18.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.252+0000 7f0ab7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c38f6ac00 0x564c38f3db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46522/0 (socket says 172.21.15.154:46522) 2023-09-23T03:17:18.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.252+0000 7f0ab7fff640 1 -- 172.21.15.154:0/3489709994 learned_addr learned my addr 172.21.15.154:0/3489709994 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:18.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.252+0000 7f0ab7fff640 1 -- 172.21.15.154:0/3489709994 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c38f3e040 msgr2=0x564c38f6ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:18.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.252+0000 7f0abcbd6640 1 --2- 172.21.15.154:0/3489709994 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c38f3e040 0x564c38f6ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:18.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.252+0000 7f0ab7fff640 1 --2- 172.21.15.154:0/3489709994 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c38f3e040 0x564c38f6ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.252+0000 7f0ab7fff640 1 -- 172.21.15.154:0/3489709994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c38f6afe0 msgr2=0x564c38f6d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:18.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.252+0000 7f0ab7fff640 1 --2- 172.21.15.154:0/3489709994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c38f6afe0 0x564c38f6d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.252+0000 7f0ab7fff640 1 -- 172.21.15.154:0/3489709994 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c38e6e8e0 con 0x564c38f6ac00 2023-09-23T03:17:18.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.252+0000 7f0ab7fff640 1 --2- 172.21.15.154:0/3489709994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c38f6ac00 0x564c38f3db00 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8007570 tx=0x7f0aa8010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe174d0edf2eca1b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:18.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.253+0000 7f0aadffb640 1 -- 172.21.15.154:0/3489709994 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa800d8a0 con 0x564c38f6ac00 2023-09-23T03:17:18.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.253+0000 7f0aadffb640 1 -- 172.21.15.154:0/3489709994 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa800da00 con 0x564c38f6ac00 2023-09-23T03:17:18.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.253+0000 7f0aadffb640 1 -- 172.21.15.154:0/3489709994 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa800ddb0 con 0x564c38f6ac00 2023-09-23T03:17:18.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.253+0000 7f0ac11cb740 1 -- 172.21.15.154:0/3489709994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c38f6ac00 msgr2=0x564c38f3db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:18.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.253+0000 7f0ac11cb740 1 --2- 172.21.15.154:0/3489709994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c38f6ac00 0x564c38f3db00 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8007570 tx=0x7f0aa8010000 comp rx=0 tx=0).stop 2023-09-23T03:17:18.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.253+0000 7f0ac11cb740 1 -- 172.21.15.154:0/3489709994 shutdown_connections 2023-09-23T03:17:18.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.253+0000 7f0ac11cb740 1 --2- 172.21.15.154:0/3489709994 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c38f3e040 0x564c38f6ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.253+0000 7f0ac11cb740 1 --2- 172.21.15.154:0/3489709994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c38f6afe0 0x564c38f6d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.253+0000 7f0ac11cb740 1 --2- 172.21.15.154:0/3489709994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c38f6ac00 0x564c38f3db00 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.253+0000 7f0ac11cb740 1 -- 172.21.15.154:0/3489709994 >> 172.21.15.154:0/3489709994 conn(0x564c38f3fe10 msgr2=0x564c38f88990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:18.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.254+0000 7f0ac11cb740 1 -- 172.21.15.154:0/3489709994 shutdown_connections 2023-09-23T03:17:18.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.254+0000 7f0ac11cb740 1 -- 172.21.15.154:0/3489709994 wait complete. 2023-09-23T03:17:18.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.254+0000 7f0ac11cb740 1 -- start start 2023-09-23T03:17:18.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.254+0000 7f0ac11cb740 1 Processor -- start 2023-09-23T03:17:18.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.255+0000 7f0ac11cb740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c38f6ac00 0x564c38f82fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:18.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.255+0000 7f0ac11cb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c38f6afe0 0x564c38f5f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:18.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.255+0000 7f0ac11cb740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c38f9be60 0x564c38f9c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:18.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.255+0000 7f0ac11cb740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c38e48940 con 0x564c38f6ac00 2023-09-23T03:17:18.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.255+0000 7f0ac11cb740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c38e485e0 con 0x564c38f6afe0 2023-09-23T03:17:18.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.255+0000 7f0ac11cb740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c38c0ef20 con 0x564c38f9be60 2023-09-23T03:17:18.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.255+0000 7f0ac11cb740 10 client.0 _ll_register_callbacks cb 0x564c38e865e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:17:18.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.256+0000 7f0ab7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c38f6ac00 0x564c38f82fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:18.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.256+0000 7f0ab7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c38f6ac00 0x564c38f82fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46538/0 (socket says 172.21.15.154:46538) 2023-09-23T03:17:18.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.256+0000 7f0ab7fff640 1 -- 172.21.15.154:0/486303764 learned_addr learned my addr 172.21.15.154:0/486303764 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:18.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.256+0000 7f0abcbd6640 1 --2- 172.21.15.154:0/486303764 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c38f9be60 0x564c38f9c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:18.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.256+0000 7f0ab77fe640 1 --2- 172.21.15.154:0/486303764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c38f6afe0 0x564c38f5f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:18.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.256+0000 7f0ab7fff640 1 -- 172.21.15.154:0/486303764 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c38f9be60 msgr2=0x564c38f9c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:18.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.256+0000 7f0ab7fff640 1 --2- 172.21.15.154:0/486303764 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c38f9be60 0x564c38f9c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.256+0000 7f0ab7fff640 1 -- 172.21.15.154:0/486303764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c38f6afe0 msgr2=0x564c38f5f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:18.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.256+0000 7f0ab7fff640 1 --2- 172.21.15.154:0/486303764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c38f6afe0 0x564c38f5f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.256+0000 7f0ab7fff640 1 -- 172.21.15.154:0/486303764 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aa80072e0 con 0x564c38f6ac00 2023-09-23T03:17:18.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.257+0000 7f0ab7fff640 1 --2- 172.21.15.154:0/486303764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c38f6ac00 0x564c38f82fc0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f0aa800dea0 tx=0x7f0aa80010a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf5613e39b48bbb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:18.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.257+0000 7f0a96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:17:18.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.257+0000 7f0a96ffd640 1 -- 172.21.15.154:0/486303764 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8027070 con 0x564c38f6ac00 2023-09-23T03:17:18.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.257+0000 7f0ac11cb740 1 -- 172.21.15.154:0/486303764 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c38ea3690 con 0x564c38f6ac00 2023-09-23T03:17:18.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.257+0000 7f0a96ffd640 1 -- 172.21.15.154:0/486303764 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0aa80224f0 con 0x564c38f6ac00 2023-09-23T03:17:18.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.257+0000 7f0a96ffd640 1 -- 172.21.15.154:0/486303764 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0aa80226b0 con 0x564c38f6ac00 2023-09-23T03:17:18.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.257+0000 7f0ac11cb740 10 client.5652 fetch_fsmap learned FSMap version 18 2023-09-23T03:17:18.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.257+0000 7f0ac11cb740 1 -- 172.21.15.154:0/486303764 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x564c38ea8d30 con 0x564c38f6ac00 2023-09-23T03:17:18.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.257+0000 7f0a96ffd640 1 -- 172.21.15.154:0/486303764 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564c38ea3690 con 0x564c38f6ac00 2023-09-23T03:17:18.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.257+0000 7f0a96ffd640 1 -- 172.21.15.154:0/486303764 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0aa8022810 con 0x564c38f6ac00 2023-09-23T03:17:18.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.257+0000 7f0ac11cb740 10 client.5652 fetch_fsmap finished waiting for FSMap version 18 2023-09-23T03:17:18.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.257+0000 7f0ac11cb740 10 client.5652 resolve_mds: resolved name 'd' to daemon mds.d 2023-09-23T03:17:18.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.258+0000 7f0ac11cb740 20 client.5652 populate_metadata read hostname 'smithi154' 2023-09-23T03:17:18.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.258+0000 7f0ac11cb740 1 --2- 172.21.15.154:0/486303764 >> [v2:172.21.15.190:6832/2131436414,v1:172.21.15.190:6834/2131436414] conn(0x564c38f6f470 0x564c38f802b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:18.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.258+0000 7f0ac11cb740 4 client.5652 mds_command: new command op to 5002 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T03:17:18.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.258+0000 7f0ac11cb740 1 -- 172.21.15.154:0/486303764 --> [v2:172.21.15.190:6832/2131436414,v1:172.21.15.190:6834/2131436414] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c38ea33b0 con 0x564c38f6f470 2023-09-23T03:17:18.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.258+0000 7f0ab77fe640 1 --2- 172.21.15.154:0/486303764 >> [v2:172.21.15.190:6832/2131436414,v1:172.21.15.190:6834/2131436414] conn(0x564c38f6f470 0x564c38f802b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:18.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.259+0000 7f0ab77fe640 1 --2- 172.21.15.154:0/486303764 >> [v2:172.21.15.190:6832/2131436414,v1:172.21.15.190:6834/2131436414] conn(0x564c38f6f470 0x564c38f802b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=769fadd6001e83c8 server_cookie=dec5a9a78af77790 in_seq=0 out_seq=0 2023-09-23T03:17:18.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.259+0000 7f0a96ffd640 10 client.5652 ms_handle_connect on v2:172.21.15.190:6832/2131436414 2023-09-23T03:17:18.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.261+0000 7f0a96ffd640 1 -- 172.21.15.154:0/486303764 <== mds.0 v2:172.21.15.190:6832/2131436414 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564c38ea33b0 con 0x564c38f6f470 2023-09-23T03:17:18.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.261+0000 7f0a96ffd640 10 client.5652 handle_command_reply: tid=0 2023-09-23T03:17:18.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.261+0000 7f0ac11cb740 1 client.5652 shutdown 2023-09-23T03:17:18.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.261+0000 7f0ac11cb740 1 -- 172.21.15.154:0/486303764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c38f6ac00 msgr2=0x564c38f82fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:18.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.261+0000 7f0ac11cb740 1 --2- 172.21.15.154:0/486303764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c38f6ac00 0x564c38f82fc0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f0aa800dea0 tx=0x7f0aa80010a0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.261+0000 7f0ac11cb740 1 -- 172.21.15.154:0/486303764 shutdown_connections 2023-09-23T03:17:18.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.261+0000 7f0ac11cb740 1 --2- 172.21.15.154:0/486303764 >> [v2:172.21.15.190:6832/2131436414,v1:172.21.15.190:6834/2131436414] conn(0x564c38f6f470 0x564c38f802b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.261+0000 7f0ac11cb740 1 --2- 172.21.15.154:0/486303764 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c38f9be60 0x564c38f9c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.262+0000 7f0a96ffd640 0 client.5652 ms_handle_reset on v2:172.21.15.190:6832/2131436414 2023-09-23T03:17:18.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.261+0000 7f0ac11cb740 1 --2- 172.21.15.154:0/486303764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c38f6afe0 0x564c38f5f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.262+0000 7f0ac11cb740 1 --2- 172.21.15.154:0/486303764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c38f6ac00 0x564c38f82fc0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.262+0000 7f0ac11cb740 1 -- 172.21.15.154:0/486303764 >> 172.21.15.154:0/486303764 conn(0x564c38f3e040 msgr2=0x564c38f827f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:18.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.262+0000 7f0ac11cb740 1 -- 172.21.15.154:0/486303764 shutdown_connections 2023-09-23T03:17:18.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.262+0000 7f0ac11cb740 1 -- 172.21.15.154:0/486303764 wait complete. 2023-09-23T03:17:18.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.262+0000 7f0ac11cb740 20 client.5652 trim_cache size 0 max 16384 2023-09-23T03:17:18.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.278+0000 7f0ac11cb740 1 Processor -- start 2023-09-23T03:17:18.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.278+0000 7f0ac11cb740 1 -- start start 2023-09-23T03:17:18.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.279+0000 7f0ac11cb740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c38f6ac00 0x564c38f88d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:18.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.279+0000 7f0ac11cb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c38f6afe0 0x564c38f7e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:18.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.279+0000 7f0ac11cb740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c38f9be60 0x564c38f80480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:18.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.279+0000 7f0ac11cb740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c38f7ea00 con 0x564c38f6ac00 2023-09-23T03:17:18.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.279+0000 7f0ac11cb740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c38f8b560 con 0x564c38f6afe0 2023-09-23T03:17:18.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.279+0000 7f0ac11cb740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c38f3ec60 con 0x564c38f9be60 2023-09-23T03:17:18.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.280+0000 7f0ab7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c38f6ac00 0x564c38f88d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:18.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.280+0000 7f0ab7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c38f6ac00 0x564c38f88d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46542/0 (socket says 172.21.15.154:46542) 2023-09-23T03:17:18.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.280+0000 7f0ab7fff640 1 -- 172.21.15.154:0/1033861994 learned_addr learned my addr 172.21.15.154:0/1033861994 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:18.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.280+0000 7f0abcbd6640 1 --2- 172.21.15.154:0/1033861994 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c38f9be60 0x564c38f80480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:18.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.280+0000 7f0ab77fe640 1 --2- 172.21.15.154:0/1033861994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c38f6afe0 0x564c38f7e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:18.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.280+0000 7f0ab7fff640 1 -- 172.21.15.154:0/1033861994 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c38f9be60 msgr2=0x564c38f80480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:18.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.280+0000 7f0ab7fff640 1 --2- 172.21.15.154:0/1033861994 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c38f9be60 0x564c38f80480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.280+0000 7f0ab7fff640 1 -- 172.21.15.154:0/1033861994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c38f6afe0 msgr2=0x564c38f7e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:18.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.280+0000 7f0ab7fff640 1 --2- 172.21.15.154:0/1033861994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c38f6afe0 0x564c38f7e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.280+0000 7f0ab7fff640 1 -- 172.21.15.154:0/1033861994 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c38f3ee50 con 0x564c38f6ac00 2023-09-23T03:17:18.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.280+0000 7f0ab7fff640 1 --2- 172.21.15.154:0/1033861994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c38f6ac00 0x564c38f88d80 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0aa800d240 tx=0x7f0aa8024050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b607def6a82c11 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:18.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.281+0000 7f0aae7fc640 1 -- 172.21.15.154:0/1033861994 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8007ea0 con 0x564c38f6ac00 2023-09-23T03:17:18.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.281+0000 7f0aae7fc640 1 -- 172.21.15.154:0/1033861994 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa8018030 con 0x564c38f6ac00 2023-09-23T03:17:18.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.281+0000 7f0aae7fc640 1 -- 172.21.15.154:0/1033861994 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8007180 con 0x564c38f6ac00 2023-09-23T03:17:18.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.281+0000 7f0ac11cb740 1 -- 172.21.15.154:0/1033861994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c38f6ac00 msgr2=0x564c38f88d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:18.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.281+0000 7f0ac11cb740 1 --2- 172.21.15.154:0/1033861994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c38f6ac00 0x564c38f88d80 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0aa800d240 tx=0x7f0aa8024050 comp rx=0 tx=0).stop 2023-09-23T03:17:18.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.281+0000 7f0ac11cb740 1 -- 172.21.15.154:0/1033861994 shutdown_connections 2023-09-23T03:17:18.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.281+0000 7f0ac11cb740 1 --2- 172.21.15.154:0/1033861994 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c38f9be60 0x564c38f80480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.281+0000 7f0ac11cb740 1 --2- 172.21.15.154:0/1033861994 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c38f6afe0 0x564c38f7e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.281+0000 7f0ac11cb740 1 --2- 172.21.15.154:0/1033861994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c38f6ac00 0x564c38f88d80 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.281+0000 7f0ac11cb740 1 -- 172.21.15.154:0/1033861994 >> 172.21.15.154:0/1033861994 conn(0x564c38f3e040 msgr2=0x564c38fab310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:18.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.282+0000 7f0ac11cb740 1 -- 172.21.15.154:0/1033861994 shutdown_connections 2023-09-23T03:17:18.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.282+0000 7f0ac11cb740 1 -- 172.21.15.154:0/1033861994 wait complete. 2023-09-23T03:17:18.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.282+0000 7f0ac11cb740 1 -- start start 2023-09-23T03:17:18.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.282+0000 7f0ac11cb740 1 Processor -- start 2023-09-23T03:17:18.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.283+0000 7f0ac11cb740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c38f6ac00 0x564c38f9a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:18.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.283+0000 7f0ac11cb740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c38f6afe0 0x564c38f7e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:18.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.283+0000 7f0ac11cb740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c38f9be60 0x564c38fefe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:18.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.283+0000 7f0ac11cb740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c38f877a0 con 0x564c38f9be60 2023-09-23T03:17:18.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.283+0000 7f0ac11cb740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c38f886e0 con 0x564c38f6ac00 2023-09-23T03:17:18.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.283+0000 7f0ac11cb740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c38f88930 con 0x564c38f6afe0 2023-09-23T03:17:18.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.284+0000 7f0ac11cb740 10 client.0 _ll_register_callbacks cb 0x7f0aa80070d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:17:18.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.284+0000 7f0abcbd6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c38f9be60 0x564c38fefe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:18.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.284+0000 7f0abcbd6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c38f9be60 0x564c38fefe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46546/0 (socket says 172.21.15.154:46546) 2023-09-23T03:17:18.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.284+0000 7f0abcbd6640 1 -- 172.21.15.154:0/3227871941 learned_addr learned my addr 172.21.15.154:0/3227871941 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:18.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.284+0000 7f0ab7fff640 1 --2- 172.21.15.154:0/3227871941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c38f6ac00 0x564c38f9a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:18.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.284+0000 7f0ab77fe640 1 --2- 172.21.15.154:0/3227871941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c38f6afe0 0x564c38f7e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:18.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.284+0000 7f0abcbd6640 1 -- 172.21.15.154:0/3227871941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c38f6afe0 msgr2=0x564c38f7e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:18.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.284+0000 7f0abcbd6640 1 --2- 172.21.15.154:0/3227871941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c38f6afe0 0x564c38f7e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.284+0000 7f0abcbd6640 1 -- 172.21.15.154:0/3227871941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c38f6ac00 msgr2=0x564c38f9a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:18.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.284+0000 7f0abcbd6640 1 --2- 172.21.15.154:0/3227871941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c38f6ac00 0x564c38f9a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.284+0000 7f0abcbd6640 1 -- 172.21.15.154:0/3227871941 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c38ea8d30 con 0x564c38f9be60 2023-09-23T03:17:18.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.284+0000 7f0ab7fff640 1 --2- 172.21.15.154:0/3227871941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c38f6ac00 0x564c38f9a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:18.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.284+0000 7f0abcbd6640 1 --2- 172.21.15.154:0/3227871941 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c38f9be60 0x564c38fefe00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0007410 tx=0x7f0ab005ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=281b051e6b6ec2c1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:18.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.284+0000 7f0a96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:17:18.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.285+0000 7f0a96ffd640 1 -- 172.21.15.154:0/3227871941 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0060840 con 0x564c38f9be60 2023-09-23T03:17:18.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.285+0000 7f0a96ffd640 1 -- 172.21.15.154:0/3227871941 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0ab0059b00 con 0x564c38f9be60 2023-09-23T03:17:18.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.285+0000 7f0ac11cb740 1 -- 172.21.15.154:0/3227871941 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0aa800da00 con 0x564c38f9be60 2023-09-23T03:17:18.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.285+0000 7f0a96ffd640 1 -- 172.21.15.154:0/3227871941 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ab0007e80 con 0x564c38f9be60 2023-09-23T03:17:18.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.285+0000 7f0ac11cb740 10 client.5658 fetch_fsmap learned FSMap version 18 2023-09-23T03:17:18.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.285+0000 7f0ac11cb740 1 -- 172.21.15.154:0/3227871941 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x564c38e47b50 con 0x564c38f9be60 2023-09-23T03:17:18.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.285+0000 7f0a96ffd640 1 -- 172.21.15.154:0/3227871941 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa800da00 con 0x564c38f9be60 2023-09-23T03:17:18.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.285+0000 7f0a96ffd640 1 -- 172.21.15.154:0/3227871941 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0ab0059cc0 con 0x564c38f9be60 2023-09-23T03:17:18.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.285+0000 7f0ac11cb740 10 client.5658 fetch_fsmap finished waiting for FSMap version 18 2023-09-23T03:17:18.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.285+0000 7f0ac11cb740 10 client.5658 resolve_mds: resolved name 'd' to daemon mds.d 2023-09-23T03:17:18.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.285+0000 7f0ac11cb740 20 client.5658 populate_metadata read hostname 'smithi154' 2023-09-23T03:17:18.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.285+0000 7f0ac11cb740 1 --2- 172.21.15.154:0/3227871941 >> [v2:172.21.15.190:6832/2131436414,v1:172.21.15.190:6834/2131436414] conn(0x564c38f80730 0x564c38ff39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:18.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.285+0000 7f0ac11cb740 4 client.5658 mds_command: new command op to 5002 tid=0[{"prefix": "status"}] 2023-09-23T03:17:18.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.285+0000 7f0ac11cb740 1 -- 172.21.15.154:0/3227871941 --> [v2:172.21.15.190:6832/2131436414,v1:172.21.15.190:6834/2131436414] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564c38bac6c0 con 0x564c38f80730 2023-09-23T03:17:18.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.286+0000 7f0ab7fff640 1 --2- 172.21.15.154:0/3227871941 >> [v2:172.21.15.190:6832/2131436414,v1:172.21.15.190:6834/2131436414] conn(0x564c38f80730 0x564c38ff39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:18.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.287+0000 7f0ab7fff640 1 --2- 172.21.15.154:0/3227871941 >> [v2:172.21.15.190:6832/2131436414,v1:172.21.15.190:6834/2131436414] conn(0x564c38f80730 0x564c38ff39c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baece84a435c11f7 server_cookie=368bfa52658f4e99 in_seq=0 out_seq=0 2023-09-23T03:17:18.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.287+0000 7f0a96ffd640 10 client.5658 ms_handle_connect on v2:172.21.15.190:6832/2131436414 2023-09-23T03:17:18.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.287+0000 7f0a96ffd640 1 -- 172.21.15.154:0/3227871941 <== mds.0 v2:172.21.15.190:6832/2131436414 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x564c38bac6c0 con 0x564c38f80730 2023-09-23T03:17:18.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.287+0000 7f0a96ffd640 10 client.5658 handle_command_reply: tid=0 2023-09-23T03:17:18.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.287+0000 7f0ac11cb740 1 client.5658 shutdown 2023-09-23T03:17:18.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.288+0000 7f0ac11cb740 1 -- 172.21.15.154:0/3227871941 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c38f9be60 msgr2=0x564c38fefe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:18.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.288+0000 7f0ac11cb740 1 --2- 172.21.15.154:0/3227871941 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c38f9be60 0x564c38fefe00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0007410 tx=0x7f0ab005ca60 comp rx=0 tx=0).stop 2023-09-23T03:17:18.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.288+0000 7f0ac11cb740 1 -- 172.21.15.154:0/3227871941 shutdown_connections 2023-09-23T03:17:18.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.288+0000 7f0ac11cb740 1 --2- 172.21.15.154:0/3227871941 >> [v2:172.21.15.190:6832/2131436414,v1:172.21.15.190:6834/2131436414] conn(0x564c38f80730 0x564c38ff39c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.288+0000 7f0ac11cb740 1 --2- 172.21.15.154:0/3227871941 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564c38f9be60 0x564c38fefe00 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.288+0000 7f0ac11cb740 1 --2- 172.21.15.154:0/3227871941 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564c38f6afe0 0x564c38f7e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.288+0000 7f0ac11cb740 1 --2- 172.21.15.154:0/3227871941 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564c38f6ac00 0x564c38f9a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.288+0000 7f0ac11cb740 1 -- 172.21.15.154:0/3227871941 >> 172.21.15.154:0/3227871941 conn(0x564c38f3e040 msgr2=0x564c38f6f850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:18.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.288+0000 7f0a96ffd640 0 client.5658 ms_handle_reset on v2:172.21.15.190:6832/2131436414 2023-09-23T03:17:18.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.288+0000 7f0ac11cb740 1 -- 172.21.15.154:0/3227871941 shutdown_connections 2023-09-23T03:17:18.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.288+0000 7f0ac11cb740 1 -- 172.21.15.154:0/3227871941 wait complete. 2023-09-23T03:17:18.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.288+0000 7f0ac11cb740 20 client.5658 trim_cache size 0 max 16384 2023-09-23T03:17:18.295 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:17:18.295 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_fsid": "8c8039d3-6c84-43db-8808-85e0dbdd47c8", 2023-09-23T03:17:18.295 INFO:teuthology.orchestra.run.smithi154.stdout: "whoami": 0, 2023-09-23T03:17:18.295 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 5002, 2023-09-23T03:17:18.295 INFO:teuthology.orchestra.run.smithi154.stdout: "want_state": "up:active", 2023-09-23T03:17:18.295 INFO:teuthology.orchestra.run.smithi154.stdout: "state": "up:active", 2023-09-23T03:17:18.295 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs", 2023-09-23T03:17:18.296 INFO:teuthology.orchestra.run.smithi154.stdout: "rank_uptime": 6.6304177119999999, 2023-09-23T03:17:18.296 INFO:teuthology.orchestra.run.smithi154.stdout: "mdsmap_epoch": 18, 2023-09-23T03:17:18.296 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_epoch": 38, 2023-09-23T03:17:18.296 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_epoch_barrier": 38, 2023-09-23T03:17:18.296 INFO:teuthology.orchestra.run.smithi154.stdout: "uptime": 15.460675277 2023-09-23T03:17:18.296 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:17:18.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.288+0000 7f0abe639640 1 -- 172.21.15.154:0/868751337 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0a9806af30 msgr2=0x7f0a9806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:18.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.289+0000 7f0abe639640 1 --2- 172.21.15.154:0/868751337 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0a9806af30 0x7f0a9806d3f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80014f0 tx=0x7f0aa8009040 comp rx=0 tx=0).stop 2023-09-23T03:17:18.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.289+0000 7f0abe639640 1 -- 172.21.15.154:0/868751337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ab80cee70 msgr2=0x7f0ab815b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:18.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.289+0000 7f0abe639640 1 --2- 172.21.15.154:0/868751337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ab80cee70 0x7f0ab815b810 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000c7c0 tx=0x7f0ab000cc80 comp rx=0 tx=0).stop 2023-09-23T03:17:18.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.289+0000 7f0abe639640 1 -- 172.21.15.154:0/868751337 shutdown_connections 2023-09-23T03:17:18.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.289+0000 7f0abe639640 1 --2- 172.21.15.154:0/868751337 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0a9806af30 0x7f0a9806d3f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.289+0000 7f0abe639640 1 --2- 172.21.15.154:0/868751337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ab80cee70 0x7f0ab815b810 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.289+0000 7f0abe639640 1 --2- 172.21.15.154:0/868751337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ab80b1b20 0x7f0ab81595e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.289+0000 7f0abe639640 1 --2- 172.21.15.154:0/868751337 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ab80acce0 0x7f0ab80c1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.289+0000 7f0abe639640 1 -- 172.21.15.154:0/868751337 >> 172.21.15.154:0/868751337 conn(0x7f0ab806a0f0 msgr2=0x7f0ab80ab140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:18.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.289+0000 7f0abe639640 1 -- 172.21.15.154:0/868751337 shutdown_connections 2023-09-23T03:17:18.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.289+0000 7f0abe639640 1 -- 172.21.15.154:0/868751337 wait complete. 2023-09-23T03:17:18.309 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-23T03:17:18.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.515+0000 7effeb59e640 1 Processor -- start 2023-09-23T03:17:18.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.515+0000 7effeb59e640 1 -- start start 2023-09-23T03:17:18.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.516+0000 7effeb59e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7effec15fe10 0x7effec0b4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:18.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.517+0000 7effeb59e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effec0ce230 0x7effec0b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:18.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.517+0000 7effeb59e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effec1601f0 con 0x7effec0d2eb0 2023-09-23T03:17:18.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.517+0000 7effeb59e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effec06ad80 con 0x7effec15fe10 2023-09-23T03:17:18.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.517+0000 7effeb59e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effec0d0cf0 con 0x7effec0ce230 2023-09-23T03:17:18.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.517+0000 7effea59c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7effec15fe10 0x7effec0b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:18.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.517+0000 7effea59c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7effec15fe10 0x7effec0b4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46558/0 (socket says 172.21.15.154:46558) 2023-09-23T03:17:18.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.517+0000 7effea59c640 1 -- 172.21.15.154:0/3081602412 learned_addr learned my addr 172.21.15.154:0/3081602412 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:18.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.517+0000 7effead9d640 1 --2- 172.21.15.154:0/3081602412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effec0ce230 0x7effec0b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:18.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.517+0000 7effea59c640 1 -- 172.21.15.154:0/3081602412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effec0ce230 msgr2=0x7effec0b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:18.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.517+0000 7effea59c640 1 --2- 172.21.15.154:0/3081602412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effec0ce230 0x7effec0b4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.517+0000 7effea59c640 1 -- 172.21.15.154:0/3081602412 >> v1:172.21.15.190:6789/0 conn(0x7effec0d2eb0 legacy=0x7effec06b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:18.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.517+0000 7effea59c640 1 -- 172.21.15.154:0/3081602412 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effec0d0fd0 con 0x7effec15fe10 2023-09-23T03:17:18.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.519+0000 7effea59c640 1 --2- 172.21.15.154:0/3081602412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effec15fe10 0x7effec0b4220 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7effdc009ea0 tx=0x7effdc02bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=126909cf92625904 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:18.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.519+0000 7effe959a640 1 -- 172.21.15.154:0/3081602412 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effdc02cb40 con 0x7effec15fe10 2023-09-23T03:17:18.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.519+0000 7effe959a640 1 -- 172.21.15.154:0/3081602412 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effdc02cca0 con 0x7effec15fe10 2023-09-23T03:17:18.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.519+0000 7effe959a640 1 -- 172.21.15.154:0/3081602412 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effdc034610 con 0x7effec15fe10 2023-09-23T03:17:18.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.519+0000 7effeb59e640 1 -- 172.21.15.154:0/3081602412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effec15fe10 msgr2=0x7effec0b4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:18.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.519+0000 7effeb59e640 1 --2- 172.21.15.154:0/3081602412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effec15fe10 0x7effec0b4220 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7effdc009ea0 tx=0x7effdc02bfa0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.520+0000 7effeb59e640 1 -- 172.21.15.154:0/3081602412 shutdown_connections 2023-09-23T03:17:18.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.520+0000 7effeb59e640 1 --2- 172.21.15.154:0/3081602412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effec0ce230 0x7effec0b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.520+0000 7effeb59e640 1 --2- 172.21.15.154:0/3081602412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effec15fe10 0x7effec0b4220 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.520+0000 7effeb59e640 1 -- 172.21.15.154:0/3081602412 >> 172.21.15.154:0/3081602412 conn(0x7effec09ba70 msgr2=0x7effec0d27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:18.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.520+0000 7effeb59e640 1 -- 172.21.15.154:0/3081602412 shutdown_connections 2023-09-23T03:17:18.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.520+0000 7effeb59e640 1 -- 172.21.15.154:0/3081602412 wait complete. 2023-09-23T03:17:18.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.521+0000 7effeb59e640 1 Processor -- start 2023-09-23T03:17:18.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.522+0000 7effeb59e640 1 -- start start 2023-09-23T03:17:18.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.523+0000 7effeb59e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effec0ce230 0x7effec0b6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:18.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.523+0000 7effeb59e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effec0d2eb0 0x7effec15ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:18.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.523+0000 7effeb59e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effec15fe10 0x7effec15e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:18.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.523+0000 7effeb59e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effec0c6b10 con 0x7effec0d2eb0 2023-09-23T03:17:18.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.523+0000 7effeb59e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effec0c6220 con 0x7effec0ce230 2023-09-23T03:17:18.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.523+0000 7effeb59e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effec0c8be0 con 0x7effec15fe10 2023-09-23T03:17:18.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.523+0000 7effe9d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effec0d2eb0 0x7effec15ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:18.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.523+0000 7effe9d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effec0d2eb0 0x7effec15ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46570/0 (socket says 172.21.15.154:46570) 2023-09-23T03:17:18.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.523+0000 7effe9d9b640 1 -- 172.21.15.154:0/2280996346 learned_addr learned my addr 172.21.15.154:0/2280996346 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:18.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.523+0000 7effea59c640 1 --2- 172.21.15.154:0/2280996346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effec0ce230 0x7effec0b6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:18.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.523+0000 7effead9d640 1 --2- 172.21.15.154:0/2280996346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effec15fe10 0x7effec15e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:18.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.523+0000 7effe9d9b640 1 -- 172.21.15.154:0/2280996346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effec15fe10 msgr2=0x7effec15e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:18.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.523+0000 7effe9d9b640 1 --2- 172.21.15.154:0/2280996346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effec15fe10 0x7effec15e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.523+0000 7effe9d9b640 1 -- 172.21.15.154:0/2280996346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effec0ce230 msgr2=0x7effec0b6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:18.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.523+0000 7effe9d9b640 1 --2- 172.21.15.154:0/2280996346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effec0ce230 0x7effec0b6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.523+0000 7effe9d9b640 1 -- 172.21.15.154:0/2280996346 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effdc009b30 con 0x7effec0d2eb0 2023-09-23T03:17:18.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.523+0000 7effead9d640 1 --2- 172.21.15.154:0/2280996346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effec15fe10 0x7effec15e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:18.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.524+0000 7effe9d9b640 1 --2- 172.21.15.154:0/2280996346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effec0d2eb0 0x7effec15ddf0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7effd400c450 tx=0x7effd400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:18.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.524+0000 7effe2ffd640 1 -- 172.21.15.154:0/2280996346 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd40052d0 con 0x7effec0d2eb0 2023-09-23T03:17:18.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.524+0000 7effe2ffd640 1 -- 172.21.15.154:0/2280996346 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd4005430 con 0x7effec0d2eb0 2023-09-23T03:17:18.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.524+0000 7effe2ffd640 1 -- 172.21.15.154:0/2280996346 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd4012870 con 0x7effec0d2eb0 2023-09-23T03:17:18.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.524+0000 7effeb59e640 1 -- 172.21.15.154:0/2280996346 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effec150880 con 0x7effec0d2eb0 2023-09-23T03:17:18.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.524+0000 7effeb59e640 1 -- 172.21.15.154:0/2280996346 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effec0c67a0 con 0x7effec0d2eb0 2023-09-23T03:17:18.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.525+0000 7effeb59e640 1 -- 172.21.15.154:0/2280996346 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effec0c6960 con 0x7effec0d2eb0 2023-09-23T03:17:18.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.526+0000 7effe2ffd640 1 -- 172.21.15.154:0/2280996346 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7effd40129d0 con 0x7effec0d2eb0 2023-09-23T03:17:18.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.527+0000 7effe2ffd640 1 --2- 172.21.15.154:0/2280996346 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7effcc06af30 0x7effcc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:18.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.527+0000 7effe2ffd640 1 -- 172.21.15.154:0/2280996346 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7effd4089270 con 0x7effec0d2eb0 2023-09-23T03:17:18.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.527+0000 7effea59c640 1 --2- 172.21.15.154:0/2280996346 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7effcc06af30 0x7effcc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:18.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.528+0000 7effea59c640 1 --2- 172.21.15.154:0/2280996346 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7effcc06af30 0x7effcc06d3f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7effdc035000 tx=0x7effdc030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:18.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.531+0000 7effe2ffd640 1 -- 172.21.15.154:0/2280996346 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7effd4054170 con 0x7effec0d2eb0 2023-09-23T03:17:18.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.764+0000 7effeb59e640 1 -- 172.21.15.154:0/2280996346 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7effec1601f0 con 0x7effec0d2eb0 2023-09-23T03:17:18.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.765+0000 7effe2ffd640 1 -- 172.21.15.154:0/2280996346 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+73 (secure 0 0 0) 0x7effec1601f0 con 0x7effec0d2eb0 2023-09-23T03:17:18.766 INFO:teuthology.orchestra.run.smithi154.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-09-23T03:17:18.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.768+0000 7effeb59e640 1 -- 172.21.15.154:0/2280996346 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7effcc06af30 msgr2=0x7effcc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:18.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.768+0000 7effeb59e640 1 --2- 172.21.15.154:0/2280996346 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7effcc06af30 0x7effcc06d3f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7effdc035000 tx=0x7effdc030000 comp rx=0 tx=0).stop 2023-09-23T03:17:18.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.768+0000 7effeb59e640 1 -- 172.21.15.154:0/2280996346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effec0d2eb0 msgr2=0x7effec15ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:18.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.768+0000 7effeb59e640 1 --2- 172.21.15.154:0/2280996346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effec0d2eb0 0x7effec15ddf0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7effd400c450 tx=0x7effd400c910 comp rx=0 tx=0).stop 2023-09-23T03:17:18.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.768+0000 7effeb59e640 1 -- 172.21.15.154:0/2280996346 shutdown_connections 2023-09-23T03:17:18.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.768+0000 7effeb59e640 1 --2- 172.21.15.154:0/2280996346 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7effcc06af30 0x7effcc06d3f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.768+0000 7effeb59e640 1 --2- 172.21.15.154:0/2280996346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effec15fe10 0x7effec15e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.768+0000 7effeb59e640 1 --2- 172.21.15.154:0/2280996346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effec0d2eb0 0x7effec15ddf0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.768+0000 7effeb59e640 1 --2- 172.21.15.154:0/2280996346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effec0ce230 0x7effec0b6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:18.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.768+0000 7effeb59e640 1 -- 172.21.15.154:0/2280996346 >> 172.21.15.154:0/2280996346 conn(0x7effec09ba70 msgr2=0x7effec0cf220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:18.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.768+0000 7effeb59e640 1 -- 172.21.15.154:0/2280996346 shutdown_connections 2023-09-23T03:17:18.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:18.768+0000 7effeb59e640 1 -- 172.21.15.154:0/2280996346 wait complete. 2023-09-23T03:17:18.788 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-23T03:17:18.788 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-23T03:17:18.789 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi154.front.sepia.ceph.com 2023-09-23T03:17:18.789 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-09-23T03:17:18.789 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-23T03:17:18.789 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-23T03:17:18.789 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-23T03:17:18.789 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-23T03:17:18.789 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-23T03:17:18.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:17:18.789 DEBUG:teuthology.orchestra.run.smithi154:> ip addr 2023-09-23T03:17:18.844 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-23T03:17:18.844 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-23T03:17:18.844 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2023-09-23T03:17:18.844 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:17:18.844 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2023-09-23T03:17:18.844 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:17:18.845 INFO:teuthology.orchestra.run.smithi154.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:17:18.845 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:54 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:17:18.845 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f0 2023-09-23T03:17:18.845 INFO:teuthology.orchestra.run.smithi154.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:17:18.845 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:55 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:17:18.845 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f1 2023-09-23T03:17:18.845 INFO:teuthology.orchestra.run.smithi154.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:17:18.845 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:60 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:17:18.845 INFO:teuthology.orchestra.run.smithi154.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-23T03:17:18.845 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:61 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:17:18.845 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.21.15.154/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-09-23T03:17:18.846 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft 42286sec preferred_lft 42286sec 2023-09-23T03:17:18.846 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::ec4:7aff:fe88:7761/64 scope link 2023-09-23T03:17:18.846 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:17:18.846 INFO:teuthology.orchestra.run.smithi154.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-23T03:17:18.846 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 5a:c6:b2:96:fb:dc brd ff:ff:ff:ff:ff:ff 2023-09-23T03:17:18.846 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-09-23T03:17:18.846 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:17:18.846 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::a801:22ff:febf:2833/64 scope link 2023-09-23T03:17:18.846 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:17:18.846 INFO:teuthology.orchestra.run.smithi154.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-23T03:17:18.846 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 5a:c6:b2:96:fb:dc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:17:18.847 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::58c6:b2ff:fe96:fbdc/64 scope link 2023-09-23T03:17:18.847 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:17:18.847 INFO:teuthology.orchestra.run.smithi154.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-23T03:17:18.847 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether b2:23:b0:e4:c2:ab brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:17:18.847 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::b023:b0ff:fee4:c2ab/64 scope link 2023-09-23T03:17:18.847 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:17:18.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:17:18.848 DEBUG:teuthology.orchestra.run.smithi154:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-23T03:17:18.937 INFO:teuthology.orchestra.run.smithi154.stdout:1 2023-09-23T03:17:18.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:17:18.942 DEBUG:teuthology.orchestra.run.smithi154:> route 2023-09-23T03:17:18.997 INFO:teuthology.orchestra.run.smithi154.stdout:Kernel IP routing table 2023-09-23T03:17:18.997 INFO:teuthology.orchestra.run.smithi154.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-23T03:17:18.997 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-23T03:17:18.997 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-09-23T03:17:18.998 INFO:teuthology.orchestra.run.smithi154.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-23T03:17:18.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:17:18.998 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-09-23T03:17:18.998 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-09-23T03:17:18.998 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-23T03:17:18.998 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-23T03:17:18.998 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-23T03:17:18.999 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-09-23T03:17:19.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:17:19.187 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2023-09-23T03:17:19.238 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-09-23T03:17:19.238 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-09-23T03:17:19.239 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-23T03:17:19.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:17:19.240 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set brx.0 up 2023-09-23T03:17:19.325 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-09-23T03:17:19.325 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-23T03:17:19.325 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-09-23T03:17:19.379 INFO:teuthology.orchestra.run.smithi154.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-09-23T03:17:19.380 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-23T03:17:19.380 DEBUG:teuthology.orchestra.run.smithi154:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-09-23T03:17:19.431 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-09-23T03:17:19.530 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2023-09-23T03:17:19.589 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2023-09-23T03:17:19.590 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2023-09-23T03:17:19.590 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2023-09-23T03:17:19.590 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2023-09-23T03:17:19.590 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2023-09-23T03:17:19.590 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2023-09-23T03:17:19.590 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2023-09-23T03:17:19.590 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2023-09-23T03:17:19.590 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2023-09-23T03:17:19.591 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2023-09-23T03:17:19.591 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2023-09-23T03:17:19.591 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T03:17:19.591 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2023-09-23T03:17:19.591 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2023-09-23T03:17:19.591 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2023-09-23T03:17:19.591 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2023-09-23T03:17:19.591 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2023-09-23T03:17:19.591 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2023-09-23T03:17:19.591 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2023-09-23T03:17:19.591 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-09-23T03:17:19.592 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-09-23T03:17:19.592 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2023-09-23T03:17:19.592 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2023-09-23T03:17:19.592 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2023-09-23T03:17:19.592 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-0 2023-09-23T03:17:19.592 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-1 2023-09-23T03:17:19.592 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-2 2023-09-23T03:17:19.592 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-3 2023-09-23T03:17:19.592 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2023-09-23T03:17:19.592 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:17:19.593 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:17:19.593 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:17:19.593 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:17:19.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:17:19.594 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-09-23T03:17:19.641 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-23T03:17:19.642 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-09-23T03:17:19.691 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-09-23T03:17:19.777 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2023-09-23T03:17:19.819 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2023-09-23T03:17:19.819 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2023-09-23T03:17:19.819 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2023-09-23T03:17:19.819 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2023-09-23T03:17:19.819 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2023-09-23T03:17:19.820 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2023-09-23T03:17:19.820 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2023-09-23T03:17:19.820 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2023-09-23T03:17:19.820 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2023-09-23T03:17:19.820 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2023-09-23T03:17:19.820 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2023-09-23T03:17:19.820 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T03:17:19.820 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2023-09-23T03:17:19.820 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2023-09-23T03:17:19.820 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2023-09-23T03:17:19.820 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2023-09-23T03:17:19.820 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2023-09-23T03:17:19.821 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2023-09-23T03:17:19.821 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2023-09-23T03:17:19.821 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-09-23T03:17:19.821 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-09-23T03:17:19.821 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2023-09-23T03:17:19.821 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2023-09-23T03:17:19.821 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2023-09-23T03:17:19.821 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-0 2023-09-23T03:17:19.821 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-1 2023-09-23T03:17:19.821 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-2 2023-09-23T03:17:19.821 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-3 2023-09-23T03:17:19.821 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2023-09-23T03:17:19.822 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:17:19.822 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:17:19.822 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:17:19.822 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:17:19.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:17:19.823 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-09-23T03:17:19.884 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:2023-09-23T03:17:19.883+0000 7f9f01e3e480 -1 init, newargv = 0x55ae26f91c00 newargc=15 2023-09-23T03:17:19.884 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:ceph-fuse[103090]: starting ceph client 2023-09-23T03:17:19.895 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:ceph-fuse[103090]: starting fuse 2023-09-23T03:17:19.908 INFO:teuthology.orchestra.run.smithi154.stdout:39 2023-09-23T03:17:19.908 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-09-23T03:17:19.908 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:17:19.908 DEBUG:teuthology.orchestra.run.smithi154:> import glob 2023-09-23T03:17:19.908 DEBUG:teuthology.orchestra.run.smithi154:> import re 2023-09-23T03:17:19.908 DEBUG:teuthology.orchestra.run.smithi154:> import os 2023-09-23T03:17:19.909 DEBUG:teuthology.orchestra.run.smithi154:> import subprocess 2023-09-23T03:17:19.909 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:17:19.909 DEBUG:teuthology.orchestra.run.smithi154:> def _find_admin_socket(client_name): 2023-09-23T03:17:19.909 DEBUG:teuthology.orchestra.run.smithi154:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-09-23T03:17:19.909 DEBUG:teuthology.orchestra.run.smithi154:> files = glob.glob(asok_path) 2023-09-23T03:17:19.909 DEBUG:teuthology.orchestra.run.smithi154:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-09-23T03:17:19.909 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:17:19.909 DEBUG:teuthology.orchestra.run.smithi154:> # Given a non-glob path, it better be there 2023-09-23T03:17:19.909 DEBUG:teuthology.orchestra.run.smithi154:> if "*" not in asok_path: 2023-09-23T03:17:19.909 DEBUG:teuthology.orchestra.run.smithi154:> assert(len(files) == 1) 2023-09-23T03:17:19.909 DEBUG:teuthology.orchestra.run.smithi154:> return files[0] 2023-09-23T03:17:19.909 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:17:19.910 DEBUG:teuthology.orchestra.run.smithi154:> for f in files: 2023-09-23T03:17:19.910 DEBUG:teuthology.orchestra.run.smithi154:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-23T03:17:19.910 DEBUG:teuthology.orchestra.run.smithi154:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-23T03:17:19.910 DEBUG:teuthology.orchestra.run.smithi154:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-23T03:17:19.910 DEBUG:teuthology.orchestra.run.smithi154:> contents = proc_f.read() 2023-09-23T03:17:19.910 DEBUG:teuthology.orchestra.run.smithi154:> if mountpoint in contents: 2023-09-23T03:17:19.910 DEBUG:teuthology.orchestra.run.smithi154:> return f 2023-09-23T03:17:19.910 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-23T03:17:19.910 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:17:19.910 DEBUG:teuthology.orchestra.run.smithi154:> print(_find_admin_socket("client.0")) 2023-09-23T03:17:19.910 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:17:20.112 INFO:teuthology.orchestra.run.smithi154.stdout:/var/run/ceph/ceph-client.0.103090.asok 2023-09-23T03:17:20.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:17:20.128 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.103090.asok 2023-09-23T03:17:20.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:17:20.129 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.103090.asok status 2023-09-23T03:17:20.368 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:17:20.368 INFO:teuthology.orchestra.run.smithi154.stdout: "metadata": { 2023-09-23T03:17:20.368 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_sha1": "862663913a74a99e1c52d50e6a7e59dd6705749a", 2023-09-23T03:17:20.369 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_version": "ceph version 18.0.0-6336-g86266391 (862663913a74a99e1c52d50e6a7e59dd6705749a) reef (dev)", 2023-09-23T03:17:20.369 INFO:teuthology.orchestra.run.smithi154.stdout: "entity_id": "0", 2023-09-23T03:17:20.369 INFO:teuthology.orchestra.run.smithi154.stdout: "hostname": "smithi154", 2023-09-23T03:17:20.369 INFO:teuthology.orchestra.run.smithi154.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-09-23T03:17:20.369 INFO:teuthology.orchestra.run.smithi154.stdout: "pid": "103090", 2023-09-23T03:17:20.369 INFO:teuthology.orchestra.run.smithi154.stdout: "root": "/" 2023-09-23T03:17:20.369 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:17:20.369 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_count": 0, 2023-09-23T03:17:20.369 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_pinned_count": 0, 2023-09-23T03:17:20.370 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 5667, 2023-09-23T03:17:20.370 INFO:teuthology.orchestra.run.smithi154.stdout: "inst": { 2023-09-23T03:17:20.370 INFO:teuthology.orchestra.run.smithi154.stdout: "name": { 2023-09-23T03:17:20.370 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "client", 2023-09-23T03:17:20.370 INFO:teuthology.orchestra.run.smithi154.stdout: "num": 5667 2023-09-23T03:17:20.370 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:17:20.370 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2023-09-23T03:17:20.370 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2023-09-23T03:17:20.370 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.1:0", 2023-09-23T03:17:20.370 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3099958610 2023-09-23T03:17:20.371 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:17:20.371 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:17:20.371 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2023-09-23T03:17:20.371 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2023-09-23T03:17:20.371 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.1:0", 2023-09-23T03:17:20.371 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3099958610 2023-09-23T03:17:20.371 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:17:20.371 INFO:teuthology.orchestra.run.smithi154.stdout: "inst_str": "client.5667 192.168.0.1:0/3099958610", 2023-09-23T03:17:20.371 INFO:teuthology.orchestra.run.smithi154.stdout: "addr_str": "192.168.0.1:0/3099958610", 2023-09-23T03:17:20.371 INFO:teuthology.orchestra.run.smithi154.stdout: "inode_count": 1, 2023-09-23T03:17:20.372 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_epoch": 18, 2023-09-23T03:17:20.372 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch": 38, 2023-09-23T03:17:20.372 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch_barrier": 0, 2023-09-23T03:17:20.372 INFO:teuthology.orchestra.run.smithi154.stdout: "blocklisted": false, 2023-09-23T03:17:20.372 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs" 2023-09-23T03:17:20.372 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:17:20.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:17:20.383 DEBUG:teuthology.orchestra.run.smithi154:> stat --file-system '--printf=%T 2023-09-23T03:17:20.383 DEBUG:teuthology.orchestra.run.smithi154:> ' -- /home/ubuntu/cephtest/mnt.0 2023-09-23T03:17:20.439 INFO:teuthology.orchestra.run.smithi154.stdout:fuseblk 2023-09-23T03:17:20.440 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-09-23T03:17:20.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:17:20.440 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-09-23T03:17:20.542 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_extra --pg_num_min 64 2023-09-23T03:17:20.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.763+0000 7f6a3cf61640 1 Processor -- start 2023-09-23T03:17:20.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.763+0000 7f6a3cf61640 1 -- start start 2023-09-23T03:17:20.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.765+0000 7f6a3cf61640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6a38069cd0 0x7f6a38071fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:20.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.765+0000 7f6a3cf61640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6a38071860 0x7f6a3806a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:20.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.765+0000 7f6a3cf61640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a38071090 con 0x7f6a3815dcd0 2023-09-23T03:17:20.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.765+0000 7f6a3cf61640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a3806a5f0 con 0x7f6a38069cd0 2023-09-23T03:17:20.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.765+0000 7f6a3cf61640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a38071610 con 0x7f6a38071860 2023-09-23T03:17:20.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.765+0000 7f6a3659c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6a38069cd0 0x7f6a38071fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:20.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.765+0000 7f6a36d9d640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f6a3815dcd0 0x7f6a3815e730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40744/0 (socket says 172.21.15.154:40744) 2023-09-23T03:17:20.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.765+0000 7f6a36d9d640 1 -- 172.21.15.154:0/3631387144 learned_addr learned my addr 172.21.15.154:0/3631387144 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:17:20.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.765+0000 7f6a2ffff640 1 --2- 172.21.15.154:0/3631387144 >> v2:172.21.15.190:3300/0 conn(0x7f6a38071860 0x7f6a3806a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:20.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.765+0000 7f6a3659c640 1 -- 172.21.15.154:0/3631387144 >> v2:172.21.15.190:3300/0 conn(0x7f6a38071860 msgr2=0x7f6a3806a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:20.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.765+0000 7f6a3659c640 1 --2- 172.21.15.154:0/3631387144 >> v2:172.21.15.190:3300/0 conn(0x7f6a38071860 0x7f6a3806a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:20.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.765+0000 7f6a3659c640 1 -- 172.21.15.154:0/3631387144 >> v1:172.21.15.154:6789/0 conn(0x7f6a3815dcd0 legacy=0x7f6a3815e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:20.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.765+0000 7f6a3659c640 1 -- 172.21.15.154:0/3631387144 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a38071330 con 0x7f6a38069cd0 2023-09-23T03:17:20.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.765+0000 7f6a2ffff640 1 --2- 172.21.15.154:0/3631387144 >> v2:172.21.15.190:3300/0 conn(0x7f6a38071860 0x7f6a3806a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:20.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.766+0000 7f6a3659c640 1 --2- 172.21.15.154:0/3631387144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a38069cd0 0x7f6a38071fd0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f6a380695b0 tx=0x7f6a2802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa84e38a6b0eb53 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:20.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.766+0000 7f6a35d9b640 1 -- 172.21.15.154:0/3631387144 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a28031830 con 0x7f6a38069cd0 2023-09-23T03:17:20.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.766+0000 7f6a35d9b640 1 -- 172.21.15.154:0/3631387144 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a28031990 con 0x7f6a38069cd0 2023-09-23T03:17:20.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.766+0000 7f6a35d9b640 1 -- 172.21.15.154:0/3631387144 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a28031d20 con 0x7f6a38069cd0 2023-09-23T03:17:20.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.766+0000 7f6a3cf61640 1 -- 172.21.15.154:0/3631387144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a38069cd0 msgr2=0x7f6a38071fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:20.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.766+0000 7f6a3cf61640 1 --2- 172.21.15.154:0/3631387144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a38069cd0 0x7f6a38071fd0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f6a380695b0 tx=0x7f6a2802b8d0 comp rx=0 tx=0).stop 2023-09-23T03:17:20.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.767+0000 7f6a3cf61640 1 -- 172.21.15.154:0/3631387144 shutdown_connections 2023-09-23T03:17:20.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.767+0000 7f6a3cf61640 1 --2- 172.21.15.154:0/3631387144 >> v2:172.21.15.190:3300/0 conn(0x7f6a38071860 0x7f6a3806a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:20.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.767+0000 7f6a3cf61640 1 --2- 172.21.15.154:0/3631387144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a38069cd0 0x7f6a38071fd0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:20.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.767+0000 7f6a3cf61640 1 -- 172.21.15.154:0/3631387144 >> 172.21.15.154:0/3631387144 conn(0x7f6a3806d010 msgr2=0x7f6a3806d420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:20.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.767+0000 7f6a3cf61640 1 -- 172.21.15.154:0/3631387144 shutdown_connections 2023-09-23T03:17:20.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.767+0000 7f6a3cf61640 1 -- 172.21.15.154:0/3631387144 wait complete. 2023-09-23T03:17:20.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.768+0000 7f6a3cf61640 1 Processor -- start 2023-09-23T03:17:20.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.769+0000 7f6a3cf61640 1 -- start start 2023-09-23T03:17:20.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.770+0000 7f6a3cf61640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a38069cd0 0x7f6a3814d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:20.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.770+0000 7f6a3cf61640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a38071860 0x7f6a380c5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:20.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.770+0000 7f6a3cf61640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a3815dcd0 0x7f6a381f63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:20.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.770+0000 7f6a3cf61640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3815ba60 con 0x7f6a38071860 2023-09-23T03:17:20.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.770+0000 7f6a3cf61640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3815bd40 con 0x7f6a3815dcd0 2023-09-23T03:17:20.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.770+0000 7f6a3cf61640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3815c020 con 0x7f6a38069cd0 2023-09-23T03:17:20.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.770+0000 7f6a2ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a38071860 0x7f6a380c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:20.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.770+0000 7f6a2ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a38071860 0x7f6a380c5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46590/0 (socket says 172.21.15.154:46590) 2023-09-23T03:17:20.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.770+0000 7f6a2ffff640 1 -- 172.21.15.154:0/4058223665 learned_addr learned my addr 172.21.15.154:0/4058223665 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:20.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.770+0000 7f6a3659c640 1 --2- 172.21.15.154:0/4058223665 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a38069cd0 0x7f6a3814d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:20.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.770+0000 7f6a36d9d640 1 --2- 172.21.15.154:0/4058223665 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a3815dcd0 0x7f6a381f63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:20.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.770+0000 7f6a2ffff640 1 -- 172.21.15.154:0/4058223665 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a38069cd0 msgr2=0x7f6a3814d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:20.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.770+0000 7f6a2ffff640 1 --2- 172.21.15.154:0/4058223665 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a38069cd0 0x7f6a3814d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:20.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.770+0000 7f6a2ffff640 1 -- 172.21.15.154:0/4058223665 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a3815dcd0 msgr2=0x7f6a381f63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:20.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.770+0000 7f6a2ffff640 1 --2- 172.21.15.154:0/4058223665 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a3815dcd0 0x7f6a381f63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:20.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.771+0000 7f6a2ffff640 1 -- 172.21.15.154:0/4058223665 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a28009400 con 0x7f6a38071860 2023-09-23T03:17:20.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.771+0000 7f6a3659c640 1 --2- 172.21.15.154:0/4058223665 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a38069cd0 0x7f6a3814d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:20.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.771+0000 7f6a36d9d640 1 --2- 172.21.15.154:0/4058223665 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a3815dcd0 0x7f6a381f63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:20.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.771+0000 7f6a2ffff640 1 --2- 172.21.15.154:0/4058223665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a38071860 0x7f6a380c5850 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6a2000c7f0 tx=0x7f6a2000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:20.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.771+0000 7f6a2effd640 1 -- 172.21.15.154:0/4058223665 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a20007a30 con 0x7f6a38071860 2023-09-23T03:17:20.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.771+0000 7f6a2effd640 1 -- 172.21.15.154:0/4058223665 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a20007b90 con 0x7f6a38071860 2023-09-23T03:17:20.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.771+0000 7f6a3cf61640 1 -- 172.21.15.154:0/4058223665 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a380c30f0 con 0x7f6a38071860 2023-09-23T03:17:20.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.771+0000 7f6a2effd640 1 -- 172.21.15.154:0/4058223665 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2000ce40 con 0x7f6a38071860 2023-09-23T03:17:20.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.771+0000 7f6a3cf61640 1 -- 172.21.15.154:0/4058223665 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a3815b680 con 0x7f6a38071860 2023-09-23T03:17:20.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.772+0000 7f6a3cf61640 1 -- 172.21.15.154:0/4058223665 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a380c3330 con 0x7f6a38071860 2023-09-23T03:17:20.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.774+0000 7f6a2effd640 1 -- 172.21.15.154:0/4058223665 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f6a200113f0 con 0x7f6a38071860 2023-09-23T03:17:20.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.774+0000 7f6a2effd640 1 --2- 172.21.15.154:0/4058223665 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6a1806af30 0x7f6a1806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:20.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.775+0000 7f6a2effd640 1 -- 172.21.15.154:0/4058223665 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6a20089ad0 con 0x7f6a38071860 2023-09-23T03:17:20.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.775+0000 7f6a3659c640 1 --2- 172.21.15.154:0/4058223665 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6a1806af30 0x7f6a1806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:20.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.775+0000 7f6a3659c640 1 --2- 172.21.15.154:0/4058223665 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6a1806af30 0x7f6a1806d3f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6a28036000 tx=0x7f6a2803c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:20.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.779+0000 7f6a2effd640 1 -- 172.21.15.154:0/4058223665 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f6a380c3330 con 0x7f6a38071860 2023-09-23T03:17:20.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:20.967+0000 7f6a3cf61640 1 -- 172.21.15.154:0/4058223665 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_extra", "pg_num_min": 64} v 0) v1 -- 0x7f6a3815bba0 con 0x7f6a38071860 2023-09-23T03:17:21.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.200+0000 7f6a2effd640 1 -- 172.21.15.154:0/4058223665 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_extra", "pg_num_min": 64}]=0 pool 'cephfs_data_extra' created v39) v1 ==== 142+0+0 (secure 0 0 0) 0x7f6a3815bba0 con 0x7f6a38071860 2023-09-23T03:17:21.201 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_data_extra' created 2023-09-23T03:17:21.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.206+0000 7f6a3cf61640 1 -- 172.21.15.154:0/4058223665 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6a1806af30 msgr2=0x7f6a1806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:21.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.206+0000 7f6a3cf61640 1 --2- 172.21.15.154:0/4058223665 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6a1806af30 0x7f6a1806d3f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6a28036000 tx=0x7f6a2803c040 comp rx=0 tx=0).stop 2023-09-23T03:17:21.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.206+0000 7f6a3cf61640 1 -- 172.21.15.154:0/4058223665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a38071860 msgr2=0x7f6a380c5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:21.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.206+0000 7f6a3cf61640 1 --2- 172.21.15.154:0/4058223665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a38071860 0x7f6a380c5850 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6a2000c7f0 tx=0x7f6a2000cbf0 comp rx=0 tx=0).stop 2023-09-23T03:17:21.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.206+0000 7f6a3cf61640 1 -- 172.21.15.154:0/4058223665 shutdown_connections 2023-09-23T03:17:21.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.206+0000 7f6a3cf61640 1 --2- 172.21.15.154:0/4058223665 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6a1806af30 0x7f6a1806d3f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:21.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.206+0000 7f6a3cf61640 1 --2- 172.21.15.154:0/4058223665 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a3815dcd0 0x7f6a381f63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:21.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.206+0000 7f6a3cf61640 1 --2- 172.21.15.154:0/4058223665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a38071860 0x7f6a380c5850 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:21.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.206+0000 7f6a3cf61640 1 --2- 172.21.15.154:0/4058223665 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a38069cd0 0x7f6a3814d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:21.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.206+0000 7f6a3cf61640 1 -- 172.21.15.154:0/4058223665 >> 172.21.15.154:0/4058223665 conn(0x7f6a3806d010 msgr2=0x7f6a38152c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:21.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.207+0000 7f6a3cf61640 1 -- 172.21.15.154:0/4058223665 shutdown_connections 2023-09-23T03:17:21.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.207+0000 7f6a3cf61640 1 -- 172.21.15.154:0/4058223665 wait complete. 2023-09-23T03:17:21.234 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_extra 2023-09-23T03:17:21.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.430+0000 7f45c2887640 1 Processor -- start 2023-09-23T03:17:21.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.430+0000 7f45c2887640 1 -- start start 2023-09-23T03:17:21.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.431+0000 7f45c2887640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f45bc070c60 0x7f45bc0caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:21.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.431+0000 7f45c2887640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45bc0712f0 con 0x7f45bc0cb400 2023-09-23T03:17:21.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.432+0000 7f45c2887640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45bc0715d0 con 0x7f45bc0ad830 2023-09-23T03:17:21.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.432+0000 7f45c2887640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45bc0710f0 con 0x7f45bc070c60 2023-09-23T03:17:21.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.432+0000 7f45bb7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f45bc070c60 0x7f45bc0caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:21.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.432+0000 7f45bb7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f45bc070c60 0x7f45bc0caec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46592/0 (socket says 172.21.15.154:46592) 2023-09-23T03:17:21.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.432+0000 7f45c0e24640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f45bc0cb400 0x7f45bc0aba70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40760/0 (socket says 172.21.15.154:40760) 2023-09-23T03:17:21.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.432+0000 7f45bb7fe640 1 -- 172.21.15.154:0/1398817756 learned_addr learned my addr 172.21.15.154:0/1398817756 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:21.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.432+0000 7f45bb7fe640 1 -- 172.21.15.154:0/1398817756 >> v1:172.21.15.190:6789/0 conn(0x7f45bc0ad830 legacy=0x7f45bc0c9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:21.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.433+0000 7f45bb7fe640 1 -- 172.21.15.154:0/1398817756 >> v1:172.21.15.154:6789/0 conn(0x7f45bc0cb400 legacy=0x7f45bc0aba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:21.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.432+0000 7f45baffd640 1 -- 172.21.15.154:0/1398817756 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1118326111 0 0) 0x7f45bc0712f0 con 0x7f45bc0cb400 2023-09-23T03:17:21.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.433+0000 7f45bb7fe640 1 -- 172.21.15.154:0/1398817756 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45bc0718b0 con 0x7f45bc070c60 2023-09-23T03:17:21.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.434+0000 7f45bb7fe640 1 --2- 172.21.15.154:0/1398817756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45bc070c60 0x7f45bc0caec0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f45b0009870 tx=0x7f45b002ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=671ce17815bb46b3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:21.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.434+0000 7f45baffd640 1 -- 172.21.15.154:0/1398817756 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b002f850 con 0x7f45bc070c60 2023-09-23T03:17:21.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.434+0000 7f45baffd640 1 -- 172.21.15.154:0/1398817756 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b002f9b0 con 0x7f45bc070c60 2023-09-23T03:17:21.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.434+0000 7f45baffd640 1 -- 172.21.15.154:0/1398817756 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b002fd40 con 0x7f45bc070c60 2023-09-23T03:17:21.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.434+0000 7f45c2887640 1 -- 172.21.15.154:0/1398817756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45bc070c60 msgr2=0x7f45bc0caec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:21.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.434+0000 7f45c2887640 1 --2- 172.21.15.154:0/1398817756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45bc070c60 0x7f45bc0caec0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f45b0009870 tx=0x7f45b002ba30 comp rx=0 tx=0).stop 2023-09-23T03:17:21.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.434+0000 7f45c2887640 1 -- 172.21.15.154:0/1398817756 shutdown_connections 2023-09-23T03:17:21.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.434+0000 7f45c2887640 1 --2- 172.21.15.154:0/1398817756 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45bc070c60 0x7f45bc0caec0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:21.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.434+0000 7f45c2887640 1 -- 172.21.15.154:0/1398817756 >> 172.21.15.154:0/1398817756 conn(0x7f45bc06c010 msgr2=0x7f45bc06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:21.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.435+0000 7f45c2887640 1 -- 172.21.15.154:0/1398817756 shutdown_connections 2023-09-23T03:17:21.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.435+0000 7f45c2887640 1 -- 172.21.15.154:0/1398817756 wait complete. 2023-09-23T03:17:21.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.436+0000 7f45c2887640 1 Processor -- start 2023-09-23T03:17:21.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.436+0000 7f45c2887640 1 -- start start 2023-09-23T03:17:21.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.437+0000 7f45c2887640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45bc070c60 0x7f45bc15d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:21.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.437+0000 7f45c2887640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45bc0ad830 0x7f45bc151880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:21.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.437+0000 7f45c2887640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45bc0cb400 0x7f45bc15b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:21.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.437+0000 7f45c2887640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc0c68c0 con 0x7f45bc0cb400 2023-09-23T03:17:21.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.437+0000 7f45c2887640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc15e090 con 0x7f45bc070c60 2023-09-23T03:17:21.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.437+0000 7f45c2887640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc15e370 con 0x7f45bc0ad830 2023-09-23T03:17:21.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.438+0000 7f45c0e24640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45bc0cb400 0x7f45bc15b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:21.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.438+0000 7f45c0e24640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45bc0cb400 0x7f45bc15b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46600/0 (socket says 172.21.15.154:46600) 2023-09-23T03:17:21.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.438+0000 7f45c0e24640 1 -- 172.21.15.154:0/1938865988 learned_addr learned my addr 172.21.15.154:0/1938865988 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:21.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.438+0000 7f45c0e24640 1 -- 172.21.15.154:0/1938865988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45bc0ad830 msgr2=0x7f45bc151880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:21.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.438+0000 7f45bb7fe640 1 --2- 172.21.15.154:0/1938865988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45bc0ad830 0x7f45bc151880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:21.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.438+0000 7f45c0e24640 1 --2- 172.21.15.154:0/1938865988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45bc0ad830 0x7f45bc151880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:21.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.438+0000 7f45c0e24640 1 -- 172.21.15.154:0/1938865988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45bc070c60 msgr2=0x7f45bc15d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:21.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.438+0000 7f45c0e24640 1 --2- 172.21.15.154:0/1938865988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45bc070c60 0x7f45bc15d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:21.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.438+0000 7f45c0e24640 1 -- 172.21.15.154:0/1938865988 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b00094d0 con 0x7f45bc0cb400 2023-09-23T03:17:21.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.438+0000 7f45c0e24640 1 --2- 172.21.15.154:0/1938865988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45bc0cb400 0x7f45bc15b950 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f45b400c7a0 tx=0x7f45b400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:21.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.439+0000 7f45b8ff9640 1 -- 172.21.15.154:0/1938865988 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b400a640 con 0x7f45bc0cb400 2023-09-23T03:17:21.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.439+0000 7f45b8ff9640 1 -- 172.21.15.154:0/1938865988 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b400a7a0 con 0x7f45bc0cb400 2023-09-23T03:17:21.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.439+0000 7f45b8ff9640 1 -- 172.21.15.154:0/1938865988 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b400abd0 con 0x7f45bc0cb400 2023-09-23T03:17:21.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.439+0000 7f45c2887640 1 -- 172.21.15.154:0/1938865988 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45bc15e650 con 0x7f45bc0cb400 2023-09-23T03:17:21.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.439+0000 7f45c2887640 1 -- 172.21.15.154:0/1938865988 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45bc15dcd0 con 0x7f45bc0cb400 2023-09-23T03:17:21.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.440+0000 7f45c2887640 1 -- 172.21.15.154:0/1938865988 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45bc15de60 con 0x7f45bc0cb400 2023-09-23T03:17:21.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.441+0000 7f45b8ff9640 1 -- 172.21.15.154:0/1938865988 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f45b400ad30 con 0x7f45bc0cb400 2023-09-23T03:17:21.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.442+0000 7f45b8ff9640 1 --2- 172.21.15.154:0/1938865988 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f45a806af30 0x7f45a806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:21.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.442+0000 7f45b8ff9640 1 -- 172.21.15.154:0/1938865988 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6401+0+0 (secure 0 0 0) 0x7f45b4088fd0 con 0x7f45bc0cb400 2023-09-23T03:17:21.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.442+0000 7f45bbfff640 1 --2- 172.21.15.154:0/1938865988 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f45a806af30 0x7f45a806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:21.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.443+0000 7f45bbfff640 1 --2- 172.21.15.154:0/1938865988 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f45a806af30 0x7f45a806d3f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f45ac000f60 tx=0x7f45ac0079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:21.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.447+0000 7f45b8ff9640 1 -- 172.21.15.154:0/1938865988 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f45b4053d40 con 0x7f45bc0cb400 2023-09-23T03:17:21.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:21.677+0000 7f45c2887640 1 -- 172.21.15.154:0/1938865988 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_extra"} v 0) v1 -- 0x7f45bc04e920 con 0x7f45bc0cb400 2023-09-23T03:17:22.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.221+0000 7f45b8ff9640 1 -- 172.21.15.154:0/1938865988 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_extra"}]=0 added data pool 7 to fsmap v19) v1 ==== 140+0+0 (secure 0 0 0) 0x7f45b4018070 con 0x7f45bc0cb400 2023-09-23T03:17:22.226 INFO:teuthology.orchestra.run.smithi154.stderr:added data pool 7 to fsmap 2023-09-23T03:17:22.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.227+0000 7f45c2887640 1 -- 172.21.15.154:0/1938865988 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f45a806af30 msgr2=0x7f45a806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:22.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.227+0000 7f45c2887640 1 --2- 172.21.15.154:0/1938865988 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f45a806af30 0x7f45a806d3f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f45ac000f60 tx=0x7f45ac0079b0 comp rx=0 tx=0).stop 2023-09-23T03:17:22.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.227+0000 7f45c2887640 1 -- 172.21.15.154:0/1938865988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45bc0cb400 msgr2=0x7f45bc15b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:22.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.227+0000 7f45c2887640 1 --2- 172.21.15.154:0/1938865988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45bc0cb400 0x7f45bc15b950 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f45b400c7a0 tx=0x7f45b400cc60 comp rx=0 tx=0).stop 2023-09-23T03:17:22.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.229+0000 7f45c2887640 1 -- 172.21.15.154:0/1938865988 shutdown_connections 2023-09-23T03:17:22.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.229+0000 7f45c2887640 1 --2- 172.21.15.154:0/1938865988 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f45a806af30 0x7f45a806d3f0 secure :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f45ac000f60 tx=0x7f45ac0079b0 comp rx=0 tx=0).stop 2023-09-23T03:17:22.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.229+0000 7f45c2887640 1 --2- 172.21.15.154:0/1938865988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45bc0cb400 0x7f45bc15b950 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:22.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.229+0000 7f45c2887640 1 --2- 172.21.15.154:0/1938865988 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45bc0ad830 0x7f45bc151880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:22.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.229+0000 7f45c2887640 1 --2- 172.21.15.154:0/1938865988 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45bc070c60 0x7f45bc15d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:22.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.229+0000 7f45c2887640 1 -- 172.21.15.154:0/1938865988 >> 172.21.15.154:0/1938865988 conn(0x7f45bc06c010 msgr2=0x7f45bc0af590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:22.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.234+0000 7f45c2887640 1 -- 172.21.15.154:0/1938865988 shutdown_connections 2023-09-23T03:17:22.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.236+0000 7f45c2887640 1 -- 172.21.15.154:0/1938865988 wait complete. 2023-09-23T03:17:22.257 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:17:22.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.447+0000 7fb94eac3640 1 Processor -- start 2023-09-23T03:17:22.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.447+0000 7fb94eac3640 1 -- start start 2023-09-23T03:17:22.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.449+0000 7fb94eac3640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb9480cf8e0 0x7fb94815c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:22.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.449+0000 7fb94eac3640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb94806be10 con 0x7fb9480cf500 2023-09-23T03:17:22.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.449+0000 7fb94eac3640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9480d17f0 con 0x7fb94815ca30 2023-09-23T03:17:22.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.449+0000 7fb94eac3640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9480d0470 con 0x7fb9480cf8e0 2023-09-23T03:17:22.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.449+0000 7fb94d2c0640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb9480cf8e0 0x7fb94815c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:22.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.449+0000 7fb94dac1640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fb9480cf500 0x7fb9480ce050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40768/0 (socket says 172.21.15.154:40768) 2023-09-23T03:17:22.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.449+0000 7fb94dac1640 1 -- 172.21.15.154:0/3750075588 learned_addr learned my addr 172.21.15.154:0/3750075588 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:17:22.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.450+0000 7fb94d2c0640 1 -- 172.21.15.154:0/3750075588 >> v1:172.21.15.190:6789/0 conn(0x7fb94815ca30 legacy=0x7fb9480b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:22.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.450+0000 7fb94d2c0640 1 -- 172.21.15.154:0/3750075588 >> v1:172.21.15.154:6789/0 conn(0x7fb9480cf500 legacy=0x7fb9480ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:22.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.450+0000 7fb94d2c0640 1 -- 172.21.15.154:0/3750075588 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9480d0750 con 0x7fb9480cf8e0 2023-09-23T03:17:22.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.451+0000 7fb94d2c0640 1 --2- 172.21.15.154:0/3750075588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9480cf8e0 0x7fb94815c4f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb9380097d0 tx=0x7fb93802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9870cf42caad9a47 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:22.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.451+0000 7fb94cabf640 1 -- 172.21.15.154:0/3750075588 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb938031780 con 0x7fb9480cf8e0 2023-09-23T03:17:22.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.451+0000 7fb94cabf640 1 -- 172.21.15.154:0/3750075588 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9380318e0 con 0x7fb9480cf8e0 2023-09-23T03:17:22.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.451+0000 7fb94cabf640 1 -- 172.21.15.154:0/3750075588 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb938031c70 con 0x7fb9480cf8e0 2023-09-23T03:17:22.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.451+0000 7fb94eac3640 1 -- 172.21.15.154:0/3750075588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9480cf8e0 msgr2=0x7fb94815c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:22.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.451+0000 7fb94eac3640 1 --2- 172.21.15.154:0/3750075588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9480cf8e0 0x7fb94815c4f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fb9380097d0 tx=0x7fb93802b820 comp rx=0 tx=0).stop 2023-09-23T03:17:22.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.452+0000 7fb94eac3640 1 -- 172.21.15.154:0/3750075588 shutdown_connections 2023-09-23T03:17:22.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.452+0000 7fb94eac3640 1 --2- 172.21.15.154:0/3750075588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9480cf8e0 0x7fb94815c4f0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:22.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.452+0000 7fb94eac3640 1 -- 172.21.15.154:0/3750075588 >> 172.21.15.154:0/3750075588 conn(0x7fb94809bbb0 msgr2=0x7fb9480d28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:22.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.452+0000 7fb94eac3640 1 -- 172.21.15.154:0/3750075588 shutdown_connections 2023-09-23T03:17:22.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.452+0000 7fb94eac3640 1 -- 172.21.15.154:0/3750075588 wait complete. 2023-09-23T03:17:22.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.453+0000 7fb94eac3640 1 Processor -- start 2023-09-23T03:17:22.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.454+0000 7fb94eac3640 1 -- start start 2023-09-23T03:17:22.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.454+0000 7fb94eac3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9480cf500 0x7fb9481525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:22.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.454+0000 7fb94eac3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9480cf8e0 0x7fb948152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:22.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.454+0000 7fb94eac3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb94815ca30 0x7fb9480c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:22.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.454+0000 7fb94eac3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9481536f0 con 0x7fb9480cf500 2023-09-23T03:17:22.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.454+0000 7fb94eac3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9481539d0 con 0x7fb9480cf8e0 2023-09-23T03:17:22.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.454+0000 7fb94eac3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb948153cb0 con 0x7fb94815ca30 2023-09-23T03:17:22.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.455+0000 7fb94dac1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9480cf500 0x7fb9481525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:22.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.455+0000 7fb94dac1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9480cf500 0x7fb9481525a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46624/0 (socket says 172.21.15.154:46624) 2023-09-23T03:17:22.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.455+0000 7fb94dac1640 1 -- 172.21.15.154:0/1248332872 learned_addr learned my addr 172.21.15.154:0/1248332872 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:22.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.455+0000 7fb94d2c0640 1 --2- 172.21.15.154:0/1248332872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9480cf8e0 0x7fb948152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:22.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.455+0000 7fb94dac1640 1 -- 172.21.15.154:0/1248332872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb94815ca30 msgr2=0x7fb9480c88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:22.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.455+0000 7fb94dac1640 1 --2- 172.21.15.154:0/1248332872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb94815ca30 0x7fb9480c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:22.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.455+0000 7fb94dac1640 1 -- 172.21.15.154:0/1248332872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9480cf8e0 msgr2=0x7fb948152df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:22.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.455+0000 7fb94dac1640 1 --2- 172.21.15.154:0/1248332872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9480cf8e0 0x7fb948152df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:22.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.455+0000 7fb94dac1640 1 -- 172.21.15.154:0/1248332872 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb938009400 con 0x7fb9480cf500 2023-09-23T03:17:22.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.455+0000 7fb94d2c0640 1 --2- 172.21.15.154:0/1248332872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9480cf8e0 0x7fb948152df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:22.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.455+0000 7fb94dac1640 1 --2- 172.21.15.154:0/1248332872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9480cf500 0x7fb9481525a0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb94400c7a0 tx=0x7fb94400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:22.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.456+0000 7fb93e7fc640 1 -- 172.21.15.154:0/1248332872 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94400a640 con 0x7fb9480cf500 2023-09-23T03:17:22.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.456+0000 7fb93e7fc640 1 -- 172.21.15.154:0/1248332872 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb94400a7a0 con 0x7fb9480cf500 2023-09-23T03:17:22.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.456+0000 7fb93e7fc640 1 -- 172.21.15.154:0/1248332872 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94400ab30 con 0x7fb9480cf500 2023-09-23T03:17:22.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.456+0000 7fb94eac3640 1 -- 172.21.15.154:0/1248332872 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb948153f90 con 0x7fb9480cf500 2023-09-23T03:17:22.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.456+0000 7fb94eac3640 1 -- 172.21.15.154:0/1248332872 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb948152ae0 con 0x7fb9480cf500 2023-09-23T03:17:22.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.457+0000 7fb94eac3640 1 -- 172.21.15.154:0/1248332872 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb94804e920 con 0x7fb9480cf500 2023-09-23T03:17:22.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.458+0000 7fb93e7fc640 1 -- 172.21.15.154:0/1248332872 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fb94400ac90 con 0x7fb9480cf500 2023-09-23T03:17:22.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.459+0000 7fb93e7fc640 1 --2- 172.21.15.154:0/1248332872 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb92c06af30 0x7fb92c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:22.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.459+0000 7fb93e7fc640 1 -- 172.21.15.154:0/1248332872 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6433+0+0 (secure 0 0 0) 0x7fb9440892e0 con 0x7fb9480cf500 2023-09-23T03:17:22.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.459+0000 7fb94d2c0640 1 --2- 172.21.15.154:0/1248332872 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb92c06af30 0x7fb92c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:22.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.460+0000 7fb94d2c0640 1 --2- 172.21.15.154:0/1248332872 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb92c06af30 0x7fb92c06d3f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb938036000 tx=0x7fb93803d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:22.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.464+0000 7fb93e7fc640 1 -- 172.21.15.154:0/1248332872 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fb944054030 con 0x7fb9480cf500 2023-09-23T03:17:22.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.687+0000 7fb94eac3640 1 -- 172.21.15.154:0/1248332872 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9480d0000 con 0x7fb9480cf500 2023-09-23T03:17:22.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.689+0000 7fb93e7fc640 1 -- 172.21.15.154:0/1248332872 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4924 (secure 0 0 0) 0x7fb944057650 con 0x7fb9480cf500 2023-09-23T03:17:22.689 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:22.689 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4760,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/4034412019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4034412019},{"type":"v1","addr":"172.21.15.154:6836","nonce":4034412019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3736308401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3736308401},{"type":"v1","addr":"172.21.15.190:6835","nonce":3736308401}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4999,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/617287200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":617287200},{"type":"v1","addr":"172.21.15.154:6837","nonce":617287200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:17:11.088637+0000","modified":"2023-09-23T03:17:21.679231+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5002},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5002":{"gid":5002,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.190:6834/2131436414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2131436414},{"type":"v1","addr":"172.21.15.190:6834","nonce":2131436414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-09-23T03:17:22.690 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 19 2023-09-23T03:17:22.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.691+0000 7fb94eac3640 1 -- 172.21.15.154:0/1248332872 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb92c06af30 msgr2=0x7fb92c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:22.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.691+0000 7fb94eac3640 1 --2- 172.21.15.154:0/1248332872 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb92c06af30 0x7fb92c06d3f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb938036000 tx=0x7fb93803d040 comp rx=0 tx=0).stop 2023-09-23T03:17:22.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.691+0000 7fb94eac3640 1 -- 172.21.15.154:0/1248332872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9480cf500 msgr2=0x7fb9481525a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:22.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.691+0000 7fb94eac3640 1 --2- 172.21.15.154:0/1248332872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9480cf500 0x7fb9481525a0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb94400c7a0 tx=0x7fb94400cc60 comp rx=0 tx=0).stop 2023-09-23T03:17:22.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.691+0000 7fb94eac3640 1 -- 172.21.15.154:0/1248332872 shutdown_connections 2023-09-23T03:17:22.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.691+0000 7fb94eac3640 1 --2- 172.21.15.154:0/1248332872 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb92c06af30 0x7fb92c06d3f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:22.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.691+0000 7fb94eac3640 1 --2- 172.21.15.154:0/1248332872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb94815ca30 0x7fb9480c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:22.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.691+0000 7fb94eac3640 1 --2- 172.21.15.154:0/1248332872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9480cf8e0 0x7fb948152df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:22.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.691+0000 7fb94eac3640 1 --2- 172.21.15.154:0/1248332872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9480cf500 0x7fb9481525a0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:22.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.691+0000 7fb94eac3640 1 -- 172.21.15.154:0/1248332872 >> 172.21.15.154:0/1248332872 conn(0x7fb94809bbb0 msgr2=0x7fb94815e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:22.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.691+0000 7fb94eac3640 1 -- 172.21.15.154:0/1248332872 shutdown_connections 2023-09-23T03:17:22.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.691+0000 7fb94eac3640 1 -- 172.21.15.154:0/1248332872 wait complete. 2023-09-23T03:17:22.711 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:17:22.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.927+0000 7f39b2667640 1 Processor -- start 2023-09-23T03:17:22.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.927+0000 7f39b2667640 1 -- start start 2023-09-23T03:17:22.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.929+0000 7f39b2667640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f39ac160fd0 0x7f39ac0c9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:22.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.929+0000 7f39b2667640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f39ac0c99d0 0x7f39ac0cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:22.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.929+0000 7f39b2667640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39ac0cc910 con 0x7f39ac15c840 2023-09-23T03:17:22.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.929+0000 7f39b2667640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39ac15cc20 con 0x7f39ac160fd0 2023-09-23T03:17:22.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.929+0000 7f39b2667640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39ac0cc6f0 con 0x7f39ac0c99d0 2023-09-23T03:17:22.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.929+0000 7f39b0e64640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f39ac160fd0 0x7f39ac0c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:22.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.929+0000 7f39b0e64640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f39ac160fd0 0x7f39ac0c9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46626/0 (socket says 172.21.15.154:46626) 2023-09-23T03:17:22.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.929+0000 7f39b0e64640 1 -- 172.21.15.154:0/3873349128 learned_addr learned my addr 172.21.15.154:0/3873349128 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:22.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.930+0000 7f39b1e66640 1 --2- 172.21.15.154:0/3873349128 >> v2:172.21.15.190:3300/0 conn(0x7f39ac0c99d0 0x7f39ac0cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:22.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.930+0000 7f39b0e64640 1 -- 172.21.15.154:0/3873349128 >> v2:172.21.15.190:3300/0 conn(0x7f39ac0c99d0 msgr2=0x7f39ac0cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:22.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.930+0000 7f39b0e64640 1 --2- 172.21.15.154:0/3873349128 >> v2:172.21.15.190:3300/0 conn(0x7f39ac0c99d0 0x7f39ac0cacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:22.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.930+0000 7f39b0e64640 1 -- 172.21.15.154:0/3873349128 >> v1:172.21.15.190:6789/0 conn(0x7f39ac15c840 legacy=0x7f39ac0cbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:22.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.930+0000 7f39b0e64640 1 -- 172.21.15.154:0/3873349128 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39ac0c89b0 con 0x7f39ac160fd0 2023-09-23T03:17:22.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.931+0000 7f39b0e64640 1 --2- 172.21.15.154:0/3873349128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39ac160fd0 0x7f39ac0c9490 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f39ac06cfe0 tx=0x7f399c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5818f117ba9659e7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:22.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.931+0000 7f39a3fff640 1 -- 172.21.15.154:0/3873349128 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399c031870 con 0x7f39ac160fd0 2023-09-23T03:17:22.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.931+0000 7f39a3fff640 1 -- 172.21.15.154:0/3873349128 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399c0319d0 con 0x7f39ac160fd0 2023-09-23T03:17:22.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.931+0000 7f39a3fff640 1 -- 172.21.15.154:0/3873349128 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399c031d60 con 0x7f39ac160fd0 2023-09-23T03:17:22.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.932+0000 7f39b2667640 1 -- 172.21.15.154:0/3873349128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39ac160fd0 msgr2=0x7f39ac0c9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:22.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.932+0000 7f39b2667640 1 --2- 172.21.15.154:0/3873349128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39ac160fd0 0x7f39ac0c9490 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f39ac06cfe0 tx=0x7f399c02b910 comp rx=0 tx=0).stop 2023-09-23T03:17:22.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.932+0000 7f39b2667640 1 -- 172.21.15.154:0/3873349128 shutdown_connections 2023-09-23T03:17:22.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.932+0000 7f39b2667640 1 --2- 172.21.15.154:0/3873349128 >> v2:172.21.15.190:3300/0 conn(0x7f39ac0c99d0 0x7f39ac0cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:22.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.932+0000 7f39b2667640 1 --2- 172.21.15.154:0/3873349128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39ac160fd0 0x7f39ac0c9490 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:22.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.932+0000 7f39b2667640 1 -- 172.21.15.154:0/3873349128 >> 172.21.15.154:0/3873349128 conn(0x7f39ac06a390 msgr2=0x7f39ac0afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:22.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.932+0000 7f39b2667640 1 -- 172.21.15.154:0/3873349128 shutdown_connections 2023-09-23T03:17:22.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.932+0000 7f39b2667640 1 -- 172.21.15.154:0/3873349128 wait complete. 2023-09-23T03:17:22.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.934+0000 7f39b2667640 1 Processor -- start 2023-09-23T03:17:22.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.934+0000 7f39b2667640 1 -- start start 2023-09-23T03:17:22.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.935+0000 7f39b2667640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39ac0c99d0 0x7f39ac146e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:22.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.935+0000 7f39b2667640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39ac15c840 0x7f39ac0c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:22.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.935+0000 7f39b2667640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39ac160fd0 0x7f39ac1496a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:22.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.935+0000 7f39b2667640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac14a640 con 0x7f39ac15c840 2023-09-23T03:17:22.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.935+0000 7f39b2667640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac0c5910 con 0x7f39ac0c99d0 2023-09-23T03:17:22.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.935+0000 7f39b2667640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac147340 con 0x7f39ac160fd0 2023-09-23T03:17:22.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.935+0000 7f39b0e64640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39ac15c840 0x7f39ac0c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:22.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.935+0000 7f39b0e64640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39ac15c840 0x7f39ac0c5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46634/0 (socket says 172.21.15.154:46634) 2023-09-23T03:17:22.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.935+0000 7f39b0e64640 1 -- 172.21.15.154:0/3225973923 learned_addr learned my addr 172.21.15.154:0/3225973923 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:22.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.935+0000 7f39b1665640 1 --2- 172.21.15.154:0/3225973923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39ac0c99d0 0x7f39ac146e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:22.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.936+0000 7f39b0e64640 1 -- 172.21.15.154:0/3225973923 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39ac160fd0 msgr2=0x7f39ac1496a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:22.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.936+0000 7f39b1e66640 1 --2- 172.21.15.154:0/3225973923 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39ac160fd0 0x7f39ac1496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:22.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.936+0000 7f39b0e64640 1 --2- 172.21.15.154:0/3225973923 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39ac160fd0 0x7f39ac1496a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:22.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.936+0000 7f39b0e64640 1 -- 172.21.15.154:0/3225973923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39ac0c99d0 msgr2=0x7f39ac146e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:22.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.936+0000 7f39b0e64640 1 --2- 172.21.15.154:0/3225973923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39ac0c99d0 0x7f39ac146e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:22.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.936+0000 7f39b0e64640 1 -- 172.21.15.154:0/3225973923 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f399c0094d0 con 0x7f39ac15c840 2023-09-23T03:17:22.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.936+0000 7f39b0e64640 1 --2- 172.21.15.154:0/3225973923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39ac15c840 0x7f39ac0c5d30 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f399c02b950 tx=0x7f399c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:22.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.936+0000 7f39a1ffb640 1 -- 172.21.15.154:0/3225973923 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399c03ac60 con 0x7f39ac15c840 2023-09-23T03:17:22.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.936+0000 7f39a1ffb640 1 -- 172.21.15.154:0/3225973923 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399c03adc0 con 0x7f39ac15c840 2023-09-23T03:17:22.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.936+0000 7f39a1ffb640 1 -- 172.21.15.154:0/3225973923 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399c044630 con 0x7f39ac15c840 2023-09-23T03:17:22.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.936+0000 7f39b2667640 1 -- 172.21.15.154:0/3225973923 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39ac14a080 con 0x7f39ac15c840 2023-09-23T03:17:22.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.936+0000 7f39b2667640 1 -- 172.21.15.154:0/3225973923 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39ac0c5530 con 0x7f39ac15c840 2023-09-23T03:17:22.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.937+0000 7f39b2667640 1 -- 172.21.15.154:0/3225973923 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39ac0736b0 con 0x7f39ac15c840 2023-09-23T03:17:22.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.938+0000 7f39a1ffb640 1 -- 172.21.15.154:0/3225973923 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f399c04d450 con 0x7f39ac15c840 2023-09-23T03:17:22.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.939+0000 7f39a1ffb640 1 --2- 172.21.15.154:0/3225973923 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f399006af30 0x7f399006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:22.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.939+0000 7f39a1ffb640 1 -- 172.21.15.154:0/3225973923 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f399c02d040 con 0x7f39ac15c840 2023-09-23T03:17:22.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.940+0000 7f39b1665640 1 --2- 172.21.15.154:0/3225973923 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f399006af30 0x7f399006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:22.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.940+0000 7f39b1665640 1 --2- 172.21.15.154:0/3225973923 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f399006af30 0x7f399006d3f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f39a8004ea0 tx=0x7f39a8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:22.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:22.943+0000 7f39a1ffb640 1 -- 172.21.15.154:0/3225973923 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f399c076d50 con 0x7f39ac15c840 2023-09-23T03:17:23.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:23.135+0000 7f39b2667640 1 -- 172.21.15.154:0/3225973923 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39ac0c5370 con 0x7f39ac15c840 2023-09-23T03:17:23.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:23.138+0000 7f39a1ffb640 1 -- 172.21.15.154:0/3225973923 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+18611 (secure 0 0 0) 0x7f399c07a370 con 0x7f39ac15c840 2023-09-23T03:17:23.139 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:23.139 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":40,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:17:21.678709+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:17:08.933690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-09-23T03:17:10.141711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data_extra","create_time":"2023-09-23T03:17:21.021560+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:17:23.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:23.141+0000 7f39b2667640 1 -- 172.21.15.154:0/3225973923 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f399006af30 msgr2=0x7f399006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:23.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:23.141+0000 7f39b2667640 1 --2- 172.21.15.154:0/3225973923 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f399006af30 0x7f399006d3f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f39a8004ea0 tx=0x7f39a8009040 comp rx=0 tx=0).stop 2023-09-23T03:17:23.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:23.141+0000 7f39b2667640 1 -- 172.21.15.154:0/3225973923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39ac15c840 msgr2=0x7f39ac0c5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:23.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:23.141+0000 7f39b2667640 1 --2- 172.21.15.154:0/3225973923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39ac15c840 0x7f39ac0c5d30 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f399c02b950 tx=0x7f399c0046c0 comp rx=0 tx=0).stop 2023-09-23T03:17:23.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:23.141+0000 7f39b2667640 1 -- 172.21.15.154:0/3225973923 shutdown_connections 2023-09-23T03:17:23.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:23.141+0000 7f39b2667640 1 --2- 172.21.15.154:0/3225973923 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f399006af30 0x7f399006d3f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:23.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:23.141+0000 7f39b2667640 1 --2- 172.21.15.154:0/3225973923 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f39ac160fd0 0x7f39ac1496a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:23.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:23.141+0000 7f39b2667640 1 --2- 172.21.15.154:0/3225973923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f39ac15c840 0x7f39ac0c5d30 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:23.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:23.141+0000 7f39b2667640 1 --2- 172.21.15.154:0/3225973923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f39ac0c99d0 0x7f39ac146e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:23.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:23.141+0000 7f39b2667640 1 -- 172.21.15.154:0/3225973923 >> 172.21.15.154:0/3225973923 conn(0x7f39ac06a390 msgr2=0x7f39ac0cac00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:23.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:23.141+0000 7f39b2667640 1 -- 172.21.15.154:0/3225973923 shutdown_connections 2023-09-23T03:17:23.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:23.141+0000 7f39b2667640 1 -- 172.21.15.154:0/3225973923 wait complete. 2023-09-23T03:17:23.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:17:23.161 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v cephfs_data_extra .) 2023-09-23T03:17:23.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:17:23.223 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2023-09-23T03:17:23.284 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/0 2023-09-23T03:17:23.284 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:17:23.284 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:17:23.285 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/0" 2023-09-23T03:17:23.285 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'w'"'"') as f: 2023-09-23T03:17:23.285 DEBUG:teuthology.orchestra.run.smithi154:> for i in range(0, 6291456): 2023-09-23T03:17:23.285 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:17:23.285 DEBUG:teuthology.orchestra.run.smithi154:> f.write(chr(val)) 2023-09-23T03:17:23.285 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:17:27.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:17:27.055 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/1 2023-09-23T03:17:27.055 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:17:27.055 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:17:27.056 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/1" 2023-09-23T03:17:27.056 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'w'"'"') as f: 2023-09-23T03:17:27.056 DEBUG:teuthology.orchestra.run.smithi154:> for i in range(0, 6291456): 2023-09-23T03:17:27.056 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:17:27.056 DEBUG:teuthology.orchestra.run.smithi154:> f.write(chr(val)) 2023-09-23T03:17:27.056 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:17:30.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:17:30.839 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/2 2023-09-23T03:17:30.839 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:17:30.839 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:17:30.839 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/2" 2023-09-23T03:17:30.839 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'w'"'"') as f: 2023-09-23T03:17:30.839 DEBUG:teuthology.orchestra.run.smithi154:> for i in range(0, 6291456): 2023-09-23T03:17:30.839 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:17:30.839 DEBUG:teuthology.orchestra.run.smithi154:> f.write(chr(val)) 2023-09-23T03:17:30.840 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:17:34.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:17:34.595 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/3 2023-09-23T03:17:34.595 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:17:34.595 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:17:34.595 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/3" 2023-09-23T03:17:34.595 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'w'"'"') as f: 2023-09-23T03:17:34.595 DEBUG:teuthology.orchestra.run.smithi154:> for i in range(0, 6291456): 2023-09-23T03:17:34.596 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:17:34.596 DEBUG:teuthology.orchestra.run.smithi154:> f.write(chr(val)) 2023-09-23T03:17:34.596 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:17:38.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:17:38.431 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/4 2023-09-23T03:17:38.431 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:17:38.431 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:17:38.431 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/4" 2023-09-23T03:17:38.431 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'w'"'"') as f: 2023-09-23T03:17:38.432 DEBUG:teuthology.orchestra.run.smithi154:> for i in range(0, 6291456): 2023-09-23T03:17:38.432 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:17:38.432 DEBUG:teuthology.orchestra.run.smithi154:> f.write(chr(val)) 2023-09-23T03:17:38.432 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:17:41.260 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:17:41.269 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:17:42.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:17:42.140 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:17:42.140 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:17:42.192 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:17:42.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.396+0000 7f132dfc9640 1 Processor -- start 2023-09-23T03:17:42.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.397+0000 7f132dfc9640 1 -- start start 2023-09-23T03:17:42.400 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.398+0000 7f132dfc9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f132815cac0 0x7f132806b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:42.400 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.398+0000 7f132dfc9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f13280cdd80 0x7f13280cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:42.400 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.398+0000 7f132dfc9640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13280ce990 con 0x7f13280cf2a0 2023-09-23T03:17:42.400 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.398+0000 7f132dfc9640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13280ce1d0 con 0x7f132815cac0 2023-09-23T03:17:42.400 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.398+0000 7f132dfc9640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13280cfc50 con 0x7f13280cdd80 2023-09-23T03:17:42.400 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.399+0000 7f13277fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f132815cac0 0x7f132806b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:42.400 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.399+0000 7f13277fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f132815cac0 0x7f132806b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56552/0 (socket says 172.21.15.154:56552) 2023-09-23T03:17:42.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.399+0000 7f13277fe640 1 -- 172.21.15.154:0/2004743631 learned_addr learned my addr 172.21.15.154:0/2004743631 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:42.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.399+0000 7f1327fff640 1 --2- 172.21.15.154:0/2004743631 >> v2:172.21.15.190:3300/0 conn(0x7f13280cdd80 0x7f13280cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:42.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.399+0000 7f13277fe640 1 -- 172.21.15.154:0/2004743631 >> v2:172.21.15.190:3300/0 conn(0x7f13280cdd80 msgr2=0x7f13280cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:42.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.399+0000 7f13277fe640 1 --2- 172.21.15.154:0/2004743631 >> v2:172.21.15.190:3300/0 conn(0x7f13280cdd80 0x7f13280cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:42.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.399+0000 7f13277fe640 1 -- 172.21.15.154:0/2004743631 >> v1:172.21.15.190:6789/0 conn(0x7f13280cf2a0 legacy=0x7f13280cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:42.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.399+0000 7f13277fe640 1 -- 172.21.15.154:0/2004743631 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13280ce3d0 con 0x7f132815cac0 2023-09-23T03:17:42.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.401+0000 7f13277fe640 1 --2- 172.21.15.154:0/2004743631 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f132815cac0 0x7f132806b280 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f13280c6190 tx=0x7f131802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c71e6a941f62acc server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:42.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.401+0000 7f1326ffd640 1 -- 172.21.15.154:0/2004743631 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1318004230 con 0x7f132815cac0 2023-09-23T03:17:42.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.401+0000 7f1326ffd640 1 -- 172.21.15.154:0/2004743631 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1318004390 con 0x7f132815cac0 2023-09-23T03:17:42.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.401+0000 7f1326ffd640 1 -- 172.21.15.154:0/2004743631 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1318003810 con 0x7f132815cac0 2023-09-23T03:17:42.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.402+0000 7f132dfc9640 1 -- 172.21.15.154:0/2004743631 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f132815cac0 msgr2=0x7f132806b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:42.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.402+0000 7f132dfc9640 1 --2- 172.21.15.154:0/2004743631 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f132815cac0 0x7f132806b280 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f13280c6190 tx=0x7f131802b8f0 comp rx=0 tx=0).stop 2023-09-23T03:17:42.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.402+0000 7f132dfc9640 1 -- 172.21.15.154:0/2004743631 shutdown_connections 2023-09-23T03:17:42.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.402+0000 7f132dfc9640 1 --2- 172.21.15.154:0/2004743631 >> v2:172.21.15.190:3300/0 conn(0x7f13280cdd80 0x7f13280cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:42.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.402+0000 7f132dfc9640 1 --2- 172.21.15.154:0/2004743631 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f132815cac0 0x7f132806b280 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:42.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.402+0000 7f132dfc9640 1 -- 172.21.15.154:0/2004743631 >> 172.21.15.154:0/2004743631 conn(0x7f1328089fd0 msgr2=0x7f132809bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:42.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.402+0000 7f132dfc9640 1 -- 172.21.15.154:0/2004743631 shutdown_connections 2023-09-23T03:17:42.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.402+0000 7f132dfc9640 1 -- 172.21.15.154:0/2004743631 wait complete. 2023-09-23T03:17:42.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.404+0000 7f132dfc9640 1 Processor -- start 2023-09-23T03:17:42.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.404+0000 7f132dfc9640 1 -- start start 2023-09-23T03:17:42.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.405+0000 7f132dfc9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13280cdd80 0x7f13281efe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:42.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.405+0000 7f132dfc9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13280cf2a0 0x7f13281f3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:42.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.405+0000 7f132dfc9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f132815cac0 0x7f13280c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:42.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.405+0000 7f132dfc9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13280c8810 con 0x7f13280cf2a0 2023-09-23T03:17:42.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.405+0000 7f132dfc9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1328152680 con 0x7f13280cdd80 2023-09-23T03:17:42.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.405+0000 7f132dfc9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13280c9980 con 0x7f132815cac0 2023-09-23T03:17:42.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.405+0000 7f131effd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13280cf2a0 0x7f13281f3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:42.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.405+0000 7f131effd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13280cf2a0 0x7f13281f3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56564/0 (socket says 172.21.15.154:56564) 2023-09-23T03:17:42.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.405+0000 7f131effd640 1 -- 172.21.15.154:0/1951068910 learned_addr learned my addr 172.21.15.154:0/1951068910 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:42.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.405+0000 7f13277fe640 1 --2- 172.21.15.154:0/1951068910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13280cdd80 0x7f13281efe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:42.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.405+0000 7f131effd640 1 -- 172.21.15.154:0/1951068910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f132815cac0 msgr2=0x7f13280c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:42.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.405+0000 7f131effd640 1 --2- 172.21.15.154:0/1951068910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f132815cac0 0x7f13280c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:42.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.405+0000 7f131effd640 1 -- 172.21.15.154:0/1951068910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13280cdd80 msgr2=0x7f13281efe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:42.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.405+0000 7f131effd640 1 --2- 172.21.15.154:0/1951068910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13280cdd80 0x7f13281efe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:42.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.405+0000 7f131effd640 1 -- 172.21.15.154:0/1951068910 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1318009460 con 0x7f13280cf2a0 2023-09-23T03:17:42.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.405+0000 7f13277fe640 1 --2- 172.21.15.154:0/1951068910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13280cdd80 0x7f13281efe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:42.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.406+0000 7f131effd640 1 --2- 172.21.15.154:0/1951068910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13280cf2a0 0x7f13281f3330 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f131000c430 tx=0x7f131000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:42.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.406+0000 7f1324ff9640 1 -- 172.21.15.154:0/1951068910 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1310011790 con 0x7f13280cf2a0 2023-09-23T03:17:42.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.406+0000 7f1324ff9640 1 -- 172.21.15.154:0/1951068910 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13100118f0 con 0x7f13280cf2a0 2023-09-23T03:17:42.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.406+0000 7f132dfc9640 1 -- 172.21.15.154:0/1951068910 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13280c7330 con 0x7f13280cf2a0 2023-09-23T03:17:42.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.406+0000 7f1324ff9640 1 -- 172.21.15.154:0/1951068910 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1310011ce0 con 0x7f13280cf2a0 2023-09-23T03:17:42.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.406+0000 7f132dfc9640 1 -- 172.21.15.154:0/1951068910 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13280c8150 con 0x7f13280cf2a0 2023-09-23T03:17:42.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.407+0000 7f132dfc9640 1 -- 172.21.15.154:0/1951068910 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13280c8950 con 0x7f13280cf2a0 2023-09-23T03:17:42.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.408+0000 7f1324ff9640 1 -- 172.21.15.154:0/1951068910 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f131001c460 con 0x7f13280cf2a0 2023-09-23T03:17:42.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.410+0000 7f1324ff9640 1 --2- 172.21.15.154:0/1951068910 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f130806af30 0x7f130806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:42.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.410+0000 7f1324ff9640 1 -- 172.21.15.154:0/1951068910 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f1310089b40 con 0x7f13280cf2a0 2023-09-23T03:17:42.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.410+0000 7f13277fe640 1 --2- 172.21.15.154:0/1951068910 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f130806af30 0x7f130806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:42.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.411+0000 7f13277fe640 1 --2- 172.21.15.154:0/1951068910 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f130806af30 0x7f130806d3f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1318035000 tx=0x7f1318008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:42.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.415+0000 7f1324ff9640 1 -- 172.21.15.154:0/1951068910 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f1310054770 con 0x7f13280cf2a0 2023-09-23T03:17:42.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.603+0000 7f132dfc9640 1 -- 172.21.15.154:0/1951068910 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1328073eb0 con 0x7f13280cf2a0 2023-09-23T03:17:42.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.606+0000 7f1324ff9640 1 -- 172.21.15.154:0/1951068910 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+18598 (secure 0 0 0) 0x7f1310057d90 con 0x7f13280cf2a0 2023-09-23T03:17:42.608 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:42.608 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":41,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:17:23.221346+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:17:08.933690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-09-23T03:17:10.141711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data_extra","create_time":"2023-09-23T03:17:21.021560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:17:42.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.608+0000 7f132dfc9640 1 -- 172.21.15.154:0/1951068910 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f130806af30 msgr2=0x7f130806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:42.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.609+0000 7f132dfc9640 1 --2- 172.21.15.154:0/1951068910 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f130806af30 0x7f130806d3f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1318035000 tx=0x7f1318008030 comp rx=0 tx=0).stop 2023-09-23T03:17:42.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.609+0000 7f132dfc9640 1 -- 172.21.15.154:0/1951068910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13280cf2a0 msgr2=0x7f13281f3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:42.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.609+0000 7f132dfc9640 1 --2- 172.21.15.154:0/1951068910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13280cf2a0 0x7f13281f3330 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f131000c430 tx=0x7f131000c8f0 comp rx=0 tx=0).stop 2023-09-23T03:17:42.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.609+0000 7f132dfc9640 1 -- 172.21.15.154:0/1951068910 shutdown_connections 2023-09-23T03:17:42.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.609+0000 7f132dfc9640 1 --2- 172.21.15.154:0/1951068910 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f130806af30 0x7f130806d3f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:42.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.609+0000 7f132dfc9640 1 --2- 172.21.15.154:0/1951068910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f132815cac0 0x7f13280c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:42.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.609+0000 7f132dfc9640 1 --2- 172.21.15.154:0/1951068910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13280cf2a0 0x7f13281f3330 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:42.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.609+0000 7f132dfc9640 1 --2- 172.21.15.154:0/1951068910 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13280cdd80 0x7f13281efe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:42.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.609+0000 7f132dfc9640 1 -- 172.21.15.154:0/1951068910 >> 172.21.15.154:0/1951068910 conn(0x7f1328089fd0 msgr2=0x7f13280b16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:42.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.609+0000 7f132dfc9640 1 -- 172.21.15.154:0/1951068910 shutdown_connections 2023-09-23T03:17:42.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.609+0000 7f132dfc9640 1 -- 172.21.15.154:0/1951068910 wait complete. 2023-09-23T03:17:42.629 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:17:08.933690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:17:10.141711+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-09-23T03:17:21.021560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T03:17:42.629 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:17:42.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.839+0000 7fa16a1f4640 1 Processor -- start 2023-09-23T03:17:42.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.839+0000 7fa16a1f4640 1 -- start start 2023-09-23T03:17:42.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.840+0000 7fa16a1f4640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa16415c6c0 0x7fa164160410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:42.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.840+0000 7fa16a1f4640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa164160950 0x7fa1640cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:42.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.841+0000 7fa16a1f4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa16415caa0 0x7fa1640cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:42.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.841+0000 7fa16a1f4640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa164070290 con 0x7fa16415c6c0 2023-09-23T03:17:42.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.841+0000 7fa16a1f4640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa16406ff40 con 0x7fa164160950 2023-09-23T03:17:42.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.841+0000 7fa16a1f4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa164160df0 con 0x7fa16415caa0 2023-09-23T03:17:42.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.841+0000 7fa1691f2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa16415c6c0 0x7fa164160410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:42.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.841+0000 7fa1691f2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa16415c6c0 0x7fa164160410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56572/0 (socket says 172.21.15.154:56572) 2023-09-23T03:17:42.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.841+0000 7fa1691f2640 1 -- 172.21.15.154:0/3262351441 learned_addr learned my addr 172.21.15.154:0/3262351441 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:42.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.841+0000 7fa1689f1640 1 --2- 172.21.15.154:0/3262351441 >> v2:172.21.15.190:3300/0 conn(0x7fa164160950 0x7fa1640cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:42.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.841+0000 7fa1691f2640 1 -- 172.21.15.154:0/3262351441 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa16415caa0 msgr2=0x7fa1640cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:42.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.841+0000 7fa1699f3640 1 --2- 172.21.15.154:0/3262351441 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa16415caa0 0x7fa1640cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:42.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.841+0000 7fa1691f2640 1 --2- 172.21.15.154:0/3262351441 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa16415caa0 0x7fa1640cc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:42.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.841+0000 7fa1691f2640 1 -- 172.21.15.154:0/3262351441 >> v2:172.21.15.190:3300/0 conn(0x7fa164160950 msgr2=0x7fa1640cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:42.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.841+0000 7fa1691f2640 1 --2- 172.21.15.154:0/3262351441 >> v2:172.21.15.190:3300/0 conn(0x7fa164160950 0x7fa1640cbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:42.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.841+0000 7fa1691f2640 1 -- 172.21.15.154:0/3262351441 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1641610d0 con 0x7fa16415c6c0 2023-09-23T03:17:42.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.841+0000 7fa1689f1640 1 --2- 172.21.15.154:0/3262351441 >> v2:172.21.15.190:3300/0 conn(0x7fa164160950 0x7fa1640cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:42.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.841+0000 7fa1699f3640 1 --2- 172.21.15.154:0/3262351441 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa16415caa0 0x7fa1640cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:42.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.842+0000 7fa1691f2640 1 --2- 172.21.15.154:0/3262351441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa16415c6c0 0x7fa164160410 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa164160370 tx=0x7fa16002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f83f7b963f387f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:42.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.843+0000 7fa15bfff640 1 -- 172.21.15.154:0/3262351441 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa160031870 con 0x7fa16415c6c0 2023-09-23T03:17:42.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.843+0000 7fa15bfff640 1 -- 172.21.15.154:0/3262351441 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1600319d0 con 0x7fa16415c6c0 2023-09-23T03:17:42.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.843+0000 7fa15bfff640 1 -- 172.21.15.154:0/3262351441 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa160031d60 con 0x7fa16415c6c0 2023-09-23T03:17:42.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.843+0000 7fa16a1f4640 1 -- 172.21.15.154:0/3262351441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa16415c6c0 msgr2=0x7fa164160410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:42.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.843+0000 7fa16a1f4640 1 --2- 172.21.15.154:0/3262351441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa16415c6c0 0x7fa164160410 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa164160370 tx=0x7fa16002b910 comp rx=0 tx=0).stop 2023-09-23T03:17:42.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.843+0000 7fa16a1f4640 1 -- 172.21.15.154:0/3262351441 shutdown_connections 2023-09-23T03:17:42.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.843+0000 7fa16a1f4640 1 --2- 172.21.15.154:0/3262351441 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa16415caa0 0x7fa1640cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:42.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.843+0000 7fa16a1f4640 1 --2- 172.21.15.154:0/3262351441 >> v2:172.21.15.190:3300/0 conn(0x7fa164160950 0x7fa1640cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:42.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.843+0000 7fa16a1f4640 1 --2- 172.21.15.154:0/3262351441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa16415c6c0 0x7fa164160410 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:42.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.843+0000 7fa16a1f4640 1 -- 172.21.15.154:0/3262351441 >> 172.21.15.154:0/3262351441 conn(0x7fa16406a300 msgr2=0x7fa1640afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:42.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.844+0000 7fa16a1f4640 1 -- 172.21.15.154:0/3262351441 shutdown_connections 2023-09-23T03:17:42.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.844+0000 7fa16a1f4640 1 -- 172.21.15.154:0/3262351441 wait complete. 2023-09-23T03:17:42.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.846+0000 7fa16a1f4640 1 Processor -- start 2023-09-23T03:17:42.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.846+0000 7fa16a1f4640 1 -- start start 2023-09-23T03:17:42.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.847+0000 7fa16a1f4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa16415c6c0 0x7fa1641f51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:42.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.847+0000 7fa16a1f4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa16415caa0 0x7fa1641f3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:42.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.847+0000 7fa16a1f4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa164160950 0x7fa1640b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:42.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.847+0000 7fa16a1f4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16415ea30 con 0x7fa164160950 2023-09-23T03:17:42.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.847+0000 7fa16a1f4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1641f3160 con 0x7fa16415caa0 2023-09-23T03:17:42.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.847+0000 7fa16a1f4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa16415e090 con 0x7fa16415c6c0 2023-09-23T03:17:42.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.847+0000 7fa1699f3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa164160950 0x7fa1640b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:42.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.847+0000 7fa1699f3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa164160950 0x7fa1640b0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56582/0 (socket says 172.21.15.154:56582) 2023-09-23T03:17:42.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.847+0000 7fa1699f3640 1 -- 172.21.15.154:0/764839403 learned_addr learned my addr 172.21.15.154:0/764839403 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:42.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.848+0000 7fa1691f2640 1 --2- 172.21.15.154:0/764839403 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa16415c6c0 0x7fa1641f51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:42.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.848+0000 7fa1699f3640 1 -- 172.21.15.154:0/764839403 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa16415c6c0 msgr2=0x7fa1641f51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:42.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.848+0000 7fa1689f1640 1 --2- 172.21.15.154:0/764839403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa16415caa0 0x7fa1641f3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:42.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.848+0000 7fa1699f3640 1 --2- 172.21.15.154:0/764839403 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa16415c6c0 0x7fa1641f51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:42.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.848+0000 7fa1699f3640 1 -- 172.21.15.154:0/764839403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa16415caa0 msgr2=0x7fa1641f3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:42.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.848+0000 7fa1699f3640 1 --2- 172.21.15.154:0/764839403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa16415caa0 0x7fa1641f3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:42.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.848+0000 7fa1699f3640 1 -- 172.21.15.154:0/764839403 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1600094d0 con 0x7fa164160950 2023-09-23T03:17:42.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.848+0000 7fa1699f3640 1 --2- 172.21.15.154:0/764839403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa164160950 0x7fa1640b0f00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00cd80 tx=0x7fa15c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:42.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.848+0000 7fa159ffb640 1 -- 172.21.15.154:0/764839403 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c00d840 con 0x7fa164160950 2023-09-23T03:17:42.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.848+0000 7fa159ffb640 1 -- 172.21.15.154:0/764839403 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15c00d9a0 con 0x7fa164160950 2023-09-23T03:17:42.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.848+0000 7fa159ffb640 1 -- 172.21.15.154:0/764839403 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c00dd50 con 0x7fa164160950 2023-09-23T03:17:42.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.848+0000 7fa16a1f4640 1 -- 172.21.15.154:0/764839403 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa16415e370 con 0x7fa164160950 2023-09-23T03:17:42.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.849+0000 7fa16a1f4640 1 -- 172.21.15.154:0/764839403 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1641f3880 con 0x7fa164160950 2023-09-23T03:17:42.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.850+0000 7fa16a1f4640 1 -- 172.21.15.154:0/764839403 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1640736b0 con 0x7fa164160950 2023-09-23T03:17:42.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.851+0000 7fa159ffb640 1 -- 172.21.15.154:0/764839403 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fa15c01e3b0 con 0x7fa164160950 2023-09-23T03:17:42.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.851+0000 7fa159ffb640 1 --2- 172.21.15.154:0/764839403 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa14806af30 0x7fa14806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:42.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.852+0000 7fa159ffb640 1 -- 172.21.15.154:0/764839403 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6433+0+0 (secure 0 0 0) 0x7fa15c08a380 con 0x7fa164160950 2023-09-23T03:17:42.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.852+0000 7fa1691f2640 1 --2- 172.21.15.154:0/764839403 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa14806af30 0x7fa14806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:42.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.852+0000 7fa1691f2640 1 --2- 172.21.15.154:0/764839403 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa14806af30 0x7fa14806d3f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa160036000 tx=0x7fa16003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:42.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:42.856+0000 7fa159ffb640 1 -- 172.21.15.154:0/764839403 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fa15c055170 con 0x7fa164160950 2023-09-23T03:17:43.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.029+0000 7fa16a1f4640 1 -- 172.21.15.154:0/764839403 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa16415eb70 con 0x7fa164160950 2023-09-23T03:17:43.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.030+0000 7fa159ffb640 1 -- 172.21.15.154:0/764839403 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa16415eb70 con 0x7fa164160950 2023-09-23T03:17:43.032 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:17:43.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.033+0000 7fa16a1f4640 1 -- 172.21.15.154:0/764839403 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa14806af30 msgr2=0x7fa14806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:43.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.033+0000 7fa16a1f4640 1 --2- 172.21.15.154:0/764839403 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa14806af30 0x7fa14806d3f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa160036000 tx=0x7fa16003c040 comp rx=0 tx=0).stop 2023-09-23T03:17:43.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.033+0000 7fa16a1f4640 1 -- 172.21.15.154:0/764839403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa164160950 msgr2=0x7fa1640b0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:43.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.033+0000 7fa16a1f4640 1 --2- 172.21.15.154:0/764839403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa164160950 0x7fa1640b0f00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00cd80 tx=0x7fa15c00a710 comp rx=0 tx=0).stop 2023-09-23T03:17:43.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.033+0000 7fa16a1f4640 1 -- 172.21.15.154:0/764839403 shutdown_connections 2023-09-23T03:17:43.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.033+0000 7fa16a1f4640 1 --2- 172.21.15.154:0/764839403 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa14806af30 0x7fa14806d3f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:43.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.033+0000 7fa16a1f4640 1 --2- 172.21.15.154:0/764839403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa164160950 0x7fa1640b0f00 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:43.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.033+0000 7fa16a1f4640 1 --2- 172.21.15.154:0/764839403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa16415caa0 0x7fa1641f3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:43.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.033+0000 7fa16a1f4640 1 --2- 172.21.15.154:0/764839403 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa16415c6c0 0x7fa1641f51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:43.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.033+0000 7fa16a1f4640 1 -- 172.21.15.154:0/764839403 >> 172.21.15.154:0/764839403 conn(0x7fa16406a300 msgr2=0x7fa16415b670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:43.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.033+0000 7fa16a1f4640 1 -- 172.21.15.154:0/764839403 shutdown_connections 2023-09-23T03:17:43.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.033+0000 7fa16a1f4640 1 -- 172.21.15.154:0/764839403 wait complete. 2023-09-23T03:17:43.052 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:17:43.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.271+0000 7f9b9252c640 1 Processor -- start 2023-09-23T03:17:43.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.271+0000 7f9b9252c640 1 -- start start 2023-09-23T03:17:43.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.272+0000 7f9b9252c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9b8c0c8d70 0x7f9b8c15c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:43.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.272+0000 7f9b9252c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b8c0c9150 0x7f9b8c15cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:43.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.272+0000 7f9b9252c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9b8c071290 0x7f9b8c0ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:43.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.272+0000 7f9b9252c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b8c0c9d20 con 0x7f9b8c0c8d70 2023-09-23T03:17:43.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.272+0000 7f9b9252c640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b8c071850 con 0x7f9b8c071290 2023-09-23T03:17:43.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.272+0000 7f9b9252c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c071b30 con 0x7f9b8c0c9150 2023-09-23T03:17:43.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.273+0000 7f9b8bfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9b8c0c8d70 0x7f9b8c15c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:43.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.273+0000 7f9b8bfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9b8c0c8d70 0x7f9b8c15c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56596/0 (socket says 172.21.15.154:56596) 2023-09-23T03:17:43.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.273+0000 7f9b8bfff640 1 -- 172.21.15.154:0/80027556 learned_addr learned my addr 172.21.15.154:0/80027556 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:43.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.273+0000 7f9b8bfff640 1 -- 172.21.15.154:0/80027556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b8c0c9150 msgr2=0x7f9b8c15cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:43.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.273+0000 7f9b83fff640 1 --2- 172.21.15.154:0/80027556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b8c0c9150 0x7f9b8c15cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:43.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.273+0000 7f9b8bfff640 1 --2- 172.21.15.154:0/80027556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b8c0c9150 0x7f9b8c15cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:43.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.273+0000 7f9b8bfff640 1 -- 172.21.15.154:0/80027556 >> v2:172.21.15.190:3300/0 conn(0x7f9b8c071290 msgr2=0x7f9b8c0ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:43.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.273+0000 7f9b8bfff640 1 --2- 172.21.15.154:0/80027556 >> v2:172.21.15.190:3300/0 conn(0x7f9b8c071290 0x7f9b8c0ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:43.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.273+0000 7f9b8bfff640 1 -- 172.21.15.154:0/80027556 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8c071e10 con 0x7f9b8c0c8d70 2023-09-23T03:17:43.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.274+0000 7f9b8bfff640 1 --2- 172.21.15.154:0/80027556 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b8c0c8d70 0x7f9b8c15c6d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c0c95d0 tx=0x7f9b7c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37b71876cc99ae7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:43.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.275+0000 7f9b8b7fe640 1 -- 172.21.15.154:0/80027556 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c0317e0 con 0x7f9b8c0c8d70 2023-09-23T03:17:43.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.275+0000 7f9b8b7fe640 1 -- 172.21.15.154:0/80027556 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b7c031940 con 0x7f9b8c0c8d70 2023-09-23T03:17:43.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.275+0000 7f9b8b7fe640 1 -- 172.21.15.154:0/80027556 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c031cd0 con 0x7f9b8c0c8d70 2023-09-23T03:17:43.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.275+0000 7f9b9252c640 1 -- 172.21.15.154:0/80027556 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b8c0c8d70 msgr2=0x7f9b8c15c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:43.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.275+0000 7f9b9252c640 1 --2- 172.21.15.154:0/80027556 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b8c0c8d70 0x7f9b8c15c6d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c0c95d0 tx=0x7f9b7c02b880 comp rx=0 tx=0).stop 2023-09-23T03:17:43.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.275+0000 7f9b9252c640 1 -- 172.21.15.154:0/80027556 shutdown_connections 2023-09-23T03:17:43.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.275+0000 7f9b9252c640 1 --2- 172.21.15.154:0/80027556 >> v2:172.21.15.190:3300/0 conn(0x7f9b8c071290 0x7f9b8c0ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:43.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.275+0000 7f9b9252c640 1 --2- 172.21.15.154:0/80027556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b8c0c9150 0x7f9b8c15cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:43.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.275+0000 7f9b9252c640 1 --2- 172.21.15.154:0/80027556 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b8c0c8d70 0x7f9b8c15c6d0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:43.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.275+0000 7f9b9252c640 1 -- 172.21.15.154:0/80027556 >> 172.21.15.154:0/80027556 conn(0x7f9b8c06c010 msgr2=0x7f9b8c06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:43.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.275+0000 7f9b9252c640 1 -- 172.21.15.154:0/80027556 shutdown_connections 2023-09-23T03:17:43.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.275+0000 7f9b9252c640 1 -- 172.21.15.154:0/80027556 wait complete. 2023-09-23T03:17:43.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.277+0000 7f9b9252c640 1 Processor -- start 2023-09-23T03:17:43.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.277+0000 7f9b9252c640 1 -- start start 2023-09-23T03:17:43.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.278+0000 7f9b9252c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b8c071290 0x7f9b8c0caac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:43.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.278+0000 7f9b9252c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b8c0c8d70 0x7f9b8c15b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:43.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.278+0000 7f9b9252c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b8c0c9150 0x7f9b8c14fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:43.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.278+0000 7f9b9252c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c0c6130 con 0x7f9b8c0c9150 2023-09-23T03:17:43.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.278+0000 7f9b9252c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c0c6410 con 0x7f9b8c071290 2023-09-23T03:17:43.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.278+0000 7f9b9252c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c0c66f0 con 0x7f9b8c0c8d70 2023-09-23T03:17:43.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.278+0000 7f9b90ac9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b8c0c9150 0x7f9b8c14fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:43.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.278+0000 7f9b90ac9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b8c0c9150 0x7f9b8c14fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56600/0 (socket says 172.21.15.154:56600) 2023-09-23T03:17:43.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.278+0000 7f9b90ac9640 1 -- 172.21.15.154:0/983192011 learned_addr learned my addr 172.21.15.154:0/983192011 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:43.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.279+0000 7f9b90ac9640 1 -- 172.21.15.154:0/983192011 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b8c0c8d70 msgr2=0x7f9b8c15b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:43.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.279+0000 7f9b90ac9640 1 --2- 172.21.15.154:0/983192011 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b8c0c8d70 0x7f9b8c15b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:43.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.279+0000 7f9b90ac9640 1 -- 172.21.15.154:0/983192011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b8c071290 msgr2=0x7f9b8c0caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:43.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.279+0000 7f9b90ac9640 1 --2- 172.21.15.154:0/983192011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b8c071290 0x7f9b8c0caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:43.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.279+0000 7f9b90ac9640 1 -- 172.21.15.154:0/983192011 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b7c0094d0 con 0x7f9b8c0c9150 2023-09-23T03:17:43.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.279+0000 7f9b90ac9640 1 --2- 172.21.15.154:0/983192011 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b8c0c9150 0x7f9b8c14fe60 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9b8400b970 tx=0x7f9b8400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:43.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.279+0000 7f9b897fa640 1 -- 172.21.15.154:0/983192011 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8400cb40 con 0x7f9b8c0c9150 2023-09-23T03:17:43.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.279+0000 7f9b897fa640 1 -- 172.21.15.154:0/983192011 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b8400ccc0 con 0x7f9b8c0c9150 2023-09-23T03:17:43.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.279+0000 7f9b897fa640 1 -- 172.21.15.154:0/983192011 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b84014680 con 0x7f9b8c0c9150 2023-09-23T03:17:43.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.279+0000 7f9b9252c640 1 -- 172.21.15.154:0/983192011 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b8c0c69d0 con 0x7f9b8c0c9150 2023-09-23T03:17:43.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.280+0000 7f9b9252c640 1 -- 172.21.15.154:0/983192011 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b8c15be90 con 0x7f9b8c0c9150 2023-09-23T03:17:43.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.281+0000 7f9b9252c640 1 -- 172.21.15.154:0/983192011 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b8c0c6ef0 con 0x7f9b8c0c9150 2023-09-23T03:17:43.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.282+0000 7f9b897fa640 1 -- 172.21.15.154:0/983192011 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f9b8401d3b0 con 0x7f9b8c0c9150 2023-09-23T03:17:43.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.283+0000 7f9b897fa640 1 --2- 172.21.15.154:0/983192011 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9b6c06af30 0x7f9b6c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:43.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.283+0000 7f9b897fa640 1 -- 172.21.15.154:0/983192011 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f9b84089a20 con 0x7f9b8c0c9150 2023-09-23T03:17:43.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.283+0000 7f9b8bfff640 1 --2- 172.21.15.154:0/983192011 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9b6c06af30 0x7f9b6c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:43.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.284+0000 7f9b8bfff640 1 --2- 172.21.15.154:0/983192011 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9b6c06af30 0x7f9b6c06d3f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c036000 tx=0x7f9b7c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:43.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.287+0000 7f9b897fa640 1 -- 172.21.15.154:0/983192011 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f9b8c0c6ef0 con 0x7f9b8c0c9150 2023-09-23T03:17:43.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.476+0000 7f9b9252c640 1 -- 172.21.15.154:0/983192011 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9b8c0c6ef0 con 0x7f9b8c0c9150 2023-09-23T03:17:43.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.477+0000 7f9b897fa640 1 -- 172.21.15.154:0/983192011 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9b8c0c6ef0 con 0x7f9b8c0c9150 2023-09-23T03:17:43.479 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:17:43.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.480+0000 7f9b9252c640 1 -- 172.21.15.154:0/983192011 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9b6c06af30 msgr2=0x7f9b6c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:43.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.480+0000 7f9b9252c640 1 --2- 172.21.15.154:0/983192011 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9b6c06af30 0x7f9b6c06d3f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c036000 tx=0x7f9b7c030000 comp rx=0 tx=0).stop 2023-09-23T03:17:43.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.480+0000 7f9b9252c640 1 -- 172.21.15.154:0/983192011 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b8c0c9150 msgr2=0x7f9b8c14fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:43.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.480+0000 7f9b9252c640 1 --2- 172.21.15.154:0/983192011 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b8c0c9150 0x7f9b8c14fe60 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9b8400b970 tx=0x7f9b8400be30 comp rx=0 tx=0).stop 2023-09-23T03:17:43.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.480+0000 7f9b9252c640 1 -- 172.21.15.154:0/983192011 shutdown_connections 2023-09-23T03:17:43.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.480+0000 7f9b9252c640 1 --2- 172.21.15.154:0/983192011 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9b6c06af30 0x7f9b6c06d3f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:43.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.480+0000 7f9b9252c640 1 --2- 172.21.15.154:0/983192011 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b8c0c9150 0x7f9b8c14fe60 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:43.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.480+0000 7f9b9252c640 1 --2- 172.21.15.154:0/983192011 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b8c0c8d70 0x7f9b8c15b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:43.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.480+0000 7f9b9252c640 1 --2- 172.21.15.154:0/983192011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b8c071290 0x7f9b8c0caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:43.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.480+0000 7f9b9252c640 1 -- 172.21.15.154:0/983192011 >> 172.21.15.154:0/983192011 conn(0x7f9b8c06c010 msgr2=0x7f9b8c0ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:43.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.480+0000 7f9b9252c640 1 -- 172.21.15.154:0/983192011 shutdown_connections 2023-09-23T03:17:43.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.480+0000 7f9b9252c640 1 -- 172.21.15.154:0/983192011 wait complete. 2023-09-23T03:17:43.499 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:17:43.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.715+0000 7f09bdd85640 1 Processor -- start 2023-09-23T03:17:43.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.716+0000 7f09bdd85640 1 -- start start 2023-09-23T03:17:43.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.717+0000 7f09bdd85640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09b8071810 0x7f09b81605e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:43.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.717+0000 7f09bdd85640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f09b8160b20 0x7f09b8162f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:43.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.717+0000 7f09bdd85640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09b8071e70 con 0x7f09b8159570 2023-09-23T03:17:43.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.717+0000 7f09bdd85640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09b8072150 con 0x7f09b8160b20 2023-09-23T03:17:43.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.717+0000 7f09bdd85640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b8071c70 con 0x7f09b8071810 2023-09-23T03:17:43.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.717+0000 7f09bcd83640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f09b8159570 0x7f09b8071100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:41030/0 (socket says 172.21.15.154:41030) 2023-09-23T03:17:43.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.717+0000 7f09bcd83640 1 -- 172.21.15.154:0/2582857215 learned_addr learned my addr 172.21.15.154:0/2582857215 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:17:43.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.717+0000 7f09bd584640 1 --2- 172.21.15.154:0/2582857215 >> v2:172.21.15.154:3300/0 conn(0x7f09b8160b20 0x7f09b8162f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:43.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.718+0000 7f09b77fe640 1 -- 172.21.15.154:0/2582857215 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2873084339 0 0) 0x7f09b8071e70 con 0x7f09b8159570 2023-09-23T03:17:43.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.718+0000 7f09b7fff640 1 --2- 172.21.15.154:0/2582857215 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09b8071810 0x7f09b81605e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:43.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.718+0000 7f09b77fe640 1 -- 172.21.15.154:0/2582857215 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09a0003520 con 0x7f09b8159570 2023-09-23T03:17:43.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.718+0000 7f09bd584640 1 -- 172.21.15.154:0/2582857215 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09b8071810 msgr2=0x7f09b81605e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:43.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.718+0000 7f09bd584640 1 --2- 172.21.15.154:0/2582857215 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09b8071810 0x7f09b81605e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:43.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.718+0000 7f09bd584640 1 -- 172.21.15.154:0/2582857215 >> v1:172.21.15.154:6789/0 conn(0x7f09b8159570 legacy=0x7f09b8071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:43.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.718+0000 7f09bd584640 1 -- 172.21.15.154:0/2582857215 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b80c8b50 con 0x7f09b8160b20 2023-09-23T03:17:43.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.719+0000 7f09bd584640 1 --2- 172.21.15.154:0/2582857215 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09b8160b20 0x7f09b8162f10 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f09b0009d90 tx=0x7f09b002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=408dad3c289b1028 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:43.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.720+0000 7f09b77fe640 1 -- 172.21.15.154:0/2582857215 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b002cbc0 con 0x7f09b8160b20 2023-09-23T03:17:43.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.720+0000 7f09b77fe640 1 -- 172.21.15.154:0/2582857215 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b002cd20 con 0x7f09b8160b20 2023-09-23T03:17:43.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.720+0000 7f09b77fe640 1 -- 172.21.15.154:0/2582857215 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b0034660 con 0x7f09b8160b20 2023-09-23T03:17:43.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.720+0000 7f09bdd85640 1 -- 172.21.15.154:0/2582857215 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09b8160b20 msgr2=0x7f09b8162f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:43.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.720+0000 7f09bdd85640 1 --2- 172.21.15.154:0/2582857215 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09b8160b20 0x7f09b8162f10 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f09b0009d90 tx=0x7f09b002be10 comp rx=0 tx=0).stop 2023-09-23T03:17:43.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.721+0000 7f09bdd85640 1 -- 172.21.15.154:0/2582857215 shutdown_connections 2023-09-23T03:17:43.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.721+0000 7f09bdd85640 1 --2- 172.21.15.154:0/2582857215 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09b8160b20 0x7f09b8162f10 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:43.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.721+0000 7f09bdd85640 1 --2- 172.21.15.154:0/2582857215 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09b8071810 0x7f09b81605e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:43.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.721+0000 7f09bdd85640 1 -- 172.21.15.154:0/2582857215 >> 172.21.15.154:0/2582857215 conn(0x7f09b80aff20 msgr2=0x7f09b80b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:43.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.721+0000 7f09bdd85640 1 -- 172.21.15.154:0/2582857215 shutdown_connections 2023-09-23T03:17:43.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.721+0000 7f09bdd85640 1 -- 172.21.15.154:0/2582857215 wait complete. 2023-09-23T03:17:43.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.722+0000 7f09bdd85640 1 Processor -- start 2023-09-23T03:17:43.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.723+0000 7f09bdd85640 1 -- start start 2023-09-23T03:17:43.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.724+0000 7f09bdd85640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09b8071810 0x7f09b80c5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:43.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.724+0000 7f09bdd85640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09b8159570 0x7f09b80c7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:43.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.724+0000 7f09bdd85640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09b8160b20 0x7f09b814b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:43.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.724+0000 7f09bdd85640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b80c7f40 con 0x7f09b8159570 2023-09-23T03:17:43.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.724+0000 7f09bdd85640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b80c8220 con 0x7f09b8160b20 2023-09-23T03:17:43.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.724+0000 7f09bdd85640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b815d000 con 0x7f09b8071810 2023-09-23T03:17:43.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.724+0000 7f09b7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09b8159570 0x7f09b80c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:43.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.724+0000 7f09b7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09b8159570 0x7f09b80c7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56612/0 (socket says 172.21.15.154:56612) 2023-09-23T03:17:43.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.724+0000 7f09b7fff640 1 -- 172.21.15.154:0/440662771 learned_addr learned my addr 172.21.15.154:0/440662771 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:43.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.724+0000 7f09bcd83640 1 --2- 172.21.15.154:0/440662771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09b8071810 0x7f09b80c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:43.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.724+0000 7f09b7fff640 1 -- 172.21.15.154:0/440662771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09b8071810 msgr2=0x7f09b80c5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:43.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.724+0000 7f09b7fff640 1 --2- 172.21.15.154:0/440662771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09b8071810 0x7f09b80c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:43.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.724+0000 7f09b7fff640 1 -- 172.21.15.154:0/440662771 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09b8160b20 msgr2=0x7f09b814b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:43.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.724+0000 7f09b7fff640 1 --2- 172.21.15.154:0/440662771 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09b8160b20 0x7f09b814b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:43.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.724+0000 7f09b7fff640 1 -- 172.21.15.154:0/440662771 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b0009a70 con 0x7f09b8159570 2023-09-23T03:17:43.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.725+0000 7f09b7fff640 1 --2- 172.21.15.154:0/440662771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09b8159570 0x7f09b80c7540 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f09a800d800 tx=0x7f09a800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:43.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.725+0000 7f09b57fa640 1 -- 172.21.15.154:0/440662771 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a800b6c0 con 0x7f09b8159570 2023-09-23T03:17:43.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.725+0000 7f09b57fa640 1 -- 172.21.15.154:0/440662771 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a800b820 con 0x7f09b8159570 2023-09-23T03:17:43.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.725+0000 7f09b57fa640 1 -- 172.21.15.154:0/440662771 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a800bbb0 con 0x7f09b8159570 2023-09-23T03:17:43.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.725+0000 7f09bdd85640 1 -- 172.21.15.154:0/440662771 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b815d2e0 con 0x7f09b8159570 2023-09-23T03:17:43.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.725+0000 7f09bdd85640 1 -- 172.21.15.154:0/440662771 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b80c5c50 con 0x7f09b8159570 2023-09-23T03:17:43.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.726+0000 7f09bdd85640 1 -- 172.21.15.154:0/440662771 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09b80c5e10 con 0x7f09b8159570 2023-09-23T03:17:43.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.727+0000 7f09b57fa640 1 -- 172.21.15.154:0/440662771 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f09a800bd10 con 0x7f09b8159570 2023-09-23T03:17:43.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.729+0000 7f09b57fa640 1 --2- 172.21.15.154:0/440662771 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f09a406af30 0x7f09a406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:43.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.729+0000 7f09b57fa640 1 -- 172.21.15.154:0/440662771 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f09a808a020 con 0x7f09b8159570 2023-09-23T03:17:43.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.729+0000 7f09bcd83640 1 --2- 172.21.15.154:0/440662771 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f09a406af30 0x7f09a406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:43.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.730+0000 7f09bcd83640 1 --2- 172.21.15.154:0/440662771 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f09a406af30 0x7f09a406d3f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f09ac0011e0 tx=0x7f09ac009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:43.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.733+0000 7f09b57fa640 1 -- 172.21.15.154:0/440662771 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f09a8054d60 con 0x7f09b8159570 2023-09-23T03:17:43.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.925+0000 7f09bdd85640 1 -- 172.21.15.154:0/440662771 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f09b804e920 con 0x7f09b8159570 2023-09-23T03:17:43.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.926+0000 7f09b57fa640 1 -- 172.21.15.154:0/440662771 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) v1 ==== 100+0+11 (secure 0 0 0) 0x7f09a8058380 con 0x7f09b8159570 2023-09-23T03:17:43.927 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:17:43.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.929+0000 7f09bdd85640 1 -- 172.21.15.154:0/440662771 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f09a406af30 msgr2=0x7f09a406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:43.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.929+0000 7f09bdd85640 1 --2- 172.21.15.154:0/440662771 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f09a406af30 0x7f09a406d3f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f09ac0011e0 tx=0x7f09ac009040 comp rx=0 tx=0).stop 2023-09-23T03:17:43.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.929+0000 7f09bdd85640 1 -- 172.21.15.154:0/440662771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09b8159570 msgr2=0x7f09b80c7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:43.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.929+0000 7f09bdd85640 1 --2- 172.21.15.154:0/440662771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09b8159570 0x7f09b80c7540 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f09a800d800 tx=0x7f09a800dcc0 comp rx=0 tx=0).stop 2023-09-23T03:17:43.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.929+0000 7f09bdd85640 1 -- 172.21.15.154:0/440662771 shutdown_connections 2023-09-23T03:17:43.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.929+0000 7f09bdd85640 1 --2- 172.21.15.154:0/440662771 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f09a406af30 0x7f09a406d3f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:43.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.929+0000 7f09bdd85640 1 --2- 172.21.15.154:0/440662771 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09b8160b20 0x7f09b814b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:43.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.929+0000 7f09bdd85640 1 --2- 172.21.15.154:0/440662771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09b8159570 0x7f09b80c7540 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:43.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.929+0000 7f09bdd85640 1 --2- 172.21.15.154:0/440662771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09b8071810 0x7f09b80c5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:43.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.929+0000 7f09bdd85640 1 -- 172.21.15.154:0/440662771 >> 172.21.15.154:0/440662771 conn(0x7f09b80aff20 msgr2=0x7f09b8157df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:43.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.929+0000 7f09bdd85640 1 -- 172.21.15.154:0/440662771 shutdown_connections 2023-09-23T03:17:43.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:43.929+0000 7f09bdd85640 1 -- 172.21.15.154:0/440662771 wait complete. 2023-09-23T03:17:43.948 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-09-23T03:17:44.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.168+0000 7fdb166d7640 1 Processor -- start 2023-09-23T03:17:44.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.168+0000 7fdb166d7640 1 -- start start 2023-09-23T03:17:44.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.170+0000 7fdb166d7640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdb100d0980 0x7fdb100b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:44.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.170+0000 7fdb166d7640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdb100d1890 0x7fdb100cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:44.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.170+0000 7fdb166d7640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb1015e330 con 0x7fdb1015df50 2023-09-23T03:17:44.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.170+0000 7fdb166d7640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb100cd870 con 0x7fdb100d0980 2023-09-23T03:17:44.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.170+0000 7fdb166d7640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb100d0e10 con 0x7fdb100d1890 2023-09-23T03:17:44.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.170+0000 7fdb14ed4640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdb100d0980 0x7fdb100b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:44.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.170+0000 7fdb14ed4640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdb100d0980 0x7fdb100b20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48820/0 (socket says 172.21.15.154:48820) 2023-09-23T03:17:44.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.170+0000 7fdb14ed4640 1 -- 172.21.15.154:0/4084243175 learned_addr learned my addr 172.21.15.154:0/4084243175 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:44.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.171+0000 7fdb15ed6640 1 --2- 172.21.15.154:0/4084243175 >> v2:172.21.15.190:3300/0 conn(0x7fdb100d1890 0x7fdb100cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:44.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.171+0000 7fdb14ed4640 1 -- 172.21.15.154:0/4084243175 >> v2:172.21.15.190:3300/0 conn(0x7fdb100d1890 msgr2=0x7fdb100cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:44.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.171+0000 7fdb14ed4640 1 --2- 172.21.15.154:0/4084243175 >> v2:172.21.15.190:3300/0 conn(0x7fdb100d1890 0x7fdb100cdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:44.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.171+0000 7fdb14ed4640 1 -- 172.21.15.154:0/4084243175 >> v1:172.21.15.154:6789/0 conn(0x7fdb1015df50 legacy=0x7fdb100d1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:44.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.171+0000 7fdb14ed4640 1 -- 172.21.15.154:0/4084243175 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb100b4c50 con 0x7fdb100d0980 2023-09-23T03:17:44.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.172+0000 7fdb14ed4640 1 --2- 172.21.15.154:0/4084243175 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb100d0980 0x7fdb100b20f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fdb101607c0 tx=0x7fdb0002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=152e2bf3bfd6e7e9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:44.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.172+0000 7fdb07fff640 1 -- 172.21.15.154:0/4084243175 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb00031870 con 0x7fdb100d0980 2023-09-23T03:17:44.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.172+0000 7fdb07fff640 1 -- 172.21.15.154:0/4084243175 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb000319d0 con 0x7fdb100d0980 2023-09-23T03:17:44.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.172+0000 7fdb07fff640 1 -- 172.21.15.154:0/4084243175 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb00031d60 con 0x7fdb100d0980 2023-09-23T03:17:44.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.173+0000 7fdb166d7640 1 -- 172.21.15.154:0/4084243175 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb100d0980 msgr2=0x7fdb100b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:44.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.173+0000 7fdb166d7640 1 --2- 172.21.15.154:0/4084243175 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb100d0980 0x7fdb100b20f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fdb101607c0 tx=0x7fdb0002b910 comp rx=0 tx=0).stop 2023-09-23T03:17:44.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.173+0000 7fdb166d7640 1 -- 172.21.15.154:0/4084243175 shutdown_connections 2023-09-23T03:17:44.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.173+0000 7fdb166d7640 1 --2- 172.21.15.154:0/4084243175 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb100d0980 0x7fdb100b20f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:44.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.173+0000 7fdb166d7640 1 --2- 172.21.15.154:0/4084243175 >> v2:172.21.15.190:3300/0 conn(0x7fdb100d1890 0x7fdb100cdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:44.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.173+0000 7fdb166d7640 1 -- 172.21.15.154:0/4084243175 >> 172.21.15.154:0/4084243175 conn(0x7fdb1009bb20 msgr2=0x7fdb100d2870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:44.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.173+0000 7fdb166d7640 1 -- 172.21.15.154:0/4084243175 shutdown_connections 2023-09-23T03:17:44.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.173+0000 7fdb166d7640 1 -- 172.21.15.154:0/4084243175 wait complete. 2023-09-23T03:17:44.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.175+0000 7fdb166d7640 1 Processor -- start 2023-09-23T03:17:44.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.175+0000 7fdb166d7640 1 -- start start 2023-09-23T03:17:44.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.176+0000 7fdb166d7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb100d1890 0x7fdb101536a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:44.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.176+0000 7fdb166d7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb1015df50 0x7fdb1015fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:44.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.176+0000 7fdb166d7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb1015e950 0x7fdb100c7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:44.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.176+0000 7fdb166d7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb100b6f10 con 0x7fdb1015e950 2023-09-23T03:17:44.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.176+0000 7fdb166d7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb100c7a70 con 0x7fdb1015df50 2023-09-23T03:17:44.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.176+0000 7fdb166d7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb100c7d50 con 0x7fdb100d1890 2023-09-23T03:17:44.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.176+0000 7fdb15ed6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb1015e950 0x7fdb100c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:44.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.176+0000 7fdb15ed6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb1015e950 0x7fdb100c7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48830/0 (socket says 172.21.15.154:48830) 2023-09-23T03:17:44.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.176+0000 7fdb15ed6640 1 -- 172.21.15.154:0/4018049187 learned_addr learned my addr 172.21.15.154:0/4018049187 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:44.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.177+0000 7fdb14ed4640 1 --2- 172.21.15.154:0/4018049187 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb1015df50 0x7fdb1015fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:44.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.177+0000 7fdb156d5640 1 --2- 172.21.15.154:0/4018049187 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb100d1890 0x7fdb101536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:44.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.177+0000 7fdb15ed6640 1 -- 172.21.15.154:0/4018049187 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb100d1890 msgr2=0x7fdb101536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:44.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.177+0000 7fdb15ed6640 1 --2- 172.21.15.154:0/4018049187 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb100d1890 0x7fdb101536a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:44.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.177+0000 7fdb15ed6640 1 -- 172.21.15.154:0/4018049187 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb1015df50 msgr2=0x7fdb1015fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:44.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.177+0000 7fdb15ed6640 1 --2- 172.21.15.154:0/4018049187 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb1015df50 0x7fdb1015fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:44.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.177+0000 7fdb15ed6640 1 -- 172.21.15.154:0/4018049187 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb000094d0 con 0x7fdb1015e950 2023-09-23T03:17:44.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.177+0000 7fdb15ed6640 1 --2- 172.21.15.154:0/4018049187 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb1015e950 0x7fdb100c7220 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fdb08007e40 tx=0x7fdb0800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:44.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.177+0000 7fdb05ffb640 1 -- 172.21.15.154:0/4018049187 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb08014620 con 0x7fdb1015e950 2023-09-23T03:17:44.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.177+0000 7fdb05ffb640 1 -- 172.21.15.154:0/4018049187 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb0800ee90 con 0x7fdb1015e950 2023-09-23T03:17:44.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.177+0000 7fdb166d7640 1 -- 172.21.15.154:0/4018049187 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb100c8030 con 0x7fdb1015e950 2023-09-23T03:17:44.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.177+0000 7fdb05ffb640 1 -- 172.21.15.154:0/4018049187 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb080149e0 con 0x7fdb1015e950 2023-09-23T03:17:44.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.178+0000 7fdb166d7640 1 -- 172.21.15.154:0/4018049187 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb100c7760 con 0x7fdb1015e950 2023-09-23T03:17:44.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.179+0000 7fdb166d7640 1 -- 172.21.15.154:0/4018049187 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb1004e920 con 0x7fdb1015e950 2023-09-23T03:17:44.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.180+0000 7fdb05ffb640 1 -- 172.21.15.154:0/4018049187 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fdb08014b40 con 0x7fdb1015e950 2023-09-23T03:17:44.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.180+0000 7fdb05ffb640 1 --2- 172.21.15.154:0/4018049187 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdaf406af60 0x7fdaf406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:44.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.181+0000 7fdb05ffb640 1 -- 172.21.15.154:0/4018049187 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6433+0+0 (secure 0 0 0) 0x7fdb0808a3e0 con 0x7fdb1015e950 2023-09-23T03:17:44.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.181+0000 7fdb156d5640 1 --2- 172.21.15.154:0/4018049187 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdaf406af60 0x7fdaf406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:44.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.181+0000 7fdb156d5640 1 --2- 172.21.15.154:0/4018049187 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdaf406af60 0x7fdaf406d420 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c000f60 tx=0x7fdb0c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:44.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.185+0000 7fdb05ffb640 1 -- 172.21.15.154:0/4018049187 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fdb08055130 con 0x7fdb1015e950 2023-09-23T03:17:44.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.374+0000 7fdb166d7640 1 -- 172.21.15.154:0/4018049187 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7fdb1015d320 con 0x7fdb1015e950 2023-09-23T03:17:44.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.375+0000 7fdb05ffb640 1 -- 172.21.15.154:0/4018049187 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v41) v1 ==== 106+0+11 (secure 0 0 0) 0x7fdb08058750 con 0x7fdb1015e950 2023-09-23T03:17:44.376 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:17:44.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.378+0000 7fdb166d7640 1 -- 172.21.15.154:0/4018049187 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdaf406af60 msgr2=0x7fdaf406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:44.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.378+0000 7fdb166d7640 1 --2- 172.21.15.154:0/4018049187 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdaf406af60 0x7fdaf406d420 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c000f60 tx=0x7fdb0c009040 comp rx=0 tx=0).stop 2023-09-23T03:17:44.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.378+0000 7fdb166d7640 1 -- 172.21.15.154:0/4018049187 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb1015e950 msgr2=0x7fdb100c7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:44.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.378+0000 7fdb166d7640 1 --2- 172.21.15.154:0/4018049187 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb1015e950 0x7fdb100c7220 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fdb08007e40 tx=0x7fdb0800e4b0 comp rx=0 tx=0).stop 2023-09-23T03:17:44.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.378+0000 7fdb166d7640 1 -- 172.21.15.154:0/4018049187 shutdown_connections 2023-09-23T03:17:44.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.378+0000 7fdb166d7640 1 --2- 172.21.15.154:0/4018049187 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdaf406af60 0x7fdaf406d420 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:44.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.378+0000 7fdb166d7640 1 --2- 172.21.15.154:0/4018049187 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb1015e950 0x7fdb100c7220 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:44.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.378+0000 7fdb166d7640 1 --2- 172.21.15.154:0/4018049187 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb1015df50 0x7fdb1015fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:44.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.378+0000 7fdb166d7640 1 --2- 172.21.15.154:0/4018049187 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb100d1890 0x7fdb101536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:44.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.378+0000 7fdb166d7640 1 -- 172.21.15.154:0/4018049187 >> 172.21.15.154:0/4018049187 conn(0x7fdb1009bb20 msgr2=0x7fdb100b1980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:44.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.378+0000 7fdb166d7640 1 -- 172.21.15.154:0/4018049187 shutdown_connections 2023-09-23T03:17:44.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.378+0000 7fdb166d7640 1 -- 172.21.15.154:0/4018049187 wait complete. 2023-09-23T03:17:44.398 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-23T03:17:44.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.618+0000 7f06790c3640 1 Processor -- start 2023-09-23T03:17:44.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.618+0000 7f06790c3640 1 -- start start 2023-09-23T03:17:44.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.619+0000 7f06790c3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06740cee20 0x7f06740cb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:44.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.619+0000 7f06790c3640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f06740aec50 0x7f06740af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:44.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.619+0000 7f06790c3640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06740cbb50 con 0x7f067415cb80 2023-09-23T03:17:44.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.619+0000 7f06790c3640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f067415cf60 con 0x7f06740aec50 2023-09-23T03:17:44.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.619+0000 7f06790c3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06740cbd10 con 0x7f06740cee20 2023-09-23T03:17:44.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.620+0000 7f06788c2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f06740aec50 0x7f06740af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:44.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.620+0000 7f06788c2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f06740aec50 0x7f06740af0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48834/0 (socket says 172.21.15.154:48834) 2023-09-23T03:17:44.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.620+0000 7f06788c2640 1 -- 172.21.15.154:0/3117409551 learned_addr learned my addr 172.21.15.154:0/3117409551 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:44.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.620+0000 7f06788c2640 1 -- 172.21.15.154:0/3117409551 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06740cee20 msgr2=0x7f06740cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:44.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.620+0000 7f06788c2640 1 --2- 172.21.15.154:0/3117409551 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06740cee20 0x7f06740cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:44.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.620+0000 7f06788c2640 1 -- 172.21.15.154:0/3117409551 >> v1:172.21.15.190:6789/0 conn(0x7f067415cb80 legacy=0x7f06740ce710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:44.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.620+0000 7f06788c2640 1 -- 172.21.15.154:0/3117409551 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06740cbff0 con 0x7f06740aec50 2023-09-23T03:17:44.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.621+0000 7f06788c2640 1 --2- 172.21.15.154:0/3117409551 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06740aec50 0x7f06740af0d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f066c009870 tx=0x7f066c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c40c7e6d1076998 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:44.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.622+0000 7f0672ffd640 1 -- 172.21.15.154:0/3117409551 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c0317c0 con 0x7f06740aec50 2023-09-23T03:17:44.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.622+0000 7f0672ffd640 1 -- 172.21.15.154:0/3117409551 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066c031920 con 0x7f06740aec50 2023-09-23T03:17:44.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.622+0000 7f0672ffd640 1 -- 172.21.15.154:0/3117409551 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c031cb0 con 0x7f06740aec50 2023-09-23T03:17:44.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.622+0000 7f06790c3640 1 -- 172.21.15.154:0/3117409551 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06740aec50 msgr2=0x7f06740af0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:44.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.622+0000 7f06790c3640 1 --2- 172.21.15.154:0/3117409551 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06740aec50 0x7f06740af0d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f066c009870 tx=0x7f066c02b860 comp rx=0 tx=0).stop 2023-09-23T03:17:44.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.622+0000 7f06790c3640 1 -- 172.21.15.154:0/3117409551 shutdown_connections 2023-09-23T03:17:44.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.622+0000 7f06790c3640 1 --2- 172.21.15.154:0/3117409551 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06740aec50 0x7f06740af0d0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:44.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.622+0000 7f06790c3640 1 --2- 172.21.15.154:0/3117409551 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06740cee20 0x7f06740cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:44.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.622+0000 7f06790c3640 1 -- 172.21.15.154:0/3117409551 >> 172.21.15.154:0/3117409551 conn(0x7f0674098a40 msgr2=0x7f06740cf510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:44.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.622+0000 7f06790c3640 1 -- 172.21.15.154:0/3117409551 shutdown_connections 2023-09-23T03:17:44.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.623+0000 7f06790c3640 1 -- 172.21.15.154:0/3117409551 wait complete. 2023-09-23T03:17:44.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.624+0000 7f06790c3640 1 Processor -- start 2023-09-23T03:17:44.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.624+0000 7f06790c3640 1 -- start start 2023-09-23T03:17:44.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.625+0000 7f06790c3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06740aec50 0x7f06740c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:44.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.625+0000 7f06790c3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06740cee20 0x7f06740c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:44.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.625+0000 7f06790c3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f067415cb80 0x7f067415a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:44.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.625+0000 7f06790c3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0674152310 con 0x7f067415cb80 2023-09-23T03:17:44.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.625+0000 7f06790c3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06740b42e0 con 0x7f06740aec50 2023-09-23T03:17:44.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.625+0000 7f06790c3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06740b36d0 con 0x7f06740cee20 2023-09-23T03:17:44.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.626+0000 7f06788c2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f067415cb80 0x7f067415a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:44.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.626+0000 7f06788c2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f067415cb80 0x7f067415a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48840/0 (socket says 172.21.15.154:48840) 2023-09-23T03:17:44.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.626+0000 7f06788c2640 1 -- 172.21.15.154:0/1091225015 learned_addr learned my addr 172.21.15.154:0/1091225015 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:44.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.626+0000 7f06737fe640 1 --2- 172.21.15.154:0/1091225015 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06740cee20 0x7f06740c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:44.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.626+0000 7f0673fff640 1 --2- 172.21.15.154:0/1091225015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06740aec50 0x7f06740c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:44.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.626+0000 7f06788c2640 1 -- 172.21.15.154:0/1091225015 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06740cee20 msgr2=0x7f06740c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:44.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.626+0000 7f06788c2640 1 --2- 172.21.15.154:0/1091225015 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06740cee20 0x7f06740c8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:44.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.626+0000 7f06788c2640 1 -- 172.21.15.154:0/1091225015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06740aec50 msgr2=0x7f06740c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:44.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.626+0000 7f06788c2640 1 --2- 172.21.15.154:0/1091225015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06740aec50 0x7f06740c8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:44.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.626+0000 7f06788c2640 1 -- 172.21.15.154:0/1091225015 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f066c0094d0 con 0x7f067415cb80 2023-09-23T03:17:44.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.626+0000 7f0673fff640 1 --2- 172.21.15.154:0/1091225015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06740aec50 0x7f06740c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:44.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.627+0000 7f06788c2640 1 --2- 172.21.15.154:0/1091225015 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f067415cb80 0x7f067415a020 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f066c009870 tx=0x7f066c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:44.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.627+0000 7f0670ff9640 1 -- 172.21.15.154:0/1091225015 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c032860 con 0x7f067415cb80 2023-09-23T03:17:44.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.627+0000 7f0670ff9640 1 -- 172.21.15.154:0/1091225015 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066c0329e0 con 0x7f067415cb80 2023-09-23T03:17:44.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.627+0000 7f0670ff9640 1 -- 172.21.15.154:0/1091225015 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c032d90 con 0x7f067415cb80 2023-09-23T03:17:44.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.627+0000 7f06790c3640 1 -- 172.21.15.154:0/1091225015 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06740b39b0 con 0x7f067415cb80 2023-09-23T03:17:44.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.627+0000 7f06790c3640 1 -- 172.21.15.154:0/1091225015 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06740b3f70 con 0x7f067415cb80 2023-09-23T03:17:44.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.628+0000 7f06790c3640 1 -- 172.21.15.154:0/1091225015 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06740b4450 con 0x7f067415cb80 2023-09-23T03:17:44.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.629+0000 7f0670ff9640 1 -- 172.21.15.154:0/1091225015 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f066c0453f0 con 0x7f067415cb80 2023-09-23T03:17:44.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.630+0000 7f0670ff9640 1 --2- 172.21.15.154:0/1091225015 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f066006af60 0x7f066006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:44.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.630+0000 7f0670ff9640 1 -- 172.21.15.154:0/1091225015 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f066c0abbf0 con 0x7f067415cb80 2023-09-23T03:17:44.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.630+0000 7f0673fff640 1 --2- 172.21.15.154:0/1091225015 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f066006af60 0x7f066006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:44.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.631+0000 7f0673fff640 1 --2- 172.21.15.154:0/1091225015 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f066006af60 0x7f066006d420 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0664004520 tx=0x7f0664009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:44.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.635+0000 7f0670ff9640 1 -- 172.21.15.154:0/1091225015 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f066c076940 con 0x7f067415cb80 2023-09-23T03:17:44.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.840+0000 7f06790c3640 1 -- 172.21.15.154:0/1091225015 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f06740cbb50 con 0x7f067415cb80 2023-09-23T03:17:44.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.841+0000 7f0670ff9640 1 -- 172.21.15.154:0/1091225015 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v41) v1 ==== 80+0+0 (secure 0 0 0) 0x7f06740cbb50 con 0x7f067415cb80 2023-09-23T03:17:44.842 INFO:teuthology.orchestra.run.smithi154.stderr:listed 0 entries 2023-09-23T03:17:44.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.844+0000 7f06790c3640 1 -- 172.21.15.154:0/1091225015 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f066006af60 msgr2=0x7f066006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:44.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.844+0000 7f06790c3640 1 --2- 172.21.15.154:0/1091225015 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f066006af60 0x7f066006d420 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0664004520 tx=0x7f0664009040 comp rx=0 tx=0).stop 2023-09-23T03:17:44.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.844+0000 7f06790c3640 1 -- 172.21.15.154:0/1091225015 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f067415cb80 msgr2=0x7f067415a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:44.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.844+0000 7f06790c3640 1 --2- 172.21.15.154:0/1091225015 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f067415cb80 0x7f067415a020 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f066c009870 tx=0x7f066c0046c0 comp rx=0 tx=0).stop 2023-09-23T03:17:44.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.844+0000 7f06790c3640 1 -- 172.21.15.154:0/1091225015 shutdown_connections 2023-09-23T03:17:44.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.844+0000 7f06790c3640 1 --2- 172.21.15.154:0/1091225015 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f066006af60 0x7f066006d420 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:44.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.844+0000 7f06790c3640 1 --2- 172.21.15.154:0/1091225015 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f067415cb80 0x7f067415a020 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:44.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.844+0000 7f06790c3640 1 --2- 172.21.15.154:0/1091225015 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06740cee20 0x7f06740c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:44.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.844+0000 7f06790c3640 1 --2- 172.21.15.154:0/1091225015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06740aec50 0x7f06740c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:44.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.844+0000 7f06790c3640 1 -- 172.21.15.154:0/1091225015 >> 172.21.15.154:0/1091225015 conn(0x7f0674098a40 msgr2=0x7f06740ca7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:44.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.844+0000 7f06790c3640 1 -- 172.21.15.154:0/1091225015 shutdown_connections 2023-09-23T03:17:44.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:44.845+0000 7f06790c3640 1 -- 172.21.15.154:0/1091225015 wait complete. 2023-09-23T03:17:44.864 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:17:44.864 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:17:44.914 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:17:45.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.178+0000 7f86a9325640 1 Processor -- start 2023-09-23T03:17:45.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.178+0000 7f86a9325640 1 -- start start 2023-09-23T03:17:45.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.179+0000 7f86a9325640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f86a40acce0 0x7f86a415ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:45.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.180+0000 7f86a9325640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86a415ac30 con 0x7f86a40cf040 2023-09-23T03:17:45.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.180+0000 7f86a9325640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86a40b1f70 con 0x7f86a40b1b20 2023-09-23T03:17:45.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.180+0000 7f86a9325640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86a40b2e90 con 0x7f86a40acce0 2023-09-23T03:17:45.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.180+0000 7f86a2ffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f86a40acce0 0x7f86a415ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:45.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.180+0000 7f86a2ffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f86a40acce0 0x7f86a415ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48850/0 (socket says 172.21.15.154:48850) 2023-09-23T03:17:45.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.180+0000 7f86a37fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f86a40cf040 0x7f86a40ab140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:43528/0 (socket says 172.21.15.154:43528) 2023-09-23T03:17:45.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.180+0000 7f86a2ffd640 1 -- 172.21.15.154:0/1446268919 learned_addr learned my addr 172.21.15.154:0/1446268919 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:45.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.180+0000 7f86a2ffd640 1 -- 172.21.15.154:0/1446268919 >> v1:172.21.15.190:6789/0 conn(0x7f86a40b1b20 legacy=0x7f86a40ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:45.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.180+0000 7f86a2ffd640 1 -- 172.21.15.154:0/1446268919 >> v1:172.21.15.154:6789/0 conn(0x7f86a40cf040 legacy=0x7f86a40ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:45.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.180+0000 7f86a2ffd640 1 -- 172.21.15.154:0/1446268919 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a415f2a0 con 0x7f86a40acce0 2023-09-23T03:17:45.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.180+0000 7f86a1ffb640 1 -- 172.21.15.154:0/1446268919 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3821787397 0 0) 0x7f869c001490 con 0x7f86a40cf040 2023-09-23T03:17:45.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.181+0000 7f86a2ffd640 1 --2- 172.21.15.154:0/1446268919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86a40acce0 0x7f86a415ed60 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f8694009850 tx=0x7f869402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb973358f8ff2685 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:45.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.182+0000 7f86a1ffb640 1 -- 172.21.15.154:0/1446268919 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8694004230 con 0x7f86a40acce0 2023-09-23T03:17:45.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.182+0000 7f86a1ffb640 1 -- 172.21.15.154:0/1446268919 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8694004390 con 0x7f86a40acce0 2023-09-23T03:17:45.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.182+0000 7f86a1ffb640 1 -- 172.21.15.154:0/1446268919 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8694003810 con 0x7f86a40acce0 2023-09-23T03:17:45.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.182+0000 7f86a9325640 1 -- 172.21.15.154:0/1446268919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86a40acce0 msgr2=0x7f86a415ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:45.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.182+0000 7f86a9325640 1 --2- 172.21.15.154:0/1446268919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86a40acce0 0x7f86a415ed60 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f8694009850 tx=0x7f869402b8f0 comp rx=0 tx=0).stop 2023-09-23T03:17:45.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.182+0000 7f86a9325640 1 -- 172.21.15.154:0/1446268919 shutdown_connections 2023-09-23T03:17:45.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.182+0000 7f86a9325640 1 --2- 172.21.15.154:0/1446268919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86a40acce0 0x7f86a415ed60 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:45.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.182+0000 7f86a9325640 1 -- 172.21.15.154:0/1446268919 >> 172.21.15.154:0/1446268919 conn(0x7f86a406a0f0 msgr2=0x7f86a406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:45.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.183+0000 7f86a9325640 1 -- 172.21.15.154:0/1446268919 shutdown_connections 2023-09-23T03:17:45.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.183+0000 7f86a9325640 1 -- 172.21.15.154:0/1446268919 wait complete. 2023-09-23T03:17:45.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.184+0000 7f86a9325640 1 Processor -- start 2023-09-23T03:17:45.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.184+0000 7f86a9325640 1 -- start start 2023-09-23T03:17:45.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.185+0000 7f86a9325640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86a40acce0 0x7f86a415b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:45.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.185+0000 7f86a9325640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86a40b1b20 0x7f86a40c3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:45.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.185+0000 7f86a9325640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86a40cf040 0x7f86a40c55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:45.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.185+0000 7f86a9325640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a415bd00 con 0x7f86a40b1b20 2023-09-23T03:17:45.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.185+0000 7f86a9325640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a40c2ae0 con 0x7f86a40cf040 2023-09-23T03:17:45.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.185+0000 7f86a9325640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a40c2d50 con 0x7f86a40acce0 2023-09-23T03:17:45.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.186+0000 7f86a27fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86a40b1b20 0x7f86a40c3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:45.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.186+0000 7f86a27fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86a40b1b20 0x7f86a40c3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48856/0 (socket says 172.21.15.154:48856) 2023-09-23T03:17:45.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.186+0000 7f86a27fc640 1 -- 172.21.15.154:0/402819790 learned_addr learned my addr 172.21.15.154:0/402819790 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:45.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.186+0000 7f86a2ffd640 1 --2- 172.21.15.154:0/402819790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86a40acce0 0x7f86a415b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:45.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.186+0000 7f86a37fe640 1 --2- 172.21.15.154:0/402819790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86a40cf040 0x7f86a40c55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:45.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.186+0000 7f86a27fc640 1 -- 172.21.15.154:0/402819790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86a40acce0 msgr2=0x7f86a415b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:45.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.186+0000 7f86a27fc640 1 --2- 172.21.15.154:0/402819790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86a40acce0 0x7f86a415b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:45.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.186+0000 7f86a27fc640 1 -- 172.21.15.154:0/402819790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86a40cf040 msgr2=0x7f86a40c55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:45.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.186+0000 7f86a27fc640 1 --2- 172.21.15.154:0/402819790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86a40cf040 0x7f86a40c55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:45.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.186+0000 7f86a27fc640 1 -- 172.21.15.154:0/402819790 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8694009460 con 0x7f86a40b1b20 2023-09-23T03:17:45.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.186+0000 7f86a37fe640 1 --2- 172.21.15.154:0/402819790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86a40cf040 0x7f86a40c55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:45.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.186+0000 7f86a27fc640 1 --2- 172.21.15.154:0/402819790 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86a40b1b20 0x7f86a40c3b10 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f869800c430 tx=0x7f869800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:45.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.187+0000 7f868b7fe640 1 -- 172.21.15.154:0/402819790 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8698011790 con 0x7f86a40b1b20 2023-09-23T03:17:45.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.187+0000 7f868b7fe640 1 -- 172.21.15.154:0/402819790 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86980118f0 con 0x7f86a40b1b20 2023-09-23T03:17:45.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.187+0000 7f868b7fe640 1 -- 172.21.15.154:0/402819790 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8698011ce0 con 0x7f86a40b1b20 2023-09-23T03:17:45.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.187+0000 7f86a9325640 1 -- 172.21.15.154:0/402819790 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86a40c3030 con 0x7f86a40b1b20 2023-09-23T03:17:45.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.187+0000 7f86a9325640 1 -- 172.21.15.154:0/402819790 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86a40c35f0 con 0x7f86a40b1b20 2023-09-23T03:17:45.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.188+0000 7f86a9325640 1 -- 172.21.15.154:0/402819790 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86a40b27b0 con 0x7f86a40b1b20 2023-09-23T03:17:45.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.189+0000 7f868b7fe640 1 -- 172.21.15.154:0/402819790 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f869801c460 con 0x7f86a40b1b20 2023-09-23T03:17:45.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.190+0000 7f868b7fe640 1 --2- 172.21.15.154:0/402819790 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f869006af30 0x7f869006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:45.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.190+0000 7f868b7fe640 1 -- 172.21.15.154:0/402819790 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f8698089a40 con 0x7f86a40b1b20 2023-09-23T03:17:45.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.190+0000 7f86a2ffd640 1 --2- 172.21.15.154:0/402819790 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f869006af30 0x7f869006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:45.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.191+0000 7f86a2ffd640 1 --2- 172.21.15.154:0/402819790 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f869006af30 0x7f869006d3f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8694035000 tx=0x7f8694008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:45.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.194+0000 7f868b7fe640 1 -- 172.21.15.154:0/402819790 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f86a40b27b0 con 0x7f86a40b1b20 2023-09-23T03:17:45.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.384+0000 7f86a9325640 1 -- 172.21.15.154:0/402819790 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86a40b27b0 con 0x7f86a40b1b20 2023-09-23T03:17:45.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.387+0000 7f868b7fe640 1 -- 172.21.15.154:0/402819790 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+18598 (secure 0 0 0) 0x7f86a40b27b0 con 0x7f86a40b1b20 2023-09-23T03:17:45.389 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:45.389 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":41,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:17:23.221346+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:17:08.933690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-09-23T03:17:10.141711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data_extra","create_time":"2023-09-23T03:17:21.021560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:17:45.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.390+0000 7f86a9325640 1 -- 172.21.15.154:0/402819790 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f869006af30 msgr2=0x7f869006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:45.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.390+0000 7f86a9325640 1 --2- 172.21.15.154:0/402819790 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f869006af30 0x7f869006d3f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8694035000 tx=0x7f8694008030 comp rx=0 tx=0).stop 2023-09-23T03:17:45.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.390+0000 7f86a9325640 1 -- 172.21.15.154:0/402819790 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86a40b1b20 msgr2=0x7f86a40c3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:45.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.390+0000 7f86a9325640 1 --2- 172.21.15.154:0/402819790 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86a40b1b20 0x7f86a40c3b10 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f869800c430 tx=0x7f869800c8f0 comp rx=0 tx=0).stop 2023-09-23T03:17:45.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.390+0000 7f86a9325640 1 -- 172.21.15.154:0/402819790 shutdown_connections 2023-09-23T03:17:45.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.390+0000 7f86a9325640 1 --2- 172.21.15.154:0/402819790 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f869006af30 0x7f869006d3f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:45.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.390+0000 7f86a9325640 1 --2- 172.21.15.154:0/402819790 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86a40cf040 0x7f86a40c55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:45.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.390+0000 7f86a9325640 1 --2- 172.21.15.154:0/402819790 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86a40b1b20 0x7f86a40c3b10 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:45.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.390+0000 7f86a9325640 1 --2- 172.21.15.154:0/402819790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86a40acce0 0x7f86a415b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:45.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.390+0000 7f86a9325640 1 -- 172.21.15.154:0/402819790 >> 172.21.15.154:0/402819790 conn(0x7f86a406a0f0 msgr2=0x7f86a40ce900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:45.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.390+0000 7f86a9325640 1 -- 172.21.15.154:0/402819790 shutdown_connections 2023-09-23T03:17:45.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.390+0000 7f86a9325640 1 -- 172.21.15.154:0/402819790 wait complete. 2023-09-23T03:17:45.411 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:17:08.933690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:17:10.141711+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-09-23T03:17:21.021560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T03:17:45.411 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:17:45.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.629+0000 7fa76c505640 1 Processor -- start 2023-09-23T03:17:45.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.629+0000 7fa76c505640 1 -- start start 2023-09-23T03:17:45.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.631+0000 7fa76c505640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa7640c8d70 0x7fa76415c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:45.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.631+0000 7fa76c505640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa764071290 0x7fa7640ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:45.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.631+0000 7fa76c505640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7640c9d20 con 0x7fa7640c9150 2023-09-23T03:17:45.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.631+0000 7fa76c505640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa764071850 con 0x7fa764071290 2023-09-23T03:17:45.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.631+0000 7fa76c505640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa764071b30 con 0x7fa7640c8d70 2023-09-23T03:17:45.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.631+0000 7fa76aaa2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa764071290 0x7fa7640ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:45.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.631+0000 7fa76aaa2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa764071290 0x7fa7640ca1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48866/0 (socket says 172.21.15.154:48866) 2023-09-23T03:17:45.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.631+0000 7fa76aaa2640 1 -- 172.21.15.154:0/1936031574 learned_addr learned my addr 172.21.15.154:0/1936031574 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:45.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.632+0000 7fa76a2a1640 1 --2- 172.21.15.154:0/1936031574 >> v2:172.21.15.190:3300/0 conn(0x7fa7640c8d70 0x7fa76415c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:45.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.632+0000 7fa76aaa2640 1 -- 172.21.15.154:0/1936031574 >> v2:172.21.15.190:3300/0 conn(0x7fa7640c8d70 msgr2=0x7fa76415c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:45.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.632+0000 7fa76aaa2640 1 --2- 172.21.15.154:0/1936031574 >> v2:172.21.15.190:3300/0 conn(0x7fa7640c8d70 0x7fa76415c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:45.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.632+0000 7fa76aaa2640 1 -- 172.21.15.154:0/1936031574 >> v1:172.21.15.154:6789/0 conn(0x7fa7640c9150 legacy=0x7fa76415cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:45.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.632+0000 7fa76aaa2640 1 -- 172.21.15.154:0/1936031574 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa764071e10 con 0x7fa764071290 2023-09-23T03:17:45.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.632+0000 7fa76929f640 1 -- 172.21.15.154:0/1936031574 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 627434555 0 0) 0x7fa7640c9d20 con 0x7fa7640c9150 2023-09-23T03:17:45.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.632+0000 7fa76a2a1640 1 --2- 172.21.15.154:0/1936031574 >> v2:172.21.15.190:3300/0 conn(0x7fa7640c8d70 0x7fa76415c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:45.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.633+0000 7fa76aaa2640 1 --2- 172.21.15.154:0/1936031574 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa764071290 0x7fa7640ca1b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa76406b790 tx=0x7fa75c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc3de2e114ffc7f7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:45.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.633+0000 7fa76929f640 1 -- 172.21.15.154:0/1936031574 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c02cb20 con 0x7fa764071290 2023-09-23T03:17:45.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.633+0000 7fa76929f640 1 -- 172.21.15.154:0/1936031574 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa75c02cc80 con 0x7fa764071290 2023-09-23T03:17:45.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.633+0000 7fa76929f640 1 -- 172.21.15.154:0/1936031574 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c0343f0 con 0x7fa764071290 2023-09-23T03:17:45.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.634+0000 7fa76c505640 1 -- 172.21.15.154:0/1936031574 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa764071290 msgr2=0x7fa7640ca1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:45.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.634+0000 7fa76c505640 1 --2- 172.21.15.154:0/1936031574 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa764071290 0x7fa7640ca1b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa76406b790 tx=0x7fa75c02bf80 comp rx=0 tx=0).stop 2023-09-23T03:17:45.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.634+0000 7fa76c505640 1 -- 172.21.15.154:0/1936031574 shutdown_connections 2023-09-23T03:17:45.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.634+0000 7fa76c505640 1 --2- 172.21.15.154:0/1936031574 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa764071290 0x7fa7640ca1b0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:45.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.634+0000 7fa76c505640 1 --2- 172.21.15.154:0/1936031574 >> v2:172.21.15.190:3300/0 conn(0x7fa7640c8d70 0x7fa76415c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:45.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.634+0000 7fa76c505640 1 -- 172.21.15.154:0/1936031574 >> 172.21.15.154:0/1936031574 conn(0x7fa76406c010 msgr2=0x7fa76406c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:45.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.634+0000 7fa76c505640 1 -- 172.21.15.154:0/1936031574 shutdown_connections 2023-09-23T03:17:45.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.634+0000 7fa76c505640 1 -- 172.21.15.154:0/1936031574 wait complete. 2023-09-23T03:17:45.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.636+0000 7fa76c505640 1 Processor -- start 2023-09-23T03:17:45.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.636+0000 7fa76c505640 1 -- start start 2023-09-23T03:17:45.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.637+0000 7fa76c505640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa764071290 0x7fa7640c7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:45.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.637+0000 7fa76c505640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7640c8d70 0x7fa7641580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:45.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.637+0000 7fa76c505640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7640c9150 0x7fa764154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:45.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.637+0000 7fa76c505640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa764158af0 con 0x7fa7640c8d70 2023-09-23T03:17:45.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.637+0000 7fa76c505640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa764158dd0 con 0x7fa7640c9150 2023-09-23T03:17:45.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.637+0000 7fa76c505640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7641590b0 con 0x7fa764071290 2023-09-23T03:17:45.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.637+0000 7fa769aa0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7640c8d70 0x7fa7641580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:45.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.637+0000 7fa769aa0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7640c8d70 0x7fa7641580f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48880/0 (socket says 172.21.15.154:48880) 2023-09-23T03:17:45.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.637+0000 7fa769aa0640 1 -- 172.21.15.154:0/4263208916 learned_addr learned my addr 172.21.15.154:0/4263208916 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:45.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.637+0000 7fa76a2a1640 1 --2- 172.21.15.154:0/4263208916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa764071290 0x7fa7640c7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:45.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.637+0000 7fa769aa0640 1 -- 172.21.15.154:0/4263208916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa764071290 msgr2=0x7fa7640c7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:45.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.637+0000 7fa769aa0640 1 --2- 172.21.15.154:0/4263208916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa764071290 0x7fa7640c7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:45.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.637+0000 7fa769aa0640 1 -- 172.21.15.154:0/4263208916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7640c9150 msgr2=0x7fa764154440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:45.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.637+0000 7fa769aa0640 1 --2- 172.21.15.154:0/4263208916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7640c9150 0x7fa764154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:45.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.637+0000 7fa769aa0640 1 -- 172.21.15.154:0/4263208916 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa75c009b30 con 0x7fa7640c8d70 2023-09-23T03:17:45.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.638+0000 7fa769aa0640 1 --2- 172.21.15.154:0/4263208916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7640c8d70 0x7fa7641580f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa76000c410 tx=0x7fa76000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:45.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.638+0000 7fa756ffd640 1 -- 172.21.15.154:0/4263208916 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7600035f0 con 0x7fa7640c8d70 2023-09-23T03:17:45.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.638+0000 7fa756ffd640 1 -- 172.21.15.154:0/4263208916 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa760003770 con 0x7fa7640c8d70 2023-09-23T03:17:45.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.638+0000 7fa756ffd640 1 -- 172.21.15.154:0/4263208916 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa760003b20 con 0x7fa7640c8d70 2023-09-23T03:17:45.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.638+0000 7fa76c505640 1 -- 172.21.15.154:0/4263208916 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7640c5700 con 0x7fa7640c8d70 2023-09-23T03:17:45.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.638+0000 7fa76c505640 1 -- 172.21.15.154:0/4263208916 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7640c8390 con 0x7fa7640c8d70 2023-09-23T03:17:45.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.639+0000 7fa76c505640 1 -- 172.21.15.154:0/4263208916 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7640c58f0 con 0x7fa7640c8d70 2023-09-23T03:17:45.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.640+0000 7fa756ffd640 1 -- 172.21.15.154:0/4263208916 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fa760003c80 con 0x7fa7640c8d70 2023-09-23T03:17:45.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.641+0000 7fa756ffd640 1 --2- 172.21.15.154:0/4263208916 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa74806af30 0x7fa74806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:45.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.641+0000 7fa756ffd640 1 -- 172.21.15.154:0/4263208916 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6433+0+0 (secure 0 0 0) 0x7fa760088f70 con 0x7fa7640c8d70 2023-09-23T03:17:45.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.641+0000 7fa76a2a1640 1 --2- 172.21.15.154:0/4263208916 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa74806af30 0x7fa74806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:45.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.642+0000 7fa76a2a1640 1 --2- 172.21.15.154:0/4263208916 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa74806af30 0x7fa74806d3f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa764165bc0 tx=0x7fa758008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:45.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.646+0000 7fa756ffd640 1 -- 172.21.15.154:0/4263208916 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fa760053c30 con 0x7fa7640c8d70 2023-09-23T03:17:45.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.833+0000 7fa76c505640 1 -- 172.21.15.154:0/4263208916 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa7640c8390 con 0x7fa7640c8d70 2023-09-23T03:17:45.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.833+0000 7fa756ffd640 1 -- 172.21.15.154:0/4263208916 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa7640c8390 con 0x7fa7640c8d70 2023-09-23T03:17:45.835 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:17:45.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.836+0000 7fa76c505640 1 -- 172.21.15.154:0/4263208916 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa74806af30 msgr2=0x7fa74806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:45.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.836+0000 7fa76c505640 1 --2- 172.21.15.154:0/4263208916 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa74806af30 0x7fa74806d3f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa764165bc0 tx=0x7fa758008040 comp rx=0 tx=0).stop 2023-09-23T03:17:45.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.836+0000 7fa76c505640 1 -- 172.21.15.154:0/4263208916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7640c8d70 msgr2=0x7fa7641580f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:45.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.836+0000 7fa76c505640 1 --2- 172.21.15.154:0/4263208916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7640c8d70 0x7fa7641580f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa76000c410 tx=0x7fa76000c8d0 comp rx=0 tx=0).stop 2023-09-23T03:17:45.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.836+0000 7fa76c505640 1 -- 172.21.15.154:0/4263208916 shutdown_connections 2023-09-23T03:17:45.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.836+0000 7fa76c505640 1 --2- 172.21.15.154:0/4263208916 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa74806af30 0x7fa74806d3f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:45.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.836+0000 7fa76c505640 1 --2- 172.21.15.154:0/4263208916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7640c9150 0x7fa764154440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:45.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.836+0000 7fa76c505640 1 --2- 172.21.15.154:0/4263208916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7640c8d70 0x7fa7641580f0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:45.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.836+0000 7fa76c505640 1 --2- 172.21.15.154:0/4263208916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa764071290 0x7fa7640c7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:45.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.836+0000 7fa76c505640 1 -- 172.21.15.154:0/4263208916 >> 172.21.15.154:0/4263208916 conn(0x7fa76406c010 msgr2=0x7fa7640cb520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:45.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.836+0000 7fa76c505640 1 -- 172.21.15.154:0/4263208916 shutdown_connections 2023-09-23T03:17:45.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:45.836+0000 7fa76c505640 1 -- 172.21.15.154:0/4263208916 wait complete. 2023-09-23T03:17:45.856 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:17:46.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.069+0000 7f20829b6640 1 Processor -- start 2023-09-23T03:17:46.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.070+0000 7f20829b6640 1 -- start start 2023-09-23T03:17:46.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.071+0000 7f20829b6640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f207c0b5140 0x7f207c0c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:46.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.071+0000 7f20829b6640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f207c069cd0 0x7f207c06a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:46.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.071+0000 7f20829b6640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f207c0cced0 con 0x7f207c15a8d0 2023-09-23T03:17:46.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.071+0000 7f20829b6640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f207c06a650 con 0x7f207c069cd0 2023-09-23T03:17:46.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.071+0000 7f20829b6640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f207c0ca300 con 0x7f207c0b5140 2023-09-23T03:17:46.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.071+0000 7f2080f53640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f207c069cd0 0x7f207c06a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:46.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.072+0000 7f2080f53640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f207c069cd0 0x7f207c06a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48890/0 (socket says 172.21.15.154:48890) 2023-09-23T03:17:46.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.072+0000 7f2080f53640 1 -- 172.21.15.154:0/4256713700 learned_addr learned my addr 172.21.15.154:0/4256713700 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:46.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.072+0000 7f2073fff640 1 --2- 172.21.15.154:0/4256713700 >> v2:172.21.15.190:3300/0 conn(0x7f207c0b5140 0x7f207c0c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:46.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.072+0000 7f2080f53640 1 -- 172.21.15.154:0/4256713700 >> v2:172.21.15.190:3300/0 conn(0x7f207c0b5140 msgr2=0x7f207c0c9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:46.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.072+0000 7f2080f53640 1 --2- 172.21.15.154:0/4256713700 >> v2:172.21.15.190:3300/0 conn(0x7f207c0b5140 0x7f207c0c9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:46.075 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.072+0000 7f2080f53640 1 -- 172.21.15.154:0/4256713700 >> v1:172.21.15.190:6789/0 conn(0x7f207c15a8d0 legacy=0x7f207c0cc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:46.075 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.072+0000 7f2080f53640 1 -- 172.21.15.154:0/4256713700 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f207c0ca5e0 con 0x7f207c069cd0 2023-09-23T03:17:46.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.073+0000 7f2080f53640 1 --2- 172.21.15.154:0/4256713700 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f207c069cd0 0x7f207c06a110 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f207c0cf420 tx=0x7f207402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4efde286c0765ef server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:46.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.073+0000 7f207b7fe640 1 -- 172.21.15.154:0/4256713700 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207402c9f0 con 0x7f207c069cd0 2023-09-23T03:17:46.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.073+0000 7f207b7fe640 1 -- 172.21.15.154:0/4256713700 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f207402cb50 con 0x7f207c069cd0 2023-09-23T03:17:46.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.074+0000 7f207b7fe640 1 -- 172.21.15.154:0/4256713700 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20740343f0 con 0x7f207c069cd0 2023-09-23T03:17:46.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.074+0000 7f20829b6640 1 -- 172.21.15.154:0/4256713700 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f207c069cd0 msgr2=0x7f207c06a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:46.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.074+0000 7f20829b6640 1 --2- 172.21.15.154:0/4256713700 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f207c069cd0 0x7f207c06a110 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f207c0cf420 tx=0x7f207402be50 comp rx=0 tx=0).stop 2023-09-23T03:17:46.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.074+0000 7f20829b6640 1 -- 172.21.15.154:0/4256713700 shutdown_connections 2023-09-23T03:17:46.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.074+0000 7f20829b6640 1 --2- 172.21.15.154:0/4256713700 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f207c069cd0 0x7f207c06a110 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:46.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.075+0000 7f20829b6640 1 --2- 172.21.15.154:0/4256713700 >> v2:172.21.15.190:3300/0 conn(0x7f207c0b5140 0x7f207c0c9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:46.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.075+0000 7f20829b6640 1 -- 172.21.15.154:0/4256713700 >> 172.21.15.154:0/4256713700 conn(0x7f207c06d120 msgr2=0x7f207c06d500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:46.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.075+0000 7f20829b6640 1 -- 172.21.15.154:0/4256713700 shutdown_connections 2023-09-23T03:17:46.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.075+0000 7f20829b6640 1 -- 172.21.15.154:0/4256713700 wait complete. 2023-09-23T03:17:46.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.076+0000 7f20829b6640 1 Processor -- start 2023-09-23T03:17:46.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.077+0000 7f20829b6640 1 -- start start 2023-09-23T03:17:46.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.077+0000 7f20829b6640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f207c069cd0 0x7f207c1f4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:46.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.077+0000 7f20829b6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f207c0b5140 0x7f207c0b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:46.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.078+0000 7f20829b6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f207c15a8d0 0x7f207c0b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:46.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.078+0000 7f20829b6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207c0b4140 con 0x7f207c15a8d0 2023-09-23T03:17:46.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.078+0000 7f20829b6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207c1f5d70 con 0x7f207c0b5140 2023-09-23T03:17:46.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.078+0000 7f20829b6640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f207c1f5440 con 0x7f207c069cd0 2023-09-23T03:17:46.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.078+0000 7f2080f53640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f207c15a8d0 0x7f207c0b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:46.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.078+0000 7f2080f53640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f207c15a8d0 0x7f207c0b3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48896/0 (socket says 172.21.15.154:48896) 2023-09-23T03:17:46.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.078+0000 7f2080f53640 1 -- 172.21.15.154:0/1103153867 learned_addr learned my addr 172.21.15.154:0/1103153867 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:46.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.078+0000 7f207bfff640 1 --2- 172.21.15.154:0/1103153867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f207c069cd0 0x7f207c1f4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:46.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.078+0000 7f2073fff640 1 --2- 172.21.15.154:0/1103153867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f207c0b5140 0x7f207c0b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:46.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.078+0000 7f2080f53640 1 -- 172.21.15.154:0/1103153867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f207c069cd0 msgr2=0x7f207c1f4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:46.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.078+0000 7f2080f53640 1 --2- 172.21.15.154:0/1103153867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f207c069cd0 0x7f207c1f4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:46.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.078+0000 7f2080f53640 1 -- 172.21.15.154:0/1103153867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f207c0b5140 msgr2=0x7f207c0b36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:46.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.078+0000 7f2080f53640 1 --2- 172.21.15.154:0/1103153867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f207c0b5140 0x7f207c0b36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:46.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.078+0000 7f2080f53640 1 -- 172.21.15.154:0/1103153867 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2074009b30 con 0x7f207c15a8d0 2023-09-23T03:17:46.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.078+0000 7f2073fff640 1 --2- 172.21.15.154:0/1103153867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f207c0b5140 0x7f207c0b36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:46.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.079+0000 7f2080f53640 1 --2- 172.21.15.154:0/1103153867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f207c15a8d0 0x7f207c0b3c00 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f2074036040 tx=0x7f2074004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:46.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.079+0000 7f20797fa640 1 -- 172.21.15.154:0/1103153867 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2074009510 con 0x7f207c15a8d0 2023-09-23T03:17:46.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.079+0000 7f20797fa640 1 -- 172.21.15.154:0/1103153867 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f207403e450 con 0x7f207c15a8d0 2023-09-23T03:17:46.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.079+0000 7f20797fa640 1 -- 172.21.15.154:0/1103153867 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207403e7e0 con 0x7f207c15a8d0 2023-09-23T03:17:46.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.079+0000 7f20829b6640 1 -- 172.21.15.154:0/1103153867 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f207c1f5720 con 0x7f207c15a8d0 2023-09-23T03:17:46.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.079+0000 7f20829b6640 1 -- 172.21.15.154:0/1103153867 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f207c1f4f40 con 0x7f207c15a8d0 2023-09-23T03:17:46.084 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.080+0000 7f20829b6640 1 -- 172.21.15.154:0/1103153867 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f207c04e920 con 0x7f207c15a8d0 2023-09-23T03:17:46.084 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.081+0000 7f20797fa640 1 -- 172.21.15.154:0/1103153867 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f207403e940 con 0x7f207c15a8d0 2023-09-23T03:17:46.084 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.082+0000 7f20797fa640 1 --2- 172.21.15.154:0/1103153867 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f205c06af30 0x7f205c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:46.084 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.082+0000 7f20797fa640 1 -- 172.21.15.154:0/1103153867 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f20740ac920 con 0x7f207c15a8d0 2023-09-23T03:17:46.084 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.082+0000 7f207bfff640 1 --2- 172.21.15.154:0/1103153867 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f205c06af30 0x7f205c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:46.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.083+0000 7f207bfff640 1 --2- 172.21.15.154:0/1103153867 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f205c06af30 0x7f205c06d3f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f206c003d90 tx=0x7f206c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:46.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.086+0000 7f20797fa640 1 -- 172.21.15.154:0/1103153867 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f20740775e0 con 0x7f207c15a8d0 2023-09-23T03:17:46.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.280+0000 7f20829b6640 1 -- 172.21.15.154:0/1103153867 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f207c0b21c0 con 0x7f207c15a8d0 2023-09-23T03:17:46.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.281+0000 7f20797fa640 1 -- 172.21.15.154:0/1103153867 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) v1 ==== 104+0+11 (secure 0 0 0) 0x7f207407ac00 con 0x7f207c15a8d0 2023-09-23T03:17:46.282 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:17:46.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.284+0000 7f20829b6640 1 -- 172.21.15.154:0/1103153867 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f205c06af30 msgr2=0x7f205c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:46.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.284+0000 7f20829b6640 1 --2- 172.21.15.154:0/1103153867 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f205c06af30 0x7f205c06d3f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f206c003d90 tx=0x7f206c009040 comp rx=0 tx=0).stop 2023-09-23T03:17:46.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.284+0000 7f20829b6640 1 -- 172.21.15.154:0/1103153867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f207c15a8d0 msgr2=0x7f207c0b3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:46.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.284+0000 7f20829b6640 1 --2- 172.21.15.154:0/1103153867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f207c15a8d0 0x7f207c0b3c00 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f2074036040 tx=0x7f2074004a40 comp rx=0 tx=0).stop 2023-09-23T03:17:46.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.284+0000 7f20829b6640 1 -- 172.21.15.154:0/1103153867 shutdown_connections 2023-09-23T03:17:46.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.284+0000 7f20829b6640 1 --2- 172.21.15.154:0/1103153867 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f205c06af30 0x7f205c06d3f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:46.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.284+0000 7f20829b6640 1 --2- 172.21.15.154:0/1103153867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f207c15a8d0 0x7f207c0b3c00 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:46.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.284+0000 7f20829b6640 1 --2- 172.21.15.154:0/1103153867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f207c0b5140 0x7f207c0b36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:46.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.284+0000 7f20829b6640 1 --2- 172.21.15.154:0/1103153867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f207c069cd0 0x7f207c1f4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:46.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.284+0000 7f20829b6640 1 -- 172.21.15.154:0/1103153867 >> 172.21.15.154:0/1103153867 conn(0x7f207c06d120 msgr2=0x7f207c0b2470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:46.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.284+0000 7f20829b6640 1 -- 172.21.15.154:0/1103153867 shutdown_connections 2023-09-23T03:17:46.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.284+0000 7f20829b6640 1 -- 172.21.15.154:0/1103153867 wait complete. 2023-09-23T03:17:46.304 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:17:46.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.516+0000 7ffb1a890640 1 Processor -- start 2023-09-23T03:17:46.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.517+0000 7ffb1a890640 1 -- start start 2023-09-23T03:17:46.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.518+0000 7ffb1a890640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ffb140ad830 0x7ffb14070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:46.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.518+0000 7ffb1a890640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb140712f0 con 0x7ffb140c9680 2023-09-23T03:17:46.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.518+0000 7ffb1a890640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb140715d0 con 0x7ffb140cb5d0 2023-09-23T03:17:46.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.518+0000 7ffb1a890640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb140718b0 con 0x7ffb140ad830 2023-09-23T03:17:46.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.519+0000 7ffb13fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ffb140ad830 0x7ffb14070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:46.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.519+0000 7ffb13fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ffb140ad830 0x7ffb14070c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48900/0 (socket says 172.21.15.154:48900) 2023-09-23T03:17:46.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.519+0000 7ffb13fff640 1 -- 172.21.15.154:0/41332378 learned_addr learned my addr 172.21.15.154:0/41332378 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:46.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.519+0000 7ffb0bfff640 1 --1- 172.21.15.154:0/41332378 >> v1:172.21.15.154:6789/0 conn(0x7ffb140c9680 0x7ffb140caec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:43548/0 (socket says 172.21.15.154:43548) 2023-09-23T03:17:46.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.519+0000 7ffb13fff640 1 -- 172.21.15.154:0/41332378 >> v1:172.21.15.190:6789/0 conn(0x7ffb140cb5d0 legacy=0x7ffb140aba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:46.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.519+0000 7ffb13fff640 1 -- 172.21.15.154:0/41332378 >> v1:172.21.15.154:6789/0 conn(0x7ffb140c9680 legacy=0x7ffb140caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:46.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.519+0000 7ffb13fff640 1 -- 172.21.15.154:0/41332378 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb14071b90 con 0x7ffb140ad830 2023-09-23T03:17:46.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.520+0000 7ffb13fff640 1 --2- 172.21.15.154:0/41332378 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb140ad830 0x7ffb14070c60 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ffb04009870 tx=0x7ffb0402b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4f007130119ca5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:46.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.520+0000 7ffb137fe640 1 -- 172.21.15.154:0/41332378 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb04030850 con 0x7ffb140ad830 2023-09-23T03:17:46.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.521+0000 7ffb137fe640 1 -- 172.21.15.154:0/41332378 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb040309b0 con 0x7ffb140ad830 2023-09-23T03:17:46.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.521+0000 7ffb137fe640 1 -- 172.21.15.154:0/41332378 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb04030d40 con 0x7ffb140ad830 2023-09-23T03:17:46.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.521+0000 7ffb1a890640 1 -- 172.21.15.154:0/41332378 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb140ad830 msgr2=0x7ffb14070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:46.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.521+0000 7ffb1a890640 1 --2- 172.21.15.154:0/41332378 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb140ad830 0x7ffb14070c60 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ffb04009870 tx=0x7ffb0402b990 comp rx=0 tx=0).stop 2023-09-23T03:17:46.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.521+0000 7ffb1a890640 1 -- 172.21.15.154:0/41332378 shutdown_connections 2023-09-23T03:17:46.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.521+0000 7ffb1a890640 1 --2- 172.21.15.154:0/41332378 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb140ad830 0x7ffb14070c60 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:46.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.521+0000 7ffb1a890640 1 -- 172.21.15.154:0/41332378 >> 172.21.15.154:0/41332378 conn(0x7ffb1406c010 msgr2=0x7ffb1406c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:46.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.521+0000 7ffb1a890640 1 -- 172.21.15.154:0/41332378 shutdown_connections 2023-09-23T03:17:46.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.521+0000 7ffb1a890640 1 -- 172.21.15.154:0/41332378 wait complete. 2023-09-23T03:17:46.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.523+0000 7ffb1a890640 1 Processor -- start 2023-09-23T03:17:46.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.523+0000 7ffb1a890640 1 -- start start 2023-09-23T03:17:46.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.524+0000 7ffb1a890640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb140ad830 0x7ffb141f1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:46.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.524+0000 7ffb1a890640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb140c9680 0x7ffb140b3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:46.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.524+0000 7ffb1a890640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb140cb5d0 0x7ffb140b3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:46.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.524+0000 7ffb1a890640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb141f3510 con 0x7ffb140ad830 2023-09-23T03:17:46.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.524+0000 7ffb1a890640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb141f5c20 con 0x7ffb140cb5d0 2023-09-23T03:17:46.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.524+0000 7ffb1a890640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb141f5f00 con 0x7ffb140c9680 2023-09-23T03:17:46.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.524+0000 7ffb13fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb140ad830 0x7ffb141f1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:46.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.524+0000 7ffb13fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb140ad830 0x7ffb141f1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48904/0 (socket says 172.21.15.154:48904) 2023-09-23T03:17:46.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.524+0000 7ffb13fff640 1 -- 172.21.15.154:0/2758585041 learned_addr learned my addr 172.21.15.154:0/2758585041 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:46.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.525+0000 7ffb0bfff640 1 --2- 172.21.15.154:0/2758585041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb140c9680 0x7ffb140b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:46.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.525+0000 7ffb13fff640 1 -- 172.21.15.154:0/2758585041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb140c9680 msgr2=0x7ffb140b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:46.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.525+0000 7ffb13fff640 1 --2- 172.21.15.154:0/2758585041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb140c9680 0x7ffb140b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:46.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.525+0000 7ffb13fff640 1 -- 172.21.15.154:0/2758585041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb140cb5d0 msgr2=0x7ffb140b3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:46.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.525+0000 7ffb13fff640 1 --2- 172.21.15.154:0/2758585041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb140cb5d0 0x7ffb140b3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:46.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.525+0000 7ffb13fff640 1 -- 172.21.15.154:0/2758585041 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb040094d0 con 0x7ffb140ad830 2023-09-23T03:17:46.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.525+0000 7ffb0bfff640 1 --2- 172.21.15.154:0/2758585041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb140c9680 0x7ffb140b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:46.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.525+0000 7ffb13fff640 1 --2- 172.21.15.154:0/2758585041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb140ad830 0x7ffb141f1d20 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ffb04009870 tx=0x7ffb04004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:46.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.525+0000 7ffb117fa640 1 -- 172.21.15.154:0/2758585041 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb04030800 con 0x7ffb140ad830 2023-09-23T03:17:46.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.526+0000 7ffb117fa640 1 -- 172.21.15.154:0/2758585041 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb04043400 con 0x7ffb140ad830 2023-09-23T03:17:46.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.526+0000 7ffb117fa640 1 -- 172.21.15.154:0/2758585041 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb040437b0 con 0x7ffb140ad830 2023-09-23T03:17:46.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.526+0000 7ffb1a890640 1 -- 172.21.15.154:0/2758585041 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb141f61e0 con 0x7ffb140ad830 2023-09-23T03:17:46.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.526+0000 7ffb1a890640 1 -- 172.21.15.154:0/2758585041 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb141f69e0 con 0x7ffb140ad830 2023-09-23T03:17:46.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.527+0000 7ffb1a890640 1 -- 172.21.15.154:0/2758585041 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb141f66a0 con 0x7ffb140ad830 2023-09-23T03:17:46.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.528+0000 7ffb117fa640 1 -- 172.21.15.154:0/2758585041 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7ffb0402e070 con 0x7ffb140ad830 2023-09-23T03:17:46.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.528+0000 7ffb117fa640 1 --2- 172.21.15.154:0/2758585041 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffaf406af60 0x7ffaf406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:46.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.529+0000 7ffb117fa640 1 -- 172.21.15.154:0/2758585041 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6433+0+0 (secure 0 0 0) 0x7ffb0403e070 con 0x7ffb140ad830 2023-09-23T03:17:46.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.529+0000 7ffb0bfff640 1 --2- 172.21.15.154:0/2758585041 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffaf406af60 0x7ffaf406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:46.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.530+0000 7ffb0bfff640 1 --2- 172.21.15.154:0/2758585041 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffaf406af60 0x7ffaf406d420 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ffb1415f070 tx=0x7ffafc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:46.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.533+0000 7ffb117fa640 1 -- 172.21.15.154:0/2758585041 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7ffb040777c0 con 0x7ffb140ad830 2023-09-23T03:17:46.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.720+0000 7ffb1a890640 1 -- 172.21.15.154:0/2758585041 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffb1404e920 con 0x7ffb140ad830 2023-09-23T03:17:46.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.721+0000 7ffb117fa640 1 -- 172.21.15.154:0/2758585041 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) v1 ==== 100+0+11 (secure 0 0 0) 0x7ffb040340a0 con 0x7ffb140ad830 2023-09-23T03:17:46.723 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:17:46.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.724+0000 7ffb1a890640 1 -- 172.21.15.154:0/2758585041 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffaf406af60 msgr2=0x7ffaf406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:46.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.724+0000 7ffb1a890640 1 --2- 172.21.15.154:0/2758585041 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffaf406af60 0x7ffaf406d420 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ffb1415f070 tx=0x7ffafc009040 comp rx=0 tx=0).stop 2023-09-23T03:17:46.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.724+0000 7ffb1a890640 1 -- 172.21.15.154:0/2758585041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb140ad830 msgr2=0x7ffb141f1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:46.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.724+0000 7ffb1a890640 1 --2- 172.21.15.154:0/2758585041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb140ad830 0x7ffb141f1d20 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ffb04009870 tx=0x7ffb04004730 comp rx=0 tx=0).stop 2023-09-23T03:17:46.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.724+0000 7ffb1a890640 1 -- 172.21.15.154:0/2758585041 shutdown_connections 2023-09-23T03:17:46.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.724+0000 7ffb1a890640 1 --2- 172.21.15.154:0/2758585041 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffaf406af60 0x7ffaf406d420 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:46.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.724+0000 7ffb1a890640 1 --2- 172.21.15.154:0/2758585041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb140cb5d0 0x7ffb140b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:46.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.724+0000 7ffb1a890640 1 --2- 172.21.15.154:0/2758585041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb140c9680 0x7ffb140b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:46.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.724+0000 7ffb1a890640 1 --2- 172.21.15.154:0/2758585041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb140ad830 0x7ffb141f1d20 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:46.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.724+0000 7ffb1a890640 1 -- 172.21.15.154:0/2758585041 >> 172.21.15.154:0/2758585041 conn(0x7ffb1406c010 msgr2=0x7ffb140ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:46.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.724+0000 7ffb1a890640 1 -- 172.21.15.154:0/2758585041 shutdown_connections 2023-09-23T03:17:46.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.724+0000 7ffb1a890640 1 -- 172.21.15.154:0/2758585041 wait complete. 2023-09-23T03:17:46.742 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-09-23T03:17:46.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.958+0000 7f9948a90640 1 Processor -- start 2023-09-23T03:17:46.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.958+0000 7f9948a90640 1 -- start start 2023-09-23T03:17:46.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.960+0000 7f9948a90640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9944071290 0x7f994415cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:46.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.960+0000 7f9948a90640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99440c9d20 con 0x7f99440c8d70 2023-09-23T03:17:46.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.960+0000 7f9948a90640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9944071850 con 0x7f99440c9150 2023-09-23T03:17:46.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.960+0000 7f9948a90640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9944071b30 con 0x7f9944071290 2023-09-23T03:17:46.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.960+0000 7f994259c640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f99440c8d70 0x7f994415c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:43562/0 (socket says 172.21.15.154:43562) 2023-09-23T03:17:46.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.960+0000 7f994259c640 1 -- 172.21.15.154:0/3977280803 learned_addr learned my addr 172.21.15.154:0/3977280803 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:17:46.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.961+0000 7f9942d9d640 1 --2- 172.21.15.154:0/3977280803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9944071290 0x7f994415cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:46.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.961+0000 7f9941d9b640 1 -- 172.21.15.154:0/3977280803 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1901419780 0 0) 0x7f99440c9d20 con 0x7f99440c8d70 2023-09-23T03:17:46.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.961+0000 7f9941d9b640 1 -- 172.21.15.154:0/3977280803 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9930005730 con 0x7f99440c8d70 2023-09-23T03:17:46.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.961+0000 7f9941d9b640 1 -- 172.21.15.154:0/3977280803 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 860822298 0 0) 0x7f9930005730 con 0x7f99440c8d70 2023-09-23T03:17:46.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.961+0000 7f9941d9b640 1 -- 172.21.15.154:0/3977280803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9944071290 msgr2=0x7f994415cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:46.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.961+0000 7f9941d9b640 1 --2- 172.21.15.154:0/3977280803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9944071290 0x7f994415cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:46.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.961+0000 7f9941d9b640 1 -- 172.21.15.154:0/3977280803 >> v1:172.21.15.190:6789/0 conn(0x7f99440c9150 legacy=0x7f99440ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:46.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.961+0000 7f9941d9b640 1 -- 172.21.15.154:0/3977280803 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9944071e10 con 0x7f99440c8d70 2023-09-23T03:17:46.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.961+0000 7f9941d9b640 1 -- 172.21.15.154:0/3977280803 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f9934001690 con 0x7f99440c8d70 2023-09-23T03:17:46.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.961+0000 7f9941d9b640 1 -- 172.21.15.154:0/3977280803 >> v1:172.21.15.154:6789/0 conn(0x7f99440c8d70 legacy=0x7f994415c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:46.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.961+0000 7f9941d9b640 1 --2- 172.21.15.154:0/3977280803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9930005bb0 0x7f9930005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:46.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.961+0000 7f9941d9b640 1 --2- 172.21.15.154:0/3977280803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9944071290 0x7f99300069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:46.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.961+0000 7f9941d9b640 1 --2- 172.21.15.154:0/3977280803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9930006f10 0x7f9930007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:46.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.961+0000 7f9941d9b640 1 -- 172.21.15.154:0/3977280803 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9944071b30 con 0x7f9930006f10 2023-09-23T03:17:46.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.961+0000 7f9941d9b640 1 -- 172.21.15.154:0/3977280803 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993000dca0 con 0x7f9930005bb0 2023-09-23T03:17:46.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.961+0000 7f9941d9b640 1 -- 172.21.15.154:0/3977280803 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f993000de60 con 0x7f9944071290 2023-09-23T03:17:46.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.962+0000 7f9942d9d640 1 --2- 172.21.15.154:0/3977280803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9930006f10 0x7f9930007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:46.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.962+0000 7f994259c640 1 --2- 172.21.15.154:0/3977280803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9930005bb0 0x7f9930005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:46.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.962+0000 7f993bfff640 1 --2- 172.21.15.154:0/3977280803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9944071290 0x7f99300069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:46.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.962+0000 7f9942d9d640 1 -- 172.21.15.154:0/3977280803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9944071290 msgr2=0x7f99300069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:46.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.962+0000 7f9942d9d640 1 --2- 172.21.15.154:0/3977280803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9944071290 0x7f99300069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:46.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.962+0000 7f9942d9d640 1 -- 172.21.15.154:0/3977280803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9930005bb0 msgr2=0x7f9930005fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:46.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.962+0000 7f9942d9d640 1 --2- 172.21.15.154:0/3977280803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9930005bb0 0x7f9930005fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:46.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.962+0000 7f9942d9d640 1 -- 172.21.15.154:0/3977280803 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9930005730 con 0x7f9930006f10 2023-09-23T03:17:46.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.962+0000 7f993bfff640 1 --2- 172.21.15.154:0/3977280803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9944071290 0x7f99300069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:46.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.963+0000 7f9942d9d640 1 --2- 172.21.15.154:0/3977280803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9930006f10 0x7f9930007390 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f993c009f90 tx=0x7f9930007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e38477a892cd7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:46.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.964+0000 7f9941d9b640 1 -- 172.21.15.154:0/3977280803 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993c02cbd0 con 0x7f9930006f10 2023-09-23T03:17:46.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.964+0000 7f9941d9b640 1 -- 172.21.15.154:0/3977280803 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f993c00f040 con 0x7f9930006f10 2023-09-23T03:17:46.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.964+0000 7f9941d9b640 1 -- 172.21.15.154:0/3977280803 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993c034440 con 0x7f9930006f10 2023-09-23T03:17:46.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.964+0000 7f9948a90640 1 -- 172.21.15.154:0/3977280803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9930006f10 msgr2=0x7f9930007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:46.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.964+0000 7f9948a90640 1 --2- 172.21.15.154:0/3977280803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9930006f10 0x7f9930007390 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f993c009f90 tx=0x7f9930007a00 comp rx=0 tx=0).stop 2023-09-23T03:17:46.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.964+0000 7f9942d9d640 1 -- 172.21.15.154:0/3977280803 reap_dead start 2023-09-23T03:17:46.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.964+0000 7f9948a90640 1 -- 172.21.15.154:0/3977280803 shutdown_connections 2023-09-23T03:17:46.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.964+0000 7f9948a90640 1 -- 172.21.15.154:0/3977280803 >> 172.21.15.154:0/3977280803 conn(0x7f994406c010 msgr2=0x7f994406c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:46.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.964+0000 7f9948a90640 1 -- 172.21.15.154:0/3977280803 shutdown_connections 2023-09-23T03:17:46.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.964+0000 7f9948a90640 1 -- 172.21.15.154:0/3977280803 wait complete. 2023-09-23T03:17:46.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.966+0000 7f9948a90640 1 Processor -- start 2023-09-23T03:17:46.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.966+0000 7f9948a90640 1 -- start start 2023-09-23T03:17:46.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.967+0000 7f9948a90640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9930006f10 0x7f9944153870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:46.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.967+0000 7f9948a90640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99440b1760 0x7f99440b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:46.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.967+0000 7f9948a90640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99440c7110 0x7f99440c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:46.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.967+0000 7f9948a90640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99440c7b00 con 0x7f99440c7110 2023-09-23T03:17:46.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.967+0000 7f9948a90640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99441f24c0 con 0x7f9930006f10 2023-09-23T03:17:46.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.967+0000 7f9948a90640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99440b22d0 con 0x7f99440b1760 2023-09-23T03:17:46.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.968+0000 7f9942d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99440c7110 0x7f99440c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:46.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.968+0000 7f9942d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99440c7110 0x7f99440c75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48932/0 (socket says 172.21.15.154:48932) 2023-09-23T03:17:46.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.968+0000 7f9942d9d640 1 -- 172.21.15.154:0/642692324 learned_addr learned my addr 172.21.15.154:0/642692324 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:46.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.968+0000 7f9942d9d640 1 -- 172.21.15.154:0/642692324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99440b1760 msgr2=0x7f99440b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:46.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.968+0000 7f9942d9d640 1 --2- 172.21.15.154:0/642692324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99440b1760 0x7f99440b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:46.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.968+0000 7f9942d9d640 1 -- 172.21.15.154:0/642692324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9930006f10 msgr2=0x7f9944153870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:46.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.968+0000 7f9942d9d640 1 --2- 172.21.15.154:0/642692324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9930006f10 0x7f9944153870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:46.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.968+0000 7f9942d9d640 1 -- 172.21.15.154:0/642692324 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99441ef340 con 0x7f99440c7110 2023-09-23T03:17:46.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.968+0000 7f9942d9d640 1 --2- 172.21.15.154:0/642692324 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99440c7110 0x7f99440c75c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f993c037040 tx=0x7f993c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:46.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.969+0000 7f993affd640 1 -- 172.21.15.154:0/642692324 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993c043070 con 0x7f99440c7110 2023-09-23T03:17:46.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.969+0000 7f993affd640 1 -- 172.21.15.154:0/642692324 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f993c02ce30 con 0x7f99440c7110 2023-09-23T03:17:46.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.969+0000 7f993affd640 1 -- 172.21.15.154:0/642692324 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993c034d10 con 0x7f99440c7110 2023-09-23T03:17:46.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.969+0000 7f9948a90640 1 -- 172.21.15.154:0/642692324 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99441ef620 con 0x7f99440c7110 2023-09-23T03:17:46.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.969+0000 7f9948a90640 1 -- 172.21.15.154:0/642692324 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99441efbe0 con 0x7f99440c7110 2023-09-23T03:17:46.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.970+0000 7f9948a90640 1 -- 172.21.15.154:0/642692324 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99440b41e0 con 0x7f99440c7110 2023-09-23T03:17:46.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.971+0000 7f993affd640 1 -- 172.21.15.154:0/642692324 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f993c03e440 con 0x7f99440c7110 2023-09-23T03:17:46.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.972+0000 7f993affd640 1 --2- 172.21.15.154:0/642692324 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f992406af60 0x7f992406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:46.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.972+0000 7f993affd640 1 -- 172.21.15.154:0/642692324 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f993c0ab3f0 con 0x7f99440c7110 2023-09-23T03:17:46.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.972+0000 7f994259c640 1 --2- 172.21.15.154:0/642692324 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f992406af60 0x7f992406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:46.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.973+0000 7f994259c640 1 --2- 172.21.15.154:0/642692324 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f992406af60 0x7f992406d420 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9934001eb0 tx=0x7f9934000f70 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:46.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:46.976+0000 7f993affd640 1 -- 172.21.15.154:0/642692324 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f99440b41e0 con 0x7f99440c7110 2023-09-23T03:17:47.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.161+0000 7f9948a90640 1 -- 172.21.15.154:0/642692324 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7f99440b41e0 con 0x7f99440c7110 2023-09-23T03:17:47.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.162+0000 7f993affd640 1 -- 172.21.15.154:0/642692324 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v41) v1 ==== 106+0+11 (secure 0 0 0) 0x7f99440b41e0 con 0x7f99440c7110 2023-09-23T03:17:47.163 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:17:47.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.164+0000 7f9948a90640 1 -- 172.21.15.154:0/642692324 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f992406af60 msgr2=0x7f992406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:47.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.164+0000 7f9948a90640 1 --2- 172.21.15.154:0/642692324 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f992406af60 0x7f992406d420 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9934001eb0 tx=0x7f9934000f70 comp rx=0 tx=0).stop 2023-09-23T03:17:47.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.164+0000 7f9948a90640 1 -- 172.21.15.154:0/642692324 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99440c7110 msgr2=0x7f99440c75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:47.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.164+0000 7f9948a90640 1 --2- 172.21.15.154:0/642692324 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99440c7110 0x7f99440c75c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f993c037040 tx=0x7f993c009750 comp rx=0 tx=0).stop 2023-09-23T03:17:47.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.164+0000 7f9948a90640 1 -- 172.21.15.154:0/642692324 shutdown_connections 2023-09-23T03:17:47.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.164+0000 7f9948a90640 1 --2- 172.21.15.154:0/642692324 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f992406af60 0x7f992406d420 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:47.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.164+0000 7f9948a90640 1 --2- 172.21.15.154:0/642692324 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99440c7110 0x7f99440c75c0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:47.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.164+0000 7f9948a90640 1 --2- 172.21.15.154:0/642692324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99440b1760 0x7f99440b1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:47.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.164+0000 7f9948a90640 1 --2- 172.21.15.154:0/642692324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9930006f10 0x7f9944153870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:47.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.164+0000 7f9948a90640 1 -- 172.21.15.154:0/642692324 >> 172.21.15.154:0/642692324 conn(0x7f994406c010 msgr2=0x7f994415c570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:47.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.165+0000 7f9948a90640 1 -- 172.21.15.154:0/642692324 shutdown_connections 2023-09-23T03:17:47.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.165+0000 7f9948a90640 1 -- 172.21.15.154:0/642692324 wait complete. 2023-09-23T03:17:47.185 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-23T03:17:47.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.391+0000 7f753b039640 1 Processor -- start 2023-09-23T03:17:47.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.391+0000 7f753b039640 1 -- start start 2023-09-23T03:17:47.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.392+0000 7f753b039640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f75340ada30 0x7f75340ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:47.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.393+0000 7f753b039640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75340b2fe0 0x7f75340cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:47.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.393+0000 7f753b039640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f75340cef60 0x7f75340b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:47.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.393+0000 7f753b039640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75340ade10 con 0x7f75340cef60 2023-09-23T03:17:47.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.393+0000 7f753b039640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f753406ceb0 con 0x7f75340ada30 2023-09-23T03:17:47.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.393+0000 7f753b039640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75340cf3b0 con 0x7f75340b2fe0 2023-09-23T03:17:47.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.393+0000 7f75395d6640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f75340cef60 0x7f75340b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:47.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.393+0000 7f75395d6640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f75340cef60 0x7f75340b18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48944/0 (socket says 172.21.15.154:48944) 2023-09-23T03:17:47.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.393+0000 7f75395d6640 1 -- 172.21.15.154:0/1193916493 learned_addr learned my addr 172.21.15.154:0/1193916493 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:47.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.393+0000 7f752bfff640 1 --2- 172.21.15.154:0/1193916493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75340b2fe0 0x7f75340cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:47.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.393+0000 7f7538dd5640 1 --2- 172.21.15.154:0/1193916493 >> v2:172.21.15.190:3300/0 conn(0x7f75340ada30 0x7f75340ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:47.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.394+0000 7f75395d6640 1 -- 172.21.15.154:0/1193916493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75340b2fe0 msgr2=0x7f75340cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:47.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.394+0000 7f75395d6640 1 --2- 172.21.15.154:0/1193916493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75340b2fe0 0x7f75340cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:47.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.394+0000 7f75395d6640 1 -- 172.21.15.154:0/1193916493 >> v2:172.21.15.190:3300/0 conn(0x7f75340ada30 msgr2=0x7f75340ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:47.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.394+0000 7f75395d6640 1 --2- 172.21.15.154:0/1193916493 >> v2:172.21.15.190:3300/0 conn(0x7f75340ada30 0x7f75340ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:47.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.394+0000 7f75395d6640 1 -- 172.21.15.154:0/1193916493 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7534159c90 con 0x7f75340cef60 2023-09-23T03:17:47.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.395+0000 7f75395d6640 1 --2- 172.21.15.154:0/1193916493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75340cef60 0x7f75340b18f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f753406d990 tx=0x7f752c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10dc28ab8873d705 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:47.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.395+0000 7f752b7fe640 1 -- 172.21.15.154:0/1193916493 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752c031860 con 0x7f75340cef60 2023-09-23T03:17:47.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.395+0000 7f752b7fe640 1 -- 172.21.15.154:0/1193916493 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f752c0319c0 con 0x7f75340cef60 2023-09-23T03:17:47.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.395+0000 7f752b7fe640 1 -- 172.21.15.154:0/1193916493 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752c031d50 con 0x7f75340cef60 2023-09-23T03:17:47.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.396+0000 7f753b039640 1 -- 172.21.15.154:0/1193916493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75340cef60 msgr2=0x7f75340b18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:47.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.396+0000 7f753b039640 1 --2- 172.21.15.154:0/1193916493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75340cef60 0x7f75340b18f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f753406d990 tx=0x7f752c02b900 comp rx=0 tx=0).stop 2023-09-23T03:17:47.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.396+0000 7f753b039640 1 -- 172.21.15.154:0/1193916493 shutdown_connections 2023-09-23T03:17:47.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.396+0000 7f753b039640 1 --2- 172.21.15.154:0/1193916493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75340cef60 0x7f75340b18f0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:47.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.396+0000 7f753b039640 1 --2- 172.21.15.154:0/1193916493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75340b2fe0 0x7f75340cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:47.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.396+0000 7f753b039640 1 --2- 172.21.15.154:0/1193916493 >> v2:172.21.15.190:3300/0 conn(0x7f75340ada30 0x7f75340ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:47.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.396+0000 7f753b039640 1 -- 172.21.15.154:0/1193916493 >> 172.21.15.154:0/1193916493 conn(0x7f753406a0f0 msgr2=0x7f753406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:47.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.396+0000 7f753b039640 1 -- 172.21.15.154:0/1193916493 shutdown_connections 2023-09-23T03:17:47.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.396+0000 7f753b039640 1 -- 172.21.15.154:0/1193916493 wait complete. 2023-09-23T03:17:47.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.398+0000 7f753b039640 1 Processor -- start 2023-09-23T03:17:47.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.398+0000 7f753b039640 1 -- start start 2023-09-23T03:17:47.400 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.399+0000 7f753b039640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f75340ada30 0x7f75340c2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:47.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.399+0000 7f753b039640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75340b2fe0 0x7f75340c2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:47.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.399+0000 7f753b039640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75340cef60 0x7f7534096d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:47.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.399+0000 7f753b039640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7534095d20 con 0x7f75340b2fe0 2023-09-23T03:17:47.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.399+0000 7f753b039640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75340968a0 con 0x7f75340ada30 2023-09-23T03:17:47.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.399+0000 7f753b039640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7534095f70 con 0x7f75340cef60 2023-09-23T03:17:47.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.399+0000 7f752bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75340b2fe0 0x7f75340c2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:47.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.399+0000 7f752bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75340b2fe0 0x7f75340c2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48954/0 (socket says 172.21.15.154:48954) 2023-09-23T03:17:47.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.399+0000 7f752bfff640 1 -- 172.21.15.154:0/1741987008 learned_addr learned my addr 172.21.15.154:0/1741987008 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:47.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.400+0000 7f752bfff640 1 -- 172.21.15.154:0/1741987008 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75340cef60 msgr2=0x7f7534096d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:47.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.400+0000 7f752bfff640 1 --2- 172.21.15.154:0/1741987008 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75340cef60 0x7f7534096d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:47.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.400+0000 7f752bfff640 1 -- 172.21.15.154:0/1741987008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f75340ada30 msgr2=0x7f75340c2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:47.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.400+0000 7f752bfff640 1 --2- 172.21.15.154:0/1741987008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f75340ada30 0x7f75340c2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:47.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.400+0000 7f752bfff640 1 -- 172.21.15.154:0/1741987008 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f752c0094d0 con 0x7f75340b2fe0 2023-09-23T03:17:47.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.400+0000 7f752bfff640 1 --2- 172.21.15.154:0/1741987008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75340b2fe0 0x7f75340c2d70 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f752400d800 tx=0x7f752400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:47.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.400+0000 7f75297fa640 1 -- 172.21.15.154:0/1741987008 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752400b6c0 con 0x7f75340b2fe0 2023-09-23T03:17:47.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.400+0000 7f75297fa640 1 -- 172.21.15.154:0/1741987008 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f752400b820 con 0x7f75340b2fe0 2023-09-23T03:17:47.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.400+0000 7f75297fa640 1 -- 172.21.15.154:0/1741987008 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752400bbb0 con 0x7f75340b2fe0 2023-09-23T03:17:47.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.400+0000 7f753b039640 1 -- 172.21.15.154:0/1741987008 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7534096250 con 0x7f75340b2fe0 2023-09-23T03:17:47.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.400+0000 7f753b039640 1 -- 172.21.15.154:0/1741987008 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7534153ee0 con 0x7f75340b2fe0 2023-09-23T03:17:47.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.401+0000 7f753b039640 1 -- 172.21.15.154:0/1741987008 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7534073eb0 con 0x7f75340b2fe0 2023-09-23T03:17:47.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.402+0000 7f75297fa640 1 -- 172.21.15.154:0/1741987008 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f752400bd10 con 0x7f75340b2fe0 2023-09-23T03:17:47.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.403+0000 7f75297fa640 1 --2- 172.21.15.154:0/1741987008 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f752006af60 0x7f752006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:47.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.403+0000 7f75297fa640 1 -- 172.21.15.154:0/1741987008 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f752408a0c0 con 0x7f75340b2fe0 2023-09-23T03:17:47.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.403+0000 7f7538dd5640 1 --2- 172.21.15.154:0/1741987008 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f752006af60 0x7f752006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:47.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.404+0000 7f7538dd5640 1 --2- 172.21.15.154:0/1741987008 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f752006af60 0x7f752006d420 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7530009540 tx=0x7f7530009450 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:47.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.408+0000 7f75297fa640 1 -- 172.21.15.154:0/1741987008 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f7524054d80 con 0x7f75340b2fe0 2023-09-23T03:17:47.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.608+0000 7f753b039640 1 -- 172.21.15.154:0/1741987008 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f75340969e0 con 0x7f75340b2fe0 2023-09-23T03:17:47.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.609+0000 7f75297fa640 1 -- 172.21.15.154:0/1741987008 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v41) v1 ==== 80+0+0 (secure 0 0 0) 0x7f75240583a0 con 0x7f75340b2fe0 2023-09-23T03:17:47.610 INFO:teuthology.orchestra.run.smithi154.stderr:listed 0 entries 2023-09-23T03:17:47.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.611+0000 7f753b039640 1 -- 172.21.15.154:0/1741987008 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f752006af60 msgr2=0x7f752006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:47.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.611+0000 7f753b039640 1 --2- 172.21.15.154:0/1741987008 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f752006af60 0x7f752006d420 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7530009540 tx=0x7f7530009450 comp rx=0 tx=0).stop 2023-09-23T03:17:47.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.611+0000 7f753b039640 1 -- 172.21.15.154:0/1741987008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75340b2fe0 msgr2=0x7f75340c2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:47.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.611+0000 7f753b039640 1 --2- 172.21.15.154:0/1741987008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75340b2fe0 0x7f75340c2d70 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f752400d800 tx=0x7f752400dcc0 comp rx=0 tx=0).stop 2023-09-23T03:17:47.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.612+0000 7f753b039640 1 -- 172.21.15.154:0/1741987008 shutdown_connections 2023-09-23T03:17:47.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.612+0000 7f753b039640 1 --2- 172.21.15.154:0/1741987008 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f752006af60 0x7f752006d420 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:47.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.612+0000 7f753b039640 1 --2- 172.21.15.154:0/1741987008 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f75340cef60 0x7f7534096d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:47.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.612+0000 7f753b039640 1 --2- 172.21.15.154:0/1741987008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f75340b2fe0 0x7f75340c2d70 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:47.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.612+0000 7f753b039640 1 --2- 172.21.15.154:0/1741987008 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f75340ada30 0x7f75340c2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:47.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.612+0000 7f753b039640 1 -- 172.21.15.154:0/1741987008 >> 172.21.15.154:0/1741987008 conn(0x7f753406a0f0 msgr2=0x7f7534070b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:47.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.612+0000 7f753b039640 1 -- 172.21.15.154:0/1741987008 shutdown_connections 2023-09-23T03:17:47.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:47.612+0000 7f753b039640 1 -- 172.21.15.154:0/1741987008 wait complete. 2023-09-23T03:17:47.630 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi154.front.sepia.ceph.com... 2023-09-23T03:17:47.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:17:47.631 DEBUG:teuthology.orchestra.run.smithi154:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-09-23T03:17:47.715 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:ceph-fuse[103090]: fuse finished with error 0 and tester_r 0 2023-09-23T03:17:47.728 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:17:53.830 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2023-09-23T03:17:53.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:17:53.831 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-09-23T03:17:53.893 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:17:54.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.101+0000 7f9f57190640 1 Processor -- start 2023-09-23T03:17:54.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.101+0000 7f9f57190640 1 -- start start 2023-09-23T03:17:54.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.102+0000 7f9f57190640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9f500acce0 0x7f9f5015ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:54.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.102+0000 7f9f57190640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9f500b1b20 0x7f9f500ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:54.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.102+0000 7f9f57190640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f500cee70 0x7f9f5006cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:54.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.102+0000 7f9f57190640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f500b2e90 con 0x7f9f500acce0 2023-09-23T03:17:54.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.102+0000 7f9f57190640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f500cf300 con 0x7f9f500b1b20 2023-09-23T03:17:54.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.102+0000 7f9f57190640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5015ac30 con 0x7f9f500cee70 2023-09-23T03:17:54.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.102+0000 7f9f54f2c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9f500acce0 0x7f9f5015ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:54.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.102+0000 7f9f54f2c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9f500acce0 0x7f9f5015ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60040/0 (socket says 172.21.15.154:60040) 2023-09-23T03:17:54.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.102+0000 7f9f54f2c640 1 -- 172.21.15.154:0/1195137334 learned_addr learned my addr 172.21.15.154:0/1195137334 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:54.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.103+0000 7f9f5572d640 1 --2- 172.21.15.154:0/1195137334 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f500cee70 0x7f9f5006cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:54.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.103+0000 7f9f4ffff640 1 --2- 172.21.15.154:0/1195137334 >> v2:172.21.15.190:3300/0 conn(0x7f9f500b1b20 0x7f9f500ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:54.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.103+0000 7f9f54f2c640 1 -- 172.21.15.154:0/1195137334 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f500cee70 msgr2=0x7f9f5006cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:54.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.103+0000 7f9f54f2c640 1 --2- 172.21.15.154:0/1195137334 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f500cee70 0x7f9f5006cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:54.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.103+0000 7f9f54f2c640 1 -- 172.21.15.154:0/1195137334 >> v2:172.21.15.190:3300/0 conn(0x7f9f500b1b20 msgr2=0x7f9f500ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:54.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.103+0000 7f9f54f2c640 1 --2- 172.21.15.154:0/1195137334 >> v2:172.21.15.190:3300/0 conn(0x7f9f500b1b20 0x7f9f500ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:54.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.103+0000 7f9f54f2c640 1 -- 172.21.15.154:0/1195137334 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f500b1f70 con 0x7f9f500acce0 2023-09-23T03:17:54.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.104+0000 7f9f54f2c640 1 --2- 172.21.15.154:0/1195137334 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f500acce0 0x7f9f5015ed60 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f9f500c9430 tx=0x7f9f4402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c4614cf08e84f7d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:54.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.104+0000 7f9f4f7fe640 1 -- 172.21.15.154:0/1195137334 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4402e070 con 0x7f9f500acce0 2023-09-23T03:17:54.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.104+0000 7f9f4f7fe640 1 -- 172.21.15.154:0/1195137334 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f44004030 con 0x7f9f500acce0 2023-09-23T03:17:54.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.104+0000 7f9f4f7fe640 1 -- 172.21.15.154:0/1195137334 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f44004420 con 0x7f9f500acce0 2023-09-23T03:17:54.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.105+0000 7f9f57190640 1 -- 172.21.15.154:0/1195137334 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f500acce0 msgr2=0x7f9f5015ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:54.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.105+0000 7f9f57190640 1 --2- 172.21.15.154:0/1195137334 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f500acce0 0x7f9f5015ed60 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f9f500c9430 tx=0x7f9f4402b840 comp rx=0 tx=0).stop 2023-09-23T03:17:54.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.105+0000 7f9f57190640 1 -- 172.21.15.154:0/1195137334 shutdown_connections 2023-09-23T03:17:54.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.105+0000 7f9f57190640 1 --2- 172.21.15.154:0/1195137334 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f500cee70 0x7f9f5006cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:54.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.105+0000 7f9f57190640 1 --2- 172.21.15.154:0/1195137334 >> v2:172.21.15.190:3300/0 conn(0x7f9f500b1b20 0x7f9f500ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:54.108 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.105+0000 7f9f57190640 1 --2- 172.21.15.154:0/1195137334 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f500acce0 0x7f9f5015ed60 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:54.108 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.105+0000 7f9f57190640 1 -- 172.21.15.154:0/1195137334 >> 172.21.15.154:0/1195137334 conn(0x7f9f5006a0f0 msgr2=0x7f9f5006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:54.108 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.105+0000 7f9f57190640 1 -- 172.21.15.154:0/1195137334 shutdown_connections 2023-09-23T03:17:54.108 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.105+0000 7f9f57190640 1 -- 172.21.15.154:0/1195137334 wait complete. 2023-09-23T03:17:54.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.107+0000 7f9f57190640 1 Processor -- start 2023-09-23T03:17:54.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.107+0000 7f9f57190640 1 -- start start 2023-09-23T03:17:54.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.108+0000 7f9f57190640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f500acce0 0x7f9f50097560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:54.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.108+0000 7f9f57190640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f500b1b20 0x7f9f5014fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:54.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.108+0000 7f9f57190640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f500cee70 0x7f9f501500c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:54.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.108+0000 7f9f57190640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f50153db0 con 0x7f9f500cee70 2023-09-23T03:17:54.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.108+0000 7f9f57190640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f50097e60 con 0x7f9f500b1b20 2023-09-23T03:17:54.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.108+0000 7f9f57190640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f500c55e0 con 0x7f9f500acce0 2023-09-23T03:17:54.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.108+0000 7f9f5572d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f500cee70 0x7f9f501500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:54.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.108+0000 7f9f5572d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f500cee70 0x7f9f501500c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60054/0 (socket says 172.21.15.154:60054) 2023-09-23T03:17:54.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.108+0000 7f9f5572d640 1 -- 172.21.15.154:0/27325646 learned_addr learned my addr 172.21.15.154:0/27325646 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:54.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.108+0000 7f9f54f2c640 1 --2- 172.21.15.154:0/27325646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f500acce0 0x7f9f50097560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:54.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.108+0000 7f9f5572d640 1 -- 172.21.15.154:0/27325646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f500acce0 msgr2=0x7f9f50097560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:54.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.108+0000 7f9f5572d640 1 --2- 172.21.15.154:0/27325646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f500acce0 0x7f9f50097560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:54.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.108+0000 7f9f5572d640 1 -- 172.21.15.154:0/27325646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f500b1b20 msgr2=0x7f9f5014fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:54.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.108+0000 7f9f4ffff640 1 --2- 172.21.15.154:0/27325646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f500b1b20 0x7f9f5014fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:54.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.108+0000 7f9f5572d640 1 --2- 172.21.15.154:0/27325646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f500b1b20 0x7f9f5014fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:54.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.108+0000 7f9f5572d640 1 -- 172.21.15.154:0/27325646 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f44009460 con 0x7f9f500cee70 2023-09-23T03:17:54.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.108+0000 7f9f54f2c640 1 --2- 172.21.15.154:0/27325646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f500acce0 0x7f9f50097560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:54.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.109+0000 7f9f5572d640 1 --2- 172.21.15.154:0/27325646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f500cee70 0x7f9f501500c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f9f4800cc30 tx=0x7f9f4800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:54.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.109+0000 7f9f4d7fa640 1 -- 172.21.15.154:0/27325646 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4800ce40 con 0x7f9f500cee70 2023-09-23T03:17:54.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.109+0000 7f9f4d7fa640 1 -- 172.21.15.154:0/27325646 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f48003990 con 0x7f9f500cee70 2023-09-23T03:17:54.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.109+0000 7f9f57190640 1 -- 172.21.15.154:0/27325646 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f501531a0 con 0x7f9f500cee70 2023-09-23T03:17:54.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.109+0000 7f9f4d7fa640 1 -- 172.21.15.154:0/27325646 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f48003d20 con 0x7f9f500cee70 2023-09-23T03:17:54.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.109+0000 7f9f57190640 1 -- 172.21.15.154:0/27325646 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f50153760 con 0x7f9f500cee70 2023-09-23T03:17:54.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.110+0000 7f9f57190640 1 -- 172.21.15.154:0/27325646 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f50151670 con 0x7f9f500cee70 2023-09-23T03:17:54.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.111+0000 7f9f4d7fa640 1 -- 172.21.15.154:0/27325646 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f9f48003e80 con 0x7f9f500cee70 2023-09-23T03:17:54.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.112+0000 7f9f4d7fa640 1 --2- 172.21.15.154:0/27325646 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9f3406af30 0x7f9f3406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:54.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.112+0000 7f9f4d7fa640 1 -- 172.21.15.154:0/27325646 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f9f48089e90 con 0x7f9f500cee70 2023-09-23T03:17:54.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.112+0000 7f9f54f2c640 1 --2- 172.21.15.154:0/27325646 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9f3406af30 0x7f9f3406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:54.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.113+0000 7f9f54f2c640 1 --2- 172.21.15.154:0/27325646 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9f3406af30 0x7f9f3406d3f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9f44034000 tx=0x7f9f44008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:54.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.117+0000 7f9f4d7fa640 1 -- 172.21.15.154:0/27325646 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f9f50151670 con 0x7f9f500cee70 2023-09-23T03:17:54.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.348+0000 7f9f57190640 1 -- 172.21.15.154:0/27325646 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f50153f60 con 0x7f9f500cee70 2023-09-23T03:17:54.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.349+0000 7f9f4d7fa640 1 -- 172.21.15.154:0/27325646 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f9f50153f60 con 0x7f9f500cee70 2023-09-23T03:17:54.353 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:54.353 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4760,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/4034412019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4034412019},{"type":"v1","addr":"172.21.15.154:6836","nonce":4034412019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3736308401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3736308401},{"type":"v1","addr":"172.21.15.190:6835","nonce":3736308401}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4999,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/617287200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":617287200},{"type":"v1","addr":"172.21.15.154:6837","nonce":617287200}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:17:11.088637+0000","modified":"2023-09-23T03:17:21.679231+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5002},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5002":{"gid":5002,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.190:6834/2131436414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2131436414},{"type":"v1","addr":"172.21.15.190:6834","nonce":2131436414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-09-23T03:17:54.354 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 19 2023-09-23T03:17:54.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.352+0000 7f9f57190640 1 -- 172.21.15.154:0/27325646 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9f3406af30 msgr2=0x7f9f3406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:54.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.352+0000 7f9f57190640 1 --2- 172.21.15.154:0/27325646 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9f3406af30 0x7f9f3406d3f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9f44034000 tx=0x7f9f44008030 comp rx=0 tx=0).stop 2023-09-23T03:17:54.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.352+0000 7f9f57190640 1 -- 172.21.15.154:0/27325646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f500cee70 msgr2=0x7f9f501500c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:54.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.352+0000 7f9f57190640 1 --2- 172.21.15.154:0/27325646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f500cee70 0x7f9f501500c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f9f4800cc30 tx=0x7f9f4800a5c0 comp rx=0 tx=0).stop 2023-09-23T03:17:54.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.352+0000 7f9f57190640 1 -- 172.21.15.154:0/27325646 shutdown_connections 2023-09-23T03:17:54.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.352+0000 7f9f57190640 1 --2- 172.21.15.154:0/27325646 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9f3406af30 0x7f9f3406d3f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:54.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.352+0000 7f9f57190640 1 --2- 172.21.15.154:0/27325646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f500cee70 0x7f9f501500c0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:54.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.352+0000 7f9f57190640 1 --2- 172.21.15.154:0/27325646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f500b1b20 0x7f9f5014fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:54.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.352+0000 7f9f57190640 1 --2- 172.21.15.154:0/27325646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f500acce0 0x7f9f50097560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:54.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.352+0000 7f9f57190640 1 -- 172.21.15.154:0/27325646 >> 172.21.15.154:0/27325646 conn(0x7f9f5006a0f0 msgr2=0x7f9f500affe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:54.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.352+0000 7f9f57190640 1 -- 172.21.15.154:0/27325646 shutdown_connections 2023-09-23T03:17:54.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.352+0000 7f9f57190640 1 -- 172.21.15.154:0/27325646 wait complete. 2023-09-23T03:17:54.372 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2023-09-23T03:17:54.639 INFO:teuthology.orchestra.run.smithi190.stdout:{"message":"","return_code":0} 2023-09-23T03:17:54.640 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-09-23T03:17:54.640 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-09-23T03:17:54.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.860+0000 7fc2d3f2c640 1 Processor -- start 2023-09-23T03:17:54.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.861+0000 7fc2d3f2c640 1 -- start start 2023-09-23T03:17:54.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.862+0000 7fc2d3f2c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2cc0acce0 0x7fc2cc15ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:54.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.862+0000 7fc2d3f2c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2cc15ac30 con 0x7fc2cc0b1b20 2023-09-23T03:17:54.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.862+0000 7fc2d3f2c640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2cc0b1f70 con 0x7fc2cc0cf040 2023-09-23T03:17:54.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.862+0000 7fc2d3f2c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2cc0b2e90 con 0x7fc2cc0acce0 2023-09-23T03:17:54.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.862+0000 7fc2d14c7640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fc2cc0b1b20 0x7fc2cc0ce930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:50630/0 (socket says 172.21.15.154:50630) 2023-09-23T03:17:54.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.862+0000 7fc2d14c7640 1 -- 172.21.15.154:0/4277373173 learned_addr learned my addr 172.21.15.154:0/4277373173 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:17:54.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.863+0000 7fc2d1cc8640 1 --2- 172.21.15.154:0/4277373173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2cc0acce0 0x7fc2cc15ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:54.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.863+0000 7fc2d0cc6640 1 -- 172.21.15.154:0/4277373173 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 466525623 0 0) 0x7fc2bc001110 con 0x7fc2cc0b1b20 2023-09-23T03:17:54.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.863+0000 7fc2d0cc6640 1 -- 172.21.15.154:0/4277373173 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2b8003520 con 0x7fc2cc0b1b20 2023-09-23T03:17:54.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.863+0000 7fc2d0cc6640 1 -- 172.21.15.154:0/4277373173 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 820287270 0 0) 0x7fc2b8003520 con 0x7fc2cc0b1b20 2023-09-23T03:17:54.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.863+0000 7fc2d0cc6640 1 -- 172.21.15.154:0/4277373173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2cc0acce0 msgr2=0x7fc2cc15ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:54.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.863+0000 7fc2d0cc6640 1 --2- 172.21.15.154:0/4277373173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2cc0acce0 0x7fc2cc15ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:54.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.863+0000 7fc2d0cc6640 1 -- 172.21.15.154:0/4277373173 >> v1:172.21.15.190:6789/0 conn(0x7fc2cc0cf040 legacy=0x7fc2cc0ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:54.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.863+0000 7fc2d0cc6640 1 -- 172.21.15.154:0/4277373173 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2cc15f2a0 con 0x7fc2cc0b1b20 2023-09-23T03:17:54.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.863+0000 7fc2d0cc6640 1 -- 172.21.15.154:0/4277373173 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fc2bc001800 con 0x7fc2cc0b1b20 2023-09-23T03:17:54.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.863+0000 7fc2d0cc6640 1 -- 172.21.15.154:0/4277373173 >> v1:172.21.15.154:6789/0 conn(0x7fc2cc0b1b20 legacy=0x7fc2cc0ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:54.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.863+0000 7fc2d0cc6640 1 --2- 172.21.15.154:0/4277373173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2cc0acce0 0x7fc2b80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:54.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.863+0000 7fc2d0cc6640 1 --2- 172.21.15.154:0/4277373173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2b8003f70 0x7fc2b80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:54.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.863+0000 7fc2d0cc6640 1 --2- 172.21.15.154:0/4277373173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2b8004d00 0x7fc2b8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:54.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.863+0000 7fc2d0cc6640 1 -- 172.21.15.154:0/4277373173 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2cc0b2e90 con 0x7fc2b8003f70 2023-09-23T03:17:54.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.863+0000 7fc2d0cc6640 1 -- 172.21.15.154:0/4277373173 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b800bb70 con 0x7fc2b8004d00 2023-09-23T03:17:54.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.863+0000 7fc2d0cc6640 1 -- 172.21.15.154:0/4277373173 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b800bd30 con 0x7fc2cc0acce0 2023-09-23T03:17:54.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.864+0000 7fc2d14c7640 1 --2- 172.21.15.154:0/4277373173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2b8003f70 0x7fc2b80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:54.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.864+0000 7fc2d1cc8640 1 --2- 172.21.15.154:0/4277373173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2cc0acce0 0x7fc2b80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:54.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.864+0000 7fc2d24c9640 1 --2- 172.21.15.154:0/4277373173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2b8004d00 0x7fc2b8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:54.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.864+0000 7fc2d14c7640 1 -- 172.21.15.154:0/4277373173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2cc0acce0 msgr2=0x7fc2b80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:54.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.864+0000 7fc2d14c7640 1 --2- 172.21.15.154:0/4277373173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2cc0acce0 0x7fc2b80039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:54.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.864+0000 7fc2d14c7640 1 -- 172.21.15.154:0/4277373173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2b8004d00 msgr2=0x7fc2b8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:54.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.864+0000 7fc2d14c7640 1 --2- 172.21.15.154:0/4277373173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2b8004d00 0x7fc2b8005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:54.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.864+0000 7fc2d14c7640 1 -- 172.21.15.154:0/4277373173 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b8003520 con 0x7fc2b8003f70 2023-09-23T03:17:54.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.865+0000 7fc2d14c7640 1 --2- 172.21.15.154:0/4277373173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2b8003f70 0x7fc2b80047c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc007f80 tx=0x7fc2b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a074ac4215e242db server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:54.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.865+0000 7fc2d0cc6640 1 -- 172.21.15.154:0/4277373173 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc005ad0 con 0x7fc2b8003f70 2023-09-23T03:17:54.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.865+0000 7fc2d0cc6640 1 -- 172.21.15.154:0/4277373173 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2bc005c50 con 0x7fc2b8003f70 2023-09-23T03:17:54.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.865+0000 7fc2d0cc6640 1 -- 172.21.15.154:0/4277373173 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc007070 con 0x7fc2b8003f70 2023-09-23T03:17:54.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.866+0000 7fc2d3f2c640 1 -- 172.21.15.154:0/4277373173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2b8003f70 msgr2=0x7fc2b80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:54.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.866+0000 7fc2d3f2c640 1 --2- 172.21.15.154:0/4277373173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2b8003f70 0x7fc2b80047c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc007f80 tx=0x7fc2b80057f0 comp rx=0 tx=0).stop 2023-09-23T03:17:54.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.866+0000 7fc2d24c9640 1 -- 172.21.15.154:0/4277373173 reap_dead start 2023-09-23T03:17:54.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.866+0000 7fc2d3f2c640 1 -- 172.21.15.154:0/4277373173 shutdown_connections 2023-09-23T03:17:54.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.866+0000 7fc2d3f2c640 1 -- 172.21.15.154:0/4277373173 >> 172.21.15.154:0/4277373173 conn(0x7fc2cc06a0f0 msgr2=0x7fc2cc06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:54.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.866+0000 7fc2d3f2c640 1 -- 172.21.15.154:0/4277373173 shutdown_connections 2023-09-23T03:17:54.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.866+0000 7fc2d3f2c640 1 -- 172.21.15.154:0/4277373173 wait complete. 2023-09-23T03:17:54.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.868+0000 7fc2d3f2c640 1 Processor -- start 2023-09-23T03:17:54.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.868+0000 7fc2d3f2c640 1 -- start start 2023-09-23T03:17:54.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.869+0000 7fc2d3f2c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2b8003f70 0x7fc2cc1f1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:54.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.869+0000 7fc2d3f2c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2cc0af330 0x7fc2cc157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:54.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.869+0000 7fc2d3f2c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2cc157b70 0x7fc2cc0cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:54.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.869+0000 7fc2d3f2c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2cc0ae2d0 con 0x7fc2b8003f70 2023-09-23T03:17:54.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.869+0000 7fc2d3f2c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2cc0ad650 con 0x7fc2cc0af330 2023-09-23T03:17:54.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.869+0000 7fc2d3f2c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2cc0ad930 con 0x7fc2cc157b70 2023-09-23T03:17:54.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.869+0000 7fc2d1cc8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2b8003f70 0x7fc2cc1f1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:54.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.869+0000 7fc2d1cc8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2b8003f70 0x7fc2cc1f1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60070/0 (socket says 172.21.15.154:60070) 2023-09-23T03:17:54.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.869+0000 7fc2d1cc8640 1 -- 172.21.15.154:0/1000282288 learned_addr learned my addr 172.21.15.154:0/1000282288 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:54.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.869+0000 7fc2d24c9640 1 --2- 172.21.15.154:0/1000282288 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2cc157b70 0x7fc2cc0cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:54.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.870+0000 7fc2d14c7640 1 --2- 172.21.15.154:0/1000282288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2cc0af330 0x7fc2cc157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:54.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.870+0000 7fc2d1cc8640 1 -- 172.21.15.154:0/1000282288 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2cc157b70 msgr2=0x7fc2cc0cc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:54.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.870+0000 7fc2d1cc8640 1 --2- 172.21.15.154:0/1000282288 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2cc157b70 0x7fc2cc0cc9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:54.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.870+0000 7fc2d1cc8640 1 -- 172.21.15.154:0/1000282288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2cc0af330 msgr2=0x7fc2cc157630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:54.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.870+0000 7fc2d1cc8640 1 --2- 172.21.15.154:0/1000282288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2cc0af330 0x7fc2cc157630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:54.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.870+0000 7fc2d1cc8640 1 -- 172.21.15.154:0/1000282288 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2cc0adc10 con 0x7fc2b8003f70 2023-09-23T03:17:54.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.870+0000 7fc2d1cc8640 1 --2- 172.21.15.154:0/1000282288 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2b8003f70 0x7fc2cc1f1d40 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc2c80115b0 tx=0x7fc2c8011a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:54.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.870+0000 7fc2c27fc640 1 -- 172.21.15.154:0/1000282288 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8005da0 con 0x7fc2b8003f70 2023-09-23T03:17:54.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.870+0000 7fc2c27fc640 1 -- 172.21.15.154:0/1000282288 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c8006050 con 0x7fc2b8003f70 2023-09-23T03:17:54.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.870+0000 7fc2c27fc640 1 -- 172.21.15.154:0/1000282288 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c80063c0 con 0x7fc2b8003f70 2023-09-23T03:17:54.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.870+0000 7fc2d3f2c640 1 -- 172.21.15.154:0/1000282288 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2cc0adef0 con 0x7fc2b8003f70 2023-09-23T03:17:54.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.870+0000 7fc2d3f2c640 1 -- 172.21.15.154:0/1000282288 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2cc1f13c0 con 0x7fc2b8003f70 2023-09-23T03:17:54.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.871+0000 7fc2d3f2c640 1 -- 172.21.15.154:0/1000282288 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2cc1edd70 con 0x7fc2b8003f70 2023-09-23T03:17:54.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.872+0000 7fc2c27fc640 1 -- 172.21.15.154:0/1000282288 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fc2c8006520 con 0x7fc2b8003f70 2023-09-23T03:17:54.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.874+0000 7fc2c27fc640 1 --2- 172.21.15.154:0/1000282288 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc2b006af30 0x7fc2b006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:54.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.874+0000 7fc2c27fc640 1 -- 172.21.15.154:0/1000282288 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6433+0+0 (secure 0 0 0) 0x7fc2c808a180 con 0x7fc2b8003f70 2023-09-23T03:17:54.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.874+0000 7fc2d14c7640 1 --2- 172.21.15.154:0/1000282288 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc2b006af30 0x7fc2b006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:54.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.875+0000 7fc2d14c7640 1 --2- 172.21.15.154:0/1000282288 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc2b006af30 0x7fc2b006d3f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc034000 tx=0x7fc2bc007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:54.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:54.878+0000 7fc2c27fc640 1 -- 172.21.15.154:0/1000282288 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fc2c8054ed0 con 0x7fc2b8003f70 2023-09-23T03:17:55.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.086+0000 7fc2d3f2c640 1 -- 172.21.15.154:0/1000282288 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc2cc073eb0 con 0x7fc2b8003f70 2023-09-23T03:17:55.088 INFO:tasks.ceph.mon.a.smithi154.stderr:2023-09-23T03:17:55.087+0000 7f3df35a6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-23T03:17:55.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.091+0000 7fc2c27fc640 1 -- 172.21.15.154:0/1000282288 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v20) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc2c80584f0 con 0x7fc2b8003f70 2023-09-23T03:17:55.092 INFO:teuthology.orchestra.run.smithi154.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-09-23T03:17:55.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.094+0000 7fc2d3f2c640 1 -- 172.21.15.154:0/1000282288 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc2b006af30 msgr2=0x7fc2b006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:55.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.094+0000 7fc2d3f2c640 1 --2- 172.21.15.154:0/1000282288 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc2b006af30 0x7fc2b006d3f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc034000 tx=0x7fc2bc007000 comp rx=0 tx=0).stop 2023-09-23T03:17:55.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.094+0000 7fc2d3f2c640 1 -- 172.21.15.154:0/1000282288 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2b8003f70 msgr2=0x7fc2cc1f1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:55.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.094+0000 7fc2d3f2c640 1 --2- 172.21.15.154:0/1000282288 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2b8003f70 0x7fc2cc1f1d40 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc2c80115b0 tx=0x7fc2c8011a70 comp rx=0 tx=0).stop 2023-09-23T03:17:55.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.094+0000 7fc2d3f2c640 1 -- 172.21.15.154:0/1000282288 shutdown_connections 2023-09-23T03:17:55.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.094+0000 7fc2d3f2c640 1 --2- 172.21.15.154:0/1000282288 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc2b006af30 0x7fc2b006d3f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:55.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.094+0000 7fc2d3f2c640 1 --2- 172.21.15.154:0/1000282288 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2cc157b70 0x7fc2cc0cc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:55.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.095+0000 7fc2d3f2c640 1 --2- 172.21.15.154:0/1000282288 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2cc0af330 0x7fc2cc157630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:55.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.095+0000 7fc2d3f2c640 1 --2- 172.21.15.154:0/1000282288 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc2b8003f70 0x7fc2cc1f1d40 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:55.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.095+0000 7fc2d3f2c640 1 -- 172.21.15.154:0/1000282288 >> 172.21.15.154:0/1000282288 conn(0x7fc2cc06a0f0 msgr2=0x7fc2cc0cb1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:55.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.095+0000 7fc2d3f2c640 1 -- 172.21.15.154:0/1000282288 shutdown_connections 2023-09-23T03:17:55.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.095+0000 7fc2d3f2c640 1 -- 172.21.15.154:0/1000282288 wait complete. 2023-09-23T03:17:55.115 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-23T03:17:55.124 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-23T03:17:55.134 INFO:tasks.ceph.mds.d.smithi190.stderr: -7070> 2023-09-23T03:17:14.616+0000 7f4ba2e71640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-23T03:17:55.207 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-09-23T03:17:55.273 INFO:teuthology.orchestra.run.smithi154.stdout:[global] 2023-09-23T03:17:55.273 INFO:teuthology.orchestra.run.smithi154.stdout: chdir = "" 2023-09-23T03:17:55.273 INFO:teuthology.orchestra.run.smithi154.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-23T03:17:55.273 INFO:teuthology.orchestra.run.smithi154.stdout: auth supported = cephx 2023-09-23T03:17:55.273 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.273 INFO:teuthology.orchestra.run.smithi154.stdout: filestore xattr use omap = true 2023-09-23T03:17:55.274 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.274 INFO:teuthology.orchestra.run.smithi154.stdout: mon clock drift allowed = 1.000 2023-09-23T03:17:55.274 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.274 INFO:teuthology.orchestra.run.smithi154.stdout: osd crush chooseleaf type = 0 2023-09-23T03:17:55.274 INFO:teuthology.orchestra.run.smithi154.stdout: auth debug = true 2023-09-23T03:17:55.274 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.274 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on old message = true 2023-09-23T03:17:55.274 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on bug = true 2023-09-23T03:17:55.274 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.274 INFO:teuthology.orchestra.run.smithi154.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-23T03:17:55.274 INFO:teuthology.orchestra.run.smithi154.stdout: mon pg warn max object skew = 0 2023-09-23T03:17:55.274 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.275 INFO:teuthology.orchestra.run.smithi154.stdout: # disable pg_autoscaler by default for new pools 2023-09-23T03:17:55.275 INFO:teuthology.orchestra.run.smithi154.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-23T03:17:55.275 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.275 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default size = 2 2023-09-23T03:17:55.275 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.275 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow primary affinity = true 2023-09-23T03:17:55.275 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow pg remap = true 2023-09-23T03:17:55.275 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on legacy crush tunables = false 2023-09-23T03:17:55.275 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on crush straw calc version zero = false 2023-09-23T03:17:55.275 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on no sortbitwise = false 2023-09-23T03:17:55.276 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on osd down out interval zero = false 2023-09-23T03:17:55.276 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on too few osds = false 2023-09-23T03:17:55.276 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-23T03:17:55.276 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-23T03:17:55.276 INFO:teuthology.orchestra.run.smithi154.stdout: mon_allow_pool_size_one = true 2023-09-23T03:17:55.276 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.276 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-23T03:17:55.276 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.276 INFO:teuthology.orchestra.run.smithi154.stdout: osd default data pool replay window = 5 2023-09-23T03:17:55.276 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.276 INFO:teuthology.orchestra.run.smithi154.stdout: mon allow pool delete = true 2023-09-23T03:17:55.276 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.277 INFO:teuthology.orchestra.run.smithi154.stdout: mon cluster log file level = debug 2023-09-23T03:17:55.277 INFO:teuthology.orchestra.run.smithi154.stdout: debug asserts on shutdown = true 2023-09-23T03:17:55.277 INFO:teuthology.orchestra.run.smithi154.stdout: mon health detail to clog = false 2023-09-23T03:17:55.277 INFO:teuthology.orchestra.run.smithi154.stdout: mon host = "172.21.15.154,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]" 2023-09-23T03:17:55.277 INFO:teuthology.orchestra.run.smithi154.stdout: fsid = 8c8039d3-6c84-43db-8808-85e0dbdd47c8 2023-09-23T03:17:55.277 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.277 INFO:teuthology.orchestra.run.smithi154.stdout:[osd] 2023-09-23T03:17:55.277 INFO:teuthology.orchestra.run.smithi154.stdout: osd journal size = 100 2023-09-23T03:17:55.277 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.277 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub load threshold = 5.0 2023-09-23T03:17:55.277 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub max interval = 600 2023-09-23T03:17:55.277 INFO:teuthology.orchestra.run.smithi154.stdout: osd mclock profile = high_recovery_ops 2023-09-23T03:17:55.278 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.278 INFO:teuthology.orchestra.run.smithi154.stdout: osd recover clone overlap = true 2023-09-23T03:17:55.278 INFO:teuthology.orchestra.run.smithi154.stdout: osd recovery max chunk = 1048576 2023-09-23T03:17:55.278 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.278 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug shutdown = true 2023-09-23T03:17:55.278 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug op order = true 2023-09-23T03:17:55.278 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug verify stray on activate = true 2023-09-23T03:17:55.278 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.278 INFO:teuthology.orchestra.run.smithi154.stdout: osd open classes on start = true 2023-09-23T03:17:55.278 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug pg log writeout = true 2023-09-23T03:17:55.278 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.278 INFO:teuthology.orchestra.run.smithi154.stdout: osd deep scrub update digest min age = 30 2023-09-23T03:17:55.279 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.279 INFO:teuthology.orchestra.run.smithi154.stdout: osd map max advance = 10 2023-09-23T03:17:55.279 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.279 INFO:teuthology.orchestra.run.smithi154.stdout: journal zero on create = true 2023-09-23T03:17:55.279 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.279 INFO:teuthology.orchestra.run.smithi154.stdout: filestore ondisk finisher threads = 3 2023-09-23T03:17:55.279 INFO:teuthology.orchestra.run.smithi154.stdout: filestore apply finisher threads = 3 2023-09-23T03:17:55.279 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.279 INFO:teuthology.orchestra.run.smithi154.stdout: bdev debug aio = true 2023-09-23T03:17:55.279 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug misdirected ops = true 2023-09-23T03:17:55.279 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore block size = 96636764160 2023-09-23T03:17:55.280 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore fsck on mount = True 2023-09-23T03:17:55.280 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluefs = 20 2023-09-23T03:17:55.280 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluestore = 20 2023-09-23T03:17:55.280 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:17:55.280 INFO:teuthology.orchestra.run.smithi154.stdout: debug osd = 20 2023-09-23T03:17:55.280 INFO:teuthology.orchestra.run.smithi154.stdout: debug rocksdb = 10 2023-09-23T03:17:55.280 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd backfillfull_ratio = 0.85 2023-09-23T03:17:55.280 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd full ratio = 0.9 2023-09-23T03:17:55.280 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd nearfull ratio = 0.8 2023-09-23T03:17:55.280 INFO:teuthology.orchestra.run.smithi154.stdout: osd failsafe full ratio = 0.95 2023-09-23T03:17:55.280 INFO:teuthology.orchestra.run.smithi154.stdout: osd objectstore = bluestore 2023-09-23T03:17:55.280 INFO:teuthology.orchestra.run.smithi154.stdout: osd op complaint time = 180 2023-09-23T03:17:55.281 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.281 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr] 2023-09-23T03:17:55.281 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:17:55.281 INFO:teuthology.orchestra.run.smithi154.stdout: debug mgr = 20 2023-09-23T03:17:55.281 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2023-09-23T03:17:55.281 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2023-09-23T03:17:55.281 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:17:55.281 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:17:55.281 INFO:teuthology.orchestra.run.smithi154.stdout: mgr/telemetry/nag = false 2023-09-23T03:17:55.281 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.281 INFO:teuthology.orchestra.run.smithi154.stdout:[mon] 2023-09-23T03:17:55.281 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:17:55.281 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2023-09-23T03:17:55.282 INFO:teuthology.orchestra.run.smithi154.stdout: debug paxos = 20 2023-09-23T03:17:55.282 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2023-09-23T03:17:55.282 INFO:teuthology.orchestra.run.smithi154.stdout: mon data avail warn = 5 2023-09-23T03:17:55.282 INFO:teuthology.orchestra.run.smithi154.stdout: mon mgr mkfs grace = 240 2023-09-23T03:17:55.282 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:17:55.282 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd reporter subtree level = osd 2023-09-23T03:17:55.282 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd prime pg temp = true 2023-09-23T03:17:55.282 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:17:55.282 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.282 INFO:teuthology.orchestra.run.smithi154.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-23T03:17:55.282 INFO:teuthology.orchestra.run.smithi154.stdout: auth mon ticket ttl = 660 # 11m 2023-09-23T03:17:55.283 INFO:teuthology.orchestra.run.smithi154.stdout: auth service ticket ttl = 240 # 4m 2023-09-23T03:17:55.283 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.283 INFO:teuthology.orchestra.run.smithi154.stdout: # don't complain about insecure global_id in the test suite 2023-09-23T03:17:55.283 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-23T03:17:55.283 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-23T03:17:55.283 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.283 INFO:teuthology.orchestra.run.smithi154.stdout: # 1m isn't quite enough 2023-09-23T03:17:55.283 INFO:teuthology.orchestra.run.smithi154.stdout: mon_down_mkfs_grace = 2m 2023-09-23T03:17:55.283 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.283 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_filestore_osds = false 2023-09-23T03:17:55.283 INFO:teuthology.orchestra.run.smithi154.stdout: mon op complaint time = 120 2023-09-23T03:17:55.283 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.283 INFO:teuthology.orchestra.run.smithi154.stdout:[client] 2023-09-23T03:17:55.284 INFO:teuthology.orchestra.run.smithi154.stdout: rgw cache enabled = true 2023-09-23T03:17:55.284 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable ops log = true 2023-09-23T03:17:55.284 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable usage log = true 2023-09-23T03:17:55.284 INFO:teuthology.orchestra.run.smithi154.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-23T03:17:55.284 INFO:teuthology.orchestra.run.smithi154.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-23T03:17:55.284 INFO:teuthology.orchestra.run.smithi154.stdout: client mount timeout = 600 2023-09-23T03:17:55.284 INFO:teuthology.orchestra.run.smithi154.stdout: debug client = 20 2023-09-23T03:17:55.284 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:17:55.284 INFO:teuthology.orchestra.run.smithi154.stdout: pid file = "" 2023-09-23T03:17:55.284 INFO:teuthology.orchestra.run.smithi154.stdout: rados mon op timeout = 900 2023-09-23T03:17:55.284 INFO:teuthology.orchestra.run.smithi154.stdout: rados osd op timeout = 900 2023-09-23T03:17:55.284 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.a] 2023-09-23T03:17:55.285 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.b] 2023-09-23T03:17:55.285 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.c] 2023-09-23T03:17:55.285 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.a] 2023-09-23T03:17:55.285 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.c] 2023-09-23T03:17:55.285 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.b] 2023-09-23T03:17:55.285 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.d] 2023-09-23T03:17:55.285 INFO:teuthology.orchestra.run.smithi154.stdout:[mds] 2023-09-23T03:17:55.285 INFO:teuthology.orchestra.run.smithi154.stdout: debug mds = 20 2023-09-23T03:17:55.285 INFO:teuthology.orchestra.run.smithi154.stdout: debug mds balancer = 20 2023-09-23T03:17:55.285 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:17:55.285 INFO:teuthology.orchestra.run.smithi154.stdout: mds debug frag = True 2023-09-23T03:17:55.285 INFO:teuthology.orchestra.run.smithi154.stdout: mds debug scatterstat = True 2023-09-23T03:17:55.285 INFO:teuthology.orchestra.run.smithi154.stdout: mds op complaint time = 180 2023-09-23T03:17:55.286 INFO:teuthology.orchestra.run.smithi154.stdout: mds verify scatter = False 2023-09-23T03:17:55.286 INFO:teuthology.orchestra.run.smithi154.stdout: osd op complaint time = 180 2023-09-23T03:17:55.286 INFO:teuthology.orchestra.run.smithi154.stdout: rados mon op timeout = 900 2023-09-23T03:17:55.286 INFO:teuthology.orchestra.run.smithi154.stdout: rados osd op timeout = 900 2023-09-23T03:17:55.289 INFO:teuthology.orchestra.run.smithi190.stdout:[global] 2023-09-23T03:17:55.289 INFO:teuthology.orchestra.run.smithi190.stdout: chdir = "" 2023-09-23T03:17:55.289 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-23T03:17:55.289 INFO:teuthology.orchestra.run.smithi190.stdout: auth supported = cephx 2023-09-23T03:17:55.289 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.289 INFO:teuthology.orchestra.run.smithi190.stdout: filestore xattr use omap = true 2023-09-23T03:17:55.289 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.289 INFO:teuthology.orchestra.run.smithi190.stdout: mon clock drift allowed = 1.000 2023-09-23T03:17:55.290 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.290 INFO:teuthology.orchestra.run.smithi190.stdout: osd crush chooseleaf type = 0 2023-09-23T03:17:55.290 INFO:teuthology.orchestra.run.smithi190.stdout: auth debug = true 2023-09-23T03:17:55.290 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.290 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on old message = true 2023-09-23T03:17:55.290 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on bug = true 2023-09-23T03:17:55.290 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.290 INFO:teuthology.orchestra.run.smithi190.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-23T03:17:55.290 INFO:teuthology.orchestra.run.smithi190.stdout: mon pg warn max object skew = 0 2023-09-23T03:17:55.290 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.290 INFO:teuthology.orchestra.run.smithi190.stdout: # disable pg_autoscaler by default for new pools 2023-09-23T03:17:55.291 INFO:teuthology.orchestra.run.smithi190.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-23T03:17:55.291 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.291 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default size = 2 2023-09-23T03:17:55.291 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.291 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow primary affinity = true 2023-09-23T03:17:55.291 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow pg remap = true 2023-09-23T03:17:55.291 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on legacy crush tunables = false 2023-09-23T03:17:55.291 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on crush straw calc version zero = false 2023-09-23T03:17:55.291 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on no sortbitwise = false 2023-09-23T03:17:55.291 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on osd down out interval zero = false 2023-09-23T03:17:55.291 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on too few osds = false 2023-09-23T03:17:55.291 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-23T03:17:55.291 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-23T03:17:55.292 INFO:teuthology.orchestra.run.smithi190.stdout: mon_allow_pool_size_one = true 2023-09-23T03:17:55.292 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.292 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-23T03:17:55.292 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.292 INFO:teuthology.orchestra.run.smithi190.stdout: osd default data pool replay window = 5 2023-09-23T03:17:55.292 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.292 INFO:teuthology.orchestra.run.smithi190.stdout: mon allow pool delete = true 2023-09-23T03:17:55.292 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.292 INFO:teuthology.orchestra.run.smithi190.stdout: mon cluster log file level = debug 2023-09-23T03:17:55.292 INFO:teuthology.orchestra.run.smithi190.stdout: debug asserts on shutdown = true 2023-09-23T03:17:55.292 INFO:teuthology.orchestra.run.smithi190.stdout: mon health detail to clog = false 2023-09-23T03:17:55.292 INFO:teuthology.orchestra.run.smithi190.stdout: mon host = "172.21.15.154,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]" 2023-09-23T03:17:55.293 INFO:teuthology.orchestra.run.smithi190.stdout: fsid = 8c8039d3-6c84-43db-8808-85e0dbdd47c8 2023-09-23T03:17:55.293 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.293 INFO:teuthology.orchestra.run.smithi190.stdout:[osd] 2023-09-23T03:17:55.293 INFO:teuthology.orchestra.run.smithi190.stdout: osd journal size = 100 2023-09-23T03:17:55.293 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.293 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub load threshold = 5.0 2023-09-23T03:17:55.293 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub max interval = 600 2023-09-23T03:17:55.293 INFO:teuthology.orchestra.run.smithi190.stdout: osd mclock profile = high_recovery_ops 2023-09-23T03:17:55.293 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.293 INFO:teuthology.orchestra.run.smithi190.stdout: osd recover clone overlap = true 2023-09-23T03:17:55.293 INFO:teuthology.orchestra.run.smithi190.stdout: osd recovery max chunk = 1048576 2023-09-23T03:17:55.293 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.294 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug shutdown = true 2023-09-23T03:17:55.294 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug op order = true 2023-09-23T03:17:55.294 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug verify stray on activate = true 2023-09-23T03:17:55.294 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.294 INFO:teuthology.orchestra.run.smithi190.stdout: osd open classes on start = true 2023-09-23T03:17:55.294 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug pg log writeout = true 2023-09-23T03:17:55.294 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.294 INFO:teuthology.orchestra.run.smithi190.stdout: osd deep scrub update digest min age = 30 2023-09-23T03:17:55.294 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.294 INFO:teuthology.orchestra.run.smithi190.stdout: osd map max advance = 10 2023-09-23T03:17:55.295 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.295 INFO:teuthology.orchestra.run.smithi190.stdout: journal zero on create = true 2023-09-23T03:17:55.295 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.295 INFO:teuthology.orchestra.run.smithi190.stdout: filestore ondisk finisher threads = 3 2023-09-23T03:17:55.295 INFO:teuthology.orchestra.run.smithi190.stdout: filestore apply finisher threads = 3 2023-09-23T03:17:55.295 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.295 INFO:teuthology.orchestra.run.smithi190.stdout: bdev debug aio = true 2023-09-23T03:17:55.295 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug misdirected ops = true 2023-09-23T03:17:55.295 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore block size = 96636764160 2023-09-23T03:17:55.295 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore fsck on mount = True 2023-09-23T03:17:55.295 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluefs = 20 2023-09-23T03:17:55.295 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluestore = 20 2023-09-23T03:17:55.295 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:17:55.296 INFO:teuthology.orchestra.run.smithi190.stdout: debug osd = 20 2023-09-23T03:17:55.296 INFO:teuthology.orchestra.run.smithi190.stdout: debug rocksdb = 10 2023-09-23T03:17:55.296 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd backfillfull_ratio = 0.85 2023-09-23T03:17:55.296 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd full ratio = 0.9 2023-09-23T03:17:55.296 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd nearfull ratio = 0.8 2023-09-23T03:17:55.296 INFO:teuthology.orchestra.run.smithi190.stdout: osd failsafe full ratio = 0.95 2023-09-23T03:17:55.296 INFO:teuthology.orchestra.run.smithi190.stdout: osd objectstore = bluestore 2023-09-23T03:17:55.296 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2023-09-23T03:17:55.296 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.296 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr] 2023-09-23T03:17:55.296 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:17:55.296 INFO:teuthology.orchestra.run.smithi190.stdout: debug mgr = 20 2023-09-23T03:17:55.297 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2023-09-23T03:17:55.297 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2023-09-23T03:17:55.297 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:17:55.297 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:17:55.297 INFO:teuthology.orchestra.run.smithi190.stdout: mgr/telemetry/nag = false 2023-09-23T03:17:55.297 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.297 INFO:teuthology.orchestra.run.smithi190.stdout:[mon] 2023-09-23T03:17:55.297 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:17:55.297 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2023-09-23T03:17:55.297 INFO:teuthology.orchestra.run.smithi190.stdout: debug paxos = 20 2023-09-23T03:17:55.297 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2023-09-23T03:17:55.297 INFO:teuthology.orchestra.run.smithi190.stdout: mon data avail warn = 5 2023-09-23T03:17:55.297 INFO:teuthology.orchestra.run.smithi190.stdout: mon mgr mkfs grace = 240 2023-09-23T03:17:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:17:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd reporter subtree level = osd 2023-09-23T03:17:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd prime pg temp = true 2023-09-23T03:17:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:17:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-23T03:17:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: auth mon ticket ttl = 660 # 11m 2023-09-23T03:17:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: auth service ticket ttl = 240 # 4m 2023-09-23T03:17:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: # don't complain about insecure global_id in the test suite 2023-09-23T03:17:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-23T03:17:55.298 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-23T03:17:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: # 1m isn't quite enough 2023-09-23T03:17:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: mon_down_mkfs_grace = 2m 2023-09-23T03:17:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_filestore_osds = false 2023-09-23T03:17:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: mon op complaint time = 120 2023-09-23T03:17:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.299 INFO:teuthology.orchestra.run.smithi190.stdout:[client] 2023-09-23T03:17:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: rgw cache enabled = true 2023-09-23T03:17:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable ops log = true 2023-09-23T03:17:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable usage log = true 2023-09-23T03:17:55.299 INFO:teuthology.orchestra.run.smithi190.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-23T03:17:55.300 INFO:teuthology.orchestra.run.smithi190.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-23T03:17:55.300 INFO:teuthology.orchestra.run.smithi190.stdout: client mount timeout = 600 2023-09-23T03:17:55.300 INFO:teuthology.orchestra.run.smithi190.stdout: debug client = 20 2023-09-23T03:17:55.300 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:17:55.300 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = "" 2023-09-23T03:17:55.300 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 900 2023-09-23T03:17:55.300 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 900 2023-09-23T03:17:55.300 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.a] 2023-09-23T03:17:55.300 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.b] 2023-09-23T03:17:55.300 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.c] 2023-09-23T03:17:55.300 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.a] 2023-09-23T03:17:55.300 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.c] 2023-09-23T03:17:55.300 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.b] 2023-09-23T03:17:55.301 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.d] 2023-09-23T03:17:55.301 INFO:teuthology.orchestra.run.smithi190.stdout:[mds] 2023-09-23T03:17:55.301 INFO:teuthology.orchestra.run.smithi190.stdout: debug mds = 20 2023-09-23T03:17:55.301 INFO:teuthology.orchestra.run.smithi190.stdout: debug mds balancer = 20 2023-09-23T03:17:55.301 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:17:55.301 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug frag = True 2023-09-23T03:17:55.301 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug scatterstat = True 2023-09-23T03:17:55.301 INFO:teuthology.orchestra.run.smithi190.stdout: mds op complaint time = 180 2023-09-23T03:17:55.301 INFO:teuthology.orchestra.run.smithi190.stdout: mds verify scatter = False 2023-09-23T03:17:55.301 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2023-09-23T03:17:55.301 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 900 2023-09-23T03:17:55.301 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 900 2023-09-23T03:17:55.327 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-23T03:17:55.335 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-23T03:17:55.501 INFO:teuthology.orchestra.run.smithi154.stdout:[global] 2023-09-23T03:17:55.501 INFO:teuthology.orchestra.run.smithi154.stdout: chdir = "" 2023-09-23T03:17:55.501 INFO:teuthology.orchestra.run.smithi154.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-23T03:17:55.501 INFO:teuthology.orchestra.run.smithi154.stdout: auth supported = cephx 2023-09-23T03:17:55.501 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.502 INFO:teuthology.orchestra.run.smithi154.stdout: filestore xattr use omap = true 2023-09-23T03:17:55.502 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.502 INFO:teuthology.orchestra.run.smithi154.stdout: mon clock drift allowed = 1.000 2023-09-23T03:17:55.502 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.502 INFO:teuthology.orchestra.run.smithi154.stdout: osd crush chooseleaf type = 0 2023-09-23T03:17:55.502 INFO:teuthology.orchestra.run.smithi154.stdout: auth debug = true 2023-09-23T03:17:55.502 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.502 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on old message = true 2023-09-23T03:17:55.502 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on bug = true 2023-09-23T03:17:55.502 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.502 INFO:teuthology.orchestra.run.smithi154.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-23T03:17:55.503 INFO:teuthology.orchestra.run.smithi154.stdout: mon pg warn max object skew = 0 2023-09-23T03:17:55.503 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.503 INFO:teuthology.orchestra.run.smithi154.stdout: # disable pg_autoscaler by default for new pools 2023-09-23T03:17:55.503 INFO:teuthology.orchestra.run.smithi154.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-23T03:17:55.503 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.503 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default size = 2 2023-09-23T03:17:55.503 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.503 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow primary affinity = true 2023-09-23T03:17:55.503 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow pg remap = true 2023-09-23T03:17:55.503 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on legacy crush tunables = false 2023-09-23T03:17:55.503 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on crush straw calc version zero = false 2023-09-23T03:17:55.504 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on no sortbitwise = false 2023-09-23T03:17:55.504 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on osd down out interval zero = false 2023-09-23T03:17:55.504 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on too few osds = false 2023-09-23T03:17:55.504 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-23T03:17:55.504 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-23T03:17:55.504 INFO:teuthology.orchestra.run.smithi154.stdout: mon_allow_pool_size_one = true 2023-09-23T03:17:55.504 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.504 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-23T03:17:55.504 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.504 INFO:teuthology.orchestra.run.smithi154.stdout: osd default data pool replay window = 5 2023-09-23T03:17:55.504 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.504 INFO:teuthology.orchestra.run.smithi154.stdout: mon allow pool delete = true 2023-09-23T03:17:55.505 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.505 INFO:teuthology.orchestra.run.smithi154.stdout: mon cluster log file level = debug 2023-09-23T03:17:55.505 INFO:teuthology.orchestra.run.smithi154.stdout: debug asserts on shutdown = true 2023-09-23T03:17:55.505 INFO:teuthology.orchestra.run.smithi154.stdout: mon health detail to clog = false 2023-09-23T03:17:55.505 INFO:teuthology.orchestra.run.smithi154.stdout: mon host = "172.21.15.154,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]" 2023-09-23T03:17:55.505 INFO:teuthology.orchestra.run.smithi154.stdout: fsid = 8c8039d3-6c84-43db-8808-85e0dbdd47c8 2023-09-23T03:17:55.505 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.505 INFO:teuthology.orchestra.run.smithi154.stdout:[osd] 2023-09-23T03:17:55.505 INFO:teuthology.orchestra.run.smithi154.stdout: osd journal size = 100 2023-09-23T03:17:55.506 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.506 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub load threshold = 5.0 2023-09-23T03:17:55.506 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub max interval = 600 2023-09-23T03:17:55.506 INFO:teuthology.orchestra.run.smithi154.stdout: osd mclock profile = high_recovery_ops 2023-09-23T03:17:55.506 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.506 INFO:teuthology.orchestra.run.smithi154.stdout: osd recover clone overlap = true 2023-09-23T03:17:55.506 INFO:teuthology.orchestra.run.smithi154.stdout: osd recovery max chunk = 1048576 2023-09-23T03:17:55.506 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.506 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug shutdown = true 2023-09-23T03:17:55.506 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug op order = true 2023-09-23T03:17:55.506 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug verify stray on activate = true 2023-09-23T03:17:55.507 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.507 INFO:teuthology.orchestra.run.smithi154.stdout: osd open classes on start = true 2023-09-23T03:17:55.507 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug pg log writeout = true 2023-09-23T03:17:55.507 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.507 INFO:teuthology.orchestra.run.smithi154.stdout: osd deep scrub update digest min age = 30 2023-09-23T03:17:55.507 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.507 INFO:teuthology.orchestra.run.smithi154.stdout: osd map max advance = 10 2023-09-23T03:17:55.507 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.507 INFO:teuthology.orchestra.run.smithi154.stdout: journal zero on create = true 2023-09-23T03:17:55.507 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.508 INFO:teuthology.orchestra.run.smithi154.stdout: filestore ondisk finisher threads = 3 2023-09-23T03:17:55.508 INFO:teuthology.orchestra.run.smithi154.stdout: filestore apply finisher threads = 3 2023-09-23T03:17:55.508 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.508 INFO:teuthology.orchestra.run.smithi154.stdout: bdev debug aio = true 2023-09-23T03:17:55.508 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug misdirected ops = true 2023-09-23T03:17:55.508 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore block size = 96636764160 2023-09-23T03:17:55.508 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore fsck on mount = True 2023-09-23T03:17:55.508 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluefs = 20 2023-09-23T03:17:55.508 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluestore = 20 2023-09-23T03:17:55.508 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:17:55.508 INFO:teuthology.orchestra.run.smithi154.stdout: debug osd = 20 2023-09-23T03:17:55.509 INFO:teuthology.orchestra.run.smithi154.stdout: debug rocksdb = 10 2023-09-23T03:17:55.509 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd backfillfull_ratio = 0.85 2023-09-23T03:17:55.509 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd full ratio = 0.9 2023-09-23T03:17:55.509 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd nearfull ratio = 0.8 2023-09-23T03:17:55.509 INFO:teuthology.orchestra.run.smithi154.stdout: osd failsafe full ratio = 0.95 2023-09-23T03:17:55.509 INFO:teuthology.orchestra.run.smithi154.stdout: osd objectstore = bluestore 2023-09-23T03:17:55.509 INFO:teuthology.orchestra.run.smithi154.stdout: osd op complaint time = 180 2023-09-23T03:17:55.509 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.509 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr] 2023-09-23T03:17:55.509 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:17:55.509 INFO:teuthology.orchestra.run.smithi154.stdout: debug mgr = 20 2023-09-23T03:17:55.510 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2023-09-23T03:17:55.510 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2023-09-23T03:17:55.510 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:17:55.510 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:17:55.510 INFO:teuthology.orchestra.run.smithi154.stdout: mgr/telemetry/nag = false 2023-09-23T03:17:55.510 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.510 INFO:teuthology.orchestra.run.smithi154.stdout:[mon] 2023-09-23T03:17:55.510 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:17:55.510 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2023-09-23T03:17:55.510 INFO:teuthology.orchestra.run.smithi154.stdout: debug paxos = 20 2023-09-23T03:17:55.510 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2023-09-23T03:17:55.511 INFO:teuthology.orchestra.run.smithi154.stdout: mon data avail warn = 5 2023-09-23T03:17:55.511 INFO:teuthology.orchestra.run.smithi154.stdout: mon mgr mkfs grace = 240 2023-09-23T03:17:55.511 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:17:55.511 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd reporter subtree level = osd 2023-09-23T03:17:55.511 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd prime pg temp = true 2023-09-23T03:17:55.511 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:17:55.511 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.511 INFO:teuthology.orchestra.run.smithi154.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-23T03:17:55.511 INFO:teuthology.orchestra.run.smithi154.stdout: auth mon ticket ttl = 660 # 11m 2023-09-23T03:17:55.511 INFO:teuthology.orchestra.run.smithi154.stdout: auth service ticket ttl = 240 # 4m 2023-09-23T03:17:55.511 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.512 INFO:teuthology.orchestra.run.smithi154.stdout: # don't complain about insecure global_id in the test suite 2023-09-23T03:17:55.512 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-23T03:17:55.512 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-23T03:17:55.512 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.512 INFO:teuthology.orchestra.run.smithi154.stdout: # 1m isn't quite enough 2023-09-23T03:17:55.512 INFO:teuthology.orchestra.run.smithi154.stdout: mon_down_mkfs_grace = 2m 2023-09-23T03:17:55.512 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.512 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_filestore_osds = false 2023-09-23T03:17:55.512 INFO:teuthology.orchestra.run.smithi154.stdout: mon op complaint time = 120 2023-09-23T03:17:55.512 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:55.512 INFO:teuthology.orchestra.run.smithi154.stdout:[client] 2023-09-23T03:17:55.513 INFO:teuthology.orchestra.run.smithi154.stdout: rgw cache enabled = true 2023-09-23T03:17:55.513 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable ops log = true 2023-09-23T03:17:55.513 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable usage log = true 2023-09-23T03:17:55.513 INFO:teuthology.orchestra.run.smithi154.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-23T03:17:55.513 INFO:teuthology.orchestra.run.smithi154.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-23T03:17:55.513 INFO:teuthology.orchestra.run.smithi154.stdout: client mount timeout = 600 2023-09-23T03:17:55.513 INFO:teuthology.orchestra.run.smithi154.stdout: debug client = 20 2023-09-23T03:17:55.513 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:17:55.513 INFO:teuthology.orchestra.run.smithi154.stdout: pid file = "" 2023-09-23T03:17:55.513 INFO:teuthology.orchestra.run.smithi154.stdout: rados mon op timeout = 900 2023-09-23T03:17:55.513 INFO:teuthology.orchestra.run.smithi154.stdout: rados osd op timeout = 900 2023-09-23T03:17:55.514 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.a] 2023-09-23T03:17:55.514 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.b] 2023-09-23T03:17:55.514 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.c] 2023-09-23T03:17:55.514 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.a] 2023-09-23T03:17:55.514 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.c] 2023-09-23T03:17:55.514 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.b] 2023-09-23T03:17:55.514 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.d] 2023-09-23T03:17:55.514 INFO:teuthology.orchestra.run.smithi154.stdout:[mds] 2023-09-23T03:17:55.514 INFO:teuthology.orchestra.run.smithi154.stdout: debug mds = 20 2023-09-23T03:17:55.514 INFO:teuthology.orchestra.run.smithi154.stdout: debug mds balancer = 20 2023-09-23T03:17:55.514 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:17:55.514 INFO:teuthology.orchestra.run.smithi154.stdout: mds debug frag = True 2023-09-23T03:17:55.515 INFO:teuthology.orchestra.run.smithi154.stdout: mds debug scatterstat = False 2023-09-23T03:17:55.515 INFO:teuthology.orchestra.run.smithi154.stdout: mds op complaint time = 180 2023-09-23T03:17:55.515 INFO:teuthology.orchestra.run.smithi154.stdout: mds verify scatter = False 2023-09-23T03:17:55.515 INFO:teuthology.orchestra.run.smithi154.stdout: osd op complaint time = 180 2023-09-23T03:17:55.515 INFO:teuthology.orchestra.run.smithi154.stdout: rados mon op timeout = 900 2023-09-23T03:17:55.515 INFO:teuthology.orchestra.run.smithi154.stdout: rados osd op timeout = 900 2023-09-23T03:17:55.515 INFO:teuthology.orchestra.run.smithi190.stdout:[global] 2023-09-23T03:17:55.515 INFO:teuthology.orchestra.run.smithi190.stdout: chdir = "" 2023-09-23T03:17:55.516 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-23T03:17:55.516 INFO:teuthology.orchestra.run.smithi190.stdout: auth supported = cephx 2023-09-23T03:17:55.516 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.516 INFO:teuthology.orchestra.run.smithi190.stdout: filestore xattr use omap = true 2023-09-23T03:17:55.516 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.516 INFO:teuthology.orchestra.run.smithi190.stdout: mon clock drift allowed = 1.000 2023-09-23T03:17:55.516 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.516 INFO:teuthology.orchestra.run.smithi190.stdout: osd crush chooseleaf type = 0 2023-09-23T03:17:55.516 INFO:teuthology.orchestra.run.smithi190.stdout: auth debug = true 2023-09-23T03:17:55.516 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.516 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on old message = true 2023-09-23T03:17:55.516 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on bug = true 2023-09-23T03:17:55.517 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.517 INFO:teuthology.orchestra.run.smithi190.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-23T03:17:55.517 INFO:teuthology.orchestra.run.smithi190.stdout: mon pg warn max object skew = 0 2023-09-23T03:17:55.517 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.517 INFO:teuthology.orchestra.run.smithi190.stdout: # disable pg_autoscaler by default for new pools 2023-09-23T03:17:55.517 INFO:teuthology.orchestra.run.smithi190.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-23T03:17:55.517 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.517 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default size = 2 2023-09-23T03:17:55.517 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.517 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow primary affinity = true 2023-09-23T03:17:55.517 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow pg remap = true 2023-09-23T03:17:55.517 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on legacy crush tunables = false 2023-09-23T03:17:55.518 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on crush straw calc version zero = false 2023-09-23T03:17:55.518 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on no sortbitwise = false 2023-09-23T03:17:55.518 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on osd down out interval zero = false 2023-09-23T03:17:55.518 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on too few osds = false 2023-09-23T03:17:55.518 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-23T03:17:55.518 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-23T03:17:55.518 INFO:teuthology.orchestra.run.smithi190.stdout: mon_allow_pool_size_one = true 2023-09-23T03:17:55.518 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.518 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-23T03:17:55.518 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.518 INFO:teuthology.orchestra.run.smithi190.stdout: osd default data pool replay window = 5 2023-09-23T03:17:55.518 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.519 INFO:teuthology.orchestra.run.smithi190.stdout: mon allow pool delete = true 2023-09-23T03:17:55.519 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.519 INFO:teuthology.orchestra.run.smithi190.stdout: mon cluster log file level = debug 2023-09-23T03:17:55.519 INFO:teuthology.orchestra.run.smithi190.stdout: debug asserts on shutdown = true 2023-09-23T03:17:55.519 INFO:teuthology.orchestra.run.smithi190.stdout: mon health detail to clog = false 2023-09-23T03:17:55.519 INFO:teuthology.orchestra.run.smithi190.stdout: mon host = "172.21.15.154,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]" 2023-09-23T03:17:55.519 INFO:teuthology.orchestra.run.smithi190.stdout: fsid = 8c8039d3-6c84-43db-8808-85e0dbdd47c8 2023-09-23T03:17:55.519 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.519 INFO:teuthology.orchestra.run.smithi190.stdout:[osd] 2023-09-23T03:17:55.519 INFO:teuthology.orchestra.run.smithi190.stdout: osd journal size = 100 2023-09-23T03:17:55.519 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.519 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub load threshold = 5.0 2023-09-23T03:17:55.520 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub max interval = 600 2023-09-23T03:17:55.520 INFO:teuthology.orchestra.run.smithi190.stdout: osd mclock profile = high_recovery_ops 2023-09-23T03:17:55.520 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.520 INFO:teuthology.orchestra.run.smithi190.stdout: osd recover clone overlap = true 2023-09-23T03:17:55.520 INFO:teuthology.orchestra.run.smithi190.stdout: osd recovery max chunk = 1048576 2023-09-23T03:17:55.520 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.520 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug shutdown = true 2023-09-23T03:17:55.520 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug op order = true 2023-09-23T03:17:55.520 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug verify stray on activate = true 2023-09-23T03:17:55.520 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.520 INFO:teuthology.orchestra.run.smithi190.stdout: osd open classes on start = true 2023-09-23T03:17:55.520 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug pg log writeout = true 2023-09-23T03:17:55.521 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.521 INFO:teuthology.orchestra.run.smithi190.stdout: osd deep scrub update digest min age = 30 2023-09-23T03:17:55.521 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.521 INFO:teuthology.orchestra.run.smithi190.stdout: osd map max advance = 10 2023-09-23T03:17:55.521 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.521 INFO:teuthology.orchestra.run.smithi190.stdout: journal zero on create = true 2023-09-23T03:17:55.521 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.521 INFO:teuthology.orchestra.run.smithi190.stdout: filestore ondisk finisher threads = 3 2023-09-23T03:17:55.521 INFO:teuthology.orchestra.run.smithi190.stdout: filestore apply finisher threads = 3 2023-09-23T03:17:55.521 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.521 INFO:teuthology.orchestra.run.smithi190.stdout: bdev debug aio = true 2023-09-23T03:17:55.521 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug misdirected ops = true 2023-09-23T03:17:55.522 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore block size = 96636764160 2023-09-23T03:17:55.522 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore fsck on mount = True 2023-09-23T03:17:55.522 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluefs = 20 2023-09-23T03:17:55.522 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluestore = 20 2023-09-23T03:17:55.522 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:17:55.522 INFO:teuthology.orchestra.run.smithi190.stdout: debug osd = 20 2023-09-23T03:17:55.522 INFO:teuthology.orchestra.run.smithi190.stdout: debug rocksdb = 10 2023-09-23T03:17:55.522 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd backfillfull_ratio = 0.85 2023-09-23T03:17:55.522 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd full ratio = 0.9 2023-09-23T03:17:55.522 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd nearfull ratio = 0.8 2023-09-23T03:17:55.522 INFO:teuthology.orchestra.run.smithi190.stdout: osd failsafe full ratio = 0.95 2023-09-23T03:17:55.523 INFO:teuthology.orchestra.run.smithi190.stdout: osd objectstore = bluestore 2023-09-23T03:17:55.523 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2023-09-23T03:17:55.523 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.523 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr] 2023-09-23T03:17:55.523 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:17:55.523 INFO:teuthology.orchestra.run.smithi190.stdout: debug mgr = 20 2023-09-23T03:17:55.523 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2023-09-23T03:17:55.523 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2023-09-23T03:17:55.523 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:17:55.523 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:17:55.523 INFO:teuthology.orchestra.run.smithi190.stdout: mgr/telemetry/nag = false 2023-09-23T03:17:55.523 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.523 INFO:teuthology.orchestra.run.smithi190.stdout:[mon] 2023-09-23T03:17:55.524 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:17:55.524 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2023-09-23T03:17:55.524 INFO:teuthology.orchestra.run.smithi190.stdout: debug paxos = 20 2023-09-23T03:17:55.524 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2023-09-23T03:17:55.524 INFO:teuthology.orchestra.run.smithi190.stdout: mon data avail warn = 5 2023-09-23T03:17:55.524 INFO:teuthology.orchestra.run.smithi190.stdout: mon mgr mkfs grace = 240 2023-09-23T03:17:55.524 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:17:55.524 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd reporter subtree level = osd 2023-09-23T03:17:55.524 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd prime pg temp = true 2023-09-23T03:17:55.524 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:17:55.524 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.524 INFO:teuthology.orchestra.run.smithi190.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-23T03:17:55.525 INFO:teuthology.orchestra.run.smithi190.stdout: auth mon ticket ttl = 660 # 11m 2023-09-23T03:17:55.525 INFO:teuthology.orchestra.run.smithi190.stdout: auth service ticket ttl = 240 # 4m 2023-09-23T03:17:55.525 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.525 INFO:teuthology.orchestra.run.smithi190.stdout: # don't complain about insecure global_id in the test suite 2023-09-23T03:17:55.525 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-23T03:17:55.525 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-23T03:17:55.525 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.525 INFO:teuthology.orchestra.run.smithi190.stdout: # 1m isn't quite enough 2023-09-23T03:17:55.525 INFO:teuthology.orchestra.run.smithi190.stdout: mon_down_mkfs_grace = 2m 2023-09-23T03:17:55.525 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.525 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_filestore_osds = false 2023-09-23T03:17:55.525 INFO:teuthology.orchestra.run.smithi190.stdout: mon op complaint time = 120 2023-09-23T03:17:55.525 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:17:55.526 INFO:teuthology.orchestra.run.smithi190.stdout:[client] 2023-09-23T03:17:55.526 INFO:teuthology.orchestra.run.smithi190.stdout: rgw cache enabled = true 2023-09-23T03:17:55.526 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable ops log = true 2023-09-23T03:17:55.526 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable usage log = true 2023-09-23T03:17:55.526 INFO:teuthology.orchestra.run.smithi190.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-23T03:17:55.526 INFO:teuthology.orchestra.run.smithi190.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-23T03:17:55.526 INFO:teuthology.orchestra.run.smithi190.stdout: client mount timeout = 600 2023-09-23T03:17:55.526 INFO:teuthology.orchestra.run.smithi190.stdout: debug client = 20 2023-09-23T03:17:55.526 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:17:55.526 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = "" 2023-09-23T03:17:55.526 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 900 2023-09-23T03:17:55.526 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 900 2023-09-23T03:17:55.527 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.a] 2023-09-23T03:17:55.527 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.b] 2023-09-23T03:17:55.527 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.c] 2023-09-23T03:17:55.527 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.a] 2023-09-23T03:17:55.527 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.c] 2023-09-23T03:17:55.527 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.b] 2023-09-23T03:17:55.527 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.d] 2023-09-23T03:17:55.527 INFO:teuthology.orchestra.run.smithi190.stdout:[mds] 2023-09-23T03:17:55.527 INFO:teuthology.orchestra.run.smithi190.stdout: debug mds = 20 2023-09-23T03:17:55.527 INFO:teuthology.orchestra.run.smithi190.stdout: debug mds balancer = 20 2023-09-23T03:17:55.527 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:17:55.528 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug frag = True 2023-09-23T03:17:55.528 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug scatterstat = False 2023-09-23T03:17:55.528 INFO:teuthology.orchestra.run.smithi190.stdout: mds op complaint time = 180 2023-09-23T03:17:55.528 INFO:teuthology.orchestra.run.smithi190.stdout: mds verify scatter = False 2023-09-23T03:17:55.528 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2023-09-23T03:17:55.528 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 900 2023-09-23T03:17:55.528 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 900 2023-09-23T03:17:55.553 DEBUG:teuthology.orchestra.run.smithi154:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph purge cephfs_metadata --yes-i-really-really-mean-it 2023-09-23T03:17:55.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.664+0000 7f3662955f40 1 Processor -- start 2023-09-23T03:17:55.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.664+0000 7f3662955f40 1 -- start start 2023-09-23T03:17:55.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.665+0000 7f3662955f40 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3648002290 0x562db87cc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:55.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.665+0000 7f3662955f40 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562db87af8c0 0x562db86f1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:55.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.665+0000 7f3662955f40 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562db86f1c60 0x562db885e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:55.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.665+0000 7f3662955f40 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x562db86a1170 con 0x562db86f1c60 2023-09-23T03:17:55.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.665+0000 7f3662955f40 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562db863c800 con 0x562db87af8c0 2023-09-23T03:17:55.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.665+0000 7f3662955f40 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x562db863c4a0 con 0x7f3648002290 2023-09-23T03:17:55.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.665+0000 7f3660c58640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562db86f1c60 0x562db885e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:55.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.665+0000 7f3660c58640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562db86f1c60 0x562db885e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60088/0 (socket says 172.21.15.154:60088) 2023-09-23T03:17:55.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.665+0000 7f3660c58640 1 -- 172.21.15.154:0/129265330 learned_addr learned my addr 172.21.15.154:0/129265330 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:55.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.665+0000 7f3660457640 1 --2- 172.21.15.154:0/129265330 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3648002290 0x562db87cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:55.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.665+0000 7f365fc56640 1 --2- 172.21.15.154:0/129265330 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562db87af8c0 0x562db86f1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:55.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.666+0000 7f3660c58640 1 -- 172.21.15.154:0/129265330 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3648002290 msgr2=0x562db87cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:55.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.666+0000 7f3660c58640 1 --2- 172.21.15.154:0/129265330 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3648002290 0x562db87cc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:55.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.666+0000 7f3660c58640 1 -- 172.21.15.154:0/129265330 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562db87af8c0 msgr2=0x562db86f1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:55.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.666+0000 7f3660c58640 1 --2- 172.21.15.154:0/129265330 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562db87af8c0 0x562db86f1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:55.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.666+0000 7f3660c58640 1 -- 172.21.15.154:0/129265330 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562db863b800 con 0x562db86f1c60 2023-09-23T03:17:55.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.666+0000 7f3660c58640 1 --2- 172.21.15.154:0/129265330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562db86f1c60 0x562db885e9f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x562db87d1de0 tx=0x7f365800c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9935fa6d68c2d4da server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:55.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.666+0000 7f365ec54640 1 -- 172.21.15.154:0/129265330 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365800cde0 con 0x562db86f1c60 2023-09-23T03:17:55.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.666+0000 7f365ec54640 1 -- 172.21.15.154:0/129265330 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f365800e440 con 0x562db86f1c60 2023-09-23T03:17:55.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.666+0000 7f365ec54640 1 -- 172.21.15.154:0/129265330 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365800e7d0 con 0x562db86f1c60 2023-09-23T03:17:55.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.667+0000 7f3662955f40 1 -- 172.21.15.154:0/129265330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562db86f1c60 msgr2=0x562db885e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:55.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.667+0000 7f3662955f40 1 --2- 172.21.15.154:0/129265330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562db86f1c60 0x562db885e9f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x562db87d1de0 tx=0x7f365800c730 comp rx=0 tx=0).stop 2023-09-23T03:17:55.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.667+0000 7f3662955f40 1 -- 172.21.15.154:0/129265330 shutdown_connections 2023-09-23T03:17:55.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.667+0000 7f3662955f40 1 --2- 172.21.15.154:0/129265330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562db86f1c60 0x562db885e9f0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:55.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.667+0000 7f3662955f40 1 --2- 172.21.15.154:0/129265330 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562db87af8c0 0x562db86f1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:55.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.667+0000 7f3662955f40 1 --2- 172.21.15.154:0/129265330 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3648002290 0x562db87cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:55.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.667+0000 7f3662955f40 1 -- 172.21.15.154:0/129265330 >> 172.21.15.154:0/129265330 conn(0x562db873cc50 msgr2=0x562db86f07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:55.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.667+0000 7f3662955f40 1 -- 172.21.15.154:0/129265330 shutdown_connections 2023-09-23T03:17:55.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.667+0000 7f3662955f40 1 -- 172.21.15.154:0/129265330 wait complete. 2023-09-23T03:17:55.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.668+0000 7f3662955f40 1 Processor -- start 2023-09-23T03:17:55.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.668+0000 7f3662955f40 1 -- start start 2023-09-23T03:17:55.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.669+0000 7f3662955f40 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562db86f1c60 0x562db86ef880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:55.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.669+0000 7f3662955f40 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562db87af8c0 0x562db87a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:55.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.669+0000 7f3662955f40 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3648002290 0x562db87d3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:55.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.669+0000 7f3662955f40 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x562db86304f0 con 0x562db87af8c0 2023-09-23T03:17:55.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.669+0000 7f3662955f40 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562db87cb790 con 0x562db86f1c60 2023-09-23T03:17:55.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.669+0000 7f3662955f40 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x562db873c7e0 con 0x7f3648002290 2023-09-23T03:17:55.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.669+0000 7f365fc56640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562db87af8c0 0x562db87a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:55.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.669+0000 7f365fc56640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562db87af8c0 0x562db87a94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60102/0 (socket says 172.21.15.154:60102) 2023-09-23T03:17:55.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.669+0000 7f365fc56640 1 -- 172.21.15.154:0/1599494751 learned_addr learned my addr 172.21.15.154:0/1599494751 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:55.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.669+0000 7f3660c58640 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3648002290 0x562db87d3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:55.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.670+0000 7f365fc56640 1 -- 172.21.15.154:0/1599494751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3648002290 msgr2=0x562db87d3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:55.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.670+0000 7f365fc56640 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3648002290 0x562db87d3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:55.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.670+0000 7f3660457640 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562db86f1c60 0x562db86ef880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:55.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.670+0000 7f365fc56640 1 -- 172.21.15.154:0/1599494751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562db86f1c60 msgr2=0x562db86ef880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:55.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.670+0000 7f365fc56640 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562db86f1c60 0x562db86ef880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:55.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.670+0000 7f365fc56640 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562db8648650 con 0x562db87af8c0 2023-09-23T03:17:55.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.670+0000 7f365fc56640 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562db87af8c0 0x562db87a94d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3654009ec0 tx=0x7f36540073f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:55.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.670+0000 7f365dc52640 1 -- 172.21.15.154:0/1599494751 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3654006790 con 0x562db87af8c0 2023-09-23T03:17:55.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.670+0000 7f365dc52640 1 -- 172.21.15.154:0/1599494751 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36540068f0 con 0x562db87af8c0 2023-09-23T03:17:55.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.670+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562db863b260 con 0x562db87af8c0 2023-09-23T03:17:55.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.670+0000 7f365dc52640 1 -- 172.21.15.154:0/1599494751 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3654006ca0 con 0x562db87af8c0 2023-09-23T03:17:55.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.670+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562db87d3100 con 0x562db87af8c0 2023-09-23T03:17:55.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.672+0000 7f365dc52640 1 -- 172.21.15.154:0/1599494751 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f3654015400 con 0x562db87af8c0 2023-09-23T03:17:55.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.673+0000 7f365dc52640 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f364c069910 0x7f364c06bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:55.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.673+0000 7f365dc52640 1 -- 172.21.15.154:0/1599494751 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6519+0+0 (secure 0 0 0) 0x7f3654088230 con 0x562db87af8c0 2023-09-23T03:17:55.675 INFO:teuthology.orchestra.run.smithi154.stdout:Warning: using slow linear search 2023-09-23T03:17:55.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.674+0000 7f3660457640 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f364c069910 0x7f364c06bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:55.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.674+0000 7f3662955f40 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x562db86efdc0 0x562db8873390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:55.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.674+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:1 5.0 5:00000000:%01:::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x562db873b5f0 con 0x562db86efdc0 2023-09-23T03:17:55.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.674+0000 7f3660c58640 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x562db86efdc0 0x562db8873390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:55.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.674+0000 7f3660457640 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f364c069910 0x7f364c06bdd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3650053000 tx=0x7f365004c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:55.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.675+0000 7f3660c58640 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x562db86efdc0 0x562db8873390 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:55.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.675+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(1 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f365800e5b0 con 0x562db86efdc0 2023-09-23T03:17:55.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.675+0000 7f3660c58640 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x562db87b74e0 0x7f3658019950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:55.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.675+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:2 5.10 5:08000000:%01:::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x562db873b5f0 con 0x562db87b74e0 2023-09-23T03:17:55.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.676+0000 7f3660457640 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x562db87b74e0 0x7f3658019950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:55.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.677+0000 7f3660457640 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x562db87b74e0 0x7f3658019950 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:55.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.684+0000 7f3660457640 1 -- 172.21.15.154:0/1599494751 <== osd.7 v2:172.21.15.190:6824/3460136272 1 ==== osd_op_reply(2 [pgnls start_epoch 42 out=177b] v41'10 uv10 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x7f3650006190 con 0x562db87b74e0 2023-09-23T03:17:55.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.684+0000 7f3660457640 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x7f3650005440 0x7f3650005860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:55.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.684+0000 7f3660457640 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:3 5.8 5:10000000:%01:::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x562db873b5f0 con 0x7f3650005440 2023-09-23T03:17:55.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.684+0000 7f365fc56640 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x7f3650005440 0x7f3650005860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:55.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.685+0000 7f365fc56640 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x7f3650005440 0x7f3650005860 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:55.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.685+0000 7f365fc56640 1 -- 172.21.15.154:0/1599494751 <== osd.2 v2:172.21.15.154:6822/2884407243 1 ==== osd_op_reply(3 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f36540864a0 con 0x7f3650005440 2023-09-23T03:17:55.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.685+0000 7f365fc56640 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x7f3654007c20 0x7f365405ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:55.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.685+0000 7f365fc56640 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:4 5.18 5:18000000:%01:::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x562db873b5f0 con 0x7f3654007c20 2023-09-23T03:17:55.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.686+0000 7f3660c58640 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x7f3654007c20 0x7f365405ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:55.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.687+0000 7f3660c58640 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x7f3654007c20 0x7f365405ba80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:55.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.692+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 <== osd.4 v2:172.21.15.190:6808/717302641 1 ==== osd_op_reply(4 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3658023040 con 0x7f3654007c20 2023-09-23T03:17:55.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.692+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:5 5.4 5:20000000:%01:::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x562db873b5f0 con 0x562db87b74e0 2023-09-23T03:17:55.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.694+0000 7f3660457640 1 -- 172.21.15.154:0/1599494751 <== osd.7 v2:172.21.15.190:6824/3460136272 2 ==== osd_op_reply(5 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3650006190 con 0x562db87b74e0 2023-09-23T03:17:55.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.694+0000 7f3660457640 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:6 5.14 5:28000000:%01:::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x562db873b5f0 con 0x562db86efdc0 2023-09-23T03:17:55.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.694+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 <== osd.3 v2:172.21.15.154:6808/1133688468 2 ==== osd_op_reply(6 [pgnls start_epoch 42 out=149b] v41'34 uv34 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x7f3658023040 con 0x562db86efdc0 2023-09-23T03:17:55.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.694+0000 7f3660c58640 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x562db87b78c0 0x7f365800a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:55.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.694+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:7 5.c 5:30000000:%01:::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x562db873b5f0 con 0x562db87b78c0 2023-09-23T03:17:55.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.695+0000 7f3660457640 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x562db87b78c0 0x7f365800a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:55.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.695+0000 7f3660457640 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x562db87b78c0 0x7f365800a400 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:55.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.696+0000 7f3660457640 1 -- 172.21.15.154:0/1599494751 <== osd.1 v2:172.21.15.154:6816/1267710005 1 ==== osd_op_reply(7 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3650075040 con 0x562db87b78c0 2023-09-23T03:17:55.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.696+0000 7f3660457640 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:8 5.1c 5:38000000:%01:::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x562db873b5f0 con 0x7f3654007c20 2023-09-23T03:17:55.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.697+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 <== osd.4 v2:172.21.15.190:6808/717302641 2 ==== osd_op_reply(8 [pgnls start_epoch 42 out=73b] v37'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f3658023040 con 0x7f3654007c20 2023-09-23T03:17:55.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.697+0000 7f3660c58640 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x562db87d1930 0x7f36580283a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:55.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.697+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:9 5.2 5:40000000:%01:::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x562db873b5f0 con 0x562db87d1930 2023-09-23T03:17:55.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.698+0000 7f365fc56640 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x562db87d1930 0x7f36580283a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:55.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.699+0000 7f365fc56640 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x562db87d1930 0x7f36580283a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:55.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.700+0000 7f365fc56640 1 -- 172.21.15.154:0/1599494751 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== osd_op_reply(9 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f36540864a0 con 0x562db87d1930 2023-09-23T03:17:55.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.700+0000 7f365fc56640 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:10 5.12 5:48000000:%01:::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x562db873b5f0 con 0x562db87b78c0 2023-09-23T03:17:55.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.701+0000 7f3660457640 1 -- 172.21.15.154:0/1599494751 <== osd.1 v2:172.21.15.154:6816/1267710005 2 ==== osd_op_reply(10 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3650075040 con 0x562db87b78c0 2023-09-23T03:17:55.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.701+0000 7f3660457640 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:11 5.a 5:50000000:%01:::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x562db873b5f0 con 0x7f3650005440 2023-09-23T03:17:55.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.702+0000 7f365fc56640 1 -- 172.21.15.154:0/1599494751 <== osd.2 v2:172.21.15.154:6822/2884407243 2 ==== osd_op_reply(11 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f36540864a0 con 0x7f3650005440 2023-09-23T03:17:55.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.702+0000 7f365fc56640 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:12 5.1a 5:58000000:%01:::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x562db873b5f0 con 0x562db87b74e0 2023-09-23T03:17:55.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.703+0000 7f3660457640 1 -- 172.21.15.154:0/1599494751 <== osd.7 v2:172.21.15.190:6824/3460136272 3 ==== osd_op_reply(12 [pgnls start_epoch 42 out=49b] v37'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3650075040 con 0x562db87b74e0 2023-09-23T03:17:55.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.703+0000 7f3660457640 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:13 5.6 5:60000000:%01:::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x562db873b5f0 con 0x7f3650005440 2023-09-23T03:17:55.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.704+0000 7f365fc56640 1 -- 172.21.15.154:0/1599494751 <== osd.2 v2:172.21.15.154:6822/2884407243 3 ==== osd_op_reply(13 [pgnls start_epoch 42 out=77b] v41'7 uv7 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x7f36540864a0 con 0x7f3650005440 2023-09-23T03:17:55.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.704+0000 7f365fc56640 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x7f3654082040 0x7f36540867f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:55.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.704+0000 7f365fc56640 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:14 5.16 5:68000000:%01:::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x562db873b5f0 con 0x7f3654082040 2023-09-23T03:17:55.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.704+0000 7f3660c58640 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x7f3654082040 0x7f36540867f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:55.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.706+0000 7f3660c58640 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x7f3654082040 0x7f36540867f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:55.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.712+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 <== osd.5 v2:172.21.15.190:6800/882536885 1 ==== osd_op_reply(14 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3658023040 con 0x7f3654082040 2023-09-23T03:17:55.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.712+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:15 5.e 5:70000000:%01:::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x562db873b5f0 con 0x7f3654007c20 2023-09-23T03:17:55.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.714+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 <== osd.4 v2:172.21.15.190:6808/717302641 3 ==== osd_op_reply(15 [pgnls start_epoch 42 out=73b] v37'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f3658023040 con 0x7f3654007c20 2023-09-23T03:17:55.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.714+0000 7f3660c58640 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x7f365803a040 0x7f365803a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:55.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.714+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:16 5.1e 5:78000000:%01:::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x562db873b5f0 con 0x7f365803a040 2023-09-23T03:17:55.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.714+0000 7f3660457640 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x7f365803a040 0x7f365803a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:55.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.715+0000 7f3660457640 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x7f365803a040 0x7f365803a930 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:55.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.715+0000 7f3660457640 1 -- 172.21.15.154:0/1599494751 <== osd.0 v2:172.21.15.154:6800/2920938138 1 ==== osd_op_reply(16 [pgnls start_epoch 42 out=49b] v37'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3650075040 con 0x7f365803a040 2023-09-23T03:17:55.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.715+0000 7f3660457640 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:17 5.1 5:80000000:%01:::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x562db873b5f0 con 0x7f3654007c20 2023-09-23T03:17:55.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.716+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 <== osd.4 v2:172.21.15.190:6808/717302641 4 ==== osd_op_reply(17 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3658023040 con 0x7f3654007c20 2023-09-23T03:17:55.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.717+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:18 5.11 5:88000000:%01:::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x562db873b5f0 con 0x562db87d1930 2023-09-23T03:17:55.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.718+0000 7f365fc56640 1 -- 172.21.15.154:0/1599494751 <== osd.6 v2:172.21.15.190:6816/1964294001 2 ==== osd_op_reply(18 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f36540864a0 con 0x562db87d1930 2023-09-23T03:17:55.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.718+0000 7f365fc56640 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:19 5.9 5:90000000:%01:::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x562db873b5f0 con 0x562db87b74e0 2023-09-23T03:17:55.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.719+0000 7f3660457640 1 -- 172.21.15.154:0/1599494751 <== osd.7 v2:172.21.15.190:6824/3460136272 4 ==== osd_op_reply(19 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3650075040 con 0x562db87b74e0 2023-09-23T03:17:55.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.719+0000 7f3660457640 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:20 5.19 5:98000000:%01:::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x562db873b5f0 con 0x562db87b78c0 2023-09-23T03:17:55.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.720+0000 7f3660457640 1 -- 172.21.15.154:0/1599494751 <== osd.1 v2:172.21.15.154:6816/1267710005 3 ==== osd_op_reply(20 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3650075040 con 0x562db87b78c0 2023-09-23T03:17:55.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.720+0000 7f3660457640 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:21 5.5 5:a0000000:%01:::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x562db873b5f0 con 0x7f365803a040 2023-09-23T03:17:55.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.721+0000 7f3660457640 1 -- 172.21.15.154:0/1599494751 <== osd.0 v2:172.21.15.154:6800/2920938138 2 ==== osd_op_reply(21 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3650075040 con 0x7f365803a040 2023-09-23T03:17:55.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.721+0000 7f3660457640 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:22 5.15 5:a8000000:%01:::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x562db873b5f0 con 0x7f3654082040 2023-09-23T03:17:55.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.722+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 <== osd.5 v2:172.21.15.190:6800/882536885 2 ==== osd_op_reply(22 [pgnls start_epoch 42 out=97b] v37'3 uv3 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x7f3658023040 con 0x7f3654082040 2023-09-23T03:17:55.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.722+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:23 5.d 5:b0000000:%01:::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x562db873b5f0 con 0x7f3650005440 2023-09-23T03:17:55.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.723+0000 7f365fc56640 1 -- 172.21.15.154:0/1599494751 <== osd.2 v2:172.21.15.154:6822/2884407243 4 ==== osd_op_reply(23 [pgnls start_epoch 42 out=74b] v37'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f36540864a0 con 0x7f3650005440 2023-09-23T03:17:55.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.723+0000 7f365fc56640 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:24 5.1d 5:b8000000:%01:::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x562db873b5f0 con 0x562db87b78c0 2023-09-23T03:17:55.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.723+0000 7f3660457640 1 -- 172.21.15.154:0/1599494751 <== osd.1 v2:172.21.15.154:6816/1267710005 4 ==== osd_op_reply(24 [pgnls start_epoch 42 out=73b] v37'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f3650075040 con 0x562db87b78c0 2023-09-23T03:17:55.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.723+0000 7f3660457640 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:25 5.3 5:c0000000:%01:::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x562db873b5f0 con 0x7f365803a040 2023-09-23T03:17:55.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.724+0000 7f3660457640 1 -- 172.21.15.154:0/1599494751 <== osd.0 v2:172.21.15.154:6800/2920938138 3 ==== osd_op_reply(25 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3650075040 con 0x7f365803a040 2023-09-23T03:17:55.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.724+0000 7f3660457640 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:26 5.13 5:c8000000:%01:::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x562db873b5f0 con 0x562db86efdc0 2023-09-23T03:17:55.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.725+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 <== osd.3 v2:172.21.15.154:6808/1133688468 3 ==== osd_op_reply(26 [pgnls start_epoch 42 out=98b] v41'3 uv3 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x7f3658023040 con 0x562db86efdc0 2023-09-23T03:17:55.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.725+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:27 5.b 5:d0000000:%01:::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x562db873b5f0 con 0x7f3650005440 2023-09-23T03:17:55.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.726+0000 7f365fc56640 1 -- 172.21.15.154:0/1599494751 <== osd.2 v2:172.21.15.154:6822/2884407243 5 ==== osd_op_reply(27 [pgnls start_epoch 42 out=76b] v41'3 uv3 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x7f36540864a0 con 0x7f3650005440 2023-09-23T03:17:55.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.726+0000 7f365fc56640 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:28 5.1b 5:d8000000:%01:::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x562db873b5f0 con 0x7f3654082040 2023-09-23T03:17:55.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.727+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 <== osd.5 v2:172.21.15.190:6800/882536885 3 ==== osd_op_reply(28 [pgnls start_epoch 42 out=49b] v37'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3658023040 con 0x7f3654082040 2023-09-23T03:17:55.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.727+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:29 5.7 5:e0000000:%01:::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x562db873b5f0 con 0x7f3654082040 2023-09-23T03:17:55.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.729+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 <== osd.5 v2:172.21.15.190:6800/882536885 4 ==== osd_op_reply(29 [pgnls start_epoch 42 out=73b] v37'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f3658023040 con 0x7f3654082040 2023-09-23T03:17:55.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.729+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:30 5.17 5:e8000000:%01:::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x562db873b5f0 con 0x562db86efdc0 2023-09-23T03:17:55.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.729+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 <== osd.3 v2:172.21.15.154:6808/1133688468 4 ==== osd_op_reply(30 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3658023040 con 0x562db86efdc0 2023-09-23T03:17:55.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.729+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:31 5.f 5:f0000000:%01:::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x562db873b5f0 con 0x7f3654082040 2023-09-23T03:17:55.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.730+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 <== osd.5 v2:172.21.15.190:6800/882536885 5 ==== osd_op_reply(31 [pgnls start_epoch 42 out=73b] v37'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f3658023040 con 0x7f3654082040 2023-09-23T03:17:55.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.730+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:32 5.1f 5:f8000000:%01:::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x562db873b5f0 con 0x562db87d1930 2023-09-23T03:17:55.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.732+0000 7f365fc56640 1 -- 172.21.15.154:0/1599494751 <== osd.6 v2:172.21.15.190:6816/1964294001 3 ==== osd_op_reply(32 [pgnls start_epoch 42 out=101b] v41'6 uv6 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x7f36540864a0 con 0x562db87d1930 2023-09-23T03:17:55.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.732+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:33 5.10 5:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e42) v8 -- 0x562db886a540 con 0x562db87b74e0 2023-09-23T03:17:55.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.732+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:34 5.10 5:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e42) v8 -- 0x562db886ad70 con 0x562db87b74e0 2023-09-23T03:17:55.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.732+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:35 5.10 5:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e42) v8 -- 0x562db8873d70 con 0x562db87b74e0 2023-09-23T03:17:55.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.732+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:36 5.10 5:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e42) v8 -- 0x562db8874570 con 0x562db87b74e0 2023-09-23T03:17:55.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.732+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:37 5.10 5:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e42) v8 -- 0x562db8874d70 con 0x562db87b74e0 2023-09-23T03:17:55.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.732+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:38 5.14 5:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e42) v8 -- 0x562db8875570 con 0x562db86efdc0 2023-09-23T03:17:55.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.732+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:39 5.14 5:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e42) v8 -- 0x562db8875d70 con 0x562db86efdc0 2023-09-23T03:17:55.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.732+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:40 5.14 5:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e42) v8 -- 0x562db8876570 con 0x562db86efdc0 2023-09-23T03:17:55.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.732+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:41 5.14 5:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e42) v8 -- 0x562db8876d70 con 0x562db86efdc0 2023-09-23T03:17:55.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.732+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:42 5.1c 5:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e42) v8 -- 0x562db8877570 con 0x7f3654007c20 2023-09-23T03:17:55.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.732+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:43 5.6 5:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e42) v8 -- 0x562db8877e40 con 0x7f3650005440 2023-09-23T03:17:55.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.732+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:44 5.e 5:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e42) v8 -- 0x562db88788c0 con 0x7f3654007c20 2023-09-23T03:17:55.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.732+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:45 5.15 5:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e42) v8 -- 0x562db8879360 con 0x7f3654082040 2023-09-23T03:17:55.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.732+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:46 5.15 5:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e42) v8 -- 0x562db8879dd0 con 0x7f3654082040 2023-09-23T03:17:55.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.732+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:47 5.d 5:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e42) v8 -- 0x562db887a8c0 con 0x7f3650005440 2023-09-23T03:17:55.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.732+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:48 5.1d 5:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e42) v8 -- 0x562db887b360 con 0x562db87b78c0 2023-09-23T03:17:55.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.739+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 <== osd.4 v2:172.21.15.190:6808/717302641 5 ==== osd_op_reply(42 607.00000000 [delete] v42'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3658023040 con 0x7f3654007c20 2023-09-23T03:17:55.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.739+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:49 5.13 5:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e42) v8 -- 0x562db8879dd0 con 0x562db86efdc0 2023-09-23T03:17:55.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.739+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 <== osd.5 v2:172.21.15.190:6800/882536885 6 ==== osd_op_reply(45 604.00000000 [delete] v42'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3658023040 con 0x7f3654082040 2023-09-23T03:17:55.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.739+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:50 5.13 5:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e42) v8 -- 0x562db8879dd0 con 0x562db86efdc0 2023-09-23T03:17:55.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.740+0000 7f3660457640 1 -- 172.21.15.154:0/1599494751 <== osd.1 v2:172.21.15.154:6816/1267710005 5 ==== osd_op_reply(48 605.00000000 [delete] v42'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3650075040 con 0x562db87b78c0 2023-09-23T03:17:55.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.740+0000 7f365fc56640 1 -- 172.21.15.154:0/1599494751 <== osd.2 v2:172.21.15.154:6822/2884407243 6 ==== osd_op_reply(47 mds_snaptable [delete] v42'2 uv2 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f36540864a0 con 0x7f3650005440 2023-09-23T03:17:55.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.740+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:51 5.b 5:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e42) v8 -- 0x562db8879dd0 con 0x7f3650005440 2023-09-23T03:17:55.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.740+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:52 5.7 5:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e42) v8 -- 0x562db887b740 con 0x7f3654082040 2023-09-23T03:17:55.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.742+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 <== osd.5 v2:172.21.15.190:6800/882536885 7 ==== osd_op_reply(46 500.00000000 [delete] v42'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3658023040 con 0x7f3654082040 2023-09-23T03:17:55.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.742+0000 7f3660457640 1 -- 172.21.15.154:0/1599494751 <== osd.7 v2:172.21.15.190:6824/3460136272 5 ==== osd_op_reply(33 601.00000000 [delete] v42'11 uv11 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3650075040 con 0x562db87b74e0 2023-09-23T03:17:55.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.742+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 <== osd.3 v2:172.21.15.154:6808/1133688468 5 ==== osd_op_reply(38 1.00000000.inode [delete] v42'35 uv35 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f3658023040 con 0x562db86efdc0 2023-09-23T03:17:55.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.742+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 <== osd.3 v2:172.21.15.154:6808/1133688468 6 ==== osd_op_reply(39 200.00000000 [delete] v42'36 uv36 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3658023040 con 0x562db86efdc0 2023-09-23T03:17:55.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.742+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 <== osd.4 v2:172.21.15.190:6808/717302641 6 ==== osd_op_reply(44 608.00000000 [delete] v42'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3658023040 con 0x7f3654007c20 2023-09-23T03:17:55.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.742+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:53 5.f 5:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e42) v8 -- 0x562db88758d0 con 0x7f3654082040 2023-09-23T03:17:55.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.742+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:54 5.1f 5:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e42) v8 -- 0x562db88750d0 con 0x562db87d1930 2023-09-23T03:17:55.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.742+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:55 5.1f 5:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e42) v8 -- 0x562db887b740 con 0x562db87d1930 2023-09-23T03:17:55.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.743+0000 7f365fc56640 1 -- 172.21.15.154:0/1599494751 <== osd.2 v2:172.21.15.154:6822/2884407243 7 ==== osd_op_reply(43 mds0_openfiles.0 [delete] v42'8 uv8 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f36540864a0 con 0x7f3650005440 2023-09-23T03:17:55.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.743+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 <== osd.3 v2:172.21.15.154:6808/1133688468 7 ==== osd_op_reply(40 200.00000001 [delete] v42'37 uv37 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3658023040 con 0x562db86efdc0 2023-09-23T03:17:55.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.743+0000 7f3660457640 1 -- 172.21.15.154:0/1599494751 <== osd.7 v2:172.21.15.190:6824/3460136272 6 ==== osd_op_reply(34 602.00000000 [delete] v42'12 uv12 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3650075040 con 0x562db87b74e0 2023-09-23T03:17:55.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.743+0000 7f3660457640 1 -- 172.21.15.154:0/1599494751 <== osd.7 v2:172.21.15.190:6824/3460136272 7 ==== osd_op_reply(35 10000000000.00000000 [delete] v42'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f3650075040 con 0x562db87b74e0 2023-09-23T03:17:55.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.743+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 <== osd.3 v2:172.21.15.154:6808/1133688468 8 ==== osd_op_reply(41 606.00000000 [delete] v42'38 uv38 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3658023040 con 0x562db86efdc0 2023-09-23T03:17:55.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.744+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 <== osd.3 v2:172.21.15.154:6808/1133688468 9 ==== osd_op_reply(49 mds0_inotable [delete] v42'4 uv4 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3658023040 con 0x562db86efdc0 2023-09-23T03:17:55.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.744+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 <== osd.3 v2:172.21.15.154:6808/1133688468 10 ==== osd_op_reply(50 100.00000000 [delete] v42'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3658023040 con 0x562db86efdc0 2023-09-23T03:17:55.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.745+0000 7f365fc56640 1 -- 172.21.15.154:0/1599494751 <== osd.2 v2:172.21.15.154:6822/2884407243 8 ==== osd_op_reply(51 mds0_sessionmap [delete] v42'4 uv4 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f36540864a0 con 0x7f3650005440 2023-09-23T03:17:55.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.746+0000 7f3660457640 1 -- 172.21.15.154:0/1599494751 <== osd.7 v2:172.21.15.190:6824/3460136272 8 ==== osd_op_reply(36 600.00000000 [delete] v42'14 uv14 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3650075040 con 0x562db87b74e0 2023-09-23T03:17:55.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.746+0000 7f3660457640 1 -- 172.21.15.154:0/1599494751 <== osd.7 v2:172.21.15.190:6824/3460136272 9 ==== osd_op_reply(37 603.00000000 [delete] v42'15 uv15 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3650075040 con 0x562db87b74e0 2023-09-23T03:17:55.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.747+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 <== osd.5 v2:172.21.15.190:6800/882536885 8 ==== osd_op_reply(52 609.00000000 [delete] v42'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3658023040 con 0x7f3654082040 2023-09-23T03:17:55.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.748+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 <== osd.5 v2:172.21.15.190:6800/882536885 9 ==== osd_op_reply(53 400.00000000 [delete] v42'2 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3658023040 con 0x7f3654082040 2023-09-23T03:17:55.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.749+0000 7f365fc56640 1 -- 172.21.15.154:0/1599494751 <== osd.6 v2:172.21.15.190:6816/1964294001 4 ==== osd_op_reply(54 100.00000000.inode [delete] v42'7 uv7 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f36540864a0 con 0x562db87d1930 2023-09-23T03:17:55.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.750+0000 7f365fc56640 1 -- 172.21.15.154:0/1599494751 <== osd.6 v2:172.21.15.190:6816/1964294001 5 ==== osd_op_reply(55 1.00000000 [delete] v42'8 uv8 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f36540864a0 con 0x562db87d1930 2023-09-23T03:17:55.752 INFO:teuthology.orchestra.run.smithi154.stdout:Removed 23 objects 2023-09-23T03:17:55.752 INFO:teuthology.orchestra.run.smithi154.stdout:successfully purged pool cephfs_metadata 2023-09-23T03:17:55.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.750+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x7f365803a040 msgr2=0x7f365803a930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:55.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.750+0000 7f3662955f40 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x7f365803a040 0x7f365803a930 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:55.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.750+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x562db87b78c0 msgr2=0x7f365800a400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:55.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.750+0000 7f3662955f40 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x562db87b78c0 0x7f365800a400 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:55.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.750+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x7f3650005440 msgr2=0x7f3650005860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:55.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.750+0000 7f3662955f40 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x7f3650005440 0x7f3650005860 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:55.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.751+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x562db86efdc0 msgr2=0x562db8873390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:55.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.751+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 reap_dead start 2023-09-23T03:17:55.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.751+0000 7f3662955f40 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x562db86efdc0 0x562db8873390 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:55.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.751+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x7f3654007c20 msgr2=0x7f365405ba80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:55.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.751+0000 7f3662955f40 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x7f3654007c20 0x7f365405ba80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:55.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.751+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x7f3654082040 msgr2=0x7f36540867f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:55.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.751+0000 7f3662955f40 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x7f3654082040 0x7f36540867f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:55.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.751+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x562db87d1930 msgr2=0x7f36580283a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:55.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.751+0000 7f3662955f40 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x562db87d1930 0x7f36580283a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:55.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.751+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x562db87b74e0 msgr2=0x7f3658019950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:55.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.751+0000 7f3662955f40 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x562db87b74e0 0x7f3658019950 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:55.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.751+0000 7f3660c58640 1 -- 172.21.15.154:0/1599494751 reap_dead start 2023-09-23T03:17:55.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.751+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f364c069910 msgr2=0x7f364c06bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:55.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.751+0000 7f3662955f40 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f364c069910 0x7f364c06bdd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3650053000 tx=0x7f365004c000 comp rx=0 tx=0).stop 2023-09-23T03:17:55.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.751+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562db87af8c0 msgr2=0x562db87a94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:55.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.751+0000 7f3662955f40 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562db87af8c0 0x562db87a94d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3654009ec0 tx=0x7f36540073f0 comp rx=0 tx=0).stop 2023-09-23T03:17:55.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.752+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 shutdown_connections 2023-09-23T03:17:55.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.752+0000 7f3662955f40 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f364c069910 0x7f364c06bdd0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:55.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.752+0000 7f3662955f40 1 --2- 172.21.15.154:0/1599494751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562db87af8c0 0x562db87a94d0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:55.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.752+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 >> 172.21.15.154:0/1599494751 conn(0x562db873cc50 msgr2=0x562db86f0740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:55.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.752+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 shutdown_connections 2023-09-23T03:17:55.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.752+0000 7f3662955f40 1 -- 172.21.15.154:0/1599494751 wait complete. 2023-09-23T03:17:55.761 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs reset cephfs --yes-i-really-mean-it 2023-09-23T03:17:55.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.970+0000 7faec3c87640 1 Processor -- start 2023-09-23T03:17:55.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.970+0000 7faec3c87640 1 -- start start 2023-09-23T03:17:55.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.971+0000 7faec3c87640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7faebc15bb30 0x7faebc0caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:55.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.972+0000 7faec3c87640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faebc0cb3e0 0x7faebc0cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:55.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.972+0000 7faec3c87640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faebc15b6a0 con 0x7faebc15aba0 2023-09-23T03:17:55.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.972+0000 7faec3c87640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faebc0cb870 con 0x7faebc15bb30 2023-09-23T03:17:55.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.972+0000 7faec3c87640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faebc15bf70 con 0x7faebc0cb3e0 2023-09-23T03:17:55.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.972+0000 7faec1a23640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7faebc15aba0 0x7faebc15e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:50640/0 (socket says 172.21.15.154:50640) 2023-09-23T03:17:55.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.972+0000 7faec1a23640 1 -- 172.21.15.154:0/1943687103 learned_addr learned my addr 172.21.15.154:0/1943687103 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:17:55.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.972+0000 7faec2224640 1 --2- 172.21.15.154:0/1943687103 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faebc0cb3e0 0x7faebc0cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:55.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.972+0000 7faec1222640 1 --2- 172.21.15.154:0/1943687103 >> v2:172.21.15.190:3300/0 conn(0x7faebc15bb30 0x7faebc0caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:55.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.972+0000 7faec0a21640 1 -- 172.21.15.154:0/1943687103 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2734425232 0 0) 0x7faebc15b6a0 con 0x7faebc15aba0 2023-09-23T03:17:55.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.972+0000 7faec0a21640 1 -- 172.21.15.154:0/1943687103 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faea8003520 con 0x7faebc15aba0 2023-09-23T03:17:55.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.973+0000 7faec0a21640 1 -- 172.21.15.154:0/1943687103 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 897549838 0 0) 0x7faea8003520 con 0x7faebc15aba0 2023-09-23T03:17:55.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.973+0000 7faec0a21640 1 -- 172.21.15.154:0/1943687103 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faebc0cb3e0 msgr2=0x7faebc0cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:55.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.973+0000 7faec0a21640 1 --2- 172.21.15.154:0/1943687103 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faebc0cb3e0 0x7faebc0cc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:55.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.973+0000 7faec0a21640 1 -- 172.21.15.154:0/1943687103 >> v2:172.21.15.190:3300/0 conn(0x7faebc15bb30 msgr2=0x7faebc0caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:55.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.973+0000 7faec0a21640 1 --2- 172.21.15.154:0/1943687103 >> v2:172.21.15.190:3300/0 conn(0x7faebc15bb30 0x7faebc0caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:55.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.973+0000 7faec0a21640 1 -- 172.21.15.154:0/1943687103 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faebc15f0b0 con 0x7faebc15aba0 2023-09-23T03:17:55.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.973+0000 7faec0a21640 1 -- 172.21.15.154:0/1943687103 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7faeb80019f0 con 0x7faebc15aba0 2023-09-23T03:17:55.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.973+0000 7faec0a21640 1 -- 172.21.15.154:0/1943687103 >> v1:172.21.15.154:6789/0 conn(0x7faebc15aba0 legacy=0x7faebc15e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:55.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.973+0000 7faec0a21640 1 --2- 172.21.15.154:0/1943687103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faea8003970 0x7faea8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:55.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.973+0000 7faec0a21640 1 --2- 172.21.15.154:0/1943687103 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faebc0cb3e0 0x7faea80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:55.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.973+0000 7faec0a21640 1 --2- 172.21.15.154:0/1943687103 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faea8004d00 0x7faea8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:55.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.973+0000 7faec0a21640 1 -- 172.21.15.154:0/1943687103 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faebc0cb870 con 0x7faea8004d00 2023-09-23T03:17:55.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.973+0000 7faec0a21640 1 -- 172.21.15.154:0/1943687103 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faebc15bf70 con 0x7faea8003970 2023-09-23T03:17:55.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.973+0000 7faec0a21640 1 -- 172.21.15.154:0/1943687103 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faea800ba30 con 0x7faebc0cb3e0 2023-09-23T03:17:55.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.973+0000 7faec2224640 1 --2- 172.21.15.154:0/1943687103 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faea8004d00 0x7faea8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:55.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.974+0000 7faec1a23640 1 --2- 172.21.15.154:0/1943687103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faea8003970 0x7faea8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:55.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.974+0000 7faec2224640 1 -- 172.21.15.154:0/1943687103 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faebc0cb3e0 msgr2=0x7faea80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:55.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.974+0000 7faec2224640 1 --2- 172.21.15.154:0/1943687103 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faebc0cb3e0 0x7faea80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:55.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.974+0000 7faec2224640 1 -- 172.21.15.154:0/1943687103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faea8003970 msgr2=0x7faea8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:55.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.974+0000 7faec2224640 1 --2- 172.21.15.154:0/1943687103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faea8003970 0x7faea8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:55.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.974+0000 7faec2224640 1 -- 172.21.15.154:0/1943687103 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea8003520 con 0x7faea8004d00 2023-09-23T03:17:55.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.975+0000 7faec2224640 1 --2- 172.21.15.154:0/1943687103 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faea8004d00 0x7faea8005180 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7faeb4009ac0 tx=0x7faea80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f4d940a1f4820c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:55.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.975+0000 7faec0a21640 1 -- 172.21.15.154:0/1943687103 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb402c6f0 con 0x7faea8004d00 2023-09-23T03:17:55.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.975+0000 7faec0a21640 1 -- 172.21.15.154:0/1943687103 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb402c850 con 0x7faea8004d00 2023-09-23T03:17:55.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.975+0000 7faec0a21640 1 -- 172.21.15.154:0/1943687103 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb402cbe0 con 0x7faea8004d00 2023-09-23T03:17:55.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.976+0000 7faec3c87640 1 -- 172.21.15.154:0/1943687103 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faea8004d00 msgr2=0x7faea8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:55.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.976+0000 7faec3c87640 1 --2- 172.21.15.154:0/1943687103 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faea8004d00 0x7faea8005180 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7faeb4009ac0 tx=0x7faea80057f0 comp rx=0 tx=0).stop 2023-09-23T03:17:55.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.976+0000 7faec2224640 1 -- 172.21.15.154:0/1943687103 reap_dead start 2023-09-23T03:17:55.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.976+0000 7faec3c87640 1 -- 172.21.15.154:0/1943687103 shutdown_connections 2023-09-23T03:17:55.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.976+0000 7faec3c87640 1 -- 172.21.15.154:0/1943687103 >> 172.21.15.154:0/1943687103 conn(0x7faebc06c010 msgr2=0x7faebc06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:55.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.976+0000 7faec3c87640 1 -- 172.21.15.154:0/1943687103 shutdown_connections 2023-09-23T03:17:55.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.976+0000 7faec3c87640 1 -- 172.21.15.154:0/1943687103 wait complete. 2023-09-23T03:17:55.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.978+0000 7faec3c87640 1 Processor -- start 2023-09-23T03:17:55.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.978+0000 7faec3c87640 1 -- start start 2023-09-23T03:17:55.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.979+0000 7faec3c87640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faea8004d00 0x7faebc154970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:55.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.979+0000 7faec3c87640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faebc0c9a60 0x7faebc0c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:55.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.979+0000 7faec3c87640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faebc1568c0 0x7faebc156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:55.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.979+0000 7faec3c87640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faebc155820 con 0x7faebc1568c0 2023-09-23T03:17:55.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.979+0000 7faec3c87640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faebc155b00 con 0x7faea8004d00 2023-09-23T03:17:55.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.979+0000 7faec3c87640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faebc155de0 con 0x7faebc0c9a60 2023-09-23T03:17:55.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.979+0000 7faec2224640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faebc1568c0 0x7faebc156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:55.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.979+0000 7faec2224640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faebc1568c0 0x7faebc156d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60130/0 (socket says 172.21.15.154:60130) 2023-09-23T03:17:55.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.979+0000 7faec2224640 1 -- 172.21.15.154:0/3048156500 learned_addr learned my addr 172.21.15.154:0/3048156500 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:55.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.980+0000 7faec1a23640 1 --2- 172.21.15.154:0/3048156500 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faea8004d00 0x7faebc154970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:55.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.980+0000 7faec1222640 1 --2- 172.21.15.154:0/3048156500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faebc0c9a60 0x7faebc0c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:55.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.980+0000 7faec2224640 1 -- 172.21.15.154:0/3048156500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faebc0c9a60 msgr2=0x7faebc0c9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:55.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.980+0000 7faec2224640 1 --2- 172.21.15.154:0/3048156500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faebc0c9a60 0x7faebc0c9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:55.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.980+0000 7faec2224640 1 -- 172.21.15.154:0/3048156500 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faea8004d00 msgr2=0x7faebc154970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:55.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.980+0000 7faec2224640 1 --2- 172.21.15.154:0/3048156500 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faea8004d00 0x7faebc154970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:55.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.980+0000 7faec2224640 1 -- 172.21.15.154:0/3048156500 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faebc1560c0 con 0x7faebc1568c0 2023-09-23T03:17:55.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.980+0000 7faec1222640 1 --2- 172.21.15.154:0/3048156500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faebc0c9a60 0x7faebc0c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:55.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.980+0000 7faec2224640 1 --2- 172.21.15.154:0/3048156500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faebc1568c0 0x7faebc156d40 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7faeb4037040 tx=0x7faeb4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:55.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.980+0000 7faeb27fc640 1 -- 172.21.15.154:0/3048156500 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb4003810 con 0x7faebc1568c0 2023-09-23T03:17:55.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.981+0000 7faeb27fc640 1 -- 172.21.15.154:0/3048156500 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb4003970 con 0x7faebc1568c0 2023-09-23T03:17:55.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.981+0000 7faeb27fc640 1 -- 172.21.15.154:0/3048156500 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb4003d20 con 0x7faebc1568c0 2023-09-23T03:17:55.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.981+0000 7faec3c87640 1 -- 172.21.15.154:0/3048156500 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faebc1563a0 con 0x7faebc1568c0 2023-09-23T03:17:55.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.981+0000 7faec3c87640 1 -- 172.21.15.154:0/3048156500 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faebc1554b0 con 0x7faebc1568c0 2023-09-23T03:17:55.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.982+0000 7faec3c87640 1 -- 172.21.15.154:0/3048156500 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faebc04e920 con 0x7faebc1568c0 2023-09-23T03:17:55.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.983+0000 7faeb27fc640 1 -- 172.21.15.154:0/3048156500 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7faeb403e440 con 0x7faebc1568c0 2023-09-23T03:17:55.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.984+0000 7faeb27fc640 1 --2- 172.21.15.154:0/3048156500 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7faea006af30 0x7faea006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:55.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.984+0000 7faeb27fc640 1 -- 172.21.15.154:0/3048156500 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6519+0+0 (secure 0 0 0) 0x7faeb4000e40 con 0x7faebc1568c0 2023-09-23T03:17:55.986 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.985+0000 7faec1a23640 1 --2- 172.21.15.154:0/3048156500 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7faea006af30 0x7faea006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:55.986 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.985+0000 7faec1a23640 1 --2- 172.21.15.154:0/3048156500 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7faea006af30 0x7faea006d3f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7faeb80013d0 tx=0x7faeb8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:55.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:55.989+0000 7faeb27fc640 1 -- 172.21.15.154:0/3048156500 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7faeb4075d90 con 0x7faebc1568c0 2023-09-23T03:17:56.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:56.219+0000 7faec3c87640 1 -- 172.21.15.154:0/3048156500 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs reset", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7faebc15db10 con 0x7faebc1568c0 2023-09-23T03:17:57.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.188+0000 7faeb27fc640 1 -- 172.21.15.154:0/3048156500 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs reset", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v22) v1 ==== 107+0+0 (secure 0 0 0) 0x7faeb40390b0 con 0x7faebc1568c0 2023-09-23T03:17:57.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.191+0000 7faec3c87640 1 -- 172.21.15.154:0/3048156500 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7faea006af30 msgr2=0x7faea006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:57.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.191+0000 7faec3c87640 1 --2- 172.21.15.154:0/3048156500 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7faea006af30 0x7faea006d3f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7faeb80013d0 tx=0x7faeb8009040 comp rx=0 tx=0).stop 2023-09-23T03:17:57.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.191+0000 7faec3c87640 1 -- 172.21.15.154:0/3048156500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faebc1568c0 msgr2=0x7faebc156d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:57.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.191+0000 7faec3c87640 1 --2- 172.21.15.154:0/3048156500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faebc1568c0 0x7faebc156d40 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7faeb4037040 tx=0x7faeb4004730 comp rx=0 tx=0).stop 2023-09-23T03:17:57.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.192+0000 7faec3c87640 1 -- 172.21.15.154:0/3048156500 shutdown_connections 2023-09-23T03:17:57.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.192+0000 7faec3c87640 1 --2- 172.21.15.154:0/3048156500 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7faea006af30 0x7faea006d3f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:57.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.192+0000 7faec3c87640 1 --2- 172.21.15.154:0/3048156500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faebc1568c0 0x7faebc156d40 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:57.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.192+0000 7faec3c87640 1 --2- 172.21.15.154:0/3048156500 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faebc0c9a60 0x7faebc0c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:57.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.192+0000 7faec3c87640 1 --2- 172.21.15.154:0/3048156500 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faea8004d00 0x7faebc154970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:57.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.192+0000 7faec3c87640 1 -- 172.21.15.154:0/3048156500 >> 172.21.15.154:0/3048156500 conn(0x7faebc06c010 msgr2=0x7faebc0cc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:57.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.192+0000 7faec3c87640 1 -- 172.21.15.154:0/3048156500 shutdown_connections 2023-09-23T03:17:57.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.192+0000 7faec3c87640 1 -- 172.21.15.154:0/3048156500 wait complete. 2023-09-23T03:17:57.205 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:17:57.204+0000 7fad8b52c640 -1 mds.0.inotable: load_2 could not read table: -2 2023-09-23T03:17:57.205 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:17:57.204+0000 7fad8b52c640 -1 log_channel(cluster) log [ERR] : error reading table object 'mds0_inotable' -2 ((2) No such file or directory) 2023-09-23T03:17:57.206 INFO:tasks.ceph.mon.a.smithi154.stderr:2023-09-23T03:17:57.205+0000 7f3df35a6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-23T03:17:57.206 INFO:tasks.ceph.mon.a.smithi154.stderr:2023-09-23T03:17:57.205+0000 7f3df35a6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-09-23T03:17:57.215 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-09-23T03:17:57.222 INFO:tasks.ceph.mds.a.smithi154.stderr: -32> 2023-09-23T03:17:57.204+0000 7fad8b52c640 -1 mds.0.inotable: load_2 could not read table: -2 2023-09-23T03:17:57.222 INFO:tasks.ceph.mds.a.smithi154.stderr: -31> 2023-09-23T03:17:57.204+0000 7fad8b52c640 -1 log_channel(cluster) log [ERR] : error reading table object 'mds0_inotable' -2 ((2) No such file or directory) 2023-09-23T03:17:57.267 INFO:tasks.ceph.mds.a.smithi154.stdout:starting mds.a at 2023-09-23T03:17:57.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.407+0000 7f463a7ce640 1 Processor -- start 2023-09-23T03:17:57.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.407+0000 7f463a7ce640 1 -- start start 2023-09-23T03:17:57.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.408+0000 7f463a7ce640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f463415bb30 0x7f46340caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:57.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.408+0000 7f463a7ce640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f46340cb3e0 0x7f46340cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:57.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.408+0000 7f463a7ce640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f463415b6a0 con 0x7f463415aba0 2023-09-23T03:17:57.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.408+0000 7f463a7ce640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46340cb870 con 0x7f463415bb30 2023-09-23T03:17:57.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.408+0000 7f463a7ce640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f463415bf70 con 0x7f46340cb3e0 2023-09-23T03:17:57.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.409+0000 7f462bfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f463415bb30 0x7f46340caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:57.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.409+0000 7f462bfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f463415bb30 0x7f46340caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60154/0 (socket says 172.21.15.154:60154) 2023-09-23T03:17:57.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.409+0000 7f462bfff640 1 -- 172.21.15.154:0/4240177050 learned_addr learned my addr 172.21.15.154:0/4240177050 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:57.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.409+0000 7f4638d6b640 1 --2- 172.21.15.154:0/4240177050 >> v2:172.21.15.190:3300/0 conn(0x7f46340cb3e0 0x7f46340cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:57.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.409+0000 7f462bfff640 1 -- 172.21.15.154:0/4240177050 >> v2:172.21.15.190:3300/0 conn(0x7f46340cb3e0 msgr2=0x7f46340cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:57.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.409+0000 7f462bfff640 1 --2- 172.21.15.154:0/4240177050 >> v2:172.21.15.190:3300/0 conn(0x7f46340cb3e0 0x7f46340cc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:57.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.409+0000 7f462bfff640 1 -- 172.21.15.154:0/4240177050 >> v1:172.21.15.190:6789/0 conn(0x7f463415aba0 legacy=0x7f463415e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:57.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.409+0000 7f462bfff640 1 -- 172.21.15.154:0/4240177050 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f463415f0b0 con 0x7f463415bb30 2023-09-23T03:17:57.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.410+0000 7f462bfff640 1 --2- 172.21.15.154:0/4240177050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f463415bb30 0x7f46340caea0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f46340c1f80 tx=0x7f461c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5454ce51c64d472 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:57.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.411+0000 7f46337fe640 1 -- 172.21.15.154:0/4240177050 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c031870 con 0x7f463415bb30 2023-09-23T03:17:57.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.411+0000 7f46337fe640 1 -- 172.21.15.154:0/4240177050 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f461c0319d0 con 0x7f463415bb30 2023-09-23T03:17:57.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.411+0000 7f46337fe640 1 -- 172.21.15.154:0/4240177050 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c031d60 con 0x7f463415bb30 2023-09-23T03:17:57.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.411+0000 7f463a7ce640 1 -- 172.21.15.154:0/4240177050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f463415bb30 msgr2=0x7f46340caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:57.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.411+0000 7f463a7ce640 1 --2- 172.21.15.154:0/4240177050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f463415bb30 0x7f46340caea0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f46340c1f80 tx=0x7f461c02b910 comp rx=0 tx=0).stop 2023-09-23T03:17:57.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.412+0000 7f463a7ce640 1 -- 172.21.15.154:0/4240177050 shutdown_connections 2023-09-23T03:17:57.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.412+0000 7f463a7ce640 1 --2- 172.21.15.154:0/4240177050 >> v2:172.21.15.190:3300/0 conn(0x7f46340cb3e0 0x7f46340cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:57.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.412+0000 7f463a7ce640 1 --2- 172.21.15.154:0/4240177050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f463415bb30 0x7f46340caea0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:57.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.412+0000 7f463a7ce640 1 -- 172.21.15.154:0/4240177050 >> 172.21.15.154:0/4240177050 conn(0x7f463406c010 msgr2=0x7f463406c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:57.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.412+0000 7f463a7ce640 1 -- 172.21.15.154:0/4240177050 shutdown_connections 2023-09-23T03:17:57.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.412+0000 7f463a7ce640 1 -- 172.21.15.154:0/4240177050 wait complete. 2023-09-23T03:17:57.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.413+0000 7f463a7ce640 1 Processor -- start 2023-09-23T03:17:57.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.414+0000 7f463a7ce640 1 -- start start 2023-09-23T03:17:57.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.414+0000 7f463a7ce640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46340cb3e0 0x7f4634155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:57.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.414+0000 7f463a7ce640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f463415aba0 0x7f4634153880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:57.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.415+0000 7f463a7ce640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f463415bb30 0x7f463414e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:57.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.415+0000 7f463a7ce640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4634153dc0 con 0x7f46340cb3e0 2023-09-23T03:17:57.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.415+0000 7f463a7ce640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463414ef00 con 0x7f463415bb30 2023-09-23T03:17:57.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.415+0000 7f463a7ce640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46341f1400 con 0x7f463415aba0 2023-09-23T03:17:57.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.415+0000 7f4633fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46340cb3e0 0x7f4634155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:57.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.415+0000 7f4633fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46340cb3e0 0x7f4634155fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60170/0 (socket says 172.21.15.154:60170) 2023-09-23T03:17:57.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.415+0000 7f4633fff640 1 -- 172.21.15.154:0/2591028603 learned_addr learned my addr 172.21.15.154:0/2591028603 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:57.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.415+0000 7f4638d6b640 1 --2- 172.21.15.154:0/2591028603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f463415bb30 0x7f463414e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:57.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.415+0000 7f462bfff640 1 --2- 172.21.15.154:0/2591028603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f463415aba0 0x7f4634153880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:57.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.415+0000 7f4633fff640 1 -- 172.21.15.154:0/2591028603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f463415aba0 msgr2=0x7f4634153880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:57.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.415+0000 7f4633fff640 1 --2- 172.21.15.154:0/2591028603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f463415aba0 0x7f4634153880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:57.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.415+0000 7f4633fff640 1 -- 172.21.15.154:0/2591028603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f463415bb30 msgr2=0x7f463414e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:57.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.415+0000 7f4633fff640 1 --2- 172.21.15.154:0/2591028603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f463415bb30 0x7f463414e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:57.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.415+0000 7f4633fff640 1 -- 172.21.15.154:0/2591028603 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461c0094d0 con 0x7f46340cb3e0 2023-09-23T03:17:57.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.415+0000 7f462bfff640 1 --2- 172.21.15.154:0/2591028603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f463415aba0 0x7f4634153880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:57.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.416+0000 7f4633fff640 1 --2- 172.21.15.154:0/2591028603 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46340cb3e0 0x7f4634155fd0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f462400c460 tx=0x7f462400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:57.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.416+0000 7f46317fa640 1 -- 172.21.15.154:0/2591028603 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4624012830 con 0x7f46340cb3e0 2023-09-23T03:17:57.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.416+0000 7f46317fa640 1 -- 172.21.15.154:0/2591028603 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46240129d0 con 0x7f46340cb3e0 2023-09-23T03:17:57.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.416+0000 7f463a7ce640 1 -- 172.21.15.154:0/2591028603 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46341f16a0 con 0x7f46340cb3e0 2023-09-23T03:17:57.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.416+0000 7f46317fa640 1 -- 172.21.15.154:0/2591028603 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4624012d80 con 0x7f46340cb3e0 2023-09-23T03:17:57.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.416+0000 7f463a7ce640 1 -- 172.21.15.154:0/2591028603 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46341f1c60 con 0x7f46340cb3e0 2023-09-23T03:17:57.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.417+0000 7f463a7ce640 1 -- 172.21.15.154:0/2591028603 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f463404e920 con 0x7f46340cb3e0 2023-09-23T03:17:57.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.418+0000 7f46317fa640 1 -- 172.21.15.154:0/2591028603 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f462401a3b0 con 0x7f46340cb3e0 2023-09-23T03:17:57.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.419+0000 7f46317fa640 1 --2- 172.21.15.154:0/2591028603 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f461406af30 0x7f461406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:57.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.419+0000 7f46317fa640 1 -- 172.21.15.154:0/2591028603 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6605+0+0 (secure 0 0 0) 0x7f4624089800 con 0x7f46340cb3e0 2023-09-23T03:17:57.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.419+0000 7f462bfff640 1 --2- 172.21.15.154:0/2591028603 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f461406af30 0x7f461406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:57.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.420+0000 7f462bfff640 1 --2- 172.21.15.154:0/2591028603 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f461406af30 0x7f461406d3f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f461c036000 tx=0x7f461c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:57.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.424+0000 7f46317fa640 1 -- 172.21.15.154:0/2591028603 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f4624054420 con 0x7f46340cb3e0 2023-09-23T03:17:57.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:57.644+0000 7f463a7ce640 1 -- 172.21.15.154:0/2591028603 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f46340cbc10 con 0x7f46340cb3e0 2023-09-23T03:17:58.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.212+0000 7f46317fa640 1 -- 172.21.15.154:0/2591028603 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v25) v1 ==== 162+0+0 (secure 0 0 0) 0x7f4624057a40 con 0x7f46340cb3e0 2023-09-23T03:17:58.213 INFO:teuthology.orchestra.run.smithi154.stderr:cephfs marked joinable; MDS may join as newly active. 2023-09-23T03:17:58.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.215+0000 7f463a7ce640 1 -- 172.21.15.154:0/2591028603 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f461406af30 msgr2=0x7f461406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:58.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.215+0000 7f463a7ce640 1 --2- 172.21.15.154:0/2591028603 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f461406af30 0x7f461406d3f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f461c036000 tx=0x7f461c008030 comp rx=0 tx=0).stop 2023-09-23T03:17:58.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.215+0000 7f463a7ce640 1 -- 172.21.15.154:0/2591028603 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46340cb3e0 msgr2=0x7f4634155fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:58.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.215+0000 7f463a7ce640 1 --2- 172.21.15.154:0/2591028603 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46340cb3e0 0x7f4634155fd0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f462400c460 tx=0x7f462400c920 comp rx=0 tx=0).stop 2023-09-23T03:17:58.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.215+0000 7f463a7ce640 1 -- 172.21.15.154:0/2591028603 shutdown_connections 2023-09-23T03:17:58.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.215+0000 7f463a7ce640 1 --2- 172.21.15.154:0/2591028603 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f461406af30 0x7f461406d3f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:58.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.215+0000 7f463a7ce640 1 --2- 172.21.15.154:0/2591028603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f463415bb30 0x7f463414e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:58.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.215+0000 7f463a7ce640 1 --2- 172.21.15.154:0/2591028603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f463415aba0 0x7f4634153880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:58.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.215+0000 7f463a7ce640 1 --2- 172.21.15.154:0/2591028603 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46340cb3e0 0x7f4634155fd0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:58.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.215+0000 7f463a7ce640 1 -- 172.21.15.154:0/2591028603 >> 172.21.15.154:0/2591028603 conn(0x7f463406c010 msgr2=0x7f46340703c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:58.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.216+0000 7f463a7ce640 1 -- 172.21.15.154:0/2591028603 shutdown_connections 2023-09-23T03:17:58.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.216+0000 7f463a7ce640 1 -- 172.21.15.154:0/2591028603 wait complete. 2023-09-23T03:17:58.238 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:17:58.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.495+0000 7f5a482f2640 1 Processor -- start 2023-09-23T03:17:58.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.495+0000 7f5a482f2640 1 -- start start 2023-09-23T03:17:58.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.496+0000 7f5a482f2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5a400b1b20 0x7f5a400ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:58.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.496+0000 7f5a482f2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a400cee70 0x7f5a400ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:58.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.496+0000 7f5a482f2640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a4015ac30 con 0x7f5a400acce0 2023-09-23T03:17:58.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.496+0000 7f5a482f2640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a400b2e90 con 0x7f5a400b1b20 2023-09-23T03:17:58.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.496+0000 7f5a482f2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a400cf320 con 0x7f5a400cee70 2023-09-23T03:17:58.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.497+0000 7f5a4588d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5a400b1b20 0x7f5a400ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:58.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.497+0000 7f5a4588d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5a400b1b20 0x7f5a400ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60184/0 (socket says 172.21.15.154:60184) 2023-09-23T03:17:58.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.497+0000 7f5a4588d640 1 -- 172.21.15.154:0/3869029831 learned_addr learned my addr 172.21.15.154:0/3869029831 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:58.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.497+0000 7f5a4688f640 1 --2- 172.21.15.154:0/3869029831 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a400cee70 0x7f5a400ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:58.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.497+0000 7f5a4588d640 1 -- 172.21.15.154:0/3869029831 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a400cee70 msgr2=0x7f5a400ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:58.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.497+0000 7f5a4588d640 1 --2- 172.21.15.154:0/3869029831 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a400cee70 0x7f5a400ab140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:58.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.497+0000 7f5a4588d640 1 -- 172.21.15.154:0/3869029831 >> v1:172.21.15.190:6789/0 conn(0x7f5a400acce0 legacy=0x7f5a4015ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:58.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.497+0000 7f5a4588d640 1 -- 172.21.15.154:0/3869029831 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a400b1f70 con 0x7f5a400b1b20 2023-09-23T03:17:58.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.498+0000 7f5a4588d640 1 --2- 172.21.15.154:0/3869029831 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a400b1b20 0x7f5a400ce930 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5a30009870 tx=0x7f5a3002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385b0150dbfb9185 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:58.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.498+0000 7f5a4508c640 1 -- 172.21.15.154:0/3869029831 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a300317c0 con 0x7f5a400b1b20 2023-09-23T03:17:58.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.498+0000 7f5a4508c640 1 -- 172.21.15.154:0/3869029831 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a30031920 con 0x7f5a400b1b20 2023-09-23T03:17:58.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.499+0000 7f5a4508c640 1 -- 172.21.15.154:0/3869029831 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a30031cb0 con 0x7f5a400b1b20 2023-09-23T03:17:58.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.499+0000 7f5a482f2640 1 -- 172.21.15.154:0/3869029831 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a400b1b20 msgr2=0x7f5a400ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:58.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.499+0000 7f5a482f2640 1 --2- 172.21.15.154:0/3869029831 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a400b1b20 0x7f5a400ce930 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5a30009870 tx=0x7f5a3002b860 comp rx=0 tx=0).stop 2023-09-23T03:17:58.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.499+0000 7f5a482f2640 1 -- 172.21.15.154:0/3869029831 shutdown_connections 2023-09-23T03:17:58.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.499+0000 7f5a482f2640 1 --2- 172.21.15.154:0/3869029831 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a400cee70 0x7f5a400ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:58.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.499+0000 7f5a482f2640 1 --2- 172.21.15.154:0/3869029831 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a400b1b20 0x7f5a400ce930 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:58.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.499+0000 7f5a482f2640 1 -- 172.21.15.154:0/3869029831 >> 172.21.15.154:0/3869029831 conn(0x7f5a4006a0f0 msgr2=0x7f5a4006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:58.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.499+0000 7f5a482f2640 1 -- 172.21.15.154:0/3869029831 shutdown_connections 2023-09-23T03:17:58.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.499+0000 7f5a482f2640 1 -- 172.21.15.154:0/3869029831 wait complete. 2023-09-23T03:17:58.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.501+0000 7f5a482f2640 1 Processor -- start 2023-09-23T03:17:58.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.501+0000 7f5a482f2640 1 -- start start 2023-09-23T03:17:58.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.502+0000 7f5a482f2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a400acce0 0x7f5a400970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:58.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.502+0000 7f5a482f2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a400b1b20 0x7f5a400c55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:58.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.502+0000 7f5a482f2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a400cee70 0x7f5a4015c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:58.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.502+0000 7f5a482f2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a400c5f30 con 0x7f5a400cee70 2023-09-23T03:17:58.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.502+0000 7f5a482f2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a400c6210 con 0x7f5a400b1b20 2023-09-23T03:17:58.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.502+0000 7f5a482f2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a400c64f0 con 0x7f5a400acce0 2023-09-23T03:17:58.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.502+0000 7f5a4688f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a400cee70 0x7f5a4015c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:58.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.502+0000 7f5a4688f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a400cee70 0x7f5a4015c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60194/0 (socket says 172.21.15.154:60194) 2023-09-23T03:17:58.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.502+0000 7f5a4688f640 1 -- 172.21.15.154:0/3748889487 learned_addr learned my addr 172.21.15.154:0/3748889487 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:58.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.503+0000 7f5a4608e640 1 --2- 172.21.15.154:0/3748889487 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a400acce0 0x7f5a400970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:58.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.503+0000 7f5a4588d640 1 --2- 172.21.15.154:0/3748889487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a400b1b20 0x7f5a400c55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:58.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.503+0000 7f5a4688f640 1 -- 172.21.15.154:0/3748889487 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a400acce0 msgr2=0x7f5a400970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:58.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.503+0000 7f5a4688f640 1 --2- 172.21.15.154:0/3748889487 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a400acce0 0x7f5a400970a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:58.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.503+0000 7f5a4688f640 1 -- 172.21.15.154:0/3748889487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a400b1b20 msgr2=0x7f5a400c55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:58.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.503+0000 7f5a4688f640 1 --2- 172.21.15.154:0/3748889487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a400b1b20 0x7f5a400c55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:58.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.503+0000 7f5a4688f640 1 -- 172.21.15.154:0/3748889487 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a300094d0 con 0x7f5a400cee70 2023-09-23T03:17:58.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.503+0000 7f5a4588d640 1 --2- 172.21.15.154:0/3748889487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a400b1b20 0x7f5a400c55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:58.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.503+0000 7f5a4688f640 1 --2- 172.21.15.154:0/3748889487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a400cee70 0x7f5a4015c660 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5a38007e60 tx=0x7f5a3800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:58.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.503+0000 7f5a36ffd640 1 -- 172.21.15.154:0/3748889487 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a38014420 con 0x7f5a400cee70 2023-09-23T03:17:58.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.503+0000 7f5a36ffd640 1 -- 172.21.15.154:0/3748889487 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a380145a0 con 0x7f5a400cee70 2023-09-23T03:17:58.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.504+0000 7f5a36ffd640 1 -- 172.21.15.154:0/3748889487 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a38014950 con 0x7f5a400cee70 2023-09-23T03:17:58.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.504+0000 7f5a482f2640 1 -- 172.21.15.154:0/3748889487 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a400c67d0 con 0x7f5a400cee70 2023-09-23T03:17:58.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.504+0000 7f5a482f2640 1 -- 172.21.15.154:0/3748889487 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a400c6fd0 con 0x7f5a400cee70 2023-09-23T03:17:58.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.505+0000 7f5a482f2640 1 -- 172.21.15.154:0/3748889487 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a400c6cf0 con 0x7f5a400cee70 2023-09-23T03:17:58.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.506+0000 7f5a36ffd640 1 -- 172.21.15.154:0/3748889487 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f5a38014ab0 con 0x7f5a400cee70 2023-09-23T03:17:58.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.507+0000 7f5a36ffd640 1 --2- 172.21.15.154:0/3748889487 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5a2406af30 0x7f5a2406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:58.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.507+0000 7f5a36ffd640 1 -- 172.21.15.154:0/3748889487 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6605+0+0 (secure 0 0 0) 0x7f5a3808a410 con 0x7f5a400cee70 2023-09-23T03:17:58.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.507+0000 7f5a4608e640 1 --2- 172.21.15.154:0/3748889487 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5a2406af30 0x7f5a2406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:58.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.508+0000 7f5a4608e640 1 --2- 172.21.15.154:0/3748889487 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5a2406af30 0x7f5a2406d3f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5a400c1f90 tx=0x7f5a3c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:58.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.512+0000 7f5a36ffd640 1 -- 172.21.15.154:0/3748889487 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f5a400c6cf0 con 0x7f5a400cee70 2023-09-23T03:17:58.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.740+0000 7f5a482f2640 1 -- 172.21.15.154:0/3748889487 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a400c6cf0 con 0x7f5a400cee70 2023-09-23T03:17:58.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.741+0000 7f5a36ffd640 1 -- 172.21.15.154:0/3748889487 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f5a400c6cf0 con 0x7f5a400cee70 2023-09-23T03:17:58.742 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:58.743 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":25,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4760,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/4034412019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4034412019},{"type":"v1","addr":"172.21.15.154:6836","nonce":4034412019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3736308401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3736308401},{"type":"v1","addr":"172.21.15.190:6835","nonce":3736308401}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/4126656125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":4126656125},{"type":"v1","addr":"172.21.15.190:6834","nonce":4126656125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5823,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/4079550625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":4079550625},{"type":"v1","addr":"172.21.15.154:6837","nonce":4079550625}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:17:56.221314+0000","modified":"2023-09-23T03:17:57.646484+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-09-23T03:17:58.743 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 25 2023-09-23T03:17:58.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.743+0000 7f5a482f2640 1 -- 172.21.15.154:0/3748889487 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5a2406af30 msgr2=0x7f5a2406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:58.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.743+0000 7f5a482f2640 1 --2- 172.21.15.154:0/3748889487 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5a2406af30 0x7f5a2406d3f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5a400c1f90 tx=0x7f5a3c009040 comp rx=0 tx=0).stop 2023-09-23T03:17:58.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.743+0000 7f5a482f2640 1 -- 172.21.15.154:0/3748889487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a400cee70 msgr2=0x7f5a4015c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:58.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.743+0000 7f5a482f2640 1 --2- 172.21.15.154:0/3748889487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a400cee70 0x7f5a4015c660 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5a38007e60 tx=0x7f5a3800e4b0 comp rx=0 tx=0).stop 2023-09-23T03:17:58.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.744+0000 7f5a482f2640 1 -- 172.21.15.154:0/3748889487 shutdown_connections 2023-09-23T03:17:58.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.744+0000 7f5a482f2640 1 --2- 172.21.15.154:0/3748889487 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5a2406af30 0x7f5a2406d3f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:58.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.744+0000 7f5a482f2640 1 --2- 172.21.15.154:0/3748889487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5a400cee70 0x7f5a4015c660 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:58.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.744+0000 7f5a482f2640 1 --2- 172.21.15.154:0/3748889487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5a400b1b20 0x7f5a400c55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:58.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.744+0000 7f5a482f2640 1 --2- 172.21.15.154:0/3748889487 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5a400acce0 0x7f5a400970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:58.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.744+0000 7f5a482f2640 1 -- 172.21.15.154:0/3748889487 >> 172.21.15.154:0/3748889487 conn(0x7f5a4006a0f0 msgr2=0x7f5a400aee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:58.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.744+0000 7f5a482f2640 1 -- 172.21.15.154:0/3748889487 shutdown_connections 2023-09-23T03:17:58.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.744+0000 7f5a482f2640 1 -- 172.21.15.154:0/3748889487 wait complete. 2023-09-23T03:17:58.761 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-09-23T03:17:58.761 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:17:58.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.973+0000 7fdf68505640 1 Processor -- start 2023-09-23T03:17:58.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.973+0000 7fdf68505640 1 -- start start 2023-09-23T03:17:58.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.974+0000 7fdf68505640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdf600b1b20 0x7fdf600ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:58.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.974+0000 7fdf68505640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdf600cee70 0x7fdf600ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:58.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.974+0000 7fdf68505640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf6015ac30 con 0x7fdf600acce0 2023-09-23T03:17:58.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.974+0000 7fdf68505640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf600b2e90 con 0x7fdf600cee70 2023-09-23T03:17:58.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.974+0000 7fdf68505640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf600cf320 con 0x7fdf600b1b20 2023-09-23T03:17:58.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.975+0000 7fdf66aa2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdf600cee70 0x7fdf600ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:58.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.975+0000 7fdf662a1640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fdf600acce0 0x7fdf6015ed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:50650/0 (socket says 172.21.15.154:50650) 2023-09-23T03:17:58.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.975+0000 7fdf662a1640 1 -- 172.21.15.154:0/2373716311 learned_addr learned my addr 172.21.15.154:0/2373716311 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:17:58.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.975+0000 7fdf65aa0640 1 --2- 172.21.15.154:0/2373716311 >> v2:172.21.15.190:3300/0 conn(0x7fdf600b1b20 0x7fdf600ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:58.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.975+0000 7fdf66aa2640 1 -- 172.21.15.154:0/2373716311 >> v2:172.21.15.190:3300/0 conn(0x7fdf600b1b20 msgr2=0x7fdf600ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:58.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.975+0000 7fdf66aa2640 1 --2- 172.21.15.154:0/2373716311 >> v2:172.21.15.190:3300/0 conn(0x7fdf600b1b20 0x7fdf600ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:58.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.975+0000 7fdf66aa2640 1 -- 172.21.15.154:0/2373716311 >> v1:172.21.15.154:6789/0 conn(0x7fdf600acce0 legacy=0x7fdf6015ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:58.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.975+0000 7fdf66aa2640 1 -- 172.21.15.154:0/2373716311 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf600b1f70 con 0x7fdf600cee70 2023-09-23T03:17:58.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.976+0000 7fdf66aa2640 1 --2- 172.21.15.154:0/2373716311 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf600cee70 0x7fdf600ab140 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fdf600cbff0 tx=0x7fdf5802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=272e01ad5e28ae7e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:58.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.976+0000 7fdf6529f640 1 -- 172.21.15.154:0/2373716311 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5802cbf0 con 0x7fdf600cee70 2023-09-23T03:17:58.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.976+0000 7fdf6529f640 1 -- 172.21.15.154:0/2373716311 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5802cd50 con 0x7fdf600cee70 2023-09-23T03:17:58.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.977+0000 7fdf6529f640 1 -- 172.21.15.154:0/2373716311 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf58034660 con 0x7fdf600cee70 2023-09-23T03:17:58.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.977+0000 7fdf68505640 1 -- 172.21.15.154:0/2373716311 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf600cee70 msgr2=0x7fdf600ab140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:58.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.977+0000 7fdf68505640 1 --2- 172.21.15.154:0/2373716311 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf600cee70 0x7fdf600ab140 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fdf600cbff0 tx=0x7fdf5802be40 comp rx=0 tx=0).stop 2023-09-23T03:17:58.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.977+0000 7fdf68505640 1 -- 172.21.15.154:0/2373716311 shutdown_connections 2023-09-23T03:17:58.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.977+0000 7fdf68505640 1 --2- 172.21.15.154:0/2373716311 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf600cee70 0x7fdf600ab140 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:58.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.977+0000 7fdf68505640 1 --2- 172.21.15.154:0/2373716311 >> v2:172.21.15.190:3300/0 conn(0x7fdf600b1b20 0x7fdf600ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:58.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.977+0000 7fdf68505640 1 -- 172.21.15.154:0/2373716311 >> 172.21.15.154:0/2373716311 conn(0x7fdf6006a0f0 msgr2=0x7fdf6006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:58.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.977+0000 7fdf68505640 1 -- 172.21.15.154:0/2373716311 shutdown_connections 2023-09-23T03:17:58.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.977+0000 7fdf68505640 1 -- 172.21.15.154:0/2373716311 wait complete. 2023-09-23T03:17:58.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.980+0000 7fdf68505640 1 Processor -- start 2023-09-23T03:17:58.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.980+0000 7fdf68505640 1 -- start start 2023-09-23T03:17:58.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.981+0000 7fdf68505640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf600acce0 0x7fdf600cd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:58.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.981+0000 7fdf68505640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf600b1b20 0x7fdf600c3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:58.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.981+0000 7fdf68505640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf600cee70 0x7fdf600c3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:58.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.981+0000 7fdf68505640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf600cc6f0 con 0x7fdf600acce0 2023-09-23T03:17:58.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.981+0000 7fdf68505640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf600cc9d0 con 0x7fdf600b1b20 2023-09-23T03:17:58.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.981+0000 7fdf68505640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf600cccb0 con 0x7fdf600cee70 2023-09-23T03:17:58.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.981+0000 7fdf662a1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf600acce0 0x7fdf600cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:58.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.981+0000 7fdf662a1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf600acce0 0x7fdf600cd790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60216/0 (socket says 172.21.15.154:60216) 2023-09-23T03:17:58.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.981+0000 7fdf662a1640 1 -- 172.21.15.154:0/3567189716 learned_addr learned my addr 172.21.15.154:0/3567189716 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:58.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.981+0000 7fdf65aa0640 1 --2- 172.21.15.154:0/3567189716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf600b1b20 0x7fdf600c3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:58.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.982+0000 7fdf662a1640 1 -- 172.21.15.154:0/3567189716 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf600cee70 msgr2=0x7fdf600c3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:58.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.982+0000 7fdf66aa2640 1 --2- 172.21.15.154:0/3567189716 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf600cee70 0x7fdf600c3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:58.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.982+0000 7fdf662a1640 1 --2- 172.21.15.154:0/3567189716 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf600cee70 0x7fdf600c3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:58.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.982+0000 7fdf662a1640 1 -- 172.21.15.154:0/3567189716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf600b1b20 msgr2=0x7fdf600c3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:58.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.982+0000 7fdf662a1640 1 --2- 172.21.15.154:0/3567189716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf600b1b20 0x7fdf600c3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:58.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.982+0000 7fdf662a1640 1 -- 172.21.15.154:0/3567189716 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf58009a70 con 0x7fdf600acce0 2023-09-23T03:17:58.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.982+0000 7fdf662a1640 1 --2- 172.21.15.154:0/3567189716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf600acce0 0x7fdf600cd790 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00c480 tx=0x7fdf5c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:58.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.982+0000 7fdf56ffd640 1 -- 172.21.15.154:0/3567189716 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c007480 con 0x7fdf600acce0 2023-09-23T03:17:58.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.982+0000 7fdf56ffd640 1 -- 172.21.15.154:0/3567189716 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5c0075e0 con 0x7fdf600acce0 2023-09-23T03:17:58.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.982+0000 7fdf68505640 1 -- 172.21.15.154:0/3567189716 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf600ccf90 con 0x7fdf600acce0 2023-09-23T03:17:58.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.982+0000 7fdf56ffd640 1 -- 172.21.15.154:0/3567189716 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c0059a0 con 0x7fdf600acce0 2023-09-23T03:17:58.986 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.983+0000 7fdf68505640 1 -- 172.21.15.154:0/3567189716 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf600cdcd0 con 0x7fdf600acce0 2023-09-23T03:17:58.986 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.984+0000 7fdf68505640 1 -- 172.21.15.154:0/3567189716 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf600cd4b0 con 0x7fdf600acce0 2023-09-23T03:17:58.986 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.985+0000 7fdf56ffd640 1 -- 172.21.15.154:0/3567189716 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fdf5c005b00 con 0x7fdf600acce0 2023-09-23T03:17:58.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.986+0000 7fdf56ffd640 1 --2- 172.21.15.154:0/3567189716 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdf4406af30 0x7fdf4406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:58.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.986+0000 7fdf56ffd640 1 -- 172.21.15.154:0/3567189716 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6605+0+0 (secure 0 0 0) 0x7fdf5c088f60 con 0x7fdf600acce0 2023-09-23T03:17:58.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.986+0000 7fdf65aa0640 1 --2- 172.21.15.154:0/3567189716 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdf4406af30 0x7fdf4406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:58.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.987+0000 7fdf65aa0640 1 --2- 172.21.15.154:0/3567189716 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdf4406af30 0x7fdf4406d3f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdf600c8ad0 tx=0x7fdf50008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:58.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:58.990+0000 7fdf56ffd640 1 -- 172.21.15.154:0/3567189716 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fdf5c057760 con 0x7fdf600acce0 2023-09-23T03:17:59.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.226+0000 7fdf68505640 1 -- 172.21.15.154:0/3567189716 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf6015ac10 con 0x7fdf600acce0 2023-09-23T03:17:59.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.227+0000 7fdf56ffd640 1 -- 172.21.15.154:0/3567189716 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4919 (secure 0 0 0) 0x7fdf5c017070 con 0x7fdf600acce0 2023-09-23T03:17:59.228 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:59.228 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":25,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4760,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/4034412019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4034412019},{"type":"v1","addr":"172.21.15.154:6836","nonce":4034412019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3736308401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3736308401},{"type":"v1","addr":"172.21.15.190:6835","nonce":3736308401}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/4126656125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":4126656125},{"type":"v1","addr":"172.21.15.190:6834","nonce":4126656125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5823,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/4079550625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":4079550625},{"type":"v1","addr":"172.21.15.154:6837","nonce":4079550625}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:17:56.221314+0000","modified":"2023-09-23T03:17:57.646484+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-09-23T03:17:59.228 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 25 2023-09-23T03:17:59.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.230+0000 7fdf68505640 1 -- 172.21.15.154:0/3567189716 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdf4406af30 msgr2=0x7fdf4406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:59.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.230+0000 7fdf68505640 1 --2- 172.21.15.154:0/3567189716 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdf4406af30 0x7fdf4406d3f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdf600c8ad0 tx=0x7fdf50008040 comp rx=0 tx=0).stop 2023-09-23T03:17:59.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.230+0000 7fdf68505640 1 -- 172.21.15.154:0/3567189716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf600acce0 msgr2=0x7fdf600cd790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:59.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.230+0000 7fdf68505640 1 --2- 172.21.15.154:0/3567189716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf600acce0 0x7fdf600cd790 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00c480 tx=0x7fdf5c00c940 comp rx=0 tx=0).stop 2023-09-23T03:17:59.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.230+0000 7fdf68505640 1 -- 172.21.15.154:0/3567189716 shutdown_connections 2023-09-23T03:17:59.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.230+0000 7fdf68505640 1 --2- 172.21.15.154:0/3567189716 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdf4406af30 0x7fdf4406d3f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:59.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.230+0000 7fdf68505640 1 --2- 172.21.15.154:0/3567189716 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdf600cee70 0x7fdf600c3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:59.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.230+0000 7fdf68505640 1 --2- 172.21.15.154:0/3567189716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdf600b1b20 0x7fdf600c3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:59.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.230+0000 7fdf68505640 1 --2- 172.21.15.154:0/3567189716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdf600acce0 0x7fdf600cd790 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:59.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.230+0000 7fdf68505640 1 -- 172.21.15.154:0/3567189716 >> 172.21.15.154:0/3567189716 conn(0x7fdf6006a0f0 msgr2=0x7fdf600c0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:59.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.231+0000 7fdf68505640 1 -- 172.21.15.154:0/3567189716 shutdown_connections 2023-09-23T03:17:59.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.231+0000 7fdf68505640 1 -- 172.21.15.154:0/3567189716 wait complete. 2023-09-23T03:17:59.248 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:17:59.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.458+0000 7efd3070f640 1 Processor -- start 2023-09-23T03:17:59.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.458+0000 7efd3070f640 1 -- start start 2023-09-23T03:17:59.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.459+0000 7efd3070f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efd28160f90 0x7efd28167950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:59.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.459+0000 7efd3070f640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd28161400 con 0x7efd280cf040 2023-09-23T03:17:59.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.459+0000 7efd3070f640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd280cd530 con 0x7efd2815caf0 2023-09-23T03:17:59.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.459+0000 7efd3070f640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd2815ced0 con 0x7efd28160f90 2023-09-23T03:17:59.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.460+0000 7efd2ecac640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efd28160f90 0x7efd28167950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:59.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.460+0000 7efd2dcaa640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7efd280cf040 0x7efd28160880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:50660/0 (socket says 172.21.15.154:50660) 2023-09-23T03:17:59.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.460+0000 7efd2dcaa640 1 -- 172.21.15.154:0/4124828011 learned_addr learned my addr 172.21.15.154:0/4124828011 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:17:59.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.460+0000 7efd2ecac640 1 -- 172.21.15.154:0/4124828011 >> v1:172.21.15.190:6789/0 conn(0x7efd2815caf0 legacy=0x7efd280ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:59.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.460+0000 7efd2ecac640 1 -- 172.21.15.154:0/4124828011 >> v1:172.21.15.154:6789/0 conn(0x7efd280cf040 legacy=0x7efd28160880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:59.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.460+0000 7efd2ecac640 1 -- 172.21.15.154:0/4124828011 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd280cd810 con 0x7efd28160f90 2023-09-23T03:17:59.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.462+0000 7efd2ecac640 1 --2- 172.21.15.154:0/4124828011 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd28160f90 0x7efd28167950 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7efd200097a0 tx=0x7efd2002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a7c3652f7b5f2e0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:59.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.462+0000 7efd2d4a9640 1 -- 172.21.15.154:0/4124828011 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd20031830 con 0x7efd28160f90 2023-09-23T03:17:59.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.462+0000 7efd2d4a9640 1 -- 172.21.15.154:0/4124828011 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd20031990 con 0x7efd28160f90 2023-09-23T03:17:59.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.462+0000 7efd2d4a9640 1 -- 172.21.15.154:0/4124828011 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd20031d20 con 0x7efd28160f90 2023-09-23T03:17:59.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.463+0000 7efd3070f640 1 -- 172.21.15.154:0/4124828011 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd28160f90 msgr2=0x7efd28167950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:59.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.463+0000 7efd3070f640 1 --2- 172.21.15.154:0/4124828011 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd28160f90 0x7efd28167950 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7efd200097a0 tx=0x7efd2002b8d0 comp rx=0 tx=0).stop 2023-09-23T03:17:59.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.463+0000 7efd3070f640 1 -- 172.21.15.154:0/4124828011 shutdown_connections 2023-09-23T03:17:59.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.463+0000 7efd3070f640 1 --2- 172.21.15.154:0/4124828011 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd28160f90 0x7efd28167950 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:59.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.463+0000 7efd3070f640 1 -- 172.21.15.154:0/4124828011 >> 172.21.15.154:0/4124828011 conn(0x7efd2806a0f0 msgr2=0x7efd2806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:59.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.463+0000 7efd3070f640 1 -- 172.21.15.154:0/4124828011 shutdown_connections 2023-09-23T03:17:59.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.463+0000 7efd3070f640 1 -- 172.21.15.154:0/4124828011 wait complete. 2023-09-23T03:17:59.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.465+0000 7efd3070f640 1 Processor -- start 2023-09-23T03:17:59.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.465+0000 7efd3070f640 1 -- start start 2023-09-23T03:17:59.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.466+0000 7efd3070f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd280cf040 0x7efd28157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:59.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.466+0000 7efd3070f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd2815caf0 0x7efd280719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:59.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.466+0000 7efd3070f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd28160f90 0x7efd280c36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:59.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.466+0000 7efd3070f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd280c3070 con 0x7efd2815caf0 2023-09-23T03:17:59.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.466+0000 7efd3070f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd280c1fd0 con 0x7efd28160f90 2023-09-23T03:17:59.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.466+0000 7efd3070f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd280c22b0 con 0x7efd280cf040 2023-09-23T03:17:59.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.466+0000 7efd2dcaa640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd2815caf0 0x7efd280719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:59.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.466+0000 7efd2dcaa640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd2815caf0 0x7efd280719a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60230/0 (socket says 172.21.15.154:60230) 2023-09-23T03:17:59.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.466+0000 7efd2dcaa640 1 -- 172.21.15.154:0/2016690127 learned_addr learned my addr 172.21.15.154:0/2016690127 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:59.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.466+0000 7efd2e4ab640 1 --2- 172.21.15.154:0/2016690127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd280cf040 0x7efd28157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:59.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.466+0000 7efd2ecac640 1 --2- 172.21.15.154:0/2016690127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd28160f90 0x7efd280c36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:59.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.466+0000 7efd2dcaa640 1 -- 172.21.15.154:0/2016690127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd280cf040 msgr2=0x7efd28157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:59.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.467+0000 7efd2dcaa640 1 --2- 172.21.15.154:0/2016690127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd280cf040 0x7efd28157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:59.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.467+0000 7efd2dcaa640 1 -- 172.21.15.154:0/2016690127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd28160f90 msgr2=0x7efd280c36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:59.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.467+0000 7efd2dcaa640 1 --2- 172.21.15.154:0/2016690127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd28160f90 0x7efd280c36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:59.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.467+0000 7efd2dcaa640 1 -- 172.21.15.154:0/2016690127 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd20009400 con 0x7efd2815caf0 2023-09-23T03:17:59.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.467+0000 7efd2e4ab640 1 --2- 172.21.15.154:0/2016690127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd280cf040 0x7efd28157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:59.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.467+0000 7efd2ecac640 1 --2- 172.21.15.154:0/2016690127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd28160f90 0x7efd280c36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:17:59.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.467+0000 7efd2dcaa640 1 --2- 172.21.15.154:0/2016690127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd2815caf0 0x7efd280719a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7efd1c00c4a0 tx=0x7efd1c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:59.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.467+0000 7efd26ffd640 1 -- 172.21.15.154:0/2016690127 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1c005ce0 con 0x7efd2815caf0 2023-09-23T03:17:59.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.467+0000 7efd26ffd640 1 -- 172.21.15.154:0/2016690127 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd1c005e40 con 0x7efd2815caf0 2023-09-23T03:17:59.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.467+0000 7efd26ffd640 1 -- 172.21.15.154:0/2016690127 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1c007270 con 0x7efd2815caf0 2023-09-23T03:17:59.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.467+0000 7efd3070f640 1 -- 172.21.15.154:0/2016690127 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd280c2590 con 0x7efd2815caf0 2023-09-23T03:17:59.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.467+0000 7efd3070f640 1 -- 172.21.15.154:0/2016690127 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd280c2b50 con 0x7efd2815caf0 2023-09-23T03:17:59.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.468+0000 7efd3070f640 1 -- 172.21.15.154:0/2016690127 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd280c2720 con 0x7efd2815caf0 2023-09-23T03:17:59.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.469+0000 7efd26ffd640 1 -- 172.21.15.154:0/2016690127 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7efd1c007490 con 0x7efd2815caf0 2023-09-23T03:17:59.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.470+0000 7efd26ffd640 1 --2- 172.21.15.154:0/2016690127 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efd1406af30 0x7efd1406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:59.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.470+0000 7efd26ffd640 1 -- 172.21.15.154:0/2016690127 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6605+0+0 (secure 0 0 0) 0x7efd1c088e00 con 0x7efd2815caf0 2023-09-23T03:17:59.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.471+0000 7efd2e4ab640 1 --2- 172.21.15.154:0/2016690127 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efd1406af30 0x7efd1406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:59.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.471+0000 7efd2e4ab640 1 --2- 172.21.15.154:0/2016690127 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efd1406af30 0x7efd1406d3f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7efd18007fd0 tx=0x7efd18009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:59.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.474+0000 7efd26ffd640 1 -- 172.21.15.154:0/2016690127 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7efd1c053b60 con 0x7efd2815caf0 2023-09-23T03:17:59.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.701+0000 7efd3070f640 1 -- 172.21.15.154:0/2016690127 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd28161400 con 0x7efd2815caf0 2023-09-23T03:17:59.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.703+0000 7efd26ffd640 1 -- 172.21.15.154:0/2016690127 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4919 (secure 0 0 0) 0x7efd28161400 con 0x7efd2815caf0 2023-09-23T03:17:59.703 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:17:59.703 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":25,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4760,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/4034412019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4034412019},{"type":"v1","addr":"172.21.15.154:6836","nonce":4034412019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3736308401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3736308401},{"type":"v1","addr":"172.21.15.190:6835","nonce":3736308401}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/4126656125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":4126656125},{"type":"v1","addr":"172.21.15.190:6834","nonce":4126656125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5823,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/4079550625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":4079550625},{"type":"v1","addr":"172.21.15.154:6837","nonce":4079550625}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:17:56.221314+0000","modified":"2023-09-23T03:17:57.646484+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-09-23T03:17:59.703 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 25 2023-09-23T03:17:59.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.705+0000 7efd3070f640 1 -- 172.21.15.154:0/2016690127 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efd1406af30 msgr2=0x7efd1406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:59.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.705+0000 7efd3070f640 1 --2- 172.21.15.154:0/2016690127 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efd1406af30 0x7efd1406d3f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7efd18007fd0 tx=0x7efd18009040 comp rx=0 tx=0).stop 2023-09-23T03:17:59.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.705+0000 7efd3070f640 1 -- 172.21.15.154:0/2016690127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd2815caf0 msgr2=0x7efd280719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:59.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.705+0000 7efd3070f640 1 --2- 172.21.15.154:0/2016690127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd2815caf0 0x7efd280719a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7efd1c00c4a0 tx=0x7efd1c00c960 comp rx=0 tx=0).stop 2023-09-23T03:17:59.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.705+0000 7efd3070f640 1 -- 172.21.15.154:0/2016690127 shutdown_connections 2023-09-23T03:17:59.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.705+0000 7efd3070f640 1 --2- 172.21.15.154:0/2016690127 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efd1406af30 0x7efd1406d3f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:59.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.705+0000 7efd3070f640 1 --2- 172.21.15.154:0/2016690127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd28160f90 0x7efd280c36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:59.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.705+0000 7efd3070f640 1 --2- 172.21.15.154:0/2016690127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd2815caf0 0x7efd280719a0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:59.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.705+0000 7efd3070f640 1 --2- 172.21.15.154:0/2016690127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd280cf040 0x7efd28157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:59.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.705+0000 7efd3070f640 1 -- 172.21.15.154:0/2016690127 >> 172.21.15.154:0/2016690127 conn(0x7efd2806a0f0 msgr2=0x7efd2806be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:59.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.706+0000 7efd3070f640 1 -- 172.21.15.154:0/2016690127 shutdown_connections 2023-09-23T03:17:59.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.706+0000 7efd3070f640 1 -- 172.21.15.154:0/2016690127 wait complete. 2023-09-23T03:17:59.724 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:17:59.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.943+0000 7f459fb41640 1 Processor -- start 2023-09-23T03:17:59.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.943+0000 7f459fb41640 1 -- start start 2023-09-23T03:17:59.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.944+0000 7f459fb41640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45980b1b20 0x7f45980ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:59.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.944+0000 7f459fb41640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f45980cee70 0x7f45980ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:59.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.944+0000 7f459fb41640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f459815ac30 con 0x7f45980acce0 2023-09-23T03:17:59.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.944+0000 7f459fb41640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45980b2e90 con 0x7f45980cee70 2023-09-23T03:17:59.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.944+0000 7f459fb41640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45980cf320 con 0x7f45980b1b20 2023-09-23T03:17:59.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.945+0000 7f459d8dd640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f45980acce0 0x7f459815ed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:50676/0 (socket says 172.21.15.154:50676) 2023-09-23T03:17:59.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.945+0000 7f459d8dd640 1 -- 172.21.15.154:0/237770659 learned_addr learned my addr 172.21.15.154:0/237770659 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:17:59.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.945+0000 7f459c8db640 1 -- 172.21.15.154:0/237770659 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1299447774 0 0) 0x7f4594001290 con 0x7f45980acce0 2023-09-23T03:17:59.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.945+0000 7f459e0de640 1 --2- 172.21.15.154:0/237770659 >> v2:172.21.15.190:3300/0 conn(0x7f45980cee70 0x7f45980ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:59.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.945+0000 7f459d0dc640 1 --2- 172.21.15.154:0/237770659 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45980b1b20 0x7f45980ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:59.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.945+0000 7f459c8db640 1 -- 172.21.15.154:0/237770659 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4584003520 con 0x7f45980acce0 2023-09-23T03:17:59.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.945+0000 7f459c8db640 1 -- 172.21.15.154:0/237770659 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2121030248 0 0) 0x7f4584003520 con 0x7f45980acce0 2023-09-23T03:17:59.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.945+0000 7f459c8db640 1 -- 172.21.15.154:0/237770659 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45980b1b20 msgr2=0x7f45980ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:59.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.945+0000 7f459c8db640 1 --2- 172.21.15.154:0/237770659 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45980b1b20 0x7f45980ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:59.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.945+0000 7f459c8db640 1 -- 172.21.15.154:0/237770659 >> v2:172.21.15.190:3300/0 conn(0x7f45980cee70 msgr2=0x7f45980ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:59.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.945+0000 7f459c8db640 1 --2- 172.21.15.154:0/237770659 >> v2:172.21.15.190:3300/0 conn(0x7f45980cee70 0x7f45980ab140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:59.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.945+0000 7f459c8db640 1 -- 172.21.15.154:0/237770659 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45980b1f70 con 0x7f45980acce0 2023-09-23T03:17:59.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.946+0000 7f459c8db640 1 -- 172.21.15.154:0/237770659 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f4594001980 con 0x7f45980acce0 2023-09-23T03:17:59.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.946+0000 7f459d0dc640 1 --2- 172.21.15.154:0/237770659 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45980b1b20 0x7f45980ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:17:59.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.946+0000 7f459c8db640 1 -- 172.21.15.154:0/237770659 >> v1:172.21.15.154:6789/0 conn(0x7f45980acce0 legacy=0x7f459815ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:59.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.946+0000 7f459c8db640 1 --2- 172.21.15.154:0/237770659 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4584003970 0x7f4584003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:59.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.946+0000 7f459c8db640 1 --2- 172.21.15.154:0/237770659 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45980b1b20 0x7f45840047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:59.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.946+0000 7f459c8db640 1 --2- 172.21.15.154:0/237770659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4584004d00 0x7f4584005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:59.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.946+0000 7f459c8db640 1 -- 172.21.15.154:0/237770659 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45980b2e90 con 0x7f4584004d00 2023-09-23T03:17:59.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.946+0000 7f459c8db640 1 -- 172.21.15.154:0/237770659 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45980cf320 con 0x7f4584003970 2023-09-23T03:17:59.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.946+0000 7f459c8db640 1 -- 172.21.15.154:0/237770659 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f458400b9d0 con 0x7f45980b1b20 2023-09-23T03:17:59.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.946+0000 7f459e0de640 1 --2- 172.21.15.154:0/237770659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4584004d00 0x7f4584005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:59.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.946+0000 7f459d8dd640 1 --2- 172.21.15.154:0/237770659 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4584003970 0x7f4584003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:59.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.946+0000 7f459d0dc640 1 --2- 172.21.15.154:0/237770659 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45980b1b20 0x7f45840047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:59.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.946+0000 7f459e0de640 1 -- 172.21.15.154:0/237770659 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45980b1b20 msgr2=0x7f45840047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:59.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.946+0000 7f459e0de640 1 --2- 172.21.15.154:0/237770659 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45980b1b20 0x7f45840047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:59.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.946+0000 7f459e0de640 1 -- 172.21.15.154:0/237770659 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4584003970 msgr2=0x7f4584003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:17:59.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.946+0000 7f459e0de640 1 --2- 172.21.15.154:0/237770659 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4584003970 0x7f4584003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:59.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.946+0000 7f459e0de640 1 -- 172.21.15.154:0/237770659 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4584003520 con 0x7f4584004d00 2023-09-23T03:17:59.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.948+0000 7f459e0de640 1 --2- 172.21.15.154:0/237770659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4584004d00 0x7f4584005180 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4590009fc0 tx=0x7f45840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c9db63c1a0b1a8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:59.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.948+0000 7f459c8db640 1 -- 172.21.15.154:0/237770659 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459002cc60 con 0x7f4584004d00 2023-09-23T03:17:59.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.948+0000 7f459c8db640 1 -- 172.21.15.154:0/237770659 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f459000f040 con 0x7f4584004d00 2023-09-23T03:17:59.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.948+0000 7f459c8db640 1 -- 172.21.15.154:0/237770659 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4590034440 con 0x7f4584004d00 2023-09-23T03:17:59.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.948+0000 7f459fb41640 1 -- 172.21.15.154:0/237770659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4584004d00 msgr2=0x7f4584005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:59.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.948+0000 7f459fb41640 1 --2- 172.21.15.154:0/237770659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4584004d00 0x7f4584005180 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4590009fc0 tx=0x7f45840057f0 comp rx=0 tx=0).stop 2023-09-23T03:17:59.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.948+0000 7f459e0de640 1 -- 172.21.15.154:0/237770659 reap_dead start 2023-09-23T03:17:59.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.948+0000 7f459fb41640 1 -- 172.21.15.154:0/237770659 shutdown_connections 2023-09-23T03:17:59.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.949+0000 7f459fb41640 1 -- 172.21.15.154:0/237770659 >> 172.21.15.154:0/237770659 conn(0x7f459806a0f0 msgr2=0x7f459806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:17:59.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.949+0000 7f459fb41640 1 -- 172.21.15.154:0/237770659 shutdown_connections 2023-09-23T03:17:59.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.949+0000 7f459fb41640 1 -- 172.21.15.154:0/237770659 wait complete. 2023-09-23T03:17:59.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.950+0000 7f459fb41640 1 Processor -- start 2023-09-23T03:17:59.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.951+0000 7f459fb41640 1 -- start start 2023-09-23T03:17:59.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.951+0000 7f459fb41640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4584004d00 0x7f45980976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:59.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.951+0000 7f459fb41640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f459815ca10 0x7f45980cd160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:59.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.951+0000 7f459fb41640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4598097be0 0x7f45980cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:59.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.951+0000 7f459fb41640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459815da10 con 0x7f459815ca10 2023-09-23T03:17:59.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.951+0000 7f459fb41640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459815ce90 con 0x7f4598097be0 2023-09-23T03:17:59.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.951+0000 7f459fb41640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f459815d170 con 0x7f4584004d00 2023-09-23T03:17:59.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.952+0000 7f459d0dc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f459815ca10 0x7f45980cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:59.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.952+0000 7f459d0dc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f459815ca10 0x7f45980cd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60244/0 (socket says 172.21.15.154:60244) 2023-09-23T03:17:59.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.952+0000 7f459d0dc640 1 -- 172.21.15.154:0/2438262406 learned_addr learned my addr 172.21.15.154:0/2438262406 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:17:59.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.952+0000 7f459e0de640 1 --2- 172.21.15.154:0/2438262406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4598097be0 0x7f45980cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:59.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.952+0000 7f459d8dd640 1 --2- 172.21.15.154:0/2438262406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4584004d00 0x7f45980976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:59.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.952+0000 7f459d0dc640 1 -- 172.21.15.154:0/2438262406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4584004d00 msgr2=0x7f45980976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:59.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.952+0000 7f459d0dc640 1 --2- 172.21.15.154:0/2438262406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4584004d00 0x7f45980976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:59.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.952+0000 7f459d0dc640 1 -- 172.21.15.154:0/2438262406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4598097be0 msgr2=0x7f45980cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:17:59.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.952+0000 7f459d0dc640 1 --2- 172.21.15.154:0/2438262406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4598097be0 0x7f45980cd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:17:59.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.952+0000 7f459d0dc640 1 -- 172.21.15.154:0/2438262406 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f459815d450 con 0x7f459815ca10 2023-09-23T03:17:59.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.953+0000 7f459d0dc640 1 --2- 172.21.15.154:0/2438262406 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f459815ca10 0x7f45980cd160 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f458800dfd0 tx=0x7f458800f970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:59.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.953+0000 7f458e7fc640 1 -- 172.21.15.154:0/2438262406 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4588014890 con 0x7f459815ca10 2023-09-23T03:17:59.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.953+0000 7f458e7fc640 1 -- 172.21.15.154:0/2438262406 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f458800c030 con 0x7f459815ca10 2023-09-23T03:17:59.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.953+0000 7f458e7fc640 1 -- 172.21.15.154:0/2438262406 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4588014c30 con 0x7f459815ca10 2023-09-23T03:17:59.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.953+0000 7f459fb41640 1 -- 172.21.15.154:0/2438262406 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f459815d730 con 0x7f459815ca10 2023-09-23T03:17:59.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.953+0000 7f459fb41640 1 -- 172.21.15.154:0/2438262406 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45980cdbe0 con 0x7f459815ca10 2023-09-23T03:17:59.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.954+0000 7f459fb41640 1 -- 172.21.15.154:0/2438262406 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4598073eb0 con 0x7f459815ca10 2023-09-23T03:17:59.958 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.955+0000 7f458e7fc640 1 -- 172.21.15.154:0/2438262406 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f45880133e0 con 0x7f459815ca10 2023-09-23T03:17:59.958 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.956+0000 7f458e7fc640 1 --2- 172.21.15.154:0/2438262406 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f457c06af30 0x7f457c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:17:59.958 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.956+0000 7f458e7fc640 1 -- 172.21.15.154:0/2438262406 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6605+0+0 (secure 0 0 0) 0x7f4588089df0 con 0x7f459815ca10 2023-09-23T03:17:59.958 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.956+0000 7f459d8dd640 1 --2- 172.21.15.154:0/2438262406 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f457c06af30 0x7f457c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:17:59.958 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.957+0000 7f459d8dd640 1 --2- 172.21.15.154:0/2438262406 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f457c06af30 0x7f457c06d3f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4594001eb0 tx=0x7f4594009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:17:59.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:17:59.960+0000 7f458e7fc640 1 -- 172.21.15.154:0/2438262406 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f4588054a10 con 0x7f459815ca10 2023-09-23T03:18:00.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.186+0000 7f459fb41640 1 -- 172.21.15.154:0/2438262406 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45981e3a70 con 0x7f459815ca10 2023-09-23T03:18:00.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.187+0000 7f458e7fc640 1 -- 172.21.15.154:0/2438262406 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f4588022090 con 0x7f459815ca10 2023-09-23T03:18:00.188 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:18:00.188 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":25,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4760,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/4034412019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4034412019},{"type":"v1","addr":"172.21.15.154:6836","nonce":4034412019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3736308401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3736308401},{"type":"v1","addr":"172.21.15.190:6835","nonce":3736308401}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/4126656125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":4126656125},{"type":"v1","addr":"172.21.15.190:6834","nonce":4126656125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5823,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/4079550625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":4079550625},{"type":"v1","addr":"172.21.15.154:6837","nonce":4079550625}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:17:56.221314+0000","modified":"2023-09-23T03:17:57.646484+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-09-23T03:18:00.188 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 25 2023-09-23T03:18:00.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.190+0000 7f459fb41640 1 -- 172.21.15.154:0/2438262406 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f457c06af30 msgr2=0x7f457c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:00.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.190+0000 7f459fb41640 1 --2- 172.21.15.154:0/2438262406 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f457c06af30 0x7f457c06d3f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4594001eb0 tx=0x7f4594009040 comp rx=0 tx=0).stop 2023-09-23T03:18:00.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.190+0000 7f459fb41640 1 -- 172.21.15.154:0/2438262406 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f459815ca10 msgr2=0x7f45980cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:00.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.190+0000 7f459fb41640 1 --2- 172.21.15.154:0/2438262406 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f459815ca10 0x7f45980cd160 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f458800dfd0 tx=0x7f458800f970 comp rx=0 tx=0).stop 2023-09-23T03:18:00.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.190+0000 7f459fb41640 1 -- 172.21.15.154:0/2438262406 shutdown_connections 2023-09-23T03:18:00.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.190+0000 7f459fb41640 1 --2- 172.21.15.154:0/2438262406 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f457c06af30 0x7f457c06d3f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.190+0000 7f459fb41640 1 --2- 172.21.15.154:0/2438262406 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4598097be0 0x7f45980cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.190+0000 7f459fb41640 1 --2- 172.21.15.154:0/2438262406 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f459815ca10 0x7f45980cd160 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.190+0000 7f459fb41640 1 --2- 172.21.15.154:0/2438262406 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4584004d00 0x7f45980976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.190+0000 7f459fb41640 1 -- 172.21.15.154:0/2438262406 >> 172.21.15.154:0/2438262406 conn(0x7f459806a0f0 msgr2=0x7f45980af580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:00.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.190+0000 7f459fb41640 1 -- 172.21.15.154:0/2438262406 shutdown_connections 2023-09-23T03:18:00.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.190+0000 7f459fb41640 1 -- 172.21.15.154:0/2438262406 wait complete. 2023-09-23T03:18:00.208 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:18:00.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.422+0000 7f0d520c2640 1 Processor -- start 2023-09-23T03:18:00.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.422+0000 7f0d520c2640 1 -- start start 2023-09-23T03:18:00.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.424+0000 7f0d520c2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d4c0acce0 0x7f0d4c15ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:00.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.424+0000 7f0d520c2640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0d4c0b1b20 0x7f0d4c0ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:00.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.424+0000 7f0d520c2640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d4c15ac30 con 0x7f0d4c0cee70 2023-09-23T03:18:00.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.424+0000 7f0d520c2640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d4c0b2e90 con 0x7f0d4c0b1b20 2023-09-23T03:18:00.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.424+0000 7f0d520c2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c0cf320 con 0x7f0d4c0acce0 2023-09-23T03:18:00.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.425+0000 7f0d42ffd640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0d4c0b1b20 0x7f0d4c0ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:00.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.425+0000 7f0d42ffd640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0d4c0b1b20 0x7f0d4c0ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.154:46162/0 (socket says 172.21.15.154:46162) 2023-09-23T03:18:00.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.425+0000 7f0d42ffd640 1 -- 172.21.15.154:0/2366650884 learned_addr learned my addr 172.21.15.154:0/2366650884 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:00.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.425+0000 7f0d4b7fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d4c0acce0 0x7f0d4c15ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:00.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.425+0000 7f0d42ffd640 1 -- 172.21.15.154:0/2366650884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d4c0acce0 msgr2=0x7f0d4c15ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:00.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.425+0000 7f0d42ffd640 1 --2- 172.21.15.154:0/2366650884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d4c0acce0 0x7f0d4c15ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.425+0000 7f0d42ffd640 1 -- 172.21.15.154:0/2366650884 >> v1:172.21.15.190:6789/0 conn(0x7f0d4c0cee70 legacy=0x7f0d4c06cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:00.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.425+0000 7f0d42ffd640 1 -- 172.21.15.154:0/2366650884 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d4c0b1f70 con 0x7f0d4c0b1b20 2023-09-23T03:18:00.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.427+0000 7f0d42ffd640 1 --2- 172.21.15.154:0/2366650884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d4c0b1b20 0x7f0d4c0ce930 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0d34009870 tx=0x7f0d3402b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe3296a35db90ca4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:00.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.427+0000 7f0d4affd640 1 -- 172.21.15.154:0/2366650884 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d34031640 con 0x7f0d4c0b1b20 2023-09-23T03:18:00.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.427+0000 7f0d4affd640 1 -- 172.21.15.154:0/2366650884 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d340317a0 con 0x7f0d4c0b1b20 2023-09-23T03:18:00.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.427+0000 7f0d4affd640 1 -- 172.21.15.154:0/2366650884 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d34031b30 con 0x7f0d4c0b1b20 2023-09-23T03:18:00.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.428+0000 7f0d520c2640 1 -- 172.21.15.154:0/2366650884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d4c0b1b20 msgr2=0x7f0d4c0ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:00.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.428+0000 7f0d520c2640 1 --2- 172.21.15.154:0/2366650884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d4c0b1b20 0x7f0d4c0ce930 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0d34009870 tx=0x7f0d3402b8f0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.428+0000 7f0d520c2640 1 -- 172.21.15.154:0/2366650884 shutdown_connections 2023-09-23T03:18:00.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.428+0000 7f0d520c2640 1 --2- 172.21.15.154:0/2366650884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d4c0b1b20 0x7f0d4c0ce930 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.428+0000 7f0d520c2640 1 --2- 172.21.15.154:0/2366650884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d4c0acce0 0x7f0d4c15ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.428+0000 7f0d520c2640 1 -- 172.21.15.154:0/2366650884 >> 172.21.15.154:0/2366650884 conn(0x7f0d4c06a0f0 msgr2=0x7f0d4c06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:00.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.428+0000 7f0d520c2640 1 -- 172.21.15.154:0/2366650884 shutdown_connections 2023-09-23T03:18:00.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.428+0000 7f0d520c2640 1 -- 172.21.15.154:0/2366650884 wait complete. 2023-09-23T03:18:00.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.430+0000 7f0d520c2640 1 Processor -- start 2023-09-23T03:18:00.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.430+0000 7f0d520c2640 1 -- start start 2023-09-23T03:18:00.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.431+0000 7f0d520c2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d4c0acce0 0x7f0d4c0c1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:00.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.431+0000 7f0d520c2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d4c0b1b20 0x7f0d4c15cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:00.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.431+0000 7f0d520c2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d4c0cee70 0x7f0d4c1f1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:00.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.431+0000 7f0d520c2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c1592e0 con 0x7f0d4c0acce0 2023-09-23T03:18:00.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.431+0000 7f0d520c2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c1f2070 con 0x7f0d4c0b1b20 2023-09-23T03:18:00.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.431+0000 7f0d520c2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c0c57f0 con 0x7f0d4c0cee70 2023-09-23T03:18:00.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.431+0000 7f0d4b7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d4c0acce0 0x7f0d4c0c1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:00.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.431+0000 7f0d4b7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d4c0acce0 0x7f0d4c0c1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60248/0 (socket says 172.21.15.154:60248) 2023-09-23T03:18:00.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.431+0000 7f0d4b7fe640 1 -- 172.21.15.154:0/369210404 learned_addr learned my addr 172.21.15.154:0/369210404 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:00.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.431+0000 7f0d42ffd640 1 --2- 172.21.15.154:0/369210404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d4c0b1b20 0x7f0d4c15cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:00.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.431+0000 7f0d4bfff640 1 --2- 172.21.15.154:0/369210404 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d4c0cee70 0x7f0d4c1f1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:00.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.432+0000 7f0d4b7fe640 1 -- 172.21.15.154:0/369210404 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d4c0cee70 msgr2=0x7f0d4c1f1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:00.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.432+0000 7f0d4b7fe640 1 --2- 172.21.15.154:0/369210404 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d4c0cee70 0x7f0d4c1f1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.432+0000 7f0d4b7fe640 1 -- 172.21.15.154:0/369210404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d4c0b1b20 msgr2=0x7f0d4c15cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:00.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.432+0000 7f0d4b7fe640 1 --2- 172.21.15.154:0/369210404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d4c0b1b20 0x7f0d4c15cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.432+0000 7f0d4b7fe640 1 -- 172.21.15.154:0/369210404 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d340094d0 con 0x7f0d4c0acce0 2023-09-23T03:18:00.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.432+0000 7f0d4b7fe640 1 --2- 172.21.15.154:0/369210404 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d4c0acce0 0x7f0d4c0c1fa0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00e900 tx=0x7f0d3c00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:00.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.432+0000 7f0d48ff9640 1 -- 172.21.15.154:0/369210404 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c009660 con 0x7f0d4c0acce0 2023-09-23T03:18:00.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.432+0000 7f0d48ff9640 1 -- 172.21.15.154:0/369210404 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d3c00ca00 con 0x7f0d4c0acce0 2023-09-23T03:18:00.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.432+0000 7f0d48ff9640 1 -- 172.21.15.154:0/369210404 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c00cd70 con 0x7f0d4c0acce0 2023-09-23T03:18:00.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.432+0000 7f0d520c2640 1 -- 172.21.15.154:0/369210404 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d4c15b540 con 0x7f0d4c0acce0 2023-09-23T03:18:00.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.433+0000 7f0d520c2640 1 -- 172.21.15.154:0/369210404 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d4c15bb00 con 0x7f0d4c0acce0 2023-09-23T03:18:00.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.434+0000 7f0d520c2640 1 -- 172.21.15.154:0/369210404 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d4c073eb0 con 0x7f0d4c0acce0 2023-09-23T03:18:00.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.435+0000 7f0d48ff9640 1 -- 172.21.15.154:0/369210404 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f0d3c013470 con 0x7f0d4c0acce0 2023-09-23T03:18:00.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.435+0000 7f0d48ff9640 1 --2- 172.21.15.154:0/369210404 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0d2c06af00 0x7f0d2c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:00.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.435+0000 7f0d48ff9640 1 -- 172.21.15.154:0/369210404 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6605+0+0 (secure 0 0 0) 0x7f0d3c08a220 con 0x7f0d4c0acce0 2023-09-23T03:18:00.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.436+0000 7f0d42ffd640 1 --2- 172.21.15.154:0/369210404 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0d2c06af00 0x7f0d2c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:00.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.436+0000 7f0d42ffd640 1 --2- 172.21.15.154:0/369210404 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0d2c06af00 0x7f0d2c06d3c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0d34036000 tx=0x7f0d3403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:00.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.440+0000 7f0d48ff9640 1 -- 172.21.15.154:0/369210404 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f0d3c054db0 con 0x7f0d4c0acce0 2023-09-23T03:18:00.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.666+0000 7f0d520c2640 1 -- 172.21.15.154:0/369210404 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d4c0cb630 con 0x7f0d4c0acce0 2023-09-23T03:18:00.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.668+0000 7f0d48ff9640 1 -- 172.21.15.154:0/369210404 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f0d3c017090 con 0x7f0d4c0acce0 2023-09-23T03:18:00.668 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:18:00.668 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":25,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4760,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/4034412019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4034412019},{"type":"v1","addr":"172.21.15.154:6836","nonce":4034412019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3736308401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3736308401},{"type":"v1","addr":"172.21.15.190:6835","nonce":3736308401}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/4126656125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":4126656125},{"type":"v1","addr":"172.21.15.190:6834","nonce":4126656125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5823,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/4079550625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":4079550625},{"type":"v1","addr":"172.21.15.154:6837","nonce":4079550625}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:17:56.221314+0000","modified":"2023-09-23T03:17:57.646484+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-09-23T03:18:00.669 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 25 2023-09-23T03:18:00.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.670+0000 7f0d520c2640 1 -- 172.21.15.154:0/369210404 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0d2c06af00 msgr2=0x7f0d2c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:00.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.670+0000 7f0d520c2640 1 --2- 172.21.15.154:0/369210404 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0d2c06af00 0x7f0d2c06d3c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0d34036000 tx=0x7f0d3403c040 comp rx=0 tx=0).stop 2023-09-23T03:18:00.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.670+0000 7f0d520c2640 1 -- 172.21.15.154:0/369210404 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d4c0acce0 msgr2=0x7f0d4c0c1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:00.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.670+0000 7f0d520c2640 1 --2- 172.21.15.154:0/369210404 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d4c0acce0 0x7f0d4c0c1fa0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00e900 tx=0x7f0d3c00edc0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.670+0000 7f0d520c2640 1 -- 172.21.15.154:0/369210404 shutdown_connections 2023-09-23T03:18:00.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.670+0000 7f0d520c2640 1 --2- 172.21.15.154:0/369210404 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0d2c06af00 0x7f0d2c06d3c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.670+0000 7f0d520c2640 1 --2- 172.21.15.154:0/369210404 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0d4c0cee70 0x7f0d4c1f1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.670+0000 7f0d520c2640 1 --2- 172.21.15.154:0/369210404 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0d4c0b1b20 0x7f0d4c15cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.670+0000 7f0d520c2640 1 --2- 172.21.15.154:0/369210404 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0d4c0acce0 0x7f0d4c0c1fa0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.670+0000 7f0d520c2640 1 -- 172.21.15.154:0/369210404 >> 172.21.15.154:0/369210404 conn(0x7f0d4c06a0f0 msgr2=0x7f0d4c0affe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:00.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.671+0000 7f0d520c2640 1 -- 172.21.15.154:0/369210404 shutdown_connections 2023-09-23T03:18:00.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.671+0000 7f0d520c2640 1 -- 172.21.15.154:0/369210404 wait complete. 2023-09-23T03:18:00.690 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset session 2023-09-23T03:18:00.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.785+0000 7f3ecfa54100 1 -- start start 2023-09-23T03:18:00.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.785+0000 7f3ecfa54100 1 Processor -- start 2023-09-23T03:18:00.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.786+0000 7f3ecfa54100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558beb046c00 0x558beb0b6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:00.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.786+0000 7f3ecfa54100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558beb047800 0x558beb0b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:00.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.786+0000 7f3ecfa54100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558beb047c00 0x558beb0b7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:00.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.786+0000 7f3ecfa54100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x558beb093180 con 0x558beb046c00 2023-09-23T03:18:00.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.786+0000 7f3ecfa54100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558beb092140 con 0x558beb047800 2023-09-23T03:18:00.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.786+0000 7f3ecfa54100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x558beb092000 con 0x558beb047c00 2023-09-23T03:18:00.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.787+0000 7f3ece1fa640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558beb046c00 0x558beb0b6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:00.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.787+0000 7f3ece1fa640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558beb046c00 0x558beb0b6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60264/0 (socket says 172.21.15.154:60264) 2023-09-23T03:18:00.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.787+0000 7f3ece1fa640 1 -- 172.21.15.154:0/1651573537 learned_addr learned my addr 172.21.15.154:0/1651573537 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:00.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.787+0000 7f3ecd9f9640 1 --2- 172.21.15.154:0/1651573537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558beb047800 0x558beb0b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:00.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.787+0000 7f3ece1fa640 1 -- 172.21.15.154:0/1651573537 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558beb047c00 msgr2=0x558beb0b7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:00.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.787+0000 7f3ece1fa640 1 --2- 172.21.15.154:0/1651573537 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558beb047c00 0x558beb0b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.787+0000 7f3ece1fa640 1 -- 172.21.15.154:0/1651573537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558beb047800 msgr2=0x558beb0b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:00.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.787+0000 7f3ece1fa640 1 --2- 172.21.15.154:0/1651573537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558beb047800 0x558beb0b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.787+0000 7f3ece1fa640 1 -- 172.21.15.154:0/1651573537 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558beb07ab60 con 0x558beb046c00 2023-09-23T03:18:00.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.787+0000 7f3ecd9f9640 1 --2- 172.21.15.154:0/1651573537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558beb047800 0x558beb0b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:18:00.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.787+0000 7f3ece1fa640 1 --2- 172.21.15.154:0/1651573537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558beb046c00 0x558beb0b6580 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x558bebe6e750 tx=0x558bebda0300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9371da81e1e4553 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:00.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.788+0000 7f3ecb9f5640 1 -- 172.21.15.154:0/1651573537 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558beb091600 con 0x558beb046c00 2023-09-23T03:18:00.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.788+0000 7f3ecb9f5640 1 -- 172.21.15.154:0/1651573537 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558beb06d800 con 0x558beb046c00 2023-09-23T03:18:00.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.788+0000 7f3ecfa54100 1 -- 172.21.15.154:0/1651573537 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558beb07a340 con 0x558beb046c00 2023-09-23T03:18:00.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.788+0000 7f3ecfa54100 1 -- 172.21.15.154:0/1651573537 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x558beb07a000 con 0x558beb046c00 2023-09-23T03:18:00.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.788+0000 7f3ecfa54100 4 waiting for MDS map... 2023-09-23T03:18:00.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.788+0000 7f3ecb9f5640 1 -- 172.21.15.154:0/1651573537 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558beb0911e0 con 0x558beb046c00 2023-09-23T03:18:00.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.788+0000 7f3ecb9f5640 1 -- 172.21.15.154:0/1651573537 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(43..43 src has 1..43) v4 ==== 6605+0+0 (secure 0 0 0) 0x558beb072380 con 0x558beb046c00 2023-09-23T03:18:00.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.788+0000 7f3ecb9f5640 1 -- 172.21.15.154:0/1651573537 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 25) v1 ==== 3306+0+0 (secure 0 0 0) 0x558beb09b200 con 0x558beb046c00 2023-09-23T03:18:00.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.788+0000 7f3ecfa54100 4 Got MDS map 25 2023-09-23T03:18:00.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.788+0000 7f3ecfa54100 10 main: main 2023-09-23T03:18:00.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.788+0000 7f3ecfa54100 4 main: connecting to RADOS... 2023-09-23T03:18:00.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.789+0000 7f3ecfa54100 1 Processor -- start 2023-09-23T03:18:00.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.789+0000 7f3ecfa54100 1 -- start start 2023-09-23T03:18:00.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.790+0000 7f3ecfa54100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558bebf09000 0x558beb0b7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:00.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.790+0000 7f3ecfa54100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558bebf09400 0x558beb0b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:00.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.790+0000 7f3ecfa54100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558bebf09800 0x558beb0b8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:00.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.790+0000 7f3ecfa54100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x558beb092a00 con 0x558bebf09800 2023-09-23T03:18:00.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.790+0000 7f3ecfa54100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558beb115180 con 0x558bebf09400 2023-09-23T03:18:00.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.790+0000 7f3ecfa54100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x558beb114dc0 con 0x558bebf09000 2023-09-23T03:18:00.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.790+0000 7f3ecd9f9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558bebf09800 0x558beb0b8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:00.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.790+0000 7f3ecd9f9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558bebf09800 0x558beb0b8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60274/0 (socket says 172.21.15.154:60274) 2023-09-23T03:18:00.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.790+0000 7f3ecd9f9640 1 -- 172.21.15.154:0/2715360741 learned_addr learned my addr 172.21.15.154:0/2715360741 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:00.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.791+0000 7f3ece9fb640 1 --2- 172.21.15.154:0/2715360741 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558bebf09000 0x558beb0b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:00.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.791+0000 7f3ece1fa640 1 --2- 172.21.15.154:0/2715360741 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558bebf09400 0x558beb0b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:00.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.791+0000 7f3ecd9f9640 1 -- 172.21.15.154:0/2715360741 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558bebf09000 msgr2=0x558beb0b7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:00.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.791+0000 7f3ecd9f9640 1 --2- 172.21.15.154:0/2715360741 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558bebf09000 0x558beb0b7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.791+0000 7f3ecd9f9640 1 -- 172.21.15.154:0/2715360741 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558bebf09400 msgr2=0x558beb0b7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:00.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.791+0000 7f3ecd9f9640 1 --2- 172.21.15.154:0/2715360741 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558bebf09400 0x558beb0b7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.791+0000 7f3ecd9f9640 1 -- 172.21.15.154:0/2715360741 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558beb07aea0 con 0x558bebf09800 2023-09-23T03:18:00.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.791+0000 7f3ecd9f9640 1 --2- 172.21.15.154:0/2715360741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558bebf09800 0x558beb0b8100 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x558bebe6fd70 tx=0x558bebda0060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=832ea772cb2e50b3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:00.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.791+0000 7f3ec99f1640 1 -- 172.21.15.154:0/2715360741 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558beb091080 con 0x558bebf09800 2023-09-23T03:18:00.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.791+0000 7f3ec99f1640 1 -- 172.21.15.154:0/2715360741 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558bebe26780 con 0x558bebf09800 2023-09-23T03:18:00.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.791+0000 7f3ec99f1640 1 -- 172.21.15.154:0/2715360741 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558beb091600 con 0x558bebf09800 2023-09-23T03:18:00.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.792+0000 7f3ecfa54100 1 -- 172.21.15.154:0/2715360741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558bebf09800 msgr2=0x558beb0b8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:00.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.792+0000 7f3ecfa54100 1 --2- 172.21.15.154:0/2715360741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558bebf09800 0x558beb0b8100 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x558bebe6fd70 tx=0x558bebda0060 comp rx=0 tx=0).stop 2023-09-23T03:18:00.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.792+0000 7f3ecfa54100 1 -- 172.21.15.154:0/2715360741 shutdown_connections 2023-09-23T03:18:00.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.792+0000 7f3ecfa54100 1 --2- 172.21.15.154:0/2715360741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558bebf09800 0x558beb0b8100 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.792+0000 7f3ecfa54100 1 --2- 172.21.15.154:0/2715360741 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558bebf09400 0x558beb0b7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.792+0000 7f3ecfa54100 1 --2- 172.21.15.154:0/2715360741 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558bebf09000 0x558beb0b7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.792+0000 7f3ecfa54100 1 -- 172.21.15.154:0/2715360741 >> 172.21.15.154:0/2715360741 conn(0x558bebf08c00 msgr2=0x558bebdbd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:00.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.792+0000 7f3ecfa54100 1 -- 172.21.15.154:0/2715360741 shutdown_connections 2023-09-23T03:18:00.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.792+0000 7f3ecfa54100 1 -- 172.21.15.154:0/2715360741 wait complete. 2023-09-23T03:18:00.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.793+0000 7f3ecfa54100 1 Processor -- start 2023-09-23T03:18:00.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.793+0000 7f3ecfa54100 1 -- start start 2023-09-23T03:18:00.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.793+0000 7f3ecfa54100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558bebf09000 0x558beb0b9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:00.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.793+0000 7f3ecfa54100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558bebf09400 0x558beb0b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:00.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.793+0000 7f3ecfa54100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558bebf09800 0x558beb0b8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:00.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.794+0000 7f3ecfa54100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x558beb114c80 con 0x558bebf09800 2023-09-23T03:18:00.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.794+0000 7f3ecfa54100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558beb092140 con 0x558bebf09000 2023-09-23T03:18:00.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.794+0000 7f3ecfa54100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x558beb092b40 con 0x558bebf09400 2023-09-23T03:18:00.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.794+0000 7f3ecd9f9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558bebf09800 0x558beb0b8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:00.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.794+0000 7f3ecd9f9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558bebf09800 0x558beb0b8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60284/0 (socket says 172.21.15.154:60284) 2023-09-23T03:18:00.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.794+0000 7f3ecd9f9640 1 -- 172.21.15.154:0/622366349 learned_addr learned my addr 172.21.15.154:0/622366349 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:00.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.794+0000 7f3ece1fa640 1 --2- 172.21.15.154:0/622366349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558bebf09400 0x558beb0b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:00.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.794+0000 7f3ece9fb640 1 --2- 172.21.15.154:0/622366349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558bebf09000 0x558beb0b9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:00.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.794+0000 7f3ecd9f9640 1 -- 172.21.15.154:0/622366349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558bebf09400 msgr2=0x558beb0b8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:00.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.794+0000 7f3ecd9f9640 1 --2- 172.21.15.154:0/622366349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558bebf09400 0x558beb0b8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.794+0000 7f3ecd9f9640 1 -- 172.21.15.154:0/622366349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558bebf09000 msgr2=0x558beb0b9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:00.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.794+0000 7f3ecd9f9640 1 --2- 172.21.15.154:0/622366349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558bebf09000 0x558beb0b9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.794+0000 7f3ecd9f9640 1 -- 172.21.15.154:0/622366349 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558beb07b1e0 con 0x558bebf09800 2023-09-23T03:18:00.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.794+0000 7f3ecd9f9640 1 --2- 172.21.15.154:0/622366349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558bebf09800 0x558beb0b8680 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x558bebf76cc0 tx=0x558bebda01e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:00.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.795+0000 7f3ec81ee640 1 -- 172.21.15.154:0/622366349 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558beb0911e0 con 0x558bebf09800 2023-09-23T03:18:00.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.795+0000 7f3ec81ee640 1 -- 172.21.15.154:0/622366349 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558beb06ca80 con 0x558bebf09800 2023-09-23T03:18:00.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.795+0000 7f3ec81ee640 1 -- 172.21.15.154:0/622366349 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558beb090dc0 con 0x558bebf09800 2023-09-23T03:18:00.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.795+0000 7f3ecfa54100 1 -- 172.21.15.154:0/622366349 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558beb07aea0 con 0x558bebf09800 2023-09-23T03:18:00.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.795+0000 7f3ecfa54100 1 -- 172.21.15.154:0/622366349 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558beb07ab60 con 0x558bebf09800 2023-09-23T03:18:00.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.795+0000 7f3ecfa54100 4 main: resolving pool 5 2023-09-23T03:18:00.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.796+0000 7f3ec81ee640 1 -- 172.21.15.154:0/622366349 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x558beb078840 con 0x558bebf09800 2023-09-23T03:18:00.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.797+0000 7f3ec81ee640 1 --2- 172.21.15.154:0/622366349 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x558bebf09c00 0x558beb0b9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:00.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.797+0000 7f3ec81ee640 1 -- 172.21.15.154:0/622366349 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6605+0+0 (secure 0 0 0) 0x558beb072540 con 0x558bebf09800 2023-09-23T03:18:00.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.797+0000 7f3ecfa54100 4 main: creating IoCtx.. 2023-09-23T03:18:00.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.797+0000 7f3ece9fb640 1 --2- 172.21.15.154:0/622366349 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x558bebf09c00 0x558beb0b9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:00.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.797+0000 7f3ecfa54100 1 --2- 172.21.15.154:0/622366349 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x558beb39e000 0x558beb0b8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:00.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.797+0000 7f3ecfa54100 1 -- 172.21.15.154:0/622366349 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:1 5.b 5:d0630e4c:::mds0_sessionmap:head [omap-clear,omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x558beb0a3880 con 0x558beb39e000 2023-09-23T03:18:00.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.798+0000 7f3ece1fa640 1 --2- 172.21.15.154:0/622366349 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x558beb39e000 0x558beb0b8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:00.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.798+0000 7f3ece9fb640 1 --2- 172.21.15.154:0/622366349 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x558bebf09c00 0x558beb0b9700 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x558bebd9e2a0 tx=0x558bebf1fe00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:00.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.798+0000 7f3ece1fa640 1 --2- 172.21.15.154:0/622366349 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x558beb39e000 0x558beb0b8100 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:00.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.800+0000 7f3ece1fa640 1 -- 172.21.15.154:0/622366349 <== osd.2 v2:172.21.15.154:6822/2884407243 1 ==== osd_op_reply(1 mds0_sessionmap [omap-clear,omap-set-header] v43'5 uv5 ondisk = 0) v8 ==== 201+0+0 (crc 0 0 0) 0x558bebefeb40 con 0x558beb39e000 2023-09-23T03:18:00.801 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:18:00.801 INFO:teuthology.orchestra.run.smithi154.stdout: "0": { 2023-09-23T03:18:00.801 INFO:teuthology.orchestra.run.smithi154.stdout: "data": {}, 2023-09-23T03:18:00.802 INFO:teuthology.orchestra.run.smithi154.stdout: "result": 0 2023-09-23T03:18:00.802 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:18:00.802 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:18:00.802 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:18:00.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.800+0000 7f3ecfa54100 1 -- 172.21.15.154:0/622366349 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x558beb39e000 msgr2=0x558beb0b8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:00.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.800+0000 7f3ecfa54100 1 --2- 172.21.15.154:0/622366349 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x558beb39e000 0x558beb0b8100 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.800+0000 7f3ecfa54100 1 -- 172.21.15.154:0/622366349 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x558bebf09c00 msgr2=0x558beb0b9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:00.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.800+0000 7f3ecfa54100 1 --2- 172.21.15.154:0/622366349 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x558bebf09c00 0x558beb0b9700 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x558bebd9e2a0 tx=0x558bebf1fe00 comp rx=0 tx=0).stop 2023-09-23T03:18:00.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.800+0000 7f3ecfa54100 1 -- 172.21.15.154:0/622366349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558bebf09800 msgr2=0x558beb0b8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:00.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.800+0000 7f3ecfa54100 1 --2- 172.21.15.154:0/622366349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558bebf09800 0x558beb0b8680 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x558bebf76cc0 tx=0x558bebda01e0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.800+0000 7f3ecd9f9640 1 -- 172.21.15.154:0/622366349 reap_dead start 2023-09-23T03:18:00.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.800+0000 7f3ecfa54100 1 -- 172.21.15.154:0/622366349 shutdown_connections 2023-09-23T03:18:00.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.800+0000 7f3ecfa54100 1 -- 172.21.15.154:0/622366349 >> 172.21.15.154:0/622366349 conn(0x558bebf08c00 msgr2=0x558bebdbd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:00.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.800+0000 7f3ecfa54100 1 -- 172.21.15.154:0/622366349 shutdown_connections 2023-09-23T03:18:00.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.800+0000 7f3ecfa54100 1 -- 172.21.15.154:0/622366349 wait complete. 2023-09-23T03:18:00.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.801+0000 7f3ecfa54100 1 -- 172.21.15.154:0/1651573537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558beb046c00 msgr2=0x558beb0b6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:00.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.801+0000 7f3ecfa54100 1 --2- 172.21.15.154:0/1651573537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558beb046c00 0x558beb0b6580 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x558bebe6e750 tx=0x558bebda0300 comp rx=0 tx=0).stop 2023-09-23T03:18:00.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.801+0000 7f3ecfa54100 1 -- 172.21.15.154:0/1651573537 shutdown_connections 2023-09-23T03:18:00.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.801+0000 7f3ecfa54100 1 --2- 172.21.15.154:0/1651573537 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558beb047c00 0x558beb0b7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.801+0000 7f3ecfa54100 1 --2- 172.21.15.154:0/1651573537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558beb047800 0x558beb0b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.801+0000 7f3ecfa54100 1 --2- 172.21.15.154:0/1651573537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558beb046c00 0x558beb0b6580 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.801+0000 7f3ecfa54100 1 -- 172.21.15.154:0/1651573537 >> 172.21.15.154:0/1651573537 conn(0x558beb047000 msgr2=0x558bebdbd000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:00.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.801+0000 7f3ecfa54100 1 -- 172.21.15.154:0/1651573537 shutdown_connections 2023-09-23T03:18:00.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.801+0000 7f3ecfa54100 1 -- 172.21.15.154:0/1651573537 wait complete. 2023-09-23T03:18:00.806 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'session'] in time 0:00:00.116210, result: { "0": { "data": {}, "result": 0 } } 2023-09-23T03:18:00.806 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset snap 2023-09-23T03:18:00.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.897+0000 7fdc7dc17100 1 -- start start 2023-09-23T03:18:00.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.897+0000 7fdc7dc17100 1 Processor -- start 2023-09-23T03:18:00.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.898+0000 7fdc7dc17100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559ca2b70c00 0x559ca2be0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:00.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.898+0000 7fdc7dc17100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ca2b71800 0x559ca2be0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:00.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.898+0000 7fdc7dc17100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ca2b71c00 0x559ca2be1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:00.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.898+0000 7fdc7dc17100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ca2bbd180 con 0x559ca2b71c00 2023-09-23T03:18:00.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.898+0000 7fdc7dc17100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ca2c3f180 con 0x559ca2b71800 2023-09-23T03:18:00.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.898+0000 7fdc7dc17100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ca2c3edc0 con 0x559ca2b70c00 2023-09-23T03:18:00.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.898+0000 7fdc7bf97640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ca2b71c00 0x559ca2be1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:00.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.898+0000 7fdc7bf97640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ca2b71c00 0x559ca2be1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60308/0 (socket says 172.21.15.154:60308) 2023-09-23T03:18:00.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.898+0000 7fdc7bf97640 1 -- 172.21.15.154:0/3998438578 learned_addr learned my addr 172.21.15.154:0/3998438578 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:00.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.898+0000 7fdc7af95640 1 --2- 172.21.15.154:0/3998438578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ca2b71800 0x559ca2be0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:00.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.898+0000 7fdc7b796640 1 --2- 172.21.15.154:0/3998438578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559ca2b70c00 0x559ca2be0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:00.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.898+0000 7fdc7bf97640 1 -- 172.21.15.154:0/3998438578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559ca2b70c00 msgr2=0x559ca2be0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:00.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.898+0000 7fdc7bf97640 1 --2- 172.21.15.154:0/3998438578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559ca2b70c00 0x559ca2be0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.898+0000 7fdc7bf97640 1 -- 172.21.15.154:0/3998438578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ca2b71800 msgr2=0x559ca2be0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:00.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.898+0000 7fdc7bf97640 1 --2- 172.21.15.154:0/3998438578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ca2b71800 0x559ca2be0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.899+0000 7fdc7bf97640 1 -- 172.21.15.154:0/3998438578 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ca2ba4b60 con 0x559ca2b71c00 2023-09-23T03:18:00.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.899+0000 7fdc7b796640 1 --2- 172.21.15.154:0/3998438578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559ca2b70c00 0x559ca2be0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:18:00.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.899+0000 7fdc7af95640 1 --2- 172.21.15.154:0/3998438578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ca2b71800 0x559ca2be0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:18:00.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.899+0000 7fdc7bf97640 1 --2- 172.21.15.154:0/3998438578 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ca2b71c00 0x559ca2be1080 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x559ca398abd0 tx=0x559ca2e54480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d77ec32e2923ca server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:00.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.899+0000 7fdc78f91640 1 -- 172.21.15.154:0/3998438578 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559ca2bbb600 con 0x559ca2b71c00 2023-09-23T03:18:00.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.899+0000 7fdc78f91640 1 -- 172.21.15.154:0/3998438578 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559ca2b97b00 con 0x559ca2b71c00 2023-09-23T03:18:00.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.899+0000 7fdc78f91640 1 -- 172.21.15.154:0/3998438578 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559ca2bbb1e0 con 0x559ca2b71c00 2023-09-23T03:18:00.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.899+0000 7fdc7dc17100 1 -- 172.21.15.154:0/3998438578 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559ca2ba4340 con 0x559ca2b71c00 2023-09-23T03:18:00.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.899+0000 7fdc7dc17100 1 -- 172.21.15.154:0/3998438578 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x559ca2ba4000 con 0x559ca2b71c00 2023-09-23T03:18:00.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.899+0000 7fdc7dc17100 4 waiting for MDS map... 2023-09-23T03:18:00.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.899+0000 7fdc78f91640 1 -- 172.21.15.154:0/3998438578 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(43..43 src has 1..43) v4 ==== 6605+0+0 (secure 0 0 0) 0x559ca2b9c380 con 0x559ca2b71c00 2023-09-23T03:18:00.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.899+0000 7fdc78f91640 1 -- 172.21.15.154:0/3998438578 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 25) v1 ==== 3306+0+0 (secure 0 0 0) 0x559ca2bc5200 con 0x559ca2b71c00 2023-09-23T03:18:00.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.899+0000 7fdc7dc17100 4 Got MDS map 25 2023-09-23T03:18:00.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.900+0000 7fdc7dc17100 10 main: main 2023-09-23T03:18:00.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.900+0000 7fdc7dc17100 4 main: connecting to RADOS... 2023-09-23T03:18:00.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.900+0000 7fdc7dc17100 1 Processor -- start 2023-09-23T03:18:00.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.900+0000 7fdc7dc17100 1 -- start start 2023-09-23T03:18:00.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.901+0000 7fdc7dc17100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559ca2e6b000 0x559ca2be1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:00.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.901+0000 7fdc7dc17100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ca2e6b400 0x559ca2be1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:00.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.901+0000 7fdc7dc17100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ca2e6b800 0x559ca2be2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:00.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.901+0000 7fdc7dc17100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ca2bbc140 con 0x559ca2e6b400 2023-09-23T03:18:00.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.901+0000 7fdc7dc17100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ca2bbcb40 con 0x559ca2e6b800 2023-09-23T03:18:00.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.901+0000 7fdc7dc17100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ca2c3f180 con 0x559ca2e6b000 2023-09-23T03:18:00.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.901+0000 7fdc7b796640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ca2e6b400 0x559ca2be1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:00.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.902+0000 7fdc7b796640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ca2e6b400 0x559ca2be1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60324/0 (socket says 172.21.15.154:60324) 2023-09-23T03:18:00.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.902+0000 7fdc7b796640 1 -- 172.21.15.154:0/1372298967 learned_addr learned my addr 172.21.15.154:0/1372298967 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:00.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.902+0000 7fdc7b796640 1 -- 172.21.15.154:0/1372298967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559ca2e6b000 msgr2=0x559ca2be1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:00.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.902+0000 7fdc7b796640 1 --2- 172.21.15.154:0/1372298967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559ca2e6b000 0x559ca2be1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.902+0000 7fdc7b796640 1 -- 172.21.15.154:0/1372298967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ca2e6b800 msgr2=0x559ca2be2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:00.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.902+0000 7fdc7b796640 1 --2- 172.21.15.154:0/1372298967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ca2e6b800 0x559ca2be2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.902+0000 7fdc7b796640 1 -- 172.21.15.154:0/1372298967 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ca2ba4ea0 con 0x559ca2e6b400 2023-09-23T03:18:00.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.902+0000 7fdc7b796640 1 --2- 172.21.15.154:0/1372298967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ca2e6b400 0x559ca2be1b80 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x559ca2ec20c0 tx=0x559ca2e542a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=623f4eb9cb975b24 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:00.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.902+0000 7fdc76f8d640 1 -- 172.21.15.154:0/1372298967 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559ca2bbb080 con 0x559ca2e6b400 2023-09-23T03:18:00.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.903+0000 7fdc76f8d640 1 -- 172.21.15.154:0/1372298967 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559ca3948900 con 0x559ca2e6b400 2023-09-23T03:18:00.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.903+0000 7fdc76f8d640 1 -- 172.21.15.154:0/1372298967 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559ca2bbb600 con 0x559ca2e6b400 2023-09-23T03:18:00.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.903+0000 7fdc7dc17100 1 -- 172.21.15.154:0/1372298967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ca2e6b400 msgr2=0x559ca2be1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:00.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.903+0000 7fdc7dc17100 1 --2- 172.21.15.154:0/1372298967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ca2e6b400 0x559ca2be1b80 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x559ca2ec20c0 tx=0x559ca2e542a0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.903+0000 7fdc7dc17100 1 -- 172.21.15.154:0/1372298967 shutdown_connections 2023-09-23T03:18:00.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.903+0000 7fdc7dc17100 1 --2- 172.21.15.154:0/1372298967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ca2e6b800 0x559ca2be2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.903+0000 7fdc7dc17100 1 --2- 172.21.15.154:0/1372298967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ca2e6b400 0x559ca2be1b80 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.903+0000 7fdc7dc17100 1 --2- 172.21.15.154:0/1372298967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559ca2e6b000 0x559ca2be1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.903+0000 7fdc7dc17100 1 -- 172.21.15.154:0/1372298967 >> 172.21.15.154:0/1372298967 conn(0x559ca2e6ac00 msgr2=0x559ca38e6800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:00.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.903+0000 7fdc7dc17100 1 -- 172.21.15.154:0/1372298967 shutdown_connections 2023-09-23T03:18:00.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.903+0000 7fdc7dc17100 1 -- 172.21.15.154:0/1372298967 wait complete. 2023-09-23T03:18:00.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.904+0000 7fdc7dc17100 1 Processor -- start 2023-09-23T03:18:00.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.904+0000 7fdc7dc17100 1 -- start start 2023-09-23T03:18:00.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.905+0000 7fdc7dc17100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559ca2e6b000 0x559ca2be3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:00.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.905+0000 7fdc7dc17100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ca2e6b400 0x559ca2be2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:00.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.905+0000 7fdc7dc17100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ca2e6b800 0x559ca2be2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:00.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.905+0000 7fdc7dc17100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ca2c3ec80 con 0x559ca2e6b800 2023-09-23T03:18:00.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.905+0000 7fdc7dc17100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ca2c3eb40 con 0x559ca2e6b400 2023-09-23T03:18:00.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.905+0000 7fdc7dc17100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x559ca2c3ea00 con 0x559ca2e6b000 2023-09-23T03:18:00.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.905+0000 7fdc7af95640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ca2e6b800 0x559ca2be2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:00.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.905+0000 7fdc7af95640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ca2e6b800 0x559ca2be2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60328/0 (socket says 172.21.15.154:60328) 2023-09-23T03:18:00.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.905+0000 7fdc7af95640 1 -- 172.21.15.154:0/660347732 learned_addr learned my addr 172.21.15.154:0/660347732 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:00.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.905+0000 7fdc7af95640 1 --2- 172.21.15.154:0/660347732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559ca2e6b000 0x559ca2be3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:00.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.905+0000 7fdc7b796640 1 --2- 172.21.15.154:0/660347732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ca2e6b400 0x559ca2be2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:00.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.905+0000 7fdc7af95640 1 -- 172.21.15.154:0/660347732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559ca2e6b000 msgr2=0x559ca2be3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:00.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.905+0000 7fdc7af95640 1 --2- 172.21.15.154:0/660347732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559ca2e6b000 0x559ca2be3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.905+0000 7fdc7af95640 1 -- 172.21.15.154:0/660347732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ca2e6b400 msgr2=0x559ca2be2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:00.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.905+0000 7fdc7af95640 1 --2- 172.21.15.154:0/660347732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ca2e6b400 0x559ca2be2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.905+0000 7fdc7af95640 1 -- 172.21.15.154:0/660347732 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ca2ba51e0 con 0x559ca2e6b800 2023-09-23T03:18:00.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.905+0000 7fdc7b796640 1 --2- 172.21.15.154:0/660347732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ca2e6b400 0x559ca2be2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:18:00.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.906+0000 7fdc7af95640 1 --2- 172.21.15.154:0/660347732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ca2e6b800 0x559ca2be2680 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x559ca2ec2ea0 tx=0x559ca2e74d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:00.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.906+0000 7fdc7578a640 1 -- 172.21.15.154:0/660347732 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559ca2bbb080 con 0x559ca2e6b800 2023-09-23T03:18:00.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.906+0000 7fdc7578a640 1 -- 172.21.15.154:0/660347732 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559ca2b97800 con 0x559ca2e6b800 2023-09-23T03:18:00.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.906+0000 7fdc7dc17100 1 -- 172.21.15.154:0/660347732 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559ca2ba4ea0 con 0x559ca2e6b800 2023-09-23T03:18:00.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.906+0000 7fdc7578a640 1 -- 172.21.15.154:0/660347732 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559ca2bbb600 con 0x559ca2e6b800 2023-09-23T03:18:00.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.906+0000 7fdc7dc17100 1 -- 172.21.15.154:0/660347732 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559ca2ba4b60 con 0x559ca2e6b800 2023-09-23T03:18:00.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.906+0000 7fdc7dc17100 4 main: resolving pool 5 2023-09-23T03:18:00.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.908+0000 7fdc7578a640 1 -- 172.21.15.154:0/660347732 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x559ca2ba2840 con 0x559ca2e6b800 2023-09-23T03:18:00.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.908+0000 7fdc7578a640 1 --2- 172.21.15.154:0/660347732 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x559ca2edc800 0x559ca2be3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:00.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.909+0000 7fdc7578a640 1 -- 172.21.15.154:0/660347732 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6605+0+0 (secure 0 0 0) 0x559ca2b9c540 con 0x559ca2e6b800 2023-09-23T03:18:00.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.909+0000 7fdc7dc17100 4 main: creating IoCtx.. 2023-09-23T03:18:00.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.909+0000 7fdc7dc17100 1 --2- 172.21.15.154:0/660347732 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x559ca2edcc00 0x559ca2be2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:00.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.909+0000 7fdc7dc17100 1 -- 172.21.15.154:0/660347732 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:1 5.d 5:b50e409b:::mds_snaptable:head [writefull 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x559ca2bcd880 con 0x559ca2edcc00 2023-09-23T03:18:00.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.909+0000 7fdc7b796640 1 --2- 172.21.15.154:0/660347732 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x559ca2edc800 0x559ca2be3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:00.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.909+0000 7fdc7af95640 1 --2- 172.21.15.154:0/660347732 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x559ca2edcc00 0x559ca2be2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:00.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.909+0000 7fdc7b796640 1 --2- 172.21.15.154:0/660347732 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x559ca2edc800 0x559ca2be3700 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x559ca2ec20c0 tx=0x559ca2e54180 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:00.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.910+0000 7fdc7af95640 1 --2- 172.21.15.154:0/660347732 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x559ca2edcc00 0x559ca2be2100 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:00.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.913+0000 7fdc7af95640 1 -- 172.21.15.154:0/660347732 <== osd.2 v2:172.21.15.154:6822/2884407243 1 ==== osd_op_reply(1 mds_snaptable [writefull 0~70] v43'3 uv3 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x559ca3944b40 con 0x559ca2edcc00 2023-09-23T03:18:00.929 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:18:00.929 INFO:teuthology.orchestra.run.smithi154.stdout: "result": 0 2023-09-23T03:18:00.929 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:18:00.930 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:18:00.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.913+0000 7fdc7dc17100 1 -- 172.21.15.154:0/660347732 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x559ca2edcc00 msgr2=0x559ca2be2100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:00.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.913+0000 7fdc7dc17100 1 --2- 172.21.15.154:0/660347732 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x559ca2edcc00 0x559ca2be2100 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.913+0000 7fdc7dc17100 1 -- 172.21.15.154:0/660347732 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x559ca2edc800 msgr2=0x559ca2be3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:00.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.913+0000 7fdc7dc17100 1 --2- 172.21.15.154:0/660347732 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x559ca2edc800 0x559ca2be3700 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x559ca2ec20c0 tx=0x559ca2e54180 comp rx=0 tx=0).stop 2023-09-23T03:18:00.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.913+0000 7fdc7dc17100 1 -- 172.21.15.154:0/660347732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ca2e6b800 msgr2=0x559ca2be2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:00.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.913+0000 7fdc7dc17100 1 --2- 172.21.15.154:0/660347732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ca2e6b800 0x559ca2be2680 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x559ca2ec2ea0 tx=0x559ca2e74d20 comp rx=0 tx=0).stop 2023-09-23T03:18:00.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.913+0000 7fdc7b796640 1 -- 172.21.15.154:0/660347732 reap_dead start 2023-09-23T03:18:00.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.913+0000 7fdc7dc17100 1 -- 172.21.15.154:0/660347732 shutdown_connections 2023-09-23T03:18:00.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.913+0000 7fdc7dc17100 1 -- 172.21.15.154:0/660347732 >> 172.21.15.154:0/660347732 conn(0x559ca2e6ac00 msgr2=0x559ca38e6800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:00.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.913+0000 7fdc7dc17100 1 -- 172.21.15.154:0/660347732 shutdown_connections 2023-09-23T03:18:00.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.913+0000 7fdc7dc17100 1 -- 172.21.15.154:0/660347732 wait complete. 2023-09-23T03:18:00.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.914+0000 7fdc7dc17100 1 -- 172.21.15.154:0/3998438578 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ca2b71c00 msgr2=0x559ca2be1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:00.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.914+0000 7fdc7dc17100 1 --2- 172.21.15.154:0/3998438578 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ca2b71c00 0x559ca2be1080 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x559ca398abd0 tx=0x559ca2e54480 comp rx=0 tx=0).stop 2023-09-23T03:18:00.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.914+0000 7fdc7dc17100 1 -- 172.21.15.154:0/3998438578 shutdown_connections 2023-09-23T03:18:00.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.914+0000 7fdc7dc17100 1 --2- 172.21.15.154:0/3998438578 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559ca2b71c00 0x559ca2be1080 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.914+0000 7fdc7dc17100 1 --2- 172.21.15.154:0/3998438578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559ca2b71800 0x559ca2be0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.914+0000 7fdc7dc17100 1 --2- 172.21.15.154:0/3998438578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559ca2b70c00 0x559ca2be0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:00.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.914+0000 7fdc7dc17100 1 -- 172.21.15.154:0/3998438578 >> 172.21.15.154:0/3998438578 conn(0x559ca2b71000 msgr2=0x559ca38e7000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:00.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.914+0000 7fdc7dc17100 1 -- 172.21.15.154:0/3998438578 shutdown_connections 2023-09-23T03:18:00.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:00.914+0000 7fdc7dc17100 1 -- 172.21.15.154:0/3998438578 wait complete. 2023-09-23T03:18:00.934 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'snap'] in time 0:00:00.127681, result: { "result": 0 } 2023-09-23T03:18:00.934 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset inode 2023-09-23T03:18:01.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.041+0000 7fdd2d647100 1 -- start start 2023-09-23T03:18:01.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.041+0000 7fdd2d647100 1 Processor -- start 2023-09-23T03:18:01.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.042+0000 7fdd2d647100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ace7858800 0x55ace6b3bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.042+0000 7fdd2d647100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ace6acbc00 0x55ace6b3ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.042+0000 7fdd2d647100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ace7859c00 0x55ace6b3b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.042+0000 7fdd2d647100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ace6b17180 con 0x55ace7858800 2023-09-23T03:18:01.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.042+0000 7fdd2d647100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ace6b16140 con 0x55ace6acbc00 2023-09-23T03:18:01.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.042+0000 7fdd2d647100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ace6b16b40 con 0x55ace7859c00 2023-09-23T03:18:01.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.042+0000 7fdd2b5ba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ace7858800 0x55ace6b3bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.042+0000 7fdd2b5ba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ace7858800 0x55ace6b3bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60332/0 (socket says 172.21.15.154:60332) 2023-09-23T03:18:01.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.042+0000 7fdd2b5ba640 1 -- 172.21.15.154:0/1461732845 learned_addr learned my addr 172.21.15.154:0/1461732845 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:01.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.043+0000 7fdd2adb9640 1 --2- 172.21.15.154:0/1461732845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ace6acbc00 0x55ace6b3ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.043+0000 7fdd2b5ba640 1 -- 172.21.15.154:0/1461732845 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ace7859c00 msgr2=0x55ace6b3b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:01.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.043+0000 7fdd2bdbb640 1 --2- 172.21.15.154:0/1461732845 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ace7859c00 0x55ace6b3b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.043+0000 7fdd2b5ba640 1 --2- 172.21.15.154:0/1461732845 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ace7859c00 0x55ace6b3b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.043+0000 7fdd2b5ba640 1 -- 172.21.15.154:0/1461732845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ace6acbc00 msgr2=0x55ace6b3ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:01.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.043+0000 7fdd2b5ba640 1 --2- 172.21.15.154:0/1461732845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ace6acbc00 0x55ace6b3ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.043+0000 7fdd2b5ba640 1 -- 172.21.15.154:0/1461732845 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ace6afeea0 con 0x55ace7858800 2023-09-23T03:18:01.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.043+0000 7fdd2adb9640 1 --2- 172.21.15.154:0/1461732845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ace6acbc00 0x55ace6b3ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:18:01.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.043+0000 7fdd2b5ba640 1 --2- 172.21.15.154:0/1461732845 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ace7858800 0x55ace6b3bb80 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x55ace78e4de0 tx=0x55ace7992300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b827034fb1fbaf36 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.043+0000 7fdd28db5640 1 -- 172.21.15.154:0/1461732845 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ace6b15600 con 0x55ace7858800 2023-09-23T03:18:01.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.043+0000 7fdd28db5640 1 -- 172.21.15.154:0/1461732845 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ace6af1800 con 0x55ace7858800 2023-09-23T03:18:01.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.043+0000 7fdd28db5640 1 -- 172.21.15.154:0/1461732845 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ace6b15080 con 0x55ace7858800 2023-09-23T03:18:01.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.043+0000 7fdd2d647100 1 -- 172.21.15.154:0/1461732845 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ace6aff520 con 0x55ace7858800 2023-09-23T03:18:01.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.043+0000 7fdd2d647100 1 -- 172.21.15.154:0/1461732845 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55ace6aff6c0 con 0x55ace7858800 2023-09-23T03:18:01.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.043+0000 7fdd2d647100 4 waiting for MDS map... 2023-09-23T03:18:01.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.044+0000 7fdd28db5640 1 -- 172.21.15.154:0/1461732845 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(43..43 src has 1..43) v4 ==== 6605+0+0 (secure 0 0 0) 0x55ace6af6380 con 0x55ace7858800 2023-09-23T03:18:01.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.044+0000 7fdd28db5640 1 -- 172.21.15.154:0/1461732845 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 25) v1 ==== 3306+0+0 (secure 0 0 0) 0x55ace6b1f200 con 0x55ace7858800 2023-09-23T03:18:01.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.044+0000 7fdd2d647100 4 Got MDS map 25 2023-09-23T03:18:01.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.044+0000 7fdd2d647100 10 main: main 2023-09-23T03:18:01.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.044+0000 7fdd2d647100 4 main: connecting to RADOS... 2023-09-23T03:18:01.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.045+0000 7fdd2d647100 1 Processor -- start 2023-09-23T03:18:01.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.045+0000 7fdd2d647100 1 -- start start 2023-09-23T03:18:01.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.046+0000 7fdd2d647100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ace7858000 0x55ace6b3b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.046+0000 7fdd2d647100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ace7994800 0x55ace6b3c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.046+0000 7fdd2d647100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ace7994c00 0x55ace6b3c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.046+0000 7fdd2d647100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ace6b16a00 con 0x55ace7994800 2023-09-23T03:18:01.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.046+0000 7fdd2d647100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ace6b16000 con 0x55ace7858000 2023-09-23T03:18:01.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.046+0000 7fdd2d647100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ace6b98dc0 con 0x55ace7994c00 2023-09-23T03:18:01.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.046+0000 7fdd2b5ba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ace7994800 0x55ace6b3c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.046+0000 7fdd2b5ba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ace7994800 0x55ace6b3c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60342/0 (socket says 172.21.15.154:60342) 2023-09-23T03:18:01.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.046+0000 7fdd2b5ba640 1 -- 172.21.15.154:0/4289875665 learned_addr learned my addr 172.21.15.154:0/4289875665 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:01.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.046+0000 7fdd2b5ba640 1 -- 172.21.15.154:0/4289875665 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ace7994c00 msgr2=0x55ace6b3c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:01.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.046+0000 7fdd2b5ba640 1 --2- 172.21.15.154:0/4289875665 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ace7994c00 0x55ace6b3c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.046+0000 7fdd2bdbb640 1 --2- 172.21.15.154:0/4289875665 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ace7858000 0x55ace6b3b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.046+0000 7fdd2b5ba640 1 -- 172.21.15.154:0/4289875665 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ace7858000 msgr2=0x55ace6b3b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:01.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.046+0000 7fdd2b5ba640 1 --2- 172.21.15.154:0/4289875665 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ace7858000 0x55ace6b3b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.046+0000 7fdd2b5ba640 1 -- 172.21.15.154:0/4289875665 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ace6aff860 con 0x55ace7994800 2023-09-23T03:18:01.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.047+0000 7fdd2b5ba640 1 --2- 172.21.15.154:0/4289875665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ace7994800 0x55ace6b3c100 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x55ace79f6150 tx=0x55ace785b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44fafbbf5f60cf90 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.047+0000 7fdd26db1640 1 -- 172.21.15.154:0/4289875665 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ace6b14f20 con 0x55ace7994800 2023-09-23T03:18:01.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.047+0000 7fdd26db1640 1 -- 172.21.15.154:0/4289875665 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ace799ac00 con 0x55ace7994800 2023-09-23T03:18:01.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.047+0000 7fdd26db1640 1 -- 172.21.15.154:0/4289875665 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ace6b15600 con 0x55ace7994800 2023-09-23T03:18:01.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.047+0000 7fdd2d647100 1 -- 172.21.15.154:0/4289875665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ace7994800 msgr2=0x55ace6b3c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.047+0000 7fdd2d647100 1 --2- 172.21.15.154:0/4289875665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ace7994800 0x55ace6b3c100 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x55ace79f6150 tx=0x55ace785b680 comp rx=0 tx=0).stop 2023-09-23T03:18:01.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.047+0000 7fdd2d647100 1 -- 172.21.15.154:0/4289875665 shutdown_connections 2023-09-23T03:18:01.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.047+0000 7fdd2d647100 1 --2- 172.21.15.154:0/4289875665 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ace7994c00 0x55ace6b3c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.047+0000 7fdd2d647100 1 --2- 172.21.15.154:0/4289875665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ace7994800 0x55ace6b3c100 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.047+0000 7fdd2d647100 1 --2- 172.21.15.154:0/4289875665 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ace7858000 0x55ace6b3b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.047+0000 7fdd2d647100 1 -- 172.21.15.154:0/4289875665 >> 172.21.15.154:0/4289875665 conn(0x55ace7858400 msgr2=0x55ace7843800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:01.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.047+0000 7fdd2d647100 1 -- 172.21.15.154:0/4289875665 shutdown_connections 2023-09-23T03:18:01.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.048+0000 7fdd2d647100 1 -- 172.21.15.154:0/4289875665 wait complete. 2023-09-23T03:18:01.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.048+0000 7fdd2d647100 1 Processor -- start 2023-09-23T03:18:01.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.048+0000 7fdd2d647100 1 -- start start 2023-09-23T03:18:01.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.049+0000 7fdd2d647100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ace7858400 0x55ace6b3d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.049+0000 7fdd2d647100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ace7995800 0x55ace6b3d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.049+0000 7fdd2d647100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ace7995400 0x55ace6b3cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.049+0000 7fdd2d647100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ace6b99180 con 0x55ace7995400 2023-09-23T03:18:01.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.049+0000 7fdd2d647100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ace6b98c80 con 0x55ace7995800 2023-09-23T03:18:01.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.049+0000 7fdd2d647100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ace6b98b40 con 0x55ace7858400 2023-09-23T03:18:01.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.049+0000 7fdd2adb9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ace7995400 0x55ace6b3cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.049+0000 7fdd2adb9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ace7995400 0x55ace6b3cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60354/0 (socket says 172.21.15.154:60354) 2023-09-23T03:18:01.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.049+0000 7fdd2adb9640 1 -- 172.21.15.154:0/1053416899 learned_addr learned my addr 172.21.15.154:0/1053416899 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:01.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.050+0000 7fdd2b5ba640 1 --2- 172.21.15.154:0/1053416899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ace7995800 0x55ace6b3d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.050+0000 7fdd2adb9640 1 -- 172.21.15.154:0/1053416899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ace7858400 msgr2=0x55ace6b3d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.050+0000 7fdd2adb9640 1 --2- 172.21.15.154:0/1053416899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ace7858400 0x55ace6b3d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.050+0000 7fdd2adb9640 1 -- 172.21.15.154:0/1053416899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ace7995800 msgr2=0x55ace6b3d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.050+0000 7fdd2adb9640 1 --2- 172.21.15.154:0/1053416899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ace7995800 0x55ace6b3d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.050+0000 7fdd2adb9640 1 -- 172.21.15.154:0/1053416899 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ace6afeea0 con 0x55ace7995400 2023-09-23T03:18:01.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.050+0000 7fdd2adb9640 1 --2- 172.21.15.154:0/1053416899 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ace7995400 0x55ace6b3cc00 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x55ace79f6d20 tx=0x55ace79b4e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.050+0000 7fdd255ae640 1 -- 172.21.15.154:0/1053416899 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ace6b14f20 con 0x55ace7995400 2023-09-23T03:18:01.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.050+0000 7fdd255ae640 1 -- 172.21.15.154:0/1053416899 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ace6af1800 con 0x55ace7995400 2023-09-23T03:18:01.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.050+0000 7fdd2d647100 1 -- 172.21.15.154:0/1053416899 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ace6aff520 con 0x55ace7995400 2023-09-23T03:18:01.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.050+0000 7fdd255ae640 1 -- 172.21.15.154:0/1053416899 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ace6b15600 con 0x55ace7995400 2023-09-23T03:18:01.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.051+0000 7fdd2d647100 1 -- 172.21.15.154:0/1053416899 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ace6affa00 con 0x55ace7995400 2023-09-23T03:18:01.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.051+0000 7fdd2d647100 4 main: resolving pool 5 2023-09-23T03:18:01.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.052+0000 7fdd255ae640 1 -- 172.21.15.154:0/1053416899 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x55ace6afc840 con 0x55ace7995400 2023-09-23T03:18:01.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.053+0000 7fdd255ae640 1 --2- 172.21.15.154:0/1053416899 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55ace79fdc00 0x55ace6e26000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.053+0000 7fdd255ae640 1 -- 172.21.15.154:0/1053416899 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6605+0+0 (secure 0 0 0) 0x55ace6af6540 con 0x55ace7995400 2023-09-23T03:18:01.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.053+0000 7fdd2d647100 4 main: creating IoCtx.. 2023-09-23T03:18:01.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.053+0000 7fdd2d647100 1 --2- 172.21.15.154:0/1053416899 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55ace7995000 0x55ace6b3c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.053+0000 7fdd2bdbb640 1 --2- 172.21.15.154:0/1053416899 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55ace79fdc00 0x55ace6e26000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.053+0000 7fdd2d647100 1 -- 172.21.15.154:0/1053416899 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:1 5.13 5:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55ace6b27880 con 0x55ace7995000 2023-09-23T03:18:01.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.053+0000 7fdd2b5ba640 1 --2- 172.21.15.154:0/1053416899 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55ace7995000 0x55ace6b3c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.054+0000 7fdd2bdbb640 1 --2- 172.21.15.154:0/1053416899 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55ace79fdc00 0x55ace6e26000 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55ace782e450 tx=0x55ace7992120 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.054+0000 7fdd2b5ba640 1 --2- 172.21.15.154:0/1053416899 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55ace7995000 0x55ace6b3c680 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.056+0000 7fdd2b5ba640 1 -- 172.21.15.154:0/1053416899 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v43'6 uv6 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x55ace789cb40 con 0x55ace7995000 2023-09-23T03:18:01.062 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:18:01.062 INFO:teuthology.orchestra.run.smithi154.stdout: "0": { 2023-09-23T03:18:01.062 INFO:teuthology.orchestra.run.smithi154.stdout: "data": {}, 2023-09-23T03:18:01.062 INFO:teuthology.orchestra.run.smithi154.stdout: "result": 0 2023-09-23T03:18:01.062 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:18:01.063 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:18:01.063 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:18:01.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.056+0000 7fdd2d647100 1 -- 172.21.15.154:0/1053416899 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55ace7995000 msgr2=0x55ace6b3c680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.056+0000 7fdd2d647100 1 --2- 172.21.15.154:0/1053416899 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55ace7995000 0x55ace6b3c680 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.056+0000 7fdd2d647100 1 -- 172.21.15.154:0/1053416899 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55ace79fdc00 msgr2=0x55ace6e26000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.056+0000 7fdd2d647100 1 --2- 172.21.15.154:0/1053416899 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55ace79fdc00 0x55ace6e26000 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55ace782e450 tx=0x55ace7992120 comp rx=0 tx=0).stop 2023-09-23T03:18:01.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.056+0000 7fdd2d647100 1 -- 172.21.15.154:0/1053416899 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ace7995400 msgr2=0x55ace6b3cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.056+0000 7fdd2d647100 1 --2- 172.21.15.154:0/1053416899 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ace7995400 0x55ace6b3cc00 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x55ace79f6d20 tx=0x55ace79b4e40 comp rx=0 tx=0).stop 2023-09-23T03:18:01.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.056+0000 7fdd2adb9640 1 -- 172.21.15.154:0/1053416899 reap_dead start 2023-09-23T03:18:01.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.056+0000 7fdd2d647100 1 -- 172.21.15.154:0/1053416899 shutdown_connections 2023-09-23T03:18:01.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.056+0000 7fdd2d647100 1 -- 172.21.15.154:0/1053416899 >> 172.21.15.154:0/1053416899 conn(0x55ace7858000 msgr2=0x55ace7843800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:01.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.056+0000 7fdd2d647100 1 -- 172.21.15.154:0/1053416899 shutdown_connections 2023-09-23T03:18:01.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.056+0000 7fdd2d647100 1 -- 172.21.15.154:0/1053416899 wait complete. 2023-09-23T03:18:01.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.057+0000 7fdd2d647100 1 -- 172.21.15.154:0/1461732845 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ace7858800 msgr2=0x55ace6b3bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.057+0000 7fdd2d647100 1 --2- 172.21.15.154:0/1461732845 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ace7858800 0x55ace6b3bb80 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x55ace78e4de0 tx=0x55ace7992300 comp rx=0 tx=0).stop 2023-09-23T03:18:01.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.057+0000 7fdd2d647100 1 -- 172.21.15.154:0/1461732845 shutdown_connections 2023-09-23T03:18:01.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.057+0000 7fdd2d647100 1 --2- 172.21.15.154:0/1461732845 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55ace7859c00 0x55ace6b3b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.057+0000 7fdd2d647100 1 --2- 172.21.15.154:0/1461732845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55ace6acbc00 0x55ace6b3ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.057+0000 7fdd2d647100 1 --2- 172.21.15.154:0/1461732845 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ace7858800 0x55ace6b3bb80 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.057+0000 7fdd2d647100 1 -- 172.21.15.154:0/1461732845 >> 172.21.15.154:0/1461732845 conn(0x55ace6acb000 msgr2=0x55ace7842000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:01.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.057+0000 7fdd2d647100 1 -- 172.21.15.154:0/1461732845 shutdown_connections 2023-09-23T03:18:01.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.057+0000 7fdd2d647100 1 -- 172.21.15.154:0/1461732845 wait complete. 2023-09-23T03:18:01.068 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'inode'] in time 0:00:00.133611, result: { "0": { "data": {}, "result": 0 } } 2023-09-23T03:18:01.068 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2023-09-23T03:18:01.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.155+0000 7f3c5144c100 1 -- start start 2023-09-23T03:18:01.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.155+0000 7f3c5144c100 1 Processor -- start 2023-09-23T03:18:01.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.156+0000 7f3c5144c100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a7cf6eb800 0x55a7cf75ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.156+0000 7f3c5144c100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a7d04ae800 0x55a7cf75bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.156+0000 7f3c5144c100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7d04afc00 0x55a7cf75b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.156+0000 7f3c5144c100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7cf737180 con 0x55a7d04ae800 2023-09-23T03:18:01.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.156+0000 7f3c5144c100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7cf736b40 con 0x55a7d04afc00 2023-09-23T03:18:01.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.156+0000 7f3c5144c100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a7cf736000 con 0x55a7cf6eb800 2023-09-23T03:18:01.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.157+0000 7f3c4ec54640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a7d04ae800 0x55a7cf75bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.157+0000 7f3c4ec54640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a7d04ae800 0x55a7cf75bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60374/0 (socket says 172.21.15.154:60374) 2023-09-23T03:18:01.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.157+0000 7f3c4ec54640 1 -- 172.21.15.154:0/1808501719 learned_addr learned my addr 172.21.15.154:0/1808501719 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:01.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.157+0000 7f3c4ec54640 1 -- 172.21.15.154:0/1808501719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a7cf6eb800 msgr2=0x55a7cf75ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:01.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.157+0000 7f3c4ec54640 1 --2- 172.21.15.154:0/1808501719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a7cf6eb800 0x55a7cf75ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.157+0000 7f3c4ec54640 1 -- 172.21.15.154:0/1808501719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7d04afc00 msgr2=0x55a7cf75b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:01.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.157+0000 7f3c4ec54640 1 --2- 172.21.15.154:0/1808501719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7d04afc00 0x55a7cf75b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.157+0000 7f3c4ec54640 1 -- 172.21.15.154:0/1808501719 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7cf71e340 con 0x55a7d04ae800 2023-09-23T03:18:01.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.157+0000 7f3c4ec54640 1 --2- 172.21.15.154:0/1808501719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a7d04ae800 0x55a7cf75bb80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x55a7d0502cc0 tx=0x55a7cf9ca000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f44ebf5ceac7f6e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.157+0000 7f3c4cc50640 1 -- 172.21.15.154:0/1808501719 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a7cf735080 con 0x55a7d04ae800 2023-09-23T03:18:01.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.157+0000 7f3c4cc50640 1 -- 172.21.15.154:0/1808501719 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a7cf711800 con 0x55a7d04ae800 2023-09-23T03:18:01.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.157+0000 7f3c5144c100 1 -- 172.21.15.154:0/1808501719 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7cf71f1e0 con 0x55a7d04ae800 2023-09-23T03:18:01.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.157+0000 7f3c5144c100 1 -- 172.21.15.154:0/1808501719 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55a7cf71e9c0 con 0x55a7d04ae800 2023-09-23T03:18:01.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.157+0000 7f3c5144c100 4 waiting for MDS map... 2023-09-23T03:18:01.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.158+0000 7f3c4cc50640 1 -- 172.21.15.154:0/1808501719 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a7cf735600 con 0x55a7d04ae800 2023-09-23T03:18:01.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.158+0000 7f3c4cc50640 1 -- 172.21.15.154:0/1808501719 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(43..43 src has 1..43) v4 ==== 6605+0+0 (secure 0 0 0) 0x55a7cf716380 con 0x55a7d04ae800 2023-09-23T03:18:01.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.158+0000 7f3c4cc50640 1 -- 172.21.15.154:0/1808501719 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 25) v1 ==== 3306+0+0 (secure 0 0 0) 0x55a7cf73f200 con 0x55a7d04ae800 2023-09-23T03:18:01.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.158+0000 7f3c5144c100 4 Got MDS map 25 2023-09-23T03:18:01.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.158+0000 7f3c5144c100 10 main: JournalTool::main 2023-09-23T03:18:01.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.158+0000 7f3c5144c100 4 main: JournalTool: connecting to RADOS... 2023-09-23T03:18:01.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.159+0000 7f3c5144c100 1 Processor -- start 2023-09-23T03:18:01.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.159+0000 7f3c5144c100 1 -- start start 2023-09-23T03:18:01.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.160+0000 7f3c5144c100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a7d04af800 0x55a7cf75b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.160+0000 7f3c5144c100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a7cf9ccc00 0x55a7cf75c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.160+0000 7f3c5144c100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7cf9cd000 0x55a7cf75c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.160+0000 7f3c5144c100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7cf736140 con 0x55a7cf9ccc00 2023-09-23T03:18:01.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.160+0000 7f3c5144c100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7cf736a00 con 0x55a7cf9cd000 2023-09-23T03:18:01.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.160+0000 7f3c5144c100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a7cf7b8dc0 con 0x55a7d04af800 2023-09-23T03:18:01.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.160+0000 7f3c4f455640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a7cf9ccc00 0x55a7cf75c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.160+0000 7f3c4f455640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a7cf9ccc00 0x55a7cf75c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60376/0 (socket says 172.21.15.154:60376) 2023-09-23T03:18:01.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.160+0000 7f3c4f455640 1 -- 172.21.15.154:0/2918681047 learned_addr learned my addr 172.21.15.154:0/2918681047 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:01.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.160+0000 7f3c4fc56640 1 --2- 172.21.15.154:0/2918681047 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a7d04af800 0x55a7cf75b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.160+0000 7f3c4ec54640 1 --2- 172.21.15.154:0/2918681047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7cf9cd000 0x55a7cf75c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.160+0000 7f3c4f455640 1 -- 172.21.15.154:0/2918681047 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a7d04af800 msgr2=0x55a7cf75b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:01.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.160+0000 7f3c4f455640 1 --2- 172.21.15.154:0/2918681047 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a7d04af800 0x55a7cf75b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.161+0000 7f3c4f455640 1 -- 172.21.15.154:0/2918681047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7cf9cd000 msgr2=0x55a7cf75c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:01.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.161+0000 7f3c4f455640 1 --2- 172.21.15.154:0/2918681047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7cf9cd000 0x55a7cf75c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.161+0000 7f3c4f455640 1 -- 172.21.15.154:0/2918681047 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7cf71e000 con 0x55a7cf9ccc00 2023-09-23T03:18:01.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.161+0000 7f3c4ec54640 1 --2- 172.21.15.154:0/2918681047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7cf9cd000 0x55a7cf75c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:18:01.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.161+0000 7f3c4f455640 1 --2- 172.21.15.154:0/2918681047 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a7cf9ccc00 0x55a7cf75c100 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x55a7cfa38270 tx=0x55a7cf9e6f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c38c9e257b9f3a2 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.161+0000 7f3c4ac4c640 1 -- 172.21.15.154:0/2918681047 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a7cf735080 con 0x55a7cf9ccc00 2023-09-23T03:18:01.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.161+0000 7f3c4ac4c640 1 -- 172.21.15.154:0/2918681047 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a7cf9d2a80 con 0x55a7cf9ccc00 2023-09-23T03:18:01.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.161+0000 7f3c4ac4c640 1 -- 172.21.15.154:0/2918681047 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a7cf735600 con 0x55a7cf9ccc00 2023-09-23T03:18:01.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.162+0000 7f3c5144c100 1 -- 172.21.15.154:0/2918681047 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a7cf9ccc00 msgr2=0x55a7cf75c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.162+0000 7f3c5144c100 1 --2- 172.21.15.154:0/2918681047 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a7cf9ccc00 0x55a7cf75c100 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x55a7cfa38270 tx=0x55a7cf9e6f00 comp rx=0 tx=0).stop 2023-09-23T03:18:01.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.162+0000 7f3c5144c100 1 -- 172.21.15.154:0/2918681047 shutdown_connections 2023-09-23T03:18:01.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.162+0000 7f3c5144c100 1 --2- 172.21.15.154:0/2918681047 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7cf9cd000 0x55a7cf75c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.162+0000 7f3c5144c100 1 --2- 172.21.15.154:0/2918681047 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a7cf9ccc00 0x55a7cf75c100 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.162+0000 7f3c5144c100 1 --2- 172.21.15.154:0/2918681047 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a7d04af800 0x55a7cf75b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.162+0000 7f3c5144c100 1 -- 172.21.15.154:0/2918681047 >> 172.21.15.154:0/2918681047 conn(0x55a7d04af400 msgr2=0x55a7d0463800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:01.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.162+0000 7f3c5144c100 1 -- 172.21.15.154:0/2918681047 shutdown_connections 2023-09-23T03:18:01.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.162+0000 7f3c5144c100 1 -- 172.21.15.154:0/2918681047 wait complete. 2023-09-23T03:18:01.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.162+0000 7f3c5144c100 1 Processor -- start 2023-09-23T03:18:01.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.163+0000 7f3c5144c100 1 -- start start 2023-09-23T03:18:01.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.163+0000 7f3c5144c100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7d04af400 0x55a7cf75c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.163+0000 7f3c5144c100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a7cf9cdc00 0x55a7cf75c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.163+0000 7f3c5144c100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a7cf9cd800 0x55a7cf75b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.163+0000 7f3c5144c100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7cf7b8c80 con 0x55a7cf9cd800 2023-09-23T03:18:01.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.163+0000 7f3c5144c100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7cf7b8b40 con 0x55a7d04af400 2023-09-23T03:18:01.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.163+0000 7f3c5144c100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a7cf7b8a00 con 0x55a7cf9cdc00 2023-09-23T03:18:01.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.163+0000 7f3c4ec54640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a7cf9cd800 0x55a7cf75b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.164+0000 7f3c4ec54640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a7cf9cd800 0x55a7cf75b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60382/0 (socket says 172.21.15.154:60382) 2023-09-23T03:18:01.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.164+0000 7f3c4ec54640 1 -- 172.21.15.154:0/1366134503 learned_addr learned my addr 172.21.15.154:0/1366134503 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:01.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.164+0000 7f3c4ec54640 1 -- 172.21.15.154:0/1366134503 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a7cf9cdc00 msgr2=0x55a7cf75c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.164+0000 7f3c4ec54640 1 --2- 172.21.15.154:0/1366134503 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a7cf9cdc00 0x55a7cf75c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.164+0000 7f3c4ec54640 1 -- 172.21.15.154:0/1366134503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7d04af400 msgr2=0x55a7cf75c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.164+0000 7f3c4ec54640 1 --2- 172.21.15.154:0/1366134503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7d04af400 0x55a7cf75c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.164+0000 7f3c4ec54640 1 -- 172.21.15.154:0/1366134503 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7cf71e340 con 0x55a7cf9cd800 2023-09-23T03:18:01.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.164+0000 7f3c4ec54640 1 --2- 172.21.15.154:0/1366134503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a7cf9cd800 0x55a7cf75b080 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x55a7cfa38120 tx=0x55a7d047b620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.164+0000 7f3c49449640 1 -- 172.21.15.154:0/1366134503 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a7cf7351e0 con 0x55a7cf9cd800 2023-09-23T03:18:01.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.164+0000 7f3c49449640 1 -- 172.21.15.154:0/1366134503 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a7cf9d2900 con 0x55a7cf9cd800 2023-09-23T03:18:01.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.164+0000 7f3c49449640 1 -- 172.21.15.154:0/1366134503 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a7cf735080 con 0x55a7cf9cd800 2023-09-23T03:18:01.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.164+0000 7f3c5144c100 1 -- 172.21.15.154:0/1366134503 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a7cf71f1e0 con 0x55a7cf9cd800 2023-09-23T03:18:01.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.165+0000 7f3c5144c100 1 -- 172.21.15.154:0/1366134503 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7cf71e9c0 con 0x55a7cf9cd800 2023-09-23T03:18:01.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.165+0000 7f3c5144c100 4 main: JournalTool: resolving pool 5 2023-09-23T03:18:01.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.166+0000 7f3c49449640 1 -- 172.21.15.154:0/1366134503 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x55a7cf71cb00 con 0x55a7cf9cd800 2023-09-23T03:18:01.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.167+0000 7f3c49449640 1 --2- 172.21.15.154:0/1366134503 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55a7cfa45c00 0x55a7cf75d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.167+0000 7f3c49449640 1 -- 172.21.15.154:0/1366134503 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6605+0+0 (secure 0 0 0) 0x55a7cf716540 con 0x55a7cf9cd800 2023-09-23T03:18:01.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.167+0000 7f3c5144c100 4 main: JournalTool: creating IoCtx.. 2023-09-23T03:18:01.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.167+0000 7f3c5144c100 4 main: Executing for rank 0 2023-09-23T03:18:01.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.167+0000 7f3c4fc56640 1 --2- 172.21.15.154:0/1366134503 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55a7cfa45c00 0x55a7cf75d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.168+0000 7f3c4fc56640 1 --2- 172.21.15.154:0/1366134503 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55a7cfa45c00 0x55a7cf75d700 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x55a7cf7c6210 tx=0x55a7cf9e6d20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.168+0000 7f3c5144c100 1 -- start start 2023-09-23T03:18:01.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.168+0000 7f3c5144c100 1 Processor -- start 2023-09-23T03:18:01.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.169+0000 7f3c5144c100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a7cf9cd000 0x55a7cf75cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.169+0000 7f3c5144c100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a7cf9ccc00 0x55a7cf75d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.169+0000 7f3c5144c100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7cfba0000 0x55a7cfba2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.169+0000 7f3c5144c100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7cf736b40 con 0x55a7cf9cd000 2023-09-23T03:18:01.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.169+0000 7f3c5144c100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7cf7b8a00 con 0x55a7cfba0000 2023-09-23T03:18:01.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.169+0000 7f3c5144c100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a7cf7b8b40 con 0x55a7cf9ccc00 2023-09-23T03:18:01.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.169+0000 7f3c4fc56640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a7cf9cd000 0x55a7cf75cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.169+0000 7f3c4fc56640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a7cf9cd000 0x55a7cf75cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60388/0 (socket says 172.21.15.154:60388) 2023-09-23T03:18:01.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.169+0000 7f3c4fc56640 1 -- 172.21.15.154:0/3318852807 learned_addr learned my addr 172.21.15.154:0/3318852807 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:01.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.170+0000 7f3c4fc56640 1 -- 172.21.15.154:0/3318852807 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a7cf9ccc00 msgr2=0x55a7cf75d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:01.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.170+0000 7f3c4fc56640 1 --2- 172.21.15.154:0/3318852807 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a7cf9ccc00 0x55a7cf75d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.170+0000 7f3c4fc56640 1 -- 172.21.15.154:0/3318852807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7cfba0000 msgr2=0x55a7cfba2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:01.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.170+0000 7f3c4fc56640 1 --2- 172.21.15.154:0/3318852807 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7cfba0000 0x55a7cfba2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.170+0000 7f3c4fc56640 1 -- 172.21.15.154:0/3318852807 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7cfa7cb60 con 0x55a7cf9cd000 2023-09-23T03:18:01.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.170+0000 7f3c4fc56640 1 --2- 172.21.15.154:0/3318852807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a7cf9cd000 0x55a7cf75cc00 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x55a7cfaf7110 tx=0x55a7cf9e6de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ac14c2605d9cedb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.170+0000 7f3c46443640 1 -- 172.21.15.154:0/3318852807 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a7cf7342c0 con 0x55a7cf9cd000 2023-09-23T03:18:01.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.170+0000 7f3c46443640 1 -- 172.21.15.154:0/3318852807 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a7cf9d3800 con 0x55a7cf9cd000 2023-09-23T03:18:01.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.170+0000 7f3c46443640 1 -- 172.21.15.154:0/3318852807 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a7cf7346e0 con 0x55a7cf9cd000 2023-09-23T03:18:01.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.170+0000 7f3c5144c100 1 -- 172.21.15.154:0/3318852807 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7cfa7c9c0 con 0x55a7cf9cd000 2023-09-23T03:18:01.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.170+0000 7f3c5144c100 1 -- 172.21.15.154:0/3318852807 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55a7cfa7c820 con 0x55a7cf9cd000 2023-09-23T03:18:01.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.170+0000 7f3c5144c100 4 waiting for MDS map... 2023-09-23T03:18:01.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.170+0000 7f3c46443640 1 -- 172.21.15.154:0/3318852807 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(43..43 src has 1..43) v4 ==== 6605+0+0 (secure 0 0 0) 0x55a7cf7168c0 con 0x55a7cf9cd000 2023-09-23T03:18:01.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.171+0000 7f3c46443640 1 -- 172.21.15.154:0/3318852807 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 25) v1 ==== 3306+0+0 (secure 0 0 0) 0x55a7cfb93800 con 0x55a7cf9cd000 2023-09-23T03:18:01.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.171+0000 7f3c5144c100 4 Got MDS map 25 2023-09-23T03:18:01.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.171+0000 7f3c5144c100 1 --2- 172.21.15.154:0/3318852807 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55a7cfba0400 0x55a7cfba2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.171+0000 7f3c5144c100 1 -- 172.21.15.154:0/3318852807 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:1 5.f 5:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55a7cfbc0000 con 0x55a7cfba0400 2023-09-23T03:18:01.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.171+0000 7f3c4f455640 1 --2- 172.21.15.154:0/3318852807 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55a7cfba0400 0x55a7cfba2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.172+0000 7f3c4f455640 1 --2- 172.21.15.154:0/3318852807 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55a7cfba0400 0x55a7cfba2580 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=26d28828f48901f9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.177+0000 7f3c4f455640 1 -- 172.21.15.154:0/3318852807 <== osd.5 v2:172.21.15.190:6800/882536885 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v43'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55a7d04beb40 con 0x55a7cfba0400 2023-09-23T03:18:01.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.177+0000 7f3c5144c100 1 --2- 172.21.15.154:0/3318852807 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55a7cfba0800 0x55a7cfba2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.177+0000 7f3c5144c100 1 -- 172.21.15.154:0/3318852807 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:2 5.14 5:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55a7cfbc0380 con 0x55a7cfba0800 2023-09-23T03:18:01.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.177+0000 7f3c4ec54640 1 --2- 172.21.15.154:0/3318852807 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55a7cfba0800 0x55a7cfba2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.177+0000 7f3c4ec54640 1 --2- 172.21.15.154:0/3318852807 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55a7cfba0800 0x55a7cfba2b00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=9b75b76aace530f0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.181+0000 7f3c4ec54640 1 -- 172.21.15.154:0/3318852807 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v43'39 uv39 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55a7d04bed80 con 0x55a7cfba0800 2023-09-23T03:18:01.181 INFO:teuthology.orchestra.run.smithi154.stdout:writing EResetJournal entry 2023-09-23T03:18:01.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.181+0000 7f3c5144c100 1 -- 172.21.15.154:0/3318852807 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:3 5.14 5:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55a7cfbc0700 con 0x55a7cfba0800 2023-09-23T03:18:01.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.181+0000 7f3c5144c100 1 -- 172.21.15.154:0/3318852807 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:4 5.1b 5:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55a7cfbc0a80 con 0x55a7cfba0400 2023-09-23T03:18:01.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.181+0000 7f3c5144c100 1 --2- 172.21.15.154:0/3318852807 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55a7cfba0c00 0x55a7cfba3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.181+0000 7f3c5144c100 1 -- 172.21.15.154:0/3318852807 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:5 5.b 5:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55a7cfbc0e00 con 0x55a7cfba0c00 2023-09-23T03:18:01.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.181+0000 7f3c5144c100 1 --2- 172.21.15.154:0/3318852807 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55a7cfba1000 0x55a7cfba3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.181+0000 7f3c5144c100 1 -- 172.21.15.154:0/3318852807 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:6 5.1f 5:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55a7cfbc1180 con 0x55a7cfba1000 2023-09-23T03:18:01.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.181+0000 7f3c5144c100 1 --2- 172.21.15.154:0/3318852807 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55a7cfba1400 0x55a7cfba3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.181+0000 7f3c5144c100 1 -- 172.21.15.154:0/3318852807 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:7 5.1a 5:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55a7cfbc1500 con 0x55a7cfba1400 2023-09-23T03:18:01.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.181+0000 7f3c5144c100 1 --2- 172.21.15.154:0/3318852807 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55a7cfba1800 0x55a7cfba4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.181+0000 7f3c5144c100 1 -- 172.21.15.154:0/3318852807 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:8 5.1e 5:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55a7cfbc0000 con 0x55a7cfba1800 2023-09-23T03:18:01.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.181+0000 7f3c4fc56640 1 --2- 172.21.15.154:0/3318852807 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55a7cfba0c00 0x55a7cfba3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.182+0000 7f3c4fc56640 1 --2- 172.21.15.154:0/3318852807 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55a7cfba1800 0x55a7cfba4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.182+0000 7f3c4f455640 1 --2- 172.21.15.154:0/3318852807 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55a7cfba1000 0x55a7cfba3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.182+0000 7f3c4fc56640 1 --2- 172.21.15.154:0/3318852807 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55a7cfba0c00 0x55a7cfba3080 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=2788060608677b95 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.182+0000 7f3c4fc56640 1 --2- 172.21.15.154:0/3318852807 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55a7cfba1800 0x55a7cfba4100 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=33398e4d427582bf server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.182+0000 7f3c4ec54640 1 --2- 172.21.15.154:0/3318852807 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55a7cfba1400 0x55a7cfba3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.184+0000 7f3c4f455640 1 --2- 172.21.15.154:0/3318852807 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55a7cfba1000 0x55a7cfba3600 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=42f817e8cff42dff server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.184+0000 7f3c4ec54640 1 --2- 172.21.15.154:0/3318852807 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55a7cfba1400 0x55a7cfba3b80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=637b987500bf9f1c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.184+0000 7f3c4ec54640 1 -- 172.21.15.154:0/3318852807 <== osd.3 v2:172.21.15.154:6808/1133688468 2 ==== osd_op_reply(3 200.00000001 [delete] v43'40 uv39 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55a7d04bed80 con 0x55a7cfba0800 2023-09-23T03:18:01.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.184+0000 7f3c4f455640 1 -- 172.21.15.154:0/3318852807 <== osd.5 v2:172.21.15.190:6800/882536885 2 ==== osd_op_reply(4 200.00000002 [delete] v43'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55a7d04beb40 con 0x55a7cfba0400 2023-09-23T03:18:01.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.184+0000 7f3c44c40640 1 -- 172.21.15.154:0/3318852807 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:9 5.14 5:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55a7cfbc0380 con 0x55a7cfba0800 2023-09-23T03:18:01.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.185+0000 7f3c4fc56640 1 -- 172.21.15.154:0/3318852807 <== osd.2 v2:172.21.15.154:6822/2884407243 1 ==== osd_op_reply(5 200.00000003 [delete] v43'6 uv5 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55a7d04bf8c0 con 0x55a7cfba0c00 2023-09-23T03:18:01.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.186+0000 7f3c4fc56640 1 -- 172.21.15.154:0/3318852807 <== osd.0 v2:172.21.15.154:6800/2920938138 1 ==== osd_op_reply(8 200.00000006 [delete] v43'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55a7d04bf8c0 con 0x55a7cfba1800 2023-09-23T03:18:01.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.188+0000 7f3c4f455640 1 -- 172.21.15.154:0/3318852807 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== osd_op_reply(6 200.00000004 [delete] v43'9 uv8 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55a7d04beb40 con 0x55a7cfba1000 2023-09-23T03:18:01.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.188+0000 7f3c4ec54640 1 -- 172.21.15.154:0/3318852807 <== osd.7 v2:172.21.15.190:6824/3460136272 1 ==== osd_op_reply(7 200.00000005 [delete] v43'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55a7d04bed80 con 0x55a7cfba1400 2023-09-23T03:18:01.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.189+0000 7f3c4ec54640 1 -- 172.21.15.154:0/3318852807 <== osd.3 v2:172.21.15.154:6808/1133688468 3 ==== osd_op_reply(9 200.00000001 [write 0~48] v43'41 uv41 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55a7d04bed80 con 0x55a7cfba0800 2023-09-23T03:18:01.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.189+0000 7f3c5144c100 4 Successfully wrote new journal pointer and header for rank 2:0 2023-09-23T03:18:01.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.189+0000 7f3c5144c100 1 -- 172.21.15.154:0/3318852807 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55a7cfba1800 msgr2=0x55a7cfba4100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.189+0000 7f3c5144c100 1 --2- 172.21.15.154:0/3318852807 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55a7cfba1800 0x55a7cfba4100 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.189+0000 7f3c5144c100 1 -- 172.21.15.154:0/3318852807 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55a7cfba0c00 msgr2=0x55a7cfba3080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.189+0000 7f3c5144c100 1 --2- 172.21.15.154:0/3318852807 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55a7cfba0c00 0x55a7cfba3080 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.189+0000 7f3c5144c100 1 -- 172.21.15.154:0/3318852807 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55a7cfba0800 msgr2=0x55a7cfba2b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.189+0000 7f3c5144c100 1 --2- 172.21.15.154:0/3318852807 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55a7cfba0800 0x55a7cfba2b00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.189+0000 7f3c5144c100 1 -- 172.21.15.154:0/3318852807 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55a7cfba0400 msgr2=0x55a7cfba2580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.189+0000 7f3c5144c100 1 --2- 172.21.15.154:0/3318852807 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55a7cfba0400 0x55a7cfba2580 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.189+0000 7f3c5144c100 1 -- 172.21.15.154:0/3318852807 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55a7cfba1000 msgr2=0x55a7cfba3600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.189+0000 7f3c5144c100 1 --2- 172.21.15.154:0/3318852807 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55a7cfba1000 0x55a7cfba3600 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.189+0000 7f3c4f455640 1 -- 172.21.15.154:0/3318852807 reap_dead start 2023-09-23T03:18:01.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.189+0000 7f3c5144c100 1 -- 172.21.15.154:0/3318852807 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55a7cfba1400 msgr2=0x55a7cfba3b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.189+0000 7f3c5144c100 1 --2- 172.21.15.154:0/3318852807 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55a7cfba1400 0x55a7cfba3b80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.189+0000 7f3c4f455640 1 -- 172.21.15.154:0/3318852807 reap_dead start 2023-09-23T03:18:01.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.189+0000 7f3c5144c100 1 -- 172.21.15.154:0/3318852807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a7cf9cd000 msgr2=0x55a7cf75cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.189+0000 7f3c5144c100 1 --2- 172.21.15.154:0/3318852807 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a7cf9cd000 0x55a7cf75cc00 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x55a7cfaf7110 tx=0x55a7cf9e6de0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.189+0000 7f3c4f455640 1 -- 172.21.15.154:0/3318852807 reap_dead start 2023-09-23T03:18:01.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.189+0000 7f3c5144c100 1 -- 172.21.15.154:0/3318852807 shutdown_connections 2023-09-23T03:18:01.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.189+0000 7f3c5144c100 1 -- 172.21.15.154:0/3318852807 >> 172.21.15.154:0/3318852807 conn(0x55a7cf9cd400 msgr2=0x55a7cfb8c000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:01.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.189+0000 7f3c5144c100 1 -- 172.21.15.154:0/3318852807 shutdown_connections 2023-09-23T03:18:01.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.189+0000 7f3c5144c100 1 -- 172.21.15.154:0/3318852807 wait complete. 2023-09-23T03:18:01.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.190+0000 7f3c5144c100 1 -- 172.21.15.154:0/1366134503 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55a7cfa45c00 msgr2=0x55a7cf75d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.190+0000 7f3c5144c100 1 --2- 172.21.15.154:0/1366134503 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55a7cfa45c00 0x55a7cf75d700 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x55a7cf7c6210 tx=0x55a7cf9e6d20 comp rx=0 tx=0).stop 2023-09-23T03:18:01.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.190+0000 7f3c5144c100 1 -- 172.21.15.154:0/1366134503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a7cf9cd800 msgr2=0x55a7cf75b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.190+0000 7f3c5144c100 1 --2- 172.21.15.154:0/1366134503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a7cf9cd800 0x55a7cf75b080 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x55a7cfa38120 tx=0x55a7d047b620 comp rx=0 tx=0).stop 2023-09-23T03:18:01.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.190+0000 7f3c5144c100 1 -- 172.21.15.154:0/1366134503 shutdown_connections 2023-09-23T03:18:01.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.190+0000 7f3c5144c100 1 --2- 172.21.15.154:0/1366134503 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55a7cfa45c00 0x55a7cf75d700 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.190+0000 7f3c5144c100 1 --2- 172.21.15.154:0/1366134503 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a7cf9cd800 0x55a7cf75b080 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.190+0000 7f3c5144c100 1 --2- 172.21.15.154:0/1366134503 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a7cf9cdc00 0x55a7cf75c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.190+0000 7f3c5144c100 1 --2- 172.21.15.154:0/1366134503 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7d04af400 0x55a7cf75c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.190+0000 7f3c5144c100 1 -- 172.21.15.154:0/1366134503 >> 172.21.15.154:0/1366134503 conn(0x55a7d04af800 msgr2=0x55a7d0463800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:01.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.190+0000 7f3c5144c100 1 -- 172.21.15.154:0/1366134503 shutdown_connections 2023-09-23T03:18:01.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.190+0000 7f3c5144c100 1 -- 172.21.15.154:0/1366134503 wait complete. 2023-09-23T03:18:01.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.190+0000 7f3c5144c100 1 -- 172.21.15.154:0/1808501719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a7d04ae800 msgr2=0x55a7cf75bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.190+0000 7f3c5144c100 1 --2- 172.21.15.154:0/1808501719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a7d04ae800 0x55a7cf75bb80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x55a7d0502cc0 tx=0x55a7cf9ca000 comp rx=0 tx=0).stop 2023-09-23T03:18:01.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.190+0000 7f3c5144c100 1 -- 172.21.15.154:0/1808501719 shutdown_connections 2023-09-23T03:18:01.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.190+0000 7f3c5144c100 1 --2- 172.21.15.154:0/1808501719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a7d04afc00 0x55a7cf75b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.190+0000 7f3c5144c100 1 --2- 172.21.15.154:0/1808501719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a7d04ae800 0x55a7cf75bb80 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.191+0000 7f3c5144c100 1 --2- 172.21.15.154:0/1808501719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a7cf6eb800 0x55a7cf75ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.191+0000 7f3c5144c100 1 -- 172.21.15.154:0/1808501719 >> 172.21.15.154:0/1808501719 conn(0x55a7cf6eb000 msgr2=0x55a7d0463000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:01.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.191+0000 7f3c5144c100 1 -- 172.21.15.154:0/1808501719 shutdown_connections 2023-09-23T03:18:01.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.191+0000 7f3c5144c100 1 -- 172.21.15.154:0/1808501719 wait complete. 2023-09-23T03:18:01.197 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.128848, result: writing EResetJournal entry 2023-09-23T03:18:01.197 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 init 2023-09-23T03:18:01.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.292+0000 7fab39a0d5c0 1 -- start start 2023-09-23T03:18:01.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.292+0000 7fab39a0d5c0 1 Processor -- start 2023-09-23T03:18:01.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.293+0000 7fab39a0d5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a530a98c00 0x55a530b02580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.293+0000 7fab39a0d5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a530a99800 0x55a530b02b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.293+0000 7fab39a0d5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a530a99c00 0x55a530b03080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.293+0000 7fab39a0d5c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a530ae3180 con 0x55a530a99800 2023-09-23T03:18:01.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.293+0000 7fab39a0d5c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a530ae2140 con 0x55a530a99c00 2023-09-23T03:18:01.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.293+0000 7fab39a0d5c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a530ae2000 con 0x55a530a98c00 2023-09-23T03:18:01.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.293+0000 7fab36db2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a530a99800 0x55a530b02b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.293+0000 7fab36db2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a530a99800 0x55a530b02b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60402/0 (socket says 172.21.15.154:60402) 2023-09-23T03:18:01.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.293+0000 7fab36db2640 1 -- 172.21.15.154:0/8380449 learned_addr learned my addr 172.21.15.154:0/8380449 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:01.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.293+0000 7fab375b3640 1 --2- 172.21.15.154:0/8380449 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a530a98c00 0x55a530b02580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.293+0000 7fab37db4640 1 --2- 172.21.15.154:0/8380449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a530a99c00 0x55a530b03080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.293+0000 7fab36db2640 1 -- 172.21.15.154:0/8380449 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a530a98c00 msgr2=0x55a530b02580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:01.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.293+0000 7fab36db2640 1 --2- 172.21.15.154:0/8380449 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a530a98c00 0x55a530b02580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.293+0000 7fab36db2640 1 -- 172.21.15.154:0/8380449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a530a99c00 msgr2=0x55a530b03080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:01.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.293+0000 7fab36db2640 1 --2- 172.21.15.154:0/8380449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a530a99c00 0x55a530b03080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.293+0000 7fab36db2640 1 -- 172.21.15.154:0/8380449 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a530accb60 con 0x55a530a99800 2023-09-23T03:18:01.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.294+0000 7fab36db2640 1 --2- 172.21.15.154:0/8380449 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a530a99800 0x55a530b02b00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x55a5318ad3e0 tx=0x55a531956360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1838f2f7a4cb7e30 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.294+0000 7fab34dae640 1 -- 172.21.15.154:0/8380449 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a530ae1600 con 0x55a530a99800 2023-09-23T03:18:01.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.294+0000 7fab34dae640 1 -- 172.21.15.154:0/8380449 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a530abf800 con 0x55a530a99800 2023-09-23T03:18:01.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.294+0000 7fab39a0d5c0 1 -- 172.21.15.154:0/8380449 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a530acc340 con 0x55a530a99800 2023-09-23T03:18:01.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.294+0000 7fab39a0d5c0 1 -- 172.21.15.154:0/8380449 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55a530acc000 con 0x55a530a99800 2023-09-23T03:18:01.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.294+0000 7fab39a0d5c0 4 waiting for MDS map... 2023-09-23T03:18:01.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.294+0000 7fab34dae640 1 -- 172.21.15.154:0/8380449 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a530ae11e0 con 0x55a530a99800 2023-09-23T03:18:01.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.294+0000 7fab34dae640 1 -- 172.21.15.154:0/8380449 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(43..43 src has 1..43) v4 ==== 6605+0+0 (secure 0 0 0) 0x55a530b58c40 con 0x55a530a99800 2023-09-23T03:18:01.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.295+0000 7fab34dae640 1 -- 172.21.15.154:0/8380449 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 25) v1 ==== 3306+0+0 (secure 0 0 0) 0x55a530aeb200 con 0x55a530a99800 2023-09-23T03:18:01.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.295+0000 7fab39a0d5c0 4 Got MDS map 25 2023-09-23T03:18:01.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.295+0000 7fab39a0d5c0 4 datascan.main: Using metadata pool output 2023-09-23T03:18:01.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.295+0000 7fab39a0d5c0 4 datascan.main: connecting to RADOS... 2023-09-23T03:18:01.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.295+0000 7fab39a0d5c0 1 Processor -- start 2023-09-23T03:18:01.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.296+0000 7fab39a0d5c0 1 -- start start 2023-09-23T03:18:01.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.296+0000 7fab39a0d5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a53195d000 0x55a530b03600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.296+0000 7fab39a0d5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a53195d400 0x55a530b03b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.296+0000 7fab39a0d5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a53195d800 0x55a530b04100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.296+0000 7fab39a0d5c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a530ae2a00 con 0x55a53195d800 2023-09-23T03:18:01.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.296+0000 7fab39a0d5c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a530b67180 con 0x55a53195d400 2023-09-23T03:18:01.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.296+0000 7fab39a0d5c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a530b66dc0 con 0x55a53195d000 2023-09-23T03:18:01.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.297+0000 7fab36db2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a53195d800 0x55a530b04100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.297+0000 7fab36db2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a53195d800 0x55a530b04100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60414/0 (socket says 172.21.15.154:60414) 2023-09-23T03:18:01.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.297+0000 7fab36db2640 1 -- 172.21.15.154:0/999439229 learned_addr learned my addr 172.21.15.154:0/999439229 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:01.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.297+0000 7fab37db4640 1 --2- 172.21.15.154:0/999439229 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a53195d000 0x55a530b03600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.297+0000 7fab36db2640 1 -- 172.21.15.154:0/999439229 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a53195d000 msgr2=0x55a530b03600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:01.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.297+0000 7fab36db2640 1 --2- 172.21.15.154:0/999439229 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a53195d000 0x55a530b03600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.297+0000 7fab36db2640 1 -- 172.21.15.154:0/999439229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a53195d400 msgr2=0x55a530b03b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:01.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.297+0000 7fab36db2640 1 --2- 172.21.15.154:0/999439229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a53195d400 0x55a530b03b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.297+0000 7fab36db2640 1 -- 172.21.15.154:0/999439229 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a530accea0 con 0x55a53195d800 2023-09-23T03:18:01.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.297+0000 7fab36db2640 1 --2- 172.21.15.154:0/999439229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a53195d800 0x55a530b04100 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x55a53196e6c0 tx=0x55a531829800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50863059f1896136 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.298+0000 7fab32daa640 1 -- 172.21.15.154:0/999439229 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a530ae0f20 con 0x55a53195d800 2023-09-23T03:18:01.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.298+0000 7fab32daa640 1 -- 172.21.15.154:0/999439229 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a531872780 con 0x55a53195d800 2023-09-23T03:18:01.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.298+0000 7fab32daa640 1 -- 172.21.15.154:0/999439229 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a530ae1600 con 0x55a53195d800 2023-09-23T03:18:01.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.298+0000 7fab39a0d5c0 1 -- 172.21.15.154:0/999439229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a53195d800 msgr2=0x55a530b04100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.298+0000 7fab39a0d5c0 1 --2- 172.21.15.154:0/999439229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a53195d800 0x55a530b04100 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x55a53196e6c0 tx=0x55a531829800 comp rx=0 tx=0).stop 2023-09-23T03:18:01.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.298+0000 7fab39a0d5c0 1 -- 172.21.15.154:0/999439229 shutdown_connections 2023-09-23T03:18:01.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.298+0000 7fab39a0d5c0 1 --2- 172.21.15.154:0/999439229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a53195d800 0x55a530b04100 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.298+0000 7fab39a0d5c0 1 --2- 172.21.15.154:0/999439229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a53195d400 0x55a530b03b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.298+0000 7fab39a0d5c0 1 --2- 172.21.15.154:0/999439229 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a53195d000 0x55a530b03600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.298+0000 7fab39a0d5c0 1 -- 172.21.15.154:0/999439229 >> 172.21.15.154:0/999439229 conn(0x55a53195cc00 msgr2=0x55a530b93000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:01.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.298+0000 7fab39a0d5c0 1 -- 172.21.15.154:0/999439229 shutdown_connections 2023-09-23T03:18:01.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.298+0000 7fab39a0d5c0 1 -- 172.21.15.154:0/999439229 wait complete. 2023-09-23T03:18:01.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.299+0000 7fab39a0d5c0 1 Processor -- start 2023-09-23T03:18:01.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.299+0000 7fab39a0d5c0 1 -- start start 2023-09-23T03:18:01.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.300+0000 7fab39a0d5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a53195d000 0x55a530b05180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.300+0000 7fab39a0d5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a53195d400 0x55a530b04c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.300+0000 7fab39a0d5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a53195d800 0x55a530b04680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.300+0000 7fab39a0d5c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a530b66c80 con 0x55a53195d400 2023-09-23T03:18:01.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.300+0000 7fab39a0d5c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a530ae2140 con 0x55a53195d000 2023-09-23T03:18:01.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.300+0000 7fab39a0d5c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a530ae2b40 con 0x55a53195d800 2023-09-23T03:18:01.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.300+0000 7fab375b3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a53195d400 0x55a530b04c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.300+0000 7fab375b3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a53195d400 0x55a530b04c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60422/0 (socket says 172.21.15.154:60422) 2023-09-23T03:18:01.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.300+0000 7fab375b3640 1 -- 172.21.15.154:0/138353970 learned_addr learned my addr 172.21.15.154:0/138353970 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:01.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.301+0000 7fab375b3640 1 -- 172.21.15.154:0/138353970 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a53195d800 msgr2=0x55a530b04680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.301+0000 7fab36db2640 1 --2- 172.21.15.154:0/138353970 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a53195d800 0x55a530b04680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.301+0000 7fab37db4640 1 --2- 172.21.15.154:0/138353970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a53195d000 0x55a530b05180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.301+0000 7fab375b3640 1 --2- 172.21.15.154:0/138353970 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a53195d800 0x55a530b04680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.301+0000 7fab375b3640 1 -- 172.21.15.154:0/138353970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a53195d000 msgr2=0x55a530b05180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.301+0000 7fab375b3640 1 --2- 172.21.15.154:0/138353970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a53195d000 0x55a530b05180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.301+0000 7fab375b3640 1 -- 172.21.15.154:0/138353970 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a530accb60 con 0x55a53195d400 2023-09-23T03:18:01.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.301+0000 7fab375b3640 1 --2- 172.21.15.154:0/138353970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a53195d400 0x55a530b04c00 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x55a53196f3b0 tx=0x55a531972f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.301+0000 7fab315a7640 1 -- 172.21.15.154:0/138353970 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a530ae0f20 con 0x55a53195d400 2023-09-23T03:18:01.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.301+0000 7fab315a7640 1 -- 172.21.15.154:0/138353970 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a531873800 con 0x55a53195d400 2023-09-23T03:18:01.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.301+0000 7fab315a7640 1 -- 172.21.15.154:0/138353970 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a530ae1600 con 0x55a53195d400 2023-09-23T03:18:01.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.301+0000 7fab39a0d5c0 1 -- 172.21.15.154:0/138353970 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a530acc340 con 0x55a53195d400 2023-09-23T03:18:01.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.302+0000 7fab39a0d5c0 1 -- 172.21.15.154:0/138353970 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a530acd520 con 0x55a53195d400 2023-09-23T03:18:01.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.302+0000 7fab39a0d5c0 4 datascan.init: resolving metadata pool 5 2023-09-23T03:18:01.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.303+0000 7fab315a7640 1 -- 172.21.15.154:0/138353970 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x55a530aca840 con 0x55a53195d400 2023-09-23T03:18:01.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.304+0000 7fab315a7640 1 --2- 172.21.15.154:0/138353970 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55a5319d2800 0x55a530b05700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.304+0000 7fab315a7640 1 -- 172.21.15.154:0/138353970 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6605+0+0 (secure 0 0 0) 0x55a530b58a80 con 0x55a53195d400 2023-09-23T03:18:01.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.304+0000 7fab39a0d5c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-09-23T03:18:01.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.304+0000 7fab39a0d5c0 1 --2- 172.21.15.154:0/138353970 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55a5319d2c00 0x55a530b04100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.304+0000 7fab39a0d5c0 1 -- 172.21.15.154:0/138353970 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:1 5.14 5:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55a530af3880 con 0x55a5319d2c00 2023-09-23T03:18:01.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.304+0000 7fab37db4640 1 --2- 172.21.15.154:0/138353970 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55a5319d2800 0x55a530b05700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.304+0000 7fab36db2640 1 --2- 172.21.15.154:0/138353970 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55a5319d2c00 0x55a530b04100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.305+0000 7fab37db4640 1 --2- 172.21.15.154:0/138353970 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55a5319d2800 0x55a530b05700 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x55a530b27e30 tx=0x55a531956180 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.305+0000 7fab36db2640 1 --2- 172.21.15.154:0/138353970 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55a5319d2c00 0x55a530b04100 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.305+0000 7fab36db2640 1 -- 172.21.15.154:0/138353970 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(1 1.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 160+0+0 (crc 0 0 0) 0x55a531946b40 con 0x55a5319d2c00 2023-09-23T03:18:01.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.305+0000 7fab39a0d5c0 1 -- 172.21.15.154:0/138353970 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:2 5.14 5:287034c4:::1.00000000.inode:head [writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55a530af3c00 con 0x55a5319d2c00 2023-09-23T03:18:01.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.309+0000 7fab36db2640 1 -- 172.21.15.154:0/138353970 <== osd.3 v2:172.21.15.154:6808/1133688468 2 ==== osd_op_reply(2 1.00000000.inode [writefull 0~582] v43'42 uv42 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x55a531946b40 con 0x55a5319d2c00 2023-09-23T03:18:01.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.309+0000 7fab39a0d5c0 1 --2- 172.21.15.154:0/138353970 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55a5319d3000 0x55a530b03b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.309+0000 7fab39a0d5c0 1 -- 172.21.15.154:0/138353970 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:3 5.1f 5:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55a530e04000 con 0x55a5319d3000 2023-09-23T03:18:01.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.309+0000 7fab37db4640 1 --2- 172.21.15.154:0/138353970 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55a5319d3000 0x55a530b03b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.310+0000 7fab37db4640 1 --2- 172.21.15.154:0/138353970 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55a5319d3000 0x55a530b03b80 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.311+0000 7fab37db4640 1 -- 172.21.15.154:0/138353970 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== osd_op_reply(3 100.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 162+0+0 (crc 0 0 0) 0x55a531947200 con 0x55a5319d3000 2023-09-23T03:18:01.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.311+0000 7fab39a0d5c0 1 -- 172.21.15.154:0/138353970 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:4 5.1f 5:fe07bba1:::100.00000000.inode:head [writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55a530e04380 con 0x55a5319d3000 2023-09-23T03:18:01.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.315+0000 7fab37db4640 1 -- 172.21.15.154:0/138353970 <== osd.6 v2:172.21.15.190:6816/1964294001 2 ==== osd_op_reply(4 100.00000000.inode [writefull 0~582] v43'10 uv10 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x55a531947200 con 0x55a5319d3000 2023-09-23T03:18:01.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.316+0000 7fab39a0d5c0 1 -- 172.21.15.154:0/138353970 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:5 5.13 5:cd5a64a3:::100.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55a530e04700 con 0x55a5319d2c00 2023-09-23T03:18:01.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.316+0000 7fab36db2640 1 -- 172.21.15.154:0/138353970 <== osd.3 v2:172.21.15.154:6808/1133688468 3 ==== osd_op_reply(5 100.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55a531946b40 con 0x55a5319d2c00 2023-09-23T03:18:01.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.316+0000 7fab39a0d5c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-09-23T03:18:01.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.316+0000 7fab39a0d5c0 1 -- 172.21.15.154:0/138353970 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:6 5.13 5:cd5a64a3:::100.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55a530e04a80 con 0x55a5319d2c00 2023-09-23T03:18:01.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.318+0000 7fab36db2640 1 -- 172.21.15.154:0/138353970 <== osd.3 v2:172.21.15.154:6808/1133688468 4 ==== osd_op_reply(6 100.00000000 [create,omap-set-header] v43'7 uv7 ondisk = 0) v8 ==== 198+0+0 (crc 0 0 0) 0x55a531946b40 con 0x55a5319d2c00 2023-09-23T03:18:01.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.318+0000 7fab39a0d5c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x100 2023-09-23T03:18:01.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.318+0000 7fab39a0d5c0 1 -- 172.21.15.154:0/138353970 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55a5319d2c00 msgr2=0x55a530b04100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.318+0000 7fab39a0d5c0 1 --2- 172.21.15.154:0/138353970 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55a5319d2c00 0x55a530b04100 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.318+0000 7fab39a0d5c0 1 -- 172.21.15.154:0/138353970 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55a5319d3000 msgr2=0x55a530b03b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.318+0000 7fab39a0d5c0 1 --2- 172.21.15.154:0/138353970 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55a5319d3000 0x55a530b03b80 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.318+0000 7fab39a0d5c0 1 -- 172.21.15.154:0/138353970 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55a5319d2800 msgr2=0x55a530b05700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.318+0000 7fab39a0d5c0 1 --2- 172.21.15.154:0/138353970 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55a5319d2800 0x55a530b05700 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x55a530b27e30 tx=0x55a531956180 comp rx=0 tx=0).stop 2023-09-23T03:18:01.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.318+0000 7fab39a0d5c0 1 -- 172.21.15.154:0/138353970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a53195d400 msgr2=0x55a530b04c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.318+0000 7fab39a0d5c0 1 --2- 172.21.15.154:0/138353970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a53195d400 0x55a530b04c00 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x55a53196f3b0 tx=0x55a531972f00 comp rx=0 tx=0).stop 2023-09-23T03:18:01.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.318+0000 7fab375b3640 1 -- 172.21.15.154:0/138353970 reap_dead start 2023-09-23T03:18:01.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.318+0000 7fab39a0d5c0 1 -- 172.21.15.154:0/138353970 shutdown_connections 2023-09-23T03:18:01.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.318+0000 7fab39a0d5c0 1 --2- 172.21.15.154:0/138353970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a53195d400 0x55a530b04c00 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.318+0000 7fab39a0d5c0 1 -- 172.21.15.154:0/138353970 >> 172.21.15.154:0/138353970 conn(0x55a53195cc00 msgr2=0x55a530b93000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:01.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.318+0000 7fab39a0d5c0 1 -- 172.21.15.154:0/138353970 shutdown_connections 2023-09-23T03:18:01.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.318+0000 7fab39a0d5c0 1 -- 172.21.15.154:0/138353970 wait complete. 2023-09-23T03:18:01.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.319+0000 7fab39a0d5c0 1 -- 172.21.15.154:0/8380449 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a530a99800 msgr2=0x55a530b02b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.319+0000 7fab39a0d5c0 1 --2- 172.21.15.154:0/8380449 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a530a99800 0x55a530b02b00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x55a5318ad3e0 tx=0x55a531956360 comp rx=0 tx=0).stop 2023-09-23T03:18:01.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.319+0000 7fab39a0d5c0 1 -- 172.21.15.154:0/8380449 shutdown_connections 2023-09-23T03:18:01.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.319+0000 7fab39a0d5c0 1 --2- 172.21.15.154:0/8380449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a530a99c00 0x55a530b03080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.319+0000 7fab39a0d5c0 1 --2- 172.21.15.154:0/8380449 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a530a99800 0x55a530b02b00 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.319+0000 7fab39a0d5c0 1 --2- 172.21.15.154:0/8380449 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a530a98c00 0x55a530b02580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.319+0000 7fab39a0d5c0 1 -- 172.21.15.154:0/8380449 >> 172.21.15.154:0/8380449 conn(0x55a530a99000 msgr2=0x55a530b92800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:01.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.319+0000 7fab39a0d5c0 1 -- 172.21.15.154:0/8380449 shutdown_connections 2023-09-23T03:18:01.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.319+0000 7fab39a0d5c0 1 -- 172.21.15.154:0/8380449 wait complete. 2023-09-23T03:18:01.324 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'init'] in time 0:00:00.127496, result: 2023-09-23T03:18:01.325 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents 2023-09-23T03:18:01.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.418+0000 7fb6e22105c0 1 -- start start 2023-09-23T03:18:01.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.418+0000 7fb6e22105c0 1 Processor -- start 2023-09-23T03:18:01.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.419+0000 7fb6e22105c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c58d9bc000 0x55c58cc49080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.419+0000 7fb6e22105c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c58cbdfc00 0x55c58cc49b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.419+0000 7fb6e22105c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c58d9bc800 0x55c58cc49600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.419+0000 7fb6e22105c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c58cc28b40 con 0x55c58cbdfc00 2023-09-23T03:18:01.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.419+0000 7fb6e22105c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c58cc28a00 con 0x55c58d9bc000 2023-09-23T03:18:01.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.419+0000 7fb6e22105c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c58cc28140 con 0x55c58d9bc800 2023-09-23T03:18:01.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.419+0000 7fb6df6df640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c58cbdfc00 0x55c58cc49b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.419+0000 7fb6df6df640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c58cbdfc00 0x55c58cc49b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60454/0 (socket says 172.21.15.154:60454) 2023-09-23T03:18:01.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.419+0000 7fb6df6df640 1 -- 172.21.15.154:0/698261334 learned_addr learned my addr 172.21.15.154:0/698261334 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:01.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.419+0000 7fb6df6df640 1 -- 172.21.15.154:0/698261334 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c58d9bc800 msgr2=0x55c58cc49600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:01.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.419+0000 7fb6df6df640 1 --2- 172.21.15.154:0/698261334 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c58d9bc800 0x55c58cc49600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.419+0000 7fb6df6df640 1 -- 172.21.15.154:0/698261334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c58d9bc000 msgr2=0x55c58cc49080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:01.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.419+0000 7fb6df6df640 1 --2- 172.21.15.154:0/698261334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c58d9bc000 0x55c58cc49080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.419+0000 7fb6df6df640 1 -- 172.21.15.154:0/698261334 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c58cc12ea0 con 0x55c58cbdfc00 2023-09-23T03:18:01.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.420+0000 7fb6df6df640 1 --2- 172.21.15.154:0/698261334 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c58cbdfc00 0x55c58cc49b80 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x55c58d9f5620 tx=0x55c58d9ba180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf8cc769c8f7080 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.420+0000 7fb6dd6db640 1 -- 172.21.15.154:0/698261334 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c58cc27600 con 0x55c58cbdfc00 2023-09-23T03:18:01.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.420+0000 7fb6dd6db640 1 -- 172.21.15.154:0/698261334 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c58cc05800 con 0x55c58cbdfc00 2023-09-23T03:18:01.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.420+0000 7fb6dd6db640 1 -- 172.21.15.154:0/698261334 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c58cc27080 con 0x55c58cbdfc00 2023-09-23T03:18:01.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.420+0000 7fb6e22105c0 1 -- 172.21.15.154:0/698261334 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c58cc13520 con 0x55c58cbdfc00 2023-09-23T03:18:01.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.420+0000 7fb6e22105c0 1 -- 172.21.15.154:0/698261334 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55c58cc136c0 con 0x55c58cbdfc00 2023-09-23T03:18:01.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.420+0000 7fb6e22105c0 4 waiting for MDS map... 2023-09-23T03:18:01.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.420+0000 7fb6dd6db640 1 -- 172.21.15.154:0/698261334 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(43..43 src has 1..43) v4 ==== 6605+0+0 (secure 0 0 0) 0x55c58cc9ec40 con 0x55c58cbdfc00 2023-09-23T03:18:01.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.421+0000 7fb6dd6db640 1 -- 172.21.15.154:0/698261334 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 25) v1 ==== 3306+0+0 (secure 0 0 0) 0x55c58cc31200 con 0x55c58cbdfc00 2023-09-23T03:18:01.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.421+0000 7fb6e22105c0 4 Got MDS map 25 2023-09-23T03:18:01.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.421+0000 7fb6e22105c0 4 datascan.main: Using metadata pool output 2023-09-23T03:18:01.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.421+0000 7fb6e22105c0 4 datascan.main: connecting to RADOS... 2023-09-23T03:18:01.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.421+0000 7fb6e22105c0 1 Processor -- start 2023-09-23T03:18:01.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.422+0000 7fb6e22105c0 1 -- start start 2023-09-23T03:18:01.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.422+0000 7fb6e22105c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c58daa2c00 0x55c58cc48b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.422+0000 7fb6e22105c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c58daa3000 0x55c58cc4a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.422+0000 7fb6e22105c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c58daa3400 0x55c58cc4a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.422+0000 7fb6e22105c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c58cc28000 con 0x55c58daa3400 2023-09-23T03:18:01.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.422+0000 7fb6e22105c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c58ccad180 con 0x55c58daa3000 2023-09-23T03:18:01.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.422+0000 7fb6e22105c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c58ccacdc0 con 0x55c58daa2c00 2023-09-23T03:18:01.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.423+0000 7fb6df6df640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c58daa3400 0x55c58cc4a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.423+0000 7fb6df6df640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c58daa3400 0x55c58cc4a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60456/0 (socket says 172.21.15.154:60456) 2023-09-23T03:18:01.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.423+0000 7fb6df6df640 1 -- 172.21.15.154:0/2022382349 learned_addr learned my addr 172.21.15.154:0/2022382349 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:01.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.423+0000 7fb6e06e1640 1 --2- 172.21.15.154:0/2022382349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c58daa2c00 0x55c58cc48b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.423+0000 7fb6dfee0640 1 --2- 172.21.15.154:0/2022382349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c58daa3000 0x55c58cc4a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.423+0000 7fb6df6df640 1 -- 172.21.15.154:0/2022382349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c58daa2c00 msgr2=0x55c58cc48b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:01.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.423+0000 7fb6df6df640 1 --2- 172.21.15.154:0/2022382349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c58daa2c00 0x55c58cc48b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.423+0000 7fb6df6df640 1 -- 172.21.15.154:0/2022382349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c58daa3000 msgr2=0x55c58cc4a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:01.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.423+0000 7fb6df6df640 1 --2- 172.21.15.154:0/2022382349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c58daa3000 0x55c58cc4a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.423+0000 7fb6df6df640 1 -- 172.21.15.154:0/2022382349 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c58cc13860 con 0x55c58daa3400 2023-09-23T03:18:01.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.423+0000 7fb6df6df640 1 --2- 172.21.15.154:0/2022382349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c58daa3400 0x55c58cc4a680 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x55c58dab0b10 tx=0x55c58cca2cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b5b7f877ab6fa72 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.424+0000 7fb6db6d7640 1 -- 172.21.15.154:0/2022382349 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c58cc26f20 con 0x55c58daa3400 2023-09-23T03:18:01.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.424+0000 7fb6db6d7640 1 -- 172.21.15.154:0/2022382349 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c58daaa180 con 0x55c58daa3400 2023-09-23T03:18:01.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.424+0000 7fb6db6d7640 1 -- 172.21.15.154:0/2022382349 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c58cc27600 con 0x55c58daa3400 2023-09-23T03:18:01.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.424+0000 7fb6e22105c0 1 -- 172.21.15.154:0/2022382349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c58daa3400 msgr2=0x55c58cc4a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.424+0000 7fb6e22105c0 1 --2- 172.21.15.154:0/2022382349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c58daa3400 0x55c58cc4a680 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x55c58dab0b10 tx=0x55c58cca2cc0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.424+0000 7fb6e22105c0 1 -- 172.21.15.154:0/2022382349 shutdown_connections 2023-09-23T03:18:01.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.424+0000 7fb6e22105c0 1 --2- 172.21.15.154:0/2022382349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c58daa3400 0x55c58cc4a680 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.424+0000 7fb6e22105c0 1 --2- 172.21.15.154:0/2022382349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c58daa3000 0x55c58cc4a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.424+0000 7fb6e22105c0 1 --2- 172.21.15.154:0/2022382349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c58daa2c00 0x55c58cc48b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.424+0000 7fb6e22105c0 1 -- 172.21.15.154:0/2022382349 >> 172.21.15.154:0/2022382349 conn(0x55c58d9bc400 msgr2=0x55c58d959800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:01.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.424+0000 7fb6e22105c0 1 -- 172.21.15.154:0/2022382349 shutdown_connections 2023-09-23T03:18:01.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.424+0000 7fb6e22105c0 1 -- 172.21.15.154:0/2022382349 wait complete. 2023-09-23T03:18:01.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.425+0000 7fb6e22105c0 1 Processor -- start 2023-09-23T03:18:01.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.425+0000 7fb6e22105c0 1 -- start start 2023-09-23T03:18:01.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.426+0000 7fb6e22105c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c58daa2c00 0x55c58cc48b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.426+0000 7fb6e22105c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c58daa3000 0x55c58cc4a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.426+0000 7fb6e22105c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c58daa3400 0x55c58cc4a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.426+0000 7fb6e22105c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c58ccacc80 con 0x55c58daa3000 2023-09-23T03:18:01.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.426+0000 7fb6e22105c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c58cc28a00 con 0x55c58daa2c00 2023-09-23T03:18:01.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.426+0000 7fb6e22105c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c58cc29180 con 0x55c58daa3400 2023-09-23T03:18:01.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.426+0000 7fb6dfee0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c58daa3000 0x55c58cc4a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.426+0000 7fb6dfee0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c58daa3000 0x55c58cc4a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60464/0 (socket says 172.21.15.154:60464) 2023-09-23T03:18:01.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.426+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 learned_addr learned my addr 172.21.15.154:0/275856908 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:01.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.426+0000 7fb6e06e1640 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c58daa2c00 0x55c58cc48b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.426+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c58daa3400 msgr2=0x55c58cc4a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.426+0000 7fb6dfee0640 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c58daa3400 0x55c58cc4a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.427+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c58daa2c00 msgr2=0x55c58cc48b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.427+0000 7fb6dfee0640 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c58daa2c00 0x55c58cc48b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.427+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c58cc12ea0 con 0x55c58daa3000 2023-09-23T03:18:01.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.427+0000 7fb6e06e1640 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c58daa2c00 0x55c58cc48b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:18:01.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.427+0000 7fb6dfee0640 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c58daa3000 0x55c58cc4a100 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x55c58d9f4e70 tx=0x55c58dabcd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.427+0000 7fb6d9ed4640 1 -- 172.21.15.154:0/275856908 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c58cc271e0 con 0x55c58daa3000 2023-09-23T03:18:01.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.427+0000 7fb6d9ed4640 1 -- 172.21.15.154:0/275856908 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c58daab680 con 0x55c58daa3000 2023-09-23T03:18:01.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.427+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c58cc13520 con 0x55c58daa3000 2023-09-23T03:18:01.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.427+0000 7fb6d9ed4640 1 -- 172.21.15.154:0/275856908 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c58cc26f20 con 0x55c58daa3000 2023-09-23T03:18:01.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.427+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c58cc13a00 con 0x55c58daa3000 2023-09-23T03:18:01.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.427+0000 7fb6e22105c0 4 datascan.init: resolving metadata pool 5 2023-09-23T03:18:01.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.429+0000 7fb6d9ed4640 1 -- 172.21.15.154:0/275856908 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x55c58cc10840 con 0x55c58daa3000 2023-09-23T03:18:01.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.429+0000 7fb6d9ed4640 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55c58db16c00 0x55c58cc4b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.429+0000 7fb6d9ed4640 1 -- 172.21.15.154:0/275856908 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6605+0+0 (secure 0 0 0) 0x55c58cc9ea80 con 0x55c58daa3000 2023-09-23T03:18:01.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.430+0000 7fb6e22105c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-09-23T03:18:01.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.430+0000 7fb6e22105c0 4 datascan.main: data pool 'cephfs_data' has ID 6 2023-09-23T03:18:01.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.430+0000 7fb6e22105c0 4 datascan.main: opening data pool 'cephfs_data' 2023-09-23T03:18:01.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.430+0000 7fb6e22105c0 4 datascan.main: data pool 'cephfs_data_extra' has ID 7 2023-09-23T03:18:01.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.430+0000 7fb6e22105c0 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-09-23T03:18:01.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.430+0000 7fb6e22105c0 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55c58daa3800 0x55c58cc4ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.430+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:1 6.0 6:00000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cc39880 con 0x55c58daa3800 2023-09-23T03:18:01.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.430+0000 7fb6e06e1640 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55c58db16c00 0x55c58cc4b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.430+0000 7fb6df6df640 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55c58daa3800 0x55c58cc4ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.430+0000 7fb6e06e1640 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55c58db16c00 0x55c58cc4b700 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55c58d95a420 tx=0x55c58cf1e060 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.430+0000 7fb6df6df640 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55c58daa3800 0x55c58cc4ac00 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.431+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.0 v2:172.21.15.154:6800/2920938138 1 ==== osd_op_reply(1 [pgnls start_epoch 43 out=49b] v41'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94b40 con 0x55c58daa3800 2023-09-23T03:18:01.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.431+0000 7fb6daed6640 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55c58db17000 0x55c58cf4a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.431+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:2 6.20 6:04000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf54000 con 0x55c58db17000 2023-09-23T03:18:01.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.431+0000 7fb6e06e1640 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55c58db17000 0x55c58cf4a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.432+0000 7fb6e06e1640 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55c58db17000 0x55c58cf4a000 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.432+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.1 v2:172.21.15.154:6816/1267710005 1 ==== osd_op_reply(2 [pgnls start_epoch 43 out=49b] v41'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94d80 con 0x55c58db17000 2023-09-23T03:18:01.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.432+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:3 6.10 6:08000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf54380 con 0x55c58daa3800 2023-09-23T03:18:01.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.432+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.0 v2:172.21.15.154:6800/2920938138 2 ==== osd_op_reply(3 [pgnls start_epoch 43 out=49b] v41'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94b40 con 0x55c58daa3800 2023-09-23T03:18:01.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.432+0000 7fb6daed6640 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55c58db17400 0x55c58cf4a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.432+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:4 6.30 6:0c000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf54700 con 0x55c58db17400 2023-09-23T03:18:01.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.433+0000 7fb6dfee0640 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55c58db17400 0x55c58cf4a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.434+0000 7fb6dfee0640 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55c58db17400 0x55c58cf4a580 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.435+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.7 v2:172.21.15.190:6824/3460136272 1 ==== osd_op_reply(4 [pgnls start_epoch 43 out=49b] v41'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58db17400 2023-09-23T03:18:01.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.435+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:5 6.8 6:10000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf54e00 con 0x55c58db17400 2023-09-23T03:18:01.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.436+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.7 v2:172.21.15.190:6824/3460136272 2 ==== osd_op_reply(5 [pgnls start_epoch 43 out=49b] v41'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58db17400 2023-09-23T03:18:01.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.436+0000 7fb6daed6640 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55c58db17800 0x55c58cf4ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.436+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:6 6.28 6:14000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf54a80 con 0x55c58db17800 2023-09-23T03:18:01.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.437+0000 7fb6df6df640 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55c58db17800 0x55c58cf4ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.437+0000 7fb6df6df640 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55c58db17800 0x55c58cf4ab00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.437+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.2 v2:172.21.15.154:6822/2884407243 1 ==== osd_op_reply(6 [pgnls start_epoch 43 out=49b] v41'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94b40 con 0x55c58db17800 2023-09-23T03:18:01.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.437+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:7 6.18 6:18000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf55180 con 0x55c58daa3800 2023-09-23T03:18:01.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.438+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.0 v2:172.21.15.154:6800/2920938138 3 ==== osd_op_reply(7 [pgnls start_epoch 43 out=49b] v41'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94b40 con 0x55c58daa3800 2023-09-23T03:18:01.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.438+0000 7fb6daed6640 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55c58db17c00 0x55c58cf4b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.438+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:8 6.38 6:1c000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf54380 con 0x55c58db17c00 2023-09-23T03:18:01.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.438+0000 7fb6e06e1640 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55c58db17c00 0x55c58cf4b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.439+0000 7fb6e06e1640 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55c58db17c00 0x55c58cf4b080 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.439+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(8 [pgnls start_epoch 43 out=49b] v41'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94d80 con 0x55c58db17c00 2023-09-23T03:18:01.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.439+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:9 6.4 6:20000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf54000 con 0x55c58db17000 2023-09-23T03:18:01.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.439+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.1 v2:172.21.15.154:6816/1267710005 2 ==== osd_op_reply(9 [pgnls start_epoch 43 out=49b] v41'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94d80 con 0x55c58db17000 2023-09-23T03:18:01.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.439+0000 7fb6daed6640 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55c58cf70000 0x55c58cf4b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.439+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:10 6.24 6:24000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf54a80 con 0x55c58cf70000 2023-09-23T03:18:01.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.440+0000 7fb6dfee0640 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55c58cf70000 0x55c58cf4b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.441+0000 7fb6dfee0640 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55c58cf70000 0x55c58cf4b600 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.441+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.5 v2:172.21.15.190:6800/882536885 1 ==== osd_op_reply(10 [pgnls start_epoch 43 out=49b] v41'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58cf70000 2023-09-23T03:18:01.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.441+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:11 6.14 6:28000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf54380 con 0x55c58db17800 2023-09-23T03:18:01.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.442+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.2 v2:172.21.15.154:6822/2884407243 2 ==== osd_op_reply(11 [pgnls start_epoch 43 out=49b] v41'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94b40 con 0x55c58db17800 2023-09-23T03:18:01.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.442+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:12 6.34 6:2c000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cc39880 con 0x55c58daa3800 2023-09-23T03:18:01.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.442+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.0 v2:172.21.15.154:6800/2920938138 4 ==== osd_op_reply(12 [pgnls start_epoch 43 out=49b] v41'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94b40 con 0x55c58daa3800 2023-09-23T03:18:01.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.442+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:13 6.c 6:30000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf54700 con 0x55c58db17c00 2023-09-23T03:18:01.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.443+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.3 v2:172.21.15.154:6808/1133688468 2 ==== osd_op_reply(13 [pgnls start_epoch 43 out=49b] v41'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94d80 con 0x55c58db17c00 2023-09-23T03:18:01.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.443+0000 7fb6daed6640 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55c58cf70400 0x55c58cf4bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.443+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:14 6.2c 6:34000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf54e00 con 0x55c58cf70400 2023-09-23T03:18:01.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.443+0000 7fb6df6df640 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55c58cf70400 0x55c58cf4bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.444+0000 7fb6df6df640 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55c58cf70400 0x55c58cf4bb80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.445+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== osd_op_reply(14 [pgnls start_epoch 43 out=49b] v41'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94b40 con 0x55c58cf70400 2023-09-23T03:18:01.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.445+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:15 6.1c 6:38000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf55180 con 0x55c58db17400 2023-09-23T03:18:01.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.446+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.7 v2:172.21.15.190:6824/3460136272 3 ==== osd_op_reply(15 [pgnls start_epoch 43 out=49b] v41'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58db17400 2023-09-23T03:18:01.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.446+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:16 6.3c 6:3c000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf55500 con 0x55c58db17800 2023-09-23T03:18:01.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.446+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.2 v2:172.21.15.154:6822/2884407243 3 ==== osd_op_reply(16 [pgnls start_epoch 43 out=49b] v41'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94b40 con 0x55c58db17800 2023-09-23T03:18:01.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.447+0000 7fb6db6d7640 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55c58cf71000 0x55c58cf4c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.447+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:17 6.2 6:40000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf54380 con 0x55c58cf71000 2023-09-23T03:18:01.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.447+0000 7fb6e06e1640 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55c58cf71000 0x55c58cf4c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.448+0000 7fb6e06e1640 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55c58cf71000 0x55c58cf4c100 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.449+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.4 v2:172.21.15.190:6808/717302641 1 ==== osd_op_reply(17 [pgnls start_epoch 43 out=49b] v41'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94d80 con 0x55c58cf71000 2023-09-23T03:18:01.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.449+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:18 6.22 6:44000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf55880 con 0x55c58cf70400 2023-09-23T03:18:01.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.450+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.6 v2:172.21.15.190:6816/1964294001 2 ==== osd_op_reply(18 [pgnls start_epoch 43 out=49b] v41'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94b40 con 0x55c58cf70400 2023-09-23T03:18:01.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.450+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:19 6.12 6:48000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf54e00 con 0x55c58db17400 2023-09-23T03:18:01.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.451+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.7 v2:172.21.15.190:6824/3460136272 4 ==== osd_op_reply(19 [pgnls start_epoch 43 out=49b] v41'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58db17400 2023-09-23T03:18:01.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.451+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:20 6.32 6:4c000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf55c00 con 0x55c58cf71000 2023-09-23T03:18:01.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.452+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.4 v2:172.21.15.190:6808/717302641 2 ==== osd_op_reply(20 [pgnls start_epoch 43 out=49b] v41'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94d80 con 0x55c58cf71000 2023-09-23T03:18:01.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.452+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:21 6.a 6:50000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cc39880 con 0x55c58cf70000 2023-09-23T03:18:01.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.453+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.5 v2:172.21.15.190:6800/882536885 2 ==== osd_op_reply(21 [pgnls start_epoch 43 out=49b] v41'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58cf70000 2023-09-23T03:18:01.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.453+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:22 6.2a 6:54000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf54a80 con 0x55c58cf71000 2023-09-23T03:18:01.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.455+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.4 v2:172.21.15.190:6808/717302641 3 ==== osd_op_reply(22 [pgnls start_epoch 43 out=81b] v41'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c58da94d80 con 0x55c58cf71000 2023-09-23T03:18:01.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.455+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:23 6.2a 6:57fe7535:::100000001f6.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cc39c00 con 0x55c58cf71000 2023-09-23T03:18:01.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.456+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.4 v2:172.21.15.190:6808/717302641 4 ==== osd_op_reply(23 100000001f6.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c58da94d80 con 0x55c58cf71000 2023-09-23T03:18:01.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.456+0000 7fb6e22105c0 10 datascan.operator(): handling object 1099511628278.0 2023-09-23T03:18:01.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.456+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:24 6.2a 6:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf54380 con 0x55c58cf71000 2023-09-23T03:18:01.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.460+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.4 v2:172.21.15.190:6808/717302641 5 ==== osd_op_reply(24 100000001f6.00000000 [call] v43'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c58da94d80 con 0x55c58cf71000 2023-09-23T03:18:01.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.460+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:25 6.1a 6:58000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf55c00 con 0x55c58cf71000 2023-09-23T03:18:01.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.461+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.4 v2:172.21.15.190:6808/717302641 6 ==== osd_op_reply(25 [pgnls start_epoch 43 out=49b] v41'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94d80 con 0x55c58cf71000 2023-09-23T03:18:01.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.461+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:26 6.3a 6:5c000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf55180 con 0x55c58db17000 2023-09-23T03:18:01.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.462+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.1 v2:172.21.15.154:6816/1267710005 3 ==== osd_op_reply(26 [pgnls start_epoch 43 out=49b] v41'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94d80 con 0x55c58db17000 2023-09-23T03:18:01.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.462+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:27 6.6 6:60000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8a000 con 0x55c58db17c00 2023-09-23T03:18:01.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.462+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.3 v2:172.21.15.154:6808/1133688468 3 ==== osd_op_reply(27 [pgnls start_epoch 43 out=49b] v41'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94d80 con 0x55c58db17c00 2023-09-23T03:18:01.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.463+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:28 6.26 6:64000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf54e00 con 0x55c58db17000 2023-09-23T03:18:01.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.463+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.1 v2:172.21.15.154:6816/1267710005 4 ==== osd_op_reply(28 [pgnls start_epoch 43 out=49b] v41'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94d80 con 0x55c58db17000 2023-09-23T03:18:01.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.463+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:29 6.16 6:68000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8a380 con 0x55c58daa3800 2023-09-23T03:18:01.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.463+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.0 v2:172.21.15.154:6800/2920938138 5 ==== osd_op_reply(29 [pgnls start_epoch 43 out=49b] v41'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94b40 con 0x55c58daa3800 2023-09-23T03:18:01.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.463+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:30 6.36 6:6c000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf54000 con 0x55c58cf70000 2023-09-23T03:18:01.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.465+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.5 v2:172.21.15.190:6800/882536885 3 ==== osd_op_reply(30 [pgnls start_epoch 43 out=101b] v41'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58cf70000 2023-09-23T03:18:01.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.465+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:31 6.36 6:6ee5e2c4:::100000001f9.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf54a80 con 0x55c58cf70000 2023-09-23T03:18:01.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.466+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.5 v2:172.21.15.190:6800/882536885 4 ==== osd_op_reply(31 100000001f9.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58cf70000 2023-09-23T03:18:01.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.466+0000 7fb6e22105c0 10 datascan.operator(): handling object 1099511628281.0 2023-09-23T03:18:01.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.466+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:32 6.36 6:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf54380 con 0x55c58cf70000 2023-09-23T03:18:01.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.470+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.5 v2:172.21.15.190:6800/882536885 5 ==== osd_op_reply(32 100000001f9.00000000 [call] v43'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58cf70000 2023-09-23T03:18:01.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.470+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:33 6.36 6:6fe38dc2::::head [pgnls start_epoch 43 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cc39c00 con 0x55c58cf70000 2023-09-23T03:18:01.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.471+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.5 v2:172.21.15.190:6800/882536885 6 ==== osd_op_reply(33 [pgnls start_epoch 43 out=81b] v43'18 uv18 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58cf70000 2023-09-23T03:18:01.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.471+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:34 6.36 6:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8b180 con 0x55c58cf70000 2023-09-23T03:18:01.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.472+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.5 v2:172.21.15.190:6800/882536885 7 ==== osd_op_reply(34 100000001f7.00000000 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58cf70000 2023-09-23T03:18:01.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.472+0000 7fb6e22105c0 10 datascan.operator(): handling object 1099511628279.0 2023-09-23T03:18:01.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.472+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:35 6.36 6:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8b500 con 0x55c58cf70000 2023-09-23T03:18:01.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.476+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.5 v2:172.21.15.190:6800/882536885 8 ==== osd_op_reply(35 100000001f7.00000000 [call] v43'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58cf70000 2023-09-23T03:18:01.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.476+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:36 6.e 6:70000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf54000 con 0x55c58cf71000 2023-09-23T03:18:01.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.477+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.4 v2:172.21.15.190:6808/717302641 7 ==== osd_op_reply(36 [pgnls start_epoch 43 out=49b] v41'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94d80 con 0x55c58cf71000 2023-09-23T03:18:01.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.477+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:37 6.2e 6:74000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8a700 con 0x55c58cf70400 2023-09-23T03:18:01.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.478+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.6 v2:172.21.15.190:6816/1964294001 3 ==== osd_op_reply(37 [pgnls start_epoch 43 out=49b] v41'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94b40 con 0x55c58cf70400 2023-09-23T03:18:01.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.479+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:38 6.1e 6:78000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf54700 con 0x55c58cf71000 2023-09-23T03:18:01.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.479+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.4 v2:172.21.15.190:6808/717302641 8 ==== osd_op_reply(38 [pgnls start_epoch 43 out=49b] v41'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94d80 con 0x55c58cf71000 2023-09-23T03:18:01.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.480+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:39 6.3e 6:7c000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8aa80 con 0x55c58db17c00 2023-09-23T03:18:01.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.480+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.3 v2:172.21.15.154:6808/1133688468 4 ==== osd_op_reply(39 [pgnls start_epoch 43 out=49b] v41'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94d80 con 0x55c58db17c00 2023-09-23T03:18:01.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.480+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:40 6.1 6:80000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8a000 con 0x55c58db17000 2023-09-23T03:18:01.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.481+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.1 v2:172.21.15.154:6816/1267710005 5 ==== osd_op_reply(40 [pgnls start_epoch 43 out=49b] v41'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94d80 con 0x55c58db17000 2023-09-23T03:18:01.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.481+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:41 6.21 6:84000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8ae00 con 0x55c58db17400 2023-09-23T03:18:01.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.482+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.7 v2:172.21.15.190:6824/3460136272 5 ==== osd_op_reply(41 [pgnls start_epoch 43 out=49b] v41'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58db17400 2023-09-23T03:18:01.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.482+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:42 6.11 6:88000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8b880 con 0x55c58db17c00 2023-09-23T03:18:01.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.483+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.3 v2:172.21.15.154:6808/1133688468 5 ==== osd_op_reply(42 [pgnls start_epoch 43 out=49b] v41'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94d80 con 0x55c58db17c00 2023-09-23T03:18:01.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.483+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:43 6.31 6:8c000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8bc00 con 0x55c58cf70000 2023-09-23T03:18:01.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.484+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.5 v2:172.21.15.190:6800/882536885 9 ==== osd_op_reply(43 [pgnls start_epoch 43 out=49b] v41'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58cf70000 2023-09-23T03:18:01.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.485+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:44 6.9 6:90000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf55c00 con 0x55c58daa3800 2023-09-23T03:18:01.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.485+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.0 v2:172.21.15.154:6800/2920938138 6 ==== osd_op_reply(44 [pgnls start_epoch 43 out=49b] v41'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94b40 con 0x55c58daa3800 2023-09-23T03:18:01.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.485+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:45 6.29 6:94000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8a380 con 0x55c58db17400 2023-09-23T03:18:01.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.486+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.7 v2:172.21.15.190:6824/3460136272 6 ==== osd_op_reply(45 [pgnls start_epoch 43 out=49b] v41'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58db17400 2023-09-23T03:18:01.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.486+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:46 6.19 6:98000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf55180 con 0x55c58cf70000 2023-09-23T03:18:01.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.488+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.5 v2:172.21.15.190:6800/882536885 10 ==== osd_op_reply(46 [pgnls start_epoch 43 out=81b] v41'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58cf70000 2023-09-23T03:18:01.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.488+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:47 6.19 6:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf54a80 con 0x55c58cf70000 2023-09-23T03:18:01.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.489+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.5 v2:172.21.15.190:6800/882536885 11 ==== osd_op_reply(47 100000001f8.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58cf70000 2023-09-23T03:18:01.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.489+0000 7fb6e22105c0 10 datascan.operator(): handling object 1099511628280.0 2023-09-23T03:18:01.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.489+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:48 6.19 6:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf54380 con 0x55c58cf70000 2023-09-23T03:18:01.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.494+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.5 v2:172.21.15.190:6800/882536885 12 ==== osd_op_reply(48 100000001f8.00000000 [call] v43'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58cf70000 2023-09-23T03:18:01.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.494+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:49 6.39 6:9c000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cc39880 con 0x55c58cf70000 2023-09-23T03:18:01.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.495+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.5 v2:172.21.15.190:6800/882536885 13 ==== osd_op_reply(49 [pgnls start_epoch 43 out=49b] v41'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58cf70000 2023-09-23T03:18:01.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.495+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:50 6.5 6:a0000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf54e00 con 0x55c58db17400 2023-09-23T03:18:01.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.496+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.7 v2:172.21.15.190:6824/3460136272 7 ==== osd_op_reply(50 [pgnls start_epoch 43 out=49b] v41'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58db17400 2023-09-23T03:18:01.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.496+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:51 6.25 6:a4000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8a700 con 0x55c58cf70400 2023-09-23T03:18:01.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.497+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.6 v2:172.21.15.190:6816/1964294001 4 ==== osd_op_reply(51 [pgnls start_epoch 43 out=49b] v41'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94b40 con 0x55c58cf70400 2023-09-23T03:18:01.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.497+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:52 6.15 6:a8000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf54000 con 0x55c58db17400 2023-09-23T03:18:01.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.498+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.7 v2:172.21.15.190:6824/3460136272 8 ==== osd_op_reply(52 [pgnls start_epoch 43 out=49b] v41'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58db17400 2023-09-23T03:18:01.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.498+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:53 6.35 6:ac000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8b180 con 0x55c58cf70000 2023-09-23T03:18:01.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.500+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.5 v2:172.21.15.190:6800/882536885 14 ==== osd_op_reply(53 [pgnls start_epoch 43 out=49b] v41'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58cf70000 2023-09-23T03:18:01.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.500+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:54 6.d 6:b0000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8a380 con 0x55c58cf70000 2023-09-23T03:18:01.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.501+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.5 v2:172.21.15.190:6800/882536885 15 ==== osd_op_reply(54 [pgnls start_epoch 43 out=49b] v41'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58cf70000 2023-09-23T03:18:01.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.501+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:55 6.2d 6:b4000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cc39c00 con 0x55c58db17800 2023-09-23T03:18:01.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.502+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.2 v2:172.21.15.154:6822/2884407243 4 ==== osd_op_reply(55 [pgnls start_epoch 43 out=49b] v41'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94b40 con 0x55c58db17800 2023-09-23T03:18:01.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.502+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:56 6.1d 6:b8000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8b500 con 0x55c58db17000 2023-09-23T03:18:01.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.503+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.1 v2:172.21.15.154:6816/1267710005 6 ==== osd_op_reply(56 [pgnls start_epoch 43 out=49b] v41'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94d80 con 0x55c58db17000 2023-09-23T03:18:01.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.503+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:57 6.3d 6:bc000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf55500 con 0x55c58daa3800 2023-09-23T03:18:01.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.504+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.0 v2:172.21.15.154:6800/2920938138 7 ==== osd_op_reply(57 [pgnls start_epoch 43 out=49b] v41'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94b40 con 0x55c58daa3800 2023-09-23T03:18:01.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.504+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:58 6.3 6:c0000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf55180 con 0x55c58db17400 2023-09-23T03:18:01.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.505+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.7 v2:172.21.15.190:6824/3460136272 9 ==== osd_op_reply(58 [pgnls start_epoch 43 out=49b] v41'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58db17400 2023-09-23T03:18:01.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.505+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:59 6.23 6:c4000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf55880 con 0x55c58cf70400 2023-09-23T03:18:01.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.506+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.6 v2:172.21.15.190:6816/1964294001 5 ==== osd_op_reply(59 [pgnls start_epoch 43 out=49b] v41'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94b40 con 0x55c58cf70400 2023-09-23T03:18:01.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.506+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:60 6.13 6:c8000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf54a80 con 0x55c58db17c00 2023-09-23T03:18:01.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.507+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.3 v2:172.21.15.154:6808/1133688468 6 ==== osd_op_reply(60 [pgnls start_epoch 43 out=49b] v41'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94d80 con 0x55c58db17c00 2023-09-23T03:18:01.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.507+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:61 6.33 6:cc000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf55c00 con 0x55c58cf70000 2023-09-23T03:18:01.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.508+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.5 v2:172.21.15.190:6800/882536885 16 ==== osd_op_reply(61 [pgnls start_epoch 43 out=49b] v41'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58cf70000 2023-09-23T03:18:01.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.509+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:62 6.b 6:d0000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf54380 con 0x55c58db17c00 2023-09-23T03:18:01.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.509+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.3 v2:172.21.15.154:6808/1133688468 7 ==== osd_op_reply(62 [pgnls start_epoch 43 out=49b] v41'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94d80 con 0x55c58db17c00 2023-09-23T03:18:01.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.509+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:63 6.2b 6:d4000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8aa80 con 0x55c58cf70000 2023-09-23T03:18:01.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.510+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.5 v2:172.21.15.190:6800/882536885 17 ==== osd_op_reply(63 [pgnls start_epoch 43 out=49b] v41'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58cf70000 2023-09-23T03:18:01.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.511+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:64 6.1b 6:d8000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf54e00 con 0x55c58db17c00 2023-09-23T03:18:01.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.511+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.3 v2:172.21.15.154:6808/1133688468 8 ==== osd_op_reply(64 [pgnls start_epoch 43 out=49b] v41'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94d80 con 0x55c58db17c00 2023-09-23T03:18:01.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.511+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:65 6.3b 6:dc000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8a000 con 0x55c58db17800 2023-09-23T03:18:01.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.512+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.2 v2:172.21.15.154:6822/2884407243 5 ==== osd_op_reply(65 [pgnls start_epoch 43 out=49b] v41'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94b40 con 0x55c58db17800 2023-09-23T03:18:01.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.512+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:66 6.7 6:e0000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cc39880 con 0x55c58cf70000 2023-09-23T03:18:01.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.514+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.5 v2:172.21.15.190:6800/882536885 18 ==== osd_op_reply(66 [pgnls start_epoch 43 out=81b] v41'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58cf70000 2023-09-23T03:18:01.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.514+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:67 6.7 6:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cc39880 con 0x55c58cf70000 2023-09-23T03:18:01.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.515+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.5 v2:172.21.15.190:6800/882536885 19 ==== osd_op_reply(67 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58cf70000 2023-09-23T03:18:01.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.515+0000 7fb6e22105c0 10 datascan.operator(): handling object 1099511627777.0 2023-09-23T03:18:01.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.515+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:68 6.7 6:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8b180 con 0x55c58cf70000 2023-09-23T03:18:01.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.519+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.5 v2:172.21.15.190:6800/882536885 20 ==== osd_op_reply(68 10000000001.00000000 [call] v43'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58cf70000 2023-09-23T03:18:01.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.520+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:69 6.27 6:e4000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8a380 con 0x55c58cf70400 2023-09-23T03:18:01.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.521+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.6 v2:172.21.15.190:6816/1964294001 6 ==== osd_op_reply(69 [pgnls start_epoch 43 out=49b] v41'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94b40 con 0x55c58cf70400 2023-09-23T03:18:01.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.521+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:70 6.17 6:e8000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8a700 con 0x55c58cf71000 2023-09-23T03:18:01.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.522+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.4 v2:172.21.15.190:6808/717302641 9 ==== osd_op_reply(70 [pgnls start_epoch 43 out=49b] v41'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94d80 con 0x55c58cf71000 2023-09-23T03:18:01.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.522+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:71 6.37 6:ec000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8b880 con 0x55c58daa3800 2023-09-23T03:18:01.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.523+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.0 v2:172.21.15.154:6800/2920938138 8 ==== osd_op_reply(71 [pgnls start_epoch 43 out=49b] v41'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94b40 con 0x55c58daa3800 2023-09-23T03:18:01.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.523+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:72 6.f 6:f0000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8a000 con 0x55c58db17800 2023-09-23T03:18:01.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.524+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.2 v2:172.21.15.154:6822/2884407243 6 ==== osd_op_reply(72 [pgnls start_epoch 43 out=49b] v41'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94b40 con 0x55c58db17800 2023-09-23T03:18:01.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.524+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:73 6.2f 6:f4000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8b500 con 0x55c58db17400 2023-09-23T03:18:01.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.525+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.7 v2:172.21.15.190:6824/3460136272 10 ==== osd_op_reply(73 [pgnls start_epoch 43 out=49b] v41'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58db17400 2023-09-23T03:18:01.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.525+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:74 6.1f 6:f8000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8a380 con 0x55c58db17c00 2023-09-23T03:18:01.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.526+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.3 v2:172.21.15.154:6808/1133688468 9 ==== osd_op_reply(74 [pgnls start_epoch 43 out=49b] v41'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94d80 con 0x55c58db17c00 2023-09-23T03:18:01.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.526+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:75 6.3f 6:fc000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8ae00 con 0x55c58db17c00 2023-09-23T03:18:01.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.527+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.3 v2:172.21.15.154:6808/1133688468 10 ==== osd_op_reply(75 [pgnls start_epoch 43 out=49b] v41'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94d80 con 0x55c58db17c00 2023-09-23T03:18:01.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.527+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:76 7.0 7:00000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8aa80 con 0x55c58cf70000 2023-09-23T03:18:01.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.529+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.5 v2:172.21.15.190:6800/882536885 21 ==== osd_op_reply(76 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58cf70000 2023-09-23T03:18:01.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.529+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:77 7.10 7:08000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8bc00 con 0x55c58db17c00 2023-09-23T03:18:01.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.530+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.3 v2:172.21.15.154:6808/1133688468 11 ==== osd_op_reply(77 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94d80 con 0x55c58db17c00 2023-09-23T03:18:01.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.530+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:78 7.8 7:10000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf55500 con 0x55c58db17800 2023-09-23T03:18:01.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.531+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.2 v2:172.21.15.154:6822/2884407243 7 ==== osd_op_reply(78 [pgnls start_epoch 43 out=81b] v41'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c58da94b40 con 0x55c58db17800 2023-09-23T03:18:01.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.531+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:79 7.8 7:1247e94b:::100000001f9.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf54000 con 0x55c58db17800 2023-09-23T03:18:01.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.532+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.2 v2:172.21.15.154:6822/2884407243 8 ==== osd_op_reply(79 100000001f9.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c58da94b40 con 0x55c58db17800 2023-09-23T03:18:01.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.532+0000 7fb6e22105c0 10 datascan.operator(): handling object 1099511628281.1 2023-09-23T03:18:01.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.533+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:80 6.36 6:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf55180 con 0x55c58cf70000 2023-09-23T03:18:01.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.537+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.5 v2:172.21.15.190:6800/882536885 22 ==== osd_op_reply(80 100000001f9.00000000 [call,setxattr (8)] v43'20 uv20 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58cf70000 2023-09-23T03:18:01.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.537+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:81 7.18 7:18000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf55c00 con 0x55c58daa3800 2023-09-23T03:18:01.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.538+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.0 v2:172.21.15.154:6800/2920938138 9 ==== osd_op_reply(81 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94b40 con 0x55c58daa3800 2023-09-23T03:18:01.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.538+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:82 7.4 7:20000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf55880 con 0x55c58daa3800 2023-09-23T03:18:01.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.539+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.0 v2:172.21.15.154:6800/2920938138 10 ==== osd_op_reply(82 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94b40 con 0x55c58daa3800 2023-09-23T03:18:01.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.539+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:83 7.14 7:28000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf54700 con 0x55c58db17c00 2023-09-23T03:18:01.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.540+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.3 v2:172.21.15.154:6808/1133688468 12 ==== osd_op_reply(83 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94d80 con 0x55c58db17c00 2023-09-23T03:18:01.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.540+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:84 7.c 7:30000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cc39c00 con 0x55c58cf70400 2023-09-23T03:18:01.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.541+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.6 v2:172.21.15.190:6816/1964294001 7 ==== osd_op_reply(84 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94b40 con 0x55c58cf70400 2023-09-23T03:18:01.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.541+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:85 7.1c 7:38000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf54a80 con 0x55c58cf70000 2023-09-23T03:18:01.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.543+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.5 v2:172.21.15.190:6800/882536885 23 ==== osd_op_reply(85 [pgnls start_epoch 43 out=81b] v41'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58cf70000 2023-09-23T03:18:01.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.543+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:86 7.1c 7:3d6db04b:::100000001f6.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf55500 con 0x55c58cf70000 2023-09-23T03:18:01.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.544+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.5 v2:172.21.15.190:6800/882536885 24 ==== osd_op_reply(86 100000001f6.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58cf70000 2023-09-23T03:18:01.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.544+0000 7fb6e22105c0 10 datascan.operator(): handling object 1099511628278.1 2023-09-23T03:18:01.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.544+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:87 6.2a 6:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8b880 con 0x55c58cf71000 2023-09-23T03:18:01.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.549+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.4 v2:172.21.15.190:6808/717302641 10 ==== osd_op_reply(87 100000001f6.00000000 [call,setxattr (8)] v43'12 uv12 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55c58da94d80 con 0x55c58cf71000 2023-09-23T03:18:01.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.549+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:88 7.2 7:40000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8a000 con 0x55c58db17c00 2023-09-23T03:18:01.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.550+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.3 v2:172.21.15.154:6808/1133688468 13 ==== osd_op_reply(88 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94d80 con 0x55c58db17c00 2023-09-23T03:18:01.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.550+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:89 7.12 7:48000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf54000 con 0x55c58cf70400 2023-09-23T03:18:01.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.551+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.6 v2:172.21.15.190:6816/1964294001 8 ==== osd_op_reply(89 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94b40 con 0x55c58cf70400 2023-09-23T03:18:01.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.551+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:90 7.a 7:50000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58d9fe000 con 0x55c58db17c00 2023-09-23T03:18:01.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.552+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.3 v2:172.21.15.154:6808/1133688468 14 ==== osd_op_reply(90 [pgnls start_epoch 43 out=81b] v41'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c58da94d80 con 0x55c58db17c00 2023-09-23T03:18:01.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.552+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:91 7.a 7:57fe7535:::100000001f6.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8a700 con 0x55c58db17c00 2023-09-23T03:18:01.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.553+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.3 v2:172.21.15.154:6808/1133688468 15 ==== osd_op_reply(91 100000001f6.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c58da94d80 con 0x55c58db17c00 2023-09-23T03:18:01.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.553+0000 7fb6e22105c0 10 datascan.operator(): handling object 1099511628278.0 2023-09-23T03:18:01.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.554+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:92 6.2a 6:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8a380 con 0x55c58cf71000 2023-09-23T03:18:01.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.558+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.4 v2:172.21.15.190:6808/717302641 11 ==== osd_op_reply(92 100000001f6.00000000 [call,setxattr (8)] v43'13 uv13 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55c58da94d80 con 0x55c58cf71000 2023-09-23T03:18:01.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.558+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:93 7.1a 7:58000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8ae00 con 0x55c58cf71000 2023-09-23T03:18:01.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.559+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.4 v2:172.21.15.190:6808/717302641 12 ==== osd_op_reply(93 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94d80 con 0x55c58cf71000 2023-09-23T03:18:01.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.559+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:94 7.6 7:60000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58d9fe380 con 0x55c58daa3800 2023-09-23T03:18:01.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.560+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.0 v2:172.21.15.154:6800/2920938138 11 ==== osd_op_reply(94 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94b40 con 0x55c58daa3800 2023-09-23T03:18:01.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.560+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:95 7.16 7:68000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf55c00 con 0x55c58cf71000 2023-09-23T03:18:01.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.562+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.4 v2:172.21.15.190:6808/717302641 13 ==== osd_op_reply(95 [pgnls start_epoch 43 out=101b] v41'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c58da94d80 con 0x55c58cf71000 2023-09-23T03:18:01.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.562+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:96 7.16 7:6ee5e2c4:::100000001f9.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8bc00 con 0x55c58cf71000 2023-09-23T03:18:01.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.563+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.4 v2:172.21.15.190:6808/717302641 14 ==== osd_op_reply(96 100000001f9.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c58da94d80 con 0x55c58cf71000 2023-09-23T03:18:01.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.563+0000 7fb6e22105c0 10 datascan.operator(): handling object 1099511628281.0 2023-09-23T03:18:01.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.563+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:97 6.36 6:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cc39c00 con 0x55c58cf70000 2023-09-23T03:18:01.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.567+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.5 v2:172.21.15.190:6800/882536885 25 ==== osd_op_reply(97 100000001f9.00000000 [call,setxattr (8)] v43'21 uv21 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58cf70000 2023-09-23T03:18:01.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.567+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:98 7.16 7:6fe38dc2::::head [pgnls start_epoch 43 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf55180 con 0x55c58cf71000 2023-09-23T03:18:01.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.568+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.4 v2:172.21.15.190:6808/717302641 15 ==== osd_op_reply(98 [pgnls start_epoch 43 out=81b] v41'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c58da94d80 con 0x55c58cf71000 2023-09-23T03:18:01.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.569+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:99 7.16 7:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf54a80 con 0x55c58cf71000 2023-09-23T03:18:01.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.570+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.4 v2:172.21.15.190:6808/717302641 16 ==== osd_op_reply(99 100000001f7.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c58da94d80 con 0x55c58cf71000 2023-09-23T03:18:01.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.570+0000 7fb6e22105c0 10 datascan.operator(): handling object 1099511628279.0 2023-09-23T03:18:01.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.570+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:100 6.36 6:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf55500 con 0x55c58cf70000 2023-09-23T03:18:01.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.574+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.5 v2:172.21.15.190:6800/882536885 26 ==== osd_op_reply(100 100000001f7.00000000 [call,setxattr (8)] v43'22 uv22 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58cf70000 2023-09-23T03:18:01.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.575+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:101 7.e 7:70000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf55c00 con 0x55c58db17800 2023-09-23T03:18:01.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.576+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.2 v2:172.21.15.154:6822/2884407243 9 ==== osd_op_reply(101 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94b40 con 0x55c58db17800 2023-09-23T03:18:01.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.576+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:102 7.1e 7:78000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58d9fe700 con 0x55c58db17000 2023-09-23T03:18:01.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.577+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.1 v2:172.21.15.154:6816/1267710005 7 ==== osd_op_reply(102 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94d80 con 0x55c58db17000 2023-09-23T03:18:01.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.577+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:103 7.1 7:80000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf55880 con 0x55c58cf71000 2023-09-23T03:18:01.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.578+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.4 v2:172.21.15.190:6808/717302641 17 ==== osd_op_reply(103 [pgnls start_epoch 43 out=81b] v41'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c58da94d80 con 0x55c58cf71000 2023-09-23T03:18:01.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.578+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:104 7.1 7:848ebb0b:::100000001f7.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58d9fe000 con 0x55c58cf71000 2023-09-23T03:18:01.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.580+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.4 v2:172.21.15.190:6808/717302641 18 ==== osd_op_reply(104 100000001f7.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c58da94d80 con 0x55c58cf71000 2023-09-23T03:18:01.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.580+0000 7fb6e22105c0 10 datascan.operator(): handling object 1099511628279.1 2023-09-23T03:18:01.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.580+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:105 6.36 6:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cc39880 con 0x55c58cf70000 2023-09-23T03:18:01.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.584+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.5 v2:172.21.15.190:6800/882536885 27 ==== osd_op_reply(105 100000001f7.00000000 [call,setxattr (8)] v43'23 uv23 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58cf70000 2023-09-23T03:18:01.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.584+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:106 7.11 7:88000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cc39c00 con 0x55c58db17400 2023-09-23T03:18:01.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.585+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.7 v2:172.21.15.190:6824/3460136272 11 ==== osd_op_reply(106 [pgnls start_epoch 43 out=81b] v41'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58db17400 2023-09-23T03:18:01.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.585+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:107 7.11 7:893be98d:::100000001f8.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8e000 con 0x55c58db17400 2023-09-23T03:18:01.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.586+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.7 v2:172.21.15.190:6824/3460136272 12 ==== osd_op_reply(107 100000001f8.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58db17400 2023-09-23T03:18:01.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.587+0000 7fb6e22105c0 10 datascan.operator(): handling object 1099511628280.1 2023-09-23T03:18:01.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.587+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:108 6.19 6:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8e380 con 0x55c58cf70000 2023-09-23T03:18:01.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.591+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.5 v2:172.21.15.190:6800/882536885 28 ==== osd_op_reply(108 100000001f8.00000000 [call,setxattr (8)] v43'13 uv13 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58cf70000 2023-09-23T03:18:01.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.591+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:109 7.9 7:90000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8e700 con 0x55c58daa3800 2023-09-23T03:18:01.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.592+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.0 v2:172.21.15.154:6800/2920938138 12 ==== osd_op_reply(109 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94b40 con 0x55c58daa3800 2023-09-23T03:18:01.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.592+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:110 7.19 7:98000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58d9fea80 con 0x55c58cf70400 2023-09-23T03:18:01.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.593+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.6 v2:172.21.15.190:6816/1964294001 9 ==== osd_op_reply(110 [pgnls start_epoch 43 out=81b] v41'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c58da94b40 con 0x55c58cf70400 2023-09-23T03:18:01.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.593+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:111 7.19 7:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8ea80 con 0x55c58cf70400 2023-09-23T03:18:01.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.594+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.6 v2:172.21.15.190:6816/1964294001 10 ==== osd_op_reply(111 100000001f8.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c58da94b40 con 0x55c58cf70400 2023-09-23T03:18:01.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.594+0000 7fb6e22105c0 10 datascan.operator(): handling object 1099511628280.0 2023-09-23T03:18:01.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.594+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:112 6.19 6:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8ee00 con 0x55c58cf70000 2023-09-23T03:18:01.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.599+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.5 v2:172.21.15.190:6800/882536885 29 ==== osd_op_reply(112 100000001f8.00000000 [call,setxattr (8)] v43'14 uv14 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58cf70000 2023-09-23T03:18:01.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.599+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:113 7.5 7:a0000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8f180 con 0x55c58db17400 2023-09-23T03:18:01.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.600+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.7 v2:172.21.15.190:6824/3460136272 13 ==== osd_op_reply(113 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58db17400 2023-09-23T03:18:01.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.600+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:114 7.15 7:a8000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58d9fee00 con 0x55c58db17400 2023-09-23T03:18:01.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.601+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.7 v2:172.21.15.190:6824/3460136272 14 ==== osd_op_reply(114 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58db17400 2023-09-23T03:18:01.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.601+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:115 7.d 7:b0000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf54380 con 0x55c58cf70400 2023-09-23T03:18:01.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.602+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.6 v2:172.21.15.190:6816/1964294001 11 ==== osd_op_reply(115 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94b40 con 0x55c58cf70400 2023-09-23T03:18:01.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.603+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:116 7.1d 7:b8000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58d9ff180 con 0x55c58db17000 2023-09-23T03:18:01.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.603+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.1 v2:172.21.15.154:6816/1267710005 8 ==== osd_op_reply(116 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94d80 con 0x55c58db17000 2023-09-23T03:18:01.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.604+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:117 7.3 7:c0000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf54e00 con 0x55c58db17c00 2023-09-23T03:18:01.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.604+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.3 v2:172.21.15.154:6808/1133688468 16 ==== osd_op_reply(117 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94d80 con 0x55c58db17c00 2023-09-23T03:18:01.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.604+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:118 7.13 7:c8000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58d9ff500 con 0x55c58daa3800 2023-09-23T03:18:01.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.605+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.0 v2:172.21.15.154:6800/2920938138 13 ==== osd_op_reply(118 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94b40 con 0x55c58daa3800 2023-09-23T03:18:01.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.605+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:119 7.b 7:d0000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf54700 con 0x55c58db17000 2023-09-23T03:18:01.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.606+0000 7fb6e06e1640 1 -- 172.21.15.154:0/275856908 <== osd.1 v2:172.21.15.154:6816/1267710005 9 ==== osd_op_reply(119 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94d80 con 0x55c58db17000 2023-09-23T03:18:01.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.606+0000 7fb6daed6640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:120 7.1b 7:d8000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58d9ff880 con 0x55c58daa3800 2023-09-23T03:18:01.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.607+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.0 v2:172.21.15.154:6800/2920938138 14 ==== osd_op_reply(120 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94b40 con 0x55c58daa3800 2023-09-23T03:18:01.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.607+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:121 7.7 7:e0000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58d9fe380 con 0x55c58cf70400 2023-09-23T03:18:01.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.609+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.6 v2:172.21.15.190:6816/1964294001 12 ==== osd_op_reply(121 [pgnls start_epoch 43 out=81b] v41'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c58da94b40 con 0x55c58cf70400 2023-09-23T03:18:01.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.609+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:122 7.7 7:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8f500 con 0x55c58cf70400 2023-09-23T03:18:01.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.610+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.6 v2:172.21.15.190:6816/1964294001 13 ==== osd_op_reply(122 10000000001.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c58da94b40 con 0x55c58cf70400 2023-09-23T03:18:01.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.610+0000 7fb6e22105c0 10 datascan.operator(): handling object 1099511627777.0 2023-09-23T03:18:01.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.610+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:123 6.7 6:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8f880 con 0x55c58cf70000 2023-09-23T03:18:01.615 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.614+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.5 v2:172.21.15.190:6800/882536885 30 ==== osd_op_reply(123 10000000001.00000000 [call,setxattr (8)] v43'8 uv8 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58cf70000 2023-09-23T03:18:01.615 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.615+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:124 7.17 7:e8000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8fc00 con 0x55c58db17400 2023-09-23T03:18:01.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.616+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.7 v2:172.21.15.190:6824/3460136272 15 ==== osd_op_reply(124 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58db17400 2023-09-23T03:18:01.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.616+0000 7fb6db6d7640 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:125 7.f 7:f0000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58d9fea80 con 0x55c58daa3800 2023-09-23T03:18:01.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.617+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.0 v2:172.21.15.154:6800/2920938138 15 ==== osd_op_reply(125 [pgnls start_epoch 43 out=81b] v41'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c58da94b40 con 0x55c58daa3800 2023-09-23T03:18:01.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.617+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:126 7.f 7:f526278d:::10000000001.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8e000 con 0x55c58daa3800 2023-09-23T03:18:01.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.618+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.0 v2:172.21.15.154:6800/2920938138 16 ==== osd_op_reply(126 10000000001.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c58da94b40 con 0x55c58daa3800 2023-09-23T03:18:01.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.618+0000 7fb6e22105c0 10 datascan.operator(): handling object 1099511627777.1 2023-09-23T03:18:01.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.618+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:127 6.7 6:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8e380 con 0x55c58cf70000 2023-09-23T03:18:01.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.622+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 <== osd.5 v2:172.21.15.190:6800/882536885 31 ==== osd_op_reply(127 10000000001.00000000 [call,setxattr (8)] v43'9 uv9 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55c58da94fc0 con 0x55c58cf70000 2023-09-23T03:18:01.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.622+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:128 7.1f 7:f8000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c58cf8ee00 con 0x55c58cf70400 2023-09-23T03:18:01.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.623+0000 7fb6df6df640 1 -- 172.21.15.154:0/275856908 <== osd.6 v2:172.21.15.190:6816/1964294001 14 ==== osd_op_reply(128 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c58da94b40 con 0x55c58cf70400 2023-09-23T03:18:01.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.624+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55c58daa3800 msgr2=0x55c58cc4ac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.624+0000 7fb6e22105c0 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55c58daa3800 0x55c58cc4ac00 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.624+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55c58db17000 msgr2=0x55c58cf4a000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.624+0000 7fb6e22105c0 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55c58db17000 0x55c58cf4a000 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.624+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55c58db17800 msgr2=0x55c58cf4ab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.624+0000 7fb6e22105c0 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55c58db17800 0x55c58cf4ab00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.624+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55c58db17c00 msgr2=0x55c58cf4b080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.624+0000 7fb6e22105c0 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55c58db17c00 0x55c58cf4b080 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.624+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55c58cf71000 msgr2=0x55c58cf4c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.624+0000 7fb6e22105c0 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55c58cf71000 0x55c58cf4c100 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.624+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55c58cf70000 msgr2=0x55c58cf4b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.624+0000 7fb6e22105c0 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55c58cf70000 0x55c58cf4b600 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.624+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 reap_dead start 2023-09-23T03:18:01.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.624+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55c58cf70400 msgr2=0x55c58cf4bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.624+0000 7fb6e22105c0 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55c58cf70400 0x55c58cf4bb80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.624+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55c58db17400 msgr2=0x55c58cf4a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.624+0000 7fb6dfee0640 1 -- 172.21.15.154:0/275856908 reap_dead start 2023-09-23T03:18:01.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.624+0000 7fb6e22105c0 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55c58db17400 0x55c58cf4a580 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.624+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55c58db16c00 msgr2=0x55c58cc4b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.624+0000 7fb6e22105c0 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55c58db16c00 0x55c58cc4b700 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55c58d95a420 tx=0x55c58cf1e060 comp rx=0 tx=0).stop 2023-09-23T03:18:01.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.625+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c58daa3000 msgr2=0x55c58cc4a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.625+0000 7fb6e22105c0 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c58daa3000 0x55c58cc4a100 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x55c58d9f4e70 tx=0x55c58dabcd20 comp rx=0 tx=0).stop 2023-09-23T03:18:01.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.625+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 shutdown_connections 2023-09-23T03:18:01.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.625+0000 7fb6e22105c0 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55c58db17400 0x55c58cf4a580 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.625+0000 7fb6e22105c0 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55c58db16c00 0x55c58cc4b700 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.625+0000 7fb6e22105c0 1 --2- 172.21.15.154:0/275856908 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c58daa3000 0x55c58cc4a100 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.625+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 >> 172.21.15.154:0/275856908 conn(0x55c58d9bc400 msgr2=0x55c58d959800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:01.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.625+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 shutdown_connections 2023-09-23T03:18:01.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.625+0000 7fb6e22105c0 1 -- 172.21.15.154:0/275856908 wait complete. 2023-09-23T03:18:01.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.626+0000 7fb6e22105c0 1 -- 172.21.15.154:0/698261334 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c58cbdfc00 msgr2=0x55c58cc49b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.626+0000 7fb6e22105c0 1 --2- 172.21.15.154:0/698261334 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c58cbdfc00 0x55c58cc49b80 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x55c58d9f5620 tx=0x55c58d9ba180 comp rx=0 tx=0).stop 2023-09-23T03:18:01.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.626+0000 7fb6e22105c0 1 -- 172.21.15.154:0/698261334 shutdown_connections 2023-09-23T03:18:01.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.626+0000 7fb6e22105c0 1 --2- 172.21.15.154:0/698261334 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c58d9bc800 0x55c58cc49600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.626+0000 7fb6e22105c0 1 --2- 172.21.15.154:0/698261334 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c58cbdfc00 0x55c58cc49b80 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.626+0000 7fb6e22105c0 1 --2- 172.21.15.154:0/698261334 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c58d9bc000 0x55c58cc49080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.626+0000 7fb6e22105c0 1 -- 172.21.15.154:0/698261334 >> 172.21.15.154:0/698261334 conn(0x55c58cbdf000 msgr2=0x55c58d957800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:01.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.627+0000 7fb6e22105c0 1 -- 172.21.15.154:0/698261334 shutdown_connections 2023-09-23T03:18:01.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.627+0000 7fb6e22105c0 1 -- 172.21.15.154:0/698261334 wait complete. 2023-09-23T03:18:01.639 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents'] in time 0:00:00.314630, result: 2023-09-23T03:18:01.639 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes 2023-09-23T03:18:01.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.742+0000 7f2f8f4435c0 1 -- start start 2023-09-23T03:18:01.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.742+0000 7f2f8f4435c0 1 Processor -- start 2023-09-23T03:18:01.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.743+0000 7f2f8f4435c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c967764c00 0x55c9677ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.743+0000 7f2f8f4435c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c967765800 0x55c9677ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.743+0000 7f2f8f4435c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c967765c00 0x55c9677cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.743+0000 7f2f8f4435c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9677af180 con 0x55c967765800 2023-09-23T03:18:01.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.743+0000 7f2f8f4435c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9677ae140 con 0x55c967764c00 2023-09-23T03:18:01.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.743+0000 7f2f8f4435c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c9677ae000 con 0x55c967765c00 2023-09-23T03:18:01.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.744+0000 7f2f8ccad640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c967765800 0x55c9677ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.744+0000 7f2f8ccad640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c967765800 0x55c9677ceb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60484/0 (socket says 172.21.15.154:60484) 2023-09-23T03:18:01.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.744+0000 7f2f8ccad640 1 -- 172.21.15.154:0/1122821403 learned_addr learned my addr 172.21.15.154:0/1122821403 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:01.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.744+0000 7f2f8dcaf640 1 --2- 172.21.15.154:0/1122821403 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c967765c00 0x55c9677cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.744+0000 7f2f8d4ae640 1 --2- 172.21.15.154:0/1122821403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c967764c00 0x55c9677ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.744+0000 7f2f8ccad640 1 -- 172.21.15.154:0/1122821403 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c967765c00 msgr2=0x55c9677cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:01.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.744+0000 7f2f8ccad640 1 --2- 172.21.15.154:0/1122821403 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c967765c00 0x55c9677cf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.744+0000 7f2f8ccad640 1 -- 172.21.15.154:0/1122821403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c967764c00 msgr2=0x55c9677ce580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:01.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.744+0000 7f2f8ccad640 1 --2- 172.21.15.154:0/1122821403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c967764c00 0x55c9677ce580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.744+0000 7f2f8ccad640 1 -- 172.21.15.154:0/1122821403 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c967798b60 con 0x55c967765800 2023-09-23T03:18:01.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.744+0000 7f2f8ccad640 1 --2- 172.21.15.154:0/1122821403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c967765800 0x55c9677ceb00 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x55c968566a80 tx=0x55c967a46120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=685f0d6e4f5adf19 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.745+0000 7f2f8aca9640 1 -- 172.21.15.154:0/1122821403 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c9677ad080 con 0x55c967765800 2023-09-23T03:18:01.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.745+0000 7f2f8aca9640 1 -- 172.21.15.154:0/1122821403 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c968534300 con 0x55c967765800 2023-09-23T03:18:01.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.745+0000 7f2f8aca9640 1 -- 172.21.15.154:0/1122821403 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c9677ad600 con 0x55c967765800 2023-09-23T03:18:01.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.745+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/1122821403 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c9677991e0 con 0x55c967765800 2023-09-23T03:18:01.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.745+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/1122821403 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55c967798b60 con 0x55c967765800 2023-09-23T03:18:01.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.745+0000 7f2f8f4435c0 4 waiting for MDS map... 2023-09-23T03:18:01.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.745+0000 7f2f8aca9640 1 -- 172.21.15.154:0/1122821403 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(43..43 src has 1..43) v4 ==== 6605+0+0 (secure 0 0 0) 0x55c967824c40 con 0x55c967765800 2023-09-23T03:18:01.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.745+0000 7f2f8aca9640 1 -- 172.21.15.154:0/1122821403 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 25) v1 ==== 3306+0+0 (secure 0 0 0) 0x55c9677b7200 con 0x55c967765800 2023-09-23T03:18:01.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.745+0000 7f2f8f4435c0 4 Got MDS map 25 2023-09-23T03:18:01.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.745+0000 7f2f8f4435c0 4 datascan.main: Using metadata pool output 2023-09-23T03:18:01.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.745+0000 7f2f8f4435c0 4 datascan.main: connecting to RADOS... 2023-09-23T03:18:01.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.746+0000 7f2f8f4435c0 1 Processor -- start 2023-09-23T03:18:01.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.746+0000 7f2f8f4435c0 1 -- start start 2023-09-23T03:18:01.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.747+0000 7f2f8f4435c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c968537c00 0x55c9677cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.747+0000 7f2f8f4435c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c967aaa000 0x55c9677cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.747+0000 7f2f8f4435c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c967aaa400 0x55c9677d0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.747+0000 7f2f8f4435c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9677aea00 con 0x55c968537c00 2023-09-23T03:18:01.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.747+0000 7f2f8f4435c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c967833180 con 0x55c967aaa000 2023-09-23T03:18:01.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.747+0000 7f2f8f4435c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c967832dc0 con 0x55c967aaa400 2023-09-23T03:18:01.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.747+0000 7f2f8dcaf640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c968537c00 0x55c9677cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.747+0000 7f2f8dcaf640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c968537c00 0x55c9677cf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60492/0 (socket says 172.21.15.154:60492) 2023-09-23T03:18:01.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.747+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/1127052269 learned_addr learned my addr 172.21.15.154:0/1127052269 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:01.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.748+0000 7f2f8ccad640 1 --2- 172.21.15.154:0/1127052269 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c967aaa400 0x55c9677d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.748+0000 7f2f8d4ae640 1 --2- 172.21.15.154:0/1127052269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c967aaa000 0x55c9677cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.748+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/1127052269 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c967aaa400 msgr2=0x55c9677d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:01.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.748+0000 7f2f8dcaf640 1 --2- 172.21.15.154:0/1127052269 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c967aaa400 0x55c9677d0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.748+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/1127052269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c967aaa000 msgr2=0x55c9677cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:01.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.748+0000 7f2f8dcaf640 1 --2- 172.21.15.154:0/1127052269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c967aaa000 0x55c9677cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.748+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/1127052269 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c967798340 con 0x55c968537c00 2023-09-23T03:18:01.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.748+0000 7f2f8d4ae640 1 --2- 172.21.15.154:0/1127052269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c967aaa000 0x55c9677cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:18:01.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.748+0000 7f2f8dcaf640 1 --2- 172.21.15.154:0/1127052269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c968537c00 0x55c9677cf600 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x55c967a508a0 tx=0x55c967a5ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=742c867addb18ff7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.748+0000 7f2f88ca5640 1 -- 172.21.15.154:0/1127052269 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c9677ad080 con 0x55c968537c00 2023-09-23T03:18:01.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.748+0000 7f2f88ca5640 1 -- 172.21.15.154:0/1127052269 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c96778aa80 con 0x55c968537c00 2023-09-23T03:18:01.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.748+0000 7f2f88ca5640 1 -- 172.21.15.154:0/1127052269 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c9677ad600 con 0x55c968537c00 2023-09-23T03:18:01.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.749+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/1127052269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c968537c00 msgr2=0x55c9677cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.749+0000 7f2f8f4435c0 1 --2- 172.21.15.154:0/1127052269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c968537c00 0x55c9677cf600 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x55c967a508a0 tx=0x55c967a5ccc0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.749+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/1127052269 shutdown_connections 2023-09-23T03:18:01.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.749+0000 7f2f8f4435c0 1 --2- 172.21.15.154:0/1127052269 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c967aaa400 0x55c9677d0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.749+0000 7f2f8f4435c0 1 --2- 172.21.15.154:0/1127052269 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c967aaa000 0x55c9677cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.749+0000 7f2f8f4435c0 1 --2- 172.21.15.154:0/1127052269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c968537c00 0x55c9677cf600 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.749+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/1127052269 >> 172.21.15.154:0/1127052269 conn(0x55c968537800 msgr2=0x55c9684ef000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:01.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.749+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/1127052269 shutdown_connections 2023-09-23T03:18:01.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.749+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/1127052269 wait complete. 2023-09-23T03:18:01.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.750+0000 7f2f8f4435c0 1 Processor -- start 2023-09-23T03:18:01.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.750+0000 7f2f8f4435c0 1 -- start start 2023-09-23T03:18:01.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.750+0000 7f2f8f4435c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c968537800 0x55c9677d1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.750+0000 7f2f8f4435c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c967aab000 0x55c9677d0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.750+0000 7f2f8f4435c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c967aaac00 0x55c9677d0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.750+0000 7f2f8f4435c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9677ae140 con 0x55c967aab000 2023-09-23T03:18:01.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.750+0000 7f2f8f4435c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9677af180 con 0x55c967aaac00 2023-09-23T03:18:01.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.750+0000 7f2f8f4435c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c9677aeb40 con 0x55c968537800 2023-09-23T03:18:01.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.751+0000 7f2f8d4ae640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c967aab000 0x55c9677d0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.751+0000 7f2f8d4ae640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c967aab000 0x55c9677d0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60496/0 (socket says 172.21.15.154:60496) 2023-09-23T03:18:01.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.751+0000 7f2f8d4ae640 1 -- 172.21.15.154:0/2871567064 learned_addr learned my addr 172.21.15.154:0/2871567064 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:01.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.751+0000 7f2f8ccad640 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c967aaac00 0x55c9677d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.751+0000 7f2f8dcaf640 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c968537800 0x55c9677d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.751+0000 7f2f8d4ae640 1 -- 172.21.15.154:0/2871567064 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c968537800 msgr2=0x55c9677d1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.751+0000 7f2f8d4ae640 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c968537800 0x55c9677d1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.751+0000 7f2f8d4ae640 1 -- 172.21.15.154:0/2871567064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c967aaac00 msgr2=0x55c9677d0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.751+0000 7f2f8d4ae640 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c967aaac00 0x55c9677d0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.751+0000 7f2f8d4ae640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c967798000 con 0x55c967aab000 2023-09-23T03:18:01.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.751+0000 7f2f8dcaf640 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c968537800 0x55c9677d1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:18:01.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.752+0000 7f2f8d4ae640 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c967aab000 0x55c9677d0c00 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x55c967a51560 tx=0x55c967a5cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.752+0000 7f2f874a2640 1 -- 172.21.15.154:0/2871567064 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c9677ad080 con 0x55c967aab000 2023-09-23T03:18:01.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.752+0000 7f2f874a2640 1 -- 172.21.15.154:0/2871567064 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c96778b800 con 0x55c967aab000 2023-09-23T03:18:01.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.752+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c967798ea0 con 0x55c967aab000 2023-09-23T03:18:01.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.752+0000 7f2f874a2640 1 -- 172.21.15.154:0/2871567064 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c9677ad600 con 0x55c967aab000 2023-09-23T03:18:01.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.752+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c967799520 con 0x55c967aab000 2023-09-23T03:18:01.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.752+0000 7f2f8f4435c0 4 datascan.init: resolving metadata pool 5 2023-09-23T03:18:01.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.754+0000 7f2f874a2640 1 -- 172.21.15.154:0/2871567064 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x55c967796840 con 0x55c967aab000 2023-09-23T03:18:01.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.754+0000 7f2f874a2640 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55c967ab8c00 0x55c9677d1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.754+0000 7f2f874a2640 1 -- 172.21.15.154:0/2871567064 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6605+0+0 (secure 0 0 0) 0x55c967824a80 con 0x55c967aab000 2023-09-23T03:18:01.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.754+0000 7f2f8f4435c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-09-23T03:18:01.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.754+0000 7f2f8f4435c0 4 datascan.main: data pool 'cephfs_data' has ID 6 2023-09-23T03:18:01.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.754+0000 7f2f8f4435c0 4 datascan.main: opening data pool 'cephfs_data' 2023-09-23T03:18:01.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.754+0000 7f2f8f4435c0 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55c967aaa800 0x55c9677cf600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.754+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:1 5.14 5:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c9677bf880 con 0x55c967aaa800 2023-09-23T03:18:01.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.755+0000 7f2f8dcaf640 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55c967ab8c00 0x55c9677d1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.755+0000 7f2f8ccad640 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55c967aaa800 0x55c9677cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.755+0000 7f2f8dcaf640 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55c967ab8c00 0x55c9677d1700 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55c967a508a0 tx=0x55c967a5cc00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.755+0000 7f2f8ccad640 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55c967aaa800 0x55c9677cf600 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.756+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv42 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x55c967a42b40 con 0x55c967aaa800 2023-09-23T03:18:01.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.756+0000 7f2f8f4435c0 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55c967aaa400 0x55c9677d0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.756+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:2 5.1f 5:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c9677bfc00 con 0x55c967aaa400 2023-09-23T03:18:01.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.757+0000 7f2f8dcaf640 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55c967aaa400 0x55c9677d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.758+0000 7f2f8dcaf640 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55c967aaa400 0x55c9677d0100 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.759+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x55c967a42d80 con 0x55c967aaa400 2023-09-23T03:18:01.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.759+0000 7f2f8f4435c0 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55c967aaa000 0x55c9677cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.759+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:3 6.0 6:00000000::::head [pgnls-filter start_epoch 43 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bca000 con 0x55c967aaa000 2023-09-23T03:18:01.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.759+0000 7f2f8d4ae640 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55c967aaa000 0x55c9677cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.759+0000 7f2f8d4ae640 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55c967aaa000 0x55c9677cfb80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.760+0000 7f2f8d4ae640 1 -- 172.21.15.154:0/2871567064 <== osd.0 v2:172.21.15.154:6800/2920938138 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 43 out=49b] v41'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42fc0 con 0x55c967aaa000 2023-09-23T03:18:01.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.760+0000 7f2f884a4640 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55c967ab9000 0x55c967bd8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.760+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:4 6.20 6:04000000::::head [pgnls-filter start_epoch 43 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bcae00 con 0x55c967ab9000 2023-09-23T03:18:01.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.760+0000 7f2f8ccad640 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55c967ab9000 0x55c967bd8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.761+0000 7f2f8ccad640 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55c967ab9000 0x55c967bd8000 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.761+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.1 v2:172.21.15.154:6816/1267710005 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 43 out=49b] v41'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42b40 con 0x55c967ab9000 2023-09-23T03:18:01.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.761+0000 7f2f88ca5640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:5 6.10 6:08000000::::head [pgnls-filter start_epoch 43 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bcb180 con 0x55c967aaa000 2023-09-23T03:18:01.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.762+0000 7f2f8d4ae640 1 -- 172.21.15.154:0/2871567064 <== osd.0 v2:172.21.15.154:6800/2920938138 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 43 out=49b] v41'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42fc0 con 0x55c967aaa000 2023-09-23T03:18:01.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.762+0000 7f2f884a4640 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55c967ab9400 0x55c967bd8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.762+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:6 6.30 6:0c000000::::head [pgnls-filter start_epoch 43 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bca000 con 0x55c967ab9400 2023-09-23T03:18:01.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.762+0000 7f2f8dcaf640 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55c967ab9400 0x55c967bd8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.763+0000 7f2f8dcaf640 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55c967ab9400 0x55c967bd8580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.764+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.7 v2:172.21.15.190:6824/3460136272 1 ==== osd_op_reply(6 [pgnls-filter start_epoch 43 out=49b] v41'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42d80 con 0x55c967ab9400 2023-09-23T03:18:01.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.764+0000 7f2f88ca5640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:7 6.8 6:10000000::::head [pgnls-filter start_epoch 43 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bca000 con 0x55c967ab9400 2023-09-23T03:18:01.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.765+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.7 v2:172.21.15.190:6824/3460136272 2 ==== osd_op_reply(7 [pgnls-filter start_epoch 43 out=49b] v41'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42d80 con 0x55c967ab9400 2023-09-23T03:18:01.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.765+0000 7f2f884a4640 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55c967ab9800 0x55c967bd8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.765+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:8 6.28 6:14000000::::head [pgnls-filter start_epoch 43 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bcb180 con 0x55c967ab9800 2023-09-23T03:18:01.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.765+0000 7f2f8d4ae640 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55c967ab9800 0x55c967bd8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.766+0000 7f2f8d4ae640 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55c967ab9800 0x55c967bd8b00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.766+0000 7f2f8d4ae640 1 -- 172.21.15.154:0/2871567064 <== osd.2 v2:172.21.15.154:6822/2884407243 1 ==== osd_op_reply(8 [pgnls-filter start_epoch 43 out=49b] v41'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42fc0 con 0x55c967ab9800 2023-09-23T03:18:01.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.766+0000 7f2f88ca5640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:9 6.18 6:18000000::::head [pgnls-filter start_epoch 43 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c9677bfc00 con 0x55c967aaa000 2023-09-23T03:18:01.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.767+0000 7f2f8d4ae640 1 -- 172.21.15.154:0/2871567064 <== osd.0 v2:172.21.15.154:6800/2920938138 3 ==== osd_op_reply(9 [pgnls-filter start_epoch 43 out=49b] v41'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42fc0 con 0x55c967aaa000 2023-09-23T03:18:01.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.767+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:10 6.38 6:1c000000::::head [pgnls-filter start_epoch 43 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bcb500 con 0x55c967aaa800 2023-09-23T03:18:01.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.767+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.3 v2:172.21.15.154:6808/1133688468 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 43 out=49b] v41'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42b40 con 0x55c967aaa800 2023-09-23T03:18:01.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.767+0000 7f2f88ca5640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:11 6.4 6:20000000::::head [pgnls-filter start_epoch 43 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bcae00 con 0x55c967ab9000 2023-09-23T03:18:01.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.767+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.1 v2:172.21.15.154:6816/1267710005 2 ==== osd_op_reply(11 [pgnls-filter start_epoch 43 out=49b] v41'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42b40 con 0x55c967ab9000 2023-09-23T03:18:01.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.768+0000 7f2f884a4640 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55c967ab9c00 0x55c967bd9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.768+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:12 6.24 6:24000000::::head [pgnls-filter start_epoch 43 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bcb880 con 0x55c967ab9c00 2023-09-23T03:18:01.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.768+0000 7f2f8ccad640 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55c967ab9c00 0x55c967bd9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.769+0000 7f2f8ccad640 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55c967ab9c00 0x55c967bd9080 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.770+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.5 v2:172.21.15.190:6800/882536885 1 ==== osd_op_reply(12 [pgnls-filter start_epoch 43 out=49b] v41'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42b40 con 0x55c967ab9c00 2023-09-23T03:18:01.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.770+0000 7f2f88ca5640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:13 6.14 6:28000000::::head [pgnls-filter start_epoch 43 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bcb500 con 0x55c967ab9800 2023-09-23T03:18:01.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.771+0000 7f2f8d4ae640 1 -- 172.21.15.154:0/2871567064 <== osd.2 v2:172.21.15.154:6822/2884407243 2 ==== osd_op_reply(13 [pgnls-filter start_epoch 43 out=49b] v41'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42fc0 con 0x55c967ab9800 2023-09-23T03:18:01.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.771+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:14 6.34 6:2c000000::::head [pgnls-filter start_epoch 43 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bcbc00 con 0x55c967aaa000 2023-09-23T03:18:01.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.771+0000 7f2f8d4ae640 1 -- 172.21.15.154:0/2871567064 <== osd.0 v2:172.21.15.154:6800/2920938138 4 ==== osd_op_reply(14 [pgnls-filter start_epoch 43 out=49b] v41'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42fc0 con 0x55c967aaa000 2023-09-23T03:18:01.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.771+0000 7f2f88ca5640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:15 6.c 6:30000000::::head [pgnls-filter start_epoch 43 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c9677bf880 con 0x55c967aaa800 2023-09-23T03:18:01.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.772+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.3 v2:172.21.15.154:6808/1133688468 3 ==== osd_op_reply(15 [pgnls-filter start_epoch 43 out=49b] v41'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42b40 con 0x55c967aaa800 2023-09-23T03:18:01.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.772+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:16 6.2c 6:34000000::::head [pgnls-filter start_epoch 43 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bcb180 con 0x55c967aaa400 2023-09-23T03:18:01.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.773+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.6 v2:172.21.15.190:6816/1964294001 2 ==== osd_op_reply(16 [pgnls-filter start_epoch 43 out=49b] v41'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42d80 con 0x55c967aaa400 2023-09-23T03:18:01.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.773+0000 7f2f88ca5640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:17 6.1c 6:38000000::::head [pgnls-filter start_epoch 43 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf2000 con 0x55c967ab9400 2023-09-23T03:18:01.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.774+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.7 v2:172.21.15.190:6824/3460136272 3 ==== osd_op_reply(17 [pgnls-filter start_epoch 43 out=49b] v41'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42d80 con 0x55c967ab9400 2023-09-23T03:18:01.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.775+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:18 6.3c 6:3c000000::::head [pgnls-filter start_epoch 43 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bcb500 con 0x55c967ab9800 2023-09-23T03:18:01.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.775+0000 7f2f8d4ae640 1 -- 172.21.15.154:0/2871567064 <== osd.2 v2:172.21.15.154:6822/2884407243 3 ==== osd_op_reply(18 [pgnls-filter start_epoch 43 out=49b] v41'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42fc0 con 0x55c967ab9800 2023-09-23T03:18:01.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.775+0000 7f2f88ca5640 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55c967bf6000 0x55c967bda100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:01.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.775+0000 7f2f88ca5640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:19 6.2 6:40000000::::head [pgnls-filter start_epoch 43 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf2380 con 0x55c967bf6000 2023-09-23T03:18:01.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.776+0000 7f2f8dcaf640 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55c967bf6000 0x55c967bda100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:01.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.777+0000 7f2f8dcaf640 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55c967bf6000 0x55c967bda100 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:01.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.778+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.4 v2:172.21.15.190:6808/717302641 1 ==== osd_op_reply(19 [pgnls-filter start_epoch 43 out=49b] v41'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42d80 con 0x55c967bf6000 2023-09-23T03:18:01.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.778+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:20 6.22 6:44000000::::head [pgnls-filter start_epoch 43 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c9677bfc00 con 0x55c967aaa400 2023-09-23T03:18:01.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.779+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.6 v2:172.21.15.190:6816/1964294001 3 ==== osd_op_reply(20 [pgnls-filter start_epoch 43 out=49b] v41'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42d80 con 0x55c967aaa400 2023-09-23T03:18:01.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.779+0000 7f2f88ca5640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:21 6.12 6:48000000::::head [pgnls-filter start_epoch 43 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf2700 con 0x55c967ab9400 2023-09-23T03:18:01.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.780+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.7 v2:172.21.15.190:6824/3460136272 4 ==== osd_op_reply(21 [pgnls-filter start_epoch 43 out=49b] v41'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42d80 con 0x55c967ab9400 2023-09-23T03:18:01.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.780+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:22 6.32 6:4c000000::::head [pgnls-filter start_epoch 43 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf2000 con 0x55c967bf6000 2023-09-23T03:18:01.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.781+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.4 v2:172.21.15.190:6808/717302641 2 ==== osd_op_reply(22 [pgnls-filter start_epoch 43 out=49b] v41'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42d80 con 0x55c967bf6000 2023-09-23T03:18:01.782 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.781+0000 7f2f88ca5640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:23 6.a 6:50000000::::head [pgnls-filter start_epoch 43 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf2a80 con 0x55c967ab9c00 2023-09-23T03:18:01.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.782+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.5 v2:172.21.15.190:6800/882536885 2 ==== osd_op_reply(23 [pgnls-filter start_epoch 43 out=49b] v41'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42b40 con 0x55c967ab9c00 2023-09-23T03:18:01.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.783+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:24 6.2a 6:54000000::::head [pgnls-filter start_epoch 43 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf2e00 con 0x55c967bf6000 2023-09-23T03:18:01.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.784+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.4 v2:172.21.15.190:6808/717302641 3 ==== osd_op_reply(24 [pgnls-filter start_epoch 43 out=81b] v43'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c967a42d80 con 0x55c967bf6000 2023-09-23T03:18:01.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.784+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:25 6.0 6:00000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bca380 con 0x55c967aaa000 2023-09-23T03:18:01.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.785+0000 7f2f8d4ae640 1 -- 172.21.15.154:0/2871567064 <== osd.0 v2:172.21.15.154:6800/2920938138 5 ==== osd_op_reply(25 [pgnls-filter start_epoch 43 out=49b] v41'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42fc0 con 0x55c967aaa000 2023-09-23T03:18:01.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.785+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:26 6.20 6:04000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf3180 con 0x55c967ab9000 2023-09-23T03:18:01.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.785+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.1 v2:172.21.15.154:6816/1267710005 3 ==== osd_op_reply(26 [pgnls-filter start_epoch 43 out=49b] v41'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42b40 con 0x55c967ab9000 2023-09-23T03:18:01.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.785+0000 7f2f88ca5640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:27 6.10 6:08000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf2a80 con 0x55c967aaa000 2023-09-23T03:18:01.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.786+0000 7f2f8d4ae640 1 -- 172.21.15.154:0/2871567064 <== osd.0 v2:172.21.15.154:6800/2920938138 6 ==== osd_op_reply(27 [pgnls-filter start_epoch 43 out=49b] v41'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42fc0 con 0x55c967aaa000 2023-09-23T03:18:01.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.786+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:28 6.30 6:0c000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bca000 con 0x55c967ab9400 2023-09-23T03:18:01.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.787+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.7 v2:172.21.15.190:6824/3460136272 5 ==== osd_op_reply(28 [pgnls-filter start_epoch 43 out=49b] v41'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42d80 con 0x55c967ab9400 2023-09-23T03:18:01.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.787+0000 7f2f88ca5640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:29 6.8 6:10000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c9677bf880 con 0x55c967ab9400 2023-09-23T03:18:01.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.788+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.7 v2:172.21.15.190:6824/3460136272 6 ==== osd_op_reply(29 [pgnls-filter start_epoch 43 out=49b] v41'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42d80 con 0x55c967ab9400 2023-09-23T03:18:01.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.789+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:30 6.28 6:14000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bcb180 con 0x55c967ab9800 2023-09-23T03:18:01.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.789+0000 7f2f8d4ae640 1 -- 172.21.15.154:0/2871567064 <== osd.2 v2:172.21.15.154:6822/2884407243 4 ==== osd_op_reply(30 [pgnls-filter start_epoch 43 out=49b] v41'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42fc0 con 0x55c967ab9800 2023-09-23T03:18:01.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.789+0000 7f2f88ca5640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:31 6.18 6:18000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c9677bfc00 con 0x55c967aaa000 2023-09-23T03:18:01.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.790+0000 7f2f8d4ae640 1 -- 172.21.15.154:0/2871567064 <== osd.0 v2:172.21.15.154:6800/2920938138 7 ==== osd_op_reply(31 [pgnls-filter start_epoch 43 out=49b] v41'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42fc0 con 0x55c967aaa000 2023-09-23T03:18:01.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.790+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:32 6.38 6:1c000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf2a80 con 0x55c967aaa800 2023-09-23T03:18:01.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.790+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.3 v2:172.21.15.154:6808/1133688468 4 ==== osd_op_reply(32 [pgnls-filter start_epoch 43 out=49b] v41'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42b40 con 0x55c967aaa800 2023-09-23T03:18:01.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.790+0000 7f2f88ca5640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:33 6.4 6:20000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bcae00 con 0x55c967ab9000 2023-09-23T03:18:01.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.791+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.1 v2:172.21.15.154:6816/1267710005 4 ==== osd_op_reply(33 [pgnls-filter start_epoch 43 out=49b] v41'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42b40 con 0x55c967ab9000 2023-09-23T03:18:01.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.791+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:34 6.24 6:24000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf2380 con 0x55c967ab9c00 2023-09-23T03:18:01.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.792+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.5 v2:172.21.15.190:6800/882536885 3 ==== osd_op_reply(34 [pgnls-filter start_epoch 43 out=49b] v41'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42b40 con 0x55c967ab9c00 2023-09-23T03:18:01.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.792+0000 7f2f88ca5640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:35 6.14 6:28000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bcb880 con 0x55c967ab9800 2023-09-23T03:18:01.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.793+0000 7f2f8d4ae640 1 -- 172.21.15.154:0/2871567064 <== osd.2 v2:172.21.15.154:6822/2884407243 5 ==== osd_op_reply(35 [pgnls-filter start_epoch 43 out=49b] v41'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42fc0 con 0x55c967ab9800 2023-09-23T03:18:01.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.793+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:36 6.34 6:2c000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf2700 con 0x55c967aaa000 2023-09-23T03:18:01.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.793+0000 7f2f8d4ae640 1 -- 172.21.15.154:0/2871567064 <== osd.0 v2:172.21.15.154:6800/2920938138 8 ==== osd_op_reply(36 [pgnls-filter start_epoch 43 out=49b] v41'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42fc0 con 0x55c967aaa000 2023-09-23T03:18:01.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.793+0000 7f2f88ca5640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:37 6.c 6:30000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf2000 con 0x55c967aaa800 2023-09-23T03:18:01.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.794+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.3 v2:172.21.15.154:6808/1133688468 5 ==== osd_op_reply(37 [pgnls-filter start_epoch 43 out=49b] v41'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42b40 con 0x55c967aaa800 2023-09-23T03:18:01.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.794+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:38 6.2c 6:34000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bcbc00 con 0x55c967aaa400 2023-09-23T03:18:01.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.795+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.6 v2:172.21.15.190:6816/1964294001 4 ==== osd_op_reply(38 [pgnls-filter start_epoch 43 out=49b] v41'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42d80 con 0x55c967aaa400 2023-09-23T03:18:01.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.796+0000 7f2f88ca5640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:39 6.1c 6:38000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf3500 con 0x55c967ab9400 2023-09-23T03:18:01.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.797+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.7 v2:172.21.15.190:6824/3460136272 7 ==== osd_op_reply(39 [pgnls-filter start_epoch 43 out=49b] v41'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42d80 con 0x55c967ab9400 2023-09-23T03:18:01.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.797+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:40 6.3c 6:3c000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bcb500 con 0x55c967ab9800 2023-09-23T03:18:01.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.797+0000 7f2f8d4ae640 1 -- 172.21.15.154:0/2871567064 <== osd.2 v2:172.21.15.154:6822/2884407243 6 ==== osd_op_reply(40 [pgnls-filter start_epoch 43 out=49b] v41'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42fc0 con 0x55c967ab9800 2023-09-23T03:18:01.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.797+0000 7f2f88ca5640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:41 6.2 6:40000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf3880 con 0x55c967bf6000 2023-09-23T03:18:01.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.799+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.4 v2:172.21.15.190:6808/717302641 4 ==== osd_op_reply(41 [pgnls-filter start_epoch 43 out=49b] v41'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42d80 con 0x55c967bf6000 2023-09-23T03:18:01.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.799+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:42 6.22 6:44000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bca380 con 0x55c967aaa400 2023-09-23T03:18:01.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.800+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.6 v2:172.21.15.190:6816/1964294001 5 ==== osd_op_reply(42 [pgnls-filter start_epoch 43 out=49b] v41'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42d80 con 0x55c967aaa400 2023-09-23T03:18:01.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.800+0000 7f2f88ca5640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:43 6.12 6:48000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf3c00 con 0x55c967ab9400 2023-09-23T03:18:01.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.801+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.7 v2:172.21.15.190:6824/3460136272 8 ==== osd_op_reply(43 [pgnls-filter start_epoch 43 out=49b] v41'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42d80 con 0x55c967ab9400 2023-09-23T03:18:01.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.801+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:44 6.32 6:4c000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bcb180 con 0x55c967bf6000 2023-09-23T03:18:01.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.802+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.4 v2:172.21.15.190:6808/717302641 5 ==== osd_op_reply(44 [pgnls-filter start_epoch 43 out=49b] v41'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42d80 con 0x55c967bf6000 2023-09-23T03:18:01.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.802+0000 7f2f88ca5640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:45 6.a 6:50000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf2700 con 0x55c967ab9c00 2023-09-23T03:18:01.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.804+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.5 v2:172.21.15.190:6800/882536885 4 ==== osd_op_reply(45 [pgnls-filter start_epoch 43 out=49b] v41'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42b40 con 0x55c967ab9c00 2023-09-23T03:18:01.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.804+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:46 6.2a 6:54000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bcb880 con 0x55c967bf6000 2023-09-23T03:18:01.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.805+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.4 v2:172.21.15.190:6808/717302641 6 ==== osd_op_reply(46 [pgnls-filter start_epoch 43 out=81b] v43'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c967a42d80 con 0x55c967bf6000 2023-09-23T03:18:01.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.805+0000 7f2f8f4435c0 20 datascan.forall_objects: OSD matched oid 100000001f6.00000000 2023-09-23T03:18:01.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.805+0000 7f2f8f4435c0 10 datascan.operator(): handling object 100000001f6.0 2023-09-23T03:18:01.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.805+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:47 6.2a 6:57fe7535:::100000001f6.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bca700 con 0x55c967bf6000 2023-09-23T03:18:01.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.806+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.4 v2:172.21.15.190:6808/717302641 7 ==== osd_op_reply(47 100000001f6.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x55c967a42d80 con 0x55c967bf6000 2023-09-23T03:18:01.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.807+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:48 7.a 7:57fe7535:::100000001f6.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf3880 con 0x55c967aaa800 2023-09-23T03:18:01.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.807+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.3 v2:172.21.15.154:6808/1133688468 6 ==== osd_op_reply(48 100000001f6.00000000 [getxattr (30) out=30b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55c967a42b40 con 0x55c967aaa800 2023-09-23T03:18:01.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.807+0000 7f2f8f4435c0 20 datascan.operator(): loaded layout from xattr: pi: 7 os: 4194304 sc: 1 su: 4194304 2023-09-23T03:18:01.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.807+0000 7f2f8f4435c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f6 2023-09-23T03:18:01.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.807+0000 7f2f8f4435c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/1 2023-09-23T03:18:01.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.807+0000 7f2f8f4435c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=1 2023-09-23T03:18:01.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.807+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:49 5.10 5:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bca380 con 0x55c967ab9400 2023-09-23T03:18:01.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.809+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.7 v2:172.21.15.190:6824/3460136272 9 ==== osd_op_reply(49 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x55c967a42d80 con 0x55c967ab9400 2023-09-23T03:18:01.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.809+0000 7f2f8f4435c0 4 datascan.get_frag_of: Unexpected error on '10000000000.00000000': (2) No such file or directory 2023-09-23T03:18:01.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.809+0000 7f2f8f4435c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/1, will insert to root 2023-09-23T03:18:01.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.809+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:50 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bcb180 con 0x55c967ab9400 2023-09-23T03:18:01.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.810+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.7 v2:172.21.15.190:6824/3460136272 10 ==== osd_op_reply(50 10000000000.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x55c967a42d80 con 0x55c967ab9400 2023-09-23T03:18:01.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.810+0000 7f2f8f4435c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-09-23T03:18:01.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.810+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:51 5.10 5:0cf6ad0f:::10000000000.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bcb880 con 0x55c967ab9400 2023-09-23T03:18:01.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.814+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.7 v2:172.21.15.190:6824/3460136272 11 ==== osd_op_reply(51 10000000000.00000000 [create,omap-set-header] v43'16 uv16 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55c967a42d80 con 0x55c967ab9400 2023-09-23T03:18:01.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.814+0000 7f2f8f4435c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x10000000000 2023-09-23T03:18:01.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.814+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:52 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bcaa80 con 0x55c967ab9400 2023-09-23T03:18:01.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.815+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.7 v2:172.21.15.190:6824/3460136272 12 ==== osd_op_reply(52 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55c967a42d80 con 0x55c967ab9400 2023-09-23T03:18:01.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.815+0000 7f2f8f4435c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=1 frag=*, r=0 2023-09-23T03:18:01.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.815+0000 7f2f8f4435c0 20 datascan.read_dentry: 1_head not found in result 2023-09-23T03:18:01.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.815+0000 7f2f8f4435c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f6 at 0x0x10000000000/1 with size=6291456 bytes 2023-09-23T03:18:01.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.815+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:53 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf3c00 con 0x55c967ab9400 2023-09-23T03:18:01.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.819+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.7 v2:172.21.15.190:6824/3460136272 13 ==== osd_op_reply(53 10000000000.00000000 [omap-set-vals] v43'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c967a42d80 con 0x55c967ab9400 2023-09-23T03:18:01.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.819+0000 7f2f8f4435c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/1 pointing to 0x0x100000001f6 2023-09-23T03:18:01.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.819+0000 7f2f8f4435c0 10 datascan.inject_with_backtrace: backptr: 0x0x1/subdir 2023-09-23T03:18:01.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.819+0000 7f2f8f4435c0 20 datascan.get_frag_of: dirino=0x1 target_dname=subdir 2023-09-23T03:18:01.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.819+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:54 5.1f 5:ff5b34d6:::1.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf2000 con 0x55c967aaa400 2023-09-23T03:18:01.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.820+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.6 v2:172.21.15.190:6816/1964294001 6 ==== osd_op_reply(54 1.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x55c967a42d80 con 0x55c967aaa400 2023-09-23T03:18:01.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.820+0000 7f2f8f4435c0 4 datascan.get_frag_of: Unexpected error on '1.00000000': (2) No such file or directory 2023-09-23T03:18:01.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.820+0000 7f2f8f4435c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x1/subdir, will insert to root 2023-09-23T03:18:01.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.820+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:55 5.1f 5:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf3c00 con 0x55c967aaa400 2023-09-23T03:18:01.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.821+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.6 v2:172.21.15.190:6816/1964294001 7 ==== osd_op_reply(55 1.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x55c967a42d80 con 0x55c967aaa400 2023-09-23T03:18:01.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.821+0000 7f2f8f4435c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-09-23T03:18:01.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.821+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:56 5.1f 5:ff5b34d6:::1.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf2000 con 0x55c967aaa400 2023-09-23T03:18:01.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.826+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.6 v2:172.21.15.190:6816/1964294001 8 ==== osd_op_reply(56 1.00000000 [create,omap-set-header] v43'11 uv11 ondisk = 0) v8 ==== 196+0+0 (crc 0 0 0) 0x55c967a42d80 con 0x55c967aaa400 2023-09-23T03:18:01.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.826+0000 7f2f8f4435c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x1 2023-09-23T03:18:01.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.826+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:57 5.1f 5:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bca700 con 0x55c967aaa400 2023-09-23T03:18:01.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.827+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.6 v2:172.21.15.190:6816/1964294001 9 ==== osd_op_reply(57 1.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv11 ondisk = 0) v8 ==== 154+0+4 (crc 0 0 0) 0x55c967a42d80 con 0x55c967aaa400 2023-09-23T03:18:01.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.827+0000 7f2f8f4435c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:18:01.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.827+0000 7f2f8f4435c0 20 datascan.read_dentry: subdir_head not found in result 2023-09-23T03:18:01.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.827+0000 7f2f8f4435c0 10 datascan.inject_with_backtrace: Linking ancestor directory of inode 0x0x10000000000 at 0x0x1:subdir 2023-09-23T03:18:01.828 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.827+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:58 5.1f 5:ff5b34d6:::1.00000000:head [omap-set-vals in=507b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bca380 con 0x55c967aaa400 2023-09-23T03:18:01.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.831+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.6 v2:172.21.15.190:6816/1964294001 10 ==== osd_op_reply(58 1.00000000 [omap-set-vals] v43'12 uv12 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x55c967a42d80 con 0x55c967aaa400 2023-09-23T03:18:01.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.831+0000 7f2f8f4435c0 20 datascan.inject_linkage: Injected dentry 0x0x1/subdir pointing to 0x0x10000000000 2023-09-23T03:18:01.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.831+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:59 6.1a 6:58000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bcb180 con 0x55c967bf6000 2023-09-23T03:18:01.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.832+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.4 v2:172.21.15.190:6808/717302641 8 ==== osd_op_reply(59 [pgnls-filter start_epoch 43 out=49b] v41'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42d80 con 0x55c967bf6000 2023-09-23T03:18:01.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.832+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:60 6.3a 6:5c000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bcb500 con 0x55c967ab9000 2023-09-23T03:18:01.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.833+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.1 v2:172.21.15.154:6816/1267710005 5 ==== osd_op_reply(60 [pgnls-filter start_epoch 43 out=49b] v41'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42b40 con 0x55c967ab9000 2023-09-23T03:18:01.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.833+0000 7f2f88ca5640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:61 6.6 6:60000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf2e00 con 0x55c967aaa800 2023-09-23T03:18:01.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.834+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.3 v2:172.21.15.154:6808/1133688468 7 ==== osd_op_reply(61 [pgnls-filter start_epoch 43 out=49b] v41'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42b40 con 0x55c967aaa800 2023-09-23T03:18:01.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.834+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:62 6.26 6:64000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bca000 con 0x55c967ab9000 2023-09-23T03:18:01.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.835+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.1 v2:172.21.15.154:6816/1267710005 6 ==== osd_op_reply(62 [pgnls-filter start_epoch 43 out=49b] v41'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42b40 con 0x55c967ab9000 2023-09-23T03:18:01.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.835+0000 7f2f88ca5640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:63 6.16 6:68000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf3500 con 0x55c967aaa000 2023-09-23T03:18:01.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.836+0000 7f2f8d4ae640 1 -- 172.21.15.154:0/2871567064 <== osd.0 v2:172.21.15.154:6800/2920938138 9 ==== osd_op_reply(63 [pgnls-filter start_epoch 43 out=49b] v41'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42fc0 con 0x55c967aaa000 2023-09-23T03:18:01.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.836+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:64 6.36 6:6c000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bcbc00 con 0x55c967ab9c00 2023-09-23T03:18:01.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.838+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.5 v2:172.21.15.190:6800/882536885 5 ==== osd_op_reply(64 [pgnls-filter start_epoch 43 out=101b] v43'23 uv23 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55c967a42b40 con 0x55c967ab9c00 2023-09-23T03:18:01.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.838+0000 7f2f8f4435c0 20 datascan.forall_objects: OSD matched oid 100000001f9.00000000 2023-09-23T03:18:01.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.838+0000 7f2f8f4435c0 10 datascan.operator(): handling object 100000001f9.0 2023-09-23T03:18:01.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.838+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:65 6.36 6:6ee5e2c4:::100000001f9.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bcb880 con 0x55c967ab9c00 2023-09-23T03:18:01.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.839+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.5 v2:172.21.15.190:6800/882536885 6 ==== osd_op_reply(65 100000001f9.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv21 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x55c967a42b40 con 0x55c967ab9c00 2023-09-23T03:18:01.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.840+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:66 7.16 7:6ee5e2c4:::100000001f9.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bcb500 con 0x55c967bf6000 2023-09-23T03:18:01.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.841+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.4 v2:172.21.15.190:6808/717302641 9 ==== osd_op_reply(66 100000001f9.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55c967a42d80 con 0x55c967bf6000 2023-09-23T03:18:01.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.841+0000 7f2f8f4435c0 20 datascan.operator(): loaded layout from xattr: pi: 7 os: 4194304 sc: 1 su: 4194304 2023-09-23T03:18:01.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.841+0000 7f2f8f4435c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f9 2023-09-23T03:18:01.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.841+0000 7f2f8f4435c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/4 2023-09-23T03:18:01.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.841+0000 7f2f8f4435c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=4 2023-09-23T03:18:01.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.841+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:67 5.10 5:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bca000 con 0x55c967ab9400 2023-09-23T03:18:01.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.842+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.7 v2:172.21.15.190:6824/3460136272 14 ==== osd_op_reply(67 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55c967a42d80 con 0x55c967ab9400 2023-09-23T03:18:01.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.842+0000 7f2f8f4435c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-09-23T03:18:01.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.842+0000 7f2f8f4435c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-09-23T03:18:01.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.842+0000 7f2f8f4435c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/4, will insert to root 2023-09-23T03:18:01.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.842+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:68 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bcbc00 con 0x55c967ab9400 2023-09-23T03:18:01.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.843+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.7 v2:172.21.15.190:6824/3460136272 15 ==== osd_op_reply(68 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55c967a42d80 con 0x55c967ab9400 2023-09-23T03:18:01.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.844+0000 7f2f8f4435c0 10 datascan.find_or_create_dirfrag: read_version = 17 2023-09-23T03:18:01.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.844+0000 7f2f8f4435c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-09-23T03:18:01.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.844+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:69 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf2700 con 0x55c967ab9400 2023-09-23T03:18:01.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.845+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.7 v2:172.21.15.190:6824/3460136272 16 ==== osd_op_reply(69 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55c967a42d80 con 0x55c967ab9400 2023-09-23T03:18:01.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.845+0000 7f2f8f4435c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=4 frag=*, r=0 2023-09-23T03:18:01.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.845+0000 7f2f8f4435c0 20 datascan.read_dentry: 4_head not found in result 2023-09-23T03:18:01.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.845+0000 7f2f8f4435c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f9 at 0x0x10000000000/4 with size=6291456 bytes 2023-09-23T03:18:01.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.845+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:70 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf3880 con 0x55c967ab9400 2023-09-23T03:18:01.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.850+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.7 v2:172.21.15.190:6824/3460136272 17 ==== osd_op_reply(70 10000000000.00000000 [omap-set-vals] v43'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c967a42d80 con 0x55c967ab9400 2023-09-23T03:18:01.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.851+0000 7f2f8f4435c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/4 pointing to 0x0x100000001f9 2023-09-23T03:18:01.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.851+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:71 6.36 6:6fe38dc2::::head [pgnls-filter start_epoch 43 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf2e00 con 0x55c967ab9c00 2023-09-23T03:18:01.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.852+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.5 v2:172.21.15.190:6800/882536885 7 ==== osd_op_reply(71 [pgnls-filter start_epoch 43 out=81b] v43'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c967a42b40 con 0x55c967ab9c00 2023-09-23T03:18:01.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.852+0000 7f2f8f4435c0 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2023-09-23T03:18:01.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.852+0000 7f2f8f4435c0 10 datascan.operator(): handling object 100000001f7.0 2023-09-23T03:18:01.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.852+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:72 6.36 6:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bca700 con 0x55c967ab9c00 2023-09-23T03:18:01.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.854+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.5 v2:172.21.15.190:6800/882536885 8 ==== osd_op_reply(72 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv23 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x55c967a42b40 con 0x55c967ab9c00 2023-09-23T03:18:01.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.854+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:73 7.16 7:6fe38dc2:::100000001f7.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bcb180 con 0x55c967bf6000 2023-09-23T03:18:01.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.855+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.4 v2:172.21.15.190:6808/717302641 10 ==== osd_op_reply(73 100000001f7.00000000 [getxattr (30) out=30b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55c967a42d80 con 0x55c967bf6000 2023-09-23T03:18:01.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.855+0000 7f2f8f4435c0 20 datascan.operator(): loaded layout from xattr: pi: 7 os: 4194304 sc: 1 su: 4194304 2023-09-23T03:18:01.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.855+0000 7f2f8f4435c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2023-09-23T03:18:01.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.855+0000 7f2f8f4435c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/2 2023-09-23T03:18:01.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.855+0000 7f2f8f4435c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=2 2023-09-23T03:18:01.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.855+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:74 5.10 5:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bcb500 con 0x55c967ab9400 2023-09-23T03:18:01.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.856+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.7 v2:172.21.15.190:6824/3460136272 18 ==== osd_op_reply(74 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55c967a42d80 con 0x55c967ab9400 2023-09-23T03:18:01.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.856+0000 7f2f8f4435c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-09-23T03:18:01.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.856+0000 7f2f8f4435c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-09-23T03:18:01.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.856+0000 7f2f8f4435c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/2, will insert to root 2023-09-23T03:18:01.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.856+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:75 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bca000 con 0x55c967ab9400 2023-09-23T03:18:01.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.858+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.7 v2:172.21.15.190:6824/3460136272 19 ==== osd_op_reply(75 10000000000.00000000 [omap-get-header out=274b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55c967a42d80 con 0x55c967ab9400 2023-09-23T03:18:01.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.858+0000 7f2f8f4435c0 10 datascan.find_or_create_dirfrag: read_version = 18 2023-09-23T03:18:01.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.858+0000 7f2f8f4435c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-09-23T03:18:01.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.858+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:76 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bcbc00 con 0x55c967ab9400 2023-09-23T03:18:01.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.859+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.7 v2:172.21.15.190:6824/3460136272 20 ==== osd_op_reply(76 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55c967a42d80 con 0x55c967ab9400 2023-09-23T03:18:01.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.859+0000 7f2f8f4435c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=2 frag=*, r=0 2023-09-23T03:18:01.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.859+0000 7f2f8f4435c0 20 datascan.read_dentry: 2_head not found in result 2023-09-23T03:18:01.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.859+0000 7f2f8f4435c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000000/2 with size=6291456 bytes 2023-09-23T03:18:01.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.859+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:77 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf3c00 con 0x55c967ab9400 2023-09-23T03:18:01.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.862+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.7 v2:172.21.15.190:6824/3460136272 21 ==== osd_op_reply(77 10000000000.00000000 [omap-set-vals] v43'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c967a42d80 con 0x55c967ab9400 2023-09-23T03:18:01.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.863+0000 7f2f8f4435c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/2 pointing to 0x0x100000001f7 2023-09-23T03:18:01.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.863+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:78 6.e 6:70000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf2000 con 0x55c967bf6000 2023-09-23T03:18:01.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.864+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.4 v2:172.21.15.190:6808/717302641 11 ==== osd_op_reply(78 [pgnls-filter start_epoch 43 out=49b] v41'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42d80 con 0x55c967bf6000 2023-09-23T03:18:01.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.864+0000 7f2f88ca5640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:79 6.2e 6:74000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf3180 con 0x55c967aaa400 2023-09-23T03:18:01.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.865+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.6 v2:172.21.15.190:6816/1964294001 11 ==== osd_op_reply(79 [pgnls-filter start_epoch 43 out=49b] v41'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42d80 con 0x55c967aaa400 2023-09-23T03:18:01.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.865+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:80 6.1e 6:78000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c9677bfc00 con 0x55c967bf6000 2023-09-23T03:18:01.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.867+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.4 v2:172.21.15.190:6808/717302641 12 ==== osd_op_reply(80 [pgnls-filter start_epoch 43 out=49b] v41'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42d80 con 0x55c967bf6000 2023-09-23T03:18:01.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.867+0000 7f2f88ca5640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:81 6.3e 6:7c000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf2a80 con 0x55c967aaa800 2023-09-23T03:18:01.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.868+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.3 v2:172.21.15.154:6808/1133688468 8 ==== osd_op_reply(81 [pgnls-filter start_epoch 43 out=49b] v41'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42b40 con 0x55c967aaa800 2023-09-23T03:18:01.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.868+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:82 6.1 6:80000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c9677bf880 con 0x55c967ab9000 2023-09-23T03:18:01.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.869+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.1 v2:172.21.15.154:6816/1267710005 7 ==== osd_op_reply(82 [pgnls-filter start_epoch 43 out=49b] v41'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42b40 con 0x55c967ab9000 2023-09-23T03:18:01.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.869+0000 7f2f88ca5640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:83 6.21 6:84000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf2380 con 0x55c967ab9400 2023-09-23T03:18:01.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.870+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.7 v2:172.21.15.190:6824/3460136272 22 ==== osd_op_reply(83 [pgnls-filter start_epoch 43 out=49b] v41'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42d80 con 0x55c967ab9400 2023-09-23T03:18:01.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.870+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:84 6.11 6:88000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bcaa80 con 0x55c967aaa800 2023-09-23T03:18:01.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.871+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.3 v2:172.21.15.154:6808/1133688468 9 ==== osd_op_reply(84 [pgnls-filter start_epoch 43 out=49b] v41'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42b40 con 0x55c967aaa800 2023-09-23T03:18:01.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.871+0000 7f2f88ca5640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:85 6.31 6:8c000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bcae00 con 0x55c967ab9c00 2023-09-23T03:18:01.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.872+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.5 v2:172.21.15.190:6800/882536885 9 ==== osd_op_reply(85 [pgnls-filter start_epoch 43 out=49b] v41'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42b40 con 0x55c967ab9c00 2023-09-23T03:18:01.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.872+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:86 6.9 6:90000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bcb880 con 0x55c967aaa000 2023-09-23T03:18:01.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.873+0000 7f2f8d4ae640 1 -- 172.21.15.154:0/2871567064 <== osd.0 v2:172.21.15.154:6800/2920938138 10 ==== osd_op_reply(86 [pgnls-filter start_epoch 43 out=49b] v41'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42fc0 con 0x55c967aaa000 2023-09-23T03:18:01.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.873+0000 7f2f88ca5640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:87 6.29 6:94000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf2e00 con 0x55c967ab9400 2023-09-23T03:18:01.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.874+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.7 v2:172.21.15.190:6824/3460136272 23 ==== osd_op_reply(87 [pgnls-filter start_epoch 43 out=49b] v41'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42d80 con 0x55c967ab9400 2023-09-23T03:18:01.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.874+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:88 6.19 6:98000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bca380 con 0x55c967ab9c00 2023-09-23T03:18:01.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.876+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.5 v2:172.21.15.190:6800/882536885 10 ==== osd_op_reply(88 [pgnls-filter start_epoch 43 out=81b] v43'14 uv14 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c967a42b40 con 0x55c967ab9c00 2023-09-23T03:18:01.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.876+0000 7f2f8f4435c0 20 datascan.forall_objects: OSD matched oid 100000001f8.00000000 2023-09-23T03:18:01.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.876+0000 7f2f8f4435c0 10 datascan.operator(): handling object 100000001f8.0 2023-09-23T03:18:01.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.876+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:89 6.19 6:997b7129:::100000001f8.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967c20000 con 0x55c967ab9c00 2023-09-23T03:18:01.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.878+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.5 v2:172.21.15.190:6800/882536885 11 ==== osd_op_reply(89 100000001f8.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv14 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x55c967a42b40 con 0x55c967ab9c00 2023-09-23T03:18:01.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.878+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:90 7.19 7:997b7129:::100000001f8.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967c20700 con 0x55c967aaa400 2023-09-23T03:18:01.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.879+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.6 v2:172.21.15.190:6816/1964294001 12 ==== osd_op_reply(90 100000001f8.00000000 [getxattr (30) out=30b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55c967a42d80 con 0x55c967aaa400 2023-09-23T03:18:01.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.879+0000 7f2f8f4435c0 20 datascan.operator(): loaded layout from xattr: pi: 7 os: 4194304 sc: 1 su: 4194304 2023-09-23T03:18:01.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.879+0000 7f2f8f4435c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f8 2023-09-23T03:18:01.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.879+0000 7f2f8f4435c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/3 2023-09-23T03:18:01.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.879+0000 7f2f8f4435c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=3 2023-09-23T03:18:01.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.879+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:91 5.10 5:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967c20a80 con 0x55c967ab9400 2023-09-23T03:18:01.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.880+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.7 v2:172.21.15.190:6824/3460136272 24 ==== osd_op_reply(91 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55c967a42d80 con 0x55c967ab9400 2023-09-23T03:18:01.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.880+0000 7f2f8f4435c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-09-23T03:18:01.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.880+0000 7f2f8f4435c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-09-23T03:18:01.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.880+0000 7f2f8f4435c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/3, will insert to root 2023-09-23T03:18:01.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.880+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:92 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967c20e00 con 0x55c967ab9400 2023-09-23T03:18:01.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.882+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.7 v2:172.21.15.190:6824/3460136272 25 ==== osd_op_reply(92 10000000000.00000000 [omap-get-header out=274b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55c967a42d80 con 0x55c967ab9400 2023-09-23T03:18:01.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.882+0000 7f2f8f4435c0 10 datascan.find_or_create_dirfrag: read_version = 19 2023-09-23T03:18:01.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.882+0000 7f2f8f4435c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-09-23T03:18:01.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.882+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:93 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967c21180 con 0x55c967ab9400 2023-09-23T03:18:01.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.883+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.7 v2:172.21.15.190:6824/3460136272 26 ==== osd_op_reply(93 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55c967a42d80 con 0x55c967ab9400 2023-09-23T03:18:01.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.883+0000 7f2f8f4435c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=3 frag=*, r=0 2023-09-23T03:18:01.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.883+0000 7f2f8f4435c0 20 datascan.read_dentry: 3_head not found in result 2023-09-23T03:18:01.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.883+0000 7f2f8f4435c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f8 at 0x0x10000000000/3 with size=6291456 bytes 2023-09-23T03:18:01.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.883+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:94 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967c21500 con 0x55c967ab9400 2023-09-23T03:18:01.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.887+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.7 v2:172.21.15.190:6824/3460136272 27 ==== osd_op_reply(94 10000000000.00000000 [omap-set-vals] v43'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c967a42d80 con 0x55c967ab9400 2023-09-23T03:18:01.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.887+0000 7f2f8f4435c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/3 pointing to 0x0x100000001f8 2023-09-23T03:18:01.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.887+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:95 6.39 6:9c000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967c21880 con 0x55c967ab9c00 2023-09-23T03:18:01.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.888+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.5 v2:172.21.15.190:6800/882536885 12 ==== osd_op_reply(95 [pgnls-filter start_epoch 43 out=49b] v41'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42b40 con 0x55c967ab9c00 2023-09-23T03:18:01.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.888+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:96 6.5 6:a0000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bca700 con 0x55c967ab9400 2023-09-23T03:18:01.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.889+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.7 v2:172.21.15.190:6824/3460136272 28 ==== osd_op_reply(96 [pgnls-filter start_epoch 43 out=49b] v41'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42d80 con 0x55c967ab9400 2023-09-23T03:18:01.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.889+0000 7f2f88ca5640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:97 6.25 6:a4000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf2a80 con 0x55c967aaa400 2023-09-23T03:18:01.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.891+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.6 v2:172.21.15.190:6816/1964294001 13 ==== osd_op_reply(97 [pgnls-filter start_epoch 43 out=49b] v41'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42d80 con 0x55c967aaa400 2023-09-23T03:18:01.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.891+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:98 6.15 6:a8000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bcb180 con 0x55c967ab9400 2023-09-23T03:18:01.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.892+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.7 v2:172.21.15.190:6824/3460136272 29 ==== osd_op_reply(98 [pgnls-filter start_epoch 43 out=49b] v41'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42d80 con 0x55c967ab9400 2023-09-23T03:18:01.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.892+0000 7f2f88ca5640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:99 6.35 6:ac000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf2700 con 0x55c967ab9c00 2023-09-23T03:18:01.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.893+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.5 v2:172.21.15.190:6800/882536885 13 ==== osd_op_reply(99 [pgnls-filter start_epoch 43 out=49b] v41'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42b40 con 0x55c967ab9c00 2023-09-23T03:18:01.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.893+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:100 6.d 6:b0000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bcb500 con 0x55c967ab9c00 2023-09-23T03:18:01.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.894+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.5 v2:172.21.15.190:6800/882536885 14 ==== osd_op_reply(100 [pgnls-filter start_epoch 43 out=49b] v41'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42b40 con 0x55c967ab9c00 2023-09-23T03:18:01.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.894+0000 7f2f88ca5640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:101 6.2d 6:b4000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf3880 con 0x55c967ab9800 2023-09-23T03:18:01.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.895+0000 7f2f8d4ae640 1 -- 172.21.15.154:0/2871567064 <== osd.2 v2:172.21.15.154:6822/2884407243 7 ==== osd_op_reply(101 [pgnls-filter start_epoch 43 out=49b] v41'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42fc0 con 0x55c967ab9800 2023-09-23T03:18:01.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.895+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:102 6.1d 6:b8000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bca000 con 0x55c967ab9000 2023-09-23T03:18:01.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.896+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.1 v2:172.21.15.154:6816/1267710005 8 ==== osd_op_reply(102 [pgnls-filter start_epoch 43 out=49b] v41'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42b40 con 0x55c967ab9000 2023-09-23T03:18:01.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.896+0000 7f2f88ca5640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:103 6.3d 6:bc000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf3c00 con 0x55c967aaa000 2023-09-23T03:18:01.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.897+0000 7f2f8d4ae640 1 -- 172.21.15.154:0/2871567064 <== osd.0 v2:172.21.15.154:6800/2920938138 11 ==== osd_op_reply(103 [pgnls-filter start_epoch 43 out=49b] v41'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42fc0 con 0x55c967aaa000 2023-09-23T03:18:01.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.897+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:104 6.3 6:c0000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c9677bf880 con 0x55c967ab9400 2023-09-23T03:18:01.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.898+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.7 v2:172.21.15.190:6824/3460136272 30 ==== osd_op_reply(104 [pgnls-filter start_epoch 43 out=49b] v41'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42d80 con 0x55c967ab9400 2023-09-23T03:18:01.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.898+0000 7f2f88ca5640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:105 6.23 6:c4000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf2000 con 0x55c967aaa400 2023-09-23T03:18:01.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.900+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.6 v2:172.21.15.190:6816/1964294001 14 ==== osd_op_reply(105 [pgnls-filter start_epoch 43 out=49b] v41'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42d80 con 0x55c967aaa400 2023-09-23T03:18:01.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.900+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:106 6.13 6:c8000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967c20380 con 0x55c967aaa800 2023-09-23T03:18:01.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.901+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.3 v2:172.21.15.154:6808/1133688468 10 ==== osd_op_reply(106 [pgnls-filter start_epoch 43 out=49b] v41'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42b40 con 0x55c967aaa800 2023-09-23T03:18:01.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.901+0000 7f2f88ca5640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:107 6.33 6:cc000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf3180 con 0x55c967ab9c00 2023-09-23T03:18:01.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.902+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.5 v2:172.21.15.190:6800/882536885 15 ==== osd_op_reply(107 [pgnls-filter start_epoch 43 out=49b] v41'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42b40 con 0x55c967ab9c00 2023-09-23T03:18:01.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.902+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:108 6.b 6:d0000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967c20000 con 0x55c967aaa800 2023-09-23T03:18:01.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.903+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.3 v2:172.21.15.154:6808/1133688468 11 ==== osd_op_reply(108 [pgnls-filter start_epoch 43 out=49b] v41'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42b40 con 0x55c967aaa800 2023-09-23T03:18:01.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.903+0000 7f2f88ca5640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:109 6.2b 6:d4000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bcbc00 con 0x55c967ab9c00 2023-09-23T03:18:01.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.904+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.5 v2:172.21.15.190:6800/882536885 16 ==== osd_op_reply(109 [pgnls-filter start_epoch 43 out=49b] v41'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42b40 con 0x55c967ab9c00 2023-09-23T03:18:01.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.904+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:110 6.1b 6:d8000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967c21880 con 0x55c967aaa800 2023-09-23T03:18:01.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.905+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.3 v2:172.21.15.154:6808/1133688468 12 ==== osd_op_reply(110 [pgnls-filter start_epoch 43 out=49b] v41'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42b40 con 0x55c967aaa800 2023-09-23T03:18:01.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.905+0000 7f2f88ca5640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:111 6.3b 6:dc000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf2700 con 0x55c967ab9800 2023-09-23T03:18:01.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.906+0000 7f2f8d4ae640 1 -- 172.21.15.154:0/2871567064 <== osd.2 v2:172.21.15.154:6822/2884407243 8 ==== osd_op_reply(111 [pgnls-filter start_epoch 43 out=49b] v41'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42fc0 con 0x55c967ab9800 2023-09-23T03:18:01.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.906+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:112 6.7 6:e0000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967c20700 con 0x55c967ab9c00 2023-09-23T03:18:01.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.907+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.5 v2:172.21.15.190:6800/882536885 17 ==== osd_op_reply(112 [pgnls-filter start_epoch 43 out=81b] v43'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c967a42b40 con 0x55c967ab9c00 2023-09-23T03:18:01.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.907+0000 7f2f8f4435c0 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2023-09-23T03:18:01.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.907+0000 7f2f8f4435c0 10 datascan.operator(): handling object 10000000001.0 2023-09-23T03:18:01.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.907+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:113 6.7 6:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967c21c00 con 0x55c967ab9c00 2023-09-23T03:18:01.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.908+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.5 v2:172.21.15.190:6800/882536885 18 ==== osd_op_reply(113 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv9 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x55c967a42b40 con 0x55c967ab9c00 2023-09-23T03:18:01.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.909+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:114 7.7 7:e2e07930:::10000000001.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967c22380 con 0x55c967aaa400 2023-09-23T03:18:01.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.910+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.6 v2:172.21.15.190:6816/1964294001 15 ==== osd_op_reply(114 10000000001.00000000 [getxattr (30) out=30b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55c967a42d80 con 0x55c967aaa400 2023-09-23T03:18:01.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.910+0000 7f2f8f4435c0 20 datascan.operator(): loaded layout from xattr: pi: 7 os: 4194304 sc: 1 su: 4194304 2023-09-23T03:18:01.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.910+0000 7f2f8f4435c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2023-09-23T03:18:01.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.910+0000 7f2f8f4435c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/0 2023-09-23T03:18:01.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.910+0000 7f2f8f4435c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=0 2023-09-23T03:18:01.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.910+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:115 5.10 5:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967c22700 con 0x55c967ab9400 2023-09-23T03:18:01.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.911+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.7 v2:172.21.15.190:6824/3460136272 31 ==== osd_op_reply(115 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55c967a42d80 con 0x55c967ab9400 2023-09-23T03:18:01.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.911+0000 7f2f8f4435c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-09-23T03:18:01.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.911+0000 7f2f8f4435c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-09-23T03:18:01.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.911+0000 7f2f8f4435c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/0, will insert to root 2023-09-23T03:18:01.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.911+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:116 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967c22a80 con 0x55c967ab9400 2023-09-23T03:18:01.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.912+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.7 v2:172.21.15.190:6824/3460136272 32 ==== osd_op_reply(116 10000000000.00000000 [omap-get-header out=274b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55c967a42d80 con 0x55c967ab9400 2023-09-23T03:18:01.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.912+0000 7f2f8f4435c0 10 datascan.find_or_create_dirfrag: read_version = 20 2023-09-23T03:18:01.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.912+0000 7f2f8f4435c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-09-23T03:18:01.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.912+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:117 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967c22e00 con 0x55c967ab9400 2023-09-23T03:18:01.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.914+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.7 v2:172.21.15.190:6824/3460136272 33 ==== osd_op_reply(117 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55c967a42d80 con 0x55c967ab9400 2023-09-23T03:18:01.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.914+0000 7f2f8f4435c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=0 frag=*, r=0 2023-09-23T03:18:01.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.914+0000 7f2f8f4435c0 20 datascan.read_dentry: 0_head not found in result 2023-09-23T03:18:01.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.914+0000 7f2f8f4435c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/0 with size=6291456 bytes 2023-09-23T03:18:01.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.914+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:118 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967c23180 con 0x55c967ab9400 2023-09-23T03:18:01.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.918+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.7 v2:172.21.15.190:6824/3460136272 34 ==== osd_op_reply(118 10000000000.00000000 [omap-set-vals] v43'21 uv21 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c967a42d80 con 0x55c967ab9400 2023-09-23T03:18:01.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.918+0000 7f2f8f4435c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/0 pointing to 0x0x10000000001 2023-09-23T03:18:01.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.918+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:119 6.27 6:e4000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967c23500 con 0x55c967aaa400 2023-09-23T03:18:01.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.919+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.6 v2:172.21.15.190:6816/1964294001 16 ==== osd_op_reply(119 [pgnls-filter start_epoch 43 out=49b] v41'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42d80 con 0x55c967aaa400 2023-09-23T03:18:01.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.919+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:120 6.17 6:e8000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967c20a80 con 0x55c967bf6000 2023-09-23T03:18:01.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.920+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.4 v2:172.21.15.190:6808/717302641 13 ==== osd_op_reply(120 [pgnls-filter start_epoch 43 out=49b] v41'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42d80 con 0x55c967bf6000 2023-09-23T03:18:01.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.921+0000 7f2f88ca5640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:121 6.37 6:ec000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf2380 con 0x55c967aaa000 2023-09-23T03:18:01.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.921+0000 7f2f8d4ae640 1 -- 172.21.15.154:0/2871567064 <== osd.0 v2:172.21.15.154:6800/2920938138 12 ==== osd_op_reply(121 [pgnls-filter start_epoch 43 out=49b] v41'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42fc0 con 0x55c967aaa000 2023-09-23T03:18:01.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.922+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:122 6.f 6:f0000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967c20e00 con 0x55c967ab9800 2023-09-23T03:18:01.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.923+0000 7f2f8d4ae640 1 -- 172.21.15.154:0/2871567064 <== osd.2 v2:172.21.15.154:6822/2884407243 9 ==== osd_op_reply(122 [pgnls-filter start_epoch 43 out=49b] v41'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42fc0 con 0x55c967ab9800 2023-09-23T03:18:01.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.923+0000 7f2f88ca5640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:123 6.2f 6:f4000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf2e00 con 0x55c967ab9400 2023-09-23T03:18:01.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.924+0000 7f2f8dcaf640 1 -- 172.21.15.154:0/2871567064 <== osd.7 v2:172.21.15.190:6824/3460136272 35 ==== osd_op_reply(123 [pgnls-filter start_epoch 43 out=49b] v41'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42d80 con 0x55c967ab9400 2023-09-23T03:18:01.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.924+0000 7f2f884a4640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:124 6.1f 6:f8000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967c21180 con 0x55c967aaa800 2023-09-23T03:18:01.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.925+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.3 v2:172.21.15.154:6808/1133688468 13 ==== osd_op_reply(124 [pgnls-filter start_epoch 43 out=49b] v41'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42b40 con 0x55c967aaa800 2023-09-23T03:18:01.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.925+0000 7f2f88ca5640 1 -- 172.21.15.154:0/2871567064 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:125 6.3f 6:fc000000::::head [pgnls-filter start_epoch 43 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x55c967bf2a80 con 0x55c967aaa800 2023-09-23T03:18:01.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.926+0000 7f2f8ccad640 1 -- 172.21.15.154:0/2871567064 <== osd.3 v2:172.21.15.154:6808/1133688468 14 ==== osd_op_reply(125 [pgnls-filter start_epoch 43 out=49b] v41'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c967a42b40 con 0x55c967aaa800 2023-09-23T03:18:01.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.926+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55c967aaa000 msgr2=0x55c9677cfb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.926+0000 7f2f8f4435c0 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55c967aaa000 0x55c9677cfb80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.926+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55c967ab9000 msgr2=0x55c967bd8000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.926+0000 7f2f8f4435c0 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55c967ab9000 0x55c967bd8000 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.926+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55c967ab9800 msgr2=0x55c967bd8b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.926+0000 7f2f8f4435c0 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55c967ab9800 0x55c967bd8b00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.926+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55c967aaa800 msgr2=0x55c9677cf600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.926+0000 7f2f8f4435c0 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55c967aaa800 0x55c9677cf600 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.926+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55c967bf6000 msgr2=0x55c967bda100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.926+0000 7f2f8f4435c0 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55c967bf6000 0x55c967bda100 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.926+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55c967ab9c00 msgr2=0x55c967bd9080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.926+0000 7f2f8f4435c0 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55c967ab9c00 0x55c967bd9080 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.926+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55c967aaa400 msgr2=0x55c9677d0100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.926+0000 7f2f8f4435c0 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55c967aaa400 0x55c9677d0100 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.926+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55c967ab9400 msgr2=0x55c967bd8580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.926+0000 7f2f8f4435c0 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55c967ab9400 0x55c967bd8580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.926+0000 7f2f8d4ae640 1 -- 172.21.15.154:0/2871567064 reap_dead start 2023-09-23T03:18:01.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.927+0000 7f2f8d4ae640 1 -- 172.21.15.154:0/2871567064 reap_dead start 2023-09-23T03:18:01.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.927+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55c967ab8c00 msgr2=0x55c9677d1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.927+0000 7f2f8f4435c0 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55c967ab8c00 0x55c9677d1700 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55c967a508a0 tx=0x55c967a5cc00 comp rx=0 tx=0).stop 2023-09-23T03:18:01.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.927+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c967aab000 msgr2=0x55c9677d0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.927+0000 7f2f8f4435c0 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c967aab000 0x55c9677d0c00 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x55c967a51560 tx=0x55c967a5cde0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.927+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 shutdown_connections 2023-09-23T03:18:01.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.927+0000 7f2f8f4435c0 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55c967ab8c00 0x55c9677d1700 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.927+0000 7f2f8f4435c0 1 --2- 172.21.15.154:0/2871567064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c967aab000 0x55c9677d0c00 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.927+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 >> 172.21.15.154:0/2871567064 conn(0x55c968537c00 msgr2=0x55c9684ef000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:01.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.928+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 shutdown_connections 2023-09-23T03:18:01.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.928+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/2871567064 wait complete. 2023-09-23T03:18:01.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.929+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/1122821403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c967765800 msgr2=0x55c9677ceb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:01.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.929+0000 7f2f8f4435c0 1 --2- 172.21.15.154:0/1122821403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c967765800 0x55c9677ceb00 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x55c968566a80 tx=0x55c967a46120 comp rx=0 tx=0).stop 2023-09-23T03:18:01.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.929+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/1122821403 shutdown_connections 2023-09-23T03:18:01.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.929+0000 7f2f8f4435c0 1 --2- 172.21.15.154:0/1122821403 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c967765c00 0x55c9677cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.929+0000 7f2f8f4435c0 1 --2- 172.21.15.154:0/1122821403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c967765800 0x55c9677ceb00 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.929+0000 7f2f8f4435c0 1 --2- 172.21.15.154:0/1122821403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c967764c00 0x55c9677ce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:01.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.929+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/1122821403 >> 172.21.15.154:0/1122821403 conn(0x55c967765000 msgr2=0x55c9684ee800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:01.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.929+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/1122821403 shutdown_connections 2023-09-23T03:18:01.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:01.930+0000 7f2f8f4435c0 1 -- 172.21.15.154:0/1122821403 wait complete. 2023-09-23T03:18:01.940 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes'] in time 0:00:00.300512, result: 2023-09-23T03:18:01.940 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2023-09-23T03:18:02.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.049+0000 7fc7e7e135c0 1 -- start start 2023-09-23T03:18:02.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.049+0000 7fc7e7e135c0 1 Processor -- start 2023-09-23T03:18:02.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.050+0000 7fc7e7e135c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f02ab8400 0x561f01d5b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:02.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.050+0000 7fc7e7e135c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x561f02ab8800 0x561f01d5ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:02.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.050+0000 7fc7e7e135c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f02ab9c00 0x561f01d5bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:02.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.050+0000 7fc7e7e135c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f01d3a140 con 0x561f02ab8400 2023-09-23T03:18:02.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.050+0000 7fc7e7e135c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f01d3b180 con 0x561f02ab9c00 2023-09-23T03:18:02.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.050+0000 7fc7e7e135c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x561f01dbedc0 con 0x561f02ab8800 2023-09-23T03:18:02.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.051+0000 7fc7e59d3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f02ab8400 0x561f01d5b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:02.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.051+0000 7fc7e59d3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f02ab8400 0x561f01d5b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60526/0 (socket says 172.21.15.154:60526) 2023-09-23T03:18:02.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.051+0000 7fc7e59d3640 1 -- 172.21.15.154:0/1483274787 learned_addr learned my addr 172.21.15.154:0/1483274787 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:02.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.051+0000 7fc7e51d2640 1 --2- 172.21.15.154:0/1483274787 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x561f02ab8800 0x561f01d5ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:02.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.051+0000 7fc7e61d4640 1 --2- 172.21.15.154:0/1483274787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f02ab9c00 0x561f01d5bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:02.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.051+0000 7fc7e59d3640 1 -- 172.21.15.154:0/1483274787 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x561f02ab8800 msgr2=0x561f01d5ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:02.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.051+0000 7fc7e59d3640 1 --2- 172.21.15.154:0/1483274787 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x561f02ab8800 0x561f01d5ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:02.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.051+0000 7fc7e59d3640 1 -- 172.21.15.154:0/1483274787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f02ab9c00 msgr2=0x561f01d5bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:02.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.051+0000 7fc7e59d3640 1 --2- 172.21.15.154:0/1483274787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f02ab9c00 0x561f01d5bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:02.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.051+0000 7fc7e59d3640 1 -- 172.21.15.154:0/1483274787 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f01d24000 con 0x561f02ab8400 2023-09-23T03:18:02.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.051+0000 7fc7e61d4640 1 --2- 172.21.15.154:0/1483274787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f02ab9c00 0x561f01d5bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:18:02.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.051+0000 7fc7e59d3640 1 --2- 172.21.15.154:0/1483274787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f02ab8400 0x561f01d5b600 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x561f02af91a0 tx=0x561f01fd83c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d29d63ad6b87d9f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:02.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.052+0000 7fc7e31ce640 1 -- 172.21.15.154:0/1483274787 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561f01d39080 con 0x561f02ab8400 2023-09-23T03:18:02.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.052+0000 7fc7e31ce640 1 -- 172.21.15.154:0/1483274787 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561f01d17800 con 0x561f02ab8400 2023-09-23T03:18:02.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.052+0000 7fc7e31ce640 1 -- 172.21.15.154:0/1483274787 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561f01d39600 con 0x561f02ab8400 2023-09-23T03:18:02.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.052+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/1483274787 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561f01d24ea0 con 0x561f02ab8400 2023-09-23T03:18:02.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.052+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/1483274787 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x561f01d25520 con 0x561f02ab8400 2023-09-23T03:18:02.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.052+0000 7fc7e7e135c0 4 waiting for MDS map... 2023-09-23T03:18:02.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.052+0000 7fc7e31ce640 1 -- 172.21.15.154:0/1483274787 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(43..43 src has 1..43) v4 ==== 6605+0+0 (secure 0 0 0) 0x561f01db0c40 con 0x561f02ab8400 2023-09-23T03:18:02.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.052+0000 7fc7e31ce640 1 -- 172.21.15.154:0/1483274787 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 25) v1 ==== 3306+0+0 (secure 0 0 0) 0x561f01d43200 con 0x561f02ab8400 2023-09-23T03:18:02.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.052+0000 7fc7e7e135c0 4 Got MDS map 25 2023-09-23T03:18:02.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.052+0000 7fc7e7e135c0 4 datascan.main: Using metadata pool output 2023-09-23T03:18:02.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.052+0000 7fc7e7e135c0 4 datascan.main: connecting to RADOS... 2023-09-23T03:18:02.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.053+0000 7fc7e7e135c0 1 Processor -- start 2023-09-23T03:18:02.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.053+0000 7fc7e7e135c0 1 -- start start 2023-09-23T03:18:02.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.054+0000 7fc7e7e135c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f02ab9800 0x561f01d5b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:02.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.054+0000 7fc7e7e135c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f01fe2c00 0x561f01d5a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:02.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.054+0000 7fc7e7e135c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x561f01fe3000 0x561f01d5c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:02.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.054+0000 7fc7e7e135c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f01dbf180 con 0x561f01fe2c00 2023-09-23T03:18:02.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.054+0000 7fc7e7e135c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f01d3ab40 con 0x561f02ab9800 2023-09-23T03:18:02.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.054+0000 7fc7e7e135c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x561f01d3aa00 con 0x561f01fe3000 2023-09-23T03:18:02.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.054+0000 7fc7e59d3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f01fe2c00 0x561f01d5a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:02.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.054+0000 7fc7e59d3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f01fe2c00 0x561f01d5a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60528/0 (socket says 172.21.15.154:60528) 2023-09-23T03:18:02.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.054+0000 7fc7e59d3640 1 -- 172.21.15.154:0/1195885699 learned_addr learned my addr 172.21.15.154:0/1195885699 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:02.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.055+0000 7fc7e61d4640 1 --2- 172.21.15.154:0/1195885699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f02ab9800 0x561f01d5b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:02.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.055+0000 7fc7e51d2640 1 --2- 172.21.15.154:0/1195885699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x561f01fe3000 0x561f01d5c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:02.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.055+0000 7fc7e59d3640 1 -- 172.21.15.154:0/1195885699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x561f01fe3000 msgr2=0x561f01d5c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:02.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.055+0000 7fc7e59d3640 1 --2- 172.21.15.154:0/1195885699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x561f01fe3000 0x561f01d5c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:02.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.055+0000 7fc7e59d3640 1 -- 172.21.15.154:0/1195885699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f02ab9800 msgr2=0x561f01d5b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:02.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.055+0000 7fc7e59d3640 1 --2- 172.21.15.154:0/1195885699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f02ab9800 0x561f01d5b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:02.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.055+0000 7fc7e59d3640 1 -- 172.21.15.154:0/1195885699 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f01d256c0 con 0x561f01fe2c00 2023-09-23T03:18:02.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.055+0000 7fc7e61d4640 1 --2- 172.21.15.154:0/1195885699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f02ab9800 0x561f01d5b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:18:02.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.055+0000 7fc7e59d3640 1 --2- 172.21.15.154:0/1195885699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f01fe2c00 0x561f01d5a580 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x561f01ff4750 tx=0x561f01fd8060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e5c0246fbdfa41 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:02.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.055+0000 7fc7e11ca640 1 -- 172.21.15.154:0/1195885699 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561f01d391e0 con 0x561f01fe2c00 2023-09-23T03:18:02.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.055+0000 7fc7e11ca640 1 -- 172.21.15.154:0/1195885699 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561f01fdac00 con 0x561f01fe2c00 2023-09-23T03:18:02.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.055+0000 7fc7e11ca640 1 -- 172.21.15.154:0/1195885699 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561f01d39080 con 0x561f01fe2c00 2023-09-23T03:18:02.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.056+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/1195885699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f01fe2c00 msgr2=0x561f01d5a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:02.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.056+0000 7fc7e7e135c0 1 --2- 172.21.15.154:0/1195885699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f01fe2c00 0x561f01d5a580 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x561f01ff4750 tx=0x561f01fd8060 comp rx=0 tx=0).stop 2023-09-23T03:18:02.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.056+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/1195885699 shutdown_connections 2023-09-23T03:18:02.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.056+0000 7fc7e7e135c0 1 --2- 172.21.15.154:0/1195885699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x561f01fe3000 0x561f01d5c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:02.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.056+0000 7fc7e7e135c0 1 --2- 172.21.15.154:0/1195885699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f01fe2c00 0x561f01d5a580 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:02.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.056+0000 7fc7e7e135c0 1 --2- 172.21.15.154:0/1195885699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f02ab9800 0x561f01d5b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:02.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.056+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/1195885699 >> 172.21.15.154:0/1195885699 conn(0x561f02ab9400 msgr2=0x561f02a6b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:02.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.056+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/1195885699 shutdown_connections 2023-09-23T03:18:02.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.056+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/1195885699 wait complete. 2023-09-23T03:18:02.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.057+0000 7fc7e7e135c0 1 Processor -- start 2023-09-23T03:18:02.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.057+0000 7fc7e7e135c0 1 -- start start 2023-09-23T03:18:02.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.058+0000 7fc7e7e135c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f02ab9400 0x561f01d5d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:02.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.058+0000 7fc7e7e135c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f01fe3c00 0x561f01d5cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:02.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.058+0000 7fc7e7e135c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x561f01fe3800 0x561f01d5c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:02.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.058+0000 7fc7e7e135c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f01dbedc0 con 0x561f01fe3c00 2023-09-23T03:18:02.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.058+0000 7fc7e7e135c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f01dbec80 con 0x561f02ab9400 2023-09-23T03:18:02.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.058+0000 7fc7e7e135c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x561f01d3a140 con 0x561f01fe3800 2023-09-23T03:18:02.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.058+0000 7fc7e59d3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f01fe3c00 0x561f01d5cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:02.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.058+0000 7fc7e59d3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f01fe3c00 0x561f01d5cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60544/0 (socket says 172.21.15.154:60544) 2023-09-23T03:18:02.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.058+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 learned_addr learned my addr 172.21.15.154:0/789620281 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:02.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.058+0000 7fc7e51d2640 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x561f01fe3800 0x561f01d5c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:02.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.058+0000 7fc7e61d4640 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f02ab9400 0x561f01d5d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:02.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.058+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x561f01fe3800 msgr2=0x561f01d5c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:02.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.058+0000 7fc7e59d3640 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x561f01fe3800 0x561f01d5c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:02.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.058+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f02ab9400 msgr2=0x561f01d5d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:02.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.058+0000 7fc7e59d3640 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f02ab9400 0x561f01d5d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:02.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.058+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f01d24000 con 0x561f01fe3c00 2023-09-23T03:18:02.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.059+0000 7fc7e59d3640 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f01fe3c00 0x561f01d5cc00 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x561f01ff5410 tx=0x561f01ff8c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:02.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.059+0000 7fc7df9c7640 1 -- 172.21.15.154:0/789620281 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561f01d39600 con 0x561f01fe3c00 2023-09-23T03:18:02.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.059+0000 7fc7df9c7640 1 -- 172.21.15.154:0/789620281 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561f01fdb380 con 0x561f01fe3c00 2023-09-23T03:18:02.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.059+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561f01d24ea0 con 0x561f01fe3c00 2023-09-23T03:18:02.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.059+0000 7fc7df9c7640 1 -- 172.21.15.154:0/789620281 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561f01d38f20 con 0x561f01fe3c00 2023-09-23T03:18:02.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.059+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561f01d25860 con 0x561f01fe3c00 2023-09-23T03:18:02.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.059+0000 7fc7e7e135c0 4 datascan.init: resolving metadata pool 5 2023-09-23T03:18:02.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.061+0000 7fc7df9c7640 1 -- 172.21.15.154:0/789620281 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x561f01d22b00 con 0x561f01fe3c00 2023-09-23T03:18:02.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.062+0000 7fc7df9c7640 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x561f02ac9000 0x561f01d5d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:02.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.062+0000 7fc7df9c7640 1 -- 172.21.15.154:0/789620281 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6605+0+0 (secure 0 0 0) 0x561f01db0a80 con 0x561f01fe3c00 2023-09-23T03:18:02.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.062+0000 7fc7e7e135c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-09-23T03:18:02.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.062+0000 7fc7e7e135c0 4 datascan.main: resolving metadata pool 5 2023-09-23T03:18:02.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.062+0000 7fc7e7e135c0 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x561f01fe3400 0x561f01d5b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:02.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.062+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:1 5.0 5:00000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f01d4b880 con 0x561f01fe3400 2023-09-23T03:18:02.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.062+0000 7fc7e61d4640 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x561f02ac9000 0x561f01d5d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:02.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.062+0000 7fc7e51d2640 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x561f01fe3400 0x561f01d5b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:02.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.063+0000 7fc7e61d4640 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x561f02ac9000 0x561f01d5d700 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x561f01de2840 tx=0x561f01fd81e0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:02.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.063+0000 7fc7e51d2640 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x561f01fe3400 0x561f01d5b080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:02.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.063+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(1 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccb40 con 0x561f01fe3400 2023-09-23T03:18:02.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.063+0000 7fc7e51d2640 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x561f02ac9400 0x561f0214c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:02.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.063+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:2 5.10 5:08000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f01d4b880 con 0x561f02ac9400 2023-09-23T03:18:02.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.064+0000 7fc7e61d4640 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x561f02ac9400 0x561f0214c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:02.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.065+0000 7fc7e61d4640 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x561f02ac9400 0x561f0214c000 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:02.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.066+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 <== osd.7 v2:172.21.15.190:6824/3460136272 1 ==== osd_op_reply(2 [pgnls start_epoch 43 out=81b] v43'21 uv21 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561f01dccd80 con 0x561f02ac9400 2023-09-23T03:18:02.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.066+0000 7fc7e61d4640 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x561f02ac9800 0x561f0214c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:02.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.066+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:3 5.8 5:10000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f01d4b880 con 0x561f02ac9800 2023-09-23T03:18:02.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.066+0000 7fc7e59d3640 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x561f02ac9800 0x561f0214c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:02.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.067+0000 7fc7e59d3640 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x561f02ac9800 0x561f0214c580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:02.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.067+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 <== osd.2 v2:172.21.15.154:6822/2884407243 1 ==== osd_op_reply(3 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccfc0 con 0x561f02ac9800 2023-09-23T03:18:02.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.067+0000 7fc7e59d3640 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x561f02ac8400 0x561f0214d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:02.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.067+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:4 5.18 5:18000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f01d4b880 con 0x561f02ac8400 2023-09-23T03:18:02.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.068+0000 7fc7e51d2640 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x561f02ac8400 0x561f0214d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:02.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.069+0000 7fc7e51d2640 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x561f02ac8400 0x561f0214d080 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:02.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.069+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 <== osd.4 v2:172.21.15.190:6808/717302641 1 ==== osd_op_reply(4 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccb40 con 0x561f02ac8400 2023-09-23T03:18:02.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.069+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:5 5.4 5:20000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f01d4b880 con 0x561f02ac9400 2023-09-23T03:18:02.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.070+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 <== osd.7 v2:172.21.15.190:6824/3460136272 2 ==== osd_op_reply(5 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccd80 con 0x561f02ac9400 2023-09-23T03:18:02.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.070+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:6 5.14 5:28000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f01d4b880 con 0x561f01fe3400 2023-09-23T03:18:02.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.071+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 <== osd.3 v2:172.21.15.154:6808/1133688468 2 ==== osd_op_reply(6 [pgnls start_epoch 43 out=125b] v43'42 uv42 ondisk = 1) v8 ==== 144+0+125 (crc 0 0 0) 0x561f01dccb40 con 0x561f01fe3400 2023-09-23T03:18:02.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.071+0000 7fc7e51d2640 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x561f02162000 0x561f0214d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:02.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.071+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:7 5.c 5:30000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f01d4b880 con 0x561f02162000 2023-09-23T03:18:02.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.071+0000 7fc7e61d4640 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x561f02162000 0x561f0214d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:02.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.072+0000 7fc7e61d4640 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x561f02162000 0x561f0214d600 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:02.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.072+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 <== osd.1 v2:172.21.15.154:6816/1267710005 1 ==== osd_op_reply(7 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccd80 con 0x561f02162000 2023-09-23T03:18:02.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.072+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:8 5.1c 5:38000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f01d4b880 con 0x561f02ac8400 2023-09-23T03:18:02.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.073+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 <== osd.4 v2:172.21.15.190:6808/717302641 2 ==== osd_op_reply(8 [pgnls start_epoch 43 out=49b] v42'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccb40 con 0x561f02ac8400 2023-09-23T03:18:02.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.073+0000 7fc7e51d2640 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x561f02162400 0x561f0214db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:02.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.073+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:9 5.2 5:40000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f01d4b880 con 0x561f02162400 2023-09-23T03:18:02.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.074+0000 7fc7e59d3640 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x561f02162400 0x561f0214db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:02.075 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.075+0000 7fc7e59d3640 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x561f02162400 0x561f0214db80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:02.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.076+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== osd_op_reply(9 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccfc0 con 0x561f02162400 2023-09-23T03:18:02.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.076+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:10 5.12 5:48000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f01d4b880 con 0x561f02162000 2023-09-23T03:18:02.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.076+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 <== osd.1 v2:172.21.15.154:6816/1267710005 2 ==== osd_op_reply(10 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccd80 con 0x561f02162000 2023-09-23T03:18:02.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.076+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:11 5.a 5:50000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f01d4b880 con 0x561f02ac9800 2023-09-23T03:18:02.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.076+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 <== osd.2 v2:172.21.15.154:6822/2884407243 2 ==== osd_op_reply(11 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccfc0 con 0x561f02ac9800 2023-09-23T03:18:02.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.077+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:12 5.1a 5:58000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f01d4b880 con 0x561f02ac9400 2023-09-23T03:18:02.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.077+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 <== osd.7 v2:172.21.15.190:6824/3460136272 3 ==== osd_op_reply(12 [pgnls start_epoch 43 out=49b] v43'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccd80 con 0x561f02ac9400 2023-09-23T03:18:02.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.077+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:13 5.6 5:60000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f01d4b880 con 0x561f02ac9800 2023-09-23T03:18:02.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.078+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 <== osd.2 v2:172.21.15.154:6822/2884407243 3 ==== osd_op_reply(13 [pgnls start_epoch 43 out=49b] v42'8 uv8 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccfc0 con 0x561f02ac9800 2023-09-23T03:18:02.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.078+0000 7fc7e59d3640 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x561f02ac8800 0x561f0214e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:02.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.078+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:14 5.16 5:68000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f01d4b880 con 0x561f02ac8800 2023-09-23T03:18:02.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.079+0000 7fc7e51d2640 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x561f02ac8800 0x561f0214e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:02.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.080+0000 7fc7e51d2640 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x561f02ac8800 0x561f0214e100 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:02.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.081+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 <== osd.5 v2:172.21.15.190:6800/882536885 1 ==== osd_op_reply(14 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccb40 con 0x561f02ac8800 2023-09-23T03:18:02.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.081+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:15 5.e 5:70000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f01d4b880 con 0x561f02ac8400 2023-09-23T03:18:02.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.082+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 <== osd.4 v2:172.21.15.190:6808/717302641 3 ==== osd_op_reply(15 [pgnls start_epoch 43 out=49b] v42'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccb40 con 0x561f02ac8400 2023-09-23T03:18:02.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.082+0000 7fc7e51d2640 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x561f02162800 0x561f0214ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:02.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.082+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:16 5.1e 5:78000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f01d4b880 con 0x561f02162800 2023-09-23T03:18:02.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.082+0000 7fc7e61d4640 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x561f02162800 0x561f0214ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:02.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.082+0000 7fc7e61d4640 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x561f02162800 0x561f0214ec00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:02.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.083+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 <== osd.0 v2:172.21.15.154:6800/2920938138 1 ==== osd_op_reply(16 [pgnls start_epoch 43 out=49b] v43'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccd80 con 0x561f02162800 2023-09-23T03:18:02.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.083+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:17 5.1 5:80000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f01d4b880 con 0x561f02ac8400 2023-09-23T03:18:02.084 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.084+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 <== osd.4 v2:172.21.15.190:6808/717302641 4 ==== osd_op_reply(17 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccb40 con 0x561f02ac8400 2023-09-23T03:18:02.084 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.084+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:18 5.11 5:88000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f01d4b880 con 0x561f02162400 2023-09-23T03:18:02.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.085+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 <== osd.6 v2:172.21.15.190:6816/1964294001 2 ==== osd_op_reply(18 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccfc0 con 0x561f02162400 2023-09-23T03:18:02.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.085+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:19 5.9 5:90000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f01d4b880 con 0x561f02ac9400 2023-09-23T03:18:02.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.086+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 <== osd.7 v2:172.21.15.190:6824/3460136272 4 ==== osd_op_reply(19 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccd80 con 0x561f02ac9400 2023-09-23T03:18:02.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.086+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:20 5.19 5:98000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f01d4b880 con 0x561f02162000 2023-09-23T03:18:02.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.086+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 <== osd.1 v2:172.21.15.154:6816/1267710005 3 ==== osd_op_reply(20 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccd80 con 0x561f02162000 2023-09-23T03:18:02.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.086+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:21 5.5 5:a0000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f01d4b880 con 0x561f02162800 2023-09-23T03:18:02.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.086+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 <== osd.0 v2:172.21.15.154:6800/2920938138 2 ==== osd_op_reply(21 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccd80 con 0x561f02162800 2023-09-23T03:18:02.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.086+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:22 5.15 5:a8000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f01d4b880 con 0x561f02ac8800 2023-09-23T03:18:02.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.087+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 <== osd.5 v2:172.21.15.190:6800/882536885 2 ==== osd_op_reply(22 [pgnls start_epoch 43 out=73b] v42'6 uv6 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x561f01dccb40 con 0x561f02ac8800 2023-09-23T03:18:02.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.087+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:23 5.d 5:b0000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f01d4b880 con 0x561f02ac9800 2023-09-23T03:18:02.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.088+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 <== osd.2 v2:172.21.15.154:6822/2884407243 4 ==== osd_op_reply(23 [pgnls start_epoch 43 out=74b] v43'3 uv3 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x561f01dccfc0 con 0x561f02ac9800 2023-09-23T03:18:02.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.088+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:24 5.1d 5:b8000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f01d4b880 con 0x561f02162000 2023-09-23T03:18:02.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.088+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 <== osd.1 v2:172.21.15.154:6816/1267710005 4 ==== osd_op_reply(24 [pgnls start_epoch 43 out=49b] v42'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccd80 con 0x561f02162000 2023-09-23T03:18:02.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.088+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:25 5.3 5:c0000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f01d4b880 con 0x561f02162800 2023-09-23T03:18:02.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.089+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 <== osd.0 v2:172.21.15.154:6800/2920938138 3 ==== osd_op_reply(25 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccd80 con 0x561f02162800 2023-09-23T03:18:02.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.089+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:26 5.13 5:c8000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f01d4b880 con 0x561f01fe3400 2023-09-23T03:18:02.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.089+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 <== osd.3 v2:172.21.15.154:6808/1133688468 3 ==== osd_op_reply(26 [pgnls start_epoch 43 out=98b] v43'7 uv7 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x561f01dccb40 con 0x561f01fe3400 2023-09-23T03:18:02.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.089+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:27 5.b 5:d0000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f01d4b880 con 0x561f02ac9800 2023-09-23T03:18:02.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.090+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 <== osd.2 v2:172.21.15.154:6822/2884407243 5 ==== osd_op_reply(27 [pgnls start_epoch 43 out=76b] v43'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x561f01dccfc0 con 0x561f02ac9800 2023-09-23T03:18:02.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.090+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:28 5.1b 5:d8000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f01d4b880 con 0x561f02ac8800 2023-09-23T03:18:02.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.091+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 <== osd.5 v2:172.21.15.190:6800/882536885 3 ==== osd_op_reply(28 [pgnls start_epoch 43 out=49b] v43'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccb40 con 0x561f02ac8800 2023-09-23T03:18:02.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.091+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:29 5.7 5:e0000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f01d4b880 con 0x561f02ac8800 2023-09-23T03:18:02.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.092+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 <== osd.5 v2:172.21.15.190:6800/882536885 4 ==== osd_op_reply(29 [pgnls start_epoch 43 out=49b] v42'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccb40 con 0x561f02ac8800 2023-09-23T03:18:02.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.092+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:30 5.17 5:e8000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f01d4b880 con 0x561f01fe3400 2023-09-23T03:18:02.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.092+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 <== osd.3 v2:172.21.15.154:6808/1133688468 4 ==== osd_op_reply(30 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccb40 con 0x561f01fe3400 2023-09-23T03:18:02.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.092+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:31 5.f 5:f0000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f01d4b880 con 0x561f02ac8800 2023-09-23T03:18:02.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.093+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 <== osd.5 v2:172.21.15.190:6800/882536885 5 ==== osd_op_reply(31 [pgnls start_epoch 43 out=73b] v43'4 uv4 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x561f01dccb40 con 0x561f02ac8800 2023-09-23T03:18:02.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.093+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:32 5.1f 5:f8000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f01d4b880 con 0x561f02162400 2023-09-23T03:18:02.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.094+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 <== osd.6 v2:172.21.15.190:6816/1964294001 3 ==== osd_op_reply(32 [pgnls start_epoch 43 out=101b] v43'12 uv12 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x561f01dccfc0 con 0x561f02162400 2023-09-23T03:18:02.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.094+0000 7fc7e7e135c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2023-09-23T03:18:02.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.094+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:33 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f01d4bc00 con 0x561f02ac9400 2023-09-23T03:18:02.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.095+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 <== osd.7 v2:172.21.15.190:6824/3460136272 5 ==== osd_op_reply(33 10000000000.00000000 [omap-get-vals out=2495b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+2495 (crc 0 0 0) 0x561f01dccd80 con 0x561f02ac9400 2023-09-23T03:18:02.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.095+0000 7fc7e7e135c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:18:02.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.095+0000 7fc7e7e135c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:18:02.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.095+0000 7fc7e7e135c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:18:02.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.095+0000 7fc7e7e135c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:18:02.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.096+0000 7fc7e7e135c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:18:02.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.096+0000 7fc7e7e135c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2023-09-23T03:18:02.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.096+0000 7fc7e7e135c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-09-23T03:18:02.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.096+0000 7fc7e7e135c0 10 datascan.scan_links: step 1: handling object 200.00000000 2023-09-23T03:18:02.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.096+0000 7fc7e7e135c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-09-23T03:18:02.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.096+0000 7fc7e7e135c0 10 datascan.scan_links: step 1: handling object 200.00000001 2023-09-23T03:18:02.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.096+0000 7fc7e7e135c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-09-23T03:18:02.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.096+0000 7fc7e7e135c0 10 datascan.scan_links: step 1: handling object 500.00000000 2023-09-23T03:18:02.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.096+0000 7fc7e7e135c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-09-23T03:18:02.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.096+0000 7fc7e7e135c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2023-09-23T03:18:02.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.096+0000 7fc7e7e135c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-09-23T03:18:02.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.096+0000 7fc7e7e135c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2023-09-23T03:18:02.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.096+0000 7fc7e7e135c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-09-23T03:18:02.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.096+0000 7fc7e7e135c0 10 datascan.scan_links: step 1: handling object 100.00000000 2023-09-23T03:18:02.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.096+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:34 5.13 5:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217a000 con 0x561f01fe3400 2023-09-23T03:18:02.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.096+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 <== osd.3 v2:172.21.15.154:6808/1133688468 5 ==== osd_op_reply(34 100.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x561f01dccb40 con 0x561f01fe3400 2023-09-23T03:18:02.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.096+0000 7fc7e7e135c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2023-09-23T03:18:02.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.096+0000 7fc7e7e135c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-09-23T03:18:02.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.096+0000 7fc7e7e135c0 10 datascan.scan_links: step 1: handling object 400.00000000 2023-09-23T03:18:02.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.096+0000 7fc7e7e135c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-09-23T03:18:02.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.096+0000 7fc7e7e135c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2023-09-23T03:18:02.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.096+0000 7fc7e7e135c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-09-23T03:18:02.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.096+0000 7fc7e7e135c0 10 datascan.scan_links: step 1: handling object 1.00000000 2023-09-23T03:18:02.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.096+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:35 5.1f 5:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217a380 con 0x561f02162400 2023-09-23T03:18:02.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.098+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 <== osd.6 v2:172.21.15.190:6816/1964294001 4 ==== osd_op_reply(35 1.00000000 [omap-get-vals out=508b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+508 (crc 0 0 0) 0x561f01dccfc0 con 0x561f02162400 2023-09-23T03:18:02.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.098+0000 7fc7e7e135c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:18:02.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.098+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:36 5.0 5:00000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217a700 con 0x561f01fe3400 2023-09-23T03:18:02.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.098+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 <== osd.3 v2:172.21.15.154:6808/1133688468 6 ==== osd_op_reply(36 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccb40 con 0x561f01fe3400 2023-09-23T03:18:02.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.098+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:37 5.10 5:08000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217a000 con 0x561f02ac9400 2023-09-23T03:18:02.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.099+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 <== osd.7 v2:172.21.15.190:6824/3460136272 6 ==== osd_op_reply(37 [pgnls start_epoch 43 out=81b] v43'21 uv21 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561f01dccd80 con 0x561f02ac9400 2023-09-23T03:18:02.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.099+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:38 5.8 5:10000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217a000 con 0x561f02ac9800 2023-09-23T03:18:02.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.100+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 <== osd.2 v2:172.21.15.154:6822/2884407243 6 ==== osd_op_reply(38 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccfc0 con 0x561f02ac9800 2023-09-23T03:18:02.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.100+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:39 5.18 5:18000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217a380 con 0x561f02ac8400 2023-09-23T03:18:02.101 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.101+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 <== osd.4 v2:172.21.15.190:6808/717302641 5 ==== osd_op_reply(39 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccb40 con 0x561f02ac8400 2023-09-23T03:18:02.101 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.101+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:40 5.4 5:20000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217a380 con 0x561f02ac9400 2023-09-23T03:18:02.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.102+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 <== osd.7 v2:172.21.15.190:6824/3460136272 7 ==== osd_op_reply(40 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccd80 con 0x561f02ac9400 2023-09-23T03:18:02.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.102+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:41 5.14 5:28000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217a380 con 0x561f01fe3400 2023-09-23T03:18:02.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.103+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 <== osd.3 v2:172.21.15.154:6808/1133688468 7 ==== osd_op_reply(41 [pgnls start_epoch 43 out=125b] v43'42 uv42 ondisk = 1) v8 ==== 144+0+125 (crc 0 0 0) 0x561f01dccb40 con 0x561f01fe3400 2023-09-23T03:18:02.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.103+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:42 5.c 5:30000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217a380 con 0x561f02162000 2023-09-23T03:18:02.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.103+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 <== osd.1 v2:172.21.15.154:6816/1267710005 5 ==== osd_op_reply(42 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccd80 con 0x561f02162000 2023-09-23T03:18:02.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.103+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:43 5.1c 5:38000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217a380 con 0x561f02ac8400 2023-09-23T03:18:02.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.104+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 <== osd.4 v2:172.21.15.190:6808/717302641 6 ==== osd_op_reply(43 [pgnls start_epoch 43 out=49b] v42'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccb40 con 0x561f02ac8400 2023-09-23T03:18:02.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.104+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:44 5.2 5:40000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217a380 con 0x561f02162400 2023-09-23T03:18:02.106 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.106+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 <== osd.6 v2:172.21.15.190:6816/1964294001 5 ==== osd_op_reply(44 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccfc0 con 0x561f02162400 2023-09-23T03:18:02.106 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.106+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:45 5.12 5:48000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217a380 con 0x561f02162000 2023-09-23T03:18:02.106 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.106+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 <== osd.1 v2:172.21.15.154:6816/1267710005 6 ==== osd_op_reply(45 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccd80 con 0x561f02162000 2023-09-23T03:18:02.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.106+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:46 5.a 5:50000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217a380 con 0x561f02ac9800 2023-09-23T03:18:02.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.107+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 <== osd.2 v2:172.21.15.154:6822/2884407243 7 ==== osd_op_reply(46 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccfc0 con 0x561f02ac9800 2023-09-23T03:18:02.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.107+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:47 5.1a 5:58000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217a380 con 0x561f02ac9400 2023-09-23T03:18:02.108 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.108+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 <== osd.7 v2:172.21.15.190:6824/3460136272 8 ==== osd_op_reply(47 [pgnls start_epoch 43 out=49b] v43'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccd80 con 0x561f02ac9400 2023-09-23T03:18:02.108 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.108+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:48 5.6 5:60000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217a380 con 0x561f02ac9800 2023-09-23T03:18:02.109 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.109+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 <== osd.2 v2:172.21.15.154:6822/2884407243 8 ==== osd_op_reply(48 [pgnls start_epoch 43 out=49b] v42'8 uv8 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccfc0 con 0x561f02ac9800 2023-09-23T03:18:02.109 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.109+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:49 5.16 5:68000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217a380 con 0x561f02ac8800 2023-09-23T03:18:02.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.110+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 <== osd.5 v2:172.21.15.190:6800/882536885 6 ==== osd_op_reply(49 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccb40 con 0x561f02ac8800 2023-09-23T03:18:02.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.110+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:50 5.e 5:70000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217a380 con 0x561f02ac8400 2023-09-23T03:18:02.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.111+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 <== osd.4 v2:172.21.15.190:6808/717302641 7 ==== osd_op_reply(50 [pgnls start_epoch 43 out=49b] v42'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccb40 con 0x561f02ac8400 2023-09-23T03:18:02.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.111+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:51 5.1e 5:78000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217a380 con 0x561f02162800 2023-09-23T03:18:02.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.111+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 <== osd.0 v2:172.21.15.154:6800/2920938138 4 ==== osd_op_reply(51 [pgnls start_epoch 43 out=49b] v43'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccd80 con 0x561f02162800 2023-09-23T03:18:02.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.111+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:52 5.1 5:80000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217a380 con 0x561f02ac8400 2023-09-23T03:18:02.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.112+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 <== osd.4 v2:172.21.15.190:6808/717302641 8 ==== osd_op_reply(52 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccb40 con 0x561f02ac8400 2023-09-23T03:18:02.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.112+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:53 5.11 5:88000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217a380 con 0x561f02162400 2023-09-23T03:18:02.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.113+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 <== osd.6 v2:172.21.15.190:6816/1964294001 6 ==== osd_op_reply(53 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccfc0 con 0x561f02162400 2023-09-23T03:18:02.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.113+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:54 5.9 5:90000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217a380 con 0x561f02ac9400 2023-09-23T03:18:02.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.114+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 <== osd.7 v2:172.21.15.190:6824/3460136272 9 ==== osd_op_reply(54 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccd80 con 0x561f02ac9400 2023-09-23T03:18:02.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.114+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:55 5.19 5:98000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217a380 con 0x561f02162000 2023-09-23T03:18:02.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.115+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 <== osd.1 v2:172.21.15.154:6816/1267710005 7 ==== osd_op_reply(55 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccd80 con 0x561f02162000 2023-09-23T03:18:02.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.115+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:56 5.5 5:a0000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217a380 con 0x561f02162800 2023-09-23T03:18:02.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.115+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 <== osd.0 v2:172.21.15.154:6800/2920938138 5 ==== osd_op_reply(56 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccd80 con 0x561f02162800 2023-09-23T03:18:02.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.115+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:57 5.15 5:a8000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217a380 con 0x561f02ac8800 2023-09-23T03:18:02.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.116+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 <== osd.5 v2:172.21.15.190:6800/882536885 7 ==== osd_op_reply(57 [pgnls start_epoch 43 out=73b] v42'6 uv6 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x561f01dccb40 con 0x561f02ac8800 2023-09-23T03:18:02.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.116+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:58 5.d 5:b0000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217a380 con 0x561f02ac9800 2023-09-23T03:18:02.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.117+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 <== osd.2 v2:172.21.15.154:6822/2884407243 9 ==== osd_op_reply(58 [pgnls start_epoch 43 out=74b] v43'3 uv3 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x561f01dccfc0 con 0x561f02ac9800 2023-09-23T03:18:02.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.117+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:59 5.1d 5:b8000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217a380 con 0x561f02162000 2023-09-23T03:18:02.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.117+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 <== osd.1 v2:172.21.15.154:6816/1267710005 8 ==== osd_op_reply(59 [pgnls start_epoch 43 out=49b] v42'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccd80 con 0x561f02162000 2023-09-23T03:18:02.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.117+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:60 5.3 5:c0000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217a380 con 0x561f02162800 2023-09-23T03:18:02.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.118+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 <== osd.0 v2:172.21.15.154:6800/2920938138 6 ==== osd_op_reply(60 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccd80 con 0x561f02162800 2023-09-23T03:18:02.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.118+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:61 5.13 5:c8000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217a380 con 0x561f01fe3400 2023-09-23T03:18:02.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.119+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 <== osd.3 v2:172.21.15.154:6808/1133688468 8 ==== osd_op_reply(61 [pgnls start_epoch 43 out=98b] v43'7 uv7 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x561f01dccb40 con 0x561f01fe3400 2023-09-23T03:18:02.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.119+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:62 5.b 5:d0000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217a380 con 0x561f02ac9800 2023-09-23T03:18:02.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.119+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 <== osd.2 v2:172.21.15.154:6822/2884407243 10 ==== osd_op_reply(62 [pgnls start_epoch 43 out=76b] v43'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x561f01dccfc0 con 0x561f02ac9800 2023-09-23T03:18:02.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.119+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:63 5.1b 5:d8000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217a380 con 0x561f02ac8800 2023-09-23T03:18:02.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.120+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 <== osd.5 v2:172.21.15.190:6800/882536885 8 ==== osd_op_reply(63 [pgnls start_epoch 43 out=49b] v43'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccb40 con 0x561f02ac8800 2023-09-23T03:18:02.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.120+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:64 5.7 5:e0000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217a380 con 0x561f02ac8800 2023-09-23T03:18:02.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.121+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 <== osd.5 v2:172.21.15.190:6800/882536885 9 ==== osd_op_reply(64 [pgnls start_epoch 43 out=49b] v42'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccb40 con 0x561f02ac8800 2023-09-23T03:18:02.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.121+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:65 5.17 5:e8000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217a380 con 0x561f01fe3400 2023-09-23T03:18:02.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.122+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 <== osd.3 v2:172.21.15.154:6808/1133688468 9 ==== osd_op_reply(65 [pgnls start_epoch 43 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561f01dccb40 con 0x561f01fe3400 2023-09-23T03:18:02.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.122+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:66 5.f 5:f0000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217a380 con 0x561f02ac8800 2023-09-23T03:18:02.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.123+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 <== osd.5 v2:172.21.15.190:6800/882536885 10 ==== osd_op_reply(66 [pgnls start_epoch 43 out=73b] v43'4 uv4 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x561f01dccb40 con 0x561f02ac8800 2023-09-23T03:18:02.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.123+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:67 5.1f 5:f8000000::::head [pgnls start_epoch 43 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217a380 con 0x561f02162400 2023-09-23T03:18:02.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.125+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 <== osd.6 v2:172.21.15.190:6816/1964294001 7 ==== osd_op_reply(67 [pgnls start_epoch 43 out=101b] v43'12 uv12 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x561f01dccfc0 con 0x561f02162400 2023-09-23T03:18:02.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.125+0000 7fc7e7e135c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2023-09-23T03:18:02.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.125+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:68 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217aa80 con 0x561f02ac9400 2023-09-23T03:18:02.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.126+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 <== osd.7 v2:172.21.15.190:6824/3460136272 10 ==== osd_op_reply(68 10000000000.00000000 [omap-get-vals out=2495b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+2495 (crc 0 0 0) 0x561f01dccd80 con 0x561f02ac9400 2023-09-23T03:18:02.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.126+0000 7fc7e7e135c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:18:02.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.126+0000 7fc7e7e135c0 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2023-09-23T03:18:02.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.126+0000 7fc7e7e135c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:18:02.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.126+0000 7fc7e7e135c0 20 datascan.scan_links: adding 0x100000001f6 for future processing to fix dnfirst 2023-09-23T03:18:02.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.126+0000 7fc7e7e135c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:18:02.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.126+0000 7fc7e7e135c0 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2023-09-23T03:18:02.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.126+0000 7fc7e7e135c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:18:02.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.126+0000 7fc7e7e135c0 20 datascan.scan_links: adding 0x100000001f8 for future processing to fix dnfirst 2023-09-23T03:18:02.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.126+0000 7fc7e7e135c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:18:02.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.126+0000 7fc7e7e135c0 20 datascan.scan_links: adding 0x100000001f9 for future processing to fix dnfirst 2023-09-23T03:18:02.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.126+0000 7fc7e7e135c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2023-09-23T03:18:02.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.126+0000 7fc7e7e135c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-09-23T03:18:02.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.126+0000 7fc7e7e135c0 10 datascan.scan_links: step 2: handling object 200.00000000 2023-09-23T03:18:02.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.126+0000 7fc7e7e135c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-09-23T03:18:02.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.126+0000 7fc7e7e135c0 10 datascan.scan_links: step 2: handling object 200.00000001 2023-09-23T03:18:02.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.126+0000 7fc7e7e135c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-09-23T03:18:02.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.126+0000 7fc7e7e135c0 10 datascan.scan_links: step 2: handling object 500.00000000 2023-09-23T03:18:02.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.126+0000 7fc7e7e135c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-09-23T03:18:02.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.126+0000 7fc7e7e135c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2023-09-23T03:18:02.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.126+0000 7fc7e7e135c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-09-23T03:18:02.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.126+0000 7fc7e7e135c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2023-09-23T03:18:02.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.126+0000 7fc7e7e135c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-09-23T03:18:02.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.126+0000 7fc7e7e135c0 10 datascan.scan_links: step 2: handling object 100.00000000 2023-09-23T03:18:02.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.127+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:69 5.13 5:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217ae00 con 0x561f01fe3400 2023-09-23T03:18:02.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.127+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 <== osd.3 v2:172.21.15.154:6808/1133688468 10 ==== osd_op_reply(69 100.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x561f01dccb40 con 0x561f01fe3400 2023-09-23T03:18:02.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.127+0000 7fc7e7e135c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2023-09-23T03:18:02.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.127+0000 7fc7e7e135c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-09-23T03:18:02.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.127+0000 7fc7e7e135c0 10 datascan.scan_links: step 2: handling object 400.00000000 2023-09-23T03:18:02.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.127+0000 7fc7e7e135c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-09-23T03:18:02.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.127+0000 7fc7e7e135c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2023-09-23T03:18:02.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.127+0000 7fc7e7e135c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-09-23T03:18:02.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.127+0000 7fc7e7e135c0 10 datascan.scan_links: step 2: handling object 1.00000000 2023-09-23T03:18:02.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.127+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:70 5.1f 5:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217b180 con 0x561f02162400 2023-09-23T03:18:02.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.129+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 <== osd.6 v2:172.21.15.190:6816/1964294001 8 ==== osd_op_reply(70 1.00000000 [omap-get-vals out=508b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+508 (crc 0 0 0) 0x561f01dccfc0 con 0x561f02162400 2023-09-23T03:18:02.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.129+0000 7fc7e7e135c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:18:02.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.129+0000 7fc7e7e135c0 20 datascan.scan_links: adding 0x10000000000 for future processing to fix dnfirst 2023-09-23T03:18:02.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.129+0000 7fc7e7e135c0 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2023-09-23T03:18:02.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.129+0000 7fc7e7e135c0 10 datascan.scan_links: removing dup dentries from 0 objects 2023-09-23T03:18:02.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.129+0000 7fc7e7e135c0 10 datascan.scan_links: processing 0 bad_nlink_inos 2023-09-23T03:18:02.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.129+0000 7fc7e7e135c0 10 datascan.scan_links: processing 6 injected_inos 2023-09-23T03:18:02.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.129+0000 7fc7e7e135c0 10 datascan.scan_links: handling injected_ino 0x10000000000 2023-09-23T03:18:02.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.129+0000 7fc7e7e135c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x1:subdir 2023-09-23T03:18:02.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.129+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:71 5.1f 5:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217b500 con 0x561f02162400 2023-09-23T03:18:02.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.130+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 <== osd.6 v2:172.21.15.190:6816/1964294001 9 ==== osd_op_reply(71 1.00000000 [omap-get-vals-by-keys out=507b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+507 (crc 0 0 0) 0x561f01dccfc0 con 0x561f02162400 2023-09-23T03:18:02.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.130+0000 7fc7e7e135c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:18:02.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.130+0000 7fc7e7e135c0 20 datascan.scan_links: first is now 2 2023-09-23T03:18:02.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.130+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:72 5.1f 5:ff5b34d6:::1.00000000:head [omap-set-vals in=507b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217b880 con 0x561f02162400 2023-09-23T03:18:02.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.134+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 <== osd.6 v2:172.21.15.190:6816/1964294001 10 ==== osd_op_reply(72 1.00000000 [omap-set-vals] v43'13 uv13 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x561f01dccfc0 con 0x561f02162400 2023-09-23T03:18:02.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.134+0000 7fc7e7e135c0 20 datascan.inject_linkage: Injected dentry 0x0x1/subdir pointing to 0x0x10000000000 2023-09-23T03:18:02.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.134+0000 7fc7e7e135c0 10 datascan.scan_links: handling injected_ino 0x10000000001 2023-09-23T03:18:02.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.134+0000 7fc7e7e135c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:0 2023-09-23T03:18:02.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.134+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:73 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217bc00 con 0x561f02ac9400 2023-09-23T03:18:02.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.135+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 <== osd.7 v2:172.21.15.190:6824/3460136272 11 ==== osd_op_reply(73 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x561f01dccd80 con 0x561f02ac9400 2023-09-23T03:18:02.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.135+0000 7fc7e7e135c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=0 frag=*, r=0 2023-09-23T03:18:02.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.135+0000 7fc7e7e135c0 20 datascan.scan_links: first is now 2 2023-09-23T03:18:02.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.135+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:74 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217a000 con 0x561f02ac9400 2023-09-23T03:18:02.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.139+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 <== osd.7 v2:172.21.15.190:6824/3460136272 12 ==== osd_op_reply(74 10000000000.00000000 [omap-set-vals] v43'22 uv22 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561f01dccd80 con 0x561f02ac9400 2023-09-23T03:18:02.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.139+0000 7fc7e7e135c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/0 pointing to 0x0x10000000001 2023-09-23T03:18:02.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.139+0000 7fc7e7e135c0 10 datascan.scan_links: handling injected_ino 0x100000001f6 2023-09-23T03:18:02.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.139+0000 7fc7e7e135c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:1 2023-09-23T03:18:02.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.139+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:75 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217a380 con 0x561f02ac9400 2023-09-23T03:18:02.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.140+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 <== osd.7 v2:172.21.15.190:6824/3460136272 13 ==== osd_op_reply(75 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x561f01dccd80 con 0x561f02ac9400 2023-09-23T03:18:02.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.140+0000 7fc7e7e135c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=1 frag=*, r=0 2023-09-23T03:18:02.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.140+0000 7fc7e7e135c0 20 datascan.scan_links: first is now 2 2023-09-23T03:18:02.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.140+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:76 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217b180 con 0x561f02ac9400 2023-09-23T03:18:02.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.144+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 <== osd.7 v2:172.21.15.190:6824/3460136272 14 ==== osd_op_reply(76 10000000000.00000000 [omap-set-vals] v43'23 uv23 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561f01dccd80 con 0x561f02ac9400 2023-09-23T03:18:02.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.144+0000 7fc7e7e135c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/1 pointing to 0x0x100000001f6 2023-09-23T03:18:02.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.144+0000 7fc7e7e135c0 10 datascan.scan_links: handling injected_ino 0x100000001f7 2023-09-23T03:18:02.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.144+0000 7fc7e7e135c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:2 2023-09-23T03:18:02.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.144+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:77 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217b500 con 0x561f02ac9400 2023-09-23T03:18:02.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.145+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 <== osd.7 v2:172.21.15.190:6824/3460136272 15 ==== osd_op_reply(77 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x561f01dccd80 con 0x561f02ac9400 2023-09-23T03:18:02.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.145+0000 7fc7e7e135c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=2 frag=*, r=0 2023-09-23T03:18:02.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.145+0000 7fc7e7e135c0 20 datascan.scan_links: first is now 2 2023-09-23T03:18:02.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.146+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:78 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f01d4b880 con 0x561f02ac9400 2023-09-23T03:18:02.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.149+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 <== osd.7 v2:172.21.15.190:6824/3460136272 16 ==== osd_op_reply(78 10000000000.00000000 [omap-set-vals] v43'24 uv24 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561f01dccd80 con 0x561f02ac9400 2023-09-23T03:18:02.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.149+0000 7fc7e7e135c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/2 pointing to 0x0x100000001f7 2023-09-23T03:18:02.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.149+0000 7fc7e7e135c0 10 datascan.scan_links: handling injected_ino 0x100000001f8 2023-09-23T03:18:02.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.149+0000 7fc7e7e135c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:3 2023-09-23T03:18:02.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.149+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:79 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217aa80 con 0x561f02ac9400 2023-09-23T03:18:02.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.150+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 <== osd.7 v2:172.21.15.190:6824/3460136272 17 ==== osd_op_reply(79 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv24 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x561f01dccd80 con 0x561f02ac9400 2023-09-23T03:18:02.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.150+0000 7fc7e7e135c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=3 frag=*, r=0 2023-09-23T03:18:02.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.150+0000 7fc7e7e135c0 20 datascan.scan_links: first is now 2 2023-09-23T03:18:02.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.150+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:80 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217bc00 con 0x561f02ac9400 2023-09-23T03:18:02.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.154+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 <== osd.7 v2:172.21.15.190:6824/3460136272 18 ==== osd_op_reply(80 10000000000.00000000 [omap-set-vals] v43'25 uv25 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561f01dccd80 con 0x561f02ac9400 2023-09-23T03:18:02.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.154+0000 7fc7e7e135c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/3 pointing to 0x0x100000001f8 2023-09-23T03:18:02.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.154+0000 7fc7e7e135c0 10 datascan.scan_links: handling injected_ino 0x100000001f9 2023-09-23T03:18:02.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.154+0000 7fc7e7e135c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:4 2023-09-23T03:18:02.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.154+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:81 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217a000 con 0x561f02ac9400 2023-09-23T03:18:02.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.155+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 <== osd.7 v2:172.21.15.190:6824/3460136272 19 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv25 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x561f01dccd80 con 0x561f02ac9400 2023-09-23T03:18:02.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.155+0000 7fc7e7e135c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=4 frag=*, r=0 2023-09-23T03:18:02.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.155+0000 7fc7e7e135c0 20 datascan.scan_links: first is now 2 2023-09-23T03:18:02.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.155+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:82 5.10 5:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f01d4bc00 con 0x561f02ac9400 2023-09-23T03:18:02.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.159+0000 7fc7e61d4640 1 -- 172.21.15.154:0/789620281 <== osd.7 v2:172.21.15.190:6824/3460136272 20 ==== osd_op_reply(82 10000000000.00000000 [omap-set-vals] v43'26 uv26 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561f01dccd80 con 0x561f02ac9400 2023-09-23T03:18:02.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.159+0000 7fc7e7e135c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/4 pointing to 0x0x100000001f9 2023-09-23T03:18:02.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.159+0000 7fc7e7e135c0 10 datascan.scan_links: updating inotable 2023-09-23T03:18:02.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.159+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:83 5.13 5:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217a380 con 0x561f01fe3400 2023-09-23T03:18:02.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.160+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 <== osd.3 v2:172.21.15.154:6808/1133688468 11 ==== osd_op_reply(83 mds0_inotable [read 0~34 out=34b] v0'0 uv6 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x561f01dccb40 con 0x561f01fe3400 2023-09-23T03:18:02.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.160+0000 7fc7e7e135c0 10 mds.0.inotable: skip_inos was [0x10000000000~0x10000000000] 2023-09-23T03:18:02.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.160+0000 7fc7e7e135c0 10 mds.0.inotable: skip_inos now [0x100000001fa~0xfffffffe06] 2023-09-23T03:18:02.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.160+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:84 5.13 5:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217b180 con 0x561f01fe3400 2023-09-23T03:18:02.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.164+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 <== osd.3 v2:172.21.15.154:6808/1133688468 12 ==== osd_op_reply(84 mds0_inotable [writefull 0~34] v43'8 uv8 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x561f01dccb40 con 0x561f01fe3400 2023-09-23T03:18:02.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.164+0000 7fc7e7e135c0 10 datascan.scan_links: updating snaptable 2023-09-23T03:18:02.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.164+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:85 5.d 5:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v8 -- 0x561f0217b500 con 0x561f02ac9800 2023-09-23T03:18:02.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.166+0000 7fc7e59d3640 1 -- 172.21.15.154:0/789620281 <== osd.2 v2:172.21.15.154:6822/2884407243 11 ==== osd_op_reply(85 mds_snaptable [read 0~70 out=70b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x561f01dccfc0 con 0x561f02ac9800 2023-09-23T03:18:02.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.166+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x561f02162800 msgr2=0x561f0214ec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:02.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.166+0000 7fc7e7e135c0 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x561f02162800 0x561f0214ec00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:02.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.166+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x561f02162000 msgr2=0x561f0214d600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:02.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.166+0000 7fc7e7e135c0 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x561f02162000 0x561f0214d600 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:02.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.166+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x561f02ac9800 msgr2=0x561f0214c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:02.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.166+0000 7fc7e7e135c0 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x561f02ac9800 0x561f0214c580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:02.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.166+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x561f01fe3400 msgr2=0x561f01d5b080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:02.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.166+0000 7fc7e7e135c0 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x561f01fe3400 0x561f01d5b080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:02.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.166+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x561f02ac8400 msgr2=0x561f0214d080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:02.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.166+0000 7fc7e7e135c0 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x561f02ac8400 0x561f0214d080 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:02.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.166+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 reap_dead start 2023-09-23T03:18:02.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.166+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x561f02ac8800 msgr2=0x561f0214e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:02.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.166+0000 7fc7e7e135c0 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x561f02ac8800 0x561f0214e100 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:02.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.166+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 reap_dead start 2023-09-23T03:18:02.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.166+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x561f02162400 msgr2=0x561f0214db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:02.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.166+0000 7fc7e7e135c0 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x561f02162400 0x561f0214db80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:02.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.166+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x561f02ac9400 msgr2=0x561f0214c000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:02.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.166+0000 7fc7e7e135c0 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x561f02ac9400 0x561f0214c000 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:02.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.167+0000 7fc7e51d2640 1 -- 172.21.15.154:0/789620281 reap_dead start 2023-09-23T03:18:02.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.167+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x561f02ac9000 msgr2=0x561f01d5d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:02.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.167+0000 7fc7e7e135c0 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x561f02ac9000 0x561f01d5d700 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x561f01de2840 tx=0x561f01fd81e0 comp rx=0 tx=0).stop 2023-09-23T03:18:02.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.167+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f01fe3c00 msgr2=0x561f01d5cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:02.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.167+0000 7fc7e7e135c0 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f01fe3c00 0x561f01d5cc00 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x561f01ff5410 tx=0x561f01ff8c00 comp rx=0 tx=0).stop 2023-09-23T03:18:02.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.167+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 shutdown_connections 2023-09-23T03:18:02.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.167+0000 7fc7e7e135c0 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x561f02ac9000 0x561f01d5d700 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:02.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.167+0000 7fc7e7e135c0 1 --2- 172.21.15.154:0/789620281 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f01fe3c00 0x561f01d5cc00 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:02.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.167+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 >> 172.21.15.154:0/789620281 conn(0x561f02ab9800 msgr2=0x561f02a6b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:02.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.167+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 shutdown_connections 2023-09-23T03:18:02.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.168+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/789620281 wait complete. 2023-09-23T03:18:02.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.169+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/1483274787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f02ab8400 msgr2=0x561f01d5b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:02.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.169+0000 7fc7e7e135c0 1 --2- 172.21.15.154:0/1483274787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f02ab8400 0x561f01d5b600 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x561f02af91a0 tx=0x561f01fd83c0 comp rx=0 tx=0).stop 2023-09-23T03:18:02.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.169+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/1483274787 shutdown_connections 2023-09-23T03:18:02.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.169+0000 7fc7e7e135c0 1 --2- 172.21.15.154:0/1483274787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561f02ab9c00 0x561f01d5bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:02.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.169+0000 7fc7e7e135c0 1 --2- 172.21.15.154:0/1483274787 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x561f02ab8800 0x561f01d5ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:02.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.169+0000 7fc7e7e135c0 1 --2- 172.21.15.154:0/1483274787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561f02ab8400 0x561f01d5b600 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:02.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.169+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/1483274787 >> 172.21.15.154:0/1483274787 conn(0x561f01cf1000 msgr2=0x561f02a6a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:02.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.169+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/1483274787 shutdown_connections 2023-09-23T03:18:02.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.169+0000 7fc7e7e135c0 1 -- 172.21.15.154:0/1483274787 wait complete. 2023-09-23T03:18:02.180 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.239540, result: 2023-09-23T03:18:02.180 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-09-23T03:18:02.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.397+0000 7f0a900e5640 1 Processor -- start 2023-09-23T03:18:02.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.397+0000 7f0a900e5640 1 -- start start 2023-09-23T03:18:02.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.398+0000 7f0a900e5640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0a8815ce90 0x7f0a880d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:02.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.398+0000 7f0a900e5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a880cf920 0x7f0a880d1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:02.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.398+0000 7f0a900e5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a880d1650 con 0x7f0a8815cab0 2023-09-23T03:18:02.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.398+0000 7f0a900e5640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a880d18e0 con 0x7f0a8815ce90 2023-09-23T03:18:02.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.398+0000 7f0a900e5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a880d1bc0 con 0x7f0a880cf920 2023-09-23T03:18:02.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.399+0000 7f0a8de81640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f0a8815cab0 0x7f0a880cf210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:50734/0 (socket says 172.21.15.154:50734) 2023-09-23T03:18:02.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.399+0000 7f0a8de81640 1 -- 172.21.15.154:0/1240168487 learned_addr learned my addr 172.21.15.154:0/1240168487 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:18:02.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.399+0000 7f0a8d680640 1 --2- 172.21.15.154:0/1240168487 >> v2:172.21.15.190:3300/0 conn(0x7f0a8815ce90 0x7f0a880d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:02.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.399+0000 7f0a8e682640 1 --2- 172.21.15.154:0/1240168487 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a880cf920 0x7f0a880d1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:02.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.399+0000 7f0a8ce7f640 1 -- 172.21.15.154:0/1240168487 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1717332678 0 0) 0x7f0a880d1650 con 0x7f0a8815cab0 2023-09-23T03:18:02.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.399+0000 7f0a8ce7f640 1 -- 172.21.15.154:0/1240168487 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a78003520 con 0x7f0a8815cab0 2023-09-23T03:18:02.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.400+0000 7f0a8ce7f640 1 -- 172.21.15.154:0/1240168487 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3454457056 0 0) 0x7f0a78003520 con 0x7f0a8815cab0 2023-09-23T03:18:02.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.400+0000 7f0a8ce7f640 1 -- 172.21.15.154:0/1240168487 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a880cf920 msgr2=0x7f0a880d1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:02.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.400+0000 7f0a8ce7f640 1 --2- 172.21.15.154:0/1240168487 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a880cf920 0x7f0a880d1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:02.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.400+0000 7f0a8ce7f640 1 -- 172.21.15.154:0/1240168487 >> v2:172.21.15.190:3300/0 conn(0x7f0a8815ce90 msgr2=0x7f0a880d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:02.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.400+0000 7f0a8ce7f640 1 --2- 172.21.15.154:0/1240168487 >> v2:172.21.15.190:3300/0 conn(0x7f0a8815ce90 0x7f0a880d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:02.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.400+0000 7f0a8ce7f640 1 -- 172.21.15.154:0/1240168487 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a880d1ea0 con 0x7f0a8815cab0 2023-09-23T03:18:02.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.400+0000 7f0a8ce7f640 1 -- 172.21.15.154:0/1240168487 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f0a7c0019f0 con 0x7f0a8815cab0 2023-09-23T03:18:02.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.400+0000 7f0a8ce7f640 1 -- 172.21.15.154:0/1240168487 >> v1:172.21.15.154:6789/0 conn(0x7f0a8815cab0 legacy=0x7f0a880cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:02.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.400+0000 7f0a8ce7f640 1 --2- 172.21.15.154:0/1240168487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a78003970 0x7f0a78003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:02.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.400+0000 7f0a8ce7f640 1 --2- 172.21.15.154:0/1240168487 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a880cf920 0x7f0a780047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:02.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.400+0000 7f0a8ce7f640 1 --2- 172.21.15.154:0/1240168487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a78004d00 0x7f0a78005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:02.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.400+0000 7f0a8ce7f640 1 -- 172.21.15.154:0/1240168487 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a880d18e0 con 0x7f0a78004d00 2023-09-23T03:18:02.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.400+0000 7f0a8ce7f640 1 -- 172.21.15.154:0/1240168487 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a880d1bc0 con 0x7f0a78003970 2023-09-23T03:18:02.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.400+0000 7f0a8ce7f640 1 -- 172.21.15.154:0/1240168487 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7800b980 con 0x7f0a880cf920 2023-09-23T03:18:02.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.400+0000 7f0a8e682640 1 --2- 172.21.15.154:0/1240168487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a78004d00 0x7f0a78005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:02.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.400+0000 7f0a8de81640 1 --2- 172.21.15.154:0/1240168487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a78003970 0x7f0a78003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:02.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.401+0000 7f0a8e682640 1 -- 172.21.15.154:0/1240168487 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a880cf920 msgr2=0x7f0a780047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:02.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.401+0000 7f0a8e682640 1 --2- 172.21.15.154:0/1240168487 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a880cf920 0x7f0a780047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:02.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.401+0000 7f0a8e682640 1 -- 172.21.15.154:0/1240168487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a78003970 msgr2=0x7f0a78003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:02.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.401+0000 7f0a8e682640 1 --2- 172.21.15.154:0/1240168487 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a78003970 0x7f0a78003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:02.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.401+0000 7f0a8e682640 1 -- 172.21.15.154:0/1240168487 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a78003520 con 0x7f0a78004d00 2023-09-23T03:18:02.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.402+0000 7f0a8e682640 1 --2- 172.21.15.154:0/1240168487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a78004d00 0x7f0a78005180 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009a90 tx=0x7f0a780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=692ba06239b03673 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:02.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.402+0000 7f0a8ce7f640 1 -- 172.21.15.154:0/1240168487 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8002c6b0 con 0x7f0a78004d00 2023-09-23T03:18:02.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.402+0000 7f0a8ce7f640 1 -- 172.21.15.154:0/1240168487 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8002c810 con 0x7f0a78004d00 2023-09-23T03:18:02.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.402+0000 7f0a8ce7f640 1 -- 172.21.15.154:0/1240168487 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8002cba0 con 0x7f0a78004d00 2023-09-23T03:18:02.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.403+0000 7f0a900e5640 1 -- 172.21.15.154:0/1240168487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a78004d00 msgr2=0x7f0a78005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:02.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.403+0000 7f0a900e5640 1 --2- 172.21.15.154:0/1240168487 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a78004d00 0x7f0a78005180 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009a90 tx=0x7f0a780057f0 comp rx=0 tx=0).stop 2023-09-23T03:18:02.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.403+0000 7f0a8e682640 1 -- 172.21.15.154:0/1240168487 reap_dead start 2023-09-23T03:18:02.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.403+0000 7f0a900e5640 1 -- 172.21.15.154:0/1240168487 shutdown_connections 2023-09-23T03:18:02.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.403+0000 7f0a900e5640 1 -- 172.21.15.154:0/1240168487 >> 172.21.15.154:0/1240168487 conn(0x7f0a88071bc0 msgr2=0x7f0a880b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:02.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.403+0000 7f0a900e5640 1 -- 172.21.15.154:0/1240168487 shutdown_connections 2023-09-23T03:18:02.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.403+0000 7f0a900e5640 1 -- 172.21.15.154:0/1240168487 wait complete. 2023-09-23T03:18:02.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.405+0000 7f0a900e5640 1 Processor -- start 2023-09-23T03:18:02.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.405+0000 7f0a900e5640 1 -- start start 2023-09-23T03:18:02.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.406+0000 7f0a900e5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a78004d00 0x7f0a88141b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:02.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.406+0000 7f0a900e5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a881f2cc0 0x7f0a880c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:02.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.406+0000 7f0a900e5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a880c7dc0 0x7f0a880c8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:02.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.406+0000 7f0a900e5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a881601a0 con 0x7f0a881f2cc0 2023-09-23T03:18:02.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.406+0000 7f0a900e5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a881f5890 con 0x7f0a78004d00 2023-09-23T03:18:02.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.406+0000 7f0a900e5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a88142140 con 0x7f0a880c7dc0 2023-09-23T03:18:02.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.406+0000 7f0a8d680640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a881f2cc0 0x7f0a880c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:02.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.406+0000 7f0a8d680640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a881f2cc0 0x7f0a880c8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60560/0 (socket says 172.21.15.154:60560) 2023-09-23T03:18:02.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.406+0000 7f0a8d680640 1 -- 172.21.15.154:0/1946746800 learned_addr learned my addr 172.21.15.154:0/1946746800 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:02.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.406+0000 7f0a8de81640 1 --2- 172.21.15.154:0/1946746800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a78004d00 0x7f0a88141b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:02.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.406+0000 7f0a8e682640 1 --2- 172.21.15.154:0/1946746800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a880c7dc0 0x7f0a880c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:02.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.406+0000 7f0a8d680640 1 -- 172.21.15.154:0/1946746800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a880c7dc0 msgr2=0x7f0a880c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:02.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.406+0000 7f0a8d680640 1 --2- 172.21.15.154:0/1946746800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a880c7dc0 0x7f0a880c8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:02.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.406+0000 7f0a8d680640 1 -- 172.21.15.154:0/1946746800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a78004d00 msgr2=0x7f0a88141b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:02.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.406+0000 7f0a8d680640 1 --2- 172.21.15.154:0/1946746800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a78004d00 0x7f0a88141b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:02.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.406+0000 7f0a8d680640 1 -- 172.21.15.154:0/1946746800 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a88142420 con 0x7f0a881f2cc0 2023-09-23T03:18:02.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.406+0000 7f0a8de81640 1 --2- 172.21.15.154:0/1946746800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a78004d00 0x7f0a88141b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:18:02.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.407+0000 7f0a8d680640 1 --2- 172.21.15.154:0/1946746800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a881f2cc0 0x7f0a880c8c60 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0a7400e860 tx=0x7f0a7400ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:02.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.407+0000 7f0a867fc640 1 -- 172.21.15.154:0/1946746800 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a74009570 con 0x7f0a881f2cc0 2023-09-23T03:18:02.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.407+0000 7f0a867fc640 1 -- 172.21.15.154:0/1946746800 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a7400c860 con 0x7f0a881f2cc0 2023-09-23T03:18:02.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.407+0000 7f0a867fc640 1 -- 172.21.15.154:0/1946746800 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7400cbf0 con 0x7f0a881f2cc0 2023-09-23T03:18:02.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.407+0000 7f0a900e5640 1 -- 172.21.15.154:0/1946746800 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a88142700 con 0x7f0a881f2cc0 2023-09-23T03:18:02.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.407+0000 7f0a900e5640 1 -- 172.21.15.154:0/1946746800 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a881429e0 con 0x7f0a881f2cc0 2023-09-23T03:18:02.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.408+0000 7f0a900e5640 1 -- 172.21.15.154:0/1946746800 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a88073eb0 con 0x7f0a881f2cc0 2023-09-23T03:18:02.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.409+0000 7f0a867fc640 1 -- 172.21.15.154:0/1946746800 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f0a7400cd50 con 0x7f0a881f2cc0 2023-09-23T03:18:02.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.410+0000 7f0a867fc640 1 --2- 172.21.15.154:0/1946746800 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0a6c06af60 0x7f0a6c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:02.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.410+0000 7f0a867fc640 1 -- 172.21.15.154:0/1946746800 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6634+0+0 (secure 0 0 0) 0x7f0a74089d20 con 0x7f0a881f2cc0 2023-09-23T03:18:02.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.411+0000 7f0a8de81640 1 --2- 172.21.15.154:0/1946746800 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0a6c06af60 0x7f0a6c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:02.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.411+0000 7f0a8de81640 1 --2- 172.21.15.154:0/1946746800 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0a6c06af60 0x7f0a6c06d420 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c001b20 tx=0x7f0a7c002000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:02.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.415+0000 7f0a867fc640 1 -- 172.21.15.154:0/1946746800 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f0a74054920 con 0x7f0a881f2cc0 2023-09-23T03:18:02.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:02.680+0000 7f0a900e5640 1 -- 172.21.15.154:0/1946746800 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f0a880b8300 con 0x7f0a881f2cc0 2023-09-23T03:18:03.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.255+0000 7f0a867fc640 1 -- 172.21.15.154:0/1946746800 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 2:0 v26) v1 ==== 101+0+0 (secure 0 0 0) 0x7f0a74021090 con 0x7f0a881f2cc0 2023-09-23T03:18:03.256 INFO:teuthology.orchestra.run.smithi154.stderr:repaired: restoring rank 2:0 2023-09-23T03:18:03.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.259+0000 7f0a900e5640 1 -- 172.21.15.154:0/1946746800 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0a6c06af60 msgr2=0x7f0a6c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:03.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.259+0000 7f0a900e5640 1 --2- 172.21.15.154:0/1946746800 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0a6c06af60 0x7f0a6c06d420 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c001b20 tx=0x7f0a7c002000 comp rx=0 tx=0).stop 2023-09-23T03:18:03.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.259+0000 7f0a900e5640 1 -- 172.21.15.154:0/1946746800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a881f2cc0 msgr2=0x7f0a880c8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:03.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.259+0000 7f0a900e5640 1 --2- 172.21.15.154:0/1946746800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a881f2cc0 0x7f0a880c8c60 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0a7400e860 tx=0x7f0a7400ed20 comp rx=0 tx=0).stop 2023-09-23T03:18:03.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.259+0000 7f0a900e5640 1 -- 172.21.15.154:0/1946746800 shutdown_connections 2023-09-23T03:18:03.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.259+0000 7f0a900e5640 1 --2- 172.21.15.154:0/1946746800 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0a6c06af60 0x7f0a6c06d420 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:03.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.259+0000 7f0a900e5640 1 --2- 172.21.15.154:0/1946746800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a880c7dc0 0x7f0a880c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:03.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.259+0000 7f0a900e5640 1 --2- 172.21.15.154:0/1946746800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a881f2cc0 0x7f0a880c8c60 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:03.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.259+0000 7f0a900e5640 1 --2- 172.21.15.154:0/1946746800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a78004d00 0x7f0a88141b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:03.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.259+0000 7f0a900e5640 1 -- 172.21.15.154:0/1946746800 >> 172.21.15.154:0/1946746800 conn(0x7f0a88071bc0 msgr2=0x7f0a880d0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:03.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.259+0000 7f0a900e5640 1 -- 172.21.15.154:0/1946746800 shutdown_connections 2023-09-23T03:18:03.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.259+0000 7f0a900e5640 1 -- 172.21.15.154:0/1946746800 wait complete. 2023-09-23T03:18:03.276 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:18:03.276+0000 7f3d6be29640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-09-23T03:18:03.284 INFO:tasks.ceph.mds.a:Restarting daemon 2023-09-23T03:18:03.284 INFO:tasks.ceph.mds.a:Stopping old one... 2023-09-23T03:18:03.284 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-09-23T03:18:03.285 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:18:03.285 INFO:tasks.ceph.mds.c:Restarting daemon 2023-09-23T03:18:03.285 INFO:tasks.ceph.mds.c:Stopping old one... 2023-09-23T03:18:03.285 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-09-23T03:18:03.285 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:18:03.285 INFO:tasks.ceph.mds.b:Restarting daemon 2023-09-23T03:18:03.286 INFO:tasks.ceph.mds.b:Stopping old one... 2023-09-23T03:18:03.286 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-09-23T03:18:03.286 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:18:03.286 INFO:tasks.ceph.mds.d:Restarting daemon 2023-09-23T03:18:03.286 INFO:tasks.ceph.mds.d:Stopping old one... 2023-09-23T03:18:03.286 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-09-23T03:18:03.287 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:18:03.301 INFO:tasks.ceph.mds.c:Stopped 2023-09-23T03:18:03.301 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-09-23T03:18:03.302 INFO:tasks.ceph.mds.a:Stopped 2023-09-23T03:18:03.302 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-09-23T03:18:03.308 INFO:tasks.ceph.mds.d:Stopped 2023-09-23T03:18:03.308 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-09-23T03:18:03.309 INFO:tasks.ceph.mds.b:Stopped 2023-09-23T03:18:03.309 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-09-23T03:18:03.347 INFO:tasks.ceph.mds.c:Started 2023-09-23T03:18:03.347 DEBUG:teuthology.parallel:result is None 2023-09-23T03:18:03.352 INFO:tasks.ceph.mds.a:Started 2023-09-23T03:18:03.353 DEBUG:teuthology.parallel:result is None 2023-09-23T03:18:03.358 INFO:tasks.ceph.mds.d:Started 2023-09-23T03:18:03.358 DEBUG:teuthology.parallel:result is None 2023-09-23T03:18:03.365 INFO:tasks.ceph.mds.b:Started 2023-09-23T03:18:03.365 DEBUG:teuthology.parallel:result is None 2023-09-23T03:18:03.365 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:18:03.508 INFO:tasks.ceph.mds.a.smithi154.stdout:starting mds.a at 2023-09-23T03:18:03.509 INFO:tasks.ceph.mds.c.smithi154.stdout:starting mds.c at 2023-09-23T03:18:03.516 INFO:tasks.ceph.mon.a.smithi154.stderr:2023-09-23T03:18:03.516+0000 7f3df35a6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-23T03:18:03.536 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-09-23T03:18:03.536 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-09-23T03:18:03.538 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:18:03.538+0000 7f35329ba640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-09-23T03:18:03.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.564+0000 7faf44d47640 1 Processor -- start 2023-09-23T03:18:03.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.564+0000 7faf44d47640 1 -- start start 2023-09-23T03:18:03.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.565+0000 7faf44d47640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf4015ca60 0x7faf400cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:03.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.565+0000 7faf44d47640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf400713f0 con 0x7faf4006cec0 2023-09-23T03:18:03.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.565+0000 7faf44d47640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf40070870 con 0x7faf400ae010 2023-09-23T03:18:03.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.565+0000 7faf44d47640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf40070b50 con 0x7faf4015ca60 2023-09-23T03:18:03.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.566+0000 7faf3ffff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7faf4006cec0 0x7faf4006a000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:50740/0 (socket says 172.21.15.154:50740) 2023-09-23T03:18:03.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.566+0000 7faf3ffff640 1 -- 172.21.15.154:0/1888741021 learned_addr learned my addr 172.21.15.154:0/1888741021 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:18:03.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.566+0000 7faf3f7fe640 1 --2- 172.21.15.154:0/1888741021 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf4015ca60 0x7faf400cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:03.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.566+0000 7faf3effd640 1 -- 172.21.15.154:0/1888741021 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3255227242 0 0) 0x7faf400713f0 con 0x7faf4006cec0 2023-09-23T03:18:03.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.566+0000 7faf3effd640 1 -- 172.21.15.154:0/1888741021 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf2c003520 con 0x7faf4006cec0 2023-09-23T03:18:03.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.566+0000 7faf3effd640 1 -- 172.21.15.154:0/1888741021 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2569880226 0 0) 0x7faf40070870 con 0x7faf400ae010 2023-09-23T03:18:03.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.566+0000 7faf3effd640 1 -- 172.21.15.154:0/1888741021 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf400713f0 con 0x7faf400ae010 2023-09-23T03:18:03.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.566+0000 7faf3f7fe640 1 -- 172.21.15.154:0/1888741021 >> v1:172.21.15.190:6789/0 conn(0x7faf400ae010 legacy=0x7faf4006c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:03.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.566+0000 7faf3f7fe640 1 -- 172.21.15.154:0/1888741021 >> v1:172.21.15.154:6789/0 conn(0x7faf4006cec0 legacy=0x7faf4006a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:03.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.566+0000 7faf3f7fe640 1 -- 172.21.15.154:0/1888741021 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf40070e30 con 0x7faf4015ca60 2023-09-23T03:18:03.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.568+0000 7faf3f7fe640 1 --2- 172.21.15.154:0/1888741021 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf4015ca60 0x7faf400cd0f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faf300097d0 tx=0x7faf3002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e34d3aba3984f7c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:03.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.568+0000 7faf3effd640 1 -- 172.21.15.154:0/1888741021 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf30031800 con 0x7faf4015ca60 2023-09-23T03:18:03.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.568+0000 7faf3effd640 1 -- 172.21.15.154:0/1888741021 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf30031960 con 0x7faf4015ca60 2023-09-23T03:18:03.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.568+0000 7faf3effd640 1 -- 172.21.15.154:0/1888741021 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf30031cf0 con 0x7faf4015ca60 2023-09-23T03:18:03.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.568+0000 7faf44d47640 1 -- 172.21.15.154:0/1888741021 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf4015ca60 msgr2=0x7faf400cd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:03.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.568+0000 7faf44d47640 1 --2- 172.21.15.154:0/1888741021 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf4015ca60 0x7faf400cd0f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faf300097d0 tx=0x7faf3002b8a0 comp rx=0 tx=0).stop 2023-09-23T03:18:03.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.568+0000 7faf44d47640 1 -- 172.21.15.154:0/1888741021 shutdown_connections 2023-09-23T03:18:03.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.568+0000 7faf44d47640 1 --2- 172.21.15.154:0/1888741021 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf4015ca60 0x7faf400cd0f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:03.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.568+0000 7faf44d47640 1 -- 172.21.15.154:0/1888741021 >> 172.21.15.154:0/1888741021 conn(0x7faf400cb620 msgr2=0x7faf400cba00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:03.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.569+0000 7faf44d47640 1 -- 172.21.15.154:0/1888741021 shutdown_connections 2023-09-23T03:18:03.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.569+0000 7faf44d47640 1 -- 172.21.15.154:0/1888741021 wait complete. 2023-09-23T03:18:03.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.570+0000 7faf44d47640 1 Processor -- start 2023-09-23T03:18:03.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.570+0000 7faf44d47640 1 -- start start 2023-09-23T03:18:03.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.571+0000 7faf44d47640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf4006cec0 0x7faf40155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:03.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.571+0000 7faf44d47640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf400ae010 0x7faf401574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:03.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.572+0000 7faf44d47640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf4015ca60 0x7faf40157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:03.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.572+0000 7faf44d47640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf401569c0 con 0x7faf4006cec0 2023-09-23T03:18:03.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.572+0000 7faf44d47640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf401564f0 con 0x7faf400ae010 2023-09-23T03:18:03.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.572+0000 7faf44d47640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf401518e0 con 0x7faf4015ca60 2023-09-23T03:18:03.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.572+0000 7faf3f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf4006cec0 0x7faf40155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:03.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.572+0000 7faf3f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf4006cec0 0x7faf40155fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60602/0 (socket says 172.21.15.154:60602) 2023-09-23T03:18:03.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.572+0000 7faf3f7fe640 1 -- 172.21.15.154:0/2490281745 learned_addr learned my addr 172.21.15.154:0/2490281745 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:03.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.572+0000 7faf37fff640 1 --2- 172.21.15.154:0/2490281745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf400ae010 0x7faf401574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:03.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.572+0000 7faf3ffff640 1 --2- 172.21.15.154:0/2490281745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf4015ca60 0x7faf40157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:03.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.572+0000 7faf3f7fe640 1 -- 172.21.15.154:0/2490281745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf4015ca60 msgr2=0x7faf40157a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:03.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.572+0000 7faf3f7fe640 1 --2- 172.21.15.154:0/2490281745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf4015ca60 0x7faf40157a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:03.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.572+0000 7faf3f7fe640 1 -- 172.21.15.154:0/2490281745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf400ae010 msgr2=0x7faf401574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:03.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.572+0000 7faf3f7fe640 1 --2- 172.21.15.154:0/2490281745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf400ae010 0x7faf401574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:03.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.572+0000 7faf3f7fe640 1 -- 172.21.15.154:0/2490281745 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf30009400 con 0x7faf4006cec0 2023-09-23T03:18:03.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.573+0000 7faf3f7fe640 1 --2- 172.21.15.154:0/2490281745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf4006cec0 0x7faf40155fb0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7faf3002beb0 tx=0x7faf300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:03.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.573+0000 7faf3cff9640 1 -- 172.21.15.154:0/2490281745 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3003bc70 con 0x7faf4006cec0 2023-09-23T03:18:03.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.573+0000 7faf3cff9640 1 -- 172.21.15.154:0/2490281745 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf3003bdd0 con 0x7faf4006cec0 2023-09-23T03:18:03.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.573+0000 7faf44d47640 1 -- 172.21.15.154:0/2490281745 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf401566e0 con 0x7faf4006cec0 2023-09-23T03:18:03.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.573+0000 7faf3cff9640 1 -- 172.21.15.154:0/2490281745 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf30045630 con 0x7faf4006cec0 2023-09-23T03:18:03.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.573+0000 7faf44d47640 1 -- 172.21.15.154:0/2490281745 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf40152480 con 0x7faf4006cec0 2023-09-23T03:18:03.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.574+0000 7faf44d47640 1 -- 172.21.15.154:0/2490281745 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf40152000 con 0x7faf4006cec0 2023-09-23T03:18:03.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.575+0000 7faf3cff9640 1 -- 172.21.15.154:0/2490281745 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7faf3004e450 con 0x7faf4006cec0 2023-09-23T03:18:03.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.577+0000 7faf3cff9640 1 --2- 172.21.15.154:0/2490281745 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7faf2006af30 0x7faf2006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:03.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.577+0000 7faf3cff9640 1 -- 172.21.15.154:0/2490281745 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6720+0+0 (secure 0 0 0) 0x7faf3002e070 con 0x7faf4006cec0 2023-09-23T03:18:03.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.577+0000 7faf37fff640 1 --2- 172.21.15.154:0/2490281745 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7faf2006af30 0x7faf2006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:03.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.578+0000 7faf37fff640 1 --2- 172.21.15.154:0/2490281745 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7faf2006af30 0x7faf2006d3f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7faf40153ca0 tx=0x7faf28009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:03.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.582+0000 7faf3cff9640 1 -- 172.21.15.154:0/2490281745 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7faf40152000 con 0x7faf4006cec0 2023-09-23T03:18:03.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.808+0000 7faf44d47640 1 -- 172.21.15.154:0/2490281745 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf40152000 con 0x7faf4006cec0 2023-09-23T03:18:03.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.810+0000 7faf3cff9640 1 -- 172.21.15.154:0/2490281745 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4928 (secure 0 0 0) 0x7faf40152000 con 0x7faf4006cec0 2023-09-23T03:18:03.810 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:18:03.810 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":29,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4760,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/4034412019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4034412019},{"type":"v1","addr":"172.21.15.154:6836","nonce":4034412019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3736308401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3736308401},{"type":"v1","addr":"172.21.15.190:6835","nonce":3736308401}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4847,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/4126656125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":4126656125},{"type":"v1","addr":"172.21.15.190:6834","nonce":4126656125}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:17:56.221314+0000","modified":"2023-09-23T03:18:03.522892+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5988},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5988":{"gid":5988,"name":"a","rank":0,"incarnation":29,"state":"up:replay","state_seq":1,"addr":"172.21.15.154:6836/4278807008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4278807008},{"type":"v1","addr":"172.21.15.154:6836","nonce":4278807008}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-09-23T03:18:03.811 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 29 2023-09-23T03:18:03.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.812+0000 7faf44d47640 1 -- 172.21.15.154:0/2490281745 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7faf2006af30 msgr2=0x7faf2006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:03.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.812+0000 7faf44d47640 1 --2- 172.21.15.154:0/2490281745 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7faf2006af30 0x7faf2006d3f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7faf40153ca0 tx=0x7faf28009040 comp rx=0 tx=0).stop 2023-09-23T03:18:03.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.812+0000 7faf44d47640 1 -- 172.21.15.154:0/2490281745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf4006cec0 msgr2=0x7faf40155fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:03.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.812+0000 7faf44d47640 1 --2- 172.21.15.154:0/2490281745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf4006cec0 0x7faf40155fb0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7faf3002beb0 tx=0x7faf300046c0 comp rx=0 tx=0).stop 2023-09-23T03:18:03.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.812+0000 7faf44d47640 1 -- 172.21.15.154:0/2490281745 shutdown_connections 2023-09-23T03:18:03.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.812+0000 7faf44d47640 1 --2- 172.21.15.154:0/2490281745 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7faf2006af30 0x7faf2006d3f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:03.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.812+0000 7faf44d47640 1 --2- 172.21.15.154:0/2490281745 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faf4015ca60 0x7faf40157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:03.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.812+0000 7faf44d47640 1 --2- 172.21.15.154:0/2490281745 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faf400ae010 0x7faf401574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:03.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.812+0000 7faf44d47640 1 --2- 172.21.15.154:0/2490281745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faf4006cec0 0x7faf40155fb0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:03.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.812+0000 7faf44d47640 1 -- 172.21.15.154:0/2490281745 >> 172.21.15.154:0/2490281745 conn(0x7faf400cb620 msgr2=0x7faf4006c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:03.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.813+0000 7faf44d47640 1 -- 172.21.15.154:0/2490281745 shutdown_connections 2023-09-23T03:18:03.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:03.813+0000 7faf44d47640 1 -- 172.21.15.154:0/2490281745 wait complete. 2023-09-23T03:18:04.233 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-23T03:17:56.221314+0000', 'modified': '2023-09-23T03:18:03.522892+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 46, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5988}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5988': {'gid': 5988, 'name': 'a', 'rank': 0, 'incarnation': 29, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.154:6836/4278807008', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6834', 'nonce': 4278807008}, {'type': 'v1', 'addr': '172.21.15.154:6836', 'nonce': 4278807008}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6, 7], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-23T03:18:04.233 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5988:up:replay 2023-09-23T03:18:05.235 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:18:05.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.460+0000 7f0ff63a9640 1 Processor -- start 2023-09-23T03:18:05.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.460+0000 7f0ff63a9640 1 -- start start 2023-09-23T03:18:05.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.462+0000 7f0ff63a9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0ff00acce0 0x7f0ff015ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:05.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.462+0000 7f0ff63a9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0ff00cf040 0x7f0ff00ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:05.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.462+0000 7f0ff63a9640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff015ac30 con 0x7f0ff00b1b20 2023-09-23T03:18:05.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.462+0000 7f0ff63a9640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff00b2e90 con 0x7f0ff00acce0 2023-09-23T03:18:05.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.462+0000 7f0ff63a9640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff00b1f70 con 0x7f0ff00cf040 2023-09-23T03:18:05.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.462+0000 7f0fef7fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f0ff00b1b20 0x7f0ff00ce930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:52836/0 (socket says 172.21.15.154:52836) 2023-09-23T03:18:05.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.462+0000 7f0fef7fe640 1 -- 172.21.15.154:0/4152943889 learned_addr learned my addr 172.21.15.154:0/4152943889 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:18:05.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.462+0000 7f0feffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0ff00acce0 0x7f0ff015ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:05.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.463+0000 7f0ff4946640 1 --2- 172.21.15.154:0/4152943889 >> v2:172.21.15.190:3300/0 conn(0x7f0ff00cf040 0x7f0ff00ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:05.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.463+0000 7f0feffff640 1 -- 172.21.15.154:0/4152943889 >> v2:172.21.15.190:3300/0 conn(0x7f0ff00cf040 msgr2=0x7f0ff00ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:05.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.463+0000 7f0feffff640 1 --2- 172.21.15.154:0/4152943889 >> v2:172.21.15.190:3300/0 conn(0x7f0ff00cf040 0x7f0ff00ab140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:05.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.463+0000 7f0feffff640 1 -- 172.21.15.154:0/4152943889 >> v1:172.21.15.154:6789/0 conn(0x7f0ff00b1b20 legacy=0x7f0ff00ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:05.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.463+0000 7f0feffff640 1 -- 172.21.15.154:0/4152943889 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff015f2a0 con 0x7f0ff00acce0 2023-09-23T03:18:05.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.464+0000 7f0feffff640 1 --2- 172.21.15.154:0/4152943889 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ff00acce0 0x7f0ff015ed60 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0ff00c9120 tx=0x7f0fe002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab11b18300f61588 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:05.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.464+0000 7f0feeffd640 1 -- 172.21.15.154:0/4152943889 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe002e070 con 0x7f0ff00acce0 2023-09-23T03:18:05.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.464+0000 7f0feeffd640 1 -- 172.21.15.154:0/4152943889 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe00315a0 con 0x7f0ff00acce0 2023-09-23T03:18:05.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.464+0000 7f0feeffd640 1 -- 172.21.15.154:0/4152943889 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe0031930 con 0x7f0ff00acce0 2023-09-23T03:18:05.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.465+0000 7f0ff63a9640 1 -- 172.21.15.154:0/4152943889 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ff00acce0 msgr2=0x7f0ff015ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:05.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.465+0000 7f0ff63a9640 1 --2- 172.21.15.154:0/4152943889 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ff00acce0 0x7f0ff015ed60 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0ff00c9120 tx=0x7f0fe002b840 comp rx=0 tx=0).stop 2023-09-23T03:18:05.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.465+0000 7f0ff63a9640 1 -- 172.21.15.154:0/4152943889 shutdown_connections 2023-09-23T03:18:05.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.465+0000 7f0ff63a9640 1 --2- 172.21.15.154:0/4152943889 >> v2:172.21.15.190:3300/0 conn(0x7f0ff00cf040 0x7f0ff00ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:05.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.465+0000 7f0ff63a9640 1 --2- 172.21.15.154:0/4152943889 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ff00acce0 0x7f0ff015ed60 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:05.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.465+0000 7f0ff63a9640 1 -- 172.21.15.154:0/4152943889 >> 172.21.15.154:0/4152943889 conn(0x7f0ff006a0f0 msgr2=0x7f0ff006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:05.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.465+0000 7f0ff63a9640 1 -- 172.21.15.154:0/4152943889 shutdown_connections 2023-09-23T03:18:05.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.465+0000 7f0ff63a9640 1 -- 172.21.15.154:0/4152943889 wait complete. 2023-09-23T03:18:05.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.466+0000 7f0ff63a9640 1 Processor -- start 2023-09-23T03:18:05.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.467+0000 7f0ff63a9640 1 -- start start 2023-09-23T03:18:05.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.468+0000 7f0ff63a9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ff00acce0 0x7f0ff0097b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:05.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.468+0000 7f0ff63a9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ff00b1b20 0x7f0ff015add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:05.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.468+0000 7f0ff63a9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ff00cf040 0x7f0ff01f4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:05.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.468+0000 7f0ff63a9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff015b310 con 0x7f0ff00acce0 2023-09-23T03:18:05.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.468+0000 7f0ff63a9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff015b590 con 0x7f0ff00cf040 2023-09-23T03:18:05.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.468+0000 7f0ff63a9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff015b870 con 0x7f0ff00b1b20 2023-09-23T03:18:05.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.468+0000 7f0feffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ff00acce0 0x7f0ff0097b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:05.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.468+0000 7f0feffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ff00acce0 0x7f0ff0097b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56710/0 (socket says 172.21.15.154:56710) 2023-09-23T03:18:05.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.468+0000 7f0feffff640 1 -- 172.21.15.154:0/2731553310 learned_addr learned my addr 172.21.15.154:0/2731553310 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:05.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.468+0000 7f0fef7fe640 1 --2- 172.21.15.154:0/2731553310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ff00b1b20 0x7f0ff015add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:05.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.468+0000 7f0feffff640 1 -- 172.21.15.154:0/2731553310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ff00b1b20 msgr2=0x7f0ff015add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:05.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.468+0000 7f0ff4946640 1 --2- 172.21.15.154:0/2731553310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ff00cf040 0x7f0ff01f4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:05.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.468+0000 7f0feffff640 1 --2- 172.21.15.154:0/2731553310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ff00b1b20 0x7f0ff015add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:05.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.468+0000 7f0feffff640 1 -- 172.21.15.154:0/2731553310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ff00cf040 msgr2=0x7f0ff01f4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:05.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.468+0000 7f0feffff640 1 --2- 172.21.15.154:0/2731553310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ff00cf040 0x7f0ff01f4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:05.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.468+0000 7f0feffff640 1 -- 172.21.15.154:0/2731553310 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe00093a0 con 0x7f0ff00acce0 2023-09-23T03:18:05.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.468+0000 7f0fef7fe640 1 --2- 172.21.15.154:0/2731553310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ff00b1b20 0x7f0ff015add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:18:05.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.469+0000 7f0feffff640 1 --2- 172.21.15.154:0/2731553310 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ff00acce0 0x7f0ff0097b00 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0000c00 tx=0x7f0fe00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:05.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.469+0000 7f0fecff9640 1 -- 172.21.15.154:0/2731553310 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe0032900 con 0x7f0ff00acce0 2023-09-23T03:18:05.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.469+0000 7f0fecff9640 1 -- 172.21.15.154:0/2731553310 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe0032a60 con 0x7f0ff00acce0 2023-09-23T03:18:05.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.469+0000 7f0fecff9640 1 -- 172.21.15.154:0/2731553310 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe0032dd0 con 0x7f0ff00acce0 2023-09-23T03:18:05.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.469+0000 7f0ff63a9640 1 -- 172.21.15.154:0/2731553310 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff015bb50 con 0x7f0ff00acce0 2023-09-23T03:18:05.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.469+0000 7f0ff63a9640 1 -- 172.21.15.154:0/2731553310 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff00c22a0 con 0x7f0ff00acce0 2023-09-23T03:18:05.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.470+0000 7f0ff63a9640 1 -- 172.21.15.154:0/2731553310 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ff0073eb0 con 0x7f0ff00acce0 2023-09-23T03:18:05.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.471+0000 7f0fecff9640 1 -- 172.21.15.154:0/2731553310 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f0fe004d3f0 con 0x7f0ff00acce0 2023-09-23T03:18:05.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.472+0000 7f0fecff9640 1 --2- 172.21.15.154:0/2731553310 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0fd006af30 0x7f0fd006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:05.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.472+0000 7f0fecff9640 1 -- 172.21.15.154:0/2731553310 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6720+0+0 (secure 0 0 0) 0x7f0fe00ab900 con 0x7f0ff00acce0 2023-09-23T03:18:05.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.473+0000 7f0fef7fe640 1 --2- 172.21.15.154:0/2731553310 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0fd006af30 0x7f0fd006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:05.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.473+0000 7f0fef7fe640 1 --2- 172.21.15.154:0/2731553310 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0fd006af30 0x7f0fd006d3f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0ff00b1990 tx=0x7f0fd8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:05.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.477+0000 7f0fecff9640 1 -- 172.21.15.154:0/2731553310 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f0fe00764d0 con 0x7f0ff00acce0 2023-09-23T03:18:05.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.708+0000 7f0ff63a9640 1 -- 172.21.15.154:0/2731553310 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ff006c930 con 0x7f0ff00acce0 2023-09-23T03:18:05.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.709+0000 7f0fecff9640 1 -- 172.21.15.154:0/2731553310 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4928 (secure 0 0 0) 0x7f0fe0079af0 con 0x7f0ff00acce0 2023-09-23T03:18:05.710 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:18:05.710 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":31,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5215,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3388818833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3388818833},{"type":"v1","addr":"172.21.15.190:6835","nonce":3388818833}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5991,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1964552060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1964552060},{"type":"v1","addr":"172.21.15.154:6837","nonce":1964552060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5994,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/1736999297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1736999297},{"type":"v1","addr":"172.21.15.190:6834","nonce":1736999297}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:17:56.221314+0000","modified":"2023-09-23T03:18:04.535861+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5988},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5988":{"gid":5988,"name":"a","rank":0,"incarnation":29,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.154:6836/4278807008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4278807008},{"type":"v1","addr":"172.21.15.154:6836","nonce":4278807008}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-09-23T03:18:05.710 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 31 2023-09-23T03:18:05.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.711+0000 7f0ff63a9640 1 -- 172.21.15.154:0/2731553310 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0fd006af30 msgr2=0x7f0fd006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:05.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.711+0000 7f0ff63a9640 1 --2- 172.21.15.154:0/2731553310 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0fd006af30 0x7f0fd006d3f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0ff00b1990 tx=0x7f0fd8009040 comp rx=0 tx=0).stop 2023-09-23T03:18:05.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.711+0000 7f0ff63a9640 1 -- 172.21.15.154:0/2731553310 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ff00acce0 msgr2=0x7f0ff0097b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:05.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.711+0000 7f0ff63a9640 1 --2- 172.21.15.154:0/2731553310 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ff00acce0 0x7f0ff0097b00 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0000c00 tx=0x7f0fe00046c0 comp rx=0 tx=0).stop 2023-09-23T03:18:05.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.712+0000 7f0ff63a9640 1 -- 172.21.15.154:0/2731553310 shutdown_connections 2023-09-23T03:18:05.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.712+0000 7f0ff63a9640 1 --2- 172.21.15.154:0/2731553310 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0fd006af30 0x7f0fd006d3f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:05.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.712+0000 7f0ff63a9640 1 --2- 172.21.15.154:0/2731553310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ff00cf040 0x7f0ff01f4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:05.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.712+0000 7f0ff63a9640 1 --2- 172.21.15.154:0/2731553310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ff00b1b20 0x7f0ff015add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:05.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.712+0000 7f0ff63a9640 1 --2- 172.21.15.154:0/2731553310 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ff00acce0 0x7f0ff0097b00 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:05.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.712+0000 7f0ff63a9640 1 -- 172.21.15.154:0/2731553310 >> 172.21.15.154:0/2731553310 conn(0x7f0ff006a0f0 msgr2=0x7f0ff00ce900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:05.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.712+0000 7f0ff63a9640 1 -- 172.21.15.154:0/2731553310 shutdown_connections 2023-09-23T03:18:05.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:05.712+0000 7f0ff63a9640 1 -- 172.21.15.154:0/2731553310 wait complete. 2023-09-23T03:18:06.134 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 31, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-23T03:17:56.221314+0000', 'modified': '2023-09-23T03:18:04.535861+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 46, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5988}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5988': {'gid': 5988, 'name': 'a', 'rank': 0, 'incarnation': 29, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.154:6836/4278807008', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6834', 'nonce': 4278807008}, {'type': 'v1', 'addr': '172.21.15.154:6836', 'nonce': 4278807008}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6, 7], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-23T03:18:06.134 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5988:up:rejoin 2023-09-23T03:18:07.135 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:18:07.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.365+0000 7f5e81f25640 1 Processor -- start 2023-09-23T03:18:07.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.365+0000 7f5e81f25640 1 -- start start 2023-09-23T03:18:07.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.366+0000 7f5e81f25640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5e7c0ada30 0x7f5e7c0ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:07.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.366+0000 7f5e81f25640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e7c0cf130 0x7f5e7c15d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:07.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.366+0000 7f5e81f25640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e7c06ceb0 con 0x7f5e7c0b2fe0 2023-09-23T03:18:07.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.366+0000 7f5e81f25640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e7c0ade10 con 0x7f5e7c0ada30 2023-09-23T03:18:07.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.366+0000 7f5e81f25640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c159c90 con 0x7f5e7c0cf130 2023-09-23T03:18:07.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.367+0000 7f5e7bfff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e7c0cf130 0x7f5e7c15d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:07.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.367+0000 7f5e7b7fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5e7c0ada30 0x7f5e7c0ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:07.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.367+0000 7f5e7bfff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e7c0cf130 0x7f5e7c15d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:60552/0 (socket says 172.21.15.154:60552) 2023-09-23T03:18:07.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.367+0000 7f5e7bfff640 1 -- 172.21.15.154:0/4102078790 learned_addr learned my addr 172.21.15.154:0/4102078790 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:07.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.368+0000 7f5e7a7fc640 1 -- 172.21.15.154:0/4102078790 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3981958533 0 0) 0x7f5e64001110 con 0x7f5e7c0b2fe0 2023-09-23T03:18:07.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.368+0000 7f5e7a7fc640 1 -- 172.21.15.154:0/4102078790 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e68003520 con 0x7f5e7c0b2fe0 2023-09-23T03:18:07.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.368+0000 7f5e7bfff640 1 -- 172.21.15.154:0/4102078790 >> v2:172.21.15.190:3300/0 conn(0x7f5e7c0ada30 msgr2=0x7f5e7c0ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:07.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.368+0000 7f5e7bfff640 1 --2- 172.21.15.154:0/4102078790 >> v2:172.21.15.190:3300/0 conn(0x7f5e7c0ada30 0x7f5e7c0ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:07.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.368+0000 7f5e7bfff640 1 -- 172.21.15.154:0/4102078790 >> v1:172.21.15.190:6789/0 conn(0x7f5e7c0b2fe0 legacy=0x7f5e7c0cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:07.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.368+0000 7f5e7bfff640 1 -- 172.21.15.154:0/4102078790 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e7c06c110 con 0x7f5e7c0cf130 2023-09-23T03:18:07.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.368+0000 7f5e7b7fe640 1 --2- 172.21.15.154:0/4102078790 >> v2:172.21.15.190:3300/0 conn(0x7f5e7c0ada30 0x7f5e7c0ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:18:07.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.369+0000 7f5e7bfff640 1 --2- 172.21.15.154:0/4102078790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e7c0cf130 0x7f5e7c15d380 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5e7400d150 tx=0x7f5e7402e160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92a641be77b5de89 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:07.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.370+0000 7f5e7a7fc640 1 -- 172.21.15.154:0/4102078790 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7400b0d0 con 0x7f5e7c0cf130 2023-09-23T03:18:07.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.370+0000 7f5e7a7fc640 1 -- 172.21.15.154:0/4102078790 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7400b230 con 0x7f5e7c0cf130 2023-09-23T03:18:07.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.370+0000 7f5e7a7fc640 1 -- 172.21.15.154:0/4102078790 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e74009600 con 0x7f5e7c0cf130 2023-09-23T03:18:07.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.371+0000 7f5e81f25640 1 -- 172.21.15.154:0/4102078790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e7c0cf130 msgr2=0x7f5e7c15d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:07.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.371+0000 7f5e81f25640 1 --2- 172.21.15.154:0/4102078790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e7c0cf130 0x7f5e7c15d380 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5e7400d150 tx=0x7f5e7402e160 comp rx=0 tx=0).stop 2023-09-23T03:18:07.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.371+0000 7f5e81f25640 1 -- 172.21.15.154:0/4102078790 shutdown_connections 2023-09-23T03:18:07.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.371+0000 7f5e81f25640 1 --2- 172.21.15.154:0/4102078790 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e7c0cf130 0x7f5e7c15d380 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:07.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.371+0000 7f5e81f25640 1 --2- 172.21.15.154:0/4102078790 >> v2:172.21.15.190:3300/0 conn(0x7f5e7c0ada30 0x7f5e7c0ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:07.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.371+0000 7f5e81f25640 1 -- 172.21.15.154:0/4102078790 >> 172.21.15.154:0/4102078790 conn(0x7f5e7c06a0f0 msgr2=0x7f5e7c06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:07.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.371+0000 7f5e81f25640 1 -- 172.21.15.154:0/4102078790 shutdown_connections 2023-09-23T03:18:07.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.371+0000 7f5e81f25640 1 -- 172.21.15.154:0/4102078790 wait complete. 2023-09-23T03:18:07.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.373+0000 7f5e81f25640 1 Processor -- start 2023-09-23T03:18:07.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.373+0000 7f5e81f25640 1 -- start start 2023-09-23T03:18:07.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.374+0000 7f5e81f25640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e7c0ada30 0x7f5e7c15c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:07.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.374+0000 7f5e81f25640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e7c0b2fe0 0x7f5e7c0ac370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:07.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.374+0000 7f5e81f25640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e7c0cf130 0x7f5e7c1f4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:07.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.374+0000 7f5e81f25640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c0c2180 con 0x7f5e7c0cf130 2023-09-23T03:18:07.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.374+0000 7f5e81f25640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c146840 con 0x7f5e7c0ada30 2023-09-23T03:18:07.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.374+0000 7f5e81f25640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c15cd20 con 0x7f5e7c0b2fe0 2023-09-23T03:18:07.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.375+0000 7f5e7bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e7c0cf130 0x7f5e7c1f4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:07.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.375+0000 7f5e7bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e7c0cf130 0x7f5e7c1f4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56726/0 (socket says 172.21.15.154:56726) 2023-09-23T03:18:07.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.375+0000 7f5e7bfff640 1 -- 172.21.15.154:0/3742032062 learned_addr learned my addr 172.21.15.154:0/3742032062 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:07.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.375+0000 7f5e7affd640 1 --2- 172.21.15.154:0/3742032062 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e7c0b2fe0 0x7f5e7c0ac370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:07.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.375+0000 7f5e7b7fe640 1 --2- 172.21.15.154:0/3742032062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e7c0ada30 0x7f5e7c15c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:07.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.375+0000 7f5e7bfff640 1 -- 172.21.15.154:0/3742032062 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e7c0b2fe0 msgr2=0x7f5e7c0ac370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:07.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.375+0000 7f5e7bfff640 1 --2- 172.21.15.154:0/3742032062 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e7c0b2fe0 0x7f5e7c0ac370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:07.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.375+0000 7f5e7bfff640 1 -- 172.21.15.154:0/3742032062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e7c0ada30 msgr2=0x7f5e7c15c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:07.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.375+0000 7f5e7bfff640 1 --2- 172.21.15.154:0/3742032062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e7c0ada30 0x7f5e7c15c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:07.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.375+0000 7f5e7bfff640 1 -- 172.21.15.154:0/3742032062 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e6c0093a0 con 0x7f5e7c0cf130 2023-09-23T03:18:07.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.375+0000 7f5e7b7fe640 1 --2- 172.21.15.154:0/3742032062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e7c0ada30 0x7f5e7c15c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:18:07.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.376+0000 7f5e7bfff640 1 --2- 172.21.15.154:0/3742032062 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e7c0cf130 0x7f5e7c1f4f20 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5e74038040 tx=0x7f5e74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:07.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.376+0000 7f5e73fff640 1 -- 172.21.15.154:0/3742032062 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7403f640 con 0x7f5e7c0cf130 2023-09-23T03:18:07.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.376+0000 7f5e73fff640 1 -- 172.21.15.154:0/3742032062 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7403f7a0 con 0x7f5e7c0cf130 2023-09-23T03:18:07.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.376+0000 7f5e81f25640 1 -- 172.21.15.154:0/3742032062 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e7400bd40 con 0x7f5e7c0cf130 2023-09-23T03:18:07.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.376+0000 7f5e73fff640 1 -- 172.21.15.154:0/3742032062 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7403fb50 con 0x7f5e7c0cf130 2023-09-23T03:18:07.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.376+0000 7f5e81f25640 1 -- 172.21.15.154:0/3742032062 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e7c146280 con 0x7f5e7c0cf130 2023-09-23T03:18:07.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.377+0000 7f5e81f25640 1 -- 172.21.15.154:0/3742032062 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e7c0c1ee0 con 0x7f5e7c0cf130 2023-09-23T03:18:07.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.378+0000 7f5e73fff640 1 -- 172.21.15.154:0/3742032062 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f5e7403fcb0 con 0x7f5e7c0cf130 2023-09-23T03:18:07.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.379+0000 7f5e73fff640 1 --2- 172.21.15.154:0/3742032062 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5e5c06af30 0x7f5e5c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:07.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.379+0000 7f5e73fff640 1 -- 172.21.15.154:0/3742032062 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6720+0+0 (secure 0 0 0) 0x7f5e740ace50 con 0x7f5e7c0cf130 2023-09-23T03:18:07.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.380+0000 7f5e7b7fe640 1 --2- 172.21.15.154:0/3742032062 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5e5c06af30 0x7f5e5c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:07.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.380+0000 7f5e7b7fe640 1 --2- 172.21.15.154:0/3742032062 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5e5c06af30 0x7f5e5c06d3f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c009ba0 tx=0x7f5e6c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:07.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.385+0000 7f5e73fff640 1 -- 172.21.15.154:0/3742032062 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f5e7c0c1ee0 con 0x7f5e7c0cf130 2023-09-23T03:18:07.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.612+0000 7f5e81f25640 1 -- 172.21.15.154:0/3742032062 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e7c0c22c0 con 0x7f5e7c0cf130 2023-09-23T03:18:07.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.613+0000 7f5e73fff640 1 -- 172.21.15.154:0/3742032062 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4928 (secure 0 0 0) 0x7f5e7c0c22c0 con 0x7f5e7c0cf130 2023-09-23T03:18:07.614 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:18:07.614 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":32,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5215,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3388818833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3388818833},{"type":"v1","addr":"172.21.15.190:6835","nonce":3388818833}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5991,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1964552060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1964552060},{"type":"v1","addr":"172.21.15.154:6837","nonce":1964552060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5994,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/1736999297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1736999297},{"type":"v1","addr":"172.21.15.190:6834","nonce":1736999297}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:17:56.221314+0000","modified":"2023-09-23T03:18:05.544508+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5988},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5988":{"gid":5988,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":4,"addr":"172.21.15.154:6836/4278807008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4278807008},{"type":"v1","addr":"172.21.15.154:6836","nonce":4278807008}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-09-23T03:18:07.614 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 32 2023-09-23T03:18:07.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.616+0000 7f5e81f25640 1 -- 172.21.15.154:0/3742032062 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5e5c06af30 msgr2=0x7f5e5c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:07.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.616+0000 7f5e81f25640 1 --2- 172.21.15.154:0/3742032062 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5e5c06af30 0x7f5e5c06d3f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c009ba0 tx=0x7f5e6c008040 comp rx=0 tx=0).stop 2023-09-23T03:18:07.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.616+0000 7f5e81f25640 1 -- 172.21.15.154:0/3742032062 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e7c0cf130 msgr2=0x7f5e7c1f4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:07.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.616+0000 7f5e81f25640 1 --2- 172.21.15.154:0/3742032062 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e7c0cf130 0x7f5e7c1f4f20 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5e74038040 tx=0x7f5e74004a40 comp rx=0 tx=0).stop 2023-09-23T03:18:07.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.616+0000 7f5e81f25640 1 -- 172.21.15.154:0/3742032062 shutdown_connections 2023-09-23T03:18:07.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.616+0000 7f5e81f25640 1 --2- 172.21.15.154:0/3742032062 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5e5c06af30 0x7f5e5c06d3f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:07.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.616+0000 7f5e81f25640 1 --2- 172.21.15.154:0/3742032062 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e7c0cf130 0x7f5e7c1f4f20 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:07.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.616+0000 7f5e81f25640 1 --2- 172.21.15.154:0/3742032062 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5e7c0b2fe0 0x7f5e7c0ac370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:07.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.616+0000 7f5e81f25640 1 --2- 172.21.15.154:0/3742032062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5e7c0ada30 0x7f5e7c15c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:07.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.616+0000 7f5e81f25640 1 -- 172.21.15.154:0/3742032062 >> 172.21.15.154:0/3742032062 conn(0x7f5e7c06a0f0 msgr2=0x7f5e7c0ccea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:07.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.616+0000 7f5e81f25640 1 -- 172.21.15.154:0/3742032062 shutdown_connections 2023-09-23T03:18:07.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:07.616+0000 7f5e81f25640 1 -- 172.21.15.154:0/3742032062 wait complete. 2023-09-23T03:18:08.037 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-23T03:17:56.221314+0000', 'modified': '2023-09-23T03:18:05.544508+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 46, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5988}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5988': {'gid': 5988, 'name': 'a', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.154:6836/4278807008', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6834', 'nonce': 4278807008}, {'type': 'v1', 'addr': '172.21.15.154:6836', 'nonce': 4278807008}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6, 7], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-23T03:18:08.037 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-23T03:18:08.038 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-09-23T03:18:08.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.265+0000 7f54449a9640 1 Processor -- start 2023-09-23T03:18:08.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.265+0000 7f54449a9640 1 -- start start 2023-09-23T03:18:08.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.266+0000 7f54449a9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54400b1b20 0x7f54400ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:08.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.266+0000 7f54449a9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f54400cee70 0x7f54400ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:08.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.266+0000 7f54449a9640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f544015ac30 con 0x7f54400acce0 2023-09-23T03:18:08.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.266+0000 7f54449a9640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54400b2e90 con 0x7f54400cee70 2023-09-23T03:18:08.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.266+0000 7f54449a9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54400cf320 con 0x7f54400b1b20 2023-09-23T03:18:08.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.267+0000 7f543e59c640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f54400acce0 0x7f544015ed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:52852/0 (socket says 172.21.15.154:52852) 2023-09-23T03:18:08.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.267+0000 7f543e59c640 1 -- 172.21.15.154:0/336078673 learned_addr learned my addr 172.21.15.154:0/336078673 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:18:08.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.267+0000 7f543ed9d640 1 --2- 172.21.15.154:0/336078673 >> v2:172.21.15.154:3300/0 conn(0x7f54400cee70 0x7f54400ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:08.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.267+0000 7f543dd9b640 1 --2- 172.21.15.154:0/336078673 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54400b1b20 0x7f54400ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:08.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.268+0000 7f543ed9d640 1 -- 172.21.15.154:0/336078673 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54400b1b20 msgr2=0x7f54400ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:08.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.268+0000 7f543ed9d640 1 --2- 172.21.15.154:0/336078673 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54400b1b20 0x7f54400ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.268+0000 7f543ed9d640 1 -- 172.21.15.154:0/336078673 >> v1:172.21.15.154:6789/0 conn(0x7f54400acce0 legacy=0x7f544015ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:08.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.268+0000 7f543ed9d640 1 -- 172.21.15.154:0/336078673 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54400b1f70 con 0x7f54400cee70 2023-09-23T03:18:08.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.269+0000 7f543ed9d640 1 --2- 172.21.15.154:0/336078673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54400cee70 0x7f54400ab140 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5438009d90 tx=0x7f543802be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4da563a913cfa018 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:08.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.271+0000 7f543d59a640 1 -- 172.21.15.154:0/336078673 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543802cbc0 con 0x7f54400cee70 2023-09-23T03:18:08.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.272+0000 7f543d59a640 1 -- 172.21.15.154:0/336078673 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f543802cd20 con 0x7f54400cee70 2023-09-23T03:18:08.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.272+0000 7f543d59a640 1 -- 172.21.15.154:0/336078673 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5438034660 con 0x7f54400cee70 2023-09-23T03:18:08.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.272+0000 7f54449a9640 1 -- 172.21.15.154:0/336078673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54400cee70 msgr2=0x7f54400ab140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:08.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.272+0000 7f54449a9640 1 --2- 172.21.15.154:0/336078673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54400cee70 0x7f54400ab140 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5438009d90 tx=0x7f543802be10 comp rx=0 tx=0).stop 2023-09-23T03:18:08.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.272+0000 7f54449a9640 1 -- 172.21.15.154:0/336078673 shutdown_connections 2023-09-23T03:18:08.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.272+0000 7f54449a9640 1 --2- 172.21.15.154:0/336078673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54400cee70 0x7f54400ab140 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.272+0000 7f54449a9640 1 --2- 172.21.15.154:0/336078673 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54400b1b20 0x7f54400ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.272+0000 7f54449a9640 1 -- 172.21.15.154:0/336078673 >> 172.21.15.154:0/336078673 conn(0x7f544006a0f0 msgr2=0x7f544006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:08.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.273+0000 7f54449a9640 1 -- 172.21.15.154:0/336078673 shutdown_connections 2023-09-23T03:18:08.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.273+0000 7f54449a9640 1 -- 172.21.15.154:0/336078673 wait complete. 2023-09-23T03:18:08.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.274+0000 7f54449a9640 1 Processor -- start 2023-09-23T03:18:08.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.275+0000 7f54449a9640 1 -- start start 2023-09-23T03:18:08.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.276+0000 7f54449a9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54400acce0 0x7f5440154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:08.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.276+0000 7f54449a9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54400b1b20 0x7f544015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:08.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.276+0000 7f54449a9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54400cee70 0x7f5440159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:08.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.276+0000 7f54449a9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54400c7cc0 con 0x7f54400acce0 2023-09-23T03:18:08.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.276+0000 7f54449a9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54401549d0 con 0x7f54400cee70 2023-09-23T03:18:08.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.276+0000 7f54449a9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54400c8150 con 0x7f54400b1b20 2023-09-23T03:18:08.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.276+0000 7f543e59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54400acce0 0x7f5440154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:08.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.276+0000 7f543e59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54400acce0 0x7f5440154490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56754/0 (socket says 172.21.15.154:56754) 2023-09-23T03:18:08.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.276+0000 7f543e59c640 1 -- 172.21.15.154:0/2565376065 learned_addr learned my addr 172.21.15.154:0/2565376065 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:08.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.276+0000 7f543dd9b640 1 --2- 172.21.15.154:0/2565376065 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54400b1b20 0x7f544015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:08.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.276+0000 7f543e59c640 1 -- 172.21.15.154:0/2565376065 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54400b1b20 msgr2=0x7f544015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:08.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.276+0000 7f543ed9d640 1 --2- 172.21.15.154:0/2565376065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54400cee70 0x7f5440159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:08.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.276+0000 7f543e59c640 1 --2- 172.21.15.154:0/2565376065 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54400b1b20 0x7f544015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.276+0000 7f543e59c640 1 -- 172.21.15.154:0/2565376065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54400cee70 msgr2=0x7f5440159ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:08.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.277+0000 7f543e59c640 1 --2- 172.21.15.154:0/2565376065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54400cee70 0x7f5440159ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.277+0000 7f543e59c640 1 -- 172.21.15.154:0/2565376065 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5438009a70 con 0x7f54400acce0 2023-09-23T03:18:08.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.277+0000 7f543dd9b640 1 --2- 172.21.15.154:0/2565376065 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54400b1b20 0x7f544015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:18:08.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.277+0000 7f543e59c640 1 --2- 172.21.15.154:0/2565376065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54400acce0 0x7f5440154490 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f543000c4b0 tx=0x7f543000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:08.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.277+0000 7f542effd640 1 -- 172.21.15.154:0/2565376065 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5430007480 con 0x7f54400acce0 2023-09-23T03:18:08.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.277+0000 7f542effd640 1 -- 172.21.15.154:0/2565376065 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54300075e0 con 0x7f54400acce0 2023-09-23T03:18:08.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.277+0000 7f542effd640 1 -- 172.21.15.154:0/2565376065 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54300059c0 con 0x7f54400acce0 2023-09-23T03:18:08.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.277+0000 7f54449a9640 1 -- 172.21.15.154:0/2565376065 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54400c70b0 con 0x7f54400acce0 2023-09-23T03:18:08.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.277+0000 7f54449a9640 1 -- 172.21.15.154:0/2565376065 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54400c7670 con 0x7f54400acce0 2023-09-23T03:18:08.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.280+0000 7f542effd640 1 -- 172.21.15.154:0/2565376065 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f5430005b20 con 0x7f54400acce0 2023-09-23T03:18:08.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.281+0000 7f542effd640 1 --2- 172.21.15.154:0/2565376065 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f542006af30 0x7f542006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:08.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.282+0000 7f542effd640 1 -- 172.21.15.154:0/2565376065 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6720+0+0 (secure 0 0 0) 0x7f5430088e50 con 0x7f54400acce0 2023-09-23T03:18:08.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.282+0000 7f543dd9b640 1 --2- 172.21.15.154:0/2565376065 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f542006af30 0x7f542006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:08.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.282+0000 7f5447613740 1 Processor -- start 2023-09-23T03:18:08.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.282+0000 7f5447613740 1 -- start start 2023-09-23T03:18:08.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.283+0000 7f543dd9b640 1 --2- 172.21.15.154:0/2565376065 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f542006af30 0x7f542006d3f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f544015a930 tx=0x7f5434008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:08.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.283+0000 7f5447613740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ab2419150 0x562ab242cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:08.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.283+0000 7f5447613740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562ab2419530 0x562ab241b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:08.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.283+0000 7f5447613740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ab241bed0 0x562ab241c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:08.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.283+0000 7f5447613740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ab235f6d0 con 0x562ab2419150 2023-09-23T03:18:08.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.283+0000 7f5447613740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ab235cdc0 con 0x562ab241bed0 2023-09-23T03:18:08.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.283+0000 7f5447613740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ab23586b0 con 0x562ab2419530 2023-09-23T03:18:08.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.284+0000 7f543e59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ab2419150 0x562ab242cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:08.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.284+0000 7f543e59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ab2419150 0x562ab242cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56770/0 (socket says 172.21.15.154:56770) 2023-09-23T03:18:08.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.284+0000 7f543e59c640 1 -- 172.21.15.154:0/3402155920 learned_addr learned my addr 172.21.15.154:0/3402155920 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:08.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.284+0000 7f543dd9b640 1 --2- 172.21.15.154:0/3402155920 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562ab2419530 0x562ab241b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:08.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.284+0000 7f543e59c640 1 -- 172.21.15.154:0/3402155920 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562ab2419530 msgr2=0x562ab241b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:08.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.284+0000 7f543e59c640 1 --2- 172.21.15.154:0/3402155920 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562ab2419530 0x562ab241b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.284+0000 7f543e59c640 1 -- 172.21.15.154:0/3402155920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ab241bed0 msgr2=0x562ab241c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:08.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.284+0000 7f543ed9d640 1 --2- 172.21.15.154:0/3402155920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ab241bed0 0x562ab241c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:08.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.284+0000 7f543e59c640 1 --2- 172.21.15.154:0/3402155920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ab241bed0 0x562ab241c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.284+0000 7f543e59c640 1 -- 172.21.15.154:0/3402155920 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ab231c8e0 con 0x562ab2419150 2023-09-23T03:18:08.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.284+0000 7f543e59c640 1 --2- 172.21.15.154:0/3402155920 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ab2419150 0x562ab242cf60 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f543005f680 tx=0x7f54300579f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c8ef367683bfd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:08.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.285+0000 7f541ffff640 1 -- 172.21.15.154:0/3402155920 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54300559f0 con 0x562ab2419150 2023-09-23T03:18:08.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.285+0000 7f541ffff640 1 -- 172.21.15.154:0/3402155920 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5430055c50 con 0x562ab2419150 2023-09-23T03:18:08.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.285+0000 7f541ffff640 1 -- 172.21.15.154:0/3402155920 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5430058be0 con 0x562ab2419150 2023-09-23T03:18:08.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.285+0000 7f5447613740 1 -- 172.21.15.154:0/3402155920 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ab2419150 msgr2=0x562ab242cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:08.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.285+0000 7f5447613740 1 --2- 172.21.15.154:0/3402155920 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ab2419150 0x562ab242cf60 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f543005f680 tx=0x7f54300579f0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.285+0000 7f5447613740 1 -- 172.21.15.154:0/3402155920 shutdown_connections 2023-09-23T03:18:08.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.285+0000 7f5447613740 1 --2- 172.21.15.154:0/3402155920 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ab241bed0 0x562ab241c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.285+0000 7f5447613740 1 --2- 172.21.15.154:0/3402155920 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562ab2419530 0x562ab241b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.285+0000 7f5447613740 1 --2- 172.21.15.154:0/3402155920 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ab2419150 0x562ab242cf60 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.286+0000 7f5447613740 1 -- 172.21.15.154:0/3402155920 >> 172.21.15.154:0/3402155920 conn(0x562ab23ede10 msgr2=0x562ab2436990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:08.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.286+0000 7f5447613740 1 -- 172.21.15.154:0/3402155920 shutdown_connections 2023-09-23T03:18:08.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.286+0000 7f5447613740 1 -- 172.21.15.154:0/3402155920 wait complete. 2023-09-23T03:18:08.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.286+0000 7f5447613740 1 -- start start 2023-09-23T03:18:08.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.286+0000 7f5447613740 1 Processor -- start 2023-09-23T03:18:08.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.287+0000 7f5447613740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ab2419530 0x562ab2419f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:08.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.287+0000 7f5447613740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ab241bed0 0x562ab2428ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:08.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.288+0000 7f5447613740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562ab2402db0 0x562ab2403230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:08.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.288+0000 7f5447613740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ab22f6940 con 0x562ab241bed0 2023-09-23T03:18:08.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.288+0000 7f5447613740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ab22f65e0 con 0x562ab2419530 2023-09-23T03:18:08.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.288+0000 7f5447613740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ab20bcf20 con 0x562ab2402db0 2023-09-23T03:18:08.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.288+0000 7f5447613740 10 client.0 _ll_register_callbacks cb 0x562ab23345e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:18:08.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.288+0000 7f543dd9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ab241bed0 0x562ab2428ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:08.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.288+0000 7f543dd9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ab241bed0 0x562ab2428ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56776/0 (socket says 172.21.15.154:56776) 2023-09-23T03:18:08.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.288+0000 7f543dd9b640 1 -- 172.21.15.154:0/1193373320 learned_addr learned my addr 172.21.15.154:0/1193373320 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:08.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.288+0000 7f543e59c640 1 --2- 172.21.15.154:0/1193373320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ab2419530 0x562ab2419f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:08.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.288+0000 7f543ed9d640 1 --2- 172.21.15.154:0/1193373320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562ab2402db0 0x562ab2403230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:08.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.288+0000 7f543dd9b640 1 -- 172.21.15.154:0/1193373320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562ab2402db0 msgr2=0x562ab2403230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:08.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.288+0000 7f543dd9b640 1 --2- 172.21.15.154:0/1193373320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562ab2402db0 0x562ab2403230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.288+0000 7f543dd9b640 1 -- 172.21.15.154:0/1193373320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ab2419530 msgr2=0x562ab2419f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:08.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.288+0000 7f543dd9b640 1 --2- 172.21.15.154:0/1193373320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ab2419530 0x562ab2419f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.288+0000 7f543dd9b640 1 -- 172.21.15.154:0/1193373320 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54400c7670 con 0x562ab241bed0 2023-09-23T03:18:08.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.289+0000 7f543dd9b640 1 --2- 172.21.15.154:0/1193373320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ab241bed0 0x562ab2428ad0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f543400ea70 tx=0x7f54340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c5734f6514fd17a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:08.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.289+0000 7f541d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:18:08.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.289+0000 7f541d7fa640 1 -- 172.21.15.154:0/1193373320 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54340044e0 con 0x562ab241bed0 2023-09-23T03:18:08.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.289+0000 7f5447613740 1 -- 172.21.15.154:0/1193373320 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562ab2351690 con 0x562ab241bed0 2023-09-23T03:18:08.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.289+0000 7f541d7fa640 1 -- 172.21.15.154:0/1193373320 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6720+0+0 (secure 0 0 0) 0x7f5434009e10 con 0x562ab241bed0 2023-09-23T03:18:08.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.289+0000 7f541d7fa640 1 -- 172.21.15.154:0/1193373320 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5434013430 con 0x562ab241bed0 2023-09-23T03:18:08.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.289+0000 7f5447613740 10 client.6030 fetch_fsmap learned FSMap version 32 2023-09-23T03:18:08.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.289+0000 7f5447613740 1 -- 172.21.15.154:0/1193373320 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x562ab2356d30 con 0x562ab241bed0 2023-09-23T03:18:08.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.289+0000 7f541d7fa640 1 -- 172.21.15.154:0/1193373320 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562ab2351690 con 0x562ab241bed0 2023-09-23T03:18:08.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.290+0000 7f541d7fa640 1 -- 172.21.15.154:0/1193373320 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 32) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f5434027250 con 0x562ab241bed0 2023-09-23T03:18:08.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.290+0000 7f5447613740 10 client.6030 fetch_fsmap finished waiting for FSMap version 32 2023-09-23T03:18:08.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.290+0000 7f5447613740 10 client.6030 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-23T03:18:08.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.290+0000 7f5447613740 20 client.6030 populate_metadata read hostname 'smithi154' 2023-09-23T03:18:08.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.290+0000 7f5447613740 1 --2- 172.21.15.154:0/1193373320 >> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] conn(0x562ab24154f0 0x562ab24158d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:08.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.290+0000 7f5447613740 4 client.6030 mds_command: new command op to 5988 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T03:18:08.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.290+0000 7f5447613740 1 -- 172.21.15.154:0/1193373320 --> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ab23513b0 con 0x562ab24154f0 2023-09-23T03:18:08.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.290+0000 7f543e59c640 1 --2- 172.21.15.154:0/1193373320 >> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] conn(0x562ab24154f0 0x562ab24158d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:08.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.290+0000 7f543e59c640 1 --2- 172.21.15.154:0/1193373320 >> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] conn(0x562ab24154f0 0x562ab24158d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=479558919d7f5e95 server_cookie=9d49b481aeaf2076 in_seq=0 out_seq=0 2023-09-23T03:18:08.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.290+0000 7f541d7fa640 10 client.6030 ms_handle_connect on v2:172.21.15.154:6834/4278807008 2023-09-23T03:18:08.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.291+0000 7f541d7fa640 1 -- 172.21.15.154:0/1193373320 <== mds.0 v2:172.21.15.154:6834/4278807008 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562ab23513b0 con 0x562ab24154f0 2023-09-23T03:18:08.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.291+0000 7f541d7fa640 10 client.6030 handle_command_reply: tid=0 2023-09-23T03:18:08.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.291+0000 7f5447613740 1 client.6030 shutdown 2023-09-23T03:18:08.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.291+0000 7f5447613740 1 -- 172.21.15.154:0/1193373320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ab241bed0 msgr2=0x562ab2428ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:08.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.291+0000 7f5447613740 1 --2- 172.21.15.154:0/1193373320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ab241bed0 0x562ab2428ad0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f543400ea70 tx=0x7f54340046c0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.292+0000 7f5447613740 1 -- 172.21.15.154:0/1193373320 shutdown_connections 2023-09-23T03:18:08.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.292+0000 7f5447613740 1 --2- 172.21.15.154:0/1193373320 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562ab2402db0 0x562ab2403230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.292+0000 7f5447613740 1 --2- 172.21.15.154:0/1193373320 >> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] conn(0x562ab24154f0 0x562ab24158d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.292+0000 7f5447613740 1 --2- 172.21.15.154:0/1193373320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ab241bed0 0x562ab2428ad0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.292+0000 7f541d7fa640 0 client.6030 ms_handle_reset on v2:172.21.15.154:6834/4278807008 2023-09-23T03:18:08.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.292+0000 7f5447613740 1 --2- 172.21.15.154:0/1193373320 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ab2419530 0x562ab2419f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.292+0000 7f5447613740 1 -- 172.21.15.154:0/1193373320 >> 172.21.15.154:0/1193373320 conn(0x562ab2419150 msgr2=0x562ab23f1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:08.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.292+0000 7f5447613740 1 -- 172.21.15.154:0/1193373320 shutdown_connections 2023-09-23T03:18:08.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.292+0000 7f5447613740 1 -- 172.21.15.154:0/1193373320 wait complete. 2023-09-23T03:18:08.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.292+0000 7f5447613740 20 client.6030 trim_cache size 0 max 16384 2023-09-23T03:18:08.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.308+0000 7f5447613740 1 Processor -- start 2023-09-23T03:18:08.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.308+0000 7f5447613740 1 -- start start 2023-09-23T03:18:08.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.309+0000 7f5447613740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ab2402db0 0x562ab23f0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:08.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.309+0000 7f5447613740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562ab2419530 0x562ab2415c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:08.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.309+0000 7f5447613740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ab241bed0 0x562ab23ef8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:08.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.309+0000 7f5447613740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ab242c530 con 0x562ab2402db0 2023-09-23T03:18:08.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.309+0000 7f5447613740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ab2417350 con 0x562ab241bed0 2023-09-23T03:18:08.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.309+0000 7f5447613740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ab2416170 con 0x562ab2419530 2023-09-23T03:18:08.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.310+0000 7f543e59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ab2402db0 0x562ab23f0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:08.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.310+0000 7f543e59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ab2402db0 0x562ab23f0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56786/0 (socket says 172.21.15.154:56786) 2023-09-23T03:18:08.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.310+0000 7f543e59c640 1 -- 172.21.15.154:0/2771251676 learned_addr learned my addr 172.21.15.154:0/2771251676 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:08.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.310+0000 7f543e59c640 1 -- 172.21.15.154:0/2771251676 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562ab2419530 msgr2=0x562ab2415c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:08.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.310+0000 7f543e59c640 1 --2- 172.21.15.154:0/2771251676 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562ab2419530 0x562ab2415c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.310+0000 7f543e59c640 1 -- 172.21.15.154:0/2771251676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ab241bed0 msgr2=0x562ab23ef8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:08.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.310+0000 7f543e59c640 1 --2- 172.21.15.154:0/2771251676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ab241bed0 0x562ab23ef8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.310+0000 7f543e59c640 1 -- 172.21.15.154:0/2771251676 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ab22f5b50 con 0x562ab2402db0 2023-09-23T03:18:08.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.310+0000 7f543e59c640 1 --2- 172.21.15.154:0/2771251676 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ab2402db0 0x562ab23f0420 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f543000ce70 tx=0x7f5430014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f3c4ef1e3d716d5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:08.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.310+0000 7f542cff9640 1 -- 172.21.15.154:0/2771251676 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5430076080 con 0x562ab2402db0 2023-09-23T03:18:08.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.310+0000 7f542cff9640 1 -- 172.21.15.154:0/2771251676 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54300761e0 con 0x562ab2402db0 2023-09-23T03:18:08.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.310+0000 7f542cff9640 1 -- 172.21.15.154:0/2771251676 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5430058be0 con 0x562ab2402db0 2023-09-23T03:18:08.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.311+0000 7f5447613740 1 -- 172.21.15.154:0/2771251676 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ab2402db0 msgr2=0x562ab23f0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:08.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.311+0000 7f5447613740 1 --2- 172.21.15.154:0/2771251676 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ab2402db0 0x562ab23f0420 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f543000ce70 tx=0x7f5430014000 comp rx=0 tx=0).stop 2023-09-23T03:18:08.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.311+0000 7f5447613740 1 -- 172.21.15.154:0/2771251676 shutdown_connections 2023-09-23T03:18:08.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.311+0000 7f5447613740 1 --2- 172.21.15.154:0/2771251676 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ab241bed0 0x562ab23ef8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.311+0000 7f5447613740 1 --2- 172.21.15.154:0/2771251676 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562ab2419530 0x562ab2415c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.311+0000 7f5447613740 1 --2- 172.21.15.154:0/2771251676 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ab2402db0 0x562ab23f0420 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.311+0000 7f5447613740 1 -- 172.21.15.154:0/2771251676 >> 172.21.15.154:0/2771251676 conn(0x562ab2419150 msgr2=0x562ab2419d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:08.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.311+0000 7f5447613740 1 -- 172.21.15.154:0/2771251676 shutdown_connections 2023-09-23T03:18:08.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.311+0000 7f5447613740 1 -- 172.21.15.154:0/2771251676 wait complete. 2023-09-23T03:18:08.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.312+0000 7f5447613740 1 -- start start 2023-09-23T03:18:08.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.312+0000 7f5447613740 1 Processor -- start 2023-09-23T03:18:08.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.313+0000 7f5447613740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ab2402db0 0x562ab244d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:08.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.313+0000 7f5447613740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562ab2419530 0x562ab2458530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:08.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.313+0000 7f5447613740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ab241bed0 0x562ab24ab840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:08.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.313+0000 7f5447613740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ab244e1a0 con 0x562ab2402db0 2023-09-23T03:18:08.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.313+0000 7f5447613740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ab23ffb90 con 0x562ab241bed0 2023-09-23T03:18:08.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.313+0000 7f5447613740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x562ab2400680 con 0x562ab2419530 2023-09-23T03:18:08.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.313+0000 7f5447613740 10 client.0 _ll_register_callbacks cb 0x7f54300573f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:18:08.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.313+0000 7f543e59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ab2402db0 0x562ab244d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:08.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.313+0000 7f543e59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ab2402db0 0x562ab244d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56794/0 (socket says 172.21.15.154:56794) 2023-09-23T03:18:08.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.313+0000 7f543e59c640 1 -- 172.21.15.154:0/463528523 learned_addr learned my addr 172.21.15.154:0/463528523 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:08.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.314+0000 7f543dd9b640 1 --2- 172.21.15.154:0/463528523 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562ab2419530 0x562ab2458530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:08.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.314+0000 7f543ed9d640 1 --2- 172.21.15.154:0/463528523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ab241bed0 0x562ab24ab840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:08.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.314+0000 7f543e59c640 1 -- 172.21.15.154:0/463528523 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562ab2419530 msgr2=0x562ab2458530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:08.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.314+0000 7f543e59c640 1 --2- 172.21.15.154:0/463528523 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562ab2419530 0x562ab2458530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.314+0000 7f543e59c640 1 -- 172.21.15.154:0/463528523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ab241bed0 msgr2=0x562ab24ab840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:08.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.314+0000 7f543e59c640 1 --2- 172.21.15.154:0/463528523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ab241bed0 0x562ab24ab840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.314+0000 7f543e59c640 1 -- 172.21.15.154:0/463528523 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ab231c8e0 con 0x562ab2402db0 2023-09-23T03:18:08.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.314+0000 7f543e59c640 1 --2- 172.21.15.154:0/463528523 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ab2402db0 0x562ab244d8a0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f5430001000 tx=0x7f5430000f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a351ec7121aee061 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:08.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.314+0000 7f541d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:18:08.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.314+0000 7f541d7fa640 1 -- 172.21.15.154:0/463528523 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54300559f0 con 0x562ab2402db0 2023-09-23T03:18:08.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.314+0000 7f541d7fa640 1 -- 172.21.15.154:0/463528523 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6720+0+0 (secure 0 0 0) 0x7f5430053fc0 con 0x562ab2402db0 2023-09-23T03:18:08.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.314+0000 7f5447613740 1 -- 172.21.15.154:0/463528523 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5430055c50 con 0x562ab2402db0 2023-09-23T03:18:08.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.314+0000 7f541d7fa640 1 -- 172.21.15.154:0/463528523 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5430054180 con 0x562ab2402db0 2023-09-23T03:18:08.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.315+0000 7f5447613740 10 client.6036 fetch_fsmap learned FSMap version 32 2023-09-23T03:18:08.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.315+0000 7f5447613740 1 -- 172.21.15.154:0/463528523 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x562ab231b140 con 0x562ab2402db0 2023-09-23T03:18:08.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.315+0000 7f541d7fa640 1 -- 172.21.15.154:0/463528523 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5430055c50 con 0x562ab2402db0 2023-09-23T03:18:08.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.315+0000 7f541d7fa640 1 -- 172.21.15.154:0/463528523 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 32) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f5430058be0 con 0x562ab2402db0 2023-09-23T03:18:08.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.315+0000 7f5447613740 10 client.6036 fetch_fsmap finished waiting for FSMap version 32 2023-09-23T03:18:08.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.315+0000 7f5447613740 10 client.6036 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-23T03:18:08.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.315+0000 7f5447613740 20 client.6036 populate_metadata read hostname 'smithi154' 2023-09-23T03:18:08.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.315+0000 7f5447613740 1 --2- 172.21.15.154:0/463528523 >> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] conn(0x562ab24a7770 0x562ab2436790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:08.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.315+0000 7f5447613740 4 client.6036 mds_command: new command op to 5988 tid=0[{"prefix": "status"}] 2023-09-23T03:18:08.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.315+0000 7f5447613740 1 -- 172.21.15.154:0/463528523 --> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562ab205a6c0 con 0x562ab24a7770 2023-09-23T03:18:08.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.315+0000 7f543dd9b640 1 --2- 172.21.15.154:0/463528523 >> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] conn(0x562ab24a7770 0x562ab2436790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:08.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.316+0000 7f543dd9b640 1 --2- 172.21.15.154:0/463528523 >> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] conn(0x562ab24a7770 0x562ab2436790 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef7cec0426e60cf2 server_cookie=6b0b32512bfdb475 in_seq=0 out_seq=0 2023-09-23T03:18:08.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.316+0000 7f541d7fa640 10 client.6036 ms_handle_connect on v2:172.21.15.154:6834/4278807008 2023-09-23T03:18:08.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.316+0000 7f541d7fa640 1 -- 172.21.15.154:0/463528523 <== mds.0 v2:172.21.15.154:6834/4278807008 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x562ab205a6c0 con 0x562ab24a7770 2023-09-23T03:18:08.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.316+0000 7f541d7fa640 10 client.6036 handle_command_reply: tid=0 2023-09-23T03:18:08.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.316+0000 7f5447613740 1 client.6036 shutdown 2023-09-23T03:18:08.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.316+0000 7f5447613740 1 -- 172.21.15.154:0/463528523 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ab2402db0 msgr2=0x562ab244d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:08.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.316+0000 7f5447613740 1 --2- 172.21.15.154:0/463528523 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ab2402db0 0x562ab244d8a0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f5430001000 tx=0x7f5430000f70 comp rx=0 tx=0).stop 2023-09-23T03:18:08.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.316+0000 7f5447613740 1 -- 172.21.15.154:0/463528523 shutdown_connections 2023-09-23T03:18:08.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.316+0000 7f5447613740 1 --2- 172.21.15.154:0/463528523 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562ab241bed0 0x562ab24ab840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.316+0000 7f5447613740 1 --2- 172.21.15.154:0/463528523 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562ab2419530 0x562ab2458530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.316+0000 7f5447613740 1 --2- 172.21.15.154:0/463528523 >> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] conn(0x562ab24a7770 0x562ab2436790 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.316+0000 7f5447613740 1 --2- 172.21.15.154:0/463528523 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ab2402db0 0x562ab244d8a0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.316+0000 7f5447613740 1 -- 172.21.15.154:0/463528523 >> 172.21.15.154:0/463528523 conn(0x562ab2419150 msgr2=0x562ab23ecb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:08.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.316+0000 7f541d7fa640 0 client.6036 ms_handle_reset on v2:172.21.15.154:6834/4278807008 2023-09-23T03:18:08.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.316+0000 7f5447613740 1 -- 172.21.15.154:0/463528523 shutdown_connections 2023-09-23T03:18:08.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.316+0000 7f5447613740 1 -- 172.21.15.154:0/463528523 wait complete. 2023-09-23T03:18:08.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.317+0000 7f5447613740 20 client.6036 trim_cache size 0 max 16384 2023-09-23T03:18:08.322 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:18:08.322 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_fsid": "8c8039d3-6c84-43db-8808-85e0dbdd47c8", 2023-09-23T03:18:08.322 INFO:teuthology.orchestra.run.smithi154.stdout: "whoami": 0, 2023-09-23T03:18:08.322 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 5988, 2023-09-23T03:18:08.322 INFO:teuthology.orchestra.run.smithi154.stdout: "want_state": "up:active", 2023-09-23T03:18:08.322 INFO:teuthology.orchestra.run.smithi154.stdout: "state": "up:active", 2023-09-23T03:18:08.322 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs", 2023-09-23T03:18:08.322 INFO:teuthology.orchestra.run.smithi154.stdout: "rank_uptime": 4.7869370460000003, 2023-09-23T03:18:08.322 INFO:teuthology.orchestra.run.smithi154.stdout: "mdsmap_epoch": 32, 2023-09-23T03:18:08.322 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_epoch": 49, 2023-09-23T03:18:08.322 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_epoch_barrier": 49, 2023-09-23T03:18:08.323 INFO:teuthology.orchestra.run.smithi154.stdout: "uptime": 4.804127577 2023-09-23T03:18:08.323 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:18:08.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.317+0000 7f54449a9640 1 -- 172.21.15.154:0/2565376065 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f542006af30 msgr2=0x7f542006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:08.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.317+0000 7f54449a9640 1 --2- 172.21.15.154:0/2565376065 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f542006af30 0x7f542006d3f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f544015a930 tx=0x7f5434008040 comp rx=0 tx=0).stop 2023-09-23T03:18:08.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.317+0000 7f54449a9640 1 -- 172.21.15.154:0/2565376065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54400acce0 msgr2=0x7f5440154490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:08.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.317+0000 7f54449a9640 1 --2- 172.21.15.154:0/2565376065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54400acce0 0x7f5440154490 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f543000c4b0 tx=0x7f543000c970 comp rx=0 tx=0).stop 2023-09-23T03:18:08.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.317+0000 7f54449a9640 1 -- 172.21.15.154:0/2565376065 shutdown_connections 2023-09-23T03:18:08.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.317+0000 7f54449a9640 1 --2- 172.21.15.154:0/2565376065 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f542006af30 0x7f542006d3f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.317+0000 7f54449a9640 1 --2- 172.21.15.154:0/2565376065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f54400cee70 0x7f5440159ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.317+0000 7f54449a9640 1 --2- 172.21.15.154:0/2565376065 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f54400b1b20 0x7f544015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.317+0000 7f54449a9640 1 --2- 172.21.15.154:0/2565376065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f54400acce0 0x7f5440154490 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.317+0000 7f54449a9640 1 -- 172.21.15.154:0/2565376065 >> 172.21.15.154:0/2565376065 conn(0x7f544006a0f0 msgr2=0x7f54400c9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:08.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.317+0000 7f54449a9640 1 -- 172.21.15.154:0/2565376065 shutdown_connections 2023-09-23T03:18:08.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.317+0000 7f54449a9640 1 -- 172.21.15.154:0/2565376065 wait complete. 2023-09-23T03:18:08.334 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:18:08.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.538+0000 7f5091e39640 1 Processor -- start 2023-09-23T03:18:08.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.539+0000 7f5091e39640 1 -- start start 2023-09-23T03:18:08.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.540+0000 7f5091e39640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f508c06c7a0 0x7f508c15eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:08.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.540+0000 7f5091e39640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f508c0704d0 0x7f508c15afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:08.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.540+0000 7f5091e39640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f508c06bff0 con 0x7f508c15ab00 2023-09-23T03:18:08.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.540+0000 7f5091e39640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f508c155070 con 0x7f508c0704d0 2023-09-23T03:18:08.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.540+0000 7f5091e39640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f508c15e0f0 con 0x7f508c06c7a0 2023-09-23T03:18:08.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.541+0000 7f508bfff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f508c15ab00 0x7f508c06fdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:52864/0 (socket says 172.21.15.154:52864) 2023-09-23T03:18:08.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.541+0000 7f508bfff640 1 -- 172.21.15.154:0/723370085 learned_addr learned my addr 172.21.15.154:0/723370085 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:18:08.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.541+0000 7f5091638640 1 --2- 172.21.15.154:0/723370085 >> v2:172.21.15.154:3300/0 conn(0x7f508c0704d0 0x7f508c15afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:08.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.541+0000 7f5090e37640 1 --2- 172.21.15.154:0/723370085 >> v2:172.21.15.190:3300/0 conn(0x7f508c06c7a0 0x7f508c15eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:08.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.541+0000 7f5091638640 1 -- 172.21.15.154:0/723370085 >> v2:172.21.15.190:3300/0 conn(0x7f508c06c7a0 msgr2=0x7f508c15eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:08.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.541+0000 7f5091638640 1 --2- 172.21.15.154:0/723370085 >> v2:172.21.15.190:3300/0 conn(0x7f508c06c7a0 0x7f508c15eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.541+0000 7f5091638640 1 -- 172.21.15.154:0/723370085 >> v1:172.21.15.154:6789/0 conn(0x7f508c15ab00 legacy=0x7f508c06fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:08.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.541+0000 7f5091638640 1 -- 172.21.15.154:0/723370085 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f508c06cb80 con 0x7f508c0704d0 2023-09-23T03:18:08.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.541+0000 7f508b7fe640 1 -- 172.21.15.154:0/723370085 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 348215965 0 0) 0x7f508c06bff0 con 0x7f508c15ab00 2023-09-23T03:18:08.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.542+0000 7f5091638640 1 --2- 172.21.15.154:0/723370085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f508c0704d0 0x7f508c15afb0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f508c06df20 tx=0x7f508402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2aad4ecf05945e8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:08.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.543+0000 7f508b7fe640 1 -- 172.21.15.154:0/723370085 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508402cc80 con 0x7f508c0704d0 2023-09-23T03:18:08.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.543+0000 7f508b7fe640 1 -- 172.21.15.154:0/723370085 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f508402cde0 con 0x7f508c0704d0 2023-09-23T03:18:08.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.543+0000 7f508b7fe640 1 -- 172.21.15.154:0/723370085 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5084034660 con 0x7f508c0704d0 2023-09-23T03:18:08.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.543+0000 7f5091e39640 1 -- 172.21.15.154:0/723370085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f508c0704d0 msgr2=0x7f508c15afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:08.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.543+0000 7f5091e39640 1 --2- 172.21.15.154:0/723370085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f508c0704d0 0x7f508c15afb0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f508c06df20 tx=0x7f508402bed0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.544+0000 7f5091e39640 1 -- 172.21.15.154:0/723370085 shutdown_connections 2023-09-23T03:18:08.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.544+0000 7f5091e39640 1 --2- 172.21.15.154:0/723370085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f508c0704d0 0x7f508c15afb0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.544+0000 7f5091e39640 1 --2- 172.21.15.154:0/723370085 >> v2:172.21.15.190:3300/0 conn(0x7f508c06c7a0 0x7f508c15eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.544+0000 7f5091e39640 1 -- 172.21.15.154:0/723370085 >> 172.21.15.154:0/723370085 conn(0x7f508c06a390 msgr2=0x7f508c0afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:08.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.544+0000 7f5091e39640 1 -- 172.21.15.154:0/723370085 shutdown_connections 2023-09-23T03:18:08.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.544+0000 7f5091e39640 1 -- 172.21.15.154:0/723370085 wait complete. 2023-09-23T03:18:08.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.546+0000 7f5091e39640 1 Processor -- start 2023-09-23T03:18:08.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.546+0000 7f5091e39640 1 -- start start 2023-09-23T03:18:08.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.547+0000 7f5091e39640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f508c06c7a0 0x7f508c143ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:08.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.547+0000 7f5091e39640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f508c0704d0 0x7f508c0c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:08.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.547+0000 7f5091e39640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f508c15ab00 0x7f508c143540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:08.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.547+0000 7f5091e39640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508c0c7250 con 0x7f508c0704d0 2023-09-23T03:18:08.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.547+0000 7f5091e39640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508c0c6f20 con 0x7f508c15ab00 2023-09-23T03:18:08.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.547+0000 7f5091e39640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f508c0c1ea0 con 0x7f508c06c7a0 2023-09-23T03:18:08.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.548+0000 7f508bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f508c0704d0 0x7f508c0c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:08.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.548+0000 7f508bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f508c0704d0 0x7f508c0c76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56814/0 (socket says 172.21.15.154:56814) 2023-09-23T03:18:08.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.548+0000 7f508bfff640 1 -- 172.21.15.154:0/3503539970 learned_addr learned my addr 172.21.15.154:0/3503539970 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:08.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.548+0000 7f5090e37640 1 --2- 172.21.15.154:0/3503539970 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f508c06c7a0 0x7f508c143ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:08.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.548+0000 7f5091638640 1 --2- 172.21.15.154:0/3503539970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f508c15ab00 0x7f508c143540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:08.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.548+0000 7f508bfff640 1 -- 172.21.15.154:0/3503539970 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f508c06c7a0 msgr2=0x7f508c143ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:08.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.548+0000 7f508bfff640 1 --2- 172.21.15.154:0/3503539970 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f508c06c7a0 0x7f508c143ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.548+0000 7f508bfff640 1 -- 172.21.15.154:0/3503539970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f508c15ab00 msgr2=0x7f508c143540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:08.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.548+0000 7f508bfff640 1 --2- 172.21.15.154:0/3503539970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f508c15ab00 0x7f508c143540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.548+0000 7f508bfff640 1 -- 172.21.15.154:0/3503539970 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5084009a70 con 0x7f508c0704d0 2023-09-23T03:18:08.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.548+0000 7f5091638640 1 --2- 172.21.15.154:0/3503539970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f508c15ab00 0x7f508c143540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:18:08.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.548+0000 7f508bfff640 1 --2- 172.21.15.154:0/3503539970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f508c0704d0 0x7f508c0c76e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f507800c430 tx=0x7f507800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:08.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.549+0000 7f50897fa640 1 -- 172.21.15.154:0/3503539970 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50780074e0 con 0x7f508c0704d0 2023-09-23T03:18:08.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.549+0000 7f50897fa640 1 -- 172.21.15.154:0/3503539970 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5078007660 con 0x7f508c0704d0 2023-09-23T03:18:08.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.549+0000 7f50897fa640 1 -- 172.21.15.154:0/3503539970 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5078004a20 con 0x7f508c0704d0 2023-09-23T03:18:08.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.549+0000 7f5091e39640 1 -- 172.21.15.154:0/3503539970 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f508c0c2180 con 0x7f508c0704d0 2023-09-23T03:18:08.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.549+0000 7f5091e39640 1 -- 172.21.15.154:0/3503539970 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f508c0c7c20 con 0x7f508c0704d0 2023-09-23T03:18:08.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.550+0000 7f5091e39640 1 -- 172.21.15.154:0/3503539970 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f508c0736b0 con 0x7f508c0704d0 2023-09-23T03:18:08.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.551+0000 7f50897fa640 1 -- 172.21.15.154:0/3503539970 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f5078004b80 con 0x7f508c0704d0 2023-09-23T03:18:08.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.552+0000 7f50897fa640 1 --2- 172.21.15.154:0/3503539970 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f507006af30 0x7f507006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:08.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.552+0000 7f50897fa640 1 -- 172.21.15.154:0/3503539970 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6720+0+0 (secure 0 0 0) 0x7f50780899e0 con 0x7f508c0704d0 2023-09-23T03:18:08.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.552+0000 7f5090e37640 1 --2- 172.21.15.154:0/3503539970 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f507006af30 0x7f507006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:08.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.553+0000 7f5090e37640 1 --2- 172.21.15.154:0/3503539970 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f507006af30 0x7f507006d3f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5080006fd0 tx=0x7f5080008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:08.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.556+0000 7f50897fa640 1 -- 172.21.15.154:0/3503539970 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f5078054640 con 0x7f508c0704d0 2023-09-23T03:18:08.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.779+0000 7f5091e39640 1 -- 172.21.15.154:0/3503539970 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f508c04e920 con 0x7f508c0704d0 2023-09-23T03:18:08.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.780+0000 7f50897fa640 1 -- 172.21.15.154:0/3503539970 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4928 (secure 0 0 0) 0x7f5078057c60 con 0x7f508c0704d0 2023-09-23T03:18:08.781 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:18:08.781 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":32,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5215,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3388818833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3388818833},{"type":"v1","addr":"172.21.15.190:6835","nonce":3388818833}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5991,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1964552060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1964552060},{"type":"v1","addr":"172.21.15.154:6837","nonce":1964552060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5994,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/1736999297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1736999297},{"type":"v1","addr":"172.21.15.190:6834","nonce":1736999297}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:17:56.221314+0000","modified":"2023-09-23T03:18:05.544508+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5988},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5988":{"gid":5988,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":4,"addr":"172.21.15.154:6836/4278807008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4278807008},{"type":"v1","addr":"172.21.15.154:6836","nonce":4278807008}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-09-23T03:18:08.781 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 32 2023-09-23T03:18:08.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.783+0000 7f5091e39640 1 -- 172.21.15.154:0/3503539970 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f507006af30 msgr2=0x7f507006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:08.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.783+0000 7f5091e39640 1 --2- 172.21.15.154:0/3503539970 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f507006af30 0x7f507006d3f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5080006fd0 tx=0x7f5080008040 comp rx=0 tx=0).stop 2023-09-23T03:18:08.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.783+0000 7f5091e39640 1 -- 172.21.15.154:0/3503539970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f508c0704d0 msgr2=0x7f508c0c76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:08.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.783+0000 7f5091e39640 1 --2- 172.21.15.154:0/3503539970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f508c0704d0 0x7f508c0c76e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f507800c430 tx=0x7f507800c8f0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.783+0000 7f5091e39640 1 -- 172.21.15.154:0/3503539970 shutdown_connections 2023-09-23T03:18:08.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.783+0000 7f5091e39640 1 --2- 172.21.15.154:0/3503539970 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f507006af30 0x7f507006d3f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.783+0000 7f5091e39640 1 --2- 172.21.15.154:0/3503539970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f508c15ab00 0x7f508c143540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.783+0000 7f5091e39640 1 --2- 172.21.15.154:0/3503539970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f508c0704d0 0x7f508c0c76e0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.784+0000 7f5091e39640 1 --2- 172.21.15.154:0/3503539970 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f508c06c7a0 0x7f508c143ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:08.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.784+0000 7f5091e39640 1 -- 172.21.15.154:0/3503539970 >> 172.21.15.154:0/3503539970 conn(0x7f508c06a390 msgr2=0x7f508c0ce210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:08.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.784+0000 7f5091e39640 1 -- 172.21.15.154:0/3503539970 shutdown_connections 2023-09-23T03:18:08.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:08.784+0000 7f5091e39640 1 -- 172.21.15.154:0/3503539970 wait complete. 2023-09-23T03:18:08.803 INFO:tasks.cephfs.test_data_scan:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 2, "epoch": 32, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-09-23T03:17:56.221314+0000", "damaged": [], "data_pools": [ 6, 7 ], "enabled": true, "epoch": 32, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5988": { "addr": "172.21.15.154:6836/4278807008", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 4278807008, "type": "v2" }, { "addr": "172.21.15.154:6836", "nonce": 4278807008, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5988, "incarnation": 29, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 4 } }, "last_failure": 0, "last_failure_osd_epoch": 46, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 5, "modified": "2023-09-23T03:18:05.544508+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5988 } } } ], "standbys": [ { "addr": "172.21.15.190:6835/3388818833", "addrs": { "addrvec": [ { "addr": "172.21.15.190:6833", "nonce": 3388818833, "type": "v2" }, { "addr": "172.21.15.190:6835", "nonce": 3388818833, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 30, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5215, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6837/1964552060", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6835", "nonce": 1964552060, "type": "v2" }, { "addr": "172.21.15.154:6837", "nonce": 1964552060, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 30, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5991, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.190:6834/1736999297", "addrs": { "addrvec": [ { "addr": "172.21.15.190:6832", "nonce": 1736999297, "type": "v2" }, { "addr": "172.21.15.190:6834", "nonce": 1736999297, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 30, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5994, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-09-23T03:18:08.804 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-23T03:18:09.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.025+0000 7f62aa54c640 1 Processor -- start 2023-09-23T03:18:09.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.025+0000 7f62aa54c640 1 -- start start 2023-09-23T03:18:09.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.027+0000 7f62aa54c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f62a40acce0 0x7f62a415ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:09.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.027+0000 7f62aa54c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62a40cf040 0x7f62a40ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:09.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.027+0000 7f62aa54c640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62a415ac30 con 0x7f62a40b1b20 2023-09-23T03:18:09.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.027+0000 7f62aa54c640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62a40b2e90 con 0x7f62a40acce0 2023-09-23T03:18:09.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.027+0000 7f62aa54c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62a40b1f70 con 0x7f62a40cf040 2023-09-23T03:18:09.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.027+0000 7f62a3fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f62a40acce0 0x7f62a415ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:09.029 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.027+0000 7f62a8ae9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62a40cf040 0x7f62a40ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:09.029 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.027+0000 7f62a3fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f62a40acce0 0x7f62a415ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.154:60960/0 (socket says 172.21.15.154:60960) 2023-09-23T03:18:09.029 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.027+0000 7f62a8ae9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62a40cf040 0x7f62a40ab140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:60624/0 (socket says 172.21.15.154:60624) 2023-09-23T03:18:09.029 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.027+0000 7f62a3fff640 1 -- 172.21.15.154:0/3866846382 learned_addr learned my addr 172.21.15.154:0/3866846382 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:09.029 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.028+0000 7f62a8ae9640 1 -- 172.21.15.154:0/3866846382 >> v2:172.21.15.190:3300/0 conn(0x7f62a40acce0 msgr2=0x7f62a415ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:09.029 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.028+0000 7f62a8ae9640 1 --2- 172.21.15.154:0/3866846382 >> v2:172.21.15.190:3300/0 conn(0x7f62a40acce0 0x7f62a415ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:09.029 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.028+0000 7f62a8ae9640 1 -- 172.21.15.154:0/3866846382 >> v1:172.21.15.190:6789/0 conn(0x7f62a40b1b20 legacy=0x7f62a40ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:09.029 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.028+0000 7f62a8ae9640 1 -- 172.21.15.154:0/3866846382 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a415f2a0 con 0x7f62a40cf040 2023-09-23T03:18:09.030 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.030+0000 7f62a8ae9640 1 --2- 172.21.15.154:0/3866846382 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62a40cf040 0x7f62a40ab140 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f629c0097a0 tx=0x7f629c02b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b0ee78232c879b2 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:09.030 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.030+0000 7f62a2ffd640 1 -- 172.21.15.154:0/3866846382 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c031780 con 0x7f62a40cf040 2023-09-23T03:18:09.030 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.030+0000 7f62a2ffd640 1 -- 172.21.15.154:0/3866846382 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629c0318e0 con 0x7f62a40cf040 2023-09-23T03:18:09.030 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.030+0000 7f62a2ffd640 1 -- 172.21.15.154:0/3866846382 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c031c70 con 0x7f62a40cf040 2023-09-23T03:18:09.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.031+0000 7f62aa54c640 1 -- 172.21.15.154:0/3866846382 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62a40cf040 msgr2=0x7f62a40ab140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:09.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.031+0000 7f62aa54c640 1 --2- 172.21.15.154:0/3866846382 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62a40cf040 0x7f62a40ab140 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f629c0097a0 tx=0x7f629c02b820 comp rx=0 tx=0).stop 2023-09-23T03:18:09.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.031+0000 7f62aa54c640 1 -- 172.21.15.154:0/3866846382 shutdown_connections 2023-09-23T03:18:09.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.031+0000 7f62aa54c640 1 --2- 172.21.15.154:0/3866846382 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62a40cf040 0x7f62a40ab140 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:09.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.031+0000 7f62aa54c640 1 --2- 172.21.15.154:0/3866846382 >> v2:172.21.15.190:3300/0 conn(0x7f62a40acce0 0x7f62a415ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:09.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.031+0000 7f62aa54c640 1 -- 172.21.15.154:0/3866846382 >> 172.21.15.154:0/3866846382 conn(0x7f62a406a0f0 msgr2=0x7f62a406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:09.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.031+0000 7f62aa54c640 1 -- 172.21.15.154:0/3866846382 shutdown_connections 2023-09-23T03:18:09.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.031+0000 7f62aa54c640 1 -- 172.21.15.154:0/3866846382 wait complete. 2023-09-23T03:18:09.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.032+0000 7f62aa54c640 1 Processor -- start 2023-09-23T03:18:09.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.033+0000 7f62aa54c640 1 -- start start 2023-09-23T03:18:09.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.034+0000 7f62aa54c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62a40acce0 0x7f62a41577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:09.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.034+0000 7f62aa54c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62a40b1b20 0x7f62a40ae600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:09.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.034+0000 7f62aa54c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62a40cf040 0x7f62a40aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:09.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.034+0000 7f62aa54c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a40cd420 con 0x7f62a40acce0 2023-09-23T03:18:09.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.034+0000 7f62aa54c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a4157d20 con 0x7f62a40cf040 2023-09-23T03:18:09.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.034+0000 7f62aa54c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62a40c5a00 con 0x7f62a40b1b20 2023-09-23T03:18:09.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.034+0000 7f62a3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62a40acce0 0x7f62a41577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:09.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.034+0000 7f62a3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62a40acce0 0x7f62a41577e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56830/0 (socket says 172.21.15.154:56830) 2023-09-23T03:18:09.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.034+0000 7f62a3fff640 1 -- 172.21.15.154:0/178225180 learned_addr learned my addr 172.21.15.154:0/178225180 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:09.036 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.034+0000 7f62a8ae9640 1 --2- 172.21.15.154:0/178225180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62a40cf040 0x7f62a40aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:09.037 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.034+0000 7f62a3fff640 1 -- 172.21.15.154:0/178225180 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62a40b1b20 msgr2=0x7f62a40ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:09.037 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.034+0000 7f62a37fe640 1 --2- 172.21.15.154:0/178225180 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62a40b1b20 0x7f62a40ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:09.037 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.034+0000 7f62a3fff640 1 --2- 172.21.15.154:0/178225180 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62a40b1b20 0x7f62a40ae600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:09.037 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.034+0000 7f62a3fff640 1 -- 172.21.15.154:0/178225180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62a40cf040 msgr2=0x7f62a40aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:09.037 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.034+0000 7f62a3fff640 1 --2- 172.21.15.154:0/178225180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62a40cf040 0x7f62a40aeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:09.037 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.034+0000 7f62a3fff640 1 -- 172.21.15.154:0/178225180 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629c009400 con 0x7f62a40acce0 2023-09-23T03:18:09.037 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.034+0000 7f62a8ae9640 1 --2- 172.21.15.154:0/178225180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62a40cf040 0x7f62a40aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:18:09.037 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.034+0000 7f62a37fe640 1 --2- 172.21.15.154:0/178225180 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62a40b1b20 0x7f62a40ae600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:18:09.037 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.035+0000 7f62a3fff640 1 --2- 172.21.15.154:0/178225180 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62a40acce0 0x7f62a41577e0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f629400efc0 tx=0x7f629400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:09.038 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.035+0000 7f62a0ff9640 1 -- 172.21.15.154:0/178225180 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629400cda0 con 0x7f62a40acce0 2023-09-23T03:18:09.038 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.035+0000 7f62a0ff9640 1 -- 172.21.15.154:0/178225180 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629400f040 con 0x7f62a40acce0 2023-09-23T03:18:09.038 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.035+0000 7f62a0ff9640 1 -- 172.21.15.154:0/178225180 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6294014680 con 0x7f62a40acce0 2023-09-23T03:18:09.038 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.035+0000 7f62aa54c640 1 -- 172.21.15.154:0/178225180 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62a415c3c0 con 0x7f62a40acce0 2023-09-23T03:18:09.038 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.035+0000 7f62aa54c640 1 -- 172.21.15.154:0/178225180 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62a415c980 con 0x7f62a40acce0 2023-09-23T03:18:09.038 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.037+0000 7f62aa54c640 1 -- 172.21.15.154:0/178225180 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62a40c5b40 con 0x7f62a40acce0 2023-09-23T03:18:09.039 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.038+0000 7f62a0ff9640 1 -- 172.21.15.154:0/178225180 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f62940147e0 con 0x7f62a40acce0 2023-09-23T03:18:09.039 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.038+0000 7f62a0ff9640 1 --2- 172.21.15.154:0/178225180 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f628406af30 0x7f628406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:09.039 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.039+0000 7f62a0ff9640 1 -- 172.21.15.154:0/178225180 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6720+0+0 (secure 0 0 0) 0x7f629408a580 con 0x7f62a40acce0 2023-09-23T03:18:09.039 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.039+0000 7f62a37fe640 1 --2- 172.21.15.154:0/178225180 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f628406af30 0x7f628406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:09.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.040+0000 7f62a37fe640 1 --2- 172.21.15.154:0/178225180 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f628406af30 0x7f628406d3f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f62a41f5860 tx=0x7f628c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:09.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.043+0000 7f62a0ff9640 1 -- 172.21.15.154:0/178225180 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f6294055150 con 0x7f62a40acce0 2023-09-23T03:18:09.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.271+0000 7f62aa54c640 1 -- 172.21.15.154:0/178225180 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f62a40ad0c0 con 0x7f62a40acce0 2023-09-23T03:18:09.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.274+0000 7f62a0ff9640 1 -- 172.21.15.154:0/178225180 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v32) v1 ==== 53+0+91 (secure 0 0 0) 0x7f6294058770 con 0x7f62a40acce0 2023-09-23T03:18:09.277 INFO:teuthology.orchestra.run.smithi154.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_extra ] 2023-09-23T03:18:09.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.277+0000 7f62aa54c640 1 -- 172.21.15.154:0/178225180 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f628406af30 msgr2=0x7f628406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:09.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.277+0000 7f62aa54c640 1 --2- 172.21.15.154:0/178225180 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f628406af30 0x7f628406d3f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f62a41f5860 tx=0x7f628c009040 comp rx=0 tx=0).stop 2023-09-23T03:18:09.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.277+0000 7f62aa54c640 1 -- 172.21.15.154:0/178225180 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62a40acce0 msgr2=0x7f62a41577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:09.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.277+0000 7f62aa54c640 1 --2- 172.21.15.154:0/178225180 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62a40acce0 0x7f62a41577e0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f629400efc0 tx=0x7f629400c490 comp rx=0 tx=0).stop 2023-09-23T03:18:09.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.277+0000 7f62aa54c640 1 -- 172.21.15.154:0/178225180 shutdown_connections 2023-09-23T03:18:09.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.277+0000 7f62aa54c640 1 --2- 172.21.15.154:0/178225180 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f628406af30 0x7f628406d3f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:09.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.277+0000 7f62aa54c640 1 --2- 172.21.15.154:0/178225180 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f62a40cf040 0x7f62a40aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:09.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.277+0000 7f62aa54c640 1 --2- 172.21.15.154:0/178225180 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f62a40b1b20 0x7f62a40ae600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:09.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.277+0000 7f62aa54c640 1 --2- 172.21.15.154:0/178225180 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62a40acce0 0x7f62a41577e0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:09.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.277+0000 7f62aa54c640 1 -- 172.21.15.154:0/178225180 >> 172.21.15.154:0/178225180 conn(0x7f62a406a0f0 msgr2=0x7f62a4071dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:09.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.277+0000 7f62aa54c640 1 -- 172.21.15.154:0/178225180 shutdown_connections 2023-09-23T03:18:09.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:09.277+0000 7f62aa54c640 1 -- 172.21.15.154:0/178225180 wait complete. 2023-09-23T03:18:09.298 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-23T03:18:09.298 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-23T03:18:09.298 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi154.front.sepia.ceph.com 2023-09-23T03:18:09.298 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-09-23T03:18:09.298 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-23T03:18:09.299 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-23T03:18:09.299 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-23T03:18:09.299 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-23T03:18:09.299 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-23T03:18:09.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:18:09.299 DEBUG:teuthology.orchestra.run.smithi154:> ip addr 2023-09-23T03:18:09.348 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-23T03:18:09.348 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-23T03:18:09.349 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2023-09-23T03:18:09.349 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:18:09.349 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2023-09-23T03:18:09.349 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:18:09.349 INFO:teuthology.orchestra.run.smithi154.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:18:09.349 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:54 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:18:09.349 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f0 2023-09-23T03:18:09.349 INFO:teuthology.orchestra.run.smithi154.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:18:09.349 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:55 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:18:09.349 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f1 2023-09-23T03:18:09.350 INFO:teuthology.orchestra.run.smithi154.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:18:09.350 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:60 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:18:09.350 INFO:teuthology.orchestra.run.smithi154.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-23T03:18:09.350 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:61 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:18:09.350 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.21.15.154/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-09-23T03:18:09.350 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft 42236sec preferred_lft 42236sec 2023-09-23T03:18:09.350 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::ec4:7aff:fe88:7761/64 scope link 2023-09-23T03:18:09.350 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:18:09.350 INFO:teuthology.orchestra.run.smithi154.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-23T03:18:09.350 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 5a:c6:b2:96:fb:dc brd ff:ff:ff:ff:ff:ff 2023-09-23T03:18:09.351 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-09-23T03:18:09.351 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:18:09.351 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::a801:22ff:febf:2833/64 scope link 2023-09-23T03:18:09.351 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:18:09.351 INFO:teuthology.orchestra.run.smithi154.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-23T03:18:09.351 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 5a:c6:b2:96:fb:dc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:18:09.351 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::58c6:b2ff:fe96:fbdc/64 scope link 2023-09-23T03:18:09.351 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:18:09.351 INFO:teuthology.orchestra.run.smithi154.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-23T03:18:09.351 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether b2:23:b0:e4:c2:ab brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:18:09.351 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::b023:b0ff:fee4:c2ab/64 scope link 2023-09-23T03:18:09.352 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:18:09.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:18:09.353 DEBUG:teuthology.orchestra.run.smithi154:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-23T03:18:09.439 INFO:teuthology.orchestra.run.smithi154.stdout:1 2023-09-23T03:18:09.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:18:09.443 DEBUG:teuthology.orchestra.run.smithi154:> route 2023-09-23T03:18:09.497 INFO:teuthology.orchestra.run.smithi154.stdout:Kernel IP routing table 2023-09-23T03:18:09.497 INFO:teuthology.orchestra.run.smithi154.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-23T03:18:09.497 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-23T03:18:09.497 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-09-23T03:18:09.497 INFO:teuthology.orchestra.run.smithi154.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-23T03:18:09.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:18:09.497 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-09-23T03:18:09.498 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-09-23T03:18:09.498 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-23T03:18:09.498 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-23T03:18:09.498 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-23T03:18:09.498 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-09-23T03:18:09.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:18:09.685 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2023-09-23T03:18:09.737 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-09-23T03:18:09.737 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-09-23T03:18:09.738 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-23T03:18:09.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:18:09.738 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set brx.0 up 2023-09-23T03:18:09.830 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-09-23T03:18:09.831 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-23T03:18:09.831 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-09-23T03:18:09.885 INFO:teuthology.orchestra.run.smithi154.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-09-23T03:18:09.885 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-23T03:18:09.885 DEBUG:teuthology.orchestra.run.smithi154:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-09-23T03:18:09.934 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-09-23T03:18:10.029 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2023-09-23T03:18:10.086 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2023-09-23T03:18:10.086 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2023-09-23T03:18:10.086 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2023-09-23T03:18:10.086 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2023-09-23T03:18:10.086 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2023-09-23T03:18:10.086 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2023-09-23T03:18:10.086 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2023-09-23T03:18:10.086 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2023-09-23T03:18:10.086 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2023-09-23T03:18:10.087 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2023-09-23T03:18:10.087 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2023-09-23T03:18:10.087 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T03:18:10.087 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2023-09-23T03:18:10.087 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2023-09-23T03:18:10.087 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2023-09-23T03:18:10.087 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2023-09-23T03:18:10.087 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2023-09-23T03:18:10.087 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2023-09-23T03:18:10.087 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2023-09-23T03:18:10.087 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-09-23T03:18:10.087 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-09-23T03:18:10.088 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2023-09-23T03:18:10.088 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2023-09-23T03:18:10.088 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2023-09-23T03:18:10.088 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-0 2023-09-23T03:18:10.088 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-1 2023-09-23T03:18:10.088 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-2 2023-09-23T03:18:10.088 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-3 2023-09-23T03:18:10.088 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2023-09-23T03:18:10.088 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:18:10.088 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:18:10.088 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:18:10.088 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:18:10.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:18:10.089 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-09-23T03:18:10.142 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-23T03:18:10.142 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-09-23T03:18:10.148 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-09-23T03:18:10.224 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2023-09-23T03:18:10.267 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2023-09-23T03:18:10.267 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2023-09-23T03:18:10.267 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2023-09-23T03:18:10.267 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2023-09-23T03:18:10.267 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2023-09-23T03:18:10.267 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2023-09-23T03:18:10.267 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2023-09-23T03:18:10.267 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2023-09-23T03:18:10.268 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2023-09-23T03:18:10.268 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2023-09-23T03:18:10.268 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2023-09-23T03:18:10.268 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T03:18:10.268 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2023-09-23T03:18:10.268 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2023-09-23T03:18:10.268 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2023-09-23T03:18:10.268 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2023-09-23T03:18:10.268 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2023-09-23T03:18:10.268 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2023-09-23T03:18:10.268 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2023-09-23T03:18:10.269 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-09-23T03:18:10.269 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-09-23T03:18:10.269 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2023-09-23T03:18:10.269 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2023-09-23T03:18:10.269 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2023-09-23T03:18:10.269 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-0 2023-09-23T03:18:10.269 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-1 2023-09-23T03:18:10.269 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-2 2023-09-23T03:18:10.269 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-3 2023-09-23T03:18:10.269 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2023-09-23T03:18:10.269 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:18:10.269 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:18:10.270 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:18:10.270 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:18:10.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:18:10.271 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-09-23T03:18:10.333 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:2023-09-23T03:18:10.333+0000 7f2868452480 -1 init, newargv = 0x55fffff9c690 newargc=15 2023-09-23T03:18:10.333 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:ceph-fuse[105601]: starting ceph client 2023-09-23T03:18:10.342 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:ceph-fuse[105601]: starting fuse 2023-09-23T03:18:10.356 INFO:teuthology.orchestra.run.smithi154.stdout:39 2023-09-23T03:18:10.357 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-09-23T03:18:10.357 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:18:10.357 DEBUG:teuthology.orchestra.run.smithi154:> import glob 2023-09-23T03:18:10.357 DEBUG:teuthology.orchestra.run.smithi154:> import re 2023-09-23T03:18:10.357 DEBUG:teuthology.orchestra.run.smithi154:> import os 2023-09-23T03:18:10.357 DEBUG:teuthology.orchestra.run.smithi154:> import subprocess 2023-09-23T03:18:10.357 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:18:10.357 DEBUG:teuthology.orchestra.run.smithi154:> def _find_admin_socket(client_name): 2023-09-23T03:18:10.357 DEBUG:teuthology.orchestra.run.smithi154:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-09-23T03:18:10.358 DEBUG:teuthology.orchestra.run.smithi154:> files = glob.glob(asok_path) 2023-09-23T03:18:10.358 DEBUG:teuthology.orchestra.run.smithi154:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-09-23T03:18:10.358 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:18:10.358 DEBUG:teuthology.orchestra.run.smithi154:> # Given a non-glob path, it better be there 2023-09-23T03:18:10.358 DEBUG:teuthology.orchestra.run.smithi154:> if "*" not in asok_path: 2023-09-23T03:18:10.358 DEBUG:teuthology.orchestra.run.smithi154:> assert(len(files) == 1) 2023-09-23T03:18:10.358 DEBUG:teuthology.orchestra.run.smithi154:> return files[0] 2023-09-23T03:18:10.358 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:18:10.358 DEBUG:teuthology.orchestra.run.smithi154:> for f in files: 2023-09-23T03:18:10.358 DEBUG:teuthology.orchestra.run.smithi154:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-23T03:18:10.358 DEBUG:teuthology.orchestra.run.smithi154:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-23T03:18:10.358 DEBUG:teuthology.orchestra.run.smithi154:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-23T03:18:10.359 DEBUG:teuthology.orchestra.run.smithi154:> contents = proc_f.read() 2023-09-23T03:18:10.359 DEBUG:teuthology.orchestra.run.smithi154:> if mountpoint in contents: 2023-09-23T03:18:10.359 DEBUG:teuthology.orchestra.run.smithi154:> return f 2023-09-23T03:18:10.359 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-23T03:18:10.359 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:18:10.359 DEBUG:teuthology.orchestra.run.smithi154:> print(_find_admin_socket("client.0")) 2023-09-23T03:18:10.359 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:18:10.554 INFO:teuthology.orchestra.run.smithi154.stdout:/var/run/ceph/ceph-client.0.105601.asok 2023-09-23T03:18:10.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:18:10.571 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.105601.asok 2023-09-23T03:18:10.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:18:10.571 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.105601.asok status 2023-09-23T03:18:10.813 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:18:10.814 INFO:teuthology.orchestra.run.smithi154.stdout: "metadata": { 2023-09-23T03:18:10.814 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_sha1": "862663913a74a99e1c52d50e6a7e59dd6705749a", 2023-09-23T03:18:10.814 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_version": "ceph version 18.0.0-6336-g86266391 (862663913a74a99e1c52d50e6a7e59dd6705749a) reef (dev)", 2023-09-23T03:18:10.814 INFO:teuthology.orchestra.run.smithi154.stdout: "entity_id": "0", 2023-09-23T03:18:10.814 INFO:teuthology.orchestra.run.smithi154.stdout: "hostname": "smithi154", 2023-09-23T03:18:10.814 INFO:teuthology.orchestra.run.smithi154.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-09-23T03:18:10.814 INFO:teuthology.orchestra.run.smithi154.stdout: "pid": "105601", 2023-09-23T03:18:10.814 INFO:teuthology.orchestra.run.smithi154.stdout: "root": "/" 2023-09-23T03:18:10.815 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:18:10.815 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_count": 0, 2023-09-23T03:18:10.815 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_pinned_count": 0, 2023-09-23T03:18:10.815 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 6057, 2023-09-23T03:18:10.815 INFO:teuthology.orchestra.run.smithi154.stdout: "inst": { 2023-09-23T03:18:10.815 INFO:teuthology.orchestra.run.smithi154.stdout: "name": { 2023-09-23T03:18:10.815 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "client", 2023-09-23T03:18:10.815 INFO:teuthology.orchestra.run.smithi154.stdout: "num": 6057 2023-09-23T03:18:10.815 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:18:10.815 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2023-09-23T03:18:10.815 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2023-09-23T03:18:10.816 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.1:0", 2023-09-23T03:18:10.816 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 4132576116 2023-09-23T03:18:10.816 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:18:10.816 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:18:10.816 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2023-09-23T03:18:10.816 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2023-09-23T03:18:10.816 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.1:0", 2023-09-23T03:18:10.816 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 4132576116 2023-09-23T03:18:10.816 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:18:10.816 INFO:teuthology.orchestra.run.smithi154.stdout: "inst_str": "client.6057 192.168.0.1:0/4132576116", 2023-09-23T03:18:10.816 INFO:teuthology.orchestra.run.smithi154.stdout: "addr_str": "192.168.0.1:0/4132576116", 2023-09-23T03:18:10.816 INFO:teuthology.orchestra.run.smithi154.stdout: "inode_count": 1, 2023-09-23T03:18:10.817 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_epoch": 32, 2023-09-23T03:18:10.817 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch": 49, 2023-09-23T03:18:10.817 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch_barrier": 0, 2023-09-23T03:18:10.817 INFO:teuthology.orchestra.run.smithi154.stdout: "blocklisted": false, 2023-09-23T03:18:10.817 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs" 2023-09-23T03:18:10.817 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:18:10.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:18:10.829 DEBUG:teuthology.orchestra.run.smithi154:> stat --file-system '--printf=%T 2023-09-23T03:18:10.829 DEBUG:teuthology.orchestra.run.smithi154:> ' -- /home/ubuntu/cephtest/mnt.0 2023-09-23T03:18:10.886 INFO:teuthology.orchestra.run.smithi154.stdout:fuseblk 2023-09-23T03:18:10.886 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-09-23T03:18:10.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:18:10.886 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-09-23T03:18:10.984 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 scrub start / repair,recursive 2023-09-23T03:18:11.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.209+0000 7ff51cd9e640 1 Processor -- start 2023-09-23T03:18:11.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.210+0000 7ff51cd9e640 1 -- start start 2023-09-23T03:18:11.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.211+0000 7ff51cd9e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff5180700f0 0x7ff5180c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.211+0000 7ff51cd9e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5180cb460 0x7ff5180b1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.211+0000 7ff51cd9e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5180c95b0 con 0x7ff51815abd0 2023-09-23T03:18:11.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.211+0000 7ff51cd9e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5180c9890 con 0x7ff5180700f0 2023-09-23T03:18:11.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.211+0000 7ff51cd9e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5180c9b70 con 0x7ff5180cb460 2023-09-23T03:18:11.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.211+0000 7ff51659c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff5180700f0 0x7ff5180c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:11.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.211+0000 7ff51659c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff5180700f0 0x7ff5180c8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56834/0 (socket says 172.21.15.154:56834) 2023-09-23T03:18:11.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.211+0000 7ff51659c640 1 -- 172.21.15.154:0/94357677 learned_addr learned my addr 172.21.15.154:0/94357677 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:11.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.211+0000 7ff50ffff640 1 --1- 172.21.15.154:0/94357677 >> v1:172.21.15.154:6789/0 conn(0x7ff51815abd0 0x7ff5180704d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:52866/0 (socket says 172.21.15.154:52866) 2023-09-23T03:18:11.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.212+0000 7ff516d9d640 1 --2- 172.21.15.154:0/94357677 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5180cb460 0x7ff5180b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:11.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.212+0000 7ff51659c640 1 -- 172.21.15.154:0/94357677 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5180cb460 msgr2=0x7ff5180b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:11.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.212+0000 7ff515d9b640 1 -- 172.21.15.154:0/94357677 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1147036317 0 0) 0x7ff5180c95b0 con 0x7ff51815abd0 2023-09-23T03:18:11.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.212+0000 7ff51659c640 1 --2- 172.21.15.154:0/94357677 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5180cb460 0x7ff5180b1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.212+0000 7ff51659c640 1 -- 172.21.15.154:0/94357677 >> v1:172.21.15.154:6789/0 conn(0x7ff51815abd0 legacy=0x7ff5180704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:11.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.212+0000 7ff51659c640 1 -- 172.21.15.154:0/94357677 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5180c9e50 con 0x7ff5180700f0 2023-09-23T03:18:11.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.212+0000 7ff516d9d640 1 --2- 172.21.15.154:0/94357677 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5180cb460 0x7ff5180b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:18:11.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.213+0000 7ff51659c640 1 --2- 172.21.15.154:0/94357677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5180700f0 0x7ff5180c8fd0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff508009870 tx=0x7ff50802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1742a03fd1436c03 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:11.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.213+0000 7ff515d9b640 1 -- 172.21.15.154:0/94357677 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff508030840 con 0x7ff5180700f0 2023-09-23T03:18:11.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.213+0000 7ff515d9b640 1 -- 172.21.15.154:0/94357677 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5080309a0 con 0x7ff5180700f0 2023-09-23T03:18:11.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.213+0000 7ff515d9b640 1 -- 172.21.15.154:0/94357677 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff508030d30 con 0x7ff5180700f0 2023-09-23T03:18:11.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.214+0000 7ff51cd9e640 1 -- 172.21.15.154:0/94357677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5180700f0 msgr2=0x7ff5180c8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:11.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.214+0000 7ff51cd9e640 1 --2- 172.21.15.154:0/94357677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5180700f0 0x7ff5180c8fd0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff508009870 tx=0x7ff50802b980 comp rx=0 tx=0).stop 2023-09-23T03:18:11.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.214+0000 7ff51cd9e640 1 -- 172.21.15.154:0/94357677 shutdown_connections 2023-09-23T03:18:11.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.214+0000 7ff51cd9e640 1 --2- 172.21.15.154:0/94357677 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5180cb460 0x7ff5180b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.214+0000 7ff51cd9e640 1 --2- 172.21.15.154:0/94357677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5180700f0 0x7ff5180c8fd0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.214+0000 7ff51cd9e640 1 -- 172.21.15.154:0/94357677 >> 172.21.15.154:0/94357677 conn(0x7ff51806a520 msgr2=0x7ff51806bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:11.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.214+0000 7ff51cd9e640 1 -- 172.21.15.154:0/94357677 shutdown_connections 2023-09-23T03:18:11.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.214+0000 7ff51cd9e640 1 -- 172.21.15.154:0/94357677 wait complete. 2023-09-23T03:18:11.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.216+0000 7ff51cd9e640 1 Processor -- start 2023-09-23T03:18:11.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.216+0000 7ff51cd9e640 1 -- start start 2023-09-23T03:18:11.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.217+0000 7ff51cd9e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5180700f0 0x7ff5180ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.217+0000 7ff51cd9e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5180cb460 0x7ff5180b0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.217+0000 7ff51cd9e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff51815abd0 0x7ff518156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.217+0000 7ff51cd9e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5180ae9e0 con 0x7ff5180700f0 2023-09-23T03:18:11.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.217+0000 7ff51cd9e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5180b1070 con 0x7ff51815abd0 2023-09-23T03:18:11.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.217+0000 7ff51cd9e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5180ac960 con 0x7ff5180cb460 2023-09-23T03:18:11.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.217+0000 7ff51659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5180700f0 0x7ff5180ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:11.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.217+0000 7ff51659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5180700f0 0x7ff5180ac2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56838/0 (socket says 172.21.15.154:56838) 2023-09-23T03:18:11.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.217+0000 7ff51659c640 1 -- 172.21.15.154:0/377000543 learned_addr learned my addr 172.21.15.154:0/377000543 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:11.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.217+0000 7ff50ffff640 1 --2- 172.21.15.154:0/377000543 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5180cb460 0x7ff5180b0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:11.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.217+0000 7ff516d9d640 1 --2- 172.21.15.154:0/377000543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff51815abd0 0x7ff518156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:11.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.218+0000 7ff51659c640 1 -- 172.21.15.154:0/377000543 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5180cb460 msgr2=0x7ff5180b0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:11.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.218+0000 7ff51659c640 1 --2- 172.21.15.154:0/377000543 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5180cb460 0x7ff5180b0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.218+0000 7ff51659c640 1 -- 172.21.15.154:0/377000543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff51815abd0 msgr2=0x7ff518156a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:11.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.218+0000 7ff51659c640 1 --2- 172.21.15.154:0/377000543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff51815abd0 0x7ff518156a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.218+0000 7ff51659c640 1 -- 172.21.15.154:0/377000543 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5080094d0 con 0x7ff5180700f0 2023-09-23T03:18:11.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.218+0000 7ff516d9d640 1 --2- 172.21.15.154:0/377000543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff51815abd0 0x7ff518156a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:18:11.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.218+0000 7ff51659c640 1 --2- 172.21.15.154:0/377000543 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5180700f0 0x7ff5180ac2d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff5080090b0 tx=0x7ff508004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:11.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.218+0000 7ff50effd640 1 -- 172.21.15.154:0/377000543 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50803e040 con 0x7ff5180700f0 2023-09-23T03:18:11.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.218+0000 7ff50effd640 1 -- 172.21.15.154:0/377000543 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff508039e20 con 0x7ff5180700f0 2023-09-23T03:18:11.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.218+0000 7ff50effd640 1 -- 172.21.15.154:0/377000543 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff508004270 con 0x7ff5180700f0 2023-09-23T03:18:11.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.218+0000 7ff51cd9e640 1 -- 172.21.15.154:0/377000543 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5180ae390 con 0x7ff5180700f0 2023-09-23T03:18:11.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.218+0000 7ff51cd9e640 1 -- 172.21.15.154:0/377000543 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff518156fb0 con 0x7ff5180700f0 2023-09-23T03:18:11.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.221+0000 7ff50effd640 1 -- 172.21.15.154:0/377000543 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7ff5080043d0 con 0x7ff5180700f0 2023-09-23T03:18:11.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.222+0000 7ff50effd640 1 --2- 172.21.15.154:0/377000543 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff4f806af30 0x7ff4f806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.222+0000 7ff50effd640 1 -- 172.21.15.154:0/377000543 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6720+0+0 (secure 0 0 0) 0x7ff5080ac580 con 0x7ff5180700f0 2023-09-23T03:18:11.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.222+0000 7ff50ffff640 1 --2- 172.21.15.154:0/377000543 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff4f806af30 0x7ff4f806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:11.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.223+0000 7ff51fa0d740 1 Processor -- start 2023-09-23T03:18:11.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.223+0000 7ff50ffff640 1 --2- 172.21.15.154:0/377000543 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff4f806af30 0x7ff4f806d3f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff500007fd0 tx=0x7ff500007c20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:11.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.223+0000 7ff51fa0d740 1 -- start start 2023-09-23T03:18:11.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.224+0000 7ff51fa0d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56424c4dfe70 0x56424c4b2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.224+0000 7ff51fa0d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56424c4e0250 0x56424c4e2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.224+0000 7ff51fa0d740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56424c4b32b0 0x56424c4e3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.224+0000 7ff51fa0d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x56424c4264e0 con 0x56424c4e0250 2023-09-23T03:18:11.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.224+0000 7ff51fa0d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56424c423bd0 con 0x56424c4dfe70 2023-09-23T03:18:11.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.224+0000 7ff51fa0d740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56424c41f4c0 con 0x56424c4b32b0 2023-09-23T03:18:11.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.224+0000 7ff50ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56424c4e0250 0x56424c4e2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:11.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.224+0000 7ff50ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56424c4e0250 0x56424c4e2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56852/0 (socket says 172.21.15.154:56852) 2023-09-23T03:18:11.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.224+0000 7ff50ffff640 1 -- 172.21.15.154:0/2077768934 learned_addr learned my addr 172.21.15.154:0/2077768934 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:11.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.224+0000 7ff51659c640 1 --2- 172.21.15.154:0/2077768934 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56424c4dfe70 0x56424c4b2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:11.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.224+0000 7ff50ffff640 1 -- 172.21.15.154:0/2077768934 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56424c4b32b0 msgr2=0x56424c4e3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:11.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.224+0000 7ff516d9d640 1 --2- 172.21.15.154:0/2077768934 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56424c4b32b0 0x56424c4e3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:11.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.225+0000 7ff50ffff640 1 --2- 172.21.15.154:0/2077768934 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56424c4b32b0 0x56424c4e3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.225+0000 7ff50ffff640 1 -- 172.21.15.154:0/2077768934 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56424c4dfe70 msgr2=0x56424c4b2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:11.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.225+0000 7ff50ffff640 1 --2- 172.21.15.154:0/2077768934 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56424c4dfe70 0x56424c4b2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.225+0000 7ff50ffff640 1 -- 172.21.15.154:0/2077768934 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56424c3bccb0 con 0x56424c4e0250 2023-09-23T03:18:11.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.225+0000 7ff50ffff640 1 --2- 172.21.15.154:0/2077768934 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56424c4e0250 0x56424c4e2c00 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff500005db0 tx=0x7ff500012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f3f6ff37eb352a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:11.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.225+0000 7ff4f7fff640 1 -- 172.21.15.154:0/2077768934 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff500007e60 con 0x56424c4e0250 2023-09-23T03:18:11.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.225+0000 7ff4f7fff640 1 -- 172.21.15.154:0/2077768934 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff500015110 con 0x56424c4e0250 2023-09-23T03:18:11.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.225+0000 7ff4f7fff640 1 -- 172.21.15.154:0/2077768934 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff500015bc0 con 0x56424c4e0250 2023-09-23T03:18:11.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.226+0000 7ff51fa0d740 1 -- 172.21.15.154:0/2077768934 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56424c4e0250 msgr2=0x56424c4e2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:11.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.226+0000 7ff51fa0d740 1 --2- 172.21.15.154:0/2077768934 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56424c4e0250 0x56424c4e2c00 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff500005db0 tx=0x7ff500012000 comp rx=0 tx=0).stop 2023-09-23T03:18:11.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.226+0000 7ff51fa0d740 1 -- 172.21.15.154:0/2077768934 shutdown_connections 2023-09-23T03:18:11.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.226+0000 7ff51fa0d740 1 --2- 172.21.15.154:0/2077768934 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56424c4b32b0 0x56424c4e3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.226+0000 7ff51fa0d740 1 --2- 172.21.15.154:0/2077768934 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56424c4e0250 0x56424c4e2c00 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.226+0000 7ff51fa0d740 1 --2- 172.21.15.154:0/2077768934 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56424c4dfe70 0x56424c4b2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.226+0000 7ff51fa0d740 1 -- 172.21.15.154:0/2077768934 >> 172.21.15.154:0/2077768934 conn(0x56424c4b5080 msgr2=0x56424c4fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:11.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.226+0000 7ff51fa0d740 1 -- 172.21.15.154:0/2077768934 shutdown_connections 2023-09-23T03:18:11.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.226+0000 7ff51fa0d740 1 -- 172.21.15.154:0/2077768934 wait complete. 2023-09-23T03:18:11.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.227+0000 7ff51fa0d740 1 -- start start 2023-09-23T03:18:11.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.227+0000 7ff51fa0d740 1 Processor -- start 2023-09-23T03:18:11.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.228+0000 7ff51fa0d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56424c4dfe70 0x56424c4e4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.228+0000 7ff51fa0d740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56424c4e0250 0x56424c4f8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.228+0000 7ff51fa0d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56424c4fdbc0 0x56424c4f8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.228+0000 7ff51fa0d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x56424c3bd1e0 con 0x56424c4fdbc0 2023-09-23T03:18:11.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.228+0000 7ff51fa0d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56424c3bce80 con 0x56424c4dfe70 2023-09-23T03:18:11.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.228+0000 7ff51fa0d740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56424c1840c0 con 0x56424c4e0250 2023-09-23T03:18:11.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.228+0000 7ff51fa0d740 10 client.0 _ll_register_callbacks cb 0x56424c414d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:18:11.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.228+0000 7ff516d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56424c4fdbc0 0x56424c4f8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:11.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.228+0000 7ff516d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56424c4fdbc0 0x56424c4f8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56862/0 (socket says 172.21.15.154:56862) 2023-09-23T03:18:11.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.228+0000 7ff516d9d640 1 -- 172.21.15.154:0/619052525 learned_addr learned my addr 172.21.15.154:0/619052525 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:11.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.229+0000 7ff51659c640 1 --2- 172.21.15.154:0/619052525 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56424c4dfe70 0x56424c4e4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:11.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.229+0000 7ff50ffff640 1 --2- 172.21.15.154:0/619052525 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56424c4e0250 0x56424c4f8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:11.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.229+0000 7ff516d9d640 1 -- 172.21.15.154:0/619052525 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56424c4e0250 msgr2=0x56424c4f8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:11.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.229+0000 7ff516d9d640 1 --2- 172.21.15.154:0/619052525 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56424c4e0250 0x56424c4f8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.229+0000 7ff516d9d640 1 -- 172.21.15.154:0/619052525 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56424c4dfe70 msgr2=0x56424c4e4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:11.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.229+0000 7ff516d9d640 1 --2- 172.21.15.154:0/619052525 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56424c4dfe70 0x56424c4e4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.229+0000 7ff516d9d640 1 -- 172.21.15.154:0/619052525 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff50000f040 con 0x56424c4fdbc0 2023-09-23T03:18:11.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.229+0000 7ff516d9d640 1 --2- 172.21.15.154:0/619052525 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56424c4fdbc0 0x56424c4f8770 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ff51000cbc0 tx=0x7ff510007730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e7e23c3bcc6421 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:11.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.229+0000 7ff4f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:18:11.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.229+0000 7ff4f57fa640 1 -- 172.21.15.154:0/619052525 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff510011070 con 0x56424c4fdbc0 2023-09-23T03:18:11.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.230+0000 7ff51fa0d740 1 -- 172.21.15.154:0/619052525 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56424c4184a0 con 0x56424c4fdbc0 2023-09-23T03:18:11.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.230+0000 7ff4f57fa640 1 -- 172.21.15.154:0/619052525 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6720+0+0 (secure 0 0 0) 0x7ff51000a400 con 0x56424c4fdbc0 2023-09-23T03:18:11.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.230+0000 7ff4f57fa640 1 -- 172.21.15.154:0/619052525 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff510009a60 con 0x56424c4fdbc0 2023-09-23T03:18:11.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.230+0000 7ff51fa0d740 10 client.6072 fetch_fsmap learned FSMap version 32 2023-09-23T03:18:11.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.230+0000 7ff51fa0d740 1 -- 172.21.15.154:0/619052525 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x56424c41db40 con 0x56424c4fdbc0 2023-09-23T03:18:11.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.230+0000 7ff4f57fa640 1 -- 172.21.15.154:0/619052525 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56424c4184a0 con 0x56424c4fdbc0 2023-09-23T03:18:11.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.230+0000 7ff4f57fa640 1 -- 172.21.15.154:0/619052525 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 32) v1 ==== 3302+0+0 (secure 0 0 0) 0x7ff510008040 con 0x56424c4fdbc0 2023-09-23T03:18:11.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.230+0000 7ff51fa0d740 10 client.6072 fetch_fsmap finished waiting for FSMap version 32 2023-09-23T03:18:11.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.230+0000 7ff51fa0d740 10 client.6072 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2023-09-23T03:18:11.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.230+0000 7ff51fa0d740 20 client.6072 populate_metadata read hostname 'smithi154' 2023-09-23T03:18:11.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.230+0000 7ff51fa0d740 1 --2- 172.21.15.154:0/619052525 >> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] conn(0x56424c4e4870 0x56424c4e6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.230+0000 7ff51fa0d740 4 client.6072 mds_command: new command op to 5988 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T03:18:11.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.230+0000 7ff51fa0d740 1 -- 172.21.15.154:0/619052525 --> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56424c4181c0 con 0x56424c4e4870 2023-09-23T03:18:11.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.231+0000 7ff51659c640 1 --2- 172.21.15.154:0/619052525 >> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] conn(0x56424c4e4870 0x56424c4e6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:11.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.231+0000 7ff51659c640 1 --2- 172.21.15.154:0/619052525 >> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] conn(0x56424c4e4870 0x56424c4e6140 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c41af4817aec4b94 server_cookie=f535cdf1e9f42b9d in_seq=0 out_seq=0 2023-09-23T03:18:11.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.231+0000 7ff4f57fa640 10 client.6072 ms_handle_connect on v2:172.21.15.154:6834/4278807008 2023-09-23T03:18:11.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.232+0000 7ff4f57fa640 1 -- 172.21.15.154:0/619052525 <== mds.0 v2:172.21.15.154:6834/4278807008 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56424c4181c0 con 0x56424c4e4870 2023-09-23T03:18:11.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.232+0000 7ff4f57fa640 10 client.6072 handle_command_reply: tid=0 2023-09-23T03:18:11.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.232+0000 7ff51fa0d740 1 client.6072 shutdown 2023-09-23T03:18:11.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.232+0000 7ff51fa0d740 1 -- 172.21.15.154:0/619052525 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56424c4fdbc0 msgr2=0x56424c4f8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:11.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.232+0000 7ff51fa0d740 1 --2- 172.21.15.154:0/619052525 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56424c4fdbc0 0x56424c4f8770 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ff51000cbc0 tx=0x7ff510007730 comp rx=0 tx=0).stop 2023-09-23T03:18:11.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.232+0000 7ff51fa0d740 1 -- 172.21.15.154:0/619052525 shutdown_connections 2023-09-23T03:18:11.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.232+0000 7ff51fa0d740 1 --2- 172.21.15.154:0/619052525 >> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] conn(0x56424c4e4870 0x56424c4e6140 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.232+0000 7ff51fa0d740 1 --2- 172.21.15.154:0/619052525 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56424c4fdbc0 0x56424c4f8770 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.232+0000 7ff51fa0d740 1 --2- 172.21.15.154:0/619052525 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56424c4e0250 0x56424c4f8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.232+0000 7ff51fa0d740 1 --2- 172.21.15.154:0/619052525 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56424c4dfe70 0x56424c4e4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.232+0000 7ff4f57fa640 0 client.6072 ms_handle_reset on v2:172.21.15.154:6834/4278807008 2023-09-23T03:18:11.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.232+0000 7ff51fa0d740 1 -- 172.21.15.154:0/619052525 >> 172.21.15.154:0/619052525 conn(0x56424c4b32b0 msgr2=0x56424c4b6c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:11.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.232+0000 7ff51fa0d740 1 -- 172.21.15.154:0/619052525 shutdown_connections 2023-09-23T03:18:11.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.232+0000 7ff51fa0d740 1 -- 172.21.15.154:0/619052525 wait complete. 2023-09-23T03:18:11.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.233+0000 7ff51fa0d740 20 client.6072 trim_cache size 0 max 16384 2023-09-23T03:18:11.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.249+0000 7ff51fa0d740 1 Processor -- start 2023-09-23T03:18:11.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.249+0000 7ff51fa0d740 1 -- start start 2023-09-23T03:18:11.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.249+0000 7ff51fa0d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56424c4dfe70 0x56424c4f67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.249+0000 7ff51fa0d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56424c4e0250 0x56424c4f8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.249+0000 7ff51fa0d740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56424c4fdbc0 0x56424c50f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.249+0000 7ff51fa0d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x56424c4b6680 con 0x56424c4e0250 2023-09-23T03:18:11.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.249+0000 7ff51fa0d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56424c4d4980 con 0x56424c4dfe70 2023-09-23T03:18:11.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.249+0000 7ff51fa0d740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56424c4f8fa0 con 0x56424c4fdbc0 2023-09-23T03:18:11.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.250+0000 7ff50ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56424c4e0250 0x56424c4f8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:11.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.250+0000 7ff50ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56424c4e0250 0x56424c4f8420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56864/0 (socket says 172.21.15.154:56864) 2023-09-23T03:18:11.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.250+0000 7ff50ffff640 1 -- 172.21.15.154:0/52636092 learned_addr learned my addr 172.21.15.154:0/52636092 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:11.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.250+0000 7ff51659c640 1 --2- 172.21.15.154:0/52636092 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56424c4dfe70 0x56424c4f67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:11.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.250+0000 7ff516d9d640 1 --2- 172.21.15.154:0/52636092 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56424c4fdbc0 0x56424c50f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:11.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.250+0000 7ff50ffff640 1 -- 172.21.15.154:0/52636092 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56424c4fdbc0 msgr2=0x56424c50f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:11.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.250+0000 7ff50ffff640 1 --2- 172.21.15.154:0/52636092 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56424c4fdbc0 0x56424c50f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.250+0000 7ff50ffff640 1 -- 172.21.15.154:0/52636092 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56424c4dfe70 msgr2=0x56424c4f67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:11.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.250+0000 7ff50ffff640 1 --2- 172.21.15.154:0/52636092 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56424c4dfe70 0x56424c4f67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.250+0000 7ff50ffff640 1 -- 172.21.15.154:0/52636092 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56424c3bc540 con 0x56424c4e0250 2023-09-23T03:18:11.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.250+0000 7ff51659c640 1 --2- 172.21.15.154:0/52636092 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56424c4dfe70 0x56424c4f67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:18:11.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.250+0000 7ff516d9d640 1 --2- 172.21.15.154:0/52636092 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56424c4fdbc0 0x56424c50f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:18:11.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.251+0000 7ff50ffff640 1 --2- 172.21.15.154:0/52636092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56424c4e0250 0x56424c4f8420 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff500005d80 tx=0x7ff500013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fd75a2d8767149e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:11.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.251+0000 7ff50cff9640 1 -- 172.21.15.154:0/52636092 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5000159c0 con 0x56424c4e0250 2023-09-23T03:18:11.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.251+0000 7ff50cff9640 1 -- 172.21.15.154:0/52636092 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff500029040 con 0x56424c4e0250 2023-09-23T03:18:11.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.251+0000 7ff50cff9640 1 -- 172.21.15.154:0/52636092 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff500015d20 con 0x56424c4e0250 2023-09-23T03:18:11.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.252+0000 7ff51fa0d740 1 -- 172.21.15.154:0/52636092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56424c4e0250 msgr2=0x56424c4f8420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:11.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.252+0000 7ff51fa0d740 1 --2- 172.21.15.154:0/52636092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56424c4e0250 0x56424c4f8420 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff500005d80 tx=0x7ff500013000 comp rx=0 tx=0).stop 2023-09-23T03:18:11.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.252+0000 7ff51fa0d740 1 -- 172.21.15.154:0/52636092 shutdown_connections 2023-09-23T03:18:11.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.252+0000 7ff51fa0d740 1 --2- 172.21.15.154:0/52636092 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56424c4fdbc0 0x56424c50f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.252+0000 7ff51fa0d740 1 --2- 172.21.15.154:0/52636092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56424c4e0250 0x56424c4f8420 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.252+0000 7ff51fa0d740 1 --2- 172.21.15.154:0/52636092 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56424c4dfe70 0x56424c4f67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.252+0000 7ff51fa0d740 1 -- 172.21.15.154:0/52636092 >> 172.21.15.154:0/52636092 conn(0x56424c4b32b0 msgr2=0x56424c4b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:11.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.252+0000 7ff51fa0d740 1 -- 172.21.15.154:0/52636092 shutdown_connections 2023-09-23T03:18:11.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.252+0000 7ff51fa0d740 1 -- 172.21.15.154:0/52636092 wait complete. 2023-09-23T03:18:11.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.252+0000 7ff51fa0d740 1 -- start start 2023-09-23T03:18:11.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.253+0000 7ff51fa0d740 1 Processor -- start 2023-09-23T03:18:11.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.253+0000 7ff51fa0d740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56424c4dfe70 0x56424c4fcc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.253+0000 7ff51fa0d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56424c4e0250 0x56424c4d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.254+0000 7ff51fa0d740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56424c4fdbc0 0x56424c4e5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.254+0000 7ff51fa0d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x56424c4e6c60 con 0x56424c4e0250 2023-09-23T03:18:11.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.254+0000 7ff51fa0d740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56424c517c00 con 0x56424c4fdbc0 2023-09-23T03:18:11.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.254+0000 7ff51fa0d740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56424c50cd10 con 0x56424c4dfe70 2023-09-23T03:18:11.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.254+0000 7ff51fa0d740 10 client.0 _ll_register_callbacks cb 0x7ff500007db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:18:11.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.254+0000 7ff50ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56424c4e0250 0x56424c4d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:11.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.254+0000 7ff50ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56424c4e0250 0x56424c4d0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56874/0 (socket says 172.21.15.154:56874) 2023-09-23T03:18:11.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.254+0000 7ff50ffff640 1 -- 172.21.15.154:0/927451429 learned_addr learned my addr 172.21.15.154:0/927451429 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:11.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.254+0000 7ff51659c640 1 --2- 172.21.15.154:0/927451429 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56424c4dfe70 0x56424c4fcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:11.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.254+0000 7ff516d9d640 1 --2- 172.21.15.154:0/927451429 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56424c4fdbc0 0x56424c4e5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:11.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.254+0000 7ff50ffff640 1 -- 172.21.15.154:0/927451429 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56424c4dfe70 msgr2=0x56424c4fcc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:11.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.254+0000 7ff50ffff640 1 --2- 172.21.15.154:0/927451429 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56424c4dfe70 0x56424c4fcc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.254+0000 7ff50ffff640 1 -- 172.21.15.154:0/927451429 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56424c4fdbc0 msgr2=0x56424c4e5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:11.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.254+0000 7ff50ffff640 1 --2- 172.21.15.154:0/927451429 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56424c4fdbc0 0x56424c4e5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.254+0000 7ff50ffff640 1 -- 172.21.15.154:0/927451429 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56424c3bccb0 con 0x56424c4e0250 2023-09-23T03:18:11.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.254+0000 7ff51659c640 1 --2- 172.21.15.154:0/927451429 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56424c4dfe70 0x56424c4fcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:18:11.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.254+0000 7ff50ffff640 1 --2- 172.21.15.154:0/927451429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56424c4e0250 0x56424c4d0ae0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff500017040 tx=0x7ff500000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=407865f4befadee7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:11.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.254+0000 7ff4f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:18:11.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.255+0000 7ff4f57fa640 1 -- 172.21.15.154:0/927451429 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff500021660 con 0x56424c4e0250 2023-09-23T03:18:11.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.255+0000 7ff4f57fa640 1 -- 172.21.15.154:0/927451429 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6720+0+0 (secure 0 0 0) 0x7ff5000217c0 con 0x56424c4e0250 2023-09-23T03:18:11.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.255+0000 7ff51fa0d740 1 -- 172.21.15.154:0/927451429 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff500015110 con 0x56424c4e0250 2023-09-23T03:18:11.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.255+0000 7ff4f57fa640 1 -- 172.21.15.154:0/927451429 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff500021980 con 0x56424c4e0250 2023-09-23T03:18:11.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.255+0000 7ff51fa0d740 10 client.6078 fetch_fsmap learned FSMap version 32 2023-09-23T03:18:11.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.255+0000 7ff51fa0d740 1 -- 172.21.15.154:0/927451429 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x56424c3e1bf0 con 0x56424c4e0250 2023-09-23T03:18:11.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.255+0000 7ff4f57fa640 1 -- 172.21.15.154:0/927451429 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff500015110 con 0x56424c4e0250 2023-09-23T03:18:11.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.255+0000 7ff4f57fa640 1 -- 172.21.15.154:0/927451429 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 32) v1 ==== 3302+0+0 (secure 0 0 0) 0x7ff500021ae0 con 0x56424c4e0250 2023-09-23T03:18:11.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.255+0000 7ff51fa0d740 10 client.6078 fetch_fsmap finished waiting for FSMap version 32 2023-09-23T03:18:11.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.255+0000 7ff51fa0d740 10 client.6078 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2023-09-23T03:18:11.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.255+0000 7ff51fa0d740 20 client.6078 populate_metadata read hostname 'smithi154' 2023-09-23T03:18:11.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.255+0000 7ff51fa0d740 1 --2- 172.21.15.154:0/927451429 >> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] conn(0x56424c550170 0x56424c4e63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.255+0000 7ff51fa0d740 4 client.6078 mds_command: new command op to 5988 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2023-09-23T03:18:11.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.255+0000 7ff51fa0d740 1 -- 172.21.15.154:0/927451429 --> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x56424c121860 con 0x56424c550170 2023-09-23T03:18:11.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.256+0000 7ff51659c640 1 --2- 172.21.15.154:0/927451429 >> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] conn(0x56424c550170 0x56424c4e63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:11.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.256+0000 7ff51659c640 1 --2- 172.21.15.154:0/927451429 >> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] conn(0x56424c550170 0x56424c4e63a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a776c2bdadfdfc3e server_cookie=46329950af15d15f in_seq=0 out_seq=0 2023-09-23T03:18:11.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.256+0000 7ff4f57fa640 10 client.6078 ms_handle_connect on v2:172.21.15.154:6834/4278807008 2023-09-23T03:18:11.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.257+0000 7ff4f57fa640 1 -- 172.21.15.154:0/927451429 <== mds.0 v2:172.21.15.154:6834/4278807008 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56424c121860 con 0x56424c550170 2023-09-23T03:18:11.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.257+0000 7ff4f57fa640 10 client.6078 handle_command_reply: tid=0 2023-09-23T03:18:11.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.257+0000 7ff51fa0d740 1 client.6078 shutdown 2023-09-23T03:18:11.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.257+0000 7ff51fa0d740 1 -- 172.21.15.154:0/927451429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56424c4e0250 msgr2=0x56424c4d0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:11.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.257+0000 7ff51fa0d740 1 --2- 172.21.15.154:0/927451429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56424c4e0250 0x56424c4d0ae0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff500017040 tx=0x7ff500000ed0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.257+0000 7ff51fa0d740 1 -- 172.21.15.154:0/927451429 shutdown_connections 2023-09-23T03:18:11.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.257+0000 7ff51fa0d740 1 --2- 172.21.15.154:0/927451429 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56424c4fdbc0 0x56424c4e5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.257+0000 7ff51fa0d740 1 --2- 172.21.15.154:0/927451429 >> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] conn(0x56424c550170 0x56424c4e63a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.257+0000 7ff51fa0d740 1 --2- 172.21.15.154:0/927451429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56424c4e0250 0x56424c4d0ae0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.257+0000 7ff51fa0d740 1 --2- 172.21.15.154:0/927451429 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56424c4dfe70 0x56424c4fcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.257+0000 7ff4f57fa640 0 client.6078 ms_handle_reset on v2:172.21.15.154:6834/4278807008 2023-09-23T03:18:11.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.257+0000 7ff51fa0d740 1 -- 172.21.15.154:0/927451429 >> 172.21.15.154:0/927451429 conn(0x56424c4b32b0 msgr2=0x56424c4e4130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:11.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.257+0000 7ff51fa0d740 1 -- 172.21.15.154:0/927451429 shutdown_connections 2023-09-23T03:18:11.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.257+0000 7ff51fa0d740 1 -- 172.21.15.154:0/927451429 wait complete. 2023-09-23T03:18:11.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.258+0000 7ff51fa0d740 20 client.6078 trim_cache size 0 max 16384 2023-09-23T03:18:11.264 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:18:11.258+0000 7f35329ba640 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2023-09-23T03:18:11.264 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:18:11.264 INFO:teuthology.orchestra.run.smithi154.stdout: "return_code": 0, 2023-09-23T03:18:11.264 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_tag": "4949800f-25c2-41ad-ba71-9a24bf740a17", 2023-09-23T03:18:11.264 INFO:teuthology.orchestra.run.smithi154.stdout: "mode": "asynchronous" 2023-09-23T03:18:11.264 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:18:11.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.258+0000 7ff51cd9e640 1 -- 172.21.15.154:0/377000543 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff4f806af30 msgr2=0x7ff4f806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:11.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.258+0000 7ff51cd9e640 1 --2- 172.21.15.154:0/377000543 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff4f806af30 0x7ff4f806d3f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff500007fd0 tx=0x7ff500007c20 comp rx=0 tx=0).stop 2023-09-23T03:18:11.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.258+0000 7ff51cd9e640 1 -- 172.21.15.154:0/377000543 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5180700f0 msgr2=0x7ff5180ac2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:11.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.258+0000 7ff51cd9e640 1 --2- 172.21.15.154:0/377000543 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5180700f0 0x7ff5180ac2d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff5080090b0 tx=0x7ff508004730 comp rx=0 tx=0).stop 2023-09-23T03:18:11.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.258+0000 7ff51cd9e640 1 -- 172.21.15.154:0/377000543 shutdown_connections 2023-09-23T03:18:11.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.258+0000 7ff51cd9e640 1 --2- 172.21.15.154:0/377000543 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff4f806af30 0x7ff4f806d3f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.258+0000 7ff51cd9e640 1 --2- 172.21.15.154:0/377000543 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff51815abd0 0x7ff518156a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.258+0000 7ff51cd9e640 1 --2- 172.21.15.154:0/377000543 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff5180cb460 0x7ff5180b0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.258+0000 7ff51cd9e640 1 --2- 172.21.15.154:0/377000543 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff5180700f0 0x7ff5180ac2d0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.258+0000 7ff51cd9e640 1 -- 172.21.15.154:0/377000543 >> 172.21.15.154:0/377000543 conn(0x7ff51806a520 msgr2=0x7ff5180cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:11.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.259+0000 7ff51cd9e640 1 -- 172.21.15.154:0/377000543 shutdown_connections 2023-09-23T03:18:11.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.259+0000 7ff51cd9e640 1 -- 172.21.15.154:0/377000543 wait complete. 2023-09-23T03:18:11.267 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:18:11.259+0000 7f35329ba640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /subdir/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x5634b4970100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(5)0x10000000000:[<0x1/subdir v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-09-23T03:18:11.267 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:18:11.260+0000 7f35329ba640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f6 [2,head] /subdir/1 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x5634b4a31700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(7)0x100000001f6:[<0x10000000000/1 v24>,<0x1/subdir v44>]//[6]","memoryvalue":"(7)0x100000001f6:[<0x10000000000/1 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:18:11.267 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:18:11.261+0000 7f35329ba640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f8 [2,head] /subdir/3 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x5634b4a30c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(7)0x100000001f8:[<0x10000000000/3 v38>,<0x1/subdir v44>]//[6]","memoryvalue":"(7)0x100000001f8:[<0x10000000000/3 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:18:11.267 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:18:11.261+0000 7f35329ba640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f7 [2,head] /subdir/2 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x5634b4a31180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(7)0x100000001f7:[<0x10000000000/2 v32>,<0x1/subdir v44>]//[6]","memoryvalue":"(7)0x100000001f7:[<0x10000000000/2 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:18:11.267 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:18:11.262+0000 7f35329ba640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/0 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x5634b4a67b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(7)0x10000000001:[<0x10000000000/0 v16>,<0x1/subdir v44>]//[6]","memoryvalue":"(7)0x10000000001:[<0x10000000000/0 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:18:11.267 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:18:11.262+0000 7f35329ba640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f9 [2,head] /subdir/4 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x5634b4a30680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(7)0x100000001f9:[<0x10000000000/4 v40>,<0x1/subdir v44>]//[6]","memoryvalue":"(7)0x100000001f9:[<0x10000000000/4 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:18:11.269 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:18:11.269+0000 7f35329ba640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v5 ap=5 snaprealm=0x5634b49686c0 DIRTYPARENT f(v1 1=0+1) n(v0 rc2023-09-23T03:18:10.979060+0000 1=0+1)/n() (idft sync r=1) (inest sync r=1) (ifile sync->mix r=1) (iversion lock) caps={6057=pLsXs/-@3} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x5634b496f600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(5)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=0+1)","ondisk_value.rstat":"n(v0 2=0+2)","memory_value.dirstat":"f(v1 1=0+1)","memory_value.rstat":"n(v0 rc2023-09-23T03:18:10.979060+0000 1=0+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-09-23T03:18:11.279 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 scrub status 2023-09-23T03:18:11.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.479+0000 7f8650cec640 1 Processor -- start 2023-09-23T03:18:11.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.479+0000 7f8650cec640 1 -- start start 2023-09-23T03:18:11.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.481+0000 7f8650cec640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f864c15bb30 0x7f864c0caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.481+0000 7f8650cec640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f864c0cb3e0 0x7f864c0cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.481+0000 7f8650cec640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f864c15b6a0 con 0x7f864c15aba0 2023-09-23T03:18:11.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.481+0000 7f8650cec640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f864c0cb870 con 0x7f864c15bb30 2023-09-23T03:18:11.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.481+0000 7f8650cec640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f864c15bf70 con 0x7f864c0cb3e0 2023-09-23T03:18:11.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.481+0000 7f864ad9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f864c0cb3e0 0x7f864c0cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:11.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.481+0000 7f8643fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f864c15bb30 0x7f864c0caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:11.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.481+0000 7f864ad9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f864c0cb3e0 0x7f864c0cc5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:60686/0 (socket says 172.21.15.154:60686) 2023-09-23T03:18:11.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.481+0000 7f864ad9d640 1 -- 172.21.15.154:0/4109144893 learned_addr learned my addr 172.21.15.154:0/4109144893 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:11.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.481+0000 7f8643fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f864c15bb30 0x7f864c0caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.154:32778/0 (socket says 172.21.15.154:32778) 2023-09-23T03:18:11.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.482+0000 7f864ad9d640 1 -- 172.21.15.154:0/4109144893 >> v2:172.21.15.190:3300/0 conn(0x7f864c15bb30 msgr2=0x7f864c0caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:11.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.482+0000 7f864ad9d640 1 --2- 172.21.15.154:0/4109144893 >> v2:172.21.15.190:3300/0 conn(0x7f864c15bb30 0x7f864c0caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.482+0000 7f864ad9d640 1 -- 172.21.15.154:0/4109144893 >> v1:172.21.15.190:6789/0 conn(0x7f864c15aba0 legacy=0x7f864c15e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:11.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.482+0000 7f864ad9d640 1 -- 172.21.15.154:0/4109144893 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f864c15f0b0 con 0x7f864c0cb3e0 2023-09-23T03:18:11.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.483+0000 7f864ad9d640 1 --2- 172.21.15.154:0/4109144893 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f864c0cb3e0 0x7f864c0cc5c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8644009e50 tx=0x7f864402bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2958c9e7e2fcc2c6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:11.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.484+0000 7f8649d9b640 1 -- 172.21.15.154:0/4109144893 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864402cb00 con 0x7f864c0cb3e0 2023-09-23T03:18:11.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.484+0000 7f8649d9b640 1 -- 172.21.15.154:0/4109144893 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f864402cc60 con 0x7f864c0cb3e0 2023-09-23T03:18:11.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.484+0000 7f8649d9b640 1 -- 172.21.15.154:0/4109144893 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86440343f0 con 0x7f864c0cb3e0 2023-09-23T03:18:11.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.484+0000 7f8650cec640 1 -- 172.21.15.154:0/4109144893 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f864c0cb3e0 msgr2=0x7f864c0cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:11.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.484+0000 7f8650cec640 1 --2- 172.21.15.154:0/4109144893 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f864c0cb3e0 0x7f864c0cc5c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8644009e50 tx=0x7f864402bf60 comp rx=0 tx=0).stop 2023-09-23T03:18:11.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.485+0000 7f8650cec640 1 -- 172.21.15.154:0/4109144893 shutdown_connections 2023-09-23T03:18:11.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.485+0000 7f8650cec640 1 --2- 172.21.15.154:0/4109144893 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f864c0cb3e0 0x7f864c0cc5c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.485+0000 7f8650cec640 1 --2- 172.21.15.154:0/4109144893 >> v2:172.21.15.190:3300/0 conn(0x7f864c15bb30 0x7f864c0caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.485+0000 7f8650cec640 1 -- 172.21.15.154:0/4109144893 >> 172.21.15.154:0/4109144893 conn(0x7f864c06c010 msgr2=0x7f864c06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:11.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.485+0000 7f8650cec640 1 -- 172.21.15.154:0/4109144893 shutdown_connections 2023-09-23T03:18:11.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.485+0000 7f8650cec640 1 -- 172.21.15.154:0/4109144893 wait complete. 2023-09-23T03:18:11.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.487+0000 7f8650cec640 1 Processor -- start 2023-09-23T03:18:11.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.487+0000 7f8650cec640 1 -- start start 2023-09-23T03:18:11.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.488+0000 7f8650cec640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f864c0cb3e0 0x7f864c14a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.488+0000 7f8650cec640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f864c15aba0 0x7f864c0c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.488+0000 7f8650cec640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f864c15bb30 0x7f864c158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.488+0000 7f8650cec640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864c14aee0 con 0x7f864c15bb30 2023-09-23T03:18:11.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.488+0000 7f8650cec640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864c0c8490 con 0x7f864c15aba0 2023-09-23T03:18:11.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.488+0000 7f8650cec640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f864c0c6eb0 con 0x7f864c0cb3e0 2023-09-23T03:18:11.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.488+0000 7f864ad9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f864c15bb30 0x7f864c158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:11.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.488+0000 7f864ad9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f864c15bb30 0x7f864c158ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56882/0 (socket says 172.21.15.154:56882) 2023-09-23T03:18:11.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.488+0000 7f864ad9d640 1 -- 172.21.15.154:0/1190832020 learned_addr learned my addr 172.21.15.154:0/1190832020 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:11.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.488+0000 7f8643fff640 1 --2- 172.21.15.154:0/1190832020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f864c15aba0 0x7f864c0c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:11.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.488+0000 7f864a59c640 1 --2- 172.21.15.154:0/1190832020 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f864c0cb3e0 0x7f864c14a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:11.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.489+0000 7f864ad9d640 1 -- 172.21.15.154:0/1190832020 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f864c0cb3e0 msgr2=0x7f864c14a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:11.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.489+0000 7f864ad9d640 1 --2- 172.21.15.154:0/1190832020 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f864c0cb3e0 0x7f864c14a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.489+0000 7f864ad9d640 1 -- 172.21.15.154:0/1190832020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f864c15aba0 msgr2=0x7f864c0c7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:11.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.489+0000 7f864ad9d640 1 --2- 172.21.15.154:0/1190832020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f864c15aba0 0x7f864c0c7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.489+0000 7f864ad9d640 1 -- 172.21.15.154:0/1190832020 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8644009b30 con 0x7f864c15bb30 2023-09-23T03:18:11.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.489+0000 7f864ad9d640 1 --2- 172.21.15.154:0/1190832020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f864c15bb30 0x7f864c158ad0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f8644009e50 tx=0x7f8644004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:11.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.489+0000 7f8642ffd640 1 -- 172.21.15.154:0/1190832020 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864403e450 con 0x7f864c15bb30 2023-09-23T03:18:11.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.489+0000 7f8642ffd640 1 -- 172.21.15.154:0/1190832020 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f864403e5b0 con 0x7f864c15bb30 2023-09-23T03:18:11.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.489+0000 7f8650cec640 1 -- 172.21.15.154:0/1190832020 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f864c0c7190 con 0x7f864c15bb30 2023-09-23T03:18:11.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.489+0000 7f8642ffd640 1 -- 172.21.15.154:0/1190832020 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864403e960 con 0x7f864c15bb30 2023-09-23T03:18:11.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.489+0000 7f8650cec640 1 -- 172.21.15.154:0/1190832020 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f864c0c7750 con 0x7f864c15bb30 2023-09-23T03:18:11.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.492+0000 7f8642ffd640 1 -- 172.21.15.154:0/1190832020 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f864403eac0 con 0x7f864c15bb30 2023-09-23T03:18:11.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.492+0000 7f8642ffd640 1 --2- 172.21.15.154:0/1190832020 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f862c06af30 0x7f862c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.493+0000 7f8642ffd640 1 -- 172.21.15.154:0/1190832020 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6720+0+0 (secure 0 0 0) 0x7f86440ac7f0 con 0x7f864c15bb30 2023-09-23T03:18:11.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.493+0000 7f864a59c640 1 --2- 172.21.15.154:0/1190832020 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f862c06af30 0x7f862c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:11.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.493+0000 7f8653883740 1 Processor -- start 2023-09-23T03:18:11.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.493+0000 7f864a59c640 1 --2- 172.21.15.154:0/1190832020 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f862c06af30 0x7f862c06d3f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f863c001640 tx=0x7f863c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:11.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.494+0000 7f8653883740 1 -- start start 2023-09-23T03:18:11.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.494+0000 7f8653883740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55abc3ebf690 0x55abc3e92590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.494+0000 7f8653883740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abc3ebfa70 0x55abc3ec2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.494+0000 7f8653883740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc3e92ad0 0x55abc3ec2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.494+0000 7f8653883740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc3e04a50 con 0x55abc3e92ad0 2023-09-23T03:18:11.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.494+0000 7f8653883740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc3e02140 con 0x55abc3ebfa70 2023-09-23T03:18:11.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.494+0000 7f8653883740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55abc3dfda30 con 0x55abc3ebf690 2023-09-23T03:18:11.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.495+0000 7f864ad9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc3e92ad0 0x55abc3ec2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:11.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.495+0000 7f864ad9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc3e92ad0 0x55abc3ec2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56890/0 (socket says 172.21.15.154:56890) 2023-09-23T03:18:11.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.495+0000 7f864ad9d640 1 -- 172.21.15.154:0/3953935029 learned_addr learned my addr 172.21.15.154:0/3953935029 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:11.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.495+0000 7f8643fff640 1 --2- 172.21.15.154:0/3953935029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abc3ebfa70 0x55abc3ec2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:11.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.495+0000 7f864a59c640 1 --2- 172.21.15.154:0/3953935029 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55abc3ebf690 0x55abc3e92590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:11.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.495+0000 7f864ad9d640 1 -- 172.21.15.154:0/3953935029 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55abc3ebf690 msgr2=0x55abc3e92590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:11.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.495+0000 7f864ad9d640 1 --2- 172.21.15.154:0/3953935029 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55abc3ebf690 0x55abc3e92590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.495+0000 7f864ad9d640 1 -- 172.21.15.154:0/3953935029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abc3ebfa70 msgr2=0x55abc3ec2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:11.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.495+0000 7f864ad9d640 1 --2- 172.21.15.154:0/3953935029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abc3ebfa70 0x55abc3ec2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.495+0000 7f864ad9d640 1 -- 172.21.15.154:0/3953935029 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abc3dc1a70 con 0x55abc3e92ad0 2023-09-23T03:18:11.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.495+0000 7f864ad9d640 1 --2- 172.21.15.154:0/3953935029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc3e92ad0 0x55abc3ec2960 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f8644082ee0 tx=0x7f8644077520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98fd4721d4c7a12 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:11.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.496+0000 7f862bfff640 1 -- 172.21.15.154:0/3953935029 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864407b810 con 0x55abc3e92ad0 2023-09-23T03:18:11.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.496+0000 7f862bfff640 1 -- 172.21.15.154:0/3953935029 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f864407b110 con 0x55abc3e92ad0 2023-09-23T03:18:11.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.496+0000 7f862bfff640 1 -- 172.21.15.154:0/3953935029 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864407f260 con 0x55abc3e92ad0 2023-09-23T03:18:11.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.496+0000 7f8653883740 1 -- 172.21.15.154:0/3953935029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc3e92ad0 msgr2=0x55abc3ec2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:11.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.496+0000 7f8653883740 1 --2- 172.21.15.154:0/3953935029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc3e92ad0 0x55abc3ec2960 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f8644082ee0 tx=0x7f8644077520 comp rx=0 tx=0).stop 2023-09-23T03:18:11.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.496+0000 7f8653883740 1 -- 172.21.15.154:0/3953935029 shutdown_connections 2023-09-23T03:18:11.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.496+0000 7f8653883740 1 --2- 172.21.15.154:0/3953935029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc3e92ad0 0x55abc3ec2960 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.496+0000 7f8653883740 1 --2- 172.21.15.154:0/3953935029 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abc3ebfa70 0x55abc3ec2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.496+0000 7f8653883740 1 --2- 172.21.15.154:0/3953935029 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55abc3ebf690 0x55abc3e92590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.496+0000 7f8653883740 1 -- 172.21.15.154:0/3953935029 >> 172.21.15.154:0/3953935029 conn(0x55abc3e948a0 msgr2=0x55abc3edd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:11.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.497+0000 7f8653883740 1 -- 172.21.15.154:0/3953935029 shutdown_connections 2023-09-23T03:18:11.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.497+0000 7f8653883740 1 -- 172.21.15.154:0/3953935029 wait complete. 2023-09-23T03:18:11.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.497+0000 7f8653883740 1 -- start start 2023-09-23T03:18:11.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.497+0000 7f8653883740 1 Processor -- start 2023-09-23T03:18:11.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.498+0000 7f8653883740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abc3ebf690 0x55abc3ec3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.498+0000 7f8653883740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc3ebfa70 0x55abc3ec5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.498+0000 7f8653883740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55abc3ebcf10 0x55abc3ec5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.498+0000 7f8653883740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc3d9bad0 con 0x55abc3ebfa70 2023-09-23T03:18:11.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.498+0000 7f8653883740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc3d9b770 con 0x55abc3ebf690 2023-09-23T03:18:11.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.498+0000 7f8653883740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55abc3af7f90 con 0x55abc3ebcf10 2023-09-23T03:18:11.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.499+0000 7f8653883740 10 client.0 _ll_register_callbacks cb 0x55abc3dbf220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:18:11.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.499+0000 7f8643fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc3ebfa70 0x55abc3ec5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:11.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.499+0000 7f8643fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc3ebfa70 0x55abc3ec5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56894/0 (socket says 172.21.15.154:56894) 2023-09-23T03:18:11.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.499+0000 7f8643fff640 1 -- 172.21.15.154:0/1080939674 learned_addr learned my addr 172.21.15.154:0/1080939674 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:11.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.499+0000 7f864a59c640 1 --2- 172.21.15.154:0/1080939674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abc3ebf690 0x55abc3ec3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:11.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.499+0000 7f8643fff640 1 -- 172.21.15.154:0/1080939674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55abc3ebcf10 msgr2=0x55abc3ec5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:11.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.499+0000 7f8643fff640 1 --2- 172.21.15.154:0/1080939674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55abc3ebcf10 0x55abc3ec5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.499+0000 7f8643fff640 1 -- 172.21.15.154:0/1080939674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abc3ebf690 msgr2=0x55abc3ec3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:11.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.499+0000 7f8643fff640 1 --2- 172.21.15.154:0/1080939674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abc3ebf690 0x55abc3ec3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.499+0000 7f8643fff640 1 -- 172.21.15.154:0/1080939674 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f864c0c7750 con 0x55abc3ebfa70 2023-09-23T03:18:11.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.499+0000 7f8643fff640 1 --2- 172.21.15.154:0/1080939674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc3ebfa70 0x55abc3ec5850 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f8634009d40 tx=0x7f86340136b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367eacba9302189b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:11.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.499+0000 7f86297fa640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:18:11.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.500+0000 7f86297fa640 1 -- 172.21.15.154:0/1080939674 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8634004020 con 0x55abc3ebfa70 2023-09-23T03:18:11.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.500+0000 7f8653883740 1 -- 172.21.15.154:0/1080939674 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55abc3e244d0 con 0x55abc3ebfa70 2023-09-23T03:18:11.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.500+0000 7f86297fa640 1 -- 172.21.15.154:0/1080939674 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6720+0+0 (secure 0 0 0) 0x7f8634004180 con 0x55abc3ebfa70 2023-09-23T03:18:11.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.500+0000 7f86297fa640 1 -- 172.21.15.154:0/1080939674 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8634004340 con 0x55abc3ebfa70 2023-09-23T03:18:11.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.500+0000 7f8653883740 10 client.6087 fetch_fsmap learned FSMap version 32 2023-09-23T03:18:11.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.500+0000 7f8653883740 1 -- 172.21.15.154:0/1080939674 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x55abc3dfc0b0 con 0x55abc3ebfa70 2023-09-23T03:18:11.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.500+0000 7f86297fa640 1 -- 172.21.15.154:0/1080939674 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55abc3e244d0 con 0x55abc3ebfa70 2023-09-23T03:18:11.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.500+0000 7f86297fa640 1 -- 172.21.15.154:0/1080939674 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 32) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f8634005990 con 0x55abc3ebfa70 2023-09-23T03:18:11.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.500+0000 7f8653883740 10 client.6087 fetch_fsmap finished waiting for FSMap version 32 2023-09-23T03:18:11.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.500+0000 7f8653883740 10 client.6087 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2023-09-23T03:18:11.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.500+0000 7f8653883740 20 client.6087 populate_metadata read hostname 'smithi154' 2023-09-23T03:18:11.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.500+0000 7f8653883740 1 --2- 172.21.15.154:0/1080939674 >> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] conn(0x55abc3ed49a0 0x55abc3ed4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.500+0000 7f8653883740 4 client.6087 mds_command: new command op to 5988 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T03:18:11.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.500+0000 7f8653883740 1 -- 172.21.15.154:0/1080939674 --> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55abc3dd0ab0 con 0x55abc3ed49a0 2023-09-23T03:18:11.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.501+0000 7f864a59c640 1 --2- 172.21.15.154:0/1080939674 >> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] conn(0x55abc3ed49a0 0x55abc3ed4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:11.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.501+0000 7f864a59c640 1 --2- 172.21.15.154:0/1080939674 >> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] conn(0x55abc3ed49a0 0x55abc3ed4d80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1c973853a5c1cc5 server_cookie=feeb93073c592e4a in_seq=0 out_seq=0 2023-09-23T03:18:11.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.501+0000 7f86297fa640 10 client.6087 ms_handle_connect on v2:172.21.15.154:6834/4278807008 2023-09-23T03:18:11.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.502+0000 7f86297fa640 1 -- 172.21.15.154:0/1080939674 <== mds.0 v2:172.21.15.154:6834/4278807008 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55abc3dd0ab0 con 0x55abc3ed49a0 2023-09-23T03:18:11.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.502+0000 7f86297fa640 10 client.6087 handle_command_reply: tid=0 2023-09-23T03:18:11.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.502+0000 7f8653883740 1 client.6087 shutdown 2023-09-23T03:18:11.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.502+0000 7f8653883740 1 -- 172.21.15.154:0/1080939674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc3ebfa70 msgr2=0x55abc3ec5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:11.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.502+0000 7f8653883740 1 --2- 172.21.15.154:0/1080939674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc3ebfa70 0x55abc3ec5850 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f8634009d40 tx=0x7f86340136b0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.502+0000 7f8653883740 1 -- 172.21.15.154:0/1080939674 shutdown_connections 2023-09-23T03:18:11.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.502+0000 7f8653883740 1 --2- 172.21.15.154:0/1080939674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55abc3ebcf10 0x55abc3ec5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.502+0000 7f8653883740 1 --2- 172.21.15.154:0/1080939674 >> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] conn(0x55abc3ed49a0 0x55abc3ed4d80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.502+0000 7f8653883740 1 --2- 172.21.15.154:0/1080939674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc3ebfa70 0x55abc3ec5850 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.502+0000 7f8653883740 1 --2- 172.21.15.154:0/1080939674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abc3ebf690 0x55abc3ec3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.502+0000 7f86297fa640 0 client.6087 ms_handle_reset on v2:172.21.15.154:6834/4278807008 2023-09-23T03:18:11.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.502+0000 7f8653883740 1 -- 172.21.15.154:0/1080939674 >> 172.21.15.154:0/1080939674 conn(0x55abc3e92ad0 msgr2=0x55abc3e96510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:11.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.502+0000 7f8653883740 1 -- 172.21.15.154:0/1080939674 shutdown_connections 2023-09-23T03:18:11.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.502+0000 7f8653883740 1 -- 172.21.15.154:0/1080939674 wait complete. 2023-09-23T03:18:11.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.503+0000 7f8653883740 20 client.6087 trim_cache size 0 max 16384 2023-09-23T03:18:11.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.518+0000 7f8653883740 1 Processor -- start 2023-09-23T03:18:11.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.519+0000 7f8653883740 1 -- start start 2023-09-23T03:18:11.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.519+0000 7f8653883740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc3ebcf10 0x55abc3eed840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.519+0000 7f8653883740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55abc3ebf690 0x55abc3eedd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.519+0000 7f8653883740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abc3ebfa70 0x55abc3eb0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.519+0000 7f8653883740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc3eecfc0 con 0x55abc3ebcf10 2023-09-23T03:18:11.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.519+0000 7f8653883740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc3e92880 con 0x55abc3ebfa70 2023-09-23T03:18:11.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.519+0000 7f8653883740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55abc3ec69c0 con 0x55abc3ebf690 2023-09-23T03:18:11.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.520+0000 7f864a59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc3ebcf10 0x55abc3eed840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:11.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.520+0000 7f864a59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc3ebcf10 0x55abc3eed840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56900/0 (socket says 172.21.15.154:56900) 2023-09-23T03:18:11.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.520+0000 7f864a59c640 1 -- 172.21.15.154:0/4011794897 learned_addr learned my addr 172.21.15.154:0/4011794897 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:11.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.520+0000 7f864a59c640 1 -- 172.21.15.154:0/4011794897 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55abc3ebf690 msgr2=0x55abc3eedd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:11.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.520+0000 7f864a59c640 1 --2- 172.21.15.154:0/4011794897 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55abc3ebf690 0x55abc3eedd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.520+0000 7f864a59c640 1 -- 172.21.15.154:0/4011794897 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abc3ebfa70 msgr2=0x55abc3eb0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:11.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.520+0000 7f864a59c640 1 --2- 172.21.15.154:0/4011794897 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abc3ebfa70 0x55abc3eb0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.520+0000 7f864a59c640 1 -- 172.21.15.154:0/4011794897 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abc3e2dda0 con 0x55abc3ebcf10 2023-09-23T03:18:11.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.520+0000 7f864a59c640 1 --2- 172.21.15.154:0/4011794897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc3ebcf10 0x55abc3eed840 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f863c001420 tx=0x7f863c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b3fbe551ed3333b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:11.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.521+0000 7f8640ff9640 1 -- 172.21.15.154:0/4011794897 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c01a040 con 0x55abc3ebcf10 2023-09-23T03:18:11.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.521+0000 7f8640ff9640 1 -- 172.21.15.154:0/4011794897 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f863c027070 con 0x55abc3ebcf10 2023-09-23T03:18:11.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.521+0000 7f8640ff9640 1 -- 172.21.15.154:0/4011794897 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c0235b0 con 0x55abc3ebcf10 2023-09-23T03:18:11.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.521+0000 7f8653883740 1 -- 172.21.15.154:0/4011794897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc3ebcf10 msgr2=0x55abc3eed840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:11.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.521+0000 7f8653883740 1 --2- 172.21.15.154:0/4011794897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc3ebcf10 0x55abc3eed840 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f863c001420 tx=0x7f863c012000 comp rx=0 tx=0).stop 2023-09-23T03:18:11.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.521+0000 7f8653883740 1 -- 172.21.15.154:0/4011794897 shutdown_connections 2023-09-23T03:18:11.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.521+0000 7f8653883740 1 --2- 172.21.15.154:0/4011794897 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abc3ebfa70 0x55abc3eb0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.521+0000 7f8653883740 1 --2- 172.21.15.154:0/4011794897 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55abc3ebf690 0x55abc3eedd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.521+0000 7f8653883740 1 --2- 172.21.15.154:0/4011794897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc3ebcf10 0x55abc3eed840 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.521+0000 7f8653883740 1 -- 172.21.15.154:0/4011794897 >> 172.21.15.154:0/4011794897 conn(0x55abc3e92ad0 msgr2=0x55abc3e8dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:11.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.521+0000 7f8653883740 1 -- 172.21.15.154:0/4011794897 shutdown_connections 2023-09-23T03:18:11.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.521+0000 7f8653883740 1 -- 172.21.15.154:0/4011794897 wait complete. 2023-09-23T03:18:11.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.522+0000 7f8653883740 1 -- start start 2023-09-23T03:18:11.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.522+0000 7f8653883740 1 Processor -- start 2023-09-23T03:18:11.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.523+0000 7f8653883740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc3ebcf10 0x55abc3eed840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.523+0000 7f8653883740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55abc3ebf690 0x55abc3eedd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.523+0000 7f8653883740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abc3ebfa70 0x55abc3e96e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.523+0000 7f8653883740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc3f4bc40 con 0x55abc3ebcf10 2023-09-23T03:18:11.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.523+0000 7f8653883740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc3eeb830 con 0x55abc3ebfa70 2023-09-23T03:18:11.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.523+0000 7f8653883740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55abc3ec3440 con 0x55abc3ebf690 2023-09-23T03:18:11.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.523+0000 7f8653883740 10 client.0 _ll_register_callbacks cb 0x7f8644007310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:18:11.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.523+0000 7f864a59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc3ebcf10 0x55abc3eed840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:11.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.523+0000 7f864a59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc3ebcf10 0x55abc3eed840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56908/0 (socket says 172.21.15.154:56908) 2023-09-23T03:18:11.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.523+0000 7f864a59c640 1 -- 172.21.15.154:0/2993582128 learned_addr learned my addr 172.21.15.154:0/2993582128 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:11.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.524+0000 7f864a59c640 1 -- 172.21.15.154:0/2993582128 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55abc3ebf690 msgr2=0x55abc3eedd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:11.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.524+0000 7f864a59c640 1 --2- 172.21.15.154:0/2993582128 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55abc3ebf690 0x55abc3eedd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.524+0000 7f864a59c640 1 -- 172.21.15.154:0/2993582128 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abc3ebfa70 msgr2=0x55abc3e96e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:11.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.524+0000 7f864ad9d640 1 --2- 172.21.15.154:0/2993582128 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abc3ebfa70 0x55abc3e96e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:11.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.524+0000 7f864a59c640 1 --2- 172.21.15.154:0/2993582128 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abc3ebfa70 0x55abc3e96e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.524+0000 7f864a59c640 1 -- 172.21.15.154:0/2993582128 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f863c00f040 con 0x55abc3ebcf10 2023-09-23T03:18:11.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.524+0000 7f864a59c640 1 --2- 172.21.15.154:0/2993582128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc3ebcf10 0x55abc3eed840 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f863c003e50 tx=0x7f863c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b95e31b648ad8d96 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:11.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.524+0000 7f86297fa640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:18:11.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.524+0000 7f86297fa640 1 -- 172.21.15.154:0/2993582128 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c023a00 con 0x55abc3ebcf10 2023-09-23T03:18:11.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.524+0000 7f86297fa640 1 -- 172.21.15.154:0/2993582128 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6720+0+0 (secure 0 0 0) 0x7f863c01a070 con 0x55abc3ebcf10 2023-09-23T03:18:11.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.524+0000 7f8653883740 1 -- 172.21.15.154:0/2993582128 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55abc3ed7740 con 0x55abc3ebcf10 2023-09-23T03:18:11.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.525+0000 7f86297fa640 1 -- 172.21.15.154:0/2993582128 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f863c023b60 con 0x55abc3ebcf10 2023-09-23T03:18:11.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.525+0000 7f8653883740 10 client.6093 fetch_fsmap learned FSMap version 32 2023-09-23T03:18:11.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.525+0000 7f8653883740 1 -- 172.21.15.154:0/2993582128 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x55abc3d9ace0 con 0x55abc3ebcf10 2023-09-23T03:18:11.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.525+0000 7f86297fa640 1 -- 172.21.15.154:0/2993582128 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55abc3ed7740 con 0x55abc3ebcf10 2023-09-23T03:18:11.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.526+0000 7f86297fa640 1 -- 172.21.15.154:0/2993582128 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 32) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f863c023cc0 con 0x55abc3ebcf10 2023-09-23T03:18:11.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.526+0000 7f8653883740 10 client.6093 fetch_fsmap finished waiting for FSMap version 32 2023-09-23T03:18:11.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.526+0000 7f8653883740 10 client.6093 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2023-09-23T03:18:11.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.526+0000 7f8653883740 20 client.6093 populate_metadata read hostname 'smithi154' 2023-09-23T03:18:11.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.526+0000 7f8653883740 1 --2- 172.21.15.154:0/2993582128 >> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] conn(0x55abc3ebda20 0x55abc3e97360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:11.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.526+0000 7f8653883740 4 client.6093 mds_command: new command op to 5988 tid=0[{"prefix": "scrub status"}] 2023-09-23T03:18:11.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.526+0000 7f8653883740 1 -- 172.21.15.154:0/2993582128 --> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f864407b110 con 0x55abc3ebda20 2023-09-23T03:18:11.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.526+0000 7f8643fff640 1 --2- 172.21.15.154:0/2993582128 >> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] conn(0x55abc3ebda20 0x55abc3e97360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:11.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.526+0000 7f8643fff640 1 --2- 172.21.15.154:0/2993582128 >> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] conn(0x55abc3ebda20 0x55abc3e97360 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80a41929a851d1fe server_cookie=7b94658828dbf9f7 in_seq=0 out_seq=0 2023-09-23T03:18:11.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.526+0000 7f86297fa640 10 client.6093 ms_handle_connect on v2:172.21.15.154:6834/4278807008 2023-09-23T03:18:11.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.527+0000 7f86297fa640 1 -- 172.21.15.154:0/2993582128 <== mds.0 v2:172.21.15.154:6834/4278807008 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x7f864407b110 con 0x55abc3ebda20 2023-09-23T03:18:11.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.527+0000 7f86297fa640 10 client.6093 handle_command_reply: tid=0 2023-09-23T03:18:11.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.527+0000 7f8653883740 1 client.6093 shutdown 2023-09-23T03:18:11.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.527+0000 7f8653883740 1 -- 172.21.15.154:0/2993582128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc3ebcf10 msgr2=0x55abc3eed840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:11.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.527+0000 7f8653883740 1 --2- 172.21.15.154:0/2993582128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc3ebcf10 0x55abc3eed840 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f863c003e50 tx=0x7f863c000bc0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.527+0000 7f8653883740 1 -- 172.21.15.154:0/2993582128 shutdown_connections 2023-09-23T03:18:11.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.527+0000 7f8653883740 1 --2- 172.21.15.154:0/2993582128 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55abc3ebfa70 0x55abc3e96e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.527+0000 7f8653883740 1 --2- 172.21.15.154:0/2993582128 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55abc3ebf690 0x55abc3eedd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.527+0000 7f8653883740 1 --2- 172.21.15.154:0/2993582128 >> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] conn(0x55abc3ebda20 0x55abc3e97360 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.527+0000 7f8653883740 1 --2- 172.21.15.154:0/2993582128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc3ebcf10 0x55abc3eed840 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.527+0000 7f8653883740 1 -- 172.21.15.154:0/2993582128 >> 172.21.15.154:0/2993582128 conn(0x55abc3e92ad0 msgr2=0x55abc3ed6650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:11.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.527+0000 7f86297fa640 0 client.6093 ms_handle_reset on v2:172.21.15.154:6834/4278807008 2023-09-23T03:18:11.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.527+0000 7f8653883740 1 -- 172.21.15.154:0/2993582128 shutdown_connections 2023-09-23T03:18:11.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.527+0000 7f8653883740 1 -- 172.21.15.154:0/2993582128 wait complete. 2023-09-23T03:18:11.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.527+0000 7f8653883740 20 client.6093 trim_cache size 0 max 16384 2023-09-23T03:18:11.532 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:18:11.533 INFO:teuthology.orchestra.run.smithi154.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T03:18:11.533 INFO:teuthology.orchestra.run.smithi154.stdout: "scrubs": { 2023-09-23T03:18:11.533 INFO:teuthology.orchestra.run.smithi154.stdout: "4949800f-25c2-41ad-ba71-9a24bf740a17": { 2023-09-23T03:18:11.533 INFO:teuthology.orchestra.run.smithi154.stdout: "path": "/", 2023-09-23T03:18:11.533 INFO:teuthology.orchestra.run.smithi154.stdout: "tag": "4949800f-25c2-41ad-ba71-9a24bf740a17", 2023-09-23T03:18:11.533 INFO:teuthology.orchestra.run.smithi154.stdout: "options": "recursive,repair" 2023-09-23T03:18:11.533 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:18:11.533 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:18:11.533 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:18:11.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.528+0000 7f8650cec640 1 -- 172.21.15.154:0/1190832020 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f862c06af30 msgr2=0x7f862c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:11.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.528+0000 7f8650cec640 1 --2- 172.21.15.154:0/1190832020 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f862c06af30 0x7f862c06d3f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f863c001640 tx=0x7f863c009040 comp rx=0 tx=0).stop 2023-09-23T03:18:11.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.528+0000 7f8650cec640 1 -- 172.21.15.154:0/1190832020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f864c15bb30 msgr2=0x7f864c158ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:11.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.528+0000 7f8650cec640 1 --2- 172.21.15.154:0/1190832020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f864c15bb30 0x7f864c158ad0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f8644009e50 tx=0x7f8644004a40 comp rx=0 tx=0).stop 2023-09-23T03:18:11.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.528+0000 7f8650cec640 1 -- 172.21.15.154:0/1190832020 shutdown_connections 2023-09-23T03:18:11.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.528+0000 7f8650cec640 1 --2- 172.21.15.154:0/1190832020 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f862c06af30 0x7f862c06d3f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.528+0000 7f8650cec640 1 --2- 172.21.15.154:0/1190832020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f864c15bb30 0x7f864c158ad0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.528+0000 7f8650cec640 1 --2- 172.21.15.154:0/1190832020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f864c15aba0 0x7f864c0c7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.528+0000 7f8650cec640 1 --2- 172.21.15.154:0/1190832020 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f864c0cb3e0 0x7f864c14a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:11.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.528+0000 7f8650cec640 1 -- 172.21.15.154:0/1190832020 >> 172.21.15.154:0/1190832020 conn(0x7f864c06c010 msgr2=0x7f864c0c2280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:11.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.528+0000 7f8650cec640 1 -- 172.21.15.154:0/1190832020 shutdown_connections 2023-09-23T03:18:11.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:11.528+0000 7f8650cec640 1 -- 172.21.15.154:0/1190832020 wait complete. 2023-09-23T03:18:11.545 INFO:tasks.cephfs.filesystem:scrub status for tag:4949800f-25c2-41ad-ba71-9a24bf740a17 - {'path': '/', 'tag': '4949800f-25c2-41ad-ba71-9a24bf740a17', 'options': 'recursive,repair'} 2023-09-23T03:18:11.628 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:18:11.637 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:18:41.547 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 scrub status 2023-09-23T03:18:41.774 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:18:41.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.773+0000 7fc2aacbb640 1 Processor -- start 2023-09-23T03:18:41.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.774+0000 7fc2aacbb640 1 -- start start 2023-09-23T03:18:41.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.775+0000 7fc2aacbb640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc2a406c570 0x7fc2a40cc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:41.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.775+0000 7fc2aacbb640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a4071150 con 0x7fc2a40cda50 2023-09-23T03:18:41.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.775+0000 7fc2aacbb640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a4159910 con 0x7fc2a4159530 2023-09-23T03:18:41.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.775+0000 7fc2aacbb640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2a40ccf30 con 0x7fc2a406c570 2023-09-23T03:18:41.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.776+0000 7fc2a94b8640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fc2a40cda50 0x7fc2a406be60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:50282/0 (socket says 172.21.15.154:50282) 2023-09-23T03:18:41.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.776+0000 7fc2a94b8640 1 -- 172.21.15.154:0/3271502388 learned_addr learned my addr 172.21.15.154:0/3271502388 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:18:41.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.776+0000 7fc2aa4ba640 1 --2- 172.21.15.154:0/3271502388 >> v2:172.21.15.190:3300/0 conn(0x7fc2a406c570 0x7fc2a40cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:41.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.776+0000 7fc2a8cb7640 1 -- 172.21.15.154:0/3271502388 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3063087792 0 0) 0x7fc2a4071150 con 0x7fc2a40cda50 2023-09-23T03:18:41.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.776+0000 7fc2a8cb7640 1 -- 172.21.15.154:0/3271502388 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc28c003520 con 0x7fc2a40cda50 2023-09-23T03:18:41.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.776+0000 7fc2a8cb7640 1 -- 172.21.15.154:0/3271502388 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1145442561 0 0) 0x7fc28c003520 con 0x7fc2a40cda50 2023-09-23T03:18:41.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.776+0000 7fc2a8cb7640 1 -- 172.21.15.154:0/3271502388 >> v2:172.21.15.190:3300/0 conn(0x7fc2a406c570 msgr2=0x7fc2a40cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:41.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.776+0000 7fc2a8cb7640 1 --2- 172.21.15.154:0/3271502388 >> v2:172.21.15.190:3300/0 conn(0x7fc2a406c570 0x7fc2a40cc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:41.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.776+0000 7fc2a8cb7640 1 -- 172.21.15.154:0/3271502388 >> v1:172.21.15.190:6789/0 conn(0x7fc2a4159530 legacy=0x7fc2a40b1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:41.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.776+0000 7fc2a8cb7640 1 -- 172.21.15.154:0/3271502388 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a40cc690 con 0x7fc2a40cda50 2023-09-23T03:18:41.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.776+0000 7fc2a8cb7640 1 -- 172.21.15.154:0/3271502388 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fc298001670 con 0x7fc2a40cda50 2023-09-23T03:18:41.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.777+0000 7fc2a8cb7640 1 -- 172.21.15.154:0/3271502388 >> v1:172.21.15.154:6789/0 conn(0x7fc2a40cda50 legacy=0x7fc2a406be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:41.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.777+0000 7fc2a8cb7640 1 --2- 172.21.15.154:0/3271502388 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc28c003970 0x7fc28c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:41.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.777+0000 7fc2a8cb7640 1 --2- 172.21.15.154:0/3271502388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc28c0047c0 0x7fc28c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:41.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.777+0000 7fc2a8cb7640 1 --2- 172.21.15.154:0/3271502388 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc28c005180 0x7fc28c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:41.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.777+0000 7fc2a8cb7640 1 -- 172.21.15.154:0/3271502388 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a40ccf30 con 0x7fc28c003970 2023-09-23T03:18:41.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.777+0000 7fc2a8cb7640 1 -- 172.21.15.154:0/3271502388 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28c00c020 con 0x7fc28c0047c0 2023-09-23T03:18:41.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.777+0000 7fc2a8cb7640 1 -- 172.21.15.154:0/3271502388 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc28c00c1e0 con 0x7fc28c005180 2023-09-23T03:18:41.780 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:18:41.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.777+0000 7fc2a9cb9640 1 --2- 172.21.15.154:0/3271502388 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc28c003970 0x7fc28c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:41.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.777+0000 7fc2a94b8640 1 --2- 172.21.15.154:0/3271502388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc28c0047c0 0x7fc28c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:41.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.777+0000 7fc2aa4ba640 1 --2- 172.21.15.154:0/3271502388 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc28c005180 0x7fc28c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:41.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.777+0000 7fc2a9cb9640 1 -- 172.21.15.154:0/3271502388 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc28c005180 msgr2=0x7fc28c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:41.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.777+0000 7fc2a9cb9640 1 --2- 172.21.15.154:0/3271502388 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc28c005180 0x7fc28c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:41.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.777+0000 7fc2a9cb9640 1 -- 172.21.15.154:0/3271502388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc28c0047c0 msgr2=0x7fc28c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:41.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.777+0000 7fc2a9cb9640 1 --2- 172.21.15.154:0/3271502388 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc28c0047c0 0x7fc28c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:41.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.778+0000 7fc2a9cb9640 1 -- 172.21.15.154:0/3271502388 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc28c003520 con 0x7fc28c003970 2023-09-23T03:18:41.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.778+0000 7fc2aa4ba640 1 --2- 172.21.15.154:0/3271502388 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc28c005180 0x7fc28c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:18:41.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.778+0000 7fc2aa4ba640 1 -- 172.21.15.154:0/3271502388 reap_dead start 2023-09-23T03:18:41.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.779+0000 7fc2a9cb9640 1 --2- 172.21.15.154:0/3271502388 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc28c003970 0x7fc28c003d90 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc294001e50 tx=0x7fc28c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a393bf01e392aa3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:41.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.779+0000 7fc2a8cb7640 1 -- 172.21.15.154:0/3271502388 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc294003b70 con 0x7fc28c003970 2023-09-23T03:18:41.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.779+0000 7fc2a8cb7640 1 -- 172.21.15.154:0/3271502388 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc294003cd0 con 0x7fc28c003970 2023-09-23T03:18:41.782 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.779+0000 7fc2a8cb7640 1 -- 172.21.15.154:0/3271502388 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc294005310 con 0x7fc28c003970 2023-09-23T03:18:41.782 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.779+0000 7fc2aacbb640 1 -- 172.21.15.154:0/3271502388 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc28c003970 msgr2=0x7fc28c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:41.782 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.779+0000 7fc2aacbb640 1 --2- 172.21.15.154:0/3271502388 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc28c003970 0x7fc28c003d90 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc294001e50 tx=0x7fc28c0057f0 comp rx=0 tx=0).stop 2023-09-23T03:18:41.782 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.780+0000 7fc2aacbb640 1 -- 172.21.15.154:0/3271502388 shutdown_connections 2023-09-23T03:18:41.782 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.780+0000 7fc2aacbb640 1 --2- 172.21.15.154:0/3271502388 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc28c003970 0x7fc28c003d90 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:41.782 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.780+0000 7fc2aacbb640 1 -- 172.21.15.154:0/3271502388 >> 172.21.15.154:0/3271502388 conn(0x7fc2a406a300 msgr2=0x7fc2a40caeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:41.782 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.780+0000 7fc2aacbb640 1 -- 172.21.15.154:0/3271502388 shutdown_connections 2023-09-23T03:18:41.782 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.780+0000 7fc2aacbb640 1 -- 172.21.15.154:0/3271502388 wait complete. 2023-09-23T03:18:41.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.781+0000 7fc2aacbb640 1 Processor -- start 2023-09-23T03:18:41.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.782+0000 7fc2aacbb640 1 -- start start 2023-09-23T03:18:41.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.782+0000 7fc2aacbb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc28c003970 0x7fc2a40c6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:41.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.782+0000 7fc2aacbb640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2a40c4540 0x7fc2a41f1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:41.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.783+0000 7fc2aacbb640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2a41f16c0 0x7fc2a41f1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:41.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.783+0000 7fc2aacbb640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a40c52c0 con 0x7fc28c003970 2023-09-23T03:18:41.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.783+0000 7fc2aacbb640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a40c4a20 con 0x7fc2a40c4540 2023-09-23T03:18:41.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.783+0000 7fc2aacbb640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a40c4d00 con 0x7fc2a41f16c0 2023-09-23T03:18:41.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.783+0000 7fc2a9cb9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc28c003970 0x7fc2a40c6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:41.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.783+0000 7fc2a9cb9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc28c003970 0x7fc2a40c6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56784/0 (socket says 172.21.15.154:56784) 2023-09-23T03:18:41.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.783+0000 7fc2a9cb9640 1 -- 172.21.15.154:0/2986091872 learned_addr learned my addr 172.21.15.154:0/2986091872 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:41.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.783+0000 7fc2aa4ba640 1 --2- 172.21.15.154:0/2986091872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2a41f16c0 0x7fc2a41f1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:41.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.783+0000 7fc2a94b8640 1 --2- 172.21.15.154:0/2986091872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2a40c4540 0x7fc2a41f1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:41.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.783+0000 7fc2a9cb9640 1 -- 172.21.15.154:0/2986091872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2a41f16c0 msgr2=0x7fc2a41f1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:41.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.783+0000 7fc2a9cb9640 1 --2- 172.21.15.154:0/2986091872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2a41f16c0 0x7fc2a41f1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:41.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.783+0000 7fc2a9cb9640 1 -- 172.21.15.154:0/2986091872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2a40c4540 msgr2=0x7fc2a41f1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:41.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.783+0000 7fc2a9cb9640 1 --2- 172.21.15.154:0/2986091872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2a40c4540 0x7fc2a41f1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:41.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.783+0000 7fc2a9cb9640 1 -- 172.21.15.154:0/2986091872 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a40c4fe0 con 0x7fc28c003970 2023-09-23T03:18:41.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.783+0000 7fc2a94b8640 1 --2- 172.21.15.154:0/2986091872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2a40c4540 0x7fc2a41f1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:18:41.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.784+0000 7fc2a9cb9640 1 --2- 172.21.15.154:0/2986091872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc28c003970 0x7fc2a40c6b60 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc29402bf70 tx=0x7fc2940014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:41.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.784+0000 7fc2a27fc640 1 -- 172.21.15.154:0/2986091872 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29403e070 con 0x7fc28c003970 2023-09-23T03:18:41.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.784+0000 7fc2a27fc640 1 -- 172.21.15.154:0/2986091872 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc294006b50 con 0x7fc28c003970 2023-09-23T03:18:41.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.784+0000 7fc2aacbb640 1 -- 172.21.15.154:0/2986091872 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a40c20e0 con 0x7fc28c003970 2023-09-23T03:18:41.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.784+0000 7fc2a27fc640 1 -- 172.21.15.154:0/2986091872 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc294007020 con 0x7fc28c003970 2023-09-23T03:18:41.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.784+0000 7fc2aacbb640 1 -- 172.21.15.154:0/2986091872 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a4143540 con 0x7fc28c003970 2023-09-23T03:18:41.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.787+0000 7fc2a27fc640 1 -- 172.21.15.154:0/2986091872 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fc294007180 con 0x7fc28c003970 2023-09-23T03:18:41.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.787+0000 7fc2a27fc640 1 --2- 172.21.15.154:0/2986091872 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc29006af30 0x7fc29006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:41.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.788+0000 7fc2a27fc640 1 -- 172.21.15.154:0/2986091872 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6720+0+0 (secure 0 0 0) 0x7fc2940ab2c0 con 0x7fc28c003970 2023-09-23T03:18:41.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.788+0000 7fc2a94b8640 1 --2- 172.21.15.154:0/2986091872 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc29006af30 0x7fc29006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:41.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.788+0000 7fc2a94b8640 1 --2- 172.21.15.154:0/2986091872 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc29006af30 0x7fc29006d3f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc298001eb0 tx=0x7fc298000f10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:41.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.789+0000 7fc2aeb87740 1 Processor -- start 2023-09-23T03:18:41.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.789+0000 7fc2aeb87740 1 -- start start 2023-09-23T03:18:41.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.790+0000 7fc2aeb87740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9661f1870 0x55b9661c4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:41.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.790+0000 7fc2aeb87740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9661f1c50 0x55b9661f4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:41.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.790+0000 7fc2aeb87740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b9661c4cb0 0x55b9661f4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:41.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.790+0000 7fc2aeb87740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b966136a50 con 0x55b9661c4cb0 2023-09-23T03:18:41.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.790+0000 7fc2aeb87740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b966134140 con 0x55b9661f1c50 2023-09-23T03:18:41.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.790+0000 7fc2aeb87740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b96612fa30 con 0x55b9661f1870 2023-09-23T03:18:41.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.790+0000 7fc2aa4ba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b9661c4cb0 0x55b9661f4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:41.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.790+0000 7fc2aa4ba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b9661c4cb0 0x55b9661f4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56790/0 (socket says 172.21.15.154:56790) 2023-09-23T03:18:41.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.790+0000 7fc2aa4ba640 1 -- 172.21.15.154:0/3728758975 learned_addr learned my addr 172.21.15.154:0/3728758975 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:41.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.790+0000 7fc2a94b8640 1 --2- 172.21.15.154:0/3728758975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9661f1c50 0x55b9661f4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:41.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.790+0000 7fc2a9cb9640 1 --2- 172.21.15.154:0/3728758975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9661f1870 0x55b9661c4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:41.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.790+0000 7fc2aa4ba640 1 -- 172.21.15.154:0/3728758975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9661f1870 msgr2=0x55b9661c4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:41.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.790+0000 7fc2aa4ba640 1 --2- 172.21.15.154:0/3728758975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9661f1870 0x55b9661c4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:41.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.790+0000 7fc2aa4ba640 1 -- 172.21.15.154:0/3728758975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9661f1c50 msgr2=0x55b9661f4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:41.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.791+0000 7fc2aa4ba640 1 --2- 172.21.15.154:0/3728758975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9661f1c50 0x55b9661f4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:41.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.791+0000 7fc2aa4ba640 1 -- 172.21.15.154:0/3728758975 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9660f3a70 con 0x55b9661c4cb0 2023-09-23T03:18:41.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.791+0000 7fc2a94b8640 1 --2- 172.21.15.154:0/3728758975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9661f1c50 0x55b9661f4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:18:41.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.791+0000 7fc2aa4ba640 1 --2- 172.21.15.154:0/3728758975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b9661c4cb0 0x55b9661f4b40 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc2a40c8a30 tx=0x7fc29c009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5c93225c0e95d6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:41.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.791+0000 7fc28b7fe640 1 -- 172.21.15.154:0/3728758975 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c01b070 con 0x55b9661c4cb0 2023-09-23T03:18:41.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.791+0000 7fc28b7fe640 1 -- 172.21.15.154:0/3728758975 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc29c006bf0 con 0x55b9661c4cb0 2023-09-23T03:18:41.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.791+0000 7fc28b7fe640 1 -- 172.21.15.154:0/3728758975 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c004220 con 0x55b9661c4cb0 2023-09-23T03:18:41.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.792+0000 7fc2aeb87740 1 -- 172.21.15.154:0/3728758975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b9661c4cb0 msgr2=0x55b9661f4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:41.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.792+0000 7fc2aeb87740 1 --2- 172.21.15.154:0/3728758975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b9661c4cb0 0x55b9661f4b40 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc2a40c8a30 tx=0x7fc29c009190 comp rx=0 tx=0).stop 2023-09-23T03:18:41.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.792+0000 7fc2aeb87740 1 -- 172.21.15.154:0/3728758975 shutdown_connections 2023-09-23T03:18:41.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.792+0000 7fc2aeb87740 1 --2- 172.21.15.154:0/3728758975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b9661c4cb0 0x55b9661f4b40 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:41.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.792+0000 7fc2aeb87740 1 --2- 172.21.15.154:0/3728758975 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9661f1c50 0x55b9661f4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:41.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.792+0000 7fc2aeb87740 1 --2- 172.21.15.154:0/3728758975 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9661f1870 0x55b9661c4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:41.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.792+0000 7fc2aeb87740 1 -- 172.21.15.154:0/3728758975 >> 172.21.15.154:0/3728758975 conn(0x55b9661c6a80 msgr2=0x55b96620f600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:41.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.792+0000 7fc2aeb87740 1 -- 172.21.15.154:0/3728758975 shutdown_connections 2023-09-23T03:18:41.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.792+0000 7fc2aeb87740 1 -- 172.21.15.154:0/3728758975 wait complete. 2023-09-23T03:18:41.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.793+0000 7fc2aeb87740 1 -- start start 2023-09-23T03:18:41.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.793+0000 7fc2aeb87740 1 Processor -- start 2023-09-23T03:18:41.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.794+0000 7fc2aeb87740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9661f1870 0x55b9661e2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:41.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.794+0000 7fc2aeb87740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b9661f1c50 0x55b966212b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:41.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.794+0000 7fc2aeb87740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9661f0910 0x55b966213050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:41.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.794+0000 7fc2aeb87740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9660cdad0 con 0x55b9661f1c50 2023-09-23T03:18:41.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.794+0000 7fc2aeb87740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9660cd770 con 0x55b9661f0910 2023-09-23T03:18:41.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.794+0000 7fc2aeb87740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b965e29f90 con 0x55b9661f1870 2023-09-23T03:18:41.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.794+0000 7fc2aeb87740 10 client.0 _ll_register_callbacks cb 0x55b9660f1220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:18:41.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.794+0000 7fc2a94b8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b9661f1c50 0x55b966212b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:41.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.795+0000 7fc2a94b8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b9661f1c50 0x55b966212b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56806/0 (socket says 172.21.15.154:56806) 2023-09-23T03:18:41.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.795+0000 7fc2a94b8640 1 -- 172.21.15.154:0/3538310600 learned_addr learned my addr 172.21.15.154:0/3538310600 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:41.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.795+0000 7fc2a9cb9640 1 --2- 172.21.15.154:0/3538310600 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9661f1870 0x55b9661e2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:41.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.795+0000 7fc2aa4ba640 1 --2- 172.21.15.154:0/3538310600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9661f0910 0x55b966213050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:41.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.795+0000 7fc2a94b8640 1 -- 172.21.15.154:0/3538310600 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9661f1870 msgr2=0x55b9661e2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:41.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.795+0000 7fc2a94b8640 1 --2- 172.21.15.154:0/3538310600 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9661f1870 0x55b9661e2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:41.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.795+0000 7fc2a94b8640 1 -- 172.21.15.154:0/3538310600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9661f0910 msgr2=0x55b966213050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:41.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.795+0000 7fc2a94b8640 1 --2- 172.21.15.154:0/3538310600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9661f0910 0x55b966213050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:41.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.795+0000 7fc2a94b8640 1 -- 172.21.15.154:0/3538310600 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc29c009540 con 0x55b9661f1c50 2023-09-23T03:18:41.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.795+0000 7fc2a94b8640 1 --2- 172.21.15.154:0/3538310600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b9661f1c50 0x55b966212b10 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fc29800ffe0 tx=0x7fc298009070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6150dbf981f1eeb0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:41.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.795+0000 7fc288ff9640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:18:41.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.795+0000 7fc2a9cb9640 1 --2- 172.21.15.154:0/3538310600 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9661f1870 0x55b9661e2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:18:41.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.796+0000 7fc288ff9640 1 -- 172.21.15.154:0/3538310600 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc298023630 con 0x55b9661f1c50 2023-09-23T03:18:41.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.796+0000 7fc2aeb87740 1 -- 172.21.15.154:0/3538310600 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b966102ab0 con 0x55b9661f1c50 2023-09-23T03:18:41.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.796+0000 7fc288ff9640 1 -- 172.21.15.154:0/3538310600 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6720+0+0 (secure 0 0 0) 0x7fc298016e30 con 0x55b9661f1c50 2023-09-23T03:18:41.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.796+0000 7fc288ff9640 1 -- 172.21.15.154:0/3538310600 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc298023790 con 0x55b9661f1c50 2023-09-23T03:18:41.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.796+0000 7fc2aeb87740 10 client.6105 fetch_fsmap learned FSMap version 32 2023-09-23T03:18:41.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.796+0000 7fc2aeb87740 1 -- 172.21.15.154:0/3538310600 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x55b96612e0b0 con 0x55b9661f1c50 2023-09-23T03:18:41.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.796+0000 7fc288ff9640 1 -- 172.21.15.154:0/3538310600 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b966102ab0 con 0x55b9661f1c50 2023-09-23T03:18:41.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.796+0000 7fc288ff9640 1 -- 172.21.15.154:0/3538310600 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 32) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fc2980238f0 con 0x55b9661f1c50 2023-09-23T03:18:41.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.796+0000 7fc2aeb87740 10 client.6105 fetch_fsmap finished waiting for FSMap version 32 2023-09-23T03:18:41.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.796+0000 7fc2aeb87740 10 client.6105 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2023-09-23T03:18:41.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.796+0000 7fc2aeb87740 20 client.6105 populate_metadata read hostname 'smithi154' 2023-09-23T03:18:41.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.796+0000 7fc2aeb87740 1 --2- 172.21.15.154:0/3538310600 >> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] conn(0x55b9661e09f0 0x55b9661e0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:41.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.796+0000 7fc2aeb87740 4 client.6105 mds_command: new command op to 5988 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T03:18:41.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.796+0000 7fc2aeb87740 1 -- 172.21.15.154:0/3538310600 --> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b966128a10 con 0x55b9661e09f0 2023-09-23T03:18:41.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.797+0000 7fc2a9cb9640 1 --2- 172.21.15.154:0/3538310600 >> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] conn(0x55b9661e09f0 0x55b9661e0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:41.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.797+0000 7fc2a9cb9640 1 --2- 172.21.15.154:0/3538310600 >> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] conn(0x55b9661e09f0 0x55b9661e0e10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1038d48bc53166c3 server_cookie=48c11803d71c0cbc in_seq=0 out_seq=0 2023-09-23T03:18:41.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.797+0000 7fc288ff9640 10 client.6105 ms_handle_connect on v2:172.21.15.154:6834/4278807008 2023-09-23T03:18:41.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.798+0000 7fc288ff9640 1 -- 172.21.15.154:0/3538310600 <== mds.0 v2:172.21.15.154:6834/4278807008 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b966128a10 con 0x55b9661e09f0 2023-09-23T03:18:41.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.798+0000 7fc288ff9640 10 client.6105 handle_command_reply: tid=0 2023-09-23T03:18:41.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.798+0000 7fc2aeb87740 1 client.6105 shutdown 2023-09-23T03:18:41.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.798+0000 7fc2aeb87740 1 -- 172.21.15.154:0/3538310600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b9661f1c50 msgr2=0x55b966212b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:41.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.798+0000 7fc2aeb87740 1 --2- 172.21.15.154:0/3538310600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b9661f1c50 0x55b966212b10 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fc29800ffe0 tx=0x7fc298009070 comp rx=0 tx=0).stop 2023-09-23T03:18:41.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.798+0000 7fc2aeb87740 1 -- 172.21.15.154:0/3538310600 shutdown_connections 2023-09-23T03:18:41.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.798+0000 7fc2aeb87740 1 --2- 172.21.15.154:0/3538310600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9661f0910 0x55b966213050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:41.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.798+0000 7fc2aeb87740 1 --2- 172.21.15.154:0/3538310600 >> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] conn(0x55b9661e09f0 0x55b9661e0e10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:41.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.798+0000 7fc2aeb87740 1 --2- 172.21.15.154:0/3538310600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b9661f1c50 0x55b966212b10 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:41.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.798+0000 7fc2aeb87740 1 --2- 172.21.15.154:0/3538310600 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9661f1870 0x55b9661e2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:41.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.798+0000 7fc288ff9640 0 client.6105 ms_handle_reset on v2:172.21.15.154:6834/4278807008 2023-09-23T03:18:41.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.798+0000 7fc2aeb87740 1 -- 172.21.15.154:0/3538310600 >> 172.21.15.154:0/3538310600 conn(0x55b9661c4cb0 msgr2=0x55b9661c8830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:41.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.799+0000 7fc2aeb87740 1 -- 172.21.15.154:0/3538310600 shutdown_connections 2023-09-23T03:18:41.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.799+0000 7fc2aeb87740 1 -- 172.21.15.154:0/3538310600 wait complete. 2023-09-23T03:18:41.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.799+0000 7fc2aeb87740 20 client.6105 trim_cache size 0 max 16384 2023-09-23T03:18:41.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.815+0000 7fc2aeb87740 1 Processor -- start 2023-09-23T03:18:41.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.815+0000 7fc2aeb87740 1 -- start start 2023-09-23T03:18:41.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.815+0000 7fc2aeb87740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9661f0910 0x55b9661c8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:41.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.815+0000 7fc2aeb87740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9661f1870 0x55b9661c8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:41.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.816+0000 7fc2aeb87740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b9661f1c50 0x55b9661ca520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:41.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.816+0000 7fc2aeb87740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b966209d80 con 0x55b9661f1c50 2023-09-23T03:18:41.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.816+0000 7fc2aeb87740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b966231fd0 con 0x55b9661f1870 2023-09-23T03:18:41.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.816+0000 7fc2aeb87740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b96621ed40 con 0x55b9661f0910 2023-09-23T03:18:41.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.816+0000 7fc2aa4ba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b9661f1c50 0x55b9661ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:41.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.816+0000 7fc2aa4ba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b9661f1c50 0x55b9661ca520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56816/0 (socket says 172.21.15.154:56816) 2023-09-23T03:18:41.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.816+0000 7fc2aa4ba640 1 -- 172.21.15.154:0/3066371923 learned_addr learned my addr 172.21.15.154:0/3066371923 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:41.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.816+0000 7fc2a94b8640 1 --2- 172.21.15.154:0/3066371923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9661f1870 0x55b9661c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:41.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.816+0000 7fc2a9cb9640 1 --2- 172.21.15.154:0/3066371923 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9661f0910 0x55b9661c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:41.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.816+0000 7fc2aa4ba640 1 -- 172.21.15.154:0/3066371923 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9661f0910 msgr2=0x55b9661c8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:41.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.816+0000 7fc2aa4ba640 1 --2- 172.21.15.154:0/3066371923 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9661f0910 0x55b9661c8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:41.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.816+0000 7fc2aa4ba640 1 -- 172.21.15.154:0/3066371923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9661f1870 msgr2=0x55b9661c8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:41.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.816+0000 7fc2aa4ba640 1 --2- 172.21.15.154:0/3066371923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9661f1870 0x55b9661c8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:41.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.816+0000 7fc2aa4ba640 1 -- 172.21.15.154:0/3066371923 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b96615fda0 con 0x55b9661f1c50 2023-09-23T03:18:41.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.817+0000 7fc2aa4ba640 1 --2- 172.21.15.154:0/3066371923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b9661f1c50 0x55b9661ca520 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc2a40ce400 tx=0x7fc29c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aec499a11f96e8f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:41.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.817+0000 7fc28bfff640 1 -- 172.21.15.154:0/3066371923 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c01b040 con 0x55b9661f1c50 2023-09-23T03:18:41.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.817+0000 7fc28bfff640 1 -- 172.21.15.154:0/3066371923 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc29c005c10 con 0x55b9661f1c50 2023-09-23T03:18:41.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.817+0000 7fc28bfff640 1 -- 172.21.15.154:0/3066371923 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c004040 con 0x55b9661f1c50 2023-09-23T03:18:41.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.817+0000 7fc2aeb87740 1 -- 172.21.15.154:0/3066371923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b9661f1c50 msgr2=0x55b9661ca520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:41.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.818+0000 7fc2aeb87740 1 --2- 172.21.15.154:0/3066371923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b9661f1c50 0x55b9661ca520 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc2a40ce400 tx=0x7fc29c016000 comp rx=0 tx=0).stop 2023-09-23T03:18:41.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.818+0000 7fc2aeb87740 1 -- 172.21.15.154:0/3066371923 shutdown_connections 2023-09-23T03:18:41.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.818+0000 7fc2aeb87740 1 --2- 172.21.15.154:0/3066371923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b9661f1c50 0x55b9661ca520 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:41.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.818+0000 7fc2aeb87740 1 --2- 172.21.15.154:0/3066371923 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9661f1870 0x55b9661c8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:41.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.818+0000 7fc2aeb87740 1 --2- 172.21.15.154:0/3066371923 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9661f0910 0x55b9661c8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:41.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.818+0000 7fc2aeb87740 1 -- 172.21.15.154:0/3066371923 >> 172.21.15.154:0/3066371923 conn(0x55b9661c4cb0 msgr2=0x55b96620fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:41.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.818+0000 7fc2aeb87740 1 -- 172.21.15.154:0/3066371923 shutdown_connections 2023-09-23T03:18:41.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.818+0000 7fc2aeb87740 1 -- 172.21.15.154:0/3066371923 wait complete. 2023-09-23T03:18:41.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.818+0000 7fc2aeb87740 1 -- start start 2023-09-23T03:18:41.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.819+0000 7fc2aeb87740 1 Processor -- start 2023-09-23T03:18:41.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.820+0000 7fc2aeb87740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9661f0910 0x55b9661c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:41.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.820+0000 7fc2aeb87740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b9661f1870 0x55b9662777a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:41.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.820+0000 7fc2aeb87740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9661f1c50 0x55b9661ded40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:41.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.820+0000 7fc2aeb87740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9661c9870 con 0x55b9661f1870 2023-09-23T03:18:41.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.820+0000 7fc2aeb87740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b966277ce0 con 0x55b9661f0910 2023-09-23T03:18:41.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.820+0000 7fc2aeb87740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b9661f43f0 con 0x55b9661f1c50 2023-09-23T03:18:41.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.820+0000 7fc2aeb87740 10 client.0 _ll_register_callbacks cb 0x7fc28c0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:18:41.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.820+0000 7fc2a94b8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b9661f1870 0x55b9662777a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:41.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.820+0000 7fc2a94b8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b9661f1870 0x55b9662777a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56832/0 (socket says 172.21.15.154:56832) 2023-09-23T03:18:41.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.820+0000 7fc2a94b8640 1 -- 172.21.15.154:0/651637419 learned_addr learned my addr 172.21.15.154:0/651637419 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:18:41.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.820+0000 7fc2a9cb9640 1 --2- 172.21.15.154:0/651637419 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9661f0910 0x55b9661c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:41.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.820+0000 7fc2aa4ba640 1 --2- 172.21.15.154:0/651637419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9661f1c50 0x55b9661ded40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:41.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.820+0000 7fc2a94b8640 1 -- 172.21.15.154:0/651637419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9661f1c50 msgr2=0x55b9661ded40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:41.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.820+0000 7fc2a94b8640 1 --2- 172.21.15.154:0/651637419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9661f1c50 0x55b9661ded40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:41.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.820+0000 7fc2a94b8640 1 -- 172.21.15.154:0/651637419 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9661f0910 msgr2=0x55b9661c89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:18:41.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.820+0000 7fc2a94b8640 1 --2- 172.21.15.154:0/651637419 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9661f0910 0x55b9661c89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:41.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.820+0000 7fc2a94b8640 1 -- 172.21.15.154:0/651637419 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b9660f3a70 con 0x55b9661f1870 2023-09-23T03:18:41.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.821+0000 7fc2a94b8640 1 --2- 172.21.15.154:0/651637419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b9661f1870 0x55b9662777a0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc29800ffe0 tx=0x7fc298001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d872961133edc412 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:18:41.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.821+0000 7fc288ff9640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:18:41.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.821+0000 7fc288ff9640 1 -- 172.21.15.154:0/651637419 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29800fb60 con 0x55b9661f1870 2023-09-23T03:18:41.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.821+0000 7fc288ff9640 1 -- 172.21.15.154:0/651637419 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 6720+0+0 (secure 0 0 0) 0x7fc298036440 con 0x55b9661f1870 2023-09-23T03:18:41.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.821+0000 7fc2aeb87740 1 -- 172.21.15.154:0/651637419 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b9661564d0 con 0x55b9661f1870 2023-09-23T03:18:41.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.821+0000 7fc288ff9640 1 -- 172.21.15.154:0/651637419 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc298036600 con 0x55b9661f1870 2023-09-23T03:18:41.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.821+0000 7fc2aeb87740 10 client.6111 fetch_fsmap learned FSMap version 32 2023-09-23T03:18:41.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.821+0000 7fc2aeb87740 1 -- 172.21.15.154:0/651637419 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x55b9660ccce0 con 0x55b9661f1870 2023-09-23T03:18:41.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.821+0000 7fc288ff9640 1 -- 172.21.15.154:0/651637419 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b9661564d0 con 0x55b9661f1870 2023-09-23T03:18:41.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.821+0000 7fc288ff9640 1 -- 172.21.15.154:0/651637419 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 32) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fc298036760 con 0x55b9661f1870 2023-09-23T03:18:41.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.822+0000 7fc2aeb87740 10 client.6111 fetch_fsmap finished waiting for FSMap version 32 2023-09-23T03:18:41.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.822+0000 7fc2aeb87740 10 client.6111 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2023-09-23T03:18:41.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.822+0000 7fc2aeb87740 20 client.6111 populate_metadata read hostname 'smithi154' 2023-09-23T03:18:41.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.822+0000 7fc2aeb87740 1 --2- 172.21.15.154:0/651637419 >> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] conn(0x55b9662290e0 0x55b966204f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:18:41.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.822+0000 7fc2aeb87740 4 client.6111 mds_command: new command op to 5988 tid=0[{"prefix": "scrub status"}] 2023-09-23T03:18:41.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.822+0000 7fc2aeb87740 1 -- 172.21.15.154:0/651637419 --> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b966128730 con 0x55b9662290e0 2023-09-23T03:18:41.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.822+0000 7fc2a9cb9640 1 --2- 172.21.15.154:0/651637419 >> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] conn(0x55b9662290e0 0x55b966204f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:18:41.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.822+0000 7fc2a9cb9640 1 --2- 172.21.15.154:0/651637419 >> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] conn(0x55b9662290e0 0x55b966204f90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae3dffa197be491a server_cookie=700a78443e934ad3 in_seq=0 out_seq=0 2023-09-23T03:18:41.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.822+0000 7fc288ff9640 10 client.6111 ms_handle_connect on v2:172.21.15.154:6834/4278807008 2023-09-23T03:18:41.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.823+0000 7fc288ff9640 1 -- 172.21.15.154:0/651637419 <== mds.0 v2:172.21.15.154:6834/4278807008 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b966128730 con 0x55b9662290e0 2023-09-23T03:18:41.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.823+0000 7fc288ff9640 10 client.6111 handle_command_reply: tid=0 2023-09-23T03:18:41.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.823+0000 7fc2aeb87740 1 client.6111 shutdown 2023-09-23T03:18:41.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.823+0000 7fc2aeb87740 1 -- 172.21.15.154:0/651637419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b9661f1870 msgr2=0x55b9662777a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:41.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.823+0000 7fc2aeb87740 1 --2- 172.21.15.154:0/651637419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b9661f1870 0x55b9662777a0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc29800ffe0 tx=0x7fc298001a00 comp rx=0 tx=0).stop 2023-09-23T03:18:41.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.823+0000 7fc2aeb87740 1 -- 172.21.15.154:0/651637419 shutdown_connections 2023-09-23T03:18:41.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.823+0000 7fc2aeb87740 1 --2- 172.21.15.154:0/651637419 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55b9661f1c50 0x55b9661ded40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:41.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.823+0000 7fc2aeb87740 1 --2- 172.21.15.154:0/651637419 >> [v2:172.21.15.154:6834/4278807008,v1:172.21.15.154:6836/4278807008] conn(0x55b9662290e0 0x55b966204f90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:41.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.823+0000 7fc2aeb87740 1 --2- 172.21.15.154:0/651637419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b9661f1870 0x55b9662777a0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:41.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.823+0000 7fc2aeb87740 1 --2- 172.21.15.154:0/651637419 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55b9661f0910 0x55b9661c89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:41.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.823+0000 7fc288ff9640 0 client.6111 ms_handle_reset on v2:172.21.15.154:6834/4278807008 2023-09-23T03:18:41.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.823+0000 7fc2aeb87740 1 -- 172.21.15.154:0/651637419 >> 172.21.15.154:0/651637419 conn(0x55b9661c4cb0 msgr2=0x55b966212fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:41.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.823+0000 7fc2aeb87740 1 -- 172.21.15.154:0/651637419 shutdown_connections 2023-09-23T03:18:41.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.823+0000 7fc2aeb87740 1 -- 172.21.15.154:0/651637419 wait complete. 2023-09-23T03:18:41.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.823+0000 7fc2aeb87740 20 client.6111 trim_cache size 0 max 16384 2023-09-23T03:18:41.828 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:18:41.828 INFO:teuthology.orchestra.run.smithi154.stdout: "status": "no active scrubs running", 2023-09-23T03:18:41.828 INFO:teuthology.orchestra.run.smithi154.stdout: "scrubs": {} 2023-09-23T03:18:41.828 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:18:41.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.824+0000 7fc2aacbb640 1 -- 172.21.15.154:0/2986091872 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc29006af30 msgr2=0x7fc29006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:41.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.824+0000 7fc2aacbb640 1 --2- 172.21.15.154:0/2986091872 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc29006af30 0x7fc29006d3f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc298001eb0 tx=0x7fc298000f10 comp rx=0 tx=0).stop 2023-09-23T03:18:41.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.824+0000 7fc2aacbb640 1 -- 172.21.15.154:0/2986091872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc28c003970 msgr2=0x7fc2a40c6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:18:41.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.824+0000 7fc2aacbb640 1 --2- 172.21.15.154:0/2986091872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc28c003970 0x7fc2a40c6b60 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc29402bf70 tx=0x7fc2940014a0 comp rx=0 tx=0).stop 2023-09-23T03:18:41.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.824+0000 7fc2aacbb640 1 -- 172.21.15.154:0/2986091872 shutdown_connections 2023-09-23T03:18:41.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.824+0000 7fc2aacbb640 1 --2- 172.21.15.154:0/2986091872 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc29006af30 0x7fc29006d3f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:41.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.824+0000 7fc2aacbb640 1 --2- 172.21.15.154:0/2986091872 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2a41f16c0 0x7fc2a41f1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:41.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.824+0000 7fc2aacbb640 1 --2- 172.21.15.154:0/2986091872 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2a40c4540 0x7fc2a41f1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:41.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.824+0000 7fc2aacbb640 1 --2- 172.21.15.154:0/2986091872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc28c003970 0x7fc2a40c6b60 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:18:41.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.824+0000 7fc2aacbb640 1 -- 172.21.15.154:0/2986091872 >> 172.21.15.154:0/2986091872 conn(0x7fc2a406a300 msgr2=0x7fc2a415bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:18:41.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.824+0000 7fc2aacbb640 1 -- 172.21.15.154:0/2986091872 shutdown_connections 2023-09-23T03:18:41.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:41.824+0000 7fc2aacbb640 1 -- 172.21.15.154:0/2986091872 wait complete. 2023-09-23T03:18:41.840 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-09-23T03:18:41.840 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/0 2023-09-23T03:18:41.841 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:18:41.841 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:18:41.841 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/0" 2023-09-23T03:18:41.841 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:18:41.841 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:18:41.841 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 6291456: 2023-09-23T03:18:41.841 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:18:41.841 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 6291456 2023-09-23T03:18:41.841 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:18:41.841 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:18:41.842 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:18:41.842 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:18:41.842 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:18:41.842 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:18:45.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:18:45.511 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/1 2023-09-23T03:18:45.511 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:18:45.512 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:18:45.512 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/1" 2023-09-23T03:18:45.512 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:18:45.512 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:18:45.512 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 6291456: 2023-09-23T03:18:45.512 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:18:45.512 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 6291456 2023-09-23T03:18:45.512 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:18:45.512 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:18:45.512 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:18:45.512 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:18:45.513 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:18:45.513 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:18:49.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:18:49.156 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/2 2023-09-23T03:18:49.156 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:18:49.156 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:18:49.156 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/2" 2023-09-23T03:18:49.157 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:18:49.157 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:18:49.157 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 6291456: 2023-09-23T03:18:49.157 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:18:49.157 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 6291456 2023-09-23T03:18:49.157 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:18:49.157 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:18:49.157 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:18:49.157 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:18:49.158 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:18:49.158 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:18:52.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:18:52.848 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/3 2023-09-23T03:18:52.848 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:18:52.848 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:18:52.848 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/3" 2023-09-23T03:18:52.849 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:18:52.849 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:18:52.849 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 6291456: 2023-09-23T03:18:52.849 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:18:52.849 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 6291456 2023-09-23T03:18:52.849 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:18:52.849 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:18:52.849 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:18:52.849 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:18:52.849 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:18:52.850 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:18:56.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:18:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:18:56.500 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/4 2023-09-23T03:18:56.500 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:18:56.500 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:18:56.500 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/4" 2023-09-23T03:18:56.501 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:18:56.501 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:18:56.501 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 6291456: 2023-09-23T03:18:56.501 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:18:56.501 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 6291456 2023-09-23T03:18:56.501 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:18:56.501 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:18:56.501 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:18:56.501 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:18:56.501 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:18:56.501 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:19:00.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:19:00.138 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-09-23T03:19:00.138 DEBUG:teuthology.orchestra.run.smithi154:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-23T03:19:00.294 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-23T03:19:00.414 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-09-23T03:19:00.414 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:19:00.414 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:19:00.478 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:19:00.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.694+0000 7f487db80640 1 Processor -- start 2023-09-23T03:19:00.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.694+0000 7f487db80640 1 -- start start 2023-09-23T03:19:00.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.696+0000 7f487db80640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f487806c7a0 0x7f487815eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:00.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.696+0000 7f487db80640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f487806bff0 con 0x7f48780704d0 2023-09-23T03:19:00.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.696+0000 7f487db80640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f487806cb80 con 0x7f487815ab00 2023-09-23T03:19:00.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.696+0000 7f487db80640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4878155070 con 0x7f487806c7a0 2023-09-23T03:19:00.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.696+0000 7f487d37f640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f48780704d0 0x7f48780ca500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:33274/0 (socket says 172.21.15.154:33274) 2023-09-23T03:19:00.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.696+0000 7f487d37f640 1 -- 172.21.15.154:0/3756249634 learned_addr learned my addr 172.21.15.154:0/3756249634 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:19:00.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.696+0000 7f487cb7e640 1 --2- 172.21.15.154:0/3756249634 >> v2:172.21.15.190:3300/0 conn(0x7f487806c7a0 0x7f487815eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:00.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.697+0000 7f48777fe640 1 -- 172.21.15.154:0/3756249634 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4019214860 0 0) 0x7f487806bff0 con 0x7f48780704d0 2023-09-23T03:19:00.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.697+0000 7f48777fe640 1 -- 172.21.15.154:0/3756249634 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4868003520 con 0x7f48780704d0 2023-09-23T03:19:00.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.697+0000 7f48777fe640 1 -- 172.21.15.154:0/3756249634 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3557762793 0 0) 0x7f4868003520 con 0x7f48780704d0 2023-09-23T03:19:00.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.697+0000 7f48777fe640 1 -- 172.21.15.154:0/3756249634 >> v2:172.21.15.190:3300/0 conn(0x7f487806c7a0 msgr2=0x7f487815eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:00.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.697+0000 7f48777fe640 1 --2- 172.21.15.154:0/3756249634 >> v2:172.21.15.190:3300/0 conn(0x7f487806c7a0 0x7f487815eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:00.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.697+0000 7f48777fe640 1 -- 172.21.15.154:0/3756249634 >> v1:172.21.15.190:6789/0 conn(0x7f487815ab00 legacy=0x7f487806fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:00.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.697+0000 7f48777fe640 1 -- 172.21.15.154:0/3756249634 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f487815b630 con 0x7f48780704d0 2023-09-23T03:19:00.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.697+0000 7f48777fe640 1 -- 172.21.15.154:0/3756249634 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f48700019f0 con 0x7f48780704d0 2023-09-23T03:19:00.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.697+0000 7f48777fe640 1 -- 172.21.15.154:0/3756249634 >> v1:172.21.15.154:6789/0 conn(0x7f48780704d0 legacy=0x7f48780ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:00.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.697+0000 7f48777fe640 1 --2- 172.21.15.154:0/3756249634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4868003970 0x7f4868003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:00.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.697+0000 7f48777fe640 1 --2- 172.21.15.154:0/3756249634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48680047c0 0x7f4868004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:00.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.697+0000 7f48777fe640 1 --2- 172.21.15.154:0/3756249634 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4868005180 0x7f486800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:00.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.697+0000 7f48777fe640 1 -- 172.21.15.154:0/3756249634 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486800bff0 con 0x7f4868005180 2023-09-23T03:19:00.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.697+0000 7f48777fe640 1 -- 172.21.15.154:0/3756249634 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486800c160 con 0x7f4868003970 2023-09-23T03:19:00.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.697+0000 7f48777fe640 1 -- 172.21.15.154:0/3756249634 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f486800c320 con 0x7f48680047c0 2023-09-23T03:19:00.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.697+0000 7f487d37f640 1 --2- 172.21.15.154:0/3756249634 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4868005180 0x7f486800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:00.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.698+0000 7f487cb7e640 1 --2- 172.21.15.154:0/3756249634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4868003970 0x7f4868003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:00.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.698+0000 7f4877fff640 1 --2- 172.21.15.154:0/3756249634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48680047c0 0x7f4868004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:00.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.698+0000 7f487d37f640 1 -- 172.21.15.154:0/3756249634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48680047c0 msgr2=0x7f4868004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:00.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.698+0000 7f487d37f640 1 --2- 172.21.15.154:0/3756249634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48680047c0 0x7f4868004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:00.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.698+0000 7f487d37f640 1 -- 172.21.15.154:0/3756249634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4868003970 msgr2=0x7f4868003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:00.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.698+0000 7f487d37f640 1 --2- 172.21.15.154:0/3756249634 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4868003970 0x7f4868003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:00.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.698+0000 7f487d37f640 1 -- 172.21.15.154:0/3756249634 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4868003520 con 0x7f4868005180 2023-09-23T03:19:00.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.698+0000 7f4877fff640 1 --2- 172.21.15.154:0/3756249634 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48680047c0 0x7f4868004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:00.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.699+0000 7f487d37f640 1 -- 172.21.15.154:0/3756249634 reap_dead start 2023-09-23T03:19:00.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.699+0000 7f487d37f640 1 --2- 172.21.15.154:0/3756249634 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4868005180 0x7f486800b980 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f487000a830 tx=0x7f48680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=129761d156d0cfc8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:00.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.699+0000 7f48777fe640 1 -- 172.21.15.154:0/3756249634 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48700072d0 con 0x7f4868005180 2023-09-23T03:19:00.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.699+0000 7f48777fe640 1 -- 172.21.15.154:0/3756249634 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f487002f070 con 0x7f4868005180 2023-09-23T03:19:00.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.699+0000 7f48777fe640 1 -- 172.21.15.154:0/3756249634 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48700076b0 con 0x7f4868005180 2023-09-23T03:19:00.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.700+0000 7f487db80640 1 -- 172.21.15.154:0/3756249634 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4868005180 msgr2=0x7f486800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:00.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.700+0000 7f487db80640 1 --2- 172.21.15.154:0/3756249634 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4868005180 0x7f486800b980 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f487000a830 tx=0x7f48680057f0 comp rx=0 tx=0).stop 2023-09-23T03:19:00.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.700+0000 7f487db80640 1 -- 172.21.15.154:0/3756249634 shutdown_connections 2023-09-23T03:19:00.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.700+0000 7f487db80640 1 --2- 172.21.15.154:0/3756249634 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4868005180 0x7f486800b980 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:00.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.700+0000 7f487db80640 1 -- 172.21.15.154:0/3756249634 >> 172.21.15.154:0/3756249634 conn(0x7f487806a390 msgr2=0x7f48780afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:00.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.700+0000 7f487db80640 1 -- 172.21.15.154:0/3756249634 shutdown_connections 2023-09-23T03:19:00.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.700+0000 7f487db80640 1 -- 172.21.15.154:0/3756249634 wait complete. 2023-09-23T03:19:00.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.702+0000 7f487db80640 1 Processor -- start 2023-09-23T03:19:00.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.702+0000 7f487db80640 1 -- start start 2023-09-23T03:19:00.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.703+0000 7f487db80640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4868005180 0x7f48781f1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:00.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.703+0000 7f487db80640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48780b3b70 0x7f48780b1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:00.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.703+0000 7f487db80640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48781443b0 0x7f4878146a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:00.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.703+0000 7f487db80640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48780b1d70 con 0x7f48780b3b70 2023-09-23T03:19:00.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.703+0000 7f487db80640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48780b3ff0 con 0x7f48781443b0 2023-09-23T03:19:00.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.703+0000 7f487db80640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48780b1210 con 0x7f4868005180 2023-09-23T03:19:00.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.703+0000 7f4877fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48780b3b70 0x7f48780b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:00.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.703+0000 7f4877fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48780b3b70 0x7f48780b1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:32882/0 (socket says 172.21.15.154:32882) 2023-09-23T03:19:00.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.703+0000 7f4877fff640 1 -- 172.21.15.154:0/1929950129 learned_addr learned my addr 172.21.15.154:0/1929950129 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:00.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.704+0000 7f4877fff640 1 -- 172.21.15.154:0/1929950129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4868005180 msgr2=0x7f48781f1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:00.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.704+0000 7f4877fff640 1 --2- 172.21.15.154:0/1929950129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4868005180 0x7f48781f1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:00.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.704+0000 7f4877fff640 1 -- 172.21.15.154:0/1929950129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48781443b0 msgr2=0x7f4878146a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:00.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.704+0000 7f4877fff640 1 --2- 172.21.15.154:0/1929950129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48781443b0 0x7f4878146a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:00.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.704+0000 7f4877fff640 1 -- 172.21.15.154:0/1929950129 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48781f2470 con 0x7f48780b3b70 2023-09-23T03:19:00.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.704+0000 7f4877fff640 1 --2- 172.21.15.154:0/1929950129 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48780b3b70 0x7f48780b1830 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f486400d560 tx=0x7f486400da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:00.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.704+0000 7f48757fa640 1 -- 172.21.15.154:0/1929950129 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4864005480 con 0x7f48780b3b70 2023-09-23T03:19:00.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.704+0000 7f48757fa640 1 -- 172.21.15.154:0/1929950129 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4864003610 con 0x7f48780b3b70 2023-09-23T03:19:00.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.704+0000 7f48757fa640 1 -- 172.21.15.154:0/1929950129 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4864003980 con 0x7f48780b3b70 2023-09-23T03:19:00.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.704+0000 7f487db80640 1 -- 172.21.15.154:0/1929950129 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48781f2750 con 0x7f48780b3b70 2023-09-23T03:19:00.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.704+0000 7f487db80640 1 -- 172.21.15.154:0/1929950129 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48781464e0 con 0x7f48780b3b70 2023-09-23T03:19:00.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.706+0000 7f487db80640 1 -- 172.21.15.154:0/1929950129 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48780736b0 con 0x7f48780b3b70 2023-09-23T03:19:00.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.707+0000 7f48757fa640 1 -- 172.21.15.154:0/1929950129 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f4864003ae0 con 0x7f48780b3b70 2023-09-23T03:19:00.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.707+0000 7f48757fa640 1 --2- 172.21.15.154:0/1929950129 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f485c06af60 0x7f485c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:00.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.707+0000 7f48757fa640 1 -- 172.21.15.154:0/1929950129 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6720+0+0 (secure 0 0 0) 0x7f4864089ab0 con 0x7f48780b3b70 2023-09-23T03:19:00.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.708+0000 7f487cb7e640 1 --2- 172.21.15.154:0/1929950129 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f485c06af60 0x7f485c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:00.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.708+0000 7f487cb7e640 1 --2- 172.21.15.154:0/1929950129 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f485c06af60 0x7f485c06d420 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4878146410 tx=0x7f486c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:00.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.712+0000 7f48757fa640 1 -- 172.21.15.154:0/1929950129 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f4864054680 con 0x7f48780b3b70 2023-09-23T03:19:00.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.899+0000 7f487db80640 1 -- 172.21.15.154:0/1929950129 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f487804e920 con 0x7f48780b3b70 2023-09-23T03:19:00.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.902+0000 7f48757fa640 1 -- 172.21.15.154:0/1929950129 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+18984 (secure 0 0 0) 0x7f4864001230 con 0x7f48780b3b70 2023-09-23T03:19:00.904 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:19:00.904 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":49,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:18:04.269705+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:17:08.933690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-09-23T03:17:10.141711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data_extra","create_time":"2023-09-23T03:17:21.021560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.35","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6837/4079550625":"2023-09-24T03:18:03.516922+0000","172.21.15.154:6835/4079550625":"2023-09-24T03:18:03.516922+0000","172.21.15.154:6837/617287200":"2023-09-24T03:17:57.206517+0000","172.21.15.190:6834/2131436414":"2023-09-24T03:17:55.088768+0000","172.21.15.154:6835/617287200":"2023-09-24T03:17:57.206517+0000","172.21.15.190:6832/2131436414":"2023-09-24T03:17:55.088768+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:19:00.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.905+0000 7f487db80640 1 -- 172.21.15.154:0/1929950129 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f485c06af60 msgr2=0x7f485c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:00.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.905+0000 7f487db80640 1 --2- 172.21.15.154:0/1929950129 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f485c06af60 0x7f485c06d420 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4878146410 tx=0x7f486c008040 comp rx=0 tx=0).stop 2023-09-23T03:19:00.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.905+0000 7f487db80640 1 -- 172.21.15.154:0/1929950129 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48780b3b70 msgr2=0x7f48780b1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:00.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.905+0000 7f487db80640 1 --2- 172.21.15.154:0/1929950129 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48780b3b70 0x7f48780b1830 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f486400d560 tx=0x7f486400da20 comp rx=0 tx=0).stop 2023-09-23T03:19:00.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.905+0000 7f487db80640 1 -- 172.21.15.154:0/1929950129 shutdown_connections 2023-09-23T03:19:00.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.905+0000 7f487db80640 1 --2- 172.21.15.154:0/1929950129 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f485c06af60 0x7f485c06d420 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:00.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.905+0000 7f487db80640 1 --2- 172.21.15.154:0/1929950129 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48781443b0 0x7f4878146a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:00.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.905+0000 7f487db80640 1 --2- 172.21.15.154:0/1929950129 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48780b3b70 0x7f48780b1830 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:00.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.905+0000 7f487db80640 1 --2- 172.21.15.154:0/1929950129 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4868005180 0x7f48781f1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:00.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.905+0000 7f487db80640 1 -- 172.21.15.154:0/1929950129 >> 172.21.15.154:0/1929950129 conn(0x7f487806a390 msgr2=0x7f48780aef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:00.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.905+0000 7f487db80640 1 -- 172.21.15.154:0/1929950129 shutdown_connections 2023-09-23T03:19:00.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:00.905+0000 7f487db80640 1 -- 172.21.15.154:0/1929950129 wait complete. 2023-09-23T03:19:00.925 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:17:08.933690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:17:10.141711+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-09-23T03:17:21.021560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T03:19:00.925 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:19:01.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.139+0000 7f2c261b6640 1 Processor -- start 2023-09-23T03:19:01.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.139+0000 7f2c261b6640 1 -- start start 2023-09-23T03:19:01.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.140+0000 7f2c261b6640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c200cc760 0x7f2c2015d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:01.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.140+0000 7f2c261b6640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c200cac50 con 0x7f2c200b3050 2023-09-23T03:19:01.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.140+0000 7f2c261b6640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c2015d9d0 con 0x7f2c200ccb40 2023-09-23T03:19:01.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.140+0000 7f2c261b6640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2015dcb0 con 0x7f2c200cc760 2023-09-23T03:19:01.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.141+0000 7f2c1effd640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f2c200b3050 0x7f2c200c8b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:33286/0 (socket says 172.21.15.154:33286) 2023-09-23T03:19:01.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.141+0000 7f2c1effd640 1 -- 172.21.15.154:0/280826980 learned_addr learned my addr 172.21.15.154:0/280826980 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:19:01.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.141+0000 7f2c1f7fe640 1 --2- 172.21.15.154:0/280826980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c200cc760 0x7f2c2015d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:01.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.141+0000 7f2c1e7fc640 1 -- 172.21.15.154:0/280826980 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4287391278 0 0) 0x7f2c200cac50 con 0x7f2c200b3050 2023-09-23T03:19:01.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.141+0000 7f2c1e7fc640 1 -- 172.21.15.154:0/280826980 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c08005730 con 0x7f2c200b3050 2023-09-23T03:19:01.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.141+0000 7f2c1e7fc640 1 -- 172.21.15.154:0/280826980 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 687397357 0 0) 0x7f2c08005730 con 0x7f2c200b3050 2023-09-23T03:19:01.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.142+0000 7f2c1e7fc640 1 -- 172.21.15.154:0/280826980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c200cc760 msgr2=0x7f2c2015d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:01.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.142+0000 7f2c1e7fc640 1 --2- 172.21.15.154:0/280826980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c200cc760 0x7f2c2015d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:01.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.142+0000 7f2c1e7fc640 1 -- 172.21.15.154:0/280826980 >> v1:172.21.15.190:6789/0 conn(0x7f2c200ccb40 legacy=0x7f2c200ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:01.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.142+0000 7f2c1e7fc640 1 -- 172.21.15.154:0/280826980 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c2015df90 con 0x7f2c200b3050 2023-09-23T03:19:01.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.142+0000 7f2c1e7fc640 1 -- 172.21.15.154:0/280826980 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f2c14001670 con 0x7f2c200b3050 2023-09-23T03:19:01.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.142+0000 7f2c1e7fc640 1 -- 172.21.15.154:0/280826980 >> v1:172.21.15.154:6789/0 conn(0x7f2c200b3050 legacy=0x7f2c200c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:01.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.142+0000 7f2c1e7fc640 1 --2- 172.21.15.154:0/280826980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c200cc760 0x7f2c08005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:01.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.142+0000 7f2c1e7fc640 1 --2- 172.21.15.154:0/280826980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c08006180 0x7f2c080069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:01.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.142+0000 7f2c1e7fc640 1 --2- 172.21.15.154:0/280826980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c08006f10 0x7f2c08007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:01.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.142+0000 7f2c1e7fc640 1 -- 172.21.15.154:0/280826980 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2015dcb0 con 0x7f2c08006f10 2023-09-23T03:19:01.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.142+0000 7f2c1e7fc640 1 -- 172.21.15.154:0/280826980 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0800dcd0 con 0x7f2c08006180 2023-09-23T03:19:01.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.142+0000 7f2c1e7fc640 1 -- 172.21.15.154:0/280826980 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0800de90 con 0x7f2c200cc760 2023-09-23T03:19:01.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.142+0000 7f2c1ffff640 1 --2- 172.21.15.154:0/280826980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c08006f10 0x7f2c08007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:01.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.142+0000 7f2c1effd640 1 --2- 172.21.15.154:0/280826980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c08006180 0x7f2c080069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:01.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.142+0000 7f2c1f7fe640 1 --2- 172.21.15.154:0/280826980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c200cc760 0x7f2c08005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:01.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.142+0000 7f2c1ffff640 1 -- 172.21.15.154:0/280826980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c200cc760 msgr2=0x7f2c08005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:01.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.142+0000 7f2c1ffff640 1 --2- 172.21.15.154:0/280826980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c200cc760 0x7f2c08005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:01.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.142+0000 7f2c1ffff640 1 -- 172.21.15.154:0/280826980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c08006180 msgr2=0x7f2c080069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:01.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.142+0000 7f2c1ffff640 1 --2- 172.21.15.154:0/280826980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c08006180 0x7f2c080069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:01.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.142+0000 7f2c1ffff640 1 -- 172.21.15.154:0/280826980 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c08005730 con 0x7f2c08006f10 2023-09-23T03:19:01.147 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.144+0000 7f2c1ffff640 1 --2- 172.21.15.154:0/280826980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c08006f10 0x7f2c08007390 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f2c18001e50 tx=0x7f2c08007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5fd6858a7a82a89 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:01.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.144+0000 7f2c1e7fc640 1 -- 172.21.15.154:0/280826980 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c18003b70 con 0x7f2c08006f10 2023-09-23T03:19:01.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.144+0000 7f2c1e7fc640 1 -- 172.21.15.154:0/280826980 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c18003cd0 con 0x7f2c08006f10 2023-09-23T03:19:01.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.144+0000 7f2c1e7fc640 1 -- 172.21.15.154:0/280826980 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c18005310 con 0x7f2c08006f10 2023-09-23T03:19:01.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.144+0000 7f2c261b6640 1 -- 172.21.15.154:0/280826980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c08006f10 msgr2=0x7f2c08007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:01.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.144+0000 7f2c261b6640 1 --2- 172.21.15.154:0/280826980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c08006f10 0x7f2c08007390 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f2c18001e50 tx=0x7f2c08007a00 comp rx=0 tx=0).stop 2023-09-23T03:19:01.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.144+0000 7f2c1ffff640 1 -- 172.21.15.154:0/280826980 reap_dead start 2023-09-23T03:19:01.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.144+0000 7f2c261b6640 1 -- 172.21.15.154:0/280826980 shutdown_connections 2023-09-23T03:19:01.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.144+0000 7f2c261b6640 1 -- 172.21.15.154:0/280826980 >> 172.21.15.154:0/280826980 conn(0x7f2c20069eb0 msgr2=0x7f2c2006a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:01.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.145+0000 7f2c261b6640 1 -- 172.21.15.154:0/280826980 shutdown_connections 2023-09-23T03:19:01.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.145+0000 7f2c261b6640 1 -- 172.21.15.154:0/280826980 wait complete. 2023-09-23T03:19:01.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.146+0000 7f2c261b6640 1 Processor -- start 2023-09-23T03:19:01.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.146+0000 7f2c261b6640 1 -- start start 2023-09-23T03:19:01.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.147+0000 7f2c261b6640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c20156f50 0x7f2c2015c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:01.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.147+0000 7f2c261b6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c2015ca70 0x7f2c200c9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:01.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.147+0000 7f2c261b6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c200c9dc0 0x7f2c200ac190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:01.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.147+0000 7f2c261b6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c200ca240 con 0x7f2c2015ca70 2023-09-23T03:19:01.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.147+0000 7f2c261b6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c201e3fc0 con 0x7f2c200c9dc0 2023-09-23T03:19:01.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.147+0000 7f2c261b6640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2015b0a0 con 0x7f2c20156f50 2023-09-23T03:19:01.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.148+0000 7f2c1effd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c2015ca70 0x7f2c200c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:01.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.148+0000 7f2c1effd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c2015ca70 0x7f2c200c9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:32912/0 (socket says 172.21.15.154:32912) 2023-09-23T03:19:01.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.148+0000 7f2c1effd640 1 -- 172.21.15.154:0/2561405774 learned_addr learned my addr 172.21.15.154:0/2561405774 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:01.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.148+0000 7f2c1effd640 1 -- 172.21.15.154:0/2561405774 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c20156f50 msgr2=0x7f2c2015c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:01.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.148+0000 7f2c1effd640 1 --2- 172.21.15.154:0/2561405774 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c20156f50 0x7f2c2015c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:01.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.148+0000 7f2c1effd640 1 -- 172.21.15.154:0/2561405774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c200c9dc0 msgr2=0x7f2c200ac190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:01.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.148+0000 7f2c1effd640 1 --2- 172.21.15.154:0/2561405774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c200c9dc0 0x7f2c200ac190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:01.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.148+0000 7f2c1effd640 1 -- 172.21.15.154:0/2561405774 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c2015a820 con 0x7f2c2015ca70 2023-09-23T03:19:01.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.148+0000 7f2c1effd640 1 --2- 172.21.15.154:0/2561405774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c2015ca70 0x7f2c200c9880 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f2c1400eb50 tx=0x7f2c1400c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:01.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.149+0000 7f2c07fff640 1 -- 172.21.15.154:0/2561405774 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c14005e10 con 0x7f2c2015ca70 2023-09-23T03:19:01.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.149+0000 7f2c07fff640 1 -- 172.21.15.154:0/2561405774 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c1400ee50 con 0x7f2c2015ca70 2023-09-23T03:19:01.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.149+0000 7f2c261b6640 1 -- 172.21.15.154:0/2561405774 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c2015aae0 con 0x7f2c2015ca70 2023-09-23T03:19:01.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.149+0000 7f2c07fff640 1 -- 172.21.15.154:0/2561405774 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1400cd20 con 0x7f2c2015ca70 2023-09-23T03:19:01.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.149+0000 7f2c261b6640 1 -- 172.21.15.154:0/2561405774 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c20157e20 con 0x7f2c2015ca70 2023-09-23T03:19:01.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.150+0000 7f2c261b6640 1 -- 172.21.15.154:0/2561405774 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c2004e920 con 0x7f2c2015ca70 2023-09-23T03:19:01.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.151+0000 7f2c07fff640 1 -- 172.21.15.154:0/2561405774 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f2c1401d460 con 0x7f2c2015ca70 2023-09-23T03:19:01.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.152+0000 7f2c07fff640 1 --2- 172.21.15.154:0/2561405774 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2c0c06af30 0x7f2c0c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:01.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.152+0000 7f2c07fff640 1 -- 172.21.15.154:0/2561405774 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6720+0+0 (secure 0 0 0) 0x7f2c14089b50 con 0x7f2c2015ca70 2023-09-23T03:19:01.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.152+0000 7f2c1f7fe640 1 --2- 172.21.15.154:0/2561405774 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2c0c06af30 0x7f2c0c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:01.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.153+0000 7f2c1f7fe640 1 --2- 172.21.15.154:0/2561405774 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2c0c06af30 0x7f2c0c06d3f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009900 tx=0x7f2c10008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:01.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.156+0000 7f2c07fff640 1 -- 172.21.15.154:0/2561405774 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f2c14054720 con 0x7f2c2015ca70 2023-09-23T03:19:01.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.353+0000 7f2c261b6640 1 -- 172.21.15.154:0/2561405774 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2c2015e240 con 0x7f2c2015ca70 2023-09-23T03:19:01.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.354+0000 7f2c07fff640 1 -- 172.21.15.154:0/2561405774 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2c14057d40 con 0x7f2c2015ca70 2023-09-23T03:19:01.356 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:19:01.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.357+0000 7f2c261b6640 1 -- 172.21.15.154:0/2561405774 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2c0c06af30 msgr2=0x7f2c0c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:01.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.357+0000 7f2c261b6640 1 --2- 172.21.15.154:0/2561405774 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2c0c06af30 0x7f2c0c06d3f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009900 tx=0x7f2c10008040 comp rx=0 tx=0).stop 2023-09-23T03:19:01.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.357+0000 7f2c261b6640 1 -- 172.21.15.154:0/2561405774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c2015ca70 msgr2=0x7f2c200c9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:01.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.357+0000 7f2c261b6640 1 --2- 172.21.15.154:0/2561405774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c2015ca70 0x7f2c200c9880 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f2c1400eb50 tx=0x7f2c1400c4b0 comp rx=0 tx=0).stop 2023-09-23T03:19:01.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.357+0000 7f2c261b6640 1 -- 172.21.15.154:0/2561405774 shutdown_connections 2023-09-23T03:19:01.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.357+0000 7f2c261b6640 1 --2- 172.21.15.154:0/2561405774 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2c0c06af30 0x7f2c0c06d3f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:01.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.357+0000 7f2c261b6640 1 --2- 172.21.15.154:0/2561405774 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c200c9dc0 0x7f2c200ac190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:01.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.357+0000 7f2c261b6640 1 --2- 172.21.15.154:0/2561405774 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c2015ca70 0x7f2c200c9880 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:01.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.357+0000 7f2c261b6640 1 --2- 172.21.15.154:0/2561405774 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c20156f50 0x7f2c2015c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:01.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.358+0000 7f2c261b6640 1 -- 172.21.15.154:0/2561405774 >> 172.21.15.154:0/2561405774 conn(0x7f2c20069eb0 msgr2=0x7f2c200b1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:01.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.358+0000 7f2c261b6640 1 -- 172.21.15.154:0/2561405774 shutdown_connections 2023-09-23T03:19:01.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.358+0000 7f2c261b6640 1 -- 172.21.15.154:0/2561405774 wait complete. 2023-09-23T03:19:01.376 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:19:01.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.592+0000 7f7e7fb97640 1 Processor -- start 2023-09-23T03:19:01.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.592+0000 7f7e7fb97640 1 -- start start 2023-09-23T03:19:01.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.594+0000 7f7e7fb97640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7e780ad830 0x7f7e78070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:01.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.594+0000 7f7e7fb97640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e780c9680 0x7f7e780caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:01.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.594+0000 7f7e7fb97640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e780712f0 con 0x7f7e780cb400 2023-09-23T03:19:01.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.594+0000 7f7e7fb97640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e780cb890 con 0x7f7e780ad830 2023-09-23T03:19:01.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.594+0000 7f7e7fb97640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e780715d0 con 0x7f7e780c9680 2023-09-23T03:19:01.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.594+0000 7f7e7d933640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7e780ad830 0x7f7e78070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:01.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.594+0000 7f7e7d933640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7e780ad830 0x7f7e78070c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:32914/0 (socket says 172.21.15.154:32914) 2023-09-23T03:19:01.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.594+0000 7f7e7d933640 1 -- 172.21.15.154:0/650074537 learned_addr learned my addr 172.21.15.154:0/650074537 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:01.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.595+0000 7f7e7d132640 1 --2- 172.21.15.154:0/650074537 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e780c9680 0x7f7e780caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:01.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.595+0000 7f7e7d933640 1 -- 172.21.15.154:0/650074537 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e780c9680 msgr2=0x7f7e780caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:01.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.595+0000 7f7e7d933640 1 --2- 172.21.15.154:0/650074537 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e780c9680 0x7f7e780caec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:01.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.595+0000 7f7e7d933640 1 -- 172.21.15.154:0/650074537 >> v1:172.21.15.190:6789/0 conn(0x7f7e780cb400 legacy=0x7f7e78160720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:01.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.595+0000 7f7e7d933640 1 -- 172.21.15.154:0/650074537 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e780718b0 con 0x7f7e780ad830 2023-09-23T03:19:01.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.597+0000 7f7e7d933640 1 --2- 172.21.15.154:0/650074537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e780ad830 0x7f7e78070c60 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f7e74009870 tx=0x7f7e7402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c68095c130b1ee server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:01.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.597+0000 7f7e7c931640 1 -- 172.21.15.154:0/650074537 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e74030840 con 0x7f7e780ad830 2023-09-23T03:19:01.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.597+0000 7f7e7c931640 1 -- 172.21.15.154:0/650074537 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e740309a0 con 0x7f7e780ad830 2023-09-23T03:19:01.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.597+0000 7f7e7c931640 1 -- 172.21.15.154:0/650074537 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e74030d30 con 0x7f7e780ad830 2023-09-23T03:19:01.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.598+0000 7f7e7fb97640 1 -- 172.21.15.154:0/650074537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e780ad830 msgr2=0x7f7e78070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:01.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.598+0000 7f7e7fb97640 1 --2- 172.21.15.154:0/650074537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e780ad830 0x7f7e78070c60 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f7e74009870 tx=0x7f7e7402b980 comp rx=0 tx=0).stop 2023-09-23T03:19:01.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.598+0000 7f7e7fb97640 1 -- 172.21.15.154:0/650074537 shutdown_connections 2023-09-23T03:19:01.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.598+0000 7f7e7fb97640 1 --2- 172.21.15.154:0/650074537 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e780c9680 0x7f7e780caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:01.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.598+0000 7f7e7fb97640 1 --2- 172.21.15.154:0/650074537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e780ad830 0x7f7e78070c60 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:01.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.598+0000 7f7e7fb97640 1 -- 172.21.15.154:0/650074537 >> 172.21.15.154:0/650074537 conn(0x7f7e7806c010 msgr2=0x7f7e7806c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:01.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.598+0000 7f7e7fb97640 1 -- 172.21.15.154:0/650074537 shutdown_connections 2023-09-23T03:19:01.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.598+0000 7f7e7fb97640 1 -- 172.21.15.154:0/650074537 wait complete. 2023-09-23T03:19:01.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.600+0000 7f7e7fb97640 1 Processor -- start 2023-09-23T03:19:01.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.600+0000 7f7e7fb97640 1 -- start start 2023-09-23T03:19:01.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.601+0000 7f7e7fb97640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e780ad830 0x7f7e780c41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:01.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.601+0000 7f7e7fb97640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e780c9680 0x7f7e781f5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:01.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.602+0000 7f7e7fb97640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e780cb400 0x7f7e780af360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:01.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.602+0000 7f7e7fb97640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e780c6b20 con 0x7f7e780c9680 2023-09-23T03:19:01.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.602+0000 7f7e7fb97640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e780c2dc0 con 0x7f7e780ad830 2023-09-23T03:19:01.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.602+0000 7f7e7fb97640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e781f63a0 con 0x7f7e780cb400 2023-09-23T03:19:01.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.602+0000 7f7e7d132640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e780c9680 0x7f7e781f5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:01.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.602+0000 7f7e7d132640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e780c9680 0x7f7e781f5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:32926/0 (socket says 172.21.15.154:32926) 2023-09-23T03:19:01.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.602+0000 7f7e7d132640 1 -- 172.21.15.154:0/3135571640 learned_addr learned my addr 172.21.15.154:0/3135571640 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:01.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.602+0000 7f7e7d933640 1 --2- 172.21.15.154:0/3135571640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e780ad830 0x7f7e780c41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:01.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.602+0000 7f7e7e134640 1 --2- 172.21.15.154:0/3135571640 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e780cb400 0x7f7e780af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:01.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.602+0000 7f7e7d132640 1 -- 172.21.15.154:0/3135571640 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e780cb400 msgr2=0x7f7e780af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:01.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.602+0000 7f7e7d132640 1 --2- 172.21.15.154:0/3135571640 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e780cb400 0x7f7e780af360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:01.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.602+0000 7f7e7d132640 1 -- 172.21.15.154:0/3135571640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e780ad830 msgr2=0x7f7e780c41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:01.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.602+0000 7f7e7d132640 1 --2- 172.21.15.154:0/3135571640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e780ad830 0x7f7e780c41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:01.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.602+0000 7f7e7d132640 1 -- 172.21.15.154:0/3135571640 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e740094d0 con 0x7f7e780c9680 2023-09-23T03:19:01.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.602+0000 7f7e7e134640 1 --2- 172.21.15.154:0/3135571640 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e780cb400 0x7f7e780af360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:01.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.602+0000 7f7e7d933640 1 --2- 172.21.15.154:0/3135571640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e780ad830 0x7f7e780c41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:01.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.603+0000 7f7e7d132640 1 --2- 172.21.15.154:0/3135571640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e780c9680 0x7f7e781f5e60 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800d5f0 tx=0x7f7e6800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:01.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.603+0000 7f7e6e7fc640 1 -- 172.21.15.154:0/3135571640 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e680125f0 con 0x7f7e780c9680 2023-09-23T03:19:01.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.603+0000 7f7e6e7fc640 1 -- 172.21.15.154:0/3135571640 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e68012750 con 0x7f7e780c9680 2023-09-23T03:19:01.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.603+0000 7f7e6e7fc640 1 -- 172.21.15.154:0/3135571640 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e68012b00 con 0x7f7e780c9680 2023-09-23T03:19:01.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.603+0000 7f7e7fb97640 1 -- 172.21.15.154:0/3135571640 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e780c6210 con 0x7f7e780c9680 2023-09-23T03:19:01.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.603+0000 7f7e7fb97640 1 -- 172.21.15.154:0/3135571640 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e780c6460 con 0x7f7e780c9680 2023-09-23T03:19:01.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.604+0000 7f7e7fb97640 1 -- 172.21.15.154:0/3135571640 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e7804e920 con 0x7f7e780c9680 2023-09-23T03:19:01.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.605+0000 7f7e6e7fc640 1 -- 172.21.15.154:0/3135571640 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f7e68012c60 con 0x7f7e780c9680 2023-09-23T03:19:01.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.606+0000 7f7e6e7fc640 1 --2- 172.21.15.154:0/3135571640 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7e5c06af30 0x7f7e5c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:01.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.607+0000 7f7e6e7fc640 1 -- 172.21.15.154:0/3135571640 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6720+0+0 (secure 0 0 0) 0x7f7e68089f80 con 0x7f7e780c9680 2023-09-23T03:19:01.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.607+0000 7f7e7d933640 1 --2- 172.21.15.154:0/3135571640 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7e5c06af30 0x7f7e5c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:01.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.607+0000 7f7e7d933640 1 --2- 172.21.15.154:0/3135571640 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7e5c06af30 0x7f7e5c06d3f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7e74036000 tx=0x7f7e7403b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:01.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.611+0000 7f7e6e7fc640 1 -- 172.21.15.154:0/3135571640 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f7e680bca50 con 0x7f7e780c9680 2023-09-23T03:19:01.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.804+0000 7f7e7fb97640 1 -- 172.21.15.154:0/3135571640 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7e7806afc0 con 0x7f7e780c9680 2023-09-23T03:19:01.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.806+0000 7f7e6e7fc640 1 -- 172.21.15.154:0/3135571640 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v49) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7e68054c60 con 0x7f7e780c9680 2023-09-23T03:19:01.807 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:19:01.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.809+0000 7f7e7fb97640 1 -- 172.21.15.154:0/3135571640 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7e5c06af30 msgr2=0x7f7e5c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:01.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.809+0000 7f7e7fb97640 1 --2- 172.21.15.154:0/3135571640 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7e5c06af30 0x7f7e5c06d3f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7e74036000 tx=0x7f7e7403b040 comp rx=0 tx=0).stop 2023-09-23T03:19:01.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.809+0000 7f7e7fb97640 1 -- 172.21.15.154:0/3135571640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e780c9680 msgr2=0x7f7e781f5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:01.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.809+0000 7f7e7fb97640 1 --2- 172.21.15.154:0/3135571640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e780c9680 0x7f7e781f5e60 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800d5f0 tx=0x7f7e6800dab0 comp rx=0 tx=0).stop 2023-09-23T03:19:01.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.809+0000 7f7e7fb97640 1 -- 172.21.15.154:0/3135571640 shutdown_connections 2023-09-23T03:19:01.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.809+0000 7f7e7fb97640 1 --2- 172.21.15.154:0/3135571640 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7e5c06af30 0x7f7e5c06d3f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:01.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.809+0000 7f7e7fb97640 1 --2- 172.21.15.154:0/3135571640 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e780cb400 0x7f7e780af360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:01.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.809+0000 7f7e7fb97640 1 --2- 172.21.15.154:0/3135571640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e780c9680 0x7f7e781f5e60 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:01.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.809+0000 7f7e7fb97640 1 --2- 172.21.15.154:0/3135571640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e780ad830 0x7f7e780c41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:01.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.809+0000 7f7e7fb97640 1 -- 172.21.15.154:0/3135571640 >> 172.21.15.154:0/3135571640 conn(0x7f7e7806c010 msgr2=0x7f7e780c4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:01.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.809+0000 7f7e7fb97640 1 -- 172.21.15.154:0/3135571640 shutdown_connections 2023-09-23T03:19:01.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:01.809+0000 7f7e7fb97640 1 -- 172.21.15.154:0/3135571640 wait complete. 2023-09-23T03:19:01.829 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:19:02.038 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.037+0000 7f8ab32ea640 1 Processor -- start 2023-09-23T03:19:02.038 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.037+0000 7f8ab32ea640 1 -- start start 2023-09-23T03:19:02.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.039+0000 7f8ab32ea640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8aac0705a0 0x7f8aac1654f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:02.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.039+0000 7f8ab32ea640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8aac0cb6c0 con 0x7f8aac15ba50 2023-09-23T03:19:02.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.039+0000 7f8ab32ea640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8aac15af70 con 0x7f8aac158480 2023-09-23T03:19:02.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.039+0000 7f8ab32ea640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac15ad80 con 0x7f8aac0705a0 2023-09-23T03:19:02.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.039+0000 7f8ab1ae7640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f8aac15ba50 0x7f8aac06fe90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:33290/0 (socket says 172.21.15.154:33290) 2023-09-23T03:19:02.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.039+0000 7f8ab1ae7640 1 -- 172.21.15.154:0/470770999 learned_addr learned my addr 172.21.15.154:0/470770999 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:19:02.042 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.039+0000 7f8ab2ae9640 1 --2- 172.21.15.154:0/470770999 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8aac0705a0 0x7f8aac1654f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:02.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.039+0000 7f8ab12e6640 1 -- 172.21.15.154:0/470770999 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3824773943 0 0) 0x7f8aac0cb6c0 con 0x7f8aac15ba50 2023-09-23T03:19:02.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.040+0000 7f8ab12e6640 1 -- 172.21.15.154:0/470770999 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a98003520 con 0x7f8aac15ba50 2023-09-23T03:19:02.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.040+0000 7f8ab12e6640 1 -- 172.21.15.154:0/470770999 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 858794215 0 0) 0x7f8a98003520 con 0x7f8aac15ba50 2023-09-23T03:19:02.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.040+0000 7f8ab12e6640 1 -- 172.21.15.154:0/470770999 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8aac0705a0 msgr2=0x7f8aac1654f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:02.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.040+0000 7f8ab12e6640 1 --2- 172.21.15.154:0/470770999 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8aac0705a0 0x7f8aac1654f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:02.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.040+0000 7f8ab12e6640 1 -- 172.21.15.154:0/470770999 >> v1:172.21.15.190:6789/0 conn(0x7f8aac158480 legacy=0x7f8aac0cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:02.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.040+0000 7f8ab12e6640 1 -- 172.21.15.154:0/470770999 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aac15b250 con 0x7f8aac15ba50 2023-09-23T03:19:02.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.040+0000 7f8ab12e6640 1 -- 172.21.15.154:0/470770999 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f8a9c001670 con 0x7f8aac15ba50 2023-09-23T03:19:02.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.040+0000 7f8ab12e6640 1 -- 172.21.15.154:0/470770999 >> v1:172.21.15.154:6789/0 conn(0x7f8aac15ba50 legacy=0x7f8aac06fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:02.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.040+0000 7f8ab12e6640 1 --2- 172.21.15.154:0/470770999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a980039a0 0x7f8a98003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:02.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.040+0000 7f8ab12e6640 1 --2- 172.21.15.154:0/470770999 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8aac0705a0 0x7f8a980047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:02.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.040+0000 7f8ab12e6640 1 --2- 172.21.15.154:0/470770999 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a98004d00 0x7f8a98005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:02.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.040+0000 7f8ab12e6640 1 -- 172.21.15.154:0/470770999 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9800fe40 con 0x7f8a98004d00 2023-09-23T03:19:02.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.040+0000 7f8ab12e6640 1 -- 172.21.15.154:0/470770999 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9800ffb0 con 0x7f8a980039a0 2023-09-23T03:19:02.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.040+0000 7f8ab12e6640 1 -- 172.21.15.154:0/470770999 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a98010170 con 0x7f8aac0705a0 2023-09-23T03:19:02.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.040+0000 7f8ab2ae9640 1 --2- 172.21.15.154:0/470770999 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a98004d00 0x7f8a98005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:02.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.041+0000 7f8ab22e8640 1 --2- 172.21.15.154:0/470770999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a980039a0 0x7f8a98003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:02.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.041+0000 7f8ab1ae7640 1 --2- 172.21.15.154:0/470770999 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8aac0705a0 0x7f8a980047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:02.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.041+0000 7f8ab2ae9640 1 -- 172.21.15.154:0/470770999 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8aac0705a0 msgr2=0x7f8a980047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:02.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.041+0000 7f8ab2ae9640 1 --2- 172.21.15.154:0/470770999 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8aac0705a0 0x7f8a980047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:02.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.041+0000 7f8ab2ae9640 1 -- 172.21.15.154:0/470770999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a980039a0 msgr2=0x7f8a98003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:02.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.041+0000 7f8ab2ae9640 1 --2- 172.21.15.154:0/470770999 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a980039a0 0x7f8a98003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:02.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.041+0000 7f8ab2ae9640 1 -- 172.21.15.154:0/470770999 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a98003520 con 0x7f8a98004d00 2023-09-23T03:19:02.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.041+0000 7f8ab1ae7640 1 --2- 172.21.15.154:0/470770999 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8aac0705a0 0x7f8a980047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:02.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.042+0000 7f8ab2ae9640 1 --2- 172.21.15.154:0/470770999 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a98004d00 0x7f8a98005180 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f8aac0c89d0 tx=0x7f8a980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88640e312031849c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:02.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.042+0000 7f8ab12e6640 1 -- 172.21.15.154:0/470770999 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa402cb90 con 0x7f8a98004d00 2023-09-23T03:19:02.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.042+0000 7f8ab12e6640 1 -- 172.21.15.154:0/470770999 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa400f040 con 0x7f8a98004d00 2023-09-23T03:19:02.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.042+0000 7f8ab12e6640 1 -- 172.21.15.154:0/470770999 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4034440 con 0x7f8a98004d00 2023-09-23T03:19:02.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.043+0000 7f8ab32ea640 1 -- 172.21.15.154:0/470770999 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a98004d00 msgr2=0x7f8a98005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:02.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.043+0000 7f8ab32ea640 1 --2- 172.21.15.154:0/470770999 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8a98004d00 0x7f8a98005180 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f8aac0c89d0 tx=0x7f8a980057f0 comp rx=0 tx=0).stop 2023-09-23T03:19:02.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.043+0000 7f8ab2ae9640 1 -- 172.21.15.154:0/470770999 reap_dead start 2023-09-23T03:19:02.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.043+0000 7f8ab32ea640 1 -- 172.21.15.154:0/470770999 shutdown_connections 2023-09-23T03:19:02.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.043+0000 7f8ab32ea640 1 -- 172.21.15.154:0/470770999 >> 172.21.15.154:0/470770999 conn(0x7f8aac0aff20 msgr2=0x7f8aac0b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:02.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.043+0000 7f8ab32ea640 1 -- 172.21.15.154:0/470770999 shutdown_connections 2023-09-23T03:19:02.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.043+0000 7f8ab32ea640 1 -- 172.21.15.154:0/470770999 wait complete. 2023-09-23T03:19:02.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.045+0000 7f8ab32ea640 1 Processor -- start 2023-09-23T03:19:02.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.045+0000 7f8ab32ea640 1 -- start start 2023-09-23T03:19:02.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.046+0000 7f8ab32ea640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a98004d00 0x7f8aac0c43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:02.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.046+0000 7f8ab32ea640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8aac150140 0x7f8aac0c1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:02.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.046+0000 7f8ab32ea640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8aac1511e0 0x7f8aac151670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:02.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.046+0000 7f8ab32ea640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac151bb0 con 0x7f8aac1511e0 2023-09-23T03:19:02.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.046+0000 7f8ab32ea640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac0c4910 con 0x7f8a98004d00 2023-09-23T03:19:02.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.046+0000 7f8ab32ea640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac0c2810 con 0x7f8aac150140 2023-09-23T03:19:02.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.046+0000 7f8ab2ae9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8aac1511e0 0x7f8aac151670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:02.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.046+0000 7f8ab2ae9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8aac1511e0 0x7f8aac151670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:32942/0 (socket says 172.21.15.154:32942) 2023-09-23T03:19:02.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.046+0000 7f8ab2ae9640 1 -- 172.21.15.154:0/2712969159 learned_addr learned my addr 172.21.15.154:0/2712969159 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:02.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.047+0000 7f8ab2ae9640 1 -- 172.21.15.154:0/2712969159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8aac150140 msgr2=0x7f8aac0c1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:02.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.047+0000 7f8ab2ae9640 1 --2- 172.21.15.154:0/2712969159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8aac150140 0x7f8aac0c1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:02.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.047+0000 7f8ab22e8640 1 --2- 172.21.15.154:0/2712969159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a98004d00 0x7f8aac0c43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:02.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.047+0000 7f8ab2ae9640 1 -- 172.21.15.154:0/2712969159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a98004d00 msgr2=0x7f8aac0c43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:02.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.047+0000 7f8ab2ae9640 1 --2- 172.21.15.154:0/2712969159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a98004d00 0x7f8aac0c43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:02.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.047+0000 7f8ab2ae9640 1 -- 172.21.15.154:0/2712969159 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aac0c2af0 con 0x7f8aac1511e0 2023-09-23T03:19:02.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.047+0000 7f8ab2ae9640 1 --2- 172.21.15.154:0/2712969159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8aac1511e0 0x7f8aac151670 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4037040 tx=0x7f8aa4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:02.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.047+0000 7f8aa2ffd640 1 -- 172.21.15.154:0/2712969159 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4043070 con 0x7f8aac1511e0 2023-09-23T03:19:02.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.047+0000 7f8aa2ffd640 1 -- 172.21.15.154:0/2712969159 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa4003990 con 0x7f8aac1511e0 2023-09-23T03:19:02.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.047+0000 7f8ab32ea640 1 -- 172.21.15.154:0/2712969159 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aac0c2dd0 con 0x7f8aac1511e0 2023-09-23T03:19:02.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.047+0000 7f8aa2ffd640 1 -- 172.21.15.154:0/2712969159 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4003d40 con 0x7f8aac1511e0 2023-09-23T03:19:02.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.048+0000 7f8ab32ea640 1 -- 172.21.15.154:0/2712969159 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aac0b1e70 con 0x7f8aac1511e0 2023-09-23T03:19:02.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.049+0000 7f8ab32ea640 1 -- 172.21.15.154:0/2712969159 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aac04e920 con 0x7f8aac1511e0 2023-09-23T03:19:02.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.050+0000 7f8aa2ffd640 1 -- 172.21.15.154:0/2712969159 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f8aa4034900 con 0x7f8aac1511e0 2023-09-23T03:19:02.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.051+0000 7f8aa2ffd640 1 --2- 172.21.15.154:0/2712969159 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8a9006af30 0x7f8a9006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:02.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.051+0000 7f8aa2ffd640 1 -- 172.21.15.154:0/2712969159 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6720+0+0 (secure 0 0 0) 0x7f8aa40ab470 con 0x7f8aac1511e0 2023-09-23T03:19:02.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.051+0000 7f8ab22e8640 1 --2- 172.21.15.154:0/2712969159 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8a9006af30 0x7f8a9006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:02.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.052+0000 7f8ab22e8640 1 --2- 172.21.15.154:0/2712969159 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8a9006af30 0x7f8a9006d3f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8007fd0 tx=0x7f8aa8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:02.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.055+0000 7f8aa2ffd640 1 -- 172.21.15.154:0/2712969159 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f8aa4076170 con 0x7f8aac1511e0 2023-09-23T03:19:02.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.246+0000 7f8ab32ea640 1 -- 172.21.15.154:0/2712969159 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8aac0ca820 con 0x7f8aac1511e0 2023-09-23T03:19:02.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.247+0000 7f8aa2ffd640 1 -- 172.21.15.154:0/2712969159 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v49) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8aa4079790 con 0x7f8aac1511e0 2023-09-23T03:19:02.248 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:19:02.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.249+0000 7f8ab32ea640 1 -- 172.21.15.154:0/2712969159 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8a9006af30 msgr2=0x7f8a9006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:02.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.250+0000 7f8ab32ea640 1 --2- 172.21.15.154:0/2712969159 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8a9006af30 0x7f8a9006d3f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8007fd0 tx=0x7f8aa8009040 comp rx=0 tx=0).stop 2023-09-23T03:19:02.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.250+0000 7f8ab32ea640 1 -- 172.21.15.154:0/2712969159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8aac1511e0 msgr2=0x7f8aac151670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:02.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.250+0000 7f8ab32ea640 1 --2- 172.21.15.154:0/2712969159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8aac1511e0 0x7f8aac151670 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4037040 tx=0x7f8aa4009750 comp rx=0 tx=0).stop 2023-09-23T03:19:02.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.250+0000 7f8ab32ea640 1 -- 172.21.15.154:0/2712969159 shutdown_connections 2023-09-23T03:19:02.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.250+0000 7f8ab32ea640 1 --2- 172.21.15.154:0/2712969159 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8a9006af30 0x7f8a9006d3f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:02.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.250+0000 7f8ab32ea640 1 --2- 172.21.15.154:0/2712969159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8aac1511e0 0x7f8aac151670 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:02.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.250+0000 7f8ab32ea640 1 --2- 172.21.15.154:0/2712969159 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8aac150140 0x7f8aac0c1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:02.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.250+0000 7f8ab32ea640 1 --2- 172.21.15.154:0/2712969159 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8a98004d00 0x7f8aac0c43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:02.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.250+0000 7f8ab32ea640 1 -- 172.21.15.154:0/2712969159 >> 172.21.15.154:0/2712969159 conn(0x7f8aac0aff20 msgr2=0x7f8aac0cdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:02.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.250+0000 7f8ab32ea640 1 -- 172.21.15.154:0/2712969159 shutdown_connections 2023-09-23T03:19:02.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.250+0000 7f8ab32ea640 1 -- 172.21.15.154:0/2712969159 wait complete. 2023-09-23T03:19:02.269 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-09-23T03:19:02.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.488+0000 7f7de280d640 1 Processor -- start 2023-09-23T03:19:02.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.488+0000 7f7de280d640 1 -- start start 2023-09-23T03:19:02.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.489+0000 7f7de280d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7ddc15ba50 0x7f7ddc06fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:02.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.490+0000 7f7de280d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ddc0cb6c0 con 0x7f7ddc158480 2023-09-23T03:19:02.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.490+0000 7f7de280d640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ddc15af70 con 0x7f7ddc15eef0 2023-09-23T03:19:02.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.490+0000 7f7de280d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ddc15ad80 con 0x7f7ddc15ba50 2023-09-23T03:19:02.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.490+0000 7f7de100a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7ddc15ba50 0x7f7ddc06fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:02.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.490+0000 7f7de100a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7ddc15ba50 0x7f7ddc06fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:32946/0 (socket says 172.21.15.154:32946) 2023-09-23T03:19:02.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.490+0000 7f7de180b640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f7ddc158480 0x7f7ddc0cade0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:33294/0 (socket says 172.21.15.154:33294) 2023-09-23T03:19:02.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.490+0000 7f7de100a640 1 -- 172.21.15.154:0/1431160522 learned_addr learned my addr 172.21.15.154:0/1431160522 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:02.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.490+0000 7f7de100a640 1 -- 172.21.15.154:0/1431160522 >> v1:172.21.15.190:6789/0 conn(0x7f7ddc15eef0 legacy=0x7f7ddc0cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:02.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.490+0000 7f7de100a640 1 -- 172.21.15.154:0/1431160522 >> v1:172.21.15.154:6789/0 conn(0x7f7ddc158480 legacy=0x7f7ddc0cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:02.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.490+0000 7f7de100a640 1 -- 172.21.15.154:0/1431160522 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ddc15b250 con 0x7f7ddc15ba50 2023-09-23T03:19:02.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.491+0000 7f7de100a640 1 --2- 172.21.15.154:0/1431160522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ddc15ba50 0x7f7ddc06fe90 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc009870 tx=0x7f7dcc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c3596aad3cc1fd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:02.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.492+0000 7f7de0809640 1 -- 172.21.15.154:0/1431160522 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc030860 con 0x7f7ddc15ba50 2023-09-23T03:19:02.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.492+0000 7f7de0809640 1 -- 172.21.15.154:0/1431160522 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dcc0309c0 con 0x7f7ddc15ba50 2023-09-23T03:19:02.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.492+0000 7f7de0809640 1 -- 172.21.15.154:0/1431160522 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc030d50 con 0x7f7ddc15ba50 2023-09-23T03:19:02.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.492+0000 7f7de280d640 1 -- 172.21.15.154:0/1431160522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ddc15ba50 msgr2=0x7f7ddc06fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:02.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.492+0000 7f7de280d640 1 --2- 172.21.15.154:0/1431160522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ddc15ba50 0x7f7ddc06fe90 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc009870 tx=0x7f7dcc02b9a0 comp rx=0 tx=0).stop 2023-09-23T03:19:02.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.493+0000 7f7de280d640 1 -- 172.21.15.154:0/1431160522 shutdown_connections 2023-09-23T03:19:02.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.493+0000 7f7de280d640 1 --2- 172.21.15.154:0/1431160522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ddc15ba50 0x7f7ddc06fe90 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:02.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.493+0000 7f7de280d640 1 -- 172.21.15.154:0/1431160522 >> 172.21.15.154:0/1431160522 conn(0x7f7ddc0aff20 msgr2=0x7f7ddc0b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:02.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.493+0000 7f7de280d640 1 -- 172.21.15.154:0/1431160522 shutdown_connections 2023-09-23T03:19:02.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.493+0000 7f7de280d640 1 -- 172.21.15.154:0/1431160522 wait complete. 2023-09-23T03:19:02.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.494+0000 7f7de280d640 1 Processor -- start 2023-09-23T03:19:02.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.495+0000 7f7de280d640 1 -- start start 2023-09-23T03:19:02.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.495+0000 7f7de280d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ddc158480 0x7f7ddc159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:02.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.495+0000 7f7de280d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ddc15ba50 0x7f7ddc0ae290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:02.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.495+0000 7f7de280d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ddc15eef0 0x7f7ddc15d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:02.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.495+0000 7f7de280d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc0ade90 con 0x7f7ddc15ba50 2023-09-23T03:19:02.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.495+0000 7f7de280d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc15a510 con 0x7f7ddc15eef0 2023-09-23T03:19:02.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.495+0000 7f7de280d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc0ae7d0 con 0x7f7ddc158480 2023-09-23T03:19:02.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.496+0000 7f7de100a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ddc15ba50 0x7f7ddc0ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:02.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.496+0000 7f7de100a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ddc15ba50 0x7f7ddc0ae290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:32962/0 (socket says 172.21.15.154:32962) 2023-09-23T03:19:02.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.496+0000 7f7de100a640 1 -- 172.21.15.154:0/3924335875 learned_addr learned my addr 172.21.15.154:0/3924335875 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:02.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.496+0000 7f7de200c640 1 --2- 172.21.15.154:0/3924335875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ddc15eef0 0x7f7ddc15d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:02.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.496+0000 7f7de100a640 1 -- 172.21.15.154:0/3924335875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ddc158480 msgr2=0x7f7ddc159fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:02.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.496+0000 7f7de180b640 1 --2- 172.21.15.154:0/3924335875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ddc158480 0x7f7ddc159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:02.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.496+0000 7f7de100a640 1 --2- 172.21.15.154:0/3924335875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ddc158480 0x7f7ddc159fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:02.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.496+0000 7f7de100a640 1 -- 172.21.15.154:0/3924335875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ddc15eef0 msgr2=0x7f7ddc15d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:02.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.496+0000 7f7de100a640 1 --2- 172.21.15.154:0/3924335875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ddc15eef0 0x7f7ddc15d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:02.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.496+0000 7f7de100a640 1 -- 172.21.15.154:0/3924335875 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dcc0094d0 con 0x7f7ddc15ba50 2023-09-23T03:19:02.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.496+0000 7f7de200c640 1 --2- 172.21.15.154:0/3924335875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ddc15eef0 0x7f7ddc15d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:02.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.497+0000 7f7de100a640 1 --2- 172.21.15.154:0/3924335875 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ddc15ba50 0x7f7ddc0ae290 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc009870 tx=0x7f7dcc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:02.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.497+0000 7f7dd27fc640 1 -- 172.21.15.154:0/3924335875 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc030810 con 0x7f7ddc15ba50 2023-09-23T03:19:02.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.497+0000 7f7dd27fc640 1 -- 172.21.15.154:0/3924335875 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dcc043400 con 0x7f7ddc15ba50 2023-09-23T03:19:02.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.497+0000 7f7dd27fc640 1 -- 172.21.15.154:0/3924335875 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc0437b0 con 0x7f7ddc15ba50 2023-09-23T03:19:02.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.497+0000 7f7de280d640 1 -- 172.21.15.154:0/3924335875 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ddc0ad860 con 0x7f7ddc15ba50 2023-09-23T03:19:02.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.497+0000 7f7de280d640 1 -- 172.21.15.154:0/3924335875 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ddc15aa20 con 0x7f7ddc15ba50 2023-09-23T03:19:02.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.498+0000 7f7de280d640 1 -- 172.21.15.154:0/3924335875 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ddc04e920 con 0x7f7ddc15ba50 2023-09-23T03:19:02.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.500+0000 7f7dd27fc640 1 -- 172.21.15.154:0/3924335875 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f7dcc043910 con 0x7f7ddc15ba50 2023-09-23T03:19:02.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.501+0000 7f7dd27fc640 1 --2- 172.21.15.154:0/3924335875 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7dc006af30 0x7f7dc006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:02.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.501+0000 7f7dd27fc640 1 -- 172.21.15.154:0/3924335875 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6720+0+0 (secure 0 0 0) 0x7f7dcc03e070 con 0x7f7ddc15ba50 2023-09-23T03:19:02.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.501+0000 7f7de180b640 1 --2- 172.21.15.154:0/3924335875 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7dc006af30 0x7f7dc006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:02.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.502+0000 7f7de180b640 1 --2- 172.21.15.154:0/3924335875 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7dc006af30 0x7f7dc006d3f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8007fd0 tx=0x7f7dd8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:02.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.506+0000 7f7dd27fc640 1 -- 172.21.15.154:0/3924335875 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f7dcc077c20 con 0x7f7ddc15ba50 2023-09-23T03:19:02.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.697+0000 7f7de280d640 1 -- 172.21.15.154:0/3924335875 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7f7ddc0b08e0 con 0x7f7ddc15ba50 2023-09-23T03:19:02.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.698+0000 7f7dd27fc640 1 -- 172.21.15.154:0/3924335875 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v49) v1 ==== 106+0+11 (secure 0 0 0) 0x7f7dcc0340b0 con 0x7f7ddc15ba50 2023-09-23T03:19:02.700 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:19:02.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.701+0000 7f7de280d640 1 -- 172.21.15.154:0/3924335875 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7dc006af30 msgr2=0x7f7dc006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:02.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.701+0000 7f7de280d640 1 --2- 172.21.15.154:0/3924335875 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7dc006af30 0x7f7dc006d3f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8007fd0 tx=0x7f7dd8009040 comp rx=0 tx=0).stop 2023-09-23T03:19:02.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.701+0000 7f7de280d640 1 -- 172.21.15.154:0/3924335875 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ddc15ba50 msgr2=0x7f7ddc0ae290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:02.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.701+0000 7f7de280d640 1 --2- 172.21.15.154:0/3924335875 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ddc15ba50 0x7f7ddc0ae290 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc009870 tx=0x7f7dcc004730 comp rx=0 tx=0).stop 2023-09-23T03:19:02.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.702+0000 7f7de280d640 1 -- 172.21.15.154:0/3924335875 shutdown_connections 2023-09-23T03:19:02.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.702+0000 7f7de280d640 1 --2- 172.21.15.154:0/3924335875 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7dc006af30 0x7f7dc006d3f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:02.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.702+0000 7f7de280d640 1 --2- 172.21.15.154:0/3924335875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ddc15eef0 0x7f7ddc15d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:02.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.702+0000 7f7de280d640 1 --2- 172.21.15.154:0/3924335875 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ddc15ba50 0x7f7ddc0ae290 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:02.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.702+0000 7f7de280d640 1 --2- 172.21.15.154:0/3924335875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ddc158480 0x7f7ddc159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:02.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.702+0000 7f7de280d640 1 -- 172.21.15.154:0/3924335875 >> 172.21.15.154:0/3924335875 conn(0x7f7ddc0aff20 msgr2=0x7f7ddc1569b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:02.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.702+0000 7f7de280d640 1 -- 172.21.15.154:0/3924335875 shutdown_connections 2023-09-23T03:19:02.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.702+0000 7f7de280d640 1 -- 172.21.15.154:0/3924335875 wait complete. 2023-09-23T03:19:02.722 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-23T03:19:02.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.933+0000 7faa3759e640 1 Processor -- start 2023-09-23T03:19:02.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.933+0000 7faa3759e640 1 -- start start 2023-09-23T03:19:02.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.935+0000 7faa3759e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa3815eb00 0x7faa3806fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:02.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.935+0000 7faa3759e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa38070750 con 0x7faa3806c7a0 2023-09-23T03:19:02.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.935+0000 7faa3759e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa3806bff0 con 0x7faa38070300 2023-09-23T03:19:02.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.935+0000 7faa3759e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa38155070 con 0x7faa3815eb00 2023-09-23T03:19:02.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.935+0000 7faa3659c640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7faa3806c7a0 0x7faa3815ab00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:33306/0 (socket says 172.21.15.154:33306) 2023-09-23T03:19:02.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.935+0000 7faa3659c640 1 -- 172.21.15.154:0/4198526365 learned_addr learned my addr 172.21.15.154:0/4198526365 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:19:02.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.935+0000 7faa2dd9b640 1 --2- 172.21.15.154:0/4198526365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa3815eb00 0x7faa3806fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:02.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.935+0000 7faa35d9b640 1 -- 172.21.15.154:0/4198526365 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1142370877 0 0) 0x7faa28001110 con 0x7faa3806c7a0 2023-09-23T03:19:02.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.935+0000 7faa35d9b640 1 -- 172.21.15.154:0/4198526365 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa24003520 con 0x7faa3806c7a0 2023-09-23T03:19:02.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.936+0000 7faa35d9b640 1 -- 172.21.15.154:0/4198526365 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3487812717 0 0) 0x7faa24003520 con 0x7faa3806c7a0 2023-09-23T03:19:02.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.936+0000 7faa35d9b640 1 -- 172.21.15.154:0/4198526365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa3815eb00 msgr2=0x7faa3806fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:02.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.936+0000 7faa35d9b640 1 --2- 172.21.15.154:0/4198526365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa3815eb00 0x7faa3806fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:02.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.936+0000 7faa35d9b640 1 -- 172.21.15.154:0/4198526365 >> v1:172.21.15.190:6789/0 conn(0x7faa38070300 legacy=0x7faa380ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:02.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.936+0000 7faa35d9b640 1 -- 172.21.15.154:0/4198526365 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa3806cb80 con 0x7faa3806c7a0 2023-09-23T03:19:02.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.936+0000 7faa35d9b640 1 -- 172.21.15.154:0/4198526365 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7faa28001800 con 0x7faa3806c7a0 2023-09-23T03:19:02.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.936+0000 7faa35d9b640 1 -- 172.21.15.154:0/4198526365 >> v1:172.21.15.154:6789/0 conn(0x7faa3806c7a0 legacy=0x7faa3815ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:02.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.936+0000 7faa35d9b640 1 --2- 172.21.15.154:0/4198526365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa240039a0 0x7faa24003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:02.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.936+0000 7faa35d9b640 1 --2- 172.21.15.154:0/4198526365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa3815eb00 0x7faa240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:02.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.936+0000 7faa35d9b640 1 --2- 172.21.15.154:0/4198526365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa24004d00 0x7faa24005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:02.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.936+0000 7faa35d9b640 1 -- 172.21.15.154:0/4198526365 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2400bb60 con 0x7faa24004d00 2023-09-23T03:19:02.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.936+0000 7faa35d9b640 1 -- 172.21.15.154:0/4198526365 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2400bcd0 con 0x7faa240039a0 2023-09-23T03:19:02.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.936+0000 7faa35d9b640 1 -- 172.21.15.154:0/4198526365 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa2400be90 con 0x7faa3815eb00 2023-09-23T03:19:02.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.936+0000 7faa36d9d640 1 --2- 172.21.15.154:0/4198526365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa24004d00 0x7faa24005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:02.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.936+0000 7faa3659c640 1 --2- 172.21.15.154:0/4198526365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa240039a0 0x7faa24003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:02.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.937+0000 7faa2dd9b640 1 --2- 172.21.15.154:0/4198526365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa3815eb00 0x7faa240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:02.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.937+0000 7faa36d9d640 1 -- 172.21.15.154:0/4198526365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa3815eb00 msgr2=0x7faa240047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:02.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.937+0000 7faa36d9d640 1 --2- 172.21.15.154:0/4198526365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa3815eb00 0x7faa240047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:02.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.937+0000 7faa36d9d640 1 -- 172.21.15.154:0/4198526365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa240039a0 msgr2=0x7faa24003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:02.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.937+0000 7faa36d9d640 1 --2- 172.21.15.154:0/4198526365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa240039a0 0x7faa24003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:02.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.937+0000 7faa36d9d640 1 -- 172.21.15.154:0/4198526365 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa24003520 con 0x7faa24004d00 2023-09-23T03:19:02.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.937+0000 7faa3659c640 1 --2- 172.21.15.154:0/4198526365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa240039a0 0x7faa24003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:19:02.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.938+0000 7faa36d9d640 1 --2- 172.21.15.154:0/4198526365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa24004d00 0x7faa24005180 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7faa30001750 tx=0x7faa240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d42fc4ffc7ba7db server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:02.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.938+0000 7faa35d9b640 1 -- 172.21.15.154:0/4198526365 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa300052d0 con 0x7faa24004d00 2023-09-23T03:19:02.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.938+0000 7faa35d9b640 1 -- 172.21.15.154:0/4198526365 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa3002e050 con 0x7faa24004d00 2023-09-23T03:19:02.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.938+0000 7faa35d9b640 1 -- 172.21.15.154:0/4198526365 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa300035f0 con 0x7faa24004d00 2023-09-23T03:19:02.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.939+0000 7faa3759e640 1 -- 172.21.15.154:0/4198526365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa24004d00 msgr2=0x7faa24005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:02.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.939+0000 7faa3759e640 1 --2- 172.21.15.154:0/4198526365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa24004d00 0x7faa24005180 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7faa30001750 tx=0x7faa240057f0 comp rx=0 tx=0).stop 2023-09-23T03:19:02.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.939+0000 7faa36d9d640 1 -- 172.21.15.154:0/4198526365 reap_dead start 2023-09-23T03:19:02.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.939+0000 7faa3759e640 1 -- 172.21.15.154:0/4198526365 shutdown_connections 2023-09-23T03:19:02.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.939+0000 7faa3759e640 1 -- 172.21.15.154:0/4198526365 >> 172.21.15.154:0/4198526365 conn(0x7faa3806a390 msgr2=0x7faa380afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:02.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.939+0000 7faa3759e640 1 -- 172.21.15.154:0/4198526365 shutdown_connections 2023-09-23T03:19:02.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.939+0000 7faa3759e640 1 -- 172.21.15.154:0/4198526365 wait complete. 2023-09-23T03:19:02.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.941+0000 7faa3759e640 1 Processor -- start 2023-09-23T03:19:02.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.941+0000 7faa3759e640 1 -- start start 2023-09-23T03:19:02.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.942+0000 7faa3759e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa24004d00 0x7faa38152730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:02.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.942+0000 7faa3759e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa38159480 0x7faa38159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:02.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.942+0000 7faa3759e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa380c2510 0x7faa380c2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:02.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.942+0000 7faa3759e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa380ccb00 con 0x7faa380c2510 2023-09-23T03:19:02.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.942+0000 7faa3759e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa380c4550 con 0x7faa24004d00 2023-09-23T03:19:02.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.942+0000 7faa3759e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa38159e40 con 0x7faa38159480 2023-09-23T03:19:02.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.942+0000 7faa36d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa380c2510 0x7faa380c2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:02.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.942+0000 7faa36d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa380c2510 0x7faa380c2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:32982/0 (socket says 172.21.15.154:32982) 2023-09-23T03:19:02.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.942+0000 7faa36d9d640 1 -- 172.21.15.154:0/1080937432 learned_addr learned my addr 172.21.15.154:0/1080937432 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:02.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.942+0000 7faa3659c640 1 --2- 172.21.15.154:0/1080937432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa24004d00 0x7faa38152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:02.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.943+0000 7faa36d9d640 1 -- 172.21.15.154:0/1080937432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa38159480 msgr2=0x7faa38159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:02.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.943+0000 7faa36d9d640 1 --2- 172.21.15.154:0/1080937432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa38159480 0x7faa38159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:02.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.943+0000 7faa36d9d640 1 -- 172.21.15.154:0/1080937432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa24004d00 msgr2=0x7faa38152730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:02.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.943+0000 7faa36d9d640 1 --2- 172.21.15.154:0/1080937432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa24004d00 0x7faa38152730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:02.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.943+0000 7faa36d9d640 1 -- 172.21.15.154:0/1080937432 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa380c2ed0 con 0x7faa380c2510 2023-09-23T03:19:02.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.943+0000 7faa36d9d640 1 --2- 172.21.15.154:0/1080937432 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa380c2510 0x7faa380c2990 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7faa30005040 tx=0x7faa30001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:02.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.943+0000 7faa2f7fe640 1 -- 172.21.15.154:0/1080937432 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa30043070 con 0x7faa380c2510 2023-09-23T03:19:02.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.943+0000 7faa2f7fe640 1 -- 172.21.15.154:0/1080937432 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa30007a30 con 0x7faa380c2510 2023-09-23T03:19:02.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.943+0000 7faa2f7fe640 1 -- 172.21.15.154:0/1080937432 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa300067d0 con 0x7faa380c2510 2023-09-23T03:19:02.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.943+0000 7faa3759e640 1 -- 172.21.15.154:0/1080937432 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa380c3190 con 0x7faa380c2510 2023-09-23T03:19:02.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.943+0000 7faa3759e640 1 -- 172.21.15.154:0/1080937432 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa380cc720 con 0x7faa380c2510 2023-09-23T03:19:02.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.945+0000 7faa3759e640 1 -- 172.21.15.154:0/1080937432 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa380736b0 con 0x7faa380c2510 2023-09-23T03:19:02.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.945+0000 7faa2f7fe640 1 -- 172.21.15.154:0/1080937432 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7faa30006930 con 0x7faa380c2510 2023-09-23T03:19:02.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.946+0000 7faa2f7fe640 1 --2- 172.21.15.154:0/1080937432 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7faa1806af30 0x7faa1806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:02.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.946+0000 7faa2f7fe640 1 -- 172.21.15.154:0/1080937432 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6720+0+0 (secure 0 0 0) 0x7faa300aaaf0 con 0x7faa380c2510 2023-09-23T03:19:02.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.947+0000 7faa3659c640 1 --2- 172.21.15.154:0/1080937432 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7faa1806af30 0x7faa1806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:02.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.947+0000 7faa3659c640 1 --2- 172.21.15.154:0/1080937432 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7faa1806af30 0x7faa1806d3f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7faa28001390 tx=0x7faa280012e0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:02.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:02.951+0000 7faa2f7fe640 1 -- 172.21.15.154:0/1080937432 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7faa300757e0 con 0x7faa380c2510 2023-09-23T03:19:03.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.151+0000 7faa3759e640 1 -- 172.21.15.154:0/1080937432 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7faa380cc8b0 con 0x7faa380c2510 2023-09-23T03:19:03.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.152+0000 7faa2f7fe640 1 -- 172.21.15.154:0/1080937432 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v49) v1 ==== 80+0+370 (secure 0 0 0) 0x7faa30078e00 con 0x7faa380c2510 2023-09-23T03:19:03.154 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.154:6837/4079550625 2023-09-24T03:18:03.516922+0000 2023-09-23T03:19:03.154 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.154:6835/4079550625 2023-09-24T03:18:03.516922+0000 2023-09-23T03:19:03.154 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.154:6837/617287200 2023-09-24T03:17:57.206517+0000 2023-09-23T03:19:03.154 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.190:6834/2131436414 2023-09-24T03:17:55.088768+0000 2023-09-23T03:19:03.154 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.154:6835/617287200 2023-09-24T03:17:57.206517+0000 2023-09-23T03:19:03.154 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.190:6832/2131436414 2023-09-24T03:17:55.088768+0000 2023-09-23T03:19:03.155 INFO:teuthology.orchestra.run.smithi154.stderr:listed 6 entries 2023-09-23T03:19:03.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.155+0000 7faa3759e640 1 -- 172.21.15.154:0/1080937432 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7faa1806af30 msgr2=0x7faa1806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:03.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.155+0000 7faa3759e640 1 --2- 172.21.15.154:0/1080937432 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7faa1806af30 0x7faa1806d3f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7faa28001390 tx=0x7faa280012e0 comp rx=0 tx=0).stop 2023-09-23T03:19:03.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.155+0000 7faa3759e640 1 -- 172.21.15.154:0/1080937432 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa380c2510 msgr2=0x7faa380c2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:03.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.155+0000 7faa3759e640 1 --2- 172.21.15.154:0/1080937432 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa380c2510 0x7faa380c2990 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7faa30005040 tx=0x7faa30001120 comp rx=0 tx=0).stop 2023-09-23T03:19:03.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.155+0000 7faa3759e640 1 -- 172.21.15.154:0/1080937432 shutdown_connections 2023-09-23T03:19:03.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.155+0000 7faa3759e640 1 --2- 172.21.15.154:0/1080937432 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7faa1806af30 0x7faa1806d3f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:03.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.155+0000 7faa3759e640 1 --2- 172.21.15.154:0/1080937432 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa380c2510 0x7faa380c2990 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:03.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.155+0000 7faa3759e640 1 --2- 172.21.15.154:0/1080937432 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa38159480 0x7faa38159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:03.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.155+0000 7faa3759e640 1 --2- 172.21.15.154:0/1080937432 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa24004d00 0x7faa38152730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:03.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.155+0000 7faa3759e640 1 -- 172.21.15.154:0/1080937432 >> 172.21.15.154:0/1080937432 conn(0x7faa3806a390 msgr2=0x7faa380cae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:03.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.155+0000 7faa3759e640 1 -- 172.21.15.154:0/1080937432 shutdown_connections 2023-09-23T03:19:03.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.155+0000 7faa3759e640 1 -- 172.21.15.154:0/1080937432 wait complete. 2023-09-23T03:19:03.176 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi154.front.sepia.ceph.com... 2023-09-23T03:19:03.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:19:03.176 DEBUG:teuthology.orchestra.run.smithi154:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-09-23T03:19:03.264 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:ceph-fuse[105601]: fuse finished with error 0 and tester_r 0 2023-09-23T03:19:03.279 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2023-09-23T03:19:03.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:19:03.280 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-09-23T03:19:03.407 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:19:03.407 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:19:03.470 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053603,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248804k,nr_inodes=812201,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-09-23T03:19:03.471 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi190.front.sepia.ceph.com 2023-09-23T03:19:03.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:19:03.471 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-09-23T03:19:03.526 INFO:teuthology.orchestra.run.smithi190.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-09-23T03:19:03.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T03:19:03.528 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:19:03.528 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:19:03.588 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053600,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-09-23T03:19:03.589 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2023-09-23T03:19:03.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:19:03.589 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-09-23T03:19:03.641 INFO:teuthology.orchestra.run.smithi154.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-09-23T03:19:03.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T03:19:03.642 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:19:03.642 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:19:03.702 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053603,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248804k,nr_inodes=812201,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-09-23T03:19:03.702 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi190.front.sepia.ceph.com 2023-09-23T03:19:03.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:19:03.702 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-09-23T03:19:03.755 INFO:teuthology.orchestra.run.smithi190.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-09-23T03:19:03.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T03:19:03.757 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:19:03.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.977+0000 7f07e6ff5640 1 Processor -- start 2023-09-23T03:19:03.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.977+0000 7f07e6ff5640 1 -- start start 2023-09-23T03:19:03.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.979+0000 7f07e6ff5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f07e00acce0 0x7f07e015ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:03.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.979+0000 7f07e6ff5640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f07e00b1b20 0x7f07e00ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:03.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.979+0000 7f07e6ff5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e015ac30 con 0x7f07e00cee70 2023-09-23T03:19:03.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.979+0000 7f07e6ff5640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07e00b2e90 con 0x7f07e00acce0 2023-09-23T03:19:03.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.979+0000 7f07e6ff5640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07e00cf300 con 0x7f07e00b1b20 2023-09-23T03:19:03.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.979+0000 7f07e5592640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f07e00cee70 0x7f07e006cb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:41556/0 (socket says 172.21.15.154:41556) 2023-09-23T03:19:03.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.979+0000 7f07e5592640 1 -- 172.21.15.154:0/1949600360 learned_addr learned my addr 172.21.15.154:0/1949600360 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:19:03.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.979+0000 7f07e4d91640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f07e00acce0 0x7f07e015ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:03.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.979+0000 7f07dffff640 1 --2- 172.21.15.154:0/1949600360 >> v2:172.21.15.190:3300/0 conn(0x7f07e00b1b20 0x7f07e00ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:03.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.980+0000 7f07df7fe640 1 -- 172.21.15.154:0/1949600360 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1757959095 0 0) 0x7f07d8001490 con 0x7f07e00cee70 2023-09-23T03:19:03.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.980+0000 7f07df7fe640 1 -- 172.21.15.154:0/1949600360 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07d0003520 con 0x7f07e00cee70 2023-09-23T03:19:03.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.980+0000 7f07e4d91640 1 -- 172.21.15.154:0/1949600360 >> v2:172.21.15.190:3300/0 conn(0x7f07e00b1b20 msgr2=0x7f07e00ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:03.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.980+0000 7f07e4d91640 1 --2- 172.21.15.154:0/1949600360 >> v2:172.21.15.190:3300/0 conn(0x7f07e00b1b20 0x7f07e00ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:03.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.980+0000 7f07e4d91640 1 -- 172.21.15.154:0/1949600360 >> v1:172.21.15.154:6789/0 conn(0x7f07e00cee70 legacy=0x7f07e006cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:03.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.980+0000 7f07e4d91640 1 -- 172.21.15.154:0/1949600360 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e00b1f70 con 0x7f07e00acce0 2023-09-23T03:19:03.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.981+0000 7f07e4d91640 1 --2- 172.21.15.154:0/1949600360 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07e00acce0 0x7f07e015ed60 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f07e00c9430 tx=0x7f07d402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bacd0816ea6ae873 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:03.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.981+0000 7f07df7fe640 1 -- 172.21.15.154:0/1949600360 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d402e070 con 0x7f07e00acce0 2023-09-23T03:19:03.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.982+0000 7f07df7fe640 1 -- 172.21.15.154:0/1949600360 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d40315a0 con 0x7f07e00acce0 2023-09-23T03:19:03.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.982+0000 7f07df7fe640 1 -- 172.21.15.154:0/1949600360 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d4031910 con 0x7f07e00acce0 2023-09-23T03:19:03.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.982+0000 7f07e6ff5640 1 -- 172.21.15.154:0/1949600360 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07e00acce0 msgr2=0x7f07e015ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:03.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.982+0000 7f07e6ff5640 1 --2- 172.21.15.154:0/1949600360 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07e00acce0 0x7f07e015ed60 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f07e00c9430 tx=0x7f07d402b840 comp rx=0 tx=0).stop 2023-09-23T03:19:03.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.982+0000 7f07e6ff5640 1 -- 172.21.15.154:0/1949600360 shutdown_connections 2023-09-23T03:19:03.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.982+0000 7f07e6ff5640 1 --2- 172.21.15.154:0/1949600360 >> v2:172.21.15.190:3300/0 conn(0x7f07e00b1b20 0x7f07e00ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:03.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.982+0000 7f07e6ff5640 1 --2- 172.21.15.154:0/1949600360 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07e00acce0 0x7f07e015ed60 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:03.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.982+0000 7f07e6ff5640 1 -- 172.21.15.154:0/1949600360 >> 172.21.15.154:0/1949600360 conn(0x7f07e006a0f0 msgr2=0x7f07e006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:03.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.982+0000 7f07e6ff5640 1 -- 172.21.15.154:0/1949600360 shutdown_connections 2023-09-23T03:19:03.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.982+0000 7f07e6ff5640 1 -- 172.21.15.154:0/1949600360 wait complete. 2023-09-23T03:19:03.986 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.984+0000 7f07e6ff5640 1 Processor -- start 2023-09-23T03:19:03.986 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.984+0000 7f07e6ff5640 1 -- start start 2023-09-23T03:19:03.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.985+0000 7f07e6ff5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07e00acce0 0x7f07e01f1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:03.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.985+0000 7f07e6ff5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07e00b1b20 0x7f07e015b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:03.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.985+0000 7f07e6ff5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07e00cee70 0x7f07e015d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:03.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.985+0000 7f07e6ff5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e015b340 con 0x7f07e00acce0 2023-09-23T03:19:03.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.985+0000 7f07e6ff5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e00cc0e0 con 0x7f07e00b1b20 2023-09-23T03:19:03.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.985+0000 7f07e6ff5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e00cc3c0 con 0x7f07e00cee70 2023-09-23T03:19:03.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.986+0000 7f07e4d91640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07e00acce0 0x7f07e01f1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:03.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.986+0000 7f07e4d91640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07e00acce0 0x7f07e01f1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54252/0 (socket says 172.21.15.154:54252) 2023-09-23T03:19:03.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.986+0000 7f07e4d91640 1 -- 172.21.15.154:0/3464534412 learned_addr learned my addr 172.21.15.154:0/3464534412 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:03.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.986+0000 7f07e4d91640 1 -- 172.21.15.154:0/3464534412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07e00cee70 msgr2=0x7f07e015d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:03.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.986+0000 7f07e4d91640 1 --2- 172.21.15.154:0/3464534412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07e00cee70 0x7f07e015d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:03.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.986+0000 7f07e4d91640 1 -- 172.21.15.154:0/3464534412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07e00b1b20 msgr2=0x7f07e015b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:03.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.986+0000 7f07e4d91640 1 --2- 172.21.15.154:0/3464534412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07e00b1b20 0x7f07e015b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:03.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.986+0000 7f07e4d91640 1 -- 172.21.15.154:0/3464534412 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d40093a0 con 0x7f07e00acce0 2023-09-23T03:19:03.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.986+0000 7f07e4d91640 1 --2- 172.21.15.154:0/3464534412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07e00acce0 0x7f07e01f1950 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f07d4034040 tx=0x7f07d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:03.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.987+0000 7f07dd7fa640 1 -- 172.21.15.154:0/3464534412 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d4003810 con 0x7f07e00acce0 2023-09-23T03:19:03.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.987+0000 7f07dd7fa640 1 -- 172.21.15.154:0/3464534412 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d4040070 con 0x7f07e00acce0 2023-09-23T03:19:03.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.987+0000 7f07dd7fa640 1 -- 172.21.15.154:0/3464534412 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d4003b90 con 0x7f07e00acce0 2023-09-23T03:19:03.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.987+0000 7f07e6ff5640 1 -- 172.21.15.154:0/3464534412 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07e00cc6a0 con 0x7f07e00acce0 2023-09-23T03:19:03.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.987+0000 7f07e6ff5640 1 -- 172.21.15.154:0/3464534412 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e01f1e90 con 0x7f07e00acce0 2023-09-23T03:19:03.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.988+0000 7f07e6ff5640 1 -- 172.21.15.154:0/3464534412 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07e00ccbc0 con 0x7f07e00acce0 2023-09-23T03:19:03.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.989+0000 7f07dd7fa640 1 -- 172.21.15.154:0/3464534412 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f07d4003cf0 con 0x7f07e00acce0 2023-09-23T03:19:03.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.990+0000 7f07dd7fa640 1 --2- 172.21.15.154:0/3464534412 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f07c406af30 0x7f07c406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:03.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.990+0000 7f07dd7fa640 1 -- 172.21.15.154:0/3464534412 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6720+0+0 (secure 0 0 0) 0x7f07d4042070 con 0x7f07e00acce0 2023-09-23T03:19:03.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.990+0000 7f07dffff640 1 --2- 172.21.15.154:0/3464534412 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f07c406af30 0x7f07c406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:03.992 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.991+0000 7f07dffff640 1 --2- 172.21.15.154:0/3464534412 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f07c406af30 0x7f07c406d3f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f07e01f5540 tx=0x7f07cc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:03.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:03.994+0000 7f07dd7fa640 1 -- 172.21.15.154:0/3464534412 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f07e00ccbc0 con 0x7f07e00acce0 2023-09-23T03:19:04.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.220+0000 7f07e6ff5640 1 -- 172.21.15.154:0/3464534412 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07e00ccbc0 con 0x7f07e00acce0 2023-09-23T03:19:04.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.222+0000 7f07dd7fa640 1 -- 172.21.15.154:0/3464534412 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4928 (secure 0 0 0) 0x7f07e00ccbc0 con 0x7f07e00acce0 2023-09-23T03:19:04.223 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:19:04.224 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":32,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5215,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3388818833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3388818833},{"type":"v1","addr":"172.21.15.190:6835","nonce":3388818833}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5991,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1964552060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1964552060},{"type":"v1","addr":"172.21.15.154:6837","nonce":1964552060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5994,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/1736999297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1736999297},{"type":"v1","addr":"172.21.15.190:6834","nonce":1736999297}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:17:56.221314+0000","modified":"2023-09-23T03:18:05.544508+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5988},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5988":{"gid":5988,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":4,"addr":"172.21.15.154:6836/4278807008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4278807008},{"type":"v1","addr":"172.21.15.154:6836","nonce":4278807008}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-09-23T03:19:04.224 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 32 2023-09-23T03:19:04.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.224+0000 7f07e6ff5640 1 -- 172.21.15.154:0/3464534412 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f07c406af30 msgr2=0x7f07c406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:04.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.224+0000 7f07e6ff5640 1 --2- 172.21.15.154:0/3464534412 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f07c406af30 0x7f07c406d3f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f07e01f5540 tx=0x7f07cc008040 comp rx=0 tx=0).stop 2023-09-23T03:19:04.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.224+0000 7f07e6ff5640 1 -- 172.21.15.154:0/3464534412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07e00acce0 msgr2=0x7f07e01f1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:04.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.224+0000 7f07e6ff5640 1 --2- 172.21.15.154:0/3464534412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07e00acce0 0x7f07e01f1950 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f07d4034040 tx=0x7f07d40046c0 comp rx=0 tx=0).stop 2023-09-23T03:19:04.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.225+0000 7f07e6ff5640 1 -- 172.21.15.154:0/3464534412 shutdown_connections 2023-09-23T03:19:04.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.225+0000 7f07e6ff5640 1 --2- 172.21.15.154:0/3464534412 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f07c406af30 0x7f07c406d3f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:04.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.225+0000 7f07e6ff5640 1 --2- 172.21.15.154:0/3464534412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f07e00cee70 0x7f07e015d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:04.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.225+0000 7f07e6ff5640 1 --2- 172.21.15.154:0/3464534412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f07e00b1b20 0x7f07e015b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:04.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.225+0000 7f07e6ff5640 1 --2- 172.21.15.154:0/3464534412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f07e00acce0 0x7f07e01f1950 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:04.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.225+0000 7f07e6ff5640 1 -- 172.21.15.154:0/3464534412 >> 172.21.15.154:0/3464534412 conn(0x7f07e006a0f0 msgr2=0x7f07e00cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:04.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.225+0000 7f07e6ff5640 1 -- 172.21.15.154:0/3464534412 shutdown_connections 2023-09-23T03:19:04.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.225+0000 7f07e6ff5640 1 -- 172.21.15.154:0/3464534412 wait complete. 2023-09-23T03:19:04.246 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:19:04.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.445+0000 7f30d73f5640 1 Processor -- start 2023-09-23T03:19:04.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.445+0000 7f30d73f5640 1 -- start start 2023-09-23T03:19:04.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.447+0000 7f30d73f5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30d00cf130 0x7f30d00ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:04.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.447+0000 7f30d73f5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f30d00b2fe0 0x7f30d015d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:04.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.447+0000 7f30d73f5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30d006ceb0 con 0x7f30d00ada30 2023-09-23T03:19:04.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.447+0000 7f30d73f5640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30d00ade10 con 0x7f30d00b2fe0 2023-09-23T03:19:04.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.447+0000 7f30d73f5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d0159c90 con 0x7f30d00cf130 2023-09-23T03:19:04.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.447+0000 7f30d5191640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f30d00ada30 0x7f30d00cea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:41558/0 (socket says 172.21.15.154:41558) 2023-09-23T03:19:04.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.447+0000 7f30d5191640 1 -- 172.21.15.154:0/2516539954 learned_addr learned my addr 172.21.15.154:0/2516539954 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:19:04.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.447+0000 7f30d5992640 1 --2- 172.21.15.154:0/2516539954 >> v2:172.21.15.154:3300/0 conn(0x7f30d00b2fe0 0x7f30d015d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:04.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.448+0000 7f30c7fff640 1 -- 172.21.15.154:0/2516539954 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 713175094 0 0) 0x7f30cc001290 con 0x7f30d00ada30 2023-09-23T03:19:04.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.448+0000 7f30c7fff640 1 -- 172.21.15.154:0/2516539954 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30bc003520 con 0x7f30d00ada30 2023-09-23T03:19:04.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.448+0000 7f30d4990640 1 --2- 172.21.15.154:0/2516539954 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30d00cf130 0x7f30d00ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:04.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.448+0000 7f30d5992640 1 -- 172.21.15.154:0/2516539954 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30d00cf130 msgr2=0x7f30d00ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:04.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.448+0000 7f30d5992640 1 --2- 172.21.15.154:0/2516539954 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30d00cf130 0x7f30d00ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:04.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.448+0000 7f30d5992640 1 -- 172.21.15.154:0/2516539954 >> v1:172.21.15.154:6789/0 conn(0x7f30d00ada30 legacy=0x7f30d00cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:04.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.448+0000 7f30d5992640 1 -- 172.21.15.154:0/2516539954 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d006c110 con 0x7f30d00b2fe0 2023-09-23T03:19:04.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.449+0000 7f30d5992640 1 --2- 172.21.15.154:0/2516539954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30d00b2fe0 0x7f30d015d380 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f30c8009d90 tx=0x7f30c802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d48324e209cbb9cd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:04.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.449+0000 7f30c7fff640 1 -- 172.21.15.154:0/2516539954 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c802cc60 con 0x7f30d00b2fe0 2023-09-23T03:19:04.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.449+0000 7f30c7fff640 1 -- 172.21.15.154:0/2516539954 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30c802cdc0 con 0x7f30d00b2fe0 2023-09-23T03:19:04.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.449+0000 7f30c7fff640 1 -- 172.21.15.154:0/2516539954 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c8034640 con 0x7f30d00b2fe0 2023-09-23T03:19:04.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.450+0000 7f30d73f5640 1 -- 172.21.15.154:0/2516539954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30d00b2fe0 msgr2=0x7f30d015d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:04.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.450+0000 7f30d73f5640 1 --2- 172.21.15.154:0/2516539954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30d00b2fe0 0x7f30d015d380 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f30c8009d90 tx=0x7f30c802beb0 comp rx=0 tx=0).stop 2023-09-23T03:19:04.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.450+0000 7f30d73f5640 1 -- 172.21.15.154:0/2516539954 shutdown_connections 2023-09-23T03:19:04.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.450+0000 7f30d73f5640 1 --2- 172.21.15.154:0/2516539954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30d00b2fe0 0x7f30d015d380 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:04.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.450+0000 7f30d73f5640 1 --2- 172.21.15.154:0/2516539954 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30d00cf130 0x7f30d00ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:04.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.450+0000 7f30d73f5640 1 -- 172.21.15.154:0/2516539954 >> 172.21.15.154:0/2516539954 conn(0x7f30d006a0f0 msgr2=0x7f30d006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:04.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.450+0000 7f30d73f5640 1 -- 172.21.15.154:0/2516539954 shutdown_connections 2023-09-23T03:19:04.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.450+0000 7f30d73f5640 1 -- 172.21.15.154:0/2516539954 wait complete. 2023-09-23T03:19:04.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.452+0000 7f30d73f5640 1 Processor -- start 2023-09-23T03:19:04.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.452+0000 7f30d73f5640 1 -- start start 2023-09-23T03:19:04.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.453+0000 7f30d73f5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30d00ada30 0x7f30d0097a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:04.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.453+0000 7f30d73f5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30d00b2fe0 0x7f30d01f2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:04.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.453+0000 7f30d73f5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30d00cf130 0x7f30d00c2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:04.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.453+0000 7f30d73f5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d01f3a70 con 0x7f30d00ada30 2023-09-23T03:19:04.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.453+0000 7f30d73f5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d0097f40 con 0x7f30d00cf130 2023-09-23T03:19:04.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.453+0000 7f30d73f5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d00c4ca0 con 0x7f30d00b2fe0 2023-09-23T03:19:04.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.453+0000 7f30d5191640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30d00ada30 0x7f30d0097a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:04.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.454+0000 7f30d5191640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30d00ada30 0x7f30d0097a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54266/0 (socket says 172.21.15.154:54266) 2023-09-23T03:19:04.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.454+0000 7f30d5191640 1 -- 172.21.15.154:0/960323662 learned_addr learned my addr 172.21.15.154:0/960323662 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:04.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.454+0000 7f30d5191640 1 -- 172.21.15.154:0/960323662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30d00b2fe0 msgr2=0x7f30d01f2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:04.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.454+0000 7f30d4990640 1 --2- 172.21.15.154:0/960323662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30d00b2fe0 0x7f30d01f2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:04.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.454+0000 7f30d5191640 1 --2- 172.21.15.154:0/960323662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30d00b2fe0 0x7f30d01f2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:04.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.454+0000 7f30d5992640 1 --2- 172.21.15.154:0/960323662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30d00cf130 0x7f30d00c2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:04.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.454+0000 7f30d5191640 1 -- 172.21.15.154:0/960323662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30d00cf130 msgr2=0x7f30d00c2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:04.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.454+0000 7f30d5191640 1 --2- 172.21.15.154:0/960323662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30d00cf130 0x7f30d00c2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:04.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.454+0000 7f30d5191640 1 -- 172.21.15.154:0/960323662 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c8009a70 con 0x7f30d00ada30 2023-09-23T03:19:04.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.454+0000 7f30d5191640 1 --2- 172.21.15.154:0/960323662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30d00ada30 0x7f30d0097a00 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f30cc00c4d0 tx=0x7f30cc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:04.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.454+0000 7f30c5ffb640 1 -- 172.21.15.154:0/960323662 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30cc007480 con 0x7f30d00ada30 2023-09-23T03:19:04.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.455+0000 7f30c5ffb640 1 -- 172.21.15.154:0/960323662 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30cc0075e0 con 0x7f30d00ada30 2023-09-23T03:19:04.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.455+0000 7f30c5ffb640 1 -- 172.21.15.154:0/960323662 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30cc0059e0 con 0x7f30d00ada30 2023-09-23T03:19:04.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.455+0000 7f30d73f5640 1 -- 172.21.15.154:0/960323662 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d01f3160 con 0x7f30d00ada30 2023-09-23T03:19:04.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.455+0000 7f30d73f5640 1 -- 172.21.15.154:0/960323662 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d00c5220 con 0x7f30d00ada30 2023-09-23T03:19:04.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.456+0000 7f30d73f5640 1 -- 172.21.15.154:0/960323662 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30d0066fd0 con 0x7f30d00ada30 2023-09-23T03:19:04.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.457+0000 7f30c5ffb640 1 -- 172.21.15.154:0/960323662 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f30cc005b40 con 0x7f30d00ada30 2023-09-23T03:19:04.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.458+0000 7f30c5ffb640 1 --2- 172.21.15.154:0/960323662 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f30b406af30 0x7f30b406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:04.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.458+0000 7f30c5ffb640 1 -- 172.21.15.154:0/960323662 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6720+0+0 (secure 0 0 0) 0x7f30cc088f50 con 0x7f30d00ada30 2023-09-23T03:19:04.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.459+0000 7f30d4990640 1 --2- 172.21.15.154:0/960323662 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f30b406af30 0x7f30b406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:04.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.459+0000 7f30d4990640 1 --2- 172.21.15.154:0/960323662 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f30b406af30 0x7f30b406d3f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f30c0006fd0 tx=0x7f30c0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:04.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.463+0000 7f30c5ffb640 1 -- 172.21.15.154:0/960323662 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f30cc053bb0 con 0x7f30d00ada30 2023-09-23T03:19:04.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.655+0000 7f30d73f5640 1 -- 172.21.15.154:0/960323662 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f30d01f3bb0 con 0x7f30d00ada30 2023-09-23T03:19:04.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.656+0000 7f30c5ffb640 1 -- 172.21.15.154:0/960323662 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+18984 (secure 0 0 0) 0x7f30d01f3bb0 con 0x7f30d00ada30 2023-09-23T03:19:04.658 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:19:04.658 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":49,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:18:04.269705+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:17:08.933690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-09-23T03:17:10.141711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data_extra","create_time":"2023-09-23T03:17:21.021560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.35","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6837/4079550625":"2023-09-24T03:18:03.516922+0000","172.21.15.154:6835/4079550625":"2023-09-24T03:18:03.516922+0000","172.21.15.154:6837/617287200":"2023-09-24T03:17:57.206517+0000","172.21.15.190:6834/2131436414":"2023-09-24T03:17:55.088768+0000","172.21.15.154:6835/617287200":"2023-09-24T03:17:57.206517+0000","172.21.15.190:6832/2131436414":"2023-09-24T03:17:55.088768+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:19:04.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.659+0000 7f30d73f5640 1 -- 172.21.15.154:0/960323662 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f30b406af30 msgr2=0x7f30b406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:04.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.659+0000 7f30d73f5640 1 --2- 172.21.15.154:0/960323662 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f30b406af30 0x7f30b406d3f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f30c0006fd0 tx=0x7f30c0008040 comp rx=0 tx=0).stop 2023-09-23T03:19:04.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.659+0000 7f30d73f5640 1 -- 172.21.15.154:0/960323662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30d00ada30 msgr2=0x7f30d0097a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:04.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.659+0000 7f30d73f5640 1 --2- 172.21.15.154:0/960323662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30d00ada30 0x7f30d0097a00 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f30cc00c4d0 tx=0x7f30cc00c990 comp rx=0 tx=0).stop 2023-09-23T03:19:04.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.659+0000 7f30d73f5640 1 -- 172.21.15.154:0/960323662 shutdown_connections 2023-09-23T03:19:04.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.659+0000 7f30d73f5640 1 --2- 172.21.15.154:0/960323662 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f30b406af30 0x7f30b406d3f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:04.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.659+0000 7f30d73f5640 1 --2- 172.21.15.154:0/960323662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f30d00cf130 0x7f30d00c2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:04.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.659+0000 7f30d73f5640 1 --2- 172.21.15.154:0/960323662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f30d00b2fe0 0x7f30d01f2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:04.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.659+0000 7f30d73f5640 1 --2- 172.21.15.154:0/960323662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f30d00ada30 0x7f30d0097a00 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:04.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.659+0000 7f30d73f5640 1 -- 172.21.15.154:0/960323662 >> 172.21.15.154:0/960323662 conn(0x7f30d006a0f0 msgr2=0x7f30d00aefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:04.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.660+0000 7f30d73f5640 1 -- 172.21.15.154:0/960323662 shutdown_connections 2023-09-23T03:19:04.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.660+0000 7f30d73f5640 1 -- 172.21.15.154:0/960323662 wait complete. 2023-09-23T03:19:04.680 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:17:08.933690+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:17:10.141711+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-09-23T03:17:21.021560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T03:19:04.680 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:19:04.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.896+0000 7f68bb70f640 1 Processor -- start 2023-09-23T03:19:04.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.897+0000 7f68bb70f640 1 -- start start 2023-09-23T03:19:04.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.898+0000 7f68bb70f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f68b40b28c0 0x7f68b40ce9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:04.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.898+0000 7f68bb70f640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68b40ce510 con 0x7f68b415abe0 2023-09-23T03:19:04.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.898+0000 7f68bb70f640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68b40cd390 con 0x7f68b40cc9b0 2023-09-23T03:19:04.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.898+0000 7f68bb70f640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68b40cd670 con 0x7f68b40b28c0 2023-09-23T03:19:04.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.898+0000 7f68b94ab640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f68b415abe0 0x7f68b40cb1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:41570/0 (socket says 172.21.15.154:41570) 2023-09-23T03:19:04.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.898+0000 7f68b94ab640 1 -- 172.21.15.154:0/1723027596 learned_addr learned my addr 172.21.15.154:0/1723027596 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:19:04.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.898+0000 7f68b9cac640 1 --2- 172.21.15.154:0/1723027596 >> v2:172.21.15.154:3300/0 conn(0x7f68b40b28c0 0x7f68b40ce9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:04.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.899+0000 7f68b3fff640 1 -- 172.21.15.154:0/1723027596 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 955218127 0 0) 0x7f68b40ce510 con 0x7f68b415abe0 2023-09-23T03:19:04.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.899+0000 7f68b3fff640 1 -- 172.21.15.154:0/1723027596 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f689c003520 con 0x7f68b415abe0 2023-09-23T03:19:04.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.899+0000 7f68b9cac640 1 -- 172.21.15.154:0/1723027596 >> v1:172.21.15.190:6789/0 conn(0x7f68b40cc9b0 legacy=0x7f68b40b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:04.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.899+0000 7f68b9cac640 1 -- 172.21.15.154:0/1723027596 >> v1:172.21.15.154:6789/0 conn(0x7f68b415abe0 legacy=0x7f68b40cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:04.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.899+0000 7f68b9cac640 1 -- 172.21.15.154:0/1723027596 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b40cd950 con 0x7f68b40b28c0 2023-09-23T03:19:04.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.900+0000 7f68b9cac640 1 --2- 172.21.15.154:0/1723027596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68b40b28c0 0x7f68b40ce9a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f68ac009d90 tx=0x7f68ac02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e5964e7a4afab3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:04.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.901+0000 7f68b3fff640 1 -- 172.21.15.154:0/1723027596 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ac02cbd0 con 0x7f68b40b28c0 2023-09-23T03:19:04.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.901+0000 7f68b3fff640 1 -- 172.21.15.154:0/1723027596 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68ac02cd30 con 0x7f68b40b28c0 2023-09-23T03:19:04.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.901+0000 7f68b3fff640 1 -- 172.21.15.154:0/1723027596 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ac02cea0 con 0x7f68b40b28c0 2023-09-23T03:19:04.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.901+0000 7f68bb70f640 1 -- 172.21.15.154:0/1723027596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68b40b28c0 msgr2=0x7f68b40ce9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:04.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.901+0000 7f68bb70f640 1 --2- 172.21.15.154:0/1723027596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68b40b28c0 0x7f68b40ce9a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f68ac009d90 tx=0x7f68ac02be20 comp rx=0 tx=0).stop 2023-09-23T03:19:04.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.901+0000 7f68bb70f640 1 -- 172.21.15.154:0/1723027596 shutdown_connections 2023-09-23T03:19:04.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.901+0000 7f68bb70f640 1 --2- 172.21.15.154:0/1723027596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68b40b28c0 0x7f68b40ce9a0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:04.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.901+0000 7f68bb70f640 1 -- 172.21.15.154:0/1723027596 >> 172.21.15.154:0/1723027596 conn(0x7f68b406fc40 msgr2=0x7f68b4070020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:04.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.902+0000 7f68bb70f640 1 -- 172.21.15.154:0/1723027596 shutdown_connections 2023-09-23T03:19:04.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.902+0000 7f68bb70f640 1 -- 172.21.15.154:0/1723027596 wait complete. 2023-09-23T03:19:04.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.903+0000 7f68bb70f640 1 Processor -- start 2023-09-23T03:19:04.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.903+0000 7f68bb70f640 1 -- start start 2023-09-23T03:19:04.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.904+0000 7f68bb70f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68b40b28c0 0x7f68b40c7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:04.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.904+0000 7f68bb70f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68b40cc9b0 0x7f68b41595e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:04.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.904+0000 7f68bb70f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68b415abe0 0x7f68b40c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:04.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.904+0000 7f68bb70f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b41f39b0 con 0x7f68b415abe0 2023-09-23T03:19:04.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.904+0000 7f68bb70f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b40c8820 con 0x7f68b40cc9b0 2023-09-23T03:19:04.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.904+0000 7f68bb70f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b41f2a70 con 0x7f68b40b28c0 2023-09-23T03:19:04.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.905+0000 7f68b9cac640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68b415abe0 0x7f68b40c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:04.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.905+0000 7f68b9cac640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68b415abe0 0x7f68b40c8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54290/0 (socket says 172.21.15.154:54290) 2023-09-23T03:19:04.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.905+0000 7f68b9cac640 1 -- 172.21.15.154:0/2155309857 learned_addr learned my addr 172.21.15.154:0/2155309857 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:04.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.905+0000 7f68b8caa640 1 --2- 172.21.15.154:0/2155309857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68b40cc9b0 0x7f68b41595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:04.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.905+0000 7f68b94ab640 1 --2- 172.21.15.154:0/2155309857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68b40b28c0 0x7f68b40c7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:04.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.905+0000 7f68b9cac640 1 -- 172.21.15.154:0/2155309857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68b40b28c0 msgr2=0x7f68b40c7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:04.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.905+0000 7f68b9cac640 1 --2- 172.21.15.154:0/2155309857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68b40b28c0 0x7f68b40c7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:04.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.905+0000 7f68b9cac640 1 -- 172.21.15.154:0/2155309857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68b40cc9b0 msgr2=0x7f68b41595e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:04.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.905+0000 7f68b9cac640 1 --2- 172.21.15.154:0/2155309857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68b40cc9b0 0x7f68b41595e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:04.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.905+0000 7f68b9cac640 1 -- 172.21.15.154:0/2155309857 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ac009a70 con 0x7f68b415abe0 2023-09-23T03:19:04.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.905+0000 7f68b8caa640 1 --2- 172.21.15.154:0/2155309857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68b40cc9b0 0x7f68b41595e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:04.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.906+0000 7f68b9cac640 1 --2- 172.21.15.154:0/2155309857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68b415abe0 0x7f68b40c8bc0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f68ac009d90 tx=0x7f68ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:04.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.906+0000 7f68b1ffb640 1 -- 172.21.15.154:0/2155309857 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ac02cea0 con 0x7f68b415abe0 2023-09-23T03:19:04.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.906+0000 7f68b1ffb640 1 -- 172.21.15.154:0/2155309857 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68ac040650 con 0x7f68b415abe0 2023-09-23T03:19:04.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.906+0000 7f68b1ffb640 1 -- 172.21.15.154:0/2155309857 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ac040a00 con 0x7f68b415abe0 2023-09-23T03:19:04.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.906+0000 7f68bb70f640 1 -- 172.21.15.154:0/2155309857 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68b41f2d30 con 0x7f68b415abe0 2023-09-23T03:19:04.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.906+0000 7f68bb70f640 1 -- 172.21.15.154:0/2155309857 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b41f32f0 con 0x7f68b415abe0 2023-09-23T03:19:04.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.907+0000 7f68bb70f640 1 -- 172.21.15.154:0/2155309857 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68b40cce20 con 0x7f68b415abe0 2023-09-23T03:19:04.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.908+0000 7f68b1ffb640 1 -- 172.21.15.154:0/2155309857 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f68ac040b60 con 0x7f68b415abe0 2023-09-23T03:19:04.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.909+0000 7f68b1ffb640 1 --2- 172.21.15.154:0/2155309857 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f68a006af30 0x7f68a006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:04.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.909+0000 7f68b1ffb640 1 -- 172.21.15.154:0/2155309857 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6720+0+0 (secure 0 0 0) 0x7f68ac0ac850 con 0x7f68b415abe0 2023-09-23T03:19:04.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.909+0000 7f68b94ab640 1 --2- 172.21.15.154:0/2155309857 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f68a006af30 0x7f68a006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:04.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.910+0000 7f68b94ab640 1 --2- 172.21.15.154:0/2155309857 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f68a006af30 0x7f68a006d3f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f68a40010e0 tx=0x7f68a4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:04.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:04.913+0000 7f68b1ffb640 1 -- 172.21.15.154:0/2155309857 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f68b40cce20 con 0x7f68b415abe0 2023-09-23T03:19:05.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.099+0000 7f68bb70f640 1 -- 172.21.15.154:0/2155309857 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f68b40cce20 con 0x7f68b415abe0 2023-09-23T03:19:05.101 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.100+0000 7f68b1ffb640 1 -- 172.21.15.154:0/2155309857 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7f68b40cce20 con 0x7f68b415abe0 2023-09-23T03:19:05.101 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:19:05.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.103+0000 7f68bb70f640 1 -- 172.21.15.154:0/2155309857 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f68a006af30 msgr2=0x7f68a006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:05.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.103+0000 7f68bb70f640 1 --2- 172.21.15.154:0/2155309857 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f68a006af30 0x7f68a006d3f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f68a40010e0 tx=0x7f68a4009040 comp rx=0 tx=0).stop 2023-09-23T03:19:05.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.103+0000 7f68bb70f640 1 -- 172.21.15.154:0/2155309857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68b415abe0 msgr2=0x7f68b40c8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:05.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.103+0000 7f68bb70f640 1 --2- 172.21.15.154:0/2155309857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68b415abe0 0x7f68b40c8bc0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f68ac009d90 tx=0x7f68ac004a40 comp rx=0 tx=0).stop 2023-09-23T03:19:05.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.103+0000 7f68bb70f640 1 -- 172.21.15.154:0/2155309857 shutdown_connections 2023-09-23T03:19:05.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.103+0000 7f68bb70f640 1 --2- 172.21.15.154:0/2155309857 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f68a006af30 0x7f68a006d3f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:05.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.103+0000 7f68bb70f640 1 --2- 172.21.15.154:0/2155309857 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68b415abe0 0x7f68b40c8bc0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:05.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.103+0000 7f68bb70f640 1 --2- 172.21.15.154:0/2155309857 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68b40cc9b0 0x7f68b41595e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:05.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.103+0000 7f68bb70f640 1 --2- 172.21.15.154:0/2155309857 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68b40b28c0 0x7f68b40c7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:05.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.103+0000 7f68bb70f640 1 -- 172.21.15.154:0/2155309857 >> 172.21.15.154:0/2155309857 conn(0x7f68b406fc40 msgr2=0x7f68b40cb430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:05.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.103+0000 7f68bb70f640 1 -- 172.21.15.154:0/2155309857 shutdown_connections 2023-09-23T03:19:05.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.103+0000 7f68bb70f640 1 -- 172.21.15.154:0/2155309857 wait complete. 2023-09-23T03:19:05.122 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:19:05.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.335+0000 7f8b33fff640 1 Processor -- start 2023-09-23T03:19:05.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.335+0000 7f8b33fff640 1 -- start start 2023-09-23T03:19:05.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.337+0000 7f8b33fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8b3406cec0 0x7f8b340cc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:05.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.337+0000 7f8b33fff640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b3406a780 con 0x7f8b3406dcd0 2023-09-23T03:19:05.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.337+0000 7f8b33fff640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b3406b620 con 0x7f8b3415a700 2023-09-23T03:19:05.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.337+0000 7f8b33fff640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b3415aae0 con 0x7f8b3406cec0 2023-09-23T03:19:05.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.337+0000 7f8b2bfff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f8b3406dcd0 0x7f8b3406c7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:41572/0 (socket says 172.21.15.154:41572) 2023-09-23T03:19:05.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.337+0000 7f8b2bfff640 1 -- 172.21.15.154:0/1266771535 learned_addr learned my addr 172.21.15.154:0/1266771535 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:19:05.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.337+0000 7f8b337fe640 1 --2- 172.21.15.154:0/1266771535 >> v2:172.21.15.190:3300/0 conn(0x7f8b3406cec0 0x7f8b340cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:05.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.337+0000 7f8b327fc640 1 -- 172.21.15.154:0/1266771535 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2962182080 0 0) 0x7f8b3406a780 con 0x7f8b3406dcd0 2023-09-23T03:19:05.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.338+0000 7f8b327fc640 1 -- 172.21.15.154:0/1266771535 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b20003520 con 0x7f8b3406dcd0 2023-09-23T03:19:05.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.338+0000 7f8b327fc640 1 -- 172.21.15.154:0/1266771535 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 946005075 0 0) 0x7f8b20003520 con 0x7f8b3406dcd0 2023-09-23T03:19:05.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.338+0000 7f8b327fc640 1 -- 172.21.15.154:0/1266771535 >> v2:172.21.15.190:3300/0 conn(0x7f8b3406cec0 msgr2=0x7f8b340cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:05.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.338+0000 7f8b327fc640 1 --2- 172.21.15.154:0/1266771535 >> v2:172.21.15.190:3300/0 conn(0x7f8b3406cec0 0x7f8b340cc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:05.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.338+0000 7f8b327fc640 1 -- 172.21.15.154:0/1266771535 >> v1:172.21.15.190:6789/0 conn(0x7f8b3415a700 legacy=0x7f8b3406af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:05.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.338+0000 7f8b327fc640 1 -- 172.21.15.154:0/1266771535 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b340ccd60 con 0x7f8b3406dcd0 2023-09-23T03:19:05.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.338+0000 7f8b327fc640 1 -- 172.21.15.154:0/1266771535 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f8b1c001670 con 0x7f8b3406dcd0 2023-09-23T03:19:05.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.338+0000 7f8b327fc640 1 -- 172.21.15.154:0/1266771535 >> v1:172.21.15.154:6789/0 conn(0x7f8b3406dcd0 legacy=0x7f8b3406c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:05.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.338+0000 7f8b327fc640 1 --2- 172.21.15.154:0/1266771535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b200039c0 0x7f8b20003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:05.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.338+0000 7f8b327fc640 1 --2- 172.21.15.154:0/1266771535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b200047c0 0x7f8b20004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:05.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.338+0000 7f8b327fc640 1 --2- 172.21.15.154:0/1266771535 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b20005180 0x7f8b2000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:05.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.338+0000 7f8b327fc640 1 -- 172.21.15.154:0/1266771535 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3415aae0 con 0x7f8b200039c0 2023-09-23T03:19:05.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.338+0000 7f8b327fc640 1 -- 172.21.15.154:0/1266771535 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2000c080 con 0x7f8b200047c0 2023-09-23T03:19:05.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.338+0000 7f8b327fc640 1 -- 172.21.15.154:0/1266771535 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2000c240 con 0x7f8b20005180 2023-09-23T03:19:05.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.339+0000 7f8b32ffd640 1 --2- 172.21.15.154:0/1266771535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b200039c0 0x7f8b20003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:05.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.339+0000 7f8b32ffd640 1 -- 172.21.15.154:0/1266771535 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b20005180 msgr2=0x7f8b2000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:05.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.339+0000 7f8b32ffd640 1 --2- 172.21.15.154:0/1266771535 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b20005180 0x7f8b2000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:05.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.339+0000 7f8b2bfff640 1 --2- 172.21.15.154:0/1266771535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b200047c0 0x7f8b20004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:05.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.339+0000 7f8b32ffd640 1 -- 172.21.15.154:0/1266771535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b200047c0 msgr2=0x7f8b20004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:05.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.339+0000 7f8b32ffd640 1 --2- 172.21.15.154:0/1266771535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b200047c0 0x7f8b20004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:05.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.339+0000 7f8b32ffd640 1 -- 172.21.15.154:0/1266771535 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b20003520 con 0x7f8b200039c0 2023-09-23T03:19:05.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.339+0000 7f8b337fe640 1 -- 172.21.15.154:0/1266771535 reap_dead start 2023-09-23T03:19:05.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.339+0000 7f8b2bfff640 1 --2- 172.21.15.154:0/1266771535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b200047c0 0x7f8b20004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:05.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.340+0000 7f8b32ffd640 1 --2- 172.21.15.154:0/1266771535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b200039c0 0x7f8b20003de0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f8b24001750 tx=0x7f8b200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2312f7b4d83d9ef3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:05.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.340+0000 7f8b327fc640 1 -- 172.21.15.154:0/1266771535 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b240052d0 con 0x7f8b200039c0 2023-09-23T03:19:05.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.340+0000 7f8b327fc640 1 -- 172.21.15.154:0/1266771535 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2402e050 con 0x7f8b200039c0 2023-09-23T03:19:05.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.340+0000 7f8b327fc640 1 -- 172.21.15.154:0/1266771535 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b24003830 con 0x7f8b200039c0 2023-09-23T03:19:05.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.341+0000 7f8b33fff640 1 -- 172.21.15.154:0/1266771535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b200039c0 msgr2=0x7f8b20003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:05.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.341+0000 7f8b33fff640 1 --2- 172.21.15.154:0/1266771535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b200039c0 0x7f8b20003de0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f8b24001750 tx=0x7f8b200057f0 comp rx=0 tx=0).stop 2023-09-23T03:19:05.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.341+0000 7f8b33fff640 1 -- 172.21.15.154:0/1266771535 shutdown_connections 2023-09-23T03:19:05.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.341+0000 7f8b33fff640 1 --2- 172.21.15.154:0/1266771535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b200039c0 0x7f8b20003de0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:05.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.341+0000 7f8b33fff640 1 -- 172.21.15.154:0/1266771535 >> 172.21.15.154:0/1266771535 conn(0x7f8b340cb630 msgr2=0x7f8b340cba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:05.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.341+0000 7f8b33fff640 1 -- 172.21.15.154:0/1266771535 shutdown_connections 2023-09-23T03:19:05.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.341+0000 7f8b33fff640 1 -- 172.21.15.154:0/1266771535 wait complete. 2023-09-23T03:19:05.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.343+0000 7f8b33fff640 1 Processor -- start 2023-09-23T03:19:05.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.343+0000 7f8b33fff640 1 -- start start 2023-09-23T03:19:05.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.344+0000 7f8b33fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b200039c0 0x7f8b340b4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:05.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.344+0000 7f8b33fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b341553a0 0x7f8b34155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:05.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.344+0000 7f8b33fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b341f57b0 0x7f8b341f5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:05.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.344+0000 7f8b33fff640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b34158a90 con 0x7f8b341553a0 2023-09-23T03:19:05.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.344+0000 7f8b33fff640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b340b4820 con 0x7f8b200039c0 2023-09-23T03:19:05.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.344+0000 7f8b33fff640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b340b2b60 con 0x7f8b341f57b0 2023-09-23T03:19:05.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.344+0000 7f8b2bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b341553a0 0x7f8b34155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:05.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.345+0000 7f8b2bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b341553a0 0x7f8b34155800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54316/0 (socket says 172.21.15.154:54316) 2023-09-23T03:19:05.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.345+0000 7f8b2bfff640 1 -- 172.21.15.154:0/681911700 learned_addr learned my addr 172.21.15.154:0/681911700 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:05.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.345+0000 7f8b32ffd640 1 --2- 172.21.15.154:0/681911700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b200039c0 0x7f8b340b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:05.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.345+0000 7f8b2bfff640 1 -- 172.21.15.154:0/681911700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b341f57b0 msgr2=0x7f8b341f5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:05.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.345+0000 7f8b337fe640 1 --2- 172.21.15.154:0/681911700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b341f57b0 0x7f8b341f5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:05.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.345+0000 7f8b2bfff640 1 --2- 172.21.15.154:0/681911700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b341f57b0 0x7f8b341f5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:05.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.345+0000 7f8b2bfff640 1 -- 172.21.15.154:0/681911700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b200039c0 msgr2=0x7f8b340b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:05.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.345+0000 7f8b2bfff640 1 --2- 172.21.15.154:0/681911700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b200039c0 0x7f8b340b4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:05.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.345+0000 7f8b2bfff640 1 -- 172.21.15.154:0/681911700 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b340b45c0 con 0x7f8b341553a0 2023-09-23T03:19:05.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.345+0000 7f8b32ffd640 1 --2- 172.21.15.154:0/681911700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b200039c0 0x7f8b340b4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:05.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.345+0000 7f8b2bfff640 1 --2- 172.21.15.154:0/681911700 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b341553a0 0x7f8b34155800 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c00d770 tx=0x7f8b1c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:05.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.345+0000 7f8b2b7fe640 1 -- 172.21.15.154:0/681911700 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c0072d0 con 0x7f8b341553a0 2023-09-23T03:19:05.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.346+0000 7f8b2b7fe640 1 -- 172.21.15.154:0/681911700 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b1c007430 con 0x7f8b341553a0 2023-09-23T03:19:05.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.346+0000 7f8b33fff640 1 -- 172.21.15.154:0/681911700 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b34158eb0 con 0x7f8b341553a0 2023-09-23T03:19:05.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.346+0000 7f8b2b7fe640 1 -- 172.21.15.154:0/681911700 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c0047e0 con 0x7f8b341553a0 2023-09-23T03:19:05.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.346+0000 7f8b33fff640 1 -- 172.21.15.154:0/681911700 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b341583d0 con 0x7f8b341553a0 2023-09-23T03:19:05.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.347+0000 7f8b33fff640 1 -- 172.21.15.154:0/681911700 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b34072390 con 0x7f8b341553a0 2023-09-23T03:19:05.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.348+0000 7f8b2b7fe640 1 -- 172.21.15.154:0/681911700 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f8b1c004940 con 0x7f8b341553a0 2023-09-23T03:19:05.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.349+0000 7f8b2b7fe640 1 --2- 172.21.15.154:0/681911700 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8b1406af30 0x7f8b1406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:05.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.349+0000 7f8b2b7fe640 1 -- 172.21.15.154:0/681911700 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6720+0+0 (secure 0 0 0) 0x7f8b1c089680 con 0x7f8b341553a0 2023-09-23T03:19:05.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.349+0000 7f8b32ffd640 1 --2- 172.21.15.154:0/681911700 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8b1406af30 0x7f8b1406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:05.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.350+0000 7f8b32ffd640 1 --2- 172.21.15.154:0/681911700 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8b1406af30 0x7f8b1406d3f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8b24033000 tx=0x7f8b24009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:05.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.353+0000 7f8b2b7fe640 1 -- 172.21.15.154:0/681911700 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f8b1c057d70 con 0x7f8b341553a0 2023-09-23T03:19:05.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.543+0000 7f8b33fff640 1 -- 172.21.15.154:0/681911700 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8b3404e920 con 0x7f8b341553a0 2023-09-23T03:19:05.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.543+0000 7f8b2b7fe640 1 -- 172.21.15.154:0/681911700 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v49) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8b1c057900 con 0x7f8b341553a0 2023-09-23T03:19:05.545 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:19:05.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.546+0000 7f8b33fff640 1 -- 172.21.15.154:0/681911700 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8b1406af30 msgr2=0x7f8b1406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:05.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.546+0000 7f8b33fff640 1 --2- 172.21.15.154:0/681911700 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8b1406af30 0x7f8b1406d3f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8b24033000 tx=0x7f8b24009030 comp rx=0 tx=0).stop 2023-09-23T03:19:05.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.546+0000 7f8b33fff640 1 -- 172.21.15.154:0/681911700 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b341553a0 msgr2=0x7f8b34155800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:05.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.546+0000 7f8b33fff640 1 --2- 172.21.15.154:0/681911700 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b341553a0 0x7f8b34155800 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c00d770 tx=0x7f8b1c00dc30 comp rx=0 tx=0).stop 2023-09-23T03:19:05.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.546+0000 7f8b33fff640 1 -- 172.21.15.154:0/681911700 shutdown_connections 2023-09-23T03:19:05.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.546+0000 7f8b33fff640 1 --2- 172.21.15.154:0/681911700 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8b1406af30 0x7f8b1406d3f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:05.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.546+0000 7f8b33fff640 1 --2- 172.21.15.154:0/681911700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8b341f57b0 0x7f8b341f5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:05.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.546+0000 7f8b33fff640 1 --2- 172.21.15.154:0/681911700 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b341553a0 0x7f8b34155800 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:05.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.546+0000 7f8b33fff640 1 --2- 172.21.15.154:0/681911700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8b200039c0 0x7f8b340b4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:05.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.546+0000 7f8b33fff640 1 -- 172.21.15.154:0/681911700 >> 172.21.15.154:0/681911700 conn(0x7f8b340cb630 msgr2=0x7f8b340c3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:05.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.546+0000 7f8b33fff640 1 -- 172.21.15.154:0/681911700 shutdown_connections 2023-09-23T03:19:05.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.546+0000 7f8b33fff640 1 -- 172.21.15.154:0/681911700 wait complete. 2023-09-23T03:19:05.565 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:19:05.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.784+0000 7fdfea8cf640 1 Processor -- start 2023-09-23T03:19:05.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.784+0000 7fdfea8cf640 1 -- start start 2023-09-23T03:19:05.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.786+0000 7fdfea8cf640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdfe40cf470 0x7fdfe40d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:05.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.786+0000 7fdfea8cf640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfe415cdd0 con 0x7fdfe415c9f0 2023-09-23T03:19:05.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.786+0000 7fdfea8cf640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfe40ce4c0 con 0x7fdfe40cf010 2023-09-23T03:19:05.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.786+0000 7fdfea8cf640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfe415dee0 con 0x7fdfe40cf470 2023-09-23T03:19:05.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.786+0000 7fdfe3fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fdfe415c9f0 0x7fdfe415d7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:41580/0 (socket says 172.21.15.154:41580) 2023-09-23T03:19:05.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.786+0000 7fdfe3fff640 1 -- 172.21.15.154:0/500990720 learned_addr learned my addr 172.21.15.154:0/500990720 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:19:05.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.786+0000 7fdfe8e6c640 1 --2- 172.21.15.154:0/500990720 >> v2:172.21.15.190:3300/0 conn(0x7fdfe40cf470 0x7fdfe40d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:05.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.786+0000 7fdfe2ffd640 1 -- 172.21.15.154:0/500990720 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1137692973 0 0) 0x7fdfe415cdd0 con 0x7fdfe415c9f0 2023-09-23T03:19:05.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.786+0000 7fdfe2ffd640 1 -- 172.21.15.154:0/500990720 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfd0003520 con 0x7fdfe415c9f0 2023-09-23T03:19:05.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.787+0000 7fdfe2ffd640 1 -- 172.21.15.154:0/500990720 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 93516295 0 0) 0x7fdfd0003520 con 0x7fdfe415c9f0 2023-09-23T03:19:05.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.787+0000 7fdfe2ffd640 1 -- 172.21.15.154:0/500990720 >> v2:172.21.15.190:3300/0 conn(0x7fdfe40cf470 msgr2=0x7fdfe40d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:05.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.787+0000 7fdfe2ffd640 1 --2- 172.21.15.154:0/500990720 >> v2:172.21.15.190:3300/0 conn(0x7fdfe40cf470 0x7fdfe40d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:05.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.787+0000 7fdfe2ffd640 1 -- 172.21.15.154:0/500990720 >> v1:172.21.15.190:6789/0 conn(0x7fdfe40cf010 legacy=0x7fdfe40d04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:05.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.787+0000 7fdfe2ffd640 1 -- 172.21.15.154:0/500990720 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe40cd8b0 con 0x7fdfe415c9f0 2023-09-23T03:19:05.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.787+0000 7fdfe2ffd640 1 -- 172.21.15.154:0/500990720 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fdfd40017f0 con 0x7fdfe415c9f0 2023-09-23T03:19:05.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.787+0000 7fdfe2ffd640 1 -- 172.21.15.154:0/500990720 >> v1:172.21.15.154:6789/0 conn(0x7fdfe415c9f0 legacy=0x7fdfe415d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:05.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.787+0000 7fdfe2ffd640 1 --2- 172.21.15.154:0/500990720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdfd0003970 0x7fdfd0003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:05.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.787+0000 7fdfe2ffd640 1 --2- 172.21.15.154:0/500990720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfd00047c0 0x7fdfd0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:05.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.787+0000 7fdfe2ffd640 1 --2- 172.21.15.154:0/500990720 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfd0005180 0x7fdfd000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:05.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.787+0000 7fdfe2ffd640 1 -- 172.21.15.154:0/500990720 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe415dee0 con 0x7fdfd00047c0 2023-09-23T03:19:05.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.787+0000 7fdfe2ffd640 1 -- 172.21.15.154:0/500990720 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd000c000 con 0x7fdfd0003970 2023-09-23T03:19:05.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.787+0000 7fdfe2ffd640 1 -- 172.21.15.154:0/500990720 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd000c1c0 con 0x7fdfd0005180 2023-09-23T03:19:05.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.787+0000 7fdfe37fe640 1 --2- 172.21.15.154:0/500990720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfd00047c0 0x7fdfd0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:05.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.788+0000 7fdfe3fff640 1 --2- 172.21.15.154:0/500990720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdfd0003970 0x7fdfd0003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:05.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.788+0000 7fdfe8e6c640 1 --2- 172.21.15.154:0/500990720 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfd0005180 0x7fdfd000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:05.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.788+0000 7fdfe37fe640 1 -- 172.21.15.154:0/500990720 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfd0005180 msgr2=0x7fdfd000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:05.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.788+0000 7fdfe37fe640 1 --2- 172.21.15.154:0/500990720 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfd0005180 0x7fdfd000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:05.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.788+0000 7fdfe37fe640 1 -- 172.21.15.154:0/500990720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdfd0003970 msgr2=0x7fdfd0003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:05.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.788+0000 7fdfe37fe640 1 --2- 172.21.15.154:0/500990720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdfd0003970 0x7fdfd0003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:05.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.788+0000 7fdfe37fe640 1 -- 172.21.15.154:0/500990720 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd0003520 con 0x7fdfd00047c0 2023-09-23T03:19:05.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.788+0000 7fdfe8e6c640 1 -- 172.21.15.154:0/500990720 reap_dead start 2023-09-23T03:19:05.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.789+0000 7fdfe37fe640 1 --2- 172.21.15.154:0/500990720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfd00047c0 0x7fdfd0004c40 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc0044b0 tx=0x7fdfd00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65ff556da4876b05 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:05.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.789+0000 7fdfe2ffd640 1 -- 172.21.15.154:0/500990720 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc030400 con 0x7fdfd00047c0 2023-09-23T03:19:05.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.789+0000 7fdfe2ffd640 1 -- 172.21.15.154:0/500990720 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfcc030560 con 0x7fdfd00047c0 2023-09-23T03:19:05.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.789+0000 7fdfe2ffd640 1 -- 172.21.15.154:0/500990720 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc030950 con 0x7fdfd00047c0 2023-09-23T03:19:05.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.790+0000 7fdfea8cf640 1 -- 172.21.15.154:0/500990720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfd00047c0 msgr2=0x7fdfd0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:05.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.790+0000 7fdfea8cf640 1 --2- 172.21.15.154:0/500990720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfd00047c0 0x7fdfd0004c40 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc0044b0 tx=0x7fdfd00057f0 comp rx=0 tx=0).stop 2023-09-23T03:19:05.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.790+0000 7fdfea8cf640 1 -- 172.21.15.154:0/500990720 shutdown_connections 2023-09-23T03:19:05.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.790+0000 7fdfea8cf640 1 --2- 172.21.15.154:0/500990720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfd00047c0 0x7fdfd0004c40 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:05.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.790+0000 7fdfea8cf640 1 -- 172.21.15.154:0/500990720 >> 172.21.15.154:0/500990720 conn(0x7fdfe409bc10 msgr2=0x7fdfe40d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:05.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.790+0000 7fdfea8cf640 1 -- 172.21.15.154:0/500990720 shutdown_connections 2023-09-23T03:19:05.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.790+0000 7fdfea8cf640 1 -- 172.21.15.154:0/500990720 wait complete. 2023-09-23T03:19:05.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.792+0000 7fdfea8cf640 1 Processor -- start 2023-09-23T03:19:05.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.792+0000 7fdfea8cf640 1 -- start start 2023-09-23T03:19:05.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.793+0000 7fdfea8cf640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfd00047c0 0x7fdfe406b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:05.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.793+0000 7fdfea8cf640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdfe40b4dc0 0x7fdfe40b5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:05.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.793+0000 7fdfea8cf640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfe41f58b0 0x7fdfe41f5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:05.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.793+0000 7fdfea8cf640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe41f2d00 con 0x7fdfe41f58b0 2023-09-23T03:19:05.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.793+0000 7fdfea8cf640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe41f2170 con 0x7fdfe40b4dc0 2023-09-23T03:19:05.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.793+0000 7fdfea8cf640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe40b5780 con 0x7fdfd00047c0 2023-09-23T03:19:05.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.793+0000 7fdfe8e6c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfe41f58b0 0x7fdfe41f5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:05.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.793+0000 7fdfe8e6c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfe41f58b0 0x7fdfe41f5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54336/0 (socket says 172.21.15.154:54336) 2023-09-23T03:19:05.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.793+0000 7fdfe8e6c640 1 -- 172.21.15.154:0/4061130027 learned_addr learned my addr 172.21.15.154:0/4061130027 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:05.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.794+0000 7fdfe37fe640 1 --2- 172.21.15.154:0/4061130027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdfe40b4dc0 0x7fdfe40b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:05.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.794+0000 7fdfe3fff640 1 --2- 172.21.15.154:0/4061130027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfd00047c0 0x7fdfe406b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:05.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.794+0000 7fdfe8e6c640 1 -- 172.21.15.154:0/4061130027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfd00047c0 msgr2=0x7fdfe406b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:05.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.794+0000 7fdfe8e6c640 1 --2- 172.21.15.154:0/4061130027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfd00047c0 0x7fdfe406b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:05.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.794+0000 7fdfe8e6c640 1 -- 172.21.15.154:0/4061130027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdfe40b4dc0 msgr2=0x7fdfe40b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:05.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.794+0000 7fdfe8e6c640 1 --2- 172.21.15.154:0/4061130027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdfe40b4dc0 0x7fdfe40b5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:05.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.794+0000 7fdfe8e6c640 1 -- 172.21.15.154:0/4061130027 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe41f6670 con 0x7fdfe41f58b0 2023-09-23T03:19:05.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.794+0000 7fdfe8e6c640 1 --2- 172.21.15.154:0/4061130027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfe41f58b0 0x7fdfe41f5d60 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4160420 tx=0x7fdfdc010b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:05.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.794+0000 7fdfe0ff9640 1 -- 172.21.15.154:0/4061130027 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc00e8a0 con 0x7fdfe41f58b0 2023-09-23T03:19:05.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.795+0000 7fdfe0ff9640 1 -- 172.21.15.154:0/4061130027 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfdc00ea20 con 0x7fdfe41f58b0 2023-09-23T03:19:05.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.795+0000 7fdfe0ff9640 1 -- 172.21.15.154:0/4061130027 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc00edd0 con 0x7fdfe41f58b0 2023-09-23T03:19:05.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.795+0000 7fdfea8cf640 1 -- 172.21.15.154:0/4061130027 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfe41f2360 con 0x7fdfe41f58b0 2023-09-23T03:19:05.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.795+0000 7fdfea8cf640 1 -- 172.21.15.154:0/4061130027 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfe41f2920 con 0x7fdfe41f58b0 2023-09-23T03:19:05.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.796+0000 7fdfea8cf640 1 -- 172.21.15.154:0/4061130027 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfe41f2e40 con 0x7fdfe41f58b0 2023-09-23T03:19:05.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.797+0000 7fdfe0ff9640 1 -- 172.21.15.154:0/4061130027 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fdfdc020410 con 0x7fdfe41f58b0 2023-09-23T03:19:05.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.798+0000 7fdfe0ff9640 1 --2- 172.21.15.154:0/4061130027 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdfc406af60 0x7fdfc406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:05.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.798+0000 7fdfe0ff9640 1 -- 172.21.15.154:0/4061130027 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6720+0+0 (secure 0 0 0) 0x7fdfdc089410 con 0x7fdfe41f58b0 2023-09-23T03:19:05.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.798+0000 7fdfe3fff640 1 --2- 172.21.15.154:0/4061130027 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdfc406af60 0x7fdfc406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:05.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.799+0000 7fdfe3fff640 1 --2- 172.21.15.154:0/4061130027 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdfc406af60 0x7fdfc406d420 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4001380 tx=0x7fdfd4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:05.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.802+0000 7fdfe0ff9640 1 -- 172.21.15.154:0/4061130027 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fdfe41f2e40 con 0x7fdfe41f58b0 2023-09-23T03:19:05.992 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.990+0000 7fdfea8cf640 1 -- 172.21.15.154:0/4061130027 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdfe41f2e40 con 0x7fdfe41f58b0 2023-09-23T03:19:05.992 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.991+0000 7fdfe0ff9640 1 -- 172.21.15.154:0/4061130027 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v49) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdfe41f2e40 con 0x7fdfe41f58b0 2023-09-23T03:19:05.993 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:19:05.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.994+0000 7fdfea8cf640 1 -- 172.21.15.154:0/4061130027 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdfc406af60 msgr2=0x7fdfc406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:05.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.994+0000 7fdfea8cf640 1 --2- 172.21.15.154:0/4061130027 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdfc406af60 0x7fdfc406d420 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4001380 tx=0x7fdfd4009040 comp rx=0 tx=0).stop 2023-09-23T03:19:05.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.994+0000 7fdfea8cf640 1 -- 172.21.15.154:0/4061130027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfe41f58b0 msgr2=0x7fdfe41f5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:05.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.994+0000 7fdfea8cf640 1 --2- 172.21.15.154:0/4061130027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfe41f58b0 0x7fdfe41f5d60 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4160420 tx=0x7fdfdc010b70 comp rx=0 tx=0).stop 2023-09-23T03:19:05.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.994+0000 7fdfea8cf640 1 -- 172.21.15.154:0/4061130027 shutdown_connections 2023-09-23T03:19:05.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.994+0000 7fdfea8cf640 1 --2- 172.21.15.154:0/4061130027 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdfc406af60 0x7fdfc406d420 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:05.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.994+0000 7fdfea8cf640 1 --2- 172.21.15.154:0/4061130027 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfe41f58b0 0x7fdfe41f5d60 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:05.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.994+0000 7fdfea8cf640 1 --2- 172.21.15.154:0/4061130027 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdfe40b4dc0 0x7fdfe40b5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:05.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.994+0000 7fdfea8cf640 1 --2- 172.21.15.154:0/4061130027 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfd00047c0 0x7fdfe406b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:05.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.994+0000 7fdfea8cf640 1 -- 172.21.15.154:0/4061130027 >> 172.21.15.154:0/4061130027 conn(0x7fdfe409bc10 msgr2=0x7fdfe415bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:05.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.994+0000 7fdfea8cf640 1 -- 172.21.15.154:0/4061130027 shutdown_connections 2023-09-23T03:19:05.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:05.994+0000 7fdfea8cf640 1 -- 172.21.15.154:0/4061130027 wait complete. 2023-09-23T03:19:06.014 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-09-23T03:19:06.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.230+0000 7fe966d16640 1 Processor -- start 2023-09-23T03:19:06.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.231+0000 7fe966d16640 1 -- start start 2023-09-23T03:19:06.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.232+0000 7fe966d16640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe96015abe0 0x7fe9600cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:06.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.232+0000 7fe966d16640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9600cc9b0 0x7fe9600b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:06.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.232+0000 7fe966d16640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe9600cce10 0x7fe9600b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:06.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.232+0000 7fe966d16640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9600ce510 con 0x7fe9600cce10 2023-09-23T03:19:06.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.232+0000 7fe966d16640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9600cb710 con 0x7fe96015abe0 2023-09-23T03:19:06.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.232+0000 7fe966d16640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9600cd390 con 0x7fe9600cc9b0 2023-09-23T03:19:06.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.233+0000 7fe9652b3640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe9600cce10 0x7fe9600b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:06.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.233+0000 7fe9652b3640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe9600cce10 0x7fe9600b26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54340/0 (socket says 172.21.15.154:54340) 2023-09-23T03:19:06.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.233+0000 7fe9652b3640 1 -- 172.21.15.154:0/3519811517 learned_addr learned my addr 172.21.15.154:0/3519811517 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:06.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.233+0000 7fe95ffff640 1 --2- 172.21.15.154:0/3519811517 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9600cc9b0 0x7fe9600b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:06.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.233+0000 7fe964ab2640 1 --2- 172.21.15.154:0/3519811517 >> v2:172.21.15.190:3300/0 conn(0x7fe96015abe0 0x7fe9600cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:06.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.233+0000 7fe9652b3640 1 -- 172.21.15.154:0/3519811517 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9600cc9b0 msgr2=0x7fe9600b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:06.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.233+0000 7fe9652b3640 1 --2- 172.21.15.154:0/3519811517 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9600cc9b0 0x7fe9600b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:06.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.233+0000 7fe9652b3640 1 -- 172.21.15.154:0/3519811517 >> v2:172.21.15.190:3300/0 conn(0x7fe96015abe0 msgr2=0x7fe9600cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:06.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.233+0000 7fe9652b3640 1 --2- 172.21.15.154:0/3519811517 >> v2:172.21.15.190:3300/0 conn(0x7fe96015abe0 0x7fe9600cb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:06.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.233+0000 7fe9652b3640 1 -- 172.21.15.154:0/3519811517 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9600cd670 con 0x7fe9600cce10 2023-09-23T03:19:06.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.234+0000 7fe9652b3640 1 --2- 172.21.15.154:0/3519811517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9600cce10 0x7fe9600b26f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe960070cc0 tx=0x7fe95802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32ff8cfc03046c74 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:06.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.235+0000 7fe95f7fe640 1 -- 172.21.15.154:0/3519811517 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95802c9e0 con 0x7fe9600cce10 2023-09-23T03:19:06.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.235+0000 7fe95f7fe640 1 -- 172.21.15.154:0/3519811517 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95802cb40 con 0x7fe9600cce10 2023-09-23T03:19:06.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.235+0000 7fe95f7fe640 1 -- 172.21.15.154:0/3519811517 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9580343f0 con 0x7fe9600cce10 2023-09-23T03:19:06.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.235+0000 7fe966d16640 1 -- 172.21.15.154:0/3519811517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9600cce10 msgr2=0x7fe9600b26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:06.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.235+0000 7fe966d16640 1 --2- 172.21.15.154:0/3519811517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9600cce10 0x7fe9600b26f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe960070cc0 tx=0x7fe95802be40 comp rx=0 tx=0).stop 2023-09-23T03:19:06.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.236+0000 7fe966d16640 1 -- 172.21.15.154:0/3519811517 shutdown_connections 2023-09-23T03:19:06.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.236+0000 7fe966d16640 1 --2- 172.21.15.154:0/3519811517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9600cce10 0x7fe9600b26f0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:06.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.236+0000 7fe966d16640 1 --2- 172.21.15.154:0/3519811517 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9600cc9b0 0x7fe9600b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:06.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.236+0000 7fe966d16640 1 --2- 172.21.15.154:0/3519811517 >> v2:172.21.15.190:3300/0 conn(0x7fe96015abe0 0x7fe9600cb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:06.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.236+0000 7fe966d16640 1 -- 172.21.15.154:0/3519811517 >> 172.21.15.154:0/3519811517 conn(0x7fe96006fc40 msgr2=0x7fe960070020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:06.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.236+0000 7fe966d16640 1 -- 172.21.15.154:0/3519811517 shutdown_connections 2023-09-23T03:19:06.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.236+0000 7fe966d16640 1 -- 172.21.15.154:0/3519811517 wait complete. 2023-09-23T03:19:06.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.238+0000 7fe966d16640 1 Processor -- start 2023-09-23T03:19:06.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.238+0000 7fe966d16640 1 -- start start 2023-09-23T03:19:06.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.239+0000 7fe966d16640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9600cc9b0 0x7fe960159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:06.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.239+0000 7fe966d16640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9600cce10 0x7fe960160f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:06.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.239+0000 7fe966d16640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe96015abe0 0x7fe9601614b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:06.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.239+0000 7fe966d16640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe96015a210 con 0x7fe96015abe0 2023-09-23T03:19:06.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.239+0000 7fe966d16640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9600b4890 con 0x7fe9600cce10 2023-09-23T03:19:06.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.239+0000 7fe966d16640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9600b4b70 con 0x7fe9600cc9b0 2023-09-23T03:19:06.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.239+0000 7fe9652b3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe96015abe0 0x7fe9601614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:06.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.239+0000 7fe9652b3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe96015abe0 0x7fe9601614b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54352/0 (socket says 172.21.15.154:54352) 2023-09-23T03:19:06.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.239+0000 7fe9652b3640 1 -- 172.21.15.154:0/2794541537 learned_addr learned my addr 172.21.15.154:0/2794541537 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:06.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.240+0000 7fe964ab2640 1 --2- 172.21.15.154:0/2794541537 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9600cc9b0 0x7fe960159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:06.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.240+0000 7fe9652b3640 1 -- 172.21.15.154:0/2794541537 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9600cc9b0 msgr2=0x7fe960159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:06.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.240+0000 7fe9652b3640 1 --2- 172.21.15.154:0/2794541537 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9600cc9b0 0x7fe960159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:06.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.240+0000 7fe9652b3640 1 -- 172.21.15.154:0/2794541537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9600cce10 msgr2=0x7fe960160f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:06.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.240+0000 7fe9652b3640 1 --2- 172.21.15.154:0/2794541537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9600cce10 0x7fe960160f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:06.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.240+0000 7fe9652b3640 1 -- 172.21.15.154:0/2794541537 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe958009b30 con 0x7fe96015abe0 2023-09-23T03:19:06.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.240+0000 7fe9652b3640 1 --2- 172.21.15.154:0/2794541537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe96015abe0 0x7fe9601614b0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fe958036040 tx=0x7fe958004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:06.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.241+0000 7fe95d7fa640 1 -- 172.21.15.154:0/2794541537 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe958009510 con 0x7fe96015abe0 2023-09-23T03:19:06.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.241+0000 7fe95d7fa640 1 -- 172.21.15.154:0/2794541537 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95803e450 con 0x7fe96015abe0 2023-09-23T03:19:06.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.241+0000 7fe95d7fa640 1 -- 172.21.15.154:0/2794541537 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95803e800 con 0x7fe96015abe0 2023-09-23T03:19:06.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.241+0000 7fe966d16640 1 -- 172.21.15.154:0/2794541537 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9600c8bf0 con 0x7fe96015abe0 2023-09-23T03:19:06.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.241+0000 7fe966d16640 1 -- 172.21.15.154:0/2794541537 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9600c91b0 con 0x7fe96015abe0 2023-09-23T03:19:06.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.242+0000 7fe966d16640 1 -- 172.21.15.154:0/2794541537 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9600c93a0 con 0x7fe96015abe0 2023-09-23T03:19:06.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.243+0000 7fe95d7fa640 1 -- 172.21.15.154:0/2794541537 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fe95803e960 con 0x7fe96015abe0 2023-09-23T03:19:06.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.244+0000 7fe95d7fa640 1 --2- 172.21.15.154:0/2794541537 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe94406af30 0x7fe94406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:06.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.244+0000 7fe95d7fa640 1 -- 172.21.15.154:0/2794541537 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6720+0+0 (secure 0 0 0) 0x7fe9580ac880 con 0x7fe96015abe0 2023-09-23T03:19:06.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.244+0000 7fe964ab2640 1 --2- 172.21.15.154:0/2794541537 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe94406af30 0x7fe94406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:06.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.245+0000 7fe964ab2640 1 --2- 172.21.15.154:0/2794541537 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe94406af30 0x7fe94406d3f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe95400b970 tx=0x7fe954008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:06.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.249+0000 7fe95d7fa640 1 -- 172.21.15.154:0/2794541537 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fe9580774d0 con 0x7fe96015abe0 2023-09-23T03:19:06.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.436+0000 7fe966d16640 1 -- 172.21.15.154:0/2794541537 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7fe96004e920 con 0x7fe96015abe0 2023-09-23T03:19:06.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.437+0000 7fe95d7fa640 1 -- 172.21.15.154:0/2794541537 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v49) v1 ==== 106+0+11 (secure 0 0 0) 0x7fe95807aaf0 con 0x7fe96015abe0 2023-09-23T03:19:06.438 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:19:06.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.439+0000 7fe966d16640 1 -- 172.21.15.154:0/2794541537 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe94406af30 msgr2=0x7fe94406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:06.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.439+0000 7fe966d16640 1 --2- 172.21.15.154:0/2794541537 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe94406af30 0x7fe94406d3f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe95400b970 tx=0x7fe954008040 comp rx=0 tx=0).stop 2023-09-23T03:19:06.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.439+0000 7fe966d16640 1 -- 172.21.15.154:0/2794541537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe96015abe0 msgr2=0x7fe9601614b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:06.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.439+0000 7fe966d16640 1 --2- 172.21.15.154:0/2794541537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe96015abe0 0x7fe9601614b0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fe958036040 tx=0x7fe958004a40 comp rx=0 tx=0).stop 2023-09-23T03:19:06.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.439+0000 7fe966d16640 1 -- 172.21.15.154:0/2794541537 shutdown_connections 2023-09-23T03:19:06.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.439+0000 7fe966d16640 1 --2- 172.21.15.154:0/2794541537 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe94406af30 0x7fe94406d3f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:06.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.439+0000 7fe966d16640 1 --2- 172.21.15.154:0/2794541537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe96015abe0 0x7fe9601614b0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:06.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.439+0000 7fe966d16640 1 --2- 172.21.15.154:0/2794541537 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9600cce10 0x7fe960160f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:06.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.439+0000 7fe966d16640 1 --2- 172.21.15.154:0/2794541537 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9600cc9b0 0x7fe960159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:06.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.439+0000 7fe966d16640 1 -- 172.21.15.154:0/2794541537 >> 172.21.15.154:0/2794541537 conn(0x7fe96006fc40 msgr2=0x7fe9600b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:06.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.439+0000 7fe966d16640 1 -- 172.21.15.154:0/2794541537 shutdown_connections 2023-09-23T03:19:06.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.439+0000 7fe966d16640 1 -- 172.21.15.154:0/2794541537 wait complete. 2023-09-23T03:19:06.458 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:19:06.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.668+0000 7f9647cd4640 1 Processor -- start 2023-09-23T03:19:06.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.669+0000 7f9647cd4640 1 -- start start 2023-09-23T03:19:06.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.670+0000 7f9647cd4640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f964015ad20 0x7f96400cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:06.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.670+0000 7f9647cd4640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f964015f880 con 0x7f96400cff90 2023-09-23T03:19:06.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.670+0000 7f9647cd4640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96400b2ff0 con 0x7f96400cd160 2023-09-23T03:19:06.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.670+0000 7f9647cd4640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f964015ed00 con 0x7f964015ad20 2023-09-23T03:19:06.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.670+0000 7f9646271640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f96400cff90 0x7f96400b28e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:41584/0 (socket says 172.21.15.154:41584) 2023-09-23T03:19:06.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.671+0000 7f9646271640 1 -- 172.21.15.154:0/996207003 learned_addr learned my addr 172.21.15.154:0/996207003 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:19:06.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.671+0000 7f9645a70640 1 --2- 172.21.15.154:0/996207003 >> v2:172.21.15.190:3300/0 conn(0x7f964015ad20 0x7f96400cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:06.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.671+0000 7f9644a6e640 1 -- 172.21.15.154:0/996207003 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1048802614 0 0) 0x7f964015f880 con 0x7f96400cff90 2023-09-23T03:19:06.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.671+0000 7f9644a6e640 1 -- 172.21.15.154:0/996207003 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f962c003520 con 0x7f96400cff90 2023-09-23T03:19:06.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.671+0000 7f9644a6e640 1 -- 172.21.15.154:0/996207003 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 429535006 0 0) 0x7f962c003520 con 0x7f96400cff90 2023-09-23T03:19:06.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.671+0000 7f9644a6e640 1 -- 172.21.15.154:0/996207003 >> v2:172.21.15.190:3300/0 conn(0x7f964015ad20 msgr2=0x7f96400cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:06.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.671+0000 7f9644a6e640 1 --2- 172.21.15.154:0/996207003 >> v2:172.21.15.190:3300/0 conn(0x7f964015ad20 0x7f96400cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:06.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.671+0000 7f9644a6e640 1 -- 172.21.15.154:0/996207003 >> v1:172.21.15.190:6789/0 conn(0x7f96400cd160 legacy=0x7f96400b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:06.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.671+0000 7f9644a6e640 1 -- 172.21.15.154:0/996207003 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f964015efe0 con 0x7f96400cff90 2023-09-23T03:19:06.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.671+0000 7f9644a6e640 1 -- 172.21.15.154:0/996207003 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f96380019f0 con 0x7f96400cff90 2023-09-23T03:19:06.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.671+0000 7f9644a6e640 1 -- 172.21.15.154:0/996207003 >> v1:172.21.15.154:6789/0 conn(0x7f96400cff90 legacy=0x7f96400b28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:06.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.672+0000 7f9644a6e640 1 --2- 172.21.15.154:0/996207003 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f962c003970 0x7f962c003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:06.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.672+0000 7f9644a6e640 1 --2- 172.21.15.154:0/996207003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f962c0047c0 0x7f962c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:06.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.672+0000 7f9644a6e640 1 --2- 172.21.15.154:0/996207003 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f962c005180 0x7f962c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:06.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.672+0000 7f9644a6e640 1 -- 172.21.15.154:0/996207003 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964015ed00 con 0x7f962c003970 2023-09-23T03:19:06.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.672+0000 7f9644a6e640 1 -- 172.21.15.154:0/996207003 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962c00c000 con 0x7f962c0047c0 2023-09-23T03:19:06.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.672+0000 7f9644a6e640 1 -- 172.21.15.154:0/996207003 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f962c00c1c0 con 0x7f962c005180 2023-09-23T03:19:06.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.672+0000 7f9645a70640 1 --2- 172.21.15.154:0/996207003 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f962c003970 0x7f962c003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:06.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.672+0000 7f964526f640 1 --2- 172.21.15.154:0/996207003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f962c0047c0 0x7f962c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:06.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.672+0000 7f9645a70640 1 -- 172.21.15.154:0/996207003 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f962c005180 msgr2=0x7f962c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:06.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.672+0000 7f9645a70640 1 --2- 172.21.15.154:0/996207003 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f962c005180 0x7f962c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:06.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.672+0000 7f9645a70640 1 -- 172.21.15.154:0/996207003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f962c0047c0 msgr2=0x7f962c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:06.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.672+0000 7f9645a70640 1 --2- 172.21.15.154:0/996207003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f962c0047c0 0x7f962c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:06.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.672+0000 7f9645a70640 1 -- 172.21.15.154:0/996207003 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f962c003520 con 0x7f962c003970 2023-09-23T03:19:06.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.672+0000 7f9646271640 1 -- 172.21.15.154:0/996207003 reap_dead start 2023-09-23T03:19:06.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.672+0000 7f964526f640 1 --2- 172.21.15.154:0/996207003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f962c0047c0 0x7f962c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:06.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.673+0000 7f9645a70640 1 --2- 172.21.15.154:0/996207003 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f962c003970 0x7f962c003d70 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f963c009ac0 tx=0x7f962c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7464a2dfa47132bd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:06.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.674+0000 7f9644a6e640 1 -- 172.21.15.154:0/996207003 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963c02c6d0 con 0x7f962c003970 2023-09-23T03:19:06.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.674+0000 7f9644a6e640 1 -- 172.21.15.154:0/996207003 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f963c02c830 con 0x7f962c003970 2023-09-23T03:19:06.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.674+0000 7f9644a6e640 1 -- 172.21.15.154:0/996207003 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963c02cbc0 con 0x7f962c003970 2023-09-23T03:19:06.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.674+0000 7f9647cd4640 1 -- 172.21.15.154:0/996207003 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f962c003970 msgr2=0x7f962c003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:06.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.674+0000 7f9647cd4640 1 --2- 172.21.15.154:0/996207003 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f962c003970 0x7f962c003d70 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f963c009ac0 tx=0x7f962c0057f0 comp rx=0 tx=0).stop 2023-09-23T03:19:06.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.674+0000 7f9647cd4640 1 -- 172.21.15.154:0/996207003 shutdown_connections 2023-09-23T03:19:06.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.674+0000 7f9647cd4640 1 --2- 172.21.15.154:0/996207003 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f962c003970 0x7f962c003d70 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:06.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.674+0000 7f9647cd4640 1 -- 172.21.15.154:0/996207003 >> 172.21.15.154:0/996207003 conn(0x7f964006fc60 msgr2=0x7f9640070040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:06.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.674+0000 7f9647cd4640 1 -- 172.21.15.154:0/996207003 shutdown_connections 2023-09-23T03:19:06.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.674+0000 7f9647cd4640 1 -- 172.21.15.154:0/996207003 wait complete. 2023-09-23T03:19:06.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.676+0000 7f9647cd4640 1 Processor -- start 2023-09-23T03:19:06.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.676+0000 7f9647cd4640 1 -- start start 2023-09-23T03:19:06.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.677+0000 7f9647cd4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f962c003970 0x7f9640155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:06.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.677+0000 7f9647cd4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9640155890 0x7f96400b1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:06.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.677+0000 7f9647cd4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96400b1960 0x7f964015bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:06.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.677+0000 7f9647cd4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96400b1de0 con 0x7f9640155890 2023-09-23T03:19:06.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.677+0000 7f9647cd4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96400c43c0 con 0x7f962c003970 2023-09-23T03:19:06.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.677+0000 7f9647cd4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96400c46a0 con 0x7f96400b1960 2023-09-23T03:19:06.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.677+0000 7f964526f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9640155890 0x7f96400b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:06.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.677+0000 7f964526f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9640155890 0x7f96400b1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54380/0 (socket says 172.21.15.154:54380) 2023-09-23T03:19:06.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.677+0000 7f964526f640 1 -- 172.21.15.154:0/4168548160 learned_addr learned my addr 172.21.15.154:0/4168548160 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:06.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.678+0000 7f964526f640 1 -- 172.21.15.154:0/4168548160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96400b1960 msgr2=0x7f964015bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:06.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.678+0000 7f964526f640 1 --2- 172.21.15.154:0/4168548160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96400b1960 0x7f964015bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:06.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.678+0000 7f964526f640 1 -- 172.21.15.154:0/4168548160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f962c003970 msgr2=0x7f9640155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:06.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.678+0000 7f964526f640 1 --2- 172.21.15.154:0/4168548160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f962c003970 0x7f9640155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:06.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.678+0000 7f964526f640 1 -- 172.21.15.154:0/4168548160 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96400c4980 con 0x7f9640155890 2023-09-23T03:19:06.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.678+0000 7f964526f640 1 --2- 172.21.15.154:0/4168548160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9640155890 0x7f96400b1420 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f963000d430 tx=0x7f963000d8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:06.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.678+0000 7f96367fc640 1 -- 172.21.15.154:0/4168548160 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9630007020 con 0x7f9640155890 2023-09-23T03:19:06.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.678+0000 7f96367fc640 1 -- 172.21.15.154:0/4168548160 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9630007180 con 0x7f9640155890 2023-09-23T03:19:06.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.678+0000 7f96367fc640 1 -- 172.21.15.154:0/4168548160 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9630007530 con 0x7f9640155890 2023-09-23T03:19:06.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.678+0000 7f9647cd4640 1 -- 172.21.15.154:0/4168548160 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96400c4c60 con 0x7f9640155890 2023-09-23T03:19:06.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.679+0000 7f9647cd4640 1 -- 172.21.15.154:0/4168548160 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f964015c2f0 con 0x7f9640155890 2023-09-23T03:19:06.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.680+0000 7f9647cd4640 1 -- 172.21.15.154:0/4168548160 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96400c5180 con 0x7f9640155890 2023-09-23T03:19:06.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.681+0000 7f96367fc640 1 -- 172.21.15.154:0/4168548160 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f96300067d0 con 0x7f9640155890 2023-09-23T03:19:06.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.682+0000 7f96367fc640 1 --2- 172.21.15.154:0/4168548160 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f962406af60 0x7f962406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:06.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.682+0000 7f96367fc640 1 -- 172.21.15.154:0/4168548160 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6720+0+0 (secure 0 0 0) 0x7f9630089a60 con 0x7f9640155890 2023-09-23T03:19:06.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.682+0000 7f9645a70640 1 --2- 172.21.15.154:0/4168548160 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f962406af60 0x7f962406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:06.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.683+0000 7f9645a70640 1 --2- 172.21.15.154:0/4168548160 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f962406af60 0x7f962406d420 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f963c036000 tx=0x7f963c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:06.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.686+0000 7f96367fc640 1 -- 172.21.15.154:0/4168548160 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f96400c5180 con 0x7f9640155890 2023-09-23T03:19:06.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.911+0000 7f9647cd4640 1 -- 172.21.15.154:0/4168548160 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96400c5180 con 0x7f9640155890 2023-09-23T03:19:06.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.912+0000 7f96367fc640 1 -- 172.21.15.154:0/4168548160 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4928 (secure 0 0 0) 0x7f96400c5180 con 0x7f9640155890 2023-09-23T03:19:06.913 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:19:06.914 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":32,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5215,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3388818833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3388818833},{"type":"v1","addr":"172.21.15.190:6835","nonce":3388818833}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5991,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1964552060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1964552060},{"type":"v1","addr":"172.21.15.154:6837","nonce":1964552060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5994,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/1736999297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1736999297},{"type":"v1","addr":"172.21.15.190:6834","nonce":1736999297}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:17:56.221314+0000","modified":"2023-09-23T03:18:05.544508+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5988},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5988":{"gid":5988,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":4,"addr":"172.21.15.154:6836/4278807008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4278807008},{"type":"v1","addr":"172.21.15.154:6836","nonce":4278807008}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-09-23T03:19:06.914 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 32 2023-09-23T03:19:06.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.914+0000 7f9647cd4640 1 -- 172.21.15.154:0/4168548160 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f962406af60 msgr2=0x7f962406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:06.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.914+0000 7f9647cd4640 1 --2- 172.21.15.154:0/4168548160 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f962406af60 0x7f962406d420 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f963c036000 tx=0x7f963c02e000 comp rx=0 tx=0).stop 2023-09-23T03:19:06.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.914+0000 7f9647cd4640 1 -- 172.21.15.154:0/4168548160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9640155890 msgr2=0x7f96400b1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:06.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.914+0000 7f9647cd4640 1 --2- 172.21.15.154:0/4168548160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9640155890 0x7f96400b1420 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f963000d430 tx=0x7f963000d8f0 comp rx=0 tx=0).stop 2023-09-23T03:19:06.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.915+0000 7f9647cd4640 1 -- 172.21.15.154:0/4168548160 shutdown_connections 2023-09-23T03:19:06.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.915+0000 7f9647cd4640 1 --2- 172.21.15.154:0/4168548160 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f962406af60 0x7f962406d420 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:06.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.915+0000 7f9647cd4640 1 --2- 172.21.15.154:0/4168548160 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f96400b1960 0x7f964015bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:06.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.915+0000 7f9647cd4640 1 --2- 172.21.15.154:0/4168548160 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9640155890 0x7f96400b1420 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:06.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.915+0000 7f9647cd4640 1 --2- 172.21.15.154:0/4168548160 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f962c003970 0x7f9640155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:06.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.915+0000 7f9647cd4640 1 -- 172.21.15.154:0/4168548160 >> 172.21.15.154:0/4168548160 conn(0x7f964006fc60 msgr2=0x7f96400cad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:06.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.915+0000 7f9647cd4640 1 -- 172.21.15.154:0/4168548160 shutdown_connections 2023-09-23T03:19:06.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:06.915+0000 7f9647cd4640 1 -- 172.21.15.154:0/4168548160 wait complete. 2023-09-23T03:19:06.934 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:19:07.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.146+0000 7fb716968640 1 Processor -- start 2023-09-23T03:19:07.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.147+0000 7fb716968640 1 -- start start 2023-09-23T03:19:07.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.148+0000 7fb716968640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb710161010 0x7fb7100cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:07.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.148+0000 7fb716968640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7100c14e0 0x7fb7100c1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:07.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.148+0000 7fb716968640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb71006c1d0 0x7fb71006c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:07.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.148+0000 7fb716968640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7100c1ea0 con 0x7fb710161010 2023-09-23T03:19:07.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.148+0000 7fb716968640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7100cb160 con 0x7fb71006c1d0 2023-09-23T03:19:07.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.148+0000 7fb716968640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb71006cb60 con 0x7fb7100c14e0 2023-09-23T03:19:07.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.149+0000 7fb70ffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb710161010 0x7fb7100cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:07.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.149+0000 7fb70ffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb710161010 0x7fb7100cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54382/0 (socket says 172.21.15.154:54382) 2023-09-23T03:19:07.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.149+0000 7fb70ffff640 1 -- 172.21.15.154:0/2704969258 learned_addr learned my addr 172.21.15.154:0/2704969258 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:07.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.149+0000 7fb70ffff640 1 -- 172.21.15.154:0/2704969258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7100c14e0 msgr2=0x7fb7100c1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:07.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.149+0000 7fb70ffff640 1 --2- 172.21.15.154:0/2704969258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7100c14e0 0x7fb7100c1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:07.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.149+0000 7fb70ffff640 1 -- 172.21.15.154:0/2704969258 >> v2:172.21.15.190:3300/0 conn(0x7fb71006c1d0 msgr2=0x7fb71006c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:07.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.149+0000 7fb714f05640 1 --2- 172.21.15.154:0/2704969258 >> v2:172.21.15.190:3300/0 conn(0x7fb71006c1d0 0x7fb71006c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:07.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.149+0000 7fb70ffff640 1 --2- 172.21.15.154:0/2704969258 >> v2:172.21.15.190:3300/0 conn(0x7fb71006c1d0 0x7fb71006c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:07.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.149+0000 7fb70ffff640 1 -- 172.21.15.154:0/2704969258 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7100ccd20 con 0x7fb710161010 2023-09-23T03:19:07.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.149+0000 7fb714f05640 1 --2- 172.21.15.154:0/2704969258 >> v2:172.21.15.190:3300/0 conn(0x7fb71006c1d0 0x7fb71006c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:07.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.150+0000 7fb70ffff640 1 --2- 172.21.15.154:0/2704969258 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb710161010 0x7fb7100cc7e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb7101648e0 tx=0x7fb70002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3978c08d6770cfa7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:07.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.151+0000 7fb70f7fe640 1 -- 172.21.15.154:0/2704969258 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb700004230 con 0x7fb710161010 2023-09-23T03:19:07.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.151+0000 7fb70f7fe640 1 -- 172.21.15.154:0/2704969258 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb700004390 con 0x7fb710161010 2023-09-23T03:19:07.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.151+0000 7fb70f7fe640 1 -- 172.21.15.154:0/2704969258 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb700003810 con 0x7fb710161010 2023-09-23T03:19:07.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.151+0000 7fb716968640 1 -- 172.21.15.154:0/2704969258 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb710161010 msgr2=0x7fb7100cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:07.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.151+0000 7fb716968640 1 --2- 172.21.15.154:0/2704969258 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb710161010 0x7fb7100cc7e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb7101648e0 tx=0x7fb70002b860 comp rx=0 tx=0).stop 2023-09-23T03:19:07.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.151+0000 7fb716968640 1 -- 172.21.15.154:0/2704969258 shutdown_connections 2023-09-23T03:19:07.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.151+0000 7fb716968640 1 --2- 172.21.15.154:0/2704969258 >> v2:172.21.15.190:3300/0 conn(0x7fb71006c1d0 0x7fb71006c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:07.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.151+0000 7fb716968640 1 --2- 172.21.15.154:0/2704969258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7100c14e0 0x7fb7100c1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:07.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.151+0000 7fb716968640 1 --2- 172.21.15.154:0/2704969258 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb710161010 0x7fb7100cc7e0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:07.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.151+0000 7fb716968640 1 -- 172.21.15.154:0/2704969258 >> 172.21.15.154:0/2704969258 conn(0x7fb71006a0f0 msgr2=0x7fb71006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:07.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.151+0000 7fb716968640 1 -- 172.21.15.154:0/2704969258 shutdown_connections 2023-09-23T03:19:07.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.152+0000 7fb716968640 1 -- 172.21.15.154:0/2704969258 wait complete. 2023-09-23T03:19:07.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.153+0000 7fb716968640 1 Processor -- start 2023-09-23T03:19:07.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.153+0000 7fb716968640 1 -- start start 2023-09-23T03:19:07.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.154+0000 7fb716968640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb71006c1d0 0x7fb71015fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:07.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.154+0000 7fb716968640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7100c14e0 0x7fb7100c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:07.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.154+0000 7fb716968640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb710161010 0x7fb71006fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:07.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.154+0000 7fb716968640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7100c3370 con 0x7fb710161010 2023-09-23T03:19:07.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.154+0000 7fb716968640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7100c51f0 con 0x7fb7100c14e0 2023-09-23T03:19:07.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.154+0000 7fb716968640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7100c59a0 con 0x7fb71006c1d0 2023-09-23T03:19:07.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.155+0000 7fb714f05640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb710161010 0x7fb71006fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:07.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.155+0000 7fb714f05640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb710161010 0x7fb71006fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54398/0 (socket says 172.21.15.154:54398) 2023-09-23T03:19:07.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.155+0000 7fb714f05640 1 -- 172.21.15.154:0/2545021597 learned_addr learned my addr 172.21.15.154:0/2545021597 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:07.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.155+0000 7fb707fff640 1 --2- 172.21.15.154:0/2545021597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7100c14e0 0x7fb7100c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:07.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.155+0000 7fb70ffff640 1 --2- 172.21.15.154:0/2545021597 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb71006c1d0 0x7fb71015fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:07.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.155+0000 7fb714f05640 1 -- 172.21.15.154:0/2545021597 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb71006c1d0 msgr2=0x7fb71015fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:07.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.155+0000 7fb714f05640 1 --2- 172.21.15.154:0/2545021597 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb71006c1d0 0x7fb71015fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:07.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.155+0000 7fb714f05640 1 -- 172.21.15.154:0/2545021597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7100c14e0 msgr2=0x7fb7100c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:07.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.155+0000 7fb714f05640 1 --2- 172.21.15.154:0/2545021597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7100c14e0 0x7fb7100c80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:07.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.155+0000 7fb714f05640 1 -- 172.21.15.154:0/2545021597 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb700009460 con 0x7fb710161010 2023-09-23T03:19:07.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.156+0000 7fb714f05640 1 --2- 172.21.15.154:0/2545021597 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb710161010 0x7fb71006fee0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb70800cef0 tx=0x7fb70800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:07.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.156+0000 7fb70d7fa640 1 -- 172.21.15.154:0/2545021597 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70800d840 con 0x7fb710161010 2023-09-23T03:19:07.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.156+0000 7fb716968640 1 -- 172.21.15.154:0/2545021597 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb71015ffd0 con 0x7fb710161010 2023-09-23T03:19:07.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.156+0000 7fb70d7fa640 1 -- 172.21.15.154:0/2545021597 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb70800d9c0 con 0x7fb710161010 2023-09-23T03:19:07.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.156+0000 7fb70d7fa640 1 -- 172.21.15.154:0/2545021597 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70800dd70 con 0x7fb710161010 2023-09-23T03:19:07.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.156+0000 7fb716968640 1 -- 172.21.15.154:0/2545021597 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7100c4c30 con 0x7fb710161010 2023-09-23T03:19:07.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.157+0000 7fb716968640 1 -- 172.21.15.154:0/2545021597 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7100c5770 con 0x7fb710161010 2023-09-23T03:19:07.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.158+0000 7fb70d7fa640 1 -- 172.21.15.154:0/2545021597 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fb70801e3b0 con 0x7fb710161010 2023-09-23T03:19:07.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.159+0000 7fb70d7fa640 1 --2- 172.21.15.154:0/2545021597 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb6f006af30 0x7fb6f006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:07.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.159+0000 7fb70d7fa640 1 -- 172.21.15.154:0/2545021597 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6720+0+0 (secure 0 0 0) 0x7fb70808a730 con 0x7fb710161010 2023-09-23T03:19:07.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.159+0000 7fb70ffff640 1 --2- 172.21.15.154:0/2545021597 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb6f006af30 0x7fb6f006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:07.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.160+0000 7fb70ffff640 1 --2- 172.21.15.154:0/2545021597 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb6f006af30 0x7fb6f006d3f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb700034000 tx=0x7fb700008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:07.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.163+0000 7fb70d7fa640 1 -- 172.21.15.154:0/2545021597 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fb7100c5770 con 0x7fb710161010 2023-09-23T03:19:07.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.355+0000 7fb716968640 1 -- 172.21.15.154:0/2545021597 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb7100c5770 con 0x7fb710161010 2023-09-23T03:19:07.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.359+0000 7fb70d7fa640 1 -- 172.21.15.154:0/2545021597 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+18984 (secure 0 0 0) 0x7fb7100c5770 con 0x7fb710161010 2023-09-23T03:19:07.360 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:19:07.361 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":49,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:18:04.269705+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:17:08.933690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-09-23T03:17:10.141711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data_extra","create_time":"2023-09-23T03:17:21.021560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.35","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6837/4079550625":"2023-09-24T03:18:03.516922+0000","172.21.15.154:6835/4079550625":"2023-09-24T03:18:03.516922+0000","172.21.15.154:6837/617287200":"2023-09-24T03:17:57.206517+0000","172.21.15.190:6834/2131436414":"2023-09-24T03:17:55.088768+0000","172.21.15.154:6835/617287200":"2023-09-24T03:17:57.206517+0000","172.21.15.190:6832/2131436414":"2023-09-24T03:17:55.088768+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:19:07.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.361+0000 7fb716968640 1 -- 172.21.15.154:0/2545021597 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb6f006af30 msgr2=0x7fb6f006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:07.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.361+0000 7fb716968640 1 --2- 172.21.15.154:0/2545021597 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb6f006af30 0x7fb6f006d3f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb700034000 tx=0x7fb700008030 comp rx=0 tx=0).stop 2023-09-23T03:19:07.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.361+0000 7fb716968640 1 -- 172.21.15.154:0/2545021597 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb710161010 msgr2=0x7fb71006fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:07.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.361+0000 7fb716968640 1 --2- 172.21.15.154:0/2545021597 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb710161010 0x7fb71006fee0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb70800cef0 tx=0x7fb70800a710 comp rx=0 tx=0).stop 2023-09-23T03:19:07.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.361+0000 7fb716968640 1 -- 172.21.15.154:0/2545021597 shutdown_connections 2023-09-23T03:19:07.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.361+0000 7fb716968640 1 --2- 172.21.15.154:0/2545021597 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb6f006af30 0x7fb6f006d3f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:07.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.361+0000 7fb716968640 1 --2- 172.21.15.154:0/2545021597 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb710161010 0x7fb71006fee0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:07.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.361+0000 7fb716968640 1 --2- 172.21.15.154:0/2545021597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7100c14e0 0x7fb7100c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:07.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.361+0000 7fb716968640 1 --2- 172.21.15.154:0/2545021597 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb71006c1d0 0x7fb71015fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:07.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.361+0000 7fb716968640 1 -- 172.21.15.154:0/2545021597 >> 172.21.15.154:0/2545021597 conn(0x7fb71006a0f0 msgr2=0x7fb71006a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:07.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.362+0000 7fb716968640 1 -- 172.21.15.154:0/2545021597 shutdown_connections 2023-09-23T03:19:07.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.362+0000 7fb716968640 1 -- 172.21.15.154:0/2545021597 wait complete. 2023-09-23T03:19:07.383 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-09-23T03:19:07.383 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:19:07.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.585+0000 7fe4312c2640 1 Processor -- start 2023-09-23T03:19:07.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.585+0000 7fe4312c2640 1 -- start start 2023-09-23T03:19:07.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.587+0000 7fe4312c2640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe42c0acce0 0x7fe42c15ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:07.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.587+0000 7fe4312c2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe42c0b1b20 0x7fe42c0ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:07.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.587+0000 7fe4312c2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe42c0cee70 0x7fe42c06cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:07.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.587+0000 7fe4312c2640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe42c0b2e90 con 0x7fe42c0cee70 2023-09-23T03:19:07.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.587+0000 7fe4312c2640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe42c0cf320 con 0x7fe42c0acce0 2023-09-23T03:19:07.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.587+0000 7fe4312c2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c15ac30 con 0x7fe42c0b1b20 2023-09-23T03:19:07.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.587+0000 7fe42b7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe42c0cee70 0x7fe42c06cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:07.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.587+0000 7fe42b7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe42c0cee70 0x7fe42c06cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54408/0 (socket says 172.21.15.154:54408) 2023-09-23T03:19:07.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.587+0000 7fe42b7fe640 1 -- 172.21.15.154:0/2444539800 learned_addr learned my addr 172.21.15.154:0/2444539800 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:07.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.588+0000 7fe4227fc640 1 --2- 172.21.15.154:0/2444539800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe42c0b1b20 0x7fe42c0ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:07.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.588+0000 7fe42affd640 1 --2- 172.21.15.154:0/2444539800 >> v2:172.21.15.190:3300/0 conn(0x7fe42c0acce0 0x7fe42c15ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:07.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.588+0000 7fe42b7fe640 1 -- 172.21.15.154:0/2444539800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe42c0b1b20 msgr2=0x7fe42c0ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:07.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.588+0000 7fe42b7fe640 1 --2- 172.21.15.154:0/2444539800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe42c0b1b20 0x7fe42c0ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:07.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.588+0000 7fe42b7fe640 1 -- 172.21.15.154:0/2444539800 >> v2:172.21.15.190:3300/0 conn(0x7fe42c0acce0 msgr2=0x7fe42c15ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:07.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.588+0000 7fe42b7fe640 1 --2- 172.21.15.154:0/2444539800 >> v2:172.21.15.190:3300/0 conn(0x7fe42c0acce0 0x7fe42c15ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:07.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.588+0000 7fe42b7fe640 1 -- 172.21.15.154:0/2444539800 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe42c0b1f70 con 0x7fe42c0cee70 2023-09-23T03:19:07.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.588+0000 7fe42affd640 1 --2- 172.21.15.154:0/2444539800 >> v2:172.21.15.190:3300/0 conn(0x7fe42c0acce0 0x7fe42c15ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:07.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.589+0000 7fe42b7fe640 1 --2- 172.21.15.154:0/2444539800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe42c0cee70 0x7fe42c06cb50 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe42c0cf570 tx=0x7fe42402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e5687766c5622b5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:07.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.589+0000 7fe42a7fc640 1 -- 172.21.15.154:0/2444539800 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42402c9e0 con 0x7fe42c0cee70 2023-09-23T03:19:07.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.589+0000 7fe42a7fc640 1 -- 172.21.15.154:0/2444539800 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe42402cb40 con 0x7fe42c0cee70 2023-09-23T03:19:07.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.589+0000 7fe42a7fc640 1 -- 172.21.15.154:0/2444539800 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4240343f0 con 0x7fe42c0cee70 2023-09-23T03:19:07.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.590+0000 7fe4312c2640 1 -- 172.21.15.154:0/2444539800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe42c0cee70 msgr2=0x7fe42c06cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:07.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.590+0000 7fe4312c2640 1 --2- 172.21.15.154:0/2444539800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe42c0cee70 0x7fe42c06cb50 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe42c0cf570 tx=0x7fe42402be40 comp rx=0 tx=0).stop 2023-09-23T03:19:07.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.590+0000 7fe4312c2640 1 -- 172.21.15.154:0/2444539800 shutdown_connections 2023-09-23T03:19:07.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.590+0000 7fe4312c2640 1 --2- 172.21.15.154:0/2444539800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe42c0cee70 0x7fe42c06cb50 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:07.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.590+0000 7fe4312c2640 1 --2- 172.21.15.154:0/2444539800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe42c0b1b20 0x7fe42c0ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:07.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.590+0000 7fe4312c2640 1 --2- 172.21.15.154:0/2444539800 >> v2:172.21.15.190:3300/0 conn(0x7fe42c0acce0 0x7fe42c15ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:07.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.590+0000 7fe4312c2640 1 -- 172.21.15.154:0/2444539800 >> 172.21.15.154:0/2444539800 conn(0x7fe42c06a0f0 msgr2=0x7fe42c06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:07.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.590+0000 7fe4312c2640 1 -- 172.21.15.154:0/2444539800 shutdown_connections 2023-09-23T03:19:07.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.590+0000 7fe4312c2640 1 -- 172.21.15.154:0/2444539800 wait complete. 2023-09-23T03:19:07.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.592+0000 7fe4312c2640 1 Processor -- start 2023-09-23T03:19:07.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.592+0000 7fe4312c2640 1 -- start start 2023-09-23T03:19:07.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.593+0000 7fe4312c2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe42c0acce0 0x7fe42c15d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:07.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.593+0000 7fe4312c2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe42c0b1b20 0x7fe42c097a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:07.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.593+0000 7fe4312c2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe42c0cee70 0x7fe42c0c37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:07.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.593+0000 7fe4312c2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c097f60 con 0x7fe42c0b1b20 2023-09-23T03:19:07.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.593+0000 7fe4312c2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c0c7ec0 con 0x7fe42c0acce0 2023-09-23T03:19:07.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.593+0000 7fe4312c2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c0c81a0 con 0x7fe42c0cee70 2023-09-23T03:19:07.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.593+0000 7fe4227fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe42c0b1b20 0x7fe42c097a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:07.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.593+0000 7fe4227fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe42c0b1b20 0x7fe42c097a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54416/0 (socket says 172.21.15.154:54416) 2023-09-23T03:19:07.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.593+0000 7fe4227fc640 1 -- 172.21.15.154:0/1425822106 learned_addr learned my addr 172.21.15.154:0/1425822106 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:07.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.594+0000 7fe4227fc640 1 -- 172.21.15.154:0/1425822106 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe42c0cee70 msgr2=0x7fe42c0c37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:07.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.594+0000 7fe4227fc640 1 --2- 172.21.15.154:0/1425822106 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe42c0cee70 0x7fe42c0c37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:07.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.594+0000 7fe4227fc640 1 -- 172.21.15.154:0/1425822106 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe42c0acce0 msgr2=0x7fe42c15d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:07.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.594+0000 7fe42affd640 1 --2- 172.21.15.154:0/1425822106 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe42c0acce0 0x7fe42c15d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:07.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.594+0000 7fe4227fc640 1 --2- 172.21.15.154:0/1425822106 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe42c0acce0 0x7fe42c15d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:07.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.594+0000 7fe4227fc640 1 -- 172.21.15.154:0/1425822106 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe424009b30 con 0x7fe42c0b1b20 2023-09-23T03:19:07.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.594+0000 7fe4227fc640 1 --2- 172.21.15.154:0/1425822106 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe42c0b1b20 0x7fe42c097a20 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe41400c7f0 tx=0x7fe41400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:07.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.594+0000 7fe423fff640 1 -- 172.21.15.154:0/1425822106 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe414004220 con 0x7fe42c0b1b20 2023-09-23T03:19:07.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.594+0000 7fe423fff640 1 -- 172.21.15.154:0/1425822106 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe414004380 con 0x7fe42c0b1b20 2023-09-23T03:19:07.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.594+0000 7fe423fff640 1 -- 172.21.15.154:0/1425822106 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe414007840 con 0x7fe42c0b1b20 2023-09-23T03:19:07.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.594+0000 7fe4312c2640 1 -- 172.21.15.154:0/1425822106 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe42c15af70 con 0x7fe42c0b1b20 2023-09-23T03:19:07.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.594+0000 7fe4312c2640 1 -- 172.21.15.154:0/1425822106 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe42c0c3d30 con 0x7fe42c0b1b20 2023-09-23T03:19:07.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.595+0000 7fe4312c2640 1 -- 172.21.15.154:0/1425822106 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe42c15add0 con 0x7fe42c0b1b20 2023-09-23T03:19:07.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.596+0000 7fe423fff640 1 -- 172.21.15.154:0/1425822106 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fe414007bb0 con 0x7fe42c0b1b20 2023-09-23T03:19:07.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.598+0000 7fe423fff640 1 --2- 172.21.15.154:0/1425822106 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe40c06af30 0x7fe40c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:07.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.598+0000 7fe423fff640 1 -- 172.21.15.154:0/1425822106 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6720+0+0 (secure 0 0 0) 0x7fe414089a80 con 0x7fe42c0b1b20 2023-09-23T03:19:07.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.598+0000 7fe42affd640 1 --2- 172.21.15.154:0/1425822106 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe40c06af30 0x7fe40c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:07.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.599+0000 7fe42affd640 1 --2- 172.21.15.154:0/1425822106 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe40c06af30 0x7fe40c06d3f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe41c006fd0 tx=0x7fe41c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:07.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.602+0000 7fe423fff640 1 -- 172.21.15.154:0/1425822106 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fe42c15add0 con 0x7fe42c0b1b20 2023-09-23T03:19:07.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.830+0000 7fe4312c2640 1 -- 172.21.15.154:0/1425822106 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe42c15b490 con 0x7fe42c0b1b20 2023-09-23T03:19:07.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.831+0000 7fe423fff640 1 -- 172.21.15.154:0/1425822106 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4928 (secure 0 0 0) 0x7fe42c15b490 con 0x7fe42c0b1b20 2023-09-23T03:19:07.832 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:19:07.833 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":32,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5215,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3388818833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3388818833},{"type":"v1","addr":"172.21.15.190:6835","nonce":3388818833}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5991,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1964552060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1964552060},{"type":"v1","addr":"172.21.15.154:6837","nonce":1964552060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5994,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/1736999297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1736999297},{"type":"v1","addr":"172.21.15.190:6834","nonce":1736999297}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:17:56.221314+0000","modified":"2023-09-23T03:18:05.544508+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5988},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5988":{"gid":5988,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":4,"addr":"172.21.15.154:6836/4278807008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4278807008},{"type":"v1","addr":"172.21.15.154:6836","nonce":4278807008}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-09-23T03:19:07.833 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 32 2023-09-23T03:19:07.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.834+0000 7fe4312c2640 1 -- 172.21.15.154:0/1425822106 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe40c06af30 msgr2=0x7fe40c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:07.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.834+0000 7fe4312c2640 1 --2- 172.21.15.154:0/1425822106 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe40c06af30 0x7fe40c06d3f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe41c006fd0 tx=0x7fe41c008040 comp rx=0 tx=0).stop 2023-09-23T03:19:07.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.834+0000 7fe4312c2640 1 -- 172.21.15.154:0/1425822106 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe42c0b1b20 msgr2=0x7fe42c097a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:07.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.834+0000 7fe4312c2640 1 --2- 172.21.15.154:0/1425822106 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe42c0b1b20 0x7fe42c097a20 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe41400c7f0 tx=0x7fe41400ccb0 comp rx=0 tx=0).stop 2023-09-23T03:19:07.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.834+0000 7fe4312c2640 1 -- 172.21.15.154:0/1425822106 shutdown_connections 2023-09-23T03:19:07.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.834+0000 7fe4312c2640 1 --2- 172.21.15.154:0/1425822106 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe40c06af30 0x7fe40c06d3f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:07.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.834+0000 7fe4312c2640 1 --2- 172.21.15.154:0/1425822106 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe42c0cee70 0x7fe42c0c37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:07.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.834+0000 7fe4312c2640 1 --2- 172.21.15.154:0/1425822106 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe42c0b1b20 0x7fe42c097a20 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:07.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.834+0000 7fe4312c2640 1 --2- 172.21.15.154:0/1425822106 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe42c0acce0 0x7fe42c15d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:07.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.834+0000 7fe4312c2640 1 -- 172.21.15.154:0/1425822106 >> 172.21.15.154:0/1425822106 conn(0x7fe42c06a0f0 msgr2=0x7fe42c0b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:07.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.834+0000 7fe4312c2640 1 -- 172.21.15.154:0/1425822106 shutdown_connections 2023-09-23T03:19:07.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:07.834+0000 7fe4312c2640 1 -- 172.21.15.154:0/1425822106 wait complete. 2023-09-23T03:19:07.854 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:19:08.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.065+0000 7faead143640 1 Processor -- start 2023-09-23T03:19:08.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.066+0000 7faead143640 1 -- start start 2023-09-23T03:19:08.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.067+0000 7faead143640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faea806c7a0 0x7faea815eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:08.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.067+0000 7faead143640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7faea815ab00 0x7faea815af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:08.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.067+0000 7faead143640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faea806bff0 con 0x7faea806fdc0 2023-09-23T03:19:08.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.067+0000 7faead143640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faea8155070 con 0x7faea815ab00 2023-09-23T03:19:08.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.067+0000 7faead143640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faea815e0f0 con 0x7faea806c7a0 2023-09-23T03:19:08.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.068+0000 7faea7fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faea806c7a0 0x7faea815eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:08.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.068+0000 7fae9ffff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7faea815ab00 0x7faea815af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:08.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.068+0000 7fae9ffff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7faea815ab00 0x7faea815af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.154:38110/0 (socket says 172.21.15.154:38110) 2023-09-23T03:19:08.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.068+0000 7fae9ffff640 1 -- 172.21.15.154:0/2838331730 learned_addr learned my addr 172.21.15.154:0/2838331730 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:08.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.069+0000 7fae9ffff640 1 -- 172.21.15.154:0/2838331730 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faea806c7a0 msgr2=0x7faea815eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:08.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.069+0000 7fae9ffff640 1 --2- 172.21.15.154:0/2838331730 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faea806c7a0 0x7faea815eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:08.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.069+0000 7fae9ffff640 1 -- 172.21.15.154:0/2838331730 >> v1:172.21.15.190:6789/0 conn(0x7faea806fdc0 legacy=0x7faea80ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:08.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.069+0000 7fae9ffff640 1 -- 172.21.15.154:0/2838331730 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea806cb80 con 0x7faea815ab00 2023-09-23T03:19:08.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.069+0000 7faea7fff640 1 --2- 172.21.15.154:0/2838331730 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faea806c7a0 0x7faea815eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:19:08.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.070+0000 7fae9ffff640 1 --2- 172.21.15.154:0/2838331730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faea815ab00 0x7faea815af80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fae90009870 tx=0x7fae9002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=812a92ca9f6e5e12 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:08.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.071+0000 7faea77fe640 1 -- 172.21.15.154:0/2838331730 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae90030840 con 0x7faea815ab00 2023-09-23T03:19:08.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.071+0000 7faea77fe640 1 -- 172.21.15.154:0/2838331730 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae900309a0 con 0x7faea815ab00 2023-09-23T03:19:08.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.071+0000 7faea77fe640 1 -- 172.21.15.154:0/2838331730 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae90030d30 con 0x7faea815ab00 2023-09-23T03:19:08.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.072+0000 7faead143640 1 -- 172.21.15.154:0/2838331730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faea815ab00 msgr2=0x7faea815af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:08.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.072+0000 7faead143640 1 --2- 172.21.15.154:0/2838331730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faea815ab00 0x7faea815af80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fae90009870 tx=0x7fae9002b980 comp rx=0 tx=0).stop 2023-09-23T03:19:08.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.072+0000 7faead143640 1 -- 172.21.15.154:0/2838331730 shutdown_connections 2023-09-23T03:19:08.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.072+0000 7faead143640 1 --2- 172.21.15.154:0/2838331730 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faea815ab00 0x7faea815af80 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:08.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.072+0000 7faead143640 1 --2- 172.21.15.154:0/2838331730 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faea806c7a0 0x7faea815eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:08.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.072+0000 7faead143640 1 -- 172.21.15.154:0/2838331730 >> 172.21.15.154:0/2838331730 conn(0x7faea806a390 msgr2=0x7faea80afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:08.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.072+0000 7faead143640 1 -- 172.21.15.154:0/2838331730 shutdown_connections 2023-09-23T03:19:08.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.072+0000 7faead143640 1 -- 172.21.15.154:0/2838331730 wait complete. 2023-09-23T03:19:08.075 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.074+0000 7faead143640 1 Processor -- start 2023-09-23T03:19:08.075 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.074+0000 7faead143640 1 -- start start 2023-09-23T03:19:08.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.075+0000 7faead143640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faea806c7a0 0x7faea81537b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:08.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.075+0000 7faead143640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faea806fdc0 0x7faea80b1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:08.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.075+0000 7faead143640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faea815ab00 0x7faea80c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:08.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.075+0000 7faead143640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea80c5df0 con 0x7faea806fdc0 2023-09-23T03:19:08.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.075+0000 7faead143640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea80c54c0 con 0x7faea806c7a0 2023-09-23T03:19:08.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.075+0000 7faead143640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faea80c57a0 con 0x7faea815ab00 2023-09-23T03:19:08.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.075+0000 7fae9ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faea806fdc0 0x7faea80b1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:08.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.075+0000 7fae9ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faea806fdc0 0x7faea80b1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54432/0 (socket says 172.21.15.154:54432) 2023-09-23T03:19:08.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.075+0000 7fae9ffff640 1 -- 172.21.15.154:0/1258143052 learned_addr learned my addr 172.21.15.154:0/1258143052 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:08.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.075+0000 7faea7fff640 1 --2- 172.21.15.154:0/1258143052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faea806c7a0 0x7faea81537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:08.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.075+0000 7faeac942640 1 --2- 172.21.15.154:0/1258143052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faea815ab00 0x7faea80c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:08.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.076+0000 7fae9ffff640 1 -- 172.21.15.154:0/1258143052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faea815ab00 msgr2=0x7faea80c76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:08.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.076+0000 7fae9ffff640 1 --2- 172.21.15.154:0/1258143052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faea815ab00 0x7faea80c76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:08.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.076+0000 7fae9ffff640 1 -- 172.21.15.154:0/1258143052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faea806c7a0 msgr2=0x7faea81537b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:08.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.076+0000 7fae9ffff640 1 --2- 172.21.15.154:0/1258143052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faea806c7a0 0x7faea81537b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:08.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.076+0000 7fae9ffff640 1 -- 172.21.15.154:0/1258143052 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae98009400 con 0x7faea806fdc0 2023-09-23T03:19:08.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.076+0000 7fae9ffff640 1 --2- 172.21.15.154:0/1258143052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faea806fdc0 0x7faea80b1a60 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fae9002b9c0 tx=0x7fae900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:08.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.076+0000 7faea57fa640 1 -- 172.21.15.154:0/1258143052 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9003e070 con 0x7faea806fdc0 2023-09-23T03:19:08.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.076+0000 7faea57fa640 1 -- 172.21.15.154:0/1258143052 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae90039a10 con 0x7faea806fdc0 2023-09-23T03:19:08.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.076+0000 7faead143640 1 -- 172.21.15.154:0/1258143052 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae900094d0 con 0x7faea806fdc0 2023-09-23T03:19:08.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.076+0000 7faea57fa640 1 -- 172.21.15.154:0/1258143052 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae90039de0 con 0x7faea806fdc0 2023-09-23T03:19:08.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.077+0000 7faead143640 1 -- 172.21.15.154:0/1258143052 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea80c4380 con 0x7faea806fdc0 2023-09-23T03:19:08.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.078+0000 7faead143640 1 -- 172.21.15.154:0/1258143052 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faea80736b0 con 0x7faea806fdc0 2023-09-23T03:19:08.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.079+0000 7faea57fa640 1 -- 172.21.15.154:0/1258143052 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fae9004b400 con 0x7faea806fdc0 2023-09-23T03:19:08.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.080+0000 7faea57fa640 1 --2- 172.21.15.154:0/1258143052 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fae8806af30 0x7fae8806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:08.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.080+0000 7faea57fa640 1 -- 172.21.15.154:0/1258143052 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6720+0+0 (secure 0 0 0) 0x7fae900ac020 con 0x7faea806fdc0 2023-09-23T03:19:08.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.080+0000 7faea7fff640 1 --2- 172.21.15.154:0/1258143052 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fae8806af30 0x7fae8806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:08.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.080+0000 7faea7fff640 1 --2- 172.21.15.154:0/1258143052 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fae8806af30 0x7fae8806d3f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faea8154480 tx=0x7fae98008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:08.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.084+0000 7faea57fa640 1 -- 172.21.15.154:0/1258143052 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fae90076d00 con 0x7faea806fdc0 2023-09-23T03:19:08.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.311+0000 7faead143640 1 -- 172.21.15.154:0/1258143052 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faea804e920 con 0x7faea806fdc0 2023-09-23T03:19:08.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.313+0000 7faea57fa640 1 -- 172.21.15.154:0/1258143052 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4928 (secure 0 0 0) 0x7fae90034090 con 0x7faea806fdc0 2023-09-23T03:19:08.314 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:19:08.314 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":32,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5215,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3388818833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3388818833},{"type":"v1","addr":"172.21.15.190:6835","nonce":3388818833}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5991,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1964552060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1964552060},{"type":"v1","addr":"172.21.15.154:6837","nonce":1964552060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5994,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/1736999297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1736999297},{"type":"v1","addr":"172.21.15.190:6834","nonce":1736999297}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:17:56.221314+0000","modified":"2023-09-23T03:18:05.544508+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5988},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5988":{"gid":5988,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":4,"addr":"172.21.15.154:6836/4278807008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4278807008},{"type":"v1","addr":"172.21.15.154:6836","nonce":4278807008}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-09-23T03:19:08.314 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 32 2023-09-23T03:19:08.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.316+0000 7faead143640 1 -- 172.21.15.154:0/1258143052 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fae8806af30 msgr2=0x7fae8806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:08.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.316+0000 7faead143640 1 --2- 172.21.15.154:0/1258143052 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fae8806af30 0x7fae8806d3f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faea8154480 tx=0x7fae98008040 comp rx=0 tx=0).stop 2023-09-23T03:19:08.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.316+0000 7faead143640 1 -- 172.21.15.154:0/1258143052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faea806fdc0 msgr2=0x7faea80b1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:08.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.316+0000 7faead143640 1 --2- 172.21.15.154:0/1258143052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faea806fdc0 0x7faea80b1a60 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fae9002b9c0 tx=0x7fae900046c0 comp rx=0 tx=0).stop 2023-09-23T03:19:08.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.316+0000 7faead143640 1 -- 172.21.15.154:0/1258143052 shutdown_connections 2023-09-23T03:19:08.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.316+0000 7faead143640 1 --2- 172.21.15.154:0/1258143052 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fae8806af30 0x7fae8806d3f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:08.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.316+0000 7faead143640 1 --2- 172.21.15.154:0/1258143052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faea815ab00 0x7faea80c76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:08.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.316+0000 7faead143640 1 --2- 172.21.15.154:0/1258143052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faea806fdc0 0x7faea80b1a60 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:08.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.316+0000 7faead143640 1 --2- 172.21.15.154:0/1258143052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faea806c7a0 0x7faea81537b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:08.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.316+0000 7faead143640 1 -- 172.21.15.154:0/1258143052 >> 172.21.15.154:0/1258143052 conn(0x7faea806a390 msgr2=0x7faea80ae2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:08.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.316+0000 7faead143640 1 -- 172.21.15.154:0/1258143052 shutdown_connections 2023-09-23T03:19:08.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.316+0000 7faead143640 1 -- 172.21.15.154:0/1258143052 wait complete. 2023-09-23T03:19:08.336 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:19:08.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.544+0000 7f81be8ca640 1 Processor -- start 2023-09-23T03:19:08.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.545+0000 7f81be8ca640 1 -- start start 2023-09-23T03:19:08.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.546+0000 7f81be8ca640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f81b80cf500 0x7f81b80ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:08.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.546+0000 7f81be8ca640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81b80cf8e0 0x7f81b80ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:08.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.546+0000 7f81be8ca640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81b80d17f0 con 0x7f81b815c4f0 2023-09-23T03:19:08.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.546+0000 7f81be8ca640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81b806be10 con 0x7f81b80cf500 2023-09-23T03:19:08.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.546+0000 7f81be8ca640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81b80d0470 con 0x7f81b80cf8e0 2023-09-23T03:19:08.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.546+0000 7f81be0c9640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f81b815c4f0 0x7f81b815c980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:41594/0 (socket says 172.21.15.154:41594) 2023-09-23T03:19:08.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.546+0000 7f81be0c9640 1 -- 172.21.15.154:0/2140360128 learned_addr learned my addr 172.21.15.154:0/2140360128 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:19:08.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.546+0000 7f81bd8c8640 1 --2- 172.21.15.154:0/2140360128 >> v2:172.21.15.154:3300/0 conn(0x7f81b80cf500 0x7f81b80ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:08.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.547+0000 7f81bd0c7640 1 --2- 172.21.15.154:0/2140360128 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81b80cf8e0 0x7f81b80ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:08.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.547+0000 7f81bc8c6640 1 -- 172.21.15.154:0/2140360128 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 26735551 0 0) 0x7f81b80d17f0 con 0x7f81b815c4f0 2023-09-23T03:19:08.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.547+0000 7f81bc8c6640 1 -- 172.21.15.154:0/2140360128 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81a0003520 con 0x7f81b815c4f0 2023-09-23T03:19:08.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.547+0000 7f81bd8c8640 1 -- 172.21.15.154:0/2140360128 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81b80cf8e0 msgr2=0x7f81b80ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:08.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.547+0000 7f81bd8c8640 1 --2- 172.21.15.154:0/2140360128 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81b80cf8e0 0x7f81b80ce590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:08.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.547+0000 7f81bd8c8640 1 -- 172.21.15.154:0/2140360128 >> v1:172.21.15.154:6789/0 conn(0x7f81b815c4f0 legacy=0x7f81b815c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:08.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.547+0000 7f81bd8c8640 1 -- 172.21.15.154:0/2140360128 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b80d0750 con 0x7f81b80cf500 2023-09-23T03:19:08.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.548+0000 7f81bd8c8640 1 --2- 172.21.15.154:0/2140360128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81b80cf500 0x7f81b80ce050 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f81a8009de0 tx=0x7f81a802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e1cfbccc93b7345 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:08.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.548+0000 7f81bc8c6640 1 -- 172.21.15.154:0/2140360128 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a802cc00 con 0x7f81b80cf500 2023-09-23T03:19:08.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.549+0000 7f81bc8c6640 1 -- 172.21.15.154:0/2140360128 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a802cd60 con 0x7f81b80cf500 2023-09-23T03:19:08.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.549+0000 7f81bc8c6640 1 -- 172.21.15.154:0/2140360128 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a8034640 con 0x7f81b80cf500 2023-09-23T03:19:08.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.549+0000 7f81be8ca640 1 -- 172.21.15.154:0/2140360128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81b80cf500 msgr2=0x7f81b80ce050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:08.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.549+0000 7f81be8ca640 1 --2- 172.21.15.154:0/2140360128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81b80cf500 0x7f81b80ce050 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f81a8009de0 tx=0x7f81a802be50 comp rx=0 tx=0).stop 2023-09-23T03:19:08.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.549+0000 7f81be8ca640 1 -- 172.21.15.154:0/2140360128 shutdown_connections 2023-09-23T03:19:08.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.549+0000 7f81be8ca640 1 --2- 172.21.15.154:0/2140360128 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81b80cf8e0 0x7f81b80ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:08.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.549+0000 7f81be8ca640 1 --2- 172.21.15.154:0/2140360128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81b80cf500 0x7f81b80ce050 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:08.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.549+0000 7f81be8ca640 1 -- 172.21.15.154:0/2140360128 >> 172.21.15.154:0/2140360128 conn(0x7f81b809bbb0 msgr2=0x7f81b80d28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:08.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.549+0000 7f81be8ca640 1 -- 172.21.15.154:0/2140360128 shutdown_connections 2023-09-23T03:19:08.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.549+0000 7f81be8ca640 1 -- 172.21.15.154:0/2140360128 wait complete. 2023-09-23T03:19:08.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.551+0000 7f81be8ca640 1 Processor -- start 2023-09-23T03:19:08.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.551+0000 7f81be8ca640 1 -- start start 2023-09-23T03:19:08.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.552+0000 7f81be8ca640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81b80cf500 0x7f81b8151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:08.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.552+0000 7f81be8ca640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81b80cf8e0 0x7f81b80cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:08.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.552+0000 7f81be8ca640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81b815c4f0 0x7f81b80c8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:08.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.552+0000 7f81be8ca640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b81522e0 con 0x7f81b80cf500 2023-09-23T03:19:08.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.552+0000 7f81be8ca640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b80cb8e0 con 0x7f81b80cf8e0 2023-09-23T03:19:08.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.552+0000 7f81be8ca640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81b80cc190 con 0x7f81b815c4f0 2023-09-23T03:19:08.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.552+0000 7f81bd8c8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81b80cf500 0x7f81b8151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:08.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.552+0000 7f81bd8c8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81b80cf500 0x7f81b8151da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54446/0 (socket says 172.21.15.154:54446) 2023-09-23T03:19:08.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.552+0000 7f81bd8c8640 1 -- 172.21.15.154:0/1711034940 learned_addr learned my addr 172.21.15.154:0/1711034940 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:08.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.553+0000 7f81bd8c8640 1 -- 172.21.15.154:0/1711034940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81b815c4f0 msgr2=0x7f81b80c8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:08.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.553+0000 7f81bd8c8640 1 --2- 172.21.15.154:0/1711034940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81b815c4f0 0x7f81b80c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:08.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.553+0000 7f81bd8c8640 1 -- 172.21.15.154:0/1711034940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81b80cf8e0 msgr2=0x7f81b80cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:08.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.553+0000 7f81bd8c8640 1 --2- 172.21.15.154:0/1711034940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81b80cf8e0 0x7f81b80cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:08.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.553+0000 7f81bd8c8640 1 -- 172.21.15.154:0/1711034940 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a8009a70 con 0x7f81b80cf500 2023-09-23T03:19:08.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.553+0000 7f81bd8c8640 1 --2- 172.21.15.154:0/1711034940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81b80cf500 0x7f81b8151da0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f81a8009de0 tx=0x7f81a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:08.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.553+0000 7f81b67fc640 1 -- 172.21.15.154:0/1711034940 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a8040650 con 0x7f81b80cf500 2023-09-23T03:19:08.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.553+0000 7f81b67fc640 1 -- 172.21.15.154:0/1711034940 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a80407d0 con 0x7f81b80cf500 2023-09-23T03:19:08.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.553+0000 7f81b67fc640 1 -- 172.21.15.154:0/1711034940 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a8040b80 con 0x7f81b80cf500 2023-09-23T03:19:08.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.553+0000 7f81be8ca640 1 -- 172.21.15.154:0/1711034940 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81b8154310 con 0x7f81b80cf500 2023-09-23T03:19:08.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.554+0000 7f81be8ca640 1 -- 172.21.15.154:0/1711034940 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81b81548d0 con 0x7f81b80cf500 2023-09-23T03:19:08.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.555+0000 7f81be8ca640 1 -- 172.21.15.154:0/1711034940 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81b804e920 con 0x7f81b80cf500 2023-09-23T03:19:08.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.556+0000 7f81b67fc640 1 -- 172.21.15.154:0/1711034940 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f81a8040ce0 con 0x7f81b80cf500 2023-09-23T03:19:08.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.557+0000 7f81b67fc640 1 --2- 172.21.15.154:0/1711034940 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f81a406af30 0x7f81a406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:08.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.557+0000 7f81b67fc640 1 -- 172.21.15.154:0/1711034940 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6720+0+0 (secure 0 0 0) 0x7f81a80ac980 con 0x7f81b80cf500 2023-09-23T03:19:08.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.557+0000 7f81bd0c7640 1 --2- 172.21.15.154:0/1711034940 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f81a406af30 0x7f81a406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:08.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.558+0000 7f81bd0c7640 1 --2- 172.21.15.154:0/1711034940 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f81a406af30 0x7f81a406d3f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f81b80cca00 tx=0x7f81ac008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:08.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.561+0000 7f81b67fc640 1 -- 172.21.15.154:0/1711034940 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f81a80775e0 con 0x7f81b80cf500 2023-09-23T03:19:08.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.750+0000 7f81be8ca640 1 -- 172.21.15.154:0/1711034940 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81b80d2c30 con 0x7f81b80cf500 2023-09-23T03:19:08.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.753+0000 7f81b67fc640 1 -- 172.21.15.154:0/1711034940 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+18984 (secure 0 0 0) 0x7f81a80aae60 con 0x7f81b80cf500 2023-09-23T03:19:08.755 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:19:08.755 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":49,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:18:04.269705+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:17:08.933690+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-09-23T03:17:10.141711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data_extra","create_time":"2023-09-23T03:17:21.021560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.35","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6837/4079550625":"2023-09-24T03:18:03.516922+0000","172.21.15.154:6835/4079550625":"2023-09-24T03:18:03.516922+0000","172.21.15.154:6837/617287200":"2023-09-24T03:17:57.206517+0000","172.21.15.190:6834/2131436414":"2023-09-24T03:17:55.088768+0000","172.21.15.154:6835/617287200":"2023-09-24T03:17:57.206517+0000","172.21.15.190:6832/2131436414":"2023-09-24T03:17:55.088768+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:19:08.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.756+0000 7f81be8ca640 1 -- 172.21.15.154:0/1711034940 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f81a406af30 msgr2=0x7f81a406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:08.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.756+0000 7f81be8ca640 1 --2- 172.21.15.154:0/1711034940 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f81a406af30 0x7f81a406d3f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f81b80cca00 tx=0x7f81ac008040 comp rx=0 tx=0).stop 2023-09-23T03:19:08.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.756+0000 7f81be8ca640 1 -- 172.21.15.154:0/1711034940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81b80cf500 msgr2=0x7f81b8151da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:08.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.756+0000 7f81be8ca640 1 --2- 172.21.15.154:0/1711034940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81b80cf500 0x7f81b8151da0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f81a8009de0 tx=0x7f81a8004a40 comp rx=0 tx=0).stop 2023-09-23T03:19:08.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.756+0000 7f81be8ca640 1 -- 172.21.15.154:0/1711034940 shutdown_connections 2023-09-23T03:19:08.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.756+0000 7f81be8ca640 1 --2- 172.21.15.154:0/1711034940 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f81a406af30 0x7f81a406d3f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:08.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.756+0000 7f81be8ca640 1 --2- 172.21.15.154:0/1711034940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f81b815c4f0 0x7f81b80c8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:08.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.756+0000 7f81be8ca640 1 --2- 172.21.15.154:0/1711034940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f81b80cf8e0 0x7f81b80cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:08.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.756+0000 7f81be8ca640 1 --2- 172.21.15.154:0/1711034940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f81b80cf500 0x7f81b8151da0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:08.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.756+0000 7f81be8ca640 1 -- 172.21.15.154:0/1711034940 >> 172.21.15.154:0/1711034940 conn(0x7f81b809bbb0 msgr2=0x7f81b815c980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:08.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.756+0000 7f81be8ca640 1 -- 172.21.15.154:0/1711034940 shutdown_connections 2023-09-23T03:19:08.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.757+0000 7f81be8ca640 1 -- 172.21.15.154:0/1711034940 wait complete. 2023-09-23T03:19:08.776 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-09-23T03:19:08.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.986+0000 7f1150f47640 1 Processor -- start 2023-09-23T03:19:08.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.986+0000 7f1150f47640 1 -- start start 2023-09-23T03:19:08.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.988+0000 7f1150f47640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f114c06c7a0 0x7f114c15eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:08.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.988+0000 7f1150f47640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f114c15ab00 0x7f114c15afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:08.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.988+0000 7f1150f47640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f114c06fdc0 0x7f114c0701f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:08.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.988+0000 7f1150f47640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f114c155070 con 0x7f114c15ab00 2023-09-23T03:19:08.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.988+0000 7f1150f47640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f114c15e0f0 con 0x7f114c06c7a0 2023-09-23T03:19:08.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.988+0000 7f1150f47640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f114c15f040 con 0x7f114c06fdc0 2023-09-23T03:19:08.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.988+0000 7f1143fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f114c15ab00 0x7f114c15afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:08.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.988+0000 7f1143fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f114c15ab00 0x7f114c15afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54460/0 (socket says 172.21.15.154:54460) 2023-09-23T03:19:08.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.988+0000 7f1143fff640 1 -- 172.21.15.154:0/80109494 learned_addr learned my addr 172.21.15.154:0/80109494 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:08.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.989+0000 7f114b7fe640 1 --2- 172.21.15.154:0/80109494 >> v2:172.21.15.190:3300/0 conn(0x7f114c06c7a0 0x7f114c15eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:08.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.989+0000 7f114bfff640 1 --2- 172.21.15.154:0/80109494 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f114c06fdc0 0x7f114c0701f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:08.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.989+0000 7f1143fff640 1 -- 172.21.15.154:0/80109494 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f114c06fdc0 msgr2=0x7f114c0701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:08.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.989+0000 7f1143fff640 1 --2- 172.21.15.154:0/80109494 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f114c06fdc0 0x7f114c0701f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:08.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.989+0000 7f1143fff640 1 -- 172.21.15.154:0/80109494 >> v2:172.21.15.190:3300/0 conn(0x7f114c06c7a0 msgr2=0x7f114c15eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:08.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.989+0000 7f1143fff640 1 --2- 172.21.15.154:0/80109494 >> v2:172.21.15.190:3300/0 conn(0x7f114c06c7a0 0x7f114c15eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:08.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.989+0000 7f1143fff640 1 -- 172.21.15.154:0/80109494 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f114c070730 con 0x7f114c15ab00 2023-09-23T03:19:08.992 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.990+0000 7f1143fff640 1 --2- 172.21.15.154:0/80109494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f114c15ab00 0x7f114c15afb0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f114c0708c0 tx=0x7f113402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682a49403fb69f97 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:08.992 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.990+0000 7f114affd640 1 -- 172.21.15.154:0/80109494 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1134031640 con 0x7f114c15ab00 2023-09-23T03:19:08.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.990+0000 7f114affd640 1 -- 172.21.15.154:0/80109494 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11340317a0 con 0x7f114c15ab00 2023-09-23T03:19:08.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.990+0000 7f114affd640 1 -- 172.21.15.154:0/80109494 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1134031b30 con 0x7f114c15ab00 2023-09-23T03:19:08.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.991+0000 7f1150f47640 1 -- 172.21.15.154:0/80109494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f114c15ab00 msgr2=0x7f114c15afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:08.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.991+0000 7f1150f47640 1 --2- 172.21.15.154:0/80109494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f114c15ab00 0x7f114c15afb0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f114c0708c0 tx=0x7f113402b8f0 comp rx=0 tx=0).stop 2023-09-23T03:19:08.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.991+0000 7f1150f47640 1 -- 172.21.15.154:0/80109494 shutdown_connections 2023-09-23T03:19:08.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.991+0000 7f1150f47640 1 --2- 172.21.15.154:0/80109494 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f114c06fdc0 0x7f114c0701f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:08.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.991+0000 7f1150f47640 1 --2- 172.21.15.154:0/80109494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f114c15ab00 0x7f114c15afb0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:08.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.991+0000 7f1150f47640 1 --2- 172.21.15.154:0/80109494 >> v2:172.21.15.190:3300/0 conn(0x7f114c06c7a0 0x7f114c15eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:08.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.991+0000 7f1150f47640 1 -- 172.21.15.154:0/80109494 >> 172.21.15.154:0/80109494 conn(0x7f114c06a390 msgr2=0x7f114c0afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:08.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.991+0000 7f1150f47640 1 -- 172.21.15.154:0/80109494 shutdown_connections 2023-09-23T03:19:08.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.991+0000 7f1150f47640 1 -- 172.21.15.154:0/80109494 wait complete. 2023-09-23T03:19:08.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.993+0000 7f1150f47640 1 Processor -- start 2023-09-23T03:19:08.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.993+0000 7f1150f47640 1 -- start start 2023-09-23T03:19:08.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.994+0000 7f1150f47640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f114c06c7a0 0x7f114c0b1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:08.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.994+0000 7f1150f47640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f114c06fdc0 0x7f114c0c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:08.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.994+0000 7f1150f47640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f114c15ab00 0x7f114c0c2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:08.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.994+0000 7f1150f47640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114c0c2240 con 0x7f114c06c7a0 2023-09-23T03:19:08.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.994+0000 7f1150f47640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114c0c1ea0 con 0x7f114c06fdc0 2023-09-23T03:19:08.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.994+0000 7f1150f47640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f114c0b1fd0 con 0x7f114c15ab00 2023-09-23T03:19:08.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.994+0000 7f114b7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f114c06c7a0 0x7f114c0b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:08.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.994+0000 7f114b7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f114c06c7a0 0x7f114c0b1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54468/0 (socket says 172.21.15.154:54468) 2023-09-23T03:19:08.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.994+0000 7f114b7fe640 1 -- 172.21.15.154:0/3743777546 learned_addr learned my addr 172.21.15.154:0/3743777546 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:08.997 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.995+0000 7f114b7fe640 1 -- 172.21.15.154:0/3743777546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f114c15ab00 msgr2=0x7f114c0c2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:08.997 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.995+0000 7f114b7fe640 1 --2- 172.21.15.154:0/3743777546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f114c15ab00 0x7f114c0c2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:08.997 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.995+0000 7f114b7fe640 1 -- 172.21.15.154:0/3743777546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f114c06fdc0 msgr2=0x7f114c0c77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:08.997 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.995+0000 7f1143fff640 1 --2- 172.21.15.154:0/3743777546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f114c06fdc0 0x7f114c0c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:08.997 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.995+0000 7f114b7fe640 1 --2- 172.21.15.154:0/3743777546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f114c06fdc0 0x7f114c0c77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:08.997 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.995+0000 7f114b7fe640 1 -- 172.21.15.154:0/3743777546 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11340094d0 con 0x7f114c06c7a0 2023-09-23T03:19:08.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.995+0000 7f114b7fe640 1 --2- 172.21.15.154:0/3743777546 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f114c06c7a0 0x7f114c0b1310 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f113c00d800 tx=0x7f113c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:08.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.995+0000 7f1148ff9640 1 -- 172.21.15.154:0/3743777546 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113c00b6c0 con 0x7f114c06c7a0 2023-09-23T03:19:08.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.995+0000 7f1148ff9640 1 -- 172.21.15.154:0/3743777546 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f113c00b820 con 0x7f114c06c7a0 2023-09-23T03:19:08.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.995+0000 7f1148ff9640 1 -- 172.21.15.154:0/3743777546 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113c00bbb0 con 0x7f114c06c7a0 2023-09-23T03:19:08.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.995+0000 7f1150f47640 1 -- 172.21.15.154:0/3743777546 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f114c0b1a10 con 0x7f114c06c7a0 2023-09-23T03:19:08.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.995+0000 7f1150f47640 1 -- 172.21.15.154:0/3743777546 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f114c14dbf0 con 0x7f114c06c7a0 2023-09-23T03:19:08.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.997+0000 7f1150f47640 1 -- 172.21.15.154:0/3743777546 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f114c066e40 con 0x7f114c06c7a0 2023-09-23T03:19:08.999 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.998+0000 7f1148ff9640 1 -- 172.21.15.154:0/3743777546 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f113c00bd10 con 0x7f114c06c7a0 2023-09-23T03:19:08.999 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.998+0000 7f1148ff9640 1 --2- 172.21.15.154:0/3743777546 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f112c06af30 0x7f112c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:09.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.999+0000 7f1148ff9640 1 -- 172.21.15.154:0/3743777546 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6720+0+0 (secure 0 0 0) 0x7f113c08a110 con 0x7f114c06c7a0 2023-09-23T03:19:09.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.999+0000 7f1143fff640 1 --2- 172.21.15.154:0/3743777546 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f112c06af30 0x7f112c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:09.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:08.999+0000 7f1143fff640 1 --2- 172.21.15.154:0/3743777546 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f112c06af30 0x7f112c06d3f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1134036000 tx=0x7f113403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:09.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.003+0000 7f1148ff9640 1 -- 172.21.15.154:0/3743777546 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f113c054d60 con 0x7f114c06c7a0 2023-09-23T03:19:09.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.227+0000 7f1150f47640 1 -- 172.21.15.154:0/3743777546 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f114c0736b0 con 0x7f114c06c7a0 2023-09-23T03:19:09.230 INFO:tasks.ceph.mon.a.smithi154.stderr:2023-09-23T03:19:09.229+0000 7f3df35a6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-23T03:19:09.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.233+0000 7f1148ff9640 1 -- 172.21.15.154:0/3743777546 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v33) v1 ==== 161+0+0 (secure 0 0 0) 0x7f113c058380 con 0x7f114c06c7a0 2023-09-23T03:19:09.234 INFO:teuthology.orchestra.run.smithi154.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-09-23T03:19:09.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.236+0000 7f1150f47640 1 -- 172.21.15.154:0/3743777546 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f112c06af30 msgr2=0x7f112c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:09.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.236+0000 7f1150f47640 1 --2- 172.21.15.154:0/3743777546 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f112c06af30 0x7f112c06d3f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1134036000 tx=0x7f113403c040 comp rx=0 tx=0).stop 2023-09-23T03:19:09.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.236+0000 7f1150f47640 1 -- 172.21.15.154:0/3743777546 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f114c06c7a0 msgr2=0x7f114c0b1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:09.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.236+0000 7f1150f47640 1 --2- 172.21.15.154:0/3743777546 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f114c06c7a0 0x7f114c0b1310 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f113c00d800 tx=0x7f113c00dcc0 comp rx=0 tx=0).stop 2023-09-23T03:19:09.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.237+0000 7f1150f47640 1 -- 172.21.15.154:0/3743777546 shutdown_connections 2023-09-23T03:19:09.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.237+0000 7f1150f47640 1 --2- 172.21.15.154:0/3743777546 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f112c06af30 0x7f112c06d3f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:09.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.237+0000 7f1150f47640 1 --2- 172.21.15.154:0/3743777546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f114c15ab00 0x7f114c0c2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:09.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.237+0000 7f1150f47640 1 --2- 172.21.15.154:0/3743777546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f114c06fdc0 0x7f114c0c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:09.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.237+0000 7f1150f47640 1 --2- 172.21.15.154:0/3743777546 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f114c06c7a0 0x7f114c0b1310 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:09.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.237+0000 7f1150f47640 1 -- 172.21.15.154:0/3743777546 >> 172.21.15.154:0/3743777546 conn(0x7f114c06a390 msgr2=0x7f114c0cae30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:09.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.237+0000 7f1150f47640 1 -- 172.21.15.154:0/3743777546 shutdown_connections 2023-09-23T03:19:09.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.237+0000 7f1150f47640 1 -- 172.21.15.154:0/3743777546 wait complete. 2023-09-23T03:19:09.245 INFO:tasks.ceph.mds.a.smithi154.stderr: -5874> 2023-09-23T03:18:03.538+0000 7f35329ba640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-09-23T03:19:09.250 INFO:tasks.ceph.mds.a.smithi154.stderr: -3980> 2023-09-23T03:18:11.258+0000 7f35329ba640 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2023-09-23T03:19:09.251 INFO:tasks.ceph.mds.a.smithi154.stderr: -3818> 2023-09-23T03:18:11.259+0000 7f35329ba640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /subdir/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x5634b4970100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(5)0x10000000000:[<0x1/subdir v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-09-23T03:19:09.251 INFO:tasks.ceph.mds.a.smithi154.stderr: -3682> 2023-09-23T03:18:11.260+0000 7f35329ba640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f6 [2,head] /subdir/1 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x5634b4a31700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(7)0x100000001f6:[<0x10000000000/1 v24>,<0x1/subdir v44>]//[6]","memoryvalue":"(7)0x100000001f6:[<0x10000000000/1 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:19:09.251 INFO:tasks.ceph.mds.a.smithi154.stderr: -3659> 2023-09-23T03:18:11.261+0000 7f35329ba640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f8 [2,head] /subdir/3 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x5634b4a30c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(7)0x100000001f8:[<0x10000000000/3 v38>,<0x1/subdir v44>]//[6]","memoryvalue":"(7)0x100000001f8:[<0x10000000000/3 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:19:09.251 INFO:tasks.ceph.mds.a.smithi154.stderr: -3642> 2023-09-23T03:18:11.261+0000 7f35329ba640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f7 [2,head] /subdir/2 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x5634b4a31180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(7)0x100000001f7:[<0x10000000000/2 v32>,<0x1/subdir v44>]//[6]","memoryvalue":"(7)0x100000001f7:[<0x10000000000/2 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:19:09.251 INFO:tasks.ceph.mds.a.smithi154.stderr: -3624> 2023-09-23T03:18:11.262+0000 7f35329ba640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/0 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x5634b4a67b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(7)0x10000000001:[<0x10000000000/0 v16>,<0x1/subdir v44>]//[6]","memoryvalue":"(7)0x10000000001:[<0x10000000000/0 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:19:09.251 INFO:tasks.ceph.mds.a.smithi154.stderr: -3608> 2023-09-23T03:18:11.262+0000 7f35329ba640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f9 [2,head] /subdir/4 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x5634b4a30680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(7)0x100000001f9:[<0x10000000000/4 v40>,<0x1/subdir v44>]//[6]","memoryvalue":"(7)0x100000001f9:[<0x10000000000/4 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:19:09.252 INFO:tasks.ceph.mds.a.smithi154.stderr: -3327> 2023-09-23T03:18:11.269+0000 7f35329ba640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v5 ap=5 snaprealm=0x5634b49686c0 DIRTYPARENT f(v1 1=0+1) n(v0 rc2023-09-23T03:18:10.979060+0000 1=0+1)/n() (idft sync r=1) (inest sync r=1) (ifile sync->mix r=1) (iversion lock) caps={6057=pLsXs/-@3} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x5634b496f600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(5)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=0+1)","ondisk_value.rstat":"n(v0 2=0+2)","memory_value.dirstat":"f(v1 1=0+1)","memory_value.rstat":"n(v0 rc2023-09-23T03:18:10.979060+0000 1=0+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-09-23T03:19:09.257 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-09-23T03:19:09.302 INFO:tasks.ceph.mds.a.smithi154.stdout:starting mds.a at 2023-09-23T03:19:09.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.449+0000 7f2b01290640 1 Processor -- start 2023-09-23T03:19:09.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.449+0000 7f2b01290640 1 -- start start 2023-09-23T03:19:09.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.451+0000 7f2b01290640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2afc0c8d70 0x7f2afc15c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:09.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.451+0000 7f2b01290640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2afc071290 0x7f2afc0ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:09.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.451+0000 7f2b01290640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2afc0c9d20 con 0x7f2afc0c9150 2023-09-23T03:19:09.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.451+0000 7f2b01290640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2afc071850 con 0x7f2afc0c8d70 2023-09-23T03:19:09.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.451+0000 7f2b01290640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2afc071b30 con 0x7f2afc071290 2023-09-23T03:19:09.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.451+0000 7f2afaffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2afc0c8d70 0x7f2afc15c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:09.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.451+0000 7f2afaffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2afc0c8d70 0x7f2afc15c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54508/0 (socket says 172.21.15.154:54508) 2023-09-23T03:19:09.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.451+0000 7f2afaffd640 1 -- 172.21.15.154:0/2881842754 learned_addr learned my addr 172.21.15.154:0/2881842754 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:09.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.451+0000 7f2afb7fe640 1 --2- 172.21.15.154:0/2881842754 >> v2:172.21.15.190:3300/0 conn(0x7f2afc071290 0x7f2afc0ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:09.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.452+0000 7f2afaffd640 1 -- 172.21.15.154:0/2881842754 >> v2:172.21.15.190:3300/0 conn(0x7f2afc071290 msgr2=0x7f2afc0ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:09.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.452+0000 7f2afaffd640 1 --2- 172.21.15.154:0/2881842754 >> v2:172.21.15.190:3300/0 conn(0x7f2afc071290 0x7f2afc0ca1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:09.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.452+0000 7f2afaffd640 1 -- 172.21.15.154:0/2881842754 >> v1:172.21.15.190:6789/0 conn(0x7f2afc0c9150 legacy=0x7f2afc15cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:09.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.452+0000 7f2afaffd640 1 -- 172.21.15.154:0/2881842754 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2afc071e10 con 0x7f2afc0c8d70 2023-09-23T03:19:09.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.452+0000 7f2afb7fe640 1 --2- 172.21.15.154:0/2881842754 >> v2:172.21.15.190:3300/0 conn(0x7f2afc071290 0x7f2afc0ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:09.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.453+0000 7f2afaffd640 1 --2- 172.21.15.154:0/2881842754 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2afc0c8d70 0x7f2afc15c6d0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2afc0b3820 tx=0x7f2aec02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5aa6be9187e2379 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:09.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.453+0000 7f2af9ffb640 1 -- 172.21.15.154:0/2881842754 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec031870 con 0x7f2afc0c8d70 2023-09-23T03:19:09.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.453+0000 7f2af9ffb640 1 -- 172.21.15.154:0/2881842754 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aec0319d0 con 0x7f2afc0c8d70 2023-09-23T03:19:09.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.453+0000 7f2af9ffb640 1 -- 172.21.15.154:0/2881842754 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec031d60 con 0x7f2afc0c8d70 2023-09-23T03:19:09.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.454+0000 7f2b01290640 1 -- 172.21.15.154:0/2881842754 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2afc0c8d70 msgr2=0x7f2afc15c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:09.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.454+0000 7f2b01290640 1 --2- 172.21.15.154:0/2881842754 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2afc0c8d70 0x7f2afc15c6d0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2afc0b3820 tx=0x7f2aec02b910 comp rx=0 tx=0).stop 2023-09-23T03:19:09.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.454+0000 7f2b01290640 1 -- 172.21.15.154:0/2881842754 shutdown_connections 2023-09-23T03:19:09.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.454+0000 7f2b01290640 1 --2- 172.21.15.154:0/2881842754 >> v2:172.21.15.190:3300/0 conn(0x7f2afc071290 0x7f2afc0ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:09.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.454+0000 7f2b01290640 1 --2- 172.21.15.154:0/2881842754 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2afc0c8d70 0x7f2afc15c6d0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:09.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.454+0000 7f2b01290640 1 -- 172.21.15.154:0/2881842754 >> 172.21.15.154:0/2881842754 conn(0x7f2afc06c010 msgr2=0x7f2afc06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:09.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.454+0000 7f2b01290640 1 -- 172.21.15.154:0/2881842754 shutdown_connections 2023-09-23T03:19:09.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.454+0000 7f2b01290640 1 -- 172.21.15.154:0/2881842754 wait complete. 2023-09-23T03:19:09.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.456+0000 7f2b01290640 1 Processor -- start 2023-09-23T03:19:09.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.456+0000 7f2b01290640 1 -- start start 2023-09-23T03:19:09.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.457+0000 7f2b01290640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2afc071290 0x7f2afc14dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:09.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.457+0000 7f2b01290640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2afc0c8d70 0x7f2afc158850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:09.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.457+0000 7f2b01290640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2afc0c9150 0x7f2afc158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:09.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.457+0000 7f2b01290640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc151f30 con 0x7f2afc071290 2023-09-23T03:19:09.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.457+0000 7f2b01290640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc151590 con 0x7f2afc0c9150 2023-09-23T03:19:09.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.457+0000 7f2b01290640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc151870 con 0x7f2afc0c8d70 2023-09-23T03:19:09.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.457+0000 7f2afaffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2afc071290 0x7f2afc14dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:09.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.457+0000 7f2afaffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2afc071290 0x7f2afc14dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54520/0 (socket says 172.21.15.154:54520) 2023-09-23T03:19:09.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.457+0000 7f2afaffd640 1 -- 172.21.15.154:0/2448950066 learned_addr learned my addr 172.21.15.154:0/2448950066 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:09.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.457+0000 7f2afaffd640 1 -- 172.21.15.154:0/2448950066 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2afc0c8d70 msgr2=0x7f2afc158850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:09.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.457+0000 7f2afaffd640 1 --2- 172.21.15.154:0/2448950066 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2afc0c8d70 0x7f2afc158850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:09.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.457+0000 7f2afaffd640 1 -- 172.21.15.154:0/2448950066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2afc0c9150 msgr2=0x7f2afc158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:09.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.457+0000 7f2afaffd640 1 --2- 172.21.15.154:0/2448950066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2afc0c9150 0x7f2afc158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:09.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.458+0000 7f2afaffd640 1 -- 172.21.15.154:0/2448950066 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aec0094d0 con 0x7f2afc071290 2023-09-23T03:19:09.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.458+0000 7f2afaffd640 1 --2- 172.21.15.154:0/2448950066 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2afc071290 0x7f2afc14dcd0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2aec000c00 tx=0x7f2aec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:09.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.458+0000 7f2ae37fe640 1 -- 172.21.15.154:0/2448950066 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec031840 con 0x7f2afc071290 2023-09-23T03:19:09.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.458+0000 7f2ae37fe640 1 -- 172.21.15.154:0/2448950066 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aec0443f0 con 0x7f2afc071290 2023-09-23T03:19:09.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.458+0000 7f2b01290640 1 -- 172.21.15.154:0/2448950066 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2afc151b50 con 0x7f2afc071290 2023-09-23T03:19:09.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.458+0000 7f2ae37fe640 1 -- 172.21.15.154:0/2448950066 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec0447a0 con 0x7f2afc071290 2023-09-23T03:19:09.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.458+0000 7f2b01290640 1 -- 172.21.15.154:0/2448950066 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2afc1511d0 con 0x7f2afc071290 2023-09-23T03:19:09.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.460+0000 7f2b01290640 1 -- 172.21.15.154:0/2448950066 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2afc04e920 con 0x7f2afc071290 2023-09-23T03:19:09.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.461+0000 7f2ae37fe640 1 -- 172.21.15.154:0/2448950066 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f2aec044900 con 0x7f2afc071290 2023-09-23T03:19:09.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.462+0000 7f2ae37fe640 1 --2- 172.21.15.154:0/2448950066 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2ae806af30 0x7f2ae806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:09.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.463+0000 7f2ae37fe640 1 -- 172.21.15.154:0/2448950066 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6806+0+0 (secure 0 0 0) 0x7f2aec02d040 con 0x7f2afc071290 2023-09-23T03:19:09.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.463+0000 7f2afa7fc640 1 --2- 172.21.15.154:0/2448950066 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2ae806af30 0x7f2ae806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:09.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.464+0000 7f2afa7fc640 1 --2- 172.21.15.154:0/2448950066 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2ae806af30 0x7f2ae806d3f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2afc0b4070 tx=0x7f2af0006ba0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:09.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.467+0000 7f2ae37fe640 1 -- 172.21.15.154:0/2448950066 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f2aec077110 con 0x7f2afc071290 2023-09-23T03:19:09.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:09.688+0000 7f2b01290640 1 -- 172.21.15.154:0/2448950066 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2afc06cbc0 con 0x7f2afc071290 2023-09-23T03:19:10.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.236+0000 7f2ae37fe640 1 -- 172.21.15.154:0/2448950066 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v34) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2aec07a730 con 0x7f2afc071290 2023-09-23T03:19:10.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.240+0000 7f2b01290640 1 -- 172.21.15.154:0/2448950066 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2ae806af30 msgr2=0x7f2ae806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:10.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.240+0000 7f2b01290640 1 --2- 172.21.15.154:0/2448950066 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2ae806af30 0x7f2ae806d3f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2afc0b4070 tx=0x7f2af0006ba0 comp rx=0 tx=0).stop 2023-09-23T03:19:10.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.240+0000 7f2b01290640 1 -- 172.21.15.154:0/2448950066 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2afc071290 msgr2=0x7f2afc14dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:10.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.240+0000 7f2b01290640 1 --2- 172.21.15.154:0/2448950066 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2afc071290 0x7f2afc14dcd0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2aec000c00 tx=0x7f2aec0046c0 comp rx=0 tx=0).stop 2023-09-23T03:19:10.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.240+0000 7f2b01290640 1 -- 172.21.15.154:0/2448950066 shutdown_connections 2023-09-23T03:19:10.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.240+0000 7f2b01290640 1 --2- 172.21.15.154:0/2448950066 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2ae806af30 0x7f2ae806d3f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:10.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.240+0000 7f2b01290640 1 --2- 172.21.15.154:0/2448950066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2afc0c9150 0x7f2afc158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:10.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.240+0000 7f2b01290640 1 --2- 172.21.15.154:0/2448950066 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2afc0c8d70 0x7f2afc158850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:10.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.240+0000 7f2b01290640 1 --2- 172.21.15.154:0/2448950066 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2afc071290 0x7f2afc14dcd0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:10.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.240+0000 7f2b01290640 1 -- 172.21.15.154:0/2448950066 >> 172.21.15.154:0/2448950066 conn(0x7f2afc06c010 msgr2=0x7f2afc0caf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:10.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.240+0000 7f2b01290640 1 -- 172.21.15.154:0/2448950066 shutdown_connections 2023-09-23T03:19:10.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.240+0000 7f2b01290640 1 -- 172.21.15.154:0/2448950066 wait complete. 2023-09-23T03:19:10.264 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-09-23T03:19:10.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.527+0000 7f0e149d0640 1 Processor -- start 2023-09-23T03:19:10.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.528+0000 7f0e149d0640 1 -- start start 2023-09-23T03:19:10.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.529+0000 7f0e149d0640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0e1015ce20 0x7f0e100ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:10.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.529+0000 7f0e149d0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e100cc8b0 0x7f0e100b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:10.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.529+0000 7f0e149d0640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0e100b25f0 0x7f0e10160a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:10.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.529+0000 7f0e149d0640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e100b2a40 con 0x7f0e100b25f0 2023-09-23T03:19:10.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.529+0000 7f0e149d0640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e100cce20 con 0x7f0e1015ce20 2023-09-23T03:19:10.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.529+0000 7f0e149d0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e100cd100 con 0x7f0e100cc8b0 2023-09-23T03:19:10.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.530+0000 7f0e0ffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0e100b25f0 0x7f0e10160a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:10.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.530+0000 7f0e0ffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0e100b25f0 0x7f0e10160a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54534/0 (socket says 172.21.15.154:54534) 2023-09-23T03:19:10.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.530+0000 7f0e0ffff640 1 -- 172.21.15.154:0/2984659624 learned_addr learned my addr 172.21.15.154:0/2984659624 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:10.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.530+0000 7f0e0effd640 1 --2- 172.21.15.154:0/2984659624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e100cc8b0 0x7f0e100b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:10.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.530+0000 7f0e0f7fe640 1 --2- 172.21.15.154:0/2984659624 >> v2:172.21.15.190:3300/0 conn(0x7f0e1015ce20 0x7f0e100ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:10.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.530+0000 7f0e0ffff640 1 -- 172.21.15.154:0/2984659624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e100cc8b0 msgr2=0x7f0e100b20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:10.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.530+0000 7f0e0ffff640 1 --2- 172.21.15.154:0/2984659624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e100cc8b0 0x7f0e100b20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:10.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.530+0000 7f0e0ffff640 1 -- 172.21.15.154:0/2984659624 >> v2:172.21.15.190:3300/0 conn(0x7f0e1015ce20 msgr2=0x7f0e100ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:10.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.530+0000 7f0e0ffff640 1 --2- 172.21.15.154:0/2984659624 >> v2:172.21.15.190:3300/0 conn(0x7f0e1015ce20 0x7f0e100ce3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:10.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.530+0000 7f0e0ffff640 1 -- 172.21.15.154:0/2984659624 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e100cd3e0 con 0x7f0e100b25f0 2023-09-23T03:19:10.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.530+0000 7f0e0effd640 1 --2- 172.21.15.154:0/2984659624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e100cc8b0 0x7f0e100b20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:10.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.531+0000 7f0e0ffff640 1 --2- 172.21.15.154:0/2984659624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e100b25f0 0x7f0e10160a00 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f0e10073930 tx=0x7f0e0802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d7fe05976c60ce0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:10.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.532+0000 7f0e0e7fc640 1 -- 172.21.15.154:0/2984659624 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e08031860 con 0x7f0e100b25f0 2023-09-23T03:19:10.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.532+0000 7f0e0e7fc640 1 -- 172.21.15.154:0/2984659624 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e080319c0 con 0x7f0e100b25f0 2023-09-23T03:19:10.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.532+0000 7f0e0e7fc640 1 -- 172.21.15.154:0/2984659624 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e08031d50 con 0x7f0e100b25f0 2023-09-23T03:19:10.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.532+0000 7f0e149d0640 1 -- 172.21.15.154:0/2984659624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e100b25f0 msgr2=0x7f0e10160a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:10.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.532+0000 7f0e149d0640 1 --2- 172.21.15.154:0/2984659624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e100b25f0 0x7f0e10160a00 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f0e10073930 tx=0x7f0e0802b900 comp rx=0 tx=0).stop 2023-09-23T03:19:10.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.532+0000 7f0e149d0640 1 -- 172.21.15.154:0/2984659624 shutdown_connections 2023-09-23T03:19:10.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.532+0000 7f0e149d0640 1 --2- 172.21.15.154:0/2984659624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e100b25f0 0x7f0e10160a00 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:10.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.532+0000 7f0e149d0640 1 --2- 172.21.15.154:0/2984659624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e100cc8b0 0x7f0e100b20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:10.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.532+0000 7f0e149d0640 1 --2- 172.21.15.154:0/2984659624 >> v2:172.21.15.190:3300/0 conn(0x7f0e1015ce20 0x7f0e100ce3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:10.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.532+0000 7f0e149d0640 1 -- 172.21.15.154:0/2984659624 >> 172.21.15.154:0/2984659624 conn(0x7f0e10069cd0 msgr2=0x7f0e1006a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:10.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.532+0000 7f0e149d0640 1 -- 172.21.15.154:0/2984659624 shutdown_connections 2023-09-23T03:19:10.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.533+0000 7f0e149d0640 1 -- 172.21.15.154:0/2984659624 wait complete. 2023-09-23T03:19:10.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.534+0000 7f0e149d0640 1 Processor -- start 2023-09-23T03:19:10.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.534+0000 7f0e149d0640 1 -- start start 2023-09-23T03:19:10.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.535+0000 7f0e149d0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e100b25f0 0x7f0e100c8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:10.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.535+0000 7f0e149d0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e100cc8b0 0x7f0e101f3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:10.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.535+0000 7f0e149d0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e1015ce20 0x7f0e101576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:10.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.535+0000 7f0e149d0640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e101f3660 con 0x7f0e1015ce20 2023-09-23T03:19:10.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.535+0000 7f0e149d0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e100c8d30 con 0x7f0e100b25f0 2023-09-23T03:19:10.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.535+0000 7f0e149d0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e100c9010 con 0x7f0e100cc8b0 2023-09-23T03:19:10.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.536+0000 7f0e0ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e1015ce20 0x7f0e101576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:10.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.536+0000 7f0e0ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e1015ce20 0x7f0e101576d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54538/0 (socket says 172.21.15.154:54538) 2023-09-23T03:19:10.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.536+0000 7f0e0ffff640 1 -- 172.21.15.154:0/2746574698 learned_addr learned my addr 172.21.15.154:0/2746574698 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:10.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.536+0000 7f0e0effd640 1 --2- 172.21.15.154:0/2746574698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e100cc8b0 0x7f0e101f3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:10.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.536+0000 7f0e0f7fe640 1 --2- 172.21.15.154:0/2746574698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e100b25f0 0x7f0e100c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:10.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.536+0000 7f0e0ffff640 1 -- 172.21.15.154:0/2746574698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e100cc8b0 msgr2=0x7f0e101f3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:10.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.536+0000 7f0e0ffff640 1 --2- 172.21.15.154:0/2746574698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e100cc8b0 0x7f0e101f3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:10.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.536+0000 7f0e0ffff640 1 -- 172.21.15.154:0/2746574698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e100b25f0 msgr2=0x7f0e100c8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:10.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.536+0000 7f0e0ffff640 1 --2- 172.21.15.154:0/2746574698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e100b25f0 0x7f0e100c8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:10.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.536+0000 7f0e0ffff640 1 -- 172.21.15.154:0/2746574698 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e080094d0 con 0x7f0e1015ce20 2023-09-23T03:19:10.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.536+0000 7f0e0f7fe640 1 --2- 172.21.15.154:0/2746574698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e100b25f0 0x7f0e100c8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:10.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.536+0000 7f0e0effd640 1 --2- 172.21.15.154:0/2746574698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e100cc8b0 0x7f0e101f3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:10.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.537+0000 7f0e0ffff640 1 --2- 172.21.15.154:0/2746574698 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e1015ce20 0x7f0e101576d0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f0e080090b0 tx=0x7f0e080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:10.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.537+0000 7f0e07fff640 1 -- 172.21.15.154:0/2746574698 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0803ac60 con 0x7f0e1015ce20 2023-09-23T03:19:10.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.537+0000 7f0e07fff640 1 -- 172.21.15.154:0/2746574698 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e0803adc0 con 0x7f0e1015ce20 2023-09-23T03:19:10.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.537+0000 7f0e149d0640 1 -- 172.21.15.154:0/2746574698 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e100c92f0 con 0x7f0e1015ce20 2023-09-23T03:19:10.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.537+0000 7f0e07fff640 1 -- 172.21.15.154:0/2746574698 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e08044630 con 0x7f0e1015ce20 2023-09-23T03:19:10.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.537+0000 7f0e149d0640 1 -- 172.21.15.154:0/2746574698 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e101e68e0 con 0x7f0e1015ce20 2023-09-23T03:19:10.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.538+0000 7f0e149d0640 1 -- 172.21.15.154:0/2746574698 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e100cf890 con 0x7f0e1015ce20 2023-09-23T03:19:10.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.539+0000 7f0e07fff640 1 -- 172.21.15.154:0/2746574698 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f0e0804d450 con 0x7f0e1015ce20 2023-09-23T03:19:10.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.540+0000 7f0e07fff640 1 --2- 172.21.15.154:0/2746574698 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0df006af30 0x7f0df006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:10.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.540+0000 7f0e07fff640 1 -- 172.21.15.154:0/2746574698 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6806+0+0 (secure 0 0 0) 0x7f0e080ac0d0 con 0x7f0e1015ce20 2023-09-23T03:19:10.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.540+0000 7f0e0f7fe640 1 --2- 172.21.15.154:0/2746574698 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0df006af30 0x7f0df006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:10.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.541+0000 7f0e0f7fe640 1 --2- 172.21.15.154:0/2746574698 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0df006af30 0x7f0df006d3f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0e00007330 tx=0x7f0e0000b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:10.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.545+0000 7f0e07fff640 1 -- 172.21.15.154:0/2746574698 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f0e08076cb0 con 0x7f0e1015ce20 2023-09-23T03:19:10.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:10.739+0000 7f0e149d0640 1 -- 172.21.15.154:0/2746574698 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0e10067390 con 0x7f0e1015ce20 2023-09-23T03:19:11.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.361+0000 7f0e07fff640 1 -- 172.21.15.154:0/2746574698 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v51) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0e080350d0 con 0x7f0e1015ce20 2023-09-23T03:19:11.362 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_metadata' removed 2023-09-23T03:19:11.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.364+0000 7f0e149d0640 1 -- 172.21.15.154:0/2746574698 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0df006af30 msgr2=0x7f0df006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:11.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.364+0000 7f0e149d0640 1 --2- 172.21.15.154:0/2746574698 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0df006af30 0x7f0df006d3f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0e00007330 tx=0x7f0e0000b040 comp rx=0 tx=0).stop 2023-09-23T03:19:11.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.364+0000 7f0e149d0640 1 -- 172.21.15.154:0/2746574698 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e1015ce20 msgr2=0x7f0e101576d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:11.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.364+0000 7f0e149d0640 1 --2- 172.21.15.154:0/2746574698 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e1015ce20 0x7f0e101576d0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f0e080090b0 tx=0x7f0e080046c0 comp rx=0 tx=0).stop 2023-09-23T03:19:11.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.365+0000 7f0e149d0640 1 -- 172.21.15.154:0/2746574698 shutdown_connections 2023-09-23T03:19:11.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.365+0000 7f0e149d0640 1 --2- 172.21.15.154:0/2746574698 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0df006af30 0x7f0df006d3f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:11.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.365+0000 7f0e149d0640 1 --2- 172.21.15.154:0/2746574698 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e1015ce20 0x7f0e101576d0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:11.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.365+0000 7f0e149d0640 1 --2- 172.21.15.154:0/2746574698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e100cc8b0 0x7f0e101f3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:11.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.365+0000 7f0e149d0640 1 --2- 172.21.15.154:0/2746574698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e100b25f0 0x7f0e100c8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:11.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.365+0000 7f0e149d0640 1 -- 172.21.15.154:0/2746574698 >> 172.21.15.154:0/2746574698 conn(0x7f0e10069cd0 msgr2=0x7f0e1015c110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:11.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.365+0000 7f0e149d0640 1 -- 172.21.15.154:0/2746574698 shutdown_connections 2023-09-23T03:19:11.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.365+0000 7f0e149d0640 1 -- 172.21.15.154:0/2746574698 wait complete. 2023-09-23T03:19:11.391 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-09-23T03:19:11.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.605+0000 7f76bfdee640 1 Processor -- start 2023-09-23T03:19:11.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.605+0000 7f76bfdee640 1 -- start start 2023-09-23T03:19:11.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.606+0000 7f76bfdee640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f76b80700f0 0x7f76b80c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:11.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.606+0000 7f76bfdee640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f76b80cb460 0x7f76b80b1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:11.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.606+0000 7f76bfdee640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76b80c95b0 con 0x7f76b815abd0 2023-09-23T03:19:11.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.606+0000 7f76bfdee640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76b80c9890 con 0x7f76b80700f0 2023-09-23T03:19:11.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.606+0000 7f76bfdee640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76b80c9b70 con 0x7f76b80cb460 2023-09-23T03:19:11.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.607+0000 7f76bdb8a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f76b80700f0 0x7f76b80c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:11.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.607+0000 7f76bdb8a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f76b80700f0 0x7f76b80c8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54548/0 (socket says 172.21.15.154:54548) 2023-09-23T03:19:11.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.607+0000 7f76bdb8a640 1 -- 172.21.15.154:0/578090588 learned_addr learned my addr 172.21.15.154:0/578090588 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:11.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.607+0000 7f76bdb8a640 1 -- 172.21.15.154:0/578090588 >> v2:172.21.15.190:3300/0 conn(0x7f76b80cb460 msgr2=0x7f76b80b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:11.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.607+0000 7f76bdb8a640 1 --2- 172.21.15.154:0/578090588 >> v2:172.21.15.190:3300/0 conn(0x7f76b80cb460 0x7f76b80b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:11.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.607+0000 7f76bdb8a640 1 -- 172.21.15.154:0/578090588 >> v1:172.21.15.190:6789/0 conn(0x7f76b815abd0 legacy=0x7f76b80704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:11.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.607+0000 7f76bdb8a640 1 -- 172.21.15.154:0/578090588 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b80c9e50 con 0x7f76b80700f0 2023-09-23T03:19:11.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.608+0000 7f76bdb8a640 1 --2- 172.21.15.154:0/578090588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76b80700f0 0x7f76b80c8fd0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f76b806e060 tx=0x7f76b402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de71fbb5a0c45946 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:11.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.609+0000 7f76bcb88640 1 -- 172.21.15.154:0/578090588 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b4031870 con 0x7f76b80700f0 2023-09-23T03:19:11.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.609+0000 7f76bcb88640 1 -- 172.21.15.154:0/578090588 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b40319d0 con 0x7f76b80700f0 2023-09-23T03:19:11.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.609+0000 7f76bcb88640 1 -- 172.21.15.154:0/578090588 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b4031d60 con 0x7f76b80700f0 2023-09-23T03:19:11.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.609+0000 7f76bfdee640 1 -- 172.21.15.154:0/578090588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76b80700f0 msgr2=0x7f76b80c8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:11.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.609+0000 7f76bfdee640 1 --2- 172.21.15.154:0/578090588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76b80700f0 0x7f76b80c8fd0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f76b806e060 tx=0x7f76b402b910 comp rx=0 tx=0).stop 2023-09-23T03:19:11.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.610+0000 7f76bfdee640 1 -- 172.21.15.154:0/578090588 shutdown_connections 2023-09-23T03:19:11.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.610+0000 7f76bfdee640 1 --2- 172.21.15.154:0/578090588 >> v2:172.21.15.190:3300/0 conn(0x7f76b80cb460 0x7f76b80b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:11.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.610+0000 7f76bfdee640 1 --2- 172.21.15.154:0/578090588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76b80700f0 0x7f76b80c8fd0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:11.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.610+0000 7f76bfdee640 1 -- 172.21.15.154:0/578090588 >> 172.21.15.154:0/578090588 conn(0x7f76b806a520 msgr2=0x7f76b806bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:11.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.610+0000 7f76bfdee640 1 -- 172.21.15.154:0/578090588 shutdown_connections 2023-09-23T03:19:11.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.610+0000 7f76bfdee640 1 -- 172.21.15.154:0/578090588 wait complete. 2023-09-23T03:19:11.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.611+0000 7f76bfdee640 1 Processor -- start 2023-09-23T03:19:11.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.612+0000 7f76bfdee640 1 -- start start 2023-09-23T03:19:11.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.612+0000 7f76bfdee640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76b80700f0 0x7f76b81e0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:11.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.612+0000 7f76bfdee640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76b80cb460 0x7f76b8145b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:11.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.613+0000 7f76bfdee640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76b815abd0 0x7f76b80c7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:11.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.613+0000 7f76bfdee640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b80c82c0 con 0x7f76b80cb460 2023-09-23T03:19:11.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.613+0000 7f76bfdee640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b80c77a0 con 0x7f76b815abd0 2023-09-23T03:19:11.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.613+0000 7f76bfdee640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76b80c7a20 con 0x7f76b80700f0 2023-09-23T03:19:11.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.613+0000 7f76bd389640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76b80cb460 0x7f76b8145b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:11.615 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.613+0000 7f76bd389640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76b80cb460 0x7f76b8145b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54552/0 (socket says 172.21.15.154:54552) 2023-09-23T03:19:11.615 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.613+0000 7f76bd389640 1 -- 172.21.15.154:0/532685229 learned_addr learned my addr 172.21.15.154:0/532685229 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:11.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.613+0000 7f76be38b640 1 --2- 172.21.15.154:0/532685229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76b815abd0 0x7f76b80c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:11.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.613+0000 7f76bdb8a640 1 --2- 172.21.15.154:0/532685229 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76b80700f0 0x7f76b81e0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:11.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.613+0000 7f76bd389640 1 -- 172.21.15.154:0/532685229 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76b80700f0 msgr2=0x7f76b81e0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:11.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.613+0000 7f76bd389640 1 --2- 172.21.15.154:0/532685229 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76b80700f0 0x7f76b81e0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:11.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.613+0000 7f76bd389640 1 -- 172.21.15.154:0/532685229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76b815abd0 msgr2=0x7f76b80c7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:11.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.613+0000 7f76bd389640 1 --2- 172.21.15.154:0/532685229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76b815abd0 0x7f76b80c7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:11.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.613+0000 7f76bd389640 1 -- 172.21.15.154:0/532685229 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b40094d0 con 0x7f76b80cb460 2023-09-23T03:19:11.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.613+0000 7f76be38b640 1 --2- 172.21.15.154:0/532685229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76b815abd0 0x7f76b80c7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:11.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.614+0000 7f76bd389640 1 --2- 172.21.15.154:0/532685229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76b80cb460 0x7f76b8145b80 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f76a800b3c0 tx=0x7f76a800b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:11.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.614+0000 7f76ae7fc640 1 -- 172.21.15.154:0/532685229 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a8011830 con 0x7f76b80cb460 2023-09-23T03:19:11.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.614+0000 7f76ae7fc640 1 -- 172.21.15.154:0/532685229 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a8011990 con 0x7f76b80cb460 2023-09-23T03:19:11.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.614+0000 7f76ae7fc640 1 -- 172.21.15.154:0/532685229 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a8011d60 con 0x7f76b80cb460 2023-09-23T03:19:11.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.614+0000 7f76bfdee640 1 -- 172.21.15.154:0/532685229 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76b80c7d00 con 0x7f76b80cb460 2023-09-23T03:19:11.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.614+0000 7f76bfdee640 1 -- 172.21.15.154:0/532685229 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76b81e1710 con 0x7f76b80cb460 2023-09-23T03:19:11.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.615+0000 7f76bfdee640 1 -- 172.21.15.154:0/532685229 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76b81e1930 con 0x7f76b80cb460 2023-09-23T03:19:11.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.616+0000 7f76ae7fc640 1 -- 172.21.15.154:0/532685229 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f76a801b460 con 0x7f76b80cb460 2023-09-23T03:19:11.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.617+0000 7f76ae7fc640 1 --2- 172.21.15.154:0/532685229 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f769c06af30 0x7f769c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:11.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.617+0000 7f76ae7fc640 1 -- 172.21.15.154:0/532685229 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6368+0+0 (secure 0 0 0) 0x7f76a8089770 con 0x7f76b80cb460 2023-09-23T03:19:11.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.617+0000 7f76bdb8a640 1 --2- 172.21.15.154:0/532685229 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f769c06af30 0x7f769c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:11.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.618+0000 7f76bdb8a640 1 --2- 172.21.15.154:0/532685229 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f769c06af30 0x7f769c06d3f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f76b4036000 tx=0x7f76b403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:11.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.622+0000 7f76ae7fc640 1 -- 172.21.15.154:0/532685229 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f76a80545b0 con 0x7f76b80cb460 2023-09-23T03:19:11.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:11.817+0000 7f76bfdee640 1 -- 172.21.15.154:0/532685229 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f76b80c8fd0 con 0x7f76b80cb460 2023-09-23T03:19:11.904 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:19:11.914 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:19:12.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.364+0000 7f76ae7fc640 1 -- 172.21.15.154:0/532685229 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v52) v1 ==== 169+0+0 (secure 0 0 0) 0x7f76a8057bd0 con 0x7f76b80cb460 2023-09-23T03:19:12.366 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_data' removed 2023-09-23T03:19:12.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.369+0000 7f76bfdee640 1 -- 172.21.15.154:0/532685229 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f769c06af30 msgr2=0x7f769c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:12.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.369+0000 7f76bfdee640 1 --2- 172.21.15.154:0/532685229 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f769c06af30 0x7f769c06d3f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f76b4036000 tx=0x7f76b403c040 comp rx=0 tx=0).stop 2023-09-23T03:19:12.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.369+0000 7f76bfdee640 1 -- 172.21.15.154:0/532685229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76b80cb460 msgr2=0x7f76b8145b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:12.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.369+0000 7f76bfdee640 1 --2- 172.21.15.154:0/532685229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76b80cb460 0x7f76b8145b80 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f76a800b3c0 tx=0x7f76a800b880 comp rx=0 tx=0).stop 2023-09-23T03:19:12.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.369+0000 7f76bfdee640 1 -- 172.21.15.154:0/532685229 shutdown_connections 2023-09-23T03:19:12.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.370+0000 7f76bfdee640 1 --2- 172.21.15.154:0/532685229 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f769c06af30 0x7f769c06d3f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:12.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.370+0000 7f76bfdee640 1 --2- 172.21.15.154:0/532685229 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76b815abd0 0x7f76b80c7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:12.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.370+0000 7f76bfdee640 1 --2- 172.21.15.154:0/532685229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76b80cb460 0x7f76b8145b80 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:12.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.370+0000 7f76bfdee640 1 --2- 172.21.15.154:0/532685229 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76b80700f0 0x7f76b81e0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:12.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.370+0000 7f76bfdee640 1 -- 172.21.15.154:0/532685229 >> 172.21.15.154:0/532685229 conn(0x7f76b806a520 msgr2=0x7f76b80c3f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:12.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.370+0000 7f76bfdee640 1 -- 172.21.15.154:0/532685229 shutdown_connections 2023-09-23T03:19:12.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.370+0000 7f76bfdee640 1 -- 172.21.15.154:0/532685229 wait complete. 2023-09-23T03:19:12.395 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_extra cephfs_data_extra --yes-i-really-really-mean-it 2023-09-23T03:19:12.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.598+0000 7f5edcc5d640 1 Processor -- start 2023-09-23T03:19:12.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.598+0000 7f5edcc5d640 1 -- start start 2023-09-23T03:19:12.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.600+0000 7f5edcc5d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5ed8069cd0 0x7f5ed80c8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:12.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.600+0000 7f5edcc5d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ed80c85b0 0x7f5ed80cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:12.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.600+0000 7f5edcc5d640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed8153f40 con 0x7f5ed8153b60 2023-09-23T03:19:12.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.600+0000 7f5edcc5d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ed80cd540 con 0x7f5ed8069cd0 2023-09-23T03:19:12.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.600+0000 7f5edcc5d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed80c8a40 con 0x7f5ed80c85b0 2023-09-23T03:19:12.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.600+0000 7f5ecdd9b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5ed8069cd0 0x7f5ed80c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:12.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.601+0000 7f5ecdd9b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5ed8069cd0 0x7f5ed80c8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54556/0 (socket says 172.21.15.154:54556) 2023-09-23T03:19:12.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.601+0000 7f5ecdd9b640 1 -- 172.21.15.154:0/1793141969 learned_addr learned my addr 172.21.15.154:0/1793141969 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:12.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.601+0000 7f5ed6d9d640 1 --2- 172.21.15.154:0/1793141969 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ed80c85b0 0x7f5ed80cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:12.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.601+0000 7f5ecdd9b640 1 -- 172.21.15.154:0/1793141969 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ed80c85b0 msgr2=0x7f5ed80cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:12.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.601+0000 7f5ecdd9b640 1 --2- 172.21.15.154:0/1793141969 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ed80c85b0 0x7f5ed80cdcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:12.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.601+0000 7f5ecdd9b640 1 -- 172.21.15.154:0/1793141969 >> v1:172.21.15.190:6789/0 conn(0x7f5ed8153b60 legacy=0x7f5ed80659f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:12.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.601+0000 7f5ecdd9b640 1 -- 172.21.15.154:0/1793141969 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed815df50 con 0x7f5ed8069cd0 2023-09-23T03:19:12.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.601+0000 7f5ed6d9d640 1 --2- 172.21.15.154:0/1793141969 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ed80c85b0 0x7f5ed80cdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:12.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.602+0000 7f5ecdd9b640 1 --2- 172.21.15.154:0/1793141969 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ed8069cd0 0x7f5ed80c8070 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0009870 tx=0x7f5ec002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f0aed8310b56132 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:12.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.603+0000 7f5ed5d9b640 1 -- 172.21.15.154:0/1793141969 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0031640 con 0x7f5ed8069cd0 2023-09-23T03:19:12.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.603+0000 7f5ed5d9b640 1 -- 172.21.15.154:0/1793141969 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec00317a0 con 0x7f5ed8069cd0 2023-09-23T03:19:12.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.603+0000 7f5ed5d9b640 1 -- 172.21.15.154:0/1793141969 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0031b30 con 0x7f5ed8069cd0 2023-09-23T03:19:12.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.603+0000 7f5edcc5d640 1 -- 172.21.15.154:0/1793141969 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ed8069cd0 msgr2=0x7f5ed80c8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:12.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.603+0000 7f5edcc5d640 1 --2- 172.21.15.154:0/1793141969 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ed8069cd0 0x7f5ed80c8070 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0009870 tx=0x7f5ec002b8f0 comp rx=0 tx=0).stop 2023-09-23T03:19:12.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.603+0000 7f5edcc5d640 1 -- 172.21.15.154:0/1793141969 shutdown_connections 2023-09-23T03:19:12.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.603+0000 7f5edcc5d640 1 --2- 172.21.15.154:0/1793141969 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ed80c85b0 0x7f5ed80cdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:12.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.603+0000 7f5edcc5d640 1 --2- 172.21.15.154:0/1793141969 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ed8069cd0 0x7f5ed80c8070 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:12.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.603+0000 7f5edcc5d640 1 -- 172.21.15.154:0/1793141969 >> 172.21.15.154:0/1793141969 conn(0x7f5ed80cb7e0 msgr2=0x7f5ed80cbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:12.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.603+0000 7f5edcc5d640 1 -- 172.21.15.154:0/1793141969 shutdown_connections 2023-09-23T03:19:12.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.604+0000 7f5edcc5d640 1 -- 172.21.15.154:0/1793141969 wait complete. 2023-09-23T03:19:12.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.605+0000 7f5edcc5d640 1 Processor -- start 2023-09-23T03:19:12.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.605+0000 7f5edcc5d640 1 -- start start 2023-09-23T03:19:12.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.606+0000 7f5edcc5d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ed8069cd0 0x7f5ed81f31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:12.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.606+0000 7f5edcc5d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ed80c85b0 0x7f5ed815f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:12.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.606+0000 7f5edcc5d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ed8153b60 0x7f5ed81f5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:12.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.606+0000 7f5edcc5d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed80b3b10 con 0x7f5ed8069cd0 2023-09-23T03:19:12.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.606+0000 7f5edcc5d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed80b49a0 con 0x7f5ed80c85b0 2023-09-23T03:19:12.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.606+0000 7f5edcc5d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed815fb60 con 0x7f5ed8153b60 2023-09-23T03:19:12.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.607+0000 7f5ed659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ed8069cd0 0x7f5ed81f31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:12.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.607+0000 7f5ed659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ed8069cd0 0x7f5ed81f31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54564/0 (socket says 172.21.15.154:54564) 2023-09-23T03:19:12.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.607+0000 7f5ed659c640 1 -- 172.21.15.154:0/2063442556 learned_addr learned my addr 172.21.15.154:0/2063442556 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:12.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.607+0000 7f5ecdd9b640 1 --2- 172.21.15.154:0/2063442556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ed80c85b0 0x7f5ed815f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:12.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.607+0000 7f5ed6d9d640 1 --2- 172.21.15.154:0/2063442556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ed8153b60 0x7f5ed81f5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:12.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.607+0000 7f5ed659c640 1 -- 172.21.15.154:0/2063442556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ed8153b60 msgr2=0x7f5ed81f5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:12.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.607+0000 7f5ed659c640 1 --2- 172.21.15.154:0/2063442556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ed8153b60 0x7f5ed81f5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:12.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.607+0000 7f5ed659c640 1 -- 172.21.15.154:0/2063442556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ed80c85b0 msgr2=0x7f5ed815f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:12.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.607+0000 7f5ed659c640 1 --2- 172.21.15.154:0/2063442556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ed80c85b0 0x7f5ed815f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:12.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.607+0000 7f5ed659c640 1 -- 172.21.15.154:0/2063442556 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec00094d0 con 0x7f5ed8069cd0 2023-09-23T03:19:12.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.607+0000 7f5ecdd9b640 1 --2- 172.21.15.154:0/2063442556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ed80c85b0 0x7f5ed815f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:12.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.608+0000 7f5ed659c640 1 --2- 172.21.15.154:0/2063442556 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ed8069cd0 0x7f5ed81f31a0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f5ec800b3f0 tx=0x7f5ec800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:12.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.608+0000 7f5ecf7fe640 1 -- 172.21.15.154:0/2063442556 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec8011830 con 0x7f5ed8069cd0 2023-09-23T03:19:12.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.608+0000 7f5ecf7fe640 1 -- 172.21.15.154:0/2063442556 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec8011990 con 0x7f5ed8069cd0 2023-09-23T03:19:12.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.608+0000 7f5ecf7fe640 1 -- 172.21.15.154:0/2063442556 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec8011d40 con 0x7f5ed8069cd0 2023-09-23T03:19:12.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.608+0000 7f5edcc5d640 1 -- 172.21.15.154:0/2063442556 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ed80b31e0 con 0x7f5ed8069cd0 2023-09-23T03:19:12.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.608+0000 7f5edcc5d640 1 -- 172.21.15.154:0/2063442556 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ed80b37a0 con 0x7f5ed8069cd0 2023-09-23T03:19:12.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.609+0000 7f5edcc5d640 1 -- 172.21.15.154:0/2063442556 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ed804e920 con 0x7f5ed8069cd0 2023-09-23T03:19:12.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.610+0000 7f5ecf7fe640 1 -- 172.21.15.154:0/2063442556 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f5ec801a400 con 0x7f5ed8069cd0 2023-09-23T03:19:12.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.611+0000 7f5ecf7fe640 1 --2- 172.21.15.154:0/2063442556 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5eb806af30 0x7f5eb806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:12.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.612+0000 7f5ecf7fe640 1 -- 172.21.15.154:0/2063442556 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5925+0+0 (secure 0 0 0) 0x7f5ec8089660 con 0x7f5ed8069cd0 2023-09-23T03:19:12.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.612+0000 7f5ecdd9b640 1 --2- 172.21.15.154:0/2063442556 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5eb806af30 0x7f5eb806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:12.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.612+0000 7f5ecdd9b640 1 --2- 172.21.15.154:0/2063442556 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5eb806af30 0x7f5eb806d3f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0036000 tx=0x7f5ec003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:12.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.616+0000 7f5ecf7fe640 1 -- 172.21.15.154:0/2063442556 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f5ec8054650 con 0x7f5ed8069cd0 2023-09-23T03:19:12.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:12.807+0000 7f5edcc5d640 1 -- 172.21.15.154:0/2063442556 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_extra", "pool2": "cephfs_data_extra", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5ed80aebb0 con 0x7f5ed8069cd0 2023-09-23T03:19:13.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.395+0000 7f5ecf7fe640 1 -- 172.21.15.154:0/2063442556 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_extra", "pool2": "cephfs_data_extra", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_extra' removed v53) v1 ==== 187+0+0 (secure 0 0 0) 0x7f5ec8057c70 con 0x7f5ed8069cd0 2023-09-23T03:19:13.397 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_data_extra' removed 2023-09-23T03:19:13.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.400+0000 7f5edcc5d640 1 -- 172.21.15.154:0/2063442556 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5eb806af30 msgr2=0x7f5eb806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:13.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.400+0000 7f5edcc5d640 1 --2- 172.21.15.154:0/2063442556 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5eb806af30 0x7f5eb806d3f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0036000 tx=0x7f5ec003c040 comp rx=0 tx=0).stop 2023-09-23T03:19:13.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.400+0000 7f5edcc5d640 1 -- 172.21.15.154:0/2063442556 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ed8069cd0 msgr2=0x7f5ed81f31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:13.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.400+0000 7f5edcc5d640 1 --2- 172.21.15.154:0/2063442556 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ed8069cd0 0x7f5ed81f31a0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f5ec800b3f0 tx=0x7f5ec800b8b0 comp rx=0 tx=0).stop 2023-09-23T03:19:13.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.401+0000 7f5edcc5d640 1 -- 172.21.15.154:0/2063442556 shutdown_connections 2023-09-23T03:19:13.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.401+0000 7f5edcc5d640 1 --2- 172.21.15.154:0/2063442556 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5eb806af30 0x7f5eb806d3f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:13.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.401+0000 7f5edcc5d640 1 --2- 172.21.15.154:0/2063442556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5ed8153b60 0x7f5ed81f5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:13.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.401+0000 7f5edcc5d640 1 --2- 172.21.15.154:0/2063442556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5ed80c85b0 0x7f5ed815f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:13.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.401+0000 7f5edcc5d640 1 --2- 172.21.15.154:0/2063442556 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5ed8069cd0 0x7f5ed81f31a0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:13.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.401+0000 7f5edcc5d640 1 -- 172.21.15.154:0/2063442556 >> 172.21.15.154:0/2063442556 conn(0x7f5ed80cb7e0 msgr2=0x7f5ed8145230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:13.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.401+0000 7f5edcc5d640 1 -- 172.21.15.154:0/2063442556 shutdown_connections 2023-09-23T03:19:13.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.401+0000 7f5edcc5d640 1 -- 172.21.15.154:0/2063442556 wait complete. 2023-09-23T03:19:13.425 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-09-23T03:19:13.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.639+0000 7f336f378640 1 Processor -- start 2023-09-23T03:19:13.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.640+0000 7f336f378640 1 -- start start 2023-09-23T03:19:13.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.641+0000 7f336f378640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f336815aba0 0x7f336815bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:13.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.641+0000 7f336f378640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f336815b6a0 con 0x7f33680cb5b0 2023-09-23T03:19:13.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.641+0000 7f336f378640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f336815f0b0 con 0x7f336815e5f0 2023-09-23T03:19:13.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.641+0000 7f336f378640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f336815f390 con 0x7f336815aba0 2023-09-23T03:19:13.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.641+0000 7f336d915640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f33680cb5b0 0x7f33680cc5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:41596/0 (socket says 172.21.15.154:41596) 2023-09-23T03:19:13.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.641+0000 7f336d915640 1 -- 172.21.15.154:0/252649793 learned_addr learned my addr 172.21.15.154:0/252649793 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:19:13.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.642+0000 7f336d114640 1 --2- 172.21.15.154:0/252649793 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f336815aba0 0x7f336815bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:13.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.642+0000 7f335ffff640 1 -- 172.21.15.154:0/252649793 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1018147203 0 0) 0x7f336815b6a0 con 0x7f33680cb5b0 2023-09-23T03:19:13.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.642+0000 7f335ffff640 1 -- 172.21.15.154:0/252649793 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3354003520 con 0x7f33680cb5b0 2023-09-23T03:19:13.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.642+0000 7f335ffff640 1 -- 172.21.15.154:0/252649793 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3545726497 0 0) 0x7f3354003520 con 0x7f33680cb5b0 2023-09-23T03:19:13.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.642+0000 7f335ffff640 1 -- 172.21.15.154:0/252649793 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f336815aba0 msgr2=0x7f336815bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:13.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.642+0000 7f335ffff640 1 --2- 172.21.15.154:0/252649793 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f336815aba0 0x7f336815bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:13.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.642+0000 7f335ffff640 1 -- 172.21.15.154:0/252649793 >> v1:172.21.15.190:6789/0 conn(0x7f336815e5f0 legacy=0x7f33680caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:13.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.642+0000 7f335ffff640 1 -- 172.21.15.154:0/252649793 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f336815b330 con 0x7f33680cb5b0 2023-09-23T03:19:13.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.642+0000 7f335ffff640 1 -- 172.21.15.154:0/252649793 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f33600019f0 con 0x7f33680cb5b0 2023-09-23T03:19:13.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.642+0000 7f335ffff640 1 -- 172.21.15.154:0/252649793 >> v1:172.21.15.154:6789/0 conn(0x7f33680cb5b0 legacy=0x7f33680cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:13.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.643+0000 7f335ffff640 1 --2- 172.21.15.154:0/252649793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33540039a0 0x7f3354003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:13.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.643+0000 7f335ffff640 1 --2- 172.21.15.154:0/252649793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33540047c0 0x7f3354004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:13.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.643+0000 7f335ffff640 1 --2- 172.21.15.154:0/252649793 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f336815aba0 0x7f3354005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:13.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.643+0000 7f335ffff640 1 -- 172.21.15.154:0/252649793 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f336815f390 con 0x7f33540039a0 2023-09-23T03:19:13.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.643+0000 7f335ffff640 1 -- 172.21.15.154:0/252649793 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335400ba30 con 0x7f33540047c0 2023-09-23T03:19:13.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.643+0000 7f335ffff640 1 -- 172.21.15.154:0/252649793 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f335400bbf0 con 0x7f336815aba0 2023-09-23T03:19:13.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.643+0000 7f336d114640 1 --2- 172.21.15.154:0/252649793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33540039a0 0x7f3354003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:13.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.643+0000 7f336d114640 1 -- 172.21.15.154:0/252649793 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f336815aba0 msgr2=0x7f3354005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:13.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.643+0000 7f336d114640 1 --2- 172.21.15.154:0/252649793 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f336815aba0 0x7f3354005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:13.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.643+0000 7f336c913640 1 --2- 172.21.15.154:0/252649793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33540047c0 0x7f3354004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:13.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.643+0000 7f336d114640 1 -- 172.21.15.154:0/252649793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33540047c0 msgr2=0x7f3354004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:13.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.643+0000 7f336d114640 1 --2- 172.21.15.154:0/252649793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33540047c0 0x7f3354004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:13.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.643+0000 7f336d114640 1 -- 172.21.15.154:0/252649793 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3354003520 con 0x7f33540039a0 2023-09-23T03:19:13.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.643+0000 7f336c913640 1 --2- 172.21.15.154:0/252649793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33540047c0 0x7f3354004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:13.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.644+0000 7f336d114640 1 --2- 172.21.15.154:0/252649793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33540039a0 0x7f3354003dc0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f3364009a90 tx=0x7f33540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0ce67be9aed391 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:13.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.645+0000 7f335ffff640 1 -- 172.21.15.154:0/252649793 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336402c6b0 con 0x7f33540039a0 2023-09-23T03:19:13.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.645+0000 7f335ffff640 1 -- 172.21.15.154:0/252649793 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f336402c810 con 0x7f33540039a0 2023-09-23T03:19:13.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.645+0000 7f335ffff640 1 -- 172.21.15.154:0/252649793 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336402cba0 con 0x7f33540039a0 2023-09-23T03:19:13.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.645+0000 7f336f378640 1 -- 172.21.15.154:0/252649793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33540039a0 msgr2=0x7f3354003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:13.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.645+0000 7f336f378640 1 --2- 172.21.15.154:0/252649793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33540039a0 0x7f3354003dc0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f3364009a90 tx=0x7f33540057f0 comp rx=0 tx=0).stop 2023-09-23T03:19:13.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.645+0000 7f336d915640 1 -- 172.21.15.154:0/252649793 reap_dead start 2023-09-23T03:19:13.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.645+0000 7f336f378640 1 -- 172.21.15.154:0/252649793 shutdown_connections 2023-09-23T03:19:13.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.645+0000 7f336f378640 1 -- 172.21.15.154:0/252649793 >> 172.21.15.154:0/252649793 conn(0x7f336806c010 msgr2=0x7f336806c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:13.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.645+0000 7f336f378640 1 -- 172.21.15.154:0/252649793 shutdown_connections 2023-09-23T03:19:13.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.645+0000 7f336f378640 1 -- 172.21.15.154:0/252649793 wait complete. 2023-09-23T03:19:13.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.647+0000 7f336f378640 1 Processor -- start 2023-09-23T03:19:13.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.647+0000 7f336f378640 1 -- start start 2023-09-23T03:19:13.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.648+0000 7f336f378640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33540039a0 0x7f33680c65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:13.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.648+0000 7f336f378640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33680acd50 0x7f336814f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:13.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.648+0000 7f336f378640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33680ad190 0x7f336814fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:13.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.648+0000 7f336f378640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33680c75f0 con 0x7f33680acd50 2023-09-23T03:19:13.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.648+0000 7f336f378640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33680c6b10 con 0x7f33540039a0 2023-09-23T03:19:13.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.648+0000 7f336f378640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33680c6d50 con 0x7f33680ad190 2023-09-23T03:19:13.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.649+0000 7f336c913640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33680acd50 0x7f336814f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:13.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.649+0000 7f336c913640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33680acd50 0x7f336814f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54588/0 (socket says 172.21.15.154:54588) 2023-09-23T03:19:13.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.649+0000 7f336c913640 1 -- 172.21.15.154:0/996962498 learned_addr learned my addr 172.21.15.154:0/996962498 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:13.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.649+0000 7f336d114640 1 --2- 172.21.15.154:0/996962498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33540039a0 0x7f33680c65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:13.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.649+0000 7f336d915640 1 --2- 172.21.15.154:0/996962498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33680ad190 0x7f336814fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:13.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.649+0000 7f336c913640 1 -- 172.21.15.154:0/996962498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33680ad190 msgr2=0x7f336814fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:13.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.649+0000 7f336c913640 1 --2- 172.21.15.154:0/996962498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33680ad190 0x7f336814fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:13.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.649+0000 7f336c913640 1 -- 172.21.15.154:0/996962498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33540039a0 msgr2=0x7f33680c65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:13.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.649+0000 7f336c913640 1 --2- 172.21.15.154:0/996962498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33540039a0 0x7f33680c65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:13.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.649+0000 7f336c913640 1 -- 172.21.15.154:0/996962498 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f336815b100 con 0x7f33680acd50 2023-09-23T03:19:13.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.649+0000 7f336c913640 1 --2- 172.21.15.154:0/996962498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33680acd50 0x7f336814f780 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f335800d7d0 tx=0x7f335800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:13.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.650+0000 7f335dffb640 1 -- 172.21.15.154:0/996962498 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3358017070 con 0x7f33680acd50 2023-09-23T03:19:13.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.650+0000 7f335dffb640 1 -- 172.21.15.154:0/996962498 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3358005070 con 0x7f33680acd50 2023-09-23T03:19:13.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.650+0000 7f335dffb640 1 -- 172.21.15.154:0/996962498 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3358005400 con 0x7f33680acd50 2023-09-23T03:19:13.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.650+0000 7f336f378640 1 -- 172.21.15.154:0/996962498 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33680c7030 con 0x7f33680acd50 2023-09-23T03:19:13.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.650+0000 7f336f378640 1 -- 172.21.15.154:0/996962498 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3368150200 con 0x7f33680acd50 2023-09-23T03:19:13.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.651+0000 7f336f378640 1 -- 172.21.15.154:0/996962498 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f336814d6d0 con 0x7f33680acd50 2023-09-23T03:19:13.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.652+0000 7f335dffb640 1 -- 172.21.15.154:0/996962498 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f3358007030 con 0x7f33680acd50 2023-09-23T03:19:13.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.653+0000 7f335dffb640 1 --2- 172.21.15.154:0/996962498 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f334c06af30 0x7f334c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:13.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.653+0000 7f335dffb640 1 -- 172.21.15.154:0/996962498 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5521+0+0 (secure 0 0 0) 0x7f3358088ed0 con 0x7f33680acd50 2023-09-23T03:19:13.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.654+0000 7f336d114640 1 --2- 172.21.15.154:0/996962498 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f334c06af30 0x7f334c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:13.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.654+0000 7f336d114640 1 --2- 172.21.15.154:0/996962498 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f334c06af30 0x7f334c06d3f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3364036000 tx=0x7f3364030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:13.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.658+0000 7f335dffb640 1 -- 172.21.15.154:0/996962498 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f336814d6d0 con 0x7f33680acd50 2023-09-23T03:19:13.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.848+0000 7f336f378640 1 -- 172.21.15.154:0/996962498 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f336814d6d0 con 0x7f33680acd50 2023-09-23T03:19:13.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.855+0000 7f335dffb640 1 -- 172.21.15.154:0/996962498 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f336814d6d0 con 0x7f33680acd50 2023-09-23T03:19:13.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.857+0000 7f336f378640 1 -- 172.21.15.154:0/996962498 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f334c06af30 msgr2=0x7f334c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:13.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.857+0000 7f336f378640 1 --2- 172.21.15.154:0/996962498 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f334c06af30 0x7f334c06d3f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3364036000 tx=0x7f3364030000 comp rx=0 tx=0).stop 2023-09-23T03:19:13.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.857+0000 7f336f378640 1 -- 172.21.15.154:0/996962498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33680acd50 msgr2=0x7f336814f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:13.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.857+0000 7f336f378640 1 --2- 172.21.15.154:0/996962498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33680acd50 0x7f336814f780 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f335800d7d0 tx=0x7f335800dc90 comp rx=0 tx=0).stop 2023-09-23T03:19:13.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.858+0000 7f336f378640 1 -- 172.21.15.154:0/996962498 shutdown_connections 2023-09-23T03:19:13.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.858+0000 7f336f378640 1 --2- 172.21.15.154:0/996962498 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f334c06af30 0x7f334c06d3f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:13.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.858+0000 7f336f378640 1 --2- 172.21.15.154:0/996962498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f33680ad190 0x7f336814fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:13.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.858+0000 7f336f378640 1 --2- 172.21.15.154:0/996962498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33680acd50 0x7f336814f780 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:13.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.858+0000 7f336f378640 1 --2- 172.21.15.154:0/996962498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f33540039a0 0x7f33680c65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:13.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.858+0000 7f336f378640 1 -- 172.21.15.154:0/996962498 >> 172.21.15.154:0/996962498 conn(0x7f336806c010 msgr2=0x7f33680c3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:13.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.858+0000 7f336f378640 1 -- 172.21.15.154:0/996962498 shutdown_connections 2023-09-23T03:19:13.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:13.858+0000 7f336f378640 1 -- 172.21.15.154:0/996962498 wait complete. 2023-09-23T03:19:13.878 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_few_files' 2023-09-23T03:19:14.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.093+0000 7f931e5fc640 1 Processor -- start 2023-09-23T03:19:14.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.093+0000 7f931e5fc640 1 -- start start 2023-09-23T03:19:14.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.095+0000 7f931e5fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93180acce0 0x7f931815ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:14.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.095+0000 7f931e5fc640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f93180b1b20 0x7f93180ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:14.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.095+0000 7f931e5fc640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f931815ac30 con 0x7f93180cee70 2023-09-23T03:19:14.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.095+0000 7f931e5fc640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93180b2e90 con 0x7f93180b1b20 2023-09-23T03:19:14.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.095+0000 7f931e5fc640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93180cf320 con 0x7f93180acce0 2023-09-23T03:19:14.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.095+0000 7f930ffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f93180b1b20 0x7f93180ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:14.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.095+0000 7f930ffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f93180b1b20 0x7f93180ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56304/0 (socket says 172.21.15.154:56304) 2023-09-23T03:19:14.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.095+0000 7f930ffff640 1 -- 172.21.15.154:0/1942019642 learned_addr learned my addr 172.21.15.154:0/1942019642 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:14.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.095+0000 7f930ffff640 1 -- 172.21.15.154:0/1942019642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93180acce0 msgr2=0x7f931815ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:14.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.095+0000 7f9317fff640 1 --2- 172.21.15.154:0/1942019642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93180acce0 0x7f931815ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:14.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.095+0000 7f930ffff640 1 --2- 172.21.15.154:0/1942019642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93180acce0 0x7f931815ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:14.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.095+0000 7f930ffff640 1 -- 172.21.15.154:0/1942019642 >> v1:172.21.15.190:6789/0 conn(0x7f93180cee70 legacy=0x7f931806cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:14.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.095+0000 7f930ffff640 1 -- 172.21.15.154:0/1942019642 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93180b1f70 con 0x7f93180b1b20 2023-09-23T03:19:14.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.095+0000 7f9317fff640 1 --2- 172.21.15.154:0/1942019642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93180acce0 0x7f931815ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:14.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.096+0000 7f930ffff640 1 --2- 172.21.15.154:0/1942019642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93180b1b20 0x7f93180ce930 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f9300009870 tx=0x7f930002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45800dfa1e2c4627 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:14.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.097+0000 7f93177fe640 1 -- 172.21.15.154:0/1942019642 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93000317c0 con 0x7f93180b1b20 2023-09-23T03:19:14.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.097+0000 7f93177fe640 1 -- 172.21.15.154:0/1942019642 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9300031920 con 0x7f93180b1b20 2023-09-23T03:19:14.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.097+0000 7f93177fe640 1 -- 172.21.15.154:0/1942019642 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9300031cb0 con 0x7f93180b1b20 2023-09-23T03:19:14.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.097+0000 7f931e5fc640 1 -- 172.21.15.154:0/1942019642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93180b1b20 msgr2=0x7f93180ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:14.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.097+0000 7f931e5fc640 1 --2- 172.21.15.154:0/1942019642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93180b1b20 0x7f93180ce930 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f9300009870 tx=0x7f930002b860 comp rx=0 tx=0).stop 2023-09-23T03:19:14.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.097+0000 7f931e5fc640 1 -- 172.21.15.154:0/1942019642 shutdown_connections 2023-09-23T03:19:14.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.097+0000 7f931e5fc640 1 --2- 172.21.15.154:0/1942019642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93180b1b20 0x7f93180ce930 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:14.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.097+0000 7f931e5fc640 1 --2- 172.21.15.154:0/1942019642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93180acce0 0x7f931815ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:14.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.097+0000 7f931e5fc640 1 -- 172.21.15.154:0/1942019642 >> 172.21.15.154:0/1942019642 conn(0x7f931806a0f0 msgr2=0x7f931806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:14.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.098+0000 7f931e5fc640 1 -- 172.21.15.154:0/1942019642 shutdown_connections 2023-09-23T03:19:14.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.098+0000 7f931e5fc640 1 -- 172.21.15.154:0/1942019642 wait complete. 2023-09-23T03:19:14.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.099+0000 7f931e5fc640 1 Processor -- start 2023-09-23T03:19:14.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.099+0000 7f931e5fc640 1 -- start start 2023-09-23T03:19:14.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.100+0000 7f931e5fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93180acce0 0x7f9318159800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:14.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.100+0000 7f931e5fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93180b1b20 0x7f93181f4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:14.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.101+0000 7f931e5fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93180cee70 0x7f93181f4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:14.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.101+0000 7f931e5fc640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93181f53c0 con 0x7f93180b1b20 2023-09-23T03:19:14.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.101+0000 7f931e5fc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93180c2960 con 0x7f93180acce0 2023-09-23T03:19:14.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.101+0000 7f931e5fc640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93180c46d0 con 0x7f93180cee70 2023-09-23T03:19:14.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.101+0000 7f930ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93180b1b20 0x7f93181f4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:14.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.101+0000 7f930ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93180b1b20 0x7f93181f4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56308/0 (socket says 172.21.15.154:56308) 2023-09-23T03:19:14.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.101+0000 7f930ffff640 1 -- 172.21.15.154:0/4227135279 learned_addr learned my addr 172.21.15.154:0/4227135279 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:14.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.101+0000 7f930ffff640 1 -- 172.21.15.154:0/4227135279 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93180cee70 msgr2=0x7f93181f4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:14.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.101+0000 7f930ffff640 1 --2- 172.21.15.154:0/4227135279 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93180cee70 0x7f93181f4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:14.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.101+0000 7f930ffff640 1 -- 172.21.15.154:0/4227135279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93180acce0 msgr2=0x7f9318159800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:14.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.101+0000 7f9317fff640 1 --2- 172.21.15.154:0/4227135279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93180acce0 0x7f9318159800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:14.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.101+0000 7f930ffff640 1 --2- 172.21.15.154:0/4227135279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93180acce0 0x7f9318159800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:14.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.101+0000 7f930ffff640 1 -- 172.21.15.154:0/4227135279 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93000094d0 con 0x7f93180b1b20 2023-09-23T03:19:14.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.102+0000 7f930ffff640 1 --2- 172.21.15.154:0/4227135279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93180b1b20 0x7f93181f4940 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f9300009870 tx=0x7f93000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:14.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.102+0000 7f93157fa640 1 -- 172.21.15.154:0/4227135279 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9300032960 con 0x7f93180b1b20 2023-09-23T03:19:14.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.102+0000 7f93157fa640 1 -- 172.21.15.154:0/4227135279 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9300032ac0 con 0x7f93180b1b20 2023-09-23T03:19:14.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.102+0000 7f931e5fc640 1 -- 172.21.15.154:0/4227135279 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93180c49b0 con 0x7f93180b1b20 2023-09-23T03:19:14.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.102+0000 7f93157fa640 1 -- 172.21.15.154:0/4227135279 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9300032e70 con 0x7f93180b1b20 2023-09-23T03:19:14.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.102+0000 7f931e5fc640 1 -- 172.21.15.154:0/4227135279 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93180c2bc0 con 0x7f93180b1b20 2023-09-23T03:19:14.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.103+0000 7f931e5fc640 1 -- 172.21.15.154:0/4227135279 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9318073eb0 con 0x7f93180b1b20 2023-09-23T03:19:14.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.104+0000 7f93157fa640 1 -- 172.21.15.154:0/4227135279 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f930004e440 con 0x7f93180b1b20 2023-09-23T03:19:14.106 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.105+0000 7f93157fa640 1 --2- 172.21.15.154:0/4227135279 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f92f806af60 0x7f92f806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:14.106 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.105+0000 7f93157fa640 1 -- 172.21.15.154:0/4227135279 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5521+0+0 (secure 0 0 0) 0x7f93000ab8d0 con 0x7f93180b1b20 2023-09-23T03:19:14.106 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.105+0000 7f9317fff640 1 --2- 172.21.15.154:0/4227135279 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f92f806af60 0x7f92f806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:14.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.106+0000 7f9317fff640 1 --2- 172.21.15.154:0/4227135279 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f92f806af60 0x7f92f806d420 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9308006fd0 tx=0x7f9308008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:14.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.109+0000 7f93157fa640 1 -- 172.21.15.154:0/4227135279 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f9300076a50 con 0x7f93180b1b20 2023-09-23T03:19:14.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.365+0000 7f931e5fc640 1 -- 172.21.15.154:0/4227135279 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_few_files"]} v 0) v1 -- 0x7f931815ac10 con 0x7f93180b1b20 2023-09-23T03:19:14.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.398+0000 7f93157fa640 1 -- 172.21.15.154:0/4227135279 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_few_files"]}]=0 v209) v1 ==== 158+0+0 (secure 0 0 0) 0x7f930007a070 con 0x7f93180b1b20 2023-09-23T03:19:14.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.400+0000 7f931e5fc640 1 -- 172.21.15.154:0/4227135279 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f92f806af60 msgr2=0x7f92f806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:14.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.400+0000 7f931e5fc640 1 --2- 172.21.15.154:0/4227135279 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f92f806af60 0x7f92f806d420 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9308006fd0 tx=0x7f9308008040 comp rx=0 tx=0).stop 2023-09-23T03:19:14.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.401+0000 7f931e5fc640 1 -- 172.21.15.154:0/4227135279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93180b1b20 msgr2=0x7f93181f4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:14.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.401+0000 7f931e5fc640 1 --2- 172.21.15.154:0/4227135279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93180b1b20 0x7f93181f4940 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f9300009870 tx=0x7f93000046c0 comp rx=0 tx=0).stop 2023-09-23T03:19:14.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.401+0000 7f931e5fc640 1 -- 172.21.15.154:0/4227135279 shutdown_connections 2023-09-23T03:19:14.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.401+0000 7f931e5fc640 1 --2- 172.21.15.154:0/4227135279 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f92f806af60 0x7f92f806d420 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:14.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.401+0000 7f931e5fc640 1 --2- 172.21.15.154:0/4227135279 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93180cee70 0x7f93181f4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:14.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.401+0000 7f931e5fc640 1 --2- 172.21.15.154:0/4227135279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93180b1b20 0x7f93181f4940 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:14.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.401+0000 7f931e5fc640 1 --2- 172.21.15.154:0/4227135279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93180acce0 0x7f9318159800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:14.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.401+0000 7f931e5fc640 1 -- 172.21.15.154:0/4227135279 >> 172.21.15.154:0/4227135279 conn(0x7f931806a0f0 msgr2=0x7f93180b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:14.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.401+0000 7f931e5fc640 1 -- 172.21.15.154:0/4227135279 shutdown_connections 2023-09-23T03:19:14.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.401+0000 7f931e5fc640 1 -- 172.21.15.154:0/4227135279 wait complete. 2023-09-23T03:19:14.420 INFO:tasks.cephfs_test_runner:test_extra_data_pool_rebuild_few_files (tasks.cephfs.test_data_scan.TestDataScan) ... ok 2023-09-23T03:19:14.420 INFO:tasks.cephfs_test_runner:Starting test: test_extra_data_pool_rebuild_many_files_many_workers (tasks.cephfs.test_data_scan.TestDataScan) 2023-09-23T03:19:14.420 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:19:14.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.621+0000 7f29b8b83640 1 Processor -- start 2023-09-23T03:19:14.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.621+0000 7f29b8b83640 1 -- start start 2023-09-23T03:19:14.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.622+0000 7f29b8b83640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29b40cf130 0x7f29b40ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:14.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.622+0000 7f29b8b83640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f29b40b2fe0 0x7f29b415d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:14.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.622+0000 7f29b8b83640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29b406ceb0 con 0x7f29b40ada30 2023-09-23T03:19:14.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.622+0000 7f29b8b83640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29b40ade10 con 0x7f29b40b2fe0 2023-09-23T03:19:14.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.622+0000 7f29b8b83640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b4159c90 con 0x7f29b40cf130 2023-09-23T03:19:14.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.623+0000 7f29b2d9d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f29b40b2fe0 0x7f29b415d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:14.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.623+0000 7f29b1d9b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29b40cf130 0x7f29b40ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:14.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.623+0000 7f29b2d9d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f29b40b2fe0 0x7f29b415d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.154:36522/0 (socket says 172.21.15.154:36522) 2023-09-23T03:19:14.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.623+0000 7f29b2d9d640 1 -- 172.21.15.154:0/123119230 learned_addr learned my addr 172.21.15.154:0/123119230 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:14.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.623+0000 7f29b2d9d640 1 -- 172.21.15.154:0/123119230 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29b40cf130 msgr2=0x7f29b40ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:14.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.623+0000 7f29b2d9d640 1 --2- 172.21.15.154:0/123119230 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29b40cf130 0x7f29b40ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:14.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.623+0000 7f29b2d9d640 1 -- 172.21.15.154:0/123119230 >> v1:172.21.15.190:6789/0 conn(0x7f29b40ada30 legacy=0x7f29b40cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:14.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.623+0000 7f29b2d9d640 1 -- 172.21.15.154:0/123119230 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b406c110 con 0x7f29b40b2fe0 2023-09-23T03:19:14.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.626+0000 7f29b2d9d640 1 --2- 172.21.15.154:0/123119230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29b40b2fe0 0x7f29b415d380 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f29ac009870 tx=0x7f29ac02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4e86329d5ee40f6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:14.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.626+0000 7f29b159a640 1 -- 172.21.15.154:0/123119230 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac030840 con 0x7f29b40b2fe0 2023-09-23T03:19:14.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.627+0000 7f29b159a640 1 -- 172.21.15.154:0/123119230 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29ac0309a0 con 0x7f29b40b2fe0 2023-09-23T03:19:14.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.627+0000 7f29b159a640 1 -- 172.21.15.154:0/123119230 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac030d30 con 0x7f29b40b2fe0 2023-09-23T03:19:14.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.627+0000 7f29b8b83640 1 -- 172.21.15.154:0/123119230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29b40b2fe0 msgr2=0x7f29b415d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:14.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.627+0000 7f29b8b83640 1 --2- 172.21.15.154:0/123119230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29b40b2fe0 0x7f29b415d380 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f29ac009870 tx=0x7f29ac02b980 comp rx=0 tx=0).stop 2023-09-23T03:19:14.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.627+0000 7f29b8b83640 1 -- 172.21.15.154:0/123119230 shutdown_connections 2023-09-23T03:19:14.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.627+0000 7f29b8b83640 1 --2- 172.21.15.154:0/123119230 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29b40b2fe0 0x7f29b415d380 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:14.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.627+0000 7f29b8b83640 1 --2- 172.21.15.154:0/123119230 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29b40cf130 0x7f29b40ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:14.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.627+0000 7f29b8b83640 1 -- 172.21.15.154:0/123119230 >> 172.21.15.154:0/123119230 conn(0x7f29b406a0f0 msgr2=0x7f29b406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:14.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.627+0000 7f29b8b83640 1 -- 172.21.15.154:0/123119230 shutdown_connections 2023-09-23T03:19:14.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.627+0000 7f29b8b83640 1 -- 172.21.15.154:0/123119230 wait complete. 2023-09-23T03:19:14.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.628+0000 7f29b8b83640 1 Processor -- start 2023-09-23T03:19:14.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.628+0000 7f29b8b83640 1 -- start start 2023-09-23T03:19:14.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.629+0000 7f29b8b83640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29b40ada30 0x7f29b40c6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:14.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.629+0000 7f29b8b83640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29b40b2fe0 0x7f29b4158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:14.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.629+0000 7f29b8b83640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29b40cf130 0x7f29b40c4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:14.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.629+0000 7f29b8b83640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b415aeb0 con 0x7f29b40cf130 2023-09-23T03:19:14.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.629+0000 7f29b8b83640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b415b100 con 0x7f29b40b2fe0 2023-09-23T03:19:14.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.629+0000 7f29b8b83640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b415b3e0 con 0x7f29b40ada30 2023-09-23T03:19:14.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.629+0000 7f29b2d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29b40cf130 0x7f29b40c4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:14.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.629+0000 7f29b2d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29b40cf130 0x7f29b40c4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56320/0 (socket says 172.21.15.154:56320) 2023-09-23T03:19:14.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.629+0000 7f29b2d9d640 1 -- 172.21.15.154:0/3235498144 learned_addr learned my addr 172.21.15.154:0/3235498144 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:14.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.630+0000 7f29b1d9b640 1 --2- 172.21.15.154:0/3235498144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29b40b2fe0 0x7f29b4158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:14.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.630+0000 7f29b259c640 1 --2- 172.21.15.154:0/3235498144 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29b40ada30 0x7f29b40c6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:14.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.630+0000 7f29b2d9d640 1 -- 172.21.15.154:0/3235498144 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29b40ada30 msgr2=0x7f29b40c6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:14.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.630+0000 7f29b2d9d640 1 --2- 172.21.15.154:0/3235498144 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29b40ada30 0x7f29b40c6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:14.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.630+0000 7f29b2d9d640 1 -- 172.21.15.154:0/3235498144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29b40b2fe0 msgr2=0x7f29b4158680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:14.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.630+0000 7f29b2d9d640 1 --2- 172.21.15.154:0/3235498144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29b40b2fe0 0x7f29b4158680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:14.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.630+0000 7f29b2d9d640 1 -- 172.21.15.154:0/3235498144 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29ac0094d0 con 0x7f29b40cf130 2023-09-23T03:19:14.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.630+0000 7f29b259c640 1 --2- 172.21.15.154:0/3235498144 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29b40ada30 0x7f29b40c6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:14.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.630+0000 7f29b1d9b640 1 --2- 172.21.15.154:0/3235498144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29b40b2fe0 0x7f29b4158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:14.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.630+0000 7f29b2d9d640 1 --2- 172.21.15.154:0/3235498144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29b40cf130 0x7f29b40c4600 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f29ac009870 tx=0x7f29ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:14.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.630+0000 7f29aaffd640 1 -- 172.21.15.154:0/3235498144 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac039c90 con 0x7f29b40cf130 2023-09-23T03:19:14.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.631+0000 7f29aaffd640 1 -- 172.21.15.154:0/3235498144 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29ac039e10 con 0x7f29b40cf130 2023-09-23T03:19:14.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.631+0000 7f29aaffd640 1 -- 172.21.15.154:0/3235498144 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac043640 con 0x7f29b40cf130 2023-09-23T03:19:14.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.631+0000 7f29b8b83640 1 -- 172.21.15.154:0/3235498144 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29b415b6c0 con 0x7f29b40cf130 2023-09-23T03:19:14.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.631+0000 7f29b8b83640 1 -- 172.21.15.154:0/3235498144 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29b4158bc0 con 0x7f29b40cf130 2023-09-23T03:19:14.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.632+0000 7f29b8b83640 1 -- 172.21.15.154:0/3235498144 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29b415bbe0 con 0x7f29b40cf130 2023-09-23T03:19:14.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.633+0000 7f29aaffd640 1 -- 172.21.15.154:0/3235498144 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f29ac0437a0 con 0x7f29b40cf130 2023-09-23T03:19:14.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.633+0000 7f29aaffd640 1 --2- 172.21.15.154:0/3235498144 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f299406af30 0x7f299406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:14.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.634+0000 7f29aaffd640 1 -- 172.21.15.154:0/3235498144 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5521+0+0 (secure 0 0 0) 0x7f29ac03e070 con 0x7f29b40cf130 2023-09-23T03:19:14.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.634+0000 7f29b259c640 1 --2- 172.21.15.154:0/3235498144 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f299406af30 0x7f299406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:14.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.634+0000 7f29b259c640 1 --2- 172.21.15.154:0/3235498144 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f299406af30 0x7f299406d3f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f29b40b2890 tx=0x7f29a4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:14.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.639+0000 7f29aaffd640 1 -- 172.21.15.154:0/3235498144 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f29b415bbe0 con 0x7f29b40cf130 2023-09-23T03:19:14.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.817+0000 7f29b8b83640 1 -- 172.21.15.154:0/3235498144 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29b415bbe0 con 0x7f29b40cf130 2023-09-23T03:19:14.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.818+0000 7f29aaffd640 1 -- 172.21.15.154:0/3235498144 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+13623 (secure 0 0 0) 0x7f29b415bbe0 con 0x7f29b40cf130 2023-09-23T03:19:14.820 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:19:14.820 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":53,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:19:13.384666+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6836/4278807008":"2023-09-24T03:19:09.230306+0000","172.21.15.154:6834/4278807008":"2023-09-24T03:19:09.230306+0000","172.21.15.154:6837/4079550625":"2023-09-24T03:18:03.516922+0000","172.21.15.154:6835/4079550625":"2023-09-24T03:18:03.516922+0000","172.21.15.154:6837/617287200":"2023-09-24T03:17:57.206517+0000","172.21.15.190:6834/2131436414":"2023-09-24T03:17:55.088768+0000","172.21.15.154:6835/617287200":"2023-09-24T03:17:57.206517+0000","172.21.15.190:6832/2131436414":"2023-09-24T03:17:55.088768+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:19:14.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.821+0000 7f29b8b83640 1 -- 172.21.15.154:0/3235498144 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f299406af30 msgr2=0x7f299406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:14.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.821+0000 7f29b8b83640 1 --2- 172.21.15.154:0/3235498144 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f299406af30 0x7f299406d3f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f29b40b2890 tx=0x7f29a4009040 comp rx=0 tx=0).stop 2023-09-23T03:19:14.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.821+0000 7f29b8b83640 1 -- 172.21.15.154:0/3235498144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29b40cf130 msgr2=0x7f29b40c4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:14.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.821+0000 7f29b8b83640 1 --2- 172.21.15.154:0/3235498144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29b40cf130 0x7f29b40c4600 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f29ac009870 tx=0x7f29ac004730 comp rx=0 tx=0).stop 2023-09-23T03:19:14.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.821+0000 7f29b8b83640 1 -- 172.21.15.154:0/3235498144 shutdown_connections 2023-09-23T03:19:14.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.821+0000 7f29b8b83640 1 --2- 172.21.15.154:0/3235498144 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f299406af30 0x7f299406d3f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:14.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.821+0000 7f29b8b83640 1 --2- 172.21.15.154:0/3235498144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f29b40cf130 0x7f29b40c4600 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:14.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.821+0000 7f29b8b83640 1 --2- 172.21.15.154:0/3235498144 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f29b40b2fe0 0x7f29b4158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:14.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.821+0000 7f29b8b83640 1 --2- 172.21.15.154:0/3235498144 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f29b40ada30 0x7f29b40c6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:14.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.821+0000 7f29b8b83640 1 -- 172.21.15.154:0/3235498144 >> 172.21.15.154:0/3235498144 conn(0x7f29b406a0f0 msgr2=0x7f29b40726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:14.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.821+0000 7f29b8b83640 1 -- 172.21.15.154:0/3235498144 shutdown_connections 2023-09-23T03:19:14.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:14.821+0000 7f29b8b83640 1 -- 172.21.15.154:0/3235498144 wait complete. 2023-09-23T03:19:14.840 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T03:19:14.841 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:19:15.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.058+0000 7f479deaf640 1 Processor -- start 2023-09-23T03:19:15.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.059+0000 7f479deaf640 1 -- start start 2023-09-23T03:19:15.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.061+0000 7f479deaf640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f479815abe0 0x7f47980cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:15.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.061+0000 7f479deaf640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f47980b28c0 0x7f47980ce9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:15.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.061+0000 7f479deaf640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47980ce510 con 0x7f47980cc9b0 2023-09-23T03:19:15.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.061+0000 7f479deaf640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47980cb710 con 0x7f47980b28c0 2023-09-23T03:19:15.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.061+0000 7f479deaf640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47980cd390 con 0x7f479815abe0 2023-09-23T03:19:15.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.061+0000 7f4797fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f47980b28c0 0x7f47980ce9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:15.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.061+0000 7f4797fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f47980b28c0 0x7f47980ce9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56336/0 (socket says 172.21.15.154:56336) 2023-09-23T03:19:15.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.061+0000 7f4797fff640 1 -- 172.21.15.154:0/1315226416 learned_addr learned my addr 172.21.15.154:0/1315226416 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:15.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.061+0000 7f47977fe640 1 --2- 172.21.15.154:0/1315226416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f479815abe0 0x7f47980cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:15.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.061+0000 7f4797fff640 1 -- 172.21.15.154:0/1315226416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f479815abe0 msgr2=0x7f47980cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:15.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.061+0000 7f4797fff640 1 --2- 172.21.15.154:0/1315226416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f479815abe0 0x7f47980cb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:15.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.061+0000 7f4797fff640 1 -- 172.21.15.154:0/1315226416 >> v1:172.21.15.190:6789/0 conn(0x7f47980cc9b0 legacy=0x7f47980b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:15.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.061+0000 7f4797fff640 1 -- 172.21.15.154:0/1315226416 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47980cd670 con 0x7f47980b28c0 2023-09-23T03:19:15.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.062+0000 7f4797fff640 1 --2- 172.21.15.154:0/1315226416 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47980b28c0 0x7f47980ce9a0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f4790009870 tx=0x7f479002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c7b7e116d40066 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:15.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.062+0000 7f47967fc640 1 -- 172.21.15.154:0/1315226416 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47900317c0 con 0x7f47980b28c0 2023-09-23T03:19:15.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.063+0000 7f47967fc640 1 -- 172.21.15.154:0/1315226416 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4790031920 con 0x7f47980b28c0 2023-09-23T03:19:15.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.063+0000 7f47967fc640 1 -- 172.21.15.154:0/1315226416 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4790031cb0 con 0x7f47980b28c0 2023-09-23T03:19:15.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.063+0000 7f479deaf640 1 -- 172.21.15.154:0/1315226416 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47980b28c0 msgr2=0x7f47980ce9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:15.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.063+0000 7f479deaf640 1 --2- 172.21.15.154:0/1315226416 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47980b28c0 0x7f47980ce9a0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f4790009870 tx=0x7f479002b860 comp rx=0 tx=0).stop 2023-09-23T03:19:15.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.063+0000 7f479deaf640 1 -- 172.21.15.154:0/1315226416 shutdown_connections 2023-09-23T03:19:15.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.063+0000 7f479deaf640 1 --2- 172.21.15.154:0/1315226416 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47980b28c0 0x7f47980ce9a0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:15.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.063+0000 7f479deaf640 1 --2- 172.21.15.154:0/1315226416 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f479815abe0 0x7f47980cb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:15.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.063+0000 7f479deaf640 1 -- 172.21.15.154:0/1315226416 >> 172.21.15.154:0/1315226416 conn(0x7f479806fc40 msgr2=0x7f4798070020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:15.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.063+0000 7f479deaf640 1 -- 172.21.15.154:0/1315226416 shutdown_connections 2023-09-23T03:19:15.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.063+0000 7f479deaf640 1 -- 172.21.15.154:0/1315226416 wait complete. 2023-09-23T03:19:15.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.065+0000 7f479deaf640 1 Processor -- start 2023-09-23T03:19:15.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.065+0000 7f479deaf640 1 -- start start 2023-09-23T03:19:15.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.066+0000 7f479deaf640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47980b28c0 0x7f4798155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:15.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.066+0000 7f479deaf640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47980cc9b0 0x7f47980b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:15.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.066+0000 7f479deaf640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f479815abe0 0x7f47980b1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:15.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.066+0000 7f479deaf640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47980c4a30 con 0x7f47980cc9b0 2023-09-23T03:19:15.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.066+0000 7f479deaf640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47980b2620 con 0x7f479815abe0 2023-09-23T03:19:15.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.066+0000 7f479deaf640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f479815cac0 con 0x7f47980b28c0 2023-09-23T03:19:15.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.066+0000 7f4796ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47980cc9b0 0x7f47980b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:15.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.066+0000 7f4796ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47980cc9b0 0x7f47980b0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56348/0 (socket says 172.21.15.154:56348) 2023-09-23T03:19:15.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.066+0000 7f4796ffd640 1 -- 172.21.15.154:0/3289647642 learned_addr learned my addr 172.21.15.154:0/3289647642 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:15.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.067+0000 7f47977fe640 1 --2- 172.21.15.154:0/3289647642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47980b28c0 0x7f4798155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:15.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.067+0000 7f4797fff640 1 --2- 172.21.15.154:0/3289647642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f479815abe0 0x7f47980b1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:15.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.067+0000 7f4796ffd640 1 -- 172.21.15.154:0/3289647642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47980b28c0 msgr2=0x7f4798155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:15.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.067+0000 7f4796ffd640 1 --2- 172.21.15.154:0/3289647642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47980b28c0 0x7f4798155c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:15.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.067+0000 7f4796ffd640 1 -- 172.21.15.154:0/3289647642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f479815abe0 msgr2=0x7f47980b1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:15.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.067+0000 7f4796ffd640 1 --2- 172.21.15.154:0/3289647642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f479815abe0 0x7f47980b1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:15.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.067+0000 7f4796ffd640 1 -- 172.21.15.154:0/3289647642 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47900094d0 con 0x7f47980cc9b0 2023-09-23T03:19:15.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.067+0000 7f4796ffd640 1 --2- 172.21.15.154:0/3289647642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47980cc9b0 0x7f47980b0f00 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f478000c460 tx=0x7f478000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:15.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.067+0000 7f478ffff640 1 -- 172.21.15.154:0/3289647642 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4780012830 con 0x7f47980cc9b0 2023-09-23T03:19:15.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.067+0000 7f478ffff640 1 -- 172.21.15.154:0/3289647642 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47800129d0 con 0x7f47980cc9b0 2023-09-23T03:19:15.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.067+0000 7f479deaf640 1 -- 172.21.15.154:0/3289647642 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f479815c7e0 con 0x7f47980cc9b0 2023-09-23T03:19:15.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.067+0000 7f478ffff640 1 -- 172.21.15.154:0/3289647642 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4780012d80 con 0x7f47980cc9b0 2023-09-23T03:19:15.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.067+0000 7f479deaf640 1 -- 172.21.15.154:0/3289647642 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47980c4650 con 0x7f47980cc9b0 2023-09-23T03:19:15.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.068+0000 7f479deaf640 1 -- 172.21.15.154:0/3289647642 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47980c4b70 con 0x7f47980cc9b0 2023-09-23T03:19:15.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.069+0000 7f478ffff640 1 -- 172.21.15.154:0/3289647642 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f478001a3b0 con 0x7f47980cc9b0 2023-09-23T03:19:15.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.070+0000 7f478ffff640 1 --2- 172.21.15.154:0/3289647642 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f477806af30 0x7f477806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:15.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.070+0000 7f478ffff640 1 -- 172.21.15.154:0/3289647642 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5521+0+0 (secure 0 0 0) 0x7f4780089330 con 0x7f47980cc9b0 2023-09-23T03:19:15.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.071+0000 7f47977fe640 1 --2- 172.21.15.154:0/3289647642 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f477806af30 0x7f477806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:15.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.071+0000 7f47977fe640 1 --2- 172.21.15.154:0/3289647642 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f477806af30 0x7f477806d3f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f479815d120 tx=0x7f47880084b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:15.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.075+0000 7f478ffff640 1 -- 172.21.15.154:0/3289647642 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f47980c4b70 con 0x7f47980cc9b0 2023-09-23T03:19:15.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.265+0000 7f479deaf640 1 -- 172.21.15.154:0/3289647642 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f47980c4b70 con 0x7f47980cc9b0 2023-09-23T03:19:15.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.266+0000 7f478ffff640 1 -- 172.21.15.154:0/3289647642 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) v1 ==== 93+0+10 (secure 0 0 0) 0x7f47980c4b70 con 0x7f47980cc9b0 2023-09-23T03:19:15.267 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:19:15.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.268+0000 7f479deaf640 1 -- 172.21.15.154:0/3289647642 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f477806af30 msgr2=0x7f477806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:15.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.268+0000 7f479deaf640 1 --2- 172.21.15.154:0/3289647642 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f477806af30 0x7f477806d3f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f479815d120 tx=0x7f47880084b0 comp rx=0 tx=0).stop 2023-09-23T03:19:15.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.268+0000 7f479deaf640 1 -- 172.21.15.154:0/3289647642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47980cc9b0 msgr2=0x7f47980b0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:15.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.268+0000 7f479deaf640 1 --2- 172.21.15.154:0/3289647642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47980cc9b0 0x7f47980b0f00 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f478000c460 tx=0x7f478000c920 comp rx=0 tx=0).stop 2023-09-23T03:19:15.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.268+0000 7f479deaf640 1 -- 172.21.15.154:0/3289647642 shutdown_connections 2023-09-23T03:19:15.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.268+0000 7f479deaf640 1 --2- 172.21.15.154:0/3289647642 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f477806af30 0x7f477806d3f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:15.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.268+0000 7f479deaf640 1 --2- 172.21.15.154:0/3289647642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f479815abe0 0x7f47980b1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:15.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.268+0000 7f479deaf640 1 --2- 172.21.15.154:0/3289647642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47980cc9b0 0x7f47980b0f00 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:15.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.268+0000 7f479deaf640 1 --2- 172.21.15.154:0/3289647642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47980b28c0 0x7f4798155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:15.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.268+0000 7f479deaf640 1 -- 172.21.15.154:0/3289647642 >> 172.21.15.154:0/3289647642 conn(0x7f479806fc40 msgr2=0x7f47980d10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:15.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.269+0000 7f479deaf640 1 -- 172.21.15.154:0/3289647642 shutdown_connections 2023-09-23T03:19:15.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.269+0000 7f479deaf640 1 -- 172.21.15.154:0/3289647642 wait complete. 2023-09-23T03:19:15.289 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_many_files_many_workers' 2023-09-23T03:19:15.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.493+0000 7f4ae277d640 1 Processor -- start 2023-09-23T03:19:15.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.493+0000 7f4ae277d640 1 -- start start 2023-09-23T03:19:15.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.495+0000 7f4ae277d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4adc0cf040 0x7f4adc1656e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:15.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.495+0000 7f4ae277d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4adc15ac30 con 0x7f4adc0b1b20 2023-09-23T03:19:15.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.495+0000 7f4ae277d640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4adc0b1f70 con 0x7f4adc0acce0 2023-09-23T03:19:15.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.495+0000 7f4ae277d640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4adc0b2e90 con 0x7f4adc0cf040 2023-09-23T03:19:15.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.495+0000 7f4adb7fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f4adc0b1b20 0x7f4adc0ce930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40034/0 (socket says 172.21.15.154:40034) 2023-09-23T03:19:15.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.495+0000 7f4adb7fe640 1 -- 172.21.15.154:0/4195035486 learned_addr learned my addr 172.21.15.154:0/4195035486 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:19:15.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.495+0000 7f4ae0d1a640 1 --2- 172.21.15.154:0/4195035486 >> v2:172.21.15.190:3300/0 conn(0x7f4adc0cf040 0x7f4adc1656e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:15.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.495+0000 7f4adaffd640 1 -- 172.21.15.154:0/4195035486 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3817227143 0 0) 0x7f4ad0001110 con 0x7f4adc0b1b20 2023-09-23T03:19:15.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.495+0000 7f4adaffd640 1 -- 172.21.15.154:0/4195035486 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ac4003520 con 0x7f4adc0b1b20 2023-09-23T03:19:15.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.496+0000 7f4adaffd640 1 -- 172.21.15.154:0/4195035486 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3470665657 0 0) 0x7f4ac4003520 con 0x7f4adc0b1b20 2023-09-23T03:19:15.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.496+0000 7f4adaffd640 1 -- 172.21.15.154:0/4195035486 >> v2:172.21.15.190:3300/0 conn(0x7f4adc0cf040 msgr2=0x7f4adc1656e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:15.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.496+0000 7f4adaffd640 1 --2- 172.21.15.154:0/4195035486 >> v2:172.21.15.190:3300/0 conn(0x7f4adc0cf040 0x7f4adc1656e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:15.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.496+0000 7f4adaffd640 1 -- 172.21.15.154:0/4195035486 >> v1:172.21.15.190:6789/0 conn(0x7f4adc0acce0 legacy=0x7f4adc15ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:15.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.496+0000 7f4adaffd640 1 -- 172.21.15.154:0/4195035486 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4adc0b2210 con 0x7f4adc0b1b20 2023-09-23T03:19:15.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.496+0000 7f4adaffd640 1 -- 172.21.15.154:0/4195035486 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f4ad0001800 con 0x7f4adc0b1b20 2023-09-23T03:19:15.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.496+0000 7f4adaffd640 1 -- 172.21.15.154:0/4195035486 >> v1:172.21.15.154:6789/0 conn(0x7f4adc0b1b20 legacy=0x7f4adc0ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:15.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.496+0000 7f4adaffd640 1 --2- 172.21.15.154:0/4195035486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ac4003970 0x7f4ac4003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:15.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.496+0000 7f4adaffd640 1 --2- 172.21.15.154:0/4195035486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ac40047c0 0x7f4ac4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:15.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.496+0000 7f4adaffd640 1 --2- 172.21.15.154:0/4195035486 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ac4005180 0x7f4ac400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:15.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.496+0000 7f4adaffd640 1 -- 172.21.15.154:0/4195035486 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc0b2e90 con 0x7f4ac40047c0 2023-09-23T03:19:15.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.496+0000 7f4adaffd640 1 -- 172.21.15.154:0/4195035486 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac400c0b0 con 0x7f4ac4003970 2023-09-23T03:19:15.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.496+0000 7f4adaffd640 1 -- 172.21.15.154:0/4195035486 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac400c270 con 0x7f4ac4005180 2023-09-23T03:19:15.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.496+0000 7f4adb7fe640 1 --2- 172.21.15.154:0/4195035486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ac40047c0 0x7f4ac4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:15.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.496+0000 7f4adbfff640 1 --2- 172.21.15.154:0/4195035486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ac4003970 0x7f4ac4003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:15.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.496+0000 7f4adb7fe640 1 -- 172.21.15.154:0/4195035486 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ac4005180 msgr2=0x7f4ac400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:15.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.496+0000 7f4adb7fe640 1 --2- 172.21.15.154:0/4195035486 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ac4005180 0x7f4ac400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:15.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.497+0000 7f4adb7fe640 1 -- 172.21.15.154:0/4195035486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ac4003970 msgr2=0x7f4ac4003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:15.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.497+0000 7f4adb7fe640 1 --2- 172.21.15.154:0/4195035486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ac4003970 0x7f4ac4003d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:15.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.497+0000 7f4adb7fe640 1 -- 172.21.15.154:0/4195035486 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac4003520 con 0x7f4ac40047c0 2023-09-23T03:19:15.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.497+0000 7f4ae0d1a640 1 -- 172.21.15.154:0/4195035486 reap_dead start 2023-09-23T03:19:15.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.497+0000 7f4adb7fe640 1 --2- 172.21.15.154:0/4195035486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ac40047c0 0x7f4ac4004c40 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f4ad000c210 tx=0x7f4ac40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dcf4bf2b1c9ad79 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:15.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.498+0000 7f4adaffd640 1 -- 172.21.15.154:0/4195035486 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad0005ad0 con 0x7f4ac40047c0 2023-09-23T03:19:15.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.498+0000 7f4adaffd640 1 -- 172.21.15.154:0/4195035486 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad0005c50 con 0x7f4ac40047c0 2023-09-23T03:19:15.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.498+0000 7f4adaffd640 1 -- 172.21.15.154:0/4195035486 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad002c830 con 0x7f4ac40047c0 2023-09-23T03:19:15.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.498+0000 7f4ae277d640 1 -- 172.21.15.154:0/4195035486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ac40047c0 msgr2=0x7f4ac4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:15.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.498+0000 7f4ae277d640 1 --2- 172.21.15.154:0/4195035486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ac40047c0 0x7f4ac4004c40 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f4ad000c210 tx=0x7f4ac40057f0 comp rx=0 tx=0).stop 2023-09-23T03:19:15.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.498+0000 7f4ae277d640 1 -- 172.21.15.154:0/4195035486 shutdown_connections 2023-09-23T03:19:15.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.498+0000 7f4ae277d640 1 --2- 172.21.15.154:0/4195035486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ac40047c0 0x7f4ac4004c40 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:15.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.498+0000 7f4ae277d640 1 -- 172.21.15.154:0/4195035486 >> 172.21.15.154:0/4195035486 conn(0x7f4adc06a0f0 msgr2=0x7f4adc06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:15.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.498+0000 7f4ae277d640 1 -- 172.21.15.154:0/4195035486 shutdown_connections 2023-09-23T03:19:15.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.498+0000 7f4ae277d640 1 -- 172.21.15.154:0/4195035486 wait complete. 2023-09-23T03:19:15.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.500+0000 7f4ae277d640 1 Processor -- start 2023-09-23T03:19:15.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.500+0000 7f4ae277d640 1 -- start start 2023-09-23T03:19:15.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.501+0000 7f4ae277d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ac40047c0 0x7f4adc156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:15.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.501+0000 7f4ae277d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4adc15d6e0 0x7f4adc1f2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:15.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.501+0000 7f4ae277d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4adc0ca6e0 0x7f4adc0cab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:15.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.501+0000 7f4ae277d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc0cb0d0 con 0x7f4adc0ca6e0 2023-09-23T03:19:15.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.501+0000 7f4ae277d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc1f3210 con 0x7f4ac40047c0 2023-09-23T03:19:15.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.501+0000 7f4ae277d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4adc156a60 con 0x7f4adc15d6e0 2023-09-23T03:19:15.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.501+0000 7f4ae0d1a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4adc0ca6e0 0x7f4adc0cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:15.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.501+0000 7f4ae0d1a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4adc0ca6e0 0x7f4adc0cab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56378/0 (socket says 172.21.15.154:56378) 2023-09-23T03:19:15.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.501+0000 7f4ae0d1a640 1 -- 172.21.15.154:0/4238671498 learned_addr learned my addr 172.21.15.154:0/4238671498 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:15.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.502+0000 7f4ae0d1a640 1 -- 172.21.15.154:0/4238671498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4adc15d6e0 msgr2=0x7f4adc1f2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:15.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.502+0000 7f4ae0d1a640 1 --2- 172.21.15.154:0/4238671498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4adc15d6e0 0x7f4adc1f2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:15.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.502+0000 7f4ae0d1a640 1 -- 172.21.15.154:0/4238671498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ac40047c0 msgr2=0x7f4adc156520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:15.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.502+0000 7f4ae0d1a640 1 --2- 172.21.15.154:0/4238671498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ac40047c0 0x7f4adc156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:15.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.502+0000 7f4ae0d1a640 1 -- 172.21.15.154:0/4238671498 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4adc1f29c0 con 0x7f4adc0ca6e0 2023-09-23T03:19:15.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.502+0000 7f4ae0d1a640 1 --2- 172.21.15.154:0/4238671498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4adc0ca6e0 0x7f4adc0cab90 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f4adc0c8e70 tx=0x7f4ad400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:15.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.502+0000 7f4ad8ff9640 1 -- 172.21.15.154:0/4238671498 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4010040 con 0x7f4adc0ca6e0 2023-09-23T03:19:15.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.502+0000 7f4ad8ff9640 1 -- 172.21.15.154:0/4238671498 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad400bce0 con 0x7f4adc0ca6e0 2023-09-23T03:19:15.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.502+0000 7f4ad8ff9640 1 -- 172.21.15.154:0/4238671498 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4014640 con 0x7f4adc0ca6e0 2023-09-23T03:19:15.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.502+0000 7f4ae277d640 1 -- 172.21.15.154:0/4238671498 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4adc1f2ca0 con 0x7f4adc0ca6e0 2023-09-23T03:19:15.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.502+0000 7f4ae277d640 1 -- 172.21.15.154:0/4238671498 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4adc0c6600 con 0x7f4adc0ca6e0 2023-09-23T03:19:15.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.503+0000 7f4ae277d640 1 -- 172.21.15.154:0/4238671498 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4adc073eb0 con 0x7f4adc0ca6e0 2023-09-23T03:19:15.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.504+0000 7f4ad8ff9640 1 -- 172.21.15.154:0/4238671498 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f4ad40147a0 con 0x7f4adc0ca6e0 2023-09-23T03:19:15.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.505+0000 7f4ad8ff9640 1 --2- 172.21.15.154:0/4238671498 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4ac806af30 0x7f4ac806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:15.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.505+0000 7f4ad8ff9640 1 -- 172.21.15.154:0/4238671498 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5521+0+0 (secure 0 0 0) 0x7f4ad4089d70 con 0x7f4adc0ca6e0 2023-09-23T03:19:15.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.506+0000 7f4adbfff640 1 --2- 172.21.15.154:0/4238671498 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4ac806af30 0x7f4ac806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:15.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.506+0000 7f4adbfff640 1 --2- 172.21.15.154:0/4238671498 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4ac806af30 0x7f4ac806d3f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4adc074b80 tx=0x7f4acc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:15.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.510+0000 7f4ad8ff9640 1 -- 172.21.15.154:0/4238671498 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f4ad4054dc0 con 0x7f4adc0ca6e0 2023-09-23T03:19:15.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:15.774+0000 7f4ae277d640 1 -- 172.21.15.154:0/4238671498 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_many_files_many_workers"]} v 0) v1 -- 0x7f4adc06c930 con 0x7f4adc0ca6e0 2023-09-23T03:19:16.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.412+0000 7f4ad8ff9640 1 -- 172.21.15.154:0/4238671498 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_many_files_many_workers"]}]=0 v211) v1 ==== 175+0+0 (secure 0 0 0) 0x7f4ad40583e0 con 0x7f4adc0ca6e0 2023-09-23T03:19:16.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.415+0000 7f4ae277d640 1 -- 172.21.15.154:0/4238671498 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4ac806af30 msgr2=0x7f4ac806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:16.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.415+0000 7f4ae277d640 1 --2- 172.21.15.154:0/4238671498 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4ac806af30 0x7f4ac806d3f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4adc074b80 tx=0x7f4acc009040 comp rx=0 tx=0).stop 2023-09-23T03:19:16.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.415+0000 7f4ae277d640 1 -- 172.21.15.154:0/4238671498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4adc0ca6e0 msgr2=0x7f4adc0cab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:16.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.415+0000 7f4ae277d640 1 --2- 172.21.15.154:0/4238671498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4adc0ca6e0 0x7f4adc0cab90 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f4adc0c8e70 tx=0x7f4ad400b480 comp rx=0 tx=0).stop 2023-09-23T03:19:16.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.416+0000 7f4ae277d640 1 -- 172.21.15.154:0/4238671498 shutdown_connections 2023-09-23T03:19:16.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.416+0000 7f4ae277d640 1 --2- 172.21.15.154:0/4238671498 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4ac806af30 0x7f4ac806d3f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:16.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.416+0000 7f4ae277d640 1 --2- 172.21.15.154:0/4238671498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4adc0ca6e0 0x7f4adc0cab90 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:16.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.416+0000 7f4ae277d640 1 --2- 172.21.15.154:0/4238671498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4adc15d6e0 0x7f4adc1f2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:16.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.416+0000 7f4ae277d640 1 --2- 172.21.15.154:0/4238671498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ac40047c0 0x7f4adc156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:16.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.416+0000 7f4ae277d640 1 -- 172.21.15.154:0/4238671498 >> 172.21.15.154:0/4238671498 conn(0x7f4adc06a0f0 msgr2=0x7f4adc0b0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:16.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.416+0000 7f4ae277d640 1 -- 172.21.15.154:0/4238671498 shutdown_connections 2023-09-23T03:19:16.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.416+0000 7f4ae277d640 1 -- 172.21.15.154:0/4238671498 wait complete. 2023-09-23T03:19:16.436 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-09-23T03:19:16.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.653+0000 7fd1877c5640 1 Processor -- start 2023-09-23T03:19:16.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.653+0000 7fd1877c5640 1 -- start start 2023-09-23T03:19:16.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.655+0000 7fd1877c5640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd180070c60 0x7fd1800caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:16.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.655+0000 7fd1877c5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1800712f0 con 0x7fd1800ad830 2023-09-23T03:19:16.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.655+0000 7fd1877c5640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1800715d0 con 0x7fd1800cb400 2023-09-23T03:19:16.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.655+0000 7fd1877c5640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1800710f0 con 0x7fd180070c60 2023-09-23T03:19:16.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.655+0000 7fd185561640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fd1800ad830 0x7fd1800c9680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40048/0 (socket says 172.21.15.154:40048) 2023-09-23T03:19:16.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.655+0000 7fd185561640 1 -- 172.21.15.154:0/1634035478 learned_addr learned my addr 172.21.15.154:0/1634035478 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:19:16.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.655+0000 7fd184d60640 1 --2- 172.21.15.154:0/1634035478 >> v2:172.21.15.190:3300/0 conn(0x7fd180070c60 0x7fd1800caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:16.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.656+0000 7fd177fff640 1 -- 172.21.15.154:0/1634035478 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 192837363 0 0) 0x7fd1800712f0 con 0x7fd1800ad830 2023-09-23T03:19:16.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.656+0000 7fd177fff640 1 -- 172.21.15.154:0/1634035478 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd16c003520 con 0x7fd1800ad830 2023-09-23T03:19:16.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.656+0000 7fd177fff640 1 -- 172.21.15.154:0/1634035478 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4164267415 0 0) 0x7fd16c003520 con 0x7fd1800ad830 2023-09-23T03:19:16.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.656+0000 7fd177fff640 1 -- 172.21.15.154:0/1634035478 >> v2:172.21.15.190:3300/0 conn(0x7fd180070c60 msgr2=0x7fd1800caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:16.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.656+0000 7fd177fff640 1 --2- 172.21.15.154:0/1634035478 >> v2:172.21.15.190:3300/0 conn(0x7fd180070c60 0x7fd1800caec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:16.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.656+0000 7fd177fff640 1 -- 172.21.15.154:0/1634035478 >> v1:172.21.15.190:6789/0 conn(0x7fd1800cb400 legacy=0x7fd1800aba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:16.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.656+0000 7fd177fff640 1 -- 172.21.15.154:0/1634035478 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1800718b0 con 0x7fd1800ad830 2023-09-23T03:19:16.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.656+0000 7fd177fff640 1 -- 172.21.15.154:0/1634035478 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fd17c001670 con 0x7fd1800ad830 2023-09-23T03:19:16.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.656+0000 7fd177fff640 1 -- 172.21.15.154:0/1634035478 >> v1:172.21.15.154:6789/0 conn(0x7fd1800ad830 legacy=0x7fd1800c9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:16.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.656+0000 7fd177fff640 1 --2- 172.21.15.154:0/1634035478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd16c003970 0x7fd16c003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:16.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.656+0000 7fd177fff640 1 --2- 172.21.15.154:0/1634035478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd16c0047c0 0x7fd16c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:16.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.656+0000 7fd177fff640 1 --2- 172.21.15.154:0/1634035478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd16c005180 0x7fd16c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:16.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.656+0000 7fd177fff640 1 -- 172.21.15.154:0/1634035478 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1800710f0 con 0x7fd16c0047c0 2023-09-23T03:19:16.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.656+0000 7fd177fff640 1 -- 172.21.15.154:0/1634035478 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c00c000 con 0x7fd16c005180 2023-09-23T03:19:16.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.656+0000 7fd177fff640 1 -- 172.21.15.154:0/1634035478 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c00c1c0 con 0x7fd16c003970 2023-09-23T03:19:16.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.656+0000 7fd184d60640 1 --2- 172.21.15.154:0/1634035478 >> v2:172.21.15.190:3300/0 conn(0x7fd180070c60 0x7fd1800caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:19:16.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.657+0000 7fd184d60640 1 --2- 172.21.15.154:0/1634035478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd16c0047c0 0x7fd16c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:16.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.657+0000 7fd185561640 1 --2- 172.21.15.154:0/1634035478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd16c003970 0x7fd16c003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:16.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.657+0000 7fd185d62640 1 --2- 172.21.15.154:0/1634035478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd16c005180 0x7fd16c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:16.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.657+0000 7fd184d60640 1 -- 172.21.15.154:0/1634035478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd16c003970 msgr2=0x7fd16c003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:16.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.657+0000 7fd184d60640 1 --2- 172.21.15.154:0/1634035478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd16c003970 0x7fd16c003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:16.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.657+0000 7fd184d60640 1 -- 172.21.15.154:0/1634035478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd16c005180 msgr2=0x7fd16c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:16.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.657+0000 7fd184d60640 1 --2- 172.21.15.154:0/1634035478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd16c005180 0x7fd16c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:16.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.657+0000 7fd184d60640 1 -- 172.21.15.154:0/1634035478 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd16c003520 con 0x7fd16c0047c0 2023-09-23T03:19:16.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.657+0000 7fd185d62640 1 -- 172.21.15.154:0/1634035478 reap_dead start 2023-09-23T03:19:16.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.658+0000 7fd184d60640 1 --2- 172.21.15.154:0/1634035478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd16c0047c0 0x7fd16c004c40 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fd170009c70 tx=0x7fd16c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8e40615810a83bd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:16.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.658+0000 7fd177fff640 1 -- 172.21.15.154:0/1634035478 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd170004020 con 0x7fd16c0047c0 2023-09-23T03:19:16.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.658+0000 7fd177fff640 1 -- 172.21.15.154:0/1634035478 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd170004180 con 0x7fd16c0047c0 2023-09-23T03:19:16.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.658+0000 7fd177fff640 1 -- 172.21.15.154:0/1634035478 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17002c8f0 con 0x7fd16c0047c0 2023-09-23T03:19:16.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.659+0000 7fd1877c5640 1 -- 172.21.15.154:0/1634035478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd16c0047c0 msgr2=0x7fd16c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:16.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.659+0000 7fd1877c5640 1 --2- 172.21.15.154:0/1634035478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd16c0047c0 0x7fd16c004c40 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fd170009c70 tx=0x7fd16c0057f0 comp rx=0 tx=0).stop 2023-09-23T03:19:16.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.659+0000 7fd1877c5640 1 -- 172.21.15.154:0/1634035478 shutdown_connections 2023-09-23T03:19:16.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.659+0000 7fd1877c5640 1 --2- 172.21.15.154:0/1634035478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd16c0047c0 0x7fd16c004c40 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:16.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.659+0000 7fd1877c5640 1 -- 172.21.15.154:0/1634035478 >> 172.21.15.154:0/1634035478 conn(0x7fd18006c010 msgr2=0x7fd18006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:16.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.659+0000 7fd1877c5640 1 -- 172.21.15.154:0/1634035478 shutdown_connections 2023-09-23T03:19:16.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.659+0000 7fd1877c5640 1 -- 172.21.15.154:0/1634035478 wait complete. 2023-09-23T03:19:16.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.661+0000 7fd1877c5640 1 Processor -- start 2023-09-23T03:19:16.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.661+0000 7fd1877c5640 1 -- start start 2023-09-23T03:19:16.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.662+0000 7fd1877c5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd16c0047c0 0x7fd180154920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:16.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.662+0000 7fd1877c5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1800b2da0 0x7fd1800b3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:16.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.662+0000 7fd1877c5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1801569d0 0x7fd180156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:16.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.662+0000 7fd1877c5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1800b3760 con 0x7fd16c0047c0 2023-09-23T03:19:16.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.662+0000 7fd1877c5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1801553f0 con 0x7fd1800b2da0 2023-09-23T03:19:16.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.662+0000 7fd1877c5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd180156250 con 0x7fd1801569d0 2023-09-23T03:19:16.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.662+0000 7fd185561640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd16c0047c0 0x7fd180154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:16.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.663+0000 7fd185561640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd16c0047c0 0x7fd180154920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56400/0 (socket says 172.21.15.154:56400) 2023-09-23T03:19:16.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.663+0000 7fd185561640 1 -- 172.21.15.154:0/1118080028 learned_addr learned my addr 172.21.15.154:0/1118080028 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:16.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.663+0000 7fd184d60640 1 --2- 172.21.15.154:0/1118080028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1800b2da0 0x7fd1800b3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:16.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.663+0000 7fd185d62640 1 --2- 172.21.15.154:0/1118080028 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1801569d0 0x7fd180156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:16.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.663+0000 7fd185561640 1 -- 172.21.15.154:0/1118080028 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1801569d0 msgr2=0x7fd180156e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:16.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.663+0000 7fd185561640 1 --2- 172.21.15.154:0/1118080028 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1801569d0 0x7fd180156e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:16.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.663+0000 7fd185561640 1 -- 172.21.15.154:0/1118080028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1800b2da0 msgr2=0x7fd1800b3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:16.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.663+0000 7fd185561640 1 --2- 172.21.15.154:0/1118080028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1800b2da0 0x7fd1800b3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:16.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.663+0000 7fd185561640 1 -- 172.21.15.154:0/1118080028 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd180155640 con 0x7fd16c0047c0 2023-09-23T03:19:16.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.663+0000 7fd184d60640 1 --2- 172.21.15.154:0/1118080028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1800b2da0 0x7fd1800b3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:19:16.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.663+0000 7fd185561640 1 --2- 172.21.15.154:0/1118080028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd16c0047c0 0x7fd180154920 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00e850 tx=0x7fd17c00ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:16.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.664+0000 7fd175ffb640 1 -- 172.21.15.154:0/1118080028 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c00c870 con 0x7fd16c0047c0 2023-09-23T03:19:16.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.664+0000 7fd175ffb640 1 -- 172.21.15.154:0/1118080028 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17c00c9f0 con 0x7fd16c0047c0 2023-09-23T03:19:16.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.664+0000 7fd175ffb640 1 -- 172.21.15.154:0/1118080028 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c00cda0 con 0x7fd16c0047c0 2023-09-23T03:19:16.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.664+0000 7fd1877c5640 1 -- 172.21.15.154:0/1118080028 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd180155920 con 0x7fd16c0047c0 2023-09-23T03:19:16.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.664+0000 7fd1877c5640 1 -- 172.21.15.154:0/1118080028 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd180155ee0 con 0x7fd16c0047c0 2023-09-23T03:19:16.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.665+0000 7fd1877c5640 1 -- 172.21.15.154:0/1118080028 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1801560a0 con 0x7fd16c0047c0 2023-09-23T03:19:16.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.666+0000 7fd175ffb640 1 -- 172.21.15.154:0/1118080028 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fd17c01d460 con 0x7fd16c0047c0 2023-09-23T03:19:16.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.667+0000 7fd175ffb640 1 --2- 172.21.15.154:0/1118080028 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd16406af30 0x7fd16406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:16.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.667+0000 7fd175ffb640 1 -- 172.21.15.154:0/1118080028 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5521+0+0 (secure 0 0 0) 0x7fd17c0895b0 con 0x7fd16c0047c0 2023-09-23T03:19:16.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.667+0000 7fd184d60640 1 --2- 172.21.15.154:0/1118080028 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd16406af30 0x7fd16406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:16.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.668+0000 7fd184d60640 1 --2- 172.21.15.154:0/1118080028 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd16406af30 0x7fd16406d3f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd170035000 tx=0x7fd170030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:16.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.672+0000 7fd175ffb640 1 -- 172.21.15.154:0/1118080028 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fd17c054730 con 0x7fd16c0047c0 2023-09-23T03:19:16.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.856+0000 7fd1877c5640 1 -- 172.21.15.154:0/1118080028 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd18004e920 con 0x7fd16c0047c0 2023-09-23T03:19:16.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.861+0000 7fd175ffb640 1 -- 172.21.15.154:0/1118080028 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd17c057d50 con 0x7fd16c0047c0 2023-09-23T03:19:16.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.864+0000 7fd1877c5640 1 -- 172.21.15.154:0/1118080028 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd16406af30 msgr2=0x7fd16406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:16.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.864+0000 7fd1877c5640 1 --2- 172.21.15.154:0/1118080028 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd16406af30 0x7fd16406d3f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd170035000 tx=0x7fd170030000 comp rx=0 tx=0).stop 2023-09-23T03:19:16.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.864+0000 7fd1877c5640 1 -- 172.21.15.154:0/1118080028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd16c0047c0 msgr2=0x7fd180154920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:16.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.864+0000 7fd1877c5640 1 --2- 172.21.15.154:0/1118080028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd16c0047c0 0x7fd180154920 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fd17c00e850 tx=0x7fd17c00ed10 comp rx=0 tx=0).stop 2023-09-23T03:19:16.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.864+0000 7fd1877c5640 1 -- 172.21.15.154:0/1118080028 shutdown_connections 2023-09-23T03:19:16.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.864+0000 7fd1877c5640 1 --2- 172.21.15.154:0/1118080028 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd16406af30 0x7fd16406d3f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:16.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.864+0000 7fd1877c5640 1 --2- 172.21.15.154:0/1118080028 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1801569d0 0x7fd180156e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:16.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.864+0000 7fd1877c5640 1 --2- 172.21.15.154:0/1118080028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1800b2da0 0x7fd1800b3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:16.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.864+0000 7fd1877c5640 1 --2- 172.21.15.154:0/1118080028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd16c0047c0 0x7fd180154920 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:16.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.864+0000 7fd1877c5640 1 -- 172.21.15.154:0/1118080028 >> 172.21.15.154:0/1118080028 conn(0x7fd18006c010 msgr2=0x7fd1800704b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:16.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.864+0000 7fd1877c5640 1 -- 172.21.15.154:0/1118080028 shutdown_connections 2023-09-23T03:19:16.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:16.864+0000 7fd1877c5640 1 -- 172.21.15.154:0/1118080028 wait complete. 2023-09-23T03:19:16.882 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-09-23T03:19:16.882 DEBUG:teuthology.orchestra.run.smithi154:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-23T03:19:16.995 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-23T03:19:17.118 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-09-23T03:19:17.118 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:19:17.118 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:19:17.172 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053600,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-09-23T03:19:17.172 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:19:17.173 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:19:17.233 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053603,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248804k,nr_inodes=812201,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-09-23T03:19:17.233 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:19:17.233 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:19:17.292 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053600,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-09-23T03:19:17.292 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:19:17.292 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:19:17.352 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053603,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248804k,nr_inodes=812201,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-09-23T03:19:17.352 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-09-23T03:19:17.352 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:19:17.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.576+0000 7fd25a821640 1 Processor -- start 2023-09-23T03:19:17.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.577+0000 7fd25a821640 1 -- start start 2023-09-23T03:19:17.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.578+0000 7fd25a821640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd2540c14e0 0x7fd2540c1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:17.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.578+0000 7fd25a821640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd25406c1d0 0x7fd25406c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:17.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.578+0000 7fd25a821640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2540cb160 con 0x7fd254161010 2023-09-23T03:19:17.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.578+0000 7fd25a821640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2540c1ea0 con 0x7fd2540c14e0 2023-09-23T03:19:17.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.578+0000 7fd25a821640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd25406cb60 con 0x7fd25406c1d0 2023-09-23T03:19:17.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.578+0000 7fd253fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fd254161010 0x7fd2540cc7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40064/0 (socket says 172.21.15.154:40064) 2023-09-23T03:19:17.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.578+0000 7fd253fff640 1 -- 172.21.15.154:0/2976749170 learned_addr learned my addr 172.21.15.154:0/2976749170 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:19:17.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.578+0000 7fd2537fe640 1 --2- 172.21.15.154:0/2976749170 >> v2:172.21.15.154:3300/0 conn(0x7fd2540c14e0 0x7fd2540c1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:17.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.579+0000 7fd258dbe640 1 --2- 172.21.15.154:0/2976749170 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd25406c1d0 0x7fd25406c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:17.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.579+0000 7fd252ffd640 1 -- 172.21.15.154:0/2976749170 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2057027505 0 0) 0x7fd2540cb160 con 0x7fd254161010 2023-09-23T03:19:17.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.579+0000 7fd252ffd640 1 -- 172.21.15.154:0/2976749170 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd240003520 con 0x7fd254161010 2023-09-23T03:19:17.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.579+0000 7fd2537fe640 1 -- 172.21.15.154:0/2976749170 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd25406c1d0 msgr2=0x7fd25406c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:17.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.579+0000 7fd2537fe640 1 --2- 172.21.15.154:0/2976749170 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd25406c1d0 0x7fd25406c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:17.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.579+0000 7fd2537fe640 1 -- 172.21.15.154:0/2976749170 >> v1:172.21.15.154:6789/0 conn(0x7fd254161010 legacy=0x7fd2540cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:17.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.579+0000 7fd2537fe640 1 -- 172.21.15.154:0/2976749170 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd25406cde0 con 0x7fd2540c14e0 2023-09-23T03:19:17.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.580+0000 7fd2537fe640 1 --2- 172.21.15.154:0/2976749170 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2540c14e0 0x7fd2540c1960 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fd23c0097a0 tx=0x7fd23c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a281449f9f7e287c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:17.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.580+0000 7fd252ffd640 1 -- 172.21.15.154:0/2976749170 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c02e070 con 0x7fd2540c14e0 2023-09-23T03:19:17.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.580+0000 7fd252ffd640 1 -- 172.21.15.154:0/2976749170 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd23c0315a0 con 0x7fd2540c14e0 2023-09-23T03:19:17.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.580+0000 7fd252ffd640 1 -- 172.21.15.154:0/2976749170 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c031910 con 0x7fd2540c14e0 2023-09-23T03:19:17.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.581+0000 7fd25a821640 1 -- 172.21.15.154:0/2976749170 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2540c14e0 msgr2=0x7fd2540c1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:17.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.581+0000 7fd25a821640 1 --2- 172.21.15.154:0/2976749170 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2540c14e0 0x7fd2540c1960 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fd23c0097a0 tx=0x7fd23c02b840 comp rx=0 tx=0).stop 2023-09-23T03:19:17.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.581+0000 7fd25a821640 1 -- 172.21.15.154:0/2976749170 shutdown_connections 2023-09-23T03:19:17.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.581+0000 7fd25a821640 1 --2- 172.21.15.154:0/2976749170 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd25406c1d0 0x7fd25406c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:17.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.581+0000 7fd25a821640 1 --2- 172.21.15.154:0/2976749170 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2540c14e0 0x7fd2540c1960 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:17.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.581+0000 7fd25a821640 1 -- 172.21.15.154:0/2976749170 >> 172.21.15.154:0/2976749170 conn(0x7fd25406a0f0 msgr2=0x7fd25406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:17.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.581+0000 7fd25a821640 1 -- 172.21.15.154:0/2976749170 shutdown_connections 2023-09-23T03:19:17.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.581+0000 7fd25a821640 1 -- 172.21.15.154:0/2976749170 wait complete. 2023-09-23T03:19:17.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.583+0000 7fd25a821640 1 Processor -- start 2023-09-23T03:19:17.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.583+0000 7fd25a821640 1 -- start start 2023-09-23T03:19:17.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.584+0000 7fd25a821640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd25406c1d0 0x7fd2540c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:17.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.584+0000 7fd25a821640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2540c14e0 0x7fd2541f9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:17.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.584+0000 7fd25a821640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd254161010 0x7fd2541fa030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:17.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.584+0000 7fd25a821640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25406fee0 con 0x7fd2540c14e0 2023-09-23T03:19:17.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.584+0000 7fd25a821640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd254070740 con 0x7fd25406c1d0 2023-09-23T03:19:17.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.584+0000 7fd25a821640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2540c4d80 con 0x7fd254161010 2023-09-23T03:19:17.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.584+0000 7fd2537fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2540c14e0 0x7fd2541f9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:17.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.584+0000 7fd2537fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2540c14e0 0x7fd2541f9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56418/0 (socket says 172.21.15.154:56418) 2023-09-23T03:19:17.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.584+0000 7fd2537fe640 1 -- 172.21.15.154:0/816020592 learned_addr learned my addr 172.21.15.154:0/816020592 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:17.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.585+0000 7fd253fff640 1 --2- 172.21.15.154:0/816020592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd25406c1d0 0x7fd2540c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:17.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.585+0000 7fd258dbe640 1 --2- 172.21.15.154:0/816020592 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd254161010 0x7fd2541fa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:17.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.585+0000 7fd2537fe640 1 -- 172.21.15.154:0/816020592 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd254161010 msgr2=0x7fd2541fa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:17.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.585+0000 7fd2537fe640 1 --2- 172.21.15.154:0/816020592 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd254161010 0x7fd2541fa030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:17.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.585+0000 7fd2537fe640 1 -- 172.21.15.154:0/816020592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd25406c1d0 msgr2=0x7fd2540c7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:17.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.585+0000 7fd2537fe640 1 --2- 172.21.15.154:0/816020592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd25406c1d0 0x7fd2540c7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:17.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.585+0000 7fd2537fe640 1 -- 172.21.15.154:0/816020592 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd23c009400 con 0x7fd2540c14e0 2023-09-23T03:19:17.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.585+0000 7fd2537fe640 1 --2- 172.21.15.154:0/816020592 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2540c14e0 0x7fd2541f9af0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fd23c0098c0 tx=0x7fd23c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:17.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.586+0000 7fd250ff9640 1 -- 172.21.15.154:0/816020592 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c003810 con 0x7fd2540c14e0 2023-09-23T03:19:17.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.586+0000 7fd250ff9640 1 -- 172.21.15.154:0/816020592 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd23c040070 con 0x7fd2540c14e0 2023-09-23T03:19:17.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.586+0000 7fd250ff9640 1 -- 172.21.15.154:0/816020592 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c003b70 con 0x7fd2540c14e0 2023-09-23T03:19:17.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.586+0000 7fd25a821640 1 -- 172.21.15.154:0/816020592 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2540700f0 con 0x7fd2540c14e0 2023-09-23T03:19:17.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.586+0000 7fd25a821640 1 -- 172.21.15.154:0/816020592 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2540c2580 con 0x7fd2540c14e0 2023-09-23T03:19:17.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.587+0000 7fd25a821640 1 -- 172.21.15.154:0/816020592 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2540708f0 con 0x7fd2540c14e0 2023-09-23T03:19:17.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.588+0000 7fd250ff9640 1 -- 172.21.15.154:0/816020592 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fd23c003cd0 con 0x7fd2540c14e0 2023-09-23T03:19:17.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.589+0000 7fd250ff9640 1 --2- 172.21.15.154:0/816020592 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd23406af60 0x7fd23406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:17.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.589+0000 7fd250ff9640 1 -- 172.21.15.154:0/816020592 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5521+0+0 (secure 0 0 0) 0x7fd23c042070 con 0x7fd2540c14e0 2023-09-23T03:19:17.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.589+0000 7fd253fff640 1 --2- 172.21.15.154:0/816020592 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd23406af60 0x7fd23406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:17.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.590+0000 7fd253fff640 1 --2- 172.21.15.154:0/816020592 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd23406af60 0x7fd23406d420 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd2440016e0 tx=0x7fd244009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:17.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.593+0000 7fd250ff9640 1 -- 172.21.15.154:0/816020592 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fd2540708f0 con 0x7fd2540c14e0 2023-09-23T03:19:17.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.817+0000 7fd25a821640 1 -- 172.21.15.154:0/816020592 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2540708b0 con 0x7fd2540c14e0 2023-09-23T03:19:17.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.819+0000 7fd250ff9640 1 -- 172.21.15.154:0/816020592 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+3711 (secure 0 0 0) 0x7fd23c04c090 con 0x7fd2540c14e0 2023-09-23T03:19:17.819 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:19:17.819 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":34,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5215,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3388818833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3388818833},{"type":"v1","addr":"172.21.15.190:6835","nonce":3388818833}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5991,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1964552060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1964552060},{"type":"v1","addr":"172.21.15.154:6837","nonce":1964552060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5994,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/1736999297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":1736999297},{"type":"v1","addr":"172.21.15.190:6834","nonce":1736999297}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6237,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/1039435342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1039435342},{"type":"v1","addr":"172.21.15.154:6836","nonce":1039435342}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34}],"filesystems":[]} 2023-09-23T03:19:17.819 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 34 2023-09-23T03:19:17.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.821+0000 7fd25a821640 1 -- 172.21.15.154:0/816020592 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd23406af60 msgr2=0x7fd23406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:17.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.821+0000 7fd25a821640 1 --2- 172.21.15.154:0/816020592 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd23406af60 0x7fd23406d420 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd2440016e0 tx=0x7fd244009040 comp rx=0 tx=0).stop 2023-09-23T03:19:17.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.821+0000 7fd25a821640 1 -- 172.21.15.154:0/816020592 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2540c14e0 msgr2=0x7fd2541f9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:17.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.821+0000 7fd25a821640 1 --2- 172.21.15.154:0/816020592 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2540c14e0 0x7fd2541f9af0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fd23c0098c0 tx=0x7fd23c0046c0 comp rx=0 tx=0).stop 2023-09-23T03:19:17.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.821+0000 7fd25a821640 1 -- 172.21.15.154:0/816020592 shutdown_connections 2023-09-23T03:19:17.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.821+0000 7fd25a821640 1 --2- 172.21.15.154:0/816020592 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd23406af60 0x7fd23406d420 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:17.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.821+0000 7fd25a821640 1 --2- 172.21.15.154:0/816020592 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd254161010 0x7fd2541fa030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:17.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.821+0000 7fd25a821640 1 --2- 172.21.15.154:0/816020592 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2540c14e0 0x7fd2541f9af0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:17.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.821+0000 7fd25a821640 1 --2- 172.21.15.154:0/816020592 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd25406c1d0 0x7fd2540c7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:17.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.821+0000 7fd25a821640 1 -- 172.21.15.154:0/816020592 >> 172.21.15.154:0/816020592 conn(0x7fd25406a0f0 msgr2=0x7fd2540c9290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:17.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.822+0000 7fd25a821640 1 -- 172.21.15.154:0/816020592 shutdown_connections 2023-09-23T03:19:17.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:17.822+0000 7fd25a821640 1 -- 172.21.15.154:0/816020592 wait complete. 2023-09-23T03:19:17.841 INFO:tasks.ceph.mds.a:Restarting daemon 2023-09-23T03:19:17.841 INFO:tasks.ceph.mds.a:Stopping old one... 2023-09-23T03:19:17.841 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-09-23T03:19:17.842 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:19:17.842 INFO:tasks.ceph.mds.c:Restarting daemon 2023-09-23T03:19:17.842 INFO:tasks.ceph.mds.c:Stopping old one... 2023-09-23T03:19:17.842 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-09-23T03:19:17.842 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:19:17.842 INFO:tasks.ceph.mds.b:Restarting daemon 2023-09-23T03:19:17.843 INFO:tasks.ceph.mds.b:Stopping old one... 2023-09-23T03:19:17.843 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-09-23T03:19:17.843 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:19:17.843 INFO:tasks.ceph.mds.d:Restarting daemon 2023-09-23T03:19:17.843 INFO:tasks.ceph.mds.d:Stopping old one... 2023-09-23T03:19:17.843 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-09-23T03:19:17.844 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:19:17.861 INFO:tasks.ceph.mds.c:Stopped 2023-09-23T03:19:17.861 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-09-23T03:19:17.865 INFO:tasks.ceph.mds.a:Stopped 2023-09-23T03:19:17.865 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-09-23T03:19:17.867 INFO:tasks.ceph.mds.d:Stopped 2023-09-23T03:19:17.868 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-09-23T03:19:17.870 INFO:tasks.ceph.mds.b:Stopped 2023-09-23T03:19:17.870 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-09-23T03:19:17.913 INFO:tasks.ceph.mds.c:Started 2023-09-23T03:19:17.913 DEBUG:teuthology.parallel:result is None 2023-09-23T03:19:17.918 INFO:tasks.ceph.mds.d:Started 2023-09-23T03:19:17.918 DEBUG:teuthology.parallel:result is None 2023-09-23T03:19:17.921 INFO:tasks.ceph.mds.a:Started 2023-09-23T03:19:17.921 DEBUG:teuthology.parallel:result is None 2023-09-23T03:19:17.925 INFO:tasks.ceph.mds.b:Started 2023-09-23T03:19:17.925 DEBUG:teuthology.parallel:result is None 2023-09-23T03:19:17.925 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-09-23T03:19:18.075 INFO:tasks.ceph.mds.a.smithi154.stdout:starting mds.a at 2023-09-23T03:19:18.075 INFO:tasks.ceph.mds.c.smithi154.stdout:starting mds.c at 2023-09-23T03:19:18.085 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-09-23T03:19:18.095 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-09-23T03:19:18.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.118+0000 7fd6cd6f3640 1 Processor -- start 2023-09-23T03:19:18.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.118+0000 7fd6cd6f3640 1 -- start start 2023-09-23T03:19:18.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.119+0000 7fd6cd6f3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6c80cf500 0x7fd6c80ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:18.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.119+0000 7fd6cd6f3640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6c806be10 con 0x7fd6c815cc00 2023-09-23T03:19:18.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.119+0000 7fd6cd6f3640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6c80d17f0 con 0x7fd6c80cf8e0 2023-09-23T03:19:18.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.120+0000 7fd6cd6f3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c80d0470 con 0x7fd6c80cf500 2023-09-23T03:19:18.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.120+0000 7fd6ccef2640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fd6c815cc00 0x7fd6c80b1430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40068/0 (socket says 172.21.15.154:40068) 2023-09-23T03:19:18.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.120+0000 7fd6ccef2640 1 -- 172.21.15.154:0/2516934364 learned_addr learned my addr 172.21.15.154:0/2516934364 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:19:18.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.120+0000 7fd6c7fff640 1 --2- 172.21.15.154:0/2516934364 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6c80cf500 0x7fd6c80ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:18.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.120+0000 7fd6c7fff640 1 -- 172.21.15.154:0/2516934364 >> v1:172.21.15.190:6789/0 conn(0x7fd6c80cf8e0 legacy=0x7fd6c815c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:18.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.120+0000 7fd6c7fff640 1 -- 172.21.15.154:0/2516934364 >> v1:172.21.15.154:6789/0 conn(0x7fd6c815cc00 legacy=0x7fd6c80b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:18.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.120+0000 7fd6c77fe640 1 -- 172.21.15.154:0/2516934364 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3957075492 0 0) 0x7fd6c806be10 con 0x7fd6c815cc00 2023-09-23T03:19:18.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.120+0000 7fd6c7fff640 1 -- 172.21.15.154:0/2516934364 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c80d0750 con 0x7fd6c80cf500 2023-09-23T03:19:18.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.122+0000 7fd6c7fff640 1 --2- 172.21.15.154:0/2516934364 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6c80cf500 0x7fd6c80ce050 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd6b80097c0 tx=0x7fd6b802b890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89a325f4c5976ee6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:18.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.122+0000 7fd6c77fe640 1 -- 172.21.15.154:0/2516934364 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b80317f0 con 0x7fd6c80cf500 2023-09-23T03:19:18.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.122+0000 7fd6c77fe640 1 -- 172.21.15.154:0/2516934364 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b8031950 con 0x7fd6c80cf500 2023-09-23T03:19:18.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.122+0000 7fd6c77fe640 1 -- 172.21.15.154:0/2516934364 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b8031ce0 con 0x7fd6c80cf500 2023-09-23T03:19:18.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.123+0000 7fd6cd6f3640 1 -- 172.21.15.154:0/2516934364 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6c80cf500 msgr2=0x7fd6c80ce050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:18.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.123+0000 7fd6cd6f3640 1 --2- 172.21.15.154:0/2516934364 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6c80cf500 0x7fd6c80ce050 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd6b80097c0 tx=0x7fd6b802b890 comp rx=0 tx=0).stop 2023-09-23T03:19:18.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.123+0000 7fd6cd6f3640 1 -- 172.21.15.154:0/2516934364 shutdown_connections 2023-09-23T03:19:18.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.123+0000 7fd6cd6f3640 1 --2- 172.21.15.154:0/2516934364 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6c80cf500 0x7fd6c80ce050 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:18.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.123+0000 7fd6cd6f3640 1 -- 172.21.15.154:0/2516934364 >> 172.21.15.154:0/2516934364 conn(0x7fd6c809bbb0 msgr2=0x7fd6c80d28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:18.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.123+0000 7fd6cd6f3640 1 -- 172.21.15.154:0/2516934364 shutdown_connections 2023-09-23T03:19:18.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.123+0000 7fd6cd6f3640 1 -- 172.21.15.154:0/2516934364 wait complete. 2023-09-23T03:19:18.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.125+0000 7fd6cd6f3640 1 Processor -- start 2023-09-23T03:19:18.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.125+0000 7fd6cd6f3640 1 -- start start 2023-09-23T03:19:18.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.126+0000 7fd6cd6f3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6c80cf500 0x7fd6c81539e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:18.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.126+0000 7fd6cd6f3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6c80cf8e0 0x7fd6c8148b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:18.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.126+0000 7fd6cd6f3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6c815cc00 0x7fd6c80cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:18.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.126+0000 7fd6cd6f3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c8153f20 con 0x7fd6c815cc00 2023-09-23T03:19:18.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.126+0000 7fd6cd6f3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c80c73c0 con 0x7fd6c80cf8e0 2023-09-23T03:19:18.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.126+0000 7fd6cd6f3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c80c9a10 con 0x7fd6c80cf500 2023-09-23T03:19:18.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.126+0000 7fd6ccef2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6c815cc00 0x7fd6c80cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:18.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.126+0000 7fd6ccef2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6c815cc00 0x7fd6c80cbe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56448/0 (socket says 172.21.15.154:56448) 2023-09-23T03:19:18.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.126+0000 7fd6ccef2640 1 -- 172.21.15.154:0/1642253303 learned_addr learned my addr 172.21.15.154:0/1642253303 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:18.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.127+0000 7fd6bffff640 1 --2- 172.21.15.154:0/1642253303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6c80cf8e0 0x7fd6c8148b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:18.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.127+0000 7fd6c7fff640 1 --2- 172.21.15.154:0/1642253303 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6c80cf500 0x7fd6c81539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:18.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.127+0000 7fd6ccef2640 1 -- 172.21.15.154:0/1642253303 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6c80cf500 msgr2=0x7fd6c81539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:18.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.127+0000 7fd6ccef2640 1 --2- 172.21.15.154:0/1642253303 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6c80cf500 0x7fd6c81539e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:18.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.127+0000 7fd6ccef2640 1 -- 172.21.15.154:0/1642253303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6c80cf8e0 msgr2=0x7fd6c8148b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:18.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.127+0000 7fd6ccef2640 1 --2- 172.21.15.154:0/1642253303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6c80cf8e0 0x7fd6c8148b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:18.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.127+0000 7fd6ccef2640 1 -- 172.21.15.154:0/1642253303 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b80093a0 con 0x7fd6c815cc00 2023-09-23T03:19:18.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.127+0000 7fd6c7fff640 1 --2- 172.21.15.154:0/1642253303 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6c80cf500 0x7fd6c81539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:18.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.127+0000 7fd6bffff640 1 --2- 172.21.15.154:0/1642253303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6c80cf8e0 0x7fd6c8148b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:19:18.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.127+0000 7fd6ccef2640 1 --2- 172.21.15.154:0/1642253303 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6c815cc00 0x7fd6c80cbe70 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fd6c000c7e0 tx=0x7fd6c000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:18.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.127+0000 7fd6c57fa640 1 -- 172.21.15.154:0/1642253303 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c000a640 con 0x7fd6c815cc00 2023-09-23T03:19:18.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.127+0000 7fd6c57fa640 1 -- 172.21.15.154:0/1642253303 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c000a7c0 con 0x7fd6c815cc00 2023-09-23T03:19:18.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.127+0000 7fd6c57fa640 1 -- 172.21.15.154:0/1642253303 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c000ab70 con 0x7fd6c815cc00 2023-09-23T03:19:18.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.128+0000 7fd6cd6f3640 1 -- 172.21.15.154:0/1642253303 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6c80c9cf0 con 0x7fd6c815cc00 2023-09-23T03:19:18.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.128+0000 7fd6cd6f3640 1 -- 172.21.15.154:0/1642253303 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6c80ca2b0 con 0x7fd6c815cc00 2023-09-23T03:19:18.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.129+0000 7fd6cd6f3640 1 -- 172.21.15.154:0/1642253303 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6c80ca4d0 con 0x7fd6c815cc00 2023-09-23T03:19:18.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.130+0000 7fd6c57fa640 1 -- 172.21.15.154:0/1642253303 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fd6c000acd0 con 0x7fd6c815cc00 2023-09-23T03:19:18.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.131+0000 7fd6c57fa640 1 --2- 172.21.15.154:0/1642253303 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd6a806af30 0x7fd6a806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:18.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.131+0000 7fd6c57fa640 1 -- 172.21.15.154:0/1642253303 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5521+0+0 (secure 0 0 0) 0x7fd6c0088d40 con 0x7fd6c815cc00 2023-09-23T03:19:18.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.131+0000 7fd6c7fff640 1 --2- 172.21.15.154:0/1642253303 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd6a806af30 0x7fd6a806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:18.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.132+0000 7fd6c7fff640 1 --2- 172.21.15.154:0/1642253303 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd6a806af30 0x7fd6a806d3f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8036000 tx=0x7fd6b803d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:18.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.135+0000 7fd6c57fa640 1 -- 172.21.15.154:0/1642253303 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fd6c0053d90 con 0x7fd6c815cc00 2023-09-23T03:19:18.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:18.334+0000 7fd6cd6f3640 1 -- 172.21.15.154:0/1642253303 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fd6c804e920 con 0x7fd6c815cc00 2023-09-23T03:19:19.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.448+0000 7fd6c57fa640 1 -- 172.21.15.154:0/1642253303 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v58) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd6c0018070 con 0x7fd6c815cc00 2023-09-23T03:19:19.448 INFO:teuthology.orchestra.run.smithi154.stderr: removed all blocklist entries 2023-09-23T03:19:19.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.451+0000 7fd6cd6f3640 1 -- 172.21.15.154:0/1642253303 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd6a806af30 msgr2=0x7fd6a806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:19.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.451+0000 7fd6cd6f3640 1 --2- 172.21.15.154:0/1642253303 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd6a806af30 0x7fd6a806d3f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8036000 tx=0x7fd6b803d040 comp rx=0 tx=0).stop 2023-09-23T03:19:19.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.451+0000 7fd6cd6f3640 1 -- 172.21.15.154:0/1642253303 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6c815cc00 msgr2=0x7fd6c80cbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:19.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.451+0000 7fd6cd6f3640 1 --2- 172.21.15.154:0/1642253303 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6c815cc00 0x7fd6c80cbe70 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fd6c000c7e0 tx=0x7fd6c000cca0 comp rx=0 tx=0).stop 2023-09-23T03:19:19.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.452+0000 7fd6cd6f3640 1 -- 172.21.15.154:0/1642253303 shutdown_connections 2023-09-23T03:19:19.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.452+0000 7fd6cd6f3640 1 --2- 172.21.15.154:0/1642253303 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd6a806af30 0x7fd6a806d3f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:19.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.452+0000 7fd6cd6f3640 1 --2- 172.21.15.154:0/1642253303 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6c815cc00 0x7fd6c80cbe70 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:19.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.452+0000 7fd6cd6f3640 1 --2- 172.21.15.154:0/1642253303 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6c80cf8e0 0x7fd6c8148b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:19.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.452+0000 7fd6cd6f3640 1 --2- 172.21.15.154:0/1642253303 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6c80cf500 0x7fd6c81539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:19.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.452+0000 7fd6cd6f3640 1 -- 172.21.15.154:0/1642253303 >> 172.21.15.154:0/1642253303 conn(0x7fd6c809bbb0 msgr2=0x7fd6c80b54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:19.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.452+0000 7fd6cd6f3640 1 -- 172.21.15.154:0/1642253303 shutdown_connections 2023-09-23T03:19:19.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.452+0000 7fd6cd6f3640 1 -- 172.21.15.154:0/1642253303 wait complete. 2023-09-23T03:19:19.471 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-09-23T03:19:19.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.684+0000 7fe2ffa44640 1 Processor -- start 2023-09-23T03:19:19.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.685+0000 7fe2ffa44640 1 -- start start 2023-09-23T03:19:19.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.686+0000 7fe2ffa44640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2f815cf40 0x7fe2f81615b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:19.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.686+0000 7fe2ffa44640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe2f80b28e0 0x7fe2f80cca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:19.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.686+0000 7fe2ffa44640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2f80ccf80 con 0x7fe2f806b1e0 2023-09-23T03:19:19.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.686+0000 7fe2ffa44640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2f80cd1c0 con 0x7fe2f80b28e0 2023-09-23T03:19:19.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.686+0000 7fe2ffa44640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f80cd4a0 con 0x7fe2f815cf40 2023-09-23T03:19:19.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.687+0000 7fe2fdfe1640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe2f80b28e0 0x7fe2f80cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:19.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.687+0000 7fe2fdfe1640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe2f80b28e0 0x7fe2f80cca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56464/0 (socket says 172.21.15.154:56464) 2023-09-23T03:19:19.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.687+0000 7fe2fdfe1640 1 -- 172.21.15.154:0/70931194 learned_addr learned my addr 172.21.15.154:0/70931194 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:19.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.687+0000 7fe2fd7e0640 1 --2- 172.21.15.154:0/70931194 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2f815cf40 0x7fe2f81615b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:19.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.687+0000 7fe2fdfe1640 1 -- 172.21.15.154:0/70931194 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2f815cf40 msgr2=0x7fe2f81615b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:19.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.687+0000 7fe2fdfe1640 1 --2- 172.21.15.154:0/70931194 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2f815cf40 0x7fe2f81615b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:19.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.687+0000 7fe2fdfe1640 1 -- 172.21.15.154:0/70931194 >> v1:172.21.15.154:6789/0 conn(0x7fe2f806b1e0 legacy=0x7fe2f80b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:19.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.687+0000 7fe2fdfe1640 1 -- 172.21.15.154:0/70931194 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f80cd780 con 0x7fe2f80b28e0 2023-09-23T03:19:19.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.687+0000 7fe2effff640 1 -- 172.21.15.154:0/70931194 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3701995420 0 0) 0x7fe2e8001110 con 0x7fe2f806b1e0 2023-09-23T03:19:19.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.688+0000 7fe2fdfe1640 1 --2- 172.21.15.154:0/70931194 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2f80b28e0 0x7fe2f80cca40 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0009e50 tx=0x7fe2f002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1de863ba92707a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:19.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.689+0000 7fe2effff640 1 -- 172.21.15.154:0/70931194 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f002cb00 con 0x7fe2f80b28e0 2023-09-23T03:19:19.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.689+0000 7fe2effff640 1 -- 172.21.15.154:0/70931194 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f002cc60 con 0x7fe2f80b28e0 2023-09-23T03:19:19.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.689+0000 7fe2effff640 1 -- 172.21.15.154:0/70931194 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f00343f0 con 0x7fe2f80b28e0 2023-09-23T03:19:19.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.689+0000 7fe2ffa44640 1 -- 172.21.15.154:0/70931194 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2f80b28e0 msgr2=0x7fe2f80cca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:19.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.689+0000 7fe2ffa44640 1 --2- 172.21.15.154:0/70931194 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2f80b28e0 0x7fe2f80cca40 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0009e50 tx=0x7fe2f002bf60 comp rx=0 tx=0).stop 2023-09-23T03:19:19.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.689+0000 7fe2ffa44640 1 -- 172.21.15.154:0/70931194 shutdown_connections 2023-09-23T03:19:19.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.689+0000 7fe2ffa44640 1 --2- 172.21.15.154:0/70931194 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2f80b28e0 0x7fe2f80cca40 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:19.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.689+0000 7fe2ffa44640 1 --2- 172.21.15.154:0/70931194 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2f815cf40 0x7fe2f81615b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:19.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.689+0000 7fe2ffa44640 1 -- 172.21.15.154:0/70931194 >> 172.21.15.154:0/70931194 conn(0x7fe2f806fc60 msgr2=0x7fe2f8070040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:19.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.689+0000 7fe2ffa44640 1 -- 172.21.15.154:0/70931194 shutdown_connections 2023-09-23T03:19:19.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.689+0000 7fe2ffa44640 1 -- 172.21.15.154:0/70931194 wait complete. 2023-09-23T03:19:19.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.691+0000 7fe2ffa44640 1 Processor -- start 2023-09-23T03:19:19.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.691+0000 7fe2ffa44640 1 -- start start 2023-09-23T03:19:19.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.692+0000 7fe2ffa44640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2f806b1e0 0x7fe2f81f20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:19.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.692+0000 7fe2ffa44640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2f80b28e0 0x7fe2f80cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:19.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.692+0000 7fe2ffa44640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2f815cf40 0x7fe2f80cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:19.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.692+0000 7fe2ffa44640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f815f0e0 con 0x7fe2f815cf40 2023-09-23T03:19:19.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.692+0000 7fe2ffa44640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f80c4280 con 0x7fe2f806b1e0 2023-09-23T03:19:19.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.692+0000 7fe2ffa44640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f81f6b80 con 0x7fe2f80b28e0 2023-09-23T03:19:19.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.692+0000 7fe2fdfe1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2f815cf40 0x7fe2f80cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:19.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.692+0000 7fe2fdfe1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2f815cf40 0x7fe2f80cc3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56470/0 (socket says 172.21.15.154:56470) 2023-09-23T03:19:19.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.692+0000 7fe2fdfe1640 1 -- 172.21.15.154:0/2448121033 learned_addr learned my addr 172.21.15.154:0/2448121033 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:19.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.693+0000 7fe2fdfe1640 1 -- 172.21.15.154:0/2448121033 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2f80b28e0 msgr2=0x7fe2f80cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:19.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.693+0000 7fe2fdfe1640 1 --2- 172.21.15.154:0/2448121033 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2f80b28e0 0x7fe2f80cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:19.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.693+0000 7fe2fdfe1640 1 -- 172.21.15.154:0/2448121033 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2f806b1e0 msgr2=0x7fe2f81f20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:19.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.693+0000 7fe2fdfe1640 1 --2- 172.21.15.154:0/2448121033 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2f806b1e0 0x7fe2f81f20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:19.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.693+0000 7fe2fdfe1640 1 -- 172.21.15.154:0/2448121033 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f0009b30 con 0x7fe2f815cf40 2023-09-23T03:19:19.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.693+0000 7fe2fdfe1640 1 --2- 172.21.15.154:0/2448121033 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2f815cf40 0x7fe2f80cc3f0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0009e50 tx=0x7fe2f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:19.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.693+0000 7fe2edffb640 1 -- 172.21.15.154:0/2448121033 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f0009510 con 0x7fe2f815cf40 2023-09-23T03:19:19.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.693+0000 7fe2edffb640 1 -- 172.21.15.154:0/2448121033 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f003e450 con 0x7fe2f815cf40 2023-09-23T03:19:19.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.693+0000 7fe2edffb640 1 -- 172.21.15.154:0/2448121033 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f003e600 con 0x7fe2f815cf40 2023-09-23T03:19:19.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.694+0000 7fe2ffa44640 1 -- 172.21.15.154:0/2448121033 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2f815ea90 con 0x7fe2f815cf40 2023-09-23T03:19:19.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.694+0000 7fe2ffa44640 1 -- 172.21.15.154:0/2448121033 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f81f25f0 con 0x7fe2f815cf40 2023-09-23T03:19:19.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.695+0000 7fe2ffa44640 1 -- 172.21.15.154:0/2448121033 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2f815e8f0 con 0x7fe2f815cf40 2023-09-23T03:19:19.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.696+0000 7fe2edffb640 1 -- 172.21.15.154:0/2448121033 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fe2f003e760 con 0x7fe2f815cf40 2023-09-23T03:19:19.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.697+0000 7fe2edffb640 1 --2- 172.21.15.154:0/2448121033 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe2dc06af30 0x7fe2dc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:19.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.697+0000 7fe2edffb640 1 -- 172.21.15.154:0/2448121033 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fe2f00abf40 con 0x7fe2f815cf40 2023-09-23T03:19:19.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.698+0000 7fe2fd7e0640 1 --2- 172.21.15.154:0/2448121033 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe2dc06af30 0x7fe2dc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:19.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.698+0000 7fe2fd7e0640 1 --2- 172.21.15.154:0/2448121033 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe2dc06af30 0x7fe2dc06d3f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4006fd0 tx=0x7fe2f4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:19.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.702+0000 7fe2edffb640 1 -- 172.21.15.154:0/2448121033 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fe2f815e8f0 con 0x7fe2f815cf40 2023-09-23T03:19:19.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.964+0000 7fe2ffa44640 1 -- 172.21.15.154:0/2448121033 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe2f815e8f0 con 0x7fe2f815cf40 2023-09-23T03:19:19.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.965+0000 7fe2edffb640 1 -- 172.21.15.154:0/2448121033 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v10) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fe2f815e8f0 con 0x7fe2f815cf40 2023-09-23T03:19:19.966 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:19:19.966 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:19:19.966 INFO:teuthology.orchestra.run.smithi154.stdout: "auth_dump": [ 2023-09-23T03:19:19.966 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:19:19.966 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mds.a", 2023-09-23T03:19:19.966 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAGWA5l9biCFhAAhD2xgY9Q/gGqK0pgHRGweQ==", 2023-09-23T03:19:19.967 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:19:19.967 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:19:19.967 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2023-09-23T03:19:19.967 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2023-09-23T03:19:19.967 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:19:19.967 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:19:19.967 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:19:19.967 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:19:19.967 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mds.b", 2023-09-23T03:19:19.967 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAGWA5lsgKCNxAAI+vD2ESh3uNsb5OCO0ydSw==", 2023-09-23T03:19:19.967 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:19:19.967 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:19:19.968 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2023-09-23T03:19:19.968 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2023-09-23T03:19:19.968 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:19:19.968 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:19:19.968 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:19:19.968 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:19:19.968 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mds.c", 2023-09-23T03:19:19.968 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAGWA5los4vJhAA2/ewjMZcc3pnm9ukhnoNgw==", 2023-09-23T03:19:19.968 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:19:19.968 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:19:19.968 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2023-09-23T03:19:19.968 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2023-09-23T03:19:19.968 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:19:19.969 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:19:19.969 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:19:19.969 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:19:19.969 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mds.d", 2023-09-23T03:19:19.969 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAHWA5lml/MDBAAHyXgd5zl/O8CeVE8Da4w5w==", 2023-09-23T03:19:19.969 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:19:19.969 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:19:19.969 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2023-09-23T03:19:19.969 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2023-09-23T03:19:19.969 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:19:19.969 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:19:19.969 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:19:19.970 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:19:19.970 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.0", 2023-09-23T03:19:19.970 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAMWA5lT1R4BBAAI+itqub/FadG3aU6LhnmGg==", 2023-09-23T03:19:19.970 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:19:19.970 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:19:19.970 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:19:19.970 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:19:19.970 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:19:19.970 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:19:19.970 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:19:19.970 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.1", 2023-09-23T03:19:19.970 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAPWA5lSablHRAAmNyr5Zopeihn5ZyoY5pyhA==", 2023-09-23T03:19:19.971 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:19:19.971 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:19:19.971 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:19:19.971 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:19:19.971 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:19:19.971 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:19:19.971 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:19:19.971 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.2", 2023-09-23T03:19:19.971 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQASWA5lSFcdJxAA3UJJ6eSvxcQ31SAm4shV2g==", 2023-09-23T03:19:19.971 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:19:19.971 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:19:19.971 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:19:19.972 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:19:19.972 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:19:19.972 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:19:19.972 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:19:19.972 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.3", 2023-09-23T03:19:19.972 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAVWA5lmSvzMRAA97YDqX3oplHCB+SENIkLYA==", 2023-09-23T03:19:19.972 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:19:19.972 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:19:19.972 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:19:19.972 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:19:19.972 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:19:19.972 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:19:19.972 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:19:19.973 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.4", 2023-09-23T03:19:19.973 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAcWA5lPZqwMBAAW95bBB0PKS/rwhXvrPf+mQ==", 2023-09-23T03:19:19.973 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:19:19.973 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:19:19.973 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:19:19.973 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:19:19.973 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:19:19.973 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:19:19.973 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:19:19.973 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.5", 2023-09-23T03:19:19.973 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAgWA5lmZySDRAAsbnUdWq9esKFSqSDeqsVlA==", 2023-09-23T03:19:19.973 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:19:19.973 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:19:19.974 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:19:19.974 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:19:19.974 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:19:19.974 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:19:19.974 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:19:19.974 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.6", 2023-09-23T03:19:19.974 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAjWA5ll9WqJRAAD9jecVxVJU+yuqANvN9QUQ==", 2023-09-23T03:19:19.974 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:19:19.974 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:19:19.974 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:19:19.974 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:19:19.974 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:19:19.974 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:19:19.975 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:19:19.975 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.7", 2023-09-23T03:19:19.975 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAnWA5lCvexAxAA/QymygbvOt1HQdCGbO8cyg==", 2023-09-23T03:19:19.975 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:19:19.975 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:19:19.975 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:19:19.975 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:19:19.975 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:19:19.975 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:19:19.975 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:19:19.975 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.0", 2023-09-23T03:19:19.975 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAHWA5lYg+qGhAAjaY0fPsmRTz3ipsKbPugMA==", 2023-09-23T03:19:19.976 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:19:19.976 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:19:19.976 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow r", 2023-09-23T03:19:19.976 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-09-23T03:19:19.976 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:19:19.976 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:19:19.976 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:19:19.976 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.1", 2023-09-23T03:19:19.976 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAHWA5lbrrGMBAAn4kvS/1xGjZ+GtRZFqy31w==", 2023-09-23T03:19:19.976 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:19:19.976 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:19:19.976 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow r", 2023-09-23T03:19:19.976 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow rw", 2023-09-23T03:19:19.977 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow rwx" 2023-09-23T03:19:19.977 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:19:19.977 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:19:19.977 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:19:19.977 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.2", 2023-09-23T03:19:19.977 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAHWA5lcLkIJRAAOknGCKwUs78D4atRzLb/Cw==", 2023-09-23T03:19:19.977 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:19:19.977 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:19:19.977 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow r", 2023-09-23T03:19:19.977 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow rw", 2023-09-23T03:19:19.977 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow rwx" 2023-09-23T03:19:19.977 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:19:19.978 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:19:19.978 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:19:19.978 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.3", 2023-09-23T03:19:19.978 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAIWA5lmFUaABAA4EBSJP5IidG2uq4g8lrFKg==", 2023-09-23T03:19:19.978 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:19:19.978 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:19:19.978 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow r", 2023-09-23T03:19:19.978 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow rw", 2023-09-23T03:19:19.978 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow rwx" 2023-09-23T03:19:19.978 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:19:19.978 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:19:19.978 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:19:19.979 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.admin", 2023-09-23T03:19:19.979 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAFWA5l0x+tCxAAGEYW1egctWzycEhM6FjJZw==", 2023-09-23T03:19:19.979 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:19:19.979 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow *", 2023-09-23T03:19:19.979 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2023-09-23T03:19:19.979 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2023-09-23T03:19:19.979 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:19:19.979 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:19:19.979 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:19:19.979 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:19:19.979 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mgr.x", 2023-09-23T03:19:19.979 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAGWA5ljcy5ChAACUOnrxEGqacSxrB9BkZ+lg==", 2023-09-23T03:19:19.979 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:19:19.980 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow *", 2023-09-23T03:19:19.980 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile mgr", 2023-09-23T03:19:19.980 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:19:19.980 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:19:19.980 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:19:19.980 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:19:19.980 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mgr.y", 2023-09-23T03:19:19.980 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAFWA5looGaOhAAdxmefaoLuXesHXCeqeCYXQ==", 2023-09-23T03:19:19.980 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:19:19.980 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow *", 2023-09-23T03:19:19.980 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile mgr", 2023-09-23T03:19:19.980 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:19:19.980 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:19:19.981 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:19:19.981 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2023-09-23T03:19:19.981 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:19:19.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.968+0000 7fe2ffa44640 1 -- 172.21.15.154:0/2448121033 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe2dc06af30 msgr2=0x7fe2dc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:19.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.968+0000 7fe2ffa44640 1 --2- 172.21.15.154:0/2448121033 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe2dc06af30 0x7fe2dc06d3f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4006fd0 tx=0x7fe2f4008040 comp rx=0 tx=0).stop 2023-09-23T03:19:19.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.968+0000 7fe2ffa44640 1 -- 172.21.15.154:0/2448121033 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2f815cf40 msgr2=0x7fe2f80cc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:19.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.968+0000 7fe2ffa44640 1 --2- 172.21.15.154:0/2448121033 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2f815cf40 0x7fe2f80cc3f0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0009e50 tx=0x7fe2f0004a40 comp rx=0 tx=0).stop 2023-09-23T03:19:19.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.968+0000 7fe2ffa44640 1 -- 172.21.15.154:0/2448121033 shutdown_connections 2023-09-23T03:19:19.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.968+0000 7fe2ffa44640 1 --2- 172.21.15.154:0/2448121033 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe2dc06af30 0x7fe2dc06d3f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:19.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.969+0000 7fe2ffa44640 1 --2- 172.21.15.154:0/2448121033 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2f815cf40 0x7fe2f80cc3f0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:19.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.969+0000 7fe2ffa44640 1 --2- 172.21.15.154:0/2448121033 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2f80b28e0 0x7fe2f80cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:19.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.969+0000 7fe2ffa44640 1 --2- 172.21.15.154:0/2448121033 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2f806b1e0 0x7fe2f81f20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:19.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.969+0000 7fe2ffa44640 1 -- 172.21.15.154:0/2448121033 >> 172.21.15.154:0/2448121033 conn(0x7fe2f806fc60 msgr2=0x7fe2f815de90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:19.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.969+0000 7fe2ffa44640 1 -- 172.21.15.154:0/2448121033 shutdown_connections 2023-09-23T03:19:19.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:19.969+0000 7fe2ffa44640 1 -- 172.21.15.154:0/2448121033 wait complete. 2023-09-23T03:19:19.988 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:19:20.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.204+0000 7f2476078640 1 Processor -- start 2023-09-23T03:19:20.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.204+0000 7f2476078640 1 -- start start 2023-09-23T03:19:20.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.205+0000 7f2476078640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f247006c7a0 0x7f247015eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:20.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.205+0000 7f2476078640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f247015ab00 0x7f247015af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:20.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.205+0000 7f2476078640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f247006bff0 con 0x7f247006fdc0 2023-09-23T03:19:20.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.205+0000 7f2476078640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2470155070 con 0x7f247006c7a0 2023-09-23T03:19:20.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.205+0000 7f2476078640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f247015e0f0 con 0x7f247015ab00 2023-09-23T03:19:20.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.206+0000 7f2475076640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f247006c7a0 0x7f247015eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:20.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.206+0000 7f2474875640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f247015ab00 0x7f247015af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:20.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.206+0000 7f2475076640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f247006c7a0 0x7f247015eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.154:36652/0 (socket says 172.21.15.154:36652) 2023-09-23T03:19:20.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.206+0000 7f2475076640 1 -- 172.21.15.154:0/1354204979 learned_addr learned my addr 172.21.15.154:0/1354204979 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:20.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.206+0000 7f2474875640 1 --2- 172.21.15.154:0/1354204979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f247015ab00 0x7f247015af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:47930/0 (socket says 172.21.15.154:47930) 2023-09-23T03:19:20.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.206+0000 7f2475877640 1 --1- 172.21.15.154:0/1354204979 >> v1:172.21.15.190:6789/0 conn(0x7f247006fdc0 0x7f24700ca500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.154:38132/0 (socket says 172.21.15.154:38132) 2023-09-23T03:19:20.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.207+0000 7f2474875640 1 -- 172.21.15.154:0/1354204979 >> v2:172.21.15.190:3300/0 conn(0x7f247006c7a0 msgr2=0x7f247015eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:20.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.207+0000 7f2474875640 1 --2- 172.21.15.154:0/1354204979 >> v2:172.21.15.190:3300/0 conn(0x7f247006c7a0 0x7f247015eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:20.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.207+0000 7f2474875640 1 -- 172.21.15.154:0/1354204979 >> v1:172.21.15.190:6789/0 conn(0x7f247006fdc0 legacy=0x7f24700ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:20.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.207+0000 7f2474875640 1 -- 172.21.15.154:0/1354204979 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f247006cb80 con 0x7f247015ab00 2023-09-23T03:19:20.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.209+0000 7f2474875640 1 --2- 172.21.15.154:0/1354204979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f247015ab00 0x7f247015af80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f24600097a0 tx=0x7f246002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=143e83965768636c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:20.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.209+0000 7f2467fff640 1 -- 172.21.15.154:0/1354204979 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246002d040 con 0x7f247015ab00 2023-09-23T03:19:20.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.209+0000 7f2467fff640 1 -- 172.21.15.154:0/1354204979 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2460031810 con 0x7f247015ab00 2023-09-23T03:19:20.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.209+0000 7f2467fff640 1 -- 172.21.15.154:0/1354204979 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2460031ba0 con 0x7f247015ab00 2023-09-23T03:19:20.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.210+0000 7f2476078640 1 -- 172.21.15.154:0/1354204979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f247015ab00 msgr2=0x7f247015af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:20.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.210+0000 7f2476078640 1 --2- 172.21.15.154:0/1354204979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f247015ab00 0x7f247015af80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f24600097a0 tx=0x7f246002b8b0 comp rx=0 tx=0).stop 2023-09-23T03:19:20.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.210+0000 7f2476078640 1 -- 172.21.15.154:0/1354204979 shutdown_connections 2023-09-23T03:19:20.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.210+0000 7f2476078640 1 --2- 172.21.15.154:0/1354204979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f247015ab00 0x7f247015af80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:20.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.210+0000 7f2476078640 1 --2- 172.21.15.154:0/1354204979 >> v2:172.21.15.190:3300/0 conn(0x7f247006c7a0 0x7f247015eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:20.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.210+0000 7f2476078640 1 -- 172.21.15.154:0/1354204979 >> 172.21.15.154:0/1354204979 conn(0x7f247006a390 msgr2=0x7f24700afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:20.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.210+0000 7f2476078640 1 -- 172.21.15.154:0/1354204979 shutdown_connections 2023-09-23T03:19:20.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.210+0000 7f2476078640 1 -- 172.21.15.154:0/1354204979 wait complete. 2023-09-23T03:19:20.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.212+0000 7f2476078640 1 Processor -- start 2023-09-23T03:19:20.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.212+0000 7f2476078640 1 -- start start 2023-09-23T03:19:20.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.213+0000 7f2476078640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f247006c7a0 0x7f2470152730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:20.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.213+0000 7f2476078640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f247006fdc0 0x7f24700c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:20.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.213+0000 7f2476078640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f247015ab00 0x7f24700c1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:20.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.213+0000 7f2476078640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24700b0f90 con 0x7f247006fdc0 2023-09-23T03:19:20.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.213+0000 7f2476078640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24700c5930 con 0x7f247006c7a0 2023-09-23T03:19:20.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.213+0000 7f2476078640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24700c5c10 con 0x7f247015ab00 2023-09-23T03:19:20.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.213+0000 7f2474875640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f247006fdc0 0x7f24700c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:20.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.213+0000 7f2474875640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f247006fdc0 0x7f24700c6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56486/0 (socket says 172.21.15.154:56486) 2023-09-23T03:19:20.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.213+0000 7f2474875640 1 -- 172.21.15.154:0/4073644833 learned_addr learned my addr 172.21.15.154:0/4073644833 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:20.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.214+0000 7f2475076640 1 --2- 172.21.15.154:0/4073644833 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f247006c7a0 0x7f2470152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:20.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.214+0000 7f2475877640 1 --2- 172.21.15.154:0/4073644833 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f247015ab00 0x7f24700c1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:20.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.214+0000 7f2474875640 1 -- 172.21.15.154:0/4073644833 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f247015ab00 msgr2=0x7f24700c1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:20.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.214+0000 7f2474875640 1 --2- 172.21.15.154:0/4073644833 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f247015ab00 0x7f24700c1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:20.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.214+0000 7f2474875640 1 -- 172.21.15.154:0/4073644833 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f247006c7a0 msgr2=0x7f2470152730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:20.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.214+0000 7f2474875640 1 --2- 172.21.15.154:0/4073644833 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f247006c7a0 0x7f2470152730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:20.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.214+0000 7f2474875640 1 -- 172.21.15.154:0/4073644833 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2460009400 con 0x7f247006fdc0 2023-09-23T03:19:20.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.214+0000 7f2475076640 1 --2- 172.21.15.154:0/4073644833 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f247006c7a0 0x7f2470152730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:20.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.214+0000 7f2474875640 1 --2- 172.21.15.154:0/4073644833 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f247006fdc0 0x7f24700c6db0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f2460000c00 tx=0x7f24600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:20.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.214+0000 7f2465ffb640 1 -- 172.21.15.154:0/4073644833 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2460006e10 con 0x7f247006fdc0 2023-09-23T03:19:20.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.214+0000 7f2465ffb640 1 -- 172.21.15.154:0/4073644833 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f246003ad60 con 0x7f247006fdc0 2023-09-23T03:19:20.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.214+0000 7f2465ffb640 1 -- 172.21.15.154:0/4073644833 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2460044630 con 0x7f247006fdc0 2023-09-23T03:19:20.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.214+0000 7f2476078640 1 -- 172.21.15.154:0/4073644833 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24700c5ef0 con 0x7f247006fdc0 2023-09-23T03:19:20.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.215+0000 7f2476078640 1 -- 172.21.15.154:0/4073644833 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2470152c70 con 0x7f247006fdc0 2023-09-23T03:19:20.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.216+0000 7f2476078640 1 -- 172.21.15.154:0/4073644833 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24700736b0 con 0x7f247006fdc0 2023-09-23T03:19:20.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.216+0000 7f2465ffb640 1 -- 172.21.15.154:0/4073644833 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f246004d450 con 0x7f247006fdc0 2023-09-23T03:19:20.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.217+0000 7f2465ffb640 1 --2- 172.21.15.154:0/4073644833 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f245406af60 0x7f245406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:20.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.217+0000 7f2465ffb640 1 -- 172.21.15.154:0/4073644833 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f24600ab8b0 con 0x7f247006fdc0 2023-09-23T03:19:20.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.218+0000 7f2475076640 1 --2- 172.21.15.154:0/4073644833 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f245406af60 0x7f245406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:20.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.218+0000 7f2475076640 1 --2- 172.21.15.154:0/4073644833 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f245406af60 0x7f245406d420 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f246c009ac0 tx=0x7f246c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:20.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.222+0000 7f2465ffb640 1 -- 172.21.15.154:0/4073644833 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f24600769d0 con 0x7f247006fdc0 2023-09-23T03:19:20.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.415+0000 7f2476078640 1 -- 172.21.15.154:0/4073644833 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f247004e920 con 0x7f247006fdc0 2023-09-23T03:19:20.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.417+0000 7f2465ffb640 1 -- 172.21.15.154:0/4073644833 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+13098 (secure 0 0 0) 0x7f2460035020 con 0x7f247006fdc0 2023-09-23T03:19:20.417 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:19:20.418 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":58,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:19:19.444908+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:19:20.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.420+0000 7f2476078640 1 -- 172.21.15.154:0/4073644833 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f245406af60 msgr2=0x7f245406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:20.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.420+0000 7f2476078640 1 --2- 172.21.15.154:0/4073644833 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f245406af60 0x7f245406d420 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f246c009ac0 tx=0x7f246c008040 comp rx=0 tx=0).stop 2023-09-23T03:19:20.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.420+0000 7f2476078640 1 -- 172.21.15.154:0/4073644833 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f247006fdc0 msgr2=0x7f24700c6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:20.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.420+0000 7f2476078640 1 --2- 172.21.15.154:0/4073644833 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f247006fdc0 0x7f24700c6db0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f2460000c00 tx=0x7f24600046c0 comp rx=0 tx=0).stop 2023-09-23T03:19:20.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.420+0000 7f2476078640 1 -- 172.21.15.154:0/4073644833 shutdown_connections 2023-09-23T03:19:20.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.420+0000 7f2476078640 1 --2- 172.21.15.154:0/4073644833 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f245406af60 0x7f245406d420 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:20.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.420+0000 7f2476078640 1 --2- 172.21.15.154:0/4073644833 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f247015ab00 0x7f24700c1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:20.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.420+0000 7f2476078640 1 --2- 172.21.15.154:0/4073644833 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f247006fdc0 0x7f24700c6db0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:20.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.420+0000 7f2476078640 1 --2- 172.21.15.154:0/4073644833 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f247006c7a0 0x7f2470152730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:20.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.420+0000 7f2476078640 1 -- 172.21.15.154:0/4073644833 >> 172.21.15.154:0/4073644833 conn(0x7f247006a390 msgr2=0x7f24700c9600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:20.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.420+0000 7f2476078640 1 -- 172.21.15.154:0/4073644833 shutdown_connections 2023-09-23T03:19:20.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.420+0000 7f2476078640 1 -- 172.21.15.154:0/4073644833 wait complete. 2023-09-23T03:19:20.438 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T03:19:20.438 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:19:20.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.657+0000 7f2666e6f640 1 Processor -- start 2023-09-23T03:19:20.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.657+0000 7f2666e6f640 1 -- start start 2023-09-23T03:19:20.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.659+0000 7f2666e6f640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f26600ad830 0x7f2660070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:20.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.659+0000 7f2666e6f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f26600c9680 0x7f26600caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:20.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.659+0000 7f2666e6f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26600cb400 0x7f2660160720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:20.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.659+0000 7f2666e6f640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26600cb890 con 0x7f26600c9680 2023-09-23T03:19:20.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.659+0000 7f2666e6f640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26600712f0 con 0x7f26600ad830 2023-09-23T03:19:20.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.659+0000 7f2666e6f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26600715d0 con 0x7f26600cb400 2023-09-23T03:19:20.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.659+0000 7f265ffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f26600c9680 0x7f26600caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:20.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.659+0000 7f265ffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f26600c9680 0x7f26600caec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56492/0 (socket says 172.21.15.154:56492) 2023-09-23T03:19:20.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.659+0000 7f265ffff640 1 -- 172.21.15.154:0/2665535215 learned_addr learned my addr 172.21.15.154:0/2665535215 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:20.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.660+0000 7f265ffff640 1 -- 172.21.15.154:0/2665535215 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26600cb400 msgr2=0x7f2660160720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:20.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.660+0000 7f265ffff640 1 --2- 172.21.15.154:0/2665535215 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26600cb400 0x7f2660160720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:20.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.660+0000 7f265ffff640 1 -- 172.21.15.154:0/2665535215 >> v2:172.21.15.190:3300/0 conn(0x7f26600ad830 msgr2=0x7f2660070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:20.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.660+0000 7f265ffff640 1 --2- 172.21.15.154:0/2665535215 >> v2:172.21.15.190:3300/0 conn(0x7f26600ad830 0x7f2660070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:20.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.660+0000 7f265ffff640 1 -- 172.21.15.154:0/2665535215 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26600718b0 con 0x7f26600c9680 2023-09-23T03:19:20.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.661+0000 7f265ffff640 1 --2- 172.21.15.154:0/2665535215 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26600c9680 0x7f26600caec0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f266006d9c0 tx=0x7f264c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25bfc4100ac386a5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:20.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.661+0000 7f265f7fe640 1 -- 172.21.15.154:0/2665535215 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264c031860 con 0x7f26600c9680 2023-09-23T03:19:20.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.661+0000 7f265f7fe640 1 -- 172.21.15.154:0/2665535215 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f264c0319c0 con 0x7f26600c9680 2023-09-23T03:19:20.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.661+0000 7f265f7fe640 1 -- 172.21.15.154:0/2665535215 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264c031d50 con 0x7f26600c9680 2023-09-23T03:19:20.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.662+0000 7f2666e6f640 1 -- 172.21.15.154:0/2665535215 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26600c9680 msgr2=0x7f26600caec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:20.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.662+0000 7f2666e6f640 1 --2- 172.21.15.154:0/2665535215 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26600c9680 0x7f26600caec0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f266006d9c0 tx=0x7f264c02b900 comp rx=0 tx=0).stop 2023-09-23T03:19:20.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.662+0000 7f2666e6f640 1 -- 172.21.15.154:0/2665535215 shutdown_connections 2023-09-23T03:19:20.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.662+0000 7f2666e6f640 1 --2- 172.21.15.154:0/2665535215 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26600cb400 0x7f2660160720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:20.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.662+0000 7f2666e6f640 1 --2- 172.21.15.154:0/2665535215 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26600c9680 0x7f26600caec0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:20.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.662+0000 7f2666e6f640 1 --2- 172.21.15.154:0/2665535215 >> v2:172.21.15.190:3300/0 conn(0x7f26600ad830 0x7f2660070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:20.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.662+0000 7f2666e6f640 1 -- 172.21.15.154:0/2665535215 >> 172.21.15.154:0/2665535215 conn(0x7f266006c010 msgr2=0x7f266006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:20.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.662+0000 7f2666e6f640 1 -- 172.21.15.154:0/2665535215 shutdown_connections 2023-09-23T03:19:20.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.662+0000 7f2666e6f640 1 -- 172.21.15.154:0/2665535215 wait complete. 2023-09-23T03:19:20.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.663+0000 7f2666e6f640 1 Processor -- start 2023-09-23T03:19:20.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.664+0000 7f2666e6f640 1 -- start start 2023-09-23T03:19:20.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.664+0000 7f2666e6f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26600ad830 0x7f26600c5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:20.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.664+0000 7f2666e6f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26600c9680 0x7f2660154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:20.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.664+0000 7f2666e6f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26600cb400 0x7f26600c2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:20.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.665+0000 7f2666e6f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26600c34d0 con 0x7f26600cb400 2023-09-23T03:19:20.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.665+0000 7f2666e6f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26600b2570 con 0x7f26600c9680 2023-09-23T03:19:20.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.665+0000 7f2666e6f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26600c31f0 con 0x7f26600ad830 2023-09-23T03:19:20.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.665+0000 7f266540c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26600cb400 0x7f26600c2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:20.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.665+0000 7f266540c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26600cb400 0x7f26600c2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56502/0 (socket says 172.21.15.154:56502) 2023-09-23T03:19:20.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.665+0000 7f266540c640 1 -- 172.21.15.154:0/3094677528 learned_addr learned my addr 172.21.15.154:0/3094677528 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:20.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.665+0000 7f265ffff640 1 --2- 172.21.15.154:0/3094677528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26600c9680 0x7f2660154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:20.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.665+0000 7f2664c0b640 1 --2- 172.21.15.154:0/3094677528 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26600ad830 0x7f26600c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:20.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.665+0000 7f266540c640 1 -- 172.21.15.154:0/3094677528 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26600ad830 msgr2=0x7f26600c5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:20.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.665+0000 7f266540c640 1 --2- 172.21.15.154:0/3094677528 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26600ad830 0x7f26600c5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:20.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.665+0000 7f266540c640 1 -- 172.21.15.154:0/3094677528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26600c9680 msgr2=0x7f2660154bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:20.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.665+0000 7f266540c640 1 --2- 172.21.15.154:0/3094677528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26600c9680 0x7f2660154bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:20.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.665+0000 7f266540c640 1 -- 172.21.15.154:0/3094677528 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f264c0094d0 con 0x7f26600cb400 2023-09-23T03:19:20.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.665+0000 7f265ffff640 1 --2- 172.21.15.154:0/3094677528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26600c9680 0x7f2660154bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:20.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.666+0000 7f266540c640 1 --2- 172.21.15.154:0/3094677528 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26600cb400 0x7f26600c2810 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f265800b970 tx=0x7f265800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:20.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.666+0000 7f265d7fa640 1 -- 172.21.15.154:0/3094677528 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265800cb40 con 0x7f26600cb400 2023-09-23T03:19:20.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.666+0000 7f265d7fa640 1 -- 172.21.15.154:0/3094677528 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f265800ccc0 con 0x7f26600cb400 2023-09-23T03:19:20.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.666+0000 7f2666e6f640 1 -- 172.21.15.154:0/3094677528 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26600b27c0 con 0x7f26600cb400 2023-09-23T03:19:20.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.666+0000 7f265d7fa640 1 -- 172.21.15.154:0/3094677528 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2658014680 con 0x7f26600cb400 2023-09-23T03:19:20.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.666+0000 7f2666e6f640 1 -- 172.21.15.154:0/3094677528 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26600b2d80 con 0x7f26600cb400 2023-09-23T03:19:20.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.667+0000 7f2666e6f640 1 -- 172.21.15.154:0/3094677528 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f266004e920 con 0x7f26600cb400 2023-09-23T03:19:20.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.668+0000 7f265d7fa640 1 -- 172.21.15.154:0/3094677528 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f265801d3b0 con 0x7f26600cb400 2023-09-23T03:19:20.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.669+0000 7f265d7fa640 1 --2- 172.21.15.154:0/3094677528 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f264406af30 0x7f264406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:20.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.669+0000 7f265d7fa640 1 -- 172.21.15.154:0/3094677528 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f2658089560 con 0x7f26600cb400 2023-09-23T03:19:20.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.669+0000 7f2664c0b640 1 --2- 172.21.15.154:0/3094677528 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f264406af30 0x7f264406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:20.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.670+0000 7f2664c0b640 1 --2- 172.21.15.154:0/3094677528 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f264406af30 0x7f264406d3f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f265400a770 tx=0x7f2654008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:20.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.673+0000 7f265d7fa640 1 -- 172.21.15.154:0/3094677528 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f2658054710 con 0x7f26600cb400 2023-09-23T03:19:20.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.845+0000 7f2666e6f640 1 -- 172.21.15.154:0/3094677528 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f26600b2f70 con 0x7f26600cb400 2023-09-23T03:19:20.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.846+0000 7f265d7fa640 1 -- 172.21.15.154:0/3094677528 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7f26580190a0 con 0x7f26600cb400 2023-09-23T03:19:20.847 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:19:20.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.849+0000 7f2666e6f640 1 -- 172.21.15.154:0/3094677528 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f264406af30 msgr2=0x7f264406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:20.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.849+0000 7f2666e6f640 1 --2- 172.21.15.154:0/3094677528 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f264406af30 0x7f264406d3f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f265400a770 tx=0x7f2654008040 comp rx=0 tx=0).stop 2023-09-23T03:19:20.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.849+0000 7f2666e6f640 1 -- 172.21.15.154:0/3094677528 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26600cb400 msgr2=0x7f26600c2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:20.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.849+0000 7f2666e6f640 1 --2- 172.21.15.154:0/3094677528 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26600cb400 0x7f26600c2810 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f265800b970 tx=0x7f265800be30 comp rx=0 tx=0).stop 2023-09-23T03:19:20.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.849+0000 7f2666e6f640 1 -- 172.21.15.154:0/3094677528 shutdown_connections 2023-09-23T03:19:20.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.849+0000 7f2666e6f640 1 --2- 172.21.15.154:0/3094677528 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f264406af30 0x7f264406d3f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:20.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.849+0000 7f2666e6f640 1 --2- 172.21.15.154:0/3094677528 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26600cb400 0x7f26600c2810 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:20.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.849+0000 7f2666e6f640 1 --2- 172.21.15.154:0/3094677528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f26600c9680 0x7f2660154bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:20.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.849+0000 7f2666e6f640 1 --2- 172.21.15.154:0/3094677528 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f26600ad830 0x7f26600c5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:20.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.849+0000 7f2666e6f640 1 -- 172.21.15.154:0/3094677528 >> 172.21.15.154:0/3094677528 conn(0x7f266006c010 msgr2=0x7f26600af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:20.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.849+0000 7f2666e6f640 1 -- 172.21.15.154:0/3094677528 shutdown_connections 2023-09-23T03:19:20.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:20.849+0000 7f2666e6f640 1 -- 172.21.15.154:0/3094677528 wait complete. 2023-09-23T03:19:20.867 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-09-23T03:19:21.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.077+0000 7f7170656640 1 Processor -- start 2023-09-23T03:19:21.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.077+0000 7f7170656640 1 -- start start 2023-09-23T03:19:21.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.078+0000 7f7170656640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71680acce0 0x7f716815ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:21.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.078+0000 7f7170656640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f71680b1b20 0x7f71680ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:21.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.079+0000 7f7170656640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f716815ac30 con 0x7f71680cee70 2023-09-23T03:19:21.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.079+0000 7f7170656640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71680b2e90 con 0x7f71680b1b20 2023-09-23T03:19:21.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.079+0000 7f7170656640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71680cf320 con 0x7f71680acce0 2023-09-23T03:19:21.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.079+0000 7f716e3f2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71680acce0 0x7f716815ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:21.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.079+0000 7f716e3f2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71680acce0 0x7f716815ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:47966/0 (socket says 172.21.15.154:47966) 2023-09-23T03:19:21.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.079+0000 7f716e3f2640 1 -- 172.21.15.154:0/2805103848 learned_addr learned my addr 172.21.15.154:0/2805103848 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:21.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.079+0000 7f716dbf1640 1 --2- 172.21.15.154:0/2805103848 >> v2:172.21.15.190:3300/0 conn(0x7f71680b1b20 0x7f71680ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:21.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.080+0000 7f716e3f2640 1 -- 172.21.15.154:0/2805103848 >> v2:172.21.15.190:3300/0 conn(0x7f71680b1b20 msgr2=0x7f71680ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:21.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.080+0000 7f716e3f2640 1 --2- 172.21.15.154:0/2805103848 >> v2:172.21.15.190:3300/0 conn(0x7f71680b1b20 0x7f71680ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:21.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.080+0000 7f716e3f2640 1 -- 172.21.15.154:0/2805103848 >> v1:172.21.15.190:6789/0 conn(0x7f71680cee70 legacy=0x7f716806cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:21.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.080+0000 7f716e3f2640 1 -- 172.21.15.154:0/2805103848 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71680b1f70 con 0x7f71680acce0 2023-09-23T03:19:21.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.080+0000 7f716dbf1640 1 --2- 172.21.15.154:0/2805103848 >> v2:172.21.15.190:3300/0 conn(0x7f71680b1b20 0x7f71680ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:19:21.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.082+0000 7f716e3f2640 1 --2- 172.21.15.154:0/2805103848 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71680acce0 0x7f716815ed60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7164009850 tx=0x7f716402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8eb5069fe2ba282d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:21.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.082+0000 7f716d3f0640 1 -- 172.21.15.154:0/2805103848 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7164004230 con 0x7f71680acce0 2023-09-23T03:19:21.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.082+0000 7f716d3f0640 1 -- 172.21.15.154:0/2805103848 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7164004390 con 0x7f71680acce0 2023-09-23T03:19:21.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.082+0000 7f716d3f0640 1 -- 172.21.15.154:0/2805103848 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7164031830 con 0x7f71680acce0 2023-09-23T03:19:21.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.083+0000 7f7170656640 1 -- 172.21.15.154:0/2805103848 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71680acce0 msgr2=0x7f716815ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:21.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.083+0000 7f7170656640 1 --2- 172.21.15.154:0/2805103848 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71680acce0 0x7f716815ed60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7164009850 tx=0x7f716402b8d0 comp rx=0 tx=0).stop 2023-09-23T03:19:21.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.083+0000 7f7170656640 1 -- 172.21.15.154:0/2805103848 shutdown_connections 2023-09-23T03:19:21.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.083+0000 7f7170656640 1 --2- 172.21.15.154:0/2805103848 >> v2:172.21.15.190:3300/0 conn(0x7f71680b1b20 0x7f71680ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:21.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.083+0000 7f7170656640 1 --2- 172.21.15.154:0/2805103848 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71680acce0 0x7f716815ed60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:21.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.083+0000 7f7170656640 1 -- 172.21.15.154:0/2805103848 >> 172.21.15.154:0/2805103848 conn(0x7f716806a0f0 msgr2=0x7f716806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:21.084 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.083+0000 7f7170656640 1 -- 172.21.15.154:0/2805103848 shutdown_connections 2023-09-23T03:19:21.084 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.083+0000 7f7170656640 1 -- 172.21.15.154:0/2805103848 wait complete. 2023-09-23T03:19:21.084 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.084+0000 7f7170656640 1 Processor -- start 2023-09-23T03:19:21.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.085+0000 7f7170656640 1 -- start start 2023-09-23T03:19:21.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.085+0000 7f7170656640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71680acce0 0x7f71681f20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:21.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.086+0000 7f7170656640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71680b1b20 0x7f716815add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:21.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.086+0000 7f7170656640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71680cee70 0x7f71680c4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:21.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.086+0000 7f7170656640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7168072f20 con 0x7f71680cee70 2023-09-23T03:19:21.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.086+0000 7f7170656640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716815b310 con 0x7f71680acce0 2023-09-23T03:19:21.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.086+0000 7f7170656640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f716815c150 con 0x7f71680b1b20 2023-09-23T03:19:21.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.086+0000 7f716ebf3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71680cee70 0x7f71680c4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:21.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.086+0000 7f716ebf3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71680cee70 0x7f71680c4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56506/0 (socket says 172.21.15.154:56506) 2023-09-23T03:19:21.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.086+0000 7f716ebf3640 1 -- 172.21.15.154:0/2797128149 learned_addr learned my addr 172.21.15.154:0/2797128149 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:21.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.086+0000 7f716e3f2640 1 --2- 172.21.15.154:0/2797128149 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71680acce0 0x7f71681f20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:21.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.086+0000 7f716dbf1640 1 --2- 172.21.15.154:0/2797128149 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71680b1b20 0x7f716815add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:21.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.086+0000 7f716ebf3640 1 -- 172.21.15.154:0/2797128149 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71680b1b20 msgr2=0x7f716815add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:21.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.086+0000 7f716ebf3640 1 --2- 172.21.15.154:0/2797128149 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71680b1b20 0x7f716815add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:21.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.086+0000 7f716ebf3640 1 -- 172.21.15.154:0/2797128149 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71680acce0 msgr2=0x7f71681f20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:21.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.086+0000 7f716ebf3640 1 --2- 172.21.15.154:0/2797128149 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71680acce0 0x7f71681f20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:21.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.086+0000 7f716ebf3640 1 -- 172.21.15.154:0/2797128149 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7158009400 con 0x7f71680cee70 2023-09-23T03:19:21.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.087+0000 7f716ebf3640 1 --2- 172.21.15.154:0/2797128149 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71680cee70 0x7f71680c4740 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f716000c6e0 tx=0x7f716000cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:21.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.087+0000 7f715effd640 1 -- 172.21.15.154:0/2797128149 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7160003b70 con 0x7f71680cee70 2023-09-23T03:19:21.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.087+0000 7f715effd640 1 -- 172.21.15.154:0/2797128149 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7160003cf0 con 0x7f71680cee70 2023-09-23T03:19:21.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.087+0000 7f715effd640 1 -- 172.21.15.154:0/2797128149 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7160005310 con 0x7f71680cee70 2023-09-23T03:19:21.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.087+0000 7f7170656640 1 -- 172.21.15.154:0/2797128149 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7164009460 con 0x7f71680cee70 2023-09-23T03:19:21.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.087+0000 7f7170656640 1 -- 172.21.15.154:0/2797128149 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f716815b820 con 0x7f71680cee70 2023-09-23T03:19:21.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.088+0000 7f7170656640 1 -- 172.21.15.154:0/2797128149 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7168073eb0 con 0x7f71680cee70 2023-09-23T03:19:21.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.089+0000 7f715effd640 1 -- 172.21.15.154:0/2797128149 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f7160005470 con 0x7f71680cee70 2023-09-23T03:19:21.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.090+0000 7f715effd640 1 --2- 172.21.15.154:0/2797128149 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f714c06af30 0x7f714c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:21.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.090+0000 7f715effd640 1 -- 172.21.15.154:0/2797128149 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f71600891d0 con 0x7f71680cee70 2023-09-23T03:19:21.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.090+0000 7f716e3f2640 1 --2- 172.21.15.154:0/2797128149 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f714c06af30 0x7f714c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:21.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.091+0000 7f716e3f2640 1 --2- 172.21.15.154:0/2797128149 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f714c06af30 0x7f714c06d3f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7164035000 tx=0x7f7164008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:21.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.095+0000 7f715effd640 1 -- 172.21.15.154:0/2797128149 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f7160054380 con 0x7f71680cee70 2023-09-23T03:19:21.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.272+0000 7f7170656640 1 -- 172.21.15.154:0/2797128149 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f716815ac30 con 0x7f71680cee70 2023-09-23T03:19:21.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.273+0000 7f715effd640 1 -- 172.21.15.154:0/2797128149 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v58) v1 ==== 84+0+65 (secure 0 0 0) 0x7f716815ac30 con 0x7f71680cee70 2023-09-23T03:19:21.273 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:19:21.273 INFO:teuthology.orchestra.run.smithi154.stdout:[ 2023-09-23T03:19:21.273 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:19:21.273 INFO:teuthology.orchestra.run.smithi154.stdout: "poolnum": 1, 2023-09-23T03:19:21.273 INFO:teuthology.orchestra.run.smithi154.stdout: "poolname": ".mgr" 2023-09-23T03:19:21.273 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:19:21.273 INFO:teuthology.orchestra.run.smithi154.stdout:] 2023-09-23T03:19:21.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.275+0000 7f7170656640 1 -- 172.21.15.154:0/2797128149 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f714c06af30 msgr2=0x7f714c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:21.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.275+0000 7f7170656640 1 --2- 172.21.15.154:0/2797128149 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f714c06af30 0x7f714c06d3f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7164035000 tx=0x7f7164008030 comp rx=0 tx=0).stop 2023-09-23T03:19:21.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.275+0000 7f7170656640 1 -- 172.21.15.154:0/2797128149 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71680cee70 msgr2=0x7f71680c4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:21.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.275+0000 7f7170656640 1 --2- 172.21.15.154:0/2797128149 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71680cee70 0x7f71680c4740 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f716000c6e0 tx=0x7f716000cba0 comp rx=0 tx=0).stop 2023-09-23T03:19:21.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.276+0000 7f7170656640 1 -- 172.21.15.154:0/2797128149 shutdown_connections 2023-09-23T03:19:21.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.276+0000 7f7170656640 1 --2- 172.21.15.154:0/2797128149 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f714c06af30 0x7f714c06d3f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:21.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.276+0000 7f7170656640 1 --2- 172.21.15.154:0/2797128149 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71680cee70 0x7f71680c4740 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:21.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.276+0000 7f7170656640 1 --2- 172.21.15.154:0/2797128149 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71680b1b20 0x7f716815add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:21.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.276+0000 7f7170656640 1 --2- 172.21.15.154:0/2797128149 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71680acce0 0x7f71681f20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:21.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.276+0000 7f7170656640 1 -- 172.21.15.154:0/2797128149 >> 172.21.15.154:0/2797128149 conn(0x7f716806a0f0 msgr2=0x7f71680adf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:21.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.276+0000 7f7170656640 1 -- 172.21.15.154:0/2797128149 shutdown_connections 2023-09-23T03:19:21.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.276+0000 7f7170656640 1 -- 172.21.15.154:0/2797128149 wait complete. 2023-09-23T03:19:21.294 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-09-23T03:19:21.294 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-09-23T03:19:21.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.512+0000 7f798ef16640 1 Processor -- start 2023-09-23T03:19:21.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.513+0000 7f798ef16640 1 -- start start 2023-09-23T03:19:21.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.514+0000 7f798ef16640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f79881572c0 0x7f7988159e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:21.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.514+0000 7f798ef16640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79880ce3c0 0x7f79880ce830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:21.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.514+0000 7f798ef16640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79880721d0 con 0x7f79881576a0 2023-09-23T03:19:21.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.514+0000 7f798ef16640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79880ced70 con 0x7f79881572c0 2023-09-23T03:19:21.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.514+0000 7f798ef16640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f798815a3c0 con 0x7f79880ce3c0 2023-09-23T03:19:21.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.514+0000 7f798ccb2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f79881572c0 0x7f7988159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:21.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.514+0000 7f798ccb2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f79881572c0 0x7f7988159e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56520/0 (socket says 172.21.15.154:56520) 2023-09-23T03:19:21.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.514+0000 7f798ccb2640 1 -- 172.21.15.154:0/520423176 learned_addr learned my addr 172.21.15.154:0/520423176 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:21.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.515+0000 7f798ccb2640 1 -- 172.21.15.154:0/520423176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79880ce3c0 msgr2=0x7f79880ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:21.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.515+0000 7f798ccb2640 1 --2- 172.21.15.154:0/520423176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79880ce3c0 0x7f79880ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:21.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.515+0000 7f798ccb2640 1 -- 172.21.15.154:0/520423176 >> v1:172.21.15.190:6789/0 conn(0x7f79881576a0 legacy=0x7f79880b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:21.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.515+0000 7f798ccb2640 1 -- 172.21.15.154:0/520423176 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7988071650 con 0x7f79881572c0 2023-09-23T03:19:21.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.516+0000 7f798ccb2640 1 --2- 172.21.15.154:0/520423176 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79881572c0 0x7f7988159e80 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f798806cdd0 tx=0x7f797c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf28921f24837ef1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:21.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.516+0000 7f79877fe640 1 -- 172.21.15.154:0/520423176 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797c0317c0 con 0x7f79881572c0 2023-09-23T03:19:21.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.516+0000 7f79877fe640 1 -- 172.21.15.154:0/520423176 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797c031920 con 0x7f79881572c0 2023-09-23T03:19:21.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.516+0000 7f79877fe640 1 -- 172.21.15.154:0/520423176 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797c031cb0 con 0x7f79881572c0 2023-09-23T03:19:21.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.517+0000 7f798ef16640 1 -- 172.21.15.154:0/520423176 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79881572c0 msgr2=0x7f7988159e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:21.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.517+0000 7f798ef16640 1 --2- 172.21.15.154:0/520423176 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79881572c0 0x7f7988159e80 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f798806cdd0 tx=0x7f797c02b860 comp rx=0 tx=0).stop 2023-09-23T03:19:21.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.517+0000 7f798ef16640 1 -- 172.21.15.154:0/520423176 shutdown_connections 2023-09-23T03:19:21.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.517+0000 7f798ef16640 1 --2- 172.21.15.154:0/520423176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79880ce3c0 0x7f79880ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:21.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.517+0000 7f798ef16640 1 --2- 172.21.15.154:0/520423176 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79881572c0 0x7f7988159e80 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:21.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.517+0000 7f798ef16640 1 -- 172.21.15.154:0/520423176 >> 172.21.15.154:0/520423176 conn(0x7f798806a520 msgr2=0x7f798806bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:21.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.517+0000 7f798ef16640 1 -- 172.21.15.154:0/520423176 shutdown_connections 2023-09-23T03:19:21.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.517+0000 7f798ef16640 1 -- 172.21.15.154:0/520423176 wait complete. 2023-09-23T03:19:21.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.519+0000 7f798ef16640 1 Processor -- start 2023-09-23T03:19:21.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.519+0000 7f798ef16640 1 -- start start 2023-09-23T03:19:21.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.520+0000 7f798ef16640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79880ce3c0 0x7f798815ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:21.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.520+0000 7f798ef16640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79881572c0 0x7f7988156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:21.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.520+0000 7f798ef16640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79881576a0 0x7f79881f5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:21.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.520+0000 7f798ef16640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79880c4e70 con 0x7f79880ce3c0 2023-09-23T03:19:21.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.520+0000 7f798ef16640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79880c4090 con 0x7f79881576a0 2023-09-23T03:19:21.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.520+0000 7f798ef16640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79880c4310 con 0x7f79881572c0 2023-09-23T03:19:21.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.520+0000 7f798ccb2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79880ce3c0 0x7f798815ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:21.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.520+0000 7f798ccb2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79880ce3c0 0x7f798815ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56522/0 (socket says 172.21.15.154:56522) 2023-09-23T03:19:21.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.520+0000 7f798ccb2640 1 -- 172.21.15.154:0/1768747350 learned_addr learned my addr 172.21.15.154:0/1768747350 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:21.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.520+0000 7f798ccb2640 1 -- 172.21.15.154:0/1768747350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79881572c0 msgr2=0x7f7988156660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:21.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.520+0000 7f798ccb2640 1 --2- 172.21.15.154:0/1768747350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79881572c0 0x7f7988156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:21.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.520+0000 7f798d4b3640 1 --2- 172.21.15.154:0/1768747350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79881576a0 0x7f79881f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:21.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.520+0000 7f798ccb2640 1 -- 172.21.15.154:0/1768747350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79881576a0 msgr2=0x7f79881f5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:21.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.520+0000 7f798ccb2640 1 --2- 172.21.15.154:0/1768747350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79881576a0 0x7f79881f5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:21.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.520+0000 7f798ccb2640 1 -- 172.21.15.154:0/1768747350 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f797c0094d0 con 0x7f79880ce3c0 2023-09-23T03:19:21.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.521+0000 7f798ccb2640 1 --2- 172.21.15.154:0/1768747350 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79880ce3c0 0x7f798815ba50 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f797c02b8a0 tx=0x7f797c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:21.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.521+0000 7f79857fa640 1 -- 172.21.15.154:0/1768747350 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797c032920 con 0x7f79880ce3c0 2023-09-23T03:19:21.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.521+0000 7f79857fa640 1 -- 172.21.15.154:0/1768747350 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797c032aa0 con 0x7f79880ce3c0 2023-09-23T03:19:21.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.521+0000 7f79857fa640 1 -- 172.21.15.154:0/1768747350 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797c032e50 con 0x7f79880ce3c0 2023-09-23T03:19:21.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.521+0000 7f798ef16640 1 -- 172.21.15.154:0/1768747350 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79880c45d0 con 0x7f79880ce3c0 2023-09-23T03:19:21.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.521+0000 7f798ef16640 1 -- 172.21.15.154:0/1768747350 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79880c4b90 con 0x7f79880ce3c0 2023-09-23T03:19:21.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.522+0000 7f798ef16640 1 -- 172.21.15.154:0/1768747350 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f798804e920 con 0x7f79880ce3c0 2023-09-23T03:19:21.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.523+0000 7f79857fa640 1 -- 172.21.15.154:0/1768747350 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f797c04e450 con 0x7f79880ce3c0 2023-09-23T03:19:21.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.524+0000 7f79857fa640 1 --2- 172.21.15.154:0/1768747350 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f796c06af30 0x7f796c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:21.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.525+0000 7f79857fa640 1 -- 172.21.15.154:0/1768747350 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f797c0ab860 con 0x7f79880ce3c0 2023-09-23T03:19:21.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.525+0000 7f7987fff640 1 --2- 172.21.15.154:0/1768747350 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f796c06af30 0x7f796c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:21.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.525+0000 7f7987fff640 1 --2- 172.21.15.154:0/1768747350 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f796c06af30 0x7f796c06d3f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f79880ada90 tx=0x7f7974007990 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:21.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.529+0000 7f79857fa640 1 -- 172.21.15.154:0/1768747350 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f797c076a90 con 0x7f79880ce3c0 2023-09-23T03:19:21.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:21.718+0000 7f798ef16640 1 -- 172.21.15.154:0/1768747350 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f79880cd340 con 0x7f79880ce3c0 2023-09-23T03:19:22.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.464+0000 7f79857fa640 1 -- 172.21.15.154:0/1768747350 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v59) v1 ==== 138+0+0 (secure 0 0 0) 0x7f797c07a0b0 con 0x7f79880ce3c0 2023-09-23T03:19:22.465 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_metadata' created 2023-09-23T03:19:22.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.468+0000 7f798ef16640 1 -- 172.21.15.154:0/1768747350 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f796c06af30 msgr2=0x7f796c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:22.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.468+0000 7f798ef16640 1 --2- 172.21.15.154:0/1768747350 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f796c06af30 0x7f796c06d3f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f79880ada90 tx=0x7f7974007990 comp rx=0 tx=0).stop 2023-09-23T03:19:22.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.468+0000 7f798ef16640 1 -- 172.21.15.154:0/1768747350 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79880ce3c0 msgr2=0x7f798815ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:22.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.468+0000 7f798ef16640 1 --2- 172.21.15.154:0/1768747350 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79880ce3c0 0x7f798815ba50 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f797c02b8a0 tx=0x7f797c0046c0 comp rx=0 tx=0).stop 2023-09-23T03:19:22.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.468+0000 7f798ef16640 1 -- 172.21.15.154:0/1768747350 shutdown_connections 2023-09-23T03:19:22.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.468+0000 7f798ef16640 1 --2- 172.21.15.154:0/1768747350 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f796c06af30 0x7f796c06d3f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:22.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.468+0000 7f798ef16640 1 --2- 172.21.15.154:0/1768747350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f79881576a0 0x7f79881f5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:22.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.468+0000 7f798ef16640 1 --2- 172.21.15.154:0/1768747350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f79881572c0 0x7f7988156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:22.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.468+0000 7f798ef16640 1 --2- 172.21.15.154:0/1768747350 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79880ce3c0 0x7f798815ba50 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:22.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.468+0000 7f798ef16640 1 -- 172.21.15.154:0/1768747350 >> 172.21.15.154:0/1768747350 conn(0x7f798806a520 msgr2=0x7f798806bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:22.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.468+0000 7f798ef16640 1 -- 172.21.15.154:0/1768747350 shutdown_connections 2023-09-23T03:19:22.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.468+0000 7f798ef16640 1 -- 172.21.15.154:0/1768747350 wait complete. 2023-09-23T03:19:22.495 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-09-23T03:19:22.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.699+0000 7f32beeda640 1 Processor -- start 2023-09-23T03:19:22.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.699+0000 7f32beeda640 1 -- start start 2023-09-23T03:19:22.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.700+0000 7f32beeda640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f32b80c8d70 0x7f32b815c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:22.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.701+0000 7f32beeda640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32b80c9150 0x7f32b815cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:22.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.701+0000 7f32beeda640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b80c9d20 con 0x7f32b8071290 2023-09-23T03:19:22.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.701+0000 7f32beeda640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32b8071850 con 0x7f32b80c8d70 2023-09-23T03:19:22.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.701+0000 7f32beeda640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b8071b30 con 0x7f32b80c9150 2023-09-23T03:19:22.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.701+0000 7f32bcc76640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f32b80c8d70 0x7f32b815c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:22.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.701+0000 7f32bd477640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f32b8071290 0x7f32b80ca1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40074/0 (socket says 172.21.15.154:40074) 2023-09-23T03:19:22.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.701+0000 7f32bd477640 1 -- 172.21.15.154:0/2382724917 learned_addr learned my addr 172.21.15.154:0/2382724917 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:19:22.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.701+0000 7f32b7fff640 1 --2- 172.21.15.154:0/2382724917 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32b80c9150 0x7f32b815cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:22.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.701+0000 7f32bcc76640 1 -- 172.21.15.154:0/2382724917 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32b80c9150 msgr2=0x7f32b815cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:22.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.701+0000 7f32bcc76640 1 --2- 172.21.15.154:0/2382724917 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32b80c9150 0x7f32b815cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:22.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.701+0000 7f32bcc76640 1 -- 172.21.15.154:0/2382724917 >> v1:172.21.15.154:6789/0 conn(0x7f32b8071290 legacy=0x7f32b80ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:22.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.702+0000 7f32bcc76640 1 -- 172.21.15.154:0/2382724917 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b8071e10 con 0x7f32b80c8d70 2023-09-23T03:19:22.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.703+0000 7f32bcc76640 1 --2- 172.21.15.154:0/2382724917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32b80c8d70 0x7f32b815c6d0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f32ac0097a0 tx=0x7f32ac02b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fdb4efa4ed16304 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:22.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.703+0000 7f32b77fe640 1 -- 172.21.15.154:0/2382724917 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac030710 con 0x7f32b80c8d70 2023-09-23T03:19:22.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.703+0000 7f32b77fe640 1 -- 172.21.15.154:0/2382724917 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ac02e050 con 0x7f32b80c8d70 2023-09-23T03:19:22.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.703+0000 7f32b77fe640 1 -- 172.21.15.154:0/2382724917 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac030a90 con 0x7f32b80c8d70 2023-09-23T03:19:22.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.703+0000 7f32beeda640 1 -- 172.21.15.154:0/2382724917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32b80c8d70 msgr2=0x7f32b815c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:22.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.703+0000 7f32beeda640 1 --2- 172.21.15.154:0/2382724917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32b80c8d70 0x7f32b815c6d0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f32ac0097a0 tx=0x7f32ac02b660 comp rx=0 tx=0).stop 2023-09-23T03:19:22.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.704+0000 7f32beeda640 1 -- 172.21.15.154:0/2382724917 shutdown_connections 2023-09-23T03:19:22.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.704+0000 7f32beeda640 1 --2- 172.21.15.154:0/2382724917 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32b80c9150 0x7f32b815cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:22.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.704+0000 7f32beeda640 1 --2- 172.21.15.154:0/2382724917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32b80c8d70 0x7f32b815c6d0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:22.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.704+0000 7f32beeda640 1 -- 172.21.15.154:0/2382724917 >> 172.21.15.154:0/2382724917 conn(0x7f32b806c010 msgr2=0x7f32b806c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:22.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.704+0000 7f32beeda640 1 -- 172.21.15.154:0/2382724917 shutdown_connections 2023-09-23T03:19:22.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.704+0000 7f32beeda640 1 -- 172.21.15.154:0/2382724917 wait complete. 2023-09-23T03:19:22.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.705+0000 7f32beeda640 1 Processor -- start 2023-09-23T03:19:22.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.706+0000 7f32beeda640 1 -- start start 2023-09-23T03:19:22.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.707+0000 7f32beeda640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32b8071290 0x7f32b81efcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:22.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.707+0000 7f32beeda640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32b80c8d70 0x7f32b80c5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:22.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.707+0000 7f32beeda640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32b80c9150 0x7f32b80b2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:22.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.707+0000 7f32beeda640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b80c5e80 con 0x7f32b80c8d70 2023-09-23T03:19:22.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.707+0000 7f32beeda640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b80c6160 con 0x7f32b8071290 2023-09-23T03:19:22.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.707+0000 7f32beeda640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b80c6440 con 0x7f32b80c9150 2023-09-23T03:19:22.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.707+0000 7f32b7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32b80c8d70 0x7f32b80c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:22.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.707+0000 7f32b7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32b80c8d70 0x7f32b80c5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56540/0 (socket says 172.21.15.154:56540) 2023-09-23T03:19:22.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.707+0000 7f32b7fff640 1 -- 172.21.15.154:0/744375349 learned_addr learned my addr 172.21.15.154:0/744375349 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:22.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.707+0000 7f32bcc76640 1 --2- 172.21.15.154:0/744375349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32b8071290 0x7f32b81efcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:22.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.707+0000 7f32bd477640 1 --2- 172.21.15.154:0/744375349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32b80c9150 0x7f32b80b2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:22.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.707+0000 7f32b7fff640 1 -- 172.21.15.154:0/744375349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32b80c9150 msgr2=0x7f32b80b2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:22.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.707+0000 7f32b7fff640 1 --2- 172.21.15.154:0/744375349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32b80c9150 0x7f32b80b2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:22.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.707+0000 7f32b7fff640 1 -- 172.21.15.154:0/744375349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32b8071290 msgr2=0x7f32b81efcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:22.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.707+0000 7f32b7fff640 1 --2- 172.21.15.154:0/744375349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32b8071290 0x7f32b81efcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:22.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.707+0000 7f32b7fff640 1 -- 172.21.15.154:0/744375349 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ac009400 con 0x7f32b80c8d70 2023-09-23T03:19:22.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.707+0000 7f32bd477640 1 --2- 172.21.15.154:0/744375349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32b80c9150 0x7f32b80b2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:22.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.708+0000 7f32b7fff640 1 --2- 172.21.15.154:0/744375349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32b80c8d70 0x7f32b80c5580 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f32a400d800 tx=0x7f32a400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:22.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.708+0000 7f32b57fa640 1 -- 172.21.15.154:0/744375349 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a400b6c0 con 0x7f32b80c8d70 2023-09-23T03:19:22.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.708+0000 7f32b57fa640 1 -- 172.21.15.154:0/744375349 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a400b820 con 0x7f32b80c8d70 2023-09-23T03:19:22.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.708+0000 7f32b57fa640 1 -- 172.21.15.154:0/744375349 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a400bbb0 con 0x7f32b80c8d70 2023-09-23T03:19:22.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.708+0000 7f32beeda640 1 -- 172.21.15.154:0/744375349 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b80c6720 con 0x7f32b80c8d70 2023-09-23T03:19:22.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.708+0000 7f32beeda640 1 -- 172.21.15.154:0/744375349 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b80c5ac0 con 0x7f32b80c8d70 2023-09-23T03:19:22.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.709+0000 7f32beeda640 1 -- 172.21.15.154:0/744375349 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b804e920 con 0x7f32b80c8d70 2023-09-23T03:19:22.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.711+0000 7f32b57fa640 1 -- 172.21.15.154:0/744375349 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f32a400bd10 con 0x7f32b80c8d70 2023-09-23T03:19:22.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.711+0000 7f32b57fa640 1 --2- 172.21.15.154:0/744375349 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f329c06af30 0x7f329c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:22.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.712+0000 7f32b57fa640 1 -- 172.21.15.154:0/744375349 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f32a4089c90 con 0x7f32b80c8d70 2023-09-23T03:19:22.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.712+0000 7f32bcc76640 1 --2- 172.21.15.154:0/744375349 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f329c06af30 0x7f329c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:22.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.712+0000 7f32bcc76640 1 --2- 172.21.15.154:0/744375349 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f329c06af30 0x7f329c06d3f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f32ac034000 tx=0x7f32ac008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:22.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.716+0000 7f32b57fa640 1 -- 172.21.15.154:0/744375349 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f32a4054d60 con 0x7f32b80c8d70 2023-09-23T03:19:22.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:22.909+0000 7f32beeda640 1 -- 172.21.15.154:0/744375349 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f32b80720c0 con 0x7f32b80c8d70 2023-09-23T03:19:23.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.478+0000 7f32b57fa640 1 -- 172.21.15.154:0/744375349 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v60) v1 ==== 170+0+0 (secure 0 0 0) 0x7f32a4058380 con 0x7f32b80c8d70 2023-09-23T03:19:23.479 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_data' created 2023-09-23T03:19:23.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.483+0000 7f32beeda640 1 -- 172.21.15.154:0/744375349 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f329c06af30 msgr2=0x7f329c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:23.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.483+0000 7f32beeda640 1 --2- 172.21.15.154:0/744375349 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f329c06af30 0x7f329c06d3f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f32ac034000 tx=0x7f32ac008030 comp rx=0 tx=0).stop 2023-09-23T03:19:23.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.483+0000 7f32beeda640 1 -- 172.21.15.154:0/744375349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32b80c8d70 msgr2=0x7f32b80c5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:23.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.483+0000 7f32beeda640 1 --2- 172.21.15.154:0/744375349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32b80c8d70 0x7f32b80c5580 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f32a400d800 tx=0x7f32a400dcc0 comp rx=0 tx=0).stop 2023-09-23T03:19:23.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.484+0000 7f32beeda640 1 -- 172.21.15.154:0/744375349 shutdown_connections 2023-09-23T03:19:23.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.484+0000 7f32beeda640 1 --2- 172.21.15.154:0/744375349 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f329c06af30 0x7f329c06d3f0 secure :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f32ac034000 tx=0x7f32ac008030 comp rx=0 tx=0).stop 2023-09-23T03:19:23.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.484+0000 7f32beeda640 1 --2- 172.21.15.154:0/744375349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32b80c9150 0x7f32b80b2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:23.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.484+0000 7f32beeda640 1 --2- 172.21.15.154:0/744375349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32b80c8d70 0x7f32b80c5580 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:23.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.484+0000 7f32beeda640 1 --2- 172.21.15.154:0/744375349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32b8071290 0x7f32b81efcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:23.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.484+0000 7f32beeda640 1 -- 172.21.15.154:0/744375349 >> 172.21.15.154:0/744375349 conn(0x7f32b806c010 msgr2=0x7f32b815b420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:23.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.485+0000 7f32beeda640 1 -- 172.21.15.154:0/744375349 shutdown_connections 2023-09-23T03:19:23.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.485+0000 7f32beeda640 1 -- 172.21.15.154:0/744375349 wait complete. 2023-09-23T03:19:23.516 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-09-23T03:19:23.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.704+0000 7f10a2d78640 1 Processor -- start 2023-09-23T03:19:23.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.704+0000 7f10a2d78640 1 -- start start 2023-09-23T03:19:23.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.706+0000 7f10a2d78640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f109c0cf9e0 0x7f109c0d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:23.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.706+0000 7f10a2d78640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f109c0d1960 con 0x7f109c0d14b0 2023-09-23T03:19:23.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.706+0000 7f10a2d78640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f109c0cd7b0 con 0x7f109c160000 2023-09-23T03:19:23.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.706+0000 7f10a2d78640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f109c0cda90 con 0x7f109c0cf9e0 2023-09-23T03:19:23.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.706+0000 7f10a1315640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f109c0d14b0 0x7f109c0b2130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40088/0 (socket says 172.21.15.154:40088) 2023-09-23T03:19:23.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.706+0000 7f10a1315640 1 -- 172.21.15.154:0/998996775 learned_addr learned my addr 172.21.15.154:0/998996775 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:19:23.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.707+0000 7f109bfff640 1 --2- 172.21.15.154:0/998996775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f109c0cf9e0 0x7f109c0d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:23.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.707+0000 7f109b7fe640 1 -- 172.21.15.154:0/998996775 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 265771021 0 0) 0x7f109c0d1960 con 0x7f109c0d14b0 2023-09-23T03:19:23.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.707+0000 7f109b7fe640 1 -- 172.21.15.154:0/998996775 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f108c003520 con 0x7f109c0d14b0 2023-09-23T03:19:23.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.707+0000 7f109b7fe640 1 -- 172.21.15.154:0/998996775 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 164284819 0 0) 0x7f108c003520 con 0x7f109c0d14b0 2023-09-23T03:19:23.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.707+0000 7f109b7fe640 1 -- 172.21.15.154:0/998996775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f109c0cf9e0 msgr2=0x7f109c0d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:23.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.707+0000 7f109b7fe640 1 --2- 172.21.15.154:0/998996775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f109c0cf9e0 0x7f109c0d0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:23.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.707+0000 7f109b7fe640 1 -- 172.21.15.154:0/998996775 >> v1:172.21.15.190:6789/0 conn(0x7f109c160000 legacy=0x7f109c0cf2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:23.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.707+0000 7f109b7fe640 1 -- 172.21.15.154:0/998996775 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f109c0cdd70 con 0x7f109c0d14b0 2023-09-23T03:19:23.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.707+0000 7f109b7fe640 1 -- 172.21.15.154:0/998996775 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f10940019f0 con 0x7f109c0d14b0 2023-09-23T03:19:23.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.707+0000 7f109b7fe640 1 -- 172.21.15.154:0/998996775 >> v1:172.21.15.154:6789/0 conn(0x7f109c0d14b0 legacy=0x7f109c0b2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:23.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.707+0000 7f109b7fe640 1 --2- 172.21.15.154:0/998996775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f109c0cf9e0 0x7f108c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:23.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.708+0000 7f109b7fe640 1 --2- 172.21.15.154:0/998996775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f108c003f20 0x7f108c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:23.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.708+0000 7f109b7fe640 1 --2- 172.21.15.154:0/998996775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f108c004d00 0x7f108c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:23.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.708+0000 7f109b7fe640 1 -- 172.21.15.154:0/998996775 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c0cda90 con 0x7f108c004d00 2023-09-23T03:19:23.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.708+0000 7f109b7fe640 1 -- 172.21.15.154:0/998996775 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108c00ba60 con 0x7f108c003f20 2023-09-23T03:19:23.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.708+0000 7f109b7fe640 1 -- 172.21.15.154:0/998996775 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f108c00bc20 con 0x7f109c0cf9e0 2023-09-23T03:19:23.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.708+0000 7f10a1315640 1 --2- 172.21.15.154:0/998996775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f108c004d00 0x7f108c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:23.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.708+0000 7f109bfff640 1 --2- 172.21.15.154:0/998996775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f108c003f20 0x7f108c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:23.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.708+0000 7f10a1315640 1 -- 172.21.15.154:0/998996775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f109c0cf9e0 msgr2=0x7f108c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:23.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.708+0000 7f10a1315640 1 --2- 172.21.15.154:0/998996775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f109c0cf9e0 0x7f108c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:23.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.708+0000 7f10a1315640 1 -- 172.21.15.154:0/998996775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f108c003f20 msgr2=0x7f108c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:23.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.708+0000 7f10a1315640 1 --2- 172.21.15.154:0/998996775 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f108c003f20 0x7f108c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:23.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.708+0000 7f10a1315640 1 -- 172.21.15.154:0/998996775 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108c003520 con 0x7f108c004d00 2023-09-23T03:19:23.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.709+0000 7f10a1315640 1 --2- 172.21.15.154:0/998996775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f108c004d00 0x7f108c005180 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f109400a830 tx=0x7f108c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d313439edf2f51f6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:23.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.710+0000 7f109b7fe640 1 -- 172.21.15.154:0/998996775 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10940072d0 con 0x7f108c004d00 2023-09-23T03:19:23.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.710+0000 7f109b7fe640 1 -- 172.21.15.154:0/998996775 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109402f070 con 0x7f108c004d00 2023-09-23T03:19:23.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.710+0000 7f109b7fe640 1 -- 172.21.15.154:0/998996775 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10940076f0 con 0x7f108c004d00 2023-09-23T03:19:23.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.710+0000 7f10a2d78640 1 -- 172.21.15.154:0/998996775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f108c004d00 msgr2=0x7f108c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:23.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.710+0000 7f10a2d78640 1 --2- 172.21.15.154:0/998996775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f108c004d00 0x7f108c005180 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f109400a830 tx=0x7f108c0057f0 comp rx=0 tx=0).stop 2023-09-23T03:19:23.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.710+0000 7f10a1315640 1 -- 172.21.15.154:0/998996775 reap_dead start 2023-09-23T03:19:23.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.710+0000 7f10a2d78640 1 -- 172.21.15.154:0/998996775 shutdown_connections 2023-09-23T03:19:23.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.710+0000 7f10a2d78640 1 -- 172.21.15.154:0/998996775 >> 172.21.15.154:0/998996775 conn(0x7f109c09bc10 msgr2=0x7f109c0d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:23.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.710+0000 7f10a2d78640 1 -- 172.21.15.154:0/998996775 shutdown_connections 2023-09-23T03:19:23.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.711+0000 7f10a2d78640 1 -- 172.21.15.154:0/998996775 wait complete. 2023-09-23T03:19:23.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.712+0000 7f10a2d78640 1 Processor -- start 2023-09-23T03:19:23.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.712+0000 7f10a2d78640 1 -- start start 2023-09-23T03:19:23.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.713+0000 7f10a2d78640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f108c004d00 0x7f109c151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:23.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.713+0000 7f10a2d78640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f109c1f2230 0x7f109c1f26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:23.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.713+0000 7f10a2d78640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f109c15d6c0 0x7f109c15db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:23.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.713+0000 7f10a2d78640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c15e080 con 0x7f109c1f2230 2023-09-23T03:19:23.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.713+0000 7f10a2d78640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c1521a0 con 0x7f108c004d00 2023-09-23T03:19:23.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.713+0000 7f10a2d78640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f109c1f2cd0 con 0x7f109c15d6c0 2023-09-23T03:19:23.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.714+0000 7f109bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f109c1f2230 0x7f109c1f26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:23.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.714+0000 7f109bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f109c1f2230 0x7f109c1f26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56550/0 (socket says 172.21.15.154:56550) 2023-09-23T03:19:23.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.714+0000 7f109bfff640 1 -- 172.21.15.154:0/1147751577 learned_addr learned my addr 172.21.15.154:0/1147751577 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:23.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.714+0000 7f10a0b14640 1 --2- 172.21.15.154:0/1147751577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f108c004d00 0x7f109c151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:23.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.714+0000 7f10a1315640 1 --2- 172.21.15.154:0/1147751577 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f109c15d6c0 0x7f109c15db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:23.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.714+0000 7f109bfff640 1 -- 172.21.15.154:0/1147751577 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f109c15d6c0 msgr2=0x7f109c15db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:23.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.714+0000 7f109bfff640 1 --2- 172.21.15.154:0/1147751577 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f109c15d6c0 0x7f109c15db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:23.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.714+0000 7f109bfff640 1 -- 172.21.15.154:0/1147751577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f108c004d00 msgr2=0x7f109c151c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:23.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.714+0000 7f109bfff640 1 --2- 172.21.15.154:0/1147751577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f108c004d00 0x7f109c151c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:23.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.714+0000 7f109bfff640 1 -- 172.21.15.154:0/1147751577 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f109c1f2fb0 con 0x7f109c1f2230 2023-09-23T03:19:23.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.714+0000 7f10a0b14640 1 --2- 172.21.15.154:0/1147751577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f108c004d00 0x7f109c151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:23.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.714+0000 7f109bfff640 1 --2- 172.21.15.154:0/1147751577 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f109c1f2230 0x7f109c1f26b0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f10880116a0 tx=0x7f1088011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:23.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.715+0000 7f10997fa640 1 -- 172.21.15.154:0/1147751577 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108800f9c0 con 0x7f109c1f2230 2023-09-23T03:19:23.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.715+0000 7f10997fa640 1 -- 172.21.15.154:0/1147751577 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108800fb20 con 0x7f109c1f2230 2023-09-23T03:19:23.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.715+0000 7f10997fa640 1 -- 172.21.15.154:0/1147751577 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108800fed0 con 0x7f109c1f2230 2023-09-23T03:19:23.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.715+0000 7f10a2d78640 1 -- 172.21.15.154:0/1147751577 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f109c09c700 con 0x7f109c1f2230 2023-09-23T03:19:23.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.715+0000 7f10a2d78640 1 -- 172.21.15.154:0/1147751577 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f109c09fe10 con 0x7f109c1f2230 2023-09-23T03:19:23.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.716+0000 7f10a2d78640 1 -- 172.21.15.154:0/1147751577 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f109c06c950 con 0x7f109c1f2230 2023-09-23T03:19:23.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.717+0000 7f10997fa640 1 -- 172.21.15.154:0/1147751577 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f1088015460 con 0x7f109c1f2230 2023-09-23T03:19:23.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.718+0000 7f10997fa640 1 --2- 172.21.15.154:0/1147751577 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f108006af60 0x7f108006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:23.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.718+0000 7f10997fa640 1 -- 172.21.15.154:0/1147751577 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f1088089d30 con 0x7f109c1f2230 2023-09-23T03:19:23.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.718+0000 7f10a0b14640 1 --2- 172.21.15.154:0/1147751577 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f108006af60 0x7f108006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:23.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.719+0000 7f10a0b14640 1 --2- 172.21.15.154:0/1147751577 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f108006af60 0x7f108006d420 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f10900016f0 tx=0x7f1090009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:23.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.723+0000 7f10997fa640 1 -- 172.21.15.154:0/1147751577 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f1088054c90 con 0x7f109c1f2230 2023-09-23T03:19:23.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:23.951+0000 7f10a2d78640 1 -- 172.21.15.154:0/1147751577 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f109c04e920 con 0x7f109c1f2230 2023-09-23T03:19:24.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.482+0000 7f10997fa640 1 -- 172.21.15.154:0/1147751577 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v36) v1 ==== 172+0+0 (secure 0 0 0) 0x7f10880220c0 con 0x7f109c1f2230 2023-09-23T03:19:24.482 INFO:teuthology.orchestra.run.smithi154.stderr:new fs with metadata pool 8 and data pool 9 2023-09-23T03:19:24.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.489+0000 7f10a2d78640 1 -- 172.21.15.154:0/1147751577 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f108006af60 msgr2=0x7f108006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:24.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.489+0000 7f10a2d78640 1 --2- 172.21.15.154:0/1147751577 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f108006af60 0x7f108006d420 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f10900016f0 tx=0x7f1090009040 comp rx=0 tx=0).stop 2023-09-23T03:19:24.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.489+0000 7f10a2d78640 1 -- 172.21.15.154:0/1147751577 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f109c1f2230 msgr2=0x7f109c1f26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:24.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.489+0000 7f10a2d78640 1 --2- 172.21.15.154:0/1147751577 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f109c1f2230 0x7f109c1f26b0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f10880116a0 tx=0x7f1088011b60 comp rx=0 tx=0).stop 2023-09-23T03:19:24.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.490+0000 7f10a2d78640 1 -- 172.21.15.154:0/1147751577 shutdown_connections 2023-09-23T03:19:24.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.490+0000 7f10a2d78640 1 --2- 172.21.15.154:0/1147751577 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f108006af60 0x7f108006d420 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:24.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.490+0000 7f10a2d78640 1 --2- 172.21.15.154:0/1147751577 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f109c15d6c0 0x7f109c15db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:24.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.490+0000 7f10a2d78640 1 --2- 172.21.15.154:0/1147751577 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f109c1f2230 0x7f109c1f26b0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:24.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.490+0000 7f10a2d78640 1 --2- 172.21.15.154:0/1147751577 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f108c004d00 0x7f109c151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:24.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.490+0000 7f10a2d78640 1 -- 172.21.15.154:0/1147751577 >> 172.21.15.154:0/1147751577 conn(0x7f109c09bc10 msgr2=0x7f109c06c240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:24.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.493+0000 7f10a2d78640 1 -- 172.21.15.154:0/1147751577 shutdown_connections 2023-09-23T03:19:24.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.494+0000 7f10a2d78640 1 -- 172.21.15.154:0/1147751577 wait complete. 2023-09-23T03:19:24.534 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:19:24.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.715+0000 7f46fdf68640 1 Processor -- start 2023-09-23T03:19:24.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.716+0000 7f46fdf68640 1 -- start start 2023-09-23T03:19:24.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.717+0000 7f46fdf68640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46f80acce0 0x7f46f815ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:24.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.717+0000 7f46fdf68640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f46f80cf040 0x7f46f80ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:24.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.717+0000 7f46fdf68640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46f815ac30 con 0x7f46f80b1b20 2023-09-23T03:19:24.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.717+0000 7f46fdf68640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46f80b2e90 con 0x7f46f80cf040 2023-09-23T03:19:24.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.717+0000 7f46fdf68640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f80b1f70 con 0x7f46f80acce0 2023-09-23T03:19:24.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.718+0000 7f46f7fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f46f80cf040 0x7f46f80ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:24.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.718+0000 7f46f77fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46f80acce0 0x7f46f815ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:24.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.718+0000 7f46f7fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f46f80cf040 0x7f46f80ab140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.154:60004/0 (socket says 172.21.15.154:60004) 2023-09-23T03:19:24.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.718+0000 7f46f77fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46f80acce0 0x7f46f815ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:54612/0 (socket says 172.21.15.154:54612) 2023-09-23T03:19:24.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.718+0000 7f46f7fff640 1 -- 172.21.15.154:0/2159787146 learned_addr learned my addr 172.21.15.154:0/2159787146 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:24.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.719+0000 7f46f77fe640 1 -- 172.21.15.154:0/2159787146 >> v2:172.21.15.190:3300/0 conn(0x7f46f80cf040 msgr2=0x7f46f80ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:24.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.719+0000 7f46f77fe640 1 --2- 172.21.15.154:0/2159787146 >> v2:172.21.15.190:3300/0 conn(0x7f46f80cf040 0x7f46f80ab140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:24.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.719+0000 7f46f77fe640 1 -- 172.21.15.154:0/2159787146 >> v1:172.21.15.190:6789/0 conn(0x7f46f80b1b20 legacy=0x7f46f80ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:24.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.719+0000 7f46f77fe640 1 -- 172.21.15.154:0/2159787146 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f815f2a0 con 0x7f46f80acce0 2023-09-23T03:19:24.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.720+0000 7f46f77fe640 1 --2- 172.21.15.154:0/2159787146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46f80acce0 0x7f46f815ed60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f46e0009790 tx=0x7f46e002b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5eb99ed4ddf05055 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:24.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.721+0000 7f46f67fc640 1 -- 172.21.15.154:0/2159787146 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e0031770 con 0x7f46f80acce0 2023-09-23T03:19:24.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.721+0000 7f46f67fc640 1 -- 172.21.15.154:0/2159787146 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e00318d0 con 0x7f46f80acce0 2023-09-23T03:19:24.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.721+0000 7f46f67fc640 1 -- 172.21.15.154:0/2159787146 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e0031c60 con 0x7f46f80acce0 2023-09-23T03:19:24.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.721+0000 7f46fdf68640 1 -- 172.21.15.154:0/2159787146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46f80acce0 msgr2=0x7f46f815ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:24.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.721+0000 7f46fdf68640 1 --2- 172.21.15.154:0/2159787146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46f80acce0 0x7f46f815ed60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f46e0009790 tx=0x7f46e002b810 comp rx=0 tx=0).stop 2023-09-23T03:19:24.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.721+0000 7f46fdf68640 1 -- 172.21.15.154:0/2159787146 shutdown_connections 2023-09-23T03:19:24.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.721+0000 7f46fdf68640 1 --2- 172.21.15.154:0/2159787146 >> v2:172.21.15.190:3300/0 conn(0x7f46f80cf040 0x7f46f80ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:24.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.721+0000 7f46fdf68640 1 --2- 172.21.15.154:0/2159787146 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46f80acce0 0x7f46f815ed60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:24.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.721+0000 7f46fdf68640 1 -- 172.21.15.154:0/2159787146 >> 172.21.15.154:0/2159787146 conn(0x7f46f806a0f0 msgr2=0x7f46f806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:24.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.722+0000 7f46fdf68640 1 -- 172.21.15.154:0/2159787146 shutdown_connections 2023-09-23T03:19:24.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.722+0000 7f46fdf68640 1 -- 172.21.15.154:0/2159787146 wait complete. 2023-09-23T03:19:24.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.723+0000 7f46fdf68640 1 Processor -- start 2023-09-23T03:19:24.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.723+0000 7f46fdf68640 1 -- start start 2023-09-23T03:19:24.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.724+0000 7f46fdf68640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46f80acce0 0x7f46f815bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:24.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.724+0000 7f46fdf68640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46f80b1b20 0x7f46f81576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:24.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.724+0000 7f46fdf68640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46f80cf040 0x7f46f815c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:24.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.724+0000 7f46fdf68640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f815add0 con 0x7f46f80cf040 2023-09-23T03:19:24.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.724+0000 7f46fdf68640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f815b020 con 0x7f46f80acce0 2023-09-23T03:19:24.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.724+0000 7f46fdf68640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f815b300 con 0x7f46f80b1b20 2023-09-23T03:19:24.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.725+0000 7f46f7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46f80cf040 0x7f46f815c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:24.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.725+0000 7f46f7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46f80cf040 0x7f46f815c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:42914/0 (socket says 172.21.15.154:42914) 2023-09-23T03:19:24.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.725+0000 7f46f7fff640 1 -- 172.21.15.154:0/1814754719 learned_addr learned my addr 172.21.15.154:0/1814754719 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:24.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.725+0000 7f46f77fe640 1 --2- 172.21.15.154:0/1814754719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46f80acce0 0x7f46f815bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:24.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.725+0000 7f46f6ffd640 1 --2- 172.21.15.154:0/1814754719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46f80b1b20 0x7f46f81576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:24.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.725+0000 7f46f7fff640 1 -- 172.21.15.154:0/1814754719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46f80b1b20 msgr2=0x7f46f81576e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:24.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.725+0000 7f46f7fff640 1 --2- 172.21.15.154:0/1814754719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46f80b1b20 0x7f46f81576e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:24.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.725+0000 7f46f7fff640 1 -- 172.21.15.154:0/1814754719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46f80acce0 msgr2=0x7f46f815bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:24.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.725+0000 7f46f7fff640 1 --2- 172.21.15.154:0/1814754719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46f80acce0 0x7f46f815bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:24.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.725+0000 7f46f7fff640 1 -- 172.21.15.154:0/1814754719 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e00093a0 con 0x7f46f80cf040 2023-09-23T03:19:24.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.725+0000 7f46f6ffd640 1 --2- 172.21.15.154:0/1814754719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46f80b1b20 0x7f46f81576e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:24.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.726+0000 7f46f7fff640 1 --2- 172.21.15.154:0/1814754719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46f80cf040 0x7f46f815c910 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f46f000e870 tx=0x7f46f000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:24.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.726+0000 7f46effff640 1 -- 172.21.15.154:0/1814754719 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f000ee60 con 0x7f46f80cf040 2023-09-23T03:19:24.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.726+0000 7f46effff640 1 -- 172.21.15.154:0/1814754719 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f000c940 con 0x7f46f80cf040 2023-09-23T03:19:24.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.726+0000 7f46effff640 1 -- 172.21.15.154:0/1814754719 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f000ccf0 con 0x7f46f80cf040 2023-09-23T03:19:24.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.726+0000 7f46fdf68640 1 -- 172.21.15.154:0/1814754719 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46f815b5e0 con 0x7f46f80cf040 2023-09-23T03:19:24.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.726+0000 7f46fdf68640 1 -- 172.21.15.154:0/1814754719 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46f8157c20 con 0x7f46f80cf040 2023-09-23T03:19:24.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.727+0000 7f46fdf68640 1 -- 172.21.15.154:0/1814754719 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46f815bb00 con 0x7f46f80cf040 2023-09-23T03:19:24.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.728+0000 7f46effff640 1 -- 172.21.15.154:0/1814754719 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f46f0013470 con 0x7f46f80cf040 2023-09-23T03:19:24.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.729+0000 7f46effff640 1 --2- 172.21.15.154:0/1814754719 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f46d806af30 0x7f46d806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:24.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.729+0000 7f46effff640 1 -- 172.21.15.154:0/1814754719 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f46f0089a30 con 0x7f46f80cf040 2023-09-23T03:19:24.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.729+0000 7f46f77fe640 1 --2- 172.21.15.154:0/1814754719 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f46d806af30 0x7f46d806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:24.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.730+0000 7f46f77fe640 1 --2- 172.21.15.154:0/1814754719 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f46d806af30 0x7f46d806d3f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f46e0036000 tx=0x7f46e003d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:24.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.734+0000 7f46effff640 1 -- 172.21.15.154:0/1814754719 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f46f815bb00 con 0x7f46f80cf040 2023-09-23T03:19:24.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.916+0000 7f46fdf68640 1 -- 172.21.15.154:0/1814754719 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46f815bb00 con 0x7f46f80cf040 2023-09-23T03:19:24.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.918+0000 7f46effff640 1 -- 172.21.15.154:0/1814754719 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+16819 (secure 0 0 0) 0x7f46f815bb00 con 0x7f46f80cf040 2023-09-23T03:19:24.919 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:19:24.919 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":61,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:19:23.952295+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:19:21.772037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-09-23T03:19:22.960250+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:19:24.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.921+0000 7f46fdf68640 1 -- 172.21.15.154:0/1814754719 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f46d806af30 msgr2=0x7f46d806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:24.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.921+0000 7f46fdf68640 1 --2- 172.21.15.154:0/1814754719 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f46d806af30 0x7f46d806d3f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f46e0036000 tx=0x7f46e003d040 comp rx=0 tx=0).stop 2023-09-23T03:19:24.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.921+0000 7f46fdf68640 1 -- 172.21.15.154:0/1814754719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46f80cf040 msgr2=0x7f46f815c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:24.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.921+0000 7f46fdf68640 1 --2- 172.21.15.154:0/1814754719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46f80cf040 0x7f46f815c910 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f46f000e870 tx=0x7f46f000ed30 comp rx=0 tx=0).stop 2023-09-23T03:19:24.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.921+0000 7f46fdf68640 1 -- 172.21.15.154:0/1814754719 shutdown_connections 2023-09-23T03:19:24.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.921+0000 7f46fdf68640 1 --2- 172.21.15.154:0/1814754719 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f46d806af30 0x7f46d806d3f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:24.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.921+0000 7f46fdf68640 1 --2- 172.21.15.154:0/1814754719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f46f80cf040 0x7f46f815c910 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:24.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.921+0000 7f46fdf68640 1 --2- 172.21.15.154:0/1814754719 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f46f80b1b20 0x7f46f81576e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:24.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.921+0000 7f46fdf68640 1 --2- 172.21.15.154:0/1814754719 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f46f80acce0 0x7f46f815bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:24.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.921+0000 7f46fdf68640 1 -- 172.21.15.154:0/1814754719 >> 172.21.15.154:0/1814754719 conn(0x7f46f806a0f0 msgr2=0x7f46f80ca7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:24.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.921+0000 7f46fdf68640 1 -- 172.21.15.154:0/1814754719 shutdown_connections 2023-09-23T03:19:24.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:24.921+0000 7f46fdf68640 1 -- 172.21.15.154:0/1814754719 wait complete. 2023-09-23T03:19:24.939 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:19:25.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.144+0000 7fa1ba91d640 1 Processor -- start 2023-09-23T03:19:25.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.145+0000 7fa1ba91d640 1 -- start start 2023-09-23T03:19:25.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.146+0000 7fa1ba91d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1b40acce0 0x7fa1b415ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:25.147 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.146+0000 7fa1ba91d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa1b40b1b20 0x7fa1b40ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:25.147 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.146+0000 7fa1ba91d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1b415ac30 con 0x7fa1b40cee70 2023-09-23T03:19:25.147 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.146+0000 7fa1ba91d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1b40b2e90 con 0x7fa1b40b1b20 2023-09-23T03:19:25.147 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.146+0000 7fa1ba91d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b40cf320 con 0x7fa1b40acce0 2023-09-23T03:19:25.147 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.146+0000 7fa1abfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa1b40b1b20 0x7fa1b40ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:25.147 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.146+0000 7fa1abfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa1b40b1b20 0x7fa1b40ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:42926/0 (socket says 172.21.15.154:42926) 2023-09-23T03:19:25.147 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.146+0000 7fa1b8eba640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fa1b40cee70 0x7fa1b406cb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:34754/0 (socket says 172.21.15.154:34754) 2023-09-23T03:19:25.147 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.146+0000 7fa1abfff640 1 -- 172.21.15.154:0/730781046 learned_addr learned my addr 172.21.15.154:0/730781046 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:25.147 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.146+0000 7fa1b3fff640 1 --2- 172.21.15.154:0/730781046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1b40acce0 0x7fa1b415ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:25.147 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.146+0000 7fa1abfff640 1 -- 172.21.15.154:0/730781046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1b40acce0 msgr2=0x7fa1b415ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:25.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.146+0000 7fa1abfff640 1 --2- 172.21.15.154:0/730781046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1b40acce0 0x7fa1b415ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:25.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.146+0000 7fa1b37fe640 1 -- 172.21.15.154:0/730781046 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2408985483 0 0) 0x7fa1ac001490 con 0x7fa1b40cee70 2023-09-23T03:19:25.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.146+0000 7fa1abfff640 1 -- 172.21.15.154:0/730781046 >> v1:172.21.15.154:6789/0 conn(0x7fa1b40cee70 legacy=0x7fa1b406cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:25.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.146+0000 7fa1abfff640 1 -- 172.21.15.154:0/730781046 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b40b1f70 con 0x7fa1b40b1b20 2023-09-23T03:19:25.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.148+0000 7fa1abfff640 1 --2- 172.21.15.154:0/730781046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1b40b1b20 0x7fa1b40ce930 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fa19c009870 tx=0x7fa19c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107fa2c3152f27f1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:25.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.148+0000 7fa1b37fe640 1 -- 172.21.15.154:0/730781046 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19c031640 con 0x7fa1b40b1b20 2023-09-23T03:19:25.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.148+0000 7fa1b37fe640 1 -- 172.21.15.154:0/730781046 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa19c0317a0 con 0x7fa1b40b1b20 2023-09-23T03:19:25.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.148+0000 7fa1b37fe640 1 -- 172.21.15.154:0/730781046 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19c031b30 con 0x7fa1b40b1b20 2023-09-23T03:19:25.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.148+0000 7fa1ba91d640 1 -- 172.21.15.154:0/730781046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1b40b1b20 msgr2=0x7fa1b40ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:25.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.148+0000 7fa1ba91d640 1 --2- 172.21.15.154:0/730781046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1b40b1b20 0x7fa1b40ce930 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fa19c009870 tx=0x7fa19c02b8f0 comp rx=0 tx=0).stop 2023-09-23T03:19:25.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.149+0000 7fa1ba91d640 1 -- 172.21.15.154:0/730781046 shutdown_connections 2023-09-23T03:19:25.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.149+0000 7fa1ba91d640 1 --2- 172.21.15.154:0/730781046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1b40b1b20 0x7fa1b40ce930 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:25.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.149+0000 7fa1ba91d640 1 --2- 172.21.15.154:0/730781046 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1b40acce0 0x7fa1b415ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:25.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.149+0000 7fa1ba91d640 1 -- 172.21.15.154:0/730781046 >> 172.21.15.154:0/730781046 conn(0x7fa1b406a0f0 msgr2=0x7fa1b406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:25.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.149+0000 7fa1ba91d640 1 -- 172.21.15.154:0/730781046 shutdown_connections 2023-09-23T03:19:25.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.149+0000 7fa1ba91d640 1 -- 172.21.15.154:0/730781046 wait complete. 2023-09-23T03:19:25.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.150+0000 7fa1ba91d640 1 Processor -- start 2023-09-23T03:19:25.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.150+0000 7fa1ba91d640 1 -- start start 2023-09-23T03:19:25.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.151+0000 7fa1ba91d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1b40acce0 0x7fa1b415bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:25.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.151+0000 7fa1ba91d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1b40b1b20 0x7fa1b40c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:25.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.151+0000 7fa1ba91d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1b40cee70 0x7fa1b40c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:25.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.151+0000 7fa1ba91d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b40c8670 con 0x7fa1b40cee70 2023-09-23T03:19:25.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.151+0000 7fa1ba91d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b40c3090 con 0x7fa1b40b1b20 2023-09-23T03:19:25.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.151+0000 7fa1ba91d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b415c050 con 0x7fa1b40acce0 2023-09-23T03:19:25.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.151+0000 7fa1b8eba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1b40cee70 0x7fa1b40c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:25.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.152+0000 7fa1b8eba640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1b40cee70 0x7fa1b40c8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:42932/0 (socket says 172.21.15.154:42932) 2023-09-23T03:19:25.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.152+0000 7fa1b8eba640 1 -- 172.21.15.154:0/3065782335 learned_addr learned my addr 172.21.15.154:0/3065782335 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:25.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.152+0000 7fa1b3fff640 1 --2- 172.21.15.154:0/3065782335 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1b40acce0 0x7fa1b415bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:25.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.152+0000 7fa1b8eba640 1 -- 172.21.15.154:0/3065782335 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1b40acce0 msgr2=0x7fa1b415bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:25.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.152+0000 7fa1b8eba640 1 --2- 172.21.15.154:0/3065782335 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1b40acce0 0x7fa1b415bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:25.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.152+0000 7fa1b8eba640 1 -- 172.21.15.154:0/3065782335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1b40b1b20 msgr2=0x7fa1b40c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:25.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.152+0000 7fa1b8eba640 1 --2- 172.21.15.154:0/3065782335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1b40b1b20 0x7fa1b40c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:25.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.152+0000 7fa1b8eba640 1 -- 172.21.15.154:0/3065782335 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa19c0094d0 con 0x7fa1b40cee70 2023-09-23T03:19:25.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.152+0000 7fa1b8eba640 1 --2- 172.21.15.154:0/3065782335 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1b40cee70 0x7fa1b40c8130 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00c700 tx=0x7fa1ac00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:25.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.152+0000 7fa1b17fa640 1 -- 172.21.15.154:0/3065782335 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac00ce10 con 0x7fa1b40cee70 2023-09-23T03:19:25.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.152+0000 7fa1b17fa640 1 -- 172.21.15.154:0/3065782335 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1ac00a640 con 0x7fa1b40cee70 2023-09-23T03:19:25.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.152+0000 7fa1b17fa640 1 -- 172.21.15.154:0/3065782335 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac00aa70 con 0x7fa1b40cee70 2023-09-23T03:19:25.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.152+0000 7fa1ba91d640 1 -- 172.21.15.154:0/3065782335 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1b40c5db0 con 0x7fa1b40cee70 2023-09-23T03:19:25.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.153+0000 7fa1ba91d640 1 -- 172.21.15.154:0/3065782335 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1b40c6370 con 0x7fa1b40cee70 2023-09-23T03:19:25.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.154+0000 7fa1ba91d640 1 -- 172.21.15.154:0/3065782335 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1b40c3250 con 0x7fa1b40cee70 2023-09-23T03:19:25.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.155+0000 7fa1b17fa640 1 -- 172.21.15.154:0/3065782335 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fa1ac00abd0 con 0x7fa1b40cee70 2023-09-23T03:19:25.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.156+0000 7fa1b17fa640 1 --2- 172.21.15.154:0/3065782335 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa19406af30 0x7fa19406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:25.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.156+0000 7fa1b17fa640 1 -- 172.21.15.154:0/3065782335 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa1ac088c20 con 0x7fa1b40cee70 2023-09-23T03:19:25.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.156+0000 7fa1b3fff640 1 --2- 172.21.15.154:0/3065782335 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa19406af30 0x7fa19406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:25.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.156+0000 7fa1b3fff640 1 --2- 172.21.15.154:0/3065782335 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa19406af30 0x7fa19406d3f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4009770 tx=0x7fa1a4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:25.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.160+0000 7fa1b17fa640 1 -- 172.21.15.154:0/3065782335 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fa1ac053b20 con 0x7fa1b40cee70 2023-09-23T03:19:25.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.353+0000 7fa1ba91d640 1 -- 172.21.15.154:0/3065782335 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1b415ac10 con 0x7fa1b40cee70 2023-09-23T03:19:25.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.356+0000 7fa1b17fa640 1 -- 172.21.15.154:0/3065782335 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+16819 (secure 0 0 0) 0x7fa1ac018070 con 0x7fa1b40cee70 2023-09-23T03:19:25.357 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:19:25.357 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":61,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:19:23.952295+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:19:21.772037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-09-23T03:19:22.960250+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:19:25.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.359+0000 7fa1ba91d640 1 -- 172.21.15.154:0/3065782335 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa19406af30 msgr2=0x7fa19406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:25.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.359+0000 7fa1ba91d640 1 --2- 172.21.15.154:0/3065782335 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa19406af30 0x7fa19406d3f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4009770 tx=0x7fa1a4008040 comp rx=0 tx=0).stop 2023-09-23T03:19:25.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.359+0000 7fa1ba91d640 1 -- 172.21.15.154:0/3065782335 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1b40cee70 msgr2=0x7fa1b40c8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:25.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.359+0000 7fa1ba91d640 1 --2- 172.21.15.154:0/3065782335 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1b40cee70 0x7fa1b40c8130 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00c700 tx=0x7fa1ac00cbc0 comp rx=0 tx=0).stop 2023-09-23T03:19:25.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.359+0000 7fa1ba91d640 1 -- 172.21.15.154:0/3065782335 shutdown_connections 2023-09-23T03:19:25.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.359+0000 7fa1ba91d640 1 --2- 172.21.15.154:0/3065782335 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa19406af30 0x7fa19406d3f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:25.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.359+0000 7fa1ba91d640 1 --2- 172.21.15.154:0/3065782335 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1b40cee70 0x7fa1b40c8130 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:25.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.359+0000 7fa1ba91d640 1 --2- 172.21.15.154:0/3065782335 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1b40b1b20 0x7fa1b40c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:25.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.359+0000 7fa1ba91d640 1 --2- 172.21.15.154:0/3065782335 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1b40acce0 0x7fa1b415bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:25.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.359+0000 7fa1ba91d640 1 -- 172.21.15.154:0/3065782335 >> 172.21.15.154:0/3065782335 conn(0x7fa1b406a0f0 msgr2=0x7fa1b40aed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:25.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.359+0000 7fa1ba91d640 1 -- 172.21.15.154:0/3065782335 shutdown_connections 2023-09-23T03:19:25.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.359+0000 7fa1ba91d640 1 -- 172.21.15.154:0/3065782335 wait complete. 2023-09-23T03:19:25.379 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-09-23T03:19:25.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.597+0000 7f0c784ec640 1 Processor -- start 2023-09-23T03:19:25.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.598+0000 7f0c784ec640 1 -- start start 2023-09-23T03:19:25.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.599+0000 7f0c784ec640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0c7015bb30 0x7f0c700caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:25.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.599+0000 7f0c784ec640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c7015b6a0 con 0x7f0c700cb3e0 2023-09-23T03:19:25.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.599+0000 7f0c784ec640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c7015f0b0 con 0x7f0c7015aba0 2023-09-23T03:19:25.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.599+0000 7f0c784ec640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c700cb870 con 0x7f0c7015bb30 2023-09-23T03:19:25.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.600+0000 7f0c76a89640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f0c700cb3e0 0x7f0c700cc5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:34758/0 (socket says 172.21.15.154:34758) 2023-09-23T03:19:25.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.600+0000 7f0c76a89640 1 -- 172.21.15.154:0/3175024771 learned_addr learned my addr 172.21.15.154:0/3175024771 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:19:25.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.600+0000 7f0c75a87640 1 --2- 172.21.15.154:0/3175024771 >> v2:172.21.15.190:3300/0 conn(0x7f0c7015bb30 0x7f0c700caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:25.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.600+0000 7f0c75286640 1 -- 172.21.15.154:0/3175024771 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4045298016 0 0) 0x7f0c7015b6a0 con 0x7f0c700cb3e0 2023-09-23T03:19:25.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.600+0000 7f0c75286640 1 -- 172.21.15.154:0/3175024771 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c5c003520 con 0x7f0c700cb3e0 2023-09-23T03:19:25.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.600+0000 7f0c75286640 1 -- 172.21.15.154:0/3175024771 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 850203237 0 0) 0x7f0c5c003520 con 0x7f0c700cb3e0 2023-09-23T03:19:25.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.600+0000 7f0c75286640 1 -- 172.21.15.154:0/3175024771 >> v2:172.21.15.190:3300/0 conn(0x7f0c7015bb30 msgr2=0x7f0c700caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:25.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.600+0000 7f0c75286640 1 --2- 172.21.15.154:0/3175024771 >> v2:172.21.15.190:3300/0 conn(0x7f0c7015bb30 0x7f0c700caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:25.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.600+0000 7f0c75286640 1 -- 172.21.15.154:0/3175024771 >> v1:172.21.15.190:6789/0 conn(0x7f0c7015aba0 legacy=0x7f0c7015e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:25.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.600+0000 7f0c75286640 1 -- 172.21.15.154:0/3175024771 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7015f390 con 0x7f0c700cb3e0 2023-09-23T03:19:25.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.600+0000 7f0c75286640 1 -- 172.21.15.154:0/3175024771 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f0c680019f0 con 0x7f0c700cb3e0 2023-09-23T03:19:25.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.600+0000 7f0c75286640 1 -- 172.21.15.154:0/3175024771 >> v1:172.21.15.154:6789/0 conn(0x7f0c700cb3e0 legacy=0x7f0c700cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:25.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.601+0000 7f0c75286640 1 --2- 172.21.15.154:0/3175024771 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c5c0039c0 0x7f0c5c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:25.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.601+0000 7f0c75286640 1 --2- 172.21.15.154:0/3175024771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c5c0047c0 0x7f0c5c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:25.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.601+0000 7f0c75286640 1 --2- 172.21.15.154:0/3175024771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c5c005180 0x7f0c5c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:25.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.601+0000 7f0c75286640 1 -- 172.21.15.154:0/3175024771 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c700cb870 con 0x7f0c5c0047c0 2023-09-23T03:19:25.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.601+0000 7f0c75286640 1 -- 172.21.15.154:0/3175024771 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c00c080 con 0x7f0c5c0039c0 2023-09-23T03:19:25.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.601+0000 7f0c75286640 1 -- 172.21.15.154:0/3175024771 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c00c240 con 0x7f0c5c005180 2023-09-23T03:19:25.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.601+0000 7f0c75a87640 1 --2- 172.21.15.154:0/3175024771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c5c0047c0 0x7f0c5c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:25.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.601+0000 7f0c76288640 1 --2- 172.21.15.154:0/3175024771 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c5c0039c0 0x7f0c5c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:25.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.601+0000 7f0c76a89640 1 --2- 172.21.15.154:0/3175024771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c5c005180 0x7f0c5c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:25.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.601+0000 7f0c75a87640 1 -- 172.21.15.154:0/3175024771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c5c005180 msgr2=0x7f0c5c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:25.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.601+0000 7f0c75a87640 1 --2- 172.21.15.154:0/3175024771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c5c005180 0x7f0c5c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:25.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.601+0000 7f0c75a87640 1 -- 172.21.15.154:0/3175024771 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c5c0039c0 msgr2=0x7f0c5c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:25.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.601+0000 7f0c75a87640 1 --2- 172.21.15.154:0/3175024771 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c5c0039c0 0x7f0c5c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:25.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.601+0000 7f0c75a87640 1 -- 172.21.15.154:0/3175024771 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c5c003520 con 0x7f0c5c0047c0 2023-09-23T03:19:25.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.601+0000 7f0c76a89640 1 --2- 172.21.15.154:0/3175024771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c5c005180 0x7f0c5c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:25.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.601+0000 7f0c76288640 1 --2- 172.21.15.154:0/3175024771 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c5c0039c0 0x7f0c5c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:25.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.601+0000 7f0c76a89640 1 -- 172.21.15.154:0/3175024771 reap_dead start 2023-09-23T03:19:25.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.602+0000 7f0c75a87640 1 --2- 172.21.15.154:0/3175024771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c5c0047c0 0x7f0c5c004c40 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f0c60009ac0 tx=0x7f0c5c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5741340596b600e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:25.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.603+0000 7f0c75286640 1 -- 172.21.15.154:0/3175024771 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6002c6b0 con 0x7f0c5c0047c0 2023-09-23T03:19:25.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.603+0000 7f0c75286640 1 -- 172.21.15.154:0/3175024771 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c6002c810 con 0x7f0c5c0047c0 2023-09-23T03:19:25.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.603+0000 7f0c75286640 1 -- 172.21.15.154:0/3175024771 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6002cba0 con 0x7f0c5c0047c0 2023-09-23T03:19:25.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.603+0000 7f0c784ec640 1 -- 172.21.15.154:0/3175024771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c5c0047c0 msgr2=0x7f0c5c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:25.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.603+0000 7f0c784ec640 1 --2- 172.21.15.154:0/3175024771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c5c0047c0 0x7f0c5c004c40 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f0c60009ac0 tx=0x7f0c5c0057f0 comp rx=0 tx=0).stop 2023-09-23T03:19:25.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.603+0000 7f0c784ec640 1 -- 172.21.15.154:0/3175024771 shutdown_connections 2023-09-23T03:19:25.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.603+0000 7f0c784ec640 1 --2- 172.21.15.154:0/3175024771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c5c0047c0 0x7f0c5c004c40 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:25.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.603+0000 7f0c784ec640 1 -- 172.21.15.154:0/3175024771 >> 172.21.15.154:0/3175024771 conn(0x7f0c7006c010 msgr2=0x7f0c7006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:25.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.603+0000 7f0c784ec640 1 -- 172.21.15.154:0/3175024771 shutdown_connections 2023-09-23T03:19:25.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.603+0000 7f0c784ec640 1 -- 172.21.15.154:0/3175024771 wait complete. 2023-09-23T03:19:25.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.605+0000 7f0c784ec640 1 Processor -- start 2023-09-23T03:19:25.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.605+0000 7f0c784ec640 1 -- start start 2023-09-23T03:19:25.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.606+0000 7f0c784ec640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c5c0047c0 0x7f0c700b3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:25.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.606+0000 7f0c784ec640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c700b23a0 0x7f0c70150040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:25.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.606+0000 7f0c784ec640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c701409b0 0x7f0c700c7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:25.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.606+0000 7f0c784ec640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c70150580 con 0x7f0c5c0047c0 2023-09-23T03:19:25.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.606+0000 7f0c784ec640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7014ed80 con 0x7f0c700b23a0 2023-09-23T03:19:25.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.606+0000 7f0c784ec640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c700b2910 con 0x7f0c701409b0 2023-09-23T03:19:25.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.606+0000 7f0c76288640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c5c0047c0 0x7f0c700b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:25.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.606+0000 7f0c76288640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c5c0047c0 0x7f0c700b3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:42960/0 (socket says 172.21.15.154:42960) 2023-09-23T03:19:25.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.606+0000 7f0c76288640 1 -- 172.21.15.154:0/3507276403 learned_addr learned my addr 172.21.15.154:0/3507276403 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:25.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.607+0000 7f0c76288640 1 -- 172.21.15.154:0/3507276403 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c701409b0 msgr2=0x7f0c700c7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:25.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.607+0000 7f0c76288640 1 --2- 172.21.15.154:0/3507276403 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c701409b0 0x7f0c700c7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:25.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.607+0000 7f0c76288640 1 -- 172.21.15.154:0/3507276403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c700b23a0 msgr2=0x7f0c70150040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:25.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.607+0000 7f0c76288640 1 --2- 172.21.15.154:0/3507276403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c700b23a0 0x7f0c70150040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:25.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.607+0000 7f0c76288640 1 -- 172.21.15.154:0/3507276403 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c700b2bf0 con 0x7f0c5c0047c0 2023-09-23T03:19:25.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.607+0000 7f0c76288640 1 --2- 172.21.15.154:0/3507276403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c5c0047c0 0x7f0c700b3980 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c000f60 tx=0x7f0c6c00d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:25.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.607+0000 7f0c66ffd640 1 -- 172.21.15.154:0/3507276403 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c005480 con 0x7f0c5c0047c0 2023-09-23T03:19:25.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.607+0000 7f0c66ffd640 1 -- 172.21.15.154:0/3507276403 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c6c003610 con 0x7f0c5c0047c0 2023-09-23T03:19:25.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.607+0000 7f0c784ec640 1 -- 172.21.15.154:0/3507276403 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c700b2ed0 con 0x7f0c5c0047c0 2023-09-23T03:19:25.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.607+0000 7f0c66ffd640 1 -- 172.21.15.154:0/3507276403 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c0039c0 con 0x7f0c5c0047c0 2023-09-23T03:19:25.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.607+0000 7f0c784ec640 1 -- 172.21.15.154:0/3507276403 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c70151960 con 0x7f0c5c0047c0 2023-09-23T03:19:25.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.609+0000 7f0c784ec640 1 -- 172.21.15.154:0/3507276403 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c7004e920 con 0x7f0c5c0047c0 2023-09-23T03:19:25.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.609+0000 7f0c66ffd640 1 -- 172.21.15.154:0/3507276403 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f0c6c003b20 con 0x7f0c5c0047c0 2023-09-23T03:19:25.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.610+0000 7f0c66ffd640 1 --2- 172.21.15.154:0/3507276403 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0c5406af30 0x7f0c5406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:25.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.610+0000 7f0c66ffd640 1 -- 172.21.15.154:0/3507276403 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0c6c089840 con 0x7f0c5c0047c0 2023-09-23T03:19:25.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.611+0000 7f0c75a87640 1 --2- 172.21.15.154:0/3507276403 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0c5406af30 0x7f0c5406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:25.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.611+0000 7f0c75a87640 1 --2- 172.21.15.154:0/3507276403 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0c5406af30 0x7f0c5406d3f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0c60036000 tx=0x7f0c6002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:25.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.614+0000 7f0c66ffd640 1 -- 172.21.15.154:0/3507276403 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f0c6c054750 con 0x7f0c5c0047c0 2023-09-23T03:19:25.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:25.839+0000 7f0c784ec640 1 -- 172.21.15.154:0/3507276403 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f0c700b4220 con 0x7f0c5c0047c0 2023-09-23T03:19:26.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.518+0000 7f0c66ffd640 1 -- 172.21.15.154:0/3507276403 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v39) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0c6c001230 con 0x7f0c5c0047c0 2023-09-23T03:19:26.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.521+0000 7f0c784ec640 1 -- 172.21.15.154:0/3507276403 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0c5406af30 msgr2=0x7f0c5406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:26.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.521+0000 7f0c784ec640 1 --2- 172.21.15.154:0/3507276403 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0c5406af30 0x7f0c5406d3f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0c60036000 tx=0x7f0c6002e000 comp rx=0 tx=0).stop 2023-09-23T03:19:26.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.521+0000 7f0c784ec640 1 -- 172.21.15.154:0/3507276403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c5c0047c0 msgr2=0x7f0c700b3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:26.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.521+0000 7f0c784ec640 1 --2- 172.21.15.154:0/3507276403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c5c0047c0 0x7f0c700b3980 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c000f60 tx=0x7f0c6c00d9f0 comp rx=0 tx=0).stop 2023-09-23T03:19:26.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.522+0000 7f0c784ec640 1 -- 172.21.15.154:0/3507276403 shutdown_connections 2023-09-23T03:19:26.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.522+0000 7f0c784ec640 1 --2- 172.21.15.154:0/3507276403 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0c5406af30 0x7f0c5406d3f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:26.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.522+0000 7f0c784ec640 1 --2- 172.21.15.154:0/3507276403 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c701409b0 0x7f0c700c7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:26.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.522+0000 7f0c784ec640 1 --2- 172.21.15.154:0/3507276403 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c700b23a0 0x7f0c70150040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:26.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.522+0000 7f0c784ec640 1 --2- 172.21.15.154:0/3507276403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c5c0047c0 0x7f0c700b3980 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:26.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.522+0000 7f0c784ec640 1 -- 172.21.15.154:0/3507276403 >> 172.21.15.154:0/3507276403 conn(0x7f0c7006c010 msgr2=0x7f0c70159db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:26.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.522+0000 7f0c784ec640 1 -- 172.21.15.154:0/3507276403 shutdown_connections 2023-09-23T03:19:26.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.522+0000 7f0c784ec640 1 -- 172.21.15.154:0/3507276403 wait complete. 2023-09-23T03:19:26.544 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-09-23T03:19:26.544 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-09-23T03:19:26.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.761+0000 7f18860df640 1 Processor -- start 2023-09-23T03:19:26.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.762+0000 7f18860df640 1 -- start start 2023-09-23T03:19:26.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.763+0000 7f18860df640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1880158480 0x7f18800cade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:26.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.763+0000 7f18860df640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18800cb6c0 con 0x7f18800705a0 2023-09-23T03:19:26.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.763+0000 7f18860df640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f188015af70 con 0x7f188015ba50 2023-09-23T03:19:26.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.763+0000 7f18860df640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f188015ad80 con 0x7f1880158480 2023-09-23T03:19:26.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.764+0000 7f18858de640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f18800705a0 0x7f18800cc2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:34770/0 (socket says 172.21.15.154:34770) 2023-09-23T03:19:26.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.764+0000 7f18858de640 1 -- 172.21.15.154:0/3886626472 learned_addr learned my addr 172.21.15.154:0/3886626472 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:19:26.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.764+0000 7f18850dd640 1 --2- 172.21.15.154:0/3886626472 >> v2:172.21.15.190:3300/0 conn(0x7f1880158480 0x7f18800cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:26.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.764+0000 7f1877fff640 1 -- 172.21.15.154:0/3886626472 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 479969226 0 0) 0x7f18800cb6c0 con 0x7f18800705a0 2023-09-23T03:19:26.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.765+0000 7f1877fff640 1 -- 172.21.15.154:0/3886626472 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f186c003520 con 0x7f18800705a0 2023-09-23T03:19:26.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.765+0000 7f18850dd640 1 -- 172.21.15.154:0/3886626472 >> v1:172.21.15.190:6789/0 conn(0x7f188015ba50 legacy=0x7f188006fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:26.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.765+0000 7f18850dd640 1 -- 172.21.15.154:0/3886626472 >> v1:172.21.15.154:6789/0 conn(0x7f18800705a0 legacy=0x7f18800cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:26.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.765+0000 7f1877fff640 1 -- 172.21.15.154:0/3886626472 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 417631809 0 0) 0x7f186c003520 con 0x7f18800705a0 2023-09-23T03:19:26.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.765+0000 7f18850dd640 1 -- 172.21.15.154:0/3886626472 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188015b250 con 0x7f1880158480 2023-09-23T03:19:26.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.766+0000 7f18850dd640 1 --2- 172.21.15.154:0/3886626472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1880158480 0x7f18800cade0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f187c009d90 tx=0x7f187c02be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d68b89d46be1017c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:26.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.767+0000 7f1877fff640 1 -- 172.21.15.154:0/3886626472 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1878001670 con 0x7f1880158480 2023-09-23T03:19:26.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.767+0000 7f1877fff640 1 -- 172.21.15.154:0/3886626472 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f187c02c940 con 0x7f1880158480 2023-09-23T03:19:26.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.767+0000 7f1877fff640 1 -- 172.21.15.154:0/3886626472 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187c02ccd0 con 0x7f1880158480 2023-09-23T03:19:26.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.767+0000 7f18860df640 1 -- 172.21.15.154:0/3886626472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1880158480 msgr2=0x7f18800cade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:26.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.767+0000 7f18860df640 1 --2- 172.21.15.154:0/3886626472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1880158480 0x7f18800cade0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f187c009d90 tx=0x7f187c02be10 comp rx=0 tx=0).stop 2023-09-23T03:19:26.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.768+0000 7f18860df640 1 -- 172.21.15.154:0/3886626472 shutdown_connections 2023-09-23T03:19:26.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.768+0000 7f18860df640 1 --2- 172.21.15.154:0/3886626472 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1880158480 0x7f18800cade0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:26.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.768+0000 7f18860df640 1 -- 172.21.15.154:0/3886626472 >> 172.21.15.154:0/3886626472 conn(0x7f18800aff20 msgr2=0x7f18800b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:26.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.768+0000 7f18860df640 1 -- 172.21.15.154:0/3886626472 shutdown_connections 2023-09-23T03:19:26.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.768+0000 7f18860df640 1 -- 172.21.15.154:0/3886626472 wait complete. 2023-09-23T03:19:26.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.769+0000 7f18860df640 1 Processor -- start 2023-09-23T03:19:26.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.770+0000 7f18860df640 1 -- start start 2023-09-23T03:19:26.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.771+0000 7f18860df640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18800705a0 0x7f18800c4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:26.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.771+0000 7f18860df640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1880158480 0x7f1880152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:26.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.771+0000 7f18860df640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f188015ba50 0x7f18800aeff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:26.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.771+0000 7f18860df640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18800c3120 con 0x7f18800705a0 2023-09-23T03:19:26.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.771+0000 7f18860df640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18800c3ca0 con 0x7f1880158480 2023-09-23T03:19:26.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.771+0000 7f18860df640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18800c3370 con 0x7f188015ba50 2023-09-23T03:19:26.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.771+0000 7f18850dd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18800705a0 0x7f18800c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:26.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.771+0000 7f18850dd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18800705a0 0x7f18800c4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:42964/0 (socket says 172.21.15.154:42964) 2023-09-23T03:19:26.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.771+0000 7f18850dd640 1 -- 172.21.15.154:0/3302014096 learned_addr learned my addr 172.21.15.154:0/3302014096 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:26.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.771+0000 7f18858de640 1 --2- 172.21.15.154:0/3302014096 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f188015ba50 0x7f18800aeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:26.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.771+0000 7f18850dd640 1 -- 172.21.15.154:0/3302014096 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f188015ba50 msgr2=0x7f18800aeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:26.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.771+0000 7f18850dd640 1 --2- 172.21.15.154:0/3302014096 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f188015ba50 0x7f18800aeff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:26.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.771+0000 7f18848dc640 1 --2- 172.21.15.154:0/3302014096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1880158480 0x7f1880152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:26.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.771+0000 7f18850dd640 1 -- 172.21.15.154:0/3302014096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1880158480 msgr2=0x7f1880152ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:26.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.771+0000 7f18850dd640 1 --2- 172.21.15.154:0/3302014096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1880158480 0x7f1880152ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:26.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.771+0000 7f18850dd640 1 -- 172.21.15.154:0/3302014096 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f187c009a70 con 0x7f18800705a0 2023-09-23T03:19:26.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.771+0000 7f18848dc640 1 --2- 172.21.15.154:0/3302014096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1880158480 0x7f1880152ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:26.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.772+0000 7f18850dd640 1 --2- 172.21.15.154:0/3302014096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18800705a0 0x7f18800c4130 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f187c037040 tx=0x7f187c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:26.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.772+0000 7f1875ffb640 1 -- 172.21.15.154:0/3302014096 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187c009450 con 0x7f18800705a0 2023-09-23T03:19:26.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.772+0000 7f1875ffb640 1 -- 172.21.15.154:0/3302014096 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f187c003990 con 0x7f18800705a0 2023-09-23T03:19:26.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.772+0000 7f1875ffb640 1 -- 172.21.15.154:0/3302014096 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187c003d40 con 0x7f18800705a0 2023-09-23T03:19:26.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.772+0000 7f18860df640 1 -- 172.21.15.154:0/3302014096 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18800c3650 con 0x7f18800705a0 2023-09-23T03:19:26.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.772+0000 7f18860df640 1 -- 172.21.15.154:0/3302014096 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1880159940 con 0x7f18800705a0 2023-09-23T03:19:26.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.773+0000 7f18860df640 1 -- 172.21.15.154:0/3302014096 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f188004e920 con 0x7f18800705a0 2023-09-23T03:19:26.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.774+0000 7f1875ffb640 1 -- 172.21.15.154:0/3302014096 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f187c03b050 con 0x7f18800705a0 2023-09-23T03:19:26.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.775+0000 7f1875ffb640 1 --2- 172.21.15.154:0/3302014096 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f186406af30 0x7f186406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:26.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.775+0000 7f1875ffb640 1 -- 172.21.15.154:0/3302014096 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f187c0abee0 con 0x7f18800705a0 2023-09-23T03:19:26.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.776+0000 7f18848dc640 1 --2- 172.21.15.154:0/3302014096 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f186406af30 0x7f186406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:26.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.776+0000 7f18848dc640 1 --2- 172.21.15.154:0/3302014096 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f186406af30 0x7f186406d3f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f188006c9a0 tx=0x7f1870009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:26.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:26.780+0000 7f1875ffb640 1 -- 172.21.15.154:0/3302014096 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f187c076f00 con 0x7f18800705a0 2023-09-23T03:19:27.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.009+0000 7f18860df640 1 -- 172.21.15.154:0/3302014096 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f1880159ad0 con 0x7f18800705a0 2023-09-23T03:19:27.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.540+0000 7f1875ffb640 1 -- 172.21.15.154:0/3302014096 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v40) v1 ==== 122+0+0 (secure 0 0 0) 0x7f187c04d0c0 con 0x7f18800705a0 2023-09-23T03:19:27.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.544+0000 7f18860df640 1 -- 172.21.15.154:0/3302014096 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f186406af30 msgr2=0x7f186406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:27.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.544+0000 7f18860df640 1 --2- 172.21.15.154:0/3302014096 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f186406af30 0x7f186406d3f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f188006c9a0 tx=0x7f1870009040 comp rx=0 tx=0).stop 2023-09-23T03:19:27.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.544+0000 7f18860df640 1 -- 172.21.15.154:0/3302014096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18800705a0 msgr2=0x7f18800c4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:27.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.544+0000 7f18860df640 1 --2- 172.21.15.154:0/3302014096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18800705a0 0x7f18800c4130 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f187c037040 tx=0x7f187c004a40 comp rx=0 tx=0).stop 2023-09-23T03:19:27.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.544+0000 7f18860df640 1 -- 172.21.15.154:0/3302014096 shutdown_connections 2023-09-23T03:19:27.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.544+0000 7f18860df640 1 --2- 172.21.15.154:0/3302014096 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f186406af30 0x7f186406d3f0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:27.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.544+0000 7f18860df640 1 --2- 172.21.15.154:0/3302014096 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f188015ba50 0x7f18800aeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:27.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.544+0000 7f18860df640 1 --2- 172.21.15.154:0/3302014096 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1880158480 0x7f1880152ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:27.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.544+0000 7f18860df640 1 --2- 172.21.15.154:0/3302014096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f18800705a0 0x7f18800c4130 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:27.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.544+0000 7f18860df640 1 -- 172.21.15.154:0/3302014096 >> 172.21.15.154:0/3302014096 conn(0x7f18800aff20 msgr2=0x7f188015d060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:27.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.544+0000 7f18860df640 1 -- 172.21.15.154:0/3302014096 shutdown_connections 2023-09-23T03:19:27.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.544+0000 7f18860df640 1 -- 172.21.15.154:0/3302014096 wait complete. 2023-09-23T03:19:27.565 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-09-23T03:19:27.565 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:19:27.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.774+0000 7fc27878e640 1 Processor -- start 2023-09-23T03:19:27.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.774+0000 7fc27878e640 1 -- start start 2023-09-23T03:19:27.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.775+0000 7fc27878e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc27015caf0 0x7fc2700ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:27.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.776+0000 7fc27878e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2700cee70 0x7fc270160880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:27.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.776+0000 7fc27878e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2700cd530 con 0x7fc270160dc0 2023-09-23T03:19:27.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.776+0000 7fc27878e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc27015ced0 con 0x7fc27015caf0 2023-09-23T03:19:27.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.776+0000 7fc27878e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2700cf2f0 con 0x7fc2700cee70 2023-09-23T03:19:27.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.776+0000 7fc27652a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc27015caf0 0x7fc2700ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:27.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.776+0000 7fc27652a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc27015caf0 0x7fc2700ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:42978/0 (socket says 172.21.15.154:42978) 2023-09-23T03:19:27.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.776+0000 7fc27652a640 1 -- 172.21.15.154:0/3937419240 learned_addr learned my addr 172.21.15.154:0/3937419240 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:27.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.776+0000 7fc276d2b640 1 --1- 172.21.15.154:0/3937419240 >> v1:172.21.15.154:6789/0 conn(0x7fc270160dc0 0x7fc27006cff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:34782/0 (socket says 172.21.15.154:34782) 2023-09-23T03:19:27.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.776+0000 7fc275d29640 1 --2- 172.21.15.154:0/3937419240 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2700cee70 0x7fc270160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:27.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.777+0000 7fc27652a640 1 -- 172.21.15.154:0/3937419240 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2700cee70 msgr2=0x7fc270160880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:27.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.777+0000 7fc27652a640 1 --2- 172.21.15.154:0/3937419240 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2700cee70 0x7fc270160880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:27.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.777+0000 7fc27652a640 1 -- 172.21.15.154:0/3937419240 >> v1:172.21.15.154:6789/0 conn(0x7fc270160dc0 legacy=0x7fc27006cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:27.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.777+0000 7fc27652a640 1 -- 172.21.15.154:0/3937419240 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2700cd810 con 0x7fc27015caf0 2023-09-23T03:19:27.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.777+0000 7fc275d29640 1 --2- 172.21.15.154:0/3937419240 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2700cee70 0x7fc270160880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:19:27.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.777+0000 7fc275528640 1 -- 172.21.15.154:0/3937419240 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 210419634 0 0) 0x7fc2700cd530 con 0x7fc270160dc0 2023-09-23T03:19:27.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.778+0000 7fc27652a640 1 --2- 172.21.15.154:0/3937419240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc27015caf0 0x7fc2700ce930 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc26c009860 tx=0x7fc26c02b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e4330a161507cf1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:27.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.778+0000 7fc275528640 1 -- 172.21.15.154:0/3937419240 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c030830 con 0x7fc27015caf0 2023-09-23T03:19:27.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.778+0000 7fc275528640 1 -- 172.21.15.154:0/3937419240 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc26c030990 con 0x7fc27015caf0 2023-09-23T03:19:27.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.778+0000 7fc275528640 1 -- 172.21.15.154:0/3937419240 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c030d20 con 0x7fc27015caf0 2023-09-23T03:19:27.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.779+0000 7fc27878e640 1 -- 172.21.15.154:0/3937419240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc27015caf0 msgr2=0x7fc2700ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:27.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.779+0000 7fc27878e640 1 --2- 172.21.15.154:0/3937419240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc27015caf0 0x7fc2700ce930 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc26c009860 tx=0x7fc26c02b970 comp rx=0 tx=0).stop 2023-09-23T03:19:27.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.779+0000 7fc27878e640 1 -- 172.21.15.154:0/3937419240 shutdown_connections 2023-09-23T03:19:27.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.779+0000 7fc27878e640 1 --2- 172.21.15.154:0/3937419240 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc2700cee70 0x7fc270160880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:27.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.779+0000 7fc27878e640 1 --2- 172.21.15.154:0/3937419240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc27015caf0 0x7fc2700ce930 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:27.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.779+0000 7fc27878e640 1 -- 172.21.15.154:0/3937419240 >> 172.21.15.154:0/3937419240 conn(0x7fc27006a0f0 msgr2=0x7fc27006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:27.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.779+0000 7fc27878e640 1 -- 172.21.15.154:0/3937419240 shutdown_connections 2023-09-23T03:19:27.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.779+0000 7fc27878e640 1 -- 172.21.15.154:0/3937419240 wait complete. 2023-09-23T03:19:27.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.781+0000 7fc27878e640 1 Processor -- start 2023-09-23T03:19:27.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.781+0000 7fc27878e640 1 -- start start 2023-09-23T03:19:27.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.783+0000 7fc27878e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2700cee70 0x7fc270155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:27.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.783+0000 7fc27878e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc27015caf0 0x7fc27015db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:27.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.783+0000 7fc27878e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc270160dc0 0x7fc27015e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:27.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.783+0000 7fc27878e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc270072420 con 0x7fc270160dc0 2023-09-23T03:19:27.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.783+0000 7fc27878e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27015a4f0 con 0x7fc2700cee70 2023-09-23T03:19:27.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.783+0000 7fc27878e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2701f20f0 con 0x7fc27015caf0 2023-09-23T03:19:27.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.783+0000 7fc276d2b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc270160dc0 0x7fc27015e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:27.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.783+0000 7fc276d2b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc270160dc0 0x7fc27015e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:42992/0 (socket says 172.21.15.154:42992) 2023-09-23T03:19:27.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.783+0000 7fc276d2b640 1 -- 172.21.15.154:0/264232892 learned_addr learned my addr 172.21.15.154:0/264232892 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:27.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.783+0000 7fc275d29640 1 --2- 172.21.15.154:0/264232892 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc27015caf0 0x7fc27015db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:27.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.783+0000 7fc27652a640 1 --2- 172.21.15.154:0/264232892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2700cee70 0x7fc270155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:27.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.784+0000 7fc276d2b640 1 -- 172.21.15.154:0/264232892 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc27015caf0 msgr2=0x7fc27015db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:27.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.784+0000 7fc276d2b640 1 --2- 172.21.15.154:0/264232892 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc27015caf0 0x7fc27015db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:27.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.784+0000 7fc276d2b640 1 -- 172.21.15.154:0/264232892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2700cee70 msgr2=0x7fc270155630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:27.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.784+0000 7fc276d2b640 1 --2- 172.21.15.154:0/264232892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2700cee70 0x7fc270155630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:27.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.784+0000 7fc276d2b640 1 -- 172.21.15.154:0/264232892 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc26c009470 con 0x7fc270160dc0 2023-09-23T03:19:27.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.784+0000 7fc276d2b640 1 --2- 172.21.15.154:0/264232892 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc270160dc0 0x7fc27015e040 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fc26800c730 tx=0x7fc26800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:27.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.784+0000 7fc266ffd640 1 -- 172.21.15.154:0/264232892 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26800ce40 con 0x7fc270160dc0 2023-09-23T03:19:27.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.784+0000 7fc266ffd640 1 -- 172.21.15.154:0/264232892 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc26800a5e0 con 0x7fc270160dc0 2023-09-23T03:19:27.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.784+0000 7fc266ffd640 1 -- 172.21.15.154:0/264232892 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26800a9d0 con 0x7fc270160dc0 2023-09-23T03:19:27.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.784+0000 7fc27878e640 1 -- 172.21.15.154:0/264232892 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc27015ff80 con 0x7fc270160dc0 2023-09-23T03:19:27.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.784+0000 7fc27878e640 1 -- 172.21.15.154:0/264232892 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc27015d910 con 0x7fc270160dc0 2023-09-23T03:19:27.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.785+0000 7fc27878e640 1 -- 172.21.15.154:0/264232892 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc27015a630 con 0x7fc270160dc0 2023-09-23T03:19:27.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.786+0000 7fc266ffd640 1 -- 172.21.15.154:0/264232892 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fc26800ab30 con 0x7fc270160dc0 2023-09-23T03:19:27.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.787+0000 7fc266ffd640 1 --2- 172.21.15.154:0/264232892 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc25406af30 0x7fc25406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:27.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.787+0000 7fc266ffd640 1 -- 172.21.15.154:0/264232892 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc268088d80 con 0x7fc270160dc0 2023-09-23T03:19:27.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.788+0000 7fc27652a640 1 --2- 172.21.15.154:0/264232892 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc25406af30 0x7fc25406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:27.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.788+0000 7fc27652a640 1 --2- 172.21.15.154:0/264232892 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc25406af30 0x7fc25406d3f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc26c036000 tx=0x7fc26c03b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:27.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:27.792+0000 7fc266ffd640 1 -- 172.21.15.154:0/264232892 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fc268053c70 con 0x7fc270160dc0 2023-09-23T03:19:28.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.017+0000 7fc27878e640 1 -- 172.21.15.154:0/264232892 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2700cec90 con 0x7fc270160dc0 2023-09-23T03:19:28.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.019+0000 7fc266ffd640 1 -- 172.21.15.154:0/264232892 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4925 (secure 0 0 0) 0x7fc268018070 con 0x7fc270160dc0 2023-09-23T03:19:28.019 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:19:28.019 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":40,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5027,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2517324433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2517324433},{"type":"v1","addr":"172.21.15.190:6833","nonce":2517324433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":5033,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4098915995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4098915995},{"type":"v1","addr":"172.21.15.190:6835","nonce":4098915995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6309,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/3034894009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":3034894009},{"type":"v1","addr":"172.21.15.154:6836","nonce":3034894009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:19:23.952758+0000","modified":"2023-09-23T03:19:27.010345+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6312":{"gid":6312,"name":"c","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6837/2183023203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":2183023203},{"type":"v1","addr":"172.21.15.154:6837","nonce":2183023203}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-09-23T03:19:28.020 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 40 2023-09-23T03:19:28.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.021+0000 7fc27878e640 1 -- 172.21.15.154:0/264232892 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc25406af30 msgr2=0x7fc25406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:28.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.021+0000 7fc27878e640 1 --2- 172.21.15.154:0/264232892 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc25406af30 0x7fc25406d3f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc26c036000 tx=0x7fc26c03b040 comp rx=0 tx=0).stop 2023-09-23T03:19:28.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.021+0000 7fc27878e640 1 -- 172.21.15.154:0/264232892 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc270160dc0 msgr2=0x7fc27015e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:28.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.021+0000 7fc27878e640 1 --2- 172.21.15.154:0/264232892 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc270160dc0 0x7fc27015e040 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fc26800c730 tx=0x7fc26800cbf0 comp rx=0 tx=0).stop 2023-09-23T03:19:28.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.021+0000 7fc27878e640 1 -- 172.21.15.154:0/264232892 shutdown_connections 2023-09-23T03:19:28.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.021+0000 7fc27878e640 1 --2- 172.21.15.154:0/264232892 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc25406af30 0x7fc25406d3f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:28.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.021+0000 7fc27878e640 1 --2- 172.21.15.154:0/264232892 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc270160dc0 0x7fc27015e040 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:28.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.021+0000 7fc27878e640 1 --2- 172.21.15.154:0/264232892 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc27015caf0 0x7fc27015db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:28.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.021+0000 7fc27878e640 1 --2- 172.21.15.154:0/264232892 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc2700cee70 0x7fc270155630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:28.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.021+0000 7fc27878e640 1 -- 172.21.15.154:0/264232892 >> 172.21.15.154:0/264232892 conn(0x7fc27006a0f0 msgr2=0x7fc2700ac440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:28.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.021+0000 7fc27878e640 1 -- 172.21.15.154:0/264232892 shutdown_connections 2023-09-23T03:19:28.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.021+0000 7fc27878e640 1 -- 172.21.15.154:0/264232892 wait complete. 2023-09-23T03:19:28.038 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:19:28.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.252+0000 7f6b6a59e640 1 Processor -- start 2023-09-23T03:19:28.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.252+0000 7f6b6a59e640 1 -- start start 2023-09-23T03:19:28.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.253+0000 7f6b6a59e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6b640b1b20 0x7f6b640ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:28.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.253+0000 7f6b6a59e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b6415ac30 con 0x7f6b640acce0 2023-09-23T03:19:28.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.253+0000 7f6b6a59e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b640b1f70 con 0x7f6b640cee70 2023-09-23T03:19:28.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.253+0000 7f6b6a59e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b640b2e90 con 0x7f6b640b1b20 2023-09-23T03:19:28.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.254+0000 7f6b63fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f6b640acce0 0x7f6b6415ed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:34790/0 (socket says 172.21.15.154:34790) 2023-09-23T03:19:28.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.254+0000 7f6b63fff640 1 -- 172.21.15.154:0/2678513964 learned_addr learned my addr 172.21.15.154:0/2678513964 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:19:28.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.254+0000 7f6b5b7fe640 1 --2- 172.21.15.154:0/2678513964 >> v2:172.21.15.154:3300/0 conn(0x7f6b640b1b20 0x7f6b640ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:28.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.254+0000 7f6b637fe640 1 -- 172.21.15.154:0/2678513964 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1852191999 0 0) 0x7f6b54001290 con 0x7f6b640acce0 2023-09-23T03:19:28.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.254+0000 7f6b637fe640 1 -- 172.21.15.154:0/2678513964 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b50003520 con 0x7f6b640acce0 2023-09-23T03:19:28.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.254+0000 7f6b5b7fe640 1 -- 172.21.15.154:0/2678513964 >> v1:172.21.15.190:6789/0 conn(0x7f6b640cee70 legacy=0x7f6b640ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:28.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.254+0000 7f6b5b7fe640 1 -- 172.21.15.154:0/2678513964 >> v1:172.21.15.154:6789/0 conn(0x7f6b640acce0 legacy=0x7f6b6415ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:28.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.254+0000 7f6b5b7fe640 1 -- 172.21.15.154:0/2678513964 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b640b2210 con 0x7f6b640b1b20 2023-09-23T03:19:28.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.255+0000 7f6b5b7fe640 1 --2- 172.21.15.154:0/2678513964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b640b1b20 0x7f6b640ce930 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c0097a0 tx=0x7f6b4c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4860dd3c7d3739d4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:28.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.256+0000 7f6b637fe640 1 -- 172.21.15.154:0/2678513964 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c02e070 con 0x7f6b640b1b20 2023-09-23T03:19:28.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.256+0000 7f6b637fe640 1 -- 172.21.15.154:0/2678513964 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b4c0315a0 con 0x7f6b640b1b20 2023-09-23T03:19:28.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.256+0000 7f6b637fe640 1 -- 172.21.15.154:0/2678513964 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c031910 con 0x7f6b640b1b20 2023-09-23T03:19:28.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.256+0000 7f6b6a59e640 1 -- 172.21.15.154:0/2678513964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b640b1b20 msgr2=0x7f6b640ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:28.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.256+0000 7f6b6a59e640 1 --2- 172.21.15.154:0/2678513964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b640b1b20 0x7f6b640ce930 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c0097a0 tx=0x7f6b4c02b840 comp rx=0 tx=0).stop 2023-09-23T03:19:28.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.256+0000 7f6b6a59e640 1 -- 172.21.15.154:0/2678513964 shutdown_connections 2023-09-23T03:19:28.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.256+0000 7f6b6a59e640 1 --2- 172.21.15.154:0/2678513964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b640b1b20 0x7f6b640ce930 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:28.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.256+0000 7f6b6a59e640 1 -- 172.21.15.154:0/2678513964 >> 172.21.15.154:0/2678513964 conn(0x7f6b6406a0f0 msgr2=0x7f6b6406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:28.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.256+0000 7f6b6a59e640 1 -- 172.21.15.154:0/2678513964 shutdown_connections 2023-09-23T03:19:28.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.257+0000 7f6b6a59e640 1 -- 172.21.15.154:0/2678513964 wait complete. 2023-09-23T03:19:28.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.258+0000 7f6b6a59e640 1 Processor -- start 2023-09-23T03:19:28.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.258+0000 7f6b6a59e640 1 -- start start 2023-09-23T03:19:28.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.259+0000 7f6b6a59e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b640acce0 0x7f6b640af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:28.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.259+0000 7f6b6a59e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b640b1b20 0x7f6b640c1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:28.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.259+0000 7f6b6a59e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b640cee70 0x7f6b640c2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:28.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.259+0000 7f6b6a59e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b640cf320 con 0x7f6b640acce0 2023-09-23T03:19:28.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.259+0000 7f6b6a59e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b640af7e0 con 0x7f6b640b1b20 2023-09-23T03:19:28.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.259+0000 7f6b6a59e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6409b970 con 0x7f6b640cee70 2023-09-23T03:19:28.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.260+0000 7f6b63fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b640acce0 0x7f6b640af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:28.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.260+0000 7f6b63fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b640acce0 0x7f6b640af2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43018/0 (socket says 172.21.15.154:43018) 2023-09-23T03:19:28.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.260+0000 7f6b63fff640 1 -- 172.21.15.154:0/2563324796 learned_addr learned my addr 172.21.15.154:0/2563324796 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:28.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.260+0000 7f6b68b3b640 1 --2- 172.21.15.154:0/2563324796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b640cee70 0x7f6b640c2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:28.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.260+0000 7f6b5b7fe640 1 --2- 172.21.15.154:0/2563324796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b640b1b20 0x7f6b640c1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:28.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.260+0000 7f6b63fff640 1 -- 172.21.15.154:0/2563324796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b640cee70 msgr2=0x7f6b640c2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:28.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.260+0000 7f6b63fff640 1 --2- 172.21.15.154:0/2563324796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b640cee70 0x7f6b640c2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:28.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.260+0000 7f6b63fff640 1 -- 172.21.15.154:0/2563324796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b640b1b20 msgr2=0x7f6b640c1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:28.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.260+0000 7f6b63fff640 1 --2- 172.21.15.154:0/2563324796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b640b1b20 0x7f6b640c1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:28.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.260+0000 7f6b63fff640 1 -- 172.21.15.154:0/2563324796 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b4c009400 con 0x7f6b640acce0 2023-09-23T03:19:28.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.260+0000 7f6b68b3b640 1 --2- 172.21.15.154:0/2563324796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b640cee70 0x7f6b640c2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:28.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.260+0000 7f6b63fff640 1 --2- 172.21.15.154:0/2563324796 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b640acce0 0x7f6b640af2a0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400c4d0 tx=0x7f6b5400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:28.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.261+0000 7f6b617fa640 1 -- 172.21.15.154:0/2563324796 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b54007480 con 0x7f6b640acce0 2023-09-23T03:19:28.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.261+0000 7f6b617fa640 1 -- 172.21.15.154:0/2563324796 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b540075e0 con 0x7f6b640acce0 2023-09-23T03:19:28.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.261+0000 7f6b617fa640 1 -- 172.21.15.154:0/2563324796 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b540059e0 con 0x7f6b640acce0 2023-09-23T03:19:28.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.261+0000 7f6b6a59e640 1 -- 172.21.15.154:0/2563324796 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b6409bc50 con 0x7f6b640acce0 2023-09-23T03:19:28.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.261+0000 7f6b6a59e640 1 -- 172.21.15.154:0/2563324796 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b6409c210 con 0x7f6b640acce0 2023-09-23T03:19:28.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.262+0000 7f6b6a59e640 1 -- 172.21.15.154:0/2563324796 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b64073eb0 con 0x7f6b640acce0 2023-09-23T03:19:28.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.263+0000 7f6b617fa640 1 -- 172.21.15.154:0/2563324796 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f6b54005b40 con 0x7f6b640acce0 2023-09-23T03:19:28.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.264+0000 7f6b617fa640 1 --2- 172.21.15.154:0/2563324796 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6b4406af30 0x7f6b4406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:28.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.264+0000 7f6b617fa640 1 -- 172.21.15.154:0/2563324796 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6b54088cf0 con 0x7f6b640acce0 2023-09-23T03:19:28.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.264+0000 7f6b5b7fe640 1 --2- 172.21.15.154:0/2563324796 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6b4406af30 0x7f6b4406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:28.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.265+0000 7f6b5b7fe640 1 --2- 172.21.15.154:0/2563324796 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6b4406af30 0x7f6b4406d3f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c036000 tx=0x7f6b4c03d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:28.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.268+0000 7f6b617fa640 1 -- 172.21.15.154:0/2563324796 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f6b5408d070 con 0x7f6b640acce0 2023-09-23T03:19:28.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.457+0000 7f6b6a59e640 1 -- 172.21.15.154:0/2563324796 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b6415bd90 con 0x7f6b640acce0 2023-09-23T03:19:28.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.460+0000 7f6b617fa640 1 -- 172.21.15.154:0/2563324796 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+16806 (secure 0 0 0) 0x7f6b54017070 con 0x7f6b640acce0 2023-09-23T03:19:28.461 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:19:28.461 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":62,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:19:25.482323+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:19:21.772037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-09-23T03:19:22.960250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:19:28.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.462+0000 7f6b6a59e640 1 -- 172.21.15.154:0/2563324796 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6b4406af30 msgr2=0x7f6b4406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:28.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.462+0000 7f6b6a59e640 1 --2- 172.21.15.154:0/2563324796 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6b4406af30 0x7f6b4406d3f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c036000 tx=0x7f6b4c03d040 comp rx=0 tx=0).stop 2023-09-23T03:19:28.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.463+0000 7f6b6a59e640 1 -- 172.21.15.154:0/2563324796 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b640acce0 msgr2=0x7f6b640af2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:28.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.463+0000 7f6b6a59e640 1 --2- 172.21.15.154:0/2563324796 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b640acce0 0x7f6b640af2a0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400c4d0 tx=0x7f6b5400c990 comp rx=0 tx=0).stop 2023-09-23T03:19:28.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.463+0000 7f6b6a59e640 1 -- 172.21.15.154:0/2563324796 shutdown_connections 2023-09-23T03:19:28.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.463+0000 7f6b6a59e640 1 --2- 172.21.15.154:0/2563324796 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6b4406af30 0x7f6b4406d3f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:28.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.463+0000 7f6b6a59e640 1 --2- 172.21.15.154:0/2563324796 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6b640cee70 0x7f6b640c2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:28.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.463+0000 7f6b6a59e640 1 --2- 172.21.15.154:0/2563324796 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6b640b1b20 0x7f6b640c1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:28.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.463+0000 7f6b6a59e640 1 --2- 172.21.15.154:0/2563324796 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6b640acce0 0x7f6b640af2a0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:28.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.463+0000 7f6b6a59e640 1 -- 172.21.15.154:0/2563324796 >> 172.21.15.154:0/2563324796 conn(0x7f6b6406a0f0 msgr2=0x7f6b6415a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:28.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.463+0000 7f6b6a59e640 1 -- 172.21.15.154:0/2563324796 shutdown_connections 2023-09-23T03:19:28.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.463+0000 7f6b6a59e640 1 -- 172.21.15.154:0/2563324796 wait complete. 2023-09-23T03:19:28.483 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-09-23T03:19:28.483 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-23T03:19:28.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.695+0000 7ffbef17c640 1 Processor -- start 2023-09-23T03:19:28.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.696+0000 7ffbef17c640 1 -- start start 2023-09-23T03:19:28.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.697+0000 7ffbef17c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe80acce0 0x7ffbe815ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:28.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.697+0000 7ffbef17c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ffbe80b1b20 0x7ffbe80ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:28.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.697+0000 7ffbef17c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbe815ac30 con 0x7ffbe80cee70 2023-09-23T03:19:28.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.697+0000 7ffbef17c640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbe80b2e90 con 0x7ffbe80b1b20 2023-09-23T03:19:28.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.697+0000 7ffbef17c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe80cf320 con 0x7ffbe80acce0 2023-09-23T03:19:28.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.697+0000 7ffbed719640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ffbe80cee70 0x7ffbe806cb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:34800/0 (socket says 172.21.15.154:34800) 2023-09-23T03:19:28.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.697+0000 7ffbed719640 1 -- 172.21.15.154:0/506764878 learned_addr learned my addr 172.21.15.154:0/506764878 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:19:28.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.698+0000 7ffbe77fe640 1 -- 172.21.15.154:0/506764878 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1010293138 0 0) 0x7ffbe0001490 con 0x7ffbe80cee70 2023-09-23T03:19:28.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.698+0000 7ffbe7fff640 1 --2- 172.21.15.154:0/506764878 >> v2:172.21.15.190:3300/0 conn(0x7ffbe80b1b20 0x7ffbe80ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:28.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.698+0000 7ffbecf18640 1 --2- 172.21.15.154:0/506764878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe80acce0 0x7ffbe815ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:28.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.698+0000 7ffbe77fe640 1 -- 172.21.15.154:0/506764878 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbd8003520 con 0x7ffbe80cee70 2023-09-23T03:19:28.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.698+0000 7ffbe77fe640 1 -- 172.21.15.154:0/506764878 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 195517490 0 0) 0x7ffbd8003520 con 0x7ffbe80cee70 2023-09-23T03:19:28.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.698+0000 7ffbe77fe640 1 -- 172.21.15.154:0/506764878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe80acce0 msgr2=0x7ffbe815ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:28.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.698+0000 7ffbe77fe640 1 --2- 172.21.15.154:0/506764878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe80acce0 0x7ffbe815ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:28.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.698+0000 7ffbe77fe640 1 -- 172.21.15.154:0/506764878 >> v2:172.21.15.190:3300/0 conn(0x7ffbe80b1b20 msgr2=0x7ffbe80ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:28.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.698+0000 7ffbe77fe640 1 --2- 172.21.15.154:0/506764878 >> v2:172.21.15.190:3300/0 conn(0x7ffbe80b1b20 0x7ffbe80ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:28.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.698+0000 7ffbe77fe640 1 -- 172.21.15.154:0/506764878 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe80b1f70 con 0x7ffbe80cee70 2023-09-23T03:19:28.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.698+0000 7ffbe77fe640 1 -- 172.21.15.154:0/506764878 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7ffbe0001b80 con 0x7ffbe80cee70 2023-09-23T03:19:28.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.698+0000 7ffbe77fe640 1 -- 172.21.15.154:0/506764878 >> v1:172.21.15.154:6789/0 conn(0x7ffbe80cee70 legacy=0x7ffbe806cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:28.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.698+0000 7ffbe77fe640 1 --2- 172.21.15.154:0/506764878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbd8003970 0x7ffbd8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:28.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.698+0000 7ffbe77fe640 1 --2- 172.21.15.154:0/506764878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe80acce0 0x7ffbd80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:28.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.698+0000 7ffbe77fe640 1 --2- 172.21.15.154:0/506764878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbd8004d00 0x7ffbd8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:28.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.698+0000 7ffbe77fe640 1 -- 172.21.15.154:0/506764878 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe80b2e90 con 0x7ffbd8004d00 2023-09-23T03:19:28.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.698+0000 7ffbe77fe640 1 -- 172.21.15.154:0/506764878 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe80cf320 con 0x7ffbd8003970 2023-09-23T03:19:28.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.698+0000 7ffbe77fe640 1 -- 172.21.15.154:0/506764878 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd800ff20 con 0x7ffbe80acce0 2023-09-23T03:19:28.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.699+0000 7ffbed719640 1 --2- 172.21.15.154:0/506764878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbd8004d00 0x7ffbd8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:28.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.699+0000 7ffbe7fff640 1 --2- 172.21.15.154:0/506764878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe80acce0 0x7ffbd80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:28.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.699+0000 7ffbecf18640 1 --2- 172.21.15.154:0/506764878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbd8003970 0x7ffbd8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:28.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.699+0000 7ffbed719640 1 -- 172.21.15.154:0/506764878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe80acce0 msgr2=0x7ffbd80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:28.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.699+0000 7ffbed719640 1 --2- 172.21.15.154:0/506764878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe80acce0 0x7ffbd80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:28.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.699+0000 7ffbed719640 1 -- 172.21.15.154:0/506764878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbd8003970 msgr2=0x7ffbd8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:28.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.699+0000 7ffbed719640 1 --2- 172.21.15.154:0/506764878 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbd8003970 0x7ffbd8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:28.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.699+0000 7ffbed719640 1 -- 172.21.15.154:0/506764878 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd8003520 con 0x7ffbd8004d00 2023-09-23T03:19:28.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.700+0000 7ffbed719640 1 --2- 172.21.15.154:0/506764878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbd8004d00 0x7ffbd8005180 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ffbe000a760 tx=0x7ffbd80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c39d6d321c046b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:28.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.700+0000 7ffbe77fe640 1 -- 172.21.15.154:0/506764878 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe0034630 con 0x7ffbd8004d00 2023-09-23T03:19:28.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.701+0000 7ffbe77fe640 1 -- 172.21.15.154:0/506764878 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbe002f070 con 0x7ffbd8004d00 2023-09-23T03:19:28.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.701+0000 7ffbe77fe640 1 -- 172.21.15.154:0/506764878 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe00349f0 con 0x7ffbd8004d00 2023-09-23T03:19:28.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.701+0000 7ffbef17c640 1 -- 172.21.15.154:0/506764878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbd8004d00 msgr2=0x7ffbd8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:28.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.701+0000 7ffbef17c640 1 --2- 172.21.15.154:0/506764878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbd8004d00 0x7ffbd8005180 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ffbe000a760 tx=0x7ffbd80057f0 comp rx=0 tx=0).stop 2023-09-23T03:19:28.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.701+0000 7ffbed719640 1 -- 172.21.15.154:0/506764878 reap_dead start 2023-09-23T03:19:28.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.701+0000 7ffbef17c640 1 -- 172.21.15.154:0/506764878 shutdown_connections 2023-09-23T03:19:28.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.701+0000 7ffbef17c640 1 -- 172.21.15.154:0/506764878 >> 172.21.15.154:0/506764878 conn(0x7ffbe806a0f0 msgr2=0x7ffbe806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:28.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.701+0000 7ffbef17c640 1 -- 172.21.15.154:0/506764878 shutdown_connections 2023-09-23T03:19:28.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.701+0000 7ffbef17c640 1 -- 172.21.15.154:0/506764878 wait complete. 2023-09-23T03:19:28.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.703+0000 7ffbef17c640 1 Processor -- start 2023-09-23T03:19:28.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.703+0000 7ffbef17c640 1 -- start start 2023-09-23T03:19:28.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.704+0000 7ffbef17c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbd8004d00 0x7ffbe8152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:28.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.704+0000 7ffbef17c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbe80ad5b0 0x7ffbe815add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:28.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.704+0000 7ffbef17c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbe815b310 0x7ffbe8071710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:28.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.704+0000 7ffbef17c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe8071280 con 0x7ffbe80ad5b0 2023-09-23T03:19:28.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.704+0000 7ffbef17c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe8070990 con 0x7ffbe815b310 2023-09-23T03:19:28.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.704+0000 7ffbef17c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe8070c30 con 0x7ffbd8004d00 2023-09-23T03:19:28.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.705+0000 7ffbe7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbe80ad5b0 0x7ffbe815add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:28.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.705+0000 7ffbe7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbe80ad5b0 0x7ffbe815add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43030/0 (socket says 172.21.15.154:43030) 2023-09-23T03:19:28.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.705+0000 7ffbe7fff640 1 -- 172.21.15.154:0/204971984 learned_addr learned my addr 172.21.15.154:0/204971984 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:28.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.705+0000 7ffbecf18640 1 --2- 172.21.15.154:0/204971984 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbd8004d00 0x7ffbe8152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:28.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.705+0000 7ffbe7fff640 1 -- 172.21.15.154:0/204971984 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbd8004d00 msgr2=0x7ffbe8152790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:28.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.705+0000 7ffbed719640 1 --2- 172.21.15.154:0/204971984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbe815b310 0x7ffbe8071710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:28.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.705+0000 7ffbe7fff640 1 --2- 172.21.15.154:0/204971984 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbd8004d00 0x7ffbe8152790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:28.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.705+0000 7ffbe7fff640 1 -- 172.21.15.154:0/204971984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbe815b310 msgr2=0x7ffbe8071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:28.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.705+0000 7ffbe7fff640 1 --2- 172.21.15.154:0/204971984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbe815b310 0x7ffbe8071710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:28.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.705+0000 7ffbe7fff640 1 -- 172.21.15.154:0/204971984 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe8070f10 con 0x7ffbe80ad5b0 2023-09-23T03:19:28.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.705+0000 7ffbecf18640 1 --2- 172.21.15.154:0/204971984 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbd8004d00 0x7ffbe8152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:28.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.705+0000 7ffbe7fff640 1 --2- 172.21.15.154:0/204971984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbe80ad5b0 0x7ffbe815add0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ffbd400f5d0 tx=0x7ffbd400fa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:28.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.706+0000 7ffbe57fa640 1 -- 172.21.15.154:0/204971984 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd4014870 con 0x7ffbe80ad5b0 2023-09-23T03:19:28.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.706+0000 7ffbe57fa640 1 -- 172.21.15.154:0/204971984 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbd40149d0 con 0x7ffbe80ad5b0 2023-09-23T03:19:28.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.706+0000 7ffbe57fa640 1 -- 172.21.15.154:0/204971984 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd4014d40 con 0x7ffbe80ad5b0 2023-09-23T03:19:28.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.706+0000 7ffbef17c640 1 -- 172.21.15.154:0/204971984 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbe80adb20 con 0x7ffbe80ad5b0 2023-09-23T03:19:28.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.706+0000 7ffbef17c640 1 -- 172.21.15.154:0/204971984 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbe80cb250 con 0x7ffbe80ad5b0 2023-09-23T03:19:28.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.707+0000 7ffbef17c640 1 -- 172.21.15.154:0/204971984 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbe80710a0 con 0x7ffbe80ad5b0 2023-09-23T03:19:28.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.708+0000 7ffbe57fa640 1 -- 172.21.15.154:0/204971984 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7ffbd400d460 con 0x7ffbe80ad5b0 2023-09-23T03:19:28.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.709+0000 7ffbe57fa640 1 --2- 172.21.15.154:0/204971984 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffbcc06af30 0x7ffbcc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:28.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.709+0000 7ffbe57fa640 1 -- 172.21.15.154:0/204971984 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ffbd408a320 con 0x7ffbe80ad5b0 2023-09-23T03:19:28.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.709+0000 7ffbecf18640 1 --2- 172.21.15.154:0/204971984 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffbcc06af30 0x7ffbcc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:28.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.710+0000 7ffbecf18640 1 --2- 172.21.15.154:0/204971984 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffbcc06af30 0x7ffbcc06d3f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ffbe80ab410 tx=0x7ffbdc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:28.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.713+0000 7ffbe57fa640 1 -- 172.21.15.154:0/204971984 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7ffbd4055210 con 0x7ffbe80ad5b0 2023-09-23T03:19:28.880 INFO:tasks.ceph.mds.c.smithi154.stderr:2023-09-23T03:19:28.880+0000 7f08227db640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-23T03:19:28.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.899+0000 7ffbef17c640 1 -- 172.21.15.154:0/204971984 --> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffbe80adcb0 con 0x7ffbcc06af30 2023-09-23T03:19:28.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.920+0000 7ffbe57fa640 1 -- 172.21.15.154:0/204971984 <== mgr.4110 v2:172.21.15.154:6832/2422652104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+281110 (secure 0 0 0) 0x7ffbe80adcb0 con 0x7ffbcc06af30 2023-09-23T03:19:28.925 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:19:28.926 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2023-09-23T03:19:28.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.927+0000 7ffbef17c640 1 -- 172.21.15.154:0/204971984 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffbcc06af30 msgr2=0x7ffbcc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:28.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.927+0000 7ffbef17c640 1 --2- 172.21.15.154:0/204971984 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffbcc06af30 0x7ffbcc06d3f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ffbe80ab410 tx=0x7ffbdc008040 comp rx=0 tx=0).stop 2023-09-23T03:19:28.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.927+0000 7ffbef17c640 1 -- 172.21.15.154:0/204971984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbe80ad5b0 msgr2=0x7ffbe815add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:28.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.927+0000 7ffbef17c640 1 --2- 172.21.15.154:0/204971984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbe80ad5b0 0x7ffbe815add0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ffbd400f5d0 tx=0x7ffbd400fa90 comp rx=0 tx=0).stop 2023-09-23T03:19:28.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.927+0000 7ffbef17c640 1 -- 172.21.15.154:0/204971984 shutdown_connections 2023-09-23T03:19:28.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.927+0000 7ffbef17c640 1 --2- 172.21.15.154:0/204971984 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffbcc06af30 0x7ffbcc06d3f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:28.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.927+0000 7ffbef17c640 1 --2- 172.21.15.154:0/204971984 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbe815b310 0x7ffbe8071710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:28.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.927+0000 7ffbef17c640 1 --2- 172.21.15.154:0/204971984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbe80ad5b0 0x7ffbe815add0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:28.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.927+0000 7ffbef17c640 1 --2- 172.21.15.154:0/204971984 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbd8004d00 0x7ffbe8152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:28.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.927+0000 7ffbef17c640 1 -- 172.21.15.154:0/204971984 >> 172.21.15.154:0/204971984 conn(0x7ffbe806a0f0 msgr2=0x7ffbe8157450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:28.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.927+0000 7ffbef17c640 1 -- 172.21.15.154:0/204971984 shutdown_connections 2023-09-23T03:19:28.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:28.927+0000 7ffbef17c640 1 -- 172.21.15.154:0/204971984 wait complete. 2023-09-23T03:19:28.947 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":184,"stamp":"2023-09-23T03:19:27.249192+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235436,"kb_used_data":6764,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754739284,"statfs":{"total":773094113280,"available":772853026816,"internally_reserved":0,"allocated":6926336,"data_stored":3834431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":234016314},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.782088"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.496299+0000","last_change":"2023-09-23T03:19:23.496299+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.496299+0000","last_undegraded":"2023-09-23T03:19:23.496299+0000","last_fullsized":"2023-09-23T03:19:23.496299+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.491358+0000","last_change":"2023-09-23T03:19:23.491358+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.491358+0000","last_undegraded":"2023-09-23T03:19:23.491358+0000","last_fullsized":"2023-09-23T03:19:23.491358+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.499260+0000","last_change":"2023-09-23T03:19:23.499260+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.499260+0000","last_undegraded":"2023-09-23T03:19:23.499260+0000","last_fullsized":"2023-09-23T03:19:23.499260+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.504046+0000","last_change":"2023-09-23T03:19:23.504046+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.504046+0000","last_undegraded":"2023-09-23T03:19:23.504046+0000","last_fullsized":"2023-09-23T03:19:23.504046+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.494815+0000","last_change":"2023-09-23T03:19:23.494815+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.494815+0000","last_undegraded":"2023-09-23T03:19:23.494815+0000","last_fullsized":"2023-09-23T03:19:23.494815+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.489767+0000","last_change":"2023-09-23T03:19:23.489767+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.489767+0000","last_undegraded":"2023-09-23T03:19:23.489767+0000","last_fullsized":"2023-09-23T03:19:23.489767+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.488689+0000","last_change":"2023-09-23T03:19:23.488689+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.488689+0000","last_undegraded":"2023-09-23T03:19:23.488689+0000","last_fullsized":"2023-09-23T03:19:23.488689+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.490760+0000","last_change":"2023-09-23T03:19:23.490760+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.490760+0000","last_undegraded":"2023-09-23T03:19:23.490760+0000","last_fullsized":"2023-09-23T03:19:23.490760+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.489816+0000","last_change":"2023-09-23T03:19:23.489816+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.489816+0000","last_undegraded":"2023-09-23T03:19:23.489816+0000","last_fullsized":"2023-09-23T03:19:23.489816+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.500805+0000","last_change":"2023-09-23T03:19:23.500805+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.500805+0000","last_undegraded":"2023-09-23T03:19:23.500805+0000","last_fullsized":"2023-09-23T03:19:23.500805+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.501343+0000","last_change":"2023-09-23T03:19:23.501343+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.501343+0000","last_undegraded":"2023-09-23T03:19:23.501343+0000","last_fullsized":"2023-09-23T03:19:23.501343+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.503530+0000","last_change":"2023-09-23T03:19:23.503530+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.503530+0000","last_undegraded":"2023-09-23T03:19:23.503530+0000","last_fullsized":"2023-09-23T03:19:23.503530+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.495939+0000","last_change":"2023-09-23T03:19:23.495939+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.495939+0000","last_undegraded":"2023-09-23T03:19:23.495939+0000","last_fullsized":"2023-09-23T03:19:23.495939+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.488295+0000","last_change":"2023-09-23T03:19:23.488295+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.488295+0000","last_undegraded":"2023-09-23T03:19:23.488295+0000","last_fullsized":"2023-09-23T03:19:23.488295+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.487204+0000","last_change":"2023-09-23T03:19:23.487204+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.487204+0000","last_undegraded":"2023-09-23T03:19:23.487204+0000","last_fullsized":"2023-09-23T03:19:23.487204+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.489319+0000","last_change":"2023-09-23T03:19:23.489319+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.489319+0000","last_undegraded":"2023-09-23T03:19:23.489319+0000","last_fullsized":"2023-09-23T03:19:23.489319+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.490227+0000","last_change":"2023-09-23T03:19:23.490227+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.490227+0000","last_undegraded":"2023-09-23T03:19:23.490227+0000","last_fullsized":"2023-09-23T03:19:23.490227+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.500342+0000","last_change":"2023-09-23T03:19:23.500342+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.500342+0000","last_undegraded":"2023-09-23T03:19:23.500342+0000","last_fullsized":"2023-09-23T03:19:23.500342+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.494436+0000","last_change":"2023-09-23T03:19:23.494436+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.494436+0000","last_undegraded":"2023-09-23T03:19:23.494436+0000","last_fullsized":"2023-09-23T03:19:23.494436+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.489696+0000","last_change":"2023-09-23T03:19:23.489696+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.489696+0000","last_undegraded":"2023-09-23T03:19:23.489696+0000","last_fullsized":"2023-09-23T03:19:23.489696+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.500116+0000","last_change":"2023-09-23T03:19:23.500116+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.500116+0000","last_undegraded":"2023-09-23T03:19:23.500116+0000","last_fullsized":"2023-09-23T03:19:23.500116+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.489656+0000","last_change":"2023-09-23T03:19:23.489656+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.489656+0000","last_undegraded":"2023-09-23T03:19:23.489656+0000","last_fullsized":"2023-09-23T03:19:23.489656+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.497801+0000","last_change":"2023-09-23T03:19:23.497801+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.497801+0000","last_undegraded":"2023-09-23T03:19:23.497801+0000","last_fullsized":"2023-09-23T03:19:23.497801+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.491559+0000","last_change":"2023-09-23T03:19:23.491559+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.491559+0000","last_undegraded":"2023-09-23T03:19:23.491559+0000","last_fullsized":"2023-09-23T03:19:23.491559+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.495800+0000","last_change":"2023-09-23T03:19:23.495800+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.495800+0000","last_undegraded":"2023-09-23T03:19:23.495800+0000","last_fullsized":"2023-09-23T03:19:23.495800+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.502761+0000","last_change":"2023-09-23T03:19:23.502761+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.502761+0000","last_undegraded":"2023-09-23T03:19:23.502761+0000","last_fullsized":"2023-09-23T03:19:23.502761+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.492766+0000","last_change":"2023-09-23T03:19:23.492766+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.492766+0000","last_undegraded":"2023-09-23T03:19:23.492766+0000","last_fullsized":"2023-09-23T03:19:23.492766+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.494163+0000","last_change":"2023-09-23T03:19:23.494163+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.494163+0000","last_undegraded":"2023-09-23T03:19:23.494163+0000","last_fullsized":"2023-09-23T03:19:23.494163+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.489233+0000","last_change":"2023-09-23T03:19:23.489233+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.489233+0000","last_undegraded":"2023-09-23T03:19:23.489233+0000","last_fullsized":"2023-09-23T03:19:23.489233+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.498098+0000","last_change":"2023-09-23T03:19:23.498098+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.498098+0000","last_undegraded":"2023-09-23T03:19:23.498098+0000","last_fullsized":"2023-09-23T03:19:23.498098+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.494358+0000","last_change":"2023-09-23T03:19:23.494358+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.494358+0000","last_undegraded":"2023-09-23T03:19:23.494358+0000","last_fullsized":"2023-09-23T03:19:23.494358+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.488708+0000","last_change":"2023-09-23T03:19:23.488708+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.488708+0000","last_undegraded":"2023-09-23T03:19:23.488708+0000","last_fullsized":"2023-09-23T03:19:23.488708+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.493052+0000","last_change":"2023-09-23T03:19:23.493052+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.493052+0000","last_undegraded":"2023-09-23T03:19:23.493052+0000","last_fullsized":"2023-09-23T03:19:23.493052+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.518586+0000","last_change":"2023-09-23T03:19:23.514227+0000","last_active":"2023-09-23T03:19:23.518586+0000","last_peered":"2023-09-23T03:19:23.518586+0000","last_clean":"2023-09-23T03:19:23.518586+0000","last_became_active":"2023-09-23T03:19:23.513867+0000","last_became_peered":"2023-09-23T03:19:23.513867+0000","last_unstale":"2023-09-23T03:19:23.518586+0000","last_undegraded":"2023-09-23T03:19:23.518586+0000","last_fullsized":"2023-09-23T03:19:23.518586+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:23:19.447543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.500949+0000","last_change":"2023-09-23T03:19:23.500949+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.500949+0000","last_undegraded":"2023-09-23T03:19:23.500949+0000","last_fullsized":"2023-09-23T03:19:23.500949+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.513306+0000","last_change":"2023-09-23T03:19:23.511815+0000","last_active":"2023-09-23T03:19:23.513306+0000","last_peered":"2023-09-23T03:19:23.513306+0000","last_clean":"2023-09-23T03:19:23.513306+0000","last_became_active":"2023-09-23T03:19:23.510810+0000","last_became_peered":"2023-09-23T03:19:23.510810+0000","last_unstale":"2023-09-23T03:19:23.513306+0000","last_undegraded":"2023-09-23T03:19:23.513306+0000","last_fullsized":"2023-09-23T03:19:23.513306+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:20:39.987490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.491398+0000","last_change":"2023-09-23T03:19:23.491398+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.491398+0000","last_undegraded":"2023-09-23T03:19:23.491398+0000","last_fullsized":"2023-09-23T03:19:23.491398+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.513486+0000","last_change":"2023-09-23T03:19:23.508844+0000","last_active":"2023-09-23T03:19:23.513486+0000","last_peered":"2023-09-23T03:19:23.513486+0000","last_clean":"2023-09-23T03:19:23.513486+0000","last_became_active":"2023-09-23T03:19:23.508557+0000","last_became_peered":"2023-09-23T03:19:23.508557+0000","last_unstale":"2023-09-23T03:19:23.513486+0000","last_undegraded":"2023-09-23T03:19:23.513486+0000","last_fullsized":"2023-09-23T03:19:23.513486+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:49:32.205787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.492304+0000","last_change":"2023-09-23T03:19:23.492304+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.492304+0000","last_undegraded":"2023-09-23T03:19:23.492304+0000","last_fullsized":"2023-09-23T03:19:23.492304+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.524534+0000","last_change":"2023-09-23T03:19:23.514514+0000","last_active":"2023-09-23T03:19:23.524534+0000","last_peered":"2023-09-23T03:19:23.524534+0000","last_clean":"2023-09-23T03:19:23.524534+0000","last_became_active":"2023-09-23T03:19:23.514219+0000","last_became_peered":"2023-09-23T03:19:23.514219+0000","last_unstale":"2023-09-23T03:19:23.524534+0000","last_undegraded":"2023-09-23T03:19:23.524534+0000","last_fullsized":"2023-09-23T03:19:23.524534+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:45:54.723198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.488324+0000","last_change":"2023-09-23T03:19:23.488324+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.488324+0000","last_undegraded":"2023-09-23T03:19:23.488324+0000","last_fullsized":"2023-09-23T03:19:23.488324+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.515178+0000","last_change":"2023-09-23T03:19:23.509524+0000","last_active":"2023-09-23T03:19:23.515178+0000","last_peered":"2023-09-23T03:19:23.515178+0000","last_clean":"2023-09-23T03:19:23.515178+0000","last_became_active":"2023-09-23T03:19:23.509229+0000","last_became_peered":"2023-09-23T03:19:23.509229+0000","last_unstale":"2023-09-23T03:19:23.515178+0000","last_undegraded":"2023-09-23T03:19:23.515178+0000","last_fullsized":"2023-09-23T03:19:23.515178+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:58:25.968168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.488600+0000","last_change":"2023-09-23T03:19:23.488600+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.488600+0000","last_undegraded":"2023-09-23T03:19:23.488600+0000","last_fullsized":"2023-09-23T03:19:23.488600+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.508057+0000","last_change":"2023-09-23T03:19:23.505508+0000","last_active":"2023-09-23T03:19:23.508057+0000","last_peered":"2023-09-23T03:19:23.508057+0000","last_clean":"2023-09-23T03:19:23.508057+0000","last_became_active":"2023-09-23T03:19:23.505249+0000","last_became_peered":"2023-09-23T03:19:23.505249+0000","last_unstale":"2023-09-23T03:19:23.508057+0000","last_undegraded":"2023-09-23T03:19:23.508057+0000","last_fullsized":"2023-09-23T03:19:23.508057+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:45:54.723198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.490303+0000","last_change":"2023-09-23T03:19:23.490303+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.490303+0000","last_undegraded":"2023-09-23T03:19:23.490303+0000","last_fullsized":"2023-09-23T03:19:23.490303+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.524401+0000","last_change":"2023-09-23T03:19:23.514127+0000","last_active":"2023-09-23T03:19:23.524401+0000","last_peered":"2023-09-23T03:19:23.524401+0000","last_clean":"2023-09-23T03:19:23.524401+0000","last_became_active":"2023-09-23T03:19:23.513630+0000","last_became_peered":"2023-09-23T03:19:23.513630+0000","last_unstale":"2023-09-23T03:19:23.524401+0000","last_undegraded":"2023-09-23T03:19:23.524401+0000","last_fullsized":"2023-09-23T03:19:23.524401+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:33:16.276149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.487896+0000","last_change":"2023-09-23T03:19:23.487896+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.487896+0000","last_undegraded":"2023-09-23T03:19:23.487896+0000","last_fullsized":"2023-09-23T03:19:23.487896+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.502692+0000","last_change":"2023-09-23T03:19:23.500111+0000","last_active":"2023-09-23T03:19:23.502692+0000","last_peered":"2023-09-23T03:19:23.502692+0000","last_clean":"2023-09-23T03:19:23.502692+0000","last_became_active":"2023-09-23T03:19:23.499792+0000","last_became_peered":"2023-09-23T03:19:23.499792+0000","last_unstale":"2023-09-23T03:19:23.502692+0000","last_undegraded":"2023-09-23T03:19:23.502692+0000","last_fullsized":"2023-09-23T03:19:23.502692+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:20:30.730712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.487758+0000","last_change":"2023-09-23T03:19:23.487758+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.487758+0000","last_undegraded":"2023-09-23T03:19:23.487758+0000","last_fullsized":"2023-09-23T03:19:23.487758+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.516664+0000","last_change":"2023-09-23T03:19:23.514447+0000","last_active":"2023-09-23T03:19:23.516664+0000","last_peered":"2023-09-23T03:19:23.516664+0000","last_clean":"2023-09-23T03:19:23.516664+0000","last_became_active":"2023-09-23T03:19:23.514067+0000","last_became_peered":"2023-09-23T03:19:23.514067+0000","last_unstale":"2023-09-23T03:19:23.516664+0000","last_undegraded":"2023-09-23T03:19:23.516664+0000","last_fullsized":"2023-09-23T03:19:23.516664+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:00:17.646758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.496971+0000","last_change":"2023-09-23T03:19:23.496971+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.496971+0000","last_undegraded":"2023-09-23T03:19:23.496971+0000","last_fullsized":"2023-09-23T03:19:23.496971+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.507007+0000","last_change":"2023-09-23T03:19:23.502457+0000","last_active":"2023-09-23T03:19:23.507007+0000","last_peered":"2023-09-23T03:19:23.507007+0000","last_clean":"2023-09-23T03:19:23.507007+0000","last_became_active":"2023-09-23T03:19:23.502185+0000","last_became_peered":"2023-09-23T03:19:23.502185+0000","last_unstale":"2023-09-23T03:19:23.507007+0000","last_undegraded":"2023-09-23T03:19:23.507007+0000","last_fullsized":"2023-09-23T03:19:23.507007+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:00:39.238009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.500995+0000","last_change":"2023-09-23T03:19:23.500995+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.500995+0000","last_undegraded":"2023-09-23T03:19:23.500995+0000","last_fullsized":"2023-09-23T03:19:23.500995+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.516693+0000","last_change":"2023-09-23T03:19:23.514525+0000","last_active":"2023-09-23T03:19:23.516693+0000","last_peered":"2023-09-23T03:19:23.516693+0000","last_clean":"2023-09-23T03:19:23.516693+0000","last_became_active":"2023-09-23T03:19:23.514117+0000","last_became_peered":"2023-09-23T03:19:23.514117+0000","last_unstale":"2023-09-23T03:19:23.516693+0000","last_undegraded":"2023-09-23T03:19:23.516693+0000","last_fullsized":"2023-09-23T03:19:23.516693+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:45:54.723198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.498493+0000","last_change":"2023-09-23T03:19:23.498493+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.498493+0000","last_undegraded":"2023-09-23T03:19:23.498493+0000","last_fullsized":"2023-09-23T03:19:23.498493+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.505498+0000","last_change":"2023-09-23T03:19:23.501871+0000","last_active":"2023-09-23T03:19:23.505498+0000","last_peered":"2023-09-23T03:19:23.505498+0000","last_clean":"2023-09-23T03:19:23.505498+0000","last_became_active":"2023-09-23T03:19:23.501537+0000","last_became_peered":"2023-09-23T03:19:23.501537+0000","last_unstale":"2023-09-23T03:19:23.505498+0000","last_undegraded":"2023-09-23T03:19:23.505498+0000","last_fullsized":"2023-09-23T03:19:23.505498+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:25:06.270537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.501561+0000","last_change":"2023-09-23T03:19:23.501561+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.501561+0000","last_undegraded":"2023-09-23T03:19:23.501561+0000","last_fullsized":"2023-09-23T03:19:23.501561+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.511665+0000","last_change":"2023-09-23T03:19:23.501583+0000","last_active":"2023-09-23T03:19:23.511665+0000","last_peered":"2023-09-23T03:19:23.511665+0000","last_clean":"2023-09-23T03:19:23.511665+0000","last_became_active":"2023-09-23T03:19:23.501037+0000","last_became_peered":"2023-09-23T03:19:23.501037+0000","last_unstale":"2023-09-23T03:19:23.511665+0000","last_undegraded":"2023-09-23T03:19:23.511665+0000","last_fullsized":"2023-09-23T03:19:23.511665+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:11:18.322404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.499265+0000","last_change":"2023-09-23T03:19:23.499265+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.499265+0000","last_undegraded":"2023-09-23T03:19:23.499265+0000","last_fullsized":"2023-09-23T03:19:23.499265+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.512892+0000","last_change":"2023-09-23T03:19:23.507592+0000","last_active":"2023-09-23T03:19:23.512892+0000","last_peered":"2023-09-23T03:19:23.512892+0000","last_clean":"2023-09-23T03:19:23.512892+0000","last_became_active":"2023-09-23T03:19:23.507316+0000","last_became_peered":"2023-09-23T03:19:23.507316+0000","last_unstale":"2023-09-23T03:19:23.512892+0000","last_undegraded":"2023-09-23T03:19:23.512892+0000","last_fullsized":"2023-09-23T03:19:23.512892+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:43:34.054466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.487374+0000","last_change":"2023-09-23T03:19:23.487374+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.487374+0000","last_undegraded":"2023-09-23T03:19:23.487374+0000","last_fullsized":"2023-09-23T03:19:23.487374+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.524603+0000","last_change":"2023-09-23T03:19:23.515237+0000","last_active":"2023-09-23T03:19:23.524603+0000","last_peered":"2023-09-23T03:19:23.524603+0000","last_clean":"2023-09-23T03:19:23.524603+0000","last_became_active":"2023-09-23T03:19:23.514946+0000","last_became_peered":"2023-09-23T03:19:23.514946+0000","last_unstale":"2023-09-23T03:19:23.524603+0000","last_undegraded":"2023-09-23T03:19:23.524603+0000","last_fullsized":"2023-09-23T03:19:23.524603+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:49:32.205787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.497140+0000","last_change":"2023-09-23T03:19:23.497140+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.497140+0000","last_undegraded":"2023-09-23T03:19:23.497140+0000","last_fullsized":"2023-09-23T03:19:23.497140+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.524349+0000","last_change":"2023-09-23T03:19:23.519665+0000","last_active":"2023-09-23T03:19:23.524349+0000","last_peered":"2023-09-23T03:19:23.524349+0000","last_clean":"2023-09-23T03:19:23.524349+0000","last_became_active":"2023-09-23T03:19:23.519393+0000","last_became_peered":"2023-09-23T03:19:23.519393+0000","last_unstale":"2023-09-23T03:19:23.524349+0000","last_undegraded":"2023-09-23T03:19:23.524349+0000","last_fullsized":"2023-09-23T03:19:23.524349+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:49:32.205787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.499552+0000","last_change":"2023-09-23T03:19:23.499552+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.499552+0000","last_undegraded":"2023-09-23T03:19:23.499552+0000","last_fullsized":"2023-09-23T03:19:23.499552+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.507040+0000","last_change":"2023-09-23T03:19:23.502992+0000","last_active":"2023-09-23T03:19:23.507040+0000","last_peered":"2023-09-23T03:19:23.507040+0000","last_clean":"2023-09-23T03:19:23.507040+0000","last_became_active":"2023-09-23T03:19:23.502688+0000","last_became_peered":"2023-09-23T03:19:23.502688+0000","last_unstale":"2023-09-23T03:19:23.507040+0000","last_undegraded":"2023-09-23T03:19:23.507040+0000","last_fullsized":"2023-09-23T03:19:23.507040+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:05:11.471065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.503106+0000","last_change":"2023-09-23T03:19:23.503106+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.503106+0000","last_undegraded":"2023-09-23T03:19:23.503106+0000","last_fullsized":"2023-09-23T03:19:23.503106+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.511859+0000","last_change":"2023-09-23T03:19:23.507612+0000","last_active":"2023-09-23T03:19:23.511859+0000","last_peered":"2023-09-23T03:19:23.511859+0000","last_clean":"2023-09-23T03:19:23.511859+0000","last_became_active":"2023-09-23T03:19:23.507211+0000","last_became_peered":"2023-09-23T03:19:23.507211+0000","last_unstale":"2023-09-23T03:19:23.511859+0000","last_undegraded":"2023-09-23T03:19:23.511859+0000","last_fullsized":"2023-09-23T03:19:23.511859+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:33:16.276149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.487706+0000","last_change":"2023-09-23T03:19:23.487706+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.487706+0000","last_undegraded":"2023-09-23T03:19:23.487706+0000","last_fullsized":"2023-09-23T03:19:23.487706+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.525158+0000","last_change":"2023-09-23T03:19:23.518829+0000","last_active":"2023-09-23T03:19:23.525158+0000","last_peered":"2023-09-23T03:19:23.525158+0000","last_clean":"2023-09-23T03:19:23.525158+0000","last_became_active":"2023-09-23T03:19:23.518512+0000","last_became_peered":"2023-09-23T03:19:23.518512+0000","last_unstale":"2023-09-23T03:19:23.525158+0000","last_undegraded":"2023-09-23T03:19:23.525158+0000","last_fullsized":"2023-09-23T03:19:23.525158+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:23:19.447543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.497713+0000","last_change":"2023-09-23T03:19:23.497713+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.497713+0000","last_undegraded":"2023-09-23T03:19:23.497713+0000","last_fullsized":"2023-09-23T03:19:23.497713+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.508037+0000","last_change":"2023-09-23T03:19:23.500147+0000","last_active":"2023-09-23T03:19:23.508037+0000","last_peered":"2023-09-23T03:19:23.508037+0000","last_clean":"2023-09-23T03:19:23.508037+0000","last_became_active":"2023-09-23T03:19:23.499826+0000","last_became_peered":"2023-09-23T03:19:23.499826+0000","last_unstale":"2023-09-23T03:19:23.508037+0000","last_undegraded":"2023-09-23T03:19:23.508037+0000","last_fullsized":"2023-09-23T03:19:23.508037+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:09:18.859490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.500650+0000","last_change":"2023-09-23T03:19:23.500650+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.500650+0000","last_undegraded":"2023-09-23T03:19:23.500650+0000","last_fullsized":"2023-09-23T03:19:23.500650+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.507070+0000","last_change":"2023-09-23T03:19:23.502333+0000","last_active":"2023-09-23T03:19:23.507070+0000","last_peered":"2023-09-23T03:19:23.507070+0000","last_clean":"2023-09-23T03:19:23.507070+0000","last_became_active":"2023-09-23T03:19:23.501997+0000","last_became_peered":"2023-09-23T03:19:23.501997+0000","last_unstale":"2023-09-23T03:19:23.507070+0000","last_undegraded":"2023-09-23T03:19:23.507070+0000","last_fullsized":"2023-09-23T03:19:23.507070+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:22:22.759701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.490693+0000","last_change":"2023-09-23T03:19:23.490693+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.490693+0000","last_undegraded":"2023-09-23T03:19:23.490693+0000","last_fullsized":"2023-09-23T03:19:23.490693+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.512081+0000","last_change":"2023-09-23T03:19:23.507705+0000","last_active":"2023-09-23T03:19:23.512081+0000","last_peered":"2023-09-23T03:19:23.512081+0000","last_clean":"2023-09-23T03:19:23.512081+0000","last_became_active":"2023-09-23T03:19:23.507409+0000","last_became_peered":"2023-09-23T03:19:23.507409+0000","last_unstale":"2023-09-23T03:19:23.512081+0000","last_undegraded":"2023-09-23T03:19:23.512081+0000","last_fullsized":"2023-09-23T03:19:23.512081+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:43:34.054466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.488782+0000","last_change":"2023-09-23T03:19:23.488782+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.488782+0000","last_undegraded":"2023-09-23T03:19:23.488782+0000","last_fullsized":"2023-09-23T03:19:23.488782+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.524549+0000","last_change":"2023-09-23T03:19:23.514960+0000","last_active":"2023-09-23T03:19:23.524549+0000","last_peered":"2023-09-23T03:19:23.524549+0000","last_clean":"2023-09-23T03:19:23.524549+0000","last_became_active":"2023-09-23T03:19:23.514644+0000","last_became_peered":"2023-09-23T03:19:23.514644+0000","last_unstale":"2023-09-23T03:19:23.524549+0000","last_undegraded":"2023-09-23T03:19:23.524549+0000","last_fullsized":"2023-09-23T03:19:23.524549+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:00:17.646758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.502007+0000","last_change":"2023-09-23T03:19:23.502007+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.502007+0000","last_undegraded":"2023-09-23T03:19:23.502007+0000","last_fullsized":"2023-09-23T03:19:23.502007+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.515734+0000","last_change":"2023-09-23T03:19:23.512352+0000","last_active":"2023-09-23T03:19:23.515734+0000","last_peered":"2023-09-23T03:19:23.515734+0000","last_clean":"2023-09-23T03:19:23.515734+0000","last_became_active":"2023-09-23T03:19:23.512039+0000","last_became_peered":"2023-09-23T03:19:23.512039+0000","last_unstale":"2023-09-23T03:19:23.515734+0000","last_undegraded":"2023-09-23T03:19:23.515734+0000","last_fullsized":"2023-09-23T03:19:23.515734+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:43:42.533621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.502444+0000","last_change":"2023-09-23T03:19:23.502444+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.502444+0000","last_undegraded":"2023-09-23T03:19:23.502444+0000","last_fullsized":"2023-09-23T03:19:23.502444+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.511839+0000","last_change":"2023-09-23T03:19:23.506342+0000","last_active":"2023-09-23T03:19:23.511839+0000","last_peered":"2023-09-23T03:19:23.511839+0000","last_clean":"2023-09-23T03:19:23.511839+0000","last_became_active":"2023-09-23T03:19:23.506078+0000","last_became_peered":"2023-09-23T03:19:23.506078+0000","last_unstale":"2023-09-23T03:19:23.511839+0000","last_undegraded":"2023-09-23T03:19:23.511839+0000","last_fullsized":"2023-09-23T03:19:23.511839+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:00:17.646758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.500536+0000","last_change":"2023-09-23T03:19:23.500536+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.500536+0000","last_undegraded":"2023-09-23T03:19:23.500536+0000","last_fullsized":"2023-09-23T03:19:23.500536+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.516019+0000","last_change":"2023-09-23T03:19:23.510497+0000","last_active":"2023-09-23T03:19:23.516019+0000","last_peered":"2023-09-23T03:19:23.516019+0000","last_clean":"2023-09-23T03:19:23.516019+0000","last_became_active":"2023-09-23T03:19:23.510155+0000","last_became_peered":"2023-09-23T03:19:23.510155+0000","last_unstale":"2023-09-23T03:19:23.516019+0000","last_undegraded":"2023-09-23T03:19:23.516019+0000","last_fullsized":"2023-09-23T03:19:23.516019+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:15:52.148791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.491878+0000","last_change":"2023-09-23T03:19:23.491878+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.491878+0000","last_undegraded":"2023-09-23T03:19:23.491878+0000","last_fullsized":"2023-09-23T03:19:23.491878+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.524455+0000","last_change":"2023-09-23T03:19:23.513693+0000","last_active":"2023-09-23T03:19:23.524455+0000","last_peered":"2023-09-23T03:19:23.524455+0000","last_clean":"2023-09-23T03:19:23.524455+0000","last_became_active":"2023-09-23T03:19:23.513359+0000","last_became_peered":"2023-09-23T03:19:23.513359+0000","last_unstale":"2023-09-23T03:19:23.524455+0000","last_undegraded":"2023-09-23T03:19:23.524455+0000","last_fullsized":"2023-09-23T03:19:23.524455+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:09:18.859490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.487319+0000","last_change":"2023-09-23T03:19:23.487319+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.487319+0000","last_undegraded":"2023-09-23T03:19:23.487319+0000","last_fullsized":"2023-09-23T03:19:23.487319+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.519999+0000","last_change":"2023-09-23T03:19:23.519087+0000","last_active":"2023-09-23T03:19:23.519999+0000","last_peered":"2023-09-23T03:19:23.519999+0000","last_clean":"2023-09-23T03:19:23.519999+0000","last_became_active":"2023-09-23T03:19:23.518827+0000","last_became_peered":"2023-09-23T03:19:23.518827+0000","last_unstale":"2023-09-23T03:19:23.519999+0000","last_undegraded":"2023-09-23T03:19:23.519999+0000","last_fullsized":"2023-09-23T03:19:23.519999+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:15:52.148791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.498413+0000","last_change":"2023-09-23T03:19:23.498413+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.498413+0000","last_undegraded":"2023-09-23T03:19:23.498413+0000","last_fullsized":"2023-09-23T03:19:23.498413+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.502654+0000","last_change":"2023-09-23T03:19:23.498946+0000","last_active":"2023-09-23T03:19:23.502654+0000","last_peered":"2023-09-23T03:19:23.502654+0000","last_clean":"2023-09-23T03:19:23.502654+0000","last_became_active":"2023-09-23T03:19:23.498522+0000","last_became_peered":"2023-09-23T03:19:23.498522+0000","last_unstale":"2023-09-23T03:19:23.502654+0000","last_undegraded":"2023-09-23T03:19:23.502654+0000","last_fullsized":"2023-09-23T03:19:23.502654+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:39:49.215616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.499307+0000","last_change":"2023-09-23T03:19:23.499307+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.499307+0000","last_undegraded":"2023-09-23T03:19:23.499307+0000","last_fullsized":"2023-09-23T03:19:23.499307+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.512203+0000","last_change":"2023-09-23T03:19:23.509484+0000","last_active":"2023-09-23T03:19:23.512203+0000","last_peered":"2023-09-23T03:19:23.512203+0000","last_clean":"2023-09-23T03:19:23.512203+0000","last_became_active":"2023-09-23T03:19:23.509203+0000","last_became_peered":"2023-09-23T03:19:23.509203+0000","last_unstale":"2023-09-23T03:19:23.512203+0000","last_undegraded":"2023-09-23T03:19:23.512203+0000","last_fullsized":"2023-09-23T03:19:23.512203+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:10:34.290176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.487164+0000","last_change":"2023-09-23T03:19:23.487164+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.487164+0000","last_undegraded":"2023-09-23T03:19:23.487164+0000","last_fullsized":"2023-09-23T03:19:23.487164+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.518826+0000","last_change":"2023-09-23T03:19:23.513953+0000","last_active":"2023-09-23T03:19:23.518826+0000","last_peered":"2023-09-23T03:19:23.518826+0000","last_clean":"2023-09-23T03:19:23.518826+0000","last_became_active":"2023-09-23T03:19:23.513616+0000","last_became_peered":"2023-09-23T03:19:23.513616+0000","last_unstale":"2023-09-23T03:19:23.518826+0000","last_undegraded":"2023-09-23T03:19:23.518826+0000","last_fullsized":"2023-09-23T03:19:23.518826+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:23:05.866775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.487239+0000","last_change":"2023-09-23T03:19:23.487239+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.487239+0000","last_undegraded":"2023-09-23T03:19:23.487239+0000","last_fullsized":"2023-09-23T03:19:23.487239+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.524435+0000","last_change":"2023-09-23T03:19:23.519480+0000","last_active":"2023-09-23T03:19:23.524435+0000","last_peered":"2023-09-23T03:19:23.524435+0000","last_clean":"2023-09-23T03:19:23.524435+0000","last_became_active":"2023-09-23T03:19:23.519191+0000","last_became_peered":"2023-09-23T03:19:23.519191+0000","last_unstale":"2023-09-23T03:19:23.524435+0000","last_undegraded":"2023-09-23T03:19:23.524435+0000","last_fullsized":"2023-09-23T03:19:23.524435+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:43:42.533621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":198,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-09-23T03:19:22.484979+0000","last_change":"2023-09-23T03:15:11.042447+0000","last_active":"2023-09-23T03:19:22.484979+0000","last_peered":"2023-09-23T03:19:22.484979+0000","last_clean":"2023-09-23T03:19:22.484979+0000","last_became_active":"2023-09-23T03:15:11.040322+0000","last_became_peered":"2023-09-23T03:15:11.040322+0000","last_unstale":"2023-09-23T03:19:22.484979+0000","last_undegraded":"2023-09-23T03:19:22.484979+0000","last_fullsized":"2023-09-23T03:19:22.484979+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_clean_scrub_stamp":"2023-09-23T03:15:09.474842+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:18:36.331645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574901,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1314816,"data_stored":923854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.98799999999999999,"15min":0.99199999999999999},"min":{"1min":0.68799999999999994,"5min":0.442,"15min":0.442},"max":{"1min":1.55,"5min":2.149,"15min":2.149},"last":1.075},{"interface":"front","average":{"1min":1.111,"5min":1.0569999999999999,"15min":1.0469999999999999},"min":{"1min":0.82099999999999995,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.681,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.95299999999999996}]},{"osd":1,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.198,"15min":1.22},"min":{"1min":0.74299999999999999,"5min":0.504,"15min":0.504},"max":{"1min":1.5649999999999999,"5min":3.1859999999999999,"15min":3.1859999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.248,"5min":1.294,"15min":1.3759999999999999},"min":{"1min":0.93899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.5840000000000001,"5min":2.613,"15min":2.613},"last":1.335}]},{"osd":2,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.1240000000000001,"15min":1.155},"min":{"1min":0.70199999999999996,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.6399999999999999,"5min":2.331,"15min":2.331},"last":0.879},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.242,"15min":1.296},"min":{"1min":0.75600000000000001,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.673,"5min":3.093,"15min":3.093},"last":1.3740000000000001}]},{"osd":3,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.248,"15min":1.2989999999999999},"min":{"1min":0.90000000000000002,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.7310000000000001,"5min":2.532,"15min":2.532},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.389,"5min":1.377,"15min":1.4410000000000001},"min":{"1min":0.95099999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.784,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":1.2829999999999999}]},{"osd":4,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.415,"15min":1.528},"min":{"1min":0.79200000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.7070000000000001,"5min":2.7210000000000001,"15min":2.7210000000000001},"last":1.5489999999999999},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.4730000000000001,"15min":1.556},"min":{"1min":0.81699999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.5880000000000001,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":1.5089999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.4179999999999999,"15min":1.4810000000000001},"min":{"1min":0.72699999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.736,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.288,"5min":1.4510000000000001,"15min":1.548},"min":{"1min":1.0069999999999999,"5min":0.497,"15min":0.497},"max":{"1min":1.7689999999999999,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":1.1819999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.5669999999999999,"15min":1.649},"min":{"1min":1.2410000000000001,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.8580000000000001,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":1.423},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.5089999999999999,"15min":1.607},"min":{"1min":1.002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.778,"5min":3.6739999999999999,"15min":3.6739999999999999},"last":1.012}]}]},{"osd":6,"up_from":13,"seq":55834574901,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":720896,"data_stored":333486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.048,"15min":1.0289999999999999},"min":{"1min":0.68899999999999995,"5min":0.38,"15min":0.38},"max":{"1min":1.6379999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.04},{"interface":"front","average":{"1min":1.083,"5min":1.0469999999999999,"15min":1.0549999999999999},"min":{"1min":0.61599999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.516,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.97899999999999998}]},{"osd":1,"last update":"Sat Sep 23 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.109,"15min":1.0980000000000001},"min":{"1min":0.65900000000000003,"5min":0.438,"15min":0.438},"max":{"1min":1.6830000000000001,"5min":1.867,"15min":1.867},"last":0.88},{"interface":"front","average":{"1min":1.23,"5min":1.2170000000000001,"15min":1.242},"min":{"1min":0.82399999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.6339999999999999,"5min":2.242,"15min":2.242},"last":1.3149999999999999}]},{"osd":2,"last update":"Sat Sep 23 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.1180000000000001,"15min":1.1060000000000001},"min":{"1min":0.82499999999999996,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4470000000000001,"5min":2.097,"15min":2.097},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.194,"5min":1.234,"15min":1.2669999999999999},"min":{"1min":0.66800000000000004,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.8149999999999999,"5min":1.903,"15min":1.903},"last":1.25}]},{"osd":3,"last update":"Sat Sep 23 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.3380000000000001,"15min":1.361},"min":{"1min":0.80200000000000005,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.8879999999999999,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":1.587},{"interface":"front","average":{"1min":1.278,"5min":1.389,"15min":1.4550000000000001},"min":{"1min":0.76600000000000001,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.657,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":1.2170000000000001}]},{"osd":4,"last update":"Sat Sep 23 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.397,"15min":1.395},"min":{"1min":0.81399999999999995,"5min":0.498,"15min":0.498},"max":{"1min":1.827,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.2849999999999999,"15min":1.294},"min":{"1min":0.75700000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.661,"5min":2.2109999999999999,"15min":2.2109999999999999},"last":0.84399999999999997}]},{"osd":5,"last update":"Sat Sep 23 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.391,"15min":1.395},"min":{"1min":0.84699999999999998,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.7390000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.4810000000000001,"15min":1.486},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.9390000000000001,"5min":4.758,"15min":4.758},"last":1.6779999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.508,"15min":1.5549999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.77,"5min":2.476,"15min":2.476},"last":1.151},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.532,"15min":1.5640000000000001},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.1739999999999999,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.472}]}]},{"osd":5,"up_from":13,"seq":55834574903,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":720896,"data_stored":333486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.044,"15min":1.03},"min":{"1min":0.80200000000000005,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.474,"5min":1.637,"15min":1.637},"last":1.075},{"interface":"front","average":{"1min":1.141,"5min":1.0609999999999999,"15min":1.032},"min":{"1min":0.78800000000000003,"5min":0.63,"15min":0.63},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":1.621}]},{"osd":1,"last update":"Sat Sep 23 03:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.212,"15min":1.224},"min":{"1min":0.95599999999999996,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.6459999999999999,"5min":1.849,"15min":1.849},"last":1.135},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.2869999999999999,"15min":1.3220000000000001},"min":{"1min":1.085,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.7410000000000001,"5min":1.893,"15min":1.893},"last":1.3759999999999999}]},{"osd":2,"last update":"Sat Sep 23 03:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.22,"15min":1.2170000000000001},"min":{"1min":1.0189999999999999,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.597,"5min":1.681,"15min":1.681},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.161,"5min":1.244,"15min":1.2969999999999999},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.5740000000000001,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":1.2090000000000001}]},{"osd":3,"last update":"Sat Sep 23 03:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.2509999999999999,"15min":1.361},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.423,"5min":1.905,"15min":1.905},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.3859999999999999,"15min":1.47},"min":{"1min":1.056,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.792,"5min":3.6110000000000002,"15min":3.6110000000000002},"last":1.4419999999999999}]},{"osd":4,"last update":"Sat Sep 23 03:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3939999999999999,"15min":1.444},"min":{"1min":0.83099999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.8500000000000001,"5min":2.069,"15min":2.069},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.401,"15min":1.4530000000000001},"min":{"1min":0.91100000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.7889999999999999,"5min":2.319,"15min":2.319},"last":1.3180000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":1.5089999999999999,"15min":1.5409999999999999},"min":{"1min":0.69899999999999995,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.948,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":1.6739999999999999},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.5309999999999999,"15min":1.613},"min":{"1min":0.97099999999999997,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.877,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.5760000000000001}]},{"osd":7,"last update":"Sat Sep 23 03:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.4570000000000001,"15min":1.538},"min":{"1min":0.91600000000000004,"5min":0.626,"15min":0.626},"max":{"1min":1.6839999999999999,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.734},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.585,"15min":1.643},"min":{"1min":1.046,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.7330000000000001,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.417}]}]},{"osd":4,"up_from":13,"seq":55834574902,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":708608,"data_stored":326899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:19:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.1279999999999999,"15min":1.1379999999999999},"min":{"1min":0.64800000000000002,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.639,"5min":1.806,"15min":1.806},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.1739999999999999,"15min":1.141},"min":{"1min":0.76500000000000001,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.905,"5min":1.9179999999999999,"15min":1.9179999999999999},"last":1.6479999999999999}]},{"osd":1,"last update":"Sat Sep 23 03:19:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.2130000000000001,"15min":1.2150000000000001},"min":{"1min":0.90200000000000002,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1910000000000001,"15min":1.2210000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.4710000000000001,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":0.97799999999999998}]},{"osd":2,"last update":"Sat Sep 23 03:19:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.1870000000000001,"15min":1.246},"min":{"1min":0.71199999999999997,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.129,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.246,"15min":1.224},"min":{"1min":0.91200000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.909,"5min":1.909,"15min":1.909},"last":0.90000000000000002}]},{"osd":3,"last update":"Sat Sep 23 03:19:15 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.1479999999999999,"15min":1.171},"min":{"1min":0.71999999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.5369999999999999,"5min":1.583,"15min":1.583},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.296,"5min":1.319,"15min":1.341},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.1779999999999999,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.302}]},{"osd":5,"last update":"Sat Sep 23 03:19:15 2023","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.371,"15min":1.345},"min":{"1min":0.99299999999999999,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.8050000000000002,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.371},{"interface":"front","average":{"1min":1.371,"5min":1.3300000000000001,"15min":1.3260000000000001},"min":{"1min":0.84299999999999997,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":2.887,"5min":2.887,"15min":2.887},"last":1.4690000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:19:15 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.397,"15min":1.391},"min":{"1min":1.048,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.4249999999999998,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.6619999999999999,"5min":1.5049999999999999,"15min":1.478},"min":{"1min":1.323,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":2.052,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.75}]},{"osd":7,"last update":"Sat Sep 23 03:19:15 2023","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.4650000000000001,"15min":1.4730000000000001},"min":{"1min":1.1819999999999999,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":2.5680000000000001,"5min":2.5680000000000001,"15min":2.5680000000000001},"last":1.716},{"interface":"front","average":{"1min":1.6379999999999999,"5min":1.5580000000000001,"15min":1.5649999999999999},"min":{"1min":1.056,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":2.4769999999999999,"5min":2.4769999999999999,"15min":2.4769999999999999},"last":1.593}]}]},{"osd":3,"up_from":13,"seq":55834574902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":708608,"data_stored":326899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.85399999999999998,"15min":0.875},"min":{"1min":0.42199999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.7090000000000001,"5min":1.744,"15min":1.744},"last":1.478},{"interface":"front","average":{"1min":0.876,"5min":0.86599999999999999,"15min":0.90200000000000002},"min":{"1min":0.51600000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.341,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.94899999999999995}]},{"osd":1,"last update":"Sat Sep 23 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.90700000000000003,"15min":0.96399999999999997},"min":{"1min":0.442,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.5640000000000001,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":1.401},{"interface":"front","average":{"1min":1.02,"5min":0.98099999999999998,"15min":0.98499999999999999},"min":{"1min":0.54700000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.3839999999999999,"5min":1.704,"15min":1.704},"last":1.127}]},{"osd":2,"last update":"Sat Sep 23 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.0609999999999999,"15min":1.113},"min":{"1min":0.54800000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.639,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.639},{"interface":"front","average":{"1min":1.165,"5min":1.097,"15min":1.129},"min":{"1min":0.64400000000000002,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.6830000000000001,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.6830000000000001}]},{"osd":4,"last update":"Sat Sep 23 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.2809999999999999,"15min":1.333},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.0099999999999998,"5min":2.698,"15min":2.698},"last":2.0099999999999998},{"interface":"front","average":{"1min":1.353,"5min":1.381,"15min":1.4470000000000001},"min":{"1min":0.95399999999999996,"5min":0.752,"15min":0.752},"max":{"1min":1.825,"5min":2.419,"15min":2.419},"last":1.825}]},{"osd":5,"last update":"Sat Sep 23 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.339,"15min":1.361},"min":{"1min":0.90300000000000002,"5min":0.753,"15min":0.753},"max":{"1min":1.8799999999999999,"5min":1.917,"15min":1.917},"last":1.867},{"interface":"front","average":{"1min":1.321,"5min":1.3049999999999999,"15min":1.3620000000000001},"min":{"1min":0.83499999999999996,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.716,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.716}]},{"osd":6,"last update":"Sat Sep 23 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.359,"15min":1.423},"min":{"1min":0.82699999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.9550000000000001,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":1.9550000000000001},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4430000000000001,"15min":1.482},"min":{"1min":1.05,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.7729999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.417,"15min":1.5009999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":2.2120000000000002,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.909},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.4399999999999999,"15min":1.526},"min":{"1min":0.85799999999999998,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.2029999999999998,"5min":2.5379999999999998,"15min":2.5379999999999998},"last":2.2029999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":720896,"data_stored":333486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.83799999999999997,"15min":0.83699999999999997},"min":{"1min":0.442,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":0.501},{"interface":"front","average":{"1min":0.998,"5min":0.86199999999999999,"15min":0.84299999999999997},"min":{"1min":0.48899999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.5329999999999999,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.77200000000000002}]},{"osd":1,"last update":"Sat Sep 23 03:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.84799999999999998,"15min":0.85999999999999999},"min":{"1min":0.55800000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.026,"5min":0.93300000000000005,"15min":0.94099999999999995},"min":{"1min":0.55100000000000005,"5min":0.38,"15min":0.38},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.95999999999999996}]},{"osd":3,"last update":"Sat Sep 23 03:19:12 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":0.96599999999999997,"15min":0.95199999999999996},"min":{"1min":0.53900000000000003,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.4279999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.073,"5min":0.98499999999999999,"15min":0.997},"min":{"1min":0.60699999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.534,"5min":1.78,"15min":1.78},"last":0.89900000000000002}]},{"osd":4,"last update":"Sat Sep 23 03:19:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2729999999999999,"15min":1.3240000000000001},"min":{"1min":0.86199999999999999,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.819,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.071},{"interface":"front","average":{"1min":1.296,"5min":1.2969999999999999,"15min":1.341},"min":{"1min":0.77600000000000002,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.631,"5min":2.234,"15min":2.234},"last":1.0029999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:19:12 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.339,"15min":1.409},"min":{"1min":0.97499999999999998,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.7849999999999999,"5min":3.9409999999999998,"15min":3.9409999999999998},"last":1.032},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.4199999999999999,"15min":1.5329999999999999},"min":{"1min":0.81299999999999994,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.9359999999999999,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":1.1759999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:19:12 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.3580000000000001,"15min":1.417},"min":{"1min":0.83299999999999996,"5min":0.755,"15min":0.755},"max":{"1min":1.8540000000000001,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.458,"5min":1.393,"15min":1.478},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.1680000000000001,"5min":4.5800000000000001,"15min":4.5800000000000001},"last":1.1279999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:19:12 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.3600000000000001,"15min":1.4019999999999999},"min":{"1min":1.0469999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.996,"5min":2.3149999999999999,"15min":2.3149999999999999},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.413,"15min":1.4550000000000001},"min":{"1min":0.873,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.9259999999999999,"5min":2.621,"15min":2.621},"last":1.109}]}]},{"osd":1,"up_from":13,"seq":55834574904,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":720896,"data_stored":333486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.80000000000000004,"15min":0.80800000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.2490000000000001,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.85099999999999998},"min":{"1min":0.34799999999999998,"5min":0.312,"15min":0.312},"max":{"1min":1.2210000000000001,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":0.623}]},{"osd":2,"last update":"Sat Sep 23 03:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.84999999999999998,"15min":0.89400000000000002},"min":{"1min":0.52000000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.409,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":1.278},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.004,"15min":1.0369999999999999},"min":{"1min":0.54500000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5600000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.524}]},{"osd":3,"last update":"Sat Sep 23 03:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.0269999999999999,"15min":1.087},"min":{"1min":0.48799999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.5389999999999999,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":1.24},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.0980000000000001,"15min":1.1619999999999999},"min":{"1min":0.56200000000000006,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.7649999999999999,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":1.6850000000000001}]},{"osd":4,"last update":"Sat Sep 23 03:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.2330000000000001,"15min":1.2829999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.7270000000000001,"5min":3.2629999999999999,"15min":3.2629999999999999},"last":1.411},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.262,"15min":1.3},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6359999999999999,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":1.456}]},{"osd":5,"last update":"Sat Sep 23 03:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.254,"15min":1.2989999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.9319999999999999,"5min":3.161,"15min":3.161},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3,"15min":1.3460000000000001},"min":{"1min":0.80400000000000005,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.698,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":1.5840000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.2969999999999999,"15min":1.3400000000000001},"min":{"1min":0.76500000000000001,"5min":0.747,"15min":0.747},"max":{"1min":2.0329999999999999,"5min":3.0179999999999998,"15min":3.0179999999999998},"last":1.637},{"interface":"front","average":{"1min":1.333,"5min":1.3520000000000001,"15min":1.3979999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.7909999999999999,"5min":3.3580000000000001,"15min":3.3580000000000001},"last":1.78}]},{"osd":7,"last update":"Sat Sep 23 03:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.363,"15min":1.413},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":2.085,"5min":3.1099999999999999,"15min":3.1099999999999999},"last":1.6619999999999999},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.415,"15min":1.4810000000000001},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.7330000000000001,"5min":3.306,"15min":3.306},"last":1.722}]}]},{"osd":0,"up_from":13,"seq":55834574904,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1310720,"data_stored":922835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.81200000000000006,"15min":0.81999999999999995},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.3180000000000001,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.86599999999999999,"15min":0.89300000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.363,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.96399999999999997}]},{"osd":2,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.81100000000000005,"15min":0.80300000000000005},"min":{"1min":0.48599999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.22,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.97299999999999998,"15min":0.98499999999999999},"min":{"1min":0.46700000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":1.3029999999999999}]},{"osd":3,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93300000000000005,"15min":0.91500000000000004},"min":{"1min":0.44800000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.226,"5min":1.591,"15min":1.591},"last":1.198},{"interface":"front","average":{"1min":1.117,"5min":0.996,"15min":1.006},"min":{"1min":0.57299999999999995,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.4990000000000001,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.147}]},{"osd":4,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.204,"15min":1.204},"min":{"1min":0.81499999999999995,"5min":0.755,"15min":0.755},"max":{"1min":1.601,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.7450000000000001},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.2410000000000001,"15min":1.274},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.4610000000000001,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.456}]},{"osd":5,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.2250000000000001,"15min":1.2290000000000001},"min":{"1min":0.872,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.6739999999999999,"5min":1.867,"15min":1.867},"last":1.474},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.294,"15min":1.272},"min":{"1min":0.76400000000000001,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":1.8080000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.3009999999999999,"15min":1.3029999999999999},"min":{"1min":0.99399999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":1.552},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.325,"15min":1.3029999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.875,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.873}]},{"osd":7,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.329,"15min":1.3160000000000001},"min":{"1min":0.93300000000000005,"5min":0.747,"15min":0.747},"max":{"1min":1.6499999999999999,"5min":1.944,"15min":1.944},"last":1.528},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.3979999999999999,"15min":1.4059999999999999},"min":{"1min":0.91800000000000004,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.756,"5min":2.048,"15min":2.048},"last":1.7170000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-23T03:19:28.955 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-23T03:19:29.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.210+0000 7f24576ec640 1 Processor -- start 2023-09-23T03:19:29.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.210+0000 7f24576ec640 1 -- start start 2023-09-23T03:19:29.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.212+0000 7f24576ec640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24500acce0 0x7f245015ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:29.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.212+0000 7f24576ec640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f24500b1b20 0x7f24500ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:29.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.212+0000 7f24576ec640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f245015ac30 con 0x7f24500cee70 2023-09-23T03:19:29.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.212+0000 7f24576ec640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24500b2e90 con 0x7f24500b1b20 2023-09-23T03:19:29.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.212+0000 7f24576ec640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24500cf320 con 0x7f24500acce0 2023-09-23T03:19:29.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.212+0000 7f2455488640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24500acce0 0x7f245015ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:29.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.212+0000 7f2454c87640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f24500b1b20 0x7f24500ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:29.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.212+0000 7f2455488640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24500acce0 0x7f245015ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:54716/0 (socket says 172.21.15.154:54716) 2023-09-23T03:19:29.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.212+0000 7f2455488640 1 -- 172.21.15.154:0/3366261876 learned_addr learned my addr 172.21.15.154:0/3366261876 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:29.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.213+0000 7f2455488640 1 -- 172.21.15.154:0/3366261876 >> v2:172.21.15.190:3300/0 conn(0x7f24500b1b20 msgr2=0x7f24500ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:29.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.213+0000 7f2455488640 1 --2- 172.21.15.154:0/3366261876 >> v2:172.21.15.190:3300/0 conn(0x7f24500b1b20 0x7f24500ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:29.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.213+0000 7f2455488640 1 -- 172.21.15.154:0/3366261876 >> v1:172.21.15.190:6789/0 conn(0x7f24500cee70 legacy=0x7f245006cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:29.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.213+0000 7f2455488640 1 -- 172.21.15.154:0/3366261876 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24500b1f70 con 0x7f24500acce0 2023-09-23T03:19:29.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.215+0000 7f2455488640 1 --2- 172.21.15.154:0/3366261876 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24500acce0 0x7f245015ed60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f244c009ea0 tx=0x7f244c02bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40795bf8bd260810 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:29.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.215+0000 7f2447fff640 1 -- 172.21.15.154:0/3366261876 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c02cab0 con 0x7f24500acce0 2023-09-23T03:19:29.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.215+0000 7f2447fff640 1 -- 172.21.15.154:0/3366261876 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f244c02cc10 con 0x7f24500acce0 2023-09-23T03:19:29.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.215+0000 7f2447fff640 1 -- 172.21.15.154:0/3366261876 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c0343f0 con 0x7f24500acce0 2023-09-23T03:19:29.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.216+0000 7f24576ec640 1 -- 172.21.15.154:0/3366261876 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24500acce0 msgr2=0x7f245015ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:29.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.216+0000 7f24576ec640 1 --2- 172.21.15.154:0/3366261876 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24500acce0 0x7f245015ed60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f244c009ea0 tx=0x7f244c02bf10 comp rx=0 tx=0).stop 2023-09-23T03:19:29.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.216+0000 7f24576ec640 1 -- 172.21.15.154:0/3366261876 shutdown_connections 2023-09-23T03:19:29.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.216+0000 7f24576ec640 1 --2- 172.21.15.154:0/3366261876 >> v2:172.21.15.190:3300/0 conn(0x7f24500b1b20 0x7f24500ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:29.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.216+0000 7f24576ec640 1 --2- 172.21.15.154:0/3366261876 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24500acce0 0x7f245015ed60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:29.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.216+0000 7f24576ec640 1 -- 172.21.15.154:0/3366261876 >> 172.21.15.154:0/3366261876 conn(0x7f245006a0f0 msgr2=0x7f245006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:29.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.216+0000 7f24576ec640 1 -- 172.21.15.154:0/3366261876 shutdown_connections 2023-09-23T03:19:29.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.216+0000 7f24576ec640 1 -- 172.21.15.154:0/3366261876 wait complete. 2023-09-23T03:19:29.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.218+0000 7f24576ec640 1 Processor -- start 2023-09-23T03:19:29.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.218+0000 7f24576ec640 1 -- start start 2023-09-23T03:19:29.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.219+0000 7f24576ec640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24500acce0 0x7f2450155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:29.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.219+0000 7f24576ec640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24500b1b20 0x7f245015d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:29.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.219+0000 7f24576ec640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24500cee70 0x7f24500c4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:29.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.219+0000 7f24576ec640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24500c55e0 con 0x7f24500cee70 2023-09-23T03:19:29.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.219+0000 7f24576ec640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2450155b70 con 0x7f24500b1b20 2023-09-23T03:19:29.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.219+0000 7f24576ec640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24500c1fa0 con 0x7f24500acce0 2023-09-23T03:19:29.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.219+0000 7f2455c89640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24500cee70 0x7f24500c4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:29.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.219+0000 7f2455c89640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24500cee70 0x7f24500c4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43038/0 (socket says 172.21.15.154:43038) 2023-09-23T03:19:29.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.219+0000 7f2455c89640 1 -- 172.21.15.154:0/2450323989 learned_addr learned my addr 172.21.15.154:0/2450323989 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:29.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.220+0000 7f2455488640 1 --2- 172.21.15.154:0/2450323989 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24500acce0 0x7f2450155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:29.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.220+0000 7f2454c87640 1 --2- 172.21.15.154:0/2450323989 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24500b1b20 0x7f245015d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:29.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.220+0000 7f2455c89640 1 -- 172.21.15.154:0/2450323989 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24500acce0 msgr2=0x7f2450155630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:29.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.220+0000 7f2455c89640 1 --2- 172.21.15.154:0/2450323989 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24500acce0 0x7f2450155630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:29.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.220+0000 7f2455c89640 1 -- 172.21.15.154:0/2450323989 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24500b1b20 msgr2=0x7f245015d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:29.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.220+0000 7f2455c89640 1 --2- 172.21.15.154:0/2450323989 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24500b1b20 0x7f245015d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:29.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.220+0000 7f2455c89640 1 -- 172.21.15.154:0/2450323989 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f244c009b30 con 0x7f24500cee70 2023-09-23T03:19:29.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.220+0000 7f2454c87640 1 --2- 172.21.15.154:0/2450323989 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24500b1b20 0x7f245015d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:29.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.220+0000 7f2455c89640 1 --2- 172.21.15.154:0/2450323989 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24500cee70 0x7f24500c4480 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f244800c3c0 tx=0x7f244800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:29.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.220+0000 7f2445ffb640 1 -- 172.21.15.154:0/2450323989 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24480052d0 con 0x7f24500cee70 2023-09-23T03:19:29.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.220+0000 7f2445ffb640 1 -- 172.21.15.154:0/2450323989 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2448005430 con 0x7f24500cee70 2023-09-23T03:19:29.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.220+0000 7f24576ec640 1 -- 172.21.15.154:0/2450323989 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24500c21f0 con 0x7f24500cee70 2023-09-23T03:19:29.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.220+0000 7f2445ffb640 1 -- 172.21.15.154:0/2450323989 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2448003810 con 0x7f24500cee70 2023-09-23T03:19:29.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.221+0000 7f24576ec640 1 -- 172.21.15.154:0/2450323989 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24500c27b0 con 0x7f24500cee70 2023-09-23T03:19:29.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.222+0000 7f24576ec640 1 -- 172.21.15.154:0/2450323989 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24500c4130 con 0x7f24500cee70 2023-09-23T03:19:29.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.223+0000 7f2445ffb640 1 -- 172.21.15.154:0/2450323989 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f2448003970 con 0x7f24500cee70 2023-09-23T03:19:29.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.223+0000 7f2445ffb640 1 --2- 172.21.15.154:0/2450323989 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f243406af60 0x7f243406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:29.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.224+0000 7f2445ffb640 1 -- 172.21.15.154:0/2450323989 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2448088be0 con 0x7f24500cee70 2023-09-23T03:19:29.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.224+0000 7f2455488640 1 --2- 172.21.15.154:0/2450323989 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f243406af60 0x7f243406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:29.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.224+0000 7f2455488640 1 --2- 172.21.15.154:0/2450323989 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f243406af60 0x7f243406d420 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f244c035000 tx=0x7f244c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:29.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.228+0000 7f2445ffb640 1 -- 172.21.15.154:0/2450323989 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f2448053ae0 con 0x7f24500cee70 2023-09-23T03:19:29.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.413+0000 7f24576ec640 1 -- 172.21.15.154:0/2450323989 --> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f24500b2210 con 0x7f243406af60 2023-09-23T03:19:29.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.425+0000 7f2445ffb640 1 -- 172.21.15.154:0/2450323989 <== mgr.4110 v2:172.21.15.154:6832/2422652104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+281110 (secure 0 0 0) 0x7f244c044090 con 0x7f243406af60 2023-09-23T03:19:29.429 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:19:29.430 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2023-09-23T03:19:29.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.430+0000 7f24576ec640 1 -- 172.21.15.154:0/2450323989 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f243406af60 msgr2=0x7f243406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:29.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.430+0000 7f24576ec640 1 --2- 172.21.15.154:0/2450323989 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f243406af60 0x7f243406d420 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f244c035000 tx=0x7f244c030000 comp rx=0 tx=0).stop 2023-09-23T03:19:29.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.430+0000 7f24576ec640 1 -- 172.21.15.154:0/2450323989 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24500cee70 msgr2=0x7f24500c4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:29.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.430+0000 7f24576ec640 1 --2- 172.21.15.154:0/2450323989 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24500cee70 0x7f24500c4480 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f244800c3c0 tx=0x7f244800c880 comp rx=0 tx=0).stop 2023-09-23T03:19:29.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.430+0000 7f24576ec640 1 -- 172.21.15.154:0/2450323989 shutdown_connections 2023-09-23T03:19:29.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.430+0000 7f24576ec640 1 --2- 172.21.15.154:0/2450323989 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f243406af60 0x7f243406d420 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:29.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.430+0000 7f24576ec640 1 --2- 172.21.15.154:0/2450323989 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24500cee70 0x7f24500c4480 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:29.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.430+0000 7f24576ec640 1 --2- 172.21.15.154:0/2450323989 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24500b1b20 0x7f245015d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:29.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.430+0000 7f24576ec640 1 --2- 172.21.15.154:0/2450323989 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24500acce0 0x7f2450155630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:29.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.430+0000 7f24576ec640 1 -- 172.21.15.154:0/2450323989 >> 172.21.15.154:0/2450323989 conn(0x7f245006a0f0 msgr2=0x7f245006a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:29.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.430+0000 7f24576ec640 1 -- 172.21.15.154:0/2450323989 shutdown_connections 2023-09-23T03:19:29.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.431+0000 7f24576ec640 1 -- 172.21.15.154:0/2450323989 wait complete. 2023-09-23T03:19:29.449 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":185,"stamp":"2023-09-23T03:19:29.249766+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235436,"kb_used_data":6764,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754739284,"statfs":{"total":773094113280,"available":772853026816,"internally_reserved":0,"allocated":6926336,"data_stored":3834431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":234016314},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001767"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.496299+0000","last_change":"2023-09-23T03:19:23.496299+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.496299+0000","last_undegraded":"2023-09-23T03:19:23.496299+0000","last_fullsized":"2023-09-23T03:19:23.496299+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.491358+0000","last_change":"2023-09-23T03:19:23.491358+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.491358+0000","last_undegraded":"2023-09-23T03:19:23.491358+0000","last_fullsized":"2023-09-23T03:19:23.491358+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.499260+0000","last_change":"2023-09-23T03:19:23.499260+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.499260+0000","last_undegraded":"2023-09-23T03:19:23.499260+0000","last_fullsized":"2023-09-23T03:19:23.499260+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.504046+0000","last_change":"2023-09-23T03:19:23.504046+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.504046+0000","last_undegraded":"2023-09-23T03:19:23.504046+0000","last_fullsized":"2023-09-23T03:19:23.504046+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.494815+0000","last_change":"2023-09-23T03:19:23.494815+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.494815+0000","last_undegraded":"2023-09-23T03:19:23.494815+0000","last_fullsized":"2023-09-23T03:19:23.494815+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.489767+0000","last_change":"2023-09-23T03:19:23.489767+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.489767+0000","last_undegraded":"2023-09-23T03:19:23.489767+0000","last_fullsized":"2023-09-23T03:19:23.489767+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.488689+0000","last_change":"2023-09-23T03:19:23.488689+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.488689+0000","last_undegraded":"2023-09-23T03:19:23.488689+0000","last_fullsized":"2023-09-23T03:19:23.488689+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.490760+0000","last_change":"2023-09-23T03:19:23.490760+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.490760+0000","last_undegraded":"2023-09-23T03:19:23.490760+0000","last_fullsized":"2023-09-23T03:19:23.490760+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.489816+0000","last_change":"2023-09-23T03:19:23.489816+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.489816+0000","last_undegraded":"2023-09-23T03:19:23.489816+0000","last_fullsized":"2023-09-23T03:19:23.489816+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.500805+0000","last_change":"2023-09-23T03:19:23.500805+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.500805+0000","last_undegraded":"2023-09-23T03:19:23.500805+0000","last_fullsized":"2023-09-23T03:19:23.500805+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.501343+0000","last_change":"2023-09-23T03:19:23.501343+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.501343+0000","last_undegraded":"2023-09-23T03:19:23.501343+0000","last_fullsized":"2023-09-23T03:19:23.501343+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.503530+0000","last_change":"2023-09-23T03:19:23.503530+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.503530+0000","last_undegraded":"2023-09-23T03:19:23.503530+0000","last_fullsized":"2023-09-23T03:19:23.503530+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.495939+0000","last_change":"2023-09-23T03:19:23.495939+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.495939+0000","last_undegraded":"2023-09-23T03:19:23.495939+0000","last_fullsized":"2023-09-23T03:19:23.495939+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.488295+0000","last_change":"2023-09-23T03:19:23.488295+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.488295+0000","last_undegraded":"2023-09-23T03:19:23.488295+0000","last_fullsized":"2023-09-23T03:19:23.488295+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.487204+0000","last_change":"2023-09-23T03:19:23.487204+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.487204+0000","last_undegraded":"2023-09-23T03:19:23.487204+0000","last_fullsized":"2023-09-23T03:19:23.487204+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.489319+0000","last_change":"2023-09-23T03:19:23.489319+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.489319+0000","last_undegraded":"2023-09-23T03:19:23.489319+0000","last_fullsized":"2023-09-23T03:19:23.489319+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.490227+0000","last_change":"2023-09-23T03:19:23.490227+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.490227+0000","last_undegraded":"2023-09-23T03:19:23.490227+0000","last_fullsized":"2023-09-23T03:19:23.490227+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.500342+0000","last_change":"2023-09-23T03:19:23.500342+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.500342+0000","last_undegraded":"2023-09-23T03:19:23.500342+0000","last_fullsized":"2023-09-23T03:19:23.500342+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.494436+0000","last_change":"2023-09-23T03:19:23.494436+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.494436+0000","last_undegraded":"2023-09-23T03:19:23.494436+0000","last_fullsized":"2023-09-23T03:19:23.494436+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.489696+0000","last_change":"2023-09-23T03:19:23.489696+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.489696+0000","last_undegraded":"2023-09-23T03:19:23.489696+0000","last_fullsized":"2023-09-23T03:19:23.489696+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.500116+0000","last_change":"2023-09-23T03:19:23.500116+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.500116+0000","last_undegraded":"2023-09-23T03:19:23.500116+0000","last_fullsized":"2023-09-23T03:19:23.500116+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.489656+0000","last_change":"2023-09-23T03:19:23.489656+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.489656+0000","last_undegraded":"2023-09-23T03:19:23.489656+0000","last_fullsized":"2023-09-23T03:19:23.489656+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.497801+0000","last_change":"2023-09-23T03:19:23.497801+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.497801+0000","last_undegraded":"2023-09-23T03:19:23.497801+0000","last_fullsized":"2023-09-23T03:19:23.497801+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.491559+0000","last_change":"2023-09-23T03:19:23.491559+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.491559+0000","last_undegraded":"2023-09-23T03:19:23.491559+0000","last_fullsized":"2023-09-23T03:19:23.491559+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.495800+0000","last_change":"2023-09-23T03:19:23.495800+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.495800+0000","last_undegraded":"2023-09-23T03:19:23.495800+0000","last_fullsized":"2023-09-23T03:19:23.495800+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.502761+0000","last_change":"2023-09-23T03:19:23.502761+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.502761+0000","last_undegraded":"2023-09-23T03:19:23.502761+0000","last_fullsized":"2023-09-23T03:19:23.502761+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.492766+0000","last_change":"2023-09-23T03:19:23.492766+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.492766+0000","last_undegraded":"2023-09-23T03:19:23.492766+0000","last_fullsized":"2023-09-23T03:19:23.492766+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.494163+0000","last_change":"2023-09-23T03:19:23.494163+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.494163+0000","last_undegraded":"2023-09-23T03:19:23.494163+0000","last_fullsized":"2023-09-23T03:19:23.494163+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.489233+0000","last_change":"2023-09-23T03:19:23.489233+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.489233+0000","last_undegraded":"2023-09-23T03:19:23.489233+0000","last_fullsized":"2023-09-23T03:19:23.489233+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.498098+0000","last_change":"2023-09-23T03:19:23.498098+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.498098+0000","last_undegraded":"2023-09-23T03:19:23.498098+0000","last_fullsized":"2023-09-23T03:19:23.498098+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.494358+0000","last_change":"2023-09-23T03:19:23.494358+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.494358+0000","last_undegraded":"2023-09-23T03:19:23.494358+0000","last_fullsized":"2023-09-23T03:19:23.494358+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.488708+0000","last_change":"2023-09-23T03:19:23.488708+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.488708+0000","last_undegraded":"2023-09-23T03:19:23.488708+0000","last_fullsized":"2023-09-23T03:19:23.488708+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.493052+0000","last_change":"2023-09-23T03:19:23.493052+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.493052+0000","last_undegraded":"2023-09-23T03:19:23.493052+0000","last_fullsized":"2023-09-23T03:19:23.493052+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.518586+0000","last_change":"2023-09-23T03:19:23.514227+0000","last_active":"2023-09-23T03:19:23.518586+0000","last_peered":"2023-09-23T03:19:23.518586+0000","last_clean":"2023-09-23T03:19:23.518586+0000","last_became_active":"2023-09-23T03:19:23.513867+0000","last_became_peered":"2023-09-23T03:19:23.513867+0000","last_unstale":"2023-09-23T03:19:23.518586+0000","last_undegraded":"2023-09-23T03:19:23.518586+0000","last_fullsized":"2023-09-23T03:19:23.518586+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:23:19.447543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.500949+0000","last_change":"2023-09-23T03:19:23.500949+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.500949+0000","last_undegraded":"2023-09-23T03:19:23.500949+0000","last_fullsized":"2023-09-23T03:19:23.500949+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.513306+0000","last_change":"2023-09-23T03:19:23.511815+0000","last_active":"2023-09-23T03:19:23.513306+0000","last_peered":"2023-09-23T03:19:23.513306+0000","last_clean":"2023-09-23T03:19:23.513306+0000","last_became_active":"2023-09-23T03:19:23.510810+0000","last_became_peered":"2023-09-23T03:19:23.510810+0000","last_unstale":"2023-09-23T03:19:23.513306+0000","last_undegraded":"2023-09-23T03:19:23.513306+0000","last_fullsized":"2023-09-23T03:19:23.513306+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:20:39.987490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.491398+0000","last_change":"2023-09-23T03:19:23.491398+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.491398+0000","last_undegraded":"2023-09-23T03:19:23.491398+0000","last_fullsized":"2023-09-23T03:19:23.491398+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.513486+0000","last_change":"2023-09-23T03:19:23.508844+0000","last_active":"2023-09-23T03:19:23.513486+0000","last_peered":"2023-09-23T03:19:23.513486+0000","last_clean":"2023-09-23T03:19:23.513486+0000","last_became_active":"2023-09-23T03:19:23.508557+0000","last_became_peered":"2023-09-23T03:19:23.508557+0000","last_unstale":"2023-09-23T03:19:23.513486+0000","last_undegraded":"2023-09-23T03:19:23.513486+0000","last_fullsized":"2023-09-23T03:19:23.513486+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:49:32.205787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.492304+0000","last_change":"2023-09-23T03:19:23.492304+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.492304+0000","last_undegraded":"2023-09-23T03:19:23.492304+0000","last_fullsized":"2023-09-23T03:19:23.492304+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.524534+0000","last_change":"2023-09-23T03:19:23.514514+0000","last_active":"2023-09-23T03:19:23.524534+0000","last_peered":"2023-09-23T03:19:23.524534+0000","last_clean":"2023-09-23T03:19:23.524534+0000","last_became_active":"2023-09-23T03:19:23.514219+0000","last_became_peered":"2023-09-23T03:19:23.514219+0000","last_unstale":"2023-09-23T03:19:23.524534+0000","last_undegraded":"2023-09-23T03:19:23.524534+0000","last_fullsized":"2023-09-23T03:19:23.524534+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:45:54.723198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.488324+0000","last_change":"2023-09-23T03:19:23.488324+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.488324+0000","last_undegraded":"2023-09-23T03:19:23.488324+0000","last_fullsized":"2023-09-23T03:19:23.488324+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.515178+0000","last_change":"2023-09-23T03:19:23.509524+0000","last_active":"2023-09-23T03:19:23.515178+0000","last_peered":"2023-09-23T03:19:23.515178+0000","last_clean":"2023-09-23T03:19:23.515178+0000","last_became_active":"2023-09-23T03:19:23.509229+0000","last_became_peered":"2023-09-23T03:19:23.509229+0000","last_unstale":"2023-09-23T03:19:23.515178+0000","last_undegraded":"2023-09-23T03:19:23.515178+0000","last_fullsized":"2023-09-23T03:19:23.515178+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:58:25.968168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.488600+0000","last_change":"2023-09-23T03:19:23.488600+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.488600+0000","last_undegraded":"2023-09-23T03:19:23.488600+0000","last_fullsized":"2023-09-23T03:19:23.488600+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.508057+0000","last_change":"2023-09-23T03:19:23.505508+0000","last_active":"2023-09-23T03:19:23.508057+0000","last_peered":"2023-09-23T03:19:23.508057+0000","last_clean":"2023-09-23T03:19:23.508057+0000","last_became_active":"2023-09-23T03:19:23.505249+0000","last_became_peered":"2023-09-23T03:19:23.505249+0000","last_unstale":"2023-09-23T03:19:23.508057+0000","last_undegraded":"2023-09-23T03:19:23.508057+0000","last_fullsized":"2023-09-23T03:19:23.508057+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:45:54.723198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.490303+0000","last_change":"2023-09-23T03:19:23.490303+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.490303+0000","last_undegraded":"2023-09-23T03:19:23.490303+0000","last_fullsized":"2023-09-23T03:19:23.490303+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.524401+0000","last_change":"2023-09-23T03:19:23.514127+0000","last_active":"2023-09-23T03:19:23.524401+0000","last_peered":"2023-09-23T03:19:23.524401+0000","last_clean":"2023-09-23T03:19:23.524401+0000","last_became_active":"2023-09-23T03:19:23.513630+0000","last_became_peered":"2023-09-23T03:19:23.513630+0000","last_unstale":"2023-09-23T03:19:23.524401+0000","last_undegraded":"2023-09-23T03:19:23.524401+0000","last_fullsized":"2023-09-23T03:19:23.524401+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:33:16.276149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.487896+0000","last_change":"2023-09-23T03:19:23.487896+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.487896+0000","last_undegraded":"2023-09-23T03:19:23.487896+0000","last_fullsized":"2023-09-23T03:19:23.487896+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.502692+0000","last_change":"2023-09-23T03:19:23.500111+0000","last_active":"2023-09-23T03:19:23.502692+0000","last_peered":"2023-09-23T03:19:23.502692+0000","last_clean":"2023-09-23T03:19:23.502692+0000","last_became_active":"2023-09-23T03:19:23.499792+0000","last_became_peered":"2023-09-23T03:19:23.499792+0000","last_unstale":"2023-09-23T03:19:23.502692+0000","last_undegraded":"2023-09-23T03:19:23.502692+0000","last_fullsized":"2023-09-23T03:19:23.502692+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:20:30.730712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.487758+0000","last_change":"2023-09-23T03:19:23.487758+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.487758+0000","last_undegraded":"2023-09-23T03:19:23.487758+0000","last_fullsized":"2023-09-23T03:19:23.487758+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.516664+0000","last_change":"2023-09-23T03:19:23.514447+0000","last_active":"2023-09-23T03:19:23.516664+0000","last_peered":"2023-09-23T03:19:23.516664+0000","last_clean":"2023-09-23T03:19:23.516664+0000","last_became_active":"2023-09-23T03:19:23.514067+0000","last_became_peered":"2023-09-23T03:19:23.514067+0000","last_unstale":"2023-09-23T03:19:23.516664+0000","last_undegraded":"2023-09-23T03:19:23.516664+0000","last_fullsized":"2023-09-23T03:19:23.516664+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:00:17.646758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.496971+0000","last_change":"2023-09-23T03:19:23.496971+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.496971+0000","last_undegraded":"2023-09-23T03:19:23.496971+0000","last_fullsized":"2023-09-23T03:19:23.496971+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.507007+0000","last_change":"2023-09-23T03:19:23.502457+0000","last_active":"2023-09-23T03:19:23.507007+0000","last_peered":"2023-09-23T03:19:23.507007+0000","last_clean":"2023-09-23T03:19:23.507007+0000","last_became_active":"2023-09-23T03:19:23.502185+0000","last_became_peered":"2023-09-23T03:19:23.502185+0000","last_unstale":"2023-09-23T03:19:23.507007+0000","last_undegraded":"2023-09-23T03:19:23.507007+0000","last_fullsized":"2023-09-23T03:19:23.507007+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:00:39.238009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.500995+0000","last_change":"2023-09-23T03:19:23.500995+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.500995+0000","last_undegraded":"2023-09-23T03:19:23.500995+0000","last_fullsized":"2023-09-23T03:19:23.500995+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.516693+0000","last_change":"2023-09-23T03:19:23.514525+0000","last_active":"2023-09-23T03:19:23.516693+0000","last_peered":"2023-09-23T03:19:23.516693+0000","last_clean":"2023-09-23T03:19:23.516693+0000","last_became_active":"2023-09-23T03:19:23.514117+0000","last_became_peered":"2023-09-23T03:19:23.514117+0000","last_unstale":"2023-09-23T03:19:23.516693+0000","last_undegraded":"2023-09-23T03:19:23.516693+0000","last_fullsized":"2023-09-23T03:19:23.516693+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:45:54.723198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.498493+0000","last_change":"2023-09-23T03:19:23.498493+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.498493+0000","last_undegraded":"2023-09-23T03:19:23.498493+0000","last_fullsized":"2023-09-23T03:19:23.498493+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.505498+0000","last_change":"2023-09-23T03:19:23.501871+0000","last_active":"2023-09-23T03:19:23.505498+0000","last_peered":"2023-09-23T03:19:23.505498+0000","last_clean":"2023-09-23T03:19:23.505498+0000","last_became_active":"2023-09-23T03:19:23.501537+0000","last_became_peered":"2023-09-23T03:19:23.501537+0000","last_unstale":"2023-09-23T03:19:23.505498+0000","last_undegraded":"2023-09-23T03:19:23.505498+0000","last_fullsized":"2023-09-23T03:19:23.505498+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:25:06.270537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.501561+0000","last_change":"2023-09-23T03:19:23.501561+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.501561+0000","last_undegraded":"2023-09-23T03:19:23.501561+0000","last_fullsized":"2023-09-23T03:19:23.501561+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.511665+0000","last_change":"2023-09-23T03:19:23.501583+0000","last_active":"2023-09-23T03:19:23.511665+0000","last_peered":"2023-09-23T03:19:23.511665+0000","last_clean":"2023-09-23T03:19:23.511665+0000","last_became_active":"2023-09-23T03:19:23.501037+0000","last_became_peered":"2023-09-23T03:19:23.501037+0000","last_unstale":"2023-09-23T03:19:23.511665+0000","last_undegraded":"2023-09-23T03:19:23.511665+0000","last_fullsized":"2023-09-23T03:19:23.511665+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:11:18.322404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.499265+0000","last_change":"2023-09-23T03:19:23.499265+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.499265+0000","last_undegraded":"2023-09-23T03:19:23.499265+0000","last_fullsized":"2023-09-23T03:19:23.499265+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.512892+0000","last_change":"2023-09-23T03:19:23.507592+0000","last_active":"2023-09-23T03:19:23.512892+0000","last_peered":"2023-09-23T03:19:23.512892+0000","last_clean":"2023-09-23T03:19:23.512892+0000","last_became_active":"2023-09-23T03:19:23.507316+0000","last_became_peered":"2023-09-23T03:19:23.507316+0000","last_unstale":"2023-09-23T03:19:23.512892+0000","last_undegraded":"2023-09-23T03:19:23.512892+0000","last_fullsized":"2023-09-23T03:19:23.512892+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:43:34.054466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.487374+0000","last_change":"2023-09-23T03:19:23.487374+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.487374+0000","last_undegraded":"2023-09-23T03:19:23.487374+0000","last_fullsized":"2023-09-23T03:19:23.487374+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.524603+0000","last_change":"2023-09-23T03:19:23.515237+0000","last_active":"2023-09-23T03:19:23.524603+0000","last_peered":"2023-09-23T03:19:23.524603+0000","last_clean":"2023-09-23T03:19:23.524603+0000","last_became_active":"2023-09-23T03:19:23.514946+0000","last_became_peered":"2023-09-23T03:19:23.514946+0000","last_unstale":"2023-09-23T03:19:23.524603+0000","last_undegraded":"2023-09-23T03:19:23.524603+0000","last_fullsized":"2023-09-23T03:19:23.524603+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:49:32.205787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.497140+0000","last_change":"2023-09-23T03:19:23.497140+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.497140+0000","last_undegraded":"2023-09-23T03:19:23.497140+0000","last_fullsized":"2023-09-23T03:19:23.497140+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.524349+0000","last_change":"2023-09-23T03:19:23.519665+0000","last_active":"2023-09-23T03:19:23.524349+0000","last_peered":"2023-09-23T03:19:23.524349+0000","last_clean":"2023-09-23T03:19:23.524349+0000","last_became_active":"2023-09-23T03:19:23.519393+0000","last_became_peered":"2023-09-23T03:19:23.519393+0000","last_unstale":"2023-09-23T03:19:23.524349+0000","last_undegraded":"2023-09-23T03:19:23.524349+0000","last_fullsized":"2023-09-23T03:19:23.524349+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:49:32.205787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.499552+0000","last_change":"2023-09-23T03:19:23.499552+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.499552+0000","last_undegraded":"2023-09-23T03:19:23.499552+0000","last_fullsized":"2023-09-23T03:19:23.499552+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.507040+0000","last_change":"2023-09-23T03:19:23.502992+0000","last_active":"2023-09-23T03:19:23.507040+0000","last_peered":"2023-09-23T03:19:23.507040+0000","last_clean":"2023-09-23T03:19:23.507040+0000","last_became_active":"2023-09-23T03:19:23.502688+0000","last_became_peered":"2023-09-23T03:19:23.502688+0000","last_unstale":"2023-09-23T03:19:23.507040+0000","last_undegraded":"2023-09-23T03:19:23.507040+0000","last_fullsized":"2023-09-23T03:19:23.507040+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:05:11.471065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.503106+0000","last_change":"2023-09-23T03:19:23.503106+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.503106+0000","last_undegraded":"2023-09-23T03:19:23.503106+0000","last_fullsized":"2023-09-23T03:19:23.503106+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.511859+0000","last_change":"2023-09-23T03:19:23.507612+0000","last_active":"2023-09-23T03:19:23.511859+0000","last_peered":"2023-09-23T03:19:23.511859+0000","last_clean":"2023-09-23T03:19:23.511859+0000","last_became_active":"2023-09-23T03:19:23.507211+0000","last_became_peered":"2023-09-23T03:19:23.507211+0000","last_unstale":"2023-09-23T03:19:23.511859+0000","last_undegraded":"2023-09-23T03:19:23.511859+0000","last_fullsized":"2023-09-23T03:19:23.511859+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:33:16.276149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.487706+0000","last_change":"2023-09-23T03:19:23.487706+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.487706+0000","last_undegraded":"2023-09-23T03:19:23.487706+0000","last_fullsized":"2023-09-23T03:19:23.487706+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.525158+0000","last_change":"2023-09-23T03:19:23.518829+0000","last_active":"2023-09-23T03:19:23.525158+0000","last_peered":"2023-09-23T03:19:23.525158+0000","last_clean":"2023-09-23T03:19:23.525158+0000","last_became_active":"2023-09-23T03:19:23.518512+0000","last_became_peered":"2023-09-23T03:19:23.518512+0000","last_unstale":"2023-09-23T03:19:23.525158+0000","last_undegraded":"2023-09-23T03:19:23.525158+0000","last_fullsized":"2023-09-23T03:19:23.525158+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:23:19.447543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.497713+0000","last_change":"2023-09-23T03:19:23.497713+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.497713+0000","last_undegraded":"2023-09-23T03:19:23.497713+0000","last_fullsized":"2023-09-23T03:19:23.497713+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.508037+0000","last_change":"2023-09-23T03:19:23.500147+0000","last_active":"2023-09-23T03:19:23.508037+0000","last_peered":"2023-09-23T03:19:23.508037+0000","last_clean":"2023-09-23T03:19:23.508037+0000","last_became_active":"2023-09-23T03:19:23.499826+0000","last_became_peered":"2023-09-23T03:19:23.499826+0000","last_unstale":"2023-09-23T03:19:23.508037+0000","last_undegraded":"2023-09-23T03:19:23.508037+0000","last_fullsized":"2023-09-23T03:19:23.508037+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:09:18.859490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.500650+0000","last_change":"2023-09-23T03:19:23.500650+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.500650+0000","last_undegraded":"2023-09-23T03:19:23.500650+0000","last_fullsized":"2023-09-23T03:19:23.500650+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.507070+0000","last_change":"2023-09-23T03:19:23.502333+0000","last_active":"2023-09-23T03:19:23.507070+0000","last_peered":"2023-09-23T03:19:23.507070+0000","last_clean":"2023-09-23T03:19:23.507070+0000","last_became_active":"2023-09-23T03:19:23.501997+0000","last_became_peered":"2023-09-23T03:19:23.501997+0000","last_unstale":"2023-09-23T03:19:23.507070+0000","last_undegraded":"2023-09-23T03:19:23.507070+0000","last_fullsized":"2023-09-23T03:19:23.507070+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:22:22.759701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.490693+0000","last_change":"2023-09-23T03:19:23.490693+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.490693+0000","last_undegraded":"2023-09-23T03:19:23.490693+0000","last_fullsized":"2023-09-23T03:19:23.490693+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.512081+0000","last_change":"2023-09-23T03:19:23.507705+0000","last_active":"2023-09-23T03:19:23.512081+0000","last_peered":"2023-09-23T03:19:23.512081+0000","last_clean":"2023-09-23T03:19:23.512081+0000","last_became_active":"2023-09-23T03:19:23.507409+0000","last_became_peered":"2023-09-23T03:19:23.507409+0000","last_unstale":"2023-09-23T03:19:23.512081+0000","last_undegraded":"2023-09-23T03:19:23.512081+0000","last_fullsized":"2023-09-23T03:19:23.512081+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:43:34.054466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.488782+0000","last_change":"2023-09-23T03:19:23.488782+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.488782+0000","last_undegraded":"2023-09-23T03:19:23.488782+0000","last_fullsized":"2023-09-23T03:19:23.488782+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.524549+0000","last_change":"2023-09-23T03:19:23.514960+0000","last_active":"2023-09-23T03:19:23.524549+0000","last_peered":"2023-09-23T03:19:23.524549+0000","last_clean":"2023-09-23T03:19:23.524549+0000","last_became_active":"2023-09-23T03:19:23.514644+0000","last_became_peered":"2023-09-23T03:19:23.514644+0000","last_unstale":"2023-09-23T03:19:23.524549+0000","last_undegraded":"2023-09-23T03:19:23.524549+0000","last_fullsized":"2023-09-23T03:19:23.524549+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:00:17.646758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.502007+0000","last_change":"2023-09-23T03:19:23.502007+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.502007+0000","last_undegraded":"2023-09-23T03:19:23.502007+0000","last_fullsized":"2023-09-23T03:19:23.502007+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.515734+0000","last_change":"2023-09-23T03:19:23.512352+0000","last_active":"2023-09-23T03:19:23.515734+0000","last_peered":"2023-09-23T03:19:23.515734+0000","last_clean":"2023-09-23T03:19:23.515734+0000","last_became_active":"2023-09-23T03:19:23.512039+0000","last_became_peered":"2023-09-23T03:19:23.512039+0000","last_unstale":"2023-09-23T03:19:23.515734+0000","last_undegraded":"2023-09-23T03:19:23.515734+0000","last_fullsized":"2023-09-23T03:19:23.515734+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:43:42.533621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.502444+0000","last_change":"2023-09-23T03:19:23.502444+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.502444+0000","last_undegraded":"2023-09-23T03:19:23.502444+0000","last_fullsized":"2023-09-23T03:19:23.502444+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.511839+0000","last_change":"2023-09-23T03:19:23.506342+0000","last_active":"2023-09-23T03:19:23.511839+0000","last_peered":"2023-09-23T03:19:23.511839+0000","last_clean":"2023-09-23T03:19:23.511839+0000","last_became_active":"2023-09-23T03:19:23.506078+0000","last_became_peered":"2023-09-23T03:19:23.506078+0000","last_unstale":"2023-09-23T03:19:23.511839+0000","last_undegraded":"2023-09-23T03:19:23.511839+0000","last_fullsized":"2023-09-23T03:19:23.511839+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:00:17.646758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.500536+0000","last_change":"2023-09-23T03:19:23.500536+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.500536+0000","last_undegraded":"2023-09-23T03:19:23.500536+0000","last_fullsized":"2023-09-23T03:19:23.500536+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.516019+0000","last_change":"2023-09-23T03:19:23.510497+0000","last_active":"2023-09-23T03:19:23.516019+0000","last_peered":"2023-09-23T03:19:23.516019+0000","last_clean":"2023-09-23T03:19:23.516019+0000","last_became_active":"2023-09-23T03:19:23.510155+0000","last_became_peered":"2023-09-23T03:19:23.510155+0000","last_unstale":"2023-09-23T03:19:23.516019+0000","last_undegraded":"2023-09-23T03:19:23.516019+0000","last_fullsized":"2023-09-23T03:19:23.516019+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:15:52.148791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.491878+0000","last_change":"2023-09-23T03:19:23.491878+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.491878+0000","last_undegraded":"2023-09-23T03:19:23.491878+0000","last_fullsized":"2023-09-23T03:19:23.491878+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.524455+0000","last_change":"2023-09-23T03:19:23.513693+0000","last_active":"2023-09-23T03:19:23.524455+0000","last_peered":"2023-09-23T03:19:23.524455+0000","last_clean":"2023-09-23T03:19:23.524455+0000","last_became_active":"2023-09-23T03:19:23.513359+0000","last_became_peered":"2023-09-23T03:19:23.513359+0000","last_unstale":"2023-09-23T03:19:23.524455+0000","last_undegraded":"2023-09-23T03:19:23.524455+0000","last_fullsized":"2023-09-23T03:19:23.524455+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:09:18.859490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.487319+0000","last_change":"2023-09-23T03:19:23.487319+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.487319+0000","last_undegraded":"2023-09-23T03:19:23.487319+0000","last_fullsized":"2023-09-23T03:19:23.487319+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.519999+0000","last_change":"2023-09-23T03:19:23.519087+0000","last_active":"2023-09-23T03:19:23.519999+0000","last_peered":"2023-09-23T03:19:23.519999+0000","last_clean":"2023-09-23T03:19:23.519999+0000","last_became_active":"2023-09-23T03:19:23.518827+0000","last_became_peered":"2023-09-23T03:19:23.518827+0000","last_unstale":"2023-09-23T03:19:23.519999+0000","last_undegraded":"2023-09-23T03:19:23.519999+0000","last_fullsized":"2023-09-23T03:19:23.519999+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:15:52.148791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.498413+0000","last_change":"2023-09-23T03:19:23.498413+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.498413+0000","last_undegraded":"2023-09-23T03:19:23.498413+0000","last_fullsized":"2023-09-23T03:19:23.498413+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.502654+0000","last_change":"2023-09-23T03:19:23.498946+0000","last_active":"2023-09-23T03:19:23.502654+0000","last_peered":"2023-09-23T03:19:23.502654+0000","last_clean":"2023-09-23T03:19:23.502654+0000","last_became_active":"2023-09-23T03:19:23.498522+0000","last_became_peered":"2023-09-23T03:19:23.498522+0000","last_unstale":"2023-09-23T03:19:23.502654+0000","last_undegraded":"2023-09-23T03:19:23.502654+0000","last_fullsized":"2023-09-23T03:19:23.502654+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:39:49.215616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.499307+0000","last_change":"2023-09-23T03:19:23.499307+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.499307+0000","last_undegraded":"2023-09-23T03:19:23.499307+0000","last_fullsized":"2023-09-23T03:19:23.499307+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.512203+0000","last_change":"2023-09-23T03:19:23.509484+0000","last_active":"2023-09-23T03:19:23.512203+0000","last_peered":"2023-09-23T03:19:23.512203+0000","last_clean":"2023-09-23T03:19:23.512203+0000","last_became_active":"2023-09-23T03:19:23.509203+0000","last_became_peered":"2023-09-23T03:19:23.509203+0000","last_unstale":"2023-09-23T03:19:23.512203+0000","last_undegraded":"2023-09-23T03:19:23.512203+0000","last_fullsized":"2023-09-23T03:19:23.512203+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:10:34.290176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.487164+0000","last_change":"2023-09-23T03:19:23.487164+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.487164+0000","last_undegraded":"2023-09-23T03:19:23.487164+0000","last_fullsized":"2023-09-23T03:19:23.487164+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.518826+0000","last_change":"2023-09-23T03:19:23.513953+0000","last_active":"2023-09-23T03:19:23.518826+0000","last_peered":"2023-09-23T03:19:23.518826+0000","last_clean":"2023-09-23T03:19:23.518826+0000","last_became_active":"2023-09-23T03:19:23.513616+0000","last_became_peered":"2023-09-23T03:19:23.513616+0000","last_unstale":"2023-09-23T03:19:23.518826+0000","last_undegraded":"2023-09-23T03:19:23.518826+0000","last_fullsized":"2023-09-23T03:19:23.518826+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:23:05.866775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.487239+0000","last_change":"2023-09-23T03:19:23.487239+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.487239+0000","last_undegraded":"2023-09-23T03:19:23.487239+0000","last_fullsized":"2023-09-23T03:19:23.487239+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.524435+0000","last_change":"2023-09-23T03:19:23.519480+0000","last_active":"2023-09-23T03:19:23.524435+0000","last_peered":"2023-09-23T03:19:23.524435+0000","last_clean":"2023-09-23T03:19:23.524435+0000","last_became_active":"2023-09-23T03:19:23.519191+0000","last_became_peered":"2023-09-23T03:19:23.519191+0000","last_unstale":"2023-09-23T03:19:23.524435+0000","last_undegraded":"2023-09-23T03:19:23.524435+0000","last_fullsized":"2023-09-23T03:19:23.524435+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:43:42.533621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":198,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-09-23T03:19:22.484979+0000","last_change":"2023-09-23T03:15:11.042447+0000","last_active":"2023-09-23T03:19:22.484979+0000","last_peered":"2023-09-23T03:19:22.484979+0000","last_clean":"2023-09-23T03:19:22.484979+0000","last_became_active":"2023-09-23T03:15:11.040322+0000","last_became_peered":"2023-09-23T03:15:11.040322+0000","last_unstale":"2023-09-23T03:19:22.484979+0000","last_undegraded":"2023-09-23T03:19:22.484979+0000","last_fullsized":"2023-09-23T03:19:22.484979+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_clean_scrub_stamp":"2023-09-23T03:15:09.474842+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:18:36.331645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574901,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1314816,"data_stored":923854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.98799999999999999,"15min":0.99199999999999999},"min":{"1min":0.68799999999999994,"5min":0.442,"15min":0.442},"max":{"1min":1.55,"5min":2.149,"15min":2.149},"last":1.075},{"interface":"front","average":{"1min":1.111,"5min":1.0569999999999999,"15min":1.0469999999999999},"min":{"1min":0.82099999999999995,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.681,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.95299999999999996}]},{"osd":1,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.198,"15min":1.22},"min":{"1min":0.74299999999999999,"5min":0.504,"15min":0.504},"max":{"1min":1.5649999999999999,"5min":3.1859999999999999,"15min":3.1859999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.248,"5min":1.294,"15min":1.3759999999999999},"min":{"1min":0.93899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.5840000000000001,"5min":2.613,"15min":2.613},"last":1.335}]},{"osd":2,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.1240000000000001,"15min":1.155},"min":{"1min":0.70199999999999996,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.6399999999999999,"5min":2.331,"15min":2.331},"last":0.879},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.242,"15min":1.296},"min":{"1min":0.75600000000000001,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.673,"5min":3.093,"15min":3.093},"last":1.3740000000000001}]},{"osd":3,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.248,"15min":1.2989999999999999},"min":{"1min":0.90000000000000002,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.7310000000000001,"5min":2.532,"15min":2.532},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.389,"5min":1.377,"15min":1.4410000000000001},"min":{"1min":0.95099999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.784,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":1.2829999999999999}]},{"osd":4,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.415,"15min":1.528},"min":{"1min":0.79200000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.7070000000000001,"5min":2.7210000000000001,"15min":2.7210000000000001},"last":1.5489999999999999},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.4730000000000001,"15min":1.556},"min":{"1min":0.81699999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.5880000000000001,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":1.5089999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.4179999999999999,"15min":1.4810000000000001},"min":{"1min":0.72699999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.736,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.288,"5min":1.4510000000000001,"15min":1.548},"min":{"1min":1.0069999999999999,"5min":0.497,"15min":0.497},"max":{"1min":1.7689999999999999,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":1.1819999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.5669999999999999,"15min":1.649},"min":{"1min":1.2410000000000001,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.8580000000000001,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":1.423},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.5089999999999999,"15min":1.607},"min":{"1min":1.002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.778,"5min":3.6739999999999999,"15min":3.6739999999999999},"last":1.012}]}]},{"osd":6,"up_from":13,"seq":55834574901,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":720896,"data_stored":333486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.048,"15min":1.0289999999999999},"min":{"1min":0.68899999999999995,"5min":0.38,"15min":0.38},"max":{"1min":1.6379999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.04},{"interface":"front","average":{"1min":1.083,"5min":1.0469999999999999,"15min":1.0549999999999999},"min":{"1min":0.61599999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.516,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.97899999999999998}]},{"osd":1,"last update":"Sat Sep 23 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.109,"15min":1.0980000000000001},"min":{"1min":0.65900000000000003,"5min":0.438,"15min":0.438},"max":{"1min":1.6830000000000001,"5min":1.867,"15min":1.867},"last":0.88},{"interface":"front","average":{"1min":1.23,"5min":1.2170000000000001,"15min":1.242},"min":{"1min":0.82399999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.6339999999999999,"5min":2.242,"15min":2.242},"last":1.3149999999999999}]},{"osd":2,"last update":"Sat Sep 23 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.1180000000000001,"15min":1.1060000000000001},"min":{"1min":0.82499999999999996,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4470000000000001,"5min":2.097,"15min":2.097},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.194,"5min":1.234,"15min":1.2669999999999999},"min":{"1min":0.66800000000000004,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.8149999999999999,"5min":1.903,"15min":1.903},"last":1.25}]},{"osd":3,"last update":"Sat Sep 23 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.3380000000000001,"15min":1.361},"min":{"1min":0.80200000000000005,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.8879999999999999,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":1.587},{"interface":"front","average":{"1min":1.278,"5min":1.389,"15min":1.4550000000000001},"min":{"1min":0.76600000000000001,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.657,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":1.2170000000000001}]},{"osd":4,"last update":"Sat Sep 23 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.397,"15min":1.395},"min":{"1min":0.81399999999999995,"5min":0.498,"15min":0.498},"max":{"1min":1.827,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.2849999999999999,"15min":1.294},"min":{"1min":0.75700000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.661,"5min":2.2109999999999999,"15min":2.2109999999999999},"last":0.84399999999999997}]},{"osd":5,"last update":"Sat Sep 23 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.391,"15min":1.395},"min":{"1min":0.84699999999999998,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.7390000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.4810000000000001,"15min":1.486},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.9390000000000001,"5min":4.758,"15min":4.758},"last":1.6779999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.508,"15min":1.5549999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.77,"5min":2.476,"15min":2.476},"last":1.151},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.532,"15min":1.5640000000000001},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.1739999999999999,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.472}]}]},{"osd":5,"up_from":13,"seq":55834574903,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":720896,"data_stored":333486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.044,"15min":1.03},"min":{"1min":0.80200000000000005,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.474,"5min":1.637,"15min":1.637},"last":1.075},{"interface":"front","average":{"1min":1.141,"5min":1.0609999999999999,"15min":1.032},"min":{"1min":0.78800000000000003,"5min":0.63,"15min":0.63},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":1.621}]},{"osd":1,"last update":"Sat Sep 23 03:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.212,"15min":1.224},"min":{"1min":0.95599999999999996,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.6459999999999999,"5min":1.849,"15min":1.849},"last":1.135},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.2869999999999999,"15min":1.3220000000000001},"min":{"1min":1.085,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.7410000000000001,"5min":1.893,"15min":1.893},"last":1.3759999999999999}]},{"osd":2,"last update":"Sat Sep 23 03:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.22,"15min":1.2170000000000001},"min":{"1min":1.0189999999999999,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.597,"5min":1.681,"15min":1.681},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.161,"5min":1.244,"15min":1.2969999999999999},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.5740000000000001,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":1.2090000000000001}]},{"osd":3,"last update":"Sat Sep 23 03:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.2509999999999999,"15min":1.361},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.423,"5min":1.905,"15min":1.905},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.3859999999999999,"15min":1.47},"min":{"1min":1.056,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.792,"5min":3.6110000000000002,"15min":3.6110000000000002},"last":1.4419999999999999}]},{"osd":4,"last update":"Sat Sep 23 03:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3939999999999999,"15min":1.444},"min":{"1min":0.83099999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.8500000000000001,"5min":2.069,"15min":2.069},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.401,"15min":1.4530000000000001},"min":{"1min":0.91100000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.7889999999999999,"5min":2.319,"15min":2.319},"last":1.3180000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":1.5089999999999999,"15min":1.5409999999999999},"min":{"1min":0.69899999999999995,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.948,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":1.6739999999999999},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.5309999999999999,"15min":1.613},"min":{"1min":0.97099999999999997,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.877,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.5760000000000001}]},{"osd":7,"last update":"Sat Sep 23 03:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.4570000000000001,"15min":1.538},"min":{"1min":0.91600000000000004,"5min":0.626,"15min":0.626},"max":{"1min":1.6839999999999999,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.734},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.585,"15min":1.643},"min":{"1min":1.046,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.7330000000000001,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.417}]}]},{"osd":4,"up_from":13,"seq":55834574902,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":708608,"data_stored":326899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:19:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.1279999999999999,"15min":1.1379999999999999},"min":{"1min":0.64800000000000002,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.639,"5min":1.806,"15min":1.806},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.1739999999999999,"15min":1.141},"min":{"1min":0.76500000000000001,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.905,"5min":1.9179999999999999,"15min":1.9179999999999999},"last":1.6479999999999999}]},{"osd":1,"last update":"Sat Sep 23 03:19:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.2130000000000001,"15min":1.2150000000000001},"min":{"1min":0.90200000000000002,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1910000000000001,"15min":1.2210000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.4710000000000001,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":0.97799999999999998}]},{"osd":2,"last update":"Sat Sep 23 03:19:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.1870000000000001,"15min":1.246},"min":{"1min":0.71199999999999997,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.129,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.246,"15min":1.224},"min":{"1min":0.91200000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.909,"5min":1.909,"15min":1.909},"last":0.90000000000000002}]},{"osd":3,"last update":"Sat Sep 23 03:19:15 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.1479999999999999,"15min":1.171},"min":{"1min":0.71999999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.5369999999999999,"5min":1.583,"15min":1.583},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.296,"5min":1.319,"15min":1.341},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.1779999999999999,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.302}]},{"osd":5,"last update":"Sat Sep 23 03:19:15 2023","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.371,"15min":1.345},"min":{"1min":0.99299999999999999,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.8050000000000002,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.371},{"interface":"front","average":{"1min":1.371,"5min":1.3300000000000001,"15min":1.3260000000000001},"min":{"1min":0.84299999999999997,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":2.887,"5min":2.887,"15min":2.887},"last":1.4690000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:19:15 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.397,"15min":1.391},"min":{"1min":1.048,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.4249999999999998,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.6619999999999999,"5min":1.5049999999999999,"15min":1.478},"min":{"1min":1.323,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":2.052,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.75}]},{"osd":7,"last update":"Sat Sep 23 03:19:15 2023","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.4650000000000001,"15min":1.4730000000000001},"min":{"1min":1.1819999999999999,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":2.5680000000000001,"5min":2.5680000000000001,"15min":2.5680000000000001},"last":1.716},{"interface":"front","average":{"1min":1.6379999999999999,"5min":1.5580000000000001,"15min":1.5649999999999999},"min":{"1min":1.056,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":2.4769999999999999,"5min":2.4769999999999999,"15min":2.4769999999999999},"last":1.593}]}]},{"osd":3,"up_from":13,"seq":55834574902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":708608,"data_stored":326899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.85399999999999998,"15min":0.875},"min":{"1min":0.42199999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.7090000000000001,"5min":1.744,"15min":1.744},"last":1.478},{"interface":"front","average":{"1min":0.876,"5min":0.86599999999999999,"15min":0.90200000000000002},"min":{"1min":0.51600000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.341,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.94899999999999995}]},{"osd":1,"last update":"Sat Sep 23 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.90700000000000003,"15min":0.96399999999999997},"min":{"1min":0.442,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.5640000000000001,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":1.401},{"interface":"front","average":{"1min":1.02,"5min":0.98099999999999998,"15min":0.98499999999999999},"min":{"1min":0.54700000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.3839999999999999,"5min":1.704,"15min":1.704},"last":1.127}]},{"osd":2,"last update":"Sat Sep 23 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.0609999999999999,"15min":1.113},"min":{"1min":0.54800000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.639,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.639},{"interface":"front","average":{"1min":1.165,"5min":1.097,"15min":1.129},"min":{"1min":0.64400000000000002,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.6830000000000001,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.6830000000000001}]},{"osd":4,"last update":"Sat Sep 23 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.2809999999999999,"15min":1.333},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.0099999999999998,"5min":2.698,"15min":2.698},"last":2.0099999999999998},{"interface":"front","average":{"1min":1.353,"5min":1.381,"15min":1.4470000000000001},"min":{"1min":0.95399999999999996,"5min":0.752,"15min":0.752},"max":{"1min":1.825,"5min":2.419,"15min":2.419},"last":1.825}]},{"osd":5,"last update":"Sat Sep 23 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.339,"15min":1.361},"min":{"1min":0.90300000000000002,"5min":0.753,"15min":0.753},"max":{"1min":1.8799999999999999,"5min":1.917,"15min":1.917},"last":1.867},{"interface":"front","average":{"1min":1.321,"5min":1.3049999999999999,"15min":1.3620000000000001},"min":{"1min":0.83499999999999996,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.716,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.716}]},{"osd":6,"last update":"Sat Sep 23 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.359,"15min":1.423},"min":{"1min":0.82699999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.9550000000000001,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":1.9550000000000001},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4430000000000001,"15min":1.482},"min":{"1min":1.05,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.7729999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.417,"15min":1.5009999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":2.2120000000000002,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.909},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.4399999999999999,"15min":1.526},"min":{"1min":0.85799999999999998,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.2029999999999998,"5min":2.5379999999999998,"15min":2.5379999999999998},"last":2.2029999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":720896,"data_stored":333486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.83799999999999997,"15min":0.83699999999999997},"min":{"1min":0.442,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":0.501},{"interface":"front","average":{"1min":0.998,"5min":0.86199999999999999,"15min":0.84299999999999997},"min":{"1min":0.48899999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.5329999999999999,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.77200000000000002}]},{"osd":1,"last update":"Sat Sep 23 03:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.84799999999999998,"15min":0.85999999999999999},"min":{"1min":0.55800000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.026,"5min":0.93300000000000005,"15min":0.94099999999999995},"min":{"1min":0.55100000000000005,"5min":0.38,"15min":0.38},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.95999999999999996}]},{"osd":3,"last update":"Sat Sep 23 03:19:12 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":0.96599999999999997,"15min":0.95199999999999996},"min":{"1min":0.53900000000000003,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.4279999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.073,"5min":0.98499999999999999,"15min":0.997},"min":{"1min":0.60699999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.534,"5min":1.78,"15min":1.78},"last":0.89900000000000002}]},{"osd":4,"last update":"Sat Sep 23 03:19:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2729999999999999,"15min":1.3240000000000001},"min":{"1min":0.86199999999999999,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.819,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.071},{"interface":"front","average":{"1min":1.296,"5min":1.2969999999999999,"15min":1.341},"min":{"1min":0.77600000000000002,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.631,"5min":2.234,"15min":2.234},"last":1.0029999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:19:12 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.339,"15min":1.409},"min":{"1min":0.97499999999999998,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.7849999999999999,"5min":3.9409999999999998,"15min":3.9409999999999998},"last":1.032},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.4199999999999999,"15min":1.5329999999999999},"min":{"1min":0.81299999999999994,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.9359999999999999,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":1.1759999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:19:12 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.3580000000000001,"15min":1.417},"min":{"1min":0.83299999999999996,"5min":0.755,"15min":0.755},"max":{"1min":1.8540000000000001,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.458,"5min":1.393,"15min":1.478},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.1680000000000001,"5min":4.5800000000000001,"15min":4.5800000000000001},"last":1.1279999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:19:12 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.3600000000000001,"15min":1.4019999999999999},"min":{"1min":1.0469999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.996,"5min":2.3149999999999999,"15min":2.3149999999999999},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.413,"15min":1.4550000000000001},"min":{"1min":0.873,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.9259999999999999,"5min":2.621,"15min":2.621},"last":1.109}]}]},{"osd":1,"up_from":13,"seq":55834574904,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":720896,"data_stored":333486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.80000000000000004,"15min":0.80800000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.2490000000000001,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.85099999999999998},"min":{"1min":0.34799999999999998,"5min":0.312,"15min":0.312},"max":{"1min":1.2210000000000001,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":0.623}]},{"osd":2,"last update":"Sat Sep 23 03:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.84999999999999998,"15min":0.89400000000000002},"min":{"1min":0.52000000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.409,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":1.278},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.004,"15min":1.0369999999999999},"min":{"1min":0.54500000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5600000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.524}]},{"osd":3,"last update":"Sat Sep 23 03:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.0269999999999999,"15min":1.087},"min":{"1min":0.48799999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.5389999999999999,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":1.24},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.0980000000000001,"15min":1.1619999999999999},"min":{"1min":0.56200000000000006,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.7649999999999999,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":1.6850000000000001}]},{"osd":4,"last update":"Sat Sep 23 03:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.2330000000000001,"15min":1.2829999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.7270000000000001,"5min":3.2629999999999999,"15min":3.2629999999999999},"last":1.411},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.262,"15min":1.3},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6359999999999999,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":1.456}]},{"osd":5,"last update":"Sat Sep 23 03:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.254,"15min":1.2989999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.9319999999999999,"5min":3.161,"15min":3.161},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3,"15min":1.3460000000000001},"min":{"1min":0.80400000000000005,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.698,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":1.5840000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.2969999999999999,"15min":1.3400000000000001},"min":{"1min":0.76500000000000001,"5min":0.747,"15min":0.747},"max":{"1min":2.0329999999999999,"5min":3.0179999999999998,"15min":3.0179999999999998},"last":1.637},{"interface":"front","average":{"1min":1.333,"5min":1.3520000000000001,"15min":1.3979999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.7909999999999999,"5min":3.3580000000000001,"15min":3.3580000000000001},"last":1.78}]},{"osd":7,"last update":"Sat Sep 23 03:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.363,"15min":1.413},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":2.085,"5min":3.1099999999999999,"15min":3.1099999999999999},"last":1.6619999999999999},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.415,"15min":1.4810000000000001},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.7330000000000001,"5min":3.306,"15min":3.306},"last":1.722}]}]},{"osd":0,"up_from":13,"seq":55834574904,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1310720,"data_stored":922835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.81200000000000006,"15min":0.81999999999999995},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.3180000000000001,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.86599999999999999,"15min":0.89300000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.363,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.96399999999999997}]},{"osd":2,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.81100000000000005,"15min":0.80300000000000005},"min":{"1min":0.48599999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.22,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.97299999999999998,"15min":0.98499999999999999},"min":{"1min":0.46700000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":1.3029999999999999}]},{"osd":3,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93300000000000005,"15min":0.91500000000000004},"min":{"1min":0.44800000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.226,"5min":1.591,"15min":1.591},"last":1.198},{"interface":"front","average":{"1min":1.117,"5min":0.996,"15min":1.006},"min":{"1min":0.57299999999999995,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.4990000000000001,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.147}]},{"osd":4,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.204,"15min":1.204},"min":{"1min":0.81499999999999995,"5min":0.755,"15min":0.755},"max":{"1min":1.601,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.7450000000000001},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.2410000000000001,"15min":1.274},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.4610000000000001,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.456}]},{"osd":5,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.2250000000000001,"15min":1.2290000000000001},"min":{"1min":0.872,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.6739999999999999,"5min":1.867,"15min":1.867},"last":1.474},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.294,"15min":1.272},"min":{"1min":0.76400000000000001,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":1.8080000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.3009999999999999,"15min":1.3029999999999999},"min":{"1min":0.99399999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":1.552},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.325,"15min":1.3029999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.875,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.873}]},{"osd":7,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.329,"15min":1.3160000000000001},"min":{"1min":0.93300000000000005,"5min":0.747,"15min":0.747},"max":{"1min":1.6499999999999999,"5min":1.944,"15min":1.944},"last":1.528},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.3979999999999999,"15min":1.4059999999999999},"min":{"1min":0.91800000000000004,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.756,"5min":2.048,"15min":2.048},"last":1.7170000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-23T03:19:29.455 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-23T03:19:29.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.712+0000 7f0c0a900640 1 Processor -- start 2023-09-23T03:19:29.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.712+0000 7f0c0a900640 1 -- start start 2023-09-23T03:19:29.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.713+0000 7f0c0a900640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c040ada30 0x7f0c040ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:29.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.713+0000 7f0c0a900640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0c040b2fe0 0x7f0c040cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:29.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.713+0000 7f0c0a900640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0c040cef60 0x7f0c040b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:29.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.713+0000 7f0c0a900640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c040ade10 con 0x7f0c040cef60 2023-09-23T03:19:29.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.713+0000 7f0c0a900640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c0406ceb0 con 0x7f0c040b2fe0 2023-09-23T03:19:29.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.713+0000 7f0c0a900640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c040cf3b0 con 0x7f0c040ada30 2023-09-23T03:19:29.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.714+0000 7f0c08e9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0c040cef60 0x7f0c040b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:29.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.714+0000 7f0c08e9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0c040cef60 0x7f0c040b18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43054/0 (socket says 172.21.15.154:43054) 2023-09-23T03:19:29.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.714+0000 7f0c08e9d640 1 -- 172.21.15.154:0/45211462 learned_addr learned my addr 172.21.15.154:0/45211462 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:29.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.714+0000 7f0c03fff640 1 --2- 172.21.15.154:0/45211462 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c040ada30 0x7f0c040ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:29.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.714+0000 7f0c037fe640 1 --2- 172.21.15.154:0/45211462 >> v2:172.21.15.190:3300/0 conn(0x7f0c040b2fe0 0x7f0c040cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:29.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.714+0000 7f0c08e9d640 1 -- 172.21.15.154:0/45211462 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c040ada30 msgr2=0x7f0c040ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:29.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.714+0000 7f0c08e9d640 1 --2- 172.21.15.154:0/45211462 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c040ada30 0x7f0c040ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:29.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.714+0000 7f0c08e9d640 1 -- 172.21.15.154:0/45211462 >> v2:172.21.15.190:3300/0 conn(0x7f0c040b2fe0 msgr2=0x7f0c040cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:29.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.714+0000 7f0c08e9d640 1 --2- 172.21.15.154:0/45211462 >> v2:172.21.15.190:3300/0 conn(0x7f0c040b2fe0 0x7f0c040cea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:29.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.714+0000 7f0c08e9d640 1 -- 172.21.15.154:0/45211462 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c04159c90 con 0x7f0c040cef60 2023-09-23T03:19:29.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.714+0000 7f0c03fff640 1 --2- 172.21.15.154:0/45211462 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c040ada30 0x7f0c040ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:29.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.715+0000 7f0c08e9d640 1 --2- 172.21.15.154:0/45211462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c040cef60 0x7f0c040b18f0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f0c0406e4f0 tx=0x7f0bfc02e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b93413712f2078c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:29.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.716+0000 7f0c02ffd640 1 -- 172.21.15.154:0/45211462 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc02ea80 con 0x7f0c040cef60 2023-09-23T03:19:29.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.716+0000 7f0c02ffd640 1 -- 172.21.15.154:0/45211462 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bfc02ebe0 con 0x7f0c040cef60 2023-09-23T03:19:29.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.716+0000 7f0c02ffd640 1 -- 172.21.15.154:0/45211462 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc009600 con 0x7f0c040cef60 2023-09-23T03:19:29.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.716+0000 7f0c0a900640 1 -- 172.21.15.154:0/45211462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c040cef60 msgr2=0x7f0c040b18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:29.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.716+0000 7f0c0a900640 1 --2- 172.21.15.154:0/45211462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c040cef60 0x7f0c040b18f0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f0c0406e4f0 tx=0x7f0bfc02e0e0 comp rx=0 tx=0).stop 2023-09-23T03:19:29.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.716+0000 7f0c0a900640 1 -- 172.21.15.154:0/45211462 shutdown_connections 2023-09-23T03:19:29.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.716+0000 7f0c0a900640 1 --2- 172.21.15.154:0/45211462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c040cef60 0x7f0c040b18f0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:29.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.716+0000 7f0c0a900640 1 --2- 172.21.15.154:0/45211462 >> v2:172.21.15.190:3300/0 conn(0x7f0c040b2fe0 0x7f0c040cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:29.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.716+0000 7f0c0a900640 1 --2- 172.21.15.154:0/45211462 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c040ada30 0x7f0c040ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:29.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.716+0000 7f0c0a900640 1 -- 172.21.15.154:0/45211462 >> 172.21.15.154:0/45211462 conn(0x7f0c0406a0f0 msgr2=0x7f0c0406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:29.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.717+0000 7f0c0a900640 1 -- 172.21.15.154:0/45211462 shutdown_connections 2023-09-23T03:19:29.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.717+0000 7f0c0a900640 1 -- 172.21.15.154:0/45211462 wait complete. 2023-09-23T03:19:29.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.718+0000 7f0c0a900640 1 Processor -- start 2023-09-23T03:19:29.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.718+0000 7f0c0a900640 1 -- start start 2023-09-23T03:19:29.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.719+0000 7f0c0a900640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c040ada30 0x7f0c04156270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:29.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.719+0000 7f0c0a900640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c040b2fe0 0x7f0c0415c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:29.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.719+0000 7f0c0a900640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c040cef60 0x7f0c040c35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:29.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.719+0000 7f0c0a900640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c040ad0c0 con 0x7f0c040b2fe0 2023-09-23T03:19:29.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.719+0000 7f0c0a900640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c040c4270 con 0x7f0c040cef60 2023-09-23T03:19:29.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.719+0000 7f0c0a900640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c040c4550 con 0x7f0c040ada30 2023-09-23T03:19:29.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.720+0000 7f0c037fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c040b2fe0 0x7f0c0415c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:29.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.720+0000 7f0c037fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c040b2fe0 0x7f0c0415c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43058/0 (socket says 172.21.15.154:43058) 2023-09-23T03:19:29.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.720+0000 7f0c037fe640 1 -- 172.21.15.154:0/3769618243 learned_addr learned my addr 172.21.15.154:0/3769618243 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:29.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.720+0000 7f0c03fff640 1 --2- 172.21.15.154:0/3769618243 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c040ada30 0x7f0c04156270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:29.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.720+0000 7f0c037fe640 1 -- 172.21.15.154:0/3769618243 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c040ada30 msgr2=0x7f0c04156270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:29.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.720+0000 7f0c037fe640 1 --2- 172.21.15.154:0/3769618243 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c040ada30 0x7f0c04156270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:29.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.720+0000 7f0c037fe640 1 -- 172.21.15.154:0/3769618243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c040cef60 msgr2=0x7f0c040c35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:29.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.720+0000 7f0c037fe640 1 --2- 172.21.15.154:0/3769618243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c040cef60 0x7f0c040c35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:29.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.720+0000 7f0c037fe640 1 -- 172.21.15.154:0/3769618243 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bfc00bd40 con 0x7f0c040b2fe0 2023-09-23T03:19:29.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.720+0000 7f0c03fff640 1 --2- 172.21.15.154:0/3769618243 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c040ada30 0x7f0c04156270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:29.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.721+0000 7f0c037fe640 1 --2- 172.21.15.154:0/3769618243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c040b2fe0 0x7f0c0415c580 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800c7f0 tx=0x7f0bf800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:29.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.721+0000 7f0c00ff9640 1 -- 172.21.15.154:0/3769618243 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8004220 con 0x7f0c040b2fe0 2023-09-23T03:19:29.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.721+0000 7f0c00ff9640 1 -- 172.21.15.154:0/3769618243 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf8004380 con 0x7f0c040b2fe0 2023-09-23T03:19:29.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.721+0000 7f0c00ff9640 1 -- 172.21.15.154:0/3769618243 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8007840 con 0x7f0c040b2fe0 2023-09-23T03:19:29.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.721+0000 7f0c0a900640 1 -- 172.21.15.154:0/3769618243 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c040c4830 con 0x7f0c040b2fe0 2023-09-23T03:19:29.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.721+0000 7f0c0a900640 1 -- 172.21.15.154:0/3769618243 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c040c4df0 con 0x7f0c040b2fe0 2023-09-23T03:19:29.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.722+0000 7f0c0a900640 1 -- 172.21.15.154:0/3769618243 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c04073eb0 con 0x7f0c040b2fe0 2023-09-23T03:19:29.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.723+0000 7f0c00ff9640 1 -- 172.21.15.154:0/3769618243 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f0bf8007bb0 con 0x7f0c040b2fe0 2023-09-23T03:19:29.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.724+0000 7f0c00ff9640 1 --2- 172.21.15.154:0/3769618243 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0bf006af30 0x7f0bf006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:29.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.724+0000 7f0c00ff9640 1 -- 172.21.15.154:0/3769618243 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0bf8089890 con 0x7f0c040b2fe0 2023-09-23T03:19:29.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.724+0000 7f0c03fff640 1 --2- 172.21.15.154:0/3769618243 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0bf006af30 0x7f0bf006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:29.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.725+0000 7f0c03fff640 1 --2- 172.21.15.154:0/3769618243 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0bf006af30 0x7f0bf006d3f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4006fd0 tx=0x7f0bf4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:29.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.728+0000 7f0c00ff9640 1 -- 172.21.15.154:0/3769618243 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f0bf80546f0 con 0x7f0c040b2fe0 2023-09-23T03:19:29.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.913+0000 7f0c0a900640 1 -- 172.21.15.154:0/3769618243 --> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0c040c46c0 con 0x7f0bf006af30 2023-09-23T03:19:29.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.931+0000 7f0c00ff9640 1 -- 172.21.15.154:0/3769618243 <== mgr.4110 v2:172.21.15.154:6832/2422652104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+281110 (secure 0 0 0) 0x7f0c040c46c0 con 0x7f0bf006af30 2023-09-23T03:19:29.934 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:19:29.937 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2023-09-23T03:19:29.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.938+0000 7f0c0a900640 1 -- 172.21.15.154:0/3769618243 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0bf006af30 msgr2=0x7f0bf006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:29.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.938+0000 7f0c0a900640 1 --2- 172.21.15.154:0/3769618243 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0bf006af30 0x7f0bf006d3f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4006fd0 tx=0x7f0bf4008040 comp rx=0 tx=0).stop 2023-09-23T03:19:29.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.938+0000 7f0c0a900640 1 -- 172.21.15.154:0/3769618243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c040b2fe0 msgr2=0x7f0c0415c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:29.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.938+0000 7f0c0a900640 1 --2- 172.21.15.154:0/3769618243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c040b2fe0 0x7f0c0415c580 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800c7f0 tx=0x7f0bf800ccb0 comp rx=0 tx=0).stop 2023-09-23T03:19:29.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.938+0000 7f0c0a900640 1 -- 172.21.15.154:0/3769618243 shutdown_connections 2023-09-23T03:19:29.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.938+0000 7f0c0a900640 1 --2- 172.21.15.154:0/3769618243 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0bf006af30 0x7f0bf006d3f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:29.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.938+0000 7f0c0a900640 1 --2- 172.21.15.154:0/3769618243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c040cef60 0x7f0c040c35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:29.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.938+0000 7f0c0a900640 1 --2- 172.21.15.154:0/3769618243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c040b2fe0 0x7f0c0415c580 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:29.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.938+0000 7f0c0a900640 1 --2- 172.21.15.154:0/3769618243 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c040ada30 0x7f0c04156270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:29.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.938+0000 7f0c0a900640 1 -- 172.21.15.154:0/3769618243 >> 172.21.15.154:0/3769618243 conn(0x7f0c0406a0f0 msgr2=0x7f0c040ca3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:29.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.938+0000 7f0c0a900640 1 -- 172.21.15.154:0/3769618243 shutdown_connections 2023-09-23T03:19:29.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:29.938+0000 7f0c0a900640 1 -- 172.21.15.154:0/3769618243 wait complete. 2023-09-23T03:19:29.958 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":185,"stamp":"2023-09-23T03:19:29.249766+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235436,"kb_used_data":6764,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754739284,"statfs":{"total":773094113280,"available":772853026816,"internally_reserved":0,"allocated":6926336,"data_stored":3834431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":234016314},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001767"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.496299+0000","last_change":"2023-09-23T03:19:23.496299+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.496299+0000","last_undegraded":"2023-09-23T03:19:23.496299+0000","last_fullsized":"2023-09-23T03:19:23.496299+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.491358+0000","last_change":"2023-09-23T03:19:23.491358+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.491358+0000","last_undegraded":"2023-09-23T03:19:23.491358+0000","last_fullsized":"2023-09-23T03:19:23.491358+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.499260+0000","last_change":"2023-09-23T03:19:23.499260+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.499260+0000","last_undegraded":"2023-09-23T03:19:23.499260+0000","last_fullsized":"2023-09-23T03:19:23.499260+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.504046+0000","last_change":"2023-09-23T03:19:23.504046+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.504046+0000","last_undegraded":"2023-09-23T03:19:23.504046+0000","last_fullsized":"2023-09-23T03:19:23.504046+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.494815+0000","last_change":"2023-09-23T03:19:23.494815+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.494815+0000","last_undegraded":"2023-09-23T03:19:23.494815+0000","last_fullsized":"2023-09-23T03:19:23.494815+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.489767+0000","last_change":"2023-09-23T03:19:23.489767+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.489767+0000","last_undegraded":"2023-09-23T03:19:23.489767+0000","last_fullsized":"2023-09-23T03:19:23.489767+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.488689+0000","last_change":"2023-09-23T03:19:23.488689+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.488689+0000","last_undegraded":"2023-09-23T03:19:23.488689+0000","last_fullsized":"2023-09-23T03:19:23.488689+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.490760+0000","last_change":"2023-09-23T03:19:23.490760+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.490760+0000","last_undegraded":"2023-09-23T03:19:23.490760+0000","last_fullsized":"2023-09-23T03:19:23.490760+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.489816+0000","last_change":"2023-09-23T03:19:23.489816+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.489816+0000","last_undegraded":"2023-09-23T03:19:23.489816+0000","last_fullsized":"2023-09-23T03:19:23.489816+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.500805+0000","last_change":"2023-09-23T03:19:23.500805+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.500805+0000","last_undegraded":"2023-09-23T03:19:23.500805+0000","last_fullsized":"2023-09-23T03:19:23.500805+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.501343+0000","last_change":"2023-09-23T03:19:23.501343+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.501343+0000","last_undegraded":"2023-09-23T03:19:23.501343+0000","last_fullsized":"2023-09-23T03:19:23.501343+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.503530+0000","last_change":"2023-09-23T03:19:23.503530+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.503530+0000","last_undegraded":"2023-09-23T03:19:23.503530+0000","last_fullsized":"2023-09-23T03:19:23.503530+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.495939+0000","last_change":"2023-09-23T03:19:23.495939+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.495939+0000","last_undegraded":"2023-09-23T03:19:23.495939+0000","last_fullsized":"2023-09-23T03:19:23.495939+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.488295+0000","last_change":"2023-09-23T03:19:23.488295+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.488295+0000","last_undegraded":"2023-09-23T03:19:23.488295+0000","last_fullsized":"2023-09-23T03:19:23.488295+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.487204+0000","last_change":"2023-09-23T03:19:23.487204+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.487204+0000","last_undegraded":"2023-09-23T03:19:23.487204+0000","last_fullsized":"2023-09-23T03:19:23.487204+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.489319+0000","last_change":"2023-09-23T03:19:23.489319+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.489319+0000","last_undegraded":"2023-09-23T03:19:23.489319+0000","last_fullsized":"2023-09-23T03:19:23.489319+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.490227+0000","last_change":"2023-09-23T03:19:23.490227+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.490227+0000","last_undegraded":"2023-09-23T03:19:23.490227+0000","last_fullsized":"2023-09-23T03:19:23.490227+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.500342+0000","last_change":"2023-09-23T03:19:23.500342+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.500342+0000","last_undegraded":"2023-09-23T03:19:23.500342+0000","last_fullsized":"2023-09-23T03:19:23.500342+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.494436+0000","last_change":"2023-09-23T03:19:23.494436+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.494436+0000","last_undegraded":"2023-09-23T03:19:23.494436+0000","last_fullsized":"2023-09-23T03:19:23.494436+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.489696+0000","last_change":"2023-09-23T03:19:23.489696+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.489696+0000","last_undegraded":"2023-09-23T03:19:23.489696+0000","last_fullsized":"2023-09-23T03:19:23.489696+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.500116+0000","last_change":"2023-09-23T03:19:23.500116+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.500116+0000","last_undegraded":"2023-09-23T03:19:23.500116+0000","last_fullsized":"2023-09-23T03:19:23.500116+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.489656+0000","last_change":"2023-09-23T03:19:23.489656+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.489656+0000","last_undegraded":"2023-09-23T03:19:23.489656+0000","last_fullsized":"2023-09-23T03:19:23.489656+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.497801+0000","last_change":"2023-09-23T03:19:23.497801+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.497801+0000","last_undegraded":"2023-09-23T03:19:23.497801+0000","last_fullsized":"2023-09-23T03:19:23.497801+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.491559+0000","last_change":"2023-09-23T03:19:23.491559+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.491559+0000","last_undegraded":"2023-09-23T03:19:23.491559+0000","last_fullsized":"2023-09-23T03:19:23.491559+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.495800+0000","last_change":"2023-09-23T03:19:23.495800+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.495800+0000","last_undegraded":"2023-09-23T03:19:23.495800+0000","last_fullsized":"2023-09-23T03:19:23.495800+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.502761+0000","last_change":"2023-09-23T03:19:23.502761+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.502761+0000","last_undegraded":"2023-09-23T03:19:23.502761+0000","last_fullsized":"2023-09-23T03:19:23.502761+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.492766+0000","last_change":"2023-09-23T03:19:23.492766+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.492766+0000","last_undegraded":"2023-09-23T03:19:23.492766+0000","last_fullsized":"2023-09-23T03:19:23.492766+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.494163+0000","last_change":"2023-09-23T03:19:23.494163+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.494163+0000","last_undegraded":"2023-09-23T03:19:23.494163+0000","last_fullsized":"2023-09-23T03:19:23.494163+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.489233+0000","last_change":"2023-09-23T03:19:23.489233+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.489233+0000","last_undegraded":"2023-09-23T03:19:23.489233+0000","last_fullsized":"2023-09-23T03:19:23.489233+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.498098+0000","last_change":"2023-09-23T03:19:23.498098+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.498098+0000","last_undegraded":"2023-09-23T03:19:23.498098+0000","last_fullsized":"2023-09-23T03:19:23.498098+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.494358+0000","last_change":"2023-09-23T03:19:23.494358+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.494358+0000","last_undegraded":"2023-09-23T03:19:23.494358+0000","last_fullsized":"2023-09-23T03:19:23.494358+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.488708+0000","last_change":"2023-09-23T03:19:23.488708+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.488708+0000","last_undegraded":"2023-09-23T03:19:23.488708+0000","last_fullsized":"2023-09-23T03:19:23.488708+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.493052+0000","last_change":"2023-09-23T03:19:23.493052+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.493052+0000","last_undegraded":"2023-09-23T03:19:23.493052+0000","last_fullsized":"2023-09-23T03:19:23.493052+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.518586+0000","last_change":"2023-09-23T03:19:23.514227+0000","last_active":"2023-09-23T03:19:23.518586+0000","last_peered":"2023-09-23T03:19:23.518586+0000","last_clean":"2023-09-23T03:19:23.518586+0000","last_became_active":"2023-09-23T03:19:23.513867+0000","last_became_peered":"2023-09-23T03:19:23.513867+0000","last_unstale":"2023-09-23T03:19:23.518586+0000","last_undegraded":"2023-09-23T03:19:23.518586+0000","last_fullsized":"2023-09-23T03:19:23.518586+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:23:19.447543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.500949+0000","last_change":"2023-09-23T03:19:23.500949+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.500949+0000","last_undegraded":"2023-09-23T03:19:23.500949+0000","last_fullsized":"2023-09-23T03:19:23.500949+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.513306+0000","last_change":"2023-09-23T03:19:23.511815+0000","last_active":"2023-09-23T03:19:23.513306+0000","last_peered":"2023-09-23T03:19:23.513306+0000","last_clean":"2023-09-23T03:19:23.513306+0000","last_became_active":"2023-09-23T03:19:23.510810+0000","last_became_peered":"2023-09-23T03:19:23.510810+0000","last_unstale":"2023-09-23T03:19:23.513306+0000","last_undegraded":"2023-09-23T03:19:23.513306+0000","last_fullsized":"2023-09-23T03:19:23.513306+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:20:39.987490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.491398+0000","last_change":"2023-09-23T03:19:23.491398+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.491398+0000","last_undegraded":"2023-09-23T03:19:23.491398+0000","last_fullsized":"2023-09-23T03:19:23.491398+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.513486+0000","last_change":"2023-09-23T03:19:23.508844+0000","last_active":"2023-09-23T03:19:23.513486+0000","last_peered":"2023-09-23T03:19:23.513486+0000","last_clean":"2023-09-23T03:19:23.513486+0000","last_became_active":"2023-09-23T03:19:23.508557+0000","last_became_peered":"2023-09-23T03:19:23.508557+0000","last_unstale":"2023-09-23T03:19:23.513486+0000","last_undegraded":"2023-09-23T03:19:23.513486+0000","last_fullsized":"2023-09-23T03:19:23.513486+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:49:32.205787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.492304+0000","last_change":"2023-09-23T03:19:23.492304+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.492304+0000","last_undegraded":"2023-09-23T03:19:23.492304+0000","last_fullsized":"2023-09-23T03:19:23.492304+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.524534+0000","last_change":"2023-09-23T03:19:23.514514+0000","last_active":"2023-09-23T03:19:23.524534+0000","last_peered":"2023-09-23T03:19:23.524534+0000","last_clean":"2023-09-23T03:19:23.524534+0000","last_became_active":"2023-09-23T03:19:23.514219+0000","last_became_peered":"2023-09-23T03:19:23.514219+0000","last_unstale":"2023-09-23T03:19:23.524534+0000","last_undegraded":"2023-09-23T03:19:23.524534+0000","last_fullsized":"2023-09-23T03:19:23.524534+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:45:54.723198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.488324+0000","last_change":"2023-09-23T03:19:23.488324+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.488324+0000","last_undegraded":"2023-09-23T03:19:23.488324+0000","last_fullsized":"2023-09-23T03:19:23.488324+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.515178+0000","last_change":"2023-09-23T03:19:23.509524+0000","last_active":"2023-09-23T03:19:23.515178+0000","last_peered":"2023-09-23T03:19:23.515178+0000","last_clean":"2023-09-23T03:19:23.515178+0000","last_became_active":"2023-09-23T03:19:23.509229+0000","last_became_peered":"2023-09-23T03:19:23.509229+0000","last_unstale":"2023-09-23T03:19:23.515178+0000","last_undegraded":"2023-09-23T03:19:23.515178+0000","last_fullsized":"2023-09-23T03:19:23.515178+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:58:25.968168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.488600+0000","last_change":"2023-09-23T03:19:23.488600+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.488600+0000","last_undegraded":"2023-09-23T03:19:23.488600+0000","last_fullsized":"2023-09-23T03:19:23.488600+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.508057+0000","last_change":"2023-09-23T03:19:23.505508+0000","last_active":"2023-09-23T03:19:23.508057+0000","last_peered":"2023-09-23T03:19:23.508057+0000","last_clean":"2023-09-23T03:19:23.508057+0000","last_became_active":"2023-09-23T03:19:23.505249+0000","last_became_peered":"2023-09-23T03:19:23.505249+0000","last_unstale":"2023-09-23T03:19:23.508057+0000","last_undegraded":"2023-09-23T03:19:23.508057+0000","last_fullsized":"2023-09-23T03:19:23.508057+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:45:54.723198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.490303+0000","last_change":"2023-09-23T03:19:23.490303+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.490303+0000","last_undegraded":"2023-09-23T03:19:23.490303+0000","last_fullsized":"2023-09-23T03:19:23.490303+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.524401+0000","last_change":"2023-09-23T03:19:23.514127+0000","last_active":"2023-09-23T03:19:23.524401+0000","last_peered":"2023-09-23T03:19:23.524401+0000","last_clean":"2023-09-23T03:19:23.524401+0000","last_became_active":"2023-09-23T03:19:23.513630+0000","last_became_peered":"2023-09-23T03:19:23.513630+0000","last_unstale":"2023-09-23T03:19:23.524401+0000","last_undegraded":"2023-09-23T03:19:23.524401+0000","last_fullsized":"2023-09-23T03:19:23.524401+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:33:16.276149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.487896+0000","last_change":"2023-09-23T03:19:23.487896+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.487896+0000","last_undegraded":"2023-09-23T03:19:23.487896+0000","last_fullsized":"2023-09-23T03:19:23.487896+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.502692+0000","last_change":"2023-09-23T03:19:23.500111+0000","last_active":"2023-09-23T03:19:23.502692+0000","last_peered":"2023-09-23T03:19:23.502692+0000","last_clean":"2023-09-23T03:19:23.502692+0000","last_became_active":"2023-09-23T03:19:23.499792+0000","last_became_peered":"2023-09-23T03:19:23.499792+0000","last_unstale":"2023-09-23T03:19:23.502692+0000","last_undegraded":"2023-09-23T03:19:23.502692+0000","last_fullsized":"2023-09-23T03:19:23.502692+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:20:30.730712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.487758+0000","last_change":"2023-09-23T03:19:23.487758+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.487758+0000","last_undegraded":"2023-09-23T03:19:23.487758+0000","last_fullsized":"2023-09-23T03:19:23.487758+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.516664+0000","last_change":"2023-09-23T03:19:23.514447+0000","last_active":"2023-09-23T03:19:23.516664+0000","last_peered":"2023-09-23T03:19:23.516664+0000","last_clean":"2023-09-23T03:19:23.516664+0000","last_became_active":"2023-09-23T03:19:23.514067+0000","last_became_peered":"2023-09-23T03:19:23.514067+0000","last_unstale":"2023-09-23T03:19:23.516664+0000","last_undegraded":"2023-09-23T03:19:23.516664+0000","last_fullsized":"2023-09-23T03:19:23.516664+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:00:17.646758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.496971+0000","last_change":"2023-09-23T03:19:23.496971+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.496971+0000","last_undegraded":"2023-09-23T03:19:23.496971+0000","last_fullsized":"2023-09-23T03:19:23.496971+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.507007+0000","last_change":"2023-09-23T03:19:23.502457+0000","last_active":"2023-09-23T03:19:23.507007+0000","last_peered":"2023-09-23T03:19:23.507007+0000","last_clean":"2023-09-23T03:19:23.507007+0000","last_became_active":"2023-09-23T03:19:23.502185+0000","last_became_peered":"2023-09-23T03:19:23.502185+0000","last_unstale":"2023-09-23T03:19:23.507007+0000","last_undegraded":"2023-09-23T03:19:23.507007+0000","last_fullsized":"2023-09-23T03:19:23.507007+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:00:39.238009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.500995+0000","last_change":"2023-09-23T03:19:23.500995+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.500995+0000","last_undegraded":"2023-09-23T03:19:23.500995+0000","last_fullsized":"2023-09-23T03:19:23.500995+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.516693+0000","last_change":"2023-09-23T03:19:23.514525+0000","last_active":"2023-09-23T03:19:23.516693+0000","last_peered":"2023-09-23T03:19:23.516693+0000","last_clean":"2023-09-23T03:19:23.516693+0000","last_became_active":"2023-09-23T03:19:23.514117+0000","last_became_peered":"2023-09-23T03:19:23.514117+0000","last_unstale":"2023-09-23T03:19:23.516693+0000","last_undegraded":"2023-09-23T03:19:23.516693+0000","last_fullsized":"2023-09-23T03:19:23.516693+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:45:54.723198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.498493+0000","last_change":"2023-09-23T03:19:23.498493+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.498493+0000","last_undegraded":"2023-09-23T03:19:23.498493+0000","last_fullsized":"2023-09-23T03:19:23.498493+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.505498+0000","last_change":"2023-09-23T03:19:23.501871+0000","last_active":"2023-09-23T03:19:23.505498+0000","last_peered":"2023-09-23T03:19:23.505498+0000","last_clean":"2023-09-23T03:19:23.505498+0000","last_became_active":"2023-09-23T03:19:23.501537+0000","last_became_peered":"2023-09-23T03:19:23.501537+0000","last_unstale":"2023-09-23T03:19:23.505498+0000","last_undegraded":"2023-09-23T03:19:23.505498+0000","last_fullsized":"2023-09-23T03:19:23.505498+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:25:06.270537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.501561+0000","last_change":"2023-09-23T03:19:23.501561+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.501561+0000","last_undegraded":"2023-09-23T03:19:23.501561+0000","last_fullsized":"2023-09-23T03:19:23.501561+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.511665+0000","last_change":"2023-09-23T03:19:23.501583+0000","last_active":"2023-09-23T03:19:23.511665+0000","last_peered":"2023-09-23T03:19:23.511665+0000","last_clean":"2023-09-23T03:19:23.511665+0000","last_became_active":"2023-09-23T03:19:23.501037+0000","last_became_peered":"2023-09-23T03:19:23.501037+0000","last_unstale":"2023-09-23T03:19:23.511665+0000","last_undegraded":"2023-09-23T03:19:23.511665+0000","last_fullsized":"2023-09-23T03:19:23.511665+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:11:18.322404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.499265+0000","last_change":"2023-09-23T03:19:23.499265+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.499265+0000","last_undegraded":"2023-09-23T03:19:23.499265+0000","last_fullsized":"2023-09-23T03:19:23.499265+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.512892+0000","last_change":"2023-09-23T03:19:23.507592+0000","last_active":"2023-09-23T03:19:23.512892+0000","last_peered":"2023-09-23T03:19:23.512892+0000","last_clean":"2023-09-23T03:19:23.512892+0000","last_became_active":"2023-09-23T03:19:23.507316+0000","last_became_peered":"2023-09-23T03:19:23.507316+0000","last_unstale":"2023-09-23T03:19:23.512892+0000","last_undegraded":"2023-09-23T03:19:23.512892+0000","last_fullsized":"2023-09-23T03:19:23.512892+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:43:34.054466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.487374+0000","last_change":"2023-09-23T03:19:23.487374+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.487374+0000","last_undegraded":"2023-09-23T03:19:23.487374+0000","last_fullsized":"2023-09-23T03:19:23.487374+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.524603+0000","last_change":"2023-09-23T03:19:23.515237+0000","last_active":"2023-09-23T03:19:23.524603+0000","last_peered":"2023-09-23T03:19:23.524603+0000","last_clean":"2023-09-23T03:19:23.524603+0000","last_became_active":"2023-09-23T03:19:23.514946+0000","last_became_peered":"2023-09-23T03:19:23.514946+0000","last_unstale":"2023-09-23T03:19:23.524603+0000","last_undegraded":"2023-09-23T03:19:23.524603+0000","last_fullsized":"2023-09-23T03:19:23.524603+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:49:32.205787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.497140+0000","last_change":"2023-09-23T03:19:23.497140+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.497140+0000","last_undegraded":"2023-09-23T03:19:23.497140+0000","last_fullsized":"2023-09-23T03:19:23.497140+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.524349+0000","last_change":"2023-09-23T03:19:23.519665+0000","last_active":"2023-09-23T03:19:23.524349+0000","last_peered":"2023-09-23T03:19:23.524349+0000","last_clean":"2023-09-23T03:19:23.524349+0000","last_became_active":"2023-09-23T03:19:23.519393+0000","last_became_peered":"2023-09-23T03:19:23.519393+0000","last_unstale":"2023-09-23T03:19:23.524349+0000","last_undegraded":"2023-09-23T03:19:23.524349+0000","last_fullsized":"2023-09-23T03:19:23.524349+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:49:32.205787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.499552+0000","last_change":"2023-09-23T03:19:23.499552+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.499552+0000","last_undegraded":"2023-09-23T03:19:23.499552+0000","last_fullsized":"2023-09-23T03:19:23.499552+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.507040+0000","last_change":"2023-09-23T03:19:23.502992+0000","last_active":"2023-09-23T03:19:23.507040+0000","last_peered":"2023-09-23T03:19:23.507040+0000","last_clean":"2023-09-23T03:19:23.507040+0000","last_became_active":"2023-09-23T03:19:23.502688+0000","last_became_peered":"2023-09-23T03:19:23.502688+0000","last_unstale":"2023-09-23T03:19:23.507040+0000","last_undegraded":"2023-09-23T03:19:23.507040+0000","last_fullsized":"2023-09-23T03:19:23.507040+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:05:11.471065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.503106+0000","last_change":"2023-09-23T03:19:23.503106+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.503106+0000","last_undegraded":"2023-09-23T03:19:23.503106+0000","last_fullsized":"2023-09-23T03:19:23.503106+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.511859+0000","last_change":"2023-09-23T03:19:23.507612+0000","last_active":"2023-09-23T03:19:23.511859+0000","last_peered":"2023-09-23T03:19:23.511859+0000","last_clean":"2023-09-23T03:19:23.511859+0000","last_became_active":"2023-09-23T03:19:23.507211+0000","last_became_peered":"2023-09-23T03:19:23.507211+0000","last_unstale":"2023-09-23T03:19:23.511859+0000","last_undegraded":"2023-09-23T03:19:23.511859+0000","last_fullsized":"2023-09-23T03:19:23.511859+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:33:16.276149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.487706+0000","last_change":"2023-09-23T03:19:23.487706+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.487706+0000","last_undegraded":"2023-09-23T03:19:23.487706+0000","last_fullsized":"2023-09-23T03:19:23.487706+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.525158+0000","last_change":"2023-09-23T03:19:23.518829+0000","last_active":"2023-09-23T03:19:23.525158+0000","last_peered":"2023-09-23T03:19:23.525158+0000","last_clean":"2023-09-23T03:19:23.525158+0000","last_became_active":"2023-09-23T03:19:23.518512+0000","last_became_peered":"2023-09-23T03:19:23.518512+0000","last_unstale":"2023-09-23T03:19:23.525158+0000","last_undegraded":"2023-09-23T03:19:23.525158+0000","last_fullsized":"2023-09-23T03:19:23.525158+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:23:19.447543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.497713+0000","last_change":"2023-09-23T03:19:23.497713+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.497713+0000","last_undegraded":"2023-09-23T03:19:23.497713+0000","last_fullsized":"2023-09-23T03:19:23.497713+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.508037+0000","last_change":"2023-09-23T03:19:23.500147+0000","last_active":"2023-09-23T03:19:23.508037+0000","last_peered":"2023-09-23T03:19:23.508037+0000","last_clean":"2023-09-23T03:19:23.508037+0000","last_became_active":"2023-09-23T03:19:23.499826+0000","last_became_peered":"2023-09-23T03:19:23.499826+0000","last_unstale":"2023-09-23T03:19:23.508037+0000","last_undegraded":"2023-09-23T03:19:23.508037+0000","last_fullsized":"2023-09-23T03:19:23.508037+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:09:18.859490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.500650+0000","last_change":"2023-09-23T03:19:23.500650+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.500650+0000","last_undegraded":"2023-09-23T03:19:23.500650+0000","last_fullsized":"2023-09-23T03:19:23.500650+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.507070+0000","last_change":"2023-09-23T03:19:23.502333+0000","last_active":"2023-09-23T03:19:23.507070+0000","last_peered":"2023-09-23T03:19:23.507070+0000","last_clean":"2023-09-23T03:19:23.507070+0000","last_became_active":"2023-09-23T03:19:23.501997+0000","last_became_peered":"2023-09-23T03:19:23.501997+0000","last_unstale":"2023-09-23T03:19:23.507070+0000","last_undegraded":"2023-09-23T03:19:23.507070+0000","last_fullsized":"2023-09-23T03:19:23.507070+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:22:22.759701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.490693+0000","last_change":"2023-09-23T03:19:23.490693+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.490693+0000","last_undegraded":"2023-09-23T03:19:23.490693+0000","last_fullsized":"2023-09-23T03:19:23.490693+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.512081+0000","last_change":"2023-09-23T03:19:23.507705+0000","last_active":"2023-09-23T03:19:23.512081+0000","last_peered":"2023-09-23T03:19:23.512081+0000","last_clean":"2023-09-23T03:19:23.512081+0000","last_became_active":"2023-09-23T03:19:23.507409+0000","last_became_peered":"2023-09-23T03:19:23.507409+0000","last_unstale":"2023-09-23T03:19:23.512081+0000","last_undegraded":"2023-09-23T03:19:23.512081+0000","last_fullsized":"2023-09-23T03:19:23.512081+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:43:34.054466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.488782+0000","last_change":"2023-09-23T03:19:23.488782+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.488782+0000","last_undegraded":"2023-09-23T03:19:23.488782+0000","last_fullsized":"2023-09-23T03:19:23.488782+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.524549+0000","last_change":"2023-09-23T03:19:23.514960+0000","last_active":"2023-09-23T03:19:23.524549+0000","last_peered":"2023-09-23T03:19:23.524549+0000","last_clean":"2023-09-23T03:19:23.524549+0000","last_became_active":"2023-09-23T03:19:23.514644+0000","last_became_peered":"2023-09-23T03:19:23.514644+0000","last_unstale":"2023-09-23T03:19:23.524549+0000","last_undegraded":"2023-09-23T03:19:23.524549+0000","last_fullsized":"2023-09-23T03:19:23.524549+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:00:17.646758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.502007+0000","last_change":"2023-09-23T03:19:23.502007+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.502007+0000","last_undegraded":"2023-09-23T03:19:23.502007+0000","last_fullsized":"2023-09-23T03:19:23.502007+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.515734+0000","last_change":"2023-09-23T03:19:23.512352+0000","last_active":"2023-09-23T03:19:23.515734+0000","last_peered":"2023-09-23T03:19:23.515734+0000","last_clean":"2023-09-23T03:19:23.515734+0000","last_became_active":"2023-09-23T03:19:23.512039+0000","last_became_peered":"2023-09-23T03:19:23.512039+0000","last_unstale":"2023-09-23T03:19:23.515734+0000","last_undegraded":"2023-09-23T03:19:23.515734+0000","last_fullsized":"2023-09-23T03:19:23.515734+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:43:42.533621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.502444+0000","last_change":"2023-09-23T03:19:23.502444+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.502444+0000","last_undegraded":"2023-09-23T03:19:23.502444+0000","last_fullsized":"2023-09-23T03:19:23.502444+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.511839+0000","last_change":"2023-09-23T03:19:23.506342+0000","last_active":"2023-09-23T03:19:23.511839+0000","last_peered":"2023-09-23T03:19:23.511839+0000","last_clean":"2023-09-23T03:19:23.511839+0000","last_became_active":"2023-09-23T03:19:23.506078+0000","last_became_peered":"2023-09-23T03:19:23.506078+0000","last_unstale":"2023-09-23T03:19:23.511839+0000","last_undegraded":"2023-09-23T03:19:23.511839+0000","last_fullsized":"2023-09-23T03:19:23.511839+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:00:17.646758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.500536+0000","last_change":"2023-09-23T03:19:23.500536+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.500536+0000","last_undegraded":"2023-09-23T03:19:23.500536+0000","last_fullsized":"2023-09-23T03:19:23.500536+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.516019+0000","last_change":"2023-09-23T03:19:23.510497+0000","last_active":"2023-09-23T03:19:23.516019+0000","last_peered":"2023-09-23T03:19:23.516019+0000","last_clean":"2023-09-23T03:19:23.516019+0000","last_became_active":"2023-09-23T03:19:23.510155+0000","last_became_peered":"2023-09-23T03:19:23.510155+0000","last_unstale":"2023-09-23T03:19:23.516019+0000","last_undegraded":"2023-09-23T03:19:23.516019+0000","last_fullsized":"2023-09-23T03:19:23.516019+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:15:52.148791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.491878+0000","last_change":"2023-09-23T03:19:23.491878+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.491878+0000","last_undegraded":"2023-09-23T03:19:23.491878+0000","last_fullsized":"2023-09-23T03:19:23.491878+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.524455+0000","last_change":"2023-09-23T03:19:23.513693+0000","last_active":"2023-09-23T03:19:23.524455+0000","last_peered":"2023-09-23T03:19:23.524455+0000","last_clean":"2023-09-23T03:19:23.524455+0000","last_became_active":"2023-09-23T03:19:23.513359+0000","last_became_peered":"2023-09-23T03:19:23.513359+0000","last_unstale":"2023-09-23T03:19:23.524455+0000","last_undegraded":"2023-09-23T03:19:23.524455+0000","last_fullsized":"2023-09-23T03:19:23.524455+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:09:18.859490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.487319+0000","last_change":"2023-09-23T03:19:23.487319+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.487319+0000","last_undegraded":"2023-09-23T03:19:23.487319+0000","last_fullsized":"2023-09-23T03:19:23.487319+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.519999+0000","last_change":"2023-09-23T03:19:23.519087+0000","last_active":"2023-09-23T03:19:23.519999+0000","last_peered":"2023-09-23T03:19:23.519999+0000","last_clean":"2023-09-23T03:19:23.519999+0000","last_became_active":"2023-09-23T03:19:23.518827+0000","last_became_peered":"2023-09-23T03:19:23.518827+0000","last_unstale":"2023-09-23T03:19:23.519999+0000","last_undegraded":"2023-09-23T03:19:23.519999+0000","last_fullsized":"2023-09-23T03:19:23.519999+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:15:52.148791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.498413+0000","last_change":"2023-09-23T03:19:23.498413+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.498413+0000","last_undegraded":"2023-09-23T03:19:23.498413+0000","last_fullsized":"2023-09-23T03:19:23.498413+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.502654+0000","last_change":"2023-09-23T03:19:23.498946+0000","last_active":"2023-09-23T03:19:23.502654+0000","last_peered":"2023-09-23T03:19:23.502654+0000","last_clean":"2023-09-23T03:19:23.502654+0000","last_became_active":"2023-09-23T03:19:23.498522+0000","last_became_peered":"2023-09-23T03:19:23.498522+0000","last_unstale":"2023-09-23T03:19:23.502654+0000","last_undegraded":"2023-09-23T03:19:23.502654+0000","last_fullsized":"2023-09-23T03:19:23.502654+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:39:49.215616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.499307+0000","last_change":"2023-09-23T03:19:23.499307+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.499307+0000","last_undegraded":"2023-09-23T03:19:23.499307+0000","last_fullsized":"2023-09-23T03:19:23.499307+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.512203+0000","last_change":"2023-09-23T03:19:23.509484+0000","last_active":"2023-09-23T03:19:23.512203+0000","last_peered":"2023-09-23T03:19:23.512203+0000","last_clean":"2023-09-23T03:19:23.512203+0000","last_became_active":"2023-09-23T03:19:23.509203+0000","last_became_peered":"2023-09-23T03:19:23.509203+0000","last_unstale":"2023-09-23T03:19:23.512203+0000","last_undegraded":"2023-09-23T03:19:23.512203+0000","last_fullsized":"2023-09-23T03:19:23.512203+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:10:34.290176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.487164+0000","last_change":"2023-09-23T03:19:23.487164+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.487164+0000","last_undegraded":"2023-09-23T03:19:23.487164+0000","last_fullsized":"2023-09-23T03:19:23.487164+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.518826+0000","last_change":"2023-09-23T03:19:23.513953+0000","last_active":"2023-09-23T03:19:23.518826+0000","last_peered":"2023-09-23T03:19:23.518826+0000","last_clean":"2023-09-23T03:19:23.518826+0000","last_became_active":"2023-09-23T03:19:23.513616+0000","last_became_peered":"2023-09-23T03:19:23.513616+0000","last_unstale":"2023-09-23T03:19:23.518826+0000","last_undegraded":"2023-09-23T03:19:23.518826+0000","last_fullsized":"2023-09-23T03:19:23.518826+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:23:05.866775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.487239+0000","last_change":"2023-09-23T03:19:23.487239+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.487239+0000","last_undegraded":"2023-09-23T03:19:23.487239+0000","last_fullsized":"2023-09-23T03:19:23.487239+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.524435+0000","last_change":"2023-09-23T03:19:23.519480+0000","last_active":"2023-09-23T03:19:23.524435+0000","last_peered":"2023-09-23T03:19:23.524435+0000","last_clean":"2023-09-23T03:19:23.524435+0000","last_became_active":"2023-09-23T03:19:23.519191+0000","last_became_peered":"2023-09-23T03:19:23.519191+0000","last_unstale":"2023-09-23T03:19:23.524435+0000","last_undegraded":"2023-09-23T03:19:23.524435+0000","last_fullsized":"2023-09-23T03:19:23.524435+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:43:42.533621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":198,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-09-23T03:19:22.484979+0000","last_change":"2023-09-23T03:15:11.042447+0000","last_active":"2023-09-23T03:19:22.484979+0000","last_peered":"2023-09-23T03:19:22.484979+0000","last_clean":"2023-09-23T03:19:22.484979+0000","last_became_active":"2023-09-23T03:15:11.040322+0000","last_became_peered":"2023-09-23T03:15:11.040322+0000","last_unstale":"2023-09-23T03:19:22.484979+0000","last_undegraded":"2023-09-23T03:19:22.484979+0000","last_fullsized":"2023-09-23T03:19:22.484979+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_clean_scrub_stamp":"2023-09-23T03:15:09.474842+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:18:36.331645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574901,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1314816,"data_stored":923854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.98799999999999999,"15min":0.99199999999999999},"min":{"1min":0.68799999999999994,"5min":0.442,"15min":0.442},"max":{"1min":1.55,"5min":2.149,"15min":2.149},"last":1.075},{"interface":"front","average":{"1min":1.111,"5min":1.0569999999999999,"15min":1.0469999999999999},"min":{"1min":0.82099999999999995,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.681,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.95299999999999996}]},{"osd":1,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.198,"15min":1.22},"min":{"1min":0.74299999999999999,"5min":0.504,"15min":0.504},"max":{"1min":1.5649999999999999,"5min":3.1859999999999999,"15min":3.1859999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.248,"5min":1.294,"15min":1.3759999999999999},"min":{"1min":0.93899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.5840000000000001,"5min":2.613,"15min":2.613},"last":1.335}]},{"osd":2,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.1240000000000001,"15min":1.155},"min":{"1min":0.70199999999999996,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.6399999999999999,"5min":2.331,"15min":2.331},"last":0.879},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.242,"15min":1.296},"min":{"1min":0.75600000000000001,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.673,"5min":3.093,"15min":3.093},"last":1.3740000000000001}]},{"osd":3,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.248,"15min":1.2989999999999999},"min":{"1min":0.90000000000000002,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.7310000000000001,"5min":2.532,"15min":2.532},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.389,"5min":1.377,"15min":1.4410000000000001},"min":{"1min":0.95099999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.784,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":1.2829999999999999}]},{"osd":4,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.415,"15min":1.528},"min":{"1min":0.79200000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.7070000000000001,"5min":2.7210000000000001,"15min":2.7210000000000001},"last":1.5489999999999999},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.4730000000000001,"15min":1.556},"min":{"1min":0.81699999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.5880000000000001,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":1.5089999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.4179999999999999,"15min":1.4810000000000001},"min":{"1min":0.72699999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.736,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.288,"5min":1.4510000000000001,"15min":1.548},"min":{"1min":1.0069999999999999,"5min":0.497,"15min":0.497},"max":{"1min":1.7689999999999999,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":1.1819999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.5669999999999999,"15min":1.649},"min":{"1min":1.2410000000000001,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.8580000000000001,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":1.423},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.5089999999999999,"15min":1.607},"min":{"1min":1.002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.778,"5min":3.6739999999999999,"15min":3.6739999999999999},"last":1.012}]}]},{"osd":6,"up_from":13,"seq":55834574901,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":720896,"data_stored":333486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.048,"15min":1.0289999999999999},"min":{"1min":0.68899999999999995,"5min":0.38,"15min":0.38},"max":{"1min":1.6379999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.04},{"interface":"front","average":{"1min":1.083,"5min":1.0469999999999999,"15min":1.0549999999999999},"min":{"1min":0.61599999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.516,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.97899999999999998}]},{"osd":1,"last update":"Sat Sep 23 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.109,"15min":1.0980000000000001},"min":{"1min":0.65900000000000003,"5min":0.438,"15min":0.438},"max":{"1min":1.6830000000000001,"5min":1.867,"15min":1.867},"last":0.88},{"interface":"front","average":{"1min":1.23,"5min":1.2170000000000001,"15min":1.242},"min":{"1min":0.82399999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.6339999999999999,"5min":2.242,"15min":2.242},"last":1.3149999999999999}]},{"osd":2,"last update":"Sat Sep 23 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.1180000000000001,"15min":1.1060000000000001},"min":{"1min":0.82499999999999996,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4470000000000001,"5min":2.097,"15min":2.097},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.194,"5min":1.234,"15min":1.2669999999999999},"min":{"1min":0.66800000000000004,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.8149999999999999,"5min":1.903,"15min":1.903},"last":1.25}]},{"osd":3,"last update":"Sat Sep 23 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.3380000000000001,"15min":1.361},"min":{"1min":0.80200000000000005,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.8879999999999999,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":1.587},{"interface":"front","average":{"1min":1.278,"5min":1.389,"15min":1.4550000000000001},"min":{"1min":0.76600000000000001,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.657,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":1.2170000000000001}]},{"osd":4,"last update":"Sat Sep 23 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.397,"15min":1.395},"min":{"1min":0.81399999999999995,"5min":0.498,"15min":0.498},"max":{"1min":1.827,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.2849999999999999,"15min":1.294},"min":{"1min":0.75700000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.661,"5min":2.2109999999999999,"15min":2.2109999999999999},"last":0.84399999999999997}]},{"osd":5,"last update":"Sat Sep 23 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.391,"15min":1.395},"min":{"1min":0.84699999999999998,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.7390000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.4810000000000001,"15min":1.486},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.9390000000000001,"5min":4.758,"15min":4.758},"last":1.6779999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.508,"15min":1.5549999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.77,"5min":2.476,"15min":2.476},"last":1.151},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.532,"15min":1.5640000000000001},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.1739999999999999,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.472}]}]},{"osd":5,"up_from":13,"seq":55834574903,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":720896,"data_stored":333486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.044,"15min":1.03},"min":{"1min":0.80200000000000005,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.474,"5min":1.637,"15min":1.637},"last":1.075},{"interface":"front","average":{"1min":1.141,"5min":1.0609999999999999,"15min":1.032},"min":{"1min":0.78800000000000003,"5min":0.63,"15min":0.63},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":1.621}]},{"osd":1,"last update":"Sat Sep 23 03:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.212,"15min":1.224},"min":{"1min":0.95599999999999996,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.6459999999999999,"5min":1.849,"15min":1.849},"last":1.135},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.2869999999999999,"15min":1.3220000000000001},"min":{"1min":1.085,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.7410000000000001,"5min":1.893,"15min":1.893},"last":1.3759999999999999}]},{"osd":2,"last update":"Sat Sep 23 03:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.22,"15min":1.2170000000000001},"min":{"1min":1.0189999999999999,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.597,"5min":1.681,"15min":1.681},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.161,"5min":1.244,"15min":1.2969999999999999},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.5740000000000001,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":1.2090000000000001}]},{"osd":3,"last update":"Sat Sep 23 03:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.2509999999999999,"15min":1.361},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.423,"5min":1.905,"15min":1.905},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.3859999999999999,"15min":1.47},"min":{"1min":1.056,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.792,"5min":3.6110000000000002,"15min":3.6110000000000002},"last":1.4419999999999999}]},{"osd":4,"last update":"Sat Sep 23 03:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3939999999999999,"15min":1.444},"min":{"1min":0.83099999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.8500000000000001,"5min":2.069,"15min":2.069},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.401,"15min":1.4530000000000001},"min":{"1min":0.91100000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.7889999999999999,"5min":2.319,"15min":2.319},"last":1.3180000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":1.5089999999999999,"15min":1.5409999999999999},"min":{"1min":0.69899999999999995,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.948,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":1.6739999999999999},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.5309999999999999,"15min":1.613},"min":{"1min":0.97099999999999997,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.877,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.5760000000000001}]},{"osd":7,"last update":"Sat Sep 23 03:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.4570000000000001,"15min":1.538},"min":{"1min":0.91600000000000004,"5min":0.626,"15min":0.626},"max":{"1min":1.6839999999999999,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.734},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.585,"15min":1.643},"min":{"1min":1.046,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.7330000000000001,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.417}]}]},{"osd":4,"up_from":13,"seq":55834574902,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":708608,"data_stored":326899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:19:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.1279999999999999,"15min":1.1379999999999999},"min":{"1min":0.64800000000000002,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.639,"5min":1.806,"15min":1.806},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.1739999999999999,"15min":1.141},"min":{"1min":0.76500000000000001,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.905,"5min":1.9179999999999999,"15min":1.9179999999999999},"last":1.6479999999999999}]},{"osd":1,"last update":"Sat Sep 23 03:19:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.2130000000000001,"15min":1.2150000000000001},"min":{"1min":0.90200000000000002,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1910000000000001,"15min":1.2210000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.4710000000000001,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":0.97799999999999998}]},{"osd":2,"last update":"Sat Sep 23 03:19:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.1870000000000001,"15min":1.246},"min":{"1min":0.71199999999999997,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.129,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.246,"15min":1.224},"min":{"1min":0.91200000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.909,"5min":1.909,"15min":1.909},"last":0.90000000000000002}]},{"osd":3,"last update":"Sat Sep 23 03:19:15 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.1479999999999999,"15min":1.171},"min":{"1min":0.71999999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.5369999999999999,"5min":1.583,"15min":1.583},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.296,"5min":1.319,"15min":1.341},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.1779999999999999,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.302}]},{"osd":5,"last update":"Sat Sep 23 03:19:15 2023","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.371,"15min":1.345},"min":{"1min":0.99299999999999999,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.8050000000000002,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.371},{"interface":"front","average":{"1min":1.371,"5min":1.3300000000000001,"15min":1.3260000000000001},"min":{"1min":0.84299999999999997,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":2.887,"5min":2.887,"15min":2.887},"last":1.4690000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:19:15 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.397,"15min":1.391},"min":{"1min":1.048,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.4249999999999998,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.6619999999999999,"5min":1.5049999999999999,"15min":1.478},"min":{"1min":1.323,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":2.052,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.75}]},{"osd":7,"last update":"Sat Sep 23 03:19:15 2023","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.4650000000000001,"15min":1.4730000000000001},"min":{"1min":1.1819999999999999,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":2.5680000000000001,"5min":2.5680000000000001,"15min":2.5680000000000001},"last":1.716},{"interface":"front","average":{"1min":1.6379999999999999,"5min":1.5580000000000001,"15min":1.5649999999999999},"min":{"1min":1.056,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":2.4769999999999999,"5min":2.4769999999999999,"15min":2.4769999999999999},"last":1.593}]}]},{"osd":3,"up_from":13,"seq":55834574902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":708608,"data_stored":326899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.85399999999999998,"15min":0.875},"min":{"1min":0.42199999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.7090000000000001,"5min":1.744,"15min":1.744},"last":1.478},{"interface":"front","average":{"1min":0.876,"5min":0.86599999999999999,"15min":0.90200000000000002},"min":{"1min":0.51600000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.341,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.94899999999999995}]},{"osd":1,"last update":"Sat Sep 23 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.90700000000000003,"15min":0.96399999999999997},"min":{"1min":0.442,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.5640000000000001,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":1.401},{"interface":"front","average":{"1min":1.02,"5min":0.98099999999999998,"15min":0.98499999999999999},"min":{"1min":0.54700000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.3839999999999999,"5min":1.704,"15min":1.704},"last":1.127}]},{"osd":2,"last update":"Sat Sep 23 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.0609999999999999,"15min":1.113},"min":{"1min":0.54800000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.639,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.639},{"interface":"front","average":{"1min":1.165,"5min":1.097,"15min":1.129},"min":{"1min":0.64400000000000002,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.6830000000000001,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.6830000000000001}]},{"osd":4,"last update":"Sat Sep 23 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.2809999999999999,"15min":1.333},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.0099999999999998,"5min":2.698,"15min":2.698},"last":2.0099999999999998},{"interface":"front","average":{"1min":1.353,"5min":1.381,"15min":1.4470000000000001},"min":{"1min":0.95399999999999996,"5min":0.752,"15min":0.752},"max":{"1min":1.825,"5min":2.419,"15min":2.419},"last":1.825}]},{"osd":5,"last update":"Sat Sep 23 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.339,"15min":1.361},"min":{"1min":0.90300000000000002,"5min":0.753,"15min":0.753},"max":{"1min":1.8799999999999999,"5min":1.917,"15min":1.917},"last":1.867},{"interface":"front","average":{"1min":1.321,"5min":1.3049999999999999,"15min":1.3620000000000001},"min":{"1min":0.83499999999999996,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.716,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.716}]},{"osd":6,"last update":"Sat Sep 23 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.359,"15min":1.423},"min":{"1min":0.82699999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.9550000000000001,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":1.9550000000000001},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4430000000000001,"15min":1.482},"min":{"1min":1.05,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.7729999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.417,"15min":1.5009999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":2.2120000000000002,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.909},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.4399999999999999,"15min":1.526},"min":{"1min":0.85799999999999998,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.2029999999999998,"5min":2.5379999999999998,"15min":2.5379999999999998},"last":2.2029999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":720896,"data_stored":333486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.83799999999999997,"15min":0.83699999999999997},"min":{"1min":0.442,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":0.501},{"interface":"front","average":{"1min":0.998,"5min":0.86199999999999999,"15min":0.84299999999999997},"min":{"1min":0.48899999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.5329999999999999,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.77200000000000002}]},{"osd":1,"last update":"Sat Sep 23 03:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.84799999999999998,"15min":0.85999999999999999},"min":{"1min":0.55800000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.026,"5min":0.93300000000000005,"15min":0.94099999999999995},"min":{"1min":0.55100000000000005,"5min":0.38,"15min":0.38},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.95999999999999996}]},{"osd":3,"last update":"Sat Sep 23 03:19:12 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":0.96599999999999997,"15min":0.95199999999999996},"min":{"1min":0.53900000000000003,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.4279999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.073,"5min":0.98499999999999999,"15min":0.997},"min":{"1min":0.60699999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.534,"5min":1.78,"15min":1.78},"last":0.89900000000000002}]},{"osd":4,"last update":"Sat Sep 23 03:19:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2729999999999999,"15min":1.3240000000000001},"min":{"1min":0.86199999999999999,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.819,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.071},{"interface":"front","average":{"1min":1.296,"5min":1.2969999999999999,"15min":1.341},"min":{"1min":0.77600000000000002,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.631,"5min":2.234,"15min":2.234},"last":1.0029999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:19:12 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.339,"15min":1.409},"min":{"1min":0.97499999999999998,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.7849999999999999,"5min":3.9409999999999998,"15min":3.9409999999999998},"last":1.032},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.4199999999999999,"15min":1.5329999999999999},"min":{"1min":0.81299999999999994,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.9359999999999999,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":1.1759999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:19:12 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.3580000000000001,"15min":1.417},"min":{"1min":0.83299999999999996,"5min":0.755,"15min":0.755},"max":{"1min":1.8540000000000001,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.458,"5min":1.393,"15min":1.478},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.1680000000000001,"5min":4.5800000000000001,"15min":4.5800000000000001},"last":1.1279999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:19:12 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.3600000000000001,"15min":1.4019999999999999},"min":{"1min":1.0469999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.996,"5min":2.3149999999999999,"15min":2.3149999999999999},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.413,"15min":1.4550000000000001},"min":{"1min":0.873,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.9259999999999999,"5min":2.621,"15min":2.621},"last":1.109}]}]},{"osd":1,"up_from":13,"seq":55834574904,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":720896,"data_stored":333486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.80000000000000004,"15min":0.80800000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.2490000000000001,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.85099999999999998},"min":{"1min":0.34799999999999998,"5min":0.312,"15min":0.312},"max":{"1min":1.2210000000000001,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":0.623}]},{"osd":2,"last update":"Sat Sep 23 03:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.84999999999999998,"15min":0.89400000000000002},"min":{"1min":0.52000000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.409,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":1.278},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.004,"15min":1.0369999999999999},"min":{"1min":0.54500000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5600000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.524}]},{"osd":3,"last update":"Sat Sep 23 03:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.0269999999999999,"15min":1.087},"min":{"1min":0.48799999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.5389999999999999,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":1.24},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.0980000000000001,"15min":1.1619999999999999},"min":{"1min":0.56200000000000006,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.7649999999999999,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":1.6850000000000001}]},{"osd":4,"last update":"Sat Sep 23 03:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.2330000000000001,"15min":1.2829999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.7270000000000001,"5min":3.2629999999999999,"15min":3.2629999999999999},"last":1.411},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.262,"15min":1.3},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6359999999999999,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":1.456}]},{"osd":5,"last update":"Sat Sep 23 03:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.254,"15min":1.2989999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.9319999999999999,"5min":3.161,"15min":3.161},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3,"15min":1.3460000000000001},"min":{"1min":0.80400000000000005,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.698,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":1.5840000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.2969999999999999,"15min":1.3400000000000001},"min":{"1min":0.76500000000000001,"5min":0.747,"15min":0.747},"max":{"1min":2.0329999999999999,"5min":3.0179999999999998,"15min":3.0179999999999998},"last":1.637},{"interface":"front","average":{"1min":1.333,"5min":1.3520000000000001,"15min":1.3979999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.7909999999999999,"5min":3.3580000000000001,"15min":3.3580000000000001},"last":1.78}]},{"osd":7,"last update":"Sat Sep 23 03:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.363,"15min":1.413},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":2.085,"5min":3.1099999999999999,"15min":3.1099999999999999},"last":1.6619999999999999},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.415,"15min":1.4810000000000001},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.7330000000000001,"5min":3.306,"15min":3.306},"last":1.722}]}]},{"osd":0,"up_from":13,"seq":55834574904,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1310720,"data_stored":922835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.81200000000000006,"15min":0.81999999999999995},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.3180000000000001,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.86599999999999999,"15min":0.89300000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.363,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.96399999999999997}]},{"osd":2,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.81100000000000005,"15min":0.80300000000000005},"min":{"1min":0.48599999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.22,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.97299999999999998,"15min":0.98499999999999999},"min":{"1min":0.46700000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":1.3029999999999999}]},{"osd":3,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93300000000000005,"15min":0.91500000000000004},"min":{"1min":0.44800000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.226,"5min":1.591,"15min":1.591},"last":1.198},{"interface":"front","average":{"1min":1.117,"5min":0.996,"15min":1.006},"min":{"1min":0.57299999999999995,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.4990000000000001,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.147}]},{"osd":4,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.204,"15min":1.204},"min":{"1min":0.81499999999999995,"5min":0.755,"15min":0.755},"max":{"1min":1.601,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.7450000000000001},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.2410000000000001,"15min":1.274},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.4610000000000001,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.456}]},{"osd":5,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.2250000000000001,"15min":1.2290000000000001},"min":{"1min":0.872,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.6739999999999999,"5min":1.867,"15min":1.867},"last":1.474},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.294,"15min":1.272},"min":{"1min":0.76400000000000001,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":1.8080000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.3009999999999999,"15min":1.3029999999999999},"min":{"1min":0.99399999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":1.552},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.325,"15min":1.3029999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.875,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.873}]},{"osd":7,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.329,"15min":1.3160000000000001},"min":{"1min":0.93300000000000005,"5min":0.747,"15min":0.747},"max":{"1min":1.6499999999999999,"5min":1.944,"15min":1.944},"last":1.528},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.3979999999999999,"15min":1.4059999999999999},"min":{"1min":0.91800000000000004,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.756,"5min":2.048,"15min":2.048},"last":1.7170000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-23T03:19:29.964 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3f is not active+clean 2023-09-23T03:19:29.965 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.496299+0000', 'last_change': '2023-09-23T03:19:23.496299+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.496299+0000', 'last_undegraded': '2023-09-23T03:19:23.496299+0000', 'last_fullsized': '2023-09-23T03:19:23.496299+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-09-23T03:19:29.965 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3e is not active+clean 2023-09-23T03:19:29.965 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.491358+0000', 'last_change': '2023-09-23T03:19:23.491358+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.491358+0000', 'last_undegraded': '2023-09-23T03:19:23.491358+0000', 'last_fullsized': '2023-09-23T03:19:23.491358+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-09-23T03:19:29.965 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3d is not active+clean 2023-09-23T03:19:29.965 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.499260+0000', 'last_change': '2023-09-23T03:19:23.499260+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.499260+0000', 'last_undegraded': '2023-09-23T03:19:23.499260+0000', 'last_fullsized': '2023-09-23T03:19:23.499260+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-09-23T03:19:29.965 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3c is not active+clean 2023-09-23T03:19:29.965 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.504046+0000', 'last_change': '2023-09-23T03:19:23.504046+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.504046+0000', 'last_undegraded': '2023-09-23T03:19:23.504046+0000', 'last_fullsized': '2023-09-23T03:19:23.504046+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-09-23T03:19:29.965 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3b is not active+clean 2023-09-23T03:19:29.965 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.494815+0000', 'last_change': '2023-09-23T03:19:23.494815+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.494815+0000', 'last_undegraded': '2023-09-23T03:19:23.494815+0000', 'last_fullsized': '2023-09-23T03:19:23.494815+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-09-23T03:19:29.966 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3a is not active+clean 2023-09-23T03:19:29.966 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.489767+0000', 'last_change': '2023-09-23T03:19:23.489767+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.489767+0000', 'last_undegraded': '2023-09-23T03:19:23.489767+0000', 'last_fullsized': '2023-09-23T03:19:23.489767+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-09-23T03:19:29.966 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.39 is not active+clean 2023-09-23T03:19:29.966 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.488689+0000', 'last_change': '2023-09-23T03:19:23.488689+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.488689+0000', 'last_undegraded': '2023-09-23T03:19:23.488689+0000', 'last_fullsized': '2023-09-23T03:19:23.488689+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-09-23T03:19:29.966 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.38 is not active+clean 2023-09-23T03:19:29.966 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.490760+0000', 'last_change': '2023-09-23T03:19:23.490760+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.490760+0000', 'last_undegraded': '2023-09-23T03:19:23.490760+0000', 'last_fullsized': '2023-09-23T03:19:23.490760+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-09-23T03:19:29.966 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.37 is not active+clean 2023-09-23T03:19:29.966 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.489816+0000', 'last_change': '2023-09-23T03:19:23.489816+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.489816+0000', 'last_undegraded': '2023-09-23T03:19:23.489816+0000', 'last_fullsized': '2023-09-23T03:19:23.489816+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-09-23T03:19:29.967 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.36 is not active+clean 2023-09-23T03:19:29.967 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.500805+0000', 'last_change': '2023-09-23T03:19:23.500805+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.500805+0000', 'last_undegraded': '2023-09-23T03:19:23.500805+0000', 'last_fullsized': '2023-09-23T03:19:23.500805+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-09-23T03:19:29.967 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.35 is not active+clean 2023-09-23T03:19:29.967 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.501343+0000', 'last_change': '2023-09-23T03:19:23.501343+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.501343+0000', 'last_undegraded': '2023-09-23T03:19:23.501343+0000', 'last_fullsized': '2023-09-23T03:19:23.501343+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-09-23T03:19:29.967 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.34 is not active+clean 2023-09-23T03:19:29.967 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.503530+0000', 'last_change': '2023-09-23T03:19:23.503530+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.503530+0000', 'last_undegraded': '2023-09-23T03:19:23.503530+0000', 'last_fullsized': '2023-09-23T03:19:23.503530+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-09-23T03:19:29.967 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.33 is not active+clean 2023-09-23T03:19:29.967 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.495939+0000', 'last_change': '2023-09-23T03:19:23.495939+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.495939+0000', 'last_undegraded': '2023-09-23T03:19:23.495939+0000', 'last_fullsized': '2023-09-23T03:19:23.495939+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-09-23T03:19:29.967 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.32 is not active+clean 2023-09-23T03:19:29.967 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.488295+0000', 'last_change': '2023-09-23T03:19:23.488295+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.488295+0000', 'last_undegraded': '2023-09-23T03:19:23.488295+0000', 'last_fullsized': '2023-09-23T03:19:23.488295+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-09-23T03:19:29.968 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.31 is not active+clean 2023-09-23T03:19:29.968 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.487204+0000', 'last_change': '2023-09-23T03:19:23.487204+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.487204+0000', 'last_undegraded': '2023-09-23T03:19:23.487204+0000', 'last_fullsized': '2023-09-23T03:19:23.487204+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-09-23T03:19:29.968 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.30 is not active+clean 2023-09-23T03:19:29.968 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.489319+0000', 'last_change': '2023-09-23T03:19:23.489319+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.489319+0000', 'last_undegraded': '2023-09-23T03:19:23.489319+0000', 'last_fullsized': '2023-09-23T03:19:23.489319+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-09-23T03:19:29.968 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2f is not active+clean 2023-09-23T03:19:29.968 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.490227+0000', 'last_change': '2023-09-23T03:19:23.490227+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.490227+0000', 'last_undegraded': '2023-09-23T03:19:23.490227+0000', 'last_fullsized': '2023-09-23T03:19:23.490227+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-09-23T03:19:29.968 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2e is not active+clean 2023-09-23T03:19:29.968 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.500342+0000', 'last_change': '2023-09-23T03:19:23.500342+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.500342+0000', 'last_undegraded': '2023-09-23T03:19:23.500342+0000', 'last_fullsized': '2023-09-23T03:19:23.500342+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-09-23T03:19:29.968 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2d is not active+clean 2023-09-23T03:19:29.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.494436+0000', 'last_change': '2023-09-23T03:19:23.494436+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.494436+0000', 'last_undegraded': '2023-09-23T03:19:23.494436+0000', 'last_fullsized': '2023-09-23T03:19:23.494436+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-09-23T03:19:29.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2c is not active+clean 2023-09-23T03:19:29.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.489696+0000', 'last_change': '2023-09-23T03:19:23.489696+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.489696+0000', 'last_undegraded': '2023-09-23T03:19:23.489696+0000', 'last_fullsized': '2023-09-23T03:19:23.489696+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-09-23T03:19:29.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2b is not active+clean 2023-09-23T03:19:29.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.500116+0000', 'last_change': '2023-09-23T03:19:23.500116+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.500116+0000', 'last_undegraded': '2023-09-23T03:19:23.500116+0000', 'last_fullsized': '2023-09-23T03:19:23.500116+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-09-23T03:19:29.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2a is not active+clean 2023-09-23T03:19:29.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.489656+0000', 'last_change': '2023-09-23T03:19:23.489656+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.489656+0000', 'last_undegraded': '2023-09-23T03:19:23.489656+0000', 'last_fullsized': '2023-09-23T03:19:23.489656+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-09-23T03:19:29.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.29 is not active+clean 2023-09-23T03:19:29.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.497801+0000', 'last_change': '2023-09-23T03:19:23.497801+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.497801+0000', 'last_undegraded': '2023-09-23T03:19:23.497801+0000', 'last_fullsized': '2023-09-23T03:19:23.497801+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-09-23T03:19:29.970 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.28 is not active+clean 2023-09-23T03:19:29.970 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.491559+0000', 'last_change': '2023-09-23T03:19:23.491559+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.491559+0000', 'last_undegraded': '2023-09-23T03:19:23.491559+0000', 'last_fullsized': '2023-09-23T03:19:23.491559+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-09-23T03:19:29.970 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.27 is not active+clean 2023-09-23T03:19:29.970 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.495800+0000', 'last_change': '2023-09-23T03:19:23.495800+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.495800+0000', 'last_undegraded': '2023-09-23T03:19:23.495800+0000', 'last_fullsized': '2023-09-23T03:19:23.495800+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-09-23T03:19:29.970 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.26 is not active+clean 2023-09-23T03:19:29.970 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.502761+0000', 'last_change': '2023-09-23T03:19:23.502761+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.502761+0000', 'last_undegraded': '2023-09-23T03:19:23.502761+0000', 'last_fullsized': '2023-09-23T03:19:23.502761+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-09-23T03:19:29.970 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.25 is not active+clean 2023-09-23T03:19:29.970 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.492766+0000', 'last_change': '2023-09-23T03:19:23.492766+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.492766+0000', 'last_undegraded': '2023-09-23T03:19:23.492766+0000', 'last_fullsized': '2023-09-23T03:19:23.492766+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-09-23T03:19:29.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.24 is not active+clean 2023-09-23T03:19:29.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.494163+0000', 'last_change': '2023-09-23T03:19:23.494163+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.494163+0000', 'last_undegraded': '2023-09-23T03:19:23.494163+0000', 'last_fullsized': '2023-09-23T03:19:23.494163+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-09-23T03:19:29.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.23 is not active+clean 2023-09-23T03:19:29.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.489233+0000', 'last_change': '2023-09-23T03:19:23.489233+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.489233+0000', 'last_undegraded': '2023-09-23T03:19:23.489233+0000', 'last_fullsized': '2023-09-23T03:19:23.489233+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-09-23T03:19:29.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.22 is not active+clean 2023-09-23T03:19:29.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.498098+0000', 'last_change': '2023-09-23T03:19:23.498098+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.498098+0000', 'last_undegraded': '2023-09-23T03:19:23.498098+0000', 'last_fullsized': '2023-09-23T03:19:23.498098+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-09-23T03:19:29.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.21 is not active+clean 2023-09-23T03:19:29.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.494358+0000', 'last_change': '2023-09-23T03:19:23.494358+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.494358+0000', 'last_undegraded': '2023-09-23T03:19:23.494358+0000', 'last_fullsized': '2023-09-23T03:19:23.494358+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-09-23T03:19:29.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.20 is not active+clean 2023-09-23T03:19:29.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.488708+0000', 'last_change': '2023-09-23T03:19:23.488708+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.488708+0000', 'last_undegraded': '2023-09-23T03:19:23.488708+0000', 'last_fullsized': '2023-09-23T03:19:23.488708+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-09-23T03:19:29.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1e is not active+clean 2023-09-23T03:19:29.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.493052+0000', 'last_change': '2023-09-23T03:19:23.493052+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.493052+0000', 'last_undegraded': '2023-09-23T03:19:23.493052+0000', 'last_fullsized': '2023-09-23T03:19:23.493052+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-09-23T03:19:29.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1f is not active+clean 2023-09-23T03:19:29.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.500949+0000', 'last_change': '2023-09-23T03:19:23.500949+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.500949+0000', 'last_undegraded': '2023-09-23T03:19:23.500949+0000', 'last_fullsized': '2023-09-23T03:19:23.500949+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-09-23T03:19:29.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1c is not active+clean 2023-09-23T03:19:29.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.491398+0000', 'last_change': '2023-09-23T03:19:23.491398+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.491398+0000', 'last_undegraded': '2023-09-23T03:19:23.491398+0000', 'last_fullsized': '2023-09-23T03:19:23.491398+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-09-23T03:19:29.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1d is not active+clean 2023-09-23T03:19:29.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.492304+0000', 'last_change': '2023-09-23T03:19:23.492304+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.492304+0000', 'last_undegraded': '2023-09-23T03:19:23.492304+0000', 'last_fullsized': '2023-09-23T03:19:23.492304+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-09-23T03:19:29.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1a is not active+clean 2023-09-23T03:19:29.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.488324+0000', 'last_change': '2023-09-23T03:19:23.488324+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.488324+0000', 'last_undegraded': '2023-09-23T03:19:23.488324+0000', 'last_fullsized': '2023-09-23T03:19:23.488324+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-09-23T03:19:29.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1b is not active+clean 2023-09-23T03:19:29.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.488600+0000', 'last_change': '2023-09-23T03:19:23.488600+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.488600+0000', 'last_undegraded': '2023-09-23T03:19:23.488600+0000', 'last_fullsized': '2023-09-23T03:19:23.488600+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-09-23T03:19:29.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.18 is not active+clean 2023-09-23T03:19:29.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.490303+0000', 'last_change': '2023-09-23T03:19:23.490303+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.490303+0000', 'last_undegraded': '2023-09-23T03:19:23.490303+0000', 'last_fullsized': '2023-09-23T03:19:23.490303+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-09-23T03:19:29.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.19 is not active+clean 2023-09-23T03:19:29.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.487896+0000', 'last_change': '2023-09-23T03:19:23.487896+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.487896+0000', 'last_undegraded': '2023-09-23T03:19:23.487896+0000', 'last_fullsized': '2023-09-23T03:19:23.487896+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-09-23T03:19:29.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.16 is not active+clean 2023-09-23T03:19:29.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.487758+0000', 'last_change': '2023-09-23T03:19:23.487758+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.487758+0000', 'last_undegraded': '2023-09-23T03:19:23.487758+0000', 'last_fullsized': '2023-09-23T03:19:23.487758+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-09-23T03:19:29.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.17 is not active+clean 2023-09-23T03:19:29.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.496971+0000', 'last_change': '2023-09-23T03:19:23.496971+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.496971+0000', 'last_undegraded': '2023-09-23T03:19:23.496971+0000', 'last_fullsized': '2023-09-23T03:19:23.496971+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-09-23T03:19:29.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.14 is not active+clean 2023-09-23T03:19:29.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.500995+0000', 'last_change': '2023-09-23T03:19:23.500995+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.500995+0000', 'last_undegraded': '2023-09-23T03:19:23.500995+0000', 'last_fullsized': '2023-09-23T03:19:23.500995+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-09-23T03:19:29.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.15 is not active+clean 2023-09-23T03:19:29.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.498493+0000', 'last_change': '2023-09-23T03:19:23.498493+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.498493+0000', 'last_undegraded': '2023-09-23T03:19:23.498493+0000', 'last_fullsized': '2023-09-23T03:19:23.498493+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-09-23T03:19:29.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.12 is not active+clean 2023-09-23T03:19:29.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.501561+0000', 'last_change': '2023-09-23T03:19:23.501561+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.501561+0000', 'last_undegraded': '2023-09-23T03:19:23.501561+0000', 'last_fullsized': '2023-09-23T03:19:23.501561+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-09-23T03:19:29.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.13 is not active+clean 2023-09-23T03:19:29.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.499265+0000', 'last_change': '2023-09-23T03:19:23.499265+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.499265+0000', 'last_undegraded': '2023-09-23T03:19:23.499265+0000', 'last_fullsized': '2023-09-23T03:19:23.499265+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-09-23T03:19:29.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.10 is not active+clean 2023-09-23T03:19:29.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.487374+0000', 'last_change': '2023-09-23T03:19:23.487374+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.487374+0000', 'last_undegraded': '2023-09-23T03:19:23.487374+0000', 'last_fullsized': '2023-09-23T03:19:23.487374+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-09-23T03:19:29.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.11 is not active+clean 2023-09-23T03:19:29.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.497140+0000', 'last_change': '2023-09-23T03:19:23.497140+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.497140+0000', 'last_undegraded': '2023-09-23T03:19:23.497140+0000', 'last_fullsized': '2023-09-23T03:19:23.497140+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-09-23T03:19:29.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.e is not active+clean 2023-09-23T03:19:29.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.499552+0000', 'last_change': '2023-09-23T03:19:23.499552+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.499552+0000', 'last_undegraded': '2023-09-23T03:19:23.499552+0000', 'last_fullsized': '2023-09-23T03:19:23.499552+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-09-23T03:19:29.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.f is not active+clean 2023-09-23T03:19:29.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.503106+0000', 'last_change': '2023-09-23T03:19:23.503106+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.503106+0000', 'last_undegraded': '2023-09-23T03:19:23.503106+0000', 'last_fullsized': '2023-09-23T03:19:23.503106+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-09-23T03:19:29.976 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.c is not active+clean 2023-09-23T03:19:29.976 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.487706+0000', 'last_change': '2023-09-23T03:19:23.487706+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.487706+0000', 'last_undegraded': '2023-09-23T03:19:23.487706+0000', 'last_fullsized': '2023-09-23T03:19:23.487706+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-09-23T03:19:29.976 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.d is not active+clean 2023-09-23T03:19:29.976 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.497713+0000', 'last_change': '2023-09-23T03:19:23.497713+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.497713+0000', 'last_undegraded': '2023-09-23T03:19:23.497713+0000', 'last_fullsized': '2023-09-23T03:19:23.497713+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-09-23T03:19:29.976 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.a is not active+clean 2023-09-23T03:19:29.976 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.500650+0000', 'last_change': '2023-09-23T03:19:23.500650+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.500650+0000', 'last_undegraded': '2023-09-23T03:19:23.500650+0000', 'last_fullsized': '2023-09-23T03:19:23.500650+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-09-23T03:19:29.976 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.b is not active+clean 2023-09-23T03:19:29.976 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.490693+0000', 'last_change': '2023-09-23T03:19:23.490693+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.490693+0000', 'last_undegraded': '2023-09-23T03:19:23.490693+0000', 'last_fullsized': '2023-09-23T03:19:23.490693+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-09-23T03:19:29.976 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.9 is not active+clean 2023-09-23T03:19:29.976 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.488782+0000', 'last_change': '2023-09-23T03:19:23.488782+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.488782+0000', 'last_undegraded': '2023-09-23T03:19:23.488782+0000', 'last_fullsized': '2023-09-23T03:19:23.488782+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-09-23T03:19:29.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.6 is not active+clean 2023-09-23T03:19:29.977 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.502007+0000', 'last_change': '2023-09-23T03:19:23.502007+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.502007+0000', 'last_undegraded': '2023-09-23T03:19:23.502007+0000', 'last_fullsized': '2023-09-23T03:19:23.502007+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-09-23T03:19:29.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.7 is not active+clean 2023-09-23T03:19:29.977 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.502444+0000', 'last_change': '2023-09-23T03:19:23.502444+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.502444+0000', 'last_undegraded': '2023-09-23T03:19:23.502444+0000', 'last_fullsized': '2023-09-23T03:19:23.502444+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-09-23T03:19:29.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.4 is not active+clean 2023-09-23T03:19:29.977 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.500536+0000', 'last_change': '2023-09-23T03:19:23.500536+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.500536+0000', 'last_undegraded': '2023-09-23T03:19:23.500536+0000', 'last_fullsized': '2023-09-23T03:19:23.500536+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-09-23T03:19:29.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.5 is not active+clean 2023-09-23T03:19:29.977 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.491878+0000', 'last_change': '2023-09-23T03:19:23.491878+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.491878+0000', 'last_undegraded': '2023-09-23T03:19:23.491878+0000', 'last_fullsized': '2023-09-23T03:19:23.491878+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-09-23T03:19:29.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2 is not active+clean 2023-09-23T03:19:29.977 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.487319+0000', 'last_change': '2023-09-23T03:19:23.487319+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.487319+0000', 'last_undegraded': '2023-09-23T03:19:23.487319+0000', 'last_fullsized': '2023-09-23T03:19:23.487319+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-09-23T03:19:29.978 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3 is not active+clean 2023-09-23T03:19:29.978 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.498413+0000', 'last_change': '2023-09-23T03:19:23.498413+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.498413+0000', 'last_undegraded': '2023-09-23T03:19:23.498413+0000', 'last_fullsized': '2023-09-23T03:19:23.498413+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-09-23T03:19:29.978 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.0 is not active+clean 2023-09-23T03:19:29.978 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.499307+0000', 'last_change': '2023-09-23T03:19:23.499307+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.499307+0000', 'last_undegraded': '2023-09-23T03:19:23.499307+0000', 'last_fullsized': '2023-09-23T03:19:23.499307+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-09-23T03:19:29.978 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1 is not active+clean 2023-09-23T03:19:29.978 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.487164+0000', 'last_change': '2023-09-23T03:19:23.487164+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.487164+0000', 'last_undegraded': '2023-09-23T03:19:23.487164+0000', 'last_fullsized': '2023-09-23T03:19:23.487164+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-09-23T03:19:29.978 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.8 is not active+clean 2023-09-23T03:19:29.978 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2023-09-23T03:19:23.487239+0000', 'last_change': '2023-09-23T03:19:23.487239+0000', 'last_active': '2023-09-23T03:19:23.465704+0000', 'last_peered': '2023-09-23T03:19:23.465704+0000', 'last_clean': '2023-09-23T03:19:23.465704+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-23T03:19:23.487239+0000', 'last_undegraded': '2023-09-23T03:19:23.487239+0000', 'last_fullsized': '2023-09-23T03:19:23.487239+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'last_clean_scrub_stamp': '2023-09-23T03:19:23.465704+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-09-23T03:19:29.979 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-09-23T03:19:30.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.218+0000 7febfcc61640 1 Processor -- start 2023-09-23T03:19:30.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.219+0000 7febfcc61640 1 -- start start 2023-09-23T03:19:30.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.220+0000 7febfcc61640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7febf80b1b20 0x7febf80ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:30.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.220+0000 7febfcc61640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7febf80cee70 0x7febf80ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:30.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.220+0000 7febfcc61640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febf815ac30 con 0x7febf80acce0 2023-09-23T03:19:30.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.220+0000 7febfcc61640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febf80b2e90 con 0x7febf80b1b20 2023-09-23T03:19:30.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.220+0000 7febfcc61640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febf80cf320 con 0x7febf80cee70 2023-09-23T03:19:30.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.220+0000 7febedd9b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7febf80b1b20 0x7febf80ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:30.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.220+0000 7febedd9b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7febf80b1b20 0x7febf80ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43072/0 (socket says 172.21.15.154:43072) 2023-09-23T03:19:30.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.220+0000 7febedd9b640 1 -- 172.21.15.154:0/2028103851 learned_addr learned my addr 172.21.15.154:0/2028103851 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:30.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.221+0000 7febf6d9d640 1 --2- 172.21.15.154:0/2028103851 >> v2:172.21.15.190:3300/0 conn(0x7febf80cee70 0x7febf80ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:30.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.221+0000 7febedd9b640 1 -- 172.21.15.154:0/2028103851 >> v2:172.21.15.190:3300/0 conn(0x7febf80cee70 msgr2=0x7febf80ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:30.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.221+0000 7febedd9b640 1 --2- 172.21.15.154:0/2028103851 >> v2:172.21.15.190:3300/0 conn(0x7febf80cee70 0x7febf80ab140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:30.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.221+0000 7febedd9b640 1 -- 172.21.15.154:0/2028103851 >> v1:172.21.15.190:6789/0 conn(0x7febf80acce0 legacy=0x7febf815ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:30.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.221+0000 7febedd9b640 1 -- 172.21.15.154:0/2028103851 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf80b1f70 con 0x7febf80b1b20 2023-09-23T03:19:30.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.221+0000 7febf6d9d640 1 --2- 172.21.15.154:0/2028103851 >> v2:172.21.15.190:3300/0 conn(0x7febf80cee70 0x7febf80ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:30.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.222+0000 7febedd9b640 1 --2- 172.21.15.154:0/2028103851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febf80b1b20 0x7febf80ce930 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7febf80c9430 tx=0x7febe002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37fc8811e8a147ee server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:30.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.222+0000 7febf5d9b640 1 -- 172.21.15.154:0/2028103851 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe002c9f0 con 0x7febf80b1b20 2023-09-23T03:19:30.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.222+0000 7febf5d9b640 1 -- 172.21.15.154:0/2028103851 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe002cb50 con 0x7febf80b1b20 2023-09-23T03:19:30.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.222+0000 7febf5d9b640 1 -- 172.21.15.154:0/2028103851 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe00343f0 con 0x7febf80b1b20 2023-09-23T03:19:30.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.223+0000 7febfcc61640 1 -- 172.21.15.154:0/2028103851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febf80b1b20 msgr2=0x7febf80ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:30.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.223+0000 7febfcc61640 1 --2- 172.21.15.154:0/2028103851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febf80b1b20 0x7febf80ce930 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7febf80c9430 tx=0x7febe002be50 comp rx=0 tx=0).stop 2023-09-23T03:19:30.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.223+0000 7febfcc61640 1 -- 172.21.15.154:0/2028103851 shutdown_connections 2023-09-23T03:19:30.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.223+0000 7febfcc61640 1 --2- 172.21.15.154:0/2028103851 >> v2:172.21.15.190:3300/0 conn(0x7febf80cee70 0x7febf80ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:30.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.223+0000 7febfcc61640 1 --2- 172.21.15.154:0/2028103851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febf80b1b20 0x7febf80ce930 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:30.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.223+0000 7febfcc61640 1 -- 172.21.15.154:0/2028103851 >> 172.21.15.154:0/2028103851 conn(0x7febf806a0f0 msgr2=0x7febf806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:30.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.223+0000 7febfcc61640 1 -- 172.21.15.154:0/2028103851 shutdown_connections 2023-09-23T03:19:30.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.223+0000 7febfcc61640 1 -- 172.21.15.154:0/2028103851 wait complete. 2023-09-23T03:19:30.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.225+0000 7febfcc61640 1 Processor -- start 2023-09-23T03:19:30.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.225+0000 7febfcc61640 1 -- start start 2023-09-23T03:19:30.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.226+0000 7febfcc61640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febf80acce0 0x7febf81578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:30.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.226+0000 7febfcc61640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febf80b1b20 0x7febf80ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:30.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.226+0000 7febfcc61640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7febf80cee70 0x7febf80cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:30.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.226+0000 7febfcc61640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf81f0130 con 0x7febf80acce0 2023-09-23T03:19:30.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.226+0000 7febfcc61640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf81f0410 con 0x7febf80cee70 2023-09-23T03:19:30.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.226+0000 7febfcc61640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf81f06f0 con 0x7febf80b1b20 2023-09-23T03:19:30.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.226+0000 7febf659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febf80acce0 0x7febf81578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:30.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.226+0000 7febf659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febf80acce0 0x7febf81578a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43074/0 (socket says 172.21.15.154:43074) 2023-09-23T03:19:30.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.226+0000 7febf659c640 1 -- 172.21.15.154:0/2579918459 learned_addr learned my addr 172.21.15.154:0/2579918459 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:30.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.227+0000 7febf659c640 1 -- 172.21.15.154:0/2579918459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febf80b1b20 msgr2=0x7febf80ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:30.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.227+0000 7febedd9b640 1 --2- 172.21.15.154:0/2579918459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febf80b1b20 0x7febf80ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:30.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.227+0000 7febf659c640 1 --2- 172.21.15.154:0/2579918459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febf80b1b20 0x7febf80ca7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:30.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.227+0000 7febf659c640 1 -- 172.21.15.154:0/2579918459 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7febf80cee70 msgr2=0x7febf80cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:30.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.227+0000 7febf6d9d640 1 --2- 172.21.15.154:0/2579918459 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7febf80cee70 0x7febf80cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:30.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.227+0000 7febf659c640 1 --2- 172.21.15.154:0/2579918459 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7febf80cee70 0x7febf80cacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:30.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.227+0000 7febf659c640 1 -- 172.21.15.154:0/2579918459 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe0009b30 con 0x7febf80acce0 2023-09-23T03:19:30.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.227+0000 7febf659c640 1 --2- 172.21.15.154:0/2579918459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febf80acce0 0x7febf81578a0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7febe800c400 tx=0x7febe800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:30.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.227+0000 7febef7fe640 1 -- 172.21.15.154:0/2579918459 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe80054e0 con 0x7febf80acce0 2023-09-23T03:19:30.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.227+0000 7febef7fe640 1 -- 172.21.15.154:0/2579918459 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe8003640 con 0x7febf80acce0 2023-09-23T03:19:30.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.227+0000 7febef7fe640 1 -- 172.21.15.154:0/2579918459 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe80039f0 con 0x7febf80acce0 2023-09-23T03:19:30.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.227+0000 7febfcc61640 1 -- 172.21.15.154:0/2579918459 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febf81f09d0 con 0x7febf80acce0 2023-09-23T03:19:30.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.227+0000 7febfcc61640 1 -- 172.21.15.154:0/2579918459 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf815d0a0 con 0x7febf80acce0 2023-09-23T03:19:30.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.229+0000 7febfcc61640 1 -- 172.21.15.154:0/2579918459 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febf81f0bc0 con 0x7febf80acce0 2023-09-23T03:19:30.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.230+0000 7febef7fe640 1 -- 172.21.15.154:0/2579918459 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7febe8003b50 con 0x7febf80acce0 2023-09-23T03:19:30.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.231+0000 7febef7fe640 1 --2- 172.21.15.154:0/2579918459 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7febd806af30 0x7febd806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:30.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.231+0000 7febef7fe640 1 -- 172.21.15.154:0/2579918459 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6029+0+0 (secure 0 0 0) 0x7febe8088980 con 0x7febf80acce0 2023-09-23T03:19:30.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.231+0000 7febedd9b640 1 --2- 172.21.15.154:0/2579918459 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7febd806af30 0x7febd806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:30.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.232+0000 7febedd9b640 1 --2- 172.21.15.154:0/2579918459 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7febd806af30 0x7febd806d3f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7febe0035000 tx=0x7febe0030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:30.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.235+0000 7febef7fe640 1 -- 172.21.15.154:0/2579918459 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7febe80539c0 con 0x7febf80acce0 2023-09-23T03:19:30.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.498+0000 7febfcc61640 1 -- 172.21.15.154:0/2579918459 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7febf81f0270 con 0x7febf80acce0 2023-09-23T03:19:30.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.499+0000 7febef7fe640 1 -- 172.21.15.154:0/2579918459 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1034 (secure 0 0 0) 0x7febf81f0270 con 0x7febf80acce0 2023-09-23T03:19:30.499 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:19:30.499 INFO:teuthology.orchestra.run.smithi154.stdout:{"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":274,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":62,"num_osds":8,"num_up_osds":8,"osd_up_since":1695438908,"num_in_osds":8,"osd_in_since":1695438899,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":241086464,"bytes_avail":772853026816,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":40,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":6312}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2023-09-23T03:19:09.243760+0000","services":{}},"progress_events":{}} 2023-09-23T03:19:30.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.501+0000 7febfcc61640 1 -- 172.21.15.154:0/2579918459 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7febd806af30 msgr2=0x7febd806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:30.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.501+0000 7febfcc61640 1 --2- 172.21.15.154:0/2579918459 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7febd806af30 0x7febd806d3f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7febe0035000 tx=0x7febe0030000 comp rx=0 tx=0).stop 2023-09-23T03:19:30.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.501+0000 7febfcc61640 1 -- 172.21.15.154:0/2579918459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febf80acce0 msgr2=0x7febf81578a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:30.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.501+0000 7febfcc61640 1 --2- 172.21.15.154:0/2579918459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febf80acce0 0x7febf81578a0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7febe800c400 tx=0x7febe800c8c0 comp rx=0 tx=0).stop 2023-09-23T03:19:30.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.501+0000 7febfcc61640 1 -- 172.21.15.154:0/2579918459 shutdown_connections 2023-09-23T03:19:30.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.501+0000 7febfcc61640 1 --2- 172.21.15.154:0/2579918459 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7febd806af30 0x7febd806d3f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:30.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.501+0000 7febfcc61640 1 --2- 172.21.15.154:0/2579918459 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7febf80cee70 0x7febf80cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:30.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.501+0000 7febfcc61640 1 --2- 172.21.15.154:0/2579918459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7febf80b1b20 0x7febf80ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:30.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.501+0000 7febfcc61640 1 --2- 172.21.15.154:0/2579918459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7febf80acce0 0x7febf81578a0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:30.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.502+0000 7febfcc61640 1 -- 172.21.15.154:0/2579918459 >> 172.21.15.154:0/2579918459 conn(0x7febf806a0f0 msgr2=0x7febf806a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:30.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.502+0000 7febfcc61640 1 -- 172.21.15.154:0/2579918459 shutdown_connections 2023-09-23T03:19:30.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.502+0000 7febfcc61640 1 -- 172.21.15.154:0/2579918459 wait complete. 2023-09-23T03:19:30.521 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-09-23T03:19:30.522 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-23T03:19:30.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.738+0000 7f4de18c2640 1 Processor -- start 2023-09-23T03:19:30.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.738+0000 7f4de18c2640 1 -- start start 2023-09-23T03:19:30.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.740+0000 7f4de18c2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4ddc0acce0 0x7f4ddc15ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:30.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.740+0000 7f4de18c2640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4ddc0b1b20 0x7f4ddc0ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:30.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.740+0000 7f4de18c2640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ddc15ac30 con 0x7f4ddc0cee70 2023-09-23T03:19:30.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.740+0000 7f4de18c2640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ddc0b2e90 con 0x7f4ddc0acce0 2023-09-23T03:19:30.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.740+0000 7f4de18c2640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ddc0cf300 con 0x7f4ddc0b1b20 2023-09-23T03:19:30.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.740+0000 7f4ddaffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4ddc0acce0 0x7f4ddc15ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:30.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.740+0000 7f4ddaffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4ddc0acce0 0x7f4ddc15ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43088/0 (socket says 172.21.15.154:43088) 2023-09-23T03:19:30.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.740+0000 7f4ddaffd640 1 -- 172.21.15.154:0/222550980 learned_addr learned my addr 172.21.15.154:0/222550980 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:30.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.741+0000 7f4dd3fff640 1 --2- 172.21.15.154:0/222550980 >> v2:172.21.15.190:3300/0 conn(0x7f4ddc0b1b20 0x7f4ddc0ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:30.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.741+0000 7f4ddaffd640 1 -- 172.21.15.154:0/222550980 >> v2:172.21.15.190:3300/0 conn(0x7f4ddc0b1b20 msgr2=0x7f4ddc0ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:30.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.741+0000 7f4ddaffd640 1 --2- 172.21.15.154:0/222550980 >> v2:172.21.15.190:3300/0 conn(0x7f4ddc0b1b20 0x7f4ddc0ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:30.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.741+0000 7f4ddaffd640 1 -- 172.21.15.154:0/222550980 >> v1:172.21.15.190:6789/0 conn(0x7f4ddc0cee70 legacy=0x7f4ddc06cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:30.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.741+0000 7f4ddaffd640 1 -- 172.21.15.154:0/222550980 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ddc0b1f70 con 0x7f4ddc0acce0 2023-09-23T03:19:30.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.741+0000 7f4dd3fff640 1 --2- 172.21.15.154:0/222550980 >> v2:172.21.15.190:3300/0 conn(0x7f4ddc0b1b20 0x7f4ddc0ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:30.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.742+0000 7f4ddaffd640 1 --2- 172.21.15.154:0/222550980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ddc0acce0 0x7f4ddc15ed60 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc0c9430 tx=0x7f4dcc02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f136e590d1a806 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:30.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.742+0000 7f4dda7fc640 1 -- 172.21.15.154:0/222550980 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc004230 con 0x7f4ddc0acce0 2023-09-23T03:19:30.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.742+0000 7f4dda7fc640 1 -- 172.21.15.154:0/222550980 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dcc004390 con 0x7f4ddc0acce0 2023-09-23T03:19:30.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.742+0000 7f4dda7fc640 1 -- 172.21.15.154:0/222550980 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dcc003810 con 0x7f4ddc0acce0 2023-09-23T03:19:30.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.743+0000 7f4de18c2640 1 -- 172.21.15.154:0/222550980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ddc0acce0 msgr2=0x7f4ddc15ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:30.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.743+0000 7f4de18c2640 1 --2- 172.21.15.154:0/222550980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ddc0acce0 0x7f4ddc15ed60 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc0c9430 tx=0x7f4dcc02b860 comp rx=0 tx=0).stop 2023-09-23T03:19:30.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.743+0000 7f4de18c2640 1 -- 172.21.15.154:0/222550980 shutdown_connections 2023-09-23T03:19:30.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.743+0000 7f4de18c2640 1 --2- 172.21.15.154:0/222550980 >> v2:172.21.15.190:3300/0 conn(0x7f4ddc0b1b20 0x7f4ddc0ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:30.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.743+0000 7f4de18c2640 1 --2- 172.21.15.154:0/222550980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ddc0acce0 0x7f4ddc15ed60 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:30.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.743+0000 7f4de18c2640 1 -- 172.21.15.154:0/222550980 >> 172.21.15.154:0/222550980 conn(0x7f4ddc06a0f0 msgr2=0x7f4ddc06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:30.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.743+0000 7f4de18c2640 1 -- 172.21.15.154:0/222550980 shutdown_connections 2023-09-23T03:19:30.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.743+0000 7f4de18c2640 1 -- 172.21.15.154:0/222550980 wait complete. 2023-09-23T03:19:30.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.745+0000 7f4de18c2640 1 Processor -- start 2023-09-23T03:19:30.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.745+0000 7f4de18c2640 1 -- start start 2023-09-23T03:19:30.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.746+0000 7f4de18c2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ddc0acce0 0x7f4ddc0c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:30.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.746+0000 7f4de18c2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ddc0b1b20 0x7f4ddc0cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:30.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.746+0000 7f4de18c2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ddc0cee70 0x7f4ddc158dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:30.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.746+0000 7f4de18c2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ddc15a030 con 0x7f4ddc0cee70 2023-09-23T03:19:30.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.746+0000 7f4de18c2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ddc0c8910 con 0x7f4ddc0b1b20 2023-09-23T03:19:30.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.746+0000 7f4de18c2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ddc0cc530 con 0x7f4ddc0acce0 2023-09-23T03:19:30.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.746+0000 7f4ddb7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ddc0cee70 0x7f4ddc158dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:30.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.746+0000 7f4ddb7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ddc0cee70 0x7f4ddc158dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43104/0 (socket says 172.21.15.154:43104) 2023-09-23T03:19:30.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.746+0000 7f4ddb7fe640 1 -- 172.21.15.154:0/937397560 learned_addr learned my addr 172.21.15.154:0/937397560 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:30.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.747+0000 7f4ddaffd640 1 --2- 172.21.15.154:0/937397560 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ddc0acce0 0x7f4ddc0c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:30.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.747+0000 7f4dd3fff640 1 --2- 172.21.15.154:0/937397560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ddc0b1b20 0x7f4ddc0cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:30.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.747+0000 7f4ddb7fe640 1 -- 172.21.15.154:0/937397560 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ddc0acce0 msgr2=0x7f4ddc0c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:30.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.747+0000 7f4ddb7fe640 1 --2- 172.21.15.154:0/937397560 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ddc0acce0 0x7f4ddc0c83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:30.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.747+0000 7f4ddb7fe640 1 -- 172.21.15.154:0/937397560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ddc0b1b20 msgr2=0x7f4ddc0cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:30.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.747+0000 7f4ddb7fe640 1 --2- 172.21.15.154:0/937397560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ddc0b1b20 0x7f4ddc0cdb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:30.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.747+0000 7f4ddb7fe640 1 -- 172.21.15.154:0/937397560 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dcc009460 con 0x7f4ddc0cee70 2023-09-23T03:19:30.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.747+0000 7f4ddaffd640 1 --2- 172.21.15.154:0/937397560 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ddc0acce0 0x7f4ddc0c83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:30.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.747+0000 7f4ddb7fe640 1 --2- 172.21.15.154:0/937397560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ddc0cee70 0x7f4ddc158dc0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f4dd400b990 tx=0x7f4dd400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:30.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.747+0000 7f4dd8ff9640 1 -- 172.21.15.154:0/937397560 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd400cba0 con 0x7f4ddc0cee70 2023-09-23T03:19:30.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.747+0000 7f4dd8ff9640 1 -- 172.21.15.154:0/937397560 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dd400cd00 con 0x7f4ddc0cee70 2023-09-23T03:19:30.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.747+0000 7f4dd8ff9640 1 -- 172.21.15.154:0/937397560 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd4014680 con 0x7f4ddc0cee70 2023-09-23T03:19:30.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.747+0000 7f4de18c2640 1 -- 172.21.15.154:0/937397560 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ddc0cc810 con 0x7f4ddc0cee70 2023-09-23T03:19:30.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.748+0000 7f4de18c2640 1 -- 172.21.15.154:0/937397560 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ddc0ccdd0 con 0x7f4ddc0cee70 2023-09-23T03:19:30.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.749+0000 7f4de18c2640 1 -- 172.21.15.154:0/937397560 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ddc073eb0 con 0x7f4ddc0cee70 2023-09-23T03:19:30.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.750+0000 7f4dd8ff9640 1 -- 172.21.15.154:0/937397560 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f4dd401d3b0 con 0x7f4ddc0cee70 2023-09-23T03:19:30.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.751+0000 7f4dd8ff9640 1 --2- 172.21.15.154:0/937397560 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4dbc06af30 0x7f4dbc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:30.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.751+0000 7f4dd8ff9640 1 -- 172.21.15.154:0/937397560 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4dd40898b0 con 0x7f4ddc0cee70 2023-09-23T03:19:30.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.752+0000 7f4ddaffd640 1 --2- 172.21.15.154:0/937397560 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4dbc06af30 0x7f4dbc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:30.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.752+0000 7f4ddaffd640 1 --2- 172.21.15.154:0/937397560 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4dbc06af30 0x7f4dbc06d3f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc034000 tx=0x7f4dcc008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:30.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.756+0000 7f4dd8ff9640 1 -- 172.21.15.154:0/937397560 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f4dd4054710 con 0x7f4ddc0cee70 2023-09-23T03:19:30.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.939+0000 7f4de18c2640 1 -- 172.21.15.154:0/937397560 --> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4ddc0b2210 con 0x7f4dbc06af30 2023-09-23T03:19:30.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.949+0000 7f4dd8ff9640 1 -- 172.21.15.154:0/937397560 <== mgr.4110 v2:172.21.15.154:6832/2422652104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+281110 (secure 0 0 0) 0x7f4dcc005a10 con 0x7f4dbc06af30 2023-09-23T03:19:30.950 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:19:30.954 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2023-09-23T03:19:30.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.955+0000 7f4de18c2640 1 -- 172.21.15.154:0/937397560 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4dbc06af30 msgr2=0x7f4dbc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:30.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.955+0000 7f4de18c2640 1 --2- 172.21.15.154:0/937397560 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4dbc06af30 0x7f4dbc06d3f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc034000 tx=0x7f4dcc008030 comp rx=0 tx=0).stop 2023-09-23T03:19:30.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.955+0000 7f4de18c2640 1 -- 172.21.15.154:0/937397560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ddc0cee70 msgr2=0x7f4ddc158dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:30.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.955+0000 7f4de18c2640 1 --2- 172.21.15.154:0/937397560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ddc0cee70 0x7f4ddc158dc0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f4dd400b990 tx=0x7f4dd400be50 comp rx=0 tx=0).stop 2023-09-23T03:19:30.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.955+0000 7f4de18c2640 1 -- 172.21.15.154:0/937397560 shutdown_connections 2023-09-23T03:19:30.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.955+0000 7f4de18c2640 1 --2- 172.21.15.154:0/937397560 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4dbc06af30 0x7f4dbc06d3f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:30.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.955+0000 7f4de18c2640 1 --2- 172.21.15.154:0/937397560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ddc0cee70 0x7f4ddc158dc0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:30.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.955+0000 7f4de18c2640 1 --2- 172.21.15.154:0/937397560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ddc0b1b20 0x7f4ddc0cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:30.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.955+0000 7f4de18c2640 1 --2- 172.21.15.154:0/937397560 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4ddc0acce0 0x7f4ddc0c83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:30.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.955+0000 7f4de18c2640 1 -- 172.21.15.154:0/937397560 >> 172.21.15.154:0/937397560 conn(0x7f4ddc06a0f0 msgr2=0x7f4ddc0aea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:30.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.956+0000 7f4de18c2640 1 -- 172.21.15.154:0/937397560 shutdown_connections 2023-09-23T03:19:30.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:30.956+0000 7f4de18c2640 1 -- 172.21.15.154:0/937397560 wait complete. 2023-09-23T03:19:30.975 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":185,"stamp":"2023-09-23T03:19:29.249766+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235436,"kb_used_data":6764,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754739284,"statfs":{"total":773094113280,"available":772853026816,"internally_reserved":0,"allocated":6926336,"data_stored":3834431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":234016314},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001767"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.496299+0000","last_change":"2023-09-23T03:19:23.496299+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.496299+0000","last_undegraded":"2023-09-23T03:19:23.496299+0000","last_fullsized":"2023-09-23T03:19:23.496299+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.491358+0000","last_change":"2023-09-23T03:19:23.491358+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.491358+0000","last_undegraded":"2023-09-23T03:19:23.491358+0000","last_fullsized":"2023-09-23T03:19:23.491358+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.499260+0000","last_change":"2023-09-23T03:19:23.499260+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.499260+0000","last_undegraded":"2023-09-23T03:19:23.499260+0000","last_fullsized":"2023-09-23T03:19:23.499260+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.504046+0000","last_change":"2023-09-23T03:19:23.504046+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.504046+0000","last_undegraded":"2023-09-23T03:19:23.504046+0000","last_fullsized":"2023-09-23T03:19:23.504046+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.494815+0000","last_change":"2023-09-23T03:19:23.494815+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.494815+0000","last_undegraded":"2023-09-23T03:19:23.494815+0000","last_fullsized":"2023-09-23T03:19:23.494815+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.489767+0000","last_change":"2023-09-23T03:19:23.489767+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.489767+0000","last_undegraded":"2023-09-23T03:19:23.489767+0000","last_fullsized":"2023-09-23T03:19:23.489767+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.488689+0000","last_change":"2023-09-23T03:19:23.488689+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.488689+0000","last_undegraded":"2023-09-23T03:19:23.488689+0000","last_fullsized":"2023-09-23T03:19:23.488689+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.490760+0000","last_change":"2023-09-23T03:19:23.490760+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.490760+0000","last_undegraded":"2023-09-23T03:19:23.490760+0000","last_fullsized":"2023-09-23T03:19:23.490760+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.489816+0000","last_change":"2023-09-23T03:19:23.489816+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.489816+0000","last_undegraded":"2023-09-23T03:19:23.489816+0000","last_fullsized":"2023-09-23T03:19:23.489816+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.500805+0000","last_change":"2023-09-23T03:19:23.500805+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.500805+0000","last_undegraded":"2023-09-23T03:19:23.500805+0000","last_fullsized":"2023-09-23T03:19:23.500805+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.501343+0000","last_change":"2023-09-23T03:19:23.501343+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.501343+0000","last_undegraded":"2023-09-23T03:19:23.501343+0000","last_fullsized":"2023-09-23T03:19:23.501343+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.503530+0000","last_change":"2023-09-23T03:19:23.503530+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.503530+0000","last_undegraded":"2023-09-23T03:19:23.503530+0000","last_fullsized":"2023-09-23T03:19:23.503530+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.495939+0000","last_change":"2023-09-23T03:19:23.495939+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.495939+0000","last_undegraded":"2023-09-23T03:19:23.495939+0000","last_fullsized":"2023-09-23T03:19:23.495939+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.488295+0000","last_change":"2023-09-23T03:19:23.488295+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.488295+0000","last_undegraded":"2023-09-23T03:19:23.488295+0000","last_fullsized":"2023-09-23T03:19:23.488295+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.487204+0000","last_change":"2023-09-23T03:19:23.487204+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.487204+0000","last_undegraded":"2023-09-23T03:19:23.487204+0000","last_fullsized":"2023-09-23T03:19:23.487204+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.489319+0000","last_change":"2023-09-23T03:19:23.489319+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.489319+0000","last_undegraded":"2023-09-23T03:19:23.489319+0000","last_fullsized":"2023-09-23T03:19:23.489319+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.490227+0000","last_change":"2023-09-23T03:19:23.490227+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.490227+0000","last_undegraded":"2023-09-23T03:19:23.490227+0000","last_fullsized":"2023-09-23T03:19:23.490227+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.500342+0000","last_change":"2023-09-23T03:19:23.500342+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.500342+0000","last_undegraded":"2023-09-23T03:19:23.500342+0000","last_fullsized":"2023-09-23T03:19:23.500342+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.494436+0000","last_change":"2023-09-23T03:19:23.494436+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.494436+0000","last_undegraded":"2023-09-23T03:19:23.494436+0000","last_fullsized":"2023-09-23T03:19:23.494436+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.489696+0000","last_change":"2023-09-23T03:19:23.489696+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.489696+0000","last_undegraded":"2023-09-23T03:19:23.489696+0000","last_fullsized":"2023-09-23T03:19:23.489696+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.500116+0000","last_change":"2023-09-23T03:19:23.500116+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.500116+0000","last_undegraded":"2023-09-23T03:19:23.500116+0000","last_fullsized":"2023-09-23T03:19:23.500116+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.489656+0000","last_change":"2023-09-23T03:19:23.489656+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.489656+0000","last_undegraded":"2023-09-23T03:19:23.489656+0000","last_fullsized":"2023-09-23T03:19:23.489656+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.497801+0000","last_change":"2023-09-23T03:19:23.497801+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.497801+0000","last_undegraded":"2023-09-23T03:19:23.497801+0000","last_fullsized":"2023-09-23T03:19:23.497801+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.491559+0000","last_change":"2023-09-23T03:19:23.491559+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.491559+0000","last_undegraded":"2023-09-23T03:19:23.491559+0000","last_fullsized":"2023-09-23T03:19:23.491559+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.495800+0000","last_change":"2023-09-23T03:19:23.495800+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.495800+0000","last_undegraded":"2023-09-23T03:19:23.495800+0000","last_fullsized":"2023-09-23T03:19:23.495800+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.502761+0000","last_change":"2023-09-23T03:19:23.502761+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.502761+0000","last_undegraded":"2023-09-23T03:19:23.502761+0000","last_fullsized":"2023-09-23T03:19:23.502761+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.492766+0000","last_change":"2023-09-23T03:19:23.492766+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.492766+0000","last_undegraded":"2023-09-23T03:19:23.492766+0000","last_fullsized":"2023-09-23T03:19:23.492766+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.494163+0000","last_change":"2023-09-23T03:19:23.494163+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.494163+0000","last_undegraded":"2023-09-23T03:19:23.494163+0000","last_fullsized":"2023-09-23T03:19:23.494163+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.489233+0000","last_change":"2023-09-23T03:19:23.489233+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.489233+0000","last_undegraded":"2023-09-23T03:19:23.489233+0000","last_fullsized":"2023-09-23T03:19:23.489233+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.498098+0000","last_change":"2023-09-23T03:19:23.498098+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.498098+0000","last_undegraded":"2023-09-23T03:19:23.498098+0000","last_fullsized":"2023-09-23T03:19:23.498098+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.494358+0000","last_change":"2023-09-23T03:19:23.494358+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.494358+0000","last_undegraded":"2023-09-23T03:19:23.494358+0000","last_fullsized":"2023-09-23T03:19:23.494358+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.488708+0000","last_change":"2023-09-23T03:19:23.488708+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.488708+0000","last_undegraded":"2023-09-23T03:19:23.488708+0000","last_fullsized":"2023-09-23T03:19:23.488708+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.493052+0000","last_change":"2023-09-23T03:19:23.493052+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.493052+0000","last_undegraded":"2023-09-23T03:19:23.493052+0000","last_fullsized":"2023-09-23T03:19:23.493052+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.518586+0000","last_change":"2023-09-23T03:19:23.514227+0000","last_active":"2023-09-23T03:19:23.518586+0000","last_peered":"2023-09-23T03:19:23.518586+0000","last_clean":"2023-09-23T03:19:23.518586+0000","last_became_active":"2023-09-23T03:19:23.513867+0000","last_became_peered":"2023-09-23T03:19:23.513867+0000","last_unstale":"2023-09-23T03:19:23.518586+0000","last_undegraded":"2023-09-23T03:19:23.518586+0000","last_fullsized":"2023-09-23T03:19:23.518586+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:23:19.447543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.500949+0000","last_change":"2023-09-23T03:19:23.500949+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.500949+0000","last_undegraded":"2023-09-23T03:19:23.500949+0000","last_fullsized":"2023-09-23T03:19:23.500949+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.513306+0000","last_change":"2023-09-23T03:19:23.511815+0000","last_active":"2023-09-23T03:19:23.513306+0000","last_peered":"2023-09-23T03:19:23.513306+0000","last_clean":"2023-09-23T03:19:23.513306+0000","last_became_active":"2023-09-23T03:19:23.510810+0000","last_became_peered":"2023-09-23T03:19:23.510810+0000","last_unstale":"2023-09-23T03:19:23.513306+0000","last_undegraded":"2023-09-23T03:19:23.513306+0000","last_fullsized":"2023-09-23T03:19:23.513306+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:20:39.987490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.491398+0000","last_change":"2023-09-23T03:19:23.491398+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.491398+0000","last_undegraded":"2023-09-23T03:19:23.491398+0000","last_fullsized":"2023-09-23T03:19:23.491398+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.513486+0000","last_change":"2023-09-23T03:19:23.508844+0000","last_active":"2023-09-23T03:19:23.513486+0000","last_peered":"2023-09-23T03:19:23.513486+0000","last_clean":"2023-09-23T03:19:23.513486+0000","last_became_active":"2023-09-23T03:19:23.508557+0000","last_became_peered":"2023-09-23T03:19:23.508557+0000","last_unstale":"2023-09-23T03:19:23.513486+0000","last_undegraded":"2023-09-23T03:19:23.513486+0000","last_fullsized":"2023-09-23T03:19:23.513486+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:49:32.205787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.492304+0000","last_change":"2023-09-23T03:19:23.492304+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.492304+0000","last_undegraded":"2023-09-23T03:19:23.492304+0000","last_fullsized":"2023-09-23T03:19:23.492304+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.524534+0000","last_change":"2023-09-23T03:19:23.514514+0000","last_active":"2023-09-23T03:19:23.524534+0000","last_peered":"2023-09-23T03:19:23.524534+0000","last_clean":"2023-09-23T03:19:23.524534+0000","last_became_active":"2023-09-23T03:19:23.514219+0000","last_became_peered":"2023-09-23T03:19:23.514219+0000","last_unstale":"2023-09-23T03:19:23.524534+0000","last_undegraded":"2023-09-23T03:19:23.524534+0000","last_fullsized":"2023-09-23T03:19:23.524534+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:45:54.723198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.488324+0000","last_change":"2023-09-23T03:19:23.488324+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.488324+0000","last_undegraded":"2023-09-23T03:19:23.488324+0000","last_fullsized":"2023-09-23T03:19:23.488324+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.515178+0000","last_change":"2023-09-23T03:19:23.509524+0000","last_active":"2023-09-23T03:19:23.515178+0000","last_peered":"2023-09-23T03:19:23.515178+0000","last_clean":"2023-09-23T03:19:23.515178+0000","last_became_active":"2023-09-23T03:19:23.509229+0000","last_became_peered":"2023-09-23T03:19:23.509229+0000","last_unstale":"2023-09-23T03:19:23.515178+0000","last_undegraded":"2023-09-23T03:19:23.515178+0000","last_fullsized":"2023-09-23T03:19:23.515178+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:58:25.968168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.488600+0000","last_change":"2023-09-23T03:19:23.488600+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.488600+0000","last_undegraded":"2023-09-23T03:19:23.488600+0000","last_fullsized":"2023-09-23T03:19:23.488600+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.508057+0000","last_change":"2023-09-23T03:19:23.505508+0000","last_active":"2023-09-23T03:19:23.508057+0000","last_peered":"2023-09-23T03:19:23.508057+0000","last_clean":"2023-09-23T03:19:23.508057+0000","last_became_active":"2023-09-23T03:19:23.505249+0000","last_became_peered":"2023-09-23T03:19:23.505249+0000","last_unstale":"2023-09-23T03:19:23.508057+0000","last_undegraded":"2023-09-23T03:19:23.508057+0000","last_fullsized":"2023-09-23T03:19:23.508057+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:45:54.723198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.490303+0000","last_change":"2023-09-23T03:19:23.490303+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.490303+0000","last_undegraded":"2023-09-23T03:19:23.490303+0000","last_fullsized":"2023-09-23T03:19:23.490303+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.524401+0000","last_change":"2023-09-23T03:19:23.514127+0000","last_active":"2023-09-23T03:19:23.524401+0000","last_peered":"2023-09-23T03:19:23.524401+0000","last_clean":"2023-09-23T03:19:23.524401+0000","last_became_active":"2023-09-23T03:19:23.513630+0000","last_became_peered":"2023-09-23T03:19:23.513630+0000","last_unstale":"2023-09-23T03:19:23.524401+0000","last_undegraded":"2023-09-23T03:19:23.524401+0000","last_fullsized":"2023-09-23T03:19:23.524401+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:33:16.276149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.487896+0000","last_change":"2023-09-23T03:19:23.487896+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.487896+0000","last_undegraded":"2023-09-23T03:19:23.487896+0000","last_fullsized":"2023-09-23T03:19:23.487896+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.502692+0000","last_change":"2023-09-23T03:19:23.500111+0000","last_active":"2023-09-23T03:19:23.502692+0000","last_peered":"2023-09-23T03:19:23.502692+0000","last_clean":"2023-09-23T03:19:23.502692+0000","last_became_active":"2023-09-23T03:19:23.499792+0000","last_became_peered":"2023-09-23T03:19:23.499792+0000","last_unstale":"2023-09-23T03:19:23.502692+0000","last_undegraded":"2023-09-23T03:19:23.502692+0000","last_fullsized":"2023-09-23T03:19:23.502692+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:20:30.730712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.487758+0000","last_change":"2023-09-23T03:19:23.487758+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.487758+0000","last_undegraded":"2023-09-23T03:19:23.487758+0000","last_fullsized":"2023-09-23T03:19:23.487758+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.516664+0000","last_change":"2023-09-23T03:19:23.514447+0000","last_active":"2023-09-23T03:19:23.516664+0000","last_peered":"2023-09-23T03:19:23.516664+0000","last_clean":"2023-09-23T03:19:23.516664+0000","last_became_active":"2023-09-23T03:19:23.514067+0000","last_became_peered":"2023-09-23T03:19:23.514067+0000","last_unstale":"2023-09-23T03:19:23.516664+0000","last_undegraded":"2023-09-23T03:19:23.516664+0000","last_fullsized":"2023-09-23T03:19:23.516664+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:00:17.646758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.496971+0000","last_change":"2023-09-23T03:19:23.496971+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.496971+0000","last_undegraded":"2023-09-23T03:19:23.496971+0000","last_fullsized":"2023-09-23T03:19:23.496971+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.507007+0000","last_change":"2023-09-23T03:19:23.502457+0000","last_active":"2023-09-23T03:19:23.507007+0000","last_peered":"2023-09-23T03:19:23.507007+0000","last_clean":"2023-09-23T03:19:23.507007+0000","last_became_active":"2023-09-23T03:19:23.502185+0000","last_became_peered":"2023-09-23T03:19:23.502185+0000","last_unstale":"2023-09-23T03:19:23.507007+0000","last_undegraded":"2023-09-23T03:19:23.507007+0000","last_fullsized":"2023-09-23T03:19:23.507007+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:00:39.238009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.500995+0000","last_change":"2023-09-23T03:19:23.500995+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.500995+0000","last_undegraded":"2023-09-23T03:19:23.500995+0000","last_fullsized":"2023-09-23T03:19:23.500995+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.516693+0000","last_change":"2023-09-23T03:19:23.514525+0000","last_active":"2023-09-23T03:19:23.516693+0000","last_peered":"2023-09-23T03:19:23.516693+0000","last_clean":"2023-09-23T03:19:23.516693+0000","last_became_active":"2023-09-23T03:19:23.514117+0000","last_became_peered":"2023-09-23T03:19:23.514117+0000","last_unstale":"2023-09-23T03:19:23.516693+0000","last_undegraded":"2023-09-23T03:19:23.516693+0000","last_fullsized":"2023-09-23T03:19:23.516693+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:45:54.723198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.498493+0000","last_change":"2023-09-23T03:19:23.498493+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.498493+0000","last_undegraded":"2023-09-23T03:19:23.498493+0000","last_fullsized":"2023-09-23T03:19:23.498493+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.505498+0000","last_change":"2023-09-23T03:19:23.501871+0000","last_active":"2023-09-23T03:19:23.505498+0000","last_peered":"2023-09-23T03:19:23.505498+0000","last_clean":"2023-09-23T03:19:23.505498+0000","last_became_active":"2023-09-23T03:19:23.501537+0000","last_became_peered":"2023-09-23T03:19:23.501537+0000","last_unstale":"2023-09-23T03:19:23.505498+0000","last_undegraded":"2023-09-23T03:19:23.505498+0000","last_fullsized":"2023-09-23T03:19:23.505498+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:25:06.270537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.501561+0000","last_change":"2023-09-23T03:19:23.501561+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.501561+0000","last_undegraded":"2023-09-23T03:19:23.501561+0000","last_fullsized":"2023-09-23T03:19:23.501561+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.511665+0000","last_change":"2023-09-23T03:19:23.501583+0000","last_active":"2023-09-23T03:19:23.511665+0000","last_peered":"2023-09-23T03:19:23.511665+0000","last_clean":"2023-09-23T03:19:23.511665+0000","last_became_active":"2023-09-23T03:19:23.501037+0000","last_became_peered":"2023-09-23T03:19:23.501037+0000","last_unstale":"2023-09-23T03:19:23.511665+0000","last_undegraded":"2023-09-23T03:19:23.511665+0000","last_fullsized":"2023-09-23T03:19:23.511665+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:11:18.322404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.499265+0000","last_change":"2023-09-23T03:19:23.499265+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.499265+0000","last_undegraded":"2023-09-23T03:19:23.499265+0000","last_fullsized":"2023-09-23T03:19:23.499265+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.512892+0000","last_change":"2023-09-23T03:19:23.507592+0000","last_active":"2023-09-23T03:19:23.512892+0000","last_peered":"2023-09-23T03:19:23.512892+0000","last_clean":"2023-09-23T03:19:23.512892+0000","last_became_active":"2023-09-23T03:19:23.507316+0000","last_became_peered":"2023-09-23T03:19:23.507316+0000","last_unstale":"2023-09-23T03:19:23.512892+0000","last_undegraded":"2023-09-23T03:19:23.512892+0000","last_fullsized":"2023-09-23T03:19:23.512892+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:43:34.054466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.487374+0000","last_change":"2023-09-23T03:19:23.487374+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.487374+0000","last_undegraded":"2023-09-23T03:19:23.487374+0000","last_fullsized":"2023-09-23T03:19:23.487374+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.524603+0000","last_change":"2023-09-23T03:19:23.515237+0000","last_active":"2023-09-23T03:19:23.524603+0000","last_peered":"2023-09-23T03:19:23.524603+0000","last_clean":"2023-09-23T03:19:23.524603+0000","last_became_active":"2023-09-23T03:19:23.514946+0000","last_became_peered":"2023-09-23T03:19:23.514946+0000","last_unstale":"2023-09-23T03:19:23.524603+0000","last_undegraded":"2023-09-23T03:19:23.524603+0000","last_fullsized":"2023-09-23T03:19:23.524603+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:49:32.205787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.497140+0000","last_change":"2023-09-23T03:19:23.497140+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.497140+0000","last_undegraded":"2023-09-23T03:19:23.497140+0000","last_fullsized":"2023-09-23T03:19:23.497140+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.524349+0000","last_change":"2023-09-23T03:19:23.519665+0000","last_active":"2023-09-23T03:19:23.524349+0000","last_peered":"2023-09-23T03:19:23.524349+0000","last_clean":"2023-09-23T03:19:23.524349+0000","last_became_active":"2023-09-23T03:19:23.519393+0000","last_became_peered":"2023-09-23T03:19:23.519393+0000","last_unstale":"2023-09-23T03:19:23.524349+0000","last_undegraded":"2023-09-23T03:19:23.524349+0000","last_fullsized":"2023-09-23T03:19:23.524349+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:49:32.205787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.499552+0000","last_change":"2023-09-23T03:19:23.499552+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.499552+0000","last_undegraded":"2023-09-23T03:19:23.499552+0000","last_fullsized":"2023-09-23T03:19:23.499552+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.507040+0000","last_change":"2023-09-23T03:19:23.502992+0000","last_active":"2023-09-23T03:19:23.507040+0000","last_peered":"2023-09-23T03:19:23.507040+0000","last_clean":"2023-09-23T03:19:23.507040+0000","last_became_active":"2023-09-23T03:19:23.502688+0000","last_became_peered":"2023-09-23T03:19:23.502688+0000","last_unstale":"2023-09-23T03:19:23.507040+0000","last_undegraded":"2023-09-23T03:19:23.507040+0000","last_fullsized":"2023-09-23T03:19:23.507040+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:05:11.471065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.503106+0000","last_change":"2023-09-23T03:19:23.503106+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.503106+0000","last_undegraded":"2023-09-23T03:19:23.503106+0000","last_fullsized":"2023-09-23T03:19:23.503106+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.511859+0000","last_change":"2023-09-23T03:19:23.507612+0000","last_active":"2023-09-23T03:19:23.511859+0000","last_peered":"2023-09-23T03:19:23.511859+0000","last_clean":"2023-09-23T03:19:23.511859+0000","last_became_active":"2023-09-23T03:19:23.507211+0000","last_became_peered":"2023-09-23T03:19:23.507211+0000","last_unstale":"2023-09-23T03:19:23.511859+0000","last_undegraded":"2023-09-23T03:19:23.511859+0000","last_fullsized":"2023-09-23T03:19:23.511859+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:33:16.276149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.487706+0000","last_change":"2023-09-23T03:19:23.487706+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.487706+0000","last_undegraded":"2023-09-23T03:19:23.487706+0000","last_fullsized":"2023-09-23T03:19:23.487706+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.525158+0000","last_change":"2023-09-23T03:19:23.518829+0000","last_active":"2023-09-23T03:19:23.525158+0000","last_peered":"2023-09-23T03:19:23.525158+0000","last_clean":"2023-09-23T03:19:23.525158+0000","last_became_active":"2023-09-23T03:19:23.518512+0000","last_became_peered":"2023-09-23T03:19:23.518512+0000","last_unstale":"2023-09-23T03:19:23.525158+0000","last_undegraded":"2023-09-23T03:19:23.525158+0000","last_fullsized":"2023-09-23T03:19:23.525158+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:23:19.447543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.497713+0000","last_change":"2023-09-23T03:19:23.497713+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.497713+0000","last_undegraded":"2023-09-23T03:19:23.497713+0000","last_fullsized":"2023-09-23T03:19:23.497713+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.508037+0000","last_change":"2023-09-23T03:19:23.500147+0000","last_active":"2023-09-23T03:19:23.508037+0000","last_peered":"2023-09-23T03:19:23.508037+0000","last_clean":"2023-09-23T03:19:23.508037+0000","last_became_active":"2023-09-23T03:19:23.499826+0000","last_became_peered":"2023-09-23T03:19:23.499826+0000","last_unstale":"2023-09-23T03:19:23.508037+0000","last_undegraded":"2023-09-23T03:19:23.508037+0000","last_fullsized":"2023-09-23T03:19:23.508037+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:09:18.859490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.500650+0000","last_change":"2023-09-23T03:19:23.500650+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.500650+0000","last_undegraded":"2023-09-23T03:19:23.500650+0000","last_fullsized":"2023-09-23T03:19:23.500650+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.507070+0000","last_change":"2023-09-23T03:19:23.502333+0000","last_active":"2023-09-23T03:19:23.507070+0000","last_peered":"2023-09-23T03:19:23.507070+0000","last_clean":"2023-09-23T03:19:23.507070+0000","last_became_active":"2023-09-23T03:19:23.501997+0000","last_became_peered":"2023-09-23T03:19:23.501997+0000","last_unstale":"2023-09-23T03:19:23.507070+0000","last_undegraded":"2023-09-23T03:19:23.507070+0000","last_fullsized":"2023-09-23T03:19:23.507070+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:22:22.759701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.490693+0000","last_change":"2023-09-23T03:19:23.490693+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.490693+0000","last_undegraded":"2023-09-23T03:19:23.490693+0000","last_fullsized":"2023-09-23T03:19:23.490693+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.512081+0000","last_change":"2023-09-23T03:19:23.507705+0000","last_active":"2023-09-23T03:19:23.512081+0000","last_peered":"2023-09-23T03:19:23.512081+0000","last_clean":"2023-09-23T03:19:23.512081+0000","last_became_active":"2023-09-23T03:19:23.507409+0000","last_became_peered":"2023-09-23T03:19:23.507409+0000","last_unstale":"2023-09-23T03:19:23.512081+0000","last_undegraded":"2023-09-23T03:19:23.512081+0000","last_fullsized":"2023-09-23T03:19:23.512081+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:43:34.054466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.488782+0000","last_change":"2023-09-23T03:19:23.488782+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.488782+0000","last_undegraded":"2023-09-23T03:19:23.488782+0000","last_fullsized":"2023-09-23T03:19:23.488782+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.524549+0000","last_change":"2023-09-23T03:19:23.514960+0000","last_active":"2023-09-23T03:19:23.524549+0000","last_peered":"2023-09-23T03:19:23.524549+0000","last_clean":"2023-09-23T03:19:23.524549+0000","last_became_active":"2023-09-23T03:19:23.514644+0000","last_became_peered":"2023-09-23T03:19:23.514644+0000","last_unstale":"2023-09-23T03:19:23.524549+0000","last_undegraded":"2023-09-23T03:19:23.524549+0000","last_fullsized":"2023-09-23T03:19:23.524549+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:00:17.646758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.502007+0000","last_change":"2023-09-23T03:19:23.502007+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.502007+0000","last_undegraded":"2023-09-23T03:19:23.502007+0000","last_fullsized":"2023-09-23T03:19:23.502007+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.515734+0000","last_change":"2023-09-23T03:19:23.512352+0000","last_active":"2023-09-23T03:19:23.515734+0000","last_peered":"2023-09-23T03:19:23.515734+0000","last_clean":"2023-09-23T03:19:23.515734+0000","last_became_active":"2023-09-23T03:19:23.512039+0000","last_became_peered":"2023-09-23T03:19:23.512039+0000","last_unstale":"2023-09-23T03:19:23.515734+0000","last_undegraded":"2023-09-23T03:19:23.515734+0000","last_fullsized":"2023-09-23T03:19:23.515734+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:43:42.533621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.502444+0000","last_change":"2023-09-23T03:19:23.502444+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.502444+0000","last_undegraded":"2023-09-23T03:19:23.502444+0000","last_fullsized":"2023-09-23T03:19:23.502444+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.511839+0000","last_change":"2023-09-23T03:19:23.506342+0000","last_active":"2023-09-23T03:19:23.511839+0000","last_peered":"2023-09-23T03:19:23.511839+0000","last_clean":"2023-09-23T03:19:23.511839+0000","last_became_active":"2023-09-23T03:19:23.506078+0000","last_became_peered":"2023-09-23T03:19:23.506078+0000","last_unstale":"2023-09-23T03:19:23.511839+0000","last_undegraded":"2023-09-23T03:19:23.511839+0000","last_fullsized":"2023-09-23T03:19:23.511839+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:00:17.646758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.500536+0000","last_change":"2023-09-23T03:19:23.500536+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.500536+0000","last_undegraded":"2023-09-23T03:19:23.500536+0000","last_fullsized":"2023-09-23T03:19:23.500536+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.516019+0000","last_change":"2023-09-23T03:19:23.510497+0000","last_active":"2023-09-23T03:19:23.516019+0000","last_peered":"2023-09-23T03:19:23.516019+0000","last_clean":"2023-09-23T03:19:23.516019+0000","last_became_active":"2023-09-23T03:19:23.510155+0000","last_became_peered":"2023-09-23T03:19:23.510155+0000","last_unstale":"2023-09-23T03:19:23.516019+0000","last_undegraded":"2023-09-23T03:19:23.516019+0000","last_fullsized":"2023-09-23T03:19:23.516019+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:15:52.148791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.491878+0000","last_change":"2023-09-23T03:19:23.491878+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.491878+0000","last_undegraded":"2023-09-23T03:19:23.491878+0000","last_fullsized":"2023-09-23T03:19:23.491878+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.524455+0000","last_change":"2023-09-23T03:19:23.513693+0000","last_active":"2023-09-23T03:19:23.524455+0000","last_peered":"2023-09-23T03:19:23.524455+0000","last_clean":"2023-09-23T03:19:23.524455+0000","last_became_active":"2023-09-23T03:19:23.513359+0000","last_became_peered":"2023-09-23T03:19:23.513359+0000","last_unstale":"2023-09-23T03:19:23.524455+0000","last_undegraded":"2023-09-23T03:19:23.524455+0000","last_fullsized":"2023-09-23T03:19:23.524455+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:09:18.859490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.487319+0000","last_change":"2023-09-23T03:19:23.487319+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.487319+0000","last_undegraded":"2023-09-23T03:19:23.487319+0000","last_fullsized":"2023-09-23T03:19:23.487319+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.519999+0000","last_change":"2023-09-23T03:19:23.519087+0000","last_active":"2023-09-23T03:19:23.519999+0000","last_peered":"2023-09-23T03:19:23.519999+0000","last_clean":"2023-09-23T03:19:23.519999+0000","last_became_active":"2023-09-23T03:19:23.518827+0000","last_became_peered":"2023-09-23T03:19:23.518827+0000","last_unstale":"2023-09-23T03:19:23.519999+0000","last_undegraded":"2023-09-23T03:19:23.519999+0000","last_fullsized":"2023-09-23T03:19:23.519999+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:15:52.148791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.498413+0000","last_change":"2023-09-23T03:19:23.498413+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.498413+0000","last_undegraded":"2023-09-23T03:19:23.498413+0000","last_fullsized":"2023-09-23T03:19:23.498413+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.502654+0000","last_change":"2023-09-23T03:19:23.498946+0000","last_active":"2023-09-23T03:19:23.502654+0000","last_peered":"2023-09-23T03:19:23.502654+0000","last_clean":"2023-09-23T03:19:23.502654+0000","last_became_active":"2023-09-23T03:19:23.498522+0000","last_became_peered":"2023-09-23T03:19:23.498522+0000","last_unstale":"2023-09-23T03:19:23.502654+0000","last_undegraded":"2023-09-23T03:19:23.502654+0000","last_fullsized":"2023-09-23T03:19:23.502654+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:39:49.215616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.499307+0000","last_change":"2023-09-23T03:19:23.499307+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.499307+0000","last_undegraded":"2023-09-23T03:19:23.499307+0000","last_fullsized":"2023-09-23T03:19:23.499307+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.512203+0000","last_change":"2023-09-23T03:19:23.509484+0000","last_active":"2023-09-23T03:19:23.512203+0000","last_peered":"2023-09-23T03:19:23.512203+0000","last_clean":"2023-09-23T03:19:23.512203+0000","last_became_active":"2023-09-23T03:19:23.509203+0000","last_became_peered":"2023-09-23T03:19:23.509203+0000","last_unstale":"2023-09-23T03:19:23.512203+0000","last_undegraded":"2023-09-23T03:19:23.512203+0000","last_fullsized":"2023-09-23T03:19:23.512203+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:10:34.290176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.487164+0000","last_change":"2023-09-23T03:19:23.487164+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.487164+0000","last_undegraded":"2023-09-23T03:19:23.487164+0000","last_fullsized":"2023-09-23T03:19:23.487164+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.518826+0000","last_change":"2023-09-23T03:19:23.513953+0000","last_active":"2023-09-23T03:19:23.518826+0000","last_peered":"2023-09-23T03:19:23.518826+0000","last_clean":"2023-09-23T03:19:23.518826+0000","last_became_active":"2023-09-23T03:19:23.513616+0000","last_became_peered":"2023-09-23T03:19:23.513616+0000","last_unstale":"2023-09-23T03:19:23.518826+0000","last_undegraded":"2023-09-23T03:19:23.518826+0000","last_fullsized":"2023-09-23T03:19:23.518826+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:23:05.866775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2023-09-23T03:19:23.487239+0000","last_change":"2023-09-23T03:19:23.487239+0000","last_active":"2023-09-23T03:19:23.465704+0000","last_peered":"2023-09-23T03:19:23.465704+0000","last_clean":"2023-09-23T03:19:23.465704+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:19:23.487239+0000","last_undegraded":"2023-09-23T03:19:23.487239+0000","last_fullsized":"2023-09-23T03:19:23.487239+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-09-23T03:19:23.524435+0000","last_change":"2023-09-23T03:19:23.519480+0000","last_active":"2023-09-23T03:19:23.524435+0000","last_peered":"2023-09-23T03:19:23.524435+0000","last_clean":"2023-09-23T03:19:23.524435+0000","last_became_active":"2023-09-23T03:19:23.519191+0000","last_became_peered":"2023-09-23T03:19:23.519191+0000","last_unstale":"2023-09-23T03:19:23.524435+0000","last_undegraded":"2023-09-23T03:19:23.524435+0000","last_fullsized":"2023-09-23T03:19:23.524435+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:43:42.533621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":198,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-09-23T03:19:22.484979+0000","last_change":"2023-09-23T03:15:11.042447+0000","last_active":"2023-09-23T03:19:22.484979+0000","last_peered":"2023-09-23T03:19:22.484979+0000","last_clean":"2023-09-23T03:19:22.484979+0000","last_became_active":"2023-09-23T03:15:11.040322+0000","last_became_peered":"2023-09-23T03:15:11.040322+0000","last_unstale":"2023-09-23T03:19:22.484979+0000","last_undegraded":"2023-09-23T03:19:22.484979+0000","last_fullsized":"2023-09-23T03:19:22.484979+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_clean_scrub_stamp":"2023-09-23T03:15:09.474842+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:18:36.331645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574901,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1314816,"data_stored":923854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.98799999999999999,"15min":0.99199999999999999},"min":{"1min":0.68799999999999994,"5min":0.442,"15min":0.442},"max":{"1min":1.55,"5min":2.149,"15min":2.149},"last":1.075},{"interface":"front","average":{"1min":1.111,"5min":1.0569999999999999,"15min":1.0469999999999999},"min":{"1min":0.82099999999999995,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.681,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.95299999999999996}]},{"osd":1,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.198,"15min":1.22},"min":{"1min":0.74299999999999999,"5min":0.504,"15min":0.504},"max":{"1min":1.5649999999999999,"5min":3.1859999999999999,"15min":3.1859999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.248,"5min":1.294,"15min":1.3759999999999999},"min":{"1min":0.93899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.5840000000000001,"5min":2.613,"15min":2.613},"last":1.335}]},{"osd":2,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.1240000000000001,"15min":1.155},"min":{"1min":0.70199999999999996,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.6399999999999999,"5min":2.331,"15min":2.331},"last":0.879},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.242,"15min":1.296},"min":{"1min":0.75600000000000001,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.673,"5min":3.093,"15min":3.093},"last":1.3740000000000001}]},{"osd":3,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.248,"15min":1.2989999999999999},"min":{"1min":0.90000000000000002,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.7310000000000001,"5min":2.532,"15min":2.532},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.389,"5min":1.377,"15min":1.4410000000000001},"min":{"1min":0.95099999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.784,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":1.2829999999999999}]},{"osd":4,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.415,"15min":1.528},"min":{"1min":0.79200000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.7070000000000001,"5min":2.7210000000000001,"15min":2.7210000000000001},"last":1.5489999999999999},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.4730000000000001,"15min":1.556},"min":{"1min":0.81699999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.5880000000000001,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":1.5089999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.4179999999999999,"15min":1.4810000000000001},"min":{"1min":0.72699999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.736,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.288,"5min":1.4510000000000001,"15min":1.548},"min":{"1min":1.0069999999999999,"5min":0.497,"15min":0.497},"max":{"1min":1.7689999999999999,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":1.1819999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.5669999999999999,"15min":1.649},"min":{"1min":1.2410000000000001,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.8580000000000001,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":1.423},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.5089999999999999,"15min":1.607},"min":{"1min":1.002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.778,"5min":3.6739999999999999,"15min":3.6739999999999999},"last":1.012}]}]},{"osd":6,"up_from":13,"seq":55834574901,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":720896,"data_stored":333486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.048,"15min":1.0289999999999999},"min":{"1min":0.68899999999999995,"5min":0.38,"15min":0.38},"max":{"1min":1.6379999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.04},{"interface":"front","average":{"1min":1.083,"5min":1.0469999999999999,"15min":1.0549999999999999},"min":{"1min":0.61599999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.516,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.97899999999999998}]},{"osd":1,"last update":"Sat Sep 23 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.109,"15min":1.0980000000000001},"min":{"1min":0.65900000000000003,"5min":0.438,"15min":0.438},"max":{"1min":1.6830000000000001,"5min":1.867,"15min":1.867},"last":0.88},{"interface":"front","average":{"1min":1.23,"5min":1.2170000000000001,"15min":1.242},"min":{"1min":0.82399999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.6339999999999999,"5min":2.242,"15min":2.242},"last":1.3149999999999999}]},{"osd":2,"last update":"Sat Sep 23 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.1180000000000001,"15min":1.1060000000000001},"min":{"1min":0.82499999999999996,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4470000000000001,"5min":2.097,"15min":2.097},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.194,"5min":1.234,"15min":1.2669999999999999},"min":{"1min":0.66800000000000004,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.8149999999999999,"5min":1.903,"15min":1.903},"last":1.25}]},{"osd":3,"last update":"Sat Sep 23 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.3380000000000001,"15min":1.361},"min":{"1min":0.80200000000000005,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.8879999999999999,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":1.587},{"interface":"front","average":{"1min":1.278,"5min":1.389,"15min":1.4550000000000001},"min":{"1min":0.76600000000000001,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.657,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":1.2170000000000001}]},{"osd":4,"last update":"Sat Sep 23 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.397,"15min":1.395},"min":{"1min":0.81399999999999995,"5min":0.498,"15min":0.498},"max":{"1min":1.827,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.2849999999999999,"15min":1.294},"min":{"1min":0.75700000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.661,"5min":2.2109999999999999,"15min":2.2109999999999999},"last":0.84399999999999997}]},{"osd":5,"last update":"Sat Sep 23 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.391,"15min":1.395},"min":{"1min":0.84699999999999998,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.7390000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.4810000000000001,"15min":1.486},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.9390000000000001,"5min":4.758,"15min":4.758},"last":1.6779999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.508,"15min":1.5549999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.77,"5min":2.476,"15min":2.476},"last":1.151},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.532,"15min":1.5640000000000001},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.1739999999999999,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.472}]}]},{"osd":5,"up_from":13,"seq":55834574903,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":720896,"data_stored":333486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.044,"15min":1.03},"min":{"1min":0.80200000000000005,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.474,"5min":1.637,"15min":1.637},"last":1.075},{"interface":"front","average":{"1min":1.141,"5min":1.0609999999999999,"15min":1.032},"min":{"1min":0.78800000000000003,"5min":0.63,"15min":0.63},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":1.621}]},{"osd":1,"last update":"Sat Sep 23 03:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.212,"15min":1.224},"min":{"1min":0.95599999999999996,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.6459999999999999,"5min":1.849,"15min":1.849},"last":1.135},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.2869999999999999,"15min":1.3220000000000001},"min":{"1min":1.085,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.7410000000000001,"5min":1.893,"15min":1.893},"last":1.3759999999999999}]},{"osd":2,"last update":"Sat Sep 23 03:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.22,"15min":1.2170000000000001},"min":{"1min":1.0189999999999999,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.597,"5min":1.681,"15min":1.681},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.161,"5min":1.244,"15min":1.2969999999999999},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.5740000000000001,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":1.2090000000000001}]},{"osd":3,"last update":"Sat Sep 23 03:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.2509999999999999,"15min":1.361},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.423,"5min":1.905,"15min":1.905},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.3859999999999999,"15min":1.47},"min":{"1min":1.056,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.792,"5min":3.6110000000000002,"15min":3.6110000000000002},"last":1.4419999999999999}]},{"osd":4,"last update":"Sat Sep 23 03:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3939999999999999,"15min":1.444},"min":{"1min":0.83099999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.8500000000000001,"5min":2.069,"15min":2.069},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.401,"15min":1.4530000000000001},"min":{"1min":0.91100000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.7889999999999999,"5min":2.319,"15min":2.319},"last":1.3180000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":1.5089999999999999,"15min":1.5409999999999999},"min":{"1min":0.69899999999999995,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.948,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":1.6739999999999999},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.5309999999999999,"15min":1.613},"min":{"1min":0.97099999999999997,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.877,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.5760000000000001}]},{"osd":7,"last update":"Sat Sep 23 03:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.4570000000000001,"15min":1.538},"min":{"1min":0.91600000000000004,"5min":0.626,"15min":0.626},"max":{"1min":1.6839999999999999,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.734},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.585,"15min":1.643},"min":{"1min":1.046,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.7330000000000001,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.417}]}]},{"osd":4,"up_from":13,"seq":55834574902,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":708608,"data_stored":326899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:19:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.1279999999999999,"15min":1.1379999999999999},"min":{"1min":0.64800000000000002,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.639,"5min":1.806,"15min":1.806},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.1739999999999999,"15min":1.141},"min":{"1min":0.76500000000000001,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.905,"5min":1.9179999999999999,"15min":1.9179999999999999},"last":1.6479999999999999}]},{"osd":1,"last update":"Sat Sep 23 03:19:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.2130000000000001,"15min":1.2150000000000001},"min":{"1min":0.90200000000000002,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1910000000000001,"15min":1.2210000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.4710000000000001,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":0.97799999999999998}]},{"osd":2,"last update":"Sat Sep 23 03:19:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.1870000000000001,"15min":1.246},"min":{"1min":0.71199999999999997,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.129,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.246,"15min":1.224},"min":{"1min":0.91200000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.909,"5min":1.909,"15min":1.909},"last":0.90000000000000002}]},{"osd":3,"last update":"Sat Sep 23 03:19:15 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.1479999999999999,"15min":1.171},"min":{"1min":0.71999999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.5369999999999999,"5min":1.583,"15min":1.583},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.296,"5min":1.319,"15min":1.341},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.1779999999999999,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.302}]},{"osd":5,"last update":"Sat Sep 23 03:19:15 2023","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.371,"15min":1.345},"min":{"1min":0.99299999999999999,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.8050000000000002,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.371},{"interface":"front","average":{"1min":1.371,"5min":1.3300000000000001,"15min":1.3260000000000001},"min":{"1min":0.84299999999999997,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":2.887,"5min":2.887,"15min":2.887},"last":1.4690000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:19:15 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.397,"15min":1.391},"min":{"1min":1.048,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.4249999999999998,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.6619999999999999,"5min":1.5049999999999999,"15min":1.478},"min":{"1min":1.323,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":2.052,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.75}]},{"osd":7,"last update":"Sat Sep 23 03:19:15 2023","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.4650000000000001,"15min":1.4730000000000001},"min":{"1min":1.1819999999999999,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":2.5680000000000001,"5min":2.5680000000000001,"15min":2.5680000000000001},"last":1.716},{"interface":"front","average":{"1min":1.6379999999999999,"5min":1.5580000000000001,"15min":1.5649999999999999},"min":{"1min":1.056,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":2.4769999999999999,"5min":2.4769999999999999,"15min":2.4769999999999999},"last":1.593}]}]},{"osd":3,"up_from":13,"seq":55834574902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":708608,"data_stored":326899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.85399999999999998,"15min":0.875},"min":{"1min":0.42199999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.7090000000000001,"5min":1.744,"15min":1.744},"last":1.478},{"interface":"front","average":{"1min":0.876,"5min":0.86599999999999999,"15min":0.90200000000000002},"min":{"1min":0.51600000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.341,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.94899999999999995}]},{"osd":1,"last update":"Sat Sep 23 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.90700000000000003,"15min":0.96399999999999997},"min":{"1min":0.442,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.5640000000000001,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":1.401},{"interface":"front","average":{"1min":1.02,"5min":0.98099999999999998,"15min":0.98499999999999999},"min":{"1min":0.54700000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.3839999999999999,"5min":1.704,"15min":1.704},"last":1.127}]},{"osd":2,"last update":"Sat Sep 23 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.0609999999999999,"15min":1.113},"min":{"1min":0.54800000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.639,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.639},{"interface":"front","average":{"1min":1.165,"5min":1.097,"15min":1.129},"min":{"1min":0.64400000000000002,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.6830000000000001,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.6830000000000001}]},{"osd":4,"last update":"Sat Sep 23 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.2809999999999999,"15min":1.333},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.0099999999999998,"5min":2.698,"15min":2.698},"last":2.0099999999999998},{"interface":"front","average":{"1min":1.353,"5min":1.381,"15min":1.4470000000000001},"min":{"1min":0.95399999999999996,"5min":0.752,"15min":0.752},"max":{"1min":1.825,"5min":2.419,"15min":2.419},"last":1.825}]},{"osd":5,"last update":"Sat Sep 23 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.339,"15min":1.361},"min":{"1min":0.90300000000000002,"5min":0.753,"15min":0.753},"max":{"1min":1.8799999999999999,"5min":1.917,"15min":1.917},"last":1.867},{"interface":"front","average":{"1min":1.321,"5min":1.3049999999999999,"15min":1.3620000000000001},"min":{"1min":0.83499999999999996,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.716,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.716}]},{"osd":6,"last update":"Sat Sep 23 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.359,"15min":1.423},"min":{"1min":0.82699999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.9550000000000001,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":1.9550000000000001},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4430000000000001,"15min":1.482},"min":{"1min":1.05,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.7729999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.417,"15min":1.5009999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":2.2120000000000002,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.909},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.4399999999999999,"15min":1.526},"min":{"1min":0.85799999999999998,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.2029999999999998,"5min":2.5379999999999998,"15min":2.5379999999999998},"last":2.2029999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":720896,"data_stored":333486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.83799999999999997,"15min":0.83699999999999997},"min":{"1min":0.442,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":0.501},{"interface":"front","average":{"1min":0.998,"5min":0.86199999999999999,"15min":0.84299999999999997},"min":{"1min":0.48899999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.5329999999999999,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.77200000000000002}]},{"osd":1,"last update":"Sat Sep 23 03:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.84799999999999998,"15min":0.85999999999999999},"min":{"1min":0.55800000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.026,"5min":0.93300000000000005,"15min":0.94099999999999995},"min":{"1min":0.55100000000000005,"5min":0.38,"15min":0.38},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.95999999999999996}]},{"osd":3,"last update":"Sat Sep 23 03:19:12 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":0.96599999999999997,"15min":0.95199999999999996},"min":{"1min":0.53900000000000003,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.4279999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.073,"5min":0.98499999999999999,"15min":0.997},"min":{"1min":0.60699999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.534,"5min":1.78,"15min":1.78},"last":0.89900000000000002}]},{"osd":4,"last update":"Sat Sep 23 03:19:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2729999999999999,"15min":1.3240000000000001},"min":{"1min":0.86199999999999999,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.819,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.071},{"interface":"front","average":{"1min":1.296,"5min":1.2969999999999999,"15min":1.341},"min":{"1min":0.77600000000000002,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.631,"5min":2.234,"15min":2.234},"last":1.0029999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:19:12 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.339,"15min":1.409},"min":{"1min":0.97499999999999998,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.7849999999999999,"5min":3.9409999999999998,"15min":3.9409999999999998},"last":1.032},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.4199999999999999,"15min":1.5329999999999999},"min":{"1min":0.81299999999999994,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.9359999999999999,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":1.1759999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:19:12 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.3580000000000001,"15min":1.417},"min":{"1min":0.83299999999999996,"5min":0.755,"15min":0.755},"max":{"1min":1.8540000000000001,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.458,"5min":1.393,"15min":1.478},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.1680000000000001,"5min":4.5800000000000001,"15min":4.5800000000000001},"last":1.1279999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:19:12 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.3600000000000001,"15min":1.4019999999999999},"min":{"1min":1.0469999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.996,"5min":2.3149999999999999,"15min":2.3149999999999999},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.413,"15min":1.4550000000000001},"min":{"1min":0.873,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.9259999999999999,"5min":2.621,"15min":2.621},"last":1.109}]}]},{"osd":1,"up_from":13,"seq":55834574904,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":720896,"data_stored":333486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.80000000000000004,"15min":0.80800000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.2490000000000001,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.85099999999999998},"min":{"1min":0.34799999999999998,"5min":0.312,"15min":0.312},"max":{"1min":1.2210000000000001,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":0.623}]},{"osd":2,"last update":"Sat Sep 23 03:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.84999999999999998,"15min":0.89400000000000002},"min":{"1min":0.52000000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.409,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":1.278},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.004,"15min":1.0369999999999999},"min":{"1min":0.54500000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5600000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.524}]},{"osd":3,"last update":"Sat Sep 23 03:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.0269999999999999,"15min":1.087},"min":{"1min":0.48799999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.5389999999999999,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":1.24},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.0980000000000001,"15min":1.1619999999999999},"min":{"1min":0.56200000000000006,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.7649999999999999,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":1.6850000000000001}]},{"osd":4,"last update":"Sat Sep 23 03:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.2330000000000001,"15min":1.2829999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.7270000000000001,"5min":3.2629999999999999,"15min":3.2629999999999999},"last":1.411},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.262,"15min":1.3},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6359999999999999,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":1.456}]},{"osd":5,"last update":"Sat Sep 23 03:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.254,"15min":1.2989999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.9319999999999999,"5min":3.161,"15min":3.161},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3,"15min":1.3460000000000001},"min":{"1min":0.80400000000000005,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.698,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":1.5840000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.2969999999999999,"15min":1.3400000000000001},"min":{"1min":0.76500000000000001,"5min":0.747,"15min":0.747},"max":{"1min":2.0329999999999999,"5min":3.0179999999999998,"15min":3.0179999999999998},"last":1.637},{"interface":"front","average":{"1min":1.333,"5min":1.3520000000000001,"15min":1.3979999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.7909999999999999,"5min":3.3580000000000001,"15min":3.3580000000000001},"last":1.78}]},{"osd":7,"last update":"Sat Sep 23 03:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.363,"15min":1.413},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":2.085,"5min":3.1099999999999999,"15min":3.1099999999999999},"last":1.6619999999999999},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.415,"15min":1.4810000000000001},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.7330000000000001,"5min":3.306,"15min":3.306},"last":1.722}]}]},{"osd":0,"up_from":13,"seq":55834574904,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1310720,"data_stored":922835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.81200000000000006,"15min":0.81999999999999995},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.3180000000000001,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.86599999999999999,"15min":0.89300000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.363,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.96399999999999997}]},{"osd":2,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.81100000000000005,"15min":0.80300000000000005},"min":{"1min":0.48599999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.22,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.97299999999999998,"15min":0.98499999999999999},"min":{"1min":0.46700000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":1.3029999999999999}]},{"osd":3,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93300000000000005,"15min":0.91500000000000004},"min":{"1min":0.44800000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.226,"5min":1.591,"15min":1.591},"last":1.198},{"interface":"front","average":{"1min":1.117,"5min":0.996,"15min":1.006},"min":{"1min":0.57299999999999995,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.4990000000000001,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.147}]},{"osd":4,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.204,"15min":1.204},"min":{"1min":0.81499999999999995,"5min":0.755,"15min":0.755},"max":{"1min":1.601,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.7450000000000001},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.2410000000000001,"15min":1.274},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.4610000000000001,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.456}]},{"osd":5,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.2250000000000001,"15min":1.2290000000000001},"min":{"1min":0.872,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.6739999999999999,"5min":1.867,"15min":1.867},"last":1.474},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.294,"15min":1.272},"min":{"1min":0.76400000000000001,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":1.8080000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.3009999999999999,"15min":1.3029999999999999},"min":{"1min":0.99399999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":1.552},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.325,"15min":1.3029999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.875,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.873}]},{"osd":7,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.329,"15min":1.3160000000000001},"min":{"1min":0.93300000000000005,"5min":0.747,"15min":0.747},"max":{"1min":1.6499999999999999,"5min":1.944,"15min":1.944},"last":1.528},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.3979999999999999,"15min":1.4059999999999999},"min":{"1min":0.91800000000000004,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.756,"5min":2.048,"15min":2.048},"last":1.7170000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-23T03:19:33.982 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-23T03:19:34.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.209+0000 7ffbecad7640 1 Processor -- start 2023-09-23T03:19:34.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.209+0000 7ffbecad7640 1 -- start start 2023-09-23T03:19:34.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.210+0000 7ffbecad7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe80cf130 0x7ffbe80ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:34.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.210+0000 7ffbecad7640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ffbe80b2fe0 0x7ffbe815d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:34.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.211+0000 7ffbecad7640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbe806ceb0 con 0x7ffbe80ada30 2023-09-23T03:19:34.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.211+0000 7ffbecad7640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbe80ade10 con 0x7ffbe80b2fe0 2023-09-23T03:19:34.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.211+0000 7ffbecad7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe8159c90 con 0x7ffbe80cf130 2023-09-23T03:19:34.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.211+0000 7ffbe659c640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ffbe80ada30 0x7ffbe80cea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:59014/0 (socket says 172.21.15.154:59014) 2023-09-23T03:19:34.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.211+0000 7ffbe659c640 1 -- 172.21.15.154:0/3156130902 learned_addr learned my addr 172.21.15.154:0/3156130902 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:19:34.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.211+0000 7ffbe5d9b640 1 --2- 172.21.15.154:0/3156130902 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe80cf130 0x7ffbe80ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:34.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.211+0000 7ffbe6d9d640 1 --2- 172.21.15.154:0/3156130902 >> v2:172.21.15.190:3300/0 conn(0x7ffbe80b2fe0 0x7ffbe815d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:34.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.211+0000 7ffbe559a640 1 -- 172.21.15.154:0/3156130902 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 243772172 0 0) 0x7ffbd8001290 con 0x7ffbe80ada30 2023-09-23T03:19:34.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.211+0000 7ffbe559a640 1 -- 172.21.15.154:0/3156130902 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbd0003520 con 0x7ffbe80ada30 2023-09-23T03:19:34.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.212+0000 7ffbe559a640 1 -- 172.21.15.154:0/3156130902 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3059040432 0 0) 0x7ffbd0003520 con 0x7ffbe80ada30 2023-09-23T03:19:34.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.212+0000 7ffbe559a640 1 -- 172.21.15.154:0/3156130902 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe80cf130 msgr2=0x7ffbe80ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:34.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.212+0000 7ffbe559a640 1 --2- 172.21.15.154:0/3156130902 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe80cf130 0x7ffbe80ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:34.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.212+0000 7ffbe559a640 1 -- 172.21.15.154:0/3156130902 >> v2:172.21.15.190:3300/0 conn(0x7ffbe80b2fe0 msgr2=0x7ffbe815d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:34.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.212+0000 7ffbe559a640 1 --2- 172.21.15.154:0/3156130902 >> v2:172.21.15.190:3300/0 conn(0x7ffbe80b2fe0 0x7ffbe815d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:34.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.212+0000 7ffbe559a640 1 -- 172.21.15.154:0/3156130902 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe806c110 con 0x7ffbe80ada30 2023-09-23T03:19:34.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.212+0000 7ffbe559a640 1 -- 172.21.15.154:0/3156130902 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7ffbd8001980 con 0x7ffbe80ada30 2023-09-23T03:19:34.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.212+0000 7ffbe559a640 1 -- 172.21.15.154:0/3156130902 >> v1:172.21.15.154:6789/0 conn(0x7ffbe80ada30 legacy=0x7ffbe80cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:34.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.212+0000 7ffbe559a640 1 --2- 172.21.15.154:0/3156130902 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe80cf130 0x7ffbd0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:34.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.212+0000 7ffbe559a640 1 --2- 172.21.15.154:0/3156130902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbd0003f40 0x7ffbd00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:34.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.212+0000 7ffbe559a640 1 --2- 172.21.15.154:0/3156130902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbd0004d00 0x7ffbd0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:34.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.212+0000 7ffbe559a640 1 -- 172.21.15.154:0/3156130902 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe80ade10 con 0x7ffbd0004d00 2023-09-23T03:19:34.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.212+0000 7ffbe559a640 1 -- 172.21.15.154:0/3156130902 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe8159c90 con 0x7ffbd0003f40 2023-09-23T03:19:34.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.212+0000 7ffbe559a640 1 -- 172.21.15.154:0/3156130902 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd000db90 con 0x7ffbe80cf130 2023-09-23T03:19:34.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.212+0000 7ffbe6d9d640 1 --2- 172.21.15.154:0/3156130902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbd0004d00 0x7ffbd0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:34.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.213+0000 7ffbe5d9b640 1 --2- 172.21.15.154:0/3156130902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbd0003f40 0x7ffbd00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:34.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.213+0000 7ffbe659c640 1 --2- 172.21.15.154:0/3156130902 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe80cf130 0x7ffbd0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:34.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.213+0000 7ffbe6d9d640 1 -- 172.21.15.154:0/3156130902 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe80cf130 msgr2=0x7ffbd0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:34.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.213+0000 7ffbe6d9d640 1 --2- 172.21.15.154:0/3156130902 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe80cf130 0x7ffbd0003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:34.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.213+0000 7ffbe6d9d640 1 -- 172.21.15.154:0/3156130902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbd0003f40 msgr2=0x7ffbd00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:34.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.213+0000 7ffbe6d9d640 1 --2- 172.21.15.154:0/3156130902 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbd0003f40 0x7ffbd00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:34.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.213+0000 7ffbe6d9d640 1 -- 172.21.15.154:0/3156130902 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd0003520 con 0x7ffbd0004d00 2023-09-23T03:19:34.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.214+0000 7ffbe6d9d640 1 --2- 172.21.15.154:0/3156130902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbd0004d00 0x7ffbd0005180 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0009fc0 tx=0x7ffbd00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9249bddc057ad731 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:34.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.214+0000 7ffbe559a640 1 -- 172.21.15.154:0/3156130902 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe002cbf0 con 0x7ffbd0004d00 2023-09-23T03:19:34.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.214+0000 7ffbe559a640 1 -- 172.21.15.154:0/3156130902 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbe000f040 con 0x7ffbd0004d00 2023-09-23T03:19:34.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.214+0000 7ffbe559a640 1 -- 172.21.15.154:0/3156130902 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe0034440 con 0x7ffbd0004d00 2023-09-23T03:19:34.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.215+0000 7ffbecad7640 1 -- 172.21.15.154:0/3156130902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbd0004d00 msgr2=0x7ffbd0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:34.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.215+0000 7ffbecad7640 1 --2- 172.21.15.154:0/3156130902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbd0004d00 0x7ffbd0005180 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0009fc0 tx=0x7ffbd00057f0 comp rx=0 tx=0).stop 2023-09-23T03:19:34.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.215+0000 7ffbe6d9d640 1 -- 172.21.15.154:0/3156130902 reap_dead start 2023-09-23T03:19:34.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.215+0000 7ffbecad7640 1 -- 172.21.15.154:0/3156130902 shutdown_connections 2023-09-23T03:19:34.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.215+0000 7ffbecad7640 1 -- 172.21.15.154:0/3156130902 >> 172.21.15.154:0/3156130902 conn(0x7ffbe806a0f0 msgr2=0x7ffbe806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:34.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.215+0000 7ffbecad7640 1 -- 172.21.15.154:0/3156130902 shutdown_connections 2023-09-23T03:19:34.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.215+0000 7ffbecad7640 1 -- 172.21.15.154:0/3156130902 wait complete. 2023-09-23T03:19:34.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.217+0000 7ffbecad7640 1 Processor -- start 2023-09-23T03:19:34.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.217+0000 7ffbecad7640 1 -- start start 2023-09-23T03:19:34.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.218+0000 7ffbecad7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbd0004d00 0x7ffbe81f1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:34.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.218+0000 7ffbecad7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbe80af1f0 0x7ffbe80af650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:34.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.218+0000 7ffbecad7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe80c3650 0x7ffbe80c3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:34.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.218+0000 7ffbecad7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe80afb90 con 0x7ffbe80af1f0 2023-09-23T03:19:34.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.218+0000 7ffbecad7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe81f2030 con 0x7ffbd0004d00 2023-09-23T03:19:34.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.218+0000 7ffbecad7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe80c43b0 con 0x7ffbe80c3650 2023-09-23T03:19:34.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.218+0000 7ffbe5d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbe80af1f0 0x7ffbe80af650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:34.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.219+0000 7ffbe5d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbe80af1f0 0x7ffbe80af650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57474/0 (socket says 172.21.15.154:57474) 2023-09-23T03:19:34.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.219+0000 7ffbe5d9b640 1 -- 172.21.15.154:0/60445170 learned_addr learned my addr 172.21.15.154:0/60445170 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:34.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.219+0000 7ffbe659c640 1 --2- 172.21.15.154:0/60445170 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbd0004d00 0x7ffbe81f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:34.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.219+0000 7ffbe6d9d640 1 --2- 172.21.15.154:0/60445170 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe80c3650 0x7ffbe80c3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:34.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.219+0000 7ffbe5d9b640 1 -- 172.21.15.154:0/60445170 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe80c3650 msgr2=0x7ffbe80c3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:34.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.219+0000 7ffbe5d9b640 1 --2- 172.21.15.154:0/60445170 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe80c3650 0x7ffbe80c3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:34.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.219+0000 7ffbe5d9b640 1 -- 172.21.15.154:0/60445170 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbd0004d00 msgr2=0x7ffbe81f1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:34.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.219+0000 7ffbe5d9b640 1 --2- 172.21.15.154:0/60445170 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbd0004d00 0x7ffbe81f1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:34.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.219+0000 7ffbe5d9b640 1 -- 172.21.15.154:0/60445170 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe80c4690 con 0x7ffbe80af1f0 2023-09-23T03:19:34.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.219+0000 7ffbe5d9b640 1 --2- 172.21.15.154:0/60445170 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbe80af1f0 0x7ffbe80af650 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc011600 tx=0x7ffbdc011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:34.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.219+0000 7ffbd6ffd640 1 -- 172.21.15.154:0/60445170 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc00f920 con 0x7ffbe80af1f0 2023-09-23T03:19:34.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.220+0000 7ffbd6ffd640 1 -- 172.21.15.154:0/60445170 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbdc00fa80 con 0x7ffbe80af1f0 2023-09-23T03:19:34.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.220+0000 7ffbd6ffd640 1 -- 172.21.15.154:0/60445170 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc00fdf0 con 0x7ffbe80af1f0 2023-09-23T03:19:34.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.220+0000 7ffbecad7640 1 -- 172.21.15.154:0/60445170 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbe80c4970 con 0x7ffbe80af1f0 2023-09-23T03:19:34.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.220+0000 7ffbecad7640 1 -- 172.21.15.154:0/60445170 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbe81f1c70 con 0x7ffbe80af1f0 2023-09-23T03:19:34.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.221+0000 7ffbecad7640 1 -- 172.21.15.154:0/60445170 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbe81f21a0 con 0x7ffbe80af1f0 2023-09-23T03:19:34.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.222+0000 7ffbd6ffd640 1 -- 172.21.15.154:0/60445170 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7ffbdc015460 con 0x7ffbe80af1f0 2023-09-23T03:19:34.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.223+0000 7ffbd6ffd640 1 --2- 172.21.15.154:0/60445170 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffbc806af60 0x7ffbc806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:34.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.223+0000 7ffbd6ffd640 1 -- 172.21.15.154:0/60445170 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ffbdc089db0 con 0x7ffbe80af1f0 2023-09-23T03:19:34.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.223+0000 7ffbe659c640 1 --2- 172.21.15.154:0/60445170 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffbc806af60 0x7ffbc806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:34.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.224+0000 7ffbe659c640 1 --2- 172.21.15.154:0/60445170 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffbc806af60 0x7ffbc806d420 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ffbd80013f0 tx=0x7ffbd8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:34.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.227+0000 7ffbd6ffd640 1 -- 172.21.15.154:0/60445170 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7ffbdc08e070 con 0x7ffbe80af1f0 2023-09-23T03:19:34.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.417+0000 7ffbecad7640 1 -- 172.21.15.154:0/60445170 --> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffbe806c6d0 con 0x7ffbc806af60 2023-09-23T03:19:34.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.439+0000 7ffbd6ffd640 1 -- 172.21.15.154:0/60445170 <== mgr.4110 v2:172.21.15.154:6832/2422652104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286426 (secure 0 0 0) 0x7ffbd80057a0 con 0x7ffbc806af60 2023-09-23T03:19:34.443 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:19:34.444 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2023-09-23T03:19:34.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.445+0000 7ffbecad7640 1 -- 172.21.15.154:0/60445170 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffbc806af60 msgr2=0x7ffbc806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:34.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.445+0000 7ffbecad7640 1 --2- 172.21.15.154:0/60445170 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffbc806af60 0x7ffbc806d420 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ffbd80013f0 tx=0x7ffbd8009040 comp rx=0 tx=0).stop 2023-09-23T03:19:34.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.445+0000 7ffbecad7640 1 -- 172.21.15.154:0/60445170 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbe80af1f0 msgr2=0x7ffbe80af650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:34.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.445+0000 7ffbecad7640 1 --2- 172.21.15.154:0/60445170 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbe80af1f0 0x7ffbe80af650 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc011600 tx=0x7ffbdc011ac0 comp rx=0 tx=0).stop 2023-09-23T03:19:34.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.445+0000 7ffbecad7640 1 -- 172.21.15.154:0/60445170 shutdown_connections 2023-09-23T03:19:34.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.445+0000 7ffbecad7640 1 --2- 172.21.15.154:0/60445170 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffbc806af60 0x7ffbc806d420 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:34.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.445+0000 7ffbecad7640 1 --2- 172.21.15.154:0/60445170 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbe80c3650 0x7ffbe80c3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:34.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.445+0000 7ffbecad7640 1 --2- 172.21.15.154:0/60445170 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbe80af1f0 0x7ffbe80af650 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:34.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.445+0000 7ffbecad7640 1 --2- 172.21.15.154:0/60445170 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbd0004d00 0x7ffbe81f1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:34.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.445+0000 7ffbecad7640 1 -- 172.21.15.154:0/60445170 >> 172.21.15.154:0/60445170 conn(0x7ffbe806a0f0 msgr2=0x7ffbe8072880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:34.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.445+0000 7ffbecad7640 1 -- 172.21.15.154:0/60445170 shutdown_connections 2023-09-23T03:19:34.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.445+0000 7ffbecad7640 1 -- 172.21.15.154:0/60445170 wait complete. 2023-09-23T03:19:34.465 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":187,"stamp":"2023-09-23T03:19:33.251576+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239852,"kb_used_data":7044,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754734868,"statfs":{"total":773094113280,"available":772848504832,"internally_reserved":0,"allocated":7213056,"data_stored":3962145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":238276154},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":54,"apply_latency_ms":54,"commit_latency_ns":54000000,"apply_latency_ns":54000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.766302"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:25.490415+0000","last_change":"2023-09-23T03:19:24.499292+0000","last_active":"2023-09-23T03:19:25.490415+0000","last_peered":"2023-09-23T03:19:25.490415+0000","last_clean":"2023-09-23T03:19:25.490415+0000","last_became_active":"2023-09-23T03:19:24.499014+0000","last_became_peered":"2023-09-23T03:19:24.499014+0000","last_unstale":"2023-09-23T03:19:25.490415+0000","last_undegraded":"2023-09-23T03:19:25.490415+0000","last_fullsized":"2023-09-23T03:19:25.490415+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:28:30.359169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.452782+0000","last_change":"2023-09-23T03:19:24.512006+0000","last_active":"2023-09-23T03:19:26.452782+0000","last_peered":"2023-09-23T03:19:26.452782+0000","last_clean":"2023-09-23T03:19:26.452782+0000","last_became_active":"2023-09-23T03:19:24.511439+0000","last_became_peered":"2023-09-23T03:19:24.511439+0000","last_unstale":"2023-09-23T03:19:26.452782+0000","last_undegraded":"2023-09-23T03:19:26.452782+0000","last_fullsized":"2023-09-23T03:19:26.452782+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:57:32.358230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.301902+0000","last_change":"2023-09-23T03:19:24.528520+0000","last_active":"2023-09-23T03:19:26.301902+0000","last_peered":"2023-09-23T03:19:26.301902+0000","last_clean":"2023-09-23T03:19:26.301902+0000","last_became_active":"2023-09-23T03:19:24.527009+0000","last_became_peered":"2023-09-23T03:19:24.527009+0000","last_unstale":"2023-09-23T03:19:26.301902+0000","last_undegraded":"2023-09-23T03:19:26.301902+0000","last_fullsized":"2023-09-23T03:19:26.301902+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:30:30.760554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:25.493977+0000","last_change":"2023-09-23T03:19:24.518783+0000","last_active":"2023-09-23T03:19:25.493977+0000","last_peered":"2023-09-23T03:19:25.493977+0000","last_clean":"2023-09-23T03:19:25.493977+0000","last_became_active":"2023-09-23T03:19:24.518419+0000","last_became_peered":"2023-09-23T03:19:24.518419+0000","last_unstale":"2023-09-23T03:19:25.493977+0000","last_undegraded":"2023-09-23T03:19:25.493977+0000","last_fullsized":"2023-09-23T03:19:25.493977+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:34:58.795835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:25.490489+0000","last_change":"2023-09-23T03:19:24.522159+0000","last_active":"2023-09-23T03:19:25.490489+0000","last_peered":"2023-09-23T03:19:25.490489+0000","last_clean":"2023-09-23T03:19:25.490489+0000","last_became_active":"2023-09-23T03:19:24.521867+0000","last_became_peered":"2023-09-23T03:19:24.521867+0000","last_unstale":"2023-09-23T03:19:25.490489+0000","last_undegraded":"2023-09-23T03:19:25.490489+0000","last_fullsized":"2023-09-23T03:19:25.490489+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:20:31.742328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.453343+0000","last_change":"2023-09-23T03:19:24.505420+0000","last_active":"2023-09-23T03:19:26.453343+0000","last_peered":"2023-09-23T03:19:26.453343+0000","last_clean":"2023-09-23T03:19:26.453343+0000","last_became_active":"2023-09-23T03:19:24.505146+0000","last_became_peered":"2023-09-23T03:19:24.505146+0000","last_unstale":"2023-09-23T03:19:26.453343+0000","last_undegraded":"2023-09-23T03:19:26.453343+0000","last_fullsized":"2023-09-23T03:19:26.453343+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:30:30.760554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.450473+0000","last_change":"2023-09-23T03:19:24.519375+0000","last_active":"2023-09-23T03:19:26.450473+0000","last_peered":"2023-09-23T03:19:26.450473+0000","last_clean":"2023-09-23T03:19:26.450473+0000","last_became_active":"2023-09-23T03:19:24.519115+0000","last_became_peered":"2023-09-23T03:19:24.519115+0000","last_unstale":"2023-09-23T03:19:26.450473+0000","last_undegraded":"2023-09-23T03:19:26.450473+0000","last_fullsized":"2023-09-23T03:19:26.450473+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:49:33.217404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.451417+0000","last_change":"2023-09-23T03:19:24.523950+0000","last_active":"2023-09-23T03:19:26.451417+0000","last_peered":"2023-09-23T03:19:26.451417+0000","last_clean":"2023-09-23T03:19:26.451417+0000","last_became_active":"2023-09-23T03:19:24.523623+0000","last_became_peered":"2023-09-23T03:19:24.523623+0000","last_unstale":"2023-09-23T03:19:26.451417+0000","last_undegraded":"2023-09-23T03:19:26.451417+0000","last_fullsized":"2023-09-23T03:19:26.451417+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:00:55.715688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.451157+0000","last_change":"2023-09-23T03:19:24.512384+0000","last_active":"2023-09-23T03:19:26.451157+0000","last_peered":"2023-09-23T03:19:26.451157+0000","last_clean":"2023-09-23T03:19:26.451157+0000","last_became_active":"2023-09-23T03:19:24.512104+0000","last_became_peered":"2023-09-23T03:19:24.512104+0000","last_unstale":"2023-09-23T03:19:26.451157+0000","last_undegraded":"2023-09-23T03:19:26.451157+0000","last_fullsized":"2023-09-23T03:19:26.451157+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:23:20.459160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.454773+0000","last_change":"2023-09-23T03:19:24.518964+0000","last_active":"2023-09-23T03:19:26.454773+0000","last_peered":"2023-09-23T03:19:26.454773+0000","last_clean":"2023-09-23T03:19:26.454773+0000","last_became_active":"2023-09-23T03:19:24.518441+0000","last_became_peered":"2023-09-23T03:19:24.518441+0000","last_unstale":"2023-09-23T03:19:26.454773+0000","last_undegraded":"2023-09-23T03:19:26.454773+0000","last_fullsized":"2023-09-23T03:19:26.454773+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:20:40.999106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.454409+0000","last_change":"2023-09-23T03:19:24.519286+0000","last_active":"2023-09-23T03:19:26.454409+0000","last_peered":"2023-09-23T03:19:26.454409+0000","last_clean":"2023-09-23T03:19:26.454409+0000","last_became_active":"2023-09-23T03:19:24.518994+0000","last_became_peered":"2023-09-23T03:19:24.518994+0000","last_unstale":"2023-09-23T03:19:26.454409+0000","last_undegraded":"2023-09-23T03:19:26.454409+0000","last_fullsized":"2023-09-23T03:19:26.454409+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:22:23.771317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:25.494189+0000","last_change":"2023-09-23T03:19:24.519257+0000","last_active":"2023-09-23T03:19:25.494189+0000","last_peered":"2023-09-23T03:19:25.494189+0000","last_clean":"2023-09-23T03:19:25.494189+0000","last_became_active":"2023-09-23T03:19:24.518171+0000","last_became_peered":"2023-09-23T03:19:24.518171+0000","last_unstale":"2023-09-23T03:19:25.494189+0000","last_undegraded":"2023-09-23T03:19:25.494189+0000","last_fullsized":"2023-09-23T03:19:25.494189+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:42:25.619408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:25.492747+0000","last_change":"2023-09-23T03:19:24.505324+0000","last_active":"2023-09-23T03:19:25.492747+0000","last_peered":"2023-09-23T03:19:25.492747+0000","last_clean":"2023-09-23T03:19:25.492747+0000","last_became_active":"2023-09-23T03:19:24.504619+0000","last_became_peered":"2023-09-23T03:19:24.504619+0000","last_unstale":"2023-09-23T03:19:25.492747+0000","last_undegraded":"2023-09-23T03:19:25.492747+0000","last_fullsized":"2023-09-23T03:19:25.492747+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:00:18.658374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.450938+0000","last_change":"2023-09-23T03:19:24.502093+0000","last_active":"2023-09-23T03:19:26.450938+0000","last_peered":"2023-09-23T03:19:26.450938+0000","last_clean":"2023-09-23T03:19:26.450938+0000","last_became_active":"2023-09-23T03:19:24.501179+0000","last_became_peered":"2023-09-23T03:19:24.501179+0000","last_unstale":"2023-09-23T03:19:26.450938+0000","last_undegraded":"2023-09-23T03:19:26.450938+0000","last_fullsized":"2023-09-23T03:19:26.450938+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:33:17.287766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.450667+0000","last_change":"2023-09-23T03:19:24.510742+0000","last_active":"2023-09-23T03:19:26.450667+0000","last_peered":"2023-09-23T03:19:26.450667+0000","last_clean":"2023-09-23T03:19:26.450667+0000","last_became_active":"2023-09-23T03:19:24.510449+0000","last_became_peered":"2023-09-23T03:19:24.510449+0000","last_unstale":"2023-09-23T03:19:26.450667+0000","last_undegraded":"2023-09-23T03:19:26.450667+0000","last_fullsized":"2023-09-23T03:19:26.450667+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:00:18.658374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.450466+0000","last_change":"2023-09-23T03:19:24.515360+0000","last_active":"2023-09-23T03:19:26.450466+0000","last_peered":"2023-09-23T03:19:26.450466+0000","last_clean":"2023-09-23T03:19:26.450466+0000","last_became_active":"2023-09-23T03:19:24.514998+0000","last_became_peered":"2023-09-23T03:19:24.514998+0000","last_unstale":"2023-09-23T03:19:26.450466+0000","last_undegraded":"2023-09-23T03:19:26.450466+0000","last_fullsized":"2023-09-23T03:19:26.450466+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:06:10.681021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.454005+0000","last_change":"2023-09-23T03:19:24.515125+0000","last_active":"2023-09-23T03:19:26.454005+0000","last_peered":"2023-09-23T03:19:26.454005+0000","last_clean":"2023-09-23T03:19:26.454005+0000","last_became_active":"2023-09-23T03:19:24.514843+0000","last_became_peered":"2023-09-23T03:19:24.514843+0000","last_unstale":"2023-09-23T03:19:26.454005+0000","last_undegraded":"2023-09-23T03:19:26.454005+0000","last_fullsized":"2023-09-23T03:19:26.454005+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:12:03.598908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.302268+0000","last_change":"2023-09-23T03:19:24.523311+0000","last_active":"2023-09-23T03:19:26.302268+0000","last_peered":"2023-09-23T03:19:26.302268+0000","last_clean":"2023-09-23T03:19:26.302268+0000","last_became_active":"2023-09-23T03:19:24.522182+0000","last_became_peered":"2023-09-23T03:19:24.522182+0000","last_unstale":"2023-09-23T03:19:26.302268+0000","last_undegraded":"2023-09-23T03:19:26.302268+0000","last_fullsized":"2023-09-23T03:19:26.302268+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:56:26.198549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:25.490481+0000","last_change":"2023-09-23T03:19:24.498578+0000","last_active":"2023-09-23T03:19:25.490481+0000","last_peered":"2023-09-23T03:19:25.490481+0000","last_clean":"2023-09-23T03:19:25.490481+0000","last_became_active":"2023-09-23T03:19:24.498295+0000","last_became_peered":"2023-09-23T03:19:24.498295+0000","last_unstale":"2023-09-23T03:19:25.490481+0000","last_undegraded":"2023-09-23T03:19:25.490481+0000","last_fullsized":"2023-09-23T03:19:25.490481+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:46:31.972531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.452384+0000","last_change":"2023-09-23T03:19:24.502402+0000","last_active":"2023-09-23T03:19:26.452384+0000","last_peered":"2023-09-23T03:19:26.452384+0000","last_clean":"2023-09-23T03:19:26.452384+0000","last_became_active":"2023-09-23T03:19:24.502127+0000","last_became_peered":"2023-09-23T03:19:24.502127+0000","last_unstale":"2023-09-23T03:19:26.452384+0000","last_undegraded":"2023-09-23T03:19:26.452384+0000","last_fullsized":"2023-09-23T03:19:26.452384+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:08:36.121832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.301859+0000","last_change":"2023-09-23T03:19:24.524839+0000","last_active":"2023-09-23T03:19:26.301859+0000","last_peered":"2023-09-23T03:19:26.301859+0000","last_clean":"2023-09-23T03:19:26.301859+0000","last_became_active":"2023-09-23T03:19:24.524514+0000","last_became_peered":"2023-09-23T03:19:24.524514+0000","last_unstale":"2023-09-23T03:19:26.301859+0000","last_undegraded":"2023-09-23T03:19:26.301859+0000","last_fullsized":"2023-09-23T03:19:26.301859+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:42:25.619408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.451200+0000","last_change":"2023-09-23T03:19:24.506503+0000","last_active":"2023-09-23T03:19:26.451200+0000","last_peered":"2023-09-23T03:19:26.451200+0000","last_clean":"2023-09-23T03:19:26.451200+0000","last_became_active":"2023-09-23T03:19:24.506071+0000","last_became_peered":"2023-09-23T03:19:24.506071+0000","last_unstale":"2023-09-23T03:19:26.451200+0000","last_undegraded":"2023-09-23T03:19:26.451200+0000","last_fullsized":"2023-09-23T03:19:26.451200+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:42:13.319156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.301836+0000","last_change":"2023-09-23T03:19:24.524122+0000","last_active":"2023-09-23T03:19:26.301836+0000","last_peered":"2023-09-23T03:19:26.301836+0000","last_clean":"2023-09-23T03:19:26.301836+0000","last_became_active":"2023-09-23T03:19:24.523438+0000","last_became_peered":"2023-09-23T03:19:24.523438+0000","last_unstale":"2023-09-23T03:19:26.301836+0000","last_undegraded":"2023-09-23T03:19:26.301836+0000","last_fullsized":"2023-09-23T03:19:26.301836+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:06:10.681021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:25.491972+0000","last_change":"2023-09-23T03:19:24.506507+0000","last_active":"2023-09-23T03:19:25.491972+0000","last_peered":"2023-09-23T03:19:25.491972+0000","last_clean":"2023-09-23T03:19:25.491972+0000","last_became_active":"2023-09-23T03:19:24.506088+0000","last_became_peered":"2023-09-23T03:19:24.506088+0000","last_unstale":"2023-09-23T03:19:25.491972+0000","last_undegraded":"2023-09-23T03:19:25.491972+0000","last_fullsized":"2023-09-23T03:19:25.491972+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:49:33.217404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:25.490614+0000","last_change":"2023-09-23T03:19:24.504575+0000","last_active":"2023-09-23T03:19:25.490614+0000","last_peered":"2023-09-23T03:19:25.490614+0000","last_clean":"2023-09-23T03:19:25.490614+0000","last_became_active":"2023-09-23T03:19:24.504280+0000","last_became_peered":"2023-09-23T03:19:24.504280+0000","last_unstale":"2023-09-23T03:19:25.490614+0000","last_undegraded":"2023-09-23T03:19:25.490614+0000","last_fullsized":"2023-09-23T03:19:25.490614+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:25:07.282154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:25.494259+0000","last_change":"2023-09-23T03:19:24.527433+0000","last_active":"2023-09-23T03:19:25.494259+0000","last_peered":"2023-09-23T03:19:25.494259+0000","last_clean":"2023-09-23T03:19:25.494259+0000","last_became_active":"2023-09-23T03:19:24.527097+0000","last_became_peered":"2023-09-23T03:19:24.527097+0000","last_unstale":"2023-09-23T03:19:25.494259+0000","last_undegraded":"2023-09-23T03:19:25.494259+0000","last_fullsized":"2023-09-23T03:19:25.494259+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:30:30.760554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.455786+0000","last_change":"2023-09-23T03:19:24.514579+0000","last_active":"2023-09-23T03:19:26.455786+0000","last_peered":"2023-09-23T03:19:26.455786+0000","last_clean":"2023-09-23T03:19:26.455786+0000","last_became_active":"2023-09-23T03:19:24.514300+0000","last_became_peered":"2023-09-23T03:19:24.514300+0000","last_unstale":"2023-09-23T03:19:26.455786+0000","last_undegraded":"2023-09-23T03:19:26.455786+0000","last_fullsized":"2023-09-23T03:19:26.455786+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:46:22.603972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:25.490471+0000","last_change":"2023-09-23T03:19:24.521270+0000","last_active":"2023-09-23T03:19:25.490471+0000","last_peered":"2023-09-23T03:19:25.490471+0000","last_clean":"2023-09-23T03:19:25.490471+0000","last_became_active":"2023-09-23T03:19:24.521004+0000","last_became_peered":"2023-09-23T03:19:24.521004+0000","last_unstale":"2023-09-23T03:19:25.490471+0000","last_undegraded":"2023-09-23T03:19:25.490471+0000","last_fullsized":"2023-09-23T03:19:25.490471+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:39:50.227232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.452642+0000","last_change":"2023-09-23T03:19:24.523241+0000","last_active":"2023-09-23T03:19:26.452642+0000","last_peered":"2023-09-23T03:19:26.452642+0000","last_clean":"2023-09-23T03:19:26.452642+0000","last_became_active":"2023-09-23T03:19:24.522977+0000","last_became_peered":"2023-09-23T03:19:24.522977+0000","last_unstale":"2023-09-23T03:19:26.452642+0000","last_undegraded":"2023-09-23T03:19:26.452642+0000","last_fullsized":"2023-09-23T03:19:26.452642+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:11:43.341253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.301781+0000","last_change":"2023-09-23T03:19:24.525854+0000","last_active":"2023-09-23T03:19:26.301781+0000","last_peered":"2023-09-23T03:19:26.301781+0000","last_clean":"2023-09-23T03:19:26.301781+0000","last_became_active":"2023-09-23T03:19:24.525566+0000","last_became_peered":"2023-09-23T03:19:24.525566+0000","last_unstale":"2023-09-23T03:19:26.301781+0000","last_undegraded":"2023-09-23T03:19:26.301781+0000","last_fullsized":"2023-09-23T03:19:26.301781+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:45:55.924724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:25.490223+0000","last_change":"2023-09-23T03:19:24.502354+0000","last_active":"2023-09-23T03:19:25.490223+0000","last_peered":"2023-09-23T03:19:25.490223+0000","last_clean":"2023-09-23T03:19:25.490223+0000","last_became_active":"2023-09-23T03:19:24.502098+0000","last_became_peered":"2023-09-23T03:19:24.502098+0000","last_unstale":"2023-09-23T03:19:25.490223+0000","last_undegraded":"2023-09-23T03:19:25.490223+0000","last_fullsized":"2023-09-23T03:19:25.490223+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:30:00.135947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.450855+0000","last_change":"2023-09-23T03:19:24.520044+0000","last_active":"2023-09-23T03:19:26.450855+0000","last_peered":"2023-09-23T03:19:26.450855+0000","last_clean":"2023-09-23T03:19:26.450855+0000","last_became_active":"2023-09-23T03:19:24.519742+0000","last_became_peered":"2023-09-23T03:19:24.519742+0000","last_unstale":"2023-09-23T03:19:26.450855+0000","last_undegraded":"2023-09-23T03:19:26.450855+0000","last_fullsized":"2023-09-23T03:19:26.450855+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:42:25.619408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.453372+0000","last_change":"2023-09-23T03:19:24.515352+0000","last_active":"2023-09-23T03:19:26.453372+0000","last_peered":"2023-09-23T03:19:26.453372+0000","last_clean":"2023-09-23T03:19:26.453372+0000","last_became_active":"2023-09-23T03:19:24.515072+0000","last_became_peered":"2023-09-23T03:19:24.515072+0000","last_unstale":"2023-09-23T03:19:26.453372+0000","last_undegraded":"2023-09-23T03:19:26.453372+0000","last_fullsized":"2023-09-23T03:19:26.453372+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:43:46.233002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"61'4","reported_seq":16,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-09-23T03:19:24.529931+0000","last_change":"2023-09-23T03:19:23.514227+0000","last_active":"2023-09-23T03:19:24.529931+0000","last_peered":"2023-09-23T03:19:24.529931+0000","last_clean":"2023-09-23T03:19:24.529931+0000","last_became_active":"2023-09-23T03:19:23.513867+0000","last_became_peered":"2023-09-23T03:19:23.513867+0000","last_unstale":"2023-09-23T03:19:24.529931+0000","last_undegraded":"2023-09-23T03:19:24.529931+0000","last_fullsized":"2023-09-23T03:19:24.529931+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:20:39.987490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.455248+0000","last_change":"2023-09-23T03:19:24.519389+0000","last_active":"2023-09-23T03:19:26.455248+0000","last_peered":"2023-09-23T03:19:26.455248+0000","last_clean":"2023-09-23T03:19:26.455248+0000","last_became_active":"2023-09-23T03:19:24.519097+0000","last_became_peered":"2023-09-23T03:19:24.519097+0000","last_unstale":"2023-09-23T03:19:26.455248+0000","last_undegraded":"2023-09-23T03:19:26.455248+0000","last_fullsized":"2023-09-23T03:19:26.455248+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:00:40.249625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"61'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-09-23T03:19:24.516751+0000","last_change":"2023-09-23T03:19:23.511815+0000","last_active":"2023-09-23T03:19:24.516751+0000","last_peered":"2023-09-23T03:19:24.516751+0000","last_clean":"2023-09-23T03:19:24.516751+0000","last_became_active":"2023-09-23T03:19:23.510810+0000","last_became_peered":"2023-09-23T03:19:23.510810+0000","last_unstale":"2023-09-23T03:19:24.516751+0000","last_undegraded":"2023-09-23T03:19:24.516751+0000","last_fullsized":"2023-09-23T03:19:24.516751+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:15:40.038449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.455239+0000","last_change":"2023-09-23T03:19:24.510348+0000","last_active":"2023-09-23T03:19:26.455239+0000","last_peered":"2023-09-23T03:19:26.455239+0000","last_clean":"2023-09-23T03:19:26.455239+0000","last_became_active":"2023-09-23T03:19:24.510097+0000","last_became_peered":"2023-09-23T03:19:24.510097+0000","last_unstale":"2023-09-23T03:19:26.455239+0000","last_undegraded":"2023-09-23T03:19:26.455239+0000","last_fullsized":"2023-09-23T03:19:26.455239+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:56:26.198549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-09-23T03:19:24.522789+0000","last_change":"2023-09-23T03:19:23.508844+0000","last_active":"2023-09-23T03:19:24.522789+0000","last_peered":"2023-09-23T03:19:24.522789+0000","last_clean":"2023-09-23T03:19:24.522789+0000","last_became_active":"2023-09-23T03:19:23.508557+0000","last_became_peered":"2023-09-23T03:19:23.508557+0000","last_unstale":"2023-09-23T03:19:24.522789+0000","last_undegraded":"2023-09-23T03:19:24.522789+0000","last_fullsized":"2023-09-23T03:19:24.522789+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:45:54.913108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.455049+0000","last_change":"2023-09-23T03:19:24.525692+0000","last_active":"2023-09-23T03:19:26.455049+0000","last_peered":"2023-09-23T03:19:26.455049+0000","last_clean":"2023-09-23T03:19:26.455049+0000","last_became_active":"2023-09-23T03:19:24.525419+0000","last_became_peered":"2023-09-23T03:19:24.525419+0000","last_unstale":"2023-09-23T03:19:26.455049+0000","last_undegraded":"2023-09-23T03:19:26.455049+0000","last_fullsized":"2023-09-23T03:19:26.455049+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:56:40.870700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-09-23T03:19:24.535629+0000","last_change":"2023-09-23T03:19:23.514514+0000","last_active":"2023-09-23T03:19:24.535629+0000","last_peered":"2023-09-23T03:19:24.535629+0000","last_clean":"2023-09-23T03:19:24.535629+0000","last_became_active":"2023-09-23T03:19:23.514219+0000","last_became_peered":"2023-09-23T03:19:23.514219+0000","last_unstale":"2023-09-23T03:19:24.535629+0000","last_undegraded":"2023-09-23T03:19:24.535629+0000","last_fullsized":"2023-09-23T03:19:24.535629+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:14:57.833336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.451556+0000","last_change":"2023-09-23T03:19:24.506339+0000","last_active":"2023-09-23T03:19:26.451556+0000","last_peered":"2023-09-23T03:19:26.451556+0000","last_clean":"2023-09-23T03:19:26.451556+0000","last_became_active":"2023-09-23T03:19:24.505704+0000","last_became_peered":"2023-09-23T03:19:24.505704+0000","last_unstale":"2023-09-23T03:19:26.451556+0000","last_undegraded":"2023-09-23T03:19:26.451556+0000","last_fullsized":"2023-09-23T03:19:26.451556+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:27:17.796622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"61'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-09-23T03:19:24.526725+0000","last_change":"2023-09-23T03:19:23.509524+0000","last_active":"2023-09-23T03:19:24.526725+0000","last_peered":"2023-09-23T03:19:24.526725+0000","last_clean":"2023-09-23T03:19:24.526725+0000","last_became_active":"2023-09-23T03:19:23.509229+0000","last_became_peered":"2023-09-23T03:19:23.509229+0000","last_unstale":"2023-09-23T03:19:24.526725+0000","last_undegraded":"2023-09-23T03:19:24.526725+0000","last_fullsized":"2023-09-23T03:19:24.526725+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:53:27.547396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.452276+0000","last_change":"2023-09-23T03:19:24.502438+0000","last_active":"2023-09-23T03:19:26.452276+0000","last_peered":"2023-09-23T03:19:26.452276+0000","last_clean":"2023-09-23T03:19:26.452276+0000","last_became_active":"2023-09-23T03:19:24.502083+0000","last_became_peered":"2023-09-23T03:19:24.502083+0000","last_unstale":"2023-09-23T03:19:26.452276+0000","last_undegraded":"2023-09-23T03:19:26.452276+0000","last_fullsized":"2023-09-23T03:19:26.452276+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:48:03.009534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"61'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-09-23T03:19:24.525518+0000","last_change":"2023-09-23T03:19:23.505508+0000","last_active":"2023-09-23T03:19:24.525518+0000","last_peered":"2023-09-23T03:19:24.525518+0000","last_clean":"2023-09-23T03:19:24.525518+0000","last_became_active":"2023-09-23T03:19:23.505249+0000","last_became_peered":"2023-09-23T03:19:23.505249+0000","last_unstale":"2023-09-23T03:19:24.525518+0000","last_undegraded":"2023-09-23T03:19:24.525518+0000","last_fullsized":"2023-09-23T03:19:24.525518+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:42:05.510039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:25.491965+0000","last_change":"2023-09-23T03:19:24.521108+0000","last_active":"2023-09-23T03:19:25.491965+0000","last_peered":"2023-09-23T03:19:25.491965+0000","last_clean":"2023-09-23T03:19:25.491965+0000","last_became_active":"2023-09-23T03:19:24.520554+0000","last_became_peered":"2023-09-23T03:19:24.520554+0000","last_unstale":"2023-09-23T03:19:25.491965+0000","last_undegraded":"2023-09-23T03:19:25.491965+0000","last_fullsized":"2023-09-23T03:19:25.491965+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:45:55.734814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-09-23T03:19:24.512999+0000","last_change":"2023-09-23T03:19:23.514127+0000","last_active":"2023-09-23T03:19:24.512999+0000","last_peered":"2023-09-23T03:19:24.512999+0000","last_clean":"2023-09-23T03:19:24.512999+0000","last_became_active":"2023-09-23T03:19:23.513630+0000","last_became_peered":"2023-09-23T03:19:23.513630+0000","last_unstale":"2023-09-23T03:19:24.512999+0000","last_undegraded":"2023-09-23T03:19:24.512999+0000","last_fullsized":"2023-09-23T03:19:24.512999+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:14:18.364509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.452766+0000","last_change":"2023-09-23T03:19:24.514284+0000","last_active":"2023-09-23T03:19:26.452766+0000","last_peered":"2023-09-23T03:19:26.452766+0000","last_clean":"2023-09-23T03:19:26.452766+0000","last_became_active":"2023-09-23T03:19:24.514029+0000","last_became_peered":"2023-09-23T03:19:24.514029+0000","last_unstale":"2023-09-23T03:19:26.452766+0000","last_undegraded":"2023-09-23T03:19:26.452766+0000","last_fullsized":"2023-09-23T03:19:26.452766+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:49:52.905733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-09-23T03:19:24.500814+0000","last_change":"2023-09-23T03:19:23.500111+0000","last_active":"2023-09-23T03:19:24.500814+0000","last_peered":"2023-09-23T03:19:24.500814+0000","last_clean":"2023-09-23T03:19:24.500814+0000","last_became_active":"2023-09-23T03:19:23.499792+0000","last_became_peered":"2023-09-23T03:19:23.499792+0000","last_unstale":"2023-09-23T03:19:24.500814+0000","last_undegraded":"2023-09-23T03:19:24.500814+0000","last_fullsized":"2023-09-23T03:19:24.500814+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:09:18.859490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.451201+0000","last_change":"2023-09-23T03:19:24.511649+0000","last_active":"2023-09-23T03:19:26.451201+0000","last_peered":"2023-09-23T03:19:26.451201+0000","last_clean":"2023-09-23T03:19:26.451201+0000","last_became_active":"2023-09-23T03:19:24.511261+0000","last_became_peered":"2023-09-23T03:19:24.511261+0000","last_unstale":"2023-09-23T03:19:26.451201+0000","last_undegraded":"2023-09-23T03:19:26.451201+0000","last_fullsized":"2023-09-23T03:19:26.451201+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:00:48.704741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-09-23T03:19:24.510084+0000","last_change":"2023-09-23T03:19:23.514447+0000","last_active":"2023-09-23T03:19:24.510084+0000","last_peered":"2023-09-23T03:19:24.510084+0000","last_clean":"2023-09-23T03:19:24.510084+0000","last_became_active":"2023-09-23T03:19:23.514067+0000","last_became_peered":"2023-09-23T03:19:23.514067+0000","last_unstale":"2023-09-23T03:19:24.510084+0000","last_undegraded":"2023-09-23T03:19:24.510084+0000","last_fullsized":"2023-09-23T03:19:24.510084+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:09:14.016886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.302432+0000","last_change":"2023-09-23T03:19:24.527123+0000","last_active":"2023-09-23T03:19:26.302432+0000","last_peered":"2023-09-23T03:19:26.302432+0000","last_clean":"2023-09-23T03:19:26.302432+0000","last_became_active":"2023-09-23T03:19:24.526829+0000","last_became_peered":"2023-09-23T03:19:24.526829+0000","last_unstale":"2023-09-23T03:19:26.302432+0000","last_undegraded":"2023-09-23T03:19:26.302432+0000","last_fullsized":"2023-09-23T03:19:26.302432+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:46:22.603972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-09-23T03:19:24.494910+0000","last_change":"2023-09-23T03:19:23.502457+0000","last_active":"2023-09-23T03:19:24.494910+0000","last_peered":"2023-09-23T03:19:24.494910+0000","last_clean":"2023-09-23T03:19:24.494910+0000","last_became_active":"2023-09-23T03:19:23.502185+0000","last_became_peered":"2023-09-23T03:19:23.502185+0000","last_unstale":"2023-09-23T03:19:24.494910+0000","last_undegraded":"2023-09-23T03:19:24.494910+0000","last_fullsized":"2023-09-23T03:19:24.494910+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:30:44.952372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.453310+0000","last_change":"2023-09-23T03:19:24.523604+0000","last_active":"2023-09-23T03:19:26.453310+0000","last_peered":"2023-09-23T03:19:26.453310+0000","last_clean":"2023-09-23T03:19:26.453310+0000","last_became_active":"2023-09-23T03:19:24.520216+0000","last_became_peered":"2023-09-23T03:19:24.520216+0000","last_unstale":"2023-09-23T03:19:26.453310+0000","last_undegraded":"2023-09-23T03:19:26.453310+0000","last_fullsized":"2023-09-23T03:19:26.453310+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:11:19.334021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"61'3","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-09-23T03:19:24.535233+0000","last_change":"2023-09-23T03:19:23.514525+0000","last_active":"2023-09-23T03:19:24.535233+0000","last_peered":"2023-09-23T03:19:24.535233+0000","last_clean":"2023-09-23T03:19:24.535233+0000","last_became_active":"2023-09-23T03:19:23.514117+0000","last_became_peered":"2023-09-23T03:19:23.514117+0000","last_unstale":"2023-09-23T03:19:24.535233+0000","last_undegraded":"2023-09-23T03:19:24.535233+0000","last_fullsized":"2023-09-23T03:19:24.535233+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:11:15.908252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.302235+0000","last_change":"2023-09-23T03:19:24.527295+0000","last_active":"2023-09-23T03:19:26.302235+0000","last_peered":"2023-09-23T03:19:26.302235+0000","last_clean":"2023-09-23T03:19:26.302235+0000","last_became_active":"2023-09-23T03:19:24.526242+0000","last_became_peered":"2023-09-23T03:19:24.526242+0000","last_unstale":"2023-09-23T03:19:26.302235+0000","last_undegraded":"2023-09-23T03:19:26.302235+0000","last_fullsized":"2023-09-23T03:19:26.302235+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:56:40.870700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"61'6","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-09-23T03:19:24.534775+0000","last_change":"2023-09-23T03:19:23.501871+0000","last_active":"2023-09-23T03:19:24.534775+0000","last_peered":"2023-09-23T03:19:24.534775+0000","last_clean":"2023-09-23T03:19:24.534775+0000","last_became_active":"2023-09-23T03:19:23.501537+0000","last_became_peered":"2023-09-23T03:19:23.501537+0000","last_unstale":"2023-09-23T03:19:24.534775+0000","last_undegraded":"2023-09-23T03:19:24.534775+0000","last_fullsized":"2023-09-23T03:19:24.534775+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:51:17.271172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:25.492990+0000","last_change":"2023-09-23T03:19:24.517548+0000","last_active":"2023-09-23T03:19:25.492990+0000","last_peered":"2023-09-23T03:19:25.492990+0000","last_clean":"2023-09-23T03:19:25.492990+0000","last_became_active":"2023-09-23T03:19:24.517214+0000","last_became_peered":"2023-09-23T03:19:24.517214+0000","last_unstale":"2023-09-23T03:19:25.492990+0000","last_undegraded":"2023-09-23T03:19:25.492990+0000","last_fullsized":"2023-09-23T03:19:25.492990+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:45:55.924724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-09-23T03:19:24.517572+0000","last_change":"2023-09-23T03:19:23.501583+0000","last_active":"2023-09-23T03:19:24.517572+0000","last_peered":"2023-09-23T03:19:24.517572+0000","last_clean":"2023-09-23T03:19:24.517572+0000","last_became_active":"2023-09-23T03:19:23.501037+0000","last_became_peered":"2023-09-23T03:19:23.501037+0000","last_unstale":"2023-09-23T03:19:24.517572+0000","last_undegraded":"2023-09-23T03:19:24.517572+0000","last_fullsized":"2023-09-23T03:19:24.517572+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:45:54.913108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.302122+0000","last_change":"2023-09-23T03:19:24.513419+0000","last_active":"2023-09-23T03:19:26.302122+0000","last_peered":"2023-09-23T03:19:26.302122+0000","last_clean":"2023-09-23T03:19:26.302122+0000","last_became_active":"2023-09-23T03:19:24.513109+0000","last_became_peered":"2023-09-23T03:19:24.513109+0000","last_unstale":"2023-09-23T03:19:26.302122+0000","last_undegraded":"2023-09-23T03:19:26.302122+0000","last_fullsized":"2023-09-23T03:19:26.302122+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:43:46.233002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-09-23T03:19:24.492914+0000","last_change":"2023-09-23T03:19:23.507592+0000","last_active":"2023-09-23T03:19:24.492914+0000","last_peered":"2023-09-23T03:19:24.492914+0000","last_clean":"2023-09-23T03:19:24.492914+0000","last_became_active":"2023-09-23T03:19:23.507316+0000","last_became_peered":"2023-09-23T03:19:23.507316+0000","last_unstale":"2023-09-23T03:19:24.492914+0000","last_undegraded":"2023-09-23T03:19:24.492914+0000","last_fullsized":"2023-09-23T03:19:24.492914+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:53:27.547396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.450425+0000","last_change":"2023-09-23T03:19:24.522883+0000","last_active":"2023-09-23T03:19:26.450425+0000","last_peered":"2023-09-23T03:19:26.450425+0000","last_clean":"2023-09-23T03:19:26.450425+0000","last_became_active":"2023-09-23T03:19:24.522621+0000","last_became_peered":"2023-09-23T03:19:24.522621+0000","last_unstale":"2023-09-23T03:19:26.450425+0000","last_undegraded":"2023-09-23T03:19:26.450425+0000","last_fullsized":"2023-09-23T03:19:26.450425+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:45:55.734814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-09-23T03:19:24.513161+0000","last_change":"2023-09-23T03:19:23.515237+0000","last_active":"2023-09-23T03:19:24.513161+0000","last_peered":"2023-09-23T03:19:24.513161+0000","last_clean":"2023-09-23T03:19:24.513161+0000","last_became_active":"2023-09-23T03:19:23.514946+0000","last_became_peered":"2023-09-23T03:19:23.514946+0000","last_unstale":"2023-09-23T03:19:24.513161+0000","last_undegraded":"2023-09-23T03:19:24.513161+0000","last_fullsized":"2023-09-23T03:19:24.513161+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:15:26.641134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.301851+0000","last_change":"2023-09-23T03:19:24.524314+0000","last_active":"2023-09-23T03:19:26.301851+0000","last_peered":"2023-09-23T03:19:26.301851+0000","last_clean":"2023-09-23T03:19:26.301851+0000","last_became_active":"2023-09-23T03:19:24.523838+0000","last_became_peered":"2023-09-23T03:19:24.523838+0000","last_unstale":"2023-09-23T03:19:26.301851+0000","last_undegraded":"2023-09-23T03:19:26.301851+0000","last_fullsized":"2023-09-23T03:19:26.301851+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:34:58.795835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"61'8","reported_seq":20,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-09-23T03:19:24.537339+0000","last_change":"2023-09-23T03:19:23.519665+0000","last_active":"2023-09-23T03:19:24.537339+0000","last_peered":"2023-09-23T03:19:24.537339+0000","last_clean":"2023-09-23T03:19:24.537339+0000","last_became_active":"2023-09-23T03:19:23.519393+0000","last_became_peered":"2023-09-23T03:19:23.519393+0000","last_unstale":"2023-09-23T03:19:24.537339+0000","last_undegraded":"2023-09-23T03:19:24.537339+0000","last_fullsized":"2023-09-23T03:19:24.537339+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:31:42.669781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.302430+0000","last_change":"2023-09-23T03:19:24.531683+0000","last_active":"2023-09-23T03:19:26.302430+0000","last_peered":"2023-09-23T03:19:26.302430+0000","last_clean":"2023-09-23T03:19:26.302430+0000","last_became_active":"2023-09-23T03:19:24.531107+0000","last_became_peered":"2023-09-23T03:19:24.531107+0000","last_unstale":"2023-09-23T03:19:26.302430+0000","last_undegraded":"2023-09-23T03:19:26.302430+0000","last_fullsized":"2023-09-23T03:19:26.302430+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:30:45.963989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"61'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-09-23T03:19:24.517770+0000","last_change":"2023-09-23T03:19:23.502992+0000","last_active":"2023-09-23T03:19:24.517770+0000","last_peered":"2023-09-23T03:19:24.517770+0000","last_clean":"2023-09-23T03:19:24.517770+0000","last_became_active":"2023-09-23T03:19:23.502688+0000","last_became_peered":"2023-09-23T03:19:23.502688+0000","last_unstale":"2023-09-23T03:19:24.517770+0000","last_undegraded":"2023-09-23T03:19:24.517770+0000","last_fullsized":"2023-09-23T03:19:24.517770+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:46:21.592356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:25.494601+0000","last_change":"2023-09-23T03:19:24.529454+0000","last_active":"2023-09-23T03:19:25.494601+0000","last_peered":"2023-09-23T03:19:25.494601+0000","last_clean":"2023-09-23T03:19:25.494601+0000","last_became_active":"2023-09-23T03:19:24.529151+0000","last_became_peered":"2023-09-23T03:19:24.529151+0000","last_unstale":"2023-09-23T03:19:25.494601+0000","last_undegraded":"2023-09-23T03:19:25.494601+0000","last_fullsized":"2023-09-23T03:19:25.494601+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:56:40.870700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-09-23T03:19:24.520055+0000","last_change":"2023-09-23T03:19:23.507612+0000","last_active":"2023-09-23T03:19:24.520055+0000","last_peered":"2023-09-23T03:19:24.520055+0000","last_clean":"2023-09-23T03:19:24.520055+0000","last_became_active":"2023-09-23T03:19:23.507211+0000","last_became_peered":"2023-09-23T03:19:23.507211+0000","last_unstale":"2023-09-23T03:19:24.520055+0000","last_undegraded":"2023-09-23T03:19:24.520055+0000","last_fullsized":"2023-09-23T03:19:24.520055+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:14:57.833336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.452687+0000","last_change":"2023-09-23T03:19:24.516541+0000","last_active":"2023-09-23T03:19:26.452687+0000","last_peered":"2023-09-23T03:19:26.452687+0000","last_clean":"2023-09-23T03:19:26.452687+0000","last_became_active":"2023-09-23T03:19:24.516285+0000","last_became_peered":"2023-09-23T03:19:24.516285+0000","last_unstale":"2023-09-23T03:19:26.452687+0000","last_undegraded":"2023-09-23T03:19:26.452687+0000","last_fullsized":"2023-09-23T03:19:26.452687+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:49:20.542937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"61'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-09-23T03:19:24.535621+0000","last_change":"2023-09-23T03:19:23.518829+0000","last_active":"2023-09-23T03:19:24.535621+0000","last_peered":"2023-09-23T03:19:24.535621+0000","last_clean":"2023-09-23T03:19:24.535621+0000","last_became_active":"2023-09-23T03:19:23.518512+0000","last_became_peered":"2023-09-23T03:19:23.518512+0000","last_unstale":"2023-09-23T03:19:24.535621+0000","last_undegraded":"2023-09-23T03:19:24.535621+0000","last_fullsized":"2023-09-23T03:19:24.535621+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:09:14.016886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.302463+0000","last_change":"2023-09-23T03:19:24.527488+0000","last_active":"2023-09-23T03:19:26.302463+0000","last_peered":"2023-09-23T03:19:26.302463+0000","last_clean":"2023-09-23T03:19:26.302463+0000","last_became_active":"2023-09-23T03:19:24.526607+0000","last_became_peered":"2023-09-23T03:19:24.526607+0000","last_unstale":"2023-09-23T03:19:26.302463+0000","last_undegraded":"2023-09-23T03:19:26.302463+0000","last_fullsized":"2023-09-23T03:19:26.302463+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:42:13.319156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-09-23T03:19:24.492237+0000","last_change":"2023-09-23T03:19:23.500147+0000","last_active":"2023-09-23T03:19:24.492237+0000","last_peered":"2023-09-23T03:19:24.492237+0000","last_clean":"2023-09-23T03:19:24.492237+0000","last_became_active":"2023-09-23T03:19:23.499826+0000","last_became_peered":"2023-09-23T03:19:23.499826+0000","last_unstale":"2023-09-23T03:19:24.492237+0000","last_undegraded":"2023-09-23T03:19:24.492237+0000","last_fullsized":"2023-09-23T03:19:24.492237+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:22:22.759701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:25.493324+0000","last_change":"2023-09-23T03:19:24.516940+0000","last_active":"2023-09-23T03:19:25.493324+0000","last_peered":"2023-09-23T03:19:25.493324+0000","last_clean":"2023-09-23T03:19:25.493324+0000","last_became_active":"2023-09-23T03:19:24.516459+0000","last_became_peered":"2023-09-23T03:19:24.516459+0000","last_unstale":"2023-09-23T03:19:25.493324+0000","last_undegraded":"2023-09-23T03:19:25.493324+0000","last_fullsized":"2023-09-23T03:19:25.493324+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:06:10.681021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-09-23T03:19:24.527028+0000","last_change":"2023-09-23T03:19:23.502333+0000","last_active":"2023-09-23T03:19:24.527028+0000","last_peered":"2023-09-23T03:19:24.527028+0000","last_clean":"2023-09-23T03:19:24.527028+0000","last_became_active":"2023-09-23T03:19:23.501997+0000","last_became_peered":"2023-09-23T03:19:23.501997+0000","last_unstale":"2023-09-23T03:19:24.527028+0000","last_undegraded":"2023-09-23T03:19:24.527028+0000","last_fullsized":"2023-09-23T03:19:24.527028+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:34:57.784219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.455426+0000","last_change":"2023-09-23T03:19:24.514598+0000","last_active":"2023-09-23T03:19:26.455426+0000","last_peered":"2023-09-23T03:19:26.455426+0000","last_clean":"2023-09-23T03:19:26.455426+0000","last_became_active":"2023-09-23T03:19:24.514234+0000","last_became_peered":"2023-09-23T03:19:24.514234+0000","last_unstale":"2023-09-23T03:19:26.455426+0000","last_undegraded":"2023-09-23T03:19:26.455426+0000","last_fullsized":"2023-09-23T03:19:26.455426+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:30:45.963989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-09-23T03:19:24.499267+0000","last_change":"2023-09-23T03:19:23.507705+0000","last_active":"2023-09-23T03:19:24.499267+0000","last_peered":"2023-09-23T03:19:24.499267+0000","last_clean":"2023-09-23T03:19:24.499267+0000","last_became_active":"2023-09-23T03:19:23.507409+0000","last_became_peered":"2023-09-23T03:19:23.507409+0000","last_unstale":"2023-09-23T03:19:24.499267+0000","last_undegraded":"2023-09-23T03:19:24.499267+0000","last_fullsized":"2023-09-23T03:19:24.499267+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:49:32.205787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.450764+0000","last_change":"2023-09-23T03:19:24.525480+0000","last_active":"2023-09-23T03:19:26.450764+0000","last_peered":"2023-09-23T03:19:26.450764+0000","last_clean":"2023-09-23T03:19:26.450764+0000","last_became_active":"2023-09-23T03:19:24.525219+0000","last_became_peered":"2023-09-23T03:19:24.525219+0000","last_unstale":"2023-09-23T03:19:26.450764+0000","last_undegraded":"2023-09-23T03:19:26.450764+0000","last_fullsized":"2023-09-23T03:19:26.450764+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:34:58.795835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-09-23T03:19:24.513085+0000","last_change":"2023-09-23T03:19:23.514960+0000","last_active":"2023-09-23T03:19:24.513085+0000","last_peered":"2023-09-23T03:19:24.513085+0000","last_clean":"2023-09-23T03:19:24.513085+0000","last_became_active":"2023-09-23T03:19:23.514644+0000","last_became_peered":"2023-09-23T03:19:23.514644+0000","last_unstale":"2023-09-23T03:19:24.513085+0000","last_undegraded":"2023-09-23T03:19:24.513085+0000","last_fullsized":"2023-09-23T03:19:24.513085+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:31:42.669781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:25.494436+0000","last_change":"2023-09-23T03:19:24.516642+0000","last_active":"2023-09-23T03:19:25.494436+0000","last_peered":"2023-09-23T03:19:25.494436+0000","last_clean":"2023-09-23T03:19:25.494436+0000","last_became_active":"2023-09-23T03:19:24.516303+0000","last_became_peered":"2023-09-23T03:19:24.516303+0000","last_unstale":"2023-09-23T03:19:25.494436+0000","last_undegraded":"2023-09-23T03:19:25.494436+0000","last_fullsized":"2023-09-23T03:19:25.494436+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:43:46.233002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"61'2","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-09-23T03:19:24.525374+0000","last_change":"2023-09-23T03:19:23.512352+0000","last_active":"2023-09-23T03:19:24.525374+0000","last_peered":"2023-09-23T03:19:24.525374+0000","last_clean":"2023-09-23T03:19:24.525374+0000","last_became_active":"2023-09-23T03:19:23.512039+0000","last_became_peered":"2023-09-23T03:19:23.512039+0000","last_unstale":"2023-09-23T03:19:24.525374+0000","last_undegraded":"2023-09-23T03:19:24.525374+0000","last_fullsized":"2023-09-23T03:19:24.525374+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:33:16.276149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:25.494665+0000","last_change":"2023-09-23T03:19:24.517809+0000","last_active":"2023-09-23T03:19:25.494665+0000","last_peered":"2023-09-23T03:19:25.494665+0000","last_clean":"2023-09-23T03:19:25.494665+0000","last_became_active":"2023-09-23T03:19:24.517492+0000","last_became_peered":"2023-09-23T03:19:24.517492+0000","last_unstale":"2023-09-23T03:19:25.494665+0000","last_undegraded":"2023-09-23T03:19:25.494665+0000","last_fullsized":"2023-09-23T03:19:25.494665+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:56:26.198549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"62'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.453973+0000","last_change":"2023-09-23T03:19:23.506342+0000","last_active":"2023-09-23T03:19:26.453973+0000","last_peered":"2023-09-23T03:19:26.453973+0000","last_clean":"2023-09-23T03:19:26.453973+0000","last_became_active":"2023-09-23T03:19:23.506078+0000","last_became_peered":"2023-09-23T03:19:23.506078+0000","last_unstale":"2023-09-23T03:19:26.453973+0000","last_undegraded":"2023-09-23T03:19:26.453973+0000","last_fullsized":"2023-09-23T03:19:26.453973+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:10:46.183841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.453657+0000","last_change":"2023-09-23T03:19:24.520483+0000","last_active":"2023-09-23T03:19:26.453657+0000","last_peered":"2023-09-23T03:19:26.453657+0000","last_clean":"2023-09-23T03:19:26.453657+0000","last_became_active":"2023-09-23T03:19:24.519840+0000","last_became_peered":"2023-09-23T03:19:24.519840+0000","last_unstale":"2023-09-23T03:19:26.453657+0000","last_undegraded":"2023-09-23T03:19:26.453657+0000","last_fullsized":"2023-09-23T03:19:26.453657+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:23:06.878391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-09-23T03:19:24.493300+0000","last_change":"2023-09-23T03:19:23.510497+0000","last_active":"2023-09-23T03:19:24.493300+0000","last_peered":"2023-09-23T03:19:24.493300+0000","last_clean":"2023-09-23T03:19:24.493300+0000","last_became_active":"2023-09-23T03:19:23.510155+0000","last_became_peered":"2023-09-23T03:19:23.510155+0000","last_unstale":"2023-09-23T03:19:24.493300+0000","last_undegraded":"2023-09-23T03:19:24.493300+0000","last_fullsized":"2023-09-23T03:19:24.493300+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:09:18.859490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.455413+0000","last_change":"2023-09-23T03:19:24.522940+0000","last_active":"2023-09-23T03:19:26.455413+0000","last_peered":"2023-09-23T03:19:26.455413+0000","last_clean":"2023-09-23T03:19:26.455413+0000","last_became_active":"2023-09-23T03:19:24.522397+0000","last_became_peered":"2023-09-23T03:19:24.522397+0000","last_unstale":"2023-09-23T03:19:26.455413+0000","last_undegraded":"2023-09-23T03:19:26.455413+0000","last_fullsized":"2023-09-23T03:19:26.455413+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:15:41.050066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-09-23T03:19:24.513771+0000","last_change":"2023-09-23T03:19:23.513693+0000","last_active":"2023-09-23T03:19:24.513771+0000","last_peered":"2023-09-23T03:19:24.513771+0000","last_clean":"2023-09-23T03:19:24.513771+0000","last_became_active":"2023-09-23T03:19:23.513359+0000","last_became_peered":"2023-09-23T03:19:23.513359+0000","last_unstale":"2023-09-23T03:19:24.513771+0000","last_undegraded":"2023-09-23T03:19:24.513771+0000","last_fullsized":"2023-09-23T03:19:24.513771+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:10:46.183841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.451872+0000","last_change":"2023-09-23T03:19:24.515785+0000","last_active":"2023-09-23T03:19:26.451872+0000","last_peered":"2023-09-23T03:19:26.451872+0000","last_clean":"2023-09-23T03:19:26.451872+0000","last_became_active":"2023-09-23T03:19:24.515483+0000","last_became_peered":"2023-09-23T03:19:24.515483+0000","last_unstale":"2023-09-23T03:19:26.451872+0000","last_undegraded":"2023-09-23T03:19:26.451872+0000","last_fullsized":"2023-09-23T03:19:26.451872+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:08:43.035639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-09-23T03:19:24.499215+0000","last_change":"2023-09-23T03:19:23.519087+0000","last_active":"2023-09-23T03:19:24.499215+0000","last_peered":"2023-09-23T03:19:24.499215+0000","last_clean":"2023-09-23T03:19:24.499215+0000","last_became_active":"2023-09-23T03:19:23.518827+0000","last_became_peered":"2023-09-23T03:19:23.518827+0000","last_unstale":"2023-09-23T03:19:24.499215+0000","last_undegraded":"2023-09-23T03:19:24.499215+0000","last_fullsized":"2023-09-23T03:19:24.499215+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:45:54.723198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.302265+0000","last_change":"2023-09-23T03:19:24.522983+0000","last_active":"2023-09-23T03:19:26.302265+0000","last_peered":"2023-09-23T03:19:26.302265+0000","last_clean":"2023-09-23T03:19:26.302265+0000","last_became_active":"2023-09-23T03:19:24.522679+0000","last_became_peered":"2023-09-23T03:19:24.522679+0000","last_unstale":"2023-09-23T03:19:26.302265+0000","last_undegraded":"2023-09-23T03:19:26.302265+0000","last_fullsized":"2023-09-23T03:19:26.302265+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:15:41.050066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-09-23T03:19:24.498720+0000","last_change":"2023-09-23T03:19:23.498946+0000","last_active":"2023-09-23T03:19:24.498720+0000","last_peered":"2023-09-23T03:19:24.498720+0000","last_clean":"2023-09-23T03:19:24.498720+0000","last_became_active":"2023-09-23T03:19:23.498522+0000","last_became_peered":"2023-09-23T03:19:23.498522+0000","last_unstale":"2023-09-23T03:19:24.498720+0000","last_undegraded":"2023-09-23T03:19:24.498720+0000","last_fullsized":"2023-09-23T03:19:24.498720+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:45:17.892103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.451792+0000","last_change":"2023-09-23T03:19:24.513670+0000","last_active":"2023-09-23T03:19:26.451792+0000","last_peered":"2023-09-23T03:19:26.451792+0000","last_clean":"2023-09-23T03:19:26.451792+0000","last_became_active":"2023-09-23T03:19:24.513372+0000","last_became_peered":"2023-09-23T03:19:24.513372+0000","last_unstale":"2023-09-23T03:19:26.451792+0000","last_undegraded":"2023-09-23T03:19:26.451792+0000","last_fullsized":"2023-09-23T03:19:26.451792+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:00:48.704741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-09-23T03:19:24.499542+0000","last_change":"2023-09-23T03:19:23.509484+0000","last_active":"2023-09-23T03:19:24.499542+0000","last_peered":"2023-09-23T03:19:24.499542+0000","last_clean":"2023-09-23T03:19:24.499542+0000","last_became_active":"2023-09-23T03:19:23.509203+0000","last_became_peered":"2023-09-23T03:19:23.509203+0000","last_unstale":"2023-09-23T03:19:24.499542+0000","last_undegraded":"2023-09-23T03:19:24.499542+0000","last_fullsized":"2023-09-23T03:19:24.499542+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:00:17.646758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.451768+0000","last_change":"2023-09-23T03:19:24.513656+0000","last_active":"2023-09-23T03:19:26.451768+0000","last_peered":"2023-09-23T03:19:26.451768+0000","last_clean":"2023-09-23T03:19:26.451768+0000","last_became_active":"2023-09-23T03:19:24.513358+0000","last_became_peered":"2023-09-23T03:19:24.513358+0000","last_unstale":"2023-09-23T03:19:26.451768+0000","last_undegraded":"2023-09-23T03:19:26.451768+0000","last_fullsized":"2023-09-23T03:19:26.451768+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:04:52.709124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-09-23T03:19:24.493912+0000","last_change":"2023-09-23T03:19:23.513953+0000","last_active":"2023-09-23T03:19:24.493912+0000","last_peered":"2023-09-23T03:19:24.493912+0000","last_clean":"2023-09-23T03:19:24.493912+0000","last_became_active":"2023-09-23T03:19:23.513616+0000","last_became_peered":"2023-09-23T03:19:23.513616+0000","last_unstale":"2023-09-23T03:19:24.493912+0000","last_undegraded":"2023-09-23T03:19:24.493912+0000","last_fullsized":"2023-09-23T03:19:24.493912+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:42:24.607791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-09-23T03:19:26.451783+0000","last_change":"2023-09-23T03:19:24.502792+0000","last_active":"2023-09-23T03:19:26.451783+0000","last_peered":"2023-09-23T03:19:26.451783+0000","last_clean":"2023-09-23T03:19:26.451783+0000","last_became_active":"2023-09-23T03:19:24.502548+0000","last_became_peered":"2023-09-23T03:19:24.502548+0000","last_unstale":"2023-09-23T03:19:26.451783+0000","last_undegraded":"2023-09-23T03:19:26.451783+0000","last_fullsized":"2023-09-23T03:19:26.451783+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:23.465704+0000","last_clean_scrub_stamp":"2023-09-23T03:19:23.465704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:56:07.141238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-09-23T03:19:24.492480+0000","last_change":"2023-09-23T03:19:23.519480+0000","last_active":"2023-09-23T03:19:24.492480+0000","last_peered":"2023-09-23T03:19:24.492480+0000","last_clean":"2023-09-23T03:19:24.492480+0000","last_became_active":"2023-09-23T03:19:23.519191+0000","last_became_peered":"2023-09-23T03:19:23.519191+0000","last_unstale":"2023-09-23T03:19:24.492480+0000","last_undegraded":"2023-09-23T03:19:24.492480+0000","last_fullsized":"2023-09-23T03:19:24.492480+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:19:22.454088+0000","last_clean_scrub_stamp":"2023-09-23T03:19:22.454088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:51:17.271172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":200,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-09-23T03:19:24.511904+0000","last_change":"2023-09-23T03:15:11.042447+0000","last_active":"2023-09-23T03:19:24.511904+0000","last_peered":"2023-09-23T03:19:24.511904+0000","last_clean":"2023-09-23T03:19:24.511904+0000","last_became_active":"2023-09-23T03:15:11.040322+0000","last_became_peered":"2023-09-23T03:15:11.040322+0000","last_unstale":"2023-09-23T03:19:24.511904+0000","last_undegraded":"2023-09-23T03:19:24.511904+0000","last_fullsized":"2023-09-23T03:19:24.511904+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_clean_scrub_stamp":"2023-09-23T03:15:09.474842+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:18:36.331645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574902,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1347584,"data_stored":937729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.98799999999999999,"15min":0.99199999999999999},"min":{"1min":0.68799999999999994,"5min":0.442,"15min":0.442},"max":{"1min":1.55,"5min":2.149,"15min":2.149},"last":0.998},{"interface":"front","average":{"1min":1.111,"5min":1.0569999999999999,"15min":1.0469999999999999},"min":{"1min":0.82099999999999995,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.681,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.82799999999999996}]},{"osd":1,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.198,"15min":1.22},"min":{"1min":0.74299999999999999,"5min":0.504,"15min":0.504},"max":{"1min":1.5649999999999999,"5min":3.1859999999999999,"15min":3.1859999999999999},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.248,"5min":1.294,"15min":1.3759999999999999},"min":{"1min":0.93899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.5840000000000001,"5min":2.613,"15min":2.613},"last":1.381}]},{"osd":2,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.1240000000000001,"15min":1.155},"min":{"1min":0.70199999999999996,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.6399999999999999,"5min":2.331,"15min":2.331},"last":1.591},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.242,"15min":1.296},"min":{"1min":0.75600000000000001,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.673,"5min":3.093,"15min":3.093},"last":1.532}]},{"osd":3,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.248,"15min":1.2989999999999999},"min":{"1min":0.90000000000000002,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.7310000000000001,"5min":2.532,"15min":2.532},"last":1.149},{"interface":"front","average":{"1min":1.389,"5min":1.377,"15min":1.4410000000000001},"min":{"1min":0.95099999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.784,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":1.3240000000000001}]},{"osd":4,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.415,"15min":1.528},"min":{"1min":0.79200000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.7070000000000001,"5min":2.7210000000000001,"15min":2.7210000000000001},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.4730000000000001,"15min":1.556},"min":{"1min":0.81699999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.5880000000000001,"5min":3.7719999999999998,"15min":3.7719999999999998},"last":1.472}]},{"osd":5,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.4179999999999999,"15min":1.4810000000000001},"min":{"1min":0.72699999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.736,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":1.6859999999999999},{"interface":"front","average":{"1min":1.288,"5min":1.4510000000000001,"15min":1.548},"min":{"1min":1.0069999999999999,"5min":0.497,"15min":0.497},"max":{"1min":1.7689999999999999,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":1.851}]},{"osd":6,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.5669999999999999,"15min":1.649},"min":{"1min":1.2410000000000001,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.8580000000000001,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":1.74},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.5089999999999999,"15min":1.607},"min":{"1min":1.002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.778,"5min":3.6739999999999999,"15min":3.6739999999999999},"last":1.284}]}]},{"osd":6,"up_from":13,"seq":55834574902,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31908,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":745472,"data_stored":347201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.048,"15min":1.0289999999999999},"min":{"1min":0.68899999999999995,"5min":0.38,"15min":0.38},"max":{"1min":1.6379999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.083,"5min":1.0469999999999999,"15min":1.0549999999999999},"min":{"1min":0.61599999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.516,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":1.0209999999999999}]},{"osd":1,"last update":"Sat Sep 23 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.109,"15min":1.0980000000000001},"min":{"1min":0.65900000000000003,"5min":0.438,"15min":0.438},"max":{"1min":1.6830000000000001,"5min":1.867,"15min":1.867},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.23,"5min":1.2170000000000001,"15min":1.242},"min":{"1min":0.82399999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.6339999999999999,"5min":2.242,"15min":2.242},"last":1.085}]},{"osd":2,"last update":"Sat Sep 23 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.1180000000000001,"15min":1.1060000000000001},"min":{"1min":0.82499999999999996,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4470000000000001,"5min":2.097,"15min":2.097},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.194,"5min":1.234,"15min":1.2669999999999999},"min":{"1min":0.66800000000000004,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.8149999999999999,"5min":1.903,"15min":1.903},"last":1.1579999999999999}]},{"osd":3,"last update":"Sat Sep 23 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.3380000000000001,"15min":1.361},"min":{"1min":0.80200000000000005,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.8879999999999999,"5min":2.6509999999999998,"15min":2.6509999999999998},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.389,"15min":1.4550000000000001},"min":{"1min":0.76600000000000001,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.657,"5min":2.3740000000000001,"15min":2.3740000000000001},"last":1.304}]},{"osd":4,"last update":"Sat Sep 23 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.397,"15min":1.395},"min":{"1min":0.81399999999999995,"5min":0.498,"15min":0.498},"max":{"1min":1.827,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.2849999999999999,"15min":1.294},"min":{"1min":0.75700000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.661,"5min":2.2109999999999999,"15min":2.2109999999999999},"last":1.4199999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.391,"15min":1.395},"min":{"1min":0.84699999999999998,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.7390000000000001,"5min":2.2050000000000001,"15min":2.2050000000000001},"last":1.607},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.4810000000000001,"15min":1.486},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.9390000000000001,"5min":4.758,"15min":4.758},"last":1.363}]},{"osd":7,"last update":"Sat Sep 23 03:19:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.508,"15min":1.5549999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.77,"5min":2.476,"15min":2.476},"last":1.722},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.532,"15min":1.5640000000000001},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.1739999999999999,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.536}]}]},{"osd":5,"up_from":13,"seq":55834574904,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27756,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344084,"statfs":{"total":96636764160,"available":96608342016,"internally_reserved":0,"allocated":753664,"data_stored":347853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.044,"15min":1.03},"min":{"1min":0.80200000000000005,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.474,"5min":1.637,"15min":1.637},"last":1.075},{"interface":"front","average":{"1min":1.141,"5min":1.0609999999999999,"15min":1.032},"min":{"1min":0.78800000000000003,"5min":0.63,"15min":0.63},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":1.621}]},{"osd":1,"last update":"Sat Sep 23 03:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.212,"15min":1.224},"min":{"1min":0.95599999999999996,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.6459999999999999,"5min":1.849,"15min":1.849},"last":1.135},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.2869999999999999,"15min":1.3220000000000001},"min":{"1min":1.085,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.7410000000000001,"5min":1.893,"15min":1.893},"last":1.3759999999999999}]},{"osd":2,"last update":"Sat Sep 23 03:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.22,"15min":1.2170000000000001},"min":{"1min":1.0189999999999999,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.597,"5min":1.681,"15min":1.681},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.161,"5min":1.244,"15min":1.2969999999999999},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.5740000000000001,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":1.2090000000000001}]},{"osd":3,"last update":"Sat Sep 23 03:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.2509999999999999,"15min":1.361},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.423,"5min":1.905,"15min":1.905},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.3859999999999999,"15min":1.47},"min":{"1min":1.056,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.792,"5min":3.6110000000000002,"15min":3.6110000000000002},"last":1.4419999999999999}]},{"osd":4,"last update":"Sat Sep 23 03:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3939999999999999,"15min":1.444},"min":{"1min":0.83099999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.8500000000000001,"5min":2.069,"15min":2.069},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.401,"15min":1.4530000000000001},"min":{"1min":0.91100000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.7889999999999999,"5min":2.319,"15min":2.319},"last":1.3180000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":1.5089999999999999,"15min":1.5409999999999999},"min":{"1min":0.69899999999999995,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.948,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":1.6739999999999999},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.5309999999999999,"15min":1.613},"min":{"1min":0.97099999999999997,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.877,"5min":2.5310000000000001,"15min":2.5310000000000001},"last":1.5760000000000001}]},{"osd":7,"last update":"Sat Sep 23 03:19:11 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.4570000000000001,"15min":1.538},"min":{"1min":0.91600000000000004,"5min":0.626,"15min":0.626},"max":{"1min":1.6839999999999999,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.734},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.585,"15min":1.643},"min":{"1min":1.046,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.7330000000000001,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.417}]}]},{"osd":4,"up_from":13,"seq":55834574903,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":757760,"data_stored":348813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:19:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.1279999999999999,"15min":1.1379999999999999},"min":{"1min":0.64800000000000002,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.639,"5min":1.806,"15min":1.806},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.1739999999999999,"15min":1.141},"min":{"1min":0.76500000000000001,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.905,"5min":1.9179999999999999,"15min":1.9179999999999999},"last":1.319}]},{"osd":1,"last update":"Sat Sep 23 03:19:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.2130000000000001,"15min":1.2150000000000001},"min":{"1min":0.90200000000000002,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.1910000000000001,"15min":1.2210000000000001},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.4710000000000001,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":1.25}]},{"osd":2,"last update":"Sat Sep 23 03:19:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.1870000000000001,"15min":1.246},"min":{"1min":0.71199999999999997,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.129,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.246,"15min":1.224},"min":{"1min":0.91200000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":1.909,"5min":1.909,"15min":1.909},"last":1.1919999999999999}]},{"osd":3,"last update":"Sat Sep 23 03:19:15 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.1479999999999999,"15min":1.171},"min":{"1min":0.71999999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.5369999999999999,"5min":1.583,"15min":1.583},"last":1.127},{"interface":"front","average":{"1min":1.296,"5min":1.319,"15min":1.341},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":2.1779999999999999,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.397}]},{"osd":5,"last update":"Sat Sep 23 03:19:15 2023","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.371,"15min":1.345},"min":{"1min":0.99299999999999999,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":2.8050000000000002,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.645},{"interface":"front","average":{"1min":1.371,"5min":1.3300000000000001,"15min":1.3260000000000001},"min":{"1min":0.84299999999999997,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":2.887,"5min":2.887,"15min":2.887},"last":1.7350000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:19:15 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.397,"15min":1.391},"min":{"1min":1.048,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.4249999999999998,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":1.351},{"interface":"front","average":{"1min":1.6619999999999999,"5min":1.5049999999999999,"15min":1.478},"min":{"1min":1.323,"5min":0.89900000000000002,"15min":0.89900000000000002},"max":{"1min":2.052,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.4319999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:19:15 2023","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.4650000000000001,"15min":1.4730000000000001},"min":{"1min":1.1819999999999999,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":2.5680000000000001,"5min":2.5680000000000001,"15min":2.5680000000000001},"last":1.365},{"interface":"front","average":{"1min":1.6379999999999999,"5min":1.5580000000000001,"15min":1.5649999999999999},"min":{"1min":1.056,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":2.4769999999999999,"5min":2.4769999999999999,"15min":2.4769999999999999},"last":1.708}]}]},{"osd":3,"up_from":13,"seq":55834574903,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31916,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339924,"statfs":{"total":96636764160,"available":96604082176,"internally_reserved":0,"allocated":753664,"data_stored":347257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.85399999999999998,"15min":0.875},"min":{"1min":0.42199999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.7090000000000001,"5min":1.744,"15min":1.744},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.86599999999999999,"15min":0.90200000000000002},"min":{"1min":0.51600000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.341,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.1319999999999999}]},{"osd":1,"last update":"Sat Sep 23 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.90700000000000003,"15min":0.96399999999999997},"min":{"1min":0.442,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.5640000000000001,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.02,"5min":0.98099999999999998,"15min":0.98499999999999999},"min":{"1min":0.54700000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.3839999999999999,"5min":1.704,"15min":1.704},"last":0.90600000000000003}]},{"osd":2,"last update":"Sat Sep 23 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.0609999999999999,"15min":1.113},"min":{"1min":0.54800000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.639,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.165,"5min":1.097,"15min":1.129},"min":{"1min":0.64400000000000002,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.6830000000000001,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.0089999999999999}]},{"osd":4,"last update":"Sat Sep 23 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.2809999999999999,"15min":1.333},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":2.0099999999999998,"5min":2.698,"15min":2.698},"last":1.232},{"interface":"front","average":{"1min":1.353,"5min":1.381,"15min":1.4470000000000001},"min":{"1min":0.95399999999999996,"5min":0.752,"15min":0.752},"max":{"1min":1.825,"5min":2.419,"15min":2.419},"last":1.645}]},{"osd":5,"last update":"Sat Sep 23 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.339,"15min":1.361},"min":{"1min":0.90300000000000002,"5min":0.753,"15min":0.753},"max":{"1min":1.8799999999999999,"5min":1.917,"15min":1.917},"last":1.369},{"interface":"front","average":{"1min":1.321,"5min":1.3049999999999999,"15min":1.3620000000000001},"min":{"1min":0.83499999999999996,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.716,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.5600000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.359,"15min":1.423},"min":{"1min":0.82699999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.9550000000000001,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":1.716},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4430000000000001,"15min":1.482},"min":{"1min":1.05,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.476}]},{"osd":7,"last update":"Sat Sep 23 03:19:22 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.417,"15min":1.5009999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":2.2120000000000002,"5min":2.7879999999999998,"15min":2.7879999999999998},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.4399999999999999,"15min":1.526},"min":{"1min":0.85799999999999998,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.2029999999999998,"5min":2.5379999999999998,"15min":2.5379999999999998},"last":1.784}]}]},{"osd":2,"up_from":13,"seq":55834574904,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27756,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344084,"statfs":{"total":96636764160,"available":96608342016,"internally_reserved":0,"allocated":753664,"data_stored":347257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.83799999999999997,"15min":0.83699999999999997},"min":{"1min":0.442,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.643,"5min":1.643,"15min":1.643},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.998,"5min":0.86199999999999999,"15min":0.84299999999999997},"min":{"1min":0.48899999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.5329999999999999,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.77600000000000002}]},{"osd":1,"last update":"Sat Sep 23 03:19:12 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.84799999999999998,"15min":0.85999999999999999},"min":{"1min":0.55800000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":1.069},{"interface":"front","average":{"1min":1.026,"5min":0.93300000000000005,"15min":0.94099999999999995},"min":{"1min":0.55100000000000005,"5min":0.38,"15min":0.38},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":1.0269999999999999}]},{"osd":3,"last update":"Sat Sep 23 03:19:12 2023","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":0.96599999999999997,"15min":0.95199999999999996},"min":{"1min":0.53900000000000003,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.4279999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":1.006},{"interface":"front","average":{"1min":1.073,"5min":0.98499999999999999,"15min":0.997},"min":{"1min":0.60699999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.534,"5min":1.78,"15min":1.78},"last":1.099}]},{"osd":4,"last update":"Sat Sep 23 03:19:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2729999999999999,"15min":1.3240000000000001},"min":{"1min":0.86199999999999999,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.819,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.296,"5min":1.2969999999999999,"15min":1.341},"min":{"1min":0.77600000000000002,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.631,"5min":2.234,"15min":2.234},"last":1.4199999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:19:12 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.339,"15min":1.409},"min":{"1min":0.97499999999999998,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.7849999999999999,"5min":3.9409999999999998,"15min":3.9409999999999998},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.4199999999999999,"15min":1.5329999999999999},"min":{"1min":0.81299999999999994,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.9359999999999999,"5min":4.9199999999999999,"15min":4.9199999999999999},"last":1.5509999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:19:12 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.3580000000000001,"15min":1.417},"min":{"1min":0.83299999999999996,"5min":0.755,"15min":0.755},"max":{"1min":1.8540000000000001,"5min":3.0009999999999999,"15min":3.0009999999999999},"last":1.256},{"interface":"front","average":{"1min":1.458,"5min":1.393,"15min":1.478},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.1680000000000001,"5min":4.5800000000000001,"15min":4.5800000000000001},"last":1.4990000000000001}]},{"osd":7,"last update":"Sat Sep 23 03:19:12 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.3600000000000001,"15min":1.4019999999999999},"min":{"1min":1.0469999999999999,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.996,"5min":2.3149999999999999,"15min":2.3149999999999999},"last":1.292},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.413,"15min":1.4550000000000001},"min":{"1min":0.873,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.9259999999999999,"5min":2.621,"15min":2.621},"last":1.341}]}]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27756,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344084,"statfs":{"total":96636764160,"available":96608342016,"internally_reserved":0,"allocated":749568,"data_stored":347291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.80000000000000004,"15min":0.80800000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.2490000000000001,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":0.878},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.85099999999999998},"min":{"1min":0.34799999999999998,"5min":0.312,"15min":0.312},"max":{"1min":1.2210000000000001,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":0.77000000000000002}]},{"osd":2,"last update":"Sat Sep 23 03:19:18 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.84999999999999998,"15min":0.89400000000000002},"min":{"1min":0.52000000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.409,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.004,"15min":1.0369999999999999},"min":{"1min":0.54500000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5600000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.127}]},{"osd":3,"last update":"Sat Sep 23 03:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.028,"5min":1.0269999999999999,"15min":1.087},"min":{"1min":0.48799999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.5389999999999999,"5min":2.2370000000000001,"15min":2.2370000000000001},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.0980000000000001,"15min":1.1619999999999999},"min":{"1min":0.56200000000000006,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.7649999999999999,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":0.82299999999999995}]},{"osd":4,"last update":"Sat Sep 23 03:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.2330000000000001,"15min":1.2829999999999999},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.7270000000000001,"5min":3.2629999999999999,"15min":3.2629999999999999},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.262,"15min":1.3},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6359999999999999,"5min":2.4689999999999999,"15min":2.4689999999999999},"last":1.0309999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.254,"15min":1.2989999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.9319999999999999,"5min":3.161,"15min":3.161},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3,"15min":1.3460000000000001},"min":{"1min":0.80400000000000005,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.698,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":0.97299999999999998}]},{"osd":6,"last update":"Sat Sep 23 03:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.2969999999999999,"15min":1.3400000000000001},"min":{"1min":0.76500000000000001,"5min":0.747,"15min":0.747},"max":{"1min":2.0329999999999999,"5min":3.0179999999999998,"15min":3.0179999999999998},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.333,"5min":1.3520000000000001,"15min":1.3979999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.7909999999999999,"5min":3.3580000000000001,"15min":3.3580000000000001},"last":1.3480000000000001}]},{"osd":7,"last update":"Sat Sep 23 03:19:18 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.363,"15min":1.413},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":2.085,"5min":3.1099999999999999,"15min":3.1099999999999999},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.415,"15min":1.4810000000000001},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.7330000000000001,"5min":3.306,"15min":3.306},"last":1.639}]}]},{"osd":0,"up_from":13,"seq":55834574905,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1351680,"data_stored":938744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.81200000000000006,"15min":0.81999999999999995},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.3180000000000001,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.137},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.86599999999999999,"15min":0.89300000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.363,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":1.1839999999999999}]},{"osd":2,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.81100000000000005,"15min":0.80300000000000005},"min":{"1min":0.48599999999999999,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.22,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.97299999999999998,"15min":0.98499999999999999},"min":{"1min":0.46700000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":1.071}]},{"osd":3,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93300000000000005,"15min":0.91500000000000004},"min":{"1min":0.44800000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.226,"5min":1.591,"15min":1.591},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.117,"5min":0.996,"15min":1.006},"min":{"1min":0.57299999999999995,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.4990000000000001,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.0069999999999999}]},{"osd":4,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.204,"15min":1.204},"min":{"1min":0.81499999999999995,"5min":0.755,"15min":0.755},"max":{"1min":1.601,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.667},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.2410000000000001,"15min":1.274},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.4610000000000001,"5min":2.1219999999999999,"15min":2.1219999999999999},"last":1.377}]},{"osd":5,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.2250000000000001,"15min":1.2290000000000001},"min":{"1min":0.872,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.6739999999999999,"5min":1.867,"15min":1.867},"last":1.6080000000000001},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.294,"15min":1.272},"min":{"1min":0.76400000000000001,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":1.5249999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.3009999999999999,"15min":1.3029999999999999},"min":{"1min":0.99399999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":1.431},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.325,"15min":1.3029999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.875,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":1.73}]},{"osd":7,"last update":"Sat Sep 23 03:19:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.329,"15min":1.3160000000000001},"min":{"1min":0.93300000000000005,"5min":0.747,"15min":0.747},"max":{"1min":1.6499999999999999,"5min":1.944,"15min":1.944},"last":1.5669999999999999},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.3979999999999999,"15min":1.4059999999999999},"min":{"1min":0.91800000000000004,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.756,"5min":2.048,"15min":2.048},"last":1.4730000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-23T03:19:34.472 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-09-23T03:19:34.472 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-09-23T03:19:34.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.737+0000 7fa30aaea640 1 Processor -- start 2023-09-23T03:19:34.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.737+0000 7fa30aaea640 1 -- start start 2023-09-23T03:19:34.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.739+0000 7fa30aaea640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3040b16c0 0x7fa30406c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:34.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.739+0000 7fa30aaea640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa30406c6a0 con 0x7fa3040d0a10 2023-09-23T03:19:34.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.739+0000 7fa30aaea640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa30406bc90 con 0x7fa3040d0df0 2023-09-23T03:19:34.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.739+0000 7fa30aaea640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3040d1930 con 0x7fa3040b16c0 2023-09-23T03:19:34.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.739+0000 7fa309ae8640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fa3040d0a10 0x7fa30406ae70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:59018/0 (socket says 172.21.15.154:59018) 2023-09-23T03:19:34.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.739+0000 7fa309ae8640 1 -- 172.21.15.154:0/1784096462 learned_addr learned my addr 172.21.15.154:0/1784096462 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:19:34.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.740+0000 7fa3092e7640 1 --2- 172.21.15.154:0/1784096462 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3040b16c0 0x7fa30406c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:34.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.740+0000 7fa308ae6640 1 -- 172.21.15.154:0/1784096462 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2550474297 0 0) 0x7fa30406c6a0 con 0x7fa3040d0a10 2023-09-23T03:19:34.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.740+0000 7fa308ae6640 1 -- 172.21.15.154:0/1784096462 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2ec003520 con 0x7fa3040d0a10 2023-09-23T03:19:34.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.740+0000 7fa308ae6640 1 -- 172.21.15.154:0/1784096462 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2208139568 0 0) 0x7fa2ec003520 con 0x7fa3040d0a10 2023-09-23T03:19:34.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.740+0000 7fa308ae6640 1 -- 172.21.15.154:0/1784096462 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3040b16c0 msgr2=0x7fa30406c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:34.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.740+0000 7fa308ae6640 1 --2- 172.21.15.154:0/1784096462 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3040b16c0 0x7fa30406c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:34.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.740+0000 7fa308ae6640 1 -- 172.21.15.154:0/1784096462 >> v1:172.21.15.190:6789/0 conn(0x7fa3040d0df0 legacy=0x7fa30406b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:34.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.740+0000 7fa308ae6640 1 -- 172.21.15.154:0/1784096462 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3040d38d0 con 0x7fa3040d0a10 2023-09-23T03:19:34.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.740+0000 7fa308ae6640 1 -- 172.21.15.154:0/1784096462 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fa2f4001b70 con 0x7fa3040d0a10 2023-09-23T03:19:34.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.740+0000 7fa3092e7640 1 --2- 172.21.15.154:0/1784096462 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3040b16c0 0x7fa30406c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:19:34.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.740+0000 7fa308ae6640 1 -- 172.21.15.154:0/1784096462 >> v1:172.21.15.154:6789/0 conn(0x7fa3040d0a10 legacy=0x7fa30406ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:34.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.740+0000 7fa308ae6640 1 --2- 172.21.15.154:0/1784096462 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3040b16c0 0x7fa2ec0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:34.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.741+0000 7fa308ae6640 1 --2- 172.21.15.154:0/1784096462 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2ec003f90 0x7fa2ec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:34.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.741+0000 7fa308ae6640 1 --2- 172.21.15.154:0/1784096462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2ec004d00 0x7fa2ec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:34.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.741+0000 7fa308ae6640 1 -- 172.21.15.154:0/1784096462 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3040d1930 con 0x7fa2ec004d00 2023-09-23T03:19:34.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.741+0000 7fa308ae6640 1 -- 172.21.15.154:0/1784096462 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec00bb70 con 0x7fa2ec003f90 2023-09-23T03:19:34.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.741+0000 7fa308ae6640 1 -- 172.21.15.154:0/1784096462 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec00bd30 con 0x7fa3040b16c0 2023-09-23T03:19:34.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.741+0000 7fa30a2e9640 1 --2- 172.21.15.154:0/1784096462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2ec004d00 0x7fa2ec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:34.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.741+0000 7fa309ae8640 1 --2- 172.21.15.154:0/1784096462 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3040b16c0 0x7fa2ec0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:34.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.741+0000 7fa3092e7640 1 --2- 172.21.15.154:0/1784096462 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2ec003f90 0x7fa2ec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:34.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.741+0000 7fa30a2e9640 1 -- 172.21.15.154:0/1784096462 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3040b16c0 msgr2=0x7fa2ec0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:34.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.741+0000 7fa30a2e9640 1 --2- 172.21.15.154:0/1784096462 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3040b16c0 0x7fa2ec0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:34.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.741+0000 7fa30a2e9640 1 -- 172.21.15.154:0/1784096462 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2ec003f90 msgr2=0x7fa2ec0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:34.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.741+0000 7fa30a2e9640 1 --2- 172.21.15.154:0/1784096462 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2ec003f90 0x7fa2ec0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:34.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.741+0000 7fa30a2e9640 1 -- 172.21.15.154:0/1784096462 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ec003520 con 0x7fa2ec004d00 2023-09-23T03:19:34.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.742+0000 7fa30a2e9640 1 --2- 172.21.15.154:0/1784096462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2ec004d00 0x7fa2ec005180 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc001750 tx=0x7fa2ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b5b4a84bfa3438 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:34.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.743+0000 7fa308ae6640 1 -- 172.21.15.154:0/1784096462 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc0052d0 con 0x7fa2ec004d00 2023-09-23T03:19:34.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.743+0000 7fa308ae6640 1 -- 172.21.15.154:0/1784096462 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2fc02e050 con 0x7fa2ec004d00 2023-09-23T03:19:34.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.743+0000 7fa308ae6640 1 -- 172.21.15.154:0/1784096462 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc003830 con 0x7fa2ec004d00 2023-09-23T03:19:34.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.743+0000 7fa30aaea640 1 -- 172.21.15.154:0/1784096462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2ec004d00 msgr2=0x7fa2ec005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:34.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.743+0000 7fa30aaea640 1 --2- 172.21.15.154:0/1784096462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2ec004d00 0x7fa2ec005180 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc001750 tx=0x7fa2ec0057f0 comp rx=0 tx=0).stop 2023-09-23T03:19:34.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.743+0000 7fa30a2e9640 1 -- 172.21.15.154:0/1784096462 reap_dead start 2023-09-23T03:19:34.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.743+0000 7fa30aaea640 1 -- 172.21.15.154:0/1784096462 shutdown_connections 2023-09-23T03:19:34.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.743+0000 7fa30aaea640 1 -- 172.21.15.154:0/1784096462 >> 172.21.15.154:0/1784096462 conn(0x7fa30409bc20 msgr2=0x7fa3040d29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:34.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.743+0000 7fa30aaea640 1 -- 172.21.15.154:0/1784096462 shutdown_connections 2023-09-23T03:19:34.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.743+0000 7fa30aaea640 1 -- 172.21.15.154:0/1784096462 wait complete. 2023-09-23T03:19:34.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.745+0000 7fa30aaea640 1 Processor -- start 2023-09-23T03:19:34.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.745+0000 7fa30aaea640 1 -- start start 2023-09-23T03:19:34.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.746+0000 7fa30aaea640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2ec004d00 0x7fa3040b4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:34.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.746+0000 7fa30aaea640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa30414cc40 0x7fa3040c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:34.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.746+0000 7fa30aaea640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3040c9510 0x7fa3040c64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:34.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.746+0000 7fa30aaea640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3040b4210 con 0x7fa3040c9510 2023-09-23T03:19:34.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.746+0000 7fa30aaea640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3040b4b70 con 0x7fa30414cc40 2023-09-23T03:19:34.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.746+0000 7fa30aaea640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3040b3590 con 0x7fa2ec004d00 2023-09-23T03:19:34.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.747+0000 7fa30a2e9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3040c9510 0x7fa3040c64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:34.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.747+0000 7fa30a2e9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3040c9510 0x7fa3040c64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57494/0 (socket says 172.21.15.154:57494) 2023-09-23T03:19:34.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.747+0000 7fa30a2e9640 1 -- 172.21.15.154:0/517379360 learned_addr learned my addr 172.21.15.154:0/517379360 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:34.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.747+0000 7fa3092e7640 1 --2- 172.21.15.154:0/517379360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa30414cc40 0x7fa3040c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:34.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.747+0000 7fa30a2e9640 1 -- 172.21.15.154:0/517379360 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2ec004d00 msgr2=0x7fa3040b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:34.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.747+0000 7fa30a2e9640 1 --2- 172.21.15.154:0/517379360 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2ec004d00 0x7fa3040b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:34.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.747+0000 7fa30a2e9640 1 -- 172.21.15.154:0/517379360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa30414cc40 msgr2=0x7fa3040c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:34.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.747+0000 7fa30a2e9640 1 --2- 172.21.15.154:0/517379360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa30414cc40 0x7fa3040c8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:34.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.747+0000 7fa30a2e9640 1 -- 172.21.15.154:0/517379360 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3040b3870 con 0x7fa3040c9510 2023-09-23T03:19:34.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.747+0000 7fa3092e7640 1 --2- 172.21.15.154:0/517379360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa30414cc40 0x7fa3040c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:34.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.747+0000 7fa30a2e9640 1 --2- 172.21.15.154:0/517379360 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3040c9510 0x7fa3040c64f0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc005040 tx=0x7fa2fc001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:34.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.748+0000 7fa3027fc640 1 -- 172.21.15.154:0/517379360 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc007a30 con 0x7fa3040c9510 2023-09-23T03:19:34.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.748+0000 7fa3027fc640 1 -- 172.21.15.154:0/517379360 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2fc0067d0 con 0x7fa3040c9510 2023-09-23T03:19:34.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.748+0000 7fa3027fc640 1 -- 172.21.15.154:0/517379360 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc006b80 con 0x7fa3040c9510 2023-09-23T03:19:34.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.748+0000 7fa30aaea640 1 -- 172.21.15.154:0/517379360 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3040b3b50 con 0x7fa3040c9510 2023-09-23T03:19:34.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.748+0000 7fa30aaea640 1 -- 172.21.15.154:0/517379360 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3040b3220 con 0x7fa3040c9510 2023-09-23T03:19:34.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.749+0000 7fa30aaea640 1 -- 172.21.15.154:0/517379360 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa30404e920 con 0x7fa3040c9510 2023-09-23T03:19:34.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.750+0000 7fa3027fc640 1 -- 172.21.15.154:0/517379360 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fa2fc006ce0 con 0x7fa3040c9510 2023-09-23T03:19:34.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.751+0000 7fa3027fc640 1 --2- 172.21.15.154:0/517379360 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa2f006af30 0x7fa2f006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:34.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.751+0000 7fa3027fc640 1 -- 172.21.15.154:0/517379360 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa2fc0ab1d0 con 0x7fa3040c9510 2023-09-23T03:19:34.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.751+0000 7fa309ae8640 1 --2- 172.21.15.154:0/517379360 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa2f006af30 0x7fa2f006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:34.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.752+0000 7fa309ae8640 1 --2- 172.21.15.154:0/517379360 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa2f006af30 0x7fa2f006d3f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4001270 tx=0x7fa2f4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:34.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:34.755+0000 7fa3027fc640 1 -- 172.21.15.154:0/517379360 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fa2fc0760c0 con 0x7fa3040c9510 2023-09-23T03:19:35.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.011+0000 7fa30aaea640 1 -- 172.21.15.154:0/517379360 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa3040d1d50 con 0x7fa3040c9510 2023-09-23T03:19:35.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.015+0000 7fa3027fc640 1 -- 172.21.15.154:0/517379360 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7fa2fc0796e0 con 0x7fa3040c9510 2023-09-23T03:19:35.016 INFO:teuthology.orchestra.run.smithi154.stdout:[client.0] 2023-09-23T03:19:35.016 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQAHWA5lYg+qGhAAjaY0fPsmRTz3ipsKbPugMA== 2023-09-23T03:19:35.016 INFO:teuthology.orchestra.run.smithi154.stdout: caps mds = "allow" 2023-09-23T03:19:35.016 INFO:teuthology.orchestra.run.smithi154.stdout: caps mon = "allow r" 2023-09-23T03:19:35.016 INFO:teuthology.orchestra.run.smithi154.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-09-23T03:19:35.016 INFO:teuthology.orchestra.run.smithi154.stderr:updated caps for client.0 2023-09-23T03:19:35.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.018+0000 7fa30aaea640 1 -- 172.21.15.154:0/517379360 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa2f006af30 msgr2=0x7fa2f006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:35.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.018+0000 7fa30aaea640 1 --2- 172.21.15.154:0/517379360 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa2f006af30 0x7fa2f006d3f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4001270 tx=0x7fa2f4009040 comp rx=0 tx=0).stop 2023-09-23T03:19:35.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.018+0000 7fa30aaea640 1 -- 172.21.15.154:0/517379360 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3040c9510 msgr2=0x7fa3040c64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:35.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.018+0000 7fa30aaea640 1 --2- 172.21.15.154:0/517379360 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3040c9510 0x7fa3040c64f0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc005040 tx=0x7fa2fc001120 comp rx=0 tx=0).stop 2023-09-23T03:19:35.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.018+0000 7fa30aaea640 1 -- 172.21.15.154:0/517379360 shutdown_connections 2023-09-23T03:19:35.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.018+0000 7fa30aaea640 1 --2- 172.21.15.154:0/517379360 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa2f006af30 0x7fa2f006d3f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:35.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.018+0000 7fa30aaea640 1 --2- 172.21.15.154:0/517379360 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3040c9510 0x7fa3040c64f0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:35.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.018+0000 7fa30aaea640 1 --2- 172.21.15.154:0/517379360 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa30414cc40 0x7fa3040c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:35.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.018+0000 7fa30aaea640 1 --2- 172.21.15.154:0/517379360 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2ec004d00 0x7fa3040b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:35.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.018+0000 7fa30aaea640 1 -- 172.21.15.154:0/517379360 >> 172.21.15.154:0/517379360 conn(0x7fa30409bc20 msgr2=0x7fa30415ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:35.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.018+0000 7fa30aaea640 1 -- 172.21.15.154:0/517379360 shutdown_connections 2023-09-23T03:19:35.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.018+0000 7fa30aaea640 1 -- 172.21.15.154:0/517379360 wait complete. 2023-09-23T03:19:35.038 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:19:35.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.251+0000 7ff4f794c640 1 Processor -- start 2023-09-23T03:19:35.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.251+0000 7ff4f794c640 1 -- start start 2023-09-23T03:19:35.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.253+0000 7ff4f794c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff4f00b1b20 0x7ff4f00ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:35.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.253+0000 7ff4f794c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4f015ac30 con 0x7ff4f00acce0 2023-09-23T03:19:35.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.253+0000 7ff4f794c640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4f00b1f70 con 0x7ff4f00cee70 2023-09-23T03:19:35.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.253+0000 7ff4f794c640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4f00b2e90 con 0x7ff4f00b1b20 2023-09-23T03:19:35.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.253+0000 7ff4f56e8640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff4f00acce0 0x7ff4f015ed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:59024/0 (socket says 172.21.15.154:59024) 2023-09-23T03:19:35.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.253+0000 7ff4f56e8640 1 -- 172.21.15.154:0/2618773940 learned_addr learned my addr 172.21.15.154:0/2618773940 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:19:35.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.253+0000 7ff4e7fff640 1 -- 172.21.15.154:0/2618773940 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1261325521 0 0) 0x7ff4ec001290 con 0x7ff4f00acce0 2023-09-23T03:19:35.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.254+0000 7ff4e7fff640 1 -- 172.21.15.154:0/2618773940 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4dc003520 con 0x7ff4f00acce0 2023-09-23T03:19:35.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.254+0000 7ff4f4ee7640 1 --2- 172.21.15.154:0/2618773940 >> v2:172.21.15.190:3300/0 conn(0x7ff4f00b1b20 0x7ff4f00ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:35.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.254+0000 7ff4e7fff640 1 -- 172.21.15.154:0/2618773940 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3166258031 0 0) 0x7ff4dc003520 con 0x7ff4f00acce0 2023-09-23T03:19:35.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.254+0000 7ff4e7fff640 1 -- 172.21.15.154:0/2618773940 >> v2:172.21.15.190:3300/0 conn(0x7ff4f00b1b20 msgr2=0x7ff4f00ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:35.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.254+0000 7ff4e7fff640 1 --2- 172.21.15.154:0/2618773940 >> v2:172.21.15.190:3300/0 conn(0x7ff4f00b1b20 0x7ff4f00ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:35.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.254+0000 7ff4e7fff640 1 -- 172.21.15.154:0/2618773940 >> v1:172.21.15.190:6789/0 conn(0x7ff4f00cee70 legacy=0x7ff4f00ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:35.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.254+0000 7ff4e7fff640 1 -- 172.21.15.154:0/2618773940 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f00b2210 con 0x7ff4f00acce0 2023-09-23T03:19:35.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.254+0000 7ff4e7fff640 1 -- 172.21.15.154:0/2618773940 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7ff4ec001980 con 0x7ff4f00acce0 2023-09-23T03:19:35.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.254+0000 7ff4e7fff640 1 -- 172.21.15.154:0/2618773940 >> v1:172.21.15.154:6789/0 conn(0x7ff4f00acce0 legacy=0x7ff4f015ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:35.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.254+0000 7ff4e7fff640 1 --2- 172.21.15.154:0/2618773940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4dc003970 0x7ff4dc003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:35.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.254+0000 7ff4e7fff640 1 --2- 172.21.15.154:0/2618773940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4dc0047c0 0x7ff4dc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:35.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.254+0000 7ff4e7fff640 1 --2- 172.21.15.154:0/2618773940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4dc0052a0 0x7ff4dc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:35.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.254+0000 7ff4e7fff640 1 -- 172.21.15.154:0/2618773940 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f00b2e90 con 0x7ff4dc0047c0 2023-09-23T03:19:35.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.254+0000 7ff4e7fff640 1 -- 172.21.15.154:0/2618773940 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc00c150 con 0x7ff4dc0052a0 2023-09-23T03:19:35.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.254+0000 7ff4e7fff640 1 -- 172.21.15.154:0/2618773940 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc00c310 con 0x7ff4dc003970 2023-09-23T03:19:35.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.254+0000 7ff4f4ee7640 1 --2- 172.21.15.154:0/2618773940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4dc0047c0 0x7ff4dc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:35.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.255+0000 7ff4f5ee9640 1 --2- 172.21.15.154:0/2618773940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4dc0052a0 0x7ff4dc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:35.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.255+0000 7ff4f56e8640 1 --2- 172.21.15.154:0/2618773940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4dc003970 0x7ff4dc003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:35.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.255+0000 7ff4f4ee7640 1 -- 172.21.15.154:0/2618773940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4dc003970 msgr2=0x7ff4dc003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:35.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.255+0000 7ff4f4ee7640 1 --2- 172.21.15.154:0/2618773940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4dc003970 0x7ff4dc003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:35.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.255+0000 7ff4f4ee7640 1 -- 172.21.15.154:0/2618773940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4dc0052a0 msgr2=0x7ff4dc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:35.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.255+0000 7ff4f4ee7640 1 --2- 172.21.15.154:0/2618773940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4dc0052a0 0x7ff4dc00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:35.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.255+0000 7ff4f4ee7640 1 -- 172.21.15.154:0/2618773940 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4dc003520 con 0x7ff4dc0047c0 2023-09-23T03:19:35.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.255+0000 7ff4f5ee9640 1 -- 172.21.15.154:0/2618773940 reap_dead start 2023-09-23T03:19:35.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.255+0000 7ff4f56e8640 1 --2- 172.21.15.154:0/2618773940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4dc003970 0x7ff4dc003d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:35.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.256+0000 7ff4f4ee7640 1 --2- 172.21.15.154:0/2618773940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4dc0047c0 0x7ff4dc004c40 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7ff4f006c660 tx=0x7ff4dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=891d5613a5893453 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:35.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.256+0000 7ff4e7fff640 1 -- 172.21.15.154:0/2618773940 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e00325f0 con 0x7ff4dc0047c0 2023-09-23T03:19:35.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.256+0000 7ff4e7fff640 1 -- 172.21.15.154:0/2618773940 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e0032750 con 0x7ff4dc0047c0 2023-09-23T03:19:35.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.256+0000 7ff4e7fff640 1 -- 172.21.15.154:0/2618773940 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e0032ae0 con 0x7ff4dc0047c0 2023-09-23T03:19:35.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.257+0000 7ff4f794c640 1 -- 172.21.15.154:0/2618773940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4dc0047c0 msgr2=0x7ff4dc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:35.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.257+0000 7ff4f794c640 1 --2- 172.21.15.154:0/2618773940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4dc0047c0 0x7ff4dc004c40 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7ff4f006c660 tx=0x7ff4dc0057f0 comp rx=0 tx=0).stop 2023-09-23T03:19:35.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.257+0000 7ff4f794c640 1 -- 172.21.15.154:0/2618773940 shutdown_connections 2023-09-23T03:19:35.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.257+0000 7ff4f794c640 1 --2- 172.21.15.154:0/2618773940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4dc0047c0 0x7ff4dc004c40 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:35.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.257+0000 7ff4f794c640 1 -- 172.21.15.154:0/2618773940 >> 172.21.15.154:0/2618773940 conn(0x7ff4f006a0f0 msgr2=0x7ff4f006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:35.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.257+0000 7ff4f794c640 1 -- 172.21.15.154:0/2618773940 shutdown_connections 2023-09-23T03:19:35.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.257+0000 7ff4f794c640 1 -- 172.21.15.154:0/2618773940 wait complete. 2023-09-23T03:19:35.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.259+0000 7ff4f794c640 1 Processor -- start 2023-09-23T03:19:35.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.259+0000 7ff4f794c640 1 -- start start 2023-09-23T03:19:35.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.260+0000 7ff4f794c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4dc0047c0 0x7ff4f0154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:35.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.260+0000 7ff4f794c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4f0154a70 0x7ff4f015add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:35.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.260+0000 7ff4f794c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4f015b310 0x7ff4f00c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:35.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.260+0000 7ff4f794c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f00cf320 con 0x7ff4f0154a70 2023-09-23T03:19:35.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.260+0000 7ff4f794c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f00c6f10 con 0x7ff4dc0047c0 2023-09-23T03:19:35.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.260+0000 7ff4f794c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f00c71f0 con 0x7ff4f015b310 2023-09-23T03:19:35.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.260+0000 7ff4f4ee7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4f0154a70 0x7ff4f015add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:35.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.260+0000 7ff4f4ee7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4f0154a70 0x7ff4f015add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57498/0 (socket says 172.21.15.154:57498) 2023-09-23T03:19:35.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.260+0000 7ff4f4ee7640 1 -- 172.21.15.154:0/2910371470 learned_addr learned my addr 172.21.15.154:0/2910371470 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:35.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.260+0000 7ff4f56e8640 1 --2- 172.21.15.154:0/2910371470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4dc0047c0 0x7ff4f0154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:35.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.260+0000 7ff4f5ee9640 1 --2- 172.21.15.154:0/2910371470 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4f015b310 0x7ff4f00c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:35.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.260+0000 7ff4f4ee7640 1 -- 172.21.15.154:0/2910371470 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4f015b310 msgr2=0x7ff4f00c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:35.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.260+0000 7ff4f4ee7640 1 --2- 172.21.15.154:0/2910371470 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4f015b310 0x7ff4f00c9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:35.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.260+0000 7ff4f4ee7640 1 -- 172.21.15.154:0/2910371470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4dc0047c0 msgr2=0x7ff4f0154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:35.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.260+0000 7ff4f4ee7640 1 --2- 172.21.15.154:0/2910371470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4dc0047c0 0x7ff4f0154f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:35.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.260+0000 7ff4f4ee7640 1 -- 172.21.15.154:0/2910371470 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f00c74d0 con 0x7ff4f0154a70 2023-09-23T03:19:35.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.260+0000 7ff4f5ee9640 1 --2- 172.21.15.154:0/2910371470 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4f015b310 0x7ff4f00c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:35.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.261+0000 7ff4f56e8640 1 --2- 172.21.15.154:0/2910371470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4dc0047c0 0x7ff4f0154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:35.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.261+0000 7ff4f4ee7640 1 --2- 172.21.15.154:0/2910371470 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4f0154a70 0x7ff4f015add0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ff4e002be70 tx=0x7ff4e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:35.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.261+0000 7ff4e5ffb640 1 -- 172.21.15.154:0/2910371470 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e0004220 con 0x7ff4f0154a70 2023-09-23T03:19:35.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.261+0000 7ff4e5ffb640 1 -- 172.21.15.154:0/2910371470 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e00043a0 con 0x7ff4f0154a70 2023-09-23T03:19:35.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.261+0000 7ff4e5ffb640 1 -- 172.21.15.154:0/2910371470 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e003ba90 con 0x7ff4f0154a70 2023-09-23T03:19:35.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.261+0000 7ff4f794c640 1 -- 172.21.15.154:0/2910371470 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4f00c77b0 con 0x7ff4f0154a70 2023-09-23T03:19:35.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.261+0000 7ff4f794c640 1 -- 172.21.15.154:0/2910371470 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4f0155670 con 0x7ff4f0154a70 2023-09-23T03:19:35.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.262+0000 7ff4f794c640 1 -- 172.21.15.154:0/2910371470 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4f0073eb0 con 0x7ff4f0154a70 2023-09-23T03:19:35.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.263+0000 7ff4e5ffb640 1 -- 172.21.15.154:0/2910371470 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7ff4e003bbf0 con 0x7ff4f0154a70 2023-09-23T03:19:35.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.264+0000 7ff4e5ffb640 1 --2- 172.21.15.154:0/2910371470 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff4d406af30 0x7ff4d406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:35.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.264+0000 7ff4e5ffb640 1 -- 172.21.15.154:0/2910371470 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff4e00abff0 con 0x7ff4f0154a70 2023-09-23T03:19:35.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.264+0000 7ff4f56e8640 1 --2- 172.21.15.154:0/2910371470 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff4d406af30 0x7ff4d406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:35.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.265+0000 7ff4f56e8640 1 --2- 172.21.15.154:0/2910371470 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff4d406af30 0x7ff4d406d3f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec001ab0 tx=0x7ff4ec009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:35.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.268+0000 7ff4e5ffb640 1 -- 172.21.15.154:0/2910371470 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7ff4e0076f40 con 0x7ff4f0154a70 2023-09-23T03:19:35.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.495+0000 7ff4f794c640 1 -- 172.21.15.154:0/2910371470 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4f0073eb0 con 0x7ff4f0154a70 2023-09-23T03:19:35.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.497+0000 7ff4e5ffb640 1 -- 172.21.15.154:0/2910371470 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4925 (secure 0 0 0) 0x7ff4e007a560 con 0x7ff4f0154a70 2023-09-23T03:19:35.499 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:19:35.499 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":40,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5027,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2517324433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2517324433},{"type":"v1","addr":"172.21.15.190:6833","nonce":2517324433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":5033,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4098915995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4098915995},{"type":"v1","addr":"172.21.15.190:6835","nonce":4098915995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6309,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/3034894009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":3034894009},{"type":"v1","addr":"172.21.15.154:6836","nonce":3034894009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:19:23.952758+0000","modified":"2023-09-23T03:19:27.010345+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6312":{"gid":6312,"name":"c","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6837/2183023203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":2183023203},{"type":"v1","addr":"172.21.15.154:6837","nonce":2183023203}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-09-23T03:19:35.499 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 40 2023-09-23T03:19:35.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.499+0000 7ff4f794c640 1 -- 172.21.15.154:0/2910371470 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff4d406af30 msgr2=0x7ff4d406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:35.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.499+0000 7ff4f794c640 1 --2- 172.21.15.154:0/2910371470 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff4d406af30 0x7ff4d406d3f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec001ab0 tx=0x7ff4ec009040 comp rx=0 tx=0).stop 2023-09-23T03:19:35.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.499+0000 7ff4f794c640 1 -- 172.21.15.154:0/2910371470 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4f0154a70 msgr2=0x7ff4f015add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:35.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.499+0000 7ff4f794c640 1 --2- 172.21.15.154:0/2910371470 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4f0154a70 0x7ff4f015add0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ff4e002be70 tx=0x7ff4e0004730 comp rx=0 tx=0).stop 2023-09-23T03:19:35.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.499+0000 7ff4f794c640 1 -- 172.21.15.154:0/2910371470 shutdown_connections 2023-09-23T03:19:35.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.499+0000 7ff4f794c640 1 --2- 172.21.15.154:0/2910371470 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff4d406af30 0x7ff4d406d3f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:35.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.499+0000 7ff4f794c640 1 --2- 172.21.15.154:0/2910371470 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff4f015b310 0x7ff4f00c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:35.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.499+0000 7ff4f794c640 1 --2- 172.21.15.154:0/2910371470 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4f0154a70 0x7ff4f015add0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:35.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.499+0000 7ff4f794c640 1 --2- 172.21.15.154:0/2910371470 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4dc0047c0 0x7ff4f0154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:35.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.499+0000 7ff4f794c640 1 -- 172.21.15.154:0/2910371470 >> 172.21.15.154:0/2910371470 conn(0x7ff4f006a0f0 msgr2=0x7ff4f00b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:35.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.500+0000 7ff4f794c640 1 -- 172.21.15.154:0/2910371470 shutdown_connections 2023-09-23T03:19:35.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:35.500+0000 7ff4f794c640 1 -- 172.21.15.154:0/2910371470 wait complete. 2023-09-23T03:19:35.920 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 40, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-23T03:19:23.952758+0000', 'modified': '2023-09-23T03:19:27.010345+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6312}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6312': {'gid': 6312, 'name': 'c', 'rank': 0, 'incarnation': 36, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.154:6837/2183023203', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6835', 'nonce': 2183023203}, {'type': 'v1', 'addr': '172.21.15.154:6837', 'nonce': 2183023203}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-23T03:19:35.920 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-23T03:19:35.920 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-09-23T03:19:36.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.149+0000 7f32c1954640 1 Processor -- start 2023-09-23T03:19:36.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.150+0000 7f32c1954640 1 -- start start 2023-09-23T03:19:36.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.151+0000 7f32c1954640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f32bc0cacd0 0x7f32bc0cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:36.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.151+0000 7f32c1954640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f32bc0c8a30 0x7f32bc0c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:36.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.151+0000 7f32c1954640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32bc06bd60 0x7f32bc06c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:36.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.151+0000 7f32c1954640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32bc1580a0 con 0x7f32bc0c8a30 2023-09-23T03:19:36.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.151+0000 7f32c1954640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32bc0c9790 con 0x7f32bc0cacd0 2023-09-23T03:19:36.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.151+0000 7f32c1954640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc165690 con 0x7f32bc06bd60 2023-09-23T03:19:36.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.152+0000 7f32bbfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f32bc0c8a30 0x7f32bc0c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:36.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.152+0000 7f32bbfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f32bc0c8a30 0x7f32bc0c8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57514/0 (socket says 172.21.15.154:57514) 2023-09-23T03:19:36.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.152+0000 7f32bbfff640 1 -- 172.21.15.154:0/2326770047 learned_addr learned my addr 172.21.15.154:0/2326770047 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:36.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.152+0000 7f32c0952640 1 --2- 172.21.15.154:0/2326770047 >> v2:172.21.15.190:3300/0 conn(0x7f32bc0cacd0 0x7f32bc0cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:36.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.152+0000 7f32c1153640 1 --2- 172.21.15.154:0/2326770047 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32bc06bd60 0x7f32bc06c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:36.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.152+0000 7f32bbfff640 1 -- 172.21.15.154:0/2326770047 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32bc06bd60 msgr2=0x7f32bc06c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:36.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.152+0000 7f32bbfff640 1 --2- 172.21.15.154:0/2326770047 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32bc06bd60 0x7f32bc06c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.152+0000 7f32bbfff640 1 -- 172.21.15.154:0/2326770047 >> v2:172.21.15.190:3300/0 conn(0x7f32bc0cacd0 msgr2=0x7f32bc0cb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:36.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.152+0000 7f32bbfff640 1 --2- 172.21.15.154:0/2326770047 >> v2:172.21.15.190:3300/0 conn(0x7f32bc0cacd0 0x7f32bc0cb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.152+0000 7f32bbfff640 1 -- 172.21.15.154:0/2326770047 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32bc0cb5f0 con 0x7f32bc0c8a30 2023-09-23T03:19:36.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.152+0000 7f32c1153640 1 --2- 172.21.15.154:0/2326770047 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32bc06bd60 0x7f32bc06c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:36.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.153+0000 7f32bbfff640 1 --2- 172.21.15.154:0/2326770047 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32bc0c8a30 0x7f32bc0c8e90 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f32bc06d060 tx=0x7f32b002b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=266e0ce56c69d58f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:36.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.154+0000 7f32bb7fe640 1 -- 172.21.15.154:0/2326770047 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b00317d0 con 0x7f32bc0c8a30 2023-09-23T03:19:36.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.154+0000 7f32bb7fe640 1 -- 172.21.15.154:0/2326770047 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b0031930 con 0x7f32bc0c8a30 2023-09-23T03:19:36.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.154+0000 7f32bb7fe640 1 -- 172.21.15.154:0/2326770047 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b0031cc0 con 0x7f32bc0c8a30 2023-09-23T03:19:36.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.154+0000 7f32c1954640 1 -- 172.21.15.154:0/2326770047 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32bc0c8a30 msgr2=0x7f32bc0c8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:36.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.154+0000 7f32c1954640 1 --2- 172.21.15.154:0/2326770047 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32bc0c8a30 0x7f32bc0c8e90 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f32bc06d060 tx=0x7f32b002b870 comp rx=0 tx=0).stop 2023-09-23T03:19:36.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.154+0000 7f32c1954640 1 -- 172.21.15.154:0/2326770047 shutdown_connections 2023-09-23T03:19:36.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.154+0000 7f32c1954640 1 --2- 172.21.15.154:0/2326770047 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32bc06bd60 0x7f32bc06c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.154+0000 7f32c1954640 1 --2- 172.21.15.154:0/2326770047 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32bc0c8a30 0x7f32bc0c8e90 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.154+0000 7f32c1954640 1 --2- 172.21.15.154:0/2326770047 >> v2:172.21.15.190:3300/0 conn(0x7f32bc0cacd0 0x7f32bc0cb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.154+0000 7f32c1954640 1 -- 172.21.15.154:0/2326770047 >> 172.21.15.154:0/2326770047 conn(0x7f32bc06a390 msgr2=0x7f32bc0afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:36.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.154+0000 7f32c1954640 1 -- 172.21.15.154:0/2326770047 shutdown_connections 2023-09-23T03:19:36.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.154+0000 7f32c1954640 1 -- 172.21.15.154:0/2326770047 wait complete. 2023-09-23T03:19:36.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.156+0000 7f32c1954640 1 Processor -- start 2023-09-23T03:19:36.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.156+0000 7f32c1954640 1 -- start start 2023-09-23T03:19:36.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.157+0000 7f32c1954640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32bc06bd60 0x7f32bc1fa840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:36.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.157+0000 7f32c1954640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32bc0c8a30 0x7f32bc155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:36.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.157+0000 7f32c1954640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32bc0cacd0 0x7f32bc163440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:36.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.157+0000 7f32c1954640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc1fad80 con 0x7f32bc0c8a30 2023-09-23T03:19:36.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.157+0000 7f32c1954640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc06fed0 con 0x7f32bc0cacd0 2023-09-23T03:19:36.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.157+0000 7f32c1954640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc0701b0 con 0x7f32bc06bd60 2023-09-23T03:19:36.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.157+0000 7f32bbfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32bc0c8a30 0x7f32bc155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:36.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.158+0000 7f32bbfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32bc0c8a30 0x7f32bc155740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57528/0 (socket says 172.21.15.154:57528) 2023-09-23T03:19:36.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.158+0000 7f32bbfff640 1 -- 172.21.15.154:0/1509912446 learned_addr learned my addr 172.21.15.154:0/1509912446 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:36.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.158+0000 7f32c0952640 1 --2- 172.21.15.154:0/1509912446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32bc06bd60 0x7f32bc1fa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:36.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.158+0000 7f32c1153640 1 --2- 172.21.15.154:0/1509912446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32bc0cacd0 0x7f32bc163440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:36.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.158+0000 7f32bbfff640 1 -- 172.21.15.154:0/1509912446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32bc06bd60 msgr2=0x7f32bc1fa840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:36.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.158+0000 7f32bbfff640 1 --2- 172.21.15.154:0/1509912446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32bc06bd60 0x7f32bc1fa840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.158+0000 7f32bbfff640 1 -- 172.21.15.154:0/1509912446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32bc0cacd0 msgr2=0x7f32bc163440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:36.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.158+0000 7f32bbfff640 1 --2- 172.21.15.154:0/1509912446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32bc0cacd0 0x7f32bc163440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.158+0000 7f32bbfff640 1 -- 172.21.15.154:0/1509912446 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b00094d0 con 0x7f32bc0c8a30 2023-09-23T03:19:36.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.158+0000 7f32c1153640 1 --2- 172.21.15.154:0/1509912446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32bc0cacd0 0x7f32bc163440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:36.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.158+0000 7f32bbfff640 1 --2- 172.21.15.154:0/1509912446 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32bc0c8a30 0x7f32bc155740 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f32b0000c00 tx=0x7f32b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:36.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.159+0000 7f32b97fa640 1 -- 172.21.15.154:0/1509912446 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b0032870 con 0x7f32bc0c8a30 2023-09-23T03:19:36.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.159+0000 7f32b97fa640 1 -- 172.21.15.154:0/1509912446 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b00329f0 con 0x7f32bc0c8a30 2023-09-23T03:19:36.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.159+0000 7f32b97fa640 1 -- 172.21.15.154:0/1509912446 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b0032da0 con 0x7f32bc0c8a30 2023-09-23T03:19:36.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.159+0000 7f32c1954640 1 -- 172.21.15.154:0/1509912446 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32bc070490 con 0x7f32bc0c8a30 2023-09-23T03:19:36.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.159+0000 7f32c1954640 1 -- 172.21.15.154:0/1509912446 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32bc070a50 con 0x7f32bc0c8a30 2023-09-23T03:19:36.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.161+0000 7f32b97fa640 1 -- 172.21.15.154:0/1509912446 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f32b0030070 con 0x7f32bc0c8a30 2023-09-23T03:19:36.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.162+0000 7f32b97fa640 1 --2- 172.21.15.154:0/1509912446 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f32a806af30 0x7f32a806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:36.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.162+0000 7f32b97fa640 1 -- 172.21.15.154:0/1509912446 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f32b00ab3a0 con 0x7f32bc0c8a30 2023-09-23T03:19:36.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.162+0000 7f32c0952640 1 --2- 172.21.15.154:0/1509912446 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f32a806af30 0x7f32a806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:36.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.163+0000 7f32c0952640 1 --2- 172.21.15.154:0/1509912446 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f32a806af30 0x7f32a806d3f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f32ac006fd0 tx=0x7f32ac008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:36.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.163+0000 7f32c5825740 1 Processor -- start 2023-09-23T03:19:36.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.163+0000 7f32c5825740 1 -- start start 2023-09-23T03:19:36.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.164+0000 7f32c5825740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5e026bda0 0x55e5e023eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:36.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.164+0000 7f32c5825740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5e026c180 0x55e5e026eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:36.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.164+0000 7f32c5825740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e5e023f1e0 0x55e5e026f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:36.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.164+0000 7f32c5825740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e01b26d0 con 0x55e5e026bda0 2023-09-23T03:19:36.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.164+0000 7f32c5825740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e01afdc0 con 0x55e5e026c180 2023-09-23T03:19:36.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.164+0000 7f32c5825740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e5e01ab6b0 con 0x55e5e023f1e0 2023-09-23T03:19:36.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.164+0000 7f32c0952640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5e026bda0 0x55e5e023eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:36.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.164+0000 7f32c0952640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5e026bda0 0x55e5e023eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57538/0 (socket says 172.21.15.154:57538) 2023-09-23T03:19:36.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.164+0000 7f32c0952640 1 -- 172.21.15.154:0/417182597 learned_addr learned my addr 172.21.15.154:0/417182597 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:36.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.164+0000 7f32c0952640 1 -- 172.21.15.154:0/417182597 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e5e023f1e0 msgr2=0x55e5e026f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:36.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.164+0000 7f32c0952640 1 --2- 172.21.15.154:0/417182597 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e5e023f1e0 0x55e5e026f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.164+0000 7f32c0952640 1 -- 172.21.15.154:0/417182597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5e026c180 msgr2=0x55e5e026eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:36.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.164+0000 7f32c0952640 1 --2- 172.21.15.154:0/417182597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5e026c180 0x55e5e026eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.164+0000 7f32c0952640 1 -- 172.21.15.154:0/417182597 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5e016f8e0 con 0x55e5e026bda0 2023-09-23T03:19:36.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.165+0000 7f32c0952640 1 --2- 172.21.15.154:0/417182597 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5e026bda0 0x55e5e023eca0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00f960 tx=0x7f32ac012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb952e8d9009709 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:36.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.165+0000 7f32a27fc640 1 -- 172.21.15.154:0/417182597 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac00fee0 con 0x55e5e026bda0 2023-09-23T03:19:36.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.165+0000 7f32a27fc640 1 -- 172.21.15.154:0/417182597 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ac014be0 con 0x55e5e026bda0 2023-09-23T03:19:36.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.165+0000 7f32a27fc640 1 -- 172.21.15.154:0/417182597 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac01f3f0 con 0x55e5e026bda0 2023-09-23T03:19:36.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.166+0000 7f32c5825740 1 -- 172.21.15.154:0/417182597 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5e026bda0 msgr2=0x55e5e023eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:36.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.166+0000 7f32c5825740 1 --2- 172.21.15.154:0/417182597 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5e026bda0 0x55e5e023eca0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00f960 tx=0x7f32ac012000 comp rx=0 tx=0).stop 2023-09-23T03:19:36.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.166+0000 7f32c5825740 1 -- 172.21.15.154:0/417182597 shutdown_connections 2023-09-23T03:19:36.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.166+0000 7f32c5825740 1 --2- 172.21.15.154:0/417182597 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e5e023f1e0 0x55e5e026f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.166+0000 7f32c5825740 1 --2- 172.21.15.154:0/417182597 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5e026c180 0x55e5e026eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.166+0000 7f32c5825740 1 --2- 172.21.15.154:0/417182597 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5e026bda0 0x55e5e023eca0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.166+0000 7f32c5825740 1 -- 172.21.15.154:0/417182597 >> 172.21.15.154:0/417182597 conn(0x55e5e0240fb0 msgr2=0x55e5e0289b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:36.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.166+0000 7f32c5825740 1 -- 172.21.15.154:0/417182597 shutdown_connections 2023-09-23T03:19:36.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.166+0000 7f32c5825740 1 -- 172.21.15.154:0/417182597 wait complete. 2023-09-23T03:19:36.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.167+0000 7f32c5825740 1 -- start start 2023-09-23T03:19:36.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.167+0000 7f32c5825740 1 Processor -- start 2023-09-23T03:19:36.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.168+0000 7f32c5825740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5e026bda0 0x55e5e025c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:36.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.168+0000 7f32c5825740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e5e026c180 0x55e5e0270f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:36.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.168+0000 7f32c5825740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5e025cd90 0x55e5e0284160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:36.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.168+0000 7f32c5825740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e0149940 con 0x55e5e026bda0 2023-09-23T03:19:36.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.168+0000 7f32c5825740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e01495e0 con 0x55e5e025cd90 2023-09-23T03:19:36.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.168+0000 7f32c5825740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e5dff0ff20 con 0x55e5e026c180 2023-09-23T03:19:36.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.168+0000 7f32c5825740 10 client.0 _ll_register_callbacks cb 0x55e5e01875e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:19:36.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.168+0000 7f32c0952640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5e026bda0 0x55e5e025c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:36.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.168+0000 7f32c0952640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5e026bda0 0x55e5e025c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57540/0 (socket says 172.21.15.154:57540) 2023-09-23T03:19:36.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.168+0000 7f32c0952640 1 -- 172.21.15.154:0/711049976 learned_addr learned my addr 172.21.15.154:0/711049976 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:36.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.168+0000 7f32c0952640 1 -- 172.21.15.154:0/711049976 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e5e026c180 msgr2=0x55e5e0270f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:36.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.168+0000 7f32c0952640 1 --2- 172.21.15.154:0/711049976 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e5e026c180 0x55e5e0270f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.168+0000 7f32c0952640 1 -- 172.21.15.154:0/711049976 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5e025cd90 msgr2=0x55e5e0284160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:36.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.168+0000 7f32c0952640 1 --2- 172.21.15.154:0/711049976 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5e025cd90 0x55e5e0284160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.168+0000 7f32c0952640 1 -- 172.21.15.154:0/711049976 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32ac00e040 con 0x55e5e026bda0 2023-09-23T03:19:36.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.169+0000 7f32c0952640 1 --2- 172.21.15.154:0/711049976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5e026bda0 0x55e5e025c850 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f32ac014830 tx=0x7f32ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32fc5930de9692c3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:36.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.169+0000 7f329b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:19:36.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.169+0000 7f329b7fe640 1 -- 172.21.15.154:0/711049976 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac02d070 con 0x55e5e026bda0 2023-09-23T03:19:36.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.169+0000 7f32c5825740 1 -- 172.21.15.154:0/711049976 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e5e01a4690 con 0x55e5e026bda0 2023-09-23T03:19:36.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.169+0000 7f329b7fe640 1 -- 172.21.15.154:0/711049976 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(62..62 src has 1..62) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f32ac01a040 con 0x55e5e026bda0 2023-09-23T03:19:36.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.169+0000 7f329b7fe640 1 -- 172.21.15.154:0/711049976 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32ac0149e0 con 0x55e5e026bda0 2023-09-23T03:19:36.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.169+0000 7f32c5825740 10 client.6468 fetch_fsmap learned FSMap version 40 2023-09-23T03:19:36.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.169+0000 7f32c5825740 1 -- 172.21.15.154:0/711049976 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x55e5e01a9d30 con 0x55e5e026bda0 2023-09-23T03:19:36.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.169+0000 7f329b7fe640 1 -- 172.21.15.154:0/711049976 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e5e01a4690 con 0x55e5e026bda0 2023-09-23T03:19:36.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.170+0000 7f329b7fe640 1 -- 172.21.15.154:0/711049976 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 40) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f32ac01f3f0 con 0x55e5e026bda0 2023-09-23T03:19:36.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.170+0000 7f32c5825740 10 client.6468 fetch_fsmap finished waiting for FSMap version 40 2023-09-23T03:19:36.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.170+0000 7f32c5825740 10 client.6468 resolve_mds: resolved name 'c' to daemon mds.c 2023-09-23T03:19:36.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.170+0000 7f32c5825740 20 client.6468 populate_metadata read hostname 'smithi154' 2023-09-23T03:19:36.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.170+0000 7f32c5825740 1 --2- 172.21.15.154:0/711049976 >> [v2:172.21.15.154:6835/2183023203,v1:172.21.15.154:6837/2183023203] conn(0x55e5e02846a0 0x55e5e025bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:36.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.170+0000 7f32c5825740 4 client.6468 mds_command: new command op to 6312 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T03:19:36.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.170+0000 7f32c5825740 1 -- 172.21.15.154:0/711049976 --> [v2:172.21.15.154:6835/2183023203,v1:172.21.15.154:6837/2183023203] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e5e01a43b0 con 0x55e5e02846a0 2023-09-23T03:19:36.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.170+0000 7f32bbfff640 1 --2- 172.21.15.154:0/711049976 >> [v2:172.21.15.154:6835/2183023203,v1:172.21.15.154:6837/2183023203] conn(0x55e5e02846a0 0x55e5e025bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:36.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.171+0000 7f32bbfff640 1 --2- 172.21.15.154:0/711049976 >> [v2:172.21.15.154:6835/2183023203,v1:172.21.15.154:6837/2183023203] conn(0x55e5e02846a0 0x55e5e025bf00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92ec1bbc4a73dc3d server_cookie=9f0065922a19c27d in_seq=0 out_seq=0 2023-09-23T03:19:36.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.171+0000 7f329b7fe640 10 client.6468 ms_handle_connect on v2:172.21.15.154:6835/2183023203 2023-09-23T03:19:36.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.171+0000 7f329b7fe640 1 -- 172.21.15.154:0/711049976 <== mds.0 v2:172.21.15.154:6835/2183023203 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e5e01a43b0 con 0x55e5e02846a0 2023-09-23T03:19:36.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.171+0000 7f329b7fe640 10 client.6468 handle_command_reply: tid=0 2023-09-23T03:19:36.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.171+0000 7f32c5825740 1 client.6468 shutdown 2023-09-23T03:19:36.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.171+0000 7f32c5825740 1 -- 172.21.15.154:0/711049976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5e026bda0 msgr2=0x55e5e025c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:36.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.172+0000 7f32c5825740 1 --2- 172.21.15.154:0/711049976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5e026bda0 0x55e5e025c850 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f32ac014830 tx=0x7f32ac004730 comp rx=0 tx=0).stop 2023-09-23T03:19:36.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.172+0000 7f32c5825740 1 -- 172.21.15.154:0/711049976 shutdown_connections 2023-09-23T03:19:36.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.172+0000 7f32c5825740 1 --2- 172.21.15.154:0/711049976 >> [v2:172.21.15.154:6835/2183023203,v1:172.21.15.154:6837/2183023203] conn(0x55e5e02846a0 0x55e5e025bf00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.172+0000 7f32c5825740 1 --2- 172.21.15.154:0/711049976 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5e025cd90 0x55e5e0284160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.172+0000 7f32c5825740 1 --2- 172.21.15.154:0/711049976 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e5e026c180 0x55e5e0270f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.172+0000 7f32c5825740 1 --2- 172.21.15.154:0/711049976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5e026bda0 0x55e5e025c850 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.172+0000 7f329b7fe640 0 client.6468 ms_handle_reset on v2:172.21.15.154:6835/2183023203 2023-09-23T03:19:36.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.172+0000 7f32c5825740 1 -- 172.21.15.154:0/711049976 >> 172.21.15.154:0/711049976 conn(0x55e5e023f1e0 msgr2=0x55e5e028d040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:36.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.172+0000 7f32c5825740 1 -- 172.21.15.154:0/711049976 shutdown_connections 2023-09-23T03:19:36.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.172+0000 7f32c5825740 1 -- 172.21.15.154:0/711049976 wait complete. 2023-09-23T03:19:36.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.172+0000 7f32c5825740 20 client.6468 trim_cache size 0 max 16384 2023-09-23T03:19:36.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.188+0000 7f32c5825740 1 Processor -- start 2023-09-23T03:19:36.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.188+0000 7f32c5825740 1 -- start start 2023-09-23T03:19:36.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.189+0000 7f32c5825740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5e023f1e0 0x55e5e028a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:36.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.189+0000 7f32c5825740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5e026bda0 0x55e5e026f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:36.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.189+0000 7f32c5825740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e5e026c180 0x55e5e028be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:36.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.189+0000 7f32c5825740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e028ca80 con 0x55e5e023f1e0 2023-09-23T03:19:36.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.189+0000 7f32c5825740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e028c860 con 0x55e5e026bda0 2023-09-23T03:19:36.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.189+0000 7f32c5825740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e5e028c3c0 con 0x55e5e026c180 2023-09-23T03:19:36.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.189+0000 7f32c0952640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5e023f1e0 0x55e5e028a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:36.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.189+0000 7f32c0952640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5e023f1e0 0x55e5e028a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57548/0 (socket says 172.21.15.154:57548) 2023-09-23T03:19:36.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.189+0000 7f32c0952640 1 -- 172.21.15.154:0/3957258291 learned_addr learned my addr 172.21.15.154:0/3957258291 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:36.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.189+0000 7f32c0952640 1 -- 172.21.15.154:0/3957258291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e5e026c180 msgr2=0x55e5e028be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:36.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.189+0000 7f32c0952640 1 --2- 172.21.15.154:0/3957258291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e5e026c180 0x55e5e028be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.189+0000 7f32c0952640 1 -- 172.21.15.154:0/3957258291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5e026bda0 msgr2=0x55e5e026f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:36.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.189+0000 7f32c0952640 1 --2- 172.21.15.154:0/3957258291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5e026bda0 0x55e5e026f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.189+0000 7f32c0952640 1 -- 172.21.15.154:0/3957258291 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5e0148b50 con 0x55e5e023f1e0 2023-09-23T03:19:36.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.190+0000 7f32c0952640 1 --2- 172.21.15.154:0/3957258291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5e023f1e0 0x55e5e028a5f0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00d5b0 tx=0x7f32ac02d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edda08f4d39fbbef server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:36.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.190+0000 7f32a2ffd640 1 -- 172.21.15.154:0/3957258291 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac028760 con 0x55e5e023f1e0 2023-09-23T03:19:36.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.190+0000 7f32a2ffd640 1 -- 172.21.15.154:0/3957258291 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ac028b10 con 0x55e5e023f1e0 2023-09-23T03:19:36.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.190+0000 7f32a2ffd640 1 -- 172.21.15.154:0/3957258291 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac01fb90 con 0x55e5e023f1e0 2023-09-23T03:19:36.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.190+0000 7f32c5825740 1 -- 172.21.15.154:0/3957258291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5e023f1e0 msgr2=0x55e5e028a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:36.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.190+0000 7f32c5825740 1 --2- 172.21.15.154:0/3957258291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5e023f1e0 0x55e5e028a5f0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00d5b0 tx=0x7f32ac02d000 comp rx=0 tx=0).stop 2023-09-23T03:19:36.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.190+0000 7f32c5825740 1 -- 172.21.15.154:0/3957258291 shutdown_connections 2023-09-23T03:19:36.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.190+0000 7f32c5825740 1 --2- 172.21.15.154:0/3957258291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e5e026c180 0x55e5e028be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.190+0000 7f32c5825740 1 --2- 172.21.15.154:0/3957258291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5e026bda0 0x55e5e026f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.190+0000 7f32c5825740 1 --2- 172.21.15.154:0/3957258291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5e023f1e0 0x55e5e028a5f0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.190+0000 7f32c5825740 1 -- 172.21.15.154:0/3957258291 >> 172.21.15.154:0/3957258291 conn(0x55e5e028cbd0 msgr2=0x55e5e0285440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:36.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.190+0000 7f32c5825740 1 -- 172.21.15.154:0/3957258291 shutdown_connections 2023-09-23T03:19:36.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.190+0000 7f32c5825740 1 -- 172.21.15.154:0/3957258291 wait complete. 2023-09-23T03:19:36.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.191+0000 7f32c5825740 1 -- start start 2023-09-23T03:19:36.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.191+0000 7f32c5825740 1 Processor -- start 2023-09-23T03:19:36.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.192+0000 7f32c5825740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5e023f1e0 0x55e5e02f59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:36.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.192+0000 7f32c5825740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5e026bda0 0x55e5e0298120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:36.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.192+0000 7f32c5825740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e5e026c180 0x55e5e02f29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:36.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.192+0000 7f32c5825740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e02e92f0 con 0x55e5e026bda0 2023-09-23T03:19:36.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.192+0000 7f32c5825740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5e028c250 con 0x55e5e023f1e0 2023-09-23T03:19:36.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.192+0000 7f32c5825740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e5e028b5a0 con 0x55e5e026c180 2023-09-23T03:19:36.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.192+0000 7f32c5825740 10 client.0 _ll_register_callbacks cb 0x7f32ac00d600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:19:36.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.192+0000 7f32bbfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5e026bda0 0x55e5e0298120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:36.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.193+0000 7f32bbfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5e026bda0 0x55e5e0298120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57554/0 (socket says 172.21.15.154:57554) 2023-09-23T03:19:36.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.193+0000 7f32bbfff640 1 -- 172.21.15.154:0/2056430931 learned_addr learned my addr 172.21.15.154:0/2056430931 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:36.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.193+0000 7f32bbfff640 1 -- 172.21.15.154:0/2056430931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e5e026c180 msgr2=0x55e5e02f29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:36.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.193+0000 7f32bbfff640 1 --2- 172.21.15.154:0/2056430931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e5e026c180 0x55e5e02f29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.193+0000 7f32bbfff640 1 -- 172.21.15.154:0/2056430931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5e023f1e0 msgr2=0x55e5e02f59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:36.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.193+0000 7f32bbfff640 1 --2- 172.21.15.154:0/2056430931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5e023f1e0 0x55e5e02f59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.193+0000 7f32bbfff640 1 -- 172.21.15.154:0/2056430931 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e5e01a9d30 con 0x55e5e026bda0 2023-09-23T03:19:36.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.193+0000 7f32bbfff640 1 --2- 172.21.15.154:0/2056430931 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5e026bda0 0x55e5e0298120 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f32b002b8b0 tx=0x7f32b007a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec2b242057fefee server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:36.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.193+0000 7f329b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:19:36.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.193+0000 7f329b7fe640 1 -- 172.21.15.154:0/2056430931 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b007eea0 con 0x55e5e026bda0 2023-09-23T03:19:36.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.193+0000 7f329b7fe640 1 -- 172.21.15.154:0/2056430931 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(62..62 src has 1..62) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f32b0082310 con 0x55e5e026bda0 2023-09-23T03:19:36.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.193+0000 7f32c5825740 1 -- 172.21.15.154:0/2056430931 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f32ac014be0 con 0x55e5e026bda0 2023-09-23T03:19:36.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.193+0000 7f329b7fe640 1 -- 172.21.15.154:0/2056430931 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32b00ab0b0 con 0x55e5e026bda0 2023-09-23T03:19:36.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.194+0000 7f32c5825740 10 client.6474 fetch_fsmap learned FSMap version 40 2023-09-23T03:19:36.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.194+0000 7f32c5825740 1 -- 172.21.15.154:0/2056430931 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x55e5e016e140 con 0x55e5e026bda0 2023-09-23T03:19:36.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.194+0000 7f329b7fe640 1 -- 172.21.15.154:0/2056430931 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ac014be0 con 0x55e5e026bda0 2023-09-23T03:19:36.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.194+0000 7f329b7fe640 1 -- 172.21.15.154:0/2056430931 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 40) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f32b00767d0 con 0x55e5e026bda0 2023-09-23T03:19:36.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.194+0000 7f32c5825740 10 client.6474 fetch_fsmap finished waiting for FSMap version 40 2023-09-23T03:19:36.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.194+0000 7f32c5825740 10 client.6474 resolve_mds: resolved name 'c' to daemon mds.c 2023-09-23T03:19:36.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.194+0000 7f32c5825740 20 client.6474 populate_metadata read hostname 'smithi154' 2023-09-23T03:19:36.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.194+0000 7f32c5825740 1 --2- 172.21.15.154:0/2056430931 >> [v2:172.21.15.154:6835/2183023203,v1:172.21.15.154:6837/2183023203] conn(0x55e5e02f2fe0 0x55e5e027f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:36.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.194+0000 7f32c5825740 4 client.6474 mds_command: new command op to 6312 tid=0[{"prefix": "status"}] 2023-09-23T03:19:36.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.194+0000 7f32c5825740 1 -- 172.21.15.154:0/2056430931 --> [v2:172.21.15.154:6835/2183023203,v1:172.21.15.154:6837/2183023203] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e5dfead6c0 con 0x55e5e02f2fe0 2023-09-23T03:19:36.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.194+0000 7f32c0952640 1 --2- 172.21.15.154:0/2056430931 >> [v2:172.21.15.154:6835/2183023203,v1:172.21.15.154:6837/2183023203] conn(0x55e5e02f2fe0 0x55e5e027f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:36.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.195+0000 7f329b7fe640 10 client.6474 ms_handle_connect on v2:172.21.15.154:6835/2183023203 2023-09-23T03:19:36.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.195+0000 7f32c0952640 1 --2- 172.21.15.154:0/2056430931 >> [v2:172.21.15.154:6835/2183023203,v1:172.21.15.154:6837/2183023203] conn(0x55e5e02f2fe0 0x55e5e027f4c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6aa43ff96038e041 server_cookie=e1f55997ddc82634 in_seq=0 out_seq=0 2023-09-23T03:19:36.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.195+0000 7f329b7fe640 1 -- 172.21.15.154:0/2056430931 <== mds.0 v2:172.21.15.154:6835/2183023203 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55e5dfead6c0 con 0x55e5e02f2fe0 2023-09-23T03:19:36.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.195+0000 7f329b7fe640 10 client.6474 handle_command_reply: tid=0 2023-09-23T03:19:36.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.195+0000 7f32c5825740 1 client.6474 shutdown 2023-09-23T03:19:36.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.195+0000 7f32c5825740 1 -- 172.21.15.154:0/2056430931 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5e026bda0 msgr2=0x55e5e0298120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:36.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.195+0000 7f32c5825740 1 --2- 172.21.15.154:0/2056430931 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5e026bda0 0x55e5e0298120 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f32b002b8b0 tx=0x7f32b007a180 comp rx=0 tx=0).stop 2023-09-23T03:19:36.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.195+0000 7f32c5825740 1 -- 172.21.15.154:0/2056430931 shutdown_connections 2023-09-23T03:19:36.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.195+0000 7f32c5825740 1 --2- 172.21.15.154:0/2056430931 >> [v2:172.21.15.154:6835/2183023203,v1:172.21.15.154:6837/2183023203] conn(0x55e5e02f2fe0 0x55e5e027f4c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.195+0000 7f32c5825740 1 --2- 172.21.15.154:0/2056430931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e5e026c180 0x55e5e02f29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.195+0000 7f32c5825740 1 --2- 172.21.15.154:0/2056430931 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e5e026bda0 0x55e5e0298120 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.195+0000 7f32c5825740 1 --2- 172.21.15.154:0/2056430931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e5e023f1e0 0x55e5e02f59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.195+0000 7f329b7fe640 0 client.6474 ms_handle_reset on v2:172.21.15.154:6835/2183023203 2023-09-23T03:19:36.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.195+0000 7f32c5825740 1 -- 172.21.15.154:0/2056430931 >> 172.21.15.154:0/2056430931 conn(0x55e5e028cbd0 msgr2=0x55e5e028a100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:36.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.195+0000 7f32c5825740 1 -- 172.21.15.154:0/2056430931 shutdown_connections 2023-09-23T03:19:36.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.195+0000 7f32c5825740 1 -- 172.21.15.154:0/2056430931 wait complete. 2023-09-23T03:19:36.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.195+0000 7f32c5825740 20 client.6474 trim_cache size 0 max 16384 2023-09-23T03:19:36.204 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:19:36.204 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_fsid": "8c8039d3-6c84-43db-8808-85e0dbdd47c8", 2023-09-23T03:19:36.204 INFO:teuthology.orchestra.run.smithi154.stdout: "whoami": 0, 2023-09-23T03:19:36.204 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 6312, 2023-09-23T03:19:36.204 INFO:teuthology.orchestra.run.smithi154.stdout: "want_state": "up:active", 2023-09-23T03:19:36.204 INFO:teuthology.orchestra.run.smithi154.stdout: "state": "up:active", 2023-09-23T03:19:36.204 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs", 2023-09-23T03:19:36.204 INFO:teuthology.orchestra.run.smithi154.stdout: "rank_uptime": 11.712850477, 2023-09-23T03:19:36.204 INFO:teuthology.orchestra.run.smithi154.stdout: "mdsmap_epoch": 40, 2023-09-23T03:19:36.205 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_epoch": 62, 2023-09-23T03:19:36.205 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_epoch_barrier": 62, 2023-09-23T03:19:36.205 INFO:teuthology.orchestra.run.smithi154.stdout: "uptime": 18.114097773000001 2023-09-23T03:19:36.205 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:19:36.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.196+0000 7f32c1954640 1 -- 172.21.15.154:0/1509912446 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f32a806af30 msgr2=0x7f32a806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:36.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.196+0000 7f32c1954640 1 --2- 172.21.15.154:0/1509912446 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f32a806af30 0x7f32a806d3f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f32ac006fd0 tx=0x7f32ac008040 comp rx=0 tx=0).stop 2023-09-23T03:19:36.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.196+0000 7f32c1954640 1 -- 172.21.15.154:0/1509912446 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32bc0c8a30 msgr2=0x7f32bc155740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:36.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.196+0000 7f32c1954640 1 --2- 172.21.15.154:0/1509912446 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32bc0c8a30 0x7f32bc155740 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f32b0000c00 tx=0x7f32b00046c0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.196+0000 7f32c1954640 1 -- 172.21.15.154:0/1509912446 shutdown_connections 2023-09-23T03:19:36.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.196+0000 7f32c1954640 1 --2- 172.21.15.154:0/1509912446 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f32a806af30 0x7f32a806d3f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.196+0000 7f32c1954640 1 --2- 172.21.15.154:0/1509912446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f32bc0cacd0 0x7f32bc163440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.196+0000 7f32c1954640 1 --2- 172.21.15.154:0/1509912446 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f32bc0c8a30 0x7f32bc155740 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.196+0000 7f32c1954640 1 --2- 172.21.15.154:0/1509912446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32bc06bd60 0x7f32bc1fa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.196+0000 7f32c1954640 1 -- 172.21.15.154:0/1509912446 >> 172.21.15.154:0/1509912446 conn(0x7f32bc06a390 msgr2=0x7f32bc0afdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:36.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.196+0000 7f32c1954640 1 -- 172.21.15.154:0/1509912446 shutdown_connections 2023-09-23T03:19:36.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.196+0000 7f32c1954640 1 -- 172.21.15.154:0/1509912446 wait complete. 2023-09-23T03:19:36.213 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-23T03:19:36.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.413+0000 7fccdd939640 1 Processor -- start 2023-09-23T03:19:36.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.413+0000 7fccdd939640 1 -- start start 2023-09-23T03:19:36.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.415+0000 7fccdd939640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fccd80cf130 0x7fccd80ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:36.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.415+0000 7fccdd939640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccd806ceb0 con 0x7fccd80b2fe0 2023-09-23T03:19:36.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.415+0000 7fccdd939640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccd8159c90 con 0x7fccd80ada30 2023-09-23T03:19:36.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.415+0000 7fccdd939640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccd80ade10 con 0x7fccd80cf130 2023-09-23T03:19:36.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.415+0000 7fccd67fc640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fccd80cf130 0x7fccd80ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:36.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.415+0000 7fccd77fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fccd80b2fe0 0x7fccd815d380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:59028/0 (socket says 172.21.15.154:59028) 2023-09-23T03:19:36.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.415+0000 7fccd77fe640 1 -- 172.21.15.154:0/1823969912 learned_addr learned my addr 172.21.15.154:0/1823969912 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:19:36.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.415+0000 7fccd67fc640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fccd80cf130 0x7fccd80ccaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57558/0 (socket says 172.21.15.154:57558) 2023-09-23T03:19:36.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.416+0000 7fccd67fc640 1 -- 172.21.15.154:0/1823969912 >> v1:172.21.15.190:6789/0 conn(0x7fccd80ada30 legacy=0x7fccd80cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:36.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.416+0000 7fccd67fc640 1 -- 172.21.15.154:0/1823969912 >> v1:172.21.15.154:6789/0 conn(0x7fccd80b2fe0 legacy=0x7fccd815d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:36.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.416+0000 7fccd67fc640 1 -- 172.21.15.154:0/1823969912 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd806c110 con 0x7fccd80cf130 2023-09-23T03:19:36.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.416+0000 7fccd5ffb640 1 -- 172.21.15.154:0/1823969912 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2024945495 0 0) 0x7fccd0001490 con 0x7fccd80b2fe0 2023-09-23T03:19:36.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.417+0000 7fccd67fc640 1 --2- 172.21.15.154:0/1823969912 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccd80cf130 0x7fccd80ccaf0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fcccc00ba40 tx=0x7fcccc02e090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee572530fd3c99c7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:36.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.417+0000 7fccd5ffb640 1 -- 172.21.15.154:0/1823969912 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcccc02ee40 con 0x7fccd80cf130 2023-09-23T03:19:36.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.417+0000 7fccd5ffb640 1 -- 172.21.15.154:0/1823969912 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcccc036440 con 0x7fccd80cf130 2023-09-23T03:19:36.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.417+0000 7fccd5ffb640 1 -- 172.21.15.154:0/1823969912 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcccc0367d0 con 0x7fccd80cf130 2023-09-23T03:19:36.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.418+0000 7fccdd939640 1 -- 172.21.15.154:0/1823969912 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccd80cf130 msgr2=0x7fccd80ccaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:36.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.418+0000 7fccdd939640 1 --2- 172.21.15.154:0/1823969912 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccd80cf130 0x7fccd80ccaf0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fcccc00ba40 tx=0x7fcccc02e090 comp rx=0 tx=0).stop 2023-09-23T03:19:36.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.418+0000 7fccdd939640 1 -- 172.21.15.154:0/1823969912 shutdown_connections 2023-09-23T03:19:36.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.418+0000 7fccdd939640 1 --2- 172.21.15.154:0/1823969912 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccd80cf130 0x7fccd80ccaf0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.418+0000 7fccdd939640 1 -- 172.21.15.154:0/1823969912 >> 172.21.15.154:0/1823969912 conn(0x7fccd806a0f0 msgr2=0x7fccd806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:36.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.418+0000 7fccdd939640 1 -- 172.21.15.154:0/1823969912 shutdown_connections 2023-09-23T03:19:36.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.418+0000 7fccdd939640 1 -- 172.21.15.154:0/1823969912 wait complete. 2023-09-23T03:19:36.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.420+0000 7fccdd939640 1 Processor -- start 2023-09-23T03:19:36.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.420+0000 7fccdd939640 1 -- start start 2023-09-23T03:19:36.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.421+0000 7fccdd939640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccd80ada30 0x7fccd815c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:36.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.421+0000 7fccdd939640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fccd80b2fe0 0x7fccd81f33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:36.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.421+0000 7fccdd939640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccd80cf130 0x7fccd81f3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:36.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.421+0000 7fccdd939640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd815b0d0 con 0x7fccd80ada30 2023-09-23T03:19:36.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.421+0000 7fccdd939640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd815a870 con 0x7fccd80cf130 2023-09-23T03:19:36.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.421+0000 7fccdd939640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccd815ab10 con 0x7fccd80b2fe0 2023-09-23T03:19:36.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.421+0000 7fccd6ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccd80ada30 0x7fccd815c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:36.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.421+0000 7fccd6ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccd80ada30 0x7fccd815c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57566/0 (socket says 172.21.15.154:57566) 2023-09-23T03:19:36.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.421+0000 7fccd6ffd640 1 -- 172.21.15.154:0/3215157699 learned_addr learned my addr 172.21.15.154:0/3215157699 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:36.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.422+0000 7fccd6ffd640 1 -- 172.21.15.154:0/3215157699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fccd80b2fe0 msgr2=0x7fccd81f33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:36.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.422+0000 7fccd6ffd640 1 --2- 172.21.15.154:0/3215157699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fccd80b2fe0 0x7fccd81f33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.422+0000 7fccd6ffd640 1 -- 172.21.15.154:0/3215157699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccd80cf130 msgr2=0x7fccd81f3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:36.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.422+0000 7fccd6ffd640 1 --2- 172.21.15.154:0/3215157699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccd80cf130 0x7fccd81f3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.422+0000 7fccd6ffd640 1 -- 172.21.15.154:0/3215157699 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcccc00b6a0 con 0x7fccd80ada30 2023-09-23T03:19:36.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.422+0000 7fccd6ffd640 1 --2- 172.21.15.154:0/3215157699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccd80ada30 0x7fccd815c5e0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fccc8004640 tx=0x7fccc800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:36.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.422+0000 7fccbffff640 1 -- 172.21.15.154:0/3215157699 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc8011790 con 0x7fccd80ada30 2023-09-23T03:19:36.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.422+0000 7fccbffff640 1 -- 172.21.15.154:0/3215157699 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc80118f0 con 0x7fccd80ada30 2023-09-23T03:19:36.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.422+0000 7fccbffff640 1 -- 172.21.15.154:0/3215157699 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc8011ce0 con 0x7fccd80ada30 2023-09-23T03:19:36.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.422+0000 7fccdd939640 1 -- 172.21.15.154:0/3215157699 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccd815adf0 con 0x7fccd80ada30 2023-09-23T03:19:36.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.422+0000 7fccdd939640 1 -- 172.21.15.154:0/3215157699 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd81f5480 con 0x7fccd80ada30 2023-09-23T03:19:36.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.423+0000 7fccdd939640 1 -- 172.21.15.154:0/3215157699 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccd81f51a0 con 0x7fccd80ada30 2023-09-23T03:19:36.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.424+0000 7fccbffff640 1 -- 172.21.15.154:0/3215157699 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fccc801c460 con 0x7fccd80ada30 2023-09-23T03:19:36.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.425+0000 7fccbffff640 1 --2- 172.21.15.154:0/3215157699 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fccc406af30 0x7fccc406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:36.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.425+0000 7fccbffff640 1 -- 172.21.15.154:0/3215157699 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fccc80897f0 con 0x7fccd80ada30 2023-09-23T03:19:36.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.425+0000 7fccd67fc640 1 --2- 172.21.15.154:0/3215157699 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fccc406af30 0x7fccc406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:36.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.426+0000 7fccd67fc640 1 --2- 172.21.15.154:0/3215157699 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fccc406af30 0x7fccc406d3f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcccc038000 tx=0x7fcccc030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:36.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.430+0000 7fccbffff640 1 -- 172.21.15.154:0/3215157699 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fccc80546f0 con 0x7fccd80ada30 2023-09-23T03:19:36.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.658+0000 7fccdd939640 1 -- 172.21.15.154:0/3215157699 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fccd81f4d90 con 0x7fccd80ada30 2023-09-23T03:19:36.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.659+0000 7fccbffff640 1 -- 172.21.15.154:0/3215157699 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v40) v1 ==== 53+0+73 (secure 0 0 0) 0x7fccc8057d10 con 0x7fccd80ada30 2023-09-23T03:19:36.661 INFO:teuthology.orchestra.run.smithi154.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-09-23T03:19:36.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.662+0000 7fccdd939640 1 -- 172.21.15.154:0/3215157699 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fccc406af30 msgr2=0x7fccc406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:36.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.662+0000 7fccdd939640 1 --2- 172.21.15.154:0/3215157699 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fccc406af30 0x7fccc406d3f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcccc038000 tx=0x7fcccc030000 comp rx=0 tx=0).stop 2023-09-23T03:19:36.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.662+0000 7fccdd939640 1 -- 172.21.15.154:0/3215157699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccd80ada30 msgr2=0x7fccd815c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:36.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.662+0000 7fccdd939640 1 --2- 172.21.15.154:0/3215157699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccd80ada30 0x7fccd815c5e0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fccc8004640 tx=0x7fccc800c8e0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.662+0000 7fccdd939640 1 -- 172.21.15.154:0/3215157699 shutdown_connections 2023-09-23T03:19:36.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.662+0000 7fccdd939640 1 --2- 172.21.15.154:0/3215157699 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fccc406af30 0x7fccc406d3f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.662+0000 7fccdd939640 1 --2- 172.21.15.154:0/3215157699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccd80cf130 0x7fccd81f3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.662+0000 7fccdd939640 1 --2- 172.21.15.154:0/3215157699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fccd80b2fe0 0x7fccd81f33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.662+0000 7fccdd939640 1 --2- 172.21.15.154:0/3215157699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccd80ada30 0x7fccd815c5e0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:36.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.662+0000 7fccdd939640 1 -- 172.21.15.154:0/3215157699 >> 172.21.15.154:0/3215157699 conn(0x7fccd806a0f0 msgr2=0x7fccd80aed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:36.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.662+0000 7fccdd939640 1 -- 172.21.15.154:0/3215157699 shutdown_connections 2023-09-23T03:19:36.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:36.662+0000 7fccdd939640 1 -- 172.21.15.154:0/3215157699 wait complete. 2023-09-23T03:19:36.682 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-23T03:19:36.682 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-23T03:19:36.682 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi154.front.sepia.ceph.com 2023-09-23T03:19:36.682 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-09-23T03:19:36.682 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-23T03:19:36.682 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-23T03:19:36.683 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-23T03:19:36.683 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-23T03:19:36.683 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-23T03:19:36.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:19:36.683 DEBUG:teuthology.orchestra.run.smithi154:> ip addr 2023-09-23T03:19:36.733 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-23T03:19:36.733 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-23T03:19:36.733 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2023-09-23T03:19:36.733 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:19:36.734 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2023-09-23T03:19:36.734 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:19:36.734 INFO:teuthology.orchestra.run.smithi154.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:19:36.734 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:54 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:19:36.734 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f0 2023-09-23T03:19:36.734 INFO:teuthology.orchestra.run.smithi154.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:19:36.734 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:55 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:19:36.734 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f1 2023-09-23T03:19:36.734 INFO:teuthology.orchestra.run.smithi154.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:19:36.734 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:60 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:19:36.734 INFO:teuthology.orchestra.run.smithi154.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-23T03:19:36.735 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:61 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:19:36.735 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.21.15.154/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-09-23T03:19:36.735 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft 42148sec preferred_lft 42148sec 2023-09-23T03:19:36.735 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::ec4:7aff:fe88:7761/64 scope link 2023-09-23T03:19:36.735 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:19:36.735 INFO:teuthology.orchestra.run.smithi154.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-23T03:19:36.735 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 5a:c6:b2:96:fb:dc brd ff:ff:ff:ff:ff:ff 2023-09-23T03:19:36.735 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-09-23T03:19:36.735 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:19:36.735 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::a801:22ff:febf:2833/64 scope link 2023-09-23T03:19:36.735 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:19:36.736 INFO:teuthology.orchestra.run.smithi154.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-23T03:19:36.736 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 5a:c6:b2:96:fb:dc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:19:36.736 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::58c6:b2ff:fe96:fbdc/64 scope link 2023-09-23T03:19:36.736 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:19:36.736 INFO:teuthology.orchestra.run.smithi154.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-23T03:19:36.736 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether b2:23:b0:e4:c2:ab brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:19:36.736 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::b023:b0ff:fee4:c2ab/64 scope link 2023-09-23T03:19:36.736 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:19:36.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:19:36.737 DEBUG:teuthology.orchestra.run.smithi154:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-23T03:19:36.832 INFO:teuthology.orchestra.run.smithi154.stdout:1 2023-09-23T03:19:36.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:19:36.836 DEBUG:teuthology.orchestra.run.smithi154:> route 2023-09-23T03:19:36.891 INFO:teuthology.orchestra.run.smithi154.stdout:Kernel IP routing table 2023-09-23T03:19:36.891 INFO:teuthology.orchestra.run.smithi154.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-23T03:19:36.891 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-23T03:19:36.892 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-09-23T03:19:36.892 INFO:teuthology.orchestra.run.smithi154.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-23T03:19:36.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:19:36.893 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-09-23T03:19:36.893 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-09-23T03:19:36.893 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-23T03:19:36.893 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-23T03:19:36.893 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-23T03:19:36.893 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-09-23T03:19:37.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:19:37.077 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2023-09-23T03:19:37.130 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-09-23T03:19:37.130 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-09-23T03:19:37.130 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-23T03:19:37.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:19:37.130 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set brx.0 up 2023-09-23T03:19:37.231 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-09-23T03:19:37.232 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-23T03:19:37.232 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-09-23T03:19:37.287 INFO:teuthology.orchestra.run.smithi154.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-09-23T03:19:37.288 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-23T03:19:37.288 DEBUG:teuthology.orchestra.run.smithi154:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-09-23T03:19:37.338 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-09-23T03:19:37.429 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2023-09-23T03:19:37.484 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2023-09-23T03:19:37.484 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2023-09-23T03:19:37.484 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2023-09-23T03:19:37.484 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2023-09-23T03:19:37.484 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2023-09-23T03:19:37.485 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2023-09-23T03:19:37.485 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2023-09-23T03:19:37.485 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2023-09-23T03:19:37.485 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2023-09-23T03:19:37.485 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2023-09-23T03:19:37.485 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2023-09-23T03:19:37.485 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T03:19:37.485 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2023-09-23T03:19:37.485 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2023-09-23T03:19:37.485 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2023-09-23T03:19:37.485 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2023-09-23T03:19:37.486 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2023-09-23T03:19:37.486 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2023-09-23T03:19:37.486 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2023-09-23T03:19:37.486 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-09-23T03:19:37.486 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-09-23T03:19:37.486 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2023-09-23T03:19:37.486 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2023-09-23T03:19:37.486 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2023-09-23T03:19:37.486 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-0 2023-09-23T03:19:37.486 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-1 2023-09-23T03:19:37.486 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-2 2023-09-23T03:19:37.487 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-3 2023-09-23T03:19:37.487 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2023-09-23T03:19:37.487 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:19:37.487 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:19:37.487 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:19:37.487 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:19:37.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:19:37.488 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-09-23T03:19:37.538 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-23T03:19:37.539 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-09-23T03:19:37.544 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-09-23T03:19:37.624 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2023-09-23T03:19:37.665 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2023-09-23T03:19:37.666 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2023-09-23T03:19:37.666 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2023-09-23T03:19:37.666 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2023-09-23T03:19:37.666 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2023-09-23T03:19:37.666 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2023-09-23T03:19:37.666 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2023-09-23T03:19:37.666 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2023-09-23T03:19:37.666 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2023-09-23T03:19:37.666 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2023-09-23T03:19:37.666 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2023-09-23T03:19:37.667 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T03:19:37.667 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2023-09-23T03:19:37.667 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2023-09-23T03:19:37.667 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2023-09-23T03:19:37.667 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2023-09-23T03:19:37.667 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2023-09-23T03:19:37.667 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2023-09-23T03:19:37.667 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2023-09-23T03:19:37.667 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-09-23T03:19:37.667 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-09-23T03:19:37.667 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2023-09-23T03:19:37.667 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2023-09-23T03:19:37.668 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2023-09-23T03:19:37.668 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-0 2023-09-23T03:19:37.668 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-1 2023-09-23T03:19:37.668 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-2 2023-09-23T03:19:37.668 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-3 2023-09-23T03:19:37.668 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2023-09-23T03:19:37.668 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:19:37.668 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:19:37.668 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:19:37.668 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:19:37.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:19:37.669 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-09-23T03:19:37.741 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:2023-09-23T03:19:37.737+0000 7f90ef838480 -1 init, newargv = 0x56109946ca60 newargc=15 2023-09-23T03:19:37.741 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:ceph-fuse[108844]: starting ceph client 2023-09-23T03:19:37.747 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:ceph-fuse[108844]: starting fuse 2023-09-23T03:19:37.761 INFO:teuthology.orchestra.run.smithi154.stdout:39 2023-09-23T03:19:37.762 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-09-23T03:19:37.762 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:19:37.762 DEBUG:teuthology.orchestra.run.smithi154:> import glob 2023-09-23T03:19:37.762 DEBUG:teuthology.orchestra.run.smithi154:> import re 2023-09-23T03:19:37.762 DEBUG:teuthology.orchestra.run.smithi154:> import os 2023-09-23T03:19:37.762 DEBUG:teuthology.orchestra.run.smithi154:> import subprocess 2023-09-23T03:19:37.762 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:19:37.762 DEBUG:teuthology.orchestra.run.smithi154:> def _find_admin_socket(client_name): 2023-09-23T03:19:37.763 DEBUG:teuthology.orchestra.run.smithi154:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-09-23T03:19:37.763 DEBUG:teuthology.orchestra.run.smithi154:> files = glob.glob(asok_path) 2023-09-23T03:19:37.763 DEBUG:teuthology.orchestra.run.smithi154:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-09-23T03:19:37.763 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:19:37.763 DEBUG:teuthology.orchestra.run.smithi154:> # Given a non-glob path, it better be there 2023-09-23T03:19:37.763 DEBUG:teuthology.orchestra.run.smithi154:> if "*" not in asok_path: 2023-09-23T03:19:37.763 DEBUG:teuthology.orchestra.run.smithi154:> assert(len(files) == 1) 2023-09-23T03:19:37.763 DEBUG:teuthology.orchestra.run.smithi154:> return files[0] 2023-09-23T03:19:37.763 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:19:37.763 DEBUG:teuthology.orchestra.run.smithi154:> for f in files: 2023-09-23T03:19:37.763 DEBUG:teuthology.orchestra.run.smithi154:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-23T03:19:37.763 DEBUG:teuthology.orchestra.run.smithi154:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-23T03:19:37.764 DEBUG:teuthology.orchestra.run.smithi154:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-23T03:19:37.764 DEBUG:teuthology.orchestra.run.smithi154:> contents = proc_f.read() 2023-09-23T03:19:37.764 DEBUG:teuthology.orchestra.run.smithi154:> if mountpoint in contents: 2023-09-23T03:19:37.764 DEBUG:teuthology.orchestra.run.smithi154:> return f 2023-09-23T03:19:37.764 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-23T03:19:37.764 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:19:37.764 DEBUG:teuthology.orchestra.run.smithi154:> print(_find_admin_socket("client.0")) 2023-09-23T03:19:37.764 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:19:37.962 INFO:teuthology.orchestra.run.smithi154.stdout:/var/run/ceph/ceph-client.0.108844.asok 2023-09-23T03:19:37.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:19:37.979 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.108844.asok 2023-09-23T03:19:37.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:19:37.979 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.108844.asok status 2023-09-23T03:19:38.221 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:19:38.221 INFO:teuthology.orchestra.run.smithi154.stdout: "metadata": { 2023-09-23T03:19:38.221 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_sha1": "862663913a74a99e1c52d50e6a7e59dd6705749a", 2023-09-23T03:19:38.221 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_version": "ceph version 18.0.0-6336-g86266391 (862663913a74a99e1c52d50e6a7e59dd6705749a) reef (dev)", 2023-09-23T03:19:38.222 INFO:teuthology.orchestra.run.smithi154.stdout: "entity_id": "0", 2023-09-23T03:19:38.222 INFO:teuthology.orchestra.run.smithi154.stdout: "hostname": "smithi154", 2023-09-23T03:19:38.222 INFO:teuthology.orchestra.run.smithi154.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-09-23T03:19:38.222 INFO:teuthology.orchestra.run.smithi154.stdout: "pid": "108844", 2023-09-23T03:19:38.222 INFO:teuthology.orchestra.run.smithi154.stdout: "root": "/" 2023-09-23T03:19:38.222 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:19:38.222 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_count": 0, 2023-09-23T03:19:38.222 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_pinned_count": 0, 2023-09-23T03:19:38.222 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 6489, 2023-09-23T03:19:38.222 INFO:teuthology.orchestra.run.smithi154.stdout: "inst": { 2023-09-23T03:19:38.222 INFO:teuthology.orchestra.run.smithi154.stdout: "name": { 2023-09-23T03:19:38.223 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "client", 2023-09-23T03:19:38.223 INFO:teuthology.orchestra.run.smithi154.stdout: "num": 6489 2023-09-23T03:19:38.223 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:19:38.223 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2023-09-23T03:19:38.223 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2023-09-23T03:19:38.223 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.1:0", 2023-09-23T03:19:38.223 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 1031151278 2023-09-23T03:19:38.223 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:19:38.223 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:19:38.223 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2023-09-23T03:19:38.223 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2023-09-23T03:19:38.223 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.1:0", 2023-09-23T03:19:38.223 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 1031151278 2023-09-23T03:19:38.224 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:19:38.224 INFO:teuthology.orchestra.run.smithi154.stdout: "inst_str": "client.6489 192.168.0.1:0/1031151278", 2023-09-23T03:19:38.224 INFO:teuthology.orchestra.run.smithi154.stdout: "addr_str": "192.168.0.1:0/1031151278", 2023-09-23T03:19:38.224 INFO:teuthology.orchestra.run.smithi154.stdout: "inode_count": 1, 2023-09-23T03:19:38.224 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_epoch": 40, 2023-09-23T03:19:38.224 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch": 62, 2023-09-23T03:19:38.224 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch_barrier": 0, 2023-09-23T03:19:38.224 INFO:teuthology.orchestra.run.smithi154.stdout: "blocklisted": false, 2023-09-23T03:19:38.224 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs" 2023-09-23T03:19:38.224 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:19:38.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:19:38.236 DEBUG:teuthology.orchestra.run.smithi154:> stat --file-system '--printf=%T 2023-09-23T03:19:38.236 DEBUG:teuthology.orchestra.run.smithi154:> ' -- /home/ubuntu/cephtest/mnt.0 2023-09-23T03:19:38.291 INFO:teuthology.orchestra.run.smithi154.stdout:fuseblk 2023-09-23T03:19:38.291 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-09-23T03:19:38.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:19:38.291 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-09-23T03:19:38.387 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_extra --pg_num_min 64 2023-09-23T03:19:38.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.606+0000 7f4502961640 1 Processor -- start 2023-09-23T03:19:38.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.606+0000 7f4502961640 1 -- start start 2023-09-23T03:19:38.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.608+0000 7f4502961640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f44fc0ce3c0 0x7f44fc0ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:38.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.608+0000 7f4502961640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44fc0ced50 con 0x7f44fc1572c0 2023-09-23T03:19:38.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.608+0000 7f4502961640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44fc0721d0 con 0x7f44fc1576a0 2023-09-23T03:19:38.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.608+0000 7f4502961640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44fc071650 con 0x7f44fc0ce3c0 2023-09-23T03:19:38.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.608+0000 7f44fbfff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f44fc1572c0 0x7f44fc159e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:59032/0 (socket says 172.21.15.154:59032) 2023-09-23T03:19:38.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.608+0000 7f44fbfff640 1 -- 172.21.15.154:0/1514528558 learned_addr learned my addr 172.21.15.154:0/1514528558 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:19:38.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.608+0000 7f4500efe640 1 --2- 172.21.15.154:0/1514528558 >> v2:172.21.15.154:3300/0 conn(0x7f44fc0ce3c0 0x7f44fc0ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:38.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.609+0000 7f44fb7fe640 1 -- 172.21.15.154:0/1514528558 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1378452879 0 0) 0x7f44ec001110 con 0x7f44fc1572c0 2023-09-23T03:19:38.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.609+0000 7f44fb7fe640 1 -- 172.21.15.154:0/1514528558 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44e8005730 con 0x7f44fc1572c0 2023-09-23T03:19:38.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.609+0000 7f4500efe640 1 -- 172.21.15.154:0/1514528558 >> v1:172.21.15.190:6789/0 conn(0x7f44fc1576a0 legacy=0x7f44fc0b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:38.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.609+0000 7f4500efe640 1 -- 172.21.15.154:0/1514528558 >> v1:172.21.15.154:6789/0 conn(0x7f44fc1572c0 legacy=0x7f44fc159e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:38.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.609+0000 7f4500efe640 1 -- 172.21.15.154:0/1514528558 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44fc071930 con 0x7f44fc0ce3c0 2023-09-23T03:19:38.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.610+0000 7f4500efe640 1 --2- 172.21.15.154:0/1514528558 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44fc0ce3c0 0x7f44fc0ce810 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f44f4009d90 tx=0x7f44f402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ddf67167455b6e4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:38.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.610+0000 7f44fb7fe640 1 -- 172.21.15.154:0/1514528558 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f402cbd0 con 0x7f44fc0ce3c0 2023-09-23T03:19:38.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.610+0000 7f44fb7fe640 1 -- 172.21.15.154:0/1514528558 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f402cd30 con 0x7f44fc0ce3c0 2023-09-23T03:19:38.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.610+0000 7f44fb7fe640 1 -- 172.21.15.154:0/1514528558 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f402cea0 con 0x7f44fc0ce3c0 2023-09-23T03:19:38.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.611+0000 7f4502961640 1 -- 172.21.15.154:0/1514528558 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44fc0ce3c0 msgr2=0x7f44fc0ce810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:38.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.611+0000 7f4502961640 1 --2- 172.21.15.154:0/1514528558 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44fc0ce3c0 0x7f44fc0ce810 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f44f4009d90 tx=0x7f44f402be20 comp rx=0 tx=0).stop 2023-09-23T03:19:38.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.611+0000 7f4502961640 1 -- 172.21.15.154:0/1514528558 shutdown_connections 2023-09-23T03:19:38.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.611+0000 7f4502961640 1 --2- 172.21.15.154:0/1514528558 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44fc0ce3c0 0x7f44fc0ce810 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:38.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.611+0000 7f4502961640 1 -- 172.21.15.154:0/1514528558 >> 172.21.15.154:0/1514528558 conn(0x7f44fc06a520 msgr2=0x7f44fc06bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:38.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.611+0000 7f4502961640 1 -- 172.21.15.154:0/1514528558 shutdown_connections 2023-09-23T03:19:38.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.611+0000 7f4502961640 1 -- 172.21.15.154:0/1514528558 wait complete. 2023-09-23T03:19:38.615 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.614+0000 7f4502961640 1 Processor -- start 2023-09-23T03:19:38.615 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.614+0000 7f4502961640 1 -- start start 2023-09-23T03:19:38.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.615+0000 7f4502961640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44fc0ce3c0 0x7f44fc0c0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:38.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.615+0000 7f4502961640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44fc1572c0 0x7f44fc1510a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:38.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.615+0000 7f4502961640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44fc1576a0 0x7f44fc1515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:38.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.615+0000 7f4502961640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44fc0c2430 con 0x7f44fc1572c0 2023-09-23T03:19:38.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.615+0000 7f4502961640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44fc1540b0 con 0x7f44fc0ce3c0 2023-09-23T03:19:38.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.615+0000 7f4502961640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44fc154390 con 0x7f44fc1576a0 2023-09-23T03:19:38.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.615+0000 7f44f37fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44fc1572c0 0x7f44fc1510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:38.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.616+0000 7f44f37fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44fc1572c0 0x7f44fc1510a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57570/0 (socket says 172.21.15.154:57570) 2023-09-23T03:19:38.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.616+0000 7f44f37fe640 1 -- 172.21.15.154:0/3141112647 learned_addr learned my addr 172.21.15.154:0/3141112647 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:38.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.616+0000 7f44fbfff640 1 --2- 172.21.15.154:0/3141112647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44fc0ce3c0 0x7f44fc0c0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:38.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.616+0000 7f4500efe640 1 --2- 172.21.15.154:0/3141112647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44fc1576a0 0x7f44fc1515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:38.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.616+0000 7f44f37fe640 1 -- 172.21.15.154:0/3141112647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44fc1576a0 msgr2=0x7f44fc1515e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:38.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.616+0000 7f44f37fe640 1 --2- 172.21.15.154:0/3141112647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44fc1576a0 0x7f44fc1515e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:38.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.616+0000 7f44f37fe640 1 -- 172.21.15.154:0/3141112647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44fc0ce3c0 msgr2=0x7f44fc0c0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:38.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.616+0000 7f44f37fe640 1 --2- 172.21.15.154:0/3141112647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44fc0ce3c0 0x7f44fc0c0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:38.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.616+0000 7f44f37fe640 1 -- 172.21.15.154:0/3141112647 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f4009a70 con 0x7f44fc1572c0 2023-09-23T03:19:38.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.617+0000 7f44f37fe640 1 --2- 172.21.15.154:0/3141112647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44fc1572c0 0x7f44fc1510a0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f44e400c430 tx=0x7f44e400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:38.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.617+0000 7f44f97fa640 1 -- 172.21.15.154:0/3141112647 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e4011790 con 0x7f44fc1572c0 2023-09-23T03:19:38.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.617+0000 7f44f97fa640 1 -- 172.21.15.154:0/3141112647 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e40118f0 con 0x7f44fc1572c0 2023-09-23T03:19:38.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.617+0000 7f44f97fa640 1 -- 172.21.15.154:0/3141112647 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e4011ce0 con 0x7f44fc1572c0 2023-09-23T03:19:38.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.617+0000 7f4502961640 1 -- 172.21.15.154:0/3141112647 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44fc0c1de0 con 0x7f44fc1572c0 2023-09-23T03:19:38.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.617+0000 7f4502961640 1 -- 172.21.15.154:0/3141112647 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44fc0c10e0 con 0x7f44fc1572c0 2023-09-23T03:19:38.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.618+0000 7f4502961640 1 -- 172.21.15.154:0/3141112647 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44fc0c25e0 con 0x7f44fc1572c0 2023-09-23T03:19:38.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.619+0000 7f44f97fa640 1 -- 172.21.15.154:0/3141112647 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f44e401c460 con 0x7f44fc1572c0 2023-09-23T03:19:38.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.620+0000 7f44f97fa640 1 --2- 172.21.15.154:0/3141112647 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f44dc06af30 0x7f44dc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:38.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.621+0000 7f44f97fa640 1 -- 172.21.15.154:0/3141112647 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f44e4089840 con 0x7f44fc1572c0 2023-09-23T03:19:38.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.621+0000 7f44fbfff640 1 --2- 172.21.15.154:0/3141112647 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f44dc06af30 0x7f44dc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:38.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.621+0000 7f44fbfff640 1 --2- 172.21.15.154:0/3141112647 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f44dc06af30 0x7f44dc06d3f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f44ec001270 tx=0x7f44ec007bb0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:38.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.625+0000 7f44f97fa640 1 -- 172.21.15.154:0/3141112647 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f44e40546a0 con 0x7f44fc1572c0 2023-09-23T03:19:38.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.815+0000 7f4502961640 1 -- 172.21.15.154:0/3141112647 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_extra", "pg_num_min": 64} v 0) v1 -- 0x7f44fc04e920 con 0x7f44fc1572c0 2023-09-23T03:19:38.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.925+0000 7f44f97fa640 1 -- 172.21.15.154:0/3141112647 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_extra", "pg_num_min": 64}]=0 pool 'cephfs_data_extra' created v63) v1 ==== 142+0+0 (secure 0 0 0) 0x7f44e4057cc0 con 0x7f44fc1572c0 2023-09-23T03:19:38.926 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_data_extra' created 2023-09-23T03:19:38.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.928+0000 7f4502961640 1 -- 172.21.15.154:0/3141112647 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f44dc06af30 msgr2=0x7f44dc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:38.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.928+0000 7f4502961640 1 --2- 172.21.15.154:0/3141112647 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f44dc06af30 0x7f44dc06d3f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f44ec001270 tx=0x7f44ec007bb0 comp rx=0 tx=0).stop 2023-09-23T03:19:38.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.928+0000 7f4502961640 1 -- 172.21.15.154:0/3141112647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44fc1572c0 msgr2=0x7f44fc1510a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:38.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.928+0000 7f4502961640 1 --2- 172.21.15.154:0/3141112647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44fc1572c0 0x7f44fc1510a0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f44e400c430 tx=0x7f44e400c8f0 comp rx=0 tx=0).stop 2023-09-23T03:19:38.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.929+0000 7f4502961640 1 -- 172.21.15.154:0/3141112647 shutdown_connections 2023-09-23T03:19:38.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.929+0000 7f4502961640 1 --2- 172.21.15.154:0/3141112647 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f44dc06af30 0x7f44dc06d3f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:38.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.929+0000 7f4502961640 1 --2- 172.21.15.154:0/3141112647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44fc1576a0 0x7f44fc1515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:38.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.929+0000 7f4502961640 1 --2- 172.21.15.154:0/3141112647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44fc1572c0 0x7f44fc1510a0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:38.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.929+0000 7f4502961640 1 --2- 172.21.15.154:0/3141112647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44fc0ce3c0 0x7f44fc0c0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:38.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.929+0000 7f4502961640 1 -- 172.21.15.154:0/3141112647 >> 172.21.15.154:0/3141112647 conn(0x7f44fc06a520 msgr2=0x7f44fc158930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:38.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.929+0000 7f4502961640 1 -- 172.21.15.154:0/3141112647 shutdown_connections 2023-09-23T03:19:38.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:38.930+0000 7f4502961640 1 -- 172.21.15.154:0/3141112647 wait complete. 2023-09-23T03:19:38.956 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_extra 2023-09-23T03:19:39.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.161+0000 7f057ffff640 1 Processor -- start 2023-09-23T03:19:39.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.161+0000 7f057ffff640 1 -- start start 2023-09-23T03:19:39.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.163+0000 7f057ffff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f05801582f0 0x7f058015acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:39.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.163+0000 7f057ffff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05800705a0 0x7f058015e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:39.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.163+0000 7f057ffff640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05800c9e80 con 0x7f05800b08c0 2023-09-23T03:19:39.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.163+0000 7f057ffff640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05801586d0 con 0x7f05801582f0 2023-09-23T03:19:39.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.163+0000 7f057ffff640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05800c9300 con 0x7f05800705a0 2023-09-23T03:19:39.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.163+0000 7f05767fc640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f05800b08c0 0x7f058006fe90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:59044/0 (socket says 172.21.15.154:59044) 2023-09-23T03:19:39.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.163+0000 7f05767fc640 1 -- 172.21.15.154:0/1249468282 learned_addr learned my addr 172.21.15.154:0/1249468282 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:19:39.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.163+0000 7f057e7fc640 1 -- 172.21.15.154:0/1249468282 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1157161207 0 0) 0x7f05800c9e80 con 0x7f05800b08c0 2023-09-23T03:19:39.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.163+0000 7f057effd640 1 --2- 172.21.15.154:0/1249468282 >> v2:172.21.15.190:3300/0 conn(0x7f05801582f0 0x7f058015acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:39.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.164+0000 7f057e7fc640 1 -- 172.21.15.154:0/1249468282 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f056c003520 con 0x7f05800b08c0 2023-09-23T03:19:39.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.163+0000 7f057f7fe640 1 --2- 172.21.15.154:0/1249468282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05800705a0 0x7f058015e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:39.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.164+0000 7f057e7fc640 1 -- 172.21.15.154:0/1249468282 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1387520712 0 0) 0x7f056c003520 con 0x7f05800b08c0 2023-09-23T03:19:39.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.164+0000 7f057e7fc640 1 -- 172.21.15.154:0/1249468282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05800705a0 msgr2=0x7f058015e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:39.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.164+0000 7f057e7fc640 1 --2- 172.21.15.154:0/1249468282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05800705a0 0x7f058015e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:39.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.164+0000 7f057e7fc640 1 -- 172.21.15.154:0/1249468282 >> v2:172.21.15.190:3300/0 conn(0x7f05801582f0 msgr2=0x7f058015acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:39.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.164+0000 7f057e7fc640 1 --2- 172.21.15.154:0/1249468282 >> v2:172.21.15.190:3300/0 conn(0x7f05801582f0 0x7f058015acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:39.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.164+0000 7f057e7fc640 1 -- 172.21.15.154:0/1249468282 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05800c95e0 con 0x7f05800b08c0 2023-09-23T03:19:39.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.164+0000 7f057e7fc640 1 -- 172.21.15.154:0/1249468282 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f0568001670 con 0x7f05800b08c0 2023-09-23T03:19:39.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.164+0000 7f057e7fc640 1 -- 172.21.15.154:0/1249468282 >> v1:172.21.15.154:6789/0 conn(0x7f05800b08c0 legacy=0x7f058006fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:39.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.164+0000 7f057e7fc640 1 --2- 172.21.15.154:0/1249468282 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f056c003970 0x7f056c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:39.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.164+0000 7f057e7fc640 1 --2- 172.21.15.154:0/1249468282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f056c0047c0 0x7f056c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:39.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.164+0000 7f057e7fc640 1 --2- 172.21.15.154:0/1249468282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05800705a0 0x7f056c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:39.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.164+0000 7f057e7fc640 1 -- 172.21.15.154:0/1249468282 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05801586d0 con 0x7f056c003970 2023-09-23T03:19:39.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.164+0000 7f057e7fc640 1 -- 172.21.15.154:0/1249468282 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05800c9300 con 0x7f056c0047c0 2023-09-23T03:19:39.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.164+0000 7f057e7fc640 1 -- 172.21.15.154:0/1249468282 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f056c00ba60 con 0x7f05800705a0 2023-09-23T03:19:39.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.164+0000 7f057effd640 1 --2- 172.21.15.154:0/1249468282 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f056c003970 0x7f056c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:39.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.165+0000 7f05767fc640 1 --2- 172.21.15.154:0/1249468282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f056c0047c0 0x7f056c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:39.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.165+0000 7f057effd640 1 -- 172.21.15.154:0/1249468282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05800705a0 msgr2=0x7f056c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:39.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.165+0000 7f057effd640 1 --2- 172.21.15.154:0/1249468282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05800705a0 0x7f056c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:39.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.165+0000 7f057effd640 1 -- 172.21.15.154:0/1249468282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f056c0047c0 msgr2=0x7f056c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:39.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.165+0000 7f057effd640 1 --2- 172.21.15.154:0/1249468282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f056c0047c0 0x7f056c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:39.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.165+0000 7f057effd640 1 -- 172.21.15.154:0/1249468282 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056c003520 con 0x7f056c003970 2023-09-23T03:19:39.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.165+0000 7f05767fc640 1 --2- 172.21.15.154:0/1249468282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f056c0047c0 0x7f056c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:39.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.166+0000 7f057effd640 1 --2- 172.21.15.154:0/1249468282 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f056c003970 0x7f056c003d90 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f0570009f90 tx=0x7f056c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=704b9c6ea6e97edf server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:39.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.166+0000 7f057e7fc640 1 -- 172.21.15.154:0/1249468282 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057002cb30 con 0x7f056c003970 2023-09-23T03:19:39.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.166+0000 7f057e7fc640 1 -- 172.21.15.154:0/1249468282 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f057000f040 con 0x7f056c003970 2023-09-23T03:19:39.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.166+0000 7f057e7fc640 1 -- 172.21.15.154:0/1249468282 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0570034440 con 0x7f056c003970 2023-09-23T03:19:39.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.167+0000 7f057ffff640 1 -- 172.21.15.154:0/1249468282 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f056c003970 msgr2=0x7f056c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:39.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.167+0000 7f057ffff640 1 --2- 172.21.15.154:0/1249468282 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f056c003970 0x7f056c003d90 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f0570009f90 tx=0x7f056c0057f0 comp rx=0 tx=0).stop 2023-09-23T03:19:39.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.167+0000 7f057f7fe640 1 -- 172.21.15.154:0/1249468282 reap_dead start 2023-09-23T03:19:39.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.167+0000 7f057ffff640 1 -- 172.21.15.154:0/1249468282 shutdown_connections 2023-09-23T03:19:39.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.167+0000 7f057ffff640 1 -- 172.21.15.154:0/1249468282 >> 172.21.15.154:0/1249468282 conn(0x7f058006a300 msgr2=0x7f05800caeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:39.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.167+0000 7f057ffff640 1 -- 172.21.15.154:0/1249468282 shutdown_connections 2023-09-23T03:19:39.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.167+0000 7f057ffff640 1 -- 172.21.15.154:0/1249468282 wait complete. 2023-09-23T03:19:39.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.169+0000 7f057ffff640 1 Processor -- start 2023-09-23T03:19:39.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.169+0000 7f057ffff640 1 -- start start 2023-09-23T03:19:39.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.170+0000 7f057ffff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f056c003970 0x7f05800c1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:39.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.170+0000 7f057ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05800cd890 0x7f058014c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:39.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.170+0000 7f057ffff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05800aca90 0x7f05800acf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:39.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.170+0000 7f057ffff640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05800ce0c0 con 0x7f05800cd890 2023-09-23T03:19:39.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.170+0000 7f057ffff640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05800ad450 con 0x7f056c003970 2023-09-23T03:19:39.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.170+0000 7f057ffff640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05800cdde0 con 0x7f05800aca90 2023-09-23T03:19:39.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.170+0000 7f05767fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05800cd890 0x7f058014c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:39.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.170+0000 7f05767fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05800cd890 0x7f058014c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57588/0 (socket says 172.21.15.154:57588) 2023-09-23T03:19:39.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.170+0000 7f05767fc640 1 -- 172.21.15.154:0/1371524326 learned_addr learned my addr 172.21.15.154:0/1371524326 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:39.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.170+0000 7f057effd640 1 --2- 172.21.15.154:0/1371524326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f056c003970 0x7f05800c1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:39.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.170+0000 7f05767fc640 1 -- 172.21.15.154:0/1371524326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05800aca90 msgr2=0x7f05800acf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:39.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.170+0000 7f05767fc640 1 --2- 172.21.15.154:0/1371524326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05800aca90 0x7f05800acf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:39.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.170+0000 7f05767fc640 1 -- 172.21.15.154:0/1371524326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f056c003970 msgr2=0x7f05800c1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:39.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.170+0000 7f05767fc640 1 --2- 172.21.15.154:0/1371524326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f056c003970 0x7f05800c1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:39.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.170+0000 7f05767fc640 1 -- 172.21.15.154:0/1371524326 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05800c4700 con 0x7f05800cd890 2023-09-23T03:19:39.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.171+0000 7f05767fc640 1 --2- 172.21.15.154:0/1371524326 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05800cd890 0x7f058014c460 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f0568000f60 tx=0x7f056800df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:39.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.171+0000 7f0577fff640 1 -- 172.21.15.154:0/1371524326 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0568007560 con 0x7f05800cd890 2023-09-23T03:19:39.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.171+0000 7f0577fff640 1 -- 172.21.15.154:0/1371524326 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05680076c0 con 0x7f05800cd890 2023-09-23T03:19:39.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.171+0000 7f0577fff640 1 -- 172.21.15.154:0/1371524326 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05680049e0 con 0x7f05800cd890 2023-09-23T03:19:39.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.171+0000 7f057ffff640 1 -- 172.21.15.154:0/1371524326 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05800c49e0 con 0x7f05800cd890 2023-09-23T03:19:39.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.171+0000 7f057ffff640 1 -- 172.21.15.154:0/1371524326 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05800c24a0 con 0x7f05800cd890 2023-09-23T03:19:39.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.172+0000 7f057ffff640 1 -- 172.21.15.154:0/1371524326 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05800c4540 con 0x7f05800cd890 2023-09-23T03:19:39.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.174+0000 7f0577fff640 1 -- 172.21.15.154:0/1371524326 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f0568004b40 con 0x7f05800cd890 2023-09-23T03:19:39.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.175+0000 7f0577fff640 1 --2- 172.21.15.154:0/1371524326 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f056006af30 0x7f056006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:39.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.175+0000 7f0577fff640 1 -- 172.21.15.154:0/1371524326 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6401+0+0 (secure 0 0 0) 0x7f0568089930 con 0x7f05800cd890 2023-09-23T03:19:39.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.175+0000 7f057effd640 1 --2- 172.21.15.154:0/1371524326 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f056006af30 0x7f056006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:39.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.176+0000 7f057effd640 1 --2- 172.21.15.154:0/1371524326 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f056006af30 0x7f056006d3f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0570036000 tx=0x7f057002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:39.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.178+0000 7f0577fff640 1 -- 172.21.15.154:0/1371524326 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f0568054720 con 0x7f05800cd890 2023-09-23T03:19:39.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.408+0000 7f057ffff640 1 -- 172.21.15.154:0/1371524326 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_extra"} v 0) v1 -- 0x7f0580067390 con 0x7f05800cd890 2023-09-23T03:19:39.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.965+0000 7f0577fff640 1 -- 172.21.15.154:0/1371524326 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_extra"}]=0 added data pool 10 to fsmap v41) v1 ==== 141+0+0 (secure 0 0 0) 0x7f0568057d40 con 0x7f05800cd890 2023-09-23T03:19:39.972 INFO:teuthology.orchestra.run.smithi154.stderr:added data pool 10 to fsmap 2023-09-23T03:19:39.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.972+0000 7f057ffff640 1 -- 172.21.15.154:0/1371524326 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f056006af30 msgr2=0x7f056006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:39.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.972+0000 7f057ffff640 1 --2- 172.21.15.154:0/1371524326 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f056006af30 0x7f056006d3f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0570036000 tx=0x7f057002e000 comp rx=0 tx=0).stop 2023-09-23T03:19:39.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.972+0000 7f057ffff640 1 -- 172.21.15.154:0/1371524326 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05800cd890 msgr2=0x7f058014c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:39.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.972+0000 7f057ffff640 1 --2- 172.21.15.154:0/1371524326 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05800cd890 0x7f058014c460 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f0568000f60 tx=0x7f056800df70 comp rx=0 tx=0).stop 2023-09-23T03:19:39.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.974+0000 7f057ffff640 1 -- 172.21.15.154:0/1371524326 shutdown_connections 2023-09-23T03:19:39.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.975+0000 7f057ffff640 1 --2- 172.21.15.154:0/1371524326 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f056006af30 0x7f056006d3f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:39.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.975+0000 7f057ffff640 1 --2- 172.21.15.154:0/1371524326 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05800aca90 0x7f05800acf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:39.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.975+0000 7f057ffff640 1 --2- 172.21.15.154:0/1371524326 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05800cd890 0x7f058014c460 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:39.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.975+0000 7f057ffff640 1 --2- 172.21.15.154:0/1371524326 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f056c003970 0x7f05800c1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:39.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.975+0000 7f057ffff640 1 -- 172.21.15.154:0/1371524326 >> 172.21.15.154:0/1371524326 conn(0x7f058006a300 msgr2=0x7f058014d900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:39.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.975+0000 7f057ffff640 1 -- 172.21.15.154:0/1371524326 shutdown_connections 2023-09-23T03:19:39.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:39.975+0000 7f057ffff640 1 -- 172.21.15.154:0/1371524326 wait complete. 2023-09-23T03:19:40.000 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:19:40.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.186+0000 7f71ff221640 1 Processor -- start 2023-09-23T03:19:40.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.187+0000 7f71ff221640 1 -- start start 2023-09-23T03:19:40.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.188+0000 7f71ff221640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f71f80b1b20 0x7f71f80ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:40.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.188+0000 7f71ff221640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71f815ac30 con 0x7f71f80cee70 2023-09-23T03:19:40.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.188+0000 7f71ff221640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71f80b1f70 con 0x7f71f80acce0 2023-09-23T03:19:40.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.188+0000 7f71ff221640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71f80b2e90 con 0x7f71f80b1b20 2023-09-23T03:19:40.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.188+0000 7f71f7fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f71f80b1b20 0x7f71f80ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:40.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.188+0000 7f71fd7be640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f71f80cee70 0x7f71f80ab140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:59046/0 (socket says 172.21.15.154:59046) 2023-09-23T03:19:40.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.189+0000 7f71fd7be640 1 -- 172.21.15.154:0/1712123217 learned_addr learned my addr 172.21.15.154:0/1712123217 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:19:40.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.189+0000 7f71f7fff640 1 -- 172.21.15.154:0/1712123217 >> v1:172.21.15.190:6789/0 conn(0x7f71f80acce0 legacy=0x7f71f815ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:40.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.189+0000 7f71f7fff640 1 -- 172.21.15.154:0/1712123217 >> v1:172.21.15.154:6789/0 conn(0x7f71f80cee70 legacy=0x7f71f80ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:40.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.189+0000 7f71f7fff640 1 -- 172.21.15.154:0/1712123217 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f80b2210 con 0x7f71f80b1b20 2023-09-23T03:19:40.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.190+0000 7f71f7fff640 1 --2- 172.21.15.154:0/1712123217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71f80b1b20 0x7f71f80ce930 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f71e40097a0 tx=0x7f71e402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c88443b9a4edcad server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:40.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.190+0000 7f71f77fe640 1 -- 172.21.15.154:0/1712123217 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e402e070 con 0x7f71f80b1b20 2023-09-23T03:19:40.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.190+0000 7f71f77fe640 1 -- 172.21.15.154:0/1712123217 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e40315a0 con 0x7f71f80b1b20 2023-09-23T03:19:40.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.190+0000 7f71f77fe640 1 -- 172.21.15.154:0/1712123217 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e4031930 con 0x7f71f80b1b20 2023-09-23T03:19:40.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.191+0000 7f71ff221640 1 -- 172.21.15.154:0/1712123217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71f80b1b20 msgr2=0x7f71f80ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:40.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.191+0000 7f71ff221640 1 --2- 172.21.15.154:0/1712123217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71f80b1b20 0x7f71f80ce930 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f71e40097a0 tx=0x7f71e402b840 comp rx=0 tx=0).stop 2023-09-23T03:19:40.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.191+0000 7f71ff221640 1 -- 172.21.15.154:0/1712123217 shutdown_connections 2023-09-23T03:19:40.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.191+0000 7f71ff221640 1 --2- 172.21.15.154:0/1712123217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71f80b1b20 0x7f71f80ce930 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:40.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.191+0000 7f71ff221640 1 -- 172.21.15.154:0/1712123217 >> 172.21.15.154:0/1712123217 conn(0x7f71f806a0f0 msgr2=0x7f71f806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:40.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.191+0000 7f71ff221640 1 -- 172.21.15.154:0/1712123217 shutdown_connections 2023-09-23T03:19:40.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.191+0000 7f71ff221640 1 -- 172.21.15.154:0/1712123217 wait complete. 2023-09-23T03:19:40.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.193+0000 7f71ff221640 1 Processor -- start 2023-09-23T03:19:40.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.193+0000 7f71ff221640 1 -- start start 2023-09-23T03:19:40.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.194+0000 7f71ff221640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71f80acce0 0x7f71f815ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:40.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.194+0000 7f71ff221640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71f80b1b20 0x7f71f80c7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:40.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.194+0000 7f71ff221640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71f80cee70 0x7f71f8158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:40.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.194+0000 7f71ff221640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f80cf320 con 0x7f71f80b1b20 2023-09-23T03:19:40.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.194+0000 7f71ff221640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f80c7d90 con 0x7f71f80cee70 2023-09-23T03:19:40.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.194+0000 7f71ff221640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f815ba50 con 0x7f71f80acce0 2023-09-23T03:19:40.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.194+0000 7f71f7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71f80b1b20 0x7f71f80c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:40.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.194+0000 7f71f7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71f80b1b20 0x7f71f80c7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57606/0 (socket says 172.21.15.154:57606) 2023-09-23T03:19:40.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.194+0000 7f71f7fff640 1 -- 172.21.15.154:0/1470601019 learned_addr learned my addr 172.21.15.154:0/1470601019 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:40.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.195+0000 7f71fcfbd640 1 --2- 172.21.15.154:0/1470601019 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71f80acce0 0x7f71f815ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:40.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.195+0000 7f71f7fff640 1 -- 172.21.15.154:0/1470601019 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71f80acce0 msgr2=0x7f71f815ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:40.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.195+0000 7f71f7fff640 1 --2- 172.21.15.154:0/1470601019 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71f80acce0 0x7f71f815ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:40.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.195+0000 7f71f7fff640 1 -- 172.21.15.154:0/1470601019 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71f80cee70 msgr2=0x7f71f8158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:40.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.195+0000 7f71fd7be640 1 --2- 172.21.15.154:0/1470601019 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71f80cee70 0x7f71f8158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:40.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.195+0000 7f71f7fff640 1 --2- 172.21.15.154:0/1470601019 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71f80cee70 0x7f71f8158bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:40.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.195+0000 7f71f7fff640 1 -- 172.21.15.154:0/1470601019 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e4009400 con 0x7f71f80b1b20 2023-09-23T03:19:40.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.195+0000 7f71f7fff640 1 --2- 172.21.15.154:0/1470601019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71f80b1b20 0x7f71f80c7850 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f71e4034040 tx=0x7f71e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:40.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.196+0000 7f71f57fa640 1 -- 172.21.15.154:0/1470601019 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e4032900 con 0x7f71f80b1b20 2023-09-23T03:19:40.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.196+0000 7f71f57fa640 1 -- 172.21.15.154:0/1470601019 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e4032a80 con 0x7f71f80b1b20 2023-09-23T03:19:40.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.196+0000 7f71ff221640 1 -- 172.21.15.154:0/1470601019 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f80c8030 con 0x7f71f80b1b20 2023-09-23T03:19:40.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.196+0000 7f71f57fa640 1 -- 172.21.15.154:0/1470601019 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e4032e30 con 0x7f71f80b1b20 2023-09-23T03:19:40.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.196+0000 7f71ff221640 1 -- 172.21.15.154:0/1470601019 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f815bd10 con 0x7f71f80b1b20 2023-09-23T03:19:40.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.197+0000 7f71ff221640 1 -- 172.21.15.154:0/1470601019 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71f8073eb0 con 0x7f71f80b1b20 2023-09-23T03:19:40.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.198+0000 7f71f57fa640 1 -- 172.21.15.154:0/1470601019 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f71e403b3f0 con 0x7f71f80b1b20 2023-09-23T03:19:40.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.199+0000 7f71f57fa640 1 --2- 172.21.15.154:0/1470601019 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f71dc06af30 0x7f71dc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:40.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.199+0000 7f71f57fa640 1 -- 172.21.15.154:0/1470601019 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f71e40abb40 con 0x7f71f80b1b20 2023-09-23T03:19:40.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.199+0000 7f71fcfbd640 1 --2- 172.21.15.154:0/1470601019 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f71dc06af30 0x7f71dc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:40.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.200+0000 7f71fcfbd640 1 --2- 172.21.15.154:0/1470601019 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f71dc06af30 0x7f71dc06d3f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f71ec004e50 tx=0x7f71ec009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:40.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.203+0000 7f71f57fa640 1 -- 172.21.15.154:0/1470601019 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f71e4076800 con 0x7f71f80b1b20 2023-09-23T03:19:40.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.428+0000 7f71ff221640 1 -- 172.21.15.154:0/1470601019 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71f806c930 con 0x7f71f80b1b20 2023-09-23T03:19:40.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.429+0000 7f71f57fa640 1 -- 172.21.15.154:0/1470601019 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4928 (secure 0 0 0) 0x7f71e4079e20 con 0x7f71f80b1b20 2023-09-23T03:19:40.430 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:19:40.431 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":41,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5027,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2517324433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2517324433},{"type":"v1","addr":"172.21.15.190:6833","nonce":2517324433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":5033,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4098915995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4098915995},{"type":"v1","addr":"172.21.15.190:6835","nonce":4098915995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6309,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/3034894009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":3034894009},{"type":"v1","addr":"172.21.15.154:6836","nonce":3034894009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:19:23.952758+0000","modified":"2023-09-23T03:19:39.412021+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6312":{"gid":6312,"name":"c","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6837/2183023203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":2183023203},{"type":"v1","addr":"172.21.15.154:6837","nonce":2183023203}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9,10],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-09-23T03:19:40.431 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 41 2023-09-23T03:19:40.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.431+0000 7f71ff221640 1 -- 172.21.15.154:0/1470601019 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f71dc06af30 msgr2=0x7f71dc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:40.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.431+0000 7f71ff221640 1 --2- 172.21.15.154:0/1470601019 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f71dc06af30 0x7f71dc06d3f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f71ec004e50 tx=0x7f71ec009040 comp rx=0 tx=0).stop 2023-09-23T03:19:40.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.431+0000 7f71ff221640 1 -- 172.21.15.154:0/1470601019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71f80b1b20 msgr2=0x7f71f80c7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:40.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.431+0000 7f71ff221640 1 --2- 172.21.15.154:0/1470601019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71f80b1b20 0x7f71f80c7850 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f71e4034040 tx=0x7f71e40046c0 comp rx=0 tx=0).stop 2023-09-23T03:19:40.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.432+0000 7f71ff221640 1 -- 172.21.15.154:0/1470601019 shutdown_connections 2023-09-23T03:19:40.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.432+0000 7f71ff221640 1 --2- 172.21.15.154:0/1470601019 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f71dc06af30 0x7f71dc06d3f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:40.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.432+0000 7f71ff221640 1 --2- 172.21.15.154:0/1470601019 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71f80cee70 0x7f71f8158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:40.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.432+0000 7f71ff221640 1 --2- 172.21.15.154:0/1470601019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71f80b1b20 0x7f71f80c7850 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:40.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.432+0000 7f71ff221640 1 --2- 172.21.15.154:0/1470601019 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71f80acce0 0x7f71f815ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:40.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.432+0000 7f71ff221640 1 -- 172.21.15.154:0/1470601019 >> 172.21.15.154:0/1470601019 conn(0x7f71f806a0f0 msgr2=0x7f71f815a170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:40.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.432+0000 7f71ff221640 1 -- 172.21.15.154:0/1470601019 shutdown_connections 2023-09-23T03:19:40.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.432+0000 7f71ff221640 1 -- 172.21.15.154:0/1470601019 wait complete. 2023-09-23T03:19:40.452 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:19:40.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.667+0000 7fd324fce640 1 Processor -- start 2023-09-23T03:19:40.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.667+0000 7fd324fce640 1 -- start start 2023-09-23T03:19:40.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.668+0000 7fd324fce640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd320159020 0x7fd32006d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:40.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.668+0000 7fd324fce640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd3200cc0e0 0x7fd3200b1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:40.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.668+0000 7fd324fce640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd320159400 con 0x7fd3200b18a0 2023-09-23T03:19:40.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.668+0000 7fd324fce640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3200ada00 con 0x7fd320159020 2023-09-23T03:19:40.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.668+0000 7fd324fce640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd32006d6b0 con 0x7fd3200cc0e0 2023-09-23T03:19:40.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.668+0000 7fd31e59c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd320159020 0x7fd32006d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:40.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.668+0000 7fd31ed9d640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fd3200b18a0 0x7fd3200cac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:59062/0 (socket says 172.21.15.154:59062) 2023-09-23T03:19:40.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.668+0000 7fd31ed9d640 1 -- 172.21.15.154:0/2838053171 learned_addr learned my addr 172.21.15.154:0/2838053171 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:19:40.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.669+0000 7fd317fff640 1 --2- 172.21.15.154:0/2838053171 >> v2:172.21.15.190:3300/0 conn(0x7fd3200cc0e0 0x7fd3200b1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:40.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.669+0000 7fd31dd9b640 1 -- 172.21.15.154:0/2838053171 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1852578794 0 0) 0x7fd320159400 con 0x7fd3200b18a0 2023-09-23T03:19:40.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.669+0000 7fd31dd9b640 1 -- 172.21.15.154:0/2838053171 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd30c003520 con 0x7fd3200b18a0 2023-09-23T03:19:40.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.669+0000 7fd31e59c640 1 -- 172.21.15.154:0/2838053171 >> v2:172.21.15.190:3300/0 conn(0x7fd3200cc0e0 msgr2=0x7fd3200b1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:19:40.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.669+0000 7fd31e59c640 1 --2- 172.21.15.154:0/2838053171 >> v2:172.21.15.190:3300/0 conn(0x7fd3200cc0e0 0x7fd3200b1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:40.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.669+0000 7fd31e59c640 1 -- 172.21.15.154:0/2838053171 >> v1:172.21.15.154:6789/0 conn(0x7fd3200b18a0 legacy=0x7fd3200cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:40.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.669+0000 7fd31e59c640 1 -- 172.21.15.154:0/2838053171 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3200cdaf0 con 0x7fd320159020 2023-09-23T03:19:40.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.670+0000 7fd31e59c640 1 --2- 172.21.15.154:0/2838053171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd320159020 0x7fd32006d170 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fd3200adf00 tx=0x7fd31002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c040c00ab74c3d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:40.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.670+0000 7fd31dd9b640 1 -- 172.21.15.154:0/2838053171 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31002e070 con 0x7fd320159020 2023-09-23T03:19:40.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.670+0000 7fd31dd9b640 1 -- 172.21.15.154:0/2838053171 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3100315a0 con 0x7fd320159020 2023-09-23T03:19:40.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.670+0000 7fd31dd9b640 1 -- 172.21.15.154:0/2838053171 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd310031930 con 0x7fd320159020 2023-09-23T03:19:40.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.671+0000 7fd324fce640 1 -- 172.21.15.154:0/2838053171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd320159020 msgr2=0x7fd32006d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:40.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.671+0000 7fd324fce640 1 --2- 172.21.15.154:0/2838053171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd320159020 0x7fd32006d170 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fd3200adf00 tx=0x7fd31002b840 comp rx=0 tx=0).stop 2023-09-23T03:19:40.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.671+0000 7fd324fce640 1 -- 172.21.15.154:0/2838053171 shutdown_connections 2023-09-23T03:19:40.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.671+0000 7fd324fce640 1 --2- 172.21.15.154:0/2838053171 >> v2:172.21.15.190:3300/0 conn(0x7fd3200cc0e0 0x7fd3200b1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:40.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.671+0000 7fd324fce640 1 --2- 172.21.15.154:0/2838053171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd320159020 0x7fd32006d170 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:40.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.671+0000 7fd324fce640 1 -- 172.21.15.154:0/2838053171 >> 172.21.15.154:0/2838053171 conn(0x7fd32006a0f0 msgr2=0x7fd32006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:40.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.671+0000 7fd324fce640 1 -- 172.21.15.154:0/2838053171 shutdown_connections 2023-09-23T03:19:40.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.671+0000 7fd324fce640 1 -- 172.21.15.154:0/2838053171 wait complete. 2023-09-23T03:19:40.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.673+0000 7fd324fce640 1 Processor -- start 2023-09-23T03:19:40.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.673+0000 7fd324fce640 1 -- start start 2023-09-23T03:19:40.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.674+0000 7fd324fce640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3200b18a0 0x7fd3200c1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:40.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.674+0000 7fd324fce640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3200cc0e0 0x7fd320148af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:40.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.674+0000 7fd324fce640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd320159020 0x7fd320097990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:40.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.674+0000 7fd324fce640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3200c4d40 con 0x7fd320159020 2023-09-23T03:19:40.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.674+0000 7fd324fce640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd320097ed0 con 0x7fd3200b18a0 2023-09-23T03:19:40.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.674+0000 7fd324fce640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3200c4a60 con 0x7fd3200cc0e0 2023-09-23T03:19:40.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.674+0000 7fd31ed9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd320159020 0x7fd320097990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:40.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.674+0000 7fd31ed9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd320159020 0x7fd320097990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57610/0 (socket says 172.21.15.154:57610) 2023-09-23T03:19:40.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.674+0000 7fd31ed9d640 1 -- 172.21.15.154:0/2938685746 learned_addr learned my addr 172.21.15.154:0/2938685746 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:19:40.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.675+0000 7fd31e59c640 1 --2- 172.21.15.154:0/2938685746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3200b18a0 0x7fd3200c1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:40.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.675+0000 7fd317fff640 1 --2- 172.21.15.154:0/2938685746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3200cc0e0 0x7fd320148af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:40.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.675+0000 7fd31ed9d640 1 -- 172.21.15.154:0/2938685746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3200cc0e0 msgr2=0x7fd320148af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:40.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.675+0000 7fd31ed9d640 1 --2- 172.21.15.154:0/2938685746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3200cc0e0 0x7fd320148af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:40.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.675+0000 7fd31ed9d640 1 -- 172.21.15.154:0/2938685746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3200b18a0 msgr2=0x7fd3200c1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:40.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.675+0000 7fd31ed9d640 1 --2- 172.21.15.154:0/2938685746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3200b18a0 0x7fd3200c1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:40.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.675+0000 7fd31ed9d640 1 -- 172.21.15.154:0/2938685746 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3100093a0 con 0x7fd320159020 2023-09-23T03:19:40.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.675+0000 7fd317fff640 1 --2- 172.21.15.154:0/2938685746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3200cc0e0 0x7fd320148af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:19:40.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.675+0000 7fd31ed9d640 1 --2- 172.21.15.154:0/2938685746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd320159020 0x7fd320097990 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fd31800cb40 tx=0x7fd31800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:40.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.675+0000 7fd316ffd640 1 -- 172.21.15.154:0/2938685746 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31800aca0 con 0x7fd320159020 2023-09-23T03:19:40.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.675+0000 7fd316ffd640 1 -- 172.21.15.154:0/2938685746 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd31800ae00 con 0x7fd320159020 2023-09-23T03:19:40.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.675+0000 7fd316ffd640 1 -- 172.21.15.154:0/2938685746 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd318012620 con 0x7fd320159020 2023-09-23T03:19:40.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.675+0000 7fd324fce640 1 -- 172.21.15.154:0/2938685746 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd32015ac10 con 0x7fd320159020 2023-09-23T03:19:40.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.676+0000 7fd324fce640 1 -- 172.21.15.154:0/2938685746 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd32015b1d0 con 0x7fd320159020 2023-09-23T03:19:40.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.677+0000 7fd324fce640 1 -- 172.21.15.154:0/2938685746 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd32015aa70 con 0x7fd320159020 2023-09-23T03:19:40.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.678+0000 7fd316ffd640 1 -- 172.21.15.154:0/2938685746 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fd318007070 con 0x7fd320159020 2023-09-23T03:19:40.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.679+0000 7fd316ffd640 1 --2- 172.21.15.154:0/2938685746 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd30006af60 0x7fd30006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:19:40.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.679+0000 7fd316ffd640 1 -- 172.21.15.154:0/2938685746 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6433+0+0 (secure 0 0 0) 0x7fd318089ab0 con 0x7fd320159020 2023-09-23T03:19:40.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.679+0000 7fd31e59c640 1 --2- 172.21.15.154:0/2938685746 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd30006af60 0x7fd30006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:19:40.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.680+0000 7fd31e59c640 1 --2- 172.21.15.154:0/2938685746 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd30006af60 0x7fd30006d420 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd310036000 tx=0x7fd31003d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:19:40.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.683+0000 7fd316ffd640 1 -- 172.21.15.154:0/2938685746 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fd32015aa70 con 0x7fd320159020 2023-09-23T03:19:40.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.861+0000 7fd324fce640 1 -- 172.21.15.154:0/2938685746 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd32015aa70 con 0x7fd320159020 2023-09-23T03:19:40.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.863+0000 7fd316ffd640 1 -- 172.21.15.154:0/2938685746 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+18605 (secure 0 0 0) 0x7fd32015aa70 con 0x7fd320159020 2023-09-23T03:19:40.865 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:19:40.865 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":64,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:19:39.411148+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:19:21.772037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-09-23T03:19:22.960250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data_extra","create_time":"2023-09-23T03:19:38.870843+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:19:40.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.865+0000 7fd324fce640 1 -- 172.21.15.154:0/2938685746 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd30006af60 msgr2=0x7fd30006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:40.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.866+0000 7fd324fce640 1 --2- 172.21.15.154:0/2938685746 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd30006af60 0x7fd30006d420 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd310036000 tx=0x7fd31003d040 comp rx=0 tx=0).stop 2023-09-23T03:19:40.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.866+0000 7fd324fce640 1 -- 172.21.15.154:0/2938685746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd320159020 msgr2=0x7fd320097990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:19:40.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.866+0000 7fd324fce640 1 --2- 172.21.15.154:0/2938685746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd320159020 0x7fd320097990 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fd31800cb40 tx=0x7fd31800a060 comp rx=0 tx=0).stop 2023-09-23T03:19:40.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.866+0000 7fd324fce640 1 -- 172.21.15.154:0/2938685746 shutdown_connections 2023-09-23T03:19:40.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.866+0000 7fd324fce640 1 --2- 172.21.15.154:0/2938685746 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd30006af60 0x7fd30006d420 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:40.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.866+0000 7fd324fce640 1 --2- 172.21.15.154:0/2938685746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd320159020 0x7fd320097990 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:40.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.866+0000 7fd324fce640 1 --2- 172.21.15.154:0/2938685746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3200cc0e0 0x7fd320148af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:40.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.866+0000 7fd324fce640 1 --2- 172.21.15.154:0/2938685746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd3200b18a0 0x7fd3200c1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:19:40.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.866+0000 7fd324fce640 1 -- 172.21.15.154:0/2938685746 >> 172.21.15.154:0/2938685746 conn(0x7fd32006a0f0 msgr2=0x7fd3200b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:19:40.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.866+0000 7fd324fce640 1 -- 172.21.15.154:0/2938685746 shutdown_connections 2023-09-23T03:19:40.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:40.866+0000 7fd324fce640 1 -- 172.21.15.154:0/2938685746 wait complete. 2023-09-23T03:19:40.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:19:40.885 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v cephfs_data_extra .) 2023-09-23T03:19:40.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:19:40.944 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2023-09-23T03:19:41.000 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/0 2023-09-23T03:19:41.001 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:19:41.001 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:19:41.001 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/0" 2023-09-23T03:19:41.001 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'w'"'"') as f: 2023-09-23T03:19:41.001 DEBUG:teuthology.orchestra.run.smithi154:> for i in range(0, 6291456): 2023-09-23T03:19:41.001 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:19:41.001 DEBUG:teuthology.orchestra.run.smithi154:> f.write(chr(val)) 2023-09-23T03:19:41.001 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:19:42.060 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:19:42.070 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:19:44.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:19:44.789 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/1 2023-09-23T03:19:44.790 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:19:44.790 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:19:44.790 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/1" 2023-09-23T03:19:44.790 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'w'"'"') as f: 2023-09-23T03:19:44.790 DEBUG:teuthology.orchestra.run.smithi154:> for i in range(0, 6291456): 2023-09-23T03:19:44.790 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:19:44.790 DEBUG:teuthology.orchestra.run.smithi154:> f.write(chr(val)) 2023-09-23T03:19:44.790 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:19:48.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:19:48.520 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/2 2023-09-23T03:19:48.521 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:19:48.521 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:19:48.521 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/2" 2023-09-23T03:19:48.521 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'w'"'"') as f: 2023-09-23T03:19:48.521 DEBUG:teuthology.orchestra.run.smithi154:> for i in range(0, 6291456): 2023-09-23T03:19:48.521 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:19:48.521 DEBUG:teuthology.orchestra.run.smithi154:> f.write(chr(val)) 2023-09-23T03:19:48.521 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:19:52.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:19:52.358 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/3 2023-09-23T03:19:52.359 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:19:52.359 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:19:52.359 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/3" 2023-09-23T03:19:52.359 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'w'"'"') as f: 2023-09-23T03:19:52.359 DEBUG:teuthology.orchestra.run.smithi154:> for i in range(0, 6291456): 2023-09-23T03:19:52.359 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:19:52.359 DEBUG:teuthology.orchestra.run.smithi154:> f.write(chr(val)) 2023-09-23T03:19:52.359 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:19:56.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:19:56.174 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/4 2023-09-23T03:19:56.174 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:19:56.174 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:19:56.174 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/4" 2023-09-23T03:19:56.174 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'w'"'"') as f: 2023-09-23T03:19:56.174 DEBUG:teuthology.orchestra.run.smithi154:> for i in range(0, 6291456): 2023-09-23T03:19:56.174 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:19:56.175 DEBUG:teuthology.orchestra.run.smithi154:> f.write(chr(val)) 2023-09-23T03:19:56.175 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:19:59.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:19:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:19:59.983 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/5 2023-09-23T03:19:59.983 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:19:59.983 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:19:59.983 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/5" 2023-09-23T03:19:59.983 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'w'"'"') as f: 2023-09-23T03:19:59.983 DEBUG:teuthology.orchestra.run.smithi154:> for i in range(0, 6291456): 2023-09-23T03:19:59.983 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:19:59.983 DEBUG:teuthology.orchestra.run.smithi154:> f.write(chr(val)) 2023-09-23T03:19:59.983 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:20:03.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:20:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:20:03.879 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/6 2023-09-23T03:20:03.879 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:20:03.879 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:20:03.879 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/6" 2023-09-23T03:20:03.879 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'w'"'"') as f: 2023-09-23T03:20:03.879 DEBUG:teuthology.orchestra.run.smithi154:> for i in range(0, 6291456): 2023-09-23T03:20:03.879 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:20:03.879 DEBUG:teuthology.orchestra.run.smithi154:> f.write(chr(val)) 2023-09-23T03:20:03.880 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:20:07.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:20:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:20:07.711 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/7 2023-09-23T03:20:07.711 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:20:07.711 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:20:07.711 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/7" 2023-09-23T03:20:07.711 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'w'"'"') as f: 2023-09-23T03:20:07.711 DEBUG:teuthology.orchestra.run.smithi154:> for i in range(0, 6291456): 2023-09-23T03:20:07.711 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:20:07.711 DEBUG:teuthology.orchestra.run.smithi154:> f.write(chr(val)) 2023-09-23T03:20:07.711 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:20:11.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:20:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:20:11.562 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/8 2023-09-23T03:20:11.562 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:20:11.562 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:20:11.562 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/8" 2023-09-23T03:20:11.562 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'w'"'"') as f: 2023-09-23T03:20:11.562 DEBUG:teuthology.orchestra.run.smithi154:> for i in range(0, 6291456): 2023-09-23T03:20:11.563 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:20:11.563 DEBUG:teuthology.orchestra.run.smithi154:> f.write(chr(val)) 2023-09-23T03:20:11.563 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:20:12.209 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:20:12.218 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:20:15.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:20:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:20:15.385 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/9 2023-09-23T03:20:15.385 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:20:15.385 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:20:15.385 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/9" 2023-09-23T03:20:15.385 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'w'"'"') as f: 2023-09-23T03:20:15.386 DEBUG:teuthology.orchestra.run.smithi154:> for i in range(0, 6291456): 2023-09-23T03:20:15.386 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:20:15.386 DEBUG:teuthology.orchestra.run.smithi154:> f.write(chr(val)) 2023-09-23T03:20:15.386 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:20:19.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:20:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:20:19.142 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/10 2023-09-23T03:20:19.143 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:20:19.143 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:20:19.143 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/10" 2023-09-23T03:20:19.143 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'w'"'"') as f: 2023-09-23T03:20:19.143 DEBUG:teuthology.orchestra.run.smithi154:> for i in range(0, 6291456): 2023-09-23T03:20:19.143 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:20:19.143 DEBUG:teuthology.orchestra.run.smithi154:> f.write(chr(val)) 2023-09-23T03:20:19.143 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:20:22.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:20:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:20:22.900 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/11 2023-09-23T03:20:22.900 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:20:22.900 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:20:22.900 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/11" 2023-09-23T03:20:22.900 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'w'"'"') as f: 2023-09-23T03:20:22.900 DEBUG:teuthology.orchestra.run.smithi154:> for i in range(0, 6291456): 2023-09-23T03:20:22.900 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:20:22.901 DEBUG:teuthology.orchestra.run.smithi154:> f.write(chr(val)) 2023-09-23T03:20:22.901 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:20:26.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:20:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:20:26.694 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/12 2023-09-23T03:20:26.695 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:20:26.695 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:20:26.695 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/12" 2023-09-23T03:20:26.695 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'w'"'"') as f: 2023-09-23T03:20:26.695 DEBUG:teuthology.orchestra.run.smithi154:> for i in range(0, 6291456): 2023-09-23T03:20:26.695 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:20:26.695 DEBUG:teuthology.orchestra.run.smithi154:> f.write(chr(val)) 2023-09-23T03:20:26.695 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:20:30.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:20:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:20:30.419 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/13 2023-09-23T03:20:30.420 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:20:30.420 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:20:30.420 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/13" 2023-09-23T03:20:30.420 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'w'"'"') as f: 2023-09-23T03:20:30.420 DEBUG:teuthology.orchestra.run.smithi154:> for i in range(0, 6291456): 2023-09-23T03:20:30.420 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:20:30.420 DEBUG:teuthology.orchestra.run.smithi154:> f.write(chr(val)) 2023-09-23T03:20:30.420 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:20:34.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:20:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:20:34.174 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/14 2023-09-23T03:20:34.174 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:20:34.174 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:20:34.174 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/14" 2023-09-23T03:20:34.175 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'w'"'"') as f: 2023-09-23T03:20:34.175 DEBUG:teuthology.orchestra.run.smithi154:> for i in range(0, 6291456): 2023-09-23T03:20:34.175 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:20:34.175 DEBUG:teuthology.orchestra.run.smithi154:> f.write(chr(val)) 2023-09-23T03:20:34.175 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:20:38.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:20:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:20:38.055 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/15 2023-09-23T03:20:38.055 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:20:38.055 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:20:38.055 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/15" 2023-09-23T03:20:38.055 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'w'"'"') as f: 2023-09-23T03:20:38.055 DEBUG:teuthology.orchestra.run.smithi154:> for i in range(0, 6291456): 2023-09-23T03:20:38.056 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:20:38.056 DEBUG:teuthology.orchestra.run.smithi154:> f.write(chr(val)) 2023-09-23T03:20:38.056 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:20:41.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:20:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:20:41.854 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/16 2023-09-23T03:20:41.854 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:20:41.854 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:20:41.854 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/16" 2023-09-23T03:20:41.854 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'w'"'"') as f: 2023-09-23T03:20:41.854 DEBUG:teuthology.orchestra.run.smithi154:> for i in range(0, 6291456): 2023-09-23T03:20:41.854 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:20:41.854 DEBUG:teuthology.orchestra.run.smithi154:> f.write(chr(val)) 2023-09-23T03:20:41.855 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:20:42.364 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:20:42.373 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:20:45.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:20:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:20:45.691 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/17 2023-09-23T03:20:45.691 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:20:45.692 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:20:45.692 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/17" 2023-09-23T03:20:45.692 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'w'"'"') as f: 2023-09-23T03:20:45.692 DEBUG:teuthology.orchestra.run.smithi154:> for i in range(0, 6291456): 2023-09-23T03:20:45.692 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:20:45.692 DEBUG:teuthology.orchestra.run.smithi154:> f.write(chr(val)) 2023-09-23T03:20:45.692 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:20:49.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:20:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:20:49.426 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/18 2023-09-23T03:20:49.426 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:20:49.427 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:20:49.427 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/18" 2023-09-23T03:20:49.427 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'w'"'"') as f: 2023-09-23T03:20:49.427 DEBUG:teuthology.orchestra.run.smithi154:> for i in range(0, 6291456): 2023-09-23T03:20:49.427 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:20:49.427 DEBUG:teuthology.orchestra.run.smithi154:> f.write(chr(val)) 2023-09-23T03:20:49.427 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:20:53.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:20:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:20:53.187 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/19 2023-09-23T03:20:53.187 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:20:53.187 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:20:53.187 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/19" 2023-09-23T03:20:53.188 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'w'"'"') as f: 2023-09-23T03:20:53.188 DEBUG:teuthology.orchestra.run.smithi154:> for i in range(0, 6291456): 2023-09-23T03:20:53.188 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:20:53.188 DEBUG:teuthology.orchestra.run.smithi154:> f.write(chr(val)) 2023-09-23T03:20:53.188 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:20:57.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:20:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:20:57.017 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/20 2023-09-23T03:20:57.017 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:20:57.017 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:20:57.018 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/20" 2023-09-23T03:20:57.018 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'w'"'"') as f: 2023-09-23T03:20:57.018 DEBUG:teuthology.orchestra.run.smithi154:> for i in range(0, 6291456): 2023-09-23T03:20:57.018 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:20:57.018 DEBUG:teuthology.orchestra.run.smithi154:> f.write(chr(val)) 2023-09-23T03:20:57.018 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:21:00.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:21:00.857 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/21 2023-09-23T03:21:00.858 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:21:00.858 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:21:00.858 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/21" 2023-09-23T03:21:00.858 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'w'"'"') as f: 2023-09-23T03:21:00.858 DEBUG:teuthology.orchestra.run.smithi154:> for i in range(0, 6291456): 2023-09-23T03:21:00.858 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:21:00.858 DEBUG:teuthology.orchestra.run.smithi154:> f.write(chr(val)) 2023-09-23T03:21:00.859 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:21:04.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:21:04.629 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/22 2023-09-23T03:21:04.629 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:21:04.629 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:21:04.629 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/22" 2023-09-23T03:21:04.629 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'w'"'"') as f: 2023-09-23T03:21:04.629 DEBUG:teuthology.orchestra.run.smithi154:> for i in range(0, 6291456): 2023-09-23T03:21:04.629 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:21:04.630 DEBUG:teuthology.orchestra.run.smithi154:> f.write(chr(val)) 2023-09-23T03:21:04.630 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:21:08.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:21:08.497 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/23 2023-09-23T03:21:08.497 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:21:08.497 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:21:08.497 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/23" 2023-09-23T03:21:08.497 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'w'"'"') as f: 2023-09-23T03:21:08.497 DEBUG:teuthology.orchestra.run.smithi154:> for i in range(0, 6291456): 2023-09-23T03:21:08.498 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:21:08.498 DEBUG:teuthology.orchestra.run.smithi154:> f.write(chr(val)) 2023-09-23T03:21:08.498 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:21:12.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:21:12.262 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/24 2023-09-23T03:21:12.262 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:21:12.262 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:21:12.262 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/24" 2023-09-23T03:21:12.262 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'w'"'"') as f: 2023-09-23T03:21:12.262 DEBUG:teuthology.orchestra.run.smithi154:> for i in range(0, 6291456): 2023-09-23T03:21:12.263 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:21:12.263 DEBUG:teuthology.orchestra.run.smithi154:> f.write(chr(val)) 2023-09-23T03:21:12.263 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:21:12.510 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:21:12.519 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:21:16.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:21:16.072 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:21:16.072 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:21:16.119 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:21:16.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.371+0000 7f53627ff640 1 Processor -- start 2023-09-23T03:21:16.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.371+0000 7f53627ff640 1 -- start start 2023-09-23T03:21:16.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.372+0000 7f53627ff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f535c160fd0 0x7f535c0c9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:16.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.372+0000 7f53627ff640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f535c0cc910 con 0x7f535c15c840 2023-09-23T03:21:16.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.372+0000 7f53627ff640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f535c0c89b0 con 0x7f535c0c99d0 2023-09-23T03:21:16.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.372+0000 7f53627ff640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f535c15cc20 con 0x7f535c160fd0 2023-09-23T03:21:16.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.373+0000 7f53617fd640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f535c15c840 0x7f535c0cbfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:49270/0 (socket says 172.21.15.154:49270) 2023-09-23T03:21:16.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.373+0000 7f53617fd640 1 -- 172.21.15.154:0/4077268674 learned_addr learned my addr 172.21.15.154:0/4077268674 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:21:16.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.373+0000 7f5353fff640 1 -- 172.21.15.154:0/4077268674 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3767396250 0 0) 0x7f535c0cc910 con 0x7f535c15c840 2023-09-23T03:21:16.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.373+0000 7f5353fff640 1 -- 172.21.15.154:0/4077268674 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5348003520 con 0x7f535c15c840 2023-09-23T03:21:16.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.373+0000 7f5360ffc640 1 --2- 172.21.15.154:0/4077268674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f535c160fd0 0x7f535c0c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:16.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.373+0000 7f5353fff640 1 -- 172.21.15.154:0/4077268674 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 199740163 0 0) 0x7f5348003520 con 0x7f535c15c840 2023-09-23T03:21:16.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.374+0000 7f5353fff640 1 -- 172.21.15.154:0/4077268674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f535c160fd0 msgr2=0x7f535c0c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:16.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.374+0000 7f5353fff640 1 --2- 172.21.15.154:0/4077268674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f535c160fd0 0x7f535c0c9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:16.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.374+0000 7f5353fff640 1 -- 172.21.15.154:0/4077268674 >> v1:172.21.15.190:6789/0 conn(0x7f535c0c99d0 legacy=0x7f535c0cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:16.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.374+0000 7f5353fff640 1 -- 172.21.15.154:0/4077268674 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f535c0c8c90 con 0x7f535c15c840 2023-09-23T03:21:16.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.374+0000 7f5353fff640 1 -- 172.21.15.154:0/4077268674 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f5358001670 con 0x7f535c15c840 2023-09-23T03:21:16.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.374+0000 7f5353fff640 1 -- 172.21.15.154:0/4077268674 >> v1:172.21.15.154:6789/0 conn(0x7f535c15c840 legacy=0x7f535c0cbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:16.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.374+0000 7f5353fff640 1 --2- 172.21.15.154:0/4077268674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53480039a0 0x7f5348003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:16.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.374+0000 7f5353fff640 1 --2- 172.21.15.154:0/4077268674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53480047c0 0x7f5348004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:16.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.374+0000 7f5353fff640 1 --2- 172.21.15.154:0/4077268674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f535c160fd0 0x7f5348005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:16.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.374+0000 7f5353fff640 1 -- 172.21.15.154:0/4077268674 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535c15cc20 con 0x7f53480047c0 2023-09-23T03:21:16.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.374+0000 7f5353fff640 1 -- 172.21.15.154:0/4077268674 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534800bb10 con 0x7f53480039a0 2023-09-23T03:21:16.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.374+0000 7f5353fff640 1 -- 172.21.15.154:0/4077268674 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f534800bca0 con 0x7f535c160fd0 2023-09-23T03:21:16.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.374+0000 7f5360ffc640 1 --2- 172.21.15.154:0/4077268674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53480047c0 0x7f5348004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:16.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.374+0000 7f53617fd640 1 --2- 172.21.15.154:0/4077268674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53480039a0 0x7f5348003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:16.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.374+0000 7f5360ffc640 1 -- 172.21.15.154:0/4077268674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f535c160fd0 msgr2=0x7f5348005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:16.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.374+0000 7f5361ffe640 1 --2- 172.21.15.154:0/4077268674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f535c160fd0 0x7f5348005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:16.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.374+0000 7f5360ffc640 1 --2- 172.21.15.154:0/4077268674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f535c160fd0 0x7f5348005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:16.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.374+0000 7f5360ffc640 1 -- 172.21.15.154:0/4077268674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53480039a0 msgr2=0x7f5348003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:16.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.374+0000 7f5360ffc640 1 --2- 172.21.15.154:0/4077268674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53480039a0 0x7f5348003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:16.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.374+0000 7f5360ffc640 1 -- 172.21.15.154:0/4077268674 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5348003520 con 0x7f53480047c0 2023-09-23T03:21:16.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.375+0000 7f5361ffe640 1 --2- 172.21.15.154:0/4077268674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f535c160fd0 0x7f5348005230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:16.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.376+0000 7f5360ffc640 1 --2- 172.21.15.154:0/4077268674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53480047c0 0x7f5348004c40 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f534c00ba10 tx=0x7f53480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71601d2674b4dd2b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:16.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.376+0000 7f5353fff640 1 -- 172.21.15.154:0/4077268674 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c032620 con 0x7f53480047c0 2023-09-23T03:21:16.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.376+0000 7f5353fff640 1 -- 172.21.15.154:0/4077268674 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f534c032780 con 0x7f53480047c0 2023-09-23T03:21:16.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.376+0000 7f5353fff640 1 -- 172.21.15.154:0/4077268674 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c032b10 con 0x7f53480047c0 2023-09-23T03:21:16.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.376+0000 7f53627ff640 1 -- 172.21.15.154:0/4077268674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53480047c0 msgr2=0x7f5348004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:16.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.376+0000 7f53627ff640 1 --2- 172.21.15.154:0/4077268674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53480047c0 0x7f5348004c40 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f534c00ba10 tx=0x7f53480057f0 comp rx=0 tx=0).stop 2023-09-23T03:21:16.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.376+0000 7f5361ffe640 1 -- 172.21.15.154:0/4077268674 reap_dead start 2023-09-23T03:21:16.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.376+0000 7f53627ff640 1 -- 172.21.15.154:0/4077268674 shutdown_connections 2023-09-23T03:21:16.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.376+0000 7f53627ff640 1 -- 172.21.15.154:0/4077268674 >> 172.21.15.154:0/4077268674 conn(0x7f535c06a390 msgr2=0x7f535c0afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:16.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.376+0000 7f53627ff640 1 -- 172.21.15.154:0/4077268674 shutdown_connections 2023-09-23T03:21:16.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.376+0000 7f53627ff640 1 -- 172.21.15.154:0/4077268674 wait complete. 2023-09-23T03:21:16.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.378+0000 7f53627ff640 1 Processor -- start 2023-09-23T03:21:16.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.378+0000 7f53627ff640 1 -- start start 2023-09-23T03:21:16.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.379+0000 7f53627ff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53480047c0 0x7f535c156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:16.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.379+0000 7f53627ff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f535c1f2a40 0x7f535c15eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:16.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.379+0000 7f53627ff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f535c0c2c40 0x7f535c0709a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:16.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.379+0000 7f53627ff640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535c15fc20 con 0x7f535c0c2c40 2023-09-23T03:21:16.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.379+0000 7f53627ff640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535c15f0e0 con 0x7f535c1f2a40 2023-09-23T03:21:16.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.379+0000 7f53627ff640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f535c15f380 con 0x7f53480047c0 2023-09-23T03:21:16.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.380+0000 7f5361ffe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f535c0c2c40 0x7f535c0709a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:16.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.380+0000 7f5361ffe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f535c0c2c40 0x7f535c0709a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:40508/0 (socket says 172.21.15.154:40508) 2023-09-23T03:21:16.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.380+0000 7f5361ffe640 1 -- 172.21.15.154:0/2155645686 learned_addr learned my addr 172.21.15.154:0/2155645686 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:16.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.380+0000 7f5360ffc640 1 --2- 172.21.15.154:0/2155645686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f535c1f2a40 0x7f535c15eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:16.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.380+0000 7f5361ffe640 1 -- 172.21.15.154:0/2155645686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53480047c0 msgr2=0x7f535c156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:16.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.380+0000 7f53617fd640 1 --2- 172.21.15.154:0/2155645686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53480047c0 0x7f535c156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:16.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.380+0000 7f5361ffe640 1 --2- 172.21.15.154:0/2155645686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53480047c0 0x7f535c156b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:16.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.380+0000 7f5361ffe640 1 -- 172.21.15.154:0/2155645686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f535c1f2a40 msgr2=0x7f535c15eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:16.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.380+0000 7f5361ffe640 1 --2- 172.21.15.154:0/2155645686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f535c1f2a40 0x7f535c15eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:16.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.380+0000 7f5361ffe640 1 -- 172.21.15.154:0/2155645686 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f535c15f660 con 0x7f535c0c2c40 2023-09-23T03:21:16.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.380+0000 7f5360ffc640 1 --2- 172.21.15.154:0/2155645686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f535c1f2a40 0x7f535c15eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:16.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.380+0000 7f5361ffe640 1 --2- 172.21.15.154:0/2155645686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f535c0c2c40 0x7f535c0709a0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f535c0cbf40 tx=0x7f535400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:16.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.381+0000 7f5351ffb640 1 -- 172.21.15.154:0/2155645686 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5354006d50 con 0x7f535c0c2c40 2023-09-23T03:21:16.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.381+0000 7f5351ffb640 1 -- 172.21.15.154:0/2155645686 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5354004910 con 0x7f535c0c2c40 2023-09-23T03:21:16.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.381+0000 7f53627ff640 1 -- 172.21.15.154:0/2155645686 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f535c15f940 con 0x7f535c0c2c40 2023-09-23T03:21:16.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.381+0000 7f5351ffb640 1 -- 172.21.15.154:0/2155645686 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5354004cc0 con 0x7f535c0c2c40 2023-09-23T03:21:16.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.381+0000 7f53627ff640 1 -- 172.21.15.154:0/2155645686 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f535c0c30c0 con 0x7f535c0c2c40 2023-09-23T03:21:16.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.382+0000 7f53627ff640 1 -- 172.21.15.154:0/2155645686 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f535c0736b0 con 0x7f535c0c2c40 2023-09-23T03:21:16.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.383+0000 7f5351ffb640 1 -- 172.21.15.154:0/2155645686 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f5354005030 con 0x7f535c0c2c40 2023-09-23T03:21:16.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.384+0000 7f5351ffb640 1 --2- 172.21.15.154:0/2155645686 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f534006af30 0x7f534006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:16.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.384+0000 7f5351ffb640 1 -- 172.21.15.154:0/2155645686 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f5354089c00 con 0x7f535c0c2c40 2023-09-23T03:21:16.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.384+0000 7f53617fd640 1 --2- 172.21.15.154:0/2155645686 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f534006af30 0x7f534006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:16.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.385+0000 7f53617fd640 1 --2- 172.21.15.154:0/2155645686 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f534006af30 0x7f534006d3f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5358001eb0 tx=0x7f5358001130 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:16.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.391+0000 7f5351ffb640 1 -- 172.21.15.154:0/2155645686 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f5354054950 con 0x7f535c0c2c40 2023-09-23T03:21:16.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.573+0000 7f53627ff640 1 -- 172.21.15.154:0/2155645686 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f535c15cd90 con 0x7f535c0c2c40 2023-09-23T03:21:16.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.576+0000 7f5351ffb640 1 -- 172.21.15.154:0/2155645686 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+18592 (secure 0 0 0) 0x7f535c15cd90 con 0x7f535c0c2c40 2023-09-23T03:21:16.576 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:16.577 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":65,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:19:40.966608+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:19:21.772037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-09-23T03:19:22.960250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data_extra","create_time":"2023-09-23T03:19:38.870843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:21:16.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.578+0000 7f53627ff640 1 -- 172.21.15.154:0/2155645686 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f534006af30 msgr2=0x7f534006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:16.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.578+0000 7f53627ff640 1 --2- 172.21.15.154:0/2155645686 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f534006af30 0x7f534006d3f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f5358001eb0 tx=0x7f5358001130 comp rx=0 tx=0).stop 2023-09-23T03:21:16.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.578+0000 7f53627ff640 1 -- 172.21.15.154:0/2155645686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f535c0c2c40 msgr2=0x7f535c0709a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:16.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.578+0000 7f53627ff640 1 --2- 172.21.15.154:0/2155645686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f535c0c2c40 0x7f535c0709a0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f535c0cbf40 tx=0x7f535400dc00 comp rx=0 tx=0).stop 2023-09-23T03:21:16.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.578+0000 7f53627ff640 1 -- 172.21.15.154:0/2155645686 shutdown_connections 2023-09-23T03:21:16.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.578+0000 7f53627ff640 1 --2- 172.21.15.154:0/2155645686 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f534006af30 0x7f534006d3f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:16.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.578+0000 7f53627ff640 1 --2- 172.21.15.154:0/2155645686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f535c0c2c40 0x7f535c0709a0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:16.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.578+0000 7f53627ff640 1 --2- 172.21.15.154:0/2155645686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f535c1f2a40 0x7f535c15eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:16.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.578+0000 7f53627ff640 1 --2- 172.21.15.154:0/2155645686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53480047c0 0x7f535c156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:16.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.578+0000 7f53627ff640 1 -- 172.21.15.154:0/2155645686 >> 172.21.15.154:0/2155645686 conn(0x7f535c06a390 msgr2=0x7f535c15af60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:16.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.578+0000 7f53627ff640 1 -- 172.21.15.154:0/2155645686 shutdown_connections 2023-09-23T03:21:16.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.578+0000 7f53627ff640 1 -- 172.21.15.154:0/2155645686 wait complete. 2023-09-23T03:21:16.595 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:19:21.772037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:19:22.960250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-09-23T03:19:38.870843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T03:21:16.596 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:21:16.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.800+0000 7fb09b52c640 1 Processor -- start 2023-09-23T03:21:16.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.800+0000 7fb09b52c640 1 -- start start 2023-09-23T03:21:16.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.801+0000 7fb09b52c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb0940cb5b0 0x7fb0940cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:16.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.801+0000 7fb09b52c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb09415b6a0 con 0x7fb09415aba0 2023-09-23T03:21:16.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.801+0000 7fb09b52c640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb09415f0b0 con 0x7fb09415bb30 2023-09-23T03:21:16.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.801+0000 7fb09b52c640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb09415bf70 con 0x7fb0940cb5b0 2023-09-23T03:21:16.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.801+0000 7fb0992c8640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fb09415aba0 0x7fb09415e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:49284/0 (socket says 172.21.15.154:49284) 2023-09-23T03:21:16.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.802+0000 7fb0992c8640 1 -- 172.21.15.154:0/2873264244 learned_addr learned my addr 172.21.15.154:0/2873264244 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:21:16.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.802+0000 7fb08bfff640 1 -- 172.21.15.154:0/2873264244 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1534242081 0 0) 0x7fb09415b6a0 con 0x7fb09415aba0 2023-09-23T03:21:16.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.802+0000 7fb08bfff640 1 -- 172.21.15.154:0/2873264244 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb080003520 con 0x7fb09415aba0 2023-09-23T03:21:16.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.802+0000 7fb099ac9640 1 --2- 172.21.15.154:0/2873264244 >> v2:172.21.15.190:3300/0 conn(0x7fb0940cb5b0 0x7fb0940cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:16.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.802+0000 7fb08bfff640 1 -- 172.21.15.154:0/2873264244 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4118189378 0 0) 0x7fb080003520 con 0x7fb09415aba0 2023-09-23T03:21:16.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.802+0000 7fb08bfff640 1 -- 172.21.15.154:0/2873264244 >> v2:172.21.15.190:3300/0 conn(0x7fb0940cb5b0 msgr2=0x7fb0940cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:16.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.802+0000 7fb08bfff640 1 --2- 172.21.15.154:0/2873264244 >> v2:172.21.15.190:3300/0 conn(0x7fb0940cb5b0 0x7fb0940cc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:16.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.802+0000 7fb08bfff640 1 -- 172.21.15.154:0/2873264244 >> v1:172.21.15.190:6789/0 conn(0x7fb09415bb30 legacy=0x7fb0940caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:16.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.802+0000 7fb08bfff640 1 -- 172.21.15.154:0/2873264244 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb09415f390 con 0x7fb09415aba0 2023-09-23T03:21:16.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.802+0000 7fb08bfff640 1 -- 172.21.15.154:0/2873264244 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fb090001670 con 0x7fb09415aba0 2023-09-23T03:21:16.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.802+0000 7fb08bfff640 1 -- 172.21.15.154:0/2873264244 >> v1:172.21.15.154:6789/0 conn(0x7fb09415aba0 legacy=0x7fb09415e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:16.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.802+0000 7fb08bfff640 1 --2- 172.21.15.154:0/2873264244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0800039c0 0x7fb080003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:16.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.802+0000 7fb08bfff640 1 --2- 172.21.15.154:0/2873264244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0800047c0 0x7fb080004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:16.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.802+0000 7fb08bfff640 1 --2- 172.21.15.154:0/2873264244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0800051d0 0x7fb08000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:16.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.803+0000 7fb08bfff640 1 -- 172.21.15.154:0/2873264244 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09415bf70 con 0x7fb0800039c0 2023-09-23T03:21:16.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.803+0000 7fb08bfff640 1 -- 172.21.15.154:0/2873264244 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08000c100 con 0x7fb0800051d0 2023-09-23T03:21:16.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.803+0000 7fb08bfff640 1 -- 172.21.15.154:0/2873264244 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb08000c2c0 con 0x7fb0800047c0 2023-09-23T03:21:16.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.803+0000 7fb0992c8640 1 --2- 172.21.15.154:0/2873264244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0800039c0 0x7fb080003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:16.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.803+0000 7fb0992c8640 1 -- 172.21.15.154:0/2873264244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0800047c0 msgr2=0x7fb080004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:16.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.803+0000 7fb0992c8640 1 --2- 172.21.15.154:0/2873264244 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb0800047c0 0x7fb080004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:16.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.803+0000 7fb0992c8640 1 -- 172.21.15.154:0/2873264244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0800051d0 msgr2=0x7fb08000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:16.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.803+0000 7fb0992c8640 1 --2- 172.21.15.154:0/2873264244 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0800051d0 0x7fb08000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:16.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.803+0000 7fb0992c8640 1 -- 172.21.15.154:0/2873264244 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb080003520 con 0x7fb0800039c0 2023-09-23T03:21:16.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.803+0000 7fb099ac9640 1 -- 172.21.15.154:0/2873264244 reap_dead start 2023-09-23T03:21:16.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.804+0000 7fb0992c8640 1 --2- 172.21.15.154:0/2873264244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0800039c0 0x7fb080003dc0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fb09000c160 tx=0x7fb0800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85468a8a1f4afe24 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:16.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.804+0000 7fb08bfff640 1 -- 172.21.15.154:0/2873264244 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb090005ad0 con 0x7fb0800039c0 2023-09-23T03:21:16.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.804+0000 7fb08bfff640 1 -- 172.21.15.154:0/2873264244 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb090005c30 con 0x7fb0800039c0 2023-09-23T03:21:16.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.804+0000 7fb08bfff640 1 -- 172.21.15.154:0/2873264244 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09002c7e0 con 0x7fb0800039c0 2023-09-23T03:21:16.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.805+0000 7fb09b52c640 1 -- 172.21.15.154:0/2873264244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0800039c0 msgr2=0x7fb080003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:16.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.805+0000 7fb09b52c640 1 --2- 172.21.15.154:0/2873264244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0800039c0 0x7fb080003dc0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fb09000c160 tx=0x7fb0800057f0 comp rx=0 tx=0).stop 2023-09-23T03:21:16.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.805+0000 7fb09b52c640 1 -- 172.21.15.154:0/2873264244 shutdown_connections 2023-09-23T03:21:16.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.805+0000 7fb09b52c640 1 --2- 172.21.15.154:0/2873264244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0800039c0 0x7fb080003dc0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:16.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.805+0000 7fb09b52c640 1 -- 172.21.15.154:0/2873264244 >> 172.21.15.154:0/2873264244 conn(0x7fb09406c010 msgr2=0x7fb09406c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:16.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.805+0000 7fb09b52c640 1 -- 172.21.15.154:0/2873264244 shutdown_connections 2023-09-23T03:21:16.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.805+0000 7fb09b52c640 1 -- 172.21.15.154:0/2873264244 wait complete. 2023-09-23T03:21:16.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.807+0000 7fb09b52c640 1 Processor -- start 2023-09-23T03:21:16.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.807+0000 7fb09b52c640 1 -- start start 2023-09-23T03:21:16.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.808+0000 7fb09b52c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0800039c0 0x7fb0940c4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:16.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.808+0000 7fb09b52c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0940c3500 0x7fb0941edf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:16.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.808+0000 7fb09b52c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb094159d10 0x7fb094150340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:16.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.808+0000 7fb09b52c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0941ee480 con 0x7fb0800039c0 2023-09-23T03:21:16.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.808+0000 7fb09b52c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0941512b0 con 0x7fb0940c3500 2023-09-23T03:21:16.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.808+0000 7fb09b52c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb094151500 con 0x7fb094159d10 2023-09-23T03:21:16.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.808+0000 7fb0992c8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0800039c0 0x7fb0940c4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:16.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.808+0000 7fb0992c8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0800039c0 0x7fb0940c4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:40516/0 (socket says 172.21.15.154:40516) 2023-09-23T03:21:16.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.808+0000 7fb0992c8640 1 -- 172.21.15.154:0/2707148075 learned_addr learned my addr 172.21.15.154:0/2707148075 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:16.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.808+0000 7fb0992c8640 1 -- 172.21.15.154:0/2707148075 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb094159d10 msgr2=0x7fb094150340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:16.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.808+0000 7fb0992c8640 1 --2- 172.21.15.154:0/2707148075 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb094159d10 0x7fb094150340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:16.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.808+0000 7fb0992c8640 1 -- 172.21.15.154:0/2707148075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0940c3500 msgr2=0x7fb0941edf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:16.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.808+0000 7fb0992c8640 1 --2- 172.21.15.154:0/2707148075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0940c3500 0x7fb0941edf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:16.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.808+0000 7fb0992c8640 1 -- 172.21.15.154:0/2707148075 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0941517e0 con 0x7fb0800039c0 2023-09-23T03:21:16.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.809+0000 7fb0992c8640 1 --2- 172.21.15.154:0/2707148075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0800039c0 0x7fb0940c4420 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb09002c380 tx=0x7fb09002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:16.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.809+0000 7fb089ffb640 1 -- 172.21.15.154:0/2707148075 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09003c040 con 0x7fb0800039c0 2023-09-23T03:21:16.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.809+0000 7fb089ffb640 1 -- 172.21.15.154:0/2707148075 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb09002ccb0 con 0x7fb0800039c0 2023-09-23T03:21:16.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.809+0000 7fb09b52c640 1 -- 172.21.15.154:0/2707148075 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb094151ac0 con 0x7fb0800039c0 2023-09-23T03:21:16.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.809+0000 7fb089ffb640 1 -- 172.21.15.154:0/2707148075 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0900072b0 con 0x7fb0800039c0 2023-09-23T03:21:16.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.809+0000 7fb09b52c640 1 -- 172.21.15.154:0/2707148075 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb094152080 con 0x7fb0800039c0 2023-09-23T03:21:16.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.810+0000 7fb09b52c640 1 -- 172.21.15.154:0/2707148075 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb094152210 con 0x7fb0800039c0 2023-09-23T03:21:16.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.811+0000 7fb089ffb640 1 -- 172.21.15.154:0/2707148075 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fb090032070 con 0x7fb0800039c0 2023-09-23T03:21:16.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.812+0000 7fb089ffb640 1 --2- 172.21.15.154:0/2707148075 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb07806af60 0x7fb07806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:16.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.812+0000 7fb089ffb640 1 -- 172.21.15.154:0/2707148075 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6433+0+0 (secure 0 0 0) 0x7fb0900aaff0 con 0x7fb0800039c0 2023-09-23T03:21:16.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.812+0000 7fb098ac7640 1 --2- 172.21.15.154:0/2707148075 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb07806af60 0x7fb07806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:16.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.813+0000 7fb098ac7640 1 --2- 172.21.15.154:0/2707148075 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb07806af60 0x7fb07806d420 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb084007fd0 tx=0x7fb084007920 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:16.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.816+0000 7fb089ffb640 1 -- 172.21.15.154:0/2707148075 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fb0900dda50 con 0x7fb0800039c0 2023-09-23T03:21:16.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.989+0000 7fb09b52c640 1 -- 172.21.15.154:0/2707148075 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb09404e920 con 0x7fb0800039c0 2023-09-23T03:21:16.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.990+0000 7fb089ffb640 1 -- 172.21.15.154:0/2707148075 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb0900460a0 con 0x7fb0800039c0 2023-09-23T03:21:16.990 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:21:16.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.993+0000 7fb09b52c640 1 -- 172.21.15.154:0/2707148075 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb07806af60 msgr2=0x7fb07806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:16.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.993+0000 7fb09b52c640 1 --2- 172.21.15.154:0/2707148075 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb07806af60 0x7fb07806d420 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb084007fd0 tx=0x7fb084007920 comp rx=0 tx=0).stop 2023-09-23T03:21:16.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.993+0000 7fb09b52c640 1 -- 172.21.15.154:0/2707148075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0800039c0 msgr2=0x7fb0940c4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:16.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.993+0000 7fb09b52c640 1 --2- 172.21.15.154:0/2707148075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0800039c0 0x7fb0940c4420 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb09002c380 tx=0x7fb09002e070 comp rx=0 tx=0).stop 2023-09-23T03:21:16.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.993+0000 7fb09b52c640 1 -- 172.21.15.154:0/2707148075 shutdown_connections 2023-09-23T03:21:16.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.993+0000 7fb09b52c640 1 --2- 172.21.15.154:0/2707148075 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb07806af60 0x7fb07806d420 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:16.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.993+0000 7fb09b52c640 1 --2- 172.21.15.154:0/2707148075 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb094159d10 0x7fb094150340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:16.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.993+0000 7fb09b52c640 1 --2- 172.21.15.154:0/2707148075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb0940c3500 0x7fb0941edf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:16.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.993+0000 7fb09b52c640 1 --2- 172.21.15.154:0/2707148075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0800039c0 0x7fb0940c4420 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:16.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.993+0000 7fb09b52c640 1 -- 172.21.15.154:0/2707148075 >> 172.21.15.154:0/2707148075 conn(0x7fb09406c010 msgr2=0x7fb09415d950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:16.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.993+0000 7fb09b52c640 1 -- 172.21.15.154:0/2707148075 shutdown_connections 2023-09-23T03:21:16.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:16.993+0000 7fb09b52c640 1 -- 172.21.15.154:0/2707148075 wait complete. 2023-09-23T03:21:17.012 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:21:17.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.229+0000 7fef5ebc0640 1 Processor -- start 2023-09-23T03:21:17.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.230+0000 7fef5ebc0640 1 -- start start 2023-09-23T03:21:17.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.231+0000 7fef5ebc0640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fef580ad830 0x7fef58070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:17.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.231+0000 7fef5ebc0640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fef580c9680 0x7fef580caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:17.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.231+0000 7fef5ebc0640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef580712f0 con 0x7fef580cb400 2023-09-23T03:21:17.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.231+0000 7fef5ebc0640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef580cb870 con 0x7fef580c9680 2023-09-23T03:21:17.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.231+0000 7fef5ebc0640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef580715d0 con 0x7fef580ad830 2023-09-23T03:21:17.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.232+0000 7fef57fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fef580c9680 0x7fef580caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:17.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.232+0000 7fef57fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fef580c9680 0x7fef580caec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:40522/0 (socket says 172.21.15.154:40522) 2023-09-23T03:21:17.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.232+0000 7fef57fff640 1 -- 172.21.15.154:0/1196435346 learned_addr learned my addr 172.21.15.154:0/1196435346 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:17.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.232+0000 7fef57fff640 1 -- 172.21.15.154:0/1196435346 >> v2:172.21.15.190:3300/0 conn(0x7fef580ad830 msgr2=0x7fef58070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:17.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.232+0000 7fef57fff640 1 --2- 172.21.15.154:0/1196435346 >> v2:172.21.15.190:3300/0 conn(0x7fef580ad830 0x7fef58070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:17.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.232+0000 7fef57fff640 1 -- 172.21.15.154:0/1196435346 >> v1:172.21.15.190:6789/0 conn(0x7fef580cb400 legacy=0x7fef58160720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:17.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.232+0000 7fef57fff640 1 -- 172.21.15.154:0/1196435346 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef580718b0 con 0x7fef580c9680 2023-09-23T03:21:17.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.233+0000 7fef57fff640 1 --2- 172.21.15.154:0/1196435346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef580c9680 0x7fef580caec0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fef5806d9c0 tx=0x7fef4802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4629274d570311d2 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:17.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.233+0000 7fef577fe640 1 -- 172.21.15.154:0/1196435346 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef48031870 con 0x7fef580c9680 2023-09-23T03:21:17.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.233+0000 7fef577fe640 1 -- 172.21.15.154:0/1196435346 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef480319d0 con 0x7fef580c9680 2023-09-23T03:21:17.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.233+0000 7fef577fe640 1 -- 172.21.15.154:0/1196435346 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef48031d60 con 0x7fef580c9680 2023-09-23T03:21:17.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.234+0000 7fef5ebc0640 1 -- 172.21.15.154:0/1196435346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef580c9680 msgr2=0x7fef580caec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:17.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.234+0000 7fef5ebc0640 1 --2- 172.21.15.154:0/1196435346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef580c9680 0x7fef580caec0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fef5806d9c0 tx=0x7fef4802b910 comp rx=0 tx=0).stop 2023-09-23T03:21:17.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.234+0000 7fef5ebc0640 1 -- 172.21.15.154:0/1196435346 shutdown_connections 2023-09-23T03:21:17.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.234+0000 7fef5ebc0640 1 --2- 172.21.15.154:0/1196435346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef580c9680 0x7fef580caec0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:17.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.234+0000 7fef5ebc0640 1 --2- 172.21.15.154:0/1196435346 >> v2:172.21.15.190:3300/0 conn(0x7fef580ad830 0x7fef58070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:17.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.234+0000 7fef5ebc0640 1 -- 172.21.15.154:0/1196435346 >> 172.21.15.154:0/1196435346 conn(0x7fef5806c010 msgr2=0x7fef5806c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:17.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.234+0000 7fef5ebc0640 1 -- 172.21.15.154:0/1196435346 shutdown_connections 2023-09-23T03:21:17.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.234+0000 7fef5ebc0640 1 -- 172.21.15.154:0/1196435346 wait complete. 2023-09-23T03:21:17.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.236+0000 7fef5ebc0640 1 Processor -- start 2023-09-23T03:21:17.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.236+0000 7fef5ebc0640 1 -- start start 2023-09-23T03:21:17.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.237+0000 7fef5ebc0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef580ad830 0x7fef580b2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:17.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.237+0000 7fef5ebc0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef580c9680 0x7fef580ce190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:17.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.237+0000 7fef5ebc0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef580cb400 0x7fef580ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:17.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.237+0000 7fef5ebc0640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef580b19c0 con 0x7fef580ad830 2023-09-23T03:21:17.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.237+0000 7fef5ebc0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef58147640 con 0x7fef580cb400 2023-09-23T03:21:17.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.237+0000 7fef5ebc0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef580b1090 con 0x7fef580c9680 2023-09-23T03:21:17.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.237+0000 7fef5c95c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef580ad830 0x7fef580b2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:17.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.237+0000 7fef5c95c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef580ad830 0x7fef580b2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:40532/0 (socket says 172.21.15.154:40532) 2023-09-23T03:21:17.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.237+0000 7fef5c95c640 1 -- 172.21.15.154:0/1167807709 learned_addr learned my addr 172.21.15.154:0/1167807709 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:17.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.238+0000 7fef57fff640 1 --2- 172.21.15.154:0/1167807709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef580c9680 0x7fef580ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:17.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.238+0000 7fef5d15d640 1 --2- 172.21.15.154:0/1167807709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef580cb400 0x7fef580ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:17.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.238+0000 7fef5c95c640 1 -- 172.21.15.154:0/1167807709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef580c9680 msgr2=0x7fef580ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:17.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.238+0000 7fef5c95c640 1 --2- 172.21.15.154:0/1167807709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef580c9680 0x7fef580ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:17.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.238+0000 7fef5c95c640 1 -- 172.21.15.154:0/1167807709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef580cb400 msgr2=0x7fef580ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:17.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.238+0000 7fef5c95c640 1 --2- 172.21.15.154:0/1167807709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef580cb400 0x7fef580ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:17.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.238+0000 7fef5c95c640 1 -- 172.21.15.154:0/1167807709 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef480094d0 con 0x7fef580ad830 2023-09-23T03:21:17.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.238+0000 7fef5d15d640 1 --2- 172.21.15.154:0/1167807709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef580cb400 0x7fef580ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:17.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.238+0000 7fef5c95c640 1 --2- 172.21.15.154:0/1167807709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef580ad830 0x7fef580b2700 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fef4c00b410 tx=0x7fef4c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:17.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.238+0000 7fef557fa640 1 -- 172.21.15.154:0/1167807709 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4c011830 con 0x7fef580ad830 2023-09-23T03:21:17.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.238+0000 7fef557fa640 1 -- 172.21.15.154:0/1167807709 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef4c0119b0 con 0x7fef580ad830 2023-09-23T03:21:17.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.238+0000 7fef557fa640 1 -- 172.21.15.154:0/1167807709 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4c011d60 con 0x7fef580ad830 2023-09-23T03:21:17.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.238+0000 7fef5ebc0640 1 -- 172.21.15.154:0/1167807709 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef580b1370 con 0x7fef580ad830 2023-09-23T03:21:17.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.239+0000 7fef5ebc0640 1 -- 172.21.15.154:0/1167807709 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef580b1e50 con 0x7fef580ad830 2023-09-23T03:21:17.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.240+0000 7fef5ebc0640 1 -- 172.21.15.154:0/1167807709 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef5804e920 con 0x7fef580ad830 2023-09-23T03:21:17.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.241+0000 7fef557fa640 1 -- 172.21.15.154:0/1167807709 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fef4c01a400 con 0x7fef580ad830 2023-09-23T03:21:17.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.241+0000 7fef557fa640 1 --2- 172.21.15.154:0/1167807709 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fef4406af30 0x7fef4406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:17.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.242+0000 7fef557fa640 1 -- 172.21.15.154:0/1167807709 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6433+0+0 (secure 0 0 0) 0x7fef4c089c40 con 0x7fef580ad830 2023-09-23T03:21:17.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.242+0000 7fef57fff640 1 --2- 172.21.15.154:0/1167807709 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fef4406af30 0x7fef4406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:17.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.243+0000 7fef57fff640 1 --2- 172.21.15.154:0/1167807709 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fef4406af30 0x7fef4406d3f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fef48036000 tx=0x7fef4803c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:17.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.246+0000 7fef557fa640 1 -- 172.21.15.154:0/1167807709 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fef4c054900 con 0x7fef580ad830 2023-09-23T03:21:17.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.436+0000 7fef5ebc0640 1 -- 172.21.15.154:0/1167807709 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fef58071b60 con 0x7fef580ad830 2023-09-23T03:21:17.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.437+0000 7fef557fa640 1 -- 172.21.15.154:0/1167807709 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v65) v1 ==== 104+0+11 (secure 0 0 0) 0x7fef4c0150a0 con 0x7fef580ad830 2023-09-23T03:21:17.437 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:21:17.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.439+0000 7fef5ebc0640 1 -- 172.21.15.154:0/1167807709 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fef4406af30 msgr2=0x7fef4406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:17.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.439+0000 7fef5ebc0640 1 --2- 172.21.15.154:0/1167807709 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fef4406af30 0x7fef4406d3f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fef48036000 tx=0x7fef4803c040 comp rx=0 tx=0).stop 2023-09-23T03:21:17.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.439+0000 7fef5ebc0640 1 -- 172.21.15.154:0/1167807709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef580ad830 msgr2=0x7fef580b2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:17.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.439+0000 7fef5ebc0640 1 --2- 172.21.15.154:0/1167807709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef580ad830 0x7fef580b2700 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fef4c00b410 tx=0x7fef4c00b8d0 comp rx=0 tx=0).stop 2023-09-23T03:21:17.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.439+0000 7fef5ebc0640 1 -- 172.21.15.154:0/1167807709 shutdown_connections 2023-09-23T03:21:17.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.439+0000 7fef5ebc0640 1 --2- 172.21.15.154:0/1167807709 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fef4406af30 0x7fef4406d3f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:17.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.439+0000 7fef5ebc0640 1 --2- 172.21.15.154:0/1167807709 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fef580cb400 0x7fef580ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:17.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.439+0000 7fef5ebc0640 1 --2- 172.21.15.154:0/1167807709 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fef580c9680 0x7fef580ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:17.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.439+0000 7fef5ebc0640 1 --2- 172.21.15.154:0/1167807709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fef580ad830 0x7fef580b2700 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:17.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.439+0000 7fef5ebc0640 1 -- 172.21.15.154:0/1167807709 >> 172.21.15.154:0/1167807709 conn(0x7fef5806c010 msgr2=0x7fef580af6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:17.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.440+0000 7fef5ebc0640 1 -- 172.21.15.154:0/1167807709 shutdown_connections 2023-09-23T03:21:17.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.440+0000 7fef5ebc0640 1 -- 172.21.15.154:0/1167807709 wait complete. 2023-09-23T03:21:17.457 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:21:17.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.663+0000 7f0610afc640 1 Processor -- start 2023-09-23T03:21:17.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.663+0000 7f0610afc640 1 -- start start 2023-09-23T03:21:17.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.665+0000 7f0610afc640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f060c15c9c0 0x7f060c0d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:17.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.665+0000 7f0610afc640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f060c0cd450 con 0x7f060c0d2d30 2023-09-23T03:21:17.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.665+0000 7f0610afc640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f060c0d0df0 con 0x7f060c15cda0 2023-09-23T03:21:17.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.665+0000 7f0610afc640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f060c0d10d0 con 0x7f060c15c9c0 2023-09-23T03:21:17.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.665+0000 7f060ad9d640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f060c0d2d30 0x7f060c0cd930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:49292/0 (socket says 172.21.15.154:49292) 2023-09-23T03:21:17.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.665+0000 7f060ad9d640 1 -- 172.21.15.154:0/2690718054 learned_addr learned my addr 172.21.15.154:0/2690718054 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:21:17.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.666+0000 7f0609d9b640 1 -- 172.21.15.154:0/2690718054 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4059584788 0 0) 0x7f060c0cd450 con 0x7f060c0d2d30 2023-09-23T03:21:17.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.666+0000 7f060a59c640 1 --2- 172.21.15.154:0/2690718054 >> v2:172.21.15.190:3300/0 conn(0x7f060c15c9c0 0x7f060c0d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:17.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.666+0000 7f0609d9b640 1 -- 172.21.15.154:0/2690718054 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05f8003520 con 0x7f060c0d2d30 2023-09-23T03:21:17.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.666+0000 7f0609d9b640 1 -- 172.21.15.154:0/2690718054 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1373928668 0 0) 0x7f05f8003520 con 0x7f060c0d2d30 2023-09-23T03:21:17.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.666+0000 7f0609d9b640 1 -- 172.21.15.154:0/2690718054 >> v2:172.21.15.190:3300/0 conn(0x7f060c15c9c0 msgr2=0x7f060c0d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:17.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.666+0000 7f0609d9b640 1 --2- 172.21.15.154:0/2690718054 >> v2:172.21.15.190:3300/0 conn(0x7f060c15c9c0 0x7f060c0d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:17.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.666+0000 7f0609d9b640 1 -- 172.21.15.154:0/2690718054 >> v1:172.21.15.190:6789/0 conn(0x7f060c15cda0 legacy=0x7f060c0d2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:17.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.666+0000 7f0609d9b640 1 -- 172.21.15.154:0/2690718054 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f060c0d13b0 con 0x7f060c0d2d30 2023-09-23T03:21:17.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.666+0000 7f0609d9b640 1 -- 172.21.15.154:0/2690718054 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f06040019f0 con 0x7f060c0d2d30 2023-09-23T03:21:17.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.666+0000 7f0609d9b640 1 -- 172.21.15.154:0/2690718054 >> v1:172.21.15.154:6789/0 conn(0x7f060c0d2d30 legacy=0x7f060c0cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:17.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.666+0000 7f0609d9b640 1 --2- 172.21.15.154:0/2690718054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05f8003970 0x7f05f8003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:17.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.666+0000 7f0609d9b640 1 --2- 172.21.15.154:0/2690718054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05f80047c0 0x7f05f8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:17.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.666+0000 7f0609d9b640 1 --2- 172.21.15.154:0/2690718054 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05f80052a0 0x7f05f800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:17.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.666+0000 7f0609d9b640 1 -- 172.21.15.154:0/2690718054 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060c0d10d0 con 0x7f05f80047c0 2023-09-23T03:21:17.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.666+0000 7f0609d9b640 1 -- 172.21.15.154:0/2690718054 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f800c150 con 0x7f05f8003970 2023-09-23T03:21:17.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.666+0000 7f0609d9b640 1 -- 172.21.15.154:0/2690718054 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05f800c310 con 0x7f05f80052a0 2023-09-23T03:21:17.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.667+0000 7f0603fff640 1 --2- 172.21.15.154:0/2690718054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05f80047c0 0x7f05f8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:17.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.667+0000 7f060a59c640 1 --2- 172.21.15.154:0/2690718054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05f8003970 0x7f05f8003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:17.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.667+0000 7f060ad9d640 1 --2- 172.21.15.154:0/2690718054 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05f80052a0 0x7f05f800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:17.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.667+0000 7f0603fff640 1 -- 172.21.15.154:0/2690718054 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05f80052a0 msgr2=0x7f05f800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:17.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.667+0000 7f0603fff640 1 --2- 172.21.15.154:0/2690718054 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05f80052a0 0x7f05f800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:17.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.667+0000 7f0603fff640 1 -- 172.21.15.154:0/2690718054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05f8003970 msgr2=0x7f05f8003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:17.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.667+0000 7f0603fff640 1 --2- 172.21.15.154:0/2690718054 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05f8003970 0x7f05f8003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:17.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.667+0000 7f0603fff640 1 -- 172.21.15.154:0/2690718054 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f8003520 con 0x7f05f80047c0 2023-09-23T03:21:17.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.667+0000 7f060ad9d640 1 -- 172.21.15.154:0/2690718054 reap_dead start 2023-09-23T03:21:17.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.668+0000 7f0603fff640 1 --2- 172.21.15.154:0/2690718054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05f80047c0 0x7f05f8004c40 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f05f4003d80 tx=0x7f05f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d60b3c051ef9547d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:17.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.668+0000 7f0609d9b640 1 -- 172.21.15.154:0/2690718054 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f4032600 con 0x7f05f80047c0 2023-09-23T03:21:17.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.668+0000 7f0609d9b640 1 -- 172.21.15.154:0/2690718054 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f402e050 con 0x7f05f80047c0 2023-09-23T03:21:17.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.668+0000 7f0609d9b640 1 -- 172.21.15.154:0/2690718054 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f40329e0 con 0x7f05f80047c0 2023-09-23T03:21:17.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.669+0000 7f0610afc640 1 -- 172.21.15.154:0/2690718054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05f80047c0 msgr2=0x7f05f8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:17.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.669+0000 7f0610afc640 1 --2- 172.21.15.154:0/2690718054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05f80047c0 0x7f05f8004c40 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f05f4003d80 tx=0x7f05f80057f0 comp rx=0 tx=0).stop 2023-09-23T03:21:17.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.669+0000 7f0610afc640 1 -- 172.21.15.154:0/2690718054 shutdown_connections 2023-09-23T03:21:17.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.669+0000 7f0610afc640 1 --2- 172.21.15.154:0/2690718054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05f80047c0 0x7f05f8004c40 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:17.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.669+0000 7f0610afc640 1 -- 172.21.15.154:0/2690718054 >> 172.21.15.154:0/2690718054 conn(0x7f060c09bc20 msgr2=0x7f060c089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:17.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.669+0000 7f0610afc640 1 -- 172.21.15.154:0/2690718054 shutdown_connections 2023-09-23T03:21:17.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.669+0000 7f0610afc640 1 -- 172.21.15.154:0/2690718054 wait complete. 2023-09-23T03:21:17.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.671+0000 7f0610afc640 1 Processor -- start 2023-09-23T03:21:17.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.671+0000 7f0610afc640 1 -- start start 2023-09-23T03:21:17.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.672+0000 7f0610afc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05f80047c0 0x7f060c15fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:17.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.672+0000 7f0610afc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f060c0d0900 0x7f060c0c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:17.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.672+0000 7f0610afc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f060c0cf2a0 0x7f060c0c6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:17.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.672+0000 7f0610afc640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060c1601a0 con 0x7f05f80047c0 2023-09-23T03:21:17.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.672+0000 7f0610afc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060c0cf860 con 0x7f060c0cf2a0 2023-09-23T03:21:17.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.672+0000 7f0610afc640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f060c0cfb40 con 0x7f060c0d0900 2023-09-23T03:21:17.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.672+0000 7f060a59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05f80047c0 0x7f060c15fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:17.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.672+0000 7f060a59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05f80047c0 0x7f060c15fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:40546/0 (socket says 172.21.15.154:40546) 2023-09-23T03:21:17.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.672+0000 7f060a59c640 1 -- 172.21.15.154:0/228151939 learned_addr learned my addr 172.21.15.154:0/228151939 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:17.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.673+0000 7f060a59c640 1 -- 172.21.15.154:0/228151939 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f060c0d0900 msgr2=0x7f060c0c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:17.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.673+0000 7f060a59c640 1 --2- 172.21.15.154:0/228151939 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f060c0d0900 0x7f060c0c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:17.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.673+0000 7f060a59c640 1 -- 172.21.15.154:0/228151939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f060c0cf2a0 msgr2=0x7f060c0c6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:17.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.673+0000 7f060a59c640 1 --2- 172.21.15.154:0/228151939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f060c0cf2a0 0x7f060c0c6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:17.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.673+0000 7f060a59c640 1 -- 172.21.15.154:0/228151939 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f060c0cfe20 con 0x7f05f80047c0 2023-09-23T03:21:17.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.673+0000 7f060a59c640 1 --2- 172.21.15.154:0/228151939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05f80047c0 0x7f060c15fc60 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f05fc00f4c0 tx=0x7f05fc00f980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:17.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.673+0000 7f0602ffd640 1 -- 172.21.15.154:0/228151939 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc005ba0 con 0x7f05f80047c0 2023-09-23T03:21:17.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.673+0000 7f0602ffd640 1 -- 172.21.15.154:0/228151939 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05fc005d00 con 0x7f05f80047c0 2023-09-23T03:21:17.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.673+0000 7f0610afc640 1 -- 172.21.15.154:0/228151939 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f060c0d0100 con 0x7f05f80047c0 2023-09-23T03:21:17.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.673+0000 7f0602ffd640 1 -- 172.21.15.154:0/228151939 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc004240 con 0x7f05f80047c0 2023-09-23T03:21:17.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.673+0000 7f0610afc640 1 -- 172.21.15.154:0/228151939 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f060c0c9290 con 0x7f05f80047c0 2023-09-23T03:21:17.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.675+0000 7f0610afc640 1 -- 172.21.15.154:0/228151939 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f060c04e920 con 0x7f05f80047c0 2023-09-23T03:21:17.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.675+0000 7f0602ffd640 1 -- 172.21.15.154:0/228151939 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f05fc0043a0 con 0x7f05f80047c0 2023-09-23T03:21:17.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.676+0000 7f0602ffd640 1 --2- 172.21.15.154:0/228151939 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f05ec06af60 0x7f05ec06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:17.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.676+0000 7f0602ffd640 1 -- 172.21.15.154:0/228151939 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f05fc089ba0 con 0x7f05f80047c0 2023-09-23T03:21:17.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.677+0000 7f0603fff640 1 --2- 172.21.15.154:0/228151939 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f05ec06af60 0x7f05ec06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:17.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.677+0000 7f0603fff640 1 --2- 172.21.15.154:0/228151939 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f05ec06af60 0x7f05ec06d420 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f05f4034000 tx=0x7f05f402f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:17.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.681+0000 7f0602ffd640 1 -- 172.21.15.154:0/228151939 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f05fc054900 con 0x7f05f80047c0 2023-09-23T03:21:17.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.871+0000 7f0610afc640 1 -- 172.21.15.154:0/228151939 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f060c0b22c0 con 0x7f05f80047c0 2023-09-23T03:21:17.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.872+0000 7f0602ffd640 1 -- 172.21.15.154:0/228151939 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v65) v1 ==== 100+0+11 (secure 0 0 0) 0x7f05fc0180a0 con 0x7f05f80047c0 2023-09-23T03:21:17.873 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:21:17.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.875+0000 7f0610afc640 1 -- 172.21.15.154:0/228151939 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f05ec06af60 msgr2=0x7f05ec06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:17.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.875+0000 7f0610afc640 1 --2- 172.21.15.154:0/228151939 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f05ec06af60 0x7f05ec06d420 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f05f4034000 tx=0x7f05f402f000 comp rx=0 tx=0).stop 2023-09-23T03:21:17.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.875+0000 7f0610afc640 1 -- 172.21.15.154:0/228151939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05f80047c0 msgr2=0x7f060c15fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:17.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.875+0000 7f0610afc640 1 --2- 172.21.15.154:0/228151939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05f80047c0 0x7f060c15fc60 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f05fc00f4c0 tx=0x7f05fc00f980 comp rx=0 tx=0).stop 2023-09-23T03:21:17.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.875+0000 7f0610afc640 1 -- 172.21.15.154:0/228151939 shutdown_connections 2023-09-23T03:21:17.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.875+0000 7f0610afc640 1 --2- 172.21.15.154:0/228151939 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f05ec06af60 0x7f05ec06d420 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:17.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.875+0000 7f0610afc640 1 --2- 172.21.15.154:0/228151939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f060c0cf2a0 0x7f060c0c6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:17.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.875+0000 7f0610afc640 1 --2- 172.21.15.154:0/228151939 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f060c0d0900 0x7f060c0c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:17.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.875+0000 7f0610afc640 1 --2- 172.21.15.154:0/228151939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05f80047c0 0x7f060c15fc60 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:17.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.875+0000 7f0610afc640 1 -- 172.21.15.154:0/228151939 >> 172.21.15.154:0/228151939 conn(0x7f060c09bc20 msgr2=0x7f060c15bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:17.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.875+0000 7f0610afc640 1 -- 172.21.15.154:0/228151939 shutdown_connections 2023-09-23T03:21:17.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:17.875+0000 7f0610afc640 1 -- 172.21.15.154:0/228151939 wait complete. 2023-09-23T03:21:17.894 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-09-23T03:21:18.108 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.107+0000 7f781efd0640 1 Processor -- start 2023-09-23T03:21:18.108 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.108+0000 7f781efd0640 1 -- start start 2023-09-23T03:21:18.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.109+0000 7f781efd0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f781815c9c0 0x7f78180d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:18.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.109+0000 7f781efd0640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f781815cda0 0x7f78180d2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:18.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.109+0000 7f781efd0640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f78180d2b60 0x7f78180cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:18.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.109+0000 7f781efd0640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78180cd450 con 0x7f781815cda0 2023-09-23T03:21:18.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.109+0000 7f781efd0640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78180d0df0 con 0x7f78180d2b60 2023-09-23T03:21:18.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.109+0000 7f781efd0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78180d10d0 con 0x7f781815c9c0 2023-09-23T03:21:18.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.110+0000 7f7817fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f781815cda0 0x7f78180d2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:18.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.110+0000 7f7817fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f781815cda0 0x7f78180d2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:40558/0 (socket says 172.21.15.154:40558) 2023-09-23T03:21:18.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.110+0000 7f7817fff640 1 -- 172.21.15.154:0/2185422466 learned_addr learned my addr 172.21.15.154:0/2185422466 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:18.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.110+0000 7f781cd6c640 1 --2- 172.21.15.154:0/2185422466 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f781815c9c0 0x7f78180d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:18.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.110+0000 7f7817fff640 1 -- 172.21.15.154:0/2185422466 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f781815c9c0 msgr2=0x7f78180d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:18.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.110+0000 7f7817fff640 1 --2- 172.21.15.154:0/2185422466 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f781815c9c0 0x7f78180d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:18.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.110+0000 7f7817fff640 1 -- 172.21.15.154:0/2185422466 >> v2:172.21.15.190:3300/0 conn(0x7f78180d2b60 msgr2=0x7f78180cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:18.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.110+0000 7f781d56d640 1 --2- 172.21.15.154:0/2185422466 >> v2:172.21.15.190:3300/0 conn(0x7f78180d2b60 0x7f78180cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:18.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.110+0000 7f7817fff640 1 --2- 172.21.15.154:0/2185422466 >> v2:172.21.15.190:3300/0 conn(0x7f78180d2b60 0x7f78180cf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:18.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.110+0000 7f7817fff640 1 -- 172.21.15.154:0/2185422466 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78180d13b0 con 0x7f781815cda0 2023-09-23T03:21:18.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.111+0000 7f7817fff640 1 --2- 172.21.15.154:0/2185422466 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f781815cda0 0x7f78180d2620 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f781809c330 tx=0x7f780402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a36730490993b66 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:18.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.112+0000 7f78177fe640 1 -- 172.21.15.154:0/2185422466 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7804031870 con 0x7f781815cda0 2023-09-23T03:21:18.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.112+0000 7f78177fe640 1 -- 172.21.15.154:0/2185422466 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78040319d0 con 0x7f781815cda0 2023-09-23T03:21:18.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.112+0000 7f78177fe640 1 -- 172.21.15.154:0/2185422466 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7804031d60 con 0x7f781815cda0 2023-09-23T03:21:18.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.112+0000 7f781efd0640 1 -- 172.21.15.154:0/2185422466 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f781815cda0 msgr2=0x7f78180d2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:18.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.112+0000 7f781efd0640 1 --2- 172.21.15.154:0/2185422466 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f781815cda0 0x7f78180d2620 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f781809c330 tx=0x7f780402b910 comp rx=0 tx=0).stop 2023-09-23T03:21:18.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.112+0000 7f781efd0640 1 -- 172.21.15.154:0/2185422466 shutdown_connections 2023-09-23T03:21:18.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.112+0000 7f781efd0640 1 --2- 172.21.15.154:0/2185422466 >> v2:172.21.15.190:3300/0 conn(0x7f78180d2b60 0x7f78180cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:18.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.112+0000 7f781efd0640 1 --2- 172.21.15.154:0/2185422466 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f781815cda0 0x7f78180d2620 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:18.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.112+0000 7f781efd0640 1 --2- 172.21.15.154:0/2185422466 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f781815c9c0 0x7f78180d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:18.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.112+0000 7f781efd0640 1 -- 172.21.15.154:0/2185422466 >> 172.21.15.154:0/2185422466 conn(0x7f781809bc20 msgr2=0x7f7818089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:18.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.112+0000 7f781efd0640 1 -- 172.21.15.154:0/2185422466 shutdown_connections 2023-09-23T03:21:18.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.113+0000 7f781efd0640 1 -- 172.21.15.154:0/2185422466 wait complete. 2023-09-23T03:21:18.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.114+0000 7f781efd0640 1 Processor -- start 2023-09-23T03:21:18.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.114+0000 7f781efd0640 1 -- start start 2023-09-23T03:21:18.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.115+0000 7f781efd0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78180d2b60 0x7f781815f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:18.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.115+0000 7f781efd0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f781815c9c0 0x7f78180c7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:18.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.115+0000 7f781efd0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f781815cda0 0x7f78181f54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:18.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.115+0000 7f781efd0640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f781815f050 con 0x7f781815cda0 2023-09-23T03:21:18.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.115+0000 7f781efd0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f781815e440 con 0x7f781815c9c0 2023-09-23T03:21:18.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.115+0000 7f781efd0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f781815e720 con 0x7f78180d2b60 2023-09-23T03:21:18.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.116+0000 7f781d56d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f781815cda0 0x7f78181f54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:18.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.116+0000 7f781d56d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f781815cda0 0x7f78181f54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:40568/0 (socket says 172.21.15.154:40568) 2023-09-23T03:21:18.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.116+0000 7f781d56d640 1 -- 172.21.15.154:0/568990394 learned_addr learned my addr 172.21.15.154:0/568990394 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:18.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.116+0000 7f7817fff640 1 --2- 172.21.15.154:0/568990394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f781815c9c0 0x7f78180c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:18.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.116+0000 7f781cd6c640 1 --2- 172.21.15.154:0/568990394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78180d2b60 0x7f781815f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:18.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.116+0000 7f781d56d640 1 -- 172.21.15.154:0/568990394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78180d2b60 msgr2=0x7f781815f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:18.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.116+0000 7f781d56d640 1 --2- 172.21.15.154:0/568990394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78180d2b60 0x7f781815f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:18.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.116+0000 7f781d56d640 1 -- 172.21.15.154:0/568990394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f781815c9c0 msgr2=0x7f78180c7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:18.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.116+0000 7f781d56d640 1 --2- 172.21.15.154:0/568990394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f781815c9c0 0x7f78180c7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:18.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.116+0000 7f781d56d640 1 -- 172.21.15.154:0/568990394 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78040094d0 con 0x7f781815cda0 2023-09-23T03:21:18.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.116+0000 7f781cd6c640 1 --2- 172.21.15.154:0/568990394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78180d2b60 0x7f781815f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:18.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.116+0000 7f781d56d640 1 --2- 172.21.15.154:0/568990394 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f781815cda0 0x7f78181f54e0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f781000cc30 tx=0x7f781000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:18.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.117+0000 7f78157fa640 1 -- 172.21.15.154:0/568990394 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781000ce40 con 0x7f781815cda0 2023-09-23T03:21:18.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.117+0000 7f78157fa640 1 -- 172.21.15.154:0/568990394 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7810003990 con 0x7f781815cda0 2023-09-23T03:21:18.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.117+0000 7f781efd0640 1 -- 172.21.15.154:0/568990394 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f781815ea00 con 0x7f781815cda0 2023-09-23T03:21:18.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.117+0000 7f78157fa640 1 -- 172.21.15.154:0/568990394 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7810003d20 con 0x7f781815cda0 2023-09-23T03:21:18.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.117+0000 7f781efd0640 1 -- 172.21.15.154:0/568990394 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f781815fa20 con 0x7f781815cda0 2023-09-23T03:21:18.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.118+0000 7f781efd0640 1 -- 172.21.15.154:0/568990394 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f781815f1c0 con 0x7f781815cda0 2023-09-23T03:21:18.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.119+0000 7f78157fa640 1 -- 172.21.15.154:0/568990394 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f7810003e80 con 0x7f781815cda0 2023-09-23T03:21:18.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.120+0000 7f78157fa640 1 --2- 172.21.15.154:0/568990394 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f77fc06af30 0x7f77fc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:18.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.120+0000 7f78157fa640 1 -- 172.21.15.154:0/568990394 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f7810089ea0 con 0x7f781815cda0 2023-09-23T03:21:18.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.120+0000 7f781cd6c640 1 --2- 172.21.15.154:0/568990394 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f77fc06af30 0x7f77fc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:18.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.121+0000 7f781cd6c640 1 --2- 172.21.15.154:0/568990394 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f77fc06af30 0x7f77fc06d3f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f780c009700 tx=0x7f780c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:18.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.124+0000 7f78157fa640 1 -- 172.21.15.154:0/568990394 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f7810054be0 con 0x7f781815cda0 2023-09-23T03:21:18.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.315+0000 7f781efd0640 1 -- 172.21.15.154:0/568990394 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7f781804e920 con 0x7f781815cda0 2023-09-23T03:21:18.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.316+0000 7f78157fa640 1 -- 172.21.15.154:0/568990394 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v65) v1 ==== 106+0+11 (secure 0 0 0) 0x7f781001d070 con 0x7f781815cda0 2023-09-23T03:21:18.317 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:21:18.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.319+0000 7f781efd0640 1 -- 172.21.15.154:0/568990394 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f77fc06af30 msgr2=0x7f77fc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:18.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.319+0000 7f781efd0640 1 --2- 172.21.15.154:0/568990394 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f77fc06af30 0x7f77fc06d3f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f780c009700 tx=0x7f780c008040 comp rx=0 tx=0).stop 2023-09-23T03:21:18.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.319+0000 7f781efd0640 1 -- 172.21.15.154:0/568990394 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f781815cda0 msgr2=0x7f78181f54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:18.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.319+0000 7f781efd0640 1 --2- 172.21.15.154:0/568990394 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f781815cda0 0x7f78181f54e0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f781000cc30 tx=0x7f781000a5c0 comp rx=0 tx=0).stop 2023-09-23T03:21:18.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.319+0000 7f781efd0640 1 -- 172.21.15.154:0/568990394 shutdown_connections 2023-09-23T03:21:18.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.319+0000 7f781efd0640 1 --2- 172.21.15.154:0/568990394 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f77fc06af30 0x7f77fc06d3f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:18.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.319+0000 7f781efd0640 1 --2- 172.21.15.154:0/568990394 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f781815cda0 0x7f78181f54e0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:18.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.319+0000 7f781efd0640 1 --2- 172.21.15.154:0/568990394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f781815c9c0 0x7f78180c7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:18.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.319+0000 7f781efd0640 1 --2- 172.21.15.154:0/568990394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78180d2b60 0x7f781815f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:18.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.319+0000 7f781efd0640 1 -- 172.21.15.154:0/568990394 >> 172.21.15.154:0/568990394 conn(0x7f781809bc20 msgr2=0x7f78180b4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:18.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.319+0000 7f781efd0640 1 -- 172.21.15.154:0/568990394 shutdown_connections 2023-09-23T03:21:18.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.319+0000 7f781efd0640 1 -- 172.21.15.154:0/568990394 wait complete. 2023-09-23T03:21:18.339 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-23T03:21:18.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.551+0000 7f8741b5e640 1 Processor -- start 2023-09-23T03:21:18.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.551+0000 7f8741b5e640 1 -- start start 2023-09-23T03:21:18.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.553+0000 7f8741b5e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f873c0cf500 0x7f873c0ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:18.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.553+0000 7f8741b5e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f873c06be10 con 0x7f873c0cf8e0 2023-09-23T03:21:18.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.553+0000 7f8741b5e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f873c0d17f0 con 0x7f873c15cc00 2023-09-23T03:21:18.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.553+0000 7f8741b5e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f873c0d0470 con 0x7f873c0cf500 2023-09-23T03:21:18.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.553+0000 7f8740b5c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f873c0cf500 0x7f873c0ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:18.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.553+0000 7f873bfff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f873c0cf8e0 0x7f873c15c4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:49306/0 (socket says 172.21.15.154:49306) 2023-09-23T03:21:18.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.553+0000 7f873bfff640 1 -- 172.21.15.154:0/3867720057 learned_addr learned my addr 172.21.15.154:0/3867720057 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:21:18.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.553+0000 7f8740b5c640 1 -- 172.21.15.154:0/3867720057 >> v1:172.21.15.190:6789/0 conn(0x7f873c15cc00 legacy=0x7f873c0b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:18.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.554+0000 7f8740b5c640 1 -- 172.21.15.154:0/3867720057 >> v1:172.21.15.154:6789/0 conn(0x7f873c0cf8e0 legacy=0x7f873c15c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:18.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.554+0000 7f8740b5c640 1 -- 172.21.15.154:0/3867720057 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f873c0d0750 con 0x7f873c0cf500 2023-09-23T03:21:18.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.554+0000 7f873b7fe640 1 -- 172.21.15.154:0/3867720057 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 350564863 0 0) 0x7f873c06be10 con 0x7f873c0cf8e0 2023-09-23T03:21:18.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.555+0000 7f8740b5c640 1 --2- 172.21.15.154:0/3867720057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f873c0cf500 0x7f873c0ce050 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f872c0097c0 tx=0x7f872c02b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd75225cf48564 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:18.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.555+0000 7f873b7fe640 1 -- 172.21.15.154:0/3867720057 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c031770 con 0x7f873c0cf500 2023-09-23T03:21:18.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.555+0000 7f873b7fe640 1 -- 172.21.15.154:0/3867720057 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872c0318d0 con 0x7f873c0cf500 2023-09-23T03:21:18.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.555+0000 7f873b7fe640 1 -- 172.21.15.154:0/3867720057 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c031c60 con 0x7f873c0cf500 2023-09-23T03:21:18.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.555+0000 7f8741b5e640 1 -- 172.21.15.154:0/3867720057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f873c0cf500 msgr2=0x7f873c0ce050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:18.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.555+0000 7f8741b5e640 1 --2- 172.21.15.154:0/3867720057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f873c0cf500 0x7f873c0ce050 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f872c0097c0 tx=0x7f872c02b810 comp rx=0 tx=0).stop 2023-09-23T03:21:18.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.555+0000 7f8741b5e640 1 -- 172.21.15.154:0/3867720057 shutdown_connections 2023-09-23T03:21:18.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.555+0000 7f8741b5e640 1 --2- 172.21.15.154:0/3867720057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f873c0cf500 0x7f873c0ce050 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:18.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.556+0000 7f8741b5e640 1 -- 172.21.15.154:0/3867720057 >> 172.21.15.154:0/3867720057 conn(0x7f873c09bbb0 msgr2=0x7f873c0d28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:18.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.556+0000 7f8741b5e640 1 -- 172.21.15.154:0/3867720057 shutdown_connections 2023-09-23T03:21:18.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.556+0000 7f8741b5e640 1 -- 172.21.15.154:0/3867720057 wait complete. 2023-09-23T03:21:18.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.557+0000 7f8741b5e640 1 Processor -- start 2023-09-23T03:21:18.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.557+0000 7f8741b5e640 1 -- start start 2023-09-23T03:21:18.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.558+0000 7f8741b5e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f873c0cf500 0x7f873c143800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:18.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.558+0000 7f8741b5e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f873c0cf8e0 0x7f873c15f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:18.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.558+0000 7f8741b5e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f873c15cc00 0x7f873c15fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:18.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.558+0000 7f8741b5e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c1444a0 con 0x7f873c0cf500 2023-09-23T03:21:18.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.558+0000 7f8741b5e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c0c6780 con 0x7f873c0cf8e0 2023-09-23T03:21:18.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.558+0000 7f8741b5e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f873c143d40 con 0x7f873c15cc00 2023-09-23T03:21:18.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.559+0000 7f8740b5c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f873c0cf500 0x7f873c143800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:18.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.559+0000 7f8740b5c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f873c0cf500 0x7f873c143800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:40588/0 (socket says 172.21.15.154:40588) 2023-09-23T03:21:18.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.559+0000 7f8740b5c640 1 -- 172.21.15.154:0/3832903349 learned_addr learned my addr 172.21.15.154:0/3832903349 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:18.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.559+0000 7f873bfff640 1 --2- 172.21.15.154:0/3832903349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f873c0cf8e0 0x7f873c15f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:18.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.559+0000 7f874135d640 1 --2- 172.21.15.154:0/3832903349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f873c15cc00 0x7f873c15fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:18.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.559+0000 7f8740b5c640 1 -- 172.21.15.154:0/3832903349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f873c15cc00 msgr2=0x7f873c15fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:18.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.559+0000 7f8740b5c640 1 --2- 172.21.15.154:0/3832903349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f873c15cc00 0x7f873c15fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:18.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.559+0000 7f8740b5c640 1 -- 172.21.15.154:0/3832903349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f873c0cf8e0 msgr2=0x7f873c15f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:18.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.559+0000 7f8740b5c640 1 --2- 172.21.15.154:0/3832903349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f873c0cf8e0 0x7f873c15f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:18.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.559+0000 7f8740b5c640 1 -- 172.21.15.154:0/3832903349 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f872c0093a0 con 0x7f873c0cf500 2023-09-23T03:21:18.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.559+0000 7f873bfff640 1 --2- 172.21.15.154:0/3832903349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f873c0cf8e0 0x7f873c15f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:18.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.559+0000 7f874135d640 1 --2- 172.21.15.154:0/3832903349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f873c15cc00 0x7f873c15fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:18.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.560+0000 7f8740b5c640 1 --2- 172.21.15.154:0/3832903349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f873c0cf500 0x7f873c143800 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f872c0098e0 tx=0x7f872c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:18.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.560+0000 7f87397fa640 1 -- 172.21.15.154:0/3832903349 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c005da0 con 0x7f873c0cf500 2023-09-23T03:21:18.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.560+0000 7f87397fa640 1 -- 172.21.15.154:0/3832903349 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872c006030 con 0x7f873c0cf500 2023-09-23T03:21:18.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.560+0000 7f87397fa640 1 -- 172.21.15.154:0/3832903349 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c0063e0 con 0x7f873c0cf500 2023-09-23T03:21:18.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.560+0000 7f8741b5e640 1 -- 172.21.15.154:0/3832903349 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f873c0c8ee0 con 0x7f873c0cf500 2023-09-23T03:21:18.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.560+0000 7f8741b5e640 1 -- 172.21.15.154:0/3832903349 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f873c0c94a0 con 0x7f873c0cf500 2023-09-23T03:21:18.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.561+0000 7f8741b5e640 1 -- 172.21.15.154:0/3832903349 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f873c04e920 con 0x7f873c0cf500 2023-09-23T03:21:18.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.562+0000 7f87397fa640 1 -- 172.21.15.154:0/3832903349 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f872c006540 con 0x7f873c0cf500 2023-09-23T03:21:18.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.563+0000 7f87397fa640 1 --2- 172.21.15.154:0/3832903349 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f871c06af30 0x7f871c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:18.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.563+0000 7f87397fa640 1 -- 172.21.15.154:0/3832903349 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f872c0abee0 con 0x7f873c0cf500 2023-09-23T03:21:18.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.563+0000 7f873bfff640 1 --2- 172.21.15.154:0/3832903349 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f871c06af30 0x7f871c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:18.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.564+0000 7f873bfff640 1 --2- 172.21.15.154:0/3832903349 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f871c06af30 0x7f871c06d3f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f873c14a730 tx=0x7f8724009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:18.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.567+0000 7f87397fa640 1 -- 172.21.15.154:0/3832903349 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f872c076cc0 con 0x7f873c0cf500 2023-09-23T03:21:18.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.769+0000 7f8741b5e640 1 -- 172.21.15.154:0/3832903349 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f873c0d0000 con 0x7f873c0cf500 2023-09-23T03:21:18.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.770+0000 7f87397fa640 1 -- 172.21.15.154:0/3832903349 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v65) v1 ==== 80+0+0 (secure 0 0 0) 0x7f872c04b070 con 0x7f873c0cf500 2023-09-23T03:21:18.771 INFO:teuthology.orchestra.run.smithi154.stderr:listed 0 entries 2023-09-23T03:21:18.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.773+0000 7f8741b5e640 1 -- 172.21.15.154:0/3832903349 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f871c06af30 msgr2=0x7f871c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:18.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.773+0000 7f8741b5e640 1 --2- 172.21.15.154:0/3832903349 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f871c06af30 0x7f871c06d3f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f873c14a730 tx=0x7f8724009040 comp rx=0 tx=0).stop 2023-09-23T03:21:18.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.773+0000 7f8741b5e640 1 -- 172.21.15.154:0/3832903349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f873c0cf500 msgr2=0x7f873c143800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:18.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.773+0000 7f8741b5e640 1 --2- 172.21.15.154:0/3832903349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f873c0cf500 0x7f873c143800 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f872c0098e0 tx=0x7f872c0046c0 comp rx=0 tx=0).stop 2023-09-23T03:21:18.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.773+0000 7f8741b5e640 1 -- 172.21.15.154:0/3832903349 shutdown_connections 2023-09-23T03:21:18.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.773+0000 7f8741b5e640 1 --2- 172.21.15.154:0/3832903349 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f871c06af30 0x7f871c06d3f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:18.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.773+0000 7f8741b5e640 1 --2- 172.21.15.154:0/3832903349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f873c15cc00 0x7f873c15fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:18.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.773+0000 7f8741b5e640 1 --2- 172.21.15.154:0/3832903349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f873c0cf8e0 0x7f873c15f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:18.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.773+0000 7f8741b5e640 1 --2- 172.21.15.154:0/3832903349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f873c0cf500 0x7f873c143800 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:18.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.773+0000 7f8741b5e640 1 -- 172.21.15.154:0/3832903349 >> 172.21.15.154:0/3832903349 conn(0x7f873c09bbb0 msgr2=0x7f873c15d030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:18.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.773+0000 7f8741b5e640 1 -- 172.21.15.154:0/3832903349 shutdown_connections 2023-09-23T03:21:18.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:18.773+0000 7f8741b5e640 1 -- 172.21.15.154:0/3832903349 wait complete. 2023-09-23T03:21:18.792 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:21:18.792 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:21:18.844 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:21:19.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.062+0000 7f47eeefc640 1 Processor -- start 2023-09-23T03:21:19.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.062+0000 7f47eeefc640 1 -- start start 2023-09-23T03:21:19.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.064+0000 7f47eeefc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47e80acce0 0x7f47e815ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:19.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.064+0000 7f47eeefc640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f47e80b1b20 0x7f47e80ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:19.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.064+0000 7f47eeefc640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f47e80cee70 0x7f47e806cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:19.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.064+0000 7f47eeefc640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47e80b2e90 con 0x7f47e80b1b20 2023-09-23T03:21:19.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.064+0000 7f47eeefc640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47e80cf320 con 0x7f47e80cee70 2023-09-23T03:21:19.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.064+0000 7f47eeefc640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e815ac30 con 0x7f47e80acce0 2023-09-23T03:21:19.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.064+0000 7f47e7fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f47e80b1b20 0x7f47e80ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:19.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.064+0000 7f47e7fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f47e80b1b20 0x7f47e80ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:40590/0 (socket says 172.21.15.154:40590) 2023-09-23T03:21:19.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.064+0000 7f47e7fff640 1 -- 172.21.15.154:0/2406578515 learned_addr learned my addr 172.21.15.154:0/2406578515 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:19.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.064+0000 7f47ed499640 1 --2- 172.21.15.154:0/2406578515 >> v2:172.21.15.190:3300/0 conn(0x7f47e80cee70 0x7f47e806cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:19.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.064+0000 7f47ecc98640 1 --2- 172.21.15.154:0/2406578515 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47e80acce0 0x7f47e815ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:19.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.065+0000 7f47e7fff640 1 -- 172.21.15.154:0/2406578515 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47e80acce0 msgr2=0x7f47e815ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:19.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.065+0000 7f47e7fff640 1 --2- 172.21.15.154:0/2406578515 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47e80acce0 0x7f47e815ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:19.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.065+0000 7f47e7fff640 1 -- 172.21.15.154:0/2406578515 >> v2:172.21.15.190:3300/0 conn(0x7f47e80cee70 msgr2=0x7f47e806cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:19.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.065+0000 7f47e7fff640 1 --2- 172.21.15.154:0/2406578515 >> v2:172.21.15.190:3300/0 conn(0x7f47e80cee70 0x7f47e806cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:19.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.065+0000 7f47e7fff640 1 -- 172.21.15.154:0/2406578515 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e80b1f70 con 0x7f47e80b1b20 2023-09-23T03:21:19.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.066+0000 7f47e7fff640 1 --2- 172.21.15.154:0/2406578515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47e80b1b20 0x7f47e80ce930 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f47e80c9150 tx=0x7f47d402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bed06781cd4491a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:19.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.066+0000 7f47e77fe640 1 -- 172.21.15.154:0/2406578515 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d40317c0 con 0x7f47e80b1b20 2023-09-23T03:21:19.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.066+0000 7f47e77fe640 1 -- 172.21.15.154:0/2406578515 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47d4031920 con 0x7f47e80b1b20 2023-09-23T03:21:19.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.066+0000 7f47e77fe640 1 -- 172.21.15.154:0/2406578515 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d4031cb0 con 0x7f47e80b1b20 2023-09-23T03:21:19.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.067+0000 7f47eeefc640 1 -- 172.21.15.154:0/2406578515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47e80b1b20 msgr2=0x7f47e80ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:19.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.067+0000 7f47eeefc640 1 --2- 172.21.15.154:0/2406578515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47e80b1b20 0x7f47e80ce930 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f47e80c9150 tx=0x7f47d402b860 comp rx=0 tx=0).stop 2023-09-23T03:21:19.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.067+0000 7f47eeefc640 1 -- 172.21.15.154:0/2406578515 shutdown_connections 2023-09-23T03:21:19.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.067+0000 7f47eeefc640 1 --2- 172.21.15.154:0/2406578515 >> v2:172.21.15.190:3300/0 conn(0x7f47e80cee70 0x7f47e806cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:19.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.067+0000 7f47eeefc640 1 --2- 172.21.15.154:0/2406578515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47e80b1b20 0x7f47e80ce930 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:19.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.067+0000 7f47eeefc640 1 --2- 172.21.15.154:0/2406578515 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47e80acce0 0x7f47e815ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:19.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.067+0000 7f47eeefc640 1 -- 172.21.15.154:0/2406578515 >> 172.21.15.154:0/2406578515 conn(0x7f47e806a0f0 msgr2=0x7f47e806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:19.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.067+0000 7f47eeefc640 1 -- 172.21.15.154:0/2406578515 shutdown_connections 2023-09-23T03:21:19.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.067+0000 7f47eeefc640 1 -- 172.21.15.154:0/2406578515 wait complete. 2023-09-23T03:21:19.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.069+0000 7f47eeefc640 1 Processor -- start 2023-09-23T03:21:19.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.069+0000 7f47eeefc640 1 -- start start 2023-09-23T03:21:19.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.070+0000 7f47eeefc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47e80acce0 0x7f47e80c5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:19.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.070+0000 7f47eeefc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47e80b1b20 0x7f47e8159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:19.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.070+0000 7f47eeefc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47e80cee70 0x7f47e8159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:19.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.070+0000 7f47eeefc640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e80c6e70 con 0x7f47e80b1b20 2023-09-23T03:21:19.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.070+0000 7f47eeefc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e80c65d0 con 0x7f47e80cee70 2023-09-23T03:21:19.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.070+0000 7f47eeefc640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e80c68b0 con 0x7f47e80acce0 2023-09-23T03:21:19.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.070+0000 7f47e7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47e80b1b20 0x7f47e8159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:19.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.070+0000 7f47e7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47e80b1b20 0x7f47e8159c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:40598/0 (socket says 172.21.15.154:40598) 2023-09-23T03:21:19.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.070+0000 7f47e7fff640 1 -- 172.21.15.154:0/1509700255 learned_addr learned my addr 172.21.15.154:0/1509700255 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:19.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.071+0000 7f47ecc98640 1 --2- 172.21.15.154:0/1509700255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47e80acce0 0x7f47e80c5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:19.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.071+0000 7f47e7fff640 1 -- 172.21.15.154:0/1509700255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47e80acce0 msgr2=0x7f47e80c5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:19.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.071+0000 7f47ed499640 1 --2- 172.21.15.154:0/1509700255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47e80cee70 0x7f47e8159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:19.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.071+0000 7f47e7fff640 1 --2- 172.21.15.154:0/1509700255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47e80acce0 0x7f47e80c5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:19.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.071+0000 7f47e7fff640 1 -- 172.21.15.154:0/1509700255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47e80cee70 msgr2=0x7f47e8159350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:19.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.071+0000 7f47e7fff640 1 --2- 172.21.15.154:0/1509700255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47e80cee70 0x7f47e8159350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:19.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.071+0000 7f47e7fff640 1 -- 172.21.15.154:0/1509700255 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d40094d0 con 0x7f47e80b1b20 2023-09-23T03:21:19.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.071+0000 7f47ecc98640 1 --2- 172.21.15.154:0/1509700255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47e80acce0 0x7f47e80c5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:19.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.071+0000 7f47ed499640 1 --2- 172.21.15.154:0/1509700255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47e80cee70 0x7f47e8159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:19.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.071+0000 7f47e7fff640 1 --2- 172.21.15.154:0/1509700255 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47e80b1b20 0x7f47e8159c00 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f47d4000c00 tx=0x7f47d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:19.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.072+0000 7f47e57fa640 1 -- 172.21.15.154:0/1509700255 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d4032860 con 0x7f47e80b1b20 2023-09-23T03:21:19.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.072+0000 7f47e57fa640 1 -- 172.21.15.154:0/1509700255 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47d40329e0 con 0x7f47e80b1b20 2023-09-23T03:21:19.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.072+0000 7f47eeefc640 1 -- 172.21.15.154:0/1509700255 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47e80c6b90 con 0x7f47e80b1b20 2023-09-23T03:21:19.075 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.072+0000 7f47e57fa640 1 -- 172.21.15.154:0/1509700255 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d4032d90 con 0x7f47e80b1b20 2023-09-23T03:21:19.075 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.072+0000 7f47eeefc640 1 -- 172.21.15.154:0/1509700255 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47e815a140 con 0x7f47e80b1b20 2023-09-23T03:21:19.075 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.073+0000 7f47eeefc640 1 -- 172.21.15.154:0/1509700255 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47e815be20 con 0x7f47e80b1b20 2023-09-23T03:21:19.075 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.074+0000 7f47e57fa640 1 -- 172.21.15.154:0/1509700255 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f47d404d3f0 con 0x7f47e80b1b20 2023-09-23T03:21:19.075 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.075+0000 7f47e57fa640 1 --2- 172.21.15.154:0/1509700255 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f47cc06af30 0x7f47cc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:19.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.075+0000 7f47e57fa640 1 -- 172.21.15.154:0/1509700255 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f47d40ab910 con 0x7f47e80b1b20 2023-09-23T03:21:19.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.075+0000 7f47ecc98640 1 --2- 172.21.15.154:0/1509700255 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f47cc06af30 0x7f47cc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:19.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.076+0000 7f47ecc98640 1 --2- 172.21.15.154:0/1509700255 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f47cc06af30 0x7f47cc06d3f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f47dc009820 tx=0x7f47dc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:19.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.079+0000 7f47e57fa640 1 -- 172.21.15.154:0/1509700255 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f47e815be20 con 0x7f47e80b1b20 2023-09-23T03:21:19.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.268+0000 7f47eeefc640 1 -- 172.21.15.154:0/1509700255 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f47e815c200 con 0x7f47e80b1b20 2023-09-23T03:21:19.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.273+0000 7f47e57fa640 1 -- 172.21.15.154:0/1509700255 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+18592 (secure 0 0 0) 0x7f47e815c200 con 0x7f47e80b1b20 2023-09-23T03:21:19.274 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:19.274 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":65,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:19:40.966608+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:19:21.772037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-09-23T03:19:22.960250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data_extra","create_time":"2023-09-23T03:19:38.870843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:21:19.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.276+0000 7f47eeefc640 1 -- 172.21.15.154:0/1509700255 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f47cc06af30 msgr2=0x7f47cc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:19.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.276+0000 7f47eeefc640 1 --2- 172.21.15.154:0/1509700255 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f47cc06af30 0x7f47cc06d3f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f47dc009820 tx=0x7f47dc008040 comp rx=0 tx=0).stop 2023-09-23T03:21:19.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.276+0000 7f47eeefc640 1 -- 172.21.15.154:0/1509700255 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47e80b1b20 msgr2=0x7f47e8159c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:19.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.276+0000 7f47eeefc640 1 --2- 172.21.15.154:0/1509700255 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47e80b1b20 0x7f47e8159c00 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f47d4000c00 tx=0x7f47d40046c0 comp rx=0 tx=0).stop 2023-09-23T03:21:19.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.276+0000 7f47eeefc640 1 -- 172.21.15.154:0/1509700255 shutdown_connections 2023-09-23T03:21:19.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.276+0000 7f47eeefc640 1 --2- 172.21.15.154:0/1509700255 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f47cc06af30 0x7f47cc06d3f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:19.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.276+0000 7f47eeefc640 1 --2- 172.21.15.154:0/1509700255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47e80cee70 0x7f47e8159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:19.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.276+0000 7f47eeefc640 1 --2- 172.21.15.154:0/1509700255 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47e80b1b20 0x7f47e8159c00 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:19.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.276+0000 7f47eeefc640 1 --2- 172.21.15.154:0/1509700255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47e80acce0 0x7f47e80c5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:19.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.276+0000 7f47eeefc640 1 -- 172.21.15.154:0/1509700255 >> 172.21.15.154:0/1509700255 conn(0x7f47e806a0f0 msgr2=0x7f47e806a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:19.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.276+0000 7f47eeefc640 1 -- 172.21.15.154:0/1509700255 shutdown_connections 2023-09-23T03:21:19.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.277+0000 7f47eeefc640 1 -- 172.21.15.154:0/1509700255 wait complete. 2023-09-23T03:21:19.295 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:19:21.772037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:19:22.960250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-09-23T03:19:38.870843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T03:21:19.296 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:21:19.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.516+0000 7f9a9fd61640 1 Processor -- start 2023-09-23T03:21:19.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.516+0000 7f9a9fd61640 1 -- start start 2023-09-23T03:21:19.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.518+0000 7f9a9fd61640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a9815bb30 0x7f9a980caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:19.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.518+0000 7f9a9fd61640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9a980cb3e0 0x7f9a980cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:19.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.518+0000 7f9a9fd61640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a9815b6a0 con 0x7f9a9815aba0 2023-09-23T03:21:19.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.518+0000 7f9a9fd61640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a980cb870 con 0x7f9a980cb3e0 2023-09-23T03:21:19.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.518+0000 7f9a9fd61640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9815bf70 con 0x7f9a9815bb30 2023-09-23T03:21:19.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.519+0000 7f9a9dafd640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f9a9815aba0 0x7f9a9815e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.154:52554/0 (socket says 172.21.15.154:52554) 2023-09-23T03:21:19.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.519+0000 7f9a9dafd640 1 -- 172.21.15.154:0/2312324438 learned_addr learned my addr 172.21.15.154:0/2312324438 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:21:19.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.519+0000 7f9a9e2fe640 1 --2- 172.21.15.154:0/2312324438 >> v2:172.21.15.190:3300/0 conn(0x7f9a980cb3e0 0x7f9a980cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:19.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.519+0000 7f9a9d2fc640 1 --2- 172.21.15.154:0/2312324438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a9815bb30 0x7f9a980caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:19.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.519+0000 7f9a9e2fe640 1 -- 172.21.15.154:0/2312324438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a9815bb30 msgr2=0x7f9a980caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:19.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.519+0000 7f9a9e2fe640 1 --2- 172.21.15.154:0/2312324438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a9815bb30 0x7f9a980caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:19.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.520+0000 7f9a9e2fe640 1 -- 172.21.15.154:0/2312324438 >> v1:172.21.15.190:6789/0 conn(0x7f9a9815aba0 legacy=0x7f9a9815e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:19.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.520+0000 7f9a9e2fe640 1 -- 172.21.15.154:0/2312324438 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a9815f0b0 con 0x7f9a980cb3e0 2023-09-23T03:21:19.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.521+0000 7f9a9e2fe640 1 --2- 172.21.15.154:0/2312324438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a980cb3e0 0x7f9a980cc5c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9a900097a0 tx=0x7f9a9002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af3585a288dd8962 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:19.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.521+0000 7f9a9cafb640 1 -- 172.21.15.154:0/2312324438 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9002d040 con 0x7f9a980cb3e0 2023-09-23T03:21:19.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.521+0000 7f9a9cafb640 1 -- 172.21.15.154:0/2312324438 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a90031810 con 0x7f9a980cb3e0 2023-09-23T03:21:19.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.521+0000 7f9a9cafb640 1 -- 172.21.15.154:0/2312324438 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a90031ba0 con 0x7f9a980cb3e0 2023-09-23T03:21:19.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.522+0000 7f9a9fd61640 1 -- 172.21.15.154:0/2312324438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a980cb3e0 msgr2=0x7f9a980cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:19.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.522+0000 7f9a9fd61640 1 --2- 172.21.15.154:0/2312324438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a980cb3e0 0x7f9a980cc5c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9a900097a0 tx=0x7f9a9002b8b0 comp rx=0 tx=0).stop 2023-09-23T03:21:19.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.522+0000 7f9a9fd61640 1 -- 172.21.15.154:0/2312324438 shutdown_connections 2023-09-23T03:21:19.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.522+0000 7f9a9fd61640 1 --2- 172.21.15.154:0/2312324438 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a980cb3e0 0x7f9a980cc5c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:19.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.522+0000 7f9a9fd61640 1 --2- 172.21.15.154:0/2312324438 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a9815bb30 0x7f9a980caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:19.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.522+0000 7f9a9fd61640 1 -- 172.21.15.154:0/2312324438 >> 172.21.15.154:0/2312324438 conn(0x7f9a9806c010 msgr2=0x7f9a9806c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:19.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.522+0000 7f9a9fd61640 1 -- 172.21.15.154:0/2312324438 shutdown_connections 2023-09-23T03:21:19.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.522+0000 7f9a9fd61640 1 -- 172.21.15.154:0/2312324438 wait complete. 2023-09-23T03:21:19.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.524+0000 7f9a9fd61640 1 Processor -- start 2023-09-23T03:21:19.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.524+0000 7f9a9fd61640 1 -- start start 2023-09-23T03:21:19.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.526+0000 7f9a9fd61640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a980cb3e0 0x7f9a98146c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:19.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.526+0000 7f9a9fd61640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a9815aba0 0x7f9a980c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:19.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.526+0000 7f9a9fd61640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a9815bb30 0x7f9a98144150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:19.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.526+0000 7f9a9fd61640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a980c9620 con 0x7f9a9815bb30 2023-09-23T03:21:19.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.526+0000 7f9a9fd61640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a980c8a10 con 0x7f9a9815aba0 2023-09-23T03:21:19.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.526+0000 7f9a9fd61640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a980c8cf0 con 0x7f9a980cb3e0 2023-09-23T03:21:19.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.526+0000 7f9a9e2fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a9815bb30 0x7f9a98144150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:19.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.526+0000 7f9a9e2fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a9815bb30 0x7f9a98144150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:40600/0 (socket says 172.21.15.154:40600) 2023-09-23T03:21:19.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.526+0000 7f9a9e2fe640 1 -- 172.21.15.154:0/1650066946 learned_addr learned my addr 172.21.15.154:0/1650066946 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:19.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.526+0000 7f9a9d2fc640 1 --2- 172.21.15.154:0/1650066946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a9815aba0 0x7f9a980c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:19.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.526+0000 7f9a9dafd640 1 --2- 172.21.15.154:0/1650066946 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a980cb3e0 0x7f9a98146c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:19.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.527+0000 7f9a9e2fe640 1 -- 172.21.15.154:0/1650066946 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a980cb3e0 msgr2=0x7f9a98146c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:19.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.527+0000 7f9a9e2fe640 1 --2- 172.21.15.154:0/1650066946 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a980cb3e0 0x7f9a98146c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:19.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.527+0000 7f9a9e2fe640 1 -- 172.21.15.154:0/1650066946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a9815aba0 msgr2=0x7f9a980c7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:19.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.527+0000 7f9a9e2fe640 1 --2- 172.21.15.154:0/1650066946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a9815aba0 0x7f9a980c7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:19.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.527+0000 7f9a9e2fe640 1 -- 172.21.15.154:0/1650066946 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a90009400 con 0x7f9a9815bb30 2023-09-23T03:21:19.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.527+0000 7f9a9e2fe640 1 --2- 172.21.15.154:0/1650066946 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a9815bb30 0x7f9a98144150 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f9a9002b8f0 tx=0x7f9a900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:19.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.527+0000 7f9a8e7fc640 1 -- 172.21.15.154:0/1650066946 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a90006e10 con 0x7f9a9815bb30 2023-09-23T03:21:19.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.527+0000 7f9a8e7fc640 1 -- 172.21.15.154:0/1650066946 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a9003ad60 con 0x7f9a9815bb30 2023-09-23T03:21:19.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.527+0000 7f9a9fd61640 1 -- 172.21.15.154:0/1650066946 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a980c8fd0 con 0x7f9a9815bb30 2023-09-23T03:21:19.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.527+0000 7f9a8e7fc640 1 -- 172.21.15.154:0/1650066946 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a90044610 con 0x7f9a9815bb30 2023-09-23T03:21:19.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.527+0000 7f9a9fd61640 1 -- 172.21.15.154:0/1650066946 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a980c9eb0 con 0x7f9a9815bb30 2023-09-23T03:21:19.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.529+0000 7f9a9fd61640 1 -- 172.21.15.154:0/1650066946 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a98071710 con 0x7f9a9815bb30 2023-09-23T03:21:19.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.530+0000 7f9a8e7fc640 1 -- 172.21.15.154:0/1650066946 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f9a9004d450 con 0x7f9a9815bb30 2023-09-23T03:21:19.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.530+0000 7f9a8e7fc640 1 --2- 172.21.15.154:0/1650066946 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9a7c06af60 0x7f9a7c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:19.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.531+0000 7f9a8e7fc640 1 -- 172.21.15.154:0/1650066946 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f9a900abca0 con 0x7f9a9815bb30 2023-09-23T03:21:19.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.531+0000 7f9a9dafd640 1 --2- 172.21.15.154:0/1650066946 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9a7c06af60 0x7f9a7c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:19.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.531+0000 7f9a9dafd640 1 --2- 172.21.15.154:0/1650066946 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9a7c06af60 0x7f9a7c06d420 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9a980c9a40 tx=0x7f9a94009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:19.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.535+0000 7f9a8e7fc640 1 -- 172.21.15.154:0/1650066946 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f9a90076960 con 0x7f9a9815bb30 2023-09-23T03:21:19.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.723+0000 7f9a9fd61640 1 -- 172.21.15.154:0/1650066946 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9a9804e920 con 0x7f9a9815bb30 2023-09-23T03:21:19.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.724+0000 7f9a8e7fc640 1 -- 172.21.15.154:0/1650066946 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9a90079f80 con 0x7f9a9815bb30 2023-09-23T03:21:19.724 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:21:19.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.727+0000 7f9a9fd61640 1 -- 172.21.15.154:0/1650066946 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9a7c06af60 msgr2=0x7f9a7c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:19.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.727+0000 7f9a9fd61640 1 --2- 172.21.15.154:0/1650066946 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9a7c06af60 0x7f9a7c06d420 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9a980c9a40 tx=0x7f9a94009040 comp rx=0 tx=0).stop 2023-09-23T03:21:19.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.727+0000 7f9a9fd61640 1 -- 172.21.15.154:0/1650066946 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a9815bb30 msgr2=0x7f9a98144150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:19.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.727+0000 7f9a9fd61640 1 --2- 172.21.15.154:0/1650066946 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a9815bb30 0x7f9a98144150 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f9a9002b8f0 tx=0x7f9a900046c0 comp rx=0 tx=0).stop 2023-09-23T03:21:19.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.727+0000 7f9a9fd61640 1 -- 172.21.15.154:0/1650066946 shutdown_connections 2023-09-23T03:21:19.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.727+0000 7f9a9fd61640 1 --2- 172.21.15.154:0/1650066946 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9a7c06af60 0x7f9a7c06d420 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:19.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.727+0000 7f9a9fd61640 1 --2- 172.21.15.154:0/1650066946 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a9815bb30 0x7f9a98144150 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:19.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.727+0000 7f9a9fd61640 1 --2- 172.21.15.154:0/1650066946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a9815aba0 0x7f9a980c7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:19.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.727+0000 7f9a9fd61640 1 --2- 172.21.15.154:0/1650066946 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a980cb3e0 0x7f9a98146c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:19.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.727+0000 7f9a9fd61640 1 -- 172.21.15.154:0/1650066946 >> 172.21.15.154:0/1650066946 conn(0x7f9a9806c010 msgr2=0x7f9a98159040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:19.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.727+0000 7f9a9fd61640 1 -- 172.21.15.154:0/1650066946 shutdown_connections 2023-09-23T03:21:19.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.727+0000 7f9a9fd61640 1 -- 172.21.15.154:0/1650066946 wait complete. 2023-09-23T03:21:19.746 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:21:19.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.964+0000 7f9e31660640 1 Processor -- start 2023-09-23T03:21:19.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.965+0000 7f9e31660640 1 -- start start 2023-09-23T03:21:19.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.966+0000 7f9e31660640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9e2c06bf30 0x7f9e2c0ceba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:19.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.966+0000 7f9e31660640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e2c15ce80 0x7f9e2c0cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:19.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.966+0000 7f9e31660640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e2c0cb010 con 0x7f9e2c15caa0 2023-09-23T03:21:19.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.966+0000 7f9e31660640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e2c06c490 con 0x7f9e2c06bf30 2023-09-23T03:21:19.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.966+0000 7f9e31660640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2c06c770 con 0x7f9e2c15ce80 2023-09-23T03:21:19.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.967+0000 7f9e2bfff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f9e2c15caa0 0x7f9e2c0d0e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:49310/0 (socket says 172.21.15.154:49310) 2023-09-23T03:21:19.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.967+0000 7f9e2bfff640 1 -- 172.21.15.154:0/761844733 learned_addr learned my addr 172.21.15.154:0/761844733 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:21:19.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.967+0000 7f9e2b7fe640 1 --2- 172.21.15.154:0/761844733 >> v2:172.21.15.190:3300/0 conn(0x7f9e2c06bf30 0x7f9e2c0ceba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:19.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.967+0000 7f9e2affd640 1 -- 172.21.15.154:0/761844733 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 725076777 0 0) 0x7f9e2c0cb010 con 0x7f9e2c15caa0 2023-09-23T03:21:19.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.967+0000 7f9e2affd640 1 -- 172.21.15.154:0/761844733 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e18003520 con 0x7f9e2c15caa0 2023-09-23T03:21:19.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.967+0000 7f9e30e5f640 1 --2- 172.21.15.154:0/761844733 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e2c15ce80 0x7f9e2c0cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:19.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.967+0000 7f9e2affd640 1 -- 172.21.15.154:0/761844733 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3521743186 0 0) 0x7f9e18003520 con 0x7f9e2c15caa0 2023-09-23T03:21:19.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.967+0000 7f9e2affd640 1 -- 172.21.15.154:0/761844733 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e2c15ce80 msgr2=0x7f9e2c0cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:19.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.967+0000 7f9e2affd640 1 --2- 172.21.15.154:0/761844733 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e2c15ce80 0x7f9e2c0cf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:19.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.967+0000 7f9e2affd640 1 -- 172.21.15.154:0/761844733 >> v2:172.21.15.190:3300/0 conn(0x7f9e2c06bf30 msgr2=0x7f9e2c0ceba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:19.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.967+0000 7f9e2affd640 1 --2- 172.21.15.154:0/761844733 >> v2:172.21.15.190:3300/0 conn(0x7f9e2c06bf30 0x7f9e2c0ceba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:19.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.967+0000 7f9e2affd640 1 -- 172.21.15.154:0/761844733 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e2c06ca50 con 0x7f9e2c15caa0 2023-09-23T03:21:19.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.967+0000 7f9e2affd640 1 -- 172.21.15.154:0/761844733 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f9e1c0019f0 con 0x7f9e2c15caa0 2023-09-23T03:21:19.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.967+0000 7f9e2b7fe640 1 --2- 172.21.15.154:0/761844733 >> v2:172.21.15.190:3300/0 conn(0x7f9e2c06bf30 0x7f9e2c0ceba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:21:19.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.967+0000 7f9e2affd640 1 -- 172.21.15.154:0/761844733 >> v1:172.21.15.154:6789/0 conn(0x7f9e2c15caa0 legacy=0x7f9e2c0d0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:19.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.968+0000 7f9e2affd640 1 --2- 172.21.15.154:0/761844733 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e2c15ce80 0x7f9e18003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:19.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.968+0000 7f9e2affd640 1 --2- 172.21.15.154:0/761844733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e18003ef0 0x7f9e180047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:19.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.968+0000 7f9e2affd640 1 --2- 172.21.15.154:0/761844733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e18004d00 0x7f9e18005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:19.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.968+0000 7f9e2affd640 1 -- 172.21.15.154:0/761844733 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2c06c490 con 0x7f9e18003ef0 2023-09-23T03:21:19.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.968+0000 7f9e2affd640 1 -- 172.21.15.154:0/761844733 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2c06c770 con 0x7f9e18004d00 2023-09-23T03:21:19.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.968+0000 7f9e2affd640 1 -- 172.21.15.154:0/761844733 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1800ba80 con 0x7f9e2c15ce80 2023-09-23T03:21:19.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.968+0000 7f9e2b7fe640 1 --2- 172.21.15.154:0/761844733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e18003ef0 0x7f9e180047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:19.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.968+0000 7f9e30e5f640 1 --2- 172.21.15.154:0/761844733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e18004d00 0x7f9e18005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:19.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.968+0000 7f9e2b7fe640 1 -- 172.21.15.154:0/761844733 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e2c15ce80 msgr2=0x7f9e18003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:19.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.968+0000 7f9e2b7fe640 1 --2- 172.21.15.154:0/761844733 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e2c15ce80 0x7f9e18003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:19.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.968+0000 7f9e2b7fe640 1 -- 172.21.15.154:0/761844733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e18004d00 msgr2=0x7f9e18005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:19.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.968+0000 7f9e2b7fe640 1 --2- 172.21.15.154:0/761844733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e18004d00 0x7f9e18005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:19.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.968+0000 7f9e2b7fe640 1 -- 172.21.15.154:0/761844733 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e18003520 con 0x7f9e18003ef0 2023-09-23T03:21:19.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.970+0000 7f9e2b7fe640 1 --2- 172.21.15.154:0/761844733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e18003ef0 0x7f9e180047c0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f9e140098d0 tx=0x7f9e180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22f47e46374af963 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:19.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.970+0000 7f9e2affd640 1 -- 172.21.15.154:0/761844733 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e14004230 con 0x7f9e18003ef0 2023-09-23T03:21:19.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.970+0000 7f9e2affd640 1 -- 172.21.15.154:0/761844733 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e14004390 con 0x7f9e18003ef0 2023-09-23T03:21:19.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.970+0000 7f9e2affd640 1 -- 172.21.15.154:0/761844733 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e14003830 con 0x7f9e18003ef0 2023-09-23T03:21:19.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.970+0000 7f9e31660640 1 -- 172.21.15.154:0/761844733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e18003ef0 msgr2=0x7f9e180047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:19.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.970+0000 7f9e31660640 1 --2- 172.21.15.154:0/761844733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e18003ef0 0x7f9e180047c0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f9e140098d0 tx=0x7f9e180057f0 comp rx=0 tx=0).stop 2023-09-23T03:21:19.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.970+0000 7f9e30e5f640 1 -- 172.21.15.154:0/761844733 reap_dead start 2023-09-23T03:21:19.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.970+0000 7f9e31660640 1 -- 172.21.15.154:0/761844733 shutdown_connections 2023-09-23T03:21:19.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.970+0000 7f9e31660640 1 -- 172.21.15.154:0/761844733 >> 172.21.15.154:0/761844733 conn(0x7f9e2c0998e0 msgr2=0x7f9e2c0d05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:19.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.971+0000 7f9e31660640 1 -- 172.21.15.154:0/761844733 shutdown_connections 2023-09-23T03:21:19.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.971+0000 7f9e31660640 1 -- 172.21.15.154:0/761844733 wait complete. 2023-09-23T03:21:19.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.972+0000 7f9e31660640 1 Processor -- start 2023-09-23T03:21:19.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.973+0000 7f9e31660640 1 -- start start 2023-09-23T03:21:19.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.974+0000 7f9e31660640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e18003ef0 0x7f9e2c0c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:19.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.974+0000 7f9e31660640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e2c06e970 0x7f9e2c06edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:19.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.974+0000 7f9e31660640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e2c0c40a0 0x7f9e2c0c4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:19.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.974+0000 7f9e31660640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2c0c7330 con 0x7f9e2c0c40a0 2023-09-23T03:21:19.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.974+0000 7f9e31660640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2c0ca110 con 0x7f9e2c06e970 2023-09-23T03:21:19.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.974+0000 7f9e31660640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2c06f330 con 0x7f9e18003ef0 2023-09-23T03:21:19.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.974+0000 7f9e30e5f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e2c0c40a0 0x7f9e2c0c4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:19.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.974+0000 7f9e30e5f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e2c0c40a0 0x7f9e2c0c4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:40618/0 (socket says 172.21.15.154:40618) 2023-09-23T03:21:19.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.974+0000 7f9e30e5f640 1 -- 172.21.15.154:0/923081718 learned_addr learned my addr 172.21.15.154:0/923081718 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:19.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.974+0000 7f9e2b7fe640 1 --2- 172.21.15.154:0/923081718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e2c06e970 0x7f9e2c06edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:19.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.974+0000 7f9e2bfff640 1 --2- 172.21.15.154:0/923081718 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e18003ef0 0x7f9e2c0c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:19.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.974+0000 7f9e30e5f640 1 -- 172.21.15.154:0/923081718 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e18003ef0 msgr2=0x7f9e2c0c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:19.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.974+0000 7f9e30e5f640 1 --2- 172.21.15.154:0/923081718 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e18003ef0 0x7f9e2c0c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:19.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.974+0000 7f9e30e5f640 1 -- 172.21.15.154:0/923081718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e2c06e970 msgr2=0x7f9e2c06edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:19.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.974+0000 7f9e30e5f640 1 --2- 172.21.15.154:0/923081718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e2c06e970 0x7f9e2c06edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:19.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.974+0000 7f9e30e5f640 1 -- 172.21.15.154:0/923081718 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e2c0c9070 con 0x7f9e2c0c40a0 2023-09-23T03:21:19.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.974+0000 7f9e2bfff640 1 --2- 172.21.15.154:0/923081718 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e18003ef0 0x7f9e2c0c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:19.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.975+0000 7f9e30e5f640 1 --2- 172.21.15.154:0/923081718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e2c0c40a0 0x7f9e2c0c4520 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c06c8b0 tx=0x7f9e24011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:19.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.975+0000 7f9e28ff9640 1 -- 172.21.15.154:0/923081718 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e24004440 con 0x7f9e2c0c40a0 2023-09-23T03:21:19.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.975+0000 7f9e28ff9640 1 -- 172.21.15.154:0/923081718 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e24003610 con 0x7f9e2c0c40a0 2023-09-23T03:21:19.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.975+0000 7f9e31660640 1 -- 172.21.15.154:0/923081718 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e2c0c9350 con 0x7f9e2c0c40a0 2023-09-23T03:21:19.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.975+0000 7f9e28ff9640 1 -- 172.21.15.154:0/923081718 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e240039c0 con 0x7f9e2c0c40a0 2023-09-23T03:21:19.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.975+0000 7f9e31660640 1 -- 172.21.15.154:0/923081718 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e2c0c9910 con 0x7f9e2c0c40a0 2023-09-23T03:21:19.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.976+0000 7f9e31660640 1 -- 172.21.15.154:0/923081718 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e2c04e920 con 0x7f9e2c0c40a0 2023-09-23T03:21:19.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.977+0000 7f9e28ff9640 1 -- 172.21.15.154:0/923081718 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f9e24003b20 con 0x7f9e2c0c40a0 2023-09-23T03:21:19.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.978+0000 7f9e28ff9640 1 --2- 172.21.15.154:0/923081718 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9e0c06af30 0x7f9e0c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:19.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.978+0000 7f9e28ff9640 1 -- 172.21.15.154:0/923081718 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f9e24089760 con 0x7f9e2c0c40a0 2023-09-23T03:21:19.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.978+0000 7f9e2bfff640 1 --2- 172.21.15.154:0/923081718 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9e0c06af30 0x7f9e0c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:19.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.979+0000 7f9e2bfff640 1 --2- 172.21.15.154:0/923081718 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9e0c06af30 0x7f9e0c06d3f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c0ae590 tx=0x7f9e1c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:19.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:19.982+0000 7f9e28ff9640 1 -- 172.21.15.154:0/923081718 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f9e24054420 con 0x7f9e2c0c40a0 2023-09-23T03:21:20.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.172+0000 7f9e31660640 1 -- 172.21.15.154:0/923081718 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9e2c08a930 con 0x7f9e2c0c40a0 2023-09-23T03:21:20.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.173+0000 7f9e28ff9640 1 -- 172.21.15.154:0/923081718 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v65) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9e24057a40 con 0x7f9e2c0c40a0 2023-09-23T03:21:20.174 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:21:20.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.176+0000 7f9e31660640 1 -- 172.21.15.154:0/923081718 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9e0c06af30 msgr2=0x7f9e0c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:20.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.176+0000 7f9e31660640 1 --2- 172.21.15.154:0/923081718 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9e0c06af30 0x7f9e0c06d3f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c0ae590 tx=0x7f9e1c009040 comp rx=0 tx=0).stop 2023-09-23T03:21:20.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.176+0000 7f9e31660640 1 -- 172.21.15.154:0/923081718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e2c0c40a0 msgr2=0x7f9e2c0c4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:20.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.176+0000 7f9e31660640 1 --2- 172.21.15.154:0/923081718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e2c0c40a0 0x7f9e2c0c4520 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c06c8b0 tx=0x7f9e24011920 comp rx=0 tx=0).stop 2023-09-23T03:21:20.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.176+0000 7f9e31660640 1 -- 172.21.15.154:0/923081718 shutdown_connections 2023-09-23T03:21:20.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.176+0000 7f9e31660640 1 --2- 172.21.15.154:0/923081718 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9e0c06af30 0x7f9e0c06d3f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:20.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.176+0000 7f9e31660640 1 --2- 172.21.15.154:0/923081718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9e2c0c40a0 0x7f9e2c0c4520 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:20.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.176+0000 7f9e31660640 1 --2- 172.21.15.154:0/923081718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9e2c06e970 0x7f9e2c06edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:20.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.176+0000 7f9e31660640 1 --2- 172.21.15.154:0/923081718 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9e18003ef0 0x7f9e2c0c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:20.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.176+0000 7f9e31660640 1 -- 172.21.15.154:0/923081718 >> 172.21.15.154:0/923081718 conn(0x7f9e2c0998e0 msgr2=0x7f9e2c06e170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:20.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.176+0000 7f9e31660640 1 -- 172.21.15.154:0/923081718 shutdown_connections 2023-09-23T03:21:20.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.176+0000 7f9e31660640 1 -- 172.21.15.154:0/923081718 wait complete. 2023-09-23T03:21:20.196 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:21:20.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.417+0000 7f632c872640 1 Processor -- start 2023-09-23T03:21:20.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.417+0000 7f632c872640 1 -- start start 2023-09-23T03:21:20.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.419+0000 7f632c872640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632815cc60 0x7f632806fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:20.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.419+0000 7f632c872640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f63280cb4f0 0x7f63280cc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:20.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.419+0000 7f632c872640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6328160bd0 con 0x7f63281606d0 2023-09-23T03:21:20.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.419+0000 7f632c872640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63280cc4e0 con 0x7f63280cb4f0 2023-09-23T03:21:20.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.419+0000 7f632c872640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6328160eb0 con 0x7f632815cc60 2023-09-23T03:21:20.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.419+0000 7f6327fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f63280cb4f0 0x7f63280cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:20.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.419+0000 7f6326ffd640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f63281606d0 0x7f63280cade0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:49316/0 (socket says 172.21.15.154:49316) 2023-09-23T03:21:20.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.419+0000 7f6326ffd640 1 -- 172.21.15.154:0/2149018943 learned_addr learned my addr 172.21.15.154:0/2149018943 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:21:20.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.419+0000 7f63277fe640 1 --2- 172.21.15.154:0/2149018943 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632815cc60 0x7f632806fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:20.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.420+0000 7f6327fff640 1 -- 172.21.15.154:0/2149018943 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632815cc60 msgr2=0x7f632806fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:20.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.420+0000 7f6327fff640 1 --2- 172.21.15.154:0/2149018943 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632815cc60 0x7f632806fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:20.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.420+0000 7f6327fff640 1 -- 172.21.15.154:0/2149018943 >> v1:172.21.15.154:6789/0 conn(0x7f63281606d0 legacy=0x7f63280cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:20.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.420+0000 7f6327fff640 1 -- 172.21.15.154:0/2149018943 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6328161190 con 0x7f63280cb4f0 2023-09-23T03:21:20.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.420+0000 7f63277fe640 1 --2- 172.21.15.154:0/2149018943 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632815cc60 0x7f632806fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:21:20.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.420+0000 7f63267fc640 1 -- 172.21.15.154:0/2149018943 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1293555094 0 0) 0x7f6328160bd0 con 0x7f63281606d0 2023-09-23T03:21:20.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.421+0000 7f6327fff640 1 --2- 172.21.15.154:0/2149018943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63280cb4f0 0x7f63280cc930 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f63200097a0 tx=0x7f632002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42bb84b0bf1bf791 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:20.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.421+0000 7f63267fc640 1 -- 172.21.15.154:0/2149018943 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632002d040 con 0x7f63280cb4f0 2023-09-23T03:21:20.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.421+0000 7f63267fc640 1 -- 172.21.15.154:0/2149018943 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6320031810 con 0x7f63280cb4f0 2023-09-23T03:21:20.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.421+0000 7f63267fc640 1 -- 172.21.15.154:0/2149018943 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6320031ba0 con 0x7f63280cb4f0 2023-09-23T03:21:20.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.422+0000 7f632c872640 1 -- 172.21.15.154:0/2149018943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63280cb4f0 msgr2=0x7f63280cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:20.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.422+0000 7f632c872640 1 --2- 172.21.15.154:0/2149018943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63280cb4f0 0x7f63280cc930 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f63200097a0 tx=0x7f632002b8b0 comp rx=0 tx=0).stop 2023-09-23T03:21:20.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.422+0000 7f632c872640 1 -- 172.21.15.154:0/2149018943 shutdown_connections 2023-09-23T03:21:20.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.422+0000 7f632c872640 1 --2- 172.21.15.154:0/2149018943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63280cb4f0 0x7f63280cc930 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:20.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.422+0000 7f632c872640 1 --2- 172.21.15.154:0/2149018943 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632815cc60 0x7f632806fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:20.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.422+0000 7f632c872640 1 -- 172.21.15.154:0/2149018943 >> 172.21.15.154:0/2149018943 conn(0x7f63280aff20 msgr2=0x7f63280b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:20.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.422+0000 7f632c872640 1 -- 172.21.15.154:0/2149018943 shutdown_connections 2023-09-23T03:21:20.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.422+0000 7f632c872640 1 -- 172.21.15.154:0/2149018943 wait complete. 2023-09-23T03:21:20.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.424+0000 7f632c872640 1 Processor -- start 2023-09-23T03:21:20.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.424+0000 7f632c872640 1 -- start start 2023-09-23T03:21:20.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.425+0000 7f632c872640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63280cb4f0 0x7f63281502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:20.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.425+0000 7f632c872640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632815cc60 0x7f632815f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:20.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.425+0000 7f632c872640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63281606d0 0x7f63280b1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:20.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.425+0000 7f632c872640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63281507f0 con 0x7f63281606d0 2023-09-23T03:21:20.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.425+0000 7f632c872640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63280c2c90 con 0x7f63280cb4f0 2023-09-23T03:21:20.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.425+0000 7f632c872640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63280c2f70 con 0x7f632815cc60 2023-09-23T03:21:20.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.425+0000 7f6327fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63281606d0 0x7f63280b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:20.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.425+0000 7f6327fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63281606d0 0x7f63280b1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:40638/0 (socket says 172.21.15.154:40638) 2023-09-23T03:21:20.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.425+0000 7f6327fff640 1 -- 172.21.15.154:0/2781233600 learned_addr learned my addr 172.21.15.154:0/2781233600 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:20.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.425+0000 7f63277fe640 1 --2- 172.21.15.154:0/2781233600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63280cb4f0 0x7f63281502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:20.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.425+0000 7f6327fff640 1 -- 172.21.15.154:0/2781233600 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632815cc60 msgr2=0x7f632815f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:20.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.425+0000 7f6327fff640 1 --2- 172.21.15.154:0/2781233600 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632815cc60 0x7f632815f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:20.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.425+0000 7f6327fff640 1 -- 172.21.15.154:0/2781233600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63280cb4f0 msgr2=0x7f63281502b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:20.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.425+0000 7f6327fff640 1 --2- 172.21.15.154:0/2781233600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63280cb4f0 0x7f63281502b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:20.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.425+0000 7f6327fff640 1 -- 172.21.15.154:0/2781233600 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6320009400 con 0x7f63281606d0 2023-09-23T03:21:20.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.426+0000 7f6327fff640 1 --2- 172.21.15.154:0/2781233600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63281606d0 0x7f63280b1010 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f63200097a0 tx=0x7f63200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:20.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.426+0000 7f631ffff640 1 -- 172.21.15.154:0/2781233600 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6320006e10 con 0x7f63281606d0 2023-09-23T03:21:20.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.426+0000 7f631ffff640 1 -- 172.21.15.154:0/2781233600 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f632003bb60 con 0x7f63281606d0 2023-09-23T03:21:20.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.426+0000 7f632c872640 1 -- 172.21.15.154:0/2781233600 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63280c3250 con 0x7f63281606d0 2023-09-23T03:21:20.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.426+0000 7f631ffff640 1 -- 172.21.15.154:0/2781233600 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6320004030 con 0x7f63281606d0 2023-09-23T03:21:20.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.426+0000 7f632c872640 1 -- 172.21.15.154:0/2781233600 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63280c3810 con 0x7f63281606d0 2023-09-23T03:21:20.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.428+0000 7f632c872640 1 -- 172.21.15.154:0/2781233600 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63280c3a00 con 0x7f63281606d0 2023-09-23T03:21:20.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.428+0000 7f631ffff640 1 -- 172.21.15.154:0/2781233600 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f6320043070 con 0x7f63281606d0 2023-09-23T03:21:20.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.429+0000 7f631ffff640 1 --2- 172.21.15.154:0/2781233600 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f630806af30 0x7f630806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:20.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.429+0000 7f631ffff640 1 -- 172.21.15.154:0/2781233600 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f63200ab920 con 0x7f63281606d0 2023-09-23T03:21:20.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.430+0000 7f63277fe640 1 --2- 172.21.15.154:0/2781233600 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f630806af30 0x7f630806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:20.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.430+0000 7f63277fe640 1 --2- 172.21.15.154:0/2781233600 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f630806af30 0x7f630806d3f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f631800d8b0 tx=0x7f6318008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:20.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.434+0000 7f631ffff640 1 -- 172.21.15.154:0/2781233600 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f63200767a0 con 0x7f63281606d0 2023-09-23T03:21:20.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.622+0000 7f632c872640 1 -- 172.21.15.154:0/2781233600 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f632804e920 con 0x7f63281606d0 2023-09-23T03:21:20.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.623+0000 7f631ffff640 1 -- 172.21.15.154:0/2781233600 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v65) v1 ==== 100+0+11 (secure 0 0 0) 0x7f63200350a0 con 0x7f63281606d0 2023-09-23T03:21:20.623 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:21:20.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.625+0000 7f632c872640 1 -- 172.21.15.154:0/2781233600 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f630806af30 msgr2=0x7f630806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:20.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.625+0000 7f632c872640 1 --2- 172.21.15.154:0/2781233600 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f630806af30 0x7f630806d3f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f631800d8b0 tx=0x7f6318008040 comp rx=0 tx=0).stop 2023-09-23T03:21:20.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.625+0000 7f632c872640 1 -- 172.21.15.154:0/2781233600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63281606d0 msgr2=0x7f63280b1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:20.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.625+0000 7f632c872640 1 --2- 172.21.15.154:0/2781233600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63281606d0 0x7f63280b1010 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f63200097a0 tx=0x7f63200046c0 comp rx=0 tx=0).stop 2023-09-23T03:21:20.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.625+0000 7f632c872640 1 -- 172.21.15.154:0/2781233600 shutdown_connections 2023-09-23T03:21:20.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.625+0000 7f632c872640 1 --2- 172.21.15.154:0/2781233600 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f630806af30 0x7f630806d3f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:20.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.625+0000 7f632c872640 1 --2- 172.21.15.154:0/2781233600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63281606d0 0x7f63280b1010 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:20.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.625+0000 7f632c872640 1 --2- 172.21.15.154:0/2781233600 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632815cc60 0x7f632815f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:20.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.625+0000 7f632c872640 1 --2- 172.21.15.154:0/2781233600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63280cb4f0 0x7f63281502b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:20.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.625+0000 7f632c872640 1 -- 172.21.15.154:0/2781233600 >> 172.21.15.154:0/2781233600 conn(0x7f63280aff20 msgr2=0x7f6328070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:20.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.626+0000 7f632c872640 1 -- 172.21.15.154:0/2781233600 shutdown_connections 2023-09-23T03:21:20.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.626+0000 7f632c872640 1 -- 172.21.15.154:0/2781233600 wait complete. 2023-09-23T03:21:20.644 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-09-23T03:21:20.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.861+0000 7fc379733640 1 Processor -- start 2023-09-23T03:21:20.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.862+0000 7fc379733640 1 -- start start 2023-09-23T03:21:20.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.863+0000 7fc379733640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc3740c8d70 0x7fc37415c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:20.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.863+0000 7fc379733640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc3740c9150 0x7fc37415cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:20.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.863+0000 7fc379733640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3740c9d20 con 0x7fc374071290 2023-09-23T03:21:20.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.863+0000 7fc379733640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc374071850 con 0x7fc3740c9150 2023-09-23T03:21:20.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.863+0000 7fc379733640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc374071b30 con 0x7fc3740c8d70 2023-09-23T03:21:20.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.864+0000 7fc3737fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fc374071290 0x7fc3740ca1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:49324/0 (socket says 172.21.15.154:49324) 2023-09-23T03:21:20.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.864+0000 7fc3737fe640 1 -- 172.21.15.154:0/925487663 learned_addr learned my addr 172.21.15.154:0/925487663 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:21:20.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.864+0000 7fc3727fc640 1 --2- 172.21.15.154:0/925487663 >> v2:172.21.15.154:3300/0 conn(0x7fc3740c9150 0x7fc37415cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:20.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.864+0000 7fc372ffd640 1 --2- 172.21.15.154:0/925487663 >> v2:172.21.15.190:3300/0 conn(0x7fc3740c8d70 0x7fc37415c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:20.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.864+0000 7fc3727fc640 1 -- 172.21.15.154:0/925487663 >> v2:172.21.15.190:3300/0 conn(0x7fc3740c8d70 msgr2=0x7fc37415c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:20.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.864+0000 7fc3727fc640 1 --2- 172.21.15.154:0/925487663 >> v2:172.21.15.190:3300/0 conn(0x7fc3740c8d70 0x7fc37415c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:20.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.864+0000 7fc3727fc640 1 -- 172.21.15.154:0/925487663 >> v1:172.21.15.154:6789/0 conn(0x7fc374071290 legacy=0x7fc3740ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:20.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.864+0000 7fc3727fc640 1 -- 172.21.15.154:0/925487663 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc374071e10 con 0x7fc3740c9150 2023-09-23T03:21:20.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.865+0000 7fc3727fc640 1 --2- 172.21.15.154:0/925487663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3740c9150 0x7fc37415cc10 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fc37406b790 tx=0x7fc36402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8585950ef3e617d7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:20.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.866+0000 7fc371ffb640 1 -- 172.21.15.154:0/925487663 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36402cc70 con 0x7fc3740c9150 2023-09-23T03:21:20.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.866+0000 7fc371ffb640 1 -- 172.21.15.154:0/925487663 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc36402cdd0 con 0x7fc3740c9150 2023-09-23T03:21:20.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.866+0000 7fc371ffb640 1 -- 172.21.15.154:0/925487663 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc364034660 con 0x7fc3740c9150 2023-09-23T03:21:20.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.866+0000 7fc379733640 1 -- 172.21.15.154:0/925487663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3740c9150 msgr2=0x7fc37415cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:20.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.866+0000 7fc379733640 1 --2- 172.21.15.154:0/925487663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3740c9150 0x7fc37415cc10 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fc37406b790 tx=0x7fc36402bec0 comp rx=0 tx=0).stop 2023-09-23T03:21:20.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.866+0000 7fc379733640 1 -- 172.21.15.154:0/925487663 shutdown_connections 2023-09-23T03:21:20.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.866+0000 7fc379733640 1 --2- 172.21.15.154:0/925487663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3740c9150 0x7fc37415cc10 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:20.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.866+0000 7fc379733640 1 --2- 172.21.15.154:0/925487663 >> v2:172.21.15.190:3300/0 conn(0x7fc3740c8d70 0x7fc37415c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:20.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.866+0000 7fc379733640 1 -- 172.21.15.154:0/925487663 >> 172.21.15.154:0/925487663 conn(0x7fc37406c010 msgr2=0x7fc37406c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:20.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.867+0000 7fc379733640 1 -- 172.21.15.154:0/925487663 shutdown_connections 2023-09-23T03:21:20.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.867+0000 7fc379733640 1 -- 172.21.15.154:0/925487663 wait complete. 2023-09-23T03:21:20.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.869+0000 7fc379733640 1 Processor -- start 2023-09-23T03:21:20.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.869+0000 7fc379733640 1 -- start start 2023-09-23T03:21:20.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.870+0000 7fc379733640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc374071290 0x7fc3741521a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:20.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.870+0000 7fc379733640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3740c8d70 0x7fc3740c45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:20.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.870+0000 7fc379733640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3740c9150 0x7fc3740ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:20.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.870+0000 7fc379733640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc374152c40 con 0x7fc3740c8d70 2023-09-23T03:21:20.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.870+0000 7fc379733640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc374152f20 con 0x7fc374071290 2023-09-23T03:21:20.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.870+0000 7fc379733640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc374153200 con 0x7fc3740c9150 2023-09-23T03:21:20.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.870+0000 7fc3727fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3740c8d70 0x7fc3740c45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:20.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.870+0000 7fc3727fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3740c8d70 0x7fc3740c45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:40658/0 (socket says 172.21.15.154:40658) 2023-09-23T03:21:20.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.870+0000 7fc3727fc640 1 -- 172.21.15.154:0/2598817769 learned_addr learned my addr 172.21.15.154:0/2598817769 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:20.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.871+0000 7fc372ffd640 1 --2- 172.21.15.154:0/2598817769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc374071290 0x7fc3741521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:20.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.871+0000 7fc3737fe640 1 --2- 172.21.15.154:0/2598817769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3740c9150 0x7fc3740ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:20.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.871+0000 7fc3727fc640 1 -- 172.21.15.154:0/2598817769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3740c9150 msgr2=0x7fc3740ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:20.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.871+0000 7fc3727fc640 1 --2- 172.21.15.154:0/2598817769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3740c9150 0x7fc3740ca870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:20.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.871+0000 7fc3727fc640 1 -- 172.21.15.154:0/2598817769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc374071290 msgr2=0x7fc3741521a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:20.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.871+0000 7fc3727fc640 1 --2- 172.21.15.154:0/2598817769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc374071290 0x7fc3741521a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:20.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.871+0000 7fc3727fc640 1 -- 172.21.15.154:0/2598817769 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc364009a70 con 0x7fc3740c8d70 2023-09-23T03:21:20.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.871+0000 7fc3727fc640 1 --2- 172.21.15.154:0/2598817769 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3740c8d70 0x7fc3740c45e0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fc364037040 tx=0x7fc364004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:20.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.871+0000 7fc36b7fe640 1 -- 172.21.15.154:0/2598817769 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc364040650 con 0x7fc3740c8d70 2023-09-23T03:21:20.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.872+0000 7fc36b7fe640 1 -- 172.21.15.154:0/2598817769 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3640407b0 con 0x7fc3740c8d70 2023-09-23T03:21:20.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.872+0000 7fc36b7fe640 1 -- 172.21.15.154:0/2598817769 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc364040b60 con 0x7fc3740c8d70 2023-09-23T03:21:20.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.872+0000 7fc379733640 1 -- 172.21.15.154:0/2598817769 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3741534e0 con 0x7fc3740c8d70 2023-09-23T03:21:20.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.872+0000 7fc379733640 1 -- 172.21.15.154:0/2598817769 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc374154600 con 0x7fc3740c8d70 2023-09-23T03:21:20.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.873+0000 7fc379733640 1 -- 172.21.15.154:0/2598817769 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc374154790 con 0x7fc3740c8d70 2023-09-23T03:21:20.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.874+0000 7fc36b7fe640 1 -- 172.21.15.154:0/2598817769 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fc364040cc0 con 0x7fc3740c8d70 2023-09-23T03:21:20.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.875+0000 7fc36b7fe640 1 --2- 172.21.15.154:0/2598817769 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc35406af30 0x7fc35406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:20.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.875+0000 7fc36b7fe640 1 -- 172.21.15.154:0/2598817769 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6433+0+0 (secure 0 0 0) 0x7fc3640ac850 con 0x7fc3740c8d70 2023-09-23T03:21:20.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.875+0000 7fc372ffd640 1 --2- 172.21.15.154:0/2598817769 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc35406af30 0x7fc35406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:20.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.876+0000 7fc372ffd640 1 --2- 172.21.15.154:0/2598817769 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc35406af30 0x7fc35406d3f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc35c006fd0 tx=0x7fc35c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:20.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:20.879+0000 7fc36b7fe640 1 -- 172.21.15.154:0/2598817769 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fc3640775b0 con 0x7fc3740c8d70 2023-09-23T03:21:21.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.068+0000 7fc379733640 1 -- 172.21.15.154:0/2598817769 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7fc374153a00 con 0x7fc3740c8d70 2023-09-23T03:21:21.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.069+0000 7fc36b7fe640 1 -- 172.21.15.154:0/2598817769 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v65) v1 ==== 106+0+11 (secure 0 0 0) 0x7fc374153a00 con 0x7fc3740c8d70 2023-09-23T03:21:21.070 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:21:21.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.072+0000 7fc379733640 1 -- 172.21.15.154:0/2598817769 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc35406af30 msgr2=0x7fc35406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:21.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.072+0000 7fc379733640 1 --2- 172.21.15.154:0/2598817769 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc35406af30 0x7fc35406d3f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc35c006fd0 tx=0x7fc35c008040 comp rx=0 tx=0).stop 2023-09-23T03:21:21.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.072+0000 7fc379733640 1 -- 172.21.15.154:0/2598817769 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3740c8d70 msgr2=0x7fc3740c45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:21.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.072+0000 7fc379733640 1 --2- 172.21.15.154:0/2598817769 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3740c8d70 0x7fc3740c45e0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fc364037040 tx=0x7fc364004a40 comp rx=0 tx=0).stop 2023-09-23T03:21:21.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.072+0000 7fc379733640 1 -- 172.21.15.154:0/2598817769 shutdown_connections 2023-09-23T03:21:21.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.072+0000 7fc379733640 1 --2- 172.21.15.154:0/2598817769 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc35406af30 0x7fc35406d3f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:21.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.072+0000 7fc379733640 1 --2- 172.21.15.154:0/2598817769 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3740c9150 0x7fc3740ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:21.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.072+0000 7fc379733640 1 --2- 172.21.15.154:0/2598817769 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3740c8d70 0x7fc3740c45e0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:21.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.072+0000 7fc379733640 1 --2- 172.21.15.154:0/2598817769 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc374071290 0x7fc3741521a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:21.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.072+0000 7fc379733640 1 -- 172.21.15.154:0/2598817769 >> 172.21.15.154:0/2598817769 conn(0x7fc37406c010 msgr2=0x7fc37415b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:21.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.073+0000 7fc379733640 1 -- 172.21.15.154:0/2598817769 shutdown_connections 2023-09-23T03:21:21.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.073+0000 7fc379733640 1 -- 172.21.15.154:0/2598817769 wait complete. 2023-09-23T03:21:21.091 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-23T03:21:21.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.301+0000 7f50f17c9640 1 Processor -- start 2023-09-23T03:21:21.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.301+0000 7f50f17c9640 1 -- start start 2023-09-23T03:21:21.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.302+0000 7f50f17c9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50ec15cf40 0x7f50ec1615b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:21.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.303+0000 7f50f17c9640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50ec0cd1c0 con 0x7f50ec0b28e0 2023-09-23T03:21:21.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.303+0000 7f50f17c9640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50ec0cd4a0 con 0x7f50ec06b1e0 2023-09-23T03:21:21.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.303+0000 7f50f17c9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec0cd780 con 0x7f50ec15cf40 2023-09-23T03:21:21.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.303+0000 7f50eb7fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f50ec0b28e0 0x7f50ec15d320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:49330/0 (socket says 172.21.15.154:49330) 2023-09-23T03:21:21.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.303+0000 7f50eb7fe640 1 -- 172.21.15.154:0/2258367735 learned_addr learned my addr 172.21.15.154:0/2258367735 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:21:21.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.303+0000 7f50eaffd640 1 --2- 172.21.15.154:0/2258367735 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50ec15cf40 0x7f50ec1615b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:21.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.303+0000 7f50e9ffb640 1 -- 172.21.15.154:0/2258367735 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3469911756 0 0) 0x7f50ec0cd1c0 con 0x7f50ec0b28e0 2023-09-23T03:21:21.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.303+0000 7f50e9ffb640 1 -- 172.21.15.154:0/2258367735 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50d8003520 con 0x7f50ec0b28e0 2023-09-23T03:21:21.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.304+0000 7f50e9ffb640 1 -- 172.21.15.154:0/2258367735 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3518911985 0 0) 0x7f50d8003520 con 0x7f50ec0b28e0 2023-09-23T03:21:21.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.304+0000 7f50e9ffb640 1 -- 172.21.15.154:0/2258367735 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50ec15cf40 msgr2=0x7f50ec1615b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:21.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.304+0000 7f50e9ffb640 1 --2- 172.21.15.154:0/2258367735 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50ec15cf40 0x7f50ec1615b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:21.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.304+0000 7f50e9ffb640 1 -- 172.21.15.154:0/2258367735 >> v1:172.21.15.190:6789/0 conn(0x7f50ec06b1e0 legacy=0x7f50ec0b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:21.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.304+0000 7f50e9ffb640 1 -- 172.21.15.154:0/2258367735 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ec0cda60 con 0x7f50ec0b28e0 2023-09-23T03:21:21.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.304+0000 7f50e9ffb640 1 -- 172.21.15.154:0/2258367735 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f50e4001670 con 0x7f50ec0b28e0 2023-09-23T03:21:21.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.304+0000 7f50e9ffb640 1 -- 172.21.15.154:0/2258367735 >> v1:172.21.15.154:6789/0 conn(0x7f50ec0b28e0 legacy=0x7f50ec15d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:21.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.304+0000 7f50e9ffb640 1 --2- 172.21.15.154:0/2258367735 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50ec15cf40 0x7f50d80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:21.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.304+0000 7f50e9ffb640 1 --2- 172.21.15.154:0/2258367735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50d8003f70 0x7f50d80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:21.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.304+0000 7f50e9ffb640 1 --2- 172.21.15.154:0/2258367735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50d8004d00 0x7f50d8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:21.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.304+0000 7f50e9ffb640 1 -- 172.21.15.154:0/2258367735 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec0cd780 con 0x7f50d8003f70 2023-09-23T03:21:21.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.304+0000 7f50e9ffb640 1 -- 172.21.15.154:0/2258367735 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d800bac0 con 0x7f50d8004d00 2023-09-23T03:21:21.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.304+0000 7f50e9ffb640 1 -- 172.21.15.154:0/2258367735 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d800bc80 con 0x7f50ec15cf40 2023-09-23T03:21:21.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.304+0000 7f50ea7fc640 1 --2- 172.21.15.154:0/2258367735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50d8003f70 0x7f50d80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:21.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.305+0000 7f50eb7fe640 1 --2- 172.21.15.154:0/2258367735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50d8004d00 0x7f50d8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:21.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.305+0000 7f50ea7fc640 1 -- 172.21.15.154:0/2258367735 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50ec15cf40 msgr2=0x7f50d80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:21.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.305+0000 7f50ea7fc640 1 --2- 172.21.15.154:0/2258367735 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50ec15cf40 0x7f50d80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:21.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.305+0000 7f50ea7fc640 1 -- 172.21.15.154:0/2258367735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50d8004d00 msgr2=0x7f50d8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:21.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.305+0000 7f50ea7fc640 1 --2- 172.21.15.154:0/2258367735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50d8004d00 0x7f50d8005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:21.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.305+0000 7f50ea7fc640 1 -- 172.21.15.154:0/2258367735 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d8003520 con 0x7f50d8003f70 2023-09-23T03:21:21.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.306+0000 7f50ea7fc640 1 --2- 172.21.15.154:0/2258367735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50d8003f70 0x7f50d80047c0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f50d4001720 tx=0x7f50d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1001e453989d849 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:21.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.306+0000 7f50e9ffb640 1 -- 172.21.15.154:0/2258367735 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d40052d0 con 0x7f50d8003f70 2023-09-23T03:21:21.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.306+0000 7f50e9ffb640 1 -- 172.21.15.154:0/2258367735 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d402e050 con 0x7f50d8003f70 2023-09-23T03:21:21.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.306+0000 7f50e9ffb640 1 -- 172.21.15.154:0/2258367735 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d4003830 con 0x7f50d8003f70 2023-09-23T03:21:21.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.306+0000 7f50f17c9640 1 -- 172.21.15.154:0/2258367735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50d8003f70 msgr2=0x7f50d80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:21.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.306+0000 7f50f17c9640 1 --2- 172.21.15.154:0/2258367735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50d8003f70 0x7f50d80047c0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f50d4001720 tx=0x7f50d80057f0 comp rx=0 tx=0).stop 2023-09-23T03:21:21.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.306+0000 7f50eb7fe640 1 -- 172.21.15.154:0/2258367735 reap_dead start 2023-09-23T03:21:21.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.307+0000 7f50f17c9640 1 -- 172.21.15.154:0/2258367735 shutdown_connections 2023-09-23T03:21:21.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.307+0000 7f50f17c9640 1 -- 172.21.15.154:0/2258367735 >> 172.21.15.154:0/2258367735 conn(0x7f50ec06fc60 msgr2=0x7f50ec070040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:21.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.307+0000 7f50f17c9640 1 -- 172.21.15.154:0/2258367735 shutdown_connections 2023-09-23T03:21:21.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.307+0000 7f50f17c9640 1 -- 172.21.15.154:0/2258367735 wait complete. 2023-09-23T03:21:21.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.308+0000 7f50f17c9640 1 Processor -- start 2023-09-23T03:21:21.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.309+0000 7f50f17c9640 1 -- start start 2023-09-23T03:21:21.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.309+0000 7f50f17c9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50d8003f70 0x7f50ec15d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:21.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.309+0000 7f50f17c9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50ec149af0 0x7f50ec15b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:21.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.310+0000 7f50f17c9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50ec14a3a0 0x7f50ec0b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:21.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.310+0000 7f50f17c9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec15e510 con 0x7f50d8003f70 2023-09-23T03:21:21.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.310+0000 7f50f17c9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec15d900 con 0x7f50ec149af0 2023-09-23T03:21:21.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.310+0000 7f50f17c9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec15dbe0 con 0x7f50ec14a3a0 2023-09-23T03:21:21.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.310+0000 7f50eaffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50d8003f70 0x7f50ec15d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:21.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.310+0000 7f50eaffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50d8003f70 0x7f50ec15d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:40666/0 (socket says 172.21.15.154:40666) 2023-09-23T03:21:21.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.310+0000 7f50eaffd640 1 -- 172.21.15.154:0/1086434493 learned_addr learned my addr 172.21.15.154:0/1086434493 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:21.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.310+0000 7f50eaffd640 1 -- 172.21.15.154:0/1086434493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50ec14a3a0 msgr2=0x7f50ec0b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:21.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.310+0000 7f50eaffd640 1 --2- 172.21.15.154:0/1086434493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50ec14a3a0 0x7f50ec0b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:21.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.310+0000 7f50eaffd640 1 -- 172.21.15.154:0/1086434493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50ec149af0 msgr2=0x7f50ec15b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:21.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.310+0000 7f50eaffd640 1 --2- 172.21.15.154:0/1086434493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50ec149af0 0x7f50ec15b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:21.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.310+0000 7f50eaffd640 1 -- 172.21.15.154:0/1086434493 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ec15dec0 con 0x7f50d8003f70 2023-09-23T03:21:21.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.310+0000 7f50eaffd640 1 --2- 172.21.15.154:0/1086434493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50d8003f70 0x7f50ec15d320 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f50dc00ef10 tx=0x7f50dc00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:21.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.311+0000 7f50e37fe640 1 -- 172.21.15.154:0/1086434493 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50dc010040 con 0x7f50d8003f70 2023-09-23T03:21:21.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.311+0000 7f50e37fe640 1 -- 172.21.15.154:0/1086434493 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50dc014400 con 0x7f50d8003f70 2023-09-23T03:21:21.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.311+0000 7f50e37fe640 1 -- 172.21.15.154:0/1086434493 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50dc014790 con 0x7f50d8003f70 2023-09-23T03:21:21.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.311+0000 7f50f17c9640 1 -- 172.21.15.154:0/1086434493 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50ec15e1a0 con 0x7f50d8003f70 2023-09-23T03:21:21.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.311+0000 7f50f17c9640 1 -- 172.21.15.154:0/1086434493 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50ec149f50 con 0x7f50d8003f70 2023-09-23T03:21:21.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.312+0000 7f50f17c9640 1 -- 172.21.15.154:0/1086434493 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50ec073eb0 con 0x7f50d8003f70 2023-09-23T03:21:21.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.313+0000 7f50e37fe640 1 -- 172.21.15.154:0/1086434493 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f50dc0148f0 con 0x7f50d8003f70 2023-09-23T03:21:21.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.314+0000 7f50e37fe640 1 --2- 172.21.15.154:0/1086434493 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f50cc06af30 0x7f50cc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:21.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.314+0000 7f50e37fe640 1 -- 172.21.15.154:0/1086434493 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f50dc08a520 con 0x7f50d8003f70 2023-09-23T03:21:21.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.314+0000 7f50ea7fc640 1 --2- 172.21.15.154:0/1086434493 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f50cc06af30 0x7f50cc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:21.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.315+0000 7f50ea7fc640 1 --2- 172.21.15.154:0/1086434493 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f50cc06af30 0x7f50cc06d3f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f50d4033000 tx=0x7f50d402f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:21.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.319+0000 7f50e37fe640 1 -- 172.21.15.154:0/1086434493 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f50dc055280 con 0x7f50d8003f70 2023-09-23T03:21:21.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.511+0000 7f50f17c9640 1 -- 172.21.15.154:0/1086434493 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f50ec0c6d90 con 0x7f50d8003f70 2023-09-23T03:21:21.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.512+0000 7f50e37fe640 1 -- 172.21.15.154:0/1086434493 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v65) v1 ==== 80+0+0 (secure 0 0 0) 0x7f50dc009c60 con 0x7f50d8003f70 2023-09-23T03:21:21.512 INFO:teuthology.orchestra.run.smithi154.stderr:listed 0 entries 2023-09-23T03:21:21.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.514+0000 7f50f17c9640 1 -- 172.21.15.154:0/1086434493 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f50cc06af30 msgr2=0x7f50cc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:21.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.514+0000 7f50f17c9640 1 --2- 172.21.15.154:0/1086434493 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f50cc06af30 0x7f50cc06d3f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f50d4033000 tx=0x7f50d402f000 comp rx=0 tx=0).stop 2023-09-23T03:21:21.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.514+0000 7f50f17c9640 1 -- 172.21.15.154:0/1086434493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50d8003f70 msgr2=0x7f50ec15d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:21.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.514+0000 7f50f17c9640 1 --2- 172.21.15.154:0/1086434493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50d8003f70 0x7f50ec15d320 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f50dc00ef10 tx=0x7f50dc00b480 comp rx=0 tx=0).stop 2023-09-23T03:21:21.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.514+0000 7f50f17c9640 1 -- 172.21.15.154:0/1086434493 shutdown_connections 2023-09-23T03:21:21.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.514+0000 7f50f17c9640 1 --2- 172.21.15.154:0/1086434493 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f50cc06af30 0x7f50cc06d3f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:21.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.514+0000 7f50f17c9640 1 --2- 172.21.15.154:0/1086434493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50ec14a3a0 0x7f50ec0b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:21.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.514+0000 7f50f17c9640 1 --2- 172.21.15.154:0/1086434493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50ec149af0 0x7f50ec15b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:21.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.514+0000 7f50f17c9640 1 --2- 172.21.15.154:0/1086434493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50d8003f70 0x7f50ec15d320 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:21.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.514+0000 7f50f17c9640 1 -- 172.21.15.154:0/1086434493 >> 172.21.15.154:0/1086434493 conn(0x7f50ec06fc60 msgr2=0x7f50ec0c8910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:21.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.514+0000 7f50f17c9640 1 -- 172.21.15.154:0/1086434493 shutdown_connections 2023-09-23T03:21:21.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:21.515+0000 7f50f17c9640 1 -- 172.21.15.154:0/1086434493 wait complete. 2023-09-23T03:21:21.533 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi154.front.sepia.ceph.com... 2023-09-23T03:21:21.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:21:21.533 DEBUG:teuthology.orchestra.run.smithi154:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-09-23T03:21:21.615 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:ceph-fuse[108844]: fuse finished with error 0 and tester_r 0 2023-09-23T03:21:21.666 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:21:27.768 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2023-09-23T03:21:27.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:21:27.768 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-09-23T03:21:27.829 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:21:28.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.050+0000 7efdffb78640 1 Processor -- start 2023-09-23T03:21:28.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.050+0000 7efdffb78640 1 -- start start 2023-09-23T03:21:28.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.052+0000 7efdffb78640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdf80c4300 0x7efdf815b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:28.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.052+0000 7efdffb78640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdf8066050 con 0x7efdf8158480 2023-09-23T03:21:28.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.052+0000 7efdffb78640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdf815e3e0 con 0x7efdf806e050 2023-09-23T03:21:28.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.052+0000 7efdffb78640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdf815bcb0 con 0x7efdf80c4300 2023-09-23T03:21:28.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.052+0000 7efdfd914640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7efdf8158480 0x7efdf80c3bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:39192/0 (socket says 172.21.15.154:39192) 2023-09-23T03:21:28.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.052+0000 7efdfd914640 1 -- 172.21.15.154:0/1635070611 learned_addr learned my addr 172.21.15.154:0/1635070611 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:21:28.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.053+0000 7efdfd113640 1 --2- 172.21.15.154:0/1635070611 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdf80c4300 0x7efdf815b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:28.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.053+0000 7efdfc912640 1 -- 172.21.15.154:0/1635070611 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2912019935 0 0) 0x7efdf8066050 con 0x7efdf8158480 2023-09-23T03:21:28.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.053+0000 7efdfc912640 1 -- 172.21.15.154:0/1635070611 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efde0003520 con 0x7efdf8158480 2023-09-23T03:21:28.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.053+0000 7efdfc912640 1 -- 172.21.15.154:0/1635070611 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 62062501 0 0) 0x7efde0003520 con 0x7efdf8158480 2023-09-23T03:21:28.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.053+0000 7efdfc912640 1 -- 172.21.15.154:0/1635070611 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdf80c4300 msgr2=0x7efdf815b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:28.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.053+0000 7efdfc912640 1 --2- 172.21.15.154:0/1635070611 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdf80c4300 0x7efdf815b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:28.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.053+0000 7efdfc912640 1 -- 172.21.15.154:0/1635070611 >> v1:172.21.15.190:6789/0 conn(0x7efdf806e050 legacy=0x7efdf8069d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:28.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.053+0000 7efdfc912640 1 -- 172.21.15.154:0/1635070611 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf815e6c0 con 0x7efdf8158480 2023-09-23T03:21:28.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.053+0000 7efdfc912640 1 -- 172.21.15.154:0/1635070611 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7efde80019f0 con 0x7efdf8158480 2023-09-23T03:21:28.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.053+0000 7efdfc912640 1 -- 172.21.15.154:0/1635070611 >> v1:172.21.15.154:6789/0 conn(0x7efdf8158480 legacy=0x7efdf80c3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:28.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.053+0000 7efdfc912640 1 --2- 172.21.15.154:0/1635070611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efde00039a0 0x7efde0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:28.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.053+0000 7efdfc912640 1 --2- 172.21.15.154:0/1635070611 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdf80c4300 0x7efde00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:28.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.053+0000 7efdfc912640 1 --2- 172.21.15.154:0/1635070611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efde0004d00 0x7efde0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:28.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.053+0000 7efdfc912640 1 -- 172.21.15.154:0/1635070611 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf815bcb0 con 0x7efde00039a0 2023-09-23T03:21:28.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.053+0000 7efdfc912640 1 -- 172.21.15.154:0/1635070611 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde000bab0 con 0x7efde0004d00 2023-09-23T03:21:28.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.053+0000 7efdfc912640 1 -- 172.21.15.154:0/1635070611 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efde000bc70 con 0x7efdf80c4300 2023-09-23T03:21:28.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.054+0000 7efdfd914640 1 --2- 172.21.15.154:0/1635070611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efde00039a0 0x7efde0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:28.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.054+0000 7efdfd914640 1 -- 172.21.15.154:0/1635070611 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdf80c4300 msgr2=0x7efde00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:28.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.054+0000 7efdfd914640 1 --2- 172.21.15.154:0/1635070611 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdf80c4300 0x7efde00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:28.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.054+0000 7efdfe115640 1 --2- 172.21.15.154:0/1635070611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efde0004d00 0x7efde0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:28.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.054+0000 7efdfd914640 1 -- 172.21.15.154:0/1635070611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efde0004d00 msgr2=0x7efde0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:28.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.054+0000 7efdfd914640 1 --2- 172.21.15.154:0/1635070611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efde0004d00 0x7efde0005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:28.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.054+0000 7efdfd914640 1 -- 172.21.15.154:0/1635070611 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde0003520 con 0x7efde00039a0 2023-09-23T03:21:28.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.055+0000 7efdfd914640 1 --2- 172.21.15.154:0/1635070611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efde00039a0 0x7efde0003dc0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7efde800a830 tx=0x7efde00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b8f05eac788aeb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:28.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.055+0000 7efdfc912640 1 -- 172.21.15.154:0/1635070611 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde80072d0 con 0x7efde00039a0 2023-09-23T03:21:28.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.055+0000 7efdfc912640 1 -- 172.21.15.154:0/1635070611 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde802f070 con 0x7efde00039a0 2023-09-23T03:21:28.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.055+0000 7efdfc912640 1 -- 172.21.15.154:0/1635070611 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde80076b0 con 0x7efde00039a0 2023-09-23T03:21:28.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.056+0000 7efdffb78640 1 -- 172.21.15.154:0/1635070611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efde00039a0 msgr2=0x7efde0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:28.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.056+0000 7efdffb78640 1 --2- 172.21.15.154:0/1635070611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efde00039a0 0x7efde0003dc0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7efde800a830 tx=0x7efde00057f0 comp rx=0 tx=0).stop 2023-09-23T03:21:28.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.056+0000 7efdfe115640 1 -- 172.21.15.154:0/1635070611 reap_dead start 2023-09-23T03:21:28.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.056+0000 7efdffb78640 1 -- 172.21.15.154:0/1635070611 shutdown_connections 2023-09-23T03:21:28.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.056+0000 7efdffb78640 1 -- 172.21.15.154:0/1635070611 >> 172.21.15.154:0/1635070611 conn(0x7efdf80cb790 msgr2=0x7efdf80cbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:28.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.056+0000 7efdffb78640 1 -- 172.21.15.154:0/1635070611 shutdown_connections 2023-09-23T03:21:28.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.056+0000 7efdffb78640 1 -- 172.21.15.154:0/1635070611 wait complete. 2023-09-23T03:21:28.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.058+0000 7efdffb78640 1 Processor -- start 2023-09-23T03:21:28.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.058+0000 7efdffb78640 1 -- start start 2023-09-23T03:21:28.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.059+0000 7efdffb78640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efde00039a0 0x7efdf8155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:28.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.059+0000 7efdffb78640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdf814f720 0x7efdf8143630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:28.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.059+0000 7efdffb78640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdf8151640 0x7efdf8154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:28.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.059+0000 7efdffb78640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf8151220 con 0x7efdf8151640 2023-09-23T03:21:28.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.059+0000 7efdffb78640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdf814ffe0 con 0x7efdf814f720 2023-09-23T03:21:28.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.059+0000 7efdffb78640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdf81505e0 con 0x7efde00039a0 2023-09-23T03:21:28.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.059+0000 7efdfe115640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdf8151640 0x7efdf8154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:28.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.060+0000 7efdfe115640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdf8151640 0x7efdf8154180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49514/0 (socket says 172.21.15.154:49514) 2023-09-23T03:21:28.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.060+0000 7efdfe115640 1 -- 172.21.15.154:0/2013386753 learned_addr learned my addr 172.21.15.154:0/2013386753 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:28.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.060+0000 7efdfd914640 1 --2- 172.21.15.154:0/2013386753 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efde00039a0 0x7efdf8155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:28.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.060+0000 7efdfd113640 1 --2- 172.21.15.154:0/2013386753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdf814f720 0x7efdf8143630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:28.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.060+0000 7efdfe115640 1 -- 172.21.15.154:0/2013386753 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efde00039a0 msgr2=0x7efdf8155c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:28.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.060+0000 7efdfe115640 1 --2- 172.21.15.154:0/2013386753 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efde00039a0 0x7efdf8155c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:28.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.060+0000 7efdfe115640 1 -- 172.21.15.154:0/2013386753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdf814f720 msgr2=0x7efdf8143630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:28.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.060+0000 7efdfe115640 1 --2- 172.21.15.154:0/2013386753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdf814f720 0x7efdf8143630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:28.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.060+0000 7efdfe115640 1 -- 172.21.15.154:0/2013386753 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf8150880 con 0x7efdf8151640 2023-09-23T03:21:28.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.060+0000 7efdfd113640 1 --2- 172.21.15.154:0/2013386753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdf814f720 0x7efdf8143630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:28.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.060+0000 7efdfe115640 1 --2- 172.21.15.154:0/2013386753 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdf8151640 0x7efdf8154180 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7efdf80c5340 tx=0x7efdf000d750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:28.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.060+0000 7efdf67fc640 1 -- 172.21.15.154:0/2013386753 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf000b070 con 0x7efdf8151640 2023-09-23T03:21:28.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.061+0000 7efdf67fc640 1 -- 172.21.15.154:0/2013386753 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf00054e0 con 0x7efdf8151640 2023-09-23T03:21:28.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.061+0000 7efdf67fc640 1 -- 172.21.15.154:0/2013386753 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf0003810 con 0x7efdf8151640 2023-09-23T03:21:28.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.061+0000 7efdffb78640 1 -- 172.21.15.154:0/2013386753 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdf8150b60 con 0x7efdf8151640 2023-09-23T03:21:28.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.061+0000 7efdffb78640 1 -- 172.21.15.154:0/2013386753 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdf8143b70 con 0x7efdf8151640 2023-09-23T03:21:28.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.062+0000 7efdffb78640 1 -- 172.21.15.154:0/2013386753 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdf8151360 con 0x7efdf8151640 2023-09-23T03:21:28.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.063+0000 7efdf67fc640 1 -- 172.21.15.154:0/2013386753 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7efdf0003970 con 0x7efdf8151640 2023-09-23T03:21:28.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.064+0000 7efdf67fc640 1 --2- 172.21.15.154:0/2013386753 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efde406af60 0x7efde406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:28.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.064+0000 7efdf67fc640 1 -- 172.21.15.154:0/2013386753 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6433+0+0 (secure 0 0 0) 0x7efdf0088fe0 con 0x7efdf8151640 2023-09-23T03:21:28.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.064+0000 7efdfd914640 1 --2- 172.21.15.154:0/2013386753 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efde406af60 0x7efde406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:28.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.065+0000 7efdfd914640 1 --2- 172.21.15.154:0/2013386753 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efde406af60 0x7efde406d420 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7efde8034000 tx=0x7efde8007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:28.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.068+0000 7efdf67fc640 1 -- 172.21.15.154:0/2013386753 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7efdf8151360 con 0x7efdf8151640 2023-09-23T03:21:28.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.294+0000 7efdffb78640 1 -- 172.21.15.154:0/2013386753 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efdf8151360 con 0x7efdf8151640 2023-09-23T03:21:28.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.295+0000 7efdf67fc640 1 -- 172.21.15.154:0/2013386753 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4928 (secure 0 0 0) 0x7efdf8151360 con 0x7efdf8151640 2023-09-23T03:21:28.296 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:28.296 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":41,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5027,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2517324433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2517324433},{"type":"v1","addr":"172.21.15.190:6833","nonce":2517324433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":5033,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4098915995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4098915995},{"type":"v1","addr":"172.21.15.190:6835","nonce":4098915995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6309,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/3034894009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":3034894009},{"type":"v1","addr":"172.21.15.154:6836","nonce":3034894009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:19:23.952758+0000","modified":"2023-09-23T03:19:39.412021+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6312},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6312":{"gid":6312,"name":"c","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6837/2183023203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":2183023203},{"type":"v1","addr":"172.21.15.154:6837","nonce":2183023203}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9,10],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-09-23T03:21:28.296 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 41 2023-09-23T03:21:28.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.298+0000 7efdffb78640 1 -- 172.21.15.154:0/2013386753 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efde406af60 msgr2=0x7efde406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:28.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.298+0000 7efdffb78640 1 --2- 172.21.15.154:0/2013386753 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efde406af60 0x7efde406d420 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7efde8034000 tx=0x7efde8007000 comp rx=0 tx=0).stop 2023-09-23T03:21:28.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.298+0000 7efdffb78640 1 -- 172.21.15.154:0/2013386753 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdf8151640 msgr2=0x7efdf8154180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:28.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.298+0000 7efdffb78640 1 --2- 172.21.15.154:0/2013386753 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdf8151640 0x7efdf8154180 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7efdf80c5340 tx=0x7efdf000d750 comp rx=0 tx=0).stop 2023-09-23T03:21:28.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.298+0000 7efdffb78640 1 -- 172.21.15.154:0/2013386753 shutdown_connections 2023-09-23T03:21:28.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.298+0000 7efdffb78640 1 --2- 172.21.15.154:0/2013386753 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efde406af60 0x7efde406d420 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:28.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.298+0000 7efdffb78640 1 --2- 172.21.15.154:0/2013386753 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdf8151640 0x7efdf8154180 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:28.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.298+0000 7efdffb78640 1 --2- 172.21.15.154:0/2013386753 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdf814f720 0x7efdf8143630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:28.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.298+0000 7efdffb78640 1 --2- 172.21.15.154:0/2013386753 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efde00039a0 0x7efdf8155c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:28.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.298+0000 7efdffb78640 1 -- 172.21.15.154:0/2013386753 >> 172.21.15.154:0/2013386753 conn(0x7efdf80cb790 msgr2=0x7efdf80c95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:28.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.298+0000 7efdffb78640 1 -- 172.21.15.154:0/2013386753 shutdown_connections 2023-09-23T03:21:28.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.298+0000 7efdffb78640 1 -- 172.21.15.154:0/2013386753 wait complete. 2023-09-23T03:21:28.319 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2023-09-23T03:21:28.583 INFO:teuthology.orchestra.run.smithi154.stdout:{"message":"","return_code":0} 2023-09-23T03:21:28.583 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-09-23T03:21:28.584 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-09-23T03:21:28.782 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.781+0000 7f6906aaf640 1 Processor -- start 2023-09-23T03:21:28.782 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.782+0000 7f6906aaf640 1 -- start start 2023-09-23T03:21:28.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.783+0000 7f6906aaf640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f69000b1b20 0x7f69000ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:28.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.783+0000 7f6906aaf640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69000cee70 0x7f69000ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:28.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.783+0000 7f6906aaf640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f690015ac30 con 0x7f69000acce0 2023-09-23T03:21:28.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.783+0000 7f6906aaf640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69000b2e90 con 0x7f69000b1b20 2023-09-23T03:21:28.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.783+0000 7f6906aaf640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69000cf320 con 0x7f69000cee70 2023-09-23T03:21:28.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.783+0000 7f68fffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f69000b1b20 0x7f69000ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:28.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.783+0000 7f68fffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f69000b1b20 0x7f69000ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49516/0 (socket says 172.21.15.154:49516) 2023-09-23T03:21:28.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.783+0000 7f68fffff640 1 -- 172.21.15.154:0/2069119865 learned_addr learned my addr 172.21.15.154:0/2069119865 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:28.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.784+0000 7f690504c640 1 --2- 172.21.15.154:0/2069119865 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69000cee70 0x7f69000ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:28.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.784+0000 7f68fffff640 1 -- 172.21.15.154:0/2069119865 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69000cee70 msgr2=0x7f69000ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:28.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.784+0000 7f68fffff640 1 --2- 172.21.15.154:0/2069119865 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69000cee70 0x7f69000ab140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:28.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.784+0000 7f68fffff640 1 -- 172.21.15.154:0/2069119865 >> v1:172.21.15.190:6789/0 conn(0x7f69000acce0 legacy=0x7f690015ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:28.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.784+0000 7f68fffff640 1 -- 172.21.15.154:0/2069119865 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69000b1f70 con 0x7f69000b1b20 2023-09-23T03:21:28.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.784+0000 7f690504c640 1 --2- 172.21.15.154:0/2069119865 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69000cee70 0x7f69000ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:28.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.785+0000 7f68fffff640 1 --2- 172.21.15.154:0/2069119865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69000b1b20 0x7f69000ce930 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f68ec009870 tx=0x7f68ec02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=192cddf03c8a947f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:28.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.785+0000 7f68ff7fe640 1 -- 172.21.15.154:0/2069119865 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ec0317c0 con 0x7f69000b1b20 2023-09-23T03:21:28.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.785+0000 7f68ff7fe640 1 -- 172.21.15.154:0/2069119865 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68ec031920 con 0x7f69000b1b20 2023-09-23T03:21:28.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.785+0000 7f68ff7fe640 1 -- 172.21.15.154:0/2069119865 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ec031cb0 con 0x7f69000b1b20 2023-09-23T03:21:28.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.786+0000 7f6906aaf640 1 -- 172.21.15.154:0/2069119865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69000b1b20 msgr2=0x7f69000ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:28.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.786+0000 7f6906aaf640 1 --2- 172.21.15.154:0/2069119865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69000b1b20 0x7f69000ce930 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f68ec009870 tx=0x7f68ec02b860 comp rx=0 tx=0).stop 2023-09-23T03:21:28.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.786+0000 7f6906aaf640 1 -- 172.21.15.154:0/2069119865 shutdown_connections 2023-09-23T03:21:28.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.786+0000 7f6906aaf640 1 --2- 172.21.15.154:0/2069119865 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69000cee70 0x7f69000ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:28.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.786+0000 7f6906aaf640 1 --2- 172.21.15.154:0/2069119865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69000b1b20 0x7f69000ce930 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:28.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.786+0000 7f6906aaf640 1 -- 172.21.15.154:0/2069119865 >> 172.21.15.154:0/2069119865 conn(0x7f690006a0f0 msgr2=0x7f690006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:28.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.786+0000 7f6906aaf640 1 -- 172.21.15.154:0/2069119865 shutdown_connections 2023-09-23T03:21:28.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.786+0000 7f6906aaf640 1 -- 172.21.15.154:0/2069119865 wait complete. 2023-09-23T03:21:28.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.788+0000 7f6906aaf640 1 Processor -- start 2023-09-23T03:21:28.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.789+0000 7f6906aaf640 1 -- start start 2023-09-23T03:21:28.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.790+0000 7f6906aaf640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69000acce0 0x7f69000c46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:28.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.790+0000 7f6906aaf640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69000b1b20 0x7f69000972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:28.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.790+0000 7f6906aaf640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69000cee70 0x7f69000977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:28.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.790+0000 7f6906aaf640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6900156940 con 0x7f69000b1b20 2023-09-23T03:21:28.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.790+0000 7f6906aaf640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69000cdde0 con 0x7f69000cee70 2023-09-23T03:21:28.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.790+0000 7f6906aaf640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6900156b90 con 0x7f69000acce0 2023-09-23T03:21:28.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.790+0000 7f68fffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69000b1b20 0x7f69000972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:28.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.791+0000 7f68fffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69000b1b20 0x7f69000972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49518/0 (socket says 172.21.15.154:49518) 2023-09-23T03:21:28.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.791+0000 7f68fffff640 1 -- 172.21.15.154:0/3693321239 learned_addr learned my addr 172.21.15.154:0/3693321239 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:28.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.791+0000 7f690484b640 1 --2- 172.21.15.154:0/3693321239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69000acce0 0x7f69000c46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:28.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.791+0000 7f690504c640 1 --2- 172.21.15.154:0/3693321239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69000cee70 0x7f69000977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:28.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.791+0000 7f68fffff640 1 -- 172.21.15.154:0/3693321239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69000acce0 msgr2=0x7f69000c46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:28.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.791+0000 7f68fffff640 1 --2- 172.21.15.154:0/3693321239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69000acce0 0x7f69000c46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:28.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.791+0000 7f68fffff640 1 -- 172.21.15.154:0/3693321239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69000cee70 msgr2=0x7f69000977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:28.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.791+0000 7f68fffff640 1 --2- 172.21.15.154:0/3693321239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69000cee70 0x7f69000977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:28.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.791+0000 7f68fffff640 1 -- 172.21.15.154:0/3693321239 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ec0094d0 con 0x7f69000b1b20 2023-09-23T03:21:28.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.791+0000 7f68fffff640 1 --2- 172.21.15.154:0/3693321239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69000b1b20 0x7f69000972b0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f68ec009870 tx=0x7f68ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:28.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.792+0000 7f68fd7fa640 1 -- 172.21.15.154:0/3693321239 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ec032960 con 0x7f69000b1b20 2023-09-23T03:21:28.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.792+0000 7f68fd7fa640 1 -- 172.21.15.154:0/3693321239 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68ec032ac0 con 0x7f69000b1b20 2023-09-23T03:21:28.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.792+0000 7f6906aaf640 1 -- 172.21.15.154:0/3693321239 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6900156e70 con 0x7f69000b1b20 2023-09-23T03:21:28.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.792+0000 7f68fd7fa640 1 -- 172.21.15.154:0/3693321239 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ec032e70 con 0x7f69000b1b20 2023-09-23T03:21:28.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.792+0000 7f6906aaf640 1 -- 172.21.15.154:0/3693321239 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69000c4c00 con 0x7f69000b1b20 2023-09-23T03:21:28.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.793+0000 7f6906aaf640 1 -- 172.21.15.154:0/3693321239 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69000c4df0 con 0x7f69000b1b20 2023-09-23T03:21:28.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.794+0000 7f68fd7fa640 1 -- 172.21.15.154:0/3693321239 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f68ec04e450 con 0x7f69000b1b20 2023-09-23T03:21:28.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.795+0000 7f68fd7fa640 1 --2- 172.21.15.154:0/3693321239 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f68e406af30 0x7f68e406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:28.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.795+0000 7f68fd7fa640 1 -- 172.21.15.154:0/3693321239 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f68ec0abcb0 con 0x7f69000b1b20 2023-09-23T03:21:28.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.795+0000 7f690484b640 1 --2- 172.21.15.154:0/3693321239 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f68e406af30 0x7f68e406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:28.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.796+0000 7f690484b640 1 --2- 172.21.15.154:0/3693321239 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f68e406af30 0x7f68e406d3f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f68f4004e50 tx=0x7f68f4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:28.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:28.799+0000 7f68fd7fa640 1 -- 172.21.15.154:0/3693321239 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f68ec076b20 con 0x7f69000b1b20 2023-09-23T03:21:29.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.031+0000 7f6906aaf640 1 -- 172.21.15.154:0/3693321239 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6900157390 con 0x7f69000b1b20 2023-09-23T03:21:29.033 INFO:tasks.ceph.mon.a.smithi154.stderr:2023-09-23T03:21:29.032+0000 7f3df35a6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-23T03:21:29.036 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.036+0000 7f68fd7fa640 1 -- 172.21.15.154:0/3693321239 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v42) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6900157390 con 0x7f69000b1b20 2023-09-23T03:21:29.036 INFO:teuthology.orchestra.run.smithi154.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-09-23T03:21:29.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.040+0000 7f6906aaf640 1 -- 172.21.15.154:0/3693321239 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f68e406af30 msgr2=0x7f68e406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:29.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.040+0000 7f6906aaf640 1 --2- 172.21.15.154:0/3693321239 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f68e406af30 0x7f68e406d3f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f68f4004e50 tx=0x7f68f4009040 comp rx=0 tx=0).stop 2023-09-23T03:21:29.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.040+0000 7f6906aaf640 1 -- 172.21.15.154:0/3693321239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69000b1b20 msgr2=0x7f69000972b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:29.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.040+0000 7f6906aaf640 1 --2- 172.21.15.154:0/3693321239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69000b1b20 0x7f69000972b0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f68ec009870 tx=0x7f68ec0046c0 comp rx=0 tx=0).stop 2023-09-23T03:21:29.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.040+0000 7f6906aaf640 1 -- 172.21.15.154:0/3693321239 shutdown_connections 2023-09-23T03:21:29.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.040+0000 7f6906aaf640 1 --2- 172.21.15.154:0/3693321239 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f68e406af30 0x7f68e406d3f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:29.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.040+0000 7f6906aaf640 1 --2- 172.21.15.154:0/3693321239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f69000cee70 0x7f69000977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:29.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.040+0000 7f6906aaf640 1 --2- 172.21.15.154:0/3693321239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69000b1b20 0x7f69000972b0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:29.042 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.040+0000 7f6906aaf640 1 --2- 172.21.15.154:0/3693321239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f69000acce0 0x7f69000c46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:29.042 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.040+0000 7f6906aaf640 1 -- 172.21.15.154:0/3693321239 >> 172.21.15.154:0/3693321239 conn(0x7f690006a0f0 msgr2=0x7f69000cb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:29.042 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.041+0000 7f6906aaf640 1 -- 172.21.15.154:0/3693321239 shutdown_connections 2023-09-23T03:21:29.042 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.041+0000 7f6906aaf640 1 -- 172.21.15.154:0/3693321239 wait complete. 2023-09-23T03:21:29.061 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-23T03:21:29.066 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-23T03:21:29.125 INFO:tasks.ceph.mds.c.smithi154.stdout:starting mds.c at 2023-09-23T03:21:29.212 INFO:teuthology.orchestra.run.smithi154.stdout:[global] 2023-09-23T03:21:29.212 INFO:teuthology.orchestra.run.smithi154.stdout: chdir = "" 2023-09-23T03:21:29.212 INFO:teuthology.orchestra.run.smithi154.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-23T03:21:29.213 INFO:teuthology.orchestra.run.smithi154.stdout: auth supported = cephx 2023-09-23T03:21:29.213 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.213 INFO:teuthology.orchestra.run.smithi154.stdout: filestore xattr use omap = true 2023-09-23T03:21:29.213 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.213 INFO:teuthology.orchestra.run.smithi154.stdout: mon clock drift allowed = 1.000 2023-09-23T03:21:29.213 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.213 INFO:teuthology.orchestra.run.smithi154.stdout: osd crush chooseleaf type = 0 2023-09-23T03:21:29.213 INFO:teuthology.orchestra.run.smithi154.stdout: auth debug = true 2023-09-23T03:21:29.213 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.213 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on old message = true 2023-09-23T03:21:29.213 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on bug = true 2023-09-23T03:21:29.213 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.214 INFO:teuthology.orchestra.run.smithi154.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-23T03:21:29.214 INFO:teuthology.orchestra.run.smithi154.stdout: mon pg warn max object skew = 0 2023-09-23T03:21:29.214 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.214 INFO:teuthology.orchestra.run.smithi154.stdout: # disable pg_autoscaler by default for new pools 2023-09-23T03:21:29.214 INFO:teuthology.orchestra.run.smithi154.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-23T03:21:29.214 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.214 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default size = 2 2023-09-23T03:21:29.214 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.214 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow primary affinity = true 2023-09-23T03:21:29.214 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow pg remap = true 2023-09-23T03:21:29.214 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on legacy crush tunables = false 2023-09-23T03:21:29.215 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on crush straw calc version zero = false 2023-09-23T03:21:29.215 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on no sortbitwise = false 2023-09-23T03:21:29.215 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on osd down out interval zero = false 2023-09-23T03:21:29.215 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on too few osds = false 2023-09-23T03:21:29.215 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-23T03:21:29.215 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-23T03:21:29.215 INFO:teuthology.orchestra.run.smithi154.stdout: mon_allow_pool_size_one = true 2023-09-23T03:21:29.215 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.215 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-23T03:21:29.215 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.215 INFO:teuthology.orchestra.run.smithi154.stdout: osd default data pool replay window = 5 2023-09-23T03:21:29.216 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.216 INFO:teuthology.orchestra.run.smithi154.stdout: mon allow pool delete = true 2023-09-23T03:21:29.216 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.216 INFO:teuthology.orchestra.run.smithi154.stdout: mon cluster log file level = debug 2023-09-23T03:21:29.216 INFO:teuthology.orchestra.run.smithi154.stdout: debug asserts on shutdown = true 2023-09-23T03:21:29.216 INFO:teuthology.orchestra.run.smithi154.stdout: mon health detail to clog = false 2023-09-23T03:21:29.216 INFO:teuthology.orchestra.run.smithi154.stdout: mon host = "172.21.15.154,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]" 2023-09-23T03:21:29.216 INFO:teuthology.orchestra.run.smithi154.stdout: fsid = 8c8039d3-6c84-43db-8808-85e0dbdd47c8 2023-09-23T03:21:29.216 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.216 INFO:teuthology.orchestra.run.smithi154.stdout:[osd] 2023-09-23T03:21:29.216 INFO:teuthology.orchestra.run.smithi154.stdout: osd journal size = 100 2023-09-23T03:21:29.216 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.216 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub load threshold = 5.0 2023-09-23T03:21:29.217 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub max interval = 600 2023-09-23T03:21:29.217 INFO:teuthology.orchestra.run.smithi154.stdout: osd mclock profile = high_recovery_ops 2023-09-23T03:21:29.217 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.217 INFO:teuthology.orchestra.run.smithi154.stdout: osd recover clone overlap = true 2023-09-23T03:21:29.217 INFO:teuthology.orchestra.run.smithi154.stdout: osd recovery max chunk = 1048576 2023-09-23T03:21:29.217 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.217 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug shutdown = true 2023-09-23T03:21:29.217 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug op order = true 2023-09-23T03:21:29.217 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug verify stray on activate = true 2023-09-23T03:21:29.217 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.217 INFO:teuthology.orchestra.run.smithi154.stdout: osd open classes on start = true 2023-09-23T03:21:29.217 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug pg log writeout = true 2023-09-23T03:21:29.218 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.218 INFO:teuthology.orchestra.run.smithi154.stdout: osd deep scrub update digest min age = 30 2023-09-23T03:21:29.218 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.218 INFO:teuthology.orchestra.run.smithi154.stdout: osd map max advance = 10 2023-09-23T03:21:29.218 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.218 INFO:teuthology.orchestra.run.smithi154.stdout: journal zero on create = true 2023-09-23T03:21:29.218 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.218 INFO:teuthology.orchestra.run.smithi154.stdout: filestore ondisk finisher threads = 3 2023-09-23T03:21:29.218 INFO:teuthology.orchestra.run.smithi154.stdout: filestore apply finisher threads = 3 2023-09-23T03:21:29.218 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.218 INFO:teuthology.orchestra.run.smithi154.stdout: bdev debug aio = true 2023-09-23T03:21:29.219 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug misdirected ops = true 2023-09-23T03:21:29.219 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore block size = 96636764160 2023-09-23T03:21:29.219 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore fsck on mount = True 2023-09-23T03:21:29.219 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluefs = 20 2023-09-23T03:21:29.219 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluestore = 20 2023-09-23T03:21:29.219 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:21:29.219 INFO:teuthology.orchestra.run.smithi154.stdout: debug osd = 20 2023-09-23T03:21:29.219 INFO:teuthology.orchestra.run.smithi154.stdout: debug rocksdb = 10 2023-09-23T03:21:29.219 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd backfillfull_ratio = 0.85 2023-09-23T03:21:29.219 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd full ratio = 0.9 2023-09-23T03:21:29.219 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd nearfull ratio = 0.8 2023-09-23T03:21:29.219 INFO:teuthology.orchestra.run.smithi154.stdout: osd failsafe full ratio = 0.95 2023-09-23T03:21:29.220 INFO:teuthology.orchestra.run.smithi154.stdout: osd objectstore = bluestore 2023-09-23T03:21:29.220 INFO:teuthology.orchestra.run.smithi154.stdout: osd op complaint time = 180 2023-09-23T03:21:29.220 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.220 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr] 2023-09-23T03:21:29.220 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:21:29.220 INFO:teuthology.orchestra.run.smithi154.stdout: debug mgr = 20 2023-09-23T03:21:29.220 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2023-09-23T03:21:29.220 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2023-09-23T03:21:29.220 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:21:29.220 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:21:29.220 INFO:teuthology.orchestra.run.smithi154.stdout: mgr/telemetry/nag = false 2023-09-23T03:21:29.220 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.220 INFO:teuthology.orchestra.run.smithi154.stdout:[mon] 2023-09-23T03:21:29.220 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:21:29.221 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2023-09-23T03:21:29.221 INFO:teuthology.orchestra.run.smithi154.stdout: debug paxos = 20 2023-09-23T03:21:29.221 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2023-09-23T03:21:29.221 INFO:teuthology.orchestra.run.smithi154.stdout: mon data avail warn = 5 2023-09-23T03:21:29.221 INFO:teuthology.orchestra.run.smithi154.stdout: mon mgr mkfs grace = 240 2023-09-23T03:21:29.221 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:21:29.221 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd reporter subtree level = osd 2023-09-23T03:21:29.221 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd prime pg temp = true 2023-09-23T03:21:29.221 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:21:29.221 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.221 INFO:teuthology.orchestra.run.smithi154.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-23T03:21:29.222 INFO:teuthology.orchestra.run.smithi154.stdout: auth mon ticket ttl = 660 # 11m 2023-09-23T03:21:29.222 INFO:teuthology.orchestra.run.smithi154.stdout: auth service ticket ttl = 240 # 4m 2023-09-23T03:21:29.222 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.222 INFO:teuthology.orchestra.run.smithi154.stdout: # don't complain about insecure global_id in the test suite 2023-09-23T03:21:29.222 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-23T03:21:29.222 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-23T03:21:29.222 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.222 INFO:teuthology.orchestra.run.smithi154.stdout: # 1m isn't quite enough 2023-09-23T03:21:29.222 INFO:teuthology.orchestra.run.smithi154.stdout: mon_down_mkfs_grace = 2m 2023-09-23T03:21:29.222 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.222 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_filestore_osds = false 2023-09-23T03:21:29.223 INFO:teuthology.orchestra.run.smithi154.stdout: mon op complaint time = 120 2023-09-23T03:21:29.223 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.223 INFO:teuthology.orchestra.run.smithi154.stdout:[client] 2023-09-23T03:21:29.223 INFO:teuthology.orchestra.run.smithi154.stdout: rgw cache enabled = true 2023-09-23T03:21:29.223 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable ops log = true 2023-09-23T03:21:29.223 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable usage log = true 2023-09-23T03:21:29.223 INFO:teuthology.orchestra.run.smithi154.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-23T03:21:29.223 INFO:teuthology.orchestra.run.smithi154.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-23T03:21:29.223 INFO:teuthology.orchestra.run.smithi154.stdout: client mount timeout = 600 2023-09-23T03:21:29.223 INFO:teuthology.orchestra.run.smithi154.stdout: debug client = 20 2023-09-23T03:21:29.223 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:21:29.223 INFO:teuthology.orchestra.run.smithi154.stdout: pid file = "" 2023-09-23T03:21:29.223 INFO:teuthology.orchestra.run.smithi154.stdout: rados mon op timeout = 900 2023-09-23T03:21:29.224 INFO:teuthology.orchestra.run.smithi154.stdout: rados osd op timeout = 900 2023-09-23T03:21:29.224 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.a] 2023-09-23T03:21:29.224 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.b] 2023-09-23T03:21:29.224 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.c] 2023-09-23T03:21:29.224 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.a] 2023-09-23T03:21:29.224 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.c] 2023-09-23T03:21:29.224 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.b] 2023-09-23T03:21:29.224 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.d] 2023-09-23T03:21:29.224 INFO:teuthology.orchestra.run.smithi154.stdout:[mds] 2023-09-23T03:21:29.224 INFO:teuthology.orchestra.run.smithi154.stdout: debug mds = 20 2023-09-23T03:21:29.224 INFO:teuthology.orchestra.run.smithi154.stdout: debug mds balancer = 20 2023-09-23T03:21:29.224 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:21:29.225 INFO:teuthology.orchestra.run.smithi154.stdout: mds debug frag = True 2023-09-23T03:21:29.225 INFO:teuthology.orchestra.run.smithi154.stdout: mds debug scatterstat = False 2023-09-23T03:21:29.225 INFO:teuthology.orchestra.run.smithi154.stdout: mds op complaint time = 180 2023-09-23T03:21:29.225 INFO:teuthology.orchestra.run.smithi154.stdout: mds verify scatter = False 2023-09-23T03:21:29.225 INFO:teuthology.orchestra.run.smithi154.stdout: osd op complaint time = 180 2023-09-23T03:21:29.225 INFO:teuthology.orchestra.run.smithi154.stdout: rados mon op timeout = 900 2023-09-23T03:21:29.225 INFO:teuthology.orchestra.run.smithi154.stdout: rados osd op timeout = 900 2023-09-23T03:21:29.238 INFO:teuthology.orchestra.run.smithi190.stdout:[global] 2023-09-23T03:21:29.238 INFO:teuthology.orchestra.run.smithi190.stdout: chdir = "" 2023-09-23T03:21:29.238 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-23T03:21:29.238 INFO:teuthology.orchestra.run.smithi190.stdout: auth supported = cephx 2023-09-23T03:21:29.238 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.239 INFO:teuthology.orchestra.run.smithi190.stdout: filestore xattr use omap = true 2023-09-23T03:21:29.239 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.239 INFO:teuthology.orchestra.run.smithi190.stdout: mon clock drift allowed = 1.000 2023-09-23T03:21:29.239 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.239 INFO:teuthology.orchestra.run.smithi190.stdout: osd crush chooseleaf type = 0 2023-09-23T03:21:29.239 INFO:teuthology.orchestra.run.smithi190.stdout: auth debug = true 2023-09-23T03:21:29.239 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.239 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on old message = true 2023-09-23T03:21:29.239 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on bug = true 2023-09-23T03:21:29.239 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.239 INFO:teuthology.orchestra.run.smithi190.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-23T03:21:29.239 INFO:teuthology.orchestra.run.smithi190.stdout: mon pg warn max object skew = 0 2023-09-23T03:21:29.240 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.240 INFO:teuthology.orchestra.run.smithi190.stdout: # disable pg_autoscaler by default for new pools 2023-09-23T03:21:29.240 INFO:teuthology.orchestra.run.smithi190.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-23T03:21:29.240 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.240 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default size = 2 2023-09-23T03:21:29.240 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.240 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow primary affinity = true 2023-09-23T03:21:29.240 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow pg remap = true 2023-09-23T03:21:29.240 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on legacy crush tunables = false 2023-09-23T03:21:29.240 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on crush straw calc version zero = false 2023-09-23T03:21:29.240 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on no sortbitwise = false 2023-09-23T03:21:29.240 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on osd down out interval zero = false 2023-09-23T03:21:29.241 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on too few osds = false 2023-09-23T03:21:29.241 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-23T03:21:29.241 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-23T03:21:29.241 INFO:teuthology.orchestra.run.smithi190.stdout: mon_allow_pool_size_one = true 2023-09-23T03:21:29.241 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.241 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-23T03:21:29.241 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.241 INFO:teuthology.orchestra.run.smithi190.stdout: osd default data pool replay window = 5 2023-09-23T03:21:29.241 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.241 INFO:teuthology.orchestra.run.smithi190.stdout: mon allow pool delete = true 2023-09-23T03:21:29.241 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.242 INFO:teuthology.orchestra.run.smithi190.stdout: mon cluster log file level = debug 2023-09-23T03:21:29.242 INFO:teuthology.orchestra.run.smithi190.stdout: debug asserts on shutdown = true 2023-09-23T03:21:29.242 INFO:teuthology.orchestra.run.smithi190.stdout: mon health detail to clog = false 2023-09-23T03:21:29.242 INFO:teuthology.orchestra.run.smithi190.stdout: mon host = "172.21.15.154,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]" 2023-09-23T03:21:29.242 INFO:teuthology.orchestra.run.smithi190.stdout: fsid = 8c8039d3-6c84-43db-8808-85e0dbdd47c8 2023-09-23T03:21:29.242 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.242 INFO:teuthology.orchestra.run.smithi190.stdout:[osd] 2023-09-23T03:21:29.242 INFO:teuthology.orchestra.run.smithi190.stdout: osd journal size = 100 2023-09-23T03:21:29.242 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.242 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub load threshold = 5.0 2023-09-23T03:21:29.242 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub max interval = 600 2023-09-23T03:21:29.242 INFO:teuthology.orchestra.run.smithi190.stdout: osd mclock profile = high_recovery_ops 2023-09-23T03:21:29.243 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.243 INFO:teuthology.orchestra.run.smithi190.stdout: osd recover clone overlap = true 2023-09-23T03:21:29.243 INFO:teuthology.orchestra.run.smithi190.stdout: osd recovery max chunk = 1048576 2023-09-23T03:21:29.243 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.243 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug shutdown = true 2023-09-23T03:21:29.243 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug op order = true 2023-09-23T03:21:29.243 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug verify stray on activate = true 2023-09-23T03:21:29.243 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.243 INFO:teuthology.orchestra.run.smithi190.stdout: osd open classes on start = true 2023-09-23T03:21:29.243 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug pg log writeout = true 2023-09-23T03:21:29.243 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.244 INFO:teuthology.orchestra.run.smithi190.stdout: osd deep scrub update digest min age = 30 2023-09-23T03:21:29.244 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.244 INFO:teuthology.orchestra.run.smithi190.stdout: osd map max advance = 10 2023-09-23T03:21:29.244 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.244 INFO:teuthology.orchestra.run.smithi190.stdout: journal zero on create = true 2023-09-23T03:21:29.244 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.244 INFO:teuthology.orchestra.run.smithi190.stdout: filestore ondisk finisher threads = 3 2023-09-23T03:21:29.244 INFO:teuthology.orchestra.run.smithi190.stdout: filestore apply finisher threads = 3 2023-09-23T03:21:29.244 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.244 INFO:teuthology.orchestra.run.smithi190.stdout: bdev debug aio = true 2023-09-23T03:21:29.244 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug misdirected ops = true 2023-09-23T03:21:29.244 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore block size = 96636764160 2023-09-23T03:21:29.245 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore fsck on mount = True 2023-09-23T03:21:29.245 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluefs = 20 2023-09-23T03:21:29.245 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluestore = 20 2023-09-23T03:21:29.245 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:21:29.245 INFO:teuthology.orchestra.run.smithi190.stdout: debug osd = 20 2023-09-23T03:21:29.245 INFO:teuthology.orchestra.run.smithi190.stdout: debug rocksdb = 10 2023-09-23T03:21:29.245 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd backfillfull_ratio = 0.85 2023-09-23T03:21:29.245 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd full ratio = 0.9 2023-09-23T03:21:29.245 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd nearfull ratio = 0.8 2023-09-23T03:21:29.245 INFO:teuthology.orchestra.run.smithi190.stdout: osd failsafe full ratio = 0.95 2023-09-23T03:21:29.245 INFO:teuthology.orchestra.run.smithi190.stdout: osd objectstore = bluestore 2023-09-23T03:21:29.245 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2023-09-23T03:21:29.245 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.246 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr] 2023-09-23T03:21:29.246 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:21:29.246 INFO:teuthology.orchestra.run.smithi190.stdout: debug mgr = 20 2023-09-23T03:21:29.246 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2023-09-23T03:21:29.246 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2023-09-23T03:21:29.246 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:21:29.246 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:21:29.246 INFO:teuthology.orchestra.run.smithi190.stdout: mgr/telemetry/nag = false 2023-09-23T03:21:29.246 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.246 INFO:teuthology.orchestra.run.smithi190.stdout:[mon] 2023-09-23T03:21:29.246 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:21:29.246 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2023-09-23T03:21:29.246 INFO:teuthology.orchestra.run.smithi190.stdout: debug paxos = 20 2023-09-23T03:21:29.247 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2023-09-23T03:21:29.247 INFO:teuthology.orchestra.run.smithi190.stdout: mon data avail warn = 5 2023-09-23T03:21:29.247 INFO:teuthology.orchestra.run.smithi190.stdout: mon mgr mkfs grace = 240 2023-09-23T03:21:29.247 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:21:29.247 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd reporter subtree level = osd 2023-09-23T03:21:29.247 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd prime pg temp = true 2023-09-23T03:21:29.247 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:21:29.247 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.247 INFO:teuthology.orchestra.run.smithi190.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-23T03:21:29.247 INFO:teuthology.orchestra.run.smithi190.stdout: auth mon ticket ttl = 660 # 11m 2023-09-23T03:21:29.247 INFO:teuthology.orchestra.run.smithi190.stdout: auth service ticket ttl = 240 # 4m 2023-09-23T03:21:29.247 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.248 INFO:teuthology.orchestra.run.smithi190.stdout: # don't complain about insecure global_id in the test suite 2023-09-23T03:21:29.248 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-23T03:21:29.248 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-23T03:21:29.248 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.248 INFO:teuthology.orchestra.run.smithi190.stdout: # 1m isn't quite enough 2023-09-23T03:21:29.248 INFO:teuthology.orchestra.run.smithi190.stdout: mon_down_mkfs_grace = 2m 2023-09-23T03:21:29.248 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.248 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_filestore_osds = false 2023-09-23T03:21:29.248 INFO:teuthology.orchestra.run.smithi190.stdout: mon op complaint time = 120 2023-09-23T03:21:29.248 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.248 INFO:teuthology.orchestra.run.smithi190.stdout:[client] 2023-09-23T03:21:29.248 INFO:teuthology.orchestra.run.smithi190.stdout: rgw cache enabled = true 2023-09-23T03:21:29.249 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable ops log = true 2023-09-23T03:21:29.249 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable usage log = true 2023-09-23T03:21:29.249 INFO:teuthology.orchestra.run.smithi190.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-23T03:21:29.249 INFO:teuthology.orchestra.run.smithi190.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-23T03:21:29.249 INFO:teuthology.orchestra.run.smithi190.stdout: client mount timeout = 600 2023-09-23T03:21:29.249 INFO:teuthology.orchestra.run.smithi190.stdout: debug client = 20 2023-09-23T03:21:29.249 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:21:29.249 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = "" 2023-09-23T03:21:29.249 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 900 2023-09-23T03:21:29.249 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 900 2023-09-23T03:21:29.249 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.a] 2023-09-23T03:21:29.249 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.b] 2023-09-23T03:21:29.249 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.c] 2023-09-23T03:21:29.250 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.a] 2023-09-23T03:21:29.250 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.c] 2023-09-23T03:21:29.250 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.b] 2023-09-23T03:21:29.250 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.d] 2023-09-23T03:21:29.250 INFO:teuthology.orchestra.run.smithi190.stdout:[mds] 2023-09-23T03:21:29.250 INFO:teuthology.orchestra.run.smithi190.stdout: debug mds = 20 2023-09-23T03:21:29.250 INFO:teuthology.orchestra.run.smithi190.stdout: debug mds balancer = 20 2023-09-23T03:21:29.250 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:21:29.250 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug frag = True 2023-09-23T03:21:29.250 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug scatterstat = False 2023-09-23T03:21:29.250 INFO:teuthology.orchestra.run.smithi190.stdout: mds op complaint time = 180 2023-09-23T03:21:29.250 INFO:teuthology.orchestra.run.smithi190.stdout: mds verify scatter = False 2023-09-23T03:21:29.251 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2023-09-23T03:21:29.251 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 900 2023-09-23T03:21:29.251 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 900 2023-09-23T03:21:29.283 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-23T03:21:29.292 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-23T03:21:29.462 INFO:teuthology.orchestra.run.smithi154.stdout:[global] 2023-09-23T03:21:29.462 INFO:teuthology.orchestra.run.smithi154.stdout: chdir = "" 2023-09-23T03:21:29.462 INFO:teuthology.orchestra.run.smithi154.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-23T03:21:29.462 INFO:teuthology.orchestra.run.smithi154.stdout: auth supported = cephx 2023-09-23T03:21:29.462 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.462 INFO:teuthology.orchestra.run.smithi154.stdout: filestore xattr use omap = true 2023-09-23T03:21:29.462 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.462 INFO:teuthology.orchestra.run.smithi154.stdout: mon clock drift allowed = 1.000 2023-09-23T03:21:29.463 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.463 INFO:teuthology.orchestra.run.smithi154.stdout: osd crush chooseleaf type = 0 2023-09-23T03:21:29.463 INFO:teuthology.orchestra.run.smithi154.stdout: auth debug = true 2023-09-23T03:21:29.463 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.463 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on old message = true 2023-09-23T03:21:29.463 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on bug = true 2023-09-23T03:21:29.463 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.463 INFO:teuthology.orchestra.run.smithi154.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-23T03:21:29.463 INFO:teuthology.orchestra.run.smithi154.stdout: mon pg warn max object skew = 0 2023-09-23T03:21:29.463 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.463 INFO:teuthology.orchestra.run.smithi154.stdout: # disable pg_autoscaler by default for new pools 2023-09-23T03:21:29.463 INFO:teuthology.orchestra.run.smithi154.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-23T03:21:29.464 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.464 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default size = 2 2023-09-23T03:21:29.464 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.464 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow primary affinity = true 2023-09-23T03:21:29.464 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow pg remap = true 2023-09-23T03:21:29.464 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on legacy crush tunables = false 2023-09-23T03:21:29.464 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on crush straw calc version zero = false 2023-09-23T03:21:29.464 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on no sortbitwise = false 2023-09-23T03:21:29.464 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on osd down out interval zero = false 2023-09-23T03:21:29.464 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on too few osds = false 2023-09-23T03:21:29.464 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-23T03:21:29.465 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-23T03:21:29.465 INFO:teuthology.orchestra.run.smithi154.stdout: mon_allow_pool_size_one = true 2023-09-23T03:21:29.465 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.465 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-23T03:21:29.465 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.465 INFO:teuthology.orchestra.run.smithi154.stdout: osd default data pool replay window = 5 2023-09-23T03:21:29.465 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.465 INFO:teuthology.orchestra.run.smithi154.stdout: mon allow pool delete = true 2023-09-23T03:21:29.465 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.465 INFO:teuthology.orchestra.run.smithi154.stdout: mon cluster log file level = debug 2023-09-23T03:21:29.465 INFO:teuthology.orchestra.run.smithi154.stdout: debug asserts on shutdown = true 2023-09-23T03:21:29.465 INFO:teuthology.orchestra.run.smithi154.stdout: mon health detail to clog = false 2023-09-23T03:21:29.465 INFO:teuthology.orchestra.run.smithi154.stdout: mon host = "172.21.15.154,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]" 2023-09-23T03:21:29.466 INFO:teuthology.orchestra.run.smithi154.stdout: fsid = 8c8039d3-6c84-43db-8808-85e0dbdd47c8 2023-09-23T03:21:29.466 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.466 INFO:teuthology.orchestra.run.smithi154.stdout:[osd] 2023-09-23T03:21:29.466 INFO:teuthology.orchestra.run.smithi154.stdout: osd journal size = 100 2023-09-23T03:21:29.466 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.466 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub load threshold = 5.0 2023-09-23T03:21:29.466 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub max interval = 600 2023-09-23T03:21:29.466 INFO:teuthology.orchestra.run.smithi154.stdout: osd mclock profile = high_recovery_ops 2023-09-23T03:21:29.466 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.466 INFO:teuthology.orchestra.run.smithi154.stdout: osd recover clone overlap = true 2023-09-23T03:21:29.466 INFO:teuthology.orchestra.run.smithi154.stdout: osd recovery max chunk = 1048576 2023-09-23T03:21:29.466 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.467 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug shutdown = true 2023-09-23T03:21:29.467 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug op order = true 2023-09-23T03:21:29.467 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug verify stray on activate = true 2023-09-23T03:21:29.467 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.467 INFO:teuthology.orchestra.run.smithi154.stdout: osd open classes on start = true 2023-09-23T03:21:29.467 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug pg log writeout = true 2023-09-23T03:21:29.467 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.467 INFO:teuthology.orchestra.run.smithi154.stdout: osd deep scrub update digest min age = 30 2023-09-23T03:21:29.467 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.467 INFO:teuthology.orchestra.run.smithi154.stdout: osd map max advance = 10 2023-09-23T03:21:29.467 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.467 INFO:teuthology.orchestra.run.smithi154.stdout: journal zero on create = true 2023-09-23T03:21:29.467 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.467 INFO:teuthology.orchestra.run.smithi154.stdout: filestore ondisk finisher threads = 3 2023-09-23T03:21:29.468 INFO:teuthology.orchestra.run.smithi154.stdout: filestore apply finisher threads = 3 2023-09-23T03:21:29.468 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.468 INFO:teuthology.orchestra.run.smithi154.stdout: bdev debug aio = true 2023-09-23T03:21:29.468 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug misdirected ops = true 2023-09-23T03:21:29.468 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore block size = 96636764160 2023-09-23T03:21:29.468 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore fsck on mount = True 2023-09-23T03:21:29.468 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluefs = 20 2023-09-23T03:21:29.468 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluestore = 20 2023-09-23T03:21:29.468 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:21:29.468 INFO:teuthology.orchestra.run.smithi154.stdout: debug osd = 20 2023-09-23T03:21:29.468 INFO:teuthology.orchestra.run.smithi154.stdout: debug rocksdb = 10 2023-09-23T03:21:29.468 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd backfillfull_ratio = 0.85 2023-09-23T03:21:29.468 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd full ratio = 0.9 2023-09-23T03:21:29.469 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd nearfull ratio = 0.8 2023-09-23T03:21:29.469 INFO:teuthology.orchestra.run.smithi154.stdout: osd failsafe full ratio = 0.95 2023-09-23T03:21:29.469 INFO:teuthology.orchestra.run.smithi154.stdout: osd objectstore = bluestore 2023-09-23T03:21:29.469 INFO:teuthology.orchestra.run.smithi154.stdout: osd op complaint time = 180 2023-09-23T03:21:29.469 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.469 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr] 2023-09-23T03:21:29.469 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:21:29.469 INFO:teuthology.orchestra.run.smithi154.stdout: debug mgr = 20 2023-09-23T03:21:29.469 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2023-09-23T03:21:29.469 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2023-09-23T03:21:29.469 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:21:29.469 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:21:29.469 INFO:teuthology.orchestra.run.smithi154.stdout: mgr/telemetry/nag = false 2023-09-23T03:21:29.469 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.470 INFO:teuthology.orchestra.run.smithi154.stdout:[mon] 2023-09-23T03:21:29.470 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:21:29.470 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2023-09-23T03:21:29.471 INFO:teuthology.orchestra.run.smithi154.stdout: debug paxos = 20 2023-09-23T03:21:29.471 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2023-09-23T03:21:29.471 INFO:teuthology.orchestra.run.smithi154.stdout: mon data avail warn = 5 2023-09-23T03:21:29.471 INFO:teuthology.orchestra.run.smithi154.stdout: mon mgr mkfs grace = 240 2023-09-23T03:21:29.471 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:21:29.471 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd reporter subtree level = osd 2023-09-23T03:21:29.471 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd prime pg temp = true 2023-09-23T03:21:29.471 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:21:29.471 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.472 INFO:teuthology.orchestra.run.smithi154.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-23T03:21:29.472 INFO:teuthology.orchestra.run.smithi154.stdout: auth mon ticket ttl = 660 # 11m 2023-09-23T03:21:29.472 INFO:teuthology.orchestra.run.smithi154.stdout: auth service ticket ttl = 240 # 4m 2023-09-23T03:21:29.472 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.472 INFO:teuthology.orchestra.run.smithi154.stdout: # don't complain about insecure global_id in the test suite 2023-09-23T03:21:29.472 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-23T03:21:29.472 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-23T03:21:29.472 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.472 INFO:teuthology.orchestra.run.smithi154.stdout: # 1m isn't quite enough 2023-09-23T03:21:29.472 INFO:teuthology.orchestra.run.smithi154.stdout: mon_down_mkfs_grace = 2m 2023-09-23T03:21:29.472 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.473 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_filestore_osds = false 2023-09-23T03:21:29.473 INFO:teuthology.orchestra.run.smithi154.stdout: mon op complaint time = 120 2023-09-23T03:21:29.473 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:29.473 INFO:teuthology.orchestra.run.smithi154.stdout:[client] 2023-09-23T03:21:29.473 INFO:teuthology.orchestra.run.smithi154.stdout: rgw cache enabled = true 2023-09-23T03:21:29.473 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable ops log = true 2023-09-23T03:21:29.473 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable usage log = true 2023-09-23T03:21:29.473 INFO:teuthology.orchestra.run.smithi154.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-23T03:21:29.473 INFO:teuthology.orchestra.run.smithi154.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-23T03:21:29.473 INFO:teuthology.orchestra.run.smithi154.stdout: client mount timeout = 600 2023-09-23T03:21:29.473 INFO:teuthology.orchestra.run.smithi154.stdout: debug client = 20 2023-09-23T03:21:29.473 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:21:29.474 INFO:teuthology.orchestra.run.smithi154.stdout: pid file = "" 2023-09-23T03:21:29.474 INFO:teuthology.orchestra.run.smithi154.stdout: rados mon op timeout = 900 2023-09-23T03:21:29.474 INFO:teuthology.orchestra.run.smithi154.stdout: rados osd op timeout = 900 2023-09-23T03:21:29.474 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.a] 2023-09-23T03:21:29.474 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.b] 2023-09-23T03:21:29.474 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.c] 2023-09-23T03:21:29.474 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.a] 2023-09-23T03:21:29.474 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.c] 2023-09-23T03:21:29.474 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.b] 2023-09-23T03:21:29.474 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.d] 2023-09-23T03:21:29.474 INFO:teuthology.orchestra.run.smithi154.stdout:[mds] 2023-09-23T03:21:29.474 INFO:teuthology.orchestra.run.smithi154.stdout: debug mds = 20 2023-09-23T03:21:29.475 INFO:teuthology.orchestra.run.smithi154.stdout: debug mds balancer = 20 2023-09-23T03:21:29.475 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:21:29.475 INFO:teuthology.orchestra.run.smithi154.stdout: mds debug frag = True 2023-09-23T03:21:29.475 INFO:teuthology.orchestra.run.smithi154.stdout: mds debug scatterstat = False 2023-09-23T03:21:29.475 INFO:teuthology.orchestra.run.smithi154.stdout: mds op complaint time = 180 2023-09-23T03:21:29.475 INFO:teuthology.orchestra.run.smithi154.stdout: mds verify scatter = False 2023-09-23T03:21:29.475 INFO:teuthology.orchestra.run.smithi154.stdout: osd op complaint time = 180 2023-09-23T03:21:29.475 INFO:teuthology.orchestra.run.smithi154.stdout: rados mon op timeout = 900 2023-09-23T03:21:29.475 INFO:teuthology.orchestra.run.smithi154.stdout: rados osd op timeout = 900 2023-09-23T03:21:29.476 INFO:teuthology.orchestra.run.smithi190.stdout:[global] 2023-09-23T03:21:29.476 INFO:teuthology.orchestra.run.smithi190.stdout: chdir = "" 2023-09-23T03:21:29.476 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-23T03:21:29.476 INFO:teuthology.orchestra.run.smithi190.stdout: auth supported = cephx 2023-09-23T03:21:29.476 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.476 INFO:teuthology.orchestra.run.smithi190.stdout: filestore xattr use omap = true 2023-09-23T03:21:29.476 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.476 INFO:teuthology.orchestra.run.smithi190.stdout: mon clock drift allowed = 1.000 2023-09-23T03:21:29.477 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.477 INFO:teuthology.orchestra.run.smithi190.stdout: osd crush chooseleaf type = 0 2023-09-23T03:21:29.477 INFO:teuthology.orchestra.run.smithi190.stdout: auth debug = true 2023-09-23T03:21:29.477 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.477 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on old message = true 2023-09-23T03:21:29.477 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on bug = true 2023-09-23T03:21:29.477 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.477 INFO:teuthology.orchestra.run.smithi190.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-23T03:21:29.477 INFO:teuthology.orchestra.run.smithi190.stdout: mon pg warn max object skew = 0 2023-09-23T03:21:29.477 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.477 INFO:teuthology.orchestra.run.smithi190.stdout: # disable pg_autoscaler by default for new pools 2023-09-23T03:21:29.477 INFO:teuthology.orchestra.run.smithi190.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-23T03:21:29.477 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.478 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default size = 2 2023-09-23T03:21:29.478 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.478 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow primary affinity = true 2023-09-23T03:21:29.478 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow pg remap = true 2023-09-23T03:21:29.478 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on legacy crush tunables = false 2023-09-23T03:21:29.478 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on crush straw calc version zero = false 2023-09-23T03:21:29.478 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on no sortbitwise = false 2023-09-23T03:21:29.478 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on osd down out interval zero = false 2023-09-23T03:21:29.478 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on too few osds = false 2023-09-23T03:21:29.478 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-23T03:21:29.478 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-23T03:21:29.478 INFO:teuthology.orchestra.run.smithi190.stdout: mon_allow_pool_size_one = true 2023-09-23T03:21:29.479 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.479 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-23T03:21:29.479 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.479 INFO:teuthology.orchestra.run.smithi190.stdout: osd default data pool replay window = 5 2023-09-23T03:21:29.479 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.479 INFO:teuthology.orchestra.run.smithi190.stdout: mon allow pool delete = true 2023-09-23T03:21:29.479 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.479 INFO:teuthology.orchestra.run.smithi190.stdout: mon cluster log file level = debug 2023-09-23T03:21:29.479 INFO:teuthology.orchestra.run.smithi190.stdout: debug asserts on shutdown = true 2023-09-23T03:21:29.479 INFO:teuthology.orchestra.run.smithi190.stdout: mon health detail to clog = false 2023-09-23T03:21:29.479 INFO:teuthology.orchestra.run.smithi190.stdout: mon host = "172.21.15.154,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]" 2023-09-23T03:21:29.479 INFO:teuthology.orchestra.run.smithi190.stdout: fsid = 8c8039d3-6c84-43db-8808-85e0dbdd47c8 2023-09-23T03:21:29.480 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.480 INFO:teuthology.orchestra.run.smithi190.stdout:[osd] 2023-09-23T03:21:29.480 INFO:teuthology.orchestra.run.smithi190.stdout: osd journal size = 100 2023-09-23T03:21:29.480 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.480 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub load threshold = 5.0 2023-09-23T03:21:29.480 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub max interval = 600 2023-09-23T03:21:29.480 INFO:teuthology.orchestra.run.smithi190.stdout: osd mclock profile = high_recovery_ops 2023-09-23T03:21:29.480 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.480 INFO:teuthology.orchestra.run.smithi190.stdout: osd recover clone overlap = true 2023-09-23T03:21:29.480 INFO:teuthology.orchestra.run.smithi190.stdout: osd recovery max chunk = 1048576 2023-09-23T03:21:29.480 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.480 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug shutdown = true 2023-09-23T03:21:29.481 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug op order = true 2023-09-23T03:21:29.481 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug verify stray on activate = true 2023-09-23T03:21:29.481 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.481 INFO:teuthology.orchestra.run.smithi190.stdout: osd open classes on start = true 2023-09-23T03:21:29.481 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug pg log writeout = true 2023-09-23T03:21:29.481 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.481 INFO:teuthology.orchestra.run.smithi190.stdout: osd deep scrub update digest min age = 30 2023-09-23T03:21:29.481 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.481 INFO:teuthology.orchestra.run.smithi190.stdout: osd map max advance = 10 2023-09-23T03:21:29.481 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.481 INFO:teuthology.orchestra.run.smithi190.stdout: journal zero on create = true 2023-09-23T03:21:29.481 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.481 INFO:teuthology.orchestra.run.smithi190.stdout: filestore ondisk finisher threads = 3 2023-09-23T03:21:29.482 INFO:teuthology.orchestra.run.smithi190.stdout: filestore apply finisher threads = 3 2023-09-23T03:21:29.482 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.482 INFO:teuthology.orchestra.run.smithi190.stdout: bdev debug aio = true 2023-09-23T03:21:29.482 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug misdirected ops = true 2023-09-23T03:21:29.482 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore block size = 96636764160 2023-09-23T03:21:29.482 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore fsck on mount = True 2023-09-23T03:21:29.482 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluefs = 20 2023-09-23T03:21:29.482 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluestore = 20 2023-09-23T03:21:29.482 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:21:29.482 INFO:teuthology.orchestra.run.smithi190.stdout: debug osd = 20 2023-09-23T03:21:29.482 INFO:teuthology.orchestra.run.smithi190.stdout: debug rocksdb = 10 2023-09-23T03:21:29.482 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd backfillfull_ratio = 0.85 2023-09-23T03:21:29.483 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd full ratio = 0.9 2023-09-23T03:21:29.483 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd nearfull ratio = 0.8 2023-09-23T03:21:29.483 INFO:teuthology.orchestra.run.smithi190.stdout: osd failsafe full ratio = 0.95 2023-09-23T03:21:29.483 INFO:teuthology.orchestra.run.smithi190.stdout: osd objectstore = bluestore 2023-09-23T03:21:29.483 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2023-09-23T03:21:29.483 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.483 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr] 2023-09-23T03:21:29.483 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:21:29.483 INFO:teuthology.orchestra.run.smithi190.stdout: debug mgr = 20 2023-09-23T03:21:29.483 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2023-09-23T03:21:29.483 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2023-09-23T03:21:29.485 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:21:29.485 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:21:29.485 INFO:teuthology.orchestra.run.smithi190.stdout: mgr/telemetry/nag = false 2023-09-23T03:21:29.486 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.486 INFO:teuthology.orchestra.run.smithi190.stdout:[mon] 2023-09-23T03:21:29.486 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:21:29.486 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2023-09-23T03:21:29.486 INFO:teuthology.orchestra.run.smithi190.stdout: debug paxos = 20 2023-09-23T03:21:29.486 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2023-09-23T03:21:29.486 INFO:teuthology.orchestra.run.smithi190.stdout: mon data avail warn = 5 2023-09-23T03:21:29.486 INFO:teuthology.orchestra.run.smithi190.stdout: mon mgr mkfs grace = 240 2023-09-23T03:21:29.486 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:21:29.486 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd reporter subtree level = osd 2023-09-23T03:21:29.486 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd prime pg temp = true 2023-09-23T03:21:29.487 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:21:29.487 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.487 INFO:teuthology.orchestra.run.smithi190.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-23T03:21:29.487 INFO:teuthology.orchestra.run.smithi190.stdout: auth mon ticket ttl = 660 # 11m 2023-09-23T03:21:29.487 INFO:teuthology.orchestra.run.smithi190.stdout: auth service ticket ttl = 240 # 4m 2023-09-23T03:21:29.487 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.487 INFO:teuthology.orchestra.run.smithi190.stdout: # don't complain about insecure global_id in the test suite 2023-09-23T03:21:29.487 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-23T03:21:29.487 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-23T03:21:29.487 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.487 INFO:teuthology.orchestra.run.smithi190.stdout: # 1m isn't quite enough 2023-09-23T03:21:29.487 INFO:teuthology.orchestra.run.smithi190.stdout: mon_down_mkfs_grace = 2m 2023-09-23T03:21:29.487 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.488 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_filestore_osds = false 2023-09-23T03:21:29.488 INFO:teuthology.orchestra.run.smithi190.stdout: mon op complaint time = 120 2023-09-23T03:21:29.488 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:21:29.488 INFO:teuthology.orchestra.run.smithi190.stdout:[client] 2023-09-23T03:21:29.488 INFO:teuthology.orchestra.run.smithi190.stdout: rgw cache enabled = true 2023-09-23T03:21:29.489 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable ops log = true 2023-09-23T03:21:29.489 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable usage log = true 2023-09-23T03:21:29.489 INFO:teuthology.orchestra.run.smithi190.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-23T03:21:29.489 INFO:teuthology.orchestra.run.smithi190.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-23T03:21:29.489 INFO:teuthology.orchestra.run.smithi190.stdout: client mount timeout = 600 2023-09-23T03:21:29.489 INFO:teuthology.orchestra.run.smithi190.stdout: debug client = 20 2023-09-23T03:21:29.489 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:21:29.489 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = "" 2023-09-23T03:21:29.489 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 900 2023-09-23T03:21:29.489 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 900 2023-09-23T03:21:29.489 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.a] 2023-09-23T03:21:29.489 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.b] 2023-09-23T03:21:29.490 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.c] 2023-09-23T03:21:29.490 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.a] 2023-09-23T03:21:29.490 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.c] 2023-09-23T03:21:29.490 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.b] 2023-09-23T03:21:29.490 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.d] 2023-09-23T03:21:29.490 INFO:teuthology.orchestra.run.smithi190.stdout:[mds] 2023-09-23T03:21:29.490 INFO:teuthology.orchestra.run.smithi190.stdout: debug mds = 20 2023-09-23T03:21:29.490 INFO:teuthology.orchestra.run.smithi190.stdout: debug mds balancer = 20 2023-09-23T03:21:29.490 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:21:29.490 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug frag = True 2023-09-23T03:21:29.490 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug scatterstat = False 2023-09-23T03:21:29.490 INFO:teuthology.orchestra.run.smithi190.stdout: mds op complaint time = 180 2023-09-23T03:21:29.491 INFO:teuthology.orchestra.run.smithi190.stdout: mds verify scatter = False 2023-09-23T03:21:29.491 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2023-09-23T03:21:29.491 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 900 2023-09-23T03:21:29.491 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 900 2023-09-23T03:21:29.518 DEBUG:teuthology.orchestra.run.smithi154:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph purge cephfs_metadata --yes-i-really-really-mean-it 2023-09-23T03:21:29.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.623+0000 7f3adda46f40 1 Processor -- start 2023-09-23T03:21:29.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.623+0000 7f3adda46f40 1 -- start start 2023-09-23T03:21:29.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.624+0000 7f3adda46f40 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c7ed593350 0x55c7ed625510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:29.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.624+0000 7f3adda46f40 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c7ed594a10 0x55c7ed622200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:29.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.624+0000 7f3adda46f40 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c7ed627880 0x55c7ed5952e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:29.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.624+0000 7f3adda46f40 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7ed49a800 con 0x55c7ed593350 2023-09-23T03:21:29.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.624+0000 7f3adda46f40 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7ed49a4a0 con 0x55c7ed594a10 2023-09-23T03:21:29.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.624+0000 7f3adda46f40 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7ed48e4f0 con 0x55c7ed627880 2023-09-23T03:21:29.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.624+0000 7f3adacae640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c7ed593350 0x55c7ed625510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:29.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.624+0000 7f3adacae640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c7ed593350 0x55c7ed625510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49560/0 (socket says 172.21.15.154:49560) 2023-09-23T03:21:29.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.624+0000 7f3adacae640 1 -- 172.21.15.154:0/3107474996 learned_addr learned my addr 172.21.15.154:0/3107474996 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:29.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.625+0000 7f3adacae640 1 -- 172.21.15.154:0/3107474996 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c7ed627880 msgr2=0x55c7ed5952e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:29.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.625+0000 7f3adacae640 1 --2- 172.21.15.154:0/3107474996 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c7ed627880 0x55c7ed5952e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:29.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.625+0000 7f3ada4ad640 1 --2- 172.21.15.154:0/3107474996 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c7ed594a10 0x55c7ed622200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:29.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.625+0000 7f3adacae640 1 -- 172.21.15.154:0/3107474996 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c7ed594a10 msgr2=0x55c7ed622200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:29.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.625+0000 7f3adacae640 1 --2- 172.21.15.154:0/3107474996 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c7ed594a10 0x55c7ed622200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:29.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.625+0000 7f3adacae640 1 -- 172.21.15.154:0/3107474996 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad0005de0 con 0x55c7ed593350 2023-09-23T03:21:29.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.625+0000 7f3ada4ad640 1 --2- 172.21.15.154:0/3107474996 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c7ed594a10 0x55c7ed622200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:29.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.625+0000 7f3adacae640 1 --2- 172.21.15.154:0/3107474996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c7ed593350 0x55c7ed625510 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00a420 tx=0x7f3acc00a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a092c2e9844ea13 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:29.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.625+0000 7f3ad94ab640 1 -- 172.21.15.154:0/3107474996 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3acc00c8f0 con 0x55c7ed593350 2023-09-23T03:21:29.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.625+0000 7f3ad94ab640 1 -- 172.21.15.154:0/3107474996 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3acc00ca50 con 0x55c7ed593350 2023-09-23T03:21:29.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.625+0000 7f3ad94ab640 1 -- 172.21.15.154:0/3107474996 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3acc00ce00 con 0x55c7ed593350 2023-09-23T03:21:29.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.626+0000 7f3adda46f40 1 -- 172.21.15.154:0/3107474996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c7ed593350 msgr2=0x55c7ed625510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:29.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.626+0000 7f3adda46f40 1 --2- 172.21.15.154:0/3107474996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c7ed593350 0x55c7ed625510 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00a420 tx=0x7f3acc00a8e0 comp rx=0 tx=0).stop 2023-09-23T03:21:29.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.626+0000 7f3adda46f40 1 -- 172.21.15.154:0/3107474996 shutdown_connections 2023-09-23T03:21:29.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.626+0000 7f3adda46f40 1 --2- 172.21.15.154:0/3107474996 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c7ed627880 0x55c7ed5952e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:29.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.626+0000 7f3adda46f40 1 --2- 172.21.15.154:0/3107474996 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c7ed594a10 0x55c7ed622200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:29.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.626+0000 7f3adda46f40 1 --2- 172.21.15.154:0/3107474996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c7ed593350 0x55c7ed625510 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:29.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.626+0000 7f3adda46f40 1 -- 172.21.15.154:0/3107474996 >> 172.21.15.154:0/3107474996 conn(0x55c7ed54df20 msgr2=0x55c7ed54e300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:29.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.626+0000 7f3adda46f40 1 -- 172.21.15.154:0/3107474996 shutdown_connections 2023-09-23T03:21:29.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.626+0000 7f3adda46f40 1 -- 172.21.15.154:0/3107474996 wait complete. 2023-09-23T03:21:29.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.627+0000 7f3adda46f40 1 Processor -- start 2023-09-23T03:21:29.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.627+0000 7f3adda46f40 1 -- start start 2023-09-23T03:21:29.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.628+0000 7f3adda46f40 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c7ed593350 0x55c7ed596ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:29.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.628+0000 7f3adda46f40 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c7ed594a10 0x55c7ed593950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:29.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.628+0000 7f3adda46f40 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c7ed627880 0x55c7ed617590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:29.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.628+0000 7f3adda46f40 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7ed581c30 con 0x55c7ed593350 2023-09-23T03:21:29.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.628+0000 7f3adda46f40 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7ed59a1c0 con 0x55c7ed627880 2023-09-23T03:21:29.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.628+0000 7f3adda46f40 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7ed581dc0 con 0x55c7ed594a10 2023-09-23T03:21:29.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.628+0000 7f3adacae640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c7ed593350 0x55c7ed596ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:29.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.628+0000 7f3adacae640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c7ed593350 0x55c7ed596ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49568/0 (socket says 172.21.15.154:49568) 2023-09-23T03:21:29.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.628+0000 7f3adacae640 1 -- 172.21.15.154:0/497163420 learned_addr learned my addr 172.21.15.154:0/497163420 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:29.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.629+0000 7f3adacae640 1 -- 172.21.15.154:0/497163420 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c7ed594a10 msgr2=0x55c7ed593950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:29.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.629+0000 7f3adacae640 1 --2- 172.21.15.154:0/497163420 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55c7ed594a10 0x55c7ed593950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:29.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.629+0000 7f3adacae640 1 -- 172.21.15.154:0/497163420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c7ed627880 msgr2=0x55c7ed617590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:29.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.629+0000 7f3adb4af640 1 --2- 172.21.15.154:0/497163420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c7ed627880 0x55c7ed617590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:29.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.629+0000 7f3adacae640 1 --2- 172.21.15.154:0/497163420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55c7ed627880 0x55c7ed617590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:29.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.629+0000 7f3adacae640 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3acc005ca0 con 0x55c7ed593350 2023-09-23T03:21:29.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.629+0000 7f3adacae640 1 --2- 172.21.15.154:0/497163420 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c7ed593350 0x55c7ed596ce0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00ade0 tx=0x7f3acc002f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:29.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.629+0000 7f3ac3fff640 1 -- 172.21.15.154:0/497163420 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3acc015d80 con 0x55c7ed593350 2023-09-23T03:21:29.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.629+0000 7f3ac3fff640 1 -- 172.21.15.154:0/497163420 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3acc021410 con 0x55c7ed593350 2023-09-23T03:21:29.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.629+0000 7f3ac3fff640 1 -- 172.21.15.154:0/497163420 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3acc0217c0 con 0x55c7ed593350 2023-09-23T03:21:29.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.630+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c7ed4a6650 con 0x55c7ed593350 2023-09-23T03:21:29.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.630+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c7ed499800 con 0x55c7ed593350 2023-09-23T03:21:29.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.632+0000 7f3ac3fff640 1 -- 172.21.15.154:0/497163420 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f3acc02a440 con 0x55c7ed593350 2023-09-23T03:21:29.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.633+0000 7f3ac3fff640 1 --2- 172.21.15.154:0/497163420 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3ac80698e0 0x7f3ac806bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:29.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.633+0000 7f3ac3fff640 1 -- 172.21.15.154:0/497163420 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6519+0+0 (secure 0 0 0) 0x7f3acc08a5c0 con 0x55c7ed593350 2023-09-23T03:21:29.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.633+0000 7f3ada4ad640 1 --2- 172.21.15.154:0/497163420 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3ac80698e0 0x7f3ac806bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:29.635 INFO:teuthology.orchestra.run.smithi154.stdout:Warning: using slow linear search 2023-09-23T03:21:29.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.633+0000 7f3adda46f40 1 --2- 172.21.15.154:0/497163420 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55c7ed593e90 0x55c7ed6cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:29.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.633+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:1 8.0 8:00000000:%01:::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55c7ed624a00 con 0x55c7ed593e90 2023-09-23T03:21:29.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.634+0000 7f3adb4af640 1 --2- 172.21.15.154:0/497163420 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55c7ed593e90 0x55c7ed6cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:29.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.634+0000 7f3ada4ad640 1 --2- 172.21.15.154:0/497163420 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3ac80698e0 0x7f3ac806bda0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0054000 tx=0x7f3ad004c000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:29.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.634+0000 7f3adb4af640 1 --2- 172.21.15.154:0/497163420 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55c7ed593e90 0x55c7ed6cc1f0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:29.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.635+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(1 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ad400b4e0 con 0x55c7ed593e90 2023-09-23T03:21:29.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.635+0000 7f3adb4af640 1 --2- 172.21.15.154:0/497163420 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x7f3ad400bbd0 0x7f3ad400e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:29.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.635+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:2 8.10 8:08000000:%01:::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55c7ed624a00 con 0x7f3ad400bbd0 2023-09-23T03:21:29.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.635+0000 7f3adacae640 1 --2- 172.21.15.154:0/497163420 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x7f3ad400bbd0 0x7f3ad400e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:29.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.636+0000 7f3adacae640 1 --2- 172.21.15.154:0/497163420 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x7f3ad400bbd0 0x7f3ad400e7b0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:29.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.637+0000 7f3adacae640 1 -- 172.21.15.154:0/497163420 <== osd.7 v2:172.21.15.190:6824/3460136272 1 ==== osd_op_reply(2 [pgnls start_epoch 66 out=177b] v65'10 uv10 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x7f3acc088830 con 0x7f3ad400bbd0 2023-09-23T03:21:29.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.637+0000 7f3adacae640 1 --2- 172.21.15.154:0/497163420 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x7f3acc021030 0x7f3acc057ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:29.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.637+0000 7f3adacae640 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:3 8.8 8:10000000:%01:::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55c7ed624a00 con 0x7f3acc021030 2023-09-23T03:21:29.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.638+0000 7f3ada4ad640 1 --2- 172.21.15.154:0/497163420 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x7f3acc021030 0x7f3acc057ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:29.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.639+0000 7f3ada4ad640 1 --2- 172.21.15.154:0/497163420 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x7f3acc021030 0x7f3acc057ee0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:29.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.640+0000 7f3ada4ad640 1 -- 172.21.15.154:0/497163420 <== osd.5 v2:172.21.15.190:6800/882536885 1 ==== osd_op_reply(3 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ad005c720 con 0x7f3acc021030 2023-09-23T03:21:29.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.640+0000 7f3ada4ad640 1 --2- 172.21.15.154:0/497163420 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x7f3ad0068040 0x7f3ad0052950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:29.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.640+0000 7f3ada4ad640 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:4 8.18 8:18000000:%01:::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55c7ed624a00 con 0x7f3ad0068040 2023-09-23T03:21:29.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.641+0000 7f3adb4af640 1 --2- 172.21.15.154:0/497163420 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x7f3ad0068040 0x7f3ad0052950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:29.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.642+0000 7f3adb4af640 1 --2- 172.21.15.154:0/497163420 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x7f3ad0068040 0x7f3ad0052950 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:29.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.643+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 <== osd.4 v2:172.21.15.190:6808/717302641 1 ==== osd_op_reply(4 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ad400b4e0 con 0x7f3ad0068040 2023-09-23T03:21:29.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.643+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:5 8.4 8:20000000:%01:::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55c7ed624a00 con 0x7f3acc021030 2023-09-23T03:21:29.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.644+0000 7f3ada4ad640 1 -- 172.21.15.154:0/497163420 <== osd.5 v2:172.21.15.190:6800/882536885 2 ==== osd_op_reply(5 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ad005c720 con 0x7f3acc021030 2023-09-23T03:21:29.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.644+0000 7f3ada4ad640 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:6 8.14 8:28000000:%01:::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55c7ed624a00 con 0x7f3ad0068040 2023-09-23T03:21:29.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.645+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 <== osd.4 v2:172.21.15.190:6808/717302641 2 ==== osd_op_reply(6 [pgnls start_epoch 66 out=149b] v65'99 uv99 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x7f3ad400b4e0 con 0x7f3ad0068040 2023-09-23T03:21:29.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.645+0000 7f3adb4af640 1 --2- 172.21.15.154:0/497163420 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x7f3ad40163d0 0x7f3ad4016850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:29.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.645+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:7 8.c 8:30000000:%01:::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55c7ed624a00 con 0x7f3ad40163d0 2023-09-23T03:21:29.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.646+0000 7f3adacae640 1 --2- 172.21.15.154:0/497163420 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x7f3ad40163d0 0x7f3ad4016850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:29.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.646+0000 7f3adacae640 1 --2- 172.21.15.154:0/497163420 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x7f3ad40163d0 0x7f3ad4016850 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:29.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.647+0000 7f3adacae640 1 -- 172.21.15.154:0/497163420 <== osd.0 v2:172.21.15.154:6800/2920938138 1 ==== osd_op_reply(7 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3acc088830 con 0x7f3ad40163d0 2023-09-23T03:21:29.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.647+0000 7f3adacae640 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:8 8.1c 8:38000000:%01:::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55c7ed624a00 con 0x7f3acc021030 2023-09-23T03:21:29.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.648+0000 7f3ada4ad640 1 -- 172.21.15.154:0/497163420 <== osd.5 v2:172.21.15.190:6800/882536885 3 ==== osd_op_reply(8 [pgnls start_epoch 66 out=73b] v61'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f3ad005c720 con 0x7f3acc021030 2023-09-23T03:21:29.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.648+0000 7f3ada4ad640 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:9 8.2 8:40000000:%01:::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55c7ed624a00 con 0x7f3ad0068040 2023-09-23T03:21:29.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.649+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 <== osd.4 v2:172.21.15.190:6808/717302641 3 ==== osd_op_reply(9 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ad400b4e0 con 0x7f3ad0068040 2023-09-23T03:21:29.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.649+0000 7f3adb4af640 1 --2- 172.21.15.154:0/497163420 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x7f3ad40128a0 0x7f3ad4015360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:29.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.649+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:10 8.12 8:48000000:%01:::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55c7ed624a00 con 0x7f3ad40128a0 2023-09-23T03:21:29.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.650+0000 7f3ada4ad640 1 --2- 172.21.15.154:0/497163420 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x7f3ad40128a0 0x7f3ad4015360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:29.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.651+0000 7f3ada4ad640 1 --2- 172.21.15.154:0/497163420 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x7f3ad40128a0 0x7f3ad4015360 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:29.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.652+0000 7f3ada4ad640 1 -- 172.21.15.154:0/497163420 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== osd_op_reply(10 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ad005c720 con 0x7f3ad40128a0 2023-09-23T03:21:29.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.652+0000 7f3ada4ad640 1 --2- 172.21.15.154:0/497163420 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x7f3ad0077430 0x7f3ad0077890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:29.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.652+0000 7f3ada4ad640 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:11 8.a 8:50000000:%01:::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55c7ed624a00 con 0x7f3ad0077430 2023-09-23T03:21:29.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.653+0000 7f3adb4af640 1 --2- 172.21.15.154:0/497163420 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x7f3ad0077430 0x7f3ad0077890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:29.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.654+0000 7f3adb4af640 1 --2- 172.21.15.154:0/497163420 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x7f3ad0077430 0x7f3ad0077890 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:29.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.654+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 <== osd.1 v2:172.21.15.154:6816/1267710005 1 ==== osd_op_reply(11 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ad400b4e0 con 0x7f3ad0077430 2023-09-23T03:21:29.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.654+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:12 8.1a 8:58000000:%01:::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55c7ed624a00 con 0x7f3ad40163d0 2023-09-23T03:21:29.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.655+0000 7f3adacae640 1 -- 172.21.15.154:0/497163420 <== osd.0 v2:172.21.15.154:6800/2920938138 2 ==== osd_op_reply(12 [pgnls start_epoch 66 out=49b] v61'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3acc088830 con 0x7f3ad40163d0 2023-09-23T03:21:29.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.655+0000 7f3adacae640 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:13 8.6 8:60000000:%01:::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55c7ed624a00 con 0x7f3ad40163d0 2023-09-23T03:21:29.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.655+0000 7f3adacae640 1 -- 172.21.15.154:0/497163420 <== osd.0 v2:172.21.15.154:6800/2920938138 3 ==== osd_op_reply(13 [pgnls start_epoch 66 out=77b] v65'23 uv23 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x7f3acc088830 con 0x7f3ad40163d0 2023-09-23T03:21:29.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.655+0000 7f3adacae640 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:14 8.16 8:68000000:%01:::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55c7ed624a00 con 0x55c7ed593e90 2023-09-23T03:21:29.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.656+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 <== osd.3 v2:172.21.15.154:6808/1133688468 2 ==== osd_op_reply(14 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ad400b4e0 con 0x55c7ed593e90 2023-09-23T03:21:29.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.656+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:15 8.e 8:70000000:%01:::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55c7ed624a00 con 0x7f3ad40163d0 2023-09-23T03:21:29.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.656+0000 7f3adacae640 1 -- 172.21.15.154:0/497163420 <== osd.0 v2:172.21.15.154:6800/2920938138 4 ==== osd_op_reply(15 [pgnls start_epoch 66 out=73b] v61'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f3acc088830 con 0x7f3ad40163d0 2023-09-23T03:21:29.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.656+0000 7f3adacae640 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:16 8.1e 8:78000000:%01:::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55c7ed624a00 con 0x55c7ed593e90 2023-09-23T03:21:29.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.656+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 <== osd.3 v2:172.21.15.154:6808/1133688468 3 ==== osd_op_reply(16 [pgnls start_epoch 66 out=49b] v61'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ad400b4e0 con 0x55c7ed593e90 2023-09-23T03:21:29.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.656+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:17 8.1 8:80000000:%01:::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55c7ed624a00 con 0x7f3ad0077430 2023-09-23T03:21:29.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.657+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 <== osd.1 v2:172.21.15.154:6816/1267710005 2 ==== osd_op_reply(17 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ad400b4e0 con 0x7f3ad0077430 2023-09-23T03:21:29.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.657+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:18 8.11 8:88000000:%01:::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55c7ed624a00 con 0x7f3acc021030 2023-09-23T03:21:29.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.658+0000 7f3ada4ad640 1 -- 172.21.15.154:0/497163420 <== osd.5 v2:172.21.15.190:6800/882536885 4 ==== osd_op_reply(18 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ad005c720 con 0x7f3acc021030 2023-09-23T03:21:29.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.658+0000 7f3ada4ad640 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:19 8.9 8:90000000:%01:::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55c7ed624a00 con 0x7f3ad40128a0 2023-09-23T03:21:29.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.659+0000 7f3ada4ad640 1 -- 172.21.15.154:0/497163420 <== osd.6 v2:172.21.15.190:6816/1964294001 2 ==== osd_op_reply(19 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ad005c720 con 0x7f3ad40128a0 2023-09-23T03:21:29.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.659+0000 7f3ada4ad640 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:20 8.19 8:98000000:%01:::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55c7ed624a00 con 0x7f3acc021030 2023-09-23T03:21:29.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.660+0000 7f3ada4ad640 1 -- 172.21.15.154:0/497163420 <== osd.5 v2:172.21.15.190:6800/882536885 5 ==== osd_op_reply(20 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ad005c720 con 0x7f3acc021030 2023-09-23T03:21:29.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.660+0000 7f3ada4ad640 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:21 8.5 8:a0000000:%01:::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55c7ed624a00 con 0x7f3ad40128a0 2023-09-23T03:21:29.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.662+0000 7f3ada4ad640 1 -- 172.21.15.154:0/497163420 <== osd.6 v2:172.21.15.190:6816/1964294001 3 ==== osd_op_reply(21 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ad005c720 con 0x7f3ad40128a0 2023-09-23T03:21:29.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.662+0000 7f3ada4ad640 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:22 8.15 8:a8000000:%01:::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55c7ed624a00 con 0x7f3ad400bbd0 2023-09-23T03:21:29.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.663+0000 7f3adacae640 1 -- 172.21.15.154:0/497163420 <== osd.7 v2:172.21.15.190:6824/3460136272 2 ==== osd_op_reply(22 [pgnls start_epoch 66 out=97b] v61'3 uv3 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x7f3acc088830 con 0x7f3ad400bbd0 2023-09-23T03:21:29.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.663+0000 7f3adacae640 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:23 8.d 8:b0000000:%01:::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55c7ed624a00 con 0x7f3acc021030 2023-09-23T03:21:29.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.664+0000 7f3ada4ad640 1 -- 172.21.15.154:0/497163420 <== osd.5 v2:172.21.15.190:6800/882536885 6 ==== osd_op_reply(23 [pgnls start_epoch 66 out=74b] v61'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f3ad005c720 con 0x7f3acc021030 2023-09-23T03:21:29.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.664+0000 7f3ada4ad640 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:24 8.1d 8:b8000000:%01:::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55c7ed624a00 con 0x7f3ad40163d0 2023-09-23T03:21:29.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.665+0000 7f3adacae640 1 -- 172.21.15.154:0/497163420 <== osd.0 v2:172.21.15.154:6800/2920938138 5 ==== osd_op_reply(24 [pgnls start_epoch 66 out=73b] v61'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f3acc088830 con 0x7f3ad40163d0 2023-09-23T03:21:29.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.665+0000 7f3adacae640 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:25 8.3 8:c0000000:%01:::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55c7ed624a00 con 0x7f3ad0077430 2023-09-23T03:21:29.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.665+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 <== osd.1 v2:172.21.15.154:6816/1267710005 3 ==== osd_op_reply(25 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ad400b4e0 con 0x7f3ad0077430 2023-09-23T03:21:29.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.666+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:26 8.13 8:c8000000:%01:::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55c7ed624a00 con 0x55c7ed593e90 2023-09-23T03:21:29.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.666+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 <== osd.3 v2:172.21.15.154:6808/1133688468 4 ==== osd_op_reply(26 [pgnls start_epoch 66 out=98b] v65'3 uv3 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x7f3ad400b4e0 con 0x55c7ed593e90 2023-09-23T03:21:29.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.666+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:27 8.b 8:d0000000:%01:::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55c7ed624a00 con 0x55c7ed593e90 2023-09-23T03:21:29.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.667+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 <== osd.3 v2:172.21.15.154:6808/1133688468 5 ==== osd_op_reply(27 [pgnls start_epoch 66 out=76b] v65'3 uv3 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x7f3ad400b4e0 con 0x55c7ed593e90 2023-09-23T03:21:29.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.667+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:28 8.1b 8:d8000000:%01:::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55c7ed624a00 con 0x7f3ad0068040 2023-09-23T03:21:29.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.668+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 <== osd.4 v2:172.21.15.190:6808/717302641 4 ==== osd_op_reply(28 [pgnls start_epoch 66 out=49b] v61'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3ad400b4e0 con 0x7f3ad0068040 2023-09-23T03:21:29.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.668+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:29 8.7 8:e0000000:%01:::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55c7ed624a00 con 0x7f3ad0077430 2023-09-23T03:21:29.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.669+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 <== osd.1 v2:172.21.15.154:6816/1267710005 4 ==== osd_op_reply(29 [pgnls start_epoch 66 out=73b] v61'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f3ad400b4e0 con 0x7f3ad0077430 2023-09-23T03:21:29.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.669+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:30 8.17 8:e8000000:%01:::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55c7ed624a00 con 0x7f3ad400bbd0 2023-09-23T03:21:29.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.671+0000 7f3adacae640 1 -- 172.21.15.154:0/497163420 <== osd.7 v2:172.21.15.190:6824/3460136272 3 ==== osd_op_reply(30 [pgnls start_epoch 66 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3acc088830 con 0x7f3ad400bbd0 2023-09-23T03:21:29.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.671+0000 7f3adacae640 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:31 8.f 8:f0000000:%01:::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55c7ed624a00 con 0x55c7ed593e90 2023-09-23T03:21:29.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.672+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 <== osd.3 v2:172.21.15.154:6808/1133688468 6 ==== osd_op_reply(31 [pgnls start_epoch 66 out=73b] v61'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f3ad400b4e0 con 0x55c7ed593e90 2023-09-23T03:21:29.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.672+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:32 8.1f 8:f8000000:%01:::head [pgnls start_epoch 66 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e66) v8 -- 0x55c7ed624a00 con 0x7f3ad40163d0 2023-09-23T03:21:29.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.672+0000 7f3adacae640 1 -- 172.21.15.154:0/497163420 <== osd.0 v2:172.21.15.154:6800/2920938138 6 ==== osd_op_reply(32 [pgnls start_epoch 66 out=101b] v65'6 uv6 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x7f3acc088830 con 0x7f3ad40163d0 2023-09-23T03:21:29.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.672+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:33 8.10 8:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e66) v8 -- 0x55c7ed583230 con 0x7f3ad400bbd0 2023-09-23T03:21:29.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.672+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:34 8.10 8:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e66) v8 -- 0x55c7ed5826f0 con 0x7f3ad400bbd0 2023-09-23T03:21:29.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.672+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:35 8.10 8:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e66) v8 -- 0x55c7ed6ccbd0 con 0x7f3ad400bbd0 2023-09-23T03:21:29.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.673+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:36 8.10 8:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e66) v8 -- 0x55c7ed6cd3d0 con 0x7f3ad400bbd0 2023-09-23T03:21:29.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.673+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:37 8.10 8:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e66) v8 -- 0x55c7ed6cdbd0 con 0x7f3ad400bbd0 2023-09-23T03:21:29.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.673+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:38 8.14 8:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e66) v8 -- 0x55c7ed6ce3d0 con 0x7f3ad0068040 2023-09-23T03:21:29.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.673+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:39 8.14 8:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e66) v8 -- 0x55c7ed6cebd0 con 0x7f3ad0068040 2023-09-23T03:21:29.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.673+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:40 8.14 8:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e66) v8 -- 0x55c7ed6cf3d0 con 0x7f3ad0068040 2023-09-23T03:21:29.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.673+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:41 8.14 8:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e66) v8 -- 0x55c7ed6cfbd0 con 0x7f3ad0068040 2023-09-23T03:21:29.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.673+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:42 8.1c 8:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e66) v8 -- 0x55c7ed6d0510 con 0x7f3acc021030 2023-09-23T03:21:29.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.673+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:43 8.6 8:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e66) v8 -- 0x55c7ed6d1040 con 0x7f3ad40163d0 2023-09-23T03:21:29.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.673+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:44 8.e 8:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e66) v8 -- 0x55c7ed6d1b10 con 0x7f3ad40163d0 2023-09-23T03:21:29.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.673+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:45 8.15 8:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e66) v8 -- 0x55c7ed6d25d0 con 0x7f3ad400bbd0 2023-09-23T03:21:29.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.673+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:46 8.15 8:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e66) v8 -- 0x55c7ed6d3060 con 0x7f3ad400bbd0 2023-09-23T03:21:29.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.673+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:47 8.d 8:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e66) v8 -- 0x55c7ed6d3b50 con 0x7f3acc021030 2023-09-23T03:21:29.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.673+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:48 8.1d 8:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e66) v8 -- 0x55c7ed6d4350 con 0x7f3ad40163d0 2023-09-23T03:21:29.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.677+0000 7f3adacae640 1 -- 172.21.15.154:0/497163420 <== osd.0 v2:172.21.15.154:6800/2920938138 7 ==== osd_op_reply(48 605.00000000 [delete] v66'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3acc088830 con 0x7f3ad40163d0 2023-09-23T03:21:29.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.677+0000 7f3adacae640 1 -- 172.21.15.154:0/497163420 <== osd.0 v2:172.21.15.154:6800/2920938138 8 ==== osd_op_reply(44 608.00000000 [delete] v66'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3acc088830 con 0x7f3ad40163d0 2023-09-23T03:21:29.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.677+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:49 8.13 8:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e66) v8 -- 0x55c7ed6d3060 con 0x55c7ed593e90 2023-09-23T03:21:29.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.677+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:50 8.13 8:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e66) v8 -- 0x55c7ed6d4350 con 0x55c7ed593e90 2023-09-23T03:21:29.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.679+0000 7f3adacae640 1 -- 172.21.15.154:0/497163420 <== osd.0 v2:172.21.15.154:6800/2920938138 9 ==== osd_op_reply(43 mds0_openfiles.0 [delete] v66'24 uv24 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f3acc088830 con 0x7f3ad40163d0 2023-09-23T03:21:29.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.679+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:51 8.b 8:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e66) v8 -- 0x55c7ed6d46b0 con 0x55c7ed593e90 2023-09-23T03:21:29.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.680+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 <== osd.3 v2:172.21.15.154:6808/1133688468 7 ==== osd_op_reply(49 mds0_inotable [delete] v66'4 uv4 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3ad400b4e0 con 0x55c7ed593e90 2023-09-23T03:21:29.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.680+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:52 8.7 8:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e66) v8 -- 0x55c7ed6d4a10 con 0x7f3ad0077430 2023-09-23T03:21:29.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.681+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 <== osd.3 v2:172.21.15.154:6808/1133688468 8 ==== osd_op_reply(50 100.00000000 [delete] v66'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3ad400b4e0 con 0x55c7ed593e90 2023-09-23T03:21:29.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.681+0000 7f3adacae640 1 -- 172.21.15.154:0/497163420 <== osd.7 v2:172.21.15.190:6824/3460136272 4 ==== osd_op_reply(45 604.00000000 [delete] v66'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3acc088830 con 0x7f3ad400bbd0 2023-09-23T03:21:29.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.681+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:53 8.f 8:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e66) v8 -- 0x55c7ed6d2010 con 0x55c7ed593e90 2023-09-23T03:21:29.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.681+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:54 8.1f 8:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e66) v8 -- 0x55c7ed6d4eb0 con 0x7f3ad40163d0 2023-09-23T03:21:29.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.682+0000 7f3ada4ad640 1 -- 172.21.15.154:0/497163420 <== osd.5 v2:172.21.15.190:6800/882536885 7 ==== osd_op_reply(42 607.00000000 [delete] v66'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3ad005c720 con 0x7f3acc021030 2023-09-23T03:21:29.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.682+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 <== osd.4 v2:172.21.15.190:6808/717302641 5 ==== osd_op_reply(38 1.00000000.inode [delete] v66'100 uv100 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f3ad400b4e0 con 0x7f3ad0068040 2023-09-23T03:21:29.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.682+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 <== osd.4 v2:172.21.15.190:6808/717302641 6 ==== osd_op_reply(39 200.00000000 [delete] v66'101 uv101 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3ad400b4e0 con 0x7f3ad0068040 2023-09-23T03:21:29.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.682+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:55 8.1f 8:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e66) v8 -- 0x55c7ed6d2010 con 0x7f3ad40163d0 2023-09-23T03:21:29.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.682+0000 7f3adacae640 1 -- 172.21.15.154:0/497163420 <== osd.7 v2:172.21.15.190:6824/3460136272 5 ==== osd_op_reply(46 500.00000000 [delete] v66'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3acc088830 con 0x7f3ad400bbd0 2023-09-23T03:21:29.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.683+0000 7f3ada4ad640 1 -- 172.21.15.154:0/497163420 <== osd.5 v2:172.21.15.190:6800/882536885 8 ==== osd_op_reply(47 mds_snaptable [delete] v66'2 uv2 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f3ad005c720 con 0x7f3acc021030 2023-09-23T03:21:29.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.684+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 <== osd.1 v2:172.21.15.154:6816/1267710005 5 ==== osd_op_reply(52 609.00000000 [delete] v66'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3ad400b4e0 con 0x7f3ad0077430 2023-09-23T03:21:29.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.684+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 <== osd.3 v2:172.21.15.154:6808/1133688468 9 ==== osd_op_reply(53 400.00000000 [delete] v66'2 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3ad400b4e0 con 0x55c7ed593e90 2023-09-23T03:21:29.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.684+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 <== osd.4 v2:172.21.15.190:6808/717302641 7 ==== osd_op_reply(40 200.00000001 [delete] v66'102 uv102 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3ad400b4e0 con 0x7f3ad0068040 2023-09-23T03:21:29.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.684+0000 7f3adacae640 1 -- 172.21.15.154:0/497163420 <== osd.7 v2:172.21.15.190:6824/3460136272 6 ==== osd_op_reply(33 601.00000000 [delete] v66'11 uv11 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3acc088830 con 0x7f3ad400bbd0 2023-09-23T03:21:29.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.684+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 <== osd.4 v2:172.21.15.190:6808/717302641 8 ==== osd_op_reply(41 606.00000000 [delete] v66'103 uv103 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3ad400b4e0 con 0x7f3ad0068040 2023-09-23T03:21:29.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.684+0000 7f3adacae640 1 -- 172.21.15.154:0/497163420 <== osd.7 v2:172.21.15.190:6824/3460136272 7 ==== osd_op_reply(34 602.00000000 [delete] v66'12 uv12 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3acc088830 con 0x7f3ad400bbd0 2023-09-23T03:21:29.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.684+0000 7f3adacae640 1 -- 172.21.15.154:0/497163420 <== osd.7 v2:172.21.15.190:6824/3460136272 8 ==== osd_op_reply(35 10000000000.00000000 [delete] v66'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f3acc088830 con 0x7f3ad400bbd0 2023-09-23T03:21:29.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.684+0000 7f3adacae640 1 -- 172.21.15.154:0/497163420 <== osd.7 v2:172.21.15.190:6824/3460136272 9 ==== osd_op_reply(36 600.00000000 [delete] v66'14 uv14 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3acc088830 con 0x7f3ad400bbd0 2023-09-23T03:21:29.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.685+0000 7f3adacae640 1 -- 172.21.15.154:0/497163420 <== osd.7 v2:172.21.15.190:6824/3460136272 10 ==== osd_op_reply(37 603.00000000 [delete] v66'15 uv15 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3acc088830 con 0x7f3ad400bbd0 2023-09-23T03:21:29.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.686+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 <== osd.3 v2:172.21.15.154:6808/1133688468 10 ==== osd_op_reply(51 mds0_sessionmap [delete] v66'4 uv4 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f3ad400b4e0 con 0x55c7ed593e90 2023-09-23T03:21:29.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.688+0000 7f3adacae640 1 -- 172.21.15.154:0/497163420 <== osd.0 v2:172.21.15.154:6800/2920938138 10 ==== osd_op_reply(54 100.00000000.inode [delete] v66'7 uv7 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f3acc088830 con 0x7f3ad40163d0 2023-09-23T03:21:29.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.689+0000 7f3adacae640 1 -- 172.21.15.154:0/497163420 <== osd.0 v2:172.21.15.154:6800/2920938138 11 ==== osd_op_reply(55 1.00000000 [delete] v66'8 uv8 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f3acc088830 con 0x7f3ad40163d0 2023-09-23T03:21:29.691 INFO:teuthology.orchestra.run.smithi154.stdout:Removed 23 objects 2023-09-23T03:21:29.691 INFO:teuthology.orchestra.run.smithi154.stdout:successfully purged pool cephfs_metadata 2023-09-23T03:21:29.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.689+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x7f3ad40163d0 msgr2=0x7f3ad4016850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:29.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.689+0000 7f3adda46f40 1 --2- 172.21.15.154:0/497163420 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x7f3ad40163d0 0x7f3ad4016850 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:29.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.689+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x7f3ad0077430 msgr2=0x7f3ad0077890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:29.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.689+0000 7f3adda46f40 1 --2- 172.21.15.154:0/497163420 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x7f3ad0077430 0x7f3ad0077890 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:29.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.689+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55c7ed593e90 msgr2=0x55c7ed6cc1f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:29.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.689+0000 7f3adda46f40 1 --2- 172.21.15.154:0/497163420 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55c7ed593e90 0x55c7ed6cc1f0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:29.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.689+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x7f3ad0068040 msgr2=0x7f3ad0052950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:29.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.689+0000 7f3adda46f40 1 --2- 172.21.15.154:0/497163420 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x7f3ad0068040 0x7f3ad0052950 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:29.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.689+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x7f3acc021030 msgr2=0x7f3acc057ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:29.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.689+0000 7f3adda46f40 1 --2- 172.21.15.154:0/497163420 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x7f3acc021030 0x7f3acc057ee0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:29.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.689+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x7f3ad40128a0 msgr2=0x7f3ad4015360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:29.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.689+0000 7f3adda46f40 1 --2- 172.21.15.154:0/497163420 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x7f3ad40128a0 0x7f3ad4015360 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:29.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.689+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x7f3ad400bbd0 msgr2=0x7f3ad400e7b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:29.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.689+0000 7f3adda46f40 1 --2- 172.21.15.154:0/497163420 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x7f3ad400bbd0 0x7f3ad400e7b0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:29.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.689+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 reap_dead start 2023-09-23T03:21:29.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.689+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3ac80698e0 msgr2=0x7f3ac806bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:29.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.689+0000 7f3adda46f40 1 --2- 172.21.15.154:0/497163420 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3ac80698e0 0x7f3ac806bda0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0054000 tx=0x7f3ad004c000 comp rx=0 tx=0).stop 2023-09-23T03:21:29.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.689+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c7ed593350 msgr2=0x55c7ed596ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:29.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.689+0000 7f3adda46f40 1 --2- 172.21.15.154:0/497163420 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c7ed593350 0x55c7ed596ce0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00ade0 tx=0x7f3acc002f30 comp rx=0 tx=0).stop 2023-09-23T03:21:29.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.689+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 reap_dead start 2023-09-23T03:21:29.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.690+0000 7f3adb4af640 1 -- 172.21.15.154:0/497163420 reap_dead start 2023-09-23T03:21:29.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.690+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 shutdown_connections 2023-09-23T03:21:29.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.690+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 >> 172.21.15.154:0/497163420 conn(0x55c7ed54df20 msgr2=0x55c7ed61ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:29.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.690+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 shutdown_connections 2023-09-23T03:21:29.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.690+0000 7f3adda46f40 1 -- 172.21.15.154:0/497163420 wait complete. 2023-09-23T03:21:29.697 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs reset cephfs --yes-i-really-mean-it 2023-09-23T03:21:29.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.909+0000 7f58af202640 1 Processor -- start 2023-09-23T03:21:29.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.910+0000 7f58af202640 1 -- start start 2023-09-23T03:21:29.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.911+0000 7f58af202640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58a80b42f0 0x7f58a80b4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:29.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.911+0000 7f58af202640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f58a80af850 0x7f58a80afcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:29.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.911+0000 7f58af202640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58a80af3c0 con 0x7f58a815cbf0 2023-09-23T03:21:29.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.911+0000 7f58af202640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58a80b5150 con 0x7f58a80af850 2023-09-23T03:21:29.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.911+0000 7f58af202640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a80b4cb0 con 0x7f58a80b42f0 2023-09-23T03:21:29.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.912+0000 7f58aea01640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f58a80af850 0x7f58a80afcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:29.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.912+0000 7f58ad9ff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58a80b42f0 0x7f58a80b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:29.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.912+0000 7f58ad9ff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58a80b42f0 0x7f58a80b4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:48280/0 (socket says 172.21.15.154:48280) 2023-09-23T03:21:29.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.912+0000 7f58ad9ff640 1 -- 172.21.15.154:0/3857921128 learned_addr learned my addr 172.21.15.154:0/3857921128 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:29.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.913+0000 7f58ad9ff640 1 -- 172.21.15.154:0/3857921128 >> v2:172.21.15.190:3300/0 conn(0x7f58a80af850 msgr2=0x7f58a80afcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:29.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.913+0000 7f58ad9ff640 1 --2- 172.21.15.154:0/3857921128 >> v2:172.21.15.190:3300/0 conn(0x7f58a80af850 0x7f58a80afcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:29.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.913+0000 7f58ad9ff640 1 -- 172.21.15.154:0/3857921128 >> v1:172.21.15.190:6789/0 conn(0x7f58a815cbf0 legacy=0x7f58a806b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:29.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.913+0000 7f58ad9ff640 1 -- 172.21.15.154:0/3857921128 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a80d14f0 con 0x7f58a80b42f0 2023-09-23T03:21:29.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.913+0000 7f58aea01640 1 --2- 172.21.15.154:0/3857921128 >> v2:172.21.15.190:3300/0 conn(0x7f58a80af850 0x7f58a80afcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:21:29.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.913+0000 7f58ad1fe640 1 -- 172.21.15.154:0/3857921128 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3228466217 0 0) 0x7f58a80af3c0 con 0x7f58a815cbf0 2023-09-23T03:21:29.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.914+0000 7f58ad9ff640 1 --2- 172.21.15.154:0/3857921128 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58a80b42f0 0x7f58a80b4770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5898009870 tx=0x7f589802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c6410c369834ef9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:29.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.915+0000 7f58ad1fe640 1 -- 172.21.15.154:0/3857921128 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5898030840 con 0x7f58a80b42f0 2023-09-23T03:21:29.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.915+0000 7f58ad1fe640 1 -- 172.21.15.154:0/3857921128 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58980309a0 con 0x7f58a80b42f0 2023-09-23T03:21:29.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.915+0000 7f58ad1fe640 1 -- 172.21.15.154:0/3857921128 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5898030d30 con 0x7f58a80b42f0 2023-09-23T03:21:29.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.916+0000 7f58af202640 1 -- 172.21.15.154:0/3857921128 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58a80b42f0 msgr2=0x7f58a80b4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:29.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.916+0000 7f58af202640 1 --2- 172.21.15.154:0/3857921128 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58a80b42f0 0x7f58a80b4770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5898009870 tx=0x7f589802b980 comp rx=0 tx=0).stop 2023-09-23T03:21:29.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.916+0000 7f58af202640 1 -- 172.21.15.154:0/3857921128 shutdown_connections 2023-09-23T03:21:29.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.916+0000 7f58af202640 1 --2- 172.21.15.154:0/3857921128 >> v2:172.21.15.190:3300/0 conn(0x7f58a80af850 0x7f58a80afcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:29.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.916+0000 7f58af202640 1 --2- 172.21.15.154:0/3857921128 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58a80b42f0 0x7f58a80b4770 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:29.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.916+0000 7f58af202640 1 -- 172.21.15.154:0/3857921128 >> 172.21.15.154:0/3857921128 conn(0x7f58a808c220 msgr2=0x7f58a809bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:29.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.916+0000 7f58af202640 1 -- 172.21.15.154:0/3857921128 shutdown_connections 2023-09-23T03:21:29.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.916+0000 7f58af202640 1 -- 172.21.15.154:0/3857921128 wait complete. 2023-09-23T03:21:29.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.918+0000 7f58af202640 1 Processor -- start 2023-09-23T03:21:29.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.918+0000 7f58af202640 1 -- start start 2023-09-23T03:21:29.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.919+0000 7f58af202640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58a80af850 0x7f58a8152020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:29.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.919+0000 7f58af202640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58a80b42f0 0x7f58a815f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:29.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.919+0000 7f58af202640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58a815cbf0 0x7f58a815f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:29.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.919+0000 7f58af202640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a815eb00 con 0x7f58a815cbf0 2023-09-23T03:21:29.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.919+0000 7f58af202640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a8152560 con 0x7f58a80af850 2023-09-23T03:21:29.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.919+0000 7f58af202640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a815e260 con 0x7f58a80b42f0 2023-09-23T03:21:29.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.919+0000 7f58ae200640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58a80af850 0x7f58a8152020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:29.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.920+0000 7f58ae200640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58a80af850 0x7f58a8152020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.154:44414/0 (socket says 172.21.15.154:44414) 2023-09-23T03:21:29.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.920+0000 7f58ae200640 1 -- 172.21.15.154:0/1414440242 learned_addr learned my addr 172.21.15.154:0/1414440242 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:29.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.920+0000 7f58aea01640 1 --2- 172.21.15.154:0/1414440242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58a815cbf0 0x7f58a815f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:29.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.920+0000 7f58ad9ff640 1 --2- 172.21.15.154:0/1414440242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58a80b42f0 0x7f58a815f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:29.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.920+0000 7f58aea01640 1 -- 172.21.15.154:0/1414440242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58a80b42f0 msgr2=0x7f58a815f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:29.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.920+0000 7f58aea01640 1 --2- 172.21.15.154:0/1414440242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58a80b42f0 0x7f58a815f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:29.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.920+0000 7f58aea01640 1 -- 172.21.15.154:0/1414440242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58a80af850 msgr2=0x7f58a8152020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:29.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.920+0000 7f58aea01640 1 --2- 172.21.15.154:0/1414440242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58a80af850 0x7f58a8152020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:29.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.920+0000 7f58aea01640 1 -- 172.21.15.154:0/1414440242 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a0009a70 con 0x7f58a815cbf0 2023-09-23T03:21:29.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.920+0000 7f58aea01640 1 --2- 172.21.15.154:0/1414440242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58a815cbf0 0x7f58a815f950 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f58a000ef90 tx=0x7f58a000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:29.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.921+0000 7f589effd640 1 -- 172.21.15.154:0/1414440242 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a0014480 con 0x7f58a815cbf0 2023-09-23T03:21:29.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.921+0000 7f589effd640 1 -- 172.21.15.154:0/1414440242 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a000f040 con 0x7f58a815cbf0 2023-09-23T03:21:29.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.921+0000 7f589effd640 1 -- 172.21.15.154:0/1414440242 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a0014840 con 0x7f58a815cbf0 2023-09-23T03:21:29.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.921+0000 7f58af202640 1 -- 172.21.15.154:0/1414440242 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58980094d0 con 0x7f58a815cbf0 2023-09-23T03:21:29.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.921+0000 7f58af202640 1 -- 172.21.15.154:0/1414440242 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a815e820 con 0x7f58a815cbf0 2023-09-23T03:21:29.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.922+0000 7f58af202640 1 -- 172.21.15.154:0/1414440242 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a8067390 con 0x7f58a815cbf0 2023-09-23T03:21:29.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.923+0000 7f589effd640 1 -- 172.21.15.154:0/1414440242 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f58a00149a0 con 0x7f58a815cbf0 2023-09-23T03:21:29.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.924+0000 7f589effd640 1 --2- 172.21.15.154:0/1414440242 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f588c06af60 0x7f588c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:29.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.924+0000 7f589effd640 1 -- 172.21.15.154:0/1414440242 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6519+0+0 (secure 0 0 0) 0x7f58a008a8f0 con 0x7f58a815cbf0 2023-09-23T03:21:29.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.925+0000 7f58ae200640 1 --2- 172.21.15.154:0/1414440242 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f588c06af60 0x7f588c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:29.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.925+0000 7f58ae200640 1 --2- 172.21.15.154:0/1414440242 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f588c06af60 0x7f588c06d420 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f58a4000fb0 tx=0x7f58a4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:29.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:29.928+0000 7f589effd640 1 -- 172.21.15.154:0/1414440242 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f58a0055610 con 0x7f58a815cbf0 2023-09-23T03:21:30.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:30.158+0000 7f58af202640 1 -- 172.21.15.154:0/1414440242 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs reset", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f58a80d00f0 con 0x7f58a815cbf0 2023-09-23T03:21:31.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.051+0000 7f589effd640 1 -- 172.21.15.154:0/1414440242 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs reset", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v44) v1 ==== 107+0+0 (secure 0 0 0) 0x7f58a0058c30 con 0x7f58a815cbf0 2023-09-23T03:21:31.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.054+0000 7f58af202640 1 -- 172.21.15.154:0/1414440242 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f588c06af60 msgr2=0x7f588c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:31.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.054+0000 7f58af202640 1 --2- 172.21.15.154:0/1414440242 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f588c06af60 0x7f588c06d420 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f58a4000fb0 tx=0x7f58a4009040 comp rx=0 tx=0).stop 2023-09-23T03:21:31.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.054+0000 7f58af202640 1 -- 172.21.15.154:0/1414440242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58a815cbf0 msgr2=0x7f58a815f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:31.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.054+0000 7f58af202640 1 --2- 172.21.15.154:0/1414440242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58a815cbf0 0x7f58a815f950 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f58a000ef90 tx=0x7f58a000c490 comp rx=0 tx=0).stop 2023-09-23T03:21:31.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.054+0000 7f58af202640 1 -- 172.21.15.154:0/1414440242 shutdown_connections 2023-09-23T03:21:31.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.054+0000 7f58af202640 1 --2- 172.21.15.154:0/1414440242 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f588c06af60 0x7f588c06d420 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:31.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.054+0000 7f58af202640 1 --2- 172.21.15.154:0/1414440242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58a815cbf0 0x7f58a815f950 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:31.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.054+0000 7f58af202640 1 --2- 172.21.15.154:0/1414440242 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58a80b42f0 0x7f58a815f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:31.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.054+0000 7f58af202640 1 --2- 172.21.15.154:0/1414440242 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58a80af850 0x7f58a8152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:31.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.055+0000 7f58af202640 1 -- 172.21.15.154:0/1414440242 >> 172.21.15.154:0/1414440242 conn(0x7f58a808c220 msgr2=0x7f58a80cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:31.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.055+0000 7f58af202640 1 -- 172.21.15.154:0/1414440242 shutdown_connections 2023-09-23T03:21:31.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.055+0000 7f58af202640 1 -- 172.21.15.154:0/1414440242 wait complete. 2023-09-23T03:21:31.067 INFO:tasks.ceph.mds.c.smithi154.stderr:2023-09-23T03:21:31.066+0000 7f7a5ce5d640 -1 mds.0.inotable: load_2 could not read table: -2 2023-09-23T03:21:31.068 INFO:tasks.ceph.mds.c.smithi154.stderr:2023-09-23T03:21:31.066+0000 7f7a5ce5d640 -1 log_channel(cluster) log [ERR] : error reading table object 'mds0_inotable' -2 ((2) No such file or directory) 2023-09-23T03:21:31.069 INFO:tasks.ceph.mon.a.smithi154.stderr:2023-09-23T03:21:31.068+0000 7f3df35a6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-23T03:21:31.069 INFO:tasks.ceph.mon.a.smithi154.stderr:2023-09-23T03:21:31.068+0000 7f3df35a6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-09-23T03:21:31.078 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-09-23T03:21:31.084 INFO:tasks.ceph.mds.c.smithi154.stderr: -30> 2023-09-23T03:21:31.066+0000 7f7a5ce5d640 -1 mds.0.inotable: load_2 could not read table: -2 2023-09-23T03:21:31.084 INFO:tasks.ceph.mds.c.smithi154.stderr: -29> 2023-09-23T03:21:31.066+0000 7f7a5ce5d640 -1 log_channel(cluster) log [ERR] : error reading table object 'mds0_inotable' -2 ((2) No such file or directory) 2023-09-23T03:21:31.128 INFO:tasks.ceph.mds.c.smithi154.stdout:starting mds.c at 2023-09-23T03:21:31.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.272+0000 7f12d15e2640 1 Processor -- start 2023-09-23T03:21:31.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.273+0000 7f12d15e2640 1 -- start start 2023-09-23T03:21:31.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.274+0000 7f12d15e2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12cc15ba50 0x7f12cc06fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:31.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.274+0000 7f12d15e2640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f12cc15eef0 0x7f12cc0703d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:31.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.274+0000 7f12d15e2640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12cc0cb6c0 con 0x7f12cc158480 2023-09-23T03:21:31.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.274+0000 7f12d15e2640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12cc15ad80 con 0x7f12cc15eef0 2023-09-23T03:21:31.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.274+0000 7f12d15e2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc15af70 con 0x7f12cc15ba50 2023-09-23T03:21:31.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.275+0000 7f12c3fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12cc15ba50 0x7f12cc06fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:31.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.275+0000 7f12c3fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12cc15ba50 0x7f12cc06fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:48332/0 (socket says 172.21.15.154:48332) 2023-09-23T03:21:31.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.275+0000 7f12c3fff640 1 -- 172.21.15.154:0/3783571241 learned_addr learned my addr 172.21.15.154:0/3783571241 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:31.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.275+0000 7f12d0de1640 1 --2- 172.21.15.154:0/3783571241 >> v2:172.21.15.190:3300/0 conn(0x7f12cc15eef0 0x7f12cc0703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:31.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.275+0000 7f12c3fff640 1 -- 172.21.15.154:0/3783571241 >> v2:172.21.15.190:3300/0 conn(0x7f12cc15eef0 msgr2=0x7f12cc0703d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:31.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.275+0000 7f12c3fff640 1 --2- 172.21.15.154:0/3783571241 >> v2:172.21.15.190:3300/0 conn(0x7f12cc15eef0 0x7f12cc0703d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:31.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.275+0000 7f12c3fff640 1 -- 172.21.15.154:0/3783571241 >> v1:172.21.15.190:6789/0 conn(0x7f12cc158480 legacy=0x7f12cc0cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:31.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.276+0000 7f12c3fff640 1 -- 172.21.15.154:0/3783571241 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12cc15b250 con 0x7f12cc15ba50 2023-09-23T03:21:31.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.277+0000 7f12c3fff640 1 --2- 172.21.15.154:0/3783571241 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12cc15ba50 0x7f12cc06fe90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f12b4009870 tx=0x7f12b402b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9522e797789bdcd0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:31.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.277+0000 7f12cb7fe640 1 -- 172.21.15.154:0/3783571241 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b40317c0 con 0x7f12cc15ba50 2023-09-23T03:21:31.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.277+0000 7f12cb7fe640 1 -- 172.21.15.154:0/3783571241 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b4031920 con 0x7f12cc15ba50 2023-09-23T03:21:31.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.277+0000 7f12cb7fe640 1 -- 172.21.15.154:0/3783571241 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b4031cb0 con 0x7f12cc15ba50 2023-09-23T03:21:31.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.278+0000 7f12d15e2640 1 -- 172.21.15.154:0/3783571241 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12cc15ba50 msgr2=0x7f12cc06fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:31.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.278+0000 7f12d15e2640 1 --2- 172.21.15.154:0/3783571241 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12cc15ba50 0x7f12cc06fe90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f12b4009870 tx=0x7f12b402b860 comp rx=0 tx=0).stop 2023-09-23T03:21:31.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.278+0000 7f12d15e2640 1 -- 172.21.15.154:0/3783571241 shutdown_connections 2023-09-23T03:21:31.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.278+0000 7f12d15e2640 1 --2- 172.21.15.154:0/3783571241 >> v2:172.21.15.190:3300/0 conn(0x7f12cc15eef0 0x7f12cc0703d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:31.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.278+0000 7f12d15e2640 1 --2- 172.21.15.154:0/3783571241 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12cc15ba50 0x7f12cc06fe90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:31.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.278+0000 7f12d15e2640 1 -- 172.21.15.154:0/3783571241 >> 172.21.15.154:0/3783571241 conn(0x7f12cc0aff20 msgr2=0x7f12cc0b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:31.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.278+0000 7f12d15e2640 1 -- 172.21.15.154:0/3783571241 shutdown_connections 2023-09-23T03:21:31.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.278+0000 7f12d15e2640 1 -- 172.21.15.154:0/3783571241 wait complete. 2023-09-23T03:21:31.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.280+0000 7f12d15e2640 1 Processor -- start 2023-09-23T03:21:31.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.281+0000 7f12d15e2640 1 -- start start 2023-09-23T03:21:31.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.282+0000 7f12d15e2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12cc158480 0x7f12cc143a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:31.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.282+0000 7f12d15e2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12cc15ba50 0x7f12cc0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:31.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.282+0000 7f12d15e2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12cc15eef0 0x7f12cc0c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:31.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.282+0000 7f12d15e2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc1f5bc0 con 0x7f12cc15eef0 2023-09-23T03:21:31.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.282+0000 7f12d15e2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc143fa0 con 0x7f12cc15ba50 2023-09-23T03:21:31.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.282+0000 7f12d15e2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc0c9590 con 0x7f12cc158480 2023-09-23T03:21:31.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.282+0000 7f12d0de1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12cc15eef0 0x7f12cc0c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:31.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.282+0000 7f12d0de1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12cc15eef0 0x7f12cc0c8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49602/0 (socket says 172.21.15.154:49602) 2023-09-23T03:21:31.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.282+0000 7f12d0de1640 1 -- 172.21.15.154:0/1200065704 learned_addr learned my addr 172.21.15.154:0/1200065704 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:31.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.282+0000 7f12c3fff640 1 --2- 172.21.15.154:0/1200065704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12cc15ba50 0x7f12cc0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:31.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.282+0000 7f12cbfff640 1 --2- 172.21.15.154:0/1200065704 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12cc158480 0x7f12cc143a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:31.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.283+0000 7f12d0de1640 1 -- 172.21.15.154:0/1200065704 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12cc158480 msgr2=0x7f12cc143a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:31.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.283+0000 7f12d0de1640 1 --2- 172.21.15.154:0/1200065704 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12cc158480 0x7f12cc143a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:31.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.283+0000 7f12d0de1640 1 -- 172.21.15.154:0/1200065704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12cc15ba50 msgr2=0x7f12cc0c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:31.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.283+0000 7f12d0de1640 1 --2- 172.21.15.154:0/1200065704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12cc15ba50 0x7f12cc0c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:31.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.283+0000 7f12d0de1640 1 -- 172.21.15.154:0/1200065704 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b40094d0 con 0x7f12cc15eef0 2023-09-23T03:21:31.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.283+0000 7f12d0de1640 1 --2- 172.21.15.154:0/1200065704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12cc15eef0 0x7f12cc0c8f60 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f12c400efc0 tx=0x7f12c400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:31.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.283+0000 7f12c97fa640 1 -- 172.21.15.154:0/1200065704 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c4009ba0 con 0x7f12cc15eef0 2023-09-23T03:21:31.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.283+0000 7f12c97fa640 1 -- 172.21.15.154:0/1200065704 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c400f040 con 0x7f12cc15eef0 2023-09-23T03:21:31.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.283+0000 7f12c97fa640 1 -- 172.21.15.154:0/1200065704 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c40146c0 con 0x7f12cc15eef0 2023-09-23T03:21:31.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.283+0000 7f12d15e2640 1 -- 172.21.15.154:0/1200065704 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12cc1f5e10 con 0x7f12cc15eef0 2023-09-23T03:21:31.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.284+0000 7f12d15e2640 1 -- 172.21.15.154:0/1200065704 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12cc1f63d0 con 0x7f12cc15eef0 2023-09-23T03:21:31.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.285+0000 7f12d15e2640 1 -- 172.21.15.154:0/1200065704 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12cc04e920 con 0x7f12cc15eef0 2023-09-23T03:21:31.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.286+0000 7f12c97fa640 1 -- 172.21.15.154:0/1200065704 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f12c4014820 con 0x7f12cc15eef0 2023-09-23T03:21:31.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.287+0000 7f12c97fa640 1 --2- 172.21.15.154:0/1200065704 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f12ac06af30 0x7f12ac06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:31.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.287+0000 7f12c97fa640 1 -- 172.21.15.154:0/1200065704 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x7f12c408a560 con 0x7f12cc15eef0 2023-09-23T03:21:31.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.287+0000 7f12cbfff640 1 --2- 172.21.15.154:0/1200065704 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f12ac06af30 0x7f12ac06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:31.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.288+0000 7f12cbfff640 1 --2- 172.21.15.154:0/1200065704 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f12ac06af30 0x7f12ac06d3f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f12bc004ea0 tx=0x7f12bc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:31.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.292+0000 7f12c97fa640 1 -- 172.21.15.154:0/1200065704 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f12c40552b0 con 0x7f12cc15eef0 2023-09-23T03:21:31.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:31.522+0000 7f12d15e2640 1 -- 172.21.15.154:0/1200065704 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f12cc1f65c0 con 0x7f12cc15eef0 2023-09-23T03:21:32.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.077+0000 7f12c97fa640 1 -- 172.21.15.154:0/1200065704 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v47) v1 ==== 162+0+0 (secure 0 0 0) 0x7f12c40588d0 con 0x7f12cc15eef0 2023-09-23T03:21:32.079 INFO:teuthology.orchestra.run.smithi154.stderr:cephfs marked joinable; MDS may join as newly active. 2023-09-23T03:21:32.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.080+0000 7f12d15e2640 1 -- 172.21.15.154:0/1200065704 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f12ac06af30 msgr2=0x7f12ac06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:32.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.080+0000 7f12d15e2640 1 --2- 172.21.15.154:0/1200065704 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f12ac06af30 0x7f12ac06d3f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f12bc004ea0 tx=0x7f12bc009040 comp rx=0 tx=0).stop 2023-09-23T03:21:32.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.080+0000 7f12d15e2640 1 -- 172.21.15.154:0/1200065704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12cc15eef0 msgr2=0x7f12cc0c8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:32.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.080+0000 7f12d15e2640 1 --2- 172.21.15.154:0/1200065704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12cc15eef0 0x7f12cc0c8f60 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f12c400efc0 tx=0x7f12c400c490 comp rx=0 tx=0).stop 2023-09-23T03:21:32.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.081+0000 7f12d15e2640 1 -- 172.21.15.154:0/1200065704 shutdown_connections 2023-09-23T03:21:32.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.081+0000 7f12d15e2640 1 --2- 172.21.15.154:0/1200065704 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f12ac06af30 0x7f12ac06d3f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:32.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.081+0000 7f12d15e2640 1 --2- 172.21.15.154:0/1200065704 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f12cc15eef0 0x7f12cc0c8f60 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:32.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.081+0000 7f12d15e2640 1 --2- 172.21.15.154:0/1200065704 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f12cc15ba50 0x7f12cc0c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:32.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.081+0000 7f12d15e2640 1 --2- 172.21.15.154:0/1200065704 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f12cc158480 0x7f12cc143a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:32.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.081+0000 7f12d15e2640 1 -- 172.21.15.154:0/1200065704 >> 172.21.15.154:0/1200065704 conn(0x7f12cc0aff20 msgr2=0x7f12cc157750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:32.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.081+0000 7f12d15e2640 1 -- 172.21.15.154:0/1200065704 shutdown_connections 2023-09-23T03:21:32.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.081+0000 7f12d15e2640 1 -- 172.21.15.154:0/1200065704 wait complete. 2023-09-23T03:21:32.102 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:21:32.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.311+0000 7fe295361640 1 Processor -- start 2023-09-23T03:21:32.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.311+0000 7fe295361640 1 -- start start 2023-09-23T03:21:32.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.313+0000 7fe295361640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe2900b1b20 0x7fe2900ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:32.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.313+0000 7fe295361640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2900cee70 0x7fe2900ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:32.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.313+0000 7fe295361640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe29015ac30 con 0x7fe2900acce0 2023-09-23T03:21:32.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.313+0000 7fe295361640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2900b2e90 con 0x7fe2900b1b20 2023-09-23T03:21:32.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.313+0000 7fe295361640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2900cf320 con 0x7fe2900cee70 2023-09-23T03:21:32.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.314+0000 7fe28effd640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fe2900acce0 0x7fe29015ed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.154:60248/0 (socket says 172.21.15.154:60248) 2023-09-23T03:21:32.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.314+0000 7fe28effd640 1 -- 172.21.15.154:0/3976705660 learned_addr learned my addr 172.21.15.154:0/3976705660 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:21:32.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.314+0000 7fe28f7fe640 1 --2- 172.21.15.154:0/3976705660 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2900cee70 0x7fe2900ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:32.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.314+0000 7fe287fff640 1 --2- 172.21.15.154:0/3976705660 >> v2:172.21.15.190:3300/0 conn(0x7fe2900b1b20 0x7fe2900ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:32.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.314+0000 7fe28e7fc640 1 -- 172.21.15.154:0/3976705660 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 294279933 0 0) 0x7fe280001160 con 0x7fe2900acce0 2023-09-23T03:21:32.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.314+0000 7fe28e7fc640 1 -- 172.21.15.154:0/3976705660 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe27c003520 con 0x7fe2900acce0 2023-09-23T03:21:32.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.314+0000 7fe287fff640 1 -- 172.21.15.154:0/3976705660 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2900cee70 msgr2=0x7fe2900ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:32.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.315+0000 7fe287fff640 1 --2- 172.21.15.154:0/3976705660 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2900cee70 0x7fe2900ab140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:32.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.315+0000 7fe287fff640 1 -- 172.21.15.154:0/3976705660 >> v1:172.21.15.190:6789/0 conn(0x7fe2900acce0 legacy=0x7fe29015ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:32.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.315+0000 7fe287fff640 1 -- 172.21.15.154:0/3976705660 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2900b1f70 con 0x7fe2900b1b20 2023-09-23T03:21:32.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.315+0000 7fe28f7fe640 1 --2- 172.21.15.154:0/3976705660 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2900cee70 0x7fe2900ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:21:32.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.316+0000 7fe287fff640 1 --2- 172.21.15.154:0/3976705660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2900b1b20 0x7fe2900ce930 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe2780097a0 tx=0x7fe27802b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c31bd74a253a7203 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:32.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.317+0000 7fe28e7fc640 1 -- 172.21.15.154:0/3976705660 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe278031780 con 0x7fe2900b1b20 2023-09-23T03:21:32.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.317+0000 7fe28e7fc640 1 -- 172.21.15.154:0/3976705660 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2780318e0 con 0x7fe2900b1b20 2023-09-23T03:21:32.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.317+0000 7fe28e7fc640 1 -- 172.21.15.154:0/3976705660 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe278031c70 con 0x7fe2900b1b20 2023-09-23T03:21:32.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.317+0000 7fe295361640 1 -- 172.21.15.154:0/3976705660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2900b1b20 msgr2=0x7fe2900ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:32.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.317+0000 7fe295361640 1 --2- 172.21.15.154:0/3976705660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2900b1b20 0x7fe2900ce930 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe2780097a0 tx=0x7fe27802b820 comp rx=0 tx=0).stop 2023-09-23T03:21:32.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.318+0000 7fe295361640 1 -- 172.21.15.154:0/3976705660 shutdown_connections 2023-09-23T03:21:32.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.318+0000 7fe295361640 1 --2- 172.21.15.154:0/3976705660 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2900cee70 0x7fe2900ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:32.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.318+0000 7fe295361640 1 --2- 172.21.15.154:0/3976705660 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2900b1b20 0x7fe2900ce930 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:32.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.318+0000 7fe295361640 1 -- 172.21.15.154:0/3976705660 >> 172.21.15.154:0/3976705660 conn(0x7fe29006a0f0 msgr2=0x7fe29006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:32.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.318+0000 7fe295361640 1 -- 172.21.15.154:0/3976705660 shutdown_connections 2023-09-23T03:21:32.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.318+0000 7fe295361640 1 -- 172.21.15.154:0/3976705660 wait complete. 2023-09-23T03:21:32.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.319+0000 7fe295361640 1 Processor -- start 2023-09-23T03:21:32.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.320+0000 7fe295361640 1 -- start start 2023-09-23T03:21:32.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.321+0000 7fe295361640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2900acce0 0x7fe290159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:32.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.321+0000 7fe295361640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2900b1b20 0x7fe2900c2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:32.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.321+0000 7fe295361640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2900cee70 0x7fe2900ae740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:32.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.321+0000 7fe295361640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29015c010 con 0x7fe2900cee70 2023-09-23T03:21:32.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.321+0000 7fe295361640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29015b390 con 0x7fe2900acce0 2023-09-23T03:21:32.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.321+0000 7fe295361640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe29015b670 con 0x7fe2900b1b20 2023-09-23T03:21:32.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.321+0000 7fe28f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2900cee70 0x7fe2900ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:32.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.321+0000 7fe28f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2900cee70 0x7fe2900ae740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49614/0 (socket says 172.21.15.154:49614) 2023-09-23T03:21:32.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.321+0000 7fe28f7fe640 1 -- 172.21.15.154:0/2598186798 learned_addr learned my addr 172.21.15.154:0/2598186798 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:32.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.321+0000 7fe28effd640 1 --2- 172.21.15.154:0/2598186798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2900acce0 0x7fe290159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:32.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.321+0000 7fe287fff640 1 --2- 172.21.15.154:0/2598186798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2900b1b20 0x7fe2900c2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:32.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.322+0000 7fe28f7fe640 1 -- 172.21.15.154:0/2598186798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2900b1b20 msgr2=0x7fe2900c2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:32.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.322+0000 7fe28f7fe640 1 --2- 172.21.15.154:0/2598186798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2900b1b20 0x7fe2900c2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:32.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.322+0000 7fe28f7fe640 1 -- 172.21.15.154:0/2598186798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2900acce0 msgr2=0x7fe290159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:32.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.322+0000 7fe28f7fe640 1 --2- 172.21.15.154:0/2598186798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2900acce0 0x7fe290159d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:32.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.322+0000 7fe28f7fe640 1 -- 172.21.15.154:0/2598186798 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe288009a70 con 0x7fe2900cee70 2023-09-23T03:21:32.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.322+0000 7fe287fff640 1 --2- 172.21.15.154:0/2598186798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2900b1b20 0x7fe2900c2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:21:32.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.322+0000 7fe28f7fe640 1 --2- 172.21.15.154:0/2598186798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2900cee70 0x7fe2900ae740 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fe28800efc0 tx=0x7fe28800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:32.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.322+0000 7fe2877fe640 1 -- 172.21.15.154:0/2598186798 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe288009cd0 con 0x7fe2900cee70 2023-09-23T03:21:32.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.322+0000 7fe2877fe640 1 -- 172.21.15.154:0/2598186798 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28800f040 con 0x7fe2900cee70 2023-09-23T03:21:32.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.322+0000 7fe295361640 1 -- 172.21.15.154:0/2598186798 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe278009400 con 0x7fe2900cee70 2023-09-23T03:21:32.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.322+0000 7fe2877fe640 1 -- 172.21.15.154:0/2598186798 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe288003990 con 0x7fe2900cee70 2023-09-23T03:21:32.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.322+0000 7fe295361640 1 -- 172.21.15.154:0/2598186798 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe29015bc30 con 0x7fe2900cee70 2023-09-23T03:21:32.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.323+0000 7fe295361640 1 -- 172.21.15.154:0/2598186798 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe29015c150 con 0x7fe2900cee70 2023-09-23T03:21:32.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.324+0000 7fe2877fe640 1 -- 172.21.15.154:0/2598186798 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fe288014480 con 0x7fe2900cee70 2023-09-23T03:21:32.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.325+0000 7fe2877fe640 1 --2- 172.21.15.154:0/2598186798 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe27006af30 0x7fe27006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:32.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.325+0000 7fe2877fe640 1 -- 172.21.15.154:0/2598186798 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x7fe28808a6b0 con 0x7fe2900cee70 2023-09-23T03:21:32.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.326+0000 7fe28effd640 1 --2- 172.21.15.154:0/2598186798 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe27006af30 0x7fe27006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:32.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.326+0000 7fe28effd640 1 --2- 172.21.15.154:0/2598186798 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe27006af30 0x7fe27006d3f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe280007fd0 tx=0x7fe280009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:32.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.330+0000 7fe2877fe640 1 -- 172.21.15.154:0/2598186798 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fe29015c150 con 0x7fe2900cee70 2023-09-23T03:21:32.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.563+0000 7fe295361640 1 -- 172.21.15.154:0/2598186798 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe29015c150 con 0x7fe2900cee70 2023-09-23T03:21:32.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.564+0000 7fe2877fe640 1 -- 172.21.15.154:0/2598186798 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4920 (secure 0 0 0) 0x7fe29015c150 con 0x7fe2900cee70 2023-09-23T03:21:32.566 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:32.566 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":47,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5027,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2517324433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2517324433},{"type":"v1","addr":"172.21.15.190:6833","nonce":2517324433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":5033,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4098915995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4098915995},{"type":"v1","addr":"172.21.15.190:6835","nonce":4098915995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6309,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/3034894009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":3034894009},{"type":"v1","addr":"172.21.15.154:6836","nonce":3034894009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6639,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/4129414148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":4129414148},{"type":"v1","addr":"172.21.15.154:6837","nonce":4129414148}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:21:30.160245+0000","modified":"2023-09-23T03:21:31.524972+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[9,10],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-09-23T03:21:32.566 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 47 2023-09-23T03:21:32.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.567+0000 7fe295361640 1 -- 172.21.15.154:0/2598186798 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe27006af30 msgr2=0x7fe27006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:32.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.567+0000 7fe295361640 1 --2- 172.21.15.154:0/2598186798 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe27006af30 0x7fe27006d3f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe280007fd0 tx=0x7fe280009040 comp rx=0 tx=0).stop 2023-09-23T03:21:32.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.567+0000 7fe295361640 1 -- 172.21.15.154:0/2598186798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2900cee70 msgr2=0x7fe2900ae740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:32.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.567+0000 7fe295361640 1 --2- 172.21.15.154:0/2598186798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2900cee70 0x7fe2900ae740 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fe28800efc0 tx=0x7fe28800c490 comp rx=0 tx=0).stop 2023-09-23T03:21:32.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.567+0000 7fe295361640 1 -- 172.21.15.154:0/2598186798 shutdown_connections 2023-09-23T03:21:32.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.567+0000 7fe295361640 1 --2- 172.21.15.154:0/2598186798 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe27006af30 0x7fe27006d3f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:32.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.567+0000 7fe295361640 1 --2- 172.21.15.154:0/2598186798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe2900cee70 0x7fe2900ae740 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:32.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.567+0000 7fe295361640 1 --2- 172.21.15.154:0/2598186798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe2900b1b20 0x7fe2900c2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:32.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.567+0000 7fe295361640 1 --2- 172.21.15.154:0/2598186798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe2900acce0 0x7fe290159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:32.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.567+0000 7fe295361640 1 -- 172.21.15.154:0/2598186798 >> 172.21.15.154:0/2598186798 conn(0x7fe29006a0f0 msgr2=0x7fe2900cb550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:32.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.567+0000 7fe295361640 1 -- 172.21.15.154:0/2598186798 shutdown_connections 2023-09-23T03:21:32.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.567+0000 7fe295361640 1 -- 172.21.15.154:0/2598186798 wait complete. 2023-09-23T03:21:32.587 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-09-23T03:21:32.587 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:21:32.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.794+0000 7fc95fd16640 1 Processor -- start 2023-09-23T03:21:32.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.794+0000 7fc95fd16640 1 -- start start 2023-09-23T03:21:32.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.795+0000 7fc95fd16640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc9580ada30 0x7fc9580ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:32.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.795+0000 7fc95fd16640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc95806ceb0 con 0x7fc9580b2fe0 2023-09-23T03:21:32.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.795+0000 7fc95fd16640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc958159c90 con 0x7fc9580cf130 2023-09-23T03:21:32.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.795+0000 7fc95fd16640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9580ade10 con 0x7fc9580ada30 2023-09-23T03:21:32.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.796+0000 7fc95d2b1640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fc9580b2fe0 0x7fc9580cea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:39220/0 (socket says 172.21.15.154:39220) 2023-09-23T03:21:32.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.796+0000 7fc95d2b1640 1 -- 172.21.15.154:0/2378084858 learned_addr learned my addr 172.21.15.154:0/2378084858 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:21:32.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.796+0000 7fc95dab2640 1 --2- 172.21.15.154:0/2378084858 >> v2:172.21.15.190:3300/0 conn(0x7fc9580ada30 0x7fc9580ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:32.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.796+0000 7fc95cab0640 1 -- 172.21.15.154:0/2378084858 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3882825795 0 0) 0x7fc948001110 con 0x7fc9580b2fe0 2023-09-23T03:21:32.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.796+0000 7fc95cab0640 1 -- 172.21.15.154:0/2378084858 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc944003520 con 0x7fc9580b2fe0 2023-09-23T03:21:32.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.796+0000 7fc95cab0640 1 -- 172.21.15.154:0/2378084858 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2634484449 0 0) 0x7fc944003520 con 0x7fc9580b2fe0 2023-09-23T03:21:32.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.796+0000 7fc95cab0640 1 -- 172.21.15.154:0/2378084858 >> v2:172.21.15.190:3300/0 conn(0x7fc9580ada30 msgr2=0x7fc9580ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:32.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.796+0000 7fc95cab0640 1 --2- 172.21.15.154:0/2378084858 >> v2:172.21.15.190:3300/0 conn(0x7fc9580ada30 0x7fc9580ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:32.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.796+0000 7fc95cab0640 1 -- 172.21.15.154:0/2378084858 >> v1:172.21.15.190:6789/0 conn(0x7fc9580cf130 legacy=0x7fc95815d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:32.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.796+0000 7fc95cab0640 1 -- 172.21.15.154:0/2378084858 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc95806c110 con 0x7fc9580b2fe0 2023-09-23T03:21:32.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.796+0000 7fc95cab0640 1 -- 172.21.15.154:0/2378084858 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fc948001800 con 0x7fc9580b2fe0 2023-09-23T03:21:32.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.796+0000 7fc95cab0640 1 -- 172.21.15.154:0/2378084858 >> v1:172.21.15.154:6789/0 conn(0x7fc9580b2fe0 legacy=0x7fc9580cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:32.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.797+0000 7fc95cab0640 1 --2- 172.21.15.154:0/2378084858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc944003970 0x7fc944003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:32.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.797+0000 7fc95cab0640 1 --2- 172.21.15.154:0/2378084858 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9440047c0 0x7fc944004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:32.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.797+0000 7fc95cab0640 1 --2- 172.21.15.154:0/2378084858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc944005180 0x7fc94400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:32.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.797+0000 7fc95cab0640 1 -- 172.21.15.154:0/2378084858 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9580ade10 con 0x7fc944005180 2023-09-23T03:21:32.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.797+0000 7fc95cab0640 1 -- 172.21.15.154:0/2378084858 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc94400c020 con 0x7fc944003970 2023-09-23T03:21:32.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.797+0000 7fc95cab0640 1 -- 172.21.15.154:0/2378084858 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc94400c1e0 con 0x7fc9440047c0 2023-09-23T03:21:32.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.797+0000 7fc95e2b3640 1 --2- 172.21.15.154:0/2378084858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc944005180 0x7fc94400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:32.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.797+0000 7fc95d2b1640 1 --2- 172.21.15.154:0/2378084858 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9440047c0 0x7fc944004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:32.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.797+0000 7fc95dab2640 1 --2- 172.21.15.154:0/2378084858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc944003970 0x7fc944003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:32.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.797+0000 7fc95e2b3640 1 -- 172.21.15.154:0/2378084858 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9440047c0 msgr2=0x7fc944004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:32.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.797+0000 7fc95e2b3640 1 --2- 172.21.15.154:0/2378084858 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc9440047c0 0x7fc944004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:32.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.797+0000 7fc95e2b3640 1 -- 172.21.15.154:0/2378084858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc944003970 msgr2=0x7fc944003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:32.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.797+0000 7fc95e2b3640 1 --2- 172.21.15.154:0/2378084858 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc944003970 0x7fc944003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:32.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.797+0000 7fc95e2b3640 1 -- 172.21.15.154:0/2378084858 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc944003520 con 0x7fc944005180 2023-09-23T03:21:32.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.798+0000 7fc95e2b3640 1 -- 172.21.15.154:0/2378084858 reap_dead start 2023-09-23T03:21:32.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.798+0000 7fc95e2b3640 1 --2- 172.21.15.154:0/2378084858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc944005180 0x7fc94400b980 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fc950001e20 tx=0x7fc9440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cfef42ed8a701fb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:32.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.799+0000 7fc95cab0640 1 -- 172.21.15.154:0/2378084858 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc950003b70 con 0x7fc944005180 2023-09-23T03:21:32.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.799+0000 7fc95cab0640 1 -- 172.21.15.154:0/2378084858 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc950003cd0 con 0x7fc944005180 2023-09-23T03:21:32.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.799+0000 7fc95cab0640 1 -- 172.21.15.154:0/2378084858 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9500052f0 con 0x7fc944005180 2023-09-23T03:21:32.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.799+0000 7fc95fd16640 1 -- 172.21.15.154:0/2378084858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc944005180 msgr2=0x7fc94400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:32.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.799+0000 7fc95fd16640 1 --2- 172.21.15.154:0/2378084858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc944005180 0x7fc94400b980 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fc950001e20 tx=0x7fc9440057f0 comp rx=0 tx=0).stop 2023-09-23T03:21:32.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.799+0000 7fc95fd16640 1 -- 172.21.15.154:0/2378084858 shutdown_connections 2023-09-23T03:21:32.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.799+0000 7fc95fd16640 1 --2- 172.21.15.154:0/2378084858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc944005180 0x7fc94400b980 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:32.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.799+0000 7fc95fd16640 1 -- 172.21.15.154:0/2378084858 >> 172.21.15.154:0/2378084858 conn(0x7fc95806a0f0 msgr2=0x7fc95806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:32.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.799+0000 7fc95fd16640 1 -- 172.21.15.154:0/2378084858 shutdown_connections 2023-09-23T03:21:32.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.799+0000 7fc95fd16640 1 -- 172.21.15.154:0/2378084858 wait complete. 2023-09-23T03:21:32.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.801+0000 7fc95fd16640 1 Processor -- start 2023-09-23T03:21:32.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.801+0000 7fc95fd16640 1 -- start start 2023-09-23T03:21:32.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.802+0000 7fc95fd16640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc944005180 0x7fc958158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:32.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.802+0000 7fc95fd16640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9580982c0 0x7fc9581f1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:32.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.802+0000 7fc95fd16640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc95809ab80 0x7fc95809b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:32.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.802+0000 7fc95fd16640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc958099d70 con 0x7fc9580982c0 2023-09-23T03:21:32.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.802+0000 7fc95fd16640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95809b540 con 0x7fc95809ab80 2023-09-23T03:21:32.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.802+0000 7fc95fd16640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9580998e0 con 0x7fc944005180 2023-09-23T03:21:32.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.802+0000 7fc95d2b1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9580982c0 0x7fc9581f1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:32.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.803+0000 7fc95d2b1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9580982c0 0x7fc9581f1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49630/0 (socket says 172.21.15.154:49630) 2023-09-23T03:21:32.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.803+0000 7fc95d2b1640 1 -- 172.21.15.154:0/1723544412 learned_addr learned my addr 172.21.15.154:0/1723544412 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:32.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.803+0000 7fc95dab2640 1 --2- 172.21.15.154:0/1723544412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc944005180 0x7fc958158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:32.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.803+0000 7fc95d2b1640 1 -- 172.21.15.154:0/1723544412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc944005180 msgr2=0x7fc958158990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:32.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.803+0000 7fc95d2b1640 1 --2- 172.21.15.154:0/1723544412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc944005180 0x7fc958158990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:32.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.803+0000 7fc95e2b3640 1 --2- 172.21.15.154:0/1723544412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc95809ab80 0x7fc95809b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:32.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.803+0000 7fc95d2b1640 1 -- 172.21.15.154:0/1723544412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc95809ab80 msgr2=0x7fc95809b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:32.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.803+0000 7fc95d2b1640 1 --2- 172.21.15.154:0/1723544412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc95809ab80 0x7fc95809b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:32.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.803+0000 7fc95d2b1640 1 -- 172.21.15.154:0/1723544412 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9581f1ea0 con 0x7fc9580982c0 2023-09-23T03:21:32.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.803+0000 7fc95d2b1640 1 --2- 172.21.15.154:0/1723544412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9580982c0 0x7fc9581f1960 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fc94800e780 tx=0x7fc94800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:32.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.803+0000 7fc94e7fc640 1 -- 172.21.15.154:0/1723544412 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94800ee30 con 0x7fc9580982c0 2023-09-23T03:21:32.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.804+0000 7fc94e7fc640 1 -- 172.21.15.154:0/1723544412 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94800c8d0 con 0x7fc9580982c0 2023-09-23T03:21:32.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.804+0000 7fc94e7fc640 1 -- 172.21.15.154:0/1723544412 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94800cc80 con 0x7fc9580982c0 2023-09-23T03:21:32.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.804+0000 7fc95fd16640 1 -- 172.21.15.154:0/1723544412 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc958098cd0 con 0x7fc9580982c0 2023-09-23T03:21:32.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.804+0000 7fc95fd16640 1 -- 172.21.15.154:0/1723544412 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc958099290 con 0x7fc9580982c0 2023-09-23T03:21:32.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.805+0000 7fc95fd16640 1 -- 172.21.15.154:0/1723544412 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc958073eb0 con 0x7fc9580982c0 2023-09-23T03:21:32.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.806+0000 7fc94e7fc640 1 -- 172.21.15.154:0/1723544412 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fc948013420 con 0x7fc9580982c0 2023-09-23T03:21:32.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.807+0000 7fc94e7fc640 1 --2- 172.21.15.154:0/1723544412 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc93c06af60 0x7fc93c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:32.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.807+0000 7fc94e7fc640 1 -- 172.21.15.154:0/1723544412 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x7fc9480895e0 con 0x7fc9580982c0 2023-09-23T03:21:32.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.807+0000 7fc95dab2640 1 --2- 172.21.15.154:0/1723544412 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc93c06af60 0x7fc93c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:32.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.808+0000 7fc95dab2640 1 --2- 172.21.15.154:0/1723544412 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc93c06af60 0x7fc93c06d420 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc9540098c0 tx=0x7fc954008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:32.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:32.811+0000 7fc94e7fc640 1 -- 172.21.15.154:0/1723544412 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fc948057d20 con 0x7fc9580982c0 2023-09-23T03:21:33.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.019+0000 7fc95fd16640 1 -- 172.21.15.154:0/1723544412 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc95806ce90 con 0x7fc9580982c0 2023-09-23T03:21:33.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.020+0000 7fc94e7fc640 1 -- 172.21.15.154:0/1723544412 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4920 (secure 0 0 0) 0x7fc948001660 con 0x7fc9580982c0 2023-09-23T03:21:33.022 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:33.022 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":47,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5027,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2517324433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2517324433},{"type":"v1","addr":"172.21.15.190:6833","nonce":2517324433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":5033,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4098915995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4098915995},{"type":"v1","addr":"172.21.15.190:6835","nonce":4098915995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6309,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/3034894009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":3034894009},{"type":"v1","addr":"172.21.15.154:6836","nonce":3034894009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6639,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/4129414148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":4129414148},{"type":"v1","addr":"172.21.15.154:6837","nonce":4129414148}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:21:30.160245+0000","modified":"2023-09-23T03:21:31.524972+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[9,10],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-09-23T03:21:33.022 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 47 2023-09-23T03:21:33.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.023+0000 7fc95fd16640 1 -- 172.21.15.154:0/1723544412 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc93c06af60 msgr2=0x7fc93c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:33.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.023+0000 7fc95fd16640 1 --2- 172.21.15.154:0/1723544412 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc93c06af60 0x7fc93c06d420 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc9540098c0 tx=0x7fc954008040 comp rx=0 tx=0).stop 2023-09-23T03:21:33.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.023+0000 7fc95fd16640 1 -- 172.21.15.154:0/1723544412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9580982c0 msgr2=0x7fc9581f1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:33.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.023+0000 7fc95fd16640 1 --2- 172.21.15.154:0/1723544412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9580982c0 0x7fc9581f1960 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fc94800e780 tx=0x7fc94800ec40 comp rx=0 tx=0).stop 2023-09-23T03:21:33.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.023+0000 7fc95fd16640 1 -- 172.21.15.154:0/1723544412 shutdown_connections 2023-09-23T03:21:33.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.023+0000 7fc95fd16640 1 --2- 172.21.15.154:0/1723544412 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc93c06af60 0x7fc93c06d420 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:33.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.023+0000 7fc95fd16640 1 --2- 172.21.15.154:0/1723544412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc95809ab80 0x7fc95809b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:33.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.023+0000 7fc95fd16640 1 --2- 172.21.15.154:0/1723544412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc9580982c0 0x7fc9581f1960 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:33.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.023+0000 7fc95fd16640 1 --2- 172.21.15.154:0/1723544412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc944005180 0x7fc958158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:33.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.023+0000 7fc95fd16640 1 -- 172.21.15.154:0/1723544412 >> 172.21.15.154:0/1723544412 conn(0x7fc95806a0f0 msgr2=0x7fc9580ae480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:33.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.023+0000 7fc95fd16640 1 -- 172.21.15.154:0/1723544412 shutdown_connections 2023-09-23T03:21:33.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.023+0000 7fc95fd16640 1 -- 172.21.15.154:0/1723544412 wait complete. 2023-09-23T03:21:33.043 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:21:33.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.250+0000 7f2b5d4b4640 1 Processor -- start 2023-09-23T03:21:33.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.251+0000 7f2b5d4b4640 1 -- start start 2023-09-23T03:21:33.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.252+0000 7f2b5d4b4640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2b580acce0 0x7f2b5815ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:33.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.252+0000 7f2b5d4b4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b580b1b20 0x7f2b580ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:33.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.252+0000 7f2b5d4b4640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2b580cee70 0x7f2b5806cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:33.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.252+0000 7f2b5d4b4640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b580b2e90 con 0x7f2b580acce0 2023-09-23T03:21:33.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.252+0000 7f2b5d4b4640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b580cf320 con 0x7f2b580cee70 2023-09-23T03:21:33.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.252+0000 7f2b5d4b4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5815ac30 con 0x7f2b580b1b20 2023-09-23T03:21:33.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.253+0000 7f2b56ffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2b580acce0 0x7f2b5815ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:33.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.253+0000 7f2b56ffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2b580acce0 0x7f2b5815ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49646/0 (socket says 172.21.15.154:49646) 2023-09-23T03:21:33.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.253+0000 7f2b56ffd640 1 -- 172.21.15.154:0/3775174795 learned_addr learned my addr 172.21.15.154:0/3775174795 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:33.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.253+0000 7f2b577fe640 1 --2- 172.21.15.154:0/3775174795 >> v2:172.21.15.190:3300/0 conn(0x7f2b580cee70 0x7f2b5806cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:33.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.253+0000 7f2b4e7fc640 1 --2- 172.21.15.154:0/3775174795 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b580b1b20 0x7f2b580ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:33.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.253+0000 7f2b56ffd640 1 -- 172.21.15.154:0/3775174795 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b580b1b20 msgr2=0x7f2b580ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:33.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.253+0000 7f2b56ffd640 1 --2- 172.21.15.154:0/3775174795 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b580b1b20 0x7f2b580ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:33.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.253+0000 7f2b56ffd640 1 -- 172.21.15.154:0/3775174795 >> v2:172.21.15.190:3300/0 conn(0x7f2b580cee70 msgr2=0x7f2b5806cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:33.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.253+0000 7f2b56ffd640 1 --2- 172.21.15.154:0/3775174795 >> v2:172.21.15.190:3300/0 conn(0x7f2b580cee70 0x7f2b5806cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:33.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.253+0000 7f2b56ffd640 1 -- 172.21.15.154:0/3775174795 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b580b1f70 con 0x7f2b580acce0 2023-09-23T03:21:33.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.255+0000 7f2b56ffd640 1 --2- 172.21.15.154:0/3775174795 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b580acce0 0x7f2b5815ed60 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f2b580c9150 tx=0x7f2b4802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5119571b690fdd1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:33.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.255+0000 7f2b567fc640 1 -- 172.21.15.154:0/3775174795 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4802e070 con 0x7f2b580acce0 2023-09-23T03:21:33.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.255+0000 7f2b567fc640 1 -- 172.21.15.154:0/3775174795 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b48004030 con 0x7f2b580acce0 2023-09-23T03:21:33.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.255+0000 7f2b567fc640 1 -- 172.21.15.154:0/3775174795 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b48004420 con 0x7f2b580acce0 2023-09-23T03:21:33.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.256+0000 7f2b5d4b4640 1 -- 172.21.15.154:0/3775174795 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b580acce0 msgr2=0x7f2b5815ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:33.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.256+0000 7f2b5d4b4640 1 --2- 172.21.15.154:0/3775174795 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b580acce0 0x7f2b5815ed60 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f2b580c9150 tx=0x7f2b4802b840 comp rx=0 tx=0).stop 2023-09-23T03:21:33.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.256+0000 7f2b5d4b4640 1 -- 172.21.15.154:0/3775174795 shutdown_connections 2023-09-23T03:21:33.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.256+0000 7f2b5d4b4640 1 --2- 172.21.15.154:0/3775174795 >> v2:172.21.15.190:3300/0 conn(0x7f2b580cee70 0x7f2b5806cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:33.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.256+0000 7f2b5d4b4640 1 --2- 172.21.15.154:0/3775174795 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b580b1b20 0x7f2b580ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:33.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.256+0000 7f2b5d4b4640 1 --2- 172.21.15.154:0/3775174795 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b580acce0 0x7f2b5815ed60 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:33.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.256+0000 7f2b5d4b4640 1 -- 172.21.15.154:0/3775174795 >> 172.21.15.154:0/3775174795 conn(0x7f2b5806a0f0 msgr2=0x7f2b5806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:33.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.256+0000 7f2b5d4b4640 1 -- 172.21.15.154:0/3775174795 shutdown_connections 2023-09-23T03:21:33.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.256+0000 7f2b5d4b4640 1 -- 172.21.15.154:0/3775174795 wait complete. 2023-09-23T03:21:33.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.258+0000 7f2b5d4b4640 1 Processor -- start 2023-09-23T03:21:33.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.258+0000 7f2b5d4b4640 1 -- start start 2023-09-23T03:21:33.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.259+0000 7f2b5d4b4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b580acce0 0x7f2b5815d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:33.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.259+0000 7f2b5d4b4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b580b1b20 0x7f2b580c4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:33.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.259+0000 7f2b5d4b4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b580cee70 0x7f2b580cd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:33.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.259+0000 7f2b5d4b4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b58097f00 con 0x7f2b580acce0 2023-09-23T03:21:33.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.259+0000 7f2b5d4b4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b58154c10 con 0x7f2b580cee70 2023-09-23T03:21:33.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.259+0000 7f2b5d4b4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b580c2370 con 0x7f2b580b1b20 2023-09-23T03:21:33.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.259+0000 7f2b56ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b580acce0 0x7f2b5815d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:33.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.259+0000 7f2b56ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b580acce0 0x7f2b5815d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49656/0 (socket says 172.21.15.154:49656) 2023-09-23T03:21:33.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.259+0000 7f2b56ffd640 1 -- 172.21.15.154:0/2735251939 learned_addr learned my addr 172.21.15.154:0/2735251939 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:33.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.260+0000 7f2b577fe640 1 --2- 172.21.15.154:0/2735251939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b580cee70 0x7f2b580cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:33.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.260+0000 7f2b4e7fc640 1 --2- 172.21.15.154:0/2735251939 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b580b1b20 0x7f2b580c4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:33.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.260+0000 7f2b56ffd640 1 -- 172.21.15.154:0/2735251939 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b580b1b20 msgr2=0x7f2b580c4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:33.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.260+0000 7f2b56ffd640 1 --2- 172.21.15.154:0/2735251939 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b580b1b20 0x7f2b580c4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:33.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.260+0000 7f2b56ffd640 1 -- 172.21.15.154:0/2735251939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b580cee70 msgr2=0x7f2b580cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:33.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.260+0000 7f2b56ffd640 1 --2- 172.21.15.154:0/2735251939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b580cee70 0x7f2b580cd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:33.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.260+0000 7f2b56ffd640 1 -- 172.21.15.154:0/2735251939 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b48009460 con 0x7f2b580acce0 2023-09-23T03:21:33.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.260+0000 7f2b577fe640 1 --2- 172.21.15.154:0/2735251939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b580cee70 0x7f2b580cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:33.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.260+0000 7f2b56ffd640 1 --2- 172.21.15.154:0/2735251939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b580acce0 0x7f2b5815d250 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f2b480090b0 tx=0x7f2b480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:33.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.260+0000 7f2b4ffff640 1 -- 172.21.15.154:0/2735251939 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b48003b20 con 0x7f2b580acce0 2023-09-23T03:21:33.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.260+0000 7f2b4ffff640 1 -- 172.21.15.154:0/2735251939 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b48003c80 con 0x7f2b580acce0 2023-09-23T03:21:33.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.260+0000 7f2b4ffff640 1 -- 172.21.15.154:0/2735251939 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b48006040 con 0x7f2b580acce0 2023-09-23T03:21:33.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.260+0000 7f2b5d4b4640 1 -- 172.21.15.154:0/2735251939 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b580c2040 con 0x7f2b580acce0 2023-09-23T03:21:33.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.261+0000 7f2b5d4b4640 1 -- 172.21.15.154:0/2735251939 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b58155140 con 0x7f2b580acce0 2023-09-23T03:21:33.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.262+0000 7f2b5d4b4640 1 -- 172.21.15.154:0/2735251939 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b58073eb0 con 0x7f2b580acce0 2023-09-23T03:21:33.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.263+0000 7f2b4ffff640 1 -- 172.21.15.154:0/2735251939 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f2b48039050 con 0x7f2b580acce0 2023-09-23T03:21:33.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.263+0000 7f2b4ffff640 1 --2- 172.21.15.154:0/2735251939 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2b3806af30 0x7f2b3806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:33.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.264+0000 7f2b4ffff640 1 -- 172.21.15.154:0/2735251939 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x7f2b480ab5c0 con 0x7f2b580acce0 2023-09-23T03:21:33.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.264+0000 7f2b4e7fc640 1 --2- 172.21.15.154:0/2735251939 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2b3806af30 0x7f2b3806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:33.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.264+0000 7f2b4e7fc640 1 --2- 172.21.15.154:0/2735251939 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2b3806af30 0x7f2b3806d3f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2b581541f0 tx=0x7f2b40008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:33.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.268+0000 7f2b4ffff640 1 -- 172.21.15.154:0/2735251939 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f2b48076270 con 0x7f2b580acce0 2023-09-23T03:21:33.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.495+0000 7f2b5d4b4640 1 -- 172.21.15.154:0/2735251939 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b5806c930 con 0x7f2b580acce0 2023-09-23T03:21:33.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.496+0000 7f2b4ffff640 1 -- 172.21.15.154:0/2735251939 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f2b48079890 con 0x7f2b580acce0 2023-09-23T03:21:33.498 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:33.498 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":47,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5027,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2517324433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2517324433},{"type":"v1","addr":"172.21.15.190:6833","nonce":2517324433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":5033,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4098915995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4098915995},{"type":"v1","addr":"172.21.15.190:6835","nonce":4098915995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6309,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/3034894009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":3034894009},{"type":"v1","addr":"172.21.15.154:6836","nonce":3034894009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6639,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/4129414148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":4129414148},{"type":"v1","addr":"172.21.15.154:6837","nonce":4129414148}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:21:30.160245+0000","modified":"2023-09-23T03:21:31.524972+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[9,10],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-09-23T03:21:33.498 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 47 2023-09-23T03:21:33.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.499+0000 7f2b5d4b4640 1 -- 172.21.15.154:0/2735251939 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2b3806af30 msgr2=0x7f2b3806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:33.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.499+0000 7f2b5d4b4640 1 --2- 172.21.15.154:0/2735251939 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2b3806af30 0x7f2b3806d3f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2b581541f0 tx=0x7f2b40008040 comp rx=0 tx=0).stop 2023-09-23T03:21:33.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.499+0000 7f2b5d4b4640 1 -- 172.21.15.154:0/2735251939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b580acce0 msgr2=0x7f2b5815d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:33.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.499+0000 7f2b5d4b4640 1 --2- 172.21.15.154:0/2735251939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b580acce0 0x7f2b5815d250 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f2b480090b0 tx=0x7f2b480046c0 comp rx=0 tx=0).stop 2023-09-23T03:21:33.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.499+0000 7f2b5d4b4640 1 -- 172.21.15.154:0/2735251939 shutdown_connections 2023-09-23T03:21:33.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.499+0000 7f2b5d4b4640 1 --2- 172.21.15.154:0/2735251939 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2b3806af30 0x7f2b3806d3f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:33.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.499+0000 7f2b5d4b4640 1 --2- 172.21.15.154:0/2735251939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b580cee70 0x7f2b580cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:33.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.499+0000 7f2b5d4b4640 1 --2- 172.21.15.154:0/2735251939 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b580b1b20 0x7f2b580c4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:33.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.499+0000 7f2b5d4b4640 1 --2- 172.21.15.154:0/2735251939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b580acce0 0x7f2b5815d250 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:33.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.499+0000 7f2b5d4b4640 1 -- 172.21.15.154:0/2735251939 >> 172.21.15.154:0/2735251939 conn(0x7f2b5806a0f0 msgr2=0x7f2b580affe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:33.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.499+0000 7f2b5d4b4640 1 -- 172.21.15.154:0/2735251939 shutdown_connections 2023-09-23T03:21:33.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.499+0000 7f2b5d4b4640 1 -- 172.21.15.154:0/2735251939 wait complete. 2023-09-23T03:21:33.519 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:21:33.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.730+0000 7efedfd6c640 1 Processor -- start 2023-09-23T03:21:33.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.730+0000 7efedfd6c640 1 -- start start 2023-09-23T03:21:33.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.731+0000 7efedfd6c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efed815caf0 0x7efed80ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:33.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.731+0000 7efedfd6c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efed80cee70 0x7efed8160880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:33.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.732+0000 7efedfd6c640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efed80cd530 con 0x7efed8160dc0 2023-09-23T03:21:33.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.732+0000 7efedfd6c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efed815ced0 con 0x7efed80cee70 2023-09-23T03:21:33.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.732+0000 7efedfd6c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed80cf2f0 con 0x7efed815caf0 2023-09-23T03:21:33.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.732+0000 7efedd307640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efed80cee70 0x7efed8160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:33.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.732+0000 7efedd307640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efed80cee70 0x7efed8160880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49658/0 (socket says 172.21.15.154:49658) 2023-09-23T03:21:33.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.732+0000 7efedd307640 1 -- 172.21.15.154:0/2436224405 learned_addr learned my addr 172.21.15.154:0/2436224405 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:33.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.732+0000 7efedd307640 1 -- 172.21.15.154:0/2436224405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efed815caf0 msgr2=0x7efed80ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:33.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.732+0000 7efeddb08640 1 --2- 172.21.15.154:0/2436224405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efed815caf0 0x7efed80ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:33.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.732+0000 7efedd307640 1 --2- 172.21.15.154:0/2436224405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efed815caf0 0x7efed80ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:33.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.732+0000 7efedd307640 1 -- 172.21.15.154:0/2436224405 >> v1:172.21.15.190:6789/0 conn(0x7efed8160dc0 legacy=0x7efed806cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:33.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.732+0000 7efedd307640 1 -- 172.21.15.154:0/2436224405 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed80cd810 con 0x7efed80cee70 2023-09-23T03:21:33.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.734+0000 7efedd307640 1 --2- 172.21.15.154:0/2436224405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efed80cee70 0x7efed8160880 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7efec8009870 tx=0x7efec802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a80bc42e98f0dfb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:33.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.734+0000 7efedcb06640 1 -- 172.21.15.154:0/2436224405 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec8031640 con 0x7efed80cee70 2023-09-23T03:21:33.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.734+0000 7efedcb06640 1 -- 172.21.15.154:0/2436224405 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec80317a0 con 0x7efed80cee70 2023-09-23T03:21:33.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.734+0000 7efedcb06640 1 -- 172.21.15.154:0/2436224405 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec8031b30 con 0x7efed80cee70 2023-09-23T03:21:33.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.734+0000 7efedfd6c640 1 -- 172.21.15.154:0/2436224405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efed80cee70 msgr2=0x7efed8160880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:33.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.734+0000 7efedfd6c640 1 --2- 172.21.15.154:0/2436224405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efed80cee70 0x7efed8160880 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7efec8009870 tx=0x7efec802b8f0 comp rx=0 tx=0).stop 2023-09-23T03:21:33.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.735+0000 7efedfd6c640 1 -- 172.21.15.154:0/2436224405 shutdown_connections 2023-09-23T03:21:33.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.735+0000 7efedfd6c640 1 --2- 172.21.15.154:0/2436224405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efed80cee70 0x7efed8160880 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:33.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.735+0000 7efedfd6c640 1 --2- 172.21.15.154:0/2436224405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efed815caf0 0x7efed80ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:33.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.735+0000 7efedfd6c640 1 -- 172.21.15.154:0/2436224405 >> 172.21.15.154:0/2436224405 conn(0x7efed806a0f0 msgr2=0x7efed806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:33.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.735+0000 7efedfd6c640 1 -- 172.21.15.154:0/2436224405 shutdown_connections 2023-09-23T03:21:33.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.735+0000 7efedfd6c640 1 -- 172.21.15.154:0/2436224405 wait complete. 2023-09-23T03:21:33.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.736+0000 7efedfd6c640 1 Processor -- start 2023-09-23T03:21:33.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.737+0000 7efedfd6c640 1 -- start start 2023-09-23T03:21:33.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.737+0000 7efedfd6c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efed80cee70 0x7efed8153610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:33.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.737+0000 7efedfd6c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efed815caf0 0x7efed815b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:33.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.737+0000 7efedfd6c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efed8160dc0 0x7efed815bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:33.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.737+0000 7efedfd6c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed80c1ff0 con 0x7efed815caf0 2023-09-23T03:21:33.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.737+0000 7efedfd6c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed8153b50 con 0x7efed80cee70 2023-09-23T03:21:33.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.737+0000 7efedfd6c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed80c2240 con 0x7efed8160dc0 2023-09-23T03:21:33.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.738+0000 7efedd307640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efed815caf0 0x7efed815b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:33.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.738+0000 7efedd307640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efed815caf0 0x7efed815b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49668/0 (socket says 172.21.15.154:49668) 2023-09-23T03:21:33.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.738+0000 7efedd307640 1 -- 172.21.15.154:0/2459616718 learned_addr learned my addr 172.21.15.154:0/2459616718 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:33.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.738+0000 7efeddb08640 1 --2- 172.21.15.154:0/2459616718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efed80cee70 0x7efed8153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:33.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.738+0000 7efedd307640 1 -- 172.21.15.154:0/2459616718 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efed8160dc0 msgr2=0x7efed815bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:33.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.738+0000 7efede309640 1 --2- 172.21.15.154:0/2459616718 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efed8160dc0 0x7efed815bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:33.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.738+0000 7efedd307640 1 --2- 172.21.15.154:0/2459616718 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efed8160dc0 0x7efed815bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:33.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.738+0000 7efedd307640 1 -- 172.21.15.154:0/2459616718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efed80cee70 msgr2=0x7efed8153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:33.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.738+0000 7efedd307640 1 --2- 172.21.15.154:0/2459616718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efed80cee70 0x7efed8153610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:33.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.738+0000 7efedd307640 1 -- 172.21.15.154:0/2459616718 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec80094d0 con 0x7efed815caf0 2023-09-23T03:21:33.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.739+0000 7efedd307640 1 --2- 172.21.15.154:0/2459616718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efed815caf0 0x7efed815b800 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7efec8000c00 tx=0x7efec80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:33.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.739+0000 7efece7fc640 1 -- 172.21.15.154:0/2459616718 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec803ac60 con 0x7efed815caf0 2023-09-23T03:21:33.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.739+0000 7efece7fc640 1 -- 172.21.15.154:0/2459616718 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec803ade0 con 0x7efed815caf0 2023-09-23T03:21:33.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.739+0000 7efece7fc640 1 -- 172.21.15.154:0/2459616718 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec8044630 con 0x7efed815caf0 2023-09-23T03:21:33.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.739+0000 7efedfd6c640 1 -- 172.21.15.154:0/2459616718 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efed80c2520 con 0x7efed815caf0 2023-09-23T03:21:33.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.739+0000 7efedfd6c640 1 -- 172.21.15.154:0/2459616718 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efed80c2ae0 con 0x7efed815caf0 2023-09-23T03:21:33.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.740+0000 7efedfd6c640 1 -- 172.21.15.154:0/2459616718 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efed80c2cd0 con 0x7efed815caf0 2023-09-23T03:21:33.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.741+0000 7efece7fc640 1 -- 172.21.15.154:0/2459616718 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7efec802f050 con 0x7efed815caf0 2023-09-23T03:21:33.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.743+0000 7efece7fc640 1 --2- 172.21.15.154:0/2459616718 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efebc06af30 0x7efebc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:33.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.743+0000 7efece7fc640 1 -- 172.21.15.154:0/2459616718 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x7efec80abcd0 con 0x7efed815caf0 2023-09-23T03:21:33.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.743+0000 7efeddb08640 1 --2- 172.21.15.154:0/2459616718 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efebc06af30 0x7efebc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:33.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.744+0000 7efeddb08640 1 --2- 172.21.15.154:0/2459616718 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efebc06af30 0x7efebc06d3f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7efed400a950 tx=0x7efed4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:33.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.747+0000 7efece7fc640 1 -- 172.21.15.154:0/2459616718 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7efec80768f0 con 0x7efed815caf0 2023-09-23T03:21:33.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.971+0000 7efedfd6c640 1 -- 172.21.15.154:0/2459616718 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efed8073eb0 con 0x7efed815caf0 2023-09-23T03:21:33.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.972+0000 7efece7fc640 1 -- 172.21.15.154:0/2459616718 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4920 (secure 0 0 0) 0x7efec8035090 con 0x7efed815caf0 2023-09-23T03:21:33.974 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:33.974 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":47,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5027,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2517324433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2517324433},{"type":"v1","addr":"172.21.15.190:6833","nonce":2517324433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":5033,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4098915995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4098915995},{"type":"v1","addr":"172.21.15.190:6835","nonce":4098915995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6309,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/3034894009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":3034894009},{"type":"v1","addr":"172.21.15.154:6836","nonce":3034894009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6639,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/4129414148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":4129414148},{"type":"v1","addr":"172.21.15.154:6837","nonce":4129414148}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:21:30.160245+0000","modified":"2023-09-23T03:21:31.524972+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[9,10],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-09-23T03:21:33.974 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 47 2023-09-23T03:21:33.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.974+0000 7efedfd6c640 1 -- 172.21.15.154:0/2459616718 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efebc06af30 msgr2=0x7efebc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:33.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.974+0000 7efedfd6c640 1 --2- 172.21.15.154:0/2459616718 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efebc06af30 0x7efebc06d3f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7efed400a950 tx=0x7efed4008040 comp rx=0 tx=0).stop 2023-09-23T03:21:33.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.974+0000 7efedfd6c640 1 -- 172.21.15.154:0/2459616718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efed815caf0 msgr2=0x7efed815b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:33.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.974+0000 7efedfd6c640 1 --2- 172.21.15.154:0/2459616718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efed815caf0 0x7efed815b800 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7efec8000c00 tx=0x7efec80046c0 comp rx=0 tx=0).stop 2023-09-23T03:21:33.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.975+0000 7efedfd6c640 1 -- 172.21.15.154:0/2459616718 shutdown_connections 2023-09-23T03:21:33.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.975+0000 7efedfd6c640 1 --2- 172.21.15.154:0/2459616718 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efebc06af30 0x7efebc06d3f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:33.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.975+0000 7efedfd6c640 1 --2- 172.21.15.154:0/2459616718 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efed8160dc0 0x7efed815bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:33.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.975+0000 7efedfd6c640 1 --2- 172.21.15.154:0/2459616718 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efed815caf0 0x7efed815b800 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:33.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.975+0000 7efedfd6c640 1 --2- 172.21.15.154:0/2459616718 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efed80cee70 0x7efed8153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:33.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.975+0000 7efedfd6c640 1 -- 172.21.15.154:0/2459616718 >> 172.21.15.154:0/2459616718 conn(0x7efed806a0f0 msgr2=0x7efed80b0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:33.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.975+0000 7efedfd6c640 1 -- 172.21.15.154:0/2459616718 shutdown_connections 2023-09-23T03:21:33.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:33.975+0000 7efedfd6c640 1 -- 172.21.15.154:0/2459616718 wait complete. 2023-09-23T03:21:33.994 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:21:34.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.206+0000 7fc82daa3640 1 Processor -- start 2023-09-23T03:21:34.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.206+0000 7fc82daa3640 1 -- start start 2023-09-23T03:21:34.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.208+0000 7fc82daa3640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc82815c840 0x7fc8280cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.208+0000 7fc82daa3640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc828160fd0 0x7fc8280c9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.208+0000 7fc82daa3640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8280cc910 con 0x7fc8280c99d0 2023-09-23T03:21:34.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.208+0000 7fc82daa3640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc82815cc20 con 0x7fc828160fd0 2023-09-23T03:21:34.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.208+0000 7fc82daa3640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8280c89b0 con 0x7fc82815c840 2023-09-23T03:21:34.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.208+0000 7fc827fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc828160fd0 0x7fc8280c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.208+0000 7fc827fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc828160fd0 0x7fc8280c9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60198/0 (socket says 172.21.15.154:60198) 2023-09-23T03:21:34.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.208+0000 7fc827fff640 1 -- 172.21.15.154:0/1021693618 learned_addr learned my addr 172.21.15.154:0/1021693618 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:34.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.209+0000 7fc82caa1640 1 --2- 172.21.15.154:0/1021693618 >> v2:172.21.15.190:3300/0 conn(0x7fc82815c840 0x7fc8280cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.209+0000 7fc827fff640 1 -- 172.21.15.154:0/1021693618 >> v2:172.21.15.190:3300/0 conn(0x7fc82815c840 msgr2=0x7fc8280cbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:34.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.209+0000 7fc827fff640 1 --2- 172.21.15.154:0/1021693618 >> v2:172.21.15.190:3300/0 conn(0x7fc82815c840 0x7fc8280cbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.209+0000 7fc827fff640 1 -- 172.21.15.154:0/1021693618 >> v1:172.21.15.190:6789/0 conn(0x7fc8280c99d0 legacy=0x7fc8280cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:34.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.209+0000 7fc827fff640 1 -- 172.21.15.154:0/1021693618 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8280c8c90 con 0x7fc828160fd0 2023-09-23T03:21:34.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.210+0000 7fc827fff640 1 --2- 172.21.15.154:0/1021693618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc828160fd0 0x7fc8280c9490 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fc82806cdd0 tx=0x7fc81402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fcdbed55a21c2ac server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:34.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.211+0000 7fc8277fe640 1 -- 172.21.15.154:0/1021693618 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc814031870 con 0x7fc828160fd0 2023-09-23T03:21:34.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.211+0000 7fc8277fe640 1 -- 172.21.15.154:0/1021693618 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8140319d0 con 0x7fc828160fd0 2023-09-23T03:21:34.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.211+0000 7fc8277fe640 1 -- 172.21.15.154:0/1021693618 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc814031d60 con 0x7fc828160fd0 2023-09-23T03:21:34.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.211+0000 7fc82daa3640 1 -- 172.21.15.154:0/1021693618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc828160fd0 msgr2=0x7fc8280c9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:34.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.211+0000 7fc82daa3640 1 --2- 172.21.15.154:0/1021693618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc828160fd0 0x7fc8280c9490 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fc82806cdd0 tx=0x7fc81402b910 comp rx=0 tx=0).stop 2023-09-23T03:21:34.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.212+0000 7fc82daa3640 1 -- 172.21.15.154:0/1021693618 shutdown_connections 2023-09-23T03:21:34.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.212+0000 7fc82daa3640 1 --2- 172.21.15.154:0/1021693618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc828160fd0 0x7fc8280c9490 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.212+0000 7fc82daa3640 1 --2- 172.21.15.154:0/1021693618 >> v2:172.21.15.190:3300/0 conn(0x7fc82815c840 0x7fc8280cbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.212+0000 7fc82daa3640 1 -- 172.21.15.154:0/1021693618 >> 172.21.15.154:0/1021693618 conn(0x7fc82806a390 msgr2=0x7fc8280afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:34.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.212+0000 7fc82daa3640 1 -- 172.21.15.154:0/1021693618 shutdown_connections 2023-09-23T03:21:34.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.212+0000 7fc82daa3640 1 -- 172.21.15.154:0/1021693618 wait complete. 2023-09-23T03:21:34.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.213+0000 7fc82daa3640 1 Processor -- start 2023-09-23T03:21:34.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.214+0000 7fc82daa3640 1 -- start start 2023-09-23T03:21:34.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.214+0000 7fc82daa3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc8280c99d0 0x7fc828150a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.214+0000 7fc82daa3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc82815c840 0x7fc8280aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.215+0000 7fc82daa3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc828160fd0 0x7fc8280af310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.215+0000 7fc82daa3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc828152150 con 0x7fc8280c99d0 2023-09-23T03:21:34.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.215+0000 7fc82daa3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc828152ea0 con 0x7fc82815c840 2023-09-23T03:21:34.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.215+0000 7fc82daa3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc828153180 con 0x7fc828160fd0 2023-09-23T03:21:34.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.215+0000 7fc82caa1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc8280c99d0 0x7fc828150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.215+0000 7fc82caa1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc8280c99d0 0x7fc828150a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60206/0 (socket says 172.21.15.154:60206) 2023-09-23T03:21:34.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.215+0000 7fc82caa1640 1 -- 172.21.15.154:0/1937649932 learned_addr learned my addr 172.21.15.154:0/1937649932 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:34.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.215+0000 7fc827fff640 1 --2- 172.21.15.154:0/1937649932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc82815c840 0x7fc8280aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.215+0000 7fc82caa1640 1 -- 172.21.15.154:0/1937649932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc828160fd0 msgr2=0x7fc8280af310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:34.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.215+0000 7fc82caa1640 1 --2- 172.21.15.154:0/1937649932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc828160fd0 0x7fc8280af310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.215+0000 7fc82caa1640 1 -- 172.21.15.154:0/1937649932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc82815c840 msgr2=0x7fc8280aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:34.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.215+0000 7fc82caa1640 1 --2- 172.21.15.154:0/1937649932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc82815c840 0x7fc8280aedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.215+0000 7fc82caa1640 1 -- 172.21.15.154:0/1937649932 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8140094d0 con 0x7fc8280c99d0 2023-09-23T03:21:34.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.215+0000 7fc827fff640 1 --2- 172.21.15.154:0/1937649932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc82815c840 0x7fc8280aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:34.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.216+0000 7fc82caa1640 1 --2- 172.21.15.154:0/1937649932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc8280c99d0 0x7fc828150a10 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fc81c00e9b0 tx=0x7fc81c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:34.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.216+0000 7fc8257fa640 1 -- 172.21.15.154:0/1937649932 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c009670 con 0x7fc8280c99d0 2023-09-23T03:21:34.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.216+0000 7fc8257fa640 1 -- 172.21.15.154:0/1937649932 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81c00c9a0 con 0x7fc8280c99d0 2023-09-23T03:21:34.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.216+0000 7fc8257fa640 1 -- 172.21.15.154:0/1937649932 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c00cd50 con 0x7fc8280c99d0 2023-09-23T03:21:34.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.216+0000 7fc82daa3640 1 -- 172.21.15.154:0/1937649932 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc828153460 con 0x7fc8280c99d0 2023-09-23T03:21:34.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.216+0000 7fc82daa3640 1 -- 172.21.15.154:0/1937649932 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc828152430 con 0x7fc8280c99d0 2023-09-23T03:21:34.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.217+0000 7fc82daa3640 1 -- 172.21.15.154:0/1937649932 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8280736b0 con 0x7fc8280c99d0 2023-09-23T03:21:34.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.218+0000 7fc8257fa640 1 -- 172.21.15.154:0/1937649932 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fc81c013470 con 0x7fc8280c99d0 2023-09-23T03:21:34.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.219+0000 7fc8257fa640 1 --2- 172.21.15.154:0/1937649932 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc80c06af30 0x7fc80c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.219+0000 7fc8257fa640 1 -- 172.21.15.154:0/1937649932 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x7fc81c089d60 con 0x7fc8280c99d0 2023-09-23T03:21:34.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.219+0000 7fc827fff640 1 --2- 172.21.15.154:0/1937649932 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc80c06af30 0x7fc80c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.220+0000 7fc827fff640 1 --2- 172.21.15.154:0/1937649932 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc80c06af30 0x7fc80c06d3f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fc814036000 tx=0x7fc81403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:34.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.223+0000 7fc8257fa640 1 -- 172.21.15.154:0/1937649932 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fc81c054a10 con 0x7fc8280c99d0 2023-09-23T03:21:34.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.453+0000 7fc82daa3640 1 -- 172.21.15.154:0/1937649932 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8280cc4a0 con 0x7fc8280c99d0 2023-09-23T03:21:34.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.454+0000 7fc8257fa640 1 -- 172.21.15.154:0/1937649932 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4920 (secure 0 0 0) 0x7fc81c017090 con 0x7fc8280c99d0 2023-09-23T03:21:34.455 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:34.456 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":47,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5027,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2517324433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2517324433},{"type":"v1","addr":"172.21.15.190:6833","nonce":2517324433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":5033,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4098915995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4098915995},{"type":"v1","addr":"172.21.15.190:6835","nonce":4098915995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6309,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/3034894009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":3034894009},{"type":"v1","addr":"172.21.15.154:6836","nonce":3034894009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6639,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/4129414148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":4129414148},{"type":"v1","addr":"172.21.15.154:6837","nonce":4129414148}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:21:30.160245+0000","modified":"2023-09-23T03:21:31.524972+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[9,10],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-09-23T03:21:34.456 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 47 2023-09-23T03:21:34.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.456+0000 7fc82daa3640 1 -- 172.21.15.154:0/1937649932 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc80c06af30 msgr2=0x7fc80c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:34.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.456+0000 7fc82daa3640 1 --2- 172.21.15.154:0/1937649932 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc80c06af30 0x7fc80c06d3f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fc814036000 tx=0x7fc81403c040 comp rx=0 tx=0).stop 2023-09-23T03:21:34.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.456+0000 7fc82daa3640 1 -- 172.21.15.154:0/1937649932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc8280c99d0 msgr2=0x7fc828150a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:34.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.456+0000 7fc82daa3640 1 --2- 172.21.15.154:0/1937649932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc8280c99d0 0x7fc828150a10 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fc81c00e9b0 tx=0x7fc81c00ee70 comp rx=0 tx=0).stop 2023-09-23T03:21:34.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.456+0000 7fc82daa3640 1 -- 172.21.15.154:0/1937649932 shutdown_connections 2023-09-23T03:21:34.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.456+0000 7fc82daa3640 1 --2- 172.21.15.154:0/1937649932 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc80c06af30 0x7fc80c06d3f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.456+0000 7fc82daa3640 1 --2- 172.21.15.154:0/1937649932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc828160fd0 0x7fc8280af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.456+0000 7fc82daa3640 1 --2- 172.21.15.154:0/1937649932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc82815c840 0x7fc8280aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.457+0000 7fc82daa3640 1 --2- 172.21.15.154:0/1937649932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc8280c99d0 0x7fc828150a10 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.457+0000 7fc82daa3640 1 -- 172.21.15.154:0/1937649932 >> 172.21.15.154:0/1937649932 conn(0x7fc82806a390 msgr2=0x7fc8280cac00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:34.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.457+0000 7fc82daa3640 1 -- 172.21.15.154:0/1937649932 shutdown_connections 2023-09-23T03:21:34.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.457+0000 7fc82daa3640 1 -- 172.21.15.154:0/1937649932 wait complete. 2023-09-23T03:21:34.477 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset session 2023-09-23T03:21:34.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.576+0000 7f0f7780c100 1 -- start start 2023-09-23T03:21:34.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.576+0000 7f0f7780c100 1 Processor -- start 2023-09-23T03:21:34.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.577+0000 7f0f7780c100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cb642a6c00 0x55cb64316580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.577+0000 7f0f7780c100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb642a7800 0x55cb64316b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.577+0000 7f0f7780c100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb642a7c00 0x55cb64317080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.577+0000 7f0f7780c100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb642f3180 con 0x55cb642a7c00 2023-09-23T03:21:34.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.577+0000 7f0f7780c100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb642f2140 con 0x55cb642a7800 2023-09-23T03:21:34.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.577+0000 7f0f7780c100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb642f2000 con 0x55cb642a6c00 2023-09-23T03:21:34.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.577+0000 7f0f75be2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb642a7c00 0x55cb64317080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.577+0000 7f0f75be2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb642a7c00 0x55cb64317080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60226/0 (socket says 172.21.15.154:60226) 2023-09-23T03:21:34.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.577+0000 7f0f75be2640 1 -- 172.21.15.154:0/634917647 learned_addr learned my addr 172.21.15.154:0/634917647 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:34.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.578+0000 7f0f74be0640 1 --2- 172.21.15.154:0/634917647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb642a7800 0x55cb64316b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.578+0000 7f0f75be2640 1 -- 172.21.15.154:0/634917647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cb642a6c00 msgr2=0x55cb64316580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:34.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.578+0000 7f0f75be2640 1 --2- 172.21.15.154:0/634917647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cb642a6c00 0x55cb64316580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.578+0000 7f0f75be2640 1 -- 172.21.15.154:0/634917647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb642a7800 msgr2=0x55cb64316b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:34.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.578+0000 7f0f75be2640 1 --2- 172.21.15.154:0/634917647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb642a7800 0x55cb64316b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.578+0000 7f0f75be2640 1 -- 172.21.15.154:0/634917647 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb642dab60 con 0x55cb642a7c00 2023-09-23T03:21:34.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.578+0000 7f0f75be2640 1 --2- 172.21.15.154:0/634917647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb642a7c00 0x55cb64317080 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x55cb650c4b70 tx=0x55cb64ffe120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71cfa2e52662022e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:34.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.578+0000 7f0f72bdc640 1 -- 172.21.15.154:0/634917647 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55cb642f1080 con 0x55cb642a7c00 2023-09-23T03:21:34.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.579+0000 7f0f72bdc640 1 -- 172.21.15.154:0/634917647 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cb642cd800 con 0x55cb642a7c00 2023-09-23T03:21:34.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.579+0000 7f0f72bdc640 1 -- 172.21.15.154:0/634917647 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55cb642f1600 con 0x55cb642a7c00 2023-09-23T03:21:34.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.579+0000 7f0f7780c100 1 -- 172.21.15.154:0/634917647 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb642db1e0 con 0x55cb642a7c00 2023-09-23T03:21:34.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.579+0000 7f0f7780c100 1 -- 172.21.15.154:0/634917647 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55cb642dab60 con 0x55cb642a7c00 2023-09-23T03:21:34.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.579+0000 7f0f7780c100 4 waiting for MDS map... 2023-09-23T03:21:34.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.579+0000 7f0f72bdc640 1 -- 172.21.15.154:0/634917647 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x55cb642d2380 con 0x55cb642a7c00 2023-09-23T03:21:34.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.579+0000 7f0f72bdc640 1 -- 172.21.15.154:0/634917647 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x55cb642fb200 con 0x55cb642a7c00 2023-09-23T03:21:34.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.579+0000 7f0f7780c100 4 Got MDS map 47 2023-09-23T03:21:34.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.579+0000 7f0f7780c100 10 main: main 2023-09-23T03:21:34.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.579+0000 7f0f7780c100 4 main: connecting to RADOS... 2023-09-23T03:21:34.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.580+0000 7f0f7780c100 1 Processor -- start 2023-09-23T03:21:34.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.580+0000 7f0f7780c100 1 -- start start 2023-09-23T03:21:34.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.581+0000 7f0f7780c100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb65085c00 0x55cb64317600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.581+0000 7f0f7780c100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb651b4000 0x55cb64317b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.581+0000 7f0f7780c100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cb651b4400 0x55cb64318100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.581+0000 7f0f7780c100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb642f2a00 con 0x55cb651b4000 2023-09-23T03:21:34.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.581+0000 7f0f7780c100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb64375180 con 0x55cb65085c00 2023-09-23T03:21:34.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.581+0000 7f0f7780c100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb64374dc0 con 0x55cb651b4400 2023-09-23T03:21:34.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.581+0000 7f0f753e1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb651b4000 0x55cb64317b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.581+0000 7f0f753e1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb651b4000 0x55cb64317b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60242/0 (socket says 172.21.15.154:60242) 2023-09-23T03:21:34.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.581+0000 7f0f753e1640 1 -- 172.21.15.154:0/3067249765 learned_addr learned my addr 172.21.15.154:0/3067249765 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:34.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.582+0000 7f0f74be0640 1 --2- 172.21.15.154:0/3067249765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb65085c00 0x55cb64317600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.582+0000 7f0f753e1640 1 -- 172.21.15.154:0/3067249765 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cb651b4400 msgr2=0x55cb64318100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:34.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.582+0000 7f0f753e1640 1 --2- 172.21.15.154:0/3067249765 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cb651b4400 0x55cb64318100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.582+0000 7f0f753e1640 1 -- 172.21.15.154:0/3067249765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb65085c00 msgr2=0x55cb64317600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:34.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.582+0000 7f0f753e1640 1 --2- 172.21.15.154:0/3067249765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb65085c00 0x55cb64317600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.582+0000 7f0f753e1640 1 -- 172.21.15.154:0/3067249765 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb642da340 con 0x55cb651b4000 2023-09-23T03:21:34.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.582+0000 7f0f74be0640 1 --2- 172.21.15.154:0/3067249765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb65085c00 0x55cb64317600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:34.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.582+0000 7f0f753e1640 1 --2- 172.21.15.154:0/3067249765 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb651b4000 0x55cb64317b80 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x55cb651be0c0 tx=0x55cb65166d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed7afa8c13c5d633 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:34.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.582+0000 7f0f70bd8640 1 -- 172.21.15.154:0/3067249765 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55cb642f1080 con 0x55cb651b4000 2023-09-23T03:21:34.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.582+0000 7f0f70bd8640 1 -- 172.21.15.154:0/3067249765 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cb65082a80 con 0x55cb651b4000 2023-09-23T03:21:34.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.582+0000 7f0f70bd8640 1 -- 172.21.15.154:0/3067249765 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55cb642f1600 con 0x55cb651b4000 2023-09-23T03:21:34.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.583+0000 7f0f7780c100 1 -- 172.21.15.154:0/3067249765 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb651b4000 msgr2=0x55cb64317b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:34.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.583+0000 7f0f7780c100 1 --2- 172.21.15.154:0/3067249765 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb651b4000 0x55cb64317b80 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x55cb651be0c0 tx=0x55cb65166d20 comp rx=0 tx=0).stop 2023-09-23T03:21:34.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.583+0000 7f0f7780c100 1 -- 172.21.15.154:0/3067249765 shutdown_connections 2023-09-23T03:21:34.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.583+0000 7f0f7780c100 1 --2- 172.21.15.154:0/3067249765 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cb651b4400 0x55cb64318100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.583+0000 7f0f7780c100 1 --2- 172.21.15.154:0/3067249765 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb651b4000 0x55cb64317b80 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.583+0000 7f0f7780c100 1 --2- 172.21.15.154:0/3067249765 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb65085c00 0x55cb64317600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.583+0000 7f0f7780c100 1 -- 172.21.15.154:0/3067249765 >> 172.21.15.154:0/3067249765 conn(0x55cb65085800 msgr2=0x55cb6501f000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:34.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.583+0000 7f0f7780c100 1 -- 172.21.15.154:0/3067249765 shutdown_connections 2023-09-23T03:21:34.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.583+0000 7f0f7780c100 1 -- 172.21.15.154:0/3067249765 wait complete. 2023-09-23T03:21:34.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.584+0000 7f0f7780c100 1 Processor -- start 2023-09-23T03:21:34.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.584+0000 7f0f7780c100 1 -- start start 2023-09-23T03:21:34.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.584+0000 7f0f7780c100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cb65085800 0x55cb64319180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.584+0000 7f0f7780c100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb651b5000 0x55cb64318c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.584+0000 7f0f7780c100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb651b4c00 0x55cb64318680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.584+0000 7f0f7780c100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb642f2140 con 0x55cb651b4c00 2023-09-23T03:21:34.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.584+0000 7f0f7780c100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb642f3180 con 0x55cb651b5000 2023-09-23T03:21:34.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.584+0000 7f0f7780c100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb642f2b40 con 0x55cb65085800 2023-09-23T03:21:34.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.585+0000 7f0f74be0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb651b4c00 0x55cb64318680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.585+0000 7f0f74be0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb651b4c00 0x55cb64318680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60252/0 (socket says 172.21.15.154:60252) 2023-09-23T03:21:34.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.585+0000 7f0f74be0640 1 -- 172.21.15.154:0/1193081412 learned_addr learned my addr 172.21.15.154:0/1193081412 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:34.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.585+0000 7f0f753e1640 1 --2- 172.21.15.154:0/1193081412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb651b5000 0x55cb64318c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.585+0000 7f0f74be0640 1 --2- 172.21.15.154:0/1193081412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cb65085800 0x55cb64319180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.585+0000 7f0f74be0640 1 -- 172.21.15.154:0/1193081412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cb65085800 msgr2=0x55cb64319180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:34.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.585+0000 7f0f74be0640 1 --2- 172.21.15.154:0/1193081412 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cb65085800 0x55cb64319180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.585+0000 7f0f74be0640 1 -- 172.21.15.154:0/1193081412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb651b5000 msgr2=0x55cb64318c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:34.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.585+0000 7f0f74be0640 1 --2- 172.21.15.154:0/1193081412 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb651b5000 0x55cb64318c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.585+0000 7f0f74be0640 1 -- 172.21.15.154:0/1193081412 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb642da000 con 0x55cb651b4c00 2023-09-23T03:21:34.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.586+0000 7f0f74be0640 1 --2- 172.21.15.154:0/1193081412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb651b4c00 0x55cb64318680 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x55cb651bede0 tx=0x55cb65166ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:34.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.586+0000 7f0f6f3d5640 1 -- 172.21.15.154:0/1193081412 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55cb642f1080 con 0x55cb651b4c00 2023-09-23T03:21:34.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.586+0000 7f0f6f3d5640 1 -- 172.21.15.154:0/1193081412 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cb65083380 con 0x55cb651b4c00 2023-09-23T03:21:34.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.586+0000 7f0f7780c100 1 -- 172.21.15.154:0/1193081412 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55cb642daea0 con 0x55cb651b4c00 2023-09-23T03:21:34.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.586+0000 7f0f6f3d5640 1 -- 172.21.15.154:0/1193081412 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55cb642f1600 con 0x55cb651b4c00 2023-09-23T03:21:34.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.586+0000 7f0f7780c100 1 -- 172.21.15.154:0/1193081412 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb642db520 con 0x55cb651b4c00 2023-09-23T03:21:34.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.586+0000 7f0f7780c100 4 main: resolving pool 8 2023-09-23T03:21:34.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.588+0000 7f0f6f3d5640 1 -- 172.21.15.154:0/1193081412 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x55cb642d8840 con 0x55cb651b4c00 2023-09-23T03:21:34.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.588+0000 7f0f6f3d5640 1 --2- 172.21.15.154:0/1193081412 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55cb651dac00 0x55cb64319700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.588+0000 7f0f6f3d5640 1 -- 172.21.15.154:0/1193081412 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x55cb642d2540 con 0x55cb651b4c00 2023-09-23T03:21:34.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.588+0000 7f0f7780c100 4 main: creating IoCtx.. 2023-09-23T03:21:34.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.588+0000 7f0f7780c100 1 --2- 172.21.15.154:0/1193081412 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55cb651b4800 0x55cb64318100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.588+0000 7f0f7780c100 1 -- 172.21.15.154:0/1193081412 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:1 8.b 8:d0630e4c:::mds0_sessionmap:head [omap-clear,omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55cb64303880 con 0x55cb651b4800 2023-09-23T03:21:34.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.588+0000 7f0f753e1640 1 --2- 172.21.15.154:0/1193081412 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55cb651dac00 0x55cb64319700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.589+0000 7f0f74be0640 1 --2- 172.21.15.154:0/1193081412 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55cb651b4800 0x55cb64318100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.589+0000 7f0f753e1640 1 --2- 172.21.15.154:0/1193081412 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55cb651dac00 0x55cb64319700 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x55cb651be0c0 tx=0x55cb65166c00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:34.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.589+0000 7f0f74be0640 1 --2- 172.21.15.154:0/1193081412 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55cb651b4800 0x55cb64318100 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:34.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.593+0000 7f0f74be0640 1 -- 172.21.15.154:0/1193081412 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(1 mds0_sessionmap [omap-clear,omap-set-header] v67'5 uv5 ondisk = 0) v8 ==== 201+0+0 (crc 0 0 0) 0x55cb64ff8b40 con 0x55cb651b4800 2023-09-23T03:21:34.594 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:21:34.594 INFO:teuthology.orchestra.run.smithi154.stdout: "0": { 2023-09-23T03:21:34.595 INFO:teuthology.orchestra.run.smithi154.stdout: "data": {}, 2023-09-23T03:21:34.595 INFO:teuthology.orchestra.run.smithi154.stdout: "result": 0 2023-09-23T03:21:34.595 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:21:34.595 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:21:34.595 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:34.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.593+0000 7f0f7780c100 1 -- 172.21.15.154:0/1193081412 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55cb651b4800 msgr2=0x55cb64318100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:34.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.593+0000 7f0f7780c100 1 --2- 172.21.15.154:0/1193081412 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55cb651b4800 0x55cb64318100 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.593+0000 7f0f7780c100 1 -- 172.21.15.154:0/1193081412 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55cb651dac00 msgr2=0x55cb64319700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:34.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.593+0000 7f0f7780c100 1 --2- 172.21.15.154:0/1193081412 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55cb651dac00 0x55cb64319700 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x55cb651be0c0 tx=0x55cb65166c00 comp rx=0 tx=0).stop 2023-09-23T03:21:34.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.593+0000 7f0f7780c100 1 -- 172.21.15.154:0/1193081412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb651b4c00 msgr2=0x55cb64318680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:34.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.593+0000 7f0f7780c100 1 --2- 172.21.15.154:0/1193081412 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb651b4c00 0x55cb64318680 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x55cb651bede0 tx=0x55cb65166ea0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.593+0000 7f0f753e1640 1 -- 172.21.15.154:0/1193081412 reap_dead start 2023-09-23T03:21:34.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.593+0000 7f0f7780c100 1 -- 172.21.15.154:0/1193081412 shutdown_connections 2023-09-23T03:21:34.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.593+0000 7f0f7780c100 1 -- 172.21.15.154:0/1193081412 >> 172.21.15.154:0/1193081412 conn(0x55cb65085c00 msgr2=0x55cb6501f000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:34.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.593+0000 7f0f7780c100 1 -- 172.21.15.154:0/1193081412 shutdown_connections 2023-09-23T03:21:34.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.593+0000 7f0f7780c100 1 -- 172.21.15.154:0/1193081412 wait complete. 2023-09-23T03:21:34.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.594+0000 7f0f7780c100 1 -- 172.21.15.154:0/634917647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb642a7c00 msgr2=0x55cb64317080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:34.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.594+0000 7f0f7780c100 1 --2- 172.21.15.154:0/634917647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb642a7c00 0x55cb64317080 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x55cb650c4b70 tx=0x55cb64ffe120 comp rx=0 tx=0).stop 2023-09-23T03:21:34.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.594+0000 7f0f7780c100 1 -- 172.21.15.154:0/634917647 shutdown_connections 2023-09-23T03:21:34.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.594+0000 7f0f7780c100 1 --2- 172.21.15.154:0/634917647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb642a7c00 0x55cb64317080 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.594+0000 7f0f7780c100 1 --2- 172.21.15.154:0/634917647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55cb642a7800 0x55cb64316b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.594+0000 7f0f7780c100 1 --2- 172.21.15.154:0/634917647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55cb642a6c00 0x55cb64316580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.594+0000 7f0f7780c100 1 -- 172.21.15.154:0/634917647 >> 172.21.15.154:0/634917647 conn(0x55cb642a7000 msgr2=0x55cb6501e800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:34.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.594+0000 7f0f7780c100 1 -- 172.21.15.154:0/634917647 shutdown_connections 2023-09-23T03:21:34.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.594+0000 7f0f7780c100 1 -- 172.21.15.154:0/634917647 wait complete. 2023-09-23T03:21:34.600 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'session'] in time 0:00:00.122599, result: { "0": { "data": {}, "result": 0 } } 2023-09-23T03:21:34.600 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset snap 2023-09-23T03:21:34.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.684+0000 7fbd21252100 1 -- start start 2023-09-23T03:21:34.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.684+0000 7fbd21252100 1 Processor -- start 2023-09-23T03:21:34.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.685+0000 7fbd21252100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f07a15cc00 0x55f07a1cc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.685+0000 7fbd21252100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f07a15d800 0x55f07a1ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.685+0000 7fbd21252100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f07a15dc00 0x55f07a1cd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.685+0000 7fbd21252100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f07a1a9180 con 0x55f07a15d800 2023-09-23T03:21:34.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.685+0000 7fbd21252100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f07a1a8140 con 0x55f07a15cc00 2023-09-23T03:21:34.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.685+0000 7fbd21252100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f07a1a8000 con 0x55f07a15dc00 2023-09-23T03:21:34.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.685+0000 7fbd1eac2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f07a15d800 0x55f07a1ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.685+0000 7fbd1eac2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f07a15d800 0x55f07a1ccb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60272/0 (socket says 172.21.15.154:60272) 2023-09-23T03:21:34.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.685+0000 7fbd1eac2640 1 -- 172.21.15.154:0/3647925275 learned_addr learned my addr 172.21.15.154:0/3647925275 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:34.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.686+0000 7fbd1fac4640 1 --2- 172.21.15.154:0/3647925275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f07a15dc00 0x55f07a1cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.686+0000 7fbd1f2c3640 1 --2- 172.21.15.154:0/3647925275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f07a15cc00 0x55f07a1cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.686+0000 7fbd1eac2640 1 -- 172.21.15.154:0/3647925275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f07a15dc00 msgr2=0x55f07a1cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:34.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.686+0000 7fbd1eac2640 1 --2- 172.21.15.154:0/3647925275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f07a15dc00 0x55f07a1cd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.686+0000 7fbd1eac2640 1 -- 172.21.15.154:0/3647925275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f07a15cc00 msgr2=0x55f07a1cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:34.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.686+0000 7fbd1eac2640 1 --2- 172.21.15.154:0/3647925275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f07a15cc00 0x55f07a1cc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.686+0000 7fbd1eac2640 1 -- 172.21.15.154:0/3647925275 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f07a190b60 con 0x55f07a15d800 2023-09-23T03:21:34.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.686+0000 7fbd1fac4640 1 --2- 172.21.15.154:0/3647925275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f07a15dc00 0x55f07a1cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:34.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.686+0000 7fbd1f2c3640 1 --2- 172.21.15.154:0/3647925275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f07a15cc00 0x55f07a1cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:34.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.686+0000 7fbd1eac2640 1 --2- 172.21.15.154:0/3647925275 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f07a15d800 0x55f07a1ccb00 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x55f07af70bd0 tx=0x55f07a43e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a84c4e1632068e2e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:34.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.686+0000 7fbd1cabe640 1 -- 172.21.15.154:0/3647925275 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f07a1a7600 con 0x55f07a15d800 2023-09-23T03:21:34.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.686+0000 7fbd1cabe640 1 -- 172.21.15.154:0/3647925275 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f07a183800 con 0x55f07a15d800 2023-09-23T03:21:34.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.686+0000 7fbd1cabe640 1 -- 172.21.15.154:0/3647925275 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f07a1a71e0 con 0x55f07a15d800 2023-09-23T03:21:34.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.687+0000 7fbd21252100 1 -- 172.21.15.154:0/3647925275 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f07a190340 con 0x55f07a15d800 2023-09-23T03:21:34.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.687+0000 7fbd21252100 1 -- 172.21.15.154:0/3647925275 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55f07a190000 con 0x55f07a15d800 2023-09-23T03:21:34.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.687+0000 7fbd21252100 4 waiting for MDS map... 2023-09-23T03:21:34.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.687+0000 7fbd1cabe640 1 -- 172.21.15.154:0/3647925275 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x55f07a188380 con 0x55f07a15d800 2023-09-23T03:21:34.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.687+0000 7fbd1cabe640 1 -- 172.21.15.154:0/3647925275 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x55f07a440000 con 0x55f07a15d800 2023-09-23T03:21:34.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.687+0000 7fbd21252100 4 Got MDS map 47 2023-09-23T03:21:34.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.687+0000 7fbd21252100 10 main: main 2023-09-23T03:21:34.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.687+0000 7fbd21252100 4 main: connecting to RADOS... 2023-09-23T03:21:34.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.688+0000 7fbd21252100 1 Processor -- start 2023-09-23T03:21:34.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.688+0000 7fbd21252100 1 -- start start 2023-09-23T03:21:34.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.689+0000 7fbd21252100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f07a453000 0x55f07a1cd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.689+0000 7fbd21252100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f07a453400 0x55f07a1cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.689+0000 7fbd21252100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f07a453800 0x55f07a1ce100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.689+0000 7fbd21252100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f07a1a8a00 con 0x55f07a453800 2023-09-23T03:21:34.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.689+0000 7fbd21252100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f07a22b180 con 0x55f07a453000 2023-09-23T03:21:34.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.689+0000 7fbd21252100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f07a22adc0 con 0x55f07a453400 2023-09-23T03:21:34.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.689+0000 7fbd1eac2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f07a453800 0x55f07a1ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.689+0000 7fbd1eac2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f07a453800 0x55f07a1ce100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60276/0 (socket says 172.21.15.154:60276) 2023-09-23T03:21:34.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.689+0000 7fbd1eac2640 1 -- 172.21.15.154:0/1405680193 learned_addr learned my addr 172.21.15.154:0/1405680193 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:34.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.689+0000 7fbd1fac4640 1 --2- 172.21.15.154:0/1405680193 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f07a453000 0x55f07a1cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.689+0000 7fbd1f2c3640 1 --2- 172.21.15.154:0/1405680193 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f07a453400 0x55f07a1cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.689+0000 7fbd1eac2640 1 -- 172.21.15.154:0/1405680193 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f07a453400 msgr2=0x55f07a1cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:34.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.689+0000 7fbd1eac2640 1 --2- 172.21.15.154:0/1405680193 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f07a453400 0x55f07a1cdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.690+0000 7fbd1eac2640 1 -- 172.21.15.154:0/1405680193 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f07a453000 msgr2=0x55f07a1cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:34.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.690+0000 7fbd1eac2640 1 --2- 172.21.15.154:0/1405680193 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f07a453000 0x55f07a1cd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.690+0000 7fbd1eac2640 1 -- 172.21.15.154:0/1405680193 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f07a190ea0 con 0x55f07a453800 2023-09-23T03:21:34.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.690+0000 7fbd1eac2640 1 --2- 172.21.15.154:0/1405680193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f07a453800 0x55f07a1ce100 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x55f07a4ae0c0 tx=0x55f07a43e060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c227dd6a85f7eca server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:34.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.690+0000 7fbd1aaba640 1 -- 172.21.15.154:0/1405680193 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f07a1a6f20 con 0x55f07a453800 2023-09-23T03:21:34.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.690+0000 7fbd1aaba640 1 -- 172.21.15.154:0/1405680193 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f07aeea600 con 0x55f07a453800 2023-09-23T03:21:34.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.690+0000 7fbd1aaba640 1 -- 172.21.15.154:0/1405680193 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f07a1a7600 con 0x55f07a453800 2023-09-23T03:21:34.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.690+0000 7fbd21252100 1 -- 172.21.15.154:0/1405680193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f07a453800 msgr2=0x55f07a1ce100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:34.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.690+0000 7fbd21252100 1 --2- 172.21.15.154:0/1405680193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f07a453800 0x55f07a1ce100 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x55f07a4ae0c0 tx=0x55f07a43e060 comp rx=0 tx=0).stop 2023-09-23T03:21:34.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.691+0000 7fbd21252100 1 -- 172.21.15.154:0/1405680193 shutdown_connections 2023-09-23T03:21:34.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.691+0000 7fbd21252100 1 --2- 172.21.15.154:0/1405680193 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f07a453800 0x55f07a1ce100 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.691+0000 7fbd21252100 1 --2- 172.21.15.154:0/1405680193 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f07a453400 0x55f07a1cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.691+0000 7fbd21252100 1 --2- 172.21.15.154:0/1405680193 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f07a453000 0x55f07a1cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.691+0000 7fbd21252100 1 -- 172.21.15.154:0/1405680193 >> 172.21.15.154:0/1405680193 conn(0x55f07a452c00 msgr2=0x55f07aed5000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:34.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.691+0000 7fbd21252100 1 -- 172.21.15.154:0/1405680193 shutdown_connections 2023-09-23T03:21:34.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.691+0000 7fbd21252100 1 -- 172.21.15.154:0/1405680193 wait complete. 2023-09-23T03:21:34.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.691+0000 7fbd21252100 1 Processor -- start 2023-09-23T03:21:34.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.692+0000 7fbd21252100 1 -- start start 2023-09-23T03:21:34.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.692+0000 7fbd21252100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f07a453000 0x55f07a1cf180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.692+0000 7fbd21252100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f07a453400 0x55f07a1cec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.692+0000 7fbd21252100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f07a453800 0x55f07a1ce680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.692+0000 7fbd21252100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f07a22ac80 con 0x55f07a453800 2023-09-23T03:21:34.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.692+0000 7fbd21252100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f07a1a8140 con 0x55f07a453400 2023-09-23T03:21:34.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.692+0000 7fbd21252100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f07a1a8b40 con 0x55f07a453000 2023-09-23T03:21:34.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.692+0000 7fbd1eac2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f07a453800 0x55f07a1ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.693+0000 7fbd1eac2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f07a453800 0x55f07a1ce680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60290/0 (socket says 172.21.15.154:60290) 2023-09-23T03:21:34.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.693+0000 7fbd1eac2640 1 -- 172.21.15.154:0/3915470323 learned_addr learned my addr 172.21.15.154:0/3915470323 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:34.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.693+0000 7fbd1fac4640 1 --2- 172.21.15.154:0/3915470323 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f07a453000 0x55f07a1cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.693+0000 7fbd1f2c3640 1 --2- 172.21.15.154:0/3915470323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f07a453400 0x55f07a1cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.693+0000 7fbd1eac2640 1 -- 172.21.15.154:0/3915470323 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f07a453000 msgr2=0x55f07a1cf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:34.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.693+0000 7fbd1eac2640 1 --2- 172.21.15.154:0/3915470323 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f07a453000 0x55f07a1cf180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.693+0000 7fbd1eac2640 1 -- 172.21.15.154:0/3915470323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f07a453400 msgr2=0x55f07a1cec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:34.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.693+0000 7fbd1eac2640 1 --2- 172.21.15.154:0/3915470323 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f07a453400 0x55f07a1cec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.693+0000 7fbd1eac2640 1 -- 172.21.15.154:0/3915470323 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f07a190b60 con 0x55f07a453800 2023-09-23T03:21:34.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.693+0000 7fbd1eac2640 1 --2- 172.21.15.154:0/3915470323 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f07a453800 0x55f07a1ce680 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x55f07a4aed80 tx=0x55f07a45ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:34.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.693+0000 7fbd192b7640 1 -- 172.21.15.154:0/3915470323 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f07a1a71e0 con 0x55f07a453800 2023-09-23T03:21:34.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.693+0000 7fbd192b7640 1 -- 172.21.15.154:0/3915470323 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f07aeea780 con 0x55f07a453800 2023-09-23T03:21:34.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.693+0000 7fbd192b7640 1 -- 172.21.15.154:0/3915470323 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f07a1a6dc0 con 0x55f07a453800 2023-09-23T03:21:34.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.693+0000 7fbd21252100 1 -- 172.21.15.154:0/3915470323 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f07a190340 con 0x55f07a453800 2023-09-23T03:21:34.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.694+0000 7fbd21252100 1 -- 172.21.15.154:0/3915470323 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f07a191520 con 0x55f07a453800 2023-09-23T03:21:34.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.694+0000 7fbd21252100 4 main: resolving pool 8 2023-09-23T03:21:34.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.695+0000 7fbd192b7640 1 -- 172.21.15.154:0/3915470323 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x55f07a18e840 con 0x55f07a453800 2023-09-23T03:21:34.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.696+0000 7fbd192b7640 1 --2- 172.21.15.154:0/3915470323 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55f07a5b0000 0x55f07a1cf700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.696+0000 7fbd192b7640 1 -- 172.21.15.154:0/3915470323 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x55f07a188540 con 0x55f07a453800 2023-09-23T03:21:34.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.696+0000 7fbd21252100 4 main: creating IoCtx.. 2023-09-23T03:21:34.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.696+0000 7fbd21252100 1 --2- 172.21.15.154:0/3915470323 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55f07a5b0400 0x55f07a1ce100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.696+0000 7fbd21252100 1 -- 172.21.15.154:0/3915470323 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:1 8.d 8:b50e409b:::mds_snaptable:head [writefull 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55f07a1b9880 con 0x55f07a5b0400 2023-09-23T03:21:34.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.696+0000 7fbd1fac4640 1 --2- 172.21.15.154:0/3915470323 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55f07a5b0000 0x55f07a1cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.697+0000 7fbd1f2c3640 1 --2- 172.21.15.154:0/3915470323 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55f07a5b0400 0x55f07a1ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.697+0000 7fbd1fac4640 1 --2- 172.21.15.154:0/3915470323 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55f07a5b0000 0x55f07a1cf700 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x55f07a229350 tx=0x55f07a43e1e0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:34.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.698+0000 7fbd1f2c3640 1 --2- 172.21.15.154:0/3915470323 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55f07a5b0400 0x55f07a1ce100 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:34.703 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:21:34.703 INFO:teuthology.orchestra.run.smithi154.stdout: "result": 0 2023-09-23T03:21:34.703 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:21:34.703 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:34.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.702+0000 7fbd1f2c3640 1 -- 172.21.15.154:0/3915470323 <== osd.5 v2:172.21.15.190:6800/882536885 1 ==== osd_op_reply(1 mds_snaptable [writefull 0~70] v67'3 uv3 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x55f07affab40 con 0x55f07a5b0400 2023-09-23T03:21:34.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.702+0000 7fbd21252100 1 -- 172.21.15.154:0/3915470323 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55f07a5b0400 msgr2=0x55f07a1ce100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:34.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.702+0000 7fbd21252100 1 --2- 172.21.15.154:0/3915470323 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55f07a5b0400 0x55f07a1ce100 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.702+0000 7fbd21252100 1 -- 172.21.15.154:0/3915470323 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55f07a5b0000 msgr2=0x55f07a1cf700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:34.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.702+0000 7fbd21252100 1 --2- 172.21.15.154:0/3915470323 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55f07a5b0000 0x55f07a1cf700 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x55f07a229350 tx=0x55f07a43e1e0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.702+0000 7fbd21252100 1 -- 172.21.15.154:0/3915470323 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f07a453800 msgr2=0x55f07a1ce680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:34.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.702+0000 7fbd21252100 1 --2- 172.21.15.154:0/3915470323 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f07a453800 0x55f07a1ce680 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x55f07a4aed80 tx=0x55f07a45ac00 comp rx=0 tx=0).stop 2023-09-23T03:21:34.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.702+0000 7fbd1f2c3640 1 -- 172.21.15.154:0/3915470323 reap_dead start 2023-09-23T03:21:34.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.702+0000 7fbd21252100 1 -- 172.21.15.154:0/3915470323 shutdown_connections 2023-09-23T03:21:34.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.702+0000 7fbd21252100 1 -- 172.21.15.154:0/3915470323 >> 172.21.15.154:0/3915470323 conn(0x55f07a452c00 msgr2=0x55f07aed5000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:34.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.702+0000 7fbd21252100 1 -- 172.21.15.154:0/3915470323 shutdown_connections 2023-09-23T03:21:34.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.702+0000 7fbd21252100 1 -- 172.21.15.154:0/3915470323 wait complete. 2023-09-23T03:21:34.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.703+0000 7fbd21252100 1 -- 172.21.15.154:0/3647925275 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f07a15d800 msgr2=0x55f07a1ccb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:34.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.703+0000 7fbd21252100 1 --2- 172.21.15.154:0/3647925275 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f07a15d800 0x55f07a1ccb00 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x55f07af70bd0 tx=0x55f07a43e420 comp rx=0 tx=0).stop 2023-09-23T03:21:34.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.703+0000 7fbd21252100 1 -- 172.21.15.154:0/3647925275 shutdown_connections 2023-09-23T03:21:34.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.703+0000 7fbd21252100 1 --2- 172.21.15.154:0/3647925275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f07a15dc00 0x55f07a1cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.703+0000 7fbd21252100 1 --2- 172.21.15.154:0/3647925275 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f07a15d800 0x55f07a1ccb00 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.703+0000 7fbd21252100 1 --2- 172.21.15.154:0/3647925275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f07a15cc00 0x55f07a1cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.703+0000 7fbd21252100 1 -- 172.21.15.154:0/3647925275 >> 172.21.15.154:0/3647925275 conn(0x55f07a15d000 msgr2=0x55f07aed4800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:34.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.703+0000 7fbd21252100 1 -- 172.21.15.154:0/3647925275 shutdown_connections 2023-09-23T03:21:34.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.703+0000 7fbd21252100 1 -- 172.21.15.154:0/3647925275 wait complete. 2023-09-23T03:21:34.709 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'snap'] in time 0:00:00.108841, result: { "result": 0 } 2023-09-23T03:21:34.709 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset inode 2023-09-23T03:21:34.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.800+0000 7fe758a3b100 1 -- start start 2023-09-23T03:21:34.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.800+0000 7fe758a3b100 1 Processor -- start 2023-09-23T03:21:34.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.801+0000 7fe758a3b100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55971e53c400 0x55971d7e1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.801+0000 7fe758a3b100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55971e53dc00 0x55971d7e0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.801+0000 7fe758a3b100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55971e62a000 0x55971d7e1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.801+0000 7fe758a3b100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55971d7bc140 con 0x55971e53dc00 2023-09-23T03:21:34.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.801+0000 7fe758a3b100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55971d7bd180 con 0x55971e62a000 2023-09-23T03:21:34.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.801+0000 7fe758a3b100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55971d83edc0 con 0x55971e53c400 2023-09-23T03:21:34.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.801+0000 7fe756204640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55971e53dc00 0x55971d7e0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.801+0000 7fe756204640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55971e53dc00 0x55971d7e0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60310/0 (socket says 172.21.15.154:60310) 2023-09-23T03:21:34.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.801+0000 7fe756204640 1 -- 172.21.15.154:0/2846582624 learned_addr learned my addr 172.21.15.154:0/2846582624 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:34.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.801+0000 7fe756204640 1 -- 172.21.15.154:0/2846582624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55971e53c400 msgr2=0x55971d7e1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:34.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.801+0000 7fe756204640 1 --2- 172.21.15.154:0/2846582624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55971e53c400 0x55971d7e1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.801+0000 7fe756204640 1 -- 172.21.15.154:0/2846582624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55971e62a000 msgr2=0x55971d7e1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:34.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.802+0000 7fe756204640 1 --2- 172.21.15.154:0/2846582624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55971e62a000 0x55971d7e1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.802+0000 7fe756204640 1 -- 172.21.15.154:0/2846582624 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55971d7a4ea0 con 0x55971e53dc00 2023-09-23T03:21:34.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.802+0000 7fe756204640 1 --2- 172.21.15.154:0/2846582624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55971e53dc00 0x55971d7e0b00 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x55971e4d5f50 tx=0x55971e630120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bcd06a7f3ae4feb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:34.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.802+0000 7fe754200640 1 -- 172.21.15.154:0/2846582624 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55971d7bb1e0 con 0x55971e53dc00 2023-09-23T03:21:34.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.802+0000 7fe754200640 1 -- 172.21.15.154:0/2846582624 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55971d796a80 con 0x55971e53dc00 2023-09-23T03:21:34.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.802+0000 7fe754200640 1 -- 172.21.15.154:0/2846582624 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55971d7bb600 con 0x55971e53dc00 2023-09-23T03:21:34.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.802+0000 7fe758a3b100 1 -- 172.21.15.154:0/2846582624 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55971d7a4000 con 0x55971e53dc00 2023-09-23T03:21:34.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.802+0000 7fe758a3b100 1 -- 172.21.15.154:0/2846582624 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55971d7a49c0 con 0x55971e53dc00 2023-09-23T03:21:34.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.802+0000 7fe758a3b100 4 waiting for MDS map... 2023-09-23T03:21:34.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.802+0000 7fe754200640 1 -- 172.21.15.154:0/2846582624 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x55971d79c380 con 0x55971e53dc00 2023-09-23T03:21:34.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.803+0000 7fe754200640 1 -- 172.21.15.154:0/2846582624 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x55971e63a000 con 0x55971e53dc00 2023-09-23T03:21:34.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.803+0000 7fe758a3b100 4 Got MDS map 47 2023-09-23T03:21:34.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.803+0000 7fe758a3b100 10 main: main 2023-09-23T03:21:34.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.803+0000 7fe758a3b100 4 main: connecting to RADOS... 2023-09-23T03:21:34.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.804+0000 7fe758a3b100 1 Processor -- start 2023-09-23T03:21:34.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.804+0000 7fe758a3b100 1 -- start start 2023-09-23T03:21:34.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.804+0000 7fe758a3b100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55971e62a800 0x55971d7e0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.805+0000 7fe758a3b100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55971e62ac00 0x55971d7e2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.805+0000 7fe758a3b100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55971e62b000 0x55971d7e2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.805+0000 7fe758a3b100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55971d7bcb40 con 0x55971e62b000 2023-09-23T03:21:34.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.805+0000 7fe758a3b100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55971d7bca00 con 0x55971e62a800 2023-09-23T03:21:34.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.805+0000 7fe758a3b100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55971d7bc000 con 0x55971e62ac00 2023-09-23T03:21:34.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.805+0000 7fe756204640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55971e62b000 0x55971d7e2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.805+0000 7fe756204640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55971e62b000 0x55971d7e2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60326/0 (socket says 172.21.15.154:60326) 2023-09-23T03:21:34.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.805+0000 7fe756204640 1 -- 172.21.15.154:0/2638551299 learned_addr learned my addr 172.21.15.154:0/2638551299 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:34.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.805+0000 7fe757206640 1 --2- 172.21.15.154:0/2638551299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55971e62a800 0x55971d7e0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.805+0000 7fe756204640 1 -- 172.21.15.154:0/2638551299 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55971e62ac00 msgr2=0x55971d7e2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:34.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.805+0000 7fe756204640 1 --2- 172.21.15.154:0/2638551299 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55971e62ac00 0x55971d7e2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.805+0000 7fe756204640 1 -- 172.21.15.154:0/2638551299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55971e62a800 msgr2=0x55971d7e0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:34.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.805+0000 7fe756204640 1 --2- 172.21.15.154:0/2638551299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55971e62a800 0x55971d7e0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.805+0000 7fe756204640 1 -- 172.21.15.154:0/2638551299 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55971d7a4ea0 con 0x55971e62b000 2023-09-23T03:21:34.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.805+0000 7fe756204640 1 --2- 172.21.15.154:0/2638551299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55971e62b000 0x55971d7e2680 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x55971e4ffb90 tx=0x55971e6301e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d6af17e9aa3cd8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:34.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.806+0000 7fe7521fc640 1 -- 172.21.15.154:0/2638551299 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55971d7bb080 con 0x55971e62b000 2023-09-23T03:21:34.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.806+0000 7fe7521fc640 1 -- 172.21.15.154:0/2638551299 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55971e636180 con 0x55971e62b000 2023-09-23T03:21:34.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.806+0000 7fe7521fc640 1 -- 172.21.15.154:0/2638551299 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55971d7baf20 con 0x55971e62b000 2023-09-23T03:21:34.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.806+0000 7fe758a3b100 1 -- 172.21.15.154:0/2638551299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55971e62b000 msgr2=0x55971d7e2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:34.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.806+0000 7fe758a3b100 1 --2- 172.21.15.154:0/2638551299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55971e62b000 0x55971d7e2680 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x55971e4ffb90 tx=0x55971e6301e0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.806+0000 7fe758a3b100 1 -- 172.21.15.154:0/2638551299 shutdown_connections 2023-09-23T03:21:34.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.806+0000 7fe758a3b100 1 --2- 172.21.15.154:0/2638551299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55971e62b000 0x55971d7e2680 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.806+0000 7fe758a3b100 1 --2- 172.21.15.154:0/2638551299 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55971e62ac00 0x55971d7e2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.806+0000 7fe758a3b100 1 --2- 172.21.15.154:0/2638551299 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55971e62a800 0x55971d7e0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.806+0000 7fe758a3b100 1 -- 172.21.15.154:0/2638551299 >> 172.21.15.154:0/2638551299 conn(0x55971e62a400 msgr2=0x55971e4e9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:34.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.806+0000 7fe758a3b100 1 -- 172.21.15.154:0/2638551299 shutdown_connections 2023-09-23T03:21:34.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.806+0000 7fe758a3b100 1 -- 172.21.15.154:0/2638551299 wait complete. 2023-09-23T03:21:34.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.807+0000 7fe758a3b100 1 Processor -- start 2023-09-23T03:21:34.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.807+0000 7fe758a3b100 1 -- start start 2023-09-23T03:21:34.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.808+0000 7fe758a3b100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55971e62a800 0x55971d7e3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.808+0000 7fe758a3b100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55971e62ac00 0x55971d7e3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.808+0000 7fe758a3b100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55971e62b000 0x55971d7e2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.808+0000 7fe758a3b100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55971d83edc0 con 0x55971e62b000 2023-09-23T03:21:34.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.808+0000 7fe758a3b100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55971d7bd180 con 0x55971e62ac00 2023-09-23T03:21:34.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.808+0000 7fe758a3b100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55971d7bc140 con 0x55971e62a800 2023-09-23T03:21:34.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.808+0000 7fe756204640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55971e62b000 0x55971d7e2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.808+0000 7fe756204640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55971e62b000 0x55971d7e2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60328/0 (socket says 172.21.15.154:60328) 2023-09-23T03:21:34.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.808+0000 7fe756204640 1 -- 172.21.15.154:0/3979212538 learned_addr learned my addr 172.21.15.154:0/3979212538 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:34.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.809+0000 7fe756a05640 1 --2- 172.21.15.154:0/3979212538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55971e62ac00 0x55971d7e3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.809+0000 7fe756204640 1 -- 172.21.15.154:0/3979212538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55971e62a800 msgr2=0x55971d7e3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:34.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.809+0000 7fe756204640 1 --2- 172.21.15.154:0/3979212538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55971e62a800 0x55971d7e3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.809+0000 7fe756204640 1 -- 172.21.15.154:0/3979212538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55971e62ac00 msgr2=0x55971d7e3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:34.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.809+0000 7fe756204640 1 --2- 172.21.15.154:0/3979212538 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55971e62ac00 0x55971d7e3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.809+0000 7fe756204640 1 -- 172.21.15.154:0/3979212538 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55971d7a5520 con 0x55971e62b000 2023-09-23T03:21:34.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.809+0000 7fe756204640 1 --2- 172.21.15.154:0/3979212538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55971e62b000 0x55971d7e2c00 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x55971e69f200 tx=0x55971e630060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:34.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.809+0000 7fe7509f9640 1 -- 172.21.15.154:0/3979212538 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55971d7bb080 con 0x55971e62b000 2023-09-23T03:21:34.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.809+0000 7fe7509f9640 1 -- 172.21.15.154:0/3979212538 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55971e636000 con 0x55971e62b000 2023-09-23T03:21:34.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.809+0000 7fe758a3b100 1 -- 172.21.15.154:0/3979212538 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55971d7a56c0 con 0x55971e62b000 2023-09-23T03:21:34.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.809+0000 7fe7509f9640 1 -- 172.21.15.154:0/3979212538 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55971d7baf20 con 0x55971e62b000 2023-09-23T03:21:34.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.809+0000 7fe758a3b100 1 -- 172.21.15.154:0/3979212538 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55971d7a5860 con 0x55971e62b000 2023-09-23T03:21:34.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.809+0000 7fe758a3b100 4 main: resolving pool 8 2023-09-23T03:21:34.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.811+0000 7fe7509f9640 1 -- 172.21.15.154:0/3979212538 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x55971d7a3080 con 0x55971e62b000 2023-09-23T03:21:34.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.812+0000 7fe7509f9640 1 --2- 172.21.15.154:0/3979212538 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55971dac0000 0x55971dac6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.812+0000 7fe7509f9640 1 -- 172.21.15.154:0/3979212538 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x55971d79c540 con 0x55971e62b000 2023-09-23T03:21:34.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.812+0000 7fe758a3b100 4 main: creating IoCtx.. 2023-09-23T03:21:34.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.812+0000 7fe758a3b100 1 --2- 172.21.15.154:0/3979212538 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55971dac0400 0x55971d7e2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.812+0000 7fe758a3b100 1 -- 172.21.15.154:0/3979212538 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:1 8.13 8:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55971d7cd880 con 0x55971dac0400 2023-09-23T03:21:34.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.812+0000 7fe757206640 1 --2- 172.21.15.154:0/3979212538 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55971dac0000 0x55971dac6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.812+0000 7fe756a05640 1 --2- 172.21.15.154:0/3979212538 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55971dac0400 0x55971d7e2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.813+0000 7fe757206640 1 --2- 172.21.15.154:0/3979212538 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55971dac0000 0x55971dac6000 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x55971d83d830 tx=0x55971e64bda0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:34.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.813+0000 7fe756a05640 1 --2- 172.21.15.154:0/3979212538 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55971dac0400 0x55971d7e2680 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:34.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.815+0000 7fe756a05640 1 -- 172.21.15.154:0/3979212538 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v67'6 uv6 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x55971e620b40 con 0x55971dac0400 2023-09-23T03:21:34.817 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:21:34.817 INFO:teuthology.orchestra.run.smithi154.stdout: "0": { 2023-09-23T03:21:34.818 INFO:teuthology.orchestra.run.smithi154.stdout: "data": {}, 2023-09-23T03:21:34.818 INFO:teuthology.orchestra.run.smithi154.stdout: "result": 0 2023-09-23T03:21:34.818 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:21:34.818 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:21:34.818 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:34.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.815+0000 7fe758a3b100 1 -- 172.21.15.154:0/3979212538 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55971dac0400 msgr2=0x55971d7e2680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:34.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.815+0000 7fe758a3b100 1 --2- 172.21.15.154:0/3979212538 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55971dac0400 0x55971d7e2680 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.815+0000 7fe758a3b100 1 -- 172.21.15.154:0/3979212538 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55971dac0000 msgr2=0x55971dac6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:34.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.815+0000 7fe758a3b100 1 --2- 172.21.15.154:0/3979212538 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55971dac0000 0x55971dac6000 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x55971d83d830 tx=0x55971e64bda0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.815+0000 7fe758a3b100 1 -- 172.21.15.154:0/3979212538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55971e62b000 msgr2=0x55971d7e2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:34.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.815+0000 7fe758a3b100 1 --2- 172.21.15.154:0/3979212538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55971e62b000 0x55971d7e2c00 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x55971e69f200 tx=0x55971e630060 comp rx=0 tx=0).stop 2023-09-23T03:21:34.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.815+0000 7fe756a05640 1 -- 172.21.15.154:0/3979212538 reap_dead start 2023-09-23T03:21:34.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.815+0000 7fe758a3b100 1 -- 172.21.15.154:0/3979212538 shutdown_connections 2023-09-23T03:21:34.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.815+0000 7fe758a3b100 1 -- 172.21.15.154:0/3979212538 >> 172.21.15.154:0/3979212538 conn(0x55971e62a400 msgr2=0x55971e4e9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:34.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.815+0000 7fe758a3b100 1 -- 172.21.15.154:0/3979212538 shutdown_connections 2023-09-23T03:21:34.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.815+0000 7fe758a3b100 1 -- 172.21.15.154:0/3979212538 wait complete. 2023-09-23T03:21:34.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.816+0000 7fe758a3b100 1 -- 172.21.15.154:0/2846582624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55971e53dc00 msgr2=0x55971d7e0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:34.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.816+0000 7fe758a3b100 1 --2- 172.21.15.154:0/2846582624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55971e53dc00 0x55971d7e0b00 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x55971e4d5f50 tx=0x55971e630120 comp rx=0 tx=0).stop 2023-09-23T03:21:34.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.816+0000 7fe758a3b100 1 -- 172.21.15.154:0/2846582624 shutdown_connections 2023-09-23T03:21:34.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.816+0000 7fe758a3b100 1 --2- 172.21.15.154:0/2846582624 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55971e62a000 0x55971d7e1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.816+0000 7fe758a3b100 1 --2- 172.21.15.154:0/2846582624 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55971e53dc00 0x55971d7e0b00 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.816+0000 7fe758a3b100 1 --2- 172.21.15.154:0/2846582624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55971e53c400 0x55971d7e1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.816+0000 7fe758a3b100 1 -- 172.21.15.154:0/2846582624 >> 172.21.15.154:0/2846582624 conn(0x55971d771000 msgr2=0x55971e4e8000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:34.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.816+0000 7fe758a3b100 1 -- 172.21.15.154:0/2846582624 shutdown_connections 2023-09-23T03:21:34.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.816+0000 7fe758a3b100 1 -- 172.21.15.154:0/2846582624 wait complete. 2023-09-23T03:21:34.822 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'inode'] in time 0:00:00.112886, result: { "0": { "data": {}, "result": 0 } } 2023-09-23T03:21:34.822 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2023-09-23T03:21:34.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.906+0000 7f62f8a15100 1 -- start start 2023-09-23T03:21:34.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.906+0000 7f62f8a15100 1 Processor -- start 2023-09-23T03:21:34.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.907+0000 7f62f8a15100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b3b0c400 0x5633b3a9f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.907+0000 7f62f8a15100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633b3b0d000 0x5633b3a9eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.907+0000 7f62f8a15100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5633b3b0d400 0x5633b3a9fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.907+0000 7f62f8a15100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b3a7b180 con 0x5633b3b0d000 2023-09-23T03:21:34.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.907+0000 7f62f8a15100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b3afcdc0 con 0x5633b3b0c400 2023-09-23T03:21:34.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.907+0000 7f62f8a15100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5633b3a7a140 con 0x5633b3b0d400 2023-09-23T03:21:34.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.907+0000 7f62f5d95640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633b3b0d000 0x5633b3a9eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.907+0000 7f62f5d95640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633b3b0d000 0x5633b3a9eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60346/0 (socket says 172.21.15.154:60346) 2023-09-23T03:21:34.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.907+0000 7f62f5d95640 1 -- 172.21.15.154:0/434966890 learned_addr learned my addr 172.21.15.154:0/434966890 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:34.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.907+0000 7f62f5d95640 1 -- 172.21.15.154:0/434966890 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5633b3b0d400 msgr2=0x5633b3a9fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:34.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.907+0000 7f62f5d95640 1 --2- 172.21.15.154:0/434966890 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5633b3b0d400 0x5633b3a9fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.907+0000 7f62f5d95640 1 -- 172.21.15.154:0/434966890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b3b0c400 msgr2=0x5633b3a9f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:34.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.907+0000 7f62f6596640 1 --2- 172.21.15.154:0/434966890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b3b0c400 0x5633b3a9f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.907+0000 7f62f5d95640 1 --2- 172.21.15.154:0/434966890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b3b0c400 0x5633b3a9f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.907+0000 7f62f5d95640 1 -- 172.21.15.154:0/434966890 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633b3a62000 con 0x5633b3b0d000 2023-09-23T03:21:34.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.908+0000 7f62f5d95640 1 --2- 172.21.15.154:0/434966890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633b3b0d000 0x5633b3a9eb00 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x5633b483e330 tx=0x5633b3d12120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d04c39043738ed4e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:34.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.908+0000 7f62f3d91640 1 -- 172.21.15.154:0/434966890 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5633b3a79080 con 0x5633b3b0d000 2023-09-23T03:21:34.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.908+0000 7f62f3d91640 1 -- 172.21.15.154:0/434966890 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5633b3d1e000 con 0x5633b3b0d000 2023-09-23T03:21:34.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.908+0000 7f62f3d91640 1 -- 172.21.15.154:0/434966890 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5633b3a79600 con 0x5633b3b0d000 2023-09-23T03:21:34.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.908+0000 7f62f8a15100 1 -- 172.21.15.154:0/434966890 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5633b3a62340 con 0x5633b3b0d000 2023-09-23T03:21:34.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.908+0000 7f62f8a15100 1 -- 172.21.15.154:0/434966890 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5633b3a631e0 con 0x5633b3b0d000 2023-09-23T03:21:34.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.908+0000 7f62f8a15100 4 waiting for MDS map... 2023-09-23T03:21:34.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.908+0000 7f62f3d91640 1 -- 172.21.15.154:0/434966890 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x5633b3a5a380 con 0x5633b3b0d000 2023-09-23T03:21:34.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.908+0000 7f62f3d91640 1 -- 172.21.15.154:0/434966890 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x5633b3d22000 con 0x5633b3b0d000 2023-09-23T03:21:34.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.908+0000 7f62f8a15100 4 Got MDS map 47 2023-09-23T03:21:34.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.909+0000 7f62f8a15100 10 main: JournalTool::main 2023-09-23T03:21:34.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.909+0000 7f62f8a15100 4 main: JournalTool: connecting to RADOS... 2023-09-23T03:21:34.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.909+0000 7f62f8a15100 1 Processor -- start 2023-09-23T03:21:34.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.909+0000 7f62f8a15100 1 -- start start 2023-09-23T03:21:34.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.910+0000 7f62f8a15100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5633b3b0dc00 0x5633b3a9f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.910+0000 7f62f8a15100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b3d7c000 0x5633b3aa0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.910+0000 7f62f8a15100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633b3d7c400 0x5633b3aa0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.910+0000 7f62f8a15100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b3a7ab40 con 0x5633b3d7c400 2023-09-23T03:21:34.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.910+0000 7f62f8a15100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b3a7aa00 con 0x5633b3d7c000 2023-09-23T03:21:34.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.910+0000 7f62f8a15100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5633b3afcdc0 con 0x5633b3b0dc00 2023-09-23T03:21:34.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.911+0000 7f62f5d95640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633b3d7c400 0x5633b3aa0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.911+0000 7f62f5d95640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633b3d7c400 0x5633b3aa0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60362/0 (socket says 172.21.15.154:60362) 2023-09-23T03:21:34.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.911+0000 7f62f5d95640 1 -- 172.21.15.154:0/1135996611 learned_addr learned my addr 172.21.15.154:0/1135996611 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:34.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.911+0000 7f62f6596640 1 --2- 172.21.15.154:0/1135996611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b3d7c000 0x5633b3aa0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.911+0000 7f62f6d97640 1 --2- 172.21.15.154:0/1135996611 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5633b3b0dc00 0x5633b3a9f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.911+0000 7f62f5d95640 1 -- 172.21.15.154:0/1135996611 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5633b3b0dc00 msgr2=0x5633b3a9f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:34.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.911+0000 7f62f5d95640 1 --2- 172.21.15.154:0/1135996611 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5633b3b0dc00 0x5633b3a9f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.911+0000 7f62f5d95640 1 -- 172.21.15.154:0/1135996611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b3d7c000 msgr2=0x5633b3aa0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:34.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.911+0000 7f62f5d95640 1 --2- 172.21.15.154:0/1135996611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b3d7c000 0x5633b3aa0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.911+0000 7f62f5d95640 1 -- 172.21.15.154:0/1135996611 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633b3a629c0 con 0x5633b3d7c400 2023-09-23T03:21:34.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.911+0000 7f62f6596640 1 --2- 172.21.15.154:0/1135996611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b3d7c000 0x5633b3aa0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:34.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.911+0000 7f62f5d95640 1 --2- 172.21.15.154:0/1135996611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633b3d7c400 0x5633b3aa0680 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x5633b483f530 tx=0x5633b3d121e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53a8566b530b9cd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:34.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.911+0000 7f62f1d8d640 1 -- 172.21.15.154:0/1135996611 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5633b3a791e0 con 0x5633b3d7c400 2023-09-23T03:21:34.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.911+0000 7f62f1d8d640 1 -- 172.21.15.154:0/1135996611 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5633b3d1ed80 con 0x5633b3d7c400 2023-09-23T03:21:34.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.911+0000 7f62f1d8d640 1 -- 172.21.15.154:0/1135996611 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5633b3a79080 con 0x5633b3d7c400 2023-09-23T03:21:34.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.912+0000 7f62f8a15100 1 -- 172.21.15.154:0/1135996611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633b3d7c400 msgr2=0x5633b3aa0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:34.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.912+0000 7f62f8a15100 1 --2- 172.21.15.154:0/1135996611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633b3d7c400 0x5633b3aa0680 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x5633b483f530 tx=0x5633b3d121e0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.912+0000 7f62f8a15100 1 -- 172.21.15.154:0/1135996611 shutdown_connections 2023-09-23T03:21:34.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.912+0000 7f62f8a15100 1 --2- 172.21.15.154:0/1135996611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633b3d7c400 0x5633b3aa0680 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.912+0000 7f62f8a15100 1 --2- 172.21.15.154:0/1135996611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b3d7c000 0x5633b3aa0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.912+0000 7f62f8a15100 1 --2- 172.21.15.154:0/1135996611 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5633b3b0dc00 0x5633b3a9f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.912+0000 7f62f8a15100 1 -- 172.21.15.154:0/1135996611 >> 172.21.15.154:0/1135996611 conn(0x5633b3b0d800 msgr2=0x5633b47a7000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:34.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.912+0000 7f62f8a15100 1 -- 172.21.15.154:0/1135996611 shutdown_connections 2023-09-23T03:21:34.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.912+0000 7f62f8a15100 1 -- 172.21.15.154:0/1135996611 wait complete. 2023-09-23T03:21:34.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.913+0000 7f62f8a15100 1 Processor -- start 2023-09-23T03:21:34.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.913+0000 7f62f8a15100 1 -- start start 2023-09-23T03:21:34.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.914+0000 7f62f8a15100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5633b3b0d800 0x5633b3aa1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.914+0000 7f62f8a15100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b3d7d000 0x5633b3aa1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.914+0000 7f62f8a15100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633b3d7cc00 0x5633b3aa0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.914+0000 7f62f8a15100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b3afd180 con 0x5633b3d7cc00 2023-09-23T03:21:34.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.914+0000 7f62f8a15100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b3afcc80 con 0x5633b3d7d000 2023-09-23T03:21:34.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.914+0000 7f62f8a15100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5633b3a7a140 con 0x5633b3b0d800 2023-09-23T03:21:34.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.914+0000 7f62f5d95640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633b3d7cc00 0x5633b3aa0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.914+0000 7f62f5d95640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633b3d7cc00 0x5633b3aa0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60372/0 (socket says 172.21.15.154:60372) 2023-09-23T03:21:34.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.914+0000 7f62f5d95640 1 -- 172.21.15.154:0/2148138337 learned_addr learned my addr 172.21.15.154:0/2148138337 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:34.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.914+0000 7f62f5d95640 1 -- 172.21.15.154:0/2148138337 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5633b3b0d800 msgr2=0x5633b3aa1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:34.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.914+0000 7f62f5d95640 1 --2- 172.21.15.154:0/2148138337 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5633b3b0d800 0x5633b3aa1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.914+0000 7f62f5d95640 1 -- 172.21.15.154:0/2148138337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b3d7d000 msgr2=0x5633b3aa1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:34.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.914+0000 7f62f6596640 1 --2- 172.21.15.154:0/2148138337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b3d7d000 0x5633b3aa1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.914+0000 7f62f5d95640 1 --2- 172.21.15.154:0/2148138337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b3d7d000 0x5633b3aa1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.914+0000 7f62f5d95640 1 -- 172.21.15.154:0/2148138337 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633b3a62000 con 0x5633b3d7cc00 2023-09-23T03:21:34.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.915+0000 7f62f5d95640 1 --2- 172.21.15.154:0/2148138337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633b3d7cc00 0x5633b3aa0c00 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x5633b3d33020 tx=0x5633b3d12060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:34.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.915+0000 7f62f058a640 1 -- 172.21.15.154:0/2148138337 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5633b3a79600 con 0x5633b3d7cc00 2023-09-23T03:21:34.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.915+0000 7f62f058a640 1 -- 172.21.15.154:0/2148138337 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5633b3d1f500 con 0x5633b3d7cc00 2023-09-23T03:21:34.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.915+0000 7f62f8a15100 1 -- 172.21.15.154:0/2148138337 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5633b3a62ea0 con 0x5633b3d7cc00 2023-09-23T03:21:34.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.915+0000 7f62f058a640 1 -- 172.21.15.154:0/2148138337 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5633b3a78f20 con 0x5633b3d7cc00 2023-09-23T03:21:34.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.915+0000 7f62f8a15100 1 -- 172.21.15.154:0/2148138337 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5633b3a63520 con 0x5633b3d7cc00 2023-09-23T03:21:34.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.915+0000 7f62f8a15100 4 main: JournalTool: resolving pool 8 2023-09-23T03:21:34.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.917+0000 7f62f058a640 1 -- 172.21.15.154:0/2148138337 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x5633b3a61080 con 0x5633b3d7cc00 2023-09-23T03:21:34.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.917+0000 7f62f058a640 1 --2- 172.21.15.154:0/2148138337 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5633b3e70000 0x5633b3e76000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.918+0000 7f62f058a640 1 -- 172.21.15.154:0/2148138337 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x5633b3a5a540 con 0x5633b3d7cc00 2023-09-23T03:21:34.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.918+0000 7f62f8a15100 4 main: JournalTool: creating IoCtx.. 2023-09-23T03:21:34.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.918+0000 7f62f8a15100 4 main: Executing for rank 0 2023-09-23T03:21:34.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.918+0000 7f62f6d97640 1 --2- 172.21.15.154:0/2148138337 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5633b3e70000 0x5633b3e76000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.918+0000 7f62f6d97640 1 --2- 172.21.15.154:0/2148138337 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5633b3e70000 0x5633b3e76000 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x5633b3afbbc0 tx=0x5633b3e89800 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:34.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.919+0000 7f62f8a15100 1 -- start start 2023-09-23T03:21:34.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.919+0000 7f62f8a15100 1 Processor -- start 2023-09-23T03:21:34.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.920+0000 7f62f8a15100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5633b3d7c400 0x5633b3aa0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.920+0000 7f62f8a15100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633b3d7c000 0x5633b3aa0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.920+0000 7f62f8a15100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b3e71000 0x5633b3a9f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.920+0000 7f62f8a15100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b3a7b180 con 0x5633b3d7c000 2023-09-23T03:21:34.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.920+0000 7f62f8a15100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b3afcdc0 con 0x5633b3e71000 2023-09-23T03:21:34.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.920+0000 7f62f8a15100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5633b3afcc80 con 0x5633b3d7c400 2023-09-23T03:21:34.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.920+0000 7f62f6596640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633b3d7c000 0x5633b3aa0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.920+0000 7f62f6596640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633b3d7c000 0x5633b3aa0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60376/0 (socket says 172.21.15.154:60376) 2023-09-23T03:21:34.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.920+0000 7f62f6596640 1 -- 172.21.15.154:0/2059760263 learned_addr learned my addr 172.21.15.154:0/2059760263 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:34.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.920+0000 7f62f6d97640 1 --2- 172.21.15.154:0/2059760263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5633b3d7c400 0x5633b3aa0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.920+0000 7f62f5d95640 1 --2- 172.21.15.154:0/2059760263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b3e71000 0x5633b3a9f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.920+0000 7f62f6596640 1 -- 172.21.15.154:0/2059760263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5633b3d7c400 msgr2=0x5633b3aa0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:34.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.920+0000 7f62f6596640 1 --2- 172.21.15.154:0/2059760263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5633b3d7c400 0x5633b3aa0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.921+0000 7f62f6596640 1 -- 172.21.15.154:0/2059760263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b3e71000 msgr2=0x5633b3a9f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:34.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.921+0000 7f62f6596640 1 --2- 172.21.15.154:0/2059760263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b3e71000 0x5633b3a9f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.921+0000 7f62f6596640 1 -- 172.21.15.154:0/2059760263 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633b3db6680 con 0x5633b3d7c000 2023-09-23T03:21:34.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.921+0000 7f62f6596640 1 --2- 172.21.15.154:0/2059760263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633b3d7c000 0x5633b3aa0100 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x5633b3e43560 tx=0x5633b3e89920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8812e8ec02aa6f57 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:34.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.921+0000 7f62ed584640 1 -- 172.21.15.154:0/2059760263 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5633b3a79080 con 0x5633b3d7c000 2023-09-23T03:21:34.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.921+0000 7f62ed584640 1 -- 172.21.15.154:0/2059760263 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5633b3d1ea80 con 0x5633b3d7c000 2023-09-23T03:21:34.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.921+0000 7f62ed584640 1 -- 172.21.15.154:0/2059760263 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5633b3a78420 con 0x5633b3d7c000 2023-09-23T03:21:34.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.921+0000 7f62f8a15100 1 -- 172.21.15.154:0/2059760263 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5633b3db64e0 con 0x5633b3d7c000 2023-09-23T03:21:34.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.921+0000 7f62f8a15100 1 -- 172.21.15.154:0/2059760263 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5633b3db6340 con 0x5633b3d7c000 2023-09-23T03:21:34.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.921+0000 7f62f8a15100 4 waiting for MDS map... 2023-09-23T03:21:34.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.921+0000 7f62ed584640 1 -- 172.21.15.154:0/2059760263 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x5633b3a5a8c0 con 0x5633b3d7c000 2023-09-23T03:21:34.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.921+0000 7f62ed584640 1 -- 172.21.15.154:0/2059760263 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x5633b3ee4000 con 0x5633b3d7c000 2023-09-23T03:21:34.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.922+0000 7f62f8a15100 4 Got MDS map 47 2023-09-23T03:21:34.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.922+0000 7f62f8a15100 1 --2- 172.21.15.154:0/2059760263 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5633b3e71400 0x5633b3e76580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.922+0000 7f62f8a15100 1 -- 172.21.15.154:0/2059760263 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:1 8.f 8:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x5633b3f02000 con 0x5633b3e71400 2023-09-23T03:21:34.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.922+0000 7f62f6d97640 1 --2- 172.21.15.154:0/2059760263 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5633b3e71400 0x5633b3e76580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.922+0000 7f62f6d97640 1 --2- 172.21.15.154:0/2059760263 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5633b3e71400 0x5633b3e76580 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=87e5a3f24b4890e4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:34.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.924+0000 7f62f6d97640 1 -- 172.21.15.154:0/2059760263 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v67'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x5633b48d2b40 con 0x5633b3e71400 2023-09-23T03:21:34.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.924+0000 7f62f8a15100 1 --2- 172.21.15.154:0/2059760263 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5633b3e71800 0x5633b3e76b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.924+0000 7f62f8a15100 1 -- 172.21.15.154:0/2059760263 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:2 8.14 8:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x5633b3f02380 con 0x5633b3e71800 2023-09-23T03:21:34.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.925+0000 7f62f5d95640 1 --2- 172.21.15.154:0/2059760263 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5633b3e71800 0x5633b3e76b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.926+0000 7f62f5d95640 1 --2- 172.21.15.154:0/2059760263 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5633b3e71800 0x5633b3e76b00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=d6019284aa71d574 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:34.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.929+0000 7f62f5d95640 1 -- 172.21.15.154:0/2059760263 <== osd.4 v2:172.21.15.190:6808/717302641 1 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v67'104 uv104 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x5633b48d2d80 con 0x5633b3e71800 2023-09-23T03:21:34.930 INFO:teuthology.orchestra.run.smithi154.stdout:writing EResetJournal entry 2023-09-23T03:21:34.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.929+0000 7f62f8a15100 1 -- 172.21.15.154:0/2059760263 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:3 8.14 8:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x5633b3f02700 con 0x5633b3e71800 2023-09-23T03:21:34.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.929+0000 7f62f8a15100 1 -- 172.21.15.154:0/2059760263 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:4 8.1b 8:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x5633b3f02a80 con 0x5633b3e71800 2023-09-23T03:21:34.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.929+0000 7f62f8a15100 1 -- 172.21.15.154:0/2059760263 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:5 8.b 8:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x5633b3f02e00 con 0x5633b3e71400 2023-09-23T03:21:34.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.929+0000 7f62f8a15100 1 --2- 172.21.15.154:0/2059760263 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5633b3e71c00 0x5633b3e77080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:34.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.929+0000 7f62f8a15100 1 -- 172.21.15.154:0/2059760263 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:6 8.1f 8:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x5633b3f03180 con 0x5633b3e71c00 2023-09-23T03:21:34.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.929+0000 7f62f8a15100 1 -- 172.21.15.154:0/2059760263 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:7 8.1a 8:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x5633b3f03500 con 0x5633b3e71c00 2023-09-23T03:21:34.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.929+0000 7f62f8a15100 1 -- 172.21.15.154:0/2059760263 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:8 8.1e 8:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x5633b3f02000 con 0x5633b3e71400 2023-09-23T03:21:34.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.930+0000 7f62f6d97640 1 --2- 172.21.15.154:0/2059760263 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5633b3e71c00 0x5633b3e77080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:34.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.930+0000 7f62f6d97640 1 --2- 172.21.15.154:0/2059760263 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5633b3e71c00 0x5633b3e77080 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=e6fea9de968db4e5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:34.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.932+0000 7f62f6d97640 1 -- 172.21.15.154:0/2059760263 <== osd.3 v2:172.21.15.154:6808/1133688468 2 ==== osd_op_reply(5 200.00000003 [delete] v67'6 uv5 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5633b48d2b40 con 0x5633b3e71400 2023-09-23T03:21:34.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.933+0000 7f62f6d97640 1 -- 172.21.15.154:0/2059760263 <== osd.3 v2:172.21.15.154:6808/1133688468 3 ==== osd_op_reply(8 200.00000006 [delete] v67'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5633b48d2b40 con 0x5633b3e71400 2023-09-23T03:21:34.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.933+0000 7f62f6d97640 1 -- 172.21.15.154:0/2059760263 <== osd.0 v2:172.21.15.154:6800/2920938138 1 ==== osd_op_reply(7 200.00000005 [delete] v67'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5633b48d2b40 con 0x5633b3e71c00 2023-09-23T03:21:34.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.933+0000 7f62f5d95640 1 -- 172.21.15.154:0/2059760263 <== osd.4 v2:172.21.15.190:6808/717302641 2 ==== osd_op_reply(4 200.00000002 [delete] v67'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5633b48d2d80 con 0x5633b3e71800 2023-09-23T03:21:34.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.933+0000 7f62f6d97640 1 -- 172.21.15.154:0/2059760263 <== osd.0 v2:172.21.15.154:6800/2920938138 2 ==== osd_op_reply(6 200.00000004 [delete] v67'9 uv8 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5633b48d2b40 con 0x5633b3e71c00 2023-09-23T03:21:34.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.933+0000 7f62f5d95640 1 -- 172.21.15.154:0/2059760263 <== osd.4 v2:172.21.15.190:6808/717302641 3 ==== osd_op_reply(3 200.00000001 [delete] v67'105 uv104 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5633b48d2d80 con 0x5633b3e71800 2023-09-23T03:21:34.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.934+0000 7f62ebd81640 1 -- 172.21.15.154:0/2059760263 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:9 8.14 8:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x5633b3f02000 con 0x5633b3e71800 2023-09-23T03:21:34.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.937+0000 7f62f5d95640 1 -- 172.21.15.154:0/2059760263 <== osd.4 v2:172.21.15.190:6808/717302641 4 ==== osd_op_reply(9 200.00000001 [write 0~48] v67'106 uv106 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x5633b48d2d80 con 0x5633b3e71800 2023-09-23T03:21:34.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.937+0000 7f62f8a15100 4 Successfully wrote new journal pointer and header for rank 3:0 2023-09-23T03:21:34.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.937+0000 7f62f8a15100 1 -- 172.21.15.154:0/2059760263 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5633b3e71c00 msgr2=0x5633b3e77080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:34.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.937+0000 7f62f8a15100 1 --2- 172.21.15.154:0/2059760263 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5633b3e71c00 0x5633b3e77080 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.937+0000 7f62f8a15100 1 -- 172.21.15.154:0/2059760263 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5633b3e71400 msgr2=0x5633b3e76580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:34.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.937+0000 7f62f8a15100 1 --2- 172.21.15.154:0/2059760263 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5633b3e71400 0x5633b3e76580 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.937+0000 7f62f8a15100 1 -- 172.21.15.154:0/2059760263 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5633b3e71800 msgr2=0x5633b3e76b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:34.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.937+0000 7f62f8a15100 1 --2- 172.21.15.154:0/2059760263 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5633b3e71800 0x5633b3e76b00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.937+0000 7f62f6596640 1 -- 172.21.15.154:0/2059760263 reap_dead start 2023-09-23T03:21:34.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.937+0000 7f62f8a15100 1 -- 172.21.15.154:0/2059760263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633b3d7c000 msgr2=0x5633b3aa0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:34.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.937+0000 7f62f8a15100 1 --2- 172.21.15.154:0/2059760263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633b3d7c000 0x5633b3aa0100 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x5633b3e43560 tx=0x5633b3e89920 comp rx=0 tx=0).stop 2023-09-23T03:21:34.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.937+0000 7f62f8a15100 1 -- 172.21.15.154:0/2059760263 shutdown_connections 2023-09-23T03:21:34.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.937+0000 7f62f8a15100 1 --2- 172.21.15.154:0/2059760263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633b3d7c000 0x5633b3aa0100 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.937+0000 7f62f8a15100 1 -- 172.21.15.154:0/2059760263 >> 172.21.15.154:0/2059760263 conn(0x5633b3d7c800 msgr2=0x5633b47a7800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:34.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.937+0000 7f62f8a15100 1 -- 172.21.15.154:0/2059760263 shutdown_connections 2023-09-23T03:21:34.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.937+0000 7f62f8a15100 1 -- 172.21.15.154:0/2059760263 wait complete. 2023-09-23T03:21:34.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.938+0000 7f62f8a15100 1 -- 172.21.15.154:0/2148138337 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5633b3e70000 msgr2=0x5633b3e76000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:34.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.938+0000 7f62f8a15100 1 --2- 172.21.15.154:0/2148138337 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5633b3e70000 0x5633b3e76000 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x5633b3afbbc0 tx=0x5633b3e89800 comp rx=0 tx=0).stop 2023-09-23T03:21:34.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.938+0000 7f62f8a15100 1 -- 172.21.15.154:0/2148138337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633b3d7cc00 msgr2=0x5633b3aa0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:34.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.938+0000 7f62f8a15100 1 --2- 172.21.15.154:0/2148138337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633b3d7cc00 0x5633b3aa0c00 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x5633b3d33020 tx=0x5633b3d12060 comp rx=0 tx=0).stop 2023-09-23T03:21:34.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.938+0000 7f62f8a15100 1 -- 172.21.15.154:0/2148138337 shutdown_connections 2023-09-23T03:21:34.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.938+0000 7f62f8a15100 1 --2- 172.21.15.154:0/2148138337 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5633b3e70000 0x5633b3e76000 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.938+0000 7f62f8a15100 1 --2- 172.21.15.154:0/2148138337 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633b3d7cc00 0x5633b3aa0c00 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.938+0000 7f62f8a15100 1 --2- 172.21.15.154:0/2148138337 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b3d7d000 0x5633b3aa1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.938+0000 7f62f8a15100 1 --2- 172.21.15.154:0/2148138337 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5633b3b0d800 0x5633b3aa1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.938+0000 7f62f8a15100 1 -- 172.21.15.154:0/2148138337 >> 172.21.15.154:0/2148138337 conn(0x5633b3b0dc00 msgr2=0x5633b47a7000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:34.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.938+0000 7f62f8a15100 1 -- 172.21.15.154:0/2148138337 shutdown_connections 2023-09-23T03:21:34.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.938+0000 7f62f8a15100 1 -- 172.21.15.154:0/2148138337 wait complete. 2023-09-23T03:21:34.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.939+0000 7f62f8a15100 1 -- 172.21.15.154:0/434966890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633b3b0d000 msgr2=0x5633b3a9eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:34.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.939+0000 7f62f8a15100 1 --2- 172.21.15.154:0/434966890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633b3b0d000 0x5633b3a9eb00 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x5633b483e330 tx=0x5633b3d12120 comp rx=0 tx=0).stop 2023-09-23T03:21:34.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.939+0000 7f62f8a15100 1 -- 172.21.15.154:0/434966890 shutdown_connections 2023-09-23T03:21:34.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.939+0000 7f62f8a15100 1 --2- 172.21.15.154:0/434966890 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5633b3b0d400 0x5633b3a9fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.939+0000 7f62f8a15100 1 --2- 172.21.15.154:0/434966890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5633b3b0d000 0x5633b3a9eb00 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.939+0000 7f62f8a15100 1 --2- 172.21.15.154:0/434966890 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5633b3b0c400 0x5633b3a9f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:34.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.939+0000 7f62f8a15100 1 -- 172.21.15.154:0/434966890 >> 172.21.15.154:0/434966890 conn(0x5633b3a2f000 msgr2=0x5633b47a6000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:34.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.939+0000 7f62f8a15100 1 -- 172.21.15.154:0/434966890 shutdown_connections 2023-09-23T03:21:34.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:34.939+0000 7f62f8a15100 1 -- 172.21.15.154:0/434966890 wait complete. 2023-09-23T03:21:34.945 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.123145, result: writing EResetJournal entry 2023-09-23T03:21:34.945 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 init 2023-09-23T03:21:35.036 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.035+0000 7f1cc4e3a5c0 1 -- start start 2023-09-23T03:21:35.036 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.035+0000 7f1cc4e3a5c0 1 Processor -- start 2023-09-23T03:21:35.037 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.036+0000 7f1cc4e3a5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55691ae48c00 0x55691aeb2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.037 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.036+0000 7f1cc4e3a5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55691ae49800 0x55691aeb2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.037 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.036+0000 7f1cc4e3a5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55691ae49c00 0x55691aeb3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.037 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.036+0000 7f1cc4e3a5c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691ae93180 con 0x55691ae49800 2023-09-23T03:21:35.037 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.036+0000 7f1cc4e3a5c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691ae92140 con 0x55691ae48c00 2023-09-23T03:21:35.037 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.036+0000 7f1cc4e3a5c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55691ae92000 con 0x55691ae49c00 2023-09-23T03:21:35.038 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.036+0000 7f1cc26a6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55691ae49800 0x55691aeb2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.038 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.036+0000 7f1cc26a6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55691ae49800 0x55691aeb2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60378/0 (socket says 172.21.15.154:60378) 2023-09-23T03:21:35.038 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.036+0000 7f1cc26a6640 1 -- 172.21.15.154:0/992684496 learned_addr learned my addr 172.21.15.154:0/992684496 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.039 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.036+0000 7f1cc36a8640 1 --2- 172.21.15.154:0/992684496 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55691ae49c00 0x55691aeb3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.039 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.036+0000 7f1cc2ea7640 1 --2- 172.21.15.154:0/992684496 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55691ae48c00 0x55691aeb2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.036+0000 7f1cc26a6640 1 -- 172.21.15.154:0/992684496 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55691ae49c00 msgr2=0x55691aeb3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.036+0000 7f1cc26a6640 1 --2- 172.21.15.154:0/992684496 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55691ae49c00 0x55691aeb3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.036+0000 7f1cc26a6640 1 -- 172.21.15.154:0/992684496 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55691ae48c00 msgr2=0x55691aeb2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.036+0000 7f1cc26a6640 1 --2- 172.21.15.154:0/992684496 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55691ae48c00 0x55691aeb2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.036+0000 7f1cc26a6640 1 -- 172.21.15.154:0/992684496 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55691ae7cb60 con 0x55691ae49800 2023-09-23T03:21:35.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.037+0000 7f1cc2ea7640 1 --2- 172.21.15.154:0/992684496 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55691ae48c00 0x55691aeb2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:35.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.037+0000 7f1cc26a6640 1 --2- 172.21.15.154:0/992684496 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55691ae49800 0x55691aeb2b00 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x55691bc53470 tx=0x55691b12a360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56dc9c73ffd5f53a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.037+0000 7f1cc06a2640 1 -- 172.21.15.154:0/992684496 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55691ae91600 con 0x55691ae49800 2023-09-23T03:21:35.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.037+0000 7f1cc06a2640 1 -- 172.21.15.154:0/992684496 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55691ae6f800 con 0x55691ae49800 2023-09-23T03:21:35.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.037+0000 7f1cc06a2640 1 -- 172.21.15.154:0/992684496 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55691ae911e0 con 0x55691ae49800 2023-09-23T03:21:35.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.037+0000 7f1cc4e3a5c0 1 -- 172.21.15.154:0/992684496 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55691ae7c340 con 0x55691ae49800 2023-09-23T03:21:35.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.037+0000 7f1cc4e3a5c0 1 -- 172.21.15.154:0/992684496 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55691ae7c000 con 0x55691ae49800 2023-09-23T03:21:35.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.037+0000 7f1cc4e3a5c0 4 waiting for MDS map... 2023-09-23T03:21:35.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.037+0000 7f1cc06a2640 1 -- 172.21.15.154:0/992684496 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x55691af08c40 con 0x55691ae49800 2023-09-23T03:21:35.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.037+0000 7f1cc06a2640 1 -- 172.21.15.154:0/992684496 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x55691b136000 con 0x55691ae49800 2023-09-23T03:21:35.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.038+0000 7f1cc4e3a5c0 4 Got MDS map 47 2023-09-23T03:21:35.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.038+0000 7f1cc4e3a5c0 4 datascan.main: Using metadata pool output 2023-09-23T03:21:35.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.038+0000 7f1cc4e3a5c0 4 datascan.main: connecting to RADOS... 2023-09-23T03:21:35.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.038+0000 7f1cc4e3a5c0 1 Processor -- start 2023-09-23T03:21:35.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.039+0000 7f1cc4e3a5c0 1 -- start start 2023-09-23T03:21:35.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.039+0000 7f1cc4e3a5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55691b135000 0x55691aeb3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.039+0000 7f1cc4e3a5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55691b135400 0x55691aeb3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.039+0000 7f1cc4e3a5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55691b135800 0x55691aeb4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.039+0000 7f1cc4e3a5c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691ae92a00 con 0x55691b135000 2023-09-23T03:21:35.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.039+0000 7f1cc4e3a5c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691af17180 con 0x55691b135400 2023-09-23T03:21:35.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.039+0000 7f1cc4e3a5c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55691af16dc0 con 0x55691b135800 2023-09-23T03:21:35.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.039+0000 7f1cc36a8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55691b135000 0x55691aeb3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.040+0000 7f1cc36a8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55691b135000 0x55691aeb3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60386/0 (socket says 172.21.15.154:60386) 2023-09-23T03:21:35.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.040+0000 7f1cc36a8640 1 -- 172.21.15.154:0/3468669255 learned_addr learned my addr 172.21.15.154:0/3468669255 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.040+0000 7f1cc36a8640 1 -- 172.21.15.154:0/3468669255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55691b135800 msgr2=0x55691aeb4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.040+0000 7f1cc36a8640 1 --2- 172.21.15.154:0/3468669255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55691b135800 0x55691aeb4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.040+0000 7f1cc36a8640 1 -- 172.21.15.154:0/3468669255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55691b135400 msgr2=0x55691aeb3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.040+0000 7f1cc36a8640 1 --2- 172.21.15.154:0/3468669255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55691b135400 0x55691aeb3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.040+0000 7f1cc36a8640 1 -- 172.21.15.154:0/3468669255 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55691ae7cea0 con 0x55691b135000 2023-09-23T03:21:35.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.040+0000 7f1cc36a8640 1 --2- 172.21.15.154:0/3468669255 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55691b135000 0x55691aeb3600 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x55691bc52a80 tx=0x55691b12a120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=353152c19d9baa4b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.040+0000 7f1cbe69e640 1 -- 172.21.15.154:0/3468669255 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55691ae91080 con 0x55691b135000 2023-09-23T03:21:35.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.040+0000 7f1cbe69e640 1 -- 172.21.15.154:0/3468669255 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55691bc20900 con 0x55691b135000 2023-09-23T03:21:35.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.040+0000 7f1cbe69e640 1 -- 172.21.15.154:0/3468669255 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55691ae91600 con 0x55691b135000 2023-09-23T03:21:35.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.041+0000 7f1cc4e3a5c0 1 -- 172.21.15.154:0/3468669255 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55691b135000 msgr2=0x55691aeb3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.041+0000 7f1cc4e3a5c0 1 --2- 172.21.15.154:0/3468669255 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55691b135000 0x55691aeb3600 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x55691bc52a80 tx=0x55691b12a120 comp rx=0 tx=0).stop 2023-09-23T03:21:35.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.041+0000 7f1cc4e3a5c0 1 -- 172.21.15.154:0/3468669255 shutdown_connections 2023-09-23T03:21:35.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.041+0000 7f1cc4e3a5c0 1 --2- 172.21.15.154:0/3468669255 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55691b135800 0x55691aeb4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.041+0000 7f1cc4e3a5c0 1 --2- 172.21.15.154:0/3468669255 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55691b135400 0x55691aeb3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.041+0000 7f1cc4e3a5c0 1 --2- 172.21.15.154:0/3468669255 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55691b135000 0x55691aeb3600 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.041+0000 7f1cc4e3a5c0 1 -- 172.21.15.154:0/3468669255 >> 172.21.15.154:0/3468669255 conn(0x55691b134c00 msgr2=0x55691bbd3000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:35.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.041+0000 7f1cc4e3a5c0 1 -- 172.21.15.154:0/3468669255 shutdown_connections 2023-09-23T03:21:35.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.041+0000 7f1cc4e3a5c0 1 -- 172.21.15.154:0/3468669255 wait complete. 2023-09-23T03:21:35.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.042+0000 7f1cc4e3a5c0 1 Processor -- start 2023-09-23T03:21:35.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.042+0000 7f1cc4e3a5c0 1 -- start start 2023-09-23T03:21:35.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.042+0000 7f1cc4e3a5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55691b135000 0x55691aeb5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.043+0000 7f1cc4e3a5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55691b135400 0x55691aeb4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.043+0000 7f1cc4e3a5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55691b135800 0x55691aeb4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.043+0000 7f1cc4e3a5c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691af16c80 con 0x55691b135400 2023-09-23T03:21:35.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.043+0000 7f1cc4e3a5c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55691ae92140 con 0x55691b135800 2023-09-23T03:21:35.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.043+0000 7f1cc4e3a5c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55691ae92b40 con 0x55691b135000 2023-09-23T03:21:35.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.043+0000 7f1cc2ea7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55691b135400 0x55691aeb4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.043+0000 7f1cc2ea7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55691b135400 0x55691aeb4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60400/0 (socket says 172.21.15.154:60400) 2023-09-23T03:21:35.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.043+0000 7f1cc2ea7640 1 -- 172.21.15.154:0/366775751 learned_addr learned my addr 172.21.15.154:0/366775751 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.043+0000 7f1cc36a8640 1 --2- 172.21.15.154:0/366775751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55691b135000 0x55691aeb5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.043+0000 7f1cc26a6640 1 --2- 172.21.15.154:0/366775751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55691b135800 0x55691aeb4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.043+0000 7f1cc2ea7640 1 -- 172.21.15.154:0/366775751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55691b135000 msgr2=0x55691aeb5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.043+0000 7f1cc2ea7640 1 --2- 172.21.15.154:0/366775751 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55691b135000 0x55691aeb5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.043+0000 7f1cc2ea7640 1 -- 172.21.15.154:0/366775751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55691b135800 msgr2=0x55691aeb4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.043+0000 7f1cc2ea7640 1 --2- 172.21.15.154:0/366775751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55691b135800 0x55691aeb4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.043+0000 7f1cc2ea7640 1 -- 172.21.15.154:0/366775751 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55691ae7d1e0 con 0x55691b135400 2023-09-23T03:21:35.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.043+0000 7f1cc26a6640 1 --2- 172.21.15.154:0/366775751 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55691b135800 0x55691aeb4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:35.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.044+0000 7f1cc2ea7640 1 --2- 172.21.15.154:0/366775751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55691b135400 0x55691aeb4c00 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x55691b13f860 tx=0x55691b142cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.044+0000 7f1cbce9b640 1 -- 172.21.15.154:0/366775751 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55691ae91080 con 0x55691b135400 2023-09-23T03:21:35.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.044+0000 7f1cbce9b640 1 -- 172.21.15.154:0/366775751 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55691ae6fb00 con 0x55691b135400 2023-09-23T03:21:35.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.044+0000 7f1cbce9b640 1 -- 172.21.15.154:0/366775751 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55691ae91600 con 0x55691b135400 2023-09-23T03:21:35.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.044+0000 7f1cc4e3a5c0 1 -- 172.21.15.154:0/366775751 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55691ae7cea0 con 0x55691b135400 2023-09-23T03:21:35.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.044+0000 7f1cc4e3a5c0 1 -- 172.21.15.154:0/366775751 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55691ae7cb60 con 0x55691b135400 2023-09-23T03:21:35.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.044+0000 7f1cc4e3a5c0 4 datascan.init: resolving metadata pool 8 2023-09-23T03:21:35.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.046+0000 7f1cbce9b640 1 -- 172.21.15.154:0/366775751 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x55691ae7a840 con 0x55691b135400 2023-09-23T03:21:35.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.047+0000 7f1cbce9b640 1 --2- 172.21.15.154:0/366775751 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55691b1aa800 0x55691aeb5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.047+0000 7f1cbce9b640 1 -- 172.21.15.154:0/366775751 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x55691af08a80 con 0x55691b135400 2023-09-23T03:21:35.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.047+0000 7f1cc4e3a5c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-09-23T03:21:35.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.047+0000 7f1cc4e3a5c0 1 --2- 172.21.15.154:0/366775751 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55691b1aac00 0x55691aeb4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.047+0000 7f1cc4e3a5c0 1 -- 172.21.15.154:0/366775751 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:1 8.14 8:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55691aea3880 con 0x55691b1aac00 2023-09-23T03:21:35.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.047+0000 7f1cc36a8640 1 --2- 172.21.15.154:0/366775751 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55691b1aa800 0x55691aeb5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.047+0000 7f1cc36a8640 1 --2- 172.21.15.154:0/366775751 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55691b1aa800 0x55691aeb5700 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x55691bc52a80 tx=0x55691b12a000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.047+0000 7f1cc26a6640 1 --2- 172.21.15.154:0/366775751 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55691b1aac00 0x55691aeb4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.048+0000 7f1cc26a6640 1 --2- 172.21.15.154:0/366775751 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55691b1aac00 0x55691aeb4100 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.049+0000 7f1cc26a6640 1 -- 172.21.15.154:0/366775751 <== osd.4 v2:172.21.15.190:6808/717302641 1 ==== osd_op_reply(1 1.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 160+0+0 (crc 0 0 0) 0x55691bce8b40 con 0x55691b1aac00 2023-09-23T03:21:35.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.049+0000 7f1cc4e3a5c0 1 -- 172.21.15.154:0/366775751 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:2 8.14 8:287034c4:::1.00000000.inode:head [writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55691aea3c00 con 0x55691b1aac00 2023-09-23T03:21:35.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.052+0000 7f1cc26a6640 1 -- 172.21.15.154:0/366775751 <== osd.4 v2:172.21.15.190:6808/717302641 2 ==== osd_op_reply(2 1.00000000.inode [writefull 0~582] v67'107 uv107 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x55691bce8b40 con 0x55691b1aac00 2023-09-23T03:21:35.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.052+0000 7f1cc4e3a5c0 1 --2- 172.21.15.154:0/366775751 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55691b1ab000 0x55691aeb3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.052+0000 7f1cc4e3a5c0 1 -- 172.21.15.154:0/366775751 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:3 8.1f 8:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55691b2aa000 con 0x55691b1ab000 2023-09-23T03:21:35.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.052+0000 7f1cc36a8640 1 --2- 172.21.15.154:0/366775751 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55691b1ab000 0x55691aeb3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.053+0000 7f1cc36a8640 1 --2- 172.21.15.154:0/366775751 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55691b1ab000 0x55691aeb3b80 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.053+0000 7f1cc36a8640 1 -- 172.21.15.154:0/366775751 <== osd.0 v2:172.21.15.154:6800/2920938138 1 ==== osd_op_reply(3 100.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 162+0+0 (crc 0 0 0) 0x55691bce9200 con 0x55691b1ab000 2023-09-23T03:21:35.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.053+0000 7f1cc4e3a5c0 1 -- 172.21.15.154:0/366775751 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:4 8.1f 8:fe07bba1:::100.00000000.inode:head [writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55691b2aa380 con 0x55691b1ab000 2023-09-23T03:21:35.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.056+0000 7f1cc36a8640 1 -- 172.21.15.154:0/366775751 <== osd.0 v2:172.21.15.154:6800/2920938138 2 ==== osd_op_reply(4 100.00000000.inode [writefull 0~582] v67'10 uv10 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x55691bce9200 con 0x55691b1ab000 2023-09-23T03:21:35.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.056+0000 7f1cc4e3a5c0 1 --2- 172.21.15.154:0/366775751 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55691b1ab400 0x55691aeb3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.056+0000 7f1cc4e3a5c0 1 -- 172.21.15.154:0/366775751 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:5 8.13 8:cd5a64a3:::100.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55691b2aa700 con 0x55691b1ab400 2023-09-23T03:21:35.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.057+0000 7f1cc2ea7640 1 --2- 172.21.15.154:0/366775751 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55691b1ab400 0x55691aeb3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.057+0000 7f1cc2ea7640 1 --2- 172.21.15.154:0/366775751 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55691b1ab400 0x55691aeb3600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.057+0000 7f1cc2ea7640 1 -- 172.21.15.154:0/366775751 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(5 100.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55691bce98c0 con 0x55691b1ab400 2023-09-23T03:21:35.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.058+0000 7f1cc4e3a5c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-09-23T03:21:35.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.058+0000 7f1cc4e3a5c0 1 -- 172.21.15.154:0/366775751 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:6 8.13 8:cd5a64a3:::100.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55691b2aaa80 con 0x55691b1ab400 2023-09-23T03:21:35.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.059+0000 7f1cc2ea7640 1 -- 172.21.15.154:0/366775751 <== osd.3 v2:172.21.15.154:6808/1133688468 2 ==== osd_op_reply(6 100.00000000 [create,omap-set-header] v67'7 uv7 ondisk = 0) v8 ==== 198+0+0 (crc 0 0 0) 0x55691bce98c0 con 0x55691b1ab400 2023-09-23T03:21:35.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.059+0000 7f1cc4e3a5c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x100 2023-09-23T03:21:35.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.059+0000 7f1cc4e3a5c0 1 -- 172.21.15.154:0/366775751 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55691b1ab000 msgr2=0x55691aeb3b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.059+0000 7f1cc4e3a5c0 1 --2- 172.21.15.154:0/366775751 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55691b1ab000 0x55691aeb3b80 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.060+0000 7f1cc4e3a5c0 1 -- 172.21.15.154:0/366775751 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55691b1ab400 msgr2=0x55691aeb3600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.060+0000 7f1cc4e3a5c0 1 --2- 172.21.15.154:0/366775751 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55691b1ab400 0x55691aeb3600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.060+0000 7f1cc4e3a5c0 1 -- 172.21.15.154:0/366775751 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55691b1aac00 msgr2=0x55691aeb4100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.060+0000 7f1cc4e3a5c0 1 --2- 172.21.15.154:0/366775751 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55691b1aac00 0x55691aeb4100 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.060+0000 7f1cc2ea7640 1 -- 172.21.15.154:0/366775751 reap_dead start 2023-09-23T03:21:35.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.060+0000 7f1cc4e3a5c0 1 -- 172.21.15.154:0/366775751 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55691b1aa800 msgr2=0x55691aeb5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.060+0000 7f1cc4e3a5c0 1 --2- 172.21.15.154:0/366775751 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55691b1aa800 0x55691aeb5700 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x55691bc52a80 tx=0x55691b12a000 comp rx=0 tx=0).stop 2023-09-23T03:21:35.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.060+0000 7f1cc4e3a5c0 1 -- 172.21.15.154:0/366775751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55691b135400 msgr2=0x55691aeb4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.060+0000 7f1cc4e3a5c0 1 --2- 172.21.15.154:0/366775751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55691b135400 0x55691aeb4c00 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x55691b13f860 tx=0x55691b142cc0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.060+0000 7f1cc4e3a5c0 1 -- 172.21.15.154:0/366775751 shutdown_connections 2023-09-23T03:21:35.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.060+0000 7f1cc4e3a5c0 1 --2- 172.21.15.154:0/366775751 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55691b1aa800 0x55691aeb5700 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.060+0000 7f1cc4e3a5c0 1 --2- 172.21.15.154:0/366775751 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55691b135400 0x55691aeb4c00 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.060+0000 7f1cc4e3a5c0 1 -- 172.21.15.154:0/366775751 >> 172.21.15.154:0/366775751 conn(0x55691b134c00 msgr2=0x55691bbd3000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:35.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.060+0000 7f1cc4e3a5c0 1 -- 172.21.15.154:0/366775751 shutdown_connections 2023-09-23T03:21:35.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.060+0000 7f1cc4e3a5c0 1 -- 172.21.15.154:0/366775751 wait complete. 2023-09-23T03:21:35.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.061+0000 7f1cc4e3a5c0 1 -- 172.21.15.154:0/992684496 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55691ae49800 msgr2=0x55691aeb2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.061+0000 7f1cc4e3a5c0 1 --2- 172.21.15.154:0/992684496 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55691ae49800 0x55691aeb2b00 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x55691bc53470 tx=0x55691b12a360 comp rx=0 tx=0).stop 2023-09-23T03:21:35.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.061+0000 7f1cc4e3a5c0 1 -- 172.21.15.154:0/992684496 shutdown_connections 2023-09-23T03:21:35.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.061+0000 7f1cc4e3a5c0 1 --2- 172.21.15.154:0/992684496 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55691ae49c00 0x55691aeb3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.061+0000 7f1cc4e3a5c0 1 --2- 172.21.15.154:0/992684496 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55691ae49800 0x55691aeb2b00 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.061+0000 7f1cc4e3a5c0 1 --2- 172.21.15.154:0/992684496 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55691ae48c00 0x55691aeb2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.061+0000 7f1cc4e3a5c0 1 -- 172.21.15.154:0/992684496 >> 172.21.15.154:0/992684496 conn(0x55691ae49000 msgr2=0x55691bbd2800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:35.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.061+0000 7f1cc4e3a5c0 1 -- 172.21.15.154:0/992684496 shutdown_connections 2023-09-23T03:21:35.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.061+0000 7f1cc4e3a5c0 1 -- 172.21.15.154:0/992684496 wait complete. 2023-09-23T03:21:35.067 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'init'] in time 0:00:00.121695, result: 2023-09-23T03:21:35.067 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents --worker_n 0 --worker_m 7 2023-09-23T03:21:35.068 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents --worker_n 1 --worker_m 7 2023-09-23T03:21:35.068 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents --worker_n 2 --worker_m 7 2023-09-23T03:21:35.068 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents --worker_n 3 --worker_m 7 2023-09-23T03:21:35.069 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents --worker_n 4 --worker_m 7 2023-09-23T03:21:35.069 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents --worker_n 5 --worker_m 7 2023-09-23T03:21:35.069 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents --worker_n 6 --worker_m 7 2023-09-23T03:21:35.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.184+0000 7efdcc0145c0 1 -- start start 2023-09-23T03:21:35.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.184+0000 7efdcc0145c0 1 Processor -- start 2023-09-23T03:21:35.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.185+0000 7efdcc0145c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564661e16c00 0x564661e80580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.185+0000 7efdcc0145c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564661e17800 0x564661e80b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.185+0000 7efdcc0145c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564661e17c00 0x564661e81080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.185+0000 7efdcc0145c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x564661e61180 con 0x564661e16c00 2023-09-23T03:21:35.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.185+0000 7efdcc0145c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564661e60000 con 0x564661e17800 2023-09-23T03:21:35.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.185+0000 7efdcc0145c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x564661e60140 con 0x564661e17c00 2023-09-23T03:21:35.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.186+0000 7efdc9cee640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564661e16c00 0x564661e80580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.186+0000 7efdc9cee640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564661e16c00 0x564661e80580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60414/0 (socket says 172.21.15.154:60414) 2023-09-23T03:21:35.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.186+0000 7efdc9cee640 1 -- 172.21.15.154:0/1163186475 learned_addr learned my addr 172.21.15.154:0/1163186475 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.187+0000 7efdc94ed640 1 --2- 172.21.15.154:0/1163186475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564661e17800 0x564661e80b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.187+0000 7efdca4ef640 1 --2- 172.21.15.154:0/1163186475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564661e17c00 0x564661e81080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.188+0000 7efdc9cee640 1 -- 172.21.15.154:0/1163186475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564661e17c00 msgr2=0x564661e81080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.188+0000 7efdc9cee640 1 --2- 172.21.15.154:0/1163186475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564661e17c00 0x564661e81080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.188+0000 7efdc9cee640 1 -- 172.21.15.154:0/1163186475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564661e17800 msgr2=0x564661e80b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.188+0000 7efdc9cee640 1 --2- 172.21.15.154:0/1163186475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564661e17800 0x564661e80b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.188+0000 7efdc9cee640 1 -- 172.21.15.154:0/1163186475 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564661e4ab60 con 0x564661e16c00 2023-09-23T03:21:35.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.188+0000 7efdc94ed640 1 --2- 172.21.15.154:0/1163186475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564661e17800 0x564661e80b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:21:35.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.188+0000 7efdc9cee640 1 --2- 172.21.15.154:0/1163186475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564661e16c00 0x564661e80580 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x564662c21470 tx=0x564662cd0420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f1e2f30710bf2a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.189+0000 7efdc74e9640 1 -- 172.21.15.154:0/1163186475 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564661e5f600 con 0x564661e16c00 2023-09-23T03:21:35.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.189+0000 7efdc74e9640 1 -- 172.21.15.154:0/1163186475 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564661e3d800 con 0x564661e16c00 2023-09-23T03:21:35.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.189+0000 7efdc74e9640 1 -- 172.21.15.154:0/1163186475 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564661e5f1e0 con 0x564661e16c00 2023-09-23T03:21:35.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.189+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1163186475 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564661e4a340 con 0x564661e16c00 2023-09-23T03:21:35.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.189+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1163186475 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x564661e4a000 con 0x564661e16c00 2023-09-23T03:21:35.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.189+0000 7efdcc0145c0 4 waiting for MDS map... 2023-09-23T03:21:35.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.190+0000 7efdc74e9640 1 -- 172.21.15.154:0/1163186475 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x564661ed6c40 con 0x564661e16c00 2023-09-23T03:21:35.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.190+0000 7efdc74e9640 1 -- 172.21.15.154:0/1163186475 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x564661e68f00 con 0x564661e16c00 2023-09-23T03:21:35.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.190+0000 7efdcc0145c0 4 Got MDS map 47 2023-09-23T03:21:35.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.190+0000 7efdcc0145c0 4 datascan.main: Using metadata pool output 2023-09-23T03:21:35.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.190+0000 7efdcc0145c0 4 datascan.main: connecting to RADOS... 2023-09-23T03:21:35.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.192+0000 7efdcc0145c0 1 Processor -- start 2023-09-23T03:21:35.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.193+0000 7efdcc0145c0 1 -- start start 2023-09-23T03:21:35.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.195+0000 7efdcc0145c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564662ce9000 0x564661e81600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.195+0000 7efdcc0145c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564662ce9400 0x564661e81b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.196+0000 7efdc9cee640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564662ce9400 0x564661e81b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.196+0000 7efdc9cee640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564662ce9400 0x564661e81b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60434/0 (socket says 172.21.15.154:60434) 2023-09-23T03:21:35.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.196+0000 7efdc9cee640 1 -- 172.21.15.154:0/4039846525 learned_addr learned my addr 172.21.15.154:0/4039846525 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.196+0000 7efdca4ef640 1 --2- 172.21.15.154:0/4039846525 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564662ce9000 0x564661e81600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.197+0000 7efdcc0145c0 1 --2- 172.21.15.154:0/4039846525 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564662ce9800 0x564661e82100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.197+0000 7efdcc0145c0 1 -- 172.21.15.154:0/4039846525 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x564661e60a00 con 0x564662ce9400 2023-09-23T03:21:35.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.197+0000 7efdcc0145c0 1 -- 172.21.15.154:0/4039846525 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564661ee5180 con 0x564662ce9000 2023-09-23T03:21:35.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.197+0000 7efdcc0145c0 1 -- 172.21.15.154:0/4039846525 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x564661ee4dc0 con 0x564662ce9800 2023-09-23T03:21:35.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.197+0000 7efdc94ed640 1 --2- 172.21.15.154:0/4039846525 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564662ce9800 0x564661e82100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.198+0000 7efdc9cee640 1 -- 172.21.15.154:0/4039846525 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564662ce9800 msgr2=0x564661e82100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.198+0000 7efdc9cee640 1 --2- 172.21.15.154:0/4039846525 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564662ce9800 0x564661e82100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.198+0000 7efdc9cee640 1 -- 172.21.15.154:0/4039846525 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564662ce9000 msgr2=0x564661e81600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.198+0000 7efdc9cee640 1 --2- 172.21.15.154:0/4039846525 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564662ce9000 0x564661e81600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.198+0000 7efdc9cee640 1 -- 172.21.15.154:0/4039846525 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564661e4aea0 con 0x564662ce9400 2023-09-23T03:21:35.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.198+0000 7efdca4ef640 1 --2- 172.21.15.154:0/4039846525 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564662ce9000 0x564661e81600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:21:35.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.199+0000 7efdc9cee640 1 --2- 172.21.15.154:0/4039846525 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564662ce9400 0x564661e81b80 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x564662be26c0 tx=0x564662ba75c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec085c5a81bc0e7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.199+0000 7efdc54e5640 1 -- 172.21.15.154:0/4039846525 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564661e5ef20 con 0x564662ce9400 2023-09-23T03:21:35.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.199+0000 7efdc54e5640 1 -- 172.21.15.154:0/4039846525 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564662bef080 con 0x564662ce9400 2023-09-23T03:21:35.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.199+0000 7efdc54e5640 1 -- 172.21.15.154:0/4039846525 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564661e5f600 con 0x564662ce9400 2023-09-23T03:21:35.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.200+0000 7fb99ca525c0 1 -- start start 2023-09-23T03:21:35.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.200+0000 7fb99ca525c0 1 Processor -- start 2023-09-23T03:21:35.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.201+0000 7efdcc0145c0 1 -- 172.21.15.154:0/4039846525 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564662ce9400 msgr2=0x564661e81b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.201+0000 7f9a4d20c5c0 1 -- start start 2023-09-23T03:21:35.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.201+0000 7efdcc0145c0 1 --2- 172.21.15.154:0/4039846525 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564662ce9400 0x564661e81b80 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x564662be26c0 tx=0x564662ba75c0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.201+0000 7f9a4d20c5c0 1 Processor -- start 2023-09-23T03:21:35.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.202+0000 7efdcc0145c0 1 -- 172.21.15.154:0/4039846525 shutdown_connections 2023-09-23T03:21:35.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.202+0000 7efdcc0145c0 1 --2- 172.21.15.154:0/4039846525 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564662ce9800 0x564661e82100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.202+0000 7efdcc0145c0 1 --2- 172.21.15.154:0/4039846525 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564662ce9400 0x564661e81b80 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.202+0000 7efdcc0145c0 1 --2- 172.21.15.154:0/4039846525 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564662ce9000 0x564661e81600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.202+0000 7efdcc0145c0 1 -- 172.21.15.154:0/4039846525 >> 172.21.15.154:0/4039846525 conn(0x564662ce8c00 msgr2=0x564662b91000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:35.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.202+0000 7fb99ca525c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5602ef07ac00 0x5602ef0e4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.202+0000 7fb99ca525c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5602ef07b800 0x5602ef0e4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.202+0000 7fb99ca525c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5602ef07bc00 0x5602ef0e5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.202+0000 7fb99a2c2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5602ef07b800 0x5602ef0e4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.202+0000 7efdcc0145c0 1 -- 172.21.15.154:0/4039846525 shutdown_connections 2023-09-23T03:21:35.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.202+0000 7efdcc0145c0 1 -- 172.21.15.154:0/4039846525 wait complete. 2023-09-23T03:21:35.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.203+0000 7fb99aac3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5602ef07ac00 0x5602ef0e4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.203+0000 7fb99ca525c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602ef0c5180 con 0x5602ef07b800 2023-09-23T03:21:35.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.203+0000 7fb99b2c4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5602ef07bc00 0x5602ef0e5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.203+0000 7fb99b2c4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5602ef07bc00 0x5602ef0e5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.154:49072/0 (socket says 172.21.15.154:49072) 2023-09-23T03:21:35.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.203+0000 7fb99b2c4640 1 -- 172.21.15.154:0/3567808904 learned_addr learned my addr 172.21.15.154:0/3567808904 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.203+0000 7f9a4d20c5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560334bec00 0x556033528580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.203+0000 7efdcc0145c0 1 Processor -- start 2023-09-23T03:21:35.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.203+0000 7fb99ca525c0 1 -- 172.21.15.154:0/3567808904 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602ef0c4140 con 0x5602ef07bc00 2023-09-23T03:21:35.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.203+0000 7fb99ca525c0 1 -- 172.21.15.154:0/3567808904 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5602ef0c4000 con 0x5602ef07ac00 2023-09-23T03:21:35.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.203+0000 7f9a4d20c5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5560334bf800 0x556033528b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.203+0000 7fb99a2c2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5602ef07b800 0x5602ef0e4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60466/0 (socket says 172.21.15.154:60466) 2023-09-23T03:21:35.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.204+0000 7efdcc0145c0 1 -- start start 2023-09-23T03:21:35.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.204+0000 7fb99a2c2640 1 -- 172.21.15.154:0/3567808904 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5602ef07ac00 msgr2=0x5602ef0e4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.204+0000 7fb99a2c2640 1 --2- 172.21.15.154:0/3567808904 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5602ef07ac00 0x5602ef0e4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.204+0000 7fb99a2c2640 1 -- 172.21.15.154:0/3567808904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5602ef07bc00 msgr2=0x5602ef0e5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.204+0000 7fb99a2c2640 1 --2- 172.21.15.154:0/3567808904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5602ef07bc00 0x5602ef0e5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.204+0000 7f9a4d20c5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560334bfc00 0x556033529080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.204+0000 7f9a4d20c5c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x556033509180 con 0x5560334bfc00 2023-09-23T03:21:35.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.204+0000 7f9a4d20c5c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55603358d180 con 0x5560334bec00 2023-09-23T03:21:35.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.204+0000 7f9a4d20c5c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55603358cdc0 con 0x5560334bf800 2023-09-23T03:21:35.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.204+0000 7f9a4b566640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560334bfc00 0x556033529080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.204+0000 7f9a4b566640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560334bfc00 0x556033529080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60470/0 (socket says 172.21.15.154:60470) 2023-09-23T03:21:35.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.204+0000 7f9a4b566640 1 -- 172.21.15.154:0/2885832469 learned_addr learned my addr 172.21.15.154:0/2885832469 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.205+0000 7fb99a2c2640 1 -- 172.21.15.154:0/3567808904 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602ef0aeb60 con 0x5602ef07b800 2023-09-23T03:21:35.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.205+0000 7f9a4a564640 1 --2- 172.21.15.154:0/2885832469 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5560334bf800 0x556033528b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.205+0000 7f9a4ad65640 1 --2- 172.21.15.154:0/2885832469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560334bec00 0x556033528580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.205+0000 7fb99a2c2640 1 --2- 172.21.15.154:0/3567808904 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5602ef07b800 0x5602ef0e4b00 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x5602efe873e0 tx=0x5602efdde480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520530c95d78917a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.205+0000 7efdcc0145c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564662ce9000 0x564661e83180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.205+0000 7efdcc0145c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564662ce9400 0x564661e82c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.205+0000 7f9a4b566640 1 -- 172.21.15.154:0/2885832469 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5560334bf800 msgr2=0x556033528b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.205+0000 7efdcc0145c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564662ce9800 0x564661e82680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.205+0000 7f9a4b566640 1 --2- 172.21.15.154:0/2885832469 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5560334bf800 0x556033528b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.205+0000 7fb99b2c4640 1 --2- 172.21.15.154:0/3567808904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5602ef07bc00 0x5602ef0e5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:35.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.205+0000 7f9a4b566640 1 -- 172.21.15.154:0/2885832469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560334bec00 msgr2=0x556033528580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.205+0000 7f9a4b566640 1 --2- 172.21.15.154:0/2885832469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560334bec00 0x556033528580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.205+0000 7f9a4b566640 1 -- 172.21.15.154:0/2885832469 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560334f2b60 con 0x5560334bfc00 2023-09-23T03:21:35.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.206+0000 7efdcc0145c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x564661ee4c80 con 0x564662ce9800 2023-09-23T03:21:35.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.206+0000 7efdcc0145c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564661e60000 con 0x564662ce9400 2023-09-23T03:21:35.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.206+0000 7fb9982be640 1 -- 172.21.15.154:0/3567808904 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5602ef0c3600 con 0x5602ef07b800 2023-09-23T03:21:35.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.206+0000 7efdcc0145c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x564661e60b40 con 0x564662ce9000 2023-09-23T03:21:35.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.206+0000 7fb99ca525c0 1 -- 172.21.15.154:0/3567808904 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5602ef0ae340 con 0x5602ef07b800 2023-09-23T03:21:35.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.206+0000 7fb99ca525c0 1 -- 172.21.15.154:0/3567808904 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5602ef0ae000 con 0x5602ef07b800 2023-09-23T03:21:35.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.206+0000 7fb99ca525c0 4 waiting for MDS map... 2023-09-23T03:21:35.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.206+0000 7f9a4b566640 1 --2- 172.21.15.154:0/2885832469 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560334bfc00 0x556033529080 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x5560342cd470 tx=0x55603429a4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ade8726537bd7fc5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.206+0000 7efdc94ed640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564662ce9800 0x564661e82680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.206+0000 7efdc94ed640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564662ce9800 0x564661e82680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60474/0 (socket says 172.21.15.154:60474) 2023-09-23T03:21:35.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.206+0000 7efdc94ed640 1 -- 172.21.15.154:0/1781234207 learned_addr learned my addr 172.21.15.154:0/1781234207 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.207+0000 7f9a48560640 1 -- 172.21.15.154:0/2885832469 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556033507600 con 0x5560334bfc00 2023-09-23T03:21:35.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.207+0000 7efdca4ef640 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564662ce9000 0x564661e83180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.207+0000 7efdc9cee640 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564662ce9400 0x564661e82c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.206+0000 7fb9982be640 1 -- 172.21.15.154:0/3567808904 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5602ef0a1800 con 0x5602ef07b800 2023-09-23T03:21:35.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.206+0000 7fb9982be640 1 -- 172.21.15.154:0/3567808904 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5602ef0c3600 con 0x5602ef07b800 2023-09-23T03:21:35.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.207+0000 7fb9982be640 1 -- 172.21.15.154:0/3567808904 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x5602ef13ac40 con 0x5602ef07b800 2023-09-23T03:21:35.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.207+0000 7fb9982be640 1 -- 172.21.15.154:0/3567808904 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x5602ef36c000 con 0x5602ef07b800 2023-09-23T03:21:35.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.207+0000 7f9a48560640 1 -- 172.21.15.154:0/2885832469 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5560334e5b00 con 0x5560334bfc00 2023-09-23T03:21:35.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.207+0000 7f9a48560640 1 -- 172.21.15.154:0/2885832469 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5560335071e0 con 0x5560334bfc00 2023-09-23T03:21:35.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.208+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/2885832469 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5560334f2340 con 0x5560334bfc00 2023-09-23T03:21:35.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.208+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/2885832469 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5560334f2000 con 0x5560334bfc00 2023-09-23T03:21:35.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.208+0000 7f9a4d20c5c0 4 waiting for MDS map... 2023-09-23T03:21:35.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.208+0000 7f48e3e0c5c0 1 -- start start 2023-09-23T03:21:35.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.207+0000 7efdc94ed640 1 -- 172.21.15.154:0/1781234207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564662ce9000 msgr2=0x564661e83180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.207+0000 7efdc94ed640 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564662ce9000 0x564661e83180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.207+0000 7efdc94ed640 1 -- 172.21.15.154:0/1781234207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564662ce9400 msgr2=0x564661e82c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.207+0000 7efdc94ed640 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564662ce9400 0x564661e82c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.207+0000 7efdc94ed640 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564661e4ab60 con 0x564662ce9800 2023-09-23T03:21:35.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.208+0000 7efdc9cee640 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564662ce9400 0x564661e82c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:21:35.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.208+0000 7f48e3e0c5c0 1 Processor -- start 2023-09-23T03:21:35.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.208+0000 7efdc94ed640 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564662ce9800 0x564661e82680 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x564662c20a80 tx=0x564662cd0120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.207+0000 7fb99ca525c0 4 Got MDS map 47 2023-09-23T03:21:35.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.208+0000 7fb99ca525c0 4 datascan.main: Using metadata pool output 2023-09-23T03:21:35.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.208+0000 7fb99ca525c0 4 datascan.main: connecting to RADOS... 2023-09-23T03:21:35.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.209+0000 7f9a48560640 1 -- 172.21.15.154:0/2885832469 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x55603357ec40 con 0x5560334bfc00 2023-09-23T03:21:35.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.209+0000 7f9a48560640 1 -- 172.21.15.154:0/2885832469 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x5560337be000 con 0x5560334bfc00 2023-09-23T03:21:35.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.209+0000 7f9a4d20c5c0 4 Got MDS map 47 2023-09-23T03:21:35.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.209+0000 7efdc3ce2640 1 -- 172.21.15.154:0/1781234207 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564661e5f080 con 0x564662ce9800 2023-09-23T03:21:35.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.209+0000 7efdc3ce2640 1 -- 172.21.15.154:0/1781234207 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564662bee300 con 0x564662ce9800 2023-09-23T03:21:35.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.209+0000 7efdc3ce2640 1 -- 172.21.15.154:0/1781234207 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564661e5ef20 con 0x564662ce9800 2023-09-23T03:21:35.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.210+0000 7fb99ca525c0 1 Processor -- start 2023-09-23T03:21:35.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.210+0000 7f9a4d20c5c0 4 datascan.main: Using metadata pool output 2023-09-23T03:21:35.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.210+0000 7f9a4d20c5c0 4 datascan.main: connecting to RADOS... 2023-09-23T03:21:35.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.210+0000 7f48e3e0c5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b08afcc00 0x556b08b66580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.210+0000 7fb99ca525c0 1 -- start start 2023-09-23T03:21:35.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.210+0000 7f48e3e0c5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b08afd800 0x556b08b66b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.210+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564661e4a340 con 0x564662ce9800 2023-09-23T03:21:35.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.211+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564661e4b520 con 0x564662ce9800 2023-09-23T03:21:35.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.211+0000 7f48e1a6f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b08afcc00 0x556b08b66580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.211+0000 7f48e1a6f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b08afcc00 0x556b08b66580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.154:49124/0 (socket says 172.21.15.154:49124) 2023-09-23T03:21:35.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.211+0000 7f48e1a6f640 1 -- 172.21.15.154:0/1067950898 learned_addr learned my addr 172.21.15.154:0/1067950898 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.211+0000 7efdcc0145c0 4 datascan.init: resolving metadata pool 8 2023-09-23T03:21:35.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.211+0000 7f48e3e0c5c0 1 --2- 172.21.15.154:0/1067950898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556b08afdc00 0x556b08b67080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.211+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/1067950898 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b08b47180 con 0x556b08afd800 2023-09-23T03:21:35.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.212+0000 7f9a4d20c5c0 1 Processor -- start 2023-09-23T03:21:35.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.212+0000 7f48e2270640 1 --2- 172.21.15.154:0/1067950898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556b08afdc00 0x556b08b67080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.212+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/1067950898 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b08bcb180 con 0x556b08afcc00 2023-09-23T03:21:35.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.212+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/1067950898 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x556b08bcadc0 con 0x556b08afdc00 2023-09-23T03:21:35.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.212+0000 7fb99ca525c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5602ef363000 0x5602ef0e5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.212+0000 7fb99ca525c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5602ef363400 0x5602ef0e5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.212+0000 7fb99ca525c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5602ef363800 0x5602ef0e6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.212+0000 7f9a4d20c5c0 1 -- start start 2023-09-23T03:21:35.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.213+0000 7fb99ca525c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602ef0c4a00 con 0x5602ef363800 2023-09-23T03:21:35.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.213+0000 7fb99ca525c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602ef149180 con 0x5602ef363000 2023-09-23T03:21:35.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.213+0000 7fb99ca525c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5602ef148dc0 con 0x5602ef363400 2023-09-23T03:21:35.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.213+0000 7fb99aac3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5602ef363400 0x5602ef0e5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.213+0000 7fb99aac3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5602ef363400 0x5602ef0e5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:33974/0 (socket says 172.21.15.154:33974) 2023-09-23T03:21:35.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.213+0000 7f48e1a6f640 1 -- 172.21.15.154:0/1067950898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556b08afdc00 msgr2=0x556b08b67080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.213+0000 7f48e1a6f640 1 --2- 172.21.15.154:0/1067950898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556b08afdc00 0x556b08b67080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.213+0000 7f48e1a6f640 1 -- 172.21.15.154:0/1067950898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b08afd800 msgr2=0x556b08b66b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.213+0000 7f48e1a6f640 1 --2- 172.21.15.154:0/1067950898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b08afd800 0x556b08b66b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.213+0000 7f48e1a6f640 1 -- 172.21.15.154:0/1067950898 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b08b30b60 con 0x556b08afcc00 2023-09-23T03:21:35.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.213+0000 7fb99aac3640 1 -- 172.21.15.154:0/746343386 learned_addr learned my addr 172.21.15.154:0/746343386 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.214+0000 7f48e1a6f640 1 --2- 172.21.15.154:0/1067950898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b08afcc00 0x556b08b66580 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x556b09905410 tx=0x556b099be240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b55d2c4f1a9e030f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.214+0000 7fb99a2c2640 1 --2- 172.21.15.154:0/746343386 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5602ef363800 0x5602ef0e6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.214+0000 7fb99aac3640 1 -- 172.21.15.154:0/746343386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5602ef363000 msgr2=0x5602ef0e5600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T03:21:35.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.214+0000 7fb99aac3640 1 --2- 172.21.15.154:0/746343386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5602ef363000 0x5602ef0e5600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.214+0000 7fb99aac3640 1 -- 172.21.15.154:0/746343386 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5602ef363800 msgr2=0x5602ef0e6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.214+0000 7fb99aac3640 1 --2- 172.21.15.154:0/746343386 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5602ef363800 0x5602ef0e6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.214+0000 7fb99aac3640 1 -- 172.21.15.154:0/746343386 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602ef0aeea0 con 0x5602ef363400 2023-09-23T03:21:35.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.214+0000 7fb99a2c2640 1 --2- 172.21.15.154:0/746343386 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5602ef363800 0x5602ef0e6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:35.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.214+0000 7f9a4d20c5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5560337b7000 0x556033529600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.214+0000 7f9a4d20c5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560337b7400 0x556033529b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.214+0000 7f9a4d20c5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560337b7800 0x55603352a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.214+0000 7f9a4d20c5c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x556033508a00 con 0x5560337b7800 2023-09-23T03:21:35.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.214+0000 7f9a4d20c5c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x556033508b40 con 0x5560337b7400 2023-09-23T03:21:35.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.214+0000 7f9a4d20c5c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55603358d180 con 0x5560337b7000 2023-09-23T03:21:35.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.214+0000 7f48df26a640 1 -- 172.21.15.154:0/1067950898 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556b08b45600 con 0x556b08afcc00 2023-09-23T03:21:35.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.214+0000 7f48df26a640 1 -- 172.21.15.154:0/1067950898 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556b08b23800 con 0x556b08afcc00 2023-09-23T03:21:35.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.214+0000 7fb99aac3640 1 --2- 172.21.15.154:0/746343386 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5602ef363400 0x5602ef0e5b80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x5602efe86a50 tx=0x5602efdde180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce892a3464427505 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.214+0000 7f48df26a640 1 -- 172.21.15.154:0/1067950898 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556b08b451e0 con 0x556b08afcc00 2023-09-23T03:21:35.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.215+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/1067950898 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b08b30340 con 0x556b08afcc00 2023-09-23T03:21:35.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.215+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/1067950898 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x556b08b30000 con 0x556b08afcc00 2023-09-23T03:21:35.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.215+0000 7f48e3e0c5c0 4 waiting for MDS map... 2023-09-23T03:21:35.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.215+0000 7fb9962ba640 1 -- 172.21.15.154:0/746343386 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5602ef0c3080 con 0x5602ef363400 2023-09-23T03:21:35.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.215+0000 7fb9962ba640 1 -- 172.21.15.154:0/746343386 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5602efe50780 con 0x5602ef363400 2023-09-23T03:21:35.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.215+0000 7fb9962ba640 1 -- 172.21.15.154:0/746343386 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5602ef0c3080 con 0x5602ef363400 2023-09-23T03:21:35.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.215+0000 7f9a4a564640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560337b7800 0x55603352a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.215+0000 7f9a4ad65640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560337b7400 0x556033529b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.215+0000 7f9a4a564640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560337b7800 0x55603352a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60514/0 (socket says 172.21.15.154:60514) 2023-09-23T03:21:35.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.215+0000 7f9a4a564640 1 -- 172.21.15.154:0/3161889150 learned_addr learned my addr 172.21.15.154:0/3161889150 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.215+0000 7f9a4a564640 1 --2- 172.21.15.154:0/3161889150 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5560337b7000 0x556033529600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.215+0000 7f48df26a640 1 -- 172.21.15.154:0/1067950898 <== mon.1 v2:172.21.15.190:3300/0 4 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x556b08bbcc40 con 0x556b08afcc00 2023-09-23T03:21:35.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.216+0000 7f48df26a640 1 -- 172.21.15.154:0/1067950898 <== mon.1 v2:172.21.15.190:3300/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x556b08b4ef00 con 0x556b08afcc00 2023-09-23T03:21:35.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.216+0000 7f48e3e0c5c0 4 Got MDS map 47 2023-09-23T03:21:35.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.216+0000 7efdc3ce2640 1 -- 172.21.15.154:0/1781234207 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x564661e49080 con 0x564662ce9800 2023-09-23T03:21:35.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.216+0000 7f48e3e0c5c0 4 datascan.main: Using metadata pool output 2023-09-23T03:21:35.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.216+0000 7f48e3e0c5c0 4 datascan.main: connecting to RADOS... 2023-09-23T03:21:35.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.216+0000 7f9a4ad65640 1 -- 172.21.15.154:0/3161889150 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5560337b7000 msgr2=0x556033529600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.216+0000 7f9a4ad65640 1 --2- 172.21.15.154:0/3161889150 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5560337b7000 0x556033529600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.216+0000 7f9a4ad65640 1 -- 172.21.15.154:0/3161889150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560337b7800 msgr2=0x55603352a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.216+0000 7f9a4ad65640 1 --2- 172.21.15.154:0/3161889150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560337b7800 0x55603352a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.216+0000 7f9a4ad65640 1 -- 172.21.15.154:0/3161889150 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560334f2ea0 con 0x5560337b7400 2023-09-23T03:21:35.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.216+0000 7fb99ca525c0 1 -- 172.21.15.154:0/746343386 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5602ef363400 msgr2=0x5602ef0e5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.216+0000 7fb99ca525c0 1 --2- 172.21.15.154:0/746343386 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5602ef363400 0x5602ef0e5b80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x5602efe86a50 tx=0x5602efdde180 comp rx=0 tx=0).stop 2023-09-23T03:21:35.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.217+0000 7fb99ca525c0 1 -- 172.21.15.154:0/746343386 shutdown_connections 2023-09-23T03:21:35.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.217+0000 7fb99ca525c0 1 --2- 172.21.15.154:0/746343386 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5602ef363800 0x5602ef0e6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.217+0000 7fb99ca525c0 1 --2- 172.21.15.154:0/746343386 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5602ef363400 0x5602ef0e5b80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.217+0000 7fb99ca525c0 1 --2- 172.21.15.154:0/746343386 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5602ef363000 0x5602ef0e5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.217+0000 7fb99ca525c0 1 -- 172.21.15.154:0/746343386 >> 172.21.15.154:0/746343386 conn(0x5602ef362c00 msgr2=0x5602efdf5000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:35.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.217+0000 7f9a4a564640 1 --2- 172.21.15.154:0/3161889150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560337b7800 0x55603352a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:21:35.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.217+0000 7f9a4ad65640 1 --2- 172.21.15.154:0/3161889150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560337b7400 0x556033529b80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x5560337c2ba0 tx=0x5560337cac00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d69353436a35089a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.217+0000 7efdc3ce2640 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x564662ce9c00 0x564661e83700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.217+0000 7f48e3e0c5c0 1 Processor -- start 2023-09-23T03:21:35.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.217+0000 7efdc3ce2640 1 -- 172.21.15.154:0/1781234207 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x564661ed6a80 con 0x564662ce9800 2023-09-23T03:21:35.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.217+0000 7efdcc0145c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-09-23T03:21:35.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.217+0000 7f48e3e0c5c0 1 -- start start 2023-09-23T03:21:35.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.217+0000 7f9a4655c640 1 -- 172.21.15.154:0/3161889150 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556033507600 con 0x5560337b7400 2023-09-23T03:21:35.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.217+0000 7efdcc0145c0 4 datascan.main: data pool 'cephfs_data' has ID 9 2023-09-23T03:21:35.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.217+0000 7fb99ca525c0 1 -- 172.21.15.154:0/746343386 shutdown_connections 2023-09-23T03:21:35.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.217+0000 7fb99ca525c0 1 -- 172.21.15.154:0/746343386 wait complete. 2023-09-23T03:21:35.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.217+0000 7efdcc0145c0 4 datascan.main: opening data pool 'cephfs_data' 2023-09-23T03:21:35.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.217+0000 7f9a4655c640 1 -- 172.21.15.154:0/3161889150 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5560334e5b00 con 0x5560337b7400 2023-09-23T03:21:35.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.217+0000 7efdcc0145c0 4 datascan.main: data pool 'cephfs_data_extra' has ID 10 2023-09-23T03:21:35.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.218+0000 7f9a4655c640 1 -- 172.21.15.154:0/3161889150 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5560335071e0 con 0x5560337b7400 2023-09-23T03:21:35.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.217+0000 7efdcc0145c0 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-09-23T03:21:35.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.217+0000 7efdcc0145c0 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x564662178000 0x564661e82100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.217+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:1 9.36 9:6db6db6d::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x564661e71880 con 0x564662178000 2023-09-23T03:21:35.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.217+0000 7efdca4ef640 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x564662ce9c00 0x564661e83700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.219+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3161889150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560337b7400 msgr2=0x556033529b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.219+0000 7f48e3e0c5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b099db000 0x556b08b67600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.219+0000 7f9a4d20c5c0 1 --2- 172.21.15.154:0/3161889150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560337b7400 0x556033529b80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x5560337c2ba0 tx=0x5560337cac00 comp rx=0 tx=0).stop 2023-09-23T03:21:35.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.219+0000 7f48e3e0c5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556b099db400 0x556b08b67b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.219+0000 7f48e3e0c5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b099db800 0x556b08b68100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.219+0000 7f48e3e0c5c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b08b46140 con 0x556b099db800 2023-09-23T03:21:35.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.219+0000 7f48e3e0c5c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b08b46b40 con 0x556b099db000 2023-09-23T03:21:35.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.219+0000 7f48e3e0c5c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x556b08bcadc0 con 0x556b099db400 2023-09-23T03:21:35.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.219+0000 7fb99ca525c0 1 Processor -- start 2023-09-23T03:21:35.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.219+0000 7fb99ca525c0 1 -- start start 2023-09-23T03:21:35.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.219+0000 7efdc9cee640 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x564662178000 0x564661e82100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.219+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3161889150 shutdown_connections 2023-09-23T03:21:35.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.219+0000 7f9a4d20c5c0 1 --2- 172.21.15.154:0/3161889150 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560337b7800 0x55603352a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.219+0000 7f9a4d20c5c0 1 --2- 172.21.15.154:0/3161889150 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560337b7400 0x556033529b80 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.220+0000 7f9a4d20c5c0 1 --2- 172.21.15.154:0/3161889150 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5560337b7000 0x556033529600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.219+0000 7f48e2270640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b099db000 0x556b08b67600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.219+0000 7f48e2270640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b099db000 0x556b08b67600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.154:49162/0 (socket says 172.21.15.154:49162) 2023-09-23T03:21:35.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.219+0000 7f48e2270640 1 -- 172.21.15.154:0/80928435 learned_addr learned my addr 172.21.15.154:0/80928435 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.220+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3161889150 >> 172.21.15.154:0/3161889150 conn(0x5560337b6c00 msgr2=0x556034237800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:35.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.220+0000 7f48e126e640 1 --2- 172.21.15.154:0/80928435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b099db800 0x556b08b68100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.220+0000 7f48e1a6f640 1 --2- 172.21.15.154:0/80928435 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556b099db400 0x556b08b67b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.220+0000 7efdca4ef640 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x564662ce9c00 0x564661e83700 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x564661f08810 tx=0x564662be7f20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.220+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3161889150 shutdown_connections 2023-09-23T03:21:35.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.220+0000 7f48e2270640 1 -- 172.21.15.154:0/80928435 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556b099db400 msgr2=0x556b08b67b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.220+0000 7f48e2270640 1 --2- 172.21.15.154:0/80928435 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556b099db400 0x556b08b67b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.220+0000 7f48e2270640 1 -- 172.21.15.154:0/80928435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b099db800 msgr2=0x556b08b68100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.220+0000 7f48e2270640 1 --2- 172.21.15.154:0/80928435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b099db800 0x556b08b68100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.220+0000 7f48e2270640 1 -- 172.21.15.154:0/80928435 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b08b30ea0 con 0x556b099db000 2023-09-23T03:21:35.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.221+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3161889150 wait complete. 2023-09-23T03:21:35.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.221+0000 7efdc9cee640 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x564662178000 0x564661e82100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.221+0000 7efdc9cee640 1 -- 172.21.15.154:0/1781234207 <== osd.1 v2:172.21.15.154:6816/1267710005 1 ==== osd_op_reply(1 [pgnls start_epoch 67 out=101b] v65'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564662cc6b40 con 0x564662178000 2023-09-23T03:21:35.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.221+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:2 9.36 9:6ee5e2c4:::100000001f9.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x564661e71c00 con 0x564662178000 2023-09-23T03:21:35.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.220+0000 7f62bf24b5c0 1 -- start start 2023-09-23T03:21:35.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.220+0000 7f62bf24b5c0 1 Processor -- start 2023-09-23T03:21:35.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.220+0000 7fb99ca525c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5602ef363000 0x5602ef0e7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.220+0000 7fb99ca525c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5602ef363400 0x5602ef0e6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.220+0000 7fb99ca525c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5602ef363800 0x5602ef0e6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.221+0000 7fb99ca525c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602ef148c80 con 0x5602ef363400 2023-09-23T03:21:35.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.221+0000 7fb99ca525c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602ef0c4140 con 0x5602ef363000 2023-09-23T03:21:35.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.221+0000 7fb99ca525c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5602ef0c4b40 con 0x5602ef363800 2023-09-23T03:21:35.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.221+0000 7fb99aac3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5602ef363400 0x5602ef0e6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.221+0000 7fb99aac3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5602ef363400 0x5602ef0e6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60550/0 (socket says 172.21.15.154:60550) 2023-09-23T03:21:35.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.221+0000 7fb99aac3640 1 -- 172.21.15.154:0/2342064636 learned_addr learned my addr 172.21.15.154:0/2342064636 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.222+0000 7f48e2270640 1 --2- 172.21.15.154:0/80928435 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b099db000 0x556b08b67600 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x556b099dc840 tx=0x556b099e0f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfdabadf22ddd937 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.222+0000 7fb99b2c4640 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5602ef363000 0x5602ef0e7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.222+0000 7fb99aac3640 1 -- 172.21.15.154:0/2342064636 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5602ef363800 msgr2=0x5602ef0e6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.222+0000 7fb99a2c2640 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5602ef363800 0x5602ef0e6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.222+0000 7f62bf24b5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b64264c00 0x555b642ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.222+0000 7efdc9cee640 1 -- 172.21.15.154:0/1781234207 <== osd.1 v2:172.21.15.154:6816/1267710005 2 ==== osd_op_reply(2 100000001f9.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564662cc6b40 con 0x564662178000 2023-09-23T03:21:35.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.222+0000 7efdcc0145c0 10 datascan.operator(): handling object 1099511628281.0 2023-09-23T03:21:35.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.222+0000 7fb99aac3640 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5602ef363800 0x5602ef0e6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.222+0000 7f62bf24b5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b64265800 0x555b642ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.222+0000 7fb99aac3640 1 -- 172.21.15.154:0/2342064636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5602ef363000 msgr2=0x5602ef0e7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.222+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:3 9.36 9:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x564662186000 con 0x564662178000 2023-09-23T03:21:35.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.222+0000 7fb99aac3640 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5602ef363000 0x5602ef0e7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.222+0000 7f9a4d20c5c0 1 Processor -- start 2023-09-23T03:21:35.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.222+0000 7fb99aac3640 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602ef0af1e0 con 0x5602ef363400 2023-09-23T03:21:35.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.221+0000 7fe181b915c0 1 -- 172.21.15.154:0/907143925 shutdown_connections 2023-09-23T03:21:35.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.221+0000 7fe181b915c0 1 -- 172.21.15.154:0/907143925 wait complete. 2023-09-23T03:21:35.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.222+0000 7fb99b2c4640 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5602ef363000 0x5602ef0e7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:35.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.222+0000 7fb99a2c2640 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5602ef363800 0x5602ef0e6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:35.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.222+0000 7f9a4d20c5c0 1 -- start start 2023-09-23T03:21:35.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.222+0000 7f48dd266640 1 -- 172.21.15.154:0/80928435 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556b08b45600 con 0x556b099db000 2023-09-23T03:21:35.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.222+0000 7f48dd266640 1 -- 172.21.15.154:0/80928435 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556b098cf080 con 0x556b099db000 2023-09-23T03:21:35.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.222+0000 7f48dd266640 1 -- 172.21.15.154:0/80928435 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556b08b451e0 con 0x556b099db000 2023-09-23T03:21:35.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.223+0000 7fb99aac3640 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5602ef363400 0x5602ef0e6c00 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x5602ef36ea50 tx=0x5602efdde060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.223+0000 7f62bcabb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b64265800 0x555b642ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.223+0000 7f62bcabb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b64265800 0x555b642ceb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60554/0 (socket says 172.21.15.154:60554) 2023-09-23T03:21:35.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.223+0000 7fe181b915c0 1 -- start start 2023-09-23T03:21:35.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.223+0000 7fe181b915c0 1 Processor -- start 2023-09-23T03:21:35.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.223+0000 7f62bcabb640 1 -- 172.21.15.154:0/4061198893 learned_addr learned my addr 172.21.15.154:0/4061198893 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.223+0000 7f62bf24b5c0 1 --2- 172.21.15.154:0/4061198893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b64265c00 0x555b642cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.223+0000 7fb994ab7640 1 -- 172.21.15.154:0/2342064636 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5602ef0c3080 con 0x5602ef363400 2023-09-23T03:21:35.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.223+0000 7fb994ab7640 1 -- 172.21.15.154:0/2342064636 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5602ef0a1800 con 0x5602ef363400 2023-09-23T03:21:35.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.223+0000 7fb994ab7640 1 -- 172.21.15.154:0/2342064636 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5602ef0c31e0 con 0x5602ef363400 2023-09-23T03:21:35.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.223+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5602ef0aeea0 con 0x5602ef363400 2023-09-23T03:21:35.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.224+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/4061198893 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b642af180 con 0x555b64265800 2023-09-23T03:21:35.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.224+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/4061198893 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b64333180 con 0x555b64265c00 2023-09-23T03:21:35.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.223+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/80928435 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b099db000 msgr2=0x556b08b67600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.223+0000 7f48e3e0c5c0 1 --2- 172.21.15.154:0/80928435 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b099db000 0x556b08b67600 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x556b099dc840 tx=0x556b099e0f00 comp rx=0 tx=0).stop 2023-09-23T03:21:35.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.224+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/4061198893 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x555b64332dc0 con 0x555b64264c00 2023-09-23T03:21:35.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.224+0000 7f9a4d20c5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5560337b7000 0x55603352b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.224+0000 7f62bdabd640 1 --2- 172.21.15.154:0/4061198893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b64265c00 0x555b642cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.224+0000 7f9a4d20c5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560337b7400 0x55603352ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.225+0000 7f9a4d20c5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560337b7800 0x55603352a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.225+0000 7fe181b915c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563379aa3800 0x563379b0cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.225+0000 7fe181b915c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563379aa3c00 0x563379b0d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.225+0000 7fe181b915c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56337a83b400 0x563379b0d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.225+0000 7fe181b915c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x563379aec140 con 0x563379aa3c00 2023-09-23T03:21:35.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.225+0000 7fe181b915c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563379aed180 con 0x563379aa3800 2023-09-23T03:21:35.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.225+0000 7fe181b915c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x563379b71180 con 0x56337a83b400 2023-09-23T03:21:35.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.224+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5602ef0aeb60 con 0x5602ef363400 2023-09-23T03:21:35.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.224+0000 7fb99ca525c0 4 datascan.init: resolving metadata pool 8 2023-09-23T03:21:35.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.225+0000 7f9a4a564640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5560337b7000 0x55603352b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.225+0000 7f9a4a564640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5560337b7000 0x55603352b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:34000/0 (socket says 172.21.15.154:34000) 2023-09-23T03:21:35.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.225+0000 7f62bcabb640 1 -- 172.21.15.154:0/4061198893 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b64264c00 msgr2=0x555b642ce580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.225+0000 7f62bcabb640 1 --2- 172.21.15.154:0/4061198893 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b64264c00 0x555b642ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.225+0000 7f9a4ad65640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560337b7400 0x55603352ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.225+0000 7f9a4ad65640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560337b7400 0x55603352ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.154:49200/0 (socket says 172.21.15.154:49200) 2023-09-23T03:21:35.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.225+0000 7f62bcabb640 1 -- 172.21.15.154:0/4061198893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b64265c00 msgr2=0x555b642cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.225+0000 7f62bcabb640 1 --2- 172.21.15.154:0/4061198893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b64265c00 0x555b642cf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.225+0000 7f62bcabb640 1 -- 172.21.15.154:0/4061198893 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b64298b60 con 0x555b64265800 2023-09-23T03:21:35.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.225+0000 7f9a4a564640 1 -- 172.21.15.154:0/3974369953 learned_addr learned my addr 172.21.15.154:0/3974369953 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.225+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55603358cc80 con 0x5560337b7800 2023-09-23T03:21:35.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.225+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55603358cb40 con 0x5560337b7400 2023-09-23T03:21:35.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.225+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55603358ca00 con 0x5560337b7000 2023-09-23T03:21:35.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.224+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/80928435 shutdown_connections 2023-09-23T03:21:35.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.224+0000 7f48e3e0c5c0 1 --2- 172.21.15.154:0/80928435 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b099db800 0x556b08b68100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.224+0000 7f48e3e0c5c0 1 --2- 172.21.15.154:0/80928435 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556b099db400 0x556b08b67b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.224+0000 7f48e3e0c5c0 1 --2- 172.21.15.154:0/80928435 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b099db000 0x556b08b67600 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.224+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/80928435 >> 172.21.15.154:0/80928435 conn(0x556b099dac00 msgr2=0x556b09874800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:35.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.226+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/80928435 shutdown_connections 2023-09-23T03:21:35.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.226+0000 7fe180b3a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56337a83b400 0x563379b0d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.226+0000 7fe180b3a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56337a83b400 0x563379b0d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:34008/0 (socket says 172.21.15.154:34008) 2023-09-23T03:21:35.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.226+0000 7fe180b3a640 1 -- 172.21.15.154:0/697319708 learned_addr learned my addr 172.21.15.154:0/697319708 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.226+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/80928435 wait complete. 2023-09-23T03:21:35.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.226+0000 7fe17fb38640 1 --2- 172.21.15.154:0/697319708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563379aa3c00 0x563379b0d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.226+0000 7f62bcabb640 1 --2- 172.21.15.154:0/4061198893 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b64265800 0x555b642ceb00 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x555b6506b440 tx=0x555b64548240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a21b4d2cd47a10b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.226+0000 7f9a4a564640 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560337b7800 0x55603352a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.226+0000 7fe180339640 1 --2- 172.21.15.154:0/697319708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563379aa3800 0x563379b0cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.227+0000 7f62baab7640 1 -- 172.21.15.154:0/4061198893 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555b642ad600 con 0x555b64265800 2023-09-23T03:21:35.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.227+0000 7f62baab7640 1 -- 172.21.15.154:0/4061198893 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555b6428b800 con 0x555b64265800 2023-09-23T03:21:35.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.227+0000 7f62baab7640 1 -- 172.21.15.154:0/4061198893 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555b642ad1e0 con 0x555b64265800 2023-09-23T03:21:35.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.227+0000 7f48e3e0c5c0 1 Processor -- start 2023-09-23T03:21:35.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.227+0000 7efdc9cee640 1 -- 172.21.15.154:0/1781234207 <== osd.1 v2:172.21.15.154:6816/1267710005 3 ==== osd_op_reply(3 100000001f9.00000000 [call] v67'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564662cc6b40 con 0x564662178000 2023-09-23T03:21:35.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.227+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:4 9.36 9:6fe38dc2::::head [pgnls start_epoch 67 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x564662186380 con 0x564662178000 2023-09-23T03:21:35.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.227+0000 7fe180b3a640 1 -- 172.21.15.154:0/697319708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563379aa3800 msgr2=0x563379b0cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.227+0000 7fe180b3a640 1 --2- 172.21.15.154:0/697319708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563379aa3800 0x563379b0cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.227+0000 7fe180b3a640 1 -- 172.21.15.154:0/697319708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563379aa3c00 msgr2=0x563379b0d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.227+0000 7fe180b3a640 1 --2- 172.21.15.154:0/697319708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563379aa3c00 0x563379b0d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.227+0000 7fe180b3a640 1 -- 172.21.15.154:0/697319708 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563379ad6b60 con 0x56337a83b400 2023-09-23T03:21:35.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.227+0000 7fe17fb38640 1 --2- 172.21.15.154:0/697319708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563379aa3c00 0x563379b0d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:21:35.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.227+0000 7f48e3e0c5c0 1 -- start start 2023-09-23T03:21:35.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.227+0000 7efdc9cee640 1 -- 172.21.15.154:0/1781234207 <== osd.1 v2:172.21.15.154:6816/1267710005 4 ==== osd_op_reply(4 [pgnls start_epoch 67 out=81b] v67'18 uv18 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564662cc6b40 con 0x564662178000 2023-09-23T03:21:35.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.227+0000 7f9a4ad65640 1 -- 172.21.15.154:0/3974369953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5560337b7000 msgr2=0x55603352b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.227+0000 7f9a4ad65640 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5560337b7000 0x55603352b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.227+0000 7f9a4ad65640 1 -- 172.21.15.154:0/3974369953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560337b7800 msgr2=0x55603352a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.227+0000 7f9a4ad65640 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560337b7800 0x55603352a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.228+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:5 9.36 9:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x564662186700 con 0x564662178000 2023-09-23T03:21:35.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.227+0000 7f9a4ad65640 1 -- 172.21.15.154:0/3974369953 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560334f2b60 con 0x5560337b7400 2023-09-23T03:21:35.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.228+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/4061198893 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555b64298340 con 0x555b64265800 2023-09-23T03:21:35.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.228+0000 7efdc9cee640 1 -- 172.21.15.154:0/1781234207 <== osd.1 v2:172.21.15.154:6816/1267710005 5 ==== osd_op_reply(5 100000001f7.00000000 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564662cc6b40 con 0x564662178000 2023-09-23T03:21:35.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.228+0000 7efdcc0145c0 10 datascan.operator(): handling object 1099511628279.0 2023-09-23T03:21:35.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.228+0000 7fb994ab7640 1 -- 172.21.15.154:0/2342064636 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x5602ef0ad080 con 0x5602ef363400 2023-09-23T03:21:35.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.228+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:6 9.36 9:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x564662186a80 con 0x564662178000 2023-09-23T03:21:35.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.228+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/4061198893 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x555b64298000 con 0x555b64265800 2023-09-23T03:21:35.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.228+0000 7f62bf24b5c0 4 waiting for MDS map... 2023-09-23T03:21:35.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.228+0000 7f62baab7640 1 -- 172.21.15.154:0/4061198893 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x555b64324c40 con 0x555b64265800 2023-09-23T03:21:35.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.228+0000 7fe180b3a640 1 --2- 172.21.15.154:0/697319708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56337a83b400 0x563379b0d600 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x56337a8a74d0 tx=0x563379d9c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d90ed76ad6897511 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.228+0000 7f9a4ad65640 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560337b7400 0x55603352ac00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x5560337c2de0 tx=0x55603429a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.228+0000 7f62baab7640 1 -- 172.21.15.154:0/4061198893 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x555b6454c000 con 0x555b64265800 2023-09-23T03:21:35.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.228+0000 7f62bf24b5c0 4 Got MDS map 47 2023-09-23T03:21:35.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.229+0000 7ffaa0a525c0 1 -- start start 2023-09-23T03:21:35.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.229+0000 7f62bf24b5c0 4 datascan.main: Using metadata pool output 2023-09-23T03:21:35.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.229+0000 7f62bf24b5c0 4 datascan.main: connecting to RADOS... 2023-09-23T03:21:35.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.229+0000 7ffaa0a525c0 1 Processor -- start 2023-09-23T03:21:35.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.229+0000 7fe17db34640 1 -- 172.21.15.154:0/697319708 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563379aeb600 con 0x56337a83b400 2023-09-23T03:21:35.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.229+0000 7fe17db34640 1 -- 172.21.15.154:0/697319708 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563379ac8a80 con 0x56337a83b400 2023-09-23T03:21:35.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.229+0000 7fe17db34640 1 -- 172.21.15.154:0/697319708 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563379aeb1e0 con 0x56337a83b400 2023-09-23T03:21:35.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.229+0000 7f9a44d59640 1 -- 172.21.15.154:0/3974369953 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556033506dc0 con 0x5560337b7400 2023-09-23T03:21:35.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.229+0000 7f9a44d59640 1 -- 172.21.15.154:0/3974369953 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55603429d680 con 0x5560337b7400 2023-09-23T03:21:35.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.229+0000 7f9a44d59640 1 -- 172.21.15.154:0/3974369953 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556033507600 con 0x5560337b7400 2023-09-23T03:21:35.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.229+0000 7f48e3e0c5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b099db000 0x556b08b69180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.229+0000 7fb994ab7640 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5602ef363c00 0x5602ef0e7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.229+0000 7f48e3e0c5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556b099db400 0x556b08b68c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.229+0000 7f48e3e0c5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b099db800 0x556b08b68680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.230+0000 7f48e3e0c5c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b08bcac80 con 0x556b099db000 2023-09-23T03:21:35.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.230+0000 7f48e3e0c5c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b08bcab40 con 0x556b099db800 2023-09-23T03:21:35.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.230+0000 7f48e3e0c5c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x556b08bcaa00 con 0x556b099db400 2023-09-23T03:21:35.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.230+0000 7f62bf24b5c0 1 Processor -- start 2023-09-23T03:21:35.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.230+0000 7fb994ab7640 1 -- 172.21.15.154:0/2342064636 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x5602ef13aa80 con 0x5602ef363400 2023-09-23T03:21:35.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.230+0000 7f62bf24b5c0 1 -- start start 2023-09-23T03:21:35.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.230+0000 7fe181b915c0 1 -- 172.21.15.154:0/697319708 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563379ad6340 con 0x56337a83b400 2023-09-23T03:21:35.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.230+0000 7fb99b2c4640 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5602ef363c00 0x5602ef0e7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.229+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5560334f2340 con 0x5560337b7400 2023-09-23T03:21:35.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.229+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5560334f3520 con 0x5560337b7400 2023-09-23T03:21:35.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.230+0000 7fe181b915c0 1 -- 172.21.15.154:0/697319708 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({fsmap=0}) v3 -- 0x563379ad6000 con 0x56337a83b400 2023-09-23T03:21:35.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.229+0000 7f9a4d20c5c0 4 datascan.init: resolving metadata pool 8 2023-09-23T03:21:35.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.230+0000 7fe181b915c0 4 waiting for MDS map... 2023-09-23T03:21:35.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.231+0000 7f48e2270640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b099db000 0x556b08b69180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.231+0000 7f48e2270640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b099db000 0x556b08b69180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60570/0 (socket says 172.21.15.154:60570) 2023-09-23T03:21:35.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.231+0000 7f48e2270640 1 -- 172.21.15.154:0/2963625397 learned_addr learned my addr 172.21.15.154:0/2963625397 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.231+0000 7f48e1a6f640 1 --2- 172.21.15.154:0/2963625397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556b099db400 0x556b08b68c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.231+0000 7f48e2270640 1 -- 172.21.15.154:0/2963625397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556b099db400 msgr2=0x556b08b68c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.231+0000 7f48e2270640 1 --2- 172.21.15.154:0/2963625397 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556b099db400 0x556b08b68c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.231+0000 7f48e2270640 1 -- 172.21.15.154:0/2963625397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b099db800 msgr2=0x556b08b68680 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-23T03:21:35.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.231+0000 7f48e2270640 1 --2- 172.21.15.154:0/2963625397 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b099db800 0x556b08b68680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.231+0000 7f48e2270640 1 -- 172.21.15.154:0/2963625397 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b08b30b60 con 0x556b099db000 2023-09-23T03:21:35.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.231+0000 7efdc9cee640 1 -- 172.21.15.154:0/1781234207 <== osd.1 v2:172.21.15.154:6816/1267710005 6 ==== osd_op_reply(6 100000001f7.00000000 [call] v67'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564662cc6b40 con 0x564662178000 2023-09-23T03:21:35.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.231+0000 7efdcc0145c0 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x564662178400 0x564661e81b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.231+0000 7fe17db34640 1 -- 172.21.15.154:0/697319708 <== mon.2 v2:172.21.15.190:3301/0 4 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x563379b62c40 con 0x56337a83b400 2023-09-23T03:21:35.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.231+0000 7fe17db34640 1 -- 172.21.15.154:0/697319708 <== mon.2 v2:172.21.15.190:3301/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x563379af4f00 con 0x56337a83b400 2023-09-23T03:21:35.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.231+0000 7ffaa0a525c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d6ae870c00 0x55d6ae8da580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.231+0000 7ffaa0a525c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d6ae871800 0x55d6ae8dab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.231+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:7 9.e 9:70000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x564662186000 con 0x564662178400 2023-09-23T03:21:35.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.232+0000 7fe181b915c0 4 Got MDS map 47 2023-09-23T03:21:35.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.232+0000 7ffaa0a525c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d6ae871c00 0x55d6ae8db080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.231+0000 7fb99ca525c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-09-23T03:21:35.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.231+0000 7fb99ca525c0 4 datascan.main: data pool 'cephfs_data' has ID 9 2023-09-23T03:21:35.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.231+0000 7fb99ca525c0 4 datascan.main: opening data pool 'cephfs_data' 2023-09-23T03:21:35.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.231+0000 7fb99ca525c0 4 datascan.main: data pool 'cephfs_data_extra' has ID 10 2023-09-23T03:21:35.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.231+0000 7fb99ca525c0 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-09-23T03:21:35.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.232+0000 7f62bf24b5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b6455d000 0x555b642cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.231+0000 7fb99ca525c0 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5602ef4ce000 0x5602ef0e6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.231+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:1 9.0 9:00000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef0d5880 con 0x5602ef4ce000 2023-09-23T03:21:35.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.232+0000 7fb99b2c4640 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5602ef363c00 0x5602ef0e7700 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x5602ef36f500 tx=0x5602ef377da0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.232+0000 7fe181b915c0 4 datascan.main: Using metadata pool output 2023-09-23T03:21:35.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.232+0000 7ffa9e2c2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d6ae871800 0x55d6ae8dab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.232+0000 7ffa9e2c2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d6ae871800 0x55d6ae8dab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60580/0 (socket says 172.21.15.154:60580) 2023-09-23T03:21:35.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.232+0000 7fe181b915c0 4 datascan.main: connecting to RADOS... 2023-09-23T03:21:35.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.232+0000 7ffaa0a525c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ae8bb180 con 0x55d6ae871800 2023-09-23T03:21:35.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.232+0000 7ffaa0a525c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ae8ba140 con 0x55d6ae871c00 2023-09-23T03:21:35.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.232+0000 7ffaa0a525c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d6ae8ba000 con 0x55d6ae870c00 2023-09-23T03:21:35.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.232+0000 7ffa9e2c2640 1 -- 172.21.15.154:0/1822334085 learned_addr learned my addr 172.21.15.154:0/1822334085 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.232+0000 7efdca4ef640 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x564662178400 0x564661e81b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.232+0000 7f62bf24b5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b6455d400 0x555b642cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.232+0000 7f62bf24b5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6455d800 0x555b642d0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.232+0000 7fb99a2c2640 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5602ef4ce000 0x5602ef0e6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.232+0000 7f48e2270640 1 --2- 172.21.15.154:0/2963625397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b099db000 0x556b08b69180 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x556b099dd500 tx=0x556b099be1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.233+0000 7f62bf24b5c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b642ae140 con 0x555b6455d000 2023-09-23T03:21:35.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.233+0000 7f62bdabd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b6455d000 0x555b642cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.233+0000 7f62bf24b5c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b642aeb40 con 0x555b6455d800 2023-09-23T03:21:35.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.233+0000 7ffa9eac3640 1 --2- 172.21.15.154:0/1822334085 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d6ae870c00 0x55d6ae8da580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.233+0000 7f62bf24b5c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x555b64333180 con 0x555b6455d400 2023-09-23T03:21:35.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.233+0000 7ffa9f2c4640 1 --2- 172.21.15.154:0/1822334085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d6ae871c00 0x55d6ae8db080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.233+0000 7f62bdabd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b6455d000 0x555b642cf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60596/0 (socket says 172.21.15.154:60596) 2023-09-23T03:21:35.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.233+0000 7f62bdabd640 1 -- 172.21.15.154:0/3815136362 learned_addr learned my addr 172.21.15.154:0/3815136362 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.233+0000 7ffa9e2c2640 1 -- 172.21.15.154:0/1822334085 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d6ae870c00 msgr2=0x55d6ae8da580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.233+0000 7ffa9e2c2640 1 --2- 172.21.15.154:0/1822334085 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d6ae870c00 0x55d6ae8da580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.233+0000 7ffa9e2c2640 1 -- 172.21.15.154:0/1822334085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d6ae871c00 msgr2=0x55d6ae8db080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.233+0000 7ffa9e2c2640 1 --2- 172.21.15.154:0/1822334085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d6ae871c00 0x55d6ae8db080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.233+0000 7ffa9e2c2640 1 -- 172.21.15.154:0/1822334085 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6ae8a4b60 con 0x55d6ae871800 2023-09-23T03:21:35.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.233+0000 7f62bd2bc640 1 --2- 172.21.15.154:0/3815136362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b6455d400 0x555b642cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.233+0000 7f48dba63640 1 -- 172.21.15.154:0/2963625397 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556b08b44f20 con 0x556b099db000 2023-09-23T03:21:35.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.233+0000 7f48dba63640 1 -- 172.21.15.154:0/2963625397 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556b098cf680 con 0x556b099db000 2023-09-23T03:21:35.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.233+0000 7f48dba63640 1 -- 172.21.15.154:0/2963625397 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556b08b45600 con 0x556b099db000 2023-09-23T03:21:35.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.233+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556b08b30340 con 0x556b099db000 2023-09-23T03:21:35.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.234+0000 7fe181b915c0 1 Processor -- start 2023-09-23T03:21:35.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.234+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b08b31520 con 0x556b099db000 2023-09-23T03:21:35.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.234+0000 7f48e3e0c5c0 4 datascan.init: resolving metadata pool 8 2023-09-23T03:21:35.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.234+0000 7efdca4ef640 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x564662178400 0x564661e81b80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.234+0000 7fe181b915c0 1 -- start start 2023-09-23T03:21:35.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.234+0000 7f62bdabd640 1 -- 172.21.15.154:0/3815136362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b6455d400 msgr2=0x555b642cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.234+0000 7f62bdabd640 1 --2- 172.21.15.154:0/3815136362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b6455d400 0x555b642cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.234+0000 7f62bdabd640 1 -- 172.21.15.154:0/3815136362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6455d800 msgr2=0x555b642d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.234+0000 7f62bdabd640 1 --2- 172.21.15.154:0/3815136362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6455d800 0x555b642d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.234+0000 7f62bdabd640 1 -- 172.21.15.154:0/3815136362 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b64298ea0 con 0x555b6455d000 2023-09-23T03:21:35.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.235+0000 7f62bd2bc640 1 --2- 172.21.15.154:0/3815136362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b6455d400 0x555b642cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:35.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.235+0000 7f62bdabd640 1 --2- 172.21.15.154:0/3815136362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b6455d000 0x555b642cf600 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x555b6455ea20 tx=0x555b64564f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8924d630fe435562 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.235+0000 7efdca4ef640 1 -- 172.21.15.154:0/1781234207 <== osd.7 v2:172.21.15.190:6824/3460136272 1 ==== osd_op_reply(7 [pgnls start_epoch 67 out=49b] v65'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564662cc6d80 con 0x564662178400 2023-09-23T03:21:35.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.235+0000 7efdc4ce4640 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:8 9.2e 9:74000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x564662187500 con 0x564662178400 2023-09-23T03:21:35.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.235+0000 7efdca4ef640 1 -- 172.21.15.154:0/1781234207 <== osd.7 v2:172.21.15.190:6824/3460136272 2 ==== osd_op_reply(8 [pgnls start_epoch 67 out=49b] v65'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564662cc6d80 con 0x564662178400 2023-09-23T03:21:35.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.235+0000 7efdc54e5640 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x564662196000 0x56466219c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.235+0000 7efdc54e5640 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:9 9.1e 9:78000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x564662186000 con 0x564662196000 2023-09-23T03:21:35.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.233+0000 7fb99a2c2640 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5602ef4ce000 0x5602ef0e6100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.234+0000 7fb99a2c2640 1 -- 172.21.15.154:0/2342064636 <== osd.1 v2:172.21.15.154:6816/1267710005 1 ==== osd_op_reply(1 [pgnls start_epoch 67 out=81b] v65'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5602efe4eb40 con 0x5602ef4ce000 2023-09-23T03:21:35.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.236+0000 7fe181b915c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56337a83a800 0x563379b0db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.235+0000 7f62b8ab3640 1 -- 172.21.15.154:0/3815136362 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555b642ad600 con 0x555b6455d000 2023-09-23T03:21:35.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.236+0000 7f62b8ab3640 1 -- 172.21.15.154:0/3815136362 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555b6428b800 con 0x555b6455d000 2023-09-23T03:21:35.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.236+0000 7fe181b915c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563379aa2c00 0x563379b0e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.236+0000 7f62b8ab3640 1 -- 172.21.15.154:0/3815136362 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555b642ad1e0 con 0x555b6455d000 2023-09-23T03:21:35.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.236+0000 7efdc9cee640 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x564662196000 0x56466219c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.236+0000 7fe181b915c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563379d9ec00 0x563379b0e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.233+0000 7ffa9e2c2640 1 --2- 172.21.15.154:0/1822334085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d6ae871800 0x55d6ae8dab00 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x55d6af669410 tx=0x55d6aeb48420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b511447a2d27b8d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.234+0000 7ffa9c2be640 1 -- 172.21.15.154:0/1822334085 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d6ae8b9600 con 0x55d6ae871800 2023-09-23T03:21:35.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.234+0000 7ffa9c2be640 1 -- 172.21.15.154:0/1822334085 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d6ae897b00 con 0x55d6ae871800 2023-09-23T03:21:35.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.234+0000 7ffa9c2be640 1 -- 172.21.15.154:0/1822334085 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d6ae8b91e0 con 0x55d6ae871800 2023-09-23T03:21:35.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.234+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/1822334085 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d6ae8a4340 con 0x55d6ae871800 2023-09-23T03:21:35.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.234+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/1822334085 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55d6ae8a4000 con 0x55d6ae871800 2023-09-23T03:21:35.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.234+0000 7ffaa0a525c0 4 waiting for MDS map... 2023-09-23T03:21:35.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.234+0000 7ffa9c2be640 1 -- 172.21.15.154:0/1822334085 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x55d6ae930c40 con 0x55d6ae871800 2023-09-23T03:21:35.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.236+0000 7ffa9c2be640 1 -- 172.21.15.154:0/1822334085 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x55d6aeb5c000 con 0x55d6ae871800 2023-09-23T03:21:35.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.236+0000 7ffaa0a525c0 4 Got MDS map 47 2023-09-23T03:21:35.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.236+0000 7ffaa0a525c0 4 datascan.main: Using metadata pool output 2023-09-23T03:21:35.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.236+0000 7ffaa0a525c0 4 datascan.main: connecting to RADOS... 2023-09-23T03:21:35.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.236+0000 7fe181b915c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x563379b70dc0 con 0x563379d9ec00 2023-09-23T03:21:35.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.236+0000 7fe181b915c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563379b70c80 con 0x56337a83a800 2023-09-23T03:21:35.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.236+0000 7fe181b915c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x563379aecb40 con 0x563379aa2c00 2023-09-23T03:21:35.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.237+0000 7fe180339640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563379aa2c00 0x563379b0e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.237+0000 7f48dba63640 1 -- 172.21.15.154:0/2963625397 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x556b08b2e840 con 0x556b099db000 2023-09-23T03:21:35.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.237+0000 7efdc9cee640 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x564662196000 0x56466219c000 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.237+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:2 9.0 9:012cdbc7:::10000000206.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef0d5c00 con 0x5602ef4ce000 2023-09-23T03:21:35.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.238+0000 7efdc9cee640 1 -- 172.21.15.154:0/1781234207 <== osd.0 v2:172.21.15.154:6800/2920938138 1 ==== osd_op_reply(9 [pgnls start_epoch 67 out=81b] v65'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564662cc6b40 con 0x564662196000 2023-09-23T03:21:35.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.238+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:10 9.1e 9:789b801b:::1000000020c.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x564662186e00 con 0x564662196000 2023-09-23T03:21:35.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.238+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/3815136362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b6455d000 msgr2=0x555b642cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.238+0000 7f62bf24b5c0 1 --2- 172.21.15.154:0/3815136362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b6455d000 0x555b642cf600 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x555b6455ea20 tx=0x555b64564f00 comp rx=0 tx=0).stop 2023-09-23T03:21:35.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.237+0000 7ffaa0a525c0 1 Processor -- start 2023-09-23T03:21:35.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.237+0000 7ffaa0a525c0 1 -- start start 2023-09-23T03:21:35.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.238+0000 7fb99a2c2640 1 -- 172.21.15.154:0/2342064636 <== osd.1 v2:172.21.15.154:6816/1267710005 2 ==== osd_op_reply(2 10000000206.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5602efe4eb40 con 0x5602ef4ce000 2023-09-23T03:21:35.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.235+0000 7f9a44d59640 1 -- 172.21.15.154:0/3974369953 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x5560334f1340 con 0x5560337b7400 2023-09-23T03:21:35.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.237+0000 7f9a44d59640 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55603382a800 0x55603352b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.237+0000 7f9a44d59640 1 -- 172.21.15.154:0/3974369953 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x55603357ea80 con 0x5560337b7400 2023-09-23T03:21:35.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.237+0000 7f9a4d20c5c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-09-23T03:21:35.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.238+0000 7fb99ca525c0 10 datascan.operator(): handling object 1099511628294.0 2023-09-23T03:21:35.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.237+0000 7f9a4d20c5c0 4 datascan.main: data pool 'cephfs_data' has ID 9 2023-09-23T03:21:35.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.237+0000 7f9a4d20c5c0 4 datascan.main: opening data pool 'cephfs_data' 2023-09-23T03:21:35.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.237+0000 7f9a4d20c5c0 4 datascan.main: data pool 'cephfs_data_extra' has ID 10 2023-09-23T03:21:35.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.237+0000 7f9a4d20c5c0 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-09-23T03:21:35.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.237+0000 7f9a4d20c5c0 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55603382ac00 0x55603352a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.237+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:1 9.24 9:24924924::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x556033519880 con 0x55603382ac00 2023-09-23T03:21:35.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.237+0000 7f9a4a564640 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55603382a800 0x55603352b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.238+0000 7f9a4a564640 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55603382ac00 0x55603352a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.238+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:3 9.0 9:012cdbc7:::10000000206.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef0d5180 con 0x5602ef4ce000 2023-09-23T03:21:35.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.238+0000 7f48dba63640 1 --2- 172.21.15.154:0/2963625397 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x556b09a40800 0x556b08b69700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.238+0000 7efdc9cee640 1 -- 172.21.15.154:0/1781234207 <== osd.0 v2:172.21.15.154:6800/2920938138 2 ==== osd_op_reply(10 1000000020c.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564662cc6b40 con 0x564662196000 2023-09-23T03:21:35.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.238+0000 7efdcc0145c0 10 datascan.operator(): handling object 1099511628300.0 2023-09-23T03:21:35.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.238+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/3815136362 shutdown_connections 2023-09-23T03:21:35.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.238+0000 7f48dba63640 1 -- 172.21.15.154:0/2963625397 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x556b08bbca80 con 0x556b099db000 2023-09-23T03:21:35.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.238+0000 7f48e3e0c5c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-09-23T03:21:35.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.238+0000 7f48e3e0c5c0 4 datascan.main: data pool 'cephfs_data' has ID 9 2023-09-23T03:21:35.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.238+0000 7f62bf24b5c0 1 --2- 172.21.15.154:0/3815136362 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6455d800 0x555b642d0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.238+0000 7f48e3e0c5c0 4 datascan.main: opening data pool 'cephfs_data' 2023-09-23T03:21:35.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.238+0000 7f48e3e0c5c0 4 datascan.main: data pool 'cephfs_data_extra' has ID 10 2023-09-23T03:21:35.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.238+0000 7f48e3e0c5c0 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-09-23T03:21:35.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.238+0000 7f48e3e0c5c0 1 --2- 172.21.15.154:0/2963625397 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x556b09a40c00 0x556b08b68100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.238+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:1 9.12 9:49249249::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x556b08b57880 con 0x556b09a40c00 2023-09-23T03:21:35.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.238+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:11 9.1e 9:789b801b:::1000000020c.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x564662187180 con 0x564662196000 2023-09-23T03:21:35.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.238+0000 7f62bf24b5c0 1 --2- 172.21.15.154:0/3815136362 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b6455d400 0x555b642cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.238+0000 7f62bf24b5c0 1 --2- 172.21.15.154:0/3815136362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b6455d000 0x555b642cf600 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.238+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/3815136362 >> 172.21.15.154:0/3815136362 conn(0x555b6455cc00 msgr2=0x555b64fdc800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:35.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.239+0000 7f48e1a6f640 1 --2- 172.21.15.154:0/2963625397 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x556b09a40800 0x556b08b69700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.238+0000 7ffaa0a525c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d6aeb63000 0x55d6ae8db600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.239+0000 7ffaa0a525c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d6aeb63400 0x55d6ae8dbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.239+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/3815136362 shutdown_connections 2023-09-23T03:21:35.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.239+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/3815136362 wait complete. 2023-09-23T03:21:35.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.239+0000 7ffaa0a525c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d6aeb63800 0x55d6ae8dc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.239+0000 7ffa9eac3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d6aeb63400 0x55d6ae8dbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.239+0000 7ffa9eac3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d6aeb63400 0x55d6ae8dbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60606/0 (socket says 172.21.15.154:60606) 2023-09-23T03:21:35.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.239+0000 7ffa9f2c4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d6aeb63000 0x55d6ae8db600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.239+0000 7ffa9f2c4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d6aeb63000 0x55d6ae8db600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:34076/0 (socket says 172.21.15.154:34076) 2023-09-23T03:21:35.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.237+0000 7fe180339640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563379aa2c00 0x563379b0e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:34064/0 (socket says 172.21.15.154:34064) 2023-09-23T03:21:35.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.238+0000 7fe180339640 1 -- 172.21.15.154:0/2416081981 learned_addr learned my addr 172.21.15.154:0/2416081981 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.237+0000 7fe17fb38640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56337a83a800 0x563379b0db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.239+0000 7f9a4a564640 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55603382ac00 0x55603352a100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.240+0000 7f9a4a564640 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55603382a800 0x55603352b700 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x5560337c2930 tx=0x55603429a120 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.240+0000 7f9a4a564640 1 -- 172.21.15.154:0/3974369953 <== osd.2 v2:172.21.15.154:6822/2884407243 1 ==== osd_op_reply(1 [pgnls start_epoch 67 out=49b] v65'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556033798b40 con 0x55603382ac00 2023-09-23T03:21:35.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.240+0000 7fe17fb38640 1 --2- 172.21.15.154:0/2416081981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563379d9ec00 0x563379b0e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.240+0000 7fe17fb38640 1 -- 172.21.15.154:0/2416081981 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563379aa2c00 msgr2=0x563379b0e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.240+0000 7f9a45d5b640 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55603392c000 0x556033932000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.240+0000 7f48e1a6f640 1 --2- 172.21.15.154:0/2963625397 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x556b09a40800 0x556b08b69700 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x556b099dc5d0 tx=0x556b099be0c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.240+0000 7ffa9e2c2640 1 --2- 172.21.15.154:0/1953746562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d6aeb63800 0x55d6ae8dc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.239+0000 7ffa9eac3640 1 -- 172.21.15.154:0/1953746562 learned_addr learned my addr 172.21.15.154:0/1953746562 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.240+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/1953746562 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ae8baa00 con 0x55d6aeb63400 2023-09-23T03:21:35.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.240+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/1953746562 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ae93f180 con 0x55d6aeb63800 2023-09-23T03:21:35.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.240+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/1953746562 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d6ae93edc0 con 0x55d6aeb63000 2023-09-23T03:21:35.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.240+0000 7f62bf24b5c0 1 Processor -- start 2023-09-23T03:21:35.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.240+0000 7fe17fb38640 1 --2- 172.21.15.154:0/2416081981 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563379aa2c00 0x563379b0e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.240+0000 7fe17fb38640 1 -- 172.21.15.154:0/2416081981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563379d9ec00 msgr2=0x563379b0e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.240+0000 7fe17fb38640 1 --2- 172.21.15.154:0/2416081981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563379d9ec00 0x563379b0e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.240+0000 7fe17fb38640 1 -- 172.21.15.154:0/2416081981 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563379ad6ea0 con 0x56337a83a800 2023-09-23T03:21:35.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.241+0000 7fe180339640 1 --2- 172.21.15.154:0/2416081981 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563379aa2c00 0x563379b0e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:21:35.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.241+0000 7f62bf24b5c0 1 -- start start 2023-09-23T03:21:35.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.241+0000 7f9a45d5b640 1 -- 172.21.15.154:0/3974369953 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:2 9.14 9:28000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x556033936000 con 0x55603392c000 2023-09-23T03:21:35.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.241+0000 7f48e126e640 1 --2- 172.21.15.154:0/2963625397 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x556b09a40c00 0x556b08b68100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.241+0000 7ffa9e2c2640 1 -- 172.21.15.154:0/1953746562 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d6aeb63000 msgr2=0x55d6ae8db600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.241+0000 7ffa9e2c2640 1 --2- 172.21.15.154:0/1953746562 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d6aeb63000 0x55d6ae8db600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.241+0000 7ffa9e2c2640 1 -- 172.21.15.154:0/1953746562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d6aeb63400 msgr2=0x55d6ae8dbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.241+0000 7ffa9e2c2640 1 --2- 172.21.15.154:0/1953746562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d6aeb63400 0x55d6ae8dbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.241+0000 7ffa9e2c2640 1 -- 172.21.15.154:0/1953746562 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6ae8a4ea0 con 0x55d6aeb63800 2023-09-23T03:21:35.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.241+0000 7ffa9eac3640 1 --2- 172.21.15.154:0/1953746562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d6aeb63400 0x55d6ae8dbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:21:35.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.241+0000 7f9a4b566640 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55603392c000 0x556033932000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.241+0000 7fe17fb38640 1 --2- 172.21.15.154:0/2416081981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56337a83a800 0x563379b0db80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x563379d967e0 tx=0x56337a86ac60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81096f3ea0a97378 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.242+0000 7ffa9e2c2640 1 --2- 172.21.15.154:0/1953746562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d6aeb63800 0x55d6ae8dc100 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x55d6aeb805a0 tx=0x55d6aeb48060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d5ea2376a5daa52 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.242+0000 7f9a4b566640 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55603392c000 0x556033932000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.242+0000 7fe17bb30640 1 -- 172.21.15.154:0/2416081981 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563379aeb600 con 0x56337a83a800 2023-09-23T03:21:35.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.242+0000 7f62bf24b5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b6455d000 0x555b642d1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.242+0000 7f62bf24b5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b6455d400 0x555b642d0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.242+0000 7f62bf24b5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6455d800 0x555b642d0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.242+0000 7fe17bb30640 1 -- 172.21.15.154:0/2416081981 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563379d92d80 con 0x56337a83a800 2023-09-23T03:21:35.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.242+0000 7f62bf24b5c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b64332c80 con 0x555b6455d400 2023-09-23T03:21:35.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.242+0000 7efdc9cee640 1 -- 172.21.15.154:0/1781234207 <== osd.0 v2:172.21.15.154:6800/2920938138 3 ==== osd_op_reply(11 1000000020c.00000000 [call] v67'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564662cc6b40 con 0x564662196000 2023-09-23T03:21:35.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.242+0000 7f62bf24b5c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b64332b40 con 0x555b6455d800 2023-09-23T03:21:35.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.242+0000 7fe17bb30640 1 -- 172.21.15.154:0/2416081981 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563379aeb1e0 con 0x56337a83a800 2023-09-23T03:21:35.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.242+0000 7f62bf24b5c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x555b64332a00 con 0x555b6455d000 2023-09-23T03:21:35.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.242+0000 7efdcc0145c0 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x564662178800 0x564661e81600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.243+0000 7ffa9a2ba640 1 -- 172.21.15.154:0/1953746562 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d6ae8b8f20 con 0x55d6aeb63800 2023-09-23T03:21:35.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.243+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:12 9.3e 9:7c000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x564661e71880 con 0x564662178800 2023-09-23T03:21:35.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.243+0000 7f62bd2bc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b6455d400 0x555b642d0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.243+0000 7ffa9a2ba640 1 -- 172.21.15.154:0/1953746562 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d6ae9ea780 con 0x55d6aeb63800 2023-09-23T03:21:35.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.243+0000 7ffa9a2ba640 1 -- 172.21.15.154:0/1953746562 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d6ae8b9600 con 0x55d6aeb63800 2023-09-23T03:21:35.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.243+0000 7f62bdabd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b6455d000 0x555b642d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.243+0000 7f62bdabd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b6455d000 0x555b642d1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:34088/0 (socket says 172.21.15.154:34088) 2023-09-23T03:21:35.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.243+0000 7f62bdabd640 1 -- 172.21.15.154:0/1011343899 learned_addr learned my addr 172.21.15.154:0/1011343899 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.243+0000 7f48e126e640 1 --2- 172.21.15.154:0/2963625397 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x556b09a40c00 0x556b08b68100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.243+0000 7efdc94ed640 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x564662178800 0x564661e81600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.243+0000 7f62bcabb640 1 --2- 172.21.15.154:0/1011343899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6455d800 0x555b642d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.243+0000 7fb99a2c2640 1 -- 172.21.15.154:0/2342064636 <== osd.1 v2:172.21.15.154:6816/1267710005 3 ==== osd_op_reply(3 10000000206.00000000 [call] v67'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5602efe4eb40 con 0x5602ef4ce000 2023-09-23T03:21:35.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.243+0000 7fb99ca525c0 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5602ef4ce400 0x5602ef0e5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.244+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:4 9.20 9:04000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef0d5880 con 0x5602ef4ce400 2023-09-23T03:21:35.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.244+0000 7fe181b915c0 1 -- 172.21.15.154:0/2416081981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56337a83a800 msgr2=0x563379b0db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.243+0000 7f9a4b566640 1 -- 172.21.15.154:0/3974369953 <== osd.1 v2:172.21.15.154:6816/1267710005 1 ==== osd_op_reply(2 [pgnls start_epoch 67 out=49b] v65'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556033798d80 con 0x55603392c000 2023-09-23T03:21:35.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.244+0000 7f9a4655c640 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55603392c400 0x556033932580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.244+0000 7f9a4655c640 1 -- 172.21.15.154:0/3974369953 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:3 9.34 9:2c000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x556033936380 con 0x55603392c400 2023-09-23T03:21:35.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.244+0000 7fe181b915c0 1 --2- 172.21.15.154:0/2416081981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56337a83a800 0x563379b0db80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x563379d967e0 tx=0x56337a86ac60 comp rx=0 tx=0).stop 2023-09-23T03:21:35.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.244+0000 7efdc94ed640 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x564662178800 0x564661e81600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.244+0000 7fe181b915c0 1 -- 172.21.15.154:0/2416081981 shutdown_connections 2023-09-23T03:21:35.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.244+0000 7fe181b915c0 1 --2- 172.21.15.154:0/2416081981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563379d9ec00 0x563379b0e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.244+0000 7fe181b915c0 1 --2- 172.21.15.154:0/2416081981 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563379aa2c00 0x563379b0e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.244+0000 7fe181b915c0 1 --2- 172.21.15.154:0/2416081981 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56337a83a800 0x563379b0db80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.244+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/1953746562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d6aeb63800 msgr2=0x55d6ae8dc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.244+0000 7fe181b915c0 1 -- 172.21.15.154:0/2416081981 >> 172.21.15.154:0/2416081981 conn(0x56337a83b000 msgr2=0x56337a81a800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:35.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.244+0000 7f62bdabd640 1 -- 172.21.15.154:0/1011343899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6455d800 msgr2=0x555b642d0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.244+0000 7f62bdabd640 1 --2- 172.21.15.154:0/1011343899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b6455d800 0x555b642d0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.244+0000 7f62bdabd640 1 -- 172.21.15.154:0/1011343899 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b6455d400 msgr2=0x555b642d0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.244+0000 7f62bdabd640 1 --2- 172.21.15.154:0/1011343899 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b6455d400 0x555b642d0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.244+0000 7f48e126e640 1 -- 172.21.15.154:0/2963625397 <== osd.5 v2:172.21.15.190:6800/882536885 1 ==== osd_op_reply(1 [pgnls start_epoch 67 out=49b] v65'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b099a6b40 con 0x556b09a40c00 2023-09-23T03:21:35.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.244+0000 7f62bdabd640 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b64298b60 con 0x555b6455d000 2023-09-23T03:21:35.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.244+0000 7ffaa0a525c0 1 --2- 172.21.15.154:0/1953746562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d6aeb63800 0x55d6ae8dc100 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x55d6aeb805a0 tx=0x55d6aeb48060 comp rx=0 tx=0).stop 2023-09-23T03:21:35.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.244+0000 7fb99b2c4640 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5602ef4ce400 0x5602ef0e5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.244+0000 7efdc94ed640 1 -- 172.21.15.154:0/1781234207 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(12 [pgnls start_epoch 67 out=49b] v65'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564662cc6fc0 con 0x564662178800 2023-09-23T03:21:35.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.244+0000 7efdc54e5640 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:13 9.1 9:80000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x564662186380 con 0x564662178800 2023-09-23T03:21:35.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.244+0000 7f48dca65640 1 --2- 172.21.15.154:0/2963625397 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x556b08e7e000 0x556b08e84000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.244+0000 7f48dca65640 1 -- 172.21.15.154:0/2963625397 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:2 9.32 9:4c000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x556b08e88000 con 0x556b08e7e000 2023-09-23T03:21:35.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.244+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/1953746562 shutdown_connections 2023-09-23T03:21:35.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.244+0000 7f9a4ad65640 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55603392c400 0x556033932580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.244+0000 7ffaa0a525c0 1 --2- 172.21.15.154:0/1953746562 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d6aeb63800 0x55d6ae8dc100 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.244+0000 7fe181b915c0 1 -- 172.21.15.154:0/2416081981 shutdown_connections 2023-09-23T03:21:35.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.244+0000 7ffaa0a525c0 1 --2- 172.21.15.154:0/1953746562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d6aeb63400 0x55d6ae8dbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.244+0000 7ffaa0a525c0 1 --2- 172.21.15.154:0/1953746562 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d6aeb63000 0x55d6ae8db600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.244+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/1953746562 >> 172.21.15.154:0/1953746562 conn(0x55d6aeb62c00 msgr2=0x55d6af5eb000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:35.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.245+0000 7fe181b915c0 1 -- 172.21.15.154:0/2416081981 wait complete. 2023-09-23T03:21:35.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.245+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/1953746562 shutdown_connections 2023-09-23T03:21:35.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.245+0000 7efdc94ed640 1 -- 172.21.15.154:0/1781234207 <== osd.3 v2:172.21.15.154:6808/1133688468 2 ==== osd_op_reply(13 [pgnls start_epoch 67 out=49b] v65'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564662cc6fc0 con 0x564662178800 2023-09-23T03:21:35.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.245+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/1953746562 wait complete. 2023-09-23T03:21:35.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.245+0000 7efdc4ce4640 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x564662196400 0x56466219c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.245+0000 7f62bdabd640 1 --2- 172.21.15.154:0/1011343899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b6455d000 0x555b642d1180 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x555b6455f6e0 tx=0x555b645481e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.245+0000 7efdc4ce4640 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:14 9.21 9:84000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x564662186380 con 0x564662196400 2023-09-23T03:21:35.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.245+0000 7fb99b2c4640 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5602ef4ce400 0x5602ef0e5b80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.245+0000 7f48e2270640 1 --2- 172.21.15.154:0/2963625397 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x556b08e7e000 0x556b08e84000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.245+0000 7efdca4ef640 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x564662196400 0x56466219c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.246+0000 7fe181b915c0 1 Processor -- start 2023-09-23T03:21:35.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.246+0000 7f9a4ad65640 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55603392c400 0x556033932580 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.246+0000 7fb99b2c4640 1 -- 172.21.15.154:0/2342064636 <== osd.0 v2:172.21.15.154:6800/2920938138 1 ==== osd_op_reply(4 [pgnls start_epoch 67 out=81b] v65'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5602efe4ed80 con 0x5602ef4ce400 2023-09-23T03:21:35.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.246+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:5 9.20 9:05552577:::1000000020b.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef0d5c00 con 0x5602ef4ce400 2023-09-23T03:21:35.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.246+0000 7f62b72b0640 1 -- 172.21.15.154:0/1011343899 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555b642acdc0 con 0x555b6455d000 2023-09-23T03:21:35.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.246+0000 7f62b72b0640 1 -- 172.21.15.154:0/1011343899 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555b64557380 con 0x555b6455d000 2023-09-23T03:21:35.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.246+0000 7f62b72b0640 1 -- 172.21.15.154:0/1011343899 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555b642ad600 con 0x555b6455d000 2023-09-23T03:21:35.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.246+0000 7fb99b2c4640 1 -- 172.21.15.154:0/2342064636 <== osd.0 v2:172.21.15.154:6800/2920938138 2 ==== osd_op_reply(5 1000000020b.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5602efe4ed80 con 0x5602ef4ce400 2023-09-23T03:21:35.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.246+0000 7fb99ca525c0 10 datascan.operator(): handling object 1099511628299.0 2023-09-23T03:21:35.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.246+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:6 9.20 9:05552577:::1000000020b.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef0d5880 con 0x5602ef4ce400 2023-09-23T03:21:35.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.246+0000 7efdca4ef640 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x564662196400 0x56466219c580 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.246+0000 7fe181b915c0 1 -- start start 2023-09-23T03:21:35.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.246+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555b64298340 con 0x555b6455d000 2023-09-23T03:21:35.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.246+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555b64299520 con 0x555b6455d000 2023-09-23T03:21:35.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.246+0000 7f62bf24b5c0 4 datascan.init: resolving metadata pool 8 2023-09-23T03:21:35.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.246+0000 7ffaa0a525c0 1 Processor -- start 2023-09-23T03:21:35.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.247+0000 7f48e2270640 1 --2- 172.21.15.154:0/2963625397 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x556b08e7e000 0x556b08e84000 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.247+0000 7ffaa0a525c0 1 -- start start 2023-09-23T03:21:35.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.247+0000 7efdca4ef640 1 -- 172.21.15.154:0/1781234207 <== osd.2 v2:172.21.15.154:6822/2884407243 1 ==== osd_op_reply(14 [pgnls start_epoch 67 out=49b] v65'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564662cc6d80 con 0x564662196400 2023-09-23T03:21:35.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.247+0000 7efdc54e5640 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:15 9.11 9:88000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x564662186700 con 0x564662178400 2023-09-23T03:21:35.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.247+0000 7f9a4ad65640 1 -- 172.21.15.154:0/3974369953 <== osd.5 v2:172.21.15.190:6800/882536885 1 ==== osd_op_reply(3 [pgnls start_epoch 67 out=49b] v65'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556033798fc0 con 0x55603392c400 2023-09-23T03:21:35.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.247+0000 7f48e2270640 1 -- 172.21.15.154:0/2963625397 <== osd.4 v2:172.21.15.190:6808/717302641 1 ==== osd_op_reply(2 [pgnls start_epoch 67 out=49b] v65'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b099a6d80 con 0x556b08e7e000 2023-09-23T03:21:35.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.247+0000 7f48dd266640 1 -- 172.21.15.154:0/2963625397 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:3 9.a 9:50000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x556b08e88380 con 0x556b09a40c00 2023-09-23T03:21:35.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.247+0000 7f9a45d5b640 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55603392c800 0x556033932b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.247+0000 7f9a45d5b640 1 -- 172.21.15.154:0/3974369953 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:4 9.c 9:30000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x556033936700 con 0x55603392c800 2023-09-23T03:21:35.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.248+0000 7efdca4ef640 1 -- 172.21.15.154:0/1781234207 <== osd.7 v2:172.21.15.190:6824/3460136272 3 ==== osd_op_reply(15 [pgnls start_epoch 67 out=49b] v65'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564662cc6d80 con 0x564662178400 2023-09-23T03:21:35.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.248+0000 7efdc4ce4640 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x564662196800 0x56466219cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.248+0000 7efdc4ce4640 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:16 9.31 9:8c000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x564661e71880 con 0x564662196800 2023-09-23T03:21:35.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.248+0000 7fe181b915c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563379aa2c00 0x563379b0e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.248+0000 7fe181b915c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563379d9fc00 0x563379b0e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.248+0000 7f9a4a564640 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55603392c800 0x556033932b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.248+0000 7f48e126e640 1 -- 172.21.15.154:0/2963625397 <== osd.5 v2:172.21.15.190:6800/882536885 2 ==== osd_op_reply(3 [pgnls start_epoch 67 out=49b] v65'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b099a6b40 con 0x556b09a40c00 2023-09-23T03:21:35.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.248+0000 7fe181b915c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563379d9f800 0x563379b0db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.248+0000 7f48dca65640 1 --2- 172.21.15.154:0/2963625397 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x556b08e7e400 0x556b08e84580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.248+0000 7f48dca65640 1 -- 172.21.15.154:0/2963625397 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:4 9.2a 9:54000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x556b08e88380 con 0x556b08e7e400 2023-09-23T03:21:35.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.248+0000 7efdc9cee640 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x564662196800 0x56466219cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.249+0000 7f48e1a6f640 1 --2- 172.21.15.154:0/2963625397 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x556b08e7e400 0x556b08e84580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.249+0000 7ffaa0a525c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d6aeb63000 0x55d6ae8dd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.249+0000 7ffaa0a525c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d6aeb63400 0x55d6ae8dcc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.249+0000 7ffaa0a525c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d6aeb63800 0x55d6ae8dc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.249+0000 7ffaa0a525c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ae93ec80 con 0x55d6aeb63400 2023-09-23T03:21:35.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.249+0000 7ffaa0a525c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ae8ba140 con 0x55d6aeb63000 2023-09-23T03:21:35.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.249+0000 7ffaa0a525c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d6ae8bab40 con 0x55d6aeb63800 2023-09-23T03:21:35.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.249+0000 7fe180339640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563379d9fc00 0x563379b0e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.249+0000 7fe180339640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563379d9fc00 0x563379b0e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60630/0 (socket says 172.21.15.154:60630) 2023-09-23T03:21:35.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.249+0000 7fe180339640 1 -- 172.21.15.154:0/3084437610 learned_addr learned my addr 172.21.15.154:0/3084437610 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.249+0000 7fe181b915c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x563379b70dc0 con 0x563379d9fc00 2023-09-23T03:21:35.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.249+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563379b70b40 con 0x563379d9f800 2023-09-23T03:21:35.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.249+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x563379b70a00 con 0x563379aa2c00 2023-09-23T03:21:35.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.249+0000 7fe17fb38640 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563379d9f800 0x563379b0db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.249+0000 7fb99b2c4640 1 -- 172.21.15.154:0/2342064636 <== osd.0 v2:172.21.15.154:6800/2920938138 3 ==== osd_op_reply(6 1000000020b.00000000 [call] v67'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5602efe4ed80 con 0x5602ef4ce400 2023-09-23T03:21:35.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.249+0000 7fb99ca525c0 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5602ef4ce800 0x5602ef0e5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.249+0000 7f48e1a6f640 1 --2- 172.21.15.154:0/2963625397 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x556b08e7e400 0x556b08e84580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.250+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:7 9.10 9:08000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef0d5c00 con 0x5602ef4ce800 2023-09-23T03:21:35.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.249+0000 7ffa9eac3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d6aeb63400 0x55d6ae8dcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.249+0000 7ffa9eac3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d6aeb63400 0x55d6ae8dcc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60632/0 (socket says 172.21.15.154:60632) 2023-09-23T03:21:35.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.249+0000 7ffa9eac3640 1 -- 172.21.15.154:0/334556304 learned_addr learned my addr 172.21.15.154:0/334556304 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.250+0000 7ffa9eac3640 1 -- 172.21.15.154:0/334556304 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d6aeb63800 msgr2=0x55d6ae8dc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.250+0000 7ffa9eac3640 1 --2- 172.21.15.154:0/334556304 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d6aeb63800 0x55d6ae8dc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.250+0000 7ffa9eac3640 1 -- 172.21.15.154:0/334556304 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d6aeb63000 msgr2=0x55d6ae8dd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.250+0000 7ffa9eac3640 1 --2- 172.21.15.154:0/334556304 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d6aeb63000 0x55d6ae8dd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.250+0000 7ffa9eac3640 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6ae8a4b60 con 0x55d6aeb63400 2023-09-23T03:21:35.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.250+0000 7f9a4a564640 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55603392c800 0x556033932b00 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.250+0000 7fe17fb38640 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563379aa2c00 0x563379b0e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.250+0000 7efdc9cee640 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x564662196800 0x56466219cb00 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.250+0000 7f48e1a6f640 1 -- 172.21.15.154:0/2963625397 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(4 [pgnls start_epoch 67 out=101b] v65'11 uv11 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x556b099a6fc0 con 0x556b08e7e400 2023-09-23T03:21:35.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.250+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:5 9.2a 9:55a6cd83:::1000000020a.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x556b08b57c00 con 0x556b08e7e400 2023-09-23T03:21:35.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.250+0000 7ffa9eac3640 1 --2- 172.21.15.154:0/334556304 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d6aeb63400 0x55d6ae8dcc00 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x55d6aeb81290 tx=0x55d6aeb48240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.250+0000 7fb99aac3640 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5602ef4ce800 0x5602ef0e5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.250+0000 7f9a4a564640 1 -- 172.21.15.154:0/3974369953 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(4 [pgnls start_epoch 67 out=81b] v65'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556033798b40 con 0x55603392c800 2023-09-23T03:21:35.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.251+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:5 9.c 9:30604137:::100000001fc.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x556033519c00 con 0x55603392c800 2023-09-23T03:21:35.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.251+0000 7f48e1a6f640 1 -- 172.21.15.154:0/2963625397 <== osd.3 v2:172.21.15.154:6808/1133688468 2 ==== osd_op_reply(5 1000000020a.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556b099a6fc0 con 0x556b08e7e400 2023-09-23T03:21:35.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.251+0000 7f48e3e0c5c0 10 datascan.operator(): handling object 1099511628298.0 2023-09-23T03:21:35.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.250+0000 7ffa98ab7640 1 -- 172.21.15.154:0/334556304 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d6ae8b9080 con 0x55d6aeb63400 2023-09-23T03:21:35.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.251+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:6 9.2a 9:55a6cd83:::1000000020a.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x556b08e88700 con 0x556b08e7e400 2023-09-23T03:21:35.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.251+0000 7fe17fb38640 1 -- 172.21.15.154:0/3084437610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563379aa2c00 msgr2=0x563379b0e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.251+0000 7fe17fb38640 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563379aa2c00 0x563379b0e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.251+0000 7fe17fb38640 1 -- 172.21.15.154:0/3084437610 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563379d9fc00 msgr2=0x563379b0e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.251+0000 7fe17fb38640 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563379d9fc00 0x563379b0e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.251+0000 7efdc9cee640 1 -- 172.21.15.154:0/1781234207 <== osd.4 v2:172.21.15.190:6808/717302641 1 ==== osd_op_reply(16 [pgnls start_epoch 67 out=49b] v65'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564662cc6b40 con 0x564662196800 2023-09-23T03:21:35.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.251+0000 7fe17fb38640 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563379ad6b60 con 0x563379d9f800 2023-09-23T03:21:35.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.251+0000 7ffa98ab7640 1 -- 172.21.15.154:0/334556304 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d6ae9ea300 con 0x55d6aeb63400 2023-09-23T03:21:35.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.251+0000 7fe180339640 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563379d9fc00 0x563379b0e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:21:35.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.251+0000 7ffa98ab7640 1 -- 172.21.15.154:0/334556304 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d6ae8b8f20 con 0x55d6aeb63400 2023-09-23T03:21:35.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.252+0000 7fb99aac3640 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5602ef4ce800 0x5602ef0e5600 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.251+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d6ae8a4340 con 0x55d6aeb63400 2023-09-23T03:21:35.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.251+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d6ae8a5520 con 0x55d6aeb63400 2023-09-23T03:21:35.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.252+0000 7efdc54e5640 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:17 9.9 9:90000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x564662186a80 con 0x564662196000 2023-09-23T03:21:35.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.252+0000 7f9a4a564640 1 -- 172.21.15.154:0/3974369953 <== osd.3 v2:172.21.15.154:6808/1133688468 2 ==== osd_op_reply(5 100000001fc.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556033798b40 con 0x55603392c800 2023-09-23T03:21:35.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.252+0000 7f9a4d20c5c0 10 datascan.operator(): handling object 1099511628284.0 2023-09-23T03:21:35.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.252+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:6 9.c 9:30604137:::100000001fc.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x556033936700 con 0x55603392c800 2023-09-23T03:21:35.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.252+0000 7fe17fb38640 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563379d9f800 0x563379b0db80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x563379d973e0 tx=0x56337a83d620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.252+0000 7f62b72b0640 1 -- 172.21.15.154:0/1011343899 <== mon.2 v2:172.21.15.190:3301/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x555b64296840 con 0x555b6455d000 2023-09-23T03:21:35.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.251+0000 7ffaa0a525c0 4 datascan.init: resolving metadata pool 8 2023-09-23T03:21:35.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.252+0000 7efdc9cee640 1 -- 172.21.15.154:0/1781234207 <== osd.0 v2:172.21.15.154:6800/2920938138 4 ==== osd_op_reply(17 [pgnls start_epoch 67 out=49b] v65'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564662cc6b40 con 0x564662196000 2023-09-23T03:21:35.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.252+0000 7fb99aac3640 1 -- 172.21.15.154:0/2342064636 <== osd.4 v2:172.21.15.190:6808/717302641 1 ==== osd_op_reply(7 [pgnls start_epoch 67 out=81b] v65'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5602efe4efc0 con 0x5602ef4ce800 2023-09-23T03:21:35.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.252+0000 7efdcc0145c0 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x564662178c00 0x56466219d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.253+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:18 10.16 10:6db6db6d::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x564661e71c00 con 0x564662178c00 2023-09-23T03:21:35.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.253+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:8 9.10 9:08b99fea:::100000001fd.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef4e4000 con 0x5602ef4ce800 2023-09-23T03:21:35.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.253+0000 7efdc94ed640 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x564662178c00 0x56466219d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.253+0000 7f48e1a6f640 1 -- 172.21.15.154:0/2963625397 <== osd.3 v2:172.21.15.154:6808/1133688468 3 ==== osd_op_reply(6 1000000020a.00000000 [call] v67'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556b099a6fc0 con 0x556b08e7e400 2023-09-23T03:21:35.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.253+0000 7f62b72b0640 1 --2- 172.21.15.154:0/1011343899 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x555b645ba800 0x555b642d1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.253+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:7 9.2a 9:57fe7535::::head [pgnls start_epoch 67 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x556b08e88a80 con 0x556b08e7e400 2023-09-23T03:21:35.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.253+0000 7f62b72b0640 1 -- 172.21.15.154:0/1011343899 <== mon.2 v2:172.21.15.190:3301/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x555b64324a80 con 0x555b6455d000 2023-09-23T03:21:35.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.253+0000 7f62bf24b5c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-09-23T03:21:35.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.253+0000 7fe17a32d640 1 -- 172.21.15.154:0/3084437610 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563379aeb1e0 con 0x563379d9f800 2023-09-23T03:21:35.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.253+0000 7f62bf24b5c0 4 datascan.main: data pool 'cephfs_data' has ID 9 2023-09-23T03:21:35.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.254+0000 7fe17a32d640 1 -- 172.21.15.154:0/3084437610 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563379d92c00 con 0x563379d9f800 2023-09-23T03:21:35.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.253+0000 7f62bf24b5c0 4 datascan.main: opening data pool 'cephfs_data' 2023-09-23T03:21:35.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.253+0000 7f62bf24b5c0 4 datascan.main: data pool 'cephfs_data_extra' has ID 10 2023-09-23T03:21:35.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.253+0000 7f62bf24b5c0 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-09-23T03:21:35.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.253+0000 7f62bf24b5c0 1 --2- 172.21.15.154:0/1011343899 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x555b645bac00 0x555b642d0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.254+0000 7f62bd2bc640 1 --2- 172.21.15.154:0/1011343899 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x555b645ba800 0x555b642d1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.254+0000 7fb99aac3640 1 -- 172.21.15.154:0/2342064636 <== osd.4 v2:172.21.15.190:6808/717302641 2 ==== osd_op_reply(8 100000001fd.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5602efe4efc0 con 0x5602ef4ce800 2023-09-23T03:21:35.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.254+0000 7fb99ca525c0 10 datascan.operator(): handling object 1099511628285.0 2023-09-23T03:21:35.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.254+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:1 9.2d 9:b6db6db6::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b642bf880 con 0x555b645bac00 2023-09-23T03:21:35.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.254+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:9 9.10 9:08b99fea:::100000001fd.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef4e4380 con 0x5602ef4ce800 2023-09-23T03:21:35.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.254+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563379ad6340 con 0x563379d9f800 2023-09-23T03:21:35.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.254+0000 7f62bcabb640 1 --2- 172.21.15.154:0/1011343899 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x555b645bac00 0x555b642d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.254+0000 7efdc94ed640 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x564662178c00 0x56466219d600 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.254+0000 7f48e1a6f640 1 -- 172.21.15.154:0/2963625397 <== osd.3 v2:172.21.15.154:6808/1133688468 4 ==== osd_op_reply(7 [pgnls start_epoch 67 out=81b] v67'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556b099a6fc0 con 0x556b08e7e400 2023-09-23T03:21:35.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.254+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:8 9.2a 9:57fe7535:::100000001f6.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x556b08e88e00 con 0x556b08e7e400 2023-09-23T03:21:35.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.254+0000 7ffa98ab7640 1 -- 172.21.15.154:0/334556304 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x55d6ae8a3080 con 0x55d6aeb63400 2023-09-23T03:21:35.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.254+0000 7ffa98ab7640 1 --2- 172.21.15.154:0/334556304 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55d6aeb63c00 0x55d6ae8dd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.255+0000 7ffa98ab7640 1 -- 172.21.15.154:0/334556304 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x55d6ae930a80 con 0x55d6aeb63400 2023-09-23T03:21:35.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.255+0000 7ffaa0a525c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-09-23T03:21:35.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.255+0000 7f48e1a6f640 1 -- 172.21.15.154:0/2963625397 <== osd.3 v2:172.21.15.154:6808/1133688468 5 ==== osd_op_reply(8 100000001f6.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556b099a6fc0 con 0x556b08e7e400 2023-09-23T03:21:35.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.255+0000 7f48e3e0c5c0 10 datascan.operator(): handling object 1099511628278.0 2023-09-23T03:21:35.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.255+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:9 9.2a 9:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x556b08e89180 con 0x556b08e7e400 2023-09-23T03:21:35.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.255+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563379ad6000 con 0x563379d9f800 2023-09-23T03:21:35.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.255+0000 7ffaa0a525c0 4 datascan.main: data pool 'cephfs_data' has ID 9 2023-09-23T03:21:35.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.255+0000 7fe181b915c0 4 datascan.init: resolving metadata pool 8 2023-09-23T03:21:35.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.255+0000 7ffaa0a525c0 4 datascan.main: opening data pool 'cephfs_data' 2023-09-23T03:21:35.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.255+0000 7fe17a32d640 1 -- 172.21.15.154:0/3084437610 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563379aeadc0 con 0x563379d9f800 2023-09-23T03:21:35.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.255+0000 7ffaa0a525c0 4 datascan.main: data pool 'cephfs_data_extra' has ID 10 2023-09-23T03:21:35.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.255+0000 7efdc94ed640 1 -- 172.21.15.154:0/1781234207 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== osd_op_reply(18 [pgnls start_epoch 67 out=101b] v65'5 uv5 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564662cc6fc0 con 0x564662178c00 2023-09-23T03:21:35.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.256+0000 7f9a4a564640 1 -- 172.21.15.154:0/3974369953 <== osd.3 v2:172.21.15.154:6808/1133688468 3 ==== osd_op_reply(6 100000001fc.00000000 [call] v67'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556033798b40 con 0x55603392c800 2023-09-23T03:21:35.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.256+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:19 10.16 10:6ee5e2c4:::100000001f9.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x564662187880 con 0x564662178c00 2023-09-23T03:21:35.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.255+0000 7ffaa0a525c0 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-09-23T03:21:35.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.255+0000 7ffaa0a525c0 1 --2- 172.21.15.154:0/334556304 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55d6aecee000 0x55d6ae8dc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.255+0000 7ffa9f2c4640 1 --2- 172.21.15.154:0/334556304 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55d6aeb63c00 0x55d6ae8dd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.255+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:1 9.1b 9:db6db6db::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6ae8cb880 con 0x55d6aecee000 2023-09-23T03:21:35.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.255+0000 7ffa9e2c2640 1 --2- 172.21.15.154:0/334556304 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55d6aecee000 0x55d6ae8dc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.256+0000 7ffa9e2c2640 1 --2- 172.21.15.154:0/334556304 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55d6aecee000 0x55d6ae8dc100 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.256+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:7 9.2c 9:34000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x556033936e00 con 0x55603392c800 2023-09-23T03:21:35.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.256+0000 7ffa9e2c2640 1 -- 172.21.15.154:0/334556304 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(1 [pgnls start_epoch 67 out=49b] v65'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d6af710b40 con 0x55d6aecee000 2023-09-23T03:21:35.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.256+0000 7f48e1a6f640 1 -- 172.21.15.154:0/2963625397 <== osd.3 v2:172.21.15.154:6808/1133688468 6 ==== osd_op_reply(9 100000001f6.00000000 [call] v67'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556b099a6fc0 con 0x556b08e7e400 2023-09-23T03:21:35.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.256+0000 7f62bd2bc640 1 --2- 172.21.15.154:0/1011343899 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x555b645ba800 0x555b642d1700 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x555b642f3b00 tx=0x555b64564d20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.257+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:10 9.1a 9:58000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x556b08e88380 con 0x556b08e7e400 2023-09-23T03:21:35.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.257+0000 7ffa99ab9640 1 --2- 172.21.15.154:0/334556304 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55d6aecf6000 0x55d6aecf8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.257+0000 7ffa99ab9640 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:2 9.3b 9:dc000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6aecf4000 con 0x55d6aecf6000 2023-09-23T03:21:35.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.257+0000 7efdc94ed640 1 -- 172.21.15.154:0/1781234207 <== osd.6 v2:172.21.15.190:6816/1964294001 2 ==== osd_op_reply(19 100000001f9.00000000 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564662cc6fc0 con 0x564662178c00 2023-09-23T03:21:35.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.257+0000 7efdcc0145c0 10 datascan.operator(): handling object 1099511628281.0 2023-09-23T03:21:35.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.257+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:20 9.36 9:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x564662187c00 con 0x564662178000 2023-09-23T03:21:35.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.257+0000 7f62bcabb640 1 --2- 172.21.15.154:0/1011343899 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x555b645bac00 0x555b642d0100 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.257+0000 7f48e1a6f640 1 -- 172.21.15.154:0/2963625397 <== osd.3 v2:172.21.15.154:6808/1133688468 7 ==== osd_op_reply(10 [pgnls start_epoch 67 out=49b] v65'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b099a6fc0 con 0x556b08e7e400 2023-09-23T03:21:35.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.257+0000 7f9a4a564640 1 -- 172.21.15.154:0/3974369953 <== osd.3 v2:172.21.15.154:6808/1133688468 4 ==== osd_op_reply(7 [pgnls start_epoch 67 out=81b] v65'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556033798b40 con 0x55603392c800 2023-09-23T03:21:35.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.257+0000 7f48dd266640 1 --2- 172.21.15.154:0/2963625397 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x556b08e7ec00 0x556b08e85080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.257+0000 7f48dd266640 1 -- 172.21.15.154:0/2963625397 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:11 9.3a 9:5c000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x556b08e88a80 con 0x556b08e7ec00 2023-09-23T03:21:35.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.257+0000 7ffa9f2c4640 1 --2- 172.21.15.154:0/334556304 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55d6aecf6000 0x55d6aecf8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.257+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:8 9.2c 9:37389ade:::10000000209.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x556033937180 con 0x55603392c800 2023-09-23T03:21:35.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.257+0000 7f62bcabb640 1 -- 172.21.15.154:0/1011343899 <== osd.2 v2:172.21.15.154:6822/2884407243 1 ==== osd_op_reply(1 [pgnls start_epoch 67 out=49b] v65'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b6453cb40 con 0x555b645bac00 2023-09-23T03:21:35.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.257+0000 7f62b82b2640 1 --2- 172.21.15.154:0/1011343899 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x555b646ce000 0x555b646d4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.257+0000 7f62b82b2640 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:2 9.1d 9:b8000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b646d8000 con 0x555b646ce000 2023-09-23T03:21:35.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.258+0000 7f48e126e640 1 --2- 172.21.15.154:0/2963625397 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x556b08e7ec00 0x556b08e85080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.258+0000 7f62bdabd640 1 --2- 172.21.15.154:0/1011343899 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x555b646ce000 0x555b646d4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.258+0000 7f9a4a564640 1 -- 172.21.15.154:0/3974369953 <== osd.3 v2:172.21.15.154:6808/1133688468 5 ==== osd_op_reply(8 10000000209.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556033798b40 con 0x55603392c800 2023-09-23T03:21:35.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.258+0000 7f9a4d20c5c0 10 datascan.operator(): handling object 1099511628297.0 2023-09-23T03:21:35.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.258+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:9 9.2c 9:37389ade:::10000000209.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x556033519880 con 0x55603392c800 2023-09-23T03:21:35.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.258+0000 7ffa9f2c4640 1 --2- 172.21.15.154:0/334556304 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55d6aeb63c00 0x55d6ae8dd700 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x55d6ae962780 tx=0x55d6aeb99f20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.258+0000 7fb99aac3640 1 -- 172.21.15.154:0/2342064636 <== osd.4 v2:172.21.15.190:6808/717302641 3 ==== osd_op_reply(9 100000001fd.00000000 [call] v67'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5602efe4efc0 con 0x5602ef4ce800 2023-09-23T03:21:35.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.259+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:10 9.30 9:0c000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef4e4700 con 0x5602ef4ce400 2023-09-23T03:21:35.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.259+0000 7f62bdabd640 1 --2- 172.21.15.154:0/1011343899 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x555b646ce000 0x555b646d4000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.258+0000 7ffa9f2c4640 1 --2- 172.21.15.154:0/334556304 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55d6aecf6000 0x55d6aecf8000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.259+0000 7f48e126e640 1 --2- 172.21.15.154:0/2963625397 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x556b08e7ec00 0x556b08e85080 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.259+0000 7ffa9f2c4640 1 -- 172.21.15.154:0/334556304 <== osd.2 v2:172.21.15.154:6822/2884407243 1 ==== osd_op_reply(2 [pgnls start_epoch 67 out=81b] v65'5 uv5 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d6af710d80 con 0x55d6aecf6000 2023-09-23T03:21:35.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.259+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:3 9.3b 9:deccbd37:::100000001fe.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6ae8cbc00 con 0x55d6aecf6000 2023-09-23T03:21:35.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.260+0000 7f62bdabd640 1 -- 172.21.15.154:0/1011343899 <== osd.0 v2:172.21.15.154:6800/2920938138 1 ==== osd_op_reply(2 [pgnls start_epoch 67 out=49b] v65'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b6453cd80 con 0x555b646ce000 2023-09-23T03:21:35.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.260+0000 7f62b8ab3640 1 --2- 172.21.15.154:0/1011343899 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x555b646ce400 0x555b646d4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.260+0000 7ffa9f2c4640 1 -- 172.21.15.154:0/334556304 <== osd.2 v2:172.21.15.154:6822/2884407243 2 ==== osd_op_reply(3 100000001fe.00000000 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d6af710d80 con 0x55d6aecf6000 2023-09-23T03:21:35.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.260+0000 7fb99b2c4640 1 -- 172.21.15.154:0/2342064636 <== osd.0 v2:172.21.15.154:6800/2920938138 4 ==== osd_op_reply(10 [pgnls start_epoch 67 out=49b] v65'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5602efe4ed80 con 0x5602ef4ce400 2023-09-23T03:21:35.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.260+0000 7fb9962ba640 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5602ef4ea000 0x5602ef4f0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.260+0000 7fb9962ba640 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:11 9.8 9:10000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef0d5c00 con 0x5602ef4ea000 2023-09-23T03:21:35.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.260+0000 7ffaa0a525c0 10 datascan.operator(): handling object 1099511628286.0 2023-09-23T03:21:35.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.260+0000 7f62b8ab3640 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:3 9.3d 9:bc000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b646d8380 con 0x555b646ce400 2023-09-23T03:21:35.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.260+0000 7efdc9cee640 1 -- 172.21.15.154:0/1781234207 <== osd.1 v2:172.21.15.154:6816/1267710005 7 ==== osd_op_reply(20 100000001f9.00000000 [call,setxattr (8)] v67'20 uv20 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x564662cc6b40 con 0x564662178000 2023-09-23T03:21:35.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.260+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:21 10.16 10:6fe38dc2::::head [pgnls start_epoch 67 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x564662186e00 con 0x564662178c00 2023-09-23T03:21:35.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.260+0000 7fe17a32d640 1 -- 172.21.15.154:0/3084437610 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x563379ad4b00 con 0x563379d9f800 2023-09-23T03:21:35.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.260+0000 7f48e126e640 1 -- 172.21.15.154:0/2963625397 <== osd.0 v2:172.21.15.154:6800/2920938138 1 ==== osd_op_reply(11 [pgnls start_epoch 67 out=49b] v65'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b099a6b40 con 0x556b08e7ec00 2023-09-23T03:21:35.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.260+0000 7f48dca65640 1 -- 172.21.15.154:0/2963625397 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:12 9.6 9:60000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x556b08b57880 con 0x556b09a40c00 2023-09-23T03:21:35.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.261+0000 7f62bd2bc640 1 --2- 172.21.15.154:0/1011343899 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x555b646ce400 0x555b646d4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.260+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:4 9.3b 9:deccbd37:::100000001fe.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6ae8cbc00 con 0x55d6aecf6000 2023-09-23T03:21:35.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.261+0000 7fb99a2c2640 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5602ef4ea000 0x5602ef4f0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.261+0000 7f9a4a564640 1 -- 172.21.15.154:0/3974369953 <== osd.3 v2:172.21.15.154:6808/1133688468 6 ==== osd_op_reply(9 10000000209.00000000 [call] v67'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556033798b40 con 0x55603392c800 2023-09-23T03:21:35.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.261+0000 7f9a4d20c5c0 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55603382b000 0x556033529b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.261+0000 7f48e126e640 1 -- 172.21.15.154:0/2963625397 <== osd.5 v2:172.21.15.190:6800/882536885 3 ==== osd_op_reply(12 [pgnls start_epoch 67 out=49b] v65'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b099a6b40 con 0x556b09a40c00 2023-09-23T03:21:35.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.261+0000 7f48dd266640 1 -- 172.21.15.154:0/2963625397 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:13 9.26 9:64000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x556b08e88e00 con 0x556b09a40c00 2023-09-23T03:21:35.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.261+0000 7fe17a32d640 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x563379dff400 0x563379b0f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.261+0000 7efdc94ed640 1 -- 172.21.15.154:0/1781234207 <== osd.6 v2:172.21.15.190:6816/1964294001 3 ==== osd_op_reply(21 [pgnls start_epoch 67 out=81b] v65'5 uv5 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564662cc6fc0 con 0x564662178c00 2023-09-23T03:21:35.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.261+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:22 10.16 10:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x564662187180 con 0x564662178c00 2023-09-23T03:21:35.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.261+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:10 9.1c 9:38000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x556033936700 con 0x55603382b000 2023-09-23T03:21:35.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.262+0000 7f9a4b566640 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55603382b000 0x556033529b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.262+0000 7fe17a32d640 1 -- 172.21.15.154:0/3084437610 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x563379b62a80 con 0x563379d9f800 2023-09-23T03:21:35.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.262+0000 7fb99a2c2640 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5602ef4ea000 0x5602ef4f0000 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.262+0000 7fe180339640 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x563379dff400 0x563379b0f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.262+0000 7fe181b915c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-09-23T03:21:35.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.262+0000 7f62bd2bc640 1 --2- 172.21.15.154:0/1011343899 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x555b646ce400 0x555b646d4580 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.262+0000 7fb99a2c2640 1 -- 172.21.15.154:0/2342064636 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(11 [pgnls start_epoch 67 out=49b] v65'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5602efe4eb40 con 0x5602ef4ea000 2023-09-23T03:21:35.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.262+0000 7fe181b915c0 4 datascan.main: data pool 'cephfs_data' has ID 9 2023-09-23T03:21:35.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.262+0000 7fe181b915c0 4 datascan.main: opening data pool 'cephfs_data' 2023-09-23T03:21:35.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.262+0000 7fe181b915c0 4 datascan.main: data pool 'cephfs_data_extra' has ID 10 2023-09-23T03:21:35.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.262+0000 7fe181b915c0 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-09-23T03:21:35.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.262+0000 7fe181b915c0 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x563379d9f400 0x563379b0ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.262+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:1 9.9 9:92492492::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379afd880 con 0x563379d9f400 2023-09-23T03:21:35.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.262+0000 7f48e126e640 1 -- 172.21.15.154:0/2963625397 <== osd.5 v2:172.21.15.190:6800/882536885 4 ==== osd_op_reply(13 [pgnls start_epoch 67 out=81b] v65'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556b099a6b40 con 0x556b09a40c00 2023-09-23T03:21:35.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.262+0000 7efdc94ed640 1 -- 172.21.15.154:0/1781234207 <== osd.6 v2:172.21.15.190:6816/1964294001 4 ==== osd_op_reply(22 100000001f7.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564662cc6fc0 con 0x564662178c00 2023-09-23T03:21:35.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.262+0000 7efdcc0145c0 10 datascan.operator(): handling object 1099511628279.0 2023-09-23T03:21:35.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.262+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:23 9.36 9:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x564662186a80 con 0x564662178000 2023-09-23T03:21:35.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.262+0000 7fb995ab9640 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x5602ef4ea400 0x5602ef4f0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.262+0000 7fe17fb38640 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x563379d9f400 0x563379b0ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.262+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:14 9.26 9:658f2455:::100000001ff.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x556b08e88700 con 0x556b09a40c00 2023-09-23T03:21:35.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.263+0000 7fb995ab9640 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:12 9.28 9:14000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef4e4000 con 0x5602ef4ea400 2023-09-23T03:21:35.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.263+0000 7f9a4b566640 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55603382b000 0x556033529b80 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.263+0000 7f9a4b566640 1 -- 172.21.15.154:0/3974369953 <== osd.0 v2:172.21.15.154:6800/2920938138 1 ==== osd_op_reply(10 [pgnls start_epoch 67 out=49b] v65'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556033798d80 con 0x55603382b000 2023-09-23T03:21:35.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.263+0000 7f9a4655c640 1 -- 172.21.15.154:0/3974369953 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:11 9.3c 9:3c000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x556033937c00 con 0x55603392c400 2023-09-23T03:21:35.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.263+0000 7fb99b2c4640 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x5602ef4ea400 0x5602ef4f0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.263+0000 7fe180339640 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x563379dff400 0x563379b0f700 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x56337a8a6990 tx=0x56337a823f80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.263+0000 7f62bd2bc640 1 -- 172.21.15.154:0/1011343899 <== osd.7 v2:172.21.15.190:6824/3460136272 1 ==== osd_op_reply(3 [pgnls start_epoch 67 out=81b] v65'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555b6453cfc0 con 0x555b646ce400 2023-09-23T03:21:35.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.263+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:4 9.3d 9:bfcd3380:::10000000205.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b642bfc00 con 0x555b646ce400 2023-09-23T03:21:35.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.263+0000 7ffa9f2c4640 1 -- 172.21.15.154:0/334556304 <== osd.2 v2:172.21.15.154:6822/2884407243 3 ==== osd_op_reply(4 100000001fe.00000000 [call] v67'6 uv6 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d6af710d80 con 0x55d6aecf6000 2023-09-23T03:21:35.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.263+0000 7f48e126e640 1 -- 172.21.15.154:0/2963625397 <== osd.5 v2:172.21.15.190:6800/882536885 5 ==== osd_op_reply(14 100000001ff.00000000 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556b099a6b40 con 0x556b09a40c00 2023-09-23T03:21:35.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.263+0000 7f48e3e0c5c0 10 datascan.operator(): handling object 1099511628287.0 2023-09-23T03:21:35.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.263+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:15 9.26 9:658f2455:::100000001ff.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x556b08e89500 con 0x556b09a40c00 2023-09-23T03:21:35.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.264+0000 7ffaa0a525c0 1 --2- 172.21.15.154:0/334556304 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55d6aecee400 0x55d6ae8dbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.264+0000 7fe17fb38640 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x563379d9f400 0x563379b0ec00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.264+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:5 9.7 9:e0000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6aed04000 con 0x55d6aecee400 2023-09-23T03:21:35.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.264+0000 7fe17fb38640 1 -- 172.21.15.154:0/3084437610 <== osd.0 v2:172.21.15.154:6800/2920938138 1 ==== osd_op_reply(1 [pgnls start_epoch 67 out=49b] v65'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563379d7ab40 con 0x563379d9f400 2023-09-23T03:21:35.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.264+0000 7fe17b32f640 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x563379f12000 0x563379f18000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.264+0000 7fe17b32f640 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:2 9.29 9:94000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379f1c000 con 0x563379f12000 2023-09-23T03:21:35.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.265+0000 7ffa9eac3640 1 --2- 172.21.15.154:0/334556304 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55d6aecee400 0x55d6ae8dbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.265+0000 7f9a4ad65640 1 -- 172.21.15.154:0/3974369953 <== osd.5 v2:172.21.15.190:6800/882536885 2 ==== osd_op_reply(11 [pgnls start_epoch 67 out=49b] v65'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556033798fc0 con 0x55603392c400 2023-09-23T03:21:35.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.265+0000 7f62bd2bc640 1 -- 172.21.15.154:0/1011343899 <== osd.7 v2:172.21.15.190:6824/3460136272 2 ==== osd_op_reply(4 10000000205.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555b6453cfc0 con 0x555b646ce400 2023-09-23T03:21:35.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.265+0000 7fb99b2c4640 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x5602ef4ea400 0x5602ef4f0580 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.265+0000 7f9a45d5b640 1 -- 172.21.15.154:0/3974369953 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:12 9.2 9:40000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x556033936a80 con 0x55603392c800 2023-09-23T03:21:35.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.265+0000 7f62bf24b5c0 10 datascan.operator(): handling object 1099511628293.0 2023-09-23T03:21:35.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.265+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:5 9.3d 9:bfcd3380:::10000000205.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b646d8380 con 0x555b646ce400 2023-09-23T03:21:35.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.265+0000 7f9a4a564640 1 -- 172.21.15.154:0/3974369953 <== osd.3 v2:172.21.15.154:6808/1133688468 7 ==== osd_op_reply(12 [pgnls start_epoch 67 out=49b] v65'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556033798b40 con 0x55603392c800 2023-09-23T03:21:35.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.265+0000 7fe180b3a640 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x563379f12000 0x563379f18000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.265+0000 7f9a4655c640 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55603392d000 0x556033933600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.265+0000 7f9a4655c640 1 -- 172.21.15.154:0/3974369953 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:13 9.22 9:44000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x556033936380 con 0x55603392d000 2023-09-23T03:21:35.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.266+0000 7fb99b2c4640 1 -- 172.21.15.154:0/2342064636 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== osd_op_reply(12 [pgnls start_epoch 67 out=101b] v65'8 uv8 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5602efe4ed80 con 0x5602ef4ea400 2023-09-23T03:21:35.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.266+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:13 9.28 9:15246602:::10000000202.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef4e4a80 con 0x5602ef4ea400 2023-09-23T03:21:35.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.266+0000 7ffa9eac3640 1 --2- 172.21.15.154:0/334556304 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55d6aecee400 0x55d6ae8dbb80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.266+0000 7f9a4ad65640 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55603392d000 0x556033933600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.267+0000 7efdc9cee640 1 -- 172.21.15.154:0/1781234207 <== osd.1 v2:172.21.15.154:6816/1267710005 8 ==== osd_op_reply(23 100000001f7.00000000 [call,setxattr (8)] v67'21 uv21 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x564662cc6b40 con 0x564662178000 2023-09-23T03:21:35.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.267+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:24 10.e 10:70000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x564662187c00 con 0x564662196800 2023-09-23T03:21:35.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.267+0000 7fe180b3a640 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x563379f12000 0x563379f18000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.268+0000 7fb99b2c4640 1 -- 172.21.15.154:0/2342064636 <== osd.6 v2:172.21.15.190:6816/1964294001 2 ==== osd_op_reply(13 10000000202.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5602efe4ed80 con 0x5602ef4ea400 2023-09-23T03:21:35.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.268+0000 7fb99ca525c0 10 datascan.operator(): handling object 1099511628290.0 2023-09-23T03:21:35.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.268+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:14 9.28 9:15246602:::10000000202.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef4e4e00 con 0x5602ef4ea400 2023-09-23T03:21:35.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.268+0000 7ffa9eac3640 1 -- 172.21.15.154:0/334556304 <== osd.5 v2:172.21.15.190:6800/882536885 1 ==== osd_op_reply(5 [pgnls start_epoch 67 out=101b] v65'7 uv7 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d6af710fc0 con 0x55d6aecee400 2023-09-23T03:21:35.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.268+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:6 9.7 9:e15a914f:::10000000207.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6aed04380 con 0x55d6aecee400 2023-09-23T03:21:35.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.269+0000 7efdc9cee640 1 -- 172.21.15.154:0/1781234207 <== osd.4 v2:172.21.15.190:6808/717302641 2 ==== osd_op_reply(24 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564662cc6b40 con 0x564662196800 2023-09-23T03:21:35.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.269+0000 7efdc54e5640 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:25 10.1e 10:78000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x564662186000 con 0x564662196000 2023-09-23T03:21:35.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.269+0000 7fe180b3a640 1 -- 172.21.15.154:0/3084437610 <== osd.7 v2:172.21.15.190:6824/3460136272 1 ==== osd_op_reply(2 [pgnls start_epoch 67 out=49b] v65'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563379d7ad80 con 0x563379f12000 2023-09-23T03:21:35.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.269+0000 7fe17bb30640 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x563379f12400 0x563379f18580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.269+0000 7fe17bb30640 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:3 9.19 9:98000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379f1c380 con 0x563379f12400 2023-09-23T03:21:35.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.269+0000 7efdc9cee640 1 -- 172.21.15.154:0/1781234207 <== osd.0 v2:172.21.15.154:6800/2920938138 5 ==== osd_op_reply(25 [pgnls start_epoch 67 out=101b] v65'3 uv3 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564662cc6b40 con 0x564662196000 2023-09-23T03:21:35.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.269+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:26 10.1e 10:789b801b:::1000000020c.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x564661e71880 con 0x564662196000 2023-09-23T03:21:35.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.269+0000 7ffa9eac3640 1 -- 172.21.15.154:0/334556304 <== osd.5 v2:172.21.15.190:6800/882536885 2 ==== osd_op_reply(6 10000000207.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d6af710fc0 con 0x55d6aecee400 2023-09-23T03:21:35.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.269+0000 7ffaa0a525c0 10 datascan.operator(): handling object 1099511628295.0 2023-09-23T03:21:35.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.269+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:7 9.7 9:e15a914f:::10000000207.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6aed04700 con 0x55d6aecee400 2023-09-23T03:21:35.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.269+0000 7fe180339640 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x563379f12400 0x563379f18580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.269+0000 7f9a4ad65640 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55603392d000 0x556033933600 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.269+0000 7efdc9cee640 1 -- 172.21.15.154:0/1781234207 <== osd.0 v2:172.21.15.154:6800/2920938138 6 ==== osd_op_reply(26 1000000020c.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564662cc6b40 con 0x564662196000 2023-09-23T03:21:35.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.269+0000 7efdcc0145c0 10 datascan.operator(): handling object 1099511628300.0 2023-09-23T03:21:35.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.269+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:27 9.1e 9:789b801b:::1000000020c.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x564662187880 con 0x564662196000 2023-09-23T03:21:35.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.270+0000 7fe180339640 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x563379f12400 0x563379f18580 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.270+0000 7f48e126e640 1 -- 172.21.15.154:0/2963625397 <== osd.5 v2:172.21.15.190:6800/882536885 6 ==== osd_op_reply(15 100000001ff.00000000 [call] v67'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556b099a6b40 con 0x556b09a40c00 2023-09-23T03:21:35.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.270+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:16 9.16 9:68000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x556b08e89880 con 0x556b08e7e000 2023-09-23T03:21:35.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.270+0000 7fe180339640 1 -- 172.21.15.154:0/3084437610 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(3 [pgnls start_epoch 67 out=101b] v65'12 uv12 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563379d7afc0 con 0x563379f12400 2023-09-23T03:21:35.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.270+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:4 9.19 9:992a6246:::10000000200.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379afdc00 con 0x563379f12400 2023-09-23T03:21:35.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.271+0000 7f9a4ad65640 1 -- 172.21.15.154:0/3974369953 <== osd.7 v2:172.21.15.190:6824/3460136272 1 ==== osd_op_reply(13 [pgnls start_epoch 67 out=49b] v65'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556033798fc0 con 0x55603392d000 2023-09-23T03:21:35.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.271+0000 7fe180339640 1 -- 172.21.15.154:0/3084437610 <== osd.3 v2:172.21.15.154:6808/1133688468 2 ==== osd_op_reply(4 10000000200.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563379d7afc0 con 0x563379f12400 2023-09-23T03:21:35.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.271+0000 7f9a45d5b640 1 -- 172.21.15.154:0/3974369953 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:14 9.12 9:48000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55603395c000 con 0x55603392c400 2023-09-23T03:21:35.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.271+0000 7fe181b915c0 10 datascan.operator(): handling object 1099511628288.0 2023-09-23T03:21:35.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.271+0000 7f48e2270640 1 -- 172.21.15.154:0/2963625397 <== osd.4 v2:172.21.15.190:6808/717302641 2 ==== osd_op_reply(16 [pgnls start_epoch 67 out=49b] v65'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b099a6d80 con 0x556b08e7e000 2023-09-23T03:21:35.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.271+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:5 9.19 9:992a6246:::10000000200.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379f1c380 con 0x563379f12400 2023-09-23T03:21:35.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.271+0000 7f48dd266640 1 --2- 172.21.15.154:0/2963625397 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x556b08e7f000 0x556b08e85600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.271+0000 7f48dd266640 1 -- 172.21.15.154:0/2963625397 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:17 9.36 9:6c000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x556b08e88000 con 0x556b08e7f000 2023-09-23T03:21:35.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.271+0000 7f48e2270640 1 --2- 172.21.15.154:0/2963625397 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x556b08e7f000 0x556b08e85600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.272+0000 7f48e2270640 1 --2- 172.21.15.154:0/2963625397 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x556b08e7f000 0x556b08e85600 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.272+0000 7f62bd2bc640 1 -- 172.21.15.154:0/1011343899 <== osd.7 v2:172.21.15.190:6824/3460136272 3 ==== osd_op_reply(5 10000000205.00000000 [call] v67'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x555b6453cfc0 con 0x555b646ce400 2023-09-23T03:21:35.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.272+0000 7f48e2270640 1 -- 172.21.15.154:0/2963625397 <== osd.1 v2:172.21.15.154:6816/1267710005 1 ==== osd_op_reply(17 [pgnls start_epoch 67 out=101b] v67'21 uv21 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x556b099a6d80 con 0x556b08e7f000 2023-09-23T03:21:35.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.272+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:6 9.3 9:c0000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b646d8700 con 0x555b646ce400 2023-09-23T03:21:35.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.272+0000 7f48e3e0c5c0 1 --2- 172.21.15.154:0/2963625397 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x556b09a41000 0x556b08b67b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.272+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:18 10.12 10:49249249::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x556b08b57c00 con 0x556b09a41000 2023-09-23T03:21:35.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.273+0000 7f9a4ad65640 1 -- 172.21.15.154:0/3974369953 <== osd.5 v2:172.21.15.190:6800/882536885 3 ==== osd_op_reply(14 [pgnls start_epoch 67 out=49b] v65'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556033798fc0 con 0x55603392c400 2023-09-23T03:21:35.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.273+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:15 10.4 10:24924924::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x556033936e00 con 0x55603382b000 2023-09-23T03:21:35.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.273+0000 7f9a4b566640 1 -- 172.21.15.154:0/3974369953 <== osd.0 v2:172.21.15.154:6800/2920938138 2 ==== osd_op_reply(15 [pgnls start_epoch 67 out=81b] v65'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556033798d80 con 0x55603382b000 2023-09-23T03:21:35.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.273+0000 7fb99b2c4640 1 -- 172.21.15.154:0/2342064636 <== osd.6 v2:172.21.15.190:6816/1964294001 3 ==== osd_op_reply(14 10000000202.00000000 [call] v67'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5602efe4ed80 con 0x5602ef4ea400 2023-09-23T03:21:35.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.273+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:16 10.4 10:24ae92ec:::100000001fb.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x556033937500 con 0x55603382b000 2023-09-23T03:21:35.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.273+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:15 9.28 9:177a762d::::head [pgnls start_epoch 67 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef4e5180 con 0x5602ef4ea400 2023-09-23T03:21:35.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.273+0000 7f9a4b566640 1 -- 172.21.15.154:0/3974369953 <== osd.0 v2:172.21.15.154:6800/2920938138 3 ==== osd_op_reply(16 100000001fb.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556033798d80 con 0x55603382b000 2023-09-23T03:21:35.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.273+0000 7f9a4d20c5c0 10 datascan.operator(): handling object 1099511628283.1 2023-09-23T03:21:35.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.273+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:17 9.13 9:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x556033937880 con 0x55603392d000 2023-09-23T03:21:35.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.274+0000 7efdc9cee640 1 -- 172.21.15.154:0/1781234207 <== osd.0 v2:172.21.15.154:6800/2920938138 7 ==== osd_op_reply(27 1000000020c.00000000 [call,setxattr (8)] v67'8 uv8 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x564662cc6b40 con 0x564662196000 2023-09-23T03:21:35.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.274+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:28 10.1e 10:7ac44a97::::head [pgnls start_epoch 67 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x564662186e00 con 0x564662196000 2023-09-23T03:21:35.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.274+0000 7f48e1a6f640 1 --2- 172.21.15.154:0/2963625397 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x556b09a41000 0x556b08b67b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.274+0000 7efdc9cee640 1 -- 172.21.15.154:0/1781234207 <== osd.0 v2:172.21.15.154:6800/2920938138 8 ==== osd_op_reply(28 [pgnls start_epoch 67 out=81b] v65'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564662cc6b40 con 0x564662196000 2023-09-23T03:21:35.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.274+0000 7fb99b2c4640 1 -- 172.21.15.154:0/2342064636 <== osd.6 v2:172.21.15.190:6816/1964294001 4 ==== osd_op_reply(15 [pgnls start_epoch 67 out=81b] v67'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5602efe4ed80 con 0x5602ef4ea400 2023-09-23T03:21:35.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.274+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:16 9.28 9:177a762d:::10000000201.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef4e4700 con 0x5602ef4ea400 2023-09-23T03:21:35.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.274+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:29 10.1e 10:7ac44a97:::10000000207.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x564661e71c00 con 0x564662196000 2023-09-23T03:21:35.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.274+0000 7efdc9cee640 1 -- 172.21.15.154:0/1781234207 <== osd.0 v2:172.21.15.154:6800/2920938138 9 ==== osd_op_reply(29 10000000207.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564662cc6b40 con 0x564662196000 2023-09-23T03:21:35.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.274+0000 7efdcc0145c0 10 datascan.operator(): handling object 1099511628295.1 2023-09-23T03:21:35.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.274+0000 7efdcc0145c0 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x564662179000 0x56466219db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.400 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.275+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:30 9.7 9:e15a914f:::10000000207.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x564662186a80 con 0x564662179000 2023-09-23T03:21:35.400 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.274+0000 7f62bd2bc640 1 -- 172.21.15.154:0/1011343899 <== osd.7 v2:172.21.15.190:6824/3460136272 4 ==== osd_op_reply(6 [pgnls start_epoch 67 out=49b] v65'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b6453cfc0 con 0x555b646ce400 2023-09-23T03:21:35.400 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.275+0000 7f62b8ab3640 1 --2- 172.21.15.154:0/1011343899 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x555b646ce800 0x555b646d4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.400 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.275+0000 7f62b8ab3640 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:7 9.23 9:c4000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b646d8e00 con 0x555b646ce800 2023-09-23T03:21:35.400 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.275+0000 7f62bcabb640 1 --2- 172.21.15.154:0/1011343899 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x555b646ce800 0x555b646d4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.275+0000 7fb99b2c4640 1 -- 172.21.15.154:0/2342064636 <== osd.6 v2:172.21.15.190:6816/1964294001 5 ==== osd_op_reply(16 10000000201.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5602efe4ed80 con 0x5602ef4ea400 2023-09-23T03:21:35.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.275+0000 7fb99ca525c0 10 datascan.operator(): handling object 1099511628289.0 2023-09-23T03:21:35.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.275+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:17 9.28 9:177a762d:::10000000201.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef4e4000 con 0x5602ef4ea400 2023-09-23T03:21:35.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.275+0000 7f48e1a6f640 1 --2- 172.21.15.154:0/2963625397 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x556b09a41000 0x556b08b67b80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.275+0000 7f62bcabb640 1 --2- 172.21.15.154:0/1011343899 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x555b646ce800 0x555b646d4b00 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.275+0000 7efdca4ef640 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x564662179000 0x56466219db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.275+0000 7ffa9eac3640 1 -- 172.21.15.154:0/334556304 <== osd.5 v2:172.21.15.190:6800/882536885 3 ==== osd_op_reply(7 10000000207.00000000 [call] v67'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d6af710fc0 con 0x55d6aecee400 2023-09-23T03:21:35.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.276+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:8 9.7 9:e2e07930::::head [pgnls start_epoch 67 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6aed04a80 con 0x55d6aecee400 2023-09-23T03:21:35.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.276+0000 7f62bcabb640 1 -- 172.21.15.154:0/1011343899 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(7 [pgnls start_epoch 67 out=49b] v65'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b6453cb40 con 0x555b646ce800 2023-09-23T03:21:35.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.276+0000 7f62b82b2640 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:8 9.13 9:c8000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b646d9500 con 0x555b646ce400 2023-09-23T03:21:35.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.277+0000 7fe180339640 1 -- 172.21.15.154:0/3084437610 <== osd.3 v2:172.21.15.154:6808/1133688468 3 ==== osd_op_reply(5 10000000200.00000000 [call] v67'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563379d7afc0 con 0x563379f12400 2023-09-23T03:21:35.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.277+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:6 9.19 9:997b7129::::head [pgnls start_epoch 67 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379f1c700 con 0x563379f12400 2023-09-23T03:21:35.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.277+0000 7ffa9eac3640 1 -- 172.21.15.154:0/334556304 <== osd.5 v2:172.21.15.190:6800/882536885 4 ==== osd_op_reply(8 [pgnls start_epoch 67 out=81b] v67'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d6af710fc0 con 0x55d6aecee400 2023-09-23T03:21:35.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.277+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:9 9.7 9:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6aed04e00 con 0x55d6aecee400 2023-09-23T03:21:35.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.277+0000 7f62bd2bc640 1 -- 172.21.15.154:0/1011343899 <== osd.7 v2:172.21.15.190:6824/3460136272 5 ==== osd_op_reply(8 [pgnls start_epoch 67 out=81b] v67'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555b6453cfc0 con 0x555b646ce400 2023-09-23T03:21:35.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.277+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:9 9.13 9:ca8fdbba:::100000001fb.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b646d8a80 con 0x555b646ce400 2023-09-23T03:21:35.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.277+0000 7f48e1a6f640 1 -- 172.21.15.154:0/2963625397 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== osd_op_reply(18 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b099a6fc0 con 0x556b09a41000 2023-09-23T03:21:35.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.277+0000 7f48dd266640 1 -- 172.21.15.154:0/2963625397 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:19 10.a 10:50000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x556b08e89880 con 0x556b09a41000 2023-09-23T03:21:35.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.277+0000 7fe180339640 1 -- 172.21.15.154:0/3084437610 <== osd.3 v2:172.21.15.154:6808/1133688468 4 ==== osd_op_reply(6 [pgnls start_epoch 67 out=81b] v67'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563379d7afc0 con 0x563379f12400 2023-09-23T03:21:35.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.277+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:7 9.19 9:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379f1ca80 con 0x563379f12400 2023-09-23T03:21:35.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.277+0000 7fe180339640 1 -- 172.21.15.154:0/3084437610 <== osd.3 v2:172.21.15.154:6808/1133688468 5 ==== osd_op_reply(7 100000001f8.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563379d7afc0 con 0x563379f12400 2023-09-23T03:21:35.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.277+0000 7fe181b915c0 10 datascan.operator(): handling object 1099511628280.0 2023-09-23T03:21:35.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.277+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:8 9.19 9:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379afdc00 con 0x563379f12400 2023-09-23T03:21:35.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.278+0000 7efdca4ef640 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x564662179000 0x56466219db80 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.406 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.278+0000 7f48e1a6f640 1 -- 172.21.15.154:0/2963625397 <== osd.6 v2:172.21.15.190:6816/1964294001 2 ==== osd_op_reply(19 [pgnls start_epoch 67 out=101b] v65'5 uv5 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x556b099a6fc0 con 0x556b09a41000 2023-09-23T03:21:35.406 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.278+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:20 10.a 10:5097359c:::100000001fd.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x556b08e88a80 con 0x556b09a41000 2023-09-23T03:21:35.406 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.279+0000 7ffa9eac3640 1 -- 172.21.15.154:0/334556304 <== osd.5 v2:172.21.15.190:6800/882536885 5 ==== osd_op_reply(9 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d6af710fc0 con 0x55d6aecee400 2023-09-23T03:21:35.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.279+0000 7ffaa0a525c0 10 datascan.operator(): handling object 1099511627777.0 2023-09-23T03:21:35.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.279+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:10 9.7 9:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6aed04000 con 0x55d6aecee400 2023-09-23T03:21:35.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.279+0000 7f48e1a6f640 1 -- 172.21.15.154:0/2963625397 <== osd.6 v2:172.21.15.190:6816/1964294001 3 ==== osd_op_reply(20 100000001fd.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556b099a6fc0 con 0x556b09a41000 2023-09-23T03:21:35.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.280+0000 7f48e3e0c5c0 10 datascan.operator(): handling object 1099511628285.1 2023-09-23T03:21:35.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.280+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:21 9.10 9:08b99fea:::100000001fd.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x556b08e88e00 con 0x556b08e7e000 2023-09-23T03:21:35.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.280+0000 7f62bd2bc640 1 -- 172.21.15.154:0/1011343899 <== osd.7 v2:172.21.15.190:6824/3460136272 6 ==== osd_op_reply(9 100000001fb.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555b6453cfc0 con 0x555b646ce400 2023-09-23T03:21:35.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.280+0000 7f62bf24b5c0 10 datascan.operator(): handling object 1099511628283.0 2023-09-23T03:21:35.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.280+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:10 9.13 9:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b642bfc00 con 0x555b646ce400 2023-09-23T03:21:35.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.280+0000 7f9a4ad65640 1 -- 172.21.15.154:0/3974369953 <== osd.7 v2:172.21.15.190:6824/3460136272 2 ==== osd_op_reply(17 100000001fb.00000000 [call,setxattr (8)] v67'17 uv17 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x556033798fc0 con 0x55603392d000 2023-09-23T03:21:35.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.280+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:18 10.14 10:28000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x556033519c00 con 0x55603392c000 2023-09-23T03:21:35.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.280+0000 7f9a4b566640 1 -- 172.21.15.154:0/3974369953 <== osd.1 v2:172.21.15.154:6816/1267710005 2 ==== osd_op_reply(18 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556033798d80 con 0x55603392c000 2023-09-23T03:21:35.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.280+0000 7f9a4655c640 1 -- 172.21.15.154:0/3974369953 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:19 10.c 10:30000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x556033937c00 con 0x55603392d000 2023-09-23T03:21:35.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.281+0000 7f9a4ad65640 1 -- 172.21.15.154:0/3974369953 <== osd.7 v2:172.21.15.190:6824/3460136272 3 ==== osd_op_reply(19 [pgnls start_epoch 67 out=101b] v65'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x556033798fc0 con 0x55603392d000 2023-09-23T03:21:35.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.281+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:20 10.c 10:30604137:::100000001fc.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55603395c000 con 0x55603392d000 2023-09-23T03:21:35.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.283+0000 7f9a4ad65640 1 -- 172.21.15.154:0/3974369953 <== osd.7 v2:172.21.15.190:6824/3460136272 4 ==== osd_op_reply(20 100000001fc.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556033798fc0 con 0x55603392d000 2023-09-23T03:21:35.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.283+0000 7f9a4d20c5c0 10 datascan.operator(): handling object 1099511628284.0 2023-09-23T03:21:35.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.283+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:21 9.c 9:30604137:::100000001fc.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x556033936380 con 0x55603392c800 2023-09-23T03:21:35.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.283+0000 7fb99b2c4640 1 -- 172.21.15.154:0/2342064636 <== osd.6 v2:172.21.15.190:6816/1964294001 6 ==== osd_op_reply(17 10000000201.00000000 [call] v67'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5602efe4ed80 con 0x5602ef4ea400 2023-09-23T03:21:35.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.283+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:18 9.18 9:18000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef4e5500 con 0x5602ef4ea400 2023-09-23T03:21:35.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.284+0000 7efdca4ef640 1 -- 172.21.15.154:0/1781234207 <== osd.5 v2:172.21.15.190:6800/882536885 1 ==== osd_op_reply(30 10000000207.00000000 [call,setxattr (8)] v67'9 uv9 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x564662cc6d80 con 0x564662179000 2023-09-23T03:21:35.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.284+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:31 10.1 10:80000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x564662187c00 con 0x564662196800 2023-09-23T03:21:35.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.284+0000 7fb99b2c4640 1 -- 172.21.15.154:0/2342064636 <== osd.6 v2:172.21.15.190:6816/1964294001 7 ==== osd_op_reply(18 [pgnls start_epoch 67 out=49b] v65'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5602efe4ed80 con 0x5602ef4ea400 2023-09-23T03:21:35.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.284+0000 7fb9962ba640 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:19 9.38 9:1c000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef4e4a80 con 0x5602ef4ea000 2023-09-23T03:21:35.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.284+0000 7fe180339640 1 -- 172.21.15.154:0/3084437610 <== osd.3 v2:172.21.15.154:6808/1133688468 6 ==== osd_op_reply(8 100000001f8.00000000 [call] v67'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563379d7afc0 con 0x563379f12400 2023-09-23T03:21:35.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.284+0000 7fe181b915c0 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x563379d9f000 0x563379b0f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.285+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:9 9.39 9:9c000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379f1c380 con 0x563379d9f000 2023-09-23T03:21:35.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.285+0000 7fb99a2c2640 1 -- 172.21.15.154:0/2342064636 <== osd.3 v2:172.21.15.154:6808/1133688468 2 ==== osd_op_reply(19 [pgnls start_epoch 67 out=49b] v65'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5602efe4eb40 con 0x5602ef4ea000 2023-09-23T03:21:35.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.285+0000 7fb995ab9640 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:20 9.4 9:20000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef4e5180 con 0x5602ef4ce000 2023-09-23T03:21:35.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.285+0000 7efdc9cee640 1 -- 172.21.15.154:0/1781234207 <== osd.4 v2:172.21.15.190:6808/717302641 3 ==== osd_op_reply(31 [pgnls start_epoch 67 out=101b] v65'2 uv2 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564662cc6b40 con 0x564662196800 2023-09-23T03:21:35.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.285+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:32 10.1 10:8410d73e:::10000000201.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x564662186000 con 0x564662196800 2023-09-23T03:21:35.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.285+0000 7f48e2270640 1 -- 172.21.15.154:0/2963625397 <== osd.4 v2:172.21.15.190:6808/717302641 3 ==== osd_op_reply(21 100000001fd.00000000 [call,setxattr (8)] v67'9 uv9 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x556b099a6d80 con 0x556b08e7e000 2023-09-23T03:21:35.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.285+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:22 10.a 10:55a6cd83::::head [pgnls start_epoch 67 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x556b08e89180 con 0x556b09a41000 2023-09-23T03:21:35.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.285+0000 7fb99a2c2640 1 -- 172.21.15.154:0/2342064636 <== osd.1 v2:172.21.15.154:6816/1267710005 4 ==== osd_op_reply(20 [pgnls start_epoch 67 out=49b] v65'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5602efe4eb40 con 0x5602ef4ce000 2023-09-23T03:21:35.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.285+0000 7fb9962ba640 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5602ef4ea800 0x5602ef4f0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.285+0000 7fb9962ba640 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:21 9.24 9:24000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef4e4e00 con 0x5602ef4ea800 2023-09-23T03:21:35.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.285+0000 7fe17fb38640 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x563379d9f000 0x563379b0f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.285+0000 7fb99aac3640 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5602ef4ea800 0x5602ef4f0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.286+0000 7efdc9cee640 1 -- 172.21.15.154:0/1781234207 <== osd.4 v2:172.21.15.190:6808/717302641 4 ==== osd_op_reply(32 10000000201.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564662cc6b40 con 0x564662196800 2023-09-23T03:21:35.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.286+0000 7efdcc0145c0 10 datascan.operator(): handling object 1099511628289.1 2023-09-23T03:21:35.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.286+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:33 9.28 9:177a762d:::10000000201.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x564662187880 con 0x564662178c00 2023-09-23T03:21:35.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.286+0000 7f48e1a6f640 1 -- 172.21.15.154:0/2963625397 <== osd.6 v2:172.21.15.190:6816/1964294001 4 ==== osd_op_reply(22 [pgnls start_epoch 67 out=101b] v65'5 uv5 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x556b099a6fc0 con 0x556b09a41000 2023-09-23T03:21:35.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.286+0000 7fb99aac3640 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5602ef4ea800 0x5602ef4f0b00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.286+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:23 10.a 10:55a6cd83:::1000000020a.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x556b08e88380 con 0x556b09a41000 2023-09-23T03:21:35.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.286+0000 7fb99aac3640 1 -- 172.21.15.154:0/2342064636 <== osd.2 v2:172.21.15.154:6822/2884407243 1 ==== osd_op_reply(21 [pgnls start_epoch 67 out=49b] v65'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5602efe4efc0 con 0x5602ef4ea800 2023-09-23T03:21:35.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.286+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:22 10.0 10:00000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef4e5880 con 0x5602ef4ea400 2023-09-23T03:21:35.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.286+0000 7f9a4a564640 1 -- 172.21.15.154:0/3974369953 <== osd.3 v2:172.21.15.154:6808/1133688468 8 ==== osd_op_reply(21 100000001fc.00000000 [call,setxattr (8)] v67'9 uv9 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x556033798b40 con 0x55603392c800 2023-09-23T03:21:35.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.287+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:22 10.c 10:37389ade::::head [pgnls start_epoch 67 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x556033937880 con 0x55603392d000 2023-09-23T03:21:35.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.287+0000 7fe17fb38640 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x563379d9f000 0x563379b0f180 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.287+0000 7ffa9eac3640 1 -- 172.21.15.154:0/334556304 <== osd.5 v2:172.21.15.190:6800/882536885 6 ==== osd_op_reply(10 10000000001.00000000 [call] v67'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d6af710fc0 con 0x55d6aecee400 2023-09-23T03:21:35.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.287+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:11 9.27 9:e4000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6aed04380 con 0x55d6aecf6000 2023-09-23T03:21:35.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.287+0000 7f62bd2bc640 1 -- 172.21.15.154:0/1011343899 <== osd.7 v2:172.21.15.190:6824/3460136272 7 ==== osd_op_reply(10 100000001fb.00000000 [call] v67'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x555b6453cfc0 con 0x555b646ce400 2023-09-23T03:21:35.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.287+0000 7f48e1a6f640 1 -- 172.21.15.154:0/2963625397 <== osd.6 v2:172.21.15.190:6816/1964294001 5 ==== osd_op_reply(23 1000000020a.00000000 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556b099a6fc0 con 0x556b09a41000 2023-09-23T03:21:35.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.287+0000 7f48e3e0c5c0 10 datascan.operator(): handling object 1099511628298.0 2023-09-23T03:21:35.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.287+0000 7f62bf24b5c0 1 --2- 172.21.15.154:0/1011343899 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x555b645bb000 0x555b642cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.287+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:11 9.33 9:cc000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b646d8380 con 0x555b645bb000 2023-09-23T03:21:35.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.287+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:24 9.2a 9:55a6cd83:::1000000020a.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x556b08e89c00 con 0x556b08e7e400 2023-09-23T03:21:35.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.287+0000 7ffa9f2c4640 1 -- 172.21.15.154:0/334556304 <== osd.2 v2:172.21.15.154:6822/2884407243 4 ==== osd_op_reply(11 [pgnls start_epoch 67 out=81b] v65'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d6af710d80 con 0x55d6aecf6000 2023-09-23T03:21:35.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.287+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:12 9.27 9:e5993619:::10000000208.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6aed04700 con 0x55d6aecf6000 2023-09-23T03:21:35.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.287+0000 7fe17fb38640 1 -- 172.21.15.154:0/3084437610 <== osd.4 v2:172.21.15.190:6808/717302641 1 ==== osd_op_reply(9 [pgnls start_epoch 67 out=49b] v65'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563379d7ab40 con 0x563379d9f000 2023-09-23T03:21:35.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.287+0000 7f9a4ad65640 1 -- 172.21.15.154:0/3974369953 <== osd.7 v2:172.21.15.190:6824/3460136272 5 ==== osd_op_reply(22 [pgnls start_epoch 67 out=81b] v65'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556033798fc0 con 0x55603392d000 2023-09-23T03:21:35.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.287+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:23 10.c 10:37389ade:::10000000209.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x556033936000 con 0x55603392d000 2023-09-23T03:21:35.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.287+0000 7fe17b32f640 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:10 9.5 9:a0000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379f1c380 con 0x563379d9f400 2023-09-23T03:21:35.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.288+0000 7f62bdabd640 1 --2- 172.21.15.154:0/1011343899 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x555b645bb000 0x555b642cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.288+0000 7ffa9f2c4640 1 -- 172.21.15.154:0/334556304 <== osd.2 v2:172.21.15.154:6822/2884407243 5 ==== osd_op_reply(12 10000000208.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d6af710d80 con 0x55d6aecf6000 2023-09-23T03:21:35.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.288+0000 7ffaa0a525c0 10 datascan.operator(): handling object 1099511628296.0 2023-09-23T03:21:35.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.288+0000 7fe17fb38640 1 -- 172.21.15.154:0/3084437610 <== osd.0 v2:172.21.15.154:6800/2920938138 2 ==== osd_op_reply(10 [pgnls start_epoch 67 out=81b] v65'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563379d7ab40 con 0x563379d9f400 2023-09-23T03:21:35.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.288+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:11 9.5 9:a0beda95:::10000000204.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379f1c700 con 0x563379d9f400 2023-09-23T03:21:35.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.288+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:13 9.27 9:e5993619:::10000000208.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6aed04a80 con 0x55d6aecf6000 2023-09-23T03:21:35.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.288+0000 7f9a4ad65640 1 -- 172.21.15.154:0/3974369953 <== osd.7 v2:172.21.15.190:6824/3460136272 6 ==== osd_op_reply(23 10000000209.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556033798fc0 con 0x55603392d000 2023-09-23T03:21:35.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.288+0000 7f9a4d20c5c0 10 datascan.operator(): handling object 1099511628297.0 2023-09-23T03:21:35.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.288+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:24 9.2c 9:37389ade:::10000000209.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x556033936700 con 0x55603392c800 2023-09-23T03:21:35.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.288+0000 7fb99b2c4640 1 -- 172.21.15.154:0/2342064636 <== osd.6 v2:172.21.15.190:6816/1964294001 8 ==== osd_op_reply(22 [pgnls start_epoch 67 out=101b] v65'5 uv5 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5602efe4ed80 con 0x5602ef4ea400 2023-09-23T03:21:35.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.289+0000 7f62bdabd640 1 --2- 172.21.15.154:0/1011343899 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x555b645bb000 0x555b642cfb80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.289+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:23 10.0 10:012cdbc7:::10000000206.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef4e5c00 con 0x5602ef4ea400 2023-09-23T03:21:35.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.289+0000 7fe17fb38640 1 -- 172.21.15.154:0/3084437610 <== osd.0 v2:172.21.15.154:6800/2920938138 3 ==== osd_op_reply(11 10000000204.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563379d7ab40 con 0x563379d9f400 2023-09-23T03:21:35.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.289+0000 7fe181b915c0 10 datascan.operator(): handling object 1099511628292.0 2023-09-23T03:21:35.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.289+0000 7f48e1a6f640 1 -- 172.21.15.154:0/2963625397 <== osd.3 v2:172.21.15.154:6808/1133688468 8 ==== osd_op_reply(24 1000000020a.00000000 [call,setxattr (8)] v67'14 uv14 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x556b099a6fc0 con 0x556b08e7e400 2023-09-23T03:21:35.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.289+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:12 9.5 9:a0beda95:::10000000204.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379f1ca80 con 0x563379d9f400 2023-09-23T03:21:35.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.289+0000 7f62bdabd640 1 -- 172.21.15.154:0/1011343899 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== osd_op_reply(11 [pgnls start_epoch 67 out=49b] v65'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b6453cd80 con 0x555b645bb000 2023-09-23T03:21:35.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.289+0000 7f62b82b2640 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:12 9.b 9:d0000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b646d9880 con 0x555b646ce000 2023-09-23T03:21:35.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.289+0000 7fb99b2c4640 1 -- 172.21.15.154:0/2342064636 <== osd.6 v2:172.21.15.190:6816/1964294001 9 ==== osd_op_reply(23 10000000206.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5602efe4ed80 con 0x5602ef4ea400 2023-09-23T03:21:35.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.289+0000 7efdc94ed640 1 -- 172.21.15.154:0/1781234207 <== osd.6 v2:172.21.15.190:6816/1964294001 5 ==== osd_op_reply(33 10000000201.00000000 [call,setxattr (8)] v67'11 uv11 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x564662cc6fc0 con 0x564662178c00 2023-09-23T03:21:35.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.290+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:34 10.1 10:848ebb0b::::head [pgnls start_epoch 67 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x564662186e00 con 0x564662196800 2023-09-23T03:21:35.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.290+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:25 10.a 10:57fe7535::::head [pgnls start_epoch 67 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x556b08e89880 con 0x556b09a41000 2023-09-23T03:21:35.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.290+0000 7fb99ca525c0 10 datascan.operator(): handling object 1099511628294.0 2023-09-23T03:21:35.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.290+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:24 9.0 9:012cdbc7:::10000000206.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef0d5880 con 0x5602ef4ce000 2023-09-23T03:21:35.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.290+0000 7efdc9cee640 1 -- 172.21.15.154:0/1781234207 <== osd.4 v2:172.21.15.190:6808/717302641 5 ==== osd_op_reply(34 [pgnls start_epoch 67 out=81b] v65'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564662cc6b40 con 0x564662196800 2023-09-23T03:21:35.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.290+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:35 10.1 10:848ebb0b:::100000001f7.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x564661e71880 con 0x564662196800 2023-09-23T03:21:35.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.290+0000 7f62bdabd640 1 -- 172.21.15.154:0/1011343899 <== osd.0 v2:172.21.15.154:6800/2920938138 2 ==== osd_op_reply(12 [pgnls start_epoch 67 out=49b] v65'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b6453cd80 con 0x555b646ce000 2023-09-23T03:21:35.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.290+0000 7f62b8ab3640 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:13 9.2b 9:d4000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b646d9180 con 0x555b646ce400 2023-09-23T03:21:35.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.290+0000 7f48e1a6f640 1 -- 172.21.15.154:0/2963625397 <== osd.6 v2:172.21.15.190:6816/1964294001 6 ==== osd_op_reply(25 [pgnls start_epoch 67 out=81b] v65'5 uv5 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556b099a6fc0 con 0x556b09a41000 2023-09-23T03:21:35.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.290+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:26 10.a 10:57fe7535:::100000001f6.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x556b08e88a80 con 0x556b09a41000 2023-09-23T03:21:35.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.290+0000 7ffa9f2c4640 1 -- 172.21.15.154:0/334556304 <== osd.2 v2:172.21.15.154:6822/2884407243 6 ==== osd_op_reply(13 10000000208.00000000 [call] v67'4 uv4 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d6af710d80 con 0x55d6aecf6000 2023-09-23T03:21:35.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.290+0000 7ffaa0a525c0 1 --2- 172.21.15.154:0/334556304 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55d6aecee800 0x55d6ae8db600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.290+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:14 9.17 9:e8000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6aed04e00 con 0x55d6aecee800 2023-09-23T03:21:35.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.291+0000 7efdc9cee640 1 -- 172.21.15.154:0/1781234207 <== osd.4 v2:172.21.15.190:6808/717302641 6 ==== osd_op_reply(35 100000001f7.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564662cc6b40 con 0x564662196800 2023-09-23T03:21:35.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.291+0000 7efdcc0145c0 10 datascan.operator(): handling object 1099511628279.1 2023-09-23T03:21:35.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.291+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:36 9.36 9:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x564662186380 con 0x564662178000 2023-09-23T03:21:35.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.291+0000 7f9a4a564640 1 -- 172.21.15.154:0/3974369953 <== osd.3 v2:172.21.15.154:6808/1133688468 9 ==== osd_op_reply(24 10000000209.00000000 [call,setxattr (8)] v67'11 uv11 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x556033798b40 con 0x55603392c800 2023-09-23T03:21:35.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.291+0000 7f9a4d20c5c0 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55603382b400 0x556033529600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.291+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:25 10.1c 10:38000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x556033936e00 con 0x55603382b400 2023-09-23T03:21:35.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.291+0000 7f62bd2bc640 1 -- 172.21.15.154:0/1011343899 <== osd.7 v2:172.21.15.190:6824/3460136272 8 ==== osd_op_reply(13 [pgnls start_epoch 67 out=81b] v65'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555b6453cfc0 con 0x555b646ce400 2023-09-23T03:21:35.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.291+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:14 9.2b 9:d4a58307:::10000000203.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b646d8700 con 0x555b646ce400 2023-09-23T03:21:35.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.291+0000 7ffa9e2c2640 1 --2- 172.21.15.154:0/334556304 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55d6aecee800 0x55d6ae8db600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.292+0000 7f48e1a6f640 1 -- 172.21.15.154:0/2963625397 <== osd.6 v2:172.21.15.190:6816/1964294001 7 ==== osd_op_reply(26 100000001f6.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556b099a6fc0 con 0x556b09a41000 2023-09-23T03:21:35.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.292+0000 7f48e3e0c5c0 10 datascan.operator(): handling object 1099511628278.0 2023-09-23T03:21:35.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.292+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:27 9.2a 9:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x556b08e89180 con 0x556b08e7e400 2023-09-23T03:21:35.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.292+0000 7f9a4a564640 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55603382b400 0x556033529600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.292+0000 7fe17fb38640 1 -- 172.21.15.154:0/3084437610 <== osd.0 v2:172.21.15.154:6800/2920938138 4 ==== osd_op_reply(12 10000000204.00000000 [call] v67'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563379d7ab40 con 0x563379d9f400 2023-09-23T03:21:35.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.292+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:13 9.25 9:a4000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379f1ce00 con 0x563379d9f400 2023-09-23T03:21:35.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.293+0000 7f62bd2bc640 1 -- 172.21.15.154:0/1011343899 <== osd.7 v2:172.21.15.190:6824/3460136272 9 ==== osd_op_reply(14 10000000203.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555b6453cfc0 con 0x555b646ce400 2023-09-23T03:21:35.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.293+0000 7f62bf24b5c0 10 datascan.operator(): handling object 1099511628291.0 2023-09-23T03:21:35.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.293+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:15 9.2b 9:d4a58307:::10000000203.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b646d9500 con 0x555b646ce400 2023-09-23T03:21:35.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.293+0000 7fe17fb38640 1 -- 172.21.15.154:0/3084437610 <== osd.0 v2:172.21.15.154:6800/2920938138 5 ==== osd_op_reply(13 [pgnls start_epoch 67 out=49b] v65'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563379d7ab40 con 0x563379d9f400 2023-09-23T03:21:35.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.293+0000 7f9a4a564640 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55603382b400 0x556033529600 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.293+0000 7fe17b32f640 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:14 9.15 9:a8000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379afd880 con 0x563379f12000 2023-09-23T03:21:35.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.293+0000 7ffa9e2c2640 1 --2- 172.21.15.154:0/334556304 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55d6aecee800 0x55d6ae8db600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.294+0000 7f48e1a6f640 1 -- 172.21.15.154:0/2963625397 <== osd.3 v2:172.21.15.154:6808/1133688468 9 ==== osd_op_reply(27 100000001f6.00000000 [call,setxattr (8)] v67'15 uv15 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x556b099a6fc0 con 0x556b08e7e400 2023-09-23T03:21:35.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.294+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:28 10.1a 10:58000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x556b08e88380 con 0x556b09a41000 2023-09-23T03:21:35.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.294+0000 7ffa9e2c2640 1 -- 172.21.15.154:0/334556304 <== osd.7 v2:172.21.15.190:6824/3460136272 1 ==== osd_op_reply(14 [pgnls start_epoch 67 out=49b] v65'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d6af710b40 con 0x55d6aecee800 2023-09-23T03:21:35.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.294+0000 7fe180b3a640 1 -- 172.21.15.154:0/3084437610 <== osd.7 v2:172.21.15.190:6824/3460136272 2 ==== osd_op_reply(14 [pgnls start_epoch 67 out=81b] v65'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563379d7ad80 con 0x563379f12000 2023-09-23T03:21:35.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.294+0000 7f9a4a564640 1 -- 172.21.15.154:0/3974369953 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== osd_op_reply(25 [pgnls start_epoch 67 out=81b] v65'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556033798b40 con 0x55603382b400 2023-09-23T03:21:35.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.294+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:26 10.1c 10:3d6db04b:::100000001f6.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x556033937c00 con 0x55603382b400 2023-09-23T03:21:35.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.294+0000 7ffa9a2ba640 1 --2- 172.21.15.154:0/334556304 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55d6aecf6400 0x55d6aecf8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.294+0000 7ffa9a2ba640 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:15 9.37 9:ec000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6ae8cbc00 con 0x55d6aecf6400 2023-09-23T03:21:35.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.294+0000 7fb99a2c2640 1 -- 172.21.15.154:0/2342064636 <== osd.1 v2:172.21.15.154:6816/1267710005 5 ==== osd_op_reply(24 10000000206.00000000 [call,setxattr (8)] v67'11 uv11 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5602efe4eb40 con 0x5602ef4ce000 2023-09-23T03:21:35.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.294+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:15 9.15 9:a8e69ea8:::1000000020d.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379f1d180 con 0x563379f12000 2023-09-23T03:21:35.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.294+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:25 10.0 10:04f22c1d::::head [pgnls start_epoch 67 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef4e4a80 con 0x5602ef4ea400 2023-09-23T03:21:35.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.294+0000 7f48e1a6f640 1 -- 172.21.15.154:0/2963625397 <== osd.6 v2:172.21.15.190:6816/1964294001 8 ==== osd_op_reply(28 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556b099a6fc0 con 0x556b09a41000 2023-09-23T03:21:35.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.295+0000 7f48dd266640 1 -- 172.21.15.154:0/2963625397 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:29 10.6 10:60000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x556b08b57880 con 0x556b09a40c00 2023-09-23T03:21:35.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.295+0000 7f9a4a564640 1 -- 172.21.15.154:0/3974369953 <== osd.6 v2:172.21.15.190:6816/1964294001 2 ==== osd_op_reply(26 100000001f6.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556033798b40 con 0x55603382b400 2023-09-23T03:21:35.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.295+0000 7f9a4d20c5c0 10 datascan.operator(): handling object 1099511628278.1 2023-09-23T03:21:35.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.295+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:27 9.2a 9:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x556033937880 con 0x55603392c800 2023-09-23T03:21:35.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.295+0000 7fe180b3a640 1 -- 172.21.15.154:0/3084437610 <== osd.7 v2:172.21.15.190:6824/3460136272 3 ==== osd_op_reply(15 1000000020d.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563379d7ad80 con 0x563379f12000 2023-09-23T03:21:35.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.295+0000 7fe181b915c0 10 datascan.operator(): handling object 1099511628301.0 2023-09-23T03:21:35.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.295+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:16 9.15 9:a8e69ea8:::1000000020d.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379f1c000 con 0x563379f12000 2023-09-23T03:21:35.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.295+0000 7fb99b2c4640 1 -- 172.21.15.154:0/2342064636 <== osd.6 v2:172.21.15.190:6816/1964294001 10 ==== osd_op_reply(25 [pgnls start_epoch 67 out=101b] v65'5 uv5 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5602efe4ed80 con 0x5602ef4ea400 2023-09-23T03:21:35.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.295+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:26 10.0 10:04f22c1d:::1000000020a.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef4e5180 con 0x5602ef4ea400 2023-09-23T03:21:35.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.295+0000 7ffa9f2c4640 1 --2- 172.21.15.154:0/334556304 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55d6aecf6400 0x55d6aecf8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.296+0000 7f48e126e640 1 -- 172.21.15.154:0/2963625397 <== osd.5 v2:172.21.15.190:6800/882536885 7 ==== osd_op_reply(29 [pgnls start_epoch 67 out=81b] v65'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556b099a6b40 con 0x556b09a40c00 2023-09-23T03:21:35.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.296+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:30 10.6 10:658f2455:::100000001ff.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x556b08b57c00 con 0x556b09a40c00 2023-09-23T03:21:35.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.296+0000 7f9a4a564640 1 -- 172.21.15.154:0/3974369953 <== osd.3 v2:172.21.15.154:6808/1133688468 10 ==== osd_op_reply(27 100000001f6.00000000 [call,setxattr (8)] v67'16 uv16 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x556033798b40 con 0x55603392c800 2023-09-23T03:21:35.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.296+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:28 10.2 10:40000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x556033936000 con 0x55603392c000 2023-09-23T03:21:35.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.296+0000 7efdc9cee640 1 -- 172.21.15.154:0/1781234207 <== osd.1 v2:172.21.15.154:6816/1267710005 9 ==== osd_op_reply(36 100000001f7.00000000 [call,setxattr (8)] v67'22 uv22 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x564662cc6b40 con 0x564662178000 2023-09-23T03:21:35.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.296+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:37 10.11 10:88000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x564662186700 con 0x564662179000 2023-09-23T03:21:35.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.297+0000 7ffa9f2c4640 1 --2- 172.21.15.154:0/334556304 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55d6aecf6400 0x55d6aecf8580 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.297+0000 7f9a4b566640 1 -- 172.21.15.154:0/3974369953 <== osd.1 v2:172.21.15.154:6816/1267710005 3 ==== osd_op_reply(28 [pgnls start_epoch 67 out=81b] v65'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556033798d80 con 0x55603392c000 2023-09-23T03:21:35.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.297+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:29 10.2 10:40d56c48:::100000001fc.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x556033937180 con 0x55603392c000 2023-09-23T03:21:35.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.297+0000 7fb99b2c4640 1 -- 172.21.15.154:0/2342064636 <== osd.6 v2:172.21.15.190:6816/1964294001 11 ==== osd_op_reply(26 1000000020a.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5602efe4ed80 con 0x5602ef4ea400 2023-09-23T03:21:35.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.297+0000 7fb99ca525c0 10 datascan.operator(): handling object 1099511628298.1 2023-09-23T03:21:35.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.297+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:27 9.2a 9:55a6cd83:::1000000020a.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef0d5180 con 0x5602ef4ea000 2023-09-23T03:21:35.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.297+0000 7f62bd2bc640 1 -- 172.21.15.154:0/1011343899 <== osd.7 v2:172.21.15.190:6824/3460136272 10 ==== osd_op_reply(15 10000000203.00000000 [call] v67'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x555b6453cfc0 con 0x555b646ce400 2023-09-23T03:21:35.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.297+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:16 9.1b 9:d8000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b646d9c00 con 0x555b646ce800 2023-09-23T03:21:35.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.297+0000 7ffa9f2c4640 1 -- 172.21.15.154:0/334556304 <== osd.4 v2:172.21.15.190:6808/717302641 1 ==== osd_op_reply(15 [pgnls start_epoch 67 out=49b] v65'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d6af710d80 con 0x55d6aecf6400 2023-09-23T03:21:35.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.297+0000 7ffa99ab9640 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:16 9.f 9:f0000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6aed04380 con 0x55d6aecee400 2023-09-23T03:21:35.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.297+0000 7f48e126e640 1 -- 172.21.15.154:0/2963625397 <== osd.5 v2:172.21.15.190:6800/882536885 8 ==== osd_op_reply(30 100000001ff.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556b099a6b40 con 0x556b09a40c00 2023-09-23T03:21:35.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.297+0000 7f9a4b566640 1 -- 172.21.15.154:0/3974369953 <== osd.1 v2:172.21.15.154:6816/1267710005 4 ==== osd_op_reply(29 100000001fc.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556033798d80 con 0x55603392c000 2023-09-23T03:21:35.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.298+0000 7f9a4d20c5c0 10 datascan.operator(): handling object 1099511628284.1 2023-09-23T03:21:35.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.298+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:30 9.c 9:30604137:::100000001fc.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x556033936a80 con 0x55603392c800 2023-09-23T03:21:35.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.298+0000 7f48e3e0c5c0 10 datascan.operator(): handling object 1099511628287.0 2023-09-23T03:21:35.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.298+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:31 9.26 9:658f2455:::100000001ff.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x556b08b57880 con 0x556b09a40c00 2023-09-23T03:21:35.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.298+0000 7efdca4ef640 1 -- 172.21.15.154:0/1781234207 <== osd.5 v2:172.21.15.190:6800/882536885 2 ==== osd_op_reply(37 [pgnls start_epoch 67 out=101b] v65'2 uv2 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564662cc6d80 con 0x564662179000 2023-09-23T03:21:35.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.298+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:38 10.11 10:893be98d:::100000001f8.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x564662186a80 con 0x564662179000 2023-09-23T03:21:35.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.298+0000 7f62bcabb640 1 -- 172.21.15.154:0/1011343899 <== osd.3 v2:172.21.15.154:6808/1133688468 2 ==== osd_op_reply(16 [pgnls start_epoch 67 out=49b] v65'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b6453cb40 con 0x555b646ce800 2023-09-23T03:21:35.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.298+0000 7ffa9eac3640 1 -- 172.21.15.154:0/334556304 <== osd.5 v2:172.21.15.190:6800/882536885 7 ==== osd_op_reply(16 [pgnls start_epoch 67 out=49b] v65'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d6af710fc0 con 0x55d6aecee400 2023-09-23T03:21:35.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.298+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:17 10.d 10:b6db6db6::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b646d8e00 con 0x555b645bb000 2023-09-23T03:21:35.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.298+0000 7ffa9a2ba640 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:17 9.2f 9:f4000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6aed05500 con 0x55d6aecee000 2023-09-23T03:21:35.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.298+0000 7efdca4ef640 1 -- 172.21.15.154:0/1781234207 <== osd.5 v2:172.21.15.190:6800/882536885 3 ==== osd_op_reply(38 100000001f8.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564662cc6d80 con 0x564662179000 2023-09-23T03:21:35.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.298+0000 7efdcc0145c0 10 datascan.operator(): handling object 1099511628280.1 2023-09-23T03:21:35.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.298+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:39 9.19 9:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x5646621c4000 con 0x564662178800 2023-09-23T03:21:35.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.299+0000 7fe180b3a640 1 -- 172.21.15.154:0/3084437610 <== osd.7 v2:172.21.15.190:6824/3460136272 4 ==== osd_op_reply(16 1000000020d.00000000 [call] v67'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563379d7ad80 con 0x563379f12000 2023-09-23T03:21:35.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.299+0000 7fe181b915c0 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x563379d9ec00 0x563379f18b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.299+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:17 9.35 9:ac000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379f1d180 con 0x563379d9ec00 2023-09-23T03:21:35.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.299+0000 7f62bdabd640 1 -- 172.21.15.154:0/1011343899 <== osd.6 v2:172.21.15.190:6816/1964294001 2 ==== osd_op_reply(17 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b6453cd80 con 0x555b645bb000 2023-09-23T03:21:35.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.299+0000 7ffa9e2c2640 1 -- 172.21.15.154:0/334556304 <== osd.3 v2:172.21.15.154:6808/1133688468 2 ==== osd_op_reply(17 [pgnls start_epoch 67 out=49b] v65'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d6af710b40 con 0x55d6aecee000 2023-09-23T03:21:35.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.299+0000 7f62b82b2640 1 --2- 172.21.15.154:0/1011343899 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x555b646cf000 0x555b646d5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.299+0000 7f62b82b2640 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:18 10.1d 10:b8000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b646d8700 con 0x555b646cf000 2023-09-23T03:21:35.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.299+0000 7fe180b3a640 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x563379d9ec00 0x563379f18b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.299+0000 7ffa99ab9640 1 --2- 172.21.15.154:0/334556304 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55d6aecf6800 0x55d6aecf8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.299+0000 7ffa99ab9640 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:18 9.1f 9:f8000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6aed04700 con 0x55d6aecf6800 2023-09-23T03:21:35.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.299+0000 7ffa9eac3640 1 --2- 172.21.15.154:0/334556304 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55d6aecf6800 0x55d6aecf8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.300+0000 7fe180b3a640 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x563379d9ec00 0x563379f18b00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.300+0000 7f62bd2bc640 1 --2- 172.21.15.154:0/1011343899 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x555b646cf000 0x555b646d5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.300+0000 7fe180b3a640 1 -- 172.21.15.154:0/3084437610 <== osd.1 v2:172.21.15.154:6816/1267710005 1 ==== osd_op_reply(17 [pgnls start_epoch 67 out=49b] v65'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563379d7ad80 con 0x563379d9ec00 2023-09-23T03:21:35.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.300+0000 7fb99a2c2640 1 -- 172.21.15.154:0/2342064636 <== osd.3 v2:172.21.15.154:6808/1133688468 3 ==== osd_op_reply(27 1000000020a.00000000 [call,setxattr (8)] v67'17 uv17 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5602efe4eb40 con 0x5602ef4ea000 2023-09-23T03:21:35.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.300+0000 7fe17b32f640 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:18 9.d 9:b0000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379f1d500 con 0x563379f12000 2023-09-23T03:21:35.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.300+0000 7ffa9eac3640 1 --2- 172.21.15.154:0/334556304 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55d6aecf6800 0x55d6aecf8b00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.300+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:28 10.0 10:05552577::::head [pgnls start_epoch 67 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef0d5c00 con 0x5602ef4ea400 2023-09-23T03:21:35.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.300+0000 7ffa9eac3640 1 -- 172.21.15.154:0/334556304 <== osd.1 v2:172.21.15.154:6816/1267710005 1 ==== osd_op_reply(18 [pgnls start_epoch 67 out=49b] v65'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d6af710fc0 con 0x55d6aecf6800 2023-09-23T03:21:35.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.300+0000 7ffa9a2ba640 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:19 9.3f 9:fc000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6aed05880 con 0x55d6aecf6000 2023-09-23T03:21:35.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.301+0000 7ffa9f2c4640 1 -- 172.21.15.154:0/334556304 <== osd.2 v2:172.21.15.154:6822/2884407243 7 ==== osd_op_reply(19 [pgnls start_epoch 67 out=81b] v65'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d6af710d80 con 0x55d6aecf6000 2023-09-23T03:21:35.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.301+0000 7f62bd2bc640 1 --2- 172.21.15.154:0/1011343899 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x555b646cf000 0x555b646d5600 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.301+0000 7f9a4a564640 1 -- 172.21.15.154:0/3974369953 <== osd.3 v2:172.21.15.154:6808/1133688468 11 ==== osd_op_reply(30 100000001fc.00000000 [call,setxattr (8)] v67'10 uv10 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x556033798b40 con 0x55603392c800 2023-09-23T03:21:35.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.301+0000 7fb99b2c4640 1 -- 172.21.15.154:0/2342064636 <== osd.6 v2:172.21.15.190:6816/1964294001 12 ==== osd_op_reply(28 [pgnls start_epoch 67 out=81b] v65'5 uv5 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5602efe4ed80 con 0x5602ef4ea400 2023-09-23T03:21:35.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.301+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:31 10.12 10:48000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x556033936380 con 0x55603382b400 2023-09-23T03:21:35.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.301+0000 7fe180b3a640 1 -- 172.21.15.154:0/3084437610 <== osd.7 v2:172.21.15.190:6824/3460136272 5 ==== osd_op_reply(18 [pgnls start_epoch 67 out=49b] v65'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563379d7ad80 con 0x563379f12000 2023-09-23T03:21:35.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.301+0000 7fe17bb30640 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x563379f12800 0x563379f1a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.301+0000 7fe17bb30640 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:19 9.2d 9:b4000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379f32000 con 0x563379f12800 2023-09-23T03:21:35.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.301+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:20 9.3f 9:feb1c803:::100000001fa.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6aed05180 con 0x55d6aecf6000 2023-09-23T03:21:35.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.301+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:29 10.0 10:05552577:::1000000020b.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef4e4000 con 0x5602ef4ea400 2023-09-23T03:21:35.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.301+0000 7ffa9f2c4640 1 -- 172.21.15.154:0/334556304 <== osd.2 v2:172.21.15.154:6822/2884407243 8 ==== osd_op_reply(20 100000001fa.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d6af710d80 con 0x55d6aecf6000 2023-09-23T03:21:35.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.301+0000 7ffaa0a525c0 10 datascan.operator(): handling object 1099511628282.0 2023-09-23T03:21:35.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.301+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:21 9.3f 9:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6aed04a80 con 0x55d6aecf6000 2023-09-23T03:21:35.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.302+0000 7fe180339640 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x563379f12800 0x563379f1a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.302+0000 7f62bd2bc640 1 -- 172.21.15.154:0/1011343899 <== osd.4 v2:172.21.15.190:6808/717302641 1 ==== osd_op_reply(18 [pgnls start_epoch 67 out=101b] v65'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x555b6453cfc0 con 0x555b646cf000 2023-09-23T03:21:35.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.302+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:19 10.1d 10:b9a363c8:::1000000020d.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b646d9c00 con 0x555b646cf000 2023-09-23T03:21:35.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.302+0000 7f9a4a564640 1 -- 172.21.15.154:0/3974369953 <== osd.6 v2:172.21.15.190:6816/1964294001 3 ==== osd_op_reply(31 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556033798b40 con 0x55603382b400 2023-09-23T03:21:35.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.302+0000 7fb99b2c4640 1 -- 172.21.15.154:0/2342064636 <== osd.6 v2:172.21.15.190:6816/1964294001 13 ==== osd_op_reply(29 1000000020b.00000000 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5602efe4ed80 con 0x5602ef4ea400 2023-09-23T03:21:35.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.302+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55603382b000 msgr2=0x556033529b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.302+0000 7f9a4d20c5c0 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55603382b000 0x556033529b80 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.302+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55603392c000 msgr2=0x556033932000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.302+0000 7f9a4d20c5c0 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55603392c000 0x556033932000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.302+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55603382ac00 msgr2=0x55603352a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.302+0000 7f9a4d20c5c0 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55603382ac00 0x55603352a100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.302+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55603392c800 msgr2=0x556033932b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.302+0000 7f9a4d20c5c0 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55603392c800 0x556033932b00 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.302+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55603392c400 msgr2=0x556033932580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.302+0000 7f9a4d20c5c0 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55603392c400 0x556033932580 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.302+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55603382b400 msgr2=0x556033529600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.302+0000 7f9a4d20c5c0 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55603382b400 0x556033529600 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.302+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55603392d000 msgr2=0x556033933600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.302+0000 7f9a4d20c5c0 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55603392d000 0x556033933600 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.302+0000 7f9a4ad65640 1 -- 172.21.15.154:0/3974369953 reap_dead start 2023-09-23T03:21:35.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.302+0000 7f9a4ad65640 1 -- 172.21.15.154:0/3974369953 reap_dead start 2023-09-23T03:21:35.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.302+0000 7f9a4ad65640 1 -- 172.21.15.154:0/3974369953 reap_dead start 2023-09-23T03:21:35.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.302+0000 7fb99ca525c0 10 datascan.operator(): handling object 1099511628299.0 2023-09-23T03:21:35.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.302+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:30 9.20 9:05552577:::1000000020b.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef4e5500 con 0x5602ef4ce400 2023-09-23T03:21:35.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.302+0000 7f48e126e640 1 -- 172.21.15.154:0/2963625397 <== osd.5 v2:172.21.15.190:6800/882536885 9 ==== osd_op_reply(31 100000001ff.00000000 [call,setxattr (8)] v67'18 uv18 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x556b099a6b40 con 0x556b09a40c00 2023-09-23T03:21:35.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.302+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:32 10.16 10:68000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x556b08b57c00 con 0x556b09a41000 2023-09-23T03:21:35.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.302+0000 7f62bd2bc640 1 -- 172.21.15.154:0/1011343899 <== osd.4 v2:172.21.15.190:6808/717302641 2 ==== osd_op_reply(19 1000000020d.00000001 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555b6453cfc0 con 0x555b646cf000 2023-09-23T03:21:35.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.302+0000 7f62bf24b5c0 10 datascan.operator(): handling object 1099511628301.1 2023-09-23T03:21:35.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.302+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:20 9.15 9:a8e69ea8:::1000000020d.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b646d8a80 con 0x555b646ce400 2023-09-23T03:21:35.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.302+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55603382a800 msgr2=0x55603352b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.302+0000 7f9a4d20c5c0 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55603382a800 0x55603352b700 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x5560337c2930 tx=0x55603429a120 comp rx=0 tx=0).stop 2023-09-23T03:21:35.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.302+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560337b7400 msgr2=0x55603352ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.302+0000 7f9a4d20c5c0 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560337b7400 0x55603352ac00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x5560337c2de0 tx=0x55603429a480 comp rx=0 tx=0).stop 2023-09-23T03:21:35.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.303+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 shutdown_connections 2023-09-23T03:21:35.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.303+0000 7efdc94ed640 1 -- 172.21.15.154:0/1781234207 <== osd.3 v2:172.21.15.154:6808/1133688468 3 ==== osd_op_reply(39 100000001f8.00000000 [call,setxattr (8)] v67'15 uv15 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x564662cc6fc0 con 0x564662178800 2023-09-23T03:21:35.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.303+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:40 10.11 10:8f3fbbd0::::head [pgnls start_epoch 67 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5646621c4380 con 0x564662179000 2023-09-23T03:21:35.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.303+0000 7f9a4d20c5c0 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55603392d000 0x556033933600 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.303+0000 7f9a4d20c5c0 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55603382a800 0x55603352b700 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.303+0000 7f9a4d20c5c0 1 --2- 172.21.15.154:0/3974369953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560337b7400 0x55603352ac00 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.303+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 >> 172.21.15.154:0/3974369953 conn(0x5560337b6c00 msgr2=0x556034237800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:35.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.303+0000 7fe180339640 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x563379f12800 0x563379f1a680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.303+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 shutdown_connections 2023-09-23T03:21:35.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.303+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/3974369953 wait complete. 2023-09-23T03:21:35.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.303+0000 7efdca4ef640 1 -- 172.21.15.154:0/1781234207 <== osd.5 v2:172.21.15.190:6800/882536885 4 ==== osd_op_reply(40 [pgnls start_epoch 67 out=81b] v65'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564662cc6d80 con 0x564662179000 2023-09-23T03:21:35.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.303+0000 7f48e1a6f640 1 -- 172.21.15.154:0/2963625397 <== osd.6 v2:172.21.15.190:6816/1964294001 9 ==== osd_op_reply(32 [pgnls start_epoch 67 out=101b] v65'5 uv5 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x556b099a6fc0 con 0x556b09a41000 2023-09-23T03:21:35.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.304+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:33 10.16 10:6bb58479:::10000000206.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x556b08e88700 con 0x556b09a41000 2023-09-23T03:21:35.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.304+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:41 10.11 10:8f3fbbd0:::1000000020c.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5646621c4700 con 0x564662179000 2023-09-23T03:21:35.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.304+0000 7fe180339640 1 -- 172.21.15.154:0/3084437610 <== osd.2 v2:172.21.15.154:6822/2884407243 1 ==== osd_op_reply(19 [pgnls start_epoch 67 out=49b] v65'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563379d7afc0 con 0x563379f12800 2023-09-23T03:21:35.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.304+0000 7fe181b915c0 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x56337a83a800 0x563379f19080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.304+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:20 10.9 10:92492492::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379f1c380 con 0x56337a83a800 2023-09-23T03:21:35.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.304+0000 7ffa9f2c4640 1 -- 172.21.15.154:0/334556304 <== osd.2 v2:172.21.15.154:6822/2884407243 9 ==== osd_op_reply(21 100000001fa.00000000 [call] v67'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d6af710d80 con 0x55d6aecf6000 2023-09-23T03:21:35.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.304+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:22 10.1b 10:db6db6db::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6aed05880 con 0x55d6aecee000 2023-09-23T03:21:35.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.304+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/2885832469 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560334bfc00 msgr2=0x556033529080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.304+0000 7f9a4d20c5c0 1 --2- 172.21.15.154:0/2885832469 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560334bfc00 0x556033529080 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x5560342cd470 tx=0x55603429a4e0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.304+0000 7f48e1a6f640 1 -- 172.21.15.154:0/2963625397 <== osd.6 v2:172.21.15.190:6816/1964294001 10 ==== osd_op_reply(33 10000000206.00000001 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x556b099a6fc0 con 0x556b09a41000 2023-09-23T03:21:35.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.304+0000 7f48e3e0c5c0 10 datascan.operator(): handling object 1099511628294.1 2023-09-23T03:21:35.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.304+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:34 9.0 9:012cdbc7:::10000000206.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x556b08e89500 con 0x556b08e7f000 2023-09-23T03:21:35.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.304+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/2885832469 shutdown_connections 2023-09-23T03:21:35.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.304+0000 7f9a4d20c5c0 1 --2- 172.21.15.154:0/2885832469 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5560334bfc00 0x556033529080 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.304+0000 7f9a4d20c5c0 1 --2- 172.21.15.154:0/2885832469 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5560334bf800 0x556033528b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.305+0000 7f9a4d20c5c0 1 --2- 172.21.15.154:0/2885832469 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5560334bec00 0x556033528580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.305+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/2885832469 >> 172.21.15.154:0/2885832469 conn(0x5560334bf000 msgr2=0x556034237000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:35.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.304+0000 7efdca4ef640 1 -- 172.21.15.154:0/1781234207 <== osd.5 v2:172.21.15.190:6800/882536885 5 ==== osd_op_reply(41 1000000020c.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564662cc6d80 con 0x564662179000 2023-09-23T03:21:35.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.305+0000 7efdcc0145c0 10 datascan.operator(): handling object 1099511628300.1 2023-09-23T03:21:35.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.305+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:42 9.1e 9:789b801b:::1000000020c.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x5646621c4a80 con 0x564662196000 2023-09-23T03:21:35.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.305+0000 7fe17fb38640 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x56337a83a800 0x563379f19080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.305+0000 7ffa9e2c2640 1 -- 172.21.15.154:0/334556304 <== osd.3 v2:172.21.15.154:6808/1133688468 3 ==== osd_op_reply(22 [pgnls start_epoch 67 out=81b] v65'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d6af710b40 con 0x55d6aecee000 2023-09-23T03:21:35.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.305+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/2885832469 shutdown_connections 2023-09-23T03:21:35.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.305+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:23 10.1b 10:deccbd37:::100000001fe.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6aed05180 con 0x55d6aecee000 2023-09-23T03:21:35.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.305+0000 7ffa9e2c2640 1 -- 172.21.15.154:0/334556304 <== osd.3 v2:172.21.15.154:6808/1133688468 4 ==== osd_op_reply(23 100000001fe.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d6af710b40 con 0x55d6aecee000 2023-09-23T03:21:35.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.305+0000 7ffaa0a525c0 10 datascan.operator(): handling object 1099511628286.0 2023-09-23T03:21:35.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.305+0000 7f9a4d20c5c0 1 -- 172.21.15.154:0/2885832469 wait complete. 2023-09-23T03:21:35.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.305+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:24 9.3b 9:deccbd37:::100000001fe.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6aed04e00 con 0x55d6aecf6000 2023-09-23T03:21:35.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.306+0000 7fe17fb38640 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x56337a83a800 0x563379f19080 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.306+0000 7f62bd2bc640 1 -- 172.21.15.154:0/1011343899 <== osd.7 v2:172.21.15.190:6824/3460136272 11 ==== osd_op_reply(20 1000000020d.00000000 [call,setxattr (8)] v67'9 uv9 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555b6453cfc0 con 0x555b646ce400 2023-09-23T03:21:35.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.306+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:21 10.1d 10:bbb18289::::head [pgnls start_epoch 67 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b646d9180 con 0x555b646cf000 2023-09-23T03:21:35.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.306+0000 7fb99b2c4640 1 -- 172.21.15.154:0/2342064636 <== osd.0 v2:172.21.15.154:6800/2920938138 5 ==== osd_op_reply(30 1000000020b.00000000 [call,setxattr (8)] v67'12 uv12 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5602efe4ed80 con 0x5602ef4ce400 2023-09-23T03:21:35.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.306+0000 7fe17fb38640 1 -- 172.21.15.154:0/3084437610 <== osd.5 v2:172.21.15.190:6800/882536885 1 ==== osd_op_reply(20 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563379d7ab40 con 0x56337a83a800 2023-09-23T03:21:35.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.306+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:31 10.10 10:08000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef4e5880 con 0x5602ef4ce800 2023-09-23T03:21:35.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.306+0000 7fe17bb30640 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:21 10.19 10:98000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379f32380 con 0x563379d9ec00 2023-09-23T03:21:35.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.307+0000 7fe180b3a640 1 -- 172.21.15.154:0/3084437610 <== osd.1 v2:172.21.15.154:6816/1267710005 2 ==== osd_op_reply(21 [pgnls start_epoch 67 out=101b] v65'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563379d7ad80 con 0x563379d9ec00 2023-09-23T03:21:35.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.307+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:22 10.19 10:992a6246:::10000000200.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379f1c700 con 0x563379d9ec00 2023-09-23T03:21:35.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.307+0000 7f62bd2bc640 1 -- 172.21.15.154:0/1011343899 <== osd.4 v2:172.21.15.190:6808/717302641 3 ==== osd_op_reply(21 [pgnls start_epoch 67 out=101b] v65'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x555b6453cfc0 con 0x555b646cf000 2023-09-23T03:21:35.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.307+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:22 10.1d 10:bbb18289:::10000000204.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b642bf880 con 0x555b646cf000 2023-09-23T03:21:35.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.307+0000 7fb99aac3640 1 -- 172.21.15.154:0/2342064636 <== osd.4 v2:172.21.15.190:6808/717302641 4 ==== osd_op_reply(31 [pgnls start_epoch 67 out=81b] v65'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5602efe4efc0 con 0x5602ef4ce800 2023-09-23T03:21:35.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.307+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:32 10.10 10:08b99fea:::100000001fd.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef4e5c00 con 0x5602ef4ce800 2023-09-23T03:21:35.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.307+0000 7efdc9cee640 1 -- 172.21.15.154:0/1781234207 <== osd.0 v2:172.21.15.154:6800/2920938138 10 ==== osd_op_reply(42 1000000020c.00000000 [call,setxattr (8)] v67'9 uv9 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x564662cc6b40 con 0x564662196000 2023-09-23T03:21:35.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.307+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:43 10.9 10:90000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5646621c4e00 con 0x564662179000 2023-09-23T03:21:35.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.307+0000 7fe180b3a640 1 -- 172.21.15.154:0/3084437610 <== osd.1 v2:172.21.15.154:6816/1267710005 3 ==== osd_op_reply(22 10000000200.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563379d7ad80 con 0x563379d9ec00 2023-09-23T03:21:35.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.307+0000 7fe181b915c0 10 datascan.operator(): handling object 1099511628288.0 2023-09-23T03:21:35.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.307+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:23 9.19 9:992a6246:::10000000200.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379f1ca80 con 0x563379f12400 2023-09-23T03:21:35.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.307+0000 7f48e2270640 1 -- 172.21.15.154:0/2963625397 <== osd.1 v2:172.21.15.154:6816/1267710005 2 ==== osd_op_reply(34 10000000206.00000000 [call,setxattr (8)] v67'12 uv12 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x556b099a6d80 con 0x556b08e7f000 2023-09-23T03:21:35.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.308+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x556b08e7ec00 msgr2=0x556b08e85080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.308+0000 7f48e3e0c5c0 1 --2- 172.21.15.154:0/2963625397 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x556b08e7ec00 0x556b08e85080 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.308+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x556b08e7f000 msgr2=0x556b08e85600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.308+0000 7f48e3e0c5c0 1 --2- 172.21.15.154:0/2963625397 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x556b08e7f000 0x556b08e85600 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.308+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x556b08e7e400 msgr2=0x556b08e84580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.308+0000 7f48e3e0c5c0 1 --2- 172.21.15.154:0/2963625397 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x556b08e7e400 0x556b08e84580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.308+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x556b08e7e000 msgr2=0x556b08e84000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.308+0000 7f48e3e0c5c0 1 --2- 172.21.15.154:0/2963625397 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x556b08e7e000 0x556b08e84000 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.308+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x556b09a40c00 msgr2=0x556b08b68100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.308+0000 7f48e3e0c5c0 1 --2- 172.21.15.154:0/2963625397 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x556b09a40c00 0x556b08b68100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.308+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x556b09a41000 msgr2=0x556b08b67b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.308+0000 7f48e3e0c5c0 1 --2- 172.21.15.154:0/2963625397 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x556b09a41000 0x556b08b67b80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.308+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x556b09a40800 msgr2=0x556b08b69700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.308+0000 7f48e3e0c5c0 1 --2- 172.21.15.154:0/2963625397 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x556b09a40800 0x556b08b69700 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x556b099dc5d0 tx=0x556b099be0c0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.308+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b099db000 msgr2=0x556b08b69180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.308+0000 7f48e3e0c5c0 1 --2- 172.21.15.154:0/2963625397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b099db000 0x556b08b69180 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x556b099dd500 tx=0x556b099be1e0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.308+0000 7f48e126e640 1 -- 172.21.15.154:0/2963625397 reap_dead start 2023-09-23T03:21:35.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.308+0000 7f48e126e640 1 -- 172.21.15.154:0/2963625397 reap_dead start 2023-09-23T03:21:35.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.308+0000 7efdca4ef640 1 -- 172.21.15.154:0/1781234207 <== osd.5 v2:172.21.15.190:6800/882536885 6 ==== osd_op_reply(43 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564662cc6d80 con 0x564662179000 2023-09-23T03:21:35.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.308+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x564662196000 msgr2=0x56466219c000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.308+0000 7efdcc0145c0 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x564662196000 0x56466219c000 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.308+0000 7f48e126e640 1 -- 172.21.15.154:0/2963625397 reap_dead start 2023-09-23T03:21:35.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.308+0000 7f48e126e640 1 -- 172.21.15.154:0/2963625397 reap_dead start 2023-09-23T03:21:35.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.308+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 shutdown_connections 2023-09-23T03:21:35.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.308+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 >> 172.21.15.154:0/2963625397 conn(0x556b099dac00 msgr2=0x556b09874800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:35.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.308+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x564662178000 msgr2=0x564661e82100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.308+0000 7efdcc0145c0 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x564662178000 0x564661e82100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.308+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x564662196400 msgr2=0x56466219c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.308+0000 7efdcc0145c0 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x564662196400 0x56466219c580 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.308+0000 7ffa9f2c4640 1 -- 172.21.15.154:0/334556304 <== osd.2 v2:172.21.15.154:6822/2884407243 10 ==== osd_op_reply(24 100000001fe.00000000 [call,setxattr (8)] v67'7 uv7 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55d6af710d80 con 0x55d6aecf6000 2023-09-23T03:21:35.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.308+0000 7efdc94ed640 1 -- 172.21.15.154:0/1781234207 reap_dead start 2023-09-23T03:21:35.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.308+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x564662178800 msgr2=0x564661e81600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.308+0000 7efdcc0145c0 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x564662178800 0x564661e81600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.308+0000 7ffaa0a525c0 1 --2- 172.21.15.154:0/334556304 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55d6aeceec00 0x55d6aecf9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.308+0000 7fb99aac3640 1 -- 172.21.15.154:0/2342064636 <== osd.4 v2:172.21.15.190:6808/717302641 5 ==== osd_op_reply(32 100000001fd.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5602efe4efc0 con 0x5602ef4ce800 2023-09-23T03:21:35.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.309+0000 7fb99ca525c0 10 datascan.operator(): handling object 1099511628285.0 2023-09-23T03:21:35.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.309+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x564662196800 msgr2=0x56466219cb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.309+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:33 9.10 9:08b99fea:::100000001fd.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef4e4a80 con 0x5602ef4ce800 2023-09-23T03:21:35.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.309+0000 7efdcc0145c0 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x564662196800 0x56466219cb00 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.309+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x564662179000 msgr2=0x56466219db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.309+0000 7efdcc0145c0 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x564662179000 0x56466219db80 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.309+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x564662178c00 msgr2=0x56466219d600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.309+0000 7efdcc0145c0 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x564662178c00 0x56466219d600 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.309+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x564662178400 msgr2=0x564661e81b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.309+0000 7efdcc0145c0 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x564662178400 0x564661e81b80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.309+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 shutdown_connections 2023-09-23T03:21:35.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.309+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/2963625397 wait complete. 2023-09-23T03:21:35.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.309+0000 7f62bd2bc640 1 -- 172.21.15.154:0/1011343899 <== osd.4 v2:172.21.15.190:6808/717302641 4 ==== osd_op_reply(22 10000000204.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555b6453cfc0 con 0x555b646cf000 2023-09-23T03:21:35.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.309+0000 7f62bf24b5c0 10 datascan.operator(): handling object 1099511628292.1 2023-09-23T03:21:35.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.309+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:25 10.7 10:e0000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6aed05500 con 0x55d6aeceec00 2023-09-23T03:21:35.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.309+0000 7efdc94ed640 1 -- 172.21.15.154:0/1781234207 reap_dead start 2023-09-23T03:21:35.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.309+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x564662ce9c00 msgr2=0x564661e83700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.309+0000 7efdcc0145c0 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x564662ce9c00 0x564661e83700 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x564661f08810 tx=0x564662be7f20 comp rx=0 tx=0).stop 2023-09-23T03:21:35.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.309+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564662ce9800 msgr2=0x564661e82680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.309+0000 7efdcc0145c0 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564662ce9800 0x564661e82680 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x564662c20a80 tx=0x564662cd0120 comp rx=0 tx=0).stop 2023-09-23T03:21:35.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.309+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:23 9.5 9:a0beda95:::10000000204.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b642bfc00 con 0x555b646ce000 2023-09-23T03:21:35.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.309+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 shutdown_connections 2023-09-23T03:21:35.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.309+0000 7efdcc0145c0 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x564662ce9c00 0x564661e83700 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.309+0000 7efdcc0145c0 1 --2- 172.21.15.154:0/1781234207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564662ce9800 0x564661e82680 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.309+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 >> 172.21.15.154:0/1781234207 conn(0x564662ce8c00 msgr2=0x564662b91000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:35.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.310+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/1067950898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b08afcc00 msgr2=0x556b08b66580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.310+0000 7f48e3e0c5c0 1 --2- 172.21.15.154:0/1067950898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b08afcc00 0x556b08b66580 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x556b09905410 tx=0x556b099be240 comp rx=0 tx=0).stop 2023-09-23T03:21:35.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.310+0000 7ffa9e2c2640 1 --2- 172.21.15.154:0/334556304 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55d6aeceec00 0x55d6aecf9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.310+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 shutdown_connections 2023-09-23T03:21:35.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.310+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1781234207 wait complete. 2023-09-23T03:21:35.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.310+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/1067950898 shutdown_connections 2023-09-23T03:21:35.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.310+0000 7f48e3e0c5c0 1 --2- 172.21.15.154:0/1067950898 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556b08afdc00 0x556b08b67080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.310+0000 7f48e3e0c5c0 1 --2- 172.21.15.154:0/1067950898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b08afd800 0x556b08b66b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.310+0000 7f48e3e0c5c0 1 --2- 172.21.15.154:0/1067950898 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556b08afcc00 0x556b08b66580 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.310+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/1067950898 >> 172.21.15.154:0/1067950898 conn(0x556b08afd000 msgr2=0x556b09875000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:35.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.310+0000 7fe180339640 1 -- 172.21.15.154:0/3084437610 <== osd.3 v2:172.21.15.154:6808/1133688468 7 ==== osd_op_reply(23 10000000200.00000000 [call,setxattr (8)] v67'16 uv16 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563379d7afc0 con 0x563379f12400 2023-09-23T03:21:35.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.310+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:24 10.19 10:997b7129::::head [pgnls start_epoch 67 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379afd880 con 0x563379d9ec00 2023-09-23T03:21:35.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.310+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/1067950898 shutdown_connections 2023-09-23T03:21:35.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.310+0000 7f48e3e0c5c0 1 -- 172.21.15.154:0/1067950898 wait complete. 2023-09-23T03:21:35.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.310+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1163186475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564661e16c00 msgr2=0x564661e80580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.311+0000 7efdcc0145c0 1 --2- 172.21.15.154:0/1163186475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564661e16c00 0x564661e80580 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x564662c21470 tx=0x564662cd0420 comp rx=0 tx=0).stop 2023-09-23T03:21:35.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.311+0000 7ffa9e2c2640 1 --2- 172.21.15.154:0/334556304 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55d6aeceec00 0x55d6aecf9600 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.311+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1163186475 shutdown_connections 2023-09-23T03:21:35.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.311+0000 7efdcc0145c0 1 --2- 172.21.15.154:0/1163186475 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564661e17c00 0x564661e81080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.311+0000 7efdcc0145c0 1 --2- 172.21.15.154:0/1163186475 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564661e17800 0x564661e80b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.311+0000 7efdcc0145c0 1 --2- 172.21.15.154:0/1163186475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564661e16c00 0x564661e80580 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.311+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1163186475 >> 172.21.15.154:0/1163186475 conn(0x564661e17000 msgr2=0x564662b90800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:35.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.311+0000 7fe180b3a640 1 -- 172.21.15.154:0/3084437610 <== osd.1 v2:172.21.15.154:6816/1267710005 4 ==== osd_op_reply(24 [pgnls start_epoch 67 out=81b] v65'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563379d7ad80 con 0x563379d9ec00 2023-09-23T03:21:35.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.311+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:25 10.19 10:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379f32380 con 0x563379d9ec00 2023-09-23T03:21:35.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.311+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1163186475 shutdown_connections 2023-09-23T03:21:35.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.311+0000 7efdcc0145c0 1 -- 172.21.15.154:0/1163186475 wait complete. 2023-09-23T03:21:35.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.311+0000 7fe180b3a640 1 -- 172.21.15.154:0/3084437610 <== osd.1 v2:172.21.15.154:6816/1267710005 5 ==== osd_op_reply(25 100000001f8.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563379d7ad80 con 0x563379d9ec00 2023-09-23T03:21:35.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.311+0000 7fe181b915c0 10 datascan.operator(): handling object 1099511628280.0 2023-09-23T03:21:35.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.311+0000 7ffa9e2c2640 1 -- 172.21.15.154:0/334556304 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== osd_op_reply(25 [pgnls start_epoch 67 out=101b] v65'7 uv7 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d6af710b40 con 0x55d6aeceec00 2023-09-23T03:21:35.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.311+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:26 10.7 10:e15a914f:::10000000207.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6ae8cbc00 con 0x55d6aeceec00 2023-09-23T03:21:35.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.311+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:26 9.19 9:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379f32700 con 0x563379f12400 2023-09-23T03:21:35.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.312+0000 7fb99aac3640 1 -- 172.21.15.154:0/2342064636 <== osd.4 v2:172.21.15.190:6808/717302641 6 ==== osd_op_reply(33 100000001fd.00000000 [call,setxattr (8)] v67'10 uv10 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5602efe4efc0 con 0x5602ef4ce800 2023-09-23T03:21:35.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.312+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:34 10.8 10:10000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef4e4380 con 0x5602ef4ce400 2023-09-23T03:21:35.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.312+0000 7f62bdabd640 1 -- 172.21.15.154:0/1011343899 <== osd.0 v2:172.21.15.154:6800/2920938138 3 ==== osd_op_reply(23 10000000204.00000000 [call,setxattr (8)] v67'8 uv8 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555b6453cd80 con 0x555b646ce000 2023-09-23T03:21:35.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.312+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:24 10.1d 10:bfcd3380::::head [pgnls start_epoch 67 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b646d9880 con 0x555b646cf000 2023-09-23T03:21:35.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.312+0000 7fb99b2c4640 1 -- 172.21.15.154:0/2342064636 <== osd.0 v2:172.21.15.154:6800/2920938138 6 ==== osd_op_reply(34 [pgnls start_epoch 67 out=101b] v65'6 uv6 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5602efe4ed80 con 0x5602ef4ce400 2023-09-23T03:21:35.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.312+0000 7ffa9e2c2640 1 -- 172.21.15.154:0/334556304 <== osd.6 v2:172.21.15.190:6816/1964294001 2 ==== osd_op_reply(26 10000000207.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d6af710b40 con 0x55d6aeceec00 2023-09-23T03:21:35.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.312+0000 7ffaa0a525c0 10 datascan.operator(): handling object 1099511628295.0 2023-09-23T03:21:35.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.312+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:35 10.8 10:101c2845:::100000001fe.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef4e4e00 con 0x5602ef4ce400 2023-09-23T03:21:35.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.312+0000 7f62bd2bc640 1 -- 172.21.15.154:0/1011343899 <== osd.4 v2:172.21.15.190:6808/717302641 5 ==== osd_op_reply(24 [pgnls start_epoch 67 out=81b] v65'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555b6453cfc0 con 0x555b646cf000 2023-09-23T03:21:35.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.312+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:27 9.7 9:e15a914f:::10000000207.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6ae8cb880 con 0x55d6aecee400 2023-09-23T03:21:35.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.312+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:25 10.1d 10:bfcd3380:::10000000205.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b646d8e00 con 0x555b646cf000 2023-09-23T03:21:35.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.313+0000 7fb99b2c4640 1 -- 172.21.15.154:0/2342064636 <== osd.0 v2:172.21.15.154:6800/2920938138 7 ==== osd_op_reply(35 100000001fe.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5602efe4ed80 con 0x5602ef4ce400 2023-09-23T03:21:35.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.313+0000 7fb99ca525c0 10 datascan.operator(): handling object 1099511628286.1 2023-09-23T03:21:35.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.313+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:36 9.3b 9:deccbd37:::100000001fe.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef4e5880 con 0x5602ef4ea800 2023-09-23T03:21:35.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.313+0000 7f62bd2bc640 1 -- 172.21.15.154:0/1011343899 <== osd.4 v2:172.21.15.190:6808/717302641 6 ==== osd_op_reply(25 10000000205.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555b6453cfc0 con 0x555b646cf000 2023-09-23T03:21:35.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.313+0000 7f62bf24b5c0 10 datascan.operator(): handling object 1099511628293.0 2023-09-23T03:21:35.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.314+0000 7fe180339640 1 -- 172.21.15.154:0/3084437610 <== osd.3 v2:172.21.15.154:6808/1133688468 8 ==== osd_op_reply(26 100000001f8.00000000 [call,setxattr (8)] v67'17 uv17 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563379d7afc0 con 0x563379f12400 2023-09-23T03:21:35.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.314+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:26 9.3d 9:bfcd3380:::10000000205.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b646d9500 con 0x555b646ce400 2023-09-23T03:21:35.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.314+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:27 10.5 10:a0000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379f32a80 con 0x563379f12800 2023-09-23T03:21:35.486 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', '--worker_n', '1', '--worker_m', '7'] in time 0:00:00.418734, result: 2023-09-23T03:21:35.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.315+0000 7fe180339640 1 -- 172.21.15.154:0/3084437610 <== osd.2 v2:172.21.15.154:6822/2884407243 2 ==== osd_op_reply(27 [pgnls start_epoch 67 out=81b] v65'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563379d7afc0 con 0x563379f12800 2023-09-23T03:21:35.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.315+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:28 10.5 10:a0beda95:::10000000204.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379f32e00 con 0x563379f12800 2023-09-23T03:21:35.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.315+0000 7fe180339640 1 -- 172.21.15.154:0/3084437610 <== osd.2 v2:172.21.15.154:6822/2884407243 3 ==== osd_op_reply(28 10000000204.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563379d7afc0 con 0x563379f12800 2023-09-23T03:21:35.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.315+0000 7ffa9eac3640 1 -- 172.21.15.154:0/334556304 <== osd.5 v2:172.21.15.190:6800/882536885 8 ==== osd_op_reply(27 10000000207.00000000 [call,setxattr (8)] v67'11 uv11 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55d6af710fc0 con 0x55d6aecee400 2023-09-23T03:21:35.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.315+0000 7fe181b915c0 10 datascan.operator(): handling object 1099511628292.0 2023-09-23T03:21:35.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.315+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:28 10.7 10:e2e07930::::head [pgnls start_epoch 67 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6aed04000 con 0x55d6aeceec00 2023-09-23T03:21:35.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.315+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:29 9.5 9:a0beda95:::10000000204.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379f1c000 con 0x563379d9f400 2023-09-23T03:21:35.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.316+0000 7ffa9e2c2640 1 -- 172.21.15.154:0/334556304 <== osd.6 v2:172.21.15.190:6816/1964294001 3 ==== osd_op_reply(28 [pgnls start_epoch 67 out=101b] v65'7 uv7 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d6af710b40 con 0x55d6aeceec00 2023-09-23T03:21:35.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.316+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:29 10.7 10:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6aed04380 con 0x55d6aeceec00 2023-09-23T03:21:35.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.316+0000 7fb99aac3640 1 -- 172.21.15.154:0/2342064636 <== osd.2 v2:172.21.15.154:6822/2884407243 2 ==== osd_op_reply(36 100000001fe.00000000 [call,setxattr (8)] v67'8 uv8 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5602efe4efc0 con 0x5602ef4ea800 2023-09-23T03:21:35.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.317+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:37 10.8 10:1247e94b::::head [pgnls start_epoch 67 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef4e5180 con 0x5602ef4ce400 2023-09-23T03:21:35.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.317+0000 7ffa9e2c2640 1 -- 172.21.15.154:0/334556304 <== osd.6 v2:172.21.15.190:6816/1964294001 4 ==== osd_op_reply(29 10000000001.00000000 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d6af710b40 con 0x55d6aeceec00 2023-09-23T03:21:35.490 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', '--worker_n', '2', '--worker_m', '7'] in time 0:00:00.421792, result: 2023-09-23T03:21:35.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.317+0000 7f62bd2bc640 1 -- 172.21.15.154:0/1011343899 <== osd.7 v2:172.21.15.190:6824/3460136272 12 ==== osd_op_reply(26 10000000205.00000000 [call,setxattr (8)] v67'10 uv10 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555b6453cfc0 con 0x555b646ce400 2023-09-23T03:21:35.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.317+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:27 10.3 10:c0000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b646d8700 con 0x555b645bac00 2023-09-23T03:21:35.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.317+0000 7ffaa0a525c0 10 datascan.operator(): handling object 1099511627777.0 2023-09-23T03:21:35.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.317+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:30 9.7 9:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6aed04700 con 0x55d6aecee400 2023-09-23T03:21:35.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.317+0000 7fe17fb38640 1 -- 172.21.15.154:0/3084437610 <== osd.0 v2:172.21.15.154:6800/2920938138 6 ==== osd_op_reply(29 10000000204.00000000 [call,setxattr (8)] v67'9 uv9 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563379d7ab40 con 0x563379d9f400 2023-09-23T03:21:35.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.317+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:30 10.15 10:a8000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379f1d180 con 0x563379f12400 2023-09-23T03:21:35.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.318+0000 7f62bcabb640 1 -- 172.21.15.154:0/1011343899 <== osd.2 v2:172.21.15.154:6822/2884407243 2 ==== osd_op_reply(27 [pgnls start_epoch 67 out=81b] v65'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555b6453cb40 con 0x555b645bac00 2023-09-23T03:21:35.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.318+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:28 10.3 10:c1028715:::10000000202.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b646d9c00 con 0x555b645bac00 2023-09-23T03:21:35.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.318+0000 7fb99b2c4640 1 -- 172.21.15.154:0/2342064636 <== osd.0 v2:172.21.15.154:6800/2920938138 8 ==== osd_op_reply(37 [pgnls start_epoch 67 out=101b] v65'6 uv6 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5602efe4ed80 con 0x5602ef4ce400 2023-09-23T03:21:35.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.318+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:38 10.8 10:1247e94b:::100000001f9.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef4e4000 con 0x5602ef4ce400 2023-09-23T03:21:35.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.318+0000 7fe180339640 1 -- 172.21.15.154:0/3084437610 <== osd.3 v2:172.21.15.154:6808/1133688468 9 ==== osd_op_reply(30 [pgnls start_epoch 67 out=101b] v65'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563379d7afc0 con 0x563379f12400 2023-09-23T03:21:35.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.318+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:31 10.15 10:a8e69ea8:::1000000020d.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379f1d500 con 0x563379f12400 2023-09-23T03:21:35.493 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', '--worker_n', '3', '--worker_m', '7'] in time 0:00:00.424647, result: 2023-09-23T03:21:35.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.318+0000 7f62bcabb640 1 -- 172.21.15.154:0/1011343899 <== osd.2 v2:172.21.15.154:6822/2884407243 3 ==== osd_op_reply(28 10000000202.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555b6453cb40 con 0x555b645bac00 2023-09-23T03:21:35.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.318+0000 7fb99b2c4640 1 -- 172.21.15.154:0/2342064636 <== osd.0 v2:172.21.15.154:6800/2920938138 9 ==== osd_op_reply(38 100000001f9.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5602efe4ed80 con 0x5602ef4ce400 2023-09-23T03:21:35.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.318+0000 7fb99ca525c0 10 datascan.operator(): handling object 1099511628281.1 2023-09-23T03:21:35.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.318+0000 7f62bf24b5c0 10 datascan.operator(): handling object 1099511628290.1 2023-09-23T03:21:35.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.318+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:39 9.36 9:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef4e5500 con 0x5602ef4ce000 2023-09-23T03:21:35.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.318+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:29 9.28 9:15246602:::10000000202.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b646d8a80 con 0x555b645bb000 2023-09-23T03:21:35.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.319+0000 7fe180339640 1 -- 172.21.15.154:0/3084437610 <== osd.3 v2:172.21.15.154:6808/1133688468 10 ==== osd_op_reply(31 1000000020d.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563379d7afc0 con 0x563379f12400 2023-09-23T03:21:35.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.319+0000 7fe181b915c0 10 datascan.operator(): handling object 1099511628301.0 2023-09-23T03:21:35.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.319+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:32 9.15 9:a8e69ea8:::1000000020d.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379f32000 con 0x563379f12000 2023-09-23T03:21:35.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.319+0000 7ffa9eac3640 1 -- 172.21.15.154:0/334556304 <== osd.5 v2:172.21.15.190:6800/882536885 9 ==== osd_op_reply(30 10000000001.00000000 [call,setxattr (8)] v67'12 uv12 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55d6af710fc0 con 0x55d6aecee400 2023-09-23T03:21:35.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.319+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:31 10.7 10:e5993619::::head [pgnls start_epoch 67 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6aed05880 con 0x55d6aeceec00 2023-09-23T03:21:35.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.320+0000 7ffa9e2c2640 1 -- 172.21.15.154:0/334556304 <== osd.6 v2:172.21.15.190:6816/1964294001 5 ==== osd_op_reply(31 [pgnls start_epoch 67 out=81b] v65'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d6af710b40 con 0x55d6aeceec00 2023-09-23T03:21:35.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.320+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:32 10.7 10:e5993619:::10000000208.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6aed05180 con 0x55d6aeceec00 2023-09-23T03:21:35.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.320+0000 7ffa9e2c2640 1 -- 172.21.15.154:0/334556304 <== osd.6 v2:172.21.15.190:6816/1964294001 6 ==== osd_op_reply(32 10000000208.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d6af710b40 con 0x55d6aeceec00 2023-09-23T03:21:35.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.320+0000 7ffaa0a525c0 10 datascan.operator(): handling object 1099511628296.0 2023-09-23T03:21:35.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.320+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:33 9.27 9:e5993619:::10000000208.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6aed05500 con 0x55d6aecf6000 2023-09-23T03:21:35.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.320+0000 7f62bdabd640 1 -- 172.21.15.154:0/1011343899 <== osd.6 v2:172.21.15.190:6816/1964294001 3 ==== osd_op_reply(29 10000000202.00000000 [call,setxattr (8)] v67'12 uv12 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555b6453cd80 con 0x555b645bb000 2023-09-23T03:21:35.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.320+0000 7f62bf24b5c0 1 --2- 172.21.15.154:0/1011343899 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x555b645bb400 0x555b642cf600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.321+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:30 10.13 10:c8000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b642bfc00 con 0x555b645bb400 2023-09-23T03:21:35.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.321+0000 7f62bcabb640 1 --2- 172.21.15.154:0/1011343899 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x555b645bb400 0x555b642cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.321+0000 7fe180b3a640 1 -- 172.21.15.154:0/3084437610 <== osd.7 v2:172.21.15.190:6824/3460136272 6 ==== osd_op_reply(32 1000000020d.00000000 [call,setxattr (8)] v67'10 uv10 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563379d7ad80 con 0x563379f12000 2023-09-23T03:21:35.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.321+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:33 10.15 10:ab3d427d::::head [pgnls start_epoch 67 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379f32700 con 0x563379f12400 2023-09-23T03:21:35.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.322+0000 7ffa9f2c4640 1 -- 172.21.15.154:0/334556304 <== osd.2 v2:172.21.15.154:6822/2884407243 11 ==== osd_op_reply(33 10000000208.00000000 [call,setxattr (8)] v67'5 uv5 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55d6af710d80 con 0x55d6aecf6000 2023-09-23T03:21:35.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.322+0000 7fe180339640 1 -- 172.21.15.154:0/3084437610 <== osd.3 v2:172.21.15.154:6808/1133688468 11 ==== osd_op_reply(33 [pgnls start_epoch 67 out=101b] v65'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563379d7afc0 con 0x563379f12400 2023-09-23T03:21:35.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.322+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:34 10.15 10:ab3d427d:::10000000203.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379f33180 con 0x563379f12400 2023-09-23T03:21:35.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.322+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:34 10.17 10:e8000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6ae8cb880 con 0x55d6aecf6400 2023-09-23T03:21:35.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.322+0000 7f62bcabb640 1 --2- 172.21.15.154:0/1011343899 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x555b645bb400 0x555b642cf600 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.322+0000 7fe180339640 1 -- 172.21.15.154:0/3084437610 <== osd.3 v2:172.21.15.154:6808/1133688468 12 ==== osd_op_reply(34 10000000203.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563379d7afc0 con 0x563379f12400 2023-09-23T03:21:35.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.322+0000 7fe181b915c0 10 datascan.operator(): handling object 1099511628291.1 2023-09-23T03:21:35.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.322+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:35 9.2b 9:d4a58307:::10000000203.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379f33500 con 0x563379f12000 2023-09-23T03:21:35.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.322+0000 7f62bcabb640 1 -- 172.21.15.154:0/1011343899 <== osd.1 v2:172.21.15.154:6816/1267710005 1 ==== osd_op_reply(30 [pgnls start_epoch 67 out=101b] v65'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x555b6453cb40 con 0x555b645bb400 2023-09-23T03:21:35.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.322+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:31 10.13 10:ca8fdbba:::100000001fb.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b646d9180 con 0x555b645bb400 2023-09-23T03:21:35.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.323+0000 7ffa9f2c4640 1 -- 172.21.15.154:0/334556304 <== osd.4 v2:172.21.15.190:6808/717302641 2 ==== osd_op_reply(34 [pgnls start_epoch 67 out=81b] v65'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d6af710d80 con 0x55d6aecf6400 2023-09-23T03:21:35.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.323+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:35 10.17 10:e88218fc:::10000000209.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6ae8cbc00 con 0x55d6aecf6400 2023-09-23T03:21:35.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.323+0000 7f62bcabb640 1 -- 172.21.15.154:0/1011343899 <== osd.1 v2:172.21.15.154:6816/1267710005 2 ==== osd_op_reply(31 100000001fb.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555b6453cb40 con 0x555b645bb400 2023-09-23T03:21:35.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.323+0000 7f62bf24b5c0 10 datascan.operator(): handling object 1099511628283.0 2023-09-23T03:21:35.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.323+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:32 9.13 9:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b646d8700 con 0x555b646ce400 2023-09-23T03:21:35.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.324+0000 7ffa9f2c4640 1 -- 172.21.15.154:0/334556304 <== osd.4 v2:172.21.15.190:6808/717302641 3 ==== osd_op_reply(35 10000000209.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d6af710d80 con 0x55d6aecf6400 2023-09-23T03:21:35.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.324+0000 7ffaa0a525c0 10 datascan.operator(): handling object 1099511628297.1 2023-09-23T03:21:35.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.324+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:36 9.2c 9:37389ade:::10000000209.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6aed04a80 con 0x55d6aecee000 2023-09-23T03:21:35.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.324+0000 7fb99a2c2640 1 -- 172.21.15.154:0/2342064636 <== osd.1 v2:172.21.15.154:6816/1267710005 6 ==== osd_op_reply(39 100000001f9.00000000 [call,setxattr (8)] v67'23 uv23 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5602efe4eb40 con 0x5602ef4ce000 2023-09-23T03:21:35.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.324+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:40 10.8 10:15246602::::head [pgnls start_epoch 67 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef4e4380 con 0x5602ef4ce400 2023-09-23T03:21:35.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.324+0000 7fb99b2c4640 1 -- 172.21.15.154:0/2342064636 <== osd.0 v2:172.21.15.154:6800/2920938138 10 ==== osd_op_reply(40 [pgnls start_epoch 67 out=101b] v65'6 uv6 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5602efe4ed80 con 0x5602ef4ce400 2023-09-23T03:21:35.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.324+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:41 10.8 10:15246602:::10000000202.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef4e4e00 con 0x5602ef4ce400 2023-09-23T03:21:35.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.325+0000 7fb99b2c4640 1 -- 172.21.15.154:0/2342064636 <== osd.0 v2:172.21.15.154:6800/2920938138 11 ==== osd_op_reply(41 10000000202.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5602efe4ed80 con 0x5602ef4ce400 2023-09-23T03:21:35.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.325+0000 7fb99ca525c0 10 datascan.operator(): handling object 1099511628290.0 2023-09-23T03:21:35.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.325+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:42 9.28 9:15246602:::10000000202.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef0d5c00 con 0x5602ef4ea400 2023-09-23T03:21:35.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.325+0000 7ffa9e2c2640 1 -- 172.21.15.154:0/334556304 <== osd.3 v2:172.21.15.154:6808/1133688468 5 ==== osd_op_reply(36 10000000209.00000000 [call,setxattr (8)] v67'12 uv12 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55d6af710b40 con 0x55d6aecee000 2023-09-23T03:21:35.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.325+0000 7fe180b3a640 1 -- 172.21.15.154:0/3084437610 <== osd.7 v2:172.21.15.190:6824/3460136272 7 ==== osd_op_reply(35 10000000203.00000000 [call,setxattr (8)] v67'14 uv14 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563379d7ad80 con 0x563379f12000 2023-09-23T03:21:35.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.325+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:36 10.15 10:ab9be7ec::::head [pgnls start_epoch 67 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379f33880 con 0x563379f12400 2023-09-23T03:21:35.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.325+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:37 10.f 10:f0000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6aed04e00 con 0x55d6aeceec00 2023-09-23T03:21:35.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.326+0000 7fe180339640 1 -- 172.21.15.154:0/3084437610 <== osd.3 v2:172.21.15.154:6808/1133688468 13 ==== osd_op_reply(36 [pgnls start_epoch 67 out=81b] v65'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563379d7afc0 con 0x563379f12400 2023-09-23T03:21:35.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.326+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:37 10.15 10:ab9be7ec:::10000000205.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379f33c00 con 0x563379f12400 2023-09-23T03:21:35.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.326+0000 7ffa9e2c2640 1 -- 172.21.15.154:0/334556304 <== osd.6 v2:172.21.15.190:6816/1964294001 7 ==== osd_op_reply(37 [pgnls start_epoch 67 out=81b] v65'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d6af710b40 con 0x55d6aeceec00 2023-09-23T03:21:35.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.326+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:38 10.f 10:f526278d:::10000000001.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6aed05500 con 0x55d6aeceec00 2023-09-23T03:21:35.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.326+0000 7fe180339640 1 -- 172.21.15.154:0/3084437610 <== osd.3 v2:172.21.15.154:6808/1133688468 14 ==== osd_op_reply(37 10000000205.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563379d7afc0 con 0x563379f12400 2023-09-23T03:21:35.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.326+0000 7fe181b915c0 10 datascan.operator(): handling object 1099511628293.1 2023-09-23T03:21:35.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.326+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:38 9.3d 9:bfcd3380:::10000000205.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379f1ca80 con 0x563379f12000 2023-09-23T03:21:35.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.326+0000 7f62bd2bc640 1 -- 172.21.15.154:0/1011343899 <== osd.7 v2:172.21.15.190:6824/3460136272 13 ==== osd_op_reply(32 100000001fb.00000000 [call,setxattr (8)] v67'19 uv19 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555b6453cfc0 con 0x555b646ce400 2023-09-23T03:21:35.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.326+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:33 10.13 10:cb24bee2::::head [pgnls start_epoch 67 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b646d9c00 con 0x555b645bb400 2023-09-23T03:21:35.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.327+0000 7ffa9e2c2640 1 -- 172.21.15.154:0/334556304 <== osd.6 v2:172.21.15.190:6816/1964294001 8 ==== osd_op_reply(38 10000000001.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d6af710b40 con 0x55d6aeceec00 2023-09-23T03:21:35.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.327+0000 7f62bcabb640 1 -- 172.21.15.154:0/1011343899 <== osd.1 v2:172.21.15.154:6816/1267710005 3 ==== osd_op_reply(33 [pgnls start_epoch 67 out=101b] v65'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x555b6453cb40 con 0x555b645bb400 2023-09-23T03:21:35.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.327+0000 7ffaa0a525c0 10 datascan.operator(): handling object 1099511627777.1 2023-09-23T03:21:35.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.327+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:34 10.13 10:cb24bee2:::100000001fa.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b646d9180 con 0x555b645bb400 2023-09-23T03:21:35.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.327+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:39 9.7 9:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6aed05c00 con 0x55d6aecee400 2023-09-23T03:21:35.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.327+0000 7f62bcabb640 1 -- 172.21.15.154:0/1011343899 <== osd.1 v2:172.21.15.154:6816/1267710005 4 ==== osd_op_reply(34 100000001fa.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555b6453cb40 con 0x555b645bb400 2023-09-23T03:21:35.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.327+0000 7f62bf24b5c0 10 datascan.operator(): handling object 1099511628282.1 2023-09-23T03:21:35.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.327+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:35 9.3f 9:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b642bfc00 con 0x555b645bac00 2023-09-23T03:21:35.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.329+0000 7f62bcabb640 1 -- 172.21.15.154:0/1011343899 <== osd.2 v2:172.21.15.154:6822/2884407243 4 ==== osd_op_reply(35 100000001fa.00000000 [call,setxattr (8)] v67'11 uv11 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555b6453cb40 con 0x555b645bac00 2023-09-23T03:21:35.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.329+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:36 10.13 10:cc6bf2c4::::head [pgnls start_epoch 67 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b645a4000 con 0x555b645bb400 2023-09-23T03:21:35.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.329+0000 7f62bcabb640 1 -- 172.21.15.154:0/1011343899 <== osd.1 v2:172.21.15.154:6816/1267710005 5 ==== osd_op_reply(36 [pgnls start_epoch 67 out=81b] v65'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555b6453cb40 con 0x555b645bb400 2023-09-23T03:21:35.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.329+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:37 10.13 10:cc6bf2c4:::1000000020b.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b645a4380 con 0x555b645bb400 2023-09-23T03:21:35.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.329+0000 7fb99b2c4640 1 -- 172.21.15.154:0/2342064636 <== osd.6 v2:172.21.15.190:6816/1964294001 14 ==== osd_op_reply(42 10000000202.00000000 [call,setxattr (8)] v67'13 uv13 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5602efe4ed80 con 0x5602ef4ea400 2023-09-23T03:21:35.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.329+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:43 10.8 10:177a762d::::head [pgnls start_epoch 67 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef508000 con 0x5602ef4ce400 2023-09-23T03:21:35.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.329+0000 7fe180b3a640 1 -- 172.21.15.154:0/3084437610 <== osd.7 v2:172.21.15.190:6824/3460136272 8 ==== osd_op_reply(38 10000000205.00000000 [call,setxattr (8)] v67'11 uv11 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563379d7ad80 con 0x563379f12000 2023-09-23T03:21:35.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.329+0000 7fe181b915c0 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x563379f13000 0x563379f19600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.329+0000 7f62bcabb640 1 -- 172.21.15.154:0/1011343899 <== osd.1 v2:172.21.15.154:6816/1267710005 6 ==== osd_op_reply(37 1000000020b.00000001 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555b6453cb40 con 0x555b645bb400 2023-09-23T03:21:35.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.329+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:39 10.d 10:b0000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x563379afdc00 con 0x563379f13000 2023-09-23T03:21:35.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.329+0000 7f62bf24b5c0 10 datascan.operator(): handling object 1099511628299.1 2023-09-23T03:21:35.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.329+0000 7fb99b2c4640 1 -- 172.21.15.154:0/2342064636 <== osd.0 v2:172.21.15.154:6800/2920938138 12 ==== osd_op_reply(43 [pgnls start_epoch 67 out=81b] v65'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5602efe4ed80 con 0x5602ef4ce400 2023-09-23T03:21:35.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.329+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:38 9.20 9:05552577:::1000000020b.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b645a4700 con 0x555b646ce000 2023-09-23T03:21:35.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.329+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:44 10.8 10:177a762d:::10000000201.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef508380 con 0x5602ef4ce400 2023-09-23T03:21:35.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.330+0000 7fe180b3a640 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x563379f13000 0x563379f19600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.330+0000 7fb99b2c4640 1 -- 172.21.15.154:0/2342064636 <== osd.0 v2:172.21.15.154:6800/2920938138 13 ==== osd_op_reply(44 10000000201.00000000 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5602efe4ed80 con 0x5602ef4ce400 2023-09-23T03:21:35.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.330+0000 7fb99ca525c0 10 datascan.operator(): handling object 1099511628289.0 2023-09-23T03:21:35.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.330+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:45 9.28 9:177a762d:::10000000201.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef508700 con 0x5602ef4ea400 2023-09-23T03:21:35.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.330+0000 7ffa9eac3640 1 -- 172.21.15.154:0/334556304 <== osd.5 v2:172.21.15.190:6800/882536885 10 ==== osd_op_reply(39 10000000001.00000000 [call,setxattr (8)] v67'13 uv13 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55d6af710fc0 con 0x55d6aecee400 2023-09-23T03:21:35.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.330+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:40 10.1f 10:f8000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6ae8cb880 con 0x55d6aecee800 2023-09-23T03:21:35.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.330+0000 7ffa9e2c2640 1 -- 172.21.15.154:0/334556304 <== osd.7 v2:172.21.15.190:6824/3460136272 2 ==== osd_op_reply(40 [pgnls start_epoch 67 out=81b] v65'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d6af710b40 con 0x55d6aecee800 2023-09-23T03:21:35.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.331+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:41 10.1f 10:feb1c803:::100000001fa.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6ae8cbc00 con 0x55d6aecee800 2023-09-23T03:21:35.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.331+0000 7fe180b3a640 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x563379f13000 0x563379f19600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.331+0000 7f62bdabd640 1 -- 172.21.15.154:0/1011343899 <== osd.0 v2:172.21.15.154:6800/2920938138 4 ==== osd_op_reply(38 1000000020b.00000000 [call,setxattr (8)] v67'13 uv13 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555b6453cd80 con 0x555b646ce000 2023-09-23T03:21:35.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.331+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:39 10.b 10:d0000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b645a4a80 con 0x555b645bb000 2023-09-23T03:21:35.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.331+0000 7ffa9e2c2640 1 -- 172.21.15.154:0/334556304 <== osd.7 v2:172.21.15.190:6824/3460136272 3 ==== osd_op_reply(41 100000001fa.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d6af710b40 con 0x55d6aecee800 2023-09-23T03:21:35.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.331+0000 7ffaa0a525c0 10 datascan.operator(): handling object 1099511628282.0 2023-09-23T03:21:35.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.331+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:42 9.3f 9:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55d6aed04000 con 0x55d6aecf6000 2023-09-23T03:21:35.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.331+0000 7fe180b3a640 1 -- 172.21.15.154:0/3084437610 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== osd_op_reply(39 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563379d7ad80 con 0x563379f13000 2023-09-23T03:21:35.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.331+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x563379d9f400 msgr2=0x563379b0ec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.331+0000 7fe181b915c0 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x563379d9f400 0x563379b0ec00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.331+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x563379d9ec00 msgr2=0x563379f18b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.331+0000 7fe181b915c0 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x563379d9ec00 0x563379f18b00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.331+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x563379f12800 msgr2=0x563379f1a680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.331+0000 7fe181b915c0 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x563379f12800 0x563379f1a680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.331+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x563379f12400 msgr2=0x563379f18580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.331+0000 7fe181b915c0 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x563379f12400 0x563379f18580 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.331+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x563379d9f000 msgr2=0x563379b0f180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.331+0000 7fe181b915c0 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x563379d9f000 0x563379b0f180 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.331+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x56337a83a800 msgr2=0x563379f19080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.331+0000 7fe181b915c0 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x56337a83a800 0x563379f19080 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.331+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x563379f13000 msgr2=0x563379f19600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.331+0000 7fe181b915c0 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x563379f13000 0x563379f19600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.331+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x563379f12000 msgr2=0x563379f18000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.331+0000 7fe181b915c0 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x563379f12000 0x563379f18000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.331+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x563379dff400 msgr2=0x563379b0f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.331+0000 7fe181b915c0 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x563379dff400 0x563379b0f700 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x56337a8a6990 tx=0x56337a823f80 comp rx=0 tx=0).stop 2023-09-23T03:21:35.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.331+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563379d9f800 msgr2=0x563379b0db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.331+0000 7fe181b915c0 1 --2- 172.21.15.154:0/3084437610 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563379d9f800 0x563379b0db80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x563379d973e0 tx=0x56337a83d620 comp rx=0 tx=0).stop 2023-09-23T03:21:35.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.332+0000 7fe180339640 1 -- 172.21.15.154:0/3084437610 reap_dead start 2023-09-23T03:21:35.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.332+0000 7fe180339640 1 -- 172.21.15.154:0/3084437610 reap_dead start 2023-09-23T03:21:35.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.332+0000 7fe180339640 1 -- 172.21.15.154:0/3084437610 reap_dead start 2023-09-23T03:21:35.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.332+0000 7fe180339640 1 -- 172.21.15.154:0/3084437610 reap_dead start 2023-09-23T03:21:35.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.332+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 shutdown_connections 2023-09-23T03:21:35.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.332+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 >> 172.21.15.154:0/3084437610 conn(0x56337a83b000 msgr2=0x56337a81a800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:35.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.332+0000 7f62bdabd640 1 -- 172.21.15.154:0/1011343899 <== osd.6 v2:172.21.15.190:6816/1964294001 4 ==== osd_op_reply(39 [pgnls start_epoch 67 out=101b] v65'3 uv3 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x555b6453cd80 con 0x555b645bb000 2023-09-23T03:21:35.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.332+0000 7fe180339640 1 -- 172.21.15.154:0/3084437610 reap_dead start 2023-09-23T03:21:35.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.332+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:40 10.b 10:d2952c40:::10000000208.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b645a4e00 con 0x555b645bb000 2023-09-23T03:21:35.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.332+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 shutdown_connections 2023-09-23T03:21:35.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.332+0000 7fe181b915c0 1 -- 172.21.15.154:0/3084437610 wait complete. 2023-09-23T03:21:35.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.333+0000 7f62bdabd640 1 -- 172.21.15.154:0/1011343899 <== osd.6 v2:172.21.15.190:6816/1964294001 5 ==== osd_op_reply(40 10000000208.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555b6453cd80 con 0x555b645bb000 2023-09-23T03:21:35.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.333+0000 7f62bf24b5c0 10 datascan.operator(): handling object 1099511628296.1 2023-09-23T03:21:35.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.333+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:41 9.27 9:e5993619:::10000000208.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b645a5180 con 0x555b645bac00 2023-09-23T03:21:35.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.333+0000 7ffa9f2c4640 1 -- 172.21.15.154:0/334556304 <== osd.2 v2:172.21.15.154:6822/2884407243 12 ==== osd_op_reply(42 100000001fa.00000000 [call,setxattr (8)] v67'12 uv12 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55d6af710d80 con 0x55d6aecf6000 2023-09-23T03:21:35.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.333+0000 7fe181b915c0 1 -- 172.21.15.154:0/697319708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56337a83b400 msgr2=0x563379b0d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.333+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55d6aecf6800 msgr2=0x55d6aecf8b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.333+0000 7fe181b915c0 1 --2- 172.21.15.154:0/697319708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56337a83b400 0x563379b0d600 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x56337a8a74d0 tx=0x563379d9c000 comp rx=0 tx=0).stop 2023-09-23T03:21:35.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.333+0000 7ffaa0a525c0 1 --2- 172.21.15.154:0/334556304 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55d6aecf6800 0x55d6aecf8b00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.333+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55d6aecf6000 msgr2=0x55d6aecf8000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.333+0000 7fe181b915c0 1 -- 172.21.15.154:0/697319708 shutdown_connections 2023-09-23T03:21:35.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.333+0000 7ffaa0a525c0 1 --2- 172.21.15.154:0/334556304 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55d6aecf6000 0x55d6aecf8000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.333+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55d6aecee000 msgr2=0x55d6ae8dc100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.333+0000 7ffaa0a525c0 1 --2- 172.21.15.154:0/334556304 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55d6aecee000 0x55d6ae8dc100 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.333+0000 7ffa9eac3640 1 -- 172.21.15.154:0/334556304 reap_dead start 2023-09-23T03:21:35.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.333+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55d6aecf6400 msgr2=0x55d6aecf8580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.333+0000 7ffaa0a525c0 1 --2- 172.21.15.154:0/334556304 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55d6aecf6400 0x55d6aecf8580 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.333+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55d6aecee400 msgr2=0x55d6ae8dbb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.333+0000 7ffaa0a525c0 1 --2- 172.21.15.154:0/334556304 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55d6aecee400 0x55d6ae8dbb80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.333+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55d6aeceec00 msgr2=0x55d6aecf9600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.333+0000 7ffaa0a525c0 1 --2- 172.21.15.154:0/334556304 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55d6aeceec00 0x55d6aecf9600 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.333+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55d6aecee800 msgr2=0x55d6ae8db600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.333+0000 7ffaa0a525c0 1 --2- 172.21.15.154:0/334556304 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55d6aecee800 0x55d6ae8db600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.334+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55d6aeb63c00 msgr2=0x55d6ae8dd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.334+0000 7ffaa0a525c0 1 --2- 172.21.15.154:0/334556304 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55d6aeb63c00 0x55d6ae8dd700 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x55d6ae962780 tx=0x55d6aeb99f20 comp rx=0 tx=0).stop 2023-09-23T03:21:35.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.334+0000 7ffa9eac3640 1 -- 172.21.15.154:0/334556304 reap_dead start 2023-09-23T03:21:35.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.334+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d6aeb63400 msgr2=0x55d6ae8dcc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.334+0000 7ffaa0a525c0 1 --2- 172.21.15.154:0/334556304 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d6aeb63400 0x55d6ae8dcc00 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x55d6aeb81290 tx=0x55d6aeb48240 comp rx=0 tx=0).stop 2023-09-23T03:21:35.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.334+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 shutdown_connections 2023-09-23T03:21:35.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.334+0000 7ffaa0a525c0 1 --2- 172.21.15.154:0/334556304 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d6aeb63400 0x55d6ae8dcc00 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.334+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 >> 172.21.15.154:0/334556304 conn(0x55d6aeb62c00 msgr2=0x55d6af5eb000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:35.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.334+0000 7fe181b915c0 1 --2- 172.21.15.154:0/697319708 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56337a83b400 0x563379b0d600 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.334+0000 7fe181b915c0 1 --2- 172.21.15.154:0/697319708 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563379aa3c00 0x563379b0d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.334+0000 7fe181b915c0 1 --2- 172.21.15.154:0/697319708 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563379aa3800 0x563379b0cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.334+0000 7fe181b915c0 1 -- 172.21.15.154:0/697319708 >> 172.21.15.154:0/697319708 conn(0x563379aa3000 msgr2=0x56337a81b000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:35.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.334+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 shutdown_connections 2023-09-23T03:21:35.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.334+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/334556304 wait complete. 2023-09-23T03:21:35.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.334+0000 7fb99b2c4640 1 -- 172.21.15.154:0/2342064636 <== osd.6 v2:172.21.15.190:6816/1964294001 15 ==== osd_op_reply(45 10000000201.00000000 [call,setxattr (8)] v67'14 uv14 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5602efe4ed80 con 0x5602ef4ea400 2023-09-23T03:21:35.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.334+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:46 10.18 10:18000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef508a80 con 0x5602ef4ea800 2023-09-23T03:21:35.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.334+0000 7fe181b915c0 1 -- 172.21.15.154:0/697319708 shutdown_connections 2023-09-23T03:21:35.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.335+0000 7fe181b915c0 1 -- 172.21.15.154:0/697319708 wait complete. 2023-09-23T03:21:35.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.335+0000 7f62bcabb640 1 -- 172.21.15.154:0/1011343899 <== osd.2 v2:172.21.15.154:6822/2884407243 5 ==== osd_op_reply(41 10000000208.00000000 [call,setxattr (8)] v67'6 uv6 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555b6453cb40 con 0x555b645bac00 2023-09-23T03:21:35.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.335+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:42 10.b 10:d4a58307::::head [pgnls start_epoch 67 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b645a5500 con 0x555b645bb000 2023-09-23T03:21:35.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.335+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/1822334085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d6ae871800 msgr2=0x55d6ae8dab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.335+0000 7ffaa0a525c0 1 --2- 172.21.15.154:0/1822334085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d6ae871800 0x55d6ae8dab00 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x55d6af669410 tx=0x55d6aeb48420 comp rx=0 tx=0).stop 2023-09-23T03:21:35.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.335+0000 7fb99aac3640 1 -- 172.21.15.154:0/2342064636 <== osd.2 v2:172.21.15.154:6822/2884407243 3 ==== osd_op_reply(46 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5602efe4efc0 con 0x5602ef4ea800 2023-09-23T03:21:35.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.335+0000 7fb9962ba640 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:47 10.4 10:20000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef4e5c00 con 0x5602ef4ce400 2023-09-23T03:21:35.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.335+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/1822334085 shutdown_connections 2023-09-23T03:21:35.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.335+0000 7ffaa0a525c0 1 --2- 172.21.15.154:0/1822334085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d6ae871c00 0x55d6ae8db080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.335+0000 7ffaa0a525c0 1 --2- 172.21.15.154:0/1822334085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d6ae871800 0x55d6ae8dab00 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.335+0000 7ffaa0a525c0 1 --2- 172.21.15.154:0/1822334085 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d6ae870c00 0x55d6ae8da580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.335+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/1822334085 >> 172.21.15.154:0/1822334085 conn(0x55d6ae871000 msgr2=0x55d6af5ea800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:35.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.335+0000 7f62bdabd640 1 -- 172.21.15.154:0/1011343899 <== osd.6 v2:172.21.15.190:6816/1964294001 6 ==== osd_op_reply(42 [pgnls start_epoch 67 out=81b] v65'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555b6453cd80 con 0x555b645bb000 2023-09-23T03:21:35.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.335+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:43 10.b 10:d4a58307:::10000000203.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b645a5880 con 0x555b645bb000 2023-09-23T03:21:35.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.335+0000 7fb99b2c4640 1 -- 172.21.15.154:0/2342064636 <== osd.0 v2:172.21.15.154:6800/2920938138 14 ==== osd_op_reply(47 [pgnls start_epoch 67 out=101b] v65'3 uv3 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5602efe4ed80 con 0x5602ef4ce400 2023-09-23T03:21:35.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.335+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/1822334085 shutdown_connections 2023-09-23T03:21:35.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.335+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:48 10.4 10:2057972a:::100000001ff.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef508e00 con 0x5602ef4ce400 2023-09-23T03:21:35.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.336+0000 7ffaa0a525c0 1 -- 172.21.15.154:0/1822334085 wait complete. 2023-09-23T03:21:35.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.336+0000 7f62bdabd640 1 -- 172.21.15.154:0/1011343899 <== osd.6 v2:172.21.15.190:6816/1964294001 7 ==== osd_op_reply(43 10000000203.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555b6453cd80 con 0x555b645bb000 2023-09-23T03:21:35.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.336+0000 7f62bf24b5c0 10 datascan.operator(): handling object 1099511628291.0 2023-09-23T03:21:35.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.336+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:44 9.2b 9:d4a58307:::10000000203.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b645a5c00 con 0x555b646ce400 2023-09-23T03:21:35.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.336+0000 7fb99b2c4640 1 -- 172.21.15.154:0/2342064636 <== osd.0 v2:172.21.15.154:6800/2920938138 15 ==== osd_op_reply(48 100000001ff.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5602efe4ed80 con 0x5602ef4ce400 2023-09-23T03:21:35.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.336+0000 7fb99ca525c0 10 datascan.operator(): handling object 1099511628287.1 2023-09-23T03:21:35.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.336+0000 7fb99ca525c0 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x5602ef4cec00 0x5602ef4f1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.336+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:49 9.26 9:658f2455:::100000001ff.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef509180 con 0x5602ef4cec00 2023-09-23T03:21:35.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.336+0000 7fb99a2c2640 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x5602ef4cec00 0x5602ef4f1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.337+0000 7fb99a2c2640 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x5602ef4cec00 0x5602ef4f1600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.338+0000 7f62bd2bc640 1 -- 172.21.15.154:0/1011343899 <== osd.7 v2:172.21.15.190:6824/3460136272 14 ==== osd_op_reply(44 10000000203.00000000 [call,setxattr (8)] v67'15 uv15 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555b6453cfc0 con 0x555b646ce400 2023-09-23T03:21:35.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.338+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:45 10.1b 10:d8000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x555b645a5c00 con 0x555b646ce800 2023-09-23T03:21:35.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.338+0000 7f62bcabb640 1 -- 172.21.15.154:0/1011343899 <== osd.3 v2:172.21.15.154:6808/1133688468 3 ==== osd_op_reply(45 [pgnls start_epoch 67 out=81b] v65'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555b6453cb40 con 0x555b646ce800 2023-09-23T03:21:35.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.339+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x555b646ce000 msgr2=0x555b646d4000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.339+0000 7f62bf24b5c0 1 --2- 172.21.15.154:0/1011343899 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x555b646ce000 0x555b646d4000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.339+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x555b645bb400 msgr2=0x555b642cf600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.339+0000 7f62bf24b5c0 1 --2- 172.21.15.154:0/1011343899 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x555b645bb400 0x555b642cf600 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.339+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x555b645bac00 msgr2=0x555b642d0100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.339+0000 7f62bf24b5c0 1 --2- 172.21.15.154:0/1011343899 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x555b645bac00 0x555b642d0100 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.339+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x555b646ce800 msgr2=0x555b646d4b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.339+0000 7f62bf24b5c0 1 --2- 172.21.15.154:0/1011343899 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x555b646ce800 0x555b646d4b00 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.339+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x555b646cf000 msgr2=0x555b646d5600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.339+0000 7f62bf24b5c0 1 --2- 172.21.15.154:0/1011343899 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x555b646cf000 0x555b646d5600 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.339+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x555b645bb000 msgr2=0x555b642cfb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.339+0000 7f62bf24b5c0 1 --2- 172.21.15.154:0/1011343899 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x555b645bb000 0x555b642cfb80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.339+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x555b646ce400 msgr2=0x555b646d4580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.339+0000 7f62bf24b5c0 1 --2- 172.21.15.154:0/1011343899 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x555b646ce400 0x555b646d4580 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.339+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x555b645ba800 msgr2=0x555b642d1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.339+0000 7f62bf24b5c0 1 --2- 172.21.15.154:0/1011343899 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x555b645ba800 0x555b642d1700 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x555b642f3b00 tx=0x555b64564d20 comp rx=0 tx=0).stop 2023-09-23T03:21:35.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.339+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b6455d000 msgr2=0x555b642d1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.339+0000 7f62bf24b5c0 1 --2- 172.21.15.154:0/1011343899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b6455d000 0x555b642d1180 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x555b6455f6e0 tx=0x555b645481e0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.339+0000 7f62bd2bc640 1 -- 172.21.15.154:0/1011343899 reap_dead start 2023-09-23T03:21:35.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.339+0000 7f62bd2bc640 1 -- 172.21.15.154:0/1011343899 reap_dead start 2023-09-23T03:21:35.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.339+0000 7f62bd2bc640 1 -- 172.21.15.154:0/1011343899 reap_dead start 2023-09-23T03:21:35.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.339+0000 7f62bd2bc640 1 -- 172.21.15.154:0/1011343899 reap_dead start 2023-09-23T03:21:35.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.339+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 shutdown_connections 2023-09-23T03:21:35.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.339+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 >> 172.21.15.154:0/1011343899 conn(0x555b6455cc00 msgr2=0x555b64fdc800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:35.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.339+0000 7f62bd2bc640 1 -- 172.21.15.154:0/1011343899 reap_dead start 2023-09-23T03:21:35.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.339+0000 7f62bd2bc640 1 -- 172.21.15.154:0/1011343899 reap_dead start 2023-09-23T03:21:35.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.339+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 shutdown_connections 2023-09-23T03:21:35.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.339+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/1011343899 wait complete. 2023-09-23T03:21:35.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.339+0000 7fb99a2c2640 1 -- 172.21.15.154:0/2342064636 <== osd.5 v2:172.21.15.190:6800/882536885 1 ==== osd_op_reply(49 100000001ff.00000000 [call,setxattr (8)] v67'19 uv19 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5602efe4eb40 con 0x5602ef4cec00 2023-09-23T03:21:35.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.339+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:50 10.4 10:218dfa3a::::head [pgnls start_epoch 67 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef509500 con 0x5602ef4ce400 2023-09-23T03:21:35.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.340+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/4061198893 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b64265800 msgr2=0x555b642ceb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.340+0000 7f62bf24b5c0 1 --2- 172.21.15.154:0/4061198893 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b64265800 0x555b642ceb00 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x555b6506b440 tx=0x555b64548240 comp rx=0 tx=0).stop 2023-09-23T03:21:35.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.340+0000 7fb99b2c4640 1 -- 172.21.15.154:0/2342064636 <== osd.0 v2:172.21.15.154:6800/2920938138 16 ==== osd_op_reply(50 [pgnls start_epoch 67 out=101b] v65'3 uv3 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5602efe4ed80 con 0x5602ef4ce400 2023-09-23T03:21:35.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.340+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/4061198893 shutdown_connections 2023-09-23T03:21:35.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.340+0000 7f62bf24b5c0 1 --2- 172.21.15.154:0/4061198893 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x555b64265c00 0x555b642cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.340+0000 7f62bf24b5c0 1 --2- 172.21.15.154:0/4061198893 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b64265800 0x555b642ceb00 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.340+0000 7f62bf24b5c0 1 --2- 172.21.15.154:0/4061198893 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x555b64264c00 0x555b642ce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.340+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/4061198893 >> 172.21.15.154:0/4061198893 conn(0x555b64265000 msgr2=0x555b64fdd000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:35.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.340+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:51 10.4 10:218dfa3a:::10000000200.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef509880 con 0x5602ef4ce400 2023-09-23T03:21:35.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.340+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/4061198893 shutdown_connections 2023-09-23T03:21:35.537 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', '--worker_n', '6', '--worker_m', '7'] in time 0:00:00.467813, result: 2023-09-23T03:21:35.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.340+0000 7f62bf24b5c0 1 -- 172.21.15.154:0/4061198893 wait complete. 2023-09-23T03:21:35.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.340+0000 7fb99b2c4640 1 -- 172.21.15.154:0/2342064636 <== osd.0 v2:172.21.15.154:6800/2920938138 17 ==== osd_op_reply(51 10000000200.00000001 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5602efe4ed80 con 0x5602ef4ce400 2023-09-23T03:21:35.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.340+0000 7fb99ca525c0 10 datascan.operator(): handling object 1099511628288.1 2023-09-23T03:21:35.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.341+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:52 9.19 9:992a6246:::10000000200.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x5602ef509c00 con 0x5602ef4ea000 2023-09-23T03:21:35.539 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', '--worker_n', '4', '--worker_m', '7'] in time 0:00:00.470185, result: 2023-09-23T03:21:35.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.342+0000 7fb99a2c2640 1 -- 172.21.15.154:0/2342064636 <== osd.3 v2:172.21.15.154:6808/1133688468 4 ==== osd_op_reply(52 10000000200.00000000 [call,setxattr (8)] v67'18 uv18 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5602efe4eb40 con 0x5602ef4ea000 2023-09-23T03:21:35.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.343+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5602ef4ce400 msgr2=0x5602ef0e5b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.343+0000 7fb99ca525c0 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5602ef4ce400 0x5602ef0e5b80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.343+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5602ef4ce000 msgr2=0x5602ef0e6100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.343+0000 7fb99ca525c0 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5602ef4ce000 0x5602ef0e6100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.343+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5602ef4ea800 msgr2=0x5602ef4f0b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.343+0000 7fb99ca525c0 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5602ef4ea800 0x5602ef4f0b00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.343+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5602ef4ea000 msgr2=0x5602ef4f0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.343+0000 7fb99ca525c0 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5602ef4ea000 0x5602ef4f0000 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.343+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5602ef4ce800 msgr2=0x5602ef0e5600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.343+0000 7fb99ca525c0 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5602ef4ce800 0x5602ef0e5600 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.343+0000 7fb99aac3640 1 -- 172.21.15.154:0/2342064636 reap_dead start 2023-09-23T03:21:35.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.343+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x5602ef4cec00 msgr2=0x5602ef4f1600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.343+0000 7fb99ca525c0 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x5602ef4cec00 0x5602ef4f1600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.343+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x5602ef4ea400 msgr2=0x5602ef4f0580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.343+0000 7fb99ca525c0 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x5602ef4ea400 0x5602ef4f0580 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.343+0000 7fb99aac3640 1 -- 172.21.15.154:0/2342064636 reap_dead start 2023-09-23T03:21:35.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.343+0000 7fb99aac3640 1 -- 172.21.15.154:0/2342064636 reap_dead start 2023-09-23T03:21:35.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.343+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5602ef363c00 msgr2=0x5602ef0e7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.343+0000 7fb99ca525c0 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5602ef363c00 0x5602ef0e7700 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x5602ef36f500 tx=0x5602ef377da0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.343+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5602ef363400 msgr2=0x5602ef0e6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.343+0000 7fb99ca525c0 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5602ef363400 0x5602ef0e6c00 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x5602ef36ea50 tx=0x5602efdde060 comp rx=0 tx=0).stop 2023-09-23T03:21:35.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.343+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 shutdown_connections 2023-09-23T03:21:35.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.343+0000 7fb99ca525c0 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5602ef363c00 0x5602ef0e7700 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.343+0000 7fb99ca525c0 1 --2- 172.21.15.154:0/2342064636 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5602ef363400 0x5602ef0e6c00 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.343+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 >> 172.21.15.154:0/2342064636 conn(0x5602ef362c00 msgr2=0x5602efdf5000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:35.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.343+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 shutdown_connections 2023-09-23T03:21:35.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.343+0000 7fb99ca525c0 1 -- 172.21.15.154:0/2342064636 wait complete. 2023-09-23T03:21:35.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.344+0000 7fb99ca525c0 1 -- 172.21.15.154:0/3567808904 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5602ef07b800 msgr2=0x5602ef0e4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.344+0000 7fb99ca525c0 1 --2- 172.21.15.154:0/3567808904 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5602ef07b800 0x5602ef0e4b00 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x5602efe873e0 tx=0x5602efdde480 comp rx=0 tx=0).stop 2023-09-23T03:21:35.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.344+0000 7fb99ca525c0 1 -- 172.21.15.154:0/3567808904 shutdown_connections 2023-09-23T03:21:35.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.344+0000 7fb99ca525c0 1 --2- 172.21.15.154:0/3567808904 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5602ef07bc00 0x5602ef0e5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.344+0000 7fb99ca525c0 1 --2- 172.21.15.154:0/3567808904 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5602ef07b800 0x5602ef0e4b00 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.344+0000 7fb99ca525c0 1 --2- 172.21.15.154:0/3567808904 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5602ef07ac00 0x5602ef0e4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.344+0000 7fb99ca525c0 1 -- 172.21.15.154:0/3567808904 >> 172.21.15.154:0/3567808904 conn(0x5602ef07b000 msgr2=0x5602efdf4800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:35.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.344+0000 7fb99ca525c0 1 -- 172.21.15.154:0/3567808904 shutdown_connections 2023-09-23T03:21:35.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.344+0000 7fb99ca525c0 1 -- 172.21.15.154:0/3567808904 wait complete. 2023-09-23T03:21:35.543 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', '--worker_n', '5', '--worker_m', '7'] in time 0:00:00.474445, result: 2023-09-23T03:21:35.544 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', '--worker_n', '0', '--worker_m', '7'] in time 0:00:00.476872, result: 2023-09-23T03:21:35.544 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --worker_n 0 --worker_m 7 2023-09-23T03:21:35.545 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --worker_n 1 --worker_m 7 2023-09-23T03:21:35.545 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --worker_n 2 --worker_m 7 2023-09-23T03:21:35.545 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --worker_n 3 --worker_m 7 2023-09-23T03:21:35.546 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --worker_n 4 --worker_m 7 2023-09-23T03:21:35.546 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --worker_n 5 --worker_m 7 2023-09-23T03:21:35.547 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --worker_n 6 --worker_m 7 2023-09-23T03:21:35.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.672+0000 7f5ee8f915c0 1 -- start start 2023-09-23T03:21:35.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.672+0000 7f5ee8f915c0 1 Processor -- start 2023-09-23T03:21:35.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.672+0000 7f9113c415c0 1 -- 172.21.15.154:0/1656660284 wait complete. 2023-09-23T03:21:35.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.674+0000 7f9113c415c0 1 -- start start 2023-09-23T03:21:35.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.674+0000 7f9113c415c0 1 Processor -- start 2023-09-23T03:21:35.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.674+0000 7f5ee8f915c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b4d7fec00 0x557b4d868580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.674+0000 7f5ee8f915c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b4d7ff800 0x557b4d868b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.675+0000 7f5ee6f38640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b4d7ff800 0x557b4d868b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.675+0000 7f5ee6f38640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b4d7ff800 0x557b4d868b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60654/0 (socket says 172.21.15.154:60654) 2023-09-23T03:21:35.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.675+0000 7f5ee6f38640 1 -- 172.21.15.154:0/2105377283 learned_addr learned my addr 172.21.15.154:0/2105377283 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.675+0000 7f5ee8f915c0 1 --2- 172.21.15.154:0/2105377283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557b4d7ffc00 0x557b4d869080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.675+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/2105377283 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b4d849180 con 0x557b4d7ff800 2023-09-23T03:21:35.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.675+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/2105377283 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b4d848140 con 0x557b4d7fec00 2023-09-23T03:21:35.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.675+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/2105377283 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b4d848000 con 0x557b4d7ffc00 2023-09-23T03:21:35.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.675+0000 7f5ee7739640 1 --2- 172.21.15.154:0/2105377283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b4d7fec00 0x557b4d868580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.676+0000 7f5ee6f38640 1 -- 172.21.15.154:0/2105377283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557b4d7ffc00 msgr2=0x557b4d869080 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T03:21:35.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.676+0000 7f5ee6f38640 1 --2- 172.21.15.154:0/2105377283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557b4d7ffc00 0x557b4d869080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.676+0000 7f5ee6f38640 1 -- 172.21.15.154:0/2105377283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b4d7fec00 msgr2=0x557b4d868580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.676+0000 7f5ee6f38640 1 --2- 172.21.15.154:0/2105377283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b4d7fec00 0x557b4d868580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.676+0000 7f5ee6f38640 1 -- 172.21.15.154:0/2105377283 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b4d832b60 con 0x557b4d7ff800 2023-09-23T03:21:35.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.676+0000 7f9113c415c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5625b70c1800 0x5625b712ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.676+0000 7f9113c415c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5625b70c1c00 0x5625b712b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.676+0000 7f9113c415c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5625b7e91400 0x5625b712b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.676+0000 7f9113c415c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b710b180 con 0x5625b7e91400 2023-09-23T03:21:35.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.676+0000 7f5ee6f38640 1 --2- 172.21.15.154:0/2105377283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b4d7ff800 0x557b4d868b00 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x557b4e604a20 tx=0x557b4dad8120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56550916783c419 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.676+0000 7f9113c415c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b710a140 con 0x5625b70c1c00 2023-09-23T03:21:35.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.676+0000 7f9113c415c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5625b710a000 con 0x5625b70c1800 2023-09-23T03:21:35.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.677+0000 7f911240d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5625b7e91400 0x5625b712b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.677+0000 7f911240d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5625b7e91400 0x5625b712b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60656/0 (socket says 172.21.15.154:60656) 2023-09-23T03:21:35.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.677+0000 7f911240d640 1 -- 172.21.15.154:0/318524665 learned_addr learned my addr 172.21.15.154:0/318524665 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.677+0000 7f911240d640 1 -- 172.21.15.154:0/318524665 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5625b70c1800 msgr2=0x5625b712ab00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T03:21:35.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.677+0000 7f911240d640 1 --2- 172.21.15.154:0/318524665 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5625b70c1800 0x5625b712ab00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.677+0000 7f911240d640 1 -- 172.21.15.154:0/318524665 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5625b70c1c00 msgr2=0x5625b712b080 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T03:21:35.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.677+0000 7f911240d640 1 --2- 172.21.15.154:0/318524665 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5625b70c1c00 0x5625b712b080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.677+0000 7f5ee4f34640 1 -- 172.21.15.154:0/2105377283 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557b4d847080 con 0x557b4d7ff800 2023-09-23T03:21:35.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.677+0000 7f5ee4f34640 1 -- 172.21.15.154:0/2105377283 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557b4d825800 con 0x557b4d7ff800 2023-09-23T03:21:35.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.677+0000 7f5ee4f34640 1 -- 172.21.15.154:0/2105377283 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557b4d847600 con 0x557b4d7ff800 2023-09-23T03:21:35.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.677+0000 7f911240d640 1 -- 172.21.15.154:0/318524665 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625b70f4b60 con 0x5625b7e91400 2023-09-23T03:21:35.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.678+0000 7f911240d640 1 --2- 172.21.15.154:0/318524665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5625b7e91400 0x5625b712b600 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x5625b7ec5440 tx=0x5625b73aa0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b6776183d58c579 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.678+0000 7f910f407640 1 -- 172.21.15.154:0/318524665 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5625b7109600 con 0x5625b7e91400 2023-09-23T03:21:35.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.678+0000 7f910f407640 1 -- 172.21.15.154:0/318524665 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5625b70e7800 con 0x5625b7e91400 2023-09-23T03:21:35.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.678+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/2105377283 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557b4d8331e0 con 0x557b4d7ff800 2023-09-23T03:21:35.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.678+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/2105377283 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x557b4d832b60 con 0x557b4d7ff800 2023-09-23T03:21:35.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.678+0000 7f5ee8f915c0 4 waiting for MDS map... 2023-09-23T03:21:35.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.678+0000 7f910f407640 1 -- 172.21.15.154:0/318524665 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5625b71091e0 con 0x5625b7e91400 2023-09-23T03:21:35.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.679+0000 7f9113c415c0 1 -- 172.21.15.154:0/318524665 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5625b70f4340 con 0x5625b7e91400 2023-09-23T03:21:35.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.679+0000 7f9113c415c0 1 -- 172.21.15.154:0/318524665 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5625b70f4000 con 0x5625b7e91400 2023-09-23T03:21:35.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.679+0000 7f9113c415c0 4 waiting for MDS map... 2023-09-23T03:21:35.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.679+0000 7f910f407640 1 -- 172.21.15.154:0/318524665 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x5625b7180c40 con 0x5625b7e91400 2023-09-23T03:21:35.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.680+0000 7f910f407640 1 -- 172.21.15.154:0/318524665 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x5625b7112f00 con 0x5625b7e91400 2023-09-23T03:21:35.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.679+0000 7f5ee4f34640 1 -- 172.21.15.154:0/2105377283 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x557b4d8bec40 con 0x557b4d7ff800 2023-09-23T03:21:35.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.680+0000 7f5ee4f34640 1 -- 172.21.15.154:0/2105377283 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x557b4d850f00 con 0x557b4d7ff800 2023-09-23T03:21:35.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.680+0000 7f5ee8f915c0 4 Got MDS map 47 2023-09-23T03:21:35.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.680+0000 7f9113c415c0 4 Got MDS map 47 2023-09-23T03:21:35.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.680+0000 7f5ee8f915c0 4 datascan.main: Using metadata pool output 2023-09-23T03:21:35.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.680+0000 7f5ee8f915c0 4 datascan.main: connecting to RADOS... 2023-09-23T03:21:35.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.680+0000 7f9113c415c0 4 datascan.main: Using metadata pool output 2023-09-23T03:21:35.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.680+0000 7f9113c415c0 4 datascan.main: connecting to RADOS... 2023-09-23T03:21:35.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.681+0000 7f5ee8f915c0 1 Processor -- start 2023-09-23T03:21:35.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.681+0000 7f9113c415c0 1 Processor -- start 2023-09-23T03:21:35.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.681+0000 7f5ee8f915c0 1 -- start start 2023-09-23T03:21:35.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.682+0000 7f9113c415c0 1 -- start start 2023-09-23T03:21:35.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.682+0000 7f9113c415c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5625b7e90800 0x5625b712bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.682+0000 7f9113c415c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5625b70c0c00 0x5625b712c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.683+0000 7f5ee8f915c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b4e5d5c00 0x557b4d869600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.683+0000 7f9113c415c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5625b7412000 0x5625b712c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.683+0000 7f9113c415c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b710aa00 con 0x5625b7e90800 2023-09-23T03:21:35.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.683+0000 7f9113c415c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b710ab40 con 0x5625b7412000 2023-09-23T03:21:35.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.683+0000 7f9113c415c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5625b718f180 con 0x5625b70c0c00 2023-09-23T03:21:35.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.683+0000 7f5ee8f915c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b4db3c000 0x557b4d869b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.683+0000 7f5ee7f3a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b4e5d5c00 0x557b4d869600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.683+0000 7f5ee7f3a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b4e5d5c00 0x557b4d869600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.154:49362/0 (socket says 172.21.15.154:49362) 2023-09-23T03:21:35.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.683+0000 7f911140b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5625b7e90800 0x5625b712bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.683+0000 7f911140b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5625b7e90800 0x5625b712bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60670/0 (socket says 172.21.15.154:60670) 2023-09-23T03:21:35.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.683+0000 7f911140b640 1 -- 172.21.15.154:0/2939442764 learned_addr learned my addr 172.21.15.154:0/2939442764 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.684+0000 7f5ee8f915c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557b4db3c400 0x557b4d86a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.684+0000 7f911140b640 1 -- 172.21.15.154:0/2939442764 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5625b70c0c00 msgr2=0x5625b712c100 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-09-23T03:21:35.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.684+0000 7f911140b640 1 --2- 172.21.15.154:0/2939442764 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5625b70c0c00 0x5625b712c100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.684+0000 7f911140b640 1 -- 172.21.15.154:0/2939442764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5625b7412000 msgr2=0x5625b712c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.684+0000 7f911140b640 1 --2- 172.21.15.154:0/2939442764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5625b7412000 0x5625b712c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.684+0000 7f911140b640 1 -- 172.21.15.154:0/2939442764 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625b70f4ea0 con 0x5625b7e90800 2023-09-23T03:21:35.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.684+0000 7f5ee8f915c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b4d848a00 con 0x557b4db3c000 2023-09-23T03:21:35.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.684+0000 7f5ee8f915c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b4d8cd180 con 0x557b4e5d5c00 2023-09-23T03:21:35.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.684+0000 7f5ee8f915c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b4d8ccdc0 con 0x557b4db3c400 2023-09-23T03:21:35.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.684+0000 7f911140b640 1 --2- 172.21.15.154:0/2939442764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5625b7e90800 0x5625b712bb80 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x5625b73c0930 tx=0x5625b73c4d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bbd157cc6a891d5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.684+0000 7f5ee7739640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b4db3c000 0x557b4d869b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.684+0000 7f5ee7739640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b4db3c000 0x557b4d869b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60684/0 (socket says 172.21.15.154:60684) 2023-09-23T03:21:35.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.684+0000 7f5ee7739640 1 -- 172.21.15.154:0/1566990310 learned_addr learned my addr 172.21.15.154:0/1566990310 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.684+0000 7f5ee6f38640 1 --2- 172.21.15.154:0/1566990310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557b4db3c400 0x557b4d86a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.685+0000 7f5ee7739640 1 -- 172.21.15.154:0/1566990310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557b4db3c400 msgr2=0x557b4d86a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.685+0000 7f5ee7739640 1 --2- 172.21.15.154:0/1566990310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557b4db3c400 0x557b4d86a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.685+0000 7f5ee7739640 1 -- 172.21.15.154:0/1566990310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b4e5d5c00 msgr2=0x557b4d869600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.685+0000 7f5ee7739640 1 --2- 172.21.15.154:0/1566990310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b4e5d5c00 0x557b4d869600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.685+0000 7f910d403640 1 -- 172.21.15.154:0/2939442764 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5625b7109600 con 0x5625b7e90800 2023-09-23T03:21:35.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.685+0000 7f5ee7739640 1 -- 172.21.15.154:0/1566990310 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b4d832340 con 0x557b4db3c000 2023-09-23T03:21:35.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.685+0000 7f910d403640 1 -- 172.21.15.154:0/2939442764 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5625b7e84d80 con 0x5625b7e90800 2023-09-23T03:21:35.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.685+0000 7f5ee7739640 1 --2- 172.21.15.154:0/1566990310 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b4db3c000 0x557b4d869b80 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x557b4e5ceab0 tx=0x557b4daeed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d10946b62be0f984 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.685+0000 7f910d403640 1 -- 172.21.15.154:0/2939442764 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5625b71091e0 con 0x5625b7e90800 2023-09-23T03:21:35.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.685+0000 7f5ee2f30640 1 -- 172.21.15.154:0/1566990310 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557b4d847080 con 0x557b4db3c000 2023-09-23T03:21:35.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.685+0000 7f5ee2f30640 1 -- 172.21.15.154:0/1566990310 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557b4e5d2c00 con 0x557b4db3c000 2023-09-23T03:21:35.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.685+0000 7f5ee2f30640 1 -- 172.21.15.154:0/1566990310 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557b4d847600 con 0x557b4db3c000 2023-09-23T03:21:35.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.686+0000 7f9113c415c0 1 -- 172.21.15.154:0/2939442764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5625b7e90800 msgr2=0x5625b712bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.686+0000 7f9113c415c0 1 --2- 172.21.15.154:0/2939442764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5625b7e90800 0x5625b712bb80 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x5625b73c0930 tx=0x5625b73c4d20 comp rx=0 tx=0).stop 2023-09-23T03:21:35.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.686+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1566990310 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b4db3c000 msgr2=0x557b4d869b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.686+0000 7f9113c415c0 1 -- 172.21.15.154:0/2939442764 shutdown_connections 2023-09-23T03:21:35.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.686+0000 7f9113c415c0 1 --2- 172.21.15.154:0/2939442764 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5625b7412000 0x5625b712c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.686+0000 7f9113c415c0 1 --2- 172.21.15.154:0/2939442764 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5625b70c0c00 0x5625b712c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.686+0000 7f9113c415c0 1 --2- 172.21.15.154:0/2939442764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5625b7e90800 0x5625b712bb80 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.686+0000 7f9113c415c0 1 -- 172.21.15.154:0/2939442764 >> 172.21.15.154:0/2939442764 conn(0x5625b7e91000 msgr2=0x5625b7e3b000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:35.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.686+0000 7f5ee8f915c0 1 --2- 172.21.15.154:0/1566990310 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b4db3c000 0x557b4d869b80 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x557b4e5ceab0 tx=0x557b4daeed20 comp rx=0 tx=0).stop 2023-09-23T03:21:35.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.687+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1566990310 shutdown_connections 2023-09-23T03:21:35.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.687+0000 7f9113c415c0 1 -- 172.21.15.154:0/2939442764 shutdown_connections 2023-09-23T03:21:35.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.687+0000 7f5ee8f915c0 1 --2- 172.21.15.154:0/1566990310 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557b4db3c400 0x557b4d86a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.687+0000 7f5ee8f915c0 1 --2- 172.21.15.154:0/1566990310 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b4db3c000 0x557b4d869b80 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.687+0000 7f5ee8f915c0 1 --2- 172.21.15.154:0/1566990310 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b4e5d5c00 0x557b4d869600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.687+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1566990310 >> 172.21.15.154:0/1566990310 conn(0x557b4e5d5800 msgr2=0x557b4e579000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:35.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.687+0000 7f9113c415c0 1 -- 172.21.15.154:0/2939442764 wait complete. 2023-09-23T03:21:35.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.687+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1566990310 shutdown_connections 2023-09-23T03:21:35.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.688+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1566990310 wait complete. 2023-09-23T03:21:35.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.688+0000 7f9113c415c0 1 Processor -- start 2023-09-23T03:21:35.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.688+0000 7f9113c415c0 1 -- start start 2023-09-23T03:21:35.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.689+0000 7f24ba63f5c0 1 -- start start 2023-09-23T03:21:35.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.689+0000 7f24ba63f5c0 1 Processor -- start 2023-09-23T03:21:35.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.689+0000 7f5ee8f915c0 1 Processor -- start 2023-09-23T03:21:35.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.689+0000 7f5ee8f915c0 1 -- start start 2023-09-23T03:21:35.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.690+0000 7f9113c415c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5625b70c0c00 0x5625b712c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.690+0000 7f9113c415c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5625b7413000 0x5625b712c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.691+0000 7f9113c415c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5625b7412c00 0x5625b712bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.691+0000 7f24ba63f5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5632fca04c00 0x5632fca6e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.691+0000 7f24ba63f5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632fca05800 0x5632fca6eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.691+0000 7f911140b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5625b70c0c00 0x5625b712c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.691+0000 7f911140b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5625b70c0c00 0x5625b712c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.154:49404/0 (socket says 172.21.15.154:49404) 2023-09-23T03:21:35.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.691+0000 7f24ba63f5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5632fca05c00 0x5632fca6f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.691+0000 7f5ee8f915c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b4e5d5800 0x557b4d86b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.691+0000 7f911140b640 1 -- 172.21.15.154:0/2694809311 learned_addr learned my addr 172.21.15.154:0/2694809311 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.691+0000 7f9111c0c640 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5625b7413000 0x5625b712c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.691+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b718edc0 con 0x5625b7412c00 2023-09-23T03:21:35.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.691+0000 7f5ee8f915c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b4db3d000 0x557b4d86ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.691+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625b718ec80 con 0x5625b70c0c00 2023-09-23T03:21:35.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.691+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5625b718eb40 con 0x5625b7413000 2023-09-23T03:21:35.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.692+0000 7f5ee8f915c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557b4db3cc00 0x557b4d86a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.692+0000 7f911140b640 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5625b7412c00 0x5625b712bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.692+0000 7f5ee7739640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b4db3d000 0x557b4d86ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.692+0000 7f5ee7739640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b4db3d000 0x557b4d86ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60698/0 (socket says 172.21.15.154:60698) 2023-09-23T03:21:35.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.692+0000 7f5ee7f3a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b4e5d5800 0x557b4d86b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.692+0000 7f5ee7739640 1 -- 172.21.15.154:0/1042754689 learned_addr learned my addr 172.21.15.154:0/1042754689 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.692+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b4d848140 con 0x557b4db3d000 2023-09-23T03:21:35.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.692+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b4d849180 con 0x557b4e5d5800 2023-09-23T03:21:35.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.692+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b4d848b40 con 0x557b4db3cc00 2023-09-23T03:21:35.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.691+0000 7f24b853b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5632fca04c00 0x5632fca6e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.692+0000 7f24b853b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5632fca04c00 0x5632fca6e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:34190/0 (socket says 172.21.15.154:34190) 2023-09-23T03:21:35.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.692+0000 7f24b853b640 1 -- 172.21.15.154:0/3563497076 learned_addr learned my addr 172.21.15.154:0/3563497076 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.693+0000 7f5ee7739640 1 -- 172.21.15.154:0/1042754689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557b4db3cc00 msgr2=0x557b4d86a680 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-23T03:21:35.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.693+0000 7f5ee7739640 1 --2- 172.21.15.154:0/1042754689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557b4db3cc00 0x557b4d86a680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.693+0000 7f5ee7739640 1 -- 172.21.15.154:0/1042754689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b4e5d5800 msgr2=0x557b4d86b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.693+0000 7f5ee7739640 1 --2- 172.21.15.154:0/1042754689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b4e5d5800 0x557b4d86b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.693+0000 7f5ee7739640 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b4d832000 con 0x557b4db3d000 2023-09-23T03:21:35.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.692+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/3563497076 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632fca4f180 con 0x5632fca05c00 2023-09-23T03:21:35.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.692+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/3563497076 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632fca4e140 con 0x5632fca05800 2023-09-23T03:21:35.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.693+0000 7f911140b640 1 -- 172.21.15.154:0/2694809311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5625b7413000 msgr2=0x5625b712c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.693+0000 7f911140b640 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5625b7413000 0x5625b712c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.693+0000 7f911140b640 1 -- 172.21.15.154:0/2694809311 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5625b7412c00 msgr2=0x5625b712bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.693+0000 7f911140b640 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5625b7412c00 0x5625b712bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.693+0000 7f911140b640 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625b70f4b60 con 0x5625b70c0c00 2023-09-23T03:21:35.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.692+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/3563497076 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632fca4e000 con 0x5632fca04c00 2023-09-23T03:21:35.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.693+0000 7f24b7d3a640 1 --2- 172.21.15.154:0/3563497076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632fca05800 0x5632fca6eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.693+0000 7f5ee7739640 1 --2- 172.21.15.154:0/1042754689 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b4db3d000 0x557b4d86ac00 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x557b4e5cf770 tx=0x557b4dad8360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.693+0000 7f24b8d3c640 1 --2- 172.21.15.154:0/3563497076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5632fca05c00 0x5632fca6f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.694+0000 7f24b853b640 1 -- 172.21.15.154:0/3563497076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632fca05800 msgr2=0x5632fca6eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.694+0000 7f24b853b640 1 --2- 172.21.15.154:0/3563497076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632fca05800 0x5632fca6eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.694+0000 7f24b853b640 1 -- 172.21.15.154:0/3563497076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5632fca05c00 msgr2=0x5632fca6f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.694+0000 7f24b853b640 1 --2- 172.21.15.154:0/3563497076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5632fca05c00 0x5632fca6f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.694+0000 7f24b853b640 1 -- 172.21.15.154:0/3563497076 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632fca38340 con 0x5632fca04c00 2023-09-23T03:21:35.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.694+0000 7f911140b640 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5625b70c0c00 0x5625b712c100 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x5625b73c15f0 tx=0x5625b73c4d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.694+0000 7f910bc00640 1 -- 172.21.15.154:0/2694809311 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5625b7108dc0 con 0x5625b70c0c00 2023-09-23T03:21:35.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.694+0000 7f910bc00640 1 -- 172.21.15.154:0/2694809311 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5625b7e85500 con 0x5625b70c0c00 2023-09-23T03:21:35.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.694+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5625b70f4340 con 0x5625b70c0c00 2023-09-23T03:21:35.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.693+0000 7f5ee172d640 1 -- 172.21.15.154:0/1042754689 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557b4d846dc0 con 0x557b4db3d000 2023-09-23T03:21:35.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.694+0000 7f5ee172d640 1 -- 172.21.15.154:0/1042754689 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557b4e5d2a80 con 0x557b4db3d000 2023-09-23T03:21:35.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.694+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557b4d832ea0 con 0x557b4db3d000 2023-09-23T03:21:35.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.694+0000 7f5ee172d640 1 -- 172.21.15.154:0/1042754689 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557b4d847080 con 0x557b4db3d000 2023-09-23T03:21:35.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.694+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557b4d833520 con 0x557b4db3d000 2023-09-23T03:21:35.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.694+0000 7f5ee8f915c0 4 datascan.init: resolving metadata pool 8 2023-09-23T03:21:35.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.695+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5625b70f5520 con 0x5625b70c0c00 2023-09-23T03:21:35.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.695+0000 7f910bc00640 1 -- 172.21.15.154:0/2694809311 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5625b7109600 con 0x5625b70c0c00 2023-09-23T03:21:35.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.695+0000 7f9113c415c0 4 datascan.init: resolving metadata pool 8 2023-09-23T03:21:35.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.695+0000 7f24b853b640 1 --2- 172.21.15.154:0/3563497076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5632fca04c00 0x5632fca6e580 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x5632fd847830 tx=0x5632fd7f8600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=601fd7eb14b355c6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.696+0000 7f24b5d36640 1 -- 172.21.15.154:0/3563497076 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5632fca4d600 con 0x5632fca04c00 2023-09-23T03:21:35.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.695+0000 7f42fcc4a5c0 1 -- start start 2023-09-23T03:21:35.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.695+0000 7f42fcc4a5c0 1 Processor -- start 2023-09-23T03:21:35.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.696+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/3563497076 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632fca38000 con 0x5632fca04c00 2023-09-23T03:21:35.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.696+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/3563497076 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5632fca38ea0 con 0x5632fca04c00 2023-09-23T03:21:35.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.696+0000 7f24ba63f5c0 4 waiting for MDS map... 2023-09-23T03:21:35.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.696+0000 7f24b5d36640 1 -- 172.21.15.154:0/3563497076 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5632fca2b800 con 0x5632fca04c00 2023-09-23T03:21:35.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.696+0000 7f24b5d36640 1 -- 172.21.15.154:0/3563497076 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5632fca4d1e0 con 0x5632fca04c00 2023-09-23T03:21:35.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.697+0000 7f24b5d36640 1 -- 172.21.15.154:0/3563497076 <== mon.2 v2:172.21.15.190:3301/0 4 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x5632fcac4c40 con 0x5632fca04c00 2023-09-23T03:21:35.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.697+0000 7f42fcc4a5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55694a0f8c00 0x55694a162580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.697+0000 7f42fcc4a5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55694a0f9800 0x55694a162b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.697+0000 7f42fcc4a5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55694a0f9c00 0x55694a163080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.697+0000 7f42fcc4a5c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55694a143180 con 0x55694a0f8c00 2023-09-23T03:21:35.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.697+0000 7f42fcc4a5c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55694a142140 con 0x55694a0f9c00 2023-09-23T03:21:35.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.697+0000 7f42fcc4a5c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55694a142000 con 0x55694a0f9800 2023-09-23T03:21:35.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.698+0000 7f24b5d36640 1 -- 172.21.15.154:0/3563497076 <== mon.2 v2:172.21.15.190:3301/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x5632fd8e8000 con 0x5632fca04c00 2023-09-23T03:21:35.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.697+0000 7f5ee172d640 1 -- 172.21.15.154:0/1042754689 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x557b4d830840 con 0x557b4db3d000 2023-09-23T03:21:35.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.698+0000 7f42fac32640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55694a0f8c00 0x55694a162580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.698+0000 7f42fa431640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55694a0f9800 0x55694a162b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.698+0000 7f42fa431640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55694a0f9800 0x55694a162b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:34210/0 (socket says 172.21.15.154:34210) 2023-09-23T03:21:35.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.698+0000 7f42fa431640 1 -- 172.21.15.154:0/1545018075 learned_addr learned my addr 172.21.15.154:0/1545018075 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.698+0000 7f24ba63f5c0 4 Got MDS map 47 2023-09-23T03:21:35.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.698+0000 7f42fb433640 1 --2- 172.21.15.154:0/1545018075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55694a0f9c00 0x55694a163080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.698+0000 7f24ba63f5c0 4 datascan.main: Using metadata pool output 2023-09-23T03:21:35.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.698+0000 7f24ba63f5c0 4 datascan.main: connecting to RADOS... 2023-09-23T03:21:35.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.698+0000 7f5ee172d640 1 --2- 172.21.15.154:0/1042754689 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x557b4dc4e000 0x557b4d86b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.698+0000 7f5ee172d640 1 -- 172.21.15.154:0/1042754689 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x557b4d8bea80 con 0x557b4db3d000 2023-09-23T03:21:35.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.699+0000 7f42fa431640 1 -- 172.21.15.154:0/1545018075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55694a0f9c00 msgr2=0x55694a163080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.699+0000 7f42fa431640 1 --2- 172.21.15.154:0/1545018075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55694a0f9c00 0x55694a163080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.699+0000 7f42fa431640 1 -- 172.21.15.154:0/1545018075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55694a0f8c00 msgr2=0x55694a162580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.699+0000 7f5ee8f915c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-09-23T03:21:35.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.699+0000 7f5ee8f915c0 4 datascan.main: data pool 'cephfs_data' has ID 9 2023-09-23T03:21:35.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.699+0000 7f5ee8f915c0 4 datascan.main: opening data pool 'cephfs_data' 2023-09-23T03:21:35.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.699+0000 7f5ee8f915c0 1 --2- 172.21.15.154:0/1042754689 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x557b4db3c800 0x557b4d869600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.699+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:1 8.14 8:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x557b4d859880 con 0x557b4db3c800 2023-09-23T03:21:35.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.699+0000 7f42fa431640 1 --2- 172.21.15.154:0/1545018075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55694a0f8c00 0x55694a162580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.699+0000 7f42fa431640 1 -- 172.21.15.154:0/1545018075 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55694a12cb60 con 0x55694a0f9800 2023-09-23T03:21:35.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.699+0000 7f5ee7f3a640 1 --2- 172.21.15.154:0/1042754689 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x557b4dc4e000 0x557b4d86b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.699+0000 7f42fac32640 1 --2- 172.21.15.154:0/1545018075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55694a0f8c00 0x55694a162580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:21:35.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.700+0000 7f42fa431640 1 --2- 172.21.15.154:0/1545018075 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55694a0f9800 0x55694a162b00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x55694af1f4d0 tx=0x55694aed6600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6825381126749afc server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.700+0000 7f910bc00640 1 -- 172.21.15.154:0/2694809311 <== mon.1 v2:172.21.15.190:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x5625b70f2b00 con 0x5625b70c0c00 2023-09-23T03:21:35.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.700+0000 7f24ba63f5c0 1 Processor -- start 2023-09-23T03:21:35.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.700+0000 7f42f842d640 1 -- 172.21.15.154:0/1545018075 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55694a141600 con 0x55694a0f9800 2023-09-23T03:21:35.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.700+0000 7f42f842d640 1 -- 172.21.15.154:0/1545018075 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55694a11f800 con 0x55694a0f9800 2023-09-23T03:21:35.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.700+0000 7f42f842d640 1 -- 172.21.15.154:0/1545018075 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55694a1411e0 con 0x55694a0f9800 2023-09-23T03:21:35.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.700+0000 7f24ba63f5c0 1 -- start start 2023-09-23T03:21:35.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.700+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/1545018075 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55694a12c340 con 0x55694a0f9800 2023-09-23T03:21:35.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.700+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/1545018075 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55694a12c000 con 0x55694a0f9800 2023-09-23T03:21:35.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.700+0000 7f42fcc4a5c0 4 waiting for MDS map... 2023-09-23T03:21:35.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.700+0000 7f5ee6f38640 1 --2- 172.21.15.154:0/1042754689 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x557b4db3c800 0x557b4d869600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.701+0000 7f910bc00640 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5625b751e000 0x5625b712d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.701+0000 7f910bc00640 1 -- 172.21.15.154:0/2694809311 <== mon.1 v2:172.21.15.190:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x5625b7180a80 con 0x5625b70c0c00 2023-09-23T03:21:35.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.701+0000 7f9113c415c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-09-23T03:21:35.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.701+0000 7f9113c415c0 4 datascan.main: data pool 'cephfs_data' has ID 9 2023-09-23T03:21:35.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.701+0000 7f9113c415c0 4 datascan.main: opening data pool 'cephfs_data' 2023-09-23T03:21:35.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.701+0000 7f9113c415c0 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5625b7412800 0x5625b712cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.701+0000 7f42f842d640 1 -- 172.21.15.154:0/1545018075 <== mon.2 v2:172.21.15.190:3301/0 4 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x55694a1b8c40 con 0x55694a0f9800 2023-09-23T03:21:35.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.701+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:1 8.14 8:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b711b880 con 0x5625b7412800 2023-09-23T03:21:35.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.701+0000 7f42f842d640 1 -- 172.21.15.154:0/1545018075 <== mon.2 v2:172.21.15.190:3301/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x55694a14af00 con 0x55694a0f9800 2023-09-23T03:21:35.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.701+0000 7f42fcc4a5c0 4 Got MDS map 47 2023-09-23T03:21:35.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.702+0000 7f42fcc4a5c0 4 datascan.main: Using metadata pool output 2023-09-23T03:21:35.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.702+0000 7f42fcc4a5c0 4 datascan.main: connecting to RADOS... 2023-09-23T03:21:35.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.702+0000 7f9111c0c640 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5625b751e000 0x5625b712d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.702+0000 7f5ee6f38640 1 --2- 172.21.15.154:0/1042754689 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x557b4db3c800 0x557b4d869600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.702+0000 7f5ee7f3a640 1 --2- 172.21.15.154:0/1042754689 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x557b4dc4e000 0x557b4d86b700 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x557b4d8f0780 tx=0x557b4dc60000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.702+0000 7f911140b640 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5625b7412800 0x5625b712cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.703+0000 7f42fcc4a5c0 1 Processor -- start 2023-09-23T03:21:35.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.703+0000 7f24ba63f5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632fd8e7000 0x5632fca6f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.703+0000 7f24ba63f5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5632fd8e7400 0x5632fca6fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.703+0000 7f42fcc4a5c0 1 -- start start 2023-09-23T03:21:35.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.703+0000 7f5ee6f38640 1 -- 172.21.15.154:0/1042754689 <== osd.4 v2:172.21.15.190:6808/717302641 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv107 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x557b4e6a2b40 con 0x557b4db3c800 2023-09-23T03:21:35.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.703+0000 7f9111c0c640 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5625b751e000 0x5625b712d700 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x5625b71b2120 tx=0x5625b73c4f60 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.703+0000 7f24b853b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5632fd8e7400 0x5632fca6fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.703+0000 7f24b853b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5632fd8e7400 0x5632fca6fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60754/0 (socket says 172.21.15.154:60754) 2023-09-23T03:21:35.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.703+0000 7f24b853b640 1 -- 172.21.15.154:0/396801176 learned_addr learned my addr 172.21.15.154:0/396801176 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.703+0000 7f24b8d3c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632fd8e7000 0x5632fca6f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.703+0000 7f5ee8f915c0 1 --2- 172.21.15.154:0/1042754689 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x557b4db3c400 0x557b4d86a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.704+0000 7f24ba63f5c0 1 --2- 172.21.15.154:0/396801176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5632fd8e7800 0x5632fca70100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.704+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/396801176 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632fca4ea00 con 0x5632fd8e7400 2023-09-23T03:21:35.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.704+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/396801176 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632fcad3180 con 0x5632fd8e7000 2023-09-23T03:21:35.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.704+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/396801176 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632fcad2dc0 con 0x5632fd8e7800 2023-09-23T03:21:35.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.704+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:2 8.1f 8:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x557b4d859c00 con 0x557b4db3c400 2023-09-23T03:21:35.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.704+0000 7f911140b640 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5625b7412800 0x5625b712cc00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.704+0000 7f42fcc4a5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55694afeb000 0x55694a163600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.704+0000 7f42fcc4a5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55694afeb400 0x55694a163b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.704+0000 7f42fcc4a5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55694afeb800 0x55694a164100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.704+0000 7f42fcc4a5c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55694a142a00 con 0x55694afeb800 2023-09-23T03:21:35.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.704+0000 7f42fcc4a5c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55694a1c7180 con 0x55694afeb000 2023-09-23T03:21:35.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.704+0000 7f42fcc4a5c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55694a1c6dc0 con 0x55694afeb400 2023-09-23T03:21:35.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.704+0000 7f24b7d3a640 1 --2- 172.21.15.154:0/396801176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5632fd8e7800 0x5632fca70100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.704+0000 7f5ee7f3a640 1 --2- 172.21.15.154:0/1042754689 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x557b4db3c400 0x557b4d86a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.705+0000 7f5ee7f3a640 1 --2- 172.21.15.154:0/1042754689 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x557b4db3c400 0x557b4d86a100 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.705+0000 7f42fa431640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55694afeb800 0x55694a164100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.705+0000 7f42fa431640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55694afeb800 0x55694a164100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60766/0 (socket says 172.21.15.154:60766) 2023-09-23T03:21:35.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.705+0000 7f42fa431640 1 -- 172.21.15.154:0/3440096348 learned_addr learned my addr 172.21.15.154:0/3440096348 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.705+0000 7f911140b640 1 -- 172.21.15.154:0/2694809311 <== osd.4 v2:172.21.15.190:6808/717302641 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv107 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x5625b7f5eb40 con 0x5625b7412800 2023-09-23T03:21:35.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.705+0000 7f9113c415c0 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5625b7412400 0x5625b712d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.705+0000 7f42fb433640 1 --2- 172.21.15.154:0/3440096348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55694afeb000 0x55694a163600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.705+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:2 8.1f 8:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b711bc00 con 0x5625b7412400 2023-09-23T03:21:35.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.705+0000 7f24b8d3c640 1 -- 172.21.15.154:0/396801176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5632fd8e7800 msgr2=0x5632fca70100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.705+0000 7f24b8d3c640 1 --2- 172.21.15.154:0/396801176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5632fd8e7800 0x5632fca70100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.705+0000 7f24b8d3c640 1 -- 172.21.15.154:0/396801176 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5632fd8e7400 msgr2=0x5632fca6fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.705+0000 7f24b8d3c640 1 --2- 172.21.15.154:0/396801176 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5632fd8e7400 0x5632fca6fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.705+0000 7f24b8d3c640 1 -- 172.21.15.154:0/396801176 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632fca39520 con 0x5632fd8e7000 2023-09-23T03:21:35.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.705+0000 7f5ee7f3a640 1 -- 172.21.15.154:0/1042754689 <== osd.0 v2:172.21.15.154:6800/2920938138 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x557b4e6a2d80 con 0x557b4db3c400 2023-09-23T03:21:35.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.705+0000 7f5ee8f915c0 1 --2- 172.21.15.154:0/1042754689 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x557b4db3c000 0x557b4d869b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.706+0000 7f911240d640 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5625b7412400 0x5625b712d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.706+0000 7f42fa431640 1 -- 172.21.15.154:0/3440096348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55694afeb400 msgr2=0x55694a163b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-23T03:21:35.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.706+0000 7f42fa431640 1 --2- 172.21.15.154:0/3440096348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55694afeb400 0x55694a163b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.706+0000 7f42fa431640 1 -- 172.21.15.154:0/3440096348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55694afeb000 msgr2=0x55694a163600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.706+0000 7f42fa431640 1 --2- 172.21.15.154:0/3440096348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55694afeb000 0x55694a163600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.706+0000 7f42fa431640 1 -- 172.21.15.154:0/3440096348 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55694a12cea0 con 0x55694afeb800 2023-09-23T03:21:35.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.706+0000 7f42fb433640 1 --2- 172.21.15.154:0/3440096348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55694afeb000 0x55694a163600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:21:35.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.706+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:3 9.0 9:00000000::::head [pgnls-filter start_epoch 67 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x557b4dc78000 con 0x557b4db3c000 2023-09-23T03:21:35.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.706+0000 7f24b8d3c640 1 --2- 172.21.15.154:0/396801176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632fd8e7000 0x5632fca6f600 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x5632fd776d50 tx=0x5632fd8eec00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc5ca6055bbf9c7e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.706+0000 7f5ee7739640 1 --2- 172.21.15.154:0/1042754689 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x557b4db3c000 0x557b4d869b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.706+0000 7f18e20b65c0 1 -- start start 2023-09-23T03:21:35.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.706+0000 7f24b3d32640 1 -- 172.21.15.154:0/396801176 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5632fca4d600 con 0x5632fd8e7000 2023-09-23T03:21:35.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.706+0000 7f24b3d32640 1 -- 172.21.15.154:0/396801176 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5632fca2b800 con 0x5632fd8e7000 2023-09-23T03:21:35.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.707+0000 7f24b3d32640 1 -- 172.21.15.154:0/396801176 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5632fca4d1e0 con 0x5632fd8e7000 2023-09-23T03:21:35.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.706+0000 7f42fa431640 1 --2- 172.21.15.154:0/3440096348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55694afeb800 0x55694a164100 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x55694aeea8a0 tx=0x55694aed6420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9817223c4d37428a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.706+0000 7f18e20b65c0 1 Processor -- start 2023-09-23T03:21:35.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.707+0000 7f42f6429640 1 -- 172.21.15.154:0/3440096348 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55694a141080 con 0x55694afeb800 2023-09-23T03:21:35.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.707+0000 7f911240d640 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5625b7412400 0x5625b712d180 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.707+0000 7f42f6429640 1 -- 172.21.15.154:0/3440096348 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55694aed2c00 con 0x55694afeb800 2023-09-23T03:21:35.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.707+0000 7f42f6429640 1 -- 172.21.15.154:0/3440096348 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55694a141600 con 0x55694afeb800 2023-09-23T03:21:35.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.708+0000 7f911240d640 1 -- 172.21.15.154:0/2694809311 <== osd.0 v2:172.21.15.154:6800/2920938138 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x5625b7f5ed80 con 0x5625b7412400 2023-09-23T03:21:35.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.708+0000 7f9113c415c0 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5625b7412000 0x5625b753e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.708+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:3 9.0 9:00000000::::head [pgnls-filter start_epoch 67 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b7542000 con 0x5625b7412000 2023-09-23T03:21:35.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.708+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/396801176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632fd8e7000 msgr2=0x5632fca6f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.708+0000 7f24ba63f5c0 1 --2- 172.21.15.154:0/396801176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632fd8e7000 0x5632fca6f600 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x5632fd776d50 tx=0x5632fd8eec00 comp rx=0 tx=0).stop 2023-09-23T03:21:35.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.708+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/396801176 shutdown_connections 2023-09-23T03:21:35.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.707+0000 7f5ee7739640 1 --2- 172.21.15.154:0/1042754689 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x557b4db3c000 0x557b4d869b80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.708+0000 7f5ee7739640 1 -- 172.21.15.154:0/1042754689 <== osd.1 v2:172.21.15.154:6816/1267710005 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 67 out=81b] v67'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557b4e6a2fc0 con 0x557b4db3c000 2023-09-23T03:21:35.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.708+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:4 9.36 9:6db6db6d::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x557b4dc78380 con 0x557b4db3c000 2023-09-23T03:21:35.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.708+0000 7f5ee7739640 1 -- 172.21.15.154:0/1042754689 <== osd.1 v2:172.21.15.154:6816/1267710005 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 67 out=101b] v67'23 uv23 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x557b4e6a2fc0 con 0x557b4db3c000 2023-09-23T03:21:35.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.708+0000 7f5ee8f915c0 20 datascan.forall_objects: OSD matched oid 100000001f9.00000000 2023-09-23T03:21:35.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.708+0000 7f5ee8f915c0 10 datascan.operator(): handling object 100000001f9.0 2023-09-23T03:21:35.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.708+0000 7f24ba63f5c0 1 --2- 172.21.15.154:0/396801176 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5632fd8e7800 0x5632fca70100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.708+0000 7f24ba63f5c0 1 --2- 172.21.15.154:0/396801176 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5632fd8e7400 0x5632fca6fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.708+0000 7f24ba63f5c0 1 --2- 172.21.15.154:0/396801176 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632fd8e7000 0x5632fca6f600 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.708+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/396801176 >> 172.21.15.154:0/396801176 conn(0x5632fd8e6c00 msgr2=0x5632fcb01800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:35.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.708+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:5 9.36 9:6ee5e2c4:::100000001f9.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x557b4dc78700 con 0x557b4db3c000 2023-09-23T03:21:35.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.708+0000 7f18e20b65c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576f6980c00 0x5576f69ea580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.709+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/396801176 shutdown_connections 2023-09-23T03:21:35.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.709+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/396801176 wait complete. 2023-09-23T03:21:35.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.709+0000 7f18e20b65c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5576f6981800 0x5576f69eab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.707+0000 7f649a4565c0 1 -- 172.21.15.154:0/3133182667 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55de25c07c00 msgr2=0x55de25c70580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.707+0000 7f649a4565c0 1 --2- 172.21.15.154:0/3133182667 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55de25c07c00 0x55de25c70580 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x55de25cf8600 tx=0x55de269e6180 comp rx=0 tx=0).stop 2023-09-23T03:21:35.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.708+0000 7f649a4565c0 1 -- 172.21.15.154:0/3133182667 shutdown_connections 2023-09-23T03:21:35.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.708+0000 7f649a4565c0 1 --2- 172.21.15.154:0/3133182667 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55de25c07c00 0x55de25c70580 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.708+0000 7f649a4565c0 1 -- 172.21.15.154:0/3133182667 >> 172.21.15.154:0/3133182667 conn(0x55de25c07000 msgr2=0x55de25cfe000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:35.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.708+0000 7f649a4565c0 1 -- 172.21.15.154:0/3133182667 shutdown_connections 2023-09-23T03:21:35.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.708+0000 7f649a4565c0 1 -- 172.21.15.154:0/3133182667 wait complete. 2023-09-23T03:21:35.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.709+0000 7f18e20b65c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5576f6981c00 0x5576f69eb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.708+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/3440096348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55694afeb800 msgr2=0x55694a164100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.708+0000 7f42fcc4a5c0 1 --2- 172.21.15.154:0/3440096348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55694afeb800 0x55694a164100 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x55694aeea8a0 tx=0x55694aed6420 comp rx=0 tx=0).stop 2023-09-23T03:21:35.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.708+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/3440096348 shutdown_connections 2023-09-23T03:21:35.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.708+0000 7f42fcc4a5c0 1 --2- 172.21.15.154:0/3440096348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55694afeb800 0x55694a164100 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.708+0000 7f42fcc4a5c0 1 --2- 172.21.15.154:0/3440096348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55694afeb400 0x55694a163b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.708+0000 7f42fcc4a5c0 1 --2- 172.21.15.154:0/3440096348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55694afeb000 0x55694a163600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.708+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/3440096348 >> 172.21.15.154:0/3440096348 conn(0x55694afeac00 msgr2=0x55694a1f5800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:35.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.709+0000 7f18e20b65c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576f69cb180 con 0x5576f6980c00 2023-09-23T03:21:35.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.709+0000 7f9111c0c640 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5625b7412000 0x5625b753e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.709+0000 7f18e085c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576f6980c00 0x5576f69ea580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.709+0000 7f18e085c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576f6980c00 0x5576f69ea580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60768/0 (socket says 172.21.15.154:60768) 2023-09-23T03:21:35.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.709+0000 7f18e085c640 1 -- 172.21.15.154:0/3234934361 learned_addr learned my addr 172.21.15.154:0/3234934361 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.709+0000 7f18e20b65c0 1 -- 172.21.15.154:0/3234934361 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576f69ca140 con 0x5576f6981c00 2023-09-23T03:21:35.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.709+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/3440096348 shutdown_connections 2023-09-23T03:21:35.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.709+0000 7f18e20b65c0 1 -- 172.21.15.154:0/3234934361 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5576f69ca000 con 0x5576f6981800 2023-09-23T03:21:35.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.709+0000 7f5ee7739640 1 -- 172.21.15.154:0/1042754689 <== osd.1 v2:172.21.15.154:6816/1267710005 3 ==== osd_op_reply(5 100000001f9.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv23 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x557b4e6a2fc0 con 0x557b4db3c000 2023-09-23T03:21:35.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.709+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/3440096348 wait complete. 2023-09-23T03:21:35.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.709+0000 7f18e005b640 1 --2- 172.21.15.154:0/3234934361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5576f6981800 0x5576f69eab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.709+0000 7f649a4565c0 1 -- start start 2023-09-23T03:21:35.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.709+0000 7f649a4565c0 1 Processor -- start 2023-09-23T03:21:35.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.710+0000 7f5ee8f915c0 1 --2- 172.21.15.154:0/1042754689 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x557b4dc4f400 0x557b4dc86000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.710+0000 7f18e105d640 1 --2- 172.21.15.154:0/3234934361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5576f6981c00 0x5576f69eb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.710+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:6 10.16 10:6ee5e2c4:::100000001f9.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x557b4dc78000 con 0x557b4dc4f400 2023-09-23T03:21:35.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.710+0000 7f9111c0c640 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5625b7412000 0x5625b753e000 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.710+0000 7f24ba63f5c0 1 Processor -- start 2023-09-23T03:21:35.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.711+0000 7f9111c0c640 1 -- 172.21.15.154:0/2694809311 <== osd.1 v2:172.21.15.154:6816/1267710005 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 67 out=81b] v67'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5625b7f5efc0 con 0x5625b7412000 2023-09-23T03:21:35.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.711+0000 7f24ba63f5c0 1 -- start start 2023-09-23T03:21:35.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.711+0000 7f42fcc4a5c0 1 Processor -- start 2023-09-23T03:21:35.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.711+0000 7f5ee6f38640 1 --2- 172.21.15.154:0/1042754689 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x557b4dc4f400 0x557b4dc86000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.711+0000 7f18e005b640 1 -- 172.21.15.154:0/3234934361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5576f6981c00 msgr2=0x5576f69eb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.711+0000 7f18e005b640 1 --2- 172.21.15.154:0/3234934361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5576f6981c00 0x5576f69eb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.711+0000 7f18e005b640 1 -- 172.21.15.154:0/3234934361 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576f6980c00 msgr2=0x5576f69ea580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.711+0000 7f18e005b640 1 --2- 172.21.15.154:0/3234934361 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576f6980c00 0x5576f69ea580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.711+0000 7f18e005b640 1 -- 172.21.15.154:0/3234934361 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576f69b51e0 con 0x5576f6981800 2023-09-23T03:21:35.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.711+0000 7f649a4565c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55de25c06c00 0x55de25c70580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.711+0000 7f649a4565c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55de25c07800 0x55de25c70b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.711+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:4 9.0 9:00000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b7542380 con 0x5625b7412000 2023-09-23T03:21:35.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.711+0000 7f42fcc4a5c0 1 -- start start 2023-09-23T03:21:35.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.711+0000 7f649a4565c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de25c07c00 0x55de25c71080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.711+0000 7f6498bfc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55de25c06c00 0x55de25c70580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.711+0000 7f6498bfc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55de25c06c00 0x55de25c70580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60782/0 (socket says 172.21.15.154:60782) 2023-09-23T03:21:35.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.711+0000 7f6498bfc640 1 -- 172.21.15.154:0/743418532 learned_addr learned my addr 172.21.15.154:0/743418532 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.711+0000 7f649a4565c0 1 -- 172.21.15.154:0/743418532 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de25c51180 con 0x55de25c06c00 2023-09-23T03:21:35.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.711+0000 7f649a4565c0 1 -- 172.21.15.154:0/743418532 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de25cd5180 con 0x55de25c07c00 2023-09-23T03:21:35.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.711+0000 7f649a4565c0 1 -- 172.21.15.154:0/743418532 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55de25cd4dc0 con 0x55de25c07800 2023-09-23T03:21:35.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.711+0000 7f18e005b640 1 --2- 172.21.15.154:0/3234934361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5576f6981800 0x5576f69eab00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x5576f77793e0 tx=0x5576f7830420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43e2c3e2a9cad334 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.712+0000 7f9111c0c640 1 -- 172.21.15.154:0/2694809311 <== osd.1 v2:172.21.15.154:6816/1267710005 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 67 out=81b] v67'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5625b7f5efc0 con 0x5625b7412000 2023-09-23T03:21:35.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.712+0000 7f64983fb640 1 --2- 172.21.15.154:0/743418532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55de25c07800 0x55de25c70b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.712+0000 7f5ee6f38640 1 --2- 172.21.15.154:0/1042754689 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x557b4dc4f400 0x557b4dc86000 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.712+0000 7f6498bfc640 1 -- 172.21.15.154:0/743418532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55de25c07800 msgr2=0x55de25c70b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.712+0000 7f6498bfc640 1 --2- 172.21.15.154:0/743418532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55de25c07800 0x55de25c70b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.712+0000 7f6498bfc640 1 -- 172.21.15.154:0/743418532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de25c07c00 msgr2=0x55de25c71080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.712+0000 7f6498bfc640 1 --2- 172.21.15.154:0/743418532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de25c07c00 0x55de25c71080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.712+0000 7f6498bfc640 1 -- 172.21.15.154:0/743418532 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de25c3ab60 con 0x55de25c06c00 2023-09-23T03:21:35.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.712+0000 7f24ba63f5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5632fd8e7000 0x5632fca71180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.712+0000 7f9113c415c0 20 datascan.forall_objects: OSD matched oid 10000000206.00000000 2023-09-23T03:21:35.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.712+0000 7f9113c415c0 10 datascan.operator(): handling object 10000000206.0 2023-09-23T03:21:35.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.712+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:5 9.0 9:012cdbc7:::10000000206.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b7542700 con 0x5625b7412000 2023-09-23T03:21:35.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.712+0000 7f24ba63f5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5632fd8e7400 0x5632fca70c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.713+0000 7f24ba63f5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632fd8e7800 0x5632fca70680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.712+0000 7f18de057640 1 -- 172.21.15.154:0/3234934361 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5576f69c9600 con 0x5576f6981800 2023-09-23T03:21:35.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.713+0000 7f18de057640 1 -- 172.21.15.154:0/3234934361 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5576f69a7b00 con 0x5576f6981800 2023-09-23T03:21:35.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.713+0000 7f18de057640 1 -- 172.21.15.154:0/3234934361 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5576f69c91e0 con 0x5576f6981800 2023-09-23T03:21:35.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.713+0000 7f9111c0c640 1 -- 172.21.15.154:0/2694809311 <== osd.1 v2:172.21.15.154:6816/1267710005 3 ==== osd_op_reply(5 10000000206.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x5625b7f5efc0 con 0x5625b7412000 2023-09-23T03:21:35.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.714+0000 7f9113c415c0 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x5625b751f800 0x5625b753e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.714+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:6 10.0 10:012cdbc7:::10000000206.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b7542000 con 0x5625b751f800 2023-09-23T03:21:35.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.713+0000 7f24ba63f5c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632fcad2c80 con 0x5632fd8e7000 2023-09-23T03:21:35.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.713+0000 7f24ba63f5c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632fca4f180 con 0x5632fd8e7800 2023-09-23T03:21:35.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.713+0000 7f24b8d3c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5632fd8e7000 0x5632fca71180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.713+0000 7f24ba63f5c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5632fca4eb40 con 0x5632fd8e7400 2023-09-23T03:21:35.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.713+0000 7f24b8d3c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5632fd8e7000 0x5632fca71180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60798/0 (socket says 172.21.15.154:60798) 2023-09-23T03:21:35.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.713+0000 7f24b8d3c640 1 -- 172.21.15.154:0/1322804614 learned_addr learned my addr 172.21.15.154:0/1322804614 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.713+0000 7f24b7d3a640 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632fd8e7800 0x5632fca70680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.714+0000 7f24b8d3c640 1 -- 172.21.15.154:0/1322804614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5632fd8e7400 msgr2=0x5632fca70c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.714+0000 7f24b8d3c640 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5632fd8e7400 0x5632fca70c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.714+0000 7f24b8d3c640 1 -- 172.21.15.154:0/1322804614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632fd8e7800 msgr2=0x5632fca70680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.714+0000 7f24b8d3c640 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632fd8e7800 0x5632fca70680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.713+0000 7f42fcc4a5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55694afeb000 0x55694a165180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.713+0000 7f42fcc4a5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55694afeb400 0x55694a164c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.713+0000 7f42fcc4a5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55694afeb800 0x55694a164680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.713+0000 7f42fcc4a5c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55694a1c6c80 con 0x55694afeb000 2023-09-23T03:21:35.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.713+0000 7f42fcc4a5c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55694a143180 con 0x55694afeb400 2023-09-23T03:21:35.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.713+0000 7f42fcc4a5c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55694a142b40 con 0x55694afeb800 2023-09-23T03:21:35.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.713+0000 7f42fb433640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55694afeb000 0x55694a165180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.714+0000 7f42fb433640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55694afeb000 0x55694a165180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60804/0 (socket says 172.21.15.154:60804) 2023-09-23T03:21:35.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.714+0000 7f42fb433640 1 -- 172.21.15.154:0/741382839 learned_addr learned my addr 172.21.15.154:0/741382839 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.714+0000 7f42fac32640 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55694afeb400 0x55694a164c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.714+0000 7f18e20b65c0 1 -- 172.21.15.154:0/3234934361 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5576f69b4b60 con 0x5576f6981800 2023-09-23T03:21:35.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.714+0000 7f18e20b65c0 1 -- 172.21.15.154:0/3234934361 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5576f69b4340 con 0x5576f6981800 2023-09-23T03:21:35.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.714+0000 7f18e20b65c0 4 waiting for MDS map... 2023-09-23T03:21:35.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.713+0000 7f6498bfc640 1 --2- 172.21.15.154:0/743418532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55de25c06c00 0x55de25c70580 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x55de26a1d470 tx=0x55de269e6240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5113a09f7adf665e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.714+0000 7f64963f7640 1 -- 172.21.15.154:0/743418532 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55de25c4f600 con 0x55de25c06c00 2023-09-23T03:21:35.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.714+0000 7f649a4565c0 1 -- 172.21.15.154:0/743418532 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de25c3a340 con 0x55de25c06c00 2023-09-23T03:21:35.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.714+0000 7f649a4565c0 1 -- 172.21.15.154:0/743418532 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55de25c3a000 con 0x55de25c06c00 2023-09-23T03:21:35.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.714+0000 7f649a4565c0 4 waiting for MDS map... 2023-09-23T03:21:35.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.714+0000 7f64963f7640 1 -- 172.21.15.154:0/743418532 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55de25c2ca80 con 0x55de25c06c00 2023-09-23T03:21:35.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.714+0000 7f64963f7640 1 -- 172.21.15.154:0/743418532 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55de25c4f1e0 con 0x55de25c06c00 2023-09-23T03:21:35.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.713+0000 7f21f4abb5c0 1 -- start start 2023-09-23T03:21:35.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.714+0000 7f21f4abb5c0 1 Processor -- start 2023-09-23T03:21:35.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.714+0000 7f24b8d3c640 1 -- 172.21.15.154:0/1322804614 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632fca38340 con 0x5632fd8e7000 2023-09-23T03:21:35.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.714+0000 7f5ee6f38640 1 -- 172.21.15.154:0/1042754689 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== osd_op_reply(6 100000001f9.00000000 [getxattr (30) out=30b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x557b4e6a2b40 con 0x557b4dc4f400 2023-09-23T03:21:35.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.714+0000 7f5ee8f915c0 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-09-23T03:21:35.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.714+0000 7f5ee8f915c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f9 2023-09-23T03:21:35.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.714+0000 7f5ee8f915c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/4 2023-09-23T03:21:35.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.714+0000 7f5ee8f915c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=4 2023-09-23T03:21:35.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.714+0000 7f5ee8f915c0 1 --2- 172.21.15.154:0/1042754689 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x557b4dc4f800 0x557b4dc86580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.714+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:7 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x557b4dc78380 con 0x557b4dc4f800 2023-09-23T03:21:35.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.714+0000 7f64963f7640 1 -- 172.21.15.154:0/743418532 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x55de25cc6c40 con 0x55de25c06c00 2023-09-23T03:21:35.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.714+0000 7f24b8d3c640 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5632fd8e7000 0x5632fca71180 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x5632fd777a10 tx=0x5632fd7f85a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.714+0000 7f64963f7640 1 -- 172.21.15.154:0/743418532 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x55de26abc000 con 0x55de25c06c00 2023-09-23T03:21:35.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.714+0000 7f911140b640 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x5625b751f800 0x5625b753e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.714+0000 7f42fa431640 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55694afeb800 0x55694a164680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.715+0000 7f649a4565c0 4 Got MDS map 47 2023-09-23T03:21:35.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.715+0000 7f24b252f640 1 -- 172.21.15.154:0/1322804614 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5632fca4cdc0 con 0x5632fd8e7000 2023-09-23T03:21:35.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.715+0000 7f21f4abb5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cd5a1cc00 0x562cd5a86580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.715+0000 7f24b252f640 1 -- 172.21.15.154:0/1322804614 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5632fd813500 con 0x5632fd8e7000 2023-09-23T03:21:35.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.715+0000 7f42fb433640 1 -- 172.21.15.154:0/741382839 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55694afeb800 msgr2=0x55694a164680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.715+0000 7f649a4565c0 4 datascan.main: Using metadata pool output 2023-09-23T03:21:35.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.715+0000 7f21f4abb5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562cd5a1d800 0x562cd5a86b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.715+0000 7f21f4abb5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562cd5a1dc00 0x562cd5a87080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.715+0000 7f42fb433640 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55694afeb800 0x55694a164680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.715+0000 7f649a4565c0 4 datascan.main: connecting to RADOS... 2023-09-23T03:21:35.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.715+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/1322804614 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5632fca38000 con 0x5632fd8e7000 2023-09-23T03:21:35.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.715+0000 7f18de057640 1 -- 172.21.15.154:0/3234934361 <== mon.2 v2:172.21.15.190:3301/0 4 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x5576f6a40c40 con 0x5576f6981800 2023-09-23T03:21:35.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.715+0000 7f18de057640 1 -- 172.21.15.154:0/3234934361 <== mon.2 v2:172.21.15.190:3301/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x5576f69d2f00 con 0x5576f6981800 2023-09-23T03:21:35.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.715+0000 7f21f4abb5c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cd5a67180 con 0x562cd5a1d800 2023-09-23T03:21:35.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.715+0000 7f21f4abb5c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cd5a66140 con 0x562cd5a1cc00 2023-09-23T03:21:35.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.715+0000 7f42fb433640 1 -- 172.21.15.154:0/741382839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55694afeb400 msgr2=0x55694a164c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.715+0000 7f5ee7f3a640 1 --2- 172.21.15.154:0/1042754689 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x557b4dc4f800 0x557b4dc86580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.715+0000 7f21f4abb5c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x562cd5a66000 con 0x562cd5a1dc00 2023-09-23T03:21:35.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.715+0000 7f42fb433640 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55694afeb400 0x55694a164c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.715+0000 7f18e20b65c0 4 Got MDS map 47 2023-09-23T03:21:35.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.715+0000 7f42fb433640 1 -- 172.21.15.154:0/741382839 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55694a12cb60 con 0x55694afeb000 2023-09-23T03:21:35.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.715+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/1322804614 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632fca396c0 con 0x5632fd8e7000 2023-09-23T03:21:35.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.715+0000 7f24ba63f5c0 4 datascan.init: resolving metadata pool 8 2023-09-23T03:21:35.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.715+0000 7f18e20b65c0 4 datascan.main: Using metadata pool output 2023-09-23T03:21:35.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.716+0000 7f18e20b65c0 4 datascan.main: connecting to RADOS... 2023-09-23T03:21:35.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.716+0000 7f24b252f640 1 -- 172.21.15.154:0/1322804614 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5632fca4d600 con 0x5632fd8e7000 2023-09-23T03:21:35.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.716+0000 7f42fb433640 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55694afeb000 0x55694a165180 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x55694aeeb5f0 tx=0x55694aff2e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.716+0000 7f21f3263640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cd5a1cc00 0x562cd5a86580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.716+0000 7f911140b640 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x5625b751f800 0x5625b753e580 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.716+0000 7f21f3a64640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562cd5a1dc00 0x562cd5a87080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.716+0000 7f42f4c26640 1 -- 172.21.15.154:0/741382839 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55694a141080 con 0x55694afeb000 2023-09-23T03:21:35.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.716+0000 7f649a4565c0 1 Processor -- start 2023-09-23T03:21:35.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.716+0000 7f21f3263640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cd5a1cc00 0x562cd5a86580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.154:49518/0 (socket says 172.21.15.154:49518) 2023-09-23T03:21:35.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.717+0000 7f42f4c26640 1 -- 172.21.15.154:0/741382839 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55694a11fb00 con 0x55694afeb000 2023-09-23T03:21:35.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.717+0000 7f5ee7f3a640 1 --2- 172.21.15.154:0/1042754689 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x557b4dc4f800 0x557b4dc86580 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.717+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55694a12c340 con 0x55694afeb000 2023-09-23T03:21:35.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.717+0000 7f649a4565c0 1 -- start start 2023-09-23T03:21:35.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.717+0000 7f18e20b65c0 1 Processor -- start 2023-09-23T03:21:35.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.716+0000 7f21f3263640 1 -- 172.21.15.154:0/2977390932 learned_addr learned my addr 172.21.15.154:0/2977390932 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.717+0000 7f911140b640 1 -- 172.21.15.154:0/2694809311 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== osd_op_reply(6 10000000206.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x5625b7f5eb40 con 0x5625b751f800 2023-09-23T03:21:35.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.717+0000 7f9113c415c0 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-09-23T03:21:35.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.717+0000 7f9113c415c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000206 2023-09-23T03:21:35.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.717+0000 7f9113c415c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/17 2023-09-23T03:21:35.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.717+0000 7f9113c415c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=17 2023-09-23T03:21:35.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.717+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55694a12c000 con 0x55694afeb000 2023-09-23T03:21:35.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.716+0000 7f21f2a62640 1 --2- 172.21.15.154:0/2977390932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562cd5a1d800 0x562cd5a86b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.717+0000 7f9113c415c0 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x5625b751fc00 0x5625b753eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.717+0000 7f21f2a62640 1 -- 172.21.15.154:0/2977390932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562cd5a1dc00 msgr2=0x562cd5a87080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.717+0000 7f21f2a62640 1 --2- 172.21.15.154:0/2977390932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562cd5a1dc00 0x562cd5a87080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.717+0000 7f21f2a62640 1 -- 172.21.15.154:0/2977390932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cd5a1cc00 msgr2=0x562cd5a86580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.717+0000 7f21f2a62640 1 --2- 172.21.15.154:0/2977390932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cd5a1cc00 0x562cd5a86580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.717+0000 7f42fcc4a5c0 4 datascan.init: resolving metadata pool 8 2023-09-23T03:21:35.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.717+0000 7f21f2a62640 1 -- 172.21.15.154:0/2977390932 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562cd5a50b60 con 0x562cd5a1d800 2023-09-23T03:21:35.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.717+0000 7f42f4c26640 1 -- 172.21.15.154:0/741382839 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55694a141600 con 0x55694afeb000 2023-09-23T03:21:35.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.717+0000 7f18e20b65c0 1 -- start start 2023-09-23T03:21:35.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.717+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:7 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b7542380 con 0x5625b751fc00 2023-09-23T03:21:35.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.718+0000 7f5ee7f3a640 1 -- 172.21.15.154:0/1042754689 <== osd.7 v2:172.21.15.190:6824/3460136272 1 ==== osd_op_reply(7 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x557b4e6a2d80 con 0x557b4dc4f800 2023-09-23T03:21:35.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.718+0000 7f5ee8f915c0 4 datascan.get_frag_of: Unexpected error on '10000000000.00000000': (2) No such file or directory 2023-09-23T03:21:35.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.718+0000 7f5ee8f915c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/4, will insert to root 2023-09-23T03:21:35.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.718+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:8 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x557b4dc78e00 con 0x557b4dc4f800 2023-09-23T03:21:35.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.718+0000 7f21f2a62640 1 --2- 172.21.15.154:0/2977390932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562cd5a1d800 0x562cd5a86b00 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x562cd682d350 tx=0x562cd5d064e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ce30c0729001de server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.718+0000 7f911240d640 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x5625b751fc00 0x5625b753eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.718+0000 7f24b252f640 1 -- 172.21.15.154:0/1322804614 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x5632fca378c0 con 0x5632fd8e7000 2023-09-23T03:21:35.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.718+0000 7f5ee7f3a640 1 -- 172.21.15.154:0/1042754689 <== osd.7 v2:172.21.15.190:6824/3460136272 2 ==== osd_op_reply(8 10000000000.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x557b4e6a2d80 con 0x557b4dc4f800 2023-09-23T03:21:35.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.718+0000 7f21f0a5e640 1 -- 172.21.15.154:0/2977390932 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562cd5a651e0 con 0x562cd5a1d800 2023-09-23T03:21:35.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.718+0000 7f649a4565c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55de26ad3000 0x55de25c71600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.719+0000 7f21f0a5e640 1 -- 172.21.15.154:0/2977390932 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562cd5a43b00 con 0x562cd5a1d800 2023-09-23T03:21:35.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.719+0000 7f649a4565c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de26ad3400 0x55de25c71b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.719+0000 7f18e20b65c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576f7849000 0x5576f69eb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.719+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/2977390932 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562cd5a50340 con 0x562cd5a1d800 2023-09-23T03:21:35.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.719+0000 7f24b252f640 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5632fd94c800 0x5632fca71700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.719+0000 7f18e20b65c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5576f7849400 0x5576f69ebb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.719+0000 7f649a4565c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55de26ad3800 0x55de25c72100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.719+0000 7f649a4565c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de25c50a00 con 0x55de26ad3800 2023-09-23T03:21:35.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.719+0000 7f24b252f640 1 -- 172.21.15.154:0/1322804614 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x5632fcac4a80 con 0x5632fd8e7000 2023-09-23T03:21:35.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.719+0000 7f24ba63f5c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-09-23T03:21:35.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.719+0000 7f649a4565c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de25c50b40 con 0x55de26ad3400 2023-09-23T03:21:35.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.719+0000 7f649a4565c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55de25cd5180 con 0x55de26ad3000 2023-09-23T03:21:35.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.719+0000 7f24ba63f5c0 4 datascan.main: data pool 'cephfs_data' has ID 9 2023-09-23T03:21:35.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.719+0000 7f24ba63f5c0 4 datascan.main: opening data pool 'cephfs_data' 2023-09-23T03:21:35.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.719+0000 7f18e20b65c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5576f7849800 0x5576f69ec100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.719+0000 7f18e20b65c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576f69caa00 con 0x5576f7849000 2023-09-23T03:21:35.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.719+0000 7f24ba63f5c0 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5632fd94cc00 0x5632fca70100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.719+0000 7f18e20b65c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576f6a4f180 con 0x5576f7849400 2023-09-23T03:21:35.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.719+0000 7f18e20b65c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5576f6a4edc0 con 0x5576f7849800 2023-09-23T03:21:35.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.719+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/1322804614 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:1 8.14 8:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5632fca5f880 con 0x5632fd94cc00 2023-09-23T03:21:35.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.719+0000 7f5ee8f915c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-09-23T03:21:35.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.719+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:9 8.10 8:0cf6ad0f:::10000000000.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x557b4dc79180 con 0x557b4dc4f800 2023-09-23T03:21:35.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.719+0000 7f18e105d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576f7849000 0x5576f69eb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.719+0000 7f18e105d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576f7849000 0x5576f69eb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60836/0 (socket says 172.21.15.154:60836) 2023-09-23T03:21:35.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.719+0000 7f18e105d640 1 -- 172.21.15.154:0/3986530590 learned_addr learned my addr 172.21.15.154:0/3986530590 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.719+0000 7f64983fb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55de26ad3800 0x55de25c72100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.719+0000 7f64983fb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55de26ad3800 0x55de25c72100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60838/0 (socket says 172.21.15.154:60838) 2023-09-23T03:21:35.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.719+0000 7f64983fb640 1 -- 172.21.15.154:0/1234265018 learned_addr learned my addr 172.21.15.154:0/1234265018 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.719+0000 7f64993fd640 1 --2- 172.21.15.154:0/1234265018 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55de26ad3000 0x55de25c71600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.719+0000 7f24b853b640 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5632fd94c800 0x5632fca71700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.720+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/2977390932 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x562cd5a50000 con 0x562cd5a1d800 2023-09-23T03:21:35.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.720+0000 7f911240d640 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x5625b751fc00 0x5625b753eb00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.720+0000 7f18e005b640 1 --2- 172.21.15.154:0/3986530590 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5576f7849800 0x5576f69ec100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.720+0000 7f18e105d640 1 -- 172.21.15.154:0/3986530590 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5576f7849800 msgr2=0x5576f69ec100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.720+0000 7f18e105d640 1 --2- 172.21.15.154:0/3986530590 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5576f7849800 0x5576f69ec100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.720+0000 7f18e105d640 1 -- 172.21.15.154:0/3986530590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5576f7849400 msgr2=0x5576f69ebb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.782 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.720+0000 7f18e105d640 1 --2- 172.21.15.154:0/3986530590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5576f7849400 0x5576f69ebb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.782 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.720+0000 7f18e105d640 1 -- 172.21.15.154:0/3986530590 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576f69b4000 con 0x5576f7849000 2023-09-23T03:21:35.782 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.720+0000 7f24b853b640 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5632fd94c800 0x5632fca71700 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x5632fd776c30 tx=0x5632fd7f8180 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.782 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.720+0000 7f21f4abb5c0 4 waiting for MDS map... 2023-09-23T03:21:35.782 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.720+0000 7f18e105d640 1 --2- 172.21.15.154:0/3986530590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576f7849000 0x5576f69eb600 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x5576f783c8d0 tx=0x5576f783ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5442e48b964acfda server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.720+0000 7f24b7d3a640 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5632fd94cc00 0x5632fca70100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.720+0000 7f21f0a5e640 1 -- 172.21.15.154:0/2977390932 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562cd5a65080 con 0x562cd5a1d800 2023-09-23T03:21:35.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.721+0000 7f42f4c26640 1 -- 172.21.15.154:0/741382839 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x55694a12b080 con 0x55694afeb000 2023-09-23T03:21:35.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.721+0000 7f18dc053640 1 -- 172.21.15.154:0/3986530590 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5576f69c9600 con 0x5576f7849000 2023-09-23T03:21:35.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.721+0000 7f18dc053640 1 -- 172.21.15.154:0/3986530590 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5576f6afb080 con 0x5576f7849000 2023-09-23T03:21:35.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.720+0000 7f6498bfc640 1 --2- 172.21.15.154:0/1234265018 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de26ad3400 0x55de25c71b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.721+0000 7f18dc053640 1 -- 172.21.15.154:0/3986530590 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5576f69c91e0 con 0x5576f7849000 2023-09-23T03:21:35.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.720+0000 7f21f0a5e640 1 -- 172.21.15.154:0/2977390932 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x562cd5adcc40 con 0x562cd5a1d800 2023-09-23T03:21:35.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.721+0000 7f21f0a5e640 1 -- 172.21.15.154:0/2977390932 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x562cd5d1e000 con 0x562cd5a1d800 2023-09-23T03:21:35.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.721+0000 7f21f4abb5c0 4 Got MDS map 47 2023-09-23T03:21:35.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.721+0000 7f64983fb640 1 -- 172.21.15.154:0/1234265018 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55de26ad3000 msgr2=0x55de25c71600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.721+0000 7f64983fb640 1 --2- 172.21.15.154:0/1234265018 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55de26ad3000 0x55de25c71600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.721+0000 7f42f4c26640 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55694a270800 0x55694a165700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.721+0000 7f64983fb640 1 -- 172.21.15.154:0/1234265018 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de26ad3400 msgr2=0x55de25c71b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.721+0000 7f64983fb640 1 --2- 172.21.15.154:0/1234265018 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de26ad3400 0x55de25c71b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.721+0000 7f64983fb640 1 -- 172.21.15.154:0/1234265018 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de25c3aea0 con 0x55de26ad3800 2023-09-23T03:21:35.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.721+0000 7f42f4c26640 1 -- 172.21.15.154:0/741382839 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x55694a1b8a80 con 0x55694afeb000 2023-09-23T03:21:35.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.721+0000 7f18e20b65c0 1 -- 172.21.15.154:0/3986530590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576f7849000 msgr2=0x5576f69eb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.721+0000 7f18e20b65c0 1 --2- 172.21.15.154:0/3986530590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576f7849000 0x5576f69eb600 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x5576f783c8d0 tx=0x5576f783ec00 comp rx=0 tx=0).stop 2023-09-23T03:21:35.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.722+0000 7f42fcc4a5c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-09-23T03:21:35.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.722+0000 7f42fcc4a5c0 4 datascan.main: data pool 'cephfs_data' has ID 9 2023-09-23T03:21:35.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.722+0000 7f42fcc4a5c0 4 datascan.main: opening data pool 'cephfs_data' 2023-09-23T03:21:35.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.722+0000 7f42fcc4a5c0 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55694a270c00 0x55694a164100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.721+0000 7f21f4abb5c0 4 datascan.main: Using metadata pool output 2023-09-23T03:21:35.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.722+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:1 8.14 8:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55694a153880 con 0x55694a270c00 2023-09-23T03:21:35.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.722+0000 7f18e20b65c0 1 -- 172.21.15.154:0/3986530590 shutdown_connections 2023-09-23T03:21:35.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.722+0000 7f18e20b65c0 1 --2- 172.21.15.154:0/3986530590 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5576f7849800 0x5576f69ec100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.722+0000 7f18e20b65c0 1 --2- 172.21.15.154:0/3986530590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5576f7849400 0x5576f69ebb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.722+0000 7f18e20b65c0 1 --2- 172.21.15.154:0/3986530590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576f7849000 0x5576f69eb600 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.722+0000 7f18e20b65c0 1 -- 172.21.15.154:0/3986530590 >> 172.21.15.154:0/3986530590 conn(0x5576f7848c00 msgr2=0x5576f6a7b000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:35.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.722+0000 7f18e20b65c0 1 -- 172.21.15.154:0/3986530590 shutdown_connections 2023-09-23T03:21:35.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.722+0000 7f64983fb640 1 --2- 172.21.15.154:0/1234265018 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55de26ad3800 0x55de25c72100 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x55de26ad8a20 tx=0x55de26adef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b15d78a219c9b62 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.722+0000 7f42fac32640 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55694a270800 0x55694a165700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.722+0000 7f64943f3640 1 -- 172.21.15.154:0/1234265018 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55de25c4f600 con 0x55de26ad3800 2023-09-23T03:21:35.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.721+0000 7f21f4abb5c0 4 datascan.main: connecting to RADOS... 2023-09-23T03:21:35.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.722+0000 7f64943f3640 1 -- 172.21.15.154:0/1234265018 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55de25c2ca80 con 0x55de26ad3800 2023-09-23T03:21:35.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.722+0000 7f21f4abb5c0 1 Processor -- start 2023-09-23T03:21:35.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.722+0000 7f21f4abb5c0 1 -- start start 2023-09-23T03:21:35.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.722+0000 7f64943f3640 1 -- 172.21.15.154:0/1234265018 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55de25c4f1e0 con 0x55de26ad3800 2023-09-23T03:21:35.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.722+0000 7f18e20b65c0 1 -- 172.21.15.154:0/3986530590 wait complete. 2023-09-23T03:21:35.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.722+0000 7f42fa431640 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55694a270c00 0x55694a164100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.722+0000 7f24b7d3a640 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5632fd94cc00 0x5632fca70100 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.723+0000 7f42fac32640 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55694a270800 0x55694a165700 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x55694ae70600 tx=0x55694aed62a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.723+0000 7f649a4565c0 1 -- 172.21.15.154:0/1234265018 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55de26ad3800 msgr2=0x55de25c72100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.723+0000 7f649a4565c0 1 --2- 172.21.15.154:0/1234265018 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55de26ad3800 0x55de25c72100 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x55de26ad8a20 tx=0x55de26adef00 comp rx=0 tx=0).stop 2023-09-23T03:21:35.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.723+0000 7f649a4565c0 1 -- 172.21.15.154:0/1234265018 shutdown_connections 2023-09-23T03:21:35.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.723+0000 7f649a4565c0 1 --2- 172.21.15.154:0/1234265018 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55de26ad3800 0x55de25c72100 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.723+0000 7f649a4565c0 1 --2- 172.21.15.154:0/1234265018 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de26ad3400 0x55de25c71b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.723+0000 7f649a4565c0 1 --2- 172.21.15.154:0/1234265018 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55de26ad3000 0x55de25c71600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.723+0000 7f649a4565c0 1 -- 172.21.15.154:0/1234265018 >> 172.21.15.154:0/1234265018 conn(0x55de26ad2c00 msgr2=0x55de25cff800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:35.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.723+0000 7f21f4abb5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cd5d1d000 0x562cd5a87600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.723+0000 7f649a4565c0 1 -- 172.21.15.154:0/1234265018 shutdown_connections 2023-09-23T03:21:35.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.723+0000 7f24b7d3a640 1 -- 172.21.15.154:0/1322804614 <== osd.4 v2:172.21.15.190:6808/717302641 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv107 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x5632fd76eb40 con 0x5632fd94cc00 2023-09-23T03:21:35.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.723+0000 7f649a4565c0 1 -- 172.21.15.154:0/1234265018 wait complete. 2023-09-23T03:21:35.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.723+0000 7f24ba63f5c0 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5632fd94d000 0x5632fca6fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.723+0000 7f18e20b65c0 1 Processor -- start 2023-09-23T03:21:35.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.724+0000 7f21f4abb5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562cd5d1d400 0x562cd5a87b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.724+0000 7f42fa431640 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55694a270c00 0x55694a164100 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.724+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/1322804614 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:2 8.1f 8:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5632fca5fc00 con 0x5632fd94d000 2023-09-23T03:21:35.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.724+0000 7f18e20b65c0 1 -- start start 2023-09-23T03:21:35.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.724+0000 7f5ee7f3a640 1 -- 172.21.15.154:0/1042754689 <== osd.7 v2:172.21.15.190:6824/3460136272 3 ==== osd_op_reply(9 10000000000.00000000 [create,omap-set-header] v67'16 uv16 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x557b4e6a2d80 con 0x557b4dc4f800 2023-09-23T03:21:35.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.724+0000 7f5ee8f915c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x10000000000 2023-09-23T03:21:35.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.724+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:10 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x557b4dc79500 con 0x557b4dc4f800 2023-09-23T03:21:35.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.724+0000 7f21f4abb5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562cd5d1d800 0x562cd5a88100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.724+0000 7f24b8d3c640 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5632fd94d000 0x5632fca6fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.724+0000 7f649a4565c0 1 Processor -- start 2023-09-23T03:21:35.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.724+0000 7f21f3a64640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cd5d1d000 0x562cd5a87600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.724+0000 7f21f3a64640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cd5d1d000 0x562cd5a87600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.154:49552/0 (socket says 172.21.15.154:49552) 2023-09-23T03:21:35.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.724+0000 7f21f3a64640 1 -- 172.21.15.154:0/2545182611 learned_addr learned my addr 172.21.15.154:0/2545182611 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.724+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/2545182611 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cd5a66a00 con 0x562cd5d1d800 2023-09-23T03:21:35.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.724+0000 7f649a4565c0 1 -- start start 2023-09-23T03:21:35.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.724+0000 7f21f3263640 1 --2- 172.21.15.154:0/2545182611 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562cd5d1d400 0x562cd5a87b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.725+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/2545182611 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cd5aeb180 con 0x562cd5d1d000 2023-09-23T03:21:35.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.725+0000 7f911240d640 1 -- 172.21.15.154:0/2694809311 <== osd.7 v2:172.21.15.190:6824/3460136272 1 ==== osd_op_reply(7 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x5625b7f5ed80 con 0x5625b751fc00 2023-09-23T03:21:35.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.725+0000 7f9113c415c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-09-23T03:21:35.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.725+0000 7f9113c415c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-09-23T03:21:35.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.725+0000 7f9113c415c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/17, will insert to root 2023-09-23T03:21:35.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.725+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/2545182611 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x562cd5aeadc0 con 0x562cd5d1d400 2023-09-23T03:21:35.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.725+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:8 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b7542e00 con 0x5625b751fc00 2023-09-23T03:21:35.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.725+0000 7f21f2a62640 1 --2- 172.21.15.154:0/2545182611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562cd5d1d800 0x562cd5a88100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.725+0000 7f24b8d3c640 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5632fd94d000 0x5632fca6fb80 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.725+0000 7f42fa431640 1 -- 172.21.15.154:0/741382839 <== osd.4 v2:172.21.15.190:6808/717302641 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv107 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x55694afb8b40 con 0x55694a270c00 2023-09-23T03:21:35.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.725+0000 7f42fcc4a5c0 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55694a271000 0x55694a163b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.725+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:2 8.1f 8:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55694a153c00 con 0x55694a271000 2023-09-23T03:21:35.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.725+0000 7f5ee7f3a640 1 -- 172.21.15.154:0/1042754689 <== osd.7 v2:172.21.15.190:6824/3460136272 4 ==== osd_op_reply(10 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x557b4e6a2d80 con 0x557b4dc4f800 2023-09-23T03:21:35.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.725+0000 7f5ee8f915c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=4 frag=*, r=0 2023-09-23T03:21:35.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.725+0000 7f5ee8f915c0 20 datascan.read_dentry: 4_head not found in result 2023-09-23T03:21:35.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.725+0000 7f5ee8f915c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f9 at 0x0x10000000000/4 with size=6291456 bytes 2023-09-23T03:21:35.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.725+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:11 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x557b4dc78e00 con 0x557b4dc4f800 2023-09-23T03:21:35.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.725+0000 7f24b8d3c640 1 -- 172.21.15.154:0/1322804614 <== osd.0 v2:172.21.15.154:6800/2920938138 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x5632fd76ed80 con 0x5632fd94d000 2023-09-23T03:21:35.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.725+0000 7f24ba63f5c0 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5632fd94d400 0x5632fca6f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.725+0000 7f42fb433640 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55694a271000 0x55694a163b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.725+0000 7f18e20b65c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5576f7849000 0x5576f69ed180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.725+0000 7f18e20b65c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5576f7849400 0x5576f69ecc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.726+0000 7f18e20b65c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576f7849800 0x5576f69ec680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.725+0000 7f21f2a62640 1 -- 172.21.15.154:0/2545182611 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562cd5d1d400 msgr2=0x562cd5a87b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.725+0000 7f21f2a62640 1 --2- 172.21.15.154:0/2545182611 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562cd5d1d400 0x562cd5a87b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.725+0000 7f21f2a62640 1 -- 172.21.15.154:0/2545182611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cd5d1d000 msgr2=0x562cd5a87600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:35.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.725+0000 7f21f2a62640 1 --2- 172.21.15.154:0/2545182611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cd5d1d000 0x562cd5a87600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.725+0000 7f21f2a62640 1 -- 172.21.15.154:0/2545182611 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562cd5a50ea0 con 0x562cd5d1d800 2023-09-23T03:21:35.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.726+0000 7f911240d640 1 -- 172.21.15.154:0/2694809311 <== osd.7 v2:172.21.15.190:6824/3460136272 2 ==== osd_op_reply(8 10000000000.00000000 [omap-get-header out=274b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5625b7f5ed80 con 0x5625b751fc00 2023-09-23T03:21:35.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.726+0000 7f649a4565c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55de26ad3000 0x55de25c73180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.726+0000 7f649a4565c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55de26ad3400 0x55de25c72c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.726+0000 7f18e20b65c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576f6a4ec80 con 0x5576f7849800 2023-09-23T03:21:35.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.726+0000 7f649a4565c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de26ad3800 0x55de25c72680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.726+0000 7f18e20b65c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5576f69cb180 con 0x5576f7849400 2023-09-23T03:21:35.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.726+0000 7f18e20b65c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5576f69cab40 con 0x5576f7849000 2023-09-23T03:21:35.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.726+0000 7f18e005b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576f7849800 0x5576f69ec680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.726+0000 7f24b853b640 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5632fd94d400 0x5632fca6f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.726+0000 7f18e005b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576f7849800 0x5576f69ec680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60854/0 (socket says 172.21.15.154:60854) 2023-09-23T03:21:35.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.726+0000 7f18e005b640 1 -- 172.21.15.154:0/1830651065 learned_addr learned my addr 172.21.15.154:0/1830651065 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.726+0000 7f9113c415c0 10 datascan.find_or_create_dirfrag: read_version = 16 2023-09-23T03:21:35.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.726+0000 7f9113c415c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-09-23T03:21:35.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.726+0000 7f649a4565c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de25cd4c80 con 0x55de26ad3400 2023-09-23T03:21:35.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.726+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:9 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b7543180 con 0x5625b751fc00 2023-09-23T03:21:35.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.726+0000 7f42fb433640 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55694a271000 0x55694a163b80 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.726+0000 7f649a4565c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de25cd4b40 con 0x55de26ad3800 2023-09-23T03:21:35.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.726+0000 7f649a4565c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55de25cd4a00 con 0x55de26ad3000 2023-09-23T03:21:35.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.726+0000 7f6498bfc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55de26ad3400 0x55de25c72c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.726+0000 7f6498bfc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55de26ad3400 0x55de25c72c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60858/0 (socket says 172.21.15.154:60858) 2023-09-23T03:21:35.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.726+0000 7f6498bfc640 1 -- 172.21.15.154:0/3049751456 learned_addr learned my addr 172.21.15.154:0/3049751456 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.726+0000 7f18e085c640 1 --2- 172.21.15.154:0/1830651065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5576f7849400 0x5576f69ecc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.727+0000 7f64993fd640 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55de26ad3000 0x55de25c73180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.727+0000 7f21f2a62640 1 --2- 172.21.15.154:0/2545182611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562cd5d1d800 0x562cd5a88100 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x562cd5d22810 tx=0x562cd5d06360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f367adeae9fb52bd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.726+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/1322804614 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:3 9.0 9:00000000::::head [pgnls-filter start_epoch 67 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5632fcd8c000 con 0x5632fd94d400 2023-09-23T03:21:35.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.727+0000 7f42fb433640 1 -- 172.21.15.154:0/741382839 <== osd.0 v2:172.21.15.154:6800/2920938138 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x55694afb8d80 con 0x55694a271000 2023-09-23T03:21:35.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.727+0000 7f42fcc4a5c0 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55694a271400 0x55694a163600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.727+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:3 9.0 9:00000000::::head [pgnls-filter start_epoch 67 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55694a484000 con 0x55694a271400 2023-09-23T03:21:35.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.727+0000 7f18e005b640 1 -- 172.21.15.154:0/1830651065 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5576f7849000 msgr2=0x5576f69ed180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.727+0000 7f18e105d640 1 --2- 172.21.15.154:0/1830651065 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5576f7849000 0x5576f69ed180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.727+0000 7f18e005b640 1 --2- 172.21.15.154:0/1830651065 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5576f7849000 0x5576f69ed180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.727+0000 7f18e005b640 1 -- 172.21.15.154:0/1830651065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5576f7849400 msgr2=0x5576f69ecc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.727+0000 7f18e005b640 1 --2- 172.21.15.154:0/1830651065 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5576f7849400 0x5576f69ecc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.727+0000 7f18e005b640 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5576f69b51e0 con 0x5576f7849800 2023-09-23T03:21:35.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.727+0000 7f18e105d640 1 --2- 172.21.15.154:0/1830651065 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5576f7849000 0x5576f69ed180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:35.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.727+0000 7f64983fb640 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de26ad3800 0x55de25c72680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.727+0000 7f6498bfc640 1 -- 172.21.15.154:0/3049751456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55de26ad3000 msgr2=0x55de25c73180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.727+0000 7f6498bfc640 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55de26ad3000 0x55de25c73180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.727+0000 7f21eea5a640 1 -- 172.21.15.154:0/2545182611 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562cd5a64f20 con 0x562cd5d1d800 2023-09-23T03:21:35.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.727+0000 7f21eea5a640 1 -- 172.21.15.154:0/2545182611 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562cd67fad80 con 0x562cd5d1d800 2023-09-23T03:21:35.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.727+0000 7f21eea5a640 1 -- 172.21.15.154:0/2545182611 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562cd5a651e0 con 0x562cd5d1d800 2023-09-23T03:21:35.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.727+0000 7f6498bfc640 1 -- 172.21.15.154:0/3049751456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de26ad3800 msgr2=0x55de25c72680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.727+0000 7f24b853b640 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5632fd94d400 0x5632fca6f600 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.728+0000 7f42fac32640 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55694a271400 0x55694a163600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.727+0000 7f18e005b640 1 --2- 172.21.15.154:0/1830651065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576f7849800 0x5576f69ec680 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x5576f783d530 tx=0x5576f7830360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.727+0000 7f6498bfc640 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de26ad3800 0x55de25c72680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.728+0000 7f18da850640 1 -- 172.21.15.154:0/1830651065 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5576f69c8f20 con 0x5576f7849800 2023-09-23T03:21:35.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.728+0000 7f18da850640 1 -- 172.21.15.154:0/1830651065 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5576f6afa900 con 0x5576f7849800 2023-09-23T03:21:35.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.728+0000 7f18da850640 1 -- 172.21.15.154:0/1830651065 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5576f69c9600 con 0x5576f7849800 2023-09-23T03:21:35.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.728+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5576f69b4b60 con 0x5576f7849800 2023-09-23T03:21:35.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.728+0000 7f24b853b640 1 -- 172.21.15.154:0/1322804614 <== osd.1 v2:172.21.15.154:6816/1267710005 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 67 out=81b] v67'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5632fd76efc0 con 0x5632fd94d400 2023-09-23T03:21:35.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.727+0000 7f6498bfc640 1 -- 172.21.15.154:0/3049751456 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de25c3ab60 con 0x55de26ad3400 2023-09-23T03:21:35.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.727+0000 7f64983fb640 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de26ad3800 0x55de25c72680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:35.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.728+0000 7f6498bfc640 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55de26ad3400 0x55de25c72c00 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x55de26ad9650 tx=0x55de269e6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.728+0000 7f24ba63f5c0 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5632fd94d800 0x5632fcd96000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.728+0000 7f6492bf0640 1 -- 172.21.15.154:0/3049751456 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55de25c4ef20 con 0x55de26ad3400 2023-09-23T03:21:35.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.728+0000 7f6492bf0640 1 -- 172.21.15.154:0/3049751456 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55de26ac2a80 con 0x55de26ad3400 2023-09-23T03:21:35.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.728+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/2545182611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562cd5d1d800 msgr2=0x562cd5a88100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.728+0000 7f21f4abb5c0 1 --2- 172.21.15.154:0/2545182611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562cd5d1d800 0x562cd5a88100 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x562cd5d22810 tx=0x562cd5d06360 comp rx=0 tx=0).stop 2023-09-23T03:21:35.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.728+0000 7f42fac32640 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55694a271400 0x55694a163600 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.728+0000 7f6492bf0640 1 -- 172.21.15.154:0/3049751456 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55de25c4f600 con 0x55de26ad3400 2023-09-23T03:21:35.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.728+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55de25c3a340 con 0x55de26ad3400 2023-09-23T03:21:35.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.728+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5576f69b4340 con 0x5576f7849800 2023-09-23T03:21:35.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.728+0000 7f18e20b65c0 4 datascan.init: resolving metadata pool 8 2023-09-23T03:21:35.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.729+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de25c3b520 con 0x55de26ad3400 2023-09-23T03:21:35.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.729+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/1322804614 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:4 9.24 9:24924924::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5632fcd8c380 con 0x5632fd94d800 2023-09-23T03:21:35.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.729+0000 7f42fac32640 1 -- 172.21.15.154:0/741382839 <== osd.1 v2:172.21.15.154:6816/1267710005 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 67 out=81b] v67'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55694afb8fc0 con 0x55694a271400 2023-09-23T03:21:35.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.729+0000 7f649a4565c0 4 datascan.init: resolving metadata pool 8 2023-09-23T03:21:35.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.729+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/2545182611 shutdown_connections 2023-09-23T03:21:35.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.729+0000 7f21f4abb5c0 1 --2- 172.21.15.154:0/2545182611 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562cd5d1d800 0x562cd5a88100 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.729+0000 7f21f4abb5c0 1 --2- 172.21.15.154:0/2545182611 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562cd5d1d400 0x562cd5a87b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.729+0000 7f21f4abb5c0 1 --2- 172.21.15.154:0/2545182611 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cd5d1d000 0x562cd5a87600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.729+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/2545182611 >> 172.21.15.154:0/2545182611 conn(0x562cd5d1cc00 msgr2=0x562cd6797000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:35.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.728+0000 7f24b7d3a640 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5632fd94d800 0x5632fcd96000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.729+0000 7f42fcc4a5c0 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55694a271800 0x55694a48c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.729+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:4 9.12 9:49249249::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55694a484380 con 0x55694a271800 2023-09-23T03:21:35.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.729+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/2545182611 shutdown_connections 2023-09-23T03:21:35.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.729+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/2545182611 wait complete. 2023-09-23T03:21:35.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.729+0000 7f24b7d3a640 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5632fd94d800 0x5632fcd96000 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.730+0000 7f42fa431640 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55694a271800 0x55694a48c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.730+0000 7f24b7d3a640 1 -- 172.21.15.154:0/1322804614 <== osd.2 v2:172.21.15.154:6822/2884407243 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 67 out=49b] v65'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5632fd76eb40 con 0x5632fd94d800 2023-09-23T03:21:35.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.730+0000 7f24b3d32640 1 -- 172.21.15.154:0/1322804614 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:5 9.14 9:28000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5632fca5f880 con 0x5632fd94d400 2023-09-23T03:21:35.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.730+0000 7f5ee7f3a640 1 -- 172.21.15.154:0/1042754689 <== osd.7 v2:172.21.15.190:6824/3460136272 5 ==== osd_op_reply(11 10000000000.00000000 [omap-set-vals] v67'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x557b4e6a2d80 con 0x557b4dc4f800 2023-09-23T03:21:35.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.730+0000 7f5ee8f915c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/4 pointing to 0x0x100000001f9 2023-09-23T03:21:35.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.730+0000 7f5ee8f915c0 10 datascan.inject_with_backtrace: backptr: 0x0x1/subdir 2023-09-23T03:21:35.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.730+0000 7f5ee8f915c0 20 datascan.get_frag_of: dirino=0x1 target_dname=subdir 2023-09-23T03:21:35.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.730+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:12 8.1f 8:ff5b34d6:::1.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x557b4dc79180 con 0x557b4db3c400 2023-09-23T03:21:35.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.730+0000 7f911240d640 1 -- 172.21.15.154:0/2694809311 <== osd.7 v2:172.21.15.190:6824/3460136272 3 ==== osd_op_reply(9 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5625b7f5ed80 con 0x5625b751fc00 2023-09-23T03:21:35.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.730+0000 7f9113c415c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=17 frag=*, r=0 2023-09-23T03:21:35.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.730+0000 7f24b853b640 1 -- 172.21.15.154:0/1322804614 <== osd.1 v2:172.21.15.154:6816/1267710005 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 67 out=49b] v65'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5632fd76efc0 con 0x5632fd94d400 2023-09-23T03:21:35.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.730+0000 7f9113c415c0 20 datascan.read_dentry: 17_head not found in result 2023-09-23T03:21:35.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.730+0000 7f9113c415c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000206 at 0x0x10000000000/17 with size=6291456 bytes 2023-09-23T03:21:35.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.730+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:10 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b7543500 con 0x5625b751fc00 2023-09-23T03:21:35.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.731+0000 7f21f4abb5c0 1 Processor -- start 2023-09-23T03:21:35.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.731+0000 7f24b3531640 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x5632fcda4000 0x5632fcd98680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.731+0000 7f24b3531640 1 -- 172.21.15.154:0/1322804614 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:6 9.34 9:2c000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5632fcd8c380 con 0x5632fcda4000 2023-09-23T03:21:35.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.731+0000 7f5ee7f3a640 1 -- 172.21.15.154:0/1042754689 <== osd.0 v2:172.21.15.154:6800/2920938138 2 ==== osd_op_reply(12 1.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x557b4e6a2d80 con 0x557b4db3c400 2023-09-23T03:21:35.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.731+0000 7f5ee8f915c0 4 datascan.get_frag_of: Unexpected error on '1.00000000': (2) No such file or directory 2023-09-23T03:21:35.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.731+0000 7f5ee8f915c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x1/subdir, will insert to root 2023-09-23T03:21:35.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.731+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:13 8.1f 8:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x557b4dc79500 con 0x557b4db3c400 2023-09-23T03:21:35.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.731+0000 7f21f4abb5c0 1 -- start start 2023-09-23T03:21:35.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.731+0000 7f42fa431640 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55694a271800 0x55694a48c000 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.731+0000 7f5ee7f3a640 1 -- 172.21.15.154:0/1042754689 <== osd.0 v2:172.21.15.154:6800/2920938138 3 ==== osd_op_reply(13 1.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x557b4e6a2d80 con 0x557b4db3c400 2023-09-23T03:21:35.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.731+0000 7f5ee8f915c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-09-23T03:21:35.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.731+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:14 8.1f 8:ff5b34d6:::1.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x557b4dc78380 con 0x557b4db3c400 2023-09-23T03:21:35.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.731+0000 7f18da850640 1 -- 172.21.15.154:0/1830651065 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x5576f69b2840 con 0x5576f7849800 2023-09-23T03:21:35.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.732+0000 7f6492bf0640 1 -- 172.21.15.154:0/3049751456 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x55de25c39080 con 0x55de26ad3400 2023-09-23T03:21:35.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.732+0000 7f24b8d3c640 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x5632fcda4000 0x5632fcd98680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.732+0000 7f6492bf0640 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55de26b43800 0x55de25c73700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.732+0000 7f18da850640 1 --2- 172.21.15.154:0/1830651065 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5576f78a5800 0x5576f69ed700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.732+0000 7f18da850640 1 -- 172.21.15.154:0/1830651065 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x5576f6a40a80 con 0x5576f7849800 2023-09-23T03:21:35.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.732+0000 7f21f4abb5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cd5d1d000 0x562cd5a89180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.732+0000 7f42fa431640 1 -- 172.21.15.154:0/741382839 <== osd.5 v2:172.21.15.190:6800/882536885 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 67 out=49b] v65'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55694afb8b40 con 0x55694a271800 2023-09-23T03:21:35.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.732+0000 7f42f6429640 1 -- 172.21.15.154:0/741382839 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:5 9.32 9:4c000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55694a484e00 con 0x55694a270c00 2023-09-23T03:21:35.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.732+0000 7f18e20b65c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-09-23T03:21:35.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.732+0000 7f18e20b65c0 4 datascan.main: data pool 'cephfs_data' has ID 9 2023-09-23T03:21:35.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.732+0000 7f18e20b65c0 4 datascan.main: opening data pool 'cephfs_data' 2023-09-23T03:21:35.828 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.732+0000 7f21f4abb5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562cd5d1d400 0x562cd5a88c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.828 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.733+0000 7f18e20b65c0 1 --2- 172.21.15.154:0/1830651065 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5576f78a5c00 0x5576f69ec100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.828 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.733+0000 7f21f4abb5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562cd5d1d800 0x562cd5a88680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.733+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:1 8.14 8:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5576f69db880 con 0x5576f78a5c00 2023-09-23T03:21:35.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.732+0000 7f6492bf0640 1 -- 172.21.15.154:0/3049751456 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x55de25cc6a80 con 0x55de26ad3400 2023-09-23T03:21:35.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.732+0000 7f64993fd640 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55de26b43800 0x55de25c73700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.733+0000 7f649a4565c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-09-23T03:21:35.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.733+0000 7f649a4565c0 4 datascan.main: data pool 'cephfs_data' has ID 9 2023-09-23T03:21:35.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.733+0000 7f649a4565c0 4 datascan.main: opening data pool 'cephfs_data' 2023-09-23T03:21:35.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.733+0000 7f649a4565c0 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55de26b43c00 0x55de25c72100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.733+0000 7f21f4abb5c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cd5aeac80 con 0x562cd5d1d400 2023-09-23T03:21:35.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.733+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:1 8.14 8:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55de25c61880 con 0x55de26b43c00 2023-09-23T03:21:35.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.733+0000 7f21f3263640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562cd5d1d400 0x562cd5a88c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.733+0000 7f21f3263640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562cd5d1d400 0x562cd5a88c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60862/0 (socket says 172.21.15.154:60862) 2023-09-23T03:21:35.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.733+0000 7f21f3263640 1 -- 172.21.15.154:0/684274785 learned_addr learned my addr 172.21.15.154:0/684274785 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:35.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.733+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x562cd5a66140 con 0x562cd5d1d000 2023-09-23T03:21:35.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.733+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x562cd5a66b40 con 0x562cd5d1d800 2023-09-23T03:21:35.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.733+0000 7f18e105d640 1 --2- 172.21.15.154:0/1830651065 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5576f78a5800 0x5576f69ed700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.733+0000 7f21f3a64640 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cd5d1d000 0x562cd5a89180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.733+0000 7f64993fd640 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55de26b43800 0x55de25c73700 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x55de25cf8600 tx=0x55de26aded20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.733+0000 7f24b8d3c640 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x5632fcda4000 0x5632fcd98680 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.733+0000 7f21f2a62640 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562cd5d1d800 0x562cd5a88680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.734+0000 7f21f3263640 1 -- 172.21.15.154:0/684274785 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562cd5d1d800 msgr2=0x562cd5a88680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.734+0000 7f21f3263640 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562cd5d1d800 0x562cd5a88680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.734+0000 7f21f3263640 1 -- 172.21.15.154:0/684274785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cd5d1d000 msgr2=0x562cd5a89180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.734+0000 7f21f3263640 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cd5d1d000 0x562cd5a89180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.734+0000 7f21f3263640 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562cd5a50b60 con 0x562cd5d1d400 2023-09-23T03:21:35.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.734+0000 7f42fa431640 1 -- 172.21.15.154:0/741382839 <== osd.4 v2:172.21.15.190:6808/717302641 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 67 out=49b] v65'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55694afb8b40 con 0x55694a270c00 2023-09-23T03:21:35.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.734+0000 7f42f5c28640 1 -- 172.21.15.154:0/741382839 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:6 9.a 9:50000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55694a153880 con 0x55694a271800 2023-09-23T03:21:35.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.734+0000 7f18e105d640 1 --2- 172.21.15.154:0/1830651065 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5576f78a5800 0x5576f69ed700 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x5576f783c8d0 tx=0x5576f77117a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.734+0000 7f64983fb640 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55de26b43c00 0x55de25c72100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.734+0000 7f21f3263640 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562cd5d1d400 0x562cd5a88c00 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x562cd5d23590 tx=0x562cd5d06120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.734+0000 7f21ed257640 1 -- 172.21.15.154:0/684274785 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562cd5a65600 con 0x562cd5d1d400 2023-09-23T03:21:35.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.734+0000 7f21ed257640 1 -- 172.21.15.154:0/684274785 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562cd67fa300 con 0x562cd5d1d400 2023-09-23T03:21:35.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.734+0000 7f21ed257640 1 -- 172.21.15.154:0/684274785 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562cd5a64f20 con 0x562cd5d1d400 2023-09-23T03:21:35.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.734+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562cd5a50340 con 0x562cd5d1d400 2023-09-23T03:21:35.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.734+0000 7f24b8d3c640 1 -- 172.21.15.154:0/1322804614 <== osd.5 v2:172.21.15.190:6800/882536885 1 ==== osd_op_reply(6 [pgnls-filter start_epoch 67 out=49b] v65'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5632fd76ed80 con 0x5632fcda4000 2023-09-23T03:21:35.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.734+0000 7f24b3d32640 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5632fcda4400 0x5632fcd98c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.735+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562cd5a51520 con 0x562cd5d1d400 2023-09-23T03:21:35.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.735+0000 7f21f4abb5c0 4 datascan.init: resolving metadata pool 8 2023-09-23T03:21:35.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.734+0000 7f24b3d32640 1 -- 172.21.15.154:0/1322804614 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:7 9.c 9:30000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5632fcd8c380 con 0x5632fcda4400 2023-09-23T03:21:35.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.735+0000 7f18e085c640 1 --2- 172.21.15.154:0/1830651065 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5576f78a5c00 0x5576f69ec100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.735+0000 7f24b853b640 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5632fcda4400 0x5632fcd98c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.735+0000 7f42fa431640 1 -- 172.21.15.154:0/741382839 <== osd.5 v2:172.21.15.190:6800/882536885 2 ==== osd_op_reply(6 [pgnls-filter start_epoch 67 out=49b] v65'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55694afb8b40 con 0x55694a271800 2023-09-23T03:21:35.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.735+0000 7f42f6429640 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55694a494000 0x55694a48e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.735+0000 7f42f6429640 1 -- 172.21.15.154:0/741382839 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:7 9.2a 9:54000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55694a484380 con 0x55694a494000 2023-09-23T03:21:35.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.735+0000 7f911240d640 1 -- 172.21.15.154:0/2694809311 <== osd.7 v2:172.21.15.190:6824/3460136272 4 ==== osd_op_reply(10 10000000000.00000000 [omap-set-vals] v67'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5625b7f5ed80 con 0x5625b751fc00 2023-09-23T03:21:35.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.735+0000 7f9113c415c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/17 pointing to 0x0x10000000206 2023-09-23T03:21:35.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.735+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:11 9.20 9:04000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b7542380 con 0x5625b7412400 2023-09-23T03:21:35.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.735+0000 7f64983fb640 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55de26b43c00 0x55de25c72100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.736+0000 7f24b853b640 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5632fcda4400 0x5632fcd98c00 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.736+0000 7f5ee7f3a640 1 -- 172.21.15.154:0/1042754689 <== osd.0 v2:172.21.15.154:6800/2920938138 4 ==== osd_op_reply(14 1.00000000 [create,omap-set-header] v67'11 uv11 ondisk = 0) v8 ==== 196+0+0 (crc 0 0 0) 0x557b4e6a2d80 con 0x557b4db3c400 2023-09-23T03:21:35.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.736+0000 7f5ee8f915c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x1 2023-09-23T03:21:35.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.736+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:15 8.1f 8:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x557b4dc79880 con 0x557b4db3c400 2023-09-23T03:21:35.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.736+0000 7f911240d640 1 -- 172.21.15.154:0/2694809311 <== osd.0 v2:172.21.15.154:6800/2920938138 2 ==== osd_op_reply(11 [pgnls-filter start_epoch 67 out=81b] v67'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5625b7f5ed80 con 0x5625b7412400 2023-09-23T03:21:35.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.735+0000 7f42fb433640 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55694a494000 0x55694a48e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.736+0000 7f9113c415c0 20 datascan.forall_objects: OSD matched oid 1000000020b.00000000 2023-09-23T03:21:35.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.736+0000 7f24b853b640 1 -- 172.21.15.154:0/1322804614 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(7 [pgnls-filter start_epoch 67 out=81b] v67'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5632fd76efc0 con 0x5632fcda4400 2023-09-23T03:21:35.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.736+0000 7f9113c415c0 10 datascan.operator(): handling object 1000000020b.0 2023-09-23T03:21:35.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.736+0000 7f64983fb640 1 -- 172.21.15.154:0/3049751456 <== osd.4 v2:172.21.15.190:6808/717302641 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv107 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x55de26ab4b40 con 0x55de26b43c00 2023-09-23T03:21:35.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.736+0000 7f649a4565c0 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55de25f7c000 0x55de25c71b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.736+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:12 9.20 9:05552577:::1000000020b.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b7542e00 con 0x5625b7412400 2023-09-23T03:21:35.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.736+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:2 8.1f 8:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55de25c61c00 con 0x55de25f7c000 2023-09-23T03:21:35.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.736+0000 7f24ba63f5c0 20 datascan.forall_objects: OSD matched oid 100000001fc.00000000 2023-09-23T03:21:35.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.736+0000 7f24ba63f5c0 10 datascan.operator(): handling object 100000001fc.0 2023-09-23T03:21:35.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.736+0000 7f18e085c640 1 --2- 172.21.15.154:0/1830651065 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5576f78a5c00 0x5576f69ec100 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.736+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/1322804614 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:8 9.c 9:30604137:::100000001fc.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5632fcd8c700 con 0x5632fcda4400 2023-09-23T03:21:35.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.736+0000 7f5ee7f3a640 1 -- 172.21.15.154:0/1042754689 <== osd.0 v2:172.21.15.154:6800/2920938138 5 ==== osd_op_reply(15 1.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv11 ondisk = 0) v8 ==== 154+0+4 (crc 0 0 0) 0x557b4e6a2d80 con 0x557b4db3c400 2023-09-23T03:21:35.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.736+0000 7f5ee8f915c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:21:35.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.736+0000 7f5ee8f915c0 20 datascan.read_dentry: subdir_head not found in result 2023-09-23T03:21:35.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.736+0000 7f5ee8f915c0 10 datascan.inject_with_backtrace: Linking ancestor directory of inode 0x0x10000000000 at 0x0x1:subdir 2023-09-23T03:21:35.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.737+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:16 8.1f 8:ff5b34d6:::1.00000000:head [omap-set-vals in=507b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x557b4d859c00 con 0x557b4db3c400 2023-09-23T03:21:35.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.737+0000 7f24b853b640 1 -- 172.21.15.154:0/1322804614 <== osd.3 v2:172.21.15.154:6808/1133688468 2 ==== osd_op_reply(8 100000001fc.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv10 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x5632fd76efc0 con 0x5632fcda4400 2023-09-23T03:21:35.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.737+0000 7f911240d640 1 -- 172.21.15.154:0/2694809311 <== osd.0 v2:172.21.15.154:6800/2920938138 3 ==== osd_op_reply(12 1000000020b.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x5625b7f5ed80 con 0x5625b7412400 2023-09-23T03:21:35.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.737+0000 7f64993fd640 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55de25f7c000 0x55de25c71b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.737+0000 7f21ed257640 1 -- 172.21.15.154:0/684274785 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x562cd5a4f080 con 0x562cd5d1d400 2023-09-23T03:21:35.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.737+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:13 10.0 10:05552577:::1000000020b.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b7543880 con 0x5625b751f800 2023-09-23T03:21:35.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.737+0000 7f24ba63f5c0 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x5632fcda4800 0x5632fcd96580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.737+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/1322804614 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:9 10.c 10:30604137:::100000001fc.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5632fcd8c000 con 0x5632fcda4800 2023-09-23T03:21:35.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.737+0000 7f18e085c640 1 -- 172.21.15.154:0/1830651065 <== osd.4 v2:172.21.15.190:6808/717302641 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv107 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x5576f779ab40 con 0x5576f78a5c00 2023-09-23T03:21:35.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.737+0000 7f42fb433640 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55694a494000 0x55694a48e680 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.737+0000 7f18e20b65c0 1 --2- 172.21.15.154:0/1830651065 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5576f6ce4000 0x5576f69ebb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.738+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:2 8.1f 8:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5576f69dbc00 con 0x5576f6ce4000 2023-09-23T03:21:35.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.738+0000 7f42fb433640 1 -- 172.21.15.154:0/741382839 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(7 [pgnls-filter start_epoch 67 out=101b] v67'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55694afb8d80 con 0x55694a494000 2023-09-23T03:21:35.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.738+0000 7f42fcc4a5c0 20 datascan.forall_objects: OSD matched oid 1000000020a.00000000 2023-09-23T03:21:35.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.738+0000 7f42fcc4a5c0 10 datascan.operator(): handling object 1000000020a.0 2023-09-23T03:21:35.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.738+0000 7f18e105d640 1 --2- 172.21.15.154:0/1830651065 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5576f6ce4000 0x5576f69ebb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.738+0000 7f21ed257640 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x562cd5d1dc00 0x562cd5a89700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.848 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.738+0000 7f24b7d3a640 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x5632fcda4800 0x5632fcd96580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.848 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.738+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:8 9.2a 9:55a6cd83:::1000000020a.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55694a484700 con 0x55694a494000 2023-09-23T03:21:35.848 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.738+0000 7f64993fd640 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55de25f7c000 0x55de25c71b80 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.848 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.738+0000 7f64993fd640 1 -- 172.21.15.154:0/3049751456 <== osd.0 v2:172.21.15.154:6800/2920938138 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x55de26ab4d80 con 0x55de25f7c000 2023-09-23T03:21:35.848 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.738+0000 7f911140b640 1 -- 172.21.15.154:0/2694809311 <== osd.6 v2:172.21.15.190:6816/1964294001 2 ==== osd_op_reply(13 1000000020b.00000000 [getxattr (30) out=30b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x5625b7f5eb40 con 0x5625b751f800 2023-09-23T03:21:35.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.738+0000 7f9113c415c0 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-09-23T03:21:35.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.738+0000 7f9113c415c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000020b 2023-09-23T03:21:35.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.738+0000 7f9113c415c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/22 2023-09-23T03:21:35.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.738+0000 7f9113c415c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=22 2023-09-23T03:21:35.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.738+0000 7f21ed257640 1 -- 172.21.15.154:0/684274785 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x562cd5adca80 con 0x562cd5d1d400 2023-09-23T03:21:35.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.738+0000 7f21f3a64640 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x562cd5d1dc00 0x562cd5a89700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.738+0000 7f21f4abb5c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-09-23T03:21:35.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.738+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:14 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b711bc00 con 0x5625b751fc00 2023-09-23T03:21:35.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.739+0000 7f21f4abb5c0 4 datascan.main: data pool 'cephfs_data' has ID 9 2023-09-23T03:21:35.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.739+0000 7f21f4abb5c0 4 datascan.main: opening data pool 'cephfs_data' 2023-09-23T03:21:35.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.739+0000 7f21f4abb5c0 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x562cd5e88000 0x562cd5a88100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.739+0000 7f42fb433640 1 -- 172.21.15.154:0/741382839 <== osd.3 v2:172.21.15.154:6808/1133688468 2 ==== osd_op_reply(8 1000000020a.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x55694afb8d80 con 0x55694a494000 2023-09-23T03:21:35.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.738+0000 7f18e105d640 1 --2- 172.21.15.154:0/1830651065 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5576f6ce4000 0x5576f69ebb80 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.739+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:1 8.14 8:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd5a77880 con 0x562cd5e88000 2023-09-23T03:21:35.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.739+0000 7f42fcc4a5c0 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55694a494400 0x55694a48c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.739+0000 7f649a4565c0 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55de25f7c400 0x55de25c71600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.739+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:9 10.a 10:55a6cd83:::1000000020a.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55694a484380 con 0x55694a494400 2023-09-23T03:21:35.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.739+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:3 9.0 9:00000000::::head [pgnls-filter start_epoch 67 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55de25f86000 con 0x55de25f7c400 2023-09-23T03:21:35.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.739+0000 7f18e105d640 1 -- 172.21.15.154:0/1830651065 <== osd.0 v2:172.21.15.154:6800/2920938138 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x5576f779ad80 con 0x5576f6ce4000 2023-09-23T03:21:35.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.739+0000 7f18e20b65c0 1 --2- 172.21.15.154:0/1830651065 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5576f6ce4400 0x5576f69eb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.739+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:3 9.0 9:00000000::::head [pgnls-filter start_epoch 67 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5576f6cf0000 con 0x5576f6ce4400 2023-09-23T03:21:35.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.739+0000 7f6498bfc640 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55de25f7c400 0x55de25c71600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.739+0000 7f21f3a64640 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x562cd5d1dc00 0x562cd5a89700 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x562cd5d227e0 tx=0x562cd5d29f20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.739+0000 7f21f2a62640 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x562cd5e88000 0x562cd5a88100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.740+0000 7f18e085c640 1 --2- 172.21.15.154:0/1830651065 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5576f6ce4400 0x5576f69eb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.740+0000 7f911240d640 1 -- 172.21.15.154:0/2694809311 <== osd.7 v2:172.21.15.190:6824/3460136272 5 ==== osd_op_reply(14 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x5625b7f5ed80 con 0x5625b751fc00 2023-09-23T03:21:35.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.740+0000 7f9113c415c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-09-23T03:21:35.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.740+0000 7f9113c415c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-09-23T03:21:35.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.740+0000 7f9113c415c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/22, will insert to root 2023-09-23T03:21:35.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.740+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:15 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b711b880 con 0x5625b751fc00 2023-09-23T03:21:35.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.740+0000 7f24b7d3a640 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x5632fcda4800 0x5632fcd96580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.740+0000 7f42fac32640 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55694a494400 0x55694a48c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.740+0000 7f6498bfc640 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55de25f7c400 0x55de25c71600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.740+0000 7f5ee7f3a640 1 -- 172.21.15.154:0/1042754689 <== osd.0 v2:172.21.15.154:6800/2920938138 6 ==== osd_op_reply(16 1.00000000 [omap-set-vals] v67'12 uv12 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x557b4e6a2d80 con 0x557b4db3c400 2023-09-23T03:21:35.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.740+0000 7f5ee8f915c0 20 datascan.inject_linkage: Injected dentry 0x0x1/subdir pointing to 0x0x10000000000 2023-09-23T03:21:35.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.740+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:17 9.36 9:6fe38dc2::::head [pgnls-filter start_epoch 67 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x557b4dc78e00 con 0x557b4db3c000 2023-09-23T03:21:35.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.740+0000 7f18e085c640 1 --2- 172.21.15.154:0/1830651065 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5576f6ce4400 0x5576f69eb600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.741+0000 7f6498bfc640 1 -- 172.21.15.154:0/3049751456 <== osd.1 v2:172.21.15.154:6816/1267710005 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 67 out=81b] v67'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55de26ab4fc0 con 0x55de25f7c400 2023-09-23T03:21:35.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.741+0000 7f5ee7739640 1 -- 172.21.15.154:0/1042754689 <== osd.1 v2:172.21.15.154:6816/1267710005 4 ==== osd_op_reply(17 [pgnls-filter start_epoch 67 out=81b] v67'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557b4e6a2fc0 con 0x557b4db3c000 2023-09-23T03:21:35.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.741+0000 7f5ee8f915c0 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2023-09-23T03:21:35.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.741+0000 7f5ee8f915c0 10 datascan.operator(): handling object 100000001f7.0 2023-09-23T03:21:35.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.741+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:18 9.36 9:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x557b4dc79180 con 0x557b4db3c000 2023-09-23T03:21:35.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.741+0000 7f649a4565c0 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55de25f7c800 0x55de25f96000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.741+0000 7f911240d640 1 -- 172.21.15.154:0/2694809311 <== osd.7 v2:172.21.15.190:6824/3460136272 6 ==== osd_op_reply(15 10000000000.00000000 [omap-get-header out=274b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5625b7f5ed80 con 0x5625b751fc00 2023-09-23T03:21:35.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.741+0000 7f24b7d3a640 1 -- 172.21.15.154:0/1322804614 <== osd.7 v2:172.21.15.190:6824/3460136272 1 ==== osd_op_reply(9 100000001fc.00000000 [getxattr (30) out=30b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x5632fd76eb40 con 0x5632fcda4800 2023-09-23T03:21:35.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.741+0000 7f24ba63f5c0 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-09-23T03:21:35.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.741+0000 7f24ba63f5c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001fc 2023-09-23T03:21:35.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.741+0000 7f24ba63f5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/7 2023-09-23T03:21:35.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.741+0000 7f24ba63f5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=7 2023-09-23T03:21:35.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.741+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/1322804614 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:10 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5632fcd8c380 con 0x5632fcda4800 2023-09-23T03:21:35.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.741+0000 7f9113c415c0 10 datascan.find_or_create_dirfrag: read_version = 18 2023-09-23T03:21:35.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.741+0000 7f9113c415c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-09-23T03:21:35.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.741+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:16 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b7543180 con 0x5625b751fc00 2023-09-23T03:21:35.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.741+0000 7f42fac32640 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55694a494400 0x55694a48c580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.741+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:4 9.2d 9:b6db6db6::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55de25f86380 con 0x55de25f7c800 2023-09-23T03:21:35.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.741+0000 7f21f2a62640 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x562cd5e88000 0x562cd5a88100 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.741+0000 7f18e085c640 1 -- 172.21.15.154:0/1830651065 <== osd.1 v2:172.21.15.154:6816/1267710005 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 67 out=81b] v67'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5576f779ab40 con 0x5576f6ce4400 2023-09-23T03:21:35.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.741+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:4 9.9 9:92492492::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5576f6cf0380 con 0x5576f6ce4000 2023-09-23T03:21:35.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.741+0000 7f64983fb640 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55de25f7c800 0x55de25f96000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.741+0000 7f5ee7739640 1 -- 172.21.15.154:0/1042754689 <== osd.1 v2:172.21.15.154:6816/1267710005 5 ==== osd_op_reply(18 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv22 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x557b4e6a2fc0 con 0x557b4db3c000 2023-09-23T03:21:35.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.741+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:19 10.16 10:6fe38dc2:::100000001f7.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x557b4dc78380 con 0x557b4dc4f400 2023-09-23T03:21:35.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.742+0000 7f18e105d640 1 -- 172.21.15.154:0/1830651065 <== osd.0 v2:172.21.15.154:6800/2920938138 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 67 out=49b] v65'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5576f779ad80 con 0x5576f6ce4000 2023-09-23T03:21:35.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.742+0000 7f18dc053640 1 --2- 172.21.15.154:0/1830651065 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x5576f6ce5800 0x5576f6cfe000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.742+0000 7f18dc053640 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:5 9.29 9:94000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5576f69dbc00 con 0x5576f6ce5800 2023-09-23T03:21:35.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.742+0000 7f42fac32640 1 -- 172.21.15.154:0/741382839 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== osd_op_reply(9 1000000020a.00000000 [getxattr (30) out=30b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55694afb8fc0 con 0x55694a494400 2023-09-23T03:21:35.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.742+0000 7f64983fb640 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55de25f7c800 0x55de25f96000 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.742+0000 7f42fcc4a5c0 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-09-23T03:21:35.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.742+0000 7f42fcc4a5c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000020a 2023-09-23T03:21:35.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.742+0000 7f42fcc4a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/21 2023-09-23T03:21:35.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.742+0000 7f42fcc4a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=21 2023-09-23T03:21:35.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.742+0000 7f42fcc4a5c0 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55694a494800 0x55694a48cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.742+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:10 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55694a485180 con 0x55694a494800 2023-09-23T03:21:35.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.742+0000 7f21f2a62640 1 -- 172.21.15.154:0/684274785 <== osd.4 v2:172.21.15.190:6808/717302641 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv107 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x562cd5cfcb40 con 0x562cd5e88000 2023-09-23T03:21:35.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.742+0000 7f5ee6f38640 1 -- 172.21.15.154:0/1042754689 <== osd.6 v2:172.21.15.190:6816/1964294001 2 ==== osd_op_reply(19 100000001f7.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x557b4e6a2b40 con 0x557b4dc4f400 2023-09-23T03:21:35.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.742+0000 7f5ee8f915c0 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-09-23T03:21:35.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.742+0000 7f64983fb640 1 -- 172.21.15.154:0/3049751456 <== osd.2 v2:172.21.15.154:6822/2884407243 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 67 out=49b] v65'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55de26ab4b40 con 0x55de25f7c800 2023-09-23T03:21:35.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.742+0000 7f21f4abb5c0 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x562cd5e88400 0x562cd5a87b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.742+0000 7f5ee8f915c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2023-09-23T03:21:35.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.742+0000 7f5ee8f915c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/2 2023-09-23T03:21:35.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.742+0000 7f5ee8f915c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=2 2023-09-23T03:21:35.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.742+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:20 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x557b4dc79c00 con 0x557b4dc4f800 2023-09-23T03:21:35.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.742+0000 7f24b7d3a640 1 -- 172.21.15.154:0/1322804614 <== osd.7 v2:172.21.15.190:6824/3460136272 2 ==== osd_op_reply(10 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x5632fd76eb40 con 0x5632fcda4800 2023-09-23T03:21:35.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.742+0000 7f24ba63f5c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-09-23T03:21:35.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.742+0000 7f24ba63f5c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-09-23T03:21:35.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.742+0000 7f24ba63f5c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/7, will insert to root 2023-09-23T03:21:35.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.742+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/1322804614 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:11 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5632fcd8ce00 con 0x5632fcda4800 2023-09-23T03:21:35.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.742+0000 7f64943f3640 1 -- 172.21.15.154:0/3049751456 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:5 9.1d 9:b8000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55de25c61880 con 0x55de25f7c000 2023-09-23T03:21:35.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.742+0000 7f18e005b640 1 --2- 172.21.15.154:0/1830651065 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x5576f6ce5800 0x5576f6cfe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.742+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:2 8.1f 8:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd5a77c00 con 0x562cd5e88400 2023-09-23T03:21:35.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.743+0000 7f21f3a64640 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x562cd5e88400 0x562cd5a87b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.743+0000 7f911240d640 1 -- 172.21.15.154:0/2694809311 <== osd.7 v2:172.21.15.190:6824/3460136272 7 ==== osd_op_reply(16 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5625b7f5ed80 con 0x5625b751fc00 2023-09-23T03:21:35.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.743+0000 7f9113c415c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=22 frag=*, r=0 2023-09-23T03:21:35.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.743+0000 7f9113c415c0 20 datascan.read_dentry: 22_head not found in result 2023-09-23T03:21:35.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.743+0000 7f42fa431640 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55694a494800 0x55694a48cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.743+0000 7f9113c415c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000020b at 0x0x10000000000/22 with size=6291456 bytes 2023-09-23T03:21:35.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.743+0000 7f64993fd640 1 -- 172.21.15.154:0/3049751456 <== osd.0 v2:172.21.15.154:6800/2920938138 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 67 out=49b] v65'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55de26ab4d80 con 0x55de25f7c000 2023-09-23T03:21:35.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.743+0000 7f6493bf2640 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55de25f7d800 0x55de25f98680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.743+0000 7f6493bf2640 1 -- 172.21.15.154:0/3049751456 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:6 9.3d 9:bc000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55de25c61c00 con 0x55de25f7d800 2023-09-23T03:21:35.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.743+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:17 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b7543500 con 0x5625b751fc00 2023-09-23T03:21:35.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.743+0000 7f21f3a64640 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x562cd5e88400 0x562cd5a87b80 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.743+0000 7f5ee7f3a640 1 -- 172.21.15.154:0/1042754689 <== osd.7 v2:172.21.15.190:6824/3460136272 6 ==== osd_op_reply(20 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x557b4e6a2d80 con 0x557b4dc4f800 2023-09-23T03:21:35.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.743+0000 7f5ee8f915c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-09-23T03:21:35.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.743+0000 7f5ee8f915c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-09-23T03:21:35.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.743+0000 7f5ee8f915c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/2, will insert to root 2023-09-23T03:21:35.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.743+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:21 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x557b4dc78000 con 0x557b4dc4f800 2023-09-23T03:21:35.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.743+0000 7f21f3a64640 1 -- 172.21.15.154:0/684274785 <== osd.0 v2:172.21.15.154:6800/2920938138 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x562cd5cfcd80 con 0x562cd5e88400 2023-09-23T03:21:35.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.744+0000 7f21f4abb5c0 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x562cd5e88800 0x562cd5a87600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.744+0000 7f64993fd640 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55de25f7d800 0x55de25f98680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.744+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:3 9.0 9:00000000::::head [pgnls-filter start_epoch 67 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd5e94000 con 0x562cd5e88800 2023-09-23T03:21:35.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.744+0000 7f18e005b640 1 --2- 172.21.15.154:0/1830651065 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x5576f6ce5800 0x5576f6cfe000 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.744+0000 7f24b7d3a640 1 -- 172.21.15.154:0/1322804614 <== osd.7 v2:172.21.15.190:6824/3460136272 3 ==== osd_op_reply(11 10000000000.00000000 [omap-get-header out=274b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5632fd76eb40 con 0x5632fcda4800 2023-09-23T03:21:35.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.744+0000 7f24ba63f5c0 10 datascan.find_or_create_dirfrag: read_version = 18 2023-09-23T03:21:35.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.744+0000 7f24ba63f5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-09-23T03:21:35.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.744+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/1322804614 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:12 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5632fcd8d180 con 0x5632fcda4800 2023-09-23T03:21:35.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.744+0000 7f21f3263640 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x562cd5e88800 0x562cd5a87600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.744+0000 7f42fa431640 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55694a494800 0x55694a48cb00 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.744+0000 7f21f3263640 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x562cd5e88800 0x562cd5a87600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.744+0000 7f18e005b640 1 -- 172.21.15.154:0/1830651065 <== osd.7 v2:172.21.15.190:6824/3460136272 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 67 out=49b] v65'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5576f779afc0 con 0x5576f6ce5800 2023-09-23T03:21:35.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.745+0000 7f18db852640 1 --2- 172.21.15.154:0/1830651065 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5576f6ce5c00 0x5576f6cfe580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.745+0000 7f18db852640 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:6 9.19 9:98000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5576f69db880 con 0x5576f6ce5c00 2023-09-23T03:21:35.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.745+0000 7f21f3263640 1 -- 172.21.15.154:0/684274785 <== osd.1 v2:172.21.15.154:6816/1267710005 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 67 out=81b] v67'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562cd5cfcfc0 con 0x562cd5e88800 2023-09-23T03:21:35.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.745+0000 7f64993fd640 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55de25f7d800 0x55de25f98680 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.745+0000 7f18e105d640 1 --2- 172.21.15.154:0/1830651065 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5576f6ce5c00 0x5576f6cfe580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.745+0000 7f21f4abb5c0 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x562cd5e88c00 0x562cd5ea8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.745+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:4 9.1b 9:db6db6db::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd5e94380 con 0x562cd5e88c00 2023-09-23T03:21:35.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.745+0000 7f21f2a62640 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x562cd5e88c00 0x562cd5ea8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.746+0000 7f18e105d640 1 --2- 172.21.15.154:0/1830651065 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5576f6ce5c00 0x5576f6cfe580 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.746+0000 7f64993fd640 1 -- 172.21.15.154:0/3049751456 <== osd.7 v2:172.21.15.190:6824/3460136272 1 ==== osd_op_reply(6 [pgnls-filter start_epoch 67 out=81b] v67'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55de26ab4d80 con 0x55de25f7d800 2023-09-23T03:21:35.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.746+0000 7f649a4565c0 20 datascan.forall_objects: OSD matched oid 10000000205.00000000 2023-09-23T03:21:35.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.746+0000 7f649a4565c0 10 datascan.operator(): handling object 10000000205.0 2023-09-23T03:21:35.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.746+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:7 9.3d 9:bfcd3380:::10000000205.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55de25f86700 con 0x55de25f7d800 2023-09-23T03:21:35.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.746+0000 7f18e105d640 1 -- 172.21.15.154:0/1830651065 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(6 [pgnls-filter start_epoch 67 out=101b] v67'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5576f779ad80 con 0x5576f6ce5c00 2023-09-23T03:21:35.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.746+0000 7f18e20b65c0 20 datascan.forall_objects: OSD matched oid 10000000200.00000000 2023-09-23T03:21:35.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.746+0000 7f18e20b65c0 10 datascan.operator(): handling object 10000000200.0 2023-09-23T03:21:35.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.746+0000 7f21f2a62640 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x562cd5e88c00 0x562cd5ea8000 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.746+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:7 9.19 9:992a6246:::10000000200.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5576f6cf0700 con 0x5576f6ce5c00 2023-09-23T03:21:35.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.746+0000 7f18e105d640 1 -- 172.21.15.154:0/1830651065 <== osd.3 v2:172.21.15.154:6808/1133688468 2 ==== osd_op_reply(7 10000000200.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv18 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x5576f779ad80 con 0x5576f6ce5c00 2023-09-23T03:21:35.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.747+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:8 10.19 10:992a6246:::10000000200.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5576f6cf0000 con 0x5576f6ce4400 2023-09-23T03:21:35.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.747+0000 7f21f2a62640 1 -- 172.21.15.154:0/684274785 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 67 out=49b] v65'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562cd5cfcb40 con 0x562cd5e88c00 2023-09-23T03:21:35.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.747+0000 7f21eea5a640 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x562cd5eb0000 0x562cd5eaa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.747+0000 7f21eea5a640 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:5 9.3b 9:dc000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd5e94e00 con 0x562cd5eb0000 2023-09-23T03:21:35.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.747+0000 7f21f3a64640 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x562cd5eb0000 0x562cd5eaa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.747+0000 7f18e085c640 1 -- 172.21.15.154:0/1830651065 <== osd.1 v2:172.21.15.154:6816/1267710005 2 ==== osd_op_reply(8 10000000200.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x5576f779ab40 con 0x5576f6ce4400 2023-09-23T03:21:35.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.747+0000 7f911240d640 1 -- 172.21.15.154:0/2694809311 <== osd.7 v2:172.21.15.190:6824/3460136272 8 ==== osd_op_reply(17 10000000000.00000000 [omap-set-vals] v67'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5625b7f5ed80 con 0x5625b751fc00 2023-09-23T03:21:35.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.747+0000 7f9113c415c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/22 pointing to 0x0x1000000020b 2023-09-23T03:21:35.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.747+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:18 9.10 9:08000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b7542380 con 0x5625b7412800 2023-09-23T03:21:35.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.747+0000 7f64993fd640 1 -- 172.21.15.154:0/3049751456 <== osd.7 v2:172.21.15.190:6824/3460136272 2 ==== osd_op_reply(7 10000000205.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x55de26ab4d80 con 0x55de25f7d800 2023-09-23T03:21:35.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.747+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:8 10.1d 10:bfcd3380:::10000000205.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55de25f86700 con 0x55de26b43c00 2023-09-23T03:21:35.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.747+0000 7f18e20b65c0 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-09-23T03:21:35.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.747+0000 7f18e20b65c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000200 2023-09-23T03:21:35.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.747+0000 7f18e20b65c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/11 2023-09-23T03:21:35.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.747+0000 7f18e20b65c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=11 2023-09-23T03:21:35.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.747+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:9 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5576f6cf0380 con 0x5576f6ce5800 2023-09-23T03:21:35.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.748+0000 7f21f3a64640 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x562cd5eb0000 0x562cd5eaa680 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.748+0000 7f5ee7f3a640 1 -- 172.21.15.154:0/1042754689 <== osd.7 v2:172.21.15.190:6824/3460136272 7 ==== osd_op_reply(21 10000000000.00000000 [omap-get-header out=274b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x557b4e6a2d80 con 0x557b4dc4f800 2023-09-23T03:21:35.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.748+0000 7f5ee8f915c0 10 datascan.find_or_create_dirfrag: read_version = 19 2023-09-23T03:21:35.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.748+0000 7f5ee8f915c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-09-23T03:21:35.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.748+0000 7f21f3a64640 1 -- 172.21.15.154:0/684274785 <== osd.2 v2:172.21.15.154:6822/2884407243 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 67 out=81b] v67'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562cd5cfcd80 con 0x562cd5eb0000 2023-09-23T03:21:35.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.748+0000 7f21f4abb5c0 20 datascan.forall_objects: OSD matched oid 100000001fe.00000000 2023-09-23T03:21:35.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.748+0000 7f21f4abb5c0 10 datascan.operator(): handling object 100000001fe.0 2023-09-23T03:21:35.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.748+0000 7f911140b640 1 -- 172.21.15.154:0/2694809311 <== osd.4 v2:172.21.15.190:6808/717302641 2 ==== osd_op_reply(18 [pgnls-filter start_epoch 67 out=81b] v67'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5625b7f5eb40 con 0x5625b7412800 2023-09-23T03:21:35.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.748+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:22 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x557b4dc78380 con 0x557b4dc4f800 2023-09-23T03:21:35.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.748+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:6 9.3b 9:deccbd37:::100000001fe.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd5e94700 con 0x562cd5eb0000 2023-09-23T03:21:35.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.748+0000 7f9113c415c0 20 datascan.forall_objects: OSD matched oid 100000001fd.00000000 2023-09-23T03:21:35.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.748+0000 7f9113c415c0 10 datascan.operator(): handling object 100000001fd.0 2023-09-23T03:21:35.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.748+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:19 9.10 9:08b99fea:::100000001fd.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b7542000 con 0x5625b7412800 2023-09-23T03:21:35.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.748+0000 7f64983fb640 1 -- 172.21.15.154:0/3049751456 <== osd.4 v2:172.21.15.190:6808/717302641 2 ==== osd_op_reply(8 10000000205.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55de26ab4b40 con 0x55de26b43c00 2023-09-23T03:21:35.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.748+0000 7f649a4565c0 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-09-23T03:21:35.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.748+0000 7f649a4565c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000205 2023-09-23T03:21:35.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.748+0000 7f649a4565c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/16 2023-09-23T03:21:35.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.748+0000 7f649a4565c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=16 2023-09-23T03:21:35.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.748+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:9 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55de25f86380 con 0x55de25f7d800 2023-09-23T03:21:35.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.748+0000 7f21f3a64640 1 -- 172.21.15.154:0/684274785 <== osd.2 v2:172.21.15.154:6822/2884407243 2 ==== osd_op_reply(6 100000001fe.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv8 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x562cd5cfcd80 con 0x562cd5eb0000 2023-09-23T03:21:35.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.748+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:7 10.1b 10:deccbd37:::100000001fe.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd5e94e00 con 0x562cd5e88c00 2023-09-23T03:21:35.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.748+0000 7f24b7d3a640 1 -- 172.21.15.154:0/1322804614 <== osd.7 v2:172.21.15.190:6824/3460136272 4 ==== osd_op_reply(12 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5632fd76eb40 con 0x5632fcda4800 2023-09-23T03:21:35.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.748+0000 7f24ba63f5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=7 frag=*, r=0 2023-09-23T03:21:35.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.748+0000 7f24ba63f5c0 20 datascan.read_dentry: 7_head not found in result 2023-09-23T03:21:35.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.748+0000 7f24ba63f5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fc at 0x0x10000000000/7 with size=6291456 bytes 2023-09-23T03:21:35.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.748+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/1322804614 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:13 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x5632fca5f880 con 0x5632fcda4800 2023-09-23T03:21:35.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.749+0000 7f21f2a62640 1 -- 172.21.15.154:0/684274785 <== osd.3 v2:172.21.15.154:6808/1133688468 2 ==== osd_op_reply(7 100000001fe.00000000 [getxattr (30) out=30b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x562cd5cfcb40 con 0x562cd5e88c00 2023-09-23T03:21:35.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.749+0000 7f21f4abb5c0 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-09-23T03:21:35.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.749+0000 7f21f4abb5c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001fe 2023-09-23T03:21:35.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.749+0000 7f21f4abb5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/9 2023-09-23T03:21:35.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.749+0000 7f21f4abb5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=9 2023-09-23T03:21:35.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.749+0000 7f21f4abb5c0 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x562cd5eb0400 0x562cd5ea8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.749+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:8 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd5e95180 con 0x562cd5eb0400 2023-09-23T03:21:35.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.749+0000 7f42fa431640 1 -- 172.21.15.154:0/741382839 <== osd.7 v2:172.21.15.190:6824/3460136272 1 ==== osd_op_reply(10 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55694afb8b40 con 0x55694a494800 2023-09-23T03:21:35.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.749+0000 7f42fcc4a5c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-09-23T03:21:35.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.749+0000 7f42fcc4a5c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-09-23T03:21:35.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.749+0000 7f42fcc4a5c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/21, will insert to root 2023-09-23T03:21:35.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.749+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:11 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55694a485500 con 0x55694a494800 2023-09-23T03:21:35.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.749+0000 7f911140b640 1 -- 172.21.15.154:0/2694809311 <== osd.4 v2:172.21.15.190:6808/717302641 3 ==== osd_op_reply(19 100000001fd.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv10 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x5625b7f5eb40 con 0x5625b7412800 2023-09-23T03:21:35.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.749+0000 7f18e005b640 1 -- 172.21.15.154:0/1830651065 <== osd.7 v2:172.21.15.190:6824/3460136272 2 ==== osd_op_reply(9 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x5576f779afc0 con 0x5576f6ce5800 2023-09-23T03:21:35.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.749+0000 7f18e20b65c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-09-23T03:21:35.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.749+0000 7f18e20b65c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-09-23T03:21:35.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.749+0000 7f18e20b65c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/11, will insert to root 2023-09-23T03:21:35.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.749+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:10 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5576f6cf0700 con 0x5576f6ce5800 2023-09-23T03:21:35.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.749+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:20 10.10 10:08b99fea:::100000001fd.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b7543880 con 0x5625b7412800 2023-09-23T03:21:35.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.749+0000 7f21f3263640 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x562cd5eb0400 0x562cd5ea8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.750+0000 7f5ee7f3a640 1 -- 172.21.15.154:0/1042754689 <== osd.7 v2:172.21.15.190:6824/3460136272 8 ==== osd_op_reply(22 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x557b4e6a2d80 con 0x557b4dc4f800 2023-09-23T03:21:35.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.750+0000 7f5ee8f915c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=2 frag=*, r=0 2023-09-23T03:21:35.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.750+0000 7f5ee8f915c0 20 datascan.read_dentry: 2_head not found in result 2023-09-23T03:21:35.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.750+0000 7f5ee8f915c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000000/2 with size=6291456 bytes 2023-09-23T03:21:35.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.750+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:23 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x557b4dc78a80 con 0x557b4dc4f800 2023-09-23T03:21:35.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.750+0000 7f64993fd640 1 -- 172.21.15.154:0/3049751456 <== osd.7 v2:172.21.15.190:6824/3460136272 3 ==== osd_op_reply(9 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55de26ab4d80 con 0x55de25f7d800 2023-09-23T03:21:35.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.750+0000 7f649a4565c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-09-23T03:21:35.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.750+0000 7f649a4565c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-09-23T03:21:35.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.750+0000 7f649a4565c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/16, will insert to root 2023-09-23T03:21:35.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.750+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:10 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55de25f86e00 con 0x55de25f7d800 2023-09-23T03:21:35.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.751+0000 7f911140b640 1 -- 172.21.15.154:0/2694809311 <== osd.4 v2:172.21.15.190:6808/717302641 4 ==== osd_op_reply(20 100000001fd.00000000 [getxattr (30) out=30b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x5625b7f5eb40 con 0x5625b7412800 2023-09-23T03:21:35.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.751+0000 7f9113c415c0 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-09-23T03:21:35.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.751+0000 7f9113c415c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001fd 2023-09-23T03:21:35.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.751+0000 7f9113c415c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/8 2023-09-23T03:21:35.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.751+0000 7f9113c415c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=8 2023-09-23T03:21:35.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.751+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:21 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b7542380 con 0x5625b751fc00 2023-09-23T03:21:35.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.751+0000 7f21f3263640 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x562cd5eb0400 0x562cd5ea8580 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.754+0000 7f24b7d3a640 1 -- 172.21.15.154:0/1322804614 <== osd.7 v2:172.21.15.190:6824/3460136272 5 ==== osd_op_reply(13 10000000000.00000000 [omap-set-vals] v67'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5632fd76eb40 con 0x5632fcda4800 2023-09-23T03:21:35.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.754+0000 7f24ba63f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/7 pointing to 0x0x100000001fc 2023-09-23T03:21:35.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.754+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/1322804614 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:14 9.2c 9:34000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5632fcd8c000 con 0x5632fcda4400 2023-09-23T03:21:35.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.754+0000 7f24b853b640 1 -- 172.21.15.154:0/1322804614 <== osd.3 v2:172.21.15.154:6808/1133688468 3 ==== osd_op_reply(14 [pgnls-filter start_epoch 67 out=81b] v67'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5632fd76efc0 con 0x5632fcda4400 2023-09-23T03:21:35.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.754+0000 7f24ba63f5c0 20 datascan.forall_objects: OSD matched oid 10000000209.00000000 2023-09-23T03:21:35.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.754+0000 7f24ba63f5c0 10 datascan.operator(): handling object 10000000209.0 2023-09-23T03:21:35.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.754+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/1322804614 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:15 9.2c 9:37389ade:::10000000209.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5632fcd8c380 con 0x5632fcda4400 2023-09-23T03:21:35.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.754+0000 7f42fa431640 1 -- 172.21.15.154:0/741382839 <== osd.7 v2:172.21.15.190:6824/3460136272 2 ==== osd_op_reply(11 10000000000.00000000 [omap-get-header out=274b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55694afb8b40 con 0x55694a494800 2023-09-23T03:21:35.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.754+0000 7f42fcc4a5c0 10 datascan.find_or_create_dirfrag: read_version = 20 2023-09-23T03:21:35.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.754+0000 7f42fcc4a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-09-23T03:21:35.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.754+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:12 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55694a485880 con 0x55694a494800 2023-09-23T03:21:35.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.754+0000 7f24b853b640 1 -- 172.21.15.154:0/1322804614 <== osd.3 v2:172.21.15.154:6808/1133688468 4 ==== osd_op_reply(15 10000000209.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x5632fd76efc0 con 0x5632fcda4400 2023-09-23T03:21:35.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.754+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/1322804614 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:16 10.c 10:37389ade:::10000000209.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5632fcd8d500 con 0x5632fcda4800 2023-09-23T03:21:35.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.755+0000 7f18e005b640 1 -- 172.21.15.154:0/1830651065 <== osd.7 v2:172.21.15.190:6824/3460136272 3 ==== osd_op_reply(10 10000000000.00000000 [omap-get-header out=274b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5576f779afc0 con 0x5576f6ce5800 2023-09-23T03:21:35.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.755+0000 7f18e20b65c0 10 datascan.find_or_create_dirfrag: read_version = 20 2023-09-23T03:21:35.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.755+0000 7f18e20b65c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-09-23T03:21:35.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.755+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:11 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5576f6cf0a80 con 0x5576f6ce5800 2023-09-23T03:21:35.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.755+0000 7f24b7d3a640 1 -- 172.21.15.154:0/1322804614 <== osd.7 v2:172.21.15.190:6824/3460136272 6 ==== osd_op_reply(16 10000000209.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x5632fd76eb40 con 0x5632fcda4800 2023-09-23T03:21:35.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.755+0000 7f24ba63f5c0 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-09-23T03:21:35.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.755+0000 7f24ba63f5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000209 2023-09-23T03:21:35.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.755+0000 7f24ba63f5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/20 2023-09-23T03:21:35.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.755+0000 7f24ba63f5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=20 2023-09-23T03:21:35.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.755+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/1322804614 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:17 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5632fcd8d880 con 0x5632fcda4800 2023-09-23T03:21:35.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.758+0000 7f5ee7f3a640 1 -- 172.21.15.154:0/1042754689 <== osd.7 v2:172.21.15.190:6824/3460136272 9 ==== osd_op_reply(23 10000000000.00000000 [omap-set-vals] v67'21 uv21 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x557b4e6a2d80 con 0x557b4dc4f800 2023-09-23T03:21:35.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.759+0000 7f5ee8f915c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/2 pointing to 0x0x100000001f7 2023-09-23T03:21:35.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.759+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:24 9.e 9:70000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x557b4dc78700 con 0x557b4dc4f800 2023-09-23T03:21:35.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.759+0000 7f64993fd640 1 -- 172.21.15.154:0/3049751456 <== osd.7 v2:172.21.15.190:6824/3460136272 4 ==== osd_op_reply(10 10000000000.00000000 [omap-get-header out=274b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55de26ab4d80 con 0x55de25f7d800 2023-09-23T03:21:35.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.759+0000 7f649a4565c0 10 datascan.find_or_create_dirfrag: read_version = 21 2023-09-23T03:21:35.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.759+0000 7f649a4565c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-09-23T03:21:35.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.759+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:11 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55de25f87180 con 0x55de25f7d800 2023-09-23T03:21:35.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.759+0000 7f21f3263640 1 -- 172.21.15.154:0/684274785 <== osd.7 v2:172.21.15.190:6824/3460136272 1 ==== osd_op_reply(8 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x562cd5cfcfc0 con 0x562cd5eb0400 2023-09-23T03:21:35.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.759+0000 7f21f4abb5c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-09-23T03:21:35.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.759+0000 7f21f4abb5c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-09-23T03:21:35.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.759+0000 7f21f4abb5c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/9, will insert to root 2023-09-23T03:21:35.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.759+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:9 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd5e95500 con 0x562cd5eb0400 2023-09-23T03:21:35.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.759+0000 7f5ee7f3a640 1 -- 172.21.15.154:0/1042754689 <== osd.7 v2:172.21.15.190:6824/3460136272 10 ==== osd_op_reply(24 [pgnls-filter start_epoch 67 out=49b] v65'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b4e6a2d80 con 0x557b4dc4f800 2023-09-23T03:21:35.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.760+0000 7f5ee2f30640 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:25 9.2e 9:74000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x557b4d859c00 con 0x557b4dc4f800 2023-09-23T03:21:35.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.760+0000 7f911240d640 1 -- 172.21.15.154:0/2694809311 <== osd.7 v2:172.21.15.190:6824/3460136272 9 ==== osd_op_reply(21 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x5625b7f5ed80 con 0x5625b751fc00 2023-09-23T03:21:35.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.760+0000 7f9113c415c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-09-23T03:21:35.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.760+0000 7f9113c415c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-09-23T03:21:35.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.760+0000 7f9113c415c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/8, will insert to root 2023-09-23T03:21:35.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.760+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:22 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b7543c00 con 0x5625b751fc00 2023-09-23T03:21:35.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.760+0000 7f5ee7f3a640 1 -- 172.21.15.154:0/1042754689 <== osd.7 v2:172.21.15.190:6824/3460136272 11 ==== osd_op_reply(25 [pgnls-filter start_epoch 67 out=49b] v65'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b4e6a2d80 con 0x557b4dc4f800 2023-09-23T03:21:35.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.760+0000 7f5ee272f640 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:26 9.1e 9:78000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x557b4dc79880 con 0x557b4db3c400 2023-09-23T03:21:35.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.760+0000 7f42fa431640 1 -- 172.21.15.154:0/741382839 <== osd.7 v2:172.21.15.190:6824/3460136272 3 ==== osd_op_reply(12 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55694afb8b40 con 0x55694a494800 2023-09-23T03:21:35.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.760+0000 7f42fcc4a5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=21 frag=*, r=0 2023-09-23T03:21:35.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.760+0000 7f42fcc4a5c0 20 datascan.read_dentry: 21_head not found in result 2023-09-23T03:21:35.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.760+0000 7f42fcc4a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000020a at 0x0x10000000000/21 with size=6291456 bytes 2023-09-23T03:21:35.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.760+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:13 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55694a153c00 con 0x55694a494800 2023-09-23T03:21:35.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.761+0000 7f5ee7f3a640 1 -- 172.21.15.154:0/1042754689 <== osd.0 v2:172.21.15.154:6800/2920938138 7 ==== osd_op_reply(26 [pgnls-filter start_epoch 67 out=81b] v67'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x557b4e6a2d80 con 0x557b4db3c400 2023-09-23T03:21:35.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.761+0000 7f5ee8f915c0 20 datascan.forall_objects: OSD matched oid 1000000020c.00000000 2023-09-23T03:21:35.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.761+0000 7f5ee8f915c0 10 datascan.operator(): handling object 1000000020c.0 2023-09-23T03:21:35.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.761+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:27 9.1e 9:789b801b:::1000000020c.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x557b4dc78e00 con 0x557b4db3c400 2023-09-23T03:21:35.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.761+0000 7f18e005b640 1 -- 172.21.15.154:0/1830651065 <== osd.7 v2:172.21.15.190:6824/3460136272 4 ==== osd_op_reply(11 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5576f779afc0 con 0x5576f6ce5800 2023-09-23T03:21:35.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.761+0000 7f18e20b65c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=11 frag=*, r=0 2023-09-23T03:21:35.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.761+0000 7f18e20b65c0 20 datascan.read_dentry: 11_head not found in result 2023-09-23T03:21:35.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.761+0000 7f18e20b65c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000200 at 0x0x10000000000/11 with size=6291456 bytes 2023-09-23T03:21:35.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.761+0000 7f5ee7f3a640 1 -- 172.21.15.154:0/1042754689 <== osd.0 v2:172.21.15.154:6800/2920938138 8 ==== osd_op_reply(27 1000000020c.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv9 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x557b4e6a2d80 con 0x557b4db3c400 2023-09-23T03:21:35.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.761+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:28 10.1e 10:789b801b:::1000000020c.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x557b4dc79c00 con 0x557b4db3c400 2023-09-23T03:21:35.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.761+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:12 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x5576f6cf0e00 con 0x5576f6ce5800 2023-09-23T03:21:35.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.761+0000 7f24b7d3a640 1 -- 172.21.15.154:0/1322804614 <== osd.7 v2:172.21.15.190:6824/3460136272 7 ==== osd_op_reply(17 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x5632fd76eb40 con 0x5632fcda4800 2023-09-23T03:21:35.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.761+0000 7f24ba63f5c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-09-23T03:21:35.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.761+0000 7f24ba63f5c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-09-23T03:21:35.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.761+0000 7f24ba63f5c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/20, will insert to root 2023-09-23T03:21:35.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.761+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/1322804614 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:18 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5632fcd8dc00 con 0x5632fcda4800 2023-09-23T03:21:35.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.761+0000 7f5ee7f3a640 1 -- 172.21.15.154:0/1042754689 <== osd.0 v2:172.21.15.154:6800/2920938138 9 ==== osd_op_reply(28 1000000020c.00000000 [getxattr (30) out=30b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x557b4e6a2d80 con 0x557b4db3c400 2023-09-23T03:21:35.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.761+0000 7f5ee8f915c0 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-09-23T03:21:35.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.761+0000 7f5ee8f915c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000020c 2023-09-23T03:21:35.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.761+0000 7f5ee8f915c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/23 2023-09-23T03:21:35.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.761+0000 7f5ee8f915c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=23 2023-09-23T03:21:35.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.761+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:29 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x557b4dc78000 con 0x557b4dc4f800 2023-09-23T03:21:35.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.762+0000 7f64993fd640 1 -- 172.21.15.154:0/3049751456 <== osd.7 v2:172.21.15.190:6824/3460136272 5 ==== osd_op_reply(11 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55de26ab4d80 con 0x55de25f7d800 2023-09-23T03:21:35.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.762+0000 7f649a4565c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=16 frag=*, r=0 2023-09-23T03:21:35.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.762+0000 7f649a4565c0 20 datascan.read_dentry: 16_head not found in result 2023-09-23T03:21:35.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.762+0000 7f649a4565c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000205 at 0x0x10000000000/16 with size=6291456 bytes 2023-09-23T03:21:35.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.762+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:12 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55de25f87500 con 0x55de25f7d800 2023-09-23T03:21:35.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.762+0000 7f21f3263640 1 -- 172.21.15.154:0/684274785 <== osd.7 v2:172.21.15.190:6824/3460136272 2 ==== osd_op_reply(9 10000000000.00000000 [omap-get-header out=274b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x562cd5cfcfc0 con 0x562cd5eb0400 2023-09-23T03:21:35.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.762+0000 7f21f4abb5c0 10 datascan.find_or_create_dirfrag: read_version = 21 2023-09-23T03:21:35.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.762+0000 7f21f4abb5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-09-23T03:21:35.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.762+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:10 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd5e95880 con 0x562cd5eb0400 2023-09-23T03:21:35.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.762+0000 7f911240d640 1 -- 172.21.15.154:0/2694809311 <== osd.7 v2:172.21.15.190:6824/3460136272 10 ==== osd_op_reply(22 10000000000.00000000 [omap-get-header out=274b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5625b7f5ed80 con 0x5625b751fc00 2023-09-23T03:21:35.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.763+0000 7f9113c415c0 10 datascan.find_or_create_dirfrag: read_version = 21 2023-09-23T03:21:35.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.763+0000 7f9113c415c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-09-23T03:21:35.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.763+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:23 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b7542e00 con 0x5625b751fc00 2023-09-23T03:21:35.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.766+0000 7f42fa431640 1 -- 172.21.15.154:0/741382839 <== osd.7 v2:172.21.15.190:6824/3460136272 4 ==== osd_op_reply(13 10000000000.00000000 [omap-set-vals] v67'22 uv22 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55694afb8b40 con 0x55694a494800 2023-09-23T03:21:35.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.766+0000 7f42fcc4a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/21 pointing to 0x0x1000000020a 2023-09-23T03:21:35.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.766+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:14 9.2a 9:57fe7535::::head [pgnls-filter start_epoch 67 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55694a485180 con 0x55694a494000 2023-09-23T03:21:35.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.766+0000 7f18e005b640 1 -- 172.21.15.154:0/1830651065 <== osd.7 v2:172.21.15.190:6824/3460136272 5 ==== osd_op_reply(12 10000000000.00000000 [omap-set-vals] v67'23 uv23 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5576f779afc0 con 0x5576f6ce5800 2023-09-23T03:21:35.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.766+0000 7f42fb433640 1 -- 172.21.15.154:0/741382839 <== osd.3 v2:172.21.15.154:6808/1133688468 3 ==== osd_op_reply(14 [pgnls-filter start_epoch 67 out=81b] v67'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55694afb8d80 con 0x55694a494000 2023-09-23T03:21:35.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.766+0000 7f18e20b65c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/11 pointing to 0x0x10000000200 2023-09-23T03:21:35.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.767+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:13 9.19 9:997b7129::::head [pgnls-filter start_epoch 67 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5576f6cf0380 con 0x5576f6ce5c00 2023-09-23T03:21:35.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.767+0000 7f42fcc4a5c0 20 datascan.forall_objects: OSD matched oid 100000001f6.00000000 2023-09-23T03:21:35.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.767+0000 7f42fcc4a5c0 10 datascan.operator(): handling object 100000001f6.0 2023-09-23T03:21:35.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.767+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:15 9.2a 9:57fe7535:::100000001f6.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55694a485500 con 0x55694a494000 2023-09-23T03:21:35.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.767+0000 7f18e105d640 1 -- 172.21.15.154:0/1830651065 <== osd.3 v2:172.21.15.154:6808/1133688468 3 ==== osd_op_reply(13 [pgnls-filter start_epoch 67 out=81b] v67'18 uv18 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5576f779ad80 con 0x5576f6ce5c00 2023-09-23T03:21:35.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.767+0000 7f42fb433640 1 -- 172.21.15.154:0/741382839 <== osd.3 v2:172.21.15.154:6808/1133688468 4 ==== osd_op_reply(15 100000001f6.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x55694afb8d80 con 0x55694a494000 2023-09-23T03:21:35.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.767+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:16 10.a 10:57fe7535:::100000001f6.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55694a484000 con 0x55694a494400 2023-09-23T03:21:35.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.767+0000 7f18e20b65c0 20 datascan.forall_objects: OSD matched oid 100000001f8.00000000 2023-09-23T03:21:35.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.767+0000 7f18e20b65c0 10 datascan.operator(): handling object 100000001f8.0 2023-09-23T03:21:35.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.767+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:14 9.19 9:997b7129:::100000001f8.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5576f6cf0700 con 0x5576f6ce5c00 2023-09-23T03:21:35.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.767+0000 7f5ee7f3a640 1 -- 172.21.15.154:0/1042754689 <== osd.7 v2:172.21.15.190:6824/3460136272 12 ==== osd_op_reply(29 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x557b4e6a2d80 con 0x557b4dc4f800 2023-09-23T03:21:35.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.767+0000 7f5ee8f915c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-09-23T03:21:35.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.767+0000 7f5ee8f915c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-09-23T03:21:35.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.767+0000 7f5ee8f915c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/23, will insert to root 2023-09-23T03:21:35.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.767+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:30 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x557b4dc78380 con 0x557b4dc4f800 2023-09-23T03:21:35.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.767+0000 7f24b7d3a640 1 -- 172.21.15.154:0/1322804614 <== osd.7 v2:172.21.15.190:6824/3460136272 8 ==== osd_op_reply(18 10000000000.00000000 [omap-get-header out=274b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5632fd76eb40 con 0x5632fcda4800 2023-09-23T03:21:35.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.767+0000 7f18e105d640 1 -- 172.21.15.154:0/1830651065 <== osd.3 v2:172.21.15.154:6808/1133688468 4 ==== osd_op_reply(14 100000001f8.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x5576f779ad80 con 0x5576f6ce5c00 2023-09-23T03:21:35.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.767+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:15 10.19 10:997b7129:::100000001f8.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5576f6cf1500 con 0x5576f6ce4400 2023-09-23T03:21:35.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.767+0000 7f24ba63f5c0 10 datascan.find_or_create_dirfrag: read_version = 23 2023-09-23T03:21:35.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.767+0000 7f24ba63f5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-09-23T03:21:35.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.767+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/1322804614 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:19 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5632fcd8d180 con 0x5632fcda4800 2023-09-23T03:21:35.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.768+0000 7f18e085c640 1 -- 172.21.15.154:0/1830651065 <== osd.1 v2:172.21.15.154:6816/1267710005 3 ==== osd_op_reply(15 100000001f8.00000000 [getxattr (30) out=30b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x5576f779ab40 con 0x5576f6ce4400 2023-09-23T03:21:35.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.768+0000 7f18e20b65c0 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-09-23T03:21:35.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.768+0000 7f18e20b65c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f8 2023-09-23T03:21:35.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.768+0000 7f18e20b65c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/3 2023-09-23T03:21:35.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.768+0000 7f18e20b65c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=3 2023-09-23T03:21:35.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.768+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:16 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5576f6cf1880 con 0x5576f6ce5800 2023-09-23T03:21:35.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.768+0000 7f42fac32640 1 -- 172.21.15.154:0/741382839 <== osd.6 v2:172.21.15.190:6816/1964294001 2 ==== osd_op_reply(16 100000001f6.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55694afb8fc0 con 0x55694a494400 2023-09-23T03:21:35.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.768+0000 7f42fcc4a5c0 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-09-23T03:21:35.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.768+0000 7f42fcc4a5c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f6 2023-09-23T03:21:35.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.768+0000 7f42fcc4a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/1 2023-09-23T03:21:35.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.768+0000 7f42fcc4a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=1 2023-09-23T03:21:35.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.768+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:17 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55694a484380 con 0x55694a494800 2023-09-23T03:21:35.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.771+0000 7f64993fd640 1 -- 172.21.15.154:0/3049751456 <== osd.7 v2:172.21.15.190:6824/3460136272 6 ==== osd_op_reply(12 10000000000.00000000 [omap-set-vals] v67'24 uv24 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55de26ab4d80 con 0x55de25f7d800 2023-09-23T03:21:35.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.771+0000 7f649a4565c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/16 pointing to 0x0x10000000205 2023-09-23T03:21:35.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.771+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:13 9.3 9:c0000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55de25f86a80 con 0x55de25f7d800 2023-09-23T03:21:35.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.772+0000 7f21f3263640 1 -- 172.21.15.154:0/684274785 <== osd.7 v2:172.21.15.190:6824/3460136272 3 ==== osd_op_reply(10 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv24 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x562cd5cfcfc0 con 0x562cd5eb0400 2023-09-23T03:21:35.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.772+0000 7f21f4abb5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=9 frag=*, r=0 2023-09-23T03:21:35.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.772+0000 7f64993fd640 1 -- 172.21.15.154:0/3049751456 <== osd.7 v2:172.21.15.190:6824/3460136272 7 ==== osd_op_reply(13 [pgnls-filter start_epoch 67 out=49b] v65'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55de26ab4d80 con 0x55de25f7d800 2023-09-23T03:21:35.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.772+0000 7f21f4abb5c0 20 datascan.read_dentry: 9_head not found in result 2023-09-23T03:21:35.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.772+0000 7f21f4abb5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fe at 0x0x10000000000/9 with size=6291456 bytes 2023-09-23T03:21:35.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.772+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:11 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd5a77c00 con 0x562cd5eb0400 2023-09-23T03:21:35.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.772+0000 7f6493bf2640 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55de25f7dc00 0x55de25f98c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.772+0000 7f6493bf2640 1 -- 172.21.15.154:0/3049751456 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:14 9.23 9:c4000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55de25f87c00 con 0x55de25f7dc00 2023-09-23T03:21:35.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.772+0000 7f6498bfc640 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55de25f7dc00 0x55de25f98c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.772+0000 7f911240d640 1 -- 172.21.15.154:0/2694809311 <== osd.7 v2:172.21.15.190:6824/3460136272 11 ==== osd_op_reply(23 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv24 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5625b7f5ed80 con 0x5625b751fc00 2023-09-23T03:21:35.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.772+0000 7f9113c415c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=8 frag=*, r=0 2023-09-23T03:21:35.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.772+0000 7f9113c415c0 20 datascan.read_dentry: 8_head not found in result 2023-09-23T03:21:35.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.772+0000 7f9113c415c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fd at 0x0x10000000000/8 with size=6291456 bytes 2023-09-23T03:21:35.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.772+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:24 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b7543180 con 0x5625b751fc00 2023-09-23T03:21:35.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.773+0000 7f6498bfc640 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55de25f7dc00 0x55de25f98c00 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.773+0000 7f5ee7f3a640 1 -- 172.21.15.154:0/1042754689 <== osd.7 v2:172.21.15.190:6824/3460136272 13 ==== osd_op_reply(30 10000000000.00000000 [omap-get-header out=274b] v0'0 uv24 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x557b4e6a2d80 con 0x557b4dc4f800 2023-09-23T03:21:35.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.773+0000 7f5ee8f915c0 10 datascan.find_or_create_dirfrag: read_version = 24 2023-09-23T03:21:35.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.773+0000 7f5ee8f915c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-09-23T03:21:35.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.773+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:31 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x557b4dc78e00 con 0x557b4dc4f800 2023-09-23T03:21:35.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.773+0000 7f6498bfc640 1 -- 172.21.15.154:0/3049751456 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(14 [pgnls-filter start_epoch 67 out=49b] v65'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55de26ab4fc0 con 0x55de25f7dc00 2023-09-23T03:21:35.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.773+0000 7f64943f3640 1 -- 172.21.15.154:0/3049751456 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:15 9.13 9:c8000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55de25f9a000 con 0x55de25f7d800 2023-09-23T03:21:35.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.773+0000 7f24b7d3a640 1 -- 172.21.15.154:0/1322804614 <== osd.7 v2:172.21.15.190:6824/3460136272 9 ==== osd_op_reply(19 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv24 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5632fd76eb40 con 0x5632fcda4800 2023-09-23T03:21:35.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.773+0000 7f24ba63f5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=20 frag=*, r=0 2023-09-23T03:21:35.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.773+0000 7f24ba63f5c0 20 datascan.read_dentry: 20_head not found in result 2023-09-23T03:21:35.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.773+0000 7f24ba63f5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000209 at 0x0x10000000000/20 with size=6291456 bytes 2023-09-23T03:21:35.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.773+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/1322804614 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:20 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x5632fcd8d500 con 0x5632fcda4800 2023-09-23T03:21:35.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.774+0000 7f18e005b640 1 -- 172.21.15.154:0/1830651065 <== osd.7 v2:172.21.15.190:6824/3460136272 6 ==== osd_op_reply(16 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x5576f779afc0 con 0x5576f6ce5800 2023-09-23T03:21:35.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.774+0000 7f18e20b65c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-09-23T03:21:35.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.774+0000 7f18e20b65c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-09-23T03:21:35.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.774+0000 7f18e20b65c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/3, will insert to root 2023-09-23T03:21:35.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.774+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:17 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5576f69dbc00 con 0x5576f6ce5800 2023-09-23T03:21:35.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.774+0000 7f64993fd640 1 -- 172.21.15.154:0/3049751456 <== osd.7 v2:172.21.15.190:6824/3460136272 8 ==== osd_op_reply(15 [pgnls-filter start_epoch 67 out=81b] v67'19 uv19 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55de26ab4d80 con 0x55de25f7d800 2023-09-23T03:21:35.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.774+0000 7f649a4565c0 20 datascan.forall_objects: OSD matched oid 100000001fb.00000000 2023-09-23T03:21:35.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.774+0000 7f649a4565c0 10 datascan.operator(): handling object 100000001fb.0 2023-09-23T03:21:35.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.774+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:16 9.13 9:ca8fdbba:::100000001fb.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55de25f86380 con 0x55de25f7d800 2023-09-23T03:21:35.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.774+0000 7f42fa431640 1 -- 172.21.15.154:0/741382839 <== osd.7 v2:172.21.15.190:6824/3460136272 5 ==== osd_op_reply(17 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55694afb8b40 con 0x55694a494800 2023-09-23T03:21:35.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.774+0000 7f42fcc4a5c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-09-23T03:21:35.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.774+0000 7f42fcc4a5c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-09-23T03:21:35.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.774+0000 7f42fcc4a5c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/1, will insert to root 2023-09-23T03:21:35.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.774+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:18 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55694a153880 con 0x55694a494800 2023-09-23T03:21:35.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.775+0000 7f64993fd640 1 -- 172.21.15.154:0/3049751456 <== osd.7 v2:172.21.15.190:6824/3460136272 9 ==== osd_op_reply(16 100000001fb.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv19 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x55de26ab4d80 con 0x55de25f7d800 2023-09-23T03:21:35.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.775+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:17 10.13 10:ca8fdbba:::100000001fb.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55de25f87880 con 0x55de25f7c400 2023-09-23T03:21:35.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.775+0000 7f6498bfc640 1 -- 172.21.15.154:0/3049751456 <== osd.1 v2:172.21.15.154:6816/1267710005 2 ==== osd_op_reply(17 100000001fb.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55de26ab4fc0 con 0x55de25f7c400 2023-09-23T03:21:35.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.775+0000 7f649a4565c0 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-09-23T03:21:35.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.775+0000 7f649a4565c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001fb 2023-09-23T03:21:35.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.775+0000 7f649a4565c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/6 2023-09-23T03:21:35.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.775+0000 7f649a4565c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=6 2023-09-23T03:21:35.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.775+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:18 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55de25c61880 con 0x55de25f7d800 2023-09-23T03:21:35.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.778+0000 7f21f3263640 1 -- 172.21.15.154:0/684274785 <== osd.7 v2:172.21.15.190:6824/3460136272 4 ==== osd_op_reply(11 10000000000.00000000 [omap-set-vals] v67'25 uv25 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562cd5cfcfc0 con 0x562cd5eb0400 2023-09-23T03:21:35.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.778+0000 7f21f4abb5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/9 pointing to 0x0x100000001fe 2023-09-23T03:21:35.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.778+0000 7f21f4abb5c0 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x562cd5eb0800 0x562cd5ea8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.778+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:12 9.7 9:e0000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd5e95500 con 0x562cd5eb0800 2023-09-23T03:21:35.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.778+0000 7f21f2a62640 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x562cd5eb0800 0x562cd5ea8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.780+0000 7f21f2a62640 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x562cd5eb0800 0x562cd5ea8b00 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.780+0000 7f911240d640 1 -- 172.21.15.154:0/2694809311 <== osd.7 v2:172.21.15.190:6824/3460136272 12 ==== osd_op_reply(24 10000000000.00000000 [omap-set-vals] v67'26 uv26 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5625b7f5ed80 con 0x5625b751fc00 2023-09-23T03:21:35.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.780+0000 7f9113c415c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/8 pointing to 0x0x100000001fd 2023-09-23T03:21:35.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.780+0000 7f21f2a62640 1 -- 172.21.15.154:0/684274785 <== osd.5 v2:172.21.15.190:6800/882536885 1 ==== osd_op_reply(12 [pgnls-filter start_epoch 67 out=101b] v67'13 uv13 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x562cd5cfcb40 con 0x562cd5eb0800 2023-09-23T03:21:35.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.780+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:25 9.30 9:0c000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b7543500 con 0x5625b7412400 2023-09-23T03:21:35.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.780+0000 7f21f4abb5c0 20 datascan.forall_objects: OSD matched oid 10000000207.00000000 2023-09-23T03:21:35.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.780+0000 7f21f4abb5c0 10 datascan.operator(): handling object 10000000207.0 2023-09-23T03:21:35.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.780+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:13 9.7 9:e15a914f:::10000000207.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd5e95880 con 0x562cd5eb0800 2023-09-23T03:21:35.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.781+0000 7f911240d640 1 -- 172.21.15.154:0/2694809311 <== osd.0 v2:172.21.15.154:6800/2920938138 4 ==== osd_op_reply(25 [pgnls-filter start_epoch 67 out=49b] v65'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5625b7f5ed80 con 0x5625b7412400 2023-09-23T03:21:35.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.781+0000 7f910d403640 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5625b7560000 0x5625b753fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.781+0000 7f910d403640 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:26 9.8 9:10000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b7566000 con 0x5625b7560000 2023-09-23T03:21:35.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.781+0000 7f5ee7f3a640 1 -- 172.21.15.154:0/1042754689 <== osd.7 v2:172.21.15.190:6824/3460136272 14 ==== osd_op_reply(31 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv26 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x557b4e6a2d80 con 0x557b4dc4f800 2023-09-23T03:21:35.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.781+0000 7f5ee8f915c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=23 frag=*, r=0 2023-09-23T03:21:35.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.781+0000 7f5ee8f915c0 20 datascan.read_dentry: 23_head not found in result 2023-09-23T03:21:35.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.781+0000 7f5ee8f915c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000020c at 0x0x10000000000/23 with size=6291456 bytes 2023-09-23T03:21:35.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.781+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:32 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x557b4dc79880 con 0x557b4dc4f800 2023-09-23T03:21:35.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.781+0000 7f9111c0c640 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5625b7560000 0x5625b753fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.781+0000 7f21f2a62640 1 -- 172.21.15.154:0/684274785 <== osd.5 v2:172.21.15.190:6800/882536885 2 ==== osd_op_reply(13 10000000207.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x562cd5cfcb40 con 0x562cd5eb0800 2023-09-23T03:21:35.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.781+0000 7f21f4abb5c0 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x562cd5eb1000 0x562cd5ea9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.781+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:14 10.7 10:e15a914f:::10000000207.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd5e95180 con 0x562cd5eb1000 2023-09-23T03:21:35.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.782+0000 7f9111c0c640 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5625b7560000 0x5625b753fb80 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.782+0000 7f9111c0c640 1 -- 172.21.15.154:0/2694809311 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(26 [pgnls-filter start_epoch 67 out=49b] v65'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5625b7f5efc0 con 0x5625b7560000 2023-09-23T03:21:35.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.782+0000 7f910cc02640 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:27 9.28 9:14000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b7542a80 con 0x5625b751f800 2023-09-23T03:21:35.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.782+0000 7f21f3a64640 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x562cd5eb1000 0x562cd5ea9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.783+0000 7f21f3a64640 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x562cd5eb1000 0x562cd5ea9080 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.783+0000 7f911140b640 1 -- 172.21.15.154:0/2694809311 <== osd.6 v2:172.21.15.190:6816/1964294001 3 ==== osd_op_reply(27 [pgnls-filter start_epoch 67 out=101b] v67'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5625b7f5eb40 con 0x5625b751f800 2023-09-23T03:21:35.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.783+0000 7f9113c415c0 20 datascan.forall_objects: OSD matched oid 10000000202.00000000 2023-09-23T03:21:35.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.783+0000 7f9113c415c0 10 datascan.operator(): handling object 10000000202.0 2023-09-23T03:21:35.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.783+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:28 9.28 9:15246602:::10000000202.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b711bc00 con 0x5625b751f800 2023-09-23T03:21:35.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.784+0000 7f21f3a64640 1 -- 172.21.15.154:0/684274785 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== osd_op_reply(14 10000000207.00000000 [getxattr (30) out=30b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x562cd5cfcd80 con 0x562cd5eb1000 2023-09-23T03:21:35.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.784+0000 7f21f4abb5c0 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-09-23T03:21:35.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.784+0000 7f21f4abb5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000207 2023-09-23T03:21:35.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.784+0000 7f21f4abb5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/18 2023-09-23T03:21:35.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.784+0000 7f21f4abb5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=18 2023-09-23T03:21:35.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.784+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:15 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd5a77c00 con 0x562cd5eb0400 2023-09-23T03:21:35.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.784+0000 7f911140b640 1 -- 172.21.15.154:0/2694809311 <== osd.6 v2:172.21.15.190:6816/1964294001 4 ==== osd_op_reply(28 10000000202.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x5625b7f5eb40 con 0x5625b751f800 2023-09-23T03:21:35.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.784+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:29 10.8 10:15246602:::10000000202.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b7542000 con 0x5625b7412400 2023-09-23T03:21:35.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.784+0000 7f911240d640 1 -- 172.21.15.154:0/2694809311 <== osd.0 v2:172.21.15.154:6800/2920938138 5 ==== osd_op_reply(29 10000000202.00000000 [getxattr (30) out=30b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x5625b7f5ed80 con 0x5625b7412400 2023-09-23T03:21:35.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.784+0000 7f9113c415c0 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-09-23T03:21:35.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.784+0000 7f9113c415c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000202 2023-09-23T03:21:35.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.784+0000 7f9113c415c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/13 2023-09-23T03:21:35.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.784+0000 7f9113c415c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=13 2023-09-23T03:21:35.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.784+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:30 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b7543880 con 0x5625b751fc00 2023-09-23T03:21:35.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.786+0000 7f24b7d3a640 1 -- 172.21.15.154:0/1322804614 <== osd.7 v2:172.21.15.190:6824/3460136272 10 ==== osd_op_reply(20 10000000000.00000000 [omap-set-vals] v67'27 uv27 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5632fd76eb40 con 0x5632fcda4800 2023-09-23T03:21:35.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.786+0000 7f24ba63f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/20 pointing to 0x0x10000000209 2023-09-23T03:21:35.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.786+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/1322804614 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:21 9.1c 9:38000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5632fcd8d880 con 0x5632fd94d000 2023-09-23T03:21:35.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.787+0000 7f24b8d3c640 1 -- 172.21.15.154:0/1322804614 <== osd.0 v2:172.21.15.154:6800/2920938138 2 ==== osd_op_reply(21 [pgnls-filter start_epoch 67 out=49b] v65'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5632fd76ed80 con 0x5632fd94d000 2023-09-23T03:21:35.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.787+0000 7f24b3531640 1 -- 172.21.15.154:0/1322804614 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:22 9.3c 9:3c000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5632fcd8ce00 con 0x5632fcda4000 2023-09-23T03:21:35.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.787+0000 7f18e005b640 1 -- 172.21.15.154:0/1830651065 <== osd.7 v2:172.21.15.190:6824/3460136272 7 ==== osd_op_reply(17 10000000000.00000000 [omap-get-header out=274b] v0'0 uv27 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5576f779afc0 con 0x5576f6ce5800 2023-09-23T03:21:35.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.787+0000 7f18e20b65c0 10 datascan.find_or_create_dirfrag: read_version = 27 2023-09-23T03:21:35.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.787+0000 7f18e20b65c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-09-23T03:21:35.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.787+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:18 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5576f6cf0a80 con 0x5576f6ce5800 2023-09-23T03:21:35.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.787+0000 7f42fa431640 1 -- 172.21.15.154:0/741382839 <== osd.7 v2:172.21.15.190:6824/3460136272 6 ==== osd_op_reply(18 10000000000.00000000 [omap-get-header out=274b] v0'0 uv27 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55694afb8b40 con 0x55694a494800 2023-09-23T03:21:35.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.787+0000 7f42fcc4a5c0 10 datascan.find_or_create_dirfrag: read_version = 27 2023-09-23T03:21:35.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.787+0000 7f24b8d3c640 1 -- 172.21.15.154:0/1322804614 <== osd.5 v2:172.21.15.190:6800/882536885 2 ==== osd_op_reply(22 [pgnls-filter start_epoch 67 out=49b] v65'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5632fd76ed80 con 0x5632fcda4000 2023-09-23T03:21:35.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.787+0000 7f42fcc4a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-09-23T03:21:35.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.787+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:19 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55694a484a80 con 0x55694a494800 2023-09-23T03:21:35.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.788+0000 7f24b3d32640 1 -- 172.21.15.154:0/1322804614 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:23 9.2 9:40000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5632fca5fc00 con 0x5632fcda4400 2023-09-23T03:21:35.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.788+0000 7f64993fd640 1 -- 172.21.15.154:0/3049751456 <== osd.7 v2:172.21.15.190:6824/3460136272 10 ==== osd_op_reply(18 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55de26ab4d80 con 0x55de25f7d800 2023-09-23T03:21:35.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.788+0000 7f649a4565c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-09-23T03:21:35.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.788+0000 7f649a4565c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-09-23T03:21:35.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.788+0000 7f649a4565c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/6, will insert to root 2023-09-23T03:21:35.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.788+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:19 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55de25c61c00 con 0x55de25f7d800 2023-09-23T03:21:35.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.788+0000 7f24b853b640 1 -- 172.21.15.154:0/1322804614 <== osd.3 v2:172.21.15.154:6808/1133688468 5 ==== osd_op_reply(23 [pgnls-filter start_epoch 67 out=49b] v65'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5632fd76efc0 con 0x5632fcda4400 2023-09-23T03:21:35.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.788+0000 7f24b3531640 1 -- 172.21.15.154:0/1322804614 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:24 9.22 9:44000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5632fcdbe000 con 0x5632fcda4800 2023-09-23T03:21:35.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.789+0000 7f24b7d3a640 1 -- 172.21.15.154:0/1322804614 <== osd.7 v2:172.21.15.190:6824/3460136272 11 ==== osd_op_reply(24 [pgnls-filter start_epoch 67 out=49b] v65'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5632fd76eb40 con 0x5632fcda4800 2023-09-23T03:21:35.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.789+0000 7f24b3d32640 1 -- 172.21.15.154:0/1322804614 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:25 9.12 9:48000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5632fcdbea80 con 0x5632fcda4000 2023-09-23T03:21:35.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.790+0000 7f24b8d3c640 1 -- 172.21.15.154:0/1322804614 <== osd.5 v2:172.21.15.190:6800/882536885 3 ==== osd_op_reply(25 [pgnls-filter start_epoch 67 out=49b] v65'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5632fd76ed80 con 0x5632fcda4000 2023-09-23T03:21:35.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.790+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/1322804614 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5632fd94d000 msgr2=0x5632fca6fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.790+0000 7f24ba63f5c0 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5632fd94d000 0x5632fca6fb80 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.790+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/1322804614 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5632fd94d400 msgr2=0x5632fca6f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.790+0000 7f24ba63f5c0 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5632fd94d400 0x5632fca6f600 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.790+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/1322804614 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5632fd94d800 msgr2=0x5632fcd96000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.790+0000 7f24ba63f5c0 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5632fd94d800 0x5632fcd96000 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.790+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/1322804614 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5632fcda4400 msgr2=0x5632fcd98c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.790+0000 7f24ba63f5c0 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5632fcda4400 0x5632fcd98c00 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.790+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/1322804614 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5632fd94cc00 msgr2=0x5632fca70100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.790+0000 7f24ba63f5c0 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5632fd94cc00 0x5632fca70100 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.790+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/1322804614 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x5632fcda4000 msgr2=0x5632fcd98680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.790+0000 7f24ba63f5c0 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x5632fcda4000 0x5632fcd98680 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.790+0000 7f24b7d3a640 1 -- 172.21.15.154:0/1322804614 reap_dead start 2023-09-23T03:21:35.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.790+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/1322804614 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x5632fcda4800 msgr2=0x5632fcd96580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.790+0000 7f24ba63f5c0 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x5632fcda4800 0x5632fcd96580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.790+0000 7f24b7d3a640 1 -- 172.21.15.154:0/1322804614 reap_dead start 2023-09-23T03:21:35.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.790+0000 7f24b7d3a640 1 -- 172.21.15.154:0/1322804614 reap_dead start 2023-09-23T03:21:35.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.790+0000 7f24b7d3a640 1 -- 172.21.15.154:0/1322804614 reap_dead start 2023-09-23T03:21:35.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.790+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/1322804614 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5632fd94c800 msgr2=0x5632fca71700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.790+0000 7f24ba63f5c0 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5632fd94c800 0x5632fca71700 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x5632fd776c30 tx=0x5632fd7f8180 comp rx=0 tx=0).stop 2023-09-23T03:21:35.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.790+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/1322804614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5632fd8e7000 msgr2=0x5632fca71180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.790+0000 7f24ba63f5c0 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5632fd8e7000 0x5632fca71180 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x5632fd777a10 tx=0x5632fd7f85a0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.790+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/1322804614 shutdown_connections 2023-09-23T03:21:35.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.790+0000 7f24ba63f5c0 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5632fd94c800 0x5632fca71700 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.790+0000 7f24ba63f5c0 1 --2- 172.21.15.154:0/1322804614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5632fd8e7000 0x5632fca71180 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.790+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/1322804614 >> 172.21.15.154:0/1322804614 conn(0x5632fd8e6c00 msgr2=0x5632fcb01800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:35.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.790+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/1322804614 shutdown_connections 2023-09-23T03:21:35.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.790+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/1322804614 wait complete. 2023-09-23T03:21:35.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.791+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/3563497076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5632fca04c00 msgr2=0x5632fca6e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.791+0000 7f24ba63f5c0 1 --2- 172.21.15.154:0/3563497076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5632fca04c00 0x5632fca6e580 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x5632fd847830 tx=0x5632fd7f8600 comp rx=0 tx=0).stop 2023-09-23T03:21:35.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.791+0000 7f5ee7f3a640 1 -- 172.21.15.154:0/1042754689 <== osd.7 v2:172.21.15.190:6824/3460136272 15 ==== osd_op_reply(32 10000000000.00000000 [omap-set-vals] v67'28 uv28 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x557b4e6a2d80 con 0x557b4dc4f800 2023-09-23T03:21:35.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.791+0000 7f5ee8f915c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/23 pointing to 0x0x1000000020c 2023-09-23T03:21:35.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.791+0000 7f5ee8f915c0 1 --2- 172.21.15.154:0/1042754689 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x557b4dc94400 0x557b4dc86b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.791+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/3563497076 shutdown_connections 2023-09-23T03:21:35.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.791+0000 7f24ba63f5c0 1 --2- 172.21.15.154:0/3563497076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5632fca05c00 0x5632fca6f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.791+0000 7f24ba63f5c0 1 --2- 172.21.15.154:0/3563497076 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5632fca05800 0x5632fca6eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.791+0000 7f24ba63f5c0 1 --2- 172.21.15.154:0/3563497076 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5632fca04c00 0x5632fca6e580 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.791+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/3563497076 >> 172.21.15.154:0/3563497076 conn(0x5632fca05000 msgr2=0x5632fcb01000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:35.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.791+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:33 9.3e 9:7c000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x557b4dc78700 con 0x557b4dc94400 2023-09-23T03:21:35.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.791+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/3563497076 shutdown_connections 2023-09-23T03:21:35.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.791+0000 7f5ee7739640 1 --2- 172.21.15.154:0/1042754689 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x557b4dc94400 0x557b4dc86b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.791+0000 7f24ba63f5c0 1 -- 172.21.15.154:0/3563497076 wait complete. 2023-09-23T03:21:35.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.791+0000 7f21f3263640 1 -- 172.21.15.154:0/684274785 <== osd.7 v2:172.21.15.190:6824/3460136272 5 ==== osd_op_reply(15 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x562cd5cfcfc0 con 0x562cd5eb0400 2023-09-23T03:21:35.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.791+0000 7f21f4abb5c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-09-23T03:21:35.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.791+0000 7f21f4abb5c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-09-23T03:21:35.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.791+0000 7f21f4abb5c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/18, will insert to root 2023-09-23T03:21:35.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.791+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:16 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd5a77880 con 0x562cd5eb0400 2023-09-23T03:21:35.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.792+0000 7f5ee7739640 1 --2- 172.21.15.154:0/1042754689 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x557b4dc94400 0x557b4dc86b00 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.792+0000 7f911240d640 1 -- 172.21.15.154:0/2694809311 <== osd.7 v2:172.21.15.190:6824/3460136272 13 ==== osd_op_reply(30 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x5625b7f5ed80 con 0x5625b751fc00 2023-09-23T03:21:35.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.792+0000 7f9113c415c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-09-23T03:21:35.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.792+0000 7f9113c415c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-09-23T03:21:35.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.792+0000 7f9113c415c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/13, will insert to root 2023-09-23T03:21:35.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.792+0000 7f5ee7739640 1 -- 172.21.15.154:0/1042754689 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(33 [pgnls-filter start_epoch 67 out=49b] v65'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b4e6a2fc0 con 0x557b4dc94400 2023-09-23T03:21:35.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.792+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:31 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b7542a80 con 0x5625b751fc00 2023-09-23T03:21:35.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.792+0000 7f5ee272f640 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:34 9.1 9:80000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x557b4dc9a000 con 0x557b4dc94400 2023-09-23T03:21:35.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.792+0000 7f18e005b640 1 -- 172.21.15.154:0/1830651065 <== osd.7 v2:172.21.15.190:6824/3460136272 8 ==== osd_op_reply(18 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv28 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5576f779afc0 con 0x5576f6ce5800 2023-09-23T03:21:35.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.792+0000 7f18e20b65c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=3 frag=*, r=0 2023-09-23T03:21:35.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.792+0000 7f18e20b65c0 20 datascan.read_dentry: 3_head not found in result 2023-09-23T03:21:35.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.792+0000 7f18e20b65c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f8 at 0x0x10000000000/3 with size=6291456 bytes 2023-09-23T03:21:35.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.792+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:19 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x5576f6cf0e00 con 0x5576f6ce5800 2023-09-23T03:21:35.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.792+0000 7f5ee7739640 1 -- 172.21.15.154:0/1042754689 <== osd.3 v2:172.21.15.154:6808/1133688468 2 ==== osd_op_reply(34 [pgnls-filter start_epoch 67 out=49b] v65'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b4e6a2fc0 con 0x557b4dc94400 2023-09-23T03:21:35.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.792+0000 7f5ee2f30640 1 --2- 172.21.15.154:0/1042754689 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x557b4dc9c000 0x557b4dc88680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.792+0000 7f5ee2f30640 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:35 9.21 9:84000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x557b4dc79500 con 0x557b4dc9c000 2023-09-23T03:21:35.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.793+0000 7f42fa431640 1 -- 172.21.15.154:0/741382839 <== osd.7 v2:172.21.15.190:6824/3460136272 7 ==== osd_op_reply(19 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv28 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55694afb8b40 con 0x55694a494800 2023-09-23T03:21:35.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.793+0000 7f42fcc4a5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=1 frag=*, r=0 2023-09-23T03:21:35.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.793+0000 7f42fcc4a5c0 20 datascan.read_dentry: 1_head not found in result 2023-09-23T03:21:35.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.793+0000 7f42fcc4a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f6 at 0x0x10000000000/1 with size=6291456 bytes 2023-09-23T03:21:35.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.793+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:20 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55694a484700 con 0x55694a494800 2023-09-23T03:21:35.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.793+0000 7f5ee6f38640 1 --2- 172.21.15.154:0/1042754689 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x557b4dc9c000 0x557b4dc88680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.793+0000 7f64993fd640 1 -- 172.21.15.154:0/3049751456 <== osd.7 v2:172.21.15.190:6824/3460136272 11 ==== osd_op_reply(19 10000000000.00000000 [omap-get-header out=274b] v0'0 uv28 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55de26ab4d80 con 0x55de25f7d800 2023-09-23T03:21:35.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.793+0000 7f649a4565c0 10 datascan.find_or_create_dirfrag: read_version = 28 2023-09-23T03:21:35.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.793+0000 7f649a4565c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-09-23T03:21:35.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.793+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:20 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55de25f86380 con 0x55de25f7d800 2023-09-23T03:21:35.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.793+0000 7f5ee6f38640 1 --2- 172.21.15.154:0/1042754689 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x557b4dc9c000 0x557b4dc88680 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.793+0000 7f21f3263640 1 -- 172.21.15.154:0/684274785 <== osd.7 v2:172.21.15.190:6824/3460136272 6 ==== osd_op_reply(16 10000000000.00000000 [omap-get-header out=274b] v0'0 uv28 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x562cd5cfcfc0 con 0x562cd5eb0400 2023-09-23T03:21:35.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.793+0000 7f21f4abb5c0 10 datascan.find_or_create_dirfrag: read_version = 28 2023-09-23T03:21:35.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.793+0000 7f21f4abb5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-09-23T03:21:35.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.793+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:17 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd5e94a80 con 0x562cd5eb0400 2023-09-23T03:21:35.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.793+0000 7f5ee6f38640 1 -- 172.21.15.154:0/1042754689 <== osd.2 v2:172.21.15.154:6822/2884407243 1 ==== osd_op_reply(35 [pgnls-filter start_epoch 67 out=49b] v65'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b4e6a2b40 con 0x557b4dc9c000 2023-09-23T03:21:35.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.793+0000 7f5ee272f640 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:36 9.11 9:88000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x557b4dc9a380 con 0x557b4dc4f800 2023-09-23T03:21:35.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.794+0000 7f911240d640 1 -- 172.21.15.154:0/2694809311 <== osd.7 v2:172.21.15.190:6824/3460136272 14 ==== osd_op_reply(31 10000000000.00000000 [omap-get-header out=274b] v0'0 uv28 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5625b7f5ed80 con 0x5625b751fc00 2023-09-23T03:21:35.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.794+0000 7f9113c415c0 10 datascan.find_or_create_dirfrag: read_version = 28 2023-09-23T03:21:35.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.794+0000 7f9113c415c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-09-23T03:21:35.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.794+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:32 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b7542700 con 0x5625b751fc00 2023-09-23T03:21:35.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.794+0000 7f5ee7f3a640 1 -- 172.21.15.154:0/1042754689 <== osd.7 v2:172.21.15.190:6824/3460136272 16 ==== osd_op_reply(36 [pgnls-filter start_epoch 67 out=49b] v65'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b4e6a2d80 con 0x557b4dc4f800 2023-09-23T03:21:35.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.794+0000 7f5ee2f30640 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:37 9.31 9:8c000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x557b4dc79180 con 0x557b4db3c800 2023-09-23T03:21:35.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.795+0000 7f5ee6f38640 1 -- 172.21.15.154:0/1042754689 <== osd.4 v2:172.21.15.190:6808/717302641 2 ==== osd_op_reply(37 [pgnls-filter start_epoch 67 out=49b] v65'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b4e6a2b40 con 0x557b4db3c800 2023-09-23T03:21:35.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.795+0000 7f5ee272f640 1 -- 172.21.15.154:0/1042754689 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:38 9.9 9:90000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x557b4dc9a000 con 0x557b4db3c400 2023-09-23T03:21:35.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.795+0000 7f5ee7f3a640 1 -- 172.21.15.154:0/1042754689 <== osd.0 v2:172.21.15.154:6800/2920938138 10 ==== osd_op_reply(38 [pgnls-filter start_epoch 67 out=49b] v65'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557b4e6a2d80 con 0x557b4db3c400 2023-09-23T03:21:35.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.795+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x557b4db3c400 msgr2=0x557b4d86a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.795+0000 7f5ee8f915c0 1 --2- 172.21.15.154:0/1042754689 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x557b4db3c400 0x557b4d86a100 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.795+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x557b4db3c000 msgr2=0x557b4d869b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.795+0000 7f5ee8f915c0 1 --2- 172.21.15.154:0/1042754689 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x557b4db3c000 0x557b4d869b80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.795+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x557b4dc9c000 msgr2=0x557b4dc88680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.795+0000 7f5ee8f915c0 1 --2- 172.21.15.154:0/1042754689 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x557b4dc9c000 0x557b4dc88680 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.795+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x557b4dc94400 msgr2=0x557b4dc86b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.795+0000 7f5ee8f915c0 1 --2- 172.21.15.154:0/1042754689 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x557b4dc94400 0x557b4dc86b00 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.795+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x557b4db3c800 msgr2=0x557b4d869600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.795+0000 7f5ee8f915c0 1 --2- 172.21.15.154:0/1042754689 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x557b4db3c800 0x557b4d869600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.795+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x557b4dc4f400 msgr2=0x557b4dc86000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.795+0000 7f5ee8f915c0 1 --2- 172.21.15.154:0/1042754689 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x557b4dc4f400 0x557b4dc86000 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.795+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x557b4dc4f800 msgr2=0x557b4dc86580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.795+0000 7f5ee8f915c0 1 --2- 172.21.15.154:0/1042754689 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x557b4dc4f800 0x557b4dc86580 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.796+0000 7f5ee7739640 1 -- 172.21.15.154:0/1042754689 reap_dead start 2023-09-23T03:21:35.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.796+0000 7f5ee7739640 1 -- 172.21.15.154:0/1042754689 reap_dead start 2023-09-23T03:21:35.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.796+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x557b4dc4e000 msgr2=0x557b4d86b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.796+0000 7f5ee8f915c0 1 --2- 172.21.15.154:0/1042754689 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x557b4dc4e000 0x557b4d86b700 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x557b4d8f0780 tx=0x557b4dc60000 comp rx=0 tx=0).stop 2023-09-23T03:21:35.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.796+0000 7f5ee7739640 1 -- 172.21.15.154:0/1042754689 reap_dead start 2023-09-23T03:21:35.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.796+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b4db3d000 msgr2=0x557b4d86ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.796+0000 7f5ee8f915c0 1 --2- 172.21.15.154:0/1042754689 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b4db3d000 0x557b4d86ac00 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x557b4e5cf770 tx=0x557b4dad8360 comp rx=0 tx=0).stop 2023-09-23T03:21:35.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.796+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 shutdown_connections 2023-09-23T03:21:35.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.796+0000 7f5ee8f915c0 1 --2- 172.21.15.154:0/1042754689 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b4db3d000 0x557b4d86ac00 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.796+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 >> 172.21.15.154:0/1042754689 conn(0x557b4e5d5c00 msgr2=0x557b4e579000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:35.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.796+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 shutdown_connections 2023-09-23T03:21:35.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.796+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/1042754689 wait complete. 2023-09-23T03:21:35.951 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--worker_n', '1', '--worker_m', '7'] in time 0:00:00.405770, result: 2023-09-23T03:21:35.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.797+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/2105377283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b4d7ff800 msgr2=0x557b4d868b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.797+0000 7f5ee8f915c0 1 --2- 172.21.15.154:0/2105377283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b4d7ff800 0x557b4d868b00 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x557b4e604a20 tx=0x557b4dad8120 comp rx=0 tx=0).stop 2023-09-23T03:21:35.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.797+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/2105377283 shutdown_connections 2023-09-23T03:21:35.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.797+0000 7f5ee8f915c0 1 --2- 172.21.15.154:0/2105377283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557b4d7ffc00 0x557b4d869080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.797+0000 7f5ee8f915c0 1 --2- 172.21.15.154:0/2105377283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557b4d7ff800 0x557b4d868b00 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.797+0000 7f5ee8f915c0 1 --2- 172.21.15.154:0/2105377283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557b4d7fec00 0x557b4d868580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.797+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/2105377283 >> 172.21.15.154:0/2105377283 conn(0x557b4d7ff000 msgr2=0x557b4e578800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:35.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.797+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/2105377283 shutdown_connections 2023-09-23T03:21:35.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.797+0000 7f5ee8f915c0 1 -- 172.21.15.154:0/2105377283 wait complete. 2023-09-23T03:21:35.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.797+0000 7f18e005b640 1 -- 172.21.15.154:0/1830651065 <== osd.7 v2:172.21.15.190:6824/3460136272 9 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v67'29 uv29 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5576f779afc0 con 0x5576f6ce5800 2023-09-23T03:21:35.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.797+0000 7f18e20b65c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/3 pointing to 0x0x100000001f8 2023-09-23T03:21:35.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.797+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:20 9.39 9:9c000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5576f6cf1880 con 0x5576f78a5c00 2023-09-23T03:21:35.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.798+0000 7f18e085c640 1 -- 172.21.15.154:0/1830651065 <== osd.4 v2:172.21.15.190:6808/717302641 2 ==== osd_op_reply(20 [pgnls-filter start_epoch 67 out=49b] v65'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5576f779ab40 con 0x5576f78a5c00 2023-09-23T03:21:35.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.798+0000 7f18dc053640 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:21 9.5 9:a0000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5576f6cf0000 con 0x5576f6ce4000 2023-09-23T03:21:35.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.799+0000 7f18e105d640 1 -- 172.21.15.154:0/1830651065 <== osd.0 v2:172.21.15.154:6800/2920938138 3 ==== osd_op_reply(21 [pgnls-filter start_epoch 67 out=81b] v67'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5576f779ad80 con 0x5576f6ce4000 2023-09-23T03:21:35.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.799+0000 7f18e20b65c0 20 datascan.forall_objects: OSD matched oid 10000000204.00000000 2023-09-23T03:21:35.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.799+0000 7f18e20b65c0 10 datascan.operator(): handling object 10000000204.0 2023-09-23T03:21:35.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.799+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:22 9.5 9:a0beda95:::10000000204.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5576f6cf1c00 con 0x5576f6ce4000 2023-09-23T03:21:35.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.799+0000 7f18e105d640 1 -- 172.21.15.154:0/1830651065 <== osd.0 v2:172.21.15.154:6800/2920938138 4 ==== osd_op_reply(22 10000000204.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv9 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x5576f779ad80 con 0x5576f6ce4000 2023-09-23T03:21:35.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.799+0000 7f18e20b65c0 1 --2- 172.21.15.154:0/1830651065 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5576f6ce5400 0x5576f6cfeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.799+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:23 10.5 10:a0beda95:::10000000204.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5576f6cf0380 con 0x5576f6ce5400 2023-09-23T03:21:35.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.799+0000 7f42fa431640 1 -- 172.21.15.154:0/741382839 <== osd.7 v2:172.21.15.190:6824/3460136272 8 ==== osd_op_reply(20 10000000000.00000000 [omap-set-vals] v67'30 uv30 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55694afb8b40 con 0x55694a494800 2023-09-23T03:21:35.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.799+0000 7f42fcc4a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/1 pointing to 0x0x100000001f6 2023-09-23T03:21:35.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.799+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:21 9.1a 9:58000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55694a485180 con 0x55694a494000 2023-09-23T03:21:35.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.799+0000 7f18e085c640 1 --2- 172.21.15.154:0/1830651065 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5576f6ce5400 0x5576f6cfeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.800+0000 7f42fb433640 1 -- 172.21.15.154:0/741382839 <== osd.3 v2:172.21.15.154:6808/1133688468 5 ==== osd_op_reply(21 [pgnls-filter start_epoch 67 out=49b] v65'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55694afb8d80 con 0x55694a494000 2023-09-23T03:21:35.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.800+0000 7f64993fd640 1 -- 172.21.15.154:0/3049751456 <== osd.7 v2:172.21.15.190:6824/3460136272 12 ==== osd_op_reply(20 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv30 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55de26ab4d80 con 0x55de25f7d800 2023-09-23T03:21:35.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.800+0000 7f649a4565c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=6 frag=*, r=0 2023-09-23T03:21:35.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.800+0000 7f649a4565c0 20 datascan.read_dentry: 6_head not found in result 2023-09-23T03:21:35.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.800+0000 7f649a4565c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fb at 0x0x10000000000/6 with size=6291456 bytes 2023-09-23T03:21:35.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.800+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:21 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55de25f86e00 con 0x55de25f7d800 2023-09-23T03:21:35.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.800+0000 7f42f5c28640 1 -- 172.21.15.154:0/741382839 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:22 9.3a 9:5c000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55694a153c00 con 0x55694a271000 2023-09-23T03:21:35.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.800+0000 7f18e085c640 1 --2- 172.21.15.154:0/1830651065 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5576f6ce5400 0x5576f6cfeb00 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.800+0000 7f911240d640 1 -- 172.21.15.154:0/2694809311 <== osd.7 v2:172.21.15.190:6824/3460136272 15 ==== osd_op_reply(32 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv30 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5625b7f5ed80 con 0x5625b751fc00 2023-09-23T03:21:35.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.800+0000 7f42fb433640 1 -- 172.21.15.154:0/741382839 <== osd.0 v2:172.21.15.154:6800/2920938138 2 ==== osd_op_reply(22 [pgnls-filter start_epoch 67 out=49b] v65'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55694afb8d80 con 0x55694a271000 2023-09-23T03:21:35.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.800+0000 7f9113c415c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=13 frag=*, r=0 2023-09-23T03:21:35.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.800+0000 7f9113c415c0 20 datascan.read_dentry: 13_head not found in result 2023-09-23T03:21:35.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.800+0000 7f9113c415c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000202 at 0x0x10000000000/13 with size=6291456 bytes 2023-09-23T03:21:35.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.800+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:33 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b711b880 con 0x5625b751fc00 2023-09-23T03:21:35.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.800+0000 7f42f6429640 1 -- 172.21.15.154:0/741382839 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:23 9.6 9:60000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55694a484e00 con 0x55694a271800 2023-09-23T03:21:35.958 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.800+0000 7f21f3263640 1 -- 172.21.15.154:0/684274785 <== osd.7 v2:172.21.15.190:6824/3460136272 7 ==== osd_op_reply(17 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv30 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x562cd5cfcfc0 con 0x562cd5eb0400 2023-09-23T03:21:35.958 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.800+0000 7f21f4abb5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=18 frag=*, r=0 2023-09-23T03:21:35.958 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.800+0000 7f21f4abb5c0 20 datascan.read_dentry: 18_head not found in result 2023-09-23T03:21:35.958 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.801+0000 7f18e085c640 1 -- 172.21.15.154:0/1830651065 <== osd.2 v2:172.21.15.154:6822/2884407243 1 ==== osd_op_reply(23 10000000204.00000000 [getxattr (30) out=30b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x5576f779ab40 con 0x5576f6ce5400 2023-09-23T03:21:35.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.801+0000 7f18e20b65c0 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-09-23T03:21:35.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.801+0000 7f18e20b65c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000204 2023-09-23T03:21:35.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.801+0000 7f18e20b65c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/15 2023-09-23T03:21:35.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.801+0000 7f18e20b65c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=15 2023-09-23T03:21:35.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.801+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:24 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5576f6cf1180 con 0x5576f6ce5800 2023-09-23T03:21:35.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.800+0000 7f21f4abb5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000207 at 0x0x10000000000/18 with size=6291456 bytes 2023-09-23T03:21:35.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.800+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:18 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd5e94700 con 0x562cd5eb0400 2023-09-23T03:21:35.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.801+0000 7f42fa431640 1 -- 172.21.15.154:0/741382839 <== osd.5 v2:172.21.15.190:6800/882536885 3 ==== osd_op_reply(23 [pgnls-filter start_epoch 67 out=49b] v65'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55694afb8b40 con 0x55694a271800 2023-09-23T03:21:35.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.801+0000 7f42f5c28640 1 -- 172.21.15.154:0/741382839 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:24 9.26 9:64000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55694a153880 con 0x55694a271800 2023-09-23T03:21:35.961 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--worker_n', '3', '--worker_m', '7'] in time 0:00:00.415116, result: 2023-09-23T03:21:35.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.802+0000 7f42fa431640 1 -- 172.21.15.154:0/741382839 <== osd.5 v2:172.21.15.190:6800/882536885 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 67 out=81b] v67'19 uv19 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55694afb8b40 con 0x55694a271800 2023-09-23T03:21:35.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.802+0000 7f42fcc4a5c0 20 datascan.forall_objects: OSD matched oid 100000001ff.00000000 2023-09-23T03:21:35.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.802+0000 7f42fcc4a5c0 10 datascan.operator(): handling object 100000001ff.0 2023-09-23T03:21:35.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.802+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:25 9.26 9:658f2455:::100000001ff.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55694a485c00 con 0x55694a271800 2023-09-23T03:21:35.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.803+0000 7f42fa431640 1 -- 172.21.15.154:0/741382839 <== osd.5 v2:172.21.15.190:6800/882536885 5 ==== osd_op_reply(25 100000001ff.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv19 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x55694afb8b40 con 0x55694a271800 2023-09-23T03:21:35.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.804+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:26 10.6 10:658f2455:::100000001ff.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55694a485500 con 0x55694a271800 2023-09-23T03:21:35.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.804+0000 7f64993fd640 1 -- 172.21.15.154:0/3049751456 <== osd.7 v2:172.21.15.190:6824/3460136272 13 ==== osd_op_reply(21 10000000000.00000000 [omap-set-vals] v67'31 uv31 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55de26ab4d80 con 0x55de25f7d800 2023-09-23T03:21:35.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.804+0000 7f649a4565c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/6 pointing to 0x0x100000001fb 2023-09-23T03:21:35.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.804+0000 7f649a4565c0 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55de25f7d400 0x55de25f96580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:35.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.804+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:22 9.33 9:cc000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55de25f86a80 con 0x55de25f7d400 2023-09-23T03:21:35.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.805+0000 7f64983fb640 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55de25f7d400 0x55de25f96580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:35.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.805+0000 7f42fa431640 1 -- 172.21.15.154:0/741382839 <== osd.5 v2:172.21.15.190:6800/882536885 6 ==== osd_op_reply(26 100000001ff.00000000 [getxattr (30) out=30b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55694afb8b40 con 0x55694a271800 2023-09-23T03:21:35.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.805+0000 7f42fcc4a5c0 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-09-23T03:21:35.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.805+0000 7f42fcc4a5c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001ff 2023-09-23T03:21:35.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.805+0000 7f42fcc4a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/10 2023-09-23T03:21:35.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.805+0000 7f42fcc4a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=10 2023-09-23T03:21:35.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.805+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:27 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55694a485180 con 0x55694a494800 2023-09-23T03:21:35.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.805+0000 7f21f3263640 1 -- 172.21.15.154:0/684274785 <== osd.7 v2:172.21.15.190:6824/3460136272 8 ==== osd_op_reply(18 10000000000.00000000 [omap-set-vals] v67'32 uv32 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562cd5cfcfc0 con 0x562cd5eb0400 2023-09-23T03:21:35.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.805+0000 7f21f4abb5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/18 pointing to 0x0x10000000207 2023-09-23T03:21:35.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.805+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:19 9.7 9:e2e07930::::head [pgnls-filter start_epoch 67 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd5e95180 con 0x562cd5eb0800 2023-09-23T03:21:35.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.806+0000 7f911240d640 1 -- 172.21.15.154:0/2694809311 <== osd.7 v2:172.21.15.190:6824/3460136272 16 ==== osd_op_reply(33 10000000000.00000000 [omap-set-vals] v67'33 uv33 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5625b7f5ed80 con 0x5625b751fc00 2023-09-23T03:21:35.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.806+0000 7f9113c415c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/13 pointing to 0x0x10000000202 2023-09-23T03:21:35.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.806+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:34 9.28 9:177a762d::::head [pgnls-filter start_epoch 67 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b7542380 con 0x5625b751f800 2023-09-23T03:21:35.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.806+0000 7f64983fb640 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55de25f7d400 0x55de25f96580 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:35.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.806+0000 7f18e005b640 1 -- 172.21.15.154:0/1830651065 <== osd.7 v2:172.21.15.190:6824/3460136272 10 ==== osd_op_reply(24 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x5576f779afc0 con 0x5576f6ce5800 2023-09-23T03:21:35.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.806+0000 7f18e20b65c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-09-23T03:21:35.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.806+0000 7f18e20b65c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-09-23T03:21:35.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.806+0000 7f18e20b65c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/15, will insert to root 2023-09-23T03:21:35.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.806+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:25 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5576f6cf0700 con 0x5576f6ce5800 2023-09-23T03:21:35.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.806+0000 7f21f2a62640 1 -- 172.21.15.154:0/684274785 <== osd.5 v2:172.21.15.190:6800/882536885 3 ==== osd_op_reply(19 [pgnls-filter start_epoch 67 out=81b] v67'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562cd5cfcb40 con 0x562cd5eb0800 2023-09-23T03:21:35.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.806+0000 7f21f4abb5c0 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2023-09-23T03:21:35.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.806+0000 7f21f4abb5c0 10 datascan.operator(): handling object 10000000001.0 2023-09-23T03:21:35.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.806+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:20 9.7 9:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd5e94380 con 0x562cd5eb0800 2023-09-23T03:21:35.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.806+0000 7f911140b640 1 -- 172.21.15.154:0/2694809311 <== osd.6 v2:172.21.15.190:6816/1964294001 5 ==== osd_op_reply(34 [pgnls-filter start_epoch 67 out=81b] v67'14 uv14 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5625b7f5eb40 con 0x5625b751f800 2023-09-23T03:21:35.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.807+0000 7f9113c415c0 20 datascan.forall_objects: OSD matched oid 10000000201.00000000 2023-09-23T03:21:35.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.807+0000 7f9113c415c0 10 datascan.operator(): handling object 10000000201.0 2023-09-23T03:21:35.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.807+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:35 9.28 9:177a762d:::10000000201.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b7543c00 con 0x5625b751f800 2023-09-23T03:21:35.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.807+0000 7f42fa431640 1 -- 172.21.15.154:0/741382839 <== osd.7 v2:172.21.15.190:6824/3460136272 9 ==== osd_op_reply(27 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55694afb8b40 con 0x55694a494800 2023-09-23T03:21:35.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.807+0000 7f42fcc4a5c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-09-23T03:21:35.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.807+0000 7f42fcc4a5c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-09-23T03:21:35.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.807+0000 7f42fcc4a5c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/10, will insert to root 2023-09-23T03:21:35.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.807+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:28 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55694a484380 con 0x55694a494800 2023-09-23T03:21:35.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.807+0000 7f64983fb640 1 -- 172.21.15.154:0/3049751456 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== osd_op_reply(22 [pgnls-filter start_epoch 67 out=49b] v65'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55de26ab4b40 con 0x55de25f7d400 2023-09-23T03:21:35.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.807+0000 7f64943f3640 1 -- 172.21.15.154:0/3049751456 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:23 9.b 9:d0000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55de25f9a380 con 0x55de25f7c000 2023-09-23T03:21:35.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.807+0000 7f18e005b640 1 -- 172.21.15.154:0/1830651065 <== osd.7 v2:172.21.15.190:6824/3460136272 11 ==== osd_op_reply(25 10000000000.00000000 [omap-get-header out=274b] v0'0 uv33 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5576f779afc0 con 0x5576f6ce5800 2023-09-23T03:21:35.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.807+0000 7f18e20b65c0 10 datascan.find_or_create_dirfrag: read_version = 33 2023-09-23T03:21:35.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.807+0000 7f18e20b65c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-09-23T03:21:35.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.807+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:26 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5576f6cf0000 con 0x5576f6ce5800 2023-09-23T03:21:35.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.807+0000 7f64993fd640 1 -- 172.21.15.154:0/3049751456 <== osd.0 v2:172.21.15.154:6800/2920938138 3 ==== osd_op_reply(23 [pgnls-filter start_epoch 67 out=49b] v65'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55de26ab4d80 con 0x55de25f7c000 2023-09-23T03:21:35.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.807+0000 7f21f2a62640 1 -- 172.21.15.154:0/684274785 <== osd.5 v2:172.21.15.190:6800/882536885 4 ==== osd_op_reply(20 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x562cd5cfcb40 con 0x562cd5eb0800 2023-09-23T03:21:35.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.807+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:21 10.7 10:e2e07930:::10000000001.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd5e95c00 con 0x562cd5eb1000 2023-09-23T03:21:35.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.807+0000 7f6493bf2640 1 -- 172.21.15.154:0/3049751456 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:24 9.2b 9:d4000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55de25f86000 con 0x55de25f7d800 2023-09-23T03:21:35.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.808+0000 7f42fa431640 1 -- 172.21.15.154:0/741382839 <== osd.7 v2:172.21.15.190:6824/3460136272 10 ==== osd_op_reply(28 10000000000.00000000 [omap-get-header out=274b] v0'0 uv33 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55694afb8b40 con 0x55694a494800 2023-09-23T03:21:35.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.808+0000 7f42fcc4a5c0 10 datascan.find_or_create_dirfrag: read_version = 33 2023-09-23T03:21:35.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.808+0000 7f42fcc4a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-09-23T03:21:35.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.808+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:29 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55694a484a80 con 0x55694a494800 2023-09-23T03:21:35.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.808+0000 7f911140b640 1 -- 172.21.15.154:0/2694809311 <== osd.6 v2:172.21.15.190:6816/1964294001 6 ==== osd_op_reply(35 10000000201.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv14 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x5625b7f5eb40 con 0x5625b751f800 2023-09-23T03:21:35.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.808+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:36 10.8 10:177a762d:::10000000201.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b7543180 con 0x5625b7412400 2023-09-23T03:21:35.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.808+0000 7f18e005b640 1 -- 172.21.15.154:0/1830651065 <== osd.7 v2:172.21.15.190:6824/3460136272 12 ==== osd_op_reply(26 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv33 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5576f779afc0 con 0x5576f6ce5800 2023-09-23T03:21:35.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.808+0000 7f18e20b65c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=15 frag=*, r=0 2023-09-23T03:21:35.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.808+0000 7f18e20b65c0 20 datascan.read_dentry: 15_head not found in result 2023-09-23T03:21:35.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.808+0000 7f18e20b65c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000204 at 0x0x10000000000/15 with size=6291456 bytes 2023-09-23T03:21:35.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.808+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:27 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x5576f6cf1880 con 0x5576f6ce5800 2023-09-23T03:21:35.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.808+0000 7f911240d640 1 -- 172.21.15.154:0/2694809311 <== osd.0 v2:172.21.15.154:6800/2920938138 6 ==== osd_op_reply(36 10000000201.00000000 [getxattr (30) out=30b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x5625b7f5ed80 con 0x5625b7412400 2023-09-23T03:21:35.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.808+0000 7f9113c415c0 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-09-23T03:21:35.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.808+0000 7f9113c415c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000201 2023-09-23T03:21:35.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.808+0000 7f9113c415c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/12 2023-09-23T03:21:35.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.808+0000 7f9113c415c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=12 2023-09-23T03:21:35.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.808+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:37 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b7543500 con 0x5625b751fc00 2023-09-23T03:21:35.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.808+0000 7f42fa431640 1 -- 172.21.15.154:0/741382839 <== osd.7 v2:172.21.15.190:6824/3460136272 11 ==== osd_op_reply(29 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv33 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55694afb8b40 con 0x55694a494800 2023-09-23T03:21:35.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.808+0000 7f42fcc4a5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=10 frag=*, r=0 2023-09-23T03:21:35.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.808+0000 7f42fcc4a5c0 20 datascan.read_dentry: 10_head not found in result 2023-09-23T03:21:35.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.808+0000 7f42fcc4a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001ff at 0x0x10000000000/10 with size=6291456 bytes 2023-09-23T03:21:35.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.808+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:30 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55694a484700 con 0x55694a494800 2023-09-23T03:21:35.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.808+0000 7f21f3a64640 1 -- 172.21.15.154:0/684274785 <== osd.6 v2:172.21.15.190:6816/1964294001 2 ==== osd_op_reply(21 10000000001.00000000 [getxattr (30) out=30b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x562cd5cfcd80 con 0x562cd5eb1000 2023-09-23T03:21:35.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.808+0000 7f21f4abb5c0 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-09-23T03:21:35.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.808+0000 7f21f4abb5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2023-09-23T03:21:35.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.808+0000 7f21f4abb5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/0 2023-09-23T03:21:35.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.808+0000 7f21f4abb5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=0 2023-09-23T03:21:35.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.808+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:22 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd5e95500 con 0x562cd5eb0400 2023-09-23T03:21:35.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.808+0000 7f64993fd640 1 -- 172.21.15.154:0/3049751456 <== osd.7 v2:172.21.15.190:6824/3460136272 14 ==== osd_op_reply(24 [pgnls-filter start_epoch 67 out=81b] v67'15 uv15 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55de26ab4d80 con 0x55de25f7d800 2023-09-23T03:21:35.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.808+0000 7f649a4565c0 20 datascan.forall_objects: OSD matched oid 10000000203.00000000 2023-09-23T03:21:35.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.808+0000 7f649a4565c0 10 datascan.operator(): handling object 10000000203.0 2023-09-23T03:21:35.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.809+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:25 9.2b 9:d4a58307:::10000000203.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55de25f87500 con 0x55de25f7d800 2023-09-23T03:21:35.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.809+0000 7f64993fd640 1 -- 172.21.15.154:0/3049751456 <== osd.7 v2:172.21.15.190:6824/3460136272 15 ==== osd_op_reply(25 10000000203.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x55de26ab4d80 con 0x55de25f7d800 2023-09-23T03:21:35.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.809+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:26 10.b 10:d4a58307:::10000000203.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55de25c61880 con 0x55de25f7d400 2023-09-23T03:21:35.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.810+0000 7f64983fb640 1 -- 172.21.15.154:0/3049751456 <== osd.6 v2:172.21.15.190:6816/1964294001 2 ==== osd_op_reply(26 10000000203.00000000 [getxattr (30) out=30b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55de26ab4b40 con 0x55de25f7d400 2023-09-23T03:21:35.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.810+0000 7f649a4565c0 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-09-23T03:21:35.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.810+0000 7f649a4565c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000203 2023-09-23T03:21:35.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.810+0000 7f649a4565c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/14 2023-09-23T03:21:35.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.810+0000 7f649a4565c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=14 2023-09-23T03:21:35.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.810+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:27 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55de25c61c00 con 0x55de25f7d800 2023-09-23T03:21:35.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.811+0000 7f18e005b640 1 -- 172.21.15.154:0/1830651065 <== osd.7 v2:172.21.15.190:6824/3460136272 13 ==== osd_op_reply(27 10000000000.00000000 [omap-set-vals] v67'34 uv34 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5576f779afc0 con 0x5576f6ce5800 2023-09-23T03:21:35.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.811+0000 7f18e20b65c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/15 pointing to 0x0x10000000204 2023-09-23T03:21:35.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.811+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:28 9.25 9:a4000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5576f69db880 con 0x5576f6ce4000 2023-09-23T03:21:35.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.811+0000 7f18e105d640 1 -- 172.21.15.154:0/1830651065 <== osd.0 v2:172.21.15.154:6800/2920938138 5 ==== osd_op_reply(28 [pgnls-filter start_epoch 67 out=49b] v65'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5576f779ad80 con 0x5576f6ce4000 2023-09-23T03:21:35.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.811+0000 7f18dc053640 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:29 9.15 9:a8000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5576f6cf1500 con 0x5576f6ce5800 2023-09-23T03:21:35.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.812+0000 7f911240d640 1 -- 172.21.15.154:0/2694809311 <== osd.7 v2:172.21.15.190:6824/3460136272 17 ==== osd_op_reply(37 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x5625b7f5ed80 con 0x5625b751fc00 2023-09-23T03:21:35.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.812+0000 7f9113c415c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-09-23T03:21:35.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.812+0000 7f9113c415c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-09-23T03:21:35.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.812+0000 7f9113c415c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/12, will insert to root 2023-09-23T03:21:35.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.812+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:38 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b7542000 con 0x5625b751fc00 2023-09-23T03:21:35.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.812+0000 7f18e005b640 1 -- 172.21.15.154:0/1830651065 <== osd.7 v2:172.21.15.190:6824/3460136272 14 ==== osd_op_reply(29 [pgnls-filter start_epoch 67 out=81b] v67'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5576f779afc0 con 0x5576f6ce5800 2023-09-23T03:21:35.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.813+0000 7f18e20b65c0 20 datascan.forall_objects: OSD matched oid 1000000020d.00000000 2023-09-23T03:21:35.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.813+0000 7f18e20b65c0 10 datascan.operator(): handling object 1000000020d.0 2023-09-23T03:21:35.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.813+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:30 9.15 9:a8e69ea8:::1000000020d.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5576f6cf0a80 con 0x5576f6ce5800 2023-09-23T03:21:35.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.813+0000 7f18e005b640 1 -- 172.21.15.154:0/1830651065 <== osd.7 v2:172.21.15.190:6824/3460136272 15 ==== osd_op_reply(30 1000000020d.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv10 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x5576f779afc0 con 0x5576f6ce5800 2023-09-23T03:21:35.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.813+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:31 10.15 10:a8e69ea8:::1000000020d.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5576f6cf1180 con 0x5576f6ce5c00 2023-09-23T03:21:35.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.814+0000 7f18e105d640 1 -- 172.21.15.154:0/1830651065 <== osd.3 v2:172.21.15.154:6808/1133688468 5 ==== osd_op_reply(31 1000000020d.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x5576f779ad80 con 0x5576f6ce5c00 2023-09-23T03:21:35.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.814+0000 7f18e20b65c0 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-09-23T03:21:35.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.814+0000 7f18e20b65c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000020d 2023-09-23T03:21:35.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.814+0000 7f18e20b65c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/24 2023-09-23T03:21:35.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.814+0000 7f18e20b65c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=24 2023-09-23T03:21:35.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.814+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:32 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5576f6cf0700 con 0x5576f6ce5800 2023-09-23T03:21:35.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.815+0000 7f42fa431640 1 -- 172.21.15.154:0/741382839 <== osd.7 v2:172.21.15.190:6824/3460136272 12 ==== osd_op_reply(30 10000000000.00000000 [omap-set-vals] v67'35 uv35 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55694afb8b40 con 0x55694a494800 2023-09-23T03:21:35.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.815+0000 7f42fcc4a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/10 pointing to 0x0x100000001ff 2023-09-23T03:21:35.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.815+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:31 9.16 9:68000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55694a153c00 con 0x55694a270c00 2023-09-23T03:21:35.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.815+0000 7f21f3263640 1 -- 172.21.15.154:0/684274785 <== osd.7 v2:172.21.15.190:6824/3460136272 9 ==== osd_op_reply(22 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x562cd5cfcfc0 con 0x562cd5eb0400 2023-09-23T03:21:35.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.815+0000 7f21f4abb5c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-09-23T03:21:35.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.815+0000 7f21f4abb5c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-09-23T03:21:35.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.815+0000 7f21f4abb5c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/0, will insert to root 2023-09-23T03:21:35.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.815+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:23 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd5e94000 con 0x562cd5eb0400 2023-09-23T03:21:35.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.816+0000 7f42fa431640 1 -- 172.21.15.154:0/741382839 <== osd.4 v2:172.21.15.190:6808/717302641 3 ==== osd_op_reply(31 [pgnls-filter start_epoch 67 out=49b] v65'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55694afb8b40 con 0x55694a270c00 2023-09-23T03:21:35.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.816+0000 7f42f5c28640 1 -- 172.21.15.154:0/741382839 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:32 9.36 9:6c000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55694a484e00 con 0x55694a271400 2023-09-23T03:21:35.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.816+0000 7f64993fd640 1 -- 172.21.15.154:0/3049751456 <== osd.7 v2:172.21.15.190:6824/3460136272 16 ==== osd_op_reply(27 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55de26ab4d80 con 0x55de25f7d800 2023-09-23T03:21:35.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.816+0000 7f649a4565c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-09-23T03:21:35.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.816+0000 7f649a4565c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-09-23T03:21:35.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.816+0000 7f649a4565c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/14, will insert to root 2023-09-23T03:21:35.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.816+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:28 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55de25f9a000 con 0x55de25f7d800 2023-09-23T03:21:35.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.816+0000 7f911240d640 1 -- 172.21.15.154:0/2694809311 <== osd.7 v2:172.21.15.190:6824/3460136272 18 ==== osd_op_reply(38 10000000000.00000000 [omap-get-header out=274b] v0'0 uv35 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5625b7f5ed80 con 0x5625b751fc00 2023-09-23T03:21:35.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.816+0000 7f9113c415c0 10 datascan.find_or_create_dirfrag: read_version = 35 2023-09-23T03:21:35.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.816+0000 7f9113c415c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-09-23T03:21:35.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.816+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:39 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b7543880 con 0x5625b751fc00 2023-09-23T03:21:35.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.816+0000 7f42fac32640 1 -- 172.21.15.154:0/741382839 <== osd.1 v2:172.21.15.154:6816/1267710005 2 ==== osd_op_reply(32 [pgnls-filter start_epoch 67 out=101b] v67'23 uv23 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55694afb8fc0 con 0x55694a271400 2023-09-23T03:21:35.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.816+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55694a271000 msgr2=0x55694a163b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.816+0000 7f42fcc4a5c0 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55694a271000 0x55694a163b80 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.816+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55694a271400 msgr2=0x55694a163600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.816+0000 7f42fcc4a5c0 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55694a271400 0x55694a163600 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.816+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55694a494000 msgr2=0x55694a48e680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.816+0000 7f42fcc4a5c0 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55694a494000 0x55694a48e680 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.816+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55694a270c00 msgr2=0x55694a164100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.816+0000 7f42fcc4a5c0 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55694a270c00 0x55694a164100 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.816+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55694a271800 msgr2=0x55694a48c000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.816+0000 7f42fcc4a5c0 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55694a271800 0x55694a48c000 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.816+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55694a494400 msgr2=0x55694a48c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.986 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.816+0000 7f42fcc4a5c0 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55694a494400 0x55694a48c580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.986 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.816+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55694a494800 msgr2=0x55694a48cb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.986 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.816+0000 7f42fcc4a5c0 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55694a494800 0x55694a48cb00 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.986 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.816+0000 7f42fac32640 1 -- 172.21.15.154:0/741382839 reap_dead start 2023-09-23T03:21:35.986 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.816+0000 7f42fac32640 1 -- 172.21.15.154:0/741382839 reap_dead start 2023-09-23T03:21:35.986 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.816+0000 7f42fac32640 1 -- 172.21.15.154:0/741382839 reap_dead start 2023-09-23T03:21:35.986 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.816+0000 7f18e005b640 1 -- 172.21.15.154:0/1830651065 <== osd.7 v2:172.21.15.190:6824/3460136272 16 ==== osd_op_reply(32 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x5576f779afc0 con 0x5576f6ce5800 2023-09-23T03:21:35.986 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.816+0000 7f18e20b65c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-09-23T03:21:35.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.816+0000 7f18e20b65c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-09-23T03:21:35.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.816+0000 7f18e20b65c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/24, will insert to root 2023-09-23T03:21:35.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.816+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55694a270800 msgr2=0x55694a165700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.816+0000 7f42fcc4a5c0 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55694a270800 0x55694a165700 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x55694ae70600 tx=0x55694aed62a0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.816+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55694afeb000 msgr2=0x55694a165180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.816+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:33 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5576f6cf0000 con 0x5576f6ce5800 2023-09-23T03:21:35.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.816+0000 7f42fcc4a5c0 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55694afeb000 0x55694a165180 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x55694aeeb5f0 tx=0x55694aff2e40 comp rx=0 tx=0).stop 2023-09-23T03:21:35.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.817+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 shutdown_connections 2023-09-23T03:21:35.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.817+0000 7f42fcc4a5c0 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55694a494800 0x55694a48cb00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.817+0000 7f42fcc4a5c0 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55694a270800 0x55694a165700 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.817+0000 7f42fcc4a5c0 1 --2- 172.21.15.154:0/741382839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55694afeb000 0x55694a165180 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.817+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 >> 172.21.15.154:0/741382839 conn(0x55694afeac00 msgr2=0x55694a1f5800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:35.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.817+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 shutdown_connections 2023-09-23T03:21:35.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.817+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/741382839 wait complete. 2023-09-23T03:21:35.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.817+0000 7f21f3263640 1 -- 172.21.15.154:0/684274785 <== osd.7 v2:172.21.15.190:6824/3460136272 10 ==== osd_op_reply(23 10000000000.00000000 [omap-get-header out=274b] v0'0 uv35 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x562cd5cfcfc0 con 0x562cd5eb0400 2023-09-23T03:21:35.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.817+0000 7f21f4abb5c0 10 datascan.find_or_create_dirfrag: read_version = 35 2023-09-23T03:21:35.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.817+0000 7f21f4abb5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-09-23T03:21:35.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.817+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:24 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd5e95880 con 0x562cd5eb0400 2023-09-23T03:21:35.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.817+0000 7f64993fd640 1 -- 172.21.15.154:0/3049751456 <== osd.7 v2:172.21.15.190:6824/3460136272 17 ==== osd_op_reply(28 10000000000.00000000 [omap-get-header out=274b] v0'0 uv35 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55de26ab4d80 con 0x55de25f7d800 2023-09-23T03:21:35.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.817+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/1545018075 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55694a0f9800 msgr2=0x55694a162b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:35.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.817+0000 7f42fcc4a5c0 1 --2- 172.21.15.154:0/1545018075 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55694a0f9800 0x55694a162b00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x55694af1f4d0 tx=0x55694aed6600 comp rx=0 tx=0).stop 2023-09-23T03:21:35.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.817+0000 7f649a4565c0 10 datascan.find_or_create_dirfrag: read_version = 35 2023-09-23T03:21:35.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.817+0000 7f649a4565c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-09-23T03:21:35.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.817+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:29 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55de25f9a700 con 0x55de25f7d800 2023-09-23T03:21:35.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.817+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/1545018075 shutdown_connections 2023-09-23T03:21:35.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.817+0000 7f42fcc4a5c0 1 --2- 172.21.15.154:0/1545018075 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55694a0f9c00 0x55694a163080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.817+0000 7f42fcc4a5c0 1 --2- 172.21.15.154:0/1545018075 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55694a0f9800 0x55694a162b00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.817+0000 7f42fcc4a5c0 1 --2- 172.21.15.154:0/1545018075 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55694a0f8c00 0x55694a162580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:35.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.817+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/1545018075 >> 172.21.15.154:0/1545018075 conn(0x55694a0f9000 msgr2=0x55694a1f5000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:35.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.817+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/1545018075 shutdown_connections 2023-09-23T03:21:35.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.818+0000 7f42fcc4a5c0 1 -- 172.21.15.154:0/1545018075 wait complete. 2023-09-23T03:21:35.992 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.818+0000 7f911240d640 1 -- 172.21.15.154:0/2694809311 <== osd.7 v2:172.21.15.190:6824/3460136272 19 ==== osd_op_reply(39 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv35 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5625b7f5ed80 con 0x5625b751fc00 2023-09-23T03:21:35.992 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.818+0000 7f9113c415c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=12 frag=*, r=0 2023-09-23T03:21:35.992 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.818+0000 7f9113c415c0 20 datascan.read_dentry: 12_head not found in result 2023-09-23T03:21:35.992 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.818+0000 7f9113c415c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000201 at 0x0x10000000000/12 with size=6291456 bytes 2023-09-23T03:21:35.992 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.818+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:40 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b7542a80 con 0x5625b751fc00 2023-09-23T03:21:35.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.818+0000 7f18e005b640 1 -- 172.21.15.154:0/1830651065 <== osd.7 v2:172.21.15.190:6824/3460136272 17 ==== osd_op_reply(33 10000000000.00000000 [omap-get-header out=274b] v0'0 uv35 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5576f779afc0 con 0x5576f6ce5800 2023-09-23T03:21:35.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.818+0000 7f18e20b65c0 10 datascan.find_or_create_dirfrag: read_version = 35 2023-09-23T03:21:35.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.818+0000 7f18e20b65c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-09-23T03:21:35.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.818+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:34 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x5576f6cf1880 con 0x5576f6ce5800 2023-09-23T03:21:35.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.818+0000 7f21f3263640 1 -- 172.21.15.154:0/684274785 <== osd.7 v2:172.21.15.190:6824/3460136272 11 ==== osd_op_reply(24 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv35 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x562cd5cfcfc0 con 0x562cd5eb0400 2023-09-23T03:21:35.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.818+0000 7f21f4abb5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=0 frag=*, r=0 2023-09-23T03:21:35.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.818+0000 7f21f4abb5c0 20 datascan.read_dentry: 0_head not found in result 2023-09-23T03:21:35.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.818+0000 7f21f4abb5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/0 with size=6291456 bytes 2023-09-23T03:21:35.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.818+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:25 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd5e95180 con 0x562cd5eb0400 2023-09-23T03:21:35.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.819+0000 7f64993fd640 1 -- 172.21.15.154:0/3049751456 <== osd.7 v2:172.21.15.190:6824/3460136272 18 ==== osd_op_reply(29 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv35 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55de26ab4d80 con 0x55de25f7d800 2023-09-23T03:21:35.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.819+0000 7f649a4565c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=14 frag=*, r=0 2023-09-23T03:21:35.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.819+0000 7f649a4565c0 20 datascan.read_dentry: 14_head not found in result 2023-09-23T03:21:35.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.819+0000 7f649a4565c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000203 at 0x0x10000000000/14 with size=6291456 bytes 2023-09-23T03:21:35.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.819+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:30 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55de25f9aa80 con 0x55de25f7d800 2023-09-23T03:21:35.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.821+0000 7f911240d640 1 -- 172.21.15.154:0/2694809311 <== osd.7 v2:172.21.15.190:6824/3460136272 20 ==== osd_op_reply(40 10000000000.00000000 [omap-set-vals] v67'36 uv36 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5625b7f5ed80 con 0x5625b751fc00 2023-09-23T03:21:35.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.821+0000 7f9113c415c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/12 pointing to 0x0x10000000201 2023-09-23T03:21:35.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.821+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:41 9.18 9:18000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b7542700 con 0x5625b751f800 2023-09-23T03:21:35.996 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--worker_n', '2', '--worker_m', '7'] in time 0:00:00.451034, result: 2023-09-23T03:21:35.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.822+0000 7f18e005b640 1 -- 172.21.15.154:0/1830651065 <== osd.7 v2:172.21.15.190:6824/3460136272 18 ==== osd_op_reply(34 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv36 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5576f779afc0 con 0x5576f6ce5800 2023-09-23T03:21:35.997 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.822+0000 7f18e20b65c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=24 frag=*, r=0 2023-09-23T03:21:35.997 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.822+0000 7f18e20b65c0 20 datascan.read_dentry: 24_head not found in result 2023-09-23T03:21:35.997 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.822+0000 7f18e20b65c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000020d at 0x0x10000000000/24 with size=6291456 bytes 2023-09-23T03:21:35.997 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.822+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:35 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x5576f6cf1500 con 0x5576f6ce5800 2023-09-23T03:21:35.999 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.822+0000 7f911140b640 1 -- 172.21.15.154:0/2694809311 <== osd.6 v2:172.21.15.190:6816/1964294001 7 ==== osd_op_reply(41 [pgnls-filter start_epoch 67 out=49b] v65'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5625b7f5eb40 con 0x5625b751f800 2023-09-23T03:21:35.999 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.822+0000 7f910d403640 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:42 9.38 9:1c000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b7542380 con 0x5625b7560000 2023-09-23T03:21:35.999 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.822+0000 7f9111c0c640 1 -- 172.21.15.154:0/2694809311 <== osd.3 v2:172.21.15.154:6808/1133688468 2 ==== osd_op_reply(42 [pgnls-filter start_epoch 67 out=49b] v65'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5625b7f5efc0 con 0x5625b7560000 2023-09-23T03:21:35.999 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.822+0000 7f910cc02640 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:43 9.4 9:20000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b7542e00 con 0x5625b7412000 2023-09-23T03:21:35.999 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.823+0000 7f9111c0c640 1 -- 172.21.15.154:0/2694809311 <== osd.1 v2:172.21.15.154:6816/1267710005 4 ==== osd_op_reply(43 [pgnls-filter start_epoch 67 out=49b] v65'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5625b7f5efc0 con 0x5625b7412000 2023-09-23T03:21:35.999 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.823+0000 7f910d403640 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5625b7560400 0x5625b7540100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:36.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.823+0000 7f910d403640 1 -- 172.21.15.154:0/2694809311 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:44 9.24 9:24000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5625b711bc00 con 0x5625b7560400 2023-09-23T03:21:36.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.823+0000 7f911140b640 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5625b7560400 0x5625b7540100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:36.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.824+0000 7f911140b640 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5625b7560400 0x5625b7540100 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:36.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.824+0000 7f911140b640 1 -- 172.21.15.154:0/2694809311 <== osd.2 v2:172.21.15.154:6822/2884407243 1 ==== osd_op_reply(44 [pgnls-filter start_epoch 67 out=49b] v65'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5625b7f5eb40 con 0x5625b7560400 2023-09-23T03:21:36.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.824+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5625b7412400 msgr2=0x5625b712d180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.824+0000 7f9113c415c0 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5625b7412400 0x5625b712d180 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.824+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5625b7412000 msgr2=0x5625b753e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.824+0000 7f9113c415c0 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5625b7412000 0x5625b753e000 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.824+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5625b7560400 msgr2=0x5625b7540100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.824+0000 7f9113c415c0 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5625b7560400 0x5625b7540100 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.824+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5625b7560000 msgr2=0x5625b753fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.824+0000 7f9113c415c0 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5625b7560000 0x5625b753fb80 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.824+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5625b7412800 msgr2=0x5625b712cc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.824+0000 7f9113c415c0 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5625b7412800 0x5625b712cc00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.824+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x5625b751f800 msgr2=0x5625b753e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.824+0000 7f9113c415c0 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x5625b751f800 0x5625b753e580 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.824+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x5625b751fc00 msgr2=0x5625b753eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.824+0000 7f9113c415c0 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x5625b751fc00 0x5625b753eb00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.824+0000 7f9111c0c640 1 -- 172.21.15.154:0/2694809311 reap_dead start 2023-09-23T03:21:36.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.824+0000 7f9111c0c640 1 -- 172.21.15.154:0/2694809311 reap_dead start 2023-09-23T03:21:36.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.824+0000 7f9111c0c640 1 -- 172.21.15.154:0/2694809311 reap_dead start 2023-09-23T03:21:36.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.825+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5625b751e000 msgr2=0x5625b712d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.825+0000 7f9113c415c0 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5625b751e000 0x5625b712d700 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x5625b71b2120 tx=0x5625b73c4f60 comp rx=0 tx=0).stop 2023-09-23T03:21:36.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.825+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5625b70c0c00 msgr2=0x5625b712c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.825+0000 7f9113c415c0 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5625b70c0c00 0x5625b712c100 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x5625b73c15f0 tx=0x5625b73c4d80 comp rx=0 tx=0).stop 2023-09-23T03:21:36.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.825+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 shutdown_connections 2023-09-23T03:21:36.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.825+0000 7f9113c415c0 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5625b751e000 0x5625b712d700 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.825+0000 7f9113c415c0 1 --2- 172.21.15.154:0/2694809311 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5625b70c0c00 0x5625b712c100 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.825+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 >> 172.21.15.154:0/2694809311 conn(0x5625b7e91000 msgr2=0x5625b7e3b000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:36.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.825+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 shutdown_connections 2023-09-23T03:21:36.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.825+0000 7f9113c415c0 1 -- 172.21.15.154:0/2694809311 wait complete. 2023-09-23T03:21:36.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.825+0000 7f9113c415c0 1 -- 172.21.15.154:0/318524665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5625b7e91400 msgr2=0x5625b712b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.825+0000 7f9113c415c0 1 --2- 172.21.15.154:0/318524665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5625b7e91400 0x5625b712b600 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x5625b7ec5440 tx=0x5625b73aa0c0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.003 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.825+0000 7f21f3263640 1 -- 172.21.15.154:0/684274785 <== osd.7 v2:172.21.15.190:6824/3460136272 12 ==== osd_op_reply(25 10000000000.00000000 [omap-set-vals] v67'37 uv37 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562cd5cfcfc0 con 0x562cd5eb0400 2023-09-23T03:21:36.003 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.825+0000 7f9113c415c0 1 -- 172.21.15.154:0/318524665 shutdown_connections 2023-09-23T03:21:36.003 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.825+0000 7f9113c415c0 1 --2- 172.21.15.154:0/318524665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5625b7e91400 0x5625b712b600 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.003 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.825+0000 7f9113c415c0 1 --2- 172.21.15.154:0/318524665 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5625b70c1c00 0x5625b712b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.003 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.825+0000 7f9113c415c0 1 --2- 172.21.15.154:0/318524665 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5625b70c1800 0x5625b712ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.003 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.825+0000 7f9113c415c0 1 -- 172.21.15.154:0/318524665 >> 172.21.15.154:0/318524665 conn(0x5625b70c1000 msgr2=0x5625b7e3a800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:36.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.825+0000 7f21f4abb5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/0 pointing to 0x0x10000000001 2023-09-23T03:21:36.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.825+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:26 9.27 9:e4000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd5e94a80 con 0x562cd5eb0000 2023-09-23T03:21:36.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.825+0000 7f9113c415c0 1 -- 172.21.15.154:0/318524665 shutdown_connections 2023-09-23T03:21:36.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.825+0000 7f9113c415c0 1 -- 172.21.15.154:0/318524665 wait complete. 2023-09-23T03:21:36.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.826+0000 7f21f3a64640 1 -- 172.21.15.154:0/684274785 <== osd.2 v2:172.21.15.154:6822/2884407243 3 ==== osd_op_reply(26 [pgnls-filter start_epoch 67 out=81b] v67'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562cd5cfcd80 con 0x562cd5eb0000 2023-09-23T03:21:36.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.826+0000 7f21f4abb5c0 20 datascan.forall_objects: OSD matched oid 10000000208.00000000 2023-09-23T03:21:36.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.826+0000 7f21f4abb5c0 10 datascan.operator(): handling object 10000000208.0 2023-09-23T03:21:36.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.826+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:27 9.27 9:e5993619:::10000000208.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd5e94700 con 0x562cd5eb0000 2023-09-23T03:21:36.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.826+0000 7f21f3a64640 1 -- 172.21.15.154:0/684274785 <== osd.2 v2:172.21.15.154:6822/2884407243 4 ==== osd_op_reply(27 10000000208.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv6 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x562cd5cfcd80 con 0x562cd5eb0000 2023-09-23T03:21:36.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.826+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:28 10.7 10:e5993619:::10000000208.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd5a77880 con 0x562cd5eb1000 2023-09-23T03:21:36.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.827+0000 7f21f3a64640 1 -- 172.21.15.154:0/684274785 <== osd.6 v2:172.21.15.190:6816/1964294001 3 ==== osd_op_reply(28 10000000208.00000000 [getxattr (30) out=30b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x562cd5cfcd80 con 0x562cd5eb1000 2023-09-23T03:21:36.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.827+0000 7f21f4abb5c0 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-09-23T03:21:36.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.827+0000 7f21f4abb5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000208 2023-09-23T03:21:36.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.827+0000 7f21f4abb5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/19 2023-09-23T03:21:36.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.827+0000 7f21f4abb5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=19 2023-09-23T03:21:36.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.827+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:29 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd6778000 con 0x562cd5eb0400 2023-09-23T03:21:36.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.827+0000 7f64993fd640 1 -- 172.21.15.154:0/3049751456 <== osd.7 v2:172.21.15.190:6824/3460136272 19 ==== osd_op_reply(30 10000000000.00000000 [omap-set-vals] v67'38 uv38 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55de26ab4d80 con 0x55de25f7d800 2023-09-23T03:21:36.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.827+0000 7f649a4565c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/14 pointing to 0x0x10000000203 2023-09-23T03:21:36.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.827+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:31 9.1b 9:d8000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55de25f9ae00 con 0x55de25f7dc00 2023-09-23T03:21:36.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.828+0000 7f6498bfc640 1 -- 172.21.15.154:0/3049751456 <== osd.3 v2:172.21.15.154:6808/1133688468 2 ==== osd_op_reply(31 [pgnls-filter start_epoch 67 out=49b] v65'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55de26ab4fc0 con 0x55de25f7dc00 2023-09-23T03:21:36.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.828+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55de25f7c000 msgr2=0x55de25c71b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.828+0000 7f649a4565c0 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55de25f7c000 0x55de25c71b80 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.828+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55de25f7c400 msgr2=0x55de25c71600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.828+0000 7f649a4565c0 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55de25f7c400 0x55de25c71600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.828+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55de25f7c800 msgr2=0x55de25f96000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.828+0000 7f649a4565c0 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55de25f7c800 0x55de25f96000 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.828+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55de25f7dc00 msgr2=0x55de25f98c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.828+0000 7f649a4565c0 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55de25f7dc00 0x55de25f98c00 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.828+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55de26b43c00 msgr2=0x55de25c72100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.828+0000 7f649a4565c0 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55de26b43c00 0x55de25c72100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.828+0000 7f64983fb640 1 -- 172.21.15.154:0/3049751456 reap_dead start 2023-09-23T03:21:36.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.828+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55de25f7d400 msgr2=0x55de25f96580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.828+0000 7f649a4565c0 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55de25f7d400 0x55de25f96580 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.828+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55de25f7d800 msgr2=0x55de25f98680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.828+0000 7f649a4565c0 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55de25f7d800 0x55de25f98680 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.828+0000 7f64983fb640 1 -- 172.21.15.154:0/3049751456 reap_dead start 2023-09-23T03:21:36.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.828+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55de26b43800 msgr2=0x55de25c73700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.828+0000 7f649a4565c0 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55de26b43800 0x55de25c73700 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x55de25cf8600 tx=0x55de26aded20 comp rx=0 tx=0).stop 2023-09-23T03:21:36.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.828+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55de26ad3400 msgr2=0x55de25c72c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.828+0000 7f649a4565c0 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55de26ad3400 0x55de25c72c00 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x55de26ad9650 tx=0x55de269e6000 comp rx=0 tx=0).stop 2023-09-23T03:21:36.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.828+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 shutdown_connections 2023-09-23T03:21:36.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.828+0000 7f649a4565c0 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55de26b43800 0x55de25c73700 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.828+0000 7f649a4565c0 1 --2- 172.21.15.154:0/3049751456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55de26ad3400 0x55de25c72c00 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.828+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 >> 172.21.15.154:0/3049751456 conn(0x55de26ad2c00 msgr2=0x55de25cff800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:36.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.829+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 shutdown_connections 2023-09-23T03:21:36.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.829+0000 7f649a4565c0 1 -- 172.21.15.154:0/3049751456 wait complete. 2023-09-23T03:21:36.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.829+0000 7f649a4565c0 1 -- 172.21.15.154:0/743418532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55de25c06c00 msgr2=0x55de25c70580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.829+0000 7f649a4565c0 1 --2- 172.21.15.154:0/743418532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55de25c06c00 0x55de25c70580 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x55de26a1d470 tx=0x55de269e6240 comp rx=0 tx=0).stop 2023-09-23T03:21:36.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.829+0000 7f649a4565c0 1 -- 172.21.15.154:0/743418532 shutdown_connections 2023-09-23T03:21:36.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.829+0000 7f649a4565c0 1 --2- 172.21.15.154:0/743418532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55de25c07c00 0x55de25c71080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.829+0000 7f649a4565c0 1 --2- 172.21.15.154:0/743418532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55de25c07800 0x55de25c70b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.829+0000 7f649a4565c0 1 --2- 172.21.15.154:0/743418532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55de25c06c00 0x55de25c70580 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.829+0000 7f649a4565c0 1 -- 172.21.15.154:0/743418532 >> 172.21.15.154:0/743418532 conn(0x55de25c07000 msgr2=0x55de25cff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:36.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.829+0000 7f649a4565c0 1 -- 172.21.15.154:0/743418532 shutdown_connections 2023-09-23T03:21:36.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.829+0000 7f649a4565c0 1 -- 172.21.15.154:0/743418532 wait complete. 2023-09-23T03:21:36.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.830+0000 7f18e005b640 1 -- 172.21.15.154:0/1830651065 <== osd.7 v2:172.21.15.190:6824/3460136272 19 ==== osd_op_reply(35 10000000000.00000000 [omap-set-vals] v67'39 uv39 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5576f779afc0 con 0x5576f6ce5800 2023-09-23T03:21:36.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.830+0000 7f18e20b65c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/24 pointing to 0x0x1000000020d 2023-09-23T03:21:36.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.830+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:36 9.35 9:ac000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5576f6cf0e00 con 0x5576f6ce4400 2023-09-23T03:21:36.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.830+0000 7f21f3263640 1 -- 172.21.15.154:0/684274785 <== osd.7 v2:172.21.15.190:6824/3460136272 13 ==== osd_op_reply(29 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x562cd5cfcfc0 con 0x562cd5eb0400 2023-09-23T03:21:36.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.830+0000 7f21f4abb5c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-09-23T03:21:36.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.830+0000 7f21f4abb5c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-09-23T03:21:36.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.830+0000 7f21f4abb5c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/19, will insert to root 2023-09-23T03:21:36.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.830+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:30 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd6778380 con 0x562cd5eb0400 2023-09-23T03:21:36.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.830+0000 7f18e085c640 1 -- 172.21.15.154:0/1830651065 <== osd.1 v2:172.21.15.154:6816/1267710005 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 67 out=49b] v65'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5576f779ab40 con 0x5576f6ce4400 2023-09-23T03:21:36.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.830+0000 7f18dc053640 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:37 9.d 9:b0000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5576f6d26000 con 0x5576f6ce5800 2023-09-23T03:21:36.014 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--worker_n', '0', '--worker_m', '7'] in time 0:00:00.469973, result: 2023-09-23T03:21:36.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.831+0000 7f21f3263640 1 -- 172.21.15.154:0/684274785 <== osd.7 v2:172.21.15.190:6824/3460136272 14 ==== osd_op_reply(30 10000000000.00000000 [omap-get-header out=274b] v0'0 uv39 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x562cd5cfcfc0 con 0x562cd5eb0400 2023-09-23T03:21:36.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.831+0000 7f21f4abb5c0 10 datascan.find_or_create_dirfrag: read_version = 39 2023-09-23T03:21:36.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.831+0000 7f21f4abb5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-09-23T03:21:36.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.831+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:31 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd6778700 con 0x562cd5eb0400 2023-09-23T03:21:36.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.831+0000 7f18e005b640 1 -- 172.21.15.154:0/1830651065 <== osd.7 v2:172.21.15.190:6824/3460136272 20 ==== osd_op_reply(37 [pgnls-filter start_epoch 67 out=49b] v65'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5576f779afc0 con 0x5576f6ce5800 2023-09-23T03:21:36.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.831+0000 7f18db852640 1 -- 172.21.15.154:0/1830651065 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:38 9.2d 9:b4000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x5576f6d26a80 con 0x5576f6ce5400 2023-09-23T03:21:36.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.831+0000 7f21f3263640 1 -- 172.21.15.154:0/684274785 <== osd.7 v2:172.21.15.190:6824/3460136272 15 ==== osd_op_reply(31 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv39 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x562cd5cfcfc0 con 0x562cd5eb0400 2023-09-23T03:21:36.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.831+0000 7f21f4abb5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=19 frag=*, r=0 2023-09-23T03:21:36.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.831+0000 7f21f4abb5c0 20 datascan.read_dentry: 19_head not found in result 2023-09-23T03:21:36.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.831+0000 7f21f4abb5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000208 at 0x0x10000000000/19 with size=6291456 bytes 2023-09-23T03:21:36.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.831+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:32 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd6778a80 con 0x562cd5eb0400 2023-09-23T03:21:36.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.831+0000 7f18e085c640 1 -- 172.21.15.154:0/1830651065 <== osd.2 v2:172.21.15.154:6822/2884407243 2 ==== osd_op_reply(38 [pgnls-filter start_epoch 67 out=49b] v65'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5576f779ab40 con 0x5576f6ce5400 2023-09-23T03:21:36.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.832+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5576f6ce4000 msgr2=0x5576f69ebb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.832+0000 7f18e20b65c0 1 --2- 172.21.15.154:0/1830651065 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5576f6ce4000 0x5576f69ebb80 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.832+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5576f6ce4400 msgr2=0x5576f69eb600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.832+0000 7f18e20b65c0 1 --2- 172.21.15.154:0/1830651065 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5576f6ce4400 0x5576f69eb600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.832+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5576f6ce5400 msgr2=0x5576f6cfeb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.832+0000 7f18e20b65c0 1 --2- 172.21.15.154:0/1830651065 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5576f6ce5400 0x5576f6cfeb00 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.832+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5576f6ce5c00 msgr2=0x5576f6cfe580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.832+0000 7f18e20b65c0 1 --2- 172.21.15.154:0/1830651065 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5576f6ce5c00 0x5576f6cfe580 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.832+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5576f78a5c00 msgr2=0x5576f69ec100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.832+0000 7f18e20b65c0 1 --2- 172.21.15.154:0/1830651065 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5576f78a5c00 0x5576f69ec100 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.832+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x5576f6ce5800 msgr2=0x5576f6cfe000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.832+0000 7f18e20b65c0 1 --2- 172.21.15.154:0/1830651065 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x5576f6ce5800 0x5576f6cfe000 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.832+0000 7f18e085c640 1 -- 172.21.15.154:0/1830651065 reap_dead start 2023-09-23T03:21:36.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.832+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5576f78a5800 msgr2=0x5576f69ed700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.832+0000 7f18e20b65c0 1 --2- 172.21.15.154:0/1830651065 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5576f78a5800 0x5576f69ed700 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x5576f783c8d0 tx=0x5576f77117a0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.832+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576f7849800 msgr2=0x5576f69ec680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.832+0000 7f18e20b65c0 1 --2- 172.21.15.154:0/1830651065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576f7849800 0x5576f69ec680 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x5576f783d530 tx=0x5576f7830360 comp rx=0 tx=0).stop 2023-09-23T03:21:36.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.832+0000 7f18e085c640 1 -- 172.21.15.154:0/1830651065 reap_dead start 2023-09-23T03:21:36.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.832+0000 7f18e085c640 1 -- 172.21.15.154:0/1830651065 reap_dead start 2023-09-23T03:21:36.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.832+0000 7f18e085c640 1 -- 172.21.15.154:0/1830651065 reap_dead start 2023-09-23T03:21:36.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.832+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 shutdown_connections 2023-09-23T03:21:36.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.832+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 >> 172.21.15.154:0/1830651065 conn(0x5576f7848c00 msgr2=0x5576f6a7b000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:36.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.832+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 shutdown_connections 2023-09-23T03:21:36.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.832+0000 7f18e20b65c0 1 -- 172.21.15.154:0/1830651065 wait complete. 2023-09-23T03:21:36.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.833+0000 7f18e20b65c0 1 -- 172.21.15.154:0/3234934361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5576f6981800 msgr2=0x5576f69eab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.833+0000 7f18e20b65c0 1 --2- 172.21.15.154:0/3234934361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5576f6981800 0x5576f69eab00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x5576f77793e0 tx=0x5576f7830420 comp rx=0 tx=0).stop 2023-09-23T03:21:36.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.833+0000 7f18e20b65c0 1 -- 172.21.15.154:0/3234934361 shutdown_connections 2023-09-23T03:21:36.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.833+0000 7f18e20b65c0 1 --2- 172.21.15.154:0/3234934361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5576f6981c00 0x5576f69eb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.833+0000 7f18e20b65c0 1 --2- 172.21.15.154:0/3234934361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5576f6981800 0x5576f69eab00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.833+0000 7f18e20b65c0 1 --2- 172.21.15.154:0/3234934361 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5576f6980c00 0x5576f69ea580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.833+0000 7f18e20b65c0 1 -- 172.21.15.154:0/3234934361 >> 172.21.15.154:0/3234934361 conn(0x5576f6981000 msgr2=0x5576f6a7a800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:36.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.833+0000 7f18e20b65c0 1 -- 172.21.15.154:0/3234934361 shutdown_connections 2023-09-23T03:21:36.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.833+0000 7f18e20b65c0 1 -- 172.21.15.154:0/3234934361 wait complete. 2023-09-23T03:21:36.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.834+0000 7f21f3263640 1 -- 172.21.15.154:0/684274785 <== osd.7 v2:172.21.15.190:6824/3460136272 16 ==== osd_op_reply(32 10000000000.00000000 [omap-set-vals] v67'40 uv40 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562cd5cfcfc0 con 0x562cd5eb0400 2023-09-23T03:21:36.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.834+0000 7f21f4abb5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/19 pointing to 0x0x10000000208 2023-09-23T03:21:36.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.834+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:33 9.17 9:e8000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd6778e00 con 0x562cd5eb0400 2023-09-23T03:21:36.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.835+0000 7f21f3263640 1 -- 172.21.15.154:0/684274785 <== osd.7 v2:172.21.15.190:6824/3460136272 17 ==== osd_op_reply(33 [pgnls-filter start_epoch 67 out=49b] v65'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562cd5cfcfc0 con 0x562cd5eb0400 2023-09-23T03:21:36.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.835+0000 7f21ee259640 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:34 9.37 9:ec000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd5e95500 con 0x562cd5e88000 2023-09-23T03:21:36.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.835+0000 7f21f2a62640 1 -- 172.21.15.154:0/684274785 <== osd.4 v2:172.21.15.190:6808/717302641 2 ==== osd_op_reply(34 [pgnls-filter start_epoch 67 out=49b] v65'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562cd5cfcb40 con 0x562cd5e88000 2023-09-23T03:21:36.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.835+0000 7f21eea5a640 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:35 9.f 9:f0000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd5e94000 con 0x562cd5eb0800 2023-09-23T03:21:36.024 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--worker_n', '5', '--worker_m', '7'] in time 0:00:00.478182, result: 2023-09-23T03:21:36.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.836+0000 7f21f2a62640 1 -- 172.21.15.154:0/684274785 <== osd.5 v2:172.21.15.190:6800/882536885 5 ==== osd_op_reply(35 [pgnls-filter start_epoch 67 out=49b] v65'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562cd5cfcb40 con 0x562cd5eb0800 2023-09-23T03:21:36.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.836+0000 7f21ee259640 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:36 9.2f 9:f4000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd6778000 con 0x562cd5e88c00 2023-09-23T03:21:36.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.836+0000 7f21f2a62640 1 -- 172.21.15.154:0/684274785 <== osd.3 v2:172.21.15.154:6808/1133688468 3 ==== osd_op_reply(36 [pgnls-filter start_epoch 67 out=49b] v65'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562cd5cfcb40 con 0x562cd5e88c00 2023-09-23T03:21:36.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.836+0000 7f21eea5a640 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:37 9.1f 9:f8000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd5e95880 con 0x562cd5e88800 2023-09-23T03:21:36.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.837+0000 7f21f3263640 1 -- 172.21.15.154:0/684274785 <== osd.1 v2:172.21.15.154:6816/1267710005 2 ==== osd_op_reply(37 [pgnls-filter start_epoch 67 out=49b] v65'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562cd5cfcfc0 con 0x562cd5e88800 2023-09-23T03:21:36.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.837+0000 7f21ee259640 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:38 9.3f 9:fc000000::::head [pgnls-filter start_epoch 67 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd5e95180 con 0x562cd5eb0000 2023-09-23T03:21:36.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.837+0000 7f21f3a64640 1 -- 172.21.15.154:0/684274785 <== osd.2 v2:172.21.15.154:6822/2884407243 5 ==== osd_op_reply(38 [pgnls-filter start_epoch 67 out=81b] v67'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562cd5cfcd80 con 0x562cd5eb0000 2023-09-23T03:21:36.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.837+0000 7f21f4abb5c0 20 datascan.forall_objects: OSD matched oid 100000001fa.00000000 2023-09-23T03:21:36.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.837+0000 7f21f4abb5c0 10 datascan.operator(): handling object 100000001fa.0 2023-09-23T03:21:36.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.837+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:39 9.3f 9:feb1c803:::100000001fa.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd6779180 con 0x562cd5eb0000 2023-09-23T03:21:36.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.838+0000 7f21f3a64640 1 -- 172.21.15.154:0/684274785 <== osd.2 v2:172.21.15.154:6822/2884407243 6 ==== osd_op_reply(39 100000001fa.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x562cd5cfcd80 con 0x562cd5eb0000 2023-09-23T03:21:36.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.838+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:40 10.1f 10:feb1c803:::100000001fa.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd6779880 con 0x562cd5eb0400 2023-09-23T03:21:36.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.838+0000 7f21f3263640 1 -- 172.21.15.154:0/684274785 <== osd.7 v2:172.21.15.190:6824/3460136272 18 ==== osd_op_reply(40 100000001fa.00000000 [getxattr (30) out=30b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x562cd5cfcfc0 con 0x562cd5eb0400 2023-09-23T03:21:36.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.838+0000 7f21f4abb5c0 20 datascan.operator(): loaded layout from xattr: pi: 10 os: 4194304 sc: 1 su: 4194304 2023-09-23T03:21:36.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.838+0000 7f21f4abb5c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001fa 2023-09-23T03:21:36.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.838+0000 7f21f4abb5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/5 2023-09-23T03:21:36.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.838+0000 7f21f4abb5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=5 2023-09-23T03:21:36.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.838+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:41 8.10 8:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd6779c00 con 0x562cd5eb0400 2023-09-23T03:21:36.028 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--worker_n', '4', '--worker_m', '7'] in time 0:00:00.482359, result: 2023-09-23T03:21:36.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.839+0000 7f21f3263640 1 -- 172.21.15.154:0/684274785 <== osd.7 v2:172.21.15.190:6824/3460136272 19 ==== osd_op_reply(41 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x562cd5cfcfc0 con 0x562cd5eb0400 2023-09-23T03:21:36.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.839+0000 7f21f4abb5c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-09-23T03:21:36.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.839+0000 7f21f4abb5c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-09-23T03:21:36.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.839+0000 7f21f4abb5c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/5, will insert to root 2023-09-23T03:21:36.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.839+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:42 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd6778380 con 0x562cd5eb0400 2023-09-23T03:21:36.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.840+0000 7f21f3263640 1 -- 172.21.15.154:0/684274785 <== osd.7 v2:172.21.15.190:6824/3460136272 20 ==== osd_op_reply(42 10000000000.00000000 [omap-get-header out=274b] v0'0 uv40 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x562cd5cfcfc0 con 0x562cd5eb0400 2023-09-23T03:21:36.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.840+0000 7f21f4abb5c0 10 datascan.find_or_create_dirfrag: read_version = 40 2023-09-23T03:21:36.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.840+0000 7f21f4abb5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-09-23T03:21:36.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.840+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:43 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd6778700 con 0x562cd5eb0400 2023-09-23T03:21:36.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.840+0000 7f21f3263640 1 -- 172.21.15.154:0/684274785 <== osd.7 v2:172.21.15.190:6824/3460136272 21 ==== osd_op_reply(43 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv40 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x562cd5cfcfc0 con 0x562cd5eb0400 2023-09-23T03:21:36.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.840+0000 7f21f4abb5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=5 frag=*, r=0 2023-09-23T03:21:36.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.840+0000 7f21f4abb5c0 20 datascan.read_dentry: 5_head not found in result 2023-09-23T03:21:36.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.840+0000 7f21f4abb5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fa at 0x0x10000000000/5 with size=6291456 bytes 2023-09-23T03:21:36.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.840+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:44 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x562cd6778a80 con 0x562cd5eb0400 2023-09-23T03:21:36.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.844+0000 7f21f3263640 1 -- 172.21.15.154:0/684274785 <== osd.7 v2:172.21.15.190:6824/3460136272 22 ==== osd_op_reply(44 10000000000.00000000 [omap-set-vals] v67'41 uv41 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562cd5cfcfc0 con 0x562cd5eb0400 2023-09-23T03:21:36.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.844+0000 7f21f4abb5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/5 pointing to 0x0x100000001fa 2023-09-23T03:21:36.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.844+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x562cd5e88400 msgr2=0x562cd5a87b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.844+0000 7f21f4abb5c0 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x562cd5e88400 0x562cd5a87b80 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.844+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x562cd5e88800 msgr2=0x562cd5a87600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.844+0000 7f21f4abb5c0 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x562cd5e88800 0x562cd5a87600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.844+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x562cd5eb0000 msgr2=0x562cd5eaa680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.844+0000 7f21f4abb5c0 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x562cd5eb0000 0x562cd5eaa680 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.844+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x562cd5e88c00 msgr2=0x562cd5ea8000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.844+0000 7f21f4abb5c0 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x562cd5e88c00 0x562cd5ea8000 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.844+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x562cd5e88000 msgr2=0x562cd5a88100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.844+0000 7f21f4abb5c0 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x562cd5e88000 0x562cd5a88100 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.844+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x562cd5eb0800 msgr2=0x562cd5ea8b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.844+0000 7f21f4abb5c0 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x562cd5eb0800 0x562cd5ea8b00 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.844+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x562cd5eb1000 msgr2=0x562cd5ea9080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.844+0000 7f21f4abb5c0 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x562cd5eb1000 0x562cd5ea9080 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.844+0000 7f21f3263640 1 -- 172.21.15.154:0/684274785 reap_dead start 2023-09-23T03:21:36.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.844+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x562cd5eb0400 msgr2=0x562cd5ea8580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.844+0000 7f21f4abb5c0 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x562cd5eb0400 0x562cd5ea8580 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.844+0000 7f21f3263640 1 -- 172.21.15.154:0/684274785 reap_dead start 2023-09-23T03:21:36.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.844+0000 7f21f3263640 1 -- 172.21.15.154:0/684274785 reap_dead start 2023-09-23T03:21:36.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.844+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x562cd5d1dc00 msgr2=0x562cd5a89700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.844+0000 7f21f4abb5c0 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x562cd5d1dc00 0x562cd5a89700 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x562cd5d227e0 tx=0x562cd5d29f20 comp rx=0 tx=0).stop 2023-09-23T03:21:36.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.844+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562cd5d1d400 msgr2=0x562cd5a88c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.844+0000 7f21f4abb5c0 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562cd5d1d400 0x562cd5a88c00 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x562cd5d23590 tx=0x562cd5d06120 comp rx=0 tx=0).stop 2023-09-23T03:21:36.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.844+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 shutdown_connections 2023-09-23T03:21:36.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.844+0000 7f21f4abb5c0 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x562cd5d1dc00 0x562cd5a89700 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.844+0000 7f21f4abb5c0 1 --2- 172.21.15.154:0/684274785 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562cd5d1d400 0x562cd5a88c00 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.844+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 >> 172.21.15.154:0/684274785 conn(0x562cd5d1cc00 msgr2=0x562cd6797000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:36.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.844+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 shutdown_connections 2023-09-23T03:21:36.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.844+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/684274785 wait complete. 2023-09-23T03:21:36.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.845+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/2977390932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562cd5a1d800 msgr2=0x562cd5a86b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.845+0000 7f21f4abb5c0 1 --2- 172.21.15.154:0/2977390932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562cd5a1d800 0x562cd5a86b00 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x562cd682d350 tx=0x562cd5d064e0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.845+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/2977390932 shutdown_connections 2023-09-23T03:21:36.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.845+0000 7f21f4abb5c0 1 --2- 172.21.15.154:0/2977390932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x562cd5a1dc00 0x562cd5a87080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.845+0000 7f21f4abb5c0 1 --2- 172.21.15.154:0/2977390932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562cd5a1d800 0x562cd5a86b00 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.845+0000 7f21f4abb5c0 1 --2- 172.21.15.154:0/2977390932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x562cd5a1cc00 0x562cd5a86580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.845+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/2977390932 >> 172.21.15.154:0/2977390932 conn(0x562cd5a1d000 msgr2=0x562cd6796800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:36.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.845+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/2977390932 shutdown_connections 2023-09-23T03:21:36.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:35.845+0000 7f21f4abb5c0 1 -- 172.21.15.154:0/2977390932 wait complete. 2023-09-23T03:21:36.035 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--worker_n', '6', '--worker_m', '7'] in time 0:00:00.488898, result: 2023-09-23T03:21:36.036 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2023-09-23T03:21:36.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.145+0000 7f5f7db6e5c0 1 -- start start 2023-09-23T03:21:36.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.145+0000 7f5f7db6e5c0 1 Processor -- start 2023-09-23T03:21:36.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.146+0000 7f5f7db6e5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55649bb10000 0x55649ba9f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:36.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.146+0000 7f5f7db6e5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55649bb11000 0x55649ba9eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:36.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.146+0000 7f5f7db6e5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55649bb11400 0x55649ba9fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:36.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.146+0000 7f5f7db6e5c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55649ba7e140 con 0x55649bb10000 2023-09-23T03:21:36.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.146+0000 7f5f7db6e5c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55649ba7f180 con 0x55649bb11400 2023-09-23T03:21:36.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.146+0000 7f5f7db6e5c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55649bb02dc0 con 0x55649bb11000 2023-09-23T03:21:36.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.146+0000 7f5f7c316640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55649bb10000 0x55649ba9f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:36.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.146+0000 7f5f7c316640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55649bb10000 0x55649ba9f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60872/0 (socket says 172.21.15.154:60872) 2023-09-23T03:21:36.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.146+0000 7f5f7c316640 1 -- 172.21.15.154:0/354970918 learned_addr learned my addr 172.21.15.154:0/354970918 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:36.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.146+0000 7f5f7bb15640 1 --2- 172.21.15.154:0/354970918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55649bb11000 0x55649ba9eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:36.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.146+0000 7f5f7c316640 1 -- 172.21.15.154:0/354970918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55649bb11000 msgr2=0x55649ba9eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:36.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.146+0000 7f5f7cb17640 1 --2- 172.21.15.154:0/354970918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55649bb11400 0x55649ba9fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:36.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.146+0000 7f5f7c316640 1 --2- 172.21.15.154:0/354970918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55649bb11000 0x55649ba9eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.146+0000 7f5f7c316640 1 -- 172.21.15.154:0/354970918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55649bb11400 msgr2=0x55649ba9fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:36.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.146+0000 7f5f7c316640 1 --2- 172.21.15.154:0/354970918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55649bb11400 0x55649ba9fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.146+0000 7f5f7c316640 1 -- 172.21.15.154:0/354970918 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55649ba68ea0 con 0x55649bb10000 2023-09-23T03:21:36.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.146+0000 7f5f7bb15640 1 --2- 172.21.15.154:0/354970918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55649bb11000 0x55649ba9eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:36.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.147+0000 7f5f7c316640 1 --2- 172.21.15.154:0/354970918 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55649bb10000 0x55649ba9f080 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x55649c840900 tx=0x55649bd1e120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b06ea3d39dd3998 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:36.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.147+0000 7f5f79b11640 1 -- 172.21.15.154:0/354970918 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55649ba7cf20 con 0x55649bb10000 2023-09-23T03:21:36.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.147+0000 7f5f79b11640 1 -- 172.21.15.154:0/354970918 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55649ba5aa80 con 0x55649bb10000 2023-09-23T03:21:36.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.147+0000 7f5f79b11640 1 -- 172.21.15.154:0/354970918 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55649ba7d080 con 0x55649bb10000 2023-09-23T03:21:36.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.147+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/354970918 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55649ba68000 con 0x55649bb10000 2023-09-23T03:21:36.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.147+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/354970918 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55649ba689c0 con 0x55649bb10000 2023-09-23T03:21:36.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.147+0000 7f5f7db6e5c0 4 waiting for MDS map... 2023-09-23T03:21:36.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.147+0000 7f5f79b11640 1 -- 172.21.15.154:0/354970918 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x55649baf4c40 con 0x55649bb10000 2023-09-23T03:21:36.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.147+0000 7f5f79b11640 1 -- 172.21.15.154:0/354970918 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x55649bd28000 con 0x55649bb10000 2023-09-23T03:21:36.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.147+0000 7f5f7db6e5c0 4 Got MDS map 47 2023-09-23T03:21:36.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.148+0000 7f5f7db6e5c0 4 datascan.main: Using metadata pool output 2023-09-23T03:21:36.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.148+0000 7f5f7db6e5c0 4 datascan.main: connecting to RADOS... 2023-09-23T03:21:36.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.148+0000 7f5f7db6e5c0 1 Processor -- start 2023-09-23T03:21:36.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.148+0000 7f5f7db6e5c0 1 -- start start 2023-09-23T03:21:36.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.149+0000 7f5f7db6e5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55649bb11c00 0x55649ba9f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:36.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.149+0000 7f5f7db6e5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55649bd84000 0x55649baa0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:36.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.149+0000 7f5f7db6e5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55649bd84400 0x55649baa0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:36.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.149+0000 7f5f7db6e5c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55649ba7eb40 con 0x55649bb11c00 2023-09-23T03:21:36.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.149+0000 7f5f7db6e5c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55649ba7ea00 con 0x55649bd84000 2023-09-23T03:21:36.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.149+0000 7f5f7db6e5c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55649ba7e000 con 0x55649bd84400 2023-09-23T03:21:36.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.149+0000 7f5f7cb17640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55649bb11c00 0x55649ba9f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:36.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.149+0000 7f5f7cb17640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55649bb11c00 0x55649ba9f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60874/0 (socket says 172.21.15.154:60874) 2023-09-23T03:21:36.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.149+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2382958245 learned_addr learned my addr 172.21.15.154:0/2382958245 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:36.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.150+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2382958245 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55649bd84400 msgr2=0x55649baa0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:36.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.150+0000 7f5f7cb17640 1 --2- 172.21.15.154:0/2382958245 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55649bd84400 0x55649baa0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.150+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2382958245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55649bd84000 msgr2=0x55649baa0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:36.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.150+0000 7f5f7cb17640 1 --2- 172.21.15.154:0/2382958245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55649bd84000 0x55649baa0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.150+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2382958245 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55649ba68ea0 con 0x55649bb11c00 2023-09-23T03:21:36.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.150+0000 7f5f7cb17640 1 --2- 172.21.15.154:0/2382958245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55649bb11c00 0x55649ba9f600 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x55649bd325a0 tx=0x55649bd38c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2170b08ed4ec9a4a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:36.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.150+0000 7f5f77b0d640 1 -- 172.21.15.154:0/2382958245 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55649ba7cf20 con 0x55649bb11c00 2023-09-23T03:21:36.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.150+0000 7f5f77b0d640 1 -- 172.21.15.154:0/2382958245 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55649bd22180 con 0x55649bb11c00 2023-09-23T03:21:36.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.150+0000 7f5f77b0d640 1 -- 172.21.15.154:0/2382958245 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55649ba7d080 con 0x55649bb11c00 2023-09-23T03:21:36.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.151+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2382958245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55649bb11c00 msgr2=0x55649ba9f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.151+0000 7f5f7db6e5c0 1 --2- 172.21.15.154:0/2382958245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55649bb11c00 0x55649ba9f600 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x55649bd325a0 tx=0x55649bd38c00 comp rx=0 tx=0).stop 2023-09-23T03:21:36.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.151+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2382958245 shutdown_connections 2023-09-23T03:21:36.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.151+0000 7f5f7db6e5c0 1 --2- 172.21.15.154:0/2382958245 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55649bd84400 0x55649baa0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.151+0000 7f5f7db6e5c0 1 --2- 172.21.15.154:0/2382958245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55649bd84000 0x55649baa0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.151+0000 7f5f7db6e5c0 1 --2- 172.21.15.154:0/2382958245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55649bb11c00 0x55649ba9f600 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.151+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2382958245 >> 172.21.15.154:0/2382958245 conn(0x55649bb11800 msgr2=0x55649c7bf000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:36.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.151+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2382958245 shutdown_connections 2023-09-23T03:21:36.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.151+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2382958245 wait complete. 2023-09-23T03:21:36.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.152+0000 7f5f7db6e5c0 1 Processor -- start 2023-09-23T03:21:36.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.152+0000 7f5f7db6e5c0 1 -- start start 2023-09-23T03:21:36.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.153+0000 7f5f7db6e5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55649bb11800 0x55649baa1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:36.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.153+0000 7f5f7db6e5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55649bd85000 0x55649baa1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:36.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.153+0000 7f5f7db6e5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55649bd84c00 0x55649baa0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:36.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.153+0000 7f5f7db6e5c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55649bb02c80 con 0x55649bd84c00 2023-09-23T03:21:36.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.153+0000 7f5f7db6e5c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55649bb02b40 con 0x55649bb11800 2023-09-23T03:21:36.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.153+0000 7f5f7db6e5c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55649bb02a00 con 0x55649bd85000 2023-09-23T03:21:36.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.153+0000 7f5f7bb15640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55649bd84c00 0x55649baa0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:36.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.153+0000 7f5f7bb15640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55649bd84c00 0x55649baa0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60888/0 (socket says 172.21.15.154:60888) 2023-09-23T03:21:36.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.153+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 learned_addr learned my addr 172.21.15.154:0/2288557327 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:36.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.153+0000 7f5f7c316640 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55649bd85000 0x55649baa1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:36.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.153+0000 7f5f7cb17640 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55649bb11800 0x55649baa1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:36.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.153+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55649bd85000 msgr2=0x55649baa1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.153+0000 7f5f7bb15640 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55649bd85000 0x55649baa1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.153+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55649bb11800 msgr2=0x55649baa1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.153+0000 7f5f7bb15640 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55649bb11800 0x55649baa1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.153+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55649ba69520 con 0x55649bd84c00 2023-09-23T03:21:36.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.154+0000 7f5f7bb15640 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55649bd84c00 0x55649baa0c00 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x55649bd33800 tx=0x55649bd38de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:36.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.154+0000 7f5f7630a640 1 -- 172.21.15.154:0/2288557327 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55649ba7cf20 con 0x55649bd84c00 2023-09-23T03:21:36.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.154+0000 7f5f7630a640 1 -- 172.21.15.154:0/2288557327 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55649bd23680 con 0x55649bd84c00 2023-09-23T03:21:36.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.154+0000 7f5f7630a640 1 -- 172.21.15.154:0/2288557327 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55649ba7d080 con 0x55649bd84c00 2023-09-23T03:21:36.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.154+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55649ba696c0 con 0x55649bd84c00 2023-09-23T03:21:36.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.154+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55649ba69860 con 0x55649bd84c00 2023-09-23T03:21:36.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.154+0000 7f5f7db6e5c0 4 datascan.init: resolving metadata pool 8 2023-09-23T03:21:36.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.156+0000 7f5f7630a640 1 -- 172.21.15.154:0/2288557327 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x55649ba66b00 con 0x55649bd84c00 2023-09-23T03:21:36.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.156+0000 7f5f7630a640 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55649bd9ac00 0x55649be88000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:36.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.156+0000 7f5f7630a640 1 -- 172.21.15.154:0/2288557327 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x55649baf4a80 con 0x55649bd84c00 2023-09-23T03:21:36.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.157+0000 7f5f7db6e5c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-09-23T03:21:36.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.157+0000 7f5f7db6e5c0 4 datascan.main: resolving metadata pool 8 2023-09-23T03:21:36.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.157+0000 7f5f7db6e5c0 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55649bd84800 0x55649baa0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:36.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.157+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:1 8.0 8:00000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8f880 con 0x55649bd84800 2023-09-23T03:21:36.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.157+0000 7f5f7cb17640 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55649bd9ac00 0x55649be88000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:36.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.157+0000 7f5f7c316640 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55649bd84800 0x55649baa0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:36.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.157+0000 7f5f7cb17640 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55649bd9ac00 0x55649be88000 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x55649bd325a0 tx=0x55649c7c55c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:36.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.158+0000 7f5f7c316640 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55649bd84800 0x55649baa0680 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:36.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.158+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(1 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12b40 con 0x55649bd84800 2023-09-23T03:21:36.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.158+0000 7f5f7c316640 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55649ba34c00 0x55649be88580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:36.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.158+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:2 8.10 8:08000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8f880 con 0x55649ba34c00 2023-09-23T03:21:36.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.159+0000 7f5f7cb17640 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55649ba34c00 0x55649be88580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:36.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.159+0000 7f5f7cb17640 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55649ba34c00 0x55649be88580 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:36.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.160+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 1 ==== osd_op_reply(2 [pgnls start_epoch 67 out=81b] v67'41 uv41 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.160+0000 7f5f7cb17640 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55649bd85c00 0x55649ba9e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:36.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.160+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:3 8.8 8:10000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8f880 con 0x55649bd85c00 2023-09-23T03:21:36.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.161+0000 7f5f7c316640 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55649bd85c00 0x55649ba9e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:36.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.162+0000 7f5f7c316640 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55649bd85c00 0x55649ba9e580 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:36.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.163+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 <== osd.5 v2:172.21.15.190:6800/882536885 1 ==== osd_op_reply(3 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12b40 con 0x55649bd85c00 2023-09-23T03:21:36.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.163+0000 7f5f7c316640 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55649bd9b000 0x55649be88b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:36.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.163+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:4 8.18 8:18000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8f880 con 0x55649bd9b000 2023-09-23T03:21:36.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.163+0000 7f5f7bb15640 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55649bd9b000 0x55649be88b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:36.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.164+0000 7f5f7bb15640 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55649bd9b000 0x55649be88b00 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:36.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.165+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 <== osd.4 v2:172.21.15.190:6808/717302641 1 ==== osd_op_reply(4 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12fc0 con 0x55649bd9b000 2023-09-23T03:21:36.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.165+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:5 8.4 8:20000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8f880 con 0x55649bd85c00 2023-09-23T03:21:36.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.166+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 <== osd.5 v2:172.21.15.190:6800/882536885 2 ==== osd_op_reply(5 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12b40 con 0x55649bd85c00 2023-09-23T03:21:36.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.166+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:6 8.14 8:28000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8f880 con 0x55649bd9b000 2023-09-23T03:21:36.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.167+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 <== osd.4 v2:172.21.15.190:6808/717302641 2 ==== osd_op_reply(6 [pgnls start_epoch 67 out=125b] v67'107 uv107 ondisk = 1) v8 ==== 144+0+125 (crc 0 0 0) 0x55649bd12fc0 con 0x55649bd9b000 2023-09-23T03:21:36.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.167+0000 7f5f7bb15640 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55649bea4000 0x55649be89600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:36.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.167+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:7 8.c 8:30000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8f880 con 0x55649bea4000 2023-09-23T03:21:36.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.167+0000 7f5f7cb17640 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55649bea4000 0x55649be89600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:36.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.168+0000 7f5f7cb17640 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55649bea4000 0x55649be89600 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:36.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.168+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.0 v2:172.21.15.154:6800/2920938138 1 ==== osd_op_reply(7 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12d80 con 0x55649bea4000 2023-09-23T03:21:36.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.168+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:8 8.1c 8:38000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8f880 con 0x55649bd85c00 2023-09-23T03:21:36.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.169+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 <== osd.5 v2:172.21.15.190:6800/882536885 3 ==== osd_op_reply(8 [pgnls start_epoch 67 out=49b] v66'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12b40 con 0x55649bd85c00 2023-09-23T03:21:36.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.169+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:9 8.2 8:40000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8f880 con 0x55649bd9b000 2023-09-23T03:21:36.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.170+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 <== osd.4 v2:172.21.15.190:6808/717302641 3 ==== osd_op_reply(9 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12fc0 con 0x55649bd9b000 2023-09-23T03:21:36.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.170+0000 7f5f7bb15640 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55649bea4400 0x55649be89b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:36.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.170+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:10 8.12 8:48000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8f880 con 0x55649bea4400 2023-09-23T03:21:36.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.171+0000 7f5f7c316640 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55649bea4400 0x55649be89b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:36.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.171+0000 7f5f7c316640 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55649bea4400 0x55649be89b80 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:36.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.172+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== osd_op_reply(10 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12b40 con 0x55649bea4400 2023-09-23T03:21:36.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.172+0000 7f5f7c316640 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55649bd9b400 0x55649be89080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:36.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.172+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:11 8.a 8:50000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8f880 con 0x55649bd9b400 2023-09-23T03:21:36.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.172+0000 7f5f7bb15640 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55649bd9b400 0x55649be89080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:36.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.173+0000 7f5f7bb15640 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55649bd9b400 0x55649be89080 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:36.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.173+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 <== osd.1 v2:172.21.15.154:6816/1267710005 1 ==== osd_op_reply(11 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12fc0 con 0x55649bd9b400 2023-09-23T03:21:36.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.173+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:12 8.1a 8:58000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8f880 con 0x55649bea4000 2023-09-23T03:21:36.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.174+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.0 v2:172.21.15.154:6800/2920938138 2 ==== osd_op_reply(12 [pgnls start_epoch 67 out=49b] v67'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12d80 con 0x55649bea4000 2023-09-23T03:21:36.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.174+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:13 8.6 8:60000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8f880 con 0x55649bea4000 2023-09-23T03:21:36.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.174+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.0 v2:172.21.15.154:6800/2920938138 3 ==== osd_op_reply(13 [pgnls start_epoch 67 out=49b] v66'24 uv24 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12d80 con 0x55649bea4000 2023-09-23T03:21:36.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.174+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:14 8.16 8:68000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8f880 con 0x55649bd84800 2023-09-23T03:21:36.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.174+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 <== osd.3 v2:172.21.15.154:6808/1133688468 2 ==== osd_op_reply(14 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12b40 con 0x55649bd84800 2023-09-23T03:21:36.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.174+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:15 8.e 8:70000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8f880 con 0x55649bea4000 2023-09-23T03:21:36.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.175+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.0 v2:172.21.15.154:6800/2920938138 4 ==== osd_op_reply(15 [pgnls start_epoch 67 out=49b] v66'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12d80 con 0x55649bea4000 2023-09-23T03:21:36.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.175+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:16 8.1e 8:78000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8f880 con 0x55649bd84800 2023-09-23T03:21:36.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.175+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 <== osd.3 v2:172.21.15.154:6808/1133688468 3 ==== osd_op_reply(16 [pgnls start_epoch 67 out=49b] v67'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12b40 con 0x55649bd84800 2023-09-23T03:21:36.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.175+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:17 8.1 8:80000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8f880 con 0x55649bd9b400 2023-09-23T03:21:36.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.175+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 <== osd.1 v2:172.21.15.154:6816/1267710005 2 ==== osd_op_reply(17 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12fc0 con 0x55649bd9b400 2023-09-23T03:21:36.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.175+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:18 8.11 8:88000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8f880 con 0x55649bd85c00 2023-09-23T03:21:36.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.176+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 <== osd.5 v2:172.21.15.190:6800/882536885 4 ==== osd_op_reply(18 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12b40 con 0x55649bd85c00 2023-09-23T03:21:36.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.176+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:19 8.9 8:90000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8f880 con 0x55649bea4400 2023-09-23T03:21:36.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.177+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 <== osd.6 v2:172.21.15.190:6816/1964294001 2 ==== osd_op_reply(19 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12b40 con 0x55649bea4400 2023-09-23T03:21:36.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.177+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:20 8.19 8:98000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8f880 con 0x55649bd85c00 2023-09-23T03:21:36.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.178+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 <== osd.5 v2:172.21.15.190:6800/882536885 5 ==== osd_op_reply(20 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12b40 con 0x55649bd85c00 2023-09-23T03:21:36.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.178+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:21 8.5 8:a0000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8f880 con 0x55649bea4400 2023-09-23T03:21:36.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.179+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 <== osd.6 v2:172.21.15.190:6816/1964294001 3 ==== osd_op_reply(21 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12b40 con 0x55649bea4400 2023-09-23T03:21:36.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.179+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:22 8.15 8:a8000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8f880 con 0x55649ba34c00 2023-09-23T03:21:36.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.181+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 2 ==== osd_op_reply(22 [pgnls start_epoch 67 out=73b] v66'6 uv6 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.181+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:23 8.d 8:b0000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8f880 con 0x55649bd85c00 2023-09-23T03:21:36.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.182+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 <== osd.5 v2:172.21.15.190:6800/882536885 6 ==== osd_op_reply(23 [pgnls start_epoch 67 out=74b] v67'3 uv3 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55649bd12b40 con 0x55649bd85c00 2023-09-23T03:21:36.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.182+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:24 8.1d 8:b8000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8f880 con 0x55649bea4000 2023-09-23T03:21:36.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.182+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.0 v2:172.21.15.154:6800/2920938138 5 ==== osd_op_reply(24 [pgnls start_epoch 67 out=49b] v66'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12d80 con 0x55649bea4000 2023-09-23T03:21:36.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.182+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:25 8.3 8:c0000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8f880 con 0x55649bd9b400 2023-09-23T03:21:36.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.183+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 <== osd.1 v2:172.21.15.154:6816/1267710005 3 ==== osd_op_reply(25 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12fc0 con 0x55649bd9b400 2023-09-23T03:21:36.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.183+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:26 8.13 8:c8000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8f880 con 0x55649bd84800 2023-09-23T03:21:36.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.183+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 <== osd.3 v2:172.21.15.154:6808/1133688468 4 ==== osd_op_reply(26 [pgnls start_epoch 67 out=98b] v67'7 uv7 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55649bd12b40 con 0x55649bd84800 2023-09-23T03:21:36.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.183+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:27 8.b 8:d0000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8f880 con 0x55649bd84800 2023-09-23T03:21:36.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.183+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 <== osd.3 v2:172.21.15.154:6808/1133688468 5 ==== osd_op_reply(27 [pgnls start_epoch 67 out=76b] v67'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55649bd12b40 con 0x55649bd84800 2023-09-23T03:21:36.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.184+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:28 8.1b 8:d8000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8f880 con 0x55649bd9b000 2023-09-23T03:21:36.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.184+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 <== osd.4 v2:172.21.15.190:6808/717302641 4 ==== osd_op_reply(28 [pgnls start_epoch 67 out=49b] v67'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12fc0 con 0x55649bd9b000 2023-09-23T03:21:36.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.184+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:29 8.7 8:e0000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8f880 con 0x55649bd9b400 2023-09-23T03:21:36.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.185+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 <== osd.1 v2:172.21.15.154:6816/1267710005 4 ==== osd_op_reply(29 [pgnls start_epoch 67 out=49b] v66'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12fc0 con 0x55649bd9b400 2023-09-23T03:21:36.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.185+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:30 8.17 8:e8000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8f880 con 0x55649ba34c00 2023-09-23T03:21:36.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.186+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 3 ==== osd_op_reply(30 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.186+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:31 8.f 8:f0000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8f880 con 0x55649bd84800 2023-09-23T03:21:36.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.186+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 <== osd.3 v2:172.21.15.154:6808/1133688468 6 ==== osd_op_reply(31 [pgnls start_epoch 67 out=73b] v67'4 uv4 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55649bd12b40 con 0x55649bd84800 2023-09-23T03:21:36.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.186+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:32 8.1f 8:f8000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8f880 con 0x55649bea4000 2023-09-23T03:21:36.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.187+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.0 v2:172.21.15.154:6800/2920938138 6 ==== osd_op_reply(32 [pgnls start_epoch 67 out=101b] v67'12 uv12 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55649bd12d80 con 0x55649bea4000 2023-09-23T03:21:36.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.187+0000 7f5f7db6e5c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2023-09-23T03:21:36.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.187+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:33 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8fc00 con 0x55649ba34c00 2023-09-23T03:21:36.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 4 ==== osd_op_reply(33 10000000000.00000000 [omap-get-vals out=12470b] v0'0 uv41 ondisk = 0) v8 ==== 164+0+12470 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2023-09-23T03:21:36.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-09-23T03:21:36.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 10 datascan.scan_links: step 1: handling object 200.00000000 2023-09-23T03:21:36.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-09-23T03:21:36.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 10 datascan.scan_links: step 1: handling object 200.00000001 2023-09-23T03:21:36.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-09-23T03:21:36.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 10 datascan.scan_links: step 1: handling object 500.00000000 2023-09-23T03:21:36.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-09-23T03:21:36.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2023-09-23T03:21:36.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-09-23T03:21:36.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2023-09-23T03:21:36.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-09-23T03:21:36.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 10 datascan.scan_links: step 1: handling object 100.00000000 2023-09-23T03:21:36.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:34 8.13 8:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8f880 con 0x55649bd84800 2023-09-23T03:21:36.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 <== osd.3 v2:172.21.15.154:6808/1133688468 7 ==== osd_op_reply(34 100.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55649bd12b40 con 0x55649bd84800 2023-09-23T03:21:36.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2023-09-23T03:21:36.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-09-23T03:21:36.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 10 datascan.scan_links: step 1: handling object 400.00000000 2023-09-23T03:21:36.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-09-23T03:21:36.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2023-09-23T03:21:36.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-09-23T03:21:36.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 10 datascan.scan_links: step 1: handling object 1.00000000 2023-09-23T03:21:36.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.189+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:35 8.1f 8:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8fc00 con 0x55649bea4000 2023-09-23T03:21:36.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.190+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.0 v2:172.21.15.154:6800/2920938138 7 ==== osd_op_reply(35 1.00000000 [omap-get-vals out=508b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+508 (crc 0 0 0) 0x55649bd12d80 con 0x55649bea4000 2023-09-23T03:21:36.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.190+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.190+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:36 8.0 8:00000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba000 con 0x55649bd84800 2023-09-23T03:21:36.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.191+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 <== osd.3 v2:172.21.15.154:6808/1133688468 8 ==== osd_op_reply(36 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12b40 con 0x55649bd84800 2023-09-23T03:21:36.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.191+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:37 8.10 8:08000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba000 con 0x55649ba34c00 2023-09-23T03:21:36.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.192+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 5 ==== osd_op_reply(37 [pgnls start_epoch 67 out=81b] v67'41 uv41 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.192+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:38 8.8 8:10000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba000 con 0x55649bd85c00 2023-09-23T03:21:36.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.193+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 <== osd.5 v2:172.21.15.190:6800/882536885 7 ==== osd_op_reply(38 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12b40 con 0x55649bd85c00 2023-09-23T03:21:36.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.193+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:39 8.18 8:18000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba000 con 0x55649bd9b000 2023-09-23T03:21:36.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.194+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 <== osd.4 v2:172.21.15.190:6808/717302641 5 ==== osd_op_reply(39 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12fc0 con 0x55649bd9b000 2023-09-23T03:21:36.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.194+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:40 8.4 8:20000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba000 con 0x55649bd85c00 2023-09-23T03:21:36.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.195+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 <== osd.5 v2:172.21.15.190:6800/882536885 8 ==== osd_op_reply(40 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12b40 con 0x55649bd85c00 2023-09-23T03:21:36.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.195+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:41 8.14 8:28000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba000 con 0x55649bd9b000 2023-09-23T03:21:36.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.196+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 <== osd.4 v2:172.21.15.190:6808/717302641 6 ==== osd_op_reply(41 [pgnls start_epoch 67 out=125b] v67'107 uv107 ondisk = 1) v8 ==== 144+0+125 (crc 0 0 0) 0x55649bd12fc0 con 0x55649bd9b000 2023-09-23T03:21:36.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.196+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:42 8.c 8:30000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba000 con 0x55649bea4000 2023-09-23T03:21:36.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.197+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.0 v2:172.21.15.154:6800/2920938138 8 ==== osd_op_reply(42 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12d80 con 0x55649bea4000 2023-09-23T03:21:36.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.197+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:43 8.1c 8:38000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba000 con 0x55649bd85c00 2023-09-23T03:21:36.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.198+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 <== osd.5 v2:172.21.15.190:6800/882536885 9 ==== osd_op_reply(43 [pgnls start_epoch 67 out=49b] v66'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12b40 con 0x55649bd85c00 2023-09-23T03:21:36.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.198+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:44 8.2 8:40000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba000 con 0x55649bd9b000 2023-09-23T03:21:36.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.199+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 <== osd.4 v2:172.21.15.190:6808/717302641 7 ==== osd_op_reply(44 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12fc0 con 0x55649bd9b000 2023-09-23T03:21:36.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.199+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:45 8.12 8:48000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba000 con 0x55649bea4400 2023-09-23T03:21:36.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.201+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 <== osd.6 v2:172.21.15.190:6816/1964294001 4 ==== osd_op_reply(45 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12b40 con 0x55649bea4400 2023-09-23T03:21:36.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.201+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:46 8.a 8:50000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba000 con 0x55649bd9b400 2023-09-23T03:21:36.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.201+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 <== osd.1 v2:172.21.15.154:6816/1267710005 5 ==== osd_op_reply(46 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12fc0 con 0x55649bd9b400 2023-09-23T03:21:36.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.201+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:47 8.1a 8:58000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba000 con 0x55649bea4000 2023-09-23T03:21:36.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.201+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.0 v2:172.21.15.154:6800/2920938138 9 ==== osd_op_reply(47 [pgnls start_epoch 67 out=49b] v67'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12d80 con 0x55649bea4000 2023-09-23T03:21:36.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.202+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:48 8.6 8:60000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba000 con 0x55649bea4000 2023-09-23T03:21:36.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.202+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.0 v2:172.21.15.154:6800/2920938138 10 ==== osd_op_reply(48 [pgnls start_epoch 67 out=49b] v66'24 uv24 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12d80 con 0x55649bea4000 2023-09-23T03:21:36.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.202+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:49 8.16 8:68000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba000 con 0x55649bd84800 2023-09-23T03:21:36.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.203+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 <== osd.3 v2:172.21.15.154:6808/1133688468 9 ==== osd_op_reply(49 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12b40 con 0x55649bd84800 2023-09-23T03:21:36.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.203+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:50 8.e 8:70000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba000 con 0x55649bea4000 2023-09-23T03:21:36.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.203+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.0 v2:172.21.15.154:6800/2920938138 11 ==== osd_op_reply(50 [pgnls start_epoch 67 out=49b] v66'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12d80 con 0x55649bea4000 2023-09-23T03:21:36.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.203+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:51 8.1e 8:78000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba000 con 0x55649bd84800 2023-09-23T03:21:36.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.204+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 <== osd.3 v2:172.21.15.154:6808/1133688468 10 ==== osd_op_reply(51 [pgnls start_epoch 67 out=49b] v67'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12b40 con 0x55649bd84800 2023-09-23T03:21:36.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.204+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:52 8.1 8:80000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba000 con 0x55649bd9b400 2023-09-23T03:21:36.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.204+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 <== osd.1 v2:172.21.15.154:6816/1267710005 6 ==== osd_op_reply(52 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12fc0 con 0x55649bd9b400 2023-09-23T03:21:36.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.204+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:53 8.11 8:88000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba000 con 0x55649bd85c00 2023-09-23T03:21:36.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.206+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 <== osd.5 v2:172.21.15.190:6800/882536885 10 ==== osd_op_reply(53 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12b40 con 0x55649bd85c00 2023-09-23T03:21:36.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.206+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:54 8.9 8:90000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba000 con 0x55649bea4400 2023-09-23T03:21:36.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.207+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 <== osd.6 v2:172.21.15.190:6816/1964294001 5 ==== osd_op_reply(54 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12b40 con 0x55649bea4400 2023-09-23T03:21:36.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.207+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:55 8.19 8:98000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba000 con 0x55649bd85c00 2023-09-23T03:21:36.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.208+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 <== osd.5 v2:172.21.15.190:6800/882536885 11 ==== osd_op_reply(55 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12b40 con 0x55649bd85c00 2023-09-23T03:21:36.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.208+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:56 8.5 8:a0000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba000 con 0x55649bea4400 2023-09-23T03:21:36.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.209+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 <== osd.6 v2:172.21.15.190:6816/1964294001 6 ==== osd_op_reply(56 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12b40 con 0x55649bea4400 2023-09-23T03:21:36.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.209+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:57 8.15 8:a8000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba000 con 0x55649ba34c00 2023-09-23T03:21:36.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.210+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 6 ==== osd_op_reply(57 [pgnls start_epoch 67 out=73b] v66'6 uv6 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.210+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:58 8.d 8:b0000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba000 con 0x55649bd85c00 2023-09-23T03:21:36.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.211+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 <== osd.5 v2:172.21.15.190:6800/882536885 12 ==== osd_op_reply(58 [pgnls start_epoch 67 out=74b] v67'3 uv3 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55649bd12b40 con 0x55649bd85c00 2023-09-23T03:21:36.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.212+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:59 8.1d 8:b8000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba000 con 0x55649bea4000 2023-09-23T03:21:36.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.212+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.0 v2:172.21.15.154:6800/2920938138 12 ==== osd_op_reply(59 [pgnls start_epoch 67 out=49b] v66'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12d80 con 0x55649bea4000 2023-09-23T03:21:36.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.212+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:60 8.3 8:c0000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba000 con 0x55649bd9b400 2023-09-23T03:21:36.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.213+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 <== osd.1 v2:172.21.15.154:6816/1267710005 7 ==== osd_op_reply(60 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12fc0 con 0x55649bd9b400 2023-09-23T03:21:36.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.213+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:61 8.13 8:c8000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba000 con 0x55649bd84800 2023-09-23T03:21:36.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.214+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 <== osd.3 v2:172.21.15.154:6808/1133688468 11 ==== osd_op_reply(61 [pgnls start_epoch 67 out=98b] v67'7 uv7 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55649bd12b40 con 0x55649bd84800 2023-09-23T03:21:36.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.214+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:62 8.b 8:d0000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba000 con 0x55649bd84800 2023-09-23T03:21:36.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.214+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 <== osd.3 v2:172.21.15.154:6808/1133688468 12 ==== osd_op_reply(62 [pgnls start_epoch 67 out=76b] v67'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55649bd12b40 con 0x55649bd84800 2023-09-23T03:21:36.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.214+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:63 8.1b 8:d8000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba000 con 0x55649bd9b000 2023-09-23T03:21:36.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.215+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 <== osd.4 v2:172.21.15.190:6808/717302641 8 ==== osd_op_reply(63 [pgnls start_epoch 67 out=49b] v67'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12fc0 con 0x55649bd9b000 2023-09-23T03:21:36.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.215+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:64 8.7 8:e0000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba000 con 0x55649bd9b400 2023-09-23T03:21:36.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.216+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 <== osd.1 v2:172.21.15.154:6816/1267710005 8 ==== osd_op_reply(64 [pgnls start_epoch 67 out=49b] v66'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12fc0 con 0x55649bd9b400 2023-09-23T03:21:36.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.216+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:65 8.17 8:e8000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba000 con 0x55649ba34c00 2023-09-23T03:21:36.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.217+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 7 ==== osd_op_reply(65 [pgnls start_epoch 67 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.217+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:66 8.f 8:f0000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba000 con 0x55649bd84800 2023-09-23T03:21:36.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.218+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 <== osd.3 v2:172.21.15.154:6808/1133688468 13 ==== osd_op_reply(66 [pgnls start_epoch 67 out=73b] v67'4 uv4 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55649bd12b40 con 0x55649bd84800 2023-09-23T03:21:36.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.218+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:67 8.1f 8:f8000000::::head [pgnls start_epoch 67 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba000 con 0x55649bea4000 2023-09-23T03:21:36.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.219+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.0 v2:172.21.15.154:6800/2920938138 13 ==== osd_op_reply(67 [pgnls start_epoch 67 out=101b] v67'12 uv12 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55649bd12d80 con 0x55649bea4000 2023-09-23T03:21:36.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.219+0000 7f5f7db6e5c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2023-09-23T03:21:36.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.219+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:68 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba380 con 0x55649ba34c00 2023-09-23T03:21:36.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.220+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 8 ==== osd_op_reply(68 10000000000.00000000 [omap-get-vals out=12470b] v0'0 uv41 ondisk = 0) v8 ==== 164+0+12470 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2023-09-23T03:21:36.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: adding 0x100000001ff for future processing to fix dnfirst 2023-09-23T03:21:36.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: adding 0x10000000200 for future processing to fix dnfirst 2023-09-23T03:21:36.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: adding 0x10000000201 for future processing to fix dnfirst 2023-09-23T03:21:36.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: adding 0x10000000202 for future processing to fix dnfirst 2023-09-23T03:21:36.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: adding 0x10000000203 for future processing to fix dnfirst 2023-09-23T03:21:36.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: adding 0x10000000204 for future processing to fix dnfirst 2023-09-23T03:21:36.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: adding 0x10000000205 for future processing to fix dnfirst 2023-09-23T03:21:36.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: adding 0x10000000206 for future processing to fix dnfirst 2023-09-23T03:21:36.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: adding 0x10000000207 for future processing to fix dnfirst 2023-09-23T03:21:36.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: adding 0x10000000208 for future processing to fix dnfirst 2023-09-23T03:21:36.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: adding 0x100000001f6 for future processing to fix dnfirst 2023-09-23T03:21:36.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: adding 0x10000000209 for future processing to fix dnfirst 2023-09-23T03:21:36.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: adding 0x1000000020a for future processing to fix dnfirst 2023-09-23T03:21:36.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: adding 0x1000000020b for future processing to fix dnfirst 2023-09-23T03:21:36.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: adding 0x1000000020c for future processing to fix dnfirst 2023-09-23T03:21:36.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: adding 0x1000000020d for future processing to fix dnfirst 2023-09-23T03:21:36.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2023-09-23T03:21:36.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: adding 0x100000001f8 for future processing to fix dnfirst 2023-09-23T03:21:36.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: adding 0x100000001f9 for future processing to fix dnfirst 2023-09-23T03:21:36.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: adding 0x100000001fa for future processing to fix dnfirst 2023-09-23T03:21:36.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: adding 0x100000001fb for future processing to fix dnfirst 2023-09-23T03:21:36.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: adding 0x100000001fc for future processing to fix dnfirst 2023-09-23T03:21:36.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: adding 0x100000001fd for future processing to fix dnfirst 2023-09-23T03:21:36.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 20 datascan.scan_links: adding 0x100000001fe for future processing to fix dnfirst 2023-09-23T03:21:36.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2023-09-23T03:21:36.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-09-23T03:21:36.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 10 datascan.scan_links: step 2: handling object 200.00000000 2023-09-23T03:21:36.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-09-23T03:21:36.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 10 datascan.scan_links: step 2: handling object 200.00000001 2023-09-23T03:21:36.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-09-23T03:21:36.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 10 datascan.scan_links: step 2: handling object 500.00000000 2023-09-23T03:21:36.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-09-23T03:21:36.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2023-09-23T03:21:36.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-09-23T03:21:36.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2023-09-23T03:21:36.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-09-23T03:21:36.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 10 datascan.scan_links: step 2: handling object 100.00000000 2023-09-23T03:21:36.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.221+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:69 8.13 8:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba700 con 0x55649bd84800 2023-09-23T03:21:36.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.222+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 <== osd.3 v2:172.21.15.154:6808/1133688468 14 ==== osd_op_reply(69 100.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55649bd12b40 con 0x55649bd84800 2023-09-23T03:21:36.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.222+0000 7f5f7db6e5c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2023-09-23T03:21:36.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.222+0000 7f5f7db6e5c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-09-23T03:21:36.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.222+0000 7f5f7db6e5c0 10 datascan.scan_links: step 2: handling object 400.00000000 2023-09-23T03:21:36.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.222+0000 7f5f7db6e5c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-09-23T03:21:36.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.222+0000 7f5f7db6e5c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2023-09-23T03:21:36.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.222+0000 7f5f7db6e5c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-09-23T03:21:36.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.222+0000 7f5f7db6e5c0 10 datascan.scan_links: step 2: handling object 1.00000000 2023-09-23T03:21:36.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.222+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:70 8.1f 8:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bebaa80 con 0x55649bea4000 2023-09-23T03:21:36.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.222+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.0 v2:172.21.15.154:6800/2920938138 14 ==== osd_op_reply(70 1.00000000 [omap-get-vals out=508b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+508 (crc 0 0 0) 0x55649bd12d80 con 0x55649bea4000 2023-09-23T03:21:36.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.223+0000 7f5f7db6e5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:21:36.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.223+0000 7f5f7db6e5c0 20 datascan.scan_links: adding 0x10000000000 for future processing to fix dnfirst 2023-09-23T03:21:36.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.223+0000 7f5f7db6e5c0 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2023-09-23T03:21:36.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.223+0000 7f5f7db6e5c0 10 datascan.scan_links: removing dup dentries from 0 objects 2023-09-23T03:21:36.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.223+0000 7f5f7db6e5c0 10 datascan.scan_links: processing 0 bad_nlink_inos 2023-09-23T03:21:36.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.223+0000 7f5f7db6e5c0 10 datascan.scan_links: processing 26 injected_inos 2023-09-23T03:21:36.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.223+0000 7f5f7db6e5c0 10 datascan.scan_links: handling injected_ino 0x10000000000 2023-09-23T03:21:36.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.223+0000 7f5f7db6e5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x1:subdir 2023-09-23T03:21:36.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.223+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:71 8.1f 8:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bebae00 con 0x55649bea4000 2023-09-23T03:21:36.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.223+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.0 v2:172.21.15.154:6800/2920938138 15 ==== osd_op_reply(71 1.00000000 [omap-get-vals-by-keys out=507b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+507 (crc 0 0 0) 0x55649bd12d80 con 0x55649bea4000 2023-09-23T03:21:36.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.223+0000 7f5f7db6e5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:21:36.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.224+0000 7f5f7db6e5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:21:36.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.224+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:72 8.1f 8:ff5b34d6:::1.00000000:head [omap-set-vals in=507b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba000 con 0x55649bea4000 2023-09-23T03:21:36.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.227+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.0 v2:172.21.15.154:6800/2920938138 16 ==== osd_op_reply(72 1.00000000 [omap-set-vals] v67'13 uv13 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x55649bd12d80 con 0x55649bea4000 2023-09-23T03:21:36.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.227+0000 7f5f7db6e5c0 20 datascan.inject_linkage: Injected dentry 0x0x1/subdir pointing to 0x0x10000000000 2023-09-23T03:21:36.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.227+0000 7f5f7db6e5c0 10 datascan.scan_links: handling injected_ino 0x10000000001 2023-09-23T03:21:36.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.227+0000 7f5f7db6e5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:0 2023-09-23T03:21:36.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.227+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:73 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba380 con 0x55649ba34c00 2023-09-23T03:21:36.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.228+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 9 ==== osd_op_reply(73 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv41 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.228+0000 7f5f7db6e5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=0 frag=*, r=0 2023-09-23T03:21:36.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.228+0000 7f5f7db6e5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:21:36.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.229+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:74 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bebb180 con 0x55649ba34c00 2023-09-23T03:21:36.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.232+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 10 ==== osd_op_reply(74 10000000000.00000000 [omap-set-vals] v67'42 uv42 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.232+0000 7f5f7db6e5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/0 pointing to 0x0x10000000001 2023-09-23T03:21:36.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.232+0000 7f5f7db6e5c0 10 datascan.scan_links: handling injected_ino 0x100000001f6 2023-09-23T03:21:36.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.232+0000 7f5f7db6e5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:1 2023-09-23T03:21:36.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.232+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:75 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bebb500 con 0x55649ba34c00 2023-09-23T03:21:36.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.233+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 11 ==== osd_op_reply(75 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv42 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.233+0000 7f5f7db6e5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=1 frag=*, r=0 2023-09-23T03:21:36.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.233+0000 7f5f7db6e5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:21:36.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.233+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:76 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bebb880 con 0x55649ba34c00 2023-09-23T03:21:36.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.238+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 12 ==== osd_op_reply(76 10000000000.00000000 [omap-set-vals] v67'43 uv43 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.238+0000 7f5f7db6e5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/1 pointing to 0x0x100000001f6 2023-09-23T03:21:36.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.238+0000 7f5f7db6e5c0 10 datascan.scan_links: handling injected_ino 0x100000001f7 2023-09-23T03:21:36.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.238+0000 7f5f7db6e5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:2 2023-09-23T03:21:36.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.238+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:77 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8fc00 con 0x55649ba34c00 2023-09-23T03:21:36.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.238+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 13 ==== osd_op_reply(77 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv43 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.238+0000 7f5f7db6e5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=2 frag=*, r=0 2023-09-23T03:21:36.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.238+0000 7f5f7db6e5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:21:36.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.238+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:78 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bebaa80 con 0x55649ba34c00 2023-09-23T03:21:36.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.243+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 14 ==== osd_op_reply(78 10000000000.00000000 [omap-set-vals] v67'44 uv44 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.243+0000 7f5f7db6e5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/2 pointing to 0x0x100000001f7 2023-09-23T03:21:36.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.243+0000 7f5f7db6e5c0 10 datascan.scan_links: handling injected_ino 0x100000001f8 2023-09-23T03:21:36.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.243+0000 7f5f7db6e5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:3 2023-09-23T03:21:36.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.243+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:79 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bebae00 con 0x55649ba34c00 2023-09-23T03:21:36.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.244+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 15 ==== osd_op_reply(79 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv44 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.244+0000 7f5f7db6e5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=3 frag=*, r=0 2023-09-23T03:21:36.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.244+0000 7f5f7db6e5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:21:36.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.244+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:80 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba000 con 0x55649ba34c00 2023-09-23T03:21:36.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.247+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 16 ==== osd_op_reply(80 10000000000.00000000 [omap-set-vals] v67'45 uv45 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.247+0000 7f5f7db6e5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/3 pointing to 0x0x100000001f8 2023-09-23T03:21:36.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.247+0000 7f5f7db6e5c0 10 datascan.scan_links: handling injected_ino 0x100000001f9 2023-09-23T03:21:36.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.247+0000 7f5f7db6e5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:4 2023-09-23T03:21:36.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.247+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:81 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba380 con 0x55649ba34c00 2023-09-23T03:21:36.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.248+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 17 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv45 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.248+0000 7f5f7db6e5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=4 frag=*, r=0 2023-09-23T03:21:36.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.248+0000 7f5f7db6e5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:21:36.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.248+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:82 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bebbc00 con 0x55649ba34c00 2023-09-23T03:21:36.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.252+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 18 ==== osd_op_reply(82 10000000000.00000000 [omap-set-vals] v67'46 uv46 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.252+0000 7f5f7db6e5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/4 pointing to 0x0x100000001f9 2023-09-23T03:21:36.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.252+0000 7f5f7db6e5c0 10 datascan.scan_links: handling injected_ino 0x100000001fa 2023-09-23T03:21:36.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.252+0000 7f5f7db6e5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:5 2023-09-23T03:21:36.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.252+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:83 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bebb180 con 0x55649ba34c00 2023-09-23T03:21:36.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.253+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 19 ==== osd_op_reply(83 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv46 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.253+0000 7f5f7db6e5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=5 frag=*, r=0 2023-09-23T03:21:36.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.253+0000 7f5f7db6e5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:21:36.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.253+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:84 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bebb500 con 0x55649ba34c00 2023-09-23T03:21:36.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.257+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 20 ==== osd_op_reply(84 10000000000.00000000 [omap-set-vals] v67'47 uv47 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.257+0000 7f5f7db6e5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/5 pointing to 0x0x100000001fa 2023-09-23T03:21:36.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.257+0000 7f5f7db6e5c0 10 datascan.scan_links: handling injected_ino 0x100000001fb 2023-09-23T03:21:36.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.257+0000 7f5f7db6e5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:6 2023-09-23T03:21:36.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.257+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:85 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bebb880 con 0x55649ba34c00 2023-09-23T03:21:36.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.258+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 21 ==== osd_op_reply(85 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv47 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.258+0000 7f5f7db6e5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=6 frag=*, r=0 2023-09-23T03:21:36.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.259+0000 7f5f7db6e5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:21:36.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.259+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:86 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bebaa80 con 0x55649ba34c00 2023-09-23T03:21:36.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.263+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 22 ==== osd_op_reply(86 10000000000.00000000 [omap-set-vals] v67'48 uv48 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.263+0000 7f5f7db6e5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/6 pointing to 0x0x100000001fb 2023-09-23T03:21:36.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.263+0000 7f5f7db6e5c0 10 datascan.scan_links: handling injected_ino 0x100000001fc 2023-09-23T03:21:36.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.263+0000 7f5f7db6e5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:7 2023-09-23T03:21:36.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.264+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:87 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8fc00 con 0x55649ba34c00 2023-09-23T03:21:36.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.265+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 23 ==== osd_op_reply(87 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv48 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.265+0000 7f5f7db6e5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=7 frag=*, r=0 2023-09-23T03:21:36.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.265+0000 7f5f7db6e5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:21:36.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.265+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:88 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bebae00 con 0x55649ba34c00 2023-09-23T03:21:36.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.268+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 24 ==== osd_op_reply(88 10000000000.00000000 [omap-set-vals] v67'49 uv49 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.268+0000 7f5f7db6e5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/7 pointing to 0x0x100000001fc 2023-09-23T03:21:36.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.268+0000 7f5f7db6e5c0 10 datascan.scan_links: handling injected_ino 0x100000001fd 2023-09-23T03:21:36.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.268+0000 7f5f7db6e5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:8 2023-09-23T03:21:36.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.268+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:89 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba000 con 0x55649ba34c00 2023-09-23T03:21:36.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.270+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 25 ==== osd_op_reply(89 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv49 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.270+0000 7f5f7db6e5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=8 frag=*, r=0 2023-09-23T03:21:36.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.270+0000 7f5f7db6e5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:21:36.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.270+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:90 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba380 con 0x55649ba34c00 2023-09-23T03:21:36.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.274+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 26 ==== osd_op_reply(90 10000000000.00000000 [omap-set-vals] v67'50 uv50 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.274+0000 7f5f7db6e5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/8 pointing to 0x0x100000001fd 2023-09-23T03:21:36.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.274+0000 7f5f7db6e5c0 10 datascan.scan_links: handling injected_ino 0x100000001fe 2023-09-23T03:21:36.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.274+0000 7f5f7db6e5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:9 2023-09-23T03:21:36.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.274+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:91 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bebbc00 con 0x55649ba34c00 2023-09-23T03:21:36.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.275+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 27 ==== osd_op_reply(91 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv50 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.275+0000 7f5f7db6e5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=9 frag=*, r=0 2023-09-23T03:21:36.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.275+0000 7f5f7db6e5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:21:36.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.275+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:92 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bebb180 con 0x55649ba34c00 2023-09-23T03:21:36.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.279+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 28 ==== osd_op_reply(92 10000000000.00000000 [omap-set-vals] v67'51 uv51 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.279+0000 7f5f7db6e5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/9 pointing to 0x0x100000001fe 2023-09-23T03:21:36.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.279+0000 7f5f7db6e5c0 10 datascan.scan_links: handling injected_ino 0x100000001ff 2023-09-23T03:21:36.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.279+0000 7f5f7db6e5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:10 2023-09-23T03:21:36.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.279+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:93 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bebb500 con 0x55649ba34c00 2023-09-23T03:21:36.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.280+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 29 ==== osd_op_reply(93 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv51 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.280+0000 7f5f7db6e5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=10 frag=*, r=0 2023-09-23T03:21:36.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.280+0000 7f5f7db6e5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:21:36.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.280+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:94 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bebb880 con 0x55649ba34c00 2023-09-23T03:21:36.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.283+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 30 ==== osd_op_reply(94 10000000000.00000000 [omap-set-vals] v67'52 uv52 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.284+0000 7f5f7db6e5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/10 pointing to 0x0x100000001ff 2023-09-23T03:21:36.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.284+0000 7f5f7db6e5c0 10 datascan.scan_links: handling injected_ino 0x10000000200 2023-09-23T03:21:36.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.284+0000 7f5f7db6e5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:11 2023-09-23T03:21:36.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.284+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:95 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bebaa80 con 0x55649ba34c00 2023-09-23T03:21:36.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.285+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 31 ==== osd_op_reply(95 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv52 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.285+0000 7f5f7db6e5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=11 frag=*, r=0 2023-09-23T03:21:36.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.285+0000 7f5f7db6e5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:21:36.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.285+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:96 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bebae00 con 0x55649ba34c00 2023-09-23T03:21:36.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.289+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 32 ==== osd_op_reply(96 10000000000.00000000 [omap-set-vals] v67'53 uv53 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.289+0000 7f5f7db6e5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/11 pointing to 0x0x10000000200 2023-09-23T03:21:36.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.289+0000 7f5f7db6e5c0 10 datascan.scan_links: handling injected_ino 0x10000000201 2023-09-23T03:21:36.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.289+0000 7f5f7db6e5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:12 2023-09-23T03:21:36.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.289+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:97 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba000 con 0x55649ba34c00 2023-09-23T03:21:36.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.291+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 33 ==== osd_op_reply(97 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv53 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.291+0000 7f5f7db6e5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=12 frag=*, r=0 2023-09-23T03:21:36.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.291+0000 7f5f7db6e5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:21:36.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.291+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:98 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba380 con 0x55649ba34c00 2023-09-23T03:21:36.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.294+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 34 ==== osd_op_reply(98 10000000000.00000000 [omap-set-vals] v67'54 uv54 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.294+0000 7f5f7db6e5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/12 pointing to 0x0x10000000201 2023-09-23T03:21:36.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.294+0000 7f5f7db6e5c0 10 datascan.scan_links: handling injected_ino 0x10000000202 2023-09-23T03:21:36.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.294+0000 7f5f7db6e5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:13 2023-09-23T03:21:36.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.294+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:99 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bebbc00 con 0x55649ba34c00 2023-09-23T03:21:36.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.295+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 35 ==== osd_op_reply(99 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv54 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.295+0000 7f5f7db6e5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=13 frag=*, r=0 2023-09-23T03:21:36.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.295+0000 7f5f7db6e5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:21:36.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.295+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:100 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8fc00 con 0x55649ba34c00 2023-09-23T03:21:36.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.299+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 36 ==== osd_op_reply(100 10000000000.00000000 [omap-set-vals] v67'55 uv55 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.299+0000 7f5f7db6e5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/13 pointing to 0x0x10000000202 2023-09-23T03:21:36.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.299+0000 7f5f7db6e5c0 10 datascan.scan_links: handling injected_ino 0x10000000203 2023-09-23T03:21:36.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.299+0000 7f5f7db6e5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:14 2023-09-23T03:21:36.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.299+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:101 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bebb180 con 0x55649ba34c00 2023-09-23T03:21:36.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.300+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 37 ==== osd_op_reply(101 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv55 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.300+0000 7f5f7db6e5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=14 frag=*, r=0 2023-09-23T03:21:36.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.300+0000 7f5f7db6e5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:21:36.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.300+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:102 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bebb500 con 0x55649ba34c00 2023-09-23T03:21:36.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.304+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 38 ==== osd_op_reply(102 10000000000.00000000 [omap-set-vals] v67'56 uv56 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.304+0000 7f5f7db6e5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/14 pointing to 0x0x10000000203 2023-09-23T03:21:36.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.304+0000 7f5f7db6e5c0 10 datascan.scan_links: handling injected_ino 0x10000000204 2023-09-23T03:21:36.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.304+0000 7f5f7db6e5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:15 2023-09-23T03:21:36.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.304+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:103 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bebb880 con 0x55649ba34c00 2023-09-23T03:21:36.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.305+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 39 ==== osd_op_reply(103 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv56 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.305+0000 7f5f7db6e5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=15 frag=*, r=0 2023-09-23T03:21:36.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.305+0000 7f5f7db6e5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:21:36.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.305+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:104 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bebaa80 con 0x55649ba34c00 2023-09-23T03:21:36.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.309+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 40 ==== osd_op_reply(104 10000000000.00000000 [omap-set-vals] v67'57 uv57 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.309+0000 7f5f7db6e5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/15 pointing to 0x0x10000000204 2023-09-23T03:21:36.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.309+0000 7f5f7db6e5c0 10 datascan.scan_links: handling injected_ino 0x10000000205 2023-09-23T03:21:36.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.309+0000 7f5f7db6e5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:16 2023-09-23T03:21:36.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.309+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:105 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bebae00 con 0x55649ba34c00 2023-09-23T03:21:36.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.310+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 41 ==== osd_op_reply(105 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv57 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.310+0000 7f5f7db6e5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=16 frag=*, r=0 2023-09-23T03:21:36.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.310+0000 7f5f7db6e5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:21:36.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.310+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:106 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba000 con 0x55649ba34c00 2023-09-23T03:21:36.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.314+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 42 ==== osd_op_reply(106 10000000000.00000000 [omap-set-vals] v67'58 uv58 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.314+0000 7f5f7db6e5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/16 pointing to 0x0x10000000205 2023-09-23T03:21:36.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.314+0000 7f5f7db6e5c0 10 datascan.scan_links: handling injected_ino 0x10000000206 2023-09-23T03:21:36.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.314+0000 7f5f7db6e5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:17 2023-09-23T03:21:36.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.314+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:107 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba380 con 0x55649ba34c00 2023-09-23T03:21:36.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.316+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 43 ==== osd_op_reply(107 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv58 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.316+0000 7f5f7db6e5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=17 frag=*, r=0 2023-09-23T03:21:36.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.316+0000 7f5f7db6e5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:21:36.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.316+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:108 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bebbc00 con 0x55649ba34c00 2023-09-23T03:21:36.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.319+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 44 ==== osd_op_reply(108 10000000000.00000000 [omap-set-vals] v67'59 uv59 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.319+0000 7f5f7db6e5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/17 pointing to 0x0x10000000206 2023-09-23T03:21:36.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.319+0000 7f5f7db6e5c0 10 datascan.scan_links: handling injected_ino 0x10000000207 2023-09-23T03:21:36.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.319+0000 7f5f7db6e5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:18 2023-09-23T03:21:36.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.319+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:109 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bebb180 con 0x55649ba34c00 2023-09-23T03:21:36.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.320+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 45 ==== osd_op_reply(109 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv59 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.320+0000 7f5f7db6e5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=18 frag=*, r=0 2023-09-23T03:21:36.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.320+0000 7f5f7db6e5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:21:36.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.320+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:110 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bebb500 con 0x55649ba34c00 2023-09-23T03:21:36.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.324+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 46 ==== osd_op_reply(110 10000000000.00000000 [omap-set-vals] v67'60 uv60 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.324+0000 7f5f7db6e5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/18 pointing to 0x0x10000000207 2023-09-23T03:21:36.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.324+0000 7f5f7db6e5c0 10 datascan.scan_links: handling injected_ino 0x10000000208 2023-09-23T03:21:36.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.324+0000 7f5f7db6e5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:19 2023-09-23T03:21:36.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.324+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:111 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bebb880 con 0x55649ba34c00 2023-09-23T03:21:36.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.325+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 47 ==== osd_op_reply(111 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv60 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.325+0000 7f5f7db6e5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=19 frag=*, r=0 2023-09-23T03:21:36.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.325+0000 7f5f7db6e5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:21:36.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.325+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:112 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bebaa80 con 0x55649ba34c00 2023-09-23T03:21:36.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.328+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 48 ==== osd_op_reply(112 10000000000.00000000 [omap-set-vals] v67'61 uv61 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.328+0000 7f5f7db6e5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/19 pointing to 0x0x10000000208 2023-09-23T03:21:36.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.328+0000 7f5f7db6e5c0 10 datascan.scan_links: handling injected_ino 0x10000000209 2023-09-23T03:21:36.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.328+0000 7f5f7db6e5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:20 2023-09-23T03:21:36.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.328+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:113 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bebae00 con 0x55649ba34c00 2023-09-23T03:21:36.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.329+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 49 ==== osd_op_reply(113 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv61 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.330+0000 7f5f7db6e5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=20 frag=*, r=0 2023-09-23T03:21:36.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.330+0000 7f5f7db6e5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:21:36.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.330+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:114 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba000 con 0x55649ba34c00 2023-09-23T03:21:36.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.334+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 50 ==== osd_op_reply(114 10000000000.00000000 [omap-set-vals] v67'62 uv62 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.334+0000 7f5f7db6e5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/20 pointing to 0x0x10000000209 2023-09-23T03:21:36.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.334+0000 7f5f7db6e5c0 10 datascan.scan_links: handling injected_ino 0x1000000020a 2023-09-23T03:21:36.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.334+0000 7f5f7db6e5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:21 2023-09-23T03:21:36.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.334+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:115 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649beba380 con 0x55649ba34c00 2023-09-23T03:21:36.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.334+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 51 ==== osd_op_reply(115 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv62 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.334+0000 7f5f7db6e5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=21 frag=*, r=0 2023-09-23T03:21:36.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.334+0000 7f5f7db6e5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:21:36.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.334+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:116 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bebbc00 con 0x55649ba34c00 2023-09-23T03:21:36.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.339+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 52 ==== osd_op_reply(116 10000000000.00000000 [omap-set-vals] v67'63 uv63 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.339+0000 7f5f7db6e5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/21 pointing to 0x0x1000000020a 2023-09-23T03:21:36.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.339+0000 7f5f7db6e5c0 10 datascan.scan_links: handling injected_ino 0x1000000020b 2023-09-23T03:21:36.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.339+0000 7f5f7db6e5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:22 2023-09-23T03:21:36.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.339+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:117 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649ba8fc00 con 0x55649ba34c00 2023-09-23T03:21:36.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.340+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 53 ==== osd_op_reply(117 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv63 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.340+0000 7f5f7db6e5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=22 frag=*, r=0 2023-09-23T03:21:36.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.340+0000 7f5f7db6e5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:21:36.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.340+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:118 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bee4000 con 0x55649ba34c00 2023-09-23T03:21:36.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.343+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 54 ==== osd_op_reply(118 10000000000.00000000 [omap-set-vals] v67'64 uv64 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.343+0000 7f5f7db6e5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/22 pointing to 0x0x1000000020b 2023-09-23T03:21:36.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.343+0000 7f5f7db6e5c0 10 datascan.scan_links: handling injected_ino 0x1000000020c 2023-09-23T03:21:36.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.343+0000 7f5f7db6e5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:23 2023-09-23T03:21:36.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.343+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:119 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bee4380 con 0x55649ba34c00 2023-09-23T03:21:36.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.344+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 55 ==== osd_op_reply(119 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv64 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.345+0000 7f5f7db6e5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=23 frag=*, r=0 2023-09-23T03:21:36.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.345+0000 7f5f7db6e5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:21:36.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.345+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:120 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bee4700 con 0x55649ba34c00 2023-09-23T03:21:36.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.349+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 56 ==== osd_op_reply(120 10000000000.00000000 [omap-set-vals] v67'65 uv65 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.349+0000 7f5f7db6e5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/23 pointing to 0x0x1000000020c 2023-09-23T03:21:36.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.349+0000 7f5f7db6e5c0 10 datascan.scan_links: handling injected_ino 0x1000000020d 2023-09-23T03:21:36.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.349+0000 7f5f7db6e5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:24 2023-09-23T03:21:36.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.349+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:121 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bee4a80 con 0x55649ba34c00 2023-09-23T03:21:36.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.349+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 57 ==== osd_op_reply(121 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv65 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.349+0000 7f5f7db6e5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=24 frag=*, r=0 2023-09-23T03:21:36.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.349+0000 7f5f7db6e5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:21:36.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.349+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:122 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bee4e00 con 0x55649ba34c00 2023-09-23T03:21:36.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.354+0000 7f5f7cb17640 1 -- 172.21.15.154:0/2288557327 <== osd.7 v2:172.21.15.190:6824/3460136272 58 ==== osd_op_reply(122 10000000000.00000000 [omap-set-vals] v67'66 uv66 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55649bd12d80 con 0x55649ba34c00 2023-09-23T03:21:36.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.354+0000 7f5f7db6e5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/24 pointing to 0x0x1000000020d 2023-09-23T03:21:36.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.354+0000 7f5f7db6e5c0 10 datascan.scan_links: updating inotable 2023-09-23T03:21:36.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.354+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:123 8.13 8:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bee5180 con 0x55649bd84800 2023-09-23T03:21:36.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.355+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 <== osd.3 v2:172.21.15.154:6808/1133688468 15 ==== osd_op_reply(123 mds0_inotable [read 0~34 out=34b] v0'0 uv6 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x55649bd12b40 con 0x55649bd84800 2023-09-23T03:21:36.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.355+0000 7f5f7db6e5c0 10 mds.0.inotable: skip_inos was [0x10000000000~0x10000000000] 2023-09-23T03:21:36.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.355+0000 7f5f7db6e5c0 10 mds.0.inotable: skip_inos now [0x1000000020e~0xfffffffdf2] 2023-09-23T03:21:36.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.355+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:124 8.13 8:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bee5500 con 0x55649bd84800 2023-09-23T03:21:36.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.359+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 <== osd.3 v2:172.21.15.154:6808/1133688468 16 ==== osd_op_reply(124 mds0_inotable [writefull 0~34] v67'8 uv8 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x55649bd12b40 con 0x55649bd84800 2023-09-23T03:21:36.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.359+0000 7f5f7db6e5c0 10 datascan.scan_links: updating snaptable 2023-09-23T03:21:36.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.359+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:125 8.d 8:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e67) v8 -- 0x55649bee5880 con 0x55649bd85c00 2023-09-23T03:21:36.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.360+0000 7f5f7c316640 1 -- 172.21.15.154:0/2288557327 <== osd.5 v2:172.21.15.190:6800/882536885 13 ==== osd_op_reply(125 mds_snaptable [read 0~70 out=70b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x55649bd12b40 con 0x55649bd85c00 2023-09-23T03:21:36.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.361+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55649bea4000 msgr2=0x55649be89600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.361+0000 7f5f7db6e5c0 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55649bea4000 0x55649be89600 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.361+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55649bd9b400 msgr2=0x55649be89080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.361+0000 7f5f7db6e5c0 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55649bd9b400 0x55649be89080 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.361+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55649bd84800 msgr2=0x55649baa0680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.361+0000 7f5f7db6e5c0 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55649bd84800 0x55649baa0680 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.361+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55649bd9b000 msgr2=0x55649be88b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.361+0000 7f5f7db6e5c0 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55649bd9b000 0x55649be88b00 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.361+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55649bd85c00 msgr2=0x55649ba9e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.361+0000 7f5f7db6e5c0 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55649bd85c00 0x55649ba9e580 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.361+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55649bea4400 msgr2=0x55649be89b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.361+0000 7f5f7db6e5c0 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55649bea4400 0x55649be89b80 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.361+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55649ba34c00 msgr2=0x55649be88580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.361+0000 7f5f7db6e5c0 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55649ba34c00 0x55649be88580 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.361+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 reap_dead start 2023-09-23T03:21:36.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.361+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 reap_dead start 2023-09-23T03:21:36.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.361+0000 7f5f7bb15640 1 -- 172.21.15.154:0/2288557327 reap_dead start 2023-09-23T03:21:36.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.361+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55649bd9ac00 msgr2=0x55649be88000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.361+0000 7f5f7db6e5c0 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55649bd9ac00 0x55649be88000 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x55649bd325a0 tx=0x55649c7c55c0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.361+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55649bd84c00 msgr2=0x55649baa0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.361+0000 7f5f7db6e5c0 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55649bd84c00 0x55649baa0c00 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x55649bd33800 tx=0x55649bd38de0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.362+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 shutdown_connections 2023-09-23T03:21:36.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.362+0000 7f5f7db6e5c0 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55649bd9ac00 0x55649be88000 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.362+0000 7f5f7db6e5c0 1 --2- 172.21.15.154:0/2288557327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55649bd84c00 0x55649baa0c00 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.362+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 >> 172.21.15.154:0/2288557327 conn(0x55649bb11c00 msgr2=0x55649c7bf000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:36.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.362+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 shutdown_connections 2023-09-23T03:21:36.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.362+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/2288557327 wait complete. 2023-09-23T03:21:36.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.363+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/354970918 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55649bb10000 msgr2=0x55649ba9f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.363+0000 7f5f7db6e5c0 1 --2- 172.21.15.154:0/354970918 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55649bb10000 0x55649ba9f080 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x55649c840900 tx=0x55649bd1e120 comp rx=0 tx=0).stop 2023-09-23T03:21:36.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.364+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/354970918 shutdown_connections 2023-09-23T03:21:36.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.364+0000 7f5f7db6e5c0 1 --2- 172.21.15.154:0/354970918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55649bb11400 0x55649ba9fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.364+0000 7f5f7db6e5c0 1 --2- 172.21.15.154:0/354970918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55649bb11000 0x55649ba9eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.364+0000 7f5f7db6e5c0 1 --2- 172.21.15.154:0/354970918 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55649bb10000 0x55649ba9f080 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.364+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/354970918 >> 172.21.15.154:0/354970918 conn(0x55649ba35000 msgr2=0x55649c7be000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:36.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.364+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/354970918 shutdown_connections 2023-09-23T03:21:36.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.364+0000 7f5f7db6e5c0 1 -- 172.21.15.154:0/354970918 wait complete. 2023-09-23T03:21:36.540 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.504162, result: 2023-09-23T03:21:36.540 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-09-23T03:21:36.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.768+0000 7f68cfb7c640 1 Processor -- start 2023-09-23T03:21:36.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.768+0000 7f68cfb7c640 1 -- start start 2023-09-23T03:21:36.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.769+0000 7f68cfb7c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68c80acce0 0x7f68c815ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:36.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.769+0000 7f68cfb7c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f68c80cf040 0x7f68c80ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:36.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.769+0000 7f68cfb7c640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68c815ac30 con 0x7f68c80b1b20 2023-09-23T03:21:36.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.769+0000 7f68cfb7c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68c80b2e90 con 0x7f68c80cf040 2023-09-23T03:21:36.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.769+0000 7f68cfb7c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68c80b1f70 con 0x7f68c80acce0 2023-09-23T03:21:36.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.770+0000 7f68ce119640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f68c80cf040 0x7f68c80ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:36.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.770+0000 7f68ce119640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f68c80cf040 0x7f68c80ab140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60890/0 (socket says 172.21.15.154:60890) 2023-09-23T03:21:36.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.770+0000 7f68ce119640 1 -- 172.21.15.154:0/2591923811 learned_addr learned my addr 172.21.15.154:0/2591923811 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:36.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.770+0000 7f68cd918640 1 --2- 172.21.15.154:0/2591923811 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68c80acce0 0x7f68c815ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:36.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.770+0000 7f68ce119640 1 -- 172.21.15.154:0/2591923811 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68c80acce0 msgr2=0x7f68c815ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:36.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.770+0000 7f68ce119640 1 --2- 172.21.15.154:0/2591923811 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68c80acce0 0x7f68c815ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.770+0000 7f68ce119640 1 -- 172.21.15.154:0/2591923811 >> v1:172.21.15.190:6789/0 conn(0x7f68c80b1b20 legacy=0x7f68c80ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:36.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.770+0000 7f68ce119640 1 -- 172.21.15.154:0/2591923811 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c815f2a0 con 0x7f68c80cf040 2023-09-23T03:21:36.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.771+0000 7f68ce119640 1 --2- 172.21.15.154:0/2591923811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68c80cf040 0x7f68c80ab140 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f68c0009e50 tx=0x7f68c002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d3f144a1ba017d1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:36.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.772+0000 7f68cc916640 1 -- 172.21.15.154:0/2591923811 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c002c9e0 con 0x7f68c80cf040 2023-09-23T03:21:36.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.772+0000 7f68cc916640 1 -- 172.21.15.154:0/2591923811 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c002cb40 con 0x7f68c80cf040 2023-09-23T03:21:36.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.772+0000 7f68cc916640 1 -- 172.21.15.154:0/2591923811 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c00343f0 con 0x7f68c80cf040 2023-09-23T03:21:36.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.772+0000 7f68cfb7c640 1 -- 172.21.15.154:0/2591923811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68c80cf040 msgr2=0x7f68c80ab140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.772+0000 7f68cfb7c640 1 --2- 172.21.15.154:0/2591923811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68c80cf040 0x7f68c80ab140 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f68c0009e50 tx=0x7f68c002be40 comp rx=0 tx=0).stop 2023-09-23T03:21:36.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.773+0000 7f68cfb7c640 1 -- 172.21.15.154:0/2591923811 shutdown_connections 2023-09-23T03:21:36.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.773+0000 7f68cfb7c640 1 --2- 172.21.15.154:0/2591923811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68c80cf040 0x7f68c80ab140 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.773+0000 7f68cfb7c640 1 --2- 172.21.15.154:0/2591923811 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68c80acce0 0x7f68c815ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.773+0000 7f68cfb7c640 1 -- 172.21.15.154:0/2591923811 >> 172.21.15.154:0/2591923811 conn(0x7f68c806a0f0 msgr2=0x7f68c806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:36.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.773+0000 7f68cfb7c640 1 -- 172.21.15.154:0/2591923811 shutdown_connections 2023-09-23T03:21:36.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.773+0000 7f68cfb7c640 1 -- 172.21.15.154:0/2591923811 wait complete. 2023-09-23T03:21:36.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.774+0000 7f68cfb7c640 1 Processor -- start 2023-09-23T03:21:36.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.774+0000 7f68cfb7c640 1 -- start start 2023-09-23T03:21:36.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.775+0000 7f68cfb7c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68c80acce0 0x7f68c80c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:36.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.775+0000 7f68cfb7c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68c80b1b20 0x7f68c81577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:36.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.775+0000 7f68cfb7c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68c80cf040 0x7f68c80c56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:36.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.775+0000 7f68cfb7c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c8157d20 con 0x7f68c80cf040 2023-09-23T03:21:36.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.775+0000 7f68cfb7c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c80cc450 con 0x7f68c80acce0 2023-09-23T03:21:36.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.775+0000 7f68cfb7c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68c80c6210 con 0x7f68c80b1b20 2023-09-23T03:21:36.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.776+0000 7f68ce119640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68c80cf040 0x7f68c80c56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:36.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.776+0000 7f68ce119640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68c80cf040 0x7f68c80c56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60906/0 (socket says 172.21.15.154:60906) 2023-09-23T03:21:36.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.776+0000 7f68ce119640 1 -- 172.21.15.154:0/56685314 learned_addr learned my addr 172.21.15.154:0/56685314 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:36.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.776+0000 7f68ce119640 1 -- 172.21.15.154:0/56685314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68c80b1b20 msgr2=0x7f68c81577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.776+0000 7f68ce119640 1 --2- 172.21.15.154:0/56685314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68c80b1b20 0x7f68c81577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.776+0000 7f68ce119640 1 -- 172.21.15.154:0/56685314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68c80acce0 msgr2=0x7f68c80c72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:36.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.776+0000 7f68ce119640 1 --2- 172.21.15.154:0/56685314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68c80acce0 0x7f68c80c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:36.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.776+0000 7f68ce119640 1 -- 172.21.15.154:0/56685314 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c0009b30 con 0x7f68c80cf040 2023-09-23T03:21:36.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.776+0000 7f68ce119640 1 --2- 172.21.15.154:0/56685314 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68c80cf040 0x7f68c80c56c0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f68c0036040 tx=0x7f68c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:36.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.777+0000 7f68c67fc640 1 -- 172.21.15.154:0/56685314 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c0009510 con 0x7f68c80cf040 2023-09-23T03:21:36.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.777+0000 7f68c67fc640 1 -- 172.21.15.154:0/56685314 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c003e450 con 0x7f68c80cf040 2023-09-23T03:21:36.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.777+0000 7f68c67fc640 1 -- 172.21.15.154:0/56685314 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c003e7e0 con 0x7f68c80cf040 2023-09-23T03:21:36.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.777+0000 7f68cfb7c640 1 -- 172.21.15.154:0/56685314 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c80c64f0 con 0x7f68c80cf040 2023-09-23T03:21:36.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.777+0000 7f68cfb7c640 1 -- 172.21.15.154:0/56685314 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c80c6ab0 con 0x7f68c80cf040 2023-09-23T03:21:36.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.778+0000 7f68cfb7c640 1 -- 172.21.15.154:0/56685314 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68c8073eb0 con 0x7f68c80cf040 2023-09-23T03:21:36.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.779+0000 7f68c67fc640 1 -- 172.21.15.154:0/56685314 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f68c003e940 con 0x7f68c80cf040 2023-09-23T03:21:36.782 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.781+0000 7f68c67fc640 1 --2- 172.21.15.154:0/56685314 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f68b406af60 0x7f68b406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:36.782 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.781+0000 7f68c67fc640 1 -- 172.21.15.154:0/56685314 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x7f68c00ac3f0 con 0x7f68c80cf040 2023-09-23T03:21:36.782 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.781+0000 7f68cd918640 1 --2- 172.21.15.154:0/56685314 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f68b406af60 0x7f68b406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:36.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.782+0000 7f68cd918640 1 --2- 172.21.15.154:0/56685314 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f68b406af60 0x7f68b406d420 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f68b8006e20 tx=0x7f68b8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:36.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:36.786+0000 7f68c67fc640 1 -- 172.21.15.154:0/56685314 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f68c00770b0 con 0x7f68c80cf040 2023-09-23T03:21:37.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.047+0000 7f68cfb7c640 1 -- 172.21.15.154:0/56685314 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f68c815ac30 con 0x7f68c80cf040 2023-09-23T03:21:37.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.271+0000 7f68c67fc640 1 -- 172.21.15.154:0/56685314 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 3:0 v48) v1 ==== 101+0+0 (secure 0 0 0) 0x7f68c815ac30 con 0x7f68c80cf040 2023-09-23T03:21:37.273 INFO:teuthology.orchestra.run.smithi154.stderr:repaired: restoring rank 3:0 2023-09-23T03:21:37.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.275+0000 7f68cfb7c640 1 -- 172.21.15.154:0/56685314 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f68b406af60 msgr2=0x7f68b406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:37.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.275+0000 7f68cfb7c640 1 --2- 172.21.15.154:0/56685314 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f68b406af60 0x7f68b406d420 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f68b8006e20 tx=0x7f68b8008040 comp rx=0 tx=0).stop 2023-09-23T03:21:37.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.275+0000 7f68cfb7c640 1 -- 172.21.15.154:0/56685314 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68c80cf040 msgr2=0x7f68c80c56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:37.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.275+0000 7f68cfb7c640 1 --2- 172.21.15.154:0/56685314 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68c80cf040 0x7f68c80c56c0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f68c0036040 tx=0x7f68c0004a40 comp rx=0 tx=0).stop 2023-09-23T03:21:37.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.276+0000 7f68cfb7c640 1 -- 172.21.15.154:0/56685314 shutdown_connections 2023-09-23T03:21:37.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.276+0000 7f68cfb7c640 1 --2- 172.21.15.154:0/56685314 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f68b406af60 0x7f68b406d420 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:37.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.276+0000 7f68cfb7c640 1 --2- 172.21.15.154:0/56685314 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68c80cf040 0x7f68c80c56c0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:37.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.276+0000 7f68cfb7c640 1 --2- 172.21.15.154:0/56685314 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68c80b1b20 0x7f68c81577e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:37.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.276+0000 7f68cfb7c640 1 --2- 172.21.15.154:0/56685314 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68c80acce0 0x7f68c80c72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:37.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.276+0000 7f68cfb7c640 1 -- 172.21.15.154:0/56685314 >> 172.21.15.154:0/56685314 conn(0x7f68c806a0f0 msgr2=0x7f68c80b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:37.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.276+0000 7f68cfb7c640 1 -- 172.21.15.154:0/56685314 shutdown_connections 2023-09-23T03:21:37.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.276+0000 7f68cfb7c640 1 -- 172.21.15.154:0/56685314 wait complete. 2023-09-23T03:21:37.284 INFO:tasks.ceph.mds.c.smithi154.stderr:2023-09-23T03:21:37.283+0000 7ff143f22640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-09-23T03:21:37.299 INFO:tasks.ceph.mds.a:Restarting daemon 2023-09-23T03:21:37.299 INFO:tasks.ceph.mds.a:Stopping old one... 2023-09-23T03:21:37.299 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-09-23T03:21:37.299 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:21:37.299 INFO:tasks.ceph.mds.c:Restarting daemon 2023-09-23T03:21:37.300 INFO:tasks.ceph.mds.c:Stopping old one... 2023-09-23T03:21:37.300 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-09-23T03:21:37.300 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:21:37.300 INFO:tasks.ceph.mds.b:Restarting daemon 2023-09-23T03:21:37.300 INFO:tasks.ceph.mds.b:Stopping old one... 2023-09-23T03:21:37.300 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-09-23T03:21:37.301 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:21:37.301 INFO:tasks.ceph.mds.d:Restarting daemon 2023-09-23T03:21:37.301 INFO:tasks.ceph.mds.d:Stopping old one... 2023-09-23T03:21:37.301 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-09-23T03:21:37.301 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:21:37.316 INFO:tasks.ceph.mds.a:Stopped 2023-09-23T03:21:37.316 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-09-23T03:21:37.317 INFO:tasks.ceph.mds.c:Stopped 2023-09-23T03:21:37.317 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-09-23T03:21:37.324 INFO:tasks.ceph.mds.b:Stopped 2023-09-23T03:21:37.324 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-09-23T03:21:37.325 INFO:tasks.ceph.mds.d:Stopped 2023-09-23T03:21:37.325 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-09-23T03:21:37.365 INFO:tasks.ceph.mds.a:Started 2023-09-23T03:21:37.365 DEBUG:teuthology.parallel:result is None 2023-09-23T03:21:37.374 INFO:tasks.ceph.mds.c:Started 2023-09-23T03:21:37.374 DEBUG:teuthology.parallel:result is None 2023-09-23T03:21:37.375 INFO:tasks.ceph.mds.b:Started 2023-09-23T03:21:37.375 DEBUG:teuthology.parallel:result is None 2023-09-23T03:21:37.382 INFO:tasks.ceph.mds.d:Started 2023-09-23T03:21:37.383 DEBUG:teuthology.parallel:result is None 2023-09-23T03:21:37.383 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:21:37.525 INFO:tasks.ceph.mds.a.smithi154.stdout:starting mds.a at 2023-09-23T03:21:37.527 INFO:tasks.ceph.mds.c.smithi154.stdout:starting mds.c at 2023-09-23T03:21:37.538 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-09-23T03:21:37.545 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-09-23T03:21:37.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.567+0000 7f67f4ccf640 1 Processor -- start 2023-09-23T03:21:37.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.567+0000 7f67f4ccf640 1 -- start start 2023-09-23T03:21:37.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.568+0000 7f67f4ccf640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f67f015cf40 0x7f67f01615b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:37.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.568+0000 7f67f4ccf640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f67f006b1e0 0x7f67f00cca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:37.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.568+0000 7f67f4ccf640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67f00b21d0 0x7f67f00b2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:37.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.569+0000 7f67f4ccf640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67f00b2b80 con 0x7f67f006b1e0 2023-09-23T03:21:37.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.569+0000 7f67f4ccf640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67f00ccf80 con 0x7f67f015cf40 2023-09-23T03:21:37.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.569+0000 7f67e7fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f67f006b1e0 0x7f67f00cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:37.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.569+0000 7f67f4ccf640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67f00cd1c0 con 0x7f67f00b21d0 2023-09-23T03:21:37.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.569+0000 7f67e7fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f67f006b1e0 0x7f67f00cca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60942/0 (socket says 172.21.15.154:60942) 2023-09-23T03:21:37.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.569+0000 7f67e7fff640 1 -- 172.21.15.154:0/1965911021 learned_addr learned my addr 172.21.15.154:0/1965911021 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:37.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.569+0000 7f67eed9d640 1 --2- 172.21.15.154:0/1965911021 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67f00b21d0 0x7f67f00b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:37.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.569+0000 7f67ee59c640 1 --2- 172.21.15.154:0/1965911021 >> v2:172.21.15.190:3300/0 conn(0x7f67f015cf40 0x7f67f01615b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:37.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.569+0000 7f67e7fff640 1 -- 172.21.15.154:0/1965911021 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67f00b21d0 msgr2=0x7f67f00b2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:37.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.569+0000 7f67e7fff640 1 --2- 172.21.15.154:0/1965911021 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67f00b21d0 0x7f67f00b2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:37.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.569+0000 7f67e7fff640 1 -- 172.21.15.154:0/1965911021 >> v2:172.21.15.190:3300/0 conn(0x7f67f015cf40 msgr2=0x7f67f01615b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:37.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.569+0000 7f67e7fff640 1 --2- 172.21.15.154:0/1965911021 >> v2:172.21.15.190:3300/0 conn(0x7f67f015cf40 0x7f67f01615b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:37.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.569+0000 7f67e7fff640 1 -- 172.21.15.154:0/1965911021 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f00cd4a0 con 0x7f67f006b1e0 2023-09-23T03:21:37.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.570+0000 7f67e7fff640 1 --2- 172.21.15.154:0/1965911021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67f006b1e0 0x7f67f00cca40 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f67f006f200 tx=0x7f67d802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d9460027a077567 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:37.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.571+0000 7f67edd9b640 1 -- 172.21.15.154:0/1965911021 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d8031870 con 0x7f67f006b1e0 2023-09-23T03:21:37.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.571+0000 7f67edd9b640 1 -- 172.21.15.154:0/1965911021 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d80319d0 con 0x7f67f006b1e0 2023-09-23T03:21:37.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.571+0000 7f67edd9b640 1 -- 172.21.15.154:0/1965911021 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d8031d60 con 0x7f67f006b1e0 2023-09-23T03:21:37.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.571+0000 7f67f4ccf640 1 -- 172.21.15.154:0/1965911021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67f006b1e0 msgr2=0x7f67f00cca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:37.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.571+0000 7f67f4ccf640 1 --2- 172.21.15.154:0/1965911021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67f006b1e0 0x7f67f00cca40 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f67f006f200 tx=0x7f67d802b910 comp rx=0 tx=0).stop 2023-09-23T03:21:37.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.571+0000 7f67f4ccf640 1 -- 172.21.15.154:0/1965911021 shutdown_connections 2023-09-23T03:21:37.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.571+0000 7f67f4ccf640 1 --2- 172.21.15.154:0/1965911021 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67f00b21d0 0x7f67f00b2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:37.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.571+0000 7f67f4ccf640 1 --2- 172.21.15.154:0/1965911021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67f006b1e0 0x7f67f00cca40 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:37.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.571+0000 7f67f4ccf640 1 --2- 172.21.15.154:0/1965911021 >> v2:172.21.15.190:3300/0 conn(0x7f67f015cf40 0x7f67f01615b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:37.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.571+0000 7f67f4ccf640 1 -- 172.21.15.154:0/1965911021 >> 172.21.15.154:0/1965911021 conn(0x7f67f006fc60 msgr2=0x7f67f0070040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:37.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.571+0000 7f67f4ccf640 1 -- 172.21.15.154:0/1965911021 shutdown_connections 2023-09-23T03:21:37.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.571+0000 7f67f4ccf640 1 -- 172.21.15.154:0/1965911021 wait complete. 2023-09-23T03:21:37.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.573+0000 7f67f4ccf640 1 Processor -- start 2023-09-23T03:21:37.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.573+0000 7f67f4ccf640 1 -- start start 2023-09-23T03:21:37.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.574+0000 7f67f4ccf640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67f006b1e0 0x7f67f014bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:37.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.574+0000 7f67f4ccf640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67f00b21d0 0x7f67f015f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:37.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.574+0000 7f67f4ccf640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67f015cf40 0x7f67f015fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:37.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.574+0000 7f67f4ccf640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f014c130 con 0x7f67f00b21d0 2023-09-23T03:21:37.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.574+0000 7f67f4ccf640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f00c5e00 con 0x7f67f015cf40 2023-09-23T03:21:37.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.574+0000 7f67f4ccf640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67f00c60e0 con 0x7f67f006b1e0 2023-09-23T03:21:37.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.575+0000 7f67e7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67f00b21d0 0x7f67f015f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:37.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.575+0000 7f67e7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67f00b21d0 0x7f67f015f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60956/0 (socket says 172.21.15.154:60956) 2023-09-23T03:21:37.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.575+0000 7f67e7fff640 1 -- 172.21.15.154:0/3134689552 learned_addr learned my addr 172.21.15.154:0/3134689552 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:37.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.575+0000 7f67ee59c640 1 --2- 172.21.15.154:0/3134689552 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67f006b1e0 0x7f67f014bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:37.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.575+0000 7f67eed9d640 1 --2- 172.21.15.154:0/3134689552 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67f015cf40 0x7f67f015fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:37.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.575+0000 7f67e7fff640 1 -- 172.21.15.154:0/3134689552 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67f006b1e0 msgr2=0x7f67f014bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:37.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.575+0000 7f67e7fff640 1 --2- 172.21.15.154:0/3134689552 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67f006b1e0 0x7f67f014bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:37.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.575+0000 7f67e7fff640 1 -- 172.21.15.154:0/3134689552 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67f015cf40 msgr2=0x7f67f015fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:37.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.575+0000 7f67e7fff640 1 --2- 172.21.15.154:0/3134689552 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67f015cf40 0x7f67f015fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:37.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.575+0000 7f67e7fff640 1 -- 172.21.15.154:0/3134689552 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d80094d0 con 0x7f67f00b21d0 2023-09-23T03:21:37.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.575+0000 7f67e7fff640 1 --2- 172.21.15.154:0/3134689552 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67f00b21d0 0x7f67f015f6a0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f67d802b950 tx=0x7f67d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:37.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.576+0000 7f67e6ffd640 1 -- 172.21.15.154:0/3134689552 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d803ac60 con 0x7f67f00b21d0 2023-09-23T03:21:37.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.576+0000 7f67e6ffd640 1 -- 172.21.15.154:0/3134689552 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d803adc0 con 0x7f67f00b21d0 2023-09-23T03:21:37.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.576+0000 7f67e6ffd640 1 -- 172.21.15.154:0/3134689552 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d8044630 con 0x7f67f00b21d0 2023-09-23T03:21:37.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.576+0000 7f67f4ccf640 1 -- 172.21.15.154:0/3134689552 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67f00c63c0 con 0x7f67f00b21d0 2023-09-23T03:21:37.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.576+0000 7f67f4ccf640 1 -- 172.21.15.154:0/3134689552 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67f00c6980 con 0x7f67f00b21d0 2023-09-23T03:21:37.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.577+0000 7f67f4ccf640 1 -- 172.21.15.154:0/3134689552 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67f0073eb0 con 0x7f67f00b21d0 2023-09-23T03:21:37.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.578+0000 7f67e6ffd640 1 -- 172.21.15.154:0/3134689552 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f67d8044790 con 0x7f67f00b21d0 2023-09-23T03:21:37.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.579+0000 7f67e6ffd640 1 --2- 172.21.15.154:0/3134689552 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f67d006af30 0x7f67d006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:37.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.579+0000 7f67e6ffd640 1 -- 172.21.15.154:0/3134689552 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6605+0+0 (secure 0 0 0) 0x7f67d802d040 con 0x7f67f00b21d0 2023-09-23T03:21:37.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.579+0000 7f67ee59c640 1 --2- 172.21.15.154:0/3134689552 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f67d006af30 0x7f67d006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:37.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.580+0000 7f67ee59c640 1 --2- 172.21.15.154:0/3134689552 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f67d006af30 0x7f67d006d3f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f67f00cbc20 tx=0x7f67e0009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:37.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.583+0000 7f67e6ffd640 1 -- 172.21.15.154:0/3134689552 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f67d8076de0 con 0x7f67f00b21d0 2023-09-23T03:21:37.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.804+0000 7f67f4ccf640 1 -- 172.21.15.154:0/3134689552 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f67f006afc0 con 0x7f67f00b21d0 2023-09-23T03:21:37.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.805+0000 7f67e6ffd640 1 -- 172.21.15.154:0/3134689552 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4929 (secure 0 0 0) 0x7f67d8035020 con 0x7f67f00b21d0 2023-09-23T03:21:37.806 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:37.807 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":49,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5027,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/2517324433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":2517324433},{"type":"v1","addr":"172.21.15.190:6833","nonce":2517324433}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":5033,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/4098915995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":4098915995},{"type":"v1","addr":"172.21.15.190:6835","nonce":4098915995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6309,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/3034894009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":3034894009},{"type":"v1","addr":"172.21.15.154:6836","nonce":3034894009}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:21:30.160245+0000","modified":"2023-09-23T03:21:37.275541+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6639},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6639":{"gid":6639,"name":"c","rank":0,"incarnation":49,"state":"up:replay","state_seq":1,"addr":"172.21.15.154:6837/4129414148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":4129414148},{"type":"v1","addr":"172.21.15.154:6837","nonce":4129414148}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9,10],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-09-23T03:21:37.807 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 49 2023-09-23T03:21:37.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.807+0000 7f67f4ccf640 1 -- 172.21.15.154:0/3134689552 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f67d006af30 msgr2=0x7f67d006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:37.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.807+0000 7f67f4ccf640 1 --2- 172.21.15.154:0/3134689552 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f67d006af30 0x7f67d006d3f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f67f00cbc20 tx=0x7f67e0009190 comp rx=0 tx=0).stop 2023-09-23T03:21:37.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.807+0000 7f67f4ccf640 1 -- 172.21.15.154:0/3134689552 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67f00b21d0 msgr2=0x7f67f015f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:37.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.807+0000 7f67f4ccf640 1 --2- 172.21.15.154:0/3134689552 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67f00b21d0 0x7f67f015f6a0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f67d802b950 tx=0x7f67d80046c0 comp rx=0 tx=0).stop 2023-09-23T03:21:37.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.807+0000 7f67f4ccf640 1 -- 172.21.15.154:0/3134689552 shutdown_connections 2023-09-23T03:21:37.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.807+0000 7f67f4ccf640 1 --2- 172.21.15.154:0/3134689552 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f67d006af30 0x7f67d006d3f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:37.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.808+0000 7f67f4ccf640 1 --2- 172.21.15.154:0/3134689552 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67f015cf40 0x7f67f015fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:37.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.808+0000 7f67f4ccf640 1 --2- 172.21.15.154:0/3134689552 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67f00b21d0 0x7f67f015f6a0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:37.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.808+0000 7f67f4ccf640 1 --2- 172.21.15.154:0/3134689552 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67f006b1e0 0x7f67f014bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:37.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.808+0000 7f67f4ccf640 1 -- 172.21.15.154:0/3134689552 >> 172.21.15.154:0/3134689552 conn(0x7f67f006fc60 msgr2=0x7f67f006b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:37.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.808+0000 7f67f4ccf640 1 -- 172.21.15.154:0/3134689552 shutdown_connections 2023-09-23T03:21:37.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:37.808+0000 7f67f4ccf640 1 -- 172.21.15.154:0/3134689552 wait complete. 2023-09-23T03:21:38.229 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 49, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-23T03:21:30.160245+0000', 'modified': '2023-09-23T03:21:37.275541+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 67, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6639}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6639': {'gid': 6639, 'name': 'c', 'rank': 0, 'incarnation': 49, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.154:6837/4129414148', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6835', 'nonce': 4129414148}, {'type': 'v1', 'addr': '172.21.15.154:6837', 'nonce': 4129414148}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9, 10], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-23T03:21:38.229 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6639:up:replay 2023-09-23T03:21:38.279 INFO:tasks.ceph.mon.a.smithi154.stderr:2023-09-23T03:21:38.277+0000 7f3df0da1640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-23T03:21:38.306 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:21:38.305+0000 7fabd8110640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-09-23T03:21:39.230 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:21:39.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.451+0000 7f64b2690640 1 Processor -- start 2023-09-23T03:21:39.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.452+0000 7f64b2690640 1 -- start start 2023-09-23T03:21:39.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.453+0000 7f64b2690640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f64ac160f90 0x7f64ac167950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:39.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.453+0000 7f64b2690640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64ac161400 con 0x7f64ac15caf0 2023-09-23T03:21:39.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.453+0000 7f64b2690640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64ac0cd530 con 0x7f64ac0cf040 2023-09-23T03:21:39.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.453+0000 7f64b2690640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64ac15ced0 con 0x7f64ac160f90 2023-09-23T03:21:39.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.453+0000 7f64abfff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f64ac15caf0 0x7f64ac0ce930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:38364/0 (socket says 172.21.15.154:38364) 2023-09-23T03:21:39.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.453+0000 7f64abfff640 1 -- 172.21.15.154:0/3811957594 learned_addr learned my addr 172.21.15.154:0/3811957594 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:21:39.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.454+0000 7f64aaffd640 1 -- 172.21.15.154:0/3811957594 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4212590753 0 0) 0x7f649c001610 con 0x7f64ac15caf0 2023-09-23T03:21:39.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.454+0000 7f64aaffd640 1 -- 172.21.15.154:0/3811957594 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6498003520 con 0x7f64ac15caf0 2023-09-23T03:21:39.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.454+0000 7f64b0c2d640 1 --2- 172.21.15.154:0/3811957594 >> v2:172.21.15.190:3300/0 conn(0x7f64ac160f90 0x7f64ac167950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:39.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.454+0000 7f64aaffd640 1 -- 172.21.15.154:0/3811957594 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1227039981 0 0) 0x7f6498003520 con 0x7f64ac15caf0 2023-09-23T03:21:39.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.454+0000 7f64aaffd640 1 -- 172.21.15.154:0/3811957594 >> v2:172.21.15.190:3300/0 conn(0x7f64ac160f90 msgr2=0x7f64ac167950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:39.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.454+0000 7f64aaffd640 1 --2- 172.21.15.154:0/3811957594 >> v2:172.21.15.190:3300/0 conn(0x7f64ac160f90 0x7f64ac167950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:39.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.454+0000 7f64aaffd640 1 -- 172.21.15.154:0/3811957594 >> v1:172.21.15.190:6789/0 conn(0x7f64ac0cf040 legacy=0x7f64ac160880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:39.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.454+0000 7f64aaffd640 1 -- 172.21.15.154:0/3811957594 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ac0cd810 con 0x7f64ac15caf0 2023-09-23T03:21:39.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.454+0000 7f64aaffd640 1 -- 172.21.15.154:0/3811957594 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f649c001d00 con 0x7f64ac15caf0 2023-09-23T03:21:39.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.454+0000 7f64aaffd640 1 -- 172.21.15.154:0/3811957594 >> v1:172.21.15.154:6789/0 conn(0x7f64ac15caf0 legacy=0x7f64ac0ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:39.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.454+0000 7f64aaffd640 1 --2- 172.21.15.154:0/3811957594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64980039c0 0x7f6498003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:39.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.454+0000 7f64aaffd640 1 --2- 172.21.15.154:0/3811957594 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64980047c0 0x7f6498004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:39.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.454+0000 7f64aaffd640 1 --2- 172.21.15.154:0/3811957594 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64980051d0 0x7f649800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:39.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.454+0000 7f64aaffd640 1 -- 172.21.15.154:0/3811957594 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac15ced0 con 0x7f64980051d0 2023-09-23T03:21:39.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.454+0000 7f64aaffd640 1 -- 172.21.15.154:0/3811957594 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649800c100 con 0x7f64980039c0 2023-09-23T03:21:39.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.454+0000 7f64aaffd640 1 -- 172.21.15.154:0/3811957594 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f649800c2c0 con 0x7f64980047c0 2023-09-23T03:21:39.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.455+0000 7f64b0c2d640 1 --2- 172.21.15.154:0/3811957594 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64980051d0 0x7f649800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:39.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.455+0000 7f64abfff640 1 --2- 172.21.15.154:0/3811957594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64980039c0 0x7f6498003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:39.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.455+0000 7f64ab7fe640 1 --2- 172.21.15.154:0/3811957594 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64980047c0 0x7f6498004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:39.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.455+0000 7f64b0c2d640 1 -- 172.21.15.154:0/3811957594 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64980047c0 msgr2=0x7f6498004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:39.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.455+0000 7f64b0c2d640 1 --2- 172.21.15.154:0/3811957594 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64980047c0 0x7f6498004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:39.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.455+0000 7f64b0c2d640 1 -- 172.21.15.154:0/3811957594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64980039c0 msgr2=0x7f6498003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:39.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.455+0000 7f64b0c2d640 1 --2- 172.21.15.154:0/3811957594 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64980039c0 0x7f6498003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:39.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.455+0000 7f64b0c2d640 1 -- 172.21.15.154:0/3811957594 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6498003520 con 0x7f64980051d0 2023-09-23T03:21:39.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.455+0000 7f64ab7fe640 1 --2- 172.21.15.154:0/3811957594 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64980047c0 0x7f6498004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:39.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.456+0000 7f64b0c2d640 1 -- 172.21.15.154:0/3811957594 reap_dead start 2023-09-23T03:21:39.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.456+0000 7f64b0c2d640 1 --2- 172.21.15.154:0/3811957594 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64980051d0 0x7f649800b980 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f64a400ba00 tx=0x7f64980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df21645796037ece server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:39.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.456+0000 7f64aaffd640 1 -- 172.21.15.154:0/3811957594 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a4004020 con 0x7f64980051d0 2023-09-23T03:21:39.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.457+0000 7f64aaffd640 1 -- 172.21.15.154:0/3811957594 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a4004180 con 0x7f64980051d0 2023-09-23T03:21:39.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.457+0000 7f64aaffd640 1 -- 172.21.15.154:0/3811957594 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a4003610 con 0x7f64980051d0 2023-09-23T03:21:39.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.457+0000 7f64b2690640 1 -- 172.21.15.154:0/3811957594 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64980051d0 msgr2=0x7f649800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:39.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.457+0000 7f64b2690640 1 --2- 172.21.15.154:0/3811957594 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64980051d0 0x7f649800b980 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f64a400ba00 tx=0x7f64980057f0 comp rx=0 tx=0).stop 2023-09-23T03:21:39.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.457+0000 7f64b2690640 1 -- 172.21.15.154:0/3811957594 shutdown_connections 2023-09-23T03:21:39.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.457+0000 7f64b2690640 1 --2- 172.21.15.154:0/3811957594 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64980051d0 0x7f649800b980 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:39.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.457+0000 7f64b2690640 1 -- 172.21.15.154:0/3811957594 >> 172.21.15.154:0/3811957594 conn(0x7f64ac06a0f0 msgr2=0x7f64ac06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:39.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.457+0000 7f64b2690640 1 -- 172.21.15.154:0/3811957594 shutdown_connections 2023-09-23T03:21:39.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.457+0000 7f64b2690640 1 -- 172.21.15.154:0/3811957594 wait complete. 2023-09-23T03:21:39.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.459+0000 7f64b2690640 1 Processor -- start 2023-09-23T03:21:39.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.459+0000 7f64b2690640 1 -- start start 2023-09-23T03:21:39.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.460+0000 7f64b2690640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64980051d0 0x7f64ac153fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:39.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.460+0000 7f64b2690640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64ac0c4690 0x7f64ac0c4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:39.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.460+0000 7f64b2690640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64ac071190 0x7f64ac071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:39.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.460+0000 7f64b2690640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac071b60 con 0x7f64980051d0 2023-09-23T03:21:39.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.460+0000 7f64b2690640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac1544f0 con 0x7f64ac071190 2023-09-23T03:21:39.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.460+0000 7f64b2690640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac154910 con 0x7f64ac0c4690 2023-09-23T03:21:39.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.461+0000 7f64abfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64980051d0 0x7f64ac153fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:39.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.461+0000 7f64abfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64980051d0 0x7f64ac153fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60976/0 (socket says 172.21.15.154:60976) 2023-09-23T03:21:39.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.461+0000 7f64abfff640 1 -- 172.21.15.154:0/3739537179 learned_addr learned my addr 172.21.15.154:0/3739537179 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:39.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.461+0000 7f64ab7fe640 1 --2- 172.21.15.154:0/3739537179 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64ac0c4690 0x7f64ac0c4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:39.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.461+0000 7f64b0c2d640 1 --2- 172.21.15.154:0/3739537179 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64ac071190 0x7f64ac071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:39.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.461+0000 7f64abfff640 1 -- 172.21.15.154:0/3739537179 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64ac0c4690 msgr2=0x7f64ac0c4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:39.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.461+0000 7f64abfff640 1 --2- 172.21.15.154:0/3739537179 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64ac0c4690 0x7f64ac0c4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:39.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.461+0000 7f64abfff640 1 -- 172.21.15.154:0/3739537179 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64ac071190 msgr2=0x7f64ac071620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:39.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.461+0000 7f64abfff640 1 --2- 172.21.15.154:0/3739537179 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64ac071190 0x7f64ac071620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:39.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.461+0000 7f64abfff640 1 -- 172.21.15.154:0/3739537179 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ac070010 con 0x7f64980051d0 2023-09-23T03:21:39.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.461+0000 7f64b0c2d640 1 --2- 172.21.15.154:0/3739537179 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64ac071190 0x7f64ac071620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:39.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.462+0000 7f64abfff640 1 --2- 172.21.15.154:0/3739537179 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64980051d0 0x7f64ac153fb0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f649c00eaa0 tx=0x7f649c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:39.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.462+0000 7f64a8ff9640 1 -- 172.21.15.154:0/3739537179 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649c005d30 con 0x7f64980051d0 2023-09-23T03:21:39.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.462+0000 7f64a8ff9640 1 -- 172.21.15.154:0/3739537179 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f649c00c730 con 0x7f64980051d0 2023-09-23T03:21:39.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.462+0000 7f64a8ff9640 1 -- 172.21.15.154:0/3739537179 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649c00cae0 con 0x7f64980051d0 2023-09-23T03:21:39.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.462+0000 7f64b2690640 1 -- 172.21.15.154:0/3739537179 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64ac0702f0 con 0x7f64980051d0 2023-09-23T03:21:39.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.462+0000 7f64b2690640 1 -- 172.21.15.154:0/3739537179 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64ac0708b0 con 0x7f64980051d0 2023-09-23T03:21:39.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.463+0000 7f64b2690640 1 -- 172.21.15.154:0/3739537179 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64ac073eb0 con 0x7f64980051d0 2023-09-23T03:21:39.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.464+0000 7f64a8ff9640 1 -- 172.21.15.154:0/3739537179 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f649c00cc40 con 0x7f64980051d0 2023-09-23T03:21:39.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.465+0000 7f64a8ff9640 1 --2- 172.21.15.154:0/3739537179 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f648c06af60 0x7f648c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:39.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.465+0000 7f64a8ff9640 1 -- 172.21.15.154:0/3739537179 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f649c089b70 con 0x7f64980051d0 2023-09-23T03:21:39.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.465+0000 7f64ab7fe640 1 --2- 172.21.15.154:0/3739537179 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f648c06af60 0x7f648c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:39.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.466+0000 7f64ab7fe640 1 --2- 172.21.15.154:0/3739537179 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f648c06af60 0x7f648c06d420 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6494004e70 tx=0x7f6494009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:39.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.470+0000 7f64a8ff9640 1 -- 172.21.15.154:0/3739537179 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f649c054860 con 0x7f64980051d0 2023-09-23T03:21:39.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.701+0000 7f64b2690640 1 -- 172.21.15.154:0/3739537179 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64ac161400 con 0x7f64980051d0 2023-09-23T03:21:39.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.703+0000 7f64a8ff9640 1 -- 172.21.15.154:0/3739537179 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4929 (secure 0 0 0) 0x7f64ac161400 con 0x7f64980051d0 2023-09-23T03:21:39.704 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:39.704 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":52,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5279,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1609915000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1609915000},{"type":"v1","addr":"172.21.15.154:6837","nonce":1609915000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":5288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1240415394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1240415394},{"type":"v1","addr":"172.21.15.190:6835","nonce":1240415394}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":5728,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/878668736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":878668736},{"type":"v1","addr":"172.21.15.190:6833","nonce":878668736}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:21:30.160245+0000","modified":"2023-09-23T03:21:38.322089+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6924},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6924":{"gid":6924,"name":"a","rank":0,"incarnation":51,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.154:6835/2633473586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2633473586},{"type":"v1","addr":"172.21.15.154:6835","nonce":2633473586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9,10],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-09-23T03:21:39.705 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 52 2023-09-23T03:21:39.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.705+0000 7f64b2690640 1 -- 172.21.15.154:0/3739537179 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f648c06af60 msgr2=0x7f648c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:39.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.705+0000 7f64b2690640 1 --2- 172.21.15.154:0/3739537179 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f648c06af60 0x7f648c06d420 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6494004e70 tx=0x7f6494009040 comp rx=0 tx=0).stop 2023-09-23T03:21:39.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.705+0000 7f64b2690640 1 -- 172.21.15.154:0/3739537179 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64980051d0 msgr2=0x7f64ac153fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:39.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.705+0000 7f64b2690640 1 --2- 172.21.15.154:0/3739537179 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64980051d0 0x7f64ac153fb0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f649c00eaa0 tx=0x7f649c00ef60 comp rx=0 tx=0).stop 2023-09-23T03:21:39.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.706+0000 7f64b2690640 1 -- 172.21.15.154:0/3739537179 shutdown_connections 2023-09-23T03:21:39.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.706+0000 7f64b2690640 1 --2- 172.21.15.154:0/3739537179 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f648c06af60 0x7f648c06d420 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:39.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.706+0000 7f64b2690640 1 --2- 172.21.15.154:0/3739537179 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f64ac071190 0x7f64ac071620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:39.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.706+0000 7f64b2690640 1 --2- 172.21.15.154:0/3739537179 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64ac0c4690 0x7f64ac0c4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:39.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.706+0000 7f64b2690640 1 --2- 172.21.15.154:0/3739537179 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64980051d0 0x7f64ac153fb0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:39.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.706+0000 7f64b2690640 1 -- 172.21.15.154:0/3739537179 >> 172.21.15.154:0/3739537179 conn(0x7f64ac06a0f0 msgr2=0x7f64ac06cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:39.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.706+0000 7f64b2690640 1 -- 172.21.15.154:0/3739537179 shutdown_connections 2023-09-23T03:21:39.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:39.706+0000 7f64b2690640 1 -- 172.21.15.154:0/3739537179 wait complete. 2023-09-23T03:21:40.128 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 52, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-23T03:21:30.160245+0000', 'modified': '2023-09-23T03:21:38.322089+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 69, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6924}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6924': {'gid': 6924, 'name': 'a', 'rank': 0, 'incarnation': 51, 'state': 'up:reconnect', 'state_seq': 2, 'addr': '172.21.15.154:6835/2633473586', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6834', 'nonce': 2633473586}, {'type': 'v1', 'addr': '172.21.15.154:6835', 'nonce': 2633473586}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9, 10], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-23T03:21:40.128 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6924:up:reconnect 2023-09-23T03:21:41.129 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:21:41.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.362+0000 7feeb369b640 1 Processor -- start 2023-09-23T03:21:41.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.363+0000 7feeb369b640 1 -- start start 2023-09-23T03:21:41.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.364+0000 7feeb369b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7feeac0cf2a0 0x7feeac0cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:41.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.364+0000 7feeb369b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feeac0cd670 0x7feeac0cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:41.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.364+0000 7feeb369b640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeac0ce990 con 0x7feeac15cac0 2023-09-23T03:21:41.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.364+0000 7feeb369b640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeac0cfc50 con 0x7feeac0cf2a0 2023-09-23T03:21:41.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.364+0000 7feeb369b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeac0ce3d0 con 0x7feeac0cd670 2023-09-23T03:21:41.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.364+0000 7feeb1437640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7feeac15cac0 0x7feeac06b280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:38378/0 (socket says 172.21.15.154:38378) 2023-09-23T03:21:41.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.364+0000 7feeb0c36640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7feeac0cf2a0 0x7feeac0cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:41.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.364+0000 7feeb1437640 1 -- 172.21.15.154:0/3254943101 learned_addr learned my addr 172.21.15.154:0/3254943101 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:21:41.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.365+0000 7feeb1c38640 1 --2- 172.21.15.154:0/3254943101 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feeac0cd670 0x7feeac0cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:41.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.365+0000 7feeb0c36640 1 -- 172.21.15.154:0/3254943101 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feeac0cd670 msgr2=0x7feeac0cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:41.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.365+0000 7feeb0c36640 1 --2- 172.21.15.154:0/3254943101 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feeac0cd670 0x7feeac0cdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:41.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.365+0000 7feeb0c36640 1 -- 172.21.15.154:0/3254943101 >> v1:172.21.15.154:6789/0 conn(0x7feeac15cac0 legacy=0x7feeac06b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:41.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.365+0000 7feeb0c36640 1 -- 172.21.15.154:0/3254943101 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeac0ce6b0 con 0x7feeac0cf2a0 2023-09-23T03:21:41.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.365+0000 7feea3fff640 1 -- 172.21.15.154:0/3254943101 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2906035644 0 0) 0x7feeac0ce990 con 0x7feeac15cac0 2023-09-23T03:21:41.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.366+0000 7feeb0c36640 1 --2- 172.21.15.154:0/3254943101 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feeac0cf2a0 0x7feeac0cf710 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fee9c0097a0 tx=0x7fee9c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b00a8cfa4933b1ad server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:41.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.366+0000 7feea3fff640 1 -- 172.21.15.154:0/3254943101 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9c031830 con 0x7feeac0cf2a0 2023-09-23T03:21:41.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.366+0000 7feea3fff640 1 -- 172.21.15.154:0/3254943101 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee9c031990 con 0x7feeac0cf2a0 2023-09-23T03:21:41.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.366+0000 7feea3fff640 1 -- 172.21.15.154:0/3254943101 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9c031d20 con 0x7feeac0cf2a0 2023-09-23T03:21:41.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.367+0000 7feeb369b640 1 -- 172.21.15.154:0/3254943101 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feeac0cf2a0 msgr2=0x7feeac0cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:41.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.367+0000 7feeb369b640 1 --2- 172.21.15.154:0/3254943101 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feeac0cf2a0 0x7feeac0cf710 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fee9c0097a0 tx=0x7fee9c02b8d0 comp rx=0 tx=0).stop 2023-09-23T03:21:41.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.367+0000 7feeb369b640 1 -- 172.21.15.154:0/3254943101 shutdown_connections 2023-09-23T03:21:41.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.367+0000 7feeb369b640 1 --2- 172.21.15.154:0/3254943101 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feeac0cd670 0x7feeac0cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:41.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.367+0000 7feeb369b640 1 --2- 172.21.15.154:0/3254943101 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feeac0cf2a0 0x7feeac0cf710 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:41.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.367+0000 7feeb369b640 1 -- 172.21.15.154:0/3254943101 >> 172.21.15.154:0/3254943101 conn(0x7feeac089fd0 msgr2=0x7feeac09bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:41.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.367+0000 7feeb369b640 1 -- 172.21.15.154:0/3254943101 shutdown_connections 2023-09-23T03:21:41.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.367+0000 7feeb369b640 1 -- 172.21.15.154:0/3254943101 wait complete. 2023-09-23T03:21:41.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.369+0000 7feeb369b640 1 Processor -- start 2023-09-23T03:21:41.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.369+0000 7feeb369b640 1 -- start start 2023-09-23T03:21:41.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.370+0000 7feeb369b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feeac0cd670 0x7feeac1f62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:41.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.370+0000 7feeb369b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feeac0cf2a0 0x7feeac0b0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:41.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.370+0000 7feeb369b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feeac15cac0 0x7feeac0c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:41.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.370+0000 7feeb369b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeac1f5ec0 con 0x7feeac0cf2a0 2023-09-23T03:21:41.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.370+0000 7feeb369b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeac15def0 con 0x7feeac0cd670 2023-09-23T03:21:41.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.370+0000 7feeb369b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeac15d930 con 0x7feeac15cac0 2023-09-23T03:21:41.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.370+0000 7feeb0c36640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feeac0cf2a0 0x7feeac0b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:41.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.370+0000 7feeb0c36640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feeac0cf2a0 0x7feeac0b0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60984/0 (socket says 172.21.15.154:60984) 2023-09-23T03:21:41.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.370+0000 7feeb0c36640 1 -- 172.21.15.154:0/528167238 learned_addr learned my addr 172.21.15.154:0/528167238 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:41.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.371+0000 7feeb1c38640 1 --2- 172.21.15.154:0/528167238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feeac15cac0 0x7feeac0c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:41.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.371+0000 7feeb1437640 1 --2- 172.21.15.154:0/528167238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feeac0cd670 0x7feeac1f62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:41.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.371+0000 7feeb0c36640 1 -- 172.21.15.154:0/528167238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feeac15cac0 msgr2=0x7feeac0c8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:41.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.371+0000 7feeb0c36640 1 --2- 172.21.15.154:0/528167238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feeac15cac0 0x7feeac0c8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:41.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.371+0000 7feeb0c36640 1 -- 172.21.15.154:0/528167238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feeac0cd670 msgr2=0x7feeac1f62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:41.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.371+0000 7feeb0c36640 1 --2- 172.21.15.154:0/528167238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feeac0cd670 0x7feeac1f62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:41.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.371+0000 7feeb0c36640 1 -- 172.21.15.154:0/528167238 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee9c009400 con 0x7feeac0cf2a0 2023-09-23T03:21:41.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.371+0000 7feeb1437640 1 --2- 172.21.15.154:0/528167238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feeac0cd670 0x7feeac1f62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:41.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.371+0000 7feeb0c36640 1 --2- 172.21.15.154:0/528167238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feeac0cf2a0 0x7feeac0b0250 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fee9c02bee0 tx=0x7fee9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:41.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.371+0000 7feea1ffb640 1 -- 172.21.15.154:0/528167238 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9c03a6f0 con 0x7feeac0cf2a0 2023-09-23T03:21:41.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.371+0000 7feea1ffb640 1 -- 172.21.15.154:0/528167238 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee9c03a870 con 0x7feeac0cf2a0 2023-09-23T03:21:41.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.371+0000 7feea1ffb640 1 -- 172.21.15.154:0/528167238 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9c03ac20 con 0x7feeac0cf2a0 2023-09-23T03:21:41.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.371+0000 7feeb369b640 1 -- 172.21.15.154:0/528167238 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeac15dc10 con 0x7feeac0cf2a0 2023-09-23T03:21:41.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.372+0000 7feeb369b640 1 -- 172.21.15.154:0/528167238 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeac15d600 con 0x7feeac0cf2a0 2023-09-23T03:21:41.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.373+0000 7feeb369b640 1 -- 172.21.15.154:0/528167238 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeac1f6000 con 0x7feeac0cf2a0 2023-09-23T03:21:41.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.374+0000 7feea1ffb640 1 -- 172.21.15.154:0/528167238 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fee9c030380 con 0x7feeac0cf2a0 2023-09-23T03:21:41.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.375+0000 7feea1ffb640 1 --2- 172.21.15.154:0/528167238 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fee9006af30 0x7fee9006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:41.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.375+0000 7feea1ffb640 1 -- 172.21.15.154:0/528167238 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fee9c0ab4f0 con 0x7feeac0cf2a0 2023-09-23T03:21:41.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.375+0000 7feeb1437640 1 --2- 172.21.15.154:0/528167238 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fee9006af30 0x7fee9006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:41.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.376+0000 7feeb1437640 1 --2- 172.21.15.154:0/528167238 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fee9006af30 0x7fee9006d3f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7feea80016b0 tx=0x7feea8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:41.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.379+0000 7feea1ffb640 1 -- 172.21.15.154:0/528167238 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7feeac1f6000 con 0x7feeac0cf2a0 2023-09-23T03:21:41.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.604+0000 7feeb369b640 1 -- 172.21.15.154:0/528167238 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feeac1f6000 con 0x7feeac0cf2a0 2023-09-23T03:21:41.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.606+0000 7feea1ffb640 1 -- 172.21.15.154:0/528167238 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4926 (secure 0 0 0) 0x7feeac1f6000 con 0x7feeac0cf2a0 2023-09-23T03:21:41.607 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:41.607 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":54,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5279,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1609915000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1609915000},{"type":"v1","addr":"172.21.15.154:6837","nonce":1609915000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":5288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1240415394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1240415394},{"type":"v1","addr":"172.21.15.190:6835","nonce":1240415394}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":5728,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/878668736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":878668736},{"type":"v1","addr":"172.21.15.190:6833","nonce":878668736}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:21:30.160245+0000","modified":"2023-09-23T03:21:40.311897+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6924},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6924":{"gid":6924,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":4,"addr":"172.21.15.154:6835/2633473586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2633473586},{"type":"v1","addr":"172.21.15.154:6835","nonce":2633473586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9,10],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-09-23T03:21:41.608 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 54 2023-09-23T03:21:41.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.608+0000 7feeb369b640 1 -- 172.21.15.154:0/528167238 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fee9006af30 msgr2=0x7fee9006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:41.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.608+0000 7feeb369b640 1 --2- 172.21.15.154:0/528167238 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fee9006af30 0x7fee9006d3f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7feea80016b0 tx=0x7feea8009040 comp rx=0 tx=0).stop 2023-09-23T03:21:41.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.608+0000 7feeb369b640 1 -- 172.21.15.154:0/528167238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feeac0cf2a0 msgr2=0x7feeac0b0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:41.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.608+0000 7feeb369b640 1 --2- 172.21.15.154:0/528167238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feeac0cf2a0 0x7feeac0b0250 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fee9c02bee0 tx=0x7fee9c0046c0 comp rx=0 tx=0).stop 2023-09-23T03:21:41.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.608+0000 7feeb369b640 1 -- 172.21.15.154:0/528167238 shutdown_connections 2023-09-23T03:21:41.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.608+0000 7feeb369b640 1 --2- 172.21.15.154:0/528167238 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fee9006af30 0x7fee9006d3f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:41.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.608+0000 7feeb369b640 1 --2- 172.21.15.154:0/528167238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feeac15cac0 0x7feeac0c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:41.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.608+0000 7feeb369b640 1 --2- 172.21.15.154:0/528167238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feeac0cf2a0 0x7feeac0b0250 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:41.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.608+0000 7feeb369b640 1 --2- 172.21.15.154:0/528167238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feeac0cd670 0x7feeac1f62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:41.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.608+0000 7feeb369b640 1 -- 172.21.15.154:0/528167238 >> 172.21.15.154:0/528167238 conn(0x7feeac089fd0 msgr2=0x7feeac0d3160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:41.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.609+0000 7feeb369b640 1 -- 172.21.15.154:0/528167238 shutdown_connections 2023-09-23T03:21:41.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:41.609+0000 7feeb369b640 1 -- 172.21.15.154:0/528167238 wait complete. 2023-09-23T03:21:42.029 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 54, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-23T03:21:30.160245+0000', 'modified': '2023-09-23T03:21:40.311897+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 69, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6924}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6924': {'gid': 6924, 'name': 'a', 'rank': 0, 'incarnation': 51, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.154:6835/2633473586', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6834', 'nonce': 2633473586}, {'type': 'v1', 'addr': '172.21.15.154:6835', 'nonce': 2633473586}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9, 10], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-23T03:21:42.029 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-23T03:21:42.030 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-09-23T03:21:42.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.257+0000 7f7a7759e640 1 Processor -- start 2023-09-23T03:21:42.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.258+0000 7f7a7759e640 1 -- start start 2023-09-23T03:21:42.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.260+0000 7f7a7759e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7a7815cc00 0x7f7a780b1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:42.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.260+0000 7f7a7759e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a7806be10 con 0x7f7a780cf8e0 2023-09-23T03:21:42.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.260+0000 7f7a7759e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a780d17f0 con 0x7f7a780cf500 2023-09-23T03:21:42.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.260+0000 7f7a7759e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a780d0470 con 0x7f7a7815cc00 2023-09-23T03:21:42.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.260+0000 7f7a76d9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7a7815cc00 0x7f7a780b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:42.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.260+0000 7f7a76d9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7a7815cc00 0x7f7a780b1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60988/0 (socket says 172.21.15.154:60988) 2023-09-23T03:21:42.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.260+0000 7f7a76d9d640 1 -- 172.21.15.154:0/2585568141 learned_addr learned my addr 172.21.15.154:0/2585568141 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:42.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.260+0000 7f7a75d9b640 1 --1- 172.21.15.154:0/2585568141 >> v1:172.21.15.154:6789/0 conn(0x7f7a780cf8e0 0x7f7a7815c4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:38382/0 (socket says 172.21.15.154:38382) 2023-09-23T03:21:42.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.261+0000 7f7a76d9d640 1 -- 172.21.15.154:0/2585568141 >> v1:172.21.15.190:6789/0 conn(0x7f7a780cf500 legacy=0x7f7a780ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:42.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.261+0000 7f7a76d9d640 1 -- 172.21.15.154:0/2585568141 >> v1:172.21.15.154:6789/0 conn(0x7f7a780cf8e0 legacy=0x7f7a7815c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:42.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.261+0000 7f7a76d9d640 1 -- 172.21.15.154:0/2585568141 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a780d0750 con 0x7f7a7815cc00 2023-09-23T03:21:42.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.262+0000 7f7a76d9d640 1 --2- 172.21.15.154:0/2585568141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a7815cc00 0x7f7a780b1430 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f7a700098a0 tx=0x7f7a7002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b71445b04b9032a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:42.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.262+0000 7f7a7559a640 1 -- 172.21.15.154:0/2585568141 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a70030840 con 0x7f7a7815cc00 2023-09-23T03:21:42.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.262+0000 7f7a7559a640 1 -- 172.21.15.154:0/2585568141 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a700309a0 con 0x7f7a7815cc00 2023-09-23T03:21:42.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.262+0000 7f7a7559a640 1 -- 172.21.15.154:0/2585568141 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a70030d30 con 0x7f7a7815cc00 2023-09-23T03:21:42.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.263+0000 7f7a7759e640 1 -- 172.21.15.154:0/2585568141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a7815cc00 msgr2=0x7f7a780b1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:42.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.263+0000 7f7a7759e640 1 --2- 172.21.15.154:0/2585568141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a7815cc00 0x7f7a780b1430 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f7a700098a0 tx=0x7f7a7002b980 comp rx=0 tx=0).stop 2023-09-23T03:21:42.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.263+0000 7f7a7759e640 1 -- 172.21.15.154:0/2585568141 shutdown_connections 2023-09-23T03:21:42.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.263+0000 7f7a7759e640 1 --2- 172.21.15.154:0/2585568141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a7815cc00 0x7f7a780b1430 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.263+0000 7f7a7759e640 1 -- 172.21.15.154:0/2585568141 >> 172.21.15.154:0/2585568141 conn(0x7f7a7809bbb0 msgr2=0x7f7a780d28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:42.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.263+0000 7f7a7759e640 1 -- 172.21.15.154:0/2585568141 shutdown_connections 2023-09-23T03:21:42.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.263+0000 7f7a7759e640 1 -- 172.21.15.154:0/2585568141 wait complete. 2023-09-23T03:21:42.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.265+0000 7f7a7759e640 1 Processor -- start 2023-09-23T03:21:42.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.265+0000 7f7a7759e640 1 -- start start 2023-09-23T03:21:42.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.266+0000 7f7a7759e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a780cf500 0x7f7a781f66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:42.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.266+0000 7f7a7759e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a780cf8e0 0x7f7a78155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:42.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.266+0000 7f7a7759e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a7815cc00 0x7f7a78153260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:42.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.266+0000 7f7a7759e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a781525e0 con 0x7f7a780cf500 2023-09-23T03:21:42.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.266+0000 7f7a7759e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a78155ad0 con 0x7f7a7815cc00 2023-09-23T03:21:42.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.266+0000 7f7a7759e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a78154210 con 0x7f7a780cf8e0 2023-09-23T03:21:42.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.266+0000 7f7a7659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a780cf500 0x7f7a781f66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:42.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.266+0000 7f7a7659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a780cf500 0x7f7a781f66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:32770/0 (socket says 172.21.15.154:32770) 2023-09-23T03:21:42.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.266+0000 7f7a7659c640 1 -- 172.21.15.154:0/912432732 learned_addr learned my addr 172.21.15.154:0/912432732 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:42.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.267+0000 7f7a76d9d640 1 --2- 172.21.15.154:0/912432732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a7815cc00 0x7f7a78153260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:42.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.267+0000 7f7a75d9b640 1 --2- 172.21.15.154:0/912432732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a780cf8e0 0x7f7a78155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:42.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.267+0000 7f7a7659c640 1 -- 172.21.15.154:0/912432732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a780cf8e0 msgr2=0x7f7a78155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:42.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.267+0000 7f7a7659c640 1 --2- 172.21.15.154:0/912432732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a780cf8e0 0x7f7a78155590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.267+0000 7f7a7659c640 1 -- 172.21.15.154:0/912432732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a7815cc00 msgr2=0x7f7a78153260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:42.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.267+0000 7f7a7659c640 1 --2- 172.21.15.154:0/912432732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a7815cc00 0x7f7a78153260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.267+0000 7f7a7659c640 1 -- 172.21.15.154:0/912432732 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a700094d0 con 0x7f7a780cf500 2023-09-23T03:21:42.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.267+0000 7f7a75d9b640 1 --2- 172.21.15.154:0/912432732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a780cf8e0 0x7f7a78155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:42.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.267+0000 7f7a7659c640 1 --2- 172.21.15.154:0/912432732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a780cf500 0x7f7a781f66d0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f7a6800c3a0 tx=0x7f7a6800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:42.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.267+0000 7f7a6effd640 1 -- 172.21.15.154:0/912432732 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a680052d0 con 0x7f7a780cf500 2023-09-23T03:21:42.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.267+0000 7f7a6effd640 1 -- 172.21.15.154:0/912432732 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a68005450 con 0x7f7a780cf500 2023-09-23T03:21:42.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.268+0000 7f7a6effd640 1 -- 172.21.15.154:0/912432732 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a68003880 con 0x7f7a780cf500 2023-09-23T03:21:42.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.268+0000 7f7a7759e640 1 -- 172.21.15.154:0/912432732 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a781544f0 con 0x7f7a780cf500 2023-09-23T03:21:42.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.268+0000 7f7a7759e640 1 -- 172.21.15.154:0/912432732 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a78154ab0 con 0x7f7a780cf500 2023-09-23T03:21:42.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.271+0000 7f7a6effd640 1 -- 172.21.15.154:0/912432732 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f7a680039e0 con 0x7f7a780cf500 2023-09-23T03:21:42.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.272+0000 7f7a6effd640 1 --2- 172.21.15.154:0/912432732 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7a5806af30 0x7f7a5806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:42.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.272+0000 7f7a7f7ad740 1 Processor -- start 2023-09-23T03:21:42.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.272+0000 7f7a6effd640 1 -- 172.21.15.154:0/912432732 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f7a68089140 con 0x7f7a780cf500 2023-09-23T03:21:42.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.272+0000 7f7a7f7ad740 1 -- start start 2023-09-23T03:21:42.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.272+0000 7f7a75d9b640 1 --2- 172.21.15.154:0/912432732 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7a5806af30 0x7f7a5806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:42.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.273+0000 7f7a75d9b640 1 --2- 172.21.15.154:0/912432732 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7a5806af30 0x7f7a5806d3f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7a60007fd0 tx=0x7f7a60009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:42.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.273+0000 7f7a7f7ad740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23bcf2da0 0x55f23bcc5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:42.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.273+0000 7f7a7f7ad740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f23bcf3180 0x55f23bcf5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:42.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.273+0000 7f7a7f7ad740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f23bcc61e0 0x55f23bcf6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:42.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.273+0000 7f7a7f7ad740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f23bc396d0 con 0x55f23bcf2da0 2023-09-23T03:21:42.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.273+0000 7f7a7f7ad740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f23bc36dc0 con 0x55f23bcf3180 2023-09-23T03:21:42.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.273+0000 7f7a7f7ad740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f23bc326b0 con 0x55f23bcc61e0 2023-09-23T03:21:42.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.273+0000 7f7a7659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23bcf2da0 0x55f23bcc5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:42.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.273+0000 7f7a7659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23bcf2da0 0x55f23bcc5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:32782/0 (socket says 172.21.15.154:32782) 2023-09-23T03:21:42.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.273+0000 7f7a7659c640 1 -- 172.21.15.154:0/2170880809 learned_addr learned my addr 172.21.15.154:0/2170880809 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:42.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.273+0000 7f7a7659c640 1 -- 172.21.15.154:0/2170880809 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f23bcc61e0 msgr2=0x55f23bcf6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:42.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.273+0000 7f7a7659c640 1 --2- 172.21.15.154:0/2170880809 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f23bcc61e0 0x55f23bcf6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.273+0000 7f7a7659c640 1 -- 172.21.15.154:0/2170880809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f23bcf3180 msgr2=0x55f23bcf5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:42.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.273+0000 7f7a7659c640 1 --2- 172.21.15.154:0/2170880809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f23bcf3180 0x55f23bcf5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.273+0000 7f7a7659c640 1 -- 172.21.15.154:0/2170880809 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f23bbf68e0 con 0x55f23bcf2da0 2023-09-23T03:21:42.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.274+0000 7f7a7659c640 1 --2- 172.21.15.154:0/2170880809 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23bcf2da0 0x55f23bcc5ca0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f7a68001110 tx=0x7f7a68058210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc1c1ff164841eb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:42.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.274+0000 7f7a57fff640 1 -- 172.21.15.154:0/2170880809 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6805fd50 con 0x55f23bcf2da0 2023-09-23T03:21:42.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.274+0000 7f7a57fff640 1 -- 172.21.15.154:0/2170880809 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a68055b90 con 0x55f23bcf2da0 2023-09-23T03:21:42.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.274+0000 7f7a57fff640 1 -- 172.21.15.154:0/2170880809 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a68054160 con 0x55f23bcf2da0 2023-09-23T03:21:42.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.275+0000 7f7a7f7ad740 1 -- 172.21.15.154:0/2170880809 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23bcf2da0 msgr2=0x55f23bcc5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:42.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.275+0000 7f7a7f7ad740 1 --2- 172.21.15.154:0/2170880809 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23bcf2da0 0x55f23bcc5ca0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f7a68001110 tx=0x7f7a68058210 comp rx=0 tx=0).stop 2023-09-23T03:21:42.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.275+0000 7f7a7f7ad740 1 -- 172.21.15.154:0/2170880809 shutdown_connections 2023-09-23T03:21:42.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.275+0000 7f7a7f7ad740 1 --2- 172.21.15.154:0/2170880809 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f23bcc61e0 0x55f23bcf6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.275+0000 7f7a7f7ad740 1 --2- 172.21.15.154:0/2170880809 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f23bcf3180 0x55f23bcf5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.275+0000 7f7a7f7ad740 1 --2- 172.21.15.154:0/2170880809 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23bcf2da0 0x55f23bcc5ca0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.275+0000 7f7a7f7ad740 1 -- 172.21.15.154:0/2170880809 >> 172.21.15.154:0/2170880809 conn(0x55f23bcc7fb0 msgr2=0x55f23bd10b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:42.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.275+0000 7f7a7f7ad740 1 -- 172.21.15.154:0/2170880809 shutdown_connections 2023-09-23T03:21:42.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.275+0000 7f7a7f7ad740 1 -- 172.21.15.154:0/2170880809 wait complete. 2023-09-23T03:21:42.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.276+0000 7f7a7f7ad740 1 -- start start 2023-09-23T03:21:42.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.276+0000 7f7a7f7ad740 1 Processor -- start 2023-09-23T03:21:42.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.277+0000 7f7a7f7ad740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23bcf2da0 0x55f23bce06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:42.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.277+0000 7f7a7f7ad740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f23bcf3180 0x55f23bcf7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:42.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.277+0000 7f7a7f7ad740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f23bce0bf0 0x55f23bd0b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:42.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.277+0000 7f7a7f7ad740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f23bbd0940 con 0x55f23bcf2da0 2023-09-23T03:21:42.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.277+0000 7f7a7f7ad740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f23bbd05e0 con 0x55f23bcf3180 2023-09-23T03:21:42.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.277+0000 7f7a7f7ad740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f23b996f20 con 0x55f23bce0bf0 2023-09-23T03:21:42.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.277+0000 7f7a7f7ad740 10 client.0 _ll_register_callbacks cb 0x55f23bc0e5e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:21:42.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.277+0000 7f7a7659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23bcf2da0 0x55f23bce06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:42.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.277+0000 7f7a7659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23bcf2da0 0x55f23bce06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:32788/0 (socket says 172.21.15.154:32788) 2023-09-23T03:21:42.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.277+0000 7f7a7659c640 1 -- 172.21.15.154:0/1808658271 learned_addr learned my addr 172.21.15.154:0/1808658271 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:42.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.278+0000 7f7a7659c640 1 -- 172.21.15.154:0/1808658271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f23bce0bf0 msgr2=0x55f23bd0b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:42.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.278+0000 7f7a7659c640 1 --2- 172.21.15.154:0/1808658271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f23bce0bf0 0x55f23bd0b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.278+0000 7f7a7659c640 1 -- 172.21.15.154:0/1808658271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f23bcf3180 msgr2=0x55f23bcf7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:42.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.278+0000 7f7a7659c640 1 --2- 172.21.15.154:0/1808658271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f23bcf3180 0x55f23bcf7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.278+0000 7f7a7659c640 1 -- 172.21.15.154:0/1808658271 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a78154ab0 con 0x55f23bcf2da0 2023-09-23T03:21:42.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.278+0000 7f7a7659c640 1 --2- 172.21.15.154:0/1808658271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23bcf2da0 0x55f23bce06b0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f7a68057700 tx=0x7f7a68000f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2072c07a99aaf77e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:42.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.278+0000 7f7a557fa640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:21:42.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.278+0000 7f7a557fa640 1 -- 172.21.15.154:0/1808658271 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6805f8c0 con 0x55f23bcf2da0 2023-09-23T03:21:42.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.278+0000 7f7a557fa640 1 -- 172.21.15.154:0/1808658271 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f7a68071eb0 con 0x55f23bcf2da0 2023-09-23T03:21:42.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.278+0000 7f7a7f7ad740 1 -- 172.21.15.154:0/1808658271 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f23bc2b690 con 0x55f23bcf2da0 2023-09-23T03:21:42.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.279+0000 7f7a557fa640 1 -- 172.21.15.154:0/1808658271 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a6805fa20 con 0x55f23bcf2da0 2023-09-23T03:21:42.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.279+0000 7f7a7f7ad740 10 client.6975 fetch_fsmap learned FSMap version 54 2023-09-23T03:21:42.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.279+0000 7f7a7f7ad740 1 -- 172.21.15.154:0/1808658271 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x55f23bc30d30 con 0x55f23bcf2da0 2023-09-23T03:21:42.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.279+0000 7f7a557fa640 1 -- 172.21.15.154:0/1808658271 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f23bc2b690 con 0x55f23bcf2da0 2023-09-23T03:21:42.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.279+0000 7f7a557fa640 1 -- 172.21.15.154:0/1808658271 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 54) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f7a68076260 con 0x55f23bcf2da0 2023-09-23T03:21:42.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.279+0000 7f7a7f7ad740 10 client.6975 fetch_fsmap finished waiting for FSMap version 54 2023-09-23T03:21:42.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.279+0000 7f7a7f7ad740 10 client.6975 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-23T03:21:42.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.279+0000 7f7a7f7ad740 20 client.6975 populate_metadata read hostname 'smithi154' 2023-09-23T03:21:42.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.279+0000 7f7a7f7ad740 1 --2- 172.21.15.154:0/1808658271 >> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] conn(0x55f23bd0b6a0 0x55f23bd246b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:42.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.279+0000 7f7a7f7ad740 4 client.6975 mds_command: new command op to 6924 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T03:21:42.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.279+0000 7f7a7f7ad740 1 -- 172.21.15.154:0/1808658271 --> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f23bc2b3b0 con 0x55f23bd0b6a0 2023-09-23T03:21:42.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.279+0000 7f7a75d9b640 1 --2- 172.21.15.154:0/1808658271 >> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] conn(0x55f23bd0b6a0 0x55f23bd246b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:42.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.280+0000 7f7a75d9b640 1 --2- 172.21.15.154:0/1808658271 >> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] conn(0x55f23bd0b6a0 0x55f23bd246b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c5901a1dd910f9e server_cookie=96ff7c15e25b86ca in_seq=0 out_seq=0 2023-09-23T03:21:42.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.280+0000 7f7a557fa640 10 client.6975 ms_handle_connect on v2:172.21.15.154:6834/2633473586 2023-09-23T03:21:42.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.281+0000 7f7a557fa640 1 -- 172.21.15.154:0/1808658271 <== mds.0 v2:172.21.15.154:6834/2633473586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f23bc2b3b0 con 0x55f23bd0b6a0 2023-09-23T03:21:42.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.281+0000 7f7a557fa640 10 client.6975 handle_command_reply: tid=0 2023-09-23T03:21:42.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.281+0000 7f7a7f7ad740 1 client.6975 shutdown 2023-09-23T03:21:42.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.281+0000 7f7a7f7ad740 1 -- 172.21.15.154:0/1808658271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23bcf2da0 msgr2=0x55f23bce06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:42.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.281+0000 7f7a7f7ad740 1 --2- 172.21.15.154:0/1808658271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23bcf2da0 0x55f23bce06b0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f7a68057700 tx=0x7f7a68000f20 comp rx=0 tx=0).stop 2023-09-23T03:21:42.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.281+0000 7f7a7f7ad740 1 -- 172.21.15.154:0/1808658271 shutdown_connections 2023-09-23T03:21:42.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.281+0000 7f7a7f7ad740 1 --2- 172.21.15.154:0/1808658271 >> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] conn(0x55f23bd0b6a0 0x55f23bd246b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.281+0000 7f7a7f7ad740 1 --2- 172.21.15.154:0/1808658271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f23bce0bf0 0x55f23bd0b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.281+0000 7f7a7f7ad740 1 --2- 172.21.15.154:0/1808658271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f23bcf3180 0x55f23bcf7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.281+0000 7f7a557fa640 0 client.6975 ms_handle_reset on v2:172.21.15.154:6834/2633473586 2023-09-23T03:21:42.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.281+0000 7f7a7f7ad740 1 --2- 172.21.15.154:0/1808658271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23bcf2da0 0x55f23bce06b0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.281+0000 7f7a7f7ad740 1 -- 172.21.15.154:0/1808658271 >> 172.21.15.154:0/1808658271 conn(0x55f23bcc61e0 msgr2=0x55f23bcc9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:42.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.281+0000 7f7a7f7ad740 1 -- 172.21.15.154:0/1808658271 shutdown_connections 2023-09-23T03:21:42.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.281+0000 7f7a7f7ad740 1 -- 172.21.15.154:0/1808658271 wait complete. 2023-09-23T03:21:42.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.282+0000 7f7a7f7ad740 20 client.6975 trim_cache size 0 max 16384 2023-09-23T03:21:42.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.298+0000 7f7a7f7ad740 1 Processor -- start 2023-09-23T03:21:42.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.298+0000 7f7a7f7ad740 1 -- start start 2023-09-23T03:21:42.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.299+0000 7f7a7f7ad740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f23bce0bf0 0x55f23bd138d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:42.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.299+0000 7f7a7f7ad740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f23bcf2da0 0x55f23bcc9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:42.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.299+0000 7f7a7f7ad740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23bcf3180 0x55f23bce0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:42.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.299+0000 7f7a7f7ad740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f23bd13250 con 0x55f23bcf3180 2023-09-23T03:21:42.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.299+0000 7f7a7f7ad740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f23bcf5e80 con 0x55f23bcf2da0 2023-09-23T03:21:42.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.299+0000 7f7a7f7ad740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f23bcf6090 con 0x55f23bce0bf0 2023-09-23T03:21:42.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.299+0000 7f7a76d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23bcf3180 0x55f23bce0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:42.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.299+0000 7f7a76d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23bcf3180 0x55f23bce0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:32796/0 (socket says 172.21.15.154:32796) 2023-09-23T03:21:42.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.300+0000 7f7a76d9d640 1 -- 172.21.15.154:0/2192993579 learned_addr learned my addr 172.21.15.154:0/2192993579 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:42.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.300+0000 7f7a76d9d640 1 -- 172.21.15.154:0/2192993579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f23bce0bf0 msgr2=0x55f23bd138d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:42.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.300+0000 7f7a76d9d640 1 --2- 172.21.15.154:0/2192993579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f23bce0bf0 0x55f23bd138d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.300+0000 7f7a76d9d640 1 -- 172.21.15.154:0/2192993579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f23bcf2da0 msgr2=0x55f23bcc9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:42.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.300+0000 7f7a75d9b640 1 --2- 172.21.15.154:0/2192993579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f23bcf2da0 0x55f23bcc9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:42.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.300+0000 7f7a76d9d640 1 --2- 172.21.15.154:0/2192993579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f23bcf2da0 0x55f23bcc9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.300+0000 7f7a76d9d640 1 -- 172.21.15.154:0/2192993579 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f23bcc6ff0 con 0x55f23bcf3180 2023-09-23T03:21:42.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.300+0000 7f7a76d9d640 1 --2- 172.21.15.154:0/2192993579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23bcf3180 0x55f23bce0030 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f7a7002b9c0 tx=0x7f7a70004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dadf7cc7a189ce4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:42.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.301+0000 7f7a6cff9640 1 -- 172.21.15.154:0/2192993579 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a70030840 con 0x55f23bcf3180 2023-09-23T03:21:42.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.301+0000 7f7a6cff9640 1 -- 172.21.15.154:0/2192993579 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a70030b10 con 0x55f23bcf3180 2023-09-23T03:21:42.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.301+0000 7f7a6cff9640 1 -- 172.21.15.154:0/2192993579 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a70030ea0 con 0x55f23bcf3180 2023-09-23T03:21:42.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.301+0000 7f7a7f7ad740 1 -- 172.21.15.154:0/2192993579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23bcf3180 msgr2=0x55f23bce0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:42.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.301+0000 7f7a7f7ad740 1 --2- 172.21.15.154:0/2192993579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23bcf3180 0x55f23bce0030 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f7a7002b9c0 tx=0x7f7a70004730 comp rx=0 tx=0).stop 2023-09-23T03:21:42.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.301+0000 7f7a7f7ad740 1 -- 172.21.15.154:0/2192993579 shutdown_connections 2023-09-23T03:21:42.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.301+0000 7f7a7f7ad740 1 --2- 172.21.15.154:0/2192993579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23bcf3180 0x55f23bce0030 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.301+0000 7f7a7f7ad740 1 --2- 172.21.15.154:0/2192993579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f23bcf2da0 0x55f23bcc9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.301+0000 7f7a7f7ad740 1 --2- 172.21.15.154:0/2192993579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f23bce0bf0 0x55f23bd138d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.301+0000 7f7a7f7ad740 1 -- 172.21.15.154:0/2192993579 >> 172.21.15.154:0/2192993579 conn(0x55f23bcc61e0 msgr2=0x55f23bcf9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:42.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.301+0000 7f7a7f7ad740 1 -- 172.21.15.154:0/2192993579 shutdown_connections 2023-09-23T03:21:42.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.301+0000 7f7a7f7ad740 1 -- 172.21.15.154:0/2192993579 wait complete. 2023-09-23T03:21:42.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.302+0000 7f7a7f7ad740 1 -- start start 2023-09-23T03:21:42.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.302+0000 7f7a7f7ad740 1 Processor -- start 2023-09-23T03:21:42.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.303+0000 7f7a7f7ad740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f23bce0bf0 0x55f23bccb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:42.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.303+0000 7f7a7f7ad740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23bcf2da0 0x55f23bd11d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:42.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.303+0000 7f7a7f7ad740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f23bcf3180 0x55f23bd12250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:42.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.303+0000 7f7a7f7ad740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f23bd2b520 con 0x55f23bcf2da0 2023-09-23T03:21:42.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.303+0000 7f7a7f7ad740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f23bd1f580 con 0x55f23bcf3180 2023-09-23T03:21:42.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.303+0000 7f7a7f7ad740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f23bccbc60 con 0x55f23bce0bf0 2023-09-23T03:21:42.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.303+0000 7f7a7f7ad740 10 client.0 _ll_register_callbacks cb 0x7f7a680011f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:21:42.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.304+0000 7f7a75d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23bcf2da0 0x55f23bd11d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:42.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.304+0000 7f7a75d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23bcf2da0 0x55f23bd11d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:32804/0 (socket says 172.21.15.154:32804) 2023-09-23T03:21:42.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.304+0000 7f7a75d9b640 1 -- 172.21.15.154:0/2788082581 learned_addr learned my addr 172.21.15.154:0/2788082581 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:42.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.304+0000 7f7a75d9b640 1 -- 172.21.15.154:0/2788082581 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f23bce0bf0 msgr2=0x55f23bccb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:42.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.304+0000 7f7a7659c640 1 --2- 172.21.15.154:0/2788082581 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f23bce0bf0 0x55f23bccb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:42.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.304+0000 7f7a76d9d640 1 --2- 172.21.15.154:0/2788082581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f23bcf3180 0x55f23bd12250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:42.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.304+0000 7f7a75d9b640 1 --2- 172.21.15.154:0/2788082581 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f23bce0bf0 0x55f23bccb3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.304+0000 7f7a75d9b640 1 -- 172.21.15.154:0/2788082581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f23bcf3180 msgr2=0x55f23bd12250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:42.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.304+0000 7f7a75d9b640 1 --2- 172.21.15.154:0/2788082581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f23bcf3180 0x55f23bd12250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.304+0000 7f7a75d9b640 1 -- 172.21.15.154:0/2788082581 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a780d0750 con 0x55f23bcf2da0 2023-09-23T03:21:42.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.304+0000 7f7a75d9b640 1 --2- 172.21.15.154:0/2788082581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23bcf2da0 0x55f23bd11d10 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f7a60005e60 tx=0x7f7a600014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b84882815fc0c827 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:42.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.304+0000 7f7a557fa640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:21:42.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.305+0000 7f7a557fa640 1 -- 172.21.15.154:0/2788082581 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a60020540 con 0x55f23bcf2da0 2023-09-23T03:21:42.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.305+0000 7f7a557fa640 1 -- 172.21.15.154:0/2788082581 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f7a600075a0 con 0x55f23bcf2da0 2023-09-23T03:21:42.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.305+0000 7f7a7f7ad740 1 -- 172.21.15.154:0/2788082581 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a68055b90 con 0x55f23bcf2da0 2023-09-23T03:21:42.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.305+0000 7f7a557fa640 1 -- 172.21.15.154:0/2788082581 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a60007760 con 0x55f23bcf2da0 2023-09-23T03:21:42.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.305+0000 7f7a7f7ad740 10 client.6981 fetch_fsmap learned FSMap version 54 2023-09-23T03:21:42.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.305+0000 7f7a7f7ad740 1 -- 172.21.15.154:0/2788082581 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x55f23bbcfb50 con 0x55f23bcf2da0 2023-09-23T03:21:42.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.305+0000 7f7a557fa640 1 -- 172.21.15.154:0/2788082581 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a68055b90 con 0x55f23bcf2da0 2023-09-23T03:21:42.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.305+0000 7f7a557fa640 1 -- 172.21.15.154:0/2788082581 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 54) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f7a600078c0 con 0x55f23bcf2da0 2023-09-23T03:21:42.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.305+0000 7f7a7f7ad740 10 client.6981 fetch_fsmap finished waiting for FSMap version 54 2023-09-23T03:21:42.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.305+0000 7f7a7f7ad740 10 client.6981 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-23T03:21:42.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.305+0000 7f7a7f7ad740 20 client.6981 populate_metadata read hostname 'smithi154' 2023-09-23T03:21:42.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.305+0000 7f7a7f7ad740 1 --2- 172.21.15.154:0/2788082581 >> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] conn(0x55f23bd2aa00 0x55f23bcdedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:42.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.306+0000 7f7a7f7ad740 4 client.6981 mds_command: new command op to 6924 tid=0[{"prefix": "status"}] 2023-09-23T03:21:42.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.306+0000 7f7a7f7ad740 1 -- 172.21.15.154:0/2788082581 --> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f23b9346c0 con 0x55f23bd2aa00 2023-09-23T03:21:42.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.306+0000 7f7a7659c640 1 --2- 172.21.15.154:0/2788082581 >> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] conn(0x55f23bd2aa00 0x55f23bcdedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:42.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.306+0000 7f7a7659c640 1 --2- 172.21.15.154:0/2788082581 >> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] conn(0x55f23bd2aa00 0x55f23bcdedf0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db6542e5bc847a05 server_cookie=de441dd51af854b2 in_seq=0 out_seq=0 2023-09-23T03:21:42.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.306+0000 7f7a557fa640 10 client.6981 ms_handle_connect on v2:172.21.15.154:6834/2633473586 2023-09-23T03:21:42.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.307+0000 7f7a557fa640 1 -- 172.21.15.154:0/2788082581 <== mds.0 v2:172.21.15.154:6834/2633473586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55f23b9346c0 con 0x55f23bd2aa00 2023-09-23T03:21:42.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.307+0000 7f7a557fa640 10 client.6981 handle_command_reply: tid=0 2023-09-23T03:21:42.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.307+0000 7f7a7f7ad740 1 client.6981 shutdown 2023-09-23T03:21:42.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.307+0000 7f7a7f7ad740 1 -- 172.21.15.154:0/2788082581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23bcf2da0 msgr2=0x55f23bd11d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:42.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.307+0000 7f7a7f7ad740 1 --2- 172.21.15.154:0/2788082581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23bcf2da0 0x55f23bd11d10 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f7a60005e60 tx=0x7f7a600014a0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.307+0000 7f7a7f7ad740 1 -- 172.21.15.154:0/2788082581 shutdown_connections 2023-09-23T03:21:42.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.307+0000 7f7a7f7ad740 1 --2- 172.21.15.154:0/2788082581 >> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] conn(0x55f23bd2aa00 0x55f23bcdedf0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.307+0000 7f7a7f7ad740 1 --2- 172.21.15.154:0/2788082581 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f23bcf3180 0x55f23bd12250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.307+0000 7f7a7f7ad740 1 --2- 172.21.15.154:0/2788082581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f23bcf2da0 0x55f23bd11d10 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.307+0000 7f7a557fa640 0 client.6981 ms_handle_reset on v2:172.21.15.154:6834/2633473586 2023-09-23T03:21:42.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.307+0000 7f7a7f7ad740 1 --2- 172.21.15.154:0/2788082581 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f23bce0bf0 0x55f23bccb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.307+0000 7f7a7f7ad740 1 -- 172.21.15.154:0/2788082581 >> 172.21.15.154:0/2788082581 conn(0x55f23bcc61e0 msgr2=0x55f23bd14040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:42.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.307+0000 7f7a7f7ad740 1 -- 172.21.15.154:0/2788082581 shutdown_connections 2023-09-23T03:21:42.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.307+0000 7f7a7f7ad740 1 -- 172.21.15.154:0/2788082581 wait complete. 2023-09-23T03:21:42.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.307+0000 7f7a7f7ad740 20 client.6981 trim_cache size 0 max 16384 2023-09-23T03:21:42.313 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:21:42.313 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_fsid": "8c8039d3-6c84-43db-8808-85e0dbdd47c8", 2023-09-23T03:21:42.314 INFO:teuthology.orchestra.run.smithi154.stdout: "whoami": 0, 2023-09-23T03:21:42.314 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 6924, 2023-09-23T03:21:42.314 INFO:teuthology.orchestra.run.smithi154.stdout: "want_state": "up:active", 2023-09-23T03:21:42.314 INFO:teuthology.orchestra.run.smithi154.stdout: "state": "up:active", 2023-09-23T03:21:42.314 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs", 2023-09-23T03:21:42.314 INFO:teuthology.orchestra.run.smithi154.stdout: "rank_uptime": 4.0070905479999999, 2023-09-23T03:21:42.314 INFO:teuthology.orchestra.run.smithi154.stdout: "mdsmap_epoch": 54, 2023-09-23T03:21:42.314 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_epoch": 71, 2023-09-23T03:21:42.314 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_epoch_barrier": 71, 2023-09-23T03:21:42.314 INFO:teuthology.orchestra.run.smithi154.stdout: "uptime": 4.7788158459999996 2023-09-23T03:21:42.314 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:21:42.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.308+0000 7f7a7759e640 1 -- 172.21.15.154:0/912432732 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7a5806af30 msgr2=0x7f7a5806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:42.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.308+0000 7f7a7759e640 1 --2- 172.21.15.154:0/912432732 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7a5806af30 0x7f7a5806d3f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7a60007fd0 tx=0x7f7a60009040 comp rx=0 tx=0).stop 2023-09-23T03:21:42.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.308+0000 7f7a7759e640 1 -- 172.21.15.154:0/912432732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a780cf500 msgr2=0x7f7a781f66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:42.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.308+0000 7f7a7759e640 1 --2- 172.21.15.154:0/912432732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a780cf500 0x7f7a781f66d0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f7a6800c3a0 tx=0x7f7a6800c860 comp rx=0 tx=0).stop 2023-09-23T03:21:42.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.308+0000 7f7a7759e640 1 -- 172.21.15.154:0/912432732 shutdown_connections 2023-09-23T03:21:42.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.308+0000 7f7a7759e640 1 --2- 172.21.15.154:0/912432732 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7a5806af30 0x7f7a5806d3f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.308+0000 7f7a7759e640 1 --2- 172.21.15.154:0/912432732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a7815cc00 0x7f7a78153260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.308+0000 7f7a7759e640 1 --2- 172.21.15.154:0/912432732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a780cf8e0 0x7f7a78155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.308+0000 7f7a7759e640 1 --2- 172.21.15.154:0/912432732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a780cf500 0x7f7a781f66d0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.308+0000 7f7a7759e640 1 -- 172.21.15.154:0/912432732 >> 172.21.15.154:0/912432732 conn(0x7f7a7809bbb0 msgr2=0x7f7a780cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:42.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.308+0000 7f7a7759e640 1 -- 172.21.15.154:0/912432732 shutdown_connections 2023-09-23T03:21:42.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.308+0000 7f7a7759e640 1 -- 172.21.15.154:0/912432732 wait complete. 2023-09-23T03:21:42.326 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:21:42.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.521+0000 7f4f2b60e640 1 Processor -- start 2023-09-23T03:21:42.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.521+0000 7f4f2b60e640 1 -- start start 2023-09-23T03:21:42.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.522+0000 7f4f2b60e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4f240cf130 0x7f4f240ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:42.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.522+0000 7f4f2b60e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f2406ceb0 con 0x7f4f240b2fe0 2023-09-23T03:21:42.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.522+0000 7f4f2b60e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f24159c90 con 0x7f4f240ada30 2023-09-23T03:21:42.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.522+0000 7f4f2b60e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f240ade10 con 0x7f4f240cf130 2023-09-23T03:21:42.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.523+0000 7f4f28ba9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4f240cf130 0x7f4f240ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:42.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.523+0000 7f4f29bab640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f4f240b2fe0 0x7f4f2415d380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:38394/0 (socket says 172.21.15.154:38394) 2023-09-23T03:21:42.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.523+0000 7f4f28ba9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4f240cf130 0x7f4f240ccaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:32818/0 (socket says 172.21.15.154:32818) 2023-09-23T03:21:42.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.523+0000 7f4f29bab640 1 -- 172.21.15.154:0/1986401014 learned_addr learned my addr 172.21.15.154:0/1986401014 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:21:42.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.523+0000 7f4f28ba9640 1 -- 172.21.15.154:0/1986401014 >> v1:172.21.15.190:6789/0 conn(0x7f4f240ada30 legacy=0x7f4f240cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:42.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.523+0000 7f4f28ba9640 1 -- 172.21.15.154:0/1986401014 >> v1:172.21.15.154:6789/0 conn(0x7f4f240b2fe0 legacy=0x7f4f2415d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:42.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.523+0000 7f4f28ba9640 1 -- 172.21.15.154:0/1986401014 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f2406c110 con 0x7f4f240cf130 2023-09-23T03:21:42.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.524+0000 7f4f28ba9640 1 --2- 172.21.15.154:0/1986401014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f240cf130 0x7f4f240ccaf0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f4f140097a0 tx=0x7f4f1402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60360b98f7900e83 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:42.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.524+0000 7f4f1bfff640 1 -- 172.21.15.154:0/1986401014 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f14031830 con 0x7f4f240cf130 2023-09-23T03:21:42.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.524+0000 7f4f1bfff640 1 -- 172.21.15.154:0/1986401014 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f14031990 con 0x7f4f240cf130 2023-09-23T03:21:42.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.525+0000 7f4f1bfff640 1 -- 172.21.15.154:0/1986401014 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f14031d20 con 0x7f4f240cf130 2023-09-23T03:21:42.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.525+0000 7f4f2b60e640 1 -- 172.21.15.154:0/1986401014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f240cf130 msgr2=0x7f4f240ccaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:42.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.525+0000 7f4f2b60e640 1 --2- 172.21.15.154:0/1986401014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f240cf130 0x7f4f240ccaf0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f4f140097a0 tx=0x7f4f1402b8d0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.525+0000 7f4f2b60e640 1 -- 172.21.15.154:0/1986401014 shutdown_connections 2023-09-23T03:21:42.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.525+0000 7f4f2b60e640 1 --2- 172.21.15.154:0/1986401014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f240cf130 0x7f4f240ccaf0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.525+0000 7f4f2b60e640 1 -- 172.21.15.154:0/1986401014 >> 172.21.15.154:0/1986401014 conn(0x7f4f2406a0f0 msgr2=0x7f4f2406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:42.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.525+0000 7f4f2b60e640 1 -- 172.21.15.154:0/1986401014 shutdown_connections 2023-09-23T03:21:42.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.525+0000 7f4f2b60e640 1 -- 172.21.15.154:0/1986401014 wait complete. 2023-09-23T03:21:42.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.527+0000 7f4f2b60e640 1 Processor -- start 2023-09-23T03:21:42.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.527+0000 7f4f2b60e640 1 -- start start 2023-09-23T03:21:42.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.528+0000 7f4f2b60e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f240ada30 0x7f4f24159110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:42.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.528+0000 7f4f2b60e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f240b2fe0 0x7f4f241f5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:42.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.528+0000 7f4f2b60e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f240cf130 0x7f4f2414e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:42.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.528+0000 7f4f2b60e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f240982c0 con 0x7f4f240ada30 2023-09-23T03:21:42.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.528+0000 7f4f2b60e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f240985a0 con 0x7f4f240b2fe0 2023-09-23T03:21:42.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.528+0000 7f4f2b60e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2414d650 con 0x7f4f240cf130 2023-09-23T03:21:42.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.528+0000 7f4f293aa640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f240ada30 0x7f4f24159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:42.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.528+0000 7f4f293aa640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f240ada30 0x7f4f24159110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:32828/0 (socket says 172.21.15.154:32828) 2023-09-23T03:21:42.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.528+0000 7f4f293aa640 1 -- 172.21.15.154:0/349546674 learned_addr learned my addr 172.21.15.154:0/349546674 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:42.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.529+0000 7f4f28ba9640 1 --2- 172.21.15.154:0/349546674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f240b2fe0 0x7f4f241f5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:42.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.529+0000 7f4f293aa640 1 -- 172.21.15.154:0/349546674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f240cf130 msgr2=0x7f4f2414e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:42.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.529+0000 7f4f29bab640 1 --2- 172.21.15.154:0/349546674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f240cf130 0x7f4f2414e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:42.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.529+0000 7f4f293aa640 1 --2- 172.21.15.154:0/349546674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f240cf130 0x7f4f2414e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.529+0000 7f4f293aa640 1 -- 172.21.15.154:0/349546674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f240b2fe0 msgr2=0x7f4f241f5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:42.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.529+0000 7f4f293aa640 1 --2- 172.21.15.154:0/349546674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f240b2fe0 0x7f4f241f5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.529+0000 7f4f293aa640 1 -- 172.21.15.154:0/349546674 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f14009400 con 0x7f4f240ada30 2023-09-23T03:21:42.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.529+0000 7f4f293aa640 1 --2- 172.21.15.154:0/349546674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f240ada30 0x7f4f24159110 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f4f20004640 tx=0x7f4f2000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:42.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.529+0000 7f4f19ffb640 1 -- 172.21.15.154:0/349546674 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f20011790 con 0x7f4f240ada30 2023-09-23T03:21:42.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.529+0000 7f4f19ffb640 1 -- 172.21.15.154:0/349546674 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f200118f0 con 0x7f4f240ada30 2023-09-23T03:21:42.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.529+0000 7f4f2b60e640 1 -- 172.21.15.154:0/349546674 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f2414d930 con 0x7f4f240ada30 2023-09-23T03:21:42.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.530+0000 7f4f19ffb640 1 -- 172.21.15.154:0/349546674 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f20011ce0 con 0x7f4f240ada30 2023-09-23T03:21:42.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.530+0000 7f4f2b60e640 1 -- 172.21.15.154:0/349546674 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f24098880 con 0x7f4f240ada30 2023-09-23T03:21:42.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.531+0000 7f4f2b60e640 1 -- 172.21.15.154:0/349546674 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f2414de50 con 0x7f4f240ada30 2023-09-23T03:21:42.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.532+0000 7f4f19ffb640 1 -- 172.21.15.154:0/349546674 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f4f2001c460 con 0x7f4f240ada30 2023-09-23T03:21:42.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.533+0000 7f4f19ffb640 1 --2- 172.21.15.154:0/349546674 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4f0806af30 0x7f4f0806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:42.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.533+0000 7f4f19ffb640 1 -- 172.21.15.154:0/349546674 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f4f20089b30 con 0x7f4f240ada30 2023-09-23T03:21:42.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.533+0000 7f4f28ba9640 1 --2- 172.21.15.154:0/349546674 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4f0806af30 0x7f4f0806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:42.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.534+0000 7f4f28ba9640 1 --2- 172.21.15.154:0/349546674 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4f0806af30 0x7f4f0806d3f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4f14036000 tx=0x7f4f14008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:42.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.537+0000 7f4f19ffb640 1 -- 172.21.15.154:0/349546674 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f4f2414de50 con 0x7f4f240ada30 2023-09-23T03:21:42.657 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:21:42.665 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:21:42.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.763+0000 7f4f2b60e640 1 -- 172.21.15.154:0/349546674 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f24073eb0 con 0x7f4f240ada30 2023-09-23T03:21:42.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.764+0000 7f4f19ffb640 1 -- 172.21.15.154:0/349546674 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4926 (secure 0 0 0) 0x7f4f200546f0 con 0x7f4f240ada30 2023-09-23T03:21:42.765 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:21:42.765 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":54,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5279,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1609915000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1609915000},{"type":"v1","addr":"172.21.15.154:6837","nonce":1609915000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":5288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1240415394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1240415394},{"type":"v1","addr":"172.21.15.190:6835","nonce":1240415394}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":5728,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/878668736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":878668736},{"type":"v1","addr":"172.21.15.190:6833","nonce":878668736}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:21:30.160245+0000","modified":"2023-09-23T03:21:40.311897+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6924},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6924":{"gid":6924,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":4,"addr":"172.21.15.154:6835/2633473586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2633473586},{"type":"v1","addr":"172.21.15.154:6835","nonce":2633473586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9,10],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-09-23T03:21:42.765 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 54 2023-09-23T03:21:42.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.766+0000 7f4f2b60e640 1 -- 172.21.15.154:0/349546674 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4f0806af30 msgr2=0x7f4f0806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:42.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.766+0000 7f4f2b60e640 1 --2- 172.21.15.154:0/349546674 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4f0806af30 0x7f4f0806d3f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4f14036000 tx=0x7f4f14008030 comp rx=0 tx=0).stop 2023-09-23T03:21:42.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.766+0000 7f4f2b60e640 1 -- 172.21.15.154:0/349546674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f240ada30 msgr2=0x7f4f24159110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:42.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.766+0000 7f4f2b60e640 1 --2- 172.21.15.154:0/349546674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f240ada30 0x7f4f24159110 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f4f20004640 tx=0x7f4f2000c8e0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.766+0000 7f4f2b60e640 1 -- 172.21.15.154:0/349546674 shutdown_connections 2023-09-23T03:21:42.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.766+0000 7f4f2b60e640 1 --2- 172.21.15.154:0/349546674 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4f0806af30 0x7f4f0806d3f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.766+0000 7f4f2b60e640 1 --2- 172.21.15.154:0/349546674 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4f240cf130 0x7f4f2414e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.766+0000 7f4f2b60e640 1 --2- 172.21.15.154:0/349546674 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4f240b2fe0 0x7f4f241f5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.766+0000 7f4f2b60e640 1 --2- 172.21.15.154:0/349546674 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f240ada30 0x7f4f24159110 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:42.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.766+0000 7f4f2b60e640 1 -- 172.21.15.154:0/349546674 >> 172.21.15.154:0/349546674 conn(0x7f4f2406a0f0 msgr2=0x7f4f240cae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:42.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.766+0000 7f4f2b60e640 1 -- 172.21.15.154:0/349546674 shutdown_connections 2023-09-23T03:21:42.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:42.767+0000 7f4f2b60e640 1 -- 172.21.15.154:0/349546674 wait complete. 2023-09-23T03:21:42.785 INFO:tasks.cephfs.test_data_scan:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 3, "epoch": 54, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 3, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-09-23T03:21:30.160245+0000", "damaged": [], "data_pools": [ 9, 10 ], "enabled": true, "epoch": 54, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_6924": { "addr": "172.21.15.154:6835/2633473586", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 2633473586, "type": "v2" }, { "addr": "172.21.15.154:6835", "nonce": 2633473586, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6924, "incarnation": 51, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 4 } }, "last_failure": 0, "last_failure_osd_epoch": 69, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 8, "modified": "2023-09-23T03:21:40.311897+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6924 } } } ], "standbys": [ { "addr": "172.21.15.154:6837/1609915000", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6836", "nonce": 1609915000, "type": "v2" }, { "addr": "172.21.15.154:6837", "nonce": 1609915000, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 50, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5279, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.190:6835/1240415394", "addrs": { "addrvec": [ { "addr": "172.21.15.190:6834", "nonce": 1240415394, "type": "v2" }, { "addr": "172.21.15.190:6835", "nonce": 1240415394, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 52, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5288, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.190:6833/878668736", "addrs": { "addrvec": [ { "addr": "172.21.15.190:6832", "nonce": 878668736, "type": "v2" }, { "addr": "172.21.15.190:6833", "nonce": 878668736, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 51, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5728, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-09-23T03:21:42.786 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-23T03:21:43.037 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.035+0000 7fad129d0640 1 Processor -- start 2023-09-23T03:21:43.037 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.035+0000 7fad129d0640 1 -- start start 2023-09-23T03:21:43.038 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.037+0000 7fad129d0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad0c15caf0 0x7fad0c0ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:43.038 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.037+0000 7fad129d0640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad0c0cd530 con 0x7fad0c160f90 2023-09-23T03:21:43.038 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.037+0000 7fad129d0640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad0c0cd810 con 0x7fad0c0cee70 2023-09-23T03:21:43.038 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.037+0000 7fad129d0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c15ced0 con 0x7fad0c15caf0 2023-09-23T03:21:43.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.037+0000 7fad10f6d640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fad0c160f90 0x7fad0c0b1360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:38396/0 (socket says 172.21.15.154:38396) 2023-09-23T03:21:43.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.037+0000 7fad10f6d640 1 -- 172.21.15.154:0/2855994184 learned_addr learned my addr 172.21.15.154:0/2855994184 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:21:43.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.038+0000 7fad0bfff640 1 --2- 172.21.15.154:0/2855994184 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad0c15caf0 0x7fad0c0ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:43.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.038+0000 7fad0affd640 1 -- 172.21.15.154:0/2855994184 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 515021441 0 0) 0x7fad0c0cd530 con 0x7fad0c160f90 2023-09-23T03:21:43.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.038+0000 7fad0affd640 1 -- 172.21.15.154:0/2855994184 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facf4003520 con 0x7fad0c160f90 2023-09-23T03:21:43.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.038+0000 7fad0affd640 1 -- 172.21.15.154:0/2855994184 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2120564632 0 0) 0x7facf4003520 con 0x7fad0c160f90 2023-09-23T03:21:43.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.038+0000 7fad0affd640 1 -- 172.21.15.154:0/2855994184 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad0c15caf0 msgr2=0x7fad0c0ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:43.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.038+0000 7fad0affd640 1 --2- 172.21.15.154:0/2855994184 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad0c15caf0 0x7fad0c0ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:43.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.038+0000 7fad0affd640 1 -- 172.21.15.154:0/2855994184 >> v1:172.21.15.190:6789/0 conn(0x7fad0c0cee70 legacy=0x7fad0c160880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:43.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.038+0000 7fad0affd640 1 -- 172.21.15.154:0/2855994184 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad0c0cdaf0 con 0x7fad0c160f90 2023-09-23T03:21:43.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.038+0000 7fad0affd640 1 -- 172.21.15.154:0/2855994184 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fad04001670 con 0x7fad0c160f90 2023-09-23T03:21:43.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.038+0000 7fad0affd640 1 -- 172.21.15.154:0/2855994184 >> v1:172.21.15.154:6789/0 conn(0x7fad0c160f90 legacy=0x7fad0c0b1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:43.042 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.038+0000 7fad0affd640 1 --2- 172.21.15.154:0/2855994184 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facf40039a0 0x7facf4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:43.042 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.039+0000 7fad0affd640 1 --2- 172.21.15.154:0/2855994184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facf40047c0 0x7facf4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:43.042 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.039+0000 7fad0affd640 1 --2- 172.21.15.154:0/2855994184 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad0c15caf0 0x7facf4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:43.042 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.039+0000 7fad0affd640 1 -- 172.21.15.154:0/2855994184 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c15ced0 con 0x7facf40039a0 2023-09-23T03:21:43.042 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.039+0000 7fad0affd640 1 -- 172.21.15.154:0/2855994184 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf400ba30 con 0x7facf40047c0 2023-09-23T03:21:43.042 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.039+0000 7fad0affd640 1 -- 172.21.15.154:0/2855994184 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf400bbf0 con 0x7fad0c15caf0 2023-09-23T03:21:43.042 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.039+0000 7fad0bfff640 1 --2- 172.21.15.154:0/2855994184 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facf40039a0 0x7facf4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:43.042 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.039+0000 7fad10f6d640 1 --2- 172.21.15.154:0/2855994184 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad0c15caf0 0x7facf4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:43.042 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.039+0000 7fad0bfff640 1 -- 172.21.15.154:0/2855994184 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad0c15caf0 msgr2=0x7facf4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:43.042 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.039+0000 7fad0b7fe640 1 --2- 172.21.15.154:0/2855994184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facf40047c0 0x7facf4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:43.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.039+0000 7fad0bfff640 1 --2- 172.21.15.154:0/2855994184 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad0c15caf0 0x7facf4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:43.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.039+0000 7fad0bfff640 1 -- 172.21.15.154:0/2855994184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facf40047c0 msgr2=0x7facf4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:43.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.039+0000 7fad0bfff640 1 --2- 172.21.15.154:0/2855994184 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facf40047c0 0x7facf4004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:43.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.039+0000 7fad0bfff640 1 -- 172.21.15.154:0/2855994184 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf4003520 con 0x7facf40039a0 2023-09-23T03:21:43.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.039+0000 7fad10f6d640 1 --2- 172.21.15.154:0/2855994184 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad0c15caf0 0x7facf4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:43.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.040+0000 7fad0bfff640 1 --2- 172.21.15.154:0/2855994184 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facf40039a0 0x7facf4003dc0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7facfc00e140 tx=0x7facf40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b638033755d91ca5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:43.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.041+0000 7fad0affd640 1 -- 172.21.15.154:0/2855994184 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facfc02cc20 con 0x7facf40039a0 2023-09-23T03:21:43.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.041+0000 7fad0affd640 1 -- 172.21.15.154:0/2855994184 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facfc00f040 con 0x7facf40039a0 2023-09-23T03:21:43.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.041+0000 7fad0affd640 1 -- 172.21.15.154:0/2855994184 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facfc034440 con 0x7facf40039a0 2023-09-23T03:21:43.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.041+0000 7fad129d0640 1 -- 172.21.15.154:0/2855994184 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facf40039a0 msgr2=0x7facf4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:43.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.041+0000 7fad129d0640 1 --2- 172.21.15.154:0/2855994184 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facf40039a0 0x7facf4003dc0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7facfc00e140 tx=0x7facf40057f0 comp rx=0 tx=0).stop 2023-09-23T03:21:43.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.041+0000 7fad10f6d640 1 -- 172.21.15.154:0/2855994184 reap_dead start 2023-09-23T03:21:43.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.041+0000 7fad129d0640 1 -- 172.21.15.154:0/2855994184 shutdown_connections 2023-09-23T03:21:43.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.041+0000 7fad129d0640 1 -- 172.21.15.154:0/2855994184 >> 172.21.15.154:0/2855994184 conn(0x7fad0c06a0f0 msgr2=0x7fad0c06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:43.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.041+0000 7fad129d0640 1 -- 172.21.15.154:0/2855994184 shutdown_connections 2023-09-23T03:21:43.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.041+0000 7fad129d0640 1 -- 172.21.15.154:0/2855994184 wait complete. 2023-09-23T03:21:43.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.043+0000 7fad129d0640 1 Processor -- start 2023-09-23T03:21:43.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.043+0000 7fad129d0640 1 -- start start 2023-09-23T03:21:43.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.044+0000 7fad129d0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facf40039a0 0x7fad0c0c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:43.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.044+0000 7fad129d0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad0c0c1ff0 0x7fad0c0c2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:43.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.044+0000 7fad129d0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad0c155ee0 0x7fad0c156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:43.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.044+0000 7fad129d0640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c0ae760 con 0x7fad0c155ee0 2023-09-23T03:21:43.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.044+0000 7fad129d0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c0c4f50 con 0x7facf40039a0 2023-09-23T03:21:43.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.044+0000 7fad129d0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c0c2990 con 0x7fad0c0c1ff0 2023-09-23T03:21:43.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.044+0000 7fad10f6d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad0c155ee0 0x7fad0c156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:43.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.045+0000 7fad10f6d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad0c155ee0 0x7fad0c156360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:32840/0 (socket says 172.21.15.154:32840) 2023-09-23T03:21:43.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.045+0000 7fad10f6d640 1 -- 172.21.15.154:0/21445855 learned_addr learned my addr 172.21.15.154:0/21445855 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:43.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.045+0000 7fad0bfff640 1 --2- 172.21.15.154:0/21445855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facf40039a0 0x7fad0c0c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:43.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.045+0000 7fad0b7fe640 1 --2- 172.21.15.154:0/21445855 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad0c0c1ff0 0x7fad0c0c2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:43.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.045+0000 7fad10f6d640 1 -- 172.21.15.154:0/21445855 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad0c0c1ff0 msgr2=0x7fad0c0c2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:43.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.045+0000 7fad10f6d640 1 --2- 172.21.15.154:0/21445855 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad0c0c1ff0 0x7fad0c0c2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:43.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.045+0000 7fad10f6d640 1 -- 172.21.15.154:0/21445855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facf40039a0 msgr2=0x7fad0c0c4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:43.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.045+0000 7fad10f6d640 1 --2- 172.21.15.154:0/21445855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facf40039a0 0x7fad0c0c4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:43.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.045+0000 7fad10f6d640 1 -- 172.21.15.154:0/21445855 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad0c156f10 con 0x7fad0c155ee0 2023-09-23T03:21:43.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.045+0000 7fad10f6d640 1 --2- 172.21.15.154:0/21445855 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad0c155ee0 0x7fad0c156360 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fad04000c00 tx=0x7fad0400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:43.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.046+0000 7fad08ff9640 1 -- 172.21.15.154:0/21445855 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad040072d0 con 0x7fad0c155ee0 2023-09-23T03:21:43.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.046+0000 7fad08ff9640 1 -- 172.21.15.154:0/21445855 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad04007450 con 0x7fad0c155ee0 2023-09-23T03:21:43.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.046+0000 7fad08ff9640 1 -- 172.21.15.154:0/21445855 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad040047e0 con 0x7fad0c155ee0 2023-09-23T03:21:43.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.046+0000 7fad129d0640 1 -- 172.21.15.154:0/21445855 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad0c156c30 con 0x7fad0c155ee0 2023-09-23T03:21:43.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.046+0000 7fad129d0640 1 -- 172.21.15.154:0/21445855 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad0c0ae380 con 0x7fad0c155ee0 2023-09-23T03:21:43.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.047+0000 7fad129d0640 1 -- 172.21.15.154:0/21445855 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad0c0ae8a0 con 0x7fad0c155ee0 2023-09-23T03:21:43.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.048+0000 7fad08ff9640 1 -- 172.21.15.154:0/21445855 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fad04004960 con 0x7fad0c155ee0 2023-09-23T03:21:43.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.049+0000 7fad08ff9640 1 --2- 172.21.15.154:0/21445855 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7facf806af30 0x7facf806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:43.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.049+0000 7fad08ff9640 1 -- 172.21.15.154:0/21445855 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fad040898f0 con 0x7fad0c155ee0 2023-09-23T03:21:43.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.049+0000 7fad0bfff640 1 --2- 172.21.15.154:0/21445855 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7facf806af30 0x7facf806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:43.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.050+0000 7fad0bfff640 1 --2- 172.21.15.154:0/21445855 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7facf806af30 0x7facf806d3f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7facfc036000 tx=0x7facfc02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:43.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.053+0000 7fad08ff9640 1 -- 172.21.15.154:0/21445855 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fad0c0ae8a0 con 0x7fad0c155ee0 2023-09-23T03:21:43.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.280+0000 7fad129d0640 1 -- 172.21.15.154:0/21445855 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fad0c0ae8a0 con 0x7fad0c155ee0 2023-09-23T03:21:43.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.281+0000 7fad08ff9640 1 -- 172.21.15.154:0/21445855 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v54) v1 ==== 53+0+91 (secure 0 0 0) 0x7fad0c0ae8a0 con 0x7fad0c155ee0 2023-09-23T03:21:43.282 INFO:teuthology.orchestra.run.smithi154.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_extra ] 2023-09-23T03:21:43.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.283+0000 7fad129d0640 1 -- 172.21.15.154:0/21445855 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7facf806af30 msgr2=0x7facf806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:43.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.283+0000 7fad129d0640 1 --2- 172.21.15.154:0/21445855 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7facf806af30 0x7facf806d3f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7facfc036000 tx=0x7facfc02e000 comp rx=0 tx=0).stop 2023-09-23T03:21:43.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.283+0000 7fad129d0640 1 -- 172.21.15.154:0/21445855 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad0c155ee0 msgr2=0x7fad0c156360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:43.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.283+0000 7fad129d0640 1 --2- 172.21.15.154:0/21445855 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad0c155ee0 0x7fad0c156360 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fad04000c00 tx=0x7fad0400dc00 comp rx=0 tx=0).stop 2023-09-23T03:21:43.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.283+0000 7fad129d0640 1 -- 172.21.15.154:0/21445855 shutdown_connections 2023-09-23T03:21:43.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.283+0000 7fad129d0640 1 --2- 172.21.15.154:0/21445855 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7facf806af30 0x7facf806d3f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:43.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.283+0000 7fad129d0640 1 --2- 172.21.15.154:0/21445855 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad0c155ee0 0x7fad0c156360 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:43.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.283+0000 7fad129d0640 1 --2- 172.21.15.154:0/21445855 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad0c0c1ff0 0x7fad0c0c2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:43.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.283+0000 7fad129d0640 1 --2- 172.21.15.154:0/21445855 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facf40039a0 0x7fad0c0c4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:43.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.283+0000 7fad129d0640 1 -- 172.21.15.154:0/21445855 >> 172.21.15.154:0/21445855 conn(0x7fad0c06a0f0 msgr2=0x7fad0c070200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:43.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.283+0000 7fad129d0640 1 -- 172.21.15.154:0/21445855 shutdown_connections 2023-09-23T03:21:43.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:43.283+0000 7fad129d0640 1 -- 172.21.15.154:0/21445855 wait complete. 2023-09-23T03:21:43.303 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-23T03:21:43.303 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-23T03:21:43.303 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi154.front.sepia.ceph.com 2023-09-23T03:21:43.303 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-09-23T03:21:43.303 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-23T03:21:43.303 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-23T03:21:43.303 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-23T03:21:43.304 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-23T03:21:43.304 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-23T03:21:43.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:21:43.304 DEBUG:teuthology.orchestra.run.smithi154:> ip addr 2023-09-23T03:21:43.353 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-23T03:21:43.353 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-23T03:21:43.353 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2023-09-23T03:21:43.353 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:21:43.353 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2023-09-23T03:21:43.353 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:21:43.353 INFO:teuthology.orchestra.run.smithi154.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:21:43.354 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:54 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:21:43.354 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f0 2023-09-23T03:21:43.354 INFO:teuthology.orchestra.run.smithi154.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:21:43.354 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:55 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:21:43.354 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f1 2023-09-23T03:21:43.354 INFO:teuthology.orchestra.run.smithi154.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:21:43.354 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:60 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:21:43.354 INFO:teuthology.orchestra.run.smithi154.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-23T03:21:43.354 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:61 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:21:43.355 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.21.15.154/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-09-23T03:21:43.355 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft 42022sec preferred_lft 42022sec 2023-09-23T03:21:43.355 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::ec4:7aff:fe88:7761/64 scope link 2023-09-23T03:21:43.355 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:21:43.355 INFO:teuthology.orchestra.run.smithi154.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-23T03:21:43.355 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 5a:c6:b2:96:fb:dc brd ff:ff:ff:ff:ff:ff 2023-09-23T03:21:43.355 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-09-23T03:21:43.355 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:21:43.355 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::a801:22ff:febf:2833/64 scope link 2023-09-23T03:21:43.356 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:21:43.356 INFO:teuthology.orchestra.run.smithi154.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-23T03:21:43.356 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 5a:c6:b2:96:fb:dc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:21:43.356 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::58c6:b2ff:fe96:fbdc/64 scope link 2023-09-23T03:21:43.356 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:21:43.356 INFO:teuthology.orchestra.run.smithi154.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-23T03:21:43.356 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether b2:23:b0:e4:c2:ab brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:21:43.356 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::b023:b0ff:fee4:c2ab/64 scope link 2023-09-23T03:21:43.356 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:21:43.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:21:43.357 DEBUG:teuthology.orchestra.run.smithi154:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-23T03:21:43.442 INFO:teuthology.orchestra.run.smithi154.stdout:1 2023-09-23T03:21:43.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:21:43.446 DEBUG:teuthology.orchestra.run.smithi154:> route 2023-09-23T03:21:43.498 INFO:teuthology.orchestra.run.smithi154.stdout:Kernel IP routing table 2023-09-23T03:21:43.498 INFO:teuthology.orchestra.run.smithi154.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-23T03:21:43.498 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-23T03:21:43.498 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-09-23T03:21:43.498 INFO:teuthology.orchestra.run.smithi154.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-23T03:21:43.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:21:43.499 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-09-23T03:21:43.499 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-09-23T03:21:43.499 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-23T03:21:43.499 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-23T03:21:43.499 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-23T03:21:43.499 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-09-23T03:21:43.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:21:43.686 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2023-09-23T03:21:43.737 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-09-23T03:21:43.738 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-09-23T03:21:43.739 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-23T03:21:43.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:21:43.739 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set brx.0 up 2023-09-23T03:21:43.834 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-09-23T03:21:43.835 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-23T03:21:43.835 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-09-23T03:21:43.889 INFO:teuthology.orchestra.run.smithi154.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-09-23T03:21:43.889 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-23T03:21:43.889 DEBUG:teuthology.orchestra.run.smithi154:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-09-23T03:21:43.941 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-09-23T03:21:44.028 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2023-09-23T03:21:44.081 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2023-09-23T03:21:44.081 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2023-09-23T03:21:44.081 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2023-09-23T03:21:44.081 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2023-09-23T03:21:44.081 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2023-09-23T03:21:44.081 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2023-09-23T03:21:44.081 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2023-09-23T03:21:44.081 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2023-09-23T03:21:44.082 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2023-09-23T03:21:44.082 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2023-09-23T03:21:44.082 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2023-09-23T03:21:44.082 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T03:21:44.082 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2023-09-23T03:21:44.082 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2023-09-23T03:21:44.082 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2023-09-23T03:21:44.082 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2023-09-23T03:21:44.082 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2023-09-23T03:21:44.082 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2023-09-23T03:21:44.082 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2023-09-23T03:21:44.082 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-09-23T03:21:44.083 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-09-23T03:21:44.083 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2023-09-23T03:21:44.083 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2023-09-23T03:21:44.083 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2023-09-23T03:21:44.083 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-0 2023-09-23T03:21:44.083 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-1 2023-09-23T03:21:44.083 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-2 2023-09-23T03:21:44.083 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-3 2023-09-23T03:21:44.083 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2023-09-23T03:21:44.083 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:21:44.083 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:21:44.084 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:21:44.084 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:21:44.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:21:44.085 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-09-23T03:21:44.134 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-23T03:21:44.134 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-09-23T03:21:44.140 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-09-23T03:21:44.221 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2023-09-23T03:21:44.265 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2023-09-23T03:21:44.265 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2023-09-23T03:21:44.265 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2023-09-23T03:21:44.265 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2023-09-23T03:21:44.265 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2023-09-23T03:21:44.265 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2023-09-23T03:21:44.265 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2023-09-23T03:21:44.265 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2023-09-23T03:21:44.265 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2023-09-23T03:21:44.265 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2023-09-23T03:21:44.266 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2023-09-23T03:21:44.266 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T03:21:44.266 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2023-09-23T03:21:44.266 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2023-09-23T03:21:44.266 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2023-09-23T03:21:44.266 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2023-09-23T03:21:44.266 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2023-09-23T03:21:44.267 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2023-09-23T03:21:44.267 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2023-09-23T03:21:44.267 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-09-23T03:21:44.267 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-09-23T03:21:44.267 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2023-09-23T03:21:44.267 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2023-09-23T03:21:44.267 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2023-09-23T03:21:44.267 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-0 2023-09-23T03:21:44.268 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-1 2023-09-23T03:21:44.268 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-2 2023-09-23T03:21:44.268 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-3 2023-09-23T03:21:44.268 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2023-09-23T03:21:44.268 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:21:44.268 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:21:44.268 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:21:44.269 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:21:44.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:21:44.270 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-09-23T03:21:44.341 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:2023-09-23T03:21:44.340+0000 7f9a0ea35480 -1 init, newargv = 0x557edeb71cc0 newargc=15 2023-09-23T03:21:44.342 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:ceph-fuse[112314]: starting ceph client 2023-09-23T03:21:44.350 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:ceph-fuse[112314]: starting fuse 2023-09-23T03:21:44.360 INFO:teuthology.orchestra.run.smithi154.stdout:39 2023-09-23T03:21:44.360 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-09-23T03:21:44.360 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:21:44.360 DEBUG:teuthology.orchestra.run.smithi154:> import glob 2023-09-23T03:21:44.360 DEBUG:teuthology.orchestra.run.smithi154:> import re 2023-09-23T03:21:44.360 DEBUG:teuthology.orchestra.run.smithi154:> import os 2023-09-23T03:21:44.360 DEBUG:teuthology.orchestra.run.smithi154:> import subprocess 2023-09-23T03:21:44.360 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:21:44.360 DEBUG:teuthology.orchestra.run.smithi154:> def _find_admin_socket(client_name): 2023-09-23T03:21:44.361 DEBUG:teuthology.orchestra.run.smithi154:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-09-23T03:21:44.361 DEBUG:teuthology.orchestra.run.smithi154:> files = glob.glob(asok_path) 2023-09-23T03:21:44.361 DEBUG:teuthology.orchestra.run.smithi154:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-09-23T03:21:44.361 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:21:44.361 DEBUG:teuthology.orchestra.run.smithi154:> # Given a non-glob path, it better be there 2023-09-23T03:21:44.361 DEBUG:teuthology.orchestra.run.smithi154:> if "*" not in asok_path: 2023-09-23T03:21:44.361 DEBUG:teuthology.orchestra.run.smithi154:> assert(len(files) == 1) 2023-09-23T03:21:44.361 DEBUG:teuthology.orchestra.run.smithi154:> return files[0] 2023-09-23T03:21:44.361 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:21:44.361 DEBUG:teuthology.orchestra.run.smithi154:> for f in files: 2023-09-23T03:21:44.361 DEBUG:teuthology.orchestra.run.smithi154:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-23T03:21:44.361 DEBUG:teuthology.orchestra.run.smithi154:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-23T03:21:44.361 DEBUG:teuthology.orchestra.run.smithi154:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-23T03:21:44.362 DEBUG:teuthology.orchestra.run.smithi154:> contents = proc_f.read() 2023-09-23T03:21:44.362 DEBUG:teuthology.orchestra.run.smithi154:> if mountpoint in contents: 2023-09-23T03:21:44.362 DEBUG:teuthology.orchestra.run.smithi154:> return f 2023-09-23T03:21:44.362 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-23T03:21:44.362 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:21:44.362 DEBUG:teuthology.orchestra.run.smithi154:> print(_find_admin_socket("client.0")) 2023-09-23T03:21:44.362 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:21:44.565 INFO:teuthology.orchestra.run.smithi154.stdout:/var/run/ceph/ceph-client.0.112314.asok 2023-09-23T03:21:44.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:21:44.582 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.112314.asok 2023-09-23T03:21:44.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:21:44.583 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.112314.asok status 2023-09-23T03:21:44.824 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:21:44.824 INFO:teuthology.orchestra.run.smithi154.stdout: "metadata": { 2023-09-23T03:21:44.824 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_sha1": "862663913a74a99e1c52d50e6a7e59dd6705749a", 2023-09-23T03:21:44.825 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_version": "ceph version 18.0.0-6336-g86266391 (862663913a74a99e1c52d50e6a7e59dd6705749a) reef (dev)", 2023-09-23T03:21:44.825 INFO:teuthology.orchestra.run.smithi154.stdout: "entity_id": "0", 2023-09-23T03:21:44.825 INFO:teuthology.orchestra.run.smithi154.stdout: "hostname": "smithi154", 2023-09-23T03:21:44.825 INFO:teuthology.orchestra.run.smithi154.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-09-23T03:21:44.825 INFO:teuthology.orchestra.run.smithi154.stdout: "pid": "112314", 2023-09-23T03:21:44.825 INFO:teuthology.orchestra.run.smithi154.stdout: "root": "/" 2023-09-23T03:21:44.825 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:21:44.825 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_count": 0, 2023-09-23T03:21:44.825 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_pinned_count": 0, 2023-09-23T03:21:44.825 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 7002, 2023-09-23T03:21:44.825 INFO:teuthology.orchestra.run.smithi154.stdout: "inst": { 2023-09-23T03:21:44.826 INFO:teuthology.orchestra.run.smithi154.stdout: "name": { 2023-09-23T03:21:44.826 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "client", 2023-09-23T03:21:44.826 INFO:teuthology.orchestra.run.smithi154.stdout: "num": 7002 2023-09-23T03:21:44.826 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:21:44.826 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2023-09-23T03:21:44.826 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2023-09-23T03:21:44.826 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.1:0", 2023-09-23T03:21:44.826 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3272805584 2023-09-23T03:21:44.826 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:21:44.826 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:21:44.827 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2023-09-23T03:21:44.827 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2023-09-23T03:21:44.827 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.1:0", 2023-09-23T03:21:44.827 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3272805584 2023-09-23T03:21:44.827 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:21:44.827 INFO:teuthology.orchestra.run.smithi154.stdout: "inst_str": "client.7002 192.168.0.1:0/3272805584", 2023-09-23T03:21:44.827 INFO:teuthology.orchestra.run.smithi154.stdout: "addr_str": "192.168.0.1:0/3272805584", 2023-09-23T03:21:44.827 INFO:teuthology.orchestra.run.smithi154.stdout: "inode_count": 1, 2023-09-23T03:21:44.827 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_epoch": 54, 2023-09-23T03:21:44.827 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch": 71, 2023-09-23T03:21:44.828 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch_barrier": 0, 2023-09-23T03:21:44.828 INFO:teuthology.orchestra.run.smithi154.stdout: "blocklisted": false, 2023-09-23T03:21:44.828 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs" 2023-09-23T03:21:44.828 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:21:44.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:21:44.839 DEBUG:teuthology.orchestra.run.smithi154:> stat --file-system '--printf=%T 2023-09-23T03:21:44.840 DEBUG:teuthology.orchestra.run.smithi154:> ' -- /home/ubuntu/cephtest/mnt.0 2023-09-23T03:21:44.894 INFO:teuthology.orchestra.run.smithi154.stdout:fuseblk 2023-09-23T03:21:44.894 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-09-23T03:21:44.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:21:44.894 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-09-23T03:21:44.991 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2023-09-23T03:21:45.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.217+0000 7f93be461640 1 Processor -- start 2023-09-23T03:21:45.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.218+0000 7f93be461640 1 -- start start 2023-09-23T03:21:45.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.219+0000 7f93be461640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f93b80b2860 0x7f93b815cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.219+0000 7f93be461640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93b8160b20 con 0x7f93b806cb60 2023-09-23T03:21:45.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.219+0000 7f93be461640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93b815dab0 con 0x7f93b815ca00 2023-09-23T03:21:45.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.219+0000 7f93be461640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93b815e070 con 0x7f93b80b2860 2023-09-23T03:21:45.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.220+0000 7f93b77fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f93b806cb60 0x7f93b80b2150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:54312/0 (socket says 172.21.15.154:54312) 2023-09-23T03:21:45.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.220+0000 7f93b77fe640 1 -- 172.21.15.154:0/1406893369 learned_addr learned my addr 172.21.15.154:0/1406893369 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:21:45.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.220+0000 7f93bc9fe640 1 --2- 172.21.15.154:0/1406893369 >> v2:172.21.15.190:3300/0 conn(0x7f93b80b2860 0x7f93b815cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:45.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.220+0000 7f93b6ffd640 1 -- 172.21.15.154:0/1406893369 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 385817051 0 0) 0x7f93b8160b20 con 0x7f93b806cb60 2023-09-23T03:21:45.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.220+0000 7f93b6ffd640 1 -- 172.21.15.154:0/1406893369 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93a4003520 con 0x7f93b806cb60 2023-09-23T03:21:45.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.220+0000 7f93b6ffd640 1 -- 172.21.15.154:0/1406893369 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3294146496 0 0) 0x7f93a4003520 con 0x7f93b806cb60 2023-09-23T03:21:45.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.220+0000 7f93b6ffd640 1 -- 172.21.15.154:0/1406893369 >> v2:172.21.15.190:3300/0 conn(0x7f93b80b2860 msgr2=0x7f93b815cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:45.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.220+0000 7f93b6ffd640 1 --2- 172.21.15.154:0/1406893369 >> v2:172.21.15.190:3300/0 conn(0x7f93b80b2860 0x7f93b815cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.220+0000 7f93b6ffd640 1 -- 172.21.15.154:0/1406893369 >> v1:172.21.15.190:6789/0 conn(0x7f93b815ca00 legacy=0x7f93b8160fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:45.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.220+0000 7f93b6ffd640 1 -- 172.21.15.154:0/1406893369 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b815dd90 con 0x7f93b806cb60 2023-09-23T03:21:45.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.221+0000 7f93b6ffd640 1 -- 172.21.15.154:0/1406893369 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f93a0001670 con 0x7f93b806cb60 2023-09-23T03:21:45.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.221+0000 7f93b6ffd640 1 -- 172.21.15.154:0/1406893369 >> v1:172.21.15.154:6789/0 conn(0x7f93b806cb60 legacy=0x7f93b80b2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:45.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.221+0000 7f93b6ffd640 1 --2- 172.21.15.154:0/1406893369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93a4003970 0x7f93a4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.221+0000 7f93b6ffd640 1 --2- 172.21.15.154:0/1406893369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93a40047c0 0x7f93a4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.221+0000 7f93b6ffd640 1 --2- 172.21.15.154:0/1406893369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93a40052a0 0x7f93a400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.221+0000 7f93b6ffd640 1 -- 172.21.15.154:0/1406893369 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b815e070 con 0x7f93a4003970 2023-09-23T03:21:45.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.221+0000 7f93b6ffd640 1 -- 172.21.15.154:0/1406893369 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a400c170 con 0x7f93a40052a0 2023-09-23T03:21:45.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.221+0000 7f93b6ffd640 1 -- 172.21.15.154:0/1406893369 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a400c330 con 0x7f93a40047c0 2023-09-23T03:21:45.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.221+0000 7f93b7fff640 1 --2- 172.21.15.154:0/1406893369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93a4003970 0x7f93a4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:45.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.221+0000 7f93b7fff640 1 -- 172.21.15.154:0/1406893369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93a40047c0 msgr2=0x7f93a4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:45.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.221+0000 7f93b7fff640 1 --2- 172.21.15.154:0/1406893369 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93a40047c0 0x7f93a4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.221+0000 7f93b7fff640 1 -- 172.21.15.154:0/1406893369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93a40052a0 msgr2=0x7f93a400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:45.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.221+0000 7f93b7fff640 1 --2- 172.21.15.154:0/1406893369 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93a40052a0 0x7f93a400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.221+0000 7f93bc9fe640 1 -- 172.21.15.154:0/1406893369 reap_dead start 2023-09-23T03:21:45.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.221+0000 7f93b7fff640 1 -- 172.21.15.154:0/1406893369 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a4003520 con 0x7f93a4003970 2023-09-23T03:21:45.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.222+0000 7f93b7fff640 1 --2- 172.21.15.154:0/1406893369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93a4003970 0x7f93a4003d90 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f93a8001720 tx=0x7f93a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b67408039da6e3da server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:45.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.223+0000 7f93b6ffd640 1 -- 172.21.15.154:0/1406893369 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a80052d0 con 0x7f93a4003970 2023-09-23T03:21:45.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.223+0000 7f93b6ffd640 1 -- 172.21.15.154:0/1406893369 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a802e050 con 0x7f93a4003970 2023-09-23T03:21:45.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.223+0000 7f93b6ffd640 1 -- 172.21.15.154:0/1406893369 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a8003830 con 0x7f93a4003970 2023-09-23T03:21:45.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.223+0000 7f93be461640 1 -- 172.21.15.154:0/1406893369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93a4003970 msgr2=0x7f93a4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:45.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.223+0000 7f93be461640 1 --2- 172.21.15.154:0/1406893369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93a4003970 0x7f93a4003d90 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f93a8001720 tx=0x7f93a40057f0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.223+0000 7f93be461640 1 -- 172.21.15.154:0/1406893369 shutdown_connections 2023-09-23T03:21:45.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.223+0000 7f93be461640 1 --2- 172.21.15.154:0/1406893369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93a4003970 0x7f93a4003d90 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.223+0000 7f93be461640 1 -- 172.21.15.154:0/1406893369 >> 172.21.15.154:0/1406893369 conn(0x7f93b80cd150 msgr2=0x7f93b80cd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:45.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.223+0000 7f93be461640 1 -- 172.21.15.154:0/1406893369 shutdown_connections 2023-09-23T03:21:45.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.224+0000 7f93be461640 1 -- 172.21.15.154:0/1406893369 wait complete. 2023-09-23T03:21:45.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.225+0000 7f93be461640 1 Processor -- start 2023-09-23T03:21:45.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.225+0000 7f93be461640 1 -- start start 2023-09-23T03:21:45.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.226+0000 7f93be461640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93a4003970 0x7f93b8150700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.226+0000 7f93be461640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93b80b3d00 0x7f93b80cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.226+0000 7f93be461640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93b80b4160 0x7f93b80cc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.226+0000 7f93be461640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b8158e40 con 0x7f93a4003970 2023-09-23T03:21:45.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.226+0000 7f93be461640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b8150c40 con 0x7f93b80b4160 2023-09-23T03:21:45.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.226+0000 7f93be461640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b80b49e0 con 0x7f93b80b3d00 2023-09-23T03:21:45.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.227+0000 7f93b7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93a4003970 0x7f93b8150700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:45.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.227+0000 7f93b7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93a4003970 0x7f93b8150700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60322/0 (socket says 172.21.15.154:60322) 2023-09-23T03:21:45.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.227+0000 7f93b7fff640 1 -- 172.21.15.154:0/2240299913 learned_addr learned my addr 172.21.15.154:0/2240299913 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:45.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.227+0000 7f93b77fe640 1 --2- 172.21.15.154:0/2240299913 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93b80b3d00 0x7f93b80cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:45.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.227+0000 7f93b7fff640 1 -- 172.21.15.154:0/2240299913 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93b80b3d00 msgr2=0x7f93b80cbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:45.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.227+0000 7f93bc9fe640 1 --2- 172.21.15.154:0/2240299913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93b80b4160 0x7f93b80cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:45.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.227+0000 7f93b7fff640 1 --2- 172.21.15.154:0/2240299913 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93b80b3d00 0x7f93b80cbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.227+0000 7f93b7fff640 1 -- 172.21.15.154:0/2240299913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93b80b4160 msgr2=0x7f93b80cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:45.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.227+0000 7f93b7fff640 1 --2- 172.21.15.154:0/2240299913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93b80b4160 0x7f93b80cc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.227+0000 7f93b7fff640 1 -- 172.21.15.154:0/2240299913 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b80b4cc0 con 0x7f93a4003970 2023-09-23T03:21:45.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.227+0000 7f93bc9fe640 1 --2- 172.21.15.154:0/2240299913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93b80b4160 0x7f93b80cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:45.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.227+0000 7f93b77fe640 1 --2- 172.21.15.154:0/2240299913 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93b80b3d00 0x7f93b80cbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:45.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.227+0000 7f93b7fff640 1 --2- 172.21.15.154:0/2240299913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93a4003970 0x7f93b8150700 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f93a802bbe0 tx=0x7f93a8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:45.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.228+0000 7f93b4ff9640 1 -- 172.21.15.154:0/2240299913 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a80076c0 con 0x7f93a4003970 2023-09-23T03:21:45.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.228+0000 7f93b4ff9640 1 -- 172.21.15.154:0/2240299913 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a8007a30 con 0x7f93a4003970 2023-09-23T03:21:45.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.228+0000 7f93b4ff9640 1 -- 172.21.15.154:0/2240299913 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a8006be0 con 0x7f93a4003970 2023-09-23T03:21:45.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.228+0000 7f93be461640 1 -- 172.21.15.154:0/2240299913 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93b80b4fa0 con 0x7f93a4003970 2023-09-23T03:21:45.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.228+0000 7f93be461640 1 -- 172.21.15.154:0/2240299913 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b80b5560 con 0x7f93a4003970 2023-09-23T03:21:45.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.230+0000 7f93b4ff9640 1 -- 172.21.15.154:0/2240299913 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f93a8006d40 con 0x7f93a4003970 2023-09-23T03:21:45.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.231+0000 7f93b4ff9640 1 --2- 172.21.15.154:0/2240299913 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f939806af30 0x7f939806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.231+0000 7f93b4ff9640 1 -- 172.21.15.154:0/2240299913 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f93a80ab8d0 con 0x7f93a4003970 2023-09-23T03:21:45.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.232+0000 7f93b77fe640 1 --2- 172.21.15.154:0/2240299913 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f939806af30 0x7f939806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:45.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.232+0000 7f93c0ff4740 1 Processor -- start 2023-09-23T03:21:45.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.232+0000 7f93b77fe640 1 --2- 172.21.15.154:0/2240299913 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f939806af30 0x7f939806d3f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f93a0001eb0 tx=0x7f93a0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:45.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.232+0000 7f93c0ff4740 1 -- start start 2023-09-23T03:21:45.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.233+0000 7f93c0ff4740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599e599ee70 0x5599e5971d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.233+0000 7f93c0ff4740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599e599f250 0x5599e59a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.233+0000 7f93c0ff4740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5599e59722b0 0x5599e59a2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.233+0000 7f93c0ff4740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e58e54e0 con 0x5599e599ee70 2023-09-23T03:21:45.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.233+0000 7f93c0ff4740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e58e2bd0 con 0x5599e599f250 2023-09-23T03:21:45.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.233+0000 7f93c0ff4740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599e58de4c0 con 0x5599e59722b0 2023-09-23T03:21:45.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.233+0000 7f93b7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599e599ee70 0x5599e5971d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:45.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.233+0000 7f93b7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599e599ee70 0x5599e5971d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60330/0 (socket says 172.21.15.154:60330) 2023-09-23T03:21:45.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.233+0000 7f93b7fff640 1 -- 172.21.15.154:0/3561301673 learned_addr learned my addr 172.21.15.154:0/3561301673 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:45.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.234+0000 7f93b7fff640 1 -- 172.21.15.154:0/3561301673 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5599e59722b0 msgr2=0x5599e59a2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:45.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.234+0000 7f93b7fff640 1 --2- 172.21.15.154:0/3561301673 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5599e59722b0 0x5599e59a2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.234+0000 7f93b7fff640 1 -- 172.21.15.154:0/3561301673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599e599f250 msgr2=0x5599e59a1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:45.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.234+0000 7f93b7fff640 1 --2- 172.21.15.154:0/3561301673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599e599f250 0x5599e59a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.234+0000 7f93b7fff640 1 -- 172.21.15.154:0/3561301673 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599e587bcb0 con 0x5599e599ee70 2023-09-23T03:21:45.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.234+0000 7f93b7fff640 1 --2- 172.21.15.154:0/3561301673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599e599ee70 0x5599e5971d70 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f93a80050a0 tx=0x7f93a807a910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de69380c2221706 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:45.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.234+0000 7f93adffb640 1 -- 172.21.15.154:0/3561301673 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a8082450 con 0x5599e599ee70 2023-09-23T03:21:45.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.234+0000 7f93adffb640 1 -- 172.21.15.154:0/3561301673 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a80784f0 con 0x5599e599ee70 2023-09-23T03:21:45.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.234+0000 7f93adffb640 1 -- 172.21.15.154:0/3561301673 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a8076860 con 0x5599e599ee70 2023-09-23T03:21:45.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.235+0000 7f93c0ff4740 1 -- 172.21.15.154:0/3561301673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599e599ee70 msgr2=0x5599e5971d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:45.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.235+0000 7f93c0ff4740 1 --2- 172.21.15.154:0/3561301673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599e599ee70 0x5599e5971d70 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f93a80050a0 tx=0x7f93a807a910 comp rx=0 tx=0).stop 2023-09-23T03:21:45.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.235+0000 7f93c0ff4740 1 -- 172.21.15.154:0/3561301673 shutdown_connections 2023-09-23T03:21:45.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.235+0000 7f93c0ff4740 1 --2- 172.21.15.154:0/3561301673 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5599e59722b0 0x5599e59a2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.235+0000 7f93c0ff4740 1 --2- 172.21.15.154:0/3561301673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599e599f250 0x5599e59a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.235+0000 7f93c0ff4740 1 --2- 172.21.15.154:0/3561301673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599e599ee70 0x5599e5971d70 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.235+0000 7f93c0ff4740 1 -- 172.21.15.154:0/3561301673 >> 172.21.15.154:0/3561301673 conn(0x5599e5974080 msgr2=0x5599e59bcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:45.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.235+0000 7f93c0ff4740 1 -- 172.21.15.154:0/3561301673 shutdown_connections 2023-09-23T03:21:45.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.235+0000 7f93c0ff4740 1 -- 172.21.15.154:0/3561301673 wait complete. 2023-09-23T03:21:45.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.236+0000 7f93c0ff4740 1 -- start start 2023-09-23T03:21:45.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.236+0000 7f93c0ff4740 1 Processor -- start 2023-09-23T03:21:45.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.237+0000 7f93c0ff4740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599e599ee70 0x5599e59b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.237+0000 7f93c0ff4740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599e599f250 0x5599e598cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.237+0000 7f93c0ff4740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5599e5992830 0x5599e5992cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.237+0000 7f93c0ff4740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e587c1e0 con 0x5599e599ee70 2023-09-23T03:21:45.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.237+0000 7f93c0ff4740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e587be80 con 0x5599e599f250 2023-09-23T03:21:45.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.237+0000 7f93c0ff4740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599e56430c0 con 0x5599e5992830 2023-09-23T03:21:45.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.237+0000 7f93c0ff4740 10 client.0 _ll_register_callbacks cb 0x5599e58d3d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:21:45.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.238+0000 7f93b7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599e599ee70 0x5599e59b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:45.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.238+0000 7f93b7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599e599ee70 0x5599e59b69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60344/0 (socket says 172.21.15.154:60344) 2023-09-23T03:21:45.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.238+0000 7f93b7fff640 1 -- 172.21.15.154:0/1608931953 learned_addr learned my addr 172.21.15.154:0/1608931953 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:45.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.238+0000 7f93b77fe640 1 --2- 172.21.15.154:0/1608931953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599e599f250 0x5599e598cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:45.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.238+0000 7f93bc9fe640 1 --2- 172.21.15.154:0/1608931953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5599e5992830 0x5599e5992cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:45.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.238+0000 7f93b7fff640 1 -- 172.21.15.154:0/1608931953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5599e5992830 msgr2=0x5599e5992cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:45.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.238+0000 7f93b7fff640 1 --2- 172.21.15.154:0/1608931953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5599e5992830 0x5599e5992cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.238+0000 7f93b7fff640 1 -- 172.21.15.154:0/1608931953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599e599f250 msgr2=0x5599e598cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:45.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.238+0000 7f93b7fff640 1 --2- 172.21.15.154:0/1608931953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599e599f250 0x5599e598cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.238+0000 7f93b7fff640 1 -- 172.21.15.154:0/1608931953 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b80b5560 con 0x5599e599ee70 2023-09-23T03:21:45.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.238+0000 7f93bc9fe640 1 --2- 172.21.15.154:0/1608931953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5599e5992830 0x5599e5992cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:45.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.238+0000 7f93b77fe640 1 --2- 172.21.15.154:0/1608931953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599e599f250 0x5599e598cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:45.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.238+0000 7f93b7fff640 1 --2- 172.21.15.154:0/1608931953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599e599ee70 0x5599e59b69d0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f93a807d040 tx=0x7f93a8007c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ed569ae9ec7880e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:45.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.238+0000 7f9396ffd640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:21:45.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.239+0000 7f9396ffd640 1 -- 172.21.15.154:0/1608931953 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a8079840 con 0x5599e599ee70 2023-09-23T03:21:45.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.239+0000 7f9396ffd640 1 -- 172.21.15.154:0/1608931953 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f93a8082450 con 0x5599e599ee70 2023-09-23T03:21:45.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.239+0000 7f93c0ff4740 1 -- 172.21.15.154:0/1608931953 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599e58d74a0 con 0x5599e599ee70 2023-09-23T03:21:45.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.239+0000 7f9396ffd640 1 -- 172.21.15.154:0/1608931953 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93a8076490 con 0x5599e599ee70 2023-09-23T03:21:45.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.239+0000 7f93c0ff4740 10 client.7014 fetch_fsmap learned FSMap version 54 2023-09-23T03:21:45.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.239+0000 7f93c0ff4740 1 -- 172.21.15.154:0/1608931953 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x5599e58dcb40 con 0x5599e599ee70 2023-09-23T03:21:45.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.239+0000 7f9396ffd640 1 -- 172.21.15.154:0/1608931953 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5599e58d74a0 con 0x5599e599ee70 2023-09-23T03:21:45.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.239+0000 7f9396ffd640 1 -- 172.21.15.154:0/1608931953 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 54) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f93a804d020 con 0x5599e599ee70 2023-09-23T03:21:45.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.239+0000 7f93c0ff4740 10 client.7014 fetch_fsmap finished waiting for FSMap version 54 2023-09-23T03:21:45.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.239+0000 7f93c0ff4740 10 client.7014 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-09-23T03:21:45.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.239+0000 7f93c0ff4740 20 client.7014 populate_metadata read hostname 'smithi154' 2023-09-23T03:21:45.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.239+0000 7f93c0ff4740 1 --2- 172.21.15.154:0/1608931953 >> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] conn(0x5599e598f880 0x5599e598fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.239+0000 7f93c0ff4740 4 client.7014 mds_command: new command op to 6924 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T03:21:45.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.239+0000 7f93c0ff4740 1 -- 172.21.15.154:0/1608931953 --> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599e58d71c0 con 0x5599e598f880 2023-09-23T03:21:45.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.240+0000 7f93b77fe640 1 --2- 172.21.15.154:0/1608931953 >> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] conn(0x5599e598f880 0x5599e598fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:45.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.240+0000 7f9396ffd640 10 client.7014 ms_handle_connect on v2:172.21.15.154:6834/2633473586 2023-09-23T03:21:45.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.240+0000 7f93b77fe640 1 --2- 172.21.15.154:0/1608931953 >> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] conn(0x5599e598f880 0x5599e598fc60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebc1f5dae6e3e645 server_cookie=90fdb130bb0369c9 in_seq=0 out_seq=0 2023-09-23T03:21:45.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.241+0000 7f9396ffd640 1 -- 172.21.15.154:0/1608931953 <== mds.0 v2:172.21.15.154:6834/2633473586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5599e58d71c0 con 0x5599e598f880 2023-09-23T03:21:45.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.241+0000 7f9396ffd640 10 client.7014 handle_command_reply: tid=0 2023-09-23T03:21:45.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.241+0000 7f93c0ff4740 1 client.7014 shutdown 2023-09-23T03:21:45.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.241+0000 7f93c0ff4740 1 -- 172.21.15.154:0/1608931953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599e599ee70 msgr2=0x5599e59b69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:45.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.241+0000 7f93c0ff4740 1 --2- 172.21.15.154:0/1608931953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599e599ee70 0x5599e59b69d0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f93a807d040 tx=0x7f93a8007c00 comp rx=0 tx=0).stop 2023-09-23T03:21:45.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.241+0000 7f93c0ff4740 1 -- 172.21.15.154:0/1608931953 shutdown_connections 2023-09-23T03:21:45.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.241+0000 7f93c0ff4740 1 --2- 172.21.15.154:0/1608931953 >> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] conn(0x5599e598f880 0x5599e598fc60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.241+0000 7f93c0ff4740 1 --2- 172.21.15.154:0/1608931953 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5599e5992830 0x5599e5992cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.241+0000 7f93c0ff4740 1 --2- 172.21.15.154:0/1608931953 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599e599f250 0x5599e598cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.241+0000 7f93c0ff4740 1 --2- 172.21.15.154:0/1608931953 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599e599ee70 0x5599e59b69d0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.241+0000 7f9396ffd640 0 client.7014 ms_handle_reset on v2:172.21.15.154:6834/2633473586 2023-09-23T03:21:45.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.241+0000 7f93c0ff4740 1 -- 172.21.15.154:0/1608931953 >> 172.21.15.154:0/1608931953 conn(0x5599e59722b0 msgr2=0x5599e5975e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:45.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.241+0000 7f93c0ff4740 1 -- 172.21.15.154:0/1608931953 shutdown_connections 2023-09-23T03:21:45.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.241+0000 7f93c0ff4740 1 -- 172.21.15.154:0/1608931953 wait complete. 2023-09-23T03:21:45.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.242+0000 7f93c0ff4740 20 client.7014 trim_cache size 0 max 16384 2023-09-23T03:21:45.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.258+0000 7f93c0ff4740 1 Processor -- start 2023-09-23T03:21:45.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.258+0000 7f93c0ff4740 1 -- start start 2023-09-23T03:21:45.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.258+0000 7f93c0ff4740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599e5992830 0x5599e5975870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.258+0000 7f93c0ff4740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5599e599ee70 0x5599e5975db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.259+0000 7f93c0ff4740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599e599f250 0x5599e598c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.259+0000 7f93c0ff4740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e598c610 con 0x5599e599f250 2023-09-23T03:21:45.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.259+0000 7f93c0ff4740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e59cfeb0 con 0x5599e5992830 2023-09-23T03:21:45.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.259+0000 7f93c0ff4740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599e58947e0 con 0x5599e599ee70 2023-09-23T03:21:45.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.259+0000 7f93bc9fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599e599f250 0x5599e598c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:45.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.259+0000 7f93bc9fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599e599f250 0x5599e598c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60356/0 (socket says 172.21.15.154:60356) 2023-09-23T03:21:45.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.259+0000 7f93bc9fe640 1 -- 172.21.15.154:0/979904777 learned_addr learned my addr 172.21.15.154:0/979904777 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:45.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.259+0000 7f93b7fff640 1 --2- 172.21.15.154:0/979904777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599e5992830 0x5599e5975870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:45.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.259+0000 7f93b77fe640 1 --2- 172.21.15.154:0/979904777 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5599e599ee70 0x5599e5975db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:45.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.259+0000 7f93bc9fe640 1 -- 172.21.15.154:0/979904777 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5599e599ee70 msgr2=0x5599e5975db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:45.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.259+0000 7f93bc9fe640 1 --2- 172.21.15.154:0/979904777 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5599e599ee70 0x5599e5975db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.259+0000 7f93bc9fe640 1 -- 172.21.15.154:0/979904777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599e5992830 msgr2=0x5599e5975870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:45.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.259+0000 7f93bc9fe640 1 --2- 172.21.15.154:0/979904777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599e5992830 0x5599e5975870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.259+0000 7f93bc9fe640 1 -- 172.21.15.154:0/979904777 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599e587b540 con 0x5599e599f250 2023-09-23T03:21:45.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.259+0000 7f93b77fe640 1 --2- 172.21.15.154:0/979904777 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5599e599ee70 0x5599e5975db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:45.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.260+0000 7f93bc9fe640 1 --2- 172.21.15.154:0/979904777 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599e599f250 0x5599e598c990 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f93b806f6e0 tx=0x7f93b0008730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=957a22c23519f898 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:45.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.260+0000 7f93ae7fc640 1 -- 172.21.15.154:0/979904777 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b0003da0 con 0x5599e599f250 2023-09-23T03:21:45.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.260+0000 7f93ae7fc640 1 -- 172.21.15.154:0/979904777 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b0008e50 con 0x5599e599f250 2023-09-23T03:21:45.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.260+0000 7f93ae7fc640 1 -- 172.21.15.154:0/979904777 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b0004270 con 0x5599e599f250 2023-09-23T03:21:45.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.261+0000 7f93c0ff4740 1 -- 172.21.15.154:0/979904777 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599e599f250 msgr2=0x5599e598c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:45.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.261+0000 7f93c0ff4740 1 --2- 172.21.15.154:0/979904777 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599e599f250 0x5599e598c990 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f93b806f6e0 tx=0x7f93b0008730 comp rx=0 tx=0).stop 2023-09-23T03:21:45.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.261+0000 7f93c0ff4740 1 -- 172.21.15.154:0/979904777 shutdown_connections 2023-09-23T03:21:45.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.261+0000 7f93c0ff4740 1 --2- 172.21.15.154:0/979904777 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599e599f250 0x5599e598c990 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.261+0000 7f93c0ff4740 1 --2- 172.21.15.154:0/979904777 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5599e599ee70 0x5599e5975db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.261+0000 7f93c0ff4740 1 --2- 172.21.15.154:0/979904777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599e5992830 0x5599e5975870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.261+0000 7f93c0ff4740 1 -- 172.21.15.154:0/979904777 >> 172.21.15.154:0/979904777 conn(0x5599e59722b0 msgr2=0x5599e5970ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:45.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.261+0000 7f93c0ff4740 1 -- 172.21.15.154:0/979904777 shutdown_connections 2023-09-23T03:21:45.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.261+0000 7f93c0ff4740 1 -- 172.21.15.154:0/979904777 wait complete. 2023-09-23T03:21:45.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.262+0000 7f93c0ff4740 1 -- start start 2023-09-23T03:21:45.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.262+0000 7f93c0ff4740 1 Processor -- start 2023-09-23T03:21:45.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.263+0000 7f93c0ff4740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5599e5992830 0x5599e598a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.263+0000 7f93c0ff4740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599e599ee70 0x5599e5988350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.263+0000 7f93c0ff4740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599e599f250 0x5599e5988890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.263+0000 7f93c0ff4740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e598ac40 con 0x5599e599ee70 2023-09-23T03:21:45.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.263+0000 7f93c0ff4740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599e599e4f0 con 0x5599e599f250 2023-09-23T03:21:45.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.263+0000 7f93c0ff4740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599e599e740 con 0x5599e5992830 2023-09-23T03:21:45.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.263+0000 7f93c0ff4740 10 client.0 _ll_register_callbacks cb 0x7f93a8079fa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:21:45.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.263+0000 7f93b77fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599e599ee70 0x5599e5988350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:45.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.263+0000 7f93b77fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599e599ee70 0x5599e5988350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60364/0 (socket says 172.21.15.154:60364) 2023-09-23T03:21:45.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.263+0000 7f93b77fe640 1 -- 172.21.15.154:0/4177047444 learned_addr learned my addr 172.21.15.154:0/4177047444 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:45.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.263+0000 7f93b7fff640 1 --2- 172.21.15.154:0/4177047444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5599e5992830 0x5599e598a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:45.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.263+0000 7f93bc9fe640 1 --2- 172.21.15.154:0/4177047444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599e599f250 0x5599e5988890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:45.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.264+0000 7f93b77fe640 1 -- 172.21.15.154:0/4177047444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5599e5992830 msgr2=0x5599e598a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:45.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.264+0000 7f93b77fe640 1 --2- 172.21.15.154:0/4177047444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5599e5992830 0x5599e598a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.264+0000 7f93b77fe640 1 -- 172.21.15.154:0/4177047444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599e599f250 msgr2=0x5599e5988890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:45.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.264+0000 7f93b77fe640 1 --2- 172.21.15.154:0/4177047444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599e599f250 0x5599e5988890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.264+0000 7f93b77fe640 1 -- 172.21.15.154:0/4177047444 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b0009c90 con 0x5599e599ee70 2023-09-23T03:21:45.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.264+0000 7f93bc9fe640 1 --2- 172.21.15.154:0/4177047444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599e599f250 0x5599e5988890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:45.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.264+0000 7f93b77fe640 1 --2- 172.21.15.154:0/4177047444 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599e599ee70 0x5599e5988350 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f93b80b6600 tx=0x7f93a0001060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fcb5238bbda5856 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:45.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.264+0000 7f9396ffd640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:21:45.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.264+0000 7f9396ffd640 1 -- 172.21.15.154:0/4177047444 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a0021b90 con 0x5599e599ee70 2023-09-23T03:21:45.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.264+0000 7f93c0ff4740 1 -- 172.21.15.154:0/4177047444 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93a80784f0 con 0x5599e599ee70 2023-09-23T03:21:45.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.264+0000 7f9396ffd640 1 -- 172.21.15.154:0/4177047444 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f93a0021cf0 con 0x5599e599ee70 2023-09-23T03:21:45.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.265+0000 7f9396ffd640 1 -- 172.21.15.154:0/4177047444 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93a000d750 con 0x5599e599ee70 2023-09-23T03:21:45.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.265+0000 7f93c0ff4740 10 client.7020 fetch_fsmap learned FSMap version 54 2023-09-23T03:21:45.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.265+0000 7f93c0ff4740 1 -- 172.21.15.154:0/4177047444 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x5599e58a0bf0 con 0x5599e599ee70 2023-09-23T03:21:45.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.265+0000 7f9396ffd640 1 -- 172.21.15.154:0/4177047444 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a80784f0 con 0x5599e599ee70 2023-09-23T03:21:45.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.265+0000 7f9396ffd640 1 -- 172.21.15.154:0/4177047444 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 54) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f93a000d8b0 con 0x5599e599ee70 2023-09-23T03:21:45.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.265+0000 7f93c0ff4740 10 client.7020 fetch_fsmap finished waiting for FSMap version 54 2023-09-23T03:21:45.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.265+0000 7f93c0ff4740 10 client.7020 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-09-23T03:21:45.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.265+0000 7f93c0ff4740 20 client.7020 populate_metadata read hostname 'smithi154' 2023-09-23T03:21:45.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.265+0000 7f93c0ff4740 1 --2- 172.21.15.154:0/4177047444 >> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] conn(0x5599e59e0530 0x5599e5987740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.265+0000 7f93c0ff4740 4 client.7020 mds_command: new command op to 6924 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2023-09-23T03:21:45.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.265+0000 7f93c0ff4740 1 -- 172.21.15.154:0/4177047444 --> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x5599e55e0860 con 0x5599e59e0530 2023-09-23T03:21:45.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.265+0000 7f93b7fff640 1 --2- 172.21.15.154:0/4177047444 >> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] conn(0x5599e59e0530 0x5599e5987740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:45.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.266+0000 7f93b7fff640 1 --2- 172.21.15.154:0/4177047444 >> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] conn(0x5599e59e0530 0x5599e5987740 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30b7d7b428862b4d server_cookie=f3d47872351134d8 in_seq=0 out_seq=0 2023-09-23T03:21:45.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.266+0000 7f9396ffd640 10 client.7020 ms_handle_connect on v2:172.21.15.154:6834/2633473586 2023-09-23T03:21:45.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.267+0000 7f9396ffd640 1 -- 172.21.15.154:0/4177047444 <== mds.0 v2:172.21.15.154:6834/2633473586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5599e55e0860 con 0x5599e59e0530 2023-09-23T03:21:45.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.267+0000 7f9396ffd640 10 client.7020 handle_command_reply: tid=0 2023-09-23T03:21:45.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.267+0000 7f93c0ff4740 1 client.7020 shutdown 2023-09-23T03:21:45.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.267+0000 7f93c0ff4740 1 -- 172.21.15.154:0/4177047444 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599e599ee70 msgr2=0x5599e5988350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:45.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.267+0000 7f93c0ff4740 1 --2- 172.21.15.154:0/4177047444 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599e599ee70 0x5599e5988350 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f93b80b6600 tx=0x7f93a0001060 comp rx=0 tx=0).stop 2023-09-23T03:21:45.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.267+0000 7f93c0ff4740 1 -- 172.21.15.154:0/4177047444 shutdown_connections 2023-09-23T03:21:45.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.267+0000 7f93c0ff4740 1 --2- 172.21.15.154:0/4177047444 >> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] conn(0x5599e59e0530 0x5599e5987740 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.267+0000 7f93c0ff4740 1 --2- 172.21.15.154:0/4177047444 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5599e599f250 0x5599e5988890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.267+0000 7f93c0ff4740 1 --2- 172.21.15.154:0/4177047444 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5599e599ee70 0x5599e5988350 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.267+0000 7f93c0ff4740 1 --2- 172.21.15.154:0/4177047444 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5599e5992830 0x5599e598a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.267+0000 7f93c0ff4740 1 -- 172.21.15.154:0/4177047444 >> 172.21.15.154:0/4177047444 conn(0x5599e59722b0 msgr2=0x5599e59cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:45.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.267+0000 7f9396ffd640 0 client.7020 ms_handle_reset on v2:172.21.15.154:6834/2633473586 2023-09-23T03:21:45.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.267+0000 7f93c0ff4740 1 -- 172.21.15.154:0/4177047444 shutdown_connections 2023-09-23T03:21:45.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.267+0000 7f93c0ff4740 1 -- 172.21.15.154:0/4177047444 wait complete. 2023-09-23T03:21:45.274 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:21:45.267+0000 7fabd8110640 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2023-09-23T03:21:45.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.267+0000 7f93c0ff4740 20 client.7020 trim_cache size 0 max 16384 2023-09-23T03:21:45.274 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:21:45.274 INFO:teuthology.orchestra.run.smithi154.stdout: "return_code": 0, 2023-09-23T03:21:45.274 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_tag": "b02bd179-3676-49e5-859e-eb986a9f809a", 2023-09-23T03:21:45.274 INFO:teuthology.orchestra.run.smithi154.stdout: "mode": "asynchronous" 2023-09-23T03:21:45.275 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:21:45.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.268+0000 7f93be461640 1 -- 172.21.15.154:0/2240299913 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f939806af30 msgr2=0x7f939806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:45.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.268+0000 7f93be461640 1 --2- 172.21.15.154:0/2240299913 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f939806af30 0x7f939806d3f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f93a0001eb0 tx=0x7f93a0009040 comp rx=0 tx=0).stop 2023-09-23T03:21:45.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.268+0000 7f93be461640 1 -- 172.21.15.154:0/2240299913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93a4003970 msgr2=0x7f93b8150700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:45.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.268+0000 7f93be461640 1 --2- 172.21.15.154:0/2240299913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93a4003970 0x7f93b8150700 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f93a802bbe0 tx=0x7f93a8001120 comp rx=0 tx=0).stop 2023-09-23T03:21:45.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.268+0000 7f93be461640 1 -- 172.21.15.154:0/2240299913 shutdown_connections 2023-09-23T03:21:45.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.268+0000 7f93be461640 1 --2- 172.21.15.154:0/2240299913 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f939806af30 0x7f939806d3f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.268+0000 7f93be461640 1 --2- 172.21.15.154:0/2240299913 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f93b80b4160 0x7f93b80cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.268+0000 7f93be461640 1 --2- 172.21.15.154:0/2240299913 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f93b80b3d00 0x7f93b80cbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.268+0000 7f93be461640 1 --2- 172.21.15.154:0/2240299913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f93a4003970 0x7f93b8150700 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.268+0000 7f93be461640 1 -- 172.21.15.154:0/2240299913 >> 172.21.15.154:0/2240299913 conn(0x7f93b80cd150 msgr2=0x7f93b815bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:45.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.268+0000 7f93be461640 1 -- 172.21.15.154:0/2240299913 shutdown_connections 2023-09-23T03:21:45.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.268+0000 7f93be461640 1 -- 172.21.15.154:0/2240299913 wait complete. 2023-09-23T03:21:45.278 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:21:45.269+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /subdir/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x562c45591b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000000:[<0x1/subdir v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-09-23T03:21:45.278 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:21:45.272+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f8 [2,head] /subdir/3 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c4566e100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001f8:[<0x10000000000/3 v40>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x100000001f8:[<0x10000000000/3 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:21:45.279 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:21:45.272+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fd [2,head] /subdir/8 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c4566c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001fd:[<0x10000000000/8 v80>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x100000001fd:[<0x10000000000/8 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:21:45.279 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:21:45.273+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fe [2,head] /subdir/9 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c4566c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001fe:[<0x10000000000/9 v88>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x100000001fe:[<0x10000000000/9 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:21:45.279 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:21:45.273+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000204 [2,head] /subdir/15 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c45676680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000204:[<0x10000000000/15 v138>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x10000000204:[<0x10000000000/15 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:21:45.279 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:21:45.274+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000201 [2,head] /subdir/12 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c45677700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000201:[<0x10000000000/12 v114>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x10000000201:[<0x10000000000/12 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:21:45.279 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:21:45.274+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000209 [2,head] /subdir/20 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c45674580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000209:[<0x10000000000/20 v178>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x10000000209:[<0x10000000000/20 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:21:45.279 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:21:45.275+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000208 [2,head] /subdir/19 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c45675080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000208:[<0x10000000000/19 v170>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x10000000208:[<0x10000000000/19 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:21:45.279 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:21:45.275+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020a [2,head] /subdir/21 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c45674000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x1000000020a:[<0x10000000000/21 v186>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x1000000020a:[<0x10000000000/21 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:21:45.279 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:21:45.275+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000200 [2,head] /subdir/11 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c4567c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000200:[<0x10000000000/11 v104>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x10000000200:[<0x10000000000/11 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:21:45.279 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:21:45.276+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001ff [2,head] /subdir/10 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c4567c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001ff:[<0x10000000000/10 v96>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x100000001ff:[<0x10000000000/10 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:21:45.279 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:21:45.276+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f7 [2,head] /subdir/2 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c4566e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001f7:[<0x10000000000/2 v32>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x100000001f7:[<0x10000000000/2 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:21:45.280 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:21:45.276+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f9 [2,head] /subdir/4 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c4566db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001f9:[<0x10000000000/4 v48>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x100000001f9:[<0x10000000000/4 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:21:45.280 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:21:45.276+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f6 [2,head] /subdir/1 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c45674b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001f6:[<0x10000000000/1 v24>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x100000001f6:[<0x10000000000/1 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:21:45.280 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:21:45.276+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020d [2,head] /subdir/24 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c4566ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x1000000020d:[<0x10000000000/24 v202>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x1000000020d:[<0x10000000000/24 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:21:45.280 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:21:45.277+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/0 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c4567cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000001:[<0x10000000000/0 v16>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x10000000001:[<0x10000000000/0 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:21:45.280 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:21:45.277+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020b [2,head] /subdir/22 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c4566f700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x1000000020b:[<0x10000000000/22 v194>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x1000000020b:[<0x10000000000/22 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:21:45.280 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:21:45.277+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000205 [2,head] /subdir/16 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c45676100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000205:[<0x10000000000/16 v146>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x10000000205:[<0x10000000000/16 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:21:45.281 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:21:45.278+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fa [2,head] /subdir/5 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c4566d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001fa:[<0x10000000000/5 v56>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x100000001fa:[<0x10000000000/5 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:21:45.281 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:21:45.278+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000202 [2,head] /subdir/13 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c45677180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000202:[<0x10000000000/13 v122>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x10000000202:[<0x10000000000/13 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:21:45.281 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:21:45.278+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020c [2,head] /subdir/23 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c4566f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x1000000020c:[<0x10000000000/23 v200>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x1000000020c:[<0x10000000000/23 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:21:45.281 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:21:45.278+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000203 [2,head] /subdir/14 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c45676c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000203:[<0x10000000000/14 v130>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x10000000203:[<0x10000000000/14 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:21:45.282 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:21:45.279+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000207 [2,head] /subdir/18 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c45675600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000207:[<0x10000000000/18 v162>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x10000000207:[<0x10000000000/18 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:21:45.282 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:21:45.279+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fb [2,head] /subdir/6 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c4566d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001fb:[<0x10000000000/6 v64>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x100000001fb:[<0x10000000000/6 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:21:45.282 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:21:45.279+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fc [2,head] /subdir/7 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c4566cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001fc:[<0x10000000000/7 v72>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x100000001fc:[<0x10000000000/7 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:21:45.282 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:21:45.279+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000206 [2,head] /subdir/17 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c45675b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000206:[<0x10000000000/17 v154>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x10000000206:[<0x10000000000/17 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:21:45.282 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:21:45.280+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v5 ap=5 snaprealm=0x562c4558a6c0 DIRTYPARENT f(v1 1=0+1) n(v0 rc2023-09-23T03:21:44.986581+0000 1=0+1)/n() (idft sync r=1) (inest sync r=1) (ifile sync->mix r=1) (iversion lock) caps={7002=pLsXs/-@3} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x562c45591600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=0+1)","ondisk_value.rstat":"n(v0 2=0+2)","memory_value.dirstat":"f(v1 1=0+1)","memory_value.rstat":"n(v0 rc2023-09-23T03:21:44.986581+0000 1=0+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-09-23T03:21:45.291 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-09-23T03:21:45.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.495+0000 7f7a095e7640 1 Processor -- start 2023-09-23T03:21:45.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.495+0000 7f7a095e7640 1 -- start start 2023-09-23T03:21:45.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.496+0000 7f7a095e7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a0415bb30 0x7f7a040caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.497+0000 7f7a095e7640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7a040cb3e0 0x7f7a040cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.497+0000 7f7a095e7640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a0415b6a0 con 0x7f7a0415aba0 2023-09-23T03:21:45.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.497+0000 7f7a095e7640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a040cb870 con 0x7f7a040cb3e0 2023-09-23T03:21:45.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.497+0000 7f7a095e7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0415bf70 con 0x7f7a0415bb30 2023-09-23T03:21:45.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.497+0000 7f7a037fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7a040cb3e0 0x7f7a040cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:45.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.497+0000 7f7a027fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a0415bb30 0x7f7a040caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:45.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.497+0000 7f7a037fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7a040cb3e0 0x7f7a040cc5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.154:50240/0 (socket says 172.21.15.154:50240) 2023-09-23T03:21:45.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.497+0000 7f7a037fe640 1 -- 172.21.15.154:0/1456792624 learned_addr learned my addr 172.21.15.154:0/1456792624 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:45.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.498+0000 7f7a027fc640 1 -- 172.21.15.154:0/1456792624 >> v2:172.21.15.190:3300/0 conn(0x7f7a040cb3e0 msgr2=0x7f7a040cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:45.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.498+0000 7f7a027fc640 1 --2- 172.21.15.154:0/1456792624 >> v2:172.21.15.190:3300/0 conn(0x7f7a040cb3e0 0x7f7a040cc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.498+0000 7f7a027fc640 1 -- 172.21.15.154:0/1456792624 >> v1:172.21.15.190:6789/0 conn(0x7f7a0415aba0 legacy=0x7f7a0415e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:45.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.498+0000 7f7a027fc640 1 -- 172.21.15.154:0/1456792624 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a0415f0b0 con 0x7f7a0415bb30 2023-09-23T03:21:45.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.498+0000 7f7a037fe640 1 --2- 172.21.15.154:0/1456792624 >> v2:172.21.15.190:3300/0 conn(0x7f7a040cb3e0 0x7f7a040cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:21:45.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.499+0000 7f7a027fc640 1 --2- 172.21.15.154:0/1456792624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a0415bb30 0x7f7a040caea0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f79f80097a0 tx=0x7f79f802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ca471da4dce1478 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:45.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.500+0000 7f7a01ffb640 1 -- 172.21.15.154:0/1456792624 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f802d040 con 0x7f7a0415bb30 2023-09-23T03:21:45.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.500+0000 7f7a01ffb640 1 -- 172.21.15.154:0/1456792624 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f8031810 con 0x7f7a0415bb30 2023-09-23T03:21:45.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.500+0000 7f7a01ffb640 1 -- 172.21.15.154:0/1456792624 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f8031ba0 con 0x7f7a0415bb30 2023-09-23T03:21:45.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.501+0000 7f7a095e7640 1 -- 172.21.15.154:0/1456792624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a0415bb30 msgr2=0x7f7a040caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:45.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.501+0000 7f7a095e7640 1 --2- 172.21.15.154:0/1456792624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a0415bb30 0x7f7a040caea0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f79f80097a0 tx=0x7f79f802b8b0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.501+0000 7f7a095e7640 1 -- 172.21.15.154:0/1456792624 shutdown_connections 2023-09-23T03:21:45.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.501+0000 7f7a095e7640 1 --2- 172.21.15.154:0/1456792624 >> v2:172.21.15.190:3300/0 conn(0x7f7a040cb3e0 0x7f7a040cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.501+0000 7f7a095e7640 1 --2- 172.21.15.154:0/1456792624 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a0415bb30 0x7f7a040caea0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.501+0000 7f7a095e7640 1 -- 172.21.15.154:0/1456792624 >> 172.21.15.154:0/1456792624 conn(0x7f7a0406c010 msgr2=0x7f7a0406c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:45.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.501+0000 7f7a095e7640 1 -- 172.21.15.154:0/1456792624 shutdown_connections 2023-09-23T03:21:45.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.501+0000 7f7a095e7640 1 -- 172.21.15.154:0/1456792624 wait complete. 2023-09-23T03:21:45.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.503+0000 7f7a095e7640 1 Processor -- start 2023-09-23T03:21:45.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.503+0000 7f7a095e7640 1 -- start start 2023-09-23T03:21:45.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.504+0000 7f7a095e7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a040cb3e0 0x7f7a04157e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.504+0000 7f7a095e7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a0415aba0 0x7f7a040c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.504+0000 7f7a095e7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a0415bb30 0x7f7a041e4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.504+0000 7f7a095e7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a040c98b0 con 0x7f7a0415aba0 2023-09-23T03:21:45.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.504+0000 7f7a095e7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a040c8c30 con 0x7f7a040cb3e0 2023-09-23T03:21:45.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.504+0000 7f7a095e7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a040c8f10 con 0x7f7a0415bb30 2023-09-23T03:21:45.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.504+0000 7f7a027fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a0415aba0 0x7f7a040c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:45.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.504+0000 7f7a027fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a0415aba0 0x7f7a040c9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60374/0 (socket says 172.21.15.154:60374) 2023-09-23T03:21:45.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.504+0000 7f7a027fc640 1 -- 172.21.15.154:0/1015361787 learned_addr learned my addr 172.21.15.154:0/1015361787 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:45.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.505+0000 7f7a02ffd640 1 --2- 172.21.15.154:0/1015361787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a040cb3e0 0x7f7a04157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:45.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.505+0000 7f7a037fe640 1 --2- 172.21.15.154:0/1015361787 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a0415bb30 0x7f7a041e4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:45.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.505+0000 7f7a027fc640 1 -- 172.21.15.154:0/1015361787 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a0415bb30 msgr2=0x7f7a041e4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:45.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.505+0000 7f7a027fc640 1 --2- 172.21.15.154:0/1015361787 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a0415bb30 0x7f7a041e4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.505+0000 7f7a027fc640 1 -- 172.21.15.154:0/1015361787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a040cb3e0 msgr2=0x7f7a04157e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:45.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.505+0000 7f7a027fc640 1 --2- 172.21.15.154:0/1015361787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a040cb3e0 0x7f7a04157e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.505+0000 7f7a027fc640 1 -- 172.21.15.154:0/1015361787 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79fc009b30 con 0x7f7a0415aba0 2023-09-23T03:21:45.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.505+0000 7f7a027fc640 1 --2- 172.21.15.154:0/1015361787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a0415aba0 0x7f7a040c9cd0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f79f8000c00 tx=0x7f79f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:45.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.505+0000 7f79eb7fe640 1 -- 172.21.15.154:0/1015361787 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f8006e10 con 0x7f7a0415aba0 2023-09-23T03:21:45.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.505+0000 7f79eb7fe640 1 -- 172.21.15.154:0/1015361787 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f803ad60 con 0x7f7a0415aba0 2023-09-23T03:21:45.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.505+0000 7f79eb7fe640 1 -- 172.21.15.154:0/1015361787 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f8044610 con 0x7f7a0415aba0 2023-09-23T03:21:45.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.505+0000 7f7a095e7640 1 -- 172.21.15.154:0/1015361787 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f8009400 con 0x7f7a0415aba0 2023-09-23T03:21:45.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.506+0000 7f7a095e7640 1 -- 172.21.15.154:0/1015361787 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a040c94d0 con 0x7f7a0415aba0 2023-09-23T03:21:45.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.508+0000 7f79eb7fe640 1 -- 172.21.15.154:0/1015361787 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f79f804d450 con 0x7f7a0415aba0 2023-09-23T03:21:45.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.509+0000 7f79eb7fe640 1 --2- 172.21.15.154:0/1015361787 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f79f006af60 0x7f79f006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.509+0000 7f79eb7fe640 1 -- 172.21.15.154:0/1015361787 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f79f80abe10 con 0x7f7a0415aba0 2023-09-23T03:21:45.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.509+0000 7f7a02ffd640 1 --2- 172.21.15.154:0/1015361787 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f79f006af60 0x7f79f006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:45.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.509+0000 7f7a0c179740 1 Processor -- start 2023-09-23T03:21:45.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.510+0000 7f7a02ffd640 1 --2- 172.21.15.154:0/1015361787 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f79f006af60 0x7f79f006d420 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7a040b4470 tx=0x7f79f4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:45.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.510+0000 7f7a0c179740 1 -- start start 2023-09-23T03:21:45.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.511+0000 7f7a0c179740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a10ddc690 0x564a10daf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.511+0000 7f7a0c179740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a10ddca70 0x564a10ddf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.511+0000 7f7a0c179740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564a10dafad0 0x564a10ddf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.511+0000 7f7a0c179740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a10d21a50 con 0x564a10ddc690 2023-09-23T03:21:45.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.511+0000 7f7a0c179740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a10d1f140 con 0x564a10ddca70 2023-09-23T03:21:45.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.511+0000 7f7a0c179740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x564a10d1aa30 con 0x564a10dafad0 2023-09-23T03:21:45.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.511+0000 7f7a02ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a10ddc690 0x564a10daf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:45.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.511+0000 7f7a02ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a10ddc690 0x564a10daf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60380/0 (socket says 172.21.15.154:60380) 2023-09-23T03:21:45.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.511+0000 7f7a02ffd640 1 -- 172.21.15.154:0/1107302457 learned_addr learned my addr 172.21.15.154:0/1107302457 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:45.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.511+0000 7f7a037fe640 1 --2- 172.21.15.154:0/1107302457 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564a10dafad0 0x564a10ddf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:45.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.511+0000 7f7a027fc640 1 --2- 172.21.15.154:0/1107302457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a10ddca70 0x564a10ddf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:45.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.511+0000 7f7a02ffd640 1 -- 172.21.15.154:0/1107302457 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564a10dafad0 msgr2=0x564a10ddf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:45.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.511+0000 7f7a02ffd640 1 --2- 172.21.15.154:0/1107302457 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564a10dafad0 0x564a10ddf960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.511+0000 7f7a02ffd640 1 -- 172.21.15.154:0/1107302457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a10ddca70 msgr2=0x564a10ddf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:45.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.511+0000 7f7a02ffd640 1 --2- 172.21.15.154:0/1107302457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a10ddca70 0x564a10ddf420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.511+0000 7f7a02ffd640 1 -- 172.21.15.154:0/1107302457 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a10cdea70 con 0x564a10ddc690 2023-09-23T03:21:45.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.511+0000 7f7a027fc640 1 --2- 172.21.15.154:0/1107302457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a10ddca70 0x564a10ddf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:45.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.512+0000 7f7a02ffd640 1 --2- 172.21.15.154:0/1107302457 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a10ddc690 0x564a10daf590 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f79f4001eb0 tx=0x7f79f4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2a18d198e1e4e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:45.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.512+0000 7f79e8ff9640 1 -- 172.21.15.154:0/1107302457 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f40035f0 con 0x564a10ddc690 2023-09-23T03:21:45.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.512+0000 7f79e8ff9640 1 -- 172.21.15.154:0/1107302457 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f4003750 con 0x564a10ddc690 2023-09-23T03:21:45.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.512+0000 7f79e8ff9640 1 -- 172.21.15.154:0/1107302457 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f4003ac0 con 0x564a10ddc690 2023-09-23T03:21:45.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.512+0000 7f7a0c179740 1 -- 172.21.15.154:0/1107302457 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a10ddc690 msgr2=0x564a10daf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:45.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.512+0000 7f7a0c179740 1 --2- 172.21.15.154:0/1107302457 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a10ddc690 0x564a10daf590 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f79f4001eb0 tx=0x7f79f4011000 comp rx=0 tx=0).stop 2023-09-23T03:21:45.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.513+0000 7f7a0c179740 1 -- 172.21.15.154:0/1107302457 shutdown_connections 2023-09-23T03:21:45.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.513+0000 7f7a0c179740 1 --2- 172.21.15.154:0/1107302457 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564a10dafad0 0x564a10ddf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.513+0000 7f7a0c179740 1 --2- 172.21.15.154:0/1107302457 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a10ddca70 0x564a10ddf420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.513+0000 7f7a0c179740 1 --2- 172.21.15.154:0/1107302457 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a10ddc690 0x564a10daf590 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.513+0000 7f7a0c179740 1 -- 172.21.15.154:0/1107302457 >> 172.21.15.154:0/1107302457 conn(0x564a10db18a0 msgr2=0x564a10dfa420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:45.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.513+0000 7f7a0c179740 1 -- 172.21.15.154:0/1107302457 shutdown_connections 2023-09-23T03:21:45.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.513+0000 7f7a0c179740 1 -- 172.21.15.154:0/1107302457 wait complete. 2023-09-23T03:21:45.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.513+0000 7f7a0c179740 1 -- start start 2023-09-23T03:21:45.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.514+0000 7f7a0c179740 1 Processor -- start 2023-09-23T03:21:45.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.515+0000 7f7a0c179740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564a10ddc690 0x564a10df8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.515+0000 7f7a0c179740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a10ddca70 0x564a10df5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.515+0000 7f7a0c179740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a10dfa050 0x564a10dfa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.515+0000 7f7a0c179740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a10cb8ad0 con 0x564a10dfa050 2023-09-23T03:21:45.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.515+0000 7f7a0c179740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a10cb8770 con 0x564a10ddca70 2023-09-23T03:21:45.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.515+0000 7f7a0c179740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x564a10a14f90 con 0x564a10ddc690 2023-09-23T03:21:45.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.515+0000 7f7a0c179740 10 client.0 _ll_register_callbacks cb 0x564a10cdc220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:21:45.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.515+0000 7f7a037fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a10dfa050 0x564a10dfa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:45.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.515+0000 7f7a037fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a10dfa050 0x564a10dfa4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60388/0 (socket says 172.21.15.154:60388) 2023-09-23T03:21:45.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.515+0000 7f7a037fe640 1 -- 172.21.15.154:0/976583426 learned_addr learned my addr 172.21.15.154:0/976583426 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:45.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.515+0000 7f7a02ffd640 1 --2- 172.21.15.154:0/976583426 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564a10ddc690 0x564a10df8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:45.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.515+0000 7f7a027fc640 1 --2- 172.21.15.154:0/976583426 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a10ddca70 0x564a10df5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:45.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.515+0000 7f7a037fe640 1 -- 172.21.15.154:0/976583426 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564a10ddc690 msgr2=0x564a10df8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:45.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.515+0000 7f7a037fe640 1 --2- 172.21.15.154:0/976583426 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564a10ddc690 0x564a10df8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.515+0000 7f7a037fe640 1 -- 172.21.15.154:0/976583426 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a10ddca70 msgr2=0x564a10df5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:45.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.516+0000 7f7a037fe640 1 --2- 172.21.15.154:0/976583426 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a10ddca70 0x564a10df5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.516+0000 7f7a037fe640 1 -- 172.21.15.154:0/976583426 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f400f040 con 0x564a10dfa050 2023-09-23T03:21:45.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.516+0000 7f7a037fe640 1 --2- 172.21.15.154:0/976583426 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a10dfa050 0x564a10dfa4d0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f79fc007980 tx=0x7f79fc007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4dc9fbd0d87a086 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:45.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.516+0000 7f79e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:21:45.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.516+0000 7f79e1ffb640 1 -- 172.21.15.154:0/976583426 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc0108d0 con 0x564a10dfa050 2023-09-23T03:21:45.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.516+0000 7f79e1ffb640 1 -- 172.21.15.154:0/976583426 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f79fc010a30 con 0x564a10dfa050 2023-09-23T03:21:45.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.516+0000 7f7a0c179740 1 -- 172.21.15.154:0/976583426 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a10d414d0 con 0x564a10dfa050 2023-09-23T03:21:45.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.516+0000 7f79e1ffb640 1 -- 172.21.15.154:0/976583426 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79fc010bf0 con 0x564a10dfa050 2023-09-23T03:21:45.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.516+0000 7f7a0c179740 10 client.7029 fetch_fsmap learned FSMap version 54 2023-09-23T03:21:45.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.516+0000 7f7a0c179740 1 -- 172.21.15.154:0/976583426 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x564a10d190b0 con 0x564a10dfa050 2023-09-23T03:21:45.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.517+0000 7f79e1ffb640 1 -- 172.21.15.154:0/976583426 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564a10d414d0 con 0x564a10dfa050 2023-09-23T03:21:45.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.517+0000 7f79e1ffb640 1 -- 172.21.15.154:0/976583426 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 54) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f79fc01c060 con 0x564a10dfa050 2023-09-23T03:21:45.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.517+0000 7f7a0c179740 10 client.7029 fetch_fsmap finished waiting for FSMap version 54 2023-09-23T03:21:45.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.517+0000 7f7a0c179740 10 client.7029 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-09-23T03:21:45.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.517+0000 7f7a0c179740 20 client.7029 populate_metadata read hostname 'smithi154' 2023-09-23T03:21:45.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.517+0000 7f7a0c179740 1 --2- 172.21.15.154:0/976583426 >> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] conn(0x564a10de1930 0x564a10de1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.517+0000 7f7a0c179740 4 client.7029 mds_command: new command op to 6924 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T03:21:45.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.517+0000 7f7a0c179740 1 -- 172.21.15.154:0/976583426 --> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a10cedab0 con 0x564a10de1930 2023-09-23T03:21:45.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.517+0000 7f7a02ffd640 1 --2- 172.21.15.154:0/976583426 >> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] conn(0x564a10de1930 0x564a10de1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:45.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.518+0000 7f79e1ffb640 10 client.7029 ms_handle_connect on v2:172.21.15.154:6834/2633473586 2023-09-23T03:21:45.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.518+0000 7f7a02ffd640 1 --2- 172.21.15.154:0/976583426 >> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] conn(0x564a10de1930 0x564a10de1d10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=375e890ce17600da server_cookie=306d31535fceb19c in_seq=0 out_seq=0 2023-09-23T03:21:45.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.518+0000 7f79e1ffb640 1 -- 172.21.15.154:0/976583426 <== mds.0 v2:172.21.15.154:6834/2633473586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564a10cedab0 con 0x564a10de1930 2023-09-23T03:21:45.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.518+0000 7f79e1ffb640 10 client.7029 handle_command_reply: tid=0 2023-09-23T03:21:45.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.518+0000 7f7a0c179740 1 client.7029 shutdown 2023-09-23T03:21:45.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.519+0000 7f7a0c179740 1 -- 172.21.15.154:0/976583426 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a10dfa050 msgr2=0x564a10dfa4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:45.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.519+0000 7f7a0c179740 1 --2- 172.21.15.154:0/976583426 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a10dfa050 0x564a10dfa4d0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f79fc007980 tx=0x7f79fc007e40 comp rx=0 tx=0).stop 2023-09-23T03:21:45.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.519+0000 7f7a0c179740 1 -- 172.21.15.154:0/976583426 shutdown_connections 2023-09-23T03:21:45.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.519+0000 7f7a0c179740 1 --2- 172.21.15.154:0/976583426 >> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] conn(0x564a10de1930 0x564a10de1d10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.519+0000 7f7a0c179740 1 --2- 172.21.15.154:0/976583426 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a10dfa050 0x564a10dfa4d0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.519+0000 7f7a0c179740 1 --2- 172.21.15.154:0/976583426 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a10ddca70 0x564a10df5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.519+0000 7f7a0c179740 1 --2- 172.21.15.154:0/976583426 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564a10ddc690 0x564a10df8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.519+0000 7f79e1ffb640 0 client.7029 ms_handle_reset on v2:172.21.15.154:6834/2633473586 2023-09-23T03:21:45.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.519+0000 7f7a0c179740 1 -- 172.21.15.154:0/976583426 >> 172.21.15.154:0/976583426 conn(0x564a10dafad0 msgr2=0x564a10db3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:45.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.519+0000 7f7a0c179740 1 -- 172.21.15.154:0/976583426 shutdown_connections 2023-09-23T03:21:45.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.519+0000 7f7a0c179740 1 -- 172.21.15.154:0/976583426 wait complete. 2023-09-23T03:21:45.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.519+0000 7f7a0c179740 20 client.7029 trim_cache size 0 max 16384 2023-09-23T03:21:45.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.535+0000 7f7a0c179740 1 Processor -- start 2023-09-23T03:21:45.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.535+0000 7f7a0c179740 1 -- start start 2023-09-23T03:21:45.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.536+0000 7f7a0c179740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564a10ddc690 0x564a10df3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.536+0000 7f7a0c179740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a10ddca70 0x564a10de1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.536+0000 7f7a0c179740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a10dfa050 0x564a10db5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.536+0000 7f7a0c179740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a10e09e80 con 0x564a10ddca70 2023-09-23T03:21:45.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.536+0000 7f7a0c179740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a10d0fd80 con 0x564a10dfa050 2023-09-23T03:21:45.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.536+0000 7f7a0c179740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x564a10de3960 con 0x564a10ddc690 2023-09-23T03:21:45.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.536+0000 7f7a027fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a10ddca70 0x564a10de1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:45.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.536+0000 7f7a027fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a10ddca70 0x564a10de1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60392/0 (socket says 172.21.15.154:60392) 2023-09-23T03:21:45.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.536+0000 7f7a027fc640 1 -- 172.21.15.154:0/1666293052 learned_addr learned my addr 172.21.15.154:0/1666293052 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:45.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.537+0000 7f7a027fc640 1 -- 172.21.15.154:0/1666293052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564a10ddc690 msgr2=0x564a10df3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:45.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.537+0000 7f7a027fc640 1 --2- 172.21.15.154:0/1666293052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564a10ddc690 0x564a10df3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.537+0000 7f7a027fc640 1 -- 172.21.15.154:0/1666293052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a10dfa050 msgr2=0x564a10db5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:45.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.537+0000 7f7a027fc640 1 --2- 172.21.15.154:0/1666293052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a10dfa050 0x564a10db5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.537+0000 7f7a027fc640 1 -- 172.21.15.154:0/1666293052 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a10df98e0 con 0x564a10ddca70 2023-09-23T03:21:45.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.537+0000 7f7a027fc640 1 --2- 172.21.15.154:0/1666293052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a10ddca70 0x564a10de1f00 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f79f802b8f0 tx=0x7f79f807a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89353fb56900a340 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:45.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.537+0000 7f79e97fa640 1 -- 172.21.15.154:0/1666293052 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f807e7e0 con 0x564a10ddca70 2023-09-23T03:21:45.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.537+0000 7f79e97fa640 1 -- 172.21.15.154:0/1666293052 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f8082990 con 0x564a10ddca70 2023-09-23T03:21:45.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.537+0000 7f79e97fa640 1 -- 172.21.15.154:0/1666293052 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f8076da0 con 0x564a10ddca70 2023-09-23T03:21:45.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.538+0000 7f7a0c179740 1 -- 172.21.15.154:0/1666293052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a10ddca70 msgr2=0x564a10de1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:45.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.538+0000 7f7a0c179740 1 --2- 172.21.15.154:0/1666293052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a10ddca70 0x564a10de1f00 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f79f802b8f0 tx=0x7f79f807a290 comp rx=0 tx=0).stop 2023-09-23T03:21:45.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.538+0000 7f7a0c179740 1 -- 172.21.15.154:0/1666293052 shutdown_connections 2023-09-23T03:21:45.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.538+0000 7f7a0c179740 1 --2- 172.21.15.154:0/1666293052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a10dfa050 0x564a10db5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.538+0000 7f7a0c179740 1 --2- 172.21.15.154:0/1666293052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a10ddca70 0x564a10de1f00 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.538+0000 7f7a0c179740 1 --2- 172.21.15.154:0/1666293052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564a10ddc690 0x564a10df3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.538+0000 7f7a0c179740 1 -- 172.21.15.154:0/1666293052 >> 172.21.15.154:0/1666293052 conn(0x564a10dafad0 msgr2=0x564a10dfadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:45.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.538+0000 7f7a0c179740 1 -- 172.21.15.154:0/1666293052 shutdown_connections 2023-09-23T03:21:45.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.538+0000 7f7a0c179740 1 -- 172.21.15.154:0/1666293052 wait complete. 2023-09-23T03:21:45.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.539+0000 7f7a0c179740 1 -- start start 2023-09-23T03:21:45.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.539+0000 7f7a0c179740 1 Processor -- start 2023-09-23T03:21:45.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.540+0000 7f7a0c179740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564a10ddc690 0x564a10df3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.540+0000 7f7a0c179740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a10ddca70 0x564a10df4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.540+0000 7f7a0c179740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a10dfa050 0x564a10ddaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.540+0000 7f7a0c179740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a10e12bc0 con 0x564a10dfa050 2023-09-23T03:21:45.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.540+0000 7f7a0c179740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a10daed00 con 0x564a10ddca70 2023-09-23T03:21:45.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.540+0000 7f7a0c179740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x564a10db4ce0 con 0x564a10ddc690 2023-09-23T03:21:45.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.540+0000 7f7a0c179740 10 client.0 _ll_register_callbacks cb 0x7f79f40013e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:21:45.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.540+0000 7f7a037fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a10dfa050 0x564a10ddaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:45.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.540+0000 7f7a037fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a10dfa050 0x564a10ddaf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60398/0 (socket says 172.21.15.154:60398) 2023-09-23T03:21:45.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.540+0000 7f7a037fe640 1 -- 172.21.15.154:0/4115855600 learned_addr learned my addr 172.21.15.154:0/4115855600 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:21:45.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.540+0000 7f7a02ffd640 1 --2- 172.21.15.154:0/4115855600 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564a10ddc690 0x564a10df3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:45.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.540+0000 7f7a027fc640 1 --2- 172.21.15.154:0/4115855600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a10ddca70 0x564a10df4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:45.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.540+0000 7f7a037fe640 1 -- 172.21.15.154:0/4115855600 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564a10ddc690 msgr2=0x564a10df3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:45.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.540+0000 7f7a037fe640 1 --2- 172.21.15.154:0/4115855600 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564a10ddc690 0x564a10df3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.540+0000 7f7a037fe640 1 -- 172.21.15.154:0/4115855600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a10ddca70 msgr2=0x564a10df4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:21:45.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.540+0000 7f7a037fe640 1 --2- 172.21.15.154:0/4115855600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a10ddca70 0x564a10df4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.540+0000 7f7a037fe640 1 -- 172.21.15.154:0/4115855600 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a040c94d0 con 0x564a10dfa050 2023-09-23T03:21:45.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.540+0000 7f7a02ffd640 1 --2- 172.21.15.154:0/4115855600 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564a10ddc690 0x564a10df3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:21:45.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.541+0000 7f7a037fe640 1 --2- 172.21.15.154:0/4115855600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a10dfa050 0x564a10ddaf90 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f79fc007e80 tx=0x7f79fc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a313f566da5b1798 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:21:45.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.541+0000 7f79e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:21:45.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.541+0000 7f79e1ffb640 1 -- 172.21.15.154:0/4115855600 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc010ec0 con 0x564a10dfa050 2023-09-23T03:21:45.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.541+0000 7f7a0c179740 1 -- 172.21.15.154:0/4115855600 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f79f4003750 con 0x564a10dfa050 2023-09-23T03:21:45.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.541+0000 7f79e1ffb640 1 -- 172.21.15.154:0/4115855600 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f79fc009790 con 0x564a10dfa050 2023-09-23T03:21:45.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.541+0000 7f79e1ffb640 1 -- 172.21.15.154:0/4115855600 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79fc010530 con 0x564a10dfa050 2023-09-23T03:21:45.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.541+0000 7f7a0c179740 10 client.7035 fetch_fsmap learned FSMap version 54 2023-09-23T03:21:45.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.541+0000 7f7a0c179740 1 -- 172.21.15.154:0/4115855600 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x564a10d4ada0 con 0x564a10dfa050 2023-09-23T03:21:45.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.542+0000 7f79e1ffb640 1 -- 172.21.15.154:0/4115855600 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f4003750 con 0x564a10dfa050 2023-09-23T03:21:45.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.542+0000 7f79e1ffb640 1 -- 172.21.15.154:0/4115855600 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 54) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f79fc003ba0 con 0x564a10dfa050 2023-09-23T03:21:45.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.542+0000 7f7a0c179740 10 client.7035 fetch_fsmap finished waiting for FSMap version 54 2023-09-23T03:21:45.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.542+0000 7f7a0c179740 10 client.7035 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-09-23T03:21:45.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.542+0000 7f7a0c179740 20 client.7035 populate_metadata read hostname 'smithi154' 2023-09-23T03:21:45.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.542+0000 7f7a0c179740 1 --2- 172.21.15.154:0/4115855600 >> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] conn(0x564a10dae760 0x564a10ddb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:21:45.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.542+0000 7f7a0c179740 4 client.7035 mds_command: new command op to 6924 tid=0[{"prefix": "scrub status"}] 2023-09-23T03:21:45.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.542+0000 7f7a0c179740 1 -- 172.21.15.154:0/4115855600 --> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564a10d13a10 con 0x564a10dae760 2023-09-23T03:21:45.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.542+0000 7f7a02ffd640 1 --2- 172.21.15.154:0/4115855600 >> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] conn(0x564a10dae760 0x564a10ddb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:21:45.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.542+0000 7f7a02ffd640 1 --2- 172.21.15.154:0/4115855600 >> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] conn(0x564a10dae760 0x564a10ddb4d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=160fef950834d85a server_cookie=81b2791d519329f6 in_seq=0 out_seq=0 2023-09-23T03:21:45.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.542+0000 7f79e1ffb640 10 client.7035 ms_handle_connect on v2:172.21.15.154:6834/2633473586 2023-09-23T03:21:45.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.543+0000 7f79e1ffb640 1 -- 172.21.15.154:0/4115855600 <== mds.0 v2:172.21.15.154:6834/2633473586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x564a10d13a10 con 0x564a10dae760 2023-09-23T03:21:45.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.543+0000 7f79e1ffb640 10 client.7035 handle_command_reply: tid=0 2023-09-23T03:21:45.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.543+0000 7f7a0c179740 1 client.7035 shutdown 2023-09-23T03:21:45.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.543+0000 7f7a0c179740 1 -- 172.21.15.154:0/4115855600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a10dfa050 msgr2=0x564a10ddaf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:45.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.543+0000 7f7a0c179740 1 --2- 172.21.15.154:0/4115855600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a10dfa050 0x564a10ddaf90 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f79fc007e80 tx=0x7f79fc004ab0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.543+0000 7f7a0c179740 1 -- 172.21.15.154:0/4115855600 shutdown_connections 2023-09-23T03:21:45.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.543+0000 7f7a0c179740 1 --2- 172.21.15.154:0/4115855600 >> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] conn(0x564a10dae760 0x564a10ddb4d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.543+0000 7f7a0c179740 1 --2- 172.21.15.154:0/4115855600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564a10dfa050 0x564a10ddaf90 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.543+0000 7f7a0c179740 1 --2- 172.21.15.154:0/4115855600 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x564a10ddca70 0x564a10df4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.543+0000 7f7a0c179740 1 --2- 172.21.15.154:0/4115855600 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x564a10ddc690 0x564a10df3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.543+0000 7f79e1ffb640 0 client.7035 ms_handle_reset on v2:172.21.15.154:6834/2633473586 2023-09-23T03:21:45.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.543+0000 7f7a0c179740 1 -- 172.21.15.154:0/4115855600 >> 172.21.15.154:0/4115855600 conn(0x564a10dafad0 msgr2=0x564a10dfadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:45.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.543+0000 7f7a0c179740 1 -- 172.21.15.154:0/4115855600 shutdown_connections 2023-09-23T03:21:45.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.543+0000 7f7a0c179740 1 -- 172.21.15.154:0/4115855600 wait complete. 2023-09-23T03:21:45.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.543+0000 7f7a0c179740 20 client.7035 trim_cache size 0 max 16384 2023-09-23T03:21:45.549 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:21:45.549 INFO:teuthology.orchestra.run.smithi154.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T03:21:45.550 INFO:teuthology.orchestra.run.smithi154.stdout: "scrubs": { 2023-09-23T03:21:45.550 INFO:teuthology.orchestra.run.smithi154.stdout: "b02bd179-3676-49e5-859e-eb986a9f809a": { 2023-09-23T03:21:45.550 INFO:teuthology.orchestra.run.smithi154.stdout: "path": "/", 2023-09-23T03:21:45.550 INFO:teuthology.orchestra.run.smithi154.stdout: "tag": "b02bd179-3676-49e5-859e-eb986a9f809a", 2023-09-23T03:21:45.550 INFO:teuthology.orchestra.run.smithi154.stdout: "options": "recursive,repair" 2023-09-23T03:21:45.550 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:21:45.550 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:21:45.550 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:21:45.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.544+0000 7f7a095e7640 1 -- 172.21.15.154:0/1015361787 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f79f006af60 msgr2=0x7f79f006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:45.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.544+0000 7f7a095e7640 1 --2- 172.21.15.154:0/1015361787 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f79f006af60 0x7f79f006d420 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7a040b4470 tx=0x7f79f4009040 comp rx=0 tx=0).stop 2023-09-23T03:21:45.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.544+0000 7f7a095e7640 1 -- 172.21.15.154:0/1015361787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a0415aba0 msgr2=0x7f7a040c9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:21:45.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.544+0000 7f7a095e7640 1 --2- 172.21.15.154:0/1015361787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a0415aba0 0x7f7a040c9cd0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f79f8000c00 tx=0x7f79f80046c0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.544+0000 7f7a095e7640 1 -- 172.21.15.154:0/1015361787 shutdown_connections 2023-09-23T03:21:45.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.544+0000 7f7a095e7640 1 --2- 172.21.15.154:0/1015361787 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f79f006af60 0x7f79f006d420 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.544+0000 7f7a095e7640 1 --2- 172.21.15.154:0/1015361787 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a0415bb30 0x7f7a041e4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.544+0000 7f7a095e7640 1 --2- 172.21.15.154:0/1015361787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a0415aba0 0x7f7a040c9cd0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.544+0000 7f7a095e7640 1 --2- 172.21.15.154:0/1015361787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a040cb3e0 0x7f7a04157e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:21:45.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.544+0000 7f7a095e7640 1 -- 172.21.15.154:0/1015361787 >> 172.21.15.154:0/1015361787 conn(0x7f7a0406c010 msgr2=0x7f7a0415c130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:21:45.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.544+0000 7f7a095e7640 1 -- 172.21.15.154:0/1015361787 shutdown_connections 2023-09-23T03:21:45.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:21:45.544+0000 7f7a095e7640 1 -- 172.21.15.154:0/1015361787 wait complete. 2023-09-23T03:21:45.562 INFO:tasks.cephfs.filesystem:scrub status for tag:b02bd179-3676-49e5-859e-eb986a9f809a - {'path': '/', 'tag': 'b02bd179-3676-49e5-859e-eb986a9f809a', 'options': 'recursive,repair'} 2023-09-23T03:22:12.825 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:22:12.834 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:22:15.563 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-09-23T03:22:15.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.790+0000 7f37c584b640 1 Processor -- start 2023-09-23T03:22:15.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.790+0000 7f37c584b640 1 -- start start 2023-09-23T03:22:15.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.791+0000 7f37c584b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f37c015bb30 0x7f37c00caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:22:15.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.791+0000 7f37c584b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37c00cb3e0 0x7f37c00cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:22:15.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.791+0000 7f37c584b640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37c015b6a0 con 0x7f37c015aba0 2023-09-23T03:22:15.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.791+0000 7f37c584b640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37c00cb870 con 0x7f37c015bb30 2023-09-23T03:22:15.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.791+0000 7f37c584b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37c015bf70 con 0x7f37c00cb3e0 2023-09-23T03:22:15.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.792+0000 7f37beffd640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f37c015aba0 0x7f37c015e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:60750/0 (socket says 172.21.15.154:60750) 2023-09-23T03:22:15.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.792+0000 7f37beffd640 1 -- 172.21.15.154:0/4178768086 learned_addr learned my addr 172.21.15.154:0/4178768086 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:22:15.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.792+0000 7f37be7fc640 1 --2- 172.21.15.154:0/4178768086 >> v2:172.21.15.154:3300/0 conn(0x7f37c015bb30 0x7f37c00caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:22:15.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.792+0000 7f37bf7fe640 1 --2- 172.21.15.154:0/4178768086 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37c00cb3e0 0x7f37c00cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:22:15.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.792+0000 7f37bdffb640 1 -- 172.21.15.154:0/4178768086 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1247453201 0 0) 0x7f37c015b6a0 con 0x7f37c015aba0 2023-09-23T03:22:15.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.792+0000 7f37bdffb640 1 -- 172.21.15.154:0/4178768086 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37ac003520 con 0x7f37c015aba0 2023-09-23T03:22:15.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.792+0000 7f37be7fc640 1 -- 172.21.15.154:0/4178768086 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37c00cb3e0 msgr2=0x7f37c00cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:22:15.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.792+0000 7f37be7fc640 1 --2- 172.21.15.154:0/4178768086 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37c00cb3e0 0x7f37c00cc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:22:15.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.792+0000 7f37be7fc640 1 -- 172.21.15.154:0/4178768086 >> v1:172.21.15.154:6789/0 conn(0x7f37c015aba0 legacy=0x7f37c015e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:22:15.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.792+0000 7f37be7fc640 1 -- 172.21.15.154:0/4178768086 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c015f0b0 con 0x7f37c015bb30 2023-09-23T03:22:15.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.794+0000 7f37be7fc640 1 --2- 172.21.15.154:0/4178768086 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37c015bb30 0x7f37c00caea0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f37a80097a0 tx=0x7f37a802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d55628b6c1da7a1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:22:15.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.794+0000 7f37bdffb640 1 -- 172.21.15.154:0/4178768086 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a802d040 con 0x7f37c015bb30 2023-09-23T03:22:15.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.794+0000 7f37bdffb640 1 -- 172.21.15.154:0/4178768086 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a8031810 con 0x7f37c015bb30 2023-09-23T03:22:15.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.794+0000 7f37bdffb640 1 -- 172.21.15.154:0/4178768086 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a8031ba0 con 0x7f37c015bb30 2023-09-23T03:22:15.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.794+0000 7f37c584b640 1 -- 172.21.15.154:0/4178768086 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37c015bb30 msgr2=0x7f37c00caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:22:15.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.794+0000 7f37c584b640 1 --2- 172.21.15.154:0/4178768086 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37c015bb30 0x7f37c00caea0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f37a80097a0 tx=0x7f37a802b8b0 comp rx=0 tx=0).stop 2023-09-23T03:22:15.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.795+0000 7f37c584b640 1 -- 172.21.15.154:0/4178768086 shutdown_connections 2023-09-23T03:22:15.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.795+0000 7f37c584b640 1 --2- 172.21.15.154:0/4178768086 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37c00cb3e0 0x7f37c00cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:22:15.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.795+0000 7f37c584b640 1 --2- 172.21.15.154:0/4178768086 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37c015bb30 0x7f37c00caea0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:22:15.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.795+0000 7f37c584b640 1 -- 172.21.15.154:0/4178768086 >> 172.21.15.154:0/4178768086 conn(0x7f37c006c010 msgr2=0x7f37c006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:22:15.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.795+0000 7f37c584b640 1 -- 172.21.15.154:0/4178768086 shutdown_connections 2023-09-23T03:22:15.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.795+0000 7f37c584b640 1 -- 172.21.15.154:0/4178768086 wait complete. 2023-09-23T03:22:15.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.796+0000 7f37c584b640 1 Processor -- start 2023-09-23T03:22:15.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.797+0000 7f37c584b640 1 -- start start 2023-09-23T03:22:15.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.798+0000 7f37c584b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37c00cb3e0 0x7f37c00c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:22:15.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.798+0000 7f37c584b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37c015aba0 0x7f37c01489f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:22:15.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.798+0000 7f37c584b640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37c015bb30 0x7f37c01f6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:22:15.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.798+0000 7f37c584b640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c00c7900 con 0x7f37c015aba0 2023-09-23T03:22:15.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.798+0000 7f37c584b640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c00c6cf0 con 0x7f37c015bb30 2023-09-23T03:22:15.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.798+0000 7f37c584b640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37c00c6fd0 con 0x7f37c00cb3e0 2023-09-23T03:22:15.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.798+0000 7f37be7fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37c015aba0 0x7f37c01489f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:22:15.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.798+0000 7f37be7fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37c015aba0 0x7f37c01489f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:41768/0 (socket says 172.21.15.154:41768) 2023-09-23T03:22:15.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.798+0000 7f37be7fc640 1 -- 172.21.15.154:0/4103126489 learned_addr learned my addr 172.21.15.154:0/4103126489 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:22:15.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.798+0000 7f37beffd640 1 --2- 172.21.15.154:0/4103126489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37c00cb3e0 0x7f37c00c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:22:15.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.798+0000 7f37bf7fe640 1 --2- 172.21.15.154:0/4103126489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37c015bb30 0x7f37c01f6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:22:15.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.799+0000 7f37be7fc640 1 -- 172.21.15.154:0/4103126489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37c00cb3e0 msgr2=0x7f37c00c7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:22:15.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.799+0000 7f37be7fc640 1 --2- 172.21.15.154:0/4103126489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37c00cb3e0 0x7f37c00c7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:22:15.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.799+0000 7f37be7fc640 1 -- 172.21.15.154:0/4103126489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37c015bb30 msgr2=0x7f37c01f6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:22:15.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.799+0000 7f37be7fc640 1 --2- 172.21.15.154:0/4103126489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37c015bb30 0x7f37c01f6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:22:15.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.799+0000 7f37be7fc640 1 -- 172.21.15.154:0/4103126489 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a8009400 con 0x7f37c015aba0 2023-09-23T03:22:15.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.799+0000 7f37be7fc640 1 --2- 172.21.15.154:0/4103126489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37c015aba0 0x7f37c01489f0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f37a8000c00 tx=0x7f37a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:22:15.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.799+0000 7f37b7fff640 1 -- 172.21.15.154:0/4103126489 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a8006e10 con 0x7f37c015aba0 2023-09-23T03:22:15.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.799+0000 7f37b7fff640 1 -- 172.21.15.154:0/4103126489 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a803ad60 con 0x7f37c015aba0 2023-09-23T03:22:15.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.799+0000 7f37b7fff640 1 -- 172.21.15.154:0/4103126489 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a8044630 con 0x7f37c015aba0 2023-09-23T03:22:15.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.799+0000 7f37c584b640 1 -- 172.21.15.154:0/4103126489 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37c00c72b0 con 0x7f37c015aba0 2023-09-23T03:22:15.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.800+0000 7f37c584b640 1 -- 172.21.15.154:0/4103126489 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37c00c82d0 con 0x7f37c015aba0 2023-09-23T03:22:15.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.802+0000 7f37b7fff640 1 -- 172.21.15.154:0/4103126489 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f37a804d450 con 0x7f37c015aba0 2023-09-23T03:22:15.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.803+0000 7f37b7fff640 1 --2- 172.21.15.154:0/4103126489 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f37a006af60 0x7f37a006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:22:15.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.804+0000 7f37b7fff640 1 -- 172.21.15.154:0/4103126489 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f37a80abd00 con 0x7f37c015aba0 2023-09-23T03:22:15.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.804+0000 7f37beffd640 1 --2- 172.21.15.154:0/4103126489 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f37a006af60 0x7f37a006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:22:15.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.804+0000 7f37c83d8740 1 Processor -- start 2023-09-23T03:22:15.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.804+0000 7f37c83d8740 1 -- start start 2023-09-23T03:22:15.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.805+0000 7f37beffd640 1 --2- 172.21.15.154:0/4103126489 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f37a006af60 0x7f37a006d420 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f37b0001560 tx=0x7f37b0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:22:15.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.805+0000 7f37c83d8740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557491705690 0x5574916d8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:22:15.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.805+0000 7f37c83d8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557491705a70 0x557491708420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:22:15.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.805+0000 7f37c83d8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5574916d8ad0 0x557491708960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:22:15.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.805+0000 7f37c83d8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55749164aa50 con 0x5574916d8ad0 2023-09-23T03:22:15.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.805+0000 7f37c83d8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557491648140 con 0x557491705a70 2023-09-23T03:22:15.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.805+0000 7f37c83d8740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x557491643a30 con 0x557491705690 2023-09-23T03:22:15.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.806+0000 7f37bf7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5574916d8ad0 0x557491708960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:22:15.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.806+0000 7f37bf7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5574916d8ad0 0x557491708960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:41772/0 (socket says 172.21.15.154:41772) 2023-09-23T03:22:15.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.806+0000 7f37bf7fe640 1 -- 172.21.15.154:0/1506309275 learned_addr learned my addr 172.21.15.154:0/1506309275 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:22:15.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.806+0000 7f37beffd640 1 --2- 172.21.15.154:0/1506309275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557491705690 0x5574916d8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:22:15.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.806+0000 7f37bf7fe640 1 -- 172.21.15.154:0/1506309275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557491705690 msgr2=0x5574916d8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:22:15.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.806+0000 7f37be7fc640 1 --2- 172.21.15.154:0/1506309275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557491705a70 0x557491708420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:22:15.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.806+0000 7f37bf7fe640 1 --2- 172.21.15.154:0/1506309275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557491705690 0x5574916d8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:22:15.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.806+0000 7f37bf7fe640 1 -- 172.21.15.154:0/1506309275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557491705a70 msgr2=0x557491708420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:22:15.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.806+0000 7f37bf7fe640 1 --2- 172.21.15.154:0/1506309275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557491705a70 0x557491708420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:22:15.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.806+0000 7f37bf7fe640 1 -- 172.21.15.154:0/1506309275 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557491607a70 con 0x5574916d8ad0 2023-09-23T03:22:15.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.806+0000 7f37beffd640 1 --2- 172.21.15.154:0/1506309275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557491705690 0x5574916d8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:22:15.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.806+0000 7f37bf7fe640 1 --2- 172.21.15.154:0/1506309275 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5574916d8ad0 0x557491708960 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f37b800b8c0 tx=0x7f37b800bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4649627dde9dff78 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:22:15.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.807+0000 7f37b57fa640 1 -- 172.21.15.154:0/1506309275 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b800f9c0 con 0x5574916d8ad0 2023-09-23T03:22:15.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.807+0000 7f37b57fa640 1 -- 172.21.15.154:0/1506309275 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37b800fb20 con 0x5574916d8ad0 2023-09-23T03:22:15.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.807+0000 7f37b57fa640 1 -- 172.21.15.154:0/1506309275 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b800feb0 con 0x5574916d8ad0 2023-09-23T03:22:15.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.807+0000 7f37c83d8740 1 -- 172.21.15.154:0/1506309275 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5574916d8ad0 msgr2=0x557491708960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:22:15.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.807+0000 7f37c83d8740 1 --2- 172.21.15.154:0/1506309275 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5574916d8ad0 0x557491708960 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f37b800b8c0 tx=0x7f37b800bd80 comp rx=0 tx=0).stop 2023-09-23T03:22:15.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.807+0000 7f37c83d8740 1 -- 172.21.15.154:0/1506309275 shutdown_connections 2023-09-23T03:22:15.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.807+0000 7f37c83d8740 1 --2- 172.21.15.154:0/1506309275 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5574916d8ad0 0x557491708960 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:22:15.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.807+0000 7f37c83d8740 1 --2- 172.21.15.154:0/1506309275 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557491705a70 0x557491708420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:22:15.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.807+0000 7f37c83d8740 1 --2- 172.21.15.154:0/1506309275 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557491705690 0x5574916d8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:22:15.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.807+0000 7f37c83d8740 1 -- 172.21.15.154:0/1506309275 >> 172.21.15.154:0/1506309275 conn(0x5574916da8a0 msgr2=0x557491723420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:22:15.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.807+0000 7f37c83d8740 1 -- 172.21.15.154:0/1506309275 shutdown_connections 2023-09-23T03:22:15.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.808+0000 7f37c83d8740 1 -- 172.21.15.154:0/1506309275 wait complete. 2023-09-23T03:22:15.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.808+0000 7f37c83d8740 1 -- start start 2023-09-23T03:22:15.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.808+0000 7f37c83d8740 1 Processor -- start 2023-09-23T03:22:15.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.809+0000 7f37c83d8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557491705690 0x557491703380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:22:15.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.809+0000 7f37c83d8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557491705a70 0x55749170bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:22:15.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.810+0000 7f37c83d8740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55749171c2b0 0x55749171c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:22:15.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.810+0000 7f37c83d8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574915e1ad0 con 0x557491705690 2023-09-23T03:22:15.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.810+0000 7f37c83d8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574915e1770 con 0x557491705a70 2023-09-23T03:22:15.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.810+0000 7f37c83d8740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55749133df90 con 0x55749171c2b0 2023-09-23T03:22:15.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.810+0000 7f37c83d8740 10 client.0 _ll_register_callbacks cb 0x557491605220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:22:15.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.810+0000 7f37beffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557491705690 0x557491703380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:22:15.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.810+0000 7f37beffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557491705690 0x557491703380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:41782/0 (socket says 172.21.15.154:41782) 2023-09-23T03:22:15.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.810+0000 7f37beffd640 1 -- 172.21.15.154:0/3185292488 learned_addr learned my addr 172.21.15.154:0/3185292488 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:22:15.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.810+0000 7f37bf7fe640 1 --2- 172.21.15.154:0/3185292488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55749171c2b0 0x55749171c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:22:15.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.810+0000 7f37be7fc640 1 --2- 172.21.15.154:0/3185292488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557491705a70 0x55749170bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:22:15.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.810+0000 7f37beffd640 1 -- 172.21.15.154:0/3185292488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55749171c2b0 msgr2=0x55749171c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:22:15.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.810+0000 7f37beffd640 1 --2- 172.21.15.154:0/3185292488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55749171c2b0 0x55749171c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:22:15.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.810+0000 7f37beffd640 1 -- 172.21.15.154:0/3185292488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557491705a70 msgr2=0x55749170bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:22:15.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.810+0000 7f37beffd640 1 --2- 172.21.15.154:0/3185292488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557491705a70 0x55749170bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:22:15.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.810+0000 7f37beffd640 1 -- 172.21.15.154:0/3185292488 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37b800e040 con 0x557491705690 2023-09-23T03:22:15.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.810+0000 7f37bf7fe640 1 --2- 172.21.15.154:0/3185292488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55749171c2b0 0x55749171c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:22:15.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.810+0000 7f37be7fc640 1 --2- 172.21.15.154:0/3185292488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557491705a70 0x55749170bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:22:15.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.811+0000 7f37beffd640 1 --2- 172.21.15.154:0/3185292488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557491705690 0x557491703380 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f37b0007680 tx=0x7f37b0001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ec8dbb36eaf64 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:22:15.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.811+0000 7f379e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:22:15.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.811+0000 7f379e7fc640 1 -- 172.21.15.154:0/3185292488 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b00159c0 con 0x557491705690 2023-09-23T03:22:15.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.811+0000 7f379e7fc640 1 -- 172.21.15.154:0/3185292488 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f37b0015b20 con 0x557491705690 2023-09-23T03:22:15.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.811+0000 7f37c83d8740 1 -- 172.21.15.154:0/3185292488 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55749166a4d0 con 0x557491705690 2023-09-23T03:22:15.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.811+0000 7f379e7fc640 1 -- 172.21.15.154:0/3185292488 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37b0015ce0 con 0x557491705690 2023-09-23T03:22:15.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.811+0000 7f37c83d8740 10 client.7050 fetch_fsmap learned FSMap version 54 2023-09-23T03:22:15.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.811+0000 7f37c83d8740 1 -- 172.21.15.154:0/3185292488 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x5574916420b0 con 0x557491705690 2023-09-23T03:22:15.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.811+0000 7f379e7fc640 1 -- 172.21.15.154:0/3185292488 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55749166a4d0 con 0x557491705690 2023-09-23T03:22:15.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.812+0000 7f379e7fc640 1 -- 172.21.15.154:0/3185292488 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 54) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f37b0029020 con 0x557491705690 2023-09-23T03:22:15.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.812+0000 7f37c83d8740 10 client.7050 fetch_fsmap finished waiting for FSMap version 54 2023-09-23T03:22:15.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.812+0000 7f37c83d8740 10 client.7050 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-09-23T03:22:15.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.812+0000 7f37c83d8740 20 client.7050 populate_metadata read hostname 'smithi154' 2023-09-23T03:22:15.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.812+0000 7f37c83d8740 1 --2- 172.21.15.154:0/3185292488 >> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] conn(0x557491733a30 0x557491733e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:22:15.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.812+0000 7f37c83d8740 4 client.7050 mds_command: new command op to 6924 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T03:22:15.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.812+0000 7f37c83d8740 1 -- 172.21.15.154:0/3185292488 --> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557491616ab0 con 0x557491733a30 2023-09-23T03:22:15.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.812+0000 7f37be7fc640 1 --2- 172.21.15.154:0/3185292488 >> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] conn(0x557491733a30 0x557491733e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:22:15.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.812+0000 7f37be7fc640 1 --2- 172.21.15.154:0/3185292488 >> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] conn(0x557491733a30 0x557491733e40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13674eb879258063 server_cookie=81e5f9ae3b408715 in_seq=0 out_seq=0 2023-09-23T03:22:15.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.812+0000 7f379e7fc640 10 client.7050 ms_handle_connect on v2:172.21.15.154:6834/2633473586 2023-09-23T03:22:15.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.813+0000 7f379e7fc640 1 -- 172.21.15.154:0/3185292488 <== mds.0 v2:172.21.15.154:6834/2633473586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x557491616ab0 con 0x557491733a30 2023-09-23T03:22:15.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.813+0000 7f379e7fc640 10 client.7050 handle_command_reply: tid=0 2023-09-23T03:22:15.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.813+0000 7f37c83d8740 1 client.7050 shutdown 2023-09-23T03:22:15.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.813+0000 7f37c83d8740 1 -- 172.21.15.154:0/3185292488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557491705690 msgr2=0x557491703380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:22:15.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.813+0000 7f37c83d8740 1 --2- 172.21.15.154:0/3185292488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557491705690 0x557491703380 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f37b0007680 tx=0x7f37b0001250 comp rx=0 tx=0).stop 2023-09-23T03:22:15.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.813+0000 7f37c83d8740 1 -- 172.21.15.154:0/3185292488 shutdown_connections 2023-09-23T03:22:15.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.813+0000 7f37c83d8740 1 --2- 172.21.15.154:0/3185292488 >> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] conn(0x557491733a30 0x557491733e40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:22:15.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.813+0000 7f37c83d8740 1 --2- 172.21.15.154:0/3185292488 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55749171c2b0 0x55749171c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:22:15.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.813+0000 7f37c83d8740 1 --2- 172.21.15.154:0/3185292488 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557491705a70 0x55749170bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:22:15.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.813+0000 7f37c83d8740 1 --2- 172.21.15.154:0/3185292488 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557491705690 0x557491703380 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:22:15.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.813+0000 7f379e7fc640 0 client.7050 ms_handle_reset on v2:172.21.15.154:6834/2633473586 2023-09-23T03:22:15.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.813+0000 7f37c83d8740 1 -- 172.21.15.154:0/3185292488 >> 172.21.15.154:0/3185292488 conn(0x5574916d8ad0 msgr2=0x5574916dc440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:22:15.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.814+0000 7f37c83d8740 1 -- 172.21.15.154:0/3185292488 shutdown_connections 2023-09-23T03:22:15.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.814+0000 7f37c83d8740 1 -- 172.21.15.154:0/3185292488 wait complete. 2023-09-23T03:22:15.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.814+0000 7f37c83d8740 20 client.7050 trim_cache size 0 max 16384 2023-09-23T03:22:15.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.830+0000 7f37c83d8740 1 Processor -- start 2023-09-23T03:22:15.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.830+0000 7f37c83d8740 1 -- start start 2023-09-23T03:22:15.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.831+0000 7f37c83d8740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557491705690 0x55749171bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:22:15.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.831+0000 7f37c83d8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557491705a70 0x55749171d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:22:15.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.831+0000 7f37c83d8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55749171c2b0 0x5574916dc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:22:15.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.831+0000 7f37c83d8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55749171b340 con 0x557491705a70 2023-09-23T03:22:15.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.831+0000 7f37c83d8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x557491709540 con 0x55749171c2b0 2023-09-23T03:22:15.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.831+0000 7f37c83d8740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55749171b130 con 0x557491705690 2023-09-23T03:22:15.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.831+0000 7f37be7fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557491705a70 0x55749171d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:22:15.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.831+0000 7f37be7fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557491705a70 0x55749171d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:41792/0 (socket says 172.21.15.154:41792) 2023-09-23T03:22:15.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.831+0000 7f37be7fc640 1 -- 172.21.15.154:0/315738252 learned_addr learned my addr 172.21.15.154:0/315738252 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:22:15.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.831+0000 7f37be7fc640 1 -- 172.21.15.154:0/315738252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557491705690 msgr2=0x55749171bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:22:15.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.831+0000 7f37be7fc640 1 --2- 172.21.15.154:0/315738252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557491705690 0x55749171bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:22:15.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.831+0000 7f37be7fc640 1 -- 172.21.15.154:0/315738252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55749171c2b0 msgr2=0x5574916dc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:22:15.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.831+0000 7f37be7fc640 1 --2- 172.21.15.154:0/315738252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55749171c2b0 0x5574916dc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:22:15.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.831+0000 7f37be7fc640 1 -- 172.21.15.154:0/315738252 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557491673da0 con 0x557491705a70 2023-09-23T03:22:15.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.831+0000 7f37be7fc640 1 --2- 172.21.15.154:0/315738252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557491705a70 0x55749171d080 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f37a8076b10 tx=0x7f37a8009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40edbdc6ecd6b813 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:22:15.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.832+0000 7f37b5ffb640 1 -- 172.21.15.154:0/315738252 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a80a9ec0 con 0x557491705a70 2023-09-23T03:22:15.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.832+0000 7f37b5ffb640 1 -- 172.21.15.154:0/315738252 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a80789b0 con 0x557491705a70 2023-09-23T03:22:15.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.832+0000 7f37b5ffb640 1 -- 172.21.15.154:0/315738252 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a807a870 con 0x557491705a70 2023-09-23T03:22:15.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.832+0000 7f37c83d8740 1 -- 172.21.15.154:0/315738252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557491705a70 msgr2=0x55749171d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:22:15.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.832+0000 7f37c83d8740 1 --2- 172.21.15.154:0/315738252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557491705a70 0x55749171d080 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f37a8076b10 tx=0x7f37a8009110 comp rx=0 tx=0).stop 2023-09-23T03:22:15.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.832+0000 7f37c83d8740 1 -- 172.21.15.154:0/315738252 shutdown_connections 2023-09-23T03:22:15.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.832+0000 7f37c83d8740 1 --2- 172.21.15.154:0/315738252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55749171c2b0 0x5574916dc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:22:15.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.832+0000 7f37c83d8740 1 --2- 172.21.15.154:0/315738252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557491705a70 0x55749171d080 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:22:15.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.832+0000 7f37c83d8740 1 --2- 172.21.15.154:0/315738252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x557491705690 0x55749171bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:22:15.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.832+0000 7f37c83d8740 1 -- 172.21.15.154:0/315738252 >> 172.21.15.154:0/315738252 conn(0x5574916d8ad0 msgr2=0x557491745b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:22:15.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.832+0000 7f37c83d8740 1 -- 172.21.15.154:0/315738252 shutdown_connections 2023-09-23T03:22:15.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.832+0000 7f37c83d8740 1 -- 172.21.15.154:0/315738252 wait complete. 2023-09-23T03:22:15.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.833+0000 7f37c83d8740 1 -- start start 2023-09-23T03:22:15.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.833+0000 7f37c83d8740 1 Processor -- start 2023-09-23T03:22:15.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.834+0000 7f37c83d8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557491705690 0x55749170bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:22:15.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.834+0000 7f37c83d8740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557491705a70 0x557491727190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:22:15.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.834+0000 7f37c83d8740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55749171c2b0 0x557491703860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:22:15.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.834+0000 7f37c83d8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5574917045f0 con 0x557491705690 2023-09-23T03:22:15.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.834+0000 7f37c83d8740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55749173cb70 con 0x557491705a70 2023-09-23T03:22:15.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.834+0000 7f37c83d8740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5574917269d0 con 0x55749171c2b0 2023-09-23T03:22:15.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.834+0000 7f37c83d8740 10 client.0 _ll_register_callbacks cb 0x7f37b800b300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:22:15.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.835+0000 7f37beffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557491705690 0x55749170bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:22:15.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.835+0000 7f37beffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557491705690 0x55749170bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:41798/0 (socket says 172.21.15.154:41798) 2023-09-23T03:22:15.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.835+0000 7f37beffd640 1 -- 172.21.15.154:0/162571767 learned_addr learned my addr 172.21.15.154:0/162571767 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:22:15.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.835+0000 7f37beffd640 1 -- 172.21.15.154:0/162571767 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55749171c2b0 msgr2=0x557491703860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:22:15.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.835+0000 7f37beffd640 1 --2- 172.21.15.154:0/162571767 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55749171c2b0 0x557491703860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:22:15.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.835+0000 7f37beffd640 1 -- 172.21.15.154:0/162571767 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557491705a70 msgr2=0x557491727190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:22:15.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.835+0000 7f37beffd640 1 --2- 172.21.15.154:0/162571767 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557491705a70 0x557491727190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:22:15.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.835+0000 7f37beffd640 1 -- 172.21.15.154:0/162571767 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37c00c82d0 con 0x557491705690 2023-09-23T03:22:15.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.835+0000 7f37beffd640 1 --2- 172.21.15.154:0/162571767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557491705690 0x55749170bef0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f37b00150e0 tx=0x7f37b00014d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f743351ededdc3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:22:15.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.835+0000 7f379e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:22:15.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.835+0000 7f379e7fc640 1 -- 172.21.15.154:0/162571767 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b0001f90 con 0x557491705690 2023-09-23T03:22:15.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.836+0000 7f379e7fc640 1 -- 172.21.15.154:0/162571767 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f37b0015a10 con 0x557491705690 2023-09-23T03:22:15.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.836+0000 7f37c83d8740 1 -- 172.21.15.154:0/162571767 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f37b800fb20 con 0x557491705690 2023-09-23T03:22:15.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.836+0000 7f379e7fc640 1 -- 172.21.15.154:0/162571767 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37b0023f10 con 0x557491705690 2023-09-23T03:22:15.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.836+0000 7f37c83d8740 10 client.7056 fetch_fsmap learned FSMap version 54 2023-09-23T03:22:15.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.836+0000 7f37c83d8740 1 -- 172.21.15.154:0/162571767 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x5574915e0ce0 con 0x557491705690 2023-09-23T03:22:15.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.836+0000 7f379e7fc640 1 -- 172.21.15.154:0/162571767 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37b800fb20 con 0x557491705690 2023-09-23T03:22:15.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.836+0000 7f379e7fc640 1 -- 172.21.15.154:0/162571767 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 54) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f37b0029070 con 0x557491705690 2023-09-23T03:22:15.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.836+0000 7f37c83d8740 10 client.7056 fetch_fsmap finished waiting for FSMap version 54 2023-09-23T03:22:15.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.836+0000 7f37c83d8740 10 client.7056 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-09-23T03:22:15.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.836+0000 7f37c83d8740 20 client.7056 populate_metadata read hostname 'smithi154' 2023-09-23T03:22:15.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.836+0000 7f37c83d8740 1 --2- 172.21.15.154:0/162571767 >> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] conn(0x557491723eb0 0x557491703da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:22:15.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.836+0000 7f37c83d8740 4 client.7056 mds_command: new command op to 6924 tid=0[{"prefix": "scrub status"}] 2023-09-23T03:22:15.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.836+0000 7f37c83d8740 1 -- 172.21.15.154:0/162571767 --> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55749163ca10 con 0x557491723eb0 2023-09-23T03:22:15.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.836+0000 7f37be7fc640 1 --2- 172.21.15.154:0/162571767 >> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] conn(0x557491723eb0 0x557491703da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:22:15.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.837+0000 7f37be7fc640 1 --2- 172.21.15.154:0/162571767 >> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] conn(0x557491723eb0 0x557491703da0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d06b8ed74932f712 server_cookie=fbd4aeeea1047e2 in_seq=0 out_seq=0 2023-09-23T03:22:15.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.837+0000 7f379e7fc640 10 client.7056 ms_handle_connect on v2:172.21.15.154:6834/2633473586 2023-09-23T03:22:15.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.837+0000 7f379e7fc640 1 -- 172.21.15.154:0/162571767 <== mds.0 v2:172.21.15.154:6834/2633473586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55749163ca10 con 0x557491723eb0 2023-09-23T03:22:15.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.837+0000 7f379e7fc640 10 client.7056 handle_command_reply: tid=0 2023-09-23T03:22:15.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.837+0000 7f37c83d8740 1 client.7056 shutdown 2023-09-23T03:22:15.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.837+0000 7f37c83d8740 1 -- 172.21.15.154:0/162571767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557491705690 msgr2=0x55749170bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:22:15.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.837+0000 7f37c83d8740 1 --2- 172.21.15.154:0/162571767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557491705690 0x55749170bef0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f37b00150e0 tx=0x7f37b00014d0 comp rx=0 tx=0).stop 2023-09-23T03:22:15.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.837+0000 7f37c83d8740 1 -- 172.21.15.154:0/162571767 shutdown_connections 2023-09-23T03:22:15.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.837+0000 7f37c83d8740 1 --2- 172.21.15.154:0/162571767 >> [v2:172.21.15.154:6834/2633473586,v1:172.21.15.154:6835/2633473586] conn(0x557491723eb0 0x557491703da0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:22:15.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.837+0000 7f37c83d8740 1 --2- 172.21.15.154:0/162571767 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55749171c2b0 0x557491703860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:22:15.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.837+0000 7f37c83d8740 1 --2- 172.21.15.154:0/162571767 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x557491705a70 0x557491727190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:22:15.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.837+0000 7f37c83d8740 1 --2- 172.21.15.154:0/162571767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557491705690 0x55749170bef0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:22:15.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.837+0000 7f37c83d8740 1 -- 172.21.15.154:0/162571767 >> 172.21.15.154:0/162571767 conn(0x5574916d8ad0 msgr2=0x5574916dc570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:22:15.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.837+0000 7f379e7fc640 0 client.7056 ms_handle_reset on v2:172.21.15.154:6834/2633473586 2023-09-23T03:22:15.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.838+0000 7f37c83d8740 1 -- 172.21.15.154:0/162571767 shutdown_connections 2023-09-23T03:22:15.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.838+0000 7f37c83d8740 1 -- 172.21.15.154:0/162571767 wait complete. 2023-09-23T03:22:15.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.838+0000 7f37c83d8740 20 client.7056 trim_cache size 0 max 16384 2023-09-23T03:22:15.843 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:22:15.844 INFO:teuthology.orchestra.run.smithi154.stdout: "status": "no active scrubs running", 2023-09-23T03:22:15.844 INFO:teuthology.orchestra.run.smithi154.stdout: "scrubs": {} 2023-09-23T03:22:15.844 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:22:15.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.838+0000 7f37c584b640 1 -- 172.21.15.154:0/4103126489 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f37a006af60 msgr2=0x7f37a006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:22:15.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.838+0000 7f37c584b640 1 --2- 172.21.15.154:0/4103126489 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f37a006af60 0x7f37a006d420 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f37b0001560 tx=0x7f37b0009040 comp rx=0 tx=0).stop 2023-09-23T03:22:15.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.838+0000 7f37c584b640 1 -- 172.21.15.154:0/4103126489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37c015aba0 msgr2=0x7f37c01489f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:22:15.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.838+0000 7f37c584b640 1 --2- 172.21.15.154:0/4103126489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37c015aba0 0x7f37c01489f0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f37a8000c00 tx=0x7f37a80046c0 comp rx=0 tx=0).stop 2023-09-23T03:22:15.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.838+0000 7f37c584b640 1 -- 172.21.15.154:0/4103126489 shutdown_connections 2023-09-23T03:22:15.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.838+0000 7f37c584b640 1 --2- 172.21.15.154:0/4103126489 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f37a006af60 0x7f37a006d420 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:22:15.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.838+0000 7f37c584b640 1 --2- 172.21.15.154:0/4103126489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37c015bb30 0x7f37c01f6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:22:15.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.838+0000 7f37c584b640 1 --2- 172.21.15.154:0/4103126489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37c015aba0 0x7f37c01489f0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:22:15.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.838+0000 7f37c584b640 1 --2- 172.21.15.154:0/4103126489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37c00cb3e0 0x7f37c00c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:22:15.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.838+0000 7f37c584b640 1 -- 172.21.15.154:0/4103126489 >> 172.21.15.154:0/4103126489 conn(0x7f37c006c010 msgr2=0x7f37c015d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:22:15.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.839+0000 7f37c584b640 1 -- 172.21.15.154:0/4103126489 shutdown_connections 2023-09-23T03:22:15.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:15.839+0000 7f37c584b640 1 -- 172.21.15.154:0/4103126489 wait complete. 2023-09-23T03:22:15.856 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-09-23T03:22:15.856 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/0 2023-09-23T03:22:15.856 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:22:15.856 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:22:15.857 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/0" 2023-09-23T03:22:15.857 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:22:15.857 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:22:15.857 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 6291456: 2023-09-23T03:22:15.857 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:22:15.857 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 6291456 2023-09-23T03:22:15.857 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:22:15.857 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:22:15.857 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:22:15.857 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:22:15.857 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:22:15.857 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:22:19.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:22:19.452 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/1 2023-09-23T03:22:19.453 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:22:19.453 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:22:19.453 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/1" 2023-09-23T03:22:19.453 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:22:19.453 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:22:19.453 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 6291456: 2023-09-23T03:22:19.453 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:22:19.454 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 6291456 2023-09-23T03:22:19.454 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:22:19.454 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:22:19.454 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:22:19.454 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:22:19.454 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:22:19.454 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:22:23.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:22:23.138 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/2 2023-09-23T03:22:23.138 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:22:23.138 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:22:23.139 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/2" 2023-09-23T03:22:23.139 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:22:23.139 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:22:23.139 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 6291456: 2023-09-23T03:22:23.139 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:22:23.139 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 6291456 2023-09-23T03:22:23.139 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:22:23.139 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:22:23.139 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:22:23.139 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:22:23.139 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:22:23.140 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:22:26.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:22:26.796 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/3 2023-09-23T03:22:26.796 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:22:26.796 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:22:26.796 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/3" 2023-09-23T03:22:26.796 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:22:26.796 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:22:26.797 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 6291456: 2023-09-23T03:22:26.797 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:22:26.797 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 6291456 2023-09-23T03:22:26.797 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:22:26.797 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:22:26.797 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:22:26.797 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:22:26.797 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:22:26.797 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:22:30.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:22:30.527 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/4 2023-09-23T03:22:30.528 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:22:30.528 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:22:30.528 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/4" 2023-09-23T03:22:30.528 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:22:30.528 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:22:30.528 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 6291456: 2023-09-23T03:22:30.529 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:22:30.529 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 6291456 2023-09-23T03:22:30.529 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:22:30.529 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:22:30.529 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:22:30.529 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:22:30.529 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:22:30.530 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:22:34.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:22:34.264 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/5 2023-09-23T03:22:34.264 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:22:34.265 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:22:34.265 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/5" 2023-09-23T03:22:34.265 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:22:34.265 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:22:34.265 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 6291456: 2023-09-23T03:22:34.265 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:22:34.265 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 6291456 2023-09-23T03:22:34.265 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:22:34.265 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:22:34.265 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:22:34.265 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:22:34.265 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:22:34.266 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:22:38.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:22:38.072 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/6 2023-09-23T03:22:38.072 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:22:38.072 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:22:38.072 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/6" 2023-09-23T03:22:38.072 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:22:38.072 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:22:38.072 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 6291456: 2023-09-23T03:22:38.072 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:22:38.073 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 6291456 2023-09-23T03:22:38.073 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:22:38.073 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:22:38.073 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:22:38.073 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:22:38.073 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:22:38.073 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:22:41.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:22:41.755 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/7 2023-09-23T03:22:41.755 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:22:41.755 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:22:41.755 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/7" 2023-09-23T03:22:41.755 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:22:41.756 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:22:41.756 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 6291456: 2023-09-23T03:22:41.756 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:22:41.756 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 6291456 2023-09-23T03:22:41.756 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:22:41.756 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:22:41.756 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:22:41.756 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:22:41.756 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:22:41.756 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:22:42.967 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:22:42.976 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:22:45.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:22:45.442 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/8 2023-09-23T03:22:45.443 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:22:45.443 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:22:45.443 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/8" 2023-09-23T03:22:45.443 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:22:45.443 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:22:45.443 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 6291456: 2023-09-23T03:22:45.443 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:22:45.443 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 6291456 2023-09-23T03:22:45.443 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:22:45.443 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:22:45.444 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:22:45.444 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:22:45.444 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:22:45.444 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:22:49.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:22:49.032 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/9 2023-09-23T03:22:49.032 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:22:49.032 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:22:49.032 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/9" 2023-09-23T03:22:49.033 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:22:49.033 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:22:49.033 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 6291456: 2023-09-23T03:22:49.033 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:22:49.033 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 6291456 2023-09-23T03:22:49.033 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:22:49.033 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:22:49.033 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:22:49.033 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:22:49.033 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:22:49.033 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:22:52.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:22:52.649 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/10 2023-09-23T03:22:52.650 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:22:52.650 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:22:52.650 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/10" 2023-09-23T03:22:52.650 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:22:52.650 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:22:52.650 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 6291456: 2023-09-23T03:22:52.650 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:22:52.650 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 6291456 2023-09-23T03:22:52.651 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:22:52.651 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:22:52.651 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:22:52.651 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:22:52.651 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:22:52.651 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:22:56.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:22:56.235 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/11 2023-09-23T03:22:56.236 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:22:56.236 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:22:56.236 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/11" 2023-09-23T03:22:56.236 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:22:56.236 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:22:56.236 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 6291456: 2023-09-23T03:22:56.236 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:22:56.236 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 6291456 2023-09-23T03:22:56.236 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:22:56.236 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:22:56.236 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:22:56.237 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:22:56.237 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:22:56.237 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:22:59.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:22:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:22:59.794 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/12 2023-09-23T03:22:59.794 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:22:59.795 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:22:59.795 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/12" 2023-09-23T03:22:59.795 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:22:59.795 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:22:59.795 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 6291456: 2023-09-23T03:22:59.795 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:22:59.795 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 6291456 2023-09-23T03:22:59.795 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:22:59.795 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:22:59.795 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:22:59.795 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:22:59.795 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:22:59.796 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:23:03.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:23:03.409 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/13 2023-09-23T03:23:03.409 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:23:03.409 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:23:03.409 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/13" 2023-09-23T03:23:03.409 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:23:03.410 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:23:03.410 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 6291456: 2023-09-23T03:23:03.410 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:23:03.410 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 6291456 2023-09-23T03:23:03.410 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:23:03.410 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:23:03.410 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:23:03.410 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:23:03.410 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:23:03.410 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:23:07.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:23:07.035 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/14 2023-09-23T03:23:07.036 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:23:07.036 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:23:07.036 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/14" 2023-09-23T03:23:07.036 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:23:07.036 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:23:07.036 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 6291456: 2023-09-23T03:23:07.036 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:23:07.036 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 6291456 2023-09-23T03:23:07.036 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:23:07.037 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:23:07.037 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:23:07.037 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:23:07.037 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:23:07.037 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:23:10.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:23:10.722 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/15 2023-09-23T03:23:10.722 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:23:10.723 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:23:10.723 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/15" 2023-09-23T03:23:10.723 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:23:10.723 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:23:10.723 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 6291456: 2023-09-23T03:23:10.723 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:23:10.723 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 6291456 2023-09-23T03:23:10.723 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:23:10.723 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:23:10.724 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:23:10.724 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:23:10.724 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:23:10.724 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:23:13.133 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:23:13.142 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:23:14.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:23:14.348 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/16 2023-09-23T03:23:14.348 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:23:14.349 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:23:14.349 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/16" 2023-09-23T03:23:14.349 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:23:14.349 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:23:14.349 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 6291456: 2023-09-23T03:23:14.349 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:23:14.349 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 6291456 2023-09-23T03:23:14.349 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:23:14.349 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:23:14.349 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:23:14.349 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:23:14.350 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:23:14.350 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:23:17.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:23:17.954 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/17 2023-09-23T03:23:17.954 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:23:17.954 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:23:17.954 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/17" 2023-09-23T03:23:17.954 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:23:17.954 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:23:17.955 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 6291456: 2023-09-23T03:23:17.955 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:23:17.955 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 6291456 2023-09-23T03:23:17.955 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:23:17.955 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:23:17.955 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:23:17.955 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:23:17.955 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:23:17.955 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:23:21.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:23:21.621 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/18 2023-09-23T03:23:21.621 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:23:21.622 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:23:21.622 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/18" 2023-09-23T03:23:21.622 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:23:21.622 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:23:21.622 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 6291456: 2023-09-23T03:23:21.622 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:23:21.622 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 6291456 2023-09-23T03:23:21.622 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:23:21.622 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:23:21.622 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:23:21.622 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:23:21.622 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:23:21.623 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:23:25.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:23:25.278 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/19 2023-09-23T03:23:25.278 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:23:25.279 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:23:25.279 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/19" 2023-09-23T03:23:25.279 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:23:25.279 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:23:25.279 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 6291456: 2023-09-23T03:23:25.279 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:23:25.279 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 6291456 2023-09-23T03:23:25.279 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:23:25.279 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:23:25.279 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:23:25.280 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:23:25.280 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:23:25.280 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:23:28.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:23:28.950 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/20 2023-09-23T03:23:28.951 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:23:28.951 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:23:28.951 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/20" 2023-09-23T03:23:28.951 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:23:28.951 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:23:28.952 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 6291456: 2023-09-23T03:23:28.952 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:23:28.952 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 6291456 2023-09-23T03:23:28.952 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:23:28.952 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:23:28.952 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:23:28.952 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:23:28.953 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:23:28.953 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:23:32.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:23:32.677 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/21 2023-09-23T03:23:32.677 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:23:32.677 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:23:32.677 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/21" 2023-09-23T03:23:32.677 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:23:32.677 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:23:32.677 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 6291456: 2023-09-23T03:23:32.677 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:23:32.677 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 6291456 2023-09-23T03:23:32.678 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:23:32.678 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:23:32.678 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:23:32.678 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:23:32.678 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:23:32.678 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:23:36.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:23:36.388 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/22 2023-09-23T03:23:36.389 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:23:36.389 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:23:36.389 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/22" 2023-09-23T03:23:36.389 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:23:36.389 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:23:36.389 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 6291456: 2023-09-23T03:23:36.389 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:23:36.390 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 6291456 2023-09-23T03:23:36.390 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:23:36.390 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:23:36.390 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:23:36.390 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:23:36.390 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:23:36.390 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:23:40.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:23:40.035 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/23 2023-09-23T03:23:40.035 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:23:40.035 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:23:40.035 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/23" 2023-09-23T03:23:40.035 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:23:40.036 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:23:40.036 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 6291456: 2023-09-23T03:23:40.036 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:23:40.036 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 6291456 2023-09-23T03:23:40.036 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:23:40.036 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:23:40.036 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:23:40.036 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:23:40.036 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:23:40.037 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:23:43.262 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:23:43.271 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:23:43.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:23:43.596 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/24 2023-09-23T03:23:43.596 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:23:43.597 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:23:43.597 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/subdir/24" 2023-09-23T03:23:43.597 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:23:43.597 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:23:43.597 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 6291456: 2023-09-23T03:23:43.597 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:23:43.597 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 6291456 2023-09-23T03:23:43.597 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:23:43.597 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:23:43.597 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:23:43.597 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:23:43.598 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:23:43.598 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:23:47.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:23:47.164 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-09-23T03:23:47.164 DEBUG:teuthology.orchestra.run.smithi154:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-23T03:23:47.320 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-23T03:23:47.444 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-09-23T03:23:47.444 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:23:47.445 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:23:47.505 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:23:47.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.722+0000 7fd6fb59e640 1 Processor -- start 2023-09-23T03:23:47.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.722+0000 7fd6fb59e640 1 -- start start 2023-09-23T03:23:47.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.723+0000 7fd6fb59e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd6fc06c7a0 0x7fd6fc15eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:47.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.723+0000 7fd6fb59e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6fc15ab00 0x7fd6fc15af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:47.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.724+0000 7fd6fb59e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6fc06bff0 con 0x7fd6fc06fdc0 2023-09-23T03:23:47.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.724+0000 7fd6fb59e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6fc155070 con 0x7fd6fc06c7a0 2023-09-23T03:23:47.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.724+0000 7fd6fb59e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc15e0f0 con 0x7fd6fc15ab00 2023-09-23T03:23:47.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.724+0000 7fd6fad9d640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fd6fc06fdc0 0x7fd6fc0ca500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:50928/0 (socket says 172.21.15.154:50928) 2023-09-23T03:23:47.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.724+0000 7fd6fad9d640 1 -- 172.21.15.154:0/2261900394 learned_addr learned my addr 172.21.15.154:0/2261900394 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:23:47.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.724+0000 7fd6fa59c640 1 --2- 172.21.15.154:0/2261900394 >> v2:172.21.15.190:3300/0 conn(0x7fd6fc06c7a0 0x7fd6fc15eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:47.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.724+0000 7fd6f9d9b640 1 --2- 172.21.15.154:0/2261900394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6fc15ab00 0x7fd6fc15af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:47.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.724+0000 7fd6f959a640 1 -- 172.21.15.154:0/2261900394 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1936769701 0 0) 0x7fd6fc06bff0 con 0x7fd6fc06fdc0 2023-09-23T03:23:47.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.725+0000 7fd6f959a640 1 -- 172.21.15.154:0/2261900394 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6e8003520 con 0x7fd6fc06fdc0 2023-09-23T03:23:47.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.725+0000 7fd6f959a640 1 -- 172.21.15.154:0/2261900394 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1566683446 0 0) 0x7fd6e8003520 con 0x7fd6fc06fdc0 2023-09-23T03:23:47.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.725+0000 7fd6f959a640 1 -- 172.21.15.154:0/2261900394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6fc15ab00 msgr2=0x7fd6fc15af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:47.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.725+0000 7fd6f959a640 1 --2- 172.21.15.154:0/2261900394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6fc15ab00 0x7fd6fc15af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:47.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.725+0000 7fd6f959a640 1 -- 172.21.15.154:0/2261900394 >> v2:172.21.15.190:3300/0 conn(0x7fd6fc06c7a0 msgr2=0x7fd6fc15eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:47.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.725+0000 7fd6f959a640 1 --2- 172.21.15.154:0/2261900394 >> v2:172.21.15.190:3300/0 conn(0x7fd6fc06c7a0 0x7fd6fc15eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:47.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.725+0000 7fd6f959a640 1 -- 172.21.15.154:0/2261900394 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6fc06cb80 con 0x7fd6fc06fdc0 2023-09-23T03:23:47.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.725+0000 7fd6f959a640 1 -- 172.21.15.154:0/2261900394 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fd6f4001670 con 0x7fd6fc06fdc0 2023-09-23T03:23:47.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.725+0000 7fd6f959a640 1 -- 172.21.15.154:0/2261900394 >> v1:172.21.15.154:6789/0 conn(0x7fd6fc06fdc0 legacy=0x7fd6fc0ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:47.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.725+0000 7fd6f959a640 1 --2- 172.21.15.154:0/2261900394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6e8003970 0x7fd6e8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:47.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.725+0000 7fd6f959a640 1 --2- 172.21.15.154:0/2261900394 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6e80047c0 0x7fd6e8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:47.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.725+0000 7fd6f959a640 1 --2- 172.21.15.154:0/2261900394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6fc15ab00 0x7fd6e8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:47.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.725+0000 7fd6f959a640 1 -- 172.21.15.154:0/2261900394 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc15e0f0 con 0x7fd6e80047c0 2023-09-23T03:23:47.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.725+0000 7fd6f959a640 1 -- 172.21.15.154:0/2261900394 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e800b980 con 0x7fd6e8003970 2023-09-23T03:23:47.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.725+0000 7fd6f959a640 1 -- 172.21.15.154:0/2261900394 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e800bac0 con 0x7fd6fc15ab00 2023-09-23T03:23:47.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.725+0000 7fd6f9d9b640 1 --2- 172.21.15.154:0/2261900394 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6e80047c0 0x7fd6e8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:47.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.726+0000 7fd6fa59c640 1 --2- 172.21.15.154:0/2261900394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6e8003970 0x7fd6e8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:47.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.726+0000 7fd6fad9d640 1 --2- 172.21.15.154:0/2261900394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6fc15ab00 0x7fd6e8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:47.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.726+0000 7fd6f9d9b640 1 -- 172.21.15.154:0/2261900394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6fc15ab00 msgr2=0x7fd6e8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:47.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.726+0000 7fd6f9d9b640 1 --2- 172.21.15.154:0/2261900394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6fc15ab00 0x7fd6e8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:47.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.726+0000 7fd6f9d9b640 1 -- 172.21.15.154:0/2261900394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6e8003970 msgr2=0x7fd6e8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:47.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.726+0000 7fd6f9d9b640 1 --2- 172.21.15.154:0/2261900394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6e8003970 0x7fd6e8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:47.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.726+0000 7fd6f9d9b640 1 -- 172.21.15.154:0/2261900394 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e8003520 con 0x7fd6e80047c0 2023-09-23T03:23:47.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.726+0000 7fd6fad9d640 1 --2- 172.21.15.154:0/2261900394 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6fc15ab00 0x7fd6e8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:23:47.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.726+0000 7fd6fa59c640 1 --2- 172.21.15.154:0/2261900394 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6e8003970 0x7fd6e8003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:23:47.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.727+0000 7fd6f9d9b640 1 --2- 172.21.15.154:0/2261900394 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6e80047c0 0x7fd6e8004c40 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009fc0 tx=0x7fd6e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3b2b863c607f2f5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:47.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.727+0000 7fd6f959a640 1 -- 172.21.15.154:0/2261900394 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e402cbf0 con 0x7fd6e80047c0 2023-09-23T03:23:47.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.727+0000 7fd6f959a640 1 -- 172.21.15.154:0/2261900394 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e400f040 con 0x7fd6e80047c0 2023-09-23T03:23:47.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.727+0000 7fd6f959a640 1 -- 172.21.15.154:0/2261900394 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4034440 con 0x7fd6e80047c0 2023-09-23T03:23:47.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.728+0000 7fd6fb59e640 1 -- 172.21.15.154:0/2261900394 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6e80047c0 msgr2=0x7fd6e8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:47.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.728+0000 7fd6fb59e640 1 --2- 172.21.15.154:0/2261900394 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6e80047c0 0x7fd6e8004c40 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009fc0 tx=0x7fd6e80057f0 comp rx=0 tx=0).stop 2023-09-23T03:23:47.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.728+0000 7fd6fad9d640 1 -- 172.21.15.154:0/2261900394 reap_dead start 2023-09-23T03:23:47.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.728+0000 7fd6fb59e640 1 -- 172.21.15.154:0/2261900394 shutdown_connections 2023-09-23T03:23:47.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.728+0000 7fd6fb59e640 1 -- 172.21.15.154:0/2261900394 >> 172.21.15.154:0/2261900394 conn(0x7fd6fc06a390 msgr2=0x7fd6fc0afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:47.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.728+0000 7fd6fb59e640 1 -- 172.21.15.154:0/2261900394 shutdown_connections 2023-09-23T03:23:47.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.728+0000 7fd6fb59e640 1 -- 172.21.15.154:0/2261900394 wait complete. 2023-09-23T03:23:47.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.730+0000 7fd6fb59e640 1 Processor -- start 2023-09-23T03:23:47.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.730+0000 7fd6fb59e640 1 -- start start 2023-09-23T03:23:47.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.731+0000 7fd6fb59e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6e80047c0 0x7fd6fc0c23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:47.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.731+0000 7fd6fb59e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6fc151b20 0x7fd6fc0b3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:47.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.731+0000 7fd6fb59e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6fc0c49d0 0x7fd6fc0c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:47.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.731+0000 7fd6fb59e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc0c50b0 con 0x7fd6fc151b20 2023-09-23T03:23:47.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.731+0000 7fd6fb59e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc0c4e50 con 0x7fd6fc0c49d0 2023-09-23T03:23:47.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.731+0000 7fd6fb59e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc0b29a0 con 0x7fd6e80047c0 2023-09-23T03:23:47.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.731+0000 7fd6f9d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6fc151b20 0x7fd6fc0b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:47.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.731+0000 7fd6f9d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6fc151b20 0x7fd6fc0b3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60082/0 (socket says 172.21.15.154:60082) 2023-09-23T03:23:47.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.731+0000 7fd6f9d9b640 1 -- 172.21.15.154:0/3585833972 learned_addr learned my addr 172.21.15.154:0/3585833972 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:23:47.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.732+0000 7fd6fa59c640 1 --2- 172.21.15.154:0/3585833972 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6e80047c0 0x7fd6fc0c23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:47.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.732+0000 7fd6f9d9b640 1 -- 172.21.15.154:0/3585833972 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6e80047c0 msgr2=0x7fd6fc0c23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:47.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.732+0000 7fd6f9d9b640 1 --2- 172.21.15.154:0/3585833972 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6e80047c0 0x7fd6fc0c23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:47.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.732+0000 7fd6fad9d640 1 --2- 172.21.15.154:0/3585833972 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6fc0c49d0 0x7fd6fc0c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:47.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.732+0000 7fd6f9d9b640 1 -- 172.21.15.154:0/3585833972 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6fc0c49d0 msgr2=0x7fd6fc0c78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:47.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.732+0000 7fd6f9d9b640 1 --2- 172.21.15.154:0/3585833972 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6fc0c49d0 0x7fd6fc0c78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:47.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.732+0000 7fd6f9d9b640 1 -- 172.21.15.154:0/3585833972 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6fc0b2c80 con 0x7fd6fc151b20 2023-09-23T03:23:47.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.732+0000 7fd6fa59c640 1 --2- 172.21.15.154:0/3585833972 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6e80047c0 0x7fd6fc0c23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:23:47.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.732+0000 7fd6f9d9b640 1 --2- 172.21.15.154:0/3585833972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6fc151b20 0x7fd6fc0b3760 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fd6e400e230 tx=0x7fd6e4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:47.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.732+0000 7fd6f2ffd640 1 -- 172.21.15.154:0/3585833972 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4034bd0 con 0x7fd6fc151b20 2023-09-23T03:23:47.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.732+0000 7fd6f2ffd640 1 -- 172.21.15.154:0/3585833972 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e4034d50 con 0x7fd6fc151b20 2023-09-23T03:23:47.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.732+0000 7fd6fb59e640 1 -- 172.21.15.154:0/3585833972 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6fc0b2f60 con 0x7fd6fc151b20 2023-09-23T03:23:47.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.733+0000 7fd6f2ffd640 1 -- 172.21.15.154:0/3585833972 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4003bd0 con 0x7fd6fc151b20 2023-09-23T03:23:47.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.733+0000 7fd6fb59e640 1 -- 172.21.15.154:0/3585833972 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6fc152270 con 0x7fd6fc151b20 2023-09-23T03:23:47.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.734+0000 7fd6fb59e640 1 -- 172.21.15.154:0/3585833972 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6fc0736b0 con 0x7fd6fc151b20 2023-09-23T03:23:47.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.735+0000 7fd6f2ffd640 1 -- 172.21.15.154:0/3585833972 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fd6e4040450 con 0x7fd6fc151b20 2023-09-23T03:23:47.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.736+0000 7fd6f2ffd640 1 --2- 172.21.15.154:0/3585833972 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd6dc06af30 0x7fd6dc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:47.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.736+0000 7fd6f2ffd640 1 -- 172.21.15.154:0/3585833972 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fd6e40ac260 con 0x7fd6fc151b20 2023-09-23T03:23:47.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.736+0000 7fd6fa59c640 1 --2- 172.21.15.154:0/3585833972 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd6dc06af30 0x7fd6dc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:47.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.737+0000 7fd6fa59c640 1 --2- 172.21.15.154:0/3585833972 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd6dc06af30 0x7fd6dc06d3f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec009b20 tx=0x7fd6ec008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:47.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.741+0000 7fd6f2ffd640 1 -- 172.21.15.154:0/3585833972 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fd6e4076eb0 con 0x7fd6fc151b20 2023-09-23T03:23:47.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.928+0000 7fd6fb59e640 1 -- 172.21.15.154:0/3585833972 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6fc04e920 con 0x7fd6fc151b20 2023-09-23T03:23:47.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.930+0000 7fd6f2ffd640 1 -- 172.21.15.154:0/3585833972 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+18987 (secure 0 0 0) 0x7fd6e407a4d0 con 0x7fd6fc151b20 2023-09-23T03:23:47.931 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:23:47.931 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":71,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:21:39.274806+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:19:21.772037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-09-23T03:19:22.960250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data_extra","create_time":"2023-09-23T03:19:38.870843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6837/4129414148":"2023-09-24T03:21:38.278599+0000","172.21.15.154:6835/4129414148":"2023-09-24T03:21:38.278599+0000","172.21.15.154:6837/2976977273":"2023-09-24T03:21:31.069171+0000","172.21.15.154:6835/2976977273":"2023-09-24T03:21:31.069171+0000","172.21.15.154:6837/2183023203":"2023-09-24T03:21:29.032871+0000","172.21.15.154:6835/2183023203":"2023-09-24T03:21:29.032871+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:23:47.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.933+0000 7fd6fb59e640 1 -- 172.21.15.154:0/3585833972 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd6dc06af30 msgr2=0x7fd6dc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:47.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.933+0000 7fd6fb59e640 1 --2- 172.21.15.154:0/3585833972 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd6dc06af30 0x7fd6dc06d3f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec009b20 tx=0x7fd6ec008040 comp rx=0 tx=0).stop 2023-09-23T03:23:47.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.933+0000 7fd6fb59e640 1 -- 172.21.15.154:0/3585833972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6fc151b20 msgr2=0x7fd6fc0b3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:47.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.933+0000 7fd6fb59e640 1 --2- 172.21.15.154:0/3585833972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6fc151b20 0x7fd6fc0b3760 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fd6e400e230 tx=0x7fd6e4009750 comp rx=0 tx=0).stop 2023-09-23T03:23:47.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.933+0000 7fd6fb59e640 1 -- 172.21.15.154:0/3585833972 shutdown_connections 2023-09-23T03:23:47.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.933+0000 7fd6fb59e640 1 --2- 172.21.15.154:0/3585833972 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd6dc06af30 0x7fd6dc06d3f0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:47.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.933+0000 7fd6fb59e640 1 --2- 172.21.15.154:0/3585833972 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6fc0c49d0 0x7fd6fc0c78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:47.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.933+0000 7fd6fb59e640 1 --2- 172.21.15.154:0/3585833972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6fc151b20 0x7fd6fc0b3760 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:47.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.933+0000 7fd6fb59e640 1 --2- 172.21.15.154:0/3585833972 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6e80047c0 0x7fd6fc0c23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:47.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.933+0000 7fd6fb59e640 1 -- 172.21.15.154:0/3585833972 >> 172.21.15.154:0/3585833972 conn(0x7fd6fc06a390 msgr2=0x7fd6fc153030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:47.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.933+0000 7fd6fb59e640 1 -- 172.21.15.154:0/3585833972 shutdown_connections 2023-09-23T03:23:47.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:47.933+0000 7fd6fb59e640 1 -- 172.21.15.154:0/3585833972 wait complete. 2023-09-23T03:23:47.953 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:19:21.772037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:19:22.960250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-09-23T03:19:38.870843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T03:23:47.953 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:23:48.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.175+0000 7f067cf33640 1 Processor -- start 2023-09-23T03:23:48.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.175+0000 7f067cf33640 1 -- start start 2023-09-23T03:23:48.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.176+0000 7f067cf33640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f06780c8d70 0x7f067815c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:48.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.176+0000 7f067cf33640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06780c9150 0x7f067815cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:48.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.176+0000 7f067cf33640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06780c9d20 con 0x7f0678071290 2023-09-23T03:23:48.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.176+0000 7f067cf33640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0678071850 con 0x7f06780c8d70 2023-09-23T03:23:48.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.176+0000 7f067cf33640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0678071b30 con 0x7f06780c9150 2023-09-23T03:23:48.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.177+0000 7f067659c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f06780c8d70 0x7f067815c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:48.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.177+0000 7f067659c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f06780c8d70 0x7f067815c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60088/0 (socket says 172.21.15.154:60088) 2023-09-23T03:23:48.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.177+0000 7f067659c640 1 -- 172.21.15.154:0/1327889882 learned_addr learned my addr 172.21.15.154:0/1327889882 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:23:48.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.177+0000 7f0675d9b640 1 --2- 172.21.15.154:0/1327889882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06780c9150 0x7f067815cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:48.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.177+0000 7f067659c640 1 -- 172.21.15.154:0/1327889882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06780c9150 msgr2=0x7f067815cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:48.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.177+0000 7f067659c640 1 --2- 172.21.15.154:0/1327889882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06780c9150 0x7f067815cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:48.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.177+0000 7f067659c640 1 -- 172.21.15.154:0/1327889882 >> v1:172.21.15.190:6789/0 conn(0x7f0678071290 legacy=0x7f06780ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:48.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.177+0000 7f067659c640 1 -- 172.21.15.154:0/1327889882 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0678071e10 con 0x7f06780c8d70 2023-09-23T03:23:48.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.177+0000 7f0675d9b640 1 --2- 172.21.15.154:0/1327889882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06780c9150 0x7f067815cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:23:48.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.178+0000 7f067659c640 1 --2- 172.21.15.154:0/1327889882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06780c8d70 0x7f067815c6d0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f0668009870 tx=0x7f066802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=933042f13e3750d5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:48.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.179+0000 7f067559a640 1 -- 172.21.15.154:0/1327889882 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06680317c0 con 0x7f06780c8d70 2023-09-23T03:23:48.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.179+0000 7f067559a640 1 -- 172.21.15.154:0/1327889882 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0668031920 con 0x7f06780c8d70 2023-09-23T03:23:48.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.179+0000 7f067559a640 1 -- 172.21.15.154:0/1327889882 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0668031cb0 con 0x7f06780c8d70 2023-09-23T03:23:48.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.179+0000 7f067cf33640 1 -- 172.21.15.154:0/1327889882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06780c8d70 msgr2=0x7f067815c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:48.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.179+0000 7f067cf33640 1 --2- 172.21.15.154:0/1327889882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06780c8d70 0x7f067815c6d0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f0668009870 tx=0x7f066802b860 comp rx=0 tx=0).stop 2023-09-23T03:23:48.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.179+0000 7f067cf33640 1 -- 172.21.15.154:0/1327889882 shutdown_connections 2023-09-23T03:23:48.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.179+0000 7f067cf33640 1 --2- 172.21.15.154:0/1327889882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06780c9150 0x7f067815cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:48.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.179+0000 7f067cf33640 1 --2- 172.21.15.154:0/1327889882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06780c8d70 0x7f067815c6d0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:48.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.180+0000 7f067cf33640 1 -- 172.21.15.154:0/1327889882 >> 172.21.15.154:0/1327889882 conn(0x7f067806c010 msgr2=0x7f067806c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:48.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.180+0000 7f067cf33640 1 -- 172.21.15.154:0/1327889882 shutdown_connections 2023-09-23T03:23:48.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.180+0000 7f067cf33640 1 -- 172.21.15.154:0/1327889882 wait complete. 2023-09-23T03:23:48.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.181+0000 7f067cf33640 1 Processor -- start 2023-09-23T03:23:48.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.181+0000 7f067cf33640 1 -- start start 2023-09-23T03:23:48.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.182+0000 7f067cf33640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0678071290 0x7f06781461a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:48.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.182+0000 7f067cf33640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06780c8d70 0x7f067814c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:48.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.182+0000 7f067cf33640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06780c9150 0x7f067814df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:48.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.182+0000 7f067cf33640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067814b9e0 con 0x7f0678071290 2023-09-23T03:23:48.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.182+0000 7f067cf33640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067814bcc0 con 0x7f06780c9150 2023-09-23T03:23:48.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.182+0000 7f067cf33640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f067814bfa0 con 0x7f06780c8d70 2023-09-23T03:23:48.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.183+0000 7f067659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0678071290 0x7f06781461a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:48.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.183+0000 7f067659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0678071290 0x7f06781461a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60104/0 (socket says 172.21.15.154:60104) 2023-09-23T03:23:48.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.183+0000 7f067659c640 1 -- 172.21.15.154:0/1180386169 learned_addr learned my addr 172.21.15.154:0/1180386169 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:23:48.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.183+0000 7f0675d9b640 1 --2- 172.21.15.154:0/1180386169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06780c8d70 0x7f067814c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:48.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.183+0000 7f0676d9d640 1 --2- 172.21.15.154:0/1180386169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06780c9150 0x7f067814df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:48.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.183+0000 7f067659c640 1 -- 172.21.15.154:0/1180386169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06780c8d70 msgr2=0x7f067814c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:48.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.183+0000 7f067659c640 1 --2- 172.21.15.154:0/1180386169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06780c8d70 0x7f067814c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:48.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.183+0000 7f067659c640 1 -- 172.21.15.154:0/1180386169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06780c9150 msgr2=0x7f067814df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:48.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.183+0000 7f067659c640 1 --2- 172.21.15.154:0/1180386169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06780c9150 0x7f067814df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:48.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.183+0000 7f067659c640 1 -- 172.21.15.154:0/1180386169 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06680094d0 con 0x7f0678071290 2023-09-23T03:23:48.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.183+0000 7f067659c640 1 --2- 172.21.15.154:0/1180386169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0678071290 0x7f06781461a0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f0668000c00 tx=0x7f06680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:48.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.184+0000 7f0666ffd640 1 -- 172.21.15.154:0/1180386169 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06680329a0 con 0x7f0678071290 2023-09-23T03:23:48.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.184+0000 7f0666ffd640 1 -- 172.21.15.154:0/1180386169 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0668032b20 con 0x7f0678071290 2023-09-23T03:23:48.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.184+0000 7f0666ffd640 1 -- 172.21.15.154:0/1180386169 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06680453f0 con 0x7f0678071290 2023-09-23T03:23:48.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.184+0000 7f067cf33640 1 -- 172.21.15.154:0/1180386169 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06780c30d0 con 0x7f0678071290 2023-09-23T03:23:48.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.184+0000 7f067cf33640 1 -- 172.21.15.154:0/1180386169 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06780c63d0 con 0x7f0678071290 2023-09-23T03:23:48.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.185+0000 7f067cf33640 1 -- 172.21.15.154:0/1180386169 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f067804e920 con 0x7f0678071290 2023-09-23T03:23:48.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.186+0000 7f0666ffd640 1 -- 172.21.15.154:0/1180386169 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f066804d3f0 con 0x7f0678071290 2023-09-23T03:23:48.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.187+0000 7f0666ffd640 1 --2- 172.21.15.154:0/1180386169 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f065806af30 0x7f065806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:48.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.187+0000 7f0666ffd640 1 -- 172.21.15.154:0/1180386169 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f06680abb10 con 0x7f0678071290 2023-09-23T03:23:48.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.187+0000 7f0675d9b640 1 --2- 172.21.15.154:0/1180386169 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f065806af30 0x7f065806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:48.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.188+0000 7f0675d9b640 1 --2- 172.21.15.154:0/1180386169 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f065806af30 0x7f065806d3f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f067814b480 tx=0x7f066c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:48.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.191+0000 7f0666ffd640 1 -- 172.21.15.154:0/1180386169 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f0668076770 con 0x7f0678071290 2023-09-23T03:23:48.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.367+0000 7f067cf33640 1 -- 172.21.15.154:0/1180386169 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f06780cf070 con 0x7f0678071290 2023-09-23T03:23:48.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.368+0000 7f0666ffd640 1 -- 172.21.15.154:0/1180386169 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0668079d90 con 0x7f0678071290 2023-09-23T03:23:48.369 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:23:48.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.370+0000 7f067cf33640 1 -- 172.21.15.154:0/1180386169 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f065806af30 msgr2=0x7f065806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:48.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.370+0000 7f067cf33640 1 --2- 172.21.15.154:0/1180386169 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f065806af30 0x7f065806d3f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f067814b480 tx=0x7f066c008040 comp rx=0 tx=0).stop 2023-09-23T03:23:48.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.371+0000 7f067cf33640 1 -- 172.21.15.154:0/1180386169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0678071290 msgr2=0x7f06781461a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:48.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.371+0000 7f067cf33640 1 --2- 172.21.15.154:0/1180386169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0678071290 0x7f06781461a0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f0668000c00 tx=0x7f06680046c0 comp rx=0 tx=0).stop 2023-09-23T03:23:48.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.371+0000 7f067cf33640 1 -- 172.21.15.154:0/1180386169 shutdown_connections 2023-09-23T03:23:48.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.371+0000 7f067cf33640 1 --2- 172.21.15.154:0/1180386169 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f065806af30 0x7f065806d3f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:48.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.371+0000 7f067cf33640 1 --2- 172.21.15.154:0/1180386169 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06780c9150 0x7f067814df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:48.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.371+0000 7f067cf33640 1 --2- 172.21.15.154:0/1180386169 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06780c8d70 0x7f067814c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:48.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.371+0000 7f067cf33640 1 --2- 172.21.15.154:0/1180386169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0678071290 0x7f06781461a0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:48.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.371+0000 7f067cf33640 1 -- 172.21.15.154:0/1180386169 >> 172.21.15.154:0/1180386169 conn(0x7f067806c010 msgr2=0x7f06780ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:48.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.371+0000 7f067cf33640 1 -- 172.21.15.154:0/1180386169 shutdown_connections 2023-09-23T03:23:48.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.371+0000 7f067cf33640 1 -- 172.21.15.154:0/1180386169 wait complete. 2023-09-23T03:23:48.390 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:23:48.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.606+0000 7fedba389640 1 Processor -- start 2023-09-23T03:23:48.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.606+0000 7fedba389640 1 -- start start 2023-09-23T03:23:48.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.607+0000 7fedba389640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fedb40c8d70 0x7fedb415c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:48.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.607+0000 7fedba389640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedb4071290 0x7fedb40ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:48.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.607+0000 7fedba389640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedb40c9d20 con 0x7fedb40c9150 2023-09-23T03:23:48.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.607+0000 7fedba389640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedb4071850 con 0x7fedb40c8d70 2023-09-23T03:23:48.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.607+0000 7fedba389640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb4071b30 con 0x7fedb4071290 2023-09-23T03:23:48.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.608+0000 7fedb3fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fedb40c8d70 0x7fedb415c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:48.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.608+0000 7fedb3fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fedb40c8d70 0x7fedb415c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60118/0 (socket says 172.21.15.154:60118) 2023-09-23T03:23:48.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.608+0000 7fedb3fff640 1 -- 172.21.15.154:0/1412101486 learned_addr learned my addr 172.21.15.154:0/1412101486 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:23:48.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.608+0000 7fedb8926640 1 --2- 172.21.15.154:0/1412101486 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedb4071290 0x7fedb40ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:48.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.608+0000 7fedb3fff640 1 -- 172.21.15.154:0/1412101486 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedb4071290 msgr2=0x7fedb40ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:48.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.608+0000 7fedb3fff640 1 --2- 172.21.15.154:0/1412101486 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedb4071290 0x7fedb40ca1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:48.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.608+0000 7fedb3fff640 1 -- 172.21.15.154:0/1412101486 >> v1:172.21.15.154:6789/0 conn(0x7fedb40c9150 legacy=0x7fedb415cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:48.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.608+0000 7fedb3fff640 1 -- 172.21.15.154:0/1412101486 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb4071e10 con 0x7fedb40c8d70 2023-09-23T03:23:48.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.608+0000 7fedb2ffd640 1 -- 172.21.15.154:0/1412101486 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 432776920 0 0) 0x7fedb40c9d20 con 0x7fedb40c9150 2023-09-23T03:23:48.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.609+0000 7fedb3fff640 1 --2- 172.21.15.154:0/1412101486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fedb40c8d70 0x7fedb415c6d0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7feda4009870 tx=0x7feda402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=254ff5beb3de2e23 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:48.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.610+0000 7fedb2ffd640 1 -- 172.21.15.154:0/1412101486 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda4030840 con 0x7fedb40c8d70 2023-09-23T03:23:48.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.610+0000 7fedb2ffd640 1 -- 172.21.15.154:0/1412101486 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feda40309a0 con 0x7fedb40c8d70 2023-09-23T03:23:48.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.610+0000 7fedb2ffd640 1 -- 172.21.15.154:0/1412101486 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda4030d30 con 0x7fedb40c8d70 2023-09-23T03:23:48.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.610+0000 7fedba389640 1 -- 172.21.15.154:0/1412101486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fedb40c8d70 msgr2=0x7fedb415c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:48.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.610+0000 7fedba389640 1 --2- 172.21.15.154:0/1412101486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fedb40c8d70 0x7fedb415c6d0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7feda4009870 tx=0x7feda402b980 comp rx=0 tx=0).stop 2023-09-23T03:23:48.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.610+0000 7fedba389640 1 -- 172.21.15.154:0/1412101486 shutdown_connections 2023-09-23T03:23:48.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.610+0000 7fedba389640 1 --2- 172.21.15.154:0/1412101486 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedb4071290 0x7fedb40ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:48.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.610+0000 7fedba389640 1 --2- 172.21.15.154:0/1412101486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fedb40c8d70 0x7fedb415c6d0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:48.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.610+0000 7fedba389640 1 -- 172.21.15.154:0/1412101486 >> 172.21.15.154:0/1412101486 conn(0x7fedb406c010 msgr2=0x7fedb406c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:48.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.611+0000 7fedba389640 1 -- 172.21.15.154:0/1412101486 shutdown_connections 2023-09-23T03:23:48.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.611+0000 7fedba389640 1 -- 172.21.15.154:0/1412101486 wait complete. 2023-09-23T03:23:48.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.612+0000 7fedba389640 1 Processor -- start 2023-09-23T03:23:48.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.613+0000 7fedba389640 1 -- start start 2023-09-23T03:23:48.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.613+0000 7fedba389640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fedb4071290 0x7fedb4158dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:48.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.613+0000 7fedba389640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedb40c8d70 0x7fedb41ee480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:48.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.614+0000 7fedba389640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedb40c9150 0x7fedb40b2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:48.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.614+0000 7fedba389640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb4159300 con 0x7fedb4071290 2023-09-23T03:23:48.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.614+0000 7fedba389640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb41ee9c0 con 0x7fedb40c9150 2023-09-23T03:23:48.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.614+0000 7fedba389640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb40b2bd0 con 0x7fedb40c8d70 2023-09-23T03:23:48.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.614+0000 7fedb3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fedb4071290 0x7fedb4158dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:48.615 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.614+0000 7fedb3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fedb4071290 0x7fedb4158dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60128/0 (socket says 172.21.15.154:60128) 2023-09-23T03:23:48.615 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.614+0000 7fedb3fff640 1 -- 172.21.15.154:0/3174509143 learned_addr learned my addr 172.21.15.154:0/3174509143 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:23:48.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.614+0000 7fedb37fe640 1 --2- 172.21.15.154:0/3174509143 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedb40c8d70 0x7fedb41ee480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:48.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.614+0000 7fedb3fff640 1 -- 172.21.15.154:0/3174509143 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedb40c8d70 msgr2=0x7fedb41ee480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:48.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.614+0000 7fedb3fff640 1 --2- 172.21.15.154:0/3174509143 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedb40c8d70 0x7fedb41ee480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:48.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.614+0000 7fedb3fff640 1 -- 172.21.15.154:0/3174509143 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedb40c9150 msgr2=0x7fedb40b2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:48.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.614+0000 7fedb8926640 1 --2- 172.21.15.154:0/3174509143 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedb40c9150 0x7fedb40b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:48.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.614+0000 7fedb3fff640 1 --2- 172.21.15.154:0/3174509143 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedb40c9150 0x7fedb40b2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:48.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.614+0000 7fedb3fff640 1 -- 172.21.15.154:0/3174509143 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda40094d0 con 0x7fedb4071290 2023-09-23T03:23:48.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.615+0000 7fedb3fff640 1 --2- 172.21.15.154:0/3174509143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fedb4071290 0x7fedb4158dc0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7feda402b9c0 tx=0x7feda4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:48.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.615+0000 7fedb0ff9640 1 -- 172.21.15.154:0/3174509143 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda4039c90 con 0x7fedb4071290 2023-09-23T03:23:48.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.615+0000 7fedb0ff9640 1 -- 172.21.15.154:0/3174509143 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feda4039df0 con 0x7fedb4071290 2023-09-23T03:23:48.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.615+0000 7fedb0ff9640 1 -- 172.21.15.154:0/3174509143 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda40446c0 con 0x7fedb4071290 2023-09-23T03:23:48.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.615+0000 7fedba389640 1 -- 172.21.15.154:0/3174509143 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedb41532d0 con 0x7fedb4071290 2023-09-23T03:23:48.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.615+0000 7fedba389640 1 -- 172.21.15.154:0/3174509143 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedb415bc10 con 0x7fedb4071290 2023-09-23T03:23:48.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.616+0000 7fedba389640 1 -- 172.21.15.154:0/3174509143 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedb40b41b0 con 0x7fedb4071290 2023-09-23T03:23:48.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.617+0000 7fedb0ff9640 1 -- 172.21.15.154:0/3174509143 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7feda402f390 con 0x7fedb4071290 2023-09-23T03:23:48.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.618+0000 7fedb0ff9640 1 --2- 172.21.15.154:0/3174509143 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7feda006af30 0x7feda006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:48.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.618+0000 7fedb0ff9640 1 -- 172.21.15.154:0/3174509143 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6691+0+0 (secure 0 0 0) 0x7feda403e070 con 0x7fedb4071290 2023-09-23T03:23:48.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.618+0000 7fedb37fe640 1 --2- 172.21.15.154:0/3174509143 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7feda006af30 0x7feda006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:48.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.619+0000 7fedb37fe640 1 --2- 172.21.15.154:0/3174509143 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7feda006af30 0x7feda006d3f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fedb41539b0 tx=0x7feda8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:48.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.622+0000 7fedb0ff9640 1 -- 172.21.15.154:0/3174509143 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7feda40770e0 con 0x7fedb4071290 2023-09-23T03:23:48.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.812+0000 7fedba389640 1 -- 172.21.15.154:0/3174509143 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fedb404e920 con 0x7fedb4071290 2023-09-23T03:23:48.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.813+0000 7fedb0ff9640 1 -- 172.21.15.154:0/3174509143 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) v1 ==== 104+0+11 (secure 0 0 0) 0x7feda407a700 con 0x7fedb4071290 2023-09-23T03:23:48.814 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:23:48.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.816+0000 7fedba389640 1 -- 172.21.15.154:0/3174509143 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7feda006af30 msgr2=0x7feda006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:48.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.816+0000 7fedba389640 1 --2- 172.21.15.154:0/3174509143 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7feda006af30 0x7feda006d3f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fedb41539b0 tx=0x7feda8009040 comp rx=0 tx=0).stop 2023-09-23T03:23:48.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.816+0000 7fedba389640 1 -- 172.21.15.154:0/3174509143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fedb4071290 msgr2=0x7fedb4158dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:48.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.816+0000 7fedba389640 1 --2- 172.21.15.154:0/3174509143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fedb4071290 0x7fedb4158dc0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7feda402b9c0 tx=0x7feda4004730 comp rx=0 tx=0).stop 2023-09-23T03:23:48.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.816+0000 7fedba389640 1 -- 172.21.15.154:0/3174509143 shutdown_connections 2023-09-23T03:23:48.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.816+0000 7fedba389640 1 --2- 172.21.15.154:0/3174509143 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7feda006af30 0x7feda006d3f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:48.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.816+0000 7fedba389640 1 --2- 172.21.15.154:0/3174509143 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fedb40c9150 0x7fedb40b2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:48.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.816+0000 7fedba389640 1 --2- 172.21.15.154:0/3174509143 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fedb40c8d70 0x7fedb41ee480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:48.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.816+0000 7fedba389640 1 --2- 172.21.15.154:0/3174509143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fedb4071290 0x7fedb4158dc0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:48.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.816+0000 7fedba389640 1 -- 172.21.15.154:0/3174509143 >> 172.21.15.154:0/3174509143 conn(0x7fedb406c010 msgr2=0x7fedb40c28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:48.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.816+0000 7fedba389640 1 -- 172.21.15.154:0/3174509143 shutdown_connections 2023-09-23T03:23:48.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:48.816+0000 7fedba389640 1 -- 172.21.15.154:0/3174509143 wait complete. 2023-09-23T03:23:48.833 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:23:49.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.050+0000 7f68fee64640 1 Processor -- start 2023-09-23T03:23:49.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.051+0000 7f68fee64640 1 -- start start 2023-09-23T03:23:49.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.052+0000 7f68fee64640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f68f815aba0 0x7f68f815bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:49.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.052+0000 7f68fee64640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f68f815e5f0 0x7f68f815ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:49.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.052+0000 7f68fee64640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68f815b6a0 con 0x7f68f80caea0 2023-09-23T03:23:49.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.052+0000 7f68fee64640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68f815f0b0 con 0x7f68f815e5f0 2023-09-23T03:23:49.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.052+0000 7f68fee64640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68f815f390 con 0x7f68f815aba0 2023-09-23T03:23:49.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.052+0000 7f68f7fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f68f815e5f0 0x7f68f815ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:49.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.052+0000 7f68fd401640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f68f80caea0 0x7f68f80cc5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:50940/0 (socket says 172.21.15.154:50940) 2023-09-23T03:23:49.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.052+0000 7f68fd401640 1 -- 172.21.15.154:0/3195951400 learned_addr learned my addr 172.21.15.154:0/3195951400 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:23:49.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.052+0000 7f68fcc00640 1 --2- 172.21.15.154:0/3195951400 >> v2:172.21.15.190:3300/0 conn(0x7f68f815aba0 0x7f68f815bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:49.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.053+0000 7f68f7fff640 1 -- 172.21.15.154:0/3195951400 >> v2:172.21.15.190:3300/0 conn(0x7f68f815aba0 msgr2=0x7f68f815bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:49.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.053+0000 7f68f7fff640 1 --2- 172.21.15.154:0/3195951400 >> v2:172.21.15.190:3300/0 conn(0x7f68f815aba0 0x7f68f815bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:49.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.053+0000 7f68f7fff640 1 -- 172.21.15.154:0/3195951400 >> v1:172.21.15.154:6789/0 conn(0x7f68f80caea0 legacy=0x7f68f80cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:49.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.053+0000 7f68f77fe640 1 -- 172.21.15.154:0/3195951400 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 231585687 0 0) 0x7f68f815b6a0 con 0x7f68f80caea0 2023-09-23T03:23:49.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.053+0000 7f68f7fff640 1 -- 172.21.15.154:0/3195951400 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f815b330 con 0x7f68f815e5f0 2023-09-23T03:23:49.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.054+0000 7f68f7fff640 1 --2- 172.21.15.154:0/3195951400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68f815e5f0 0x7f68f815ea30 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f68f806e1f0 tx=0x7f68ec02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0fe48b108b6fd07 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:49.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.054+0000 7f68f77fe640 1 -- 172.21.15.154:0/3195951400 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ec031830 con 0x7f68f815e5f0 2023-09-23T03:23:49.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.054+0000 7f68f77fe640 1 -- 172.21.15.154:0/3195951400 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68ec031990 con 0x7f68f815e5f0 2023-09-23T03:23:49.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.054+0000 7f68f77fe640 1 -- 172.21.15.154:0/3195951400 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ec031d20 con 0x7f68f815e5f0 2023-09-23T03:23:49.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.055+0000 7f68fee64640 1 -- 172.21.15.154:0/3195951400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68f815e5f0 msgr2=0x7f68f815ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:49.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.055+0000 7f68fee64640 1 --2- 172.21.15.154:0/3195951400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68f815e5f0 0x7f68f815ea30 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f68f806e1f0 tx=0x7f68ec02b8d0 comp rx=0 tx=0).stop 2023-09-23T03:23:49.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.055+0000 7f68fee64640 1 -- 172.21.15.154:0/3195951400 shutdown_connections 2023-09-23T03:23:49.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.055+0000 7f68fee64640 1 --2- 172.21.15.154:0/3195951400 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68f815e5f0 0x7f68f815ea30 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:49.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.055+0000 7f68fee64640 1 --2- 172.21.15.154:0/3195951400 >> v2:172.21.15.190:3300/0 conn(0x7f68f815aba0 0x7f68f815bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:49.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.055+0000 7f68fee64640 1 -- 172.21.15.154:0/3195951400 >> 172.21.15.154:0/3195951400 conn(0x7f68f806c010 msgr2=0x7f68f806c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:49.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.055+0000 7f68fee64640 1 -- 172.21.15.154:0/3195951400 shutdown_connections 2023-09-23T03:23:49.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.055+0000 7f68fee64640 1 -- 172.21.15.154:0/3195951400 wait complete. 2023-09-23T03:23:49.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.057+0000 7f68fee64640 1 Processor -- start 2023-09-23T03:23:49.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.057+0000 7f68fee64640 1 -- start start 2023-09-23T03:23:49.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.058+0000 7f68fee64640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68f80caea0 0x7f68f81440c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:49.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.058+0000 7f68fee64640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68f815aba0 0x7f68f80b3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:49.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.058+0000 7f68fee64640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68f815e5f0 0x7f68f80c2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:49.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.058+0000 7f68fee64640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f80c4f60 con 0x7f68f815aba0 2023-09-23T03:23:49.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.058+0000 7f68fee64640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f8144600 con 0x7f68f80caea0 2023-09-23T03:23:49.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.058+0000 7f68fee64640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f81f2e80 con 0x7f68f815e5f0 2023-09-23T03:23:49.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.058+0000 7f68f7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68f815aba0 0x7f68f80b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:49.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.058+0000 7f68f7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68f815aba0 0x7f68f80b3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60152/0 (socket says 172.21.15.154:60152) 2023-09-23T03:23:49.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.058+0000 7f68f7fff640 1 -- 172.21.15.154:0/58906041 learned_addr learned my addr 172.21.15.154:0/58906041 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:23:49.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.058+0000 7f68fcc00640 1 --2- 172.21.15.154:0/58906041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68f80caea0 0x7f68f81440c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:49.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.058+0000 7f68f7fff640 1 -- 172.21.15.154:0/58906041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68f815e5f0 msgr2=0x7f68f80c2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:49.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.058+0000 7f68fd401640 1 --2- 172.21.15.154:0/58906041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68f815e5f0 0x7f68f80c2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:49.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.059+0000 7f68f7fff640 1 --2- 172.21.15.154:0/58906041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68f815e5f0 0x7f68f80c2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:49.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.059+0000 7f68f7fff640 1 -- 172.21.15.154:0/58906041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68f80caea0 msgr2=0x7f68f81440c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:49.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.059+0000 7f68f7fff640 1 --2- 172.21.15.154:0/58906041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68f80caea0 0x7f68f81440c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:49.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.059+0000 7f68f7fff640 1 -- 172.21.15.154:0/58906041 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ec009400 con 0x7f68f815aba0 2023-09-23T03:23:49.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.059+0000 7f68fcc00640 1 --2- 172.21.15.154:0/58906041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68f80caea0 0x7f68f81440c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:23:49.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.059+0000 7f68f7fff640 1 --2- 172.21.15.154:0/58906041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68f815aba0 0x7f68f80b3750 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f68ec000c00 tx=0x7f68ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:49.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.059+0000 7f68f57fa640 1 -- 172.21.15.154:0/58906041 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ec0317d0 con 0x7f68f815aba0 2023-09-23T03:23:49.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.059+0000 7f68f57fa640 1 -- 172.21.15.154:0/58906041 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68ec03aa60 con 0x7f68f815aba0 2023-09-23T03:23:49.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.059+0000 7f68f57fa640 1 -- 172.21.15.154:0/58906041 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ec03ae10 con 0x7f68f815aba0 2023-09-23T03:23:49.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.059+0000 7f68fee64640 1 -- 172.21.15.154:0/58906041 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68f815b100 con 0x7f68f815aba0 2023-09-23T03:23:49.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.060+0000 7f68fee64640 1 -- 172.21.15.154:0/58906041 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68f81f25e0 con 0x7f68f815aba0 2023-09-23T03:23:49.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.061+0000 7f68fee64640 1 -- 172.21.15.154:0/58906041 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68f804e920 con 0x7f68f815aba0 2023-09-23T03:23:49.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.062+0000 7f68f57fa640 1 -- 172.21.15.154:0/58906041 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f68ec04d450 con 0x7f68f815aba0 2023-09-23T03:23:49.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.062+0000 7f68f57fa640 1 --2- 172.21.15.154:0/58906041 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f68e406af30 0x7f68e406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:49.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.063+0000 7f68f57fa640 1 -- 172.21.15.154:0/58906041 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f68ec0abc60 con 0x7f68f815aba0 2023-09-23T03:23:49.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.063+0000 7f68fcc00640 1 --2- 172.21.15.154:0/58906041 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f68e406af30 0x7f68e406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:49.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.063+0000 7f68fcc00640 1 --2- 172.21.15.154:0/58906041 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f68e406af30 0x7f68e406d3f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f68e8007330 tx=0x7f68e800b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:49.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.067+0000 7f68f57fa640 1 -- 172.21.15.154:0/58906041 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f68ec0768b0 con 0x7f68f815aba0 2023-09-23T03:23:49.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.256+0000 7f68fee64640 1 -- 172.21.15.154:0/58906041 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f68f80cbc10 con 0x7f68f815aba0 2023-09-23T03:23:49.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.257+0000 7f68f57fa640 1 -- 172.21.15.154:0/58906041 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) v1 ==== 100+0+11 (secure 0 0 0) 0x7f68ec035020 con 0x7f68f815aba0 2023-09-23T03:23:49.257 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:23:49.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.260+0000 7f68fee64640 1 -- 172.21.15.154:0/58906041 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f68e406af30 msgr2=0x7f68e406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:49.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.260+0000 7f68fee64640 1 --2- 172.21.15.154:0/58906041 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f68e406af30 0x7f68e406d3f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f68e8007330 tx=0x7f68e800b040 comp rx=0 tx=0).stop 2023-09-23T03:23:49.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.260+0000 7f68fee64640 1 -- 172.21.15.154:0/58906041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68f815aba0 msgr2=0x7f68f80b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:49.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.260+0000 7f68fee64640 1 --2- 172.21.15.154:0/58906041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68f815aba0 0x7f68f80b3750 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f68ec000c00 tx=0x7f68ec0046c0 comp rx=0 tx=0).stop 2023-09-23T03:23:49.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.260+0000 7f68fee64640 1 -- 172.21.15.154:0/58906041 shutdown_connections 2023-09-23T03:23:49.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.260+0000 7f68fee64640 1 --2- 172.21.15.154:0/58906041 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f68e406af30 0x7f68e406d3f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:49.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.260+0000 7f68fee64640 1 --2- 172.21.15.154:0/58906041 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68f815e5f0 0x7f68f80c2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:49.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.260+0000 7f68fee64640 1 --2- 172.21.15.154:0/58906041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68f815aba0 0x7f68f80b3750 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:49.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.260+0000 7f68fee64640 1 --2- 172.21.15.154:0/58906041 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68f80caea0 0x7f68f81440c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:49.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.260+0000 7f68fee64640 1 -- 172.21.15.154:0/58906041 >> 172.21.15.154:0/58906041 conn(0x7f68f806c010 msgr2=0x7f68f80c1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:49.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.260+0000 7f68fee64640 1 -- 172.21.15.154:0/58906041 shutdown_connections 2023-09-23T03:23:49.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.260+0000 7f68fee64640 1 -- 172.21.15.154:0/58906041 wait complete. 2023-09-23T03:23:49.279 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-09-23T03:23:49.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.491+0000 7f4c78d16640 1 Processor -- start 2023-09-23T03:23:49.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.491+0000 7f4c78d16640 1 -- start start 2023-09-23T03:23:49.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.492+0000 7f4c78d16640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4c740c9150 0x7f4c7415cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:49.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.492+0000 7f4c78d16640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4c74071290 0x7f4c740ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:49.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.492+0000 7f4c78d16640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c740c9d20 con 0x7f4c740c8d70 2023-09-23T03:23:49.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.492+0000 7f4c78d16640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c74071850 con 0x7f4c74071290 2023-09-23T03:23:49.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.492+0000 7f4c78d16640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c74071b30 con 0x7f4c740c9150 2023-09-23T03:23:49.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.493+0000 7f4c7259c640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f4c740c8d70 0x7f4c7415c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:50950/0 (socket says 172.21.15.154:50950) 2023-09-23T03:23:49.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.493+0000 7f4c72d9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4c74071290 0x7f4c740ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:49.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.493+0000 7f4c7259c640 1 -- 172.21.15.154:0/274165163 learned_addr learned my addr 172.21.15.154:0/274165163 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:23:49.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.493+0000 7f4c6bfff640 1 --2- 172.21.15.154:0/274165163 >> v2:172.21.15.190:3300/0 conn(0x7f4c740c9150 0x7f4c7415cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:49.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.493+0000 7f4c72d9d640 1 -- 172.21.15.154:0/274165163 >> v2:172.21.15.190:3300/0 conn(0x7f4c740c9150 msgr2=0x7f4c7415cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:49.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.493+0000 7f4c72d9d640 1 --2- 172.21.15.154:0/274165163 >> v2:172.21.15.190:3300/0 conn(0x7f4c740c9150 0x7f4c7415cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:49.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.493+0000 7f4c72d9d640 1 -- 172.21.15.154:0/274165163 >> v1:172.21.15.154:6789/0 conn(0x7f4c740c8d70 legacy=0x7f4c7415c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:49.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.493+0000 7f4c72d9d640 1 -- 172.21.15.154:0/274165163 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c74071e10 con 0x7f4c74071290 2023-09-23T03:23:49.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.493+0000 7f4c6bfff640 1 --2- 172.21.15.154:0/274165163 >> v2:172.21.15.190:3300/0 conn(0x7f4c740c9150 0x7f4c7415cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:23:49.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.494+0000 7f4c72d9d640 1 --2- 172.21.15.154:0/274165163 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c74071290 0x7f4c740ca1b0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f4c740b3ce0 tx=0x7f4c6c02bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ea26873f665ee9b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:49.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.495+0000 7f4c71d9b640 1 -- 172.21.15.154:0/274165163 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c02cc70 con 0x7f4c74071290 2023-09-23T03:23:49.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.495+0000 7f4c71d9b640 1 -- 172.21.15.154:0/274165163 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c6c02cdd0 con 0x7f4c74071290 2023-09-23T03:23:49.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.495+0000 7f4c71d9b640 1 -- 172.21.15.154:0/274165163 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c034660 con 0x7f4c74071290 2023-09-23T03:23:49.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.495+0000 7f4c78d16640 1 -- 172.21.15.154:0/274165163 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c74071290 msgr2=0x7f4c740ca1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:49.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.495+0000 7f4c78d16640 1 --2- 172.21.15.154:0/274165163 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c74071290 0x7f4c740ca1b0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f4c740b3ce0 tx=0x7f4c6c02bec0 comp rx=0 tx=0).stop 2023-09-23T03:23:49.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.495+0000 7f4c78d16640 1 -- 172.21.15.154:0/274165163 shutdown_connections 2023-09-23T03:23:49.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.495+0000 7f4c78d16640 1 --2- 172.21.15.154:0/274165163 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c74071290 0x7f4c740ca1b0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:49.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.495+0000 7f4c78d16640 1 --2- 172.21.15.154:0/274165163 >> v2:172.21.15.190:3300/0 conn(0x7f4c740c9150 0x7f4c7415cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:49.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.495+0000 7f4c78d16640 1 -- 172.21.15.154:0/274165163 >> 172.21.15.154:0/274165163 conn(0x7f4c7406c010 msgr2=0x7f4c7406c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:49.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.495+0000 7f4c78d16640 1 -- 172.21.15.154:0/274165163 shutdown_connections 2023-09-23T03:23:49.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.495+0000 7f4c78d16640 1 -- 172.21.15.154:0/274165163 wait complete. 2023-09-23T03:23:49.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.497+0000 7f4c78d16640 1 Processor -- start 2023-09-23T03:23:49.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.497+0000 7f4c78d16640 1 -- start start 2023-09-23T03:23:49.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.498+0000 7f4c78d16640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c74071290 0x7f4c740c8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:49.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.498+0000 7f4c78d16640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c740c8d70 0x7f4c741f2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:49.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.498+0000 7f4c78d16640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c740c9150 0x7f4c7415c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:49.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.498+0000 7f4c78d16640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7415a690 con 0x7f4c740c9150 2023-09-23T03:23:49.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.498+0000 7f4c78d16640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7415c590 con 0x7f4c74071290 2023-09-23T03:23:49.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.498+0000 7f4c78d16640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c741f29c0 con 0x7f4c740c8d70 2023-09-23T03:23:49.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.498+0000 7f4c72d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c740c9150 0x7f4c7415c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:49.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.499+0000 7f4c72d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c740c9150 0x7f4c7415c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60172/0 (socket says 172.21.15.154:60172) 2023-09-23T03:23:49.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.499+0000 7f4c72d9d640 1 -- 172.21.15.154:0/3318382348 learned_addr learned my addr 172.21.15.154:0/3318382348 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:23:49.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.499+0000 7f4c72d9d640 1 -- 172.21.15.154:0/3318382348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c740c8d70 msgr2=0x7f4c741f2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:49.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.499+0000 7f4c72d9d640 1 --2- 172.21.15.154:0/3318382348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c740c8d70 0x7f4c741f2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:49.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.499+0000 7f4c72d9d640 1 -- 172.21.15.154:0/3318382348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c74071290 msgr2=0x7f4c740c8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:49.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.499+0000 7f4c7259c640 1 --2- 172.21.15.154:0/3318382348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c74071290 0x7f4c740c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:49.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.499+0000 7f4c72d9d640 1 --2- 172.21.15.154:0/3318382348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c74071290 0x7f4c740c8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:49.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.499+0000 7f4c72d9d640 1 -- 172.21.15.154:0/3318382348 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6c009a70 con 0x7f4c740c9150 2023-09-23T03:23:49.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.499+0000 7f4c72d9d640 1 --2- 172.21.15.154:0/3318382348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c740c9150 0x7f4c7415c050 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c009e80 tx=0x7f4c6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:49.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.500+0000 7f4c6affd640 1 -- 172.21.15.154:0/3318382348 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c040650 con 0x7f4c740c9150 2023-09-23T03:23:49.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.500+0000 7f4c6affd640 1 -- 172.21.15.154:0/3318382348 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c6c0407b0 con 0x7f4c740c9150 2023-09-23T03:23:49.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.500+0000 7f4c6affd640 1 -- 172.21.15.154:0/3318382348 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c040b60 con 0x7f4c740c9150 2023-09-23T03:23:49.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.500+0000 7f4c78d16640 1 -- 172.21.15.154:0/3318382348 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c741f2ca0 con 0x7f4c740c9150 2023-09-23T03:23:49.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.500+0000 7f4c78d16640 1 -- 172.21.15.154:0/3318382348 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c741f3260 con 0x7f4c740c9150 2023-09-23T03:23:49.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.501+0000 7f4c78d16640 1 -- 172.21.15.154:0/3318382348 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c7415a7d0 con 0x7f4c740c9150 2023-09-23T03:23:49.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.502+0000 7f4c6affd640 1 -- 172.21.15.154:0/3318382348 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f4c6c040cc0 con 0x7f4c740c9150 2023-09-23T03:23:49.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.503+0000 7f4c6affd640 1 --2- 172.21.15.154:0/3318382348 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4c5406af30 0x7f4c5406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:49.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.503+0000 7f4c6affd640 1 -- 172.21.15.154:0/3318382348 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f4c6c0acab0 con 0x7f4c740c9150 2023-09-23T03:23:49.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.503+0000 7f4c7259c640 1 --2- 172.21.15.154:0/3318382348 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4c5406af30 0x7f4c5406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:49.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.504+0000 7f4c7259c640 1 --2- 172.21.15.154:0/3318382348 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4c5406af30 0x7f4c5406d3f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4c64005880 tx=0x7f4c64009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:49.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.507+0000 7f4c6affd640 1 -- 172.21.15.154:0/3318382348 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f4c7415a7d0 con 0x7f4c740c9150 2023-09-23T03:23:49.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.697+0000 7f4c78d16640 1 -- 172.21.15.154:0/3318382348 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7f4c7415a7d0 con 0x7f4c740c9150 2023-09-23T03:23:49.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.698+0000 7f4c6affd640 1 -- 172.21.15.154:0/3318382348 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v71) v1 ==== 106+0+11 (secure 0 0 0) 0x7f4c7415a7d0 con 0x7f4c740c9150 2023-09-23T03:23:49.698 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:23:49.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.701+0000 7f4c78d16640 1 -- 172.21.15.154:0/3318382348 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4c5406af30 msgr2=0x7f4c5406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:49.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.701+0000 7f4c78d16640 1 --2- 172.21.15.154:0/3318382348 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4c5406af30 0x7f4c5406d3f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4c64005880 tx=0x7f4c64009040 comp rx=0 tx=0).stop 2023-09-23T03:23:49.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.701+0000 7f4c78d16640 1 -- 172.21.15.154:0/3318382348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c740c9150 msgr2=0x7f4c7415c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:49.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.701+0000 7f4c78d16640 1 --2- 172.21.15.154:0/3318382348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c740c9150 0x7f4c7415c050 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c009e80 tx=0x7f4c6c004a40 comp rx=0 tx=0).stop 2023-09-23T03:23:49.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.701+0000 7f4c78d16640 1 -- 172.21.15.154:0/3318382348 shutdown_connections 2023-09-23T03:23:49.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.701+0000 7f4c78d16640 1 --2- 172.21.15.154:0/3318382348 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4c5406af30 0x7f4c5406d3f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:49.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.701+0000 7f4c78d16640 1 --2- 172.21.15.154:0/3318382348 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c740c9150 0x7f4c7415c050 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:49.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.701+0000 7f4c78d16640 1 --2- 172.21.15.154:0/3318382348 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c740c8d70 0x7f4c741f2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:49.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.701+0000 7f4c78d16640 1 --2- 172.21.15.154:0/3318382348 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c74071290 0x7f4c740c8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:49.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.701+0000 7f4c78d16640 1 -- 172.21.15.154:0/3318382348 >> 172.21.15.154:0/3318382348 conn(0x7f4c7406c010 msgr2=0x7f4c740cdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:49.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.701+0000 7f4c78d16640 1 -- 172.21.15.154:0/3318382348 shutdown_connections 2023-09-23T03:23:49.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.701+0000 7f4c78d16640 1 -- 172.21.15.154:0/3318382348 wait complete. 2023-09-23T03:23:49.719 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-23T03:23:49.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.933+0000 7f6331007640 1 Processor -- start 2023-09-23T03:23:49.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.934+0000 7f6331007640 1 -- start start 2023-09-23T03:23:49.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.935+0000 7f6331007640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632c0acce0 0x7f632c15ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:49.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.935+0000 7f6331007640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f632c15ac30 con 0x7f632c0cf040 2023-09-23T03:23:49.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.935+0000 7f6331007640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f632c0b1f70 con 0x7f632c0b1b20 2023-09-23T03:23:49.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.935+0000 7f6331007640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f632c0b2e90 con 0x7f632c0acce0 2023-09-23T03:23:49.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.936+0000 7f632b59e640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f632c0cf040 0x7f632c0ab140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:50952/0 (socket says 172.21.15.154:50952) 2023-09-23T03:23:49.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.936+0000 7f632b59e640 1 -- 172.21.15.154:0/2622211652 learned_addr learned my addr 172.21.15.154:0/2622211652 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:23:49.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.936+0000 7f632ad9d640 1 --2- 172.21.15.154:0/2622211652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632c0acce0 0x7f632c15ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:49.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.936+0000 7f6329d9b640 1 -- 172.21.15.154:0/2622211652 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 345232825 0 0) 0x7f6324001110 con 0x7f632c0cf040 2023-09-23T03:23:49.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.936+0000 7f6329d9b640 1 -- 172.21.15.154:0/2622211652 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6314003520 con 0x7f632c0cf040 2023-09-23T03:23:49.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.936+0000 7f6329d9b640 1 -- 172.21.15.154:0/2622211652 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 836547661 0 0) 0x7f6314003520 con 0x7f632c0cf040 2023-09-23T03:23:49.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.936+0000 7f6329d9b640 1 -- 172.21.15.154:0/2622211652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632c0acce0 msgr2=0x7f632c15ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:49.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.936+0000 7f6329d9b640 1 --2- 172.21.15.154:0/2622211652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632c0acce0 0x7f632c15ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:49.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.936+0000 7f6329d9b640 1 -- 172.21.15.154:0/2622211652 >> v1:172.21.15.190:6789/0 conn(0x7f632c0b1b20 legacy=0x7f632c0ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:49.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.936+0000 7f6329d9b640 1 -- 172.21.15.154:0/2622211652 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f632c15f2a0 con 0x7f632c0cf040 2023-09-23T03:23:49.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.936+0000 7f6329d9b640 1 -- 172.21.15.154:0/2622211652 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f6324001800 con 0x7f632c0cf040 2023-09-23T03:23:49.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.937+0000 7f6329d9b640 1 -- 172.21.15.154:0/2622211652 >> v1:172.21.15.154:6789/0 conn(0x7f632c0cf040 legacy=0x7f632c0ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:49.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.937+0000 7f6329d9b640 1 --2- 172.21.15.154:0/2622211652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63140039a0 0x7f6314003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:49.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.937+0000 7f6329d9b640 1 --2- 172.21.15.154:0/2622211652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63140047c0 0x7f6314004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:49.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.937+0000 7f6329d9b640 1 --2- 172.21.15.154:0/2622211652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632c0acce0 0x7f6314005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:49.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.937+0000 7f6329d9b640 1 -- 172.21.15.154:0/2622211652 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f632c0b2e90 con 0x7f63140047c0 2023-09-23T03:23:49.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.937+0000 7f6329d9b640 1 -- 172.21.15.154:0/2622211652 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631400bb10 con 0x7f63140039a0 2023-09-23T03:23:49.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.937+0000 7f6329d9b640 1 -- 172.21.15.154:0/2622211652 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f631400bcd0 con 0x7f632c0acce0 2023-09-23T03:23:49.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.937+0000 7f632a59c640 1 --2- 172.21.15.154:0/2622211652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63140047c0 0x7f6314004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:49.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.937+0000 7f632b59e640 1 --2- 172.21.15.154:0/2622211652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632c0acce0 0x7f6314005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:49.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.937+0000 7f632ad9d640 1 --2- 172.21.15.154:0/2622211652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63140039a0 0x7f6314003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:49.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.937+0000 7f632a59c640 1 -- 172.21.15.154:0/2622211652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632c0acce0 msgr2=0x7f6314005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:49.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.937+0000 7f632a59c640 1 --2- 172.21.15.154:0/2622211652 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632c0acce0 0x7f6314005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:49.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.937+0000 7f632a59c640 1 -- 172.21.15.154:0/2622211652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63140039a0 msgr2=0x7f6314003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:49.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.937+0000 7f632a59c640 1 --2- 172.21.15.154:0/2622211652 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63140039a0 0x7f6314003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:49.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.937+0000 7f632a59c640 1 -- 172.21.15.154:0/2622211652 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6314003520 con 0x7f63140047c0 2023-09-23T03:23:49.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.938+0000 7f632a59c640 1 --2- 172.21.15.154:0/2622211652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63140047c0 0x7f6314004c40 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f6320001720 tx=0x7f63140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66f322cf2d19aeae server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:49.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.939+0000 7f6329d9b640 1 -- 172.21.15.154:0/2622211652 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63200052d0 con 0x7f63140047c0 2023-09-23T03:23:49.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.939+0000 7f6329d9b640 1 -- 172.21.15.154:0/2622211652 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f632002e050 con 0x7f63140047c0 2023-09-23T03:23:49.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.939+0000 7f6329d9b640 1 -- 172.21.15.154:0/2622211652 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63200035f0 con 0x7f63140047c0 2023-09-23T03:23:49.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.939+0000 7f6331007640 1 -- 172.21.15.154:0/2622211652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63140047c0 msgr2=0x7f6314004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:49.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.939+0000 7f6331007640 1 --2- 172.21.15.154:0/2622211652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63140047c0 0x7f6314004c40 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f6320001720 tx=0x7f63140057f0 comp rx=0 tx=0).stop 2023-09-23T03:23:49.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.939+0000 7f632b59e640 1 -- 172.21.15.154:0/2622211652 reap_dead start 2023-09-23T03:23:49.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.939+0000 7f6331007640 1 -- 172.21.15.154:0/2622211652 shutdown_connections 2023-09-23T03:23:49.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.939+0000 7f6331007640 1 -- 172.21.15.154:0/2622211652 >> 172.21.15.154:0/2622211652 conn(0x7f632c06a0f0 msgr2=0x7f632c06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:49.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.939+0000 7f6331007640 1 -- 172.21.15.154:0/2622211652 shutdown_connections 2023-09-23T03:23:49.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.939+0000 7f6331007640 1 -- 172.21.15.154:0/2622211652 wait complete. 2023-09-23T03:23:49.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.941+0000 7f6331007640 1 Processor -- start 2023-09-23T03:23:49.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.941+0000 7f6331007640 1 -- start start 2023-09-23T03:23:49.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.942+0000 7f6331007640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63140047c0 0x7f632c0c55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:49.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.942+0000 7f6331007640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632c0ac110 0x7f632c14cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:49.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.942+0000 7f6331007640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f632c0ac590 0x7f632c14d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:49.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.942+0000 7f6331007640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f632c0c3490 con 0x7f632c0ac590 2023-09-23T03:23:49.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.942+0000 7f6331007640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f632c15bd70 con 0x7f63140047c0 2023-09-23T03:23:49.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.942+0000 7f6331007640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f632c0c28e0 con 0x7f632c0ac110 2023-09-23T03:23:49.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.943+0000 7f632b59e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f632c0ac590 0x7f632c14d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:49.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.943+0000 7f632b59e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f632c0ac590 0x7f632c14d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60194/0 (socket says 172.21.15.154:60194) 2023-09-23T03:23:49.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.943+0000 7f632b59e640 1 -- 172.21.15.154:0/3424282588 learned_addr learned my addr 172.21.15.154:0/3424282588 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:23:49.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.943+0000 7f632b59e640 1 -- 172.21.15.154:0/3424282588 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632c0ac110 msgr2=0x7f632c14cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:49.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.943+0000 7f632b59e640 1 --2- 172.21.15.154:0/3424282588 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632c0ac110 0x7f632c14cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:49.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.943+0000 7f632ad9d640 1 --2- 172.21.15.154:0/3424282588 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63140047c0 0x7f632c0c55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:49.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.943+0000 7f632b59e640 1 -- 172.21.15.154:0/3424282588 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63140047c0 msgr2=0x7f632c0c55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:49.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.943+0000 7f632b59e640 1 --2- 172.21.15.154:0/3424282588 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63140047c0 0x7f632c0c55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:49.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.943+0000 7f632b59e640 1 -- 172.21.15.154:0/3424282588 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f632c0c2af0 con 0x7f632c0ac590 2023-09-23T03:23:49.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.943+0000 7f632b59e640 1 --2- 172.21.15.154:0/3424282588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f632c0ac590 0x7f632c14d2a0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f6324001eb0 tx=0x7f632400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:49.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.944+0000 7f63137fe640 1 -- 172.21.15.154:0/3424282588 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632400c630 con 0x7f632c0ac590 2023-09-23T03:23:49.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.944+0000 7f63137fe640 1 -- 172.21.15.154:0/3424282588 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f632400c7b0 con 0x7f632c0ac590 2023-09-23T03:23:49.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.944+0000 7f63137fe640 1 -- 172.21.15.154:0/3424282588 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632400cb60 con 0x7f632c0ac590 2023-09-23T03:23:49.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.944+0000 7f6331007640 1 -- 172.21.15.154:0/3424282588 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f632c0c2dd0 con 0x7f632c0ac590 2023-09-23T03:23:49.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.944+0000 7f6331007640 1 -- 172.21.15.154:0/3424282588 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f632c0c5b20 con 0x7f632c0ac590 2023-09-23T03:23:49.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.945+0000 7f6331007640 1 -- 172.21.15.154:0/3424282588 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f632c0c35d0 con 0x7f632c0ac590 2023-09-23T03:23:49.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.946+0000 7f63137fe640 1 -- 172.21.15.154:0/3424282588 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f632400ccc0 con 0x7f632c0ac590 2023-09-23T03:23:49.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.947+0000 7f63137fe640 1 --2- 172.21.15.154:0/3424282588 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f631806af30 0x7f631806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:49.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.947+0000 7f63137fe640 1 -- 172.21.15.154:0/3424282588 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f63240895d0 con 0x7f632c0ac590 2023-09-23T03:23:49.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.947+0000 7f632ad9d640 1 --2- 172.21.15.154:0/3424282588 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f631806af30 0x7f631806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:49.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.948+0000 7f632ad9d640 1 --2- 172.21.15.154:0/3424282588 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f631806af30 0x7f631806d3f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f631c003aa0 tx=0x7f631c00b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:49.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:49.951+0000 7f63137fe640 1 -- 172.21.15.154:0/3424282588 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f632c0c35d0 con 0x7f632c0ac590 2023-09-23T03:23:50.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:50.157+0000 7f6331007640 1 -- 172.21.15.154:0/3424282588 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f632c15ac30 con 0x7f632c0ac590 2023-09-23T03:23:50.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:50.158+0000 7f63137fe640 1 -- 172.21.15.154:0/3424282588 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v71) v1 ==== 80+0+372 (secure 0 0 0) 0x7f632c15ac30 con 0x7f632c0ac590 2023-09-23T03:23:50.158 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.154:6837/4129414148 2023-09-24T03:21:38.278599+0000 2023-09-23T03:23:50.158 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.154:6835/4129414148 2023-09-24T03:21:38.278599+0000 2023-09-23T03:23:50.159 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.154:6837/2976977273 2023-09-24T03:21:31.069171+0000 2023-09-23T03:23:50.159 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.154:6835/2976977273 2023-09-24T03:21:31.069171+0000 2023-09-23T03:23:50.159 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.154:6837/2183023203 2023-09-24T03:21:29.032871+0000 2023-09-23T03:23:50.159 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.154:6835/2183023203 2023-09-24T03:21:29.032871+0000 2023-09-23T03:23:50.159 INFO:teuthology.orchestra.run.smithi154.stderr:listed 6 entries 2023-09-23T03:23:50.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:50.160+0000 7f6331007640 1 -- 172.21.15.154:0/3424282588 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f631806af30 msgr2=0x7f631806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:50.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:50.160+0000 7f6331007640 1 --2- 172.21.15.154:0/3424282588 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f631806af30 0x7f631806d3f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f631c003aa0 tx=0x7f631c00b040 comp rx=0 tx=0).stop 2023-09-23T03:23:50.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:50.161+0000 7f6331007640 1 -- 172.21.15.154:0/3424282588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f632c0ac590 msgr2=0x7f632c14d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:50.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:50.161+0000 7f6331007640 1 --2- 172.21.15.154:0/3424282588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f632c0ac590 0x7f632c14d2a0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f6324001eb0 tx=0x7f632400ec10 comp rx=0 tx=0).stop 2023-09-23T03:23:50.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:50.161+0000 7f6331007640 1 -- 172.21.15.154:0/3424282588 shutdown_connections 2023-09-23T03:23:50.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:50.161+0000 7f6331007640 1 --2- 172.21.15.154:0/3424282588 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f631806af30 0x7f631806d3f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:50.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:50.161+0000 7f6331007640 1 --2- 172.21.15.154:0/3424282588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f632c0ac590 0x7f632c14d2a0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:50.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:50.161+0000 7f6331007640 1 --2- 172.21.15.154:0/3424282588 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f632c0ac110 0x7f632c14cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:50.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:50.161+0000 7f6331007640 1 --2- 172.21.15.154:0/3424282588 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63140047c0 0x7f632c0c55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:50.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:50.161+0000 7f6331007640 1 -- 172.21.15.154:0/3424282588 >> 172.21.15.154:0/3424282588 conn(0x7f632c06a0f0 msgr2=0x7f632c0b0870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:50.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:50.161+0000 7f6331007640 1 -- 172.21.15.154:0/3424282588 shutdown_connections 2023-09-23T03:23:50.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:50.161+0000 7f6331007640 1 -- 172.21.15.154:0/3424282588 wait complete. 2023-09-23T03:23:50.180 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi154.front.sepia.ceph.com... 2023-09-23T03:23:50.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:23:50.181 DEBUG:teuthology.orchestra.run.smithi154:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-09-23T03:23:50.271 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:ceph-fuse[112314]: fuse finished with error 0 and tester_r 0 2023-09-23T03:23:50.310 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2023-09-23T03:23:50.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:23:50.311 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-09-23T03:23:50.488 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:23:50.488 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:23:50.548 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053603,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248804k,nr_inodes=812201,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-09-23T03:23:50.549 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi190.front.sepia.ceph.com 2023-09-23T03:23:50.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:23:50.549 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-09-23T03:23:50.603 INFO:teuthology.orchestra.run.smithi190.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-09-23T03:23:50.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T03:23:50.604 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:23:50.604 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:23:50.664 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053600,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-09-23T03:23:50.665 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2023-09-23T03:23:50.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:23:50.665 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-09-23T03:23:50.719 INFO:teuthology.orchestra.run.smithi154.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-09-23T03:23:50.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T03:23:50.720 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:23:50.720 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:23:50.781 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053603,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248804k,nr_inodes=812201,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-09-23T03:23:50.781 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi190.front.sepia.ceph.com 2023-09-23T03:23:50.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:23:50.781 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-09-23T03:23:50.839 INFO:teuthology.orchestra.run.smithi190.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-09-23T03:23:50.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T03:23:50.841 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:23:51.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.057+0000 7f66960c9640 1 Processor -- start 2023-09-23T03:23:51.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.057+0000 7f66960c9640 1 -- start start 2023-09-23T03:23:51.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.059+0000 7f66960c9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f66900acce0 0x7f669015ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:51.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.059+0000 7f66960c9640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f669015ac30 con 0x7f66900b1b20 2023-09-23T03:23:51.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.059+0000 7f66960c9640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66900b1f70 con 0x7f66900cf040 2023-09-23T03:23:51.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.059+0000 7f66960c9640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66900b2e90 con 0x7f66900acce0 2023-09-23T03:23:51.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.059+0000 7f6687fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f66900b1b20 0x7f66900ce930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:50956/0 (socket says 172.21.15.154:50956) 2023-09-23T03:23:51.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.059+0000 7f6687fff640 1 -- 172.21.15.154:0/1259903533 learned_addr learned my addr 172.21.15.154:0/1259903533 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:23:51.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.059+0000 7f668f7fe640 1 --2- 172.21.15.154:0/1259903533 >> v2:172.21.15.190:3300/0 conn(0x7f66900acce0 0x7f669015ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:51.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.059+0000 7f668effd640 1 -- 172.21.15.154:0/1259903533 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4225044808 0 0) 0x7f6678001110 con 0x7f66900b1b20 2023-09-23T03:23:51.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.060+0000 7f668effd640 1 -- 172.21.15.154:0/1259903533 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f667c003520 con 0x7f66900b1b20 2023-09-23T03:23:51.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.060+0000 7f668effd640 1 -- 172.21.15.154:0/1259903533 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2940146925 0 0) 0x7f667c003520 con 0x7f66900b1b20 2023-09-23T03:23:51.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.060+0000 7f668effd640 1 -- 172.21.15.154:0/1259903533 >> v2:172.21.15.190:3300/0 conn(0x7f66900acce0 msgr2=0x7f669015ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:51.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.060+0000 7f668effd640 1 --2- 172.21.15.154:0/1259903533 >> v2:172.21.15.190:3300/0 conn(0x7f66900acce0 0x7f669015ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:51.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.060+0000 7f668effd640 1 -- 172.21.15.154:0/1259903533 >> v1:172.21.15.190:6789/0 conn(0x7f66900cf040 legacy=0x7f66900ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:51.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.060+0000 7f668effd640 1 -- 172.21.15.154:0/1259903533 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f669015f2a0 con 0x7f66900b1b20 2023-09-23T03:23:51.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.060+0000 7f668effd640 1 -- 172.21.15.154:0/1259903533 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f6678001800 con 0x7f66900b1b20 2023-09-23T03:23:51.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.060+0000 7f668effd640 1 -- 172.21.15.154:0/1259903533 >> v1:172.21.15.154:6789/0 conn(0x7f66900b1b20 legacy=0x7f66900ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:51.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.060+0000 7f668effd640 1 --2- 172.21.15.154:0/1259903533 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f667c003970 0x7f667c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:51.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.060+0000 7f668effd640 1 --2- 172.21.15.154:0/1259903533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f667c0047c0 0x7f667c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:51.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.060+0000 7f668effd640 1 --2- 172.21.15.154:0/1259903533 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f667c005180 0x7f667c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:51.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.060+0000 7f668effd640 1 -- 172.21.15.154:0/1259903533 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66900b2e90 con 0x7f667c005180 2023-09-23T03:23:51.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.060+0000 7f668effd640 1 -- 172.21.15.154:0/1259903533 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667c00c120 con 0x7f667c0047c0 2023-09-23T03:23:51.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.060+0000 7f668effd640 1 -- 172.21.15.154:0/1259903533 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f667c00c2e0 con 0x7f667c003970 2023-09-23T03:23:51.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.060+0000 7f668ffff640 1 --2- 172.21.15.154:0/1259903533 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f667c005180 0x7f667c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:51.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.061+0000 7f6687fff640 1 --2- 172.21.15.154:0/1259903533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f667c0047c0 0x7f667c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:51.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.061+0000 7f668ffff640 1 -- 172.21.15.154:0/1259903533 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f667c003970 msgr2=0x7f667c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:51.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.061+0000 7f668f7fe640 1 --2- 172.21.15.154:0/1259903533 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f667c003970 0x7f667c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:51.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.061+0000 7f668ffff640 1 --2- 172.21.15.154:0/1259903533 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f667c003970 0x7f667c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:51.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.061+0000 7f668ffff640 1 -- 172.21.15.154:0/1259903533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f667c0047c0 msgr2=0x7f667c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:51.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.061+0000 7f668ffff640 1 --2- 172.21.15.154:0/1259903533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f667c0047c0 0x7f667c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:51.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.061+0000 7f668ffff640 1 -- 172.21.15.154:0/1259903533 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f667c003520 con 0x7f667c005180 2023-09-23T03:23:51.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.061+0000 7f668f7fe640 1 --2- 172.21.15.154:0/1259903533 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f667c003970 0x7f667c003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:23:51.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.061+0000 7f6687fff640 1 --2- 172.21.15.154:0/1259903533 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f667c0047c0 0x7f667c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:23:51.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.062+0000 7f668ffff640 1 -- 172.21.15.154:0/1259903533 reap_dead start 2023-09-23T03:23:51.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.062+0000 7f668ffff640 1 --2- 172.21.15.154:0/1259903533 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f667c005180 0x7f667c00b980 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f6688001e20 tx=0x7f667c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d447ff7d34907d04 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:51.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.062+0000 7f668effd640 1 -- 172.21.15.154:0/1259903533 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6688003b70 con 0x7f667c005180 2023-09-23T03:23:51.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.062+0000 7f668effd640 1 -- 172.21.15.154:0/1259903533 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6688003cd0 con 0x7f667c005180 2023-09-23T03:23:51.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.062+0000 7f668effd640 1 -- 172.21.15.154:0/1259903533 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66880052f0 con 0x7f667c005180 2023-09-23T03:23:51.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.063+0000 7f66960c9640 1 -- 172.21.15.154:0/1259903533 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f667c005180 msgr2=0x7f667c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:51.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.063+0000 7f66960c9640 1 --2- 172.21.15.154:0/1259903533 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f667c005180 0x7f667c00b980 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f6688001e20 tx=0x7f667c0057f0 comp rx=0 tx=0).stop 2023-09-23T03:23:51.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.063+0000 7f66960c9640 1 -- 172.21.15.154:0/1259903533 shutdown_connections 2023-09-23T03:23:51.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.063+0000 7f66960c9640 1 --2- 172.21.15.154:0/1259903533 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f667c005180 0x7f667c00b980 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:51.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.063+0000 7f66960c9640 1 -- 172.21.15.154:0/1259903533 >> 172.21.15.154:0/1259903533 conn(0x7f669006a0f0 msgr2=0x7f669006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:51.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.063+0000 7f66960c9640 1 -- 172.21.15.154:0/1259903533 shutdown_connections 2023-09-23T03:23:51.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.063+0000 7f66960c9640 1 -- 172.21.15.154:0/1259903533 wait complete. 2023-09-23T03:23:51.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.065+0000 7f66960c9640 1 Processor -- start 2023-09-23T03:23:51.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.066+0000 7f66960c9640 1 -- start start 2023-09-23T03:23:51.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.067+0000 7f66960c9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f667c005180 0x7f6690159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:51.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.067+0000 7f66960c9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66900c7a30 0x7f66900c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:51.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.067+0000 7f66960c9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f669015c950 0x7f669015cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:51.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.067+0000 7f66960c9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66900c83f0 con 0x7f667c005180 2023-09-23T03:23:51.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.067+0000 7f66960c9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669015d6f0 con 0x7f669015c950 2023-09-23T03:23:51.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.067+0000 7f66960c9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66900cc3a0 con 0x7f66900c7a30 2023-09-23T03:23:51.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.067+0000 7f668f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f667c005180 0x7f6690159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:51.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.067+0000 7f668f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f667c005180 0x7f6690159b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60208/0 (socket says 172.21.15.154:60208) 2023-09-23T03:23:51.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.067+0000 7f668f7fe640 1 -- 172.21.15.154:0/925721053 learned_addr learned my addr 172.21.15.154:0/925721053 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:23:51.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.067+0000 7f668ffff640 1 --2- 172.21.15.154:0/925721053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f669015c950 0x7f669015cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:51.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.067+0000 7f668f7fe640 1 -- 172.21.15.154:0/925721053 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66900c7a30 msgr2=0x7f66900c7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:51.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.067+0000 7f6687fff640 1 --2- 172.21.15.154:0/925721053 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66900c7a30 0x7f66900c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:51.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.068+0000 7f668f7fe640 1 --2- 172.21.15.154:0/925721053 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66900c7a30 0x7f66900c7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:51.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.068+0000 7f668f7fe640 1 -- 172.21.15.154:0/925721053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f669015c950 msgr2=0x7f669015cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:51.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.068+0000 7f668f7fe640 1 --2- 172.21.15.154:0/925721053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f669015c950 0x7f669015cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:51.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.068+0000 7f668f7fe640 1 -- 172.21.15.154:0/925721053 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66900cc680 con 0x7f667c005180 2023-09-23T03:23:51.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.068+0000 7f668f7fe640 1 --2- 172.21.15.154:0/925721053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f667c005180 0x7f6690159b50 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f6680010900 tx=0x7f6680010dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:51.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.068+0000 7f668cff9640 1 -- 172.21.15.154:0/925721053 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6680004220 con 0x7f667c005180 2023-09-23T03:23:51.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.068+0000 7f668cff9640 1 -- 172.21.15.154:0/925721053 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6680004380 con 0x7f667c005180 2023-09-23T03:23:51.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.068+0000 7f668cff9640 1 -- 172.21.15.154:0/925721053 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6680005bd0 con 0x7f667c005180 2023-09-23T03:23:51.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.068+0000 7f66960c9640 1 -- 172.21.15.154:0/925721053 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66900cc960 con 0x7f667c005180 2023-09-23T03:23:51.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.068+0000 7f66960c9640 1 -- 172.21.15.154:0/925721053 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66900c55e0 con 0x7f667c005180 2023-09-23T03:23:51.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.070+0000 7f66960c9640 1 -- 172.21.15.154:0/925721053 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66900ccb20 con 0x7f667c005180 2023-09-23T03:23:51.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.070+0000 7f668cff9640 1 -- 172.21.15.154:0/925721053 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f6680005d30 con 0x7f667c005180 2023-09-23T03:23:51.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.071+0000 7f668cff9640 1 --2- 172.21.15.154:0/925721053 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f667006af60 0x7f667006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:51.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.072+0000 7f668cff9640 1 -- 172.21.15.154:0/925721053 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f6680089e50 con 0x7f667c005180 2023-09-23T03:23:51.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.072+0000 7f6687fff640 1 --2- 172.21.15.154:0/925721053 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f667006af60 0x7f667006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:51.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.072+0000 7f6687fff640 1 --2- 172.21.15.154:0/925721053 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f667006af60 0x7f667006d420 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6678001eb0 tx=0x7f6678009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:51.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.076+0000 7f668cff9640 1 -- 172.21.15.154:0/925721053 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f6680054b40 con 0x7f667c005180 2023-09-23T03:23:51.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.302+0000 7f66960c9640 1 -- 172.21.15.154:0/925721053 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66900cc200 con 0x7f667c005180 2023-09-23T03:23:51.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.303+0000 7f668cff9640 1 -- 172.21.15.154:0/925721053 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4926 (secure 0 0 0) 0x7f66900cc200 con 0x7f667c005180 2023-09-23T03:23:51.304 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:23:51.304 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":54,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5279,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1609915000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1609915000},{"type":"v1","addr":"172.21.15.154:6837","nonce":1609915000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":5288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1240415394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1240415394},{"type":"v1","addr":"172.21.15.190:6835","nonce":1240415394}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":5728,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/878668736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":878668736},{"type":"v1","addr":"172.21.15.190:6833","nonce":878668736}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:21:30.160245+0000","modified":"2023-09-23T03:21:40.311897+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6924},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6924":{"gid":6924,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":4,"addr":"172.21.15.154:6835/2633473586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2633473586},{"type":"v1","addr":"172.21.15.154:6835","nonce":2633473586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9,10],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-09-23T03:23:51.304 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 54 2023-09-23T03:23:51.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.306+0000 7f66960c9640 1 -- 172.21.15.154:0/925721053 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f667006af60 msgr2=0x7f667006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:51.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.306+0000 7f66960c9640 1 --2- 172.21.15.154:0/925721053 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f667006af60 0x7f667006d420 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6678001eb0 tx=0x7f6678009040 comp rx=0 tx=0).stop 2023-09-23T03:23:51.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.306+0000 7f66960c9640 1 -- 172.21.15.154:0/925721053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f667c005180 msgr2=0x7f6690159b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:51.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.306+0000 7f66960c9640 1 --2- 172.21.15.154:0/925721053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f667c005180 0x7f6690159b50 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f6680010900 tx=0x7f6680010dc0 comp rx=0 tx=0).stop 2023-09-23T03:23:51.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.306+0000 7f66960c9640 1 -- 172.21.15.154:0/925721053 shutdown_connections 2023-09-23T03:23:51.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.306+0000 7f66960c9640 1 --2- 172.21.15.154:0/925721053 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f667006af60 0x7f667006d420 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:51.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.306+0000 7f66960c9640 1 --2- 172.21.15.154:0/925721053 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f669015c950 0x7f669015cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:51.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.306+0000 7f66960c9640 1 --2- 172.21.15.154:0/925721053 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f66900c7a30 0x7f66900c7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:51.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.306+0000 7f66960c9640 1 --2- 172.21.15.154:0/925721053 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f667c005180 0x7f6690159b50 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:51.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.306+0000 7f66960c9640 1 -- 172.21.15.154:0/925721053 >> 172.21.15.154:0/925721053 conn(0x7f669006a0f0 msgr2=0x7f66900affe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:51.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.306+0000 7f66960c9640 1 -- 172.21.15.154:0/925721053 shutdown_connections 2023-09-23T03:23:51.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.306+0000 7f66960c9640 1 -- 172.21.15.154:0/925721053 wait complete. 2023-09-23T03:23:51.326 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:23:51.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.545+0000 7f4c4b24f640 1 Processor -- start 2023-09-23T03:23:51.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.545+0000 7f4c4b24f640 1 -- start start 2023-09-23T03:23:51.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.546+0000 7f4c4b24f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4c440b2420 0x7f4c441605d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:51.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.546+0000 7f4c4b24f640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4c440b2800 0x7f4c4415d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:51.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.546+0000 7f4c4b24f640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c4406b350 con 0x7f4c440ceff0 2023-09-23T03:23:51.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.546+0000 7f4c4b24f640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c4406bfd0 con 0x7f4c440b2420 2023-09-23T03:23:51.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.546+0000 7f4c4b24f640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c4406b630 con 0x7f4c440b2800 2023-09-23T03:23:51.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.546+0000 7f4c48feb640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4c440b2420 0x7f4c441605d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:51.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.546+0000 7f4c43fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f4c440ceff0 0x7f4c4406c510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:50960/0 (socket says 172.21.15.154:50960) 2023-09-23T03:23:51.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.546+0000 7f4c43fff640 1 -- 172.21.15.154:0/2908331648 learned_addr learned my addr 172.21.15.154:0/2908331648 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:23:51.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.547+0000 7f4c497ec640 1 --2- 172.21.15.154:0/2908331648 >> v2:172.21.15.190:3300/0 conn(0x7f4c440b2800 0x7f4c4415d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:51.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.547+0000 7f4c437fe640 1 -- 172.21.15.154:0/2908331648 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1840305100 0 0) 0x7f4c4406b350 con 0x7f4c440ceff0 2023-09-23T03:23:51.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.547+0000 7f4c437fe640 1 -- 172.21.15.154:0/2908331648 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c30003520 con 0x7f4c440ceff0 2023-09-23T03:23:51.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.547+0000 7f4c48feb640 1 -- 172.21.15.154:0/2908331648 >> v2:172.21.15.190:3300/0 conn(0x7f4c440b2800 msgr2=0x7f4c4415d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:51.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.547+0000 7f4c48feb640 1 --2- 172.21.15.154:0/2908331648 >> v2:172.21.15.190:3300/0 conn(0x7f4c440b2800 0x7f4c4415d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:51.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.547+0000 7f4c48feb640 1 -- 172.21.15.154:0/2908331648 >> v1:172.21.15.154:6789/0 conn(0x7f4c440ceff0 legacy=0x7f4c4406c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:51.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.547+0000 7f4c48feb640 1 -- 172.21.15.154:0/2908331648 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c4406b910 con 0x7f4c440b2420 2023-09-23T03:23:51.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.548+0000 7f4c48feb640 1 --2- 172.21.15.154:0/2908331648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c440b2420 0x7f4c441605d0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f4c440721a0 tx=0x7f4c3402bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1ec99a4e10b29b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:51.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.548+0000 7f4c437fe640 1 -- 172.21.15.154:0/2908331648 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3402cb50 con 0x7f4c440b2420 2023-09-23T03:23:51.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.548+0000 7f4c437fe640 1 -- 172.21.15.154:0/2908331648 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3402ccb0 con 0x7f4c440b2420 2023-09-23T03:23:51.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.548+0000 7f4c437fe640 1 -- 172.21.15.154:0/2908331648 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c34034660 con 0x7f4c440b2420 2023-09-23T03:23:51.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.549+0000 7f4c4b24f640 1 -- 172.21.15.154:0/2908331648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c440b2420 msgr2=0x7f4c441605d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:51.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.549+0000 7f4c4b24f640 1 --2- 172.21.15.154:0/2908331648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c440b2420 0x7f4c441605d0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f4c440721a0 tx=0x7f4c3402bda0 comp rx=0 tx=0).stop 2023-09-23T03:23:51.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.549+0000 7f4c4b24f640 1 -- 172.21.15.154:0/2908331648 shutdown_connections 2023-09-23T03:23:51.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.549+0000 7f4c4b24f640 1 --2- 172.21.15.154:0/2908331648 >> v2:172.21.15.190:3300/0 conn(0x7f4c440b2800 0x7f4c4415d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:51.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.549+0000 7f4c4b24f640 1 --2- 172.21.15.154:0/2908331648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c440b2420 0x7f4c441605d0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:51.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.549+0000 7f4c4b24f640 1 -- 172.21.15.154:0/2908331648 >> 172.21.15.154:0/2908331648 conn(0x7f4c440cd1b0 msgr2=0x7f4c440cd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:51.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.549+0000 7f4c4b24f640 1 -- 172.21.15.154:0/2908331648 shutdown_connections 2023-09-23T03:23:51.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.549+0000 7f4c4b24f640 1 -- 172.21.15.154:0/2908331648 wait complete. 2023-09-23T03:23:51.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.551+0000 7f4c4b24f640 1 Processor -- start 2023-09-23T03:23:51.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.551+0000 7f4c4b24f640 1 -- start start 2023-09-23T03:23:51.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.552+0000 7f4c4b24f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c440b2420 0x7f4c441f68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:51.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.552+0000 7f4c4b24f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c440b2800 0x7f4c440b5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:51.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.552+0000 7f4c4b24f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c440c6720 0x7f4c440c6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:51.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.552+0000 7f4c4b24f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c440c7110 con 0x7f4c440b2420 2023-09-23T03:23:51.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.552+0000 7f4c4b24f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c440b42d0 con 0x7f4c440c6720 2023-09-23T03:23:51.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.552+0000 7f4c4b24f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c441f6e10 con 0x7f4c440b2800 2023-09-23T03:23:51.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.552+0000 7f4c48feb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c440b2420 0x7f4c441f68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:51.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.552+0000 7f4c48feb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c440b2420 0x7f4c441f68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60234/0 (socket says 172.21.15.154:60234) 2023-09-23T03:23:51.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.552+0000 7f4c48feb640 1 -- 172.21.15.154:0/914487278 learned_addr learned my addr 172.21.15.154:0/914487278 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:23:51.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.552+0000 7f4c48feb640 1 -- 172.21.15.154:0/914487278 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c440b2800 msgr2=0x7f4c440b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:51.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.552+0000 7f4c48feb640 1 --2- 172.21.15.154:0/914487278 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c440b2800 0x7f4c440b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:51.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.552+0000 7f4c48feb640 1 -- 172.21.15.154:0/914487278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c440c6720 msgr2=0x7f4c440c6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:51.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.553+0000 7f4c48feb640 1 --2- 172.21.15.154:0/914487278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c440c6720 0x7f4c440c6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:51.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.553+0000 7f4c48feb640 1 -- 172.21.15.154:0/914487278 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c34009a70 con 0x7f4c440b2420 2023-09-23T03:23:51.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.553+0000 7f4c48feb640 1 --2- 172.21.15.154:0/914487278 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c440b2420 0x7f4c441f68d0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f4c34009e80 tx=0x7f4c34004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:51.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.553+0000 7f4c417fa640 1 -- 172.21.15.154:0/914487278 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c34040650 con 0x7f4c440b2420 2023-09-23T03:23:51.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.553+0000 7f4c417fa640 1 -- 172.21.15.154:0/914487278 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c340407b0 con 0x7f4c440b2420 2023-09-23T03:23:51.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.553+0000 7f4c417fa640 1 -- 172.21.15.154:0/914487278 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c34040b60 con 0x7f4c440b2420 2023-09-23T03:23:51.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.553+0000 7f4c4b24f640 1 -- 172.21.15.154:0/914487278 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c440b36c0 con 0x7f4c440b2420 2023-09-23T03:23:51.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.553+0000 7f4c4b24f640 1 -- 172.21.15.154:0/914487278 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c440b3c80 con 0x7f4c440b2420 2023-09-23T03:23:51.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.554+0000 7f4c4b24f640 1 -- 172.21.15.154:0/914487278 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c44073eb0 con 0x7f4c440b2420 2023-09-23T03:23:51.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.555+0000 7f4c417fa640 1 -- 172.21.15.154:0/914487278 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f4c34040cc0 con 0x7f4c440b2420 2023-09-23T03:23:51.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.556+0000 7f4c417fa640 1 --2- 172.21.15.154:0/914487278 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4c2406af30 0x7f4c2406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:51.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.556+0000 7f4c417fa640 1 -- 172.21.15.154:0/914487278 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f4c340aca90 con 0x7f4c440b2420 2023-09-23T03:23:51.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.557+0000 7f4c43fff640 1 --2- 172.21.15.154:0/914487278 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4c2406af30 0x7f4c2406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:51.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.557+0000 7f4c43fff640 1 --2- 172.21.15.154:0/914487278 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4c2406af30 0x7f4c2406d3f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4c4415e1f0 tx=0x7f4c2c007930 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:51.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.561+0000 7f4c417fa640 1 -- 172.21.15.154:0/914487278 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f4c34077650 con 0x7f4c440b2420 2023-09-23T03:23:51.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.753+0000 7f4c4b24f640 1 -- 172.21.15.154:0/914487278 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c440b4d00 con 0x7f4c440b2420 2023-09-23T03:23:51.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.754+0000 7f4c417fa640 1 -- 172.21.15.154:0/914487278 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+18987 (secure 0 0 0) 0x7f4c3407ac70 con 0x7f4c440b2420 2023-09-23T03:23:51.755 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:23:51.755 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":71,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:21:39.274806+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:19:21.772037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-09-23T03:19:22.960250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data_extra","create_time":"2023-09-23T03:19:38.870843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6837/4129414148":"2023-09-24T03:21:38.278599+0000","172.21.15.154:6835/4129414148":"2023-09-24T03:21:38.278599+0000","172.21.15.154:6837/2976977273":"2023-09-24T03:21:31.069171+0000","172.21.15.154:6835/2976977273":"2023-09-24T03:21:31.069171+0000","172.21.15.154:6837/2183023203":"2023-09-24T03:21:29.032871+0000","172.21.15.154:6835/2183023203":"2023-09-24T03:21:29.032871+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:23:51.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.757+0000 7f4c4b24f640 1 -- 172.21.15.154:0/914487278 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4c2406af30 msgr2=0x7f4c2406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:51.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.757+0000 7f4c4b24f640 1 --2- 172.21.15.154:0/914487278 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4c2406af30 0x7f4c2406d3f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4c4415e1f0 tx=0x7f4c2c007930 comp rx=0 tx=0).stop 2023-09-23T03:23:51.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.757+0000 7f4c4b24f640 1 -- 172.21.15.154:0/914487278 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c440b2420 msgr2=0x7f4c441f68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:51.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.757+0000 7f4c4b24f640 1 --2- 172.21.15.154:0/914487278 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c440b2420 0x7f4c441f68d0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f4c34009e80 tx=0x7f4c34004a40 comp rx=0 tx=0).stop 2023-09-23T03:23:51.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.758+0000 7f4c4b24f640 1 -- 172.21.15.154:0/914487278 shutdown_connections 2023-09-23T03:23:51.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.758+0000 7f4c4b24f640 1 --2- 172.21.15.154:0/914487278 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4c2406af30 0x7f4c2406d3f0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:51.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.758+0000 7f4c4b24f640 1 --2- 172.21.15.154:0/914487278 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c440c6720 0x7f4c440c6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:51.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.758+0000 7f4c4b24f640 1 --2- 172.21.15.154:0/914487278 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c440b2800 0x7f4c440b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:51.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.758+0000 7f4c4b24f640 1 --2- 172.21.15.154:0/914487278 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c440b2420 0x7f4c441f68d0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:51.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.758+0000 7f4c4b24f640 1 -- 172.21.15.154:0/914487278 >> 172.21.15.154:0/914487278 conn(0x7f4c440cd1b0 msgr2=0x7f4c440b1930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:51.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.758+0000 7f4c4b24f640 1 -- 172.21.15.154:0/914487278 shutdown_connections 2023-09-23T03:23:51.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.758+0000 7f4c4b24f640 1 -- 172.21.15.154:0/914487278 wait complete. 2023-09-23T03:23:51.778 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:19:21.772037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:19:22.960250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-09-23T03:19:38.870843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T03:23:51.778 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:23:51.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.995+0000 7f35b4451640 1 Processor -- start 2023-09-23T03:23:51.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.996+0000 7f35b4451640 1 -- start start 2023-09-23T03:23:51.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.997+0000 7f35b4451640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f35ac0cc9b0 0x7f35ac0b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:51.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.997+0000 7f35b4451640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35ac0cce40 0x7f35ac0b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:51.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.997+0000 7f35b4451640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35ac0ce510 con 0x7f35ac15abe0 2023-09-23T03:23:51.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.997+0000 7f35b4451640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35ac0cd390 con 0x7f35ac0cc9b0 2023-09-23T03:23:51.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.997+0000 7f35b4451640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac0cd670 con 0x7f35ac0cce40 2023-09-23T03:23:52.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.998+0000 7f35b21ed640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f35ac15abe0 0x7f35ac0cb1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:50974/0 (socket says 172.21.15.154:50974) 2023-09-23T03:23:52.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.998+0000 7f35b21ed640 1 -- 172.21.15.154:0/4130935946 learned_addr learned my addr 172.21.15.154:0/4130935946 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:23:52.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.998+0000 7f35b29ee640 1 --2- 172.21.15.154:0/4130935946 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35ac0cce40 0x7f35ac0b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:52.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.998+0000 7f35b19ec640 1 --2- 172.21.15.154:0/4130935946 >> v2:172.21.15.190:3300/0 conn(0x7f35ac0cc9b0 0x7f35ac0b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:52.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.998+0000 7f35b11eb640 1 -- 172.21.15.154:0/4130935946 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4038009004 0 0) 0x7f35ac0ce510 con 0x7f35ac15abe0 2023-09-23T03:23:52.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.998+0000 7f35b11eb640 1 -- 172.21.15.154:0/4130935946 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3594003520 con 0x7f35ac15abe0 2023-09-23T03:23:52.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.999+0000 7f35b11eb640 1 -- 172.21.15.154:0/4130935946 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3613019023 0 0) 0x7f3594003520 con 0x7f35ac15abe0 2023-09-23T03:23:52.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.999+0000 7f35b11eb640 1 -- 172.21.15.154:0/4130935946 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35ac0cce40 msgr2=0x7f35ac0b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:52.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.999+0000 7f35b11eb640 1 --2- 172.21.15.154:0/4130935946 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35ac0cce40 0x7f35ac0b26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:52.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.999+0000 7f35b11eb640 1 -- 172.21.15.154:0/4130935946 >> v2:172.21.15.190:3300/0 conn(0x7f35ac0cc9b0 msgr2=0x7f35ac0b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:52.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.999+0000 7f35b11eb640 1 --2- 172.21.15.154:0/4130935946 >> v2:172.21.15.190:3300/0 conn(0x7f35ac0cc9b0 0x7f35ac0b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:52.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.999+0000 7f35b11eb640 1 -- 172.21.15.154:0/4130935946 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35ac0cd950 con 0x7f35ac15abe0 2023-09-23T03:23:52.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.999+0000 7f35b11eb640 1 -- 172.21.15.154:0/4130935946 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f359c001b70 con 0x7f35ac15abe0 2023-09-23T03:23:52.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.999+0000 7f35b11eb640 1 -- 172.21.15.154:0/4130935946 >> v1:172.21.15.154:6789/0 conn(0x7f35ac15abe0 legacy=0x7f35ac0cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:52.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.999+0000 7f35b11eb640 1 --2- 172.21.15.154:0/4130935946 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35ac0cce40 0x7f3594003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:52.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.999+0000 7f35b11eb640 1 --2- 172.21.15.154:0/4130935946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3594003ef0 0x7f35940047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:52.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.999+0000 7f35b11eb640 1 --2- 172.21.15.154:0/4130935946 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3594004d00 0x7f3594005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:52.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.999+0000 7f35b11eb640 1 -- 172.21.15.154:0/4130935946 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac0cd390 con 0x7f3594004d00 2023-09-23T03:23:52.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.999+0000 7f35b11eb640 1 -- 172.21.15.154:0/4130935946 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac0cd670 con 0x7f3594003ef0 2023-09-23T03:23:52.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.999+0000 7f35b11eb640 1 -- 172.21.15.154:0/4130935946 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f359400b9d0 con 0x7f35ac0cce40 2023-09-23T03:23:52.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.999+0000 7f35b29ee640 1 --2- 172.21.15.154:0/4130935946 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3594004d00 0x7f3594005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:52.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.999+0000 7f35b29ee640 1 -- 172.21.15.154:0/4130935946 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35ac0cce40 msgr2=0x7f3594003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:52.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:51.999+0000 7f35b29ee640 1 --2- 172.21.15.154:0/4130935946 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35ac0cce40 0x7f3594003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:52.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.000+0000 7f35b29ee640 1 -- 172.21.15.154:0/4130935946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3594003ef0 msgr2=0x7f35940047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:52.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.000+0000 7f35b29ee640 1 --2- 172.21.15.154:0/4130935946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3594003ef0 0x7f35940047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:52.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.000+0000 7f35b29ee640 1 -- 172.21.15.154:0/4130935946 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3594003520 con 0x7f3594004d00 2023-09-23T03:23:52.003 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.001+0000 7f35b29ee640 1 --2- 172.21.15.154:0/4130935946 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3594004d00 0x7f3594005180 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f35a4009af0 tx=0x7f35940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f48b1ccfe0ca14 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:52.003 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.001+0000 7f35b11eb640 1 -- 172.21.15.154:0/4130935946 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a402c710 con 0x7f3594004d00 2023-09-23T03:23:52.003 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.001+0000 7f35b11eb640 1 -- 172.21.15.154:0/4130935946 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a402c870 con 0x7f3594004d00 2023-09-23T03:23:52.003 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.001+0000 7f35b11eb640 1 -- 172.21.15.154:0/4130935946 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a402cc00 con 0x7f3594004d00 2023-09-23T03:23:52.003 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.001+0000 7f35b4451640 1 -- 172.21.15.154:0/4130935946 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3594004d00 msgr2=0x7f3594005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:52.003 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.001+0000 7f35b4451640 1 --2- 172.21.15.154:0/4130935946 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3594004d00 0x7f3594005180 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f35a4009af0 tx=0x7f35940057f0 comp rx=0 tx=0).stop 2023-09-23T03:23:52.003 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.001+0000 7f35b29ee640 1 -- 172.21.15.154:0/4130935946 reap_dead start 2023-09-23T03:23:52.003 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.002+0000 7f35b4451640 1 -- 172.21.15.154:0/4130935946 shutdown_connections 2023-09-23T03:23:52.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.002+0000 7f35b4451640 1 -- 172.21.15.154:0/4130935946 >> 172.21.15.154:0/4130935946 conn(0x7f35ac06fc40 msgr2=0x7f35ac070020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:52.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.002+0000 7f35b4451640 1 -- 172.21.15.154:0/4130935946 shutdown_connections 2023-09-23T03:23:52.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.002+0000 7f35b4451640 1 -- 172.21.15.154:0/4130935946 wait complete. 2023-09-23T03:23:52.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.003+0000 7f35b4451640 1 Processor -- start 2023-09-23T03:23:52.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.004+0000 7f35b4451640 1 -- start start 2023-09-23T03:23:52.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.005+0000 7f35b4451640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3594004d00 0x7f35ac155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:52.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.005+0000 7f35b4451640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35ac0ca4b0 0x7f35ac0c8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:52.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.005+0000 7f35b4451640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35ac15c200 0x7f35ac1f5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:52.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.005+0000 7f35b4451640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac1560f0 con 0x7f3594004d00 2023-09-23T03:23:52.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.005+0000 7f35b4451640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac15b160 con 0x7f35ac0ca4b0 2023-09-23T03:23:52.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.005+0000 7f35b4451640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac15b440 con 0x7f35ac15c200 2023-09-23T03:23:52.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.005+0000 7f35b21ed640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3594004d00 0x7f35ac155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:52.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.005+0000 7f35b21ed640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3594004d00 0x7f35ac155bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60242/0 (socket says 172.21.15.154:60242) 2023-09-23T03:23:52.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.005+0000 7f35b21ed640 1 -- 172.21.15.154:0/333154454 learned_addr learned my addr 172.21.15.154:0/333154454 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:23:52.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.005+0000 7f35b19ec640 1 --2- 172.21.15.154:0/333154454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35ac0ca4b0 0x7f35ac0c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:52.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.005+0000 7f35b29ee640 1 --2- 172.21.15.154:0/333154454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35ac15c200 0x7f35ac1f5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:52.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.005+0000 7f35b21ed640 1 -- 172.21.15.154:0/333154454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35ac15c200 msgr2=0x7f35ac1f5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:52.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.005+0000 7f35b21ed640 1 --2- 172.21.15.154:0/333154454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35ac15c200 0x7f35ac1f5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:52.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.006+0000 7f35b21ed640 1 -- 172.21.15.154:0/333154454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35ac0ca4b0 msgr2=0x7f35ac0c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:52.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.006+0000 7f35b21ed640 1 --2- 172.21.15.154:0/333154454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35ac0ca4b0 0x7f35ac0c8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:52.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.006+0000 7f35b21ed640 1 -- 172.21.15.154:0/333154454 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35ac15b720 con 0x7f3594004d00 2023-09-23T03:23:52.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.006+0000 7f35b21ed640 1 --2- 172.21.15.154:0/333154454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3594004d00 0x7f35ac155bb0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f359c00ca70 tx=0x7f359c00cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:52.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.006+0000 7f35aaffd640 1 -- 172.21.15.154:0/333154454 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c00a760 con 0x7f3594004d00 2023-09-23T03:23:52.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.006+0000 7f35aaffd640 1 -- 172.21.15.154:0/333154454 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f359c00a8c0 con 0x7f3594004d00 2023-09-23T03:23:52.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.006+0000 7f35aaffd640 1 -- 172.21.15.154:0/333154454 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359c00acd0 con 0x7f3594004d00 2023-09-23T03:23:52.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.006+0000 7f35b4451640 1 -- 172.21.15.154:0/333154454 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35ac15ba00 con 0x7f3594004d00 2023-09-23T03:23:52.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.006+0000 7f35b4451640 1 -- 172.21.15.154:0/333154454 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35ac0c8c50 con 0x7f3594004d00 2023-09-23T03:23:52.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.007+0000 7f35b4451640 1 -- 172.21.15.154:0/333154454 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35ac15bf20 con 0x7f3594004d00 2023-09-23T03:23:52.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.008+0000 7f35aaffd640 1 -- 172.21.15.154:0/333154454 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f359c007070 con 0x7f3594004d00 2023-09-23T03:23:52.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.010+0000 7f35aaffd640 1 --2- 172.21.15.154:0/333154454 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f359806af30 0x7f359806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:52.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.010+0000 7f35aaffd640 1 -- 172.21.15.154:0/333154454 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f359c089730 con 0x7f3594004d00 2023-09-23T03:23:52.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.010+0000 7f35b19ec640 1 --2- 172.21.15.154:0/333154454 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f359806af30 0x7f359806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:52.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.011+0000 7f35b19ec640 1 --2- 172.21.15.154:0/333154454 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f359806af30 0x7f359806d3f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f35a0004770 tx=0x7f35a0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:52.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.014+0000 7f35aaffd640 1 -- 172.21.15.154:0/333154454 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f35ac15bf20 con 0x7f3594004d00 2023-09-23T03:23:52.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.195+0000 7f35b4451640 1 -- 172.21.15.154:0/333154454 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f35ac15bf20 con 0x7f3594004d00 2023-09-23T03:23:52.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.195+0000 7f35aaffd640 1 -- 172.21.15.154:0/333154454 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) v1 ==== 93+0+10 (secure 0 0 0) 0x7f359c017020 con 0x7f3594004d00 2023-09-23T03:23:52.196 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:23:52.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.198+0000 7f35b4451640 1 -- 172.21.15.154:0/333154454 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f359806af30 msgr2=0x7f359806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:52.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.198+0000 7f35b4451640 1 --2- 172.21.15.154:0/333154454 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f359806af30 0x7f359806d3f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f35a0004770 tx=0x7f35a0008040 comp rx=0 tx=0).stop 2023-09-23T03:23:52.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.198+0000 7f35b4451640 1 -- 172.21.15.154:0/333154454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3594004d00 msgr2=0x7f35ac155bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:52.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.198+0000 7f35b4451640 1 --2- 172.21.15.154:0/333154454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3594004d00 0x7f35ac155bb0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f359c00ca70 tx=0x7f359c00cf30 comp rx=0 tx=0).stop 2023-09-23T03:23:52.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.198+0000 7f35b4451640 1 -- 172.21.15.154:0/333154454 shutdown_connections 2023-09-23T03:23:52.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.198+0000 7f35b4451640 1 --2- 172.21.15.154:0/333154454 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f359806af30 0x7f359806d3f0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:52.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.198+0000 7f35b4451640 1 --2- 172.21.15.154:0/333154454 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35ac15c200 0x7f35ac1f5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:52.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.198+0000 7f35b4451640 1 --2- 172.21.15.154:0/333154454 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35ac0ca4b0 0x7f35ac0c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:52.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.198+0000 7f35b4451640 1 --2- 172.21.15.154:0/333154454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3594004d00 0x7f35ac155bb0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:52.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.198+0000 7f35b4451640 1 -- 172.21.15.154:0/333154454 >> 172.21.15.154:0/333154454 conn(0x7f35ac06fc40 msgr2=0x7f35ac1588b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:52.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.198+0000 7f35b4451640 1 -- 172.21.15.154:0/333154454 shutdown_connections 2023-09-23T03:23:52.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.198+0000 7f35b4451640 1 -- 172.21.15.154:0/333154454 wait complete. 2023-09-23T03:23:52.216 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:23:52.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.430+0000 7ff6ec88f640 1 Processor -- start 2023-09-23T03:23:52.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.430+0000 7ff6ec88f640 1 -- start start 2023-09-23T03:23:52.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.432+0000 7ff6ec88f640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff6e815ba50 0x7ff6e806fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:52.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.432+0000 7ff6ec88f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6e815eef0 0x7ff6e80703d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:52.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.432+0000 7ff6ec88f640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e80cb6c0 con 0x7ff6e8158480 2023-09-23T03:23:52.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.432+0000 7ff6ec88f640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e815ad80 con 0x7ff6e815ba50 2023-09-23T03:23:52.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.432+0000 7ff6ec88f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e815af70 con 0x7ff6e815eef0 2023-09-23T03:23:52.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.432+0000 7ff6dffff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff6e815ba50 0x7ff6e806fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:52.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.432+0000 7ff6dffff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff6e815ba50 0x7ff6e806fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.154:47704/0 (socket says 172.21.15.154:47704) 2023-09-23T03:23:52.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.432+0000 7ff6dffff640 1 -- 172.21.15.154:0/245975622 learned_addr learned my addr 172.21.15.154:0/245975622 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:23:52.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.432+0000 7ff6e759e640 1 --2- 172.21.15.154:0/245975622 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6e815eef0 0x7ff6e80703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:52.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.433+0000 7ff6e759e640 1 -- 172.21.15.154:0/245975622 >> v2:172.21.15.190:3300/0 conn(0x7ff6e815ba50 msgr2=0x7ff6e806fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:52.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.433+0000 7ff6e759e640 1 --2- 172.21.15.154:0/245975622 >> v2:172.21.15.190:3300/0 conn(0x7ff6e815ba50 0x7ff6e806fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:52.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.433+0000 7ff6e759e640 1 -- 172.21.15.154:0/245975622 >> v1:172.21.15.190:6789/0 conn(0x7ff6e8158480 legacy=0x7ff6e80cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:52.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.433+0000 7ff6e759e640 1 -- 172.21.15.154:0/245975622 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e815b250 con 0x7ff6e815eef0 2023-09-23T03:23:52.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.435+0000 7ff6e759e640 1 --2- 172.21.15.154:0/245975622 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6e815eef0 0x7ff6e80703d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0009d90 tx=0x7ff6e002bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fadf6cd4d963ab4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:52.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.438+0000 7ff6e659c640 1 -- 172.21.15.154:0/245975622 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e002cc50 con 0x7ff6e815eef0 2023-09-23T03:23:52.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.438+0000 7ff6e659c640 1 -- 172.21.15.154:0/245975622 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e002cdb0 con 0x7ff6e815eef0 2023-09-23T03:23:52.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.438+0000 7ff6e659c640 1 -- 172.21.15.154:0/245975622 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e0034660 con 0x7ff6e815eef0 2023-09-23T03:23:52.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.439+0000 7ff6ec88f640 1 -- 172.21.15.154:0/245975622 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6e815eef0 msgr2=0x7ff6e80703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:52.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.439+0000 7ff6ec88f640 1 --2- 172.21.15.154:0/245975622 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6e815eef0 0x7ff6e80703d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0009d90 tx=0x7ff6e002bea0 comp rx=0 tx=0).stop 2023-09-23T03:23:52.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.439+0000 7ff6ec88f640 1 -- 172.21.15.154:0/245975622 shutdown_connections 2023-09-23T03:23:52.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.439+0000 7ff6ec88f640 1 --2- 172.21.15.154:0/245975622 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6e815eef0 0x7ff6e80703d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:52.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.439+0000 7ff6ec88f640 1 --2- 172.21.15.154:0/245975622 >> v2:172.21.15.190:3300/0 conn(0x7ff6e815ba50 0x7ff6e806fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:52.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.439+0000 7ff6ec88f640 1 -- 172.21.15.154:0/245975622 >> 172.21.15.154:0/245975622 conn(0x7ff6e80aff20 msgr2=0x7ff6e80b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:52.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.439+0000 7ff6ec88f640 1 -- 172.21.15.154:0/245975622 shutdown_connections 2023-09-23T03:23:52.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.439+0000 7ff6ec88f640 1 -- 172.21.15.154:0/245975622 wait complete. 2023-09-23T03:23:52.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.441+0000 7ff6ec88f640 1 Processor -- start 2023-09-23T03:23:52.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.441+0000 7ff6ec88f640 1 -- start start 2023-09-23T03:23:52.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.442+0000 7ff6ec88f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6e8158480 0x7ff6e81f2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:52.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.442+0000 7ff6ec88f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6e815ba50 0x7ff6e81f37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:52.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.442+0000 7ff6ec88f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6e815eef0 0x7ff6e81567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:52.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.442+0000 7ff6ec88f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e8156360 con 0x7ff6e815ba50 2023-09-23T03:23:52.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.442+0000 7ff6ec88f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e81f3290 con 0x7ff6e815eef0 2023-09-23T03:23:52.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.442+0000 7ff6ec88f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e815d790 con 0x7ff6e8158480 2023-09-23T03:23:52.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.442+0000 7ff6dffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6e815ba50 0x7ff6e81f37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:52.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.442+0000 7ff6dffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6e815ba50 0x7ff6e81f37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60248/0 (socket says 172.21.15.154:60248) 2023-09-23T03:23:52.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.442+0000 7ff6dffff640 1 -- 172.21.15.154:0/2740205509 learned_addr learned my addr 172.21.15.154:0/2740205509 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:23:52.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.442+0000 7ff6e759e640 1 --2- 172.21.15.154:0/2740205509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6e815eef0 0x7ff6e81567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:52.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.442+0000 7ff6dffff640 1 -- 172.21.15.154:0/2740205509 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6e8158480 msgr2=0x7ff6e81f2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:52.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.442+0000 7ff6dffff640 1 --2- 172.21.15.154:0/2740205509 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6e8158480 0x7ff6e81f2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:52.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.442+0000 7ff6dffff640 1 -- 172.21.15.154:0/2740205509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6e815eef0 msgr2=0x7ff6e81567f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:52.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.442+0000 7ff6dffff640 1 --2- 172.21.15.154:0/2740205509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6e815eef0 0x7ff6e81567f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:52.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.442+0000 7ff6dffff640 1 -- 172.21.15.154:0/2740205509 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e0009a70 con 0x7ff6e815ba50 2023-09-23T03:23:52.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.443+0000 7ff6dffff640 1 --2- 172.21.15.154:0/2740205509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6e815ba50 0x7ff6e81f37a0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0009fd0 tx=0x7ff6d000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:52.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.443+0000 7ff6df7fe640 1 -- 172.21.15.154:0/2740205509 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d000c940 con 0x7ff6e815ba50 2023-09-23T03:23:52.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.443+0000 7ff6df7fe640 1 -- 172.21.15.154:0/2740205509 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d000caa0 con 0x7ff6e815ba50 2023-09-23T03:23:52.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.443+0000 7ff6ec88f640 1 -- 172.21.15.154:0/2740205509 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6e8155d10 con 0x7ff6e815ba50 2023-09-23T03:23:52.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.443+0000 7ff6df7fe640 1 -- 172.21.15.154:0/2740205509 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d000ce30 con 0x7ff6e815ba50 2023-09-23T03:23:52.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.443+0000 7ff6ec88f640 1 -- 172.21.15.154:0/2740205509 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e8155920 con 0x7ff6e815ba50 2023-09-23T03:23:52.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.445+0000 7ff6ec88f640 1 -- 172.21.15.154:0/2740205509 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6e80c5380 con 0x7ff6e815ba50 2023-09-23T03:23:52.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.445+0000 7ff6df7fe640 1 -- 172.21.15.154:0/2740205509 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7ff6d0013470 con 0x7ff6e815ba50 2023-09-23T03:23:52.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.447+0000 7ff6df7fe640 1 --2- 172.21.15.154:0/2740205509 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff6c806af30 0x7ff6c806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:52.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.447+0000 7ff6df7fe640 1 -- 172.21.15.154:0/2740205509 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6691+0+0 (secure 0 0 0) 0x7ff6d0089dd0 con 0x7ff6e815ba50 2023-09-23T03:23:52.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.447+0000 7ff6e6d9d640 1 --2- 172.21.15.154:0/2740205509 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff6c806af30 0x7ff6c806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:52.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.448+0000 7ff6e6d9d640 1 --2- 172.21.15.154:0/2740205509 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff6c806af30 0x7ff6c806d3f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff6e806f9f0 tx=0x7ff6d8007950 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:52.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.451+0000 7ff6df7fe640 1 -- 172.21.15.154:0/2740205509 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7ff6d0054900 con 0x7ff6e815ba50 2023-09-23T03:23:52.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.641+0000 7ff6ec88f640 1 -- 172.21.15.154:0/2740205509 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff6e804e920 con 0x7ff6e815ba50 2023-09-23T03:23:52.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.642+0000 7ff6df7fe640 1 -- 172.21.15.154:0/2740205509 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff6d0017020 con 0x7ff6e815ba50 2023-09-23T03:23:52.642 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:23:52.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.644+0000 7ff6ec88f640 1 -- 172.21.15.154:0/2740205509 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff6c806af30 msgr2=0x7ff6c806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:52.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.644+0000 7ff6ec88f640 1 --2- 172.21.15.154:0/2740205509 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff6c806af30 0x7ff6c806d3f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff6e806f9f0 tx=0x7ff6d8007950 comp rx=0 tx=0).stop 2023-09-23T03:23:52.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.644+0000 7ff6ec88f640 1 -- 172.21.15.154:0/2740205509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6e815ba50 msgr2=0x7ff6e81f37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:52.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.644+0000 7ff6ec88f640 1 --2- 172.21.15.154:0/2740205509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6e815ba50 0x7ff6e81f37a0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0009fd0 tx=0x7ff6d000ed40 comp rx=0 tx=0).stop 2023-09-23T03:23:52.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.644+0000 7ff6ec88f640 1 -- 172.21.15.154:0/2740205509 shutdown_connections 2023-09-23T03:23:52.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.644+0000 7ff6ec88f640 1 --2- 172.21.15.154:0/2740205509 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff6c806af30 0x7ff6c806d3f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:52.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.644+0000 7ff6ec88f640 1 --2- 172.21.15.154:0/2740205509 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6e815eef0 0x7ff6e81567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:52.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.644+0000 7ff6ec88f640 1 --2- 172.21.15.154:0/2740205509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6e815ba50 0x7ff6e81f37a0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:52.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.644+0000 7ff6ec88f640 1 --2- 172.21.15.154:0/2740205509 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6e8158480 0x7ff6e81f2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:52.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.644+0000 7ff6ec88f640 1 -- 172.21.15.154:0/2740205509 >> 172.21.15.154:0/2740205509 conn(0x7ff6e80aff20 msgr2=0x7ff6e806fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:52.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.644+0000 7ff6ec88f640 1 -- 172.21.15.154:0/2740205509 shutdown_connections 2023-09-23T03:23:52.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.644+0000 7ff6ec88f640 1 -- 172.21.15.154:0/2740205509 wait complete. 2023-09-23T03:23:52.661 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:23:52.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.874+0000 7fd93e258640 1 Processor -- start 2023-09-23T03:23:52.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.874+0000 7fd93e258640 1 -- start start 2023-09-23T03:23:52.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.876+0000 7fd93e258640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd93815abe0 0x7fd9380cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:52.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.876+0000 7fd93e258640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd9380b28c0 0x7fd9380ce9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:52.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.876+0000 7fd93e258640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9380ce510 con 0x7fd9380cc9b0 2023-09-23T03:23:52.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.876+0000 7fd93e258640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9380cb710 con 0x7fd93815abe0 2023-09-23T03:23:52.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.876+0000 7fd93e258640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9380cd390 con 0x7fd9380b28c0 2023-09-23T03:23:52.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.877+0000 7fd9377fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd93815abe0 0x7fd9380cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:52.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.877+0000 7fd9377fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd93815abe0 0x7fd9380cb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60256/0 (socket says 172.21.15.154:60256) 2023-09-23T03:23:52.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.877+0000 7fd9377fe640 1 -- 172.21.15.154:0/674347695 learned_addr learned my addr 172.21.15.154:0/674347695 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:23:52.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.877+0000 7fd937fff640 1 --2- 172.21.15.154:0/674347695 >> v2:172.21.15.190:3300/0 conn(0x7fd9380b28c0 0x7fd9380ce9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:52.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.877+0000 7fd9377fe640 1 -- 172.21.15.154:0/674347695 >> v2:172.21.15.190:3300/0 conn(0x7fd9380b28c0 msgr2=0x7fd9380ce9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:52.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.877+0000 7fd9377fe640 1 --2- 172.21.15.154:0/674347695 >> v2:172.21.15.190:3300/0 conn(0x7fd9380b28c0 0x7fd9380ce9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:52.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.877+0000 7fd9377fe640 1 -- 172.21.15.154:0/674347695 >> v1:172.21.15.190:6789/0 conn(0x7fd9380cc9b0 legacy=0x7fd9380b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:52.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.877+0000 7fd9377fe640 1 -- 172.21.15.154:0/674347695 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9380cd670 con 0x7fd93815abe0 2023-09-23T03:23:52.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.879+0000 7fd9377fe640 1 --2- 172.21.15.154:0/674347695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd93815abe0 0x7fd9380cb1d0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fd9380676f0 tx=0x7fd92802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddfd3620bde9036 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:52.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.879+0000 7fd936ffd640 1 -- 172.21.15.154:0/674347695 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9280317e0 con 0x7fd93815abe0 2023-09-23T03:23:52.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.879+0000 7fd936ffd640 1 -- 172.21.15.154:0/674347695 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd928031940 con 0x7fd93815abe0 2023-09-23T03:23:52.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.879+0000 7fd936ffd640 1 -- 172.21.15.154:0/674347695 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd928031cd0 con 0x7fd93815abe0 2023-09-23T03:23:52.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.879+0000 7fd93e258640 1 -- 172.21.15.154:0/674347695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd93815abe0 msgr2=0x7fd9380cb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:52.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.879+0000 7fd93e258640 1 --2- 172.21.15.154:0/674347695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd93815abe0 0x7fd9380cb1d0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fd9380676f0 tx=0x7fd92802b880 comp rx=0 tx=0).stop 2023-09-23T03:23:52.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.880+0000 7fd93e258640 1 -- 172.21.15.154:0/674347695 shutdown_connections 2023-09-23T03:23:52.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.880+0000 7fd93e258640 1 --2- 172.21.15.154:0/674347695 >> v2:172.21.15.190:3300/0 conn(0x7fd9380b28c0 0x7fd9380ce9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:52.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.880+0000 7fd93e258640 1 --2- 172.21.15.154:0/674347695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd93815abe0 0x7fd9380cb1d0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:52.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.880+0000 7fd93e258640 1 -- 172.21.15.154:0/674347695 >> 172.21.15.154:0/674347695 conn(0x7fd93806fc40 msgr2=0x7fd938070020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:52.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.880+0000 7fd93e258640 1 -- 172.21.15.154:0/674347695 shutdown_connections 2023-09-23T03:23:52.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.880+0000 7fd93e258640 1 -- 172.21.15.154:0/674347695 wait complete. 2023-09-23T03:23:52.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.882+0000 7fd93e258640 1 Processor -- start 2023-09-23T03:23:52.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.882+0000 7fd93e258640 1 -- start start 2023-09-23T03:23:52.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.883+0000 7fd93e258640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9380b28c0 0x7fd9381f2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:52.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.883+0000 7fd93e258640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9380cc9b0 0x7fd9380b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:52.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.883+0000 7fd93e258640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd93815abe0 0x7fd93814a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:52.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.883+0000 7fd93e258640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9380b5be0 con 0x7fd9380cc9b0 2023-09-23T03:23:52.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.883+0000 7fd93e258640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93814a8f0 con 0x7fd93815abe0 2023-09-23T03:23:52.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.883+0000 7fd93e258640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9381f31a0 con 0x7fd9380b28c0 2023-09-23T03:23:52.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.883+0000 7fd92ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9380cc9b0 0x7fd9380b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:52.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.883+0000 7fd92ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9380cc9b0 0x7fd9380b56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60262/0 (socket says 172.21.15.154:60262) 2023-09-23T03:23:52.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.883+0000 7fd92ffff640 1 -- 172.21.15.154:0/3428130535 learned_addr learned my addr 172.21.15.154:0/3428130535 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:23:52.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.884+0000 7fd92ffff640 1 -- 172.21.15.154:0/3428130535 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9380b28c0 msgr2=0x7fd9381f2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:52.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.884+0000 7fd937fff640 1 --2- 172.21.15.154:0/3428130535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd93815abe0 0x7fd93814a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:52.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.884+0000 7fd92ffff640 1 --2- 172.21.15.154:0/3428130535 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9380b28c0 0x7fd9381f2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:52.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.884+0000 7fd92ffff640 1 -- 172.21.15.154:0/3428130535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd93815abe0 msgr2=0x7fd93814a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:52.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.884+0000 7fd92ffff640 1 --2- 172.21.15.154:0/3428130535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd93815abe0 0x7fd93814a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:52.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.884+0000 7fd92ffff640 1 -- 172.21.15.154:0/3428130535 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd928009470 con 0x7fd9380cc9b0 2023-09-23T03:23:52.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.884+0000 7fd92ffff640 1 --2- 172.21.15.154:0/3428130535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9380cc9b0 0x7fd9380b56a0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fd92000b3f0 tx=0x7fd92000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:52.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.884+0000 7fd934ff9640 1 -- 172.21.15.154:0/3428130535 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd920011830 con 0x7fd9380cc9b0 2023-09-23T03:23:52.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.884+0000 7fd934ff9640 1 -- 172.21.15.154:0/3428130535 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd920011990 con 0x7fd9380cc9b0 2023-09-23T03:23:52.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.884+0000 7fd934ff9640 1 -- 172.21.15.154:0/3428130535 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd920011d40 con 0x7fd9380cc9b0 2023-09-23T03:23:52.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.884+0000 7fd93e258640 1 -- 172.21.15.154:0/3428130535 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9381f3480 con 0x7fd9380cc9b0 2023-09-23T03:23:52.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.885+0000 7fd93e258640 1 -- 172.21.15.154:0/3428130535 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9381f3a40 con 0x7fd9380cc9b0 2023-09-23T03:23:52.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.886+0000 7fd93e258640 1 -- 172.21.15.154:0/3428130535 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9380caaf0 con 0x7fd9380cc9b0 2023-09-23T03:23:52.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.887+0000 7fd934ff9640 1 -- 172.21.15.154:0/3428130535 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fd92001a400 con 0x7fd9380cc9b0 2023-09-23T03:23:52.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.887+0000 7fd934ff9640 1 --2- 172.21.15.154:0/3428130535 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd91806af30 0x7fd91806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:52.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.888+0000 7fd934ff9640 1 -- 172.21.15.154:0/3428130535 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fd9200899c0 con 0x7fd9380cc9b0 2023-09-23T03:23:52.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.888+0000 7fd9377fe640 1 --2- 172.21.15.154:0/3428130535 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd91806af30 0x7fd91806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:52.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.888+0000 7fd9377fe640 1 --2- 172.21.15.154:0/3428130535 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd91806af30 0x7fd91806d3f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd928036000 tx=0x7fd928030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:52.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:52.892+0000 7fd934ff9640 1 -- 172.21.15.154:0/3428130535 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fd920054610 con 0x7fd9380cc9b0 2023-09-23T03:23:53.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.082+0000 7fd93e258640 1 -- 172.21.15.154:0/3428130535 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd93804e920 con 0x7fd9380cc9b0 2023-09-23T03:23:53.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.083+0000 7fd934ff9640 1 -- 172.21.15.154:0/3428130535 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd920057c30 con 0x7fd9380cc9b0 2023-09-23T03:23:53.083 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:23:53.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.085+0000 7fd93e258640 1 -- 172.21.15.154:0/3428130535 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd91806af30 msgr2=0x7fd91806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:53.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.086+0000 7fd93e258640 1 --2- 172.21.15.154:0/3428130535 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd91806af30 0x7fd91806d3f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd928036000 tx=0x7fd928030000 comp rx=0 tx=0).stop 2023-09-23T03:23:53.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.086+0000 7fd93e258640 1 -- 172.21.15.154:0/3428130535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9380cc9b0 msgr2=0x7fd9380b56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:53.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.086+0000 7fd93e258640 1 --2- 172.21.15.154:0/3428130535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9380cc9b0 0x7fd9380b56a0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fd92000b3f0 tx=0x7fd92000b8b0 comp rx=0 tx=0).stop 2023-09-23T03:23:53.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.086+0000 7fd93e258640 1 -- 172.21.15.154:0/3428130535 shutdown_connections 2023-09-23T03:23:53.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.086+0000 7fd93e258640 1 --2- 172.21.15.154:0/3428130535 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd91806af30 0x7fd91806d3f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:53.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.086+0000 7fd93e258640 1 --2- 172.21.15.154:0/3428130535 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd93815abe0 0x7fd93814a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:53.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.086+0000 7fd93e258640 1 --2- 172.21.15.154:0/3428130535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9380cc9b0 0x7fd9380b56a0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:53.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.086+0000 7fd93e258640 1 --2- 172.21.15.154:0/3428130535 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9380b28c0 0x7fd9381f2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:53.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.086+0000 7fd93e258640 1 -- 172.21.15.154:0/3428130535 >> 172.21.15.154:0/3428130535 conn(0x7fd93806fc40 msgr2=0x7fd9380ce9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:53.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.086+0000 7fd93e258640 1 -- 172.21.15.154:0/3428130535 shutdown_connections 2023-09-23T03:23:53.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.086+0000 7fd93e258640 1 -- 172.21.15.154:0/3428130535 wait complete. 2023-09-23T03:23:53.105 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-09-23T03:23:53.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.324+0000 7ff88b789640 1 Processor -- start 2023-09-23T03:23:53.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.325+0000 7ff88b789640 1 -- start start 2023-09-23T03:23:53.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.326+0000 7ff88b789640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff8840ce940 0x7ff8841677f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:53.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.326+0000 7ff88b789640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8840d0bb0 con 0x7ff88415cda0 2023-09-23T03:23:53.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.326+0000 7ff88b789640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8840cd450 con 0x7ff88415c9c0 2023-09-23T03:23:53.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.326+0000 7ff88b789640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8840d0df0 con 0x7ff8840ce940 2023-09-23T03:23:53.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.326+0000 7ff888d24640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff88415cda0 0x7ff8840d2620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:50978/0 (socket says 172.21.15.154:50978) 2023-09-23T03:23:53.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.326+0000 7ff888d24640 1 -- 172.21.15.154:0/2450398309 learned_addr learned my addr 172.21.15.154:0/2450398309 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:23:53.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.327+0000 7ff889d26640 1 --2- 172.21.15.154:0/2450398309 >> v2:172.21.15.190:3300/0 conn(0x7ff8840ce940 0x7ff8841677f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:53.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.327+0000 7ff883fff640 1 -- 172.21.15.154:0/2450398309 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 95633501 0 0) 0x7ff878001110 con 0x7ff88415cda0 2023-09-23T03:23:53.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.327+0000 7ff883fff640 1 -- 172.21.15.154:0/2450398309 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff86c003520 con 0x7ff88415cda0 2023-09-23T03:23:53.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.327+0000 7ff883fff640 1 -- 172.21.15.154:0/2450398309 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1337109149 0 0) 0x7ff86c003520 con 0x7ff88415cda0 2023-09-23T03:23:53.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.327+0000 7ff883fff640 1 -- 172.21.15.154:0/2450398309 >> v2:172.21.15.190:3300/0 conn(0x7ff8840ce940 msgr2=0x7ff8841677f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:53.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.327+0000 7ff883fff640 1 --2- 172.21.15.154:0/2450398309 >> v2:172.21.15.190:3300/0 conn(0x7ff8840ce940 0x7ff8841677f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:53.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.327+0000 7ff883fff640 1 -- 172.21.15.154:0/2450398309 >> v1:172.21.15.190:6789/0 conn(0x7ff88415c9c0 legacy=0x7ff8840d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:53.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.327+0000 7ff883fff640 1 -- 172.21.15.154:0/2450398309 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8840d10d0 con 0x7ff88415cda0 2023-09-23T03:23:53.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.327+0000 7ff883fff640 1 -- 172.21.15.154:0/2450398309 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7ff878001800 con 0x7ff88415cda0 2023-09-23T03:23:53.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.327+0000 7ff889d26640 1 --2- 172.21.15.154:0/2450398309 >> v2:172.21.15.190:3300/0 conn(0x7ff8840ce940 0x7ff8841677f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:23:53.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.327+0000 7ff883fff640 1 -- 172.21.15.154:0/2450398309 >> v1:172.21.15.154:6789/0 conn(0x7ff88415cda0 legacy=0x7ff8840d2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:53.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.327+0000 7ff883fff640 1 --2- 172.21.15.154:0/2450398309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff86c0039c0 0x7ff86c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:53.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.327+0000 7ff883fff640 1 --2- 172.21.15.154:0/2450398309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff86c0047c0 0x7ff86c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:53.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.327+0000 7ff883fff640 1 --2- 172.21.15.154:0/2450398309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff86c005250 0x7ff86c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:53.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.327+0000 7ff883fff640 1 -- 172.21.15.154:0/2450398309 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8840d0df0 con 0x7ff86c005250 2023-09-23T03:23:53.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.327+0000 7ff883fff640 1 -- 172.21.15.154:0/2450398309 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff86c00c1a0 con 0x7ff86c0039c0 2023-09-23T03:23:53.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.327+0000 7ff883fff640 1 -- 172.21.15.154:0/2450398309 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff86c00c360 con 0x7ff86c0047c0 2023-09-23T03:23:53.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.328+0000 7ff889d26640 1 --2- 172.21.15.154:0/2450398309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff86c005250 0x7ff86c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:53.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.328+0000 7ff889525640 1 --2- 172.21.15.154:0/2450398309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff86c0039c0 0x7ff86c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:53.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.328+0000 7ff889d26640 1 -- 172.21.15.154:0/2450398309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff86c0047c0 msgr2=0x7ff86c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:53.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.328+0000 7ff889d26640 1 --2- 172.21.15.154:0/2450398309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff86c0047c0 0x7ff86c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:53.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.328+0000 7ff889d26640 1 -- 172.21.15.154:0/2450398309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff86c0039c0 msgr2=0x7ff86c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:53.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.328+0000 7ff889d26640 1 --2- 172.21.15.154:0/2450398309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff86c0039c0 0x7ff86c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:53.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.328+0000 7ff889d26640 1 -- 172.21.15.154:0/2450398309 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff86c003520 con 0x7ff86c005250 2023-09-23T03:23:53.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.328+0000 7ff889525640 1 --2- 172.21.15.154:0/2450398309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff86c0039c0 0x7ff86c003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:23:53.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.329+0000 7ff889d26640 1 -- 172.21.15.154:0/2450398309 reap_dead start 2023-09-23T03:23:53.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.329+0000 7ff889d26640 1 --2- 172.21.15.154:0/2450398309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff86c005250 0x7ff86c00b980 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0098a0 tx=0x7ff86c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b39bb2b697a4b3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:53.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.330+0000 7ff883fff640 1 -- 172.21.15.154:0/2450398309 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c004220 con 0x7ff86c005250 2023-09-23T03:23:53.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.330+0000 7ff883fff640 1 -- 172.21.15.154:0/2450398309 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87c0043c0 con 0x7ff86c005250 2023-09-23T03:23:53.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.330+0000 7ff883fff640 1 -- 172.21.15.154:0/2450398309 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c003850 con 0x7ff86c005250 2023-09-23T03:23:53.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.330+0000 7ff88b789640 1 -- 172.21.15.154:0/2450398309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff86c005250 msgr2=0x7ff86c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:53.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.330+0000 7ff88b789640 1 --2- 172.21.15.154:0/2450398309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff86c005250 0x7ff86c00b980 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0098a0 tx=0x7ff86c0057f0 comp rx=0 tx=0).stop 2023-09-23T03:23:53.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.330+0000 7ff88b789640 1 -- 172.21.15.154:0/2450398309 shutdown_connections 2023-09-23T03:23:53.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.330+0000 7ff88b789640 1 --2- 172.21.15.154:0/2450398309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff86c005250 0x7ff86c00b980 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:53.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.330+0000 7ff88b789640 1 -- 172.21.15.154:0/2450398309 >> 172.21.15.154:0/2450398309 conn(0x7ff88409bc20 msgr2=0x7ff884089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:53.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.330+0000 7ff88b789640 1 -- 172.21.15.154:0/2450398309 shutdown_connections 2023-09-23T03:23:53.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.330+0000 7ff88b789640 1 -- 172.21.15.154:0/2450398309 wait complete. 2023-09-23T03:23:53.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.332+0000 7ff88b789640 1 Processor -- start 2023-09-23T03:23:53.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.333+0000 7ff88b789640 1 -- start start 2023-09-23T03:23:53.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.333+0000 7ff88b789640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff86c005250 0x7ff88415e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:53.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.334+0000 7ff88b789640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff884156150 0x7ff8840b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:53.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.334+0000 7ff88b789640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff88415eaf0 0x7ff8840b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:53.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.334+0000 7ff88b789640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8840b75e0 con 0x7ff88415eaf0 2023-09-23T03:23:53.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.334+0000 7ff88b789640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8840b7300 con 0x7ff884156150 2023-09-23T03:23:53.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.334+0000 7ff88b789640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8840c9b40 con 0x7ff86c005250 2023-09-23T03:23:53.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.334+0000 7ff889d26640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff88415eaf0 0x7ff8840b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:53.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.334+0000 7ff889d26640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff88415eaf0 0x7ff8840b3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60278/0 (socket says 172.21.15.154:60278) 2023-09-23T03:23:53.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.334+0000 7ff889d26640 1 -- 172.21.15.154:0/953546651 learned_addr learned my addr 172.21.15.154:0/953546651 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:23:53.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.334+0000 7ff888d24640 1 --2- 172.21.15.154:0/953546651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff884156150 0x7ff8840b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:53.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.334+0000 7ff889525640 1 --2- 172.21.15.154:0/953546651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff86c005250 0x7ff88415e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:53.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.334+0000 7ff889d26640 1 -- 172.21.15.154:0/953546651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff86c005250 msgr2=0x7ff88415e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:53.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.334+0000 7ff889d26640 1 --2- 172.21.15.154:0/953546651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff86c005250 0x7ff88415e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:53.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.334+0000 7ff889d26640 1 -- 172.21.15.154:0/953546651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff884156150 msgr2=0x7ff8840b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:53.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.334+0000 7ff889d26640 1 --2- 172.21.15.154:0/953546651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff884156150 0x7ff8840b3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:53.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.334+0000 7ff889d26640 1 -- 172.21.15.154:0/953546651 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8840c9e20 con 0x7ff88415eaf0 2023-09-23T03:23:53.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.335+0000 7ff889d26640 1 --2- 172.21.15.154:0/953546651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff88415eaf0 0x7ff8840b3f40 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0099c0 tx=0x7ff87c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:53.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.335+0000 7ff881ffb640 1 -- 172.21.15.154:0/953546651 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c02f800 con 0x7ff88415eaf0 2023-09-23T03:23:53.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.335+0000 7ff881ffb640 1 -- 172.21.15.154:0/953546651 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87c02f980 con 0x7ff88415eaf0 2023-09-23T03:23:53.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.335+0000 7ff88b789640 1 -- 172.21.15.154:0/953546651 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8840ca100 con 0x7ff88415eaf0 2023-09-23T03:23:53.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.335+0000 7ff881ffb640 1 -- 172.21.15.154:0/953546651 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c02fd30 con 0x7ff88415eaf0 2023-09-23T03:23:53.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.335+0000 7ff88b789640 1 -- 172.21.15.154:0/953546651 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8840ca800 con 0x7ff88415eaf0 2023-09-23T03:23:53.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.336+0000 7ff88b789640 1 -- 172.21.15.154:0/953546651 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8840c99a0 con 0x7ff88415eaf0 2023-09-23T03:23:53.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.337+0000 7ff881ffb640 1 -- 172.21.15.154:0/953546651 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7ff87c0423b0 con 0x7ff88415eaf0 2023-09-23T03:23:53.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.338+0000 7ff881ffb640 1 --2- 172.21.15.154:0/953546651 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff87006af30 0x7ff87006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:53.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.338+0000 7ff881ffb640 1 -- 172.21.15.154:0/953546651 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6691+0+0 (secure 0 0 0) 0x7ff87c0ab130 con 0x7ff88415eaf0 2023-09-23T03:23:53.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.338+0000 7ff889525640 1 --2- 172.21.15.154:0/953546651 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff87006af30 0x7ff87006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:53.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.339+0000 7ff889525640 1 --2- 172.21.15.154:0/953546651 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff87006af30 0x7ff87006d3f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff874007fd0 tx=0x7ff874009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:53.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.343+0000 7ff881ffb640 1 -- 172.21.15.154:0/953546651 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7ff8840c99a0 con 0x7ff88415eaf0 2023-09-23T03:23:53.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.536+0000 7ff88b789640 1 -- 172.21.15.154:0/953546651 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7ff8840c99a0 con 0x7ff88415eaf0 2023-09-23T03:23:53.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.537+0000 7ff881ffb640 1 -- 172.21.15.154:0/953546651 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v71) v1 ==== 106+0+11 (secure 0 0 0) 0x7ff8840c99a0 con 0x7ff88415eaf0 2023-09-23T03:23:53.537 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:23:53.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.539+0000 7ff88b789640 1 -- 172.21.15.154:0/953546651 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff87006af30 msgr2=0x7ff87006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:53.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.539+0000 7ff88b789640 1 --2- 172.21.15.154:0/953546651 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff87006af30 0x7ff87006d3f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff874007fd0 tx=0x7ff874009040 comp rx=0 tx=0).stop 2023-09-23T03:23:53.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.539+0000 7ff88b789640 1 -- 172.21.15.154:0/953546651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff88415eaf0 msgr2=0x7ff8840b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:53.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.539+0000 7ff88b789640 1 --2- 172.21.15.154:0/953546651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff88415eaf0 0x7ff8840b3f40 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7ff87c0099c0 tx=0x7ff87c004730 comp rx=0 tx=0).stop 2023-09-23T03:23:53.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.539+0000 7ff88b789640 1 -- 172.21.15.154:0/953546651 shutdown_connections 2023-09-23T03:23:53.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.539+0000 7ff88b789640 1 --2- 172.21.15.154:0/953546651 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff87006af30 0x7ff87006d3f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:53.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.539+0000 7ff88b789640 1 --2- 172.21.15.154:0/953546651 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff88415eaf0 0x7ff8840b3f40 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:53.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.539+0000 7ff88b789640 1 --2- 172.21.15.154:0/953546651 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff884156150 0x7ff8840b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:53.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.539+0000 7ff88b789640 1 --2- 172.21.15.154:0/953546651 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff86c005250 0x7ff88415e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:53.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.539+0000 7ff88b789640 1 -- 172.21.15.154:0/953546651 >> 172.21.15.154:0/953546651 conn(0x7ff88409bc20 msgr2=0x7ff88415c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:53.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.540+0000 7ff88b789640 1 -- 172.21.15.154:0/953546651 shutdown_connections 2023-09-23T03:23:53.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.540+0000 7ff88b789640 1 -- 172.21.15.154:0/953546651 wait complete. 2023-09-23T03:23:53.558 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:23:53.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.767+0000 7f92655f5640 1 Processor -- start 2023-09-23T03:23:53.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.767+0000 7f92655f5640 1 -- start start 2023-09-23T03:23:53.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.769+0000 7f92655f5640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f92600cf040 0x7f92601656e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:53.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.769+0000 7f92655f5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f926015ac30 con 0x7f92600acce0 2023-09-23T03:23:53.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.769+0000 7f92655f5640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92600b1f70 con 0x7f92600b1b20 2023-09-23T03:23:53.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.769+0000 7f92655f5640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92600b2e90 con 0x7f92600cf040 2023-09-23T03:23:53.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.769+0000 7f925effd640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f92600acce0 0x7f926015ed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:50986/0 (socket says 172.21.15.154:50986) 2023-09-23T03:23:53.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.769+0000 7f925effd640 1 -- 172.21.15.154:0/2901868442 learned_addr learned my addr 172.21.15.154:0/2901868442 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:23:53.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.769+0000 7f925f7fe640 1 --2- 172.21.15.154:0/2901868442 >> v2:172.21.15.190:3300/0 conn(0x7f92600cf040 0x7f92601656e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:53.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.770+0000 7f925dffb640 1 -- 172.21.15.154:0/2901868442 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1477151760 0 0) 0x7f9250001290 con 0x7f92600acce0 2023-09-23T03:23:53.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.770+0000 7f925dffb640 1 -- 172.21.15.154:0/2901868442 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9248003520 con 0x7f92600acce0 2023-09-23T03:23:53.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.770+0000 7f925dffb640 1 -- 172.21.15.154:0/2901868442 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2933821006 0 0) 0x7f9248003520 con 0x7f92600acce0 2023-09-23T03:23:53.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.770+0000 7f925dffb640 1 -- 172.21.15.154:0/2901868442 >> v2:172.21.15.190:3300/0 conn(0x7f92600cf040 msgr2=0x7f92601656e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:53.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.770+0000 7f925dffb640 1 --2- 172.21.15.154:0/2901868442 >> v2:172.21.15.190:3300/0 conn(0x7f92600cf040 0x7f92601656e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:53.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.770+0000 7f925dffb640 1 -- 172.21.15.154:0/2901868442 >> v1:172.21.15.190:6789/0 conn(0x7f92600b1b20 legacy=0x7f92600ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:53.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.770+0000 7f925dffb640 1 -- 172.21.15.154:0/2901868442 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92600b2210 con 0x7f92600acce0 2023-09-23T03:23:53.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.770+0000 7f925dffb640 1 -- 172.21.15.154:0/2901868442 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f9250001980 con 0x7f92600acce0 2023-09-23T03:23:53.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.770+0000 7f925dffb640 1 -- 172.21.15.154:0/2901868442 >> v1:172.21.15.154:6789/0 conn(0x7f92600acce0 legacy=0x7f926015ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:53.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.770+0000 7f925dffb640 1 --2- 172.21.15.154:0/2901868442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9248003970 0x7f9248003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:53.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.770+0000 7f925dffb640 1 --2- 172.21.15.154:0/2901868442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92480047c0 0x7f9248004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:53.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.770+0000 7f925dffb640 1 --2- 172.21.15.154:0/2901868442 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92480052a0 0x7f924800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:53.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.770+0000 7f925dffb640 1 -- 172.21.15.154:0/2901868442 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92600b2e90 con 0x7f92480047c0 2023-09-23T03:23:53.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.770+0000 7f925dffb640 1 -- 172.21.15.154:0/2901868442 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924800c170 con 0x7f9248003970 2023-09-23T03:23:53.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.770+0000 7f925dffb640 1 -- 172.21.15.154:0/2901868442 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f924800c330 con 0x7f92480052a0 2023-09-23T03:23:53.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.770+0000 7f925f7fe640 1 --2- 172.21.15.154:0/2901868442 >> v2:172.21.15.190:3300/0 conn(0x7f92600cf040 0x7f92601656e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:23:53.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.770+0000 7f925e7fc640 1 --2- 172.21.15.154:0/2901868442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92480047c0 0x7f9248004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:53.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.771+0000 7f925e7fc640 1 -- 172.21.15.154:0/2901868442 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92480052a0 msgr2=0x7f924800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:53.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.771+0000 7f925e7fc640 1 --2- 172.21.15.154:0/2901868442 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92480052a0 0x7f924800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:53.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.771+0000 7f925e7fc640 1 -- 172.21.15.154:0/2901868442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9248003970 msgr2=0x7f9248003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:53.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.771+0000 7f925e7fc640 1 --2- 172.21.15.154:0/2901868442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9248003970 0x7f9248003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:53.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.771+0000 7f925e7fc640 1 -- 172.21.15.154:0/2901868442 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9248003520 con 0x7f92480047c0 2023-09-23T03:23:53.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.771+0000 7f925f7fe640 1 -- 172.21.15.154:0/2901868442 reap_dead start 2023-09-23T03:23:53.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.772+0000 7f925e7fc640 1 --2- 172.21.15.154:0/2901868442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92480047c0 0x7f9248004c40 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f926006c660 tx=0x7f92480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec0870088bf0816 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:53.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.772+0000 7f925dffb640 1 -- 172.21.15.154:0/2901868442 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92540052d0 con 0x7f92480047c0 2023-09-23T03:23:53.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.772+0000 7f925dffb640 1 -- 172.21.15.154:0/2901868442 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f925402e050 con 0x7f92480047c0 2023-09-23T03:23:53.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.772+0000 7f925dffb640 1 -- 172.21.15.154:0/2901868442 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9254003830 con 0x7f92480047c0 2023-09-23T03:23:53.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.773+0000 7f92655f5640 1 -- 172.21.15.154:0/2901868442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92480047c0 msgr2=0x7f9248004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:53.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.773+0000 7f92655f5640 1 --2- 172.21.15.154:0/2901868442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92480047c0 0x7f9248004c40 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f926006c660 tx=0x7f92480057f0 comp rx=0 tx=0).stop 2023-09-23T03:23:53.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.773+0000 7f92655f5640 1 -- 172.21.15.154:0/2901868442 shutdown_connections 2023-09-23T03:23:53.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.773+0000 7f92655f5640 1 --2- 172.21.15.154:0/2901868442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92480047c0 0x7f9248004c40 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:53.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.773+0000 7f92655f5640 1 -- 172.21.15.154:0/2901868442 >> 172.21.15.154:0/2901868442 conn(0x7f926006a0f0 msgr2=0x7f926006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:53.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.773+0000 7f92655f5640 1 -- 172.21.15.154:0/2901868442 shutdown_connections 2023-09-23T03:23:53.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.773+0000 7f92655f5640 1 -- 172.21.15.154:0/2901868442 wait complete. 2023-09-23T03:23:53.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.775+0000 7f92655f5640 1 Processor -- start 2023-09-23T03:23:53.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.775+0000 7f92655f5640 1 -- start start 2023-09-23T03:23:53.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.776+0000 7f92655f5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92480047c0 0x7f9260155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:53.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.776+0000 7f92655f5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9260156000 0x7f92600c3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:53.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.776+0000 7f92655f5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92600c4390 0x7f92600affe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:53.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.776+0000 7f92655f5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92600c4fd0 con 0x7f92600c4390 2023-09-23T03:23:53.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.776+0000 7f92655f5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92600c5b60 con 0x7f9260156000 2023-09-23T03:23:53.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.776+0000 7f92655f5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92600c4a10 con 0x7f92480047c0 2023-09-23T03:23:53.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.776+0000 7f925f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92600c4390 0x7f92600affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:53.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.776+0000 7f925f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92600c4390 0x7f92600affe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:60300/0 (socket says 172.21.15.154:60300) 2023-09-23T03:23:53.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.776+0000 7f925f7fe640 1 -- 172.21.15.154:0/1757056356 learned_addr learned my addr 172.21.15.154:0/1757056356 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:23:53.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.776+0000 7f925effd640 1 --2- 172.21.15.154:0/1757056356 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92480047c0 0x7f9260155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:53.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.776+0000 7f925e7fc640 1 --2- 172.21.15.154:0/1757056356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9260156000 0x7f92600c3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:53.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.776+0000 7f925f7fe640 1 -- 172.21.15.154:0/1757056356 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92480047c0 msgr2=0x7f9260155ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:53.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.776+0000 7f925f7fe640 1 --2- 172.21.15.154:0/1757056356 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92480047c0 0x7f9260155ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:53.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.776+0000 7f925f7fe640 1 -- 172.21.15.154:0/1757056356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9260156000 msgr2=0x7f92600c3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:53.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.776+0000 7f925f7fe640 1 --2- 172.21.15.154:0/1757056356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9260156000 0x7f92600c3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:53.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.776+0000 7f925f7fe640 1 -- 172.21.15.154:0/1757056356 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92600c4cf0 con 0x7f92600c4390 2023-09-23T03:23:53.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.777+0000 7f925f7fe640 1 --2- 172.21.15.154:0/1757056356 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92600c4390 0x7f92600affe0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f92600c8e70 tx=0x7f925800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:53.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.777+0000 7f92477fe640 1 -- 172.21.15.154:0/1757056356 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925800de90 con 0x7f92600c4390 2023-09-23T03:23:53.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.777+0000 7f92477fe640 1 -- 172.21.15.154:0/1757056356 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f925800f040 con 0x7f92600c4390 2023-09-23T03:23:53.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.777+0000 7f92477fe640 1 -- 172.21.15.154:0/1757056356 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9258003bd0 con 0x7f92600c4390 2023-09-23T03:23:53.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.777+0000 7f92655f5640 1 -- 172.21.15.154:0/1757056356 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92600c5780 con 0x7f92600c4390 2023-09-23T03:23:53.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.777+0000 7f92655f5640 1 -- 172.21.15.154:0/1757056356 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92600b0520 con 0x7f92600c4390 2023-09-23T03:23:53.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.778+0000 7f92655f5640 1 -- 172.21.15.154:0/1757056356 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92600c55e0 con 0x7f92600c4390 2023-09-23T03:23:53.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.779+0000 7f92477fe640 1 -- 172.21.15.154:0/1757056356 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f9258003d30 con 0x7f92600c4390 2023-09-23T03:23:53.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.780+0000 7f92477fe640 1 --2- 172.21.15.154:0/1757056356 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f924c06af30 0x7f924c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:53.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.780+0000 7f92477fe640 1 -- 172.21.15.154:0/1757056356 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f9258089ad0 con 0x7f92600c4390 2023-09-23T03:23:53.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.780+0000 7f925effd640 1 --2- 172.21.15.154:0/1757056356 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f924c06af30 0x7f924c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:53.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.781+0000 7f925effd640 1 --2- 172.21.15.154:0/1757056356 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f924c06af30 0x7f924c06d3f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9250001170 tx=0x7f9250009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:53.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:53.784+0000 7f92477fe640 1 -- 172.21.15.154:0/1757056356 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f92600c55e0 con 0x7f92600c4390 2023-09-23T03:23:54.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.009+0000 7f92655f5640 1 -- 172.21.15.154:0/1757056356 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f92600c5ca0 con 0x7f92600c4390 2023-09-23T03:23:54.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.010+0000 7f92477fe640 1 -- 172.21.15.154:0/1757056356 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4926 (secure 0 0 0) 0x7f92600c5ca0 con 0x7f92600c4390 2023-09-23T03:23:54.010 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:23:54.011 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":54,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5279,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1609915000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1609915000},{"type":"v1","addr":"172.21.15.154:6837","nonce":1609915000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":5288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1240415394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1240415394},{"type":"v1","addr":"172.21.15.190:6835","nonce":1240415394}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":5728,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/878668736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":878668736},{"type":"v1","addr":"172.21.15.190:6833","nonce":878668736}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:21:30.160245+0000","modified":"2023-09-23T03:21:40.311897+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6924},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6924":{"gid":6924,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":4,"addr":"172.21.15.154:6835/2633473586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2633473586},{"type":"v1","addr":"172.21.15.154:6835","nonce":2633473586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9,10],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-09-23T03:23:54.011 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 54 2023-09-23T03:23:54.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.012+0000 7f92655f5640 1 -- 172.21.15.154:0/1757056356 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f924c06af30 msgr2=0x7f924c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:54.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.012+0000 7f92655f5640 1 --2- 172.21.15.154:0/1757056356 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f924c06af30 0x7f924c06d3f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9250001170 tx=0x7f9250009040 comp rx=0 tx=0).stop 2023-09-23T03:23:54.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.012+0000 7f92655f5640 1 -- 172.21.15.154:0/1757056356 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92600c4390 msgr2=0x7f92600affe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:54.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.012+0000 7f92655f5640 1 --2- 172.21.15.154:0/1757056356 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92600c4390 0x7f92600affe0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f92600c8e70 tx=0x7f925800d4a0 comp rx=0 tx=0).stop 2023-09-23T03:23:54.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.013+0000 7f92655f5640 1 -- 172.21.15.154:0/1757056356 shutdown_connections 2023-09-23T03:23:54.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.013+0000 7f92655f5640 1 --2- 172.21.15.154:0/1757056356 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f924c06af30 0x7f924c06d3f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:54.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.013+0000 7f92655f5640 1 --2- 172.21.15.154:0/1757056356 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92600c4390 0x7f92600affe0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:54.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.013+0000 7f92655f5640 1 --2- 172.21.15.154:0/1757056356 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9260156000 0x7f92600c3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:54.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.013+0000 7f92655f5640 1 --2- 172.21.15.154:0/1757056356 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92480047c0 0x7f9260155ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:54.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.013+0000 7f92655f5640 1 -- 172.21.15.154:0/1757056356 >> 172.21.15.154:0/1757056356 conn(0x7f926006a0f0 msgr2=0x7f92600726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:54.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.013+0000 7f92655f5640 1 -- 172.21.15.154:0/1757056356 shutdown_connections 2023-09-23T03:23:54.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.013+0000 7f92655f5640 1 -- 172.21.15.154:0/1757056356 wait complete. 2023-09-23T03:23:54.030 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:23:54.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.235+0000 7f51b315e640 1 Processor -- start 2023-09-23T03:23:54.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.236+0000 7f51b315e640 1 -- start start 2023-09-23T03:23:54.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.237+0000 7f51b315e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f51ac06c7a0 0x7f51ac15eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:54.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.237+0000 7f51b315e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51ac15ab00 0x7f51ac15af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:54.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.237+0000 7f51b315e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51ac06bff0 con 0x7f51ac06fdc0 2023-09-23T03:23:54.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.237+0000 7f51b315e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51ac155070 con 0x7f51ac06c7a0 2023-09-23T03:23:54.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.237+0000 7f51b315e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac15e0f0 con 0x7f51ac15ab00 2023-09-23T03:23:54.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.238+0000 7f51b195b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51ac15ab00 0x7f51ac15af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:54.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.238+0000 7f51b215c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f51ac06c7a0 0x7f51ac15eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:54.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.238+0000 7f51b195b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51ac15ab00 0x7f51ac15af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:51092/0 (socket says 172.21.15.154:51092) 2023-09-23T03:23:54.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.238+0000 7f51b195b640 1 -- 172.21.15.154:0/613179567 learned_addr learned my addr 172.21.15.154:0/613179567 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:23:54.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.239+0000 7f51b195b640 1 -- 172.21.15.154:0/613179567 >> v2:172.21.15.190:3300/0 conn(0x7f51ac06c7a0 msgr2=0x7f51ac15eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:54.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.239+0000 7f51b195b640 1 --2- 172.21.15.154:0/613179567 >> v2:172.21.15.190:3300/0 conn(0x7f51ac06c7a0 0x7f51ac15eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:54.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.239+0000 7f51b195b640 1 -- 172.21.15.154:0/613179567 >> v1:172.21.15.190:6789/0 conn(0x7f51ac06fdc0 legacy=0x7f51ac0ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:54.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.239+0000 7f51b195b640 1 -- 172.21.15.154:0/613179567 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ac06cb80 con 0x7f51ac15ab00 2023-09-23T03:23:54.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.240+0000 7f51b195b640 1 --2- 172.21.15.154:0/613179567 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51ac15ab00 0x7f51ac15af80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f519c009870 tx=0x7f519c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7eab44e141c317dc server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:54.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.240+0000 7f51b115a640 1 -- 172.21.15.154:0/613179567 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519c030840 con 0x7f51ac15ab00 2023-09-23T03:23:54.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.241+0000 7f51b115a640 1 -- 172.21.15.154:0/613179567 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519c0309a0 con 0x7f51ac15ab00 2023-09-23T03:23:54.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.241+0000 7f51b115a640 1 -- 172.21.15.154:0/613179567 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519c030d30 con 0x7f51ac15ab00 2023-09-23T03:23:54.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.241+0000 7f51b315e640 1 -- 172.21.15.154:0/613179567 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51ac15ab00 msgr2=0x7f51ac15af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:54.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.241+0000 7f51b315e640 1 --2- 172.21.15.154:0/613179567 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51ac15ab00 0x7f51ac15af80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f519c009870 tx=0x7f519c02b980 comp rx=0 tx=0).stop 2023-09-23T03:23:54.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.242+0000 7f51b315e640 1 -- 172.21.15.154:0/613179567 shutdown_connections 2023-09-23T03:23:54.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.242+0000 7f51b315e640 1 --2- 172.21.15.154:0/613179567 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51ac15ab00 0x7f51ac15af80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:54.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.242+0000 7f51b315e640 1 --2- 172.21.15.154:0/613179567 >> v2:172.21.15.190:3300/0 conn(0x7f51ac06c7a0 0x7f51ac15eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:54.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.242+0000 7f51b315e640 1 -- 172.21.15.154:0/613179567 >> 172.21.15.154:0/613179567 conn(0x7f51ac06a390 msgr2=0x7f51ac0afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:54.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.242+0000 7f51b315e640 1 -- 172.21.15.154:0/613179567 shutdown_connections 2023-09-23T03:23:54.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.242+0000 7f51b315e640 1 -- 172.21.15.154:0/613179567 wait complete. 2023-09-23T03:23:54.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.244+0000 7f51b315e640 1 Processor -- start 2023-09-23T03:23:54.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.244+0000 7f51b315e640 1 -- start start 2023-09-23T03:23:54.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.245+0000 7f51b315e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51ac06c7a0 0x7f51ac14b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:54.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.245+0000 7f51b315e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51ac06fdc0 0x7f51ac0c7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:54.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.245+0000 7f51b315e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51ac15ab00 0x7f51ac0c9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:54.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.245+0000 7f51b315e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac0b30d0 con 0x7f51ac06c7a0 2023-09-23T03:23:54.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.245+0000 7f51b315e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac0b24e0 con 0x7f51ac15ab00 2023-09-23T03:23:54.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.245+0000 7f51b315e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac14be90 con 0x7f51ac06fdc0 2023-09-23T03:23:54.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.245+0000 7f51b215c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51ac06c7a0 0x7f51ac14b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:54.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.245+0000 7f51b215c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51ac06c7a0 0x7f51ac14b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:51390/0 (socket says 172.21.15.154:51390) 2023-09-23T03:23:54.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.245+0000 7f51b215c640 1 -- 172.21.15.154:0/610623967 learned_addr learned my addr 172.21.15.154:0/610623967 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:23:54.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.246+0000 7f51b195b640 1 --2- 172.21.15.154:0/610623967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51ac06fdc0 0x7f51ac0c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:54.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.246+0000 7f51b295d640 1 --2- 172.21.15.154:0/610623967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51ac15ab00 0x7f51ac0c9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:54.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.246+0000 7f51b215c640 1 -- 172.21.15.154:0/610623967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51ac06fdc0 msgr2=0x7f51ac0c7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:54.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.246+0000 7f51b215c640 1 --2- 172.21.15.154:0/610623967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51ac06fdc0 0x7f51ac0c7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:54.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.246+0000 7f51b215c640 1 -- 172.21.15.154:0/610623967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51ac15ab00 msgr2=0x7f51ac0c9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:54.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.246+0000 7f51b215c640 1 --2- 172.21.15.154:0/610623967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51ac15ab00 0x7f51ac0c9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:54.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.246+0000 7f51b215c640 1 -- 172.21.15.154:0/610623967 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f519c0094d0 con 0x7f51ac06c7a0 2023-09-23T03:23:54.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.246+0000 7f51b295d640 1 --2- 172.21.15.154:0/610623967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51ac15ab00 0x7f51ac0c9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:23:54.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.246+0000 7f51b215c640 1 --2- 172.21.15.154:0/610623967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51ac06c7a0 0x7f51ac14b950 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f51a800e9b0 tx=0x7f51a800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:54.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.246+0000 7f51a2ffd640 1 -- 172.21.15.154:0/610623967 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a8009670 con 0x7f51ac06c7a0 2023-09-23T03:23:54.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.246+0000 7f51a2ffd640 1 -- 172.21.15.154:0/610623967 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a800c9a0 con 0x7f51ac06c7a0 2023-09-23T03:23:54.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.246+0000 7f51b315e640 1 -- 172.21.15.154:0/610623967 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51ac0b27a0 con 0x7f51ac06c7a0 2023-09-23T03:23:54.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.246+0000 7f51a2ffd640 1 -- 172.21.15.154:0/610623967 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a800cd50 con 0x7f51ac06c7a0 2023-09-23T03:23:54.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.247+0000 7f51b315e640 1 -- 172.21.15.154:0/610623967 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51ac0b2d60 con 0x7f51ac06c7a0 2023-09-23T03:23:54.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.248+0000 7f51b315e640 1 -- 172.21.15.154:0/610623967 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51ac0736b0 con 0x7f51ac06c7a0 2023-09-23T03:23:54.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.249+0000 7f51a2ffd640 1 -- 172.21.15.154:0/610623967 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f51a8013470 con 0x7f51ac06c7a0 2023-09-23T03:23:54.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.249+0000 7f51a2ffd640 1 --2- 172.21.15.154:0/610623967 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f519006af30 0x7f519006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:54.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.250+0000 7f51a2ffd640 1 -- 172.21.15.154:0/610623967 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f51a8089e30 con 0x7f51ac06c7a0 2023-09-23T03:23:54.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.250+0000 7f51b195b640 1 --2- 172.21.15.154:0/610623967 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f519006af30 0x7f519006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:54.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.250+0000 7f51b195b640 1 --2- 172.21.15.154:0/610623967 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f519006af30 0x7f519006d3f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f519c036000 tx=0x7f519c03b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:54.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.254+0000 7f51a2ffd640 1 -- 172.21.15.154:0/610623967 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f51a8054a70 con 0x7f51ac06c7a0 2023-09-23T03:23:54.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.447+0000 7f51b315e640 1 -- 172.21.15.154:0/610623967 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51ac04e920 con 0x7f51ac06c7a0 2023-09-23T03:23:54.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.450+0000 7f51a2ffd640 1 -- 172.21.15.154:0/610623967 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+18987 (secure 0 0 0) 0x7f51a8017090 con 0x7f51ac06c7a0 2023-09-23T03:23:54.450 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:23:54.451 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":71,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:21:39.274806+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:19:21.772037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-09-23T03:19:22.960250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data_extra","create_time":"2023-09-23T03:19:38.870843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6837/4129414148":"2023-09-24T03:21:38.278599+0000","172.21.15.154:6835/4129414148":"2023-09-24T03:21:38.278599+0000","172.21.15.154:6837/2976977273":"2023-09-24T03:21:31.069171+0000","172.21.15.154:6835/2976977273":"2023-09-24T03:21:31.069171+0000","172.21.15.154:6837/2183023203":"2023-09-24T03:21:29.032871+0000","172.21.15.154:6835/2183023203":"2023-09-24T03:21:29.032871+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:23:54.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.452+0000 7f51b315e640 1 -- 172.21.15.154:0/610623967 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f519006af30 msgr2=0x7f519006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:54.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.452+0000 7f51b315e640 1 --2- 172.21.15.154:0/610623967 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f519006af30 0x7f519006d3f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f519c036000 tx=0x7f519c03b040 comp rx=0 tx=0).stop 2023-09-23T03:23:54.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.452+0000 7f51b315e640 1 -- 172.21.15.154:0/610623967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51ac06c7a0 msgr2=0x7f51ac14b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:54.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.452+0000 7f51b315e640 1 --2- 172.21.15.154:0/610623967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51ac06c7a0 0x7f51ac14b950 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f51a800e9b0 tx=0x7f51a800ee70 comp rx=0 tx=0).stop 2023-09-23T03:23:54.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.452+0000 7f51b315e640 1 -- 172.21.15.154:0/610623967 shutdown_connections 2023-09-23T03:23:54.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.452+0000 7f51b315e640 1 --2- 172.21.15.154:0/610623967 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f519006af30 0x7f519006d3f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:54.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.452+0000 7f51b315e640 1 --2- 172.21.15.154:0/610623967 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51ac15ab00 0x7f51ac0c9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:54.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.452+0000 7f51b315e640 1 --2- 172.21.15.154:0/610623967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51ac06fdc0 0x7f51ac0c7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:54.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.452+0000 7f51b315e640 1 --2- 172.21.15.154:0/610623967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51ac06c7a0 0x7f51ac14b950 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:54.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.452+0000 7f51b315e640 1 -- 172.21.15.154:0/610623967 >> 172.21.15.154:0/610623967 conn(0x7f51ac06a390 msgr2=0x7f51ac0add80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:54.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.453+0000 7f51b315e640 1 -- 172.21.15.154:0/610623967 shutdown_connections 2023-09-23T03:23:54.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.453+0000 7f51b315e640 1 -- 172.21.15.154:0/610623967 wait complete. 2023-09-23T03:23:54.472 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-09-23T03:23:54.472 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:23:54.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.692+0000 7fcafc807640 1 Processor -- start 2023-09-23T03:23:54.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.693+0000 7fcafc807640 1 -- start start 2023-09-23T03:23:54.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.694+0000 7fcafc807640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcaf80acce0 0x7fcaf815ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:54.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.694+0000 7fcafc807640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcaf80b1b20 0x7fcaf80ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:54.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.694+0000 7fcafc807640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf815ac30 con 0x7fcaf80cee70 2023-09-23T03:23:54.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.694+0000 7fcafc807640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaf80b2e90 con 0x7fcaf80acce0 2023-09-23T03:23:54.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.694+0000 7fcafc807640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaf80cf300 con 0x7fcaf80b1b20 2023-09-23T03:23:54.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.694+0000 7fcaf6d9d640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fcaf80cee70 0x7fcaf806cb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46006/0 (socket says 172.21.15.154:46006) 2023-09-23T03:23:54.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.694+0000 7fcaf6d9d640 1 -- 172.21.15.154:0/1114470339 learned_addr learned my addr 172.21.15.154:0/1114470339 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:23:54.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.695+0000 7fcaf659c640 1 --2- 172.21.15.154:0/1114470339 >> v2:172.21.15.154:3300/0 conn(0x7fcaf80acce0 0x7fcaf815ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:54.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.695+0000 7fcaeffff640 1 --2- 172.21.15.154:0/1114470339 >> v2:172.21.15.190:3300/0 conn(0x7fcaf80b1b20 0x7fcaf80ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:54.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.695+0000 7fcaf5d9b640 1 -- 172.21.15.154:0/1114470339 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1874927221 0 0) 0x7fcaf0001490 con 0x7fcaf80cee70 2023-09-23T03:23:54.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.695+0000 7fcaf5d9b640 1 -- 172.21.15.154:0/1114470339 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcae4003520 con 0x7fcaf80cee70 2023-09-23T03:23:54.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.695+0000 7fcaf659c640 1 -- 172.21.15.154:0/1114470339 >> v2:172.21.15.190:3300/0 conn(0x7fcaf80b1b20 msgr2=0x7fcaf80ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:54.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.695+0000 7fcaf659c640 1 --2- 172.21.15.154:0/1114470339 >> v2:172.21.15.190:3300/0 conn(0x7fcaf80b1b20 0x7fcaf80ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:54.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.695+0000 7fcaf659c640 1 -- 172.21.15.154:0/1114470339 >> v1:172.21.15.154:6789/0 conn(0x7fcaf80cee70 legacy=0x7fcaf806cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:54.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.695+0000 7fcaf659c640 1 -- 172.21.15.154:0/1114470339 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf80b1f70 con 0x7fcaf80acce0 2023-09-23T03:23:54.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.696+0000 7fcaf659c640 1 --2- 172.21.15.154:0/1114470339 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaf80acce0 0x7fcaf815ed60 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fcaf80c9430 tx=0x7fcae802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab26c169ce2fa1be server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:54.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.696+0000 7fcaf5d9b640 1 -- 172.21.15.154:0/1114470339 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae802e070 con 0x7fcaf80acce0 2023-09-23T03:23:54.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.696+0000 7fcaf5d9b640 1 -- 172.21.15.154:0/1114470339 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae80315a0 con 0x7fcaf80acce0 2023-09-23T03:23:54.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.696+0000 7fcaf5d9b640 1 -- 172.21.15.154:0/1114470339 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae8031910 con 0x7fcaf80acce0 2023-09-23T03:23:54.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.697+0000 7fcafc807640 1 -- 172.21.15.154:0/1114470339 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaf80acce0 msgr2=0x7fcaf815ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:54.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.697+0000 7fcafc807640 1 --2- 172.21.15.154:0/1114470339 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaf80acce0 0x7fcaf815ed60 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fcaf80c9430 tx=0x7fcae802b840 comp rx=0 tx=0).stop 2023-09-23T03:23:54.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.697+0000 7fcafc807640 1 -- 172.21.15.154:0/1114470339 shutdown_connections 2023-09-23T03:23:54.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.697+0000 7fcafc807640 1 --2- 172.21.15.154:0/1114470339 >> v2:172.21.15.190:3300/0 conn(0x7fcaf80b1b20 0x7fcaf80ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:54.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.697+0000 7fcafc807640 1 --2- 172.21.15.154:0/1114470339 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaf80acce0 0x7fcaf815ed60 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:54.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.697+0000 7fcafc807640 1 -- 172.21.15.154:0/1114470339 >> 172.21.15.154:0/1114470339 conn(0x7fcaf806a0f0 msgr2=0x7fcaf806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:54.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.697+0000 7fcafc807640 1 -- 172.21.15.154:0/1114470339 shutdown_connections 2023-09-23T03:23:54.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.697+0000 7fcafc807640 1 -- 172.21.15.154:0/1114470339 wait complete. 2023-09-23T03:23:54.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.699+0000 7fcafc807640 1 Processor -- start 2023-09-23T03:23:54.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.699+0000 7fcafc807640 1 -- start start 2023-09-23T03:23:54.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.700+0000 7fcafc807640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcaf80acce0 0x7fcaf8157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:54.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.700+0000 7fcafc807640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaf80b1b20 0x7fcaf80cd800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:54.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.700+0000 7fcafc807640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaf80cee70 0x7fcaf80cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:54.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.700+0000 7fcafc807640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf8157bb0 con 0x7fcaf80b1b20 2023-09-23T03:23:54.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.700+0000 7fcafc807640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf81f7060 con 0x7fcaf80acce0 2023-09-23T03:23:54.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.700+0000 7fcafc807640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf80ccf20 con 0x7fcaf80cee70 2023-09-23T03:23:54.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.700+0000 7fcaeffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaf80b1b20 0x7fcaf80cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:54.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.700+0000 7fcaeffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaf80b1b20 0x7fcaf80cd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:51410/0 (socket says 172.21.15.154:51410) 2023-09-23T03:23:54.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.700+0000 7fcaeffff640 1 -- 172.21.15.154:0/1715573791 learned_addr learned my addr 172.21.15.154:0/1715573791 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:23:54.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.701+0000 7fcaeffff640 1 -- 172.21.15.154:0/1715573791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaf80cee70 msgr2=0x7fcaf80cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:54.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.701+0000 7fcaeffff640 1 --2- 172.21.15.154:0/1715573791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaf80cee70 0x7fcaf80cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:54.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.701+0000 7fcaeffff640 1 -- 172.21.15.154:0/1715573791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcaf80acce0 msgr2=0x7fcaf8157670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:54.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.701+0000 7fcaeffff640 1 --2- 172.21.15.154:0/1715573791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcaf80acce0 0x7fcaf8157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:54.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.701+0000 7fcaeffff640 1 -- 172.21.15.154:0/1715573791 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae80093a0 con 0x7fcaf80b1b20 2023-09-23T03:23:54.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.701+0000 7fcaeffff640 1 --2- 172.21.15.154:0/1715573791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaf80b1b20 0x7fcaf80cd800 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fcae000d830 tx=0x7fcae000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:54.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.701+0000 7fcaeeffd640 1 -- 172.21.15.154:0/1715573791 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae000b6c0 con 0x7fcaf80b1b20 2023-09-23T03:23:54.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.701+0000 7fcaeeffd640 1 -- 172.21.15.154:0/1715573791 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae000b820 con 0x7fcaf80b1b20 2023-09-23T03:23:54.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.701+0000 7fcafc807640 1 -- 172.21.15.154:0/1715573791 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf81f6490 con 0x7fcaf80b1b20 2023-09-23T03:23:54.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.701+0000 7fcaeeffd640 1 -- 172.21.15.154:0/1715573791 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae000bbb0 con 0x7fcaf80b1b20 2023-09-23T03:23:54.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.701+0000 7fcafc807640 1 -- 172.21.15.154:0/1715573791 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf81f6a10 con 0x7fcaf80b1b20 2023-09-23T03:23:54.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.702+0000 7fcafc807640 1 -- 172.21.15.154:0/1715573791 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaf80c1990 con 0x7fcaf80b1b20 2023-09-23T03:23:54.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.703+0000 7fcaeeffd640 1 -- 172.21.15.154:0/1715573791 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fcae001e460 con 0x7fcaf80b1b20 2023-09-23T03:23:54.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.704+0000 7fcaeeffd640 1 --2- 172.21.15.154:0/1715573791 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcad806af30 0x7fcad806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:54.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.704+0000 7fcaeeffd640 1 -- 172.21.15.154:0/1715573791 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fcae008a2e0 con 0x7fcaf80b1b20 2023-09-23T03:23:54.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.705+0000 7fcaf659c640 1 --2- 172.21.15.154:0/1715573791 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcad806af30 0x7fcad806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:54.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.705+0000 7fcaf659c640 1 --2- 172.21.15.154:0/1715573791 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcad806af30 0x7fcad806d3f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcae8036000 tx=0x7fcae8030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:54.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.709+0000 7fcaeeffd640 1 -- 172.21.15.154:0/1715573791 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fcae0054f30 con 0x7fcaf80b1b20 2023-09-23T03:23:54.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.921+0000 7fcafc807640 1 -- 172.21.15.154:0/1715573791 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcaf8073eb0 con 0x7fcaf80b1b20 2023-09-23T03:23:54.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.923+0000 7fcaeeffd640 1 -- 172.21.15.154:0/1715573791 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4926 (secure 0 0 0) 0x7fcae0058550 con 0x7fcaf80b1b20 2023-09-23T03:23:54.923 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:23:54.923 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":54,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5279,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1609915000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1609915000},{"type":"v1","addr":"172.21.15.154:6837","nonce":1609915000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":5288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1240415394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1240415394},{"type":"v1","addr":"172.21.15.190:6835","nonce":1240415394}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":5728,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/878668736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":878668736},{"type":"v1","addr":"172.21.15.190:6833","nonce":878668736}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:21:30.160245+0000","modified":"2023-09-23T03:21:40.311897+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6924},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6924":{"gid":6924,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":4,"addr":"172.21.15.154:6835/2633473586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2633473586},{"type":"v1","addr":"172.21.15.154:6835","nonce":2633473586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9,10],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-09-23T03:23:54.924 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 54 2023-09-23T03:23:54.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.925+0000 7fcafc807640 1 -- 172.21.15.154:0/1715573791 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcad806af30 msgr2=0x7fcad806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:54.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.925+0000 7fcafc807640 1 --2- 172.21.15.154:0/1715573791 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcad806af30 0x7fcad806d3f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcae8036000 tx=0x7fcae8030000 comp rx=0 tx=0).stop 2023-09-23T03:23:54.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.925+0000 7fcafc807640 1 -- 172.21.15.154:0/1715573791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaf80b1b20 msgr2=0x7fcaf80cd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:54.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.925+0000 7fcafc807640 1 --2- 172.21.15.154:0/1715573791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaf80b1b20 0x7fcaf80cd800 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fcae000d830 tx=0x7fcae000dcf0 comp rx=0 tx=0).stop 2023-09-23T03:23:54.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.926+0000 7fcafc807640 1 -- 172.21.15.154:0/1715573791 shutdown_connections 2023-09-23T03:23:54.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.926+0000 7fcafc807640 1 --2- 172.21.15.154:0/1715573791 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcad806af30 0x7fcad806d3f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:54.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.926+0000 7fcafc807640 1 --2- 172.21.15.154:0/1715573791 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaf80cee70 0x7fcaf80cdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:54.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.926+0000 7fcafc807640 1 --2- 172.21.15.154:0/1715573791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaf80b1b20 0x7fcaf80cd800 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:54.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.926+0000 7fcafc807640 1 --2- 172.21.15.154:0/1715573791 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcaf80acce0 0x7fcaf8157670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:54.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.926+0000 7fcafc807640 1 -- 172.21.15.154:0/1715573791 >> 172.21.15.154:0/1715573791 conn(0x7fcaf806a0f0 msgr2=0x7fcaf80cb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:54.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.926+0000 7fcafc807640 1 -- 172.21.15.154:0/1715573791 shutdown_connections 2023-09-23T03:23:54.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:54.926+0000 7fcafc807640 1 -- 172.21.15.154:0/1715573791 wait complete. 2023-09-23T03:23:54.942 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:23:55.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.148+0000 7fe94f5f5640 1 Processor -- start 2023-09-23T03:23:55.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.148+0000 7fe94f5f5640 1 -- start start 2023-09-23T03:23:55.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.149+0000 7fe94f5f5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9480ada30 0x7fe9480ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:55.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.149+0000 7fe94f5f5640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe9480b2fe0 0x7fe9480cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:55.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.150+0000 7fe94f5f5640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe94806ceb0 con 0x7fe9480cef60 2023-09-23T03:23:55.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.150+0000 7fe94f5f5640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9480ade10 con 0x7fe9480b2fe0 2023-09-23T03:23:55.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.150+0000 7fe94f5f5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9480cf3b0 con 0x7fe9480ada30 2023-09-23T03:23:55.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.150+0000 7fe94d391640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9480ada30 0x7fe9480ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:55.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.150+0000 7fe94d391640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9480ada30 0x7fe9480ccaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:51124/0 (socket says 172.21.15.154:51124) 2023-09-23T03:23:55.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.150+0000 7fe94cb90640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe9480b2fe0 0x7fe9480cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:55.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.150+0000 7fe94d391640 1 -- 172.21.15.154:0/3676328980 learned_addr learned my addr 172.21.15.154:0/3676328980 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:23:55.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.151+0000 7fe94d391640 1 -- 172.21.15.154:0/3676328980 >> v2:172.21.15.190:3300/0 conn(0x7fe9480b2fe0 msgr2=0x7fe9480cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:55.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.151+0000 7fe94d391640 1 --2- 172.21.15.154:0/3676328980 >> v2:172.21.15.190:3300/0 conn(0x7fe9480b2fe0 0x7fe9480cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:55.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.151+0000 7fe94d391640 1 -- 172.21.15.154:0/3676328980 >> v1:172.21.15.190:6789/0 conn(0x7fe9480cef60 legacy=0x7fe9480b18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:55.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.151+0000 7fe94d391640 1 -- 172.21.15.154:0/3676328980 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe948159c90 con 0x7fe9480ada30 2023-09-23T03:23:55.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.152+0000 7fe94d391640 1 --2- 172.21.15.154:0/3676328980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9480ada30 0x7fe9480ccaf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe944009850 tx=0x7fe94402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f031bb2c612ba8e1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:55.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.153+0000 7fe93ffff640 1 -- 172.21.15.154:0/3676328980 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe944004230 con 0x7fe9480ada30 2023-09-23T03:23:55.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.153+0000 7fe93ffff640 1 -- 172.21.15.154:0/3676328980 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe944004390 con 0x7fe9480ada30 2023-09-23T03:23:55.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.153+0000 7fe93ffff640 1 -- 172.21.15.154:0/3676328980 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe944031830 con 0x7fe9480ada30 2023-09-23T03:23:55.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.153+0000 7fe94f5f5640 1 -- 172.21.15.154:0/3676328980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9480ada30 msgr2=0x7fe9480ccaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:55.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.153+0000 7fe94f5f5640 1 --2- 172.21.15.154:0/3676328980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9480ada30 0x7fe9480ccaf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe944009850 tx=0x7fe94402b8d0 comp rx=0 tx=0).stop 2023-09-23T03:23:55.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.153+0000 7fe94f5f5640 1 -- 172.21.15.154:0/3676328980 shutdown_connections 2023-09-23T03:23:55.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.153+0000 7fe94f5f5640 1 --2- 172.21.15.154:0/3676328980 >> v2:172.21.15.190:3300/0 conn(0x7fe9480b2fe0 0x7fe9480cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:55.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.153+0000 7fe94f5f5640 1 --2- 172.21.15.154:0/3676328980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9480ada30 0x7fe9480ccaf0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:55.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.153+0000 7fe94f5f5640 1 -- 172.21.15.154:0/3676328980 >> 172.21.15.154:0/3676328980 conn(0x7fe94806a0f0 msgr2=0x7fe94806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:55.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.154+0000 7fe94f5f5640 1 -- 172.21.15.154:0/3676328980 shutdown_connections 2023-09-23T03:23:55.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.154+0000 7fe94f5f5640 1 -- 172.21.15.154:0/3676328980 wait complete. 2023-09-23T03:23:55.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.155+0000 7fe94f5f5640 1 Processor -- start 2023-09-23T03:23:55.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.155+0000 7fe94f5f5640 1 -- start start 2023-09-23T03:23:55.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.156+0000 7fe94f5f5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9480ada30 0x7fe948155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:55.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.156+0000 7fe94f5f5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9480b2fe0 0x7fe94815bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:55.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.156+0000 7fe94f5f5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9480cef60 0x7fe9480ac2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:55.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.156+0000 7fe94f5f5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94815a650 con 0x7fe9480cef60 2023-09-23T03:23:55.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.156+0000 7fe94f5f5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94815c3d0 con 0x7fe9480b2fe0 2023-09-23T03:23:55.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.156+0000 7fe94f5f5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe948159f90 con 0x7fe9480ada30 2023-09-23T03:23:55.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.157+0000 7fe94db92640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9480cef60 0x7fe9480ac2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:55.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.157+0000 7fe94db92640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9480cef60 0x7fe9480ac2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:51416/0 (socket says 172.21.15.154:51416) 2023-09-23T03:23:55.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.157+0000 7fe94db92640 1 -- 172.21.15.154:0/774326767 learned_addr learned my addr 172.21.15.154:0/774326767 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:23:55.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.157+0000 7fe94d391640 1 --2- 172.21.15.154:0/774326767 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9480ada30 0x7fe948155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:55.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.157+0000 7fe94cb90640 1 --2- 172.21.15.154:0/774326767 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9480b2fe0 0x7fe94815bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:55.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.157+0000 7fe94db92640 1 -- 172.21.15.154:0/774326767 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9480ada30 msgr2=0x7fe948155f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:55.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.157+0000 7fe94db92640 1 --2- 172.21.15.154:0/774326767 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9480ada30 0x7fe948155f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:55.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.157+0000 7fe94db92640 1 -- 172.21.15.154:0/774326767 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9480b2fe0 msgr2=0x7fe94815bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:55.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.157+0000 7fe94db92640 1 --2- 172.21.15.154:0/774326767 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9480b2fe0 0x7fe94815bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:55.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.157+0000 7fe94db92640 1 -- 172.21.15.154:0/774326767 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe944009460 con 0x7fe9480cef60 2023-09-23T03:23:55.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.158+0000 7fe94db92640 1 --2- 172.21.15.154:0/774326767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9480cef60 0x7fe9480ac2c0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fe94000b970 tx=0x7fe94000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:55.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.158+0000 7fe93dffb640 1 -- 172.21.15.154:0/774326767 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94000cb40 con 0x7fe9480cef60 2023-09-23T03:23:55.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.158+0000 7fe93dffb640 1 -- 172.21.15.154:0/774326767 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe94000cca0 con 0x7fe9480cef60 2023-09-23T03:23:55.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.158+0000 7fe94f5f5640 1 -- 172.21.15.154:0/774326767 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe94815a270 con 0x7fe9480cef60 2023-09-23T03:23:55.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.158+0000 7fe93dffb640 1 -- 172.21.15.154:0/774326767 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe940014680 con 0x7fe9480cef60 2023-09-23T03:23:55.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.158+0000 7fe94f5f5640 1 -- 172.21.15.154:0/774326767 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe94815c6b0 con 0x7fe9480cef60 2023-09-23T03:23:55.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.159+0000 7fe94f5f5640 1 -- 172.21.15.154:0/774326767 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe94815a790 con 0x7fe9480cef60 2023-09-23T03:23:55.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.160+0000 7fe93dffb640 1 -- 172.21.15.154:0/774326767 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fe94001d3b0 con 0x7fe9480cef60 2023-09-23T03:23:55.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.161+0000 7fe93dffb640 1 --2- 172.21.15.154:0/774326767 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe92c06af60 0x7fe92c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:55.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.161+0000 7fe93dffb640 1 -- 172.21.15.154:0/774326767 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fe940089a80 con 0x7fe9480cef60 2023-09-23T03:23:55.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.161+0000 7fe94d391640 1 --2- 172.21.15.154:0/774326767 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe92c06af60 0x7fe92c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:55.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.162+0000 7fe94d391640 1 --2- 172.21.15.154:0/774326767 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe92c06af60 0x7fe92c06d420 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe944035000 tx=0x7fe944008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:55.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.165+0000 7fe93dffb640 1 -- 172.21.15.154:0/774326767 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fe94815a790 con 0x7fe9480cef60 2023-09-23T03:23:55.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.385+0000 7fe94f5f5640 1 -- 172.21.15.154:0/774326767 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe94815a790 con 0x7fe9480cef60 2023-09-23T03:23:55.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.386+0000 7fe93dffb640 1 -- 172.21.15.154:0/774326767 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4926 (secure 0 0 0) 0x7fe94815a790 con 0x7fe9480cef60 2023-09-23T03:23:55.387 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:23:55.387 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":54,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5279,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1609915000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1609915000},{"type":"v1","addr":"172.21.15.154:6837","nonce":1609915000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":5288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1240415394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1240415394},{"type":"v1","addr":"172.21.15.190:6835","nonce":1240415394}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":5728,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/878668736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":878668736},{"type":"v1","addr":"172.21.15.190:6833","nonce":878668736}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:21:30.160245+0000","modified":"2023-09-23T03:21:40.311897+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6924},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6924":{"gid":6924,"name":"a","rank":0,"incarnation":51,"state":"up:active","state_seq":4,"addr":"172.21.15.154:6835/2633473586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2633473586},{"type":"v1","addr":"172.21.15.154:6835","nonce":2633473586}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9,10],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-09-23T03:23:55.387 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 54 2023-09-23T03:23:55.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.389+0000 7fe94f5f5640 1 -- 172.21.15.154:0/774326767 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe92c06af60 msgr2=0x7fe92c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:55.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.389+0000 7fe94f5f5640 1 --2- 172.21.15.154:0/774326767 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe92c06af60 0x7fe92c06d420 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe944035000 tx=0x7fe944008030 comp rx=0 tx=0).stop 2023-09-23T03:23:55.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.389+0000 7fe94f5f5640 1 -- 172.21.15.154:0/774326767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9480cef60 msgr2=0x7fe9480ac2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:55.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.389+0000 7fe94f5f5640 1 --2- 172.21.15.154:0/774326767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9480cef60 0x7fe9480ac2c0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fe94000b970 tx=0x7fe94000be30 comp rx=0 tx=0).stop 2023-09-23T03:23:55.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.389+0000 7fe94f5f5640 1 -- 172.21.15.154:0/774326767 shutdown_connections 2023-09-23T03:23:55.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.389+0000 7fe94f5f5640 1 --2- 172.21.15.154:0/774326767 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe92c06af60 0x7fe92c06d420 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:55.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.389+0000 7fe94f5f5640 1 --2- 172.21.15.154:0/774326767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe9480cef60 0x7fe9480ac2c0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:55.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.389+0000 7fe94f5f5640 1 --2- 172.21.15.154:0/774326767 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe9480b2fe0 0x7fe94815bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:55.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.389+0000 7fe94f5f5640 1 --2- 172.21.15.154:0/774326767 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe9480ada30 0x7fe948155f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:55.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.389+0000 7fe94f5f5640 1 -- 172.21.15.154:0/774326767 >> 172.21.15.154:0/774326767 conn(0x7fe94806a0f0 msgr2=0x7fe9480af7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:55.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.389+0000 7fe94f5f5640 1 -- 172.21.15.154:0/774326767 shutdown_connections 2023-09-23T03:23:55.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.389+0000 7fe94f5f5640 1 -- 172.21.15.154:0/774326767 wait complete. 2023-09-23T03:23:55.408 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:23:55.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.611+0000 7f7a5e661640 1 Processor -- start 2023-09-23T03:23:55.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.612+0000 7f7a5e661640 1 -- start start 2023-09-23T03:23:55.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.613+0000 7f7a5e661640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7a580cf130 0x7f7a580ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:55.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.613+0000 7f7a5e661640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a5806ceb0 con 0x7f7a580b2fe0 2023-09-23T03:23:55.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.613+0000 7f7a5e661640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a58159c90 con 0x7f7a580ada30 2023-09-23T03:23:55.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.613+0000 7f7a5e661640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a580ade10 con 0x7f7a580cf130 2023-09-23T03:23:55.615 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.613+0000 7f7a5cbfe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f7a580b2fe0 0x7f7a5815d380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46022/0 (socket says 172.21.15.154:46022) 2023-09-23T03:23:55.615 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.613+0000 7f7a5cbfe640 1 -- 172.21.15.154:0/4159904829 learned_addr learned my addr 172.21.15.154:0/4159904829 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:23:55.615 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.614+0000 7f7a4ffff640 1 --2- 172.21.15.154:0/4159904829 >> v2:172.21.15.190:3300/0 conn(0x7f7a580cf130 0x7f7a580ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:55.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.614+0000 7f7a577fe640 1 -- 172.21.15.154:0/4159904829 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 196995018 0 0) 0x7f7a50001490 con 0x7f7a580b2fe0 2023-09-23T03:23:55.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.614+0000 7f7a577fe640 1 -- 172.21.15.154:0/4159904829 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a44003520 con 0x7f7a580b2fe0 2023-09-23T03:23:55.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.614+0000 7f7a577fe640 1 -- 172.21.15.154:0/4159904829 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3335540636 0 0) 0x7f7a44003520 con 0x7f7a580b2fe0 2023-09-23T03:23:55.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.614+0000 7f7a577fe640 1 -- 172.21.15.154:0/4159904829 >> v2:172.21.15.190:3300/0 conn(0x7f7a580cf130 msgr2=0x7f7a580ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:55.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.614+0000 7f7a577fe640 1 --2- 172.21.15.154:0/4159904829 >> v2:172.21.15.190:3300/0 conn(0x7f7a580cf130 0x7f7a580ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:55.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.614+0000 7f7a577fe640 1 -- 172.21.15.154:0/4159904829 >> v1:172.21.15.190:6789/0 conn(0x7f7a580ada30 legacy=0x7f7a580cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:55.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.614+0000 7f7a577fe640 1 -- 172.21.15.154:0/4159904829 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5806c110 con 0x7f7a580b2fe0 2023-09-23T03:23:55.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.614+0000 7f7a577fe640 1 -- 172.21.15.154:0/4159904829 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f7a50001b80 con 0x7f7a580b2fe0 2023-09-23T03:23:55.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.614+0000 7f7a577fe640 1 -- 172.21.15.154:0/4159904829 >> v1:172.21.15.154:6789/0 conn(0x7f7a580b2fe0 legacy=0x7f7a5815d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:55.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.614+0000 7f7a577fe640 1 --2- 172.21.15.154:0/4159904829 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a44003970 0x7f7a44003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:55.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.614+0000 7f7a577fe640 1 --2- 172.21.15.154:0/4159904829 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a440047c0 0x7f7a44004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:55.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.614+0000 7f7a577fe640 1 --2- 172.21.15.154:0/4159904829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a44005180 0x7f7a4400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:55.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.614+0000 7f7a577fe640 1 -- 172.21.15.154:0/4159904829 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a580ade10 con 0x7f7a440047c0 2023-09-23T03:23:55.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.614+0000 7f7a577fe640 1 -- 172.21.15.154:0/4159904829 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4400c020 con 0x7f7a44005180 2023-09-23T03:23:55.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.614+0000 7f7a577fe640 1 -- 172.21.15.154:0/4159904829 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4400e3f0 con 0x7f7a44003970 2023-09-23T03:23:55.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.615+0000 7f7a4ffff640 1 --2- 172.21.15.154:0/4159904829 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a440047c0 0x7f7a44004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:55.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.615+0000 7f7a4ffff640 1 -- 172.21.15.154:0/4159904829 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a44003970 msgr2=0x7f7a44003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:55.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.615+0000 7f7a57fff640 1 --2- 172.21.15.154:0/4159904829 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a44003970 0x7f7a44003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:55.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.615+0000 7f7a5cbfe640 1 --2- 172.21.15.154:0/4159904829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a44005180 0x7f7a4400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:55.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.615+0000 7f7a4ffff640 1 --2- 172.21.15.154:0/4159904829 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a44003970 0x7f7a44003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:55.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.615+0000 7f7a4ffff640 1 -- 172.21.15.154:0/4159904829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a44005180 msgr2=0x7f7a4400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:55.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.615+0000 7f7a4ffff640 1 --2- 172.21.15.154:0/4159904829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a44005180 0x7f7a4400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:55.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.615+0000 7f7a4ffff640 1 -- 172.21.15.154:0/4159904829 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a44003520 con 0x7f7a440047c0 2023-09-23T03:23:55.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.615+0000 7f7a5cbfe640 1 -- 172.21.15.154:0/4159904829 reap_dead start 2023-09-23T03:23:55.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.615+0000 7f7a57fff640 1 --2- 172.21.15.154:0/4159904829 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a44003970 0x7f7a44003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:23:55.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.616+0000 7f7a4ffff640 1 --2- 172.21.15.154:0/4159904829 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a440047c0 0x7f7a44004c40 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f7a5806e070 tx=0x7f7a440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19b3401627ce25e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:55.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.616+0000 7f7a577fe640 1 -- 172.21.15.154:0/4159904829 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4002c660 con 0x7f7a440047c0 2023-09-23T03:23:55.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.617+0000 7f7a577fe640 1 -- 172.21.15.154:0/4159904829 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4002c7c0 con 0x7f7a440047c0 2023-09-23T03:23:55.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.617+0000 7f7a577fe640 1 -- 172.21.15.154:0/4159904829 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4002cb50 con 0x7f7a440047c0 2023-09-23T03:23:55.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.617+0000 7f7a5e661640 1 -- 172.21.15.154:0/4159904829 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a440047c0 msgr2=0x7f7a44004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:55.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.617+0000 7f7a5e661640 1 --2- 172.21.15.154:0/4159904829 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a440047c0 0x7f7a44004c40 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f7a5806e070 tx=0x7f7a440057f0 comp rx=0 tx=0).stop 2023-09-23T03:23:55.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.617+0000 7f7a5e661640 1 -- 172.21.15.154:0/4159904829 shutdown_connections 2023-09-23T03:23:55.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.617+0000 7f7a5e661640 1 --2- 172.21.15.154:0/4159904829 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a440047c0 0x7f7a44004c40 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:55.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.617+0000 7f7a5e661640 1 -- 172.21.15.154:0/4159904829 >> 172.21.15.154:0/4159904829 conn(0x7f7a5806a0f0 msgr2=0x7f7a5806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:55.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.617+0000 7f7a5e661640 1 -- 172.21.15.154:0/4159904829 shutdown_connections 2023-09-23T03:23:55.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.617+0000 7f7a5e661640 1 -- 172.21.15.154:0/4159904829 wait complete. 2023-09-23T03:23:55.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.619+0000 7f7a5e661640 1 Processor -- start 2023-09-23T03:23:55.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.619+0000 7f7a5e661640 1 -- start start 2023-09-23T03:23:55.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.620+0000 7f7a5e661640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a440047c0 0x7f7a581f1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:55.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.620+0000 7f7a5e661640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a58099a10 0x7f7a581f5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:55.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.620+0000 7f7a5e661640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a581f5ca0 0x7f7a5815b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:55.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.620+0000 7f7a5e661640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a581f6150 con 0x7f7a440047c0 2023-09-23T03:23:55.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.620+0000 7f7a5e661640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a580c2560 con 0x7f7a581f5ca0 2023-09-23T03:23:55.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.620+0000 7f7a5e661640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a580c2840 con 0x7f7a58099a10 2023-09-23T03:23:55.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.620+0000 7f7a57fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a440047c0 0x7f7a581f1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:55.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.620+0000 7f7a57fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a440047c0 0x7f7a581f1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:51434/0 (socket says 172.21.15.154:51434) 2023-09-23T03:23:55.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.620+0000 7f7a57fff640 1 -- 172.21.15.154:0/3328504598 learned_addr learned my addr 172.21.15.154:0/3328504598 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:23:55.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.621+0000 7f7a57fff640 1 -- 172.21.15.154:0/3328504598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a58099a10 msgr2=0x7f7a581f5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:55.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.621+0000 7f7a57fff640 1 --2- 172.21.15.154:0/3328504598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a58099a10 0x7f7a581f5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:55.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.621+0000 7f7a57fff640 1 -- 172.21.15.154:0/3328504598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a581f5ca0 msgr2=0x7f7a5815b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:55.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.621+0000 7f7a57fff640 1 --2- 172.21.15.154:0/3328504598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a581f5ca0 0x7f7a5815b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:55.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.621+0000 7f7a57fff640 1 -- 172.21.15.154:0/3328504598 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a580c2b20 con 0x7f7a440047c0 2023-09-23T03:23:55.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.621+0000 7f7a57fff640 1 --2- 172.21.15.154:0/3328504598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a440047c0 0x7f7a581f1960 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f7a4800d470 tx=0x7f7a4800d930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:55.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.621+0000 7f7a557fa640 1 -- 172.21.15.154:0/3328504598 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a48005480 con 0x7f7a440047c0 2023-09-23T03:23:55.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.621+0000 7f7a557fa640 1 -- 172.21.15.154:0/3328504598 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a48003660 con 0x7f7a440047c0 2023-09-23T03:23:55.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.621+0000 7f7a5e661640 1 -- 172.21.15.154:0/3328504598 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a580c2e00 con 0x7f7a440047c0 2023-09-23T03:23:55.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.621+0000 7f7a557fa640 1 -- 172.21.15.154:0/3328504598 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a480039f0 con 0x7f7a440047c0 2023-09-23T03:23:55.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.622+0000 7f7a5e661640 1 -- 172.21.15.154:0/3328504598 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a580c21a0 con 0x7f7a440047c0 2023-09-23T03:23:55.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.623+0000 7f7a5e661640 1 -- 172.21.15.154:0/3328504598 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a580c2330 con 0x7f7a440047c0 2023-09-23T03:23:55.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.624+0000 7f7a557fa640 1 -- 172.21.15.154:0/3328504598 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f7a48003b50 con 0x7f7a440047c0 2023-09-23T03:23:55.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.624+0000 7f7a557fa640 1 --2- 172.21.15.154:0/3328504598 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7a3806af60 0x7f7a3806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:55.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.625+0000 7f7a557fa640 1 -- 172.21.15.154:0/3328504598 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f7a48089930 con 0x7f7a440047c0 2023-09-23T03:23:55.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.625+0000 7f7a4ffff640 1 --2- 172.21.15.154:0/3328504598 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7a3806af60 0x7f7a3806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:55.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.625+0000 7f7a4ffff640 1 --2- 172.21.15.154:0/3328504598 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7a3806af60 0x7f7a3806d420 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7a40036000 tx=0x7f7a4002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:55.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.629+0000 7f7a557fa640 1 -- 172.21.15.154:0/3328504598 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f7a48054590 con 0x7f7a440047c0 2023-09-23T03:23:55.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.803+0000 7f7a5e661640 1 -- 172.21.15.154:0/3328504598 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a58073eb0 con 0x7f7a440047c0 2023-09-23T03:23:55.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.807+0000 7f7a557fa640 1 -- 172.21.15.154:0/3328504598 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+18987 (secure 0 0 0) 0x7f7a48057bb0 con 0x7f7a440047c0 2023-09-23T03:23:55.808 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:23:55.808 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":71,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:21:39.274806+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:19:21.772037+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-09-23T03:19:22.960250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data_extra","create_time":"2023-09-23T03:19:38.870843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6837/4129414148":"2023-09-24T03:21:38.278599+0000","172.21.15.154:6835/4129414148":"2023-09-24T03:21:38.278599+0000","172.21.15.154:6837/2976977273":"2023-09-24T03:21:31.069171+0000","172.21.15.154:6835/2976977273":"2023-09-24T03:21:31.069171+0000","172.21.15.154:6837/2183023203":"2023-09-24T03:21:29.032871+0000","172.21.15.154:6835/2183023203":"2023-09-24T03:21:29.032871+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:23:55.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.810+0000 7f7a5e661640 1 -- 172.21.15.154:0/3328504598 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7a3806af60 msgr2=0x7f7a3806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:55.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.810+0000 7f7a5e661640 1 --2- 172.21.15.154:0/3328504598 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7a3806af60 0x7f7a3806d420 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7a40036000 tx=0x7f7a4002e000 comp rx=0 tx=0).stop 2023-09-23T03:23:55.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.810+0000 7f7a5e661640 1 -- 172.21.15.154:0/3328504598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a440047c0 msgr2=0x7f7a581f1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:55.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.810+0000 7f7a5e661640 1 --2- 172.21.15.154:0/3328504598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a440047c0 0x7f7a581f1960 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f7a4800d470 tx=0x7f7a4800d930 comp rx=0 tx=0).stop 2023-09-23T03:23:55.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.810+0000 7f7a5e661640 1 -- 172.21.15.154:0/3328504598 shutdown_connections 2023-09-23T03:23:55.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.810+0000 7f7a5e661640 1 --2- 172.21.15.154:0/3328504598 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7a3806af60 0x7f7a3806d420 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:55.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.810+0000 7f7a5e661640 1 --2- 172.21.15.154:0/3328504598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a581f5ca0 0x7f7a5815b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:55.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.810+0000 7f7a5e661640 1 --2- 172.21.15.154:0/3328504598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a58099a10 0x7f7a581f5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:55.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.810+0000 7f7a5e661640 1 --2- 172.21.15.154:0/3328504598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a440047c0 0x7f7a581f1960 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:55.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.810+0000 7f7a5e661640 1 -- 172.21.15.154:0/3328504598 >> 172.21.15.154:0/3328504598 conn(0x7f7a5806a0f0 msgr2=0x7f7a580af390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:55.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.811+0000 7f7a5e661640 1 -- 172.21.15.154:0/3328504598 shutdown_connections 2023-09-23T03:23:55.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:55.811+0000 7f7a5e661640 1 -- 172.21.15.154:0/3328504598 wait complete. 2023-09-23T03:23:55.828 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-09-23T03:23:56.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.043+0000 7fa44074c640 1 Processor -- start 2023-09-23T03:23:56.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.043+0000 7fa44074c640 1 -- start start 2023-09-23T03:23:56.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.045+0000 7fa44074c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa43815ad20 0x7fa4380cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:56.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.045+0000 7fa44074c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4380cd160 0x7fa4380cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:56.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.045+0000 7fa44074c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4380cdb00 con 0x7fa4380b21d0 2023-09-23T03:23:56.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.045+0000 7fa44074c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa43815f880 con 0x7fa43815ad20 2023-09-23T03:23:56.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.045+0000 7fa44074c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa43815ed00 con 0x7fa4380cd160 2023-09-23T03:23:56.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.045+0000 7fa43e4e8640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa43815ad20 0x7fa4380cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:56.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.045+0000 7fa43ece9640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fa4380b21d0 0x7fa4380b2660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46030/0 (socket says 172.21.15.154:46030) 2023-09-23T03:23:56.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.045+0000 7fa43ece9640 1 -- 172.21.15.154:0/2558635803 learned_addr learned my addr 172.21.15.154:0/2558635803 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:23:56.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.046+0000 7fa43e4e8640 1 -- 172.21.15.154:0/2558635803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4380cd160 msgr2=0x7fa4380cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:56.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.046+0000 7fa43e4e8640 1 --2- 172.21.15.154:0/2558635803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4380cd160 0x7fa4380cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:56.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.046+0000 7fa43e4e8640 1 -- 172.21.15.154:0/2558635803 >> v1:172.21.15.154:6789/0 conn(0x7fa4380b21d0 legacy=0x7fa4380b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:56.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.046+0000 7fa43e4e8640 1 -- 172.21.15.154:0/2558635803 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa43815efe0 con 0x7fa43815ad20 2023-09-23T03:23:56.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.046+0000 7fa43d4e6640 1 -- 172.21.15.154:0/2558635803 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2412770418 0 0) 0x7fa4380cdb00 con 0x7fa4380b21d0 2023-09-23T03:23:56.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.047+0000 7fa43e4e8640 1 --2- 172.21.15.154:0/2558635803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa43815ad20 0x7fa4380cafb0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fa4340097a0 tx=0x7fa43402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf4b4eabd5099321 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:56.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.047+0000 7fa43d4e6640 1 -- 172.21.15.154:0/2558635803 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43402d040 con 0x7fa43815ad20 2023-09-23T03:23:56.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.047+0000 7fa43d4e6640 1 -- 172.21.15.154:0/2558635803 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa434031810 con 0x7fa43815ad20 2023-09-23T03:23:56.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.047+0000 7fa43d4e6640 1 -- 172.21.15.154:0/2558635803 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa434031ba0 con 0x7fa43815ad20 2023-09-23T03:23:56.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.048+0000 7fa44074c640 1 -- 172.21.15.154:0/2558635803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa43815ad20 msgr2=0x7fa4380cafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:56.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.048+0000 7fa44074c640 1 --2- 172.21.15.154:0/2558635803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa43815ad20 0x7fa4380cafb0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fa4340097a0 tx=0x7fa43402b8b0 comp rx=0 tx=0).stop 2023-09-23T03:23:56.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.048+0000 7fa44074c640 1 -- 172.21.15.154:0/2558635803 shutdown_connections 2023-09-23T03:23:56.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.048+0000 7fa44074c640 1 --2- 172.21.15.154:0/2558635803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4380cd160 0x7fa4380cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:56.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.048+0000 7fa44074c640 1 --2- 172.21.15.154:0/2558635803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa43815ad20 0x7fa4380cafb0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:56.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.048+0000 7fa44074c640 1 -- 172.21.15.154:0/2558635803 >> 172.21.15.154:0/2558635803 conn(0x7fa43806fc60 msgr2=0x7fa438070040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:56.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.048+0000 7fa44074c640 1 -- 172.21.15.154:0/2558635803 shutdown_connections 2023-09-23T03:23:56.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.048+0000 7fa44074c640 1 -- 172.21.15.154:0/2558635803 wait complete. 2023-09-23T03:23:56.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.050+0000 7fa44074c640 1 Processor -- start 2023-09-23T03:23:56.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.050+0000 7fa44074c640 1 -- start start 2023-09-23T03:23:56.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.051+0000 7fa44074c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4380b21d0 0x7fa438152c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:56.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.051+0000 7fa44074c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4380cd160 0x7fa4380c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:56.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.051+0000 7fa44074c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa43815ad20 0x7fa4380c4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:56.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.051+0000 7fa44074c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4380c47c0 con 0x7fa43815ad20 2023-09-23T03:23:56.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.051+0000 7fa44074c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4380c8f80 con 0x7fa4380cd160 2023-09-23T03:23:56.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.051+0000 7fa44074c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4380b4680 con 0x7fa4380b21d0 2023-09-23T03:23:56.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.051+0000 7fa43ece9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa43815ad20 0x7fa4380c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:56.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.051+0000 7fa43ece9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa43815ad20 0x7fa4380c4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:51464/0 (socket says 172.21.15.154:51464) 2023-09-23T03:23:56.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.051+0000 7fa43ece9640 1 -- 172.21.15.154:0/35058347 learned_addr learned my addr 172.21.15.154:0/35058347 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:23:56.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.051+0000 7fa43ece9640 1 -- 172.21.15.154:0/35058347 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4380b21d0 msgr2=0x7fa438152c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:56.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.051+0000 7fa43ece9640 1 --2- 172.21.15.154:0/35058347 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4380b21d0 0x7fa438152c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:56.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.051+0000 7fa43ece9640 1 -- 172.21.15.154:0/35058347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4380cd160 msgr2=0x7fa4380c7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:56.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.051+0000 7fa43ece9640 1 --2- 172.21.15.154:0/35058347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4380cd160 0x7fa4380c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:56.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.051+0000 7fa43ece9640 1 -- 172.21.15.154:0/35058347 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa434009400 con 0x7fa43815ad20 2023-09-23T03:23:56.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.052+0000 7fa43ece9640 1 --2- 172.21.15.154:0/35058347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa43815ad20 0x7fa4380c4280 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fa43000c7d0 tx=0x7fa43000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:56.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.052+0000 7fa42effd640 1 -- 172.21.15.154:0/35058347 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43000a640 con 0x7fa43815ad20 2023-09-23T03:23:56.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.052+0000 7fa42effd640 1 -- 172.21.15.154:0/35058347 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa43000a7a0 con 0x7fa43815ad20 2023-09-23T03:23:56.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.052+0000 7fa42effd640 1 -- 172.21.15.154:0/35058347 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43000ab90 con 0x7fa43815ad20 2023-09-23T03:23:56.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.052+0000 7fa44074c640 1 -- 172.21.15.154:0/35058347 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4380b4960 con 0x7fa43815ad20 2023-09-23T03:23:56.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.052+0000 7fa44074c640 1 -- 172.21.15.154:0/35058347 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4380b4f20 con 0x7fa43815ad20 2023-09-23T03:23:56.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.053+0000 7fa44074c640 1 -- 172.21.15.154:0/35058347 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4380b5140 con 0x7fa43815ad20 2023-09-23T03:23:56.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.054+0000 7fa42effd640 1 -- 172.21.15.154:0/35058347 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fa43000acf0 con 0x7fa43815ad20 2023-09-23T03:23:56.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.055+0000 7fa42effd640 1 --2- 172.21.15.154:0/35058347 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa41c06af30 0x7fa41c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:56.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.055+0000 7fa42effd640 1 -- 172.21.15.154:0/35058347 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fa430089100 con 0x7fa43815ad20 2023-09-23T03:23:56.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.055+0000 7fa43e4e8640 1 --2- 172.21.15.154:0/35058347 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa41c06af30 0x7fa41c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:56.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.056+0000 7fa43e4e8640 1 --2- 172.21.15.154:0/35058347 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa41c06af30 0x7fa41c06d3f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa434036000 tx=0x7fa434030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:56.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.059+0000 7fa42effd640 1 -- 172.21.15.154:0/35058347 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fa430053d40 con 0x7fa43815ad20 2023-09-23T03:23:56.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.282+0000 7fa44074c640 1 -- 172.21.15.154:0/35058347 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa438073eb0 con 0x7fa43815ad20 2023-09-23T03:23:56.284 INFO:tasks.ceph.mon.a.smithi154.stderr:2023-09-23T03:23:56.284+0000 7f3df35a6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-23T03:23:56.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.287+0000 7fa42effd640 1 -- 172.21.15.154:0/35058347 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v55) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa430018090 con 0x7fa43815ad20 2023-09-23T03:23:56.288 INFO:teuthology.orchestra.run.smithi154.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-09-23T03:23:56.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.290+0000 7fa44074c640 1 -- 172.21.15.154:0/35058347 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa41c06af30 msgr2=0x7fa41c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:56.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.290+0000 7fa44074c640 1 --2- 172.21.15.154:0/35058347 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa41c06af30 0x7fa41c06d3f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa434036000 tx=0x7fa434030000 comp rx=0 tx=0).stop 2023-09-23T03:23:56.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.290+0000 7fa44074c640 1 -- 172.21.15.154:0/35058347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa43815ad20 msgr2=0x7fa4380c4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:56.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.290+0000 7fa44074c640 1 --2- 172.21.15.154:0/35058347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa43815ad20 0x7fa4380c4280 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fa43000c7d0 tx=0x7fa43000cc90 comp rx=0 tx=0).stop 2023-09-23T03:23:56.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.291+0000 7fa44074c640 1 -- 172.21.15.154:0/35058347 shutdown_connections 2023-09-23T03:23:56.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.291+0000 7fa44074c640 1 --2- 172.21.15.154:0/35058347 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa41c06af30 0x7fa41c06d3f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:56.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.291+0000 7fa44074c640 1 --2- 172.21.15.154:0/35058347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa43815ad20 0x7fa4380c4280 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:56.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.291+0000 7fa44074c640 1 --2- 172.21.15.154:0/35058347 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa4380cd160 0x7fa4380c7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:56.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.291+0000 7fa44074c640 1 --2- 172.21.15.154:0/35058347 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa4380b21d0 0x7fa438152c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:56.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.291+0000 7fa44074c640 1 -- 172.21.15.154:0/35058347 >> 172.21.15.154:0/35058347 conn(0x7fa43806fc60 msgr2=0x7fa438159f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:56.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.291+0000 7fa44074c640 1 -- 172.21.15.154:0/35058347 shutdown_connections 2023-09-23T03:23:56.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.291+0000 7fa44074c640 1 -- 172.21.15.154:0/35058347 wait complete. 2023-09-23T03:23:56.310 INFO:tasks.ceph.mds.a.smithi154.stderr: -9543> 2023-09-23T03:21:45.267+0000 7fabd8110640 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2023-09-23T03:23:56.310 INFO:tasks.ceph.mds.a.smithi154.stderr: -9383> 2023-09-23T03:21:45.269+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /subdir/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x562c45591b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000000:[<0x1/subdir v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-09-23T03:23:56.311 INFO:tasks.ceph.mds.a.smithi154.stderr: -9005> 2023-09-23T03:21:45.272+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f8 [2,head] /subdir/3 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c4566e100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001f8:[<0x10000000000/3 v40>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x100000001f8:[<0x10000000000/3 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:23:56.311 INFO:tasks.ceph.mds.a.smithi154.stderr: -8976> 2023-09-23T03:21:45.272+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fd [2,head] /subdir/8 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c4566c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001fd:[<0x10000000000/8 v80>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x100000001fd:[<0x10000000000/8 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:23:56.311 INFO:tasks.ceph.mds.a.smithi154.stderr: -8952> 2023-09-23T03:21:45.273+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fe [2,head] /subdir/9 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c4566c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001fe:[<0x10000000000/9 v88>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x100000001fe:[<0x10000000000/9 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:23:56.311 INFO:tasks.ceph.mds.a.smithi154.stderr: -8928> 2023-09-23T03:21:45.273+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000204 [2,head] /subdir/15 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c45676680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000204:[<0x10000000000/15 v138>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x10000000204:[<0x10000000000/15 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:23:56.313 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-09-23T03:23:56.315 INFO:tasks.ceph.mds.a.smithi154.stderr: -8719> 2023-09-23T03:21:45.274+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000201 [2,head] /subdir/12 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c45677700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000201:[<0x10000000000/12 v114>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x10000000201:[<0x10000000000/12 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:23:56.315 INFO:tasks.ceph.mds.a.smithi154.stderr: -8697> 2023-09-23T03:21:45.274+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000209 [2,head] /subdir/20 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c45674580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000209:[<0x10000000000/20 v178>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x10000000209:[<0x10000000000/20 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:23:56.315 INFO:tasks.ceph.mds.a.smithi154.stderr: -8674> 2023-09-23T03:21:45.275+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000208 [2,head] /subdir/19 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c45675080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000208:[<0x10000000000/19 v170>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x10000000208:[<0x10000000000/19 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:23:56.315 INFO:tasks.ceph.mds.a.smithi154.stderr: -8652> 2023-09-23T03:21:45.275+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020a [2,head] /subdir/21 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c45674000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x1000000020a:[<0x10000000000/21 v186>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x1000000020a:[<0x10000000000/21 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:23:56.315 INFO:tasks.ceph.mds.a.smithi154.stderr: -8629> 2023-09-23T03:21:45.275+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000200 [2,head] /subdir/11 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c4567c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000200:[<0x10000000000/11 v104>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x10000000200:[<0x10000000000/11 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:23:56.315 INFO:tasks.ceph.mds.a.smithi154.stderr: -8605> 2023-09-23T03:21:45.276+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001ff [2,head] /subdir/10 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c4567c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001ff:[<0x10000000000/10 v96>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x100000001ff:[<0x10000000000/10 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:23:56.315 INFO:tasks.ceph.mds.a.smithi154.stderr: -8582> 2023-09-23T03:21:45.276+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f7 [2,head] /subdir/2 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c4566e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001f7:[<0x10000000000/2 v32>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x100000001f7:[<0x10000000000/2 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:23:56.315 INFO:tasks.ceph.mds.a.smithi154.stderr: -8560> 2023-09-23T03:21:45.276+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f9 [2,head] /subdir/4 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c4566db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001f9:[<0x10000000000/4 v48>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x100000001f9:[<0x10000000000/4 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:23:56.315 INFO:tasks.ceph.mds.a.smithi154.stderr: -8537> 2023-09-23T03:21:45.276+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f6 [2,head] /subdir/1 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c45674b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001f6:[<0x10000000000/1 v24>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x100000001f6:[<0x10000000000/1 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:23:56.316 INFO:tasks.ceph.mds.a.smithi154.stderr: -8514> 2023-09-23T03:21:45.276+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020d [2,head] /subdir/24 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c4566ec00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x1000000020d:[<0x10000000000/24 v202>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x1000000020d:[<0x10000000000/24 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:23:56.316 INFO:tasks.ceph.mds.a.smithi154.stderr: -8490> 2023-09-23T03:21:45.277+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/0 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c4567cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000001:[<0x10000000000/0 v16>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x10000000001:[<0x10000000000/0 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:23:56.316 INFO:tasks.ceph.mds.a.smithi154.stderr: -8467> 2023-09-23T03:21:45.277+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020b [2,head] /subdir/22 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c4566f700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x1000000020b:[<0x10000000000/22 v194>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x1000000020b:[<0x10000000000/22 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:23:56.316 INFO:tasks.ceph.mds.a.smithi154.stderr: -8445> 2023-09-23T03:21:45.277+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000205 [2,head] /subdir/16 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c45676100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000205:[<0x10000000000/16 v146>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x10000000205:[<0x10000000000/16 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:23:56.316 INFO:tasks.ceph.mds.a.smithi154.stderr: -8422> 2023-09-23T03:21:45.278+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fa [2,head] /subdir/5 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c4566d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001fa:[<0x10000000000/5 v56>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x100000001fa:[<0x10000000000/5 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:23:56.316 INFO:tasks.ceph.mds.a.smithi154.stderr: -8399> 2023-09-23T03:21:45.278+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000202 [2,head] /subdir/13 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c45677180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000202:[<0x10000000000/13 v122>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x10000000202:[<0x10000000000/13 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:23:56.316 INFO:tasks.ceph.mds.a.smithi154.stderr: -8376> 2023-09-23T03:21:45.278+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020c [2,head] /subdir/23 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c4566f180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x1000000020c:[<0x10000000000/23 v200>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x1000000020c:[<0x10000000000/23 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:23:56.316 INFO:tasks.ceph.mds.a.smithi154.stderr: -8353> 2023-09-23T03:21:45.278+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000203 [2,head] /subdir/14 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c45676c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000203:[<0x10000000000/14 v130>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x10000000203:[<0x10000000000/14 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:23:56.316 INFO:tasks.ceph.mds.a.smithi154.stderr: -8329> 2023-09-23T03:21:45.279+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000207 [2,head] /subdir/18 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c45675600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000207:[<0x10000000000/18 v162>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x10000000207:[<0x10000000000/18 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:23:56.316 INFO:tasks.ceph.mds.a.smithi154.stderr: -8312> 2023-09-23T03:21:45.279+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fb [2,head] /subdir/6 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c4566d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001fb:[<0x10000000000/6 v64>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x100000001fb:[<0x10000000000/6 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:23:56.316 INFO:tasks.ceph.mds.a.smithi154.stderr: -8295> 2023-09-23T03:21:45.279+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fc [2,head] /subdir/7 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c4566cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x100000001fc:[<0x10000000000/7 v72>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x100000001fc:[<0x10000000000/7 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:23:56.317 INFO:tasks.ceph.mds.a.smithi154.stderr: -8278> 2023-09-23T03:21:45.279+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000206 [2,head] /subdir/17 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x562c45675b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(10)0x10000000206:[<0x10000000000/17 v154>,<0x1/subdir v214>]//[9]","memoryvalue":"(10)0x10000000206:[<0x10000000000/17 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:23:56.317 INFO:tasks.ceph.mds.a.smithi154.stderr: -8183> 2023-09-23T03:21:45.280+0000 7fabd8110640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v5 ap=5 snaprealm=0x562c4558a6c0 DIRTYPARENT f(v1 1=0+1) n(v0 rc2023-09-23T03:21:44.986581+0000 1=0+1)/n() (idft sync r=1) (inest sync r=1) (ifile sync->mix r=1) (iversion lock) caps={7002=pLsXs/-@3} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x562c45591600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=0+1)","ondisk_value.rstat":"n(v0 2=0+2)","memory_value.dirstat":"f(v1 1=0+1)","memory_value.rstat":"n(v0 rc2023-09-23T03:21:44.986581+0000 1=0+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-09-23T03:23:56.374 INFO:tasks.ceph.mds.a.smithi154.stdout:starting mds.a at 2023-09-23T03:23:56.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.501+0000 7fb1ff347640 1 Processor -- start 2023-09-23T03:23:56.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.502+0000 7fb1ff347640 1 -- start start 2023-09-23T03:23:56.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.503+0000 7fb1ff347640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb1f815cbf0 0x7fb1f806b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:56.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.503+0000 7fb1ff347640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb1f80b0860 0x7fb1f80b4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:56.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.503+0000 7fb1ff347640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1f80af3c0 con 0x7fb1f80b42f0 2023-09-23T03:23:56.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.503+0000 7fb1ff347640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1f80b5150 con 0x7fb1f815cbf0 2023-09-23T03:23:56.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.503+0000 7fb1ff347640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1f80b4cb0 con 0x7fb1f80b0860 2023-09-23T03:23:56.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.503+0000 7fb1fe345640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb1f815cbf0 0x7fb1f806b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:56.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.503+0000 7fb1fdb44640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fb1f80b42f0 0x7fb1f8160650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46042/0 (socket says 172.21.15.154:46042) 2023-09-23T03:23:56.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.503+0000 7fb1fdb44640 1 -- 172.21.15.154:0/2885195124 learned_addr learned my addr 172.21.15.154:0/2885195124 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:23:56.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.503+0000 7fb1fe345640 1 -- 172.21.15.154:0/2885195124 >> v2:172.21.15.190:3300/0 conn(0x7fb1f80b0860 msgr2=0x7fb1f80b4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:56.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.503+0000 7fb1fe345640 1 --2- 172.21.15.154:0/2885195124 >> v2:172.21.15.190:3300/0 conn(0x7fb1f80b0860 0x7fb1f80b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:56.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.504+0000 7fb1fe345640 1 -- 172.21.15.154:0/2885195124 >> v1:172.21.15.154:6789/0 conn(0x7fb1f80b42f0 legacy=0x7fb1f8160650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:56.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.504+0000 7fb1fd343640 1 -- 172.21.15.154:0/2885195124 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1696826626 0 0) 0x7fb1f80af3c0 con 0x7fb1f80b42f0 2023-09-23T03:23:56.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.504+0000 7fb1fe345640 1 -- 172.21.15.154:0/2885195124 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f80d14f0 con 0x7fb1f815cbf0 2023-09-23T03:23:56.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.505+0000 7fb1fe345640 1 --2- 172.21.15.154:0/2885195124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1f815cbf0 0x7fb1f806b7b0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec009de0 tx=0x7fb1ec02bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a8661fb6f051a2 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:56.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.505+0000 7fb1fd343640 1 -- 172.21.15.154:0/2885195124 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec02ccb0 con 0x7fb1f815cbf0 2023-09-23T03:23:56.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.505+0000 7fb1fd343640 1 -- 172.21.15.154:0/2885195124 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1ec02ce10 con 0x7fb1f815cbf0 2023-09-23T03:23:56.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.505+0000 7fb1fd343640 1 -- 172.21.15.154:0/2885195124 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec034660 con 0x7fb1f815cbf0 2023-09-23T03:23:56.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.505+0000 7fb1ff347640 1 -- 172.21.15.154:0/2885195124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1f815cbf0 msgr2=0x7fb1f806b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:56.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.505+0000 7fb1ff347640 1 --2- 172.21.15.154:0/2885195124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1f815cbf0 0x7fb1f806b7b0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec009de0 tx=0x7fb1ec02bf00 comp rx=0 tx=0).stop 2023-09-23T03:23:56.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.506+0000 7fb1ff347640 1 -- 172.21.15.154:0/2885195124 shutdown_connections 2023-09-23T03:23:56.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.506+0000 7fb1ff347640 1 --2- 172.21.15.154:0/2885195124 >> v2:172.21.15.190:3300/0 conn(0x7fb1f80b0860 0x7fb1f80b4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:56.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.506+0000 7fb1ff347640 1 --2- 172.21.15.154:0/2885195124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1f815cbf0 0x7fb1f806b7b0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:56.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.506+0000 7fb1ff347640 1 -- 172.21.15.154:0/2885195124 >> 172.21.15.154:0/2885195124 conn(0x7fb1f808c220 msgr2=0x7fb1f809bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:56.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.506+0000 7fb1ff347640 1 -- 172.21.15.154:0/2885195124 shutdown_connections 2023-09-23T03:23:56.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.506+0000 7fb1ff347640 1 -- 172.21.15.154:0/2885195124 wait complete. 2023-09-23T03:23:56.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.508+0000 7fb1ff347640 1 Processor -- start 2023-09-23T03:23:56.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.508+0000 7fb1ff347640 1 -- start start 2023-09-23T03:23:56.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.509+0000 7fb1ff347640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1f80b0860 0x7fb1f8159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:56.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.509+0000 7fb1ff347640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1f80b42f0 0x7fb1f806b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:56.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.509+0000 7fb1ff347640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1f815cbf0 0x7fb1f806b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:56.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.509+0000 7fb1ff347640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f80b7df0 con 0x7fb1f80b0860 2023-09-23T03:23:56.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.509+0000 7fb1ff347640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f8159e70 con 0x7fb1f815cbf0 2023-09-23T03:23:56.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.509+0000 7fb1ff347640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f80b77e0 con 0x7fb1f80b42f0 2023-09-23T03:23:56.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.509+0000 7fb1fe345640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1f80b0860 0x7fb1f8159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:56.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.509+0000 7fb1fe345640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1f80b0860 0x7fb1f8159930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:51504/0 (socket says 172.21.15.154:51504) 2023-09-23T03:23:56.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.509+0000 7fb1fe345640 1 -- 172.21.15.154:0/2000335946 learned_addr learned my addr 172.21.15.154:0/2000335946 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:23:56.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.510+0000 7fb1fe345640 1 -- 172.21.15.154:0/2000335946 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1f80b42f0 msgr2=0x7fb1f806b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:56.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.510+0000 7fb1fe345640 1 --2- 172.21.15.154:0/2000335946 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1f80b42f0 0x7fb1f806b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:56.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.510+0000 7fb1fe345640 1 -- 172.21.15.154:0/2000335946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1f815cbf0 msgr2=0x7fb1f806b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:56.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.510+0000 7fb1fe345640 1 --2- 172.21.15.154:0/2000335946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1f815cbf0 0x7fb1f806b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:56.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.510+0000 7fb1fe345640 1 -- 172.21.15.154:0/2000335946 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1ec009a70 con 0x7fb1f80b0860 2023-09-23T03:23:56.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.510+0000 7fb1fe345640 1 --2- 172.21.15.154:0/2000335946 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1f80b0860 0x7fb1f8159930 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec037040 tx=0x7fb1ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:56.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.510+0000 7fb1f6ffd640 1 -- 172.21.15.154:0/2000335946 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec040650 con 0x7fb1f80b0860 2023-09-23T03:23:56.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.510+0000 7fb1f6ffd640 1 -- 172.21.15.154:0/2000335946 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1ec0407d0 con 0x7fb1f80b0860 2023-09-23T03:23:56.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.510+0000 7fb1f6ffd640 1 -- 172.21.15.154:0/2000335946 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec040b80 con 0x7fb1f80b0860 2023-09-23T03:23:56.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.510+0000 7fb1ff347640 1 -- 172.21.15.154:0/2000335946 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1f80a0200 con 0x7fb1f80b0860 2023-09-23T03:23:56.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.510+0000 7fb1ff347640 1 -- 172.21.15.154:0/2000335946 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1f80a04e0 con 0x7fb1f80b0860 2023-09-23T03:23:56.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.512+0000 7fb1ff347640 1 -- 172.21.15.154:0/2000335946 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1f80b7640 con 0x7fb1f80b0860 2023-09-23T03:23:56.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.513+0000 7fb1f6ffd640 1 -- 172.21.15.154:0/2000335946 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fb1ec040ce0 con 0x7fb1f80b0860 2023-09-23T03:23:56.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.514+0000 7fb1f6ffd640 1 --2- 172.21.15.154:0/2000335946 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb1dc06af30 0x7fb1dc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:56.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.514+0000 7fb1f6ffd640 1 -- 172.21.15.154:0/2000335946 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6777+0+0 (secure 0 0 0) 0x7fb1ec0acb10 con 0x7fb1f80b0860 2023-09-23T03:23:56.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.514+0000 7fb1fdb44640 1 --2- 172.21.15.154:0/2000335946 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb1dc06af30 0x7fb1dc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:56.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.515+0000 7fb1fdb44640 1 --2- 172.21.15.154:0/2000335946 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb1dc06af30 0x7fb1dc06d3f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4007fd0 tx=0x7fb1e4007960 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:56.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.518+0000 7fb1f6ffd640 1 -- 172.21.15.154:0/2000335946 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fb1f80b7640 con 0x7fb1f80b0860 2023-09-23T03:23:56.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:56.741+0000 7fb1ff347640 1 -- 172.21.15.154:0/2000335946 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb1f80b7640 con 0x7fb1f80b0860 2023-09-23T03:23:57.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.319+0000 7fb1f6ffd640 1 -- 172.21.15.154:0/2000335946 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v56) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb1f80b7640 con 0x7fb1f80b0860 2023-09-23T03:23:57.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.323+0000 7fb1ff347640 1 -- 172.21.15.154:0/2000335946 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb1dc06af30 msgr2=0x7fb1dc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:57.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.323+0000 7fb1ff347640 1 --2- 172.21.15.154:0/2000335946 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb1dc06af30 0x7fb1dc06d3f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4007fd0 tx=0x7fb1e4007960 comp rx=0 tx=0).stop 2023-09-23T03:23:57.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.323+0000 7fb1ff347640 1 -- 172.21.15.154:0/2000335946 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1f80b0860 msgr2=0x7fb1f8159930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:57.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.323+0000 7fb1ff347640 1 --2- 172.21.15.154:0/2000335946 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1f80b0860 0x7fb1f8159930 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec037040 tx=0x7fb1ec004a40 comp rx=0 tx=0).stop 2023-09-23T03:23:57.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.324+0000 7fb1ff347640 1 -- 172.21.15.154:0/2000335946 shutdown_connections 2023-09-23T03:23:57.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.324+0000 7fb1ff347640 1 --2- 172.21.15.154:0/2000335946 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb1dc06af30 0x7fb1dc06d3f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:57.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.324+0000 7fb1ff347640 1 --2- 172.21.15.154:0/2000335946 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb1f815cbf0 0x7fb1f806b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:57.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.324+0000 7fb1ff347640 1 --2- 172.21.15.154:0/2000335946 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb1f80b42f0 0x7fb1f806b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:57.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.324+0000 7fb1ff347640 1 --2- 172.21.15.154:0/2000335946 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb1f80b0860 0x7fb1f8159930 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:57.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.324+0000 7fb1ff347640 1 -- 172.21.15.154:0/2000335946 >> 172.21.15.154:0/2000335946 conn(0x7fb1f808c220 msgr2=0x7fb1f80cf2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:57.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.324+0000 7fb1ff347640 1 -- 172.21.15.154:0/2000335946 shutdown_connections 2023-09-23T03:23:57.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.324+0000 7fb1ff347640 1 -- 172.21.15.154:0/2000335946 wait complete. 2023-09-23T03:23:57.345 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-09-23T03:23:57.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.559+0000 7efcfc690640 1 Processor -- start 2023-09-23T03:23:57.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.559+0000 7efcfc690640 1 -- start start 2023-09-23T03:23:57.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.561+0000 7efcfc690640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efcf40cb370 0x7efcf415d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:57.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.561+0000 7efcfc690640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efcf415dae0 0x7efcf40cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:57.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.561+0000 7efcfc690640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcf40c9cf0 con 0x7efcf4157300 2023-09-23T03:23:57.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.561+0000 7efcfc690640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcf41617c0 con 0x7efcf40cb370 2023-09-23T03:23:57.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.561+0000 7efcfc690640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf4160a00 con 0x7efcf415dae0 2023-09-23T03:23:57.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.561+0000 7efcfa42c640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7efcf4157300 0x7efcf40cac60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46056/0 (socket says 172.21.15.154:46056) 2023-09-23T03:23:57.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.561+0000 7efcfa42c640 1 -- 172.21.15.154:0/969816761 learned_addr learned my addr 172.21.15.154:0/969816761 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:23:57.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.561+0000 7efcf942a640 1 -- 172.21.15.154:0/969816761 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2967588096 0 0) 0x7efcf40c9cf0 con 0x7efcf4157300 2023-09-23T03:23:57.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.561+0000 7efcf942a640 1 -- 172.21.15.154:0/969816761 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efce0003520 con 0x7efcf4157300 2023-09-23T03:23:57.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.562+0000 7efcf9c2b640 1 --2- 172.21.15.154:0/969816761 >> v2:172.21.15.190:3300/0 conn(0x7efcf40cb370 0x7efcf415d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:57.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.562+0000 7efcf942a640 1 -- 172.21.15.154:0/969816761 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1828239601 0 0) 0x7efce0003520 con 0x7efcf4157300 2023-09-23T03:23:57.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.562+0000 7efcf942a640 1 -- 172.21.15.154:0/969816761 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efcf415dae0 msgr2=0x7efcf40cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:57.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.562+0000 7efcfac2d640 1 --2- 172.21.15.154:0/969816761 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efcf415dae0 0x7efcf40cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:57.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.562+0000 7efcf942a640 1 --2- 172.21.15.154:0/969816761 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efcf415dae0 0x7efcf40cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:57.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.562+0000 7efcf942a640 1 -- 172.21.15.154:0/969816761 >> v2:172.21.15.190:3300/0 conn(0x7efcf40cb370 msgr2=0x7efcf415d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:57.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.562+0000 7efcf942a640 1 --2- 172.21.15.154:0/969816761 >> v2:172.21.15.190:3300/0 conn(0x7efcf40cb370 0x7efcf415d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:57.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.562+0000 7efcf942a640 1 -- 172.21.15.154:0/969816761 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf4160ce0 con 0x7efcf4157300 2023-09-23T03:23:57.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.562+0000 7efcf942a640 1 -- 172.21.15.154:0/969816761 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7efcf0001670 con 0x7efcf4157300 2023-09-23T03:23:57.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.562+0000 7efcf942a640 1 -- 172.21.15.154:0/969816761 >> v1:172.21.15.154:6789/0 conn(0x7efcf4157300 legacy=0x7efcf40cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:57.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.562+0000 7efcf942a640 1 --2- 172.21.15.154:0/969816761 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efce0003970 0x7efce0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:57.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.562+0000 7efcf942a640 1 --2- 172.21.15.154:0/969816761 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efcf415dae0 0x7efce00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:57.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.562+0000 7efcf942a640 1 --2- 172.21.15.154:0/969816761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efce0004d50 0x7efce00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:57.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.562+0000 7efcf942a640 1 -- 172.21.15.154:0/969816761 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf41617c0 con 0x7efce0003970 2023-09-23T03:23:57.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.562+0000 7efcf942a640 1 -- 172.21.15.154:0/969816761 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf4160a00 con 0x7efce0004d50 2023-09-23T03:23:57.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.562+0000 7efcf942a640 1 -- 172.21.15.154:0/969816761 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce000bc90 con 0x7efcf415dae0 2023-09-23T03:23:57.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.562+0000 7efcfa42c640 1 --2- 172.21.15.154:0/969816761 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efce0003970 0x7efce0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:57.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.563+0000 7efcfa42c640 1 -- 172.21.15.154:0/969816761 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efcf415dae0 msgr2=0x7efce00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:57.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.563+0000 7efcfa42c640 1 --2- 172.21.15.154:0/969816761 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efcf415dae0 0x7efce00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:57.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.563+0000 7efcfa42c640 1 -- 172.21.15.154:0/969816761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efce0004d50 msgr2=0x7efce00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:57.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.563+0000 7efcfa42c640 1 --2- 172.21.15.154:0/969816761 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efce0004d50 0x7efce00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:57.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.563+0000 7efcfa42c640 1 -- 172.21.15.154:0/969816761 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce0003520 con 0x7efce0003970 2023-09-23T03:23:57.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.564+0000 7efcfa42c640 1 --2- 172.21.15.154:0/969816761 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efce0003970 0x7efce0003d90 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7efcf000c240 tx=0x7efce00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aceac6c81f988c5f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:57.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.564+0000 7efcf942a640 1 -- 172.21.15.154:0/969816761 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf002cb00 con 0x7efce0003970 2023-09-23T03:23:57.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.564+0000 7efcf942a640 1 -- 172.21.15.154:0/969816761 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcf002cc60 con 0x7efce0003970 2023-09-23T03:23:57.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.564+0000 7efcf942a640 1 -- 172.21.15.154:0/969816761 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf0005af0 con 0x7efce0003970 2023-09-23T03:23:57.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.564+0000 7efcfc690640 1 -- 172.21.15.154:0/969816761 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efce0003970 msgr2=0x7efce0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:57.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.564+0000 7efcfc690640 1 --2- 172.21.15.154:0/969816761 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efce0003970 0x7efce0003d90 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7efcf000c240 tx=0x7efce00057f0 comp rx=0 tx=0).stop 2023-09-23T03:23:57.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.564+0000 7efcfac2d640 1 -- 172.21.15.154:0/969816761 reap_dead start 2023-09-23T03:23:57.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.564+0000 7efcfc690640 1 -- 172.21.15.154:0/969816761 shutdown_connections 2023-09-23T03:23:57.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.564+0000 7efcfc690640 1 -- 172.21.15.154:0/969816761 >> 172.21.15.154:0/969816761 conn(0x7efcf406a520 msgr2=0x7efcf406bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:57.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.565+0000 7efcfc690640 1 -- 172.21.15.154:0/969816761 shutdown_connections 2023-09-23T03:23:57.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.565+0000 7efcfc690640 1 -- 172.21.15.154:0/969816761 wait complete. 2023-09-23T03:23:57.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.566+0000 7efcfc690640 1 Processor -- start 2023-09-23T03:23:57.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.566+0000 7efcfc690640 1 -- start start 2023-09-23T03:23:57.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.567+0000 7efcfc690640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efce0003970 0x7efcf414c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:57.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.567+0000 7efcfc690640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcf4157b20 0x7efcf40b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:57.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.567+0000 7efcfc690640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcf4145d40 0x7efcf415eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:57.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.567+0000 7efcfc690640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf40b2c00 con 0x7efcf4157b20 2023-09-23T03:23:57.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.567+0000 7efcfc690640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf4148740 con 0x7efcf4145d40 2023-09-23T03:23:57.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.567+0000 7efcfc690640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf4148a20 con 0x7efce0003970 2023-09-23T03:23:57.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.567+0000 7efcf9c2b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcf4157b20 0x7efcf40b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:57.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.568+0000 7efcf9c2b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcf4157b20 0x7efcf40b26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:51512/0 (socket says 172.21.15.154:51512) 2023-09-23T03:23:57.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.568+0000 7efcf9c2b640 1 -- 172.21.15.154:0/3456364867 learned_addr learned my addr 172.21.15.154:0/3456364867 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:23:57.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.568+0000 7efcf9c2b640 1 -- 172.21.15.154:0/3456364867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efce0003970 msgr2=0x7efcf414c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:57.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.568+0000 7efcf9c2b640 1 --2- 172.21.15.154:0/3456364867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efce0003970 0x7efcf414c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:57.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.568+0000 7efcf9c2b640 1 -- 172.21.15.154:0/3456364867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcf4145d40 msgr2=0x7efcf415eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:57.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.568+0000 7efcf9c2b640 1 --2- 172.21.15.154:0/3456364867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcf4145d40 0x7efcf415eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:57.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.568+0000 7efcf9c2b640 1 -- 172.21.15.154:0/3456364867 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf4148d00 con 0x7efcf4157b20 2023-09-23T03:23:57.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.568+0000 7efcf9c2b640 1 --2- 172.21.15.154:0/3456364867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcf4157b20 0x7efcf40b26c0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7efce400d7d0 tx=0x7efce400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:57.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.568+0000 7efceaffd640 1 -- 172.21.15.154:0/3456364867 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce400b6c0 con 0x7efcf4157b20 2023-09-23T03:23:57.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.569+0000 7efceaffd640 1 -- 172.21.15.154:0/3456364867 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce400b860 con 0x7efcf4157b20 2023-09-23T03:23:57.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.569+0000 7efcfc690640 1 -- 172.21.15.154:0/3456364867 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcf4148fe0 con 0x7efcf4157b20 2023-09-23T03:23:57.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.569+0000 7efceaffd640 1 -- 172.21.15.154:0/3456364867 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce400bc10 con 0x7efcf4157b20 2023-09-23T03:23:57.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.569+0000 7efcfc690640 1 -- 172.21.15.154:0/3456364867 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcf41483d0 con 0x7efcf4157b20 2023-09-23T03:23:57.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.570+0000 7efcfc690640 1 -- 172.21.15.154:0/3456364867 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcf404e920 con 0x7efcf4157b20 2023-09-23T03:23:57.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.571+0000 7efceaffd640 1 -- 172.21.15.154:0/3456364867 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7efce401e460 con 0x7efcf4157b20 2023-09-23T03:23:57.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.572+0000 7efceaffd640 1 --2- 172.21.15.154:0/3456364867 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efcd806af30 0x7efcd806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:57.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.572+0000 7efceaffd640 1 -- 172.21.15.154:0/3456364867 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6777+0+0 (secure 0 0 0) 0x7efce408a080 con 0x7efcf4157b20 2023-09-23T03:23:57.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.572+0000 7efcfa42c640 1 --2- 172.21.15.154:0/3456364867 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efcd806af30 0x7efcd806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:57.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.573+0000 7efcfa42c640 1 --2- 172.21.15.154:0/3456364867 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efcd806af30 0x7efcd806d3f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7efcf0034000 tx=0x7efcf002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:57.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.576+0000 7efceaffd640 1 -- 172.21.15.154:0/3456364867 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7efce40bca50 con 0x7efcf4157b20 2023-09-23T03:23:57.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:57.749+0000 7efcfc690640 1 -- 172.21.15.154:0/3456364867 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7efcf40cc1d0 con 0x7efcf4157b20 2023-09-23T03:23:58.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.470+0000 7efceaffd640 1 -- 172.21.15.154:0/3456364867 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v73) v1 ==== 181+0+0 (secure 0 0 0) 0x7efce4058680 con 0x7efcf4157b20 2023-09-23T03:23:58.471 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_metadata' removed 2023-09-23T03:23:58.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.473+0000 7efcfc690640 1 -- 172.21.15.154:0/3456364867 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efcd806af30 msgr2=0x7efcd806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:58.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.473+0000 7efcfc690640 1 --2- 172.21.15.154:0/3456364867 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efcd806af30 0x7efcd806d3f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7efcf0034000 tx=0x7efcf002e000 comp rx=0 tx=0).stop 2023-09-23T03:23:58.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.473+0000 7efcfc690640 1 -- 172.21.15.154:0/3456364867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcf4157b20 msgr2=0x7efcf40b26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:58.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.473+0000 7efcfc690640 1 --2- 172.21.15.154:0/3456364867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcf4157b20 0x7efcf40b26c0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7efce400d7d0 tx=0x7efce400dc90 comp rx=0 tx=0).stop 2023-09-23T03:23:58.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.474+0000 7efcfc690640 1 -- 172.21.15.154:0/3456364867 shutdown_connections 2023-09-23T03:23:58.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.474+0000 7efcfc690640 1 --2- 172.21.15.154:0/3456364867 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efcd806af30 0x7efcd806d3f0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:58.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.474+0000 7efcfc690640 1 --2- 172.21.15.154:0/3456364867 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcf4145d40 0x7efcf415eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:58.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.474+0000 7efcfc690640 1 --2- 172.21.15.154:0/3456364867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcf4157b20 0x7efcf40b26c0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:58.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.474+0000 7efcfc690640 1 --2- 172.21.15.154:0/3456364867 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efce0003970 0x7efcf414c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:58.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.474+0000 7efcfc690640 1 -- 172.21.15.154:0/3456364867 >> 172.21.15.154:0/3456364867 conn(0x7efcf406a520 msgr2=0x7efcf4070160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:58.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.474+0000 7efcfc690640 1 -- 172.21.15.154:0/3456364867 shutdown_connections 2023-09-23T03:23:58.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.475+0000 7efcfc690640 1 -- 172.21.15.154:0/3456364867 wait complete. 2023-09-23T03:23:58.497 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-09-23T03:23:58.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.708+0000 7f19729a5640 1 Processor -- start 2023-09-23T03:23:58.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.708+0000 7f19729a5640 1 -- start start 2023-09-23T03:23:58.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.709+0000 7f19729a5640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f196c0ce3c0 0x7f196c0ce810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:58.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.709+0000 7f19729a5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f196c0ced50 con 0x7f196c1576a0 2023-09-23T03:23:58.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.709+0000 7f19729a5640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f196c0721d0 con 0x7f196c1572c0 2023-09-23T03:23:58.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.709+0000 7f19729a5640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f196c071650 con 0x7f196c0ce3c0 2023-09-23T03:23:58.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.710+0000 7f196b7fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f196c1576a0 0x7f196c0b3e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46058/0 (socket says 172.21.15.154:46058) 2023-09-23T03:23:58.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.710+0000 7f196b7fe640 1 -- 172.21.15.154:0/347889687 learned_addr learned my addr 172.21.15.154:0/347889687 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:23:58.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.710+0000 7f1970f42640 1 --2- 172.21.15.154:0/347889687 >> v2:172.21.15.190:3300/0 conn(0x7f196c0ce3c0 0x7f196c0ce810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:58.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.710+0000 7f196affd640 1 -- 172.21.15.154:0/347889687 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1757419459 0 0) 0x7f1960001110 con 0x7f196c1576a0 2023-09-23T03:23:58.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.710+0000 7f196affd640 1 -- 172.21.15.154:0/347889687 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1954003520 con 0x7f196c1576a0 2023-09-23T03:23:58.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.711+0000 7f196affd640 1 -- 172.21.15.154:0/347889687 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3291441973 0 0) 0x7f1954003520 con 0x7f196c1576a0 2023-09-23T03:23:58.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.711+0000 7f196affd640 1 -- 172.21.15.154:0/347889687 >> v2:172.21.15.190:3300/0 conn(0x7f196c0ce3c0 msgr2=0x7f196c0ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:58.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.711+0000 7f196affd640 1 --2- 172.21.15.154:0/347889687 >> v2:172.21.15.190:3300/0 conn(0x7f196c0ce3c0 0x7f196c0ce810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:58.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.711+0000 7f196affd640 1 -- 172.21.15.154:0/347889687 >> v1:172.21.15.190:6789/0 conn(0x7f196c1572c0 legacy=0x7f196c159e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:58.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.711+0000 7f196affd640 1 -- 172.21.15.154:0/347889687 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196c071930 con 0x7f196c1576a0 2023-09-23T03:23:58.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.711+0000 7f196affd640 1 -- 172.21.15.154:0/347889687 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f1960001800 con 0x7f196c1576a0 2023-09-23T03:23:58.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.711+0000 7f196affd640 1 -- 172.21.15.154:0/347889687 >> v1:172.21.15.154:6789/0 conn(0x7f196c1576a0 legacy=0x7f196c0b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:58.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.711+0000 7f196affd640 1 --2- 172.21.15.154:0/347889687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19540039c0 0x7f1954003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:58.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.711+0000 7f196affd640 1 --2- 172.21.15.154:0/347889687 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19540047c0 0x7f1954004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:58.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.711+0000 7f196affd640 1 --2- 172.21.15.154:0/347889687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1954005250 0x7f195400db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:58.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.711+0000 7f196affd640 1 -- 172.21.15.154:0/347889687 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c071650 con 0x7f19540039c0 2023-09-23T03:23:58.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.711+0000 7f196affd640 1 -- 172.21.15.154:0/347889687 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195400e3b0 con 0x7f1954005250 2023-09-23T03:23:58.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.711+0000 7f196affd640 1 -- 172.21.15.154:0/347889687 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f195400e570 con 0x7f19540047c0 2023-09-23T03:23:58.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.711+0000 7f196bfff640 1 --2- 172.21.15.154:0/347889687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19540039c0 0x7f1954003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:58.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.711+0000 7f196bfff640 1 -- 172.21.15.154:0/347889687 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19540047c0 msgr2=0x7f1954004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:58.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.711+0000 7f196bfff640 1 --2- 172.21.15.154:0/347889687 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19540047c0 0x7f1954004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:58.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.711+0000 7f196bfff640 1 -- 172.21.15.154:0/347889687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1954005250 msgr2=0x7f195400db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:58.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.711+0000 7f196bfff640 1 --2- 172.21.15.154:0/347889687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1954005250 0x7f195400db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:58.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.712+0000 7f196bfff640 1 -- 172.21.15.154:0/347889687 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1954003520 con 0x7f19540039c0 2023-09-23T03:23:58.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.712+0000 7f1970f42640 1 -- 172.21.15.154:0/347889687 reap_dead start 2023-09-23T03:23:58.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.713+0000 7f196bfff640 1 --2- 172.21.15.154:0/347889687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19540039c0 0x7f1954003dc0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f195c003db0 tx=0x7f19540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=352fc398d0e68f15 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:58.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.713+0000 7f196affd640 1 -- 172.21.15.154:0/347889687 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c032600 con 0x7f19540039c0 2023-09-23T03:23:58.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.713+0000 7f196affd640 1 -- 172.21.15.154:0/347889687 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195c02e050 con 0x7f19540039c0 2023-09-23T03:23:58.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.713+0000 7f196affd640 1 -- 172.21.15.154:0/347889687 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c0329e0 con 0x7f19540039c0 2023-09-23T03:23:58.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.713+0000 7f19729a5640 1 -- 172.21.15.154:0/347889687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19540039c0 msgr2=0x7f1954003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:58.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.713+0000 7f19729a5640 1 --2- 172.21.15.154:0/347889687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19540039c0 0x7f1954003dc0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f195c003db0 tx=0x7f19540057f0 comp rx=0 tx=0).stop 2023-09-23T03:23:58.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.714+0000 7f19729a5640 1 -- 172.21.15.154:0/347889687 shutdown_connections 2023-09-23T03:23:58.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.714+0000 7f19729a5640 1 --2- 172.21.15.154:0/347889687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19540039c0 0x7f1954003dc0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:58.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.714+0000 7f19729a5640 1 -- 172.21.15.154:0/347889687 >> 172.21.15.154:0/347889687 conn(0x7f196c06a520 msgr2=0x7f196c06bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:58.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.714+0000 7f19729a5640 1 -- 172.21.15.154:0/347889687 shutdown_connections 2023-09-23T03:23:58.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.714+0000 7f19729a5640 1 -- 172.21.15.154:0/347889687 wait complete. 2023-09-23T03:23:58.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.715+0000 7f19729a5640 1 Processor -- start 2023-09-23T03:23:58.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.716+0000 7f19729a5640 1 -- start start 2023-09-23T03:23:58.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.717+0000 7f19729a5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19540039c0 0x7f196c1569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:58.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.717+0000 7f19729a5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f196c1e5600 0x7f196c0b10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:58.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.717+0000 7f19729a5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f196c0ad630 0x7f196c0b1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:58.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.717+0000 7f19729a5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c1e66f0 con 0x7f196c0ad630 2023-09-23T03:23:58.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.717+0000 7f19729a5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c0addf0 con 0x7f19540039c0 2023-09-23T03:23:58.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.717+0000 7f19729a5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196c0c3600 con 0x7f196c1e5600 2023-09-23T03:23:58.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.717+0000 7f1970f42640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f196c0ad630 0x7f196c0b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:58.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.717+0000 7f1970f42640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f196c0ad630 0x7f196c0b1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:51536/0 (socket says 172.21.15.154:51536) 2023-09-23T03:23:58.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.717+0000 7f1970f42640 1 -- 172.21.15.154:0/3270088504 learned_addr learned my addr 172.21.15.154:0/3270088504 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:23:58.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.717+0000 7f196b7fe640 1 --2- 172.21.15.154:0/3270088504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f196c1e5600 0x7f196c0b10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:58.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.717+0000 7f196bfff640 1 --2- 172.21.15.154:0/3270088504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19540039c0 0x7f196c1569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:58.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.717+0000 7f1970f42640 1 -- 172.21.15.154:0/3270088504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f196c1e5600 msgr2=0x7f196c0b10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:58.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.717+0000 7f1970f42640 1 --2- 172.21.15.154:0/3270088504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f196c1e5600 0x7f196c0b10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:58.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.717+0000 7f1970f42640 1 -- 172.21.15.154:0/3270088504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19540039c0 msgr2=0x7f196c1569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:58.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.717+0000 7f1970f42640 1 --2- 172.21.15.154:0/3270088504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19540039c0 0x7f196c1569e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:58.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.717+0000 7f1970f42640 1 -- 172.21.15.154:0/3270088504 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196c156f20 con 0x7f196c0ad630 2023-09-23T03:23:58.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.717+0000 7f196b7fe640 1 --2- 172.21.15.154:0/3270088504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f196c1e5600 0x7f196c0b10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:23:58.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.717+0000 7f196bfff640 1 --2- 172.21.15.154:0/3270088504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19540039c0 0x7f196c1569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:23:58.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.718+0000 7f1970f42640 1 --2- 172.21.15.154:0/3270088504 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f196c0ad630 0x7f196c0b1620 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f19540057a0 tx=0x7f196400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:58.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.718+0000 7f1968ff9640 1 -- 172.21.15.154:0/3270088504 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196400edc0 con 0x7f196c0ad630 2023-09-23T03:23:58.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.718+0000 7f1968ff9640 1 -- 172.21.15.154:0/3270088504 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1964014420 con 0x7f196c0ad630 2023-09-23T03:23:58.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.718+0000 7f1968ff9640 1 -- 172.21.15.154:0/3270088504 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19640147d0 con 0x7f196c0ad630 2023-09-23T03:23:58.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.718+0000 7f19729a5640 1 -- 172.21.15.154:0/3270088504 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f196c1e5b70 con 0x7f196c0ad630 2023-09-23T03:23:58.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.718+0000 7f19729a5640 1 -- 172.21.15.154:0/3270088504 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f196c1e6130 con 0x7f196c0ad630 2023-09-23T03:23:58.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.719+0000 7f19729a5640 1 -- 172.21.15.154:0/3270088504 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f196c0adf30 con 0x7f196c0ad630 2023-09-23T03:23:58.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.720+0000 7f1968ff9640 1 -- 172.21.15.154:0/3270088504 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f1964014930 con 0x7f196c0ad630 2023-09-23T03:23:58.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.721+0000 7f1968ff9640 1 --2- 172.21.15.154:0/3270088504 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f195806af60 0x7f195806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:58.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.721+0000 7f1968ff9640 1 -- 172.21.15.154:0/3270088504 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6339+0+0 (secure 0 0 0) 0x7f1964089fc0 con 0x7f196c0ad630 2023-09-23T03:23:58.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.721+0000 7f196bfff640 1 --2- 172.21.15.154:0/3270088504 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f195806af60 0x7f195806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:58.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.722+0000 7f196bfff640 1 --2- 172.21.15.154:0/3270088504 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f195806af60 0x7f195806d420 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f195c034000 tx=0x7f195c009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:58.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.725+0000 7f1968ff9640 1 -- 172.21.15.154:0/3270088504 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f1964054ce0 con 0x7f196c0ad630 2023-09-23T03:23:58.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:58.917+0000 7f19729a5640 1 -- 172.21.15.154:0/3270088504 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f196c04e920 con 0x7f196c0ad630 2023-09-23T03:23:59.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.473+0000 7f1968ff9640 1 -- 172.21.15.154:0/3270088504 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v74) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1964058300 con 0x7f196c0ad630 2023-09-23T03:23:59.474 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_data' removed 2023-09-23T03:23:59.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.480+0000 7f19729a5640 1 -- 172.21.15.154:0/3270088504 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f195806af60 msgr2=0x7f195806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:59.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.480+0000 7f19729a5640 1 --2- 172.21.15.154:0/3270088504 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f195806af60 0x7f195806d420 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f195c034000 tx=0x7f195c009030 comp rx=0 tx=0).stop 2023-09-23T03:23:59.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.480+0000 7f19729a5640 1 -- 172.21.15.154:0/3270088504 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f196c0ad630 msgr2=0x7f196c0b1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:59.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.480+0000 7f19729a5640 1 --2- 172.21.15.154:0/3270088504 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f196c0ad630 0x7f196c0b1620 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f19540057a0 tx=0x7f196400e4b0 comp rx=0 tx=0).stop 2023-09-23T03:23:59.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.481+0000 7f19729a5640 1 -- 172.21.15.154:0/3270088504 shutdown_connections 2023-09-23T03:23:59.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.481+0000 7f19729a5640 1 --2- 172.21.15.154:0/3270088504 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f195806af60 0x7f195806d420 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:59.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.481+0000 7f19729a5640 1 --2- 172.21.15.154:0/3270088504 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f196c0ad630 0x7f196c0b1620 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:59.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.481+0000 7f19729a5640 1 --2- 172.21.15.154:0/3270088504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f196c1e5600 0x7f196c0b10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:59.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.481+0000 7f19729a5640 1 --2- 172.21.15.154:0/3270088504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19540039c0 0x7f196c1569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:59.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.481+0000 7f19729a5640 1 -- 172.21.15.154:0/3270088504 >> 172.21.15.154:0/3270088504 conn(0x7f196c06a520 msgr2=0x7f196c0c7c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:59.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.481+0000 7f19729a5640 1 -- 172.21.15.154:0/3270088504 shutdown_connections 2023-09-23T03:23:59.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.481+0000 7f19729a5640 1 -- 172.21.15.154:0/3270088504 wait complete. 2023-09-23T03:23:59.505 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_extra cephfs_data_extra --yes-i-really-really-mean-it 2023-09-23T03:23:59.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.723+0000 7f4df8e13640 1 Processor -- start 2023-09-23T03:23:59.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.723+0000 7f4df8e13640 1 -- start start 2023-09-23T03:23:59.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.724+0000 7f4df8e13640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4df406c740 0x7f4df40ab030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:59.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.724+0000 7f4df8e13640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4df40acc40 0x7f4df40ab570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:59.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.724+0000 7f4df8e13640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4df406c0e0 con 0x7f4df406cb20 2023-09-23T03:23:59.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.725+0000 7f4df8e13640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4df40abba0 con 0x7f4df40acc40 2023-09-23T03:23:59.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.725+0000 7f4df8e13640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df40abe80 con 0x7f4df406c740 2023-09-23T03:23:59.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.725+0000 7f4df3fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f4df406cb20 0x7f4df40b1160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46064/0 (socket says 172.21.15.154:46064) 2023-09-23T03:23:59.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.725+0000 7f4df3fff640 1 -- 172.21.15.154:0/1871712712 learned_addr learned my addr 172.21.15.154:0/1871712712 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:23:59.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.725+0000 7f4df37fe640 1 --2- 172.21.15.154:0/1871712712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4df406c740 0x7f4df40ab030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:59.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.725+0000 7f4df2ffd640 1 --2- 172.21.15.154:0/1871712712 >> v2:172.21.15.190:3300/0 conn(0x7f4df40acc40 0x7f4df40ab570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:59.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.725+0000 7f4df27fc640 1 -- 172.21.15.154:0/1871712712 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3019039112 0 0) 0x7f4df406c0e0 con 0x7f4df406cb20 2023-09-23T03:23:59.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.726+0000 7f4df27fc640 1 -- 172.21.15.154:0/1871712712 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ddc005730 con 0x7f4df406cb20 2023-09-23T03:23:59.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.726+0000 7f4df27fc640 1 -- 172.21.15.154:0/1871712712 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2016828480 0 0) 0x7f4ddc005730 con 0x7f4df406cb20 2023-09-23T03:23:59.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.726+0000 7f4df27fc640 1 -- 172.21.15.154:0/1871712712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4df406c740 msgr2=0x7f4df40ab030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:59.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.726+0000 7f4df27fc640 1 --2- 172.21.15.154:0/1871712712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4df406c740 0x7f4df40ab030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:59.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.726+0000 7f4df27fc640 1 -- 172.21.15.154:0/1871712712 >> v2:172.21.15.190:3300/0 conn(0x7f4df40acc40 msgr2=0x7f4df40ab570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:59.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.726+0000 7f4df27fc640 1 --2- 172.21.15.154:0/1871712712 >> v2:172.21.15.190:3300/0 conn(0x7f4df40acc40 0x7f4df40ab570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:59.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.726+0000 7f4df27fc640 1 -- 172.21.15.154:0/1871712712 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df40ac160 con 0x7f4df406cb20 2023-09-23T03:23:59.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.726+0000 7f4df27fc640 1 -- 172.21.15.154:0/1871712712 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f4dec001a10 con 0x7f4df406cb20 2023-09-23T03:23:59.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.726+0000 7f4df27fc640 1 -- 172.21.15.154:0/1871712712 >> v1:172.21.15.154:6789/0 conn(0x7f4df406cb20 legacy=0x7f4df40b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:59.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.726+0000 7f4df27fc640 1 --2- 172.21.15.154:0/1871712712 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ddc005b80 0x7f4ddc005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:59.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.726+0000 7f4df27fc640 1 --2- 172.21.15.154:0/1871712712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ddc0069d0 0x7f4ddc006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:59.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.726+0000 7f4df27fc640 1 --2- 172.21.15.154:0/1871712712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4df406c740 0x7f4ddc007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:59.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.726+0000 7f4df27fc640 1 -- 172.21.15.154:0/1871712712 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df40abba0 con 0x7f4ddc005b80 2023-09-23T03:23:59.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.726+0000 7f4df27fc640 1 -- 172.21.15.154:0/1871712712 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df40abe80 con 0x7f4ddc0069d0 2023-09-23T03:23:59.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.726+0000 7f4df27fc640 1 -- 172.21.15.154:0/1871712712 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ddc00db90 con 0x7f4df406c740 2023-09-23T03:23:59.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.726+0000 7f4df37fe640 1 --2- 172.21.15.154:0/1871712712 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ddc005b80 0x7f4ddc005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:59.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.727+0000 7f4df37fe640 1 -- 172.21.15.154:0/1871712712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4df406c740 msgr2=0x7f4ddc007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:59.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.727+0000 7f4df37fe640 1 --2- 172.21.15.154:0/1871712712 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4df406c740 0x7f4ddc007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:59.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.727+0000 7f4df2ffd640 1 --2- 172.21.15.154:0/1871712712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ddc0069d0 0x7f4ddc006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:59.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.727+0000 7f4df37fe640 1 -- 172.21.15.154:0/1871712712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ddc0069d0 msgr2=0x7f4ddc006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:23:59.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.727+0000 7f4df37fe640 1 --2- 172.21.15.154:0/1871712712 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ddc0069d0 0x7f4ddc006e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:59.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.727+0000 7f4df37fe640 1 -- 172.21.15.154:0/1871712712 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ddc005730 con 0x7f4ddc005b80 2023-09-23T03:23:59.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.728+0000 7f4df37fe640 1 --2- 172.21.15.154:0/1871712712 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ddc005b80 0x7f4ddc005fa0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f4de4009ac0 tx=0x7f4ddc007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ed44488a8a4368 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:59.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.728+0000 7f4df27fc640 1 -- 172.21.15.154:0/1871712712 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de402c6d0 con 0x7f4ddc005b80 2023-09-23T03:23:59.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.728+0000 7f4df27fc640 1 -- 172.21.15.154:0/1871712712 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de402c830 con 0x7f4ddc005b80 2023-09-23T03:23:59.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.728+0000 7f4df27fc640 1 -- 172.21.15.154:0/1871712712 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de402cbc0 con 0x7f4ddc005b80 2023-09-23T03:23:59.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.729+0000 7f4df8e13640 1 -- 172.21.15.154:0/1871712712 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ddc005b80 msgr2=0x7f4ddc005fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:59.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.729+0000 7f4df8e13640 1 --2- 172.21.15.154:0/1871712712 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ddc005b80 0x7f4ddc005fa0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f4de4009ac0 tx=0x7f4ddc007a00 comp rx=0 tx=0).stop 2023-09-23T03:23:59.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.729+0000 7f4df3fff640 1 -- 172.21.15.154:0/1871712712 reap_dead start 2023-09-23T03:23:59.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.729+0000 7f4df8e13640 1 -- 172.21.15.154:0/1871712712 shutdown_connections 2023-09-23T03:23:59.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.729+0000 7f4df8e13640 1 -- 172.21.15.154:0/1871712712 >> 172.21.15.154:0/1871712712 conn(0x7f4df4069eb0 msgr2=0x7f4df406a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:23:59.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.729+0000 7f4df8e13640 1 -- 172.21.15.154:0/1871712712 shutdown_connections 2023-09-23T03:23:59.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.729+0000 7f4df8e13640 1 -- 172.21.15.154:0/1871712712 wait complete. 2023-09-23T03:23:59.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.731+0000 7f4df8e13640 1 Processor -- start 2023-09-23T03:23:59.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.731+0000 7f4df8e13640 1 -- start start 2023-09-23T03:23:59.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.732+0000 7f4df8e13640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ddc005b80 0x7f4df41579b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:59.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.732+0000 7f4df8e13640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4df41585e0 0x7f4df415b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:59.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.732+0000 7f4df8e13640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4df4156db0 0x7f4df4157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:59.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.732+0000 7f4df8e13640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df4158230 con 0x7f4df4156db0 2023-09-23T03:23:59.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.732+0000 7f4df8e13640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df4158a60 con 0x7f4ddc005b80 2023-09-23T03:23:59.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.732+0000 7f4df8e13640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df4157770 con 0x7f4df41585e0 2023-09-23T03:23:59.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.732+0000 7f4df3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4df4156db0 0x7f4df4157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:59.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.732+0000 7f4df3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4df4156db0 0x7f4df4157230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:51554/0 (socket says 172.21.15.154:51554) 2023-09-23T03:23:59.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.732+0000 7f4df3fff640 1 -- 172.21.15.154:0/154851735 learned_addr learned my addr 172.21.15.154:0/154851735 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:23:59.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.732+0000 7f4df37fe640 1 --2- 172.21.15.154:0/154851735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ddc005b80 0x7f4df41579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:59.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.732+0000 7f4df3fff640 1 -- 172.21.15.154:0/154851735 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4df41585e0 msgr2=0x7f4df415b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:59.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.732+0000 7f4df3fff640 1 --2- 172.21.15.154:0/154851735 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4df41585e0 0x7f4df415b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:59.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.732+0000 7f4df3fff640 1 -- 172.21.15.154:0/154851735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ddc005b80 msgr2=0x7f4df41579b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:23:59.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.732+0000 7f4df3fff640 1 --2- 172.21.15.154:0/154851735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ddc005b80 0x7f4df41579b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:23:59.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.732+0000 7f4df3fff640 1 -- 172.21.15.154:0/154851735 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df4158ce0 con 0x7f4df4156db0 2023-09-23T03:23:59.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.733+0000 7f4df37fe640 1 --2- 172.21.15.154:0/154851735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ddc005b80 0x7f4df41579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:23:59.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.733+0000 7f4df3fff640 1 --2- 172.21.15.154:0/154851735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4df4156db0 0x7f4df4157230 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f4df40c8890 tx=0x7f4dec00cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:59.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.733+0000 7f4ddbfff640 1 -- 172.21.15.154:0/154851735 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec007c00 con 0x7f4df4156db0 2023-09-23T03:23:59.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.733+0000 7f4ddbfff640 1 -- 172.21.15.154:0/154851735 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dec004d60 con 0x7f4df4156db0 2023-09-23T03:23:59.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.733+0000 7f4ddbfff640 1 -- 172.21.15.154:0/154851735 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec00a830 con 0x7f4df4156db0 2023-09-23T03:23:59.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.733+0000 7f4df8e13640 1 -- 172.21.15.154:0/154851735 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df4158fc0 con 0x7f4df4156db0 2023-09-23T03:23:59.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.733+0000 7f4df8e13640 1 -- 172.21.15.154:0/154851735 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df4159580 con 0x7f4df4156db0 2023-09-23T03:23:59.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.734+0000 7f4df8e13640 1 -- 172.21.15.154:0/154851735 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4df41597a0 con 0x7f4df4156db0 2023-09-23T03:23:59.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.735+0000 7f4ddbfff640 1 -- 172.21.15.154:0/154851735 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f4dec00a990 con 0x7f4df4156db0 2023-09-23T03:23:59.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.736+0000 7f4ddbfff640 1 --2- 172.21.15.154:0/154851735 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4de006af30 0x7f4de006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:23:59.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.736+0000 7f4ddbfff640 1 -- 172.21.15.154:0/154851735 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5925+0+0 (secure 0 0 0) 0x7f4dec089090 con 0x7f4df4156db0 2023-09-23T03:23:59.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.736+0000 7f4df37fe640 1 --2- 172.21.15.154:0/154851735 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4de006af30 0x7f4de006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:23:59.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.737+0000 7f4df37fe640 1 --2- 172.21.15.154:0/154851735 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4de006af30 0x7f4de006d3f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4de4036000 tx=0x7f4de4030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:23:59.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.741+0000 7f4ddbfff640 1 -- 172.21.15.154:0/154851735 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f4dec053fe0 con 0x7f4df4156db0 2023-09-23T03:23:59.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:23:59.937+0000 7f4df8e13640 1 -- 172.21.15.154:0/154851735 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_extra", "pool2": "cephfs_data_extra", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4df4067390 con 0x7f4df4156db0 2023-09-23T03:24:00.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.499+0000 7f4ddbfff640 1 -- 172.21.15.154:0/154851735 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_extra", "pool2": "cephfs_data_extra", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_extra' removed v75) v1 ==== 187+0+0 (secure 0 0 0) 0x7f4dec057600 con 0x7f4df4156db0 2023-09-23T03:24:00.500 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_data_extra' removed 2023-09-23T03:24:00.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.503+0000 7f4df8e13640 1 -- 172.21.15.154:0/154851735 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4de006af30 msgr2=0x7f4de006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:00.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.503+0000 7f4df8e13640 1 --2- 172.21.15.154:0/154851735 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4de006af30 0x7f4de006d3f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4de4036000 tx=0x7f4de4030000 comp rx=0 tx=0).stop 2023-09-23T03:24:00.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.503+0000 7f4df8e13640 1 -- 172.21.15.154:0/154851735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4df4156db0 msgr2=0x7f4df4157230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:00.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.503+0000 7f4df8e13640 1 --2- 172.21.15.154:0/154851735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4df4156db0 0x7f4df4157230 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f4df40c8890 tx=0x7f4dec00cbe0 comp rx=0 tx=0).stop 2023-09-23T03:24:00.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.503+0000 7f4df8e13640 1 -- 172.21.15.154:0/154851735 shutdown_connections 2023-09-23T03:24:00.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.503+0000 7f4df8e13640 1 --2- 172.21.15.154:0/154851735 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4de006af30 0x7f4de006d3f0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:00.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.503+0000 7f4df8e13640 1 --2- 172.21.15.154:0/154851735 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4df4156db0 0x7f4df4157230 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:00.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.504+0000 7f4df8e13640 1 --2- 172.21.15.154:0/154851735 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4df41585e0 0x7f4df415b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:00.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.504+0000 7f4df8e13640 1 --2- 172.21.15.154:0/154851735 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4ddc005b80 0x7f4df41579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:00.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.504+0000 7f4df8e13640 1 -- 172.21.15.154:0/154851735 >> 172.21.15.154:0/154851735 conn(0x7f4df4069eb0 msgr2=0x7f4df40aed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:00.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.504+0000 7f4df8e13640 1 -- 172.21.15.154:0/154851735 shutdown_connections 2023-09-23T03:24:00.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.504+0000 7f4df8e13640 1 -- 172.21.15.154:0/154851735 wait complete. 2023-09-23T03:24:00.539 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-09-23T03:24:00.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.732+0000 7fb3c2ee0640 1 Processor -- start 2023-09-23T03:24:00.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.732+0000 7fb3c2ee0640 1 -- start start 2023-09-23T03:24:00.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.734+0000 7fb3c2ee0640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb3bc071080 0x7fb3bc0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:00.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.734+0000 7fb3c2ee0640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3bc15e090 con 0x7fb3bc06a490 2023-09-23T03:24:00.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.734+0000 7fb3c2ee0640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3bc15d240 con 0x7fb3bc0cd8d0 2023-09-23T03:24:00.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.734+0000 7fb3c2ee0640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3bc0cdd40 con 0x7fb3bc071080 2023-09-23T03:24:00.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.734+0000 7fb3c0c7c640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fb3bc06a490 0x7fb3bc0cb6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46080/0 (socket says 172.21.15.154:46080) 2023-09-23T03:24:00.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.734+0000 7fb3c0c7c640 1 -- 172.21.15.154:0/4292661566 learned_addr learned my addr 172.21.15.154:0/4292661566 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:24:00.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.735+0000 7fb3bb7fe640 1 -- 172.21.15.154:0/4292661566 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4253073020 0 0) 0x7fb3bc15e090 con 0x7fb3bc06a490 2023-09-23T03:24:00.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.735+0000 7fb3bbfff640 1 --2- 172.21.15.154:0/4292661566 >> v2:172.21.15.190:3300/0 conn(0x7fb3bc071080 0x7fb3bc0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:00.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.735+0000 7fb3bb7fe640 1 -- 172.21.15.154:0/4292661566 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3ac003520 con 0x7fb3bc06a490 2023-09-23T03:24:00.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.735+0000 7fb3bb7fe640 1 -- 172.21.15.154:0/4292661566 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3358938161 0 0) 0x7fb3ac003520 con 0x7fb3bc06a490 2023-09-23T03:24:00.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.735+0000 7fb3bb7fe640 1 -- 172.21.15.154:0/4292661566 >> v2:172.21.15.190:3300/0 conn(0x7fb3bc071080 msgr2=0x7fb3bc0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:00.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.735+0000 7fb3bb7fe640 1 --2- 172.21.15.154:0/4292661566 >> v2:172.21.15.190:3300/0 conn(0x7fb3bc071080 0x7fb3bc0cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:00.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.735+0000 7fb3bb7fe640 1 -- 172.21.15.154:0/4292661566 >> v1:172.21.15.190:6789/0 conn(0x7fb3bc0cd8d0 legacy=0x7fb3bc160950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:00.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.735+0000 7fb3bb7fe640 1 -- 172.21.15.154:0/4292661566 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3bc071500 con 0x7fb3bc06a490 2023-09-23T03:24:00.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.735+0000 7fb3bb7fe640 1 -- 172.21.15.154:0/4292661566 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fb3b0001670 con 0x7fb3bc06a490 2023-09-23T03:24:00.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.736+0000 7fb3bb7fe640 1 -- 172.21.15.154:0/4292661566 >> v1:172.21.15.154:6789/0 conn(0x7fb3bc06a490 legacy=0x7fb3bc0cb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:00.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.736+0000 7fb3bb7fe640 1 --2- 172.21.15.154:0/4292661566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3ac003970 0x7fb3ac003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:00.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.736+0000 7fb3bb7fe640 1 --2- 172.21.15.154:0/4292661566 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3ac0047c0 0x7fb3ac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:00.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.736+0000 7fb3bb7fe640 1 --2- 172.21.15.154:0/4292661566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3ac005180 0x7fb3ac00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:00.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.736+0000 7fb3bb7fe640 1 -- 172.21.15.154:0/4292661566 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3bc0cdd40 con 0x7fb3ac0047c0 2023-09-23T03:24:00.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.736+0000 7fb3bb7fe640 1 -- 172.21.15.154:0/4292661566 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac00c120 con 0x7fb3ac005180 2023-09-23T03:24:00.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.736+0000 7fb3bb7fe640 1 -- 172.21.15.154:0/4292661566 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ac00c2e0 con 0x7fb3ac003970 2023-09-23T03:24:00.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.736+0000 7fb3bbfff640 1 --2- 172.21.15.154:0/4292661566 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3ac0047c0 0x7fb3ac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:00.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.736+0000 7fb3c0c7c640 1 --2- 172.21.15.154:0/4292661566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3ac003970 0x7fb3ac003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:00.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.736+0000 7fb3c147d640 1 --2- 172.21.15.154:0/4292661566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3ac005180 0x7fb3ac00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:00.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.736+0000 7fb3bbfff640 1 -- 172.21.15.154:0/4292661566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3ac003970 msgr2=0x7fb3ac003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:00.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.736+0000 7fb3bbfff640 1 --2- 172.21.15.154:0/4292661566 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3ac003970 0x7fb3ac003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:00.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.736+0000 7fb3bbfff640 1 -- 172.21.15.154:0/4292661566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3ac005180 msgr2=0x7fb3ac00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:00.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.736+0000 7fb3bbfff640 1 --2- 172.21.15.154:0/4292661566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3ac005180 0x7fb3ac00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:00.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.736+0000 7fb3bbfff640 1 -- 172.21.15.154:0/4292661566 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3ac003520 con 0x7fb3ac0047c0 2023-09-23T03:24:00.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.736+0000 7fb3c147d640 1 --2- 172.21.15.154:0/4292661566 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3ac005180 0x7fb3ac00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:00.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.736+0000 7fb3c147d640 1 -- 172.21.15.154:0/4292661566 reap_dead start 2023-09-23T03:24:00.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.738+0000 7fb3bbfff640 1 --2- 172.21.15.154:0/4292661566 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3ac0047c0 0x7fb3ac004c40 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8009ac0 tx=0x7fb3ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f422f1ca73f402a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:00.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.738+0000 7fb3bb7fe640 1 -- 172.21.15.154:0/4292661566 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a802c6b0 con 0x7fb3ac0047c0 2023-09-23T03:24:00.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.738+0000 7fb3bb7fe640 1 -- 172.21.15.154:0/4292661566 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a802c810 con 0x7fb3ac0047c0 2023-09-23T03:24:00.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.738+0000 7fb3bb7fe640 1 -- 172.21.15.154:0/4292661566 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a802cba0 con 0x7fb3ac0047c0 2023-09-23T03:24:00.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.738+0000 7fb3c2ee0640 1 -- 172.21.15.154:0/4292661566 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3ac0047c0 msgr2=0x7fb3ac004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:00.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.738+0000 7fb3c2ee0640 1 --2- 172.21.15.154:0/4292661566 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3ac0047c0 0x7fb3ac004c40 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8009ac0 tx=0x7fb3ac0057f0 comp rx=0 tx=0).stop 2023-09-23T03:24:00.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.739+0000 7fb3c2ee0640 1 -- 172.21.15.154:0/4292661566 shutdown_connections 2023-09-23T03:24:00.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.739+0000 7fb3c2ee0640 1 --2- 172.21.15.154:0/4292661566 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3ac0047c0 0x7fb3ac004c40 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:00.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.739+0000 7fb3c2ee0640 1 -- 172.21.15.154:0/4292661566 >> 172.21.15.154:0/4292661566 conn(0x7fb3bc0cc030 msgr2=0x7fb3bc0cc410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:00.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.739+0000 7fb3c2ee0640 1 -- 172.21.15.154:0/4292661566 shutdown_connections 2023-09-23T03:24:00.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.739+0000 7fb3c2ee0640 1 -- 172.21.15.154:0/4292661566 wait complete. 2023-09-23T03:24:00.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.740+0000 7fb3c2ee0640 1 Processor -- start 2023-09-23T03:24:00.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.741+0000 7fb3c2ee0640 1 -- start start 2023-09-23T03:24:00.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.741+0000 7fb3c2ee0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3ac0047c0 0x7fb3bc155a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:00.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.741+0000 7fb3c2ee0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3bc0c7c60 0x7fb3bc0afd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:00.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.742+0000 7fb3c2ee0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3bc15f450 0x7fb3bc15f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:00.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.742+0000 7fb3c2ee0640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3bc15fe10 con 0x7fb3ac0047c0 2023-09-23T03:24:00.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.742+0000 7fb3c2ee0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3bc0b0e70 con 0x7fb3bc0c7c60 2023-09-23T03:24:00.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.742+0000 7fb3c2ee0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3bc155f60 con 0x7fb3bc15f450 2023-09-23T03:24:00.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.742+0000 7fb3c0c7c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3ac0047c0 0x7fb3bc155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:00.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.742+0000 7fb3c0c7c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3ac0047c0 0x7fb3bc155a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:51576/0 (socket says 172.21.15.154:51576) 2023-09-23T03:24:00.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.742+0000 7fb3c0c7c640 1 -- 172.21.15.154:0/1206844449 learned_addr learned my addr 172.21.15.154:0/1206844449 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:00.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.742+0000 7fb3bbfff640 1 --2- 172.21.15.154:0/1206844449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3bc0c7c60 0x7fb3bc0afd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:00.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.742+0000 7fb3c147d640 1 --2- 172.21.15.154:0/1206844449 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3bc15f450 0x7fb3bc15f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:00.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.742+0000 7fb3c0c7c640 1 -- 172.21.15.154:0/1206844449 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3bc15f450 msgr2=0x7fb3bc15f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:00.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.742+0000 7fb3c0c7c640 1 --2- 172.21.15.154:0/1206844449 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3bc15f450 0x7fb3bc15f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:00.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.742+0000 7fb3c0c7c640 1 -- 172.21.15.154:0/1206844449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3bc0c7c60 msgr2=0x7fb3bc0afd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:00.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.742+0000 7fb3c0c7c640 1 --2- 172.21.15.154:0/1206844449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3bc0c7c60 0x7fb3bc0afd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:00.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.742+0000 7fb3c0c7c640 1 -- 172.21.15.154:0/1206844449 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3bc0b02d0 con 0x7fb3ac0047c0 2023-09-23T03:24:00.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.742+0000 7fb3bbfff640 1 --2- 172.21.15.154:0/1206844449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3bc0c7c60 0x7fb3bc0afd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:00.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.742+0000 7fb3c147d640 1 --2- 172.21.15.154:0/1206844449 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3bc15f450 0x7fb3bc15f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:00.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.743+0000 7fb3c0c7c640 1 --2- 172.21.15.154:0/1206844449 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3ac0047c0 0x7fb3bc155a20 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fb3b000e730 tx=0x7fb3b000ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:00.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.743+0000 7fb3b97fa640 1 -- 172.21.15.154:0/1206844449 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b000ee40 con 0x7fb3ac0047c0 2023-09-23T03:24:00.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.743+0000 7fb3b97fa640 1 -- 172.21.15.154:0/1206844449 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b000c850 con 0x7fb3ac0047c0 2023-09-23T03:24:00.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.743+0000 7fb3b97fa640 1 -- 172.21.15.154:0/1206844449 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b000cc00 con 0x7fb3ac0047c0 2023-09-23T03:24:00.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.743+0000 7fb3c2ee0640 1 -- 172.21.15.154:0/1206844449 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3bc0c8120 con 0x7fb3ac0047c0 2023-09-23T03:24:00.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.743+0000 7fb3c2ee0640 1 -- 172.21.15.154:0/1206844449 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3bc0c86e0 con 0x7fb3ac0047c0 2023-09-23T03:24:00.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.744+0000 7fb3c2ee0640 1 -- 172.21.15.154:0/1206844449 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3bc04e920 con 0x7fb3ac0047c0 2023-09-23T03:24:00.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.745+0000 7fb3b97fa640 1 -- 172.21.15.154:0/1206844449 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fb3b001d460 con 0x7fb3ac0047c0 2023-09-23T03:24:00.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.746+0000 7fb3b97fa640 1 --2- 172.21.15.154:0/1206844449 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb3a006af30 0x7fb3a006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:00.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.747+0000 7fb3b97fa640 1 -- 172.21.15.154:0/1206844449 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5521+0+0 (secure 0 0 0) 0x7fb3b00896f0 con 0x7fb3ac0047c0 2023-09-23T03:24:00.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.747+0000 7fb3bbfff640 1 --2- 172.21.15.154:0/1206844449 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb3a006af30 0x7fb3a006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:00.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.747+0000 7fb3bbfff640 1 --2- 172.21.15.154:0/1206844449 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb3a006af30 0x7fb3a006d3f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8036000 tx=0x7fb3a8030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:00.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.751+0000 7fb3b97fa640 1 -- 172.21.15.154:0/1206844449 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fb3b0054740 con 0x7fb3ac0047c0 2023-09-23T03:24:00.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.941+0000 7fb3c2ee0640 1 -- 172.21.15.154:0/1206844449 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb3bc15ef30 con 0x7fb3ac0047c0 2023-09-23T03:24:00.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.947+0000 7fb3b97fa640 1 -- 172.21.15.154:0/1206844449 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb3b0057d60 con 0x7fb3ac0047c0 2023-09-23T03:24:00.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.950+0000 7fb3c2ee0640 1 -- 172.21.15.154:0/1206844449 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb3a006af30 msgr2=0x7fb3a006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:00.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.950+0000 7fb3c2ee0640 1 --2- 172.21.15.154:0/1206844449 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb3a006af30 0x7fb3a006d3f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8036000 tx=0x7fb3a8030000 comp rx=0 tx=0).stop 2023-09-23T03:24:00.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.950+0000 7fb3c2ee0640 1 -- 172.21.15.154:0/1206844449 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3ac0047c0 msgr2=0x7fb3bc155a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:00.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.950+0000 7fb3c2ee0640 1 --2- 172.21.15.154:0/1206844449 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3ac0047c0 0x7fb3bc155a20 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fb3b000e730 tx=0x7fb3b000ebf0 comp rx=0 tx=0).stop 2023-09-23T03:24:00.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.950+0000 7fb3c2ee0640 1 -- 172.21.15.154:0/1206844449 shutdown_connections 2023-09-23T03:24:00.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.950+0000 7fb3c2ee0640 1 --2- 172.21.15.154:0/1206844449 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb3a006af30 0x7fb3a006d3f0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:00.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.950+0000 7fb3c2ee0640 1 --2- 172.21.15.154:0/1206844449 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb3bc15f450 0x7fb3bc15f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:00.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.950+0000 7fb3c2ee0640 1 --2- 172.21.15.154:0/1206844449 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3bc0c7c60 0x7fb3bc0afd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:00.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.950+0000 7fb3c2ee0640 1 --2- 172.21.15.154:0/1206844449 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb3ac0047c0 0x7fb3bc155a20 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:00.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.950+0000 7fb3c2ee0640 1 -- 172.21.15.154:0/1206844449 >> 172.21.15.154:0/1206844449 conn(0x7fb3bc0cc030 msgr2=0x7fb3bc0b18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:00.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.951+0000 7fb3c2ee0640 1 -- 172.21.15.154:0/1206844449 shutdown_connections 2023-09-23T03:24:00.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:00.951+0000 7fb3c2ee0640 1 -- 172.21.15.154:0/1206844449 wait complete. 2023-09-23T03:24:00.969 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_many_files_many_workers' 2023-09-23T03:24:01.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.181+0000 7fa43a176640 1 Processor -- start 2023-09-23T03:24:01.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.182+0000 7fa43a176640 1 -- start start 2023-09-23T03:24:01.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.183+0000 7fa43a176640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa4340cf040 0x7fa434160880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:01.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.183+0000 7fa43a176640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4340cd530 con 0x7fa434160dc0 2023-09-23T03:24:01.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.183+0000 7fa43a176640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4340cd810 con 0x7fa43415caf0 2023-09-23T03:24:01.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.183+0000 7fa43a176640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa43415ced0 con 0x7fa4340cf040 2023-09-23T03:24:01.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.183+0000 7fa433fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fa434160dc0 0x7fa4340b1360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46092/0 (socket says 172.21.15.154:46092) 2023-09-23T03:24:01.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.183+0000 7fa433fff640 1 -- 172.21.15.154:0/738548911 learned_addr learned my addr 172.21.15.154:0/738548911 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:24:01.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.184+0000 7fa432ffd640 1 --2- 172.21.15.154:0/738548911 >> v2:172.21.15.154:3300/0 conn(0x7fa4340cf040 0x7fa434160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:01.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.184+0000 7fa4327fc640 1 -- 172.21.15.154:0/738548911 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4043895626 0 0) 0x7fa4340cd530 con 0x7fa434160dc0 2023-09-23T03:24:01.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.184+0000 7fa4327fc640 1 -- 172.21.15.154:0/738548911 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa420003520 con 0x7fa434160dc0 2023-09-23T03:24:01.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.184+0000 7fa4327fc640 1 -- 172.21.15.154:0/738548911 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 846209646 0 0) 0x7fa420003520 con 0x7fa434160dc0 2023-09-23T03:24:01.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.184+0000 7fa4327fc640 1 -- 172.21.15.154:0/738548911 >> v2:172.21.15.154:3300/0 conn(0x7fa4340cf040 msgr2=0x7fa434160880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:01.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.184+0000 7fa4327fc640 1 --2- 172.21.15.154:0/738548911 >> v2:172.21.15.154:3300/0 conn(0x7fa4340cf040 0x7fa434160880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:01.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.184+0000 7fa4327fc640 1 -- 172.21.15.154:0/738548911 >> v1:172.21.15.190:6789/0 conn(0x7fa43415caf0 legacy=0x7fa4340ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:01.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.184+0000 7fa4327fc640 1 -- 172.21.15.154:0/738548911 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4340cdaf0 con 0x7fa434160dc0 2023-09-23T03:24:01.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.185+0000 7fa4327fc640 1 -- 172.21.15.154:0/738548911 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fa42c001670 con 0x7fa434160dc0 2023-09-23T03:24:01.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.185+0000 7fa4327fc640 1 -- 172.21.15.154:0/738548911 >> v1:172.21.15.154:6789/0 conn(0x7fa434160dc0 legacy=0x7fa4340b1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:01.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.185+0000 7fa4327fc640 1 --2- 172.21.15.154:0/738548911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa420003970 0x7fa420003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:01.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.185+0000 7fa4327fc640 1 --2- 172.21.15.154:0/738548911 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa420004370 0x7fa420004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:01.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.185+0000 7fa4327fc640 1 --2- 172.21.15.154:0/738548911 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa420005040 0x7fa42000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:01.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.185+0000 7fa4327fc640 1 -- 172.21.15.154:0/738548911 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43415ced0 con 0x7fa420005040 2023-09-23T03:24:01.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.185+0000 7fa4327fc640 1 -- 172.21.15.154:0/738548911 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42000bfa0 con 0x7fa420003970 2023-09-23T03:24:01.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.185+0000 7fa4327fc640 1 -- 172.21.15.154:0/738548911 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa42000c160 con 0x7fa420004370 2023-09-23T03:24:01.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.185+0000 7fa433fff640 1 --2- 172.21.15.154:0/738548911 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa420005040 0x7fa42000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:01.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.185+0000 7fa4337fe640 1 --2- 172.21.15.154:0/738548911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa420003970 0x7fa420003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:01.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.185+0000 7fa432ffd640 1 --2- 172.21.15.154:0/738548911 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa420004370 0x7fa420004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:01.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.186+0000 7fa433fff640 1 -- 172.21.15.154:0/738548911 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa420004370 msgr2=0x7fa420004840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:01.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.186+0000 7fa433fff640 1 --2- 172.21.15.154:0/738548911 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa420004370 0x7fa420004840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:01.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.186+0000 7fa433fff640 1 -- 172.21.15.154:0/738548911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa420003970 msgr2=0x7fa420003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:01.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.186+0000 7fa433fff640 1 --2- 172.21.15.154:0/738548911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa420003970 0x7fa420003d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:01.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.186+0000 7fa433fff640 1 -- 172.21.15.154:0/738548911 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa420003520 con 0x7fa420005040 2023-09-23T03:24:01.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.186+0000 7fa4337fe640 1 --2- 172.21.15.154:0/738548911 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa420003970 0x7fa420003d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:24:01.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.187+0000 7fa433fff640 1 -- 172.21.15.154:0/738548911 reap_dead start 2023-09-23T03:24:01.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.187+0000 7fa433fff640 1 --2- 172.21.15.154:0/738548911 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa420005040 0x7fa42000b980 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fa42c00c210 tx=0x7fa4200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6470b710b6440b8a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:01.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.188+0000 7fa4327fc640 1 -- 172.21.15.154:0/738548911 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42c005ad0 con 0x7fa420005040 2023-09-23T03:24:01.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.188+0000 7fa4327fc640 1 -- 172.21.15.154:0/738548911 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa42c005c50 con 0x7fa420005040 2023-09-23T03:24:01.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.188+0000 7fa4327fc640 1 -- 172.21.15.154:0/738548911 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42c02c8d0 con 0x7fa420005040 2023-09-23T03:24:01.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.188+0000 7fa43a176640 1 -- 172.21.15.154:0/738548911 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa420005040 msgr2=0x7fa42000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:01.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.188+0000 7fa43a176640 1 --2- 172.21.15.154:0/738548911 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa420005040 0x7fa42000b980 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fa42c00c210 tx=0x7fa4200057f0 comp rx=0 tx=0).stop 2023-09-23T03:24:01.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.188+0000 7fa43a176640 1 -- 172.21.15.154:0/738548911 shutdown_connections 2023-09-23T03:24:01.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.188+0000 7fa43a176640 1 --2- 172.21.15.154:0/738548911 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa420005040 0x7fa42000b980 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:01.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.188+0000 7fa43a176640 1 -- 172.21.15.154:0/738548911 >> 172.21.15.154:0/738548911 conn(0x7fa43406a0f0 msgr2=0x7fa43406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:01.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.188+0000 7fa43a176640 1 -- 172.21.15.154:0/738548911 shutdown_connections 2023-09-23T03:24:01.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.188+0000 7fa43a176640 1 -- 172.21.15.154:0/738548911 wait complete. 2023-09-23T03:24:01.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.190+0000 7fa43a176640 1 Processor -- start 2023-09-23T03:24:01.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.190+0000 7fa43a176640 1 -- start start 2023-09-23T03:24:01.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.191+0000 7fa43a176640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa420005040 0x7fa434155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:01.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.191+0000 7fa43a176640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa43415f770 0x7fa4340ae510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:01.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.191+0000 7fa43a176640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4340aea50 0x7fa434156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:01.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.191+0000 7fa43a176640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43415d770 con 0x7fa4340aea50 2023-09-23T03:24:01.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.191+0000 7fa43a176640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4340aef00 con 0x7fa420005040 2023-09-23T03:24:01.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.191+0000 7fa43a176640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4341557b0 con 0x7fa43415f770 2023-09-23T03:24:01.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.192+0000 7fa433fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4340aea50 0x7fa434156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:01.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.192+0000 7fa433fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4340aea50 0x7fa434156ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:51588/0 (socket says 172.21.15.154:51588) 2023-09-23T03:24:01.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.192+0000 7fa433fff640 1 -- 172.21.15.154:0/2785055862 learned_addr learned my addr 172.21.15.154:0/2785055862 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:01.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.192+0000 7fa4337fe640 1 --2- 172.21.15.154:0/2785055862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa420005040 0x7fa434155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:01.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.192+0000 7fa432ffd640 1 --2- 172.21.15.154:0/2785055862 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa43415f770 0x7fa4340ae510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:01.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.192+0000 7fa433fff640 1 -- 172.21.15.154:0/2785055862 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa43415f770 msgr2=0x7fa4340ae510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:01.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.192+0000 7fa433fff640 1 --2- 172.21.15.154:0/2785055862 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa43415f770 0x7fa4340ae510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:01.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.192+0000 7fa433fff640 1 -- 172.21.15.154:0/2785055862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa420005040 msgr2=0x7fa434155270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:01.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.192+0000 7fa433fff640 1 --2- 172.21.15.154:0/2785055862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa420005040 0x7fa434155270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:01.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.192+0000 7fa433fff640 1 -- 172.21.15.154:0/2785055862 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4340c1fd0 con 0x7fa4340aea50 2023-09-23T03:24:01.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.193+0000 7fa433fff640 1 --2- 172.21.15.154:0/2785055862 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4340aea50 0x7fa434156ba0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fa42c001eb0 tx=0x7fa42c001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:01.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.193+0000 7fa42bfff640 1 -- 172.21.15.154:0/2785055862 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42c0049e0 con 0x7fa4340aea50 2023-09-23T03:24:01.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.193+0000 7fa42bfff640 1 -- 172.21.15.154:0/2785055862 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa42c004b40 con 0x7fa4340aea50 2023-09-23T03:24:01.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.193+0000 7fa42bfff640 1 -- 172.21.15.154:0/2785055862 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42c007070 con 0x7fa4340aea50 2023-09-23T03:24:01.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.193+0000 7fa43a176640 1 -- 172.21.15.154:0/2785055862 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa43415c300 con 0x7fa4340aea50 2023-09-23T03:24:01.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.193+0000 7fa43a176640 1 -- 172.21.15.154:0/2785055862 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa43415d0b0 con 0x7fa4340aea50 2023-09-23T03:24:01.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.194+0000 7fa43a176640 1 -- 172.21.15.154:0/2785055862 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa434073eb0 con 0x7fa4340aea50 2023-09-23T03:24:01.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.195+0000 7fa42bfff640 1 -- 172.21.15.154:0/2785055862 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fa42c032070 con 0x7fa4340aea50 2023-09-23T03:24:01.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.196+0000 7fa42bfff640 1 --2- 172.21.15.154:0/2785055862 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa41406af60 0x7fa41406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:01.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.196+0000 7fa42bfff640 1 -- 172.21.15.154:0/2785055862 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5521+0+0 (secure 0 0 0) 0x7fa42c03c070 con 0x7fa4340aea50 2023-09-23T03:24:01.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.196+0000 7fa4337fe640 1 --2- 172.21.15.154:0/2785055862 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa41406af60 0x7fa41406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:01.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.197+0000 7fa4337fe640 1 --2- 172.21.15.154:0/2785055862 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa41406af60 0x7fa41406d420 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa424007fd0 tx=0x7fa424009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:01.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.201+0000 7fa42bfff640 1 -- 172.21.15.154:0/2785055862 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fa42c075780 con 0x7fa4340aea50 2023-09-23T03:24:01.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.464+0000 7fa43a176640 1 -- 172.21.15.154:0/2785055862 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_many_files_many_workers"]} v 0) v1 -- 0x7fa43406cb10 con 0x7fa4340aea50 2023-09-23T03:24:01.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.503+0000 7fa42bfff640 1 -- 172.21.15.154:0/2785055862 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_many_files_many_workers"]}]=0 v384) v1 ==== 172+0+0 (secure 0 0 0) 0x7fa42c078da0 con 0x7fa4340aea50 2023-09-23T03:24:01.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.508+0000 7fa43a176640 1 -- 172.21.15.154:0/2785055862 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa41406af60 msgr2=0x7fa41406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:01.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.508+0000 7fa43a176640 1 --2- 172.21.15.154:0/2785055862 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa41406af60 0x7fa41406d420 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa424007fd0 tx=0x7fa424009040 comp rx=0 tx=0).stop 2023-09-23T03:24:01.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.508+0000 7fa43a176640 1 -- 172.21.15.154:0/2785055862 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4340aea50 msgr2=0x7fa434156ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:01.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.508+0000 7fa43a176640 1 --2- 172.21.15.154:0/2785055862 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4340aea50 0x7fa434156ba0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fa42c001eb0 tx=0x7fa42c001a00 comp rx=0 tx=0).stop 2023-09-23T03:24:01.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.508+0000 7fa43a176640 1 -- 172.21.15.154:0/2785055862 shutdown_connections 2023-09-23T03:24:01.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.508+0000 7fa43a176640 1 --2- 172.21.15.154:0/2785055862 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa41406af60 0x7fa41406d420 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:01.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.508+0000 7fa43a176640 1 --2- 172.21.15.154:0/2785055862 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa4340aea50 0x7fa434156ba0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:01.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.508+0000 7fa43a176640 1 --2- 172.21.15.154:0/2785055862 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa43415f770 0x7fa4340ae510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:01.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.508+0000 7fa43a176640 1 --2- 172.21.15.154:0/2785055862 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa420005040 0x7fa434155270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:01.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.508+0000 7fa43a176640 1 -- 172.21.15.154:0/2785055862 >> 172.21.15.154:0/2785055862 conn(0x7fa43406a0f0 msgr2=0x7fa4340ce900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:01.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.509+0000 7fa43a176640 1 -- 172.21.15.154:0/2785055862 shutdown_connections 2023-09-23T03:24:01.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.509+0000 7fa43a176640 1 -- 172.21.15.154:0/2785055862 wait complete. 2023-09-23T03:24:01.527 INFO:tasks.cephfs_test_runner:test_extra_data_pool_rebuild_many_files_many_workers (tasks.cephfs.test_data_scan.TestDataScan) ... ok 2023-09-23T03:24:01.527 INFO:tasks.cephfs_test_runner:Starting test: test_extra_data_pool_rebuild_simple (tasks.cephfs.test_data_scan.TestDataScan) 2023-09-23T03:24:01.528 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:24:01.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.743+0000 7fdb8a2f3640 1 Processor -- start 2023-09-23T03:24:01.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.743+0000 7fdb8a2f3640 1 -- start start 2023-09-23T03:24:01.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.744+0000 7fdb8a2f3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb840ada30 0x7fdb840ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:01.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.744+0000 7fdb8a2f3640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdb840cf130 0x7fdb8415d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:01.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.744+0000 7fdb8a2f3640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb8406ceb0 con 0x7fdb840b2fe0 2023-09-23T03:24:01.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.744+0000 7fdb8a2f3640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb840ade10 con 0x7fdb840cf130 2023-09-23T03:24:01.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.744+0000 7fdb8a2f3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb84159c90 con 0x7fdb840ada30 2023-09-23T03:24:01.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.744+0000 7fdb88890640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdb840cf130 0x7fdb8415d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:01.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.745+0000 7fdb88890640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdb840cf130 0x7fdb8415d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:51602/0 (socket says 172.21.15.154:51602) 2023-09-23T03:24:01.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.745+0000 7fdb88890640 1 -- 172.21.15.154:0/1758486677 learned_addr learned my addr 172.21.15.154:0/1758486677 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:01.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.745+0000 7fdb88890640 1 -- 172.21.15.154:0/1758486677 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb840ada30 msgr2=0x7fdb840ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:01.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.745+0000 7fdb88890640 1 --2- 172.21.15.154:0/1758486677 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb840ada30 0x7fdb840ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:01.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.745+0000 7fdb88890640 1 -- 172.21.15.154:0/1758486677 >> v1:172.21.15.190:6789/0 conn(0x7fdb840b2fe0 legacy=0x7fdb840cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:01.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.745+0000 7fdb88890640 1 -- 172.21.15.154:0/1758486677 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb8406c110 con 0x7fdb840cf130 2023-09-23T03:24:01.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.746+0000 7fdb88890640 1 --2- 172.21.15.154:0/1758486677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb840cf130 0x7fdb8415d380 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c009870 tx=0x7fdb7c02db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee9c2b50e379784 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:01.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.746+0000 7fdb82ffd640 1 -- 172.21.15.154:0/1758486677 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c02f8c0 con 0x7fdb840cf130 2023-09-23T03:24:01.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.746+0000 7fdb82ffd640 1 -- 172.21.15.154:0/1758486677 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb7c02fa20 con 0x7fdb840cf130 2023-09-23T03:24:01.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.746+0000 7fdb82ffd640 1 -- 172.21.15.154:0/1758486677 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c02fdb0 con 0x7fdb840cf130 2023-09-23T03:24:01.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.747+0000 7fdb8a2f3640 1 -- 172.21.15.154:0/1758486677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb840cf130 msgr2=0x7fdb8415d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:01.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.747+0000 7fdb8a2f3640 1 --2- 172.21.15.154:0/1758486677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb840cf130 0x7fdb8415d380 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c009870 tx=0x7fdb7c02db00 comp rx=0 tx=0).stop 2023-09-23T03:24:01.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.747+0000 7fdb8a2f3640 1 -- 172.21.15.154:0/1758486677 shutdown_connections 2023-09-23T03:24:01.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.747+0000 7fdb8a2f3640 1 --2- 172.21.15.154:0/1758486677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb840cf130 0x7fdb8415d380 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:01.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.747+0000 7fdb8a2f3640 1 --2- 172.21.15.154:0/1758486677 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb840ada30 0x7fdb840ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:01.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.747+0000 7fdb8a2f3640 1 -- 172.21.15.154:0/1758486677 >> 172.21.15.154:0/1758486677 conn(0x7fdb8406a0f0 msgr2=0x7fdb8406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:01.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.747+0000 7fdb8a2f3640 1 -- 172.21.15.154:0/1758486677 shutdown_connections 2023-09-23T03:24:01.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.747+0000 7fdb8a2f3640 1 -- 172.21.15.154:0/1758486677 wait complete. 2023-09-23T03:24:01.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.749+0000 7fdb8a2f3640 1 Processor -- start 2023-09-23T03:24:01.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.749+0000 7fdb8a2f3640 1 -- start start 2023-09-23T03:24:01.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.750+0000 7fdb8a2f3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb840ada30 0x7fdb84157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:01.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.750+0000 7fdb8a2f3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb840b2fe0 0x7fdb840982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:01.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.750+0000 7fdb8a2f3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb840cf130 0x7fdb840c4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:01.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.750+0000 7fdb8a2f3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb840c2c00 con 0x7fdb840ada30 2023-09-23T03:24:01.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.750+0000 7fdb8a2f3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb840c2e80 con 0x7fdb840b2fe0 2023-09-23T03:24:01.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.750+0000 7fdb8a2f3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb840c3160 con 0x7fdb840cf130 2023-09-23T03:24:01.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.750+0000 7fdb83fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb840ada30 0x7fdb84157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:01.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.750+0000 7fdb83fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb840ada30 0x7fdb84157ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:51608/0 (socket says 172.21.15.154:51608) 2023-09-23T03:24:01.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.750+0000 7fdb83fff640 1 -- 172.21.15.154:0/3375380049 learned_addr learned my addr 172.21.15.154:0/3375380049 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:01.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.751+0000 7fdb83fff640 1 -- 172.21.15.154:0/3375380049 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb840cf130 msgr2=0x7fdb840c4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:01.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.751+0000 7fdb83fff640 1 --2- 172.21.15.154:0/3375380049 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb840cf130 0x7fdb840c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:01.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.751+0000 7fdb83fff640 1 -- 172.21.15.154:0/3375380049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb840b2fe0 msgr2=0x7fdb840982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:01.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.751+0000 7fdb83fff640 1 --2- 172.21.15.154:0/3375380049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb840b2fe0 0x7fdb840982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:01.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.751+0000 7fdb83fff640 1 -- 172.21.15.154:0/3375380049 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb7c0094d0 con 0x7fdb840ada30 2023-09-23T03:24:01.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.751+0000 7fdb83fff640 1 --2- 172.21.15.154:0/3375380049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb840ada30 0x7fdb84157ef0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fdb7400ba50 tx=0x7fdb7400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:01.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.751+0000 7fdb80ff9640 1 -- 172.21.15.154:0/3375380049 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7400cbe0 con 0x7fdb840ada30 2023-09-23T03:24:01.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.751+0000 7fdb80ff9640 1 -- 172.21.15.154:0/3375380049 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb7400cd60 con 0x7fdb840ada30 2023-09-23T03:24:01.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.751+0000 7fdb80ff9640 1 -- 172.21.15.154:0/3375380049 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb74014680 con 0x7fdb840ada30 2023-09-23T03:24:01.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.751+0000 7fdb8a2f3640 1 -- 172.21.15.154:0/3375380049 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb840c3440 con 0x7fdb840ada30 2023-09-23T03:24:01.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.751+0000 7fdb8a2f3640 1 -- 172.21.15.154:0/3375380049 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb84158430 con 0x7fdb840ada30 2023-09-23T03:24:01.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.752+0000 7fdb8a2f3640 1 -- 172.21.15.154:0/3375380049 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb840c3960 con 0x7fdb840ada30 2023-09-23T03:24:01.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.754+0000 7fdb80ff9640 1 -- 172.21.15.154:0/3375380049 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fdb7401d3b0 con 0x7fdb840ada30 2023-09-23T03:24:01.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.754+0000 7fdb80ff9640 1 --2- 172.21.15.154:0/3375380049 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdb6406af30 0x7fdb6406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:01.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.754+0000 7fdb80ff9640 1 -- 172.21.15.154:0/3375380049 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5521+0+0 (secure 0 0 0) 0x7fdb740897d0 con 0x7fdb840ada30 2023-09-23T03:24:01.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.755+0000 7fdb837fe640 1 --2- 172.21.15.154:0/3375380049 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdb6406af30 0x7fdb6406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:01.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.756+0000 7fdb837fe640 1 --2- 172.21.15.154:0/3375380049 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdb6406af30 0x7fdb6406d3f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fdb840af460 tx=0x7fdb6c007990 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:01.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.760+0000 7fdb80ff9640 1 -- 172.21.15.154:0/3375380049 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fdb74054790 con 0x7fdb840ada30 2023-09-23T03:24:01.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.951+0000 7fdb8a2f3640 1 -- 172.21.15.154:0/3375380049 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb84073eb0 con 0x7fdb840ada30 2023-09-23T03:24:01.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.953+0000 7fdb80ff9640 1 -- 172.21.15.154:0/3375380049 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+13626 (secure 0 0 0) 0x7fdb74019070 con 0x7fdb840ada30 2023-09-23T03:24:01.954 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:01.954 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":75,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:24:00.488723+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6835/2633473586":"2023-09-24T03:23:56.284319+0000","172.21.15.154:6834/2633473586":"2023-09-24T03:23:56.284319+0000","172.21.15.154:6837/4129414148":"2023-09-24T03:21:38.278599+0000","172.21.15.154:6835/4129414148":"2023-09-24T03:21:38.278599+0000","172.21.15.154:6837/2976977273":"2023-09-24T03:21:31.069171+0000","172.21.15.154:6835/2976977273":"2023-09-24T03:21:31.069171+0000","172.21.15.154:6837/2183023203":"2023-09-24T03:21:29.032871+0000","172.21.15.154:6835/2183023203":"2023-09-24T03:21:29.032871+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:24:01.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.956+0000 7fdb8a2f3640 1 -- 172.21.15.154:0/3375380049 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdb6406af30 msgr2=0x7fdb6406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:01.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.956+0000 7fdb8a2f3640 1 --2- 172.21.15.154:0/3375380049 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdb6406af30 0x7fdb6406d3f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fdb840af460 tx=0x7fdb6c007990 comp rx=0 tx=0).stop 2023-09-23T03:24:01.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.956+0000 7fdb8a2f3640 1 -- 172.21.15.154:0/3375380049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb840ada30 msgr2=0x7fdb84157ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:01.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.956+0000 7fdb8a2f3640 1 --2- 172.21.15.154:0/3375380049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb840ada30 0x7fdb84157ef0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fdb7400ba50 tx=0x7fdb7400bf10 comp rx=0 tx=0).stop 2023-09-23T03:24:01.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.956+0000 7fdb8a2f3640 1 -- 172.21.15.154:0/3375380049 shutdown_connections 2023-09-23T03:24:01.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.956+0000 7fdb8a2f3640 1 --2- 172.21.15.154:0/3375380049 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdb6406af30 0x7fdb6406d3f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:01.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.956+0000 7fdb8a2f3640 1 --2- 172.21.15.154:0/3375380049 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb840cf130 0x7fdb840c4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:01.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.956+0000 7fdb8a2f3640 1 --2- 172.21.15.154:0/3375380049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb840b2fe0 0x7fdb840982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:01.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.956+0000 7fdb8a2f3640 1 --2- 172.21.15.154:0/3375380049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb840ada30 0x7fdb84157ef0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:01.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.956+0000 7fdb8a2f3640 1 -- 172.21.15.154:0/3375380049 >> 172.21.15.154:0/3375380049 conn(0x7fdb8406a0f0 msgr2=0x7fdb840b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:01.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.956+0000 7fdb8a2f3640 1 -- 172.21.15.154:0/3375380049 shutdown_connections 2023-09-23T03:24:01.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:01.956+0000 7fdb8a2f3640 1 -- 172.21.15.154:0/3375380049 wait complete. 2023-09-23T03:24:01.975 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T03:24:01.975 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:24:02.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.190+0000 7ff49c826640 1 Processor -- start 2023-09-23T03:24:02.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.191+0000 7ff49c826640 1 -- start start 2023-09-23T03:24:02.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.192+0000 7ff49c826640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff4980c8d70 0x7ff49815c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:02.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.192+0000 7ff49c826640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff498071290 0x7ff4980ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:02.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.192+0000 7ff49c826640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4980c9d20 con 0x7ff4980c9150 2023-09-23T03:24:02.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.192+0000 7ff49c826640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff498071850 con 0x7ff498071290 2023-09-23T03:24:02.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.192+0000 7ff49c826640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff498071b30 con 0x7ff4980c8d70 2023-09-23T03:24:02.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.192+0000 7ff496d9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff498071290 0x7ff4980ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:02.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.193+0000 7ff496d9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff498071290 0x7ff4980ca1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:51622/0 (socket says 172.21.15.154:51622) 2023-09-23T03:24:02.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.193+0000 7ff496d9d640 1 -- 172.21.15.154:0/4255092573 learned_addr learned my addr 172.21.15.154:0/4255092573 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:02.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.193+0000 7ff49659c640 1 --2- 172.21.15.154:0/4255092573 >> v2:172.21.15.190:3300/0 conn(0x7ff4980c8d70 0x7ff49815c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:02.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.193+0000 7ff496d9d640 1 -- 172.21.15.154:0/4255092573 >> v2:172.21.15.190:3300/0 conn(0x7ff4980c8d70 msgr2=0x7ff49815c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:02.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.193+0000 7ff496d9d640 1 --2- 172.21.15.154:0/4255092573 >> v2:172.21.15.190:3300/0 conn(0x7ff4980c8d70 0x7ff49815c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:02.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.193+0000 7ff496d9d640 1 -- 172.21.15.154:0/4255092573 >> v1:172.21.15.190:6789/0 conn(0x7ff4980c9150 legacy=0x7ff49815cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:02.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.193+0000 7ff496d9d640 1 -- 172.21.15.154:0/4255092573 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff498071e10 con 0x7ff498071290 2023-09-23T03:24:02.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.194+0000 7ff496d9d640 1 --2- 172.21.15.154:0/4255092573 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff498071290 0x7ff4980ca1b0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7ff49806b790 tx=0x7ff49002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d40fd0216438503 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:02.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.194+0000 7ff495d9b640 1 -- 172.21.15.154:0/4255092573 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49002ca80 con 0x7ff498071290 2023-09-23T03:24:02.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.194+0000 7ff495d9b640 1 -- 172.21.15.154:0/4255092573 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49002cbe0 con 0x7ff498071290 2023-09-23T03:24:02.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.194+0000 7ff495d9b640 1 -- 172.21.15.154:0/4255092573 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4900343f0 con 0x7ff498071290 2023-09-23T03:24:02.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.195+0000 7ff49c826640 1 -- 172.21.15.154:0/4255092573 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff498071290 msgr2=0x7ff4980ca1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:02.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.195+0000 7ff49c826640 1 --2- 172.21.15.154:0/4255092573 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff498071290 0x7ff4980ca1b0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7ff49806b790 tx=0x7ff49002bee0 comp rx=0 tx=0).stop 2023-09-23T03:24:02.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.195+0000 7ff49c826640 1 -- 172.21.15.154:0/4255092573 shutdown_connections 2023-09-23T03:24:02.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.195+0000 7ff49c826640 1 --2- 172.21.15.154:0/4255092573 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff498071290 0x7ff4980ca1b0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:02.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.195+0000 7ff49c826640 1 --2- 172.21.15.154:0/4255092573 >> v2:172.21.15.190:3300/0 conn(0x7ff4980c8d70 0x7ff49815c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:02.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.195+0000 7ff49c826640 1 -- 172.21.15.154:0/4255092573 >> 172.21.15.154:0/4255092573 conn(0x7ff49806c010 msgr2=0x7ff49806c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:02.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.195+0000 7ff49c826640 1 -- 172.21.15.154:0/4255092573 shutdown_connections 2023-09-23T03:24:02.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.195+0000 7ff49c826640 1 -- 172.21.15.154:0/4255092573 wait complete. 2023-09-23T03:24:02.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.197+0000 7ff49c826640 1 Processor -- start 2023-09-23T03:24:02.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.197+0000 7ff49c826640 1 -- start start 2023-09-23T03:24:02.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.198+0000 7ff49c826640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff498071290 0x7ff4980cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:02.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.198+0000 7ff49c826640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4980c8d70 0x7ff498147010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:02.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.198+0000 7ff49c826640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4980c9150 0x7ff498147550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:02.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.198+0000 7ff49c826640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4980cf0e0 con 0x7ff4980c8d70 2023-09-23T03:24:02.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.198+0000 7ff49c826640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4980cde10 con 0x7ff4980c9150 2023-09-23T03:24:02.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.198+0000 7ff49c826640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4980ce040 con 0x7ff498071290 2023-09-23T03:24:02.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.198+0000 7ff48ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4980c8d70 0x7ff498147010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:02.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.199+0000 7ff48ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4980c8d70 0x7ff498147010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:51634/0 (socket says 172.21.15.154:51634) 2023-09-23T03:24:02.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.199+0000 7ff48ffff640 1 -- 172.21.15.154:0/2844432240 learned_addr learned my addr 172.21.15.154:0/2844432240 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:02.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.199+0000 7ff496d9d640 1 --2- 172.21.15.154:0/2844432240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4980c9150 0x7ff498147550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:02.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.199+0000 7ff48ffff640 1 -- 172.21.15.154:0/2844432240 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff498071290 msgr2=0x7ff4980cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:02.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.199+0000 7ff48ffff640 1 --2- 172.21.15.154:0/2844432240 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff498071290 0x7ff4980cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:02.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.199+0000 7ff48ffff640 1 -- 172.21.15.154:0/2844432240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4980c9150 msgr2=0x7ff498147550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:02.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.199+0000 7ff48ffff640 1 --2- 172.21.15.154:0/2844432240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4980c9150 0x7ff498147550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:02.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.199+0000 7ff48ffff640 1 -- 172.21.15.154:0/2844432240 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff490009b30 con 0x7ff4980c8d70 2023-09-23T03:24:02.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.199+0000 7ff48ffff640 1 --2- 172.21.15.154:0/2844432240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4980c8d70 0x7ff498147010 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7ff48000b420 tx=0x7ff48000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:02.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.200+0000 7ff48effd640 1 -- 172.21.15.154:0/2844432240 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff480011830 con 0x7ff4980c8d70 2023-09-23T03:24:02.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.200+0000 7ff48effd640 1 -- 172.21.15.154:0/2844432240 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff480011990 con 0x7ff4980c8d70 2023-09-23T03:24:02.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.200+0000 7ff48effd640 1 -- 172.21.15.154:0/2844432240 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff480011d40 con 0x7ff4980c8d70 2023-09-23T03:24:02.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.200+0000 7ff49c826640 1 -- 172.21.15.154:0/2844432240 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4980ce320 con 0x7ff4980c8d70 2023-09-23T03:24:02.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.200+0000 7ff49c826640 1 -- 172.21.15.154:0/2844432240 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4980ce8e0 con 0x7ff4980c8d70 2023-09-23T03:24:02.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.201+0000 7ff49c826640 1 -- 172.21.15.154:0/2844432240 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4980cb3e0 con 0x7ff4980c8d70 2023-09-23T03:24:02.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.202+0000 7ff48effd640 1 -- 172.21.15.154:0/2844432240 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7ff48001b460 con 0x7ff4980c8d70 2023-09-23T03:24:02.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.203+0000 7ff48effd640 1 --2- 172.21.15.154:0/2844432240 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff47806af30 0x7ff47806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:02.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.203+0000 7ff48effd640 1 -- 172.21.15.154:0/2844432240 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5521+0+0 (secure 0 0 0) 0x7ff4800893a0 con 0x7ff4980c8d70 2023-09-23T03:24:02.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.203+0000 7ff49659c640 1 --2- 172.21.15.154:0/2844432240 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff47806af30 0x7ff47806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:02.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.204+0000 7ff49659c640 1 --2- 172.21.15.154:0/2844432240 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff47806af30 0x7ff47806d3f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff4880098f0 tx=0x7ff488008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:02.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.207+0000 7ff48effd640 1 -- 172.21.15.154:0/2844432240 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7ff480054360 con 0x7ff4980c8d70 2023-09-23T03:24:02.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.403+0000 7ff49c826640 1 -- 172.21.15.154:0/2844432240 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff49804e920 con 0x7ff4980c8d70 2023-09-23T03:24:02.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.404+0000 7ff48effd640 1 -- 172.21.15.154:0/2844432240 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff480057980 con 0x7ff4980c8d70 2023-09-23T03:24:02.405 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:24:02.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.407+0000 7ff49c826640 1 -- 172.21.15.154:0/2844432240 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff47806af30 msgr2=0x7ff47806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:02.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.407+0000 7ff49c826640 1 --2- 172.21.15.154:0/2844432240 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff47806af30 0x7ff47806d3f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff4880098f0 tx=0x7ff488008040 comp rx=0 tx=0).stop 2023-09-23T03:24:02.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.407+0000 7ff49c826640 1 -- 172.21.15.154:0/2844432240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4980c8d70 msgr2=0x7ff498147010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:02.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.407+0000 7ff49c826640 1 --2- 172.21.15.154:0/2844432240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4980c8d70 0x7ff498147010 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7ff48000b420 tx=0x7ff48000b8e0 comp rx=0 tx=0).stop 2023-09-23T03:24:02.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.407+0000 7ff49c826640 1 -- 172.21.15.154:0/2844432240 shutdown_connections 2023-09-23T03:24:02.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.407+0000 7ff49c826640 1 --2- 172.21.15.154:0/2844432240 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff47806af30 0x7ff47806d3f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:02.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.407+0000 7ff49c826640 1 --2- 172.21.15.154:0/2844432240 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff4980c9150 0x7ff498147550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:02.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.407+0000 7ff49c826640 1 --2- 172.21.15.154:0/2844432240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff4980c8d70 0x7ff498147010 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:02.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.407+0000 7ff49c826640 1 --2- 172.21.15.154:0/2844432240 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff498071290 0x7ff4980cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:02.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.407+0000 7ff49c826640 1 -- 172.21.15.154:0/2844432240 >> 172.21.15.154:0/2844432240 conn(0x7ff49806c010 msgr2=0x7ff4980cacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:02.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.407+0000 7ff49c826640 1 -- 172.21.15.154:0/2844432240 shutdown_connections 2023-09-23T03:24:02.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.407+0000 7ff49c826640 1 -- 172.21.15.154:0/2844432240 wait complete. 2023-09-23T03:24:02.427 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_simple' 2023-09-23T03:24:02.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.644+0000 7f8739ce0640 1 Processor -- start 2023-09-23T03:24:02.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.645+0000 7f8739ce0640 1 -- start start 2023-09-23T03:24:02.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.646+0000 7f8739ce0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f873415cf40 0x7f87341615b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:02.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.646+0000 7f8739ce0640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f87340b28e0 0x7f87340cca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:02.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.646+0000 7f8739ce0640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87340ccf80 con 0x7f873406b1e0 2023-09-23T03:24:02.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.646+0000 7f8739ce0640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87340cd1c0 con 0x7f87340b28e0 2023-09-23T03:24:02.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.646+0000 7f8739ce0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87340cd4a0 con 0x7f873415cf40 2023-09-23T03:24:02.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.646+0000 7f8733fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f87340b28e0 0x7f87340cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:02.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.647+0000 7f8733fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f87340b28e0 0x7f87340cca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:51640/0 (socket says 172.21.15.154:51640) 2023-09-23T03:24:02.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.647+0000 7f8733fff640 1 -- 172.21.15.154:0/4115970073 learned_addr learned my addr 172.21.15.154:0/4115970073 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:02.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.647+0000 7f8733fff640 1 -- 172.21.15.154:0/4115970073 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f873415cf40 msgr2=0x7f87341615b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:02.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.647+0000 7f8733fff640 1 --2- 172.21.15.154:0/4115970073 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f873415cf40 0x7f87341615b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:02.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.647+0000 7f8733fff640 1 -- 172.21.15.154:0/4115970073 >> v1:172.21.15.190:6789/0 conn(0x7f873406b1e0 legacy=0x7f87340b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:02.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.647+0000 7f8733fff640 1 -- 172.21.15.154:0/4115970073 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87340cd780 con 0x7f87340b28e0 2023-09-23T03:24:02.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.648+0000 7f8733fff640 1 --2- 172.21.15.154:0/4115970073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87340b28e0 0x7f87340cca40 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f872c009870 tx=0x7f872c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac995ba85e673c9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:02.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.648+0000 7f87327fc640 1 -- 172.21.15.154:0/4115970073 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c031640 con 0x7f87340b28e0 2023-09-23T03:24:02.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.648+0000 7f87327fc640 1 -- 172.21.15.154:0/4115970073 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872c0317a0 con 0x7f87340b28e0 2023-09-23T03:24:02.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.648+0000 7f87327fc640 1 -- 172.21.15.154:0/4115970073 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c031b30 con 0x7f87340b28e0 2023-09-23T03:24:02.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.649+0000 7f8739ce0640 1 -- 172.21.15.154:0/4115970073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87340b28e0 msgr2=0x7f87340cca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:02.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.649+0000 7f8739ce0640 1 --2- 172.21.15.154:0/4115970073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87340b28e0 0x7f87340cca40 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f872c009870 tx=0x7f872c02b8f0 comp rx=0 tx=0).stop 2023-09-23T03:24:02.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.649+0000 7f8739ce0640 1 -- 172.21.15.154:0/4115970073 shutdown_connections 2023-09-23T03:24:02.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.649+0000 7f8739ce0640 1 --2- 172.21.15.154:0/4115970073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87340b28e0 0x7f87340cca40 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:02.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.649+0000 7f8739ce0640 1 --2- 172.21.15.154:0/4115970073 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f873415cf40 0x7f87341615b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:02.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.649+0000 7f8739ce0640 1 -- 172.21.15.154:0/4115970073 >> 172.21.15.154:0/4115970073 conn(0x7f873406fc60 msgr2=0x7f8734070040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:02.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.649+0000 7f8739ce0640 1 -- 172.21.15.154:0/4115970073 shutdown_connections 2023-09-23T03:24:02.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.649+0000 7f8739ce0640 1 -- 172.21.15.154:0/4115970073 wait complete. 2023-09-23T03:24:02.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.651+0000 7f8739ce0640 1 Processor -- start 2023-09-23T03:24:02.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.651+0000 7f8739ce0640 1 -- start start 2023-09-23T03:24:02.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.652+0000 7f8739ce0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f873406b1e0 0x7f87340b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:02.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.652+0000 7f8739ce0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87340b28e0 0x7f87340cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:02.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.652+0000 7f8739ce0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f873415cf40 0x7f87340cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:02.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.652+0000 7f8739ce0640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87341f3ad0 con 0x7f873406b1e0 2023-09-23T03:24:02.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.652+0000 7f8739ce0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87341f70a0 con 0x7f87340b28e0 2023-09-23T03:24:02.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.652+0000 7f8739ce0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87341f62e0 con 0x7f873415cf40 2023-09-23T03:24:02.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.652+0000 7f87337fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f873406b1e0 0x7f87340b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:02.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.652+0000 7f87337fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f873406b1e0 0x7f87340b45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:51646/0 (socket says 172.21.15.154:51646) 2023-09-23T03:24:02.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.652+0000 7f87337fe640 1 -- 172.21.15.154:0/1905813294 learned_addr learned my addr 172.21.15.154:0/1905813294 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:02.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.653+0000 7f8732ffd640 1 --2- 172.21.15.154:0/1905813294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87340b28e0 0x7f87340cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:02.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.653+0000 7f8733fff640 1 --2- 172.21.15.154:0/1905813294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f873415cf40 0x7f87340cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:02.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.653+0000 7f87337fe640 1 -- 172.21.15.154:0/1905813294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f873415cf40 msgr2=0x7f87340cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:02.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.653+0000 7f87337fe640 1 --2- 172.21.15.154:0/1905813294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f873415cf40 0x7f87340cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:02.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.653+0000 7f87337fe640 1 -- 172.21.15.154:0/1905813294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87340b28e0 msgr2=0x7f87340cbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:02.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.653+0000 7f87337fe640 1 --2- 172.21.15.154:0/1905813294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87340b28e0 0x7f87340cbfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:02.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.653+0000 7f87337fe640 1 -- 172.21.15.154:0/1905813294 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f872c0094d0 con 0x7f873406b1e0 2023-09-23T03:24:02.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.653+0000 7f8732ffd640 1 --2- 172.21.15.154:0/1905813294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87340b28e0 0x7f87340cbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:02.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.653+0000 7f87337fe640 1 --2- 172.21.15.154:0/1905813294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f873406b1e0 0x7f87340b45c0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f872400ba50 tx=0x7f872400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:02.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.653+0000 7f872bfff640 1 -- 172.21.15.154:0/1905813294 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872400cbe0 con 0x7f873406b1e0 2023-09-23T03:24:02.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.653+0000 7f872bfff640 1 -- 172.21.15.154:0/1905813294 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872400cd60 con 0x7f873406b1e0 2023-09-23T03:24:02.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.653+0000 7f872bfff640 1 -- 172.21.15.154:0/1905813294 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8724014680 con 0x7f873406b1e0 2023-09-23T03:24:02.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.653+0000 7f8739ce0640 1 -- 172.21.15.154:0/1905813294 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87341f65c0 con 0x7f873406b1e0 2023-09-23T03:24:02.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.654+0000 7f8739ce0640 1 -- 172.21.15.154:0/1905813294 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87341f6b80 con 0x7f873406b1e0 2023-09-23T03:24:02.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.655+0000 7f8739ce0640 1 -- 172.21.15.154:0/1905813294 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87341f6d90 con 0x7f873406b1e0 2023-09-23T03:24:02.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.656+0000 7f872bfff640 1 -- 172.21.15.154:0/1905813294 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f872401d3b0 con 0x7f873406b1e0 2023-09-23T03:24:02.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.657+0000 7f872bfff640 1 --2- 172.21.15.154:0/1905813294 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f871406af30 0x7f871406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:02.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.657+0000 7f872bfff640 1 -- 172.21.15.154:0/1905813294 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5521+0+0 (secure 0 0 0) 0x7f87240896b0 con 0x7f873406b1e0 2023-09-23T03:24:02.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.657+0000 7f8732ffd640 1 --2- 172.21.15.154:0/1905813294 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f871406af30 0x7f871406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:02.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.658+0000 7f8732ffd640 1 --2- 172.21.15.154:0/1905813294 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f871406af30 0x7f871406d3f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f871c007fd0 tx=0x7f871c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:02.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.662+0000 7f872bfff640 1 -- 172.21.15.154:0/1905813294 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f8724054790 con 0x7f873406b1e0 2023-09-23T03:24:02.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:02.924+0000 7f8739ce0640 1 -- 172.21.15.154:0/1905813294 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_simple"]} v 0) v1 -- 0x7f8734073eb0 con 0x7f873406b1e0 2023-09-23T03:24:03.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.516+0000 7f872bfff640 1 -- 172.21.15.154:0/1905813294 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_simple"]}]=0 v386) v1 ==== 158+0+0 (secure 0 0 0) 0x7f87240190e0 con 0x7f873406b1e0 2023-09-23T03:24:03.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.520+0000 7f8739ce0640 1 -- 172.21.15.154:0/1905813294 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f871406af30 msgr2=0x7f871406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:03.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.520+0000 7f8739ce0640 1 --2- 172.21.15.154:0/1905813294 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f871406af30 0x7f871406d3f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f871c007fd0 tx=0x7f871c009040 comp rx=0 tx=0).stop 2023-09-23T03:24:03.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.520+0000 7f8739ce0640 1 -- 172.21.15.154:0/1905813294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f873406b1e0 msgr2=0x7f87340b45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:03.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.520+0000 7f8739ce0640 1 --2- 172.21.15.154:0/1905813294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f873406b1e0 0x7f87340b45c0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f872400ba50 tx=0x7f872400bf10 comp rx=0 tx=0).stop 2023-09-23T03:24:03.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.520+0000 7f8739ce0640 1 -- 172.21.15.154:0/1905813294 shutdown_connections 2023-09-23T03:24:03.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.520+0000 7f8739ce0640 1 --2- 172.21.15.154:0/1905813294 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f871406af30 0x7f871406d3f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:03.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.520+0000 7f8739ce0640 1 --2- 172.21.15.154:0/1905813294 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f873415cf40 0x7f87340cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:03.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.520+0000 7f8739ce0640 1 --2- 172.21.15.154:0/1905813294 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87340b28e0 0x7f87340cbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:03.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.520+0000 7f8739ce0640 1 --2- 172.21.15.154:0/1905813294 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f873406b1e0 0x7f87340b45c0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:03.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.520+0000 7f8739ce0640 1 -- 172.21.15.154:0/1905813294 >> 172.21.15.154:0/1905813294 conn(0x7f873406fc60 msgr2=0x7f873415dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:03.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.520+0000 7f8739ce0640 1 -- 172.21.15.154:0/1905813294 shutdown_connections 2023-09-23T03:24:03.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.520+0000 7f8739ce0640 1 -- 172.21.15.154:0/1905813294 wait complete. 2023-09-23T03:24:03.543 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-09-23T03:24:03.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.760+0000 7f0eb8f97640 1 Processor -- start 2023-09-23T03:24:03.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.760+0000 7f0eb8f97640 1 -- start start 2023-09-23T03:24:03.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.762+0000 7f0eb8f97640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0eb415aba0 0x7f0eb415bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:03.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.762+0000 7f0eb8f97640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0eb415e5f0 0x7f0eb415ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:03.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.762+0000 7f0eb8f97640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0eb40caea0 0x7f0eb40cb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:03.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.762+0000 7f0eb8f97640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eb40cb870 con 0x7f0eb415aba0 2023-09-23T03:24:03.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.762+0000 7f0eb8f97640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eb415b6a0 con 0x7f0eb40caea0 2023-09-23T03:24:03.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.762+0000 7f0eb8f97640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb415f0b0 con 0x7f0eb415e5f0 2023-09-23T03:24:03.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.762+0000 7f0eb259c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0eb415aba0 0x7f0eb415bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:03.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.762+0000 7f0eb259c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0eb415aba0 0x7f0eb415bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:51654/0 (socket says 172.21.15.154:51654) 2023-09-23T03:24:03.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.762+0000 7f0eb259c640 1 -- 172.21.15.154:0/1786691773 learned_addr learned my addr 172.21.15.154:0/1786691773 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:03.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.763+0000 7f0eb2d9d640 1 --2- 172.21.15.154:0/1786691773 >> v2:172.21.15.190:3300/0 conn(0x7f0eb40caea0 0x7f0eb40cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:03.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.763+0000 7f0eabfff640 1 --2- 172.21.15.154:0/1786691773 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0eb415e5f0 0x7f0eb415ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:03.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.763+0000 7f0eb259c640 1 -- 172.21.15.154:0/1786691773 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0eb415e5f0 msgr2=0x7f0eb415ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:03.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.763+0000 7f0eb259c640 1 --2- 172.21.15.154:0/1786691773 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0eb415e5f0 0x7f0eb415ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:03.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.763+0000 7f0eb259c640 1 -- 172.21.15.154:0/1786691773 >> v2:172.21.15.190:3300/0 conn(0x7f0eb40caea0 msgr2=0x7f0eb40cb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:03.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.763+0000 7f0eb259c640 1 --2- 172.21.15.154:0/1786691773 >> v2:172.21.15.190:3300/0 conn(0x7f0eb40caea0 0x7f0eb40cb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:03.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.763+0000 7f0eb259c640 1 -- 172.21.15.154:0/1786691773 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb415f390 con 0x7f0eb415aba0 2023-09-23T03:24:03.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.763+0000 7f0eb2d9d640 1 --2- 172.21.15.154:0/1786691773 >> v2:172.21.15.190:3300/0 conn(0x7f0eb40caea0 0x7f0eb40cb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:03.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.764+0000 7f0eb259c640 1 --2- 172.21.15.154:0/1786691773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0eb415aba0 0x7f0eb415bb30 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f0eb40adf40 tx=0x7f0ea402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3db97de4c30b55cf server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:03.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.765+0000 7f0eb1d9b640 1 -- 172.21.15.154:0/1786691773 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4031860 con 0x7f0eb415aba0 2023-09-23T03:24:03.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.765+0000 7f0eb1d9b640 1 -- 172.21.15.154:0/1786691773 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea40319c0 con 0x7f0eb415aba0 2023-09-23T03:24:03.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.765+0000 7f0eb1d9b640 1 -- 172.21.15.154:0/1786691773 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4031d50 con 0x7f0eb415aba0 2023-09-23T03:24:03.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.765+0000 7f0eb8f97640 1 -- 172.21.15.154:0/1786691773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0eb415aba0 msgr2=0x7f0eb415bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:03.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.765+0000 7f0eb8f97640 1 --2- 172.21.15.154:0/1786691773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0eb415aba0 0x7f0eb415bb30 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f0eb40adf40 tx=0x7f0ea402b900 comp rx=0 tx=0).stop 2023-09-23T03:24:03.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.765+0000 7f0eb8f97640 1 -- 172.21.15.154:0/1786691773 shutdown_connections 2023-09-23T03:24:03.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.765+0000 7f0eb8f97640 1 --2- 172.21.15.154:0/1786691773 >> v2:172.21.15.190:3300/0 conn(0x7f0eb40caea0 0x7f0eb40cb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:03.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.765+0000 7f0eb8f97640 1 --2- 172.21.15.154:0/1786691773 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0eb415e5f0 0x7f0eb415ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:03.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.765+0000 7f0eb8f97640 1 --2- 172.21.15.154:0/1786691773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0eb415aba0 0x7f0eb415bb30 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:03.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.765+0000 7f0eb8f97640 1 -- 172.21.15.154:0/1786691773 >> 172.21.15.154:0/1786691773 conn(0x7f0eb406c010 msgr2=0x7f0eb406c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:03.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.765+0000 7f0eb8f97640 1 -- 172.21.15.154:0/1786691773 shutdown_connections 2023-09-23T03:24:03.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.765+0000 7f0eb8f97640 1 -- 172.21.15.154:0/1786691773 wait complete. 2023-09-23T03:24:03.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.767+0000 7f0eb8f97640 1 Processor -- start 2023-09-23T03:24:03.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.767+0000 7f0eb8f97640 1 -- start start 2023-09-23T03:24:03.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.768+0000 7f0eb8f97640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0eb40caea0 0x7f0eb4149d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:03.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.768+0000 7f0eb8f97640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0eb415aba0 0x7f0eb415d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:03.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.768+0000 7f0eb8f97640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0eb415e5f0 0x7f0eb415d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:03.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.768+0000 7f0eb8f97640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb41490a0 con 0x7f0eb415aba0 2023-09-23T03:24:03.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.768+0000 7f0eb8f97640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb414a290 con 0x7f0eb40caea0 2023-09-23T03:24:03.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.768+0000 7f0eb8f97640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb414a530 con 0x7f0eb415e5f0 2023-09-23T03:24:03.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.769+0000 7f0eabfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0eb415aba0 0x7f0eb415d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:03.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.769+0000 7f0eabfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0eb415aba0 0x7f0eb415d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:51658/0 (socket says 172.21.15.154:51658) 2023-09-23T03:24:03.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.769+0000 7f0eabfff640 1 -- 172.21.15.154:0/2721394864 learned_addr learned my addr 172.21.15.154:0/2721394864 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:03.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.769+0000 7f0eabfff640 1 -- 172.21.15.154:0/2721394864 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0eb415e5f0 msgr2=0x7f0eb415d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:03.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.769+0000 7f0eabfff640 1 --2- 172.21.15.154:0/2721394864 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0eb415e5f0 0x7f0eb415d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:03.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.769+0000 7f0eb259c640 1 --2- 172.21.15.154:0/2721394864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0eb40caea0 0x7f0eb4149d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:03.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.769+0000 7f0eabfff640 1 -- 172.21.15.154:0/2721394864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0eb40caea0 msgr2=0x7f0eb4149d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:03.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.769+0000 7f0eabfff640 1 --2- 172.21.15.154:0/2721394864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0eb40caea0 0x7f0eb4149d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:03.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.769+0000 7f0eabfff640 1 -- 172.21.15.154:0/2721394864 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea40094d0 con 0x7f0eb415aba0 2023-09-23T03:24:03.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.769+0000 7f0eabfff640 1 --2- 172.21.15.154:0/2721394864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0eb415aba0 0x7f0eb415d130 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c00d800 tx=0x7f0e9c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:03.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.770+0000 7f0eaaffd640 1 -- 172.21.15.154:0/2721394864 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c00b6c0 con 0x7f0eb415aba0 2023-09-23T03:24:03.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.770+0000 7f0eaaffd640 1 -- 172.21.15.154:0/2721394864 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e9c00b820 con 0x7f0eb415aba0 2023-09-23T03:24:03.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.770+0000 7f0eb8f97640 1 -- 172.21.15.154:0/2721394864 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb414a810 con 0x7f0eb415aba0 2023-09-23T03:24:03.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.770+0000 7f0eaaffd640 1 -- 172.21.15.154:0/2721394864 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c00bbb0 con 0x7f0eb415aba0 2023-09-23T03:24:03.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.770+0000 7f0eb8f97640 1 -- 172.21.15.154:0/2721394864 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb414add0 con 0x7f0eb415aba0 2023-09-23T03:24:03.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.771+0000 7f0eb8f97640 1 -- 172.21.15.154:0/2721394864 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb404e920 con 0x7f0eb415aba0 2023-09-23T03:24:03.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.772+0000 7f0eaaffd640 1 -- 172.21.15.154:0/2721394864 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f0e9c00bd10 con 0x7f0eb415aba0 2023-09-23T03:24:03.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.773+0000 7f0eaaffd640 1 --2- 172.21.15.154:0/2721394864 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0e9406af30 0x7f0e9406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:03.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.773+0000 7f0eaaffd640 1 -- 172.21.15.154:0/2721394864 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5521+0+0 (secure 0 0 0) 0x7f0e9c089cb0 con 0x7f0eb415aba0 2023-09-23T03:24:03.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.773+0000 7f0eb259c640 1 --2- 172.21.15.154:0/2721394864 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0e9406af30 0x7f0e9406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:03.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.774+0000 7f0eb259c640 1 --2- 172.21.15.154:0/2721394864 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0e9406af30 0x7f0e9406d3f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4036000 tx=0x7f0ea403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:03.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.777+0000 7f0eaaffd640 1 -- 172.21.15.154:0/2721394864 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f0e9c054d10 con 0x7f0eb415aba0 2023-09-23T03:24:03.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.967+0000 7f0eb8f97640 1 -- 172.21.15.154:0/2721394864 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0eb40cbc10 con 0x7f0eb415aba0 2023-09-23T03:24:03.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.975+0000 7f0eaaffd640 1 -- 172.21.15.154:0/2721394864 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0e9c058330 con 0x7f0eb415aba0 2023-09-23T03:24:03.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.978+0000 7f0eb8f97640 1 -- 172.21.15.154:0/2721394864 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0e9406af30 msgr2=0x7f0e9406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:03.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.978+0000 7f0eb8f97640 1 --2- 172.21.15.154:0/2721394864 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0e9406af30 0x7f0e9406d3f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4036000 tx=0x7f0ea403c040 comp rx=0 tx=0).stop 2023-09-23T03:24:03.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.978+0000 7f0eb8f97640 1 -- 172.21.15.154:0/2721394864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0eb415aba0 msgr2=0x7f0eb415d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:03.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.978+0000 7f0eb8f97640 1 --2- 172.21.15.154:0/2721394864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0eb415aba0 0x7f0eb415d130 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c00d800 tx=0x7f0e9c00dcc0 comp rx=0 tx=0).stop 2023-09-23T03:24:03.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.979+0000 7f0eb8f97640 1 -- 172.21.15.154:0/2721394864 shutdown_connections 2023-09-23T03:24:03.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.979+0000 7f0eb8f97640 1 --2- 172.21.15.154:0/2721394864 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0e9406af30 0x7f0e9406d3f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:03.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.979+0000 7f0eb8f97640 1 --2- 172.21.15.154:0/2721394864 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0eb415e5f0 0x7f0eb415d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:03.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.979+0000 7f0eb8f97640 1 --2- 172.21.15.154:0/2721394864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0eb415aba0 0x7f0eb415d130 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:03.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.979+0000 7f0eb8f97640 1 --2- 172.21.15.154:0/2721394864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0eb40caea0 0x7f0eb4149d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:03.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.979+0000 7f0eb8f97640 1 -- 172.21.15.154:0/2721394864 >> 172.21.15.154:0/2721394864 conn(0x7f0eb406c010 msgr2=0x7f0eb40715a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:03.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.979+0000 7f0eb8f97640 1 -- 172.21.15.154:0/2721394864 shutdown_connections 2023-09-23T03:24:03.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:03.979+0000 7f0eb8f97640 1 -- 172.21.15.154:0/2721394864 wait complete. 2023-09-23T03:24:04.000 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-09-23T03:24:04.000 DEBUG:teuthology.orchestra.run.smithi154:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-23T03:24:04.110 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-23T03:24:04.241 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-09-23T03:24:04.241 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:24:04.242 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:24:04.301 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053600,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-09-23T03:24:04.302 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:24:04.302 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:24:04.364 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053603,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248804k,nr_inodes=812201,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-09-23T03:24:04.365 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:24:04.365 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:24:04.426 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053600,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-09-23T03:24:04.426 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:24:04.426 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:24:04.486 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053603,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248804k,nr_inodes=812201,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-09-23T03:24:04.486 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-09-23T03:24:04.486 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:24:04.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.705+0000 7f7009c44640 1 Processor -- start 2023-09-23T03:24:04.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.705+0000 7f7009c44640 1 -- start start 2023-09-23T03:24:04.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.707+0000 7f7009c44640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f700415caf0 0x7f70040ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:04.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.707+0000 7f7009c44640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f70040cee70 0x7f7004160880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:04.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.707+0000 7f7009c44640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70040cd530 con 0x7f7004160dc0 2023-09-23T03:24:04.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.707+0000 7f7009c44640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f700415ced0 con 0x7f70040cee70 2023-09-23T03:24:04.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.707+0000 7f7009c44640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70040cf2f0 con 0x7f700415caf0 2023-09-23T03:24:04.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.707+0000 7f6ffbfff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f70040cee70 0x7f7004160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:04.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.707+0000 7f70037fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f700415caf0 0x7f70040ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:04.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.708+0000 7f6ffbfff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f70040cee70 0x7f7004160880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.154:49884/0 (socket says 172.21.15.154:49884) 2023-09-23T03:24:04.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.708+0000 7f6ffbfff640 1 -- 172.21.15.154:0/3769840635 learned_addr learned my addr 172.21.15.154:0/3769840635 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:04.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.708+0000 7f6ffbfff640 1 -- 172.21.15.154:0/3769840635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f700415caf0 msgr2=0x7f70040ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:04.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.708+0000 7f6ffbfff640 1 --2- 172.21.15.154:0/3769840635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f700415caf0 0x7f70040ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:04.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.708+0000 7f6ffbfff640 1 -- 172.21.15.154:0/3769840635 >> v1:172.21.15.190:6789/0 conn(0x7f7004160dc0 legacy=0x7f700406cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:04.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.708+0000 7f6ffbfff640 1 -- 172.21.15.154:0/3769840635 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70040cd810 con 0x7f70040cee70 2023-09-23T03:24:04.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.710+0000 7f6ffbfff640 1 --2- 172.21.15.154:0/3769840635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70040cee70 0x7f7004160880 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6fec009870 tx=0x7f6fec02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3749506bff593f3a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:04.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.711+0000 7f7002ffd640 1 -- 172.21.15.154:0/3769840635 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec030840 con 0x7f70040cee70 2023-09-23T03:24:04.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.711+0000 7f7002ffd640 1 -- 172.21.15.154:0/3769840635 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fec0309a0 con 0x7f70040cee70 2023-09-23T03:24:04.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.711+0000 7f7002ffd640 1 -- 172.21.15.154:0/3769840635 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec030d30 con 0x7f70040cee70 2023-09-23T03:24:04.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.711+0000 7f7009c44640 1 -- 172.21.15.154:0/3769840635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70040cee70 msgr2=0x7f7004160880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:04.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.711+0000 7f7009c44640 1 --2- 172.21.15.154:0/3769840635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70040cee70 0x7f7004160880 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6fec009870 tx=0x7f6fec02b980 comp rx=0 tx=0).stop 2023-09-23T03:24:04.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.711+0000 7f7009c44640 1 -- 172.21.15.154:0/3769840635 shutdown_connections 2023-09-23T03:24:04.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.711+0000 7f7009c44640 1 --2- 172.21.15.154:0/3769840635 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f70040cee70 0x7f7004160880 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:04.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.711+0000 7f7009c44640 1 --2- 172.21.15.154:0/3769840635 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f700415caf0 0x7f70040ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:04.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.711+0000 7f7009c44640 1 -- 172.21.15.154:0/3769840635 >> 172.21.15.154:0/3769840635 conn(0x7f700406a0f0 msgr2=0x7f700406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:04.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.712+0000 7f7009c44640 1 -- 172.21.15.154:0/3769840635 shutdown_connections 2023-09-23T03:24:04.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.712+0000 7f7009c44640 1 -- 172.21.15.154:0/3769840635 wait complete. 2023-09-23T03:24:04.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.713+0000 7f7009c44640 1 Processor -- start 2023-09-23T03:24:04.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.713+0000 7f7009c44640 1 -- start start 2023-09-23T03:24:04.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.714+0000 7f7009c44640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70040cee70 0x7f70040c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:04.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.714+0000 7f7009c44640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f700415caf0 0x7f7004155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:04.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.714+0000 7f7009c44640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7004160dc0 0x7f7004156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:04.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.714+0000 7f7009c44640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700415fcc0 con 0x7f700415caf0 2023-09-23T03:24:04.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.714+0000 7f7009c44640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70041580a0 con 0x7f7004160dc0 2023-09-23T03:24:04.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.714+0000 7f7009c44640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70040c4560 con 0x7f70040cee70 2023-09-23T03:24:04.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.715+0000 7f6ffbfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f700415caf0 0x7f7004155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:04.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.715+0000 7f6ffbfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f700415caf0 0x7f7004155ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36202/0 (socket says 172.21.15.154:36202) 2023-09-23T03:24:04.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.715+0000 7f6ffbfff640 1 -- 172.21.15.154:0/3362615355 learned_addr learned my addr 172.21.15.154:0/3362615355 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:04.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.715+0000 7f7003fff640 1 --2- 172.21.15.154:0/3362615355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7004160dc0 0x7f7004156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:04.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.715+0000 7f70037fe640 1 --2- 172.21.15.154:0/3362615355 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70040cee70 0x7f70040c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:04.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.715+0000 7f6ffbfff640 1 -- 172.21.15.154:0/3362615355 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70040cee70 msgr2=0x7f70040c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:04.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.715+0000 7f6ffbfff640 1 --2- 172.21.15.154:0/3362615355 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70040cee70 0x7f70040c49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:04.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.715+0000 7f6ffbfff640 1 -- 172.21.15.154:0/3362615355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7004160dc0 msgr2=0x7f7004156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:04.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.715+0000 7f6ffbfff640 1 --2- 172.21.15.154:0/3362615355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7004160dc0 0x7f7004156b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:04.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.715+0000 7f6ffbfff640 1 -- 172.21.15.154:0/3362615355 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fec0094d0 con 0x7f700415caf0 2023-09-23T03:24:04.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.715+0000 7f7003fff640 1 --2- 172.21.15.154:0/3362615355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7004160dc0 0x7f7004156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:04.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.716+0000 7f6ffbfff640 1 --2- 172.21.15.154:0/3362615355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f700415caf0 0x7f7004155ed0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f6fec000c00 tx=0x7f6fec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:04.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.716+0000 7f7000ff9640 1 -- 172.21.15.154:0/3362615355 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec039c90 con 0x7f700415caf0 2023-09-23T03:24:04.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.716+0000 7f7000ff9640 1 -- 172.21.15.154:0/3362615355 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fec039df0 con 0x7f700415caf0 2023-09-23T03:24:04.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.716+0000 7f7000ff9640 1 -- 172.21.15.154:0/3362615355 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec043640 con 0x7f700415caf0 2023-09-23T03:24:04.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.716+0000 7f7009c44640 1 -- 172.21.15.154:0/3362615355 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70041582f0 con 0x7f700415caf0 2023-09-23T03:24:04.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.716+0000 7f7009c44640 1 -- 172.21.15.154:0/3362615355 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70040c41f0 con 0x7f700415caf0 2023-09-23T03:24:04.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.717+0000 7f7009c44640 1 -- 172.21.15.154:0/3362615355 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70041613c0 con 0x7f700415caf0 2023-09-23T03:24:04.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.718+0000 7f7000ff9640 1 -- 172.21.15.154:0/3362615355 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f6fec0437a0 con 0x7f700415caf0 2023-09-23T03:24:04.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.719+0000 7f7000ff9640 1 --2- 172.21.15.154:0/3362615355 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6fe406af60 0x7f6fe406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:04.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.719+0000 7f7000ff9640 1 -- 172.21.15.154:0/3362615355 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5521+0+0 (secure 0 0 0) 0x7f6fec03e070 con 0x7f700415caf0 2023-09-23T03:24:04.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.719+0000 7f70037fe640 1 --2- 172.21.15.154:0/3362615355 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6fe406af60 0x7f6fe406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:04.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.720+0000 7f70037fe640 1 --2- 172.21.15.154:0/3362615355 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6fe406af60 0x7f6fe406d420 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4009a00 tx=0x7f6ff4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:04.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.724+0000 7f7000ff9640 1 -- 172.21.15.154:0/3362615355 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f6fec077740 con 0x7f700415caf0 2023-09-23T03:24:04.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.955+0000 7f7009c44640 1 -- 172.21.15.154:0/3362615355 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70040ab6a0 con 0x7f700415caf0 2023-09-23T03:24:04.958 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.957+0000 7f7000ff9640 1 -- 172.21.15.154:0/3362615355 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+3708 (secure 0 0 0) 0x7f6fec07ad60 con 0x7f700415caf0 2023-09-23T03:24:04.959 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:04.959 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":56,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5279,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1609915000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1609915000},{"type":"v1","addr":"172.21.15.154:6837","nonce":1609915000}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":5288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/1240415394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":1240415394},{"type":"v1","addr":"172.21.15.190:6835","nonce":1240415394}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":5728,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/878668736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":878668736},{"type":"v1","addr":"172.21.15.190:6833","nonce":878668736}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":7179,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/1983484795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1983484795},{"type":"v1","addr":"172.21.15.154:6835","nonce":1983484795}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[]} 2023-09-23T03:24:04.959 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 56 2023-09-23T03:24:04.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.959+0000 7f7009c44640 1 -- 172.21.15.154:0/3362615355 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6fe406af60 msgr2=0x7f6fe406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:04.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.959+0000 7f7009c44640 1 --2- 172.21.15.154:0/3362615355 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6fe406af60 0x7f6fe406d420 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4009a00 tx=0x7f6ff4008040 comp rx=0 tx=0).stop 2023-09-23T03:24:04.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.959+0000 7f7009c44640 1 -- 172.21.15.154:0/3362615355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f700415caf0 msgr2=0x7f7004155ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:04.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.959+0000 7f7009c44640 1 --2- 172.21.15.154:0/3362615355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f700415caf0 0x7f7004155ed0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f6fec000c00 tx=0x7f6fec004730 comp rx=0 tx=0).stop 2023-09-23T03:24:04.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.959+0000 7f7009c44640 1 -- 172.21.15.154:0/3362615355 shutdown_connections 2023-09-23T03:24:04.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.959+0000 7f7009c44640 1 --2- 172.21.15.154:0/3362615355 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6fe406af60 0x7f6fe406d420 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:04.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.959+0000 7f7009c44640 1 --2- 172.21.15.154:0/3362615355 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7004160dc0 0x7f7004156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:04.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.959+0000 7f7009c44640 1 --2- 172.21.15.154:0/3362615355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f700415caf0 0x7f7004155ed0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:04.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.959+0000 7f7009c44640 1 --2- 172.21.15.154:0/3362615355 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70040cee70 0x7f70040c49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:04.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.959+0000 7f7009c44640 1 -- 172.21.15.154:0/3362615355 >> 172.21.15.154:0/3362615355 conn(0x7f700406a0f0 msgr2=0x7f70040adf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:04.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.960+0000 7f7009c44640 1 -- 172.21.15.154:0/3362615355 shutdown_connections 2023-09-23T03:24:04.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:04.960+0000 7f7009c44640 1 -- 172.21.15.154:0/3362615355 wait complete. 2023-09-23T03:24:04.979 INFO:tasks.ceph.mds.a:Restarting daemon 2023-09-23T03:24:04.979 INFO:tasks.ceph.mds.a:Stopping old one... 2023-09-23T03:24:04.979 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-09-23T03:24:04.980 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:24:04.980 INFO:tasks.ceph.mds.c:Restarting daemon 2023-09-23T03:24:04.980 INFO:tasks.ceph.mds.c:Stopping old one... 2023-09-23T03:24:04.980 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-09-23T03:24:04.980 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:24:04.980 INFO:tasks.ceph.mds.b:Restarting daemon 2023-09-23T03:24:04.981 INFO:tasks.ceph.mds.b:Stopping old one... 2023-09-23T03:24:04.981 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-09-23T03:24:04.981 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:24:04.981 INFO:tasks.ceph.mds.d:Restarting daemon 2023-09-23T03:24:04.981 INFO:tasks.ceph.mds.d:Stopping old one... 2023-09-23T03:24:04.982 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-09-23T03:24:04.982 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:24:04.994 INFO:tasks.ceph.mds.c:Stopped 2023-09-23T03:24:04.994 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-09-23T03:24:05.000 INFO:tasks.ceph.mds.a:Stopped 2023-09-23T03:24:05.000 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-09-23T03:24:05.019 INFO:tasks.ceph.mds.b:Stopped 2023-09-23T03:24:05.019 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-09-23T03:24:05.021 INFO:tasks.ceph.mds.d:Stopped 2023-09-23T03:24:05.021 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-09-23T03:24:05.048 INFO:tasks.ceph.mds.c:Started 2023-09-23T03:24:05.048 DEBUG:teuthology.parallel:result is None 2023-09-23T03:24:05.055 INFO:tasks.ceph.mds.a:Started 2023-09-23T03:24:05.056 DEBUG:teuthology.parallel:result is None 2023-09-23T03:24:05.069 INFO:tasks.ceph.mds.b:Started 2023-09-23T03:24:05.069 DEBUG:teuthology.parallel:result is None 2023-09-23T03:24:05.077 INFO:tasks.ceph.mds.d:Started 2023-09-23T03:24:05.077 DEBUG:teuthology.parallel:result is None 2023-09-23T03:24:05.077 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-09-23T03:24:05.206 INFO:tasks.ceph.mds.c.smithi154.stdout:starting mds.c at 2023-09-23T03:24:05.216 INFO:tasks.ceph.mds.a.smithi154.stdout:starting mds.a at 2023-09-23T03:24:05.231 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-09-23T03:24:05.235 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-09-23T03:24:05.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.271+0000 7ff53417d640 1 Processor -- start 2023-09-23T03:24:05.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.271+0000 7ff53417d640 1 -- start start 2023-09-23T03:24:05.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.272+0000 7ff53417d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff52c0b1b20 0x7ff52c0ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:05.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.272+0000 7ff53417d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff52c0cee70 0x7ff52c0ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:05.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.272+0000 7ff53417d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff52c15ac30 con 0x7ff52c0acce0 2023-09-23T03:24:05.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.272+0000 7ff53417d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff52c0b2e90 con 0x7ff52c0cee70 2023-09-23T03:24:05.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.272+0000 7ff53417d640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff52c0cf320 con 0x7ff52c0b1b20 2023-09-23T03:24:05.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.272+0000 7ff531f19640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff52c0acce0 0x7ff52c15ed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:54430/0 (socket says 172.21.15.154:54430) 2023-09-23T03:24:05.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.272+0000 7ff531f19640 1 -- 172.21.15.154:0/3529419701 learned_addr learned my addr 172.21.15.154:0/3529419701 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:24:05.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.272+0000 7ff53271a640 1 --2- 172.21.15.154:0/3529419701 >> v2:172.21.15.154:3300/0 conn(0x7ff52c0cee70 0x7ff52c0ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:05.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.273+0000 7ff531718640 1 --2- 172.21.15.154:0/3529419701 >> v2:172.21.15.190:3300/0 conn(0x7ff52c0b1b20 0x7ff52c0ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:05.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.273+0000 7ff53271a640 1 -- 172.21.15.154:0/3529419701 >> v2:172.21.15.190:3300/0 conn(0x7ff52c0b1b20 msgr2=0x7ff52c0ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:05.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.273+0000 7ff53271a640 1 --2- 172.21.15.154:0/3529419701 >> v2:172.21.15.190:3300/0 conn(0x7ff52c0b1b20 0x7ff52c0ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:05.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.273+0000 7ff53271a640 1 -- 172.21.15.154:0/3529419701 >> v1:172.21.15.154:6789/0 conn(0x7ff52c0acce0 legacy=0x7ff52c15ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:05.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.273+0000 7ff53271a640 1 -- 172.21.15.154:0/3529419701 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff52c0b1f70 con 0x7ff52c0cee70 2023-09-23T03:24:05.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.274+0000 7ff53271a640 1 --2- 172.21.15.154:0/3529419701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff52c0cee70 0x7ff52c0ab140 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7ff52c0cbff0 tx=0x7ff52402be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee708856824f7338 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:05.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.274+0000 7ff530f17640 1 -- 172.21.15.154:0/3529419701 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52402cbe0 con 0x7ff52c0cee70 2023-09-23T03:24:05.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.274+0000 7ff530f17640 1 -- 172.21.15.154:0/3529419701 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52402cd40 con 0x7ff52c0cee70 2023-09-23T03:24:05.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.274+0000 7ff530f17640 1 -- 172.21.15.154:0/3529419701 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff524034660 con 0x7ff52c0cee70 2023-09-23T03:24:05.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.275+0000 7ff53417d640 1 -- 172.21.15.154:0/3529419701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff52c0cee70 msgr2=0x7ff52c0ab140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:05.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.275+0000 7ff53417d640 1 --2- 172.21.15.154:0/3529419701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff52c0cee70 0x7ff52c0ab140 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7ff52c0cbff0 tx=0x7ff52402be30 comp rx=0 tx=0).stop 2023-09-23T03:24:05.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.275+0000 7ff53417d640 1 -- 172.21.15.154:0/3529419701 shutdown_connections 2023-09-23T03:24:05.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.275+0000 7ff53417d640 1 --2- 172.21.15.154:0/3529419701 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff52c0cee70 0x7ff52c0ab140 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:05.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.275+0000 7ff53417d640 1 --2- 172.21.15.154:0/3529419701 >> v2:172.21.15.190:3300/0 conn(0x7ff52c0b1b20 0x7ff52c0ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:05.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.275+0000 7ff53417d640 1 -- 172.21.15.154:0/3529419701 >> 172.21.15.154:0/3529419701 conn(0x7ff52c06a0f0 msgr2=0x7ff52c06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:05.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.275+0000 7ff53417d640 1 -- 172.21.15.154:0/3529419701 shutdown_connections 2023-09-23T03:24:05.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.275+0000 7ff53417d640 1 -- 172.21.15.154:0/3529419701 wait complete. 2023-09-23T03:24:05.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.277+0000 7ff53417d640 1 Processor -- start 2023-09-23T03:24:05.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.277+0000 7ff53417d640 1 -- start start 2023-09-23T03:24:05.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.278+0000 7ff53417d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff52c0acce0 0x7ff52c097a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:05.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.278+0000 7ff53417d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff52c0b1b20 0x7ff52c0c55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:05.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.278+0000 7ff53417d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff52c0cee70 0x7ff52c0c5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:05.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.278+0000 7ff53417d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c0c3720 con 0x7ff52c0b1b20 2023-09-23T03:24:05.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.278+0000 7ff53417d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c0c8550 con 0x7ff52c0acce0 2023-09-23T03:24:05.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.278+0000 7ff53417d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c0c8830 con 0x7ff52c0cee70 2023-09-23T03:24:05.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.278+0000 7ff531718640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff52c0b1b20 0x7ff52c0c55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:05.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.278+0000 7ff531718640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff52c0b1b20 0x7ff52c0c55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36246/0 (socket says 172.21.15.154:36246) 2023-09-23T03:24:05.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.278+0000 7ff531718640 1 -- 172.21.15.154:0/2515890048 learned_addr learned my addr 172.21.15.154:0/2515890048 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:05.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.278+0000 7ff531f19640 1 --2- 172.21.15.154:0/2515890048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff52c0acce0 0x7ff52c097a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:05.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.278+0000 7ff53271a640 1 --2- 172.21.15.154:0/2515890048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff52c0cee70 0x7ff52c0c5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:05.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.278+0000 7ff531718640 1 -- 172.21.15.154:0/2515890048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff52c0cee70 msgr2=0x7ff52c0c5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:05.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.278+0000 7ff531718640 1 --2- 172.21.15.154:0/2515890048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff52c0cee70 0x7ff52c0c5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:05.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.278+0000 7ff531718640 1 -- 172.21.15.154:0/2515890048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff52c0acce0 msgr2=0x7ff52c097a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:05.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.278+0000 7ff531718640 1 --2- 172.21.15.154:0/2515890048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff52c0acce0 0x7ff52c097a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:05.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.279+0000 7ff531718640 1 -- 172.21.15.154:0/2515890048 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff524009a70 con 0x7ff52c0b1b20 2023-09-23T03:24:05.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.279+0000 7ff531f19640 1 --2- 172.21.15.154:0/2515890048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff52c0acce0 0x7ff52c097a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:24:05.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.279+0000 7ff531718640 1 --2- 172.21.15.154:0/2515890048 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff52c0b1b20 0x7ff52c0c55e0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7ff51c00d7d0 tx=0x7ff51c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:05.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.279+0000 7ff5227fc640 1 -- 172.21.15.154:0/2515890048 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c00b6c0 con 0x7ff52c0b1b20 2023-09-23T03:24:05.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.279+0000 7ff5227fc640 1 -- 172.21.15.154:0/2515890048 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff51c00b820 con 0x7ff52c0b1b20 2023-09-23T03:24:05.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.279+0000 7ff5227fc640 1 -- 172.21.15.154:0/2515890048 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c00bbb0 con 0x7ff52c0b1b20 2023-09-23T03:24:05.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.279+0000 7ff53417d640 1 -- 172.21.15.154:0/2515890048 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff52c0c30d0 con 0x7ff52c0b1b20 2023-09-23T03:24:05.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.279+0000 7ff53417d640 1 -- 172.21.15.154:0/2515890048 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff52c1f0dd0 con 0x7ff52c0b1b20 2023-09-23T03:24:05.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.280+0000 7ff53417d640 1 -- 172.21.15.154:0/2515890048 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff52c0c38d0 con 0x7ff52c0b1b20 2023-09-23T03:24:05.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.281+0000 7ff5227fc640 1 -- 172.21.15.154:0/2515890048 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7ff51c00bd10 con 0x7ff52c0b1b20 2023-09-23T03:24:05.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.282+0000 7ff5227fc640 1 --2- 172.21.15.154:0/2515890048 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff51006af30 0x7ff51006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:05.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.282+0000 7ff5227fc640 1 -- 172.21.15.154:0/2515890048 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5521+0+0 (secure 0 0 0) 0x7ff51c089b40 con 0x7ff52c0b1b20 2023-09-23T03:24:05.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.282+0000 7ff531f19640 1 --2- 172.21.15.154:0/2515890048 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff51006af30 0x7ff51006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:05.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.283+0000 7ff531f19640 1 --2- 172.21.15.154:0/2515890048 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff51006af30 0x7ff51006d3f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff528007fd0 tx=0x7ff528009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:05.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.287+0000 7ff5227fc640 1 -- 172.21.15.154:0/2515890048 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7ff51c054c20 con 0x7ff52c0b1b20 2023-09-23T03:24:05.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.488+0000 7ff53417d640 1 -- 172.21.15.154:0/2515890048 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff52c073eb0 con 0x7ff52c0b1b20 2023-09-23T03:24:05.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.539+0000 7ff5227fc640 1 -- 172.21.15.154:0/2515890048 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v77) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff51c058240 con 0x7ff52c0b1b20 2023-09-23T03:24:05.540 INFO:teuthology.orchestra.run.smithi154.stderr: removed all blocklist entries 2023-09-23T03:24:05.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.542+0000 7ff53417d640 1 -- 172.21.15.154:0/2515890048 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff51006af30 msgr2=0x7ff51006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:05.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.542+0000 7ff53417d640 1 --2- 172.21.15.154:0/2515890048 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff51006af30 0x7ff51006d3f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff528007fd0 tx=0x7ff528009040 comp rx=0 tx=0).stop 2023-09-23T03:24:05.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.542+0000 7ff53417d640 1 -- 172.21.15.154:0/2515890048 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff52c0b1b20 msgr2=0x7ff52c0c55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:05.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.542+0000 7ff53417d640 1 --2- 172.21.15.154:0/2515890048 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff52c0b1b20 0x7ff52c0c55e0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7ff51c00d7d0 tx=0x7ff51c00dc90 comp rx=0 tx=0).stop 2023-09-23T03:24:05.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.542+0000 7ff53417d640 1 -- 172.21.15.154:0/2515890048 shutdown_connections 2023-09-23T03:24:05.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.542+0000 7ff53417d640 1 --2- 172.21.15.154:0/2515890048 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff51006af30 0x7ff51006d3f0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:05.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.542+0000 7ff53417d640 1 --2- 172.21.15.154:0/2515890048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff52c0cee70 0x7ff52c0c5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:05.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.542+0000 7ff53417d640 1 --2- 172.21.15.154:0/2515890048 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff52c0b1b20 0x7ff52c0c55e0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:05.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.542+0000 7ff53417d640 1 --2- 172.21.15.154:0/2515890048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff52c0acce0 0x7ff52c097a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:05.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.542+0000 7ff53417d640 1 -- 172.21.15.154:0/2515890048 >> 172.21.15.154:0/2515890048 conn(0x7ff52c06a0f0 msgr2=0x7ff52c0ae100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:05.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.542+0000 7ff53417d640 1 -- 172.21.15.154:0/2515890048 shutdown_connections 2023-09-23T03:24:05.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.543+0000 7ff53417d640 1 -- 172.21.15.154:0/2515890048 wait complete. 2023-09-23T03:24:05.562 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-09-23T03:24:05.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.760+0000 7f3e5d11d640 1 Processor -- start 2023-09-23T03:24:05.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.760+0000 7f3e5d11d640 1 -- start start 2023-09-23T03:24:05.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.762+0000 7f3e5d11d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e580acce0 0x7f3e5815ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:05.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.762+0000 7f3e5d11d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3e580b1b20 0x7f3e580ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:05.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.762+0000 7f3e5d11d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3e580cee70 0x7f3e5806cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:05.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.762+0000 7f3e5d11d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e580b2e90 con 0x7f3e580cee70 2023-09-23T03:24:05.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.762+0000 7f3e5d11d640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e580cf320 con 0x7f3e580b1b20 2023-09-23T03:24:05.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.762+0000 7f3e5d11d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5815ac30 con 0x7f3e580acce0 2023-09-23T03:24:05.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.762+0000 7f3e5759e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3e580cee70 0x7f3e5806cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:05.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.762+0000 7f3e5759e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3e580cee70 0x7f3e5806cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36258/0 (socket says 172.21.15.154:36258) 2023-09-23T03:24:05.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.762+0000 7f3e5759e640 1 -- 172.21.15.154:0/4093780125 learned_addr learned my addr 172.21.15.154:0/4093780125 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:05.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.762+0000 7f3e4ffff640 1 --2- 172.21.15.154:0/4093780125 >> v2:172.21.15.190:3300/0 conn(0x7f3e580b1b20 0x7f3e580ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:05.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.762+0000 7f3e5759e640 1 -- 172.21.15.154:0/4093780125 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e580acce0 msgr2=0x7f3e5815ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:05.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.763+0000 7f3e5759e640 1 --2- 172.21.15.154:0/4093780125 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e580acce0 0x7f3e5815ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:05.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.763+0000 7f3e5759e640 1 -- 172.21.15.154:0/4093780125 >> v2:172.21.15.190:3300/0 conn(0x7f3e580b1b20 msgr2=0x7f3e580ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:05.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.763+0000 7f3e5759e640 1 --2- 172.21.15.154:0/4093780125 >> v2:172.21.15.190:3300/0 conn(0x7f3e580b1b20 0x7f3e580ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:05.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.763+0000 7f3e5759e640 1 -- 172.21.15.154:0/4093780125 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e580b1f70 con 0x7f3e580cee70 2023-09-23T03:24:05.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.764+0000 7f3e5759e640 1 --2- 172.21.15.154:0/4093780125 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e580cee70 0x7f3e5806cb50 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f3e580cc020 tx=0x7f3e5002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77f27083bbdbd6f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:05.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.764+0000 7f3e5659c640 1 -- 172.21.15.154:0/4093780125 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5002c9f0 con 0x7f3e580cee70 2023-09-23T03:24:05.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.764+0000 7f3e5659c640 1 -- 172.21.15.154:0/4093780125 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e5002cb50 con 0x7f3e580cee70 2023-09-23T03:24:05.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.764+0000 7f3e5659c640 1 -- 172.21.15.154:0/4093780125 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e500343f0 con 0x7f3e580cee70 2023-09-23T03:24:05.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.765+0000 7f3e5d11d640 1 -- 172.21.15.154:0/4093780125 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e580cee70 msgr2=0x7f3e5806cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:05.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.765+0000 7f3e5d11d640 1 --2- 172.21.15.154:0/4093780125 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e580cee70 0x7f3e5806cb50 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f3e580cc020 tx=0x7f3e5002be50 comp rx=0 tx=0).stop 2023-09-23T03:24:05.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.765+0000 7f3e5d11d640 1 -- 172.21.15.154:0/4093780125 shutdown_connections 2023-09-23T03:24:05.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.765+0000 7f3e5d11d640 1 --2- 172.21.15.154:0/4093780125 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e580cee70 0x7f3e5806cb50 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:05.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.765+0000 7f3e5d11d640 1 --2- 172.21.15.154:0/4093780125 >> v2:172.21.15.190:3300/0 conn(0x7f3e580b1b20 0x7f3e580ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:05.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.765+0000 7f3e5d11d640 1 --2- 172.21.15.154:0/4093780125 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e580acce0 0x7f3e5815ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:05.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.765+0000 7f3e5d11d640 1 -- 172.21.15.154:0/4093780125 >> 172.21.15.154:0/4093780125 conn(0x7f3e5806a0f0 msgr2=0x7f3e5806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:05.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.765+0000 7f3e5d11d640 1 -- 172.21.15.154:0/4093780125 shutdown_connections 2023-09-23T03:24:05.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.765+0000 7f3e5d11d640 1 -- 172.21.15.154:0/4093780125 wait complete. 2023-09-23T03:24:05.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.766+0000 7f3e5d11d640 1 Processor -- start 2023-09-23T03:24:05.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.767+0000 7f3e5d11d640 1 -- start start 2023-09-23T03:24:05.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.767+0000 7f3e5d11d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e580acce0 0x7f3e5815d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:05.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.768+0000 7f3e5d11d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e580b1b20 0x7f3e580cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:05.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.768+0000 7f3e5d11d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e580cee70 0x7f3e581f14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:05.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.768+0000 7f3e5d11d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e580c2040 con 0x7f3e580b1b20 2023-09-23T03:24:05.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.768+0000 7f3e5d11d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e580c2320 con 0x7f3e580acce0 2023-09-23T03:24:05.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.768+0000 7f3e5d11d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e580c2600 con 0x7f3e580cee70 2023-09-23T03:24:05.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.768+0000 7f3e4ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e580b1b20 0x7f3e580cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:05.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.768+0000 7f3e4ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e580b1b20 0x7f3e580cd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36274/0 (socket says 172.21.15.154:36274) 2023-09-23T03:24:05.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.768+0000 7f3e4ffff640 1 -- 172.21.15.154:0/2269914152 learned_addr learned my addr 172.21.15.154:0/2269914152 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:05.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.768+0000 7f3e56d9d640 1 --2- 172.21.15.154:0/2269914152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e580acce0 0x7f3e5815d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:05.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.768+0000 7f3e5759e640 1 --2- 172.21.15.154:0/2269914152 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e580cee70 0x7f3e581f14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:05.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.768+0000 7f3e4ffff640 1 -- 172.21.15.154:0/2269914152 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e580cee70 msgr2=0x7f3e581f14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:05.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.768+0000 7f3e4ffff640 1 --2- 172.21.15.154:0/2269914152 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e580cee70 0x7f3e581f14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:05.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.768+0000 7f3e4ffff640 1 -- 172.21.15.154:0/2269914152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e580acce0 msgr2=0x7f3e5815d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:05.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.768+0000 7f3e4ffff640 1 --2- 172.21.15.154:0/2269914152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e580acce0 0x7f3e5815d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:05.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.768+0000 7f3e4ffff640 1 -- 172.21.15.154:0/2269914152 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e50009b30 con 0x7f3e580b1b20 2023-09-23T03:24:05.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.769+0000 7f3e4ffff640 1 --2- 172.21.15.154:0/2269914152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e580b1b20 0x7f3e580cd3a0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f3e4000c7f0 tx=0x7f3e4000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:05.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.769+0000 7f3e4f7fe640 1 -- 172.21.15.154:0/2269914152 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e40004220 con 0x7f3e580b1b20 2023-09-23T03:24:05.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.769+0000 7f3e4f7fe640 1 -- 172.21.15.154:0/2269914152 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e40004380 con 0x7f3e580b1b20 2023-09-23T03:24:05.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.769+0000 7f3e4f7fe640 1 -- 172.21.15.154:0/2269914152 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e40007840 con 0x7f3e580b1b20 2023-09-23T03:24:05.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.769+0000 7f3e5d11d640 1 -- 172.21.15.154:0/2269914152 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e580c28e0 con 0x7f3e580b1b20 2023-09-23T03:24:05.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.769+0000 7f3e5d11d640 1 -- 172.21.15.154:0/2269914152 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e5815dbf0 con 0x7f3e580b1b20 2023-09-23T03:24:05.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.770+0000 7f3e5d11d640 1 -- 172.21.15.154:0/2269914152 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e580b24b0 con 0x7f3e580b1b20 2023-09-23T03:24:05.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.771+0000 7f3e4f7fe640 1 -- 172.21.15.154:0/2269914152 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f3e400079a0 con 0x7f3e580b1b20 2023-09-23T03:24:05.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.772+0000 7f3e4f7fe640 1 --2- 172.21.15.154:0/2269914152 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3e3806af30 0x7f3e3806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:05.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.772+0000 7f3e4f7fe640 1 -- 172.21.15.154:0/2269914152 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f3e40089310 con 0x7f3e580b1b20 2023-09-23T03:24:05.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.772+0000 7f3e56d9d640 1 --2- 172.21.15.154:0/2269914152 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3e3806af30 0x7f3e3806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:05.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.773+0000 7f3e56d9d640 1 --2- 172.21.15.154:0/2269914152 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3e3806af30 0x7f3e3806d3f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3e48006fd0 tx=0x7f3e48008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:05.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:05.776+0000 7f3e4f7fe640 1 -- 172.21.15.154:0/2269914152 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f3e40054550 con 0x7f3e580b1b20 2023-09-23T03:24:06.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.032+0000 7f3e5d11d640 1 -- 172.21.15.154:0/2269914152 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f3e58073eb0 con 0x7f3e580b1b20 2023-09-23T03:24:06.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.034+0000 7f3e4f7fe640 1 -- 172.21.15.154:0/2269914152 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f3e40057b70 con 0x7f3e580b1b20 2023-09-23T03:24:06.036 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:06.036 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:24:06.036 INFO:teuthology.orchestra.run.smithi154.stdout: "auth_dump": [ 2023-09-23T03:24:06.036 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:24:06.036 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mds.a", 2023-09-23T03:24:06.036 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAGWA5l9biCFhAAhD2xgY9Q/gGqK0pgHRGweQ==", 2023-09-23T03:24:06.037 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:24:06.037 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:24:06.037 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2023-09-23T03:24:06.037 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2023-09-23T03:24:06.037 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:24:06.037 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:24:06.037 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:24:06.037 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:24:06.037 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mds.b", 2023-09-23T03:24:06.037 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAGWA5lsgKCNxAAI+vD2ESh3uNsb5OCO0ydSw==", 2023-09-23T03:24:06.038 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:24:06.038 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:24:06.038 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2023-09-23T03:24:06.038 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2023-09-23T03:24:06.038 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:24:06.038 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:24:06.038 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:24:06.038 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:24:06.038 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mds.c", 2023-09-23T03:24:06.038 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAGWA5los4vJhAA2/ewjMZcc3pnm9ukhnoNgw==", 2023-09-23T03:24:06.039 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:24:06.039 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:24:06.039 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2023-09-23T03:24:06.039 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2023-09-23T03:24:06.039 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:24:06.039 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:24:06.039 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:24:06.039 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:24:06.039 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mds.d", 2023-09-23T03:24:06.039 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAHWA5lml/MDBAAHyXgd5zl/O8CeVE8Da4w5w==", 2023-09-23T03:24:06.040 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:24:06.040 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:24:06.040 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2023-09-23T03:24:06.040 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2023-09-23T03:24:06.040 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:24:06.040 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:24:06.040 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:24:06.040 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:24:06.040 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.0", 2023-09-23T03:24:06.040 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAMWA5lT1R4BBAAI+itqub/FadG3aU6LhnmGg==", 2023-09-23T03:24:06.041 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:24:06.041 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:24:06.041 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:24:06.041 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:24:06.041 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:24:06.041 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:24:06.041 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:24:06.041 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.1", 2023-09-23T03:24:06.041 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAPWA5lSablHRAAmNyr5Zopeihn5ZyoY5pyhA==", 2023-09-23T03:24:06.041 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:24:06.041 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:24:06.042 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:24:06.042 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:24:06.042 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:24:06.042 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:24:06.042 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:24:06.042 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.2", 2023-09-23T03:24:06.042 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQASWA5lSFcdJxAA3UJJ6eSvxcQ31SAm4shV2g==", 2023-09-23T03:24:06.042 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:24:06.042 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:24:06.042 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:24:06.043 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:24:06.043 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:24:06.043 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:24:06.043 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:24:06.043 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.3", 2023-09-23T03:24:06.043 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAVWA5lmSvzMRAA97YDqX3oplHCB+SENIkLYA==", 2023-09-23T03:24:06.043 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:24:06.043 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:24:06.043 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:24:06.043 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:24:06.044 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:24:06.044 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:24:06.044 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:24:06.044 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.4", 2023-09-23T03:24:06.044 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAcWA5lPZqwMBAAW95bBB0PKS/rwhXvrPf+mQ==", 2023-09-23T03:24:06.044 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:24:06.044 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:24:06.044 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:24:06.044 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:24:06.044 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:24:06.044 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:24:06.045 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:24:06.045 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.5", 2023-09-23T03:24:06.045 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAgWA5lmZySDRAAsbnUdWq9esKFSqSDeqsVlA==", 2023-09-23T03:24:06.045 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:24:06.045 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:24:06.045 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:24:06.045 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:24:06.045 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:24:06.045 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:24:06.045 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:24:06.046 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.6", 2023-09-23T03:24:06.046 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAjWA5ll9WqJRAAD9jecVxVJU+yuqANvN9QUQ==", 2023-09-23T03:24:06.046 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:24:06.046 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:24:06.046 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:24:06.046 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:24:06.046 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:24:06.046 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:24:06.046 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:24:06.046 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.7", 2023-09-23T03:24:06.047 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAnWA5lCvexAxAA/QymygbvOt1HQdCGbO8cyg==", 2023-09-23T03:24:06.047 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:24:06.047 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:24:06.047 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:24:06.047 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:24:06.047 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:24:06.048 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:24:06.048 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:24:06.048 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.0", 2023-09-23T03:24:06.048 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAHWA5lYg+qGhAAjaY0fPsmRTz3ipsKbPugMA==", 2023-09-23T03:24:06.048 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:24:06.048 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:24:06.048 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow r", 2023-09-23T03:24:06.048 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-09-23T03:24:06.048 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:24:06.049 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:24:06.049 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:24:06.049 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.1", 2023-09-23T03:24:06.049 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAHWA5lbrrGMBAAn4kvS/1xGjZ+GtRZFqy31w==", 2023-09-23T03:24:06.049 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:24:06.049 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:24:06.049 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow r", 2023-09-23T03:24:06.049 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow rw", 2023-09-23T03:24:06.049 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow rwx" 2023-09-23T03:24:06.050 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:24:06.050 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:24:06.050 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:24:06.050 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.2", 2023-09-23T03:24:06.050 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAHWA5lcLkIJRAAOknGCKwUs78D4atRzLb/Cw==", 2023-09-23T03:24:06.050 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:24:06.050 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:24:06.050 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow r", 2023-09-23T03:24:06.050 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow rw", 2023-09-23T03:24:06.050 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow rwx" 2023-09-23T03:24:06.051 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:24:06.051 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:24:06.051 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:24:06.051 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.3", 2023-09-23T03:24:06.051 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAIWA5lmFUaABAA4EBSJP5IidG2uq4g8lrFKg==", 2023-09-23T03:24:06.051 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:24:06.051 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:24:06.051 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow r", 2023-09-23T03:24:06.051 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow rw", 2023-09-23T03:24:06.052 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow rwx" 2023-09-23T03:24:06.052 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:24:06.052 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:24:06.052 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:24:06.052 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.admin", 2023-09-23T03:24:06.052 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAFWA5l0x+tCxAAGEYW1egctWzycEhM6FjJZw==", 2023-09-23T03:24:06.052 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:24:06.052 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow *", 2023-09-23T03:24:06.052 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2023-09-23T03:24:06.052 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2023-09-23T03:24:06.052 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:24:06.053 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:24:06.053 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:24:06.053 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:24:06.053 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mgr.x", 2023-09-23T03:24:06.053 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAGWA5ljcy5ChAACUOnrxEGqacSxrB9BkZ+lg==", 2023-09-23T03:24:06.053 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:24:06.053 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow *", 2023-09-23T03:24:06.053 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile mgr", 2023-09-23T03:24:06.053 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:24:06.053 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:24:06.054 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:24:06.054 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:24:06.054 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mgr.y", 2023-09-23T03:24:06.054 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAFWA5looGaOhAAdxmefaoLuXesHXCeqeCYXQ==", 2023-09-23T03:24:06.054 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:24:06.054 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow *", 2023-09-23T03:24:06.054 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile mgr", 2023-09-23T03:24:06.054 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:24:06.054 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:24:06.054 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:24:06.055 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2023-09-23T03:24:06.055 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:24:06.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.038+0000 7f3e5d11d640 1 -- 172.21.15.154:0/2269914152 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3e3806af30 msgr2=0x7f3e3806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:06.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.038+0000 7f3e5d11d640 1 --2- 172.21.15.154:0/2269914152 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3e3806af30 0x7f3e3806d3f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3e48006fd0 tx=0x7f3e48008040 comp rx=0 tx=0).stop 2023-09-23T03:24:06.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.038+0000 7f3e5d11d640 1 -- 172.21.15.154:0/2269914152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e580b1b20 msgr2=0x7f3e580cd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:06.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.038+0000 7f3e5d11d640 1 --2- 172.21.15.154:0/2269914152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e580b1b20 0x7f3e580cd3a0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f3e4000c7f0 tx=0x7f3e4000ccb0 comp rx=0 tx=0).stop 2023-09-23T03:24:06.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.038+0000 7f3e5d11d640 1 -- 172.21.15.154:0/2269914152 shutdown_connections 2023-09-23T03:24:06.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.038+0000 7f3e5d11d640 1 --2- 172.21.15.154:0/2269914152 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3e3806af30 0x7f3e3806d3f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:06.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.038+0000 7f3e5d11d640 1 --2- 172.21.15.154:0/2269914152 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3e580cee70 0x7f3e581f14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:06.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.038+0000 7f3e5d11d640 1 --2- 172.21.15.154:0/2269914152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e580b1b20 0x7f3e580cd3a0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:06.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.038+0000 7f3e5d11d640 1 --2- 172.21.15.154:0/2269914152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3e580acce0 0x7f3e5815d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:06.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.038+0000 7f3e5d11d640 1 -- 172.21.15.154:0/2269914152 >> 172.21.15.154:0/2269914152 conn(0x7f3e5806a0f0 msgr2=0x7f3e580b0870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:06.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.038+0000 7f3e5d11d640 1 -- 172.21.15.154:0/2269914152 shutdown_connections 2023-09-23T03:24:06.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.038+0000 7f3e5d11d640 1 -- 172.21.15.154:0/2269914152 wait complete. 2023-09-23T03:24:06.058 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:24:06.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.264+0000 7f52b06d4640 1 Processor -- start 2023-09-23T03:24:06.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.264+0000 7f52b06d4640 1 -- start start 2023-09-23T03:24:06.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.265+0000 7f52b06d4640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f52a80acce0 0x7f52a815ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:06.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.266+0000 7f52b06d4640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f52a80cf040 0x7f52a80ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:06.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.266+0000 7f52b06d4640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52a815ac30 con 0x7f52a80b1b20 2023-09-23T03:24:06.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.266+0000 7f52b06d4640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52a80b2e90 con 0x7f52a80acce0 2023-09-23T03:24:06.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.266+0000 7f52b06d4640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52a80b1f70 con 0x7f52a80cf040 2023-09-23T03:24:06.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.266+0000 7f52ae470640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f52a80acce0 0x7f52a815ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:06.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.266+0000 7f52ae470640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f52a80acce0 0x7f52a815ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36282/0 (socket says 172.21.15.154:36282) 2023-09-23T03:24:06.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.266+0000 7f52ae470640 1 -- 172.21.15.154:0/4212020890 learned_addr learned my addr 172.21.15.154:0/4212020890 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:06.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.266+0000 7f52aec71640 1 --2- 172.21.15.154:0/4212020890 >> v2:172.21.15.190:3300/0 conn(0x7f52a80cf040 0x7f52a80ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:06.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.266+0000 7f52ae470640 1 -- 172.21.15.154:0/4212020890 >> v2:172.21.15.190:3300/0 conn(0x7f52a80cf040 msgr2=0x7f52a80ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:06.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.266+0000 7f52ae470640 1 --2- 172.21.15.154:0/4212020890 >> v2:172.21.15.190:3300/0 conn(0x7f52a80cf040 0x7f52a80ab140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:06.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.266+0000 7f52ae470640 1 -- 172.21.15.154:0/4212020890 >> v1:172.21.15.190:6789/0 conn(0x7f52a80b1b20 legacy=0x7f52a80ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:06.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.266+0000 7f52ae470640 1 -- 172.21.15.154:0/4212020890 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a815f2a0 con 0x7f52a80acce0 2023-09-23T03:24:06.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.267+0000 7f52ae470640 1 --2- 172.21.15.154:0/4212020890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52a80acce0 0x7f52a815ed60 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f52a80c9120 tx=0x7f52a402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b377e5123b050c2 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:06.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.268+0000 7f52ad46e640 1 -- 172.21.15.154:0/4212020890 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a40317e0 con 0x7f52a80acce0 2023-09-23T03:24:06.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.268+0000 7f52ad46e640 1 -- 172.21.15.154:0/4212020890 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a4031940 con 0x7f52a80acce0 2023-09-23T03:24:06.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.268+0000 7f52ad46e640 1 -- 172.21.15.154:0/4212020890 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a4031cd0 con 0x7f52a80acce0 2023-09-23T03:24:06.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.268+0000 7f52b06d4640 1 -- 172.21.15.154:0/4212020890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52a80acce0 msgr2=0x7f52a815ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:06.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.268+0000 7f52b06d4640 1 --2- 172.21.15.154:0/4212020890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52a80acce0 0x7f52a815ed60 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f52a80c9120 tx=0x7f52a402b880 comp rx=0 tx=0).stop 2023-09-23T03:24:06.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.268+0000 7f52b06d4640 1 -- 172.21.15.154:0/4212020890 shutdown_connections 2023-09-23T03:24:06.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.268+0000 7f52b06d4640 1 --2- 172.21.15.154:0/4212020890 >> v2:172.21.15.190:3300/0 conn(0x7f52a80cf040 0x7f52a80ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:06.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.268+0000 7f52b06d4640 1 --2- 172.21.15.154:0/4212020890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52a80acce0 0x7f52a815ed60 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:06.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.268+0000 7f52b06d4640 1 -- 172.21.15.154:0/4212020890 >> 172.21.15.154:0/4212020890 conn(0x7f52a806a0f0 msgr2=0x7f52a806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:06.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.269+0000 7f52b06d4640 1 -- 172.21.15.154:0/4212020890 shutdown_connections 2023-09-23T03:24:06.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.269+0000 7f52b06d4640 1 -- 172.21.15.154:0/4212020890 wait complete. 2023-09-23T03:24:06.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.270+0000 7f52b06d4640 1 Processor -- start 2023-09-23T03:24:06.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.271+0000 7f52b06d4640 1 -- start start 2023-09-23T03:24:06.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.272+0000 7f52b06d4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52a80acce0 0x7f52a8097990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:06.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.272+0000 7f52b06d4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52a80b1b20 0x7f52a81f54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:06.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.272+0000 7f52b06d4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52a80cf040 0x7f52a81f2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:06.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.272+0000 7f52b06d4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a81f2270 con 0x7f52a80b1b20 2023-09-23T03:24:06.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.272+0000 7f52b06d4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a8097ed0 con 0x7f52a80acce0 2023-09-23T03:24:06.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.272+0000 7f52b06d4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52a81f5d10 con 0x7f52a80cf040 2023-09-23T03:24:06.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.272+0000 7f52adc6f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52a80b1b20 0x7f52a81f54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:06.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.272+0000 7f52adc6f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52a80b1b20 0x7f52a81f54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36296/0 (socket says 172.21.15.154:36296) 2023-09-23T03:24:06.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.272+0000 7f52adc6f640 1 -- 172.21.15.154:0/2939909546 learned_addr learned my addr 172.21.15.154:0/2939909546 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:06.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.272+0000 7f52ae470640 1 --2- 172.21.15.154:0/2939909546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52a80acce0 0x7f52a8097990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:06.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.272+0000 7f52adc6f640 1 -- 172.21.15.154:0/2939909546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52a80cf040 msgr2=0x7f52a81f2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:06.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.272+0000 7f52adc6f640 1 --2- 172.21.15.154:0/2939909546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52a80cf040 0x7f52a81f2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:06.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.272+0000 7f52adc6f640 1 -- 172.21.15.154:0/2939909546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52a80acce0 msgr2=0x7f52a8097990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:06.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.272+0000 7f52adc6f640 1 --2- 172.21.15.154:0/2939909546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52a80acce0 0x7f52a8097990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:06.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.272+0000 7f52adc6f640 1 -- 172.21.15.154:0/2939909546 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a4009470 con 0x7f52a80b1b20 2023-09-23T03:24:06.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.272+0000 7f52ae470640 1 --2- 172.21.15.154:0/2939909546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52a80acce0 0x7f52a8097990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:24:06.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.273+0000 7f52adc6f640 1 --2- 172.21.15.154:0/2939909546 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52a80b1b20 0x7f52a81f54c0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f529800c430 tx=0x7f529800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:06.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.273+0000 7f529effd640 1 -- 172.21.15.154:0/2939909546 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5298011790 con 0x7f52a80b1b20 2023-09-23T03:24:06.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.273+0000 7f529effd640 1 -- 172.21.15.154:0/2939909546 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52980118f0 con 0x7f52a80b1b20 2023-09-23T03:24:06.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.273+0000 7f529effd640 1 -- 172.21.15.154:0/2939909546 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5298011ce0 con 0x7f52a80b1b20 2023-09-23T03:24:06.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.273+0000 7f52b06d4640 1 -- 172.21.15.154:0/2939909546 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52a81f5ff0 con 0x7f52a80b1b20 2023-09-23T03:24:06.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.273+0000 7f52b06d4640 1 -- 172.21.15.154:0/2939909546 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52a81f1ed0 con 0x7f52a80b1b20 2023-09-23T03:24:06.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.274+0000 7f52b06d4640 1 -- 172.21.15.154:0/2939909546 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52a81f2420 con 0x7f52a80b1b20 2023-09-23T03:24:06.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.275+0000 7f529effd640 1 -- 172.21.15.154:0/2939909546 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f529801c460 con 0x7f52a80b1b20 2023-09-23T03:24:06.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.276+0000 7f529effd640 1 --2- 172.21.15.154:0/2939909546 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f528c06af60 0x7f528c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:06.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.276+0000 7f529effd640 1 -- 172.21.15.154:0/2939909546 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f52980893f0 con 0x7f52a80b1b20 2023-09-23T03:24:06.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.277+0000 7f52ae470640 1 --2- 172.21.15.154:0/2939909546 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f528c06af60 0x7f528c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:06.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.277+0000 7f52ae470640 1 --2- 172.21.15.154:0/2939909546 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f528c06af60 0x7f528c06d420 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f52a4036000 tx=0x7f52a403d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:06.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.280+0000 7f529effd640 1 -- 172.21.15.154:0/2939909546 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f52980546d0 con 0x7f52a80b1b20 2023-09-23T03:24:06.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.469+0000 7f52b06d4640 1 -- 172.21.15.154:0/2939909546 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52a8073eb0 con 0x7f52a80b1b20 2023-09-23T03:24:06.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.471+0000 7f529effd640 1 -- 172.21.15.154:0/2939909546 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+13099 (secure 0 0 0) 0x7f5298057cf0 con 0x7f52a80b1b20 2023-09-23T03:24:06.473 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:06.473 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":79,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:24:05.554359+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:24:06.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.474+0000 7f52b06d4640 1 -- 172.21.15.154:0/2939909546 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f528c06af60 msgr2=0x7f528c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:06.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.474+0000 7f52b06d4640 1 --2- 172.21.15.154:0/2939909546 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f528c06af60 0x7f528c06d420 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f52a4036000 tx=0x7f52a403d040 comp rx=0 tx=0).stop 2023-09-23T03:24:06.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.474+0000 7f52b06d4640 1 -- 172.21.15.154:0/2939909546 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52a80b1b20 msgr2=0x7f52a81f54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:06.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.474+0000 7f52b06d4640 1 --2- 172.21.15.154:0/2939909546 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52a80b1b20 0x7f52a81f54c0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f529800c430 tx=0x7f529800c8f0 comp rx=0 tx=0).stop 2023-09-23T03:24:06.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.474+0000 7f52b06d4640 1 -- 172.21.15.154:0/2939909546 shutdown_connections 2023-09-23T03:24:06.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.474+0000 7f52b06d4640 1 --2- 172.21.15.154:0/2939909546 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f528c06af60 0x7f528c06d420 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:06.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.474+0000 7f52b06d4640 1 --2- 172.21.15.154:0/2939909546 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f52a80cf040 0x7f52a81f2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:06.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.474+0000 7f52b06d4640 1 --2- 172.21.15.154:0/2939909546 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52a80b1b20 0x7f52a81f54c0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:06.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.474+0000 7f52b06d4640 1 --2- 172.21.15.154:0/2939909546 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f52a80acce0 0x7f52a8097990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:06.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.474+0000 7f52b06d4640 1 -- 172.21.15.154:0/2939909546 >> 172.21.15.154:0/2939909546 conn(0x7f52a806a0f0 msgr2=0x7f52a80ce900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:06.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.474+0000 7f52b06d4640 1 -- 172.21.15.154:0/2939909546 shutdown_connections 2023-09-23T03:24:06.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.474+0000 7f52b06d4640 1 -- 172.21.15.154:0/2939909546 wait complete. 2023-09-23T03:24:06.492 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T03:24:06.493 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:24:06.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.702+0000 7f1f48b4c640 1 Processor -- start 2023-09-23T03:24:06.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.702+0000 7f1f48b4c640 1 -- start start 2023-09-23T03:24:06.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.703+0000 7f1f48b4c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1f440c8d70 0x7f1f4415c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:06.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.703+0000 7f1f48b4c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f44071290 0x7f1f440ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:06.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.703+0000 7f1f48b4c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f440c9d20 con 0x7f1f440c9150 2023-09-23T03:24:06.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.703+0000 7f1f48b4c640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f44071850 con 0x7f1f440c8d70 2023-09-23T03:24:06.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.703+0000 7f1f48b4c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f44071b30 con 0x7f1f44071290 2023-09-23T03:24:06.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.704+0000 7f1f3bfff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f1f440c9150 0x7f1f4415cc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:54432/0 (socket says 172.21.15.154:54432) 2023-09-23T03:24:06.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.704+0000 7f1f3bfff640 1 -- 172.21.15.154:0/3617824089 learned_addr learned my addr 172.21.15.154:0/3617824089 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:24:06.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.704+0000 7f1f4259c640 1 --2- 172.21.15.154:0/3617824089 >> v2:172.21.15.190:3300/0 conn(0x7f1f440c8d70 0x7f1f4415c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:06.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.704+0000 7f1f42d9d640 1 --2- 172.21.15.154:0/3617824089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f44071290 0x7f1f440ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:06.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.704+0000 7f1f41d9b640 1 -- 172.21.15.154:0/3617824089 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4080339229 0 0) 0x7f1f440c9d20 con 0x7f1f440c9150 2023-09-23T03:24:06.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.704+0000 7f1f41d9b640 1 -- 172.21.15.154:0/3617824089 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f30003520 con 0x7f1f440c9150 2023-09-23T03:24:06.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.705+0000 7f1f41d9b640 1 -- 172.21.15.154:0/3617824089 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 756614212 0 0) 0x7f1f30003520 con 0x7f1f440c9150 2023-09-23T03:24:06.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.705+0000 7f1f41d9b640 1 -- 172.21.15.154:0/3617824089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f44071290 msgr2=0x7f1f440ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:06.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.705+0000 7f1f41d9b640 1 --2- 172.21.15.154:0/3617824089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f44071290 0x7f1f440ca1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:06.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.705+0000 7f1f41d9b640 1 -- 172.21.15.154:0/3617824089 >> v2:172.21.15.190:3300/0 conn(0x7f1f440c8d70 msgr2=0x7f1f4415c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:06.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.705+0000 7f1f41d9b640 1 --2- 172.21.15.154:0/3617824089 >> v2:172.21.15.190:3300/0 conn(0x7f1f440c8d70 0x7f1f4415c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:06.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.705+0000 7f1f41d9b640 1 -- 172.21.15.154:0/3617824089 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f44071e10 con 0x7f1f440c9150 2023-09-23T03:24:06.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.705+0000 7f1f41d9b640 1 -- 172.21.15.154:0/3617824089 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f1f2c001690 con 0x7f1f440c9150 2023-09-23T03:24:06.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.705+0000 7f1f41d9b640 1 -- 172.21.15.154:0/3617824089 >> v1:172.21.15.154:6789/0 conn(0x7f1f440c9150 legacy=0x7f1f4415cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:06.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.705+0000 7f1f41d9b640 1 --2- 172.21.15.154:0/3617824089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f30003970 0x7f1f30003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:06.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.705+0000 7f1f41d9b640 1 --2- 172.21.15.154:0/3617824089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f44071290 0x7f1f300047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:06.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.705+0000 7f1f41d9b640 1 --2- 172.21.15.154:0/3617824089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f30004d00 0x7f1f30005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:06.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.705+0000 7f1f41d9b640 1 -- 172.21.15.154:0/3617824089 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f44071850 con 0x7f1f30004d00 2023-09-23T03:24:06.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.705+0000 7f1f41d9b640 1 -- 172.21.15.154:0/3617824089 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f44071b30 con 0x7f1f30003970 2023-09-23T03:24:06.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.705+0000 7f1f41d9b640 1 -- 172.21.15.154:0/3617824089 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3000db90 con 0x7f1f44071290 2023-09-23T03:24:06.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.705+0000 7f1f42d9d640 1 --2- 172.21.15.154:0/3617824089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f30004d00 0x7f1f30005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:06.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.705+0000 7f1f42d9d640 1 -- 172.21.15.154:0/3617824089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f44071290 msgr2=0x7f1f300047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:06.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.705+0000 7f1f3bfff640 1 --2- 172.21.15.154:0/3617824089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f44071290 0x7f1f300047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:06.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.705+0000 7f1f42d9d640 1 --2- 172.21.15.154:0/3617824089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f44071290 0x7f1f300047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:06.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.705+0000 7f1f42d9d640 1 -- 172.21.15.154:0/3617824089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f30003970 msgr2=0x7f1f30003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:06.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.705+0000 7f1f42d9d640 1 --2- 172.21.15.154:0/3617824089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f30003970 0x7f1f30003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:06.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.705+0000 7f1f42d9d640 1 -- 172.21.15.154:0/3617824089 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f30003520 con 0x7f1f30004d00 2023-09-23T03:24:06.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.706+0000 7f1f3bfff640 1 --2- 172.21.15.154:0/3617824089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f44071290 0x7f1f300047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:06.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.706+0000 7f1f42d9d640 1 --2- 172.21.15.154:0/3617824089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f30004d00 0x7f1f30005180 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00e170 tx=0x7f1f300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2e2d4fee8932575 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:06.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.707+0000 7f1f41d9b640 1 -- 172.21.15.154:0/3617824089 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c02cbf0 con 0x7f1f30004d00 2023-09-23T03:24:06.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.707+0000 7f1f41d9b640 1 -- 172.21.15.154:0/3617824089 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f3c00f040 con 0x7f1f30004d00 2023-09-23T03:24:06.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.707+0000 7f1f41d9b640 1 -- 172.21.15.154:0/3617824089 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c034440 con 0x7f1f30004d00 2023-09-23T03:24:06.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.707+0000 7f1f48b4c640 1 -- 172.21.15.154:0/3617824089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f30004d00 msgr2=0x7f1f30005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:06.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.707+0000 7f1f48b4c640 1 --2- 172.21.15.154:0/3617824089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f30004d00 0x7f1f30005180 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00e170 tx=0x7f1f300057f0 comp rx=0 tx=0).stop 2023-09-23T03:24:06.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.707+0000 7f1f42d9d640 1 -- 172.21.15.154:0/3617824089 reap_dead start 2023-09-23T03:24:06.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.707+0000 7f1f48b4c640 1 -- 172.21.15.154:0/3617824089 shutdown_connections 2023-09-23T03:24:06.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.707+0000 7f1f48b4c640 1 -- 172.21.15.154:0/3617824089 >> 172.21.15.154:0/3617824089 conn(0x7f1f4406c010 msgr2=0x7f1f4406c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:06.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.707+0000 7f1f48b4c640 1 -- 172.21.15.154:0/3617824089 shutdown_connections 2023-09-23T03:24:06.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.707+0000 7f1f48b4c640 1 -- 172.21.15.154:0/3617824089 wait complete. 2023-09-23T03:24:06.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.709+0000 7f1f48b4c640 1 Processor -- start 2023-09-23T03:24:06.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.709+0000 7f1f48b4c640 1 -- start start 2023-09-23T03:24:06.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.710+0000 7f1f48b4c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f30004d00 0x7f1f441491a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:06.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.710+0000 7f1f48b4c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f441496e0 0x7f1f4414a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:06.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.710+0000 7f1f48b4c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f4414a730 0x7f1f4414abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:06.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.710+0000 7f1f48b4c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f440c5f50 con 0x7f1f4414a730 2023-09-23T03:24:06.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.710+0000 7f1f48b4c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f440c5620 con 0x7f1f30004d00 2023-09-23T03:24:06.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.710+0000 7f1f48b4c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f440c5890 con 0x7f1f441496e0 2023-09-23T03:24:06.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.711+0000 7f1f42d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f4414a730 0x7f1f4414abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:06.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.711+0000 7f1f42d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f4414a730 0x7f1f4414abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36308/0 (socket says 172.21.15.154:36308) 2023-09-23T03:24:06.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.711+0000 7f1f42d9d640 1 -- 172.21.15.154:0/1793842606 learned_addr learned my addr 172.21.15.154:0/1793842606 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:06.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.711+0000 7f1f42d9d640 1 -- 172.21.15.154:0/1793842606 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f441496e0 msgr2=0x7f1f4414a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:06.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.711+0000 7f1f3bfff640 1 --2- 172.21.15.154:0/1793842606 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f441496e0 0x7f1f4414a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:06.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.711+0000 7f1f4259c640 1 --2- 172.21.15.154:0/1793842606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f30004d00 0x7f1f441491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:06.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.711+0000 7f1f42d9d640 1 --2- 172.21.15.154:0/1793842606 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f441496e0 0x7f1f4414a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:06.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.711+0000 7f1f42d9d640 1 -- 172.21.15.154:0/1793842606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f30004d00 msgr2=0x7f1f441491a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:06.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.711+0000 7f1f42d9d640 1 --2- 172.21.15.154:0/1793842606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f30004d00 0x7f1f441491a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:06.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.711+0000 7f1f42d9d640 1 -- 172.21.15.154:0/1793842606 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f440c5b70 con 0x7f1f4414a730 2023-09-23T03:24:06.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.711+0000 7f1f42d9d640 1 --2- 172.21.15.154:0/1793842606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f4414a730 0x7f1f4414abb0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c009f90 tx=0x7f1f3c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:06.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.712+0000 7f1f3affd640 1 -- 172.21.15.154:0/1793842606 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c02de80 con 0x7f1f4414a730 2023-09-23T03:24:06.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.712+0000 7f1f3affd640 1 -- 172.21.15.154:0/1793842606 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f3c034ad0 con 0x7f1f4414a730 2023-09-23T03:24:06.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.712+0000 7f1f3affd640 1 -- 172.21.15.154:0/1793842606 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c034e80 con 0x7f1f4414a730 2023-09-23T03:24:06.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.712+0000 7f1f48b4c640 1 -- 172.21.15.154:0/1793842606 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f441f1d80 con 0x7f1f4414a730 2023-09-23T03:24:06.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.712+0000 7f1f48b4c640 1 -- 172.21.15.154:0/1793842606 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f44149b60 con 0x7f1f4414a730 2023-09-23T03:24:06.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.713+0000 7f1f48b4c640 1 -- 172.21.15.154:0/1793842606 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f440c6090 con 0x7f1f4414a730 2023-09-23T03:24:06.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.714+0000 7f1f3affd640 1 -- 172.21.15.154:0/1793842606 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f1f3c03e440 con 0x7f1f4414a730 2023-09-23T03:24:06.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.715+0000 7f1f3affd640 1 --2- 172.21.15.154:0/1793842606 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1f2406af30 0x7f1f2406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:06.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.715+0000 7f1f3affd640 1 -- 172.21.15.154:0/1793842606 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f1f3c0ab4a0 con 0x7f1f4414a730 2023-09-23T03:24:06.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.715+0000 7f1f4259c640 1 --2- 172.21.15.154:0/1793842606 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1f2406af30 0x7f1f2406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:06.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.716+0000 7f1f4259c640 1 --2- 172.21.15.154:0/1793842606 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1f2406af30 0x7f1f2406d3f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1f34004770 tx=0x7f1f34008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:06.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.719+0000 7f1f3affd640 1 -- 172.21.15.154:0/1793842606 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f1f440c6090 con 0x7f1f4414a730 2023-09-23T03:24:06.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.906+0000 7f1f48b4c640 1 -- 172.21.15.154:0/1793842606 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1f440c6090 con 0x7f1f4414a730 2023-09-23T03:24:06.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.907+0000 7f1f3affd640 1 -- 172.21.15.154:0/1793842606 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1f440c6090 con 0x7f1f4414a730 2023-09-23T03:24:06.909 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:24:06.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.909+0000 7f1f48b4c640 1 -- 172.21.15.154:0/1793842606 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1f2406af30 msgr2=0x7f1f2406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:06.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.909+0000 7f1f48b4c640 1 --2- 172.21.15.154:0/1793842606 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1f2406af30 0x7f1f2406d3f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1f34004770 tx=0x7f1f34008040 comp rx=0 tx=0).stop 2023-09-23T03:24:06.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.909+0000 7f1f48b4c640 1 -- 172.21.15.154:0/1793842606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f4414a730 msgr2=0x7f1f4414abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:06.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.909+0000 7f1f48b4c640 1 --2- 172.21.15.154:0/1793842606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f4414a730 0x7f1f4414abb0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c009f90 tx=0x7f1f3c009750 comp rx=0 tx=0).stop 2023-09-23T03:24:06.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.910+0000 7f1f48b4c640 1 -- 172.21.15.154:0/1793842606 shutdown_connections 2023-09-23T03:24:06.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.910+0000 7f1f48b4c640 1 --2- 172.21.15.154:0/1793842606 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1f2406af30 0x7f1f2406d3f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:06.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.910+0000 7f1f48b4c640 1 --2- 172.21.15.154:0/1793842606 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f4414a730 0x7f1f4414abb0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:06.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.910+0000 7f1f48b4c640 1 --2- 172.21.15.154:0/1793842606 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f441496e0 0x7f1f4414a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:06.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.910+0000 7f1f48b4c640 1 --2- 172.21.15.154:0/1793842606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f30004d00 0x7f1f441491a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:06.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.910+0000 7f1f48b4c640 1 -- 172.21.15.154:0/1793842606 >> 172.21.15.154:0/1793842606 conn(0x7f1f4406c010 msgr2=0x7f1f440c1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:06.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.910+0000 7f1f48b4c640 1 -- 172.21.15.154:0/1793842606 shutdown_connections 2023-09-23T03:24:06.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:06.910+0000 7f1f48b4c640 1 -- 172.21.15.154:0/1793842606 wait complete. 2023-09-23T03:24:06.929 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-09-23T03:24:07.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.136+0000 7f2cbc939640 1 Processor -- start 2023-09-23T03:24:07.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.136+0000 7f2cbc939640 1 -- start start 2023-09-23T03:24:07.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.137+0000 7f2cbc939640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cb80acce0 0x7f2cb815ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:07.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.137+0000 7f2cbc939640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2cb80b1b20 0x7f2cb80ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:07.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.137+0000 7f2cbc939640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2cb80cee70 0x7f2cb806cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:07.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.137+0000 7f2cbc939640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cb80b2e90 con 0x7f2cb80b1b20 2023-09-23T03:24:07.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.137+0000 7f2cbc939640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cb80cf320 con 0x7f2cb80cee70 2023-09-23T03:24:07.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.137+0000 7f2cbc939640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb815ac30 con 0x7f2cb80acce0 2023-09-23T03:24:07.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.138+0000 7f2caffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2cb80b1b20 0x7f2cb80ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:07.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.138+0000 7f2caffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2cb80b1b20 0x7f2cb80ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36316/0 (socket says 172.21.15.154:36316) 2023-09-23T03:24:07.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.138+0000 7f2caffff640 1 -- 172.21.15.154:0/1695943996 learned_addr learned my addr 172.21.15.154:0/1695943996 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:07.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.138+0000 7f2cb659c640 1 --2- 172.21.15.154:0/1695943996 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cb80acce0 0x7f2cb815ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:07.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.138+0000 7f2cb6d9d640 1 --2- 172.21.15.154:0/1695943996 >> v2:172.21.15.190:3300/0 conn(0x7f2cb80cee70 0x7f2cb806cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:07.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.138+0000 7f2caffff640 1 -- 172.21.15.154:0/1695943996 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cb80acce0 msgr2=0x7f2cb815ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:07.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.138+0000 7f2caffff640 1 --2- 172.21.15.154:0/1695943996 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cb80acce0 0x7f2cb815ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:07.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.138+0000 7f2caffff640 1 -- 172.21.15.154:0/1695943996 >> v2:172.21.15.190:3300/0 conn(0x7f2cb80cee70 msgr2=0x7f2cb806cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:07.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.138+0000 7f2caffff640 1 --2- 172.21.15.154:0/1695943996 >> v2:172.21.15.190:3300/0 conn(0x7f2cb80cee70 0x7f2cb806cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:07.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.138+0000 7f2caffff640 1 -- 172.21.15.154:0/1695943996 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb80b1f70 con 0x7f2cb80b1b20 2023-09-23T03:24:07.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.138+0000 7f2cb6d9d640 1 --2- 172.21.15.154:0/1695943996 >> v2:172.21.15.190:3300/0 conn(0x7f2cb80cee70 0x7f2cb806cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:07.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.139+0000 7f2caffff640 1 --2- 172.21.15.154:0/1695943996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cb80b1b20 0x7f2cb80ce930 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f2cb80c9150 tx=0x7f2ca002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78cd1df3e1a048b0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:07.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.139+0000 7f2cb5d9b640 1 -- 172.21.15.154:0/1695943996 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca00317c0 con 0x7f2cb80b1b20 2023-09-23T03:24:07.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.140+0000 7f2cb5d9b640 1 -- 172.21.15.154:0/1695943996 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca0031920 con 0x7f2cb80b1b20 2023-09-23T03:24:07.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.140+0000 7f2cb5d9b640 1 -- 172.21.15.154:0/1695943996 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0031cb0 con 0x7f2cb80b1b20 2023-09-23T03:24:07.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.140+0000 7f2cbc939640 1 -- 172.21.15.154:0/1695943996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cb80b1b20 msgr2=0x7f2cb80ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:07.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.140+0000 7f2cbc939640 1 --2- 172.21.15.154:0/1695943996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cb80b1b20 0x7f2cb80ce930 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f2cb80c9150 tx=0x7f2ca002b860 comp rx=0 tx=0).stop 2023-09-23T03:24:07.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.140+0000 7f2cbc939640 1 -- 172.21.15.154:0/1695943996 shutdown_connections 2023-09-23T03:24:07.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.140+0000 7f2cbc939640 1 --2- 172.21.15.154:0/1695943996 >> v2:172.21.15.190:3300/0 conn(0x7f2cb80cee70 0x7f2cb806cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:07.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.140+0000 7f2cbc939640 1 --2- 172.21.15.154:0/1695943996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cb80b1b20 0x7f2cb80ce930 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:07.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.140+0000 7f2cbc939640 1 --2- 172.21.15.154:0/1695943996 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cb80acce0 0x7f2cb815ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:07.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.140+0000 7f2cbc939640 1 -- 172.21.15.154:0/1695943996 >> 172.21.15.154:0/1695943996 conn(0x7f2cb806a0f0 msgr2=0x7f2cb806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:07.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.140+0000 7f2cbc939640 1 -- 172.21.15.154:0/1695943996 shutdown_connections 2023-09-23T03:24:07.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.140+0000 7f2cbc939640 1 -- 172.21.15.154:0/1695943996 wait complete. 2023-09-23T03:24:07.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.142+0000 7f2cbc939640 1 Processor -- start 2023-09-23T03:24:07.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.142+0000 7f2cbc939640 1 -- start start 2023-09-23T03:24:07.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.143+0000 7f2cbc939640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cb80acce0 0x7f2cb8159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:07.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.143+0000 7f2cbc939640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cb80b1b20 0x7f2cb80abc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:07.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.143+0000 7f2cbc939640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cb80cee70 0x7f2cb80ac1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:07.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.143+0000 7f2cbc939640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb81f2b00 con 0x7f2cb80acce0 2023-09-23T03:24:07.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.143+0000 7f2cbc939640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb80c8530 con 0x7f2cb80cee70 2023-09-23T03:24:07.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.143+0000 7f2cbc939640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb81f2160 con 0x7f2cb80b1b20 2023-09-23T03:24:07.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.143+0000 7f2cb659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cb80acce0 0x7f2cb8159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:07.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.143+0000 7f2cb659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cb80acce0 0x7f2cb8159c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36320/0 (socket says 172.21.15.154:36320) 2023-09-23T03:24:07.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.143+0000 7f2cb659c640 1 -- 172.21.15.154:0/1772327504 learned_addr learned my addr 172.21.15.154:0/1772327504 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:07.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.144+0000 7f2cb659c640 1 -- 172.21.15.154:0/1772327504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cb80b1b20 msgr2=0x7f2cb80abc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:07.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.144+0000 7f2cb659c640 1 --2- 172.21.15.154:0/1772327504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cb80b1b20 0x7f2cb80abc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:07.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.144+0000 7f2cb659c640 1 -- 172.21.15.154:0/1772327504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cb80cee70 msgr2=0x7f2cb80ac1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:07.147 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.144+0000 7f2cb659c640 1 --2- 172.21.15.154:0/1772327504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cb80cee70 0x7f2cb80ac1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:07.147 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.144+0000 7f2cb659c640 1 -- 172.21.15.154:0/1772327504 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca00094d0 con 0x7f2cb80acce0 2023-09-23T03:24:07.147 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.144+0000 7f2cb659c640 1 --2- 172.21.15.154:0/1772327504 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cb80acce0 0x7f2cb8159c50 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f2ca800d800 tx=0x7f2ca800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:07.147 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.144+0000 7f2caeffd640 1 -- 172.21.15.154:0/1772327504 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca800b6c0 con 0x7f2cb80acce0 2023-09-23T03:24:07.147 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.144+0000 7f2caeffd640 1 -- 172.21.15.154:0/1772327504 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca800b820 con 0x7f2cb80acce0 2023-09-23T03:24:07.147 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.144+0000 7f2caeffd640 1 -- 172.21.15.154:0/1772327504 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca800bbd0 con 0x7f2cb80acce0 2023-09-23T03:24:07.147 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.144+0000 7f2cbc939640 1 -- 172.21.15.154:0/1772327504 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cb81f2440 con 0x7f2cb80acce0 2023-09-23T03:24:07.147 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.145+0000 7f2cbc939640 1 -- 172.21.15.154:0/1772327504 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cb80c63a0 con 0x7f2cb80acce0 2023-09-23T03:24:07.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.146+0000 7f2cbc939640 1 -- 172.21.15.154:0/1772327504 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cb8073eb0 con 0x7f2cb80acce0 2023-09-23T03:24:07.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.147+0000 7f2caeffd640 1 -- 172.21.15.154:0/1772327504 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f2ca800bd30 con 0x7f2cb80acce0 2023-09-23T03:24:07.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.147+0000 7f2caeffd640 1 --2- 172.21.15.154:0/1772327504 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2c9806af30 0x7f2c9806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:07.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.148+0000 7f2caeffd640 1 -- 172.21.15.154:0/1772327504 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f2ca8089b70 con 0x7f2cb80acce0 2023-09-23T03:24:07.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.148+0000 7f2caffff640 1 --2- 172.21.15.154:0/1772327504 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2c9806af30 0x7f2c9806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:07.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.148+0000 7f2caffff640 1 --2- 172.21.15.154:0/1772327504 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2c9806af30 0x7f2c9806d3f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0036000 tx=0x7f2ca0030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:07.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.152+0000 7f2caeffd640 1 -- 172.21.15.154:0/1772327504 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f2ca8054d20 con 0x7f2cb80acce0 2023-09-23T03:24:07.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.343+0000 7f2cbc939640 1 -- 172.21.15.154:0/1772327504 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2cb806c930 con 0x7f2cb80acce0 2023-09-23T03:24:07.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.344+0000 7f2caeffd640 1 -- 172.21.15.154:0/1772327504 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) v1 ==== 84+0+65 (secure 0 0 0) 0x7f2ca8058340 con 0x7f2cb80acce0 2023-09-23T03:24:07.345 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:07.345 INFO:teuthology.orchestra.run.smithi154.stdout:[ 2023-09-23T03:24:07.345 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:24:07.345 INFO:teuthology.orchestra.run.smithi154.stdout: "poolnum": 1, 2023-09-23T03:24:07.345 INFO:teuthology.orchestra.run.smithi154.stdout: "poolname": ".mgr" 2023-09-23T03:24:07.346 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:24:07.346 INFO:teuthology.orchestra.run.smithi154.stdout:] 2023-09-23T03:24:07.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.346+0000 7f2cbc939640 1 -- 172.21.15.154:0/1772327504 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2c9806af30 msgr2=0x7f2c9806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:07.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.346+0000 7f2cbc939640 1 --2- 172.21.15.154:0/1772327504 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2c9806af30 0x7f2c9806d3f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0036000 tx=0x7f2ca0030000 comp rx=0 tx=0).stop 2023-09-23T03:24:07.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.346+0000 7f2cbc939640 1 -- 172.21.15.154:0/1772327504 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cb80acce0 msgr2=0x7f2cb8159c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:07.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.346+0000 7f2cbc939640 1 --2- 172.21.15.154:0/1772327504 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cb80acce0 0x7f2cb8159c50 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f2ca800d800 tx=0x7f2ca800dcc0 comp rx=0 tx=0).stop 2023-09-23T03:24:07.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.346+0000 7f2cbc939640 1 -- 172.21.15.154:0/1772327504 shutdown_connections 2023-09-23T03:24:07.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.346+0000 7f2cbc939640 1 --2- 172.21.15.154:0/1772327504 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2c9806af30 0x7f2c9806d3f0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:07.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.346+0000 7f2cbc939640 1 --2- 172.21.15.154:0/1772327504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2cb80cee70 0x7f2cb80ac1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:07.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.346+0000 7f2cbc939640 1 --2- 172.21.15.154:0/1772327504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2cb80b1b20 0x7f2cb80abc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:07.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.346+0000 7f2cbc939640 1 --2- 172.21.15.154:0/1772327504 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cb80acce0 0x7f2cb8159c50 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:07.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.346+0000 7f2cbc939640 1 -- 172.21.15.154:0/1772327504 >> 172.21.15.154:0/1772327504 conn(0x7f2cb806a0f0 msgr2=0x7f2cb806a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:07.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.346+0000 7f2cbc939640 1 -- 172.21.15.154:0/1772327504 shutdown_connections 2023-09-23T03:24:07.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.346+0000 7f2cbc939640 1 -- 172.21.15.154:0/1772327504 wait complete. 2023-09-23T03:24:07.366 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-09-23T03:24:07.366 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-09-23T03:24:07.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.572+0000 7f3ab25c4640 1 Processor -- start 2023-09-23T03:24:07.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.572+0000 7f3ab25c4640 1 -- start start 2023-09-23T03:24:07.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.573+0000 7f3ab25c4640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3aac0afd70 0x7f3aac0b0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:07.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.574+0000 7f3ab25c4640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3aac15adf0 0x7f3aac0b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:07.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.574+0000 7f3ab25c4640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aac0cc1f0 con 0x7f3aac0b2300 2023-09-23T03:24:07.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.574+0000 7f3ab25c4640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aac0b0690 con 0x7f3aac0afd70 2023-09-23T03:24:07.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.574+0000 7f3ab25c4640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aac0b2830 con 0x7f3aac15adf0 2023-09-23T03:24:07.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.574+0000 7f3ab1dc3640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f3aac0b2300 0x7f3aac0cad00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:54446/0 (socket says 172.21.15.154:54446) 2023-09-23T03:24:07.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.574+0000 7f3ab1dc3640 1 -- 172.21.15.154:0/2921907054 learned_addr learned my addr 172.21.15.154:0/2921907054 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:24:07.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.574+0000 7f3ab15c2640 1 --2- 172.21.15.154:0/2921907054 >> v2:172.21.15.154:3300/0 conn(0x7f3aac0afd70 0x7f3aac0b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:07.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.574+0000 7f3aa3fff640 1 -- 172.21.15.154:0/2921907054 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1206522928 0 0) 0x7f3aac0cc1f0 con 0x7f3aac0b2300 2023-09-23T03:24:07.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.574+0000 7f3ab0dc1640 1 --2- 172.21.15.154:0/2921907054 >> v2:172.21.15.190:3300/0 conn(0x7f3aac15adf0 0x7f3aac0b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:07.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.574+0000 7f3aa3fff640 1 -- 172.21.15.154:0/2921907054 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a98003520 con 0x7f3aac0b2300 2023-09-23T03:24:07.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.575+0000 7f3ab15c2640 1 -- 172.21.15.154:0/2921907054 >> v2:172.21.15.190:3300/0 conn(0x7f3aac15adf0 msgr2=0x7f3aac0b1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:07.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.575+0000 7f3ab15c2640 1 --2- 172.21.15.154:0/2921907054 >> v2:172.21.15.190:3300/0 conn(0x7f3aac15adf0 0x7f3aac0b1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:07.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.575+0000 7f3ab15c2640 1 -- 172.21.15.154:0/2921907054 >> v1:172.21.15.154:6789/0 conn(0x7f3aac0b2300 legacy=0x7f3aac0cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:07.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.575+0000 7f3aa3fff640 1 -- 172.21.15.154:0/2921907054 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 653794836 0 0) 0x7f3a98003520 con 0x7f3aac0b2300 2023-09-23T03:24:07.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.575+0000 7f3ab15c2640 1 -- 172.21.15.154:0/2921907054 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aac0b2b10 con 0x7f3aac0afd70 2023-09-23T03:24:07.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.576+0000 7f3ab15c2640 1 --2- 172.21.15.154:0/2921907054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3aac0afd70 0x7f3aac0b0150 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f3aac0c1ce0 tx=0x7f3aa802b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd8ca4d4e30407e7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:07.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.576+0000 7f3aa3fff640 1 -- 172.21.15.154:0/2921907054 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa40019f0 con 0x7f3aac0afd70 2023-09-23T03:24:07.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.576+0000 7f3aa3fff640 1 -- 172.21.15.154:0/2921907054 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa8033440 con 0x7f3aac0afd70 2023-09-23T03:24:07.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.576+0000 7f3aa3fff640 1 -- 172.21.15.154:0/2921907054 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa80337b0 con 0x7f3aac0afd70 2023-09-23T03:24:07.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.577+0000 7f3ab25c4640 1 -- 172.21.15.154:0/2921907054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3aac0afd70 msgr2=0x7f3aac0b0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:07.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.577+0000 7f3ab25c4640 1 --2- 172.21.15.154:0/2921907054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3aac0afd70 0x7f3aac0b0150 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f3aac0c1ce0 tx=0x7f3aa802b7a0 comp rx=0 tx=0).stop 2023-09-23T03:24:07.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.577+0000 7f3ab25c4640 1 -- 172.21.15.154:0/2921907054 shutdown_connections 2023-09-23T03:24:07.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.577+0000 7f3ab25c4640 1 --2- 172.21.15.154:0/2921907054 >> v2:172.21.15.190:3300/0 conn(0x7f3aac15adf0 0x7f3aac0b1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:07.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.577+0000 7f3ab25c4640 1 --2- 172.21.15.154:0/2921907054 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3aac0afd70 0x7f3aac0b0150 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:07.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.577+0000 7f3ab25c4640 1 -- 172.21.15.154:0/2921907054 >> 172.21.15.154:0/2921907054 conn(0x7f3aac069cd0 msgr2=0x7f3aac06a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:07.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.577+0000 7f3ab25c4640 1 -- 172.21.15.154:0/2921907054 shutdown_connections 2023-09-23T03:24:07.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.577+0000 7f3ab25c4640 1 -- 172.21.15.154:0/2921907054 wait complete. 2023-09-23T03:24:07.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.578+0000 7f3ab25c4640 1 Processor -- start 2023-09-23T03:24:07.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.579+0000 7f3ab25c4640 1 -- start start 2023-09-23T03:24:07.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.580+0000 7f3ab25c4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3aac0afd70 0x7f3aac0c6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:07.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.580+0000 7f3ab25c4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3aac0b2300 0x7f3aac0cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:07.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.580+0000 7f3ab25c4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3aac15adf0 0x7f3aac1575f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:07.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.580+0000 7f3ab25c4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac0cef30 con 0x7f3aac0b2300 2023-09-23T03:24:07.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.580+0000 7f3ab25c4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac06d5b0 con 0x7f3aac15adf0 2023-09-23T03:24:07.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.580+0000 7f3ab25c4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac0cec50 con 0x7f3aac0afd70 2023-09-23T03:24:07.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.580+0000 7f3ab0dc1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3aac0b2300 0x7f3aac0cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:07.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.580+0000 7f3ab0dc1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3aac0b2300 0x7f3aac0cdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36344/0 (socket says 172.21.15.154:36344) 2023-09-23T03:24:07.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.580+0000 7f3ab0dc1640 1 -- 172.21.15.154:0/148553209 learned_addr learned my addr 172.21.15.154:0/148553209 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:07.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.580+0000 7f3ab15c2640 1 --2- 172.21.15.154:0/148553209 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3aac0afd70 0x7f3aac0c6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:07.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.580+0000 7f3ab0dc1640 1 -- 172.21.15.154:0/148553209 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3aac0afd70 msgr2=0x7f3aac0c6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:07.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.580+0000 7f3ab0dc1640 1 --2- 172.21.15.154:0/148553209 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3aac0afd70 0x7f3aac0c6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:07.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.580+0000 7f3ab0dc1640 1 -- 172.21.15.154:0/148553209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3aac15adf0 msgr2=0x7f3aac1575f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:07.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.580+0000 7f3ab0dc1640 1 --2- 172.21.15.154:0/148553209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3aac15adf0 0x7f3aac1575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:07.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.580+0000 7f3ab0dc1640 1 -- 172.21.15.154:0/148553209 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa8009400 con 0x7f3aac0b2300 2023-09-23T03:24:07.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.581+0000 7f3ab0dc1640 1 --2- 172.21.15.154:0/148553209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3aac0b2300 0x7f3aac0cdc10 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c00d7d0 tx=0x7f3a9c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:07.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.581+0000 7f3aa1ffb640 1 -- 172.21.15.154:0/148553209 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c00b6c0 con 0x7f3aac0b2300 2023-09-23T03:24:07.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.581+0000 7f3aa1ffb640 1 -- 172.21.15.154:0/148553209 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9c00b820 con 0x7f3aac0b2300 2023-09-23T03:24:07.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.581+0000 7f3ab25c4640 1 -- 172.21.15.154:0/148553209 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aac0c5560 con 0x7f3aac0b2300 2023-09-23T03:24:07.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.581+0000 7f3aa1ffb640 1 -- 172.21.15.154:0/148553209 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c00bbb0 con 0x7f3aac0b2300 2023-09-23T03:24:07.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.581+0000 7f3ab25c4640 1 -- 172.21.15.154:0/148553209 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aac0c5b20 con 0x7f3aac0b2300 2023-09-23T03:24:07.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.582+0000 7f3ab25c4640 1 -- 172.21.15.154:0/148553209 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aac0c6580 con 0x7f3aac0b2300 2023-09-23T03:24:07.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.583+0000 7f3aa1ffb640 1 -- 172.21.15.154:0/148553209 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f3a9c00bd10 con 0x7f3aac0b2300 2023-09-23T03:24:07.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.584+0000 7f3aa1ffb640 1 --2- 172.21.15.154:0/148553209 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3a9006af60 0x7f3a9006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:07.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.584+0000 7f3aa1ffb640 1 -- 172.21.15.154:0/148553209 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f3a9c089b10 con 0x7f3aac0b2300 2023-09-23T03:24:07.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.584+0000 7f3ab15c2640 1 --2- 172.21.15.154:0/148553209 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3a9006af60 0x7f3a9006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:07.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.585+0000 7f3ab15c2640 1 --2- 172.21.15.154:0/148553209 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3a9006af60 0x7f3a9006d420 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8035000 tx=0x7f3aa802f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:07.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.589+0000 7f3aa1ffb640 1 -- 172.21.15.154:0/148553209 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f3a9c054cc0 con 0x7f3aac0b2300 2023-09-23T03:24:07.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:07.779+0000 7f3ab25c4640 1 -- 172.21.15.154:0/148553209 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3aac0c53a0 con 0x7f3aac0b2300 2023-09-23T03:24:08.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.572+0000 7f3aa1ffb640 1 -- 172.21.15.154:0/148553209 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3a9c0582e0 con 0x7f3aac0b2300 2023-09-23T03:24:08.574 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_metadata' created 2023-09-23T03:24:08.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.576+0000 7f3ab25c4640 1 -- 172.21.15.154:0/148553209 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3a9006af60 msgr2=0x7f3a9006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:08.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.576+0000 7f3ab25c4640 1 --2- 172.21.15.154:0/148553209 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3a9006af60 0x7f3a9006d420 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8035000 tx=0x7f3aa802f000 comp rx=0 tx=0).stop 2023-09-23T03:24:08.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.576+0000 7f3ab25c4640 1 -- 172.21.15.154:0/148553209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3aac0b2300 msgr2=0x7f3aac0cdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:08.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.576+0000 7f3ab25c4640 1 --2- 172.21.15.154:0/148553209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3aac0b2300 0x7f3aac0cdc10 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c00d7d0 tx=0x7f3a9c00dc90 comp rx=0 tx=0).stop 2023-09-23T03:24:08.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.576+0000 7f3ab25c4640 1 -- 172.21.15.154:0/148553209 shutdown_connections 2023-09-23T03:24:08.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.577+0000 7f3ab25c4640 1 --2- 172.21.15.154:0/148553209 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3a9006af60 0x7f3a9006d420 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:08.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.577+0000 7f3ab25c4640 1 --2- 172.21.15.154:0/148553209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3aac15adf0 0x7f3aac1575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:08.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.577+0000 7f3ab25c4640 1 --2- 172.21.15.154:0/148553209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3aac0b2300 0x7f3aac0cdc10 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:08.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.577+0000 7f3ab25c4640 1 --2- 172.21.15.154:0/148553209 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3aac0afd70 0x7f3aac0c6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:08.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.577+0000 7f3ab25c4640 1 -- 172.21.15.154:0/148553209 >> 172.21.15.154:0/148553209 conn(0x7f3aac069cd0 msgr2=0x7f3aac06cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:08.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.577+0000 7f3ab25c4640 1 -- 172.21.15.154:0/148553209 shutdown_connections 2023-09-23T03:24:08.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.577+0000 7f3ab25c4640 1 -- 172.21.15.154:0/148553209 wait complete. 2023-09-23T03:24:08.603 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-09-23T03:24:08.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.811+0000 7fbaacea2640 1 Processor -- start 2023-09-23T03:24:08.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.812+0000 7fbaacea2640 1 -- start start 2023-09-23T03:24:08.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.813+0000 7fbaacea2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaa815ad00 0x7fbaa80cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:08.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.813+0000 7fbaacea2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbaa80cbc80 0x7fbaa80cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:08.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.813+0000 7fbaacea2640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaa80b21b0 con 0x7fbaa80b3960 2023-09-23T03:24:08.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.813+0000 7fbaacea2640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaa80b2470 con 0x7fbaa80cbc80 2023-09-23T03:24:08.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.813+0000 7fbaacea2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa80b2750 con 0x7fbaa815ad00 2023-09-23T03:24:08.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.813+0000 7fbaa6d9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbaa80cbc80 0x7fbaa80cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:08.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.813+0000 7fbaa5d9b640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fbaa80b3960 0x7fbaa80cb570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:54448/0 (socket says 172.21.15.154:54448) 2023-09-23T03:24:08.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.813+0000 7fbaa5d9b640 1 -- 172.21.15.154:0/1571697984 learned_addr learned my addr 172.21.15.154:0/1571697984 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:24:08.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.814+0000 7fbaa6d9d640 1 -- 172.21.15.154:0/1571697984 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaa815ad00 msgr2=0x7fbaa80cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:08.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.814+0000 7fbaa659c640 1 --2- 172.21.15.154:0/1571697984 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaa815ad00 0x7fbaa80cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:08.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.814+0000 7fbaa6d9d640 1 --2- 172.21.15.154:0/1571697984 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaa815ad00 0x7fbaa80cd0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:08.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.814+0000 7fbaa6d9d640 1 -- 172.21.15.154:0/1571697984 >> v1:172.21.15.154:6789/0 conn(0x7fbaa80b3960 legacy=0x7fbaa80cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:08.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.814+0000 7fbaa6d9d640 1 -- 172.21.15.154:0/1571697984 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa80b2a30 con 0x7fbaa80cbc80 2023-09-23T03:24:08.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.814+0000 7fbaa559a640 1 -- 172.21.15.154:0/1571697984 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 748253622 0 0) 0x7fbaa80b21b0 con 0x7fbaa80b3960 2023-09-23T03:24:08.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.815+0000 7fbaa6d9d640 1 --2- 172.21.15.154:0/1571697984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbaa80cbc80 0x7fbaa80cd620 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0009d90 tx=0x7fbaa002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83946084764a0650 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:08.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.815+0000 7fbaa559a640 1 -- 172.21.15.154:0/1571697984 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa002cc50 con 0x7fbaa80cbc80 2023-09-23T03:24:08.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.815+0000 7fbaa559a640 1 -- 172.21.15.154:0/1571697984 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaa002cdb0 con 0x7fbaa80cbc80 2023-09-23T03:24:08.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.815+0000 7fbaa559a640 1 -- 172.21.15.154:0/1571697984 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0034660 con 0x7fbaa80cbc80 2023-09-23T03:24:08.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.816+0000 7fbaacea2640 1 -- 172.21.15.154:0/1571697984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbaa80cbc80 msgr2=0x7fbaa80cd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:08.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.816+0000 7fbaacea2640 1 --2- 172.21.15.154:0/1571697984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbaa80cbc80 0x7fbaa80cd620 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0009d90 tx=0x7fbaa002bea0 comp rx=0 tx=0).stop 2023-09-23T03:24:08.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.816+0000 7fbaacea2640 1 -- 172.21.15.154:0/1571697984 shutdown_connections 2023-09-23T03:24:08.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.816+0000 7fbaacea2640 1 --2- 172.21.15.154:0/1571697984 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbaa80cbc80 0x7fbaa80cd620 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:08.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.816+0000 7fbaacea2640 1 --2- 172.21.15.154:0/1571697984 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaa815ad00 0x7fbaa80cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:08.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.816+0000 7fbaacea2640 1 -- 172.21.15.154:0/1571697984 >> 172.21.15.154:0/1571697984 conn(0x7fbaa806fc40 msgr2=0x7fbaa8070020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:08.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.816+0000 7fbaacea2640 1 -- 172.21.15.154:0/1571697984 shutdown_connections 2023-09-23T03:24:08.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.816+0000 7fbaacea2640 1 -- 172.21.15.154:0/1571697984 wait complete. 2023-09-23T03:24:08.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.818+0000 7fbaacea2640 1 Processor -- start 2023-09-23T03:24:08.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.818+0000 7fbaacea2640 1 -- start start 2023-09-23T03:24:08.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.819+0000 7fbaacea2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbaa80b3960 0x7fbaa80c9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:08.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.819+0000 7fbaacea2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaa80cbc80 0x7fbaa806c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:08.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.819+0000 7fbaacea2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbaa815ad00 0x7fbaa814ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:08.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.819+0000 7fbaacea2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa806ce70 con 0x7fbaa815ad00 2023-09-23T03:24:08.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.819+0000 7fbaacea2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa80c9c80 con 0x7fbaa80b3960 2023-09-23T03:24:08.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.819+0000 7fbaacea2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa80c9f60 con 0x7fbaa80cbc80 2023-09-23T03:24:08.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.819+0000 7fbaa6d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbaa815ad00 0x7fbaa814ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:08.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.819+0000 7fbaa6d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbaa815ad00 0x7fbaa814ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36366/0 (socket says 172.21.15.154:36366) 2023-09-23T03:24:08.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.819+0000 7fbaa6d9d640 1 -- 172.21.15.154:0/605473647 learned_addr learned my addr 172.21.15.154:0/605473647 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:08.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.819+0000 7fbaa659c640 1 --2- 172.21.15.154:0/605473647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbaa80b3960 0x7fbaa80c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:08.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.819+0000 7fbaa5d9b640 1 --2- 172.21.15.154:0/605473647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaa80cbc80 0x7fbaa806c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:08.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.819+0000 7fbaa6d9d640 1 -- 172.21.15.154:0/605473647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaa80cbc80 msgr2=0x7fbaa806c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:08.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.819+0000 7fbaa6d9d640 1 --2- 172.21.15.154:0/605473647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaa80cbc80 0x7fbaa806c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:08.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.819+0000 7fbaa6d9d640 1 -- 172.21.15.154:0/605473647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbaa80b3960 msgr2=0x7fbaa80c9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:08.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.819+0000 7fbaa6d9d640 1 --2- 172.21.15.154:0/605473647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbaa80b3960 0x7fbaa80c9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:08.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.819+0000 7fbaa6d9d640 1 -- 172.21.15.154:0/605473647 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa0009a70 con 0x7fbaa815ad00 2023-09-23T03:24:08.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.820+0000 7fbaa6d9d640 1 --2- 172.21.15.154:0/605473647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbaa815ad00 0x7fbaa814ab00 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0009d90 tx=0x7fbaa0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:08.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.820+0000 7fba9effd640 1 -- 172.21.15.154:0/605473647 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0003b90 con 0x7fbaa815ad00 2023-09-23T03:24:08.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.820+0000 7fba9effd640 1 -- 172.21.15.154:0/605473647 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaa0003cf0 con 0x7fbaa815ad00 2023-09-23T03:24:08.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.820+0000 7fba9effd640 1 -- 172.21.15.154:0/605473647 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0004270 con 0x7fbaa815ad00 2023-09-23T03:24:08.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.820+0000 7fbaacea2640 1 -- 172.21.15.154:0/605473647 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaa80ca240 con 0x7fbaa815ad00 2023-09-23T03:24:08.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.820+0000 7fbaacea2640 1 -- 172.21.15.154:0/605473647 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaa80c3dc0 con 0x7fbaa815ad00 2023-09-23T03:24:08.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.821+0000 7fbaacea2640 1 -- 172.21.15.154:0/605473647 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaa804e920 con 0x7fbaa815ad00 2023-09-23T03:24:08.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.822+0000 7fba9effd640 1 -- 172.21.15.154:0/605473647 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fbaa003b050 con 0x7fbaa815ad00 2023-09-23T03:24:08.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.823+0000 7fba9effd640 1 --2- 172.21.15.154:0/605473647 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fba8806af30 0x7fba8806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:08.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.823+0000 7fba9effd640 1 -- 172.21.15.154:0/605473647 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5547+0+0 (secure 0 0 0) 0x7fbaa00abc70 con 0x7fbaa815ad00 2023-09-23T03:24:08.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.824+0000 7fbaa659c640 1 --2- 172.21.15.154:0/605473647 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fba8806af30 0x7fba8806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:08.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.824+0000 7fbaa659c640 1 --2- 172.21.15.154:0/605473647 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fba8806af30 0x7fba8806d3f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fbaa80cda40 tx=0x7fba98008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:08.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:08.828+0000 7fba9effd640 1 -- 172.21.15.154:0/605473647 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fbaa0076d40 con 0x7fbaa815ad00 2023-09-23T03:24:09.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.003+0000 7fbaacea2640 1 -- 172.21.15.154:0/605473647 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fbaa80ce4b0 con 0x7fbaa815ad00 2023-09-23T03:24:09.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.587+0000 7fba9effd640 1 -- 172.21.15.154:0/605473647 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v81) v1 ==== 170+0+0 (secure 0 0 0) 0x7fbaa0039020 con 0x7fbaa815ad00 2023-09-23T03:24:09.589 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_data' created 2023-09-23T03:24:09.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.591+0000 7fbaacea2640 1 -- 172.21.15.154:0/605473647 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fba8806af30 msgr2=0x7fba8806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:09.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.591+0000 7fbaacea2640 1 --2- 172.21.15.154:0/605473647 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fba8806af30 0x7fba8806d3f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fbaa80cda40 tx=0x7fba98008040 comp rx=0 tx=0).stop 2023-09-23T03:24:09.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.591+0000 7fbaacea2640 1 -- 172.21.15.154:0/605473647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbaa815ad00 msgr2=0x7fbaa814ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:09.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.591+0000 7fbaacea2640 1 --2- 172.21.15.154:0/605473647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbaa815ad00 0x7fbaa814ab00 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0009d90 tx=0x7fbaa0004a40 comp rx=0 tx=0).stop 2023-09-23T03:24:09.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.591+0000 7fbaacea2640 1 -- 172.21.15.154:0/605473647 shutdown_connections 2023-09-23T03:24:09.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.591+0000 7fbaacea2640 1 --2- 172.21.15.154:0/605473647 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fba8806af30 0x7fba8806d3f0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:09.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.591+0000 7fbaacea2640 1 --2- 172.21.15.154:0/605473647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbaa815ad00 0x7fbaa814ab00 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:09.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.591+0000 7fbaacea2640 1 --2- 172.21.15.154:0/605473647 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbaa80cbc80 0x7fbaa806c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:09.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.591+0000 7fbaacea2640 1 --2- 172.21.15.154:0/605473647 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbaa80b3960 0x7fbaa80c9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:09.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.591+0000 7fbaacea2640 1 -- 172.21.15.154:0/605473647 >> 172.21.15.154:0/605473647 conn(0x7fbaa806fc40 msgr2=0x7fbaa80cb520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:09.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.592+0000 7fbaacea2640 1 -- 172.21.15.154:0/605473647 shutdown_connections 2023-09-23T03:24:09.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.593+0000 7fbaacea2640 1 -- 172.21.15.154:0/605473647 wait complete. 2023-09-23T03:24:09.628 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-09-23T03:24:09.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.812+0000 7f0e5f6a2640 1 Processor -- start 2023-09-23T03:24:09.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.812+0000 7f0e5f6a2640 1 -- start start 2023-09-23T03:24:09.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.814+0000 7f0e5f6a2640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0e5815bb30 0x7f0e580caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:09.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.814+0000 7f0e5f6a2640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e5815b6a0 con 0x7f0e5815aba0 2023-09-23T03:24:09.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.814+0000 7f0e5f6a2640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e5815f0b0 con 0x7f0e580cb3e0 2023-09-23T03:24:09.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.814+0000 7f0e5f6a2640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e580cb870 con 0x7f0e5815bb30 2023-09-23T03:24:09.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.814+0000 7f0e5d43e640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f0e5815aba0 0x7f0e5815e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:54456/0 (socket says 172.21.15.154:54456) 2023-09-23T03:24:09.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.814+0000 7f0e5d43e640 1 -- 172.21.15.154:0/2790523202 learned_addr learned my addr 172.21.15.154:0/2790523202 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:24:09.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.814+0000 7f0e4ffff640 1 -- 172.21.15.154:0/2790523202 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4263967912 0 0) 0x7f0e5815b6a0 con 0x7f0e5815aba0 2023-09-23T03:24:09.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.815+0000 7f0e4ffff640 1 -- 172.21.15.154:0/2790523202 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e44003520 con 0x7f0e5815aba0 2023-09-23T03:24:09.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.815+0000 7f0e5cc3d640 1 --2- 172.21.15.154:0/2790523202 >> v2:172.21.15.190:3300/0 conn(0x7f0e5815bb30 0x7f0e580caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:09.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.815+0000 7f0e4ffff640 1 -- 172.21.15.154:0/2790523202 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1070469757 0 0) 0x7f0e44003520 con 0x7f0e5815aba0 2023-09-23T03:24:09.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.815+0000 7f0e4ffff640 1 -- 172.21.15.154:0/2790523202 >> v2:172.21.15.190:3300/0 conn(0x7f0e5815bb30 msgr2=0x7f0e580caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:09.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.815+0000 7f0e4ffff640 1 --2- 172.21.15.154:0/2790523202 >> v2:172.21.15.190:3300/0 conn(0x7f0e5815bb30 0x7f0e580caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:09.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.815+0000 7f0e4ffff640 1 -- 172.21.15.154:0/2790523202 >> v1:172.21.15.190:6789/0 conn(0x7f0e580cb3e0 legacy=0x7f0e580cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:09.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.815+0000 7f0e4ffff640 1 -- 172.21.15.154:0/2790523202 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e5815f390 con 0x7f0e5815aba0 2023-09-23T03:24:09.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.815+0000 7f0e4ffff640 1 -- 172.21.15.154:0/2790523202 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f0e54001670 con 0x7f0e5815aba0 2023-09-23T03:24:09.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.815+0000 7f0e4ffff640 1 -- 172.21.15.154:0/2790523202 >> v1:172.21.15.154:6789/0 conn(0x7f0e5815aba0 legacy=0x7f0e5815e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:09.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.815+0000 7f0e4ffff640 1 --2- 172.21.15.154:0/2790523202 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e440039c0 0x7f0e44003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:09.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.815+0000 7f0e4ffff640 1 --2- 172.21.15.154:0/2790523202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e440047c0 0x7f0e44004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:09.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.815+0000 7f0e4ffff640 1 --2- 172.21.15.154:0/2790523202 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e440051d0 0x7f0e4400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:09.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.815+0000 7f0e4ffff640 1 -- 172.21.15.154:0/2790523202 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e580cb870 con 0x7f0e440051d0 2023-09-23T03:24:09.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.815+0000 7f0e4ffff640 1 -- 172.21.15.154:0/2790523202 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4400c100 con 0x7f0e440047c0 2023-09-23T03:24:09.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.815+0000 7f0e4ffff640 1 -- 172.21.15.154:0/2790523202 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4400c2c0 con 0x7f0e440039c0 2023-09-23T03:24:09.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.815+0000 7f0e5dc3f640 1 --2- 172.21.15.154:0/2790523202 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e440051d0 0x7f0e4400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:09.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.816+0000 7f0e5dc3f640 1 -- 172.21.15.154:0/2790523202 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e440039c0 msgr2=0x7f0e44003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:09.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.816+0000 7f0e5d43e640 1 --2- 172.21.15.154:0/2790523202 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e440039c0 0x7f0e44003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:09.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.816+0000 7f0e5dc3f640 1 --2- 172.21.15.154:0/2790523202 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e440039c0 0x7f0e44003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:09.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.816+0000 7f0e5cc3d640 1 --2- 172.21.15.154:0/2790523202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e440047c0 0x7f0e44004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:09.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.816+0000 7f0e5dc3f640 1 -- 172.21.15.154:0/2790523202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e440047c0 msgr2=0x7f0e44004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:09.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.816+0000 7f0e5dc3f640 1 --2- 172.21.15.154:0/2790523202 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e440047c0 0x7f0e44004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:09.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.816+0000 7f0e5dc3f640 1 -- 172.21.15.154:0/2790523202 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e44003520 con 0x7f0e440051d0 2023-09-23T03:24:09.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.816+0000 7f0e5d43e640 1 --2- 172.21.15.154:0/2790523202 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e440039c0 0x7f0e44003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:09.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.817+0000 7f0e5dc3f640 1 -- 172.21.15.154:0/2790523202 reap_dead start 2023-09-23T03:24:09.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.817+0000 7f0e5dc3f640 1 --2- 172.21.15.154:0/2790523202 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e440051d0 0x7f0e4400b980 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f0e50004480 tx=0x7f0e440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05cee7088e3401d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:09.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.817+0000 7f0e4ffff640 1 -- 172.21.15.154:0/2790523202 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e50006b50 con 0x7f0e440051d0 2023-09-23T03:24:09.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.817+0000 7f0e4ffff640 1 -- 172.21.15.154:0/2790523202 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e50006cd0 con 0x7f0e440051d0 2023-09-23T03:24:09.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.817+0000 7f0e4ffff640 1 -- 172.21.15.154:0/2790523202 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e50007260 con 0x7f0e440051d0 2023-09-23T03:24:09.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.818+0000 7f0e5f6a2640 1 -- 172.21.15.154:0/2790523202 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e440051d0 msgr2=0x7f0e4400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:09.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.818+0000 7f0e5f6a2640 1 --2- 172.21.15.154:0/2790523202 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e440051d0 0x7f0e4400b980 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f0e50004480 tx=0x7f0e440057f0 comp rx=0 tx=0).stop 2023-09-23T03:24:09.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.818+0000 7f0e5f6a2640 1 -- 172.21.15.154:0/2790523202 shutdown_connections 2023-09-23T03:24:09.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.818+0000 7f0e5f6a2640 1 --2- 172.21.15.154:0/2790523202 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e440051d0 0x7f0e4400b980 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:09.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.818+0000 7f0e5f6a2640 1 -- 172.21.15.154:0/2790523202 >> 172.21.15.154:0/2790523202 conn(0x7f0e5806c010 msgr2=0x7f0e5806c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:09.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.818+0000 7f0e5f6a2640 1 -- 172.21.15.154:0/2790523202 shutdown_connections 2023-09-23T03:24:09.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.818+0000 7f0e5f6a2640 1 -- 172.21.15.154:0/2790523202 wait complete. 2023-09-23T03:24:09.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.820+0000 7f0e5f6a2640 1 Processor -- start 2023-09-23T03:24:09.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.820+0000 7f0e5f6a2640 1 -- start start 2023-09-23T03:24:09.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.821+0000 7f0e5f6a2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e440051d0 0x7f0e58145830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:09.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.821+0000 7f0e5f6a2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e58070240 0x7f0e580706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:09.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.821+0000 7f0e5f6a2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e580c9b00 0x7f0e580c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:09.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.821+0000 7f0e5f6a2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e581f30c0 con 0x7f0e440051d0 2023-09-23T03:24:09.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.821+0000 7f0e5f6a2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e58070c00 con 0x7f0e580c9b00 2023-09-23T03:24:09.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.821+0000 7f0e5f6a2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e58145d70 con 0x7f0e58070240 2023-09-23T03:24:09.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.821+0000 7f0e5d43e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e440051d0 0x7f0e58145830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:09.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.821+0000 7f0e5d43e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e440051d0 0x7f0e58145830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36376/0 (socket says 172.21.15.154:36376) 2023-09-23T03:24:09.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.821+0000 7f0e5d43e640 1 -- 172.21.15.154:0/3878024668 learned_addr learned my addr 172.21.15.154:0/3878024668 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:09.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.821+0000 7f0e5d43e640 1 -- 172.21.15.154:0/3878024668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e58070240 msgr2=0x7f0e580706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:09.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.821+0000 7f0e5d43e640 1 --2- 172.21.15.154:0/3878024668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e58070240 0x7f0e580706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:09.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.821+0000 7f0e5d43e640 1 -- 172.21.15.154:0/3878024668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e580c9b00 msgr2=0x7f0e580c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:09.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.821+0000 7f0e5d43e640 1 --2- 172.21.15.154:0/3878024668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e580c9b00 0x7f0e580c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:09.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.821+0000 7f0e5d43e640 1 -- 172.21.15.154:0/3878024668 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e580ca4a0 con 0x7f0e440051d0 2023-09-23T03:24:09.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.822+0000 7f0e5d43e640 1 --2- 172.21.15.154:0/3878024668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e440051d0 0x7f0e58145830 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400d720 tx=0x7f0e5400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:09.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.822+0000 7f0e4dffb640 1 -- 172.21.15.154:0/3878024668 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e540074e0 con 0x7f0e440051d0 2023-09-23T03:24:09.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.822+0000 7f0e4dffb640 1 -- 172.21.15.154:0/3878024668 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e54007640 con 0x7f0e440051d0 2023-09-23T03:24:09.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.822+0000 7f0e5f6a2640 1 -- 172.21.15.154:0/3878024668 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e581f2180 con 0x7f0e440051d0 2023-09-23T03:24:09.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.822+0000 7f0e4dffb640 1 -- 172.21.15.154:0/3878024668 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e54004a00 con 0x7f0e440051d0 2023-09-23T03:24:09.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.822+0000 7f0e5f6a2640 1 -- 172.21.15.154:0/3878024668 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e581f2720 con 0x7f0e440051d0 2023-09-23T03:24:09.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.823+0000 7f0e5f6a2640 1 -- 172.21.15.154:0/3878024668 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e5804e920 con 0x7f0e440051d0 2023-09-23T03:24:09.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.824+0000 7f0e4dffb640 1 -- 172.21.15.154:0/3878024668 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f0e54004b60 con 0x7f0e440051d0 2023-09-23T03:24:09.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.825+0000 7f0e4dffb640 1 --2- 172.21.15.154:0/3878024668 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0e3c06af30 0x7f0e3c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:09.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.826+0000 7f0e4dffb640 1 -- 172.21.15.154:0/3878024668 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f0e540895b0 con 0x7f0e440051d0 2023-09-23T03:24:09.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.826+0000 7f0e5cc3d640 1 --2- 172.21.15.154:0/3878024668 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0e3c06af30 0x7f0e3c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:09.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.826+0000 7f0e5cc3d640 1 --2- 172.21.15.154:0/3878024668 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0e3c06af30 0x7f0e3c06d3f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0e48006a90 tx=0x7f0e48006a20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:09.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:09.831+0000 7f0e4dffb640 1 -- 172.21.15.154:0/3878024668 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f0e54054470 con 0x7f0e440051d0 2023-09-23T03:24:10.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.058+0000 7f0e5f6a2640 1 -- 172.21.15.154:0/3878024668 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f0e580cbc10 con 0x7f0e440051d0 2023-09-23T03:24:10.622 INFO:teuthology.orchestra.run.smithi154.stderr:new fs with metadata pool 11 and data pool 12 2023-09-23T03:24:10.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.621+0000 7f0e4dffb640 1 -- 172.21.15.154:0/3878024668 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v59) v1 ==== 174+0+0 (secure 0 0 0) 0x7f0e54057a90 con 0x7f0e440051d0 2023-09-23T03:24:10.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.635+0000 7f0e5f6a2640 1 -- 172.21.15.154:0/3878024668 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0e3c06af30 msgr2=0x7f0e3c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:10.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.635+0000 7f0e5f6a2640 1 --2- 172.21.15.154:0/3878024668 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0e3c06af30 0x7f0e3c06d3f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0e48006a90 tx=0x7f0e48006a20 comp rx=0 tx=0).stop 2023-09-23T03:24:10.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.635+0000 7f0e5f6a2640 1 -- 172.21.15.154:0/3878024668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e440051d0 msgr2=0x7f0e58145830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:10.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.635+0000 7f0e5f6a2640 1 --2- 172.21.15.154:0/3878024668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e440051d0 0x7f0e58145830 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400d720 tx=0x7f0e5400dbe0 comp rx=0 tx=0).stop 2023-09-23T03:24:10.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.636+0000 7f0e5f6a2640 1 -- 172.21.15.154:0/3878024668 shutdown_connections 2023-09-23T03:24:10.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.636+0000 7f0e5f6a2640 1 --2- 172.21.15.154:0/3878024668 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0e3c06af30 0x7f0e3c06d3f0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:10.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.636+0000 7f0e5f6a2640 1 --2- 172.21.15.154:0/3878024668 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0e580c9b00 0x7f0e580c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:10.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.636+0000 7f0e5f6a2640 1 --2- 172.21.15.154:0/3878024668 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0e58070240 0x7f0e580706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:10.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.636+0000 7f0e5f6a2640 1 --2- 172.21.15.154:0/3878024668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e440051d0 0x7f0e58145830 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:10.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.636+0000 7f0e5f6a2640 1 -- 172.21.15.154:0/3878024668 >> 172.21.15.154:0/3878024668 conn(0x7f0e5806c010 msgr2=0x7f0e58157cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:10.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.638+0000 7f0e5f6a2640 1 -- 172.21.15.154:0/3878024668 shutdown_connections 2023-09-23T03:24:10.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.638+0000 7f0e5f6a2640 1 -- 172.21.15.154:0/3878024668 wait complete. 2023-09-23T03:24:10.666 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:24:10.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.839+0000 7fe358b39640 1 Processor -- start 2023-09-23T03:24:10.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.839+0000 7fe358b39640 1 -- start start 2023-09-23T03:24:10.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.841+0000 7fe358b39640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3540b1b20 0x7fe3540ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:10.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.841+0000 7fe358b39640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe35415ac30 con 0x7fe3540cee70 2023-09-23T03:24:10.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.841+0000 7fe358b39640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3540b1f70 con 0x7fe3540acce0 2023-09-23T03:24:10.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.841+0000 7fe358b39640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3540b2e90 con 0x7fe3540b1b20 2023-09-23T03:24:10.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.841+0000 7fe352d9d640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fe3540cee70 0x7fe3540ab140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:54472/0 (socket says 172.21.15.154:54472) 2023-09-23T03:24:10.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.841+0000 7fe352d9d640 1 -- 172.21.15.154:0/4115587428 learned_addr learned my addr 172.21.15.154:0/4115587428 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:24:10.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.841+0000 7fe351d9b640 1 -- 172.21.15.154:0/4115587428 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1686311437 0 0) 0x7fe34c001490 con 0x7fe3540cee70 2023-09-23T03:24:10.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.841+0000 7fe34bfff640 1 --2- 172.21.15.154:0/4115587428 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3540b1b20 0x7fe3540ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:10.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.842+0000 7fe351d9b640 1 -- 172.21.15.154:0/4115587428 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe340003520 con 0x7fe3540cee70 2023-09-23T03:24:10.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.842+0000 7fe351d9b640 1 -- 172.21.15.154:0/4115587428 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3677103991 0 0) 0x7fe340003520 con 0x7fe3540cee70 2023-09-23T03:24:10.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.842+0000 7fe351d9b640 1 -- 172.21.15.154:0/4115587428 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3540b1b20 msgr2=0x7fe3540ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:10.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.842+0000 7fe351d9b640 1 --2- 172.21.15.154:0/4115587428 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3540b1b20 0x7fe3540ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:10.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.842+0000 7fe351d9b640 1 -- 172.21.15.154:0/4115587428 >> v1:172.21.15.190:6789/0 conn(0x7fe3540acce0 legacy=0x7fe35415ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:10.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.842+0000 7fe351d9b640 1 -- 172.21.15.154:0/4115587428 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3540b2210 con 0x7fe3540cee70 2023-09-23T03:24:10.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.842+0000 7fe351d9b640 1 -- 172.21.15.154:0/4115587428 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fe34c001b80 con 0x7fe3540cee70 2023-09-23T03:24:10.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.842+0000 7fe351d9b640 1 -- 172.21.15.154:0/4115587428 >> v1:172.21.15.154:6789/0 conn(0x7fe3540cee70 legacy=0x7fe3540ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:10.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.842+0000 7fe351d9b640 1 --2- 172.21.15.154:0/4115587428 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3540b1b20 0x7fe3400039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:10.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.842+0000 7fe351d9b640 1 --2- 172.21.15.154:0/4115587428 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe340003fd0 0x7fe3400047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:10.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.842+0000 7fe351d9b640 1 --2- 172.21.15.154:0/4115587428 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe340004d00 0x7fe340005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:10.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.842+0000 7fe351d9b640 1 -- 172.21.15.154:0/4115587428 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3540b2e90 con 0x7fe340003fd0 2023-09-23T03:24:10.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.842+0000 7fe351d9b640 1 -- 172.21.15.154:0/4115587428 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34000bbc0 con 0x7fe340004d00 2023-09-23T03:24:10.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.842+0000 7fe351d9b640 1 -- 172.21.15.154:0/4115587428 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe34000bd80 con 0x7fe3540b1b20 2023-09-23T03:24:10.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.842+0000 7fe34bfff640 1 --2- 172.21.15.154:0/4115587428 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe340003fd0 0x7fe3400047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:10.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.843+0000 7fe34bfff640 1 -- 172.21.15.154:0/4115587428 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3540b1b20 msgr2=0x7fe3400039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:10.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.843+0000 7fe35259c640 1 --2- 172.21.15.154:0/4115587428 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3540b1b20 0x7fe3400039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:10.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.843+0000 7fe34bfff640 1 --2- 172.21.15.154:0/4115587428 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3540b1b20 0x7fe3400039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:10.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.843+0000 7fe352d9d640 1 --2- 172.21.15.154:0/4115587428 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe340004d00 0x7fe340005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:10.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.843+0000 7fe34bfff640 1 -- 172.21.15.154:0/4115587428 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe340004d00 msgr2=0x7fe340005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:10.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.843+0000 7fe34bfff640 1 --2- 172.21.15.154:0/4115587428 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe340004d00 0x7fe340005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:10.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.843+0000 7fe34bfff640 1 -- 172.21.15.154:0/4115587428 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe340003520 con 0x7fe340003fd0 2023-09-23T03:24:10.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.843+0000 7fe35259c640 1 --2- 172.21.15.154:0/4115587428 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3540b1b20 0x7fe3400039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:10.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.844+0000 7fe34bfff640 1 --2- 172.21.15.154:0/4115587428 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe340003fd0 0x7fe3400047c0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fe33c00ba00 tx=0x7fe3400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=844d9e70f685f2eb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:10.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.844+0000 7fe351d9b640 1 -- 172.21.15.154:0/4115587428 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33c032620 con 0x7fe340003fd0 2023-09-23T03:24:10.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.844+0000 7fe351d9b640 1 -- 172.21.15.154:0/4115587428 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe33c032780 con 0x7fe340003fd0 2023-09-23T03:24:10.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.844+0000 7fe351d9b640 1 -- 172.21.15.154:0/4115587428 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33c032b10 con 0x7fe340003fd0 2023-09-23T03:24:10.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.845+0000 7fe358b39640 1 -- 172.21.15.154:0/4115587428 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe340003fd0 msgr2=0x7fe3400047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:10.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.845+0000 7fe358b39640 1 --2- 172.21.15.154:0/4115587428 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe340003fd0 0x7fe3400047c0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fe33c00ba00 tx=0x7fe3400057f0 comp rx=0 tx=0).stop 2023-09-23T03:24:10.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.845+0000 7fe352d9d640 1 -- 172.21.15.154:0/4115587428 reap_dead start 2023-09-23T03:24:10.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.845+0000 7fe358b39640 1 -- 172.21.15.154:0/4115587428 shutdown_connections 2023-09-23T03:24:10.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.845+0000 7fe358b39640 1 -- 172.21.15.154:0/4115587428 >> 172.21.15.154:0/4115587428 conn(0x7fe35406a0f0 msgr2=0x7fe35406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:10.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.845+0000 7fe358b39640 1 -- 172.21.15.154:0/4115587428 shutdown_connections 2023-09-23T03:24:10.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.845+0000 7fe358b39640 1 -- 172.21.15.154:0/4115587428 wait complete. 2023-09-23T03:24:10.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.847+0000 7fe358b39640 1 Processor -- start 2023-09-23T03:24:10.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.847+0000 7fe358b39640 1 -- start start 2023-09-23T03:24:10.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.848+0000 7fe358b39640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe340003fd0 0x7fe3540c2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:10.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.848+0000 7fe358b39640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3540c5830 0x7fe3540c3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:10.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.848+0000 7fe358b39640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe35415a230 0x7fe3540aea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:10.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.848+0000 7fe358b39640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3540cf320 con 0x7fe35415a230 2023-09-23T03:24:10.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.848+0000 7fe358b39640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3540c3e80 con 0x7fe340003fd0 2023-09-23T03:24:10.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.848+0000 7fe358b39640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35415b7a0 con 0x7fe3540c5830 2023-09-23T03:24:10.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.848+0000 7fe352d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe35415a230 0x7fe3540aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:10.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.848+0000 7fe352d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe35415a230 0x7fe3540aea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36386/0 (socket says 172.21.15.154:36386) 2023-09-23T03:24:10.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.848+0000 7fe352d9d640 1 -- 172.21.15.154:0/620520021 learned_addr learned my addr 172.21.15.154:0/620520021 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:10.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.848+0000 7fe352d9d640 1 -- 172.21.15.154:0/620520021 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3540c5830 msgr2=0x7fe3540c3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:10.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.848+0000 7fe34bfff640 1 --2- 172.21.15.154:0/620520021 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3540c5830 0x7fe3540c3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:10.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.848+0000 7fe35259c640 1 --2- 172.21.15.154:0/620520021 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe340003fd0 0x7fe3540c2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:10.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.848+0000 7fe352d9d640 1 --2- 172.21.15.154:0/620520021 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3540c5830 0x7fe3540c3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:10.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.848+0000 7fe352d9d640 1 -- 172.21.15.154:0/620520021 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe340003fd0 msgr2=0x7fe3540c2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:10.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.848+0000 7fe352d9d640 1 --2- 172.21.15.154:0/620520021 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe340003fd0 0x7fe3540c2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:10.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.848+0000 7fe352d9d640 1 -- 172.21.15.154:0/620520021 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3540c4070 con 0x7fe35415a230 2023-09-23T03:24:10.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.849+0000 7fe352d9d640 1 --2- 172.21.15.154:0/620520021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe35415a230 0x7fe3540aea80 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fe34c000f80 tx=0x7fe34c0021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:10.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.849+0000 7fe34affd640 1 -- 172.21.15.154:0/620520021 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c004d60 con 0x7fe35415a230 2023-09-23T03:24:10.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.849+0000 7fe34affd640 1 -- 172.21.15.154:0/620520021 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34c00bb10 con 0x7fe35415a230 2023-09-23T03:24:10.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.849+0000 7fe34affd640 1 -- 172.21.15.154:0/620520021 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c00bec0 con 0x7fe35415a230 2023-09-23T03:24:10.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.849+0000 7fe358b39640 1 -- 172.21.15.154:0/620520021 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe35415ae70 con 0x7fe35415a230 2023-09-23T03:24:10.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.849+0000 7fe358b39640 1 -- 172.21.15.154:0/620520021 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe35415b430 con 0x7fe35415a230 2023-09-23T03:24:10.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.850+0000 7fe358b39640 1 -- 172.21.15.154:0/620520021 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe35415b930 con 0x7fe35415a230 2023-09-23T03:24:10.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.851+0000 7fe34affd640 1 -- 172.21.15.154:0/620520021 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fe34c007070 con 0x7fe35415a230 2023-09-23T03:24:10.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.852+0000 7fe34affd640 1 --2- 172.21.15.154:0/620520021 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe33406af30 0x7fe33406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:10.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.852+0000 7fe34affd640 1 -- 172.21.15.154:0/620520021 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe34c089aa0 con 0x7fe35415a230 2023-09-23T03:24:10.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.852+0000 7fe35259c640 1 --2- 172.21.15.154:0/620520021 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe33406af30 0x7fe33406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:10.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.853+0000 7fe35259c640 1 --2- 172.21.15.154:0/620520021 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe33406af30 0x7fe33406d3f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe3540c38d0 tx=0x7fe344009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:10.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:10.857+0000 7fe34affd640 1 -- 172.21.15.154:0/620520021 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fe34c054a60 con 0x7fe35415a230 2023-09-23T03:24:11.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.030+0000 7fe358b39640 1 -- 172.21.15.154:0/620520021 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe35415ac30 con 0x7fe35415a230 2023-09-23T03:24:11.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.032+0000 7fe34affd640 1 -- 172.21.15.154:0/620520021 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+16826 (secure 0 0 0) 0x7fe35415ac30 con 0x7fe35415a230 2023-09-23T03:24:11.033 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:11.034 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":82,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:24:10.060649+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:24:07.834123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-09-23T03:24:09.058217+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:24:11.036 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.034+0000 7fe358b39640 1 -- 172.21.15.154:0/620520021 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe33406af30 msgr2=0x7fe33406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:11.036 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.034+0000 7fe358b39640 1 --2- 172.21.15.154:0/620520021 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe33406af30 0x7fe33406d3f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe3540c38d0 tx=0x7fe344009040 comp rx=0 tx=0).stop 2023-09-23T03:24:11.036 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.034+0000 7fe358b39640 1 -- 172.21.15.154:0/620520021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe35415a230 msgr2=0x7fe3540aea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:11.036 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.034+0000 7fe358b39640 1 --2- 172.21.15.154:0/620520021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe35415a230 0x7fe3540aea80 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fe34c000f80 tx=0x7fe34c0021f0 comp rx=0 tx=0).stop 2023-09-23T03:24:11.036 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.034+0000 7fe358b39640 1 -- 172.21.15.154:0/620520021 shutdown_connections 2023-09-23T03:24:11.036 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.034+0000 7fe358b39640 1 --2- 172.21.15.154:0/620520021 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe33406af30 0x7fe33406d3f0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:11.036 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.034+0000 7fe358b39640 1 --2- 172.21.15.154:0/620520021 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe35415a230 0x7fe3540aea80 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:11.037 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.034+0000 7fe358b39640 1 --2- 172.21.15.154:0/620520021 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3540c5830 0x7fe3540c3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:11.037 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.034+0000 7fe358b39640 1 --2- 172.21.15.154:0/620520021 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe340003fd0 0x7fe3540c2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:11.037 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.034+0000 7fe358b39640 1 -- 172.21.15.154:0/620520021 >> 172.21.15.154:0/620520021 conn(0x7fe35406a0f0 msgr2=0x7fe3540ab140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:11.037 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.035+0000 7fe358b39640 1 -- 172.21.15.154:0/620520021 shutdown_connections 2023-09-23T03:24:11.037 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.035+0000 7fe358b39640 1 -- 172.21.15.154:0/620520021 wait complete. 2023-09-23T03:24:11.053 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:24:11.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.259+0000 7f87dd9e7640 1 Processor -- start 2023-09-23T03:24:11.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.259+0000 7f87dd9e7640 1 -- start start 2023-09-23T03:24:11.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.261+0000 7f87dd9e7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87d80b2fe0 0x7f87d80ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:11.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.261+0000 7f87dd9e7640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87d806ceb0 con 0x7f87d80ada30 2023-09-23T03:24:11.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.261+0000 7f87dd9e7640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87d8159c90 con 0x7f87d80cf130 2023-09-23T03:24:11.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.261+0000 7f87dd9e7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d80ade10 con 0x7f87d80b2fe0 2023-09-23T03:24:11.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.261+0000 7f87d6ffd640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f87d80ada30 0x7f87d80cea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:54488/0 (socket says 172.21.15.154:54488) 2023-09-23T03:24:11.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.261+0000 7f87d6ffd640 1 -- 172.21.15.154:0/4173433965 learned_addr learned my addr 172.21.15.154:0/4173433965 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:24:11.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.261+0000 7f87d77fe640 1 --2- 172.21.15.154:0/4173433965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87d80b2fe0 0x7f87d80ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:11.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.262+0000 7f87d5ffb640 1 -- 172.21.15.154:0/4173433965 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2665125855 0 0) 0x7f87c8001290 con 0x7f87d80ada30 2023-09-23T03:24:11.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.262+0000 7f87d5ffb640 1 -- 172.21.15.154:0/4173433965 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87c0005730 con 0x7f87d80ada30 2023-09-23T03:24:11.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.262+0000 7f87d5ffb640 1 -- 172.21.15.154:0/4173433965 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1252534764 0 0) 0x7f87c0005730 con 0x7f87d80ada30 2023-09-23T03:24:11.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.262+0000 7f87d5ffb640 1 -- 172.21.15.154:0/4173433965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87d80b2fe0 msgr2=0x7f87d80ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:11.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.262+0000 7f87d5ffb640 1 --2- 172.21.15.154:0/4173433965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87d80b2fe0 0x7f87d80ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:11.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.262+0000 7f87d5ffb640 1 -- 172.21.15.154:0/4173433965 >> v1:172.21.15.190:6789/0 conn(0x7f87d80cf130 legacy=0x7f87d815d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:11.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.262+0000 7f87d5ffb640 1 -- 172.21.15.154:0/4173433965 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d806c110 con 0x7f87d80ada30 2023-09-23T03:24:11.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.262+0000 7f87d5ffb640 1 -- 172.21.15.154:0/4173433965 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f87c8001980 con 0x7f87d80ada30 2023-09-23T03:24:11.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.262+0000 7f87d5ffb640 1 -- 172.21.15.154:0/4173433965 >> v1:172.21.15.154:6789/0 conn(0x7f87d80ada30 legacy=0x7f87d80cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:11.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.262+0000 7f87d5ffb640 1 --2- 172.21.15.154:0/4173433965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87d80b2fe0 0x7f87c0005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:11.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.262+0000 7f87d5ffb640 1 --2- 172.21.15.154:0/4173433965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87c0006180 0x7f87c00069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:11.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.262+0000 7f87d5ffb640 1 --2- 172.21.15.154:0/4173433965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87c0006f10 0x7f87c0007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:11.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.262+0000 7f87d5ffb640 1 -- 172.21.15.154:0/4173433965 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d80ade10 con 0x7f87c0006180 2023-09-23T03:24:11.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.262+0000 7f87d5ffb640 1 -- 172.21.15.154:0/4173433965 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c000dcd0 con 0x7f87c0006f10 2023-09-23T03:24:11.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.262+0000 7f87d5ffb640 1 -- 172.21.15.154:0/4173433965 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c000de90 con 0x7f87d80b2fe0 2023-09-23T03:24:11.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.262+0000 7f87d67fc640 1 --2- 172.21.15.154:0/4173433965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87c0006180 0x7f87c00069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:11.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.263+0000 7f87d77fe640 1 --2- 172.21.15.154:0/4173433965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87c0006f10 0x7f87c0007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:11.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.263+0000 7f87d6ffd640 1 --2- 172.21.15.154:0/4173433965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87d80b2fe0 0x7f87c0005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:11.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.263+0000 7f87d67fc640 1 -- 172.21.15.154:0/4173433965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87d80b2fe0 msgr2=0x7f87c0005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:11.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.263+0000 7f87d67fc640 1 --2- 172.21.15.154:0/4173433965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87d80b2fe0 0x7f87c0005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:11.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.263+0000 7f87d67fc640 1 -- 172.21.15.154:0/4173433965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87c0006f10 msgr2=0x7f87c0007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:11.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.263+0000 7f87d67fc640 1 --2- 172.21.15.154:0/4173433965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87c0006f10 0x7f87c0007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:11.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.263+0000 7f87d67fc640 1 -- 172.21.15.154:0/4173433965 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c0005730 con 0x7f87c0006180 2023-09-23T03:24:11.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.264+0000 7f87d67fc640 1 --2- 172.21.15.154:0/4173433965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87c0006180 0x7f87c00069d0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f87d806e070 tx=0x7f87c0007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c57462ada24b557 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:11.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.264+0000 7f87d5ffb640 1 -- 172.21.15.154:0/4173433965 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc0052d0 con 0x7f87c0006180 2023-09-23T03:24:11.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.264+0000 7f87d5ffb640 1 -- 172.21.15.154:0/4173433965 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87cc02e050 con 0x7f87c0006180 2023-09-23T03:24:11.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.264+0000 7f87d5ffb640 1 -- 172.21.15.154:0/4173433965 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc003830 con 0x7f87c0006180 2023-09-23T03:24:11.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.265+0000 7f87dd9e7640 1 -- 172.21.15.154:0/4173433965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87c0006180 msgr2=0x7f87c00069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:11.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.265+0000 7f87dd9e7640 1 --2- 172.21.15.154:0/4173433965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87c0006180 0x7f87c00069d0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f87d806e070 tx=0x7f87c0007a00 comp rx=0 tx=0).stop 2023-09-23T03:24:11.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.265+0000 7f87d77fe640 1 -- 172.21.15.154:0/4173433965 reap_dead start 2023-09-23T03:24:11.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.265+0000 7f87dd9e7640 1 -- 172.21.15.154:0/4173433965 shutdown_connections 2023-09-23T03:24:11.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.265+0000 7f87dd9e7640 1 -- 172.21.15.154:0/4173433965 >> 172.21.15.154:0/4173433965 conn(0x7f87d806a0f0 msgr2=0x7f87d806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:11.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.265+0000 7f87dd9e7640 1 -- 172.21.15.154:0/4173433965 shutdown_connections 2023-09-23T03:24:11.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.265+0000 7f87dd9e7640 1 -- 172.21.15.154:0/4173433965 wait complete. 2023-09-23T03:24:11.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.267+0000 7f87dd9e7640 1 Processor -- start 2023-09-23T03:24:11.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.267+0000 7f87dd9e7640 1 -- start start 2023-09-23T03:24:11.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.268+0000 7f87dd9e7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87c0006180 0x7f87d8097b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:11.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.268+0000 7f87dd9e7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87d815b980 0x7f87d815be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:11.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.268+0000 7f87dd9e7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87d80af160 0x7f87d80af5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:11.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.268+0000 7f87dd9e7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d815c340 con 0x7f87d80af160 2023-09-23T03:24:11.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.268+0000 7f87dd9e7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d80aecd0 con 0x7f87d815b980 2023-09-23T03:24:11.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.268+0000 7f87dd9e7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d80b07c0 con 0x7f87c0006180 2023-09-23T03:24:11.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.268+0000 7f87d77fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87d80af160 0x7f87d80af5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:11.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.268+0000 7f87d77fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87d80af160 0x7f87d80af5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36406/0 (socket says 172.21.15.154:36406) 2023-09-23T03:24:11.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.268+0000 7f87d77fe640 1 -- 172.21.15.154:0/965165463 learned_addr learned my addr 172.21.15.154:0/965165463 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:11.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.268+0000 7f87d6ffd640 1 --2- 172.21.15.154:0/965165463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87c0006180 0x7f87d8097b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:11.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.268+0000 7f87d77fe640 1 -- 172.21.15.154:0/965165463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87c0006180 msgr2=0x7f87d8097b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:11.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.268+0000 7f87d77fe640 1 --2- 172.21.15.154:0/965165463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87c0006180 0x7f87d8097b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:11.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.268+0000 7f87d77fe640 1 -- 172.21.15.154:0/965165463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87d815b980 msgr2=0x7f87d815be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:11.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.268+0000 7f87d67fc640 1 --2- 172.21.15.154:0/965165463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87d815b980 0x7f87d815be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:11.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.268+0000 7f87d77fe640 1 --2- 172.21.15.154:0/965165463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87d815b980 0x7f87d815be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:11.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.268+0000 7f87d77fe640 1 -- 172.21.15.154:0/965165463 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d80b0aa0 con 0x7f87d80af160 2023-09-23T03:24:11.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.269+0000 7f87d77fe640 1 --2- 172.21.15.154:0/965165463 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87d80af160 0x7f87d80af5e0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f87d80ce410 tx=0x7f87d00115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:11.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.269+0000 7f87bffff640 1 -- 172.21.15.154:0/965165463 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d000e070 con 0x7f87d80af160 2023-09-23T03:24:11.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.269+0000 7f87bffff640 1 -- 172.21.15.154:0/965165463 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d0011e10 con 0x7f87d80af160 2023-09-23T03:24:11.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.269+0000 7f87bffff640 1 -- 172.21.15.154:0/965165463 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d0003990 con 0x7f87d80af160 2023-09-23T03:24:11.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.269+0000 7f87dd9e7640 1 -- 172.21.15.154:0/965165463 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87d80b0d80 con 0x7f87d80af160 2023-09-23T03:24:11.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.269+0000 7f87dd9e7640 1 -- 172.21.15.154:0/965165463 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87d815c560 con 0x7f87d80af160 2023-09-23T03:24:11.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.270+0000 7f87dd9e7640 1 -- 172.21.15.154:0/965165463 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87d8073eb0 con 0x7f87d80af160 2023-09-23T03:24:11.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.271+0000 7f87bffff640 1 -- 172.21.15.154:0/965165463 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f87d0003af0 con 0x7f87d80af160 2023-09-23T03:24:11.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.272+0000 7f87bffff640 1 --2- 172.21.15.154:0/965165463 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f87c406af60 0x7f87c406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:11.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.272+0000 7f87bffff640 1 -- 172.21.15.154:0/965165463 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f87d0089f10 con 0x7f87d80af160 2023-09-23T03:24:11.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.272+0000 7f87d6ffd640 1 --2- 172.21.15.154:0/965165463 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f87c406af60 0x7f87c406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:11.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.273+0000 7f87d6ffd640 1 --2- 172.21.15.154:0/965165463 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f87c406af60 0x7f87c406d420 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f87c8001510 tx=0x7f87c8000ff0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:11.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.276+0000 7f87bffff640 1 -- 172.21.15.154:0/965165463 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f87d0054e00 con 0x7f87d80af160 2023-09-23T03:24:11.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.456+0000 7f87dd9e7640 1 -- 172.21.15.154:0/965165463 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87d8159240 con 0x7f87d80af160 2023-09-23T03:24:11.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.459+0000 7f87bffff640 1 -- 172.21.15.154:0/965165463 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+16826 (secure 0 0 0) 0x7f87d0058420 con 0x7f87d80af160 2023-09-23T03:24:11.461 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:11.461 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":82,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:24:10.060649+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:24:07.834123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-09-23T03:24:09.058217+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:24:11.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.461+0000 7f87dd9e7640 1 -- 172.21.15.154:0/965165463 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f87c406af60 msgr2=0x7f87c406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:11.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.461+0000 7f87dd9e7640 1 --2- 172.21.15.154:0/965165463 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f87c406af60 0x7f87c406d420 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f87c8001510 tx=0x7f87c8000ff0 comp rx=0 tx=0).stop 2023-09-23T03:24:11.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.461+0000 7f87dd9e7640 1 -- 172.21.15.154:0/965165463 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87d80af160 msgr2=0x7f87d80af5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:11.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.461+0000 7f87dd9e7640 1 --2- 172.21.15.154:0/965165463 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87d80af160 0x7f87d80af5e0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f87d80ce410 tx=0x7f87d00115a0 comp rx=0 tx=0).stop 2023-09-23T03:24:11.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.462+0000 7f87dd9e7640 1 -- 172.21.15.154:0/965165463 shutdown_connections 2023-09-23T03:24:11.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.462+0000 7f87dd9e7640 1 --2- 172.21.15.154:0/965165463 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f87c406af60 0x7f87c406d420 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:11.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.462+0000 7f87dd9e7640 1 --2- 172.21.15.154:0/965165463 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87d80af160 0x7f87d80af5e0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:11.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.462+0000 7f87dd9e7640 1 --2- 172.21.15.154:0/965165463 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87d815b980 0x7f87d815be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:11.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.462+0000 7f87dd9e7640 1 --2- 172.21.15.154:0/965165463 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87c0006180 0x7f87d8097b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:11.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.462+0000 7f87dd9e7640 1 -- 172.21.15.154:0/965165463 >> 172.21.15.154:0/965165463 conn(0x7f87d806a0f0 msgr2=0x7f87d80ac1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:11.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.462+0000 7f87dd9e7640 1 -- 172.21.15.154:0/965165463 shutdown_connections 2023-09-23T03:24:11.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.462+0000 7f87dd9e7640 1 -- 172.21.15.154:0/965165463 wait complete. 2023-09-23T03:24:11.481 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-09-23T03:24:11.552 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:24:11.551+0000 7fe1a6776640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-23T03:24:11.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.705+0000 7fcd20ee1640 1 Processor -- start 2023-09-23T03:24:11.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.705+0000 7fcd20ee1640 1 -- start start 2023-09-23T03:24:11.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.706+0000 7fcd20ee1640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd1c15aba0 0x7fcd1c15bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:11.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.706+0000 7fcd20ee1640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcd1c0cb5b0 0x7fcd1c15ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:11.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.706+0000 7fcd20ee1640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd1c15b6a0 con 0x7fcd1c15e5f0 2023-09-23T03:24:11.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.706+0000 7fcd20ee1640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd1c15f0b0 con 0x7fcd1c0cb5b0 2023-09-23T03:24:11.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.706+0000 7fcd20ee1640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c15f390 con 0x7fcd1c15aba0 2023-09-23T03:24:11.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.707+0000 7fcd1ad9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcd1c0cb5b0 0x7fcd1c15ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:11.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.707+0000 7fcd1ad9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcd1c0cb5b0 0x7fcd1c15ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36410/0 (socket says 172.21.15.154:36410) 2023-09-23T03:24:11.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.707+0000 7fcd1ad9d640 1 -- 172.21.15.154:0/2032480420 learned_addr learned my addr 172.21.15.154:0/2032480420 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:11.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.707+0000 7fcd1a59c640 1 --2- 172.21.15.154:0/2032480420 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd1c15aba0 0x7fcd1c15bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:11.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.707+0000 7fcd1ad9d640 1 -- 172.21.15.154:0/2032480420 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd1c15aba0 msgr2=0x7fcd1c15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:11.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.707+0000 7fcd1ad9d640 1 --2- 172.21.15.154:0/2032480420 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd1c15aba0 0x7fcd1c15bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:11.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.707+0000 7fcd1ad9d640 1 -- 172.21.15.154:0/2032480420 >> v1:172.21.15.154:6789/0 conn(0x7fcd1c15e5f0 legacy=0x7fcd1c0caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:11.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.707+0000 7fcd1ad9d640 1 -- 172.21.15.154:0/2032480420 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1c15b330 con 0x7fcd1c0cb5b0 2023-09-23T03:24:11.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.708+0000 7fcd1ad9d640 1 --2- 172.21.15.154:0/2032480420 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd1c0cb5b0 0x7fcd1c15ea30 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fcd14009e50 tx=0x7fcd1402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e76bfd2f27dc313 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:11.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.709+0000 7fcd19d9b640 1 -- 172.21.15.154:0/2032480420 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1402cb00 con 0x7fcd1c0cb5b0 2023-09-23T03:24:11.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.709+0000 7fcd19d9b640 1 -- 172.21.15.154:0/2032480420 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1402cc60 con 0x7fcd1c0cb5b0 2023-09-23T03:24:11.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.709+0000 7fcd19d9b640 1 -- 172.21.15.154:0/2032480420 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd140343f0 con 0x7fcd1c0cb5b0 2023-09-23T03:24:11.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.709+0000 7fcd20ee1640 1 -- 172.21.15.154:0/2032480420 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd1c0cb5b0 msgr2=0x7fcd1c15ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:11.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.709+0000 7fcd20ee1640 1 --2- 172.21.15.154:0/2032480420 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd1c0cb5b0 0x7fcd1c15ea30 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fcd14009e50 tx=0x7fcd1402bf60 comp rx=0 tx=0).stop 2023-09-23T03:24:11.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.709+0000 7fcd20ee1640 1 -- 172.21.15.154:0/2032480420 shutdown_connections 2023-09-23T03:24:11.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.709+0000 7fcd20ee1640 1 --2- 172.21.15.154:0/2032480420 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd1c0cb5b0 0x7fcd1c15ea30 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:11.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.709+0000 7fcd20ee1640 1 --2- 172.21.15.154:0/2032480420 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd1c15aba0 0x7fcd1c15bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:11.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.709+0000 7fcd20ee1640 1 -- 172.21.15.154:0/2032480420 >> 172.21.15.154:0/2032480420 conn(0x7fcd1c06c010 msgr2=0x7fcd1c06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:11.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.709+0000 7fcd20ee1640 1 -- 172.21.15.154:0/2032480420 shutdown_connections 2023-09-23T03:24:11.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.709+0000 7fcd20ee1640 1 -- 172.21.15.154:0/2032480420 wait complete. 2023-09-23T03:24:11.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.711+0000 7fcd20ee1640 1 Processor -- start 2023-09-23T03:24:11.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.711+0000 7fcd20ee1640 1 -- start start 2023-09-23T03:24:11.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.712+0000 7fcd20ee1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd1c0cb5b0 0x7fcd1c0c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:11.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.712+0000 7fcd20ee1640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd1c15aba0 0x7fcd1c156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:11.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.712+0000 7fcd20ee1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd1c15e5f0 0x7fcd1c14eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:11.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.712+0000 7fcd20ee1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c150140 con 0x7fcd1c15e5f0 2023-09-23T03:24:11.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.712+0000 7fcd20ee1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c0c6c50 con 0x7fcd1c0cb5b0 2023-09-23T03:24:11.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.712+0000 7fcd20ee1640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c150390 con 0x7fcd1c15aba0 2023-09-23T03:24:11.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.712+0000 7fcd1ad9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd1c15e5f0 0x7fcd1c14eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:11.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.712+0000 7fcd1ad9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd1c15e5f0 0x7fcd1c14eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36418/0 (socket says 172.21.15.154:36418) 2023-09-23T03:24:11.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.712+0000 7fcd1ad9d640 1 -- 172.21.15.154:0/65974875 learned_addr learned my addr 172.21.15.154:0/65974875 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:11.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.713+0000 7fcd1a59c640 1 --2- 172.21.15.154:0/65974875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd1c0cb5b0 0x7fcd1c0c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:11.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.713+0000 7fcd13fff640 1 --2- 172.21.15.154:0/65974875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd1c15aba0 0x7fcd1c156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:11.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.713+0000 7fcd1ad9d640 1 -- 172.21.15.154:0/65974875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd1c15aba0 msgr2=0x7fcd1c156ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:11.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.713+0000 7fcd1ad9d640 1 --2- 172.21.15.154:0/65974875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd1c15aba0 0x7fcd1c156ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:11.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.713+0000 7fcd1ad9d640 1 -- 172.21.15.154:0/65974875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd1c0cb5b0 msgr2=0x7fcd1c0c9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:11.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.713+0000 7fcd1ad9d640 1 --2- 172.21.15.154:0/65974875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd1c0cb5b0 0x7fcd1c0c9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:11.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.713+0000 7fcd1ad9d640 1 -- 172.21.15.154:0/65974875 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd14009b30 con 0x7fcd1c15e5f0 2023-09-23T03:24:11.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.713+0000 7fcd1a59c640 1 --2- 172.21.15.154:0/65974875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd1c0cb5b0 0x7fcd1c0c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:11.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.713+0000 7fcd1ad9d640 1 --2- 172.21.15.154:0/65974875 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd1c15e5f0 0x7fcd1c14eba0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fcd14036040 tx=0x7fcd14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:11.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.713+0000 7fcd12ffd640 1 -- 172.21.15.154:0/65974875 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd14009510 con 0x7fcd1c15e5f0 2023-09-23T03:24:11.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.714+0000 7fcd12ffd640 1 -- 172.21.15.154:0/65974875 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1403e450 con 0x7fcd1c15e5f0 2023-09-23T03:24:11.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.714+0000 7fcd12ffd640 1 -- 172.21.15.154:0/65974875 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1403e7e0 con 0x7fcd1c15e5f0 2023-09-23T03:24:11.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.714+0000 7fcd20ee1640 1 -- 172.21.15.154:0/65974875 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd1c15b100 con 0x7fcd1c15e5f0 2023-09-23T03:24:11.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.714+0000 7fcd20ee1640 1 -- 172.21.15.154:0/65974875 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd1c150950 con 0x7fcd1c15e5f0 2023-09-23T03:24:11.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.715+0000 7fcd20ee1640 1 -- 172.21.15.154:0/65974875 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd1c04e920 con 0x7fcd1c15e5f0 2023-09-23T03:24:11.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.716+0000 7fcd12ffd640 1 -- 172.21.15.154:0/65974875 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fcd1403e940 con 0x7fcd1c15e5f0 2023-09-23T03:24:11.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.717+0000 7fcd12ffd640 1 --2- 172.21.15.154:0/65974875 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fccfc06af30 0x7fccfc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:11.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.717+0000 7fcd12ffd640 1 -- 172.21.15.154:0/65974875 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fcd140ac6f0 con 0x7fcd1c15e5f0 2023-09-23T03:24:11.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.717+0000 7fcd1a59c640 1 --2- 172.21.15.154:0/65974875 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fccfc06af30 0x7fccfc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:11.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.718+0000 7fcd1a59c640 1 --2- 172.21.15.154:0/65974875 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fccfc06af30 0x7fccfc06d3f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c006fd0 tx=0x7fcd0c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:11.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.721+0000 7fcd12ffd640 1 -- 172.21.15.154:0/65974875 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fcd140775e0 con 0x7fcd1c15e5f0 2023-09-23T03:24:11.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:11.948+0000 7fcd20ee1640 1 -- 172.21.15.154:0/65974875 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fcd1c165760 con 0x7fcd1c15e5f0 2023-09-23T03:24:12.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.607+0000 7fcd12ffd640 1 -- 172.21.15.154:0/65974875 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v61) v1 ==== 118+0+0 (secure 0 0 0) 0x7fcd140aae60 con 0x7fcd1c15e5f0 2023-09-23T03:24:12.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.611+0000 7fcd20ee1640 1 -- 172.21.15.154:0/65974875 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fccfc06af30 msgr2=0x7fccfc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:12.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.611+0000 7fcd20ee1640 1 --2- 172.21.15.154:0/65974875 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fccfc06af30 0x7fccfc06d3f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c006fd0 tx=0x7fcd0c008040 comp rx=0 tx=0).stop 2023-09-23T03:24:12.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.611+0000 7fcd20ee1640 1 -- 172.21.15.154:0/65974875 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd1c15e5f0 msgr2=0x7fcd1c14eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:12.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.611+0000 7fcd20ee1640 1 --2- 172.21.15.154:0/65974875 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd1c15e5f0 0x7fcd1c14eba0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fcd14036040 tx=0x7fcd14004a40 comp rx=0 tx=0).stop 2023-09-23T03:24:12.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.611+0000 7fcd20ee1640 1 -- 172.21.15.154:0/65974875 shutdown_connections 2023-09-23T03:24:12.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.611+0000 7fcd20ee1640 1 --2- 172.21.15.154:0/65974875 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fccfc06af30 0x7fccfc06d3f0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:12.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.611+0000 7fcd20ee1640 1 --2- 172.21.15.154:0/65974875 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcd1c15e5f0 0x7fcd1c14eba0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:12.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.611+0000 7fcd20ee1640 1 --2- 172.21.15.154:0/65974875 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcd1c15aba0 0x7fcd1c156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:12.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.611+0000 7fcd20ee1640 1 --2- 172.21.15.154:0/65974875 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcd1c0cb5b0 0x7fcd1c0c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:12.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.611+0000 7fcd20ee1640 1 -- 172.21.15.154:0/65974875 >> 172.21.15.154:0/65974875 conn(0x7fcd1c06c010 msgr2=0x7fcd1c15c130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:12.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.611+0000 7fcd20ee1640 1 -- 172.21.15.154:0/65974875 shutdown_connections 2023-09-23T03:24:12.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.611+0000 7fcd20ee1640 1 -- 172.21.15.154:0/65974875 wait complete. 2023-09-23T03:24:12.633 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-09-23T03:24:12.633 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-09-23T03:24:12.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.848+0000 7fa862db7640 1 Processor -- start 2023-09-23T03:24:12.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.848+0000 7fa862db7640 1 -- start start 2023-09-23T03:24:12.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.850+0000 7fa862db7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa85c0cb5b0 0x7fa85c0cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:12.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.850+0000 7fa862db7640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa85c15b6a0 con 0x7fa85c15bb30 2023-09-23T03:24:12.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.850+0000 7fa862db7640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa85c15f0b0 con 0x7fa85c15aba0 2023-09-23T03:24:12.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.850+0000 7fa862db7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c15bf70 con 0x7fa85c0cb5b0 2023-09-23T03:24:12.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.850+0000 7fa85bfff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fa85c15bb30 0x7fa85c0caea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:54496/0 (socket says 172.21.15.154:54496) 2023-09-23T03:24:12.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.850+0000 7fa85bfff640 1 -- 172.21.15.154:0/3724479279 learned_addr learned my addr 172.21.15.154:0/3724479279 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:24:12.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.850+0000 7fa861354640 1 --2- 172.21.15.154:0/3724479279 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa85c0cb5b0 0x7fa85c0cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:12.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.851+0000 7fa85b7fe640 1 -- 172.21.15.154:0/3724479279 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4118614459 0 0) 0x7fa85c15b6a0 con 0x7fa85c15bb30 2023-09-23T03:24:12.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.851+0000 7fa85b7fe640 1 -- 172.21.15.154:0/3724479279 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa84c003520 con 0x7fa85c15bb30 2023-09-23T03:24:12.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.851+0000 7fa85b7fe640 1 -- 172.21.15.154:0/3724479279 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3089839992 0 0) 0x7fa84c003520 con 0x7fa85c15bb30 2023-09-23T03:24:12.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.851+0000 7fa85b7fe640 1 -- 172.21.15.154:0/3724479279 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa85c0cb5b0 msgr2=0x7fa85c0cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:12.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.851+0000 7fa85b7fe640 1 --2- 172.21.15.154:0/3724479279 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa85c0cb5b0 0x7fa85c0cc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:12.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.851+0000 7fa85b7fe640 1 -- 172.21.15.154:0/3724479279 >> v1:172.21.15.190:6789/0 conn(0x7fa85c15aba0 legacy=0x7fa85c15e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:12.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.851+0000 7fa85b7fe640 1 -- 172.21.15.154:0/3724479279 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa85c15f390 con 0x7fa85c15bb30 2023-09-23T03:24:12.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.851+0000 7fa85b7fe640 1 -- 172.21.15.154:0/3724479279 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fa848001670 con 0x7fa85c15bb30 2023-09-23T03:24:12.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.851+0000 7fa85b7fe640 1 -- 172.21.15.154:0/3724479279 >> v1:172.21.15.154:6789/0 conn(0x7fa85c15bb30 legacy=0x7fa85c0caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:12.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.851+0000 7fa85b7fe640 1 --2- 172.21.15.154:0/3724479279 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa85c0cb5b0 0x7fa84c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:12.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.851+0000 7fa85b7fe640 1 --2- 172.21.15.154:0/3724479279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa84c003f20 0x7fa84c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:12.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.851+0000 7fa85b7fe640 1 --2- 172.21.15.154:0/3724479279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa84c004d00 0x7fa84c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:12.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.851+0000 7fa85b7fe640 1 -- 172.21.15.154:0/3724479279 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c15bf70 con 0x7fa84c004d00 2023-09-23T03:24:12.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.851+0000 7fa85b7fe640 1 -- 172.21.15.154:0/3724479279 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c00ba60 con 0x7fa84c003f20 2023-09-23T03:24:12.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.851+0000 7fa85b7fe640 1 -- 172.21.15.154:0/3724479279 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c00bc20 con 0x7fa85c0cb5b0 2023-09-23T03:24:12.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.852+0000 7fa861354640 1 --2- 172.21.15.154:0/3724479279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa84c004d00 0x7fa84c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:12.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.852+0000 7fa860b53640 1 --2- 172.21.15.154:0/3724479279 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa85c0cb5b0 0x7fa84c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:12.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.852+0000 7fa85bfff640 1 --2- 172.21.15.154:0/3724479279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa84c003f20 0x7fa84c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:12.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.852+0000 7fa861354640 1 -- 172.21.15.154:0/3724479279 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa85c0cb5b0 msgr2=0x7fa84c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:12.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.852+0000 7fa861354640 1 --2- 172.21.15.154:0/3724479279 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa85c0cb5b0 0x7fa84c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:12.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.852+0000 7fa861354640 1 -- 172.21.15.154:0/3724479279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa84c003f20 msgr2=0x7fa84c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:12.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.852+0000 7fa861354640 1 --2- 172.21.15.154:0/3724479279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa84c003f20 0x7fa84c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:12.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.852+0000 7fa861354640 1 -- 172.21.15.154:0/3724479279 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa84c003520 con 0x7fa84c004d00 2023-09-23T03:24:12.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.852+0000 7fa85bfff640 1 --2- 172.21.15.154:0/3724479279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa84c003f20 0x7fa84c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:12.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.853+0000 7fa861354640 1 --2- 172.21.15.154:0/3724479279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa84c004d00 0x7fa84c005180 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fa854009fc0 tx=0x7fa84c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b45cb31d928128d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:12.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.853+0000 7fa85b7fe640 1 -- 172.21.15.154:0/3724479279 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85402cc20 con 0x7fa84c004d00 2023-09-23T03:24:12.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.853+0000 7fa85b7fe640 1 -- 172.21.15.154:0/3724479279 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa85400f040 con 0x7fa84c004d00 2023-09-23T03:24:12.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.853+0000 7fa85b7fe640 1 -- 172.21.15.154:0/3724479279 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa854034440 con 0x7fa84c004d00 2023-09-23T03:24:12.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.854+0000 7fa862db7640 1 -- 172.21.15.154:0/3724479279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa84c004d00 msgr2=0x7fa84c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:12.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.854+0000 7fa862db7640 1 --2- 172.21.15.154:0/3724479279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa84c004d00 0x7fa84c005180 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fa854009fc0 tx=0x7fa84c0057f0 comp rx=0 tx=0).stop 2023-09-23T03:24:12.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.854+0000 7fa861354640 1 -- 172.21.15.154:0/3724479279 reap_dead start 2023-09-23T03:24:12.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.854+0000 7fa862db7640 1 -- 172.21.15.154:0/3724479279 shutdown_connections 2023-09-23T03:24:12.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.854+0000 7fa862db7640 1 -- 172.21.15.154:0/3724479279 >> 172.21.15.154:0/3724479279 conn(0x7fa85c06c010 msgr2=0x7fa85c06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:12.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.854+0000 7fa862db7640 1 -- 172.21.15.154:0/3724479279 shutdown_connections 2023-09-23T03:24:12.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.854+0000 7fa862db7640 1 -- 172.21.15.154:0/3724479279 wait complete. 2023-09-23T03:24:12.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.856+0000 7fa862db7640 1 Processor -- start 2023-09-23T03:24:12.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.856+0000 7fa862db7640 1 -- start start 2023-09-23T03:24:12.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.857+0000 7fa862db7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa84c004d00 0x7fa85c0c5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:12.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.857+0000 7fa862db7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa85c158ca0 0x7fa85c14d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:12.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.857+0000 7fa862db7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa85c0c67f0 0x7fa85c0c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:12.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.857+0000 7fa862db7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c0c71b0 con 0x7fa85c0c67f0 2023-09-23T03:24:12.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.857+0000 7fa862db7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c14d800 con 0x7fa85c158ca0 2023-09-23T03:24:12.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.857+0000 7fa862db7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c0c8a20 con 0x7fa84c004d00 2023-09-23T03:24:12.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.857+0000 7fa861354640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa85c0c67f0 0x7fa85c0c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:12.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.858+0000 7fa861354640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa85c0c67f0 0x7fa85c0c6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:36448/0 (socket says 172.21.15.154:36448) 2023-09-23T03:24:12.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.858+0000 7fa861354640 1 -- 172.21.15.154:0/2756166889 learned_addr learned my addr 172.21.15.154:0/2756166889 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:12.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.858+0000 7fa85bfff640 1 --2- 172.21.15.154:0/2756166889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa85c158ca0 0x7fa85c14d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:12.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.858+0000 7fa860b53640 1 --2- 172.21.15.154:0/2756166889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa84c004d00 0x7fa85c0c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:12.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.858+0000 7fa861354640 1 -- 172.21.15.154:0/2756166889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa84c004d00 msgr2=0x7fa85c0c5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:12.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.858+0000 7fa861354640 1 --2- 172.21.15.154:0/2756166889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa84c004d00 0x7fa85c0c5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:12.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.858+0000 7fa861354640 1 -- 172.21.15.154:0/2756166889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa85c158ca0 msgr2=0x7fa85c14d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:12.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.858+0000 7fa861354640 1 --2- 172.21.15.154:0/2756166889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa85c158ca0 0x7fa85c14d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:12.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.858+0000 7fa861354640 1 -- 172.21.15.154:0/2756166889 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa85c0cd280 con 0x7fa85c0c67f0 2023-09-23T03:24:12.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.858+0000 7fa861354640 1 --2- 172.21.15.154:0/2756166889 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa85c0c67f0 0x7fa85c0c6c70 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fa85402c690 tx=0x7fa854009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:12.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.859+0000 7fa8597fa640 1 -- 172.21.15.154:0/2756166889 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa854034cd0 con 0x7fa85c0c67f0 2023-09-23T03:24:12.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.859+0000 7fa8597fa640 1 -- 172.21.15.154:0/2756166889 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa854034e30 con 0x7fa85c0c67f0 2023-09-23T03:24:12.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.859+0000 7fa8597fa640 1 -- 172.21.15.154:0/2756166889 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa854003b90 con 0x7fa85c0c67f0 2023-09-23T03:24:12.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.859+0000 7fa862db7640 1 -- 172.21.15.154:0/2756166889 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa85c0c5c80 con 0x7fa85c0c67f0 2023-09-23T03:24:12.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.859+0000 7fa862db7640 1 -- 172.21.15.154:0/2756166889 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa85c0c6240 con 0x7fa85c0c67f0 2023-09-23T03:24:12.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.860+0000 7fa862db7640 1 -- 172.21.15.154:0/2756166889 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa85c0c8860 con 0x7fa85c0c67f0 2023-09-23T03:24:12.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.861+0000 7fa8597fa640 1 -- 172.21.15.154:0/2756166889 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fa85403e440 con 0x7fa85c0c67f0 2023-09-23T03:24:12.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.862+0000 7fa8597fa640 1 --2- 172.21.15.154:0/2756166889 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa84006af30 0x7fa84006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:12.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.862+0000 7fa8597fa640 1 -- 172.21.15.154:0/2756166889 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa8540ab5b0 con 0x7fa85c0c67f0 2023-09-23T03:24:12.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.862+0000 7fa860b53640 1 --2- 172.21.15.154:0/2756166889 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa84006af30 0x7fa84006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:12.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.863+0000 7fa860b53640 1 --2- 172.21.15.154:0/2756166889 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa84006af30 0x7fa84006d3f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa850007fd0 tx=0x7fa850009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:12.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:12.866+0000 7fa8597fa640 1 -- 172.21.15.154:0/2756166889 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fa85c0c8860 con 0x7fa85c0c67f0 2023-09-23T03:24:13.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.094+0000 7fa862db7640 1 -- 172.21.15.154:0/2756166889 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa85c0c8860 con 0x7fa85c0c67f0 2023-09-23T03:24:13.420 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:24:13.431 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:24:13.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.626+0000 7fa8597fa640 1 -- 172.21.15.154:0/2756166889 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v62) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa85c0c8860 con 0x7fa85c0c67f0 2023-09-23T03:24:13.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.630+0000 7fa862db7640 1 -- 172.21.15.154:0/2756166889 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa84006af30 msgr2=0x7fa84006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:13.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.630+0000 7fa862db7640 1 --2- 172.21.15.154:0/2756166889 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa84006af30 0x7fa84006d3f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa850007fd0 tx=0x7fa850009040 comp rx=0 tx=0).stop 2023-09-23T03:24:13.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.630+0000 7fa862db7640 1 -- 172.21.15.154:0/2756166889 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa85c0c67f0 msgr2=0x7fa85c0c6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:13.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.630+0000 7fa862db7640 1 --2- 172.21.15.154:0/2756166889 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa85c0c67f0 0x7fa85c0c6c70 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fa85402c690 tx=0x7fa854009750 comp rx=0 tx=0).stop 2023-09-23T03:24:13.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.630+0000 7fa862db7640 1 -- 172.21.15.154:0/2756166889 shutdown_connections 2023-09-23T03:24:13.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.630+0000 7fa862db7640 1 --2- 172.21.15.154:0/2756166889 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa84006af30 0x7fa84006d3f0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:13.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.630+0000 7fa862db7640 1 --2- 172.21.15.154:0/2756166889 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa85c0c67f0 0x7fa85c0c6c70 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:13.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.631+0000 7fa862db7640 1 --2- 172.21.15.154:0/2756166889 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa85c158ca0 0x7fa85c14d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:13.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.631+0000 7fa862db7640 1 --2- 172.21.15.154:0/2756166889 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa84c004d00 0x7fa85c0c5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:13.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.631+0000 7fa862db7640 1 -- 172.21.15.154:0/2756166889 >> 172.21.15.154:0/2756166889 conn(0x7fa85c06c010 msgr2=0x7fa85c15e360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:13.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.631+0000 7fa862db7640 1 -- 172.21.15.154:0/2756166889 shutdown_connections 2023-09-23T03:24:13.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.631+0000 7fa862db7640 1 -- 172.21.15.154:0/2756166889 wait complete. 2023-09-23T03:24:13.653 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-09-23T03:24:13.653 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:24:13.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.917+0000 7f3f2f554640 1 Processor -- start 2023-09-23T03:24:13.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.917+0000 7f3f2f554640 1 -- start start 2023-09-23T03:24:13.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.918+0000 7f3f2f554640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f280c4970 0x7f3f28071870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:13.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.918+0000 7f3f2f554640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3f2806c470 0x7f3f280ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:13.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.918+0000 7f3f2f554640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f28159a00 con 0x7f3f280ca260 2023-09-23T03:24:13.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.918+0000 7f3f2f554640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f280cbe90 con 0x7f3f2806c470 2023-09-23T03:24:13.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.918+0000 7f3f2f554640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f28071f70 con 0x7f3f280c4970 2023-09-23T03:24:13.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.919+0000 7f3f2ed53640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3f2806c470 0x7f3f280ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:13.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.919+0000 7f3f2ed53640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3f2806c470 0x7f3f280ccc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50772/0 (socket says 172.21.15.154:50772) 2023-09-23T03:24:13.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.919+0000 7f3f2dd51640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f3f280ca260 0x7f3f2806bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:59510/0 (socket says 172.21.15.154:59510) 2023-09-23T03:24:13.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.919+0000 7f3f2ed53640 1 -- 172.21.15.154:0/3056852827 learned_addr learned my addr 172.21.15.154:0/3056852827 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:13.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.919+0000 7f3f2e552640 1 --2- 172.21.15.154:0/3056852827 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f280c4970 0x7f3f28071870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:13.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.919+0000 7f3f2ed53640 1 -- 172.21.15.154:0/3056852827 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f280c4970 msgr2=0x7f3f28071870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:13.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.919+0000 7f3f2ed53640 1 --2- 172.21.15.154:0/3056852827 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f280c4970 0x7f3f28071870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:13.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.919+0000 7f3f2ed53640 1 -- 172.21.15.154:0/3056852827 >> v1:172.21.15.154:6789/0 conn(0x7f3f280ca260 legacy=0x7f3f2806bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:13.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.919+0000 7f3f2ed53640 1 -- 172.21.15.154:0/3056852827 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f280cc150 con 0x7f3f2806c470 2023-09-23T03:24:13.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.920+0000 7f3f2ed53640 1 --2- 172.21.15.154:0/3056852827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f2806c470 0x7f3f280ccc30 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f3f20009e50 tx=0x7f3f2002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe1db57ea8b37a2 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:13.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.921+0000 7f3f2d550640 1 -- 172.21.15.154:0/3056852827 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2002cb00 con 0x7f3f2806c470 2023-09-23T03:24:13.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.921+0000 7f3f2d550640 1 -- 172.21.15.154:0/3056852827 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f2002cc60 con 0x7f3f2806c470 2023-09-23T03:24:13.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.921+0000 7f3f2d550640 1 -- 172.21.15.154:0/3056852827 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f200343f0 con 0x7f3f2806c470 2023-09-23T03:24:13.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.921+0000 7f3f2f554640 1 -- 172.21.15.154:0/3056852827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f2806c470 msgr2=0x7f3f280ccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:13.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.921+0000 7f3f2f554640 1 --2- 172.21.15.154:0/3056852827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f2806c470 0x7f3f280ccc30 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f3f20009e50 tx=0x7f3f2002bf60 comp rx=0 tx=0).stop 2023-09-23T03:24:13.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.921+0000 7f3f2f554640 1 -- 172.21.15.154:0/3056852827 shutdown_connections 2023-09-23T03:24:13.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.921+0000 7f3f2f554640 1 --2- 172.21.15.154:0/3056852827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f2806c470 0x7f3f280ccc30 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:13.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.921+0000 7f3f2f554640 1 --2- 172.21.15.154:0/3056852827 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f280c4970 0x7f3f28071870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:13.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.921+0000 7f3f2f554640 1 -- 172.21.15.154:0/3056852827 >> 172.21.15.154:0/3056852827 conn(0x7f3f2806a390 msgr2=0x7f3f280afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:13.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.921+0000 7f3f2f554640 1 -- 172.21.15.154:0/3056852827 shutdown_connections 2023-09-23T03:24:13.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.921+0000 7f3f2f554640 1 -- 172.21.15.154:0/3056852827 wait complete. 2023-09-23T03:24:13.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.923+0000 7f3f2f554640 1 Processor -- start 2023-09-23T03:24:13.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.923+0000 7f3f2f554640 1 -- start start 2023-09-23T03:24:13.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.924+0000 7f3f2f554640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f2806c470 0x7f3f280be240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:13.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.924+0000 7f3f2f554640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f280c4970 0x7f3f280c6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:13.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.924+0000 7f3f2f554640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f280ca260 0x7f3f280aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:13.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.924+0000 7f3f2f554640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f280afb50 con 0x7f3f2806c470 2023-09-23T03:24:13.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.924+0000 7f3f2f554640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f280ac070 con 0x7f3f280c4970 2023-09-23T03:24:13.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.924+0000 7f3f2f554640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f280aba60 con 0x7f3f280ca260 2023-09-23T03:24:13.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.924+0000 7f3f2e552640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f2806c470 0x7f3f280be240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:13.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.924+0000 7f3f2e552640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f2806c470 0x7f3f280be240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50776/0 (socket says 172.21.15.154:50776) 2023-09-23T03:24:13.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.924+0000 7f3f2e552640 1 -- 172.21.15.154:0/4158547598 learned_addr learned my addr 172.21.15.154:0/4158547598 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:13.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.925+0000 7f3f2ed53640 1 --2- 172.21.15.154:0/4158547598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f280ca260 0x7f3f280aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:13.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.925+0000 7f3f2dd51640 1 --2- 172.21.15.154:0/4158547598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f280c4970 0x7f3f280c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:13.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.925+0000 7f3f2e552640 1 -- 172.21.15.154:0/4158547598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f280ca260 msgr2=0x7f3f280aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:13.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.925+0000 7f3f2e552640 1 --2- 172.21.15.154:0/4158547598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f280ca260 0x7f3f280aecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:13.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.925+0000 7f3f2e552640 1 -- 172.21.15.154:0/4158547598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f280c4970 msgr2=0x7f3f280c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:13.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.925+0000 7f3f2e552640 1 --2- 172.21.15.154:0/4158547598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f280c4970 0x7f3f280c6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:13.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.925+0000 7f3f2e552640 1 -- 172.21.15.154:0/4158547598 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f20009b30 con 0x7f3f2806c470 2023-09-23T03:24:13.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.925+0000 7f3f2e552640 1 --2- 172.21.15.154:0/4158547598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f2806c470 0x7f3f280be240 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f3f2400d800 tx=0x7f3f2400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:13.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.925+0000 7f3f1effd640 1 -- 172.21.15.154:0/4158547598 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2400b6c0 con 0x7f3f2806c470 2023-09-23T03:24:13.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.925+0000 7f3f1effd640 1 -- 172.21.15.154:0/4158547598 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f2400b820 con 0x7f3f2806c470 2023-09-23T03:24:13.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.925+0000 7f3f1effd640 1 -- 172.21.15.154:0/4158547598 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2400bbb0 con 0x7f3f2806c470 2023-09-23T03:24:13.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.925+0000 7f3f2f554640 1 -- 172.21.15.154:0/4158547598 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f280af870 con 0x7f3f2806c470 2023-09-23T03:24:13.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.926+0000 7f3f2f554640 1 -- 172.21.15.154:0/4158547598 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f280c0a60 con 0x7f3f2806c470 2023-09-23T03:24:13.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.927+0000 7f3f2f554640 1 -- 172.21.15.154:0/4158547598 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f280736b0 con 0x7f3f2806c470 2023-09-23T03:24:13.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.928+0000 7f3f1effd640 1 -- 172.21.15.154:0/4158547598 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f3f2400bd10 con 0x7f3f2806c470 2023-09-23T03:24:13.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.929+0000 7f3f1effd640 1 --2- 172.21.15.154:0/4158547598 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3f0c06af60 0x7f3f0c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:13.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.929+0000 7f3f1effd640 1 -- 172.21.15.154:0/4158547598 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f3f24089dd0 con 0x7f3f2806c470 2023-09-23T03:24:13.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.929+0000 7f3f2dd51640 1 --2- 172.21.15.154:0/4158547598 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3f0c06af60 0x7f3f0c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:13.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.930+0000 7f3f2dd51640 1 --2- 172.21.15.154:0/4158547598 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3f0c06af60 0x7f3f0c06d420 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3f18007fd0 tx=0x7f3f18009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:13.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:13.933+0000 7f3f1effd640 1 -- 172.21.15.154:0/4158547598 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f3f24054cc0 con 0x7f3f2806c470 2023-09-23T03:24:14.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.158+0000 7f3f2f554640 1 -- 172.21.15.154:0/4158547598 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f280ab8c0 con 0x7f3f2806c470 2023-09-23T03:24:14.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.160+0000 7f3f1effd640 1 -- 172.21.15.154:0/4158547598 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f3f280ab8c0 con 0x7f3f2806c470 2023-09-23T03:24:14.161 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:14.161 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":62,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/728936136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":728936136},{"type":"v1","addr":"172.21.15.190:6833","nonce":728936136}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":5890,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2277229238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2277229238},{"type":"v1","addr":"172.21.15.190:6835","nonce":2277229238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":7257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/4245928089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4245928089},{"type":"v1","addr":"172.21.15.154:6835","nonce":4245928089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:24:10.061474+0000","modified":"2023-09-23T03:24:13.096320+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7260":{"gid":7260,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6837/2213283923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2213283923},{"type":"v1","addr":"172.21.15.154:6837","nonce":2213283923}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-09-23T03:24:14.162 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 62 2023-09-23T03:24:14.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.162+0000 7f3f2f554640 1 -- 172.21.15.154:0/4158547598 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3f0c06af60 msgr2=0x7f3f0c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:14.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.162+0000 7f3f2f554640 1 --2- 172.21.15.154:0/4158547598 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3f0c06af60 0x7f3f0c06d420 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3f18007fd0 tx=0x7f3f18009040 comp rx=0 tx=0).stop 2023-09-23T03:24:14.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.162+0000 7f3f2f554640 1 -- 172.21.15.154:0/4158547598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f2806c470 msgr2=0x7f3f280be240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:14.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.162+0000 7f3f2f554640 1 --2- 172.21.15.154:0/4158547598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f2806c470 0x7f3f280be240 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f3f2400d800 tx=0x7f3f2400dcc0 comp rx=0 tx=0).stop 2023-09-23T03:24:14.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.163+0000 7f3f2f554640 1 -- 172.21.15.154:0/4158547598 shutdown_connections 2023-09-23T03:24:14.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.163+0000 7f3f2f554640 1 --2- 172.21.15.154:0/4158547598 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3f0c06af60 0x7f3f0c06d420 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:14.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.163+0000 7f3f2f554640 1 --2- 172.21.15.154:0/4158547598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f280ca260 0x7f3f280aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:14.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.163+0000 7f3f2f554640 1 --2- 172.21.15.154:0/4158547598 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f280c4970 0x7f3f280c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:14.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.163+0000 7f3f2f554640 1 --2- 172.21.15.154:0/4158547598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f2806c470 0x7f3f280be240 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:14.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.163+0000 7f3f2f554640 1 -- 172.21.15.154:0/4158547598 >> 172.21.15.154:0/4158547598 conn(0x7f3f2806a390 msgr2=0x7f3f28071790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:14.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.163+0000 7f3f2f554640 1 -- 172.21.15.154:0/4158547598 shutdown_connections 2023-09-23T03:24:14.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.163+0000 7f3f2f554640 1 -- 172.21.15.154:0/4158547598 wait complete. 2023-09-23T03:24:14.183 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:24:14.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.401+0000 7feac592c640 1 Processor -- start 2023-09-23T03:24:14.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.401+0000 7feac592c640 1 -- start start 2023-09-23T03:24:14.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.402+0000 7feac592c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7feac00ada30 0x7feac00ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:14.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.402+0000 7feac592c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feac00cf130 0x7feac015d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:14.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.402+0000 7feac592c640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feac006ceb0 con 0x7feac00b2fe0 2023-09-23T03:24:14.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.402+0000 7feac592c640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feac00ade10 con 0x7feac00ada30 2023-09-23T03:24:14.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.402+0000 7feac592c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac0159c90 con 0x7feac00cf130 2023-09-23T03:24:14.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.403+0000 7feabf7fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feac00cf130 0x7feac015d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:14.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.403+0000 7feabf7fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feac00cf130 0x7feac015d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:54678/0 (socket says 172.21.15.154:54678) 2023-09-23T03:24:14.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.403+0000 7feabe7fc640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7feac00b2fe0 0x7feac00cea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.154:38886/0 (socket says 172.21.15.154:38886) 2023-09-23T03:24:14.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.403+0000 7feabf7fe640 1 -- 172.21.15.154:0/1259578706 learned_addr learned my addr 172.21.15.154:0/1259578706 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:14.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.403+0000 7feabeffd640 1 --2- 172.21.15.154:0/1259578706 >> v2:172.21.15.190:3300/0 conn(0x7feac00ada30 0x7feac00ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:14.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.404+0000 7feabf7fe640 1 -- 172.21.15.154:0/1259578706 >> v2:172.21.15.190:3300/0 conn(0x7feac00ada30 msgr2=0x7feac00ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:14.406 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.404+0000 7feabf7fe640 1 --2- 172.21.15.154:0/1259578706 >> v2:172.21.15.190:3300/0 conn(0x7feac00ada30 0x7feac00ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:14.406 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.404+0000 7feabf7fe640 1 -- 172.21.15.154:0/1259578706 >> v1:172.21.15.190:6789/0 conn(0x7feac00b2fe0 legacy=0x7feac00cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:14.406 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.404+0000 7feabf7fe640 1 -- 172.21.15.154:0/1259578706 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac006c110 con 0x7feac00cf130 2023-09-23T03:24:14.406 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.404+0000 7feabdffb640 1 -- 172.21.15.154:0/1259578706 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 419859347 0 0) 0x7feab0001490 con 0x7feac00b2fe0 2023-09-23T03:24:14.406 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.405+0000 7feabf7fe640 1 --2- 172.21.15.154:0/1259578706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feac00cf130 0x7feac015d380 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7feab8009870 tx=0x7feab802db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2947d6c3b1ddd4c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:14.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.406+0000 7feabdffb640 1 -- 172.21.15.154:0/1259578706 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab802e940 con 0x7feac00cf130 2023-09-23T03:24:14.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.406+0000 7feabdffb640 1 -- 172.21.15.154:0/1259578706 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab802eaa0 con 0x7feac00cf130 2023-09-23T03:24:14.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.406+0000 7feabdffb640 1 -- 172.21.15.154:0/1259578706 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab802ee30 con 0x7feac00cf130 2023-09-23T03:24:14.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.406+0000 7feac592c640 1 -- 172.21.15.154:0/1259578706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feac00cf130 msgr2=0x7feac015d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:14.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.406+0000 7feac592c640 1 --2- 172.21.15.154:0/1259578706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feac00cf130 0x7feac015d380 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7feab8009870 tx=0x7feab802db90 comp rx=0 tx=0).stop 2023-09-23T03:24:14.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.406+0000 7feac592c640 1 -- 172.21.15.154:0/1259578706 shutdown_connections 2023-09-23T03:24:14.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.406+0000 7feac592c640 1 --2- 172.21.15.154:0/1259578706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feac00cf130 0x7feac015d380 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:14.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.406+0000 7feac592c640 1 --2- 172.21.15.154:0/1259578706 >> v2:172.21.15.190:3300/0 conn(0x7feac00ada30 0x7feac00ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:14.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.406+0000 7feac592c640 1 -- 172.21.15.154:0/1259578706 >> 172.21.15.154:0/1259578706 conn(0x7feac006a0f0 msgr2=0x7feac006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:14.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.406+0000 7feac592c640 1 -- 172.21.15.154:0/1259578706 shutdown_connections 2023-09-23T03:24:14.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.407+0000 7feac592c640 1 -- 172.21.15.154:0/1259578706 wait complete. 2023-09-23T03:24:14.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.408+0000 7feac592c640 1 Processor -- start 2023-09-23T03:24:14.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.408+0000 7feac592c640 1 -- start start 2023-09-23T03:24:14.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.409+0000 7feac592c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feac00ada30 0x7feac0099940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:14.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.409+0000 7feac592c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feac00b2fe0 0x7feac01f16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:14.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.409+0000 7feac592c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feac00cf130 0x7feac00c46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:14.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.409+0000 7feac592c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac015cd00 con 0x7feac00cf130 2023-09-23T03:24:14.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.409+0000 7feac592c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac00c2c70 con 0x7feac00ada30 2023-09-23T03:24:14.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.409+0000 7feac592c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac00c2060 con 0x7feac00b2fe0 2023-09-23T03:24:14.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.410+0000 7feabf7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feac00cf130 0x7feac00c46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:14.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.410+0000 7feabf7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feac00cf130 0x7feac00c46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50778/0 (socket says 172.21.15.154:50778) 2023-09-23T03:24:14.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.410+0000 7feabf7fe640 1 -- 172.21.15.154:0/3099895015 learned_addr learned my addr 172.21.15.154:0/3099895015 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:14.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.410+0000 7feabeffd640 1 --2- 172.21.15.154:0/3099895015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feac00ada30 0x7feac0099940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:14.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.410+0000 7feabe7fc640 1 --2- 172.21.15.154:0/3099895015 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feac00b2fe0 0x7feac01f16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:14.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.410+0000 7feabf7fe640 1 -- 172.21.15.154:0/3099895015 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feac00b2fe0 msgr2=0x7feac01f16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:14.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.410+0000 7feabf7fe640 1 --2- 172.21.15.154:0/3099895015 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feac00b2fe0 0x7feac01f16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:14.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.410+0000 7feabf7fe640 1 -- 172.21.15.154:0/3099895015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feac00ada30 msgr2=0x7feac0099940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:14.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.410+0000 7feabf7fe640 1 --2- 172.21.15.154:0/3099895015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feac00ada30 0x7feac0099940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:14.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.410+0000 7feabf7fe640 1 -- 172.21.15.154:0/3099895015 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab80094d0 con 0x7feac00cf130 2023-09-23T03:24:14.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.410+0000 7feabeffd640 1 --2- 172.21.15.154:0/3099895015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feac00ada30 0x7feac0099940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:24:14.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.410+0000 7feabf7fe640 1 --2- 172.21.15.154:0/3099895015 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feac00cf130 0x7feac00c46f0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7feab802dbd0 tx=0x7feab80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:14.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.411+0000 7feab7fff640 1 -- 172.21.15.154:0/3099895015 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab8041680 con 0x7feac00cf130 2023-09-23T03:24:14.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.411+0000 7feab7fff640 1 -- 172.21.15.154:0/3099895015 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab8041800 con 0x7feac00cf130 2023-09-23T03:24:14.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.411+0000 7feab7fff640 1 -- 172.21.15.154:0/3099895015 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab8041bb0 con 0x7feac00cf130 2023-09-23T03:24:14.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.411+0000 7feac592c640 1 -- 172.21.15.154:0/3099895015 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feac00c2340 con 0x7feac00cf130 2023-09-23T03:24:14.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.411+0000 7feac592c640 1 -- 172.21.15.154:0/3099895015 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac00c2900 con 0x7feac00cf130 2023-09-23T03:24:14.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.412+0000 7feac592c640 1 -- 172.21.15.154:0/3099895015 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feac00c2e00 con 0x7feac00cf130 2023-09-23T03:24:14.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.413+0000 7feab7fff640 1 -- 172.21.15.154:0/3099895015 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7feab8041d10 con 0x7feac00cf130 2023-09-23T03:24:14.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.414+0000 7feab7fff640 1 --2- 172.21.15.154:0/3099895015 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7feaa006af60 0x7feaa006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:14.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.414+0000 7feab7fff640 1 -- 172.21.15.154:0/3099895015 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7feab80ac700 con 0x7feac00cf130 2023-09-23T03:24:14.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.414+0000 7feabeffd640 1 --2- 172.21.15.154:0/3099895015 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7feaa006af60 0x7feaa006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:14.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.415+0000 7feabeffd640 1 --2- 172.21.15.154:0/3099895015 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7feaa006af60 0x7feaa006d420 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7feaa8009b80 tx=0x7feaa8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:14.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.419+0000 7feab7fff640 1 -- 172.21.15.154:0/3099895015 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7feab80777f0 con 0x7feac00cf130 2023-09-23T03:24:14.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.609+0000 7feac592c640 1 -- 172.21.15.154:0/3099895015 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feac0073eb0 con 0x7feac00cf130 2023-09-23T03:24:14.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.612+0000 7feab7fff640 1 -- 172.21.15.154:0/3099895015 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16813 (secure 0 0 0) 0x7feab807ae10 con 0x7feac00cf130 2023-09-23T03:24:14.613 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:14.614 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":83,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:24:11.592387+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:24:07.834123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-09-23T03:24:09.058217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:24:14.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.614+0000 7feac592c640 1 -- 172.21.15.154:0/3099895015 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7feaa006af60 msgr2=0x7feaa006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:14.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.614+0000 7feac592c640 1 --2- 172.21.15.154:0/3099895015 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7feaa006af60 0x7feaa006d420 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7feaa8009b80 tx=0x7feaa8008040 comp rx=0 tx=0).stop 2023-09-23T03:24:14.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.614+0000 7feac592c640 1 -- 172.21.15.154:0/3099895015 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feac00cf130 msgr2=0x7feac00c46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:14.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.614+0000 7feac592c640 1 --2- 172.21.15.154:0/3099895015 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feac00cf130 0x7feac00c46f0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7feab802dbd0 tx=0x7feab80046c0 comp rx=0 tx=0).stop 2023-09-23T03:24:14.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.614+0000 7feac592c640 1 -- 172.21.15.154:0/3099895015 shutdown_connections 2023-09-23T03:24:14.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.614+0000 7feac592c640 1 --2- 172.21.15.154:0/3099895015 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7feaa006af60 0x7feaa006d420 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:14.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.614+0000 7feac592c640 1 --2- 172.21.15.154:0/3099895015 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feac00cf130 0x7feac00c46f0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:14.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.614+0000 7feac592c640 1 --2- 172.21.15.154:0/3099895015 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feac00b2fe0 0x7feac01f16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:14.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.614+0000 7feac592c640 1 --2- 172.21.15.154:0/3099895015 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feac00ada30 0x7feac0099940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:14.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.614+0000 7feac592c640 1 -- 172.21.15.154:0/3099895015 >> 172.21.15.154:0/3099895015 conn(0x7feac006a0f0 msgr2=0x7feac00af000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:14.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.615+0000 7feac592c640 1 -- 172.21.15.154:0/3099895015 shutdown_connections 2023-09-23T03:24:14.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.615+0000 7feac592c640 1 -- 172.21.15.154:0/3099895015 wait complete. 2023-09-23T03:24:14.638 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-09-23T03:24:14.638 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-23T03:24:14.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.846+0000 7ff21ed5e640 1 Processor -- start 2023-09-23T03:24:14.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.846+0000 7ff21ed5e640 1 -- start start 2023-09-23T03:24:14.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.848+0000 7ff21ed5e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff21815cf40 0x7ff2181615b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:14.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.848+0000 7ff21ed5e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff21806b1e0 0x7ff2180cca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:14.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.848+0000 7ff21ed5e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2180ccf80 con 0x7ff2180b21d0 2023-09-23T03:24:14.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.848+0000 7ff21ed5e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2180cd1c0 con 0x7ff21815cf40 2023-09-23T03:24:14.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.848+0000 7ff21ed5e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2180cd4a0 con 0x7ff21806b1e0 2023-09-23T03:24:14.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.848+0000 7ff21cafa640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff21815cf40 0x7ff2181615b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:14.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.848+0000 7ff21cafa640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff21815cf40 0x7ff2181615b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50794/0 (socket says 172.21.15.154:50794) 2023-09-23T03:24:14.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.848+0000 7ff21cafa640 1 -- 172.21.15.154:0/2224228967 learned_addr learned my addr 172.21.15.154:0/2224228967 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:14.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.848+0000 7ff21d2fb640 1 --1- 172.21.15.154:0/2224228967 >> v1:172.21.15.154:6789/0 conn(0x7ff2180b21d0 0x7ff2180cb7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:59516/0 (socket says 172.21.15.154:59516) 2023-09-23T03:24:14.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.849+0000 7ff20ffff640 1 --2- 172.21.15.154:0/2224228967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff21806b1e0 0x7ff2180cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:14.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.849+0000 7ff21cafa640 1 -- 172.21.15.154:0/2224228967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff21806b1e0 msgr2=0x7ff2180cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:14.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.849+0000 7ff21cafa640 1 --2- 172.21.15.154:0/2224228967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff21806b1e0 0x7ff2180cca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:14.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.849+0000 7ff21cafa640 1 -- 172.21.15.154:0/2224228967 >> v1:172.21.15.154:6789/0 conn(0x7ff2180b21d0 legacy=0x7ff2180cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:14.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.849+0000 7ff20f7fe640 1 -- 172.21.15.154:0/2224228967 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1577087335 0 0) 0x7ff210001490 con 0x7ff2180b21d0 2023-09-23T03:24:14.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.849+0000 7ff21cafa640 1 -- 172.21.15.154:0/2224228967 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2180cd780 con 0x7ff21815cf40 2023-09-23T03:24:14.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.849+0000 7ff20ffff640 1 --2- 172.21.15.154:0/2224228967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff21806b1e0 0x7ff2180cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:14.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.850+0000 7ff21cafa640 1 --2- 172.21.15.154:0/2224228967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff21815cf40 0x7ff2181615b0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7ff214009870 tx=0x7ff21402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b4778df7583acbd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:14.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.850+0000 7ff20f7fe640 1 -- 172.21.15.154:0/2224228967 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff214030840 con 0x7ff21815cf40 2023-09-23T03:24:14.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.850+0000 7ff20f7fe640 1 -- 172.21.15.154:0/2224228967 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2140309a0 con 0x7ff21815cf40 2023-09-23T03:24:14.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.850+0000 7ff20f7fe640 1 -- 172.21.15.154:0/2224228967 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff214030d30 con 0x7ff21815cf40 2023-09-23T03:24:14.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.851+0000 7ff21ed5e640 1 -- 172.21.15.154:0/2224228967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff21815cf40 msgr2=0x7ff2181615b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:14.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.851+0000 7ff21ed5e640 1 --2- 172.21.15.154:0/2224228967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff21815cf40 0x7ff2181615b0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7ff214009870 tx=0x7ff21402b980 comp rx=0 tx=0).stop 2023-09-23T03:24:14.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.851+0000 7ff21ed5e640 1 -- 172.21.15.154:0/2224228967 shutdown_connections 2023-09-23T03:24:14.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.851+0000 7ff21ed5e640 1 --2- 172.21.15.154:0/2224228967 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff21806b1e0 0x7ff2180cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:14.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.851+0000 7ff21ed5e640 1 --2- 172.21.15.154:0/2224228967 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff21815cf40 0x7ff2181615b0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:14.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.851+0000 7ff21ed5e640 1 -- 172.21.15.154:0/2224228967 >> 172.21.15.154:0/2224228967 conn(0x7ff21806fc60 msgr2=0x7ff218070040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:14.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.851+0000 7ff21ed5e640 1 -- 172.21.15.154:0/2224228967 shutdown_connections 2023-09-23T03:24:14.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.851+0000 7ff21ed5e640 1 -- 172.21.15.154:0/2224228967 wait complete. 2023-09-23T03:24:14.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.853+0000 7ff21ed5e640 1 Processor -- start 2023-09-23T03:24:14.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.853+0000 7ff21ed5e640 1 -- start start 2023-09-23T03:24:14.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.854+0000 7ff21ed5e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff21806b1e0 0x7ff2180c9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:14.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.854+0000 7ff21ed5e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2180b21d0 0x7ff218156c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:14.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.854+0000 7ff21ed5e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff21815cf40 0x7ff2180c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:14.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.854+0000 7ff21ed5e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2181f6d80 con 0x7ff21815cf40 2023-09-23T03:24:14.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.854+0000 7ff21ed5e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2181f6170 con 0x7ff2180b21d0 2023-09-23T03:24:14.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.854+0000 7ff21ed5e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2181f6450 con 0x7ff21806b1e0 2023-09-23T03:24:14.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.854+0000 7ff21d2fb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff21815cf40 0x7ff2180c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:14.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.854+0000 7ff21d2fb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff21815cf40 0x7ff2180c8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50806/0 (socket says 172.21.15.154:50806) 2023-09-23T03:24:14.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.854+0000 7ff21d2fb640 1 -- 172.21.15.154:0/2053689453 learned_addr learned my addr 172.21.15.154:0/2053689453 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:14.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.855+0000 7ff20ffff640 1 --2- 172.21.15.154:0/2053689453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2180b21d0 0x7ff218156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:14.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.855+0000 7ff21cafa640 1 --2- 172.21.15.154:0/2053689453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff21806b1e0 0x7ff2180c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:14.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.855+0000 7ff21d2fb640 1 -- 172.21.15.154:0/2053689453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff21806b1e0 msgr2=0x7ff2180c9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:14.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.855+0000 7ff21d2fb640 1 --2- 172.21.15.154:0/2053689453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff21806b1e0 0x7ff2180c9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:14.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.855+0000 7ff21d2fb640 1 -- 172.21.15.154:0/2053689453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2180b21d0 msgr2=0x7ff218156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:14.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.855+0000 7ff21d2fb640 1 --2- 172.21.15.154:0/2053689453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2180b21d0 0x7ff218156c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:14.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.855+0000 7ff21d2fb640 1 -- 172.21.15.154:0/2053689453 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2140094d0 con 0x7ff21815cf40 2023-09-23T03:24:14.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.855+0000 7ff20ffff640 1 --2- 172.21.15.154:0/2053689453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2180b21d0 0x7ff218156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:14.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.855+0000 7ff21cafa640 1 --2- 172.21.15.154:0/2053689453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff21806b1e0 0x7ff2180c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:14.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.855+0000 7ff21d2fb640 1 --2- 172.21.15.154:0/2053689453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff21815cf40 0x7ff2180c8870 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7ff21000c7d0 tx=0x7ff21000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:14.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.855+0000 7ff20d7fa640 1 -- 172.21.15.154:0/2053689453 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21000a640 con 0x7ff21815cf40 2023-09-23T03:24:14.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.855+0000 7ff20d7fa640 1 -- 172.21.15.154:0/2053689453 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff21000a7c0 con 0x7ff21815cf40 2023-09-23T03:24:14.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.855+0000 7ff20d7fa640 1 -- 172.21.15.154:0/2053689453 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21000abb0 con 0x7ff21815cf40 2023-09-23T03:24:14.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.855+0000 7ff21ed5e640 1 -- 172.21.15.154:0/2053689453 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2181f6730 con 0x7ff21815cf40 2023-09-23T03:24:14.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.856+0000 7ff21ed5e640 1 -- 172.21.15.154:0/2053689453 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff218157190 con 0x7ff21815cf40 2023-09-23T03:24:14.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.857+0000 7ff21ed5e640 1 -- 172.21.15.154:0/2053689453 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff218073eb0 con 0x7ff21815cf40 2023-09-23T03:24:14.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.858+0000 7ff20d7fa640 1 -- 172.21.15.154:0/2053689453 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7ff21000ad10 con 0x7ff21815cf40 2023-09-23T03:24:14.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.859+0000 7ff20d7fa640 1 --2- 172.21.15.154:0/2053689453 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff20406af30 0x7ff20406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:14.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.859+0000 7ff20d7fa640 1 -- 172.21.15.154:0/2053689453 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff210088c10 con 0x7ff21815cf40 2023-09-23T03:24:14.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.859+0000 7ff21cafa640 1 --2- 172.21.15.154:0/2053689453 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff20406af30 0x7ff20406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:14.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.860+0000 7ff21cafa640 1 --2- 172.21.15.154:0/2053689453 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff20406af30 0x7ff20406d3f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff214036000 tx=0x7ff21403b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:14.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:14.863+0000 7ff20d7fa640 1 -- 172.21.15.154:0/2053689453 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7ff210053ba0 con 0x7ff21815cf40 2023-09-23T03:24:15.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.048+0000 7ff21ed5e640 1 -- 172.21.15.154:0/2053689453 --> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff2180cb260 con 0x7ff20406af30 2023-09-23T03:24:15.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.061+0000 7ff20d7fa640 1 -- 172.21.15.154:0/2053689453 <== mgr.4110 v2:172.21.15.154:6832/2422652104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+274520 (secure 0 0 0) 0x7ff214030c10 con 0x7ff20406af30 2023-09-23T03:24:15.066 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:15.067 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2023-09-23T03:24:15.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.067+0000 7ff21ed5e640 1 -- 172.21.15.154:0/2053689453 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff20406af30 msgr2=0x7ff20406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:15.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.067+0000 7ff21ed5e640 1 --2- 172.21.15.154:0/2053689453 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff20406af30 0x7ff20406d3f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff214036000 tx=0x7ff21403b040 comp rx=0 tx=0).stop 2023-09-23T03:24:15.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.067+0000 7ff21ed5e640 1 -- 172.21.15.154:0/2053689453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff21815cf40 msgr2=0x7ff2180c8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:15.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.067+0000 7ff21ed5e640 1 --2- 172.21.15.154:0/2053689453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff21815cf40 0x7ff2180c8870 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7ff21000c7d0 tx=0x7ff21000cc90 comp rx=0 tx=0).stop 2023-09-23T03:24:15.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.067+0000 7ff21ed5e640 1 -- 172.21.15.154:0/2053689453 shutdown_connections 2023-09-23T03:24:15.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.067+0000 7ff21ed5e640 1 --2- 172.21.15.154:0/2053689453 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff20406af30 0x7ff20406d3f0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:15.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.067+0000 7ff21ed5e640 1 --2- 172.21.15.154:0/2053689453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff21815cf40 0x7ff2180c8870 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:15.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.067+0000 7ff21ed5e640 1 --2- 172.21.15.154:0/2053689453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff2180b21d0 0x7ff218156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:15.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.067+0000 7ff21ed5e640 1 --2- 172.21.15.154:0/2053689453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff21806b1e0 0x7ff2180c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:15.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.067+0000 7ff21ed5e640 1 -- 172.21.15.154:0/2053689453 >> 172.21.15.154:0/2053689453 conn(0x7ff21806fc60 msgr2=0x7ff218070040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:15.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.067+0000 7ff21ed5e640 1 -- 172.21.15.154:0/2053689453 shutdown_connections 2023-09-23T03:24:15.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.067+0000 7ff21ed5e640 1 -- 172.21.15.154:0/2053689453 wait complete. 2023-09-23T03:24:15.087 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":348,"stamp":"2023-09-23T03:24:13.354263+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258240,"kb_used_data":8792,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716480,"statfs":{"total":773094113280,"available":772829675520,"internally_reserved":0,"allocated":9003008,"data_stored":4889828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":255315514},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.778235"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2023-09-23T03:24:08.586103+0000","last_change":"2023-09-23T03:24:08.586103+0000","last_active":"2023-09-23T03:24:08.558342+0000","last_peered":"2023-09-23T03:24:08.558342+0000","last_clean":"2023-09-23T03:24:08.558342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:08.586103+0000","last_undegraded":"2023-09-23T03:24:08.586103+0000","last_fullsized":"2023-09-23T03:24:08.586103+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2023-09-23T03:24:08.585892+0000","last_change":"2023-09-23T03:24:08.585892+0000","last_active":"2023-09-23T03:24:08.558342+0000","last_peered":"2023-09-23T03:24:08.558342+0000","last_clean":"2023-09-23T03:24:08.558342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:08.585892+0000","last_undegraded":"2023-09-23T03:24:08.585892+0000","last_fullsized":"2023-09-23T03:24:08.585892+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2023-09-23T03:24:08.587268+0000","last_change":"2023-09-23T03:24:08.587268+0000","last_active":"2023-09-23T03:24:08.558342+0000","last_peered":"2023-09-23T03:24:08.558342+0000","last_clean":"2023-09-23T03:24:08.558342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:08.587268+0000","last_undegraded":"2023-09-23T03:24:08.587268+0000","last_fullsized":"2023-09-23T03:24:08.587268+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2023-09-23T03:24:08.581873+0000","last_change":"2023-09-23T03:24:08.581873+0000","last_active":"2023-09-23T03:24:08.558342+0000","last_peered":"2023-09-23T03:24:08.558342+0000","last_clean":"2023-09-23T03:24:08.558342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:08.581873+0000","last_undegraded":"2023-09-23T03:24:08.581873+0000","last_fullsized":"2023-09-23T03:24:08.581873+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2023-09-23T03:24:08.590791+0000","last_change":"2023-09-23T03:24:08.590791+0000","last_active":"2023-09-23T03:24:08.558342+0000","last_peered":"2023-09-23T03:24:08.558342+0000","last_clean":"2023-09-23T03:24:08.558342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:08.590791+0000","last_undegraded":"2023-09-23T03:24:08.590791+0000","last_fullsized":"2023-09-23T03:24:08.590791+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2023-09-23T03:24:08.594907+0000","last_change":"2023-09-23T03:24:08.594907+0000","last_active":"2023-09-23T03:24:08.558342+0000","last_peered":"2023-09-23T03:24:08.558342+0000","last_clean":"2023-09-23T03:24:08.558342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:08.594907+0000","last_undegraded":"2023-09-23T03:24:08.594907+0000","last_fullsized":"2023-09-23T03:24:08.594907+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2023-09-23T03:24:08.596006+0000","last_change":"2023-09-23T03:24:08.596006+0000","last_active":"2023-09-23T03:24:08.558342+0000","last_peered":"2023-09-23T03:24:08.558342+0000","last_clean":"2023-09-23T03:24:08.558342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:08.596006+0000","last_undegraded":"2023-09-23T03:24:08.596006+0000","last_fullsized":"2023-09-23T03:24:08.596006+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2023-09-23T03:24:08.585777+0000","last_change":"2023-09-23T03:24:08.585777+0000","last_active":"2023-09-23T03:24:08.558342+0000","last_peered":"2023-09-23T03:24:08.558342+0000","last_clean":"2023-09-23T03:24:08.558342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:08.585777+0000","last_undegraded":"2023-09-23T03:24:08.585777+0000","last_fullsized":"2023-09-23T03:24:08.585777+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2023-09-23T03:24:08.587214+0000","last_change":"2023-09-23T03:24:08.587214+0000","last_active":"2023-09-23T03:24:08.558342+0000","last_peered":"2023-09-23T03:24:08.558342+0000","last_clean":"2023-09-23T03:24:08.558342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:08.587214+0000","last_undegraded":"2023-09-23T03:24:08.587214+0000","last_fullsized":"2023-09-23T03:24:08.587214+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2023-09-23T03:24:08.595082+0000","last_change":"2023-09-23T03:24:08.595082+0000","last_active":"2023-09-23T03:24:08.558342+0000","last_peered":"2023-09-23T03:24:08.558342+0000","last_clean":"2023-09-23T03:24:08.558342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:08.595082+0000","last_undegraded":"2023-09-23T03:24:08.595082+0000","last_fullsized":"2023-09-23T03:24:08.595082+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2023-09-23T03:24:08.585147+0000","last_change":"2023-09-23T03:24:08.585147+0000","last_active":"2023-09-23T03:24:08.558342+0000","last_peered":"2023-09-23T03:24:08.558342+0000","last_clean":"2023-09-23T03:24:08.558342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:08.585147+0000","last_undegraded":"2023-09-23T03:24:08.585147+0000","last_fullsized":"2023-09-23T03:24:08.585147+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2023-09-23T03:24:08.580778+0000","last_change":"2023-09-23T03:24:08.580778+0000","last_active":"2023-09-23T03:24:08.558342+0000","last_peered":"2023-09-23T03:24:08.558342+0000","last_clean":"2023-09-23T03:24:08.558342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:08.580778+0000","last_undegraded":"2023-09-23T03:24:08.580778+0000","last_fullsized":"2023-09-23T03:24:08.580778+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2023-09-23T03:24:08.594954+0000","last_change":"2023-09-23T03:24:08.594954+0000","last_active":"2023-09-23T03:24:08.558342+0000","last_peered":"2023-09-23T03:24:08.558342+0000","last_clean":"2023-09-23T03:24:08.558342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:08.594954+0000","last_undegraded":"2023-09-23T03:24:08.594954+0000","last_fullsized":"2023-09-23T03:24:08.594954+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2023-09-23T03:24:08.585772+0000","last_change":"2023-09-23T03:24:08.585772+0000","last_active":"2023-09-23T03:24:08.558342+0000","last_peered":"2023-09-23T03:24:08.558342+0000","last_clean":"2023-09-23T03:24:08.558342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:08.585772+0000","last_undegraded":"2023-09-23T03:24:08.585772+0000","last_fullsized":"2023-09-23T03:24:08.585772+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2023-09-23T03:24:08.584965+0000","last_change":"2023-09-23T03:24:08.584965+0000","last_active":"2023-09-23T03:24:08.558342+0000","last_peered":"2023-09-23T03:24:08.558342+0000","last_clean":"2023-09-23T03:24:08.558342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:08.584965+0000","last_undegraded":"2023-09-23T03:24:08.584965+0000","last_fullsized":"2023-09-23T03:24:08.584965+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2023-09-23T03:24:08.596717+0000","last_change":"2023-09-23T03:24:08.596717+0000","last_active":"2023-09-23T03:24:08.558342+0000","last_peered":"2023-09-23T03:24:08.558342+0000","last_clean":"2023-09-23T03:24:08.558342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:08.596717+0000","last_undegraded":"2023-09-23T03:24:08.596717+0000","last_fullsized":"2023-09-23T03:24:08.596717+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2023-09-23T03:24:08.586046+0000","last_change":"2023-09-23T03:24:08.586046+0000","last_active":"2023-09-23T03:24:08.558342+0000","last_peered":"2023-09-23T03:24:08.558342+0000","last_clean":"2023-09-23T03:24:08.558342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:08.586046+0000","last_undegraded":"2023-09-23T03:24:08.586046+0000","last_fullsized":"2023-09-23T03:24:08.586046+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2023-09-23T03:24:08.591032+0000","last_change":"2023-09-23T03:24:08.591032+0000","last_active":"2023-09-23T03:24:08.558342+0000","last_peered":"2023-09-23T03:24:08.558342+0000","last_clean":"2023-09-23T03:24:08.558342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:08.591032+0000","last_undegraded":"2023-09-23T03:24:08.591032+0000","last_fullsized":"2023-09-23T03:24:08.591032+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2023-09-23T03:24:08.584454+0000","last_change":"2023-09-23T03:24:08.584454+0000","last_active":"2023-09-23T03:24:08.558342+0000","last_peered":"2023-09-23T03:24:08.558342+0000","last_clean":"2023-09-23T03:24:08.558342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:08.584454+0000","last_undegraded":"2023-09-23T03:24:08.584454+0000","last_fullsized":"2023-09-23T03:24:08.584454+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2023-09-23T03:24:08.580595+0000","last_change":"2023-09-23T03:24:08.580595+0000","last_active":"2023-09-23T03:24:08.558342+0000","last_peered":"2023-09-23T03:24:08.558342+0000","last_clean":"2023-09-23T03:24:08.558342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:08.580595+0000","last_undegraded":"2023-09-23T03:24:08.580595+0000","last_fullsized":"2023-09-23T03:24:08.580595+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2023-09-23T03:24:08.587469+0000","last_change":"2023-09-23T03:24:08.587469+0000","last_active":"2023-09-23T03:24:08.558342+0000","last_peered":"2023-09-23T03:24:08.558342+0000","last_clean":"2023-09-23T03:24:08.558342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:08.587469+0000","last_undegraded":"2023-09-23T03:24:08.587469+0000","last_fullsized":"2023-09-23T03:24:08.587469+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2023-09-23T03:24:08.586552+0000","last_change":"2023-09-23T03:24:08.586552+0000","last_active":"2023-09-23T03:24:08.558342+0000","last_peered":"2023-09-23T03:24:08.558342+0000","last_clean":"2023-09-23T03:24:08.558342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:08.586552+0000","last_undegraded":"2023-09-23T03:24:08.586552+0000","last_fullsized":"2023-09-23T03:24:08.586552+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2023-09-23T03:24:08.586097+0000","last_change":"2023-09-23T03:24:08.586097+0000","last_active":"2023-09-23T03:24:08.558342+0000","last_peered":"2023-09-23T03:24:08.558342+0000","last_clean":"2023-09-23T03:24:08.558342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:08.586097+0000","last_undegraded":"2023-09-23T03:24:08.586097+0000","last_fullsized":"2023-09-23T03:24:08.586097+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2023-09-23T03:24:08.595105+0000","last_change":"2023-09-23T03:24:08.595105+0000","last_active":"2023-09-23T03:24:08.558342+0000","last_peered":"2023-09-23T03:24:08.558342+0000","last_clean":"2023-09-23T03:24:08.558342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:08.595105+0000","last_undegraded":"2023-09-23T03:24:08.595105+0000","last_fullsized":"2023-09-23T03:24:08.595105+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2023-09-23T03:24:08.596857+0000","last_change":"2023-09-23T03:24:08.596857+0000","last_active":"2023-09-23T03:24:08.558342+0000","last_peered":"2023-09-23T03:24:08.558342+0000","last_clean":"2023-09-23T03:24:08.558342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:08.596857+0000","last_undegraded":"2023-09-23T03:24:08.596857+0000","last_fullsized":"2023-09-23T03:24:08.596857+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2023-09-23T03:24:08.585973+0000","last_change":"2023-09-23T03:24:08.585973+0000","last_active":"2023-09-23T03:24:08.558342+0000","last_peered":"2023-09-23T03:24:08.558342+0000","last_clean":"2023-09-23T03:24:08.558342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:08.585973+0000","last_undegraded":"2023-09-23T03:24:08.585973+0000","last_fullsized":"2023-09-23T03:24:08.585973+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2023-09-23T03:24:08.586097+0000","last_change":"2023-09-23T03:24:08.586097+0000","last_active":"2023-09-23T03:24:08.558342+0000","last_peered":"2023-09-23T03:24:08.558342+0000","last_clean":"2023-09-23T03:24:08.558342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:08.586097+0000","last_undegraded":"2023-09-23T03:24:08.586097+0000","last_fullsized":"2023-09-23T03:24:08.586097+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2023-09-23T03:24:08.580597+0000","last_change":"2023-09-23T03:24:08.580597+0000","last_active":"2023-09-23T03:24:08.558342+0000","last_peered":"2023-09-23T03:24:08.558342+0000","last_clean":"2023-09-23T03:24:08.558342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:08.580597+0000","last_undegraded":"2023-09-23T03:24:08.580597+0000","last_fullsized":"2023-09-23T03:24:08.580597+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2023-09-23T03:24:08.584735+0000","last_change":"2023-09-23T03:24:08.584735+0000","last_active":"2023-09-23T03:24:08.558342+0000","last_peered":"2023-09-23T03:24:08.558342+0000","last_clean":"2023-09-23T03:24:08.558342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:08.584735+0000","last_undegraded":"2023-09-23T03:24:08.584735+0000","last_fullsized":"2023-09-23T03:24:08.584735+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2023-09-23T03:24:08.580724+0000","last_change":"2023-09-23T03:24:08.580724+0000","last_active":"2023-09-23T03:24:08.558342+0000","last_peered":"2023-09-23T03:24:08.558342+0000","last_clean":"2023-09-23T03:24:08.558342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:08.580724+0000","last_undegraded":"2023-09-23T03:24:08.580724+0000","last_fullsized":"2023-09-23T03:24:08.580724+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2023-09-23T03:24:08.580164+0000","last_change":"2023-09-23T03:24:08.580164+0000","last_active":"2023-09-23T03:24:08.558342+0000","last_peered":"2023-09-23T03:24:08.558342+0000","last_clean":"2023-09-23T03:24:08.558342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:08.580164+0000","last_undegraded":"2023-09-23T03:24:08.580164+0000","last_fullsized":"2023-09-23T03:24:08.580164+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-23T03:24:09.574937+0000","last_change":"2023-09-23T03:24:09.574937+0000","last_active":"2023-09-23T03:24:09.574937+0000","last_peered":"2023-09-23T03:24:09.574937+0000","last_clean":"2023-09-23T03:24:09.574937+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:09.574937+0000","last_undegraded":"2023-09-23T03:24:09.574937+0000","last_fullsized":"2023-09-23T03:24:09.574937+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2023-09-23T03:24:08.586883+0000","last_change":"2023-09-23T03:24:08.586883+0000","last_active":"2023-09-23T03:24:08.558342+0000","last_peered":"2023-09-23T03:24:08.558342+0000","last_clean":"2023-09-23T03:24:08.558342+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-23T03:24:08.586883+0000","last_undegraded":"2023-09-23T03:24:08.586883+0000","last_fullsized":"2023-09-23T03:24:08.586883+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":217,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-09-23T03:24:06.038749+0000","last_change":"2023-09-23T03:15:11.042447+0000","last_active":"2023-09-23T03:24:06.038749+0000","last_peered":"2023-09-23T03:24:06.038749+0000","last_clean":"2023-09-23T03:24:06.038749+0000","last_became_active":"2023-09-23T03:15:11.040322+0000","last_became_peered":"2023-09-23T03:15:11.040322+0000","last_unstale":"2023-09-23T03:24:06.038749+0000","last_undegraded":"2023-09-23T03:24:06.038749+0000","last_fullsized":"2023-09-23T03:24:06.038749+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_clean_scrub_stamp":"2023-09-23T03:15:09.474842+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:18:36.331645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574958,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32716,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339124,"statfs":{"total":96636764160,"available":96603262976,"internally_reserved":0,"allocated":1572864,"data_stored":1054899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:23:25 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.012,"15min":1.0029999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.442},"max":{"1min":1.556,"5min":1.6890000000000001,"15min":2.149},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0660000000000001,"15min":1.0509999999999999},"min":{"1min":0.78200000000000003,"5min":0.56200000000000006,"15min":0.52000000000000002},"max":{"1min":1.4870000000000001,"5min":1.681,"15min":2.1480000000000001},"last":0.74199999999999999}]},{"osd":1,"last update":"Sat Sep 23 03:23:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.214,"15min":1.2150000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.504},"max":{"1min":1.4990000000000001,"5min":1.671,"15min":3.1859999999999999},"last":1.004},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.246,"15min":1.3300000000000001},"min":{"1min":0.78400000000000003,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.49,"5min":1.736,"15min":2.613},"last":1.6220000000000001}]},{"osd":2,"last update":"Sat Sep 23 03:23:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.099,"15min":1.135},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.42399999999999999},"max":{"1min":1.3149999999999999,"5min":1.667,"15min":2.331},"last":1.5389999999999999},{"interface":"front","average":{"1min":1.151,"5min":1.1779999999999999,"15min":1.254},"min":{"1min":0.749,"5min":0.621,"15min":0.55200000000000005},"max":{"1min":1.613,"5min":1.724,"15min":3.093},"last":1.3}]},{"osd":3,"last update":"Sat Sep 23 03:23:25 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.208,"15min":1.266},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.4450000000000001,"5min":1.9039999999999999,"15min":2.532},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.286,"5min":1.3080000000000001,"15min":1.3919999999999999},"min":{"1min":0.76500000000000001,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.633,"5min":1.8049999999999999,"15min":3.2349999999999999},"last":1.173}]},{"osd":4,"last update":"Sat Sep 23 03:23:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2949999999999999,"15min":1.448},"min":{"1min":0.89600000000000002,"5min":0.79200000000000004,"15min":0.45400000000000001},"max":{"1min":1.5800000000000001,"5min":1.867,"15min":2.7210000000000001},"last":1.7290000000000001},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.427,"15min":1.514},"min":{"1min":1.157,"5min":0.81699999999999995,"15min":0.48099999999999998},"max":{"1min":1.6459999999999999,"5min":1.944,"15min":3.7719999999999998},"last":1.679}]},{"osd":5,"last update":"Sat Sep 23 03:23:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.351,"15min":1.4390000000000001},"min":{"1min":0.749,"5min":0.72699999999999998,"15min":0.47999999999999998},"max":{"1min":1.798,"5min":1.8799999999999999,"15min":2.3690000000000002},"last":1.383},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.367,"15min":1.492},"min":{"1min":1.1930000000000001,"5min":1.0069999999999999,"15min":0.497},"max":{"1min":1.679,"5min":2.008,"15min":2.4689999999999999},"last":1.7989999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:23:25 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.4830000000000001,"15min":1.5920000000000001},"min":{"1min":1.026,"5min":1.026,"15min":0.53800000000000003},"max":{"1min":1.7769999999999999,"5min":1.9399999999999999,"15min":2.8420000000000001},"last":1.8500000000000001},{"interface":"front","average":{"1min":1.413,"5min":1.4159999999999999,"15min":1.5429999999999999},"min":{"1min":1.089,"5min":0.92400000000000004,"15min":0.45900000000000002},"max":{"1min":1.7869999999999999,"5min":2.0699999999999998,"15min":3.6739999999999999},"last":1.77}]}]},{"osd":6,"up_from":13,"seq":55834574958,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":978944,"data_stored":464531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:23:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.111,"15min":1.0529999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.38},"max":{"1min":1.4850000000000001,"5min":1.778,"15min":1.778},"last":0.72099999999999997},{"interface":"front","average":{"1min":1.081,"5min":1.0880000000000001,"15min":1.0629999999999999},"min":{"1min":0.71499999999999997,"5min":0.61599999999999999,"15min":0.46100000000000002},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":2.0529999999999999},"last":0.79800000000000004}]},{"osd":1,"last update":"Sat Sep 23 03:23:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.1220000000000001,"15min":1.1040000000000001},"min":{"1min":0.64100000000000001,"5min":0.59099999999999997,"15min":0.438},"max":{"1min":1.468,"5min":1.7030000000000001,"15min":1.867},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.093,"5min":1.153,"15min":1.21},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.5269999999999999,"5min":1.7190000000000001,"15min":2.242},"last":0.57099999999999995}]},{"osd":2,"last update":"Sat Sep 23 03:23:20 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.109,"15min":1.1080000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.51300000000000001},"max":{"1min":1.5329999999999999,"5min":1.6659999999999999,"15min":2.097},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.173,"15min":1.236},"min":{"1min":0.71999999999999997,"5min":0.66200000000000003,"15min":0.64300000000000002},"max":{"1min":1.375,"5min":1.8149999999999999,"15min":1.903},"last":1.214}]},{"osd":3,"last update":"Sat Sep 23 03:23:20 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.278,"15min":1.3360000000000001},"min":{"1min":0.92400000000000004,"5min":0.65800000000000003,"15min":0.54600000000000004},"max":{"1min":1.77,"5min":1.8879999999999999,"15min":2.6509999999999998},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3260000000000001,"15min":1.415},"min":{"1min":0.95599999999999996,"5min":0.745,"15min":0.74299999999999999},"max":{"1min":1.665,"5min":2.0339999999999998,"15min":2.3740000000000001},"last":1.1679999999999999}]},{"osd":4,"last update":"Sat Sep 23 03:23:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3839999999999999},"min":{"1min":0.98199999999999998,"5min":0.81399999999999995,"15min":0.498},"max":{"1min":1.7190000000000001,"5min":1.843,"15min":2.3210000000000002},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.276,"5min":1.2809999999999999,"15min":1.2889999999999999},"min":{"1min":0.89300000000000002,"5min":0.74399999999999999,"15min":0.56000000000000005},"max":{"1min":1.6719999999999999,"5min":1.708,"15min":2.2109999999999999},"last":0.96199999999999997}]},{"osd":5,"last update":"Sat Sep 23 03:23:20 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.405,"15min":1.3979999999999999},"min":{"1min":1.171,"5min":0.84699999999999998,"15min":0.69199999999999995},"max":{"1min":1.857,"5min":1.857,"15min":2.2050000000000001},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.361,"5min":1.353,"15min":1.45},"min":{"1min":0.95499999999999996,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.887,"5min":1.9410000000000001,"15min":4.758},"last":1.3220000000000001}]},{"osd":7,"last update":"Sat Sep 23 03:23:20 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.4810000000000001,"15min":1.5329999999999999},"min":{"1min":1.1100000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.821,"5min":1.978,"15min":2.476},"last":1.254},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.4870000000000001,"15min":1.5409999999999999},"min":{"1min":1.278,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.9470000000000001,"5min":2.1739999999999999,"15min":2.4060000000000001},"last":1.4239999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574960,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":978944,"data_stored":464531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":1.006,"15min":1.0269999999999999},"min":{"1min":0.64200000000000002,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.27,"5min":1.732,"15min":1.732},"last":1.179},{"interface":"front","average":{"1min":1.139,"5min":1.099,"15min":1.0509999999999999},"min":{"1min":0.69199999999999995,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.5569999999999999,"5min":2.2320000000000002,"15min":2.2320000000000002},"last":1.1379999999999999}]},{"osd":1,"last update":"Sat Sep 23 03:23:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.254,"15min":1.2270000000000001},"min":{"1min":0.68300000000000005,"5min":0.62,"15min":0.62},"max":{"1min":1.599,"5min":1.7869999999999999,"15min":1.849},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.284,"15min":1.298},"min":{"1min":0.83199999999999996,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.663,"5min":1.7410000000000001,"15min":1.893},"last":0.94199999999999995}]},{"osd":2,"last update":"Sat Sep 23 03:23:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.2,"15min":1.2050000000000001},"min":{"1min":0.65800000000000003,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.6910000000000001,"5min":1.726,"15min":1.726},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.2,"15min":1.2669999999999999},"min":{"1min":0.73699999999999999,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.464,"5min":1.7529999999999999,"15min":1.7629999999999999},"last":0.89700000000000002}]},{"osd":3,"last update":"Sat Sep 23 03:23:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.198,"15min":1.304},"min":{"1min":0.68500000000000005,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.5980000000000001,"5min":1.696,"15min":1.905},"last":1.327},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.302,"15min":1.411},"min":{"1min":0.93799999999999994,"5min":0.71699999999999997,"15min":0.69799999999999995},"max":{"1min":1.492,"5min":1.792,"15min":3.6110000000000002},"last":1.099}]},{"osd":4,"last update":"Sat Sep 23 03:23:17 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.351,"15min":1.415},"min":{"1min":0.98399999999999999,"5min":0.83099999999999996,"15min":0.54100000000000004},"max":{"1min":1.843,"5min":1.9359999999999999,"15min":2.069},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2829999999999999,"15min":1.401},"min":{"1min":0.82799999999999996,"5min":0.60299999999999998,"15min":0.48899999999999999},"max":{"1min":1.6699999999999999,"5min":1.7889999999999999,"15min":2.319},"last":1.0569999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:23:17 2023","interfaces":[{"interface":"back","average":{"1min":1.488,"5min":1.49,"15min":1.5229999999999999},"min":{"1min":1.2450000000000001,"5min":0.69899999999999995,"15min":0.59199999999999997},"max":{"1min":1.7729999999999999,"5min":2.0350000000000001,"15min":2.1509999999999998},"last":1.387},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.524,"15min":1.587},"min":{"1min":1.119,"5min":0.78700000000000003,"15min":0.55100000000000005},"max":{"1min":1.605,"5min":9.6560000000000006,"15min":9.6560000000000006},"last":1.292}]},{"osd":7,"last update":"Sat Sep 23 03:23:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.369,"15min":1.4830000000000001},"min":{"1min":1.0860000000000001,"5min":0.89100000000000001,"15min":0.626},"max":{"1min":1.784,"5min":1.8480000000000001,"15min":2.0350000000000001},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4850000000000001,"15min":1.5960000000000001},"min":{"1min":1.228,"5min":0.85999999999999999,"15min":0.60799999999999998},"max":{"1min":1.7390000000000001,"5min":1.919,"15min":2.1960000000000002},"last":1.244}]}]},{"osd":4,"up_from":13,"seq":55834574959,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":978944,"data_stored":464531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:23:20 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.093,"15min":1.117},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.59899999999999998},"max":{"1min":1.4770000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.339},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.181,"15min":1.155},"min":{"1min":0.67400000000000004,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.581,"5min":1.905,"15min":1.9179999999999999},"last":2.004}]},{"osd":1,"last update":"Sat Sep 23 03:23:20 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.232,"15min":1.2110000000000001},"min":{"1min":0.65400000000000003,"5min":0.64000000000000001,"15min":0.61799999999999999},"max":{"1min":1.609,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.8200000000000001},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.0900000000000001,"15min":1.1799999999999999},"min":{"1min":0.72299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.625,"5min":1.625,"15min":3.2189999999999999},"last":1.252}]},{"osd":2,"last update":"Sat Sep 23 03:23:20 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.111,"15min":1.2},"min":{"1min":0.90600000000000003,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.5509999999999999,"5min":2.129,"15min":2.5920000000000001},"last":1.647},{"interface":"front","average":{"1min":1.202,"5min":1.2470000000000001,"15min":1.228},"min":{"1min":0.73499999999999999,"5min":0.69999999999999996,"15min":0.68300000000000005},"max":{"1min":1.784,"5min":1.909,"15min":1.909},"last":1.9530000000000001}]},{"osd":3,"last update":"Sat Sep 23 03:23:20 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1719999999999999,"15min":1.1699999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.69199999999999995},"max":{"1min":1.504,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.216,"5min":1.27,"15min":1.3169999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.6359999999999999,"5min":2.1779999999999999,"15min":2.4969999999999999},"last":1.5389999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:23:20 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.395,"15min":1.3560000000000001},"min":{"1min":0.876,"5min":0.83099999999999996,"15min":0.70199999999999996},"max":{"1min":1.7,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.6020000000000001},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.355,"15min":1.333},"min":{"1min":0.97999999999999998,"5min":0.84299999999999997,"15min":0.81799999999999995},"max":{"1min":1.736,"5min":2.887,"15min":2.887},"last":1.486}]},{"osd":6,"last update":"Sat Sep 23 03:23:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.3680000000000001,"15min":1.377},"min":{"1min":0.86499999999999999,"5min":0.81299999999999994,"15min":0.66400000000000003},"max":{"1min":1.7509999999999999,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":1.7250000000000001},{"interface":"front","average":{"1min":1.476,"5min":1.5349999999999999,"15min":1.488},"min":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":0.89900000000000002},"max":{"1min":1.8580000000000001,"5min":2.0579999999999998,"15min":2.1779999999999999},"last":2.0800000000000001}]},{"osd":7,"last update":"Sat Sep 23 03:23:20 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.484,"15min":1.47},"min":{"1min":1.0880000000000001,"5min":0.98299999999999998,"15min":0.81399999999999995},"max":{"1min":1.849,"5min":2.5680000000000001,"15min":2.5680000000000001},"last":1.7669999999999999},{"interface":"front","average":{"1min":1.486,"5min":1.5549999999999999,"15min":1.556},"min":{"1min":1.1619999999999999,"5min":1.056,"15min":0.84199999999999997},"max":{"1min":1.8340000000000001,"5min":2.4769999999999999,"15min":2.4769999999999999},"last":1.8680000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574959,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":978944,"data_stored":464531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.88900000000000001,"15min":0.876},"min":{"1min":0.68799999999999994,"5min":0.42199999999999999,"15min":0.40200000000000002},"max":{"1min":1.7050000000000001,"5min":2.952,"15min":2.952},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.58699999999999997,"5min":0.377,"15min":0.29299999999999998},"max":{"1min":1.419,"5min":1.419,"15min":1.8979999999999999},"last":0.52600000000000002}]},{"osd":1,"last update":"Sat Sep 23 03:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85399999999999998,"15min":0.92500000000000004},"min":{"1min":0.51700000000000002,"5min":0.442,"15min":0.42299999999999999},"max":{"1min":1.603,"5min":1.603,"15min":1.6839999999999999},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.018,"5min":0.99299999999999999,"15min":0.98499999999999999},"min":{"1min":0.67100000000000004,"5min":0.54700000000000004,"15min":0.45400000000000001},"max":{"1min":1.605,"5min":2.903,"15min":2.903},"last":1.2709999999999999}]},{"osd":2,"last update":"Sat Sep 23 03:23:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.048,"15min":1.087},"min":{"1min":0.70699999999999996,"5min":0.54800000000000004,"15min":0.46500000000000002},"max":{"1min":1.9850000000000001,"5min":1.9850000000000001,"15min":2.2240000000000002},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.056,"5min":1.1120000000000001,"15min":1.1160000000000001},"min":{"1min":0.76500000000000001,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3360000000000001,"5min":3.129,"15min":3.129},"last":0.88800000000000001}]},{"osd":4,"last update":"Sat Sep 23 03:23:30 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.1850000000000001,"15min":1.2789999999999999},"min":{"1min":0.873,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.671,"5min":2.0099999999999998,"15min":2.698},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.3100000000000001,"15min":1.399},"min":{"1min":0.93999999999999995,"5min":0.93899999999999995,"15min":0.752},"max":{"1min":1.587,"5min":1.825,"15min":2.419},"last":1.23}]},{"osd":5,"last update":"Sat Sep 23 03:23:30 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.361,"15min":1.355},"min":{"1min":1.004,"5min":0.90300000000000002,"15min":0.753},"max":{"1min":2.04,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":1.093},{"interface":"front","average":{"1min":1.345,"5min":1.27,"15min":1.3260000000000001},"min":{"1min":1.028,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":2.0070000000000001},"last":1.008}]},{"osd":6,"last update":"Sat Sep 23 03:23:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.3360000000000001,"15min":1.3939999999999999},"min":{"1min":1.0940000000000001,"5min":0.82699999999999996,"15min":0.69799999999999995},"max":{"1min":1.7869999999999999,"5min":1.9550000000000001,"15min":2.6440000000000001},"last":1.167},{"interface":"front","average":{"1min":1.45,"5min":1.4279999999999999,"15min":1.462},"min":{"1min":0.999,"5min":0.94499999999999995,"15min":0.67800000000000005},"max":{"1min":2.1360000000000001,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":1.385}]},{"osd":7,"last update":"Sat Sep 23 03:23:30 2023","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.345,"15min":1.4430000000000001},"min":{"1min":1.097,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.845,"5min":2.2120000000000002,"15min":2.7879999999999998},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.411,"5min":1.3959999999999999,"15min":1.48},"min":{"1min":1.0700000000000001,"5min":0.81200000000000006,"15min":0.80900000000000005},"max":{"1min":1.738,"5min":2.2029999999999998,"15min":2.5379999999999998},"last":1.3200000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574960,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":966656,"data_stored":458394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.86699999999999999,"15min":0.83699999999999997},"min":{"1min":0.50800000000000001,"5min":0.371,"15min":0.30299999999999999},"max":{"1min":1.284,"5min":1.643,"15min":1.643},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.90600000000000003,"15min":0.85599999999999998},"min":{"1min":0.46500000000000002,"5min":0.39300000000000002,"15min":0.36299999999999999},"max":{"1min":1.1919999999999999,"5min":1.5649999999999999,"15min":1.5840000000000001},"last":0.77300000000000002}]},{"osd":1,"last update":"Sat Sep 23 03:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.877,"15min":0.85899999999999999},"min":{"1min":0.55400000000000005,"5min":0.437,"15min":0.437},"max":{"1min":1.139,"5min":1.478,"15min":1.478},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.999,"15min":0.95399999999999996},"min":{"1min":0.59899999999999998,"5min":0.49399999999999999,"15min":0.38},"max":{"1min":1.26,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.68400000000000005}]},{"osd":3,"last update":"Sat Sep 23 03:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.008,"15min":0.96299999999999997},"min":{"1min":0.59799999999999998,"5min":0.51600000000000001,"15min":0.40699999999999997},"max":{"1min":1.542,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.0149999999999999,"15min":0.997},"min":{"1min":0.56399999999999995,"5min":0.51000000000000001,"15min":0.34999999999999998},"max":{"1min":1.2290000000000001,"5min":1.6779999999999999,"15min":1.78},"last":0.63100000000000001}]},{"osd":4,"last update":"Sat Sep 23 03:23:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.25,"15min":1.2949999999999999},"min":{"1min":0.88200000000000001,"5min":0.82099999999999995,"15min":0.55800000000000005},"max":{"1min":1.629,"5min":1.819,"15min":2.3109999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.218,"5min":1.272,"15min":1.3149999999999999},"min":{"1min":0.90700000000000003,"5min":0.77200000000000002,"15min":0.76100000000000001},"max":{"1min":1.6180000000000001,"5min":1.8080000000000001,"15min":2.234},"last":0.95499999999999996}]},{"osd":5,"last update":"Sat Sep 23 03:23:15 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.3340000000000001,"15min":1.375},"min":{"1min":1.0169999999999999,"5min":0.78600000000000003,"15min":0.71499999999999997},"max":{"1min":1.6899999999999999,"5min":1.8049999999999999,"15min":3.9409999999999998},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.349,"15min":1.466},"min":{"1min":1.1479999999999999,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.7390000000000001,"5min":1.9359999999999999,"15min":4.9199999999999999},"last":0.878}]},{"osd":6,"last update":"Sat Sep 23 03:23:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.3560000000000001,"15min":1.3879999999999999},"min":{"1min":0.88,"5min":0.83299999999999996,"15min":0.755},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":3.0009999999999999},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.399,"5min":1.403,"15min":1.4430000000000001},"min":{"1min":1.0389999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.839,"5min":2.1680000000000001,"15min":4.5800000000000001},"last":1.0980000000000001}]},{"osd":7,"last update":"Sat Sep 23 03:23:15 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.4019999999999999,"15min":1.395},"min":{"1min":1.2,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.9219999999999999,"5min":1.996,"15min":2.3149999999999999},"last":1.173},{"interface":"front","average":{"1min":1.417,"5min":1.4379999999999999,"15min":1.4430000000000001},"min":{"1min":0.87,"5min":0.82599999999999996,"15min":0.74399999999999999},"max":{"1min":2.0590000000000002,"5min":2.0590000000000002,"15min":2.621},"last":1.006}]}]},{"osd":1,"up_from":13,"seq":55834574961,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":978944,"data_stored":464531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:23:27 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.80100000000000005,"15min":0.80000000000000004},"min":{"1min":0.45000000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.081,"5min":1.377,"15min":2.8279999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.81999999999999995,"15min":0.83999999999999997},"min":{"1min":0.57899999999999996,"5min":0.34799999999999998,"15min":0.312},"max":{"1min":1.147,"5min":1.4390000000000001,"15min":1.8919999999999999},"last":0.91000000000000003}]},{"osd":2,"last update":"Sat Sep 23 03:23:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.88700000000000001,"15min":0.88600000000000001},"min":{"1min":0.52500000000000002,"5min":0.503,"15min":0.36699999999999999},"max":{"1min":1.107,"5min":1.536,"15min":2.3340000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.0449999999999999,"15min":1.0309999999999999},"min":{"1min":0.61199999999999999,"5min":0.54500000000000004,"15min":0.45600000000000002},"max":{"1min":1.3140000000000001,"5min":1.6499999999999999,"15min":1.9570000000000001},"last":1.1739999999999999}]},{"osd":3,"last update":"Sat Sep 23 03:23:27 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":1.01,"15min":1.0569999999999999},"min":{"1min":0.65800000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.413,"5min":1.595,"15min":2.2370000000000001},"last":1.022},{"interface":"front","average":{"1min":1.008,"5min":1.0760000000000001,"15min":1.127},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.45900000000000002},"max":{"1min":1.6339999999999999,"5min":1.7649999999999999,"15min":3.2130000000000001},"last":0.85399999999999998}]},{"osd":4,"last update":"Sat Sep 23 03:23:27 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.216,"15min":1.2589999999999999},"min":{"1min":0.88700000000000001,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.476,"5min":1.7270000000000001,"15min":3.2629999999999999},"last":1.119},{"interface":"front","average":{"1min":1.266,"5min":1.2350000000000001,"15min":1.278},"min":{"1min":0.92200000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":2.4689999999999999},"last":1.2629999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:23:27 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.2669999999999999,"15min":1.2889999999999999},"min":{"1min":0.92400000000000004,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.657,"5min":1.9319999999999999,"15min":3.161},"last":1.2},{"interface":"front","average":{"1min":1.325,"5min":1.335,"15min":1.339},"min":{"1min":0.998,"5min":0.80400000000000005,"15min":0.72399999999999998},"max":{"1min":1.7929999999999999,"5min":1.9890000000000001,"15min":2.4289999999999998},"last":1.343}]},{"osd":6,"last update":"Sat Sep 23 03:23:27 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.3169999999999999,"15min":1.3280000000000001},"min":{"1min":1.0720000000000001,"5min":0.76500000000000001,"15min":0.747},"max":{"1min":1.667,"5min":2.0329999999999999,"15min":3.0179999999999998},"last":1.411},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.357,"15min":1.383},"min":{"1min":0.96099999999999997,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.694,"5min":2.0720000000000001,"15min":3.3580000000000001},"last":1.4470000000000001}]},{"osd":7,"last update":"Sat Sep 23 03:23:27 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.3740000000000001,"15min":1.393},"min":{"1min":1.0760000000000001,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.669,"5min":2.085,"15min":3.1099999999999999},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.391,"5min":1.425,"15min":1.4550000000000001},"min":{"1min":1.008,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.657,"5min":1.9419999999999999,"15min":3.306},"last":1.0669999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574961,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32712,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1568768,"data_stored":1053880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 23 03:23:22 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.88600000000000001,"15min":0.83999999999999997},"min":{"1min":0.45800000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.2629999999999999,"5min":1.919,"15min":1.919},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.90000000000000002,"15min":0.89400000000000002},"min":{"1min":0.56200000000000006,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.139,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.89100000000000001}]},{"osd":2,"last update":"Sat Sep 23 03:23:22 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.89600000000000002,"15min":0.82899999999999996},"min":{"1min":0.55000000000000004,"5min":0.48599999999999999,"15min":0.46400000000000002},"max":{"1min":1.327,"5min":3.089,"15min":3.089},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.90300000000000002,"5min":1.0449999999999999,"15min":1.0049999999999999},"min":{"1min":0.61299999999999999,"5min":0.46700000000000003,"15min":0.42499999999999999},"max":{"1min":1.238,"5min":2.633,"15min":2.633},"last":0.81499999999999995}]},{"osd":3,"last update":"Sat Sep 23 03:23:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0289999999999999,"15min":0.95399999999999996},"min":{"1min":0.61499999999999999,"5min":0.44800000000000001,"15min":0.44700000000000001},"max":{"1min":1.45,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.0489999999999999,"15min":1.012},"min":{"1min":0.55400000000000005,"5min":0.53500000000000003,"15min":0.51900000000000002},"max":{"1min":1.5,"5min":2.012,"15min":2.012},"last":1.4630000000000001}]},{"osd":4,"last update":"Sat Sep 23 03:23:22 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.333,"15min":1.246},"min":{"1min":0.82899999999999996,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.7869999999999999,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":1.885},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.294,"15min":1.2829999999999999},"min":{"1min":0.86299999999999999,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.976,"5min":3.0230000000000001,"15min":3.0230000000000001},"last":1.321}]},{"osd":5,"last update":"Sat Sep 23 03:23:22 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.319,"15min":1.2529999999999999},"min":{"1min":1.0289999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.859,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":1.6220000000000001},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.409,"15min":1.3140000000000001},"min":{"1min":1.0860000000000001,"5min":0.75700000000000001,"15min":0.71999999999999997},"max":{"1min":1.772,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.722}]},{"osd":6,"last update":"Sat Sep 23 03:23:22 2023","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.3779999999999999,"15min":1.3220000000000001},"min":{"1min":0.95599999999999996,"5min":0.77400000000000002,"15min":0.75600000000000001},"max":{"1min":1.9299999999999999,"5min":2.9489999999999998,"15min":2.9489999999999998},"last":1.5640000000000001},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.4690000000000001,"15min":1.3560000000000001},"min":{"1min":1.1140000000000001,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.8300000000000001,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.786}]},{"osd":7,"last update":"Sat Sep 23 03:23:22 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.4319999999999999,"15min":1.353},"min":{"1min":1.145,"5min":0.90500000000000003,"15min":0.747},"max":{"1min":1.8240000000000001,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":1.236},{"interface":"front","average":{"1min":1.52,"5min":1.4990000000000001,"15min":1.4379999999999999},"min":{"1min":1.196,"5min":0.91800000000000004,"15min":0.81699999999999995},"max":{"1min":2.0390000000000001,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":1.5389999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-23T03:24:15.094 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-23T03:24:15.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.342+0000 7f4cdb324640 1 Processor -- start 2023-09-23T03:24:15.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.342+0000 7f4cdb324640 1 -- start start 2023-09-23T03:24:15.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.343+0000 7f4cdb324640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4cd415cac0 0x7f4cd406b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:15.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.343+0000 7f4cdb324640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4cd40cf2a0 0x7f4cd40cf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:15.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.344+0000 7f4cdb324640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4cd40cd670 0x7f4cd40cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:15.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.344+0000 7f4cdb324640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cd40ce990 con 0x7f4cd415cac0 2023-09-23T03:24:15.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.344+0000 7f4cdb324640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cd40cfc20 con 0x7f4cd40cd670 2023-09-23T03:24:15.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.344+0000 7f4cdb324640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd40ce3d0 con 0x7f4cd40cf2a0 2023-09-23T03:24:15.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.344+0000 7f4cd90c0640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4cd415cac0 0x7f4cd406b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:15.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.344+0000 7f4cd90c0640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4cd415cac0 0x7f4cd406b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50820/0 (socket says 172.21.15.154:50820) 2023-09-23T03:24:15.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.344+0000 7f4cd90c0640 1 -- 172.21.15.154:0/1783384189 learned_addr learned my addr 172.21.15.154:0/1783384189 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:15.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.344+0000 7f4cd90c0640 1 -- 172.21.15.154:0/1783384189 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4cd40cf2a0 msgr2=0x7f4cd40cf6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:15.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.344+0000 7f4cd90c0640 1 --2- 172.21.15.154:0/1783384189 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4cd40cf2a0 0x7f4cd40cf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:15.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.344+0000 7f4cd90c0640 1 -- 172.21.15.154:0/1783384189 >> v2:172.21.15.190:3300/0 conn(0x7f4cd40cd670 msgr2=0x7f4cd40cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:15.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.344+0000 7f4cd90c0640 1 --2- 172.21.15.154:0/1783384189 >> v2:172.21.15.190:3300/0 conn(0x7f4cd40cd670 0x7f4cd40cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:15.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.344+0000 7f4cd90c0640 1 -- 172.21.15.154:0/1783384189 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd40ce6b0 con 0x7f4cd415cac0 2023-09-23T03:24:15.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.345+0000 7f4cd90c0640 1 --2- 172.21.15.154:0/1783384189 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cd415cac0 0x7f4cd406b280 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f4cd406cd70 tx=0x7f4cd002bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=597ad1e236e8f55b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:15.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.346+0000 7f4ccbfff640 1 -- 172.21.15.154:0/1783384189 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd002cac0 con 0x7f4cd415cac0 2023-09-23T03:24:15.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.346+0000 7f4ccbfff640 1 -- 172.21.15.154:0/1783384189 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd002cc20 con 0x7f4cd415cac0 2023-09-23T03:24:15.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.346+0000 7f4ccbfff640 1 -- 172.21.15.154:0/1783384189 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd00343f0 con 0x7f4cd415cac0 2023-09-23T03:24:15.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.346+0000 7f4cdb324640 1 -- 172.21.15.154:0/1783384189 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cd415cac0 msgr2=0x7f4cd406b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:15.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.346+0000 7f4cdb324640 1 --2- 172.21.15.154:0/1783384189 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cd415cac0 0x7f4cd406b280 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f4cd406cd70 tx=0x7f4cd002bf20 comp rx=0 tx=0).stop 2023-09-23T03:24:15.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.346+0000 7f4cdb324640 1 -- 172.21.15.154:0/1783384189 shutdown_connections 2023-09-23T03:24:15.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.346+0000 7f4cdb324640 1 --2- 172.21.15.154:0/1783384189 >> v2:172.21.15.190:3300/0 conn(0x7f4cd40cd670 0x7f4cd40cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:15.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.346+0000 7f4cdb324640 1 --2- 172.21.15.154:0/1783384189 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4cd40cf2a0 0x7f4cd40cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:15.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.346+0000 7f4cdb324640 1 --2- 172.21.15.154:0/1783384189 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cd415cac0 0x7f4cd406b280 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:15.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.346+0000 7f4cdb324640 1 -- 172.21.15.154:0/1783384189 >> 172.21.15.154:0/1783384189 conn(0x7f4cd4089fd0 msgr2=0x7f4cd409bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:15.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.346+0000 7f4cdb324640 1 -- 172.21.15.154:0/1783384189 shutdown_connections 2023-09-23T03:24:15.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.346+0000 7f4cdb324640 1 -- 172.21.15.154:0/1783384189 wait complete. 2023-09-23T03:24:15.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.348+0000 7f4cdb324640 1 Processor -- start 2023-09-23T03:24:15.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.348+0000 7f4cdb324640 1 -- start start 2023-09-23T03:24:15.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.349+0000 7f4cdb324640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cd40cd670 0x7f4cd40b77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:15.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.349+0000 7f4cdb324640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4cd40cf2a0 0x7f4cd40c6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:15.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.349+0000 7f4cdb324640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4cd415cac0 0x7f4cd40c9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:15.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.349+0000 7f4cdb324640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd40b4b80 con 0x7f4cd40cd670 2023-09-23T03:24:15.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.349+0000 7f4cdb324640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd41572f0 con 0x7f4cd415cac0 2023-09-23T03:24:15.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.349+0000 7f4cdb324640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd40b3f00 con 0x7f4cd40cf2a0 2023-09-23T03:24:15.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.349+0000 7f4cd90c0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cd40cd670 0x7f4cd40b77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:15.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.349+0000 7f4cd90c0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cd40cd670 0x7f4cd40b77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50836/0 (socket says 172.21.15.154:50836) 2023-09-23T03:24:15.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.349+0000 7f4cd90c0640 1 -- 172.21.15.154:0/2771111813 learned_addr learned my addr 172.21.15.154:0/2771111813 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:15.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.350+0000 7f4cd90c0640 1 -- 172.21.15.154:0/2771111813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4cd40cf2a0 msgr2=0x7f4cd40c6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:15.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.350+0000 7f4cd90c0640 1 --2- 172.21.15.154:0/2771111813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4cd40cf2a0 0x7f4cd40c6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:15.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.350+0000 7f4cd90c0640 1 -- 172.21.15.154:0/2771111813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4cd415cac0 msgr2=0x7f4cd40c9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:15.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.350+0000 7f4cd90c0640 1 --2- 172.21.15.154:0/2771111813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4cd415cac0 0x7f4cd40c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:15.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.350+0000 7f4cd90c0640 1 -- 172.21.15.154:0/2771111813 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd0009b30 con 0x7f4cd40cd670 2023-09-23T03:24:15.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.350+0000 7f4cd90c0640 1 --2- 172.21.15.154:0/2771111813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cd40cd670 0x7f4cd40b77d0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f4cd00093e0 tx=0x7f4cd0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:15.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.351+0000 7f4cc9ffb640 1 -- 172.21.15.154:0/2771111813 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd003e450 con 0x7f4cd40cd670 2023-09-23T03:24:15.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.351+0000 7f4cc9ffb640 1 -- 172.21.15.154:0/2771111813 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd003e5d0 con 0x7f4cd40cd670 2023-09-23T03:24:15.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.351+0000 7f4cc9ffb640 1 -- 172.21.15.154:0/2771111813 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd003e980 con 0x7f4cd40cd670 2023-09-23T03:24:15.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.351+0000 7f4cdb324640 1 -- 172.21.15.154:0/2771111813 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cd40b41e0 con 0x7f4cd40cd670 2023-09-23T03:24:15.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.351+0000 7f4cdb324640 1 -- 172.21.15.154:0/2771111813 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cd40b47a0 con 0x7f4cd40cd670 2023-09-23T03:24:15.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.352+0000 7f4cdb324640 1 -- 172.21.15.154:0/2771111813 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cd4073eb0 con 0x7f4cd40cd670 2023-09-23T03:24:15.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.353+0000 7f4cc9ffb640 1 -- 172.21.15.154:0/2771111813 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f4cd003eae0 con 0x7f4cd40cd670 2023-09-23T03:24:15.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.354+0000 7f4cc9ffb640 1 --2- 172.21.15.154:0/2771111813 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4cb806af30 0x7f4cb806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:15.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.354+0000 7f4cc9ffb640 1 -- 172.21.15.154:0/2771111813 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4cd00ac970 con 0x7f4cd40cd670 2023-09-23T03:24:15.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.354+0000 7f4cd88bf640 1 --2- 172.21.15.154:0/2771111813 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4cb806af30 0x7f4cb806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:15.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.354+0000 7f4cd88bf640 1 --2- 172.21.15.154:0/2771111813 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4cb806af30 0x7f4cb806d3f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4cd40c6900 tx=0x7f4cc4006c90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:15.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.358+0000 7f4cc9ffb640 1 -- 172.21.15.154:0/2771111813 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f4cd0077740 con 0x7f4cd40cd670 2023-09-23T03:24:15.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.546+0000 7f4cdb324640 1 -- 172.21.15.154:0/2771111813 --> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4cd40d0890 con 0x7f4cb806af30 2023-09-23T03:24:15.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.569+0000 7f4cc9ffb640 1 -- 172.21.15.154:0/2771111813 <== mgr.4110 v2:172.21.15.154:6832/2422652104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286435 (secure 0 0 0) 0x7f4cc4013400 con 0x7f4cb806af30 2023-09-23T03:24:15.574 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:15.575 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2023-09-23T03:24:15.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.574+0000 7f4cdb324640 1 -- 172.21.15.154:0/2771111813 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4cb806af30 msgr2=0x7f4cb806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:15.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.574+0000 7f4cdb324640 1 --2- 172.21.15.154:0/2771111813 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4cb806af30 0x7f4cb806d3f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4cd40c6900 tx=0x7f4cc4006c90 comp rx=0 tx=0).stop 2023-09-23T03:24:15.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.574+0000 7f4cdb324640 1 -- 172.21.15.154:0/2771111813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cd40cd670 msgr2=0x7f4cd40b77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:15.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.574+0000 7f4cdb324640 1 --2- 172.21.15.154:0/2771111813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cd40cd670 0x7f4cd40b77d0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f4cd00093e0 tx=0x7f4cd0004a40 comp rx=0 tx=0).stop 2023-09-23T03:24:15.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.575+0000 7f4cdb324640 1 -- 172.21.15.154:0/2771111813 shutdown_connections 2023-09-23T03:24:15.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.575+0000 7f4cdb324640 1 --2- 172.21.15.154:0/2771111813 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4cb806af30 0x7f4cb806d3f0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:15.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.575+0000 7f4cdb324640 1 --2- 172.21.15.154:0/2771111813 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4cd415cac0 0x7f4cd40c9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:15.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.575+0000 7f4cdb324640 1 --2- 172.21.15.154:0/2771111813 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4cd40cf2a0 0x7f4cd40c6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:15.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.575+0000 7f4cdb324640 1 --2- 172.21.15.154:0/2771111813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4cd40cd670 0x7f4cd40b77d0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:15.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.575+0000 7f4cdb324640 1 -- 172.21.15.154:0/2771111813 >> 172.21.15.154:0/2771111813 conn(0x7f4cd4089fd0 msgr2=0x7f4cd40b1490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:15.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.575+0000 7f4cdb324640 1 -- 172.21.15.154:0/2771111813 shutdown_connections 2023-09-23T03:24:15.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.575+0000 7f4cdb324640 1 -- 172.21.15.154:0/2771111813 wait complete. 2023-09-23T03:24:15.595 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2023-09-23T03:24:15.354833+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258612,"kb_used_data":9156,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716108,"statfs":{"total":773094113280,"available":772829294592,"internally_reserved":0,"allocated":9375744,"data_stored":5063265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":255315514},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001846"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.602877+0000","last_change":"2023-09-23T03:24:10.628373+0000","last_active":"2023-09-23T03:24:11.602877+0000","last_peered":"2023-09-23T03:24:11.602877+0000","last_clean":"2023-09-23T03:24:11.602877+0000","last_became_active":"2023-09-23T03:24:10.627205+0000","last_became_peered":"2023-09-23T03:24:10.627205+0000","last_unstale":"2023-09-23T03:24:11.602877+0000","last_undegraded":"2023-09-23T03:24:11.602877+0000","last_fullsized":"2023-09-23T03:24:11.602877+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:47:52.730063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.619193+0000","last_change":"2023-09-23T03:24:10.621346+0000","last_active":"2023-09-23T03:24:11.619193+0000","last_peered":"2023-09-23T03:24:11.619193+0000","last_clean":"2023-09-23T03:24:11.619193+0000","last_became_active":"2023-09-23T03:24:10.620390+0000","last_became_peered":"2023-09-23T03:24:10.620390+0000","last_unstale":"2023-09-23T03:24:11.619193+0000","last_undegraded":"2023-09-23T03:24:11.619193+0000","last_fullsized":"2023-09-23T03:24:11.619193+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:11:52.573631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:12.741487+0000","last_change":"2023-09-23T03:24:10.633282+0000","last_active":"2023-09-23T03:24:12.741487+0000","last_peered":"2023-09-23T03:24:12.741487+0000","last_clean":"2023-09-23T03:24:12.741487+0000","last_became_active":"2023-09-23T03:24:10.632987+0000","last_became_peered":"2023-09-23T03:24:10.632987+0000","last_unstale":"2023-09-23T03:24:12.741487+0000","last_undegraded":"2023-09-23T03:24:12.741487+0000","last_fullsized":"2023-09-23T03:24:12.741487+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:00:41.189671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:12.743462+0000","last_change":"2023-09-23T03:24:10.624519+0000","last_active":"2023-09-23T03:24:12.743462+0000","last_peered":"2023-09-23T03:24:12.743462+0000","last_clean":"2023-09-23T03:24:12.743462+0000","last_became_active":"2023-09-23T03:24:10.624221+0000","last_became_peered":"2023-09-23T03:24:10.624221+0000","last_unstale":"2023-09-23T03:24:12.743462+0000","last_undegraded":"2023-09-23T03:24:12.743462+0000","last_fullsized":"2023-09-23T03:24:12.743462+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:50:23.861373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.641756+0000","last_change":"2023-09-23T03:24:10.631095+0000","last_active":"2023-09-23T03:24:11.641756+0000","last_peered":"2023-09-23T03:24:11.641756+0000","last_clean":"2023-09-23T03:24:11.641756+0000","last_became_active":"2023-09-23T03:24:10.627182+0000","last_became_peered":"2023-09-23T03:24:10.627182+0000","last_unstale":"2023-09-23T03:24:11.641756+0000","last_undegraded":"2023-09-23T03:24:11.641756+0000","last_fullsized":"2023-09-23T03:24:11.641756+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:40:47.654169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.600915+0000","last_change":"2023-09-23T03:24:10.648138+0000","last_active":"2023-09-23T03:24:11.600915+0000","last_peered":"2023-09-23T03:24:11.600915+0000","last_clean":"2023-09-23T03:24:11.600915+0000","last_became_active":"2023-09-23T03:24:10.647677+0000","last_became_peered":"2023-09-23T03:24:10.647677+0000","last_unstale":"2023-09-23T03:24:11.600915+0000","last_undegraded":"2023-09-23T03:24:11.600915+0000","last_fullsized":"2023-09-23T03:24:11.600915+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:07:39.105141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.604917+0000","last_change":"2023-09-23T03:24:10.630162+0000","last_active":"2023-09-23T03:24:11.604917+0000","last_peered":"2023-09-23T03:24:11.604917+0000","last_clean":"2023-09-23T03:24:11.604917+0000","last_became_active":"2023-09-23T03:24:10.629851+0000","last_became_peered":"2023-09-23T03:24:10.629851+0000","last_unstale":"2023-09-23T03:24:11.604917+0000","last_undegraded":"2023-09-23T03:24:11.604917+0000","last_fullsized":"2023-09-23T03:24:11.604917+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:22:19.837319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.641078+0000","last_change":"2023-09-23T03:24:10.623179+0000","last_active":"2023-09-23T03:24:11.641078+0000","last_peered":"2023-09-23T03:24:11.641078+0000","last_clean":"2023-09-23T03:24:11.641078+0000","last_became_active":"2023-09-23T03:24:10.622902+0000","last_became_peered":"2023-09-23T03:24:10.622902+0000","last_unstale":"2023-09-23T03:24:11.641078+0000","last_undegraded":"2023-09-23T03:24:11.641078+0000","last_fullsized":"2023-09-23T03:24:11.641078+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:23:08.637838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:12.744534+0000","last_change":"2023-09-23T03:24:10.647930+0000","last_active":"2023-09-23T03:24:12.744534+0000","last_peered":"2023-09-23T03:24:12.744534+0000","last_clean":"2023-09-23T03:24:12.744534+0000","last_became_active":"2023-09-23T03:24:10.647608+0000","last_became_peered":"2023-09-23T03:24:10.647608+0000","last_unstale":"2023-09-23T03:24:12.744534+0000","last_undegraded":"2023-09-23T03:24:12.744534+0000","last_fullsized":"2023-09-23T03:24:12.744534+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:00:41.189671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:12.742795+0000","last_change":"2023-09-23T03:24:10.624256+0000","last_active":"2023-09-23T03:24:12.742795+0000","last_peered":"2023-09-23T03:24:12.742795+0000","last_clean":"2023-09-23T03:24:12.742795+0000","last_became_active":"2023-09-23T03:24:10.623949+0000","last_became_peered":"2023-09-23T03:24:10.623949+0000","last_unstale":"2023-09-23T03:24:12.742795+0000","last_undegraded":"2023-09-23T03:24:12.742795+0000","last_fullsized":"2023-09-23T03:24:12.742795+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:57:37.372085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.619191+0000","last_change":"2023-09-23T03:24:10.635663+0000","last_active":"2023-09-23T03:24:11.619191+0000","last_peered":"2023-09-23T03:24:11.619191+0000","last_clean":"2023-09-23T03:24:11.619191+0000","last_became_active":"2023-09-23T03:24:10.635371+0000","last_became_peered":"2023-09-23T03:24:10.635371+0000","last_unstale":"2023-09-23T03:24:11.619191+0000","last_undegraded":"2023-09-23T03:24:11.619191+0000","last_fullsized":"2023-09-23T03:24:11.619191+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:39:34.790190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:12.743500+0000","last_change":"2023-09-23T03:24:10.630219+0000","last_active":"2023-09-23T03:24:12.743500+0000","last_peered":"2023-09-23T03:24:12.743500+0000","last_clean":"2023-09-23T03:24:12.743500+0000","last_became_active":"2023-09-23T03:24:10.629909+0000","last_became_peered":"2023-09-23T03:24:10.629909+0000","last_unstale":"2023-09-23T03:24:12.743500+0000","last_undegraded":"2023-09-23T03:24:12.743500+0000","last_fullsized":"2023-09-23T03:24:12.743500+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:31:35.390459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.641931+0000","last_change":"2023-09-23T03:24:10.650876+0000","last_active":"2023-09-23T03:24:11.641931+0000","last_peered":"2023-09-23T03:24:11.641931+0000","last_clean":"2023-09-23T03:24:11.641931+0000","last_became_active":"2023-09-23T03:24:10.650572+0000","last_became_peered":"2023-09-23T03:24:10.650572+0000","last_unstale":"2023-09-23T03:24:11.641931+0000","last_undegraded":"2023-09-23T03:24:11.641931+0000","last_fullsized":"2023-09-23T03:24:11.641931+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:12:44.714263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.600940+0000","last_change":"2023-09-23T03:24:10.647949+0000","last_active":"2023-09-23T03:24:11.600940+0000","last_peered":"2023-09-23T03:24:11.600940+0000","last_clean":"2023-09-23T03:24:11.600940+0000","last_became_active":"2023-09-23T03:24:10.647454+0000","last_became_peered":"2023-09-23T03:24:10.647454+0000","last_unstale":"2023-09-23T03:24:11.600940+0000","last_undegraded":"2023-09-23T03:24:11.600940+0000","last_fullsized":"2023-09-23T03:24:11.600940+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:27:49.005576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.618872+0000","last_change":"2023-09-23T03:24:10.624609+0000","last_active":"2023-09-23T03:24:11.618872+0000","last_peered":"2023-09-23T03:24:11.618872+0000","last_clean":"2023-09-23T03:24:11.618872+0000","last_became_active":"2023-09-23T03:24:10.624308+0000","last_became_peered":"2023-09-23T03:24:10.624308+0000","last_unstale":"2023-09-23T03:24:11.618872+0000","last_undegraded":"2023-09-23T03:24:11.618872+0000","last_fullsized":"2023-09-23T03:24:11.618872+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:46:38.114645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.602137+0000","last_change":"2023-09-23T03:24:10.630268+0000","last_active":"2023-09-23T03:24:11.602137+0000","last_peered":"2023-09-23T03:24:11.602137+0000","last_clean":"2023-09-23T03:24:11.602137+0000","last_became_active":"2023-09-23T03:24:10.629961+0000","last_became_peered":"2023-09-23T03:24:10.629961+0000","last_unstale":"2023-09-23T03:24:11.602137+0000","last_undegraded":"2023-09-23T03:24:11.602137+0000","last_fullsized":"2023-09-23T03:24:11.602137+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:55:55.838696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.642316+0000","last_change":"2023-09-23T03:24:10.623345+0000","last_active":"2023-09-23T03:24:11.642316+0000","last_peered":"2023-09-23T03:24:11.642316+0000","last_clean":"2023-09-23T03:24:11.642316+0000","last_became_active":"2023-09-23T03:24:10.623080+0000","last_became_peered":"2023-09-23T03:24:10.623080+0000","last_unstale":"2023-09-23T03:24:11.642316+0000","last_undegraded":"2023-09-23T03:24:11.642316+0000","last_fullsized":"2023-09-23T03:24:11.642316+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:58:07.593331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:12.741488+0000","last_change":"2023-09-23T03:24:10.626829+0000","last_active":"2023-09-23T03:24:12.741488+0000","last_peered":"2023-09-23T03:24:12.741488+0000","last_clean":"2023-09-23T03:24:12.741488+0000","last_became_active":"2023-09-23T03:24:10.626509+0000","last_became_peered":"2023-09-23T03:24:10.626509+0000","last_unstale":"2023-09-23T03:24:12.741488+0000","last_undegraded":"2023-09-23T03:24:12.741488+0000","last_fullsized":"2023-09-23T03:24:12.741488+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:42:28.257850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.601543+0000","last_change":"2023-09-23T03:24:10.637617+0000","last_active":"2023-09-23T03:24:11.601543+0000","last_peered":"2023-09-23T03:24:11.601543+0000","last_clean":"2023-09-23T03:24:11.601543+0000","last_became_active":"2023-09-23T03:24:10.637258+0000","last_became_peered":"2023-09-23T03:24:10.637258+0000","last_unstale":"2023-09-23T03:24:11.601543+0000","last_undegraded":"2023-09-23T03:24:11.601543+0000","last_fullsized":"2023-09-23T03:24:11.601543+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:44:10.719896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.641788+0000","last_change":"2023-09-23T03:24:10.623904+0000","last_active":"2023-09-23T03:24:11.641788+0000","last_peered":"2023-09-23T03:24:11.641788+0000","last_clean":"2023-09-23T03:24:11.641788+0000","last_became_active":"2023-09-23T03:24:10.623612+0000","last_became_peered":"2023-09-23T03:24:10.623612+0000","last_unstale":"2023-09-23T03:24:11.641788+0000","last_undegraded":"2023-09-23T03:24:11.641788+0000","last_fullsized":"2023-09-23T03:24:11.641788+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:25:34.151156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:12.743224+0000","last_change":"2023-09-23T03:24:10.628256+0000","last_active":"2023-09-23T03:24:12.743224+0000","last_peered":"2023-09-23T03:24:12.743224+0000","last_clean":"2023-09-23T03:24:12.743224+0000","last_became_active":"2023-09-23T03:24:10.627971+0000","last_became_peered":"2023-09-23T03:24:10.627971+0000","last_unstale":"2023-09-23T03:24:12.743224+0000","last_undegraded":"2023-09-23T03:24:12.743224+0000","last_fullsized":"2023-09-23T03:24:12.743224+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:09:46.315596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.641016+0000","last_change":"2023-09-23T03:24:10.651166+0000","last_active":"2023-09-23T03:24:11.641016+0000","last_peered":"2023-09-23T03:24:11.641016+0000","last_clean":"2023-09-23T03:24:11.641016+0000","last_became_active":"2023-09-23T03:24:10.650390+0000","last_became_peered":"2023-09-23T03:24:10.650390+0000","last_unstale":"2023-09-23T03:24:11.641016+0000","last_undegraded":"2023-09-23T03:24:11.641016+0000","last_fullsized":"2023-09-23T03:24:11.641016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:34:43.304464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:12.742467+0000","last_change":"2023-09-23T03:24:10.624759+0000","last_active":"2023-09-23T03:24:12.742467+0000","last_peered":"2023-09-23T03:24:12.742467+0000","last_clean":"2023-09-23T03:24:12.742467+0000","last_became_active":"2023-09-23T03:24:10.624478+0000","last_became_peered":"2023-09-23T03:24:10.624478+0000","last_unstale":"2023-09-23T03:24:12.742467+0000","last_undegraded":"2023-09-23T03:24:12.742467+0000","last_fullsized":"2023-09-23T03:24:12.742467+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:02:16.748244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.618706+0000","last_change":"2023-09-23T03:24:10.634932+0000","last_active":"2023-09-23T03:24:11.618706+0000","last_peered":"2023-09-23T03:24:11.618706+0000","last_clean":"2023-09-23T03:24:11.618706+0000","last_became_active":"2023-09-23T03:24:10.634663+0000","last_became_peered":"2023-09-23T03:24:10.634663+0000","last_unstale":"2023-09-23T03:24:11.618706+0000","last_undegraded":"2023-09-23T03:24:11.618706+0000","last_fullsized":"2023-09-23T03:24:11.618706+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:09:53.551798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.619562+0000","last_change":"2023-09-23T03:24:10.647802+0000","last_active":"2023-09-23T03:24:11.619562+0000","last_peered":"2023-09-23T03:24:11.619562+0000","last_clean":"2023-09-23T03:24:11.619562+0000","last_became_active":"2023-09-23T03:24:10.647515+0000","last_became_peered":"2023-09-23T03:24:10.647515+0000","last_unstale":"2023-09-23T03:24:11.619562+0000","last_undegraded":"2023-09-23T03:24:11.619562+0000","last_fullsized":"2023-09-23T03:24:11.619562+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:18:22.755023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:12.744138+0000","last_change":"2023-09-23T03:24:10.630209+0000","last_active":"2023-09-23T03:24:12.744138+0000","last_peered":"2023-09-23T03:24:12.744138+0000","last_clean":"2023-09-23T03:24:12.744138+0000","last_became_active":"2023-09-23T03:24:10.629913+0000","last_became_peered":"2023-09-23T03:24:10.629913+0000","last_unstale":"2023-09-23T03:24:12.744138+0000","last_undegraded":"2023-09-23T03:24:12.744138+0000","last_fullsized":"2023-09-23T03:24:12.744138+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:17:44.867044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.603281+0000","last_change":"2023-09-23T03:24:10.628176+0000","last_active":"2023-09-23T03:24:11.603281+0000","last_peered":"2023-09-23T03:24:11.603281+0000","last_clean":"2023-09-23T03:24:11.603281+0000","last_became_active":"2023-09-23T03:24:10.627864+0000","last_became_peered":"2023-09-23T03:24:10.627864+0000","last_unstale":"2023-09-23T03:24:11.603281+0000","last_undegraded":"2023-09-23T03:24:11.603281+0000","last_fullsized":"2023-09-23T03:24:11.603281+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:09:53.551798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.641712+0000","last_change":"2023-09-23T03:24:10.629122+0000","last_active":"2023-09-23T03:24:11.641712+0000","last_peered":"2023-09-23T03:24:11.641712+0000","last_clean":"2023-09-23T03:24:11.641712+0000","last_became_active":"2023-09-23T03:24:10.628800+0000","last_became_peered":"2023-09-23T03:24:10.628800+0000","last_unstale":"2023-09-23T03:24:11.641712+0000","last_undegraded":"2023-09-23T03:24:11.641712+0000","last_fullsized":"2023-09-23T03:24:11.641712+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:58:53.138465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.618786+0000","last_change":"2023-09-23T03:24:10.615962+0000","last_active":"2023-09-23T03:24:11.618786+0000","last_peered":"2023-09-23T03:24:11.618786+0000","last_clean":"2023-09-23T03:24:11.618786+0000","last_became_active":"2023-09-23T03:24:10.615613+0000","last_became_peered":"2023-09-23T03:24:10.615613+0000","last_unstale":"2023-09-23T03:24:11.618786+0000","last_undegraded":"2023-09-23T03:24:11.618786+0000","last_fullsized":"2023-09-23T03:24:11.618786+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:07:39.105141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.602436+0000","last_change":"2023-09-23T03:24:10.633764+0000","last_active":"2023-09-23T03:24:11.602436+0000","last_peered":"2023-09-23T03:24:11.602436+0000","last_clean":"2023-09-23T03:24:11.602436+0000","last_became_active":"2023-09-23T03:24:10.633384+0000","last_became_peered":"2023-09-23T03:24:10.633384+0000","last_unstale":"2023-09-23T03:24:11.602436+0000","last_undegraded":"2023-09-23T03:24:11.602436+0000","last_fullsized":"2023-09-23T03:24:11.602436+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:42:42.691073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:12.741299+0000","last_change":"2023-09-23T03:24:10.625904+0000","last_active":"2023-09-23T03:24:12.741299+0000","last_peered":"2023-09-23T03:24:12.741299+0000","last_clean":"2023-09-23T03:24:12.741299+0000","last_became_active":"2023-09-23T03:24:10.625617+0000","last_became_peered":"2023-09-23T03:24:10.625617+0000","last_unstale":"2023-09-23T03:24:12.741299+0000","last_undegraded":"2023-09-23T03:24:12.741299+0000","last_fullsized":"2023-09-23T03:24:12.741299+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:43:55.950015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.602221+0000","last_change":"2023-09-23T03:24:10.630300+0000","last_active":"2023-09-23T03:24:11.602221+0000","last_peered":"2023-09-23T03:24:11.602221+0000","last_clean":"2023-09-23T03:24:11.602221+0000","last_became_active":"2023-09-23T03:24:10.630043+0000","last_became_peered":"2023-09-23T03:24:10.630043+0000","last_unstale":"2023-09-23T03:24:11.602221+0000","last_undegraded":"2023-09-23T03:24:11.602221+0000","last_fullsized":"2023-09-23T03:24:11.602221+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:59:53.324438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:12.741412+0000","last_change":"2023-09-23T03:24:10.626713+0000","last_active":"2023-09-23T03:24:12.741412+0000","last_peered":"2023-09-23T03:24:12.741412+0000","last_clean":"2023-09-23T03:24:12.741412+0000","last_became_active":"2023-09-23T03:24:10.626402+0000","last_became_peered":"2023-09-23T03:24:10.626402+0000","last_unstale":"2023-09-23T03:24:12.741412+0000","last_undegraded":"2023-09-23T03:24:12.741412+0000","last_fullsized":"2023-09-23T03:24:12.741412+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:42:05.028735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"82'4","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T03:24:10.651584+0000","last_change":"2023-09-23T03:24:09.618168+0000","last_active":"2023-09-23T03:24:10.651584+0000","last_peered":"2023-09-23T03:24:10.651584+0000","last_clean":"2023-09-23T03:24:10.651584+0000","last_became_active":"2023-09-23T03:24:09.617890+0000","last_became_peered":"2023-09-23T03:24:09.617890+0000","last_unstale":"2023-09-23T03:24:10.651584+0000","last_undegraded":"2023-09-23T03:24:10.651584+0000","last_fullsized":"2023-09-23T03:24:10.651584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:57:18.929554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.618760+0000","last_change":"2023-09-23T03:24:10.619879+0000","last_active":"2023-09-23T03:24:11.618760+0000","last_peered":"2023-09-23T03:24:11.618760+0000","last_clean":"2023-09-23T03:24:11.618760+0000","last_became_active":"2023-09-23T03:24:10.619535+0000","last_became_peered":"2023-09-23T03:24:10.619535+0000","last_unstale":"2023-09-23T03:24:11.618760+0000","last_undegraded":"2023-09-23T03:24:11.618760+0000","last_fullsized":"2023-09-23T03:24:11.618760+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:28:19.431732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T03:24:10.641081+0000","last_change":"2023-09-23T03:24:09.630210+0000","last_active":"2023-09-23T03:24:10.641081+0000","last_peered":"2023-09-23T03:24:10.641081+0000","last_clean":"2023-09-23T03:24:10.641081+0000","last_became_active":"2023-09-23T03:24:09.629872+0000","last_became_peered":"2023-09-23T03:24:09.629872+0000","last_unstale":"2023-09-23T03:24:10.641081+0000","last_undegraded":"2023-09-23T03:24:10.641081+0000","last_fullsized":"2023-09-23T03:24:10.641081+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:27:04.141923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:12.742714+0000","last_change":"2023-09-23T03:24:10.648198+0000","last_active":"2023-09-23T03:24:12.742714+0000","last_peered":"2023-09-23T03:24:12.742714+0000","last_clean":"2023-09-23T03:24:12.742714+0000","last_became_active":"2023-09-23T03:24:10.647746+0000","last_became_peered":"2023-09-23T03:24:10.647746+0000","last_unstale":"2023-09-23T03:24:12.742714+0000","last_undegraded":"2023-09-23T03:24:12.742714+0000","last_fullsized":"2023-09-23T03:24:12.742714+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:16:06.466323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T03:24:10.642346+0000","last_change":"2023-09-23T03:24:09.626670+0000","last_active":"2023-09-23T03:24:10.642346+0000","last_peered":"2023-09-23T03:24:10.642346+0000","last_clean":"2023-09-23T03:24:10.642346+0000","last_became_active":"2023-09-23T03:24:09.625967+0000","last_became_peered":"2023-09-23T03:24:09.625967+0000","last_unstale":"2023-09-23T03:24:10.642346+0000","last_undegraded":"2023-09-23T03:24:10.642346+0000","last_fullsized":"2023-09-23T03:24:10.642346+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:01:47.705432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:12.743409+0000","last_change":"2023-09-23T03:24:10.628235+0000","last_active":"2023-09-23T03:24:12.743409+0000","last_peered":"2023-09-23T03:24:12.743409+0000","last_clean":"2023-09-23T03:24:12.743409+0000","last_became_active":"2023-09-23T03:24:10.627946+0000","last_became_peered":"2023-09-23T03:24:10.627946+0000","last_unstale":"2023-09-23T03:24:12.743409+0000","last_undegraded":"2023-09-23T03:24:12.743409+0000","last_fullsized":"2023-09-23T03:24:12.743409+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:43:55.950015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T03:24:10.643768+0000","last_change":"2023-09-23T03:24:09.623911+0000","last_active":"2023-09-23T03:24:10.643768+0000","last_peered":"2023-09-23T03:24:10.643768+0000","last_clean":"2023-09-23T03:24:10.643768+0000","last_became_active":"2023-09-23T03:24:09.623610+0000","last_became_peered":"2023-09-23T03:24:09.623610+0000","last_unstale":"2023-09-23T03:24:10.643768+0000","last_undegraded":"2023-09-23T03:24:10.643768+0000","last_fullsized":"2023-09-23T03:24:10.643768+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:37:57.025987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.618829+0000","last_change":"2023-09-23T03:24:10.628206+0000","last_active":"2023-09-23T03:24:11.618829+0000","last_peered":"2023-09-23T03:24:11.618829+0000","last_clean":"2023-09-23T03:24:11.618829+0000","last_became_active":"2023-09-23T03:24:10.627941+0000","last_became_peered":"2023-09-23T03:24:10.627941+0000","last_unstale":"2023-09-23T03:24:11.618829+0000","last_undegraded":"2023-09-23T03:24:11.618829+0000","last_fullsized":"2023-09-23T03:24:11.618829+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:27:49.005576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T03:24:10.640096+0000","last_change":"2023-09-23T03:24:09.626230+0000","last_active":"2023-09-23T03:24:10.640096+0000","last_peered":"2023-09-23T03:24:10.640096+0000","last_clean":"2023-09-23T03:24:10.640096+0000","last_became_active":"2023-09-23T03:24:09.625906+0000","last_became_peered":"2023-09-23T03:24:09.625906+0000","last_unstale":"2023-09-23T03:24:10.640096+0000","last_undegraded":"2023-09-23T03:24:10.640096+0000","last_fullsized":"2023-09-23T03:24:10.640096+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:55:39.471977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.619383+0000","last_change":"2023-09-23T03:24:10.621041+0000","last_active":"2023-09-23T03:24:11.619383+0000","last_peered":"2023-09-23T03:24:11.619383+0000","last_clean":"2023-09-23T03:24:11.619383+0000","last_became_active":"2023-09-23T03:24:10.620774+0000","last_became_peered":"2023-09-23T03:24:10.620774+0000","last_unstale":"2023-09-23T03:24:11.619383+0000","last_undegraded":"2023-09-23T03:24:11.619383+0000","last_fullsized":"2023-09-23T03:24:11.619383+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:54:22.321881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T03:24:10.650095+0000","last_change":"2023-09-23T03:24:09.625217+0000","last_active":"2023-09-23T03:24:10.650095+0000","last_peered":"2023-09-23T03:24:10.650095+0000","last_clean":"2023-09-23T03:24:10.650095+0000","last_became_active":"2023-09-23T03:24:09.624756+0000","last_became_peered":"2023-09-23T03:24:09.624756+0000","last_unstale":"2023-09-23T03:24:10.650095+0000","last_undegraded":"2023-09-23T03:24:10.650095+0000","last_fullsized":"2023-09-23T03:24:10.650095+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:15:33.296587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:12.744174+0000","last_change":"2023-09-23T03:24:10.647849+0000","last_active":"2023-09-23T03:24:12.744174+0000","last_peered":"2023-09-23T03:24:12.744174+0000","last_clean":"2023-09-23T03:24:12.744174+0000","last_became_active":"2023-09-23T03:24:10.647537+0000","last_became_peered":"2023-09-23T03:24:10.647537+0000","last_unstale":"2023-09-23T03:24:12.744174+0000","last_undegraded":"2023-09-23T03:24:12.744174+0000","last_fullsized":"2023-09-23T03:24:12.744174+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:42:05.028735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T03:24:10.598122+0000","last_change":"2023-09-23T03:24:09.614441+0000","last_active":"2023-09-23T03:24:10.598122+0000","last_peered":"2023-09-23T03:24:10.598122+0000","last_clean":"2023-09-23T03:24:10.598122+0000","last_became_active":"2023-09-23T03:24:09.614141+0000","last_became_peered":"2023-09-23T03:24:09.614141+0000","last_unstale":"2023-09-23T03:24:10.598122+0000","last_undegraded":"2023-09-23T03:24:10.598122+0000","last_fullsized":"2023-09-23T03:24:10.598122+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:01:47.705432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.606184+0000","last_change":"2023-09-23T03:24:10.629124+0000","last_active":"2023-09-23T03:24:11.606184+0000","last_peered":"2023-09-23T03:24:11.606184+0000","last_clean":"2023-09-23T03:24:11.606184+0000","last_became_active":"2023-09-23T03:24:10.628851+0000","last_became_peered":"2023-09-23T03:24:10.628851+0000","last_unstale":"2023-09-23T03:24:11.606184+0000","last_undegraded":"2023-09-23T03:24:11.606184+0000","last_fullsized":"2023-09-23T03:24:11.606184+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:27:05.158518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T03:24:10.609195+0000","last_change":"2023-09-23T03:24:09.628674+0000","last_active":"2023-09-23T03:24:10.609195+0000","last_peered":"2023-09-23T03:24:10.609195+0000","last_clean":"2023-09-23T03:24:10.609195+0000","last_became_active":"2023-09-23T03:24:09.628382+0000","last_became_peered":"2023-09-23T03:24:09.628382+0000","last_unstale":"2023-09-23T03:24:10.609195+0000","last_undegraded":"2023-09-23T03:24:10.609195+0000","last_fullsized":"2023-09-23T03:24:10.609195+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:51:58.042054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.619257+0000","last_change":"2023-09-23T03:24:10.629371+0000","last_active":"2023-09-23T03:24:11.619257+0000","last_peered":"2023-09-23T03:24:11.619257+0000","last_clean":"2023-09-23T03:24:11.619257+0000","last_became_active":"2023-09-23T03:24:10.629099+0000","last_became_peered":"2023-09-23T03:24:10.629099+0000","last_unstale":"2023-09-23T03:24:11.619257+0000","last_undegraded":"2023-09-23T03:24:11.619257+0000","last_fullsized":"2023-09-23T03:24:11.619257+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:28:36.312059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T03:24:10.619018+0000","last_change":"2023-09-23T03:24:09.631273+0000","last_active":"2023-09-23T03:24:10.619018+0000","last_peered":"2023-09-23T03:24:10.619018+0000","last_clean":"2023-09-23T03:24:10.619018+0000","last_became_active":"2023-09-23T03:24:09.630968+0000","last_became_peered":"2023-09-23T03:24:09.630968+0000","last_unstale":"2023-09-23T03:24:10.619018+0000","last_undegraded":"2023-09-23T03:24:10.619018+0000","last_fullsized":"2023-09-23T03:24:10.619018+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:17:43.850449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:12.743004+0000","last_change":"2023-09-23T03:24:10.628113+0000","last_active":"2023-09-23T03:24:12.743004+0000","last_peered":"2023-09-23T03:24:12.743004+0000","last_clean":"2023-09-23T03:24:12.743004+0000","last_became_active":"2023-09-23T03:24:10.627856+0000","last_became_peered":"2023-09-23T03:24:10.627856+0000","last_unstale":"2023-09-23T03:24:12.743004+0000","last_undegraded":"2023-09-23T03:24:12.743004+0000","last_fullsized":"2023-09-23T03:24:12.743004+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:17:18.981524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T03:24:10.614989+0000","last_change":"2023-09-23T03:24:09.625645+0000","last_active":"2023-09-23T03:24:10.614989+0000","last_peered":"2023-09-23T03:24:10.614989+0000","last_clean":"2023-09-23T03:24:10.614989+0000","last_became_active":"2023-09-23T03:24:09.625325+0000","last_became_peered":"2023-09-23T03:24:09.625325+0000","last_unstale":"2023-09-23T03:24:10.614989+0000","last_undegraded":"2023-09-23T03:24:10.614989+0000","last_fullsized":"2023-09-23T03:24:10.614989+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:01:47.705432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.601115+0000","last_change":"2023-09-23T03:24:10.647948+0000","last_active":"2023-09-23T03:24:11.601115+0000","last_peered":"2023-09-23T03:24:11.601115+0000","last_clean":"2023-09-23T03:24:11.601115+0000","last_became_active":"2023-09-23T03:24:10.647333+0000","last_became_peered":"2023-09-23T03:24:10.647333+0000","last_unstale":"2023-09-23T03:24:11.601115+0000","last_undegraded":"2023-09-23T03:24:11.601115+0000","last_fullsized":"2023-09-23T03:24:11.601115+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:46:38.114645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"82'3","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T03:24:10.651384+0000","last_change":"2023-09-23T03:24:09.611434+0000","last_active":"2023-09-23T03:24:10.651384+0000","last_peered":"2023-09-23T03:24:10.651384+0000","last_clean":"2023-09-23T03:24:10.651384+0000","last_became_active":"2023-09-23T03:24:09.611165+0000","last_became_peered":"2023-09-23T03:24:09.611165+0000","last_unstale":"2023-09-23T03:24:10.651384+0000","last_undegraded":"2023-09-23T03:24:10.651384+0000","last_fullsized":"2023-09-23T03:24:10.651384+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:26:56.162449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:12.741834+0000","last_change":"2023-09-23T03:24:10.639215+0000","last_active":"2023-09-23T03:24:12.741834+0000","last_peered":"2023-09-23T03:24:12.741834+0000","last_clean":"2023-09-23T03:24:12.741834+0000","last_became_active":"2023-09-23T03:24:10.638639+0000","last_became_peered":"2023-09-23T03:24:10.638639+0000","last_unstale":"2023-09-23T03:24:12.741834+0000","last_undegraded":"2023-09-23T03:24:12.741834+0000","last_fullsized":"2023-09-23T03:24:12.741834+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:51:51.428693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"82'6","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T03:24:10.649757+0000","last_change":"2023-09-23T03:24:09.619477+0000","last_active":"2023-09-23T03:24:10.649757+0000","last_peered":"2023-09-23T03:24:10.649757+0000","last_clean":"2023-09-23T03:24:10.649757+0000","last_became_active":"2023-09-23T03:24:09.619209+0000","last_became_peered":"2023-09-23T03:24:09.619209+0000","last_unstale":"2023-09-23T03:24:10.649757+0000","last_undegraded":"2023-09-23T03:24:10.649757+0000","last_fullsized":"2023-09-23T03:24:10.649757+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:17:05.233886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:12.743825+0000","last_change":"2023-09-23T03:24:10.648103+0000","last_active":"2023-09-23T03:24:12.743825+0000","last_peered":"2023-09-23T03:24:12.743825+0000","last_clean":"2023-09-23T03:24:12.743825+0000","last_became_active":"2023-09-23T03:24:10.647806+0000","last_became_peered":"2023-09-23T03:24:10.647806+0000","last_unstale":"2023-09-23T03:24:12.743825+0000","last_undegraded":"2023-09-23T03:24:12.743825+0000","last_fullsized":"2023-09-23T03:24:12.743825+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:24:24.925081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T03:24:10.642547+0000","last_change":"2023-09-23T03:24:09.627198+0000","last_active":"2023-09-23T03:24:10.642547+0000","last_peered":"2023-09-23T03:24:10.642547+0000","last_clean":"2023-09-23T03:24:10.642547+0000","last_became_active":"2023-09-23T03:24:09.626867+0000","last_became_peered":"2023-09-23T03:24:09.626867+0000","last_unstale":"2023-09-23T03:24:10.642547+0000","last_undegraded":"2023-09-23T03:24:10.642547+0000","last_fullsized":"2023-09-23T03:24:10.642547+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:24:28.730813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.605652+0000","last_change":"2023-09-23T03:24:10.628212+0000","last_active":"2023-09-23T03:24:11.605652+0000","last_peered":"2023-09-23T03:24:11.605652+0000","last_clean":"2023-09-23T03:24:11.605652+0000","last_became_active":"2023-09-23T03:24:10.627897+0000","last_became_peered":"2023-09-23T03:24:10.627897+0000","last_unstale":"2023-09-23T03:24:11.605652+0000","last_undegraded":"2023-09-23T03:24:11.605652+0000","last_fullsized":"2023-09-23T03:24:11.605652+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:28:06.095616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T03:24:10.616048+0000","last_change":"2023-09-23T03:24:09.620859+0000","last_active":"2023-09-23T03:24:10.616048+0000","last_peered":"2023-09-23T03:24:10.616048+0000","last_clean":"2023-09-23T03:24:10.616048+0000","last_became_active":"2023-09-23T03:24:09.620521+0000","last_became_peered":"2023-09-23T03:24:09.620521+0000","last_unstale":"2023-09-23T03:24:10.616048+0000","last_undegraded":"2023-09-23T03:24:10.616048+0000","last_fullsized":"2023-09-23T03:24:10.616048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:15:33.296587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.606058+0000","last_change":"2023-09-23T03:24:10.615365+0000","last_active":"2023-09-23T03:24:11.606058+0000","last_peered":"2023-09-23T03:24:11.606058+0000","last_clean":"2023-09-23T03:24:11.606058+0000","last_became_active":"2023-09-23T03:24:10.615034+0000","last_became_peered":"2023-09-23T03:24:10.615034+0000","last_unstale":"2023-09-23T03:24:11.606058+0000","last_undegraded":"2023-09-23T03:24:11.606058+0000","last_fullsized":"2023-09-23T03:24:11.606058+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:24:29.747408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T03:24:10.613296+0000","last_change":"2023-09-23T03:24:09.615616+0000","last_active":"2023-09-23T03:24:10.613296+0000","last_peered":"2023-09-23T03:24:10.613296+0000","last_clean":"2023-09-23T03:24:10.613296+0000","last_became_active":"2023-09-23T03:24:09.615328+0000","last_became_peered":"2023-09-23T03:24:09.615328+0000","last_unstale":"2023-09-23T03:24:10.613296+0000","last_undegraded":"2023-09-23T03:24:10.613296+0000","last_fullsized":"2023-09-23T03:24:10.613296+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:29:22.145180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.606284+0000","last_change":"2023-09-23T03:24:10.648299+0000","last_active":"2023-09-23T03:24:11.606284+0000","last_peered":"2023-09-23T03:24:11.606284+0000","last_clean":"2023-09-23T03:24:11.606284+0000","last_became_active":"2023-09-23T03:24:10.648025+0000","last_became_peered":"2023-09-23T03:24:10.648025+0000","last_unstale":"2023-09-23T03:24:11.606284+0000","last_undegraded":"2023-09-23T03:24:11.606284+0000","last_fullsized":"2023-09-23T03:24:11.606284+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:40:59.292853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"82'8","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T03:24:10.647658+0000","last_change":"2023-09-23T03:24:09.624709+0000","last_active":"2023-09-23T03:24:10.647658+0000","last_peered":"2023-09-23T03:24:10.647658+0000","last_clean":"2023-09-23T03:24:10.647658+0000","last_became_active":"2023-09-23T03:24:09.624429+0000","last_became_peered":"2023-09-23T03:24:09.624429+0000","last_unstale":"2023-09-23T03:24:10.647658+0000","last_undegraded":"2023-09-23T03:24:10.647658+0000","last_fullsized":"2023-09-23T03:24:10.647658+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:15:53.388690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.600542+0000","last_change":"2023-09-23T03:24:10.631299+0000","last_active":"2023-09-23T03:24:11.600542+0000","last_peered":"2023-09-23T03:24:11.600542+0000","last_clean":"2023-09-23T03:24:11.600542+0000","last_became_active":"2023-09-23T03:24:10.631027+0000","last_became_peered":"2023-09-23T03:24:10.631027+0000","last_unstale":"2023-09-23T03:24:11.600542+0000","last_undegraded":"2023-09-23T03:24:11.600542+0000","last_fullsized":"2023-09-23T03:24:11.600542+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:28:19.431732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T03:24:10.645524+0000","last_change":"2023-09-23T03:24:09.624769+0000","last_active":"2023-09-23T03:24:10.645524+0000","last_peered":"2023-09-23T03:24:10.645524+0000","last_clean":"2023-09-23T03:24:10.645524+0000","last_became_active":"2023-09-23T03:24:09.624256+0000","last_became_peered":"2023-09-23T03:24:09.624256+0000","last_unstale":"2023-09-23T03:24:10.645524+0000","last_undegraded":"2023-09-23T03:24:10.645524+0000","last_fullsized":"2023-09-23T03:24:10.645524+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:17:17.964929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.641136+0000","last_change":"2023-09-23T03:24:10.622254+0000","last_active":"2023-09-23T03:24:11.641136+0000","last_peered":"2023-09-23T03:24:11.641136+0000","last_clean":"2023-09-23T03:24:11.641136+0000","last_became_active":"2023-09-23T03:24:10.621872+0000","last_became_peered":"2023-09-23T03:24:10.621872+0000","last_unstale":"2023-09-23T03:24:11.641136+0000","last_undegraded":"2023-09-23T03:24:11.641136+0000","last_fullsized":"2023-09-23T03:24:11.641136+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:49:34.554628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T03:24:10.644659+0000","last_change":"2023-09-23T03:24:09.626909+0000","last_active":"2023-09-23T03:24:10.644659+0000","last_peered":"2023-09-23T03:24:10.644659+0000","last_clean":"2023-09-23T03:24:10.644659+0000","last_became_active":"2023-09-23T03:24:09.626625+0000","last_became_peered":"2023-09-23T03:24:09.626625+0000","last_unstale":"2023-09-23T03:24:10.644659+0000","last_undegraded":"2023-09-23T03:24:10.644659+0000","last_fullsized":"2023-09-23T03:24:10.644659+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:25:33.134561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.641295+0000","last_change":"2023-09-23T03:24:10.633083+0000","last_active":"2023-09-23T03:24:11.641295+0000","last_peered":"2023-09-23T03:24:11.641295+0000","last_clean":"2023-09-23T03:24:11.641295+0000","last_became_active":"2023-09-23T03:24:10.632784+0000","last_became_peered":"2023-09-23T03:24:10.632784+0000","last_unstale":"2023-09-23T03:24:11.641295+0000","last_undegraded":"2023-09-23T03:24:11.641295+0000","last_fullsized":"2023-09-23T03:24:11.641295+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:15:34.232813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T03:24:10.648996+0000","last_change":"2023-09-23T03:24:09.627217+0000","last_active":"2023-09-23T03:24:10.648996+0000","last_peered":"2023-09-23T03:24:10.648996+0000","last_clean":"2023-09-23T03:24:10.648996+0000","last_became_active":"2023-09-23T03:24:09.626895+0000","last_became_peered":"2023-09-23T03:24:09.626895+0000","last_unstale":"2023-09-23T03:24:10.648996+0000","last_undegraded":"2023-09-23T03:24:10.648996+0000","last_fullsized":"2023-09-23T03:24:10.648996+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:29:54.816640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.601585+0000","last_change":"2023-09-23T03:24:10.647197+0000","last_active":"2023-09-23T03:24:11.601585+0000","last_peered":"2023-09-23T03:24:11.601585+0000","last_clean":"2023-09-23T03:24:11.601585+0000","last_became_active":"2023-09-23T03:24:10.646838+0000","last_became_peered":"2023-09-23T03:24:10.646838+0000","last_unstale":"2023-09-23T03:24:11.601585+0000","last_undegraded":"2023-09-23T03:24:11.601585+0000","last_fullsized":"2023-09-23T03:24:11.601585+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:41:37.473221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T03:24:10.615439+0000","last_change":"2023-09-23T03:24:09.619402+0000","last_active":"2023-09-23T03:24:10.615439+0000","last_peered":"2023-09-23T03:24:10.615439+0000","last_clean":"2023-09-23T03:24:10.615439+0000","last_became_active":"2023-09-23T03:24:09.619126+0000","last_became_peered":"2023-09-23T03:24:09.619126+0000","last_unstale":"2023-09-23T03:24:10.615439+0000","last_undegraded":"2023-09-23T03:24:10.615439+0000","last_fullsized":"2023-09-23T03:24:10.615439+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:55:39.471977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.619043+0000","last_change":"2023-09-23T03:24:10.625700+0000","last_active":"2023-09-23T03:24:11.619043+0000","last_peered":"2023-09-23T03:24:11.619043+0000","last_clean":"2023-09-23T03:24:11.619043+0000","last_became_active":"2023-09-23T03:24:10.625412+0000","last_became_peered":"2023-09-23T03:24:10.625412+0000","last_unstale":"2023-09-23T03:24:11.619043+0000","last_undegraded":"2023-09-23T03:24:11.619043+0000","last_fullsized":"2023-09-23T03:24:11.619043+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:44:10.719896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T03:24:10.650011+0000","last_change":"2023-09-23T03:24:09.621763+0000","last_active":"2023-09-23T03:24:10.650011+0000","last_peered":"2023-09-23T03:24:10.650011+0000","last_clean":"2023-09-23T03:24:10.650011+0000","last_became_active":"2023-09-23T03:24:09.621417+0000","last_became_peered":"2023-09-23T03:24:09.621417+0000","last_unstale":"2023-09-23T03:24:10.650011+0000","last_undegraded":"2023-09-23T03:24:10.650011+0000","last_fullsized":"2023-09-23T03:24:10.650011+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:55:54.822101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.619338+0000","last_change":"2023-09-23T03:24:10.614623+0000","last_active":"2023-09-23T03:24:11.619338+0000","last_peered":"2023-09-23T03:24:11.619338+0000","last_clean":"2023-09-23T03:24:11.619338+0000","last_became_active":"2023-09-23T03:24:10.614200+0000","last_became_peered":"2023-09-23T03:24:10.614200+0000","last_unstale":"2023-09-23T03:24:11.619338+0000","last_undegraded":"2023-09-23T03:24:11.619338+0000","last_fullsized":"2023-09-23T03:24:11.619338+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:37:45.052571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T03:24:10.616135+0000","last_change":"2023-09-23T03:24:09.629509+0000","last_active":"2023-09-23T03:24:10.616135+0000","last_peered":"2023-09-23T03:24:10.616135+0000","last_clean":"2023-09-23T03:24:10.616135+0000","last_became_active":"2023-09-23T03:24:09.629213+0000","last_became_peered":"2023-09-23T03:24:09.629213+0000","last_unstale":"2023-09-23T03:24:10.616135+0000","last_undegraded":"2023-09-23T03:24:10.616135+0000","last_fullsized":"2023-09-23T03:24:10.616135+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:17:43.850449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.606365+0000","last_change":"2023-09-23T03:24:10.625381+0000","last_active":"2023-09-23T03:24:11.606365+0000","last_peered":"2023-09-23T03:24:11.606365+0000","last_clean":"2023-09-23T03:24:11.606365+0000","last_became_active":"2023-09-23T03:24:10.624894+0000","last_became_peered":"2023-09-23T03:24:10.624894+0000","last_unstale":"2023-09-23T03:24:11.606365+0000","last_undegraded":"2023-09-23T03:24:11.606365+0000","last_fullsized":"2023-09-23T03:24:11.606365+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:15:54.405284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T03:24:10.611874+0000","last_change":"2023-09-23T03:24:09.628404+0000","last_active":"2023-09-23T03:24:10.611874+0000","last_peered":"2023-09-23T03:24:10.611874+0000","last_clean":"2023-09-23T03:24:10.611874+0000","last_became_active":"2023-09-23T03:24:09.628044+0000","last_became_peered":"2023-09-23T03:24:09.628044+0000","last_unstale":"2023-09-23T03:24:10.611874+0000","last_undegraded":"2023-09-23T03:24:10.611874+0000","last_fullsized":"2023-09-23T03:24:10.611874+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:31:34.373865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:12.742783+0000","last_change":"2023-09-23T03:24:10.616680+0000","last_active":"2023-09-23T03:24:12.742783+0000","last_peered":"2023-09-23T03:24:12.742783+0000","last_clean":"2023-09-23T03:24:12.742783+0000","last_became_active":"2023-09-23T03:24:10.616339+0000","last_became_peered":"2023-09-23T03:24:10.616339+0000","last_unstale":"2023-09-23T03:24:12.742783+0000","last_undegraded":"2023-09-23T03:24:12.742783+0000","last_fullsized":"2023-09-23T03:24:12.742783+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:34:09.802297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"82'2","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T03:24:10.645993+0000","last_change":"2023-09-23T03:24:09.622375+0000","last_active":"2023-09-23T03:24:10.645993+0000","last_peered":"2023-09-23T03:24:10.645993+0000","last_clean":"2023-09-23T03:24:10.645993+0000","last_became_active":"2023-09-23T03:24:09.622010+0000","last_became_peered":"2023-09-23T03:24:09.622010+0000","last_unstale":"2023-09-23T03:24:10.645993+0000","last_undegraded":"2023-09-23T03:24:10.645993+0000","last_fullsized":"2023-09-23T03:24:10.645993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:40:58.276258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.604986+0000","last_change":"2023-09-23T03:24:10.633148+0000","last_active":"2023-09-23T03:24:11.604986+0000","last_peered":"2023-09-23T03:24:11.604986+0000","last_clean":"2023-09-23T03:24:11.604986+0000","last_became_active":"2023-09-23T03:24:10.632837+0000","last_became_peered":"2023-09-23T03:24:10.632837+0000","last_unstale":"2023-09-23T03:24:11.604986+0000","last_undegraded":"2023-09-23T03:24:11.604986+0000","last_fullsized":"2023-09-23T03:24:11.604986+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:06:39.323186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T03:24:10.601123+0000","last_change":"2023-09-23T03:24:09.620514+0000","last_active":"2023-09-23T03:24:10.601123+0000","last_peered":"2023-09-23T03:24:10.601123+0000","last_clean":"2023-09-23T03:24:10.601123+0000","last_became_active":"2023-09-23T03:24:09.620227+0000","last_became_peered":"2023-09-23T03:24:09.620227+0000","last_unstale":"2023-09-23T03:24:10.601123+0000","last_undegraded":"2023-09-23T03:24:10.601123+0000","last_fullsized":"2023-09-23T03:24:10.601123+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:59:52.307843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:12.744201+0000","last_change":"2023-09-23T03:24:10.649477+0000","last_active":"2023-09-23T03:24:12.744201+0000","last_peered":"2023-09-23T03:24:12.744201+0000","last_clean":"2023-09-23T03:24:12.744201+0000","last_became_active":"2023-09-23T03:24:10.649203+0000","last_became_peered":"2023-09-23T03:24:10.649203+0000","last_unstale":"2023-09-23T03:24:12.744201+0000","last_undegraded":"2023-09-23T03:24:12.744201+0000","last_fullsized":"2023-09-23T03:24:12.744201+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:42:28.257850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T03:24:10.618820+0000","last_change":"2023-09-23T03:24:09.626835+0000","last_active":"2023-09-23T03:24:10.618820+0000","last_peered":"2023-09-23T03:24:10.618820+0000","last_clean":"2023-09-23T03:24:10.618820+0000","last_became_active":"2023-09-23T03:24:09.626400+0000","last_became_peered":"2023-09-23T03:24:09.626400+0000","last_unstale":"2023-09-23T03:24:10.618820+0000","last_undegraded":"2023-09-23T03:24:10.618820+0000","last_fullsized":"2023-09-23T03:24:10.618820+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:31:34.373865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.619057+0000","last_change":"2023-09-23T03:24:10.637030+0000","last_active":"2023-09-23T03:24:11.619057+0000","last_peered":"2023-09-23T03:24:11.619057+0000","last_clean":"2023-09-23T03:24:11.619057+0000","last_became_active":"2023-09-23T03:24:10.636721+0000","last_became_peered":"2023-09-23T03:24:10.636721+0000","last_unstale":"2023-09-23T03:24:11.619057+0000","last_undegraded":"2023-09-23T03:24:11.619057+0000","last_fullsized":"2023-09-23T03:24:11.619057+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:41:37.473221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T03:24:10.616407+0000","last_change":"2023-09-23T03:24:09.629743+0000","last_active":"2023-09-23T03:24:10.616407+0000","last_peered":"2023-09-23T03:24:10.616407+0000","last_clean":"2023-09-23T03:24:10.616407+0000","last_became_active":"2023-09-23T03:24:09.629390+0000","last_became_peered":"2023-09-23T03:24:09.629390+0000","last_unstale":"2023-09-23T03:24:10.616407+0000","last_undegraded":"2023-09-23T03:24:10.616407+0000","last_fullsized":"2023-09-23T03:24:10.616407+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:24:28.730813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.605319+0000","last_change":"2023-09-23T03:24:10.649860+0000","last_active":"2023-09-23T03:24:11.605319+0000","last_peered":"2023-09-23T03:24:11.605319+0000","last_clean":"2023-09-23T03:24:11.605319+0000","last_became_active":"2023-09-23T03:24:10.648440+0000","last_became_peered":"2023-09-23T03:24:10.648440+0000","last_unstale":"2023-09-23T03:24:11.605319+0000","last_undegraded":"2023-09-23T03:24:11.605319+0000","last_fullsized":"2023-09-23T03:24:11.605319+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:13:56.017901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T03:24:10.607502+0000","last_change":"2023-09-23T03:24:09.624729+0000","last_active":"2023-09-23T03:24:10.607502+0000","last_peered":"2023-09-23T03:24:10.607502+0000","last_clean":"2023-09-23T03:24:10.607502+0000","last_became_active":"2023-09-23T03:24:09.624381+0000","last_became_peered":"2023-09-23T03:24:09.624381+0000","last_unstale":"2023-09-23T03:24:10.607502+0000","last_undegraded":"2023-09-23T03:24:10.607502+0000","last_fullsized":"2023-09-23T03:24:10.607502+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:42:04.012140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.605744+0000","last_change":"2023-09-23T03:24:10.629560+0000","last_active":"2023-09-23T03:24:11.605744+0000","last_peered":"2023-09-23T03:24:11.605744+0000","last_clean":"2023-09-23T03:24:11.605744+0000","last_became_active":"2023-09-23T03:24:10.629283+0000","last_became_peered":"2023-09-23T03:24:10.629283+0000","last_unstale":"2023-09-23T03:24:11.605744+0000","last_undegraded":"2023-09-23T03:24:11.605744+0000","last_fullsized":"2023-09-23T03:24:11.605744+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:15:34.313182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T03:24:10.613652+0000","last_change":"2023-09-23T03:24:09.620607+0000","last_active":"2023-09-23T03:24:10.613652+0000","last_peered":"2023-09-23T03:24:10.613652+0000","last_clean":"2023-09-23T03:24:10.613652+0000","last_became_active":"2023-09-23T03:24:09.618446+0000","last_became_peered":"2023-09-23T03:24:09.618446+0000","last_unstale":"2023-09-23T03:24:10.613652+0000","last_undegraded":"2023-09-23T03:24:10.613652+0000","last_fullsized":"2023-09-23T03:24:10.613652+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:38:56.452836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:12.741694+0000","last_change":"2023-09-23T03:24:10.631790+0000","last_active":"2023-09-23T03:24:12.741694+0000","last_peered":"2023-09-23T03:24:12.741694+0000","last_clean":"2023-09-23T03:24:12.741694+0000","last_became_active":"2023-09-23T03:24:10.631381+0000","last_became_peered":"2023-09-23T03:24:10.631381+0000","last_unstale":"2023-09-23T03:24:12.741694+0000","last_undegraded":"2023-09-23T03:24:12.741694+0000","last_fullsized":"2023-09-23T03:24:12.741694+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:37:58.042582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T03:24:10.610796+0000","last_change":"2023-09-23T03:24:09.621858+0000","last_active":"2023-09-23T03:24:10.610796+0000","last_peered":"2023-09-23T03:24:10.610796+0000","last_clean":"2023-09-23T03:24:10.610796+0000","last_became_active":"2023-09-23T03:24:09.621588+0000","last_became_peered":"2023-09-23T03:24:09.621588+0000","last_unstale":"2023-09-23T03:24:10.610796+0000","last_undegraded":"2023-09-23T03:24:10.610796+0000","last_fullsized":"2023-09-23T03:24:10.610796+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:51:50.412098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:12.741733+0000","last_change":"2023-09-23T03:24:10.627061+0000","last_active":"2023-09-23T03:24:12.741733+0000","last_peered":"2023-09-23T03:24:12.741733+0000","last_clean":"2023-09-23T03:24:12.741733+0000","last_became_active":"2023-09-23T03:24:10.626765+0000","last_became_peered":"2023-09-23T03:24:10.626765+0000","last_unstale":"2023-09-23T03:24:12.741733+0000","last_undegraded":"2023-09-23T03:24:12.741733+0000","last_fullsized":"2023-09-23T03:24:12.741733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:17:06.250481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T03:24:10.616297+0000","last_change":"2023-09-23T03:24:09.615983+0000","last_active":"2023-09-23T03:24:10.616297+0000","last_peered":"2023-09-23T03:24:10.616297+0000","last_clean":"2023-09-23T03:24:10.616297+0000","last_became_active":"2023-09-23T03:24:09.615701+0000","last_became_peered":"2023-09-23T03:24:09.615701+0000","last_unstale":"2023-09-23T03:24:10.616297+0000","last_undegraded":"2023-09-23T03:24:10.616297+0000","last_fullsized":"2023-09-23T03:24:10.616297+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:02:15.731649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-09-23T03:24:11.603705+0000","last_change":"2023-09-23T03:24:10.627187+0000","last_active":"2023-09-23T03:24:11.603705+0000","last_peered":"2023-09-23T03:24:11.603705+0000","last_clean":"2023-09-23T03:24:11.603705+0000","last_became_active":"2023-09-23T03:24:10.626902+0000","last_became_peered":"2023-09-23T03:24:10.626902+0000","last_unstale":"2023-09-23T03:24:11.603705+0000","last_undegraded":"2023-09-23T03:24:11.603705+0000","last_fullsized":"2023-09-23T03:24:11.603705+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:09.574937+0000","last_clean_scrub_stamp":"2023-09-23T03:24:09.574937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:28:19.431732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T03:24:10.624763+0000","last_change":"2023-09-23T03:24:09.628343+0000","last_active":"2023-09-23T03:24:10.624763+0000","last_peered":"2023-09-23T03:24:10.624763+0000","last_clean":"2023-09-23T03:24:10.624763+0000","last_became_active":"2023-09-23T03:24:09.628047+0000","last_became_peered":"2023-09-23T03:24:09.628047+0000","last_unstale":"2023-09-23T03:24:10.624763+0000","last_undegraded":"2023-09-23T03:24:10.624763+0000","last_fullsized":"2023-09-23T03:24:10.624763+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:24:08.558342+0000","last_clean_scrub_stamp":"2023-09-23T03:24:08.558342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:09:52.535203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":220,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-09-23T03:24:10.618839+0000","last_change":"2023-09-23T03:15:11.042447+0000","last_active":"2023-09-23T03:24:10.618839+0000","last_peered":"2023-09-23T03:24:10.618839+0000","last_clean":"2023-09-23T03:24:10.618839+0000","last_became_active":"2023-09-23T03:15:11.040322+0000","last_became_peered":"2023-09-23T03:15:11.040322+0000","last_unstale":"2023-09-23T03:24:10.618839+0000","last_undegraded":"2023-09-23T03:24:10.618839+0000","last_fullsized":"2023-09-23T03:24:10.618839+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_clean_scrub_stamp":"2023-09-23T03:15:09.474842+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:18:36.331645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574959,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32760,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339080,"statfs":{"total":96636764160,"available":96603217920,"internally_reserved":0,"allocated":1613824,"data_stored":1075231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:23:25 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.012,"15min":1.0029999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.442},"max":{"1min":1.556,"5min":1.6890000000000001,"15min":2.149},"last":0.87},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0660000000000001,"15min":1.0509999999999999},"min":{"1min":0.78200000000000003,"5min":0.56200000000000006,"15min":0.52000000000000002},"max":{"1min":1.4870000000000001,"5min":1.681,"15min":2.1480000000000001},"last":0.81699999999999995}]},{"osd":1,"last update":"Sat Sep 23 03:23:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.214,"15min":1.2150000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.504},"max":{"1min":1.4990000000000001,"5min":1.671,"15min":3.1859999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.246,"15min":1.3300000000000001},"min":{"1min":0.78400000000000003,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.49,"5min":1.736,"15min":2.613},"last":0.71299999999999997}]},{"osd":2,"last update":"Sat Sep 23 03:23:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.099,"15min":1.135},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.42399999999999999},"max":{"1min":1.3149999999999999,"5min":1.667,"15min":2.331},"last":0.61199999999999999},{"interface":"front","average":{"1min":1.151,"5min":1.1779999999999999,"15min":1.254},"min":{"1min":0.749,"5min":0.621,"15min":0.55200000000000005},"max":{"1min":1.613,"5min":1.724,"15min":3.093},"last":0.79600000000000004}]},{"osd":3,"last update":"Sat Sep 23 03:23:25 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.208,"15min":1.266},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.4450000000000001,"5min":1.9039999999999999,"15min":2.532},"last":0.66800000000000004},{"interface":"front","average":{"1min":1.286,"5min":1.3080000000000001,"15min":1.3919999999999999},"min":{"1min":0.76500000000000001,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.633,"5min":1.8049999999999999,"15min":3.2349999999999999},"last":0.91200000000000003}]},{"osd":4,"last update":"Sat Sep 23 03:23:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2949999999999999,"15min":1.448},"min":{"1min":0.89600000000000002,"5min":0.79200000000000004,"15min":0.45400000000000001},"max":{"1min":1.5800000000000001,"5min":1.867,"15min":2.7210000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.427,"15min":1.514},"min":{"1min":1.157,"5min":0.81699999999999995,"15min":0.48099999999999998},"max":{"1min":1.6459999999999999,"5min":1.944,"15min":3.7719999999999998},"last":0.73599999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:23:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.351,"15min":1.4390000000000001},"min":{"1min":0.749,"5min":0.72699999999999998,"15min":0.47999999999999998},"max":{"1min":1.798,"5min":1.8799999999999999,"15min":2.3690000000000002},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.367,"15min":1.492},"min":{"1min":1.1930000000000001,"5min":1.0069999999999999,"15min":0.497},"max":{"1min":1.679,"5min":2.008,"15min":2.4689999999999999},"last":0.77000000000000002}]},{"osd":6,"last update":"Sat Sep 23 03:23:25 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.4830000000000001,"15min":1.5920000000000001},"min":{"1min":1.026,"5min":1.026,"15min":0.53800000000000003},"max":{"1min":1.7769999999999999,"5min":1.9399999999999999,"15min":2.8420000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.413,"5min":1.4159999999999999,"15min":1.5429999999999999},"min":{"1min":1.089,"5min":0.92400000000000004,"15min":0.45900000000000002},"max":{"1min":1.7869999999999999,"5min":2.0699999999999998,"15min":3.6739999999999999},"last":0.78900000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574959,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1028096,"data_stored":486453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:23:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":1.111,"15min":1.0529999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.38},"max":{"1min":1.4850000000000001,"5min":1.778,"15min":1.778},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.081,"5min":1.0880000000000001,"15min":1.0629999999999999},"min":{"1min":0.71499999999999997,"5min":0.61599999999999999,"15min":0.46100000000000002},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":2.0529999999999999},"last":1.2909999999999999}]},{"osd":1,"last update":"Sat Sep 23 03:23:20 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.1220000000000001,"15min":1.1040000000000001},"min":{"1min":0.64100000000000001,"5min":0.59099999999999997,"15min":0.438},"max":{"1min":1.468,"5min":1.7030000000000001,"15min":1.867},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.093,"5min":1.153,"15min":1.21},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.5269999999999999,"5min":1.7190000000000001,"15min":2.242},"last":1.1879999999999999}]},{"osd":2,"last update":"Sat Sep 23 03:23:20 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.109,"15min":1.1080000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.51300000000000001},"max":{"1min":1.5329999999999999,"5min":1.6659999999999999,"15min":2.097},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.173,"15min":1.236},"min":{"1min":0.71999999999999997,"5min":0.66200000000000003,"15min":0.64300000000000002},"max":{"1min":1.375,"5min":1.8149999999999999,"15min":1.903},"last":1.351}]},{"osd":3,"last update":"Sat Sep 23 03:23:20 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.278,"15min":1.3360000000000001},"min":{"1min":0.92400000000000004,"5min":0.65800000000000003,"15min":0.54600000000000004},"max":{"1min":1.77,"5min":1.8879999999999999,"15min":2.6509999999999998},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3260000000000001,"15min":1.415},"min":{"1min":0.95599999999999996,"5min":0.745,"15min":0.74299999999999999},"max":{"1min":1.665,"5min":2.0339999999999998,"15min":2.3740000000000001},"last":1.0580000000000001}]},{"osd":4,"last update":"Sat Sep 23 03:23:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3839999999999999},"min":{"1min":0.98199999999999998,"5min":0.81399999999999995,"15min":0.498},"max":{"1min":1.7190000000000001,"5min":1.843,"15min":2.3210000000000002},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.276,"5min":1.2809999999999999,"15min":1.2889999999999999},"min":{"1min":0.89300000000000002,"5min":0.74399999999999999,"15min":0.56000000000000005},"max":{"1min":1.6719999999999999,"5min":1.708,"15min":2.2109999999999999},"last":1.47}]},{"osd":5,"last update":"Sat Sep 23 03:23:20 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.405,"15min":1.3979999999999999},"min":{"1min":1.171,"5min":0.84699999999999998,"15min":0.69199999999999995},"max":{"1min":1.857,"5min":1.857,"15min":2.2050000000000001},"last":1.415},{"interface":"front","average":{"1min":1.361,"5min":1.353,"15min":1.45},"min":{"1min":0.95499999999999996,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.887,"5min":1.9410000000000001,"15min":4.758},"last":1.5169999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:23:20 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.4810000000000001,"15min":1.5329999999999999},"min":{"1min":1.1100000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.821,"5min":1.978,"15min":2.476},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.4870000000000001,"15min":1.5409999999999999},"min":{"1min":1.278,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.9470000000000001,"5min":2.1739999999999999,"15min":2.4060000000000001},"last":1.6299999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574961,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1019904,"data_stored":484875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:23:17 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":1.006,"15min":1.0269999999999999},"min":{"1min":0.64200000000000002,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.27,"5min":1.732,"15min":1.732},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.139,"5min":1.099,"15min":1.0509999999999999},"min":{"1min":0.69199999999999995,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.5569999999999999,"5min":2.2320000000000002,"15min":2.2320000000000002},"last":1.4630000000000001}]},{"osd":1,"last update":"Sat Sep 23 03:23:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.254,"15min":1.2270000000000001},"min":{"1min":0.68300000000000005,"5min":0.62,"15min":0.62},"max":{"1min":1.599,"5min":1.7869999999999999,"15min":1.849},"last":1.208},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.284,"15min":1.298},"min":{"1min":0.83199999999999996,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.663,"5min":1.7410000000000001,"15min":1.893},"last":1.075}]},{"osd":2,"last update":"Sat Sep 23 03:23:17 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.2,"15min":1.2050000000000001},"min":{"1min":0.65800000000000003,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.6910000000000001,"5min":1.726,"15min":1.726},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.2,"15min":1.2669999999999999},"min":{"1min":0.73699999999999999,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.464,"5min":1.7529999999999999,"15min":1.7629999999999999},"last":1.141}]},{"osd":3,"last update":"Sat Sep 23 03:23:17 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.198,"15min":1.304},"min":{"1min":0.68500000000000005,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.5980000000000001,"5min":1.696,"15min":1.905},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.302,"15min":1.411},"min":{"1min":0.93799999999999994,"5min":0.71699999999999997,"15min":0.69799999999999995},"max":{"1min":1.492,"5min":1.792,"15min":3.6110000000000002},"last":1.345}]},{"osd":4,"last update":"Sat Sep 23 03:23:17 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.351,"15min":1.415},"min":{"1min":0.98399999999999999,"5min":0.83099999999999996,"15min":0.54100000000000004},"max":{"1min":1.843,"5min":1.9359999999999999,"15min":2.069},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2829999999999999,"15min":1.401},"min":{"1min":0.82799999999999996,"5min":0.60299999999999998,"15min":0.48899999999999999},"max":{"1min":1.6699999999999999,"5min":1.7889999999999999,"15min":2.319},"last":1.264}]},{"osd":6,"last update":"Sat Sep 23 03:23:17 2023","interfaces":[{"interface":"back","average":{"1min":1.488,"5min":1.49,"15min":1.5229999999999999},"min":{"1min":1.2450000000000001,"5min":0.69899999999999995,"15min":0.59199999999999997},"max":{"1min":1.7729999999999999,"5min":2.0350000000000001,"15min":2.1509999999999998},"last":1.6419999999999999},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.524,"15min":1.587},"min":{"1min":1.119,"5min":0.78700000000000003,"15min":0.55100000000000005},"max":{"1min":1.605,"5min":9.6560000000000006,"15min":9.6560000000000006},"last":1.4019999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:23:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.369,"15min":1.4830000000000001},"min":{"1min":1.0860000000000001,"5min":0.89100000000000001,"15min":0.626},"max":{"1min":1.784,"5min":1.8480000000000001,"15min":2.0350000000000001},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4850000000000001,"15min":1.5960000000000001},"min":{"1min":1.228,"5min":0.85999999999999999,"15min":0.60799999999999998},"max":{"1min":1.7390000000000001,"5min":1.919,"15min":2.1960000000000002},"last":1.601}]}]},{"osd":4,"up_from":13,"seq":55834574960,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1036288,"data_stored":486557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:23:20 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.093,"15min":1.117},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.59899999999999998},"max":{"1min":1.4770000000000001,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.181,"15min":1.155},"min":{"1min":0.67400000000000004,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.581,"5min":1.905,"15min":1.9179999999999999},"last":1.1899999999999999}]},{"osd":1,"last update":"Sat Sep 23 03:23:20 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.232,"15min":1.2110000000000001},"min":{"1min":0.65400000000000003,"5min":0.64000000000000001,"15min":0.61799999999999999},"max":{"1min":1.609,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.0900000000000001,"15min":1.1799999999999999},"min":{"1min":0.72299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.625,"5min":1.625,"15min":3.2189999999999999},"last":0.82299999999999995}]},{"osd":2,"last update":"Sat Sep 23 03:23:20 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.111,"15min":1.2},"min":{"1min":0.90600000000000003,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.5509999999999999,"5min":2.129,"15min":2.5920000000000001},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.202,"5min":1.2470000000000001,"15min":1.228},"min":{"1min":0.73499999999999999,"5min":0.69999999999999996,"15min":0.68300000000000005},"max":{"1min":1.784,"5min":1.909,"15min":1.909},"last":1.508}]},{"osd":3,"last update":"Sat Sep 23 03:23:20 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1719999999999999,"15min":1.1699999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.69199999999999995},"max":{"1min":1.504,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.216,"5min":1.27,"15min":1.3169999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.6359999999999999,"5min":2.1779999999999999,"15min":2.4969999999999999},"last":1.603}]},{"osd":5,"last update":"Sat Sep 23 03:23:20 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.395,"15min":1.3560000000000001},"min":{"1min":0.876,"5min":0.83099999999999996,"15min":0.70199999999999996},"max":{"1min":1.7,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.6779999999999999},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.355,"15min":1.333},"min":{"1min":0.97999999999999998,"5min":0.84299999999999997,"15min":0.81799999999999995},"max":{"1min":1.736,"5min":2.887,"15min":2.887},"last":1.2589999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:23:20 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.3680000000000001,"15min":1.377},"min":{"1min":0.86499999999999999,"5min":0.81299999999999994,"15min":0.66400000000000003},"max":{"1min":1.7509999999999999,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":1.831},{"interface":"front","average":{"1min":1.476,"5min":1.5349999999999999,"15min":1.488},"min":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":0.89900000000000002},"max":{"1min":1.8580000000000001,"5min":2.0579999999999998,"15min":2.1779999999999999},"last":1.7949999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:23:20 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.484,"15min":1.47},"min":{"1min":1.0880000000000001,"5min":0.98299999999999998,"15min":0.81399999999999995},"max":{"1min":1.849,"5min":2.5680000000000001,"15min":2.5680000000000001},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.486,"5min":1.5549999999999999,"15min":1.556},"min":{"1min":1.1619999999999999,"5min":1.056,"15min":0.84199999999999997},"max":{"1min":1.8340000000000001,"5min":2.4769999999999999,"15min":2.4769999999999999},"last":1.76}]}]},{"osd":3,"up_from":13,"seq":55834574960,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1019904,"data_stored":485423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.88900000000000001,"15min":0.876},"min":{"1min":0.68799999999999994,"5min":0.42199999999999999,"15min":0.40200000000000002},"max":{"1min":1.7050000000000001,"5min":2.952,"15min":2.952},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.58699999999999997,"5min":0.377,"15min":0.29299999999999998},"max":{"1min":1.419,"5min":1.419,"15min":1.8979999999999999},"last":1.026}]},{"osd":1,"last update":"Sat Sep 23 03:23:30 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85399999999999998,"15min":0.92500000000000004},"min":{"1min":0.51700000000000002,"5min":0.442,"15min":0.42299999999999999},"max":{"1min":1.603,"5min":1.603,"15min":1.6839999999999999},"last":1.141},{"interface":"front","average":{"1min":1.018,"5min":0.99299999999999999,"15min":0.98499999999999999},"min":{"1min":0.67100000000000004,"5min":0.54700000000000004,"15min":0.45400000000000001},"max":{"1min":1.605,"5min":2.903,"15min":2.903},"last":1.079}]},{"osd":2,"last update":"Sat Sep 23 03:23:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.048,"15min":1.087},"min":{"1min":0.70699999999999996,"5min":0.54800000000000004,"15min":0.46500000000000002},"max":{"1min":1.9850000000000001,"5min":1.9850000000000001,"15min":2.2240000000000002},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.056,"5min":1.1120000000000001,"15min":1.1160000000000001},"min":{"1min":0.76500000000000001,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3360000000000001,"5min":3.129,"15min":3.129},"last":1.3109999999999999}]},{"osd":4,"last update":"Sat Sep 23 03:23:30 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.1850000000000001,"15min":1.2789999999999999},"min":{"1min":0.873,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.671,"5min":2.0099999999999998,"15min":2.698},"last":1.212},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.3100000000000001,"15min":1.399},"min":{"1min":0.93999999999999995,"5min":0.93899999999999995,"15min":0.752},"max":{"1min":1.587,"5min":1.825,"15min":2.419},"last":1.4019999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:23:30 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.361,"15min":1.355},"min":{"1min":1.004,"5min":0.90300000000000002,"15min":0.753},"max":{"1min":2.04,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":1.879},{"interface":"front","average":{"1min":1.345,"5min":1.27,"15min":1.3260000000000001},"min":{"1min":1.028,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":2.0070000000000001},"last":1.841}]},{"osd":6,"last update":"Sat Sep 23 03:23:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.3360000000000001,"15min":1.3939999999999999},"min":{"1min":1.0940000000000001,"5min":0.82699999999999996,"15min":0.69799999999999995},"max":{"1min":1.7869999999999999,"5min":1.9550000000000001,"15min":2.6440000000000001},"last":1.75},{"interface":"front","average":{"1min":1.45,"5min":1.4279999999999999,"15min":1.462},"min":{"1min":0.999,"5min":0.94499999999999995,"15min":0.67800000000000005},"max":{"1min":2.1360000000000001,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":1.8049999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:23:30 2023","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.345,"15min":1.4430000000000001},"min":{"1min":1.097,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.845,"5min":2.2120000000000002,"15min":2.7879999999999998},"last":1.982},{"interface":"front","average":{"1min":1.411,"5min":1.3959999999999999,"15min":1.48},"min":{"1min":1.0700000000000001,"5min":0.81200000000000006,"15min":0.80900000000000005},"max":{"1min":1.738,"5min":2.2029999999999998,"15min":2.5379999999999998},"last":1.698}]}]},{"osd":2,"up_from":13,"seq":55834574961,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1028096,"data_stored":485583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.86699999999999999,"15min":0.83699999999999997},"min":{"1min":0.50800000000000001,"5min":0.371,"15min":0.30299999999999999},"max":{"1min":1.284,"5min":1.643,"15min":1.643},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.90600000000000003,"15min":0.85599999999999998},"min":{"1min":0.46500000000000002,"5min":0.39300000000000002,"15min":0.36299999999999999},"max":{"1min":1.1919999999999999,"5min":1.5649999999999999,"15min":1.5840000000000001},"last":0.72399999999999998}]},{"osd":1,"last update":"Sat Sep 23 03:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.877,"15min":0.85899999999999999},"min":{"1min":0.55400000000000005,"5min":0.437,"15min":0.437},"max":{"1min":1.139,"5min":1.478,"15min":1.478},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.999,"15min":0.95399999999999996},"min":{"1min":0.59899999999999998,"5min":0.49399999999999999,"15min":0.38},"max":{"1min":1.26,"5min":1.6990000000000001,"15min":1.6990000000000001},"last":0.86799999999999999}]},{"osd":3,"last update":"Sat Sep 23 03:23:15 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.008,"15min":0.96299999999999997},"min":{"1min":0.59799999999999998,"5min":0.51600000000000001,"15min":0.40699999999999997},"max":{"1min":1.542,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.0149999999999999,"15min":0.997},"min":{"1min":0.56399999999999995,"5min":0.51000000000000001,"15min":0.34999999999999998},"max":{"1min":1.2290000000000001,"5min":1.6779999999999999,"15min":1.78},"last":0.752}]},{"osd":4,"last update":"Sat Sep 23 03:23:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.25,"15min":1.2949999999999999},"min":{"1min":0.88200000000000001,"5min":0.82099999999999995,"15min":0.55800000000000005},"max":{"1min":1.629,"5min":1.819,"15min":2.3109999999999999},"last":1.016},{"interface":"front","average":{"1min":1.218,"5min":1.272,"15min":1.3149999999999999},"min":{"1min":0.90700000000000003,"5min":0.77200000000000002,"15min":0.76100000000000001},"max":{"1min":1.6180000000000001,"5min":1.8080000000000001,"15min":2.234},"last":1.266}]},{"osd":5,"last update":"Sat Sep 23 03:23:15 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.3340000000000001,"15min":1.375},"min":{"1min":1.0169999999999999,"5min":0.78600000000000003,"15min":0.71499999999999997},"max":{"1min":1.6899999999999999,"5min":1.8049999999999999,"15min":3.9409999999999998},"last":1.7050000000000001},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.349,"15min":1.466},"min":{"1min":1.1479999999999999,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.7390000000000001,"5min":1.9359999999999999,"15min":4.9199999999999999},"last":1.448}]},{"osd":6,"last update":"Sat Sep 23 03:23:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.3560000000000001,"15min":1.3879999999999999},"min":{"1min":0.88,"5min":0.83299999999999996,"15min":0.755},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":3.0009999999999999},"last":1.6679999999999999},{"interface":"front","average":{"1min":1.399,"5min":1.403,"15min":1.4430000000000001},"min":{"1min":1.0389999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.839,"5min":2.1680000000000001,"15min":4.5800000000000001},"last":1.323}]},{"osd":7,"last update":"Sat Sep 23 03:23:15 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.4019999999999999,"15min":1.395},"min":{"1min":1.2,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.9219999999999999,"5min":1.996,"15min":2.3149999999999999},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.417,"5min":1.4379999999999999,"15min":1.4430000000000001},"min":{"1min":0.87,"5min":0.82599999999999996,"15min":0.74399999999999999},"max":{"1min":2.0590000000000002,"5min":2.0590000000000002,"15min":2.621},"last":1.4239999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574962,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1024000,"data_stored":484953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:23:27 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.80100000000000005,"15min":0.80000000000000004},"min":{"1min":0.45000000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.081,"5min":1.377,"15min":2.8279999999999998},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.81999999999999995,"15min":0.83999999999999997},"min":{"1min":0.57899999999999996,"5min":0.34799999999999998,"15min":0.312},"max":{"1min":1.147,"5min":1.4390000000000001,"15min":1.8919999999999999},"last":0.61099999999999999}]},{"osd":2,"last update":"Sat Sep 23 03:23:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.88700000000000001,"15min":0.88600000000000001},"min":{"1min":0.52500000000000002,"5min":0.503,"15min":0.36699999999999999},"max":{"1min":1.107,"5min":1.536,"15min":2.3340000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.99399999999999999,"5min":1.0449999999999999,"15min":1.0309999999999999},"min":{"1min":0.61199999999999999,"5min":0.54500000000000004,"15min":0.45600000000000002},"max":{"1min":1.3140000000000001,"5min":1.6499999999999999,"15min":1.9570000000000001},"last":0.73399999999999999}]},{"osd":3,"last update":"Sat Sep 23 03:23:27 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":1.01,"15min":1.0569999999999999},"min":{"1min":0.65800000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.413,"5min":1.595,"15min":2.2370000000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.008,"5min":1.0760000000000001,"15min":1.127},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.45900000000000002},"max":{"1min":1.6339999999999999,"5min":1.7649999999999999,"15min":3.2130000000000001},"last":0.84899999999999998}]},{"osd":4,"last update":"Sat Sep 23 03:23:27 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.216,"15min":1.2589999999999999},"min":{"1min":0.88700000000000001,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.476,"5min":1.7270000000000001,"15min":3.2629999999999999},"last":0.875},{"interface":"front","average":{"1min":1.266,"5min":1.2350000000000001,"15min":1.278},"min":{"1min":0.92200000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":2.4689999999999999},"last":0.77400000000000002}]},{"osd":5,"last update":"Sat Sep 23 03:23:27 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.2669999999999999,"15min":1.2889999999999999},"min":{"1min":0.92400000000000004,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.657,"5min":1.9319999999999999,"15min":3.161},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.325,"5min":1.335,"15min":1.339},"min":{"1min":0.998,"5min":0.80400000000000005,"15min":0.72399999999999998},"max":{"1min":1.7929999999999999,"5min":1.9890000000000001,"15min":2.4289999999999998},"last":0.90700000000000003}]},{"osd":6,"last update":"Sat Sep 23 03:23:27 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.3169999999999999,"15min":1.3280000000000001},"min":{"1min":1.0720000000000001,"5min":0.76500000000000001,"15min":0.747},"max":{"1min":1.667,"5min":2.0329999999999999,"15min":3.0179999999999998},"last":1.052},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.357,"15min":1.383},"min":{"1min":0.96099999999999997,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.694,"5min":2.0720000000000001,"15min":3.3580000000000001},"last":1.012}]},{"osd":7,"last update":"Sat Sep 23 03:23:27 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.3740000000000001,"15min":1.393},"min":{"1min":1.0760000000000001,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.669,"5min":2.085,"15min":3.1099999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.391,"5min":1.425,"15min":1.4550000000000001},"min":{"1min":1.008,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.657,"5min":1.9419999999999999,"15min":3.306},"last":0.82899999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574962,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32748,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1605632,"data_stored":1074190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 23 03:23:22 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.88600000000000001,"15min":0.83999999999999997},"min":{"1min":0.45800000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.2629999999999999,"5min":1.919,"15min":1.919},"last":1.0740000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.90000000000000002,"15min":0.89400000000000002},"min":{"1min":0.56200000000000006,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.139,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.94199999999999995}]},{"osd":2,"last update":"Sat Sep 23 03:23:22 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.89600000000000002,"15min":0.82899999999999996},"min":{"1min":0.55000000000000004,"5min":0.48599999999999999,"15min":0.46400000000000002},"max":{"1min":1.327,"5min":3.089,"15min":3.089},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.90300000000000002,"5min":1.0449999999999999,"15min":1.0049999999999999},"min":{"1min":0.61299999999999999,"5min":0.46700000000000003,"15min":0.42499999999999999},"max":{"1min":1.238,"5min":2.633,"15min":2.633},"last":0.82099999999999995}]},{"osd":3,"last update":"Sat Sep 23 03:23:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0289999999999999,"15min":0.95399999999999996},"min":{"1min":0.61499999999999999,"5min":0.44800000000000001,"15min":0.44700000000000001},"max":{"1min":1.45,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.615},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.0489999999999999,"15min":1.012},"min":{"1min":0.55400000000000005,"5min":0.53500000000000003,"15min":0.51900000000000002},"max":{"1min":1.5,"5min":2.012,"15min":2.012},"last":0.88200000000000001}]},{"osd":4,"last update":"Sat Sep 23 03:23:22 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.333,"15min":1.246},"min":{"1min":0.82899999999999996,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.7869999999999999,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":1.24},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.294,"15min":1.2829999999999999},"min":{"1min":0.86299999999999999,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.976,"5min":3.0230000000000001,"15min":3.0230000000000001},"last":1.2909999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:23:22 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.319,"15min":1.2529999999999999},"min":{"1min":1.0289999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.859,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":0.999},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.409,"15min":1.3140000000000001},"min":{"1min":1.0860000000000001,"5min":0.75700000000000001,"15min":0.71999999999999997},"max":{"1min":1.772,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.4219999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:23:22 2023","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.3779999999999999,"15min":1.3220000000000001},"min":{"1min":0.95599999999999996,"5min":0.77400000000000002,"15min":0.75600000000000001},"max":{"1min":1.9299999999999999,"5min":2.9489999999999998,"15min":2.9489999999999998},"last":1.157},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.4690000000000001,"15min":1.3560000000000001},"min":{"1min":1.1140000000000001,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.8300000000000001,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.3360000000000001}]},{"osd":7,"last update":"Sat Sep 23 03:23:22 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.4319999999999999,"15min":1.353},"min":{"1min":1.145,"5min":0.90500000000000003,"15min":0.747},"max":{"1min":1.8240000000000001,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":1.367},{"interface":"front","average":{"1min":1.52,"5min":1.4990000000000001,"15min":1.4379999999999999},"min":{"1min":1.196,"5min":0.91800000000000004,"15min":0.81699999999999995},"max":{"1min":2.0390000000000001,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":1.5029999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-23T03:24:15.601 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-09-23T03:24:15.602 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-09-23T03:24:15.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.866+0000 7f2bbb59e640 1 Processor -- start 2023-09-23T03:24:15.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.866+0000 7f2bbb59e640 1 -- start start 2023-09-23T03:24:15.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.867+0000 7f2bbb59e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bbc15df50 0x7f2bbc0b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:15.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.867+0000 7f2bbb59e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bbc15e330 con 0x7f2bbc0d0980 2023-09-23T03:24:15.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.867+0000 7f2bbb59e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bbc0d0e10 con 0x7f2bbc0d1890 2023-09-23T03:24:15.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.867+0000 7f2bbb59e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc0cd870 con 0x7f2bbc15df50 2023-09-23T03:24:15.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.868+0000 7f2bb9d9b640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f2bbc0d0980 0x7f2bbc0d1180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:59528/0 (socket says 172.21.15.154:59528) 2023-09-23T03:24:15.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.868+0000 7f2bb9d9b640 1 -- 172.21.15.154:0/1566006969 learned_addr learned my addr 172.21.15.154:0/1566006969 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:24:15.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.868+0000 7f2bba59c640 1 --2- 172.21.15.154:0/1566006969 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bbc15df50 0x7f2bbc0b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:15.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.868+0000 7f2bb959a640 1 -- 172.21.15.154:0/1566006969 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3784274714 0 0) 0x7f2bbc15e330 con 0x7f2bbc0d0980 2023-09-23T03:24:15.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.868+0000 7f2bb959a640 1 -- 172.21.15.154:0/1566006969 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ba8003520 con 0x7f2bbc0d0980 2023-09-23T03:24:15.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.868+0000 7f2bb959a640 1 -- 172.21.15.154:0/1566006969 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1728545081 0 0) 0x7f2ba8003520 con 0x7f2bbc0d0980 2023-09-23T03:24:15.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.868+0000 7f2bb959a640 1 -- 172.21.15.154:0/1566006969 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bbc15df50 msgr2=0x7f2bbc0b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:15.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.868+0000 7f2bb959a640 1 --2- 172.21.15.154:0/1566006969 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bbc15df50 0x7f2bbc0b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:15.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.868+0000 7f2bb959a640 1 -- 172.21.15.154:0/1566006969 >> v1:172.21.15.190:6789/0 conn(0x7f2bbc0d1890 legacy=0x7f2bbc0cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:15.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.868+0000 7f2bb959a640 1 -- 172.21.15.154:0/1566006969 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bbc0b4c50 con 0x7f2bbc0d0980 2023-09-23T03:24:15.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.868+0000 7f2bb959a640 1 -- 172.21.15.154:0/1566006969 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f2ba4001670 con 0x7f2bbc0d0980 2023-09-23T03:24:15.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.868+0000 7f2bb959a640 1 -- 172.21.15.154:0/1566006969 >> v1:172.21.15.154:6789/0 conn(0x7f2bbc0d0980 legacy=0x7f2bbc0d1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:15.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.869+0000 7f2bb959a640 1 --2- 172.21.15.154:0/1566006969 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ba80039a0 0x7f2ba8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:15.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.869+0000 7f2bb959a640 1 --2- 172.21.15.154:0/1566006969 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bbc15df50 0x7f2ba80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:15.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.869+0000 7f2bb959a640 1 --2- 172.21.15.154:0/1566006969 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ba8004d00 0x7f2ba8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:15.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.869+0000 7f2bb959a640 1 -- 172.21.15.154:0/1566006969 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc0cd870 con 0x7f2ba8004d00 2023-09-23T03:24:15.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.869+0000 7f2bb959a640 1 -- 172.21.15.154:0/1566006969 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba800ba90 con 0x7f2ba80039a0 2023-09-23T03:24:15.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.869+0000 7f2bb959a640 1 -- 172.21.15.154:0/1566006969 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba800bc50 con 0x7f2bbc15df50 2023-09-23T03:24:15.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.869+0000 7f2bbad9d640 1 --2- 172.21.15.154:0/1566006969 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ba8004d00 0x7f2ba8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:15.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.869+0000 7f2bb9d9b640 1 --2- 172.21.15.154:0/1566006969 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bbc15df50 0x7f2ba80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:15.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.869+0000 7f2bba59c640 1 --2- 172.21.15.154:0/1566006969 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ba80039a0 0x7f2ba8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:15.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.869+0000 7f2bbad9d640 1 -- 172.21.15.154:0/1566006969 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bbc15df50 msgr2=0x7f2ba80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:15.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.869+0000 7f2bbad9d640 1 --2- 172.21.15.154:0/1566006969 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bbc15df50 0x7f2ba80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:15.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.869+0000 7f2bbad9d640 1 -- 172.21.15.154:0/1566006969 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ba80039a0 msgr2=0x7f2ba8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:15.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.869+0000 7f2bbad9d640 1 --2- 172.21.15.154:0/1566006969 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ba80039a0 0x7f2ba8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:15.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.869+0000 7f2bbad9d640 1 -- 172.21.15.154:0/1566006969 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bbc0b4c50 con 0x7f2ba8004d00 2023-09-23T03:24:15.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.870+0000 7f2bbad9d640 1 --2- 172.21.15.154:0/1566006969 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ba8004d00 0x7f2ba8005180 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4001720 tx=0x7f2ba80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99e37cefe4dca03 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:15.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.871+0000 7f2bb959a640 1 -- 172.21.15.154:0/1566006969 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb40052d0 con 0x7f2ba8004d00 2023-09-23T03:24:15.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.871+0000 7f2bb959a640 1 -- 172.21.15.154:0/1566006969 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb402e050 con 0x7f2ba8004d00 2023-09-23T03:24:15.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.871+0000 7f2bb959a640 1 -- 172.21.15.154:0/1566006969 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb40035f0 con 0x7f2ba8004d00 2023-09-23T03:24:15.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.871+0000 7f2bbb59e640 1 -- 172.21.15.154:0/1566006969 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ba8004d00 msgr2=0x7f2ba8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:15.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.871+0000 7f2bbb59e640 1 --2- 172.21.15.154:0/1566006969 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ba8004d00 0x7f2ba8005180 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4001720 tx=0x7f2ba80057f0 comp rx=0 tx=0).stop 2023-09-23T03:24:15.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.871+0000 7f2bbad9d640 1 -- 172.21.15.154:0/1566006969 reap_dead start 2023-09-23T03:24:15.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.871+0000 7f2bbb59e640 1 -- 172.21.15.154:0/1566006969 shutdown_connections 2023-09-23T03:24:15.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.871+0000 7f2bbb59e640 1 -- 172.21.15.154:0/1566006969 >> 172.21.15.154:0/1566006969 conn(0x7f2bbc09bb20 msgr2=0x7f2bbc0d2870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:15.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.871+0000 7f2bbb59e640 1 -- 172.21.15.154:0/1566006969 shutdown_connections 2023-09-23T03:24:15.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.871+0000 7f2bbb59e640 1 -- 172.21.15.154:0/1566006969 wait complete. 2023-09-23T03:24:15.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.873+0000 7f2bbb59e640 1 Processor -- start 2023-09-23T03:24:15.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.873+0000 7f2bbb59e640 1 -- start start 2023-09-23T03:24:15.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.874+0000 7f2bbb59e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ba8004d00 0x7f2bbc1602a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:15.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.874+0000 7f2bbb59e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bbc14eff0 0x7f2bbc0ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:15.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.874+0000 7f2bbb59e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bbc0c81e0 0x7f2bbc0c8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:15.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.874+0000 7f2bbb59e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc0c8bd0 con 0x7f2bbc0c81e0 2023-09-23T03:24:15.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.874+0000 7f2bbb59e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc0cab10 con 0x7f2ba8004d00 2023-09-23T03:24:15.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.874+0000 7f2bbb59e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc0cadf0 con 0x7f2bbc14eff0 2023-09-23T03:24:15.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.875+0000 7f2bbad9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bbc0c81e0 0x7f2bbc0c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:15.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.875+0000 7f2bbad9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bbc0c81e0 0x7f2bbc0c8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50852/0 (socket says 172.21.15.154:50852) 2023-09-23T03:24:15.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.875+0000 7f2bbad9d640 1 -- 172.21.15.154:0/94559755 learned_addr learned my addr 172.21.15.154:0/94559755 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:15.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.875+0000 7f2bb9d9b640 1 --2- 172.21.15.154:0/94559755 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bbc14eff0 0x7f2bbc0ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:15.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.875+0000 7f2bba59c640 1 --2- 172.21.15.154:0/94559755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ba8004d00 0x7f2bbc1602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:15.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.875+0000 7f2bbad9d640 1 -- 172.21.15.154:0/94559755 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bbc14eff0 msgr2=0x7f2bbc0ca2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:15.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.875+0000 7f2bbad9d640 1 --2- 172.21.15.154:0/94559755 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bbc14eff0 0x7f2bbc0ca2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:15.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.875+0000 7f2bbad9d640 1 -- 172.21.15.154:0/94559755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ba8004d00 msgr2=0x7f2bbc1602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:15.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.875+0000 7f2bbad9d640 1 --2- 172.21.15.154:0/94559755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ba8004d00 0x7f2bbc1602a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:15.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.875+0000 7f2bbad9d640 1 -- 172.21.15.154:0/94559755 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bbc0cb0d0 con 0x7f2bbc0c81e0 2023-09-23T03:24:15.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.876+0000 7f2bbad9d640 1 --2- 172.21.15.154:0/94559755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bbc0c81e0 0x7f2bbc0c8690 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4005040 tx=0x7f2bb4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:15.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.876+0000 7f2bb2ffd640 1 -- 172.21.15.154:0/94559755 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb4043070 con 0x7f2bbc0c81e0 2023-09-23T03:24:15.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.876+0000 7f2bb2ffd640 1 -- 172.21.15.154:0/94559755 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb4007a30 con 0x7f2bbc0c81e0 2023-09-23T03:24:15.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.876+0000 7f2bbb59e640 1 -- 172.21.15.154:0/94559755 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bbc0cb3b0 con 0x7f2bbc0c81e0 2023-09-23T03:24:15.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.876+0000 7f2bb2ffd640 1 -- 172.21.15.154:0/94559755 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb40067d0 con 0x7f2bbc0c81e0 2023-09-23T03:24:15.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.876+0000 7f2bbb59e640 1 -- 172.21.15.154:0/94559755 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bbc0ca800 con 0x7f2bbc0c81e0 2023-09-23T03:24:15.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.877+0000 7f2bbb59e640 1 -- 172.21.15.154:0/94559755 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bbc0cb8d0 con 0x7f2bbc0c81e0 2023-09-23T03:24:15.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.878+0000 7f2bb2ffd640 1 -- 172.21.15.154:0/94559755 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f2bb4006950 con 0x7f2bbc0c81e0 2023-09-23T03:24:15.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.879+0000 7f2bb2ffd640 1 --2- 172.21.15.154:0/94559755 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2b9c06af30 0x7f2b9c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:15.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.879+0000 7f2bb2ffd640 1 -- 172.21.15.154:0/94559755 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2bb40aa6f0 con 0x7f2bbc0c81e0 2023-09-23T03:24:15.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.879+0000 7f2bba59c640 1 --2- 172.21.15.154:0/94559755 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2b9c06af30 0x7f2b9c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:15.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.880+0000 7f2bba59c640 1 --2- 172.21.15.154:0/94559755 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2b9c06af30 0x7f2b9c06d3f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2bac004330 tx=0x7f2bac00b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:15.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:15.884+0000 7f2bb2ffd640 1 -- 172.21.15.154:0/94559755 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f2bbc0cb8d0 con 0x7f2bbc0c81e0 2023-09-23T03:24:16.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.142+0000 7f2bbb59e640 1 -- 172.21.15.154:0/94559755 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f2bbc0cb8d0 con 0x7f2bbc0c81e0 2023-09-23T03:24:16.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.148+0000 7f2bb2ffd640 1 -- 172.21.15.154:0/94559755 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7f2bbc0cb8d0 con 0x7f2bbc0c81e0 2023-09-23T03:24:16.149 INFO:teuthology.orchestra.run.smithi154.stdout:[client.0] 2023-09-23T03:24:16.149 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQAHWA5lYg+qGhAAjaY0fPsmRTz3ipsKbPugMA== 2023-09-23T03:24:16.149 INFO:teuthology.orchestra.run.smithi154.stdout: caps mds = "allow" 2023-09-23T03:24:16.149 INFO:teuthology.orchestra.run.smithi154.stdout: caps mon = "allow r" 2023-09-23T03:24:16.149 INFO:teuthology.orchestra.run.smithi154.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-09-23T03:24:16.150 INFO:teuthology.orchestra.run.smithi154.stderr:updated caps for client.0 2023-09-23T03:24:16.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.151+0000 7f2bbb59e640 1 -- 172.21.15.154:0/94559755 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2b9c06af30 msgr2=0x7f2b9c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:16.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.151+0000 7f2bbb59e640 1 --2- 172.21.15.154:0/94559755 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2b9c06af30 0x7f2b9c06d3f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2bac004330 tx=0x7f2bac00b040 comp rx=0 tx=0).stop 2023-09-23T03:24:16.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.151+0000 7f2bbb59e640 1 -- 172.21.15.154:0/94559755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bbc0c81e0 msgr2=0x7f2bbc0c8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:16.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.151+0000 7f2bbb59e640 1 --2- 172.21.15.154:0/94559755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bbc0c81e0 0x7f2bbc0c8690 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4005040 tx=0x7f2bb4001120 comp rx=0 tx=0).stop 2023-09-23T03:24:16.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.151+0000 7f2bbb59e640 1 -- 172.21.15.154:0/94559755 shutdown_connections 2023-09-23T03:24:16.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.151+0000 7f2bbb59e640 1 --2- 172.21.15.154:0/94559755 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2b9c06af30 0x7f2b9c06d3f0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:16.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.151+0000 7f2bbb59e640 1 --2- 172.21.15.154:0/94559755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2bbc0c81e0 0x7f2bbc0c8690 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:16.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.151+0000 7f2bbb59e640 1 --2- 172.21.15.154:0/94559755 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2bbc14eff0 0x7f2bbc0ca2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:16.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.151+0000 7f2bbb59e640 1 --2- 172.21.15.154:0/94559755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ba8004d00 0x7f2bbc1602a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:16.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.151+0000 7f2bbb59e640 1 -- 172.21.15.154:0/94559755 >> 172.21.15.154:0/94559755 conn(0x7f2bbc09bb20 msgr2=0x7f2bbc0b18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:16.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.151+0000 7f2bbb59e640 1 -- 172.21.15.154:0/94559755 shutdown_connections 2023-09-23T03:24:16.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.151+0000 7f2bbb59e640 1 -- 172.21.15.154:0/94559755 wait complete. 2023-09-23T03:24:16.170 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:24:16.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.376+0000 7f061a81a640 1 Processor -- start 2023-09-23T03:24:16.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.376+0000 7f061a81a640 1 -- start start 2023-09-23T03:24:16.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.377+0000 7f061a81a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f06140ada30 0x7f06140ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:16.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.377+0000 7f061a81a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f06140b2fe0 0x7f06140cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:16.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.377+0000 7f061a81a640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f061406ceb0 con 0x7f06140cef60 2023-09-23T03:24:16.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.377+0000 7f061a81a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06140cf410 con 0x7f06140b2fe0 2023-09-23T03:24:16.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.377+0000 7f061a81a640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06140ade10 con 0x7f06140ada30 2023-09-23T03:24:16.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.378+0000 7f0618db7640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f06140cef60 0x7f06140b18f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:59538/0 (socket says 172.21.15.154:59538) 2023-09-23T03:24:16.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.378+0000 7f0618db7640 1 -- 172.21.15.154:0/1200401615 learned_addr learned my addr 172.21.15.154:0/1200401615 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:24:16.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.378+0000 7f060bfff640 1 --2- 172.21.15.154:0/1200401615 >> v2:172.21.15.154:3300/0 conn(0x7f06140b2fe0 0x7f06140cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:16.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.378+0000 7f0613fff640 1 --2- 172.21.15.154:0/1200401615 >> v2:172.21.15.190:3300/0 conn(0x7f06140ada30 0x7f06140ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:16.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.378+0000 7f06137fe640 1 -- 172.21.15.154:0/1200401615 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2657991227 0 0) 0x7f060c001490 con 0x7f06140cef60 2023-09-23T03:24:16.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.378+0000 7f06137fe640 1 -- 172.21.15.154:0/1200401615 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0600003520 con 0x7f06140cef60 2023-09-23T03:24:16.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.378+0000 7f060bfff640 1 -- 172.21.15.154:0/1200401615 >> v2:172.21.15.190:3300/0 conn(0x7f06140ada30 msgr2=0x7f06140ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:16.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.378+0000 7f060bfff640 1 --2- 172.21.15.154:0/1200401615 >> v2:172.21.15.190:3300/0 conn(0x7f06140ada30 0x7f06140ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:16.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.378+0000 7f060bfff640 1 -- 172.21.15.154:0/1200401615 >> v1:172.21.15.154:6789/0 conn(0x7f06140cef60 legacy=0x7f06140b18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:16.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.378+0000 7f060bfff640 1 -- 172.21.15.154:0/1200401615 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0614159c90 con 0x7f06140b2fe0 2023-09-23T03:24:16.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.380+0000 7f060bfff640 1 --2- 172.21.15.154:0/1200401615 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06140b2fe0 0x7f06140cea20 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f061406d990 tx=0x7f05fc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb9494a6a107ef69 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:16.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.380+0000 7f06137fe640 1 -- 172.21.15.154:0/1200401615 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc031830 con 0x7f06140b2fe0 2023-09-23T03:24:16.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.380+0000 7f06137fe640 1 -- 172.21.15.154:0/1200401615 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05fc031990 con 0x7f06140b2fe0 2023-09-23T03:24:16.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.380+0000 7f06137fe640 1 -- 172.21.15.154:0/1200401615 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc031d20 con 0x7f06140b2fe0 2023-09-23T03:24:16.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.380+0000 7f061a81a640 1 -- 172.21.15.154:0/1200401615 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06140b2fe0 msgr2=0x7f06140cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:16.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.380+0000 7f061a81a640 1 --2- 172.21.15.154:0/1200401615 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06140b2fe0 0x7f06140cea20 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f061406d990 tx=0x7f05fc02b8d0 comp rx=0 tx=0).stop 2023-09-23T03:24:16.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.380+0000 7f061a81a640 1 -- 172.21.15.154:0/1200401615 shutdown_connections 2023-09-23T03:24:16.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.380+0000 7f061a81a640 1 --2- 172.21.15.154:0/1200401615 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06140b2fe0 0x7f06140cea20 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:16.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.380+0000 7f061a81a640 1 --2- 172.21.15.154:0/1200401615 >> v2:172.21.15.190:3300/0 conn(0x7f06140ada30 0x7f06140ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:16.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.380+0000 7f061a81a640 1 -- 172.21.15.154:0/1200401615 >> 172.21.15.154:0/1200401615 conn(0x7f061406a0f0 msgr2=0x7f061406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:16.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.381+0000 7f061a81a640 1 -- 172.21.15.154:0/1200401615 shutdown_connections 2023-09-23T03:24:16.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.381+0000 7f061a81a640 1 -- 172.21.15.154:0/1200401615 wait complete. 2023-09-23T03:24:16.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.382+0000 7f061a81a640 1 Processor -- start 2023-09-23T03:24:16.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.382+0000 7f061a81a640 1 -- start start 2023-09-23T03:24:16.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.383+0000 7f061a81a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06140ada30 0x7f06140ab960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:16.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.383+0000 7f061a81a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06140b2fe0 0x7f0614157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:16.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.383+0000 7f061a81a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06140cef60 0x7f0614158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:16.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.383+0000 7f061a81a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06141f6270 con 0x7f06140cef60 2023-09-23T03:24:16.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.383+0000 7f061a81a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06141f5680 con 0x7f06140b2fe0 2023-09-23T03:24:16.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.383+0000 7f061a81a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06141f5940 con 0x7f06140ada30 2023-09-23T03:24:16.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.384+0000 7f0618db7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06140cef60 0x7f0614158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:16.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.384+0000 7f0618db7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06140cef60 0x7f0614158500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50874/0 (socket says 172.21.15.154:50874) 2023-09-23T03:24:16.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.384+0000 7f0618db7640 1 -- 172.21.15.154:0/4039382346 learned_addr learned my addr 172.21.15.154:0/4039382346 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:16.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.384+0000 7f0613fff640 1 --2- 172.21.15.154:0/4039382346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06140ada30 0x7f06140ab960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:16.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.384+0000 7f0618db7640 1 -- 172.21.15.154:0/4039382346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06140ada30 msgr2=0x7f06140ab960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:16.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.384+0000 7f060bfff640 1 --2- 172.21.15.154:0/4039382346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06140b2fe0 0x7f0614157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:16.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.384+0000 7f0618db7640 1 --2- 172.21.15.154:0/4039382346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06140ada30 0x7f06140ab960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:16.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.384+0000 7f0618db7640 1 -- 172.21.15.154:0/4039382346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06140b2fe0 msgr2=0x7f0614157fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:16.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.384+0000 7f0618db7640 1 --2- 172.21.15.154:0/4039382346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06140b2fe0 0x7f0614157fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:16.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.384+0000 7f0618db7640 1 -- 172.21.15.154:0/4039382346 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05fc009400 con 0x7f06140cef60 2023-09-23T03:24:16.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.384+0000 7f0613fff640 1 --2- 172.21.15.154:0/4039382346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06140ada30 0x7f06140ab960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:16.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.384+0000 7f0618db7640 1 --2- 172.21.15.154:0/4039382346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06140cef60 0x7f0614158500 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f060c00cb40 tx=0x7f060c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:16.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.385+0000 7f06117fa640 1 -- 172.21.15.154:0/4039382346 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060c00aca0 con 0x7f06140cef60 2023-09-23T03:24:16.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.385+0000 7f06117fa640 1 -- 172.21.15.154:0/4039382346 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f060c00ae00 con 0x7f06140cef60 2023-09-23T03:24:16.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.385+0000 7f06117fa640 1 -- 172.21.15.154:0/4039382346 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060c012620 con 0x7f06140cef60 2023-09-23T03:24:16.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.385+0000 7f061a81a640 1 -- 172.21.15.154:0/4039382346 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06141f5c20 con 0x7f06140cef60 2023-09-23T03:24:16.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.385+0000 7f061a81a640 1 -- 172.21.15.154:0/4039382346 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f061415b390 con 0x7f06140cef60 2023-09-23T03:24:16.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.386+0000 7f061a81a640 1 -- 172.21.15.154:0/4039382346 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06141f63e0 con 0x7f06140cef60 2023-09-23T03:24:16.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.387+0000 7f06117fa640 1 -- 172.21.15.154:0/4039382346 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f060c007070 con 0x7f06140cef60 2023-09-23T03:24:16.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.388+0000 7f06117fa640 1 --2- 172.21.15.154:0/4039382346 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f05f406af30 0x7f05f406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:16.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.388+0000 7f06117fa640 1 -- 172.21.15.154:0/4039382346 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f060c089770 con 0x7f06140cef60 2023-09-23T03:24:16.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.388+0000 7f0613fff640 1 --2- 172.21.15.154:0/4039382346 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f05f406af30 0x7f05f406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:16.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.389+0000 7f0613fff640 1 --2- 172.21.15.154:0/4039382346 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f05f406af30 0x7f05f406d3f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f061409b730 tx=0x7f0604008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:16.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.393+0000 7f06117fa640 1 -- 172.21.15.154:0/4039382346 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f060c054670 con 0x7f06140cef60 2023-09-23T03:24:16.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.620+0000 7f061a81a640 1 -- 172.21.15.154:0/4039382346 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f061406ceb0 con 0x7f06140cef60 2023-09-23T03:24:16.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.621+0000 7f06117fa640 1 -- 172.21.15.154:0/4039382346 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f061406ceb0 con 0x7f06140cef60 2023-09-23T03:24:16.623 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:16.623 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":62,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/728936136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":728936136},{"type":"v1","addr":"172.21.15.190:6833","nonce":728936136}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":5890,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2277229238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2277229238},{"type":"v1","addr":"172.21.15.190:6835","nonce":2277229238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":7257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/4245928089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4245928089},{"type":"v1","addr":"172.21.15.154:6835","nonce":4245928089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:24:10.061474+0000","modified":"2023-09-23T03:24:13.096320+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7260":{"gid":7260,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6837/2213283923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2213283923},{"type":"v1","addr":"172.21.15.154:6837","nonce":2213283923}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-09-23T03:24:16.623 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 62 2023-09-23T03:24:16.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.624+0000 7f061a81a640 1 -- 172.21.15.154:0/4039382346 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f05f406af30 msgr2=0x7f05f406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:16.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.624+0000 7f061a81a640 1 --2- 172.21.15.154:0/4039382346 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f05f406af30 0x7f05f406d3f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f061409b730 tx=0x7f0604008040 comp rx=0 tx=0).stop 2023-09-23T03:24:16.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.624+0000 7f061a81a640 1 -- 172.21.15.154:0/4039382346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06140cef60 msgr2=0x7f0614158500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:16.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.624+0000 7f061a81a640 1 --2- 172.21.15.154:0/4039382346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06140cef60 0x7f0614158500 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f060c00cb40 tx=0x7f060c00a060 comp rx=0 tx=0).stop 2023-09-23T03:24:16.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.624+0000 7f061a81a640 1 -- 172.21.15.154:0/4039382346 shutdown_connections 2023-09-23T03:24:16.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.624+0000 7f061a81a640 1 --2- 172.21.15.154:0/4039382346 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f05f406af30 0x7f05f406d3f0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:16.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.624+0000 7f061a81a640 1 --2- 172.21.15.154:0/4039382346 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06140cef60 0x7f0614158500 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:16.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.624+0000 7f061a81a640 1 --2- 172.21.15.154:0/4039382346 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06140b2fe0 0x7f0614157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:16.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.624+0000 7f061a81a640 1 --2- 172.21.15.154:0/4039382346 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06140ada30 0x7f06140ab960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:16.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.624+0000 7f061a81a640 1 -- 172.21.15.154:0/4039382346 >> 172.21.15.154:0/4039382346 conn(0x7f061406a0f0 msgr2=0x7f06140adf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:16.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.624+0000 7f061a81a640 1 -- 172.21.15.154:0/4039382346 shutdown_connections 2023-09-23T03:24:16.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:16.624+0000 7f061a81a640 1 -- 172.21.15.154:0/4039382346 wait complete. 2023-09-23T03:24:17.045 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 62, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-23T03:24:10.061474+0000', 'modified': '2023-09-23T03:24:13.096320+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7260}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7260': {'gid': 7260, 'name': 'a', 'rank': 0, 'incarnation': 59, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.154:6837/2213283923', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6836', 'nonce': 2213283923}, {'type': 'v1', 'addr': '172.21.15.154:6837', 'nonce': 2213283923}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-23T03:24:17.045 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-23T03:24:17.045 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-09-23T03:24:17.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.268+0000 7ff687580640 1 Processor -- start 2023-09-23T03:24:17.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.269+0000 7ff687580640 1 -- start start 2023-09-23T03:24:17.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.270+0000 7ff687580640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff6800cf5a0 0x7ff680167800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:17.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.270+0000 7ff687580640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6800d34f0 con 0x7ff68015ffd0 2023-09-23T03:24:17.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.270+0000 7ff687580640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6800cdf40 con 0x7ff6800b5520 2023-09-23T03:24:17.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.270+0000 7ff687580640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6800d04d0 con 0x7ff6800cf5a0 2023-09-23T03:24:17.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.270+0000 7ff68657e640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff68015ffd0 0x7ff6800d2de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:59548/0 (socket says 172.21.15.154:59548) 2023-09-23T03:24:17.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.270+0000 7ff68657e640 1 -- 172.21.15.154:0/2587414513 learned_addr learned my addr 172.21.15.154:0/2587414513 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:24:17.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.271+0000 7ff68557c640 1 -- 172.21.15.154:0/2587414513 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2205167547 0 0) 0x7ff6800d34f0 con 0x7ff68015ffd0 2023-09-23T03:24:17.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.271+0000 7ff68557c640 1 -- 172.21.15.154:0/2587414513 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff668003520 con 0x7ff68015ffd0 2023-09-23T03:24:17.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.271+0000 7ff686d7f640 1 --2- 172.21.15.154:0/2587414513 >> v2:172.21.15.190:3300/0 conn(0x7ff6800cf5a0 0x7ff680167800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:17.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.271+0000 7ff68557c640 1 -- 172.21.15.154:0/2587414513 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1357336646 0 0) 0x7ff668003520 con 0x7ff68015ffd0 2023-09-23T03:24:17.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.271+0000 7ff68557c640 1 -- 172.21.15.154:0/2587414513 >> v2:172.21.15.190:3300/0 conn(0x7ff6800cf5a0 msgr2=0x7ff680167800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:17.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.271+0000 7ff68557c640 1 --2- 172.21.15.154:0/2587414513 >> v2:172.21.15.190:3300/0 conn(0x7ff6800cf5a0 0x7ff680167800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.271+0000 7ff68557c640 1 -- 172.21.15.154:0/2587414513 >> v1:172.21.15.190:6789/0 conn(0x7ff6800b5520 legacy=0x7ff6800cee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:17.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.271+0000 7ff68557c640 1 -- 172.21.15.154:0/2587414513 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6800ce220 con 0x7ff68015ffd0 2023-09-23T03:24:17.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.271+0000 7ff68557c640 1 -- 172.21.15.154:0/2587414513 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7ff670001b70 con 0x7ff68015ffd0 2023-09-23T03:24:17.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.271+0000 7ff68557c640 1 -- 172.21.15.154:0/2587414513 >> v1:172.21.15.154:6789/0 conn(0x7ff68015ffd0 legacy=0x7ff6800d2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:17.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.271+0000 7ff68557c640 1 --2- 172.21.15.154:0/2587414513 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff668003970 0x7ff668003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:17.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.271+0000 7ff68557c640 1 --2- 172.21.15.154:0/2587414513 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6680047c0 0x7ff668004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:17.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.271+0000 7ff68557c640 1 --2- 172.21.15.154:0/2587414513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff668005180 0x7ff66800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:17.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.271+0000 7ff68557c640 1 -- 172.21.15.154:0/2587414513 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6800d04d0 con 0x7ff6680047c0 2023-09-23T03:24:17.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.271+0000 7ff68557c640 1 -- 172.21.15.154:0/2587414513 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66800c0d0 con 0x7ff668005180 2023-09-23T03:24:17.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.271+0000 7ff68557c640 1 -- 172.21.15.154:0/2587414513 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff66800c290 con 0x7ff668003970 2023-09-23T03:24:17.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.272+0000 7ff685d7d640 1 --2- 172.21.15.154:0/2587414513 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6680047c0 0x7ff668004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:17.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.272+0000 7ff68657e640 1 --2- 172.21.15.154:0/2587414513 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff668003970 0x7ff668003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:17.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.272+0000 7ff686d7f640 1 --2- 172.21.15.154:0/2587414513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff668005180 0x7ff66800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:17.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.272+0000 7ff685d7d640 1 -- 172.21.15.154:0/2587414513 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff668003970 msgr2=0x7ff668003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:17.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.272+0000 7ff685d7d640 1 --2- 172.21.15.154:0/2587414513 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff668003970 0x7ff668003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.272+0000 7ff685d7d640 1 -- 172.21.15.154:0/2587414513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff668005180 msgr2=0x7ff66800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:17.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.272+0000 7ff685d7d640 1 --2- 172.21.15.154:0/2587414513 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff668005180 0x7ff66800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.272+0000 7ff685d7d640 1 -- 172.21.15.154:0/2587414513 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff668003520 con 0x7ff6680047c0 2023-09-23T03:24:17.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.272+0000 7ff686d7f640 1 -- 172.21.15.154:0/2587414513 reap_dead start 2023-09-23T03:24:17.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.273+0000 7ff685d7d640 1 --2- 172.21.15.154:0/2587414513 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6680047c0 0x7ff668004c40 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7ff674001720 tx=0x7ff6680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb63b861725d82b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:17.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.273+0000 7ff68557c640 1 -- 172.21.15.154:0/2587414513 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6740052d0 con 0x7ff6680047c0 2023-09-23T03:24:17.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.273+0000 7ff68557c640 1 -- 172.21.15.154:0/2587414513 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67402e050 con 0x7ff6680047c0 2023-09-23T03:24:17.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.273+0000 7ff68557c640 1 -- 172.21.15.154:0/2587414513 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674003830 con 0x7ff6680047c0 2023-09-23T03:24:17.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.274+0000 7ff687580640 1 -- 172.21.15.154:0/2587414513 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6680047c0 msgr2=0x7ff668004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:17.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.274+0000 7ff687580640 1 --2- 172.21.15.154:0/2587414513 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6680047c0 0x7ff668004c40 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7ff674001720 tx=0x7ff6680057f0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.274+0000 7ff687580640 1 -- 172.21.15.154:0/2587414513 shutdown_connections 2023-09-23T03:24:17.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.274+0000 7ff687580640 1 --2- 172.21.15.154:0/2587414513 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6680047c0 0x7ff668004c40 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.274+0000 7ff687580640 1 -- 172.21.15.154:0/2587414513 >> 172.21.15.154:0/2587414513 conn(0x7ff68009ba50 msgr2=0x7ff6800d26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:17.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.274+0000 7ff687580640 1 -- 172.21.15.154:0/2587414513 shutdown_connections 2023-09-23T03:24:17.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.274+0000 7ff687580640 1 -- 172.21.15.154:0/2587414513 wait complete. 2023-09-23T03:24:17.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.276+0000 7ff687580640 1 Processor -- start 2023-09-23T03:24:17.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.276+0000 7ff687580640 1 -- start start 2023-09-23T03:24:17.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.277+0000 7ff687580640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6680047c0 0x7ff68014d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:17.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.277+0000 7ff687580640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6800ca1b0 0x7ff6800c8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:17.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.277+0000 7ff687580640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6800c8550 0x7ff68015d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:17.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.277+0000 7ff687580640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6800c89e0 con 0x7ff6800ca1b0 2023-09-23T03:24:17.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.277+0000 7ff687580640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6800cb2c0 con 0x7ff6800c8550 2023-09-23T03:24:17.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.277+0000 7ff687580640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6800ca6b0 con 0x7ff6680047c0 2023-09-23T03:24:17.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.277+0000 7ff685d7d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6800ca1b0 0x7ff6800c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:17.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.278+0000 7ff685d7d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6800ca1b0 0x7ff6800c8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50892/0 (socket says 172.21.15.154:50892) 2023-09-23T03:24:17.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.278+0000 7ff685d7d640 1 -- 172.21.15.154:0/3010326572 learned_addr learned my addr 172.21.15.154:0/3010326572 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:17.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.278+0000 7ff68657e640 1 --2- 172.21.15.154:0/3010326572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6680047c0 0x7ff68014d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:17.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.278+0000 7ff686d7f640 1 --2- 172.21.15.154:0/3010326572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6800c8550 0x7ff68015d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:17.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.278+0000 7ff685d7d640 1 -- 172.21.15.154:0/3010326572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6680047c0 msgr2=0x7ff68014d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:17.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.278+0000 7ff685d7d640 1 --2- 172.21.15.154:0/3010326572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6680047c0 0x7ff68014d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.278+0000 7ff685d7d640 1 -- 172.21.15.154:0/3010326572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6800c8550 msgr2=0x7ff68015d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:17.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.278+0000 7ff685d7d640 1 --2- 172.21.15.154:0/3010326572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6800c8550 0x7ff68015d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.278+0000 7ff685d7d640 1 -- 172.21.15.154:0/3010326572 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6800ca990 con 0x7ff6800ca1b0 2023-09-23T03:24:17.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.278+0000 7ff686d7f640 1 --2- 172.21.15.154:0/3010326572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6800c8550 0x7ff68015d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:17.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.278+0000 7ff685d7d640 1 --2- 172.21.15.154:0/3010326572 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6800ca1b0 0x7ff6800c8010 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7ff67402bbc0 tx=0x7ff674001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:17.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.279+0000 7ff67effd640 1 -- 172.21.15.154:0/3010326572 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6740076c0 con 0x7ff6800ca1b0 2023-09-23T03:24:17.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.279+0000 7ff67effd640 1 -- 172.21.15.154:0/3010326572 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff674007a30 con 0x7ff6800ca1b0 2023-09-23T03:24:17.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.279+0000 7ff687580640 1 -- 172.21.15.154:0/3010326572 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6800cac70 con 0x7ff6800ca1b0 2023-09-23T03:24:17.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.279+0000 7ff67effd640 1 -- 172.21.15.154:0/3010326572 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674006be0 con 0x7ff6800ca1b0 2023-09-23T03:24:17.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.279+0000 7ff687580640 1 -- 172.21.15.154:0/3010326572 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff68014df30 con 0x7ff6800ca1b0 2023-09-23T03:24:17.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.281+0000 7ff67effd640 1 -- 172.21.15.154:0/3010326572 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7ff674006d40 con 0x7ff6800ca1b0 2023-09-23T03:24:17.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.282+0000 7ff67effd640 1 --2- 172.21.15.154:0/3010326572 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff66c06af60 0x7ff66c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:17.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.282+0000 7ff67effd640 1 -- 172.21.15.154:0/3010326572 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff6740ab500 con 0x7ff6800ca1b0 2023-09-23T03:24:17.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.282+0000 7ff68657e640 1 --2- 172.21.15.154:0/3010326572 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff66c06af60 0x7ff66c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:17.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.283+0000 7ff68b373740 1 Processor -- start 2023-09-23T03:24:17.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.283+0000 7ff68b373740 1 -- start start 2023-09-23T03:24:17.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.283+0000 7ff68657e640 1 --2- 172.21.15.154:0/3010326572 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff66c06af60 0x7ff66c06d420 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff670001270 tx=0x7ff670009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:17.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.284+0000 7ff68b373740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdbf1e0da0 0x55bdbf1b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:17.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.284+0000 7ff68b373740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bdbf1e1180 0x55bdbf1e3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:17.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.284+0000 7ff68b373740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bdbf1b41e0 0x55bdbf1e4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:17.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.284+0000 7ff68b373740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdbf1276d0 con 0x55bdbf1e1180 2023-09-23T03:24:17.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.284+0000 7ff68b373740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdbf124dc0 con 0x55bdbf1e0da0 2023-09-23T03:24:17.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.284+0000 7ff68b373740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bdbf1206b0 con 0x55bdbf1b41e0 2023-09-23T03:24:17.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.284+0000 7ff685d7d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bdbf1e1180 0x55bdbf1e3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:17.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.284+0000 7ff685d7d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bdbf1e1180 0x55bdbf1e3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50894/0 (socket says 172.21.15.154:50894) 2023-09-23T03:24:17.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.284+0000 7ff685d7d640 1 -- 172.21.15.154:0/2362744231 learned_addr learned my addr 172.21.15.154:0/2362744231 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:17.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.285+0000 7ff685d7d640 1 -- 172.21.15.154:0/2362744231 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bdbf1b41e0 msgr2=0x55bdbf1e4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:17.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.285+0000 7ff68657e640 1 --2- 172.21.15.154:0/2362744231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdbf1e0da0 0x55bdbf1b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:17.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.285+0000 7ff685d7d640 1 --2- 172.21.15.154:0/2362744231 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bdbf1b41e0 0x55bdbf1e4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.285+0000 7ff685d7d640 1 -- 172.21.15.154:0/2362744231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdbf1e0da0 msgr2=0x55bdbf1b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:17.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.285+0000 7ff685d7d640 1 --2- 172.21.15.154:0/2362744231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdbf1e0da0 0x55bdbf1b3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.285+0000 7ff685d7d640 1 -- 172.21.15.154:0/2362744231 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdbf0e48e0 con 0x55bdbf1e1180 2023-09-23T03:24:17.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.285+0000 7ff68657e640 1 --2- 172.21.15.154:0/2362744231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdbf1e0da0 0x55bdbf1b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:17.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.285+0000 7ff685d7d640 1 --2- 172.21.15.154:0/2362744231 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bdbf1e1180 0x55bdbf1e3b30 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7ff674081e80 tx=0x7ff67407a870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7f660bba0bba8c5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:17.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.285+0000 7ff667fff640 1 -- 172.21.15.154:0/2362744231 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6740823b0 con 0x55bdbf1e1180 2023-09-23T03:24:17.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.285+0000 7ff667fff640 1 -- 172.21.15.154:0/2362744231 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff674001370 con 0x55bdbf1e1180 2023-09-23T03:24:17.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.285+0000 7ff667fff640 1 -- 172.21.15.154:0/2362744231 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6740767c0 con 0x55bdbf1e1180 2023-09-23T03:24:17.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.286+0000 7ff68b373740 1 -- 172.21.15.154:0/2362744231 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bdbf1e1180 msgr2=0x55bdbf1e3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:17.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.286+0000 7ff68b373740 1 --2- 172.21.15.154:0/2362744231 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bdbf1e1180 0x55bdbf1e3b30 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7ff674081e80 tx=0x7ff67407a870 comp rx=0 tx=0).stop 2023-09-23T03:24:17.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.286+0000 7ff68b373740 1 -- 172.21.15.154:0/2362744231 shutdown_connections 2023-09-23T03:24:17.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.286+0000 7ff68b373740 1 --2- 172.21.15.154:0/2362744231 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bdbf1b41e0 0x55bdbf1e4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.286+0000 7ff68b373740 1 --2- 172.21.15.154:0/2362744231 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bdbf1e1180 0x55bdbf1e3b30 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.286+0000 7ff68b373740 1 --2- 172.21.15.154:0/2362744231 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdbf1e0da0 0x55bdbf1b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.286+0000 7ff68b373740 1 -- 172.21.15.154:0/2362744231 >> 172.21.15.154:0/2362744231 conn(0x55bdbf1b5fb0 msgr2=0x55bdbf1feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:17.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.286+0000 7ff68b373740 1 -- 172.21.15.154:0/2362744231 shutdown_connections 2023-09-23T03:24:17.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.286+0000 7ff68b373740 1 -- 172.21.15.154:0/2362744231 wait complete. 2023-09-23T03:24:17.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.287+0000 7ff68b373740 1 -- start start 2023-09-23T03:24:17.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.287+0000 7ff68b373740 1 Processor -- start 2023-09-23T03:24:17.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.288+0000 7ff68b373740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bdbf1e0da0 0x55bdbf1e6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:17.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.288+0000 7ff68b373740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bdbf1e1180 0x55bdbf211d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:17.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.288+0000 7ff68b373740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdbf2122a0 0x55bdbf1f8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:17.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.288+0000 7ff68b373740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdbf0be940 con 0x55bdbf1e0da0 2023-09-23T03:24:17.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.288+0000 7ff68b373740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdbf0be5e0 con 0x55bdbf2122a0 2023-09-23T03:24:17.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.288+0000 7ff68b373740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bdbee84f20 con 0x55bdbf1e1180 2023-09-23T03:24:17.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.288+0000 7ff68b373740 10 client.0 _ll_register_callbacks cb 0x55bdbf0fc5e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:24:17.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.288+0000 7ff68657e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bdbf1e0da0 0x55bdbf1e6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:17.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.288+0000 7ff68657e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bdbf1e0da0 0x55bdbf1e6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50906/0 (socket says 172.21.15.154:50906) 2023-09-23T03:24:17.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.288+0000 7ff68657e640 1 -- 172.21.15.154:0/1278034482 learned_addr learned my addr 172.21.15.154:0/1278034482 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:17.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.289+0000 7ff686d7f640 1 --2- 172.21.15.154:0/1278034482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdbf2122a0 0x55bdbf1f8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:17.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.289+0000 7ff68657e640 1 -- 172.21.15.154:0/1278034482 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bdbf1e1180 msgr2=0x55bdbf211d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:17.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.289+0000 7ff68657e640 1 --2- 172.21.15.154:0/1278034482 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bdbf1e1180 0x55bdbf211d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.289+0000 7ff68657e640 1 -- 172.21.15.154:0/1278034482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdbf2122a0 msgr2=0x55bdbf1f8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:17.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.289+0000 7ff68657e640 1 --2- 172.21.15.154:0/1278034482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdbf2122a0 0x55bdbf1f8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.289+0000 7ff68657e640 1 -- 172.21.15.154:0/1278034482 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff68014df30 con 0x55bdbf1e0da0 2023-09-23T03:24:17.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.289+0000 7ff686d7f640 1 --2- 172.21.15.154:0/1278034482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdbf2122a0 0x55bdbf1f8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:17.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.289+0000 7ff68657e640 1 --2- 172.21.15.154:0/1278034482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bdbf1e0da0 0x55bdbf1e6580 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7ff6700144a0 tx=0x7ff670002120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db4e5cd77e2c8e3c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:17.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.289+0000 7ff6657fa640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:24:17.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.289+0000 7ff6657fa640 1 -- 172.21.15.154:0/1278034482 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67001f790 con 0x55bdbf1e0da0 2023-09-23T03:24:17.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.289+0000 7ff68b373740 1 -- 172.21.15.154:0/1278034482 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bdbf119690 con 0x55bdbf1e0da0 2023-09-23T03:24:17.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.289+0000 7ff6657fa640 1 -- 172.21.15.154:0/1278034482 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff67001f8f0 con 0x55bdbf1e0da0 2023-09-23T03:24:17.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.290+0000 7ff6657fa640 1 -- 172.21.15.154:0/1278034482 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff67001fab0 con 0x55bdbf1e0da0 2023-09-23T03:24:17.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.290+0000 7ff68b373740 10 client.7413 fetch_fsmap learned FSMap version 62 2023-09-23T03:24:17.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.290+0000 7ff68b373740 1 -- 172.21.15.154:0/1278034482 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x55bdbf11ed30 con 0x55bdbf1e0da0 2023-09-23T03:24:17.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.290+0000 7ff6657fa640 1 -- 172.21.15.154:0/1278034482 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bdbf119690 con 0x55bdbf1e0da0 2023-09-23T03:24:17.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.290+0000 7ff6657fa640 1 -- 172.21.15.154:0/1278034482 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff67001fc10 con 0x55bdbf1e0da0 2023-09-23T03:24:17.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.290+0000 7ff68b373740 10 client.7413 fetch_fsmap finished waiting for FSMap version 62 2023-09-23T03:24:17.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.290+0000 7ff68b373740 10 client.7413 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-23T03:24:17.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.290+0000 7ff68b373740 20 client.7413 populate_metadata read hostname 'smithi154' 2023-09-23T03:24:17.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.290+0000 7ff68b373740 1 --2- 172.21.15.154:0/1278034482 >> [v2:172.21.15.154:6836/2213283923,v1:172.21.15.154:6837/2213283923] conn(0x55bdbf1d10c0 0x55bdbf1d14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:17.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.290+0000 7ff68b373740 4 client.7413 mds_command: new command op to 7260 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T03:24:17.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.290+0000 7ff68b373740 1 -- 172.21.15.154:0/1278034482 --> [v2:172.21.15.154:6836/2213283923,v1:172.21.15.154:6837/2213283923] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bdbf1193b0 con 0x55bdbf1d10c0 2023-09-23T03:24:17.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.290+0000 7ff685d7d640 1 --2- 172.21.15.154:0/1278034482 >> [v2:172.21.15.154:6836/2213283923,v1:172.21.15.154:6837/2213283923] conn(0x55bdbf1d10c0 0x55bdbf1d14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:17.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.291+0000 7ff685d7d640 1 --2- 172.21.15.154:0/1278034482 >> [v2:172.21.15.154:6836/2213283923,v1:172.21.15.154:6837/2213283923] conn(0x55bdbf1d10c0 0x55bdbf1d14a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=221653a4c7420150 server_cookie=c29f2b8c354e9297 in_seq=0 out_seq=0 2023-09-23T03:24:17.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.291+0000 7ff6657fa640 10 client.7413 ms_handle_connect on v2:172.21.15.154:6836/2213283923 2023-09-23T03:24:17.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.292+0000 7ff6657fa640 1 -- 172.21.15.154:0/1278034482 <== mds.0 v2:172.21.15.154:6836/2213283923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bdbf1193b0 con 0x55bdbf1d10c0 2023-09-23T03:24:17.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.292+0000 7ff6657fa640 10 client.7413 handle_command_reply: tid=0 2023-09-23T03:24:17.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.292+0000 7ff68b373740 1 client.7413 shutdown 2023-09-23T03:24:17.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.292+0000 7ff68b373740 1 -- 172.21.15.154:0/1278034482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bdbf1e0da0 msgr2=0x55bdbf1e6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:17.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.292+0000 7ff68b373740 1 --2- 172.21.15.154:0/1278034482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bdbf1e0da0 0x55bdbf1e6580 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7ff6700144a0 tx=0x7ff670002120 comp rx=0 tx=0).stop 2023-09-23T03:24:17.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.292+0000 7ff68b373740 1 -- 172.21.15.154:0/1278034482 shutdown_connections 2023-09-23T03:24:17.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.292+0000 7ff68b373740 1 --2- 172.21.15.154:0/1278034482 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdbf2122a0 0x55bdbf1f8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.292+0000 7ff68b373740 1 --2- 172.21.15.154:0/1278034482 >> [v2:172.21.15.154:6836/2213283923,v1:172.21.15.154:6837/2213283923] conn(0x55bdbf1d10c0 0x55bdbf1d14a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.292+0000 7ff68b373740 1 --2- 172.21.15.154:0/1278034482 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bdbf1e1180 0x55bdbf211d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.292+0000 7ff6657fa640 0 client.7413 ms_handle_reset on v2:172.21.15.154:6836/2213283923 2023-09-23T03:24:17.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.292+0000 7ff68b373740 1 --2- 172.21.15.154:0/1278034482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bdbf1e0da0 0x55bdbf1e6580 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.292+0000 7ff68b373740 1 -- 172.21.15.154:0/1278034482 >> 172.21.15.154:0/1278034482 conn(0x55bdbf1b41e0 msgr2=0x55bdbf1b7b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:17.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.292+0000 7ff68b373740 1 -- 172.21.15.154:0/1278034482 shutdown_connections 2023-09-23T03:24:17.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.292+0000 7ff68b373740 1 -- 172.21.15.154:0/1278034482 wait complete. 2023-09-23T03:24:17.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.292+0000 7ff68b373740 20 client.7413 trim_cache size 0 max 16384 2023-09-23T03:24:17.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.309+0000 7ff68b373740 1 Processor -- start 2023-09-23T03:24:17.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.309+0000 7ff68b373740 1 -- start start 2023-09-23T03:24:17.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.310+0000 7ff68b373740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bdbf1e0da0 0x55bdbf1f9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:17.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.310+0000 7ff68b373740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bdbf1e1180 0x55bdbf1b7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:17.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.310+0000 7ff68b373740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdbf2122a0 0x55bdbf20d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:17.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.310+0000 7ff68b373740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdbf1fa210 con 0x55bdbf1e0da0 2023-09-23T03:24:17.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.310+0000 7ff68b373740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdbf1e3f70 con 0x55bdbf2122a0 2023-09-23T03:24:17.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.310+0000 7ff68b373740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bdbf1e76d0 con 0x55bdbf1e1180 2023-09-23T03:24:17.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.310+0000 7ff68657e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bdbf1e0da0 0x55bdbf1f9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:17.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.311+0000 7ff68657e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bdbf1e0da0 0x55bdbf1f9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50918/0 (socket says 172.21.15.154:50918) 2023-09-23T03:24:17.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.311+0000 7ff68657e640 1 -- 172.21.15.154:0/2226428917 learned_addr learned my addr 172.21.15.154:0/2226428917 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:17.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.311+0000 7ff685d7d640 1 --2- 172.21.15.154:0/2226428917 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bdbf1e1180 0x55bdbf1b7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:17.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.311+0000 7ff686d7f640 1 --2- 172.21.15.154:0/2226428917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdbf2122a0 0x55bdbf20d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:17.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.311+0000 7ff68657e640 1 -- 172.21.15.154:0/2226428917 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bdbf1e1180 msgr2=0x55bdbf1b7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:17.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.311+0000 7ff68657e640 1 --2- 172.21.15.154:0/2226428917 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bdbf1e1180 0x55bdbf1b7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.311+0000 7ff68657e640 1 -- 172.21.15.154:0/2226428917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdbf2122a0 msgr2=0x55bdbf20d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:17.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.311+0000 7ff68657e640 1 --2- 172.21.15.154:0/2226428917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdbf2122a0 0x55bdbf20d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.311+0000 7ff68657e640 1 -- 172.21.15.154:0/2226428917 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdbf0bdb50 con 0x55bdbf1e0da0 2023-09-23T03:24:17.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.311+0000 7ff68657e640 1 --2- 172.21.15.154:0/2226428917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bdbf1e0da0 0x55bdbf1f9670 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7ff670014470 tx=0x7ff670016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=122374fc631d5abf server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:17.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.311+0000 7ff67cff9640 1 -- 172.21.15.154:0/2226428917 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff670036070 con 0x55bdbf1e0da0 2023-09-23T03:24:17.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.312+0000 7ff67cff9640 1 -- 172.21.15.154:0/2226428917 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff670014d30 con 0x55bdbf1e0da0 2023-09-23T03:24:17.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.312+0000 7ff67cff9640 1 -- 172.21.15.154:0/2226428917 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6700207d0 con 0x55bdbf1e0da0 2023-09-23T03:24:17.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.312+0000 7ff68b373740 1 -- 172.21.15.154:0/2226428917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bdbf1e0da0 msgr2=0x55bdbf1f9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:17.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.312+0000 7ff68b373740 1 --2- 172.21.15.154:0/2226428917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bdbf1e0da0 0x55bdbf1f9670 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7ff670014470 tx=0x7ff670016000 comp rx=0 tx=0).stop 2023-09-23T03:24:17.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.312+0000 7ff68b373740 1 -- 172.21.15.154:0/2226428917 shutdown_connections 2023-09-23T03:24:17.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.312+0000 7ff68b373740 1 --2- 172.21.15.154:0/2226428917 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdbf2122a0 0x55bdbf20d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.312+0000 7ff68b373740 1 --2- 172.21.15.154:0/2226428917 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bdbf1e1180 0x55bdbf1b7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.312+0000 7ff68b373740 1 --2- 172.21.15.154:0/2226428917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bdbf1e0da0 0x55bdbf1f9670 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.312+0000 7ff68b373740 1 -- 172.21.15.154:0/2226428917 >> 172.21.15.154:0/2226428917 conn(0x55bdbf1b41e0 msgr2=0x55bdbf201090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:17.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.312+0000 7ff68b373740 1 -- 172.21.15.154:0/2226428917 shutdown_connections 2023-09-23T03:24:17.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.312+0000 7ff68b373740 1 -- 172.21.15.154:0/2226428917 wait complete. 2023-09-23T03:24:17.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.313+0000 7ff68b373740 1 -- start start 2023-09-23T03:24:17.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.313+0000 7ff68b373740 1 Processor -- start 2023-09-23T03:24:17.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.314+0000 7ff68b373740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdbf1e0da0 0x55bdbf1df510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:17.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.314+0000 7ff68b373740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bdbf1e1180 0x55bdbf201090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:17.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.314+0000 7ff68b373740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bdbf2122a0 0x55bdbf2015d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:17.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.314+0000 7ff68b373740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdbf2675e0 con 0x55bdbf2122a0 2023-09-23T03:24:17.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.314+0000 7ff68b373740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdbf1fdcf0 con 0x55bdbf1e0da0 2023-09-23T03:24:17.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.314+0000 7ff68b373740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bdbf1fdae0 con 0x55bdbf1e1180 2023-09-23T03:24:17.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.314+0000 7ff68b373740 10 client.0 _ll_register_callbacks cb 0x7ff6740763f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:24:17.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.314+0000 7ff686d7f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bdbf2122a0 0x55bdbf2015d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:17.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.314+0000 7ff686d7f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bdbf2122a0 0x55bdbf2015d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50926/0 (socket says 172.21.15.154:50926) 2023-09-23T03:24:17.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.314+0000 7ff686d7f640 1 -- 172.21.15.154:0/1803980891 learned_addr learned my addr 172.21.15.154:0/1803980891 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:17.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.315+0000 7ff685d7d640 1 --2- 172.21.15.154:0/1803980891 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bdbf1e1180 0x55bdbf201090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:17.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.315+0000 7ff68657e640 1 --2- 172.21.15.154:0/1803980891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdbf1e0da0 0x55bdbf1df510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:17.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.315+0000 7ff686d7f640 1 -- 172.21.15.154:0/1803980891 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bdbf1e1180 msgr2=0x55bdbf201090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:17.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.315+0000 7ff686d7f640 1 --2- 172.21.15.154:0/1803980891 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bdbf1e1180 0x55bdbf201090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.315+0000 7ff686d7f640 1 -- 172.21.15.154:0/1803980891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdbf1e0da0 msgr2=0x55bdbf1df510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:17.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.315+0000 7ff686d7f640 1 --2- 172.21.15.154:0/1803980891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdbf1e0da0 0x55bdbf1df510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.315+0000 7ff686d7f640 1 -- 172.21.15.154:0/1803980891 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bdbf11ed30 con 0x55bdbf2122a0 2023-09-23T03:24:17.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.315+0000 7ff68657e640 1 --2- 172.21.15.154:0/1803980891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdbf1e0da0 0x55bdbf1df510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:17.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.315+0000 7ff686d7f640 1 --2- 172.21.15.154:0/1803980891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bdbf2122a0 0x55bdbf2015d0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7ff68008bba0 tx=0x7ff67800df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb0224b6a7a5452 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:17.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.315+0000 7ff6657fa640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:24:17.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.315+0000 7ff6657fa640 1 -- 172.21.15.154:0/1803980891 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff678013c00 con 0x55bdbf2122a0 2023-09-23T03:24:17.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.315+0000 7ff6657fa640 1 -- 172.21.15.154:0/1803980891 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff678013d60 con 0x55bdbf2122a0 2023-09-23T03:24:17.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.315+0000 7ff68b373740 1 -- 172.21.15.154:0/1803980891 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff674001370 con 0x55bdbf2122a0 2023-09-23T03:24:17.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.316+0000 7ff6657fa640 1 -- 172.21.15.154:0/1803980891 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff678004020 con 0x55bdbf2122a0 2023-09-23T03:24:17.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.316+0000 7ff68b373740 10 client.7419 fetch_fsmap learned FSMap version 62 2023-09-23T03:24:17.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.316+0000 7ff68b373740 1 -- 172.21.15.154:0/1803980891 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x55bdbf0e3140 con 0x55bdbf2122a0 2023-09-23T03:24:17.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.316+0000 7ff6657fa640 1 -- 172.21.15.154:0/1803980891 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff674001370 con 0x55bdbf2122a0 2023-09-23T03:24:17.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.316+0000 7ff6657fa640 1 -- 172.21.15.154:0/1803980891 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff678004180 con 0x55bdbf2122a0 2023-09-23T03:24:17.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.316+0000 7ff68b373740 10 client.7419 fetch_fsmap finished waiting for FSMap version 62 2023-09-23T03:24:17.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.316+0000 7ff68b373740 10 client.7419 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-23T03:24:17.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.316+0000 7ff68b373740 20 client.7419 populate_metadata read hostname 'smithi154' 2023-09-23T03:24:17.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.316+0000 7ff68b373740 1 --2- 172.21.15.154:0/1803980891 >> [v2:172.21.15.154:6836/2213283923,v1:172.21.15.154:6837/2213283923] conn(0x55bdbf211150 0x55bdbf20d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:17.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.316+0000 7ff68b373740 4 client.7419 mds_command: new command op to 7260 tid=0[{"prefix": "status"}] 2023-09-23T03:24:17.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.316+0000 7ff68b373740 1 -- 172.21.15.154:0/1803980891 --> [v2:172.21.15.154:6836/2213283923,v1:172.21.15.154:6837/2213283923] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bdbee226c0 con 0x55bdbf211150 2023-09-23T03:24:17.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.317+0000 7ff68657e640 1 --2- 172.21.15.154:0/1803980891 >> [v2:172.21.15.154:6836/2213283923,v1:172.21.15.154:6837/2213283923] conn(0x55bdbf211150 0x55bdbf20d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:17.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.317+0000 7ff68657e640 1 --2- 172.21.15.154:0/1803980891 >> [v2:172.21.15.154:6836/2213283923,v1:172.21.15.154:6837/2213283923] conn(0x55bdbf211150 0x55bdbf20d200 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=309a7e931b6e84a5 server_cookie=81e700f2c244ae63 in_seq=0 out_seq=0 2023-09-23T03:24:17.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.317+0000 7ff6657fa640 10 client.7419 ms_handle_connect on v2:172.21.15.154:6836/2213283923 2023-09-23T03:24:17.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.317+0000 7ff6657fa640 1 -- 172.21.15.154:0/1803980891 <== mds.0 v2:172.21.15.154:6836/2213283923 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55bdbee226c0 con 0x55bdbf211150 2023-09-23T03:24:17.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.317+0000 7ff6657fa640 10 client.7419 handle_command_reply: tid=0 2023-09-23T03:24:17.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.317+0000 7ff68b373740 1 client.7419 shutdown 2023-09-23T03:24:17.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.317+0000 7ff68b373740 1 -- 172.21.15.154:0/1803980891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bdbf2122a0 msgr2=0x55bdbf2015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:17.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.317+0000 7ff68b373740 1 --2- 172.21.15.154:0/1803980891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bdbf2122a0 0x55bdbf2015d0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7ff68008bba0 tx=0x7ff67800df80 comp rx=0 tx=0).stop 2023-09-23T03:24:17.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.318+0000 7ff68b373740 1 -- 172.21.15.154:0/1803980891 shutdown_connections 2023-09-23T03:24:17.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.318+0000 7ff68b373740 1 --2- 172.21.15.154:0/1803980891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55bdbf2122a0 0x55bdbf2015d0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.318+0000 7ff68b373740 1 --2- 172.21.15.154:0/1803980891 >> [v2:172.21.15.154:6836/2213283923,v1:172.21.15.154:6837/2213283923] conn(0x55bdbf211150 0x55bdbf20d200 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.318+0000 7ff68b373740 1 --2- 172.21.15.154:0/1803980891 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55bdbf1e1180 0x55bdbf201090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.318+0000 7ff68b373740 1 --2- 172.21.15.154:0/1803980891 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55bdbf1e0da0 0x55bdbf1df510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.318+0000 7ff6657fa640 0 client.7419 ms_handle_reset on v2:172.21.15.154:6836/2213283923 2023-09-23T03:24:17.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.318+0000 7ff68b373740 1 -- 172.21.15.154:0/1803980891 >> 172.21.15.154:0/1803980891 conn(0x55bdbf1b41e0 msgr2=0x55bdbf1f9400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:17.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.318+0000 7ff68b373740 1 -- 172.21.15.154:0/1803980891 shutdown_connections 2023-09-23T03:24:17.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.318+0000 7ff68b373740 1 -- 172.21.15.154:0/1803980891 wait complete. 2023-09-23T03:24:17.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.318+0000 7ff68b373740 20 client.7419 trim_cache size 0 max 16384 2023-09-23T03:24:17.324 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:24:17.324 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_fsid": "8c8039d3-6c84-43db-8808-85e0dbdd47c8", 2023-09-23T03:24:17.325 INFO:teuthology.orchestra.run.smithi154.stdout: "whoami": 0, 2023-09-23T03:24:17.325 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 7260, 2023-09-23T03:24:17.325 INFO:teuthology.orchestra.run.smithi154.stdout: "want_state": "up:active", 2023-09-23T03:24:17.325 INFO:teuthology.orchestra.run.smithi154.stdout: "state": "up:active", 2023-09-23T03:24:17.325 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs", 2023-09-23T03:24:17.325 INFO:teuthology.orchestra.run.smithi154.stdout: "rank_uptime": 6.6986846849999999, 2023-09-23T03:24:17.325 INFO:teuthology.orchestra.run.smithi154.stdout: "mdsmap_epoch": 62, 2023-09-23T03:24:17.325 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_epoch": 83, 2023-09-23T03:24:17.325 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_epoch_barrier": 83, 2023-09-23T03:24:17.325 INFO:teuthology.orchestra.run.smithi154.stdout: "uptime": 12.097534496 2023-09-23T03:24:17.325 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:24:17.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.318+0000 7ff687580640 1 -- 172.21.15.154:0/3010326572 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff66c06af60 msgr2=0x7ff66c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:17.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.318+0000 7ff687580640 1 --2- 172.21.15.154:0/3010326572 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff66c06af60 0x7ff66c06d420 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff670001270 tx=0x7ff670009040 comp rx=0 tx=0).stop 2023-09-23T03:24:17.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.318+0000 7ff687580640 1 -- 172.21.15.154:0/3010326572 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6800ca1b0 msgr2=0x7ff6800c8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:17.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.318+0000 7ff687580640 1 --2- 172.21.15.154:0/3010326572 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6800ca1b0 0x7ff6800c8010 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7ff67402bbc0 tx=0x7ff674001120 comp rx=0 tx=0).stop 2023-09-23T03:24:17.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.319+0000 7ff687580640 1 -- 172.21.15.154:0/3010326572 shutdown_connections 2023-09-23T03:24:17.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.319+0000 7ff687580640 1 --2- 172.21.15.154:0/3010326572 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff66c06af60 0x7ff66c06d420 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.319+0000 7ff687580640 1 --2- 172.21.15.154:0/3010326572 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6800c8550 0x7ff68015d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.319+0000 7ff687580640 1 --2- 172.21.15.154:0/3010326572 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6800ca1b0 0x7ff6800c8010 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.319+0000 7ff687580640 1 --2- 172.21.15.154:0/3010326572 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6680047c0 0x7ff68014d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.319+0000 7ff687580640 1 -- 172.21.15.154:0/3010326572 >> 172.21.15.154:0/3010326572 conn(0x7ff68009ba50 msgr2=0x7ff68015a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:17.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.319+0000 7ff687580640 1 -- 172.21.15.154:0/3010326572 shutdown_connections 2023-09-23T03:24:17.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.319+0000 7ff687580640 1 -- 172.21.15.154:0/3010326572 wait complete. 2023-09-23T03:24:17.337 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-23T03:24:17.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.540+0000 7fa2dd5ae640 1 Processor -- start 2023-09-23T03:24:17.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.540+0000 7fa2dd5ae640 1 -- start start 2023-09-23T03:24:17.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.541+0000 7fa2dd5ae640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa2d806fdc0 0x7fa2d80ab030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:17.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.541+0000 7fa2dd5ae640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2d8159c50 con 0x7fa2d8159870 2023-09-23T03:24:17.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.541+0000 7fa2dd5ae640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2d8071c00 con 0x7fa2d80ab570 2023-09-23T03:24:17.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.541+0000 7fa2dd5ae640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2d8070250 con 0x7fa2d806fdc0 2023-09-23T03:24:17.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.542+0000 7fa2d7fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fa2d8159870 0x7fa2d80714f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:59556/0 (socket says 172.21.15.154:59556) 2023-09-23T03:24:17.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.542+0000 7fa2d7fff640 1 -- 172.21.15.154:0/3717114646 learned_addr learned my addr 172.21.15.154:0/3717114646 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:24:17.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.542+0000 7fa2d77fe640 1 -- 172.21.15.154:0/3717114646 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1524698824 0 0) 0x7fa2c8001110 con 0x7fa2d8159870 2023-09-23T03:24:17.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.542+0000 7fa2d77fe640 1 -- 172.21.15.154:0/3717114646 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2c4003520 con 0x7fa2d8159870 2023-09-23T03:24:17.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.542+0000 7fa2cffff640 1 --2- 172.21.15.154:0/3717114646 >> v2:172.21.15.190:3300/0 conn(0x7fa2d806fdc0 0x7fa2d80ab030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:17.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.542+0000 7fa2d77fe640 1 -- 172.21.15.154:0/3717114646 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3018134240 0 0) 0x7fa2c4003520 con 0x7fa2d8159870 2023-09-23T03:24:17.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.542+0000 7fa2d77fe640 1 -- 172.21.15.154:0/3717114646 >> v2:172.21.15.190:3300/0 conn(0x7fa2d806fdc0 msgr2=0x7fa2d80ab030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:17.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.542+0000 7fa2d77fe640 1 --2- 172.21.15.154:0/3717114646 >> v2:172.21.15.190:3300/0 conn(0x7fa2d806fdc0 0x7fa2d80ab030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.542+0000 7fa2d77fe640 1 -- 172.21.15.154:0/3717114646 >> v1:172.21.15.190:6789/0 conn(0x7fa2d80ab570 legacy=0x7fa2d80cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:17.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.542+0000 7fa2d77fe640 1 -- 172.21.15.154:0/3717114646 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d8070450 con 0x7fa2d8159870 2023-09-23T03:24:17.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.542+0000 7fa2d77fe640 1 -- 172.21.15.154:0/3717114646 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fa2c8001800 con 0x7fa2d8159870 2023-09-23T03:24:17.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.542+0000 7fa2d77fe640 1 -- 172.21.15.154:0/3717114646 >> v1:172.21.15.154:6789/0 conn(0x7fa2d8159870 legacy=0x7fa2d80714f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:17.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.543+0000 7fa2d77fe640 1 --2- 172.21.15.154:0/3717114646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2c4003970 0x7fa2c4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:17.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.543+0000 7fa2d77fe640 1 --2- 172.21.15.154:0/3717114646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2c40047c0 0x7fa2c4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:17.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.543+0000 7fa2d77fe640 1 --2- 172.21.15.154:0/3717114646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2c4005180 0x7fa2c400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:17.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.543+0000 7fa2d77fe640 1 -- 172.21.15.154:0/3717114646 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d8070250 con 0x7fa2c4003970 2023-09-23T03:24:17.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.543+0000 7fa2d77fe640 1 -- 172.21.15.154:0/3717114646 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c400c0d0 con 0x7fa2c40047c0 2023-09-23T03:24:17.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.543+0000 7fa2d77fe640 1 -- 172.21.15.154:0/3717114646 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c400e4a0 con 0x7fa2c4005180 2023-09-23T03:24:17.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.543+0000 7fa2d7fff640 1 --2- 172.21.15.154:0/3717114646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2c4003970 0x7fa2c4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:17.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.543+0000 7fa2cffff640 1 --2- 172.21.15.154:0/3717114646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2c40047c0 0x7fa2c4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:17.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.543+0000 7fa2d7fff640 1 -- 172.21.15.154:0/3717114646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2c4005180 msgr2=0x7fa2c400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:17.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.543+0000 7fa2d7fff640 1 --2- 172.21.15.154:0/3717114646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2c4005180 0x7fa2c400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.543+0000 7fa2d7fff640 1 -- 172.21.15.154:0/3717114646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2c40047c0 msgr2=0x7fa2c4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:17.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.543+0000 7fa2d7fff640 1 --2- 172.21.15.154:0/3717114646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2c40047c0 0x7fa2c4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.543+0000 7fa2d7fff640 1 -- 172.21.15.154:0/3717114646 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c4003520 con 0x7fa2c4003970 2023-09-23T03:24:17.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.543+0000 7fa2cffff640 1 --2- 172.21.15.154:0/3717114646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2c40047c0 0x7fa2c4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:17.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.543+0000 7fa2dcdad640 1 -- 172.21.15.154:0/3717114646 reap_dead start 2023-09-23T03:24:17.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.544+0000 7fa2d7fff640 1 --2- 172.21.15.154:0/3717114646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2c4003970 0x7fa2c4003d90 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fa2c800c240 tx=0x7fa2c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=683b8f251b7902db server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:17.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.545+0000 7fa2d77fe640 1 -- 172.21.15.154:0/3717114646 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8005ad0 con 0x7fa2c4003970 2023-09-23T03:24:17.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.545+0000 7fa2d77fe640 1 -- 172.21.15.154:0/3717114646 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2c8005c50 con 0x7fa2c4003970 2023-09-23T03:24:17.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.545+0000 7fa2d77fe640 1 -- 172.21.15.154:0/3717114646 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c802c8b0 con 0x7fa2c4003970 2023-09-23T03:24:17.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.545+0000 7fa2dd5ae640 1 -- 172.21.15.154:0/3717114646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2c4003970 msgr2=0x7fa2c4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:17.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.545+0000 7fa2dd5ae640 1 --2- 172.21.15.154:0/3717114646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2c4003970 0x7fa2c4003d90 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fa2c800c240 tx=0x7fa2c40057f0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.545+0000 7fa2dd5ae640 1 -- 172.21.15.154:0/3717114646 shutdown_connections 2023-09-23T03:24:17.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.545+0000 7fa2dd5ae640 1 --2- 172.21.15.154:0/3717114646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2c4003970 0x7fa2c4003d90 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.545+0000 7fa2dd5ae640 1 -- 172.21.15.154:0/3717114646 >> 172.21.15.154:0/3717114646 conn(0x7fa2d8069eb0 msgr2=0x7fa2d806a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:17.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.545+0000 7fa2dd5ae640 1 -- 172.21.15.154:0/3717114646 shutdown_connections 2023-09-23T03:24:17.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.545+0000 7fa2dd5ae640 1 -- 172.21.15.154:0/3717114646 wait complete. 2023-09-23T03:24:17.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.547+0000 7fa2dd5ae640 1 Processor -- start 2023-09-23T03:24:17.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.547+0000 7fa2dd5ae640 1 -- start start 2023-09-23T03:24:17.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.548+0000 7fa2dd5ae640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2c4003970 0x7fa2d8098450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:17.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.548+0000 7fa2dd5ae640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2d8158ca0 0x7fa2d8098990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:17.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.548+0000 7fa2dd5ae640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2d80ae000 0x7fa2d81ef2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:17.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.548+0000 7fa2dd5ae640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d81eeec0 con 0x7fa2c4003970 2023-09-23T03:24:17.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.548+0000 7fa2dd5ae640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d809b460 con 0x7fa2d8158ca0 2023-09-23T03:24:17.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.548+0000 7fa2dd5ae640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d809a480 con 0x7fa2d80ae000 2023-09-23T03:24:17.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.548+0000 7fa2d7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2c4003970 0x7fa2d8098450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:17.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.549+0000 7fa2d7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2c4003970 0x7fa2d8098450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50954/0 (socket says 172.21.15.154:50954) 2023-09-23T03:24:17.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.549+0000 7fa2d7fff640 1 -- 172.21.15.154:0/1363634991 learned_addr learned my addr 172.21.15.154:0/1363634991 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:17.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.549+0000 7fa2d7fff640 1 -- 172.21.15.154:0/1363634991 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2d80ae000 msgr2=0x7fa2d81ef2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:17.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.549+0000 7fa2d7fff640 1 --2- 172.21.15.154:0/1363634991 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2d80ae000 0x7fa2d81ef2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.549+0000 7fa2d7fff640 1 -- 172.21.15.154:0/1363634991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2d8158ca0 msgr2=0x7fa2d8098990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:17.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.549+0000 7fa2cffff640 1 --2- 172.21.15.154:0/1363634991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2d8158ca0 0x7fa2d8098990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:17.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.549+0000 7fa2d7fff640 1 --2- 172.21.15.154:0/1363634991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2d8158ca0 0x7fa2d8098990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.549+0000 7fa2d7fff640 1 -- 172.21.15.154:0/1363634991 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d809a720 con 0x7fa2c4003970 2023-09-23T03:24:17.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.549+0000 7fa2d7fff640 1 --2- 172.21.15.154:0/1363634991 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2c4003970 0x7fa2d8098450 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fa2c800c240 tx=0x7fa2c802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:17.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.549+0000 7fa2d57fa640 1 -- 172.21.15.154:0/1363634991 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c803c040 con 0x7fa2c4003970 2023-09-23T03:24:17.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.550+0000 7fa2d57fa640 1 -- 172.21.15.154:0/1363634991 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2c802ccc0 con 0x7fa2c4003970 2023-09-23T03:24:17.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.550+0000 7fa2d57fa640 1 -- 172.21.15.154:0/1363634991 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c80072b0 con 0x7fa2c4003970 2023-09-23T03:24:17.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.550+0000 7fa2dd5ae640 1 -- 172.21.15.154:0/1363634991 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d809b6f0 con 0x7fa2c4003970 2023-09-23T03:24:17.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.550+0000 7fa2dd5ae640 1 -- 172.21.15.154:0/1363634991 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d80afa10 con 0x7fa2c4003970 2023-09-23T03:24:17.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.551+0000 7fa2dd5ae640 1 -- 172.21.15.154:0/1363634991 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2d804e920 con 0x7fa2c4003970 2023-09-23T03:24:17.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.552+0000 7fa2d57fa640 1 -- 172.21.15.154:0/1363634991 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fa2c8032070 con 0x7fa2c4003970 2023-09-23T03:24:17.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.552+0000 7fa2d57fa640 1 --2- 172.21.15.154:0/1363634991 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa2b806af60 0x7fa2b806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:17.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.553+0000 7fa2d57fa640 1 -- 172.21.15.154:0/1363634991 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa2c80aabd0 con 0x7fa2c4003970 2023-09-23T03:24:17.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.553+0000 7fa2cffff640 1 --2- 172.21.15.154:0/1363634991 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa2b806af60 0x7fa2b806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:17.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.554+0000 7fa2cffff640 1 --2- 172.21.15.154:0/1363634991 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa2b806af60 0x7fa2b806d420 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8159280 tx=0x7fa2c0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:17.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.557+0000 7fa2d57fa640 1 -- 172.21.15.154:0/1363634991 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fa2c8075be0 con 0x7fa2c4003970 2023-09-23T03:24:17.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.789+0000 7fa2dd5ae640 1 -- 172.21.15.154:0/1363634991 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa2d8159c50 con 0x7fa2c4003970 2023-09-23T03:24:17.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.790+0000 7fa2d57fa640 1 -- 172.21.15.154:0/1363634991 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v62) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa2d8159c50 con 0x7fa2c4003970 2023-09-23T03:24:17.791 INFO:teuthology.orchestra.run.smithi154.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-09-23T03:24:17.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.793+0000 7fa2dd5ae640 1 -- 172.21.15.154:0/1363634991 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa2b806af60 msgr2=0x7fa2b806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:17.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.793+0000 7fa2dd5ae640 1 --2- 172.21.15.154:0/1363634991 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa2b806af60 0x7fa2b806d420 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8159280 tx=0x7fa2c0008040 comp rx=0 tx=0).stop 2023-09-23T03:24:17.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.793+0000 7fa2dd5ae640 1 -- 172.21.15.154:0/1363634991 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2c4003970 msgr2=0x7fa2d8098450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:17.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.793+0000 7fa2dd5ae640 1 --2- 172.21.15.154:0/1363634991 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2c4003970 0x7fa2d8098450 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fa2c800c240 tx=0x7fa2c802e070 comp rx=0 tx=0).stop 2023-09-23T03:24:17.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.793+0000 7fa2dd5ae640 1 -- 172.21.15.154:0/1363634991 shutdown_connections 2023-09-23T03:24:17.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.793+0000 7fa2dd5ae640 1 --2- 172.21.15.154:0/1363634991 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa2b806af60 0x7fa2b806d420 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.793+0000 7fa2dd5ae640 1 --2- 172.21.15.154:0/1363634991 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa2d80ae000 0x7fa2d81ef2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.793+0000 7fa2dd5ae640 1 --2- 172.21.15.154:0/1363634991 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa2d8158ca0 0x7fa2d8098990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.793+0000 7fa2dd5ae640 1 --2- 172.21.15.154:0/1363634991 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa2c4003970 0x7fa2d8098450 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:17.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.793+0000 7fa2dd5ae640 1 -- 172.21.15.154:0/1363634991 >> 172.21.15.154:0/1363634991 conn(0x7fa2d8069eb0 msgr2=0x7fa2d80af270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:17.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.793+0000 7fa2dd5ae640 1 -- 172.21.15.154:0/1363634991 shutdown_connections 2023-09-23T03:24:17.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:17.793+0000 7fa2dd5ae640 1 -- 172.21.15.154:0/1363634991 wait complete. 2023-09-23T03:24:17.812 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-23T03:24:17.812 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-23T03:24:17.812 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi154.front.sepia.ceph.com 2023-09-23T03:24:17.812 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-09-23T03:24:17.812 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-23T03:24:17.812 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-23T03:24:17.812 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-23T03:24:17.813 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-23T03:24:17.813 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-23T03:24:17.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:24:17.813 DEBUG:teuthology.orchestra.run.smithi154:> ip addr 2023-09-23T03:24:17.869 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-23T03:24:17.869 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-23T03:24:17.869 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2023-09-23T03:24:17.869 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:24:17.869 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2023-09-23T03:24:17.869 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:24:17.869 INFO:teuthology.orchestra.run.smithi154.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:24:17.869 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:54 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:24:17.870 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f0 2023-09-23T03:24:17.870 INFO:teuthology.orchestra.run.smithi154.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:24:17.870 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:55 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:24:17.870 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f1 2023-09-23T03:24:17.870 INFO:teuthology.orchestra.run.smithi154.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:24:17.870 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:60 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:24:17.870 INFO:teuthology.orchestra.run.smithi154.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-23T03:24:17.870 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:61 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:24:17.870 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.21.15.154/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-09-23T03:24:17.870 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft 41867sec preferred_lft 41867sec 2023-09-23T03:24:17.870 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::ec4:7aff:fe88:7761/64 scope link 2023-09-23T03:24:17.870 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:24:17.871 INFO:teuthology.orchestra.run.smithi154.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-23T03:24:17.871 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 5a:c6:b2:96:fb:dc brd ff:ff:ff:ff:ff:ff 2023-09-23T03:24:17.871 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-09-23T03:24:17.871 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:24:17.871 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::a801:22ff:febf:2833/64 scope link 2023-09-23T03:24:17.871 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:24:17.871 INFO:teuthology.orchestra.run.smithi154.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-23T03:24:17.871 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 5a:c6:b2:96:fb:dc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:24:17.871 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::58c6:b2ff:fe96:fbdc/64 scope link 2023-09-23T03:24:17.871 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:24:17.871 INFO:teuthology.orchestra.run.smithi154.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-23T03:24:17.871 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether b2:23:b0:e4:c2:ab brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:24:17.872 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::b023:b0ff:fee4:c2ab/64 scope link 2023-09-23T03:24:17.872 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:24:17.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:24:17.873 DEBUG:teuthology.orchestra.run.smithi154:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-23T03:24:17.964 INFO:teuthology.orchestra.run.smithi154.stdout:1 2023-09-23T03:24:17.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:24:17.968 DEBUG:teuthology.orchestra.run.smithi154:> route 2023-09-23T03:24:18.021 INFO:teuthology.orchestra.run.smithi154.stdout:Kernel IP routing table 2023-09-23T03:24:18.022 INFO:teuthology.orchestra.run.smithi154.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-23T03:24:18.022 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-23T03:24:18.022 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-09-23T03:24:18.022 INFO:teuthology.orchestra.run.smithi154.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-23T03:24:18.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:24:18.023 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-09-23T03:24:18.023 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-09-23T03:24:18.023 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-23T03:24:18.023 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-23T03:24:18.023 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-23T03:24:18.023 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-09-23T03:24:18.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:24:18.218 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2023-09-23T03:24:18.267 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-09-23T03:24:18.267 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-09-23T03:24:18.268 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-23T03:24:18.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:24:18.268 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set brx.0 up 2023-09-23T03:24:18.363 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-09-23T03:24:18.363 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-23T03:24:18.363 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-09-23T03:24:18.419 INFO:teuthology.orchestra.run.smithi154.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-09-23T03:24:18.420 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-23T03:24:18.420 DEBUG:teuthology.orchestra.run.smithi154:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-09-23T03:24:18.470 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-09-23T03:24:18.570 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2023-09-23T03:24:18.632 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2023-09-23T03:24:18.632 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2023-09-23T03:24:18.632 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2023-09-23T03:24:18.632 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2023-09-23T03:24:18.632 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2023-09-23T03:24:18.632 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2023-09-23T03:24:18.632 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2023-09-23T03:24:18.633 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2023-09-23T03:24:18.633 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2023-09-23T03:24:18.633 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2023-09-23T03:24:18.633 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2023-09-23T03:24:18.633 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T03:24:18.633 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2023-09-23T03:24:18.633 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2023-09-23T03:24:18.633 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2023-09-23T03:24:18.633 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2023-09-23T03:24:18.633 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2023-09-23T03:24:18.633 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2023-09-23T03:24:18.634 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2023-09-23T03:24:18.634 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-09-23T03:24:18.634 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-09-23T03:24:18.634 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2023-09-23T03:24:18.634 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2023-09-23T03:24:18.634 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2023-09-23T03:24:18.634 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-0 2023-09-23T03:24:18.634 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-1 2023-09-23T03:24:18.634 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-2 2023-09-23T03:24:18.634 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-3 2023-09-23T03:24:18.634 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2023-09-23T03:24:18.635 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:24:18.635 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:24:18.635 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:24:18.635 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:24:18.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:24:18.636 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-09-23T03:24:18.685 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-23T03:24:18.685 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-09-23T03:24:18.690 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-09-23T03:24:18.765 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2023-09-23T03:24:18.805 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2023-09-23T03:24:18.805 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2023-09-23T03:24:18.806 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2023-09-23T03:24:18.806 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2023-09-23T03:24:18.806 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2023-09-23T03:24:18.806 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2023-09-23T03:24:18.806 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2023-09-23T03:24:18.806 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2023-09-23T03:24:18.806 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2023-09-23T03:24:18.806 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2023-09-23T03:24:18.806 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2023-09-23T03:24:18.806 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T03:24:18.806 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2023-09-23T03:24:18.807 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2023-09-23T03:24:18.807 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2023-09-23T03:24:18.807 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2023-09-23T03:24:18.807 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2023-09-23T03:24:18.807 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2023-09-23T03:24:18.807 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2023-09-23T03:24:18.807 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-09-23T03:24:18.807 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-09-23T03:24:18.807 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2023-09-23T03:24:18.807 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2023-09-23T03:24:18.807 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2023-09-23T03:24:18.808 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-0 2023-09-23T03:24:18.808 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-1 2023-09-23T03:24:18.808 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-2 2023-09-23T03:24:18.808 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-3 2023-09-23T03:24:18.808 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2023-09-23T03:24:18.808 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:24:18.808 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:24:18.808 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:24:18.808 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:24:18.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:24:18.809 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-09-23T03:24:18.878 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:2023-09-23T03:24:18.877+0000 7f76dcf65480 -1 init, newargv = 0x55ac0d5e1ba0 newargc=15 2023-09-23T03:24:18.879 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:ceph-fuse[115807]: starting ceph client 2023-09-23T03:24:18.888 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:ceph-fuse[115807]: starting fuse 2023-09-23T03:24:18.894 INFO:teuthology.orchestra.run.smithi154.stdout:39 2023-09-23T03:24:18.895 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-09-23T03:24:18.895 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:24:18.895 DEBUG:teuthology.orchestra.run.smithi154:> import glob 2023-09-23T03:24:18.895 DEBUG:teuthology.orchestra.run.smithi154:> import re 2023-09-23T03:24:18.895 DEBUG:teuthology.orchestra.run.smithi154:> import os 2023-09-23T03:24:18.895 DEBUG:teuthology.orchestra.run.smithi154:> import subprocess 2023-09-23T03:24:18.895 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:24:18.895 DEBUG:teuthology.orchestra.run.smithi154:> def _find_admin_socket(client_name): 2023-09-23T03:24:18.895 DEBUG:teuthology.orchestra.run.smithi154:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-09-23T03:24:18.895 DEBUG:teuthology.orchestra.run.smithi154:> files = glob.glob(asok_path) 2023-09-23T03:24:18.896 DEBUG:teuthology.orchestra.run.smithi154:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-09-23T03:24:18.896 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:24:18.896 DEBUG:teuthology.orchestra.run.smithi154:> # Given a non-glob path, it better be there 2023-09-23T03:24:18.896 DEBUG:teuthology.orchestra.run.smithi154:> if "*" not in asok_path: 2023-09-23T03:24:18.896 DEBUG:teuthology.orchestra.run.smithi154:> assert(len(files) == 1) 2023-09-23T03:24:18.896 DEBUG:teuthology.orchestra.run.smithi154:> return files[0] 2023-09-23T03:24:18.896 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:24:18.896 DEBUG:teuthology.orchestra.run.smithi154:> for f in files: 2023-09-23T03:24:18.896 DEBUG:teuthology.orchestra.run.smithi154:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-23T03:24:18.896 DEBUG:teuthology.orchestra.run.smithi154:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-23T03:24:18.896 DEBUG:teuthology.orchestra.run.smithi154:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-23T03:24:18.896 DEBUG:teuthology.orchestra.run.smithi154:> contents = proc_f.read() 2023-09-23T03:24:18.897 DEBUG:teuthology.orchestra.run.smithi154:> if mountpoint in contents: 2023-09-23T03:24:18.897 DEBUG:teuthology.orchestra.run.smithi154:> return f 2023-09-23T03:24:18.897 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-23T03:24:18.897 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:24:18.897 DEBUG:teuthology.orchestra.run.smithi154:> print(_find_admin_socket("client.0")) 2023-09-23T03:24:18.897 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:24:19.094 INFO:teuthology.orchestra.run.smithi154.stdout:/var/run/ceph/ceph-client.0.115807.asok 2023-09-23T03:24:19.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:24:19.110 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.115807.asok 2023-09-23T03:24:19.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:24:19.111 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.115807.asok status 2023-09-23T03:24:19.351 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:24:19.351 INFO:teuthology.orchestra.run.smithi154.stdout: "metadata": { 2023-09-23T03:24:19.351 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_sha1": "862663913a74a99e1c52d50e6a7e59dd6705749a", 2023-09-23T03:24:19.351 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_version": "ceph version 18.0.0-6336-g86266391 (862663913a74a99e1c52d50e6a7e59dd6705749a) reef (dev)", 2023-09-23T03:24:19.351 INFO:teuthology.orchestra.run.smithi154.stdout: "entity_id": "0", 2023-09-23T03:24:19.351 INFO:teuthology.orchestra.run.smithi154.stdout: "hostname": "smithi154", 2023-09-23T03:24:19.351 INFO:teuthology.orchestra.run.smithi154.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-09-23T03:24:19.351 INFO:teuthology.orchestra.run.smithi154.stdout: "pid": "115807", 2023-09-23T03:24:19.351 INFO:teuthology.orchestra.run.smithi154.stdout: "root": "/" 2023-09-23T03:24:19.351 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:24:19.352 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_count": 0, 2023-09-23T03:24:19.352 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_pinned_count": 0, 2023-09-23T03:24:19.352 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 7431, 2023-09-23T03:24:19.352 INFO:teuthology.orchestra.run.smithi154.stdout: "inst": { 2023-09-23T03:24:19.352 INFO:teuthology.orchestra.run.smithi154.stdout: "name": { 2023-09-23T03:24:19.352 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "client", 2023-09-23T03:24:19.352 INFO:teuthology.orchestra.run.smithi154.stdout: "num": 7431 2023-09-23T03:24:19.352 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:24:19.352 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2023-09-23T03:24:19.352 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2023-09-23T03:24:19.352 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.1:0", 2023-09-23T03:24:19.353 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 563899610 2023-09-23T03:24:19.353 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:24:19.353 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:24:19.353 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2023-09-23T03:24:19.353 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2023-09-23T03:24:19.353 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.1:0", 2023-09-23T03:24:19.353 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 563899610 2023-09-23T03:24:19.353 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:24:19.353 INFO:teuthology.orchestra.run.smithi154.stdout: "inst_str": "client.7431 192.168.0.1:0/563899610", 2023-09-23T03:24:19.353 INFO:teuthology.orchestra.run.smithi154.stdout: "addr_str": "192.168.0.1:0/563899610", 2023-09-23T03:24:19.353 INFO:teuthology.orchestra.run.smithi154.stdout: "inode_count": 1, 2023-09-23T03:24:19.353 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_epoch": 62, 2023-09-23T03:24:19.354 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch": 83, 2023-09-23T03:24:19.354 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch_barrier": 0, 2023-09-23T03:24:19.354 INFO:teuthology.orchestra.run.smithi154.stdout: "blocklisted": false, 2023-09-23T03:24:19.354 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs" 2023-09-23T03:24:19.354 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:24:19.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:24:19.366 DEBUG:teuthology.orchestra.run.smithi154:> stat --file-system '--printf=%T 2023-09-23T03:24:19.366 DEBUG:teuthology.orchestra.run.smithi154:> ' -- /home/ubuntu/cephtest/mnt.0 2023-09-23T03:24:19.421 INFO:teuthology.orchestra.run.smithi154.stdout:fuseblk 2023-09-23T03:24:19.421 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-09-23T03:24:19.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:24:19.421 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-09-23T03:24:19.514 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_extra --pg_num_min 64 2023-09-23T03:24:19.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.736+0000 7efcb8c00640 1 Processor -- start 2023-09-23T03:24:19.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.736+0000 7efcb8c00640 1 -- start start 2023-09-23T03:24:19.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.737+0000 7efcb8c00640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efcb40700f0 0x7efcb40c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:19.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.737+0000 7efcb8c00640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efcb415abd0 0x7efcb40cb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:19.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.737+0000 7efcb8c00640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efcb40704d0 0x7efcb40ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:19.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.737+0000 7efcb8c00640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcb40cb9a0 con 0x7efcb40704d0 2023-09-23T03:24:19.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.737+0000 7efcb8c00640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcb40c95b0 con 0x7efcb415abd0 2023-09-23T03:24:19.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.737+0000 7efcb8c00640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcb40c9890 con 0x7efcb40700f0 2023-09-23T03:24:19.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.738+0000 7efcb2d9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efcb40704d0 0x7efcb40ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:19.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.738+0000 7efcb2d9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efcb40704d0 0x7efcb40ce3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50968/0 (socket says 172.21.15.154:50968) 2023-09-23T03:24:19.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.738+0000 7efcb2d9d640 1 -- 172.21.15.154:0/3285331960 learned_addr learned my addr 172.21.15.154:0/3285331960 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:19.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.738+0000 7efcb259c640 1 --2- 172.21.15.154:0/3285331960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efcb40700f0 0x7efcb40c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:19.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.738+0000 7efcabfff640 1 --2- 172.21.15.154:0/3285331960 >> v2:172.21.15.190:3300/0 conn(0x7efcb415abd0 0x7efcb40cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:19.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.738+0000 7efcb2d9d640 1 -- 172.21.15.154:0/3285331960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efcb40700f0 msgr2=0x7efcb40c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:19.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.738+0000 7efcb2d9d640 1 --2- 172.21.15.154:0/3285331960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efcb40700f0 0x7efcb40c8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:19.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.738+0000 7efcb2d9d640 1 -- 172.21.15.154:0/3285331960 >> v2:172.21.15.190:3300/0 conn(0x7efcb415abd0 msgr2=0x7efcb40cb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:19.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.738+0000 7efcb2d9d640 1 --2- 172.21.15.154:0/3285331960 >> v2:172.21.15.190:3300/0 conn(0x7efcb415abd0 0x7efcb40cb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:19.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.738+0000 7efcb2d9d640 1 -- 172.21.15.154:0/3285331960 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb40c9b70 con 0x7efcb40704d0 2023-09-23T03:24:19.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.739+0000 7efcb2d9d640 1 --2- 172.21.15.154:0/3285331960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcb40704d0 0x7efcb40ce3a0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7efcb40ce340 tx=0x7efcac02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12906602c1f15a90 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:19.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.740+0000 7efcb1d9b640 1 -- 172.21.15.154:0/3285331960 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcac02ca70 con 0x7efcb40704d0 2023-09-23T03:24:19.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.740+0000 7efcb1d9b640 1 -- 172.21.15.154:0/3285331960 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcac02cbd0 con 0x7efcb40704d0 2023-09-23T03:24:19.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.740+0000 7efcb1d9b640 1 -- 172.21.15.154:0/3285331960 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcac0343f0 con 0x7efcb40704d0 2023-09-23T03:24:19.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.740+0000 7efcb8c00640 1 -- 172.21.15.154:0/3285331960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcb40704d0 msgr2=0x7efcb40ce3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:19.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.740+0000 7efcb8c00640 1 --2- 172.21.15.154:0/3285331960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcb40704d0 0x7efcb40ce3a0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7efcb40ce340 tx=0x7efcac02bed0 comp rx=0 tx=0).stop 2023-09-23T03:24:19.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.741+0000 7efcb8c00640 1 -- 172.21.15.154:0/3285331960 shutdown_connections 2023-09-23T03:24:19.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.741+0000 7efcb8c00640 1 --2- 172.21.15.154:0/3285331960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcb40704d0 0x7efcb40ce3a0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:19.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.741+0000 7efcb8c00640 1 --2- 172.21.15.154:0/3285331960 >> v2:172.21.15.190:3300/0 conn(0x7efcb415abd0 0x7efcb40cb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:19.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.741+0000 7efcb8c00640 1 --2- 172.21.15.154:0/3285331960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efcb40700f0 0x7efcb40c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:19.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.741+0000 7efcb8c00640 1 -- 172.21.15.154:0/3285331960 >> 172.21.15.154:0/3285331960 conn(0x7efcb406a520 msgr2=0x7efcb406bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:19.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.741+0000 7efcb8c00640 1 -- 172.21.15.154:0/3285331960 shutdown_connections 2023-09-23T03:24:19.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.741+0000 7efcb8c00640 1 -- 172.21.15.154:0/3285331960 wait complete. 2023-09-23T03:24:19.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.742+0000 7efcb8c00640 1 Processor -- start 2023-09-23T03:24:19.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.743+0000 7efcb8c00640 1 -- start start 2023-09-23T03:24:19.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.744+0000 7efcb8c00640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcb40700f0 0x7efcb40b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:19.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.744+0000 7efcb8c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcb40704d0 0x7efcb40aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:19.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.744+0000 7efcb8c00640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efcb415abd0 0x7efcb40b2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:19.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.744+0000 7efcb8c00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb40c8510 con 0x7efcb40704d0 2023-09-23T03:24:19.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.744+0000 7efcb8c00640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb40c7990 con 0x7efcb40700f0 2023-09-23T03:24:19.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.744+0000 7efcb8c00640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcb40c7c70 con 0x7efcb415abd0 2023-09-23T03:24:19.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.744+0000 7efcabfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcb40704d0 0x7efcb40aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:19.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.744+0000 7efcabfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcb40704d0 0x7efcb40aef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50984/0 (socket says 172.21.15.154:50984) 2023-09-23T03:24:19.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.744+0000 7efcabfff640 1 -- 172.21.15.154:0/3430362030 learned_addr learned my addr 172.21.15.154:0/3430362030 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:19.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.744+0000 7efcb259c640 1 --2- 172.21.15.154:0/3430362030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcb40700f0 0x7efcb40b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:19.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.744+0000 7efcb2d9d640 1 --2- 172.21.15.154:0/3430362030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efcb415abd0 0x7efcb40b2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:19.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.744+0000 7efcabfff640 1 -- 172.21.15.154:0/3430362030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efcb415abd0 msgr2=0x7efcb40b2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:19.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.744+0000 7efcabfff640 1 --2- 172.21.15.154:0/3430362030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efcb415abd0 0x7efcb40b2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:19.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.744+0000 7efcabfff640 1 -- 172.21.15.154:0/3430362030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcb40700f0 msgr2=0x7efcb40b02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:19.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.744+0000 7efcabfff640 1 --2- 172.21.15.154:0/3430362030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcb40700f0 0x7efcb40b02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:19.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.744+0000 7efcabfff640 1 -- 172.21.15.154:0/3430362030 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcac009b30 con 0x7efcb40704d0 2023-09-23T03:24:19.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.744+0000 7efcb2d9d640 1 --2- 172.21.15.154:0/3430362030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efcb415abd0 0x7efcb40b2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:19.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.745+0000 7efcabfff640 1 --2- 172.21.15.154:0/3430362030 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcb40704d0 0x7efcb40aef90 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7efc9c00d7d0 tx=0x7efc9c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:19.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.745+0000 7efcaaffd640 1 -- 172.21.15.154:0/3430362030 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9c00b6c0 con 0x7efcb40704d0 2023-09-23T03:24:19.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.745+0000 7efcaaffd640 1 -- 172.21.15.154:0/3430362030 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc9c00b820 con 0x7efcb40704d0 2023-09-23T03:24:19.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.745+0000 7efcaaffd640 1 -- 172.21.15.154:0/3430362030 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9c00bbb0 con 0x7efcb40704d0 2023-09-23T03:24:19.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.745+0000 7efcb8c00640 1 -- 172.21.15.154:0/3430362030 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcb40c7f50 con 0x7efcb40704d0 2023-09-23T03:24:19.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.745+0000 7efcb8c00640 1 -- 172.21.15.154:0/3430362030 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcb40b07e0 con 0x7efcb40704d0 2023-09-23T03:24:19.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.746+0000 7efcb8c00640 1 -- 172.21.15.154:0/3430362030 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcb404e920 con 0x7efcb40704d0 2023-09-23T03:24:19.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.747+0000 7efcaaffd640 1 -- 172.21.15.154:0/3430362030 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7efc9c00bd10 con 0x7efcb40704d0 2023-09-23T03:24:19.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.748+0000 7efcaaffd640 1 --2- 172.21.15.154:0/3430362030 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efc9406af30 0x7efc9406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:19.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.748+0000 7efcaaffd640 1 -- 172.21.15.154:0/3430362030 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6029+0+0 (secure 0 0 0) 0x7efc9c089e60 con 0x7efcb40704d0 2023-09-23T03:24:19.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.748+0000 7efcb259c640 1 --2- 172.21.15.154:0/3430362030 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efc9406af30 0x7efc9406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:19.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.749+0000 7efcb259c640 1 --2- 172.21.15.154:0/3430362030 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efc9406af30 0x7efc9406d3f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7efca4006fd0 tx=0x7efca4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:19.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.753+0000 7efcaaffd640 1 -- 172.21.15.154:0/3430362030 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7efc9c054cc0 con 0x7efcb40704d0 2023-09-23T03:24:19.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:19.945+0000 7efcb8c00640 1 -- 172.21.15.154:0/3430362030 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_extra", "pg_num_min": 64} v 0) v1 -- 0x7efcb4142520 con 0x7efcb40704d0 2023-09-23T03:24:20.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.680+0000 7efcaaffd640 1 -- 172.21.15.154:0/3430362030 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_extra", "pg_num_min": 64}]=0 pool 'cephfs_data_extra' created v84) v1 ==== 142+0+0 (secure 0 0 0) 0x7efc9c0582e0 con 0x7efcb40704d0 2023-09-23T03:24:20.681 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_data_extra' created 2023-09-23T03:24:20.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.684+0000 7efcb8c00640 1 -- 172.21.15.154:0/3430362030 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efc9406af30 msgr2=0x7efc9406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:20.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.684+0000 7efcb8c00640 1 --2- 172.21.15.154:0/3430362030 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efc9406af30 0x7efc9406d3f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7efca4006fd0 tx=0x7efca4008040 comp rx=0 tx=0).stop 2023-09-23T03:24:20.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.684+0000 7efcb8c00640 1 -- 172.21.15.154:0/3430362030 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcb40704d0 msgr2=0x7efcb40aef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:20.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.684+0000 7efcb8c00640 1 --2- 172.21.15.154:0/3430362030 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcb40704d0 0x7efcb40aef90 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7efc9c00d7d0 tx=0x7efc9c00dc90 comp rx=0 tx=0).stop 2023-09-23T03:24:20.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.685+0000 7efcb8c00640 1 -- 172.21.15.154:0/3430362030 shutdown_connections 2023-09-23T03:24:20.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.685+0000 7efcb8c00640 1 --2- 172.21.15.154:0/3430362030 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efc9406af30 0x7efc9406d3f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:20.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.685+0000 7efcb8c00640 1 --2- 172.21.15.154:0/3430362030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efcb415abd0 0x7efcb40b2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:20.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.685+0000 7efcb8c00640 1 --2- 172.21.15.154:0/3430362030 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efcb40704d0 0x7efcb40aef90 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:20.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.685+0000 7efcb8c00640 1 --2- 172.21.15.154:0/3430362030 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efcb40700f0 0x7efcb40b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:20.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.685+0000 7efcb8c00640 1 -- 172.21.15.154:0/3430362030 >> 172.21.15.154:0/3430362030 conn(0x7efcb406a520 msgr2=0x7efcb40c2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:20.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.686+0000 7efcb8c00640 1 -- 172.21.15.154:0/3430362030 shutdown_connections 2023-09-23T03:24:20.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.686+0000 7efcb8c00640 1 -- 172.21.15.154:0/3430362030 wait complete. 2023-09-23T03:24:20.711 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_extra 2023-09-23T03:24:20.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.912+0000 7fd39ebc0640 1 Processor -- start 2023-09-23T03:24:20.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.912+0000 7fd39ebc0640 1 -- start start 2023-09-23T03:24:20.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.914+0000 7fd39ebc0640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd3980cb5b0 0x7fd3980cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:20.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.914+0000 7fd39ebc0640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd39815b6a0 con 0x7fd39815bb30 2023-09-23T03:24:20.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.914+0000 7fd39ebc0640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd39815f0b0 con 0x7fd39815aba0 2023-09-23T03:24:20.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.914+0000 7fd39ebc0640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd39815bf70 con 0x7fd3980cb5b0 2023-09-23T03:24:20.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.914+0000 7fd397fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fd39815bb30 0x7fd3980caea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:59558/0 (socket says 172.21.15.154:59558) 2023-09-23T03:24:20.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.914+0000 7fd397fff640 1 -- 172.21.15.154:0/2726711128 learned_addr learned my addr 172.21.15.154:0/2726711128 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:24:20.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.915+0000 7fd39d15d640 1 --2- 172.21.15.154:0/2726711128 >> v2:172.21.15.154:3300/0 conn(0x7fd3980cb5b0 0x7fd3980cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:20.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.915+0000 7fd39d15d640 1 -- 172.21.15.154:0/2726711128 >> v1:172.21.15.190:6789/0 conn(0x7fd39815aba0 legacy=0x7fd39815e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:20.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.915+0000 7fd3977fe640 1 -- 172.21.15.154:0/2726711128 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3660831137 0 0) 0x7fd39815b6a0 con 0x7fd39815bb30 2023-09-23T03:24:20.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.915+0000 7fd39d15d640 1 -- 172.21.15.154:0/2726711128 >> v1:172.21.15.154:6789/0 conn(0x7fd39815bb30 legacy=0x7fd3980caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:20.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.915+0000 7fd39d15d640 1 -- 172.21.15.154:0/2726711128 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd39815f390 con 0x7fd3980cb5b0 2023-09-23T03:24:20.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.917+0000 7fd39d15d640 1 --2- 172.21.15.154:0/2726711128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3980cb5b0 0x7fd3980cc5c0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fd390009d90 tx=0x7fd39002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d27f0903da68f5d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:20.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.917+0000 7fd3977fe640 1 -- 172.21.15.154:0/2726711128 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39002cd00 con 0x7fd3980cb5b0 2023-09-23T03:24:20.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.917+0000 7fd3977fe640 1 -- 172.21.15.154:0/2726711128 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd39002ce60 con 0x7fd3980cb5b0 2023-09-23T03:24:20.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.917+0000 7fd3977fe640 1 -- 172.21.15.154:0/2726711128 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd390034660 con 0x7fd3980cb5b0 2023-09-23T03:24:20.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.917+0000 7fd39ebc0640 1 -- 172.21.15.154:0/2726711128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3980cb5b0 msgr2=0x7fd3980cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:20.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.917+0000 7fd39ebc0640 1 --2- 172.21.15.154:0/2726711128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3980cb5b0 0x7fd3980cc5c0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fd390009d90 tx=0x7fd39002bf50 comp rx=0 tx=0).stop 2023-09-23T03:24:20.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.918+0000 7fd39ebc0640 1 -- 172.21.15.154:0/2726711128 shutdown_connections 2023-09-23T03:24:20.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.918+0000 7fd39ebc0640 1 --2- 172.21.15.154:0/2726711128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd3980cb5b0 0x7fd3980cc5c0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:20.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.918+0000 7fd39ebc0640 1 -- 172.21.15.154:0/2726711128 >> 172.21.15.154:0/2726711128 conn(0x7fd39806c010 msgr2=0x7fd39806c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:20.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.918+0000 7fd39ebc0640 1 -- 172.21.15.154:0/2726711128 shutdown_connections 2023-09-23T03:24:20.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.918+0000 7fd39ebc0640 1 -- 172.21.15.154:0/2726711128 wait complete. 2023-09-23T03:24:20.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.919+0000 7fd39ebc0640 1 Processor -- start 2023-09-23T03:24:20.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.920+0000 7fd39ebc0640 1 -- start start 2023-09-23T03:24:20.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.921+0000 7fd39ebc0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3980cb5b0 0x7fd39815e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:20.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.921+0000 7fd39ebc0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd39815aba0 0x7fd3980ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:20.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.921+0000 7fd39ebc0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd39815bb30 0x7fd3981f1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:20.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.921+0000 7fd39ebc0640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3980c97b0 con 0x7fd39815bb30 2023-09-23T03:24:20.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.921+0000 7fd39ebc0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3980c8ba0 con 0x7fd39815aba0 2023-09-23T03:24:20.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.921+0000 7fd39ebc0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3980c8e80 con 0x7fd3980cb5b0 2023-09-23T03:24:20.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.921+0000 7fd39d15d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd39815bb30 0x7fd3981f1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:20.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.921+0000 7fd39d15d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd39815bb30 0x7fd3981f1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:51012/0 (socket says 172.21.15.154:51012) 2023-09-23T03:24:20.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.921+0000 7fd39d15d640 1 -- 172.21.15.154:0/1920003238 learned_addr learned my addr 172.21.15.154:0/1920003238 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:20.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.921+0000 7fd397fff640 1 --2- 172.21.15.154:0/1920003238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd39815aba0 0x7fd3980ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:20.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.921+0000 7fd39c95c640 1 --2- 172.21.15.154:0/1920003238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3980cb5b0 0x7fd39815e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:20.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.921+0000 7fd39d15d640 1 -- 172.21.15.154:0/1920003238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3980cb5b0 msgr2=0x7fd39815e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:20.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.921+0000 7fd39d15d640 1 --2- 172.21.15.154:0/1920003238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3980cb5b0 0x7fd39815e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:20.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.921+0000 7fd39d15d640 1 -- 172.21.15.154:0/1920003238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd39815aba0 msgr2=0x7fd3980ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:20.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.921+0000 7fd39d15d640 1 --2- 172.21.15.154:0/1920003238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd39815aba0 0x7fd3980ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:20.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.921+0000 7fd39d15d640 1 -- 172.21.15.154:0/1920003238 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd390009a70 con 0x7fd39815bb30 2023-09-23T03:24:20.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.922+0000 7fd397fff640 1 --2- 172.21.15.154:0/1920003238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd39815aba0 0x7fd3980ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:20.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.922+0000 7fd39c95c640 1 --2- 172.21.15.154:0/1920003238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3980cb5b0 0x7fd39815e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:20.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.922+0000 7fd39d15d640 1 --2- 172.21.15.154:0/1920003238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd39815bb30 0x7fd3981f1810 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fd390009d90 tx=0x7fd390004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:20.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.922+0000 7fd3957fa640 1 -- 172.21.15.154:0/1920003238 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd390003b90 con 0x7fd39815bb30 2023-09-23T03:24:20.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.922+0000 7fd3957fa640 1 -- 172.21.15.154:0/1920003238 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd390003cf0 con 0x7fd39815bb30 2023-09-23T03:24:20.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.922+0000 7fd39ebc0640 1 -- 172.21.15.154:0/1920003238 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3980c9160 con 0x7fd39815bb30 2023-09-23T03:24:20.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.922+0000 7fd3957fa640 1 -- 172.21.15.154:0/1920003238 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd390004230 con 0x7fd39815bb30 2023-09-23T03:24:20.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.922+0000 7fd39ebc0640 1 -- 172.21.15.154:0/1920003238 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3980adbf0 con 0x7fd39815bb30 2023-09-23T03:24:20.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.923+0000 7fd39ebc0640 1 -- 172.21.15.154:0/1920003238 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3980add80 con 0x7fd39815bb30 2023-09-23T03:24:20.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.925+0000 7fd3957fa640 1 -- 172.21.15.154:0/1920003238 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fd39003b050 con 0x7fd39815bb30 2023-09-23T03:24:20.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.926+0000 7fd3957fa640 1 --2- 172.21.15.154:0/1920003238 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd37c06af30 0x7fd37c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:20.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.926+0000 7fd3957fa640 1 -- 172.21.15.154:0/1920003238 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6401+0+0 (secure 0 0 0) 0x7fd3900acaa0 con 0x7fd39815bb30 2023-09-23T03:24:20.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.926+0000 7fd39c95c640 1 --2- 172.21.15.154:0/1920003238 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd37c06af30 0x7fd37c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:20.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.927+0000 7fd39c95c640 1 --2- 172.21.15.154:0/1920003238 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd37c06af30 0x7fd37c06d3f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd38c000f60 tx=0x7fd38c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:20.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:20.930+0000 7fd3957fa640 1 -- 172.21.15.154:0/1920003238 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fd390077920 con 0x7fd39815bb30 2023-09-23T03:24:21.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.158+0000 7fd39ebc0640 1 -- 172.21.15.154:0/1920003238 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_extra"} v 0) v1 -- 0x7fd39804e920 con 0x7fd39815bb30 2023-09-23T03:24:21.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.695+0000 7fd3957fa640 1 -- 172.21.15.154:0/1920003238 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_extra"}]=0 added data pool 13 to fsmap v63) v1 ==== 141+0+0 (secure 0 0 0) 0x7fd390039020 con 0x7fd39815bb30 2023-09-23T03:24:21.695 INFO:teuthology.orchestra.run.smithi154.stderr:added data pool 13 to fsmap 2023-09-23T03:24:21.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.698+0000 7fd39ebc0640 1 -- 172.21.15.154:0/1920003238 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd37c06af30 msgr2=0x7fd37c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:21.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.698+0000 7fd39ebc0640 1 --2- 172.21.15.154:0/1920003238 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd37c06af30 0x7fd37c06d3f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd38c000f60 tx=0x7fd38c009040 comp rx=0 tx=0).stop 2023-09-23T03:24:21.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.698+0000 7fd39ebc0640 1 -- 172.21.15.154:0/1920003238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd39815bb30 msgr2=0x7fd3981f1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:21.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.698+0000 7fd39ebc0640 1 --2- 172.21.15.154:0/1920003238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd39815bb30 0x7fd3981f1810 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fd390009d90 tx=0x7fd390004a40 comp rx=0 tx=0).stop 2023-09-23T03:24:21.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.698+0000 7fd39ebc0640 1 -- 172.21.15.154:0/1920003238 shutdown_connections 2023-09-23T03:24:21.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.698+0000 7fd39ebc0640 1 --2- 172.21.15.154:0/1920003238 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd37c06af30 0x7fd37c06d3f0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:21.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.698+0000 7fd39ebc0640 1 --2- 172.21.15.154:0/1920003238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd39815bb30 0x7fd3981f1810 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:21.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.698+0000 7fd39ebc0640 1 --2- 172.21.15.154:0/1920003238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd39815aba0 0x7fd3980ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:21.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.698+0000 7fd39ebc0640 1 --2- 172.21.15.154:0/1920003238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3980cb5b0 0x7fd39815e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:21.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.698+0000 7fd39ebc0640 1 -- 172.21.15.154:0/1920003238 >> 172.21.15.154:0/1920003238 conn(0x7fd39806c010 msgr2=0x7fd39815d240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:21.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.699+0000 7fd39ebc0640 1 -- 172.21.15.154:0/1920003238 shutdown_connections 2023-09-23T03:24:21.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.699+0000 7fd39ebc0640 1 -- 172.21.15.154:0/1920003238 wait complete. 2023-09-23T03:24:21.719 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:24:21.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.932+0000 7f6622553640 1 Processor -- start 2023-09-23T03:24:21.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.932+0000 7f6622553640 1 -- start start 2023-09-23T03:24:21.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.933+0000 7f6622553640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f661c0ada30 0x7f661c0ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:21.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.934+0000 7f6622553640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f661c0b2fe0 0x7f661c0cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:21.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.934+0000 7f6622553640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f661c06ceb0 con 0x7f661c0cef60 2023-09-23T03:24:21.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.934+0000 7f6622553640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f661c0ade10 con 0x7f661c0b2fe0 2023-09-23T03:24:21.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.934+0000 7f6622553640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f661c0cf3b0 con 0x7f661c0ada30 2023-09-23T03:24:21.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.934+0000 7f6620af0640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f661c0cef60 0x7f661c0b18f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:59566/0 (socket says 172.21.15.154:59566) 2023-09-23T03:24:21.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.934+0000 7f6620af0640 1 -- 172.21.15.154:0/3332681579 learned_addr learned my addr 172.21.15.154:0/3332681579 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:24:21.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.934+0000 7f66137fe640 1 --2- 172.21.15.154:0/3332681579 >> v2:172.21.15.190:3300/0 conn(0x7f661c0b2fe0 0x7f661c0cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:21.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.934+0000 7f661bfff640 1 --2- 172.21.15.154:0/3332681579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f661c0ada30 0x7f661c0ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:21.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.935+0000 7f661b7fe640 1 -- 172.21.15.154:0/3332681579 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2720249145 0 0) 0x7f6614001490 con 0x7f661c0cef60 2023-09-23T03:24:21.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.935+0000 7f661b7fe640 1 -- 172.21.15.154:0/3332681579 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6608005730 con 0x7f661c0cef60 2023-09-23T03:24:21.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.935+0000 7f661b7fe640 1 -- 172.21.15.154:0/3332681579 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 745992596 0 0) 0x7f6608005730 con 0x7f661c0cef60 2023-09-23T03:24:21.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.935+0000 7f661b7fe640 1 -- 172.21.15.154:0/3332681579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f661c0ada30 msgr2=0x7f661c0ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:21.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.935+0000 7f661b7fe640 1 --2- 172.21.15.154:0/3332681579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f661c0ada30 0x7f661c0ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:21.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.935+0000 7f661b7fe640 1 -- 172.21.15.154:0/3332681579 >> v2:172.21.15.190:3300/0 conn(0x7f661c0b2fe0 msgr2=0x7f661c0cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:21.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.935+0000 7f661b7fe640 1 --2- 172.21.15.154:0/3332681579 >> v2:172.21.15.190:3300/0 conn(0x7f661c0b2fe0 0x7f661c0cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:21.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.935+0000 7f661b7fe640 1 -- 172.21.15.154:0/3332681579 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f661c159c90 con 0x7f661c0cef60 2023-09-23T03:24:21.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.935+0000 7f661b7fe640 1 -- 172.21.15.154:0/3332681579 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f6614001b80 con 0x7f661c0cef60 2023-09-23T03:24:21.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.935+0000 7f661b7fe640 1 -- 172.21.15.154:0/3332681579 >> v1:172.21.15.154:6789/0 conn(0x7f661c0cef60 legacy=0x7f661c0b18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:21.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.935+0000 7f661b7fe640 1 --2- 172.21.15.154:0/3332681579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f661c0ada30 0x7f6608005b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:21.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.935+0000 7f661b7fe640 1 --2- 172.21.15.154:0/3332681579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6608006100 0x7f66080069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:21.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.935+0000 7f661b7fe640 1 --2- 172.21.15.154:0/3332681579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6608006f10 0x7f6608007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:21.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.935+0000 7f661b7fe640 1 -- 172.21.15.154:0/3332681579 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661c0ade10 con 0x7f6608006100 2023-09-23T03:24:21.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.935+0000 7f661b7fe640 1 -- 172.21.15.154:0/3332681579 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661c0cf3b0 con 0x7f6608006f10 2023-09-23T03:24:21.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.935+0000 7f661b7fe640 1 -- 172.21.15.154:0/3332681579 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f660800db90 con 0x7f661c0ada30 2023-09-23T03:24:21.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.936+0000 7f66137fe640 1 --2- 172.21.15.154:0/3332681579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6608006100 0x7f66080069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:21.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.936+0000 7f6620af0640 1 --2- 172.21.15.154:0/3332681579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6608006f10 0x7f6608007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:21.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.936+0000 7f661bfff640 1 --2- 172.21.15.154:0/3332681579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f661c0ada30 0x7f6608005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:21.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.936+0000 7f66137fe640 1 -- 172.21.15.154:0/3332681579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f661c0ada30 msgr2=0x7f6608005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:21.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.936+0000 7f66137fe640 1 --2- 172.21.15.154:0/3332681579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f661c0ada30 0x7f6608005b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:21.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.936+0000 7f66137fe640 1 -- 172.21.15.154:0/3332681579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6608006f10 msgr2=0x7f6608007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:21.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.936+0000 7f66137fe640 1 --2- 172.21.15.154:0/3332681579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6608006f10 0x7f6608007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:21.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.936+0000 7f66137fe640 1 -- 172.21.15.154:0/3332681579 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6608005730 con 0x7f6608006100 2023-09-23T03:24:21.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.937+0000 7f66137fe640 1 --2- 172.21.15.154:0/3332681579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6608006100 0x7f66080069d0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f6604009ac0 tx=0x7f6608007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=791e88a7a2c33d0c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:21.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.937+0000 7f661b7fe640 1 -- 172.21.15.154:0/3332681579 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660402c6d0 con 0x7f6608006100 2023-09-23T03:24:21.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.937+0000 7f661b7fe640 1 -- 172.21.15.154:0/3332681579 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f660402c830 con 0x7f6608006100 2023-09-23T03:24:21.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.937+0000 7f661b7fe640 1 -- 172.21.15.154:0/3332681579 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660402cbc0 con 0x7f6608006100 2023-09-23T03:24:21.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.938+0000 7f6622553640 1 -- 172.21.15.154:0/3332681579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6608006100 msgr2=0x7f66080069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:21.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.938+0000 7f6622553640 1 --2- 172.21.15.154:0/3332681579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6608006100 0x7f66080069d0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f6604009ac0 tx=0x7f6608007a00 comp rx=0 tx=0).stop 2023-09-23T03:24:21.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.938+0000 7f6620af0640 1 -- 172.21.15.154:0/3332681579 reap_dead start 2023-09-23T03:24:21.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.938+0000 7f6622553640 1 -- 172.21.15.154:0/3332681579 shutdown_connections 2023-09-23T03:24:21.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.938+0000 7f6622553640 1 -- 172.21.15.154:0/3332681579 >> 172.21.15.154:0/3332681579 conn(0x7f661c06a0f0 msgr2=0x7f661c06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:21.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.938+0000 7f6622553640 1 -- 172.21.15.154:0/3332681579 shutdown_connections 2023-09-23T03:24:21.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.938+0000 7f6622553640 1 -- 172.21.15.154:0/3332681579 wait complete. 2023-09-23T03:24:21.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.940+0000 7f6622553640 1 Processor -- start 2023-09-23T03:24:21.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.940+0000 7f6622553640 1 -- start start 2023-09-23T03:24:21.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.941+0000 7f6622553640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6608006100 0x7f661c1f5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:21.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.941+0000 7f6622553640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f661c0af400 0x7f661c0af860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:21.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.941+0000 7f6622553640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f661c15b560 0x7f661c15b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:21.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.941+0000 7f6622553640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661c0afda0 con 0x7f661c0af400 2023-09-23T03:24:21.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.941+0000 7f6622553640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661c0c1ee0 con 0x7f6608006100 2023-09-23T03:24:21.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.941+0000 7f6622553640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f661c15c2d0 con 0x7f661c15b560 2023-09-23T03:24:21.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.941+0000 7f66137fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f661c0af400 0x7f661c0af860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:21.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.941+0000 7f66137fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f661c0af400 0x7f661c0af860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:51036/0 (socket says 172.21.15.154:51036) 2023-09-23T03:24:21.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.941+0000 7f66137fe640 1 -- 172.21.15.154:0/3785018191 learned_addr learned my addr 172.21.15.154:0/3785018191 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:21.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.942+0000 7f66137fe640 1 -- 172.21.15.154:0/3785018191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f661c15b560 msgr2=0x7f661c15b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:21.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.942+0000 7f66137fe640 1 --2- 172.21.15.154:0/3785018191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f661c15b560 0x7f661c15b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:21.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.942+0000 7f66137fe640 1 -- 172.21.15.154:0/3785018191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6608006100 msgr2=0x7f661c1f5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:21.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.942+0000 7f66137fe640 1 --2- 172.21.15.154:0/3785018191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6608006100 0x7f661c1f5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:21.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.942+0000 7f66137fe640 1 -- 172.21.15.154:0/3785018191 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f661c15c5b0 con 0x7f661c0af400 2023-09-23T03:24:21.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.942+0000 7f66137fe640 1 --2- 172.21.15.154:0/3785018191 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f661c0af400 0x7f661c0af860 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f660402c2d0 tx=0x7f6604004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:21.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.942+0000 7f66197fa640 1 -- 172.21.15.154:0/3785018191 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6604003810 con 0x7f661c0af400 2023-09-23T03:24:21.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.942+0000 7f66197fa640 1 -- 172.21.15.154:0/3785018191 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6604003970 con 0x7f661c0af400 2023-09-23T03:24:21.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.942+0000 7f6622553640 1 -- 172.21.15.154:0/3785018191 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f661c15c890 con 0x7f661c0af400 2023-09-23T03:24:21.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.942+0000 7f66197fa640 1 -- 172.21.15.154:0/3785018191 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6604003ce0 con 0x7f661c0af400 2023-09-23T03:24:21.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.943+0000 7f6622553640 1 -- 172.21.15.154:0/3785018191 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f661c15bf20 con 0x7f661c0af400 2023-09-23T03:24:21.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.944+0000 7f6622553640 1 -- 172.21.15.154:0/3785018191 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f661c15cdb0 con 0x7f661c0af400 2023-09-23T03:24:21.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.945+0000 7f66197fa640 1 -- 172.21.15.154:0/3785018191 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f6604040450 con 0x7f661c0af400 2023-09-23T03:24:21.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.946+0000 7f66197fa640 1 --2- 172.21.15.154:0/3785018191 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f65fc06af60 0x7f65fc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:21.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.946+0000 7f66197fa640 1 -- 172.21.15.154:0/3785018191 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f66040ab8a0 con 0x7f661c0af400 2023-09-23T03:24:21.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.946+0000 7f661bfff640 1 --2- 172.21.15.154:0/3785018191 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f65fc06af60 0x7f65fc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:21.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.947+0000 7f661bfff640 1 --2- 172.21.15.154:0/3785018191 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f65fc06af60 0x7f65fc06d420 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f660c009820 tx=0x7f660c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:21.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:21.950+0000 7f66197fa640 1 -- 172.21.15.154:0/3785018191 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f660407a120 con 0x7f661c0af400 2023-09-23T03:24:22.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.179+0000 7f6622553640 1 -- 172.21.15.154:0/3785018191 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f661c06c6a0 con 0x7f661c0af400 2023-09-23T03:24:22.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.180+0000 7f66197fa640 1 -- 172.21.15.154:0/3785018191 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f66040044e0 con 0x7f661c0af400 2023-09-23T03:24:22.180 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:22.180 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":63,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/728936136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":728936136},{"type":"v1","addr":"172.21.15.190:6833","nonce":728936136}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":5890,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2277229238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2277229238},{"type":"v1","addr":"172.21.15.190:6835","nonce":2277229238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":7257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/4245928089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4245928089},{"type":"v1","addr":"172.21.15.154:6835","nonce":4245928089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:24:10.061474+0000","modified":"2023-09-23T03:24:21.160286+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7260":{"gid":7260,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6837/2213283923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2213283923},{"type":"v1","addr":"172.21.15.154:6837","nonce":2213283923}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12,13],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-09-23T03:24:22.181 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 63 2023-09-23T03:24:22.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.182+0000 7f6622553640 1 -- 172.21.15.154:0/3785018191 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f65fc06af60 msgr2=0x7f65fc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:22.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.182+0000 7f6622553640 1 --2- 172.21.15.154:0/3785018191 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f65fc06af60 0x7f65fc06d420 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f660c009820 tx=0x7f660c008040 comp rx=0 tx=0).stop 2023-09-23T03:24:22.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.182+0000 7f6622553640 1 -- 172.21.15.154:0/3785018191 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f661c0af400 msgr2=0x7f661c0af860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:22.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.182+0000 7f6622553640 1 --2- 172.21.15.154:0/3785018191 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f661c0af400 0x7f661c0af860 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f660402c2d0 tx=0x7f6604004730 comp rx=0 tx=0).stop 2023-09-23T03:24:22.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.183+0000 7f6622553640 1 -- 172.21.15.154:0/3785018191 shutdown_connections 2023-09-23T03:24:22.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.183+0000 7f6622553640 1 --2- 172.21.15.154:0/3785018191 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f65fc06af60 0x7f65fc06d420 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:22.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.183+0000 7f6622553640 1 --2- 172.21.15.154:0/3785018191 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f661c15b560 0x7f661c15b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:22.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.183+0000 7f6622553640 1 --2- 172.21.15.154:0/3785018191 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f661c0af400 0x7f661c0af860 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:22.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.183+0000 7f6622553640 1 --2- 172.21.15.154:0/3785018191 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6608006100 0x7f661c1f5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:22.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.183+0000 7f6622553640 1 -- 172.21.15.154:0/3785018191 >> 172.21.15.154:0/3785018191 conn(0x7f661c06a0f0 msgr2=0x7f661c0c9820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:22.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.183+0000 7f6622553640 1 -- 172.21.15.154:0/3785018191 shutdown_connections 2023-09-23T03:24:22.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.183+0000 7f6622553640 1 -- 172.21.15.154:0/3785018191 wait complete. 2023-09-23T03:24:22.201 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:24:22.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.420+0000 7f8d6f0c2640 1 Processor -- start 2023-09-23T03:24:22.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.420+0000 7f8d6f0c2640 1 -- start start 2023-09-23T03:24:22.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.422+0000 7f8d6f0c2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d680acce0 0x7f8d6815ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:22.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.422+0000 7f8d6f0c2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8d680b1b20 0x7f8d680ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:22.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.422+0000 7f8d6f0c2640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d6815ac30 con 0x7f8d680cee70 2023-09-23T03:24:22.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.422+0000 7f8d6f0c2640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d680b2e90 con 0x7f8d680b1b20 2023-09-23T03:24:22.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.422+0000 7f8d6f0c2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d680cf320 con 0x7f8d680acce0 2023-09-23T03:24:22.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.422+0000 7f8d67fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8d680b1b20 0x7f8d680ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:22.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.422+0000 7f8d67fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8d680b1b20 0x7f8d680ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:51040/0 (socket says 172.21.15.154:51040) 2023-09-23T03:24:22.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.422+0000 7f8d67fff640 1 -- 172.21.15.154:0/1158855102 learned_addr learned my addr 172.21.15.154:0/1158855102 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:22.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.422+0000 7f8d6ce5e640 1 --2- 172.21.15.154:0/1158855102 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d680acce0 0x7f8d6815ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:22.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.422+0000 7f8d67fff640 1 -- 172.21.15.154:0/1158855102 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d680acce0 msgr2=0x7f8d6815ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:22.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.422+0000 7f8d67fff640 1 --2- 172.21.15.154:0/1158855102 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d680acce0 0x7f8d6815ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:22.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.422+0000 7f8d67fff640 1 -- 172.21.15.154:0/1158855102 >> v1:172.21.15.190:6789/0 conn(0x7f8d680cee70 legacy=0x7f8d6806cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:22.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.423+0000 7f8d67fff640 1 -- 172.21.15.154:0/1158855102 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d680b1f70 con 0x7f8d680b1b20 2023-09-23T03:24:22.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.424+0000 7f8d67fff640 1 --2- 172.21.15.154:0/1158855102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d680b1b20 0x7f8d680ce930 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f8d54009870 tx=0x7f8d5402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3875344c4ba500d3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:22.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.424+0000 7f8d677fe640 1 -- 172.21.15.154:0/1158855102 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d540317c0 con 0x7f8d680b1b20 2023-09-23T03:24:22.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.424+0000 7f8d677fe640 1 -- 172.21.15.154:0/1158855102 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d54031920 con 0x7f8d680b1b20 2023-09-23T03:24:22.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.424+0000 7f8d677fe640 1 -- 172.21.15.154:0/1158855102 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d54031cb0 con 0x7f8d680b1b20 2023-09-23T03:24:22.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.424+0000 7f8d6f0c2640 1 -- 172.21.15.154:0/1158855102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d680b1b20 msgr2=0x7f8d680ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:22.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.424+0000 7f8d6f0c2640 1 --2- 172.21.15.154:0/1158855102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d680b1b20 0x7f8d680ce930 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f8d54009870 tx=0x7f8d5402b860 comp rx=0 tx=0).stop 2023-09-23T03:24:22.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.425+0000 7f8d6f0c2640 1 -- 172.21.15.154:0/1158855102 shutdown_connections 2023-09-23T03:24:22.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.425+0000 7f8d6f0c2640 1 --2- 172.21.15.154:0/1158855102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d680b1b20 0x7f8d680ce930 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:22.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.425+0000 7f8d6f0c2640 1 --2- 172.21.15.154:0/1158855102 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d680acce0 0x7f8d6815ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:22.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.425+0000 7f8d6f0c2640 1 -- 172.21.15.154:0/1158855102 >> 172.21.15.154:0/1158855102 conn(0x7f8d6806a0f0 msgr2=0x7f8d6806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:22.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.425+0000 7f8d6f0c2640 1 -- 172.21.15.154:0/1158855102 shutdown_connections 2023-09-23T03:24:22.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.425+0000 7f8d6f0c2640 1 -- 172.21.15.154:0/1158855102 wait complete. 2023-09-23T03:24:22.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.426+0000 7f8d6f0c2640 1 Processor -- start 2023-09-23T03:24:22.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.426+0000 7f8d6f0c2640 1 -- start start 2023-09-23T03:24:22.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.427+0000 7f8d6f0c2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d680acce0 0x7f8d6815cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:22.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.427+0000 7f8d6f0c2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d680b1b20 0x7f8d680c7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:22.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.427+0000 7f8d6f0c2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d680cee70 0x7f8d681598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:22.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.427+0000 7f8d6f0c2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6815d230 con 0x7f8d680acce0 2023-09-23T03:24:22.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.427+0000 7f8d6f0c2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6815d510 con 0x7f8d680b1b20 2023-09-23T03:24:22.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.427+0000 7f8d6f0c2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6815d7f0 con 0x7f8d680cee70 2023-09-23T03:24:22.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.428+0000 7f8d6ce5e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d680acce0 0x7f8d6815cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:22.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.428+0000 7f8d6ce5e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d680acce0 0x7f8d6815cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:51048/0 (socket says 172.21.15.154:51048) 2023-09-23T03:24:22.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.428+0000 7f8d6ce5e640 1 -- 172.21.15.154:0/3033126420 learned_addr learned my addr 172.21.15.154:0/3033126420 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:22.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.428+0000 7f8d6d65f640 1 --2- 172.21.15.154:0/3033126420 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d680cee70 0x7f8d681598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:22.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.428+0000 7f8d67fff640 1 --2- 172.21.15.154:0/3033126420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d680b1b20 0x7f8d680c7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:22.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.428+0000 7f8d6ce5e640 1 -- 172.21.15.154:0/3033126420 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d680cee70 msgr2=0x7f8d681598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:22.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.428+0000 7f8d6ce5e640 1 --2- 172.21.15.154:0/3033126420 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d680cee70 0x7f8d681598c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:22.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.428+0000 7f8d6ce5e640 1 -- 172.21.15.154:0/3033126420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d680b1b20 msgr2=0x7f8d680c7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:22.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.428+0000 7f8d6ce5e640 1 --2- 172.21.15.154:0/3033126420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d680b1b20 0x7f8d680c7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:22.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.428+0000 7f8d6ce5e640 1 -- 172.21.15.154:0/3033126420 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d540094d0 con 0x7f8d680acce0 2023-09-23T03:24:22.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.428+0000 7f8d6ce5e640 1 --2- 172.21.15.154:0/3033126420 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d680acce0 0x7f8d6815cc50 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c00c7f0 tx=0x7f8d5c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:22.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.429+0000 7f8d657fa640 1 -- 172.21.15.154:0/3033126420 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c007820 con 0x7f8d680acce0 2023-09-23T03:24:22.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.429+0000 7f8d657fa640 1 -- 172.21.15.154:0/3033126420 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d5c007980 con 0x7f8d680acce0 2023-09-23T03:24:22.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.429+0000 7f8d657fa640 1 -- 172.21.15.154:0/3033126420 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c007d30 con 0x7f8d680acce0 2023-09-23T03:24:22.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.429+0000 7f8d6f0c2640 1 -- 172.21.15.154:0/3033126420 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d6815dad0 con 0x7f8d680acce0 2023-09-23T03:24:22.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.429+0000 7f8d6f0c2640 1 -- 172.21.15.154:0/3033126420 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d680c31e0 con 0x7f8d680acce0 2023-09-23T03:24:22.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.430+0000 7f8d6f0c2640 1 -- 172.21.15.154:0/3033126420 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d6815dd10 con 0x7f8d680acce0 2023-09-23T03:24:22.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.431+0000 7f8d657fa640 1 -- 172.21.15.154:0/3033126420 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f8d5c004020 con 0x7f8d680acce0 2023-09-23T03:24:22.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.432+0000 7f8d657fa640 1 --2- 172.21.15.154:0/3033126420 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8d4c06af30 0x7f8d4c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:22.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.432+0000 7f8d657fa640 1 -- 172.21.15.154:0/3033126420 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f8d5c089d90 con 0x7f8d680acce0 2023-09-23T03:24:22.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.432+0000 7f8d67fff640 1 --2- 172.21.15.154:0/3033126420 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8d4c06af30 0x7f8d4c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:22.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.433+0000 7f8d67fff640 1 --2- 172.21.15.154:0/3033126420 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8d4c06af30 0x7f8d4c06d3f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8d54036000 tx=0x7f8d5403d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:22.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.436+0000 7f8d657fa640 1 -- 172.21.15.154:0/3033126420 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f8d5c054ae0 con 0x7f8d680acce0 2023-09-23T03:24:22.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.625+0000 7f8d6f0c2640 1 -- 172.21.15.154:0/3033126420 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d68073eb0 con 0x7f8d680acce0 2023-09-23T03:24:22.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.628+0000 7f8d657fa640 1 -- 172.21.15.154:0/3033126420 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+18598 (secure 0 0 0) 0x7f8d5c01f020 con 0x7f8d680acce0 2023-09-23T03:24:22.629 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:22.629 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":86,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:24:22.327793+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:24:07.834123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-09-23T03:24:09.058217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data_extra","create_time":"2023-09-23T03:24:19.995906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:24:22.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.630+0000 7f8d6f0c2640 1 -- 172.21.15.154:0/3033126420 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8d4c06af30 msgr2=0x7f8d4c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:22.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.630+0000 7f8d6f0c2640 1 --2- 172.21.15.154:0/3033126420 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8d4c06af30 0x7f8d4c06d3f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8d54036000 tx=0x7f8d5403d040 comp rx=0 tx=0).stop 2023-09-23T03:24:22.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.630+0000 7f8d6f0c2640 1 -- 172.21.15.154:0/3033126420 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d680acce0 msgr2=0x7f8d6815cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:22.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.630+0000 7f8d6f0c2640 1 --2- 172.21.15.154:0/3033126420 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d680acce0 0x7f8d6815cc50 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c00c7f0 tx=0x7f8d5c00ccb0 comp rx=0 tx=0).stop 2023-09-23T03:24:22.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.631+0000 7f8d6f0c2640 1 -- 172.21.15.154:0/3033126420 shutdown_connections 2023-09-23T03:24:22.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.631+0000 7f8d6f0c2640 1 --2- 172.21.15.154:0/3033126420 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8d4c06af30 0x7f8d4c06d3f0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:22.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.631+0000 7f8d6f0c2640 1 --2- 172.21.15.154:0/3033126420 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8d680cee70 0x7f8d681598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:22.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.631+0000 7f8d6f0c2640 1 --2- 172.21.15.154:0/3033126420 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8d680b1b20 0x7f8d680c7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:22.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.631+0000 7f8d6f0c2640 1 --2- 172.21.15.154:0/3033126420 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8d680acce0 0x7f8d6815cc50 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:22.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.631+0000 7f8d6f0c2640 1 -- 172.21.15.154:0/3033126420 >> 172.21.15.154:0/3033126420 conn(0x7f8d6806a0f0 msgr2=0x7f8d680cb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:22.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.631+0000 7f8d6f0c2640 1 -- 172.21.15.154:0/3033126420 shutdown_connections 2023-09-23T03:24:22.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:22.631+0000 7f8d6f0c2640 1 -- 172.21.15.154:0/3033126420 wait complete. 2023-09-23T03:24:22.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:24:22.651 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v cephfs_data_extra .) 2023-09-23T03:24:22.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:24:22.716 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2023-09-23T03:24:22.778 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:24:22.778 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:24:22.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:24:22.830 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=subdir/sixmegs bs=1M conv=fdatasync count=6 seek=0) 2023-09-23T03:24:23.052 INFO:teuthology.orchestra.run.smithi154.stderr:6+0 records in 2023-09-23T03:24:23.052 INFO:teuthology.orchestra.run.smithi154.stderr:6+0 records out 2023-09-23T03:24:23.052 INFO:teuthology.orchestra.run.smithi154.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.163904 s, 38.4 MB/s 2023-09-23T03:24:23.054 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:24:23.054 DEBUG:teuthology.orchestra.run.smithi154:> import os 2023-09-23T03:24:23.054 DEBUG:teuthology.orchestra.run.smithi154:> import stat 2023-09-23T03:24:23.054 DEBUG:teuthology.orchestra.run.smithi154:> import json 2023-09-23T03:24:23.054 DEBUG:teuthology.orchestra.run.smithi154:> import sys 2023-09-23T03:24:23.054 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:24:23.054 DEBUG:teuthology.orchestra.run.smithi154:> try: 2023-09-23T03:24:23.054 DEBUG:teuthology.orchestra.run.smithi154:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2023-09-23T03:24:23.055 DEBUG:teuthology.orchestra.run.smithi154:> except OSError as e: 2023-09-23T03:24:23.055 DEBUG:teuthology.orchestra.run.smithi154:> sys.exit(e.errno) 2023-09-23T03:24:23.055 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:24:23.055 DEBUG:teuthology.orchestra.run.smithi154:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2023-09-23T03:24:23.055 DEBUG:teuthology.orchestra.run.smithi154:> print(json.dumps( 2023-09-23T03:24:23.055 DEBUG:teuthology.orchestra.run.smithi154:> dict([(a, getattr(s, a)) for a in attrs]), 2023-09-23T03:24:23.055 DEBUG:teuthology.orchestra.run.smithi154:> indent=2)) 2023-09-23T03:24:23.055 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:24:23.203 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:24:23.204 INFO:teuthology.orchestra.run.smithi154.stdout: "st_mode": 33188, 2023-09-23T03:24:23.204 INFO:teuthology.orchestra.run.smithi154.stdout: "st_ino": 1099511627777, 2023-09-23T03:24:23.204 INFO:teuthology.orchestra.run.smithi154.stdout: "st_dev": 39, 2023-09-23T03:24:23.204 INFO:teuthology.orchestra.run.smithi154.stdout: "st_nlink": 1, 2023-09-23T03:24:23.204 INFO:teuthology.orchestra.run.smithi154.stdout: "st_uid": 1000, 2023-09-23T03:24:23.204 INFO:teuthology.orchestra.run.smithi154.stdout: "st_gid": 1301, 2023-09-23T03:24:23.204 INFO:teuthology.orchestra.run.smithi154.stdout: "st_size": 6291456, 2023-09-23T03:24:23.204 INFO:teuthology.orchestra.run.smithi154.stdout: "st_atime": 1695439462.8858285, 2023-09-23T03:24:23.204 INFO:teuthology.orchestra.run.smithi154.stdout: "st_mtime": 1695439462.9955971, 2023-09-23T03:24:23.205 INFO:teuthology.orchestra.run.smithi154.stdout: "st_ctime": 1695439462.9955971 2023-09-23T03:24:23.205 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:24:23.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:24:23.217 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:24:23.217 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:24:23.315 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:24:23.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.526+0000 7fdfbd410640 1 Processor -- start 2023-09-23T03:24:23.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.526+0000 7fdfbd410640 1 -- start start 2023-09-23T03:24:23.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.527+0000 7fdfbd410640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfb80acce0 0x7fdfb815ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:23.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.527+0000 7fdfbd410640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdfb80cf040 0x7fdfb80ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:23.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.527+0000 7fdfbd410640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfb815ac30 con 0x7fdfb80b1b20 2023-09-23T03:24:23.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.527+0000 7fdfbd410640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfb80b2e90 con 0x7fdfb80cf040 2023-09-23T03:24:23.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.527+0000 7fdfbd410640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb80b1f70 con 0x7fdfb80acce0 2023-09-23T03:24:23.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.528+0000 7fdfb77fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdfb80cf040 0x7fdfb80ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:23.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.528+0000 7fdfaffff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fdfb80b1b20 0x7fdfb80ce930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:59568/0 (socket says 172.21.15.154:59568) 2023-09-23T03:24:23.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.528+0000 7fdfaffff640 1 -- 172.21.15.154:0/991925658 learned_addr learned my addr 172.21.15.154:0/991925658 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:24:23.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.528+0000 7fdfb6ffd640 1 --2- 172.21.15.154:0/991925658 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfb80acce0 0x7fdfb815ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:23.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.528+0000 7fdfb77fe640 1 -- 172.21.15.154:0/991925658 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfb80acce0 msgr2=0x7fdfb815ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:23.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.528+0000 7fdfb77fe640 1 --2- 172.21.15.154:0/991925658 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfb80acce0 0x7fdfb815ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:23.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.528+0000 7fdfb77fe640 1 -- 172.21.15.154:0/991925658 >> v1:172.21.15.154:6789/0 conn(0x7fdfb80b1b20 legacy=0x7fdfb80ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:23.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.528+0000 7fdfb77fe640 1 -- 172.21.15.154:0/991925658 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb815f2a0 con 0x7fdfb80cf040 2023-09-23T03:24:23.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.528+0000 7fdfb6ffd640 1 --2- 172.21.15.154:0/991925658 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfb80acce0 0x7fdfb815ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:23.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.529+0000 7fdfb77fe640 1 --2- 172.21.15.154:0/991925658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfb80cf040 0x7fdfb80ab140 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009d90 tx=0x7fdfb002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bbfd55eafa5b5c4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:23.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.530+0000 7fdfb67fc640 1 -- 172.21.15.154:0/991925658 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb002cbc0 con 0x7fdfb80cf040 2023-09-23T03:24:23.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.530+0000 7fdfb67fc640 1 -- 172.21.15.154:0/991925658 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb002cd20 con 0x7fdfb80cf040 2023-09-23T03:24:23.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.530+0000 7fdfb67fc640 1 -- 172.21.15.154:0/991925658 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0034660 con 0x7fdfb80cf040 2023-09-23T03:24:23.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.530+0000 7fdfbd410640 1 -- 172.21.15.154:0/991925658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfb80cf040 msgr2=0x7fdfb80ab140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:23.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.530+0000 7fdfbd410640 1 --2- 172.21.15.154:0/991925658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfb80cf040 0x7fdfb80ab140 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009d90 tx=0x7fdfb002be10 comp rx=0 tx=0).stop 2023-09-23T03:24:23.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.530+0000 7fdfbd410640 1 -- 172.21.15.154:0/991925658 shutdown_connections 2023-09-23T03:24:23.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.530+0000 7fdfbd410640 1 --2- 172.21.15.154:0/991925658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfb80cf040 0x7fdfb80ab140 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:23.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.530+0000 7fdfbd410640 1 --2- 172.21.15.154:0/991925658 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfb80acce0 0x7fdfb815ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:23.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.530+0000 7fdfbd410640 1 -- 172.21.15.154:0/991925658 >> 172.21.15.154:0/991925658 conn(0x7fdfb806a0f0 msgr2=0x7fdfb806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:23.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.530+0000 7fdfbd410640 1 -- 172.21.15.154:0/991925658 shutdown_connections 2023-09-23T03:24:23.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.530+0000 7fdfbd410640 1 -- 172.21.15.154:0/991925658 wait complete. 2023-09-23T03:24:23.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.532+0000 7fdfbd410640 1 Processor -- start 2023-09-23T03:24:23.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.532+0000 7fdfbd410640 1 -- start start 2023-09-23T03:24:23.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.533+0000 7fdfbd410640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfb80acce0 0x7fdfb81f1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:23.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.533+0000 7fdfbd410640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdfb80b1b20 0x7fdfb814eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:23.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.533+0000 7fdfbd410640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfb80cf040 0x7fdfb80cd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:23.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.533+0000 7fdfbd410640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb8151d80 con 0x7fdfb80cf040 2023-09-23T03:24:23.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.533+0000 7fdfbd410640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb80c3f50 con 0x7fdfb80b1b20 2023-09-23T03:24:23.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.533+0000 7fdfbd410640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb80c3620 con 0x7fdfb80acce0 2023-09-23T03:24:23.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.533+0000 7fdfb77fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfb80cf040 0x7fdfb80cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:23.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.534+0000 7fdfb77fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfb80cf040 0x7fdfb80cd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:51052/0 (socket says 172.21.15.154:51052) 2023-09-23T03:24:23.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.534+0000 7fdfb77fe640 1 -- 172.21.15.154:0/4150240771 learned_addr learned my addr 172.21.15.154:0/4150240771 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:23.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.534+0000 7fdfaffff640 1 --2- 172.21.15.154:0/4150240771 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdfb80b1b20 0x7fdfb814eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:23.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.534+0000 7fdfb6ffd640 1 --2- 172.21.15.154:0/4150240771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfb80acce0 0x7fdfb81f1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:23.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.534+0000 7fdfb77fe640 1 -- 172.21.15.154:0/4150240771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfb80acce0 msgr2=0x7fdfb81f1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:23.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.534+0000 7fdfb77fe640 1 --2- 172.21.15.154:0/4150240771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfb80acce0 0x7fdfb81f1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:23.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.534+0000 7fdfb77fe640 1 -- 172.21.15.154:0/4150240771 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdfb80b1b20 msgr2=0x7fdfb814eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:23.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.534+0000 7fdfb77fe640 1 --2- 172.21.15.154:0/4150240771 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdfb80b1b20 0x7fdfb814eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:23.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.534+0000 7fdfb77fe640 1 -- 172.21.15.154:0/4150240771 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb0009a70 con 0x7fdfb80cf040 2023-09-23T03:24:23.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.534+0000 7fdfb77fe640 1 --2- 172.21.15.154:0/4150240771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfb80cf040 0x7fdfb80cd130 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009d90 tx=0x7fdfb0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:23.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.535+0000 7fdfaf7fe640 1 -- 172.21.15.154:0/4150240771 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0040650 con 0x7fdfb80cf040 2023-09-23T03:24:23.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.535+0000 7fdfaf7fe640 1 -- 172.21.15.154:0/4150240771 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb002ce90 con 0x7fdfb80cf040 2023-09-23T03:24:23.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.535+0000 7fdfbd410640 1 -- 172.21.15.154:0/4150240771 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfb80c3900 con 0x7fdfb80cf040 2023-09-23T03:24:23.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.535+0000 7fdfaf7fe640 1 -- 172.21.15.154:0/4150240771 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0040a10 con 0x7fdfb80cf040 2023-09-23T03:24:23.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.535+0000 7fdfbd410640 1 -- 172.21.15.154:0/4150240771 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfb814f3e0 con 0x7fdfb80cf040 2023-09-23T03:24:23.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.536+0000 7fdfbd410640 1 -- 172.21.15.154:0/4150240771 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfb80c4100 con 0x7fdfb80cf040 2023-09-23T03:24:23.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.537+0000 7fdfaf7fe640 1 -- 172.21.15.154:0/4150240771 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fdfb0040b70 con 0x7fdfb80cf040 2023-09-23T03:24:23.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.538+0000 7fdfaf7fe640 1 --2- 172.21.15.154:0/4150240771 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdf9806af30 0x7fdf9806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:23.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.538+0000 7fdfaf7fe640 1 -- 172.21.15.154:0/4150240771 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6433+0+0 (secure 0 0 0) 0x7fdfb00acaa0 con 0x7fdfb80cf040 2023-09-23T03:24:23.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.539+0000 7fdfb6ffd640 1 --2- 172.21.15.154:0/4150240771 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdf9806af30 0x7fdf9806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:23.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.539+0000 7fdfb6ffd640 1 --2- 172.21.15.154:0/4150240771 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdf9806af30 0x7fdf9806d3f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8006fd0 tx=0x7fdfa8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:23.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.541+0000 7fdfaf7fe640 1 -- 172.21.15.154:0/4150240771 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fdfb0077990 con 0x7fdfb80cf040 2023-09-23T03:24:23.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.738+0000 7fdfbd410640 1 -- 172.21.15.154:0/4150240771 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdfb80c40c0 con 0x7fdfb80cf040 2023-09-23T03:24:23.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.741+0000 7fdfaf7fe640 1 -- 172.21.15.154:0/4150240771 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+18598 (secure 0 0 0) 0x7fdfb007afb0 con 0x7fdfb80cf040 2023-09-23T03:24:23.742 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:23.742 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":86,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:24:22.327793+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:24:07.834123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-09-23T03:24:09.058217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data_extra","create_time":"2023-09-23T03:24:19.995906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:24:23.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.744+0000 7fdfbd410640 1 -- 172.21.15.154:0/4150240771 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdf9806af30 msgr2=0x7fdf9806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:23.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.744+0000 7fdfbd410640 1 --2- 172.21.15.154:0/4150240771 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdf9806af30 0x7fdf9806d3f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8006fd0 tx=0x7fdfa8008040 comp rx=0 tx=0).stop 2023-09-23T03:24:23.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.744+0000 7fdfbd410640 1 -- 172.21.15.154:0/4150240771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfb80cf040 msgr2=0x7fdfb80cd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:23.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.744+0000 7fdfbd410640 1 --2- 172.21.15.154:0/4150240771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfb80cf040 0x7fdfb80cd130 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009d90 tx=0x7fdfb0004a40 comp rx=0 tx=0).stop 2023-09-23T03:24:23.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.744+0000 7fdfbd410640 1 -- 172.21.15.154:0/4150240771 shutdown_connections 2023-09-23T03:24:23.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.744+0000 7fdfbd410640 1 --2- 172.21.15.154:0/4150240771 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdf9806af30 0x7fdf9806d3f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:23.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.744+0000 7fdfbd410640 1 --2- 172.21.15.154:0/4150240771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdfb80cf040 0x7fdfb80cd130 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:23.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.744+0000 7fdfbd410640 1 --2- 172.21.15.154:0/4150240771 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdfb80b1b20 0x7fdfb814eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:23.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.744+0000 7fdfbd410640 1 --2- 172.21.15.154:0/4150240771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdfb80acce0 0x7fdfb81f1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:23.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.744+0000 7fdfbd410640 1 -- 172.21.15.154:0/4150240771 >> 172.21.15.154:0/4150240771 conn(0x7fdfb806a0f0 msgr2=0x7fdfb80726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:23.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.744+0000 7fdfbd410640 1 -- 172.21.15.154:0/4150240771 shutdown_connections 2023-09-23T03:24:23.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.744+0000 7fdfbd410640 1 -- 172.21.15.154:0/4150240771 wait complete. 2023-09-23T03:24:23.764 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:24:07.834123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:24:09.058217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-09-23T03:24:19.995906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T03:24:23.764 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:24:23.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.981+0000 7f03d47d7640 1 Processor -- start 2023-09-23T03:24:23.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.982+0000 7f03d47d7640 1 -- start start 2023-09-23T03:24:23.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.983+0000 7f03d47d7640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f03cc06a490 0x7f03cc071080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:23.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.983+0000 7f03d47d7640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03cc15e090 con 0x7f03cc0cb6b0 2023-09-23T03:24:23.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.983+0000 7f03d47d7640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03cc15d240 con 0x7f03cc0cdaa0 2023-09-23T03:24:23.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.983+0000 7f03d47d7640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03cc15d480 con 0x7f03cc06a490 2023-09-23T03:24:23.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.983+0000 7f03d1d72640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f03cc0cb6b0 0x7f03cc0cd390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:38978/0 (socket says 172.21.15.154:38978) 2023-09-23T03:24:23.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.983+0000 7f03d1d72640 1 -- 172.21.15.154:0/750040471 learned_addr learned my addr 172.21.15.154:0/750040471 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:24:23.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.984+0000 7f03d1571640 1 -- 172.21.15.154:0/750040471 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4050035480 0 0) 0x7f03cc15e090 con 0x7f03cc0cb6b0 2023-09-23T03:24:23.986 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.984+0000 7f03d1571640 1 -- 172.21.15.154:0/750040471 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03b4003520 con 0x7f03cc0cb6b0 2023-09-23T03:24:23.986 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.984+0000 7f03d1571640 1 -- 172.21.15.154:0/750040471 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2979553373 0 0) 0x7f03b4003520 con 0x7f03cc0cb6b0 2023-09-23T03:24:23.986 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.984+0000 7f03d1571640 1 -- 172.21.15.154:0/750040471 >> v2:172.21.15.190:3300/0 conn(0x7f03cc06a490 msgr2=0x7f03cc071080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:23.986 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.984+0000 7f03d1571640 1 --2- 172.21.15.154:0/750040471 >> v2:172.21.15.190:3300/0 conn(0x7f03cc06a490 0x7f03cc071080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:23.986 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.984+0000 7f03d1571640 1 -- 172.21.15.154:0/750040471 >> v1:172.21.15.190:6789/0 conn(0x7f03cc0cdaa0 legacy=0x7f03cc160950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:23.986 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.984+0000 7f03d1571640 1 -- 172.21.15.154:0/750040471 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03cc15d760 con 0x7f03cc0cb6b0 2023-09-23T03:24:23.986 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.984+0000 7f03d1571640 1 -- 172.21.15.154:0/750040471 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f03c0001670 con 0x7f03cc0cb6b0 2023-09-23T03:24:23.986 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.984+0000 7f03d1571640 1 -- 172.21.15.154:0/750040471 >> v1:172.21.15.154:6789/0 conn(0x7f03cc0cb6b0 legacy=0x7f03cc0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:23.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.984+0000 7f03d1571640 1 --2- 172.21.15.154:0/750040471 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03b4003970 0x7f03b4003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:23.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.984+0000 7f03d1571640 1 --2- 172.21.15.154:0/750040471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03b40047c0 0x7f03b4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:23.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.984+0000 7f03d1571640 1 --2- 172.21.15.154:0/750040471 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03b4005380 0x7f03b400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:23.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.984+0000 7f03d1571640 1 -- 172.21.15.154:0/750040471 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03cc15d480 con 0x7f03b4005380 2023-09-23T03:24:23.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.984+0000 7f03d1571640 1 -- 172.21.15.154:0/750040471 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b400c220 con 0x7f03b40047c0 2023-09-23T03:24:23.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.984+0000 7f03d1571640 1 -- 172.21.15.154:0/750040471 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b400c3e0 con 0x7f03b4003970 2023-09-23T03:24:23.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.984+0000 7f03d2d74640 1 --2- 172.21.15.154:0/750040471 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03b4005380 0x7f03b400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:23.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.985+0000 7f03d2573640 1 --2- 172.21.15.154:0/750040471 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03b4003970 0x7f03b4003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:23.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.985+0000 7f03d1d72640 1 --2- 172.21.15.154:0/750040471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03b40047c0 0x7f03b4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:23.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.985+0000 7f03d2d74640 1 -- 172.21.15.154:0/750040471 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03b4003970 msgr2=0x7f03b4003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:23.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.985+0000 7f03d2d74640 1 --2- 172.21.15.154:0/750040471 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03b4003970 0x7f03b4003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:23.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.985+0000 7f03d2d74640 1 -- 172.21.15.154:0/750040471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03b40047c0 msgr2=0x7f03b4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:23.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.985+0000 7f03d2d74640 1 --2- 172.21.15.154:0/750040471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03b40047c0 0x7f03b4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:23.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.985+0000 7f03d2d74640 1 -- 172.21.15.154:0/750040471 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b4003520 con 0x7f03b4005380 2023-09-23T03:24:23.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.985+0000 7f03d2573640 1 --2- 172.21.15.154:0/750040471 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03b4003970 0x7f03b4003d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:23.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.985+0000 7f03d1d72640 1 --2- 172.21.15.154:0/750040471 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03b40047c0 0x7f03b4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:23.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.986+0000 7f03d2d74640 1 -- 172.21.15.154:0/750040471 reap_dead start 2023-09-23T03:24:23.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.986+0000 7f03d2d74640 1 --2- 172.21.15.154:0/750040471 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03b4005380 0x7f03b400b980 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f03cc0c3030 tx=0x7f03b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba718a8bfef701f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:23.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.986+0000 7f03d1571640 1 -- 172.21.15.154:0/750040471 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c4032600 con 0x7f03b4005380 2023-09-23T03:24:23.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.986+0000 7f03d1571640 1 -- 172.21.15.154:0/750040471 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c402e050 con 0x7f03b4005380 2023-09-23T03:24:23.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.986+0000 7f03d1571640 1 -- 172.21.15.154:0/750040471 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c4032980 con 0x7f03b4005380 2023-09-23T03:24:23.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.987+0000 7f03d47d7640 1 -- 172.21.15.154:0/750040471 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03b4005380 msgr2=0x7f03b400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:23.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.987+0000 7f03d47d7640 1 --2- 172.21.15.154:0/750040471 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03b4005380 0x7f03b400b980 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f03cc0c3030 tx=0x7f03b40057f0 comp rx=0 tx=0).stop 2023-09-23T03:24:23.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.987+0000 7f03d47d7640 1 -- 172.21.15.154:0/750040471 shutdown_connections 2023-09-23T03:24:23.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.987+0000 7f03d47d7640 1 --2- 172.21.15.154:0/750040471 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03b4005380 0x7f03b400b980 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:23.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.987+0000 7f03d47d7640 1 -- 172.21.15.154:0/750040471 >> 172.21.15.154:0/750040471 conn(0x7f03cc0cc030 msgr2=0x7f03cc0cc410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:23.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.987+0000 7f03d47d7640 1 -- 172.21.15.154:0/750040471 shutdown_connections 2023-09-23T03:24:23.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.987+0000 7f03d47d7640 1 -- 172.21.15.154:0/750040471 wait complete. 2023-09-23T03:24:23.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.989+0000 7f03d47d7640 1 Processor -- start 2023-09-23T03:24:23.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.989+0000 7f03d47d7640 1 -- start start 2023-09-23T03:24:23.992 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.990+0000 7f03d47d7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03b4005380 0x7f03cc1518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:23.992 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.990+0000 7f03d47d7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03cc0ab8c0 0x7f03cc0b2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:23.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.990+0000 7f03d47d7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03cc15bc40 0x7f03cc15eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:23.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.990+0000 7f03d47d7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03cc0c4400 con 0x7f03b4005380 2023-09-23T03:24:23.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.990+0000 7f03d47d7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03cc0c46e0 con 0x7f03cc0ab8c0 2023-09-23T03:24:23.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.990+0000 7f03d47d7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03cc0c49c0 con 0x7f03cc15bc40 2023-09-23T03:24:23.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.990+0000 7f03d2573640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03b4005380 0x7f03cc1518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:23.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.990+0000 7f03d2573640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03b4005380 0x7f03cc1518d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46686/0 (socket says 172.21.15.154:46686) 2023-09-23T03:24:23.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.990+0000 7f03d2573640 1 -- 172.21.15.154:0/2085728171 learned_addr learned my addr 172.21.15.154:0/2085728171 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:23.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.991+0000 7f03d2573640 1 -- 172.21.15.154:0/2085728171 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03cc15bc40 msgr2=0x7f03cc15eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:23.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.991+0000 7f03d2573640 1 --2- 172.21.15.154:0/2085728171 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03cc15bc40 0x7f03cc15eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:23.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.991+0000 7f03d2573640 1 -- 172.21.15.154:0/2085728171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03cc0ab8c0 msgr2=0x7f03cc0b2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:23.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.991+0000 7f03d2573640 1 --2- 172.21.15.154:0/2085728171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03cc0ab8c0 0x7f03cc0b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:23.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.991+0000 7f03d2573640 1 -- 172.21.15.154:0/2085728171 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03cc0c4ca0 con 0x7f03b4005380 2023-09-23T03:24:23.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.991+0000 7f03d2573640 1 --2- 172.21.15.154:0/2085728171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03b4005380 0x7f03cc1518d0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f03bc00cdb0 tx=0x7f03bc00a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:23.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.991+0000 7f03caffd640 1 -- 172.21.15.154:0/2085728171 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03bc003970 con 0x7f03b4005380 2023-09-23T03:24:23.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.991+0000 7f03caffd640 1 -- 172.21.15.154:0/2085728171 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03bc003ad0 con 0x7f03b4005380 2023-09-23T03:24:23.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.991+0000 7f03caffd640 1 -- 172.21.15.154:0/2085728171 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03bc003e80 con 0x7f03b4005380 2023-09-23T03:24:23.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.991+0000 7f03d47d7640 1 -- 172.21.15.154:0/2085728171 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03cc0c4f80 con 0x7f03b4005380 2023-09-23T03:24:23.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.991+0000 7f03d47d7640 1 -- 172.21.15.154:0/2085728171 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03cc151e10 con 0x7f03b4005380 2023-09-23T03:24:23.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.993+0000 7f03d47d7640 1 -- 172.21.15.154:0/2085728171 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03cc0c51c0 con 0x7f03b4005380 2023-09-23T03:24:23.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.994+0000 7f03caffd640 1 -- 172.21.15.154:0/2085728171 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f03bc00d040 con 0x7f03b4005380 2023-09-23T03:24:23.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.995+0000 7f03caffd640 1 --2- 172.21.15.154:0/2085728171 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f03b806af30 0x7f03b806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:23.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.995+0000 7f03caffd640 1 -- 172.21.15.154:0/2085728171 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f03bc089ab0 con 0x7f03b4005380 2023-09-23T03:24:23.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.995+0000 7f03d1d72640 1 --2- 172.21.15.154:0/2085728171 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f03b806af30 0x7f03b806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:23.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:23.996+0000 7f03d1d72640 1 --2- 172.21.15.154:0/2085728171 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f03b806af30 0x7f03b806d3f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f03c00018e0 tx=0x7f03c0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:24.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.000+0000 7f03caffd640 1 -- 172.21.15.154:0/2085728171 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f03cc0c51c0 con 0x7f03b4005380 2023-09-23T03:24:24.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.194+0000 7f03d47d7640 1 -- 172.21.15.154:0/2085728171 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f03cc0c51c0 con 0x7f03b4005380 2023-09-23T03:24:24.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.195+0000 7f03caffd640 1 -- 172.21.15.154:0/2085728171 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f03cc0c51c0 con 0x7f03b4005380 2023-09-23T03:24:24.195 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:24:24.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.198+0000 7f03d47d7640 1 -- 172.21.15.154:0/2085728171 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f03b806af30 msgr2=0x7f03b806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:24.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.198+0000 7f03d47d7640 1 --2- 172.21.15.154:0/2085728171 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f03b806af30 0x7f03b806d3f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f03c00018e0 tx=0x7f03c0009040 comp rx=0 tx=0).stop 2023-09-23T03:24:24.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.198+0000 7f03d47d7640 1 -- 172.21.15.154:0/2085728171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03b4005380 msgr2=0x7f03cc1518d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:24.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.198+0000 7f03d47d7640 1 --2- 172.21.15.154:0/2085728171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03b4005380 0x7f03cc1518d0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f03bc00cdb0 tx=0x7f03bc00a5d0 comp rx=0 tx=0).stop 2023-09-23T03:24:24.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.198+0000 7f03d47d7640 1 -- 172.21.15.154:0/2085728171 shutdown_connections 2023-09-23T03:24:24.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.198+0000 7f03d47d7640 1 --2- 172.21.15.154:0/2085728171 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f03b806af30 0x7f03b806d3f0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:24.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.198+0000 7f03d47d7640 1 --2- 172.21.15.154:0/2085728171 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f03cc15bc40 0x7f03cc15eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:24.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.198+0000 7f03d47d7640 1 --2- 172.21.15.154:0/2085728171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f03cc0ab8c0 0x7f03cc0b2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:24.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.198+0000 7f03d47d7640 1 --2- 172.21.15.154:0/2085728171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03b4005380 0x7f03cc1518d0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:24.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.198+0000 7f03d47d7640 1 -- 172.21.15.154:0/2085728171 >> 172.21.15.154:0/2085728171 conn(0x7f03cc0cc030 msgr2=0x7f03cc0cd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:24.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.198+0000 7f03d47d7640 1 -- 172.21.15.154:0/2085728171 shutdown_connections 2023-09-23T03:24:24.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.198+0000 7f03d47d7640 1 -- 172.21.15.154:0/2085728171 wait complete. 2023-09-23T03:24:24.217 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:24:24.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.439+0000 7f05f4664640 1 Processor -- start 2023-09-23T03:24:24.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.439+0000 7f05f4664640 1 -- start start 2023-09-23T03:24:24.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.441+0000 7f05f4664640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f05ec0c9150 0x7f05ec15cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:24.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.441+0000 7f05f4664640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f05ec071290 0x7f05ec0ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:24.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.441+0000 7f05f4664640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05ec0c9d20 con 0x7f05ec0c8d70 2023-09-23T03:24:24.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.441+0000 7f05f4664640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05ec071850 con 0x7f05ec071290 2023-09-23T03:24:24.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.441+0000 7f05f4664640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05ec071b30 con 0x7f05ec0c9150 2023-09-23T03:24:24.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.441+0000 7f05f2c01640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f05ec071290 0x7f05ec0ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:24.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.441+0000 7f05f2c01640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f05ec071290 0x7f05ec0ca1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46702/0 (socket says 172.21.15.154:46702) 2023-09-23T03:24:24.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.441+0000 7f05f2c01640 1 -- 172.21.15.154:0/406228458 learned_addr learned my addr 172.21.15.154:0/406228458 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:24.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.441+0000 7f05f1bff640 1 --2- 172.21.15.154:0/406228458 >> v2:172.21.15.190:3300/0 conn(0x7f05ec0c9150 0x7f05ec15cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:24.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.442+0000 7f05f2c01640 1 -- 172.21.15.154:0/406228458 >> v2:172.21.15.190:3300/0 conn(0x7f05ec0c9150 msgr2=0x7f05ec15cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:24.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.442+0000 7f05f2c01640 1 --2- 172.21.15.154:0/406228458 >> v2:172.21.15.190:3300/0 conn(0x7f05ec0c9150 0x7f05ec15cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:24.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.442+0000 7f05f2c01640 1 -- 172.21.15.154:0/406228458 >> v1:172.21.15.190:6789/0 conn(0x7f05ec0c8d70 legacy=0x7f05ec15c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:24.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.442+0000 7f05f2c01640 1 -- 172.21.15.154:0/406228458 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ec071e10 con 0x7f05ec071290 2023-09-23T03:24:24.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.442+0000 7f05f1bff640 1 --2- 172.21.15.154:0/406228458 >> v2:172.21.15.190:3300/0 conn(0x7f05ec0c9150 0x7f05ec15cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:24.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.443+0000 7f05f2c01640 1 --2- 172.21.15.154:0/406228458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05ec071290 0x7f05ec0ca1b0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f05ec0b3ce0 tx=0x7f05e402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80448f77293dac83 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:24.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.443+0000 7f05f13fe640 1 -- 172.21.15.154:0/406228458 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e402ca80 con 0x7f05ec071290 2023-09-23T03:24:24.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.443+0000 7f05f13fe640 1 -- 172.21.15.154:0/406228458 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e402cbe0 con 0x7f05ec071290 2023-09-23T03:24:24.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.443+0000 7f05f13fe640 1 -- 172.21.15.154:0/406228458 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e40343f0 con 0x7f05ec071290 2023-09-23T03:24:24.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.444+0000 7f05f4664640 1 -- 172.21.15.154:0/406228458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05ec071290 msgr2=0x7f05ec0ca1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:24.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.444+0000 7f05f4664640 1 --2- 172.21.15.154:0/406228458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05ec071290 0x7f05ec0ca1b0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f05ec0b3ce0 tx=0x7f05e402bee0 comp rx=0 tx=0).stop 2023-09-23T03:24:24.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.444+0000 7f05f4664640 1 -- 172.21.15.154:0/406228458 shutdown_connections 2023-09-23T03:24:24.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.444+0000 7f05f4664640 1 --2- 172.21.15.154:0/406228458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05ec071290 0x7f05ec0ca1b0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:24.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.444+0000 7f05f4664640 1 --2- 172.21.15.154:0/406228458 >> v2:172.21.15.190:3300/0 conn(0x7f05ec0c9150 0x7f05ec15cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:24.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.444+0000 7f05f4664640 1 -- 172.21.15.154:0/406228458 >> 172.21.15.154:0/406228458 conn(0x7f05ec06c010 msgr2=0x7f05ec06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:24.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.444+0000 7f05f4664640 1 -- 172.21.15.154:0/406228458 shutdown_connections 2023-09-23T03:24:24.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.444+0000 7f05f4664640 1 -- 172.21.15.154:0/406228458 wait complete. 2023-09-23T03:24:24.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.446+0000 7f05f4664640 1 Processor -- start 2023-09-23T03:24:24.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.446+0000 7f05f4664640 1 -- start start 2023-09-23T03:24:24.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.447+0000 7f05f4664640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05ec071290 0x7f05ec0c1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:24.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.447+0000 7f05f4664640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05ec0c8d70 0x7f05ec14fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:24.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.447+0000 7f05f4664640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05ec0c9150 0x7f05ec0c1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:24.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.447+0000 7f05f4664640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec0c5a50 con 0x7f05ec0c8d70 2023-09-23T03:24:24.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.447+0000 7f05f4664640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec0c56b0 con 0x7f05ec071290 2023-09-23T03:24:24.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.447+0000 7f05f4664640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec0c2830 con 0x7f05ec0c9150 2023-09-23T03:24:24.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.447+0000 7f05f1bff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05ec0c8d70 0x7f05ec14fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:24.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.447+0000 7f05f1bff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05ec0c8d70 0x7f05ec14fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46710/0 (socket says 172.21.15.154:46710) 2023-09-23T03:24:24.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.447+0000 7f05f1bff640 1 -- 172.21.15.154:0/2343835865 learned_addr learned my addr 172.21.15.154:0/2343835865 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:24.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.448+0000 7f05f1bff640 1 -- 172.21.15.154:0/2343835865 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05ec0c9150 msgr2=0x7f05ec0c1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:24.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.448+0000 7f05f1bff640 1 --2- 172.21.15.154:0/2343835865 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05ec0c9150 0x7f05ec0c1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:24.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.448+0000 7f05f1bff640 1 -- 172.21.15.154:0/2343835865 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05ec071290 msgr2=0x7f05ec0c1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:24.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.448+0000 7f05f1bff640 1 --2- 172.21.15.154:0/2343835865 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05ec071290 0x7f05ec0c1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:24.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.448+0000 7f05f1bff640 1 -- 172.21.15.154:0/2343835865 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e4009b30 con 0x7f05ec0c8d70 2023-09-23T03:24:24.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.448+0000 7f05f1bff640 1 --2- 172.21.15.154:0/2343835865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05ec0c8d70 0x7f05ec14fb80 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00c7f0 tx=0x7f05dc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:24.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.448+0000 7f05e2ffd640 1 -- 172.21.15.154:0/2343835865 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05dc007820 con 0x7f05ec0c8d70 2023-09-23T03:24:24.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.448+0000 7f05e2ffd640 1 -- 172.21.15.154:0/2343835865 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05dc007980 con 0x7f05ec0c8d70 2023-09-23T03:24:24.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.448+0000 7f05e2ffd640 1 -- 172.21.15.154:0/2343835865 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05dc007cf0 con 0x7f05ec0c8d70 2023-09-23T03:24:24.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.448+0000 7f05f4664640 1 -- 172.21.15.154:0/2343835865 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05ec0c2b10 con 0x7f05ec0c8d70 2023-09-23T03:24:24.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.448+0000 7f05f4664640 1 -- 172.21.15.154:0/2343835865 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05ec0c2df0 con 0x7f05ec0c8d70 2023-09-23T03:24:24.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.449+0000 7f05f4664640 1 -- 172.21.15.154:0/2343835865 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05ec04e920 con 0x7f05ec0c8d70 2023-09-23T03:24:24.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.450+0000 7f05e2ffd640 1 -- 172.21.15.154:0/2343835865 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f05dc0113f0 con 0x7f05ec0c8d70 2023-09-23T03:24:24.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.451+0000 7f05e2ffd640 1 --2- 172.21.15.154:0/2343835865 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f05d006af30 0x7f05d006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:24.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.451+0000 7f05e2ffd640 1 -- 172.21.15.154:0/2343835865 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f05dc089db0 con 0x7f05ec0c8d70 2023-09-23T03:24:24.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.452+0000 7f05f2400640 1 --2- 172.21.15.154:0/2343835865 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f05d006af30 0x7f05d006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:24.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.452+0000 7f05f2400640 1 --2- 172.21.15.154:0/2343835865 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f05d006af30 0x7f05d006d3f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f05e8003cc0 tx=0x7f05e80079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:24.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.456+0000 7f05e2ffd640 1 -- 172.21.15.154:0/2343835865 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f05dc08e070 con 0x7f05ec0c8d70 2023-09-23T03:24:24.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.644+0000 7f05f4664640 1 -- 172.21.15.154:0/2343835865 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f05ec0720c0 con 0x7f05ec0c8d70 2023-09-23T03:24:24.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.645+0000 7f05e2ffd640 1 -- 172.21.15.154:0/2343835865 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7f05dc01f020 con 0x7f05ec0c8d70 2023-09-23T03:24:24.646 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:24:24.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.648+0000 7f05f4664640 1 -- 172.21.15.154:0/2343835865 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f05d006af30 msgr2=0x7f05d006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:24.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.648+0000 7f05f4664640 1 --2- 172.21.15.154:0/2343835865 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f05d006af30 0x7f05d006d3f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f05e8003cc0 tx=0x7f05e80079b0 comp rx=0 tx=0).stop 2023-09-23T03:24:24.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.648+0000 7f05f4664640 1 -- 172.21.15.154:0/2343835865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05ec0c8d70 msgr2=0x7f05ec14fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:24.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.648+0000 7f05f4664640 1 --2- 172.21.15.154:0/2343835865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05ec0c8d70 0x7f05ec14fb80 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f05dc00c7f0 tx=0x7f05dc00ccb0 comp rx=0 tx=0).stop 2023-09-23T03:24:24.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.648+0000 7f05f4664640 1 -- 172.21.15.154:0/2343835865 shutdown_connections 2023-09-23T03:24:24.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.648+0000 7f05f4664640 1 --2- 172.21.15.154:0/2343835865 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f05d006af30 0x7f05d006d3f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:24.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.648+0000 7f05f4664640 1 --2- 172.21.15.154:0/2343835865 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05ec0c9150 0x7f05ec0c1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:24.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.648+0000 7f05f4664640 1 --2- 172.21.15.154:0/2343835865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05ec0c8d70 0x7f05ec14fb80 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:24.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.648+0000 7f05f4664640 1 --2- 172.21.15.154:0/2343835865 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05ec071290 0x7f05ec0c1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:24.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.648+0000 7f05f4664640 1 -- 172.21.15.154:0/2343835865 >> 172.21.15.154:0/2343835865 conn(0x7f05ec06c010 msgr2=0x7f05ec0caab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:24.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.648+0000 7f05f4664640 1 -- 172.21.15.154:0/2343835865 shutdown_connections 2023-09-23T03:24:24.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.648+0000 7f05f4664640 1 -- 172.21.15.154:0/2343835865 wait complete. 2023-09-23T03:24:24.667 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:24:24.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.882+0000 7feedd389640 1 Processor -- start 2023-09-23T03:24:24.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.882+0000 7feedd389640 1 -- start start 2023-09-23T03:24:24.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.883+0000 7feedd389640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feed815ad00 0x7feed80cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:24.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.884+0000 7feedd389640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feed80b21b0 con 0x7feed80b3960 2023-09-23T03:24:24.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.884+0000 7feedd389640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feed80b2470 con 0x7feed80cbc80 2023-09-23T03:24:24.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.884+0000 7feedd389640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed80b2750 con 0x7feed815ad00 2023-09-23T03:24:24.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.884+0000 7feece7fc640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7feed80b3960 0x7feed80cb570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:38986/0 (socket says 172.21.15.154:38986) 2023-09-23T03:24:24.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.884+0000 7feece7fc640 1 -- 172.21.15.154:0/3931380951 learned_addr learned my addr 172.21.15.154:0/3931380951 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:24:24.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.884+0000 7feed6ffd640 1 --2- 172.21.15.154:0/3931380951 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feed815ad00 0x7feed80cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:24.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.884+0000 7feed67fc640 1 -- 172.21.15.154:0/3931380951 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4178600663 0 0) 0x7feed80b21b0 con 0x7feed80b3960 2023-09-23T03:24:24.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.884+0000 7feed67fc640 1 -- 172.21.15.154:0/3931380951 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feec4003520 con 0x7feed80b3960 2023-09-23T03:24:24.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.885+0000 7feed67fc640 1 -- 172.21.15.154:0/3931380951 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2707173710 0 0) 0x7feec4003520 con 0x7feed80b3960 2023-09-23T03:24:24.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.885+0000 7feed67fc640 1 -- 172.21.15.154:0/3931380951 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feed815ad00 msgr2=0x7feed80cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:24.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.885+0000 7feed67fc640 1 --2- 172.21.15.154:0/3931380951 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feed815ad00 0x7feed80cd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:24.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.885+0000 7feed67fc640 1 -- 172.21.15.154:0/3931380951 >> v1:172.21.15.190:6789/0 conn(0x7feed80cbc80 legacy=0x7feed815e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:24.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.885+0000 7feed67fc640 1 -- 172.21.15.154:0/3931380951 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed80b2a30 con 0x7feed80b3960 2023-09-23T03:24:24.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.885+0000 7feed67fc640 1 -- 172.21.15.154:0/3931380951 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7feec0001670 con 0x7feed80b3960 2023-09-23T03:24:24.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.885+0000 7feed67fc640 1 -- 172.21.15.154:0/3931380951 >> v1:172.21.15.154:6789/0 conn(0x7feed80b3960 legacy=0x7feed80cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:24.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.885+0000 7feed67fc640 1 --2- 172.21.15.154:0/3931380951 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feec40039a0 0x7feec4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:24.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.885+0000 7feed67fc640 1 --2- 172.21.15.154:0/3931380951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feec40047c0 0x7feec4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:24.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.885+0000 7feed67fc640 1 --2- 172.21.15.154:0/3931380951 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feed815ad00 0x7feec4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:24.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.885+0000 7feed67fc640 1 -- 172.21.15.154:0/3931380951 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed80b2750 con 0x7feec40039a0 2023-09-23T03:24:24.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.885+0000 7feed67fc640 1 -- 172.21.15.154:0/3931380951 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec400ba30 con 0x7feec40047c0 2023-09-23T03:24:24.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.885+0000 7feed67fc640 1 -- 172.21.15.154:0/3931380951 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec400bbf0 con 0x7feed815ad00 2023-09-23T03:24:24.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.885+0000 7feed6ffd640 1 --2- 172.21.15.154:0/3931380951 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feec40039a0 0x7feec4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:24.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.886+0000 7feece7fc640 1 --2- 172.21.15.154:0/3931380951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feec40047c0 0x7feec4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:24.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.886+0000 7feed77fe640 1 --2- 172.21.15.154:0/3931380951 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feed815ad00 0x7feec4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:24.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.886+0000 7feed6ffd640 1 -- 172.21.15.154:0/3931380951 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feed815ad00 msgr2=0x7feec4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:24.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.886+0000 7feed6ffd640 1 --2- 172.21.15.154:0/3931380951 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feed815ad00 0x7feec4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:24.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.886+0000 7feed6ffd640 1 -- 172.21.15.154:0/3931380951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feec40047c0 msgr2=0x7feec4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:24.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.886+0000 7feed6ffd640 1 --2- 172.21.15.154:0/3931380951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feec40047c0 0x7feec4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:24.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.886+0000 7feed6ffd640 1 -- 172.21.15.154:0/3931380951 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec4003520 con 0x7feec40039a0 2023-09-23T03:24:24.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.886+0000 7feece7fc640 1 --2- 172.21.15.154:0/3931380951 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feec40047c0 0x7feec4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:24.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.886+0000 7feed77fe640 1 --2- 172.21.15.154:0/3931380951 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feed815ad00 0x7feec4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:24.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.887+0000 7feed6ffd640 1 --2- 172.21.15.154:0/3931380951 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feec40039a0 0x7feec4003dc0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7feec800e140 tx=0x7feec40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a6e06b9117a04d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:24.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.887+0000 7feed67fc640 1 -- 172.21.15.154:0/3931380951 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec802cbf0 con 0x7feec40039a0 2023-09-23T03:24:24.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.887+0000 7feed67fc640 1 -- 172.21.15.154:0/3931380951 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec800f040 con 0x7feec40039a0 2023-09-23T03:24:24.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.887+0000 7feed67fc640 1 -- 172.21.15.154:0/3931380951 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec8034440 con 0x7feec40039a0 2023-09-23T03:24:24.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.888+0000 7feedd389640 1 -- 172.21.15.154:0/3931380951 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feec40039a0 msgr2=0x7feec4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:24.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.888+0000 7feedd389640 1 --2- 172.21.15.154:0/3931380951 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feec40039a0 0x7feec4003dc0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7feec800e140 tx=0x7feec40057f0 comp rx=0 tx=0).stop 2023-09-23T03:24:24.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.888+0000 7feed77fe640 1 -- 172.21.15.154:0/3931380951 reap_dead start 2023-09-23T03:24:24.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.888+0000 7feedd389640 1 -- 172.21.15.154:0/3931380951 shutdown_connections 2023-09-23T03:24:24.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.888+0000 7feedd389640 1 -- 172.21.15.154:0/3931380951 >> 172.21.15.154:0/3931380951 conn(0x7feed806fc40 msgr2=0x7feed8070020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:24.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.888+0000 7feedd389640 1 -- 172.21.15.154:0/3931380951 shutdown_connections 2023-09-23T03:24:24.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.888+0000 7feedd389640 1 -- 172.21.15.154:0/3931380951 wait complete. 2023-09-23T03:24:24.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.890+0000 7feedd389640 1 Processor -- start 2023-09-23T03:24:24.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.890+0000 7feedd389640 1 -- start start 2023-09-23T03:24:24.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.891+0000 7feedd389640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feec40039a0 0x7feed8158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:24.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.891+0000 7feedd389640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feed81f2860 0x7feed8158290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:24.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.891+0000 7feedd389640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feed815b490 0x7feed815b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:24.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.891+0000 7feedd389640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed815be50 con 0x7feed81f2860 2023-09-23T03:24:24.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.891+0000 7feedd389640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed81590b0 con 0x7feec40039a0 2023-09-23T03:24:24.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.891+0000 7feedd389640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed8159390 con 0x7feed815b490 2023-09-23T03:24:24.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.891+0000 7feece7fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feed81f2860 0x7feed8158290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:24.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.891+0000 7feece7fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feed81f2860 0x7feed8158290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46734/0 (socket says 172.21.15.154:46734) 2023-09-23T03:24:24.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.891+0000 7feece7fc640 1 -- 172.21.15.154:0/2425260339 learned_addr learned my addr 172.21.15.154:0/2425260339 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:24.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.892+0000 7feed6ffd640 1 --2- 172.21.15.154:0/2425260339 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feec40039a0 0x7feed8158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:24.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.892+0000 7feed77fe640 1 --2- 172.21.15.154:0/2425260339 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feed815b490 0x7feed815b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:24.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.892+0000 7feece7fc640 1 -- 172.21.15.154:0/2425260339 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feed815b490 msgr2=0x7feed815b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:24.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.892+0000 7feece7fc640 1 --2- 172.21.15.154:0/2425260339 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feed815b490 0x7feed815b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:24.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.892+0000 7feece7fc640 1 -- 172.21.15.154:0/2425260339 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feec40039a0 msgr2=0x7feed8158ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:24.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.892+0000 7feece7fc640 1 --2- 172.21.15.154:0/2425260339 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feec40039a0 0x7feed8158ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:24.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.892+0000 7feece7fc640 1 -- 172.21.15.154:0/2425260339 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed8159670 con 0x7feed81f2860 2023-09-23T03:24:24.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.892+0000 7feece7fc640 1 --2- 172.21.15.154:0/2425260339 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feed81f2860 0x7feed8158290 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7feec000d860 tx=0x7feec000dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:24.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.892+0000 7feecffff640 1 -- 172.21.15.154:0/2425260339 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec00074e0 con 0x7feed81f2860 2023-09-23T03:24:24.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.892+0000 7feecffff640 1 -- 172.21.15.154:0/2425260339 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec0007640 con 0x7feed81f2860 2023-09-23T03:24:24.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.892+0000 7feecffff640 1 -- 172.21.15.154:0/2425260339 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec00049e0 con 0x7feed81f2860 2023-09-23T03:24:24.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.892+0000 7feedd389640 1 -- 172.21.15.154:0/2425260339 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed8159950 con 0x7feed81f2860 2023-09-23T03:24:24.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.893+0000 7feedd389640 1 -- 172.21.15.154:0/2425260339 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed80c6c30 con 0x7feed81f2860 2023-09-23T03:24:24.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.894+0000 7feedd389640 1 -- 172.21.15.154:0/2425260339 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed804e920 con 0x7feed81f2860 2023-09-23T03:24:24.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.895+0000 7feecffff640 1 -- 172.21.15.154:0/2425260339 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7feec0004b40 con 0x7feed81f2860 2023-09-23T03:24:24.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.896+0000 7feecffff640 1 --2- 172.21.15.154:0/2425260339 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7feeb806af30 0x7feeb806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:24.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.896+0000 7feecffff640 1 -- 172.21.15.154:0/2425260339 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6433+0+0 (secure 0 0 0) 0x7feec00894c0 con 0x7feed81f2860 2023-09-23T03:24:24.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.896+0000 7feed6ffd640 1 --2- 172.21.15.154:0/2425260339 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7feeb806af30 0x7feeb806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:24.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.897+0000 7feed6ffd640 1 --2- 172.21.15.154:0/2425260339 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7feeb806af30 0x7feeb806d3f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7feec8036000 tx=0x7feec802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:24.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:24.900+0000 7feecffff640 1 -- 172.21.15.154:0/2425260339 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7feec0054180 con 0x7feed81f2860 2023-09-23T03:24:25.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.087+0000 7feedd389640 1 -- 172.21.15.154:0/2425260339 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7feed80ce4b0 con 0x7feed81f2860 2023-09-23T03:24:25.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.087+0000 7feecffff640 1 -- 172.21.15.154:0/2425260339 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7feec00577a0 con 0x7feed81f2860 2023-09-23T03:24:25.088 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:24:25.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.090+0000 7feedd389640 1 -- 172.21.15.154:0/2425260339 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7feeb806af30 msgr2=0x7feeb806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:25.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.090+0000 7feedd389640 1 --2- 172.21.15.154:0/2425260339 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7feeb806af30 0x7feeb806d3f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7feec8036000 tx=0x7feec802e000 comp rx=0 tx=0).stop 2023-09-23T03:24:25.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.090+0000 7feedd389640 1 -- 172.21.15.154:0/2425260339 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feed81f2860 msgr2=0x7feed8158290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:25.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.090+0000 7feedd389640 1 --2- 172.21.15.154:0/2425260339 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feed81f2860 0x7feed8158290 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7feec000d860 tx=0x7feec000dd20 comp rx=0 tx=0).stop 2023-09-23T03:24:25.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.090+0000 7feedd389640 1 -- 172.21.15.154:0/2425260339 shutdown_connections 2023-09-23T03:24:25.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.090+0000 7feedd389640 1 --2- 172.21.15.154:0/2425260339 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7feeb806af30 0x7feeb806d3f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:25.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.090+0000 7feedd389640 1 --2- 172.21.15.154:0/2425260339 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feed815b490 0x7feed815b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:25.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.090+0000 7feedd389640 1 --2- 172.21.15.154:0/2425260339 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feed81f2860 0x7feed8158290 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:25.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.090+0000 7feedd389640 1 --2- 172.21.15.154:0/2425260339 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feec40039a0 0x7feed8158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:25.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.090+0000 7feedd389640 1 -- 172.21.15.154:0/2425260339 >> 172.21.15.154:0/2425260339 conn(0x7feed806fc40 msgr2=0x7feed806c880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:25.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.090+0000 7feedd389640 1 -- 172.21.15.154:0/2425260339 shutdown_connections 2023-09-23T03:24:25.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.090+0000 7feedd389640 1 -- 172.21.15.154:0/2425260339 wait complete. 2023-09-23T03:24:25.110 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-09-23T03:24:25.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.326+0000 7efe79797640 1 Processor -- start 2023-09-23T03:24:25.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.326+0000 7efe79797640 1 -- start start 2023-09-23T03:24:25.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.328+0000 7efe79797640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe7415c9c0 0x7efe740d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:25.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.328+0000 7efe79797640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efe7415cda0 0x7efe740d2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:25.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.328+0000 7efe79797640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe740cd450 con 0x7efe740d2b60 2023-09-23T03:24:25.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.328+0000 7efe79797640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe740d0df0 con 0x7efe7415cda0 2023-09-23T03:24:25.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.328+0000 7efe79797640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe740d10d0 con 0x7efe7415c9c0 2023-09-23T03:24:25.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.328+0000 7efe737fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7efe740d2b60 0x7efe740cf2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:38990/0 (socket says 172.21.15.154:38990) 2023-09-23T03:24:25.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.328+0000 7efe737fe640 1 -- 172.21.15.154:0/3636624942 learned_addr learned my addr 172.21.15.154:0/3636624942 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:24:25.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.328+0000 7efe6bfff640 1 --2- 172.21.15.154:0/3636624942 >> v2:172.21.15.154:3300/0 conn(0x7efe7415cda0 0x7efe740d2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:25.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.329+0000 7efe72ffd640 1 --2- 172.21.15.154:0/3636624942 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe7415c9c0 0x7efe740d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:25.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.329+0000 7efe6bfff640 1 -- 172.21.15.154:0/3636624942 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe7415c9c0 msgr2=0x7efe740d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:25.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.329+0000 7efe6bfff640 1 --2- 172.21.15.154:0/3636624942 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe7415c9c0 0x7efe740d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:25.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.329+0000 7efe6bfff640 1 -- 172.21.15.154:0/3636624942 >> v1:172.21.15.154:6789/0 conn(0x7efe740d2b60 legacy=0x7efe740cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:25.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.329+0000 7efe6bfff640 1 -- 172.21.15.154:0/3636624942 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe740d13b0 con 0x7efe7415cda0 2023-09-23T03:24:25.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.329+0000 7efe727fc640 1 -- 172.21.15.154:0/3636624942 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1971884146 0 0) 0x7efe740cd450 con 0x7efe740d2b60 2023-09-23T03:24:25.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.330+0000 7efe6bfff640 1 --2- 172.21.15.154:0/3636624942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe7415cda0 0x7efe740d2620 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7efe5c0097a0 tx=0x7efe5c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32abe3ee9b53a5b7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:25.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.330+0000 7efe727fc640 1 -- 172.21.15.154:0/3636624942 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5c031830 con 0x7efe7415cda0 2023-09-23T03:24:25.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.330+0000 7efe727fc640 1 -- 172.21.15.154:0/3636624942 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe5c031990 con 0x7efe7415cda0 2023-09-23T03:24:25.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.330+0000 7efe727fc640 1 -- 172.21.15.154:0/3636624942 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5c031d20 con 0x7efe7415cda0 2023-09-23T03:24:25.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.331+0000 7efe79797640 1 -- 172.21.15.154:0/3636624942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe7415cda0 msgr2=0x7efe740d2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:25.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.331+0000 7efe79797640 1 --2- 172.21.15.154:0/3636624942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe7415cda0 0x7efe740d2620 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7efe5c0097a0 tx=0x7efe5c02b8d0 comp rx=0 tx=0).stop 2023-09-23T03:24:25.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.331+0000 7efe79797640 1 -- 172.21.15.154:0/3636624942 shutdown_connections 2023-09-23T03:24:25.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.331+0000 7efe79797640 1 --2- 172.21.15.154:0/3636624942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe7415cda0 0x7efe740d2620 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:25.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.331+0000 7efe79797640 1 --2- 172.21.15.154:0/3636624942 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe7415c9c0 0x7efe740d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:25.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.331+0000 7efe79797640 1 -- 172.21.15.154:0/3636624942 >> 172.21.15.154:0/3636624942 conn(0x7efe7409bc20 msgr2=0x7efe74089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:25.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.331+0000 7efe79797640 1 -- 172.21.15.154:0/3636624942 shutdown_connections 2023-09-23T03:24:25.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.331+0000 7efe79797640 1 -- 172.21.15.154:0/3636624942 wait complete. 2023-09-23T03:24:25.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.333+0000 7efe79797640 1 Processor -- start 2023-09-23T03:24:25.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.333+0000 7efe79797640 1 -- start start 2023-09-23T03:24:25.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.334+0000 7efe79797640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe740d2b60 0x7efe740b4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:25.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.334+0000 7efe79797640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe7415c9c0 0x7efe7415f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:25.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.334+0000 7efe79797640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe7415cda0 0x7efe7415f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:25.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.334+0000 7efe79797640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe741497b0 con 0x7efe7415cda0 2023-09-23T03:24:25.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.334+0000 7efe79797640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe740cc260 con 0x7efe740d2b60 2023-09-23T03:24:25.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.334+0000 7efe79797640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe740cc540 con 0x7efe7415c9c0 2023-09-23T03:24:25.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.334+0000 7efe737fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe7415cda0 0x7efe7415f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:25.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.334+0000 7efe737fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe7415cda0 0x7efe7415f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46752/0 (socket says 172.21.15.154:46752) 2023-09-23T03:24:25.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.334+0000 7efe737fe640 1 -- 172.21.15.154:0/1960048852 learned_addr learned my addr 172.21.15.154:0/1960048852 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:25.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.335+0000 7efe72ffd640 1 --2- 172.21.15.154:0/1960048852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe740d2b60 0x7efe740b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:25.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.335+0000 7efe6bfff640 1 --2- 172.21.15.154:0/1960048852 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe7415c9c0 0x7efe7415f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:25.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.335+0000 7efe737fe640 1 -- 172.21.15.154:0/1960048852 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe7415c9c0 msgr2=0x7efe7415f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:25.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.335+0000 7efe737fe640 1 --2- 172.21.15.154:0/1960048852 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe7415c9c0 0x7efe7415f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:25.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.335+0000 7efe737fe640 1 -- 172.21.15.154:0/1960048852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe740d2b60 msgr2=0x7efe740b4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:25.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.335+0000 7efe737fe640 1 --2- 172.21.15.154:0/1960048852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe740d2b60 0x7efe740b4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:25.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.335+0000 7efe737fe640 1 -- 172.21.15.154:0/1960048852 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe5c009400 con 0x7efe7415cda0 2023-09-23T03:24:25.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.335+0000 7efe72ffd640 1 --2- 172.21.15.154:0/1960048852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe740d2b60 0x7efe740b4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:25.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.335+0000 7efe737fe640 1 --2- 172.21.15.154:0/1960048852 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe7415cda0 0x7efe7415f9a0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7efe6c00c700 tx=0x7efe6c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:25.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.335+0000 7efe6b7fe640 1 -- 172.21.15.154:0/1960048852 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6c00ce10 con 0x7efe7415cda0 2023-09-23T03:24:25.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.335+0000 7efe6b7fe640 1 -- 172.21.15.154:0/1960048852 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe6c00a640 con 0x7efe7415cda0 2023-09-23T03:24:25.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.335+0000 7efe6b7fe640 1 -- 172.21.15.154:0/1960048852 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6c00aa70 con 0x7efe7415cda0 2023-09-23T03:24:25.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.335+0000 7efe79797640 1 -- 172.21.15.154:0/1960048852 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe740cc820 con 0x7efe7415cda0 2023-09-23T03:24:25.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.336+0000 7efe79797640 1 -- 172.21.15.154:0/1960048852 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe740b4790 con 0x7efe7415cda0 2023-09-23T03:24:25.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.337+0000 7efe79797640 1 -- 172.21.15.154:0/1960048852 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe74149920 con 0x7efe7415cda0 2023-09-23T03:24:25.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.338+0000 7efe6b7fe640 1 -- 172.21.15.154:0/1960048852 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7efe6c00abd0 con 0x7efe7415cda0 2023-09-23T03:24:25.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.338+0000 7efe6b7fe640 1 --2- 172.21.15.154:0/1960048852 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efe5406af30 0x7efe5406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:25.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.339+0000 7efe6b7fe640 1 -- 172.21.15.154:0/1960048852 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6433+0+0 (secure 0 0 0) 0x7efe6c088ee0 con 0x7efe7415cda0 2023-09-23T03:24:25.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.339+0000 7efe72ffd640 1 --2- 172.21.15.154:0/1960048852 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efe5406af30 0x7efe5406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:25.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.340+0000 7efe72ffd640 1 --2- 172.21.15.154:0/1960048852 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efe5406af30 0x7efe5406d3f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7efe740ceda0 tx=0x7efe64008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:25.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.343+0000 7efe6b7fe640 1 -- 172.21.15.154:0/1960048852 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7efe6c053cd0 con 0x7efe7415cda0 2023-09-23T03:24:25.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.535+0000 7efe79797640 1 -- 172.21.15.154:0/1960048852 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7efe7404e920 con 0x7efe7415cda0 2023-09-23T03:24:25.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.536+0000 7efe6b7fe640 1 -- 172.21.15.154:0/1960048852 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v86) v1 ==== 106+0+11 (secure 0 0 0) 0x7efe6c018070 con 0x7efe7415cda0 2023-09-23T03:24:25.536 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:24:25.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.538+0000 7efe79797640 1 -- 172.21.15.154:0/1960048852 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efe5406af30 msgr2=0x7efe5406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:25.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.538+0000 7efe79797640 1 --2- 172.21.15.154:0/1960048852 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efe5406af30 0x7efe5406d3f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7efe740ceda0 tx=0x7efe64008040 comp rx=0 tx=0).stop 2023-09-23T03:24:25.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.538+0000 7efe79797640 1 -- 172.21.15.154:0/1960048852 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe7415cda0 msgr2=0x7efe7415f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:25.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.538+0000 7efe79797640 1 --2- 172.21.15.154:0/1960048852 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe7415cda0 0x7efe7415f9a0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7efe6c00c700 tx=0x7efe6c00cbc0 comp rx=0 tx=0).stop 2023-09-23T03:24:25.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.538+0000 7efe79797640 1 -- 172.21.15.154:0/1960048852 shutdown_connections 2023-09-23T03:24:25.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.538+0000 7efe79797640 1 --2- 172.21.15.154:0/1960048852 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efe5406af30 0x7efe5406d3f0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:25.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.538+0000 7efe79797640 1 --2- 172.21.15.154:0/1960048852 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe7415cda0 0x7efe7415f9a0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:25.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.538+0000 7efe79797640 1 --2- 172.21.15.154:0/1960048852 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe7415c9c0 0x7efe7415f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:25.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.538+0000 7efe79797640 1 --2- 172.21.15.154:0/1960048852 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe740d2b60 0x7efe740b4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:25.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.538+0000 7efe79797640 1 -- 172.21.15.154:0/1960048852 >> 172.21.15.154:0/1960048852 conn(0x7efe7409bc20 msgr2=0x7efe740b4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:25.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.538+0000 7efe79797640 1 -- 172.21.15.154:0/1960048852 shutdown_connections 2023-09-23T03:24:25.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.538+0000 7efe79797640 1 -- 172.21.15.154:0/1960048852 wait complete. 2023-09-23T03:24:25.557 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-23T03:24:25.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.780+0000 7ffa02f97640 1 Processor -- start 2023-09-23T03:24:25.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.780+0000 7ffa02f97640 1 -- start start 2023-09-23T03:24:25.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.782+0000 7ffa02f97640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff9fc0cdfa0 0x7ff9fc0659f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:25.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.782+0000 7ffa02f97640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff9fc15cf80 0x7ff9fc06c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:25.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.782+0000 7ffa02f97640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9fc065f30 con 0x7ff9fc0b1400 2023-09-23T03:24:25.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.782+0000 7ffa02f97640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9fc15d550 con 0x7ff9fc0cdfa0 2023-09-23T03:24:25.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.782+0000 7ffa02f97640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9fc15d830 con 0x7ff9fc15cf80 2023-09-23T03:24:25.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.782+0000 7ffa00d33640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff9fc0cdfa0 0x7ff9fc0659f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:25.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.782+0000 7ffa00d33640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff9fc0cdfa0 0x7ff9fc0659f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46756/0 (socket says 172.21.15.154:46756) 2023-09-23T03:24:25.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.782+0000 7ffa00d33640 1 -- 172.21.15.154:0/3073685932 learned_addr learned my addr 172.21.15.154:0/3073685932 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:25.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.782+0000 7ffa00d33640 1 -- 172.21.15.154:0/3073685932 >> v2:172.21.15.190:3300/0 conn(0x7ff9fc15cf80 msgr2=0x7ff9fc06c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:25.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.782+0000 7ffa01534640 1 --2- 172.21.15.154:0/3073685932 >> v2:172.21.15.190:3300/0 conn(0x7ff9fc15cf80 0x7ff9fc06c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:25.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.783+0000 7ffa00d33640 1 --2- 172.21.15.154:0/3073685932 >> v2:172.21.15.190:3300/0 conn(0x7ff9fc15cf80 0x7ff9fc06c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:25.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.783+0000 7ffa00d33640 1 -- 172.21.15.154:0/3073685932 >> v1:172.21.15.190:6789/0 conn(0x7ff9fc0b1400 legacy=0x7ff9fc0ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:25.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.783+0000 7ffa00d33640 1 -- 172.21.15.154:0/3073685932 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9fc15db10 con 0x7ff9fc0cdfa0 2023-09-23T03:24:25.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.783+0000 7ffa01534640 1 --2- 172.21.15.154:0/3073685932 >> v2:172.21.15.190:3300/0 conn(0x7ff9fc15cf80 0x7ff9fc06c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:25.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.784+0000 7ffa00d33640 1 --2- 172.21.15.154:0/3073685932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9fc0cdfa0 0x7ff9fc0659f0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc15e310 tx=0x7ff9f002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4c18f14ac2e9635 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:25.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.785+0000 7ff9fb7fe640 1 -- 172.21.15.154:0/3073685932 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0031750 con 0x7ff9fc0cdfa0 2023-09-23T03:24:25.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.785+0000 7ff9fb7fe640 1 -- 172.21.15.154:0/3073685932 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f00318b0 con 0x7ff9fc0cdfa0 2023-09-23T03:24:25.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.785+0000 7ff9fb7fe640 1 -- 172.21.15.154:0/3073685932 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0031c40 con 0x7ff9fc0cdfa0 2023-09-23T03:24:25.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.785+0000 7ffa02f97640 1 -- 172.21.15.154:0/3073685932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9fc0cdfa0 msgr2=0x7ff9fc0659f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:25.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.785+0000 7ffa02f97640 1 --2- 172.21.15.154:0/3073685932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9fc0cdfa0 0x7ff9fc0659f0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc15e310 tx=0x7ff9f002b7f0 comp rx=0 tx=0).stop 2023-09-23T03:24:25.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.786+0000 7ffa02f97640 1 -- 172.21.15.154:0/3073685932 shutdown_connections 2023-09-23T03:24:25.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.786+0000 7ffa02f97640 1 --2- 172.21.15.154:0/3073685932 >> v2:172.21.15.190:3300/0 conn(0x7ff9fc15cf80 0x7ff9fc06c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:25.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.786+0000 7ffa02f97640 1 --2- 172.21.15.154:0/3073685932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9fc0cdfa0 0x7ff9fc0659f0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:25.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.786+0000 7ffa02f97640 1 -- 172.21.15.154:0/3073685932 >> 172.21.15.154:0/3073685932 conn(0x7ff9fc0b08e0 msgr2=0x7ff9fc0b0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:25.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.786+0000 7ffa02f97640 1 -- 172.21.15.154:0/3073685932 shutdown_connections 2023-09-23T03:24:25.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.786+0000 7ffa02f97640 1 -- 172.21.15.154:0/3073685932 wait complete. 2023-09-23T03:24:25.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.787+0000 7ffa02f97640 1 Processor -- start 2023-09-23T03:24:25.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.788+0000 7ffa02f97640 1 -- start start 2023-09-23T03:24:25.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.789+0000 7ffa02f97640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9fc0b1400 0x7ff9fc1e7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:25.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.789+0000 7ffa02f97640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9fc0cdfa0 0x7ff9fc0c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:25.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.789+0000 7ffa02f97640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9fc15cf80 0x7ff9fc154d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:25.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.789+0000 7ffa02f97640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc1e7670 con 0x7ff9fc15cf80 2023-09-23T03:24:25.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.789+0000 7ffa02f97640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc1561f0 con 0x7ff9fc0b1400 2023-09-23T03:24:25.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.789+0000 7ffa02f97640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc160dd0 con 0x7ff9fc0cdfa0 2023-09-23T03:24:25.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.789+0000 7ffa01534640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9fc15cf80 0x7ff9fc154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:25.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.789+0000 7ffa01534640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9fc15cf80 0x7ff9fc154d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46762/0 (socket says 172.21.15.154:46762) 2023-09-23T03:24:25.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.789+0000 7ffa01534640 1 -- 172.21.15.154:0/948262336 learned_addr learned my addr 172.21.15.154:0/948262336 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:25.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.789+0000 7ffa00d33640 1 --2- 172.21.15.154:0/948262336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9fc0b1400 0x7ff9fc1e7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:25.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.789+0000 7ff9fbfff640 1 --2- 172.21.15.154:0/948262336 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9fc0cdfa0 0x7ff9fc0c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:25.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.789+0000 7ffa01534640 1 -- 172.21.15.154:0/948262336 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9fc0cdfa0 msgr2=0x7ff9fc0c7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:25.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.789+0000 7ffa01534640 1 --2- 172.21.15.154:0/948262336 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9fc0cdfa0 0x7ff9fc0c7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:25.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.789+0000 7ffa01534640 1 -- 172.21.15.154:0/948262336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9fc0b1400 msgr2=0x7ff9fc1e7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:25.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.789+0000 7ffa01534640 1 --2- 172.21.15.154:0/948262336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9fc0b1400 0x7ff9fc1e7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:25.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.789+0000 7ffa01534640 1 -- 172.21.15.154:0/948262336 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f00094d0 con 0x7ff9fc15cf80 2023-09-23T03:24:25.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.790+0000 7ffa01534640 1 --2- 172.21.15.154:0/948262336 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9fc15cf80 0x7ff9fc154d20 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400cd50 tx=0x7ff9f400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:25.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.790+0000 7ff9f97fa640 1 -- 172.21.15.154:0/948262336 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f400f680 con 0x7ff9fc15cf80 2023-09-23T03:24:25.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.790+0000 7ff9f97fa640 1 -- 172.21.15.154:0/948262336 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f400f800 con 0x7ff9fc15cf80 2023-09-23T03:24:25.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.790+0000 7ff9f97fa640 1 -- 172.21.15.154:0/948262336 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f400fbb0 con 0x7ff9fc15cf80 2023-09-23T03:24:25.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.790+0000 7ffa02f97640 1 -- 172.21.15.154:0/948262336 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9fc161090 con 0x7ff9fc15cf80 2023-09-23T03:24:25.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.790+0000 7ffa02f97640 1 -- 172.21.15.154:0/948262336 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9fc155ec0 con 0x7ff9fc15cf80 2023-09-23T03:24:25.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.791+0000 7ffa02f97640 1 -- 172.21.15.154:0/948262336 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9fc155ce0 con 0x7ff9fc15cf80 2023-09-23T03:24:25.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.792+0000 7ff9f97fa640 1 -- 172.21.15.154:0/948262336 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7ff9f400b040 con 0x7ff9fc15cf80 2023-09-23T03:24:25.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.793+0000 7ff9f97fa640 1 --2- 172.21.15.154:0/948262336 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff9e006af30 0x7ff9e006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:25.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.793+0000 7ff9f97fa640 1 -- 172.21.15.154:0/948262336 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6433+0+0 (secure 0 0 0) 0x7ff9f408a480 con 0x7ff9fc15cf80 2023-09-23T03:24:25.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.794+0000 7ffa00d33640 1 --2- 172.21.15.154:0/948262336 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff9e006af30 0x7ff9e006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:25.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.794+0000 7ffa00d33640 1 --2- 172.21.15.154:0/948262336 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff9e006af30 0x7ff9e006d3f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0035000 tx=0x7ff9f0008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:25.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:25.798+0000 7ff9f97fa640 1 -- 172.21.15.154:0/948262336 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7ff9f40551d0 con 0x7ff9fc15cf80 2023-09-23T03:24:26.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.000+0000 7ffa02f97640 1 -- 172.21.15.154:0/948262336 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff9fc073eb0 con 0x7ff9fc15cf80 2023-09-23T03:24:26.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.001+0000 7ff9f97fa640 1 -- 172.21.15.154:0/948262336 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v86) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff9f40587f0 con 0x7ff9fc15cf80 2023-09-23T03:24:26.001 INFO:teuthology.orchestra.run.smithi154.stderr:listed 0 entries 2023-09-23T03:24:26.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.003+0000 7ffa02f97640 1 -- 172.21.15.154:0/948262336 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff9e006af30 msgr2=0x7ff9e006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:26.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.003+0000 7ffa02f97640 1 --2- 172.21.15.154:0/948262336 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff9e006af30 0x7ff9e006d3f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0035000 tx=0x7ff9f0008030 comp rx=0 tx=0).stop 2023-09-23T03:24:26.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.003+0000 7ffa02f97640 1 -- 172.21.15.154:0/948262336 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9fc15cf80 msgr2=0x7ff9fc154d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:26.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.003+0000 7ffa02f97640 1 --2- 172.21.15.154:0/948262336 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9fc15cf80 0x7ff9fc154d20 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400cd50 tx=0x7ff9f400a650 comp rx=0 tx=0).stop 2023-09-23T03:24:26.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.003+0000 7ffa02f97640 1 -- 172.21.15.154:0/948262336 shutdown_connections 2023-09-23T03:24:26.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.003+0000 7ffa02f97640 1 --2- 172.21.15.154:0/948262336 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff9e006af30 0x7ff9e006d3f0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:26.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.004+0000 7ffa02f97640 1 --2- 172.21.15.154:0/948262336 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9fc15cf80 0x7ff9fc154d20 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:26.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.004+0000 7ffa02f97640 1 --2- 172.21.15.154:0/948262336 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9fc0cdfa0 0x7ff9fc0c7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:26.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.004+0000 7ffa02f97640 1 --2- 172.21.15.154:0/948262336 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9fc0b1400 0x7ff9fc1e7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:26.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.004+0000 7ffa02f97640 1 -- 172.21.15.154:0/948262336 >> 172.21.15.154:0/948262336 conn(0x7ff9fc0b08e0 msgr2=0x7ff9fc15eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:26.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.004+0000 7ffa02f97640 1 -- 172.21.15.154:0/948262336 shutdown_connections 2023-09-23T03:24:26.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.004+0000 7ffa02f97640 1 -- 172.21.15.154:0/948262336 wait complete. 2023-09-23T03:24:26.023 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:24:26.024 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:24:26.077 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:24:26.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.341+0000 7f82a76af640 1 Processor -- start 2023-09-23T03:24:26.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.341+0000 7f82a76af640 1 -- start start 2023-09-23T03:24:26.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.342+0000 7f82a76af640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f82a00acce0 0x7f82a015ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:26.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.342+0000 7f82a76af640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82a00b1b20 0x7f82a00ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:26.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.342+0000 7f82a76af640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82a015ac30 con 0x7f82a00cee70 2023-09-23T03:24:26.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.342+0000 7f82a76af640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82a00b2e90 con 0x7f82a00acce0 2023-09-23T03:24:26.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.342+0000 7f82a76af640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82a00cf320 con 0x7f82a00b1b20 2023-09-23T03:24:26.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.343+0000 7f82a4c4a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82a00b1b20 0x7f82a00ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:26.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.343+0000 7f82a4c4a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82a00b1b20 0x7f82a00ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:51892/0 (socket says 172.21.15.154:51892) 2023-09-23T03:24:26.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.343+0000 7f82a4c4a640 1 -- 172.21.15.154:0/218386671 learned_addr learned my addr 172.21.15.154:0/218386671 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:26.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.343+0000 7f82a544b640 1 --2- 172.21.15.154:0/218386671 >> v2:172.21.15.190:3300/0 conn(0x7f82a00acce0 0x7f82a015ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:26.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.344+0000 7f82a4c4a640 1 -- 172.21.15.154:0/218386671 >> v2:172.21.15.190:3300/0 conn(0x7f82a00acce0 msgr2=0x7f82a015ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:26.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.344+0000 7f82a4c4a640 1 --2- 172.21.15.154:0/218386671 >> v2:172.21.15.190:3300/0 conn(0x7f82a00acce0 0x7f82a015ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:26.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.344+0000 7f8297fff640 1 -- 172.21.15.154:0/218386671 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1225240691 0 0) 0x7f8298001490 con 0x7f82a00cee70 2023-09-23T03:24:26.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.344+0000 7f82a4c4a640 1 -- 172.21.15.154:0/218386671 >> v1:172.21.15.190:6789/0 conn(0x7f82a00cee70 legacy=0x7f82a006cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:26.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.344+0000 7f82a4c4a640 1 -- 172.21.15.154:0/218386671 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a00b1f70 con 0x7f82a00b1b20 2023-09-23T03:24:26.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.345+0000 7f82a4c4a640 1 --2- 172.21.15.154:0/218386671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82a00b1b20 0x7f82a00ce930 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8290009870 tx=0x7f829002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfaf51a33f7767a3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:26.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.346+0000 7f8297fff640 1 -- 172.21.15.154:0/218386671 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8290031640 con 0x7f82a00b1b20 2023-09-23T03:24:26.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.346+0000 7f8297fff640 1 -- 172.21.15.154:0/218386671 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82900317a0 con 0x7f82a00b1b20 2023-09-23T03:24:26.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.346+0000 7f8297fff640 1 -- 172.21.15.154:0/218386671 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8290031b30 con 0x7f82a00b1b20 2023-09-23T03:24:26.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.347+0000 7f82a76af640 1 -- 172.21.15.154:0/218386671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82a00b1b20 msgr2=0x7f82a00ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:26.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.347+0000 7f82a76af640 1 --2- 172.21.15.154:0/218386671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82a00b1b20 0x7f82a00ce930 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8290009870 tx=0x7f829002b8f0 comp rx=0 tx=0).stop 2023-09-23T03:24:26.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.347+0000 7f82a76af640 1 -- 172.21.15.154:0/218386671 shutdown_connections 2023-09-23T03:24:26.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.347+0000 7f82a76af640 1 --2- 172.21.15.154:0/218386671 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82a00b1b20 0x7f82a00ce930 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:26.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.347+0000 7f82a76af640 1 --2- 172.21.15.154:0/218386671 >> v2:172.21.15.190:3300/0 conn(0x7f82a00acce0 0x7f82a015ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:26.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.347+0000 7f82a76af640 1 -- 172.21.15.154:0/218386671 >> 172.21.15.154:0/218386671 conn(0x7f82a006a0f0 msgr2=0x7f82a006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:26.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.347+0000 7f82a76af640 1 -- 172.21.15.154:0/218386671 shutdown_connections 2023-09-23T03:24:26.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.347+0000 7f82a76af640 1 -- 172.21.15.154:0/218386671 wait complete. 2023-09-23T03:24:26.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.349+0000 7f82a76af640 1 Processor -- start 2023-09-23T03:24:26.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.349+0000 7f82a76af640 1 -- start start 2023-09-23T03:24:26.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.350+0000 7f82a76af640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82a00acce0 0x7f82a01ed780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:26.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.350+0000 7f82a76af640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82a00b1b20 0x7f82a0070f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:26.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.350+0000 7f82a76af640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82a00cee70 0x7f82a0071470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:26.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.350+0000 7f82a76af640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a0159320 con 0x7f82a00acce0 2023-09-23T03:24:26.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.350+0000 7f82a76af640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a00c3590 con 0x7f82a00cee70 2023-09-23T03:24:26.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.350+0000 7f82a76af640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82a00c3870 con 0x7f82a00b1b20 2023-09-23T03:24:26.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.350+0000 7f82a544b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82a00acce0 0x7f82a01ed780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:26.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.350+0000 7f82a544b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82a00acce0 0x7f82a01ed780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46772/0 (socket says 172.21.15.154:46772) 2023-09-23T03:24:26.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.350+0000 7f82a544b640 1 -- 172.21.15.154:0/222496100 learned_addr learned my addr 172.21.15.154:0/222496100 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:26.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.351+0000 7f82a4c4a640 1 --2- 172.21.15.154:0/222496100 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82a00b1b20 0x7f82a0070f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:26.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.351+0000 7f82a544b640 1 -- 172.21.15.154:0/222496100 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82a00b1b20 msgr2=0x7f82a0070f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:26.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.351+0000 7f82a544b640 1 --2- 172.21.15.154:0/222496100 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82a00b1b20 0x7f82a0070f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:26.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.351+0000 7f82a544b640 1 -- 172.21.15.154:0/222496100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82a00cee70 msgr2=0x7f82a0071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:26.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.351+0000 7f82a5c4c640 1 --2- 172.21.15.154:0/222496100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82a00cee70 0x7f82a0071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:26.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.351+0000 7f82a544b640 1 --2- 172.21.15.154:0/222496100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82a00cee70 0x7f82a0071470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:26.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.351+0000 7f82a544b640 1 -- 172.21.15.154:0/222496100 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82900094d0 con 0x7f82a00acce0 2023-09-23T03:24:26.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.351+0000 7f82a544b640 1 --2- 172.21.15.154:0/222496100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82a00acce0 0x7f82a01ed780 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f829c00e900 tx=0x7f829c00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:26.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.351+0000 7f8295ffb640 1 -- 172.21.15.154:0/222496100 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829c009660 con 0x7f82a00acce0 2023-09-23T03:24:26.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.351+0000 7f8295ffb640 1 -- 172.21.15.154:0/222496100 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f829c00ca00 con 0x7f82a00acce0 2023-09-23T03:24:26.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.351+0000 7f8295ffb640 1 -- 172.21.15.154:0/222496100 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829c00cd70 con 0x7f82a00acce0 2023-09-23T03:24:26.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.351+0000 7f82a76af640 1 -- 172.21.15.154:0/222496100 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82a00c3b50 con 0x7f82a00acce0 2023-09-23T03:24:26.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.352+0000 7f82a76af640 1 -- 172.21.15.154:0/222496100 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82a015b320 con 0x7f82a00acce0 2023-09-23T03:24:26.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.353+0000 7f82a76af640 1 -- 172.21.15.154:0/222496100 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82a0073eb0 con 0x7f82a00acce0 2023-09-23T03:24:26.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.354+0000 7f8295ffb640 1 -- 172.21.15.154:0/222496100 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f829c013470 con 0x7f82a00acce0 2023-09-23T03:24:26.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.354+0000 7f8295ffb640 1 --2- 172.21.15.154:0/222496100 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f828406af30 0x7f828406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:26.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.355+0000 7f8295ffb640 1 -- 172.21.15.154:0/222496100 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f829c08a110 con 0x7f82a00acce0 2023-09-23T03:24:26.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.355+0000 7f82a4c4a640 1 --2- 172.21.15.154:0/222496100 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f828406af30 0x7f828406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:26.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.355+0000 7f82a4c4a640 1 --2- 172.21.15.154:0/222496100 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f828406af30 0x7f828406d3f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8290036000 tx=0x7f829003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:26.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.359+0000 7f8295ffb640 1 -- 172.21.15.154:0/222496100 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f829c054dd0 con 0x7f82a00acce0 2023-09-23T03:24:26.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.549+0000 7f82a76af640 1 -- 172.21.15.154:0/222496100 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82a006c930 con 0x7f82a00acce0 2023-09-23T03:24:26.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.551+0000 7f8295ffb640 1 -- 172.21.15.154:0/222496100 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+18598 (secure 0 0 0) 0x7f829c017090 con 0x7f82a00acce0 2023-09-23T03:24:26.552 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:26.552 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":86,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:24:22.327793+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:24:07.834123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-09-23T03:24:09.058217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data_extra","create_time":"2023-09-23T03:24:19.995906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:24:26.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.553+0000 7f82a76af640 1 -- 172.21.15.154:0/222496100 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f828406af30 msgr2=0x7f828406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:26.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.553+0000 7f82a76af640 1 --2- 172.21.15.154:0/222496100 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f828406af30 0x7f828406d3f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8290036000 tx=0x7f829003c040 comp rx=0 tx=0).stop 2023-09-23T03:24:26.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.554+0000 7f82a76af640 1 -- 172.21.15.154:0/222496100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82a00acce0 msgr2=0x7f82a01ed780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:26.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.554+0000 7f82a76af640 1 --2- 172.21.15.154:0/222496100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82a00acce0 0x7f82a01ed780 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f829c00e900 tx=0x7f829c00edc0 comp rx=0 tx=0).stop 2023-09-23T03:24:26.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.554+0000 7f82a76af640 1 -- 172.21.15.154:0/222496100 shutdown_connections 2023-09-23T03:24:26.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.554+0000 7f82a76af640 1 --2- 172.21.15.154:0/222496100 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f828406af30 0x7f828406d3f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:26.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.554+0000 7f82a76af640 1 --2- 172.21.15.154:0/222496100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f82a00cee70 0x7f82a0071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:26.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.554+0000 7f82a76af640 1 --2- 172.21.15.154:0/222496100 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f82a00b1b20 0x7f82a0070f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:26.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.554+0000 7f82a76af640 1 --2- 172.21.15.154:0/222496100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82a00acce0 0x7f82a01ed780 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:26.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.554+0000 7f82a76af640 1 -- 172.21.15.154:0/222496100 >> 172.21.15.154:0/222496100 conn(0x7f82a006a0f0 msgr2=0x7f82a00cb310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:26.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.554+0000 7f82a76af640 1 -- 172.21.15.154:0/222496100 shutdown_connections 2023-09-23T03:24:26.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.554+0000 7f82a76af640 1 -- 172.21.15.154:0/222496100 wait complete. 2023-09-23T03:24:26.574 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:24:07.834123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:24:09.058217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-09-23T03:24:19.995906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T03:24:26.574 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:24:26.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.801+0000 7fdd77961640 1 Processor -- start 2023-09-23T03:24:26.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.802+0000 7fdd77961640 1 -- start start 2023-09-23T03:24:26.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.803+0000 7fdd77961640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd700c8d70 0x7fdd7015c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:26.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.803+0000 7fdd77961640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdd700c9150 0x7fdd7015cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:26.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.803+0000 7fdd77961640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdd70071290 0x7fdd700ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:26.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.803+0000 7fdd77961640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd700c9d20 con 0x7fdd700c9150 2023-09-23T03:24:26.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.803+0000 7fdd77961640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd70071850 con 0x7fdd70071290 2023-09-23T03:24:26.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.803+0000 7fdd77961640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd70071b30 con 0x7fdd700c8d70 2023-09-23T03:24:26.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.804+0000 7fdd74efc640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdd700c9150 0x7fdd7015cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:26.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.804+0000 7fdd74efc640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdd700c9150 0x7fdd7015cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46778/0 (socket says 172.21.15.154:46778) 2023-09-23T03:24:26.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.804+0000 7fdd74efc640 1 -- 172.21.15.154:0/1452942434 learned_addr learned my addr 172.21.15.154:0/1452942434 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:26.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.804+0000 7fdd75efe640 1 --2- 172.21.15.154:0/1452942434 >> v2:172.21.15.190:3300/0 conn(0x7fdd70071290 0x7fdd700ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:26.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.804+0000 7fdd74efc640 1 -- 172.21.15.154:0/1452942434 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd700c8d70 msgr2=0x7fdd7015c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:26.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.804+0000 7fdd756fd640 1 --2- 172.21.15.154:0/1452942434 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd700c8d70 0x7fdd7015c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:26.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.804+0000 7fdd74efc640 1 --2- 172.21.15.154:0/1452942434 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd700c8d70 0x7fdd7015c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:26.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.804+0000 7fdd74efc640 1 -- 172.21.15.154:0/1452942434 >> v2:172.21.15.190:3300/0 conn(0x7fdd70071290 msgr2=0x7fdd700ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:26.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.804+0000 7fdd74efc640 1 --2- 172.21.15.154:0/1452942434 >> v2:172.21.15.190:3300/0 conn(0x7fdd70071290 0x7fdd700ca1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:26.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.804+0000 7fdd74efc640 1 -- 172.21.15.154:0/1452942434 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd70071e10 con 0x7fdd700c9150 2023-09-23T03:24:26.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.804+0000 7fdd75efe640 1 --2- 172.21.15.154:0/1452942434 >> v2:172.21.15.190:3300/0 conn(0x7fdd70071290 0x7fdd700ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:26.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.805+0000 7fdd74efc640 1 --2- 172.21.15.154:0/1452942434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd700c9150 0x7fdd7015cc10 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fdd700c95d0 tx=0x7fdd6002b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0948d326f351f0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:26.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.806+0000 7fdd67fff640 1 -- 172.21.15.154:0/1452942434 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd600317f0 con 0x7fdd700c9150 2023-09-23T03:24:26.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.806+0000 7fdd67fff640 1 -- 172.21.15.154:0/1452942434 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd60031950 con 0x7fdd700c9150 2023-09-23T03:24:26.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.806+0000 7fdd67fff640 1 -- 172.21.15.154:0/1452942434 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd60031ce0 con 0x7fdd700c9150 2023-09-23T03:24:26.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.806+0000 7fdd77961640 1 -- 172.21.15.154:0/1452942434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd700c9150 msgr2=0x7fdd7015cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:26.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.806+0000 7fdd77961640 1 --2- 172.21.15.154:0/1452942434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd700c9150 0x7fdd7015cc10 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fdd700c95d0 tx=0x7fdd6002b890 comp rx=0 tx=0).stop 2023-09-23T03:24:26.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.806+0000 7fdd77961640 1 -- 172.21.15.154:0/1452942434 shutdown_connections 2023-09-23T03:24:26.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.806+0000 7fdd77961640 1 --2- 172.21.15.154:0/1452942434 >> v2:172.21.15.190:3300/0 conn(0x7fdd70071290 0x7fdd700ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:26.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.806+0000 7fdd77961640 1 --2- 172.21.15.154:0/1452942434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd700c9150 0x7fdd7015cc10 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:26.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.806+0000 7fdd77961640 1 --2- 172.21.15.154:0/1452942434 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd700c8d70 0x7fdd7015c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:26.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.806+0000 7fdd77961640 1 -- 172.21.15.154:0/1452942434 >> 172.21.15.154:0/1452942434 conn(0x7fdd7006c010 msgr2=0x7fdd7006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:26.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.806+0000 7fdd77961640 1 -- 172.21.15.154:0/1452942434 shutdown_connections 2023-09-23T03:24:26.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.806+0000 7fdd77961640 1 -- 172.21.15.154:0/1452942434 wait complete. 2023-09-23T03:24:26.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.808+0000 7fdd77961640 1 Processor -- start 2023-09-23T03:24:26.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.808+0000 7fdd77961640 1 -- start start 2023-09-23T03:24:26.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.809+0000 7fdd77961640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd70071290 0x7fdd701f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:26.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.809+0000 7fdd77961640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd700c8d70 0x7fdd700c7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:26.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.809+0000 7fdd77961640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd700c9150 0x7fdd701f5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:26.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.809+0000 7fdd77961640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd700c6fb0 con 0x7fdd700c8d70 2023-09-23T03:24:26.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.809+0000 7fdd77961640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd701f56f0 con 0x7fdd70071290 2023-09-23T03:24:26.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.809+0000 7fdd77961640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd701f6750 con 0x7fdd700c9150 2023-09-23T03:24:26.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.810+0000 7fdd74efc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd700c8d70 0x7fdd700c7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:26.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.810+0000 7fdd74efc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd700c8d70 0x7fdd700c7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46794/0 (socket says 172.21.15.154:46794) 2023-09-23T03:24:26.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.810+0000 7fdd74efc640 1 -- 172.21.15.154:0/3043741494 learned_addr learned my addr 172.21.15.154:0/3043741494 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:26.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.810+0000 7fdd756fd640 1 --2- 172.21.15.154:0/3043741494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd70071290 0x7fdd701f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:26.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.810+0000 7fdd74efc640 1 -- 172.21.15.154:0/3043741494 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd700c9150 msgr2=0x7fdd701f5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:26.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.810+0000 7fdd75efe640 1 --2- 172.21.15.154:0/3043741494 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd700c9150 0x7fdd701f5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:26.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.810+0000 7fdd74efc640 1 --2- 172.21.15.154:0/3043741494 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd700c9150 0x7fdd701f5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:26.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.810+0000 7fdd74efc640 1 -- 172.21.15.154:0/3043741494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd70071290 msgr2=0x7fdd701f51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:26.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.810+0000 7fdd74efc640 1 --2- 172.21.15.154:0/3043741494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd70071290 0x7fdd701f51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:26.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.810+0000 7fdd74efc640 1 -- 172.21.15.154:0/3043741494 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd600094d0 con 0x7fdd700c8d70 2023-09-23T03:24:26.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.810+0000 7fdd756fd640 1 --2- 172.21.15.154:0/3043741494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd70071290 0x7fdd701f51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:26.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.810+0000 7fdd74efc640 1 --2- 172.21.15.154:0/3043741494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd700c8d70 0x7fdd700c7390 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fdd6002b8d0 tx=0x7fdd600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:26.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.811+0000 7fdd65ffb640 1 -- 172.21.15.154:0/3043741494 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd60032910 con 0x7fdd700c8d70 2023-09-23T03:24:26.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.811+0000 7fdd65ffb640 1 -- 172.21.15.154:0/3043741494 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd60032a70 con 0x7fdd700c8d70 2023-09-23T03:24:26.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.811+0000 7fdd65ffb640 1 -- 172.21.15.154:0/3043741494 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd60032e20 con 0x7fdd700c8d70 2023-09-23T03:24:26.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.811+0000 7fdd77961640 1 -- 172.21.15.154:0/3043741494 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd700c78d0 con 0x7fdd700c8d70 2023-09-23T03:24:26.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.811+0000 7fdd77961640 1 -- 172.21.15.154:0/3043741494 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd700c6610 con 0x7fdd700c8d70 2023-09-23T03:24:26.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.812+0000 7fdd77961640 1 -- 172.21.15.154:0/3043741494 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd7004e920 con 0x7fdd700c8d70 2023-09-23T03:24:26.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.813+0000 7fdd65ffb640 1 -- 172.21.15.154:0/3043741494 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fdd600453f0 con 0x7fdd700c8d70 2023-09-23T03:24:26.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.814+0000 7fdd65ffb640 1 --2- 172.21.15.154:0/3043741494 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdd5406af30 0x7fdd5406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:26.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.814+0000 7fdd65ffb640 1 -- 172.21.15.154:0/3043741494 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6433+0+0 (secure 0 0 0) 0x7fdd600abf30 con 0x7fdd700c8d70 2023-09-23T03:24:26.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.815+0000 7fdd756fd640 1 --2- 172.21.15.154:0/3043741494 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdd5406af30 0x7fdd5406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:26.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.815+0000 7fdd756fd640 1 --2- 172.21.15.154:0/3043741494 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdd5406af30 0x7fdd5406d3f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c00b820 tx=0x7fdd6c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:26.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:26.819+0000 7fdd65ffb640 1 -- 172.21.15.154:0/3043741494 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fdd60076e40 con 0x7fdd700c8d70 2023-09-23T03:24:27.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.010+0000 7fdd77961640 1 -- 172.21.15.154:0/3043741494 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdd7006cbc0 con 0x7fdd700c8d70 2023-09-23T03:24:27.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.010+0000 7fdd65ffb640 1 -- 172.21.15.154:0/3043741494 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdd6007a460 con 0x7fdd700c8d70 2023-09-23T03:24:27.011 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:24:27.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.013+0000 7fdd77961640 1 -- 172.21.15.154:0/3043741494 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdd5406af30 msgr2=0x7fdd5406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:27.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.013+0000 7fdd77961640 1 --2- 172.21.15.154:0/3043741494 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdd5406af30 0x7fdd5406d3f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c00b820 tx=0x7fdd6c008040 comp rx=0 tx=0).stop 2023-09-23T03:24:27.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.013+0000 7fdd77961640 1 -- 172.21.15.154:0/3043741494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd700c8d70 msgr2=0x7fdd700c7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:27.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.013+0000 7fdd77961640 1 --2- 172.21.15.154:0/3043741494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd700c8d70 0x7fdd700c7390 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fdd6002b8d0 tx=0x7fdd600046c0 comp rx=0 tx=0).stop 2023-09-23T03:24:27.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.013+0000 7fdd77961640 1 -- 172.21.15.154:0/3043741494 shutdown_connections 2023-09-23T03:24:27.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.013+0000 7fdd77961640 1 --2- 172.21.15.154:0/3043741494 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdd5406af30 0x7fdd5406d3f0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:27.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.013+0000 7fdd77961640 1 --2- 172.21.15.154:0/3043741494 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd700c9150 0x7fdd701f5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:27.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.013+0000 7fdd77961640 1 --2- 172.21.15.154:0/3043741494 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd700c8d70 0x7fdd700c7390 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:27.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.013+0000 7fdd77961640 1 --2- 172.21.15.154:0/3043741494 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd70071290 0x7fdd701f51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:27.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.013+0000 7fdd77961640 1 -- 172.21.15.154:0/3043741494 >> 172.21.15.154:0/3043741494 conn(0x7fdd7006c010 msgr2=0x7fdd7015b020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:27.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.014+0000 7fdd77961640 1 -- 172.21.15.154:0/3043741494 shutdown_connections 2023-09-23T03:24:27.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.014+0000 7fdd77961640 1 -- 172.21.15.154:0/3043741494 wait complete. 2023-09-23T03:24:27.033 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:24:27.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.252+0000 7f04612bb640 1 Processor -- start 2023-09-23T03:24:27.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.252+0000 7f04612bb640 1 -- start start 2023-09-23T03:24:27.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.254+0000 7f04612bb640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f045c15aba0 0x7f045c15bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:27.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.254+0000 7f04612bb640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f045c15e5f0 0x7f045c15ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:27.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.254+0000 7f04612bb640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f045c15b6a0 con 0x7f045c0caea0 2023-09-23T03:24:27.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.254+0000 7f04612bb640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f045c15f0b0 con 0x7f045c15e5f0 2023-09-23T03:24:27.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.254+0000 7f04612bb640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f045c15f390 con 0x7f045c15aba0 2023-09-23T03:24:27.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.254+0000 7f045b7fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f045c0caea0 0x7f045c0cc5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:39006/0 (socket says 172.21.15.154:39006) 2023-09-23T03:24:27.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.254+0000 7f045b7fe640 1 -- 172.21.15.154:0/4086302498 learned_addr learned my addr 172.21.15.154:0/4086302498 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:24:27.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.255+0000 7f0453fff640 1 --2- 172.21.15.154:0/4086302498 >> v2:172.21.15.190:3300/0 conn(0x7f045c15e5f0 0x7f045c15ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:27.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.255+0000 7f045affd640 1 --2- 172.21.15.154:0/4086302498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f045c15aba0 0x7f045c15bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:27.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.255+0000 7f045a7fc640 1 -- 172.21.15.154:0/4086302498 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2249810831 0 0) 0x7f045c15b6a0 con 0x7f045c0caea0 2023-09-23T03:24:27.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.255+0000 7f045a7fc640 1 -- 172.21.15.154:0/4086302498 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0448003520 con 0x7f045c0caea0 2023-09-23T03:24:27.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.255+0000 7f045a7fc640 1 -- 172.21.15.154:0/4086302498 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2548218825 0 0) 0x7f0448003520 con 0x7f045c0caea0 2023-09-23T03:24:27.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.255+0000 7f045a7fc640 1 -- 172.21.15.154:0/4086302498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f045c15aba0 msgr2=0x7f045c15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:27.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.255+0000 7f045a7fc640 1 --2- 172.21.15.154:0/4086302498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f045c15aba0 0x7f045c15bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:27.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.255+0000 7f045a7fc640 1 -- 172.21.15.154:0/4086302498 >> v2:172.21.15.190:3300/0 conn(0x7f045c15e5f0 msgr2=0x7f045c15ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:27.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.255+0000 7f045a7fc640 1 --2- 172.21.15.154:0/4086302498 >> v2:172.21.15.190:3300/0 conn(0x7f045c15e5f0 0x7f045c15ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:27.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.255+0000 7f045a7fc640 1 -- 172.21.15.154:0/4086302498 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f045c15b330 con 0x7f045c0caea0 2023-09-23T03:24:27.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.255+0000 7f045a7fc640 1 -- 172.21.15.154:0/4086302498 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f04540019f0 con 0x7f045c0caea0 2023-09-23T03:24:27.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.255+0000 7f045a7fc640 1 -- 172.21.15.154:0/4086302498 >> v1:172.21.15.154:6789/0 conn(0x7f045c0caea0 legacy=0x7f045c0cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:27.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.255+0000 7f045a7fc640 1 --2- 172.21.15.154:0/4086302498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0448003970 0x7f0448003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:27.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.255+0000 7f045a7fc640 1 --2- 172.21.15.154:0/4086302498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04480047c0 0x7f0448004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:27.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.255+0000 7f045a7fc640 1 --2- 172.21.15.154:0/4086302498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f045c15aba0 0x7f0448005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:27.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.255+0000 7f045a7fc640 1 -- 172.21.15.154:0/4086302498 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045c15f0b0 con 0x7f0448003970 2023-09-23T03:24:27.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.255+0000 7f045a7fc640 1 -- 172.21.15.154:0/4086302498 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045c15f390 con 0x7f04480047c0 2023-09-23T03:24:27.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.255+0000 7f045a7fc640 1 -- 172.21.15.154:0/4086302498 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f044800b980 con 0x7f045c15aba0 2023-09-23T03:24:27.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.256+0000 7f045affd640 1 --2- 172.21.15.154:0/4086302498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0448003970 0x7f0448003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:27.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.256+0000 7f0453fff640 1 --2- 172.21.15.154:0/4086302498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04480047c0 0x7f0448004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:27.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.256+0000 7f045affd640 1 -- 172.21.15.154:0/4086302498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f045c15aba0 msgr2=0x7f0448005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:27.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.256+0000 7f045affd640 1 --2- 172.21.15.154:0/4086302498 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f045c15aba0 0x7f0448005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:27.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.256+0000 7f045affd640 1 -- 172.21.15.154:0/4086302498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04480047c0 msgr2=0x7f0448004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:27.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.256+0000 7f045affd640 1 --2- 172.21.15.154:0/4086302498 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f04480047c0 0x7f0448004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:27.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.256+0000 7f045affd640 1 -- 172.21.15.154:0/4086302498 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0448003520 con 0x7f0448003970 2023-09-23T03:24:27.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.257+0000 7f045affd640 1 --2- 172.21.15.154:0/4086302498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0448003970 0x7f0448003d90 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f044c009ac0 tx=0x7f04480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab62f4462d9d2b39 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:27.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.257+0000 7f045a7fc640 1 -- 172.21.15.154:0/4086302498 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044c02c6d0 con 0x7f0448003970 2023-09-23T03:24:27.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.258+0000 7f045a7fc640 1 -- 172.21.15.154:0/4086302498 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f044c02c830 con 0x7f0448003970 2023-09-23T03:24:27.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.258+0000 7f045a7fc640 1 -- 172.21.15.154:0/4086302498 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044c02cbc0 con 0x7f0448003970 2023-09-23T03:24:27.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.258+0000 7f04612bb640 1 -- 172.21.15.154:0/4086302498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0448003970 msgr2=0x7f0448003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:27.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.258+0000 7f04612bb640 1 --2- 172.21.15.154:0/4086302498 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0448003970 0x7f0448003d90 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f044c009ac0 tx=0x7f04480057f0 comp rx=0 tx=0).stop 2023-09-23T03:24:27.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.258+0000 7f045b7fe640 1 -- 172.21.15.154:0/4086302498 reap_dead start 2023-09-23T03:24:27.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.258+0000 7f04612bb640 1 -- 172.21.15.154:0/4086302498 shutdown_connections 2023-09-23T03:24:27.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.258+0000 7f04612bb640 1 -- 172.21.15.154:0/4086302498 >> 172.21.15.154:0/4086302498 conn(0x7f045c06c010 msgr2=0x7f045c06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:27.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.258+0000 7f04612bb640 1 -- 172.21.15.154:0/4086302498 shutdown_connections 2023-09-23T03:24:27.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.258+0000 7f04612bb640 1 -- 172.21.15.154:0/4086302498 wait complete. 2023-09-23T03:24:27.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.260+0000 7f04612bb640 1 Processor -- start 2023-09-23T03:24:27.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.260+0000 7f04612bb640 1 -- start start 2023-09-23T03:24:27.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.261+0000 7f04612bb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0448003970 0x7f045c14da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:27.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.261+0000 7f04612bb640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f045c0c7290 0x7f045c1507e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:27.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.261+0000 7f04612bb640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f045c0c90c0 0x7f045c15c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:27.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.261+0000 7f04612bb640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045c0c8020 con 0x7f0448003970 2023-09-23T03:24:27.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.261+0000 7f04612bb640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045c0c8300 con 0x7f045c0c7290 2023-09-23T03:24:27.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.261+0000 7f04612bb640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f045c0c85e0 con 0x7f045c0c90c0 2023-09-23T03:24:27.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.261+0000 7f045affd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0448003970 0x7f045c14da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:27.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.261+0000 7f045affd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0448003970 0x7f045c14da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46818/0 (socket says 172.21.15.154:46818) 2023-09-23T03:24:27.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.261+0000 7f045affd640 1 -- 172.21.15.154:0/544557381 learned_addr learned my addr 172.21.15.154:0/544557381 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:27.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.262+0000 7f045affd640 1 -- 172.21.15.154:0/544557381 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f045c0c90c0 msgr2=0x7f045c15c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:27.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.262+0000 7f045affd640 1 --2- 172.21.15.154:0/544557381 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f045c0c90c0 0x7f045c15c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:27.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.262+0000 7f0453fff640 1 --2- 172.21.15.154:0/544557381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f045c0c7290 0x7f045c1507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:27.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.262+0000 7f045affd640 1 -- 172.21.15.154:0/544557381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f045c0c7290 msgr2=0x7f045c1507e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:27.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.262+0000 7f045affd640 1 --2- 172.21.15.154:0/544557381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f045c0c7290 0x7f045c1507e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:27.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.262+0000 7f045affd640 1 -- 172.21.15.154:0/544557381 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f045c15b100 con 0x7f0448003970 2023-09-23T03:24:27.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.262+0000 7f045affd640 1 --2- 172.21.15.154:0/544557381 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0448003970 0x7f045c14da40 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f044c02c2d0 tx=0x7f044c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:27.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.262+0000 7f04537fe640 1 -- 172.21.15.154:0/544557381 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044c003810 con 0x7f0448003970 2023-09-23T03:24:27.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.262+0000 7f04537fe640 1 -- 172.21.15.154:0/544557381 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f044c003970 con 0x7f0448003970 2023-09-23T03:24:27.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.262+0000 7f04537fe640 1 -- 172.21.15.154:0/544557381 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044c003ce0 con 0x7f0448003970 2023-09-23T03:24:27.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.263+0000 7f04612bb640 1 -- 172.21.15.154:0/544557381 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f045c0c88c0 con 0x7f0448003970 2023-09-23T03:24:27.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.263+0000 7f04612bb640 1 -- 172.21.15.154:0/544557381 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f045c150d20 con 0x7f0448003970 2023-09-23T03:24:27.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.264+0000 7f04612bb640 1 -- 172.21.15.154:0/544557381 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f045c150eb0 con 0x7f0448003970 2023-09-23T03:24:27.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.265+0000 7f04537fe640 1 -- 172.21.15.154:0/544557381 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f044c040450 con 0x7f0448003970 2023-09-23T03:24:27.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.265+0000 7f04537fe640 1 --2- 172.21.15.154:0/544557381 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f043c06af60 0x7f043c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:27.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.266+0000 7f04537fe640 1 -- 172.21.15.154:0/544557381 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f044c0aba10 con 0x7f0448003970 2023-09-23T03:24:27.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.266+0000 7f0453fff640 1 --2- 172.21.15.154:0/544557381 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f043c06af60 0x7f043c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:27.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.266+0000 7f0453fff640 1 --2- 172.21.15.154:0/544557381 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f043c06af60 0x7f043c06d420 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f045c159b80 tx=0x7f0444008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:27.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.270+0000 7f04537fe640 1 -- 172.21.15.154:0/544557381 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f044c076760 con 0x7f0448003970 2023-09-23T03:24:27.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.464+0000 7f04612bb640 1 -- 172.21.15.154:0/544557381 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f045c04e920 con 0x7f0448003970 2023-09-23T03:24:27.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.465+0000 7f04537fe640 1 -- 172.21.15.154:0/544557381 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7f044c0044e0 con 0x7f0448003970 2023-09-23T03:24:27.466 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:24:27.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.468+0000 7f04612bb640 1 -- 172.21.15.154:0/544557381 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f043c06af60 msgr2=0x7f043c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:27.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.468+0000 7f04612bb640 1 --2- 172.21.15.154:0/544557381 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f043c06af60 0x7f043c06d420 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f045c159b80 tx=0x7f0444008040 comp rx=0 tx=0).stop 2023-09-23T03:24:27.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.468+0000 7f04612bb640 1 -- 172.21.15.154:0/544557381 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0448003970 msgr2=0x7f045c14da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:27.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.468+0000 7f04612bb640 1 --2- 172.21.15.154:0/544557381 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0448003970 0x7f045c14da40 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f044c02c2d0 tx=0x7f044c004730 comp rx=0 tx=0).stop 2023-09-23T03:24:27.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.468+0000 7f04612bb640 1 -- 172.21.15.154:0/544557381 shutdown_connections 2023-09-23T03:24:27.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.468+0000 7f04612bb640 1 --2- 172.21.15.154:0/544557381 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f043c06af60 0x7f043c06d420 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:27.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.468+0000 7f04612bb640 1 --2- 172.21.15.154:0/544557381 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f045c0c90c0 0x7f045c15c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:27.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.468+0000 7f04612bb640 1 --2- 172.21.15.154:0/544557381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f045c0c7290 0x7f045c1507e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:27.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.468+0000 7f04612bb640 1 --2- 172.21.15.154:0/544557381 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0448003970 0x7f045c14da40 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:27.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.468+0000 7f04612bb640 1 -- 172.21.15.154:0/544557381 >> 172.21.15.154:0/544557381 conn(0x7f045c06c010 msgr2=0x7f045c0ab8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:27.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.468+0000 7f04612bb640 1 -- 172.21.15.154:0/544557381 shutdown_connections 2023-09-23T03:24:27.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.468+0000 7f04612bb640 1 -- 172.21.15.154:0/544557381 wait complete. 2023-09-23T03:24:27.487 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:24:27.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.698+0000 7f88fe267640 1 Processor -- start 2023-09-23T03:24:27.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.698+0000 7f88fe267640 1 -- start start 2023-09-23T03:24:27.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.700+0000 7f88fe267640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f88f815ba50 0x7f88f806fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:27.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.700+0000 7f88fe267640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88f815eef0 0x7f88f80703d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:27.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.700+0000 7f88fe267640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88f80cb6c0 con 0x7f88f8158480 2023-09-23T03:24:27.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.700+0000 7f88fe267640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88f815ad80 con 0x7f88f815ba50 2023-09-23T03:24:27.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.700+0000 7f88fe267640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88f815af70 con 0x7f88f815eef0 2023-09-23T03:24:27.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.700+0000 7f88fca64640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f88f815ba50 0x7f88f806fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:27.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.700+0000 7f88fd265640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f88f8158480 0x7f88f80cade0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:39016/0 (socket says 172.21.15.154:39016) 2023-09-23T03:24:27.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.700+0000 7f88fd265640 1 -- 172.21.15.154:0/1338109089 learned_addr learned my addr 172.21.15.154:0/1338109089 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:24:27.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.700+0000 7f88fda66640 1 --2- 172.21.15.154:0/1338109089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88f815eef0 0x7f88f80703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:27.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.701+0000 7f88fca64640 1 -- 172.21.15.154:0/1338109089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88f815eef0 msgr2=0x7f88f80703d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:27.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.701+0000 7f88fca64640 1 --2- 172.21.15.154:0/1338109089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88f815eef0 0x7f88f80703d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:27.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.701+0000 7f88fca64640 1 -- 172.21.15.154:0/1338109089 >> v1:172.21.15.154:6789/0 conn(0x7f88f8158480 legacy=0x7f88f80cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:27.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.701+0000 7f88fca64640 1 -- 172.21.15.154:0/1338109089 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f815b250 con 0x7f88f815ba50 2023-09-23T03:24:27.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.701+0000 7f88effff640 1 -- 172.21.15.154:0/1338109089 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2132436840 0 0) 0x7f88f80cb6c0 con 0x7f88f8158480 2023-09-23T03:24:27.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.701+0000 7f88fda66640 1 --2- 172.21.15.154:0/1338109089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88f815eef0 0x7f88f80703d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:24:27.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.702+0000 7f88fca64640 1 --2- 172.21.15.154:0/1338109089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88f815ba50 0x7f88f806fe90 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f88e80097a0 tx=0x7f88e802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8398b3ed2e385a7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:27.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.702+0000 7f88effff640 1 -- 172.21.15.154:0/1338109089 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e802d040 con 0x7f88f815ba50 2023-09-23T03:24:27.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.702+0000 7f88effff640 1 -- 172.21.15.154:0/1338109089 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e8031810 con 0x7f88f815ba50 2023-09-23T03:24:27.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.702+0000 7f88effff640 1 -- 172.21.15.154:0/1338109089 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e8031ba0 con 0x7f88f815ba50 2023-09-23T03:24:27.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.703+0000 7f88fe267640 1 -- 172.21.15.154:0/1338109089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88f815ba50 msgr2=0x7f88f806fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:27.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.703+0000 7f88fe267640 1 --2- 172.21.15.154:0/1338109089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88f815ba50 0x7f88f806fe90 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f88e80097a0 tx=0x7f88e802b8b0 comp rx=0 tx=0).stop 2023-09-23T03:24:27.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.703+0000 7f88fe267640 1 -- 172.21.15.154:0/1338109089 shutdown_connections 2023-09-23T03:24:27.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.703+0000 7f88fe267640 1 --2- 172.21.15.154:0/1338109089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88f815eef0 0x7f88f80703d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:27.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.703+0000 7f88fe267640 1 --2- 172.21.15.154:0/1338109089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88f815ba50 0x7f88f806fe90 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:27.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.703+0000 7f88fe267640 1 -- 172.21.15.154:0/1338109089 >> 172.21.15.154:0/1338109089 conn(0x7f88f80aff20 msgr2=0x7f88f80b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:27.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.703+0000 7f88fe267640 1 -- 172.21.15.154:0/1338109089 shutdown_connections 2023-09-23T03:24:27.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.703+0000 7f88fe267640 1 -- 172.21.15.154:0/1338109089 wait complete. 2023-09-23T03:24:27.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.705+0000 7f88fe267640 1 Processor -- start 2023-09-23T03:24:27.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.705+0000 7f88fe267640 1 -- start start 2023-09-23T03:24:27.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.706+0000 7f88fe267640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88f8158480 0x7f88f80c1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:27.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.706+0000 7f88fe267640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88f815ba50 0x7f88f80c2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:27.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.706+0000 7f88fe267640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88f815eef0 0x7f88f815be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:27.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.706+0000 7f88fe267640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f81f0150 con 0x7f88f8158480 2023-09-23T03:24:27.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.706+0000 7f88fe267640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f80c3c60 con 0x7f88f815eef0 2023-09-23T03:24:27.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.706+0000 7f88fe267640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88f80c32c0 con 0x7f88f815ba50 2023-09-23T03:24:27.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.706+0000 7f88fd265640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88f8158480 0x7f88f80c1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:27.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.706+0000 7f88fd265640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88f8158480 0x7f88f80c1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46844/0 (socket says 172.21.15.154:46844) 2023-09-23T03:24:27.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.706+0000 7f88fd265640 1 -- 172.21.15.154:0/3515666966 learned_addr learned my addr 172.21.15.154:0/3515666966 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:27.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.706+0000 7f88fd265640 1 -- 172.21.15.154:0/3515666966 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88f815ba50 msgr2=0x7f88f80c2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:27.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.706+0000 7f88fd265640 1 --2- 172.21.15.154:0/3515666966 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88f815ba50 0x7f88f80c2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:27.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.706+0000 7f88fd265640 1 -- 172.21.15.154:0/3515666966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88f815eef0 msgr2=0x7f88f815be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:27.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.706+0000 7f88fd265640 1 --2- 172.21.15.154:0/3515666966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88f815eef0 0x7f88f815be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:27.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.707+0000 7f88fd265640 1 -- 172.21.15.154:0/3515666966 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e8009400 con 0x7f88f8158480 2023-09-23T03:24:27.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.707+0000 7f88fd265640 1 --2- 172.21.15.154:0/3515666966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88f8158480 0x7f88f80c1f00 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f88f400c4a0 tx=0x7f88f400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:27.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.707+0000 7f88edffb640 1 -- 172.21.15.154:0/3515666966 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f4005ce0 con 0x7f88f8158480 2023-09-23T03:24:27.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.707+0000 7f88edffb640 1 -- 172.21.15.154:0/3515666966 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f4005e40 con 0x7f88f8158480 2023-09-23T03:24:27.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.707+0000 7f88edffb640 1 -- 172.21.15.154:0/3515666966 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f40072b0 con 0x7f88f8158480 2023-09-23T03:24:27.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.707+0000 7f88fe267640 1 -- 172.21.15.154:0/3515666966 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88f80c35a0 con 0x7f88f8158480 2023-09-23T03:24:27.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.707+0000 7f88fe267640 1 -- 172.21.15.154:0/3515666966 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88f81efd50 con 0x7f88f8158480 2023-09-23T03:24:27.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.708+0000 7f88fe267640 1 -- 172.21.15.154:0/3515666966 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88f81efee0 con 0x7f88f8158480 2023-09-23T03:24:27.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.709+0000 7f88edffb640 1 -- 172.21.15.154:0/3515666966 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f88f4007410 con 0x7f88f8158480 2023-09-23T03:24:27.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.711+0000 7f88edffb640 1 --2- 172.21.15.154:0/3515666966 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f88dc06af30 0x7f88dc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:27.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.711+0000 7f88edffb640 1 -- 172.21.15.154:0/3515666966 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f88f4088ff0 con 0x7f88f8158480 2023-09-23T03:24:27.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.711+0000 7f88fca64640 1 --2- 172.21.15.154:0/3515666966 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f88dc06af30 0x7f88dc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:27.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.712+0000 7f88fca64640 1 --2- 172.21.15.154:0/3515666966 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f88dc06af30 0x7f88dc06d3f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f88e8036000 tx=0x7f88e8030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:27.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.716+0000 7f88edffb640 1 -- 172.21.15.154:0/3515666966 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f88f40bba50 con 0x7f88f8158480 2023-09-23T03:24:27.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.906+0000 7f88fe267640 1 -- 172.21.15.154:0/3515666966 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f88f804e920 con 0x7f88f8158480 2023-09-23T03:24:27.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.906+0000 7f88edffb640 1 -- 172.21.15.154:0/3515666966 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7f88f4017020 con 0x7f88f8158480 2023-09-23T03:24:27.907 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:24:27.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.909+0000 7f88fe267640 1 -- 172.21.15.154:0/3515666966 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f88dc06af30 msgr2=0x7f88dc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:27.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.909+0000 7f88fe267640 1 --2- 172.21.15.154:0/3515666966 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f88dc06af30 0x7f88dc06d3f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f88e8036000 tx=0x7f88e8030000 comp rx=0 tx=0).stop 2023-09-23T03:24:27.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.909+0000 7f88fe267640 1 -- 172.21.15.154:0/3515666966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88f8158480 msgr2=0x7f88f80c1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:27.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.909+0000 7f88fe267640 1 --2- 172.21.15.154:0/3515666966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88f8158480 0x7f88f80c1f00 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f88f400c4a0 tx=0x7f88f400c960 comp rx=0 tx=0).stop 2023-09-23T03:24:27.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.909+0000 7f88fe267640 1 -- 172.21.15.154:0/3515666966 shutdown_connections 2023-09-23T03:24:27.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.909+0000 7f88fe267640 1 --2- 172.21.15.154:0/3515666966 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f88dc06af30 0x7f88dc06d3f0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:27.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.909+0000 7f88fe267640 1 --2- 172.21.15.154:0/3515666966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f88f815eef0 0x7f88f815be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:27.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.909+0000 7f88fe267640 1 --2- 172.21.15.154:0/3515666966 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f88f815ba50 0x7f88f80c2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:27.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.909+0000 7f88fe267640 1 --2- 172.21.15.154:0/3515666966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f88f8158480 0x7f88f80c1f00 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:27.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.909+0000 7f88fe267640 1 -- 172.21.15.154:0/3515666966 >> 172.21.15.154:0/3515666966 conn(0x7f88f80aff20 msgr2=0x7f88f8156bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:27.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.909+0000 7f88fe267640 1 -- 172.21.15.154:0/3515666966 shutdown_connections 2023-09-23T03:24:27.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:27.910+0000 7f88fe267640 1 -- 172.21.15.154:0/3515666966 wait complete. 2023-09-23T03:24:27.928 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-09-23T03:24:28.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.145+0000 7f95e62d4640 1 Processor -- start 2023-09-23T03:24:28.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.145+0000 7f95e62d4640 1 -- start start 2023-09-23T03:24:28.147 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.147+0000 7f95e62d4640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f95e00cb5b0 0x7f95e00cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:28.147 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.147+0000 7f95e62d4640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e015b6a0 con 0x7f95e015aba0 2023-09-23T03:24:28.147 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.147+0000 7f95e62d4640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e015f0b0 con 0x7f95e015bb30 2023-09-23T03:24:28.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.147+0000 7f95e62d4640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95e015bf70 con 0x7f95e00cb5b0 2023-09-23T03:24:28.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.147+0000 7f95dffff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f95e015aba0 0x7f95e015e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:39028/0 (socket says 172.21.15.154:39028) 2023-09-23T03:24:28.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.147+0000 7f95dffff640 1 -- 172.21.15.154:0/125101614 learned_addr learned my addr 172.21.15.154:0/125101614 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:24:28.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.148+0000 7f95e4871640 1 --2- 172.21.15.154:0/125101614 >> v2:172.21.15.190:3300/0 conn(0x7f95e00cb5b0 0x7f95e00cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:28.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.148+0000 7f95df7fe640 1 -- 172.21.15.154:0/125101614 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 654386595 0 0) 0x7f95e015b6a0 con 0x7f95e015aba0 2023-09-23T03:24:28.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.148+0000 7f95df7fe640 1 -- 172.21.15.154:0/125101614 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95cc003520 con 0x7f95e015aba0 2023-09-23T03:24:28.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.148+0000 7f95df7fe640 1 -- 172.21.15.154:0/125101614 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 854873076 0 0) 0x7f95cc003520 con 0x7f95e015aba0 2023-09-23T03:24:28.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.148+0000 7f95df7fe640 1 -- 172.21.15.154:0/125101614 >> v2:172.21.15.190:3300/0 conn(0x7f95e00cb5b0 msgr2=0x7f95e00cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:28.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.148+0000 7f95df7fe640 1 --2- 172.21.15.154:0/125101614 >> v2:172.21.15.190:3300/0 conn(0x7f95e00cb5b0 0x7f95e00cc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:28.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.148+0000 7f95df7fe640 1 -- 172.21.15.154:0/125101614 >> v1:172.21.15.190:6789/0 conn(0x7f95e015bb30 legacy=0x7f95e00caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:28.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.148+0000 7f95df7fe640 1 -- 172.21.15.154:0/125101614 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e015f390 con 0x7f95e015aba0 2023-09-23T03:24:28.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.148+0000 7f95df7fe640 1 -- 172.21.15.154:0/125101614 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f95d0001670 con 0x7f95e015aba0 2023-09-23T03:24:28.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.148+0000 7f95df7fe640 1 -- 172.21.15.154:0/125101614 >> v1:172.21.15.154:6789/0 conn(0x7f95e015aba0 legacy=0x7f95e015e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:28.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.148+0000 7f95df7fe640 1 --2- 172.21.15.154:0/125101614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95cc003970 0x7f95cc003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:28.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.148+0000 7f95df7fe640 1 --2- 172.21.15.154:0/125101614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95cc004370 0x7f95cc004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:28.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.148+0000 7f95df7fe640 1 --2- 172.21.15.154:0/125101614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95cc005040 0x7f95cc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:28.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.148+0000 7f95df7fe640 1 -- 172.21.15.154:0/125101614 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e015bf70 con 0x7f95cc005040 2023-09-23T03:24:28.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.148+0000 7f95df7fe640 1 -- 172.21.15.154:0/125101614 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc00bfa0 con 0x7f95cc004370 2023-09-23T03:24:28.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.148+0000 7f95df7fe640 1 -- 172.21.15.154:0/125101614 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc00c160 con 0x7f95cc003970 2023-09-23T03:24:28.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.149+0000 7f95e4871640 1 --2- 172.21.15.154:0/125101614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95cc005040 0x7f95cc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:28.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.149+0000 7f95dffff640 1 --2- 172.21.15.154:0/125101614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95cc003970 0x7f95cc003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:28.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.149+0000 7f95d7fff640 1 --2- 172.21.15.154:0/125101614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95cc004370 0x7f95cc004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:28.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.149+0000 7f95e4871640 1 -- 172.21.15.154:0/125101614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95cc003970 msgr2=0x7f95cc003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:28.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.149+0000 7f95e4871640 1 --2- 172.21.15.154:0/125101614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95cc003970 0x7f95cc003d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:28.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.149+0000 7f95e4871640 1 -- 172.21.15.154:0/125101614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95cc004370 msgr2=0x7f95cc004840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:28.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.149+0000 7f95e4871640 1 --2- 172.21.15.154:0/125101614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95cc004370 0x7f95cc004840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:28.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.149+0000 7f95e4871640 1 -- 172.21.15.154:0/125101614 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95cc003520 con 0x7f95cc005040 2023-09-23T03:24:28.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.150+0000 7f95e4871640 1 -- 172.21.15.154:0/125101614 reap_dead start 2023-09-23T03:24:28.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.150+0000 7f95e4871640 1 --2- 172.21.15.154:0/125101614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95cc005040 0x7f95cc00b980 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f95e006e1c0 tx=0x7f95cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d60332017fa83cf server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:28.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.151+0000 7f95df7fe640 1 -- 172.21.15.154:0/125101614 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d802cbd0 con 0x7f95cc005040 2023-09-23T03:24:28.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.151+0000 7f95df7fe640 1 -- 172.21.15.154:0/125101614 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d800f040 con 0x7f95cc005040 2023-09-23T03:24:28.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.151+0000 7f95df7fe640 1 -- 172.21.15.154:0/125101614 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d8003990 con 0x7f95cc005040 2023-09-23T03:24:28.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.151+0000 7f95e62d4640 1 -- 172.21.15.154:0/125101614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95cc005040 msgr2=0x7f95cc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:28.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.151+0000 7f95e62d4640 1 --2- 172.21.15.154:0/125101614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95cc005040 0x7f95cc00b980 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f95e006e1c0 tx=0x7f95cc0057f0 comp rx=0 tx=0).stop 2023-09-23T03:24:28.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.151+0000 7f95e62d4640 1 -- 172.21.15.154:0/125101614 shutdown_connections 2023-09-23T03:24:28.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.151+0000 7f95e62d4640 1 --2- 172.21.15.154:0/125101614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95cc005040 0x7f95cc00b980 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:28.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.151+0000 7f95e62d4640 1 -- 172.21.15.154:0/125101614 >> 172.21.15.154:0/125101614 conn(0x7f95e006c010 msgr2=0x7f95e006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:28.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.151+0000 7f95e62d4640 1 -- 172.21.15.154:0/125101614 shutdown_connections 2023-09-23T03:24:28.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.151+0000 7f95e62d4640 1 -- 172.21.15.154:0/125101614 wait complete. 2023-09-23T03:24:28.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.153+0000 7f95e62d4640 1 Processor -- start 2023-09-23T03:24:28.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.153+0000 7f95e62d4640 1 -- start start 2023-09-23T03:24:28.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.154+0000 7f95e62d4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95cc005040 0x7f95e00b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:28.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.154+0000 7f95e62d4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95e01f3e40 0x7f95e00b0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:28.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.154+0000 7f95e62d4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95e00b0bd0 0x7f95e014b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:28.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.154+0000 7f95e62d4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e014c7f0 con 0x7f95cc005040 2023-09-23T03:24:28.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.154+0000 7f95e62d4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e00b1050 con 0x7f95e00b0bd0 2023-09-23T03:24:28.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.154+0000 7f95e62d4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95e00b2cb0 con 0x7f95e01f3e40 2023-09-23T03:24:28.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.154+0000 7f95dffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95cc005040 0x7f95e00b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:28.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.155+0000 7f95dffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95cc005040 0x7f95e00b26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46866/0 (socket says 172.21.15.154:46866) 2023-09-23T03:24:28.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.155+0000 7f95dffff640 1 -- 172.21.15.154:0/3431643139 learned_addr learned my addr 172.21.15.154:0/3431643139 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:28.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.155+0000 7f95dffff640 1 -- 172.21.15.154:0/3431643139 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95e01f3e40 msgr2=0x7f95e00b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:28.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.155+0000 7f95dffff640 1 --2- 172.21.15.154:0/3431643139 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95e01f3e40 0x7f95e00b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:28.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.155+0000 7f95dffff640 1 -- 172.21.15.154:0/3431643139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95e00b0bd0 msgr2=0x7f95e014b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:28.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.155+0000 7f95dffff640 1 --2- 172.21.15.154:0/3431643139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95e00b0bd0 0x7f95e014b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:28.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.155+0000 7f95dffff640 1 -- 172.21.15.154:0/3431643139 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e00b2f90 con 0x7f95cc005040 2023-09-23T03:24:28.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.155+0000 7f95dffff640 1 --2- 172.21.15.154:0/3431643139 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95cc005040 0x7f95e00b26b0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f95d00072c0 tx=0x7f95d0007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:28.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.155+0000 7f95dd7fa640 1 -- 172.21.15.154:0/3431643139 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d000f960 con 0x7f95cc005040 2023-09-23T03:24:28.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.156+0000 7f95dd7fa640 1 -- 172.21.15.154:0/3431643139 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d000fb00 con 0x7f95cc005040 2023-09-23T03:24:28.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.156+0000 7f95dd7fa640 1 -- 172.21.15.154:0/3431643139 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d000feb0 con 0x7f95cc005040 2023-09-23T03:24:28.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.156+0000 7f95e62d4640 1 -- 172.21.15.154:0/3431643139 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e00b3270 con 0x7f95cc005040 2023-09-23T03:24:28.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.156+0000 7f95e62d4640 1 -- 172.21.15.154:0/3431643139 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e00b3830 con 0x7f95cc005040 2023-09-23T03:24:28.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.157+0000 7f95e62d4640 1 -- 172.21.15.154:0/3431643139 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e004e920 con 0x7f95cc005040 2023-09-23T03:24:28.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.158+0000 7f95dd7fa640 1 -- 172.21.15.154:0/3431643139 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f95d001d460 con 0x7f95cc005040 2023-09-23T03:24:28.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.158+0000 7f95dd7fa640 1 --2- 172.21.15.154:0/3431643139 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f95c006af60 0x7f95c006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:28.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.159+0000 7f95dd7fa640 1 -- 172.21.15.154:0/3431643139 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f95d0089bd0 con 0x7f95cc005040 2023-09-23T03:24:28.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.159+0000 7f95d7fff640 1 --2- 172.21.15.154:0/3431643139 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f95c006af60 0x7f95c006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:28.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.159+0000 7f95d7fff640 1 --2- 172.21.15.154:0/3431643139 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f95c006af60 0x7f95c006d420 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f95e0071680 tx=0x7f95c8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:28.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.163+0000 7f95dd7fa640 1 -- 172.21.15.154:0/3431643139 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f95d0054890 con 0x7f95cc005040 2023-09-23T03:24:28.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.351+0000 7f95e62d4640 1 -- 172.21.15.154:0/3431643139 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7f95e0072e80 con 0x7f95cc005040 2023-09-23T03:24:28.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.352+0000 7f95dd7fa640 1 -- 172.21.15.154:0/3431643139 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v86) v1 ==== 106+0+11 (secure 0 0 0) 0x7f95d0057eb0 con 0x7f95cc005040 2023-09-23T03:24:28.352 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:24:28.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.355+0000 7f95e62d4640 1 -- 172.21.15.154:0/3431643139 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f95c006af60 msgr2=0x7f95c006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:28.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.355+0000 7f95e62d4640 1 --2- 172.21.15.154:0/3431643139 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f95c006af60 0x7f95c006d420 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f95e0071680 tx=0x7f95c8009040 comp rx=0 tx=0).stop 2023-09-23T03:24:28.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.355+0000 7f95e62d4640 1 -- 172.21.15.154:0/3431643139 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95cc005040 msgr2=0x7f95e00b26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:28.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.355+0000 7f95e62d4640 1 --2- 172.21.15.154:0/3431643139 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95cc005040 0x7f95e00b26b0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f95d00072c0 tx=0x7f95d0007780 comp rx=0 tx=0).stop 2023-09-23T03:24:28.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.355+0000 7f95e62d4640 1 -- 172.21.15.154:0/3431643139 shutdown_connections 2023-09-23T03:24:28.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.355+0000 7f95e62d4640 1 --2- 172.21.15.154:0/3431643139 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f95c006af60 0x7f95c006d420 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:28.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.355+0000 7f95e62d4640 1 --2- 172.21.15.154:0/3431643139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95e00b0bd0 0x7f95e014b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:28.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.355+0000 7f95e62d4640 1 --2- 172.21.15.154:0/3431643139 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95e01f3e40 0x7f95e00b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:28.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.355+0000 7f95e62d4640 1 --2- 172.21.15.154:0/3431643139 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95cc005040 0x7f95e00b26b0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:28.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.355+0000 7f95e62d4640 1 -- 172.21.15.154:0/3431643139 >> 172.21.15.154:0/3431643139 conn(0x7f95e006c010 msgr2=0x7f95e00c4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:28.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.355+0000 7f95e62d4640 1 -- 172.21.15.154:0/3431643139 shutdown_connections 2023-09-23T03:24:28.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.355+0000 7f95e62d4640 1 -- 172.21.15.154:0/3431643139 wait complete. 2023-09-23T03:24:28.375 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-23T03:24:28.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.593+0000 7f4484de7640 1 Processor -- start 2023-09-23T03:24:28.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.593+0000 7f4484de7640 1 -- start start 2023-09-23T03:24:28.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.594+0000 7f4484de7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44800cf040 0x7f44801656e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:28.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.595+0000 7f4484de7640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f448015ac30 con 0x7f44800acce0 2023-09-23T03:24:28.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.595+0000 7f4484de7640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44800b1f70 con 0x7f44800b1b20 2023-09-23T03:24:28.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.595+0000 7f4484de7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44800b2e90 con 0x7f44800cf040 2023-09-23T03:24:28.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.595+0000 7f447e59c640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f44800acce0 0x7f448015ed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:39042/0 (socket says 172.21.15.154:39042) 2023-09-23T03:24:28.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.595+0000 7f447e59c640 1 -- 172.21.15.154:0/4111992738 learned_addr learned my addr 172.21.15.154:0/4111992738 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:24:28.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.595+0000 7f447ed9d640 1 --2- 172.21.15.154:0/4111992738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44800cf040 0x7f44801656e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:28.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.595+0000 7f447d59a640 1 -- 172.21.15.154:0/4111992738 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3609534856 0 0) 0x7f4470001290 con 0x7f44800acce0 2023-09-23T03:24:28.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.595+0000 7f447d59a640 1 -- 172.21.15.154:0/4111992738 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4468003520 con 0x7f44800acce0 2023-09-23T03:24:28.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.596+0000 7f447d59a640 1 -- 172.21.15.154:0/4111992738 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2454437031 0 0) 0x7f4468003520 con 0x7f44800acce0 2023-09-23T03:24:28.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.596+0000 7f447d59a640 1 -- 172.21.15.154:0/4111992738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44800cf040 msgr2=0x7f44801656e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:28.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.596+0000 7f447d59a640 1 --2- 172.21.15.154:0/4111992738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44800cf040 0x7f44801656e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:28.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.596+0000 7f447d59a640 1 -- 172.21.15.154:0/4111992738 >> v1:172.21.15.190:6789/0 conn(0x7f44800b1b20 legacy=0x7f44800ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:28.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.596+0000 7f447d59a640 1 -- 172.21.15.154:0/4111992738 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44800b2210 con 0x7f44800acce0 2023-09-23T03:24:28.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.596+0000 7f447d59a640 1 -- 172.21.15.154:0/4111992738 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f4470001980 con 0x7f44800acce0 2023-09-23T03:24:28.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.596+0000 7f447d59a640 1 -- 172.21.15.154:0/4111992738 >> v1:172.21.15.154:6789/0 conn(0x7f44800acce0 legacy=0x7f448015ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:28.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.596+0000 7f447d59a640 1 --2- 172.21.15.154:0/4111992738 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44680039a0 0x7f4468003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:28.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.596+0000 7f447d59a640 1 --2- 172.21.15.154:0/4111992738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44800cf040 0x7f44680047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:28.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.596+0000 7f447d59a640 1 --2- 172.21.15.154:0/4111992738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4468004d00 0x7f4468005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:28.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.596+0000 7f447d59a640 1 -- 172.21.15.154:0/4111992738 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44800b2e90 con 0x7f44680039a0 2023-09-23T03:24:28.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.596+0000 7f447d59a640 1 -- 172.21.15.154:0/4111992738 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446800bb40 con 0x7f4468004d00 2023-09-23T03:24:28.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.596+0000 7f447d59a640 1 -- 172.21.15.154:0/4111992738 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f446800bd00 con 0x7f44800cf040 2023-09-23T03:24:28.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.596+0000 7f447e59c640 1 --2- 172.21.15.154:0/4111992738 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44680039a0 0x7f4468003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:28.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.597+0000 7f447e59c640 1 -- 172.21.15.154:0/4111992738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44800cf040 msgr2=0x7f44680047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:28.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.597+0000 7f447dd9b640 1 --2- 172.21.15.154:0/4111992738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44800cf040 0x7f44680047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:28.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.597+0000 7f447e59c640 1 --2- 172.21.15.154:0/4111992738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44800cf040 0x7f44680047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:28.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.597+0000 7f447e59c640 1 -- 172.21.15.154:0/4111992738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4468004d00 msgr2=0x7f4468005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:28.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.597+0000 7f447e59c640 1 --2- 172.21.15.154:0/4111992738 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4468004d00 0x7f4468005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:28.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.597+0000 7f447e59c640 1 -- 172.21.15.154:0/4111992738 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4468003520 con 0x7f44680039a0 2023-09-23T03:24:28.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.597+0000 7f447dd9b640 1 --2- 172.21.15.154:0/4111992738 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44800cf040 0x7f44680047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:28.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.598+0000 7f447e59c640 1 --2- 172.21.15.154:0/4111992738 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44680039a0 0x7f4468003dc0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f4470007fd0 tx=0x7f44680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e7dee17218226 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:28.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.598+0000 7f447d59a640 1 -- 172.21.15.154:0/4111992738 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470007270 con 0x7f44680039a0 2023-09-23T03:24:28.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.598+0000 7f447d59a640 1 -- 172.21.15.154:0/4111992738 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44700073f0 con 0x7f44680039a0 2023-09-23T03:24:28.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.598+0000 7f447d59a640 1 -- 172.21.15.154:0/4111992738 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447002c7c0 con 0x7f44680039a0 2023-09-23T03:24:28.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.599+0000 7f4484de7640 1 -- 172.21.15.154:0/4111992738 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44680039a0 msgr2=0x7f4468003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:28.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.599+0000 7f4484de7640 1 --2- 172.21.15.154:0/4111992738 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44680039a0 0x7f4468003dc0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f4470007fd0 tx=0x7f44680057f0 comp rx=0 tx=0).stop 2023-09-23T03:24:28.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.599+0000 7f447ed9d640 1 -- 172.21.15.154:0/4111992738 reap_dead start 2023-09-23T03:24:28.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.599+0000 7f4484de7640 1 -- 172.21.15.154:0/4111992738 shutdown_connections 2023-09-23T03:24:28.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.599+0000 7f4484de7640 1 -- 172.21.15.154:0/4111992738 >> 172.21.15.154:0/4111992738 conn(0x7f448006a0f0 msgr2=0x7f448006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:28.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.599+0000 7f4484de7640 1 -- 172.21.15.154:0/4111992738 shutdown_connections 2023-09-23T03:24:28.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.599+0000 7f4484de7640 1 -- 172.21.15.154:0/4111992738 wait complete. 2023-09-23T03:24:28.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.601+0000 7f4484de7640 1 Processor -- start 2023-09-23T03:24:28.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.601+0000 7f4484de7640 1 -- start start 2023-09-23T03:24:28.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.602+0000 7f4484de7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44680039a0 0x7f4480158480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:28.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.602+0000 7f4484de7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44800af600 0x7f4480070f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:28.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.602+0000 7f4484de7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44800afa80 0x7f4480071470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:28.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.602+0000 7f4484de7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448015c490 con 0x7f44800afa80 2023-09-23T03:24:28.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.602+0000 7f4484de7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448015b910 con 0x7f44680039a0 2023-09-23T03:24:28.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.602+0000 7f4484de7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f448015bbf0 con 0x7f44800af600 2023-09-23T03:24:28.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.602+0000 7f447ed9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44800afa80 0x7f4480071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:28.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.602+0000 7f447ed9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44800afa80 0x7f4480071470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46886/0 (socket says 172.21.15.154:46886) 2023-09-23T03:24:28.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.602+0000 7f447ed9d640 1 -- 172.21.15.154:0/748330002 learned_addr learned my addr 172.21.15.154:0/748330002 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:28.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.603+0000 7f447dd9b640 1 --2- 172.21.15.154:0/748330002 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44800af600 0x7f4480070f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:28.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.603+0000 7f447e59c640 1 --2- 172.21.15.154:0/748330002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44680039a0 0x7f4480158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:28.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.603+0000 7f447ed9d640 1 -- 172.21.15.154:0/748330002 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44800af600 msgr2=0x7f4480070f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:28.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.603+0000 7f447ed9d640 1 --2- 172.21.15.154:0/748330002 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44800af600 0x7f4480070f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:28.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.603+0000 7f447ed9d640 1 -- 172.21.15.154:0/748330002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44680039a0 msgr2=0x7f4480158480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:28.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.603+0000 7f447ed9d640 1 --2- 172.21.15.154:0/748330002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44680039a0 0x7f4480158480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:28.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.603+0000 7f447ed9d640 1 -- 172.21.15.154:0/748330002 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f448015bed0 con 0x7f44800afa80 2023-09-23T03:24:28.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.603+0000 7f447e59c640 1 --2- 172.21.15.154:0/748330002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44680039a0 0x7f4480158480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:28.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.603+0000 7f447ed9d640 1 --2- 172.21.15.154:0/748330002 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44800afa80 0x7f4480071470 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f44800c9140 tx=0x7f447800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:28.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.603+0000 7f446effd640 1 -- 172.21.15.154:0/748330002 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4478010040 con 0x7f44800afa80 2023-09-23T03:24:28.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.603+0000 7f446effd640 1 -- 172.21.15.154:0/748330002 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447800bae0 con 0x7f44800afa80 2023-09-23T03:24:28.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.603+0000 7f446effd640 1 -- 172.21.15.154:0/748330002 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447800be70 con 0x7f44800afa80 2023-09-23T03:24:28.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.603+0000 7f4484de7640 1 -- 172.21.15.154:0/748330002 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f448015c1b0 con 0x7f44800afa80 2023-09-23T03:24:28.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.603+0000 7f4484de7640 1 -- 172.21.15.154:0/748330002 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4480096670 con 0x7f44800afa80 2023-09-23T03:24:28.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.605+0000 7f4484de7640 1 -- 172.21.15.154:0/748330002 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4480073eb0 con 0x7f44800afa80 2023-09-23T03:24:28.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.606+0000 7f446effd640 1 -- 172.21.15.154:0/748330002 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f447801d400 con 0x7f44800afa80 2023-09-23T03:24:28.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.606+0000 7f446effd640 1 --2- 172.21.15.154:0/748330002 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f446006af60 0x7f446006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:28.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.607+0000 7f446effd640 1 -- 172.21.15.154:0/748330002 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f4478089ff0 con 0x7f44800afa80 2023-09-23T03:24:28.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.607+0000 7f447e59c640 1 --2- 172.21.15.154:0/748330002 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f446006af60 0x7f446006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:28.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.608+0000 7f447e59c640 1 --2- 172.21.15.154:0/748330002 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f446006af60 0x7f446006d420 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4470034000 tx=0x7f4470009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:28.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.611+0000 7f446effd640 1 -- 172.21.15.154:0/748330002 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f44780bca50 con 0x7f44800afa80 2023-09-23T03:24:28.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.803+0000 7f4484de7640 1 -- 172.21.15.154:0/748330002 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f44800ae180 con 0x7f44800afa80 2023-09-23T03:24:28.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.803+0000 7f446effd640 1 -- 172.21.15.154:0/748330002 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v86) v1 ==== 80+0+0 (secure 0 0 0) 0x7f44780587f0 con 0x7f44800afa80 2023-09-23T03:24:28.804 INFO:teuthology.orchestra.run.smithi154.stderr:listed 0 entries 2023-09-23T03:24:28.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.806+0000 7f4484de7640 1 -- 172.21.15.154:0/748330002 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f446006af60 msgr2=0x7f446006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:28.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.806+0000 7f4484de7640 1 --2- 172.21.15.154:0/748330002 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f446006af60 0x7f446006d420 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4470034000 tx=0x7f4470009030 comp rx=0 tx=0).stop 2023-09-23T03:24:28.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.806+0000 7f4484de7640 1 -- 172.21.15.154:0/748330002 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44800afa80 msgr2=0x7f4480071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:28.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.806+0000 7f4484de7640 1 --2- 172.21.15.154:0/748330002 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44800afa80 0x7f4480071470 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f44800c9140 tx=0x7f447800b480 comp rx=0 tx=0).stop 2023-09-23T03:24:28.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.806+0000 7f4484de7640 1 -- 172.21.15.154:0/748330002 shutdown_connections 2023-09-23T03:24:28.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.806+0000 7f4484de7640 1 --2- 172.21.15.154:0/748330002 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f446006af60 0x7f446006d420 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:28.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.806+0000 7f4484de7640 1 --2- 172.21.15.154:0/748330002 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f44800afa80 0x7f4480071470 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:28.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.806+0000 7f4484de7640 1 --2- 172.21.15.154:0/748330002 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f44800af600 0x7f4480070f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:28.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.806+0000 7f4484de7640 1 --2- 172.21.15.154:0/748330002 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f44680039a0 0x7f4480158480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:28.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.806+0000 7f4484de7640 1 -- 172.21.15.154:0/748330002 >> 172.21.15.154:0/748330002 conn(0x7f448006a0f0 msgr2=0x7f448015a150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:28.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.806+0000 7f4484de7640 1 -- 172.21.15.154:0/748330002 shutdown_connections 2023-09-23T03:24:28.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:28.806+0000 7f4484de7640 1 -- 172.21.15.154:0/748330002 wait complete. 2023-09-23T03:24:28.824 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi154.front.sepia.ceph.com... 2023-09-23T03:24:28.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:24:28.824 DEBUG:teuthology.orchestra.run.smithi154:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-09-23T03:24:28.909 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:ceph-fuse[115807]: fuse finished with error 0 and tester_r 0 2023-09-23T03:24:28.915 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:24:35.017 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2023-09-23T03:24:35.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:24:35.018 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-09-23T03:24:35.081 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:24:35.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.300+0000 7f7d57fff640 1 Processor -- start 2023-09-23T03:24:35.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.300+0000 7f7d57fff640 1 -- start start 2023-09-23T03:24:35.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.302+0000 7f7d57fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d58071870 0x7f7d5806bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:35.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.302+0000 7f7d57fff640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d58159a00 con 0x7f7d580c4970 2023-09-23T03:24:35.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.302+0000 7f7d57fff640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d580cbe90 con 0x7f7d5806c2a0 2023-09-23T03:24:35.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.302+0000 7f7d57fff640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5806c710 con 0x7f7d58071870 2023-09-23T03:24:35.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.302+0000 7f7d56ffd640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f7d580c4970 0x7f7d580ca260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:45532/0 (socket says 172.21.15.154:45532) 2023-09-23T03:24:35.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.302+0000 7f7d56ffd640 1 -- 172.21.15.154:0/3879063205 learned_addr learned my addr 172.21.15.154:0/3879063205 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:24:35.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.302+0000 7f7d567fc640 1 -- 172.21.15.154:0/3879063205 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2792138021 0 0) 0x7f7d58159a00 con 0x7f7d580c4970 2023-09-23T03:24:35.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.303+0000 7f7d567fc640 1 -- 172.21.15.154:0/3879063205 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d44003520 con 0x7f7d580c4970 2023-09-23T03:24:35.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.303+0000 7f7d4ffff640 1 --2- 172.21.15.154:0/3879063205 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d58071870 0x7f7d5806bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:35.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.303+0000 7f7d567fc640 1 -- 172.21.15.154:0/3879063205 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4108085235 0 0) 0x7f7d44003520 con 0x7f7d580c4970 2023-09-23T03:24:35.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.303+0000 7f7d567fc640 1 -- 172.21.15.154:0/3879063205 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d58071870 msgr2=0x7f7d5806bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:35.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.303+0000 7f7d567fc640 1 --2- 172.21.15.154:0/3879063205 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d58071870 0x7f7d5806bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:35.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.303+0000 7f7d567fc640 1 -- 172.21.15.154:0/3879063205 >> v1:172.21.15.190:6789/0 conn(0x7f7d5806c2a0 legacy=0x7f7d580ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:35.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.303+0000 7f7d567fc640 1 -- 172.21.15.154:0/3879063205 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d58071f70 con 0x7f7d580c4970 2023-09-23T03:24:35.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.303+0000 7f7d567fc640 1 -- 172.21.15.154:0/3879063205 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f7d48001670 con 0x7f7d580c4970 2023-09-23T03:24:35.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.303+0000 7f7d567fc640 1 -- 172.21.15.154:0/3879063205 >> v1:172.21.15.154:6789/0 conn(0x7f7d580c4970 legacy=0x7f7d580ca260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:35.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.303+0000 7f7d567fc640 1 --2- 172.21.15.154:0/3879063205 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d58071870 0x7f7d440039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:35.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.303+0000 7f7d567fc640 1 --2- 172.21.15.154:0/3879063205 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d44003fd0 0x7f7d440047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:35.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.303+0000 7f7d567fc640 1 --2- 172.21.15.154:0/3879063205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d44004d00 0x7f7d44005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:35.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.303+0000 7f7d567fc640 1 -- 172.21.15.154:0/3879063205 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5806c710 con 0x7f7d44003fd0 2023-09-23T03:24:35.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.303+0000 7f7d567fc640 1 -- 172.21.15.154:0/3879063205 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4400bb40 con 0x7f7d44004d00 2023-09-23T03:24:35.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.303+0000 7f7d567fc640 1 -- 172.21.15.154:0/3879063205 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4400bd00 con 0x7f7d58071870 2023-09-23T03:24:35.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.303+0000 7f7d4ffff640 1 --2- 172.21.15.154:0/3879063205 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d44003fd0 0x7f7d440047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:35.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.304+0000 7f7d56ffd640 1 --2- 172.21.15.154:0/3879063205 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d58071870 0x7f7d440039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:35.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.304+0000 7f7d4ffff640 1 -- 172.21.15.154:0/3879063205 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d58071870 msgr2=0x7f7d440039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:35.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.304+0000 7f7d4ffff640 1 --2- 172.21.15.154:0/3879063205 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d58071870 0x7f7d440039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:35.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.304+0000 7f7d4ffff640 1 -- 172.21.15.154:0/3879063205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d44004d00 msgr2=0x7f7d44005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:35.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.304+0000 7f7d4ffff640 1 --2- 172.21.15.154:0/3879063205 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d44004d00 0x7f7d44005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:35.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.304+0000 7f7d4ffff640 1 -- 172.21.15.154:0/3879063205 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d44003520 con 0x7f7d44003fd0 2023-09-23T03:24:35.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.305+0000 7f7d4ffff640 1 --2- 172.21.15.154:0/3879063205 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d44003fd0 0x7f7d440047c0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f7d4000ba30 tx=0x7f7d440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a6cc055ca19834 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:35.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.305+0000 7f7d567fc640 1 -- 172.21.15.154:0/3879063205 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d40032640 con 0x7f7d44003fd0 2023-09-23T03:24:35.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.305+0000 7f7d567fc640 1 -- 172.21.15.154:0/3879063205 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d400327a0 con 0x7f7d44003fd0 2023-09-23T03:24:35.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.305+0000 7f7d567fc640 1 -- 172.21.15.154:0/3879063205 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d40032b30 con 0x7f7d44003fd0 2023-09-23T03:24:35.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.306+0000 7f7d57fff640 1 -- 172.21.15.154:0/3879063205 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d44003fd0 msgr2=0x7f7d440047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:35.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.306+0000 7f7d57fff640 1 --2- 172.21.15.154:0/3879063205 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d44003fd0 0x7f7d440047c0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f7d4000ba30 tx=0x7f7d440057f0 comp rx=0 tx=0).stop 2023-09-23T03:24:35.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.306+0000 7f7d577fe640 1 -- 172.21.15.154:0/3879063205 reap_dead start 2023-09-23T03:24:35.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.306+0000 7f7d57fff640 1 -- 172.21.15.154:0/3879063205 shutdown_connections 2023-09-23T03:24:35.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.306+0000 7f7d57fff640 1 -- 172.21.15.154:0/3879063205 >> 172.21.15.154:0/3879063205 conn(0x7f7d5806a390 msgr2=0x7f7d580afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:35.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.306+0000 7f7d57fff640 1 -- 172.21.15.154:0/3879063205 shutdown_connections 2023-09-23T03:24:35.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.306+0000 7f7d57fff640 1 -- 172.21.15.154:0/3879063205 wait complete. 2023-09-23T03:24:35.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.308+0000 7f7d57fff640 1 Processor -- start 2023-09-23T03:24:35.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.308+0000 7f7d57fff640 1 -- start start 2023-09-23T03:24:35.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.309+0000 7f7d57fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d44003fd0 0x7f7d5814ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:35.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.309+0000 7f7d57fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d58150be0 0x7f7d581ece20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:35.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.309+0000 7f7d57fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d58151060 0x7f7d581ed360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:35.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.309+0000 7f7d57fff640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d580c8220 con 0x7f7d58151060 2023-09-23T03:24:35.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.309+0000 7f7d57fff640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5814f1e0 con 0x7f7d44003fd0 2023-09-23T03:24:35.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.309+0000 7f7d57fff640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d581515d0 con 0x7f7d58150be0 2023-09-23T03:24:35.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.309+0000 7f7d577fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d58151060 0x7f7d581ed360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:35.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.309+0000 7f7d577fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d58151060 0x7f7d581ed360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:42928/0 (socket says 172.21.15.154:42928) 2023-09-23T03:24:35.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.309+0000 7f7d577fe640 1 -- 172.21.15.154:0/1207116152 learned_addr learned my addr 172.21.15.154:0/1207116152 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:35.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.309+0000 7f7d56ffd640 1 --2- 172.21.15.154:0/1207116152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d44003fd0 0x7f7d5814ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:35.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.309+0000 7f7d4ffff640 1 --2- 172.21.15.154:0/1207116152 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d58150be0 0x7f7d581ece20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:35.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.310+0000 7f7d577fe640 1 -- 172.21.15.154:0/1207116152 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d58150be0 msgr2=0x7f7d581ece20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:35.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.310+0000 7f7d577fe640 1 --2- 172.21.15.154:0/1207116152 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d58150be0 0x7f7d581ece20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:35.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.310+0000 7f7d577fe640 1 -- 172.21.15.154:0/1207116152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d44003fd0 msgr2=0x7f7d5814ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:35.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.310+0000 7f7d577fe640 1 --2- 172.21.15.154:0/1207116152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d44003fd0 0x7f7d5814ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:35.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.310+0000 7f7d577fe640 1 -- 172.21.15.154:0/1207116152 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d581518b0 con 0x7f7d58151060 2023-09-23T03:24:35.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.310+0000 7f7d577fe640 1 --2- 172.21.15.154:0/1207116152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d58151060 0x7f7d581ed360 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f7d5806dfd0 tx=0x7f7d5000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:35.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.310+0000 7f7d4f7fe640 1 -- 172.21.15.154:0/1207116152 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d50005d40 con 0x7f7d58151060 2023-09-23T03:24:35.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.310+0000 7f7d4f7fe640 1 -- 172.21.15.154:0/1207116152 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d5000ced0 con 0x7f7d58151060 2023-09-23T03:24:35.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.310+0000 7f7d57fff640 1 -- 172.21.15.154:0/1207116152 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d58151b90 con 0x7f7d58151060 2023-09-23T03:24:35.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.310+0000 7f7d4f7fe640 1 -- 172.21.15.154:0/1207116152 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5000a850 con 0x7f7d58151060 2023-09-23T03:24:35.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.310+0000 7f7d57fff640 1 -- 172.21.15.154:0/1207116152 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d581ed8a0 con 0x7f7d58151060 2023-09-23T03:24:35.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.311+0000 7f7d57fff640 1 -- 172.21.15.154:0/1207116152 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d580736b0 con 0x7f7d58151060 2023-09-23T03:24:35.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.313+0000 7f7d4f7fe640 1 -- 172.21.15.154:0/1207116152 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f7d5000a9b0 con 0x7f7d58151060 2023-09-23T03:24:35.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.313+0000 7f7d4f7fe640 1 --2- 172.21.15.154:0/1207116152 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7d3806af60 0x7f7d3806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:35.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.314+0000 7f7d4f7fe640 1 -- 172.21.15.154:0/1207116152 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f7d50088f30 con 0x7f7d58151060 2023-09-23T03:24:35.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.314+0000 7f7d56ffd640 1 --2- 172.21.15.154:0/1207116152 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7d3806af60 0x7f7d3806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:35.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.314+0000 7f7d56ffd640 1 --2- 172.21.15.154:0/1207116152 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7d3806af60 0x7f7d3806d420 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7d48001200 tx=0x7f7d48009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:35.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.318+0000 7f7d4f7fe640 1 -- 172.21.15.154:0/1207116152 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f7d50053c90 con 0x7f7d58151060 2023-09-23T03:24:35.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.548+0000 7f7d57fff640 1 -- 172.21.15.154:0/1207116152 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d580cc120 con 0x7f7d58151060 2023-09-23T03:24:35.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.549+0000 7f7d4f7fe640 1 -- 172.21.15.154:0/1207116152 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f7d50017020 con 0x7f7d58151060 2023-09-23T03:24:35.550 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:35.550 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":63,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/728936136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":728936136},{"type":"v1","addr":"172.21.15.190:6833","nonce":728936136}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":5890,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2277229238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2277229238},{"type":"v1","addr":"172.21.15.190:6835","nonce":2277229238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":7257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/4245928089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4245928089},{"type":"v1","addr":"172.21.15.154:6835","nonce":4245928089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:24:10.061474+0000","modified":"2023-09-23T03:24:21.160286+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7260":{"gid":7260,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.154:6837/2213283923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":2213283923},{"type":"v1","addr":"172.21.15.154:6837","nonce":2213283923}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12,13],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-09-23T03:24:35.550 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 63 2023-09-23T03:24:35.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.552+0000 7f7d57fff640 1 -- 172.21.15.154:0/1207116152 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7d3806af60 msgr2=0x7f7d3806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:35.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.552+0000 7f7d57fff640 1 --2- 172.21.15.154:0/1207116152 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7d3806af60 0x7f7d3806d420 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7d48001200 tx=0x7f7d48009040 comp rx=0 tx=0).stop 2023-09-23T03:24:35.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.552+0000 7f7d57fff640 1 -- 172.21.15.154:0/1207116152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d58151060 msgr2=0x7f7d581ed360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:35.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.552+0000 7f7d57fff640 1 --2- 172.21.15.154:0/1207116152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d58151060 0x7f7d581ed360 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f7d5806dfd0 tx=0x7f7d5000cb90 comp rx=0 tx=0).stop 2023-09-23T03:24:35.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.552+0000 7f7d57fff640 1 -- 172.21.15.154:0/1207116152 shutdown_connections 2023-09-23T03:24:35.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.552+0000 7f7d57fff640 1 --2- 172.21.15.154:0/1207116152 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7d3806af60 0x7f7d3806d420 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:35.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.552+0000 7f7d57fff640 1 --2- 172.21.15.154:0/1207116152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d58151060 0x7f7d581ed360 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:35.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.552+0000 7f7d57fff640 1 --2- 172.21.15.154:0/1207116152 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7d58150be0 0x7f7d581ece20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:35.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.552+0000 7f7d57fff640 1 --2- 172.21.15.154:0/1207116152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7d44003fd0 0x7f7d5814ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:35.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.552+0000 7f7d57fff640 1 -- 172.21.15.154:0/1207116152 >> 172.21.15.154:0/1207116152 conn(0x7f7d5806a390 msgr2=0x7f7d580afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:35.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.552+0000 7f7d57fff640 1 -- 172.21.15.154:0/1207116152 shutdown_connections 2023-09-23T03:24:35.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:35.552+0000 7f7d57fff640 1 -- 172.21.15.154:0/1207116152 wait complete. 2023-09-23T03:24:35.570 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2023-09-23T03:24:35.806 INFO:teuthology.orchestra.run.smithi154.stdout:{"message":"","return_code":0} 2023-09-23T03:24:35.807 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-09-23T03:24:35.807 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-09-23T03:24:36.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.021+0000 7f122a6d1640 1 Processor -- start 2023-09-23T03:24:36.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.021+0000 7f122a6d1640 1 -- start start 2023-09-23T03:24:36.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.023+0000 7f122a6d1640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f122415eb00 0x7f122406fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:36.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.023+0000 7f122a6d1640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1224070300 0x7f12240ca500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:36.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.023+0000 7f122a6d1640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1224070730 con 0x7f122406c7a0 2023-09-23T03:24:36.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.023+0000 7f122a6d1640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1224155070 con 0x7f122415eb00 2023-09-23T03:24:36.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.023+0000 7f122a6d1640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f122415e0f0 con 0x7f1224070300 2023-09-23T03:24:36.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.023+0000 7f1228ece640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f122415eb00 0x7f122406fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:36.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.023+0000 7f12296cf640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f122406c7a0 0x7f122415ab00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:45542/0 (socket says 172.21.15.154:45542) 2023-09-23T03:24:36.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.023+0000 7f12296cf640 1 -- 172.21.15.154:0/2441611563 learned_addr learned my addr 172.21.15.154:0/2441611563 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:24:36.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.024+0000 7f1228ece640 1 -- 172.21.15.154:0/2441611563 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1224070300 msgr2=0x7f12240ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:36.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.024+0000 7f1229ed0640 1 --2- 172.21.15.154:0/2441611563 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1224070300 0x7f12240ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:36.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.024+0000 7f1228ece640 1 --2- 172.21.15.154:0/2441611563 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1224070300 0x7f12240ca500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:36.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.024+0000 7f1228ece640 1 -- 172.21.15.154:0/2441611563 >> v1:172.21.15.154:6789/0 conn(0x7f122406c7a0 legacy=0x7f122415ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:36.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.024+0000 7f121bfff640 1 -- 172.21.15.154:0/2441611563 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3899723582 0 0) 0x7f1224070730 con 0x7f122406c7a0 2023-09-23T03:24:36.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.024+0000 7f1228ece640 1 -- 172.21.15.154:0/2441611563 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f122406bff0 con 0x7f122415eb00 2023-09-23T03:24:36.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.025+0000 7f1228ece640 1 --2- 172.21.15.154:0/2441611563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f122415eb00 0x7f122406fdc0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f122415e230 tx=0x7f121402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b4dee62c5b8814 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:36.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.025+0000 7f121bfff640 1 -- 172.21.15.154:0/2441611563 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121402d040 con 0x7f122415eb00 2023-09-23T03:24:36.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.025+0000 7f121bfff640 1 -- 172.21.15.154:0/2441611563 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1214031810 con 0x7f122415eb00 2023-09-23T03:24:36.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.025+0000 7f121bfff640 1 -- 172.21.15.154:0/2441611563 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1214031ba0 con 0x7f122415eb00 2023-09-23T03:24:36.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.026+0000 7f122a6d1640 1 -- 172.21.15.154:0/2441611563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f122415eb00 msgr2=0x7f122406fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:36.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.026+0000 7f122a6d1640 1 --2- 172.21.15.154:0/2441611563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f122415eb00 0x7f122406fdc0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f122415e230 tx=0x7f121402b8b0 comp rx=0 tx=0).stop 2023-09-23T03:24:36.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.026+0000 7f122a6d1640 1 -- 172.21.15.154:0/2441611563 shutdown_connections 2023-09-23T03:24:36.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.026+0000 7f122a6d1640 1 --2- 172.21.15.154:0/2441611563 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1224070300 0x7f12240ca500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:36.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.026+0000 7f122a6d1640 1 --2- 172.21.15.154:0/2441611563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f122415eb00 0x7f122406fdc0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:36.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.026+0000 7f122a6d1640 1 -- 172.21.15.154:0/2441611563 >> 172.21.15.154:0/2441611563 conn(0x7f122406a390 msgr2=0x7f12240afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:36.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.026+0000 7f122a6d1640 1 -- 172.21.15.154:0/2441611563 shutdown_connections 2023-09-23T03:24:36.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.026+0000 7f122a6d1640 1 -- 172.21.15.154:0/2441611563 wait complete. 2023-09-23T03:24:36.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.028+0000 7f122a6d1640 1 Processor -- start 2023-09-23T03:24:36.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.028+0000 7f122a6d1640 1 -- start start 2023-09-23T03:24:36.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.029+0000 7f122a6d1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f122406c7a0 0x7f12240c7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:36.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.029+0000 7f122a6d1640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1224070300 0x7f12241f17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:36.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.029+0000 7f122a6d1640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f122415eb00 0x7f1224159c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:36.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.029+0000 7f122a6d1640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12240c7c90 con 0x7f122406c7a0 2023-09-23T03:24:36.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.029+0000 7f122a6d1640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12240c72c0 con 0x7f122415eb00 2023-09-23T03:24:36.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.029+0000 7f122a6d1640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12240c8190 con 0x7f1224070300 2023-09-23T03:24:36.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.029+0000 7f12296cf640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f122406c7a0 0x7f12240c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:36.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.029+0000 7f12296cf640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f122406c7a0 0x7f12240c7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:42940/0 (socket says 172.21.15.154:42940) 2023-09-23T03:24:36.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.029+0000 7f12296cf640 1 -- 172.21.15.154:0/3534862268 learned_addr learned my addr 172.21.15.154:0/3534862268 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:36.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.029+0000 7f12296cf640 1 -- 172.21.15.154:0/3534862268 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1224070300 msgr2=0x7f12241f17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:36.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.029+0000 7f1229ed0640 1 --2- 172.21.15.154:0/3534862268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f122415eb00 0x7f1224159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:36.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.029+0000 7f1228ece640 1 --2- 172.21.15.154:0/3534862268 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1224070300 0x7f12241f17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:36.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.030+0000 7f12296cf640 1 --2- 172.21.15.154:0/3534862268 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1224070300 0x7f12241f17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:36.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.030+0000 7f12296cf640 1 -- 172.21.15.154:0/3534862268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f122415eb00 msgr2=0x7f1224159c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:36.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.030+0000 7f12296cf640 1 --2- 172.21.15.154:0/3534862268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f122415eb00 0x7f1224159c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:36.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.030+0000 7f12296cf640 1 -- 172.21.15.154:0/3534862268 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1214009400 con 0x7f122406c7a0 2023-09-23T03:24:36.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.030+0000 7f1229ed0640 1 --2- 172.21.15.154:0/3534862268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f122415eb00 0x7f1224159c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:36.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.030+0000 7f12296cf640 1 --2- 172.21.15.154:0/3534862268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f122406c7a0 0x7f12240c7750 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f122000c4a0 tx=0x7f122000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:36.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.030+0000 7f1219ffb640 1 -- 172.21.15.154:0/3534862268 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1220005ce0 con 0x7f122406c7a0 2023-09-23T03:24:36.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.030+0000 7f1219ffb640 1 -- 172.21.15.154:0/3534862268 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1220005e40 con 0x7f122406c7a0 2023-09-23T03:24:36.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.030+0000 7f1219ffb640 1 -- 172.21.15.154:0/3534862268 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12200072b0 con 0x7f122406c7a0 2023-09-23T03:24:36.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.030+0000 7f122a6d1640 1 -- 172.21.15.154:0/3534862268 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12240c66b0 con 0x7f122406c7a0 2023-09-23T03:24:36.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.030+0000 7f122a6d1640 1 -- 172.21.15.154:0/3534862268 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12240c6c70 con 0x7f122406c7a0 2023-09-23T03:24:36.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.032+0000 7f122a6d1640 1 -- 172.21.15.154:0/3534862268 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12240736b0 con 0x7f122406c7a0 2023-09-23T03:24:36.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.032+0000 7f1219ffb640 1 -- 172.21.15.154:0/3534862268 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f1220007410 con 0x7f122406c7a0 2023-09-23T03:24:36.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.033+0000 7f1219ffb640 1 --2- 172.21.15.154:0/3534862268 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f120806af30 0x7f120806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:36.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.033+0000 7f1219ffb640 1 -- 172.21.15.154:0/3534862268 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f12200890c0 con 0x7f122406c7a0 2023-09-23T03:24:36.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.034+0000 7f1228ece640 1 --2- 172.21.15.154:0/3534862268 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f120806af30 0x7f120806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:36.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.034+0000 7f1228ece640 1 --2- 172.21.15.154:0/3534862268 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f120806af30 0x7f120806d3f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f1214036000 tx=0x7f121403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:36.038 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.038+0000 7f1219ffb640 1 -- 172.21.15.154:0/3534862268 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f1220053d80 con 0x7f122406c7a0 2023-09-23T03:24:36.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.267+0000 7f122a6d1640 1 -- 172.21.15.154:0/3534862268 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f122404e920 con 0x7f122406c7a0 2023-09-23T03:24:36.268 INFO:tasks.ceph.mon.a.smithi154.stderr:2023-09-23T03:24:36.268+0000 7f3df35a6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-23T03:24:36.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.271+0000 7f1219ffb640 1 -- 172.21.15.154:0/3534862268 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v64) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1220017090 con 0x7f122406c7a0 2023-09-23T03:24:36.272 INFO:teuthology.orchestra.run.smithi154.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-09-23T03:24:36.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.275+0000 7f122a6d1640 1 -- 172.21.15.154:0/3534862268 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f120806af30 msgr2=0x7f120806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:36.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.275+0000 7f122a6d1640 1 --2- 172.21.15.154:0/3534862268 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f120806af30 0x7f120806d3f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f1214036000 tx=0x7f121403c040 comp rx=0 tx=0).stop 2023-09-23T03:24:36.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.275+0000 7f122a6d1640 1 -- 172.21.15.154:0/3534862268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f122406c7a0 msgr2=0x7f12240c7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:36.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.275+0000 7f122a6d1640 1 --2- 172.21.15.154:0/3534862268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f122406c7a0 0x7f12240c7750 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f122000c4a0 tx=0x7f122000c960 comp rx=0 tx=0).stop 2023-09-23T03:24:36.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.276+0000 7f122a6d1640 1 -- 172.21.15.154:0/3534862268 shutdown_connections 2023-09-23T03:24:36.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.276+0000 7f122a6d1640 1 --2- 172.21.15.154:0/3534862268 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f120806af30 0x7f120806d3f0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:36.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.276+0000 7f122a6d1640 1 --2- 172.21.15.154:0/3534862268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f122415eb00 0x7f1224159c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:36.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.276+0000 7f122a6d1640 1 --2- 172.21.15.154:0/3534862268 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1224070300 0x7f12241f17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:36.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.276+0000 7f122a6d1640 1 --2- 172.21.15.154:0/3534862268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f122406c7a0 0x7f12240c7750 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:36.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.276+0000 7f122a6d1640 1 -- 172.21.15.154:0/3534862268 >> 172.21.15.154:0/3534862268 conn(0x7f122406a390 msgr2=0x7f12241547e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:36.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.276+0000 7f122a6d1640 1 -- 172.21.15.154:0/3534862268 shutdown_connections 2023-09-23T03:24:36.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.276+0000 7f122a6d1640 1 -- 172.21.15.154:0/3534862268 wait complete. 2023-09-23T03:24:36.285 INFO:tasks.ceph.mds.a.smithi154.stderr: -3617> 2023-09-23T03:24:11.551+0000 7fe1a6776640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-23T03:24:36.297 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-23T03:24:36.302 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-23T03:24:36.341 INFO:tasks.ceph.mds.a.smithi154.stdout:starting mds.a at 2023-09-23T03:24:36.442 INFO:teuthology.orchestra.run.smithi154.stdout:[global] 2023-09-23T03:24:36.442 INFO:teuthology.orchestra.run.smithi154.stdout: chdir = "" 2023-09-23T03:24:36.442 INFO:teuthology.orchestra.run.smithi154.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-23T03:24:36.442 INFO:teuthology.orchestra.run.smithi154.stdout: auth supported = cephx 2023-09-23T03:24:36.442 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.442 INFO:teuthology.orchestra.run.smithi154.stdout: filestore xattr use omap = true 2023-09-23T03:24:36.443 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.443 INFO:teuthology.orchestra.run.smithi154.stdout: mon clock drift allowed = 1.000 2023-09-23T03:24:36.443 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.443 INFO:teuthology.orchestra.run.smithi154.stdout: osd crush chooseleaf type = 0 2023-09-23T03:24:36.443 INFO:teuthology.orchestra.run.smithi154.stdout: auth debug = true 2023-09-23T03:24:36.443 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.443 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on old message = true 2023-09-23T03:24:36.443 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on bug = true 2023-09-23T03:24:36.443 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.444 INFO:teuthology.orchestra.run.smithi154.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-23T03:24:36.444 INFO:teuthology.orchestra.run.smithi154.stdout: mon pg warn max object skew = 0 2023-09-23T03:24:36.444 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.444 INFO:teuthology.orchestra.run.smithi154.stdout: # disable pg_autoscaler by default for new pools 2023-09-23T03:24:36.444 INFO:teuthology.orchestra.run.smithi154.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-23T03:24:36.444 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.444 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default size = 2 2023-09-23T03:24:36.444 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.444 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow primary affinity = true 2023-09-23T03:24:36.444 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow pg remap = true 2023-09-23T03:24:36.444 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on legacy crush tunables = false 2023-09-23T03:24:36.445 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on crush straw calc version zero = false 2023-09-23T03:24:36.445 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on no sortbitwise = false 2023-09-23T03:24:36.445 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on osd down out interval zero = false 2023-09-23T03:24:36.445 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on too few osds = false 2023-09-23T03:24:36.445 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-23T03:24:36.445 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-23T03:24:36.445 INFO:teuthology.orchestra.run.smithi154.stdout: mon_allow_pool_size_one = true 2023-09-23T03:24:36.445 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.445 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-23T03:24:36.445 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.445 INFO:teuthology.orchestra.run.smithi154.stdout: osd default data pool replay window = 5 2023-09-23T03:24:36.445 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.446 INFO:teuthology.orchestra.run.smithi154.stdout: mon allow pool delete = true 2023-09-23T03:24:36.446 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.446 INFO:teuthology.orchestra.run.smithi154.stdout: mon cluster log file level = debug 2023-09-23T03:24:36.446 INFO:teuthology.orchestra.run.smithi154.stdout: debug asserts on shutdown = true 2023-09-23T03:24:36.446 INFO:teuthology.orchestra.run.smithi154.stdout: mon health detail to clog = false 2023-09-23T03:24:36.446 INFO:teuthology.orchestra.run.smithi154.stdout: mon host = "172.21.15.154,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]" 2023-09-23T03:24:36.446 INFO:teuthology.orchestra.run.smithi154.stdout: fsid = 8c8039d3-6c84-43db-8808-85e0dbdd47c8 2023-09-23T03:24:36.446 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.446 INFO:teuthology.orchestra.run.smithi154.stdout:[osd] 2023-09-23T03:24:36.446 INFO:teuthology.orchestra.run.smithi154.stdout: osd journal size = 100 2023-09-23T03:24:36.446 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.446 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub load threshold = 5.0 2023-09-23T03:24:36.447 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub max interval = 600 2023-09-23T03:24:36.447 INFO:teuthology.orchestra.run.smithi154.stdout: osd mclock profile = high_recovery_ops 2023-09-23T03:24:36.447 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.447 INFO:teuthology.orchestra.run.smithi154.stdout: osd recover clone overlap = true 2023-09-23T03:24:36.447 INFO:teuthology.orchestra.run.smithi154.stdout: osd recovery max chunk = 1048576 2023-09-23T03:24:36.447 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.447 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug shutdown = true 2023-09-23T03:24:36.447 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug op order = true 2023-09-23T03:24:36.447 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug verify stray on activate = true 2023-09-23T03:24:36.447 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.447 INFO:teuthology.orchestra.run.smithi154.stdout: osd open classes on start = true 2023-09-23T03:24:36.447 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug pg log writeout = true 2023-09-23T03:24:36.447 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.448 INFO:teuthology.orchestra.run.smithi154.stdout: osd deep scrub update digest min age = 30 2023-09-23T03:24:36.448 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.448 INFO:teuthology.orchestra.run.smithi154.stdout: osd map max advance = 10 2023-09-23T03:24:36.448 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.448 INFO:teuthology.orchestra.run.smithi154.stdout: journal zero on create = true 2023-09-23T03:24:36.448 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.448 INFO:teuthology.orchestra.run.smithi154.stdout: filestore ondisk finisher threads = 3 2023-09-23T03:24:36.448 INFO:teuthology.orchestra.run.smithi154.stdout: filestore apply finisher threads = 3 2023-09-23T03:24:36.448 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.448 INFO:teuthology.orchestra.run.smithi154.stdout: bdev debug aio = true 2023-09-23T03:24:36.448 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug misdirected ops = true 2023-09-23T03:24:36.448 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore block size = 96636764160 2023-09-23T03:24:36.449 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore fsck on mount = True 2023-09-23T03:24:36.449 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluefs = 20 2023-09-23T03:24:36.449 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluestore = 20 2023-09-23T03:24:36.449 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:24:36.449 INFO:teuthology.orchestra.run.smithi154.stdout: debug osd = 20 2023-09-23T03:24:36.449 INFO:teuthology.orchestra.run.smithi154.stdout: debug rocksdb = 10 2023-09-23T03:24:36.449 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd backfillfull_ratio = 0.85 2023-09-23T03:24:36.449 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd full ratio = 0.9 2023-09-23T03:24:36.449 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd nearfull ratio = 0.8 2023-09-23T03:24:36.449 INFO:teuthology.orchestra.run.smithi154.stdout: osd failsafe full ratio = 0.95 2023-09-23T03:24:36.449 INFO:teuthology.orchestra.run.smithi154.stdout: osd objectstore = bluestore 2023-09-23T03:24:36.449 INFO:teuthology.orchestra.run.smithi154.stdout: osd op complaint time = 180 2023-09-23T03:24:36.449 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.450 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr] 2023-09-23T03:24:36.450 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:24:36.450 INFO:teuthology.orchestra.run.smithi154.stdout: debug mgr = 20 2023-09-23T03:24:36.450 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2023-09-23T03:24:36.450 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2023-09-23T03:24:36.450 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:24:36.450 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:24:36.450 INFO:teuthology.orchestra.run.smithi154.stdout: mgr/telemetry/nag = false 2023-09-23T03:24:36.450 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.450 INFO:teuthology.orchestra.run.smithi154.stdout:[mon] 2023-09-23T03:24:36.450 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:24:36.450 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2023-09-23T03:24:36.451 INFO:teuthology.orchestra.run.smithi154.stdout: debug paxos = 20 2023-09-23T03:24:36.451 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2023-09-23T03:24:36.451 INFO:teuthology.orchestra.run.smithi154.stdout: mon data avail warn = 5 2023-09-23T03:24:36.451 INFO:teuthology.orchestra.run.smithi154.stdout: mon mgr mkfs grace = 240 2023-09-23T03:24:36.451 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:24:36.451 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd reporter subtree level = osd 2023-09-23T03:24:36.451 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd prime pg temp = true 2023-09-23T03:24:36.451 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:24:36.451 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.451 INFO:teuthology.orchestra.run.smithi154.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-23T03:24:36.451 INFO:teuthology.orchestra.run.smithi154.stdout: auth mon ticket ttl = 660 # 11m 2023-09-23T03:24:36.451 INFO:teuthology.orchestra.run.smithi154.stdout: auth service ticket ttl = 240 # 4m 2023-09-23T03:24:36.451 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.452 INFO:teuthology.orchestra.run.smithi154.stdout: # don't complain about insecure global_id in the test suite 2023-09-23T03:24:36.452 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-23T03:24:36.452 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-23T03:24:36.452 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.452 INFO:teuthology.orchestra.run.smithi154.stdout: # 1m isn't quite enough 2023-09-23T03:24:36.452 INFO:teuthology.orchestra.run.smithi154.stdout: mon_down_mkfs_grace = 2m 2023-09-23T03:24:36.452 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.452 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_filestore_osds = false 2023-09-23T03:24:36.452 INFO:teuthology.orchestra.run.smithi154.stdout: mon op complaint time = 120 2023-09-23T03:24:36.452 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.452 INFO:teuthology.orchestra.run.smithi154.stdout:[client] 2023-09-23T03:24:36.452 INFO:teuthology.orchestra.run.smithi154.stdout: rgw cache enabled = true 2023-09-23T03:24:36.452 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable ops log = true 2023-09-23T03:24:36.453 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable usage log = true 2023-09-23T03:24:36.453 INFO:teuthology.orchestra.run.smithi154.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-23T03:24:36.453 INFO:teuthology.orchestra.run.smithi154.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-23T03:24:36.453 INFO:teuthology.orchestra.run.smithi154.stdout: client mount timeout = 600 2023-09-23T03:24:36.453 INFO:teuthology.orchestra.run.smithi154.stdout: debug client = 20 2023-09-23T03:24:36.453 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:24:36.453 INFO:teuthology.orchestra.run.smithi154.stdout: pid file = "" 2023-09-23T03:24:36.453 INFO:teuthology.orchestra.run.smithi154.stdout: rados mon op timeout = 900 2023-09-23T03:24:36.453 INFO:teuthology.orchestra.run.smithi154.stdout: rados osd op timeout = 900 2023-09-23T03:24:36.453 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.a] 2023-09-23T03:24:36.453 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.b] 2023-09-23T03:24:36.454 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.c] 2023-09-23T03:24:36.454 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.a] 2023-09-23T03:24:36.454 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.c] 2023-09-23T03:24:36.454 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.b] 2023-09-23T03:24:36.454 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.d] 2023-09-23T03:24:36.454 INFO:teuthology.orchestra.run.smithi154.stdout:[mds] 2023-09-23T03:24:36.454 INFO:teuthology.orchestra.run.smithi154.stdout: debug mds = 20 2023-09-23T03:24:36.454 INFO:teuthology.orchestra.run.smithi154.stdout: debug mds balancer = 20 2023-09-23T03:24:36.454 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:24:36.454 INFO:teuthology.orchestra.run.smithi154.stdout: mds debug frag = True 2023-09-23T03:24:36.454 INFO:teuthology.orchestra.run.smithi154.stdout: mds debug scatterstat = False 2023-09-23T03:24:36.454 INFO:teuthology.orchestra.run.smithi154.stdout: mds op complaint time = 180 2023-09-23T03:24:36.455 INFO:teuthology.orchestra.run.smithi154.stdout: mds verify scatter = False 2023-09-23T03:24:36.455 INFO:teuthology.orchestra.run.smithi154.stdout: osd op complaint time = 180 2023-09-23T03:24:36.455 INFO:teuthology.orchestra.run.smithi154.stdout: rados mon op timeout = 900 2023-09-23T03:24:36.455 INFO:teuthology.orchestra.run.smithi154.stdout: rados osd op timeout = 900 2023-09-23T03:24:36.476 INFO:teuthology.orchestra.run.smithi190.stdout:[global] 2023-09-23T03:24:36.476 INFO:teuthology.orchestra.run.smithi190.stdout: chdir = "" 2023-09-23T03:24:36.476 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-23T03:24:36.476 INFO:teuthology.orchestra.run.smithi190.stdout: auth supported = cephx 2023-09-23T03:24:36.476 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.476 INFO:teuthology.orchestra.run.smithi190.stdout: filestore xattr use omap = true 2023-09-23T03:24:36.477 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.477 INFO:teuthology.orchestra.run.smithi190.stdout: mon clock drift allowed = 1.000 2023-09-23T03:24:36.477 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.477 INFO:teuthology.orchestra.run.smithi190.stdout: osd crush chooseleaf type = 0 2023-09-23T03:24:36.477 INFO:teuthology.orchestra.run.smithi190.stdout: auth debug = true 2023-09-23T03:24:36.477 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.477 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on old message = true 2023-09-23T03:24:36.477 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on bug = true 2023-09-23T03:24:36.477 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.477 INFO:teuthology.orchestra.run.smithi190.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-23T03:24:36.478 INFO:teuthology.orchestra.run.smithi190.stdout: mon pg warn max object skew = 0 2023-09-23T03:24:36.478 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.478 INFO:teuthology.orchestra.run.smithi190.stdout: # disable pg_autoscaler by default for new pools 2023-09-23T03:24:36.478 INFO:teuthology.orchestra.run.smithi190.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-23T03:24:36.478 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.478 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default size = 2 2023-09-23T03:24:36.478 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.478 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow primary affinity = true 2023-09-23T03:24:36.478 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow pg remap = true 2023-09-23T03:24:36.478 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on legacy crush tunables = false 2023-09-23T03:24:36.478 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on crush straw calc version zero = false 2023-09-23T03:24:36.478 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on no sortbitwise = false 2023-09-23T03:24:36.479 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on osd down out interval zero = false 2023-09-23T03:24:36.479 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on too few osds = false 2023-09-23T03:24:36.479 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-23T03:24:36.479 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-23T03:24:36.479 INFO:teuthology.orchestra.run.smithi190.stdout: mon_allow_pool_size_one = true 2023-09-23T03:24:36.479 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.479 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-23T03:24:36.479 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.479 INFO:teuthology.orchestra.run.smithi190.stdout: osd default data pool replay window = 5 2023-09-23T03:24:36.479 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.479 INFO:teuthology.orchestra.run.smithi190.stdout: mon allow pool delete = true 2023-09-23T03:24:36.479 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.479 INFO:teuthology.orchestra.run.smithi190.stdout: mon cluster log file level = debug 2023-09-23T03:24:36.480 INFO:teuthology.orchestra.run.smithi190.stdout: debug asserts on shutdown = true 2023-09-23T03:24:36.480 INFO:teuthology.orchestra.run.smithi190.stdout: mon health detail to clog = false 2023-09-23T03:24:36.480 INFO:teuthology.orchestra.run.smithi190.stdout: mon host = "172.21.15.154,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]" 2023-09-23T03:24:36.480 INFO:teuthology.orchestra.run.smithi190.stdout: fsid = 8c8039d3-6c84-43db-8808-85e0dbdd47c8 2023-09-23T03:24:36.480 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.480 INFO:teuthology.orchestra.run.smithi190.stdout:[osd] 2023-09-23T03:24:36.480 INFO:teuthology.orchestra.run.smithi190.stdout: osd journal size = 100 2023-09-23T03:24:36.480 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.480 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub load threshold = 5.0 2023-09-23T03:24:36.480 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub max interval = 600 2023-09-23T03:24:36.480 INFO:teuthology.orchestra.run.smithi190.stdout: osd mclock profile = high_recovery_ops 2023-09-23T03:24:36.480 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.480 INFO:teuthology.orchestra.run.smithi190.stdout: osd recover clone overlap = true 2023-09-23T03:24:36.481 INFO:teuthology.orchestra.run.smithi190.stdout: osd recovery max chunk = 1048576 2023-09-23T03:24:36.481 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.481 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug shutdown = true 2023-09-23T03:24:36.481 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug op order = true 2023-09-23T03:24:36.481 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug verify stray on activate = true 2023-09-23T03:24:36.481 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.481 INFO:teuthology.orchestra.run.smithi190.stdout: osd open classes on start = true 2023-09-23T03:24:36.481 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug pg log writeout = true 2023-09-23T03:24:36.481 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.481 INFO:teuthology.orchestra.run.smithi190.stdout: osd deep scrub update digest min age = 30 2023-09-23T03:24:36.481 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.481 INFO:teuthology.orchestra.run.smithi190.stdout: osd map max advance = 10 2023-09-23T03:24:36.482 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.482 INFO:teuthology.orchestra.run.smithi190.stdout: journal zero on create = true 2023-09-23T03:24:36.482 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.482 INFO:teuthology.orchestra.run.smithi190.stdout: filestore ondisk finisher threads = 3 2023-09-23T03:24:36.482 INFO:teuthology.orchestra.run.smithi190.stdout: filestore apply finisher threads = 3 2023-09-23T03:24:36.482 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.482 INFO:teuthology.orchestra.run.smithi190.stdout: bdev debug aio = true 2023-09-23T03:24:36.482 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug misdirected ops = true 2023-09-23T03:24:36.482 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore block size = 96636764160 2023-09-23T03:24:36.482 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore fsck on mount = True 2023-09-23T03:24:36.482 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluefs = 20 2023-09-23T03:24:36.482 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluestore = 20 2023-09-23T03:24:36.482 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:24:36.483 INFO:teuthology.orchestra.run.smithi190.stdout: debug osd = 20 2023-09-23T03:24:36.483 INFO:teuthology.orchestra.run.smithi190.stdout: debug rocksdb = 10 2023-09-23T03:24:36.483 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd backfillfull_ratio = 0.85 2023-09-23T03:24:36.483 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd full ratio = 0.9 2023-09-23T03:24:36.483 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd nearfull ratio = 0.8 2023-09-23T03:24:36.483 INFO:teuthology.orchestra.run.smithi190.stdout: osd failsafe full ratio = 0.95 2023-09-23T03:24:36.483 INFO:teuthology.orchestra.run.smithi190.stdout: osd objectstore = bluestore 2023-09-23T03:24:36.483 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2023-09-23T03:24:36.483 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.483 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr] 2023-09-23T03:24:36.483 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:24:36.483 INFO:teuthology.orchestra.run.smithi190.stdout: debug mgr = 20 2023-09-23T03:24:36.483 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2023-09-23T03:24:36.484 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2023-09-23T03:24:36.484 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:24:36.484 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:24:36.484 INFO:teuthology.orchestra.run.smithi190.stdout: mgr/telemetry/nag = false 2023-09-23T03:24:36.484 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.484 INFO:teuthology.orchestra.run.smithi190.stdout:[mon] 2023-09-23T03:24:36.484 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:24:36.484 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2023-09-23T03:24:36.484 INFO:teuthology.orchestra.run.smithi190.stdout: debug paxos = 20 2023-09-23T03:24:36.484 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2023-09-23T03:24:36.484 INFO:teuthology.orchestra.run.smithi190.stdout: mon data avail warn = 5 2023-09-23T03:24:36.484 INFO:teuthology.orchestra.run.smithi190.stdout: mon mgr mkfs grace = 240 2023-09-23T03:24:36.484 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:24:36.485 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd reporter subtree level = osd 2023-09-23T03:24:36.485 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd prime pg temp = true 2023-09-23T03:24:36.485 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:24:36.485 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.485 INFO:teuthology.orchestra.run.smithi190.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-23T03:24:36.485 INFO:teuthology.orchestra.run.smithi190.stdout: auth mon ticket ttl = 660 # 11m 2023-09-23T03:24:36.485 INFO:teuthology.orchestra.run.smithi190.stdout: auth service ticket ttl = 240 # 4m 2023-09-23T03:24:36.485 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.485 INFO:teuthology.orchestra.run.smithi190.stdout: # don't complain about insecure global_id in the test suite 2023-09-23T03:24:36.485 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-23T03:24:36.485 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-23T03:24:36.486 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.486 INFO:teuthology.orchestra.run.smithi190.stdout: # 1m isn't quite enough 2023-09-23T03:24:36.486 INFO:teuthology.orchestra.run.smithi190.stdout: mon_down_mkfs_grace = 2m 2023-09-23T03:24:36.486 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.486 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_filestore_osds = false 2023-09-23T03:24:36.486 INFO:teuthology.orchestra.run.smithi190.stdout: mon op complaint time = 120 2023-09-23T03:24:36.486 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.486 INFO:teuthology.orchestra.run.smithi190.stdout:[client] 2023-09-23T03:24:36.486 INFO:teuthology.orchestra.run.smithi190.stdout: rgw cache enabled = true 2023-09-23T03:24:36.486 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable ops log = true 2023-09-23T03:24:36.486 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable usage log = true 2023-09-23T03:24:36.486 INFO:teuthology.orchestra.run.smithi190.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-23T03:24:36.486 INFO:teuthology.orchestra.run.smithi190.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-23T03:24:36.487 INFO:teuthology.orchestra.run.smithi190.stdout: client mount timeout = 600 2023-09-23T03:24:36.487 INFO:teuthology.orchestra.run.smithi190.stdout: debug client = 20 2023-09-23T03:24:36.487 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:24:36.487 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = "" 2023-09-23T03:24:36.487 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 900 2023-09-23T03:24:36.487 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 900 2023-09-23T03:24:36.487 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.a] 2023-09-23T03:24:36.487 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.b] 2023-09-23T03:24:36.487 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.c] 2023-09-23T03:24:36.487 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.a] 2023-09-23T03:24:36.487 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.c] 2023-09-23T03:24:36.487 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.b] 2023-09-23T03:24:36.488 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.d] 2023-09-23T03:24:36.488 INFO:teuthology.orchestra.run.smithi190.stdout:[mds] 2023-09-23T03:24:36.488 INFO:teuthology.orchestra.run.smithi190.stdout: debug mds = 20 2023-09-23T03:24:36.488 INFO:teuthology.orchestra.run.smithi190.stdout: debug mds balancer = 20 2023-09-23T03:24:36.488 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:24:36.488 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug frag = True 2023-09-23T03:24:36.488 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug scatterstat = False 2023-09-23T03:24:36.488 INFO:teuthology.orchestra.run.smithi190.stdout: mds op complaint time = 180 2023-09-23T03:24:36.488 INFO:teuthology.orchestra.run.smithi190.stdout: mds verify scatter = False 2023-09-23T03:24:36.488 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2023-09-23T03:24:36.488 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 900 2023-09-23T03:24:36.488 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 900 2023-09-23T03:24:36.523 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-23T03:24:36.538 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-23T03:24:36.708 INFO:teuthology.orchestra.run.smithi154.stdout:[global] 2023-09-23T03:24:36.708 INFO:teuthology.orchestra.run.smithi154.stdout: chdir = "" 2023-09-23T03:24:36.708 INFO:teuthology.orchestra.run.smithi154.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-23T03:24:36.708 INFO:teuthology.orchestra.run.smithi154.stdout: auth supported = cephx 2023-09-23T03:24:36.708 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.708 INFO:teuthology.orchestra.run.smithi154.stdout: filestore xattr use omap = true 2023-09-23T03:24:36.709 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.709 INFO:teuthology.orchestra.run.smithi154.stdout: mon clock drift allowed = 1.000 2023-09-23T03:24:36.709 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.709 INFO:teuthology.orchestra.run.smithi154.stdout: osd crush chooseleaf type = 0 2023-09-23T03:24:36.709 INFO:teuthology.orchestra.run.smithi154.stdout: auth debug = true 2023-09-23T03:24:36.709 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.709 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on old message = true 2023-09-23T03:24:36.709 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on bug = true 2023-09-23T03:24:36.710 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.710 INFO:teuthology.orchestra.run.smithi154.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-23T03:24:36.710 INFO:teuthology.orchestra.run.smithi154.stdout: mon pg warn max object skew = 0 2023-09-23T03:24:36.710 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.710 INFO:teuthology.orchestra.run.smithi154.stdout: # disable pg_autoscaler by default for new pools 2023-09-23T03:24:36.710 INFO:teuthology.orchestra.run.smithi154.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-23T03:24:36.710 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.710 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default size = 2 2023-09-23T03:24:36.710 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.711 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow primary affinity = true 2023-09-23T03:24:36.711 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow pg remap = true 2023-09-23T03:24:36.711 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on legacy crush tunables = false 2023-09-23T03:24:36.711 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on crush straw calc version zero = false 2023-09-23T03:24:36.711 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on no sortbitwise = false 2023-09-23T03:24:36.711 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on osd down out interval zero = false 2023-09-23T03:24:36.711 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on too few osds = false 2023-09-23T03:24:36.711 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-23T03:24:36.711 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-23T03:24:36.712 INFO:teuthology.orchestra.run.smithi154.stdout: mon_allow_pool_size_one = true 2023-09-23T03:24:36.712 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.712 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-23T03:24:36.712 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.712 INFO:teuthology.orchestra.run.smithi154.stdout: osd default data pool replay window = 5 2023-09-23T03:24:36.712 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.712 INFO:teuthology.orchestra.run.smithi154.stdout: mon allow pool delete = true 2023-09-23T03:24:36.712 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.712 INFO:teuthology.orchestra.run.smithi154.stdout: mon cluster log file level = debug 2023-09-23T03:24:36.712 INFO:teuthology.orchestra.run.smithi154.stdout: debug asserts on shutdown = true 2023-09-23T03:24:36.712 INFO:teuthology.orchestra.run.smithi154.stdout: mon health detail to clog = false 2023-09-23T03:24:36.712 INFO:teuthology.orchestra.run.smithi154.stdout: mon host = "172.21.15.154,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]" 2023-09-23T03:24:36.713 INFO:teuthology.orchestra.run.smithi154.stdout: fsid = 8c8039d3-6c84-43db-8808-85e0dbdd47c8 2023-09-23T03:24:36.713 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.713 INFO:teuthology.orchestra.run.smithi154.stdout:[osd] 2023-09-23T03:24:36.713 INFO:teuthology.orchestra.run.smithi154.stdout: osd journal size = 100 2023-09-23T03:24:36.713 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.713 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub load threshold = 5.0 2023-09-23T03:24:36.713 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub max interval = 600 2023-09-23T03:24:36.713 INFO:teuthology.orchestra.run.smithi154.stdout: osd mclock profile = high_recovery_ops 2023-09-23T03:24:36.713 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.713 INFO:teuthology.orchestra.run.smithi154.stdout: osd recover clone overlap = true 2023-09-23T03:24:36.713 INFO:teuthology.orchestra.run.smithi154.stdout: osd recovery max chunk = 1048576 2023-09-23T03:24:36.713 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.713 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug shutdown = true 2023-09-23T03:24:36.714 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug op order = true 2023-09-23T03:24:36.714 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug verify stray on activate = true 2023-09-23T03:24:36.714 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.714 INFO:teuthology.orchestra.run.smithi154.stdout: osd open classes on start = true 2023-09-23T03:24:36.714 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug pg log writeout = true 2023-09-23T03:24:36.714 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.714 INFO:teuthology.orchestra.run.smithi154.stdout: osd deep scrub update digest min age = 30 2023-09-23T03:24:36.714 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.714 INFO:teuthology.orchestra.run.smithi154.stdout: osd map max advance = 10 2023-09-23T03:24:36.714 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.715 INFO:teuthology.orchestra.run.smithi154.stdout: journal zero on create = true 2023-09-23T03:24:36.715 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.715 INFO:teuthology.orchestra.run.smithi154.stdout: filestore ondisk finisher threads = 3 2023-09-23T03:24:36.715 INFO:teuthology.orchestra.run.smithi154.stdout: filestore apply finisher threads = 3 2023-09-23T03:24:36.715 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.715 INFO:teuthology.orchestra.run.smithi154.stdout: bdev debug aio = true 2023-09-23T03:24:36.715 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug misdirected ops = true 2023-09-23T03:24:36.715 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore block size = 96636764160 2023-09-23T03:24:36.715 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore fsck on mount = True 2023-09-23T03:24:36.715 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluefs = 20 2023-09-23T03:24:36.715 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluestore = 20 2023-09-23T03:24:36.715 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:24:36.716 INFO:teuthology.orchestra.run.smithi154.stdout: debug osd = 20 2023-09-23T03:24:36.716 INFO:teuthology.orchestra.run.smithi154.stdout: debug rocksdb = 10 2023-09-23T03:24:36.716 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd backfillfull_ratio = 0.85 2023-09-23T03:24:36.716 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd full ratio = 0.9 2023-09-23T03:24:36.716 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd nearfull ratio = 0.8 2023-09-23T03:24:36.716 INFO:teuthology.orchestra.run.smithi154.stdout: osd failsafe full ratio = 0.95 2023-09-23T03:24:36.716 INFO:teuthology.orchestra.run.smithi154.stdout: osd objectstore = bluestore 2023-09-23T03:24:36.716 INFO:teuthology.orchestra.run.smithi154.stdout: osd op complaint time = 180 2023-09-23T03:24:36.716 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.716 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr] 2023-09-23T03:24:36.716 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:24:36.716 INFO:teuthology.orchestra.run.smithi154.stdout: debug mgr = 20 2023-09-23T03:24:36.717 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2023-09-23T03:24:36.717 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2023-09-23T03:24:36.717 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:24:36.717 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:24:36.717 INFO:teuthology.orchestra.run.smithi154.stdout: mgr/telemetry/nag = false 2023-09-23T03:24:36.717 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.717 INFO:teuthology.orchestra.run.smithi154.stdout:[mon] 2023-09-23T03:24:36.717 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:24:36.717 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2023-09-23T03:24:36.717 INFO:teuthology.orchestra.run.smithi154.stdout: debug paxos = 20 2023-09-23T03:24:36.717 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2023-09-23T03:24:36.717 INFO:teuthology.orchestra.run.smithi154.stdout: mon data avail warn = 5 2023-09-23T03:24:36.717 INFO:teuthology.orchestra.run.smithi154.stdout: mon mgr mkfs grace = 240 2023-09-23T03:24:36.718 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:24:36.718 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd reporter subtree level = osd 2023-09-23T03:24:36.718 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd prime pg temp = true 2023-09-23T03:24:36.718 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:24:36.718 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.718 INFO:teuthology.orchestra.run.smithi154.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-23T03:24:36.718 INFO:teuthology.orchestra.run.smithi154.stdout: auth mon ticket ttl = 660 # 11m 2023-09-23T03:24:36.718 INFO:teuthology.orchestra.run.smithi154.stdout: auth service ticket ttl = 240 # 4m 2023-09-23T03:24:36.718 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.718 INFO:teuthology.orchestra.run.smithi154.stdout: # don't complain about insecure global_id in the test suite 2023-09-23T03:24:36.718 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-23T03:24:36.718 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-23T03:24:36.719 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.719 INFO:teuthology.orchestra.run.smithi154.stdout: # 1m isn't quite enough 2023-09-23T03:24:36.719 INFO:teuthology.orchestra.run.smithi154.stdout: mon_down_mkfs_grace = 2m 2023-09-23T03:24:36.719 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.719 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_filestore_osds = false 2023-09-23T03:24:36.719 INFO:teuthology.orchestra.run.smithi154.stdout: mon op complaint time = 120 2023-09-23T03:24:36.719 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:36.719 INFO:teuthology.orchestra.run.smithi154.stdout:[client] 2023-09-23T03:24:36.719 INFO:teuthology.orchestra.run.smithi154.stdout: rgw cache enabled = true 2023-09-23T03:24:36.719 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable ops log = true 2023-09-23T03:24:36.719 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable usage log = true 2023-09-23T03:24:36.719 INFO:teuthology.orchestra.run.smithi154.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-23T03:24:36.719 INFO:teuthology.orchestra.run.smithi154.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-23T03:24:36.720 INFO:teuthology.orchestra.run.smithi154.stdout: client mount timeout = 600 2023-09-23T03:24:36.720 INFO:teuthology.orchestra.run.smithi154.stdout: debug client = 20 2023-09-23T03:24:36.720 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:24:36.720 INFO:teuthology.orchestra.run.smithi154.stdout: pid file = "" 2023-09-23T03:24:36.720 INFO:teuthology.orchestra.run.smithi154.stdout: rados mon op timeout = 900 2023-09-23T03:24:36.720 INFO:teuthology.orchestra.run.smithi154.stdout: rados osd op timeout = 900 2023-09-23T03:24:36.720 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.a] 2023-09-23T03:24:36.720 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.b] 2023-09-23T03:24:36.720 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.c] 2023-09-23T03:24:36.720 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.a] 2023-09-23T03:24:36.720 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.c] 2023-09-23T03:24:36.720 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.b] 2023-09-23T03:24:36.720 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.d] 2023-09-23T03:24:36.721 INFO:teuthology.orchestra.run.smithi154.stdout:[mds] 2023-09-23T03:24:36.721 INFO:teuthology.orchestra.run.smithi154.stdout: debug mds = 20 2023-09-23T03:24:36.721 INFO:teuthology.orchestra.run.smithi154.stdout: debug mds balancer = 20 2023-09-23T03:24:36.721 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:24:36.721 INFO:teuthology.orchestra.run.smithi154.stdout: mds debug frag = True 2023-09-23T03:24:36.721 INFO:teuthology.orchestra.run.smithi154.stdout: mds debug scatterstat = False 2023-09-23T03:24:36.721 INFO:teuthology.orchestra.run.smithi154.stdout: mds op complaint time = 180 2023-09-23T03:24:36.721 INFO:teuthology.orchestra.run.smithi154.stdout: mds verify scatter = False 2023-09-23T03:24:36.721 INFO:teuthology.orchestra.run.smithi154.stdout: osd op complaint time = 180 2023-09-23T03:24:36.721 INFO:teuthology.orchestra.run.smithi154.stdout: rados mon op timeout = 900 2023-09-23T03:24:36.721 INFO:teuthology.orchestra.run.smithi154.stdout: rados osd op timeout = 900 2023-09-23T03:24:36.722 INFO:teuthology.orchestra.run.smithi190.stdout:[global] 2023-09-23T03:24:36.722 INFO:teuthology.orchestra.run.smithi190.stdout: chdir = "" 2023-09-23T03:24:36.722 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-23T03:24:36.722 INFO:teuthology.orchestra.run.smithi190.stdout: auth supported = cephx 2023-09-23T03:24:36.722 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.722 INFO:teuthology.orchestra.run.smithi190.stdout: filestore xattr use omap = true 2023-09-23T03:24:36.722 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.722 INFO:teuthology.orchestra.run.smithi190.stdout: mon clock drift allowed = 1.000 2023-09-23T03:24:36.722 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.723 INFO:teuthology.orchestra.run.smithi190.stdout: osd crush chooseleaf type = 0 2023-09-23T03:24:36.723 INFO:teuthology.orchestra.run.smithi190.stdout: auth debug = true 2023-09-23T03:24:36.723 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.723 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on old message = true 2023-09-23T03:24:36.723 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on bug = true 2023-09-23T03:24:36.723 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.723 INFO:teuthology.orchestra.run.smithi190.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-23T03:24:36.723 INFO:teuthology.orchestra.run.smithi190.stdout: mon pg warn max object skew = 0 2023-09-23T03:24:36.723 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.723 INFO:teuthology.orchestra.run.smithi190.stdout: # disable pg_autoscaler by default for new pools 2023-09-23T03:24:36.723 INFO:teuthology.orchestra.run.smithi190.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-23T03:24:36.724 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.724 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default size = 2 2023-09-23T03:24:36.724 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.724 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow primary affinity = true 2023-09-23T03:24:36.724 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow pg remap = true 2023-09-23T03:24:36.724 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on legacy crush tunables = false 2023-09-23T03:24:36.724 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on crush straw calc version zero = false 2023-09-23T03:24:36.724 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on no sortbitwise = false 2023-09-23T03:24:36.724 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on osd down out interval zero = false 2023-09-23T03:24:36.724 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on too few osds = false 2023-09-23T03:24:36.725 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-23T03:24:36.725 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-23T03:24:36.725 INFO:teuthology.orchestra.run.smithi190.stdout: mon_allow_pool_size_one = true 2023-09-23T03:24:36.725 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.725 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-23T03:24:36.725 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.725 INFO:teuthology.orchestra.run.smithi190.stdout: osd default data pool replay window = 5 2023-09-23T03:24:36.725 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.725 INFO:teuthology.orchestra.run.smithi190.stdout: mon allow pool delete = true 2023-09-23T03:24:36.726 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.726 INFO:teuthology.orchestra.run.smithi190.stdout: mon cluster log file level = debug 2023-09-23T03:24:36.726 INFO:teuthology.orchestra.run.smithi190.stdout: debug asserts on shutdown = true 2023-09-23T03:24:36.726 INFO:teuthology.orchestra.run.smithi190.stdout: mon health detail to clog = false 2023-09-23T03:24:36.726 INFO:teuthology.orchestra.run.smithi190.stdout: mon host = "172.21.15.154,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]" 2023-09-23T03:24:36.726 INFO:teuthology.orchestra.run.smithi190.stdout: fsid = 8c8039d3-6c84-43db-8808-85e0dbdd47c8 2023-09-23T03:24:36.726 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.726 INFO:teuthology.orchestra.run.smithi190.stdout:[osd] 2023-09-23T03:24:36.726 INFO:teuthology.orchestra.run.smithi190.stdout: osd journal size = 100 2023-09-23T03:24:36.727 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.727 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub load threshold = 5.0 2023-09-23T03:24:36.727 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub max interval = 600 2023-09-23T03:24:36.727 INFO:teuthology.orchestra.run.smithi190.stdout: osd mclock profile = high_recovery_ops 2023-09-23T03:24:36.727 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.727 INFO:teuthology.orchestra.run.smithi190.stdout: osd recover clone overlap = true 2023-09-23T03:24:36.727 INFO:teuthology.orchestra.run.smithi190.stdout: osd recovery max chunk = 1048576 2023-09-23T03:24:36.727 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.727 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug shutdown = true 2023-09-23T03:24:36.728 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug op order = true 2023-09-23T03:24:36.728 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug verify stray on activate = true 2023-09-23T03:24:36.728 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.728 INFO:teuthology.orchestra.run.smithi190.stdout: osd open classes on start = true 2023-09-23T03:24:36.728 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug pg log writeout = true 2023-09-23T03:24:36.728 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.728 INFO:teuthology.orchestra.run.smithi190.stdout: osd deep scrub update digest min age = 30 2023-09-23T03:24:36.728 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.728 INFO:teuthology.orchestra.run.smithi190.stdout: osd map max advance = 10 2023-09-23T03:24:36.728 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.729 INFO:teuthology.orchestra.run.smithi190.stdout: journal zero on create = true 2023-09-23T03:24:36.729 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.729 INFO:teuthology.orchestra.run.smithi190.stdout: filestore ondisk finisher threads = 3 2023-09-23T03:24:36.729 INFO:teuthology.orchestra.run.smithi190.stdout: filestore apply finisher threads = 3 2023-09-23T03:24:36.729 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.729 INFO:teuthology.orchestra.run.smithi190.stdout: bdev debug aio = true 2023-09-23T03:24:36.729 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug misdirected ops = true 2023-09-23T03:24:36.729 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore block size = 96636764160 2023-09-23T03:24:36.729 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore fsck on mount = True 2023-09-23T03:24:36.729 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluefs = 20 2023-09-23T03:24:36.730 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluestore = 20 2023-09-23T03:24:36.730 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:24:36.730 INFO:teuthology.orchestra.run.smithi190.stdout: debug osd = 20 2023-09-23T03:24:36.730 INFO:teuthology.orchestra.run.smithi190.stdout: debug rocksdb = 10 2023-09-23T03:24:36.730 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd backfillfull_ratio = 0.85 2023-09-23T03:24:36.730 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd full ratio = 0.9 2023-09-23T03:24:36.730 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd nearfull ratio = 0.8 2023-09-23T03:24:36.730 INFO:teuthology.orchestra.run.smithi190.stdout: osd failsafe full ratio = 0.95 2023-09-23T03:24:36.730 INFO:teuthology.orchestra.run.smithi190.stdout: osd objectstore = bluestore 2023-09-23T03:24:36.730 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2023-09-23T03:24:36.730 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.730 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr] 2023-09-23T03:24:36.731 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:24:36.731 INFO:teuthology.orchestra.run.smithi190.stdout: debug mgr = 20 2023-09-23T03:24:36.731 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2023-09-23T03:24:36.731 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2023-09-23T03:24:36.731 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:24:36.731 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:24:36.731 INFO:teuthology.orchestra.run.smithi190.stdout: mgr/telemetry/nag = false 2023-09-23T03:24:36.731 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.731 INFO:teuthology.orchestra.run.smithi190.stdout:[mon] 2023-09-23T03:24:36.732 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:24:36.732 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2023-09-23T03:24:36.732 INFO:teuthology.orchestra.run.smithi190.stdout: debug paxos = 20 2023-09-23T03:24:36.732 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2023-09-23T03:24:36.732 INFO:teuthology.orchestra.run.smithi190.stdout: mon data avail warn = 5 2023-09-23T03:24:36.732 INFO:teuthology.orchestra.run.smithi190.stdout: mon mgr mkfs grace = 240 2023-09-23T03:24:36.732 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:24:36.732 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd reporter subtree level = osd 2023-09-23T03:24:36.732 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd prime pg temp = true 2023-09-23T03:24:36.732 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:24:36.733 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.733 INFO:teuthology.orchestra.run.smithi190.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-23T03:24:36.733 INFO:teuthology.orchestra.run.smithi190.stdout: auth mon ticket ttl = 660 # 11m 2023-09-23T03:24:36.733 INFO:teuthology.orchestra.run.smithi190.stdout: auth service ticket ttl = 240 # 4m 2023-09-23T03:24:36.733 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.733 INFO:teuthology.orchestra.run.smithi190.stdout: # don't complain about insecure global_id in the test suite 2023-09-23T03:24:36.733 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-23T03:24:36.733 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-23T03:24:36.733 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.734 INFO:teuthology.orchestra.run.smithi190.stdout: # 1m isn't quite enough 2023-09-23T03:24:36.734 INFO:teuthology.orchestra.run.smithi190.stdout: mon_down_mkfs_grace = 2m 2023-09-23T03:24:36.734 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.734 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_filestore_osds = false 2023-09-23T03:24:36.734 INFO:teuthology.orchestra.run.smithi190.stdout: mon op complaint time = 120 2023-09-23T03:24:36.734 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:24:36.734 INFO:teuthology.orchestra.run.smithi190.stdout:[client] 2023-09-23T03:24:36.734 INFO:teuthology.orchestra.run.smithi190.stdout: rgw cache enabled = true 2023-09-23T03:24:36.734 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable ops log = true 2023-09-23T03:24:36.735 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable usage log = true 2023-09-23T03:24:36.735 INFO:teuthology.orchestra.run.smithi190.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-23T03:24:36.735 INFO:teuthology.orchestra.run.smithi190.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-23T03:24:36.735 INFO:teuthology.orchestra.run.smithi190.stdout: client mount timeout = 600 2023-09-23T03:24:36.735 INFO:teuthology.orchestra.run.smithi190.stdout: debug client = 20 2023-09-23T03:24:36.735 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:24:36.735 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = "" 2023-09-23T03:24:36.735 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 900 2023-09-23T03:24:36.735 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 900 2023-09-23T03:24:36.736 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.a] 2023-09-23T03:24:36.736 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.b] 2023-09-23T03:24:36.736 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.c] 2023-09-23T03:24:36.736 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.a] 2023-09-23T03:24:36.736 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.c] 2023-09-23T03:24:36.736 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.b] 2023-09-23T03:24:36.736 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.d] 2023-09-23T03:24:36.736 INFO:teuthology.orchestra.run.smithi190.stdout:[mds] 2023-09-23T03:24:36.736 INFO:teuthology.orchestra.run.smithi190.stdout: debug mds = 20 2023-09-23T03:24:36.737 INFO:teuthology.orchestra.run.smithi190.stdout: debug mds balancer = 20 2023-09-23T03:24:36.737 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:24:36.737 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug frag = True 2023-09-23T03:24:36.737 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug scatterstat = False 2023-09-23T03:24:36.737 INFO:teuthology.orchestra.run.smithi190.stdout: mds op complaint time = 180 2023-09-23T03:24:36.737 INFO:teuthology.orchestra.run.smithi190.stdout: mds verify scatter = False 2023-09-23T03:24:36.737 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2023-09-23T03:24:36.737 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 900 2023-09-23T03:24:36.738 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 900 2023-09-23T03:24:36.759 DEBUG:teuthology.orchestra.run.smithi154:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph purge cephfs_metadata --yes-i-really-really-mean-it 2023-09-23T03:24:36.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.854+0000 7f3763140f40 1 Processor -- start 2023-09-23T03:24:36.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.854+0000 7f3763140f40 1 -- start start 2023-09-23T03:24:36.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.854+0000 7f3763140f40 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x561d62e94590 0x561d62f27060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:36.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.854+0000 7f3763140f40 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d62f28a30 0x561d62f275a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:36.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.854+0000 7f3763140f40 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561d62f29950 0x561d62e821c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:36.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.855+0000 7f3763140f40 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d62d9b800 con 0x561d62f29950 2023-09-23T03:24:36.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.855+0000 7f3763140f40 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d62d9b4a0 con 0x561d62f28a30 2023-09-23T03:24:36.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.855+0000 7f3763140f40 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x561d62d8f4f0 con 0x561d62e94590 2023-09-23T03:24:36.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.855+0000 7f3761445640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561d62f29950 0x561d62e821c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:36.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.855+0000 7f3761445640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561d62f29950 0x561d62e821c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:42972/0 (socket says 172.21.15.154:42972) 2023-09-23T03:24:36.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.855+0000 7f3761445640 1 -- 172.21.15.154:0/442403367 learned_addr learned my addr 172.21.15.154:0/442403367 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:36.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.855+0000 7f3760c44640 1 --2- 172.21.15.154:0/442403367 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x561d62e94590 0x561d62f27060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:36.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.855+0000 7f3760443640 1 --2- 172.21.15.154:0/442403367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d62f28a30 0x561d62f275a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:36.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.855+0000 7f3761445640 1 -- 172.21.15.154:0/442403367 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x561d62e94590 msgr2=0x561d62f27060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:36.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.855+0000 7f3761445640 1 --2- 172.21.15.154:0/442403367 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x561d62e94590 0x561d62f27060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:36.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.855+0000 7f3761445640 1 -- 172.21.15.154:0/442403367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d62f28a30 msgr2=0x561d62f275a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:36.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.855+0000 7f3761445640 1 --2- 172.21.15.154:0/442403367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d62f28a30 0x561d62f275a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:36.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.855+0000 7f3761445640 1 -- 172.21.15.154:0/442403367 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3758005de0 con 0x561d62f29950 2023-09-23T03:24:36.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.856+0000 7f3761445640 1 --2- 172.21.15.154:0/442403367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561d62f29950 0x561d62e821c0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f375801d4a0 tx=0x7f375804c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faf36b1a1a2f58b6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:36.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.856+0000 7f375f441640 1 -- 172.21.15.154:0/442403367 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37580527b0 con 0x561d62f29950 2023-09-23T03:24:36.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.856+0000 7f375f441640 1 -- 172.21.15.154:0/442403367 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3758052910 con 0x561d62f29950 2023-09-23T03:24:36.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.856+0000 7f375f441640 1 -- 172.21.15.154:0/442403367 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3758052c80 con 0x561d62f29950 2023-09-23T03:24:36.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.857+0000 7f3763140f40 1 -- 172.21.15.154:0/442403367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561d62f29950 msgr2=0x561d62e821c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:36.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.857+0000 7f3763140f40 1 --2- 172.21.15.154:0/442403367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561d62f29950 0x561d62e821c0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f375801d4a0 tx=0x7f375804c070 comp rx=0 tx=0).stop 2023-09-23T03:24:36.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.857+0000 7f3763140f40 1 -- 172.21.15.154:0/442403367 shutdown_connections 2023-09-23T03:24:36.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.857+0000 7f3763140f40 1 --2- 172.21.15.154:0/442403367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561d62f29950 0x561d62e821c0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:36.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.857+0000 7f3763140f40 1 --2- 172.21.15.154:0/442403367 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d62f28a30 0x561d62f275a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:36.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.857+0000 7f3763140f40 1 --2- 172.21.15.154:0/442403367 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x561d62e94590 0x561d62f27060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:36.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.857+0000 7f3763140f40 1 -- 172.21.15.154:0/442403367 >> 172.21.15.154:0/442403367 conn(0x561d62e4e690 msgr2=0x561d62f27e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:36.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.857+0000 7f3763140f40 1 -- 172.21.15.154:0/442403367 shutdown_connections 2023-09-23T03:24:36.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.857+0000 7f3763140f40 1 -- 172.21.15.154:0/442403367 wait complete. 2023-09-23T03:24:36.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.858+0000 7f3763140f40 1 Processor -- start 2023-09-23T03:24:36.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.858+0000 7f3763140f40 1 -- start start 2023-09-23T03:24:36.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.859+0000 7f3763140f40 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x561d62e94590 0x561d62e841a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:36.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.859+0000 7f3763140f40 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d62f28a30 0x561d62e815a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:36.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.859+0000 7f3763140f40 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561d62f29950 0x561d62f23d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:36.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.859+0000 7f3763140f40 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d62f250d0 con 0x561d62f29950 2023-09-23T03:24:36.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.859+0000 7f3763140f40 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d62e81e20 con 0x561d62f28a30 2023-09-23T03:24:36.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.859+0000 7f3763140f40 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x561d62e81b40 con 0x561d62e94590 2023-09-23T03:24:36.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.859+0000 7f3761445640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561d62f29950 0x561d62f23d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:36.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.859+0000 7f3761445640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561d62f29950 0x561d62f23d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:42978/0 (socket says 172.21.15.154:42978) 2023-09-23T03:24:36.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.859+0000 7f3761445640 1 -- 172.21.15.154:0/2423586916 learned_addr learned my addr 172.21.15.154:0/2423586916 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:36.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.859+0000 7f3760c44640 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x561d62e94590 0x561d62e841a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:36.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.859+0000 7f3760443640 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d62f28a30 0x561d62e815a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:36.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.859+0000 7f3761445640 1 -- 172.21.15.154:0/2423586916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x561d62e94590 msgr2=0x561d62e841a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:36.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.859+0000 7f3761445640 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x561d62e94590 0x561d62e841a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:36.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.859+0000 7f3761445640 1 -- 172.21.15.154:0/2423586916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d62f28a30 msgr2=0x561d62e815a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:36.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.859+0000 7f3761445640 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d62f28a30 0x561d62e815a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:36.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.859+0000 7f3761445640 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d62dc1640 con 0x561d62f29950 2023-09-23T03:24:36.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.860+0000 7f3760443640 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561d62f28a30 0x561d62e815a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:36.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.860+0000 7f3761445640 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561d62f29950 0x561d62f23d00 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f375800a4f0 tx=0x561d62f2c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:36.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.860+0000 7f375e43f640 1 -- 172.21.15.154:0/2423586916 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375805c850 con 0x561d62f29950 2023-09-23T03:24:36.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.860+0000 7f375e43f640 1 -- 172.21.15.154:0/2423586916 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f375805c9b0 con 0x561d62f29950 2023-09-23T03:24:36.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.860+0000 7f375e43f640 1 -- 172.21.15.154:0/2423586916 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375805cd60 con 0x561d62f29950 2023-09-23T03:24:36.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.860+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561d62da7650 con 0x561d62f29950 2023-09-23T03:24:36.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.860+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d62d9a800 con 0x561d62f29950 2023-09-23T03:24:36.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.862+0000 7f375e43f640 1 -- 172.21.15.154:0/2423586916 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f37580653f0 con 0x561d62f29950 2023-09-23T03:24:36.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.863+0000 7f375e43f640 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f37400698e0 0x7f374006bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:36.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.863+0000 7f375e43f640 1 -- 172.21.15.154:0/2423586916 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6519+0+0 (secure 0 0 0) 0x7f3758053e40 con 0x561d62f29950 2023-09-23T03:24:36.866 INFO:teuthology.orchestra.run.smithi154.stdout:Warning: using slow linear search 2023-09-23T03:24:36.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.863+0000 7f3763140f40 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x561d62f2bed0 0x561d62e9b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:36.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.863+0000 7f3760c44640 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f37400698e0 0x7f374006bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:36.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.863+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:1 11.0 11:00000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x561d62e9bc80 con 0x561d62f2bed0 2023-09-23T03:24:36.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.864+0000 7f3760443640 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x561d62f2bed0 0x561d62e9b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:36.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.864+0000 7f3760c44640 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f37400698e0 0x7f374006bda0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3750006f90 tx=0x7f3750008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:36.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.864+0000 7f3760443640 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x561d62f2bed0 0x561d62e9b740 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:36.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.865+0000 7f3760443640 1 -- 172.21.15.154:0/2423586916 <== osd.1 v2:172.21.15.154:6816/1267710005 1 ==== osd_op_reply(1 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3748007610 con 0x561d62f2bed0 2023-09-23T03:24:36.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.865+0000 7f3760443640 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x7f3748006890 0x7f37480122a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:36.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.865+0000 7f3760443640 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:2 11.10 11:08000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x561d62e9bc80 con 0x7f3748006890 2023-09-23T03:24:36.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.865+0000 7f3760c44640 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x7f3748006890 0x7f37480122a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:36.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.866+0000 7f3760c44640 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x7f3748006890 0x7f37480122a0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:36.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.866+0000 7f3760c44640 1 -- 172.21.15.154:0/2423586916 <== osd.0 v2:172.21.15.154:6800/2920938138 1 ==== osd_op_reply(2 [pgnls start_epoch 87 out=177b] v86'10 uv10 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x7f3750006040 con 0x7f3748006890 2023-09-23T03:24:36.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.866+0000 7f3760c44640 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x7f3750005ac0 0x7f37500133f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:36.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.866+0000 7f3760c44640 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:3 11.8 11:10000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x561d62e9bc80 con 0x7f3750005ac0 2023-09-23T03:24:36.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.866+0000 7f3760443640 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x7f3750005ac0 0x7f37500133f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:36.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.867+0000 7f3760443640 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x7f3750005ac0 0x7f37500133f0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:36.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.867+0000 7f3760443640 1 -- 172.21.15.154:0/2423586916 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(3 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3748007610 con 0x7f3750005ac0 2023-09-23T03:24:36.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.867+0000 7f3760443640 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x7f3748019040 0x7f374801a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:36.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.867+0000 7f3760443640 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:4 11.18 11:18000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x561d62e9bc80 con 0x7f3748019040 2023-09-23T03:24:36.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.867+0000 7f3761445640 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x7f3748019040 0x7f374801a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:36.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.868+0000 7f3761445640 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x7f3748019040 0x7f374801a940 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:36.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.868+0000 7f3761445640 1 -- 172.21.15.154:0/2423586916 <== osd.2 v2:172.21.15.154:6822/2884407243 1 ==== osd_op_reply(4 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f375806c020 con 0x7f3748019040 2023-09-23T03:24:36.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.868+0000 7f3761445640 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x7f375809aa90 0x7f3758098e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:36.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.868+0000 7f3761445640 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:5 11.4 11:20000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x561d62e9bc80 con 0x7f375809aa90 2023-09-23T03:24:36.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.869+0000 7f3760c44640 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x7f375809aa90 0x7f3758098e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:36.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.870+0000 7f3760c44640 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x7f375809aa90 0x7f3758098e90 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:36.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.871+0000 7f3760c44640 1 -- 172.21.15.154:0/2423586916 <== osd.7 v2:172.21.15.190:6824/3460136272 1 ==== osd_op_reply(5 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3750006040 con 0x7f375809aa90 2023-09-23T03:24:36.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.871+0000 7f3760c44640 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x7f37500233c0 0x7f3750023800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:36.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.871+0000 7f3760c44640 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:6 11.14 11:28000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x561d62e9bc80 con 0x7f37500233c0 2023-09-23T03:24:36.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.872+0000 7f3760443640 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x7f37500233c0 0x7f3750023800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:36.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.873+0000 7f3760443640 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x7f37500233c0 0x7f3750023800 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:36.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.874+0000 7f3760443640 1 -- 172.21.15.154:0/2423586916 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== osd_op_reply(6 [pgnls start_epoch 87 out=149b] v86'22 uv22 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x7f3748007610 con 0x7f37500233c0 2023-09-23T03:24:36.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.874+0000 7f3760443640 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:7 11.c 11:30000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x561d62e9bc80 con 0x561d62f2bed0 2023-09-23T03:24:36.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.874+0000 7f3760443640 1 -- 172.21.15.154:0/2423586916 <== osd.1 v2:172.21.15.154:6816/1267710005 2 ==== osd_op_reply(7 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3748007610 con 0x561d62f2bed0 2023-09-23T03:24:36.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.874+0000 7f3760443640 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:8 11.1c 11:38000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x561d62e9bc80 con 0x7f37500233c0 2023-09-23T03:24:36.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.875+0000 7f3760443640 1 -- 172.21.15.154:0/2423586916 <== osd.6 v2:172.21.15.190:6816/1964294001 2 ==== osd_op_reply(8 [pgnls start_epoch 87 out=73b] v82'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f3748007610 con 0x7f37500233c0 2023-09-23T03:24:36.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.875+0000 7f3760443640 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:9 11.2 11:40000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x561d62e9bc80 con 0x7f3748019040 2023-09-23T03:24:36.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.875+0000 7f3761445640 1 -- 172.21.15.154:0/2423586916 <== osd.2 v2:172.21.15.154:6822/2884407243 2 ==== osd_op_reply(9 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f375806c020 con 0x7f3748019040 2023-09-23T03:24:36.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.875+0000 7f3761445640 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:10 11.12 11:48000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x561d62e9bc80 con 0x7f375809aa90 2023-09-23T03:24:36.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.876+0000 7f3760c44640 1 -- 172.21.15.154:0/2423586916 <== osd.7 v2:172.21.15.190:6824/3460136272 2 ==== osd_op_reply(10 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3750006040 con 0x7f375809aa90 2023-09-23T03:24:36.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.876+0000 7f3760c44640 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:11 11.a 11:50000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x561d62e9bc80 con 0x7f3750005ac0 2023-09-23T03:24:36.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.877+0000 7f3760443640 1 -- 172.21.15.154:0/2423586916 <== osd.3 v2:172.21.15.154:6808/1133688468 2 ==== osd_op_reply(11 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3748007610 con 0x7f3750005ac0 2023-09-23T03:24:36.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.877+0000 7f3760443640 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x7f3748018410 0x7f3748018870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:36.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.877+0000 7f3760443640 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:12 11.1a 11:58000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x561d62e9bc80 con 0x7f3748018410 2023-09-23T03:24:36.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.877+0000 7f3761445640 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x7f3748018410 0x7f3748018870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:36.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.878+0000 7f3761445640 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x7f3748018410 0x7f3748018870 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:36.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.879+0000 7f3761445640 1 -- 172.21.15.154:0/2423586916 <== osd.5 v2:172.21.15.190:6800/882536885 1 ==== osd_op_reply(12 [pgnls start_epoch 87 out=49b] v82'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f37580c3db0 con 0x7f3748018410 2023-09-23T03:24:36.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.879+0000 7f3761445640 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:13 11.6 11:60000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x561d62e9bc80 con 0x7f3748006890 2023-09-23T03:24:36.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.880+0000 7f3760c44640 1 -- 172.21.15.154:0/2423586916 <== osd.0 v2:172.21.15.154:6800/2920938138 2 ==== osd_op_reply(13 [pgnls start_epoch 87 out=77b] v86'5 uv5 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x7f3750006040 con 0x7f3748006890 2023-09-23T03:24:36.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.880+0000 7f3760c44640 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:14 11.16 11:68000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x561d62e9bc80 con 0x7f3748018410 2023-09-23T03:24:36.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.881+0000 7f3761445640 1 -- 172.21.15.154:0/2423586916 <== osd.5 v2:172.21.15.190:6800/882536885 2 ==== osd_op_reply(14 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f37580c3db0 con 0x7f3748018410 2023-09-23T03:24:36.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.881+0000 7f3761445640 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x7f37580b1040 0x7f375809e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:36.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.881+0000 7f3761445640 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:15 11.e 11:70000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x561d62e9bc80 con 0x7f37580b1040 2023-09-23T03:24:36.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.882+0000 7f3760c44640 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x7f37580b1040 0x7f375809e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:36.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.883+0000 7f3760c44640 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x7f37580b1040 0x7f375809e120 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:36.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.884+0000 7f3760c44640 1 -- 172.21.15.154:0/2423586916 <== osd.4 v2:172.21.15.190:6808/717302641 1 ==== osd_op_reply(15 [pgnls start_epoch 87 out=73b] v82'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f3750006040 con 0x7f37580b1040 2023-09-23T03:24:36.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.884+0000 7f3760c44640 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:16 11.1e 11:78000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x561d62e9bc80 con 0x7f375809aa90 2023-09-23T03:24:36.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.885+0000 7f3760c44640 1 -- 172.21.15.154:0/2423586916 <== osd.7 v2:172.21.15.190:6824/3460136272 3 ==== osd_op_reply(16 [pgnls start_epoch 87 out=49b] v82'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3750006040 con 0x7f375809aa90 2023-09-23T03:24:36.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.885+0000 7f3760c44640 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:17 11.1 11:80000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x561d62e9bc80 con 0x7f37500233c0 2023-09-23T03:24:36.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.886+0000 7f3760443640 1 -- 172.21.15.154:0/2423586916 <== osd.6 v2:172.21.15.190:6816/1964294001 3 ==== osd_op_reply(17 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3748007610 con 0x7f37500233c0 2023-09-23T03:24:36.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.886+0000 7f3760443640 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:18 11.11 11:88000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x561d62e9bc80 con 0x7f3748019040 2023-09-23T03:24:36.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.887+0000 7f3761445640 1 -- 172.21.15.154:0/2423586916 <== osd.2 v2:172.21.15.154:6822/2884407243 3 ==== osd_op_reply(18 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f37580c3db0 con 0x7f3748019040 2023-09-23T03:24:36.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.887+0000 7f3761445640 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:19 11.9 11:90000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x561d62e9bc80 con 0x7f3750005ac0 2023-09-23T03:24:36.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.887+0000 7f3760443640 1 -- 172.21.15.154:0/2423586916 <== osd.3 v2:172.21.15.154:6808/1133688468 3 ==== osd_op_reply(19 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3748007610 con 0x7f3750005ac0 2023-09-23T03:24:36.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.887+0000 7f3760443640 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:20 11.19 11:98000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x561d62e9bc80 con 0x7f3748006890 2023-09-23T03:24:36.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.888+0000 7f3760c44640 1 -- 172.21.15.154:0/2423586916 <== osd.0 v2:172.21.15.154:6800/2920938138 3 ==== osd_op_reply(20 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3750006040 con 0x7f3748006890 2023-09-23T03:24:36.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.888+0000 7f3760c44640 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:21 11.5 11:a0000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x561d62e9bc80 con 0x7f375809aa90 2023-09-23T03:24:36.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.889+0000 7f3760c44640 1 -- 172.21.15.154:0/2423586916 <== osd.7 v2:172.21.15.190:6824/3460136272 4 ==== osd_op_reply(21 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3750006040 con 0x7f375809aa90 2023-09-23T03:24:36.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.889+0000 7f3760c44640 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:22 11.15 11:a8000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x561d62e9bc80 con 0x7f3748019040 2023-09-23T03:24:36.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.890+0000 7f3761445640 1 -- 172.21.15.154:0/2423586916 <== osd.2 v2:172.21.15.154:6822/2884407243 4 ==== osd_op_reply(22 [pgnls start_epoch 87 out=97b] v82'3 uv3 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x7f37580c3db0 con 0x7f3748019040 2023-09-23T03:24:36.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.890+0000 7f3761445640 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:23 11.d 11:b0000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x561d62e9bc80 con 0x7f3748019040 2023-09-23T03:24:36.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.890+0000 7f3761445640 1 -- 172.21.15.154:0/2423586916 <== osd.2 v2:172.21.15.154:6822/2884407243 5 ==== osd_op_reply(23 [pgnls start_epoch 87 out=74b] v82'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f37580c3db0 con 0x7f3748019040 2023-09-23T03:24:36.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.890+0000 7f3761445640 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:24 11.1d 11:b8000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x561d62e9bc80 con 0x7f375809aa90 2023-09-23T03:24:36.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.892+0000 7f3760c44640 1 -- 172.21.15.154:0/2423586916 <== osd.7 v2:172.21.15.190:6824/3460136272 5 ==== osd_op_reply(24 [pgnls start_epoch 87 out=73b] v82'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f3750006040 con 0x7f375809aa90 2023-09-23T03:24:36.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.892+0000 7f3760c44640 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:25 11.3 11:c0000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x561d62e9bc80 con 0x7f37500233c0 2023-09-23T03:24:36.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.892+0000 7f3760443640 1 -- 172.21.15.154:0/2423586916 <== osd.6 v2:172.21.15.190:6816/1964294001 4 ==== osd_op_reply(25 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f3748007610 con 0x7f37500233c0 2023-09-23T03:24:36.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.892+0000 7f3760443640 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:26 11.13 11:c8000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x561d62e9bc80 con 0x7f3748018410 2023-09-23T03:24:36.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.894+0000 7f3761445640 1 -- 172.21.15.154:0/2423586916 <== osd.5 v2:172.21.15.190:6800/882536885 3 ==== osd_op_reply(26 [pgnls start_epoch 87 out=98b] v86'3 uv3 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x7f37580c3db0 con 0x7f3748018410 2023-09-23T03:24:36.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.894+0000 7f3761445640 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:27 11.b 11:d0000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x561d62e9bc80 con 0x7f3750005ac0 2023-09-23T03:24:36.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.895+0000 7f3760443640 1 -- 172.21.15.154:0/2423586916 <== osd.3 v2:172.21.15.154:6808/1133688468 4 ==== osd_op_reply(27 [pgnls start_epoch 87 out=76b] v86'3 uv3 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x7f3748007610 con 0x7f3750005ac0 2023-09-23T03:24:36.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.895+0000 7f3760443640 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:28 11.1b 11:d8000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x561d62e9bc80 con 0x7f37580b1040 2023-09-23T03:24:36.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.896+0000 7f3760c44640 1 -- 172.21.15.154:0/2423586916 <== osd.4 v2:172.21.15.190:6808/717302641 2 ==== osd_op_reply(28 [pgnls start_epoch 87 out=49b] v82'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f375003e040 con 0x7f37580b1040 2023-09-23T03:24:36.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.896+0000 7f3760c44640 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:29 11.7 11:e0000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x561d62e9bc80 con 0x7f3748018410 2023-09-23T03:24:36.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.897+0000 7f3761445640 1 -- 172.21.15.154:0/2423586916 <== osd.5 v2:172.21.15.190:6800/882536885 4 ==== osd_op_reply(29 [pgnls start_epoch 87 out=73b] v82'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f37580c3db0 con 0x7f3748018410 2023-09-23T03:24:36.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.897+0000 7f3761445640 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:30 11.17 11:e8000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x561d62e9bc80 con 0x7f375809aa90 2023-09-23T03:24:36.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.898+0000 7f3760c44640 1 -- 172.21.15.154:0/2423586916 <== osd.7 v2:172.21.15.190:6824/3460136272 6 ==== osd_op_reply(30 [pgnls start_epoch 87 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f375003e040 con 0x7f375809aa90 2023-09-23T03:24:36.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.898+0000 7f3760c44640 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:31 11.f 11:f0000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x561d62e9bc80 con 0x7f375809aa90 2023-09-23T03:24:36.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.899+0000 7f3760c44640 1 -- 172.21.15.154:0/2423586916 <== osd.7 v2:172.21.15.190:6824/3460136272 7 ==== osd_op_reply(31 [pgnls start_epoch 87 out=73b] v82'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f375003e040 con 0x7f375809aa90 2023-09-23T03:24:36.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.899+0000 7f3760c44640 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:32 11.1f 11:f8000000:%01:::head [pgnls start_epoch 87 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e87) v8 -- 0x561d62e9bc80 con 0x7f3748019040 2023-09-23T03:24:36.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.900+0000 7f3761445640 1 -- 172.21.15.154:0/2423586916 <== osd.2 v2:172.21.15.154:6822/2884407243 6 ==== osd_op_reply(32 [pgnls start_epoch 87 out=101b] v86'6 uv6 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x7f37580c3db0 con 0x7f3748019040 2023-09-23T03:24:36.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.900+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:33 11.10 11:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x561d62e9c0a0 con 0x7f3748006890 2023-09-23T03:24:36.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.900+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:34 11.10 11:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x561d62fcb050 con 0x7f3748006890 2023-09-23T03:24:36.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.900+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:35 11.10 11:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x561d62fcdbd0 con 0x7f3748006890 2023-09-23T03:24:36.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.900+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:36 11.10 11:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x561d62fce3d0 con 0x7f3748006890 2023-09-23T03:24:36.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.900+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:37 11.10 11:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x561d62fcebd0 con 0x7f3748006890 2023-09-23T03:24:36.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.900+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:38 11.14 11:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x561d62fcf3d0 con 0x7f37500233c0 2023-09-23T03:24:36.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.901+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:39 11.14 11:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x561d62fcfbd0 con 0x7f37500233c0 2023-09-23T03:24:36.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.901+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:40 11.14 11:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x561d62fd03d0 con 0x7f37500233c0 2023-09-23T03:24:36.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.901+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:41 11.14 11:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x561d62fd0bd0 con 0x7f37500233c0 2023-09-23T03:24:36.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.901+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:42 11.1c 11:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x561d62fd13d0 con 0x7f37500233c0 2023-09-23T03:24:36.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.901+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:43 11.6 11:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x561d62fd1c80 con 0x7f3748006890 2023-09-23T03:24:36.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.901+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:44 11.e 11:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x561d62fd2700 con 0x7f37580b1040 2023-09-23T03:24:36.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.901+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:45 11.15 11:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x561d62fd31f0 con 0x7f3748019040 2023-09-23T03:24:36.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.901+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:46 11.15 11:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x561d62fd3cb0 con 0x7f3748019040 2023-09-23T03:24:36.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.901+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:47 11.d 11:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x561d62fd47a0 con 0x7f3748019040 2023-09-23T03:24:36.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.901+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:48 11.1d 11:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x561d62fd5210 con 0x7f375809aa90 2023-09-23T03:24:36.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.904+0000 7f3761445640 1 -- 172.21.15.154:0/2423586916 <== osd.2 v2:172.21.15.154:6822/2884407243 7 ==== osd_op_reply(45 604.00000000 [delete] v87'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f37580c3db0 con 0x7f3748019040 2023-09-23T03:24:36.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.905+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:49 11.13 11:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x561d62fd52b0 con 0x7f3748018410 2023-09-23T03:24:36.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.906+0000 7f3760c44640 1 -- 172.21.15.154:0/2423586916 <== osd.0 v2:172.21.15.154:6800/2920938138 4 ==== osd_op_reply(33 601.00000000 [delete] v87'11 uv11 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f375003e040 con 0x7f3748006890 2023-09-23T03:24:36.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.907+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:50 11.13 11:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x561d62fd56d0 con 0x7f3748018410 2023-09-23T03:24:36.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.907+0000 7f3760c44640 1 -- 172.21.15.154:0/2423586916 <== osd.0 v2:172.21.15.154:6800/2920938138 5 ==== osd_op_reply(34 602.00000000 [delete] v87'12 uv12 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f375003e040 con 0x7f3748006890 2023-09-23T03:24:36.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.907+0000 7f3760c44640 1 -- 172.21.15.154:0/2423586916 <== osd.0 v2:172.21.15.154:6800/2920938138 6 ==== osd_op_reply(35 10000000000.00000000 [delete] v87'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f375003e040 con 0x7f3748006890 2023-09-23T03:24:36.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.907+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:51 11.b 11:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x561d62fcb500 con 0x7f3750005ac0 2023-09-23T03:24:36.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.907+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:52 11.7 11:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x561d62fd5f50 con 0x7f3748018410 2023-09-23T03:24:36.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.908+0000 7f3761445640 1 -- 172.21.15.154:0/2423586916 <== osd.2 v2:172.21.15.154:6822/2884407243 8 ==== osd_op_reply(46 500.00000000 [delete] v87'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f37580c3db0 con 0x7f3748019040 2023-09-23T03:24:36.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.908+0000 7f3760c44640 1 -- 172.21.15.154:0/2423586916 <== osd.0 v2:172.21.15.154:6800/2920938138 7 ==== osd_op_reply(36 600.00000000 [delete] v87'14 uv14 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f375003e040 con 0x7f3748006890 2023-09-23T03:24:36.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.908+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:53 11.f 11:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x561d62fd5f50 con 0x7f375809aa90 2023-09-23T03:24:36.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.908+0000 7f3760c44640 1 -- 172.21.15.154:0/2423586916 <== osd.0 v2:172.21.15.154:6800/2920938138 8 ==== osd_op_reply(43 mds0_openfiles.0 [delete] v87'6 uv6 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f375003e040 con 0x7f3748006890 2023-09-23T03:24:36.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.908+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:54 11.1f 11:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x561d62fd6330 con 0x7f3748019040 2023-09-23T03:24:36.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.908+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:55 11.1f 11:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e87) v8 -- 0x561d62fd6b30 con 0x7f3748019040 2023-09-23T03:24:36.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.909+0000 7f3760c44640 1 -- 172.21.15.154:0/2423586916 <== osd.7 v2:172.21.15.190:6824/3460136272 8 ==== osd_op_reply(48 605.00000000 [delete] v87'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f375003e040 con 0x7f375809aa90 2023-09-23T03:24:36.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.910+0000 7f3760c44640 1 -- 172.21.15.154:0/2423586916 <== osd.0 v2:172.21.15.154:6800/2920938138 9 ==== osd_op_reply(37 603.00000000 [delete] v87'15 uv15 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f375003e040 con 0x7f3748006890 2023-09-23T03:24:36.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.910+0000 7f3760c44640 1 -- 172.21.15.154:0/2423586916 <== osd.4 v2:172.21.15.190:6808/717302641 3 ==== osd_op_reply(44 608.00000000 [delete] v87'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f375003e040 con 0x7f37580b1040 2023-09-23T03:24:36.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.910+0000 7f3761445640 1 -- 172.21.15.154:0/2423586916 <== osd.2 v2:172.21.15.154:6822/2884407243 9 ==== osd_op_reply(47 mds_snaptable [delete] v87'2 uv2 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f37580c3db0 con 0x7f3748019040 2023-09-23T03:24:36.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.910+0000 7f3760443640 1 -- 172.21.15.154:0/2423586916 <== osd.6 v2:172.21.15.190:6816/1964294001 5 ==== osd_op_reply(38 1.00000000.inode [delete] v87'23 uv23 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f3748007610 con 0x7f37500233c0 2023-09-23T03:24:36.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.911+0000 7f3761445640 1 -- 172.21.15.154:0/2423586916 <== osd.2 v2:172.21.15.154:6822/2884407243 10 ==== osd_op_reply(54 100.00000000.inode [delete] v87'7 uv7 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f37580c3db0 con 0x7f3748019040 2023-09-23T03:24:36.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.911+0000 7f3760443640 1 -- 172.21.15.154:0/2423586916 <== osd.6 v2:172.21.15.190:6816/1964294001 6 ==== osd_op_reply(42 607.00000000 [delete] v87'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3748007610 con 0x7f37500233c0 2023-09-23T03:24:36.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.911+0000 7f3761445640 1 -- 172.21.15.154:0/2423586916 <== osd.2 v2:172.21.15.154:6822/2884407243 11 ==== osd_op_reply(55 1.00000000 [delete] v87'8 uv8 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f37580c3db0 con 0x7f3748019040 2023-09-23T03:24:36.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.912+0000 7f3760443640 1 -- 172.21.15.154:0/2423586916 <== osd.3 v2:172.21.15.154:6808/1133688468 5 ==== osd_op_reply(51 mds0_sessionmap [delete] v87'4 uv4 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f3748007610 con 0x7f3750005ac0 2023-09-23T03:24:36.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.912+0000 7f3760443640 1 -- 172.21.15.154:0/2423586916 <== osd.6 v2:172.21.15.190:6816/1964294001 7 ==== osd_op_reply(39 200.00000000 [delete] v87'24 uv24 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3748007610 con 0x7f37500233c0 2023-09-23T03:24:36.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.913+0000 7f3761445640 1 -- 172.21.15.154:0/2423586916 <== osd.5 v2:172.21.15.190:6800/882536885 5 ==== osd_op_reply(49 mds0_inotable [delete] v87'4 uv4 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f37580c3db0 con 0x7f3748018410 2023-09-23T03:24:36.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.913+0000 7f3760443640 1 -- 172.21.15.154:0/2423586916 <== osd.6 v2:172.21.15.190:6816/1964294001 8 ==== osd_op_reply(40 200.00000001 [delete] v87'25 uv25 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3748007610 con 0x7f37500233c0 2023-09-23T03:24:36.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.913+0000 7f3760443640 1 -- 172.21.15.154:0/2423586916 <== osd.6 v2:172.21.15.190:6816/1964294001 9 ==== osd_op_reply(41 606.00000000 [delete] v87'26 uv26 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f3748007610 con 0x7f37500233c0 2023-09-23T03:24:36.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.914+0000 7f3761445640 1 -- 172.21.15.154:0/2423586916 <== osd.5 v2:172.21.15.190:6800/882536885 6 ==== osd_op_reply(50 100.00000000 [delete] v87'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f37580c3db0 con 0x7f3748018410 2023-09-23T03:24:36.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.914+0000 7f3761445640 1 -- 172.21.15.154:0/2423586916 <== osd.5 v2:172.21.15.190:6800/882536885 7 ==== osd_op_reply(52 609.00000000 [delete] v87'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f37580c3db0 con 0x7f3748018410 2023-09-23T03:24:36.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.914+0000 7f3760c44640 1 -- 172.21.15.154:0/2423586916 <== osd.7 v2:172.21.15.190:6824/3460136272 9 ==== osd_op_reply(53 400.00000000 [delete] v87'2 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f375003e040 con 0x7f375809aa90 2023-09-23T03:24:36.915 INFO:teuthology.orchestra.run.smithi154.stdout:Removed 23 objects 2023-09-23T03:24:36.915 INFO:teuthology.orchestra.run.smithi154.stdout:successfully purged pool cephfs_metadata 2023-09-23T03:24:36.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.914+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x7f3748006890 msgr2=0x7f37480122a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:36.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.914+0000 7f3763140f40 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x7f3748006890 0x7f37480122a0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:36.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.914+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x561d62f2bed0 msgr2=0x561d62e9b740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:36.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.914+0000 7f3763140f40 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x561d62f2bed0 0x561d62e9b740 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:36.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.914+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x7f3748019040 msgr2=0x7f374801a940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:36.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.914+0000 7f3763140f40 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x7f3748019040 0x7f374801a940 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:36.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.914+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x7f3750005ac0 msgr2=0x7f37500133f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:36.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.914+0000 7f3763140f40 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x7f3750005ac0 0x7f37500133f0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:36.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.914+0000 7f3761445640 1 -- 172.21.15.154:0/2423586916 reap_dead start 2023-09-23T03:24:36.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.914+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x7f37580b1040 msgr2=0x7f375809e120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:36.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.914+0000 7f3763140f40 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x7f37580b1040 0x7f375809e120 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:36.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.914+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x7f3748018410 msgr2=0x7f3748018870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:36.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.914+0000 7f3763140f40 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x7f3748018410 0x7f3748018870 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:36.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.914+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x7f37500233c0 msgr2=0x7f3750023800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:36.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.914+0000 7f3763140f40 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x7f37500233c0 0x7f3750023800 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:36.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.914+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x7f375809aa90 msgr2=0x7f3758098e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:36.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.914+0000 7f3763140f40 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x7f375809aa90 0x7f3758098e90 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:36.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.914+0000 7f3761445640 1 -- 172.21.15.154:0/2423586916 reap_dead start 2023-09-23T03:24:36.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.914+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f37400698e0 msgr2=0x7f374006bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:36.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.914+0000 7f3763140f40 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f37400698e0 0x7f374006bda0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3750006f90 tx=0x7f3750008040 comp rx=0 tx=0).stop 2023-09-23T03:24:36.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.914+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561d62f29950 msgr2=0x561d62f23d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:36.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.914+0000 7f3763140f40 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561d62f29950 0x561d62f23d00 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f375800a4f0 tx=0x561d62f2c3b0 comp rx=0 tx=0).stop 2023-09-23T03:24:36.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.915+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 shutdown_connections 2023-09-23T03:24:36.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.915+0000 7f3763140f40 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f37400698e0 0x7f374006bda0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:36.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.915+0000 7f3763140f40 1 --2- 172.21.15.154:0/2423586916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561d62f29950 0x561d62f23d00 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:36.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.915+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 >> 172.21.15.154:0/2423586916 conn(0x561d62e4e690 msgr2=0x561d62f21190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:36.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.915+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 shutdown_connections 2023-09-23T03:24:36.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:36.915+0000 7f3763140f40 1 -- 172.21.15.154:0/2423586916 wait complete. 2023-09-23T03:24:36.920 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs reset cephfs --yes-i-really-mean-it 2023-09-23T03:24:37.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.129+0000 7fe747ae5640 1 Processor -- start 2023-09-23T03:24:37.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.129+0000 7fe747ae5640 1 -- start start 2023-09-23T03:24:37.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.131+0000 7fe747ae5640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe74015abe0 0x7fe7400cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:37.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.131+0000 7fe747ae5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7400cc9b0 0x7fe7400b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:37.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.131+0000 7fe747ae5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe7400cce10 0x7fe7400b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:37.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.131+0000 7fe747ae5640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7400ce510 con 0x7fe7400cce10 2023-09-23T03:24:37.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.131+0000 7fe747ae5640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7400cb710 con 0x7fe74015abe0 2023-09-23T03:24:37.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.131+0000 7fe747ae5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7400cd390 con 0x7fe7400cc9b0 2023-09-23T03:24:37.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.131+0000 7fe746082640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe7400cce10 0x7fe7400b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:37.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.132+0000 7fe746082640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe7400cce10 0x7fe7400b26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:42992/0 (socket says 172.21.15.154:42992) 2023-09-23T03:24:37.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.132+0000 7fe746082640 1 -- 172.21.15.154:0/3956278817 learned_addr learned my addr 172.21.15.154:0/3956278817 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:37.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.132+0000 7fe745881640 1 --2- 172.21.15.154:0/3956278817 >> v2:172.21.15.190:3300/0 conn(0x7fe74015abe0 0x7fe7400cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:37.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.132+0000 7fe745080640 1 --2- 172.21.15.154:0/3956278817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7400cc9b0 0x7fe7400b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:37.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.132+0000 7fe746082640 1 -- 172.21.15.154:0/3956278817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7400cc9b0 msgr2=0x7fe7400b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:37.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.132+0000 7fe746082640 1 --2- 172.21.15.154:0/3956278817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7400cc9b0 0x7fe7400b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:37.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.132+0000 7fe746082640 1 -- 172.21.15.154:0/3956278817 >> v2:172.21.15.190:3300/0 conn(0x7fe74015abe0 msgr2=0x7fe7400cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:37.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.132+0000 7fe746082640 1 --2- 172.21.15.154:0/3956278817 >> v2:172.21.15.190:3300/0 conn(0x7fe74015abe0 0x7fe7400cb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:37.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.132+0000 7fe746082640 1 -- 172.21.15.154:0/3956278817 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7400cd670 con 0x7fe7400cce10 2023-09-23T03:24:37.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.132+0000 7fe745080640 1 --2- 172.21.15.154:0/3956278817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7400cc9b0 0x7fe7400b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:37.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.133+0000 7fe746082640 1 --2- 172.21.15.154:0/3956278817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7400cce10 0x7fe7400b26f0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fe740070cc0 tx=0x7fe73802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27732161d15d3672 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:37.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.133+0000 7fe74487f640 1 -- 172.21.15.154:0/3956278817 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73802c9f0 con 0x7fe7400cce10 2023-09-23T03:24:37.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.133+0000 7fe74487f640 1 -- 172.21.15.154:0/3956278817 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe73802cb50 con 0x7fe7400cce10 2023-09-23T03:24:37.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.133+0000 7fe74487f640 1 -- 172.21.15.154:0/3956278817 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7380343f0 con 0x7fe7400cce10 2023-09-23T03:24:37.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.134+0000 7fe747ae5640 1 -- 172.21.15.154:0/3956278817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7400cce10 msgr2=0x7fe7400b26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:37.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.134+0000 7fe747ae5640 1 --2- 172.21.15.154:0/3956278817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7400cce10 0x7fe7400b26f0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fe740070cc0 tx=0x7fe73802be50 comp rx=0 tx=0).stop 2023-09-23T03:24:37.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.134+0000 7fe747ae5640 1 -- 172.21.15.154:0/3956278817 shutdown_connections 2023-09-23T03:24:37.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.134+0000 7fe747ae5640 1 --2- 172.21.15.154:0/3956278817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7400cce10 0x7fe7400b26f0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:37.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.134+0000 7fe747ae5640 1 --2- 172.21.15.154:0/3956278817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7400cc9b0 0x7fe7400b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:37.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.134+0000 7fe747ae5640 1 --2- 172.21.15.154:0/3956278817 >> v2:172.21.15.190:3300/0 conn(0x7fe74015abe0 0x7fe7400cb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:37.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.134+0000 7fe747ae5640 1 -- 172.21.15.154:0/3956278817 >> 172.21.15.154:0/3956278817 conn(0x7fe74006fc40 msgr2=0x7fe740070020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:37.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.134+0000 7fe747ae5640 1 -- 172.21.15.154:0/3956278817 shutdown_connections 2023-09-23T03:24:37.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.134+0000 7fe747ae5640 1 -- 172.21.15.154:0/3956278817 wait complete. 2023-09-23T03:24:37.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.136+0000 7fe747ae5640 1 Processor -- start 2023-09-23T03:24:37.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.136+0000 7fe747ae5640 1 -- start start 2023-09-23T03:24:37.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.137+0000 7fe747ae5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7400cc9b0 0x7fe7400b2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:37.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.137+0000 7fe747ae5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7400cce10 0x7fe7401f35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:37.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.137+0000 7fe747ae5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe74015abe0 0x7fe7400b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:37.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.137+0000 7fe747ae5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7401f3b10 con 0x7fe7400cc9b0 2023-09-23T03:24:37.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.137+0000 7fe747ae5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7400b5f40 con 0x7fe74015abe0 2023-09-23T03:24:37.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.137+0000 7fe747ae5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7400b15c0 con 0x7fe7400cce10 2023-09-23T03:24:37.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.137+0000 7fe745881640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7400cc9b0 0x7fe7400b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:37.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.138+0000 7fe745881640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7400cc9b0 0x7fe7400b2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43008/0 (socket says 172.21.15.154:43008) 2023-09-23T03:24:37.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.138+0000 7fe745881640 1 -- 172.21.15.154:0/1886757814 learned_addr learned my addr 172.21.15.154:0/1886757814 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:37.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.138+0000 7fe746082640 1 --2- 172.21.15.154:0/1886757814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe74015abe0 0x7fe7400b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:37.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.138+0000 7fe745881640 1 -- 172.21.15.154:0/1886757814 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7400cce10 msgr2=0x7fe7401f35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:37.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.138+0000 7fe745881640 1 --2- 172.21.15.154:0/1886757814 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7400cce10 0x7fe7401f35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:37.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.138+0000 7fe745881640 1 -- 172.21.15.154:0/1886757814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe74015abe0 msgr2=0x7fe7400b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:37.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.138+0000 7fe745881640 1 --2- 172.21.15.154:0/1886757814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe74015abe0 0x7fe7400b3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:37.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.138+0000 7fe745881640 1 -- 172.21.15.154:0/1886757814 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe738009b30 con 0x7fe7400cc9b0 2023-09-23T03:24:37.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.138+0000 7fe745881640 1 --2- 172.21.15.154:0/1886757814 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7400cc9b0 0x7fe7400b2660 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fe73c00d7d0 tx=0x7fe73c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:37.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.138+0000 7fe7367fc640 1 -- 172.21.15.154:0/1886757814 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73c0096c0 con 0x7fe7400cc9b0 2023-09-23T03:24:37.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.139+0000 7fe7367fc640 1 -- 172.21.15.154:0/1886757814 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe73c009820 con 0x7fe7400cc9b0 2023-09-23T03:24:37.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.139+0000 7fe747ae5640 1 -- 172.21.15.154:0/1886757814 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7400b18a0 con 0x7fe7400cc9b0 2023-09-23T03:24:37.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.139+0000 7fe7367fc640 1 -- 172.21.15.154:0/1886757814 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73c009bd0 con 0x7fe7400cc9b0 2023-09-23T03:24:37.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.139+0000 7fe747ae5640 1 -- 172.21.15.154:0/1886757814 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7400b1e60 con 0x7fe7400cc9b0 2023-09-23T03:24:37.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.140+0000 7fe747ae5640 1 -- 172.21.15.154:0/1886757814 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe74004e920 con 0x7fe7400cc9b0 2023-09-23T03:24:37.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.141+0000 7fe7367fc640 1 -- 172.21.15.154:0/1886757814 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fe73c009d30 con 0x7fe7400cc9b0 2023-09-23T03:24:37.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.142+0000 7fe7367fc640 1 --2- 172.21.15.154:0/1886757814 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe72406af60 0x7fe72406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:37.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.142+0000 7fe7367fc640 1 -- 172.21.15.154:0/1886757814 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6519+0+0 (secure 0 0 0) 0x7fe73c08a120 con 0x7fe7400cc9b0 2023-09-23T03:24:37.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.142+0000 7fe745080640 1 --2- 172.21.15.154:0/1886757814 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe72406af60 0x7fe72406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:37.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.143+0000 7fe745080640 1 --2- 172.21.15.154:0/1886757814 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe72406af60 0x7fe72406d420 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe730006fd0 tx=0x7fe730008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:37.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.146+0000 7fe7367fc640 1 -- 172.21.15.154:0/1886757814 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fe73c054ec0 con 0x7fe7400cc9b0 2023-09-23T03:24:37.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:37.384+0000 7fe747ae5640 1 -- 172.21.15.154:0/1886757814 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs reset", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe74015e7c0 con 0x7fe7400cc9b0 2023-09-23T03:24:38.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.346+0000 7fe7367fc640 1 -- 172.21.15.154:0/1886757814 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs reset", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v66) v1 ==== 107+0+0 (secure 0 0 0) 0x7fe73c0584e0 con 0x7fe7400cc9b0 2023-09-23T03:24:38.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.349+0000 7fe747ae5640 1 -- 172.21.15.154:0/1886757814 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe72406af60 msgr2=0x7fe72406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:38.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.349+0000 7fe747ae5640 1 --2- 172.21.15.154:0/1886757814 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe72406af60 0x7fe72406d420 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe730006fd0 tx=0x7fe730008040 comp rx=0 tx=0).stop 2023-09-23T03:24:38.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.349+0000 7fe747ae5640 1 -- 172.21.15.154:0/1886757814 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7400cc9b0 msgr2=0x7fe7400b2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:38.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.349+0000 7fe747ae5640 1 --2- 172.21.15.154:0/1886757814 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7400cc9b0 0x7fe7400b2660 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fe73c00d7d0 tx=0x7fe73c00dc90 comp rx=0 tx=0).stop 2023-09-23T03:24:38.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.349+0000 7fe747ae5640 1 -- 172.21.15.154:0/1886757814 shutdown_connections 2023-09-23T03:24:38.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.349+0000 7fe747ae5640 1 --2- 172.21.15.154:0/1886757814 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe72406af60 0x7fe72406d420 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:38.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.349+0000 7fe747ae5640 1 --2- 172.21.15.154:0/1886757814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe74015abe0 0x7fe7400b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:38.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.349+0000 7fe747ae5640 1 --2- 172.21.15.154:0/1886757814 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe7400cce10 0x7fe7401f35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:38.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.349+0000 7fe747ae5640 1 --2- 172.21.15.154:0/1886757814 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7400cc9b0 0x7fe7400b2660 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:38.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.349+0000 7fe747ae5640 1 -- 172.21.15.154:0/1886757814 >> 172.21.15.154:0/1886757814 conn(0x7fe74006fc40 msgr2=0x7fe740070020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:38.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.350+0000 7fe747ae5640 1 -- 172.21.15.154:0/1886757814 shutdown_connections 2023-09-23T03:24:38.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.350+0000 7fe747ae5640 1 -- 172.21.15.154:0/1886757814 wait complete. 2023-09-23T03:24:38.360 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:24:38.359+0000 7f33eccb1640 -1 mds.0.sessionmap _load_finish got (2) No such file or directory 2023-09-23T03:24:38.360 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:24:38.359+0000 7f33eccb1640 -1 log_channel(cluster) log [ERR] : error reading sessionmap 'mds0_sessionmap' -2 ((2) No such file or directory) 2023-09-23T03:24:38.360 INFO:tasks.ceph.mon.a.smithi154.stderr:2023-09-23T03:24:38.360+0000 7f3df35a6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-23T03:24:38.361 INFO:tasks.ceph.mon.a.smithi154.stderr:2023-09-23T03:24:38.360+0000 7f3df35a6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-09-23T03:24:38.370 INFO:tasks.ceph.mds.a.smithi154.stderr: -33> 2023-09-23T03:24:38.359+0000 7f33eccb1640 -1 mds.0.sessionmap _load_finish got (2) No such file or directory 2023-09-23T03:24:38.370 INFO:tasks.ceph.mds.a.smithi154.stderr: -32> 2023-09-23T03:24:38.359+0000 7f33eccb1640 -1 log_channel(cluster) log [ERR] : error reading sessionmap 'mds0_sessionmap' -2 ((2) No such file or directory) 2023-09-23T03:24:38.372 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-09-23T03:24:38.418 INFO:tasks.ceph.mds.a.smithi154.stdout:starting mds.a at 2023-09-23T03:24:38.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.567+0000 7ff423b89640 1 Processor -- start 2023-09-23T03:24:38.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.567+0000 7ff423b89640 1 -- start start 2023-09-23T03:24:38.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.568+0000 7ff423b89640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff41c0c8d70 0x7ff41c15c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:38.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.568+0000 7ff423b89640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff41c071290 0x7ff41c0ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:38.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.568+0000 7ff423b89640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff41c0c9d20 con 0x7ff41c0c9150 2023-09-23T03:24:38.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.568+0000 7ff423b89640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff41c071850 con 0x7ff41c071290 2023-09-23T03:24:38.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.568+0000 7ff423b89640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c071b30 con 0x7ff41c0c8d70 2023-09-23T03:24:38.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.568+0000 7ff422126640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff41c071290 0x7ff41c0ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:38.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.569+0000 7ff422126640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff41c071290 0x7ff41c0ca1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43020/0 (socket says 172.21.15.154:43020) 2023-09-23T03:24:38.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.569+0000 7ff422126640 1 -- 172.21.15.154:0/3683746753 learned_addr learned my addr 172.21.15.154:0/3683746753 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:38.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.569+0000 7ff421925640 1 --2- 172.21.15.154:0/3683746753 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff41c0c8d70 0x7ff41c15c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:38.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.569+0000 7ff422126640 1 -- 172.21.15.154:0/3683746753 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff41c0c8d70 msgr2=0x7ff41c15c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:38.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.569+0000 7ff422126640 1 --2- 172.21.15.154:0/3683746753 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff41c0c8d70 0x7ff41c15c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:38.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.569+0000 7ff422126640 1 -- 172.21.15.154:0/3683746753 >> v1:172.21.15.190:6789/0 conn(0x7ff41c0c9150 legacy=0x7ff41c15cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:38.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.569+0000 7ff422126640 1 -- 172.21.15.154:0/3683746753 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff41c071e10 con 0x7ff41c071290 2023-09-23T03:24:38.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.570+0000 7ff422126640 1 --2- 172.21.15.154:0/3683746753 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff41c071290 0x7ff41c0ca1b0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7ff414009e50 tx=0x7ff41402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dfcacd4d77c4b2a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:38.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.570+0000 7ff420923640 1 -- 172.21.15.154:0/3683746753 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41402c9e0 con 0x7ff41c071290 2023-09-23T03:24:38.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.570+0000 7ff420923640 1 -- 172.21.15.154:0/3683746753 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff41402cb40 con 0x7ff41c071290 2023-09-23T03:24:38.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.570+0000 7ff420923640 1 -- 172.21.15.154:0/3683746753 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4140343f0 con 0x7ff41c071290 2023-09-23T03:24:38.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.571+0000 7ff423b89640 1 -- 172.21.15.154:0/3683746753 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff41c071290 msgr2=0x7ff41c0ca1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:38.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.571+0000 7ff423b89640 1 --2- 172.21.15.154:0/3683746753 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff41c071290 0x7ff41c0ca1b0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7ff414009e50 tx=0x7ff41402be40 comp rx=0 tx=0).stop 2023-09-23T03:24:38.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.571+0000 7ff423b89640 1 -- 172.21.15.154:0/3683746753 shutdown_connections 2023-09-23T03:24:38.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.571+0000 7ff423b89640 1 --2- 172.21.15.154:0/3683746753 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff41c071290 0x7ff41c0ca1b0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:38.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.571+0000 7ff423b89640 1 --2- 172.21.15.154:0/3683746753 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff41c0c8d70 0x7ff41c15c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:38.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.571+0000 7ff423b89640 1 -- 172.21.15.154:0/3683746753 >> 172.21.15.154:0/3683746753 conn(0x7ff41c06c010 msgr2=0x7ff41c06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:38.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.571+0000 7ff423b89640 1 -- 172.21.15.154:0/3683746753 shutdown_connections 2023-09-23T03:24:38.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.571+0000 7ff423b89640 1 -- 172.21.15.154:0/3683746753 wait complete. 2023-09-23T03:24:38.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.573+0000 7ff423b89640 1 Processor -- start 2023-09-23T03:24:38.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.573+0000 7ff423b89640 1 -- start start 2023-09-23T03:24:38.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.574+0000 7ff423b89640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff41c071290 0x7ff41c0b02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:38.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.574+0000 7ff423b89640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff41c0c8d70 0x7ff41c15cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:38.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.574+0000 7ff423b89640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff41c0c9150 0x7ff41c0c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:38.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.574+0000 7ff423b89640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c15d510 con 0x7ff41c0c8d70 2023-09-23T03:24:38.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.574+0000 7ff423b89640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c0b09d0 con 0x7ff41c0c9150 2023-09-23T03:24:38.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.574+0000 7ff423b89640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c0b0cb0 con 0x7ff41c071290 2023-09-23T03:24:38.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.574+0000 7ff421124640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff41c0c8d70 0x7ff41c15cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:38.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.574+0000 7ff421124640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff41c0c8d70 0x7ff41c15cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43030/0 (socket says 172.21.15.154:43030) 2023-09-23T03:24:38.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.574+0000 7ff421124640 1 -- 172.21.15.154:0/1924650510 learned_addr learned my addr 172.21.15.154:0/1924650510 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:38.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.575+0000 7ff421925640 1 --2- 172.21.15.154:0/1924650510 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff41c071290 0x7ff41c0b02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:38.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.575+0000 7ff421124640 1 -- 172.21.15.154:0/1924650510 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff41c071290 msgr2=0x7ff41c0b02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:38.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.575+0000 7ff421124640 1 --2- 172.21.15.154:0/1924650510 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff41c071290 0x7ff41c0b02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:38.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.575+0000 7ff421124640 1 -- 172.21.15.154:0/1924650510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff41c0c9150 msgr2=0x7ff41c0c5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:38.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.575+0000 7ff421124640 1 --2- 172.21.15.154:0/1924650510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff41c0c9150 0x7ff41c0c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:38.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.575+0000 7ff421124640 1 -- 172.21.15.154:0/1924650510 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff414009b30 con 0x7ff41c0c8d70 2023-09-23T03:24:38.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.575+0000 7ff421925640 1 --2- 172.21.15.154:0/1924650510 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff41c071290 0x7ff41c0b02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:38.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.575+0000 7ff421124640 1 --2- 172.21.15.154:0/1924650510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff41c0c8d70 0x7ff41c15cfd0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7ff40c00b3f0 tx=0x7ff40c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:38.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.575+0000 7ff4127fc640 1 -- 172.21.15.154:0/1924650510 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40c011830 con 0x7ff41c0c8d70 2023-09-23T03:24:38.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.575+0000 7ff4127fc640 1 -- 172.21.15.154:0/1924650510 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff40c011990 con 0x7ff41c0c8d70 2023-09-23T03:24:38.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.575+0000 7ff4127fc640 1 -- 172.21.15.154:0/1924650510 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40c011d40 con 0x7ff41c0c8d70 2023-09-23T03:24:38.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.575+0000 7ff423b89640 1 -- 172.21.15.154:0/1924650510 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff41c0b0f90 con 0x7ff41c0c8d70 2023-09-23T03:24:38.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.576+0000 7ff423b89640 1 -- 172.21.15.154:0/1924650510 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff41c0b1550 con 0x7ff41c0c8d70 2023-09-23T03:24:38.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.577+0000 7ff423b89640 1 -- 172.21.15.154:0/1924650510 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff41c0b1770 con 0x7ff41c0c8d70 2023-09-23T03:24:38.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.577+0000 7ff4127fc640 1 -- 172.21.15.154:0/1924650510 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7ff40c01a400 con 0x7ff41c0c8d70 2023-09-23T03:24:38.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.578+0000 7ff4127fc640 1 --2- 172.21.15.154:0/1924650510 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff40006af30 0x7ff40006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:38.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.578+0000 7ff4127fc640 1 -- 172.21.15.154:0/1924650510 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6605+0+0 (secure 0 0 0) 0x7ff40c089940 con 0x7ff41c0c8d70 2023-09-23T03:24:38.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.579+0000 7ff421925640 1 --2- 172.21.15.154:0/1924650510 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff40006af30 0x7ff40006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:38.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.579+0000 7ff421925640 1 --2- 172.21.15.154:0/1924650510 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff40006af30 0x7ff40006d3f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff418006fd0 tx=0x7ff418008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:38.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.583+0000 7ff4127fc640 1 -- 172.21.15.154:0/1924650510 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7ff40c054600 con 0x7ff41c0c8d70 2023-09-23T03:24:38.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:38.800+0000 7ff423b89640 1 -- 172.21.15.154:0/1924650510 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7ff41c04e920 con 0x7ff41c0c8d70 2023-09-23T03:24:39.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.367+0000 7ff4127fc640 1 -- 172.21.15.154:0/1924650510 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v69) v1 ==== 162+0+0 (secure 0 0 0) 0x7ff40c057c20 con 0x7ff41c0c8d70 2023-09-23T03:24:39.368 INFO:teuthology.orchestra.run.smithi154.stderr:cephfs marked joinable; MDS may join as newly active. 2023-09-23T03:24:39.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.370+0000 7ff423b89640 1 -- 172.21.15.154:0/1924650510 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff40006af30 msgr2=0x7ff40006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:39.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.370+0000 7ff423b89640 1 --2- 172.21.15.154:0/1924650510 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff40006af30 0x7ff40006d3f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff418006fd0 tx=0x7ff418008040 comp rx=0 tx=0).stop 2023-09-23T03:24:39.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.370+0000 7ff423b89640 1 -- 172.21.15.154:0/1924650510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff41c0c8d70 msgr2=0x7ff41c15cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:39.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.370+0000 7ff423b89640 1 --2- 172.21.15.154:0/1924650510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff41c0c8d70 0x7ff41c15cfd0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7ff40c00b3f0 tx=0x7ff40c00b8b0 comp rx=0 tx=0).stop 2023-09-23T03:24:39.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.371+0000 7ff423b89640 1 -- 172.21.15.154:0/1924650510 shutdown_connections 2023-09-23T03:24:39.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.371+0000 7ff423b89640 1 --2- 172.21.15.154:0/1924650510 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff40006af30 0x7ff40006d3f0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:39.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.371+0000 7ff423b89640 1 --2- 172.21.15.154:0/1924650510 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff41c0c9150 0x7ff41c0c5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:39.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.371+0000 7ff423b89640 1 --2- 172.21.15.154:0/1924650510 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff41c0c8d70 0x7ff41c15cfd0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:39.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.371+0000 7ff423b89640 1 --2- 172.21.15.154:0/1924650510 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff41c071290 0x7ff41c0b02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:39.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.371+0000 7ff423b89640 1 -- 172.21.15.154:0/1924650510 >> 172.21.15.154:0/1924650510 conn(0x7ff41c06c010 msgr2=0x7ff41c0c41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:39.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.371+0000 7ff423b89640 1 -- 172.21.15.154:0/1924650510 shutdown_connections 2023-09-23T03:24:39.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.371+0000 7ff423b89640 1 -- 172.21.15.154:0/1924650510 wait complete. 2023-09-23T03:24:39.391 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:24:39.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.644+0000 7f095359e640 1 Processor -- start 2023-09-23T03:24:39.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.644+0000 7f095359e640 1 -- start start 2023-09-23T03:24:39.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.646+0000 7f095359e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f09540cf500 0x7f09540ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:39.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.646+0000 7f095359e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f09540cf8e0 0x7f09540ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:39.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.646+0000 7f095359e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09540d17f0 con 0x7f095415c4f0 2023-09-23T03:24:39.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.646+0000 7f095359e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f095406be10 con 0x7f09540cf8e0 2023-09-23T03:24:39.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.646+0000 7f095359e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09540d0470 con 0x7f09540cf500 2023-09-23T03:24:39.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.646+0000 7f0951d9b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f09540cf8e0 0x7f09540ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:39.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.646+0000 7f0951d9b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f09540cf8e0 0x7f09540ce590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43044/0 (socket says 172.21.15.154:43044) 2023-09-23T03:24:39.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.646+0000 7f0951d9b640 1 -- 172.21.15.154:0/3379517301 learned_addr learned my addr 172.21.15.154:0/3379517301 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:39.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.646+0000 7f0951d9b640 1 -- 172.21.15.154:0/3379517301 >> v2:172.21.15.190:3300/0 conn(0x7f09540cf500 msgr2=0x7f09540ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:39.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.646+0000 7f0951d9b640 1 --2- 172.21.15.154:0/3379517301 >> v2:172.21.15.190:3300/0 conn(0x7f09540cf500 0x7f09540ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:39.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.646+0000 7f0951d9b640 1 -- 172.21.15.154:0/3379517301 >> v1:172.21.15.190:6789/0 conn(0x7f095415c4f0 legacy=0x7f095415c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:39.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.646+0000 7f0951d9b640 1 -- 172.21.15.154:0/3379517301 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09540d0750 con 0x7f09540cf8e0 2023-09-23T03:24:39.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.647+0000 7f0951d9b640 1 --2- 172.21.15.154:0/3379517301 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09540cf8e0 0x7f09540ce590 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f0944009e50 tx=0x7f094402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef455e25a95128f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:39.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.648+0000 7f095159a640 1 -- 172.21.15.154:0/3379517301 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094402c9f0 con 0x7f09540cf8e0 2023-09-23T03:24:39.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.648+0000 7f095159a640 1 -- 172.21.15.154:0/3379517301 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094402cb50 con 0x7f09540cf8e0 2023-09-23T03:24:39.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.648+0000 7f095159a640 1 -- 172.21.15.154:0/3379517301 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09440343f0 con 0x7f09540cf8e0 2023-09-23T03:24:39.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.648+0000 7f095359e640 1 -- 172.21.15.154:0/3379517301 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09540cf8e0 msgr2=0x7f09540ce590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:39.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.648+0000 7f095359e640 1 --2- 172.21.15.154:0/3379517301 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09540cf8e0 0x7f09540ce590 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f0944009e50 tx=0x7f094402be50 comp rx=0 tx=0).stop 2023-09-23T03:24:39.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.649+0000 7f095359e640 1 -- 172.21.15.154:0/3379517301 shutdown_connections 2023-09-23T03:24:39.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.649+0000 7f095359e640 1 --2- 172.21.15.154:0/3379517301 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f09540cf8e0 0x7f09540ce590 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:39.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.649+0000 7f095359e640 1 --2- 172.21.15.154:0/3379517301 >> v2:172.21.15.190:3300/0 conn(0x7f09540cf500 0x7f09540ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:39.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.649+0000 7f095359e640 1 -- 172.21.15.154:0/3379517301 >> 172.21.15.154:0/3379517301 conn(0x7f095409bbb0 msgr2=0x7f09540d28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:39.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.649+0000 7f095359e640 1 -- 172.21.15.154:0/3379517301 shutdown_connections 2023-09-23T03:24:39.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.649+0000 7f095359e640 1 -- 172.21.15.154:0/3379517301 wait complete. 2023-09-23T03:24:39.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.650+0000 7f095359e640 1 Processor -- start 2023-09-23T03:24:39.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.650+0000 7f095359e640 1 -- start start 2023-09-23T03:24:39.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.651+0000 7f095359e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09540cf500 0x7f095406b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:39.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.651+0000 7f095359e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09540cf8e0 0x7f09541f16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:39.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.651+0000 7f095359e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f095415c4f0 0x7f09541f1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:39.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.651+0000 7f095359e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09540cbbd0 con 0x7f095415c4f0 2023-09-23T03:24:39.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.651+0000 7f095359e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09541f2160 con 0x7f09540cf500 2023-09-23T03:24:39.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.651+0000 7f095359e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f095415c940 con 0x7f09540cf8e0 2023-09-23T03:24:39.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.652+0000 7f0952d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f095415c4f0 0x7f09541f1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:39.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.652+0000 7f0952d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f095415c4f0 0x7f09541f1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43060/0 (socket says 172.21.15.154:43060) 2023-09-23T03:24:39.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.652+0000 7f0952d9d640 1 -- 172.21.15.154:0/3858076433 learned_addr learned my addr 172.21.15.154:0/3858076433 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:39.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.652+0000 7f095259c640 1 --2- 172.21.15.154:0/3858076433 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09540cf500 0x7f095406b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:39.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.652+0000 7f0952d9d640 1 -- 172.21.15.154:0/3858076433 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09540cf8e0 msgr2=0x7f09541f16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:39.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.652+0000 7f0952d9d640 1 --2- 172.21.15.154:0/3858076433 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09540cf8e0 0x7f09541f16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:39.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.652+0000 7f0952d9d640 1 -- 172.21.15.154:0/3858076433 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09540cf500 msgr2=0x7f095406b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:39.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.652+0000 7f0952d9d640 1 --2- 172.21.15.154:0/3858076433 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09540cf500 0x7f095406b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:39.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.652+0000 7f0952d9d640 1 -- 172.21.15.154:0/3858076433 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0944009b30 con 0x7f095415c4f0 2023-09-23T03:24:39.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.652+0000 7f0952d9d640 1 --2- 172.21.15.154:0/3858076433 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f095415c4f0 0x7f09541f1c20 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f094c00c390 tx=0x7f094c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:39.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.653+0000 7f094affd640 1 -- 172.21.15.154:0/3858076433 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c0035f0 con 0x7f095415c4f0 2023-09-23T03:24:39.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.653+0000 7f094affd640 1 -- 172.21.15.154:0/3858076433 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094c003770 con 0x7f095415c4f0 2023-09-23T03:24:39.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.653+0000 7f095359e640 1 -- 172.21.15.154:0/3858076433 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f095415cc20 con 0x7f095415c4f0 2023-09-23T03:24:39.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.653+0000 7f094affd640 1 -- 172.21.15.154:0/3858076433 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c003b20 con 0x7f095415c4f0 2023-09-23T03:24:39.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.653+0000 7f095359e640 1 -- 172.21.15.154:0/3858076433 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f095415d1e0 con 0x7f095415c4f0 2023-09-23T03:24:39.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.654+0000 7f095359e640 1 -- 172.21.15.154:0/3858076433 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f095404e920 con 0x7f095415c4f0 2023-09-23T03:24:39.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.655+0000 7f094affd640 1 -- 172.21.15.154:0/3858076433 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f094c003c80 con 0x7f095415c4f0 2023-09-23T03:24:39.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.656+0000 7f094affd640 1 --2- 172.21.15.154:0/3858076433 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f093406af30 0x7f093406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:39.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.656+0000 7f094affd640 1 -- 172.21.15.154:0/3858076433 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6605+0+0 (secure 0 0 0) 0x7f094c088dc0 con 0x7f095415c4f0 2023-09-23T03:24:39.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.656+0000 7f095259c640 1 --2- 172.21.15.154:0/3858076433 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f093406af30 0x7f093406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:39.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.657+0000 7f095259c640 1 --2- 172.21.15.154:0/3858076433 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f093406af30 0x7f093406d3f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f093c006fd0 tx=0x7f093c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:39.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.660+0000 7f094affd640 1 -- 172.21.15.154:0/3858076433 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f094c053a00 con 0x7f095415c4f0 2023-09-23T03:24:39.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.885+0000 7f095359e640 1 -- 172.21.15.154:0/3858076433 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09540d0000 con 0x7f095415c4f0 2023-09-23T03:24:39.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.886+0000 7f094affd640 1 -- 172.21.15.154:0/3858076433 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f094c057020 con 0x7f095415c4f0 2023-09-23T03:24:39.886 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:39.887 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":69,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/728936136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":728936136},{"type":"v1","addr":"172.21.15.190:6833","nonce":728936136}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":5890,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2277229238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2277229238},{"type":"v1","addr":"172.21.15.190:6835","nonce":2277229238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":7257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/4245928089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4245928089},{"type":"v1","addr":"172.21.15.154:6835","nonce":4245928089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7575,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1588996498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1588996498},{"type":"v1","addr":"172.21.15.154:6837","nonce":1588996498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:24:37.386034+0000","modified":"2023-09-23T03:24:38.801289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[12,13],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-09-23T03:24:39.887 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 69 2023-09-23T03:24:39.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.888+0000 7f095359e640 1 -- 172.21.15.154:0/3858076433 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f093406af30 msgr2=0x7f093406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:39.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.888+0000 7f095359e640 1 --2- 172.21.15.154:0/3858076433 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f093406af30 0x7f093406d3f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f093c006fd0 tx=0x7f093c008040 comp rx=0 tx=0).stop 2023-09-23T03:24:39.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.888+0000 7f095359e640 1 -- 172.21.15.154:0/3858076433 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f095415c4f0 msgr2=0x7f09541f1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:39.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.888+0000 7f095359e640 1 --2- 172.21.15.154:0/3858076433 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f095415c4f0 0x7f09541f1c20 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f094c00c390 tx=0x7f094c00c850 comp rx=0 tx=0).stop 2023-09-23T03:24:39.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.889+0000 7f095359e640 1 -- 172.21.15.154:0/3858076433 shutdown_connections 2023-09-23T03:24:39.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.889+0000 7f095359e640 1 --2- 172.21.15.154:0/3858076433 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f093406af30 0x7f093406d3f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:39.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.889+0000 7f095359e640 1 --2- 172.21.15.154:0/3858076433 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f095415c4f0 0x7f09541f1c20 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:39.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.889+0000 7f095359e640 1 --2- 172.21.15.154:0/3858076433 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f09540cf8e0 0x7f09541f16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:39.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.889+0000 7f095359e640 1 --2- 172.21.15.154:0/3858076433 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f09540cf500 0x7f095406b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:39.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.889+0000 7f095359e640 1 -- 172.21.15.154:0/3858076433 >> 172.21.15.154:0/3858076433 conn(0x7f095409bbb0 msgr2=0x7f09540b75f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:39.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.889+0000 7f095359e640 1 -- 172.21.15.154:0/3858076433 shutdown_connections 2023-09-23T03:24:39.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:39.889+0000 7f095359e640 1 -- 172.21.15.154:0/3858076433 wait complete. 2023-09-23T03:24:39.908 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-09-23T03:24:39.908 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:24:40.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.117+0000 7fd69fb65640 1 Processor -- start 2023-09-23T03:24:40.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.118+0000 7fd69fb65640 1 -- start start 2023-09-23T03:24:40.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.119+0000 7fd69fb65640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6980b1b20 0x7fd6980ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:40.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.119+0000 7fd69fb65640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd69815ac30 con 0x7fd6980cee70 2023-09-23T03:24:40.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.119+0000 7fd69fb65640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6980b1f70 con 0x7fd6980acce0 2023-09-23T03:24:40.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.119+0000 7fd69fb65640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6980b2e90 con 0x7fd6980b1b20 2023-09-23T03:24:40.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.119+0000 7fd69e102640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fd6980cee70 0x7fd6980ab140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:45574/0 (socket says 172.21.15.154:45574) 2023-09-23T03:24:40.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.119+0000 7fd69e102640 1 -- 172.21.15.154:0/4039728643 learned_addr learned my addr 172.21.15.154:0/4039728643 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:24:40.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.120+0000 7fd69d100640 1 --2- 172.21.15.154:0/4039728643 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6980b1b20 0x7fd6980ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:40.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.120+0000 7fd69c8ff640 1 -- 172.21.15.154:0/4039728643 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1323123957 0 0) 0x7fd690001490 con 0x7fd6980cee70 2023-09-23T03:24:40.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.120+0000 7fd69c8ff640 1 -- 172.21.15.154:0/4039728643 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd684003520 con 0x7fd6980cee70 2023-09-23T03:24:40.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.120+0000 7fd69c8ff640 1 -- 172.21.15.154:0/4039728643 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3339961847 0 0) 0x7fd684003520 con 0x7fd6980cee70 2023-09-23T03:24:40.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.120+0000 7fd69c8ff640 1 -- 172.21.15.154:0/4039728643 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6980b1b20 msgr2=0x7fd6980ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:40.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.120+0000 7fd69c8ff640 1 --2- 172.21.15.154:0/4039728643 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6980b1b20 0x7fd6980ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:40.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.120+0000 7fd69c8ff640 1 -- 172.21.15.154:0/4039728643 >> v1:172.21.15.190:6789/0 conn(0x7fd6980acce0 legacy=0x7fd69815ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:40.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.120+0000 7fd69c8ff640 1 -- 172.21.15.154:0/4039728643 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6980b2210 con 0x7fd6980cee70 2023-09-23T03:24:40.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.120+0000 7fd69c8ff640 1 -- 172.21.15.154:0/4039728643 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fd690001b80 con 0x7fd6980cee70 2023-09-23T03:24:40.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.120+0000 7fd69c8ff640 1 -- 172.21.15.154:0/4039728643 >> v1:172.21.15.154:6789/0 conn(0x7fd6980cee70 legacy=0x7fd6980ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:40.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.120+0000 7fd69c8ff640 1 --2- 172.21.15.154:0/4039728643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6840039a0 0x7fd684003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:40.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.120+0000 7fd69c8ff640 1 --2- 172.21.15.154:0/4039728643 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6980b1b20 0x7fd6840047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:40.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.120+0000 7fd69c8ff640 1 --2- 172.21.15.154:0/4039728643 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd684004d00 0x7fd684005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:40.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.120+0000 7fd69c8ff640 1 -- 172.21.15.154:0/4039728643 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6980b2e90 con 0x7fd684004d00 2023-09-23T03:24:40.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.120+0000 7fd69c8ff640 1 -- 172.21.15.154:0/4039728643 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68400bab0 con 0x7fd6840039a0 2023-09-23T03:24:40.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.120+0000 7fd69c8ff640 1 -- 172.21.15.154:0/4039728643 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd68400bc70 con 0x7fd6980b1b20 2023-09-23T03:24:40.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.121+0000 7fd69e102640 1 --2- 172.21.15.154:0/4039728643 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd684004d00 0x7fd684005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:40.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.121+0000 7fd69d901640 1 --2- 172.21.15.154:0/4039728643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6840039a0 0x7fd684003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:40.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.121+0000 7fd69e102640 1 -- 172.21.15.154:0/4039728643 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6980b1b20 msgr2=0x7fd6840047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:40.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.121+0000 7fd69e102640 1 --2- 172.21.15.154:0/4039728643 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd6980b1b20 0x7fd6840047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:40.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.121+0000 7fd69e102640 1 -- 172.21.15.154:0/4039728643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6840039a0 msgr2=0x7fd684003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:40.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.121+0000 7fd69e102640 1 --2- 172.21.15.154:0/4039728643 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd6840039a0 0x7fd684003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:40.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.121+0000 7fd69e102640 1 -- 172.21.15.154:0/4039728643 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd684003520 con 0x7fd684004d00 2023-09-23T03:24:40.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.122+0000 7fd69e102640 1 --2- 172.21.15.154:0/4039728643 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd684004d00 0x7fd684005180 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fd69000a830 tx=0x7fd6840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce905ffd16dd9a81 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:40.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.122+0000 7fd69c8ff640 1 -- 172.21.15.154:0/4039728643 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd690034630 con 0x7fd684004d00 2023-09-23T03:24:40.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.122+0000 7fd69c8ff640 1 -- 172.21.15.154:0/4039728643 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd69002f070 con 0x7fd684004d00 2023-09-23T03:24:40.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.122+0000 7fd69c8ff640 1 -- 172.21.15.154:0/4039728643 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6900349f0 con 0x7fd684004d00 2023-09-23T03:24:40.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.123+0000 7fd69fb65640 1 -- 172.21.15.154:0/4039728643 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd684004d00 msgr2=0x7fd684005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:40.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.123+0000 7fd69fb65640 1 --2- 172.21.15.154:0/4039728643 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd684004d00 0x7fd684005180 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fd69000a830 tx=0x7fd6840057f0 comp rx=0 tx=0).stop 2023-09-23T03:24:40.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.123+0000 7fd69e102640 1 -- 172.21.15.154:0/4039728643 reap_dead start 2023-09-23T03:24:40.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.123+0000 7fd69fb65640 1 -- 172.21.15.154:0/4039728643 shutdown_connections 2023-09-23T03:24:40.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.123+0000 7fd69fb65640 1 -- 172.21.15.154:0/4039728643 >> 172.21.15.154:0/4039728643 conn(0x7fd69806a0f0 msgr2=0x7fd69806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:40.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.123+0000 7fd69fb65640 1 -- 172.21.15.154:0/4039728643 shutdown_connections 2023-09-23T03:24:40.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.123+0000 7fd69fb65640 1 -- 172.21.15.154:0/4039728643 wait complete. 2023-09-23T03:24:40.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.125+0000 7fd69fb65640 1 Processor -- start 2023-09-23T03:24:40.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.125+0000 7fd69fb65640 1 -- start start 2023-09-23T03:24:40.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.126+0000 7fd69fb65640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd684004d00 0x7fd6981f1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:40.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.126+0000 7fd69fb65640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd69815a3d0 0x7fd6980c2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:40.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.126+0000 7fd69fb65640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6980c2cd0 0x7fd6980ae600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:40.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.126+0000 7fd69fb65640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6980cf320 con 0x7fd6980c2cd0 2023-09-23T03:24:40.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.126+0000 7fd69fb65640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd698155550 con 0x7fd69815a3d0 2023-09-23T03:24:40.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.126+0000 7fd69fb65640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd698155220 con 0x7fd684004d00 2023-09-23T03:24:40.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.126+0000 7fd69e102640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6980c2cd0 0x7fd6980ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:40.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.127+0000 7fd69e102640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6980c2cd0 0x7fd6980ae600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43086/0 (socket says 172.21.15.154:43086) 2023-09-23T03:24:40.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.127+0000 7fd69e102640 1 -- 172.21.15.154:0/3356218052 learned_addr learned my addr 172.21.15.154:0/3356218052 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:40.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.127+0000 7fd69d901640 1 --2- 172.21.15.154:0/3356218052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd684004d00 0x7fd6981f1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:40.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.127+0000 7fd69d100640 1 --2- 172.21.15.154:0/3356218052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd69815a3d0 0x7fd6980c2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:40.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.127+0000 7fd69e102640 1 -- 172.21.15.154:0/3356218052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd684004d00 msgr2=0x7fd6981f1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:40.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.127+0000 7fd69e102640 1 --2- 172.21.15.154:0/3356218052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd684004d00 0x7fd6981f1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:40.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.127+0000 7fd69e102640 1 -- 172.21.15.154:0/3356218052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd69815a3d0 msgr2=0x7fd6980c2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:40.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.127+0000 7fd69e102640 1 --2- 172.21.15.154:0/3356218052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd69815a3d0 0x7fd6980c2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:40.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.127+0000 7fd69e102640 1 -- 172.21.15.154:0/3356218052 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6981ee590 con 0x7fd6980c2cd0 2023-09-23T03:24:40.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.127+0000 7fd69e102640 1 --2- 172.21.15.154:0/3356218052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6980c2cd0 0x7fd6980ae600 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fd69000a830 tx=0x7fd690001f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:40.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.128+0000 7fd68e7fc640 1 -- 172.21.15.154:0/3356218052 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd690001b40 con 0x7fd6980c2cd0 2023-09-23T03:24:40.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.128+0000 7fd68e7fc640 1 -- 172.21.15.154:0/3356218052 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd690001490 con 0x7fd6980c2cd0 2023-09-23T03:24:40.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.128+0000 7fd68e7fc640 1 -- 172.21.15.154:0/3356218052 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd690034630 con 0x7fd6980c2cd0 2023-09-23T03:24:40.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.128+0000 7fd69fb65640 1 -- 172.21.15.154:0/3356218052 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6981f2510 con 0x7fd6980c2cd0 2023-09-23T03:24:40.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.128+0000 7fd69fb65640 1 -- 172.21.15.154:0/3356218052 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6981f2ad0 con 0x7fd6980c2cd0 2023-09-23T03:24:40.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.129+0000 7fd69fb65640 1 -- 172.21.15.154:0/3356218052 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd698073eb0 con 0x7fd6980c2cd0 2023-09-23T03:24:40.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.130+0000 7fd68e7fc640 1 -- 172.21.15.154:0/3356218052 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fd690031030 con 0x7fd6980c2cd0 2023-09-23T03:24:40.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.131+0000 7fd68e7fc640 1 --2- 172.21.15.154:0/3356218052 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd67c06af60 0x7fd67c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:40.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.131+0000 7fd68e7fc640 1 -- 172.21.15.154:0/3356218052 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6605+0+0 (secure 0 0 0) 0x7fd6900ab690 con 0x7fd6980c2cd0 2023-09-23T03:24:40.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.131+0000 7fd69d901640 1 --2- 172.21.15.154:0/3356218052 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd67c06af60 0x7fd67c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:40.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.132+0000 7fd69d901640 1 --2- 172.21.15.154:0/3356218052 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd67c06af60 0x7fd67c06d420 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd6940055b0 tx=0x7fd694009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:40.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.136+0000 7fd68e7fc640 1 -- 172.21.15.154:0/3356218052 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fd6900762b0 con 0x7fd6980c2cd0 2023-09-23T03:24:40.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.362+0000 7fd69fb65640 1 -- 172.21.15.154:0/3356218052 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd698073eb0 con 0x7fd6980c2cd0 2023-09-23T03:24:40.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.363+0000 7fd68e7fc640 1 -- 172.21.15.154:0/3356218052 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4919 (secure 0 0 0) 0x7fd6900798d0 con 0x7fd6980c2cd0 2023-09-23T03:24:40.365 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:40.365 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":69,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/728936136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":728936136},{"type":"v1","addr":"172.21.15.190:6833","nonce":728936136}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":5890,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2277229238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2277229238},{"type":"v1","addr":"172.21.15.190:6835","nonce":2277229238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":7257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/4245928089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4245928089},{"type":"v1","addr":"172.21.15.154:6835","nonce":4245928089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7575,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1588996498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1588996498},{"type":"v1","addr":"172.21.15.154:6837","nonce":1588996498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:24:37.386034+0000","modified":"2023-09-23T03:24:38.801289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[12,13],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-09-23T03:24:40.365 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 69 2023-09-23T03:24:40.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.366+0000 7fd69fb65640 1 -- 172.21.15.154:0/3356218052 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd67c06af60 msgr2=0x7fd67c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:40.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.366+0000 7fd69fb65640 1 --2- 172.21.15.154:0/3356218052 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd67c06af60 0x7fd67c06d420 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd6940055b0 tx=0x7fd694009040 comp rx=0 tx=0).stop 2023-09-23T03:24:40.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.366+0000 7fd69fb65640 1 -- 172.21.15.154:0/3356218052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6980c2cd0 msgr2=0x7fd6980ae600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:40.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.366+0000 7fd69fb65640 1 --2- 172.21.15.154:0/3356218052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6980c2cd0 0x7fd6980ae600 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fd69000a830 tx=0x7fd690001f10 comp rx=0 tx=0).stop 2023-09-23T03:24:40.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.366+0000 7fd69fb65640 1 -- 172.21.15.154:0/3356218052 shutdown_connections 2023-09-23T03:24:40.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.366+0000 7fd69fb65640 1 --2- 172.21.15.154:0/3356218052 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd67c06af60 0x7fd67c06d420 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:40.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.366+0000 7fd69fb65640 1 --2- 172.21.15.154:0/3356218052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6980c2cd0 0x7fd6980ae600 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:40.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.366+0000 7fd69fb65640 1 --2- 172.21.15.154:0/3356218052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd69815a3d0 0x7fd6980c2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:40.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.366+0000 7fd69fb65640 1 --2- 172.21.15.154:0/3356218052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd684004d00 0x7fd6981f1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:40.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.366+0000 7fd69fb65640 1 -- 172.21.15.154:0/3356218052 >> 172.21.15.154:0/3356218052 conn(0x7fd69806a0f0 msgr2=0x7fd6980cd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:40.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.366+0000 7fd69fb65640 1 -- 172.21.15.154:0/3356218052 shutdown_connections 2023-09-23T03:24:40.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.366+0000 7fd69fb65640 1 -- 172.21.15.154:0/3356218052 wait complete. 2023-09-23T03:24:40.386 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:24:40.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.578+0000 7f1b0a3b6640 1 Processor -- start 2023-09-23T03:24:40.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.579+0000 7f1b0a3b6640 1 -- start start 2023-09-23T03:24:40.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.580+0000 7f1b0a3b6640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1b040cf040 0x7f1b04160880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:40.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.580+0000 7f1b0a3b6640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1b04160dc0 0x7f1b040b1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:40.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.580+0000 7f1b0a3b6640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b040cd530 con 0x7f1b0415caf0 2023-09-23T03:24:40.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.580+0000 7f1b0a3b6640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b0415ced0 con 0x7f1b040cf040 2023-09-23T03:24:40.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.580+0000 7f1b0a3b6640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b040ce2f0 con 0x7f1b04160dc0 2023-09-23T03:24:40.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.580+0000 7f1afbfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1b040cf040 0x7f1b04160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:40.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.580+0000 7f1afbfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1b040cf040 0x7f1b04160880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43098/0 (socket says 172.21.15.154:43098) 2023-09-23T03:24:40.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.580+0000 7f1afbfff640 1 -- 172.21.15.154:0/4237800323 learned_addr learned my addr 172.21.15.154:0/4237800323 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:40.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.581+0000 7f1afbfff640 1 -- 172.21.15.154:0/4237800323 >> v2:172.21.15.190:3300/0 conn(0x7f1b04160dc0 msgr2=0x7f1b040b1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:40.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.581+0000 7f1b08953640 1 --2- 172.21.15.154:0/4237800323 >> v2:172.21.15.190:3300/0 conn(0x7f1b04160dc0 0x7f1b040b1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:40.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.581+0000 7f1afbfff640 1 --2- 172.21.15.154:0/4237800323 >> v2:172.21.15.190:3300/0 conn(0x7f1b04160dc0 0x7f1b040b1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:40.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.581+0000 7f1afbfff640 1 -- 172.21.15.154:0/4237800323 >> v1:172.21.15.190:6789/0 conn(0x7f1b0415caf0 legacy=0x7f1b040ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:40.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.581+0000 7f1afbfff640 1 -- 172.21.15.154:0/4237800323 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b040cd810 con 0x7f1b040cf040 2023-09-23T03:24:40.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.581+0000 7f1b08953640 1 --2- 172.21.15.154:0/4237800323 >> v2:172.21.15.190:3300/0 conn(0x7f1b04160dc0 0x7f1b040b1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:40.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.582+0000 7f1afbfff640 1 --2- 172.21.15.154:0/4237800323 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b040cf040 0x7f1b04160880 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f1b040ad620 tx=0x7f1aec02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fa44d9459332530 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:40.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.582+0000 7f1b037fe640 1 -- 172.21.15.154:0/4237800323 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aec031870 con 0x7f1b040cf040 2023-09-23T03:24:40.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.582+0000 7f1b037fe640 1 -- 172.21.15.154:0/4237800323 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aec0319d0 con 0x7f1b040cf040 2023-09-23T03:24:40.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.582+0000 7f1b037fe640 1 -- 172.21.15.154:0/4237800323 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aec031d60 con 0x7f1b040cf040 2023-09-23T03:24:40.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.583+0000 7f1b0a3b6640 1 -- 172.21.15.154:0/4237800323 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b040cf040 msgr2=0x7f1b04160880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:40.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.583+0000 7f1b0a3b6640 1 --2- 172.21.15.154:0/4237800323 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b040cf040 0x7f1b04160880 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f1b040ad620 tx=0x7f1aec02b910 comp rx=0 tx=0).stop 2023-09-23T03:24:40.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.583+0000 7f1b0a3b6640 1 -- 172.21.15.154:0/4237800323 shutdown_connections 2023-09-23T03:24:40.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.583+0000 7f1b0a3b6640 1 --2- 172.21.15.154:0/4237800323 >> v2:172.21.15.190:3300/0 conn(0x7f1b04160dc0 0x7f1b040b1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:40.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.583+0000 7f1b0a3b6640 1 --2- 172.21.15.154:0/4237800323 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b040cf040 0x7f1b04160880 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:40.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.583+0000 7f1b0a3b6640 1 -- 172.21.15.154:0/4237800323 >> 172.21.15.154:0/4237800323 conn(0x7f1b0406a0f0 msgr2=0x7f1b0406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:40.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.583+0000 7f1b0a3b6640 1 -- 172.21.15.154:0/4237800323 shutdown_connections 2023-09-23T03:24:40.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.583+0000 7f1b0a3b6640 1 -- 172.21.15.154:0/4237800323 wait complete. 2023-09-23T03:24:40.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.585+0000 7f1b0a3b6640 1 Processor -- start 2023-09-23T03:24:40.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.585+0000 7f1b0a3b6640 1 -- start start 2023-09-23T03:24:40.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.586+0000 7f1b0a3b6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b040cf040 0x7f1b041f4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:40.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.586+0000 7f1b0a3b6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b0415caf0 0x7f1b0415b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:40.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.586+0000 7f1b0a3b6640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b04160dc0 0x7f1b0415b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:40.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.586+0000 7f1b0a3b6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0415ed40 con 0x7f1b0415caf0 2023-09-23T03:24:40.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.586+0000 7f1b0a3b6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0415e0c0 con 0x7f1b040cf040 2023-09-23T03:24:40.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.586+0000 7f1b0a3b6640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0415e3a0 con 0x7f1b04160dc0 2023-09-23T03:24:40.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.587+0000 7f1afbfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b0415caf0 0x7f1b0415b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:40.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.587+0000 7f1afbfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b0415caf0 0x7f1b0415b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43114/0 (socket says 172.21.15.154:43114) 2023-09-23T03:24:40.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.587+0000 7f1afbfff640 1 -- 172.21.15.154:0/840454262 learned_addr learned my addr 172.21.15.154:0/840454262 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:40.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.587+0000 7f1b03fff640 1 --2- 172.21.15.154:0/840454262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b040cf040 0x7f1b041f4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:40.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.587+0000 7f1afbfff640 1 -- 172.21.15.154:0/840454262 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b04160dc0 msgr2=0x7f1b0415b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:40.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.587+0000 7f1afbfff640 1 --2- 172.21.15.154:0/840454262 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b04160dc0 0x7f1b0415b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:40.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.587+0000 7f1afbfff640 1 -- 172.21.15.154:0/840454262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b040cf040 msgr2=0x7f1b041f4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:40.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.587+0000 7f1afbfff640 1 --2- 172.21.15.154:0/840454262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b040cf040 0x7f1b041f4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:40.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.587+0000 7f1afbfff640 1 -- 172.21.15.154:0/840454262 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aec0094d0 con 0x7f1b0415caf0 2023-09-23T03:24:40.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.587+0000 7f1b03fff640 1 --2- 172.21.15.154:0/840454262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b040cf040 0x7f1b041f4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:40.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.587+0000 7f1afbfff640 1 --2- 172.21.15.154:0/840454262 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b0415caf0 0x7f1b0415b130 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f1aec000c00 tx=0x7f1aec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:40.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.588+0000 7f1b017fa640 1 -- 172.21.15.154:0/840454262 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aec031840 con 0x7f1b0415caf0 2023-09-23T03:24:40.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.588+0000 7f1b017fa640 1 -- 172.21.15.154:0/840454262 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aec0443f0 con 0x7f1b0415caf0 2023-09-23T03:24:40.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.588+0000 7f1b017fa640 1 -- 172.21.15.154:0/840454262 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aec0447a0 con 0x7f1b0415caf0 2023-09-23T03:24:40.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.588+0000 7f1b0a3b6640 1 -- 172.21.15.154:0/840454262 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b0415e680 con 0x7f1b0415caf0 2023-09-23T03:24:40.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.588+0000 7f1b0a3b6640 1 -- 172.21.15.154:0/840454262 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b041f52f0 con 0x7f1b0415caf0 2023-09-23T03:24:40.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.589+0000 7f1b0a3b6640 1 -- 172.21.15.154:0/840454262 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b04073eb0 con 0x7f1b0415caf0 2023-09-23T03:24:40.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.590+0000 7f1b017fa640 1 -- 172.21.15.154:0/840454262 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f1aec04d450 con 0x7f1b0415caf0 2023-09-23T03:24:40.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.591+0000 7f1b017fa640 1 --2- 172.21.15.154:0/840454262 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1ae406af30 0x7f1ae406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:40.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.591+0000 7f1b017fa640 1 -- 172.21.15.154:0/840454262 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6605+0+0 (secure 0 0 0) 0x7f1aec02d040 con 0x7f1b0415caf0 2023-09-23T03:24:40.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.591+0000 7f1b03fff640 1 --2- 172.21.15.154:0/840454262 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1ae406af30 0x7f1ae406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:40.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.592+0000 7f1b03fff640 1 --2- 172.21.15.154:0/840454262 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1ae406af30 0x7f1ae406d3f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f1b040c50d0 tx=0x7f1af4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:40.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.595+0000 7f1b017fa640 1 -- 172.21.15.154:0/840454262 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f1aec076f80 con 0x7f1b0415caf0 2023-09-23T03:24:40.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.810+0000 7f1b0a3b6640 1 -- 172.21.15.154:0/840454262 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b0406cb10 con 0x7f1b0415caf0 2023-09-23T03:24:40.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.811+0000 7f1b017fa640 1 -- 172.21.15.154:0/840454262 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f1aec07a5a0 con 0x7f1b0415caf0 2023-09-23T03:24:40.813 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:40.813 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":69,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/728936136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":728936136},{"type":"v1","addr":"172.21.15.190:6833","nonce":728936136}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":5890,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2277229238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2277229238},{"type":"v1","addr":"172.21.15.190:6835","nonce":2277229238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":7257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/4245928089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4245928089},{"type":"v1","addr":"172.21.15.154:6835","nonce":4245928089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7575,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1588996498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1588996498},{"type":"v1","addr":"172.21.15.154:6837","nonce":1588996498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:24:37.386034+0000","modified":"2023-09-23T03:24:38.801289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[12,13],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-09-23T03:24:40.813 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 69 2023-09-23T03:24:40.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.814+0000 7f1b0a3b6640 1 -- 172.21.15.154:0/840454262 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1ae406af30 msgr2=0x7f1ae406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:40.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.814+0000 7f1b0a3b6640 1 --2- 172.21.15.154:0/840454262 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1ae406af30 0x7f1ae406d3f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f1b040c50d0 tx=0x7f1af4009040 comp rx=0 tx=0).stop 2023-09-23T03:24:40.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.814+0000 7f1b0a3b6640 1 -- 172.21.15.154:0/840454262 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b0415caf0 msgr2=0x7f1b0415b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:40.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.814+0000 7f1b0a3b6640 1 --2- 172.21.15.154:0/840454262 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b0415caf0 0x7f1b0415b130 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f1aec000c00 tx=0x7f1aec0046c0 comp rx=0 tx=0).stop 2023-09-23T03:24:40.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.814+0000 7f1b0a3b6640 1 -- 172.21.15.154:0/840454262 shutdown_connections 2023-09-23T03:24:40.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.814+0000 7f1b0a3b6640 1 --2- 172.21.15.154:0/840454262 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1ae406af30 0x7f1ae406d3f0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:40.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.814+0000 7f1b0a3b6640 1 --2- 172.21.15.154:0/840454262 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b04160dc0 0x7f1b0415b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:40.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.814+0000 7f1b0a3b6640 1 --2- 172.21.15.154:0/840454262 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b0415caf0 0x7f1b0415b130 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:40.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.814+0000 7f1b0a3b6640 1 --2- 172.21.15.154:0/840454262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b040cf040 0x7f1b041f4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:40.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.814+0000 7f1b0a3b6640 1 -- 172.21.15.154:0/840454262 >> 172.21.15.154:0/840454262 conn(0x7f1b0406a0f0 msgr2=0x7f1b0406cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:40.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.814+0000 7f1b0a3b6640 1 -- 172.21.15.154:0/840454262 shutdown_connections 2023-09-23T03:24:40.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:40.814+0000 7f1b0a3b6640 1 -- 172.21.15.154:0/840454262 wait complete. 2023-09-23T03:24:40.833 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:24:41.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.043+0000 7fac7e37c640 1 Processor -- start 2023-09-23T03:24:41.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.043+0000 7fac7e37c640 1 -- start start 2023-09-23T03:24:41.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.045+0000 7fac7e37c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fac780ada30 0x7fac780ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:41.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.045+0000 7fac7e37c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fac780cf130 0x7fac7815d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:41.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.045+0000 7fac7e37c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac7806ceb0 con 0x7fac780b2fe0 2023-09-23T03:24:41.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.045+0000 7fac7e37c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac780ade10 con 0x7fac780cf130 2023-09-23T03:24:41.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.045+0000 7fac7e37c640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac78159c90 con 0x7fac780ada30 2023-09-23T03:24:41.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.045+0000 7fac7c919640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fac780cf130 0x7fac7815d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:41.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.045+0000 7fac7c919640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fac780cf130 0x7fac7815d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43124/0 (socket says 172.21.15.154:43124) 2023-09-23T03:24:41.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.045+0000 7fac7c919640 1 -- 172.21.15.154:0/1874364508 learned_addr learned my addr 172.21.15.154:0/1874364508 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:41.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.045+0000 7fac777fe640 1 --1- 172.21.15.154:0/1874364508 >> v1:172.21.15.154:6789/0 conn(0x7fac780b2fe0 0x7fac780cea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:45584/0 (socket says 172.21.15.154:45584) 2023-09-23T03:24:41.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.045+0000 7fac77fff640 1 --2- 172.21.15.154:0/1874364508 >> v2:172.21.15.190:3300/0 conn(0x7fac780ada30 0x7fac780ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:41.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.046+0000 7fac7c919640 1 -- 172.21.15.154:0/1874364508 >> v2:172.21.15.190:3300/0 conn(0x7fac780ada30 msgr2=0x7fac780ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:41.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.046+0000 7fac7c919640 1 --2- 172.21.15.154:0/1874364508 >> v2:172.21.15.190:3300/0 conn(0x7fac780ada30 0x7fac780ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:41.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.046+0000 7fac7c919640 1 -- 172.21.15.154:0/1874364508 >> v1:172.21.15.154:6789/0 conn(0x7fac780b2fe0 legacy=0x7fac780cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:41.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.046+0000 7fac7c919640 1 -- 172.21.15.154:0/1874364508 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac7806c110 con 0x7fac780cf130 2023-09-23T03:24:41.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.046+0000 7fac76ffd640 1 -- 172.21.15.154:0/1874364508 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1568742140 0 0) 0x7fac6c001110 con 0x7fac780b2fe0 2023-09-23T03:24:41.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.047+0000 7fac7c919640 1 --2- 172.21.15.154:0/1874364508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac780cf130 0x7fac7815d380 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fac7806d990 tx=0x7fac7002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21f682e5f63decf9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:41.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.047+0000 7fac76ffd640 1 -- 172.21.15.154:0/1874364508 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7002cb20 con 0x7fac780cf130 2023-09-23T03:24:41.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.047+0000 7fac76ffd640 1 -- 172.21.15.154:0/1874364508 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac7002cc80 con 0x7fac780cf130 2023-09-23T03:24:41.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.047+0000 7fac76ffd640 1 -- 172.21.15.154:0/1874364508 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac700343f0 con 0x7fac780cf130 2023-09-23T03:24:41.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.048+0000 7fac7e37c640 1 -- 172.21.15.154:0/1874364508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac780cf130 msgr2=0x7fac7815d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:41.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.048+0000 7fac7e37c640 1 --2- 172.21.15.154:0/1874364508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac780cf130 0x7fac7815d380 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fac7806d990 tx=0x7fac7002bf80 comp rx=0 tx=0).stop 2023-09-23T03:24:41.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.048+0000 7fac7e37c640 1 -- 172.21.15.154:0/1874364508 shutdown_connections 2023-09-23T03:24:41.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.048+0000 7fac7e37c640 1 --2- 172.21.15.154:0/1874364508 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac780cf130 0x7fac7815d380 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:41.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.048+0000 7fac7e37c640 1 --2- 172.21.15.154:0/1874364508 >> v2:172.21.15.190:3300/0 conn(0x7fac780ada30 0x7fac780ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:41.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.048+0000 7fac7e37c640 1 -- 172.21.15.154:0/1874364508 >> 172.21.15.154:0/1874364508 conn(0x7fac7806a0f0 msgr2=0x7fac7806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:41.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.048+0000 7fac7e37c640 1 -- 172.21.15.154:0/1874364508 shutdown_connections 2023-09-23T03:24:41.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.048+0000 7fac7e37c640 1 -- 172.21.15.154:0/1874364508 wait complete. 2023-09-23T03:24:41.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.050+0000 7fac7e37c640 1 Processor -- start 2023-09-23T03:24:41.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.050+0000 7fac7e37c640 1 -- start start 2023-09-23T03:24:41.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.051+0000 7fac7e37c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac780ada30 0x7fac78094610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:41.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.051+0000 7fac7e37c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac780b2fe0 0x7fac780ac670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:41.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.051+0000 7fac7e37c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac780cf130 0x7fac780acbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:41.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.051+0000 7fac7e37c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac780c1bd0 con 0x7fac780b2fe0 2023-09-23T03:24:41.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.051+0000 7fac7e37c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac78070f30 con 0x7fac780ada30 2023-09-23T03:24:41.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.051+0000 7fac7e37c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac78071180 con 0x7fac780cf130 2023-09-23T03:24:41.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.051+0000 7fac777fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac780b2fe0 0x7fac780ac670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:41.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.051+0000 7fac777fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac780b2fe0 0x7fac780ac670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43126/0 (socket says 172.21.15.154:43126) 2023-09-23T03:24:41.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.051+0000 7fac777fe640 1 -- 172.21.15.154:0/1479697152 learned_addr learned my addr 172.21.15.154:0/1479697152 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:41.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.051+0000 7fac77fff640 1 --2- 172.21.15.154:0/1479697152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac780ada30 0x7fac78094610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:41.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.051+0000 7fac777fe640 1 -- 172.21.15.154:0/1479697152 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac780cf130 msgr2=0x7fac780acbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:41.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.051+0000 7fac777fe640 1 --2- 172.21.15.154:0/1479697152 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac780cf130 0x7fac780acbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:41.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.051+0000 7fac777fe640 1 -- 172.21.15.154:0/1479697152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac780ada30 msgr2=0x7fac78094610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:41.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.051+0000 7fac777fe640 1 --2- 172.21.15.154:0/1479697152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac780ada30 0x7fac78094610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:41.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.051+0000 7fac777fe640 1 -- 172.21.15.154:0/1479697152 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac70009b30 con 0x7fac780b2fe0 2023-09-23T03:24:41.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.052+0000 7fac777fe640 1 --2- 172.21.15.154:0/1479697152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac780b2fe0 0x7fac780ac670 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fac6c00c7b0 tx=0x7fac6c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:41.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.052+0000 7fac74ff9640 1 -- 172.21.15.154:0/1479697152 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6c0072d0 con 0x7fac780b2fe0 2023-09-23T03:24:41.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.052+0000 7fac74ff9640 1 -- 172.21.15.154:0/1479697152 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac6c007430 con 0x7fac780b2fe0 2023-09-23T03:24:41.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.052+0000 7fac74ff9640 1 -- 172.21.15.154:0/1479697152 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6c001d10 con 0x7fac780b2fe0 2023-09-23T03:24:41.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.052+0000 7fac7e37c640 1 -- 172.21.15.154:0/1479697152 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac78071460 con 0x7fac780b2fe0 2023-09-23T03:24:41.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.052+0000 7fac7e37c640 1 -- 172.21.15.154:0/1479697152 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac78071a20 con 0x7fac780b2fe0 2023-09-23T03:24:41.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.054+0000 7fac7e37c640 1 -- 172.21.15.154:0/1479697152 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac78073eb0 con 0x7fac780b2fe0 2023-09-23T03:24:41.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.054+0000 7fac74ff9640 1 -- 172.21.15.154:0/1479697152 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fac6c0047e0 con 0x7fac780b2fe0 2023-09-23T03:24:41.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.055+0000 7fac74ff9640 1 --2- 172.21.15.154:0/1479697152 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fac6406af60 0x7fac6406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:41.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.055+0000 7fac74ff9640 1 -- 172.21.15.154:0/1479697152 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6605+0+0 (secure 0 0 0) 0x7fac6c088a10 con 0x7fac780b2fe0 2023-09-23T03:24:41.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.056+0000 7fac77fff640 1 --2- 172.21.15.154:0/1479697152 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fac6406af60 0x7fac6406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:41.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.056+0000 7fac77fff640 1 --2- 172.21.15.154:0/1479697152 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fac6406af60 0x7fac6406d420 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fac68009770 tx=0x7fac68008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:41.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.060+0000 7fac74ff9640 1 -- 172.21.15.154:0/1479697152 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fac6c053760 con 0x7fac780b2fe0 2023-09-23T03:24:41.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.287+0000 7fac7e37c640 1 -- 172.21.15.154:0/1479697152 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac7806ceb0 con 0x7fac780b2fe0 2023-09-23T03:24:41.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.288+0000 7fac74ff9640 1 -- 172.21.15.154:0/1479697152 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4919 (secure 0 0 0) 0x7fac7806ceb0 con 0x7fac780b2fe0 2023-09-23T03:24:41.290 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:41.290 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":69,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/728936136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":728936136},{"type":"v1","addr":"172.21.15.190:6833","nonce":728936136}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":5890,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2277229238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2277229238},{"type":"v1","addr":"172.21.15.190:6835","nonce":2277229238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":7257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/4245928089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4245928089},{"type":"v1","addr":"172.21.15.154:6835","nonce":4245928089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7575,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1588996498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1588996498},{"type":"v1","addr":"172.21.15.154:6837","nonce":1588996498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:24:37.386034+0000","modified":"2023-09-23T03:24:38.801289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[12,13],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-09-23T03:24:41.290 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 69 2023-09-23T03:24:41.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.291+0000 7fac7e37c640 1 -- 172.21.15.154:0/1479697152 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fac6406af60 msgr2=0x7fac6406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:41.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.291+0000 7fac7e37c640 1 --2- 172.21.15.154:0/1479697152 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fac6406af60 0x7fac6406d420 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fac68009770 tx=0x7fac68008040 comp rx=0 tx=0).stop 2023-09-23T03:24:41.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.291+0000 7fac7e37c640 1 -- 172.21.15.154:0/1479697152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac780b2fe0 msgr2=0x7fac780ac670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:41.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.291+0000 7fac7e37c640 1 --2- 172.21.15.154:0/1479697152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac780b2fe0 0x7fac780ac670 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fac6c00c7b0 tx=0x7fac6c00cc70 comp rx=0 tx=0).stop 2023-09-23T03:24:41.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.291+0000 7fac7e37c640 1 -- 172.21.15.154:0/1479697152 shutdown_connections 2023-09-23T03:24:41.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.291+0000 7fac7e37c640 1 --2- 172.21.15.154:0/1479697152 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fac6406af60 0x7fac6406d420 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:41.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.291+0000 7fac7e37c640 1 --2- 172.21.15.154:0/1479697152 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fac780cf130 0x7fac780acbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:41.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.291+0000 7fac7e37c640 1 --2- 172.21.15.154:0/1479697152 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac780b2fe0 0x7fac780ac670 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:41.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.291+0000 7fac7e37c640 1 --2- 172.21.15.154:0/1479697152 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fac780ada30 0x7fac78094610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:41.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.291+0000 7fac7e37c640 1 -- 172.21.15.154:0/1479697152 >> 172.21.15.154:0/1479697152 conn(0x7fac7806a0f0 msgr2=0x7fac78151af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:41.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.292+0000 7fac7e37c640 1 -- 172.21.15.154:0/1479697152 shutdown_connections 2023-09-23T03:24:41.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.292+0000 7fac7e37c640 1 -- 172.21.15.154:0/1479697152 wait complete. 2023-09-23T03:24:41.313 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:24:41.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.532+0000 7f2baa27c640 1 Processor -- start 2023-09-23T03:24:41.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.532+0000 7f2baa27c640 1 -- start start 2023-09-23T03:24:41.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.534+0000 7f2baa27c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2ba415caf0 0x7f2ba40ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:41.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.534+0000 7f2baa27c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ba4160f90 0x7f2ba40b1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:41.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.534+0000 7f2baa27c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ba40cd530 con 0x7f2ba40cee70 2023-09-23T03:24:41.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.534+0000 7f2baa27c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ba415ced0 con 0x7f2ba415caf0 2023-09-23T03:24:41.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.534+0000 7f2baa27c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba40cf2f0 con 0x7f2ba4160f90 2023-09-23T03:24:41.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.534+0000 7f2ba3fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2ba415caf0 0x7f2ba40ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:41.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.534+0000 7f2b9bfff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f2ba40cee70 0x7f2ba4160880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:45586/0 (socket says 172.21.15.154:45586) 2023-09-23T03:24:41.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.534+0000 7f2b9bfff640 1 -- 172.21.15.154:0/2028980698 learned_addr learned my addr 172.21.15.154:0/2028980698 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:24:41.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.535+0000 7f2ba8819640 1 --2- 172.21.15.154:0/2028980698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ba4160f90 0x7f2ba40b1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:41.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.535+0000 7f2ba37fe640 1 -- 172.21.15.154:0/2028980698 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2189922896 0 0) 0x7f2ba40cd530 con 0x7f2ba40cee70 2023-09-23T03:24:41.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.535+0000 7f2ba37fe640 1 -- 172.21.15.154:0/2028980698 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b90003520 con 0x7f2ba40cee70 2023-09-23T03:24:41.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.535+0000 7f2ba3fff640 1 -- 172.21.15.154:0/2028980698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ba4160f90 msgr2=0x7f2ba40b1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:41.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.535+0000 7f2ba3fff640 1 --2- 172.21.15.154:0/2028980698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ba4160f90 0x7f2ba40b1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:41.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.535+0000 7f2ba3fff640 1 -- 172.21.15.154:0/2028980698 >> v1:172.21.15.154:6789/0 conn(0x7f2ba40cee70 legacy=0x7f2ba4160880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:41.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.535+0000 7f2ba3fff640 1 -- 172.21.15.154:0/2028980698 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba40cd810 con 0x7f2ba415caf0 2023-09-23T03:24:41.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.537+0000 7f2ba3fff640 1 --2- 172.21.15.154:0/2028980698 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ba415caf0 0x7f2ba40ce930 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f2b94009790 tx=0x7f2b9402b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5441dc390ebc3080 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:41.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.537+0000 7f2ba37fe640 1 -- 172.21.15.154:0/2028980698 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b94031800 con 0x7f2ba415caf0 2023-09-23T03:24:41.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.537+0000 7f2ba37fe640 1 -- 172.21.15.154:0/2028980698 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b94031960 con 0x7f2ba415caf0 2023-09-23T03:24:41.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.537+0000 7f2ba37fe640 1 -- 172.21.15.154:0/2028980698 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b94031cf0 con 0x7f2ba415caf0 2023-09-23T03:24:41.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.538+0000 7f2baa27c640 1 -- 172.21.15.154:0/2028980698 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ba415caf0 msgr2=0x7f2ba40ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:41.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.538+0000 7f2baa27c640 1 --2- 172.21.15.154:0/2028980698 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ba415caf0 0x7f2ba40ce930 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f2b94009790 tx=0x7f2b9402b8a0 comp rx=0 tx=0).stop 2023-09-23T03:24:41.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.538+0000 7f2baa27c640 1 -- 172.21.15.154:0/2028980698 shutdown_connections 2023-09-23T03:24:41.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.538+0000 7f2baa27c640 1 --2- 172.21.15.154:0/2028980698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ba4160f90 0x7f2ba40b1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:41.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.538+0000 7f2baa27c640 1 --2- 172.21.15.154:0/2028980698 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ba415caf0 0x7f2ba40ce930 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:41.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.538+0000 7f2baa27c640 1 -- 172.21.15.154:0/2028980698 >> 172.21.15.154:0/2028980698 conn(0x7f2ba406a0f0 msgr2=0x7f2ba406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:41.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.538+0000 7f2baa27c640 1 -- 172.21.15.154:0/2028980698 shutdown_connections 2023-09-23T03:24:41.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.538+0000 7f2baa27c640 1 -- 172.21.15.154:0/2028980698 wait complete. 2023-09-23T03:24:41.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.540+0000 7f2baa27c640 1 Processor -- start 2023-09-23T03:24:41.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.540+0000 7f2baa27c640 1 -- start start 2023-09-23T03:24:41.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.541+0000 7f2baa27c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ba40cee70 0x7f2ba415fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:41.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.541+0000 7f2baa27c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ba415caf0 0x7f2ba415ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:41.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.541+0000 7f2baa27c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ba4160f90 0x7f2ba4159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:41.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.541+0000 7f2baa27c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba415c1d0 con 0x7f2ba415caf0 2023-09-23T03:24:41.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.541+0000 7f2baa27c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba415b650 con 0x7f2ba4160f90 2023-09-23T03:24:41.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.541+0000 7f2baa27c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba415b930 con 0x7f2ba40cee70 2023-09-23T03:24:41.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.541+0000 7f2b9bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ba415caf0 0x7f2ba415ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:41.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.541+0000 7f2b9bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ba415caf0 0x7f2ba415ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43146/0 (socket says 172.21.15.154:43146) 2023-09-23T03:24:41.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.541+0000 7f2b9bfff640 1 -- 172.21.15.154:0/1086973502 learned_addr learned my addr 172.21.15.154:0/1086973502 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:41.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.541+0000 7f2b9bfff640 1 -- 172.21.15.154:0/1086973502 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ba40cee70 msgr2=0x7f2ba415fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:41.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.541+0000 7f2b9bfff640 1 --2- 172.21.15.154:0/1086973502 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ba40cee70 0x7f2ba415fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:41.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.542+0000 7f2b9bfff640 1 -- 172.21.15.154:0/1086973502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ba4160f90 msgr2=0x7f2ba4159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:41.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.542+0000 7f2b9bfff640 1 --2- 172.21.15.154:0/1086973502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ba4160f90 0x7f2ba4159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:41.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.542+0000 7f2b9bfff640 1 -- 172.21.15.154:0/1086973502 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b940093a0 con 0x7f2ba415caf0 2023-09-23T03:24:41.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.542+0000 7f2b9bfff640 1 --2- 172.21.15.154:0/1086973502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ba415caf0 0x7f2ba415ad90 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c00c500 tx=0x7f2b8c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:41.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.542+0000 7f2ba17fa640 1 -- 172.21.15.154:0/1086973502 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c005ce0 con 0x7f2ba415caf0 2023-09-23T03:24:41.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.542+0000 7f2ba17fa640 1 -- 172.21.15.154:0/1086973502 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b8c005e40 con 0x7f2ba415caf0 2023-09-23T03:24:41.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.542+0000 7f2ba17fa640 1 -- 172.21.15.154:0/1086973502 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c0072b0 con 0x7f2ba415caf0 2023-09-23T03:24:41.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.542+0000 7f2baa27c640 1 -- 172.21.15.154:0/1086973502 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ba415bc10 con 0x7f2ba415caf0 2023-09-23T03:24:41.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.542+0000 7f2baa27c640 1 -- 172.21.15.154:0/1086973502 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ba415b2d0 con 0x7f2ba415caf0 2023-09-23T03:24:41.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.543+0000 7f2baa27c640 1 -- 172.21.15.154:0/1086973502 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ba4073eb0 con 0x7f2ba415caf0 2023-09-23T03:24:41.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.544+0000 7f2ba17fa640 1 -- 172.21.15.154:0/1086973502 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f2b8c0074d0 con 0x7f2ba415caf0 2023-09-23T03:24:41.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.545+0000 7f2ba17fa640 1 --2- 172.21.15.154:0/1086973502 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2b8406af30 0x7f2b8406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:41.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.545+0000 7f2ba17fa640 1 -- 172.21.15.154:0/1086973502 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6605+0+0 (secure 0 0 0) 0x7f2b8c089150 con 0x7f2ba415caf0 2023-09-23T03:24:41.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.546+0000 7f2ba3fff640 1 --2- 172.21.15.154:0/1086973502 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2b8406af30 0x7f2b8406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:41.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.546+0000 7f2ba3fff640 1 --2- 172.21.15.154:0/1086973502 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2b8406af30 0x7f2b8406d3f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2b94036000 tx=0x7f2b94030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:41.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.550+0000 7f2ba17fa640 1 -- 172.21.15.154:0/1086973502 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f2b8c053ec0 con 0x7f2ba415caf0 2023-09-23T03:24:41.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.779+0000 7f2baa27c640 1 -- 172.21.15.154:0/1086973502 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ba40afd70 con 0x7f2ba415caf0 2023-09-23T03:24:41.782 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.781+0000 7f2ba17fa640 1 -- 172.21.15.154:0/1086973502 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f2b8c0574e0 con 0x7f2ba415caf0 2023-09-23T03:24:41.783 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:41.783 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":69,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/728936136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":728936136},{"type":"v1","addr":"172.21.15.190:6833","nonce":728936136}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":5890,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2277229238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2277229238},{"type":"v1","addr":"172.21.15.190:6835","nonce":2277229238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":7257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/4245928089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4245928089},{"type":"v1","addr":"172.21.15.154:6835","nonce":4245928089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":7575,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1588996498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1588996498},{"type":"v1","addr":"172.21.15.154:6837","nonce":1588996498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:24:37.386034+0000","modified":"2023-09-23T03:24:38.801289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[12,13],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-09-23T03:24:41.783 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 69 2023-09-23T03:24:41.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.784+0000 7f2baa27c640 1 -- 172.21.15.154:0/1086973502 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2b8406af30 msgr2=0x7f2b8406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:41.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.784+0000 7f2baa27c640 1 --2- 172.21.15.154:0/1086973502 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2b8406af30 0x7f2b8406d3f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2b94036000 tx=0x7f2b94030000 comp rx=0 tx=0).stop 2023-09-23T03:24:41.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.784+0000 7f2baa27c640 1 -- 172.21.15.154:0/1086973502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ba415caf0 msgr2=0x7f2ba415ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:41.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.784+0000 7f2baa27c640 1 --2- 172.21.15.154:0/1086973502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ba415caf0 0x7f2ba415ad90 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c00c500 tx=0x7f2b8c00c9c0 comp rx=0 tx=0).stop 2023-09-23T03:24:41.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.784+0000 7f2baa27c640 1 -- 172.21.15.154:0/1086973502 shutdown_connections 2023-09-23T03:24:41.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.784+0000 7f2baa27c640 1 --2- 172.21.15.154:0/1086973502 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2b8406af30 0x7f2b8406d3f0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:41.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.784+0000 7f2baa27c640 1 --2- 172.21.15.154:0/1086973502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ba4160f90 0x7f2ba4159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:41.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.784+0000 7f2baa27c640 1 --2- 172.21.15.154:0/1086973502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ba415caf0 0x7f2ba415ad90 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:41.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.784+0000 7f2baa27c640 1 --2- 172.21.15.154:0/1086973502 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ba40cee70 0x7f2ba415fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:41.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.784+0000 7f2baa27c640 1 -- 172.21.15.154:0/1086973502 >> 172.21.15.154:0/1086973502 conn(0x7f2ba406a0f0 msgr2=0x7f2ba40af640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:41.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.784+0000 7f2baa27c640 1 -- 172.21.15.154:0/1086973502 shutdown_connections 2023-09-23T03:24:41.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.784+0000 7f2baa27c640 1 -- 172.21.15.154:0/1086973502 wait complete. 2023-09-23T03:24:41.803 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset session 2023-09-23T03:24:41.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.892+0000 7f25fce10100 1 -- start start 2023-09-23T03:24:41.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.892+0000 7f25fce10100 1 Processor -- start 2023-09-23T03:24:41.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.893+0000 7f25fce10100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569b590ec00 0x5569b597e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:41.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.893+0000 7f25fce10100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5569b590f800 0x5569b597eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:41.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.893+0000 7f25fce10100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5569b590fc00 0x5569b597f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:41.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.893+0000 7f25fce10100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569b595b180 con 0x5569b590ec00 2023-09-23T03:24:41.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.893+0000 7f25fce10100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569b595a140 con 0x5569b590fc00 2023-09-23T03:24:41.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.893+0000 7f25fce10100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5569b595a000 con 0x5569b590f800 2023-09-23T03:24:41.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.893+0000 7f25fa98f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569b590ec00 0x5569b597e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:41.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.893+0000 7f25fa98f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569b590ec00 0x5569b597e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43164/0 (socket says 172.21.15.154:43164) 2023-09-23T03:24:41.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.893+0000 7f25fa98f640 1 -- 172.21.15.154:0/2208377618 learned_addr learned my addr 172.21.15.154:0/2208377618 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:41.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.893+0000 7f25fa98f640 1 -- 172.21.15.154:0/2208377618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5569b590f800 msgr2=0x5569b597eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:41.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.893+0000 7f25fa98f640 1 --2- 172.21.15.154:0/2208377618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5569b590f800 0x5569b597eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:41.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.893+0000 7f25fa98f640 1 -- 172.21.15.154:0/2208377618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5569b590fc00 msgr2=0x5569b597f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:41.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.893+0000 7f25fa98f640 1 --2- 172.21.15.154:0/2208377618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5569b590fc00 0x5569b597f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:41.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.894+0000 7f25fa98f640 1 -- 172.21.15.154:0/2208377618 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569b5942b60 con 0x5569b590ec00 2023-09-23T03:24:41.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.894+0000 7f25fa98f640 1 --2- 172.21.15.154:0/2208377618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569b590ec00 0x5569b597e580 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x5569b6722b70 tx=0x5569b67b2120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6fe2e56374bf57 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:41.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.894+0000 7f25f818a640 1 -- 172.21.15.154:0/2208377618 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5569b5959080 con 0x5569b590ec00 2023-09-23T03:24:41.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.894+0000 7f25f818a640 1 -- 172.21.15.154:0/2208377618 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5569b5935800 con 0x5569b590ec00 2023-09-23T03:24:41.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.894+0000 7f25fce10100 1 -- 172.21.15.154:0/2208377618 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5569b59431e0 con 0x5569b590ec00 2023-09-23T03:24:41.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.894+0000 7f25fce10100 1 -- 172.21.15.154:0/2208377618 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5569b5942b60 con 0x5569b590ec00 2023-09-23T03:24:41.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.894+0000 7f25fce10100 4 waiting for MDS map... 2023-09-23T03:24:41.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.894+0000 7f25f818a640 1 -- 172.21.15.154:0/2208377618 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5569b5959600 con 0x5569b590ec00 2023-09-23T03:24:41.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.894+0000 7f25f818a640 1 -- 172.21.15.154:0/2208377618 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(88..88 src has 1..88) v4 ==== 6605+0+0 (secure 0 0 0) 0x5569b593a380 con 0x5569b590ec00 2023-09-23T03:24:41.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.894+0000 7f25f818a640 1 -- 172.21.15.154:0/2208377618 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 69) v1 ==== 3306+0+0 (secure 0 0 0) 0x5569b5963200 con 0x5569b590ec00 2023-09-23T03:24:41.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.894+0000 7f25fce10100 4 Got MDS map 69 2023-09-23T03:24:41.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.895+0000 7f25fce10100 10 main: main 2023-09-23T03:24:41.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.895+0000 7f25fce10100 4 main: connecting to RADOS... 2023-09-23T03:24:41.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.895+0000 7f25fce10100 1 Processor -- start 2023-09-23T03:24:41.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.895+0000 7f25fce10100 1 -- start start 2023-09-23T03:24:41.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.896+0000 7f25fce10100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5569b66e3c00 0x5569b597f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:41.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.896+0000 7f25fce10100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5569b5c4c000 0x5569b597fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:41.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.896+0000 7f25fce10100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569b5c4c400 0x5569b5980100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:41.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.896+0000 7f25fce10100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569b595aa00 con 0x5569b5c4c400 2023-09-23T03:24:41.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.896+0000 7f25fce10100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569b59dd180 con 0x5569b66e3c00 2023-09-23T03:24:41.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.896+0000 7f25fce10100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5569b59dcdc0 con 0x5569b5c4c000 2023-09-23T03:24:41.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.896+0000 7f25fa18e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569b5c4c400 0x5569b5980100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:41.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.897+0000 7f25fa18e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569b5c4c400 0x5569b5980100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43172/0 (socket says 172.21.15.154:43172) 2023-09-23T03:24:41.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.897+0000 7f25fa18e640 1 -- 172.21.15.154:0/141689609 learned_addr learned my addr 172.21.15.154:0/141689609 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:41.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.897+0000 7f25fb190640 1 --2- 172.21.15.154:0/141689609 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5569b66e3c00 0x5569b597f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:41.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.897+0000 7f25fa18e640 1 -- 172.21.15.154:0/141689609 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5569b5c4c000 msgr2=0x5569b597fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:41.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.897+0000 7f25fa18e640 1 --2- 172.21.15.154:0/141689609 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5569b5c4c000 0x5569b597fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:41.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.897+0000 7f25fa18e640 1 -- 172.21.15.154:0/141689609 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5569b66e3c00 msgr2=0x5569b597f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:41.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.897+0000 7f25fa18e640 1 --2- 172.21.15.154:0/141689609 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5569b66e3c00 0x5569b597f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:41.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.897+0000 7f25fa18e640 1 -- 172.21.15.154:0/141689609 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569b5942340 con 0x5569b5c4c400 2023-09-23T03:24:41.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.897+0000 7f25fa18e640 1 --2- 172.21.15.154:0/141689609 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569b5c4c400 0x5569b5980100 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x5569b5c560c0 tx=0x5569b5bfede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b00a8a7a4aa2ae server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:41.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.897+0000 7f25f6186640 1 -- 172.21.15.154:0/141689609 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5569b5959080 con 0x5569b5c4c400 2023-09-23T03:24:41.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.897+0000 7f25f6186640 1 -- 172.21.15.154:0/141689609 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5569b669c480 con 0x5569b5c4c400 2023-09-23T03:24:41.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.897+0000 7f25f6186640 1 -- 172.21.15.154:0/141689609 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5569b5959600 con 0x5569b5c4c400 2023-09-23T03:24:41.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.898+0000 7f25fce10100 1 -- 172.21.15.154:0/141689609 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569b5c4c400 msgr2=0x5569b5980100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:41.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.898+0000 7f25fce10100 1 --2- 172.21.15.154:0/141689609 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569b5c4c400 0x5569b5980100 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x5569b5c560c0 tx=0x5569b5bfede0 comp rx=0 tx=0).stop 2023-09-23T03:24:41.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.898+0000 7f25fce10100 1 -- 172.21.15.154:0/141689609 shutdown_connections 2023-09-23T03:24:41.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.898+0000 7f25fce10100 1 --2- 172.21.15.154:0/141689609 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569b5c4c400 0x5569b5980100 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:41.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.898+0000 7f25fce10100 1 --2- 172.21.15.154:0/141689609 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5569b5c4c000 0x5569b597fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:41.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.898+0000 7f25fce10100 1 --2- 172.21.15.154:0/141689609 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5569b66e3c00 0x5569b597f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:41.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.898+0000 7f25fce10100 1 -- 172.21.15.154:0/141689609 >> 172.21.15.154:0/141689609 conn(0x5569b66e3800 msgr2=0x5569b59ed000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:41.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.898+0000 7f25fce10100 1 -- 172.21.15.154:0/141689609 shutdown_connections 2023-09-23T03:24:41.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.898+0000 7f25fce10100 1 -- 172.21.15.154:0/141689609 wait complete. 2023-09-23T03:24:41.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.899+0000 7f25fce10100 1 Processor -- start 2023-09-23T03:24:41.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.899+0000 7f25fce10100 1 -- start start 2023-09-23T03:24:41.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.899+0000 7f25fce10100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5569b66e3800 0x5569b5981180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:41.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.899+0000 7f25fce10100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569b5c4d000 0x5569b5980c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:41.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.899+0000 7f25fce10100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5569b5c4cc00 0x5569b5980680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:41.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.899+0000 7f25fce10100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569b595a140 con 0x5569b5c4d000 2023-09-23T03:24:41.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.899+0000 7f25fce10100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569b595b180 con 0x5569b5c4cc00 2023-09-23T03:24:41.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.900+0000 7f25fce10100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5569b595ab40 con 0x5569b66e3800 2023-09-23T03:24:41.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.900+0000 7f25fa98f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569b5c4d000 0x5569b5980c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:41.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.900+0000 7f25fa98f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569b5c4d000 0x5569b5980c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43186/0 (socket says 172.21.15.154:43186) 2023-09-23T03:24:41.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.900+0000 7f25fa98f640 1 -- 172.21.15.154:0/1942459007 learned_addr learned my addr 172.21.15.154:0/1942459007 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:41.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.900+0000 7f25fa98f640 1 -- 172.21.15.154:0/1942459007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5569b66e3800 msgr2=0x5569b5981180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:41.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.900+0000 7f25fa98f640 1 --2- 172.21.15.154:0/1942459007 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5569b66e3800 0x5569b5981180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:41.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.900+0000 7f25fa98f640 1 -- 172.21.15.154:0/1942459007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5569b5c4cc00 msgr2=0x5569b5980680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:41.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.900+0000 7f25fa18e640 1 --2- 172.21.15.154:0/1942459007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5569b5c4cc00 0x5569b5980680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:41.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.900+0000 7f25fa98f640 1 --2- 172.21.15.154:0/1942459007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5569b5c4cc00 0x5569b5980680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:41.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.900+0000 7f25fa98f640 1 -- 172.21.15.154:0/1942459007 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569b5942000 con 0x5569b5c4d000 2023-09-23T03:24:41.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.900+0000 7f25fa18e640 1 --2- 172.21.15.154:0/1942459007 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5569b5c4cc00 0x5569b5980680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:41.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.900+0000 7f25fa98f640 1 --2- 172.21.15.154:0/1942459007 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569b5c4d000 0x5569b5980c00 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x5569b6722ff0 tx=0x5569b67b21e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:41.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.901+0000 7f25f4983640 1 -- 172.21.15.154:0/1942459007 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5569b59591e0 con 0x5569b5c4d000 2023-09-23T03:24:41.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.901+0000 7f25f4983640 1 -- 172.21.15.154:0/1942459007 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5569b669d380 con 0x5569b5c4d000 2023-09-23T03:24:41.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.901+0000 7f25f4983640 1 -- 172.21.15.154:0/1942459007 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5569b5958f20 con 0x5569b5c4d000 2023-09-23T03:24:41.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.901+0000 7f25fce10100 1 -- 172.21.15.154:0/1942459007 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5569b5942ea0 con 0x5569b5c4d000 2023-09-23T03:24:41.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.901+0000 7f25fce10100 1 -- 172.21.15.154:0/1942459007 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5569b5943520 con 0x5569b5c4d000 2023-09-23T03:24:41.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.901+0000 7f25fce10100 4 main: resolving pool 11 2023-09-23T03:24:41.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.902+0000 7f25f4983640 1 -- 172.21.15.154:0/1942459007 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x5569b5941080 con 0x5569b5c4d000 2023-09-23T03:24:41.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.903+0000 7f25f4983640 1 --2- 172.21.15.154:0/1942459007 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5569b5c69000 0x5569b5981700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:41.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.903+0000 7f25f4983640 1 -- 172.21.15.154:0/1942459007 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6605+0+0 (secure 0 0 0) 0x5569b593a540 con 0x5569b5c4d000 2023-09-23T03:24:41.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.903+0000 7f25fce10100 4 main: creating IoCtx.. 2023-09-23T03:24:41.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.903+0000 7f25fce10100 1 --2- 172.21.15.154:0/1942459007 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5569b5c4c800 0x5569b597f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:41.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.903+0000 7f25fce10100 1 -- 172.21.15.154:0/1942459007 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:1 11.b 11:d0630e4c:::mds0_sessionmap:head [omap-clear,omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x5569b596b880 con 0x5569b5c4c800 2023-09-23T03:24:41.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.903+0000 7f25fb190640 1 --2- 172.21.15.154:0/1942459007 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5569b5c69000 0x5569b5981700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:41.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.904+0000 7f25fa18e640 1 --2- 172.21.15.154:0/1942459007 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5569b5c4c800 0x5569b597f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:41.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.904+0000 7f25fb190640 1 --2- 172.21.15.154:0/1942459007 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5569b5c69000 0x5569b5981700 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x5569b66680f0 tx=0x5569b5d66000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:41.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.904+0000 7f25fa18e640 1 --2- 172.21.15.154:0/1942459007 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5569b5c4c800 0x5569b597f600 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:41.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.906+0000 7f25fa18e640 1 -- 172.21.15.154:0/1942459007 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(1 mds0_sessionmap [omap-clear,omap-set-header] v88'5 uv5 ondisk = 0) v8 ==== 201+0+0 (crc 0 0 0) 0x5569b67b0b40 con 0x5569b5c4c800 2023-09-23T03:24:41.910 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:24:41.911 INFO:teuthology.orchestra.run.smithi154.stdout: "0": { 2023-09-23T03:24:41.911 INFO:teuthology.orchestra.run.smithi154.stdout: "data": {}, 2023-09-23T03:24:41.911 INFO:teuthology.orchestra.run.smithi154.stdout: "result": 0 2023-09-23T03:24:41.911 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:24:41.911 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:24:41.911 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:41.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.906+0000 7f25fce10100 1 -- 172.21.15.154:0/1942459007 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5569b5c4c800 msgr2=0x5569b597f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:41.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.906+0000 7f25fce10100 1 --2- 172.21.15.154:0/1942459007 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5569b5c4c800 0x5569b597f600 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:41.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.906+0000 7f25fce10100 1 -- 172.21.15.154:0/1942459007 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5569b5c69000 msgr2=0x5569b5981700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:41.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.906+0000 7f25fce10100 1 --2- 172.21.15.154:0/1942459007 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5569b5c69000 0x5569b5981700 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x5569b66680f0 tx=0x5569b5d66000 comp rx=0 tx=0).stop 2023-09-23T03:24:41.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.906+0000 7f25fce10100 1 -- 172.21.15.154:0/1942459007 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569b5c4d000 msgr2=0x5569b5980c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:41.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.906+0000 7f25fce10100 1 --2- 172.21.15.154:0/1942459007 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569b5c4d000 0x5569b5980c00 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x5569b6722ff0 tx=0x5569b67b21e0 comp rx=0 tx=0).stop 2023-09-23T03:24:41.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.906+0000 7f25fa18e640 1 -- 172.21.15.154:0/1942459007 reap_dead start 2023-09-23T03:24:41.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.906+0000 7f25fce10100 1 -- 172.21.15.154:0/1942459007 shutdown_connections 2023-09-23T03:24:41.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.906+0000 7f25fce10100 1 -- 172.21.15.154:0/1942459007 >> 172.21.15.154:0/1942459007 conn(0x5569b66e3c00 msgr2=0x5569b59ed000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:41.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.906+0000 7f25fce10100 1 -- 172.21.15.154:0/1942459007 shutdown_connections 2023-09-23T03:24:41.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.906+0000 7f25fce10100 1 -- 172.21.15.154:0/1942459007 wait complete. 2023-09-23T03:24:41.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.907+0000 7f25fce10100 1 -- 172.21.15.154:0/2208377618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569b590ec00 msgr2=0x5569b597e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:41.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.907+0000 7f25fce10100 1 --2- 172.21.15.154:0/2208377618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569b590ec00 0x5569b597e580 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x5569b6722b70 tx=0x5569b67b2120 comp rx=0 tx=0).stop 2023-09-23T03:24:41.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.907+0000 7f25fce10100 1 -- 172.21.15.154:0/2208377618 shutdown_connections 2023-09-23T03:24:41.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.907+0000 7f25fce10100 1 --2- 172.21.15.154:0/2208377618 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5569b590fc00 0x5569b597f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:41.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.907+0000 7f25fce10100 1 --2- 172.21.15.154:0/2208377618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5569b590f800 0x5569b597eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:41.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.907+0000 7f25fce10100 1 --2- 172.21.15.154:0/2208377618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5569b590ec00 0x5569b597e580 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:41.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.907+0000 7f25fce10100 1 -- 172.21.15.154:0/2208377618 >> 172.21.15.154:0/2208377618 conn(0x5569b590f000 msgr2=0x5569b59ec800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:41.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.907+0000 7f25fce10100 1 -- 172.21.15.154:0/2208377618 shutdown_connections 2023-09-23T03:24:41.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:41.907+0000 7f25fce10100 1 -- 172.21.15.154:0/2208377618 wait complete. 2023-09-23T03:24:41.915 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'session'] in time 0:00:00.112684, result: { "0": { "data": {}, "result": 0 } } 2023-09-23T03:24:41.916 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset snap 2023-09-23T03:24:42.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.009+0000 7f784ae3b100 1 -- start start 2023-09-23T03:24:42.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.009+0000 7f784ae3b100 1 Processor -- start 2023-09-23T03:24:42.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.010+0000 7f784ae3b100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556604d48800 0x556603ff5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.010+0000 7f784ae3b100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556604d49c00 0x556603ff5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.010+0000 7f784ae3b100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556604d49400 0x556603ff4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.010+0000 7f784ae3b100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x556603fd1180 con 0x556604d49c00 2023-09-23T03:24:42.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.010+0000 7f784ae3b100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x556604052dc0 con 0x556604d49400 2023-09-23T03:24:42.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.010+0000 7f784ae3b100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x556603fd0b40 con 0x556604d48800 2023-09-23T03:24:42.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.010+0000 7f7848686640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556604d49c00 0x556603ff5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.010+0000 7f7848686640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556604d49c00 0x556603ff5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43220/0 (socket says 172.21.15.154:43220) 2023-09-23T03:24:42.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.010+0000 7f7848686640 1 -- 172.21.15.154:0/418666678 learned_addr learned my addr 172.21.15.154:0/418666678 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:42.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.010+0000 7f7849688640 1 --2- 172.21.15.154:0/418666678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556604d49400 0x556603ff4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.010+0000 7f7848e87640 1 --2- 172.21.15.154:0/418666678 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556604d48800 0x556603ff5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.010+0000 7f7848686640 1 -- 172.21.15.154:0/418666678 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556604d48800 msgr2=0x556603ff5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:42.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.010+0000 7f7848686640 1 --2- 172.21.15.154:0/418666678 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556604d48800 0x556603ff5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.010+0000 7f7848686640 1 -- 172.21.15.154:0/418666678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556604d49400 msgr2=0x556603ff4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:42.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.010+0000 7f7848686640 1 --2- 172.21.15.154:0/418666678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556604d49400 0x556603ff4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.010+0000 7f7848686640 1 -- 172.21.15.154:0/418666678 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556603fb8000 con 0x556604d49c00 2023-09-23T03:24:42.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.010+0000 7f7849688640 1 --2- 172.21.15.154:0/418666678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556604d49400 0x556603ff4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:42.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.011+0000 7f7848686640 1 --2- 172.21.15.154:0/418666678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556604d49c00 0x556603ff5080 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x556604da2c60 tx=0x556604e4a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb62a5b705544f7b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.011+0000 7f7846682640 1 -- 172.21.15.154:0/418666678 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556603fcf080 con 0x556604d49c00 2023-09-23T03:24:42.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.011+0000 7f7846682640 1 -- 172.21.15.154:0/418666678 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556603fab800 con 0x556604d49c00 2023-09-23T03:24:42.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.011+0000 7f7846682640 1 -- 172.21.15.154:0/418666678 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556603fcf600 con 0x556604d49c00 2023-09-23T03:24:42.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.011+0000 7f784ae3b100 1 -- 172.21.15.154:0/418666678 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556603fb8ea0 con 0x556604d49c00 2023-09-23T03:24:42.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.011+0000 7f784ae3b100 1 -- 172.21.15.154:0/418666678 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x556603fb9520 con 0x556604d49c00 2023-09-23T03:24:42.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.011+0000 7f784ae3b100 4 waiting for MDS map... 2023-09-23T03:24:42.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.011+0000 7f7846682640 1 -- 172.21.15.154:0/418666678 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(88..88 src has 1..88) v4 ==== 6605+0+0 (secure 0 0 0) 0x556603fb0380 con 0x556604d49c00 2023-09-23T03:24:42.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.011+0000 7f7846682640 1 -- 172.21.15.154:0/418666678 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 69) v1 ==== 3306+0+0 (secure 0 0 0) 0x556603fd9200 con 0x556604d49c00 2023-09-23T03:24:42.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.012+0000 7f784ae3b100 4 Got MDS map 69 2023-09-23T03:24:42.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.012+0000 7f784ae3b100 10 main: main 2023-09-23T03:24:42.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.012+0000 7f784ae3b100 4 main: connecting to RADOS... 2023-09-23T03:24:42.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.012+0000 7f784ae3b100 1 Processor -- start 2023-09-23T03:24:42.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.013+0000 7f784ae3b100 1 -- start start 2023-09-23T03:24:42.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.013+0000 7f784ae3b100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556604e52c00 0x556603ff6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.013+0000 7f784ae3b100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556604e53000 0x556603ff6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.013+0000 7f784ae3b100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556604e53400 0x556603ff6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.013+0000 7f784ae3b100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x556603fd0000 con 0x556604e52c00 2023-09-23T03:24:42.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.013+0000 7f784ae3b100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x556603fd0a00 con 0x556604e53000 2023-09-23T03:24:42.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.013+0000 7f784ae3b100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x556604052dc0 con 0x556604e53400 2023-09-23T03:24:42.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.014+0000 7f7849688640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556604e52c00 0x556603ff6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.014+0000 7f7849688640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556604e52c00 0x556603ff6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43236/0 (socket says 172.21.15.154:43236) 2023-09-23T03:24:42.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.014+0000 7f7849688640 1 -- 172.21.15.154:0/3412098750 learned_addr learned my addr 172.21.15.154:0/3412098750 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:42.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.014+0000 7f7849688640 1 -- 172.21.15.154:0/3412098750 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556604e53400 msgr2=0x556603ff6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:42.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.014+0000 7f7849688640 1 --2- 172.21.15.154:0/3412098750 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556604e53400 0x556603ff6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.014+0000 7f7849688640 1 -- 172.21.15.154:0/3412098750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556604e53000 msgr2=0x556603ff6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:42.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.014+0000 7f7848e87640 1 --2- 172.21.15.154:0/3412098750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556604e53000 0x556603ff6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.014+0000 7f7849688640 1 --2- 172.21.15.154:0/3412098750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556604e53000 0x556603ff6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.014+0000 7f7849688640 1 -- 172.21.15.154:0/3412098750 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556603fb96c0 con 0x556604e52c00 2023-09-23T03:24:42.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.014+0000 7f7849688640 1 --2- 172.21.15.154:0/3412098750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556604e52c00 0x556603ff6100 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x556604051380 tx=0x556604e4a120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=534d1e908af02350 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.014+0000 7f784467e640 1 -- 172.21.15.154:0/3412098750 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556603fcf080 con 0x556604e52c00 2023-09-23T03:24:42.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.014+0000 7f784467e640 1 -- 172.21.15.154:0/3412098750 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556603fab800 con 0x556604e52c00 2023-09-23T03:24:42.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.014+0000 7f784467e640 1 -- 172.21.15.154:0/3412098750 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556603fcf600 con 0x556604e52c00 2023-09-23T03:24:42.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.015+0000 7f784ae3b100 1 -- 172.21.15.154:0/3412098750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556604e52c00 msgr2=0x556603ff6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.015+0000 7f784ae3b100 1 --2- 172.21.15.154:0/3412098750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556604e52c00 0x556603ff6100 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x556604051380 tx=0x556604e4a120 comp rx=0 tx=0).stop 2023-09-23T03:24:42.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.015+0000 7f784ae3b100 1 -- 172.21.15.154:0/3412098750 shutdown_connections 2023-09-23T03:24:42.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.015+0000 7f784ae3b100 1 --2- 172.21.15.154:0/3412098750 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556604e53400 0x556603ff6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.015+0000 7f784ae3b100 1 --2- 172.21.15.154:0/3412098750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556604e53000 0x556603ff6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.015+0000 7f784ae3b100 1 --2- 172.21.15.154:0/3412098750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556604e52c00 0x556603ff6100 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.015+0000 7f784ae3b100 1 -- 172.21.15.154:0/3412098750 >> 172.21.15.154:0/3412098750 conn(0x556604d49800 msgr2=0x556604cfd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:42.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.015+0000 7f784ae3b100 1 -- 172.21.15.154:0/3412098750 shutdown_connections 2023-09-23T03:24:42.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.015+0000 7f784ae3b100 1 -- 172.21.15.154:0/3412098750 wait complete. 2023-09-23T03:24:42.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.016+0000 7f784ae3b100 1 Processor -- start 2023-09-23T03:24:42.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.016+0000 7f784ae3b100 1 -- start start 2023-09-23T03:24:42.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.017+0000 7f784ae3b100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556604e52c00 0x556603ff6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.017+0000 7f784ae3b100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556604e53000 0x556603ff6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.017+0000 7f784ae3b100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556604e53400 0x556603ff6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.017+0000 7f784ae3b100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x556604053180 con 0x556604e53400 2023-09-23T03:24:42.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.017+0000 7f784ae3b100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x556604052c80 con 0x556604e53000 2023-09-23T03:24:42.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.017+0000 7f784ae3b100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x556604052b40 con 0x556604e52c00 2023-09-23T03:24:42.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.017+0000 7f7848686640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556604e53400 0x556603ff6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.017+0000 7f7848686640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556604e53400 0x556603ff6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43240/0 (socket says 172.21.15.154:43240) 2023-09-23T03:24:42.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.017+0000 7f7848686640 1 -- 172.21.15.154:0/3959550731 learned_addr learned my addr 172.21.15.154:0/3959550731 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:42.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.017+0000 7f7848e87640 1 --2- 172.21.15.154:0/3959550731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556604e53000 0x556603ff6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.017+0000 7f7849688640 1 --2- 172.21.15.154:0/3959550731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556604e52c00 0x556603ff6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.017+0000 7f7848686640 1 -- 172.21.15.154:0/3959550731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556604e52c00 msgr2=0x556603ff6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.017+0000 7f7848686640 1 --2- 172.21.15.154:0/3959550731 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556604e52c00 0x556603ff6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.017+0000 7f7848686640 1 -- 172.21.15.154:0/3959550731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556604e53000 msgr2=0x556603ff6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.017+0000 7f7848686640 1 --2- 172.21.15.154:0/3959550731 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556604e53000 0x556603ff6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.017+0000 7f7848686640 1 -- 172.21.15.154:0/3959550731 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556603fb8340 con 0x556604e53400 2023-09-23T03:24:42.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.018+0000 7f7848686640 1 --2- 172.21.15.154:0/3959550731 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556604e53400 0x556603ff6c00 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x556604eb4120 tx=0x556604d156e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.018+0000 7f7842e7b640 1 -- 172.21.15.154:0/3959550731 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556603fcf1e0 con 0x556604e53400 2023-09-23T03:24:42.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.018+0000 7f7842e7b640 1 -- 172.21.15.154:0/3959550731 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556604e58900 con 0x556604e53400 2023-09-23T03:24:42.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.018+0000 7f784ae3b100 1 -- 172.21.15.154:0/3959550731 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556603fb91e0 con 0x556604e53400 2023-09-23T03:24:42.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.018+0000 7f7842e7b640 1 -- 172.21.15.154:0/3959550731 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556603fcf080 con 0x556604e53400 2023-09-23T03:24:42.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.018+0000 7f784ae3b100 1 -- 172.21.15.154:0/3959550731 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556603fb89c0 con 0x556604e53400 2023-09-23T03:24:42.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.018+0000 7f784ae3b100 4 main: resolving pool 11 2023-09-23T03:24:42.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.020+0000 7f7842e7b640 1 -- 172.21.15.154:0/3959550731 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x556603fb6840 con 0x556604e53400 2023-09-23T03:24:42.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.020+0000 7f7842e7b640 1 --2- 172.21.15.154:0/3959550731 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x556604ec1000 0x556603ff7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.020+0000 7f7842e7b640 1 -- 172.21.15.154:0/3959550731 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6605+0+0 (secure 0 0 0) 0x556603fb0540 con 0x556604e53400 2023-09-23T03:24:42.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.020+0000 7f784ae3b100 4 main: creating IoCtx.. 2023-09-23T03:24:42.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.020+0000 7f784ae3b100 1 --2- 172.21.15.154:0/3959550731 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x556604e53800 0x556603ff7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.020+0000 7f784ae3b100 1 -- 172.21.15.154:0/3959550731 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:1 11.d 11:b50e409b:::mds_snaptable:head [writefull 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x556603fe1880 con 0x556604e53800 2023-09-23T03:24:42.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.020+0000 7f7849688640 1 --2- 172.21.15.154:0/3959550731 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x556604ec1000 0x556603ff7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.021+0000 7f7848e87640 1 --2- 172.21.15.154:0/3959550731 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x556604e53800 0x556603ff7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.021+0000 7f7849688640 1 --2- 172.21.15.154:0/3959550731 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x556604ec1000 0x556603ff7700 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x556604051380 tx=0x556604e4a000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.021+0000 7f7848e87640 1 --2- 172.21.15.154:0/3959550731 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x556604e53800 0x556603ff7180 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.024+0000 7f7848e87640 1 -- 172.21.15.154:0/3959550731 <== osd.2 v2:172.21.15.154:6822/2884407243 1 ==== osd_op_reply(1 mds_snaptable [writefull 0~70] v88'3 uv3 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x556604d68b40 con 0x556604e53800 2023-09-23T03:24:42.028 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:24:42.028 INFO:teuthology.orchestra.run.smithi154.stdout: "result": 0 2023-09-23T03:24:42.028 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:24:42.028 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:42.030 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.024+0000 7f784ae3b100 1 -- 172.21.15.154:0/3959550731 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x556604e53800 msgr2=0x556603ff7180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.030 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.024+0000 7f784ae3b100 1 --2- 172.21.15.154:0/3959550731 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x556604e53800 0x556603ff7180 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.030 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.024+0000 7f784ae3b100 1 -- 172.21.15.154:0/3959550731 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x556604ec1000 msgr2=0x556603ff7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.030 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.024+0000 7f784ae3b100 1 --2- 172.21.15.154:0/3959550731 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x556604ec1000 0x556603ff7700 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x556604051380 tx=0x556604e4a000 comp rx=0 tx=0).stop 2023-09-23T03:24:42.030 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.024+0000 7f784ae3b100 1 -- 172.21.15.154:0/3959550731 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556604e53400 msgr2=0x556603ff6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.030 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.024+0000 7f784ae3b100 1 --2- 172.21.15.154:0/3959550731 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556604e53400 0x556603ff6c00 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x556604eb4120 tx=0x556604d156e0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.024+0000 7f7848e87640 1 -- 172.21.15.154:0/3959550731 reap_dead start 2023-09-23T03:24:42.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.025+0000 7f784ae3b100 1 -- 172.21.15.154:0/3959550731 shutdown_connections 2023-09-23T03:24:42.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.025+0000 7f784ae3b100 1 -- 172.21.15.154:0/3959550731 >> 172.21.15.154:0/3959550731 conn(0x556604d49800 msgr2=0x556604cfd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:42.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.025+0000 7f784ae3b100 1 -- 172.21.15.154:0/3959550731 shutdown_connections 2023-09-23T03:24:42.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.025+0000 7f784ae3b100 1 -- 172.21.15.154:0/3959550731 wait complete. 2023-09-23T03:24:42.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.025+0000 7f784ae3b100 1 -- 172.21.15.154:0/418666678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556604d49c00 msgr2=0x556603ff5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.025+0000 7f784ae3b100 1 --2- 172.21.15.154:0/418666678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556604d49c00 0x556603ff5080 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x556604da2c60 tx=0x556604e4a300 comp rx=0 tx=0).stop 2023-09-23T03:24:42.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.025+0000 7f784ae3b100 1 -- 172.21.15.154:0/418666678 shutdown_connections 2023-09-23T03:24:42.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.025+0000 7f784ae3b100 1 --2- 172.21.15.154:0/418666678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556604d49400 0x556603ff4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.025+0000 7f784ae3b100 1 --2- 172.21.15.154:0/418666678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556604d49c00 0x556603ff5080 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.025+0000 7f784ae3b100 1 --2- 172.21.15.154:0/418666678 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556604d48800 0x556603ff5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.025+0000 7f784ae3b100 1 -- 172.21.15.154:0/418666678 >> 172.21.15.154:0/418666678 conn(0x556603f85000 msgr2=0x556604cfd000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:42.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.025+0000 7f784ae3b100 1 -- 172.21.15.154:0/418666678 shutdown_connections 2023-09-23T03:24:42.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.025+0000 7f784ae3b100 1 -- 172.21.15.154:0/418666678 wait complete. 2023-09-23T03:24:42.033 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'snap'] in time 0:00:00.117254, result: { "result": 0 } 2023-09-23T03:24:42.033 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset inode 2023-09-23T03:24:42.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.112+0000 7f2b71412100 1 -- start start 2023-09-23T03:24:42.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.112+0000 7f2b71412100 1 Processor -- start 2023-09-23T03:24:42.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.113+0000 7f2b71412100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56551d640c00 0x56551d6b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.113+0000 7f2b71412100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56551d641800 0x56551d6b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.113+0000 7f2b71412100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56551d641c00 0x56551d6b1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.113+0000 7f2b71412100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x56551d68d180 con 0x56551d641c00 2023-09-23T03:24:42.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.113+0000 7f2b71412100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56551d68c140 con 0x56551d640c00 2023-09-23T03:24:42.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.113+0000 7f2b71412100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56551d68c000 con 0x56551d641800 2023-09-23T03:24:42.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.113+0000 7f2b6f8e1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56551d641c00 0x56551d6b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.113+0000 7f2b6f8e1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56551d641c00 0x56551d6b1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43266/0 (socket says 172.21.15.154:43266) 2023-09-23T03:24:42.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.113+0000 7f2b6f8e1640 1 -- 172.21.15.154:0/3825266099 learned_addr learned my addr 172.21.15.154:0/3825266099 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:42.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.113+0000 7f2b6f8e1640 1 -- 172.21.15.154:0/3825266099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56551d641800 msgr2=0x56551d6b0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:42.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.113+0000 7f2b6f8e1640 1 --2- 172.21.15.154:0/3825266099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56551d641800 0x56551d6b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.113+0000 7f2b6f8e1640 1 -- 172.21.15.154:0/3825266099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56551d640c00 msgr2=0x56551d6b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:42.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.113+0000 7f2b6f8e1640 1 --2- 172.21.15.154:0/3825266099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56551d640c00 0x56551d6b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.113+0000 7f2b6f8e1640 1 -- 172.21.15.154:0/3825266099 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56551d674b60 con 0x56551d641c00 2023-09-23T03:24:42.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.113+0000 7f2b6f8e1640 1 --2- 172.21.15.154:0/3825266099 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56551d641c00 0x56551d6b1080 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x56551e3ce420 tx=0x56551d918120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9584258c576c2506 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.114+0000 7f2b6c8db640 1 -- 172.21.15.154:0/3825266099 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56551d68b080 con 0x56551d641c00 2023-09-23T03:24:42.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.114+0000 7f2b6c8db640 1 -- 172.21.15.154:0/3825266099 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56551d667800 con 0x56551d641c00 2023-09-23T03:24:42.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.114+0000 7f2b6c8db640 1 -- 172.21.15.154:0/3825266099 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56551d68b600 con 0x56551d641c00 2023-09-23T03:24:42.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.114+0000 7f2b71412100 1 -- 172.21.15.154:0/3825266099 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56551d6751e0 con 0x56551d641c00 2023-09-23T03:24:42.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.114+0000 7f2b71412100 1 -- 172.21.15.154:0/3825266099 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x56551d674b60 con 0x56551d641c00 2023-09-23T03:24:42.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.114+0000 7f2b71412100 4 waiting for MDS map... 2023-09-23T03:24:42.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.114+0000 7f2b6c8db640 1 -- 172.21.15.154:0/3825266099 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(88..88 src has 1..88) v4 ==== 6605+0+0 (secure 0 0 0) 0x56551d66c380 con 0x56551d641c00 2023-09-23T03:24:42.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.114+0000 7f2b6c8db640 1 -- 172.21.15.154:0/3825266099 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 69) v1 ==== 3306+0+0 (secure 0 0 0) 0x56551d695200 con 0x56551d641c00 2023-09-23T03:24:42.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.114+0000 7f2b71412100 4 Got MDS map 69 2023-09-23T03:24:42.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.114+0000 7f2b71412100 10 main: main 2023-09-23T03:24:42.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.114+0000 7f2b71412100 4 main: connecting to RADOS... 2023-09-23T03:24:42.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.115+0000 7f2b71412100 1 Processor -- start 2023-09-23T03:24:42.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.115+0000 7f2b71412100 1 -- start start 2023-09-23T03:24:42.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.116+0000 7f2b71412100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56551e41bc00 0x56551d6b1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.116+0000 7f2b71412100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56551d97e000 0x56551d6b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.116+0000 7f2b71412100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56551d97e400 0x56551d6b2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.116+0000 7f2b71412100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x56551d68ca00 con 0x56551e41bc00 2023-09-23T03:24:42.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.116+0000 7f2b71412100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56551d70f180 con 0x56551d97e400 2023-09-23T03:24:42.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.116+0000 7f2b71412100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56551d70edc0 con 0x56551d97e000 2023-09-23T03:24:42.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.116+0000 7f2b6e8df640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56551e41bc00 0x56551d6b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.116+0000 7f2b6e8df640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56551e41bc00 0x56551d6b1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43272/0 (socket says 172.21.15.154:43272) 2023-09-23T03:24:42.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.116+0000 7f2b6e8df640 1 -- 172.21.15.154:0/1991566928 learned_addr learned my addr 172.21.15.154:0/1991566928 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:42.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.116+0000 7f2b6e8df640 1 -- 172.21.15.154:0/1991566928 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56551d97e000 msgr2=0x56551d6b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:42.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.116+0000 7f2b6e8df640 1 --2- 172.21.15.154:0/1991566928 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56551d97e000 0x56551d6b1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.116+0000 7f2b6e8df640 1 -- 172.21.15.154:0/1991566928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56551d97e400 msgr2=0x56551d6b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:42.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.116+0000 7f2b6e8df640 1 --2- 172.21.15.154:0/1991566928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56551d97e400 0x56551d6b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.116+0000 7f2b6e8df640 1 -- 172.21.15.154:0/1991566928 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56551d674340 con 0x56551e41bc00 2023-09-23T03:24:42.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.117+0000 7f2b6e8df640 1 --2- 172.21.15.154:0/1991566928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56551e41bc00 0x56551d6b1600 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x56551d9880f0 tx=0x56551d930d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c388fb6f9e68b46 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.117+0000 7f2b6a8d7640 1 -- 172.21.15.154:0/1991566928 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56551d68b080 con 0x56551e41bc00 2023-09-23T03:24:42.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.117+0000 7f2b6a8d7640 1 -- 172.21.15.154:0/1991566928 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56551e418480 con 0x56551e41bc00 2023-09-23T03:24:42.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.117+0000 7f2b6a8d7640 1 -- 172.21.15.154:0/1991566928 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56551d68b600 con 0x56551e41bc00 2023-09-23T03:24:42.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.117+0000 7f2b71412100 1 -- 172.21.15.154:0/1991566928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56551e41bc00 msgr2=0x56551d6b1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.117+0000 7f2b71412100 1 --2- 172.21.15.154:0/1991566928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56551e41bc00 0x56551d6b1600 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x56551d9880f0 tx=0x56551d930d20 comp rx=0 tx=0).stop 2023-09-23T03:24:42.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.117+0000 7f2b71412100 1 -- 172.21.15.154:0/1991566928 shutdown_connections 2023-09-23T03:24:42.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.117+0000 7f2b71412100 1 --2- 172.21.15.154:0/1991566928 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56551d97e400 0x56551d6b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.117+0000 7f2b71412100 1 --2- 172.21.15.154:0/1991566928 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56551d97e000 0x56551d6b1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.117+0000 7f2b71412100 1 --2- 172.21.15.154:0/1991566928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56551e41bc00 0x56551d6b1600 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.117+0000 7f2b71412100 1 -- 172.21.15.154:0/1991566928 >> 172.21.15.154:0/1991566928 conn(0x56551e41b800 msgr2=0x56551d71f000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:42.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.117+0000 7f2b71412100 1 -- 172.21.15.154:0/1991566928 shutdown_connections 2023-09-23T03:24:42.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.117+0000 7f2b71412100 1 -- 172.21.15.154:0/1991566928 wait complete. 2023-09-23T03:24:42.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.118+0000 7f2b71412100 1 Processor -- start 2023-09-23T03:24:42.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.118+0000 7f2b71412100 1 -- start start 2023-09-23T03:24:42.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.119+0000 7f2b71412100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56551e41b800 0x56551d6b3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.119+0000 7f2b71412100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56551d97f000 0x56551d6b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.119+0000 7f2b71412100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56551d97ec00 0x56551d6b2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.119+0000 7f2b71412100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x56551d68c140 con 0x56551e41b800 2023-09-23T03:24:42.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.119+0000 7f2b71412100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56551d68d180 con 0x56551d97f000 2023-09-23T03:24:42.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.119+0000 7f2b71412100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56551d68cb40 con 0x56551d97ec00 2023-09-23T03:24:42.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.119+0000 7f2b6e8df640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56551e41b800 0x56551d6b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.119+0000 7f2b6e8df640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56551e41b800 0x56551d6b3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43282/0 (socket says 172.21.15.154:43282) 2023-09-23T03:24:42.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.119+0000 7f2b6e8df640 1 -- 172.21.15.154:0/4279406629 learned_addr learned my addr 172.21.15.154:0/4279406629 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:42.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.120+0000 7f2b6e8df640 1 -- 172.21.15.154:0/4279406629 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56551d97ec00 msgr2=0x56551d6b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.120+0000 7f2b6e8df640 1 --2- 172.21.15.154:0/4279406629 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56551d97ec00 0x56551d6b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.120+0000 7f2b6e8df640 1 -- 172.21.15.154:0/4279406629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56551d97f000 msgr2=0x56551d6b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.120+0000 7f2b6e8df640 1 --2- 172.21.15.154:0/4279406629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56551d97f000 0x56551d6b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.120+0000 7f2b6e8df640 1 -- 172.21.15.154:0/4279406629 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56551d674000 con 0x56551e41b800 2023-09-23T03:24:42.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.120+0000 7f2b6e8df640 1 --2- 172.21.15.154:0/4279406629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56551e41b800 0x56551d6b3180 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x56551d9893b0 tx=0x56551d930c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.120+0000 7f2b690d4640 1 -- 172.21.15.154:0/4279406629 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56551d68adc0 con 0x56551e41b800 2023-09-23T03:24:42.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.120+0000 7f2b690d4640 1 -- 172.21.15.154:0/4279406629 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56551e418780 con 0x56551e41b800 2023-09-23T03:24:42.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.120+0000 7f2b690d4640 1 -- 172.21.15.154:0/4279406629 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56551d68b080 con 0x56551e41b800 2023-09-23T03:24:42.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.120+0000 7f2b71412100 1 -- 172.21.15.154:0/4279406629 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56551d674ea0 con 0x56551e41b800 2023-09-23T03:24:42.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.120+0000 7f2b71412100 1 -- 172.21.15.154:0/4279406629 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56551d675520 con 0x56551e41b800 2023-09-23T03:24:42.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.120+0000 7f2b71412100 4 main: resolving pool 11 2023-09-23T03:24:42.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.122+0000 7f2b690d4640 1 -- 172.21.15.154:0/4279406629 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x56551d672840 con 0x56551e41b800 2023-09-23T03:24:42.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.123+0000 7f2b690d4640 1 --2- 172.21.15.154:0/4279406629 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x56551da8a000 0x56551d6b3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.123+0000 7f2b690d4640 1 -- 172.21.15.154:0/4279406629 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6605+0+0 (secure 0 0 0) 0x56551d66c540 con 0x56551e41b800 2023-09-23T03:24:42.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.123+0000 7f2b71412100 4 main: creating IoCtx.. 2023-09-23T03:24:42.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.123+0000 7f2b6f0e0640 1 --2- 172.21.15.154:0/4279406629 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x56551da8a000 0x56551d6b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.123+0000 7f2b71412100 1 --2- 172.21.15.154:0/4279406629 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x56551d97e800 0x56551d6b1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.123+0000 7f2b71412100 1 -- 172.21.15.154:0/4279406629 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:1 11.13 11:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x56551d69d880 con 0x56551d97e800 2023-09-23T03:24:42.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.123+0000 7f2b6f0e0640 1 --2- 172.21.15.154:0/4279406629 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x56551da8a000 0x56551d6b3700 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x56551e39a0c0 tx=0x56551da940c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.124+0000 7f2b6e8df640 1 --2- 172.21.15.154:0/4279406629 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x56551d97e800 0x56551d6b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.125+0000 7f2b6e8df640 1 --2- 172.21.15.154:0/4279406629 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x56551d97e800 0x56551d6b1600 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.128+0000 7f2b6e8df640 1 -- 172.21.15.154:0/4279406629 <== osd.5 v2:172.21.15.190:6800/882536885 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v88'6 uv6 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x56551d916b40 con 0x56551d97e800 2023-09-23T03:24:42.129 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:24:42.129 INFO:teuthology.orchestra.run.smithi154.stdout: "0": { 2023-09-23T03:24:42.129 INFO:teuthology.orchestra.run.smithi154.stdout: "data": {}, 2023-09-23T03:24:42.129 INFO:teuthology.orchestra.run.smithi154.stdout: "result": 0 2023-09-23T03:24:42.130 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:24:42.130 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:24:42.130 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:42.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.128+0000 7f2b71412100 1 -- 172.21.15.154:0/4279406629 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x56551d97e800 msgr2=0x56551d6b1600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.128+0000 7f2b71412100 1 --2- 172.21.15.154:0/4279406629 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x56551d97e800 0x56551d6b1600 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.128+0000 7f2b71412100 1 -- 172.21.15.154:0/4279406629 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x56551da8a000 msgr2=0x56551d6b3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.128+0000 7f2b71412100 1 --2- 172.21.15.154:0/4279406629 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x56551da8a000 0x56551d6b3700 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x56551e39a0c0 tx=0x56551da940c0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.128+0000 7f2b71412100 1 -- 172.21.15.154:0/4279406629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56551e41b800 msgr2=0x56551d6b3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.128+0000 7f2b71412100 1 --2- 172.21.15.154:0/4279406629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56551e41b800 0x56551d6b3180 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x56551d9893b0 tx=0x56551d930c00 comp rx=0 tx=0).stop 2023-09-23T03:24:42.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.128+0000 7f2b6f0e0640 1 -- 172.21.15.154:0/4279406629 reap_dead start 2023-09-23T03:24:42.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.128+0000 7f2b71412100 1 -- 172.21.15.154:0/4279406629 shutdown_connections 2023-09-23T03:24:42.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.128+0000 7f2b71412100 1 -- 172.21.15.154:0/4279406629 >> 172.21.15.154:0/4279406629 conn(0x56551e41bc00 msgr2=0x56551d71f000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:42.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.128+0000 7f2b71412100 1 -- 172.21.15.154:0/4279406629 shutdown_connections 2023-09-23T03:24:42.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.128+0000 7f2b71412100 1 -- 172.21.15.154:0/4279406629 wait complete. 2023-09-23T03:24:42.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.129+0000 7f2b71412100 1 -- 172.21.15.154:0/3825266099 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56551d641c00 msgr2=0x56551d6b1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.129+0000 7f2b71412100 1 --2- 172.21.15.154:0/3825266099 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56551d641c00 0x56551d6b1080 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x56551e3ce420 tx=0x56551d918120 comp rx=0 tx=0).stop 2023-09-23T03:24:42.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.129+0000 7f2b71412100 1 -- 172.21.15.154:0/3825266099 shutdown_connections 2023-09-23T03:24:42.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.129+0000 7f2b71412100 1 --2- 172.21.15.154:0/3825266099 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56551d641c00 0x56551d6b1080 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.129+0000 7f2b71412100 1 --2- 172.21.15.154:0/3825266099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56551d641800 0x56551d6b0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.129+0000 7f2b71412100 1 --2- 172.21.15.154:0/3825266099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56551d640c00 0x56551d6b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.129+0000 7f2b71412100 1 -- 172.21.15.154:0/3825266099 >> 172.21.15.154:0/3825266099 conn(0x56551d641000 msgr2=0x56551d71e800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:42.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.129+0000 7f2b71412100 1 -- 172.21.15.154:0/3825266099 shutdown_connections 2023-09-23T03:24:42.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.129+0000 7f2b71412100 1 -- 172.21.15.154:0/3825266099 wait complete. 2023-09-23T03:24:42.134 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'inode'] in time 0:00:00.101600, result: { "0": { "data": {}, "result": 0 } } 2023-09-23T03:24:42.135 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2023-09-23T03:24:42.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.215+0000 7f637ea93100 1 -- start start 2023-09-23T03:24:42.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.215+0000 7f637ea93100 1 Processor -- start 2023-09-23T03:24:42.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.216+0000 7f637ea93100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9600aac00 0x55e96011a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.216+0000 7f637ea93100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e9600ab800 0x55e96011ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.216+0000 7f637ea93100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e9600abc00 0x55e96011b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.216+0000 7f637ea93100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9600f7180 con 0x55e9600abc00 2023-09-23T03:24:42.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.216+0000 7f637ea93100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9600f6140 con 0x55e9600aac00 2023-09-23T03:24:42.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.216+0000 7f637ea93100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e9600f6000 con 0x55e9600ab800 2023-09-23T03:24:42.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.216+0000 7f637da3c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e9600abc00 0x55e96011b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.216+0000 7f637da3c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e9600abc00 0x55e96011b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43302/0 (socket says 172.21.15.154:43302) 2023-09-23T03:24:42.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.216+0000 7f637da3c640 1 -- 172.21.15.154:0/340599504 learned_addr learned my addr 172.21.15.154:0/340599504 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:42.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.216+0000 7f637d23b640 1 --2- 172.21.15.154:0/340599504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9600aac00 0x55e96011a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.216+0000 7f637da3c640 1 -- 172.21.15.154:0/340599504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e9600ab800 msgr2=0x55e96011ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:42.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.216+0000 7f637da3c640 1 --2- 172.21.15.154:0/340599504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e9600ab800 0x55e96011ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.216+0000 7f637da3c640 1 -- 172.21.15.154:0/340599504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9600aac00 msgr2=0x55e96011a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:42.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.216+0000 7f637da3c640 1 --2- 172.21.15.154:0/340599504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9600aac00 0x55e96011a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.216+0000 7f637da3c640 1 -- 172.21.15.154:0/340599504 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9600deb60 con 0x55e9600abc00 2023-09-23T03:24:42.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.216+0000 7f637d23b640 1 --2- 172.21.15.154:0/340599504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9600aac00 0x55e96011a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:42.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.217+0000 7f637da3c640 1 --2- 172.21.15.154:0/340599504 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e9600abc00 0x55e96011b080 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x55e960ebad50 tx=0x55e960f4c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b0fa06d119decfb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.217+0000 7f637aa36640 1 -- 172.21.15.154:0/340599504 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e9600f5600 con 0x55e9600abc00 2023-09-23T03:24:42.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.217+0000 7f637aa36640 1 -- 172.21.15.154:0/340599504 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e9600d1b00 con 0x55e9600abc00 2023-09-23T03:24:42.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.217+0000 7f637aa36640 1 -- 172.21.15.154:0/340599504 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e9600f51e0 con 0x55e9600abc00 2023-09-23T03:24:42.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.217+0000 7f637ea93100 1 -- 172.21.15.154:0/340599504 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e9600de340 con 0x55e9600abc00 2023-09-23T03:24:42.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.217+0000 7f637ea93100 1 -- 172.21.15.154:0/340599504 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55e9600de000 con 0x55e9600abc00 2023-09-23T03:24:42.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.217+0000 7f637ea93100 4 waiting for MDS map... 2023-09-23T03:24:42.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.217+0000 7f637aa36640 1 -- 172.21.15.154:0/340599504 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(88..88 src has 1..88) v4 ==== 6605+0+0 (secure 0 0 0) 0x55e9600d6380 con 0x55e9600abc00 2023-09-23T03:24:42.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.217+0000 7f637aa36640 1 -- 172.21.15.154:0/340599504 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 69) v1 ==== 3306+0+0 (secure 0 0 0) 0x55e96038c000 con 0x55e9600abc00 2023-09-23T03:24:42.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.217+0000 7f637ea93100 4 Got MDS map 69 2023-09-23T03:24:42.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.217+0000 7f637ea93100 10 main: JournalTool::main 2023-09-23T03:24:42.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.217+0000 7f637ea93100 4 main: JournalTool: connecting to RADOS... 2023-09-23T03:24:42.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.218+0000 7f637ea93100 1 Processor -- start 2023-09-23T03:24:42.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.218+0000 7f637ea93100 1 -- start start 2023-09-23T03:24:42.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.219+0000 7f637ea93100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e960395000 0x55e96011b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.219+0000 7f637ea93100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e960395400 0x55e96011bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.219+0000 7f637ea93100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e960395800 0x55e96011c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.219+0000 7f637ea93100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9600f6a00 con 0x55e960395400 2023-09-23T03:24:42.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.219+0000 7f637ea93100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e960179180 con 0x55e960395800 2023-09-23T03:24:42.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.219+0000 7f637ea93100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e960178dc0 con 0x55e960395000 2023-09-23T03:24:42.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.219+0000 7f637d23b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e960395400 0x55e96011bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.219+0000 7f637d23b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e960395400 0x55e96011bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43308/0 (socket says 172.21.15.154:43308) 2023-09-23T03:24:42.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.219+0000 7f637d23b640 1 -- 172.21.15.154:0/3109938291 learned_addr learned my addr 172.21.15.154:0/3109938291 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:42.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.219+0000 7f637d23b640 1 -- 172.21.15.154:0/3109938291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e960395000 msgr2=0x55e96011b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:42.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.219+0000 7f637d23b640 1 --2- 172.21.15.154:0/3109938291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e960395000 0x55e96011b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.220+0000 7f637d23b640 1 -- 172.21.15.154:0/3109938291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e960395800 msgr2=0x55e96011c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:42.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.220+0000 7f637d23b640 1 --2- 172.21.15.154:0/3109938291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e960395800 0x55e96011c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.220+0000 7f637d23b640 1 -- 172.21.15.154:0/3109938291 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9600deea0 con 0x55e960395400 2023-09-23T03:24:42.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.220+0000 7f637d23b640 1 --2- 172.21.15.154:0/3109938291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e960395400 0x55e96011bb80 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x55e9603f00c0 tx=0x55e960f4c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5692c1809f72100b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.220+0000 7f6378a32640 1 -- 172.21.15.154:0/3109938291 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e9600f5080 con 0x55e960395400 2023-09-23T03:24:42.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.220+0000 7f6378a32640 1 -- 172.21.15.154:0/3109938291 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e9600d0a80 con 0x55e960395400 2023-09-23T03:24:42.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.220+0000 7f6378a32640 1 -- 172.21.15.154:0/3109938291 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e9600f5600 con 0x55e960395400 2023-09-23T03:24:42.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.220+0000 7f637ea93100 1 -- 172.21.15.154:0/3109938291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e960395400 msgr2=0x55e96011bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.220+0000 7f637ea93100 1 --2- 172.21.15.154:0/3109938291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e960395400 0x55e96011bb80 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x55e9603f00c0 tx=0x55e960f4c240 comp rx=0 tx=0).stop 2023-09-23T03:24:42.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.220+0000 7f637ea93100 1 -- 172.21.15.154:0/3109938291 shutdown_connections 2023-09-23T03:24:42.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.220+0000 7f637ea93100 1 --2- 172.21.15.154:0/3109938291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e960395800 0x55e96011c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.220+0000 7f637ea93100 1 --2- 172.21.15.154:0/3109938291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e960395400 0x55e96011bb80 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.220+0000 7f637ea93100 1 --2- 172.21.15.154:0/3109938291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e960395000 0x55e96011b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.220+0000 7f637ea93100 1 -- 172.21.15.154:0/3109938291 >> 172.21.15.154:0/3109938291 conn(0x55e960394c00 msgr2=0x55e960e23000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:42.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.221+0000 7f637ea93100 1 -- 172.21.15.154:0/3109938291 shutdown_connections 2023-09-23T03:24:42.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.221+0000 7f637ea93100 1 -- 172.21.15.154:0/3109938291 wait complete. 2023-09-23T03:24:42.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.221+0000 7f637ea93100 1 Processor -- start 2023-09-23T03:24:42.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.221+0000 7f637ea93100 1 -- start start 2023-09-23T03:24:42.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.222+0000 7f637ea93100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e960395000 0x55e96011d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.222+0000 7f637ea93100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e960395400 0x55e96011cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.222+0000 7f637ea93100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e960395800 0x55e96011c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.222+0000 7f637ea93100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e960178c80 con 0x55e960395800 2023-09-23T03:24:42.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.222+0000 7f637ea93100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9600f6140 con 0x55e960395400 2023-09-23T03:24:42.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.222+0000 7f637ea93100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e9600f6b40 con 0x55e960395000 2023-09-23T03:24:42.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.222+0000 7f637ca3a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e960395800 0x55e96011c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.222+0000 7f637ca3a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e960395800 0x55e96011c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43312/0 (socket says 172.21.15.154:43312) 2023-09-23T03:24:42.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.222+0000 7f637ca3a640 1 -- 172.21.15.154:0/1764528864 learned_addr learned my addr 172.21.15.154:0/1764528864 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:42.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.223+0000 7f637ca3a640 1 --2- 172.21.15.154:0/1764528864 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e960395000 0x55e96011d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.223+0000 7f637d23b640 1 --2- 172.21.15.154:0/1764528864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e960395400 0x55e96011cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.223+0000 7f637ca3a640 1 -- 172.21.15.154:0/1764528864 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e960395000 msgr2=0x55e96011d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.223+0000 7f637ca3a640 1 --2- 172.21.15.154:0/1764528864 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e960395000 0x55e96011d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.223+0000 7f637ca3a640 1 -- 172.21.15.154:0/1764528864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e960395400 msgr2=0x55e96011cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.223+0000 7f637ca3a640 1 --2- 172.21.15.154:0/1764528864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e960395400 0x55e96011cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.223+0000 7f637ca3a640 1 -- 172.21.15.154:0/1764528864 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9600df1e0 con 0x55e960395800 2023-09-23T03:24:42.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.223+0000 7f637d23b640 1 --2- 172.21.15.154:0/1764528864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e960395400 0x55e96011cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:42.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.223+0000 7f637ca3a640 1 --2- 172.21.15.154:0/1764528864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e960395800 0x55e96011c680 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x55e9603f1470 tx=0x55e9603a0d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.223+0000 7f637722f640 1 -- 172.21.15.154:0/1764528864 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e9600f5080 con 0x55e960395800 2023-09-23T03:24:42.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.223+0000 7f637722f640 1 -- 172.21.15.154:0/1764528864 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e96018d800 con 0x55e960395800 2023-09-23T03:24:42.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.223+0000 7f637ea93100 1 -- 172.21.15.154:0/1764528864 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e9600deea0 con 0x55e960395800 2023-09-23T03:24:42.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.223+0000 7f637722f640 1 -- 172.21.15.154:0/1764528864 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e9600f5600 con 0x55e960395800 2023-09-23T03:24:42.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.224+0000 7f637ea93100 1 -- 172.21.15.154:0/1764528864 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e9600deb60 con 0x55e960395800 2023-09-23T03:24:42.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.224+0000 7f637ea93100 4 main: JournalTool: resolving pool 11 2023-09-23T03:24:42.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.225+0000 7f637722f640 1 -- 172.21.15.154:0/1764528864 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x55e9600dc840 con 0x55e960395800 2023-09-23T03:24:42.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.226+0000 7f637722f640 1 --2- 172.21.15.154:0/1764528864 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55e96040e800 0x55e96011d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.226+0000 7f637722f640 1 -- 172.21.15.154:0/1764528864 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6605+0+0 (secure 0 0 0) 0x55e9600d6540 con 0x55e960395800 2023-09-23T03:24:42.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.226+0000 7f637ea93100 4 main: JournalTool: creating IoCtx.. 2023-09-23T03:24:42.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.226+0000 7f637ea93100 4 main: Executing for rank 0 2023-09-23T03:24:42.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.226+0000 7f637d23b640 1 --2- 172.21.15.154:0/1764528864 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55e96040e800 0x55e96011d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.227+0000 7f637d23b640 1 --2- 172.21.15.154:0/1764528864 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55e96040e800 0x55e96011d700 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x55e9603f00c0 tx=0x55e960f4c120 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.227+0000 7f637ea93100 1 -- start start 2023-09-23T03:24:42.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.227+0000 7f637ea93100 1 Processor -- start 2023-09-23T03:24:42.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.228+0000 7f637ea93100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e96040f000 0x55e96011c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.228+0000 7f637ea93100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e96040f400 0x55e96011bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.228+0000 7f637ea93100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e96040f800 0x55e96011b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.228+0000 7f637ea93100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e960178dc0 con 0x55e96040f400 2023-09-23T03:24:42.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.228+0000 7f637ea93100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e960179180 con 0x55e96040f000 2023-09-23T03:24:42.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.228+0000 7f637ea93100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e960178b40 con 0x55e96040f800 2023-09-23T03:24:42.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.228+0000 7f637d23b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e96040f400 0x55e96011bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.228+0000 7f637d23b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e96040f400 0x55e96011bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43314/0 (socket says 172.21.15.154:43314) 2023-09-23T03:24:42.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.228+0000 7f637d23b640 1 -- 172.21.15.154:0/1156076349 learned_addr learned my addr 172.21.15.154:0/1156076349 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:42.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.229+0000 7f637da3c640 1 --2- 172.21.15.154:0/1156076349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e96040f000 0x55e96011c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.229+0000 7f637ca3a640 1 --2- 172.21.15.154:0/1156076349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e96040f800 0x55e96011b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.229+0000 7f637d23b640 1 -- 172.21.15.154:0/1156076349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e96040f800 msgr2=0x55e96011b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:42.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.229+0000 7f637d23b640 1 --2- 172.21.15.154:0/1156076349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e96040f800 0x55e96011b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.229+0000 7f637d23b640 1 -- 172.21.15.154:0/1156076349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e96040f000 msgr2=0x55e96011c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:42.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.229+0000 7f637d23b640 1 --2- 172.21.15.154:0/1156076349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e96040f000 0x55e96011c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.229+0000 7f637d23b640 1 -- 172.21.15.154:0/1156076349 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9604429c0 con 0x55e96040f400 2023-09-23T03:24:42.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.229+0000 7f637d23b640 1 --2- 172.21.15.154:0/1156076349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e96040f400 0x55e96011bb80 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x55e9604b7170 tx=0x55e960f4c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79eb994bcd8df946 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.229+0000 7f6374229640 1 -- 172.21.15.154:0/1156076349 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e9600f51e0 con 0x55e96040f400 2023-09-23T03:24:42.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.229+0000 7f6374229640 1 -- 172.21.15.154:0/1156076349 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e9600d1800 con 0x55e96040f400 2023-09-23T03:24:42.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.229+0000 7f6374229640 1 -- 172.21.15.154:0/1156076349 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e9600f4dc0 con 0x55e96040f400 2023-09-23T03:24:42.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.229+0000 7f637ea93100 1 -- 172.21.15.154:0/1156076349 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e960442820 con 0x55e96040f400 2023-09-23T03:24:42.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.229+0000 7f637ea93100 1 -- 172.21.15.154:0/1156076349 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55e960442680 con 0x55e96040f400 2023-09-23T03:24:42.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.229+0000 7f637ea93100 4 waiting for MDS map... 2023-09-23T03:24:42.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.230+0000 7f6374229640 1 -- 172.21.15.154:0/1156076349 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(88..88 src has 1..88) v4 ==== 6605+0+0 (secure 0 0 0) 0x55e9600d6a80 con 0x55e96040f400 2023-09-23T03:24:42.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.230+0000 7f6374229640 1 -- 172.21.15.154:0/1156076349 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 69) v1 ==== 3306+0+0 (secure 0 0 0) 0x55e960568000 con 0x55e96040f400 2023-09-23T03:24:42.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.230+0000 7f637ea93100 4 Got MDS map 69 2023-09-23T03:24:42.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.230+0000 7f637ea93100 1 --2- 172.21.15.154:0/1156076349 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55e96040fc00 0x55e96056e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.230+0000 7f637ea93100 1 -- 172.21.15.154:0/1156076349 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:1 11.f 11:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x55e960572000 con 0x55e96040fc00 2023-09-23T03:24:42.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.231+0000 7f637da3c640 1 --2- 172.21.15.154:0/1156076349 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55e96040fc00 0x55e96056e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.232+0000 7f637da3c640 1 --2- 172.21.15.154:0/1156076349 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55e96040fc00 0x55e96056e000 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=2cbab75b79cd8421 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.237+0000 7f637da3c640 1 -- 172.21.15.154:0/1156076349 <== osd.7 v2:172.21.15.190:6824/3460136272 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v88'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55e960f44b40 con 0x55e96040fc00 2023-09-23T03:24:42.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.237+0000 7f637ea93100 1 --2- 172.21.15.154:0/1156076349 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55e960559000 0x55e96056e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.237+0000 7f637ea93100 1 -- 172.21.15.154:0/1156076349 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:2 11.14 11:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x55e960572380 con 0x55e960559000 2023-09-23T03:24:42.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.238+0000 7f637ca3a640 1 --2- 172.21.15.154:0/1156076349 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55e960559000 0x55e96056e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.239+0000 7f637ca3a640 1 --2- 172.21.15.154:0/1156076349 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55e960559000 0x55e96056e580 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=b15b1218de98c957 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.243+0000 7f637ca3a640 1 -- 172.21.15.154:0/1156076349 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v88'27 uv27 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55e960f44d80 con 0x55e960559000 2023-09-23T03:24:42.244 INFO:teuthology.orchestra.run.smithi154.stdout:writing EResetJournal entry 2023-09-23T03:24:42.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.243+0000 7f637ea93100 1 -- 172.21.15.154:0/1156076349 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:3 11.14 11:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x55e960572700 con 0x55e960559000 2023-09-23T03:24:42.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.243+0000 7f637ea93100 1 --2- 172.21.15.154:0/1156076349 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55e960559400 0x55e96056eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.243+0000 7f637ea93100 1 -- 172.21.15.154:0/1156076349 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:4 11.1b 11:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x55e960572a80 con 0x55e960559400 2023-09-23T03:24:42.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.243+0000 7f637ea93100 1 --2- 172.21.15.154:0/1156076349 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55e960559800 0x55e96056f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.243+0000 7f637ea93100 1 -- 172.21.15.154:0/1156076349 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:5 11.b 11:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x55e960572e00 con 0x55e960559800 2023-09-23T03:24:42.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.243+0000 7f637ea93100 1 --2- 172.21.15.154:0/1156076349 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55e960559c00 0x55e96056f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.243+0000 7f637ea93100 1 -- 172.21.15.154:0/1156076349 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:6 11.1f 11:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x55e960573180 con 0x55e960559c00 2023-09-23T03:24:42.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.243+0000 7f637ea93100 1 --2- 172.21.15.154:0/1156076349 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55e960596000 0x55e96056fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.243+0000 7f637ea93100 1 -- 172.21.15.154:0/1156076349 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:7 11.1a 11:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x55e960573500 con 0x55e960596000 2023-09-23T03:24:42.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.243+0000 7f637ea93100 1 -- 172.21.15.154:0/1156076349 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:8 11.1e 11:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x55e960572380 con 0x55e96040fc00 2023-09-23T03:24:42.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.243+0000 7f637ca3a640 1 --2- 172.21.15.154:0/1156076349 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55e960559c00 0x55e96056f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.244+0000 7f637d23b640 1 --2- 172.21.15.154:0/1156076349 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55e960559800 0x55e96056f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.244+0000 7f637da3c640 1 --2- 172.21.15.154:0/1156076349 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55e960559400 0x55e96056eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.244+0000 7f637da3c640 1 --2- 172.21.15.154:0/1156076349 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55e960596000 0x55e96056fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.244+0000 7f637ca3a640 1 --2- 172.21.15.154:0/1156076349 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55e960559c00 0x55e96056f600 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=9f1ba3d9a2d859c4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.244+0000 7f637d23b640 1 --2- 172.21.15.154:0/1156076349 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55e960559800 0x55e96056f080 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=6f2d7fe46b9b233 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.245+0000 7f637da3c640 1 --2- 172.21.15.154:0/1156076349 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55e960559400 0x55e96056eb00 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=326d4d7b823f4f39 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.245+0000 7f637ca3a640 1 -- 172.21.15.154:0/1156076349 <== osd.2 v2:172.21.15.154:6822/2884407243 1 ==== osd_op_reply(6 200.00000004 [delete] v88'9 uv8 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55e960f44d80 con 0x55e960559c00 2023-09-23T03:24:42.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.246+0000 7f637d23b640 1 -- 172.21.15.154:0/1156076349 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(5 200.00000003 [delete] v88'6 uv5 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55e960f45440 con 0x55e960559800 2023-09-23T03:24:42.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.246+0000 7f637da3c640 1 --2- 172.21.15.154:0/1156076349 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55e960596000 0x55e96056fb80 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=68d02b00522f21c1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.248+0000 7f637da3c640 1 -- 172.21.15.154:0/1156076349 <== osd.7 v2:172.21.15.190:6824/3460136272 2 ==== osd_op_reply(8 200.00000006 [delete] v88'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55e960f44b40 con 0x55e96040fc00 2023-09-23T03:24:42.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.248+0000 7f637ca3a640 1 -- 172.21.15.154:0/1156076349 <== osd.6 v2:172.21.15.190:6816/1964294001 2 ==== osd_op_reply(3 200.00000001 [delete] v88'28 uv27 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55e960f44d80 con 0x55e960559000 2023-09-23T03:24:42.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.249+0000 7f637da3c640 1 -- 172.21.15.154:0/1156076349 <== osd.4 v2:172.21.15.190:6808/717302641 1 ==== osd_op_reply(4 200.00000002 [delete] v88'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55e960f44b40 con 0x55e960559400 2023-09-23T03:24:42.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.249+0000 7f6372a26640 1 -- 172.21.15.154:0/1156076349 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:9 11.14 11:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x55e960572000 con 0x55e960559000 2023-09-23T03:24:42.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.249+0000 7f637da3c640 1 -- 172.21.15.154:0/1156076349 <== osd.5 v2:172.21.15.190:6800/882536885 1 ==== osd_op_reply(7 200.00000005 [delete] v88'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55e960f44b40 con 0x55e960596000 2023-09-23T03:24:42.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.253+0000 7f637ca3a640 1 -- 172.21.15.154:0/1156076349 <== osd.6 v2:172.21.15.190:6816/1964294001 3 ==== osd_op_reply(9 200.00000001 [write 0~48] v88'29 uv29 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55e960f44d80 con 0x55e960559000 2023-09-23T03:24:42.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.253+0000 7f637ea93100 4 Successfully wrote new journal pointer and header for rank 4:0 2023-09-23T03:24:42.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.253+0000 7f637ea93100 1 -- 172.21.15.154:0/1156076349 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55e960559c00 msgr2=0x55e96056f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.253+0000 7f637ea93100 1 --2- 172.21.15.154:0/1156076349 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55e960559c00 0x55e96056f600 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.253+0000 7f637ea93100 1 -- 172.21.15.154:0/1156076349 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55e960559800 msgr2=0x55e96056f080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.253+0000 7f637ea93100 1 --2- 172.21.15.154:0/1156076349 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55e960559800 0x55e96056f080 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.253+0000 7f637ea93100 1 -- 172.21.15.154:0/1156076349 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55e960559400 msgr2=0x55e96056eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.253+0000 7f637ea93100 1 --2- 172.21.15.154:0/1156076349 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55e960559400 0x55e96056eb00 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.253+0000 7f637ea93100 1 -- 172.21.15.154:0/1156076349 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55e960596000 msgr2=0x55e96056fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.253+0000 7f637ea93100 1 --2- 172.21.15.154:0/1156076349 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55e960596000 0x55e96056fb80 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.253+0000 7f637ea93100 1 -- 172.21.15.154:0/1156076349 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55e960559000 msgr2=0x55e96056e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.253+0000 7f637ea93100 1 --2- 172.21.15.154:0/1156076349 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55e960559000 0x55e96056e580 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.253+0000 7f637ea93100 1 -- 172.21.15.154:0/1156076349 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55e96040fc00 msgr2=0x55e96056e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.253+0000 7f637ea93100 1 --2- 172.21.15.154:0/1156076349 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55e96040fc00 0x55e96056e000 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.253+0000 7f637ea93100 1 -- 172.21.15.154:0/1156076349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e96040f400 msgr2=0x55e96011bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.253+0000 7f637ea93100 1 --2- 172.21.15.154:0/1156076349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e96040f400 0x55e96011bb80 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x55e9604b7170 tx=0x55e960f4c300 comp rx=0 tx=0).stop 2023-09-23T03:24:42.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.253+0000 7f637ca3a640 1 -- 172.21.15.154:0/1156076349 reap_dead start 2023-09-23T03:24:42.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.253+0000 7f637ca3a640 1 -- 172.21.15.154:0/1156076349 reap_dead start 2023-09-23T03:24:42.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.253+0000 7f637ca3a640 1 -- 172.21.15.154:0/1156076349 reap_dead start 2023-09-23T03:24:42.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.253+0000 7f637ea93100 1 -- 172.21.15.154:0/1156076349 shutdown_connections 2023-09-23T03:24:42.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.253+0000 7f637ea93100 1 -- 172.21.15.154:0/1156076349 >> 172.21.15.154:0/1156076349 conn(0x55e96040ec00 msgr2=0x55e960e23800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:42.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.253+0000 7f637ea93100 1 -- 172.21.15.154:0/1156076349 shutdown_connections 2023-09-23T03:24:42.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.253+0000 7f637ea93100 1 -- 172.21.15.154:0/1156076349 wait complete. 2023-09-23T03:24:42.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.254+0000 7f637ea93100 1 -- 172.21.15.154:0/1764528864 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55e96040e800 msgr2=0x55e96011d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.254+0000 7f637ea93100 1 --2- 172.21.15.154:0/1764528864 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55e96040e800 0x55e96011d700 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x55e9603f00c0 tx=0x55e960f4c120 comp rx=0 tx=0).stop 2023-09-23T03:24:42.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.254+0000 7f637ea93100 1 -- 172.21.15.154:0/1764528864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e960395800 msgr2=0x55e96011c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.254+0000 7f637ea93100 1 --2- 172.21.15.154:0/1764528864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e960395800 0x55e96011c680 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x55e9603f1470 tx=0x55e9603a0d20 comp rx=0 tx=0).stop 2023-09-23T03:24:42.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.254+0000 7f637ea93100 1 -- 172.21.15.154:0/1764528864 shutdown_connections 2023-09-23T03:24:42.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.254+0000 7f637ea93100 1 --2- 172.21.15.154:0/1764528864 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55e96040e800 0x55e96011d700 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.254+0000 7f637ea93100 1 --2- 172.21.15.154:0/1764528864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e960395800 0x55e96011c680 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.254+0000 7f637ea93100 1 --2- 172.21.15.154:0/1764528864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e960395400 0x55e96011cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.254+0000 7f637ea93100 1 --2- 172.21.15.154:0/1764528864 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e960395000 0x55e96011d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.254+0000 7f637ea93100 1 -- 172.21.15.154:0/1764528864 >> 172.21.15.154:0/1764528864 conn(0x55e960394c00 msgr2=0x55e960e23000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:42.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.254+0000 7f637ea93100 1 -- 172.21.15.154:0/1764528864 shutdown_connections 2023-09-23T03:24:42.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.254+0000 7f637ea93100 1 -- 172.21.15.154:0/1764528864 wait complete. 2023-09-23T03:24:42.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.254+0000 7f637ea93100 1 -- 172.21.15.154:0/340599504 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e9600abc00 msgr2=0x55e96011b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.254+0000 7f637ea93100 1 --2- 172.21.15.154:0/340599504 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e9600abc00 0x55e96011b080 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x55e960ebad50 tx=0x55e960f4c420 comp rx=0 tx=0).stop 2023-09-23T03:24:42.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.254+0000 7f637ea93100 1 -- 172.21.15.154:0/340599504 shutdown_connections 2023-09-23T03:24:42.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.254+0000 7f637ea93100 1 --2- 172.21.15.154:0/340599504 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e9600abc00 0x55e96011b080 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.254+0000 7f637ea93100 1 --2- 172.21.15.154:0/340599504 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e9600ab800 0x55e96011ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.254+0000 7f637ea93100 1 --2- 172.21.15.154:0/340599504 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e9600aac00 0x55e96011a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.254+0000 7f637ea93100 1 -- 172.21.15.154:0/340599504 >> 172.21.15.154:0/340599504 conn(0x55e9600ab000 msgr2=0x55e960e22800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:42.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.255+0000 7f637ea93100 1 -- 172.21.15.154:0/340599504 shutdown_connections 2023-09-23T03:24:42.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.255+0000 7f637ea93100 1 -- 172.21.15.154:0/340599504 wait complete. 2023-09-23T03:24:42.261 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.126026, result: writing EResetJournal entry 2023-09-23T03:24:42.261 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 init 2023-09-23T03:24:42.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.355+0000 7f44df40c5c0 1 -- start start 2023-09-23T03:24:42.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.355+0000 7f44df40c5c0 1 Processor -- start 2023-09-23T03:24:42.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.356+0000 7f44df40c5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ec077ac00 0x560ec07e4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.356+0000 7f44df40c5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560ec077b800 0x560ec07e4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.356+0000 7f44df40c5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ec077bc00 0x560ec07e5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.356+0000 7f44df40c5c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ec07c5180 con 0x560ec077ac00 2023-09-23T03:24:42.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.356+0000 7f44df40c5c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ec07c4140 con 0x560ec077bc00 2023-09-23T03:24:42.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.356+0000 7f44df40c5c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x560ec07c4000 con 0x560ec077b800 2023-09-23T03:24:42.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.356+0000 7f44dcfe1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ec077ac00 0x560ec07e4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.356+0000 7f44dcfe1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ec077ac00 0x560ec07e4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43318/0 (socket says 172.21.15.154:43318) 2023-09-23T03:24:42.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.356+0000 7f44dcfe1640 1 -- 172.21.15.154:0/3998605711 learned_addr learned my addr 172.21.15.154:0/3998605711 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:42.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.356+0000 7f44dcfe1640 1 -- 172.21.15.154:0/3998605711 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560ec077b800 msgr2=0x560ec07e4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:42.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.356+0000 7f44dcfe1640 1 --2- 172.21.15.154:0/3998605711 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560ec077b800 0x560ec07e4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.356+0000 7f44dcfe1640 1 -- 172.21.15.154:0/3998605711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ec077bc00 msgr2=0x560ec07e5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:42.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.356+0000 7f44dcfe1640 1 --2- 172.21.15.154:0/3998605711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ec077bc00 0x560ec07e5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.356+0000 7f44dcfe1640 1 -- 172.21.15.154:0/3998605711 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ec07aeb60 con 0x560ec077ac00 2023-09-23T03:24:42.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.357+0000 7f44dcfe1640 1 --2- 172.21.15.154:0/3998605711 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ec077ac00 0x560ec07e4580 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x560ec157d3e0 tx=0x560ec0a5c120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c934795f034a1156 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.357+0000 7f44da7dc640 1 -- 172.21.15.154:0/3998605711 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560ec07c3600 con 0x560ec077ac00 2023-09-23T03:24:42.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.357+0000 7f44da7dc640 1 -- 172.21.15.154:0/3998605711 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560ec07a1800 con 0x560ec077ac00 2023-09-23T03:24:42.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.357+0000 7f44da7dc640 1 -- 172.21.15.154:0/3998605711 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560ec07c31e0 con 0x560ec077ac00 2023-09-23T03:24:42.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.357+0000 7f44df40c5c0 1 -- 172.21.15.154:0/3998605711 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560ec07ae340 con 0x560ec077ac00 2023-09-23T03:24:42.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.357+0000 7f44df40c5c0 1 -- 172.21.15.154:0/3998605711 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x560ec07ae000 con 0x560ec077ac00 2023-09-23T03:24:42.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.357+0000 7f44df40c5c0 4 waiting for MDS map... 2023-09-23T03:24:42.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.357+0000 7f44da7dc640 1 -- 172.21.15.154:0/3998605711 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(88..88 src has 1..88) v4 ==== 6605+0+0 (secure 0 0 0) 0x560ec083ac40 con 0x560ec077ac00 2023-09-23T03:24:42.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.357+0000 7f44da7dc640 1 -- 172.21.15.154:0/3998605711 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 69) v1 ==== 3306+0+0 (secure 0 0 0) 0x560ec07cd200 con 0x560ec077ac00 2023-09-23T03:24:42.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.357+0000 7f44df40c5c0 4 Got MDS map 69 2023-09-23T03:24:42.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.357+0000 7f44df40c5c0 4 datascan.main: Using metadata pool output 2023-09-23T03:24:42.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.357+0000 7f44df40c5c0 4 datascan.main: connecting to RADOS... 2023-09-23T03:24:42.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.358+0000 7f44df40c5c0 1 Processor -- start 2023-09-23T03:24:42.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.358+0000 7f44df40c5c0 1 -- start start 2023-09-23T03:24:42.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.359+0000 7f44df40c5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ec0a67000 0x560ec07e5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.359+0000 7f44df40c5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560ec0a67400 0x560ec07e5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.359+0000 7f44df40c5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ec0a67800 0x560ec07e6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.359+0000 7f44df40c5c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ec07c4a00 con 0x560ec0a67000 2023-09-23T03:24:42.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.359+0000 7f44df40c5c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ec0849180 con 0x560ec0a67800 2023-09-23T03:24:42.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.359+0000 7f44df40c5c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x560ec0848dc0 con 0x560ec0a67400 2023-09-23T03:24:42.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.359+0000 7f44dd7e2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ec0a67000 0x560ec07e5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.359+0000 7f44dd7e2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ec0a67000 0x560ec07e5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43328/0 (socket says 172.21.15.154:43328) 2023-09-23T03:24:42.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.359+0000 7f44dd7e2640 1 -- 172.21.15.154:0/1172982080 learned_addr learned my addr 172.21.15.154:0/1172982080 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:42.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.360+0000 7f44dd7e2640 1 -- 172.21.15.154:0/1172982080 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560ec0a67400 msgr2=0x560ec07e5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:42.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.360+0000 7f44dcfe1640 1 --2- 172.21.15.154:0/1172982080 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560ec0a67400 0x560ec07e5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.360+0000 7f44dd7e2640 1 --2- 172.21.15.154:0/1172982080 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560ec0a67400 0x560ec07e5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.360+0000 7f44dd7e2640 1 -- 172.21.15.154:0/1172982080 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ec0a67800 msgr2=0x560ec07e6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:42.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.360+0000 7f44dd7e2640 1 --2- 172.21.15.154:0/1172982080 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ec0a67800 0x560ec07e6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.360+0000 7f44dd7e2640 1 -- 172.21.15.154:0/1172982080 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ec07aeea0 con 0x560ec0a67000 2023-09-23T03:24:42.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.360+0000 7f44dd7e2640 1 --2- 172.21.15.154:0/1172982080 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ec0a67000 0x560ec07e5600 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x560ec157ca50 tx=0x560ec0a74d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=294ba1fca3c1b625 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.360+0000 7f44d87d8640 1 -- 172.21.15.154:0/1172982080 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560ec07c3080 con 0x560ec0a67000 2023-09-23T03:24:42.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.360+0000 7f44d87d8640 1 -- 172.21.15.154:0/1172982080 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560ec1542900 con 0x560ec0a67000 2023-09-23T03:24:42.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.360+0000 7f44d87d8640 1 -- 172.21.15.154:0/1172982080 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560ec07c3600 con 0x560ec0a67000 2023-09-23T03:24:42.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.361+0000 7f44df40c5c0 1 -- 172.21.15.154:0/1172982080 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ec0a67000 msgr2=0x560ec07e5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.361+0000 7f44df40c5c0 1 --2- 172.21.15.154:0/1172982080 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ec0a67000 0x560ec07e5600 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x560ec157ca50 tx=0x560ec0a74d20 comp rx=0 tx=0).stop 2023-09-23T03:24:42.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.361+0000 7f44df40c5c0 1 -- 172.21.15.154:0/1172982080 shutdown_connections 2023-09-23T03:24:42.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.361+0000 7f44df40c5c0 1 --2- 172.21.15.154:0/1172982080 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ec0a67800 0x560ec07e6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.361+0000 7f44df40c5c0 1 --2- 172.21.15.154:0/1172982080 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560ec0a67400 0x560ec07e5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.361+0000 7f44df40c5c0 1 --2- 172.21.15.154:0/1172982080 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ec0a67000 0x560ec07e5600 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.361+0000 7f44df40c5c0 1 -- 172.21.15.154:0/1172982080 >> 172.21.15.154:0/1172982080 conn(0x560ec0a66c00 msgr2=0x560ec14f5000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:42.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.361+0000 7f44df40c5c0 1 -- 172.21.15.154:0/1172982080 shutdown_connections 2023-09-23T03:24:42.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.361+0000 7f44df40c5c0 1 -- 172.21.15.154:0/1172982080 wait complete. 2023-09-23T03:24:42.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.362+0000 7f44df40c5c0 1 Processor -- start 2023-09-23T03:24:42.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.362+0000 7f44df40c5c0 1 -- start start 2023-09-23T03:24:42.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.362+0000 7f44df40c5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ec0a67000 0x560ec07e7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.362+0000 7f44df40c5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560ec0a67400 0x560ec07e6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.363+0000 7f44df40c5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ec0a67800 0x560ec07e6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.363+0000 7f44df40c5c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ec0848c80 con 0x560ec0a67800 2023-09-23T03:24:42.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.363+0000 7f44df40c5c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ec07c4140 con 0x560ec0a67000 2023-09-23T03:24:42.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.363+0000 7f44df40c5c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x560ec07c4b40 con 0x560ec0a67400 2023-09-23T03:24:42.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.363+0000 7f44dc7e0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ec0a67800 0x560ec07e6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.363+0000 7f44dc7e0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ec0a67800 0x560ec07e6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43330/0 (socket says 172.21.15.154:43330) 2023-09-23T03:24:42.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.363+0000 7f44dc7e0640 1 -- 172.21.15.154:0/2524886068 learned_addr learned my addr 172.21.15.154:0/2524886068 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:42.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.363+0000 7f44dd7e2640 1 --2- 172.21.15.154:0/2524886068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ec0a67000 0x560ec07e7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.363+0000 7f44dcfe1640 1 --2- 172.21.15.154:0/2524886068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560ec0a67400 0x560ec07e6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.363+0000 7f44dc7e0640 1 -- 172.21.15.154:0/2524886068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560ec0a67400 msgr2=0x560ec07e6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.363+0000 7f44dc7e0640 1 --2- 172.21.15.154:0/2524886068 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560ec0a67400 0x560ec07e6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.363+0000 7f44dc7e0640 1 -- 172.21.15.154:0/2524886068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ec0a67000 msgr2=0x560ec07e7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.363+0000 7f44dc7e0640 1 --2- 172.21.15.154:0/2524886068 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ec0a67000 0x560ec07e7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.363+0000 7f44dc7e0640 1 -- 172.21.15.154:0/2524886068 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ec07af1e0 con 0x560ec0a67800 2023-09-23T03:24:42.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.364+0000 7f44dc7e0640 1 --2- 172.21.15.154:0/2524886068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ec0a67800 0x560ec07e6680 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x560ec0a715c0 tx=0x560ec0a75020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.364+0000 7f44d6fd5640 1 -- 172.21.15.154:0/2524886068 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560ec07c3080 con 0x560ec0a67800 2023-09-23T03:24:42.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.364+0000 7f44d6fd5640 1 -- 172.21.15.154:0/2524886068 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560ec07a1b00 con 0x560ec0a67800 2023-09-23T03:24:42.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.364+0000 7f44d6fd5640 1 -- 172.21.15.154:0/2524886068 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560ec07c3600 con 0x560ec0a67800 2023-09-23T03:24:42.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.364+0000 7f44df40c5c0 1 -- 172.21.15.154:0/2524886068 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560ec07aeea0 con 0x560ec0a67800 2023-09-23T03:24:42.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.364+0000 7f44df40c5c0 1 -- 172.21.15.154:0/2524886068 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560ec07aeb60 con 0x560ec0a67800 2023-09-23T03:24:42.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.364+0000 7f44df40c5c0 4 datascan.init: resolving metadata pool 11 2023-09-23T03:24:42.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.366+0000 7f44d6fd5640 1 -- 172.21.15.154:0/2524886068 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x560ec07ac840 con 0x560ec0a67800 2023-09-23T03:24:42.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.366+0000 7f44d6fd5640 1 --2- 172.21.15.154:0/2524886068 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x560ec0ada800 0x560ec07e7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.367+0000 7f44d6fd5640 1 -- 172.21.15.154:0/2524886068 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6605+0+0 (secure 0 0 0) 0x560ec083aa80 con 0x560ec0a67800 2023-09-23T03:24:42.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.367+0000 7f44df40c5c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-09-23T03:24:42.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.367+0000 7f44df40c5c0 1 --2- 172.21.15.154:0/2524886068 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x560ec0adac00 0x560ec07e6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.367+0000 7f44df40c5c0 1 -- 172.21.15.154:0/2524886068 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:1 11.14 11:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x560ec07d5880 con 0x560ec0adac00 2023-09-23T03:24:42.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.367+0000 7f44dd7e2640 1 --2- 172.21.15.154:0/2524886068 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x560ec0ada800 0x560ec07e7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.367+0000 7f44dd7e2640 1 --2- 172.21.15.154:0/2524886068 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x560ec0ada800 0x560ec07e7700 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x560ec157ca50 tx=0x560ec0a74c00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.367+0000 7f44dcfe1640 1 --2- 172.21.15.154:0/2524886068 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x560ec0adac00 0x560ec07e6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.368+0000 7f44dcfe1640 1 --2- 172.21.15.154:0/2524886068 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x560ec0adac00 0x560ec07e6100 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.369+0000 7f44dcfe1640 1 -- 172.21.15.154:0/2524886068 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== osd_op_reply(1 1.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 160+0+0 (crc 0 0 0) 0x560ec161cb40 con 0x560ec0adac00 2023-09-23T03:24:42.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.369+0000 7f44df40c5c0 1 -- 172.21.15.154:0/2524886068 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:2 11.14 11:287034c4:::1.00000000.inode:head [writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x560ec07d5c00 con 0x560ec0adac00 2023-09-23T03:24:42.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.373+0000 7f44dcfe1640 1 -- 172.21.15.154:0/2524886068 <== osd.6 v2:172.21.15.190:6816/1964294001 2 ==== osd_op_reply(2 1.00000000.inode [writefull 0~582] v88'30 uv30 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x560ec161cb40 con 0x560ec0adac00 2023-09-23T03:24:42.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.373+0000 7f44df40c5c0 1 --2- 172.21.15.154:0/2524886068 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x560ec0adb000 0x560ec07e5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.373+0000 7f44df40c5c0 1 -- 172.21.15.154:0/2524886068 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:3 11.1f 11:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x560ec0bde000 con 0x560ec0adb000 2023-09-23T03:24:42.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.373+0000 7f44dd7e2640 1 --2- 172.21.15.154:0/2524886068 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x560ec0adb000 0x560ec07e5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.373+0000 7f44dd7e2640 1 --2- 172.21.15.154:0/2524886068 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x560ec0adb000 0x560ec07e5b80 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.374+0000 7f44dd7e2640 1 -- 172.21.15.154:0/2524886068 <== osd.2 v2:172.21.15.154:6822/2884407243 1 ==== osd_op_reply(3 100.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 162+0+0 (crc 0 0 0) 0x560ec161d200 con 0x560ec0adb000 2023-09-23T03:24:42.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.374+0000 7f44df40c5c0 1 -- 172.21.15.154:0/2524886068 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:4 11.1f 11:fe07bba1:::100.00000000.inode:head [writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x560ec0bde380 con 0x560ec0adb000 2023-09-23T03:24:42.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.375+0000 7f44dd7e2640 1 -- 172.21.15.154:0/2524886068 <== osd.2 v2:172.21.15.154:6822/2884407243 2 ==== osd_op_reply(4 100.00000000.inode [writefull 0~582] v88'10 uv10 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x560ec161d200 con 0x560ec0adb000 2023-09-23T03:24:42.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.375+0000 7f44df40c5c0 1 --2- 172.21.15.154:0/2524886068 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x560ec0adb400 0x560ec07e5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.375+0000 7f44df40c5c0 1 -- 172.21.15.154:0/2524886068 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:5 11.13 11:cd5a64a3:::100.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x560ec0bde700 con 0x560ec0adb400 2023-09-23T03:24:42.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.376+0000 7f44dcfe1640 1 --2- 172.21.15.154:0/2524886068 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x560ec0adb400 0x560ec07e5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.377+0000 7f44dcfe1640 1 --2- 172.21.15.154:0/2524886068 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x560ec0adb400 0x560ec07e5600 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.378+0000 7f44dcfe1640 1 -- 172.21.15.154:0/2524886068 <== osd.5 v2:172.21.15.190:6800/882536885 1 ==== osd_op_reply(5 100.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x560ec161cb40 con 0x560ec0adb400 2023-09-23T03:24:42.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.378+0000 7f44df40c5c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-09-23T03:24:42.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.378+0000 7f44df40c5c0 1 -- 172.21.15.154:0/2524886068 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:6 11.13 11:cd5a64a3:::100.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x560ec0bdea80 con 0x560ec0adb400 2023-09-23T03:24:42.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.382+0000 7f44dcfe1640 1 -- 172.21.15.154:0/2524886068 <== osd.5 v2:172.21.15.190:6800/882536885 2 ==== osd_op_reply(6 100.00000000 [create,omap-set-header] v88'7 uv7 ondisk = 0) v8 ==== 198+0+0 (crc 0 0 0) 0x560ec161cb40 con 0x560ec0adb400 2023-09-23T03:24:42.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.382+0000 7f44df40c5c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x100 2023-09-23T03:24:42.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.382+0000 7f44df40c5c0 1 -- 172.21.15.154:0/2524886068 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x560ec0adb000 msgr2=0x560ec07e5b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.382+0000 7f44df40c5c0 1 --2- 172.21.15.154:0/2524886068 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x560ec0adb000 0x560ec07e5b80 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.382+0000 7f44df40c5c0 1 -- 172.21.15.154:0/2524886068 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x560ec0adb400 msgr2=0x560ec07e5600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.382+0000 7f44df40c5c0 1 --2- 172.21.15.154:0/2524886068 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x560ec0adb400 0x560ec07e5600 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.382+0000 7f44df40c5c0 1 -- 172.21.15.154:0/2524886068 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x560ec0adac00 msgr2=0x560ec07e6100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.382+0000 7f44df40c5c0 1 --2- 172.21.15.154:0/2524886068 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x560ec0adac00 0x560ec07e6100 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.382+0000 7f44dc7e0640 1 -- 172.21.15.154:0/2524886068 reap_dead start 2023-09-23T03:24:42.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.382+0000 7f44df40c5c0 1 -- 172.21.15.154:0/2524886068 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x560ec0ada800 msgr2=0x560ec07e7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.382+0000 7f44df40c5c0 1 --2- 172.21.15.154:0/2524886068 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x560ec0ada800 0x560ec07e7700 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x560ec157ca50 tx=0x560ec0a74c00 comp rx=0 tx=0).stop 2023-09-23T03:24:42.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.382+0000 7f44df40c5c0 1 -- 172.21.15.154:0/2524886068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ec0a67800 msgr2=0x560ec07e6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.382+0000 7f44df40c5c0 1 --2- 172.21.15.154:0/2524886068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ec0a67800 0x560ec07e6680 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x560ec0a715c0 tx=0x560ec0a75020 comp rx=0 tx=0).stop 2023-09-23T03:24:42.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.382+0000 7f44df40c5c0 1 -- 172.21.15.154:0/2524886068 shutdown_connections 2023-09-23T03:24:42.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.382+0000 7f44df40c5c0 1 --2- 172.21.15.154:0/2524886068 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x560ec0ada800 0x560ec07e7700 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.382+0000 7f44df40c5c0 1 --2- 172.21.15.154:0/2524886068 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ec0a67800 0x560ec07e6680 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.382+0000 7f44df40c5c0 1 -- 172.21.15.154:0/2524886068 >> 172.21.15.154:0/2524886068 conn(0x560ec0a66c00 msgr2=0x560ec14f5000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:42.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.383+0000 7f44df40c5c0 1 -- 172.21.15.154:0/2524886068 shutdown_connections 2023-09-23T03:24:42.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.383+0000 7f44df40c5c0 1 -- 172.21.15.154:0/2524886068 wait complete. 2023-09-23T03:24:42.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.383+0000 7f44df40c5c0 1 -- 172.21.15.154:0/3998605711 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ec077ac00 msgr2=0x560ec07e4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.383+0000 7f44df40c5c0 1 --2- 172.21.15.154:0/3998605711 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ec077ac00 0x560ec07e4580 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x560ec157d3e0 tx=0x560ec0a5c120 comp rx=0 tx=0).stop 2023-09-23T03:24:42.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.383+0000 7f44df40c5c0 1 -- 172.21.15.154:0/3998605711 shutdown_connections 2023-09-23T03:24:42.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.383+0000 7f44df40c5c0 1 --2- 172.21.15.154:0/3998605711 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560ec077bc00 0x560ec07e5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.383+0000 7f44df40c5c0 1 --2- 172.21.15.154:0/3998605711 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560ec077b800 0x560ec07e4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.383+0000 7f44df40c5c0 1 --2- 172.21.15.154:0/3998605711 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560ec077ac00 0x560ec07e4580 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.383+0000 7f44df40c5c0 1 -- 172.21.15.154:0/3998605711 >> 172.21.15.154:0/3998605711 conn(0x560ec077b000 msgr2=0x560ec14f4800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:42.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.383+0000 7f44df40c5c0 1 -- 172.21.15.154:0/3998605711 shutdown_connections 2023-09-23T03:24:42.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.383+0000 7f44df40c5c0 1 -- 172.21.15.154:0/3998605711 wait complete. 2023-09-23T03:24:42.390 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'init'] in time 0:00:00.129016, result: 2023-09-23T03:24:42.390 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents 2023-09-23T03:24:42.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.472+0000 7f8f994da5c0 1 -- start start 2023-09-23T03:24:42.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.472+0000 7f8f994da5c0 1 Processor -- start 2023-09-23T03:24:42.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.472+0000 7f8f994da5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598b4c94c00 0x5598b4cffb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.472+0000 7f8f994da5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5598b4c95800 0x5598b4cff600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.472+0000 7f8f994da5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598b4c95c00 0x5598b4cff080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.472+0000 7f8f994da5c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598b4d63180 con 0x5598b4c95c00 2023-09-23T03:24:42.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.472+0000 7f8f994da5c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598b4d62dc0 con 0x5598b4c94c00 2023-09-23T03:24:42.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.472+0000 7f8f994da5c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598b4cdea00 con 0x5598b4c95800 2023-09-23T03:24:42.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.473+0000 7f8f98483640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598b4c95c00 0x5598b4cff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.473+0000 7f8f98483640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598b4c95c00 0x5598b4cff080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43338/0 (socket says 172.21.15.154:43338) 2023-09-23T03:24:42.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.473+0000 7f8f98483640 1 -- 172.21.15.154:0/921388744 learned_addr learned my addr 172.21.15.154:0/921388744 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:42.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.473+0000 7f8f97c82640 1 --2- 172.21.15.154:0/921388744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598b4c94c00 0x5598b4cffb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.473+0000 7f8f98483640 1 -- 172.21.15.154:0/921388744 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5598b4c95800 msgr2=0x5598b4cff600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:42.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.473+0000 7f8f98483640 1 --2- 172.21.15.154:0/921388744 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5598b4c95800 0x5598b4cff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.473+0000 7f8f98483640 1 -- 172.21.15.154:0/921388744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598b4c94c00 msgr2=0x5598b4cffb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:42.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.473+0000 7f8f98483640 1 --2- 172.21.15.154:0/921388744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598b4c94c00 0x5598b4cffb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.473+0000 7f8f98483640 1 -- 172.21.15.154:0/921388744 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598b4cc8b60 con 0x5598b4c95c00 2023-09-23T03:24:42.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.473+0000 7f8f98483640 1 --2- 172.21.15.154:0/921388744 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598b4c95c00 0x5598b4cff080 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x5598b5a9f320 tx=0x5598b4f6c120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3493381a88448063 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.473+0000 7f8f9547d640 1 -- 172.21.15.154:0/921388744 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5598b4cdd080 con 0x5598b4c95c00 2023-09-23T03:24:42.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.474+0000 7f8f9547d640 1 -- 172.21.15.154:0/921388744 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5598b4cbaa80 con 0x5598b4c95c00 2023-09-23T03:24:42.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.474+0000 7f8f9547d640 1 -- 172.21.15.154:0/921388744 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5598b4cdd600 con 0x5598b4c95c00 2023-09-23T03:24:42.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.474+0000 7f8f994da5c0 1 -- 172.21.15.154:0/921388744 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598b4cc91e0 con 0x5598b4c95c00 2023-09-23T03:24:42.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.474+0000 7f8f994da5c0 1 -- 172.21.15.154:0/921388744 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5598b4cc8b60 con 0x5598b4c95c00 2023-09-23T03:24:42.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.474+0000 7f8f994da5c0 4 waiting for MDS map... 2023-09-23T03:24:42.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.474+0000 7f8f9547d640 1 -- 172.21.15.154:0/921388744 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(88..88 src has 1..88) v4 ==== 6605+0+0 (secure 0 0 0) 0x5598b4d54c40 con 0x5598b4c95c00 2023-09-23T03:24:42.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.474+0000 7f8f9547d640 1 -- 172.21.15.154:0/921388744 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 69) v1 ==== 3306+0+0 (secure 0 0 0) 0x5598b5a66000 con 0x5598b4c95c00 2023-09-23T03:24:42.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.474+0000 7f8f994da5c0 4 Got MDS map 69 2023-09-23T03:24:42.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.474+0000 7f8f994da5c0 4 datascan.main: Using metadata pool output 2023-09-23T03:24:42.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.474+0000 7f8f994da5c0 4 datascan.main: connecting to RADOS... 2023-09-23T03:24:42.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.475+0000 7f8f994da5c0 1 Processor -- start 2023-09-23T03:24:42.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.475+0000 7f8f994da5c0 1 -- start start 2023-09-23T03:24:42.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.476+0000 7f8f994da5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5598b4e0fc00 0x5598b4cfeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.476+0000 7f8f994da5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598b4fde000 0x5598b4cfe580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.476+0000 7f8f994da5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598b4fde400 0x5598b4d00100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.476+0000 7f8f994da5c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598b4cdeb40 con 0x5598b4fde000 2023-09-23T03:24:42.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.476+0000 7f8f994da5c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598b4d62c80 con 0x5598b4fde400 2023-09-23T03:24:42.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.476+0000 7f8f994da5c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598b4d62b40 con 0x5598b4e0fc00 2023-09-23T03:24:42.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.476+0000 7f8f97c82640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598b4fde000 0x5598b4cfe580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.476+0000 7f8f97c82640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598b4fde000 0x5598b4cfe580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43346/0 (socket says 172.21.15.154:43346) 2023-09-23T03:24:42.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.476+0000 7f8f97c82640 1 -- 172.21.15.154:0/4141729894 learned_addr learned my addr 172.21.15.154:0/4141729894 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:42.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.476+0000 7f8f97c82640 1 -- 172.21.15.154:0/4141729894 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5598b4e0fc00 msgr2=0x5598b4cfeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:42.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.476+0000 7f8f97c82640 1 --2- 172.21.15.154:0/4141729894 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5598b4e0fc00 0x5598b4cfeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.476+0000 7f8f97c82640 1 -- 172.21.15.154:0/4141729894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598b4fde400 msgr2=0x5598b4d00100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:42.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.476+0000 7f8f97c82640 1 --2- 172.21.15.154:0/4141729894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598b4fde400 0x5598b4d00100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.476+0000 7f8f97c82640 1 -- 172.21.15.154:0/4141729894 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598b4cc8340 con 0x5598b4fde000 2023-09-23T03:24:42.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.476+0000 7f8f97c82640 1 --2- 172.21.15.154:0/4141729894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598b4fde000 0x5598b4cfe580 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x5598b4f82870 tx=0x5598b4f92cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e6be9a563c397f9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.477+0000 7f8f93479640 1 -- 172.21.15.154:0/4141729894 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5598b4cdd080 con 0x5598b4fde000 2023-09-23T03:24:42.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.477+0000 7f8f93479640 1 -- 172.21.15.154:0/4141729894 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5598b5a64300 con 0x5598b4fde000 2023-09-23T03:24:42.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.477+0000 7f8f93479640 1 -- 172.21.15.154:0/4141729894 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5598b4cdd600 con 0x5598b4fde000 2023-09-23T03:24:42.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.477+0000 7f8f994da5c0 1 -- 172.21.15.154:0/4141729894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598b4fde000 msgr2=0x5598b4cfe580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.477+0000 7f8f994da5c0 1 --2- 172.21.15.154:0/4141729894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598b4fde000 0x5598b4cfe580 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x5598b4f82870 tx=0x5598b4f92cc0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.477+0000 7f8f994da5c0 1 -- 172.21.15.154:0/4141729894 shutdown_connections 2023-09-23T03:24:42.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.477+0000 7f8f994da5c0 1 --2- 172.21.15.154:0/4141729894 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598b4fde400 0x5598b4d00100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.477+0000 7f8f994da5c0 1 --2- 172.21.15.154:0/4141729894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598b4fde000 0x5598b4cfe580 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.477+0000 7f8f994da5c0 1 --2- 172.21.15.154:0/4141729894 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5598b4e0fc00 0x5598b4cfeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.477+0000 7f8f994da5c0 1 -- 172.21.15.154:0/4141729894 >> 172.21.15.154:0/4141729894 conn(0x5598b4e0f800 msgr2=0x5598b5a0f000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:42.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.477+0000 7f8f994da5c0 1 -- 172.21.15.154:0/4141729894 shutdown_connections 2023-09-23T03:24:42.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.477+0000 7f8f994da5c0 1 -- 172.21.15.154:0/4141729894 wait complete. 2023-09-23T03:24:42.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.478+0000 7f8f994da5c0 1 Processor -- start 2023-09-23T03:24:42.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.478+0000 7f8f994da5c0 1 -- start start 2023-09-23T03:24:42.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.479+0000 7f8f994da5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598b4e0f800 0x5598b4cfeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.479+0000 7f8f994da5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598b4fdf000 0x5598b4cfe580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.479+0000 7f8f994da5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5598b4fdec00 0x5598b4d00100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.479+0000 7f8f994da5c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598b4cdf180 con 0x5598b4e0f800 2023-09-23T03:24:42.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.479+0000 7f8f994da5c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598b4cde000 con 0x5598b4fdf000 2023-09-23T03:24:42.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.479+0000 7f8f994da5c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598b4cdea00 con 0x5598b4fdec00 2023-09-23T03:24:42.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.479+0000 7f8f97481640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598b4e0f800 0x5598b4cfeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.479+0000 7f8f97481640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598b4e0f800 0x5598b4cfeb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43352/0 (socket says 172.21.15.154:43352) 2023-09-23T03:24:42.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.479+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 learned_addr learned my addr 172.21.15.154:0/1527791939 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:42.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.479+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5598b4fdec00 msgr2=0x5598b4d00100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.479+0000 7f8f97481640 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5598b4fdec00 0x5598b4d00100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.479+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598b4fdf000 msgr2=0x5598b4cfe580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.479+0000 7f8f97481640 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598b4fdf000 0x5598b4cfe580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.479+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598b4cc8000 con 0x5598b4e0f800 2023-09-23T03:24:42.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.480+0000 7f8f97481640 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598b4e0f800 0x5598b4cfeb00 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x5598b4f83590 tx=0x5598b4f92ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.480+0000 7f8f91c76640 1 -- 172.21.15.154:0/1527791939 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5598b4cdd080 con 0x5598b4e0f800 2023-09-23T03:24:42.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.480+0000 7f8f91c76640 1 -- 172.21.15.154:0/1527791939 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5598b4cbb800 con 0x5598b4e0f800 2023-09-23T03:24:42.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.480+0000 7f8f994da5c0 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5598b4cc8ea0 con 0x5598b4e0f800 2023-09-23T03:24:42.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.480+0000 7f8f91c76640 1 -- 172.21.15.154:0/1527791939 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5598b4cdd600 con 0x5598b4e0f800 2023-09-23T03:24:42.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.480+0000 7f8f994da5c0 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598b4cc9520 con 0x5598b4e0f800 2023-09-23T03:24:42.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.480+0000 7f8f994da5c0 4 datascan.init: resolving metadata pool 11 2023-09-23T03:24:42.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.481+0000 7f8f91c76640 1 -- 172.21.15.154:0/1527791939 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x5598b4cc6840 con 0x5598b4e0f800 2023-09-23T03:24:42.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.482+0000 7f8f91c76640 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5598b5004c00 0x5598b50f2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.482+0000 7f8f91c76640 1 -- 172.21.15.154:0/1527791939 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6605+0+0 (secure 0 0 0) 0x5598b4d54a80 con 0x5598b4e0f800 2023-09-23T03:24:42.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.482+0000 7f8f994da5c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-09-23T03:24:42.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.482+0000 7f8f994da5c0 4 datascan.main: data pool 'cephfs_data' has ID 12 2023-09-23T03:24:42.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.482+0000 7f8f994da5c0 4 datascan.main: opening data pool 'cephfs_data' 2023-09-23T03:24:42.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.482+0000 7f8f994da5c0 4 datascan.main: data pool 'cephfs_data_extra' has ID 13 2023-09-23T03:24:42.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.482+0000 7f8f994da5c0 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-09-23T03:24:42.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.482+0000 7f8f994da5c0 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5598b4fde800 0x5598b4d00680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.482+0000 7f8f97c82640 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5598b5004c00 0x5598b50f2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.482+0000 7f8f994da5c0 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:1 12.0 12:00000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b4cef880 con 0x5598b4fde800 2023-09-23T03:24:42.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.483+0000 7f8f97c82640 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5598b5004c00 0x5598b50f2000 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x5598b4f82870 tx=0x5598b4f92c00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.483+0000 7f8f97481640 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5598b4fde800 0x5598b4d00680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.484+0000 7f8f97481640 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5598b4fde800 0x5598b4d00680 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.485+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 <== osd.4 v2:172.21.15.190:6808/717302641 1 ==== osd_op_reply(1 [pgnls start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eab40 con 0x5598b4fde800 2023-09-23T03:24:42.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.485+0000 7f8f92c78640 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x5598b5005000 0x5598b50f2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.485+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:2 12.20 12:04000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5066000 con 0x5598b5005000 2023-09-23T03:24:42.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.485+0000 7f8f98483640 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x5598b5005000 0x5598b50f2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.486+0000 7f8f98483640 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x5598b5005000 0x5598b50f2580 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.487+0000 7f8f98483640 1 -- 172.21.15.154:0/1527791939 <== osd.7 v2:172.21.15.190:6824/3460136272 1 ==== osd_op_reply(2 [pgnls start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59ead80 con 0x5598b5005000 2023-09-23T03:24:42.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.487+0000 7f8f93479640 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5598b5005400 0x5598b50f2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.487+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:3 12.10 12:08000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5066380 con 0x5598b5005400 2023-09-23T03:24:42.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.487+0000 7f8f97c82640 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5598b5005400 0x5598b50f2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.488+0000 7f8f97c82640 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5598b5005400 0x5598b50f2b00 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.488+0000 7f8f97c82640 1 -- 172.21.15.154:0/1527791939 <== osd.0 v2:172.21.15.154:6800/2920938138 1 ==== osd_op_reply(3 [pgnls start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eafc0 con 0x5598b5005400 2023-09-23T03:24:42.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.488+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:4 12.30 12:0c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5066700 con 0x5598b5005000 2023-09-23T03:24:42.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.489+0000 7f8f98483640 1 -- 172.21.15.154:0/1527791939 <== osd.7 v2:172.21.15.190:6824/3460136272 2 ==== osd_op_reply(4 [pgnls start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59ead80 con 0x5598b5005000 2023-09-23T03:24:42.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.489+0000 7f8f93479640 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5598b5005800 0x5598b50f3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.489+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:5 12.8 12:10000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5066e00 con 0x5598b5005800 2023-09-23T03:24:42.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.489+0000 7f8f97481640 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5598b5005800 0x5598b50f3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.490+0000 7f8f97481640 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5598b5005800 0x5598b50f3080 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.490+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(5 [pgnls start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eab40 con 0x5598b5005800 2023-09-23T03:24:42.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.490+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:6 12.28 12:14000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5066a80 con 0x5598b5005400 2023-09-23T03:24:42.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.490+0000 7f8f97c82640 1 -- 172.21.15.154:0/1527791939 <== osd.0 v2:172.21.15.154:6800/2920938138 2 ==== osd_op_reply(6 [pgnls start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eafc0 con 0x5598b5005400 2023-09-23T03:24:42.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.490+0000 7f8f93479640 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5598b5005c00 0x5598b50f3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.491+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:7 12.18 12:18000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5067180 con 0x5598b5005c00 2023-09-23T03:24:42.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.491+0000 7f8f98483640 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5598b5005c00 0x5598b50f3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.491+0000 7f8f98483640 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5598b5005c00 0x5598b50f3600 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.491+0000 7f8f98483640 1 -- 172.21.15.154:0/1527791939 <== osd.1 v2:172.21.15.154:6816/1267710005 1 ==== osd_op_reply(7 [pgnls start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59ead80 con 0x5598b5005c00 2023-09-23T03:24:42.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.491+0000 7f8f92c78640 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5598b511a000 0x5598b50f3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.492+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:8 12.38 12:1c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b4cef880 con 0x5598b511a000 2023-09-23T03:24:42.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.492+0000 7f8f97c82640 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5598b511a000 0x5598b50f3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.492+0000 7f8f97c82640 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5598b511a000 0x5598b50f3b80 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.492+0000 7f8f97c82640 1 -- 172.21.15.154:0/1527791939 <== osd.2 v2:172.21.15.154:6822/2884407243 1 ==== osd_op_reply(8 [pgnls start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eafc0 con 0x5598b511a000 2023-09-23T03:24:42.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.492+0000 7f8f93479640 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x5598b511a800 0x5598b50f4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.493+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:9 12.4 12:20000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5124000 con 0x5598b511a800 2023-09-23T03:24:42.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.493+0000 7f8f97481640 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x5598b511a800 0x5598b50f4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.494+0000 7f8f97481640 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x5598b511a800 0x5598b50f4680 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.495+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== osd_op_reply(9 [pgnls start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eab40 con 0x5598b511a800 2023-09-23T03:24:42.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.495+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:10 12.24 12:24000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5124a80 con 0x5598b511a000 2023-09-23T03:24:42.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.495+0000 7f8f97c82640 1 -- 172.21.15.154:0/1527791939 <== osd.2 v2:172.21.15.154:6822/2884407243 2 ==== osd_op_reply(10 [pgnls start_epoch 88 out=49b] v86'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eafc0 con 0x5598b511a000 2023-09-23T03:24:42.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.495+0000 7f8f93479640 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x5598b511ac00 0x5598b50f4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.495+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:11 12.14 12:28000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5124380 con 0x5598b511ac00 2023-09-23T03:24:42.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.496+0000 7f8f98483640 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x5598b511ac00 0x5598b50f4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.497+0000 7f8f98483640 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x5598b511ac00 0x5598b50f4c00 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.497+0000 7f8f98483640 1 -- 172.21.15.154:0/1527791939 <== osd.5 v2:172.21.15.190:6800/882536885 1 ==== osd_op_reply(11 [pgnls start_epoch 88 out=49b] v86'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59ead80 con 0x5598b511ac00 2023-09-23T03:24:42.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.497+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:12 12.34 12:2c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5124e00 con 0x5598b511ac00 2023-09-23T03:24:42.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.498+0000 7f8f98483640 1 -- 172.21.15.154:0/1527791939 <== osd.5 v2:172.21.15.190:6800/882536885 2 ==== osd_op_reply(12 [pgnls start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59ead80 con 0x5598b511ac00 2023-09-23T03:24:42.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.498+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:13 12.c 12:30000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5124700 con 0x5598b5005400 2023-09-23T03:24:42.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.498+0000 7f8f97c82640 1 -- 172.21.15.154:0/1527791939 <== osd.0 v2:172.21.15.154:6800/2920938138 3 ==== osd_op_reply(13 [pgnls start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eafc0 con 0x5598b5005400 2023-09-23T03:24:42.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.498+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:14 12.2c 12:34000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5125180 con 0x5598b511a000 2023-09-23T03:24:42.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.499+0000 7f8f97c82640 1 -- 172.21.15.154:0/1527791939 <== osd.2 v2:172.21.15.154:6822/2884407243 3 ==== osd_op_reply(14 [pgnls start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eafc0 con 0x5598b511a000 2023-09-23T03:24:42.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.499+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:15 12.1c 12:38000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5124a80 con 0x5598b5005400 2023-09-23T03:24:42.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.499+0000 7f8f97c82640 1 -- 172.21.15.154:0/1527791939 <== osd.0 v2:172.21.15.154:6800/2920938138 4 ==== osd_op_reply(15 [pgnls start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eafc0 con 0x5598b5005400 2023-09-23T03:24:42.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.499+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:16 12.3c 12:3c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5125500 con 0x5598b511ac00 2023-09-23T03:24:42.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.500+0000 7f8f98483640 1 -- 172.21.15.154:0/1527791939 <== osd.5 v2:172.21.15.190:6800/882536885 3 ==== osd_op_reply(16 [pgnls start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59ead80 con 0x5598b511ac00 2023-09-23T03:24:42.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.500+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:17 12.2 12:40000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5124700 con 0x5598b4fde800 2023-09-23T03:24:42.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.501+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 <== osd.4 v2:172.21.15.190:6808/717302641 2 ==== osd_op_reply(17 [pgnls start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eab40 con 0x5598b4fde800 2023-09-23T03:24:42.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.501+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:18 12.22 12:44000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5125c00 con 0x5598b5005000 2023-09-23T03:24:42.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.502+0000 7f8f98483640 1 -- 172.21.15.154:0/1527791939 <== osd.7 v2:172.21.15.190:6824/3460136272 3 ==== osd_op_reply(18 [pgnls start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59ead80 con 0x5598b5005000 2023-09-23T03:24:42.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.502+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:19 12.12 12:48000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5125880 con 0x5598b5005800 2023-09-23T03:24:42.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.503+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 <== osd.3 v2:172.21.15.154:6808/1133688468 2 ==== osd_op_reply(19 [pgnls start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eab40 con 0x5598b5005800 2023-09-23T03:24:42.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.503+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:20 12.32 12:4c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5124000 con 0x5598b5005800 2023-09-23T03:24:42.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.503+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 <== osd.3 v2:172.21.15.154:6808/1133688468 3 ==== osd_op_reply(20 [pgnls start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eab40 con 0x5598b5005800 2023-09-23T03:24:42.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.503+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:21 12.a 12:50000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b4cef880 con 0x5598b511a000 2023-09-23T03:24:42.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.504+0000 7f8f97c82640 1 -- 172.21.15.154:0/1527791939 <== osd.2 v2:172.21.15.154:6822/2884407243 4 ==== osd_op_reply(21 [pgnls start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eafc0 con 0x5598b511a000 2023-09-23T03:24:42.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.504+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:22 12.2a 12:54000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5124700 con 0x5598b511a000 2023-09-23T03:24:42.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.504+0000 7f8f97c82640 1 -- 172.21.15.154:0/1527791939 <== osd.2 v2:172.21.15.154:6822/2884407243 5 ==== osd_op_reply(22 [pgnls start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eafc0 con 0x5598b511a000 2023-09-23T03:24:42.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.504+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:23 12.1a 12:58000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5125180 con 0x5598b4fde800 2023-09-23T03:24:42.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.505+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 <== osd.4 v2:172.21.15.190:6808/717302641 3 ==== osd_op_reply(23 [pgnls start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eab40 con 0x5598b4fde800 2023-09-23T03:24:42.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.505+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:24 12.3a 12:5c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5125880 con 0x5598b5005800 2023-09-23T03:24:42.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.505+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 <== osd.3 v2:172.21.15.154:6808/1133688468 4 ==== osd_op_reply(24 [pgnls start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eab40 con 0x5598b5005800 2023-09-23T03:24:42.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.505+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:25 12.6 12:60000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5124a80 con 0x5598b5005c00 2023-09-23T03:24:42.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.506+0000 7f8f98483640 1 -- 172.21.15.154:0/1527791939 <== osd.1 v2:172.21.15.154:6816/1267710005 2 ==== osd_op_reply(25 [pgnls start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59ead80 con 0x5598b5005c00 2023-09-23T03:24:42.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.506+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:26 12.26 12:64000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5124380 con 0x5598b511ac00 2023-09-23T03:24:42.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.507+0000 7f8f98483640 1 -- 172.21.15.154:0/1527791939 <== osd.5 v2:172.21.15.190:6800/882536885 4 ==== osd_op_reply(26 [pgnls start_epoch 88 out=49b] v86'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59ead80 con 0x5598b511ac00 2023-09-23T03:24:42.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.507+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:27 12.16 12:68000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5124700 con 0x5598b511a800 2023-09-23T03:24:42.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.508+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 <== osd.6 v2:172.21.15.190:6816/1964294001 2 ==== osd_op_reply(27 [pgnls start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eab40 con 0x5598b511a800 2023-09-23T03:24:42.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.508+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:28 12.36 12:6c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5124e00 con 0x5598b511ac00 2023-09-23T03:24:42.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.509+0000 7f8f98483640 1 -- 172.21.15.154:0/1527791939 <== osd.5 v2:172.21.15.190:6800/882536885 5 ==== osd_op_reply(28 [pgnls start_epoch 88 out=49b] v86'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59ead80 con 0x5598b511ac00 2023-09-23T03:24:42.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.509+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:29 12.e 12:70000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5125c00 con 0x5598b5005400 2023-09-23T03:24:42.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.509+0000 7f8f97c82640 1 -- 172.21.15.154:0/1527791939 <== osd.0 v2:172.21.15.154:6800/2920938138 5 ==== osd_op_reply(29 [pgnls start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eafc0 con 0x5598b5005400 2023-09-23T03:24:42.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.509+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:30 12.2e 12:74000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5125500 con 0x5598b5005c00 2023-09-23T03:24:42.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.510+0000 7f8f98483640 1 -- 172.21.15.154:0/1527791939 <== osd.1 v2:172.21.15.154:6816/1267710005 3 ==== osd_op_reply(30 [pgnls start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59ead80 con 0x5598b5005c00 2023-09-23T03:24:42.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.510+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:31 12.1e 12:78000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b4cef880 con 0x5598b4fde800 2023-09-23T03:24:42.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.510+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 <== osd.4 v2:172.21.15.190:6808/717302641 4 ==== osd_op_reply(31 [pgnls start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eab40 con 0x5598b4fde800 2023-09-23T03:24:42.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.511+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:32 12.3e 12:7c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5124000 con 0x5598b5005400 2023-09-23T03:24:42.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.511+0000 7f8f97c82640 1 -- 172.21.15.154:0/1527791939 <== osd.0 v2:172.21.15.154:6800/2920938138 6 ==== osd_op_reply(32 [pgnls start_epoch 88 out=49b] v86'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eafc0 con 0x5598b5005400 2023-09-23T03:24:42.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.511+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:33 12.1 12:80000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5134000 con 0x5598b511a800 2023-09-23T03:24:42.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.512+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 <== osd.6 v2:172.21.15.190:6816/1964294001 3 ==== osd_op_reply(33 [pgnls start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eab40 con 0x5598b511a800 2023-09-23T03:24:42.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.512+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:34 12.21 12:84000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5125180 con 0x5598b5005c00 2023-09-23T03:24:42.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.512+0000 7f8f98483640 1 -- 172.21.15.154:0/1527791939 <== osd.1 v2:172.21.15.154:6816/1267710005 4 ==== osd_op_reply(34 [pgnls start_epoch 88 out=49b] v86'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59ead80 con 0x5598b5005c00 2023-09-23T03:24:42.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.512+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:35 12.11 12:88000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5134380 con 0x5598b511ac00 2023-09-23T03:24:42.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.514+0000 7f8f98483640 1 -- 172.21.15.154:0/1527791939 <== osd.5 v2:172.21.15.190:6800/882536885 6 ==== osd_op_reply(35 [pgnls start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59ead80 con 0x5598b511ac00 2023-09-23T03:24:42.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.514+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:36 12.31 12:8c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5125880 con 0x5598b5005400 2023-09-23T03:24:42.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.514+0000 7f8f97c82640 1 -- 172.21.15.154:0/1527791939 <== osd.0 v2:172.21.15.154:6800/2920938138 7 ==== osd_op_reply(36 [pgnls start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eafc0 con 0x5598b5005400 2023-09-23T03:24:42.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.514+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:37 12.9 12:90000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5134700 con 0x5598b511a000 2023-09-23T03:24:42.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.514+0000 7f8f97c82640 1 -- 172.21.15.154:0/1527791939 <== osd.2 v2:172.21.15.154:6822/2884407243 6 ==== osd_op_reply(37 [pgnls start_epoch 88 out=49b] v86'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eafc0 con 0x5598b511a000 2023-09-23T03:24:42.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.515+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:38 12.29 12:94000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5124700 con 0x5598b4fde800 2023-09-23T03:24:42.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.515+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 <== osd.4 v2:172.21.15.190:6808/717302641 5 ==== osd_op_reply(38 [pgnls start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eab40 con 0x5598b4fde800 2023-09-23T03:24:42.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.516+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:39 12.19 12:98000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5134a80 con 0x5598b5005400 2023-09-23T03:24:42.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.516+0000 7f8f97c82640 1 -- 172.21.15.154:0/1527791939 <== osd.0 v2:172.21.15.154:6800/2920938138 8 ==== osd_op_reply(39 [pgnls start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eafc0 con 0x5598b5005400 2023-09-23T03:24:42.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.516+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:40 12.39 12:9c000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5125c00 con 0x5598b511a800 2023-09-23T03:24:42.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.517+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 <== osd.6 v2:172.21.15.190:6816/1964294001 4 ==== osd_op_reply(40 [pgnls start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eab40 con 0x5598b511a800 2023-09-23T03:24:42.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.517+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:41 12.5 12:a0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5134e00 con 0x5598b511a800 2023-09-23T03:24:42.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.518+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 <== osd.6 v2:172.21.15.190:6816/1964294001 5 ==== osd_op_reply(41 [pgnls start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eab40 con 0x5598b511a800 2023-09-23T03:24:42.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.519+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:42 12.25 12:a4000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5124000 con 0x5598b5005000 2023-09-23T03:24:42.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.520+0000 7f8f98483640 1 -- 172.21.15.154:0/1527791939 <== osd.7 v2:172.21.15.190:6824/3460136272 4 ==== osd_op_reply(42 [pgnls start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59ead80 con 0x5598b5005000 2023-09-23T03:24:42.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.520+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:43 12.15 12:a8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5135180 con 0x5598b511a800 2023-09-23T03:24:42.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.521+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 <== osd.6 v2:172.21.15.190:6816/1964294001 6 ==== osd_op_reply(43 [pgnls start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eab40 con 0x5598b511a800 2023-09-23T03:24:42.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.521+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:44 12.35 12:ac000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5125880 con 0x5598b5005400 2023-09-23T03:24:42.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.521+0000 7f8f97c82640 1 -- 172.21.15.154:0/1527791939 <== osd.0 v2:172.21.15.154:6800/2920938138 9 ==== osd_op_reply(44 [pgnls start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eafc0 con 0x5598b5005400 2023-09-23T03:24:42.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.521+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:45 12.d 12:b0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5134000 con 0x5598b5005000 2023-09-23T03:24:42.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.522+0000 7f8f98483640 1 -- 172.21.15.154:0/1527791939 <== osd.7 v2:172.21.15.190:6824/3460136272 5 ==== osd_op_reply(45 [pgnls start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59ead80 con 0x5598b5005000 2023-09-23T03:24:42.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.522+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:46 12.2d 12:b4000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5124a80 con 0x5598b5005800 2023-09-23T03:24:42.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.523+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 <== osd.3 v2:172.21.15.154:6808/1133688468 5 ==== osd_op_reply(46 [pgnls start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eab40 con 0x5598b5005800 2023-09-23T03:24:42.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.523+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:47 12.1d 12:b8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5134e00 con 0x5598b5005400 2023-09-23T03:24:42.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.524+0000 7f8f97c82640 1 -- 172.21.15.154:0/1527791939 <== osd.0 v2:172.21.15.154:6800/2920938138 10 ==== osd_op_reply(47 [pgnls start_epoch 88 out=49b] v86'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eafc0 con 0x5598b5005400 2023-09-23T03:24:42.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.524+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:48 12.3d 12:bc000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5124380 con 0x5598b5005c00 2023-09-23T03:24:42.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.524+0000 7f8f98483640 1 -- 172.21.15.154:0/1527791939 <== osd.1 v2:172.21.15.154:6816/1267710005 5 ==== osd_op_reply(48 [pgnls start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59ead80 con 0x5598b5005c00 2023-09-23T03:24:42.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.524+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:49 12.3 12:c0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5134700 con 0x5598b5005400 2023-09-23T03:24:42.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.525+0000 7f8f97c82640 1 -- 172.21.15.154:0/1527791939 <== osd.0 v2:172.21.15.154:6800/2920938138 11 ==== osd_op_reply(49 [pgnls start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eafc0 con 0x5598b5005400 2023-09-23T03:24:42.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.525+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:50 12.23 12:c4000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5124e00 con 0x5598b5005400 2023-09-23T03:24:42.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.525+0000 7f8f97c82640 1 -- 172.21.15.154:0/1527791939 <== osd.0 v2:172.21.15.154:6800/2920938138 12 ==== osd_op_reply(50 [pgnls start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eafc0 con 0x5598b5005400 2023-09-23T03:24:42.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.525+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:51 12.13 12:c8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5134a80 con 0x5598b5005c00 2023-09-23T03:24:42.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.526+0000 7f8f98483640 1 -- 172.21.15.154:0/1527791939 <== osd.1 v2:172.21.15.154:6816/1267710005 6 ==== osd_op_reply(51 [pgnls start_epoch 88 out=49b] v86'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59ead80 con 0x5598b5005c00 2023-09-23T03:24:42.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.526+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:52 12.33 12:cc000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5125500 con 0x5598b511a000 2023-09-23T03:24:42.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.526+0000 7f8f97c82640 1 -- 172.21.15.154:0/1527791939 <== osd.2 v2:172.21.15.154:6822/2884407243 7 ==== osd_op_reply(52 [pgnls start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eafc0 con 0x5598b511a000 2023-09-23T03:24:42.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.526+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:53 12.b 12:d0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5124700 con 0x5598b5005800 2023-09-23T03:24:42.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.527+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 <== osd.3 v2:172.21.15.154:6808/1133688468 6 ==== osd_op_reply(53 [pgnls start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eab40 con 0x5598b5005800 2023-09-23T03:24:42.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.527+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:54 12.2b 12:d4000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5125180 con 0x5598b4fde800 2023-09-23T03:24:42.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.528+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 <== osd.4 v2:172.21.15.190:6808/717302641 6 ==== osd_op_reply(54 [pgnls start_epoch 88 out=49b] v86'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eab40 con 0x5598b4fde800 2023-09-23T03:24:42.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.528+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:55 12.1b 12:d8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5125c00 con 0x5598b4fde800 2023-09-23T03:24:42.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.529+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 <== osd.4 v2:172.21.15.190:6808/717302641 7 ==== osd_op_reply(55 [pgnls start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eab40 con 0x5598b4fde800 2023-09-23T03:24:42.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.529+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:56 12.3b 12:dc000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5135500 con 0x5598b511a000 2023-09-23T03:24:42.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.530+0000 7f8f97c82640 1 -- 172.21.15.154:0/1527791939 <== osd.2 v2:172.21.15.154:6822/2884407243 8 ==== osd_op_reply(56 [pgnls start_epoch 88 out=49b] v86'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eafc0 con 0x5598b511a000 2023-09-23T03:24:42.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.530+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:57 12.7 12:e0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b4cef880 con 0x5598b5005c00 2023-09-23T03:24:42.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.531+0000 7f8f98483640 1 -- 172.21.15.154:0/1527791939 <== osd.1 v2:172.21.15.154:6816/1267710005 7 ==== osd_op_reply(57 [pgnls start_epoch 88 out=81b] v86'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5598b59ead80 con 0x5598b5005c00 2023-09-23T03:24:42.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.531+0000 7f8f994da5c0 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:58 12.7 12:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b4cefc00 con 0x5598b5005c00 2023-09-23T03:24:42.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.531+0000 7f8f98483640 1 -- 172.21.15.154:0/1527791939 <== osd.1 v2:172.21.15.154:6816/1267710005 8 ==== osd_op_reply(58 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5598b59ead80 con 0x5598b5005c00 2023-09-23T03:24:42.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.531+0000 7f8f994da5c0 10 datascan.operator(): handling object 1099511627777.0 2023-09-23T03:24:42.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.532+0000 7f8f994da5c0 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:59 12.7 12:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5124000 con 0x5598b5005c00 2023-09-23T03:24:42.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.535+0000 7f8f98483640 1 -- 172.21.15.154:0/1527791939 <== osd.1 v2:172.21.15.154:6816/1267710005 9 ==== osd_op_reply(59 10000000001.00000000 [call] v88'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5598b59ead80 con 0x5598b5005c00 2023-09-23T03:24:42.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.535+0000 7f8f994da5c0 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:60 12.27 12:e4000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5124380 con 0x5598b5005400 2023-09-23T03:24:42.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.536+0000 7f8f97c82640 1 -- 172.21.15.154:0/1527791939 <== osd.0 v2:172.21.15.154:6800/2920938138 13 ==== osd_op_reply(60 [pgnls start_epoch 88 out=49b] v86'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eafc0 con 0x5598b5005400 2023-09-23T03:24:42.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.536+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:61 12.17 12:e8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5134380 con 0x5598b511a800 2023-09-23T03:24:42.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.537+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 <== osd.6 v2:172.21.15.190:6816/1964294001 7 ==== osd_op_reply(61 [pgnls start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eab40 con 0x5598b511a800 2023-09-23T03:24:42.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.537+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:62 12.37 12:ec000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5135880 con 0x5598b4fde800 2023-09-23T03:24:42.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.538+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 <== osd.4 v2:172.21.15.190:6808/717302641 8 ==== osd_op_reply(62 [pgnls start_epoch 88 out=49b] v86'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eab40 con 0x5598b4fde800 2023-09-23T03:24:42.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.538+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:63 12.f 12:f0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5134000 con 0x5598b511a800 2023-09-23T03:24:42.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.539+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 <== osd.6 v2:172.21.15.190:6816/1964294001 8 ==== osd_op_reply(63 [pgnls start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eab40 con 0x5598b511a800 2023-09-23T03:24:42.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.539+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:64 12.2f 12:f4000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5135c00 con 0x5598b511a000 2023-09-23T03:24:42.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.540+0000 7f8f97c82640 1 -- 172.21.15.154:0/1527791939 <== osd.2 v2:172.21.15.154:6822/2884407243 9 ==== osd_op_reply(64 [pgnls start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eafc0 con 0x5598b511a000 2023-09-23T03:24:42.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.540+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:65 12.1f 12:f8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5134a80 con 0x5598b511a800 2023-09-23T03:24:42.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.541+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 <== osd.6 v2:172.21.15.190:6816/1964294001 9 ==== osd_op_reply(65 [pgnls start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eab40 con 0x5598b511a800 2023-09-23T03:24:42.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.541+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:66 12.3f 12:fc000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5135180 con 0x5598b5005000 2023-09-23T03:24:42.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.542+0000 7f8f98483640 1 -- 172.21.15.154:0/1527791939 <== osd.7 v2:172.21.15.190:6824/3460136272 6 ==== osd_op_reply(66 [pgnls start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59ead80 con 0x5598b5005000 2023-09-23T03:24:42.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.543+0000 7f8f994da5c0 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:67 13.0 13:00000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5125880 con 0x5598b5005000 2023-09-23T03:24:42.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.544+0000 7f8f98483640 1 -- 172.21.15.154:0/1527791939 <== osd.7 v2:172.21.15.190:6824/3460136272 7 ==== osd_op_reply(67 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59ead80 con 0x5598b5005000 2023-09-23T03:24:42.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.544+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:68 13.10 13:08000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5134380 con 0x5598b5005c00 2023-09-23T03:24:42.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.545+0000 7f8f98483640 1 -- 172.21.15.154:0/1527791939 <== osd.1 v2:172.21.15.154:6816/1267710005 10 ==== osd_op_reply(68 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59ead80 con 0x5598b5005c00 2023-09-23T03:24:42.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.545+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:69 13.8 13:10000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5134e00 con 0x5598b511a000 2023-09-23T03:24:42.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.545+0000 7f8f97c82640 1 -- 172.21.15.154:0/1527791939 <== osd.2 v2:172.21.15.154:6822/2884407243 10 ==== osd_op_reply(69 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eafc0 con 0x5598b511a000 2023-09-23T03:24:42.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.545+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:70 13.18 13:18000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5124a80 con 0x5598b5005000 2023-09-23T03:24:42.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.547+0000 7f8f98483640 1 -- 172.21.15.154:0/1527791939 <== osd.7 v2:172.21.15.190:6824/3460136272 8 ==== osd_op_reply(70 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59ead80 con 0x5598b5005000 2023-09-23T03:24:42.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.547+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:71 13.4 13:20000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5134700 con 0x5598b511a800 2023-09-23T03:24:42.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.548+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 <== osd.6 v2:172.21.15.190:6816/1964294001 10 ==== osd_op_reply(71 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eab40 con 0x5598b511a800 2023-09-23T03:24:42.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.548+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:72 13.14 13:28000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5124700 con 0x5598b5005400 2023-09-23T03:24:42.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.549+0000 7f8f97c82640 1 -- 172.21.15.154:0/1527791939 <== osd.0 v2:172.21.15.154:6800/2920938138 14 ==== osd_op_reply(72 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eafc0 con 0x5598b5005400 2023-09-23T03:24:42.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.549+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:73 13.c 13:30000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5135500 con 0x5598b5005c00 2023-09-23T03:24:42.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.549+0000 7f8f98483640 1 -- 172.21.15.154:0/1527791939 <== osd.1 v2:172.21.15.154:6816/1267710005 11 ==== osd_op_reply(73 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59ead80 con 0x5598b5005c00 2023-09-23T03:24:42.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.549+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:74 13.1c 13:38000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5125180 con 0x5598b511a800 2023-09-23T03:24:42.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.551+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 <== osd.6 v2:172.21.15.190:6816/1964294001 11 ==== osd_op_reply(74 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eab40 con 0x5598b511a800 2023-09-23T03:24:42.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.551+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:75 13.2 13:40000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5124e00 con 0x5598b511ac00 2023-09-23T03:24:42.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.552+0000 7f8f98483640 1 -- 172.21.15.154:0/1527791939 <== osd.5 v2:172.21.15.190:6800/882536885 7 ==== osd_op_reply(75 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59ead80 con 0x5598b511ac00 2023-09-23T03:24:42.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.552+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:76 13.12 13:48000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5125c00 con 0x5598b5005400 2023-09-23T03:24:42.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.553+0000 7f8f97c82640 1 -- 172.21.15.154:0/1527791939 <== osd.0 v2:172.21.15.154:6800/2920938138 15 ==== osd_op_reply(76 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eafc0 con 0x5598b5005400 2023-09-23T03:24:42.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.553+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:77 13.a 13:50000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5125500 con 0x5598b511a000 2023-09-23T03:24:42.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.553+0000 7f8f97c82640 1 -- 172.21.15.154:0/1527791939 <== osd.2 v2:172.21.15.154:6822/2884407243 11 ==== osd_op_reply(77 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eafc0 con 0x5598b511a000 2023-09-23T03:24:42.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.553+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:78 13.1a 13:58000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5135180 con 0x5598b5005c00 2023-09-23T03:24:42.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.554+0000 7f8f98483640 1 -- 172.21.15.154:0/1527791939 <== osd.1 v2:172.21.15.154:6816/1267710005 12 ==== osd_op_reply(78 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59ead80 con 0x5598b5005c00 2023-09-23T03:24:42.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.554+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:79 13.6 13:60000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5142000 con 0x5598b4fde800 2023-09-23T03:24:42.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.555+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 <== osd.4 v2:172.21.15.190:6808/717302641 9 ==== osd_op_reply(79 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eab40 con 0x5598b4fde800 2023-09-23T03:24:42.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.556+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:80 13.16 13:68000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5134380 con 0x5598b5005800 2023-09-23T03:24:42.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.556+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 <== osd.3 v2:172.21.15.154:6808/1133688468 7 ==== osd_op_reply(80 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eab40 con 0x5598b5005800 2023-09-23T03:24:42.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.556+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:81 13.e 13:70000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5142380 con 0x5598b5005000 2023-09-23T03:24:42.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.558+0000 7f8f98483640 1 -- 172.21.15.154:0/1527791939 <== osd.7 v2:172.21.15.190:6824/3460136272 9 ==== osd_op_reply(81 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59ead80 con 0x5598b5005000 2023-09-23T03:24:42.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.558+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:82 13.1e 13:78000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5135500 con 0x5598b4fde800 2023-09-23T03:24:42.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.559+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 <== osd.4 v2:172.21.15.190:6808/717302641 10 ==== osd_op_reply(82 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eab40 con 0x5598b4fde800 2023-09-23T03:24:42.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.559+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:83 13.1 13:80000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5142700 con 0x5598b511a800 2023-09-23T03:24:42.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.560+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 <== osd.6 v2:172.21.15.190:6816/1964294001 12 ==== osd_op_reply(83 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eab40 con 0x5598b511a800 2023-09-23T03:24:42.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.560+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:84 13.11 13:88000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5124000 con 0x5598b5005c00 2023-09-23T03:24:42.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.561+0000 7f8f98483640 1 -- 172.21.15.154:0/1527791939 <== osd.1 v2:172.21.15.154:6816/1267710005 13 ==== osd_op_reply(84 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59ead80 con 0x5598b5005c00 2023-09-23T03:24:42.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.561+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:85 13.9 13:90000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5142a80 con 0x5598b511ac00 2023-09-23T03:24:42.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.562+0000 7f8f98483640 1 -- 172.21.15.154:0/1527791939 <== osd.5 v2:172.21.15.190:6800/882536885 8 ==== osd_op_reply(85 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59ead80 con 0x5598b511ac00 2023-09-23T03:24:42.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.562+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:86 13.19 13:98000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5125880 con 0x5598b5005800 2023-09-23T03:24:42.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.563+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 <== osd.3 v2:172.21.15.154:6808/1133688468 8 ==== osd_op_reply(86 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eab40 con 0x5598b5005800 2023-09-23T03:24:42.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.563+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:87 13.5 13:a0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5142e00 con 0x5598b4fde800 2023-09-23T03:24:42.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.564+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 <== osd.4 v2:172.21.15.190:6808/717302641 11 ==== osd_op_reply(87 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eab40 con 0x5598b4fde800 2023-09-23T03:24:42.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.564+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:88 13.15 13:a8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5124a80 con 0x5598b4fde800 2023-09-23T03:24:42.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.565+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 <== osd.4 v2:172.21.15.190:6808/717302641 12 ==== osd_op_reply(88 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eab40 con 0x5598b4fde800 2023-09-23T03:24:42.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.566+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:89 13.d 13:b0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5143180 con 0x5598b4fde800 2023-09-23T03:24:42.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.567+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 <== osd.4 v2:172.21.15.190:6808/717302641 13 ==== osd_op_reply(89 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eab40 con 0x5598b4fde800 2023-09-23T03:24:42.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.567+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:90 13.1d 13:b8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b4cefc00 con 0x5598b5005000 2023-09-23T03:24:42.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.568+0000 7f8f98483640 1 -- 172.21.15.154:0/1527791939 <== osd.7 v2:172.21.15.190:6824/3460136272 10 ==== osd_op_reply(90 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59ead80 con 0x5598b5005000 2023-09-23T03:24:42.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.568+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:91 13.3 13:c0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5143500 con 0x5598b5005400 2023-09-23T03:24:42.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.569+0000 7f8f97c82640 1 -- 172.21.15.154:0/1527791939 <== osd.0 v2:172.21.15.154:6800/2920938138 16 ==== osd_op_reply(91 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eafc0 con 0x5598b5005400 2023-09-23T03:24:42.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.569+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:92 13.13 13:c8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5135c00 con 0x5598b5005000 2023-09-23T03:24:42.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.570+0000 7f8f98483640 1 -- 172.21.15.154:0/1527791939 <== osd.7 v2:172.21.15.190:6824/3460136272 11 ==== osd_op_reply(92 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59ead80 con 0x5598b5005000 2023-09-23T03:24:42.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.570+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:93 13.b 13:d0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5143880 con 0x5598b511ac00 2023-09-23T03:24:42.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.571+0000 7f8f98483640 1 -- 172.21.15.154:0/1527791939 <== osd.5 v2:172.21.15.190:6800/882536885 9 ==== osd_op_reply(93 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59ead80 con 0x5598b511ac00 2023-09-23T03:24:42.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.571+0000 7f8f92c78640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:94 13.1b 13:d8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5134e00 con 0x5598b4fde800 2023-09-23T03:24:42.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.572+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 <== osd.4 v2:172.21.15.190:6808/717302641 14 ==== osd_op_reply(94 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eab40 con 0x5598b4fde800 2023-09-23T03:24:42.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.572+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:95 13.7 13:e0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5143c00 con 0x5598b5005000 2023-09-23T03:24:42.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.574+0000 7f8f98483640 1 -- 172.21.15.154:0/1527791939 <== osd.7 v2:172.21.15.190:6824/3460136272 12 ==== osd_op_reply(95 [pgnls start_epoch 88 out=81b] v86'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5598b59ead80 con 0x5598b5005000 2023-09-23T03:24:42.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.574+0000 7f8f994da5c0 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:96 13.7 13:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b4cef880 con 0x5598b5005000 2023-09-23T03:24:42.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.575+0000 7f8f98483640 1 -- 172.21.15.154:0/1527791939 <== osd.7 v2:172.21.15.190:6824/3460136272 13 ==== osd_op_reply(96 10000000001.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5598b59ead80 con 0x5598b5005000 2023-09-23T03:24:42.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.575+0000 7f8f994da5c0 10 datascan.operator(): handling object 1099511627777.0 2023-09-23T03:24:42.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.575+0000 7f8f994da5c0 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:97 12.7 12:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5142380 con 0x5598b5005c00 2023-09-23T03:24:42.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.579+0000 7f8f98483640 1 -- 172.21.15.154:0/1527791939 <== osd.1 v2:172.21.15.154:6816/1267710005 14 ==== osd_op_reply(97 10000000001.00000000 [call,setxattr (8)] v88'8 uv8 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5598b59ead80 con 0x5598b5005c00 2023-09-23T03:24:42.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.579+0000 7f8f994da5c0 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:98 13.17 13:e8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5142a80 con 0x5598b511ac00 2023-09-23T03:24:42.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.580+0000 7f8f98483640 1 -- 172.21.15.154:0/1527791939 <== osd.5 v2:172.21.15.190:6800/882536885 10 ==== osd_op_reply(98 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59ead80 con 0x5598b511ac00 2023-09-23T03:24:42.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.580+0000 7f8f93479640 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:99 13.f 13:f0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5135180 con 0x5598b511a800 2023-09-23T03:24:42.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.581+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 <== osd.6 v2:172.21.15.190:6816/1964294001 13 ==== osd_op_reply(99 [pgnls start_epoch 88 out=81b] v86'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5598b59eab40 con 0x5598b511a800 2023-09-23T03:24:42.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.581+0000 7f8f994da5c0 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:100 13.f 13:f526278d:::10000000001.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b4cefc00 con 0x5598b511a800 2023-09-23T03:24:42.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.582+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 <== osd.6 v2:172.21.15.190:6816/1964294001 14 ==== osd_op_reply(100 10000000001.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5598b59eab40 con 0x5598b511a800 2023-09-23T03:24:42.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.582+0000 7f8f994da5c0 10 datascan.operator(): handling object 1099511627777.1 2023-09-23T03:24:42.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.582+0000 7f8f994da5c0 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:101 12.7 12:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5124e00 con 0x5598b5005c00 2023-09-23T03:24:42.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.586+0000 7f8f98483640 1 -- 172.21.15.154:0/1527791939 <== osd.1 v2:172.21.15.154:6816/1267710005 15 ==== osd_op_reply(101 10000000001.00000000 [call,setxattr (8)] v88'9 uv9 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5598b59ead80 con 0x5598b5005c00 2023-09-23T03:24:42.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.586+0000 7f8f994da5c0 1 -- 172.21.15.154:0/1527791939 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:102 13.1f 13:f8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5598b5124000 con 0x5598b511a800 2023-09-23T03:24:42.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.587+0000 7f8f97481640 1 -- 172.21.15.154:0/1527791939 <== osd.6 v2:172.21.15.190:6816/1964294001 15 ==== osd_op_reply(102 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5598b59eab40 con 0x5598b511a800 2023-09-23T03:24:42.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.587+0000 7f8f994da5c0 1 -- 172.21.15.154:0/1527791939 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5598b5005400 msgr2=0x5598b50f2b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.587+0000 7f8f994da5c0 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5598b5005400 0x5598b50f2b00 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.587+0000 7f8f994da5c0 1 -- 172.21.15.154:0/1527791939 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5598b5005c00 msgr2=0x5598b50f3600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.587+0000 7f8f994da5c0 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5598b5005c00 0x5598b50f3600 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.587+0000 7f8f994da5c0 1 -- 172.21.15.154:0/1527791939 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5598b511a000 msgr2=0x5598b50f3b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.587+0000 7f8f994da5c0 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5598b511a000 0x5598b50f3b80 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.587+0000 7f8f994da5c0 1 -- 172.21.15.154:0/1527791939 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5598b5005800 msgr2=0x5598b50f3080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.587+0000 7f8f994da5c0 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5598b5005800 0x5598b50f3080 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.587+0000 7f8f994da5c0 1 -- 172.21.15.154:0/1527791939 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5598b4fde800 msgr2=0x5598b4d00680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.587+0000 7f8f994da5c0 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5598b4fde800 0x5598b4d00680 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.587+0000 7f8f994da5c0 1 -- 172.21.15.154:0/1527791939 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x5598b511ac00 msgr2=0x5598b50f4c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.587+0000 7f8f994da5c0 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x5598b511ac00 0x5598b50f4c00 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.587+0000 7f8f994da5c0 1 -- 172.21.15.154:0/1527791939 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x5598b511a800 msgr2=0x5598b50f4680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.587+0000 7f8f994da5c0 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x5598b511a800 0x5598b50f4680 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.587+0000 7f8f97c82640 1 -- 172.21.15.154:0/1527791939 reap_dead start 2023-09-23T03:24:42.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.588+0000 7f8f994da5c0 1 -- 172.21.15.154:0/1527791939 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x5598b5005000 msgr2=0x5598b50f2580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.588+0000 7f8f994da5c0 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x5598b5005000 0x5598b50f2580 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.588+0000 7f8f994da5c0 1 -- 172.21.15.154:0/1527791939 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5598b5004c00 msgr2=0x5598b50f2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.588+0000 7f8f97c82640 1 -- 172.21.15.154:0/1527791939 reap_dead start 2023-09-23T03:24:42.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.588+0000 7f8f994da5c0 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5598b5004c00 0x5598b50f2000 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x5598b4f82870 tx=0x5598b4f92c00 comp rx=0 tx=0).stop 2023-09-23T03:24:42.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.588+0000 7f8f994da5c0 1 -- 172.21.15.154:0/1527791939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598b4e0f800 msgr2=0x5598b4cfeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.588+0000 7f8f994da5c0 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598b4e0f800 0x5598b4cfeb00 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x5598b4f83590 tx=0x5598b4f92ea0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.588+0000 7f8f994da5c0 1 -- 172.21.15.154:0/1527791939 shutdown_connections 2023-09-23T03:24:42.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.588+0000 7f8f994da5c0 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5598b5004c00 0x5598b50f2000 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.588+0000 7f8f994da5c0 1 --2- 172.21.15.154:0/1527791939 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598b4e0f800 0x5598b4cfeb00 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.588+0000 7f8f994da5c0 1 -- 172.21.15.154:0/1527791939 >> 172.21.15.154:0/1527791939 conn(0x5598b4e0fc00 msgr2=0x5598b5a0f000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:42.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.589+0000 7f8f994da5c0 1 -- 172.21.15.154:0/1527791939 shutdown_connections 2023-09-23T03:24:42.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.589+0000 7f8f994da5c0 1 -- 172.21.15.154:0/1527791939 wait complete. 2023-09-23T03:24:42.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.590+0000 7f8f994da5c0 1 -- 172.21.15.154:0/921388744 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598b4c95c00 msgr2=0x5598b4cff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.590+0000 7f8f994da5c0 1 --2- 172.21.15.154:0/921388744 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598b4c95c00 0x5598b4cff080 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x5598b5a9f320 tx=0x5598b4f6c120 comp rx=0 tx=0).stop 2023-09-23T03:24:42.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.590+0000 7f8f994da5c0 1 -- 172.21.15.154:0/921388744 shutdown_connections 2023-09-23T03:24:42.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.590+0000 7f8f994da5c0 1 --2- 172.21.15.154:0/921388744 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5598b4c95c00 0x5598b4cff080 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.590+0000 7f8f994da5c0 1 --2- 172.21.15.154:0/921388744 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5598b4c95800 0x5598b4cff600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.590+0000 7f8f994da5c0 1 --2- 172.21.15.154:0/921388744 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5598b4c94c00 0x5598b4cffb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.590+0000 7f8f994da5c0 1 -- 172.21.15.154:0/921388744 >> 172.21.15.154:0/921388744 conn(0x5598b4c95000 msgr2=0x5598b5a0e800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:42.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.590+0000 7f8f994da5c0 1 -- 172.21.15.154:0/921388744 shutdown_connections 2023-09-23T03:24:42.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.590+0000 7f8f994da5c0 1 -- 172.21.15.154:0/921388744 wait complete. 2023-09-23T03:24:42.603 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents'] in time 0:00:00.212565, result: 2023-09-23T03:24:42.603 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes 2023-09-23T03:24:42.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.696+0000 7fa6474bb5c0 1 -- start start 2023-09-23T03:24:42.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.696+0000 7fa6474bb5c0 1 Processor -- start 2023-09-23T03:24:42.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.697+0000 7fa6474bb5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626f63aec00 0x5626f6418580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.697+0000 7fa6474bb5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5626f63af800 0x5626f6418b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.697+0000 7fa6474bb5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626f63afc00 0x5626f6419080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.697+0000 7fa6474bb5c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626f63f9180 con 0x5626f63afc00 2023-09-23T03:24:42.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.697+0000 7fa6474bb5c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626f63f8140 con 0x5626f63aec00 2023-09-23T03:24:42.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.697+0000 7fa6474bb5c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626f63f8000 con 0x5626f63af800 2023-09-23T03:24:42.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.697+0000 7fa646462640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626f63afc00 0x5626f6419080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.697+0000 7fa646462640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626f63afc00 0x5626f6419080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43370/0 (socket says 172.21.15.154:43370) 2023-09-23T03:24:42.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.697+0000 7fa646462640 1 -- 172.21.15.154:0/1470646990 learned_addr learned my addr 172.21.15.154:0/1470646990 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:42.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.698+0000 7fa645460640 1 --2- 172.21.15.154:0/1470646990 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5626f63af800 0x5626f6418b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.698+0000 7fa646462640 1 -- 172.21.15.154:0/1470646990 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5626f63af800 msgr2=0x5626f6418b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:42.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.698+0000 7fa646462640 1 --2- 172.21.15.154:0/1470646990 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5626f63af800 0x5626f6418b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.698+0000 7fa646462640 1 -- 172.21.15.154:0/1470646990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626f63aec00 msgr2=0x5626f6418580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:42.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.698+0000 7fa646462640 1 --2- 172.21.15.154:0/1470646990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626f63aec00 0x5626f6418580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.698+0000 7fa646462640 1 -- 172.21.15.154:0/1470646990 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626f63e2b60 con 0x5626f63afc00 2023-09-23T03:24:42.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.698+0000 7fa646462640 1 --2- 172.21.15.154:0/1470646990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626f63afc00 0x5626f6419080 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x5626f71b4a50 tx=0x5626f6688120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad5a97f377faaf2 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.698+0000 7fa64345c640 1 -- 172.21.15.154:0/1470646990 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5626f63f7080 con 0x5626f63afc00 2023-09-23T03:24:42.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.698+0000 7fa64345c640 1 -- 172.21.15.154:0/1470646990 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5626f63d5800 con 0x5626f63afc00 2023-09-23T03:24:42.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.698+0000 7fa64345c640 1 -- 172.21.15.154:0/1470646990 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5626f63f7600 con 0x5626f63afc00 2023-09-23T03:24:42.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.698+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/1470646990 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626f63e31e0 con 0x5626f63afc00 2023-09-23T03:24:42.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.698+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/1470646990 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5626f63e2b60 con 0x5626f63afc00 2023-09-23T03:24:42.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.698+0000 7fa6474bb5c0 4 waiting for MDS map... 2023-09-23T03:24:42.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.699+0000 7fa64345c640 1 -- 172.21.15.154:0/1470646990 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(88..88 src has 1..88) v4 ==== 6605+0+0 (secure 0 0 0) 0x5626f646ec40 con 0x5626f63afc00 2023-09-23T03:24:42.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.699+0000 7fa64345c640 1 -- 172.21.15.154:0/1470646990 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 69) v1 ==== 3306+0+0 (secure 0 0 0) 0x5626f6401200 con 0x5626f63afc00 2023-09-23T03:24:42.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.699+0000 7fa6474bb5c0 4 Got MDS map 69 2023-09-23T03:24:42.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.699+0000 7fa6474bb5c0 4 datascan.main: Using metadata pool output 2023-09-23T03:24:42.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.699+0000 7fa6474bb5c0 4 datascan.main: connecting to RADOS... 2023-09-23T03:24:42.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.700+0000 7fa6474bb5c0 1 Processor -- start 2023-09-23T03:24:42.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.700+0000 7fa6474bb5c0 1 -- start start 2023-09-23T03:24:42.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.701+0000 7fa6474bb5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626f7185c00 0x5626f6419600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.701+0000 7fa6474bb5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626f66f0000 0x5626f6419b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.701+0000 7fa6474bb5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5626f66f0400 0x5626f641a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.701+0000 7fa6474bb5c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626f63f8a00 con 0x5626f7185c00 2023-09-23T03:24:42.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.701+0000 7fa6474bb5c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626f647d180 con 0x5626f66f0000 2023-09-23T03:24:42.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.701+0000 7fa6474bb5c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626f647cdc0 con 0x5626f66f0400 2023-09-23T03:24:42.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.701+0000 7fa645460640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626f7185c00 0x5626f6419600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.701+0000 7fa645460640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626f7185c00 0x5626f6419600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43386/0 (socket says 172.21.15.154:43386) 2023-09-23T03:24:42.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.701+0000 7fa645460640 1 -- 172.21.15.154:0/1913916077 learned_addr learned my addr 172.21.15.154:0/1913916077 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:42.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.701+0000 7fa645c61640 1 --2- 172.21.15.154:0/1913916077 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626f66f0000 0x5626f6419b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.701+0000 7fa645460640 1 -- 172.21.15.154:0/1913916077 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5626f66f0400 msgr2=0x5626f641a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:42.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.701+0000 7fa645460640 1 --2- 172.21.15.154:0/1913916077 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5626f66f0400 0x5626f641a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.701+0000 7fa645460640 1 -- 172.21.15.154:0/1913916077 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626f66f0000 msgr2=0x5626f6419b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:42.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.701+0000 7fa645460640 1 --2- 172.21.15.154:0/1913916077 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626f66f0000 0x5626f6419b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.701+0000 7fa645460640 1 -- 172.21.15.154:0/1913916077 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626f63e2340 con 0x5626f7185c00 2023-09-23T03:24:42.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.702+0000 7fa645460640 1 --2- 172.21.15.154:0/1913916077 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626f7185c00 0x5626f6419600 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x5626f669aa80 tx=0x5626f66a2cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5752bb00965906 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.702+0000 7fa641458640 1 -- 172.21.15.154:0/1913916077 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5626f63f7080 con 0x5626f7185c00 2023-09-23T03:24:42.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.702+0000 7fa641458640 1 -- 172.21.15.154:0/1913916077 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5626f717ea80 con 0x5626f7185c00 2023-09-23T03:24:42.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.702+0000 7fa641458640 1 -- 172.21.15.154:0/1913916077 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5626f63f7600 con 0x5626f7185c00 2023-09-23T03:24:42.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.702+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/1913916077 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626f7185c00 msgr2=0x5626f6419600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.702+0000 7fa6474bb5c0 1 --2- 172.21.15.154:0/1913916077 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626f7185c00 0x5626f6419600 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x5626f669aa80 tx=0x5626f66a2cc0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.702+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/1913916077 shutdown_connections 2023-09-23T03:24:42.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.702+0000 7fa6474bb5c0 1 --2- 172.21.15.154:0/1913916077 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5626f66f0400 0x5626f641a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.702+0000 7fa6474bb5c0 1 --2- 172.21.15.154:0/1913916077 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626f66f0000 0x5626f6419b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.702+0000 7fa6474bb5c0 1 --2- 172.21.15.154:0/1913916077 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626f7185c00 0x5626f6419600 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.702+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/1913916077 >> 172.21.15.154:0/1913916077 conn(0x5626f7185800 msgr2=0x5626f7129000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:42.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.703+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/1913916077 shutdown_connections 2023-09-23T03:24:42.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.703+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/1913916077 wait complete. 2023-09-23T03:24:42.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.703+0000 7fa6474bb5c0 1 Processor -- start 2023-09-23T03:24:42.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.703+0000 7fa6474bb5c0 1 -- start start 2023-09-23T03:24:42.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.704+0000 7fa6474bb5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5626f7185800 0x5626f641b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.704+0000 7fa6474bb5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626f66f1000 0x5626f641ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.704+0000 7fa6474bb5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626f66f0c00 0x5626f641a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.704+0000 7fa6474bb5c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626f63f8140 con 0x5626f66f0c00 2023-09-23T03:24:42.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.704+0000 7fa6474bb5c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626f63f9180 con 0x5626f66f1000 2023-09-23T03:24:42.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.704+0000 7fa6474bb5c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626f63f8b40 con 0x5626f7185800 2023-09-23T03:24:42.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.704+0000 7fa645460640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626f66f0c00 0x5626f641a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.705+0000 7fa645460640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626f66f0c00 0x5626f641a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43396/0 (socket says 172.21.15.154:43396) 2023-09-23T03:24:42.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.705+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 learned_addr learned my addr 172.21.15.154:0/3134866139 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:42.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.705+0000 7fa645460640 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5626f7185800 0x5626f641b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.705+0000 7fa645c61640 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626f66f1000 0x5626f641ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.705+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5626f7185800 msgr2=0x5626f641b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.705+0000 7fa645460640 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5626f7185800 0x5626f641b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.705+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626f66f1000 msgr2=0x5626f641ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.705+0000 7fa645460640 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626f66f1000 0x5626f641ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.705+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626f63e2000 con 0x5626f66f0c00 2023-09-23T03:24:42.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.705+0000 7fa645c61640 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626f66f1000 0x5626f641ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:42.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.705+0000 7fa645460640 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626f66f0c00 0x5626f641a680 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x5626f669b740 tx=0x5626f66a2d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.705+0000 7fa63fc55640 1 -- 172.21.15.154:0/3134866139 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5626f63f6dc0 con 0x5626f66f0c00 2023-09-23T03:24:42.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.706+0000 7fa63fc55640 1 -- 172.21.15.154:0/3134866139 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5626f717e900 con 0x5626f66f0c00 2023-09-23T03:24:42.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.706+0000 7fa63fc55640 1 -- 172.21.15.154:0/3134866139 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5626f63f7080 con 0x5626f66f0c00 2023-09-23T03:24:42.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.706+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5626f63e2ea0 con 0x5626f66f0c00 2023-09-23T03:24:42.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.706+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626f63e3520 con 0x5626f66f0c00 2023-09-23T03:24:42.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.706+0000 7fa6474bb5c0 4 datascan.init: resolving metadata pool 11 2023-09-23T03:24:42.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.707+0000 7fa63fc55640 1 -- 172.21.15.154:0/3134866139 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x5626f63e0840 con 0x5626f66f0c00 2023-09-23T03:24:42.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.708+0000 7fa63fc55640 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5626f67f6000 0x5626f641b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.708+0000 7fa63fc55640 1 -- 172.21.15.154:0/3134866139 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6605+0+0 (secure 0 0 0) 0x5626f646ea80 con 0x5626f66f0c00 2023-09-23T03:24:42.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.708+0000 7fa6474bb5c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-09-23T03:24:42.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.708+0000 7fa6474bb5c0 4 datascan.main: data pool 'cephfs_data' has ID 12 2023-09-23T03:24:42.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.708+0000 7fa6474bb5c0 4 datascan.main: opening data pool 'cephfs_data' 2023-09-23T03:24:42.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.708+0000 7fa6474bb5c0 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x5626f66f0800 0x5626f6419600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.708+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:1 11.14 11:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6409880 con 0x5626f66f0800 2023-09-23T03:24:42.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.708+0000 7fa645c61640 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5626f67f6000 0x5626f641b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.709+0000 7fa645c61640 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5626f67f6000 0x5626f641b700 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x5626f64a06f0 tx=0x5626f6800060 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.709+0000 7fa645460640 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x5626f66f0800 0x5626f6419600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.710+0000 7fa645460640 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x5626f66f0800 0x5626f6419600 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.711+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv30 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x5626f724cb40 con 0x5626f66f0800 2023-09-23T03:24:42.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.711+0000 7fa6474bb5c0 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5626f66f0400 0x5626f641a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.711+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:2 11.1f 11:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6409c00 con 0x5626f66f0400 2023-09-23T03:24:42.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.711+0000 7fa646462640 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5626f66f0400 0x5626f641a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.712+0000 7fa646462640 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5626f66f0400 0x5626f641a100 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.712+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.2 v2:172.21.15.154:6822/2884407243 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x5626f724cd80 con 0x5626f66f0400 2023-09-23T03:24:42.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.712+0000 7fa6474bb5c0 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5626f66f0000 0x5626f6419b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.712+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:3 12.0 12:00000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681a000 con 0x5626f66f0000 2023-09-23T03:24:42.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.713+0000 7fa645c61640 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5626f66f0000 0x5626f6419b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.714+0000 7fa645c61640 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5626f66f0000 0x5626f6419b80 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.715+0000 7fa645c61640 1 -- 172.21.15.154:0/3134866139 <== osd.4 v2:172.21.15.190:6808/717302641 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cfc0 con 0x5626f66f0000 2023-09-23T03:24:42.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.715+0000 7fa640c57640 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x5626f67f7000 0x5626f682a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.715+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:4 12.20 12:04000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681ae00 con 0x5626f67f7000 2023-09-23T03:24:42.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.715+0000 7fa645460640 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x5626f67f7000 0x5626f682a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.716+0000 7fa645460640 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x5626f67f7000 0x5626f682a000 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.717+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.7 v2:172.21.15.190:6824/3460136272 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cb40 con 0x5626f67f7000 2023-09-23T03:24:42.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.717+0000 7fa641458640 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5626f67f7400 0x5626f682a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.717+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:5 12.10 12:08000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6409880 con 0x5626f67f7400 2023-09-23T03:24:42.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.717+0000 7fa646462640 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5626f67f7400 0x5626f682a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.718+0000 7fa646462640 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5626f67f7400 0x5626f682a580 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.718+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.0 v2:172.21.15.154:6800/2920938138 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f67f7400 2023-09-23T03:24:42.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.718+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:6 12.30 12:0c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681ae00 con 0x5626f67f7000 2023-09-23T03:24:42.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.719+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.7 v2:172.21.15.190:6824/3460136272 2 ==== osd_op_reply(6 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cb40 con 0x5626f67f7000 2023-09-23T03:24:42.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.719+0000 7fa641458640 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5626f67f7800 0x5626f682ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.719+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:7 12.8 12:10000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681b500 con 0x5626f67f7800 2023-09-23T03:24:42.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.719+0000 7fa645c61640 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5626f67f7800 0x5626f682ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.720+0000 7fa645c61640 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5626f67f7800 0x5626f682ab00 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.720+0000 7fa645c61640 1 -- 172.21.15.154:0/3134866139 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(7 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cfc0 con 0x5626f67f7800 2023-09-23T03:24:42.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.720+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:8 12.28 12:14000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681b180 con 0x5626f67f7400 2023-09-23T03:24:42.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.721+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.0 v2:172.21.15.154:6800/2920938138 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f67f7400 2023-09-23T03:24:42.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.721+0000 7fa641458640 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5626f67f7c00 0x5626f682b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.721+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:9 12.18 12:18000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681b880 con 0x5626f67f7c00 2023-09-23T03:24:42.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.721+0000 7fa645460640 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5626f67f7c00 0x5626f682b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.721+0000 7fa645460640 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5626f67f7c00 0x5626f682b080 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.722+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.1 v2:172.21.15.154:6816/1267710005 1 ==== osd_op_reply(9 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cb40 con 0x5626f67f7c00 2023-09-23T03:24:42.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.722+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:10 12.38 12:1c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681b180 con 0x5626f66f0400 2023-09-23T03:24:42.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.722+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.2 v2:172.21.15.154:6822/2884407243 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f66f0400 2023-09-23T03:24:42.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.722+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:11 12.4 12:20000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681a000 con 0x5626f66f0800 2023-09-23T03:24:42.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.723+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.6 v2:172.21.15.190:6816/1964294001 2 ==== osd_op_reply(11 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cb40 con 0x5626f66f0800 2023-09-23T03:24:42.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.723+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:12 12.24 12:24000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6409c00 con 0x5626f66f0400 2023-09-23T03:24:42.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.723+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.2 v2:172.21.15.154:6822/2884407243 3 ==== osd_op_reply(12 [pgnls-filter start_epoch 88 out=49b] v86'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f66f0400 2023-09-23T03:24:42.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.723+0000 7fa641458640 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x5626f6840000 0x5626f682b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.723+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:13 12.14 12:28000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681b500 con 0x5626f6840000 2023-09-23T03:24:42.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.724+0000 7fa646462640 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x5626f6840000 0x5626f682b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.725+0000 7fa646462640 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x5626f6840000 0x5626f682b600 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.726+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.5 v2:172.21.15.190:6800/882536885 1 ==== osd_op_reply(13 [pgnls-filter start_epoch 88 out=49b] v86'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f6840000 2023-09-23T03:24:42.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.726+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:14 12.34 12:2c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6409880 con 0x5626f6840000 2023-09-23T03:24:42.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.726+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.5 v2:172.21.15.190:6800/882536885 2 ==== osd_op_reply(14 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f6840000 2023-09-23T03:24:42.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.726+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:15 12.c 12:30000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681bc00 con 0x5626f67f7400 2023-09-23T03:24:42.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.727+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.0 v2:172.21.15.154:6800/2920938138 3 ==== osd_op_reply(15 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f67f7400 2023-09-23T03:24:42.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.727+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:16 12.2c 12:34000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681b180 con 0x5626f66f0400 2023-09-23T03:24:42.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.727+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.2 v2:172.21.15.154:6822/2884407243 4 ==== osd_op_reply(16 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f66f0400 2023-09-23T03:24:42.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.727+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:17 12.1c 12:38000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681ae00 con 0x5626f67f7400 2023-09-23T03:24:42.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.727+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.0 v2:172.21.15.154:6800/2920938138 4 ==== osd_op_reply(17 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f67f7400 2023-09-23T03:24:42.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.727+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:18 12.3c 12:3c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681b500 con 0x5626f6840000 2023-09-23T03:24:42.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.728+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.5 v2:172.21.15.190:6800/882536885 3 ==== osd_op_reply(18 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f6840000 2023-09-23T03:24:42.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.728+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:19 12.2 12:40000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681b880 con 0x5626f66f0000 2023-09-23T03:24:42.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.729+0000 7fa645c61640 1 -- 172.21.15.154:0/3134866139 <== osd.4 v2:172.21.15.190:6808/717302641 2 ==== osd_op_reply(19 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cfc0 con 0x5626f66f0000 2023-09-23T03:24:42.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.729+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:20 12.22 12:44000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6409c00 con 0x5626f67f7000 2023-09-23T03:24:42.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.730+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.7 v2:172.21.15.190:6824/3460136272 3 ==== osd_op_reply(20 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cb40 con 0x5626f67f7000 2023-09-23T03:24:42.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.730+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:21 12.12 12:48000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681a000 con 0x5626f67f7800 2023-09-23T03:24:42.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.731+0000 7fa645c61640 1 -- 172.21.15.154:0/3134866139 <== osd.3 v2:172.21.15.154:6808/1133688468 2 ==== osd_op_reply(21 [pgnls-filter start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cfc0 con 0x5626f67f7800 2023-09-23T03:24:42.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.731+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:22 12.32 12:4c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6409880 con 0x5626f67f7800 2023-09-23T03:24:42.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.731+0000 7fa645c61640 1 -- 172.21.15.154:0/3134866139 <== osd.3 v2:172.21.15.154:6808/1133688468 3 ==== osd_op_reply(22 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cfc0 con 0x5626f67f7800 2023-09-23T03:24:42.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.731+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:23 12.a 12:50000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6409880 con 0x5626f66f0400 2023-09-23T03:24:42.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.731+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.2 v2:172.21.15.154:6822/2884407243 5 ==== osd_op_reply(23 [pgnls-filter start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f66f0400 2023-09-23T03:24:42.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.731+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:24 12.2a 12:54000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6409880 con 0x5626f66f0400 2023-09-23T03:24:42.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.732+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.2 v2:172.21.15.154:6822/2884407243 6 ==== osd_op_reply(24 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f66f0400 2023-09-23T03:24:42.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.732+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:25 12.1a 12:58000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681b880 con 0x5626f66f0000 2023-09-23T03:24:42.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.733+0000 7fa645c61640 1 -- 172.21.15.154:0/3134866139 <== osd.4 v2:172.21.15.190:6808/717302641 3 ==== osd_op_reply(25 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cfc0 con 0x5626f66f0000 2023-09-23T03:24:42.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.733+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:26 12.3a 12:5c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681bc00 con 0x5626f67f7800 2023-09-23T03:24:42.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.733+0000 7fa645c61640 1 -- 172.21.15.154:0/3134866139 <== osd.3 v2:172.21.15.154:6808/1133688468 4 ==== osd_op_reply(26 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cfc0 con 0x5626f67f7800 2023-09-23T03:24:42.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.733+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:27 12.6 12:60000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681a000 con 0x5626f67f7c00 2023-09-23T03:24:42.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.734+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.1 v2:172.21.15.154:6816/1267710005 2 ==== osd_op_reply(27 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cb40 con 0x5626f67f7c00 2023-09-23T03:24:42.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.734+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:28 12.26 12:64000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681b180 con 0x5626f6840000 2023-09-23T03:24:42.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.735+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.5 v2:172.21.15.190:6800/882536885 4 ==== osd_op_reply(28 [pgnls-filter start_epoch 88 out=49b] v86'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f6840000 2023-09-23T03:24:42.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.735+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:29 12.16 12:68000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f684c000 con 0x5626f66f0800 2023-09-23T03:24:42.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.736+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.6 v2:172.21.15.190:6816/1964294001 3 ==== osd_op_reply(29 [pgnls-filter start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cb40 con 0x5626f66f0800 2023-09-23T03:24:42.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.736+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:30 12.36 12:6c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681ae00 con 0x5626f6840000 2023-09-23T03:24:42.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.737+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.5 v2:172.21.15.190:6800/882536885 5 ==== osd_op_reply(30 [pgnls-filter start_epoch 88 out=49b] v86'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f6840000 2023-09-23T03:24:42.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.737+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:31 12.e 12:70000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f684c380 con 0x5626f67f7400 2023-09-23T03:24:42.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.738+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.0 v2:172.21.15.154:6800/2920938138 5 ==== osd_op_reply(31 [pgnls-filter start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f67f7400 2023-09-23T03:24:42.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.738+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:32 12.2e 12:74000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681b500 con 0x5626f67f7c00 2023-09-23T03:24:42.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.738+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.1 v2:172.21.15.154:6816/1267710005 3 ==== osd_op_reply(32 [pgnls-filter start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cb40 con 0x5626f67f7c00 2023-09-23T03:24:42.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.738+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:33 12.1e 12:78000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f684c700 con 0x5626f66f0000 2023-09-23T03:24:42.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.739+0000 7fa645c61640 1 -- 172.21.15.154:0/3134866139 <== osd.4 v2:172.21.15.190:6808/717302641 4 ==== osd_op_reply(33 [pgnls-filter start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cfc0 con 0x5626f66f0000 2023-09-23T03:24:42.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.740+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:34 12.3e 12:7c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f684c000 con 0x5626f67f7400 2023-09-23T03:24:42.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.740+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.0 v2:172.21.15.154:6800/2920938138 6 ==== osd_op_reply(34 [pgnls-filter start_epoch 88 out=49b] v86'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f67f7400 2023-09-23T03:24:42.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.740+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:35 12.1 12:80000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f684ca80 con 0x5626f66f0800 2023-09-23T03:24:42.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.741+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.6 v2:172.21.15.190:6816/1964294001 4 ==== osd_op_reply(35 [pgnls-filter start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cb40 con 0x5626f66f0800 2023-09-23T03:24:42.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.741+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:36 12.21 12:84000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f684d500 con 0x5626f67f7c00 2023-09-23T03:24:42.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.742+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.1 v2:172.21.15.154:6816/1267710005 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 88 out=49b] v86'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cb40 con 0x5626f67f7c00 2023-09-23T03:24:42.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.742+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:37 12.11 12:88000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f684ce00 con 0x5626f6840000 2023-09-23T03:24:42.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.743+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.5 v2:172.21.15.190:6800/882536885 6 ==== osd_op_reply(37 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f6840000 2023-09-23T03:24:42.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.743+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:38 12.31 12:8c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f684d880 con 0x5626f67f7400 2023-09-23T03:24:42.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.744+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.0 v2:172.21.15.154:6800/2920938138 7 ==== osd_op_reply(38 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f67f7400 2023-09-23T03:24:42.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.744+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:39 12.9 12:90000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f684d180 con 0x5626f66f0400 2023-09-23T03:24:42.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.744+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.2 v2:172.21.15.154:6822/2884407243 7 ==== osd_op_reply(39 [pgnls-filter start_epoch 88 out=49b] v86'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f66f0400 2023-09-23T03:24:42.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.744+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:40 12.29 12:94000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681a000 con 0x5626f66f0000 2023-09-23T03:24:42.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.745+0000 7fa645c61640 1 -- 172.21.15.154:0/3134866139 <== osd.4 v2:172.21.15.190:6808/717302641 5 ==== osd_op_reply(40 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cfc0 con 0x5626f66f0000 2023-09-23T03:24:42.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.745+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:41 12.19 12:98000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f684c700 con 0x5626f67f7400 2023-09-23T03:24:42.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.746+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.0 v2:172.21.15.154:6800/2920938138 8 ==== osd_op_reply(41 [pgnls-filter start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f67f7400 2023-09-23T03:24:42.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.746+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:42 12.39 12:9c000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681b500 con 0x5626f66f0800 2023-09-23T03:24:42.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.747+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.6 v2:172.21.15.190:6816/1964294001 5 ==== osd_op_reply(42 [pgnls-filter start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cb40 con 0x5626f66f0800 2023-09-23T03:24:42.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.747+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:43 12.5 12:a0000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681b880 con 0x5626f66f0800 2023-09-23T03:24:42.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.748+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.6 v2:172.21.15.190:6816/1964294001 6 ==== osd_op_reply(43 [pgnls-filter start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cb40 con 0x5626f66f0800 2023-09-23T03:24:42.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.748+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:44 12.25 12:a4000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6409c00 con 0x5626f67f7000 2023-09-23T03:24:42.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.749+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.7 v2:172.21.15.190:6824/3460136272 4 ==== osd_op_reply(44 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cb40 con 0x5626f67f7000 2023-09-23T03:24:42.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.749+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:45 12.15 12:a8000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681bc00 con 0x5626f66f0800 2023-09-23T03:24:42.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.750+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.6 v2:172.21.15.190:6816/1964294001 7 ==== osd_op_reply(45 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cb40 con 0x5626f66f0800 2023-09-23T03:24:42.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.750+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:46 12.35 12:ac000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f684ca80 con 0x5626f67f7400 2023-09-23T03:24:42.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.750+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.0 v2:172.21.15.154:6800/2920938138 9 ==== osd_op_reply(46 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f67f7400 2023-09-23T03:24:42.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.750+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:47 12.d 12:b0000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681a000 con 0x5626f67f7000 2023-09-23T03:24:42.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.751+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.7 v2:172.21.15.190:6824/3460136272 5 ==== osd_op_reply(47 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cb40 con 0x5626f67f7000 2023-09-23T03:24:42.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.751+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:48 12.2d 12:b4000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f684d500 con 0x5626f67f7800 2023-09-23T03:24:42.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.752+0000 7fa645c61640 1 -- 172.21.15.154:0/3134866139 <== osd.3 v2:172.21.15.154:6808/1133688468 5 ==== osd_op_reply(48 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cfc0 con 0x5626f67f7800 2023-09-23T03:24:42.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.752+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:49 12.1d 12:b8000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681b180 con 0x5626f67f7400 2023-09-23T03:24:42.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.752+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.0 v2:172.21.15.154:6800/2920938138 10 ==== osd_op_reply(49 [pgnls-filter start_epoch 88 out=49b] v86'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f67f7400 2023-09-23T03:24:42.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.752+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:50 12.3d 12:bc000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f684c380 con 0x5626f67f7c00 2023-09-23T03:24:42.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.753+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.1 v2:172.21.15.154:6816/1267710005 5 ==== osd_op_reply(50 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cb40 con 0x5626f67f7c00 2023-09-23T03:24:42.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.753+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:51 12.3 12:c0000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681ae00 con 0x5626f67f7400 2023-09-23T03:24:42.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.754+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.0 v2:172.21.15.154:6800/2920938138 11 ==== osd_op_reply(51 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f67f7400 2023-09-23T03:24:42.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.754+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:52 12.23 12:c4000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f684c000 con 0x5626f67f7400 2023-09-23T03:24:42.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.754+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.0 v2:172.21.15.154:6800/2920938138 12 ==== osd_op_reply(52 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f67f7400 2023-09-23T03:24:42.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.754+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:53 12.13 12:c8000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6409880 con 0x5626f67f7c00 2023-09-23T03:24:42.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.755+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.1 v2:172.21.15.154:6816/1267710005 6 ==== osd_op_reply(53 [pgnls-filter start_epoch 88 out=49b] v86'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cb40 con 0x5626f67f7c00 2023-09-23T03:24:42.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.755+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:54 12.33 12:cc000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6409c00 con 0x5626f66f0400 2023-09-23T03:24:42.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.756+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.2 v2:172.21.15.154:6822/2884407243 8 ==== osd_op_reply(54 [pgnls-filter start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f66f0400 2023-09-23T03:24:42.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.756+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:55 12.b 12:d0000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f684d880 con 0x5626f67f7800 2023-09-23T03:24:42.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.756+0000 7fa645c61640 1 -- 172.21.15.154:0/3134866139 <== osd.3 v2:172.21.15.154:6808/1133688468 6 ==== osd_op_reply(55 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cfc0 con 0x5626f67f7800 2023-09-23T03:24:42.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.756+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:56 12.2b 12:d4000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681b500 con 0x5626f66f0000 2023-09-23T03:24:42.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.757+0000 7fa645c61640 1 -- 172.21.15.154:0/3134866139 <== osd.4 v2:172.21.15.190:6808/717302641 6 ==== osd_op_reply(56 [pgnls-filter start_epoch 88 out=49b] v86'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cfc0 con 0x5626f66f0000 2023-09-23T03:24:42.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.758+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:57 12.1b 12:d8000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f684d180 con 0x5626f66f0000 2023-09-23T03:24:42.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.759+0000 7fa645c61640 1 -- 172.21.15.154:0/3134866139 <== osd.4 v2:172.21.15.190:6808/717302641 7 ==== osd_op_reply(57 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cfc0 con 0x5626f66f0000 2023-09-23T03:24:42.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.759+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:58 12.3b 12:dc000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681b880 con 0x5626f66f0400 2023-09-23T03:24:42.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.759+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.2 v2:172.21.15.154:6822/2884407243 9 ==== osd_op_reply(58 [pgnls-filter start_epoch 88 out=49b] v86'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f66f0400 2023-09-23T03:24:42.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.760+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:59 12.7 12:e0000000::::head [pgnls-filter start_epoch 88 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f684c700 con 0x5626f67f7c00 2023-09-23T03:24:42.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.760+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.1 v2:172.21.15.154:6816/1267710005 7 ==== osd_op_reply(59 [pgnls-filter start_epoch 88 out=81b] v88'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5626f724cb40 con 0x5626f67f7c00 2023-09-23T03:24:42.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.761+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:60 12.0 12:00000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681a380 con 0x5626f66f0000 2023-09-23T03:24:42.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.762+0000 7fa645c61640 1 -- 172.21.15.154:0/3134866139 <== osd.4 v2:172.21.15.190:6808/717302641 8 ==== osd_op_reply(60 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cfc0 con 0x5626f66f0000 2023-09-23T03:24:42.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.762+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:61 12.20 12:04000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f684ca80 con 0x5626f67f7000 2023-09-23T03:24:42.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.763+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.7 v2:172.21.15.190:6824/3460136272 6 ==== osd_op_reply(61 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cb40 con 0x5626f67f7000 2023-09-23T03:24:42.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.763+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:62 12.10 12:08000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f684c380 con 0x5626f67f7400 2023-09-23T03:24:42.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.764+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.0 v2:172.21.15.154:6800/2920938138 13 ==== osd_op_reply(62 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f67f7400 2023-09-23T03:24:42.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.764+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:63 12.30 12:0c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f684c000 con 0x5626f67f7000 2023-09-23T03:24:42.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.765+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.7 v2:172.21.15.190:6824/3460136272 7 ==== osd_op_reply(63 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cb40 con 0x5626f67f7000 2023-09-23T03:24:42.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.765+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:64 12.8 12:10000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f684c700 con 0x5626f67f7800 2023-09-23T03:24:42.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.766+0000 7fa645c61640 1 -- 172.21.15.154:0/3134866139 <== osd.3 v2:172.21.15.154:6808/1133688468 7 ==== osd_op_reply(64 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cfc0 con 0x5626f67f7800 2023-09-23T03:24:42.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.766+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:65 12.28 12:14000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f684ce00 con 0x5626f67f7400 2023-09-23T03:24:42.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.766+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.0 v2:172.21.15.154:6800/2920938138 14 ==== osd_op_reply(65 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f67f7400 2023-09-23T03:24:42.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.767+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:66 12.18 12:18000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f684ca80 con 0x5626f67f7c00 2023-09-23T03:24:42.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.767+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.1 v2:172.21.15.154:6816/1267710005 8 ==== osd_op_reply(66 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cb40 con 0x5626f67f7c00 2023-09-23T03:24:42.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.767+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:67 12.38 12:1c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681b180 con 0x5626f66f0400 2023-09-23T03:24:42.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.768+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.2 v2:172.21.15.154:6822/2884407243 10 ==== osd_op_reply(67 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f66f0400 2023-09-23T03:24:42.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.768+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:68 12.4 12:20000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681bc00 con 0x5626f66f0800 2023-09-23T03:24:42.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.769+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.6 v2:172.21.15.190:6816/1964294001 8 ==== osd_op_reply(68 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cb40 con 0x5626f66f0800 2023-09-23T03:24:42.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.769+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:69 12.24 12:24000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681ae00 con 0x5626f66f0400 2023-09-23T03:24:42.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.770+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.2 v2:172.21.15.154:6822/2884407243 11 ==== osd_op_reply(69 [pgnls-filter start_epoch 88 out=49b] v86'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f66f0400 2023-09-23T03:24:42.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.770+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:70 12.14 12:28000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681a000 con 0x5626f6840000 2023-09-23T03:24:42.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.771+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.5 v2:172.21.15.190:6800/882536885 7 ==== osd_op_reply(70 [pgnls-filter start_epoch 88 out=49b] v86'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f6840000 2023-09-23T03:24:42.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.771+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:71 12.34 12:2c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f684d500 con 0x5626f6840000 2023-09-23T03:24:42.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.772+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.5 v2:172.21.15.190:6800/882536885 8 ==== osd_op_reply(71 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f6840000 2023-09-23T03:24:42.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.772+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:72 12.c 12:30000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681b500 con 0x5626f67f7400 2023-09-23T03:24:42.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.773+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.0 v2:172.21.15.154:6800/2920938138 15 ==== osd_op_reply(72 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f67f7400 2023-09-23T03:24:42.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.773+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:73 12.2c 12:34000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f684d880 con 0x5626f66f0400 2023-09-23T03:24:42.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.774+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.2 v2:172.21.15.154:6822/2884407243 12 ==== osd_op_reply(73 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f66f0400 2023-09-23T03:24:42.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.774+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:74 12.1c 12:38000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681a380 con 0x5626f67f7400 2023-09-23T03:24:42.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.775+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.0 v2:172.21.15.154:6800/2920938138 16 ==== osd_op_reply(74 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f67f7400 2023-09-23T03:24:42.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.775+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:75 12.3c 12:3c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f684d180 con 0x5626f6840000 2023-09-23T03:24:42.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.776+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.5 v2:172.21.15.190:6800/882536885 9 ==== osd_op_reply(75 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f6840000 2023-09-23T03:24:42.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.776+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:76 12.2 12:40000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6409880 con 0x5626f66f0000 2023-09-23T03:24:42.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.777+0000 7fa645c61640 1 -- 172.21.15.154:0/3134866139 <== osd.4 v2:172.21.15.190:6808/717302641 9 ==== osd_op_reply(76 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cfc0 con 0x5626f66f0000 2023-09-23T03:24:42.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.777+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:77 12.22 12:44000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f684dc00 con 0x5626f67f7000 2023-09-23T03:24:42.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.778+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.7 v2:172.21.15.190:6824/3460136272 8 ==== osd_op_reply(77 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cb40 con 0x5626f67f7000 2023-09-23T03:24:42.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.778+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:78 12.12 12:48000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f684c380 con 0x5626f67f7800 2023-09-23T03:24:42.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.779+0000 7fa645c61640 1 -- 172.21.15.154:0/3134866139 <== osd.3 v2:172.21.15.154:6808/1133688468 8 ==== osd_op_reply(78 [pgnls-filter start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cfc0 con 0x5626f67f7800 2023-09-23T03:24:42.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.779+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:79 12.32 12:4c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6854000 con 0x5626f67f7800 2023-09-23T03:24:42.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.780+0000 7fa645c61640 1 -- 172.21.15.154:0/3134866139 <== osd.3 v2:172.21.15.154:6808/1133688468 9 ==== osd_op_reply(79 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cfc0 con 0x5626f67f7800 2023-09-23T03:24:42.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.780+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:80 12.a 12:50000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f684ce00 con 0x5626f66f0400 2023-09-23T03:24:42.782 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.781+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.2 v2:172.21.15.154:6822/2884407243 13 ==== osd_op_reply(80 [pgnls-filter start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f66f0400 2023-09-23T03:24:42.782 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.781+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:81 12.2a 12:54000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6854380 con 0x5626f66f0400 2023-09-23T03:24:42.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.782+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.2 v2:172.21.15.154:6822/2884407243 14 ==== osd_op_reply(81 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f66f0400 2023-09-23T03:24:42.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.782+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:82 12.1a 12:58000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f684d500 con 0x5626f66f0000 2023-09-23T03:24:42.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.783+0000 7fa645c61640 1 -- 172.21.15.154:0/3134866139 <== osd.4 v2:172.21.15.190:6808/717302641 10 ==== osd_op_reply(82 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cfc0 con 0x5626f66f0000 2023-09-23T03:24:42.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.783+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:83 12.3a 12:5c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6854700 con 0x5626f67f7800 2023-09-23T03:24:42.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.784+0000 7fa645c61640 1 -- 172.21.15.154:0/3134866139 <== osd.3 v2:172.21.15.154:6808/1133688468 10 ==== osd_op_reply(83 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cfc0 con 0x5626f67f7800 2023-09-23T03:24:42.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.784+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:84 12.6 12:60000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681b880 con 0x5626f67f7c00 2023-09-23T03:24:42.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.785+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.1 v2:172.21.15.154:6816/1267710005 9 ==== osd_op_reply(84 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cb40 con 0x5626f67f7c00 2023-09-23T03:24:42.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.785+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:85 12.26 12:64000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6854a80 con 0x5626f6840000 2023-09-23T03:24:42.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.786+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.5 v2:172.21.15.190:6800/882536885 10 ==== osd_op_reply(85 [pgnls-filter start_epoch 88 out=49b] v86'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f6840000 2023-09-23T03:24:42.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.786+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:86 12.16 12:68000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681b180 con 0x5626f66f0800 2023-09-23T03:24:42.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.787+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.6 v2:172.21.15.190:6816/1964294001 9 ==== osd_op_reply(86 [pgnls-filter start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cb40 con 0x5626f66f0800 2023-09-23T03:24:42.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.787+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:87 12.36 12:6c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6854e00 con 0x5626f6840000 2023-09-23T03:24:42.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.788+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.5 v2:172.21.15.190:6800/882536885 11 ==== osd_op_reply(87 [pgnls-filter start_epoch 88 out=49b] v86'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f6840000 2023-09-23T03:24:42.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.788+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:88 12.e 12:70000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681ae00 con 0x5626f67f7400 2023-09-23T03:24:42.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.789+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.0 v2:172.21.15.154:6800/2920938138 17 ==== osd_op_reply(88 [pgnls-filter start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f67f7400 2023-09-23T03:24:42.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.789+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:89 12.2e 12:74000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6855180 con 0x5626f67f7c00 2023-09-23T03:24:42.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.790+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.1 v2:172.21.15.154:6816/1267710005 10 ==== osd_op_reply(89 [pgnls-filter start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cb40 con 0x5626f67f7c00 2023-09-23T03:24:42.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.790+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:90 12.1e 12:78000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681a000 con 0x5626f66f0000 2023-09-23T03:24:42.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.791+0000 7fa645c61640 1 -- 172.21.15.154:0/3134866139 <== osd.4 v2:172.21.15.190:6808/717302641 11 ==== osd_op_reply(90 [pgnls-filter start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cfc0 con 0x5626f66f0000 2023-09-23T03:24:42.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.791+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:91 12.3e 12:7c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6855500 con 0x5626f67f7400 2023-09-23T03:24:42.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.792+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.0 v2:172.21.15.154:6800/2920938138 18 ==== osd_op_reply(91 [pgnls-filter start_epoch 88 out=49b] v86'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f67f7400 2023-09-23T03:24:42.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.792+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:92 12.1 12:80000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6409c00 con 0x5626f66f0800 2023-09-23T03:24:42.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.793+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.6 v2:172.21.15.190:6816/1964294001 10 ==== osd_op_reply(92 [pgnls-filter start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cb40 con 0x5626f66f0800 2023-09-23T03:24:42.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.794+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:93 12.21 12:84000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6855880 con 0x5626f67f7c00 2023-09-23T03:24:42.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.795+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.1 v2:172.21.15.154:6816/1267710005 11 ==== osd_op_reply(93 [pgnls-filter start_epoch 88 out=49b] v86'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cb40 con 0x5626f67f7c00 2023-09-23T03:24:42.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.795+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:94 12.11 12:88000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6855180 con 0x5626f6840000 2023-09-23T03:24:42.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.796+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.5 v2:172.21.15.190:6800/882536885 12 ==== osd_op_reply(94 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f6840000 2023-09-23T03:24:42.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.796+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:95 12.31 12:8c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6855c00 con 0x5626f67f7400 2023-09-23T03:24:42.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.797+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.0 v2:172.21.15.154:6800/2920938138 19 ==== osd_op_reply(95 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f67f7400 2023-09-23T03:24:42.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.797+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:96 12.9 12:90000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6854380 con 0x5626f66f0400 2023-09-23T03:24:42.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.797+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.2 v2:172.21.15.154:6822/2884407243 15 ==== osd_op_reply(96 [pgnls-filter start_epoch 88 out=49b] v86'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f66f0400 2023-09-23T03:24:42.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.798+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:97 12.29 12:94000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6854e00 con 0x5626f66f0000 2023-09-23T03:24:42.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.799+0000 7fa645c61640 1 -- 172.21.15.154:0/3134866139 <== osd.4 v2:172.21.15.190:6808/717302641 12 ==== osd_op_reply(97 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cfc0 con 0x5626f66f0000 2023-09-23T03:24:42.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.799+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:98 12.19 12:98000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6854a80 con 0x5626f67f7400 2023-09-23T03:24:42.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.799+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.0 v2:172.21.15.154:6800/2920938138 20 ==== osd_op_reply(98 [pgnls-filter start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f67f7400 2023-09-23T03:24:42.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.800+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:99 12.39 12:9c000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6854000 con 0x5626f66f0800 2023-09-23T03:24:42.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.801+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.6 v2:172.21.15.190:6816/1964294001 11 ==== osd_op_reply(99 [pgnls-filter start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cb40 con 0x5626f66f0800 2023-09-23T03:24:42.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.801+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:100 12.5 12:a0000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6409880 con 0x5626f66f0800 2023-09-23T03:24:42.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.802+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.6 v2:172.21.15.190:6816/1964294001 12 ==== osd_op_reply(100 [pgnls-filter start_epoch 88 out=49b] v86'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cb40 con 0x5626f66f0800 2023-09-23T03:24:42.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.802+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:101 12.25 12:a4000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6854700 con 0x5626f67f7000 2023-09-23T03:24:42.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.803+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.7 v2:172.21.15.190:6824/3460136272 9 ==== osd_op_reply(101 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cb40 con 0x5626f67f7000 2023-09-23T03:24:42.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.803+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:102 12.15 12:a8000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681bc00 con 0x5626f66f0800 2023-09-23T03:24:42.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.804+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.6 v2:172.21.15.190:6816/1964294001 13 ==== osd_op_reply(102 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cb40 con 0x5626f66f0800 2023-09-23T03:24:42.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.804+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:103 12.35 12:ac000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f685c000 con 0x5626f67f7400 2023-09-23T03:24:42.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.805+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.0 v2:172.21.15.154:6800/2920938138 21 ==== osd_op_reply(103 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f67f7400 2023-09-23T03:24:42.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.805+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:104 12.d 12:b0000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681b880 con 0x5626f67f7000 2023-09-23T03:24:42.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.806+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.7 v2:172.21.15.190:6824/3460136272 10 ==== osd_op_reply(104 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cb40 con 0x5626f67f7000 2023-09-23T03:24:42.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.807+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:105 12.2d 12:b4000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f685c380 con 0x5626f67f7800 2023-09-23T03:24:42.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.808+0000 7fa645c61640 1 -- 172.21.15.154:0/3134866139 <== osd.3 v2:172.21.15.154:6808/1133688468 11 ==== osd_op_reply(105 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cfc0 con 0x5626f67f7800 2023-09-23T03:24:42.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.808+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:106 12.1d 12:b8000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681b180 con 0x5626f67f7400 2023-09-23T03:24:42.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.809+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.0 v2:172.21.15.154:6800/2920938138 22 ==== osd_op_reply(106 [pgnls-filter start_epoch 88 out=49b] v86'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f67f7400 2023-09-23T03:24:42.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.809+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:107 12.3d 12:bc000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f685c700 con 0x5626f67f7c00 2023-09-23T03:24:42.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.810+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.1 v2:172.21.15.154:6816/1267710005 12 ==== osd_op_reply(107 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cb40 con 0x5626f67f7c00 2023-09-23T03:24:42.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.810+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:108 12.3 12:c0000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681b500 con 0x5626f67f7400 2023-09-23T03:24:42.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.810+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.0 v2:172.21.15.154:6800/2920938138 23 ==== osd_op_reply(108 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f67f7400 2023-09-23T03:24:42.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.811+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:109 12.23 12:c4000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f685ca80 con 0x5626f67f7400 2023-09-23T03:24:42.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.811+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.0 v2:172.21.15.154:6800/2920938138 24 ==== osd_op_reply(109 [pgnls-filter start_epoch 88 out=49b] v86'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f67f7400 2023-09-23T03:24:42.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.811+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:110 12.13 12:c8000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681a380 con 0x5626f67f7c00 2023-09-23T03:24:42.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.812+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.1 v2:172.21.15.154:6816/1267710005 13 ==== osd_op_reply(110 [pgnls-filter start_epoch 88 out=49b] v86'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cb40 con 0x5626f67f7c00 2023-09-23T03:24:42.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.812+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:111 12.33 12:cc000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f685ce00 con 0x5626f66f0400 2023-09-23T03:24:42.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.813+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.2 v2:172.21.15.154:6822/2884407243 16 ==== osd_op_reply(111 [pgnls-filter start_epoch 88 out=49b] v86'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f66f0400 2023-09-23T03:24:42.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.813+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:112 12.b 12:d0000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681ae00 con 0x5626f67f7800 2023-09-23T03:24:42.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.814+0000 7fa645c61640 1 -- 172.21.15.154:0/3134866139 <== osd.3 v2:172.21.15.154:6808/1133688468 12 ==== osd_op_reply(112 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cfc0 con 0x5626f67f7800 2023-09-23T03:24:42.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.814+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:113 12.2b 12:d4000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f685d180 con 0x5626f66f0000 2023-09-23T03:24:42.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.816+0000 7fa645c61640 1 -- 172.21.15.154:0/3134866139 <== osd.4 v2:172.21.15.190:6808/717302641 13 ==== osd_op_reply(113 [pgnls-filter start_epoch 88 out=49b] v86'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cfc0 con 0x5626f66f0000 2023-09-23T03:24:42.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.816+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:114 12.1b 12:d8000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681b180 con 0x5626f66f0000 2023-09-23T03:24:42.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.817+0000 7fa645c61640 1 -- 172.21.15.154:0/3134866139 <== osd.4 v2:172.21.15.190:6808/717302641 14 ==== osd_op_reply(114 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cfc0 con 0x5626f66f0000 2023-09-23T03:24:42.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.817+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:115 12.3b 12:dc000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f685d500 con 0x5626f66f0400 2023-09-23T03:24:42.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.818+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.2 v2:172.21.15.154:6822/2884407243 17 ==== osd_op_reply(115 [pgnls-filter start_epoch 88 out=49b] v86'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f66f0400 2023-09-23T03:24:42.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.818+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:116 12.7 12:e0000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6855500 con 0x5626f67f7c00 2023-09-23T03:24:42.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.819+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.1 v2:172.21.15.154:6816/1267710005 14 ==== osd_op_reply(116 [pgnls-filter start_epoch 88 out=81b] v88'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5626f724cb40 con 0x5626f67f7c00 2023-09-23T03:24:42.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.819+0000 7fa6474bb5c0 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2023-09-23T03:24:42.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.819+0000 7fa6474bb5c0 10 datascan.operator(): handling object 10000000001.0 2023-09-23T03:24:42.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.819+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:117 12.7 12:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681a700 con 0x5626f67f7c00 2023-09-23T03:24:42.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.821+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.1 v2:172.21.15.154:6816/1267710005 15 ==== osd_op_reply(117 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (103) out=103b,getxattr,getxattr] v0'0 uv9 ondisk = 0) v8 ==== 416+0+149 (crc 0 0 0) 0x5626f724cb40 con 0x5626f67f7c00 2023-09-23T03:24:42.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.821+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:118 13.7 13:e2e07930:::10000000001.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f685c000 con 0x5626f67f7000 2023-09-23T03:24:42.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.822+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.7 v2:172.21.15.190:6824/3460136272 11 ==== osd_op_reply(118 10000000001.00000000 [getxattr (30) out=30b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x5626f724cb40 con 0x5626f67f7000 2023-09-23T03:24:42.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.822+0000 7fa6474bb5c0 20 datascan.operator(): loaded layout from xattr: pi: 13 os: 4194304 sc: 1 su: 4194304 2023-09-23T03:24:42.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.822+0000 7fa6474bb5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2023-09-23T03:24:42.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.822+0000 7fa6474bb5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/sixmegs 2023-09-23T03:24:42.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.822+0000 7fa6474bb5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=sixmegs 2023-09-23T03:24:42.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.822+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:119 11.10 11:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f685ca80 con 0x5626f67f7400 2023-09-23T03:24:42.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.823+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.0 v2:172.21.15.154:6800/2920938138 25 ==== osd_op_reply(119 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x5626f724cd80 con 0x5626f67f7400 2023-09-23T03:24:42.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.823+0000 7fa6474bb5c0 4 datascan.get_frag_of: Unexpected error on '10000000000.00000000': (2) No such file or directory 2023-09-23T03:24:42.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.823+0000 7fa6474bb5c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/sixmegs, will insert to root 2023-09-23T03:24:42.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.823+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:120 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681b500 con 0x5626f67f7400 2023-09-23T03:24:42.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.823+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.0 v2:172.21.15.154:6800/2920938138 26 ==== osd_op_reply(120 10000000000.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x5626f724cd80 con 0x5626f67f7400 2023-09-23T03:24:42.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.823+0000 7fa6474bb5c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-09-23T03:24:42.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.824+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:121 11.10 11:0cf6ad0f:::10000000000.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681bc00 con 0x5626f67f7400 2023-09-23T03:24:42.828 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.827+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.0 v2:172.21.15.154:6800/2920938138 27 ==== osd_op_reply(121 10000000000.00000000 [create,omap-set-header] v88'16 uv16 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5626f724cd80 con 0x5626f67f7400 2023-09-23T03:24:42.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.827+0000 7fa6474bb5c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x10000000000 2023-09-23T03:24:42.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.827+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:122 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f681b880 con 0x5626f67f7400 2023-09-23T03:24:42.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.828+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.0 v2:172.21.15.154:6800/2920938138 28 ==== osd_op_reply(122 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5626f724cd80 con 0x5626f67f7400 2023-09-23T03:24:42.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.828+0000 7fa6474bb5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=sixmegs frag=*, r=0 2023-09-23T03:24:42.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.828+0000 7fa6474bb5c0 20 datascan.read_dentry: sixmegs_head not found in result 2023-09-23T03:24:42.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.828+0000 7fa6474bb5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/sixmegs with size=6291456 bytes 2023-09-23T03:24:42.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.828+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:123 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=508b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f685ce00 con 0x5626f67f7400 2023-09-23T03:24:42.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.832+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.0 v2:172.21.15.154:6800/2920938138 29 ==== osd_op_reply(123 10000000000.00000000 [omap-set-vals] v88'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5626f724cd80 con 0x5626f67f7400 2023-09-23T03:24:42.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.832+0000 7fa6474bb5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/sixmegs pointing to 0x0x10000000001 2023-09-23T03:24:42.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.832+0000 7fa6474bb5c0 10 datascan.inject_with_backtrace: backptr: 0x0x1/subdir 2023-09-23T03:24:42.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.832+0000 7fa6474bb5c0 20 datascan.get_frag_of: dirino=0x1 target_dname=subdir 2023-09-23T03:24:42.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.832+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:124 11.1f 11:ff5b34d6:::1.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f685c700 con 0x5626f66f0400 2023-09-23T03:24:42.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.832+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.2 v2:172.21.15.154:6822/2884407243 18 ==== osd_op_reply(124 1.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x5626f724cd80 con 0x5626f66f0400 2023-09-23T03:24:42.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.832+0000 7fa6474bb5c0 4 datascan.get_frag_of: Unexpected error on '1.00000000': (2) No such file or directory 2023-09-23T03:24:42.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.832+0000 7fa6474bb5c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x1/subdir, will insert to root 2023-09-23T03:24:42.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.833+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:125 11.1f 11:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6868000 con 0x5626f66f0400 2023-09-23T03:24:42.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.833+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.2 v2:172.21.15.154:6822/2884407243 19 ==== osd_op_reply(125 1.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x5626f724cd80 con 0x5626f66f0400 2023-09-23T03:24:42.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.833+0000 7fa6474bb5c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-09-23T03:24:42.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.833+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:126 11.1f 11:ff5b34d6:::1.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6868380 con 0x5626f66f0400 2023-09-23T03:24:42.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.836+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.2 v2:172.21.15.154:6822/2884407243 20 ==== osd_op_reply(126 1.00000000 [create,omap-set-header] v88'11 uv11 ondisk = 0) v8 ==== 196+0+0 (crc 0 0 0) 0x5626f724cd80 con 0x5626f66f0400 2023-09-23T03:24:42.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.836+0000 7fa6474bb5c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x1 2023-09-23T03:24:42.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.836+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:127 11.1f 11:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6868700 con 0x5626f66f0400 2023-09-23T03:24:42.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.837+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.2 v2:172.21.15.154:6822/2884407243 21 ==== osd_op_reply(127 1.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv11 ondisk = 0) v8 ==== 154+0+4 (crc 0 0 0) 0x5626f724cd80 con 0x5626f66f0400 2023-09-23T03:24:42.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.837+0000 7fa6474bb5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:24:42.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.837+0000 7fa6474bb5c0 20 datascan.read_dentry: subdir_head not found in result 2023-09-23T03:24:42.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.837+0000 7fa6474bb5c0 10 datascan.inject_with_backtrace: Linking ancestor directory of inode 0x0x10000000000 at 0x0x1:subdir 2023-09-23T03:24:42.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.837+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:128 11.1f 11:ff5b34d6:::1.00000000:head [omap-set-vals in=507b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6868a80 con 0x5626f66f0400 2023-09-23T03:24:42.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.841+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.2 v2:172.21.15.154:6822/2884407243 22 ==== osd_op_reply(128 1.00000000 [omap-set-vals] v88'12 uv12 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x5626f724cd80 con 0x5626f66f0400 2023-09-23T03:24:42.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.841+0000 7fa6474bb5c0 20 datascan.inject_linkage: Injected dentry 0x0x1/subdir pointing to 0x0x10000000000 2023-09-23T03:24:42.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.841+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:129 12.27 12:e4000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6868e00 con 0x5626f67f7400 2023-09-23T03:24:42.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.842+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.0 v2:172.21.15.154:6800/2920938138 30 ==== osd_op_reply(129 [pgnls-filter start_epoch 88 out=49b] v86'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f67f7400 2023-09-23T03:24:42.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.842+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:130 12.17 12:e8000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6855180 con 0x5626f66f0800 2023-09-23T03:24:42.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.843+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.6 v2:172.21.15.190:6816/1964294001 14 ==== osd_op_reply(130 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cb40 con 0x5626f66f0800 2023-09-23T03:24:42.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.843+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:131 12.37 12:ec000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f685d880 con 0x5626f66f0000 2023-09-23T03:24:42.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.844+0000 7fa645c61640 1 -- 172.21.15.154:0/3134866139 <== osd.4 v2:172.21.15.190:6808/717302641 15 ==== osd_op_reply(131 [pgnls-filter start_epoch 88 out=49b] v86'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cfc0 con 0x5626f66f0000 2023-09-23T03:24:42.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.844+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:132 12.f 12:f0000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6855c00 con 0x5626f66f0800 2023-09-23T03:24:42.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.845+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.6 v2:172.21.15.190:6816/1964294001 15 ==== osd_op_reply(132 [pgnls-filter start_epoch 88 out=49b] v86'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cb40 con 0x5626f66f0800 2023-09-23T03:24:42.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.845+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:133 12.2f 12:f4000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f685dc00 con 0x5626f66f0400 2023-09-23T03:24:42.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.846+0000 7fa646462640 1 -- 172.21.15.154:0/3134866139 <== osd.2 v2:172.21.15.154:6822/2884407243 23 ==== osd_op_reply(133 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cd80 con 0x5626f66f0400 2023-09-23T03:24:42.848 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.846+0000 7fa640c57640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:134 12.1f 12:f8000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f6854380 con 0x5626f66f0800 2023-09-23T03:24:42.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.847+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.6 v2:172.21.15.190:6816/1964294001 16 ==== osd_op_reply(134 [pgnls-filter start_epoch 88 out=49b] v86'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cb40 con 0x5626f66f0800 2023-09-23T03:24:42.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.848+0000 7fa641458640 1 -- 172.21.15.154:0/3134866139 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:135 12.3f 12:fc000000::::head [pgnls-filter start_epoch 88 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x5626f685c380 con 0x5626f67f7000 2023-09-23T03:24:42.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.849+0000 7fa645460640 1 -- 172.21.15.154:0/3134866139 <== osd.7 v2:172.21.15.190:6824/3460136272 12 ==== osd_op_reply(135 [pgnls-filter start_epoch 88 out=49b] v86'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5626f724cb40 con 0x5626f67f7000 2023-09-23T03:24:42.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.849+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/3134866139 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5626f67f7400 msgr2=0x5626f682a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.849+0000 7fa6474bb5c0 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5626f67f7400 0x5626f682a580 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.849+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/3134866139 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5626f67f7c00 msgr2=0x5626f682b080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.849+0000 7fa6474bb5c0 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5626f67f7c00 0x5626f682b080 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.849+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/3134866139 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5626f66f0400 msgr2=0x5626f641a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.849+0000 7fa6474bb5c0 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5626f66f0400 0x5626f641a100 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.849+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/3134866139 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5626f67f7800 msgr2=0x5626f682ab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.849+0000 7fa6474bb5c0 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5626f67f7800 0x5626f682ab00 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.849+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/3134866139 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5626f66f0000 msgr2=0x5626f6419b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.849+0000 7fa6474bb5c0 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5626f66f0000 0x5626f6419b80 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.849+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/3134866139 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x5626f6840000 msgr2=0x5626f682b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.849+0000 7fa6474bb5c0 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x5626f6840000 0x5626f682b600 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.849+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/3134866139 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x5626f66f0800 msgr2=0x5626f6419600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.849+0000 7fa6474bb5c0 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x5626f66f0800 0x5626f6419600 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.849+0000 7fa645c61640 1 -- 172.21.15.154:0/3134866139 reap_dead start 2023-09-23T03:24:42.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.849+0000 7fa645c61640 1 -- 172.21.15.154:0/3134866139 reap_dead start 2023-09-23T03:24:42.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.849+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/3134866139 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x5626f67f7000 msgr2=0x5626f682a000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.849+0000 7fa6474bb5c0 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x5626f67f7000 0x5626f682a000 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.849+0000 7fa645c61640 1 -- 172.21.15.154:0/3134866139 reap_dead start 2023-09-23T03:24:42.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.849+0000 7fa645c61640 1 -- 172.21.15.154:0/3134866139 reap_dead start 2023-09-23T03:24:42.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.850+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/3134866139 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5626f67f6000 msgr2=0x5626f641b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.850+0000 7fa6474bb5c0 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5626f67f6000 0x5626f641b700 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x5626f64a06f0 tx=0x5626f6800060 comp rx=0 tx=0).stop 2023-09-23T03:24:42.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.850+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/3134866139 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626f66f0c00 msgr2=0x5626f641a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.850+0000 7fa6474bb5c0 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626f66f0c00 0x5626f641a680 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x5626f669b740 tx=0x5626f66a2d20 comp rx=0 tx=0).stop 2023-09-23T03:24:42.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.850+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/3134866139 shutdown_connections 2023-09-23T03:24:42.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.850+0000 7fa6474bb5c0 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5626f67f6000 0x5626f641b700 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.850+0000 7fa6474bb5c0 1 --2- 172.21.15.154:0/3134866139 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626f66f0c00 0x5626f641a680 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.850+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/3134866139 >> 172.21.15.154:0/3134866139 conn(0x5626f7185c00 msgr2=0x5626f7129000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:42.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.851+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/3134866139 shutdown_connections 2023-09-23T03:24:42.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.851+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/3134866139 wait complete. 2023-09-23T03:24:42.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.852+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/1470646990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626f63afc00 msgr2=0x5626f6419080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.852+0000 7fa6474bb5c0 1 --2- 172.21.15.154:0/1470646990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626f63afc00 0x5626f6419080 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x5626f71b4a50 tx=0x5626f6688120 comp rx=0 tx=0).stop 2023-09-23T03:24:42.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.852+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/1470646990 shutdown_connections 2023-09-23T03:24:42.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.852+0000 7fa6474bb5c0 1 --2- 172.21.15.154:0/1470646990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5626f63afc00 0x5626f6419080 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.852+0000 7fa6474bb5c0 1 --2- 172.21.15.154:0/1470646990 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5626f63af800 0x5626f6418b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.852+0000 7fa6474bb5c0 1 --2- 172.21.15.154:0/1470646990 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5626f63aec00 0x5626f6418580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.852+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/1470646990 >> 172.21.15.154:0/1470646990 conn(0x5626f63af000 msgr2=0x5626f7128800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:42.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.852+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/1470646990 shutdown_connections 2023-09-23T03:24:42.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.852+0000 7fa6474bb5c0 1 -- 172.21.15.154:0/1470646990 wait complete. 2023-09-23T03:24:42.864 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes'] in time 0:00:00.261002, result: 2023-09-23T03:24:42.864 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2023-09-23T03:24:42.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.967+0000 7f3c9b8495c0 1 -- start start 2023-09-23T03:24:42.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.967+0000 7f3c9b8495c0 1 Processor -- start 2023-09-23T03:24:42.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.968+0000 7f3c9b8495c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x561377c5c800 0x561376ef5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.968+0000 7f3c9b8495c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561376e8bc00 0x561376ef4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.968+0000 7f3c9b8495c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561377c5dc00 0x561376ef5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.968+0000 7f3c9b8495c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x561376ed5180 con 0x561377c5dc00 2023-09-23T03:24:42.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.968+0000 7f3c9b8495c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561376ed4140 con 0x561376e8bc00 2023-09-23T03:24:42.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.968+0000 7f3c9b8495c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x561376ed4b40 con 0x561377c5c800 2023-09-23T03:24:42.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.968+0000 7f3c99f78640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561377c5dc00 0x561376ef5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.968+0000 7f3c99f78640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561377c5dc00 0x561376ef5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43410/0 (socket says 172.21.15.154:43410) 2023-09-23T03:24:42.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.968+0000 7f3c99f78640 1 -- 172.21.15.154:0/2164676632 learned_addr learned my addr 172.21.15.154:0/2164676632 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:42.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.969+0000 7f3c99777640 1 --2- 172.21.15.154:0/2164676632 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x561377c5c800 0x561376ef5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.969+0000 7f3c99f78640 1 -- 172.21.15.154:0/2164676632 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x561377c5c800 msgr2=0x561376ef5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:42.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.969+0000 7f3c99f78640 1 --2- 172.21.15.154:0/2164676632 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x561377c5c800 0x561376ef5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.969+0000 7f3c99f78640 1 -- 172.21.15.154:0/2164676632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561376e8bc00 msgr2=0x561376ef4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:42.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.969+0000 7f3c98f76640 1 --2- 172.21.15.154:0/2164676632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561376e8bc00 0x561376ef4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.969+0000 7f3c99f78640 1 --2- 172.21.15.154:0/2164676632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561376e8bc00 0x561376ef4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.969+0000 7f3c99f78640 1 -- 172.21.15.154:0/2164676632 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561376ebeea0 con 0x561377c5dc00 2023-09-23T03:24:42.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.969+0000 7f3c99f78640 1 --2- 172.21.15.154:0/2164676632 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561377c5dc00 0x561376ef5600 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x561377bde7b0 tx=0x561377d3e120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb51da8b88815c28 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.969+0000 7f3c96f72640 1 -- 172.21.15.154:0/2164676632 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561376ed31e0 con 0x561377c5dc00 2023-09-23T03:24:42.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.969+0000 7f3c96f72640 1 -- 172.21.15.154:0/2164676632 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561376eb0a80 con 0x561377c5dc00 2023-09-23T03:24:42.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.969+0000 7f3c96f72640 1 -- 172.21.15.154:0/2164676632 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561376ed3600 con 0x561377c5dc00 2023-09-23T03:24:42.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.969+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/2164676632 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561376ebe000 con 0x561377c5dc00 2023-09-23T03:24:42.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.969+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/2164676632 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x561376ebe9c0 con 0x561377c5dc00 2023-09-23T03:24:42.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.969+0000 7f3c9b8495c0 4 waiting for MDS map... 2023-09-23T03:24:42.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.970+0000 7f3c96f72640 1 -- 172.21.15.154:0/2164676632 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(88..88 src has 1..88) v4 ==== 6605+0+0 (secure 0 0 0) 0x561376f4ac40 con 0x561377c5dc00 2023-09-23T03:24:42.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.970+0000 7f3c96f72640 1 -- 172.21.15.154:0/2164676632 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 69) v1 ==== 3306+0+0 (secure 0 0 0) 0x561377d52000 con 0x561377c5dc00 2023-09-23T03:24:42.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.970+0000 7f3c9b8495c0 4 Got MDS map 69 2023-09-23T03:24:42.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.970+0000 7f3c9b8495c0 4 datascan.main: Using metadata pool output 2023-09-23T03:24:42.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.970+0000 7f3c9b8495c0 4 datascan.main: connecting to RADOS... 2023-09-23T03:24:42.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.971+0000 7f3c9b8495c0 1 Processor -- start 2023-09-23T03:24:42.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.971+0000 7f3c9b8495c0 1 -- start start 2023-09-23T03:24:42.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.972+0000 7f3c9b8495c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561377c5c000 0x561376ef5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.972+0000 7f3c9b8495c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x561376e8b800 0x561376ef6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.972+0000 7f3c9b8495c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561377daa000 0x561376ef6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.972+0000 7f3c9b8495c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x561376ed4a00 con 0x561377daa000 2023-09-23T03:24:42.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.972+0000 7f3c9b8495c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561376ed4000 con 0x561377c5c000 2023-09-23T03:24:42.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.972+0000 7f3c9b8495c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x561376f59180 con 0x561376e8b800 2023-09-23T03:24:42.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.972+0000 7f3c98f76640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561377daa000 0x561376ef6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.972+0000 7f3c98f76640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561377daa000 0x561376ef6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43416/0 (socket says 172.21.15.154:43416) 2023-09-23T03:24:42.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.972+0000 7f3c98f76640 1 -- 172.21.15.154:0/986143245 learned_addr learned my addr 172.21.15.154:0/986143245 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:42.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.972+0000 7f3c99777640 1 --2- 172.21.15.154:0/986143245 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x561376e8b800 0x561376ef6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.972+0000 7f3c98f76640 1 -- 172.21.15.154:0/986143245 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x561376e8b800 msgr2=0x561376ef6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:42.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.972+0000 7f3c98f76640 1 --2- 172.21.15.154:0/986143245 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x561376e8b800 0x561376ef6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.972+0000 7f3c98f76640 1 -- 172.21.15.154:0/986143245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561377c5c000 msgr2=0x561376ef5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:42.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.972+0000 7f3c98f76640 1 --2- 172.21.15.154:0/986143245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561377c5c000 0x561376ef5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.973+0000 7f3c98f76640 1 -- 172.21.15.154:0/986143245 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561376ebeea0 con 0x561377daa000 2023-09-23T03:24:42.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.973+0000 7f3c99777640 1 --2- 172.21.15.154:0/986143245 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x561376e8b800 0x561376ef6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:42.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.973+0000 7f3c98f76640 1 --2- 172.21.15.154:0/986143245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561377daa000 0x561376ef6680 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x561377d605d0 tx=0x561377d62cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c8f3aa8aca6da6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.973+0000 7f3c94f6e640 1 -- 172.21.15.154:0/986143245 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561376ed31e0 con 0x561377daa000 2023-09-23T03:24:42.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.973+0000 7f3c94f6e640 1 -- 172.21.15.154:0/986143245 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561377d4ea80 con 0x561377daa000 2023-09-23T03:24:42.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.973+0000 7f3c94f6e640 1 -- 172.21.15.154:0/986143245 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561376ed3600 con 0x561377daa000 2023-09-23T03:24:42.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.974+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/986143245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561377daa000 msgr2=0x561376ef6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.974+0000 7f3c9b8495c0 1 --2- 172.21.15.154:0/986143245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561377daa000 0x561376ef6680 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x561377d605d0 tx=0x561377d62cc0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.974+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/986143245 shutdown_connections 2023-09-23T03:24:42.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.974+0000 7f3c9b8495c0 1 --2- 172.21.15.154:0/986143245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561377daa000 0x561376ef6680 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.974+0000 7f3c9b8495c0 1 --2- 172.21.15.154:0/986143245 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x561376e8b800 0x561376ef6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.974+0000 7f3c9b8495c0 1 --2- 172.21.15.154:0/986143245 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561377c5c000 0x561376ef5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.974+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/986143245 >> 172.21.15.154:0/986143245 conn(0x561377c5c400 msgr2=0x561377c15800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:42.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.974+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/986143245 shutdown_connections 2023-09-23T03:24:42.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.974+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/986143245 wait complete. 2023-09-23T03:24:42.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.975+0000 7f3c9b8495c0 1 Processor -- start 2023-09-23T03:24:42.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.975+0000 7f3c9b8495c0 1 -- start start 2023-09-23T03:24:42.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.975+0000 7f3c9b8495c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x561376e8b800 0x561376ef7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.975+0000 7f3c9b8495c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561377c5c000 0x561376ef7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.975+0000 7f3c9b8495c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561377dab400 0x561376ef6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.976+0000 7f3c9b8495c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x561376f58c80 con 0x561377c5c000 2023-09-23T03:24:42.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.976+0000 7f3c9b8495c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x561376f58b40 con 0x561377dab400 2023-09-23T03:24:42.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.976+0000 7f3c9b8495c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x561376f58a00 con 0x561376e8b800 2023-09-23T03:24:42.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.976+0000 7f3c99777640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561377c5c000 0x561376ef7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.976+0000 7f3c99777640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561377c5c000 0x561376ef7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43422/0 (socket says 172.21.15.154:43422) 2023-09-23T03:24:42.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.976+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 learned_addr learned my addr 172.21.15.154:0/1220270411 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:42.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.976+0000 7f3c98f76640 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x561376e8b800 0x561376ef7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.976+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x561376e8b800 msgr2=0x561376ef7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.976+0000 7f3c99777640 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x561376e8b800 0x561376ef7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.976+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561377dab400 msgr2=0x561376ef6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:42.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.976+0000 7f3c99777640 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561377dab400 0x561376ef6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:42.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.976+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561376ebf520 con 0x561377c5c000 2023-09-23T03:24:42.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.977+0000 7f3c99777640 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561377c5c000 0x561376ef7180 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x561377d618c0 tx=0x561377d62de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.977+0000 7f3c9376b640 1 -- 172.21.15.154:0/1220270411 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561376ed31e0 con 0x561377c5c000 2023-09-23T03:24:42.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.977+0000 7f3c9376b640 1 -- 172.21.15.154:0/1220270411 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561377d4f500 con 0x561377c5c000 2023-09-23T03:24:42.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.977+0000 7f3c9376b640 1 -- 172.21.15.154:0/1220270411 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561376ed3600 con 0x561377c5c000 2023-09-23T03:24:42.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.977+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561376ebf6c0 con 0x561377c5c000 2023-09-23T03:24:42.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.977+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561376ebf860 con 0x561377c5c000 2023-09-23T03:24:42.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.977+0000 7f3c9b8495c0 4 datascan.init: resolving metadata pool 11 2023-09-23T03:24:42.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.979+0000 7f3c9376b640 1 -- 172.21.15.154:0/1220270411 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x561376ebc840 con 0x561377c5c000 2023-09-23T03:24:42.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.979+0000 7f3c9376b640 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x561377dc6c00 0x5613771e4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.979+0000 7f3c9376b640 1 -- 172.21.15.154:0/1220270411 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6605+0+0 (secure 0 0 0) 0x561376f4aa80 con 0x561377c5c000 2023-09-23T03:24:42.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.979+0000 7f3c9b8495c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-09-23T03:24:42.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.979+0000 7f3c9b8495c0 4 datascan.main: resolving metadata pool 11 2023-09-23T03:24:42.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.979+0000 7f3c9b8495c0 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x561377dab000 0x561376ef6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.979+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:1 11.0 11:00000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561376ee5880 con 0x561377dab000 2023-09-23T03:24:42.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.980+0000 7f3c98f76640 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x561377dc6c00 0x5613771e4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.980+0000 7f3c98f76640 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x561377dab000 0x561376ef6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.980+0000 7f3c98f76640 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x561377dc6c00 0x5613771e4000 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x561377d60720 tx=0x561377d62c60 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.980+0000 7f3c98f76640 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x561377dab000 0x561376ef6680 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.981+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 <== osd.1 v2:172.21.15.154:6816/1267710005 1 ==== osd_op_reply(1 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38b40 con 0x561377dab000 2023-09-23T03:24:42.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.981+0000 7f3c98f76640 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x561377dc7000 0x5613771e4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.981+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:2 11.10 11:08000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561376ee5880 con 0x561377dc7000 2023-09-23T03:24:42.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.981+0000 7f3c99f78640 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x561377dc7000 0x5613771e4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.982+0000 7f3c99f78640 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x561377dc7000 0x5613771e4580 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.982+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 <== osd.0 v2:172.21.15.154:6800/2920938138 1 ==== osd_op_reply(2 [pgnls start_epoch 88 out=81b] v88'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561377d38d80 con 0x561377dc7000 2023-09-23T03:24:42.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.982+0000 7f3c99f78640 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x561377dc7c00 0x561376ef4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.982+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:3 11.8 11:10000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561376ee5880 con 0x561377dc7c00 2023-09-23T03:24:42.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.982+0000 7f3c99777640 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x561377dc7c00 0x561376ef4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.983+0000 7f3c99777640 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x561377dc7c00 0x561376ef4580 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.983+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(3 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38fc0 con 0x561377dc7c00 2023-09-23T03:24:42.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.983+0000 7f3c99777640 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5613771fe000 0x5613771e4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.983+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:4 11.18 11:18000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561376ee5880 con 0x5613771fe000 2023-09-23T03:24:42.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.984+0000 7f3c98f76640 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5613771fe000 0x5613771e4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.984+0000 7f3c98f76640 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5613771fe000 0x5613771e4b00 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.984+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 <== osd.2 v2:172.21.15.154:6822/2884407243 1 ==== osd_op_reply(4 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38b40 con 0x5613771fe000 2023-09-23T03:24:42.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.984+0000 7f3c98f76640 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x561377dc7400 0x5613771e5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.984+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:5 11.4 11:20000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561376ee5880 con 0x561377dc7400 2023-09-23T03:24:42.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.985+0000 7f3c99f78640 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x561377dc7400 0x5613771e5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.986+0000 7f3c99f78640 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x561377dc7400 0x5613771e5080 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.992 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.987+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 <== osd.7 v2:172.21.15.190:6824/3460136272 1 ==== osd_op_reply(5 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38d80 con 0x561377dc7400 2023-09-23T03:24:42.992 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.987+0000 7f3c99f78640 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x5613771fec00 0x5613771e5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.992 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.987+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:6 11.14 11:28000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561376ee5880 con 0x5613771fec00 2023-09-23T03:24:42.992 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.987+0000 7f3c99777640 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x5613771fec00 0x5613771e5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.988+0000 7f3c99777640 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x5613771fec00 0x5613771e5b80 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.989+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== osd_op_reply(6 [pgnls start_epoch 88 out=125b] v88'30 uv30 ondisk = 1) v8 ==== 144+0+125 (crc 0 0 0) 0x561377d38fc0 con 0x5613771fec00 2023-09-23T03:24:42.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.989+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:7 11.c 11:30000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561376ee5880 con 0x561377dab000 2023-09-23T03:24:42.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.989+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 <== osd.1 v2:172.21.15.154:6816/1267710005 2 ==== osd_op_reply(7 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38b40 con 0x561377dab000 2023-09-23T03:24:42.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.989+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:8 11.1c 11:38000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561376ee5880 con 0x5613771fec00 2023-09-23T03:24:42.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.990+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 <== osd.6 v2:172.21.15.190:6816/1964294001 2 ==== osd_op_reply(8 [pgnls start_epoch 88 out=49b] v87'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38fc0 con 0x5613771fec00 2023-09-23T03:24:42.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.990+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:9 11.2 11:40000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561376ee5880 con 0x5613771fe000 2023-09-23T03:24:42.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.990+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 <== osd.2 v2:172.21.15.154:6822/2884407243 2 ==== osd_op_reply(9 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38b40 con 0x5613771fe000 2023-09-23T03:24:42.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.990+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:10 11.12 11:48000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561376ee5880 con 0x561377dc7400 2023-09-23T03:24:42.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.991+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 <== osd.7 v2:172.21.15.190:6824/3460136272 2 ==== osd_op_reply(10 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38d80 con 0x561377dc7400 2023-09-23T03:24:42.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.991+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:11 11.a 11:50000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561376ee5880 con 0x561377dc7c00 2023-09-23T03:24:42.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.991+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 <== osd.3 v2:172.21.15.154:6808/1133688468 2 ==== osd_op_reply(11 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38fc0 con 0x561377dc7c00 2023-09-23T03:24:42.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.991+0000 7f3c99777640 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x5613771fe400 0x5613771e6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.992+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:12 11.1a 11:58000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561376ee5880 con 0x5613771fe400 2023-09-23T03:24:42.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.992+0000 7f3c98f76640 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x5613771fe400 0x5613771e6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.993+0000 7f3c98f76640 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x5613771fe400 0x5613771e6680 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.994+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 <== osd.5 v2:172.21.15.190:6800/882536885 1 ==== osd_op_reply(12 [pgnls start_epoch 88 out=49b] v88'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38b40 con 0x5613771fe400 2023-09-23T03:24:42.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.994+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:13 11.6 11:60000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561376ee5880 con 0x561377dc7000 2023-09-23T03:24:42.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.994+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 <== osd.0 v2:172.21.15.154:6800/2920938138 2 ==== osd_op_reply(13 [pgnls start_epoch 88 out=49b] v87'6 uv6 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38d80 con 0x561377dc7000 2023-09-23T03:24:42.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.994+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:14 11.16 11:68000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561376ee5880 con 0x5613771fe400 2023-09-23T03:24:42.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.995+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 <== osd.5 v2:172.21.15.190:6800/882536885 2 ==== osd_op_reply(14 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38b40 con 0x5613771fe400 2023-09-23T03:24:42.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.995+0000 7f3c98f76640 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x561377dc7800 0x5613771e5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:42.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.995+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:15 11.e 11:70000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561376ee5880 con 0x561377dc7800 2023-09-23T03:24:42.997 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.996+0000 7f3c99f78640 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x561377dc7800 0x5613771e5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:42.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.997+0000 7f3c99f78640 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x561377dc7800 0x5613771e5600 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:42.999 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.998+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 <== osd.4 v2:172.21.15.190:6808/717302641 1 ==== osd_op_reply(15 [pgnls start_epoch 88 out=49b] v87'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38d80 con 0x561377dc7800 2023-09-23T03:24:42.999 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.998+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:16 11.1e 11:78000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561376ee5880 con 0x561377dc7400 2023-09-23T03:24:43.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.999+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 <== osd.7 v2:172.21.15.190:6824/3460136272 3 ==== osd_op_reply(16 [pgnls start_epoch 88 out=49b] v88'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38d80 con 0x561377dc7400 2023-09-23T03:24:43.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:42.999+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:17 11.1 11:80000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561376ee5880 con 0x5613771fec00 2023-09-23T03:24:43.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.000+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 <== osd.6 v2:172.21.15.190:6816/1964294001 3 ==== osd_op_reply(17 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38fc0 con 0x5613771fec00 2023-09-23T03:24:43.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.000+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:18 11.11 11:88000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561376ee5880 con 0x5613771fe000 2023-09-23T03:24:43.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.000+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 <== osd.2 v2:172.21.15.154:6822/2884407243 3 ==== osd_op_reply(18 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38b40 con 0x5613771fe000 2023-09-23T03:24:43.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.000+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:19 11.9 11:90000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561376ee5880 con 0x561377dc7c00 2023-09-23T03:24:43.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.001+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 <== osd.3 v2:172.21.15.154:6808/1133688468 3 ==== osd_op_reply(19 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38fc0 con 0x561377dc7c00 2023-09-23T03:24:43.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.001+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:20 11.19 11:98000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561376ee5880 con 0x561377dc7000 2023-09-23T03:24:43.003 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.001+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 <== osd.0 v2:172.21.15.154:6800/2920938138 3 ==== osd_op_reply(20 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38d80 con 0x561377dc7000 2023-09-23T03:24:43.003 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.001+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:21 11.5 11:a0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561376ee5880 con 0x561377dc7400 2023-09-23T03:24:43.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.002+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 <== osd.7 v2:172.21.15.190:6824/3460136272 4 ==== osd_op_reply(21 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38d80 con 0x561377dc7400 2023-09-23T03:24:43.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.002+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:22 11.15 11:a8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561376ee5880 con 0x5613771fe000 2023-09-23T03:24:43.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.003+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 <== osd.2 v2:172.21.15.154:6822/2884407243 4 ==== osd_op_reply(22 [pgnls start_epoch 88 out=73b] v87'6 uv6 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x561377d38b40 con 0x5613771fe000 2023-09-23T03:24:43.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.003+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:23 11.d 11:b0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561376ee5880 con 0x5613771fe000 2023-09-23T03:24:43.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.003+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 <== osd.2 v2:172.21.15.154:6822/2884407243 5 ==== osd_op_reply(23 [pgnls start_epoch 88 out=74b] v88'3 uv3 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x561377d38b40 con 0x5613771fe000 2023-09-23T03:24:43.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.003+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:24 11.1d 11:b8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561376ee5880 con 0x561377dc7400 2023-09-23T03:24:43.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.004+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 <== osd.7 v2:172.21.15.190:6824/3460136272 5 ==== osd_op_reply(24 [pgnls start_epoch 88 out=49b] v87'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38d80 con 0x561377dc7400 2023-09-23T03:24:43.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.004+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:25 11.3 11:c0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561376ee5880 con 0x5613771fec00 2023-09-23T03:24:43.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.005+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 <== osd.6 v2:172.21.15.190:6816/1964294001 4 ==== osd_op_reply(25 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38fc0 con 0x5613771fec00 2023-09-23T03:24:43.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.005+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:26 11.13 11:c8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561376ee5880 con 0x5613771fe400 2023-09-23T03:24:43.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.006+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 <== osd.5 v2:172.21.15.190:6800/882536885 3 ==== osd_op_reply(26 [pgnls start_epoch 88 out=98b] v88'7 uv7 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x561377d38b40 con 0x5613771fe400 2023-09-23T03:24:43.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.007+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:27 11.b 11:d0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561376ee5880 con 0x561377dc7c00 2023-09-23T03:24:43.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.007+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 <== osd.3 v2:172.21.15.154:6808/1133688468 4 ==== osd_op_reply(27 [pgnls start_epoch 88 out=76b] v88'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x561377d38fc0 con 0x561377dc7c00 2023-09-23T03:24:43.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.007+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:28 11.1b 11:d8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561376ee5880 con 0x561377dc7800 2023-09-23T03:24:43.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.008+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 <== osd.4 v2:172.21.15.190:6808/717302641 2 ==== osd_op_reply(28 [pgnls start_epoch 88 out=49b] v88'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38d80 con 0x561377dc7800 2023-09-23T03:24:43.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.008+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:29 11.7 11:e0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561376ee5880 con 0x5613771fe400 2023-09-23T03:24:43.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.009+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 <== osd.5 v2:172.21.15.190:6800/882536885 4 ==== osd_op_reply(29 [pgnls start_epoch 88 out=49b] v87'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38b40 con 0x5613771fe400 2023-09-23T03:24:43.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.009+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:30 11.17 11:e8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561376ee5880 con 0x561377dc7400 2023-09-23T03:24:43.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.010+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 <== osd.7 v2:172.21.15.190:6824/3460136272 6 ==== osd_op_reply(30 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38d80 con 0x561377dc7400 2023-09-23T03:24:43.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.010+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:31 11.f 11:f0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561376ee5880 con 0x561377dc7400 2023-09-23T03:24:43.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.011+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 <== osd.7 v2:172.21.15.190:6824/3460136272 7 ==== osd_op_reply(31 [pgnls start_epoch 88 out=73b] v88'4 uv4 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x561377d38d80 con 0x561377dc7400 2023-09-23T03:24:43.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.011+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:32 11.1f 11:f8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561376ee5880 con 0x5613771fe000 2023-09-23T03:24:43.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.011+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 <== osd.2 v2:172.21.15.154:6822/2884407243 6 ==== osd_op_reply(32 [pgnls start_epoch 88 out=101b] v88'12 uv12 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x561377d38b40 con 0x5613771fe000 2023-09-23T03:24:43.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.011+0000 7f3c9b8495c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2023-09-23T03:24:43.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.011+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:33 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x561376ee5c00 con 0x561377dc7000 2023-09-23T03:24:43.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.012+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 <== osd.0 v2:172.21.15.154:6800/2920938138 4 ==== osd_op_reply(33 10000000000.00000000 [omap-get-vals out=509b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+509 (crc 0 0 0) 0x561377d38d80 con 0x561377dc7000 2023-09-23T03:24:43.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.012+0000 7f3c9b8495c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:24:43.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.012+0000 7f3c9b8495c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2023-09-23T03:24:43.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.012+0000 7f3c9b8495c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-09-23T03:24:43.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.012+0000 7f3c9b8495c0 10 datascan.scan_links: step 1: handling object 200.00000000 2023-09-23T03:24:43.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.012+0000 7f3c9b8495c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-09-23T03:24:43.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.012+0000 7f3c9b8495c0 10 datascan.scan_links: step 1: handling object 200.00000001 2023-09-23T03:24:43.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.012+0000 7f3c9b8495c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-09-23T03:24:43.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.012+0000 7f3c9b8495c0 10 datascan.scan_links: step 1: handling object 500.00000000 2023-09-23T03:24:43.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.012+0000 7f3c9b8495c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-09-23T03:24:43.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.012+0000 7f3c9b8495c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2023-09-23T03:24:43.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.012+0000 7f3c9b8495c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-09-23T03:24:43.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.012+0000 7f3c9b8495c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2023-09-23T03:24:43.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.012+0000 7f3c9b8495c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-09-23T03:24:43.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.012+0000 7f3c9b8495c0 10 datascan.scan_links: step 1: handling object 100.00000000 2023-09-23T03:24:43.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.012+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:34 11.13 11:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210000 con 0x5613771fe400 2023-09-23T03:24:43.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.013+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 <== osd.5 v2:172.21.15.190:6800/882536885 5 ==== osd_op_reply(34 100.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x561377d38b40 con 0x5613771fe400 2023-09-23T03:24:43.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.014+0000 7f3c9b8495c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2023-09-23T03:24:43.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.014+0000 7f3c9b8495c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-09-23T03:24:43.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.014+0000 7f3c9b8495c0 10 datascan.scan_links: step 1: handling object 400.00000000 2023-09-23T03:24:43.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.014+0000 7f3c9b8495c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-09-23T03:24:43.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.014+0000 7f3c9b8495c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2023-09-23T03:24:43.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.014+0000 7f3c9b8495c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-09-23T03:24:43.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.014+0000 7f3c9b8495c0 10 datascan.scan_links: step 1: handling object 1.00000000 2023-09-23T03:24:43.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.014+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:35 11.1f 11:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210380 con 0x5613771fe000 2023-09-23T03:24:43.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.014+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 <== osd.2 v2:172.21.15.154:6822/2884407243 7 ==== osd_op_reply(35 1.00000000 [omap-get-vals out=508b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+508 (crc 0 0 0) 0x561377d38b40 con 0x5613771fe000 2023-09-23T03:24:43.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.014+0000 7f3c9b8495c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:24:43.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.014+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:36 11.0 11:00000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210700 con 0x561377dab000 2023-09-23T03:24:43.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.015+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 <== osd.1 v2:172.21.15.154:6816/1267710005 3 ==== osd_op_reply(36 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38b40 con 0x561377dab000 2023-09-23T03:24:43.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.015+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:37 11.10 11:08000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210700 con 0x561377dc7000 2023-09-23T03:24:43.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.015+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 <== osd.0 v2:172.21.15.154:6800/2920938138 5 ==== osd_op_reply(37 [pgnls start_epoch 88 out=81b] v88'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561377d38d80 con 0x561377dc7000 2023-09-23T03:24:43.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.015+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:38 11.8 11:10000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210700 con 0x561377dc7c00 2023-09-23T03:24:43.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.016+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 <== osd.3 v2:172.21.15.154:6808/1133688468 5 ==== osd_op_reply(38 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38fc0 con 0x561377dc7c00 2023-09-23T03:24:43.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.016+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:39 11.18 11:18000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210700 con 0x5613771fe000 2023-09-23T03:24:43.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.017+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 <== osd.2 v2:172.21.15.154:6822/2884407243 8 ==== osd_op_reply(39 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38b40 con 0x5613771fe000 2023-09-23T03:24:43.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.017+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:40 11.4 11:20000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210700 con 0x561377dc7400 2023-09-23T03:24:43.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.017+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 <== osd.7 v2:172.21.15.190:6824/3460136272 8 ==== osd_op_reply(40 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38d80 con 0x561377dc7400 2023-09-23T03:24:43.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.017+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:41 11.14 11:28000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210700 con 0x5613771fec00 2023-09-23T03:24:43.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.019+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 <== osd.6 v2:172.21.15.190:6816/1964294001 5 ==== osd_op_reply(41 [pgnls start_epoch 88 out=125b] v88'30 uv30 ondisk = 1) v8 ==== 144+0+125 (crc 0 0 0) 0x561377d38fc0 con 0x5613771fec00 2023-09-23T03:24:43.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.019+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:42 11.c 11:30000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210700 con 0x561377dab000 2023-09-23T03:24:43.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.019+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 <== osd.1 v2:172.21.15.154:6816/1267710005 4 ==== osd_op_reply(42 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38b40 con 0x561377dab000 2023-09-23T03:24:43.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.019+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:43 11.1c 11:38000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210700 con 0x5613771fec00 2023-09-23T03:24:43.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.020+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 <== osd.6 v2:172.21.15.190:6816/1964294001 6 ==== osd_op_reply(43 [pgnls start_epoch 88 out=49b] v87'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38fc0 con 0x5613771fec00 2023-09-23T03:24:43.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.020+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:44 11.2 11:40000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210700 con 0x5613771fe000 2023-09-23T03:24:43.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.021+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 <== osd.2 v2:172.21.15.154:6822/2884407243 9 ==== osd_op_reply(44 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38b40 con 0x5613771fe000 2023-09-23T03:24:43.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.021+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:45 11.12 11:48000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210700 con 0x561377dc7400 2023-09-23T03:24:43.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.022+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 <== osd.7 v2:172.21.15.190:6824/3460136272 9 ==== osd_op_reply(45 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38d80 con 0x561377dc7400 2023-09-23T03:24:43.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.022+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:46 11.a 11:50000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210700 con 0x561377dc7c00 2023-09-23T03:24:43.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.022+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 <== osd.3 v2:172.21.15.154:6808/1133688468 6 ==== osd_op_reply(46 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38fc0 con 0x561377dc7c00 2023-09-23T03:24:43.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.022+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:47 11.1a 11:58000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210700 con 0x5613771fe400 2023-09-23T03:24:43.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.023+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 <== osd.5 v2:172.21.15.190:6800/882536885 6 ==== osd_op_reply(47 [pgnls start_epoch 88 out=49b] v88'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38b40 con 0x5613771fe400 2023-09-23T03:24:43.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.023+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:48 11.6 11:60000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210700 con 0x561377dc7000 2023-09-23T03:24:43.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.024+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 <== osd.0 v2:172.21.15.154:6800/2920938138 6 ==== osd_op_reply(48 [pgnls start_epoch 88 out=49b] v87'6 uv6 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38d80 con 0x561377dc7000 2023-09-23T03:24:43.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.024+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:49 11.16 11:68000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210700 con 0x5613771fe400 2023-09-23T03:24:43.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.025+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 <== osd.5 v2:172.21.15.190:6800/882536885 7 ==== osd_op_reply(49 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38b40 con 0x5613771fe400 2023-09-23T03:24:43.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.025+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:50 11.e 11:70000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210700 con 0x561377dc7800 2023-09-23T03:24:43.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.026+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 <== osd.4 v2:172.21.15.190:6808/717302641 3 ==== osd_op_reply(50 [pgnls start_epoch 88 out=49b] v87'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38d80 con 0x561377dc7800 2023-09-23T03:24:43.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.026+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:51 11.1e 11:78000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210700 con 0x561377dc7400 2023-09-23T03:24:43.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.027+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 <== osd.7 v2:172.21.15.190:6824/3460136272 10 ==== osd_op_reply(51 [pgnls start_epoch 88 out=49b] v88'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38d80 con 0x561377dc7400 2023-09-23T03:24:43.029 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.027+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:52 11.1 11:80000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210700 con 0x5613771fec00 2023-09-23T03:24:43.030 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.028+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 <== osd.6 v2:172.21.15.190:6816/1964294001 7 ==== osd_op_reply(52 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38fc0 con 0x5613771fec00 2023-09-23T03:24:43.030 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.028+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:53 11.11 11:88000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210700 con 0x5613771fe000 2023-09-23T03:24:43.030 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.029+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 <== osd.2 v2:172.21.15.154:6822/2884407243 10 ==== osd_op_reply(53 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38b40 con 0x5613771fe000 2023-09-23T03:24:43.030 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.029+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:54 11.9 11:90000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210700 con 0x561377dc7c00 2023-09-23T03:24:43.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.030+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 <== osd.3 v2:172.21.15.154:6808/1133688468 7 ==== osd_op_reply(54 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38fc0 con 0x561377dc7c00 2023-09-23T03:24:43.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.030+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:55 11.19 11:98000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210700 con 0x561377dc7000 2023-09-23T03:24:43.031 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.030+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 <== osd.0 v2:172.21.15.154:6800/2920938138 7 ==== osd_op_reply(55 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38d80 con 0x561377dc7000 2023-09-23T03:24:43.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.030+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:56 11.5 11:a0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210700 con 0x561377dc7400 2023-09-23T03:24:43.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.032+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 <== osd.7 v2:172.21.15.190:6824/3460136272 11 ==== osd_op_reply(56 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38d80 con 0x561377dc7400 2023-09-23T03:24:43.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.032+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:57 11.15 11:a8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210700 con 0x5613771fe000 2023-09-23T03:24:43.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.032+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 <== osd.2 v2:172.21.15.154:6822/2884407243 11 ==== osd_op_reply(57 [pgnls start_epoch 88 out=73b] v87'6 uv6 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x561377d38b40 con 0x5613771fe000 2023-09-23T03:24:43.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.032+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:58 11.d 11:b0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210700 con 0x5613771fe000 2023-09-23T03:24:43.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.033+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 <== osd.2 v2:172.21.15.154:6822/2884407243 12 ==== osd_op_reply(58 [pgnls start_epoch 88 out=74b] v88'3 uv3 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x561377d38b40 con 0x5613771fe000 2023-09-23T03:24:43.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.033+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:59 11.1d 11:b8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210700 con 0x561377dc7400 2023-09-23T03:24:43.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.034+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 <== osd.7 v2:172.21.15.190:6824/3460136272 12 ==== osd_op_reply(59 [pgnls start_epoch 88 out=49b] v87'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38d80 con 0x561377dc7400 2023-09-23T03:24:43.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.034+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:60 11.3 11:c0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210700 con 0x5613771fec00 2023-09-23T03:24:43.036 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.035+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 <== osd.6 v2:172.21.15.190:6816/1964294001 8 ==== osd_op_reply(60 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38fc0 con 0x5613771fec00 2023-09-23T03:24:43.037 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.035+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:61 11.13 11:c8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210700 con 0x5613771fe400 2023-09-23T03:24:43.037 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.036+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 <== osd.5 v2:172.21.15.190:6800/882536885 8 ==== osd_op_reply(61 [pgnls start_epoch 88 out=98b] v88'7 uv7 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x561377d38b40 con 0x5613771fe400 2023-09-23T03:24:43.038 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.036+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:62 11.b 11:d0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210700 con 0x561377dc7c00 2023-09-23T03:24:43.038 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.037+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 <== osd.3 v2:172.21.15.154:6808/1133688468 8 ==== osd_op_reply(62 [pgnls start_epoch 88 out=76b] v88'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x561377d38fc0 con 0x561377dc7c00 2023-09-23T03:24:43.038 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.037+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:63 11.1b 11:d8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210700 con 0x561377dc7800 2023-09-23T03:24:43.039 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.038+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 <== osd.4 v2:172.21.15.190:6808/717302641 4 ==== osd_op_reply(63 [pgnls start_epoch 88 out=49b] v88'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38d80 con 0x561377dc7800 2023-09-23T03:24:43.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.038+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:64 11.7 11:e0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210700 con 0x5613771fe400 2023-09-23T03:24:43.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.039+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 <== osd.5 v2:172.21.15.190:6800/882536885 9 ==== osd_op_reply(64 [pgnls start_epoch 88 out=49b] v87'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38b40 con 0x5613771fe400 2023-09-23T03:24:43.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.039+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:65 11.17 11:e8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210700 con 0x561377dc7400 2023-09-23T03:24:43.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.040+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 <== osd.7 v2:172.21.15.190:6824/3460136272 13 ==== osd_op_reply(65 [pgnls start_epoch 88 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561377d38d80 con 0x561377dc7400 2023-09-23T03:24:43.042 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.040+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:66 11.f 11:f0000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210700 con 0x561377dc7400 2023-09-23T03:24:43.042 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.041+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 <== osd.7 v2:172.21.15.190:6824/3460136272 14 ==== osd_op_reply(66 [pgnls start_epoch 88 out=73b] v88'4 uv4 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x561377d38d80 con 0x561377dc7400 2023-09-23T03:24:43.042 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.041+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:67 11.1f 11:f8000000::::head [pgnls start_epoch 88 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210700 con 0x5613771fe000 2023-09-23T03:24:43.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.042+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 <== osd.2 v2:172.21.15.154:6822/2884407243 13 ==== osd_op_reply(67 [pgnls start_epoch 88 out=101b] v88'12 uv12 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x561377d38b40 con 0x5613771fe000 2023-09-23T03:24:43.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.042+0000 7f3c9b8495c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2023-09-23T03:24:43.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.042+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:68 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210a80 con 0x561377dc7000 2023-09-23T03:24:43.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.043+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 <== osd.0 v2:172.21.15.154:6800/2920938138 8 ==== osd_op_reply(68 10000000000.00000000 [omap-get-vals out=509b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+509 (crc 0 0 0) 0x561377d38d80 con 0x561377dc7000 2023-09-23T03:24:43.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.043+0000 7f3c9b8495c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:24:43.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.043+0000 7f3c9b8495c0 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2023-09-23T03:24:43.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.043+0000 7f3c9b8495c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2023-09-23T03:24:43.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.043+0000 7f3c9b8495c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-09-23T03:24:43.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.043+0000 7f3c9b8495c0 10 datascan.scan_links: step 2: handling object 200.00000000 2023-09-23T03:24:43.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.043+0000 7f3c9b8495c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-09-23T03:24:43.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.043+0000 7f3c9b8495c0 10 datascan.scan_links: step 2: handling object 200.00000001 2023-09-23T03:24:43.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.043+0000 7f3c9b8495c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-09-23T03:24:43.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.043+0000 7f3c9b8495c0 10 datascan.scan_links: step 2: handling object 500.00000000 2023-09-23T03:24:43.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.043+0000 7f3c9b8495c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-09-23T03:24:43.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.043+0000 7f3c9b8495c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2023-09-23T03:24:43.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.043+0000 7f3c9b8495c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-09-23T03:24:43.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.043+0000 7f3c9b8495c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2023-09-23T03:24:43.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.043+0000 7f3c9b8495c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-09-23T03:24:43.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.043+0000 7f3c9b8495c0 10 datascan.scan_links: step 2: handling object 100.00000000 2023-09-23T03:24:43.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.044+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:69 11.13 11:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210000 con 0x5613771fe400 2023-09-23T03:24:43.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.045+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 <== osd.5 v2:172.21.15.190:6800/882536885 10 ==== osd_op_reply(69 100.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x561377d38b40 con 0x5613771fe400 2023-09-23T03:24:43.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.045+0000 7f3c9b8495c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2023-09-23T03:24:43.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.045+0000 7f3c9b8495c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-09-23T03:24:43.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.045+0000 7f3c9b8495c0 10 datascan.scan_links: step 2: handling object 400.00000000 2023-09-23T03:24:43.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.045+0000 7f3c9b8495c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-09-23T03:24:43.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.045+0000 7f3c9b8495c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2023-09-23T03:24:43.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.045+0000 7f3c9b8495c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-09-23T03:24:43.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.045+0000 7f3c9b8495c0 10 datascan.scan_links: step 2: handling object 1.00000000 2023-09-23T03:24:43.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.045+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:70 11.1f 11:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210e00 con 0x5613771fe000 2023-09-23T03:24:43.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.046+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 <== osd.2 v2:172.21.15.154:6822/2884407243 14 ==== osd_op_reply(70 1.00000000 [omap-get-vals out=508b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+508 (crc 0 0 0) 0x561377d38b40 con 0x5613771fe000 2023-09-23T03:24:43.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.046+0000 7f3c9b8495c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:24:43.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.046+0000 7f3c9b8495c0 20 datascan.scan_links: adding 0x10000000000 for future processing to fix dnfirst 2023-09-23T03:24:43.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.046+0000 7f3c9b8495c0 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2023-09-23T03:24:43.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.046+0000 7f3c9b8495c0 10 datascan.scan_links: removing dup dentries from 0 objects 2023-09-23T03:24:43.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.046+0000 7f3c9b8495c0 10 datascan.scan_links: processing 0 bad_nlink_inos 2023-09-23T03:24:43.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.046+0000 7f3c9b8495c0 10 datascan.scan_links: processing 2 injected_inos 2023-09-23T03:24:43.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.046+0000 7f3c9b8495c0 10 datascan.scan_links: handling injected_ino 0x10000000000 2023-09-23T03:24:43.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.046+0000 7f3c9b8495c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x1:subdir 2023-09-23T03:24:43.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.046+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:71 11.1f 11:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377211180 con 0x5613771fe000 2023-09-23T03:24:43.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.047+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 <== osd.2 v2:172.21.15.154:6822/2884407243 15 ==== osd_op_reply(71 1.00000000 [omap-get-vals-by-keys out=507b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+507 (crc 0 0 0) 0x561377d38b40 con 0x5613771fe000 2023-09-23T03:24:43.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.047+0000 7f3c9b8495c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:24:43.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.047+0000 7f3c9b8495c0 20 datascan.scan_links: first is now 2 2023-09-23T03:24:43.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.047+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:72 11.1f 11:ff5b34d6:::1.00000000:head [omap-set-vals in=507b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377211500 con 0x5613771fe000 2023-09-23T03:24:43.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.050+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 <== osd.2 v2:172.21.15.154:6822/2884407243 16 ==== osd_op_reply(72 1.00000000 [omap-set-vals] v88'13 uv13 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x561377d38b40 con 0x5613771fe000 2023-09-23T03:24:43.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.050+0000 7f3c9b8495c0 20 datascan.inject_linkage: Injected dentry 0x0x1/subdir pointing to 0x0x10000000000 2023-09-23T03:24:43.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.050+0000 7f3c9b8495c0 10 datascan.scan_links: handling injected_ino 0x10000000001 2023-09-23T03:24:43.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.050+0000 7f3c9b8495c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:sixmegs 2023-09-23T03:24:43.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.050+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:73 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x561376ee5880 con 0x561377dc7000 2023-09-23T03:24:43.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.051+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 <== osd.0 v2:172.21.15.154:6800/2920938138 9 ==== osd_op_reply(73 10000000000.00000000 [omap-get-vals-by-keys out=508b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+508 (crc 0 0 0) 0x561377d38d80 con 0x561377dc7000 2023-09-23T03:24:43.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.051+0000 7f3c9b8495c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=sixmegs frag=*, r=0 2023-09-23T03:24:43.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.051+0000 7f3c9b8495c0 20 datascan.scan_links: first is now 2 2023-09-23T03:24:43.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.051+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:74 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=508b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210a80 con 0x561377dc7000 2023-09-23T03:24:43.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.055+0000 7f3c99f78640 1 -- 172.21.15.154:0/1220270411 <== osd.0 v2:172.21.15.154:6800/2920938138 10 ==== osd_op_reply(74 10000000000.00000000 [omap-set-vals] v88'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561377d38d80 con 0x561377dc7000 2023-09-23T03:24:43.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.055+0000 7f3c9b8495c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/sixmegs pointing to 0x0x10000000001 2023-09-23T03:24:43.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.055+0000 7f3c9b8495c0 10 datascan.scan_links: updating inotable 2023-09-23T03:24:43.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.055+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:75 11.13 11:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210380 con 0x5613771fe400 2023-09-23T03:24:43.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.056+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 <== osd.5 v2:172.21.15.190:6800/882536885 11 ==== osd_op_reply(75 mds0_inotable [read 0~34 out=34b] v0'0 uv6 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x561377d38b40 con 0x5613771fe400 2023-09-23T03:24:43.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.056+0000 7f3c9b8495c0 10 mds.0.inotable: skip_inos was [0x10000000000~0x10000000000] 2023-09-23T03:24:43.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.056+0000 7f3c9b8495c0 10 mds.0.inotable: skip_inos now [0x10000000002~0xfffffffffe] 2023-09-23T03:24:43.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.056+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:76 11.13 11:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210700 con 0x5613771fe400 2023-09-23T03:24:43.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.061+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 <== osd.5 v2:172.21.15.190:6800/882536885 12 ==== osd_op_reply(76 mds0_inotable [writefull 0~34] v88'8 uv8 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x561377d38b40 con 0x5613771fe400 2023-09-23T03:24:43.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.061+0000 7f3c9b8495c0 10 datascan.scan_links: updating snaptable 2023-09-23T03:24:43.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.061+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/1220270411 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:77 11.d 11:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) v8 -- 0x561377210000 con 0x5613771fe000 2023-09-23T03:24:43.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.062+0000 7f3c98f76640 1 -- 172.21.15.154:0/1220270411 <== osd.2 v2:172.21.15.154:6822/2884407243 17 ==== osd_op_reply(77 mds_snaptable [read 0~70 out=70b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x561377d38b40 con 0x5613771fe000 2023-09-23T03:24:43.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.062+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/1220270411 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x561377dc7000 msgr2=0x5613771e4580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:43.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.062+0000 7f3c9b8495c0 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x561377dc7000 0x5613771e4580 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:43.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.062+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/1220270411 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x561377dab000 msgr2=0x561376ef6680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:43.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.062+0000 7f3c9b8495c0 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x561377dab000 0x561376ef6680 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:43.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.062+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/1220270411 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5613771fe000 msgr2=0x5613771e4b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:43.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.062+0000 7f3c9b8495c0 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5613771fe000 0x5613771e4b00 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:43.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.062+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/1220270411 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x561377dc7c00 msgr2=0x561376ef4580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:43.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.062+0000 7f3c9b8495c0 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x561377dc7c00 0x561376ef4580 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:43.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.062+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/1220270411 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x561377dc7800 msgr2=0x5613771e5600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:43.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.062+0000 7f3c9b8495c0 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x561377dc7800 0x5613771e5600 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:43.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.062+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 reap_dead start 2023-09-23T03:24:43.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.062+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/1220270411 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x5613771fe400 msgr2=0x5613771e6680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:43.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.062+0000 7f3c9b8495c0 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x5613771fe400 0x5613771e6680 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:43.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.062+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 reap_dead start 2023-09-23T03:24:43.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.062+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/1220270411 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x5613771fec00 msgr2=0x5613771e5b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:43.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.062+0000 7f3c9b8495c0 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x5613771fec00 0x5613771e5b80 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:43.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.062+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/1220270411 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x561377dc7400 msgr2=0x5613771e5080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:43.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.062+0000 7f3c9b8495c0 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x561377dc7400 0x5613771e5080 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:43.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.063+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/1220270411 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x561377dc6c00 msgr2=0x5613771e4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:43.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.063+0000 7f3c9b8495c0 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x561377dc6c00 0x5613771e4000 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x561377d60720 tx=0x561377d62c60 comp rx=0 tx=0).stop 2023-09-23T03:24:43.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.063+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/1220270411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561377c5c000 msgr2=0x561376ef7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:43.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.063+0000 7f3c9b8495c0 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561377c5c000 0x561376ef7180 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x561377d618c0 tx=0x561377d62de0 comp rx=0 tx=0).stop 2023-09-23T03:24:43.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.063+0000 7f3c99777640 1 -- 172.21.15.154:0/1220270411 reap_dead start 2023-09-23T03:24:43.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.063+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/1220270411 shutdown_connections 2023-09-23T03:24:43.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.063+0000 7f3c9b8495c0 1 --2- 172.21.15.154:0/1220270411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561377c5c000 0x561376ef7180 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:43.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.063+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/1220270411 >> 172.21.15.154:0/1220270411 conn(0x561377c5c400 msgr2=0x561377c15800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:43.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.064+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/1220270411 shutdown_connections 2023-09-23T03:24:43.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.064+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/1220270411 wait complete. 2023-09-23T03:24:43.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.065+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/2164676632 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561377c5dc00 msgr2=0x561376ef5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:43.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.065+0000 7f3c9b8495c0 1 --2- 172.21.15.154:0/2164676632 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561377c5dc00 0x561376ef5600 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x561377bde7b0 tx=0x561377d3e120 comp rx=0 tx=0).stop 2023-09-23T03:24:43.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.065+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/2164676632 shutdown_connections 2023-09-23T03:24:43.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.065+0000 7f3c9b8495c0 1 --2- 172.21.15.154:0/2164676632 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561377c5dc00 0x561376ef5600 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:43.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.065+0000 7f3c9b8495c0 1 --2- 172.21.15.154:0/2164676632 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x561376e8bc00 0x561376ef4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:43.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.065+0000 7f3c9b8495c0 1 --2- 172.21.15.154:0/2164676632 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x561377c5c800 0x561376ef5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:43.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.065+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/2164676632 >> 172.21.15.154:0/2164676632 conn(0x561376e8b000 msgr2=0x561377c14000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:43.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.066+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/2164676632 shutdown_connections 2023-09-23T03:24:43.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.066+0000 7f3c9b8495c0 1 -- 172.21.15.154:0/2164676632 wait complete. 2023-09-23T03:24:43.078 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.214187, result: 2023-09-23T03:24:43.078 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-09-23T03:24:43.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.302+0000 7fc00091d640 1 Processor -- start 2023-09-23T03:24:43.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.303+0000 7fc00091d640 1 -- start start 2023-09-23T03:24:43.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.304+0000 7fc00091d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbffc0b1b20 0x7fbffc0ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:43.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.304+0000 7fc00091d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbffc0cee70 0x7fbffc0ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:43.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.304+0000 7fc00091d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbffc15ac30 con 0x7fbffc0acce0 2023-09-23T03:24:43.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.304+0000 7fc00091d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbffc0b2e90 con 0x7fbffc0b1b20 2023-09-23T03:24:43.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.304+0000 7fc00091d640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbffc0cf320 con 0x7fbffc0cee70 2023-09-23T03:24:43.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.304+0000 7fbff3fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbffc0b1b20 0x7fbffc0ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:43.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.304+0000 7fbffa59c640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fbffc0acce0 0x7fbffc15ed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:45630/0 (socket says 172.21.15.154:45630) 2023-09-23T03:24:43.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.304+0000 7fbffa59c640 1 -- 172.21.15.154:0/2378776672 learned_addr learned my addr 172.21.15.154:0/2378776672 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:24:43.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.305+0000 7fbff3fff640 1 -- 172.21.15.154:0/2378776672 >> v2:172.21.15.190:3300/0 conn(0x7fbffc0cee70 msgr2=0x7fbffc0ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:43.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.305+0000 7fbff3fff640 1 --2- 172.21.15.154:0/2378776672 >> v2:172.21.15.190:3300/0 conn(0x7fbffc0cee70 0x7fbffc0ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:43.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.305+0000 7fbff3fff640 1 -- 172.21.15.154:0/2378776672 >> v1:172.21.15.154:6789/0 conn(0x7fbffc0acce0 legacy=0x7fbffc15ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:43.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.305+0000 7fbff3fff640 1 -- 172.21.15.154:0/2378776672 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbffc0b1f70 con 0x7fbffc0b1b20 2023-09-23T03:24:43.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.306+0000 7fbff3fff640 1 --2- 172.21.15.154:0/2378776672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbffc0b1b20 0x7fbffc0ce930 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fbffc0c9430 tx=0x7fbfe402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dec97b90807c1e48 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:43.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.306+0000 7fbff9d9b640 1 -- 172.21.15.154:0/2378776672 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe402e070 con 0x7fbffc0b1b20 2023-09-23T03:24:43.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.306+0000 7fbff9d9b640 1 -- 172.21.15.154:0/2378776672 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe40315a0 con 0x7fbffc0b1b20 2023-09-23T03:24:43.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.306+0000 7fbff9d9b640 1 -- 172.21.15.154:0/2378776672 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4031930 con 0x7fbffc0b1b20 2023-09-23T03:24:43.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.307+0000 7fc00091d640 1 -- 172.21.15.154:0/2378776672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbffc0b1b20 msgr2=0x7fbffc0ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:43.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.307+0000 7fc00091d640 1 --2- 172.21.15.154:0/2378776672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbffc0b1b20 0x7fbffc0ce930 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fbffc0c9430 tx=0x7fbfe402b840 comp rx=0 tx=0).stop 2023-09-23T03:24:43.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.307+0000 7fc00091d640 1 -- 172.21.15.154:0/2378776672 shutdown_connections 2023-09-23T03:24:43.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.307+0000 7fc00091d640 1 --2- 172.21.15.154:0/2378776672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbffc0b1b20 0x7fbffc0ce930 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:43.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.307+0000 7fc00091d640 1 --2- 172.21.15.154:0/2378776672 >> v2:172.21.15.190:3300/0 conn(0x7fbffc0cee70 0x7fbffc0ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:43.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.307+0000 7fc00091d640 1 -- 172.21.15.154:0/2378776672 >> 172.21.15.154:0/2378776672 conn(0x7fbffc06a0f0 msgr2=0x7fbffc06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:43.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.307+0000 7fc00091d640 1 -- 172.21.15.154:0/2378776672 shutdown_connections 2023-09-23T03:24:43.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.307+0000 7fc00091d640 1 -- 172.21.15.154:0/2378776672 wait complete. 2023-09-23T03:24:43.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.309+0000 7fc00091d640 1 Processor -- start 2023-09-23T03:24:43.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.309+0000 7fc00091d640 1 -- start start 2023-09-23T03:24:43.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.310+0000 7fc00091d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbffc0acce0 0x7fbffc159670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:43.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.310+0000 7fc00091d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbffc0b1b20 0x7fbffc157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:43.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.310+0000 7fc00091d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbffc0cee70 0x7fbffc0c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:43.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.310+0000 7fc00091d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc0c62c0 con 0x7fbffc0cee70 2023-09-23T03:24:43.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.310+0000 7fc00091d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc0c65a0 con 0x7fbffc0b1b20 2023-09-23T03:24:43.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.310+0000 7fc00091d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc0c3240 con 0x7fbffc0acce0 2023-09-23T03:24:43.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.310+0000 7fbffad9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbffc0cee70 0x7fbffc0c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:43.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.310+0000 7fbffad9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbffc0cee70 0x7fbffc0c7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43446/0 (socket says 172.21.15.154:43446) 2023-09-23T03:24:43.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.310+0000 7fbffad9d640 1 -- 172.21.15.154:0/2358182655 learned_addr learned my addr 172.21.15.154:0/2358182655 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:43.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.310+0000 7fbffa59c640 1 --2- 172.21.15.154:0/2358182655 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbffc0acce0 0x7fbffc159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:43.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.310+0000 7fbff3fff640 1 --2- 172.21.15.154:0/2358182655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbffc0b1b20 0x7fbffc157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:43.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.311+0000 7fbffad9d640 1 -- 172.21.15.154:0/2358182655 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbffc0acce0 msgr2=0x7fbffc159670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:43.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.311+0000 7fbffad9d640 1 --2- 172.21.15.154:0/2358182655 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbffc0acce0 0x7fbffc159670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:43.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.311+0000 7fbffad9d640 1 -- 172.21.15.154:0/2358182655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbffc0b1b20 msgr2=0x7fbffc157630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:43.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.311+0000 7fbffad9d640 1 --2- 172.21.15.154:0/2358182655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbffc0b1b20 0x7fbffc157630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:43.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.311+0000 7fbffad9d640 1 -- 172.21.15.154:0/2358182655 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe4009400 con 0x7fbffc0cee70 2023-09-23T03:24:43.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.311+0000 7fbffad9d640 1 --2- 172.21.15.154:0/2358182655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbffc0cee70 0x7fbffc0c7d40 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fbff400b970 tx=0x7fbff400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:43.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.311+0000 7fbff2ffd640 1 -- 172.21.15.154:0/2358182655 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff400cb40 con 0x7fbffc0cee70 2023-09-23T03:24:43.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.311+0000 7fbff2ffd640 1 -- 172.21.15.154:0/2358182655 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff400ccc0 con 0x7fbffc0cee70 2023-09-23T03:24:43.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.311+0000 7fbff2ffd640 1 -- 172.21.15.154:0/2358182655 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff4014680 con 0x7fbffc0cee70 2023-09-23T03:24:43.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.311+0000 7fc00091d640 1 -- 172.21.15.154:0/2358182655 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbffc0c3520 con 0x7fbffc0cee70 2023-09-23T03:24:43.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.311+0000 7fc00091d640 1 -- 172.21.15.154:0/2358182655 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbffc15add0 con 0x7fbffc0cee70 2023-09-23T03:24:43.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.312+0000 7fc00091d640 1 -- 172.21.15.154:0/2358182655 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbffc15b080 con 0x7fbffc0cee70 2023-09-23T03:24:43.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.313+0000 7fbff2ffd640 1 -- 172.21.15.154:0/2358182655 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fbff401d3b0 con 0x7fbffc0cee70 2023-09-23T03:24:43.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.314+0000 7fbff2ffd640 1 --2- 172.21.15.154:0/2358182655 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbfdc06af30 0x7fbfdc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:43.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.314+0000 7fbff2ffd640 1 -- 172.21.15.154:0/2358182655 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6605+0+0 (secure 0 0 0) 0x7fbff4089ac0 con 0x7fbffc0cee70 2023-09-23T03:24:43.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.314+0000 7fbffa59c640 1 --2- 172.21.15.154:0/2358182655 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbfdc06af30 0x7fbfdc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:43.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.315+0000 7fbffa59c640 1 --2- 172.21.15.154:0/2358182655 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbfdc06af30 0x7fbfdc06d3f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0010e0 tx=0x7fbfec009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:43.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.319+0000 7fbff2ffd640 1 -- 172.21.15.154:0/2358182655 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fbffc15b080 con 0x7fbffc0cee70 2023-09-23T03:24:43.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:43.581+0000 7fc00091d640 1 -- 172.21.15.154:0/2358182655 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7fbffc15b080 con 0x7fbffc0cee70 2023-09-23T03:24:43.591 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:24:43.597 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:24:44.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.394+0000 7fbff2ffd640 1 -- 172.21.15.154:0/2358182655 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 4:0 v70) v1 ==== 101+0+0 (secure 0 0 0) 0x7fbffc15b080 con 0x7fbffc0cee70 2023-09-23T03:24:44.397 INFO:teuthology.orchestra.run.smithi154.stderr:repaired: restoring rank 4:0 2023-09-23T03:24:44.400 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.398+0000 7fc00091d640 1 -- 172.21.15.154:0/2358182655 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbfdc06af30 msgr2=0x7fbfdc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:44.400 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.398+0000 7fc00091d640 1 --2- 172.21.15.154:0/2358182655 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbfdc06af30 0x7fbfdc06d3f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0010e0 tx=0x7fbfec009040 comp rx=0 tx=0).stop 2023-09-23T03:24:44.400 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.398+0000 7fc00091d640 1 -- 172.21.15.154:0/2358182655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbffc0cee70 msgr2=0x7fbffc0c7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:44.400 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.398+0000 7fc00091d640 1 --2- 172.21.15.154:0/2358182655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbffc0cee70 0x7fbffc0c7d40 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fbff400b970 tx=0x7fbff400be30 comp rx=0 tx=0).stop 2023-09-23T03:24:44.400 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.398+0000 7fc00091d640 1 -- 172.21.15.154:0/2358182655 shutdown_connections 2023-09-23T03:24:44.400 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.398+0000 7fc00091d640 1 --2- 172.21.15.154:0/2358182655 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbfdc06af30 0x7fbfdc06d3f0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:44.400 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.398+0000 7fc00091d640 1 --2- 172.21.15.154:0/2358182655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbffc0cee70 0x7fbffc0c7d40 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:44.400 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.398+0000 7fc00091d640 1 --2- 172.21.15.154:0/2358182655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbffc0b1b20 0x7fbffc157630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:44.400 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.398+0000 7fc00091d640 1 --2- 172.21.15.154:0/2358182655 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbffc0acce0 0x7fbffc159670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:44.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.398+0000 7fc00091d640 1 -- 172.21.15.154:0/2358182655 >> 172.21.15.154:0/2358182655 conn(0x7fbffc06a0f0 msgr2=0x7fbffc0ca520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:44.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.399+0000 7fc00091d640 1 -- 172.21.15.154:0/2358182655 shutdown_connections 2023-09-23T03:24:44.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.399+0000 7fc00091d640 1 -- 172.21.15.154:0/2358182655 wait complete. 2023-09-23T03:24:44.409 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:24:44.408+0000 7f5ce453a640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-09-23T03:24:44.421 INFO:tasks.ceph.mds.a:Restarting daemon 2023-09-23T03:24:44.422 INFO:tasks.ceph.mds.a:Stopping old one... 2023-09-23T03:24:44.422 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-09-23T03:24:44.422 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:24:44.422 INFO:tasks.ceph.mds.c:Restarting daemon 2023-09-23T03:24:44.422 INFO:tasks.ceph.mds.c:Stopping old one... 2023-09-23T03:24:44.422 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-09-23T03:24:44.422 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:24:44.423 INFO:tasks.ceph.mds.b:Restarting daemon 2023-09-23T03:24:44.423 INFO:tasks.ceph.mds.b:Stopping old one... 2023-09-23T03:24:44.423 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-09-23T03:24:44.423 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:24:44.423 INFO:tasks.ceph.mds.d:Restarting daemon 2023-09-23T03:24:44.423 INFO:tasks.ceph.mds.d:Stopping old one... 2023-09-23T03:24:44.424 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-09-23T03:24:44.424 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:24:44.439 INFO:tasks.ceph.mds.c:Stopped 2023-09-23T03:24:44.439 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-09-23T03:24:44.440 INFO:tasks.ceph.mds.a:Stopped 2023-09-23T03:24:44.440 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-09-23T03:24:44.444 INFO:tasks.ceph.mds.d:Stopped 2023-09-23T03:24:44.445 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-09-23T03:24:44.445 INFO:tasks.ceph.mds.b:Stopped 2023-09-23T03:24:44.445 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-09-23T03:24:44.484 INFO:tasks.ceph.mds.c:Started 2023-09-23T03:24:44.484 DEBUG:teuthology.parallel:result is None 2023-09-23T03:24:44.488 INFO:tasks.ceph.mds.a:Started 2023-09-23T03:24:44.488 DEBUG:teuthology.parallel:result is None 2023-09-23T03:24:44.493 INFO:tasks.ceph.mds.d:Started 2023-09-23T03:24:44.494 DEBUG:teuthology.parallel:result is None 2023-09-23T03:24:44.500 INFO:tasks.ceph.mds.b:Started 2023-09-23T03:24:44.501 DEBUG:teuthology.parallel:result is None 2023-09-23T03:24:44.501 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:24:44.643 INFO:tasks.ceph.mds.c.smithi154.stdout:starting mds.c at 2023-09-23T03:24:44.643 INFO:tasks.ceph.mds.a.smithi154.stdout:starting mds.a at 2023-09-23T03:24:44.656 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-09-23T03:24:44.656 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-09-23T03:24:44.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.687+0000 7fc5b9935640 1 Processor -- start 2023-09-23T03:24:44.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.687+0000 7fc5b9935640 1 -- start start 2023-09-23T03:24:44.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.688+0000 7fc5b9935640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc5b40cf500 0x7fc5b40ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:44.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.688+0000 7fc5b9935640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc5b40cf8e0 0x7fc5b40ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:44.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.688+0000 7fc5b9935640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5b415c4f0 0x7fc5b415c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:44.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.689+0000 7fc5b8933640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc5b40cf500 0x7fc5b40ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:44.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.689+0000 7fc5b9935640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5b40d17f0 con 0x7fc5b40cf500 2023-09-23T03:24:44.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.689+0000 7fc5b3fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc5b40cf8e0 0x7fc5b40ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:44.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.689+0000 7fc5b8933640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc5b40cf500 0x7fc5b40ce050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52508/0 (socket says 172.21.15.154:52508) 2023-09-23T03:24:44.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.689+0000 7fc5b8933640 1 -- 172.21.15.154:0/2245428057 learned_addr learned my addr 172.21.15.154:0/2245428057 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:44.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.689+0000 7fc5b9134640 1 --2- 172.21.15.154:0/2245428057 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5b415c4f0 0x7fc5b415c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:44.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.689+0000 7fc5b9935640 1 -- 172.21.15.154:0/2245428057 --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5b40d0470 con 0x7fc5b40cf8e0 2023-09-23T03:24:44.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.689+0000 7fc5b9935640 1 -- 172.21.15.154:0/2245428057 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b40d0750 con 0x7fc5b415c4f0 2023-09-23T03:24:44.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.689+0000 7fc5b3fff640 1 -- 172.21.15.154:0/2245428057 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5b415c4f0 msgr2=0x7fc5b415c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:44.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.689+0000 7fc5b3fff640 1 --2- 172.21.15.154:0/2245428057 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5b415c4f0 0x7fc5b415c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:44.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.689+0000 7fc5b3fff640 1 -- 172.21.15.154:0/2245428057 >> v2:172.21.15.154:3300/0 conn(0x7fc5b40cf500 msgr2=0x7fc5b40ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:44.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.689+0000 7fc5b3fff640 1 --2- 172.21.15.154:0/2245428057 >> v2:172.21.15.154:3300/0 conn(0x7fc5b40cf500 0x7fc5b40ce050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:44.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.689+0000 7fc5b3fff640 1 -- 172.21.15.154:0/2245428057 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b40d0a30 con 0x7fc5b40cf8e0 2023-09-23T03:24:44.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.691+0000 7fc5b3fff640 1 --2- 172.21.15.154:0/2245428057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5b40cf8e0 0x7fc5b40ce590 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc5a00097a0 tx=0x7fc5a002b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5627307cf562627 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:44.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.691+0000 7fc5b37fe640 1 -- 172.21.15.154:0/2245428057 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a002f050 con 0x7fc5b40cf8e0 2023-09-23T03:24:44.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.691+0000 7fc5b37fe640 1 -- 172.21.15.154:0/2245428057 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a0031700 con 0x7fc5b40cf8e0 2023-09-23T03:24:44.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.691+0000 7fc5b37fe640 1 -- 172.21.15.154:0/2245428057 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0031a90 con 0x7fc5b40cf8e0 2023-09-23T03:24:44.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.691+0000 7fc5b9935640 1 -- 172.21.15.154:0/2245428057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5b40cf8e0 msgr2=0x7fc5b40ce590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:44.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.691+0000 7fc5b9935640 1 --2- 172.21.15.154:0/2245428057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5b40cf8e0 0x7fc5b40ce590 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc5a00097a0 tx=0x7fc5a002b7a0 comp rx=0 tx=0).stop 2023-09-23T03:24:44.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.692+0000 7fc5b9935640 1 -- 172.21.15.154:0/2245428057 shutdown_connections 2023-09-23T03:24:44.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.692+0000 7fc5b9935640 1 --2- 172.21.15.154:0/2245428057 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5b415c4f0 0x7fc5b415c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:44.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.692+0000 7fc5b9935640 1 --2- 172.21.15.154:0/2245428057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5b40cf8e0 0x7fc5b40ce590 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:44.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.692+0000 7fc5b9935640 1 --2- 172.21.15.154:0/2245428057 >> v2:172.21.15.154:3300/0 conn(0x7fc5b40cf500 0x7fc5b40ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:44.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.692+0000 7fc5b9935640 1 -- 172.21.15.154:0/2245428057 >> 172.21.15.154:0/2245428057 conn(0x7fc5b409bbb0 msgr2=0x7fc5b40d28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:44.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.692+0000 7fc5b9935640 1 -- 172.21.15.154:0/2245428057 shutdown_connections 2023-09-23T03:24:44.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.692+0000 7fc5b9935640 1 -- 172.21.15.154:0/2245428057 wait complete. 2023-09-23T03:24:44.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.693+0000 7fc5b9935640 1 Processor -- start 2023-09-23T03:24:44.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.694+0000 7fc5b9935640 1 -- start start 2023-09-23T03:24:44.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.694+0000 7fc5b9935640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5b40cf500 0x7fc5b4156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:44.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.695+0000 7fc5b9935640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5b40cf8e0 0x7fc5b40b71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:44.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.695+0000 7fc5b9935640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5b415c4f0 0x7fc5b415e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:44.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.695+0000 7fc5b9935640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b40cb9b0 con 0x7fc5b40cf8e0 2023-09-23T03:24:44.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.695+0000 7fc5b9935640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b4157530 con 0x7fc5b415c4f0 2023-09-23T03:24:44.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.695+0000 7fc5b9935640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b40cad30 con 0x7fc5b40cf500 2023-09-23T03:24:44.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.695+0000 7fc5b3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5b40cf8e0 0x7fc5b40b71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:44.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.695+0000 7fc5b3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5b40cf8e0 0x7fc5b40b71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52510/0 (socket says 172.21.15.154:52510) 2023-09-23T03:24:44.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.695+0000 7fc5b3fff640 1 -- 172.21.15.154:0/2546948212 learned_addr learned my addr 172.21.15.154:0/2546948212 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:44.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.695+0000 7fc5b8933640 1 --2- 172.21.15.154:0/2546948212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5b40cf500 0x7fc5b4156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:44.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.695+0000 7fc5b9134640 1 --2- 172.21.15.154:0/2546948212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5b415c4f0 0x7fc5b415e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:44.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.695+0000 7fc5b3fff640 1 -- 172.21.15.154:0/2546948212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5b40cf500 msgr2=0x7fc5b4156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:44.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.695+0000 7fc5b3fff640 1 --2- 172.21.15.154:0/2546948212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5b40cf500 0x7fc5b4156ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:44.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.695+0000 7fc5b3fff640 1 -- 172.21.15.154:0/2546948212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5b415c4f0 msgr2=0x7fc5b415e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:44.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.695+0000 7fc5b3fff640 1 --2- 172.21.15.154:0/2546948212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5b415c4f0 0x7fc5b415e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:44.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.695+0000 7fc5b3fff640 1 -- 172.21.15.154:0/2546948212 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a0009400 con 0x7fc5b40cf8e0 2023-09-23T03:24:44.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.696+0000 7fc5b3fff640 1 --2- 172.21.15.154:0/2546948212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5b40cf8e0 0x7fc5b40b71c0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0036040 tx=0x7fc5a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:44.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.696+0000 7fc5b17fa640 1 -- 172.21.15.154:0/2546948212 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0006e10 con 0x7fc5b40cf8e0 2023-09-23T03:24:44.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.696+0000 7fc5b17fa640 1 -- 172.21.15.154:0/2546948212 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a0031500 con 0x7fc5b40cf8e0 2023-09-23T03:24:44.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.696+0000 7fc5b17fa640 1 -- 172.21.15.154:0/2546948212 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0033c90 con 0x7fc5b40cf8e0 2023-09-23T03:24:44.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.696+0000 7fc5b9935640 1 -- 172.21.15.154:0/2546948212 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5b40cb010 con 0x7fc5b40cf8e0 2023-09-23T03:24:44.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.696+0000 7fc5b9935640 1 -- 172.21.15.154:0/2546948212 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5b40cb5d0 con 0x7fc5b40cf8e0 2023-09-23T03:24:44.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.697+0000 7fc5b9935640 1 -- 172.21.15.154:0/2546948212 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5b415c960 con 0x7fc5b40cf8e0 2023-09-23T03:24:44.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.698+0000 7fc5b17fa640 1 -- 172.21.15.154:0/2546948212 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fc5a003a040 con 0x7fc5b40cf8e0 2023-09-23T03:24:44.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.699+0000 7fc5b17fa640 1 --2- 172.21.15.154:0/2546948212 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc59806af30 0x7fc59806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:44.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.699+0000 7fc5b17fa640 1 -- 172.21.15.154:0/2546948212 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6605+0+0 (secure 0 0 0) 0x7fc5a00aab40 con 0x7fc5b40cf8e0 2023-09-23T03:24:44.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.700+0000 7fc5b8933640 1 --2- 172.21.15.154:0/2546948212 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc59806af30 0x7fc59806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:44.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.700+0000 7fc5b8933640 1 --2- 172.21.15.154:0/2546948212 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc59806af30 0x7fc59806d3f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8009a90 tx=0x7fc5a8009210 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:44.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.704+0000 7fc5b17fa640 1 -- 172.21.15.154:0/2546948212 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fc5b415c960 con 0x7fc5b40cf8e0 2023-09-23T03:24:44.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.928+0000 7fc5b9935640 1 -- 172.21.15.154:0/2546948212 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5b415c960 con 0x7fc5b40cf8e0 2023-09-23T03:24:44.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.929+0000 7fc5b17fa640 1 -- 172.21.15.154:0/2546948212 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4928 (secure 0 0 0) 0x7fc5b415c960 con 0x7fc5b40cf8e0 2023-09-23T03:24:44.931 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:44.931 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":71,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/728936136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":728936136},{"type":"v1","addr":"172.21.15.190:6833","nonce":728936136}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":5890,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2277229238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2277229238},{"type":"v1","addr":"172.21.15.190:6835","nonce":2277229238}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":7257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/4245928089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":4245928089},{"type":"v1","addr":"172.21.15.154:6835","nonce":4245928089}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:24:37.386034+0000","modified":"2023-09-23T03:24:44.400599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7575":{"gid":7575,"name":"a","rank":0,"incarnation":71,"state":"up:replay","state_seq":1,"addr":"172.21.15.154:6837/1588996498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1588996498},{"type":"v1","addr":"172.21.15.154:6837","nonce":1588996498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12,13],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-09-23T03:24:44.931 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 71 2023-09-23T03:24:44.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.932+0000 7fc5b9935640 1 -- 172.21.15.154:0/2546948212 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc59806af30 msgr2=0x7fc59806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:44.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.932+0000 7fc5b9935640 1 --2- 172.21.15.154:0/2546948212 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc59806af30 0x7fc59806d3f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8009a90 tx=0x7fc5a8009210 comp rx=0 tx=0).stop 2023-09-23T03:24:44.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.932+0000 7fc5b9935640 1 -- 172.21.15.154:0/2546948212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5b40cf8e0 msgr2=0x7fc5b40b71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:44.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.932+0000 7fc5b9935640 1 --2- 172.21.15.154:0/2546948212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5b40cf8e0 0x7fc5b40b71c0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0036040 tx=0x7fc5a00046c0 comp rx=0 tx=0).stop 2023-09-23T03:24:44.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.932+0000 7fc5b9935640 1 -- 172.21.15.154:0/2546948212 shutdown_connections 2023-09-23T03:24:44.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.932+0000 7fc5b9935640 1 --2- 172.21.15.154:0/2546948212 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc59806af30 0x7fc59806d3f0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:44.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.932+0000 7fc5b9935640 1 --2- 172.21.15.154:0/2546948212 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc5b415c4f0 0x7fc5b415e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:44.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.932+0000 7fc5b9935640 1 --2- 172.21.15.154:0/2546948212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc5b40cf8e0 0x7fc5b40b71c0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:44.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.932+0000 7fc5b9935640 1 --2- 172.21.15.154:0/2546948212 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc5b40cf500 0x7fc5b4156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:44.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.932+0000 7fc5b9935640 1 -- 172.21.15.154:0/2546948212 >> 172.21.15.154:0/2546948212 conn(0x7fc5b409bbb0 msgr2=0x7fc5b40b54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:44.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.932+0000 7fc5b9935640 1 -- 172.21.15.154:0/2546948212 shutdown_connections 2023-09-23T03:24:44.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:44.932+0000 7fc5b9935640 1 -- 172.21.15.154:0/2546948212 wait complete. 2023-09-23T03:24:45.352 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-23T03:24:37.386034+0000', 'modified': '2023-09-23T03:24:44.400599+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 88, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7575}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7575': {'gid': 7575, 'name': 'a', 'rank': 0, 'incarnation': 71, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.154:6837/1588996498', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6836', 'nonce': 1588996498}, {'type': 'v1', 'addr': '172.21.15.154:6837', 'nonce': 1588996498}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12, 13], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-23T03:24:45.352 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7575:up:replay 2023-09-23T03:24:45.399 INFO:tasks.ceph.mon.a.smithi154.stderr:2023-09-23T03:24:45.397+0000 7f3df0da1640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-23T03:24:45.439 INFO:tasks.ceph.mds.b.smithi190.stderr:2023-09-23T03:24:45.438+0000 7fce45bea640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-09-23T03:24:46.353 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:24:46.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.572+0000 7f497c200640 1 Processor -- start 2023-09-23T03:24:46.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.572+0000 7f497c200640 1 -- start start 2023-09-23T03:24:46.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.573+0000 7f497c200640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49740b1b20 0x7f49740ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:46.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.573+0000 7f497c200640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f49740cee70 0x7f49740ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:46.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.573+0000 7f497c200640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f497415ac30 con 0x7f49740acce0 2023-09-23T03:24:46.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.573+0000 7f497c200640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49740b2e90 con 0x7f49740cee70 2023-09-23T03:24:46.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.573+0000 7f497c200640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49740cf320 con 0x7f49740b1b20 2023-09-23T03:24:46.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.574+0000 7f4979f9c640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f49740acce0 0x7f497415ed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:60308/0 (socket says 172.21.15.154:60308) 2023-09-23T03:24:46.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.574+0000 7f4979f9c640 1 -- 172.21.15.154:0/174341844 learned_addr learned my addr 172.21.15.154:0/174341844 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:24:46.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.574+0000 7f4978f9a640 1 -- 172.21.15.154:0/174341844 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2322926439 0 0) 0x7f4964001610 con 0x7f49740acce0 2023-09-23T03:24:46.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.574+0000 7f4978f9a640 1 -- 172.21.15.154:0/174341844 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f495c003520 con 0x7f49740acce0 2023-09-23T03:24:46.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.574+0000 7f497a79d640 1 --2- 172.21.15.154:0/174341844 >> v2:172.21.15.190:3300/0 conn(0x7f49740cee70 0x7f49740ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:46.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.574+0000 7f497979b640 1 --2- 172.21.15.154:0/174341844 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49740b1b20 0x7f49740ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:46.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.574+0000 7f4978f9a640 1 -- 172.21.15.154:0/174341844 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 324350326 0 0) 0x7f495c003520 con 0x7f49740acce0 2023-09-23T03:24:46.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.574+0000 7f4978f9a640 1 -- 172.21.15.154:0/174341844 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49740b1b20 msgr2=0x7f49740ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:46.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.574+0000 7f4978f9a640 1 --2- 172.21.15.154:0/174341844 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49740b1b20 0x7f49740ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:46.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.574+0000 7f4978f9a640 1 -- 172.21.15.154:0/174341844 >> v2:172.21.15.190:3300/0 conn(0x7f49740cee70 msgr2=0x7f49740ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:46.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.574+0000 7f4978f9a640 1 --2- 172.21.15.154:0/174341844 >> v2:172.21.15.190:3300/0 conn(0x7f49740cee70 0x7f49740ab140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:46.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.575+0000 7f4978f9a640 1 -- 172.21.15.154:0/174341844 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49740b1f70 con 0x7f49740acce0 2023-09-23T03:24:46.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.575+0000 7f4978f9a640 1 -- 172.21.15.154:0/174341844 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f4964001d00 con 0x7f49740acce0 2023-09-23T03:24:46.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.575+0000 7f4978f9a640 1 -- 172.21.15.154:0/174341844 >> v1:172.21.15.154:6789/0 conn(0x7f49740acce0 legacy=0x7f497415ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:46.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.575+0000 7f4978f9a640 1 --2- 172.21.15.154:0/174341844 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49740b1b20 0x7f495c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:46.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.575+0000 7f4978f9a640 1 --2- 172.21.15.154:0/174341844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f495c003f40 0x7f495c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:46.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.575+0000 7f4978f9a640 1 --2- 172.21.15.154:0/174341844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f495c004d00 0x7f495c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:46.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.575+0000 7f4978f9a640 1 -- 172.21.15.154:0/174341844 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49740b2e90 con 0x7f495c003f40 2023-09-23T03:24:46.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.575+0000 7f4978f9a640 1 -- 172.21.15.154:0/174341844 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49740cf320 con 0x7f495c004d00 2023-09-23T03:24:46.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.575+0000 7f4978f9a640 1 -- 172.21.15.154:0/174341844 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f495c00bae0 con 0x7f49740b1b20 2023-09-23T03:24:46.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.575+0000 7f497979b640 1 --2- 172.21.15.154:0/174341844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f495c003f40 0x7f495c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:46.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.575+0000 7f497a79d640 1 --2- 172.21.15.154:0/174341844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f495c004d00 0x7f495c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:46.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.575+0000 7f497979b640 1 -- 172.21.15.154:0/174341844 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49740b1b20 msgr2=0x7f495c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:46.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.575+0000 7f497979b640 1 --2- 172.21.15.154:0/174341844 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f49740b1b20 0x7f495c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:46.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.575+0000 7f497979b640 1 -- 172.21.15.154:0/174341844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f495c004d00 msgr2=0x7f495c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:46.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.575+0000 7f497979b640 1 --2- 172.21.15.154:0/174341844 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f495c004d00 0x7f495c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:46.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.575+0000 7f497979b640 1 -- 172.21.15.154:0/174341844 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f495c003520 con 0x7f495c003f40 2023-09-23T03:24:46.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.576+0000 7f497979b640 1 --2- 172.21.15.154:0/174341844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f495c003f40 0x7f495c0047c0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f496800ba00 tx=0x7f495c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a118dd9e4cd803 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:46.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.577+0000 7f4978f9a640 1 -- 172.21.15.154:0/174341844 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4968032640 con 0x7f495c003f40 2023-09-23T03:24:46.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.577+0000 7f4978f9a640 1 -- 172.21.15.154:0/174341844 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49680327a0 con 0x7f495c003f40 2023-09-23T03:24:46.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.577+0000 7f4978f9a640 1 -- 172.21.15.154:0/174341844 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4968032b30 con 0x7f495c003f40 2023-09-23T03:24:46.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.577+0000 7f497c200640 1 -- 172.21.15.154:0/174341844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f495c003f40 msgr2=0x7f495c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:46.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.577+0000 7f497c200640 1 --2- 172.21.15.154:0/174341844 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f495c003f40 0x7f495c0047c0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f496800ba00 tx=0x7f495c0057f0 comp rx=0 tx=0).stop 2023-09-23T03:24:46.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.577+0000 7f497a79d640 1 -- 172.21.15.154:0/174341844 reap_dead start 2023-09-23T03:24:46.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.578+0000 7f497c200640 1 -- 172.21.15.154:0/174341844 shutdown_connections 2023-09-23T03:24:46.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.578+0000 7f497c200640 1 -- 172.21.15.154:0/174341844 >> 172.21.15.154:0/174341844 conn(0x7f497406a0f0 msgr2=0x7f497406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:46.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.578+0000 7f497c200640 1 -- 172.21.15.154:0/174341844 shutdown_connections 2023-09-23T03:24:46.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.578+0000 7f497c200640 1 -- 172.21.15.154:0/174341844 wait complete. 2023-09-23T03:24:46.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.580+0000 7f497c200640 1 Processor -- start 2023-09-23T03:24:46.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.580+0000 7f497c200640 1 -- start start 2023-09-23T03:24:46.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.581+0000 7f497c200640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f495c003f40 0x7f49740c1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:46.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.581+0000 7f497c200640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49741574d0 0x7f49741f0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:46.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.581+0000 7f497c200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f497414d1f0 0x7f49740cac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:46.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.581+0000 7f497c200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497415c220 con 0x7f497414d1f0 2023-09-23T03:24:46.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.581+0000 7f497c200640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497415cbb0 con 0x7f49741574d0 2023-09-23T03:24:46.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.582+0000 7f497c200640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49740c5100 con 0x7f495c003f40 2023-09-23T03:24:46.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.582+0000 7f497a79d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f497414d1f0 0x7f49740cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:46.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.582+0000 7f497a79d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f497414d1f0 0x7f49740cac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52530/0 (socket says 172.21.15.154:52530) 2023-09-23T03:24:46.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.582+0000 7f497a79d640 1 -- 172.21.15.154:0/3070192114 learned_addr learned my addr 172.21.15.154:0/3070192114 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:46.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.582+0000 7f4979f9c640 1 --2- 172.21.15.154:0/3070192114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f495c003f40 0x7f49740c1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:46.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.582+0000 7f497979b640 1 --2- 172.21.15.154:0/3070192114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49741574d0 0x7f49741f0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:46.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.582+0000 7f497a79d640 1 -- 172.21.15.154:0/3070192114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f495c003f40 msgr2=0x7f49740c1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:46.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.582+0000 7f497a79d640 1 --2- 172.21.15.154:0/3070192114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f495c003f40 0x7f49740c1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:46.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.582+0000 7f497a79d640 1 -- 172.21.15.154:0/3070192114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49741574d0 msgr2=0x7f49741f0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:46.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.582+0000 7f497a79d640 1 --2- 172.21.15.154:0/3070192114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49741574d0 0x7f49741f0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:46.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.582+0000 7f497a79d640 1 -- 172.21.15.154:0/3070192114 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f497415c430 con 0x7f497414d1f0 2023-09-23T03:24:46.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.582+0000 7f4979f9c640 1 --2- 172.21.15.154:0/3070192114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f495c003f40 0x7f49740c1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:24:46.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.582+0000 7f497979b640 1 --2- 172.21.15.154:0/3070192114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49741574d0 0x7f49741f0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:24:46.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.583+0000 7f497a79d640 1 --2- 172.21.15.154:0/3070192114 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f497414d1f0 0x7f49740cac60 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f49740cf250 tx=0x7f496c010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:46.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.583+0000 7f49727fc640 1 -- 172.21.15.154:0/3070192114 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496c00e840 con 0x7f497414d1f0 2023-09-23T03:24:46.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.583+0000 7f49727fc640 1 -- 172.21.15.154:0/3070192114 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f496c00e9c0 con 0x7f497414d1f0 2023-09-23T03:24:46.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.583+0000 7f49727fc640 1 -- 172.21.15.154:0/3070192114 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496c00ed70 con 0x7f497414d1f0 2023-09-23T03:24:46.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.583+0000 7f497c200640 1 -- 172.21.15.154:0/3070192114 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f497415c710 con 0x7f497414d1f0 2023-09-23T03:24:46.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.584+0000 7f497c200640 1 -- 172.21.15.154:0/3070192114 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49740c4d20 con 0x7f497414d1f0 2023-09-23T03:24:46.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.585+0000 7f497c200640 1 -- 172.21.15.154:0/3070192114 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4974073eb0 con 0x7f497414d1f0 2023-09-23T03:24:46.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.586+0000 7f49727fc640 1 -- 172.21.15.154:0/3070192114 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f496c020410 con 0x7f497414d1f0 2023-09-23T03:24:46.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.587+0000 7f49727fc640 1 --2- 172.21.15.154:0/3070192114 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f496006af30 0x7f496006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:46.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.587+0000 7f49727fc640 1 -- 172.21.15.154:0/3070192114 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f496c089430 con 0x7f497414d1f0 2023-09-23T03:24:46.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.587+0000 7f4979f9c640 1 --2- 172.21.15.154:0/3070192114 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f496006af30 0x7f496006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:46.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.588+0000 7f4979f9c640 1 --2- 172.21.15.154:0/3070192114 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f496006af30 0x7f496006d3f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f49640017b0 tx=0x7f4964009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:46.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.591+0000 7f49727fc640 1 -- 172.21.15.154:0/3070192114 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f496c054140 con 0x7f497414d1f0 2023-09-23T03:24:46.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.819+0000 7f497c200640 1 -- 172.21.15.154:0/3070192114 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f497415a960 con 0x7f497414d1f0 2023-09-23T03:24:46.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.820+0000 7f49727fc640 1 -- 172.21.15.154:0/3070192114 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4928 (secure 0 0 0) 0x7f496c000e60 con 0x7f497414d1f0 2023-09-23T03:24:46.822 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:46.822 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":74,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7743,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/31599836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":31599836},{"type":"v1","addr":"172.21.15.154:6836","nonce":31599836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":7752,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/4266451911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":4266451911},{"type":"v1","addr":"172.21.15.154:6837","nonce":4266451911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":7755,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3029600822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3029600822},{"type":"v1","addr":"172.21.15.190:6834","nonce":3029600822}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:24:37.386034+0000","modified":"2023-09-23T03:24:45.444142+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":90,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7758},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7758":{"gid":7758,"name":"b","rank":0,"incarnation":73,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.190:6835/2202597679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2202597679},{"type":"v1","addr":"172.21.15.190:6835","nonce":2202597679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12,13],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-09-23T03:24:46.822 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 74 2023-09-23T03:24:46.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.823+0000 7f497c200640 1 -- 172.21.15.154:0/3070192114 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f496006af30 msgr2=0x7f496006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:46.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.823+0000 7f497c200640 1 --2- 172.21.15.154:0/3070192114 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f496006af30 0x7f496006d3f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f49640017b0 tx=0x7f4964009040 comp rx=0 tx=0).stop 2023-09-23T03:24:46.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.823+0000 7f497c200640 1 -- 172.21.15.154:0/3070192114 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f497414d1f0 msgr2=0x7f49740cac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:46.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.823+0000 7f497c200640 1 --2- 172.21.15.154:0/3070192114 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f497414d1f0 0x7f49740cac60 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f49740cf250 tx=0x7f496c010c20 comp rx=0 tx=0).stop 2023-09-23T03:24:46.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.823+0000 7f497c200640 1 -- 172.21.15.154:0/3070192114 shutdown_connections 2023-09-23T03:24:46.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.823+0000 7f497c200640 1 --2- 172.21.15.154:0/3070192114 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f496006af30 0x7f496006d3f0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:46.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.823+0000 7f497c200640 1 --2- 172.21.15.154:0/3070192114 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f497414d1f0 0x7f49740cac60 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:46.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.823+0000 7f497c200640 1 --2- 172.21.15.154:0/3070192114 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f49741574d0 0x7f49741f0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:46.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.823+0000 7f497c200640 1 --2- 172.21.15.154:0/3070192114 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f495c003f40 0x7f49740c1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:46.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.823+0000 7f497c200640 1 -- 172.21.15.154:0/3070192114 >> 172.21.15.154:0/3070192114 conn(0x7f497406a0f0 msgr2=0x7f49740c9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:46.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.823+0000 7f497c200640 1 -- 172.21.15.154:0/3070192114 shutdown_connections 2023-09-23T03:24:46.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:46.823+0000 7f497c200640 1 -- 172.21.15.154:0/3070192114 wait complete. 2023-09-23T03:24:47.246 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-23T03:24:37.386034+0000', 'modified': '2023-09-23T03:24:45.444142+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 90, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7758}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7758': {'gid': 7758, 'name': 'b', 'rank': 0, 'incarnation': 73, 'state': 'up:reconnect', 'state_seq': 2, 'addr': '172.21.15.190:6835/2202597679', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6833', 'nonce': 2202597679}, {'type': 'v1', 'addr': '172.21.15.190:6835', 'nonce': 2202597679}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12, 13], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-23T03:24:47.246 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7758:up:reconnect 2023-09-23T03:24:48.247 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:24:48.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.471+0000 7fb95cc1a640 1 Processor -- start 2023-09-23T03:24:48.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.471+0000 7fb95cc1a640 1 -- start start 2023-09-23T03:24:48.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.472+0000 7fb95cc1a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb9580b1b20 0x7fb9580ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:48.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.472+0000 7fb95cc1a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb9580cee70 0x7fb9580ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:48.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.473+0000 7fb95cc1a640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb95815ac30 con 0x7fb9580acce0 2023-09-23T03:24:48.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.473+0000 7fb95cc1a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9580b2e90 con 0x7fb9580cee70 2023-09-23T03:24:48.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.473+0000 7fb95cc1a640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9580cf320 con 0x7fb9580b1b20 2023-09-23T03:24:48.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.473+0000 7fb956d9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb9580cee70 0x7fb9580ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:48.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.473+0000 7fb956d9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb9580cee70 0x7fb9580ab140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52542/0 (socket says 172.21.15.154:52542) 2023-09-23T03:24:48.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.473+0000 7fb956d9d640 1 -- 172.21.15.154:0/411059980 learned_addr learned my addr 172.21.15.154:0/411059980 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:48.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.473+0000 7fb94ffff640 1 --2- 172.21.15.154:0/411059980 >> v2:172.21.15.190:3300/0 conn(0x7fb9580b1b20 0x7fb9580ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:48.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.473+0000 7fb956d9d640 1 -- 172.21.15.154:0/411059980 >> v2:172.21.15.190:3300/0 conn(0x7fb9580b1b20 msgr2=0x7fb9580ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:48.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.473+0000 7fb956d9d640 1 --2- 172.21.15.154:0/411059980 >> v2:172.21.15.190:3300/0 conn(0x7fb9580b1b20 0x7fb9580ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:48.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.473+0000 7fb956d9d640 1 -- 172.21.15.154:0/411059980 >> v1:172.21.15.190:6789/0 conn(0x7fb9580acce0 legacy=0x7fb95815ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:48.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.473+0000 7fb956d9d640 1 -- 172.21.15.154:0/411059980 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9580b1f70 con 0x7fb9580cee70 2023-09-23T03:24:48.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.474+0000 7fb956d9d640 1 --2- 172.21.15.154:0/411059980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9580cee70 0x7fb9580ab140 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fb9580cbff0 tx=0x7fb95002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d32e801656fff31d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:48.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.475+0000 7fb955d9b640 1 -- 172.21.15.154:0/411059980 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95002c9f0 con 0x7fb9580cee70 2023-09-23T03:24:48.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.475+0000 7fb955d9b640 1 -- 172.21.15.154:0/411059980 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb95002cb50 con 0x7fb9580cee70 2023-09-23T03:24:48.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.475+0000 7fb955d9b640 1 -- 172.21.15.154:0/411059980 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9500343f0 con 0x7fb9580cee70 2023-09-23T03:24:48.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.475+0000 7fb95cc1a640 1 -- 172.21.15.154:0/411059980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9580cee70 msgr2=0x7fb9580ab140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:48.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.475+0000 7fb95cc1a640 1 --2- 172.21.15.154:0/411059980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9580cee70 0x7fb9580ab140 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fb9580cbff0 tx=0x7fb95002be50 comp rx=0 tx=0).stop 2023-09-23T03:24:48.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.476+0000 7fb95cc1a640 1 -- 172.21.15.154:0/411059980 shutdown_connections 2023-09-23T03:24:48.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.476+0000 7fb95cc1a640 1 --2- 172.21.15.154:0/411059980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9580cee70 0x7fb9580ab140 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:48.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.476+0000 7fb95cc1a640 1 --2- 172.21.15.154:0/411059980 >> v2:172.21.15.190:3300/0 conn(0x7fb9580b1b20 0x7fb9580ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:48.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.476+0000 7fb95cc1a640 1 -- 172.21.15.154:0/411059980 >> 172.21.15.154:0/411059980 conn(0x7fb95806a0f0 msgr2=0x7fb95806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:48.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.476+0000 7fb95cc1a640 1 -- 172.21.15.154:0/411059980 shutdown_connections 2023-09-23T03:24:48.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.476+0000 7fb95cc1a640 1 -- 172.21.15.154:0/411059980 wait complete. 2023-09-23T03:24:48.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.477+0000 7fb95cc1a640 1 Processor -- start 2023-09-23T03:24:48.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.478+0000 7fb95cc1a640 1 -- start start 2023-09-23T03:24:48.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.478+0000 7fb95cc1a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9580acce0 0x7fb958097900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:48.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.479+0000 7fb95cc1a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9580b1b20 0x7fb95815d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:48.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.479+0000 7fb95cc1a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9580cee70 0x7fb9580c55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:48.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.479+0000 7fb95cc1a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95815c340 con 0x7fb9580acce0 2023-09-23T03:24:48.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.479+0000 7fb95cc1a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95815d1a0 con 0x7fb9580cee70 2023-09-23T03:24:48.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.479+0000 7fb95cc1a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb95815c590 con 0x7fb9580b1b20 2023-09-23T03:24:48.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.479+0000 7fb95659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9580acce0 0x7fb958097900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:48.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.479+0000 7fb95659c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9580acce0 0x7fb958097900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52546/0 (socket says 172.21.15.154:52546) 2023-09-23T03:24:48.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.479+0000 7fb95659c640 1 -- 172.21.15.154:0/682310043 learned_addr learned my addr 172.21.15.154:0/682310043 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:48.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.479+0000 7fb94ffff640 1 --2- 172.21.15.154:0/682310043 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9580b1b20 0x7fb95815d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:48.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.479+0000 7fb95659c640 1 -- 172.21.15.154:0/682310043 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9580b1b20 msgr2=0x7fb95815d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:48.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.479+0000 7fb95659c640 1 --2- 172.21.15.154:0/682310043 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9580b1b20 0x7fb95815d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:48.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.479+0000 7fb95659c640 1 -- 172.21.15.154:0/682310043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9580cee70 msgr2=0x7fb9580c55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:48.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.479+0000 7fb95659c640 1 --2- 172.21.15.154:0/682310043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9580cee70 0x7fb9580c55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:48.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.479+0000 7fb95659c640 1 -- 172.21.15.154:0/682310043 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb950009b30 con 0x7fb9580acce0 2023-09-23T03:24:48.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.480+0000 7fb95659c640 1 --2- 172.21.15.154:0/682310043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9580acce0 0x7fb958097900 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fb948004640 tx=0x7fb94800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:48.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.480+0000 7fb94effd640 1 -- 172.21.15.154:0/682310043 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb948011790 con 0x7fb9580acce0 2023-09-23T03:24:48.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.480+0000 7fb94effd640 1 -- 172.21.15.154:0/682310043 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9480118f0 con 0x7fb9580acce0 2023-09-23T03:24:48.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.480+0000 7fb95cc1a640 1 -- 172.21.15.154:0/682310043 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb95815c870 con 0x7fb9580acce0 2023-09-23T03:24:48.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.480+0000 7fb94effd640 1 -- 172.21.15.154:0/682310043 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb948011ce0 con 0x7fb9580acce0 2023-09-23T03:24:48.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.480+0000 7fb95cc1a640 1 -- 172.21.15.154:0/682310043 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb95815ce30 con 0x7fb9580acce0 2023-09-23T03:24:48.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.481+0000 7fb95cc1a640 1 -- 172.21.15.154:0/682310043 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9580ad0c0 con 0x7fb9580acce0 2023-09-23T03:24:48.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.482+0000 7fb94effd640 1 -- 172.21.15.154:0/682310043 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fb94801c460 con 0x7fb9580acce0 2023-09-23T03:24:48.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.483+0000 7fb94effd640 1 --2- 172.21.15.154:0/682310043 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb93806af30 0x7fb93806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:48.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.483+0000 7fb94effd640 1 -- 172.21.15.154:0/682310043 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fb948089b10 con 0x7fb9580acce0 2023-09-23T03:24:48.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.483+0000 7fb94ffff640 1 --2- 172.21.15.154:0/682310043 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb93806af30 0x7fb93806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:48.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.484+0000 7fb94ffff640 1 --2- 172.21.15.154:0/682310043 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb93806af30 0x7fb93806d3f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb9580cd170 tx=0x7fb940008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:48.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.487+0000 7fb94effd640 1 -- 172.21.15.154:0/682310043 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fb9480546e0 con 0x7fb9580acce0 2023-09-23T03:24:48.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.712+0000 7fb95cc1a640 1 -- 172.21.15.154:0/682310043 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb958073eb0 con 0x7fb9580acce0 2023-09-23T03:24:48.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.713+0000 7fb94effd640 1 -- 172.21.15.154:0/682310043 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4925 (secure 0 0 0) 0x7fb948057d00 con 0x7fb9580acce0 2023-09-23T03:24:48.715 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:48.715 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":76,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7743,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/31599836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":31599836},{"type":"v1","addr":"172.21.15.154:6836","nonce":31599836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":7752,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/4266451911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":4266451911},{"type":"v1","addr":"172.21.15.154:6837","nonce":4266451911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":7755,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3029600822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3029600822},{"type":"v1","addr":"172.21.15.190:6834","nonce":3029600822}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:24:37.386034+0000","modified":"2023-09-23T03:24:47.439439+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":90,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7758},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7758":{"gid":7758,"name":"b","rank":0,"incarnation":73,"state":"up:active","state_seq":4,"addr":"172.21.15.190:6835/2202597679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2202597679},{"type":"v1","addr":"172.21.15.190:6835","nonce":2202597679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12,13],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-09-23T03:24:48.715 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 76 2023-09-23T03:24:48.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.716+0000 7fb95cc1a640 1 -- 172.21.15.154:0/682310043 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb93806af30 msgr2=0x7fb93806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:48.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.716+0000 7fb95cc1a640 1 --2- 172.21.15.154:0/682310043 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb93806af30 0x7fb93806d3f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb9580cd170 tx=0x7fb940008040 comp rx=0 tx=0).stop 2023-09-23T03:24:48.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.716+0000 7fb95cc1a640 1 -- 172.21.15.154:0/682310043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9580acce0 msgr2=0x7fb958097900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:48.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.716+0000 7fb95cc1a640 1 --2- 172.21.15.154:0/682310043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9580acce0 0x7fb958097900 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fb948004640 tx=0x7fb94800c8e0 comp rx=0 tx=0).stop 2023-09-23T03:24:48.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.716+0000 7fb95cc1a640 1 -- 172.21.15.154:0/682310043 shutdown_connections 2023-09-23T03:24:48.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.716+0000 7fb95cc1a640 1 --2- 172.21.15.154:0/682310043 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb93806af30 0x7fb93806d3f0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:48.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.716+0000 7fb95cc1a640 1 --2- 172.21.15.154:0/682310043 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9580cee70 0x7fb9580c55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:48.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.716+0000 7fb95cc1a640 1 --2- 172.21.15.154:0/682310043 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9580b1b20 0x7fb95815d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:48.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.716+0000 7fb95cc1a640 1 --2- 172.21.15.154:0/682310043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9580acce0 0x7fb958097900 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:48.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.716+0000 7fb95cc1a640 1 -- 172.21.15.154:0/682310043 >> 172.21.15.154:0/682310043 conn(0x7fb95806a0f0 msgr2=0x7fb9580cad70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:48.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.716+0000 7fb95cc1a640 1 -- 172.21.15.154:0/682310043 shutdown_connections 2023-09-23T03:24:48.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:48.716+0000 7fb95cc1a640 1 -- 172.21.15.154:0/682310043 wait complete. 2023-09-23T03:24:49.138 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 76, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-23T03:24:37.386034+0000', 'modified': '2023-09-23T03:24:47.439439+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 90, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7758}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7758': {'gid': 7758, 'name': 'b', 'rank': 0, 'incarnation': 73, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.190:6835/2202597679', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6833', 'nonce': 2202597679}, {'type': 'v1', 'addr': '172.21.15.190:6835', 'nonce': 2202597679}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12, 13], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-23T03:24:49.138 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-23T03:24:49.138 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-09-23T03:24:49.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.362+0000 7f0639155640 1 Processor -- start 2023-09-23T03:24:49.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.362+0000 7f0639155640 1 -- start start 2023-09-23T03:24:49.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.363+0000 7f0639155640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06340ada30 0x7f06340ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:49.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.363+0000 7f0639155640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f06340b2fe0 0x7f06340cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:49.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.363+0000 7f0639155640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f063406ceb0 con 0x7f06340cef60 2023-09-23T03:24:49.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.363+0000 7f0639155640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06340ade10 con 0x7f06340b2fe0 2023-09-23T03:24:49.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.363+0000 7f0639155640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06340cf3b0 con 0x7f06340ada30 2023-09-23T03:24:49.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.363+0000 7f063359e640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f06340cef60 0x7f06340b18f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:60318/0 (socket says 172.21.15.154:60318) 2023-09-23T03:24:49.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.363+0000 7f063359e640 1 -- 172.21.15.154:0/2398687653 learned_addr learned my addr 172.21.15.154:0/2398687653 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:24:49.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.364+0000 7f062bfff640 1 --2- 172.21.15.154:0/2398687653 >> v2:172.21.15.154:3300/0 conn(0x7f06340b2fe0 0x7f06340cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:49.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.364+0000 7f0632d9d640 1 --2- 172.21.15.154:0/2398687653 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06340ada30 0x7f06340ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:49.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.364+0000 7f063259c640 1 -- 172.21.15.154:0/2398687653 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2695634412 0 0) 0x7f062c001490 con 0x7f06340cef60 2023-09-23T03:24:49.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.364+0000 7f063259c640 1 -- 172.21.15.154:0/2398687653 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0620003520 con 0x7f06340cef60 2023-09-23T03:24:49.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.364+0000 7f062bfff640 1 -- 172.21.15.154:0/2398687653 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06340ada30 msgr2=0x7f06340ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:49.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.364+0000 7f062bfff640 1 --2- 172.21.15.154:0/2398687653 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06340ada30 0x7f06340ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.364+0000 7f062bfff640 1 -- 172.21.15.154:0/2398687653 >> v1:172.21.15.154:6789/0 conn(0x7f06340cef60 legacy=0x7f06340b18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:49.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.364+0000 7f062bfff640 1 -- 172.21.15.154:0/2398687653 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0634159c90 con 0x7f06340b2fe0 2023-09-23T03:24:49.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.365+0000 7f062bfff640 1 --2- 172.21.15.154:0/2398687653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06340b2fe0 0x7f06340cea20 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f061c0097a0 tx=0x7f061c02dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b5365ae332524a1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:49.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.366+0000 7f063259c640 1 -- 172.21.15.154:0/2398687653 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c02f8a0 con 0x7f06340b2fe0 2023-09-23T03:24:49.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.366+0000 7f063259c640 1 -- 172.21.15.154:0/2398687653 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f061c02fa00 con 0x7f06340b2fe0 2023-09-23T03:24:49.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.366+0000 7f063259c640 1 -- 172.21.15.154:0/2398687653 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c02fd70 con 0x7f06340b2fe0 2023-09-23T03:24:49.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.366+0000 7f0639155640 1 -- 172.21.15.154:0/2398687653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06340b2fe0 msgr2=0x7f06340cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:49.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.366+0000 7f0639155640 1 --2- 172.21.15.154:0/2398687653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06340b2fe0 0x7f06340cea20 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f061c0097a0 tx=0x7f061c02dae0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.366+0000 7f0639155640 1 -- 172.21.15.154:0/2398687653 shutdown_connections 2023-09-23T03:24:49.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.366+0000 7f0639155640 1 --2- 172.21.15.154:0/2398687653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06340b2fe0 0x7f06340cea20 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.366+0000 7f0639155640 1 --2- 172.21.15.154:0/2398687653 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06340ada30 0x7f06340ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.366+0000 7f0639155640 1 -- 172.21.15.154:0/2398687653 >> 172.21.15.154:0/2398687653 conn(0x7f063406a0f0 msgr2=0x7f063406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:49.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.367+0000 7f0639155640 1 -- 172.21.15.154:0/2398687653 shutdown_connections 2023-09-23T03:24:49.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.367+0000 7f0639155640 1 -- 172.21.15.154:0/2398687653 wait complete. 2023-09-23T03:24:49.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.368+0000 7f0639155640 1 Processor -- start 2023-09-23T03:24:49.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.368+0000 7f0639155640 1 -- start start 2023-09-23T03:24:49.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.369+0000 7f0639155640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06340ada30 0x7f06341547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:49.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.369+0000 7f0639155640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06340b2fe0 0x7f06341f1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:49.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.369+0000 7f0639155640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06340cef60 0x7f06341554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:49.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.369+0000 7f0639155640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06340c4540 con 0x7f06340cef60 2023-09-23T03:24:49.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.369+0000 7f0639155640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0634154d30 con 0x7f06340b2fe0 2023-09-23T03:24:49.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.369+0000 7f0639155640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06341f2460 con 0x7f06340ada30 2023-09-23T03:24:49.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.370+0000 7f063359e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06340cef60 0x7f06341554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:49.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.370+0000 7f063359e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06340cef60 0x7f06341554c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52560/0 (socket says 172.21.15.154:52560) 2023-09-23T03:24:49.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.370+0000 7f063359e640 1 -- 172.21.15.154:0/1242651121 learned_addr learned my addr 172.21.15.154:0/1242651121 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:49.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.370+0000 7f0632d9d640 1 --2- 172.21.15.154:0/1242651121 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06340ada30 0x7f06341547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:49.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.370+0000 7f062bfff640 1 --2- 172.21.15.154:0/1242651121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06340b2fe0 0x7f06341f1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:49.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.370+0000 7f063359e640 1 -- 172.21.15.154:0/1242651121 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06340ada30 msgr2=0x7f06341547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:49.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.370+0000 7f063359e640 1 --2- 172.21.15.154:0/1242651121 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06340ada30 0x7f06341547f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.370+0000 7f063359e640 1 -- 172.21.15.154:0/1242651121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06340b2fe0 msgr2=0x7f06341f1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:49.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.370+0000 7f063359e640 1 --2- 172.21.15.154:0/1242651121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06340b2fe0 0x7f06341f1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.370+0000 7f063359e640 1 -- 172.21.15.154:0/1242651121 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f061c009400 con 0x7f06340cef60 2023-09-23T03:24:49.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.370+0000 7f063359e640 1 --2- 172.21.15.154:0/1242651121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06340cef60 0x7f06341554c0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f062c00cb10 tx=0x7f062c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:49.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.371+0000 7f062b7fe640 1 -- 172.21.15.154:0/1242651121 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c00ac70 con 0x7f06340cef60 2023-09-23T03:24:49.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.371+0000 7f062b7fe640 1 -- 172.21.15.154:0/1242651121 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062c00add0 con 0x7f06340cef60 2023-09-23T03:24:49.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.371+0000 7f062b7fe640 1 -- 172.21.15.154:0/1242651121 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c012620 con 0x7f06340cef60 2023-09-23T03:24:49.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.371+0000 7f0639155640 1 -- 172.21.15.154:0/1242651121 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0634155a00 con 0x7f06340cef60 2023-09-23T03:24:49.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.371+0000 7f0639155640 1 -- 172.21.15.154:0/1242651121 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06340c3c10 con 0x7f06340cef60 2023-09-23T03:24:49.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.374+0000 7f062b7fe640 1 -- 172.21.15.154:0/1242651121 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f062c007070 con 0x7f06340cef60 2023-09-23T03:24:49.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.375+0000 7f062b7fe640 1 --2- 172.21.15.154:0/1242651121 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f061406af30 0x7f061406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:49.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.375+0000 7f062b7fe640 1 -- 172.21.15.154:0/1242651121 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f062c089a10 con 0x7f06340cef60 2023-09-23T03:24:49.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.375+0000 7f0632d9d640 1 --2- 172.21.15.154:0/1242651121 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f061406af30 0x7f061406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:49.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.376+0000 7f063bdbf740 1 Processor -- start 2023-09-23T03:24:49.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.376+0000 7f063bdbf740 1 -- start start 2023-09-23T03:24:49.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.376+0000 7f0632d9d640 1 --2- 172.21.15.154:0/1242651121 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f061406af30 0x7f061406d3f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f062400b9b0 tx=0x7f0624008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:49.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.377+0000 7f063bdbf740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607bc09c010 0x5607bc06ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:49.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.377+0000 7f063bdbf740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5607bc09c3f0 0x5607bc09eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:49.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.377+0000 7f063bdbf740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5607bc06f450 0x5607bc09f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:49.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.377+0000 7f063bdbf740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607bbfde6d0 con 0x5607bc09c010 2023-09-23T03:24:49.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.377+0000 7f063bdbf740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607bbfdbdc0 con 0x5607bc09c3f0 2023-09-23T03:24:49.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.377+0000 7f063bdbf740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5607bbfd76b0 con 0x5607bc06f450 2023-09-23T03:24:49.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.377+0000 7f0632d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607bc09c010 0x5607bc06ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:49.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.377+0000 7f0632d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607bc09c010 0x5607bc06ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52566/0 (socket says 172.21.15.154:52566) 2023-09-23T03:24:49.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.377+0000 7f0632d9d640 1 -- 172.21.15.154:0/2666687233 learned_addr learned my addr 172.21.15.154:0/2666687233 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:49.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.377+0000 7f0632d9d640 1 -- 172.21.15.154:0/2666687233 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5607bc06f450 msgr2=0x5607bc09f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:49.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.377+0000 7f0632d9d640 1 --2- 172.21.15.154:0/2666687233 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5607bc06f450 0x5607bc09f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.377+0000 7f062bfff640 1 --2- 172.21.15.154:0/2666687233 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5607bc09c3f0 0x5607bc09eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:49.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.377+0000 7f0632d9d640 1 -- 172.21.15.154:0/2666687233 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5607bc09c3f0 msgr2=0x5607bc09eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:49.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.377+0000 7f0632d9d640 1 --2- 172.21.15.154:0/2666687233 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5607bc09c3f0 0x5607bc09eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.377+0000 7f0632d9d640 1 -- 172.21.15.154:0/2666687233 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607bbf9b8e0 con 0x5607bc09c010 2023-09-23T03:24:49.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.378+0000 7f0632d9d640 1 --2- 172.21.15.154:0/2666687233 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607bc09c010 0x5607bc06ef10 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f062400fa90 tx=0x7f0624012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcc5b9e23f4971cc server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:49.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.378+0000 7f0628ff9640 1 -- 172.21.15.154:0/2666687233 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0624014be0 con 0x5607bc09c010 2023-09-23T03:24:49.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.378+0000 7f0628ff9640 1 -- 172.21.15.154:0/2666687233 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0624019070 con 0x5607bc09c010 2023-09-23T03:24:49.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.378+0000 7f0628ff9640 1 -- 172.21.15.154:0/2666687233 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0624020440 con 0x5607bc09c010 2023-09-23T03:24:49.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.379+0000 7f063bdbf740 1 -- 172.21.15.154:0/2666687233 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607bc09c010 msgr2=0x5607bc06ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:49.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.379+0000 7f063bdbf740 1 --2- 172.21.15.154:0/2666687233 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607bc09c010 0x5607bc06ef10 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f062400fa90 tx=0x7f0624012000 comp rx=0 tx=0).stop 2023-09-23T03:24:49.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.379+0000 7f063bdbf740 1 -- 172.21.15.154:0/2666687233 shutdown_connections 2023-09-23T03:24:49.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.379+0000 7f063bdbf740 1 --2- 172.21.15.154:0/2666687233 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5607bc06f450 0x5607bc09f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.379+0000 7f063bdbf740 1 --2- 172.21.15.154:0/2666687233 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5607bc09c3f0 0x5607bc09eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.379+0000 7f063bdbf740 1 --2- 172.21.15.154:0/2666687233 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607bc09c010 0x5607bc06ef10 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.379+0000 7f063bdbf740 1 -- 172.21.15.154:0/2666687233 >> 172.21.15.154:0/2666687233 conn(0x5607bc071220 msgr2=0x5607bc0b9da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:49.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.379+0000 7f063bdbf740 1 -- 172.21.15.154:0/2666687233 shutdown_connections 2023-09-23T03:24:49.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.379+0000 7f063bdbf740 1 -- 172.21.15.154:0/2666687233 wait complete. 2023-09-23T03:24:49.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.380+0000 7f063bdbf740 1 -- start start 2023-09-23T03:24:49.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.380+0000 7f063bdbf740 1 Processor -- start 2023-09-23T03:24:49.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.381+0000 7f063bdbf740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607bc09c010 0x5607bc0b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:49.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.381+0000 7f063bdbf740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5607bc09c3f0 0x5607bc0b7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:49.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.381+0000 7f063bdbf740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5607bc09b430 0x5607bc0ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:49.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.381+0000 7f063bdbf740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607bbf75940 con 0x5607bc09c010 2023-09-23T03:24:49.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.381+0000 7f063bdbf740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607bbf755e0 con 0x5607bc09c3f0 2023-09-23T03:24:49.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.381+0000 7f063bdbf740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5607bbd3bf20 con 0x5607bc09b430 2023-09-23T03:24:49.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.381+0000 7f063bdbf740 10 client.0 _ll_register_callbacks cb 0x5607bbfb35e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:24:49.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.381+0000 7f0632d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607bc09c010 0x5607bc0b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:49.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.381+0000 7f0632d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607bc09c010 0x5607bc0b78d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52574/0 (socket says 172.21.15.154:52574) 2023-09-23T03:24:49.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.381+0000 7f0632d9d640 1 -- 172.21.15.154:0/1277987688 learned_addr learned my addr 172.21.15.154:0/1277987688 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:49.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.382+0000 7f0632d9d640 1 -- 172.21.15.154:0/1277987688 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5607bc09b430 msgr2=0x5607bc0ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:49.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.382+0000 7f0632d9d640 1 --2- 172.21.15.154:0/1277987688 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5607bc09b430 0x5607bc0ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.382+0000 7f0632d9d640 1 -- 172.21.15.154:0/1277987688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5607bc09c3f0 msgr2=0x5607bc0b7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:49.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.382+0000 7f0632d9d640 1 --2- 172.21.15.154:0/1277987688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5607bc09c3f0 0x5607bc0b7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.382+0000 7f0632d9d640 1 -- 172.21.15.154:0/1277987688 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f062400e040 con 0x5607bc09c010 2023-09-23T03:24:49.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.382+0000 7f0632d9d640 1 --2- 172.21.15.154:0/1277987688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607bc09c010 0x5607bc0b78d0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f0624014830 tx=0x7f0624004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d583e1a43499d28e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:49.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.382+0000 7f0611ffb640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:24:49.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.382+0000 7f0611ffb640 1 -- 172.21.15.154:0/1277987688 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0624020cb0 con 0x5607bc09c010 2023-09-23T03:24:49.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.382+0000 7f0611ffb640 1 -- 172.21.15.154:0/1277987688 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f0624020e10 con 0x5607bc09c010 2023-09-23T03:24:49.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.382+0000 7f063bdbf740 1 -- 172.21.15.154:0/1277987688 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5607bbfd0690 con 0x5607bc09c010 2023-09-23T03:24:49.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.382+0000 7f0611ffb640 1 -- 172.21.15.154:0/1277987688 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0624014e90 con 0x5607bc09c010 2023-09-23T03:24:49.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.382+0000 7f063bdbf740 10 client.7791 fetch_fsmap learned FSMap version 76 2023-09-23T03:24:49.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.382+0000 7f063bdbf740 1 -- 172.21.15.154:0/1277987688 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x5607bbfd5d30 con 0x5607bc09c010 2023-09-23T03:24:49.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.383+0000 7f0611ffb640 1 -- 172.21.15.154:0/1277987688 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5607bbfd0690 con 0x5607bc09c010 2023-09-23T03:24:49.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.383+0000 7f0611ffb640 1 -- 172.21.15.154:0/1277987688 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 76) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f062403e470 con 0x5607bc09c010 2023-09-23T03:24:49.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.383+0000 7f063bdbf740 10 client.7791 fetch_fsmap finished waiting for FSMap version 76 2023-09-23T03:24:49.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.383+0000 7f063bdbf740 10 client.7791 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-23T03:24:49.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.383+0000 7f063bdbf740 20 client.7791 populate_metadata read hostname 'smithi154' 2023-09-23T03:24:49.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.383+0000 7f063bdbf740 1 --2- 172.21.15.154:0/1277987688 >> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] conn(0x5607bc0cd980 0x5607bc0d71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:49.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.383+0000 7f063bdbf740 4 client.7791 mds_command: new command op to 7758 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T03:24:49.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.383+0000 7f063bdbf740 1 -- 172.21.15.154:0/1277987688 --> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5607bbfd03b0 con 0x5607bc0cd980 2023-09-23T03:24:49.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.384+0000 7f062bfff640 1 --2- 172.21.15.154:0/1277987688 >> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] conn(0x5607bc0cd980 0x5607bc0d71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:49.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.385+0000 7f062bfff640 1 --2- 172.21.15.154:0/1277987688 >> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] conn(0x5607bc0cd980 0x5607bc0d71d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3447d7095fcebe2c server_cookie=e591b404eb405610 in_seq=0 out_seq=0 2023-09-23T03:24:49.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.385+0000 7f0611ffb640 10 client.7791 ms_handle_connect on v2:172.21.15.190:6833/2202597679 2023-09-23T03:24:49.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.387+0000 7f0611ffb640 1 -- 172.21.15.154:0/1277987688 <== mds.0 v2:172.21.15.190:6833/2202597679 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5607bbfd03b0 con 0x5607bc0cd980 2023-09-23T03:24:49.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.387+0000 7f0611ffb640 10 client.7791 handle_command_reply: tid=0 2023-09-23T03:24:49.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.387+0000 7f063bdbf740 1 client.7791 shutdown 2023-09-23T03:24:49.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.387+0000 7f063bdbf740 1 -- 172.21.15.154:0/1277987688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607bc09c010 msgr2=0x5607bc0b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:49.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.387+0000 7f063bdbf740 1 --2- 172.21.15.154:0/1277987688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607bc09c010 0x5607bc0b78d0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f0624014830 tx=0x7f0624004730 comp rx=0 tx=0).stop 2023-09-23T03:24:49.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.387+0000 7f063bdbf740 1 -- 172.21.15.154:0/1277987688 shutdown_connections 2023-09-23T03:24:49.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.387+0000 7f063bdbf740 1 --2- 172.21.15.154:0/1277987688 >> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] conn(0x5607bc0cd980 0x5607bc0d71d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.387+0000 7f063bdbf740 1 --2- 172.21.15.154:0/1277987688 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5607bc09b430 0x5607bc0ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.387+0000 7f063bdbf740 1 --2- 172.21.15.154:0/1277987688 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5607bc09c3f0 0x5607bc0b7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.387+0000 7f063bdbf740 1 --2- 172.21.15.154:0/1277987688 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607bc09c010 0x5607bc0b78d0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.387+0000 7f0611ffb640 0 client.7791 ms_handle_reset on v2:172.21.15.190:6833/2202597679 2023-09-23T03:24:49.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.387+0000 7f063bdbf740 1 -- 172.21.15.154:0/1277987688 >> 172.21.15.154:0/1277987688 conn(0x5607bc06f450 msgr2=0x5607bc072e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:49.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.387+0000 7f063bdbf740 1 -- 172.21.15.154:0/1277987688 shutdown_connections 2023-09-23T03:24:49.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.387+0000 7f063bdbf740 1 -- 172.21.15.154:0/1277987688 wait complete. 2023-09-23T03:24:49.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.387+0000 7f063bdbf740 20 client.7791 trim_cache size 0 max 16384 2023-09-23T03:24:49.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.404+0000 7f063bdbf740 1 Processor -- start 2023-09-23T03:24:49.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.404+0000 7f063bdbf740 1 -- start start 2023-09-23T03:24:49.406 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.404+0000 7f063bdbf740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5607bc09b430 0x5607bc0a2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:49.406 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.404+0000 7f063bdbf740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607bc09c010 0x5607bc0cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:49.406 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.404+0000 7f063bdbf740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5607bc09c3f0 0x5607bc120960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:49.406 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.404+0000 7f063bdbf740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607bc09a870 con 0x5607bc09c010 2023-09-23T03:24:49.406 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.404+0000 7f063bdbf740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607bc074be0 con 0x5607bc09b430 2023-09-23T03:24:49.406 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.404+0000 7f063bdbf740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5607bbfd9590 con 0x5607bc09c3f0 2023-09-23T03:24:49.406 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.405+0000 7f062bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607bc09c010 0x5607bc0cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:49.406 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.405+0000 7f062bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607bc09c010 0x5607bc0cce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52588/0 (socket says 172.21.15.154:52588) 2023-09-23T03:24:49.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.405+0000 7f062bfff640 1 -- 172.21.15.154:0/421921222 learned_addr learned my addr 172.21.15.154:0/421921222 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:49.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.405+0000 7f062bfff640 1 -- 172.21.15.154:0/421921222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5607bc09c3f0 msgr2=0x5607bc120960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:49.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.405+0000 7f063359e640 1 --2- 172.21.15.154:0/421921222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5607bc09c3f0 0x5607bc120960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:49.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.405+0000 7f062bfff640 1 --2- 172.21.15.154:0/421921222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5607bc09c3f0 0x5607bc120960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.405+0000 7f062bfff640 1 -- 172.21.15.154:0/421921222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5607bc09b430 msgr2=0x5607bc0a2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:49.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.405+0000 7f062bfff640 1 --2- 172.21.15.154:0/421921222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5607bc09b430 0x5607bc0a2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.405+0000 7f062bfff640 1 -- 172.21.15.154:0/421921222 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607bbf74b50 con 0x5607bc09c010 2023-09-23T03:24:49.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.405+0000 7f062bfff640 1 --2- 172.21.15.154:0/421921222 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607bc09c010 0x5607bc0cce70 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f061c0097a0 tx=0x7f061c030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecb90bfa9ea26add server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:49.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.406+0000 7f06297fa640 1 -- 172.21.15.154:0/421921222 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c03d040 con 0x5607bc09c010 2023-09-23T03:24:49.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.406+0000 7f06297fa640 1 -- 172.21.15.154:0/421921222 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f061c02fb70 con 0x5607bc09c010 2023-09-23T03:24:49.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.406+0000 7f06297fa640 1 -- 172.21.15.154:0/421921222 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c047a60 con 0x5607bc09c010 2023-09-23T03:24:49.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.406+0000 7f063bdbf740 1 -- 172.21.15.154:0/421921222 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607bc09c010 msgr2=0x5607bc0cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:49.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.406+0000 7f063bdbf740 1 --2- 172.21.15.154:0/421921222 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607bc09c010 0x5607bc0cce70 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f061c0097a0 tx=0x7f061c030000 comp rx=0 tx=0).stop 2023-09-23T03:24:49.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.406+0000 7f063bdbf740 1 -- 172.21.15.154:0/421921222 shutdown_connections 2023-09-23T03:24:49.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.406+0000 7f063bdbf740 1 --2- 172.21.15.154:0/421921222 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5607bc09c3f0 0x5607bc120960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.406+0000 7f063bdbf740 1 --2- 172.21.15.154:0/421921222 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607bc09c010 0x5607bc0cce70 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.406+0000 7f063bdbf740 1 --2- 172.21.15.154:0/421921222 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5607bc09b430 0x5607bc0a2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.406+0000 7f063bdbf740 1 -- 172.21.15.154:0/421921222 >> 172.21.15.154:0/421921222 conn(0x5607bc06f450 msgr2=0x5607bc074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:49.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.406+0000 7f063bdbf740 1 -- 172.21.15.154:0/421921222 shutdown_connections 2023-09-23T03:24:49.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.406+0000 7f063bdbf740 1 -- 172.21.15.154:0/421921222 wait complete. 2023-09-23T03:24:49.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.407+0000 7f063bdbf740 1 -- start start 2023-09-23T03:24:49.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.407+0000 7f063bdbf740 1 Processor -- start 2023-09-23T03:24:49.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.408+0000 7f063bdbf740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5607bc09b430 0x5607bc1095b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:49.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.408+0000 7f063bdbf740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607bc09c010 0x5607bc089270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:49.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.408+0000 7f063bdbf740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5607bc09c3f0 0x5607bc0b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:49.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.408+0000 7f063bdbf740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607bc0c88b0 con 0x5607bc09c010 2023-09-23T03:24:49.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.408+0000 7f063bdbf740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607bc10c130 con 0x5607bc09c3f0 2023-09-23T03:24:49.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.408+0000 7f063bdbf740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5607bc10cc20 con 0x5607bc09b430 2023-09-23T03:24:49.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.408+0000 7f063bdbf740 10 client.0 _ll_register_callbacks cb 0x7f062400b1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:24:49.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.409+0000 7f062bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607bc09c010 0x5607bc089270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:49.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.409+0000 7f062bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607bc09c010 0x5607bc089270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52592/0 (socket says 172.21.15.154:52592) 2023-09-23T03:24:49.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.409+0000 7f062bfff640 1 -- 172.21.15.154:0/2388443203 learned_addr learned my addr 172.21.15.154:0/2388443203 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:49.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.409+0000 7f063359e640 1 --2- 172.21.15.154:0/2388443203 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5607bc09c3f0 0x5607bc0b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:49.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.409+0000 7f062bfff640 1 -- 172.21.15.154:0/2388443203 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5607bc09b430 msgr2=0x5607bc1095b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:49.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.409+0000 7f062bfff640 1 --2- 172.21.15.154:0/2388443203 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5607bc09b430 0x5607bc1095b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.409+0000 7f062bfff640 1 -- 172.21.15.154:0/2388443203 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5607bc09c3f0 msgr2=0x5607bc0b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:49.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.409+0000 7f062bfff640 1 --2- 172.21.15.154:0/2388443203 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5607bc09c3f0 0x5607bc0b3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.409+0000 7f062bfff640 1 -- 172.21.15.154:0/2388443203 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0634159c90 con 0x5607bc09c010 2023-09-23T03:24:49.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.409+0000 7f062bfff640 1 --2- 172.21.15.154:0/2388443203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607bc09c010 0x5607bc089270 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f061c02e0c0 tx=0x7f061c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e87e9cd600848 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:49.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.409+0000 7f0611ffb640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:24:49.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.410+0000 7f0611ffb640 1 -- 172.21.15.154:0/2388443203 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c03d040 con 0x5607bc09c010 2023-09-23T03:24:49.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.410+0000 7f0611ffb640 1 -- 172.21.15.154:0/2388443203 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f061c047a60 con 0x5607bc09c010 2023-09-23T03:24:49.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.410+0000 7f063bdbf740 1 -- 172.21.15.154:0/2388443203 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5607bc09b2c0 con 0x5607bc09c010 2023-09-23T03:24:49.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.410+0000 7f0611ffb640 1 -- 172.21.15.154:0/2388443203 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f061c047c20 con 0x5607bc09c010 2023-09-23T03:24:49.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.410+0000 7f063bdbf740 10 client.7797 fetch_fsmap learned FSMap version 76 2023-09-23T03:24:49.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.410+0000 7f063bdbf740 1 -- 172.21.15.154:0/2388443203 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x5607bbf9a140 con 0x5607bc09c010 2023-09-23T03:24:49.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.410+0000 7f0611ffb640 1 -- 172.21.15.154:0/2388443203 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5607bc09b2c0 con 0x5607bc09c010 2023-09-23T03:24:49.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.410+0000 7f0611ffb640 1 -- 172.21.15.154:0/2388443203 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 76) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f061c04f030 con 0x5607bc09c010 2023-09-23T03:24:49.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.410+0000 7f063bdbf740 10 client.7797 fetch_fsmap finished waiting for FSMap version 76 2023-09-23T03:24:49.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.410+0000 7f063bdbf740 10 client.7797 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-23T03:24:49.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.410+0000 7f063bdbf740 20 client.7797 populate_metadata read hostname 'smithi154' 2023-09-23T03:24:49.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.410+0000 7f063bdbf740 1 --2- 172.21.15.154:0/2388443203 >> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] conn(0x5607bc0b9e30 0x5607bc0ba210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:49.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.410+0000 7f063bdbf740 4 client.7797 mds_command: new command op to 7758 tid=0[{"prefix": "status"}] 2023-09-23T03:24:49.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.410+0000 7f063bdbf740 1 -- 172.21.15.154:0/2388443203 --> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f0624019070 con 0x5607bc0b9e30 2023-09-23T03:24:49.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.411+0000 7f0632d9d640 1 --2- 172.21.15.154:0/2388443203 >> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] conn(0x5607bc0b9e30 0x5607bc0ba210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:49.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.412+0000 7f0632d9d640 1 --2- 172.21.15.154:0/2388443203 >> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] conn(0x5607bc0b9e30 0x5607bc0ba210 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c7b410ccdead3a0 server_cookie=ce70de396e5b0d5e in_seq=0 out_seq=0 2023-09-23T03:24:49.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.412+0000 7f0611ffb640 10 client.7797 ms_handle_connect on v2:172.21.15.190:6833/2202597679 2023-09-23T03:24:49.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.412+0000 7f0611ffb640 1 -- 172.21.15.154:0/2388443203 <== mds.0 v2:172.21.15.190:6833/2202597679 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x7f0624019070 con 0x5607bc0b9e30 2023-09-23T03:24:49.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.412+0000 7f0611ffb640 10 client.7797 handle_command_reply: tid=0 2023-09-23T03:24:49.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.413+0000 7f063bdbf740 1 client.7797 shutdown 2023-09-23T03:24:49.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.413+0000 7f063bdbf740 1 -- 172.21.15.154:0/2388443203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607bc09c010 msgr2=0x5607bc089270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:49.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.413+0000 7f063bdbf740 1 --2- 172.21.15.154:0/2388443203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607bc09c010 0x5607bc089270 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f061c02e0c0 tx=0x7f061c004730 comp rx=0 tx=0).stop 2023-09-23T03:24:49.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.413+0000 7f063bdbf740 1 -- 172.21.15.154:0/2388443203 shutdown_connections 2023-09-23T03:24:49.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.413+0000 7f063bdbf740 1 --2- 172.21.15.154:0/2388443203 >> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] conn(0x5607bc0b9e30 0x5607bc0ba210 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.413+0000 7f063bdbf740 1 --2- 172.21.15.154:0/2388443203 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5607bc09c3f0 0x5607bc0b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.413+0000 7f063bdbf740 1 --2- 172.21.15.154:0/2388443203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5607bc09c010 0x5607bc089270 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.413+0000 7f063bdbf740 1 --2- 172.21.15.154:0/2388443203 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5607bc09b430 0x5607bc1095b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.413+0000 7f063bdbf740 1 -- 172.21.15.154:0/2388443203 >> 172.21.15.154:0/2388443203 conn(0x5607bc06f450 msgr2=0x5607bc0db510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:49.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.413+0000 7f0611ffb640 0 client.7797 ms_handle_reset on v2:172.21.15.190:6833/2202597679 2023-09-23T03:24:49.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.413+0000 7f063bdbf740 1 -- 172.21.15.154:0/2388443203 shutdown_connections 2023-09-23T03:24:49.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.413+0000 7f063bdbf740 1 -- 172.21.15.154:0/2388443203 wait complete. 2023-09-23T03:24:49.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.413+0000 7f063bdbf740 20 client.7797 trim_cache size 0 max 16384 2023-09-23T03:24:49.417 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:24:49.417 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_fsid": "8c8039d3-6c84-43db-8808-85e0dbdd47c8", 2023-09-23T03:24:49.418 INFO:teuthology.orchestra.run.smithi154.stdout: "whoami": 0, 2023-09-23T03:24:49.418 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 7758, 2023-09-23T03:24:49.418 INFO:teuthology.orchestra.run.smithi154.stdout: "want_state": "up:active", 2023-09-23T03:24:49.418 INFO:teuthology.orchestra.run.smithi154.stdout: "state": "up:active", 2023-09-23T03:24:49.418 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs", 2023-09-23T03:24:49.418 INFO:teuthology.orchestra.run.smithi154.stdout: "rank_uptime": 3.991111852, 2023-09-23T03:24:49.418 INFO:teuthology.orchestra.run.smithi154.stdout: "mdsmap_epoch": 76, 2023-09-23T03:24:49.418 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_epoch": 92, 2023-09-23T03:24:49.418 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_epoch_barrier": 92, 2023-09-23T03:24:49.418 INFO:teuthology.orchestra.run.smithi154.stdout: "uptime": 4.7535953839999996 2023-09-23T03:24:49.418 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:24:49.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.414+0000 7f0639155640 1 -- 172.21.15.154:0/1242651121 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f061406af30 msgr2=0x7f061406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:49.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.414+0000 7f0639155640 1 --2- 172.21.15.154:0/1242651121 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f061406af30 0x7f061406d3f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f062400b9b0 tx=0x7f0624008040 comp rx=0 tx=0).stop 2023-09-23T03:24:49.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.414+0000 7f0639155640 1 -- 172.21.15.154:0/1242651121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06340cef60 msgr2=0x7f06341554c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:49.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.414+0000 7f0639155640 1 --2- 172.21.15.154:0/1242651121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06340cef60 0x7f06341554c0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f062c00cb10 tx=0x7f062c00a060 comp rx=0 tx=0).stop 2023-09-23T03:24:49.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.414+0000 7f0639155640 1 -- 172.21.15.154:0/1242651121 shutdown_connections 2023-09-23T03:24:49.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.414+0000 7f0639155640 1 --2- 172.21.15.154:0/1242651121 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f061406af30 0x7f061406d3f0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.414+0000 7f0639155640 1 --2- 172.21.15.154:0/1242651121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06340cef60 0x7f06341554c0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.414+0000 7f0639155640 1 --2- 172.21.15.154:0/1242651121 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f06340b2fe0 0x7f06341f1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.414+0000 7f0639155640 1 --2- 172.21.15.154:0/1242651121 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f06340ada30 0x7f06341547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.414+0000 7f0639155640 1 -- 172.21.15.154:0/1242651121 >> 172.21.15.154:0/1242651121 conn(0x7f063406a0f0 msgr2=0x7f06340af240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:49.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.414+0000 7f0639155640 1 -- 172.21.15.154:0/1242651121 shutdown_connections 2023-09-23T03:24:49.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.414+0000 7f0639155640 1 -- 172.21.15.154:0/1242651121 wait complete. 2023-09-23T03:24:49.432 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:24:49.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.643+0000 7fd076602640 1 Processor -- start 2023-09-23T03:24:49.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.643+0000 7fd076602640 1 -- start start 2023-09-23T03:24:49.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.645+0000 7fd076602640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd07015eb00 0x7fd07006fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:49.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.645+0000 7fd076602640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd070070750 con 0x7fd07006c7a0 2023-09-23T03:24:49.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.645+0000 7fd076602640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd07006bff0 con 0x7fd070070300 2023-09-23T03:24:49.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.645+0000 7fd076602640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd070155070 con 0x7fd07015eb00 2023-09-23T03:24:49.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.645+0000 7fd075600640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fd07006c7a0 0x7fd07015ab00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:60324/0 (socket says 172.21.15.154:60324) 2023-09-23T03:24:49.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.645+0000 7fd075600640 1 -- 172.21.15.154:0/89087257 learned_addr learned my addr 172.21.15.154:0/89087257 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:24:49.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.646+0000 7fd06ffff640 1 -- 172.21.15.154:0/89087257 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3149275615 0 0) 0x7fd060001490 con 0x7fd07006c7a0 2023-09-23T03:24:49.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.646+0000 7fd074dff640 1 --2- 172.21.15.154:0/89087257 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd07015eb00 0x7fd07006fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:49.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.646+0000 7fd06ffff640 1 -- 172.21.15.154:0/89087257 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd058003520 con 0x7fd07006c7a0 2023-09-23T03:24:49.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.646+0000 7fd06ffff640 1 -- 172.21.15.154:0/89087257 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1414693564 0 0) 0x7fd058003520 con 0x7fd07006c7a0 2023-09-23T03:24:49.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.646+0000 7fd06ffff640 1 -- 172.21.15.154:0/89087257 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd07015eb00 msgr2=0x7fd07006fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:49.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.646+0000 7fd06ffff640 1 --2- 172.21.15.154:0/89087257 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd07015eb00 0x7fd07006fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.646+0000 7fd06ffff640 1 -- 172.21.15.154:0/89087257 >> v1:172.21.15.190:6789/0 conn(0x7fd070070300 legacy=0x7fd0700ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:49.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.646+0000 7fd06ffff640 1 -- 172.21.15.154:0/89087257 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd07006cb80 con 0x7fd07006c7a0 2023-09-23T03:24:49.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.646+0000 7fd06ffff640 1 -- 172.21.15.154:0/89087257 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fd060001b80 con 0x7fd07006c7a0 2023-09-23T03:24:49.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.646+0000 7fd06ffff640 1 -- 172.21.15.154:0/89087257 >> v1:172.21.15.154:6789/0 conn(0x7fd07006c7a0 legacy=0x7fd07015ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:49.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.646+0000 7fd06ffff640 1 --2- 172.21.15.154:0/89087257 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd07015eb00 0x7fd0580039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:49.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.646+0000 7fd06ffff640 1 --2- 172.21.15.154:0/89087257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd058003fd0 0x7fd0580047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:49.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.646+0000 7fd06ffff640 1 --2- 172.21.15.154:0/89087257 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd058004d00 0x7fd058005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:49.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.646+0000 7fd06ffff640 1 -- 172.21.15.154:0/89087257 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05800bb10 con 0x7fd058004d00 2023-09-23T03:24:49.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.646+0000 7fd06ffff640 1 -- 172.21.15.154:0/89087257 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05800bc80 con 0x7fd058003fd0 2023-09-23T03:24:49.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.646+0000 7fd06ffff640 1 -- 172.21.15.154:0/89087257 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd05800be40 con 0x7fd07015eb00 2023-09-23T03:24:49.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.647+0000 7fd075e01640 1 --2- 172.21.15.154:0/89087257 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd058004d00 0x7fd058005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:49.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.647+0000 7fd075600640 1 --2- 172.21.15.154:0/89087257 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd07015eb00 0x7fd0580039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:49.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.647+0000 7fd074dff640 1 --2- 172.21.15.154:0/89087257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd058003fd0 0x7fd0580047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:49.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.647+0000 7fd075e01640 1 -- 172.21.15.154:0/89087257 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd07015eb00 msgr2=0x7fd0580039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:49.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.647+0000 7fd075e01640 1 --2- 172.21.15.154:0/89087257 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd07015eb00 0x7fd0580039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.647+0000 7fd075e01640 1 -- 172.21.15.154:0/89087257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd058003fd0 msgr2=0x7fd0580047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:49.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.647+0000 7fd075e01640 1 --2- 172.21.15.154:0/89087257 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd058003fd0 0x7fd0580047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.647+0000 7fd075e01640 1 -- 172.21.15.154:0/89087257 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd058003520 con 0x7fd058004d00 2023-09-23T03:24:49.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.648+0000 7fd075e01640 1 --2- 172.21.15.154:0/89087257 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd058004d00 0x7fd058005180 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fd068001750 tx=0x7fd0580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d47640e1ed6c7ca server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:49.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.648+0000 7fd06ffff640 1 -- 172.21.15.154:0/89087257 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0680052d0 con 0x7fd058004d00 2023-09-23T03:24:49.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.648+0000 7fd06ffff640 1 -- 172.21.15.154:0/89087257 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06802e050 con 0x7fd058004d00 2023-09-23T03:24:49.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.648+0000 7fd06ffff640 1 -- 172.21.15.154:0/89087257 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd068003830 con 0x7fd058004d00 2023-09-23T03:24:49.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.649+0000 7fd076602640 1 -- 172.21.15.154:0/89087257 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd058004d00 msgr2=0x7fd058005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:49.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.649+0000 7fd076602640 1 --2- 172.21.15.154:0/89087257 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd058004d00 0x7fd058005180 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fd068001750 tx=0x7fd0580057f0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.649+0000 7fd075e01640 1 -- 172.21.15.154:0/89087257 reap_dead start 2023-09-23T03:24:49.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.649+0000 7fd076602640 1 -- 172.21.15.154:0/89087257 shutdown_connections 2023-09-23T03:24:49.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.649+0000 7fd076602640 1 -- 172.21.15.154:0/89087257 >> 172.21.15.154:0/89087257 conn(0x7fd07006a390 msgr2=0x7fd0700afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:49.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.649+0000 7fd076602640 1 -- 172.21.15.154:0/89087257 shutdown_connections 2023-09-23T03:24:49.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.649+0000 7fd076602640 1 -- 172.21.15.154:0/89087257 wait complete. 2023-09-23T03:24:49.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.651+0000 7fd076602640 1 Processor -- start 2023-09-23T03:24:49.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.651+0000 7fd076602640 1 -- start start 2023-09-23T03:24:49.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.652+0000 7fd076602640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd058004d00 0x7fd0700cced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:49.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.652+0000 7fd076602640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0701f1650 0x7fd0700ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:49.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.652+0000 7fd076602640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0701f1a90 0x7fd0700ca830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:49.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.652+0000 7fd076602640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0700cd410 con 0x7fd0701f1650 2023-09-23T03:24:49.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.652+0000 7fd076602640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0700c6340 con 0x7fd058004d00 2023-09-23T03:24:49.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.652+0000 7fd076602640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0700c5730 con 0x7fd0701f1a90 2023-09-23T03:24:49.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.652+0000 7fd074dff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0701f1650 0x7fd0700ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:49.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.652+0000 7fd074dff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0701f1650 0x7fd0700ca2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52618/0 (socket says 172.21.15.154:52618) 2023-09-23T03:24:49.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.652+0000 7fd074dff640 1 -- 172.21.15.154:0/2137443733 learned_addr learned my addr 172.21.15.154:0/2137443733 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:49.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.653+0000 7fd075e01640 1 --2- 172.21.15.154:0/2137443733 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0701f1a90 0x7fd0700ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:49.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.653+0000 7fd075600640 1 --2- 172.21.15.154:0/2137443733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd058004d00 0x7fd0700cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:49.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.653+0000 7fd074dff640 1 -- 172.21.15.154:0/2137443733 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0701f1a90 msgr2=0x7fd0700ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:49.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.653+0000 7fd074dff640 1 --2- 172.21.15.154:0/2137443733 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0701f1a90 0x7fd0700ca830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.653+0000 7fd074dff640 1 -- 172.21.15.154:0/2137443733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd058004d00 msgr2=0x7fd0700cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:49.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.653+0000 7fd074dff640 1 --2- 172.21.15.154:0/2137443733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd058004d00 0x7fd0700cced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.653+0000 7fd074dff640 1 -- 172.21.15.154:0/2137443733 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0700c5a10 con 0x7fd0701f1650 2023-09-23T03:24:49.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.653+0000 7fd074dff640 1 --2- 172.21.15.154:0/2137443733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0701f1650 0x7fd0700ca2f0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fd06400f580 tx=0x7fd06400fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:49.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.653+0000 7fd06dffb640 1 -- 172.21.15.154:0/2137443733 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd064014870 con 0x7fd0701f1650 2023-09-23T03:24:49.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.653+0000 7fd06dffb640 1 -- 172.21.15.154:0/2137443733 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0640149d0 con 0x7fd0701f1650 2023-09-23T03:24:49.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.653+0000 7fd06dffb640 1 -- 172.21.15.154:0/2137443733 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd064014d80 con 0x7fd0701f1650 2023-09-23T03:24:49.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.654+0000 7fd076602640 1 -- 172.21.15.154:0/2137443733 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0700c5cf0 con 0x7fd0701f1650 2023-09-23T03:24:49.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.654+0000 7fd076602640 1 -- 172.21.15.154:0/2137443733 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0700c4a70 con 0x7fd0701f1650 2023-09-23T03:24:49.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.655+0000 7fd076602640 1 -- 172.21.15.154:0/2137443733 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0700736b0 con 0x7fd0701f1650 2023-09-23T03:24:49.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.656+0000 7fd06dffb640 1 -- 172.21.15.154:0/2137443733 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fd064015410 con 0x7fd0701f1650 2023-09-23T03:24:49.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.657+0000 7fd06dffb640 1 --2- 172.21.15.154:0/2137443733 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd05c06af30 0x7fd05c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:49.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.657+0000 7fd06dffb640 1 -- 172.21.15.154:0/2137443733 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fd06408a650 con 0x7fd0701f1650 2023-09-23T03:24:49.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.657+0000 7fd075600640 1 --2- 172.21.15.154:0/2137443733 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd05c06af30 0x7fd05c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:49.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.658+0000 7fd075600640 1 --2- 172.21.15.154:0/2137443733 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd05c06af30 0x7fd05c06d3f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd0600021a0 tx=0x7fd060001490 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:49.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.661+0000 7fd06dffb640 1 -- 172.21.15.154:0/2137443733 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fd064055190 con 0x7fd0701f1650 2023-09-23T03:24:49.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.887+0000 7fd076602640 1 -- 172.21.15.154:0/2137443733 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0700c64b0 con 0x7fd0701f1650 2023-09-23T03:24:49.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.889+0000 7fd06dffb640 1 -- 172.21.15.154:0/2137443733 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4925 (secure 0 0 0) 0x7fd0640587b0 con 0x7fd0701f1650 2023-09-23T03:24:49.890 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:24:49.891 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":76,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7743,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/31599836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":31599836},{"type":"v1","addr":"172.21.15.154:6836","nonce":31599836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":7752,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/4266451911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":4266451911},{"type":"v1","addr":"172.21.15.154:6837","nonce":4266451911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":7755,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3029600822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3029600822},{"type":"v1","addr":"172.21.15.190:6834","nonce":3029600822}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:24:37.386034+0000","modified":"2023-09-23T03:24:47.439439+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":90,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7758},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7758":{"gid":7758,"name":"b","rank":0,"incarnation":73,"state":"up:active","state_seq":4,"addr":"172.21.15.190:6835/2202597679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2202597679},{"type":"v1","addr":"172.21.15.190:6835","nonce":2202597679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12,13],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-09-23T03:24:49.891 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 76 2023-09-23T03:24:49.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.891+0000 7fd076602640 1 -- 172.21.15.154:0/2137443733 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd05c06af30 msgr2=0x7fd05c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:49.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.891+0000 7fd076602640 1 --2- 172.21.15.154:0/2137443733 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd05c06af30 0x7fd05c06d3f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd0600021a0 tx=0x7fd060001490 comp rx=0 tx=0).stop 2023-09-23T03:24:49.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.891+0000 7fd076602640 1 -- 172.21.15.154:0/2137443733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0701f1650 msgr2=0x7fd0700ca2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:49.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.891+0000 7fd076602640 1 --2- 172.21.15.154:0/2137443733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0701f1650 0x7fd0700ca2f0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fd06400f580 tx=0x7fd06400fa40 comp rx=0 tx=0).stop 2023-09-23T03:24:49.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.892+0000 7fd076602640 1 -- 172.21.15.154:0/2137443733 shutdown_connections 2023-09-23T03:24:49.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.892+0000 7fd076602640 1 --2- 172.21.15.154:0/2137443733 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd05c06af30 0x7fd05c06d3f0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.892+0000 7fd076602640 1 --2- 172.21.15.154:0/2137443733 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd0701f1a90 0x7fd0700ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.892+0000 7fd076602640 1 --2- 172.21.15.154:0/2137443733 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd0701f1650 0x7fd0700ca2f0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.892+0000 7fd076602640 1 --2- 172.21.15.154:0/2137443733 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd058004d00 0x7fd0700cced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:49.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.892+0000 7fd076602640 1 -- 172.21.15.154:0/2137443733 >> 172.21.15.154:0/2137443733 conn(0x7fd07006a390 msgr2=0x7fd0700cc200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:49.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.892+0000 7fd076602640 1 -- 172.21.15.154:0/2137443733 shutdown_connections 2023-09-23T03:24:49.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:49.892+0000 7fd076602640 1 -- 172.21.15.154:0/2137443733 wait complete. 2023-09-23T03:24:49.912 INFO:tasks.cephfs.test_data_scan:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 4, "epoch": 76, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 4, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-09-23T03:24:37.386034+0000", "damaged": [], "data_pools": [ 12, 13 ], "enabled": true, "epoch": 76, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_7758": { "addr": "172.21.15.190:6835/2202597679", "addrs": { "addrvec": [ { "addr": "172.21.15.190:6833", "nonce": 2202597679, "type": "v2" }, { "addr": "172.21.15.190:6835", "nonce": 2202597679, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7758, "incarnation": 73, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:active", "state_seq": 4 } }, "last_failure": 0, "last_failure_osd_epoch": 90, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 11, "modified": "2023-09-23T03:24:47.439439+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7758 } } } ], "standbys": [ { "addr": "172.21.15.154:6836/31599836", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 31599836, "type": "v2" }, { "addr": "172.21.15.154:6836", "nonce": 31599836, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 72, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7743, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6837/4266451911", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6835", "nonce": 4266451911, "type": "v2" }, { "addr": "172.21.15.154:6837", "nonce": 4266451911, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 72, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7752, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.190:6834/3029600822", "addrs": { "addrvec": [ { "addr": "172.21.15.190:6832", "nonce": 3029600822, "type": "v2" }, { "addr": "172.21.15.190:6834", "nonce": 3029600822, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 74, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7755, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-09-23T03:24:49.912 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-23T03:24:50.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.124+0000 7fd9ac75e640 1 Processor -- start 2023-09-23T03:24:50.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.124+0000 7fd9ac75e640 1 -- start start 2023-09-23T03:24:50.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.126+0000 7fd9ac75e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd9a40b1b20 0x7fd9a40ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:50.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.126+0000 7fd9ac75e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd9a40cee70 0x7fd9a40ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:50.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.126+0000 7fd9ac75e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9a415ac30 con 0x7fd9a40acce0 2023-09-23T03:24:50.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.126+0000 7fd9ac75e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9a40b2e90 con 0x7fd9a40b1b20 2023-09-23T03:24:50.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.126+0000 7fd9ac75e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9a40cf320 con 0x7fd9a40cee70 2023-09-23T03:24:50.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.126+0000 7fd9aa4fa640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fd9a40acce0 0x7fd9a415ed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:60340/0 (socket says 172.21.15.154:60340) 2023-09-23T03:24:50.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.126+0000 7fd9aa4fa640 1 -- 172.21.15.154:0/535505236 learned_addr learned my addr 172.21.15.154:0/535505236 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:24:50.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.126+0000 7fd9a9cf9640 1 --2- 172.21.15.154:0/535505236 >> v2:172.21.15.154:3300/0 conn(0x7fd9a40b1b20 0x7fd9a40ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:50.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.126+0000 7fd9aacfb640 1 --2- 172.21.15.154:0/535505236 >> v2:172.21.15.190:3300/0 conn(0x7fd9a40cee70 0x7fd9a40ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:50.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.126+0000 7fd9a94f8640 1 -- 172.21.15.154:0/535505236 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2918804369 0 0) 0x7fd9a0001290 con 0x7fd9a40acce0 2023-09-23T03:24:50.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.126+0000 7fd9a94f8640 1 -- 172.21.15.154:0/535505236 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd990003520 con 0x7fd9a40acce0 2023-09-23T03:24:50.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.127+0000 7fd9a9cf9640 1 -- 172.21.15.154:0/535505236 >> v2:172.21.15.190:3300/0 conn(0x7fd9a40cee70 msgr2=0x7fd9a40ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:50.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.127+0000 7fd9a9cf9640 1 --2- 172.21.15.154:0/535505236 >> v2:172.21.15.190:3300/0 conn(0x7fd9a40cee70 0x7fd9a40ab140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:50.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.127+0000 7fd9a9cf9640 1 -- 172.21.15.154:0/535505236 >> v1:172.21.15.154:6789/0 conn(0x7fd9a40acce0 legacy=0x7fd9a415ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:50.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.127+0000 7fd9a9cf9640 1 -- 172.21.15.154:0/535505236 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a40b1f70 con 0x7fd9a40b1b20 2023-09-23T03:24:50.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.128+0000 7fd9a9cf9640 1 --2- 172.21.15.154:0/535505236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9a40b1b20 0x7fd9a40ce930 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fd9a40c9430 tx=0x7fd99402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec78ac3b1b07bec0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:50.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.128+0000 7fd9a94f8640 1 -- 172.21.15.154:0/535505236 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99402e070 con 0x7fd9a40b1b20 2023-09-23T03:24:50.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.128+0000 7fd9a94f8640 1 -- 172.21.15.154:0/535505236 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9940315a0 con 0x7fd9a40b1b20 2023-09-23T03:24:50.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.128+0000 7fd9a94f8640 1 -- 172.21.15.154:0/535505236 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd994031910 con 0x7fd9a40b1b20 2023-09-23T03:24:50.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.128+0000 7fd9ac75e640 1 -- 172.21.15.154:0/535505236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9a40b1b20 msgr2=0x7fd9a40ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:50.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.128+0000 7fd9ac75e640 1 --2- 172.21.15.154:0/535505236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9a40b1b20 0x7fd9a40ce930 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fd9a40c9430 tx=0x7fd99402b840 comp rx=0 tx=0).stop 2023-09-23T03:24:50.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.129+0000 7fd9ac75e640 1 -- 172.21.15.154:0/535505236 shutdown_connections 2023-09-23T03:24:50.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.129+0000 7fd9ac75e640 1 --2- 172.21.15.154:0/535505236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9a40b1b20 0x7fd9a40ce930 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:50.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.129+0000 7fd9ac75e640 1 --2- 172.21.15.154:0/535505236 >> v2:172.21.15.190:3300/0 conn(0x7fd9a40cee70 0x7fd9a40ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:50.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.129+0000 7fd9ac75e640 1 -- 172.21.15.154:0/535505236 >> 172.21.15.154:0/535505236 conn(0x7fd9a406a0f0 msgr2=0x7fd9a406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:50.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.129+0000 7fd9ac75e640 1 -- 172.21.15.154:0/535505236 shutdown_connections 2023-09-23T03:24:50.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.129+0000 7fd9ac75e640 1 -- 172.21.15.154:0/535505236 wait complete. 2023-09-23T03:24:50.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.130+0000 7fd9ac75e640 1 Processor -- start 2023-09-23T03:24:50.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.131+0000 7fd9ac75e640 1 -- start start 2023-09-23T03:24:50.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.132+0000 7fd9ac75e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9a40acce0 0x7fd9a4147c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:50.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.132+0000 7fd9ac75e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9a40b1b20 0x7fd9a41f1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:50.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.132+0000 7fd9ac75e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9a40cee70 0x7fd9a40cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:50.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.132+0000 7fd9ac75e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a415b250 con 0x7fd9a40cee70 2023-09-23T03:24:50.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.132+0000 7fd9ac75e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a4148b90 con 0x7fd9a40acce0 2023-09-23T03:24:50.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.132+0000 7fd9ac75e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a41485d0 con 0x7fd9a40b1b20 2023-09-23T03:24:50.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.132+0000 7fd9aacfb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9a40cee70 0x7fd9a40cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:50.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.132+0000 7fd9aacfb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9a40cee70 0x7fd9a40cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52640/0 (socket says 172.21.15.154:52640) 2023-09-23T03:24:50.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.132+0000 7fd9aacfb640 1 -- 172.21.15.154:0/3879284899 learned_addr learned my addr 172.21.15.154:0/3879284899 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:50.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.132+0000 7fd9aa4fa640 1 --2- 172.21.15.154:0/3879284899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9a40acce0 0x7fd9a4147c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:50.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.132+0000 7fd9aacfb640 1 -- 172.21.15.154:0/3879284899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9a40b1b20 msgr2=0x7fd9a41f1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:50.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.132+0000 7fd9aacfb640 1 --2- 172.21.15.154:0/3879284899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9a40b1b20 0x7fd9a41f1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:50.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.132+0000 7fd9aacfb640 1 -- 172.21.15.154:0/3879284899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9a40acce0 msgr2=0x7fd9a4147c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:50.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.132+0000 7fd9aacfb640 1 --2- 172.21.15.154:0/3879284899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9a40acce0 0x7fd9a4147c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:50.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.132+0000 7fd9aacfb640 1 -- 172.21.15.154:0/3879284899 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd994009400 con 0x7fd9a40cee70 2023-09-23T03:24:50.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.133+0000 7fd9aacfb640 1 --2- 172.21.15.154:0/3879284899 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9a40cee70 0x7fd9a40cd6d0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fd99c007e60 tx=0x7fd99c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:50.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.133+0000 7fd99affd640 1 -- 172.21.15.154:0/3879284899 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c014420 con 0x7fd9a40cee70 2023-09-23T03:24:50.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.133+0000 7fd99affd640 1 -- 172.21.15.154:0/3879284899 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd99c0145a0 con 0x7fd9a40cee70 2023-09-23T03:24:50.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.133+0000 7fd99affd640 1 -- 172.21.15.154:0/3879284899 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c014950 con 0x7fd9a40cee70 2023-09-23T03:24:50.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.133+0000 7fd9ac75e640 1 -- 172.21.15.154:0/3879284899 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9a41488b0 con 0x7fd9a40cee70 2023-09-23T03:24:50.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.133+0000 7fd9ac75e640 1 -- 172.21.15.154:0/3879284899 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9a4148190 con 0x7fd9a40cee70 2023-09-23T03:24:50.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.134+0000 7fd9ac75e640 1 -- 172.21.15.154:0/3879284899 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9a415b390 con 0x7fd9a40cee70 2023-09-23T03:24:50.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.135+0000 7fd99affd640 1 -- 172.21.15.154:0/3879284899 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fd99c014ab0 con 0x7fd9a40cee70 2023-09-23T03:24:50.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.136+0000 7fd99affd640 1 --2- 172.21.15.154:0/3879284899 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd98806af30 0x7fd98806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:50.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.136+0000 7fd99affd640 1 -- 172.21.15.154:0/3879284899 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fd99c08a520 con 0x7fd9a40cee70 2023-09-23T03:24:50.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.136+0000 7fd9aa4fa640 1 --2- 172.21.15.154:0/3879284899 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd98806af30 0x7fd98806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:50.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.137+0000 7fd9aa4fa640 1 --2- 172.21.15.154:0/3879284899 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd98806af30 0x7fd98806d3f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd9a00014f0 tx=0x7fd9a0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:50.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.141+0000 7fd99affd640 1 -- 172.21.15.154:0/3879284899 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fd9a415b390 con 0x7fd9a40cee70 2023-09-23T03:24:50.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.369+0000 7fd9ac75e640 1 -- 172.21.15.154:0/3879284899 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd9a415b390 con 0x7fd9a40cee70 2023-09-23T03:24:50.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.369+0000 7fd99affd640 1 -- 172.21.15.154:0/3879284899 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v76) v1 ==== 53+0+91 (secure 0 0 0) 0x7fd9a415b390 con 0x7fd9a40cee70 2023-09-23T03:24:50.371 INFO:teuthology.orchestra.run.smithi154.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_extra ] 2023-09-23T03:24:50.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.372+0000 7fd9ac75e640 1 -- 172.21.15.154:0/3879284899 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd98806af30 msgr2=0x7fd98806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:50.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.372+0000 7fd9ac75e640 1 --2- 172.21.15.154:0/3879284899 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd98806af30 0x7fd98806d3f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd9a00014f0 tx=0x7fd9a0009040 comp rx=0 tx=0).stop 2023-09-23T03:24:50.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.372+0000 7fd9ac75e640 1 -- 172.21.15.154:0/3879284899 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9a40cee70 msgr2=0x7fd9a40cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:50.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.372+0000 7fd9ac75e640 1 --2- 172.21.15.154:0/3879284899 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9a40cee70 0x7fd9a40cd6d0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fd99c007e60 tx=0x7fd99c00e4b0 comp rx=0 tx=0).stop 2023-09-23T03:24:50.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.372+0000 7fd9ac75e640 1 -- 172.21.15.154:0/3879284899 shutdown_connections 2023-09-23T03:24:50.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.372+0000 7fd9ac75e640 1 --2- 172.21.15.154:0/3879284899 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd98806af30 0x7fd98806d3f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:50.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.372+0000 7fd9ac75e640 1 --2- 172.21.15.154:0/3879284899 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9a40cee70 0x7fd9a40cd6d0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:50.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.372+0000 7fd9ac75e640 1 --2- 172.21.15.154:0/3879284899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9a40b1b20 0x7fd9a41f1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:50.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.372+0000 7fd9ac75e640 1 --2- 172.21.15.154:0/3879284899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9a40acce0 0x7fd9a4147c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:50.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.372+0000 7fd9ac75e640 1 -- 172.21.15.154:0/3879284899 >> 172.21.15.154:0/3879284899 conn(0x7fd9a406a0f0 msgr2=0x7fd9a40ab9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:50.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.372+0000 7fd9ac75e640 1 -- 172.21.15.154:0/3879284899 shutdown_connections 2023-09-23T03:24:50.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:50.372+0000 7fd9ac75e640 1 -- 172.21.15.154:0/3879284899 wait complete. 2023-09-23T03:24:50.392 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-23T03:24:50.392 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-23T03:24:50.392 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi154.front.sepia.ceph.com 2023-09-23T03:24:50.392 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-09-23T03:24:50.393 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-23T03:24:50.393 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-23T03:24:50.393 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-23T03:24:50.393 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-23T03:24:50.393 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-23T03:24:50.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:24:50.393 DEBUG:teuthology.orchestra.run.smithi154:> ip addr 2023-09-23T03:24:50.446 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-23T03:24:50.446 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-23T03:24:50.446 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2023-09-23T03:24:50.446 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:24:50.446 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2023-09-23T03:24:50.446 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:24:50.446 INFO:teuthology.orchestra.run.smithi154.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:24:50.446 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:54 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:24:50.446 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f0 2023-09-23T03:24:50.447 INFO:teuthology.orchestra.run.smithi154.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:24:50.447 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:55 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:24:50.447 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f1 2023-09-23T03:24:50.447 INFO:teuthology.orchestra.run.smithi154.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:24:50.447 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:60 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:24:50.447 INFO:teuthology.orchestra.run.smithi154.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-23T03:24:50.447 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:61 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:24:50.447 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.21.15.154/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-09-23T03:24:50.447 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft 41835sec preferred_lft 41835sec 2023-09-23T03:24:50.447 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::ec4:7aff:fe88:7761/64 scope link 2023-09-23T03:24:50.447 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:24:50.447 INFO:teuthology.orchestra.run.smithi154.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-23T03:24:50.449 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 5a:c6:b2:96:fb:dc brd ff:ff:ff:ff:ff:ff 2023-09-23T03:24:50.449 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-09-23T03:24:50.449 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:24:50.449 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::a801:22ff:febf:2833/64 scope link 2023-09-23T03:24:50.449 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:24:50.449 INFO:teuthology.orchestra.run.smithi154.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-23T03:24:50.449 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 5a:c6:b2:96:fb:dc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:24:50.449 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::58c6:b2ff:fe96:fbdc/64 scope link 2023-09-23T03:24:50.449 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:24:50.450 INFO:teuthology.orchestra.run.smithi154.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-23T03:24:50.450 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether b2:23:b0:e4:c2:ab brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:24:50.450 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::b023:b0ff:fee4:c2ab/64 scope link 2023-09-23T03:24:50.450 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:24:50.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:24:50.451 DEBUG:teuthology.orchestra.run.smithi154:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-23T03:24:50.539 INFO:teuthology.orchestra.run.smithi154.stdout:1 2023-09-23T03:24:50.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:24:50.543 DEBUG:teuthology.orchestra.run.smithi154:> route 2023-09-23T03:24:50.598 INFO:teuthology.orchestra.run.smithi154.stdout:Kernel IP routing table 2023-09-23T03:24:50.599 INFO:teuthology.orchestra.run.smithi154.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-23T03:24:50.599 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-23T03:24:50.599 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-09-23T03:24:50.599 INFO:teuthology.orchestra.run.smithi154.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-23T03:24:50.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:24:50.600 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-09-23T03:24:50.600 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-09-23T03:24:50.600 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-23T03:24:50.600 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-23T03:24:50.600 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-23T03:24:50.600 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-09-23T03:24:50.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:24:50.782 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2023-09-23T03:24:50.832 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-09-23T03:24:50.833 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-09-23T03:24:50.833 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-23T03:24:50.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:24:50.834 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set brx.0 up 2023-09-23T03:24:50.930 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-09-23T03:24:50.931 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-23T03:24:50.931 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-09-23T03:24:50.986 INFO:teuthology.orchestra.run.smithi154.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-09-23T03:24:50.986 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-23T03:24:50.986 DEBUG:teuthology.orchestra.run.smithi154:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-09-23T03:24:51.037 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-09-23T03:24:51.133 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2023-09-23T03:24:51.192 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2023-09-23T03:24:51.192 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2023-09-23T03:24:51.192 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2023-09-23T03:24:51.192 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2023-09-23T03:24:51.192 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2023-09-23T03:24:51.192 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2023-09-23T03:24:51.193 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2023-09-23T03:24:51.193 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2023-09-23T03:24:51.193 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2023-09-23T03:24:51.193 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2023-09-23T03:24:51.193 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2023-09-23T03:24:51.193 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T03:24:51.193 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2023-09-23T03:24:51.193 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2023-09-23T03:24:51.193 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2023-09-23T03:24:51.193 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2023-09-23T03:24:51.193 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2023-09-23T03:24:51.194 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2023-09-23T03:24:51.194 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2023-09-23T03:24:51.194 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-09-23T03:24:51.194 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-09-23T03:24:51.194 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2023-09-23T03:24:51.194 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2023-09-23T03:24:51.194 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2023-09-23T03:24:51.194 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-0 2023-09-23T03:24:51.194 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-1 2023-09-23T03:24:51.194 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-2 2023-09-23T03:24:51.194 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-3 2023-09-23T03:24:51.194 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2023-09-23T03:24:51.195 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:24:51.195 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:24:51.195 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:24:51.195 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:24:51.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:24:51.196 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-09-23T03:24:51.248 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-23T03:24:51.248 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-09-23T03:24:51.253 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-09-23T03:24:51.329 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2023-09-23T03:24:51.372 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2023-09-23T03:24:51.372 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2023-09-23T03:24:51.372 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2023-09-23T03:24:51.372 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2023-09-23T03:24:51.372 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2023-09-23T03:24:51.372 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2023-09-23T03:24:51.372 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2023-09-23T03:24:51.373 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2023-09-23T03:24:51.373 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2023-09-23T03:24:51.373 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2023-09-23T03:24:51.373 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2023-09-23T03:24:51.373 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T03:24:51.373 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2023-09-23T03:24:51.373 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2023-09-23T03:24:51.373 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2023-09-23T03:24:51.373 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2023-09-23T03:24:51.373 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2023-09-23T03:24:51.373 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2023-09-23T03:24:51.374 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2023-09-23T03:24:51.374 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-09-23T03:24:51.374 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-09-23T03:24:51.374 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2023-09-23T03:24:51.374 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2023-09-23T03:24:51.374 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2023-09-23T03:24:51.374 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-0 2023-09-23T03:24:51.374 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-1 2023-09-23T03:24:51.374 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-2 2023-09-23T03:24:51.374 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-3 2023-09-23T03:24:51.374 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2023-09-23T03:24:51.374 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:24:51.375 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:24:51.375 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:24:51.375 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:24:51.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:24:51.376 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-09-23T03:24:51.444 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:2023-09-23T03:24:51.443+0000 7f6553635480 -1 init, newargv = 0x55c4ed1d9da0 newargc=15 2023-09-23T03:24:51.444 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:ceph-fuse[118311]: starting ceph client 2023-09-23T03:24:51.455 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:ceph-fuse[118311]: starting fuse 2023-09-23T03:24:51.464 INFO:teuthology.orchestra.run.smithi154.stdout:39 2023-09-23T03:24:51.464 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-09-23T03:24:51.465 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:24:51.465 DEBUG:teuthology.orchestra.run.smithi154:> import glob 2023-09-23T03:24:51.465 DEBUG:teuthology.orchestra.run.smithi154:> import re 2023-09-23T03:24:51.465 DEBUG:teuthology.orchestra.run.smithi154:> import os 2023-09-23T03:24:51.465 DEBUG:teuthology.orchestra.run.smithi154:> import subprocess 2023-09-23T03:24:51.465 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:24:51.465 DEBUG:teuthology.orchestra.run.smithi154:> def _find_admin_socket(client_name): 2023-09-23T03:24:51.465 DEBUG:teuthology.orchestra.run.smithi154:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-09-23T03:24:51.465 DEBUG:teuthology.orchestra.run.smithi154:> files = glob.glob(asok_path) 2023-09-23T03:24:51.465 DEBUG:teuthology.orchestra.run.smithi154:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-09-23T03:24:51.465 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:24:51.466 DEBUG:teuthology.orchestra.run.smithi154:> # Given a non-glob path, it better be there 2023-09-23T03:24:51.466 DEBUG:teuthology.orchestra.run.smithi154:> if "*" not in asok_path: 2023-09-23T03:24:51.466 DEBUG:teuthology.orchestra.run.smithi154:> assert(len(files) == 1) 2023-09-23T03:24:51.466 DEBUG:teuthology.orchestra.run.smithi154:> return files[0] 2023-09-23T03:24:51.466 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:24:51.466 DEBUG:teuthology.orchestra.run.smithi154:> for f in files: 2023-09-23T03:24:51.466 DEBUG:teuthology.orchestra.run.smithi154:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-23T03:24:51.466 DEBUG:teuthology.orchestra.run.smithi154:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-23T03:24:51.466 DEBUG:teuthology.orchestra.run.smithi154:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-23T03:24:51.466 DEBUG:teuthology.orchestra.run.smithi154:> contents = proc_f.read() 2023-09-23T03:24:51.466 DEBUG:teuthology.orchestra.run.smithi154:> if mountpoint in contents: 2023-09-23T03:24:51.466 DEBUG:teuthology.orchestra.run.smithi154:> return f 2023-09-23T03:24:51.466 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-23T03:24:51.467 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:24:51.467 DEBUG:teuthology.orchestra.run.smithi154:> print(_find_admin_socket("client.0")) 2023-09-23T03:24:51.467 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:24:51.668 INFO:teuthology.orchestra.run.smithi154.stdout:/var/run/ceph/ceph-client.0.118311.asok 2023-09-23T03:24:51.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:24:51.684 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.118311.asok 2023-09-23T03:24:51.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:24:51.685 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.118311.asok status 2023-09-23T03:24:51.924 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:24:51.924 INFO:teuthology.orchestra.run.smithi154.stdout: "metadata": { 2023-09-23T03:24:51.924 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_sha1": "862663913a74a99e1c52d50e6a7e59dd6705749a", 2023-09-23T03:24:51.924 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_version": "ceph version 18.0.0-6336-g86266391 (862663913a74a99e1c52d50e6a7e59dd6705749a) reef (dev)", 2023-09-23T03:24:51.924 INFO:teuthology.orchestra.run.smithi154.stdout: "entity_id": "0", 2023-09-23T03:24:51.925 INFO:teuthology.orchestra.run.smithi154.stdout: "hostname": "smithi154", 2023-09-23T03:24:51.925 INFO:teuthology.orchestra.run.smithi154.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-09-23T03:24:51.925 INFO:teuthology.orchestra.run.smithi154.stdout: "pid": "118311", 2023-09-23T03:24:51.925 INFO:teuthology.orchestra.run.smithi154.stdout: "root": "/" 2023-09-23T03:24:51.925 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:24:51.925 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_count": 0, 2023-09-23T03:24:51.925 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_pinned_count": 0, 2023-09-23T03:24:51.925 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 7818, 2023-09-23T03:24:51.925 INFO:teuthology.orchestra.run.smithi154.stdout: "inst": { 2023-09-23T03:24:51.925 INFO:teuthology.orchestra.run.smithi154.stdout: "name": { 2023-09-23T03:24:51.925 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "client", 2023-09-23T03:24:51.926 INFO:teuthology.orchestra.run.smithi154.stdout: "num": 7818 2023-09-23T03:24:51.926 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:24:51.926 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2023-09-23T03:24:51.926 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2023-09-23T03:24:51.926 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.1:0", 2023-09-23T03:24:51.926 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3761712380 2023-09-23T03:24:51.926 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:24:51.926 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:24:51.926 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2023-09-23T03:24:51.926 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2023-09-23T03:24:51.926 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.1:0", 2023-09-23T03:24:51.926 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3761712380 2023-09-23T03:24:51.927 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:24:51.927 INFO:teuthology.orchestra.run.smithi154.stdout: "inst_str": "client.7818 192.168.0.1:0/3761712380", 2023-09-23T03:24:51.927 INFO:teuthology.orchestra.run.smithi154.stdout: "addr_str": "192.168.0.1:0/3761712380", 2023-09-23T03:24:51.927 INFO:teuthology.orchestra.run.smithi154.stdout: "inode_count": 1, 2023-09-23T03:24:51.927 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_epoch": 76, 2023-09-23T03:24:51.927 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch": 92, 2023-09-23T03:24:51.927 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch_barrier": 0, 2023-09-23T03:24:51.927 INFO:teuthology.orchestra.run.smithi154.stdout: "blocklisted": false, 2023-09-23T03:24:51.927 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs" 2023-09-23T03:24:51.927 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:24:51.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:24:51.939 DEBUG:teuthology.orchestra.run.smithi154:> stat --file-system '--printf=%T 2023-09-23T03:24:51.939 DEBUG:teuthology.orchestra.run.smithi154:> ' -- /home/ubuntu/cephtest/mnt.0 2023-09-23T03:24:51.991 INFO:teuthology.orchestra.run.smithi154.stdout:fuseblk 2023-09-23T03:24:51.991 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-09-23T03:24:51.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:24:51.992 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-09-23T03:24:52.082 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub start / repair,recursive 2023-09-23T03:24:52.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.307+0000 7fb94432c640 1 Processor -- start 2023-09-23T03:24:52.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.307+0000 7fb94432c640 1 -- start start 2023-09-23T03:24:52.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.309+0000 7fb94432c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb93c15abd0 0x7fb93c0c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.309+0000 7fb94432c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb93c0c95b0 con 0x7fb93c0cb460 2023-09-23T03:24:52.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.309+0000 7fb94432c640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb93c0c9890 con 0x7fb93c0700f0 2023-09-23T03:24:52.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.309+0000 7fb94432c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c0c9b70 con 0x7fb93c15abd0 2023-09-23T03:24:52.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.309+0000 7fb9428c9640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fb93c0cb460 0x7fb93c0b1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:60346/0 (socket says 172.21.15.154:60346) 2023-09-23T03:24:52.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.309+0000 7fb9428c9640 1 -- 172.21.15.154:0/3886258932 learned_addr learned my addr 172.21.15.154:0/3886258932 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:24:52.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.309+0000 7fb9418c7640 1 --2- 172.21.15.154:0/3886258932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb93c15abd0 0x7fb93c0c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:52.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.310+0000 7fb9410c6640 1 -- 172.21.15.154:0/3886258932 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 91798677 0 0) 0x7fb93c0c95b0 con 0x7fb93c0cb460 2023-09-23T03:24:52.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.310+0000 7fb9410c6640 1 -- 172.21.15.154:0/3886258932 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb928003520 con 0x7fb93c0cb460 2023-09-23T03:24:52.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.310+0000 7fb9410c6640 1 -- 172.21.15.154:0/3886258932 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4220487653 0 0) 0x7fb928003520 con 0x7fb93c0cb460 2023-09-23T03:24:52.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.310+0000 7fb9410c6640 1 -- 172.21.15.154:0/3886258932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb93c15abd0 msgr2=0x7fb93c0c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:52.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.310+0000 7fb9410c6640 1 --2- 172.21.15.154:0/3886258932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb93c15abd0 0x7fb93c0c8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.310+0000 7fb9410c6640 1 -- 172.21.15.154:0/3886258932 >> v1:172.21.15.190:6789/0 conn(0x7fb93c0700f0 legacy=0x7fb93c0704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:52.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.310+0000 7fb9410c6640 1 -- 172.21.15.154:0/3886258932 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb93c0c9e50 con 0x7fb93c0cb460 2023-09-23T03:24:52.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.310+0000 7fb9410c6640 1 -- 172.21.15.154:0/3886258932 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fb9340019f0 con 0x7fb93c0cb460 2023-09-23T03:24:52.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.310+0000 7fb9410c6640 1 -- 172.21.15.154:0/3886258932 >> v1:172.21.15.154:6789/0 conn(0x7fb93c0cb460 legacy=0x7fb93c0b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:52.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.310+0000 7fb9410c6640 1 --2- 172.21.15.154:0/3886258932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb93c15abd0 0x7fb9280039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.310+0000 7fb9410c6640 1 --2- 172.21.15.154:0/3886258932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb928003f70 0x7fb9280047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.310+0000 7fb9410c6640 1 --2- 172.21.15.154:0/3886258932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb928004d00 0x7fb928005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.310+0000 7fb9410c6640 1 -- 172.21.15.154:0/3886258932 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c0c9b70 con 0x7fb928004d00 2023-09-23T03:24:52.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.310+0000 7fb9410c6640 1 -- 172.21.15.154:0/3886258932 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92800bac0 con 0x7fb928003f70 2023-09-23T03:24:52.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.310+0000 7fb9410c6640 1 -- 172.21.15.154:0/3886258932 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb92800bc80 con 0x7fb93c15abd0 2023-09-23T03:24:52.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.310+0000 7fb9428c9640 1 --2- 172.21.15.154:0/3886258932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb928004d00 0x7fb928005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:52.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.311+0000 7fb9420c8640 1 --2- 172.21.15.154:0/3886258932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb93c15abd0 0x7fb9280039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:52.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.311+0000 7fb9418c7640 1 --2- 172.21.15.154:0/3886258932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb928003f70 0x7fb9280047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:52.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.311+0000 7fb9428c9640 1 -- 172.21.15.154:0/3886258932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb93c15abd0 msgr2=0x7fb9280039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:52.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.311+0000 7fb9428c9640 1 --2- 172.21.15.154:0/3886258932 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb93c15abd0 0x7fb9280039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.311+0000 7fb9428c9640 1 -- 172.21.15.154:0/3886258932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb928003f70 msgr2=0x7fb9280047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:52.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.311+0000 7fb9428c9640 1 --2- 172.21.15.154:0/3886258932 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb928003f70 0x7fb9280047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.311+0000 7fb9428c9640 1 -- 172.21.15.154:0/3886258932 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb928003520 con 0x7fb928004d00 2023-09-23T03:24:52.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.312+0000 7fb9428c9640 1 --2- 172.21.15.154:0/3886258932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb928004d00 0x7fb928005180 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fb93400a760 tx=0x7fb9280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e42647b58a3dd8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:52.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.312+0000 7fb9410c6640 1 -- 172.21.15.154:0/3886258932 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb934034630 con 0x7fb928004d00 2023-09-23T03:24:52.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.312+0000 7fb9410c6640 1 -- 172.21.15.154:0/3886258932 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93402f070 con 0x7fb928004d00 2023-09-23T03:24:52.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.312+0000 7fb9410c6640 1 -- 172.21.15.154:0/3886258932 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb934034a50 con 0x7fb928004d00 2023-09-23T03:24:52.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.313+0000 7fb94432c640 1 -- 172.21.15.154:0/3886258932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb928004d00 msgr2=0x7fb928005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:52.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.313+0000 7fb94432c640 1 --2- 172.21.15.154:0/3886258932 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb928004d00 0x7fb928005180 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fb93400a760 tx=0x7fb9280057f0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.313+0000 7fb9428c9640 1 -- 172.21.15.154:0/3886258932 reap_dead start 2023-09-23T03:24:52.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.313+0000 7fb94432c640 1 -- 172.21.15.154:0/3886258932 shutdown_connections 2023-09-23T03:24:52.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.313+0000 7fb94432c640 1 -- 172.21.15.154:0/3886258932 >> 172.21.15.154:0/3886258932 conn(0x7fb93c06a520 msgr2=0x7fb93c06bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:52.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.313+0000 7fb94432c640 1 -- 172.21.15.154:0/3886258932 shutdown_connections 2023-09-23T03:24:52.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.313+0000 7fb94432c640 1 -- 172.21.15.154:0/3886258932 wait complete. 2023-09-23T03:24:52.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.315+0000 7fb94432c640 1 Processor -- start 2023-09-23T03:24:52.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.315+0000 7fb94432c640 1 -- start start 2023-09-23T03:24:52.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.316+0000 7fb94432c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb928004d00 0x7fb93c14b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.316+0000 7fb94432c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb93c0c5f30 0x7fb93c0c63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.316+0000 7fb94432c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb93c0c6ef0 0x7fb93c0c7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.316+0000 7fb94432c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c0c78b0 con 0x7fb93c0c5f30 2023-09-23T03:24:52.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.316+0000 7fb94432c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c1f2c90 con 0x7fb928004d00 2023-09-23T03:24:52.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.316+0000 7fb94432c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c1f2f70 con 0x7fb93c0c6ef0 2023-09-23T03:24:52.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.316+0000 7fb9418c7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb93c0c5f30 0x7fb93c0c63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:52.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.316+0000 7fb9418c7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb93c0c5f30 0x7fb93c0c63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52668/0 (socket says 172.21.15.154:52668) 2023-09-23T03:24:52.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.316+0000 7fb9418c7640 1 -- 172.21.15.154:0/912174691 learned_addr learned my addr 172.21.15.154:0/912174691 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:52.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.317+0000 7fb9420c8640 1 --2- 172.21.15.154:0/912174691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb928004d00 0x7fb93c14b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:52.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.317+0000 7fb9428c9640 1 --2- 172.21.15.154:0/912174691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb93c0c6ef0 0x7fb93c0c7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:52.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.317+0000 7fb9418c7640 1 -- 172.21.15.154:0/912174691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb93c0c6ef0 msgr2=0x7fb93c0c7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:52.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.317+0000 7fb9418c7640 1 --2- 172.21.15.154:0/912174691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb93c0c6ef0 0x7fb93c0c7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.317+0000 7fb9418c7640 1 -- 172.21.15.154:0/912174691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb928004d00 msgr2=0x7fb93c14b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:52.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.317+0000 7fb9418c7640 1 --2- 172.21.15.154:0/912174691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb928004d00 0x7fb93c14b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.317+0000 7fb9418c7640 1 -- 172.21.15.154:0/912174691 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb93c1f3250 con 0x7fb93c0c5f30 2023-09-23T03:24:52.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.317+0000 7fb9418c7640 1 --2- 172.21.15.154:0/912174691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb93c0c5f30 0x7fb93c0c63b0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fb92c011560 tx=0x7fb92c011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:52.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.317+0000 7fb932ffd640 1 -- 172.21.15.154:0/912174691 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92c004220 con 0x7fb93c0c5f30 2023-09-23T03:24:52.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.317+0000 7fb932ffd640 1 -- 172.21.15.154:0/912174691 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb92c004380 con 0x7fb93c0c5f30 2023-09-23T03:24:52.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.317+0000 7fb932ffd640 1 -- 172.21.15.154:0/912174691 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92c015870 con 0x7fb93c0c5f30 2023-09-23T03:24:52.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.317+0000 7fb94432c640 1 -- 172.21.15.154:0/912174691 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb93c1f3530 con 0x7fb93c0c5f30 2023-09-23T03:24:52.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.318+0000 7fb94432c640 1 -- 172.21.15.154:0/912174691 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb93c1f2940 con 0x7fb93c0c5f30 2023-09-23T03:24:52.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.320+0000 7fb932ffd640 1 -- 172.21.15.154:0/912174691 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fb92c0159d0 con 0x7fb93c0c5f30 2023-09-23T03:24:52.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.321+0000 7fb932ffd640 1 --2- 172.21.15.154:0/912174691 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb92006af30 0x7fb92006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.321+0000 7fb932ffd640 1 -- 172.21.15.154:0/912174691 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fb92c089de0 con 0x7fb93c0c5f30 2023-09-23T03:24:52.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.321+0000 7fb9420c8640 1 --2- 172.21.15.154:0/912174691 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb92006af30 0x7fb92006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:52.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.322+0000 7fb946f96740 1 Processor -- start 2023-09-23T03:24:52.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.322+0000 7fb9420c8640 1 --2- 172.21.15.154:0/912174691 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb92006af30 0x7fb92006d3f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb938000c00 tx=0x7fb938009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:52.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.322+0000 7fb946f96740 1 -- start start 2023-09-23T03:24:52.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.323+0000 7fb946f96740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5646e2e9de70 0x5646e2e70d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.323+0000 7fb946f96740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646e2e9e250 0x5646e2ea0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.323+0000 7fb946f96740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5646e2e712b0 0x5646e2ea1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.323+0000 7fb946f96740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e2de44e0 con 0x5646e2e9de70 2023-09-23T03:24:52.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.323+0000 7fb946f96740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e2de1bd0 con 0x5646e2e9e250 2023-09-23T03:24:52.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.323+0000 7fb946f96740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5646e2ddd4c0 con 0x5646e2e712b0 2023-09-23T03:24:52.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.323+0000 7fb9420c8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5646e2e9de70 0x5646e2e70d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:52.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.323+0000 7fb9420c8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5646e2e9de70 0x5646e2e70d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52680/0 (socket says 172.21.15.154:52680) 2023-09-23T03:24:52.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.323+0000 7fb9420c8640 1 -- 172.21.15.154:0/2136363899 learned_addr learned my addr 172.21.15.154:0/2136363899 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:52.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.324+0000 7fb9428c9640 1 --2- 172.21.15.154:0/2136363899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5646e2e712b0 0x5646e2ea1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:52.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.324+0000 7fb9418c7640 1 --2- 172.21.15.154:0/2136363899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646e2e9e250 0x5646e2ea0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:52.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.324+0000 7fb9420c8640 1 -- 172.21.15.154:0/2136363899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5646e2e712b0 msgr2=0x5646e2ea1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:52.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.324+0000 7fb9420c8640 1 --2- 172.21.15.154:0/2136363899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5646e2e712b0 0x5646e2ea1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.324+0000 7fb9420c8640 1 -- 172.21.15.154:0/2136363899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646e2e9e250 msgr2=0x5646e2ea0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:52.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.324+0000 7fb9420c8640 1 --2- 172.21.15.154:0/2136363899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646e2e9e250 0x5646e2ea0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.324+0000 7fb9420c8640 1 -- 172.21.15.154:0/2136363899 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646e2d7acb0 con 0x5646e2e9de70 2023-09-23T03:24:52.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.324+0000 7fb9428c9640 1 --2- 172.21.15.154:0/2136363899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5646e2e712b0 0x5646e2ea1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:52.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.324+0000 7fb9418c7640 1 --2- 172.21.15.154:0/2136363899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646e2e9e250 0x5646e2ea0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:52.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.324+0000 7fb9420c8640 1 --2- 172.21.15.154:0/2136363899 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5646e2e9de70 0x5646e2e70d70 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fb9380015f0 tx=0x7fb938013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a432d44879552c13 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:52.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.324+0000 7fb91ffff640 1 -- 172.21.15.154:0/2136363899 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9380147c0 con 0x5646e2e9de70 2023-09-23T03:24:52.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.324+0000 7fb91ffff640 1 -- 172.21.15.154:0/2136363899 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb938014920 con 0x5646e2e9de70 2023-09-23T03:24:52.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.324+0000 7fb91ffff640 1 -- 172.21.15.154:0/2136363899 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb938014c90 con 0x5646e2e9de70 2023-09-23T03:24:52.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.325+0000 7fb946f96740 1 -- 172.21.15.154:0/2136363899 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5646e2e9de70 msgr2=0x5646e2e70d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:52.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.325+0000 7fb946f96740 1 --2- 172.21.15.154:0/2136363899 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5646e2e9de70 0x5646e2e70d70 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fb9380015f0 tx=0x7fb938013000 comp rx=0 tx=0).stop 2023-09-23T03:24:52.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.325+0000 7fb946f96740 1 -- 172.21.15.154:0/2136363899 shutdown_connections 2023-09-23T03:24:52.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.325+0000 7fb946f96740 1 --2- 172.21.15.154:0/2136363899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5646e2e712b0 0x5646e2ea1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.325+0000 7fb946f96740 1 --2- 172.21.15.154:0/2136363899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646e2e9e250 0x5646e2ea0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.325+0000 7fb946f96740 1 --2- 172.21.15.154:0/2136363899 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5646e2e9de70 0x5646e2e70d70 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.325+0000 7fb946f96740 1 -- 172.21.15.154:0/2136363899 >> 172.21.15.154:0/2136363899 conn(0x5646e2e73080 msgr2=0x5646e2ebbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:52.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.325+0000 7fb946f96740 1 -- 172.21.15.154:0/2136363899 shutdown_connections 2023-09-23T03:24:52.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.325+0000 7fb946f96740 1 -- 172.21.15.154:0/2136363899 wait complete. 2023-09-23T03:24:52.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.326+0000 7fb946f96740 1 -- start start 2023-09-23T03:24:52.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.326+0000 7fb946f96740 1 Processor -- start 2023-09-23T03:24:52.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.327+0000 7fb946f96740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5646e2e9de70 0x5646e2e8f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.327+0000 7fb946f96740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5646e2e9e250 0x5646e2eb5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.327+0000 7fb946f96740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646e2eb3cd0 0x5646e2e8faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.327+0000 7fb946f96740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e2d7b1e0 con 0x5646e2e9e250 2023-09-23T03:24:52.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.327+0000 7fb946f96740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e2d7ae80 con 0x5646e2eb3cd0 2023-09-23T03:24:52.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.327+0000 7fb946f96740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5646e2b420c0 con 0x5646e2e9de70 2023-09-23T03:24:52.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.328+0000 7fb946f96740 10 client.0 _ll_register_callbacks cb 0x5646e2dd2d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:24:52.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.328+0000 7fb9418c7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5646e2e9e250 0x5646e2eb5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:52.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.328+0000 7fb9418c7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5646e2e9e250 0x5646e2eb5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52694/0 (socket says 172.21.15.154:52694) 2023-09-23T03:24:52.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.328+0000 7fb9418c7640 1 -- 172.21.15.154:0/4167181644 learned_addr learned my addr 172.21.15.154:0/4167181644 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:52.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.328+0000 7fb9420c8640 1 --2- 172.21.15.154:0/4167181644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5646e2e9de70 0x5646e2e8f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:52.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.328+0000 7fb9428c9640 1 --2- 172.21.15.154:0/4167181644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646e2eb3cd0 0x5646e2e8faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:52.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.328+0000 7fb9418c7640 1 -- 172.21.15.154:0/4167181644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5646e2e9de70 msgr2=0x5646e2e8f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:52.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.328+0000 7fb9418c7640 1 --2- 172.21.15.154:0/4167181644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5646e2e9de70 0x5646e2e8f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.328+0000 7fb9418c7640 1 -- 172.21.15.154:0/4167181644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646e2eb3cd0 msgr2=0x5646e2e8faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:52.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.328+0000 7fb9418c7640 1 --2- 172.21.15.154:0/4167181644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646e2eb3cd0 0x5646e2e8faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.328+0000 7fb9418c7640 1 -- 172.21.15.154:0/4167181644 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb93800f040 con 0x5646e2e9e250 2023-09-23T03:24:52.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.328+0000 7fb9420c8640 1 --2- 172.21.15.154:0/4167181644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5646e2e9de70 0x5646e2e8f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:52.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.328+0000 7fb9428c9640 1 --2- 172.21.15.154:0/4167181644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646e2eb3cd0 0x5646e2e8faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:52.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.328+0000 7fb9418c7640 1 --2- 172.21.15.154:0/4167181644 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5646e2e9e250 0x5646e2eb5830 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fb92c058360 tx=0x7fb92c05c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2974ef84d25eb79f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:52.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.328+0000 7fb91d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:24:52.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.329+0000 7fb91d7fa640 1 -- 172.21.15.154:0/4167181644 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92c0606d0 con 0x5646e2e9e250 2023-09-23T03:24:52.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.329+0000 7fb91d7fa640 1 -- 172.21.15.154:0/4167181644 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fb92c054450 con 0x5646e2e9e250 2023-09-23T03:24:52.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.329+0000 7fb946f96740 1 -- 172.21.15.154:0/4167181644 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5646e2dd64a0 con 0x5646e2e9e250 2023-09-23T03:24:52.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.329+0000 7fb91d7fa640 1 -- 172.21.15.154:0/4167181644 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb92c054610 con 0x5646e2e9e250 2023-09-23T03:24:52.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.329+0000 7fb946f96740 10 client.7830 fetch_fsmap learned FSMap version 76 2023-09-23T03:24:52.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.329+0000 7fb946f96740 1 -- 172.21.15.154:0/4167181644 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x5646e2ddbb40 con 0x5646e2e9e250 2023-09-23T03:24:52.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.329+0000 7fb91d7fa640 1 -- 172.21.15.154:0/4167181644 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5646e2dd64a0 con 0x5646e2e9e250 2023-09-23T03:24:52.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.329+0000 7fb91d7fa640 1 -- 172.21.15.154:0/4167181644 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 76) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fb92c06e020 con 0x5646e2e9e250 2023-09-23T03:24:52.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.329+0000 7fb946f96740 10 client.7830 fetch_fsmap finished waiting for FSMap version 76 2023-09-23T03:24:52.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.329+0000 7fb946f96740 10 client.7830 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2023-09-23T03:24:52.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.329+0000 7fb946f96740 20 client.7830 populate_metadata read hostname 'smithi154' 2023-09-23T03:24:52.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.329+0000 7fb946f96740 1 --2- 172.21.15.154:0/4167181644 >> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] conn(0x5646e2e90030 0x5646e2ea36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.329+0000 7fb946f96740 4 client.7830 mds_command: new command op to 7758 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T03:24:52.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.329+0000 7fb946f96740 1 -- 172.21.15.154:0/4167181644 --> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5646e2dd61c0 con 0x5646e2e90030 2023-09-23T03:24:52.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.330+0000 7fb9420c8640 1 --2- 172.21.15.154:0/4167181644 >> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] conn(0x5646e2e90030 0x5646e2ea36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:52.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.331+0000 7fb9420c8640 1 --2- 172.21.15.154:0/4167181644 >> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] conn(0x5646e2e90030 0x5646e2ea36b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c47a9872877602a0 server_cookie=450496a34c6a4182 in_seq=0 out_seq=0 2023-09-23T03:24:52.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.331+0000 7fb91d7fa640 10 client.7830 ms_handle_connect on v2:172.21.15.190:6833/2202597679 2023-09-23T03:24:52.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.333+0000 7fb91d7fa640 1 -- 172.21.15.154:0/4167181644 <== mds.0 v2:172.21.15.190:6833/2202597679 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5646e2dd61c0 con 0x5646e2e90030 2023-09-23T03:24:52.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.333+0000 7fb91d7fa640 10 client.7830 handle_command_reply: tid=0 2023-09-23T03:24:52.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.333+0000 7fb946f96740 1 client.7830 shutdown 2023-09-23T03:24:52.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.333+0000 7fb946f96740 1 -- 172.21.15.154:0/4167181644 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5646e2e9e250 msgr2=0x5646e2eb5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:52.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.333+0000 7fb946f96740 1 --2- 172.21.15.154:0/4167181644 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5646e2e9e250 0x5646e2eb5830 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fb92c058360 tx=0x7fb92c05c7b0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.333+0000 7fb946f96740 1 -- 172.21.15.154:0/4167181644 shutdown_connections 2023-09-23T03:24:52.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.333+0000 7fb946f96740 1 --2- 172.21.15.154:0/4167181644 >> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] conn(0x5646e2e90030 0x5646e2ea36b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.333+0000 7fb946f96740 1 --2- 172.21.15.154:0/4167181644 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646e2eb3cd0 0x5646e2e8faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.333+0000 7fb946f96740 1 --2- 172.21.15.154:0/4167181644 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5646e2e9e250 0x5646e2eb5830 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.333+0000 7fb946f96740 1 --2- 172.21.15.154:0/4167181644 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5646e2e9de70 0x5646e2e8f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.333+0000 7fb91d7fa640 0 client.7830 ms_handle_reset on v2:172.21.15.190:6833/2202597679 2023-09-23T03:24:52.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.333+0000 7fb946f96740 1 -- 172.21.15.154:0/4167181644 >> 172.21.15.154:0/4167181644 conn(0x5646e2e712b0 msgr2=0x5646e2e74e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:52.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.334+0000 7fb946f96740 1 -- 172.21.15.154:0/4167181644 shutdown_connections 2023-09-23T03:24:52.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.334+0000 7fb946f96740 1 -- 172.21.15.154:0/4167181644 wait complete. 2023-09-23T03:24:52.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.334+0000 7fb946f96740 20 client.7830 trim_cache size 0 max 16384 2023-09-23T03:24:52.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.350+0000 7fb946f96740 1 Processor -- start 2023-09-23T03:24:52.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.350+0000 7fb946f96740 1 -- start start 2023-09-23T03:24:52.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.351+0000 7fb946f96740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646e2e9de70 0x5646e2ece4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.351+0000 7fb946f96740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5646e2e9e250 0x5646e2ebf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.351+0000 7fb946f96740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5646e2eb3cd0 0x5646e2ebf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.351+0000 7fb946f96740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e2e74ed0 con 0x5646e2eb3cd0 2023-09-23T03:24:52.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.351+0000 7fb946f96740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e2e70b90 con 0x5646e2e9de70 2023-09-23T03:24:52.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.351+0000 7fb946f96740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5646e2e6fa80 con 0x5646e2e9e250 2023-09-23T03:24:52.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.352+0000 7fb9428c9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5646e2eb3cd0 0x5646e2ebf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:52.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.352+0000 7fb9428c9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5646e2eb3cd0 0x5646e2ebf650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52710/0 (socket says 172.21.15.154:52710) 2023-09-23T03:24:52.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.352+0000 7fb9428c9640 1 -- 172.21.15.154:0/1058778772 learned_addr learned my addr 172.21.15.154:0/1058778772 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:52.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.352+0000 7fb9418c7640 1 --2- 172.21.15.154:0/1058778772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5646e2e9e250 0x5646e2ebf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:52.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.352+0000 7fb9420c8640 1 --2- 172.21.15.154:0/1058778772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646e2e9de70 0x5646e2ece4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:52.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.352+0000 7fb9428c9640 1 -- 172.21.15.154:0/1058778772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5646e2e9e250 msgr2=0x5646e2ebf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:52.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.352+0000 7fb9428c9640 1 --2- 172.21.15.154:0/1058778772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5646e2e9e250 0x5646e2ebf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.352+0000 7fb9428c9640 1 -- 172.21.15.154:0/1058778772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646e2e9de70 msgr2=0x5646e2ece4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:52.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.352+0000 7fb9428c9640 1 --2- 172.21.15.154:0/1058778772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646e2e9de70 0x5646e2ece4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.352+0000 7fb9428c9640 1 -- 172.21.15.154:0/1058778772 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646e2d7a540 con 0x5646e2eb3cd0 2023-09-23T03:24:52.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.352+0000 7fb9420c8640 1 --2- 172.21.15.154:0/1058778772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646e2e9de70 0x5646e2ece4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:52.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.352+0000 7fb9418c7640 1 --2- 172.21.15.154:0/1058778772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5646e2e9e250 0x5646e2ebf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:52.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.352+0000 7fb9428c9640 1 --2- 172.21.15.154:0/1058778772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5646e2eb3cd0 0x5646e2ebf650 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fb93402cb80 tx=0x7fb934009030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ad90fcd75ecbf3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:52.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.353+0000 7fb930e19640 1 -- 172.21.15.154:0/1058778772 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9340017f0 con 0x5646e2eb3cd0 2023-09-23T03:24:52.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.353+0000 7fb930e19640 1 -- 172.21.15.154:0/1058778772 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb934001950 con 0x5646e2eb3cd0 2023-09-23T03:24:52.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.353+0000 7fb930e19640 1 -- 172.21.15.154:0/1058778772 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb934040660 con 0x5646e2eb3cd0 2023-09-23T03:24:52.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.353+0000 7fb946f96740 1 -- 172.21.15.154:0/1058778772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5646e2eb3cd0 msgr2=0x5646e2ebf650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:52.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.353+0000 7fb946f96740 1 --2- 172.21.15.154:0/1058778772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5646e2eb3cd0 0x5646e2ebf650 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fb93402cb80 tx=0x7fb934009030 comp rx=0 tx=0).stop 2023-09-23T03:24:52.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.353+0000 7fb946f96740 1 -- 172.21.15.154:0/1058778772 shutdown_connections 2023-09-23T03:24:52.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.353+0000 7fb946f96740 1 --2- 172.21.15.154:0/1058778772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5646e2eb3cd0 0x5646e2ebf650 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.354+0000 7fb946f96740 1 --2- 172.21.15.154:0/1058778772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5646e2e9e250 0x5646e2ebf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.354+0000 7fb946f96740 1 --2- 172.21.15.154:0/1058778772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646e2e9de70 0x5646e2ece4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.354+0000 7fb946f96740 1 -- 172.21.15.154:0/1058778772 >> 172.21.15.154:0/1058778772 conn(0x5646e2e712b0 msgr2=0x5646e2e8f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:52.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.354+0000 7fb946f96740 1 -- 172.21.15.154:0/1058778772 shutdown_connections 2023-09-23T03:24:52.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.354+0000 7fb946f96740 1 -- 172.21.15.154:0/1058778772 wait complete. 2023-09-23T03:24:52.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.354+0000 7fb946f96740 1 -- start start 2023-09-23T03:24:52.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.355+0000 7fb946f96740 1 Processor -- start 2023-09-23T03:24:52.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.355+0000 7fb946f96740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5646e2e9de70 0x5646e2ebae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.355+0000 7fb946f96740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646e2e9e250 0x5646e2e76d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.355+0000 7fb946f96740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5646e2eb3cd0 0x5646e2ed4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.355+0000 7fb946f96740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e2ebb920 con 0x5646e2eb3cd0 2023-09-23T03:24:52.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.355+0000 7fb946f96740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646e2eb5eb0 con 0x5646e2e9e250 2023-09-23T03:24:52.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.355+0000 7fb946f96740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5646e2e75d40 con 0x5646e2e9de70 2023-09-23T03:24:52.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.356+0000 7fb946f96740 10 client.0 _ll_register_callbacks cb 0x7fb9380052d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:24:52.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.356+0000 7fb9428c9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5646e2eb3cd0 0x5646e2ed4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:52.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.356+0000 7fb9428c9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5646e2eb3cd0 0x5646e2ed4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52720/0 (socket says 172.21.15.154:52720) 2023-09-23T03:24:52.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.356+0000 7fb9428c9640 1 -- 172.21.15.154:0/2995130459 learned_addr learned my addr 172.21.15.154:0/2995130459 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:52.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.356+0000 7fb9428c9640 1 -- 172.21.15.154:0/2995130459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5646e2e9de70 msgr2=0x5646e2ebae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:52.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.356+0000 7fb9418c7640 1 --2- 172.21.15.154:0/2995130459 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646e2e9e250 0x5646e2e76d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:52.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.356+0000 7fb9420c8640 1 --2- 172.21.15.154:0/2995130459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5646e2e9de70 0x5646e2ebae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:52.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.356+0000 7fb9428c9640 1 --2- 172.21.15.154:0/2995130459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5646e2e9de70 0x5646e2ebae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.356+0000 7fb9428c9640 1 -- 172.21.15.154:0/2995130459 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646e2e9e250 msgr2=0x5646e2e76d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:52.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.356+0000 7fb9428c9640 1 --2- 172.21.15.154:0/2995130459 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646e2e9e250 0x5646e2e76d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.356+0000 7fb9428c9640 1 -- 172.21.15.154:0/2995130459 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb934038040 con 0x5646e2eb3cd0 2023-09-23T03:24:52.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.356+0000 7fb9428c9640 1 --2- 172.21.15.154:0/2995130459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5646e2eb3cd0 0x5646e2ed4fd0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fb934043040 tx=0x7fb93402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a7adb8265fb2cd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:52.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.357+0000 7fb91d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:24:52.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.357+0000 7fb91d7fa640 1 -- 172.21.15.154:0/2995130459 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb934001280 con 0x5646e2eb3cd0 2023-09-23T03:24:52.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.357+0000 7fb91d7fa640 1 -- 172.21.15.154:0/2995130459 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fb934040460 con 0x5646e2eb3cd0 2023-09-23T03:24:52.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.357+0000 7fb946f96740 1 -- 172.21.15.154:0/2995130459 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb938014920 con 0x5646e2eb3cd0 2023-09-23T03:24:52.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.357+0000 7fb91d7fa640 1 -- 172.21.15.154:0/2995130459 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb934040620 con 0x5646e2eb3cd0 2023-09-23T03:24:52.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.357+0000 7fb946f96740 10 client.7836 fetch_fsmap learned FSMap version 76 2023-09-23T03:24:52.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.357+0000 7fb946f96740 1 -- 172.21.15.154:0/2995130459 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x5646e2d9fbf0 con 0x5646e2eb3cd0 2023-09-23T03:24:52.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.357+0000 7fb91d7fa640 1 -- 172.21.15.154:0/2995130459 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb938014920 con 0x5646e2eb3cd0 2023-09-23T03:24:52.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.357+0000 7fb91d7fa640 1 -- 172.21.15.154:0/2995130459 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 76) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fb934049040 con 0x5646e2eb3cd0 2023-09-23T03:24:52.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.357+0000 7fb946f96740 10 client.7836 fetch_fsmap finished waiting for FSMap version 76 2023-09-23T03:24:52.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.357+0000 7fb946f96740 10 client.7836 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2023-09-23T03:24:52.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.357+0000 7fb946f96740 20 client.7836 populate_metadata read hostname 'smithi154' 2023-09-23T03:24:52.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.357+0000 7fb946f96740 1 --2- 172.21.15.154:0/2995130459 >> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] conn(0x5646e2f17780 0x5646e2ed5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.357+0000 7fb946f96740 4 client.7836 mds_command: new command op to 7758 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2023-09-23T03:24:52.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.357+0000 7fb946f96740 1 -- 172.21.15.154:0/2995130459 --> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x5646e2adf860 con 0x5646e2f17780 2023-09-23T03:24:52.363 INFO:tasks.ceph.mds.b.smithi190.stderr:2023-09-23T03:24:52.362+0000 7fce45bea640 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2023-09-23T03:24:52.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.358+0000 7fb9420c8640 1 --2- 172.21.15.154:0/2995130459 >> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] conn(0x5646e2f17780 0x5646e2ed5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:52.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.359+0000 7fb91d7fa640 10 client.7836 ms_handle_connect on v2:172.21.15.190:6833/2202597679 2023-09-23T03:24:52.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.359+0000 7fb9420c8640 1 --2- 172.21.15.154:0/2995130459 >> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] conn(0x5646e2f17780 0x5646e2ed5510 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3dea3db8c37c7c server_cookie=1a34089ab3722a56 in_seq=0 out_seq=0 2023-09-23T03:24:52.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.360+0000 7fb91d7fa640 1 -- 172.21.15.154:0/2995130459 <== mds.0 v2:172.21.15.190:6833/2202597679 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5646e2adf860 con 0x5646e2f17780 2023-09-23T03:24:52.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.360+0000 7fb91d7fa640 10 client.7836 handle_command_reply: tid=0 2023-09-23T03:24:52.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.360+0000 7fb946f96740 1 client.7836 shutdown 2023-09-23T03:24:52.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.360+0000 7fb946f96740 1 -- 172.21.15.154:0/2995130459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5646e2eb3cd0 msgr2=0x5646e2ed4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:52.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.360+0000 7fb946f96740 1 --2- 172.21.15.154:0/2995130459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5646e2eb3cd0 0x5646e2ed4fd0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fb934043040 tx=0x7fb93402e000 comp rx=0 tx=0).stop 2023-09-23T03:24:52.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.361+0000 7fb946f96740 1 -- 172.21.15.154:0/2995130459 shutdown_connections 2023-09-23T03:24:52.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.361+0000 7fb946f96740 1 --2- 172.21.15.154:0/2995130459 >> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] conn(0x5646e2f17780 0x5646e2ed5510 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.361+0000 7fb946f96740 1 --2- 172.21.15.154:0/2995130459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5646e2eb3cd0 0x5646e2ed4fd0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.361+0000 7fb946f96740 1 --2- 172.21.15.154:0/2995130459 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5646e2e9e250 0x5646e2e76d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.361+0000 7fb946f96740 1 --2- 172.21.15.154:0/2995130459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5646e2e9de70 0x5646e2ebae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.361+0000 7fb91d7fa640 0 client.7836 ms_handle_reset on v2:172.21.15.190:6833/2202597679 2023-09-23T03:24:52.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.361+0000 7fb946f96740 1 -- 172.21.15.154:0/2995130459 >> 172.21.15.154:0/2995130459 conn(0x5646e2e712b0 msgr2=0x5646e2f186f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:52.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.361+0000 7fb946f96740 1 -- 172.21.15.154:0/2995130459 shutdown_connections 2023-09-23T03:24:52.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.361+0000 7fb946f96740 1 -- 172.21.15.154:0/2995130459 wait complete. 2023-09-23T03:24:52.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.361+0000 7fb946f96740 20 client.7836 trim_cache size 0 max 16384 2023-09-23T03:24:52.366 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:24:52.366 INFO:teuthology.orchestra.run.smithi154.stdout: "return_code": 0, 2023-09-23T03:24:52.366 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_tag": "dc1a4afd-c3bb-4268-a271-e1eaf4b50151", 2023-09-23T03:24:52.366 INFO:teuthology.orchestra.run.smithi154.stdout: "mode": "asynchronous" 2023-09-23T03:24:52.367 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:24:52.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.361+0000 7fb94432c640 1 -- 172.21.15.154:0/912174691 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb92006af30 msgr2=0x7fb92006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:52.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.361+0000 7fb94432c640 1 --2- 172.21.15.154:0/912174691 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb92006af30 0x7fb92006d3f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb938000c00 tx=0x7fb938009040 comp rx=0 tx=0).stop 2023-09-23T03:24:52.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.362+0000 7fb94432c640 1 -- 172.21.15.154:0/912174691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb93c0c5f30 msgr2=0x7fb93c0c63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:52.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.362+0000 7fb94432c640 1 --2- 172.21.15.154:0/912174691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb93c0c5f30 0x7fb93c0c63b0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fb92c011560 tx=0x7fb92c011a20 comp rx=0 tx=0).stop 2023-09-23T03:24:52.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.362+0000 7fb94432c640 1 -- 172.21.15.154:0/912174691 shutdown_connections 2023-09-23T03:24:52.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.362+0000 7fb94432c640 1 --2- 172.21.15.154:0/912174691 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb92006af30 0x7fb92006d3f0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.362+0000 7fb94432c640 1 --2- 172.21.15.154:0/912174691 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb93c0c6ef0 0x7fb93c0c7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.362+0000 7fb94432c640 1 --2- 172.21.15.154:0/912174691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb93c0c5f30 0x7fb93c0c63b0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.362+0000 7fb94432c640 1 --2- 172.21.15.154:0/912174691 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb928004d00 0x7fb93c14b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.362+0000 7fb94432c640 1 -- 172.21.15.154:0/912174691 >> 172.21.15.154:0/912174691 conn(0x7fb93c06a520 msgr2=0x7fb93c0b1310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:52.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.362+0000 7fb94432c640 1 -- 172.21.15.154:0/912174691 shutdown_connections 2023-09-23T03:24:52.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.362+0000 7fb94432c640 1 -- 172.21.15.154:0/912174691 wait complete. 2023-09-23T03:24:52.369 INFO:tasks.ceph.mds.b.smithi190.stderr:2023-09-23T03:24:52.365+0000 7fce45bea640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /subdir/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x5584d4d38580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(11)0x10000000000:[<0x1/subdir v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-09-23T03:24:52.369 INFO:tasks.ceph.mds.b.smithi190.stderr:2023-09-23T03:24:52.367+0000 7fce45bea640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/sixmegs auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x5584d4de2680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(13)0x10000000001:[<0x10000000000/sixmegs v8>,<0x1/subdir v10>]//[12]","memoryvalue":"(13)0x10000000001:[<0x10000000000/sixmegs v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:24:52.376 INFO:tasks.ceph.mds.b.smithi190.stderr:2023-09-23T03:24:52.376+0000 7fce45bea640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v5 ap=5 snaprealm=0x5584d4d52d80 DIRTYPARENT f(v1 1=0+1) n(v0 rc2023-09-23T03:24:52.077396+0000 1=0+1)/n() (idft sync r=1) (inest sync r=1) (ifile sync->mix r=1) (iversion lock) caps={7818=pLsXs/-@3} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x5584d4d3a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(11)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=0+1)","ondisk_value.rstat":"n(v0 2=0+2)","memory_value.dirstat":"f(v1 1=0+1)","memory_value.rstat":"n(v0 rc2023-09-23T03:24:52.077396+0000 1=0+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-09-23T03:24:52.379 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2023-09-23T03:24:52.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.577+0000 7f745d3f5640 1 Processor -- start 2023-09-23T03:24:52.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.577+0000 7f745d3f5640 1 -- start start 2023-09-23T03:24:52.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.579+0000 7f745d3f5640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f74580cae90 0x7f74580cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.579+0000 7f745d3f5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f74580c9980 0x7f7458165350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.579+0000 7f745d3f5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74580c1a40 con 0x7f74580c9510 2023-09-23T03:24:52.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.579+0000 7f745d3f5640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74581624f0 con 0x7f74580c9980 2023-09-23T03:24:52.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.579+0000 7f745d3f5640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7458162be0 con 0x7f74580cae90 2023-09-23T03:24:52.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.580+0000 7f74577fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f74580c9980 0x7f7458165350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:52.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.580+0000 7f744e7fc640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f74580c9510 0x7f7458164c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:60360/0 (socket says 172.21.15.154:60360) 2023-09-23T03:24:52.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.580+0000 7f74577fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f74580c9980 0x7f7458165350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52726/0 (socket says 172.21.15.154:52726) 2023-09-23T03:24:52.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.580+0000 7f744e7fc640 1 -- 172.21.15.154:0/1556341640 learned_addr learned my addr 172.21.15.154:0/1556341640 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:24:52.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.580+0000 7f7456ffd640 1 --2- 172.21.15.154:0/1556341640 >> v2:172.21.15.190:3300/0 conn(0x7f74580cae90 0x7f74580cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:52.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.580+0000 7f74577fe640 1 -- 172.21.15.154:0/1556341640 >> v2:172.21.15.190:3300/0 conn(0x7f74580cae90 msgr2=0x7f74580cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:52.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.580+0000 7f74577fe640 1 --2- 172.21.15.154:0/1556341640 >> v2:172.21.15.190:3300/0 conn(0x7f74580cae90 0x7f74580cb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.580+0000 7f74577fe640 1 -- 172.21.15.154:0/1556341640 >> v1:172.21.15.154:6789/0 conn(0x7f74580c9510 legacy=0x7f7458164c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:52.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.580+0000 7f74577fe640 1 -- 172.21.15.154:0/1556341640 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7458162ec0 con 0x7f74580c9980 2023-09-23T03:24:52.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.581+0000 7f74577fe640 1 --2- 172.21.15.154:0/1556341640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74580c9980 0x7f7458165350 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f74580adf60 tx=0x7f745002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a1d6262f5f5e9d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:52.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.581+0000 7f74567fc640 1 -- 172.21.15.154:0/1556341640 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7450030610 con 0x7f74580c9980 2023-09-23T03:24:52.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.581+0000 7f74567fc640 1 -- 172.21.15.154:0/1556341640 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7450030770 con 0x7f74580c9980 2023-09-23T03:24:52.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.581+0000 7f74567fc640 1 -- 172.21.15.154:0/1556341640 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7450030b00 con 0x7f74580c9980 2023-09-23T03:24:52.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.582+0000 7f745d3f5640 1 -- 172.21.15.154:0/1556341640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74580c9980 msgr2=0x7f7458165350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:52.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.582+0000 7f745d3f5640 1 --2- 172.21.15.154:0/1556341640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74580c9980 0x7f7458165350 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f74580adf60 tx=0x7f745002b960 comp rx=0 tx=0).stop 2023-09-23T03:24:52.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.582+0000 7f745d3f5640 1 -- 172.21.15.154:0/1556341640 shutdown_connections 2023-09-23T03:24:52.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.582+0000 7f745d3f5640 1 --2- 172.21.15.154:0/1556341640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74580c9980 0x7f7458165350 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.582+0000 7f745d3f5640 1 --2- 172.21.15.154:0/1556341640 >> v2:172.21.15.190:3300/0 conn(0x7f74580cae90 0x7f74580cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.582+0000 7f745d3f5640 1 -- 172.21.15.154:0/1556341640 >> 172.21.15.154:0/1556341640 conn(0x7f745806c010 msgr2=0x7f745806c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:52.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.582+0000 7f745d3f5640 1 -- 172.21.15.154:0/1556341640 shutdown_connections 2023-09-23T03:24:52.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.582+0000 7f745d3f5640 1 -- 172.21.15.154:0/1556341640 wait complete. 2023-09-23T03:24:52.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.584+0000 7f745d3f5640 1 Processor -- start 2023-09-23T03:24:52.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.584+0000 7f745d3f5640 1 -- start start 2023-09-23T03:24:52.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.585+0000 7f745d3f5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74580c9510 0x7f74581f72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.585+0000 7f745d3f5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74580c9980 0x7f74580b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.585+0000 7f745d3f5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74580cae90 0x7f745815e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.585+0000 7f745d3f5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74580c7c30 con 0x7f74580cae90 2023-09-23T03:24:52.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.585+0000 7f745d3f5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74580c6d80 con 0x7f74580c9980 2023-09-23T03:24:52.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.585+0000 7f745d3f5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74580c7020 con 0x7f74580c9510 2023-09-23T03:24:52.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.586+0000 7f74577fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74580cae90 0x7f745815e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:52.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.586+0000 7f74577fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74580cae90 0x7f745815e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52742/0 (socket says 172.21.15.154:52742) 2023-09-23T03:24:52.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.586+0000 7f74577fe640 1 -- 172.21.15.154:0/2473654613 learned_addr learned my addr 172.21.15.154:0/2473654613 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:52.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.586+0000 7f7456ffd640 1 --2- 172.21.15.154:0/2473654613 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74580c9510 0x7f74581f72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:52.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.586+0000 7f744e7fc640 1 --2- 172.21.15.154:0/2473654613 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74580c9980 0x7f74580b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:52.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.586+0000 7f74577fe640 1 -- 172.21.15.154:0/2473654613 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74580c9510 msgr2=0x7f74581f72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:52.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.586+0000 7f74577fe640 1 --2- 172.21.15.154:0/2473654613 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74580c9510 0x7f74581f72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.586+0000 7f74577fe640 1 -- 172.21.15.154:0/2473654613 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74580c9980 msgr2=0x7f74580b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:52.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.586+0000 7f74577fe640 1 --2- 172.21.15.154:0/2473654613 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74580c9980 0x7f74580b36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.586+0000 7f74577fe640 1 -- 172.21.15.154:0/2473654613 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7450009400 con 0x7f74580cae90 2023-09-23T03:24:52.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.586+0000 7f744e7fc640 1 --2- 172.21.15.154:0/2473654613 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74580c9980 0x7f74580b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:24:52.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.586+0000 7f74577fe640 1 --2- 172.21.15.154:0/2473654613 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74580cae90 0x7f745815e520 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f745002bf70 tx=0x7f7450004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:52.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.587+0000 7f744ffff640 1 -- 172.21.15.154:0/2473654613 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745003e040 con 0x7f74580cae90 2023-09-23T03:24:52.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.587+0000 7f744ffff640 1 -- 172.21.15.154:0/2473654613 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7450039c30 con 0x7f74580cae90 2023-09-23T03:24:52.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.587+0000 7f744ffff640 1 -- 172.21.15.154:0/2473654613 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7450006e10 con 0x7f74580cae90 2023-09-23T03:24:52.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.587+0000 7f745d3f5640 1 -- 172.21.15.154:0/2473654613 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74580c7300 con 0x7f74580cae90 2023-09-23T03:24:52.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.587+0000 7f745d3f5640 1 -- 172.21.15.154:0/2473654613 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74580c78c0 con 0x7f74580cae90 2023-09-23T03:24:52.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.589+0000 7f744ffff640 1 -- 172.21.15.154:0/2473654613 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f74500443f0 con 0x7f74580cae90 2023-09-23T03:24:52.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.590+0000 7f744ffff640 1 --2- 172.21.15.154:0/2473654613 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f743806af60 0x7f743806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.590+0000 7f744ffff640 1 -- 172.21.15.154:0/2473654613 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f74500abf30 con 0x7f74580cae90 2023-09-23T03:24:52.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.590+0000 7f7456ffd640 1 --2- 172.21.15.154:0/2473654613 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f743806af60 0x7f743806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:52.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.591+0000 7f745ff83740 1 Processor -- start 2023-09-23T03:24:52.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.591+0000 7f7456ffd640 1 --2- 172.21.15.154:0/2473654613 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f743806af60 0x7f743806d420 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f7448006fd0 tx=0x7f7448008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:52.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.591+0000 7f745ff83740 1 -- start start 2023-09-23T03:24:52.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.592+0000 7f745ff83740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fabe49690 0x560fabe1c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.592+0000 7f745ff83740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fabe49a70 0x560fabe4c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.592+0000 7f745ff83740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560fabe1cad0 0x560fabe4c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.592+0000 7f745ff83740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fabd8ea50 con 0x560fabe49690 2023-09-23T03:24:52.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.592+0000 7f745ff83740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fabd8c140 con 0x560fabe49a70 2023-09-23T03:24:52.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.592+0000 7f745ff83740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x560fabd87a30 con 0x560fabe1cad0 2023-09-23T03:24:52.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.592+0000 7f7456ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fabe49690 0x560fabe1c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:52.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.592+0000 7f7456ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fabe49690 0x560fabe1c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52752/0 (socket says 172.21.15.154:52752) 2023-09-23T03:24:52.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.592+0000 7f7456ffd640 1 -- 172.21.15.154:0/3459529965 learned_addr learned my addr 172.21.15.154:0/3459529965 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:52.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.593+0000 7f744e7fc640 1 --2- 172.21.15.154:0/3459529965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fabe49a70 0x560fabe4c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:52.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.593+0000 7f7456ffd640 1 -- 172.21.15.154:0/3459529965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560fabe1cad0 msgr2=0x560fabe4c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:52.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.593+0000 7f74577fe640 1 --2- 172.21.15.154:0/3459529965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560fabe1cad0 0x560fabe4c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:52.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.593+0000 7f7456ffd640 1 --2- 172.21.15.154:0/3459529965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560fabe1cad0 0x560fabe4c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.593+0000 7f7456ffd640 1 -- 172.21.15.154:0/3459529965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fabe49a70 msgr2=0x560fabe4c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:52.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.593+0000 7f7456ffd640 1 --2- 172.21.15.154:0/3459529965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fabe49a70 0x560fabe4c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.593+0000 7f7456ffd640 1 -- 172.21.15.154:0/3459529965 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fabd4ba70 con 0x560fabe49690 2023-09-23T03:24:52.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.593+0000 7f7456ffd640 1 --2- 172.21.15.154:0/3459529965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fabe49690 0x560fabe1c590 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f744800e920 tx=0x7f7448011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=588947bd576616b1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:52.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.593+0000 7f744cff9640 1 -- 172.21.15.154:0/3459529965 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7448015700 con 0x560fabe49690 2023-09-23T03:24:52.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.593+0000 7f744cff9640 1 -- 172.21.15.154:0/3459529965 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7448015860 con 0x560fabe49690 2023-09-23T03:24:52.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.593+0000 7f744cff9640 1 -- 172.21.15.154:0/3459529965 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7448015c10 con 0x560fabe49690 2023-09-23T03:24:52.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.594+0000 7f745ff83740 1 -- 172.21.15.154:0/3459529965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fabe49690 msgr2=0x560fabe1c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:52.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.594+0000 7f745ff83740 1 --2- 172.21.15.154:0/3459529965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fabe49690 0x560fabe1c590 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f744800e920 tx=0x7f7448011000 comp rx=0 tx=0).stop 2023-09-23T03:24:52.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.594+0000 7f745ff83740 1 -- 172.21.15.154:0/3459529965 shutdown_connections 2023-09-23T03:24:52.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.594+0000 7f745ff83740 1 --2- 172.21.15.154:0/3459529965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560fabe1cad0 0x560fabe4c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.594+0000 7f745ff83740 1 --2- 172.21.15.154:0/3459529965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fabe49a70 0x560fabe4c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.594+0000 7f745ff83740 1 --2- 172.21.15.154:0/3459529965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fabe49690 0x560fabe1c590 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.594+0000 7f745ff83740 1 -- 172.21.15.154:0/3459529965 >> 172.21.15.154:0/3459529965 conn(0x560fabe1e8a0 msgr2=0x560fabe67420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:52.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.594+0000 7f745ff83740 1 -- 172.21.15.154:0/3459529965 shutdown_connections 2023-09-23T03:24:52.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.594+0000 7f745ff83740 1 -- 172.21.15.154:0/3459529965 wait complete. 2023-09-23T03:24:52.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.595+0000 7f745ff83740 1 -- start start 2023-09-23T03:24:52.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.595+0000 7f745ff83740 1 Processor -- start 2023-09-23T03:24:52.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.596+0000 7f745ff83740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560fabe49690 0x560fabe375d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.596+0000 7f745ff83740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fabe49a70 0x560fabe5ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.596+0000 7f745ff83740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fabe61cf0 0x560fabe39300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.596+0000 7f745ff83740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fabd25ad0 con 0x560fabe61cf0 2023-09-23T03:24:52.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.596+0000 7f745ff83740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fabd25770 con 0x560fabe49a70 2023-09-23T03:24:52.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.596+0000 7f745ff83740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x560faba81f90 con 0x560fabe49690 2023-09-23T03:24:52.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.596+0000 7f745ff83740 10 client.0 _ll_register_callbacks cb 0x560fabd49220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:24:52.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.597+0000 7f74577fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fabe61cf0 0x560fabe39300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:52.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.597+0000 7f74577fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fabe61cf0 0x560fabe39300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52756/0 (socket says 172.21.15.154:52756) 2023-09-23T03:24:52.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.597+0000 7f74577fe640 1 -- 172.21.15.154:0/841492312 learned_addr learned my addr 172.21.15.154:0/841492312 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:52.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.597+0000 7f74577fe640 1 -- 172.21.15.154:0/841492312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560fabe49690 msgr2=0x560fabe375d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:52.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.597+0000 7f74577fe640 1 --2- 172.21.15.154:0/841492312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560fabe49690 0x560fabe375d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.597+0000 7f74577fe640 1 -- 172.21.15.154:0/841492312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fabe49a70 msgr2=0x560fabe5ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:52.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.597+0000 7f74577fe640 1 --2- 172.21.15.154:0/841492312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fabe49a70 0x560fabe5ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.597+0000 7f74577fe640 1 -- 172.21.15.154:0/841492312 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560fabd860b0 con 0x560fabe61cf0 2023-09-23T03:24:52.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.597+0000 7f74577fe640 1 --2- 172.21.15.154:0/841492312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fabe61cf0 0x560fabe39300 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f7450076d40 tx=0x7f745007b080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29897f64ddf0301a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:52.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.597+0000 7f7435ffb640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:24:52.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.597+0000 7f7435ffb640 1 -- 172.21.15.154:0/841492312 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74500828b0 con 0x560fabe61cf0 2023-09-23T03:24:52.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.597+0000 7f7435ffb640 1 -- 172.21.15.154:0/841492312 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f7450082b40 con 0x560fabe61cf0 2023-09-23T03:24:52.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.597+0000 7f745ff83740 1 -- 172.21.15.154:0/841492312 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560fabdae4d0 con 0x560fabe61cf0 2023-09-23T03:24:52.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.598+0000 7f7435ffb640 1 -- 172.21.15.154:0/841492312 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f745007bb70 con 0x560fabe61cf0 2023-09-23T03:24:52.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.598+0000 7f745ff83740 10 client.7851 fetch_fsmap learned FSMap version 76 2023-09-23T03:24:52.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.598+0000 7f745ff83740 1 -- 172.21.15.154:0/841492312 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x560fabd24ce0 con 0x560fabe61cf0 2023-09-23T03:24:52.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.598+0000 7f7435ffb640 1 -- 172.21.15.154:0/841492312 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560fabdae4d0 con 0x560fabe61cf0 2023-09-23T03:24:52.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.598+0000 7f7435ffb640 1 -- 172.21.15.154:0/841492312 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 76) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f745007bcd0 con 0x560fabe61cf0 2023-09-23T03:24:52.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.598+0000 7f745ff83740 10 client.7851 fetch_fsmap finished waiting for FSMap version 76 2023-09-23T03:24:52.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.598+0000 7f745ff83740 10 client.7851 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2023-09-23T03:24:52.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.598+0000 7f745ff83740 20 client.7851 populate_metadata read hostname 'smithi154' 2023-09-23T03:24:52.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.598+0000 7f745ff83740 1 --2- 172.21.15.154:0/841492312 >> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] conn(0x560fabe39840 0x560fabe3a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.598+0000 7f745ff83740 4 client.7851 mds_command: new command op to 7758 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T03:24:52.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.598+0000 7f745ff83740 1 -- 172.21.15.154:0/841492312 --> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560fabd5aab0 con 0x560fabe39840 2023-09-23T03:24:52.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.599+0000 7f7456ffd640 1 --2- 172.21.15.154:0/841492312 >> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] conn(0x560fabe39840 0x560fabe3a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:52.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.600+0000 7f7456ffd640 1 --2- 172.21.15.154:0/841492312 >> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] conn(0x560fabe39840 0x560fabe3a1e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e55cfcefe2536922 server_cookie=b6d8e605586a16d6 in_seq=0 out_seq=0 2023-09-23T03:24:52.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.600+0000 7f7435ffb640 10 client.7851 ms_handle_connect on v2:172.21.15.190:6833/2202597679 2023-09-23T03:24:52.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.601+0000 7f7435ffb640 1 -- 172.21.15.154:0/841492312 <== mds.0 v2:172.21.15.190:6833/2202597679 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560fabd5aab0 con 0x560fabe39840 2023-09-23T03:24:52.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.601+0000 7f7435ffb640 10 client.7851 handle_command_reply: tid=0 2023-09-23T03:24:52.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.602+0000 7f745ff83740 1 client.7851 shutdown 2023-09-23T03:24:52.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.602+0000 7f745ff83740 1 -- 172.21.15.154:0/841492312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fabe61cf0 msgr2=0x560fabe39300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:52.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.602+0000 7f745ff83740 1 --2- 172.21.15.154:0/841492312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fabe61cf0 0x560fabe39300 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f7450076d40 tx=0x7f745007b080 comp rx=0 tx=0).stop 2023-09-23T03:24:52.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.602+0000 7f745ff83740 1 -- 172.21.15.154:0/841492312 shutdown_connections 2023-09-23T03:24:52.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.602+0000 7f745ff83740 1 --2- 172.21.15.154:0/841492312 >> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] conn(0x560fabe39840 0x560fabe3a1e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.602+0000 7f745ff83740 1 --2- 172.21.15.154:0/841492312 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fabe61cf0 0x560fabe39300 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.602+0000 7f745ff83740 1 --2- 172.21.15.154:0/841492312 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fabe49a70 0x560fabe5ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.602+0000 7f745ff83740 1 --2- 172.21.15.154:0/841492312 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560fabe49690 0x560fabe375d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.602+0000 7f7435ffb640 0 client.7851 ms_handle_reset on v2:172.21.15.190:6833/2202597679 2023-09-23T03:24:52.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.602+0000 7f745ff83740 1 -- 172.21.15.154:0/841492312 >> 172.21.15.154:0/841492312 conn(0x560fabe1cad0 msgr2=0x560fabe20670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:52.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.602+0000 7f745ff83740 1 -- 172.21.15.154:0/841492312 shutdown_connections 2023-09-23T03:24:52.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.602+0000 7f745ff83740 1 -- 172.21.15.154:0/841492312 wait complete. 2023-09-23T03:24:52.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.602+0000 7f745ff83740 20 client.7851 trim_cache size 0 max 16384 2023-09-23T03:24:52.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.618+0000 7f745ff83740 1 Processor -- start 2023-09-23T03:24:52.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.618+0000 7f745ff83740 1 -- start start 2023-09-23T03:24:52.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.619+0000 7f745ff83740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fabe49690 0x560fabe4cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.619+0000 7f745ff83740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560fabe49a70 0x560fabe373e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.619+0000 7f745ff83740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fabe61cf0 0x560fabe37920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.619+0000 7f745ff83740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fabe67f00 con 0x560fabe61cf0 2023-09-23T03:24:52.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.619+0000 7f745ff83740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fabe3a1e0 con 0x560fabe49690 2023-09-23T03:24:52.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.619+0000 7f745ff83740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x560fabe504b0 con 0x560fabe49a70 2023-09-23T03:24:52.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.619+0000 7f74577fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fabe61cf0 0x560fabe37920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:52.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.619+0000 7f74577fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fabe61cf0 0x560fabe37920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52758/0 (socket says 172.21.15.154:52758) 2023-09-23T03:24:52.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.619+0000 7f74577fe640 1 -- 172.21.15.154:0/3673344315 learned_addr learned my addr 172.21.15.154:0/3673344315 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:52.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.620+0000 7f74577fe640 1 -- 172.21.15.154:0/3673344315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560fabe49a70 msgr2=0x560fabe373e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:52.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.620+0000 7f744e7fc640 1 --2- 172.21.15.154:0/3673344315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560fabe49a70 0x560fabe373e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:52.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.620+0000 7f7456ffd640 1 --2- 172.21.15.154:0/3673344315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fabe49690 0x560fabe4cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:52.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.620+0000 7f74577fe640 1 --2- 172.21.15.154:0/3673344315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560fabe49a70 0x560fabe373e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.620+0000 7f74577fe640 1 -- 172.21.15.154:0/3673344315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fabe49690 msgr2=0x560fabe4cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:52.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.620+0000 7f74577fe640 1 --2- 172.21.15.154:0/3673344315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fabe49690 0x560fabe4cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.620+0000 7f74577fe640 1 -- 172.21.15.154:0/3673344315 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fabdb7da0 con 0x560fabe61cf0 2023-09-23T03:24:52.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.620+0000 7f74577fe640 1 --2- 172.21.15.154:0/3673344315 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fabe61cf0 0x560fabe37920 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f745002b9a0 tx=0x7f745007e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f764b0f809addf96 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:52.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.620+0000 7f744d7fa640 1 -- 172.21.15.154:0/3673344315 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74500828b0 con 0x560fabe61cf0 2023-09-23T03:24:52.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.620+0000 7f744d7fa640 1 -- 172.21.15.154:0/3673344315 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7450076b80 con 0x560fabe61cf0 2023-09-23T03:24:52.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.620+0000 7f744d7fa640 1 -- 172.21.15.154:0/3673344315 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7450082b40 con 0x560fabe61cf0 2023-09-23T03:24:52.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.621+0000 7f745ff83740 1 -- 172.21.15.154:0/3673344315 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fabe61cf0 msgr2=0x560fabe37920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:52.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.621+0000 7f745ff83740 1 --2- 172.21.15.154:0/3673344315 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fabe61cf0 0x560fabe37920 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f745002b9a0 tx=0x7f745007e000 comp rx=0 tx=0).stop 2023-09-23T03:24:52.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.621+0000 7f745ff83740 1 -- 172.21.15.154:0/3673344315 shutdown_connections 2023-09-23T03:24:52.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.621+0000 7f745ff83740 1 --2- 172.21.15.154:0/3673344315 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fabe61cf0 0x560fabe37920 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.621+0000 7f745ff83740 1 --2- 172.21.15.154:0/3673344315 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560fabe49a70 0x560fabe373e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.621+0000 7f745ff83740 1 --2- 172.21.15.154:0/3673344315 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fabe49690 0x560fabe4cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.621+0000 7f745ff83740 1 -- 172.21.15.154:0/3673344315 >> 172.21.15.154:0/3673344315 conn(0x560fabe1cad0 msgr2=0x560fabe39130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:52.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.621+0000 7f745ff83740 1 -- 172.21.15.154:0/3673344315 shutdown_connections 2023-09-23T03:24:52.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.621+0000 7f745ff83740 1 -- 172.21.15.154:0/3673344315 wait complete. 2023-09-23T03:24:52.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.622+0000 7f745ff83740 1 -- start start 2023-09-23T03:24:52.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.622+0000 7f745ff83740 1 Processor -- start 2023-09-23T03:24:52.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.623+0000 7f745ff83740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fabe49690 0x560fabe22750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.623+0000 7f745ff83740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560fabe49a70 0x560fabe75e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.623+0000 7f745ff83740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fabe61cf0 0x560fabe356e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.623+0000 7f745ff83740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fabe22c90 con 0x560fabe49690 2023-09-23T03:24:52.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.623+0000 7f745ff83740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fabe22e00 con 0x560fabe61cf0 2023-09-23T03:24:52.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.623+0000 7f745ff83740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x560fabe658d0 con 0x560fabe49a70 2023-09-23T03:24:52.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.623+0000 7f745ff83740 10 client.0 _ll_register_callbacks cb 0x7f7448009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:24:52.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.623+0000 7f7456ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fabe49690 0x560fabe22750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:52.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.623+0000 7f7456ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fabe49690 0x560fabe22750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52766/0 (socket says 172.21.15.154:52766) 2023-09-23T03:24:52.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.623+0000 7f7456ffd640 1 -- 172.21.15.154:0/194119899 learned_addr learned my addr 172.21.15.154:0/194119899 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:24:52.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.624+0000 7f7456ffd640 1 -- 172.21.15.154:0/194119899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560fabe49a70 msgr2=0x560fabe75e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:52.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.624+0000 7f7456ffd640 1 --2- 172.21.15.154:0/194119899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560fabe49a70 0x560fabe75e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.624+0000 7f7456ffd640 1 -- 172.21.15.154:0/194119899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fabe61cf0 msgr2=0x560fabe356e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:24:52.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.624+0000 7f7456ffd640 1 --2- 172.21.15.154:0/194119899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fabe61cf0 0x560fabe356e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.624+0000 7f7456ffd640 1 -- 172.21.15.154:0/194119899 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560fabd24ce0 con 0x560fabe49690 2023-09-23T03:24:52.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.624+0000 7f7456ffd640 1 --2- 172.21.15.154:0/194119899 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fabe49690 0x560fabe22750 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f7448020000 tx=0x7f74480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc04e11bf89dc7db server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:24:52.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.624+0000 7f7435ffb640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:24:52.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.624+0000 7f7435ffb640 1 -- 172.21.15.154:0/194119899 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744802f190 con 0x560fabe49690 2023-09-23T03:24:52.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.624+0000 7f7435ffb640 1 -- 172.21.15.154:0/194119899 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f744802f2f0 con 0x560fabe49690 2023-09-23T03:24:52.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.624+0000 7f745ff83740 1 -- 172.21.15.154:0/194119899 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7448015860 con 0x560fabe49690 2023-09-23T03:24:52.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.624+0000 7f7435ffb640 1 -- 172.21.15.154:0/194119899 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f744802f4b0 con 0x560fabe49690 2023-09-23T03:24:52.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.624+0000 7f745ff83740 10 client.7857 fetch_fsmap learned FSMap version 76 2023-09-23T03:24:52.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.624+0000 7f745ff83740 1 -- 172.21.15.154:0/194119899 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x560fabd4a2d0 con 0x560fabe49690 2023-09-23T03:24:52.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.625+0000 7f7435ffb640 1 -- 172.21.15.154:0/194119899 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7448015860 con 0x560fabe49690 2023-09-23T03:24:52.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.625+0000 7f7435ffb640 1 -- 172.21.15.154:0/194119899 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 76) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f74480159d0 con 0x560fabe49690 2023-09-23T03:24:52.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.625+0000 7f745ff83740 10 client.7857 fetch_fsmap finished waiting for FSMap version 76 2023-09-23T03:24:52.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.625+0000 7f745ff83740 10 client.7857 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2023-09-23T03:24:52.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.625+0000 7f745ff83740 20 client.7857 populate_metadata read hostname 'smithi154' 2023-09-23T03:24:52.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.625+0000 7f745ff83740 1 --2- 172.21.15.154:0/194119899 >> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] conn(0x560fabe35c20 0x560fabe79290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:24:52.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.625+0000 7f745ff83740 4 client.7857 mds_command: new command op to 7758 tid=0[{"prefix": "scrub status"}] 2023-09-23T03:24:52.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.625+0000 7f745ff83740 1 -- 172.21.15.154:0/194119899 --> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560fabd80a10 con 0x560fabe35c20 2023-09-23T03:24:52.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.626+0000 7f744e7fc640 1 --2- 172.21.15.154:0/194119899 >> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] conn(0x560fabe35c20 0x560fabe79290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:24:52.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.627+0000 7f744e7fc640 1 --2- 172.21.15.154:0/194119899 >> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] conn(0x560fabe35c20 0x560fabe79290 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f36333539df45c6d server_cookie=35ced9786717965 in_seq=0 out_seq=0 2023-09-23T03:24:52.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.627+0000 7f7435ffb640 10 client.7857 ms_handle_connect on v2:172.21.15.190:6833/2202597679 2023-09-23T03:24:52.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.627+0000 7f7435ffb640 1 -- 172.21.15.154:0/194119899 <== mds.0 v2:172.21.15.190:6833/2202597679 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x560fabd80a10 con 0x560fabe35c20 2023-09-23T03:24:52.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.627+0000 7f7435ffb640 10 client.7857 handle_command_reply: tid=0 2023-09-23T03:24:52.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.627+0000 7f745ff83740 1 client.7857 shutdown 2023-09-23T03:24:52.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.627+0000 7f745ff83740 1 -- 172.21.15.154:0/194119899 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fabe49690 msgr2=0x560fabe22750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:52.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.627+0000 7f745ff83740 1 --2- 172.21.15.154:0/194119899 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fabe49690 0x560fabe22750 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f7448020000 tx=0x7f74480046c0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.628+0000 7f745ff83740 1 -- 172.21.15.154:0/194119899 shutdown_connections 2023-09-23T03:24:52.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.628+0000 7f745ff83740 1 --2- 172.21.15.154:0/194119899 >> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] conn(0x560fabe35c20 0x560fabe79290 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.628+0000 7f745ff83740 1 --2- 172.21.15.154:0/194119899 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560fabe61cf0 0x560fabe356e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.628+0000 7f745ff83740 1 --2- 172.21.15.154:0/194119899 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560fabe49a70 0x560fabe75e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.628+0000 7f745ff83740 1 --2- 172.21.15.154:0/194119899 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560fabe49690 0x560fabe22750 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.628+0000 7f7435ffb640 0 client.7857 ms_handle_reset on v2:172.21.15.190:6833/2202597679 2023-09-23T03:24:52.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.628+0000 7f745ff83740 1 -- 172.21.15.154:0/194119899 >> 172.21.15.154:0/194119899 conn(0x560fabe1cad0 msgr2=0x560fabe373e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:52.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.628+0000 7f745ff83740 1 -- 172.21.15.154:0/194119899 shutdown_connections 2023-09-23T03:24:52.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.628+0000 7f745ff83740 1 -- 172.21.15.154:0/194119899 wait complete. 2023-09-23T03:24:52.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.628+0000 7f745ff83740 20 client.7857 trim_cache size 0 max 16384 2023-09-23T03:24:52.633 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:24:52.633 INFO:teuthology.orchestra.run.smithi154.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T03:24:52.633 INFO:teuthology.orchestra.run.smithi154.stdout: "scrubs": { 2023-09-23T03:24:52.633 INFO:teuthology.orchestra.run.smithi154.stdout: "dc1a4afd-c3bb-4268-a271-e1eaf4b50151": { 2023-09-23T03:24:52.633 INFO:teuthology.orchestra.run.smithi154.stdout: "path": "/", 2023-09-23T03:24:52.633 INFO:teuthology.orchestra.run.smithi154.stdout: "tag": "dc1a4afd-c3bb-4268-a271-e1eaf4b50151", 2023-09-23T03:24:52.633 INFO:teuthology.orchestra.run.smithi154.stdout: "options": "recursive,repair" 2023-09-23T03:24:52.633 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:24:52.634 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:24:52.634 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:24:52.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.628+0000 7f745d3f5640 1 -- 172.21.15.154:0/2473654613 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f743806af60 msgr2=0x7f743806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:52.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.628+0000 7f745d3f5640 1 --2- 172.21.15.154:0/2473654613 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f743806af60 0x7f743806d420 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f7448006fd0 tx=0x7f7448008040 comp rx=0 tx=0).stop 2023-09-23T03:24:52.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.628+0000 7f745d3f5640 1 -- 172.21.15.154:0/2473654613 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74580cae90 msgr2=0x7f745815e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:24:52.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.628+0000 7f745d3f5640 1 --2- 172.21.15.154:0/2473654613 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74580cae90 0x7f745815e520 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f745002bf70 tx=0x7f7450004730 comp rx=0 tx=0).stop 2023-09-23T03:24:52.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.629+0000 7f745d3f5640 1 -- 172.21.15.154:0/2473654613 shutdown_connections 2023-09-23T03:24:52.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.629+0000 7f745d3f5640 1 --2- 172.21.15.154:0/2473654613 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f743806af60 0x7f743806d420 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.629+0000 7f745d3f5640 1 --2- 172.21.15.154:0/2473654613 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74580cae90 0x7f745815e520 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.629+0000 7f745d3f5640 1 --2- 172.21.15.154:0/2473654613 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f74580c9980 0x7f74580b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.629+0000 7f745d3f5640 1 --2- 172.21.15.154:0/2473654613 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f74580c9510 0x7f74581f72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:24:52.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.629+0000 7f745d3f5640 1 -- 172.21.15.154:0/2473654613 >> 172.21.15.154:0/2473654613 conn(0x7f745806c010 msgr2=0x7f745815f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:24:52.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.629+0000 7f745d3f5640 1 -- 172.21.15.154:0/2473654613 shutdown_connections 2023-09-23T03:24:52.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:24:52.629+0000 7f745d3f5640 1 -- 172.21.15.154:0/2473654613 wait complete. 2023-09-23T03:24:52.646 INFO:tasks.cephfs.filesystem:scrub status for tag:dc1a4afd-c3bb-4268-a271-e1eaf4b50151 - {'path': '/', 'tag': 'dc1a4afd-c3bb-4268-a271-e1eaf4b50151', 'options': 'recursive,repair'} 2023-09-23T03:25:13.747 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:25:13.756 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:25:22.647 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2023-09-23T03:25:22.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.871+0000 7f80828ec640 1 Processor -- start 2023-09-23T03:25:22.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.871+0000 7f80828ec640 1 -- start start 2023-09-23T03:25:22.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.872+0000 7f80828ec640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f807c0c8d70 0x7f807c15c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:22.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.873+0000 7f80828ec640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f807c0c9150 0x7f807c15cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:22.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.873+0000 7f80828ec640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f807c071290 0x7f807c0ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:22.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.873+0000 7f80828ec640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f807c0c9d20 con 0x7f807c0c8d70 2023-09-23T03:25:22.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.873+0000 7f80828ec640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f807c071850 con 0x7f807c0c9150 2023-09-23T03:25:22.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.873+0000 7f80828ec640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f807c071b30 con 0x7f807c071290 2023-09-23T03:25:22.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.873+0000 7f807bfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f807c0c8d70 0x7f807c15c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:22.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.873+0000 7f807bfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f807c0c8d70 0x7f807c15c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:38312/0 (socket says 172.21.15.154:38312) 2023-09-23T03:25:22.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.873+0000 7f807bfff640 1 -- 172.21.15.154:0/2602020102 learned_addr learned my addr 172.21.15.154:0/2602020102 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:22.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.873+0000 7f807bfff640 1 -- 172.21.15.154:0/2602020102 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f807c071290 msgr2=0x7f807c0ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:22.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.873+0000 7f8080e89640 1 --2- 172.21.15.154:0/2602020102 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f807c071290 0x7f807c0ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:22.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.873+0000 7f807b7fe640 1 --2- 172.21.15.154:0/2602020102 >> v2:172.21.15.190:3300/0 conn(0x7f807c0c9150 0x7f807c15cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:22.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.873+0000 7f807bfff640 1 --2- 172.21.15.154:0/2602020102 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f807c071290 0x7f807c0ca1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:22.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.873+0000 7f807bfff640 1 -- 172.21.15.154:0/2602020102 >> v2:172.21.15.190:3300/0 conn(0x7f807c0c9150 msgr2=0x7f807c15cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:22.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.873+0000 7f807bfff640 1 --2- 172.21.15.154:0/2602020102 >> v2:172.21.15.190:3300/0 conn(0x7f807c0c9150 0x7f807c15cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:22.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.874+0000 7f807bfff640 1 -- 172.21.15.154:0/2602020102 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f807c071e10 con 0x7f807c0c8d70 2023-09-23T03:25:22.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.875+0000 7f807bfff640 1 --2- 172.21.15.154:0/2602020102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f807c0c8d70 0x7f807c15c6d0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f807c0b3ce0 tx=0x7f806c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfac0b8071a0e671 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:22.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.875+0000 7f807affd640 1 -- 172.21.15.154:0/2602020102 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806c031860 con 0x7f807c0c8d70 2023-09-23T03:25:22.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.875+0000 7f807affd640 1 -- 172.21.15.154:0/2602020102 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f806c0319c0 con 0x7f807c0c8d70 2023-09-23T03:25:22.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.875+0000 7f807affd640 1 -- 172.21.15.154:0/2602020102 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806c031d50 con 0x7f807c0c8d70 2023-09-23T03:25:22.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.876+0000 7f80828ec640 1 -- 172.21.15.154:0/2602020102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f807c0c8d70 msgr2=0x7f807c15c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:22.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.876+0000 7f80828ec640 1 --2- 172.21.15.154:0/2602020102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f807c0c8d70 0x7f807c15c6d0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f807c0b3ce0 tx=0x7f806c02b900 comp rx=0 tx=0).stop 2023-09-23T03:25:22.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.876+0000 7f80828ec640 1 -- 172.21.15.154:0/2602020102 shutdown_connections 2023-09-23T03:25:22.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.876+0000 7f80828ec640 1 --2- 172.21.15.154:0/2602020102 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f807c071290 0x7f807c0ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:22.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.876+0000 7f80828ec640 1 --2- 172.21.15.154:0/2602020102 >> v2:172.21.15.190:3300/0 conn(0x7f807c0c9150 0x7f807c15cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:22.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.876+0000 7f80828ec640 1 --2- 172.21.15.154:0/2602020102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f807c0c8d70 0x7f807c15c6d0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:22.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.876+0000 7f80828ec640 1 -- 172.21.15.154:0/2602020102 >> 172.21.15.154:0/2602020102 conn(0x7f807c06c010 msgr2=0x7f807c06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:22.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.876+0000 7f80828ec640 1 -- 172.21.15.154:0/2602020102 shutdown_connections 2023-09-23T03:25:22.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.876+0000 7f80828ec640 1 -- 172.21.15.154:0/2602020102 wait complete. 2023-09-23T03:25:22.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.878+0000 7f80828ec640 1 Processor -- start 2023-09-23T03:25:22.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.878+0000 7f80828ec640 1 -- start start 2023-09-23T03:25:22.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.879+0000 7f80828ec640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f807c071290 0x7f807c0c24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:22.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.879+0000 7f80828ec640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f807c0c8d70 0x7f807c146f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:22.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.880+0000 7f80828ec640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f807c0c9150 0x7f807c0ce160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:22.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.880+0000 7f80828ec640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807c0c5580 con 0x7f807c0c8d70 2023-09-23T03:25:22.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.880+0000 7f80828ec640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807c0c2a30 con 0x7f807c0c9150 2023-09-23T03:25:22.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.880+0000 7f80828ec640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f807c0c57d0 con 0x7f807c071290 2023-09-23T03:25:22.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.880+0000 7f807b7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f807c0c8d70 0x7f807c146f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:22.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.880+0000 7f807b7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f807c0c8d70 0x7f807c146f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:38318/0 (socket says 172.21.15.154:38318) 2023-09-23T03:25:22.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.880+0000 7f807b7fe640 1 -- 172.21.15.154:0/4266789766 learned_addr learned my addr 172.21.15.154:0/4266789766 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:22.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.880+0000 7f807bfff640 1 --2- 172.21.15.154:0/4266789766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f807c071290 0x7f807c0c24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:22.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.880+0000 7f8080e89640 1 --2- 172.21.15.154:0/4266789766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f807c0c9150 0x7f807c0ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:22.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.880+0000 7f807b7fe640 1 -- 172.21.15.154:0/4266789766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f807c071290 msgr2=0x7f807c0c24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:22.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.880+0000 7f807b7fe640 1 --2- 172.21.15.154:0/4266789766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f807c071290 0x7f807c0c24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:22.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.880+0000 7f807b7fe640 1 -- 172.21.15.154:0/4266789766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f807c0c9150 msgr2=0x7f807c0ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:22.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.880+0000 7f807b7fe640 1 --2- 172.21.15.154:0/4266789766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f807c0c9150 0x7f807c0ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:22.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.880+0000 7f807b7fe640 1 -- 172.21.15.154:0/4266789766 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f806c0094d0 con 0x7f807c0c8d70 2023-09-23T03:25:22.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.881+0000 7f807b7fe640 1 --2- 172.21.15.154:0/4266789766 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f807c0c8d70 0x7f807c146f20 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f807000c820 tx=0x7f807000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:22.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.881+0000 7f8078ff9640 1 -- 172.21.15.154:0/4266789766 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8070004220 con 0x7f807c0c8d70 2023-09-23T03:25:22.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.881+0000 7f8078ff9640 1 -- 172.21.15.154:0/4266789766 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80700043a0 con 0x7f807c0c8d70 2023-09-23T03:25:22.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.881+0000 7f8078ff9640 1 -- 172.21.15.154:0/4266789766 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8070007a70 con 0x7f807c0c8d70 2023-09-23T03:25:22.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.881+0000 7f80828ec640 1 -- 172.21.15.154:0/4266789766 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f807c0c5ab0 con 0x7f807c0c8d70 2023-09-23T03:25:22.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.881+0000 7f80828ec640 1 -- 172.21.15.154:0/4266789766 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f807c0c6070 con 0x7f807c0c8d70 2023-09-23T03:25:22.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.883+0000 7f8078ff9640 1 -- 172.21.15.154:0/4266789766 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f8070007bd0 con 0x7f807c0c8d70 2023-09-23T03:25:22.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.885+0000 7f8078ff9640 1 --2- 172.21.15.154:0/4266789766 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f806806af30 0x7f806806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:22.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.885+0000 7f8078ff9640 1 -- 172.21.15.154:0/4266789766 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f80700899a0 con 0x7f807c0c8d70 2023-09-23T03:25:22.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.885+0000 7f807bfff640 1 --2- 172.21.15.154:0/4266789766 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f806806af30 0x7f806806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:22.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.886+0000 7f8085481740 1 Processor -- start 2023-09-23T03:25:22.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.886+0000 7f807bfff640 1 --2- 172.21.15.154:0/4266789766 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f806806af30 0x7f806806d3f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f806c036000 tx=0x7f806c03c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:22.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.886+0000 7f8085481740 1 -- start start 2023-09-23T03:25:22.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.887+0000 7f8085481740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56266e846690 0x56266e819590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:22.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.887+0000 7f8085481740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56266e846a70 0x56266e849420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:22.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.887+0000 7f8085481740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56266e819ad0 0x56266e849960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:22.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.887+0000 7f8085481740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x56266e78ba50 con 0x56266e846a70 2023-09-23T03:25:22.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.887+0000 7f8085481740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56266e789140 con 0x56266e819ad0 2023-09-23T03:25:22.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.887+0000 7f8085481740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56266e784a30 con 0x56266e846690 2023-09-23T03:25:22.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.887+0000 7f807b7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56266e846a70 0x56266e849420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:22.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.887+0000 7f807b7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56266e846a70 0x56266e849420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:38332/0 (socket says 172.21.15.154:38332) 2023-09-23T03:25:22.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.887+0000 7f807b7fe640 1 -- 172.21.15.154:0/3358230350 learned_addr learned my addr 172.21.15.154:0/3358230350 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:22.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.888+0000 7f807b7fe640 1 -- 172.21.15.154:0/3358230350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56266e846690 msgr2=0x56266e819590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:22.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.888+0000 7f807b7fe640 1 --2- 172.21.15.154:0/3358230350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56266e846690 0x56266e819590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:22.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.888+0000 7f807b7fe640 1 -- 172.21.15.154:0/3358230350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56266e819ad0 msgr2=0x56266e849960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:22.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.888+0000 7f807b7fe640 1 --2- 172.21.15.154:0/3358230350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56266e819ad0 0x56266e849960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:22.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.888+0000 7f807b7fe640 1 -- 172.21.15.154:0/3358230350 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56266e748a70 con 0x56266e846a70 2023-09-23T03:25:22.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.888+0000 7f807b7fe640 1 --2- 172.21.15.154:0/3358230350 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56266e846a70 0x56266e849420 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f807005fff0 tx=0x7f8070058120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92aedf0b62b1bdd4 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:22.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.888+0000 7f8061ffb640 1 -- 172.21.15.154:0/3358230350 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807005c370 con 0x56266e846a70 2023-09-23T03:25:22.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.888+0000 7f8061ffb640 1 -- 172.21.15.154:0/3358230350 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8070060520 con 0x56266e846a70 2023-09-23T03:25:22.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.888+0000 7f8061ffb640 1 -- 172.21.15.154:0/3358230350 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807000ce10 con 0x56266e846a70 2023-09-23T03:25:22.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.889+0000 7f8085481740 1 -- 172.21.15.154:0/3358230350 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56266e846a70 msgr2=0x56266e849420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:22.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.889+0000 7f8085481740 1 --2- 172.21.15.154:0/3358230350 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56266e846a70 0x56266e849420 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f807005fff0 tx=0x7f8070058120 comp rx=0 tx=0).stop 2023-09-23T03:25:22.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.889+0000 7f8085481740 1 -- 172.21.15.154:0/3358230350 shutdown_connections 2023-09-23T03:25:22.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.889+0000 7f8085481740 1 --2- 172.21.15.154:0/3358230350 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56266e819ad0 0x56266e849960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:22.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.889+0000 7f8085481740 1 --2- 172.21.15.154:0/3358230350 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56266e846a70 0x56266e849420 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:22.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.889+0000 7f8085481740 1 --2- 172.21.15.154:0/3358230350 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56266e846690 0x56266e819590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:22.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.889+0000 7f8085481740 1 -- 172.21.15.154:0/3358230350 >> 172.21.15.154:0/3358230350 conn(0x56266e81b8a0 msgr2=0x56266e864420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:22.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.889+0000 7f8085481740 1 -- 172.21.15.154:0/3358230350 shutdown_connections 2023-09-23T03:25:22.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.889+0000 7f8085481740 1 -- 172.21.15.154:0/3358230350 wait complete. 2023-09-23T03:25:22.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.890+0000 7f8085481740 1 -- start start 2023-09-23T03:25:22.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.890+0000 7f8085481740 1 Processor -- start 2023-09-23T03:25:22.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.891+0000 7f8085481740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56266e846690 0x56266e83a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:22.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.891+0000 7f8085481740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56266e846a70 0x56266e84c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:22.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.891+0000 7f8085481740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56266e85e0e0 0x56266e85e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:22.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.891+0000 7f8085481740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x56266e722ad0 con 0x56266e846a70 2023-09-23T03:25:22.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.891+0000 7f8085481740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56266e722770 con 0x56266e85e0e0 2023-09-23T03:25:22.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.891+0000 7f8085481740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56266e47ef90 con 0x56266e846690 2023-09-23T03:25:22.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.891+0000 7f8085481740 10 client.0 _ll_register_callbacks cb 0x56266e746220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:25:22.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.891+0000 7f807b7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56266e846a70 0x56266e84c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:22.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.891+0000 7f807b7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56266e846a70 0x56266e84c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:38338/0 (socket says 172.21.15.154:38338) 2023-09-23T03:25:22.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.891+0000 7f807b7fe640 1 -- 172.21.15.154:0/2754812154 learned_addr learned my addr 172.21.15.154:0/2754812154 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:22.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.892+0000 7f807b7fe640 1 -- 172.21.15.154:0/2754812154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56266e846690 msgr2=0x56266e83a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:22.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.892+0000 7f807bfff640 1 --2- 172.21.15.154:0/2754812154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56266e846690 0x56266e83a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:22.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.892+0000 7f8080e89640 1 --2- 172.21.15.154:0/2754812154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56266e85e0e0 0x56266e85e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:22.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.892+0000 7f807b7fe640 1 --2- 172.21.15.154:0/2754812154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56266e846690 0x56266e83a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:22.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.892+0000 7f807b7fe640 1 -- 172.21.15.154:0/2754812154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56266e85e0e0 msgr2=0x56266e85e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:22.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.892+0000 7f807b7fe640 1 --2- 172.21.15.154:0/2754812154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56266e85e0e0 0x56266e85e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:22.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.892+0000 7f807b7fe640 1 -- 172.21.15.154:0/2754812154 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f807c0c6070 con 0x56266e846a70 2023-09-23T03:25:22.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.892+0000 7f8080e89640 1 --2- 172.21.15.154:0/2754812154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56266e85e0e0 0x56266e85e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:22.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.892+0000 7f807b7fe640 1 --2- 172.21.15.154:0/2754812154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56266e846a70 0x56266e84c690 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f8070054720 tx=0x7f8070006f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1cbdbd95670ef00 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:22.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.892+0000 7f805affd640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:25:22.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.892+0000 7f805affd640 1 -- 172.21.15.154:0/2754812154 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8070060290 con 0x56266e846a70 2023-09-23T03:25:22.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.892+0000 7f8085481740 1 -- 172.21.15.154:0/2754812154 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56266e7ab4d0 con 0x56266e846a70 2023-09-23T03:25:22.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.892+0000 7f805affd640 1 -- 172.21.15.154:0/2754812154 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f807005c370 con 0x56266e846a70 2023-09-23T03:25:22.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.892+0000 7f805affd640 1 -- 172.21.15.154:0/2754812154 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f807000ce10 con 0x56266e846a70 2023-09-23T03:25:22.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.892+0000 7f8085481740 10 client.7869 fetch_fsmap learned FSMap version 76 2023-09-23T03:25:22.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.892+0000 7f8085481740 1 -- 172.21.15.154:0/2754812154 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x56266e7830b0 con 0x56266e846a70 2023-09-23T03:25:22.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.893+0000 7f805affd640 1 -- 172.21.15.154:0/2754812154 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56266e7ab4d0 con 0x56266e846a70 2023-09-23T03:25:22.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.893+0000 7f805affd640 1 -- 172.21.15.154:0/2754812154 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 76) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f807006fba0 con 0x56266e846a70 2023-09-23T03:25:22.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.893+0000 7f8085481740 10 client.7869 fetch_fsmap finished waiting for FSMap version 76 2023-09-23T03:25:22.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.893+0000 7f8085481740 10 client.7869 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2023-09-23T03:25:22.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.893+0000 7f8085481740 20 client.7869 populate_metadata read hostname 'smithi154' 2023-09-23T03:25:22.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.893+0000 7f8085481740 1 --2- 172.21.15.154:0/2754812154 >> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] conn(0x56266e876a20 0x56266e876e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:22.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.893+0000 7f8085481740 4 client.7869 mds_command: new command op to 7758 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T03:25:22.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.893+0000 7f8085481740 1 -- 172.21.15.154:0/2754812154 --> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56266e757ab0 con 0x56266e876a20 2023-09-23T03:25:22.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.894+0000 7f807bfff640 1 --2- 172.21.15.154:0/2754812154 >> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] conn(0x56266e876a20 0x56266e876e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:22.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.895+0000 7f805affd640 10 client.7869 ms_handle_connect on v2:172.21.15.190:6833/2202597679 2023-09-23T03:25:22.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.895+0000 7f807bfff640 1 --2- 172.21.15.154:0/2754812154 >> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] conn(0x56266e876a20 0x56266e876e30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd1e5cbad43de1f2 server_cookie=639a0b6d1e61c2eb in_seq=0 out_seq=0 2023-09-23T03:25:22.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.896+0000 7f805affd640 1 -- 172.21.15.154:0/2754812154 <== mds.0 v2:172.21.15.190:6833/2202597679 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56266e757ab0 con 0x56266e876a20 2023-09-23T03:25:22.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.896+0000 7f805affd640 10 client.7869 handle_command_reply: tid=0 2023-09-23T03:25:22.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.896+0000 7f8085481740 1 client.7869 shutdown 2023-09-23T03:25:22.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.897+0000 7f8085481740 1 -- 172.21.15.154:0/2754812154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56266e846a70 msgr2=0x56266e84c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:22.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.897+0000 7f8085481740 1 --2- 172.21.15.154:0/2754812154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56266e846a70 0x56266e84c690 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f8070054720 tx=0x7f8070006f30 comp rx=0 tx=0).stop 2023-09-23T03:25:22.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.897+0000 7f8085481740 1 -- 172.21.15.154:0/2754812154 shutdown_connections 2023-09-23T03:25:22.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.897+0000 7f8085481740 1 --2- 172.21.15.154:0/2754812154 >> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] conn(0x56266e876a20 0x56266e876e30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:22.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.897+0000 7f8085481740 1 --2- 172.21.15.154:0/2754812154 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56266e85e0e0 0x56266e85e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:22.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.897+0000 7f8085481740 1 --2- 172.21.15.154:0/2754812154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56266e846a70 0x56266e84c690 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:22.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.897+0000 7f805affd640 0 client.7869 ms_handle_reset on v2:172.21.15.190:6833/2202597679 2023-09-23T03:25:22.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.897+0000 7f8085481740 1 --2- 172.21.15.154:0/2754812154 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56266e846690 0x56266e83a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:22.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.897+0000 7f8085481740 1 -- 172.21.15.154:0/2754812154 >> 172.21.15.154:0/2754812154 conn(0x56266e819ad0 msgr2=0x56266e81d540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:22.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.897+0000 7f8085481740 1 -- 172.21.15.154:0/2754812154 shutdown_connections 2023-09-23T03:25:22.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.897+0000 7f8085481740 1 -- 172.21.15.154:0/2754812154 wait complete. 2023-09-23T03:25:22.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.897+0000 7f8085481740 20 client.7869 trim_cache size 0 max 16384 2023-09-23T03:25:22.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.913+0000 7f8085481740 1 Processor -- start 2023-09-23T03:25:22.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.913+0000 7f8085481740 1 -- start start 2023-09-23T03:25:22.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.914+0000 7f8085481740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56266e846690 0x56266e8cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:22.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.914+0000 7f8085481740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56266e846a70 0x56266e84c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:22.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.914+0000 7f8085481740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56266e85e0e0 0x56266e85d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:22.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.914+0000 7f8085481740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x56266e818f10 con 0x56266e846690 2023-09-23T03:25:22.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.914+0000 7f8085481740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56266e81afd0 con 0x56266e846a70 2023-09-23T03:25:22.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.914+0000 7f8085481740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56266e84d3d0 con 0x56266e85e0e0 2023-09-23T03:25:22.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.914+0000 7f807bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56266e846690 0x56266e8cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:22.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.914+0000 7f807bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56266e846690 0x56266e8cbec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:38346/0 (socket says 172.21.15.154:38346) 2023-09-23T03:25:22.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.914+0000 7f807bfff640 1 -- 172.21.15.154:0/1177626142 learned_addr learned my addr 172.21.15.154:0/1177626142 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:22.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.914+0000 7f807bfff640 1 -- 172.21.15.154:0/1177626142 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56266e85e0e0 msgr2=0x56266e85d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:22.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.914+0000 7f807bfff640 1 --2- 172.21.15.154:0/1177626142 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56266e85e0e0 0x56266e85d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:22.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.914+0000 7f807bfff640 1 -- 172.21.15.154:0/1177626142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56266e846a70 msgr2=0x56266e84c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:22.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.914+0000 7f807b7fe640 1 --2- 172.21.15.154:0/1177626142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56266e846a70 0x56266e84c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:22.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.914+0000 7f807bfff640 1 --2- 172.21.15.154:0/1177626142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56266e846a70 0x56266e84c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:22.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.914+0000 7f807bfff640 1 -- 172.21.15.154:0/1177626142 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56266e7b4da0 con 0x56266e846690 2023-09-23T03:25:22.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.914+0000 7f807b7fe640 1 --2- 172.21.15.154:0/1177626142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56266e846a70 0x56266e84c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:22.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.915+0000 7f807bfff640 1 --2- 172.21.15.154:0/1177626142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56266e846690 0x56266e8cbec0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f806c02b940 tx=0x7f806c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=341d964d3477d3a9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:22.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.915+0000 7f80627fc640 1 -- 172.21.15.154:0/1177626142 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806c053070 con 0x56266e846690 2023-09-23T03:25:22.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.915+0000 7f80627fc640 1 -- 172.21.15.154:0/1177626142 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f806c04f3b0 con 0x56266e846690 2023-09-23T03:25:22.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.915+0000 7f80627fc640 1 -- 172.21.15.154:0/1177626142 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806c04a040 con 0x56266e846690 2023-09-23T03:25:22.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.916+0000 7f8085481740 1 -- 172.21.15.154:0/1177626142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56266e846690 msgr2=0x56266e8cbec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:22.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.916+0000 7f8085481740 1 --2- 172.21.15.154:0/1177626142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56266e846690 0x56266e8cbec0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f806c02b940 tx=0x7f806c02e000 comp rx=0 tx=0).stop 2023-09-23T03:25:22.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.916+0000 7f8085481740 1 -- 172.21.15.154:0/1177626142 shutdown_connections 2023-09-23T03:25:22.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.916+0000 7f8085481740 1 --2- 172.21.15.154:0/1177626142 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56266e85e0e0 0x56266e85d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:22.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.916+0000 7f8085481740 1 --2- 172.21.15.154:0/1177626142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56266e846a70 0x56266e84c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:22.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.916+0000 7f8085481740 1 --2- 172.21.15.154:0/1177626142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56266e846690 0x56266e8cbec0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:22.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.916+0000 7f8085481740 1 -- 172.21.15.154:0/1177626142 >> 172.21.15.154:0/1177626142 conn(0x56266e819ad0 msgr2=0x56266e848e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:22.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.916+0000 7f8085481740 1 -- 172.21.15.154:0/1177626142 shutdown_connections 2023-09-23T03:25:22.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.916+0000 7f8085481740 1 -- 172.21.15.154:0/1177626142 wait complete. 2023-09-23T03:25:22.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.916+0000 7f8085481740 1 -- start start 2023-09-23T03:25:22.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.917+0000 7f8085481740 1 Processor -- start 2023-09-23T03:25:22.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.918+0000 7f8085481740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56266e846690 0x56266e8c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:22.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.918+0000 7f8085481740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56266e846a70 0x56266e8ced00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:22.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.918+0000 7f8085481740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56266e85e0e0 0x56266e8626a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:22.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.918+0000 7f8085481740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x56266e863200 con 0x56266e846690 2023-09-23T03:25:22.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.918+0000 7f8085481740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56266e8cdc80 con 0x56266e85e0e0 2023-09-23T03:25:22.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.918+0000 7f8085481740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56266e8cdf40 con 0x56266e846a70 2023-09-23T03:25:22.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.918+0000 7f8085481740 10 client.0 _ll_register_callbacks cb 0x7f8070007f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:25:22.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.918+0000 7f807bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56266e846690 0x56266e8c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:22.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.918+0000 7f807bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56266e846690 0x56266e8c8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:38362/0 (socket says 172.21.15.154:38362) 2023-09-23T03:25:22.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.918+0000 7f807bfff640 1 -- 172.21.15.154:0/1392257125 learned_addr learned my addr 172.21.15.154:0/1392257125 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:22.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.918+0000 7f807bfff640 1 -- 172.21.15.154:0/1392257125 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56266e846a70 msgr2=0x56266e8ced00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:22.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.918+0000 7f807bfff640 1 --2- 172.21.15.154:0/1392257125 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56266e846a70 0x56266e8ced00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:22.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.918+0000 7f807bfff640 1 -- 172.21.15.154:0/1392257125 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56266e85e0e0 msgr2=0x56266e8626a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:22.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.918+0000 7f807bfff640 1 --2- 172.21.15.154:0/1392257125 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56266e85e0e0 0x56266e8626a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:22.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.918+0000 7f807bfff640 1 -- 172.21.15.154:0/1392257125 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f807c071e10 con 0x56266e846690 2023-09-23T03:25:22.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.919+0000 7f807bfff640 1 --2- 172.21.15.154:0/1392257125 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56266e846690 0x56266e8c8dc0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f806c051040 tx=0x7f806c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505c89a99532e086 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:22.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.919+0000 7f805affd640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:25:22.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.919+0000 7f805affd640 1 -- 172.21.15.154:0/1392257125 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806c053040 con 0x56266e846690 2023-09-23T03:25:22.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.919+0000 7f805affd640 1 -- 172.21.15.154:0/1392257125 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f806c03f070 con 0x56266e846690 2023-09-23T03:25:22.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.919+0000 7f8085481740 1 -- 172.21.15.154:0/1392257125 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8070060520 con 0x56266e846690 2023-09-23T03:25:22.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.919+0000 7f805affd640 1 -- 172.21.15.154:0/1392257125 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f806c04a040 con 0x56266e846690 2023-09-23T03:25:22.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.919+0000 7f8085481740 10 client.7875 fetch_fsmap learned FSMap version 76 2023-09-23T03:25:22.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.919+0000 7f8085481740 1 -- 172.21.15.154:0/1392257125 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x56266e721ce0 con 0x56266e846690 2023-09-23T03:25:22.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.919+0000 7f805affd640 1 -- 172.21.15.154:0/1392257125 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8070060520 con 0x56266e846690 2023-09-23T03:25:22.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.919+0000 7f805affd640 1 -- 172.21.15.154:0/1392257125 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 76) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f806c04f900 con 0x56266e846690 2023-09-23T03:25:22.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.919+0000 7f8085481740 10 client.7875 fetch_fsmap finished waiting for FSMap version 76 2023-09-23T03:25:22.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.919+0000 7f8085481740 10 client.7875 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.b 2023-09-23T03:25:22.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.919+0000 7f8085481740 20 client.7875 populate_metadata read hostname 'smithi154' 2023-09-23T03:25:22.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.919+0000 7f8085481740 1 --2- 172.21.15.154:0/1392257125 >> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] conn(0x56266e875ec0 0x56266e844720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:22.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.919+0000 7f8085481740 4 client.7875 mds_command: new command op to 7758 tid=0[{"prefix": "scrub status"}] 2023-09-23T03:25:22.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.919+0000 7f8085481740 1 -- 172.21.15.154:0/1392257125 --> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56266e77da10 con 0x56266e875ec0 2023-09-23T03:25:22.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.920+0000 7f807b7fe640 1 --2- 172.21.15.154:0/1392257125 >> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] conn(0x56266e875ec0 0x56266e844720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:22.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.921+0000 7f807b7fe640 1 --2- 172.21.15.154:0/1392257125 >> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] conn(0x56266e875ec0 0x56266e844720 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bba78336f693e093 server_cookie=75ce88f2bdc9b054 in_seq=0 out_seq=0 2023-09-23T03:25:22.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.921+0000 7f805affd640 10 client.7875 ms_handle_connect on v2:172.21.15.190:6833/2202597679 2023-09-23T03:25:22.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.921+0000 7f805affd640 1 -- 172.21.15.154:0/1392257125 <== mds.0 v2:172.21.15.190:6833/2202597679 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56266e77da10 con 0x56266e875ec0 2023-09-23T03:25:22.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.921+0000 7f805affd640 10 client.7875 handle_command_reply: tid=0 2023-09-23T03:25:22.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.922+0000 7f8085481740 1 client.7875 shutdown 2023-09-23T03:25:22.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.922+0000 7f8085481740 1 -- 172.21.15.154:0/1392257125 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56266e846690 msgr2=0x56266e8c8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:22.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.922+0000 7f8085481740 1 --2- 172.21.15.154:0/1392257125 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56266e846690 0x56266e8c8dc0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f806c051040 tx=0x7f806c004730 comp rx=0 tx=0).stop 2023-09-23T03:25:22.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.922+0000 7f8085481740 1 -- 172.21.15.154:0/1392257125 shutdown_connections 2023-09-23T03:25:22.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.922+0000 7f8085481740 1 --2- 172.21.15.154:0/1392257125 >> [v2:172.21.15.190:6833/2202597679,v1:172.21.15.190:6835/2202597679] conn(0x56266e875ec0 0x56266e844720 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:22.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.922+0000 7f8085481740 1 --2- 172.21.15.154:0/1392257125 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56266e85e0e0 0x56266e8626a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:22.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.922+0000 7f8085481740 1 --2- 172.21.15.154:0/1392257125 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56266e846a70 0x56266e8ced00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:22.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.922+0000 7f8085481740 1 --2- 172.21.15.154:0/1392257125 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56266e846690 0x56266e8c8dc0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:22.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.922+0000 7f805affd640 0 client.7875 ms_handle_reset on v2:172.21.15.190:6833/2202597679 2023-09-23T03:25:22.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.922+0000 7f8085481740 1 -- 172.21.15.154:0/1392257125 >> 172.21.15.154:0/1392257125 conn(0x56266e819ad0 msgr2=0x56266e81d230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:22.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.922+0000 7f8085481740 1 -- 172.21.15.154:0/1392257125 shutdown_connections 2023-09-23T03:25:22.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.922+0000 7f8085481740 1 -- 172.21.15.154:0/1392257125 wait complete. 2023-09-23T03:25:22.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.922+0000 7f8085481740 20 client.7875 trim_cache size 0 max 16384 2023-09-23T03:25:22.927 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:25:22.927 INFO:teuthology.orchestra.run.smithi154.stdout: "status": "no active scrubs running", 2023-09-23T03:25:22.927 INFO:teuthology.orchestra.run.smithi154.stdout: "scrubs": {} 2023-09-23T03:25:22.928 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:25:22.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.923+0000 7f80828ec640 1 -- 172.21.15.154:0/4266789766 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f806806af30 msgr2=0x7f806806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:22.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.923+0000 7f80828ec640 1 --2- 172.21.15.154:0/4266789766 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f806806af30 0x7f806806d3f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f806c036000 tx=0x7f806c03c040 comp rx=0 tx=0).stop 2023-09-23T03:25:22.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.923+0000 7f80828ec640 1 -- 172.21.15.154:0/4266789766 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f807c0c8d70 msgr2=0x7f807c146f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:22.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.923+0000 7f80828ec640 1 --2- 172.21.15.154:0/4266789766 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f807c0c8d70 0x7f807c146f20 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f807000c820 tx=0x7f807000cce0 comp rx=0 tx=0).stop 2023-09-23T03:25:22.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.923+0000 7f80828ec640 1 -- 172.21.15.154:0/4266789766 shutdown_connections 2023-09-23T03:25:22.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.923+0000 7f80828ec640 1 --2- 172.21.15.154:0/4266789766 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f806806af30 0x7f806806d3f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:22.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.923+0000 7f80828ec640 1 --2- 172.21.15.154:0/4266789766 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f807c0c9150 0x7f807c0ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:22.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.923+0000 7f80828ec640 1 --2- 172.21.15.154:0/4266789766 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f807c0c8d70 0x7f807c146f20 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:22.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.923+0000 7f80828ec640 1 --2- 172.21.15.154:0/4266789766 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f807c071290 0x7f807c0c24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:22.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.923+0000 7f80828ec640 1 -- 172.21.15.154:0/4266789766 >> 172.21.15.154:0/4266789766 conn(0x7f807c06c010 msgr2=0x7f807c0b2380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:22.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.923+0000 7f80828ec640 1 -- 172.21.15.154:0/4266789766 shutdown_connections 2023-09-23T03:25:22.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:22.923+0000 7f80828ec640 1 -- 172.21.15.154:0/4266789766 wait complete. 2023-09-23T03:25:22.942 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-09-23T03:25:22.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:25:22.942 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo ls subdir) 2023-09-23T03:25:23.043 INFO:teuthology.orchestra.run.smithi154.stdout:sixmegs 2023-09-23T03:25:23.047 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:25:23.048 DEBUG:teuthology.orchestra.run.smithi154:> import os 2023-09-23T03:25:23.048 DEBUG:teuthology.orchestra.run.smithi154:> import stat 2023-09-23T03:25:23.048 DEBUG:teuthology.orchestra.run.smithi154:> import json 2023-09-23T03:25:23.048 DEBUG:teuthology.orchestra.run.smithi154:> import sys 2023-09-23T03:25:23.048 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:25:23.048 DEBUG:teuthology.orchestra.run.smithi154:> try: 2023-09-23T03:25:23.048 DEBUG:teuthology.orchestra.run.smithi154:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2023-09-23T03:25:23.048 DEBUG:teuthology.orchestra.run.smithi154:> except OSError as e: 2023-09-23T03:25:23.048 DEBUG:teuthology.orchestra.run.smithi154:> sys.exit(e.errno) 2023-09-23T03:25:23.048 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:25:23.048 DEBUG:teuthology.orchestra.run.smithi154:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2023-09-23T03:25:23.049 DEBUG:teuthology.orchestra.run.smithi154:> print(json.dumps( 2023-09-23T03:25:23.049 DEBUG:teuthology.orchestra.run.smithi154:> dict([(a, getattr(s, a)) for a in attrs]), 2023-09-23T03:25:23.049 DEBUG:teuthology.orchestra.run.smithi154:> indent=2)) 2023-09-23T03:25:23.049 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:25:23.228 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:25:23.228 INFO:teuthology.orchestra.run.smithi154.stdout: "st_mode": 33088, 2023-09-23T03:25:23.228 INFO:teuthology.orchestra.run.smithi154.stdout: "st_ino": 1099511627777, 2023-09-23T03:25:23.229 INFO:teuthology.orchestra.run.smithi154.stdout: "st_dev": 39, 2023-09-23T03:25:23.229 INFO:teuthology.orchestra.run.smithi154.stdout: "st_nlink": 1, 2023-09-23T03:25:23.229 INFO:teuthology.orchestra.run.smithi154.stdout: "st_uid": 0, 2023-09-23T03:25:23.229 INFO:teuthology.orchestra.run.smithi154.stdout: "st_gid": 0, 2023-09-23T03:25:23.229 INFO:teuthology.orchestra.run.smithi154.stdout: "st_size": 6291456, 2023-09-23T03:25:23.229 INFO:teuthology.orchestra.run.smithi154.stdout: "st_atime": 1695439475.0, 2023-09-23T03:25:23.229 INFO:teuthology.orchestra.run.smithi154.stdout: "st_mtime": 1695439475.0, 2023-09-23T03:25:23.229 INFO:teuthology.orchestra.run.smithi154.stdout: "st_ctime": 1695439475.0 2023-09-23T03:25:23.229 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:25:23.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:25:23.247 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-09-23T03:25:23.247 DEBUG:teuthology.orchestra.run.smithi154:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-23T03:25:23.402 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-23T03:25:23.518 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-09-23T03:25:23.518 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:25:23.519 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:25:23.579 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:25:23.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.794+0000 7f1539539640 1 Processor -- start 2023-09-23T03:25:23.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.795+0000 7f1539539640 1 -- start start 2023-09-23T03:25:23.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.796+0000 7f1539539640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15340ada30 0x7f15340ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:23.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.796+0000 7f1539539640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f15340cf130 0x7f153415d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:23.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.796+0000 7f1539539640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f153406ceb0 con 0x7f15340b2fe0 2023-09-23T03:25:23.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.796+0000 7f1539539640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15340ade10 con 0x7f15340cf130 2023-09-23T03:25:23.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.796+0000 7f1539539640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1534159c90 con 0x7f15340ada30 2023-09-23T03:25:23.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.797+0000 7f15337fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f15340cf130 0x7f153415d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:23.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.797+0000 7f1532ffd640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15340ada30 0x7f15340ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:23.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.797+0000 7f152bfff640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f15340b2fe0 0x7f15340cea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.154:56214/0 (socket says 172.21.15.154:56214) 2023-09-23T03:25:23.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.797+0000 7f152bfff640 1 -- 172.21.15.154:0/2472400815 learned_addr learned my addr 172.21.15.154:0/2472400815 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:25:23.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.798+0000 7f15327fc640 1 -- 172.21.15.154:0/2472400815 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4244592777 0 0) 0x7f151c001110 con 0x7f15340b2fe0 2023-09-23T03:25:23.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.798+0000 7f15327fc640 1 -- 172.21.15.154:0/2472400815 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1520005730 con 0x7f15340b2fe0 2023-09-23T03:25:23.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.798+0000 7f1532ffd640 1 -- 172.21.15.154:0/2472400815 >> v2:172.21.15.190:3300/0 conn(0x7f15340cf130 msgr2=0x7f153415d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:23.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.798+0000 7f1532ffd640 1 --2- 172.21.15.154:0/2472400815 >> v2:172.21.15.190:3300/0 conn(0x7f15340cf130 0x7f153415d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:23.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.798+0000 7f1532ffd640 1 -- 172.21.15.154:0/2472400815 >> v1:172.21.15.190:6789/0 conn(0x7f15340b2fe0 legacy=0x7f15340cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:23.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.798+0000 7f1532ffd640 1 -- 172.21.15.154:0/2472400815 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f153406c110 con 0x7f15340ada30 2023-09-23T03:25:23.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.800+0000 7f1532ffd640 1 --2- 172.21.15.154:0/2472400815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15340ada30 0x7f15340ccaf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1524009790 tx=0x7f152402b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9561f605cf389482 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:23.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.800+0000 7f15327fc640 1 -- 172.21.15.154:0/2472400815 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1524031800 con 0x7f15340ada30 2023-09-23T03:25:23.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.801+0000 7f15327fc640 1 -- 172.21.15.154:0/2472400815 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1524031960 con 0x7f15340ada30 2023-09-23T03:25:23.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.801+0000 7f15327fc640 1 -- 172.21.15.154:0/2472400815 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1524031cd0 con 0x7f15340ada30 2023-09-23T03:25:23.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.801+0000 7f1539539640 1 -- 172.21.15.154:0/2472400815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15340ada30 msgr2=0x7f15340ccaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:23.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.801+0000 7f1539539640 1 --2- 172.21.15.154:0/2472400815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15340ada30 0x7f15340ccaf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1524009790 tx=0x7f152402b8a0 comp rx=0 tx=0).stop 2023-09-23T03:25:23.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.801+0000 7f1539539640 1 -- 172.21.15.154:0/2472400815 shutdown_connections 2023-09-23T03:25:23.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.801+0000 7f1539539640 1 --2- 172.21.15.154:0/2472400815 >> v2:172.21.15.190:3300/0 conn(0x7f15340cf130 0x7f153415d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:23.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.801+0000 7f1539539640 1 --2- 172.21.15.154:0/2472400815 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15340ada30 0x7f15340ccaf0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:23.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.801+0000 7f1539539640 1 -- 172.21.15.154:0/2472400815 >> 172.21.15.154:0/2472400815 conn(0x7f153406a0f0 msgr2=0x7f153406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:23.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.801+0000 7f1539539640 1 -- 172.21.15.154:0/2472400815 shutdown_connections 2023-09-23T03:25:23.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.801+0000 7f1539539640 1 -- 172.21.15.154:0/2472400815 wait complete. 2023-09-23T03:25:23.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.803+0000 7f1539539640 1 Processor -- start 2023-09-23T03:25:23.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.803+0000 7f1539539640 1 -- start start 2023-09-23T03:25:23.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.804+0000 7f1539539640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15340ada30 0x7f1534156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:23.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.804+0000 7f1539539640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15340b2fe0 0x7f15340c4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:23.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.804+0000 7f1539539640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15340cf130 0x7f15340ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:23.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.804+0000 7f1539539640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15340c4ec0 con 0x7f15340b2fe0 2023-09-23T03:25:23.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.804+0000 7f1539539640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15340c46b0 con 0x7f15340cf130 2023-09-23T03:25:23.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.804+0000 7f1539539640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1534158390 con 0x7f15340ada30 2023-09-23T03:25:23.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.804+0000 7f152bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15340b2fe0 0x7f15340c4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:23.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.805+0000 7f152bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15340b2fe0 0x7f15340c4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:38374/0 (socket says 172.21.15.154:38374) 2023-09-23T03:25:23.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.805+0000 7f152bfff640 1 -- 172.21.15.154:0/263509259 learned_addr learned my addr 172.21.15.154:0/263509259 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:23.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.805+0000 7f15337fe640 1 --2- 172.21.15.154:0/263509259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15340cf130 0x7f15340ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:23.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.805+0000 7f1532ffd640 1 --2- 172.21.15.154:0/263509259 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15340ada30 0x7f1534156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:23.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.805+0000 7f152bfff640 1 -- 172.21.15.154:0/263509259 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15340ada30 msgr2=0x7f1534156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:23.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.805+0000 7f152bfff640 1 --2- 172.21.15.154:0/263509259 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15340ada30 0x7f1534156b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:23.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.805+0000 7f152bfff640 1 -- 172.21.15.154:0/263509259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15340cf130 msgr2=0x7f15340ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:23.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.805+0000 7f152bfff640 1 --2- 172.21.15.154:0/263509259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15340cf130 0x7f15340ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:23.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.805+0000 7f152bfff640 1 -- 172.21.15.154:0/263509259 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15240093a0 con 0x7f15340b2fe0 2023-09-23T03:25:23.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.805+0000 7f152bfff640 1 --2- 172.21.15.154:0/263509259 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15340b2fe0 0x7f15340c4170 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f151c00c730 tx=0x7f151c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:23.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.806+0000 7f152b7fe640 1 -- 172.21.15.154:0/263509259 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151c005ab0 con 0x7f15340b2fe0 2023-09-23T03:25:23.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.806+0000 7f152b7fe640 1 -- 172.21.15.154:0/263509259 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f151c005c10 con 0x7f15340b2fe0 2023-09-23T03:25:23.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.806+0000 7f152b7fe640 1 -- 172.21.15.154:0/263509259 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151c001d50 con 0x7f15340b2fe0 2023-09-23T03:25:23.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.806+0000 7f1539539640 1 -- 172.21.15.154:0/263509259 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15340c4900 con 0x7f15340b2fe0 2023-09-23T03:25:23.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.806+0000 7f1539539640 1 -- 172.21.15.154:0/263509259 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15341f3120 con 0x7f15340b2fe0 2023-09-23T03:25:23.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.807+0000 7f1539539640 1 -- 172.21.15.154:0/263509259 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15340b27f0 con 0x7f15340b2fe0 2023-09-23T03:25:23.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.808+0000 7f152b7fe640 1 -- 172.21.15.154:0/263509259 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f151c007070 con 0x7f15340b2fe0 2023-09-23T03:25:23.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.809+0000 7f152b7fe640 1 --2- 172.21.15.154:0/263509259 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f151406af30 0x7f151406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:23.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.809+0000 7f152b7fe640 1 -- 172.21.15.154:0/263509259 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f151c088e40 con 0x7f15340b2fe0 2023-09-23T03:25:23.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.809+0000 7f1532ffd640 1 --2- 172.21.15.154:0/263509259 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f151406af30 0x7f151406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:23.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.810+0000 7f1532ffd640 1 --2- 172.21.15.154:0/263509259 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f151406af30 0x7f151406d3f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f1524036000 tx=0x7f1524008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:23.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:23.814+0000 7f152b7fe640 1 -- 172.21.15.154:0/263509259 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f151c053b30 con 0x7f15340b2fe0 2023-09-23T03:25:24.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.005+0000 7f1539539640 1 -- 172.21.15.154:0/263509259 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f153406ceb0 con 0x7f15340b2fe0 2023-09-23T03:25:24.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.007+0000 7f152b7fe640 1 -- 172.21.15.154:0/263509259 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+18993 (secure 0 0 0) 0x7f153406ceb0 con 0x7f15340b2fe0 2023-09-23T03:25:24.009 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:25:24.009 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":92,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:24:46.339781+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:24:07.834123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-09-23T03:24:09.058217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data_extra","create_time":"2023-09-23T03:24:19.995906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6837/1588996498":"2023-09-24T03:24:45.398760+0000","172.21.15.154:6836/1588996498":"2023-09-24T03:24:45.398760+0000","172.21.15.154:6837/3812680297":"2023-09-24T03:24:38.360723+0000","172.21.15.154:6836/3812680297":"2023-09-24T03:24:38.360723+0000","172.21.15.154:6837/2213283923":"2023-09-24T03:24:36.268262+0000","172.21.15.154:6836/2213283923":"2023-09-24T03:24:36.268262+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:25:24.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.010+0000 7f1539539640 1 -- 172.21.15.154:0/263509259 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f151406af30 msgr2=0x7f151406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:24.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.010+0000 7f1539539640 1 --2- 172.21.15.154:0/263509259 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f151406af30 0x7f151406d3f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f1524036000 tx=0x7f1524008030 comp rx=0 tx=0).stop 2023-09-23T03:25:24.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.010+0000 7f1539539640 1 -- 172.21.15.154:0/263509259 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15340b2fe0 msgr2=0x7f15340c4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:24.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.010+0000 7f1539539640 1 --2- 172.21.15.154:0/263509259 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15340b2fe0 0x7f15340c4170 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f151c00c730 tx=0x7f151c00cbf0 comp rx=0 tx=0).stop 2023-09-23T03:25:24.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.010+0000 7f1539539640 1 -- 172.21.15.154:0/263509259 shutdown_connections 2023-09-23T03:25:24.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.010+0000 7f1539539640 1 --2- 172.21.15.154:0/263509259 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f151406af30 0x7f151406d3f0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:24.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.010+0000 7f1539539640 1 --2- 172.21.15.154:0/263509259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15340cf130 0x7f15340ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:24.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.010+0000 7f1539539640 1 --2- 172.21.15.154:0/263509259 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15340b2fe0 0x7f15340c4170 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:24.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.010+0000 7f1539539640 1 --2- 172.21.15.154:0/263509259 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15340ada30 0x7f1534156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:24.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.010+0000 7f1539539640 1 -- 172.21.15.154:0/263509259 >> 172.21.15.154:0/263509259 conn(0x7f153406a0f0 msgr2=0x7f15340adf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:24.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.010+0000 7f1539539640 1 -- 172.21.15.154:0/263509259 shutdown_connections 2023-09-23T03:25:24.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.010+0000 7f1539539640 1 -- 172.21.15.154:0/263509259 wait complete. 2023-09-23T03:25:24.030 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:24:07.834123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:24:09.058217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-09-23T03:24:19.995906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T03:25:24.030 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:25:24.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.252+0000 7fddfb483640 1 Processor -- start 2023-09-23T03:25:24.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.252+0000 7fddfb483640 1 -- start start 2023-09-23T03:25:24.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.254+0000 7fddfb483640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fddf406fdc0 0x7fddf4070200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:24.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.254+0000 7fddfb483640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddf4070740 con 0x7fddf415e130 2023-09-23T03:25:24.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.254+0000 7fddfb483640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddf4158540 con 0x7fddf4158160 2023-09-23T03:25:24.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.254+0000 7fddfb483640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddf40c9c50 con 0x7fddf406fdc0 2023-09-23T03:25:24.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.254+0000 7fddf9c80640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fddf415e130 0x7fddf40c9540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40930/0 (socket says 172.21.15.154:40930) 2023-09-23T03:25:24.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.254+0000 7fddf9c80640 1 -- 172.21.15.154:0/4137101976 learned_addr learned my addr 172.21.15.154:0/4137101976 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:25:24.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.255+0000 7fddfac82640 1 --2- 172.21.15.154:0/4137101976 >> v2:172.21.15.190:3300/0 conn(0x7fddf406fdc0 0x7fddf4070200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:24.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.255+0000 7fddf947f640 1 -- 172.21.15.154:0/4137101976 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 933650423 0 0) 0x7fddf4070740 con 0x7fddf415e130 2023-09-23T03:25:24.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.255+0000 7fddf947f640 1 -- 172.21.15.154:0/4137101976 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdde0003520 con 0x7fddf415e130 2023-09-23T03:25:24.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.255+0000 7fddf947f640 1 -- 172.21.15.154:0/4137101976 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3399941304 0 0) 0x7fdde0003520 con 0x7fddf415e130 2023-09-23T03:25:24.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.255+0000 7fddf947f640 1 -- 172.21.15.154:0/4137101976 >> v2:172.21.15.190:3300/0 conn(0x7fddf406fdc0 msgr2=0x7fddf4070200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:24.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.255+0000 7fddf947f640 1 --2- 172.21.15.154:0/4137101976 >> v2:172.21.15.190:3300/0 conn(0x7fddf406fdc0 0x7fddf4070200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:24.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.255+0000 7fddf947f640 1 -- 172.21.15.154:0/4137101976 >> v1:172.21.15.190:6789/0 conn(0x7fddf4158160 legacy=0x7fddf415ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:24.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.255+0000 7fddf947f640 1 -- 172.21.15.154:0/4137101976 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf415b3e0 con 0x7fddf415e130 2023-09-23T03:25:24.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.255+0000 7fddf947f640 1 -- 172.21.15.154:0/4137101976 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fdde4001670 con 0x7fddf415e130 2023-09-23T03:25:24.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.255+0000 7fddf947f640 1 -- 172.21.15.154:0/4137101976 >> v1:172.21.15.154:6789/0 conn(0x7fddf415e130 legacy=0x7fddf40c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:24.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.255+0000 7fddf947f640 1 --2- 172.21.15.154:0/4137101976 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdde0003970 0x7fdde0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:24.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.255+0000 7fddf947f640 1 --2- 172.21.15.154:0/4137101976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdde00047c0 0x7fdde0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:24.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.255+0000 7fddf947f640 1 --2- 172.21.15.154:0/4137101976 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdde0005180 0x7fdde000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:24.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.255+0000 7fddf947f640 1 -- 172.21.15.154:0/4137101976 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf40c9c50 con 0x7fdde00047c0 2023-09-23T03:25:24.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.255+0000 7fddf947f640 1 -- 172.21.15.154:0/4137101976 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde000c020 con 0x7fdde0003970 2023-09-23T03:25:24.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.255+0000 7fddf947f640 1 -- 172.21.15.154:0/4137101976 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdde000c1e0 con 0x7fdde0005180 2023-09-23T03:25:24.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.256+0000 7fddf9c80640 1 --2- 172.21.15.154:0/4137101976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdde00047c0 0x7fdde0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:24.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.256+0000 7fddfa481640 1 --2- 172.21.15.154:0/4137101976 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdde0003970 0x7fdde0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:24.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.256+0000 7fddf9c80640 1 -- 172.21.15.154:0/4137101976 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdde0005180 msgr2=0x7fdde000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:24.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.256+0000 7fddfac82640 1 --2- 172.21.15.154:0/4137101976 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdde0005180 0x7fdde000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:24.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.256+0000 7fddf9c80640 1 --2- 172.21.15.154:0/4137101976 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdde0005180 0x7fdde000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:24.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.256+0000 7fddf9c80640 1 -- 172.21.15.154:0/4137101976 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdde0003970 msgr2=0x7fdde0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:24.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.256+0000 7fddf9c80640 1 --2- 172.21.15.154:0/4137101976 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdde0003970 0x7fdde0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:24.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.256+0000 7fddf9c80640 1 -- 172.21.15.154:0/4137101976 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde0003520 con 0x7fdde00047c0 2023-09-23T03:25:24.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.256+0000 7fddfac82640 1 -- 172.21.15.154:0/4137101976 reap_dead start 2023-09-23T03:25:24.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.257+0000 7fddf9c80640 1 --2- 172.21.15.154:0/4137101976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdde00047c0 0x7fdde0004c40 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fdde400c210 tx=0x7fdde00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6c2369416143b06 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:24.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.257+0000 7fddf947f640 1 -- 172.21.15.154:0/4137101976 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde4005ad0 con 0x7fdde00047c0 2023-09-23T03:25:24.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.257+0000 7fddf947f640 1 -- 172.21.15.154:0/4137101976 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde4005c30 con 0x7fdde00047c0 2023-09-23T03:25:24.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.257+0000 7fddf947f640 1 -- 172.21.15.154:0/4137101976 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde402c9e0 con 0x7fdde00047c0 2023-09-23T03:25:24.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.258+0000 7fddfb483640 1 -- 172.21.15.154:0/4137101976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdde00047c0 msgr2=0x7fdde0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:24.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.258+0000 7fddfb483640 1 --2- 172.21.15.154:0/4137101976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdde00047c0 0x7fdde0004c40 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fdde400c210 tx=0x7fdde00057f0 comp rx=0 tx=0).stop 2023-09-23T03:25:24.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.258+0000 7fddfb483640 1 -- 172.21.15.154:0/4137101976 shutdown_connections 2023-09-23T03:25:24.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.258+0000 7fddfb483640 1 --2- 172.21.15.154:0/4137101976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdde00047c0 0x7fdde0004c40 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:24.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.258+0000 7fddfb483640 1 -- 172.21.15.154:0/4137101976 >> 172.21.15.154:0/4137101976 conn(0x7fddf406a300 msgr2=0x7fddf40afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:24.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.258+0000 7fddfb483640 1 -- 172.21.15.154:0/4137101976 shutdown_connections 2023-09-23T03:25:24.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.258+0000 7fddfb483640 1 -- 172.21.15.154:0/4137101976 wait complete. 2023-09-23T03:25:24.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.260+0000 7fddfb483640 1 Processor -- start 2023-09-23T03:25:24.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.260+0000 7fddfb483640 1 -- start start 2023-09-23T03:25:24.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.261+0000 7fddfb483640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdde00047c0 0x7fddf40c71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:24.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.261+0000 7fddfb483640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fddf40b0f00 0x7fddf40ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:24.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.261+0000 7fddfb483640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fddf4145210 0x7fddf41456c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:24.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.261+0000 7fddfb483640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf40b2460 con 0x7fdde00047c0 2023-09-23T03:25:24.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.261+0000 7fddfb483640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf40b1fd0 con 0x7fddf4145210 2023-09-23T03:25:24.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.261+0000 7fddfb483640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf4145c00 con 0x7fddf40b0f00 2023-09-23T03:25:24.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.261+0000 7fddfa481640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdde00047c0 0x7fddf40c71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:24.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.261+0000 7fddfa481640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdde00047c0 0x7fddf40c71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50568/0 (socket says 172.21.15.154:50568) 2023-09-23T03:25:24.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.261+0000 7fddfa481640 1 -- 172.21.15.154:0/418992942 learned_addr learned my addr 172.21.15.154:0/418992942 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:24.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.262+0000 7fddfa481640 1 -- 172.21.15.154:0/418992942 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fddf40b0f00 msgr2=0x7fddf40ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:24.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.262+0000 7fddfa481640 1 --2- 172.21.15.154:0/418992942 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fddf40b0f00 0x7fddf40ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:24.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.262+0000 7fddfa481640 1 -- 172.21.15.154:0/418992942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fddf4145210 msgr2=0x7fddf41456c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:24.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.262+0000 7fddfa481640 1 --2- 172.21.15.154:0/418992942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fddf4145210 0x7fddf41456c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:24.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.262+0000 7fddfa481640 1 -- 172.21.15.154:0/418992942 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf40b13c0 con 0x7fdde00047c0 2023-09-23T03:25:24.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.262+0000 7fddfa481640 1 --2- 172.21.15.154:0/418992942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdde00047c0 0x7fddf40c71d0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fddf000e4e0 tx=0x7fddf000e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:24.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.262+0000 7fddeaffd640 1 -- 172.21.15.154:0/418992942 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf00052e0 con 0x7fdde00047c0 2023-09-23T03:25:24.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.262+0000 7fddeaffd640 1 -- 172.21.15.154:0/418992942 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf0005440 con 0x7fdde00047c0 2023-09-23T03:25:24.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.262+0000 7fddeaffd640 1 -- 172.21.15.154:0/418992942 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf0003830 con 0x7fdde00047c0 2023-09-23T03:25:24.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.262+0000 7fddfb483640 1 -- 172.21.15.154:0/418992942 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddf40b16a0 con 0x7fdde00047c0 2023-09-23T03:25:24.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.263+0000 7fddfb483640 1 -- 172.21.15.154:0/418992942 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddf40b1c60 con 0x7fdde00047c0 2023-09-23T03:25:24.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.264+0000 7fddfb483640 1 -- 172.21.15.154:0/418992942 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddf40736b0 con 0x7fdde00047c0 2023-09-23T03:25:24.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.265+0000 7fddeaffd640 1 -- 172.21.15.154:0/418992942 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fddf0003990 con 0x7fdde00047c0 2023-09-23T03:25:24.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.265+0000 7fddeaffd640 1 --2- 172.21.15.154:0/418992942 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fddd806af60 0x7fddd806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:24.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.266+0000 7fddeaffd640 1 -- 172.21.15.154:0/418992942 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fddf0089830 con 0x7fdde00047c0 2023-09-23T03:25:24.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.266+0000 7fddf9c80640 1 --2- 172.21.15.154:0/418992942 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fddd806af60 0x7fddd806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:24.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.267+0000 7fddf9c80640 1 --2- 172.21.15.154:0/418992942 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fddd806af60 0x7fddd806d420 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fdde4034000 tx=0x7fdde4039040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:24.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.270+0000 7fddeaffd640 1 -- 172.21.15.154:0/418992942 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fddf0054550 con 0x7fdde00047c0 2023-09-23T03:25:24.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.464+0000 7fddfb483640 1 -- 172.21.15.154:0/418992942 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fddf4067390 con 0x7fdde00047c0 2023-09-23T03:25:24.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.465+0000 7fddeaffd640 1 -- 172.21.15.154:0/418992942 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7fddf0057b70 con 0x7fdde00047c0 2023-09-23T03:25:24.467 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:25:24.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.468+0000 7fddfb483640 1 -- 172.21.15.154:0/418992942 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fddd806af60 msgr2=0x7fddd806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:24.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.468+0000 7fddfb483640 1 --2- 172.21.15.154:0/418992942 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fddd806af60 0x7fddd806d420 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fdde4034000 tx=0x7fdde4039040 comp rx=0 tx=0).stop 2023-09-23T03:25:24.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.468+0000 7fddfb483640 1 -- 172.21.15.154:0/418992942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdde00047c0 msgr2=0x7fddf40c71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:24.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.468+0000 7fddfb483640 1 --2- 172.21.15.154:0/418992942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdde00047c0 0x7fddf40c71d0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fddf000e4e0 tx=0x7fddf000e9a0 comp rx=0 tx=0).stop 2023-09-23T03:25:24.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.468+0000 7fddfb483640 1 -- 172.21.15.154:0/418992942 shutdown_connections 2023-09-23T03:25:24.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.468+0000 7fddfb483640 1 --2- 172.21.15.154:0/418992942 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fddd806af60 0x7fddd806d420 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:24.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.468+0000 7fddfb483640 1 --2- 172.21.15.154:0/418992942 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fddf4145210 0x7fddf41456c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:24.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.468+0000 7fddfb483640 1 --2- 172.21.15.154:0/418992942 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fddf40b0f00 0x7fddf40ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:24.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.468+0000 7fddfb483640 1 --2- 172.21.15.154:0/418992942 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdde00047c0 0x7fddf40c71d0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:24.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.468+0000 7fddfb483640 1 -- 172.21.15.154:0/418992942 >> 172.21.15.154:0/418992942 conn(0x7fddf406a300 msgr2=0x7fddf4159d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:24.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.469+0000 7fddfb483640 1 -- 172.21.15.154:0/418992942 shutdown_connections 2023-09-23T03:25:24.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.469+0000 7fddfb483640 1 -- 172.21.15.154:0/418992942 wait complete. 2023-09-23T03:25:24.490 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:25:24.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.704+0000 7f3620b8a640 1 Processor -- start 2023-09-23T03:25:24.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.704+0000 7f3620b8a640 1 -- start start 2023-09-23T03:25:24.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.705+0000 7f3620b8a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f361c15bb30 0x7f361c0caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:24.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.705+0000 7f3620b8a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f361c0cb3e0 0x7f361c0cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:24.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.705+0000 7f3620b8a640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f361c15b6a0 con 0x7f361c15aba0 2023-09-23T03:25:24.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.705+0000 7f3620b8a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f361c0cb870 con 0x7f361c15bb30 2023-09-23T03:25:24.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.705+0000 7f3620b8a640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f361c15bf70 con 0x7f361c0cb3e0 2023-09-23T03:25:24.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.706+0000 7f361a59c640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f361c15aba0 0x7f361c15e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40932/0 (socket says 172.21.15.154:40932) 2023-09-23T03:25:24.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.706+0000 7f361a59c640 1 -- 172.21.15.154:0/4183434509 learned_addr learned my addr 172.21.15.154:0/4183434509 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:25:24.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.706+0000 7f3619d9b640 1 --2- 172.21.15.154:0/4183434509 >> v2:172.21.15.154:3300/0 conn(0x7f361c15bb30 0x7f361c0caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:24.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.706+0000 7f3619d9b640 1 -- 172.21.15.154:0/4183434509 >> v2:172.21.15.190:3300/0 conn(0x7f361c0cb3e0 msgr2=0x7f361c0cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:24.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.706+0000 7f361ad9d640 1 --2- 172.21.15.154:0/4183434509 >> v2:172.21.15.190:3300/0 conn(0x7f361c0cb3e0 0x7f361c0cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:24.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.706+0000 7f3619d9b640 1 --2- 172.21.15.154:0/4183434509 >> v2:172.21.15.190:3300/0 conn(0x7f361c0cb3e0 0x7f361c0cc5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:24.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.706+0000 7f3619d9b640 1 -- 172.21.15.154:0/4183434509 >> v1:172.21.15.154:6789/0 conn(0x7f361c15aba0 legacy=0x7f361c15e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:24.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.706+0000 7f361959a640 1 -- 172.21.15.154:0/4183434509 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 330432474 0 0) 0x7f361c15b6a0 con 0x7f361c15aba0 2023-09-23T03:25:24.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.706+0000 7f3619d9b640 1 -- 172.21.15.154:0/4183434509 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361c15f0b0 con 0x7f361c15bb30 2023-09-23T03:25:24.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.707+0000 7f3619d9b640 1 --2- 172.21.15.154:0/4183434509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f361c15bb30 0x7f361c0caea0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f361c0c1f80 tx=0x7f361002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b28949a75c62c53e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:24.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.708+0000 7f361959a640 1 -- 172.21.15.154:0/4183434509 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3610031830 con 0x7f361c15bb30 2023-09-23T03:25:24.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.708+0000 7f361959a640 1 -- 172.21.15.154:0/4183434509 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3610031990 con 0x7f361c15bb30 2023-09-23T03:25:24.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.708+0000 7f361959a640 1 -- 172.21.15.154:0/4183434509 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3610031d20 con 0x7f361c15bb30 2023-09-23T03:25:24.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.708+0000 7f3620b8a640 1 -- 172.21.15.154:0/4183434509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f361c15bb30 msgr2=0x7f361c0caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:24.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.708+0000 7f3620b8a640 1 --2- 172.21.15.154:0/4183434509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f361c15bb30 0x7f361c0caea0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f361c0c1f80 tx=0x7f361002b8d0 comp rx=0 tx=0).stop 2023-09-23T03:25:24.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.708+0000 7f3620b8a640 1 -- 172.21.15.154:0/4183434509 shutdown_connections 2023-09-23T03:25:24.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.708+0000 7f3620b8a640 1 --2- 172.21.15.154:0/4183434509 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f361c15bb30 0x7f361c0caea0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:24.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.708+0000 7f3620b8a640 1 --2- 172.21.15.154:0/4183434509 >> v2:172.21.15.190:3300/0 conn(0x7f361c0cb3e0 0x7f361c0cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:24.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.708+0000 7f3620b8a640 1 -- 172.21.15.154:0/4183434509 >> 172.21.15.154:0/4183434509 conn(0x7f361c06c010 msgr2=0x7f361c06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:24.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.708+0000 7f3620b8a640 1 -- 172.21.15.154:0/4183434509 shutdown_connections 2023-09-23T03:25:24.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.708+0000 7f3620b8a640 1 -- 172.21.15.154:0/4183434509 wait complete. 2023-09-23T03:25:24.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.710+0000 7f3620b8a640 1 Processor -- start 2023-09-23T03:25:24.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.710+0000 7f3620b8a640 1 -- start start 2023-09-23T03:25:24.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.711+0000 7f3620b8a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f361c0cb3e0 0x7f361c156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:24.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.711+0000 7f3620b8a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f361c15aba0 0x7f361c153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:24.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.711+0000 7f3620b8a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f361c15bb30 0x7f361c153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:24.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.711+0000 7f3620b8a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361c1f41e0 con 0x7f361c15bb30 2023-09-23T03:25:24.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.711+0000 7f3620b8a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361c1542b0 con 0x7f361c15aba0 2023-09-23T03:25:24.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.711+0000 7f3620b8a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f361c1f38b0 con 0x7f361c0cb3e0 2023-09-23T03:25:24.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.712+0000 7f361ad9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f361c15bb30 0x7f361c153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:24.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.712+0000 7f361ad9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f361c15bb30 0x7f361c153d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50586/0 (socket says 172.21.15.154:50586) 2023-09-23T03:25:24.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.712+0000 7f361ad9d640 1 -- 172.21.15.154:0/2216942607 learned_addr learned my addr 172.21.15.154:0/2216942607 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:24.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.712+0000 7f361a59c640 1 --2- 172.21.15.154:0/2216942607 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f361c0cb3e0 0x7f361c156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:24.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.712+0000 7f3619d9b640 1 --2- 172.21.15.154:0/2216942607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f361c15aba0 0x7f361c153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:24.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.712+0000 7f361ad9d640 1 -- 172.21.15.154:0/2216942607 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f361c0cb3e0 msgr2=0x7f361c156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:24.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.712+0000 7f361ad9d640 1 --2- 172.21.15.154:0/2216942607 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f361c0cb3e0 0x7f361c156720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:24.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.712+0000 7f361ad9d640 1 -- 172.21.15.154:0/2216942607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f361c15aba0 msgr2=0x7f361c153830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:24.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.712+0000 7f361ad9d640 1 --2- 172.21.15.154:0/2216942607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f361c15aba0 0x7f361c153830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:24.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.712+0000 7f361ad9d640 1 -- 172.21.15.154:0/2216942607 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3610009400 con 0x7f361c15bb30 2023-09-23T03:25:24.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.712+0000 7f3619d9b640 1 --2- 172.21.15.154:0/2216942607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f361c15aba0 0x7f361c153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:24.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.712+0000 7f361ad9d640 1 --2- 172.21.15.154:0/2216942607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f361c15bb30 0x7f361c153d70 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f361400cc30 tx=0x7f361400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:24.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.713+0000 7f360affd640 1 -- 172.21.15.154:0/2216942607 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361400ce40 con 0x7f361c15bb30 2023-09-23T03:25:24.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.713+0000 7f360affd640 1 -- 172.21.15.154:0/2216942607 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3614003990 con 0x7f361c15bb30 2023-09-23T03:25:24.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.713+0000 7f3620b8a640 1 -- 172.21.15.154:0/2216942607 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f361c1f3b90 con 0x7f361c15bb30 2023-09-23T03:25:24.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.713+0000 7f360affd640 1 -- 172.21.15.154:0/2216942607 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3614003d20 con 0x7f361c15bb30 2023-09-23T03:25:24.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.713+0000 7f3620b8a640 1 -- 172.21.15.154:0/2216942607 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f361c1f35b0 con 0x7f361c15bb30 2023-09-23T03:25:24.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.714+0000 7f3620b8a640 1 -- 172.21.15.154:0/2216942607 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f361c1f4350 con 0x7f361c15bb30 2023-09-23T03:25:24.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.715+0000 7f360affd640 1 -- 172.21.15.154:0/2216942607 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f3614003e80 con 0x7f361c15bb30 2023-09-23T03:25:24.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.716+0000 7f360affd640 1 --2- 172.21.15.154:0/2216942607 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f35fc06af30 0x7f35fc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:24.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.717+0000 7f360affd640 1 -- 172.21.15.154:0/2216942607 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f3614089f30 con 0x7f361c15bb30 2023-09-23T03:25:24.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.717+0000 7f361a59c640 1 --2- 172.21.15.154:0/2216942607 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f35fc06af30 0x7f35fc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:24.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.718+0000 7f361a59c640 1 --2- 172.21.15.154:0/2216942607 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f35fc06af30 0x7f35fc06d3f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f360c000f60 tx=0x7f360c007b50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:24.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.721+0000 7f360affd640 1 -- 172.21.15.154:0/2216942607 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f3614054b80 con 0x7f361c15bb30 2023-09-23T03:25:24.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.908+0000 7f3620b8a640 1 -- 172.21.15.154:0/2216942607 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f361c04e920 con 0x7f361c15bb30 2023-09-23T03:25:24.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.909+0000 7f360affd640 1 -- 172.21.15.154:0/2216942607 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f361401d020 con 0x7f361c15bb30 2023-09-23T03:25:24.910 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:25:24.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.912+0000 7f3620b8a640 1 -- 172.21.15.154:0/2216942607 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f35fc06af30 msgr2=0x7f35fc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:24.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.912+0000 7f3620b8a640 1 --2- 172.21.15.154:0/2216942607 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f35fc06af30 0x7f35fc06d3f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f360c000f60 tx=0x7f360c007b50 comp rx=0 tx=0).stop 2023-09-23T03:25:24.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.912+0000 7f3620b8a640 1 -- 172.21.15.154:0/2216942607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f361c15bb30 msgr2=0x7f361c153d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:24.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.912+0000 7f3620b8a640 1 --2- 172.21.15.154:0/2216942607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f361c15bb30 0x7f361c153d70 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f361400cc30 tx=0x7f361400a5c0 comp rx=0 tx=0).stop 2023-09-23T03:25:24.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.912+0000 7f3620b8a640 1 -- 172.21.15.154:0/2216942607 shutdown_connections 2023-09-23T03:25:24.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.912+0000 7f3620b8a640 1 --2- 172.21.15.154:0/2216942607 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f35fc06af30 0x7f35fc06d3f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:24.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.912+0000 7f3620b8a640 1 --2- 172.21.15.154:0/2216942607 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f361c15bb30 0x7f361c153d70 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:24.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.912+0000 7f3620b8a640 1 --2- 172.21.15.154:0/2216942607 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f361c15aba0 0x7f361c153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:24.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.912+0000 7f3620b8a640 1 --2- 172.21.15.154:0/2216942607 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f361c0cb3e0 0x7f361c156720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:24.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.912+0000 7f3620b8a640 1 -- 172.21.15.154:0/2216942607 >> 172.21.15.154:0/2216942607 conn(0x7f361c06c010 msgr2=0x7f361c0701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:24.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.912+0000 7f3620b8a640 1 -- 172.21.15.154:0/2216942607 shutdown_connections 2023-09-23T03:25:24.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:24.912+0000 7f3620b8a640 1 -- 172.21.15.154:0/2216942607 wait complete. 2023-09-23T03:25:24.931 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:25:25.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.148+0000 7f92b8de7640 1 Processor -- start 2023-09-23T03:25:25.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.149+0000 7f92b8de7640 1 -- start start 2023-09-23T03:25:25.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.150+0000 7f92b8de7640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f92b4071290 0x7f92b415cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:25.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.150+0000 7f92b8de7640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92b40c9d20 con 0x7f92b40c9150 2023-09-23T03:25:25.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.150+0000 7f92b8de7640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92b4071850 con 0x7f92b40c8d70 2023-09-23T03:25:25.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.150+0000 7f92b8de7640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92b4071b30 con 0x7f92b4071290 2023-09-23T03:25:25.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.150+0000 7f92b1d9b640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f92b40c9150 0x7f92b40ca1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40948/0 (socket says 172.21.15.154:40948) 2023-09-23T03:25:25.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.150+0000 7f92b1d9b640 1 -- 172.21.15.154:0/1232787137 learned_addr learned my addr 172.21.15.154:0/1232787137 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:25:25.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.151+0000 7f92b2d9d640 1 --2- 172.21.15.154:0/1232787137 >> v2:172.21.15.190:3300/0 conn(0x7f92b4071290 0x7f92b415cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:25.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.151+0000 7f92b159a640 1 -- 172.21.15.154:0/1232787137 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1364634374 0 0) 0x7f92b40c9d20 con 0x7f92b40c9150 2023-09-23T03:25:25.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.151+0000 7f92b159a640 1 -- 172.21.15.154:0/1232787137 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92a0003520 con 0x7f92b40c9150 2023-09-23T03:25:25.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.151+0000 7f92b159a640 1 -- 172.21.15.154:0/1232787137 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2672646880 0 0) 0x7f92a0003520 con 0x7f92b40c9150 2023-09-23T03:25:25.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.151+0000 7f92b159a640 1 -- 172.21.15.154:0/1232787137 >> v2:172.21.15.190:3300/0 conn(0x7f92b4071290 msgr2=0x7f92b415cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:25.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.151+0000 7f92b159a640 1 --2- 172.21.15.154:0/1232787137 >> v2:172.21.15.190:3300/0 conn(0x7f92b4071290 0x7f92b415cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:25.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.151+0000 7f92b159a640 1 -- 172.21.15.154:0/1232787137 >> v1:172.21.15.190:6789/0 conn(0x7f92b40c8d70 legacy=0x7f92b415c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:25.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.151+0000 7f92b159a640 1 -- 172.21.15.154:0/1232787137 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b4071e10 con 0x7f92b40c9150 2023-09-23T03:25:25.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.151+0000 7f92b159a640 1 -- 172.21.15.154:0/1232787137 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f92a40019f0 con 0x7f92b40c9150 2023-09-23T03:25:25.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.151+0000 7f92b159a640 1 -- 172.21.15.154:0/1232787137 >> v1:172.21.15.154:6789/0 conn(0x7f92b40c9150 legacy=0x7f92b40ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:25.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.151+0000 7f92b159a640 1 --2- 172.21.15.154:0/1232787137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92a0003970 0x7f92a0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:25.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.151+0000 7f92b159a640 1 --2- 172.21.15.154:0/1232787137 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92a00047c0 0x7f92a0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:25.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.151+0000 7f92b159a640 1 --2- 172.21.15.154:0/1232787137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92a0005180 0x7f92a000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:25.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.151+0000 7f92b159a640 1 -- 172.21.15.154:0/1232787137 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b4071b30 con 0x7f92a00047c0 2023-09-23T03:25:25.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.151+0000 7f92b159a640 1 -- 172.21.15.154:0/1232787137 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a000c020 con 0x7f92a0003970 2023-09-23T03:25:25.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.151+0000 7f92b159a640 1 -- 172.21.15.154:0/1232787137 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92a000c1e0 con 0x7f92a0005180 2023-09-23T03:25:25.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.152+0000 7f92b1d9b640 1 --2- 172.21.15.154:0/1232787137 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92a00047c0 0x7f92a0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:25.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.152+0000 7f92b1d9b640 1 -- 172.21.15.154:0/1232787137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92a0005180 msgr2=0x7f92a000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:25.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.152+0000 7f92b1d9b640 1 --2- 172.21.15.154:0/1232787137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92a0005180 0x7f92a000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:25.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.152+0000 7f92b1d9b640 1 -- 172.21.15.154:0/1232787137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92a0003970 msgr2=0x7f92a0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:25.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.152+0000 7f92b259c640 1 --2- 172.21.15.154:0/1232787137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92a0003970 0x7f92a0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:25.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.152+0000 7f92b1d9b640 1 --2- 172.21.15.154:0/1232787137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92a0003970 0x7f92a0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:25.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.152+0000 7f92b1d9b640 1 -- 172.21.15.154:0/1232787137 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a0003520 con 0x7f92a00047c0 2023-09-23T03:25:25.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.152+0000 7f92b2d9d640 1 -- 172.21.15.154:0/1232787137 reap_dead start 2023-09-23T03:25:25.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.152+0000 7f92b259c640 1 --2- 172.21.15.154:0/1232787137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92a0003970 0x7f92a0003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:25.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.153+0000 7f92b1d9b640 1 --2- 172.21.15.154:0/1232787137 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92a00047c0 0x7f92a0004c40 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f92a400a830 tx=0x7f92a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60acf0c471339010 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:25.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.154+0000 7f92b159a640 1 -- 172.21.15.154:0/1232787137 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a40072d0 con 0x7f92a00047c0 2023-09-23T03:25:25.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.154+0000 7f92b159a640 1 -- 172.21.15.154:0/1232787137 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a402f070 con 0x7f92a00047c0 2023-09-23T03:25:25.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.154+0000 7f92b159a640 1 -- 172.21.15.154:0/1232787137 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a4007690 con 0x7f92a00047c0 2023-09-23T03:25:25.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.154+0000 7f92b8de7640 1 -- 172.21.15.154:0/1232787137 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92a00047c0 msgr2=0x7f92a0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:25.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.154+0000 7f92b8de7640 1 --2- 172.21.15.154:0/1232787137 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92a00047c0 0x7f92a0004c40 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f92a400a830 tx=0x7f92a00057f0 comp rx=0 tx=0).stop 2023-09-23T03:25:25.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.154+0000 7f92b8de7640 1 -- 172.21.15.154:0/1232787137 shutdown_connections 2023-09-23T03:25:25.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.154+0000 7f92b8de7640 1 --2- 172.21.15.154:0/1232787137 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92a00047c0 0x7f92a0004c40 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:25.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.154+0000 7f92b8de7640 1 -- 172.21.15.154:0/1232787137 >> 172.21.15.154:0/1232787137 conn(0x7f92b406c010 msgr2=0x7f92b406c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:25.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.154+0000 7f92b8de7640 1 -- 172.21.15.154:0/1232787137 shutdown_connections 2023-09-23T03:25:25.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.154+0000 7f92b8de7640 1 -- 172.21.15.154:0/1232787137 wait complete. 2023-09-23T03:25:25.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.156+0000 7f92b8de7640 1 Processor -- start 2023-09-23T03:25:25.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.156+0000 7f92b8de7640 1 -- start start 2023-09-23T03:25:25.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.157+0000 7f92b8de7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92a00047c0 0x7f92b40c1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:25.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.157+0000 7f92b8de7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92b415b6e0 0x7f92b415bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:25.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.157+0000 7f92b8de7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92b40c25a0 0x7f92b40c2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:25.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.157+0000 7f92b8de7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b415c080 con 0x7f92a00047c0 2023-09-23T03:25:25.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.157+0000 7f92b8de7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b4145fe0 con 0x7f92b415b6e0 2023-09-23T03:25:25.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.157+0000 7f92b8de7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92b40c17b0 con 0x7f92b40c25a0 2023-09-23T03:25:25.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.157+0000 7f92b259c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92a00047c0 0x7f92b40c1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:25.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.157+0000 7f92b259c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92a00047c0 0x7f92b40c1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50606/0 (socket says 172.21.15.154:50606) 2023-09-23T03:25:25.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.157+0000 7f92b259c640 1 -- 172.21.15.154:0/2828548714 learned_addr learned my addr 172.21.15.154:0/2828548714 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:25.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.158+0000 7f92b2d9d640 1 --2- 172.21.15.154:0/2828548714 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92b40c25a0 0x7f92b40c2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:25.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.158+0000 7f92b259c640 1 -- 172.21.15.154:0/2828548714 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92b40c25a0 msgr2=0x7f92b40c2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:25.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.158+0000 7f92b1d9b640 1 --2- 172.21.15.154:0/2828548714 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92b415b6e0 0x7f92b415bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:25.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.158+0000 7f92b259c640 1 --2- 172.21.15.154:0/2828548714 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92b40c25a0 0x7f92b40c2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:25.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.158+0000 7f92b259c640 1 -- 172.21.15.154:0/2828548714 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92b415b6e0 msgr2=0x7f92b415bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:25.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.158+0000 7f92b259c640 1 --2- 172.21.15.154:0/2828548714 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92b415b6e0 0x7f92b415bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:25.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.158+0000 7f92b259c640 1 -- 172.21.15.154:0/2828548714 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b4145740 con 0x7f92a00047c0 2023-09-23T03:25:25.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.158+0000 7f92b259c640 1 --2- 172.21.15.154:0/2828548714 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92a00047c0 0x7f92b40c1270 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f929c00d4a0 tx=0x7f929c00d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:25.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.158+0000 7f92aaffd640 1 -- 172.21.15.154:0/2828548714 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929c005480 con 0x7f92a00047c0 2023-09-23T03:25:25.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.159+0000 7f92aaffd640 1 -- 172.21.15.154:0/2828548714 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f929c007030 con 0x7f92a00047c0 2023-09-23T03:25:25.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.159+0000 7f92aaffd640 1 -- 172.21.15.154:0/2828548714 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929c0073c0 con 0x7f92a00047c0 2023-09-23T03:25:25.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.159+0000 7f92b8de7640 1 -- 172.21.15.154:0/2828548714 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92b4145a20 con 0x7f92a00047c0 2023-09-23T03:25:25.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.159+0000 7f92b8de7640 1 -- 172.21.15.154:0/2828548714 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92b40c2080 con 0x7f92a00047c0 2023-09-23T03:25:25.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.160+0000 7f92b8de7640 1 -- 172.21.15.154:0/2828548714 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92b404e920 con 0x7f92a00047c0 2023-09-23T03:25:25.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.161+0000 7f92aaffd640 1 -- 172.21.15.154:0/2828548714 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f929c007520 con 0x7f92a00047c0 2023-09-23T03:25:25.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.162+0000 7f92aaffd640 1 --2- 172.21.15.154:0/2828548714 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f929406af30 0x7f929406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:25.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.162+0000 7f92aaffd640 1 -- 172.21.15.154:0/2828548714 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f929c088ff0 con 0x7f92a00047c0 2023-09-23T03:25:25.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.162+0000 7f92b1d9b640 1 --2- 172.21.15.154:0/2828548714 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f929406af30 0x7f929406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:25.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.163+0000 7f92b1d9b640 1 --2- 172.21.15.154:0/2828548714 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f929406af30 0x7f929406d3f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f92a4034000 tx=0x7f92a4007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:25.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.166+0000 7f92aaffd640 1 -- 172.21.15.154:0/2828548714 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f929c0bba50 con 0x7f92a00047c0 2023-09-23T03:25:25.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.357+0000 7f92b8de7640 1 -- 172.21.15.154:0/2828548714 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f92b406cbc0 con 0x7f92a00047c0 2023-09-23T03:25:25.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.358+0000 7f92aaffd640 1 -- 172.21.15.154:0/2828548714 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f929c019020 con 0x7f92a00047c0 2023-09-23T03:25:25.359 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:25:25.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.361+0000 7f92b8de7640 1 -- 172.21.15.154:0/2828548714 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f929406af30 msgr2=0x7f929406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:25.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.361+0000 7f92b8de7640 1 --2- 172.21.15.154:0/2828548714 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f929406af30 0x7f929406d3f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f92a4034000 tx=0x7f92a4007000 comp rx=0 tx=0).stop 2023-09-23T03:25:25.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.361+0000 7f92b8de7640 1 -- 172.21.15.154:0/2828548714 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92a00047c0 msgr2=0x7f92b40c1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:25.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.361+0000 7f92b8de7640 1 --2- 172.21.15.154:0/2828548714 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92a00047c0 0x7f92b40c1270 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f929c00d4a0 tx=0x7f929c00d960 comp rx=0 tx=0).stop 2023-09-23T03:25:25.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.361+0000 7f92b8de7640 1 -- 172.21.15.154:0/2828548714 shutdown_connections 2023-09-23T03:25:25.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.361+0000 7f92b8de7640 1 --2- 172.21.15.154:0/2828548714 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f929406af30 0x7f929406d3f0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:25.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.361+0000 7f92b8de7640 1 --2- 172.21.15.154:0/2828548714 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f92b40c25a0 0x7f92b40c2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:25.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.361+0000 7f92b8de7640 1 --2- 172.21.15.154:0/2828548714 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f92b415b6e0 0x7f92b415bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:25.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.361+0000 7f92b8de7640 1 --2- 172.21.15.154:0/2828548714 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f92a00047c0 0x7f92b40c1270 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:25.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.361+0000 7f92b8de7640 1 -- 172.21.15.154:0/2828548714 >> 172.21.15.154:0/2828548714 conn(0x7f92b406c010 msgr2=0x7f92b414a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:25.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.361+0000 7f92b8de7640 1 -- 172.21.15.154:0/2828548714 shutdown_connections 2023-09-23T03:25:25.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.361+0000 7f92b8de7640 1 -- 172.21.15.154:0/2828548714 wait complete. 2023-09-23T03:25:25.381 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-09-23T03:25:25.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.593+0000 7efd1eed7640 1 Processor -- start 2023-09-23T03:25:25.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.594+0000 7efd1eed7640 1 -- start start 2023-09-23T03:25:25.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.595+0000 7efd1eed7640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efd1815ba50 0x7efd1806fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:25.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.595+0000 7efd1eed7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd1815eef0 0x7efd180703d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:25.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.595+0000 7efd1eed7640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd180cb6c0 con 0x7efd18158480 2023-09-23T03:25:25.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.595+0000 7efd1eed7640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd1815ad80 con 0x7efd1815ba50 2023-09-23T03:25:25.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.595+0000 7efd1eed7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd1815af70 con 0x7efd1815eef0 2023-09-23T03:25:25.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.596+0000 7efd1d6d4640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efd1815ba50 0x7efd1806fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:25.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.596+0000 7efd1ded5640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7efd18158480 0x7efd180cade0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40950/0 (socket says 172.21.15.154:40950) 2023-09-23T03:25:25.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.596+0000 7efd1ded5640 1 -- 172.21.15.154:0/263121687 learned_addr learned my addr 172.21.15.154:0/263121687 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:25:25.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.596+0000 7efd1e6d6640 1 --2- 172.21.15.154:0/263121687 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd1815eef0 0x7efd180703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:25.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.596+0000 7efd1d6d4640 1 -- 172.21.15.154:0/263121687 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd1815eef0 msgr2=0x7efd180703d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:25.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.596+0000 7efd1d6d4640 1 --2- 172.21.15.154:0/263121687 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd1815eef0 0x7efd180703d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:25.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.596+0000 7efd1d6d4640 1 -- 172.21.15.154:0/263121687 >> v1:172.21.15.154:6789/0 conn(0x7efd18158480 legacy=0x7efd180cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:25.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.596+0000 7efd1d6d4640 1 -- 172.21.15.154:0/263121687 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd1815b250 con 0x7efd1815ba50 2023-09-23T03:25:25.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.596+0000 7efd1ced3640 1 -- 172.21.15.154:0/263121687 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2212582188 0 0) 0x7efd180cb6c0 con 0x7efd18158480 2023-09-23T03:25:25.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.597+0000 7efd1d6d4640 1 --2- 172.21.15.154:0/263121687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd1815ba50 0x7efd1806fe90 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7efd080097a0 tx=0x7efd0802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3630a95090b5dda7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:25.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.598+0000 7efd1ced3640 1 -- 172.21.15.154:0/263121687 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0802d040 con 0x7efd1815ba50 2023-09-23T03:25:25.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.598+0000 7efd1ced3640 1 -- 172.21.15.154:0/263121687 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd08031810 con 0x7efd1815ba50 2023-09-23T03:25:25.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.598+0000 7efd1ced3640 1 -- 172.21.15.154:0/263121687 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd08031ba0 con 0x7efd1815ba50 2023-09-23T03:25:25.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.598+0000 7efd1eed7640 1 -- 172.21.15.154:0/263121687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd1815ba50 msgr2=0x7efd1806fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:25.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.598+0000 7efd1eed7640 1 --2- 172.21.15.154:0/263121687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd1815ba50 0x7efd1806fe90 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7efd080097a0 tx=0x7efd0802b8b0 comp rx=0 tx=0).stop 2023-09-23T03:25:25.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.598+0000 7efd1eed7640 1 -- 172.21.15.154:0/263121687 shutdown_connections 2023-09-23T03:25:25.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.598+0000 7efd1eed7640 1 --2- 172.21.15.154:0/263121687 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd1815eef0 0x7efd180703d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:25.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.598+0000 7efd1eed7640 1 --2- 172.21.15.154:0/263121687 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd1815ba50 0x7efd1806fe90 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:25.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.598+0000 7efd1eed7640 1 -- 172.21.15.154:0/263121687 >> 172.21.15.154:0/263121687 conn(0x7efd180aff20 msgr2=0x7efd180b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:25.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.598+0000 7efd1eed7640 1 -- 172.21.15.154:0/263121687 shutdown_connections 2023-09-23T03:25:25.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.599+0000 7efd1eed7640 1 -- 172.21.15.154:0/263121687 wait complete. 2023-09-23T03:25:25.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.600+0000 7efd1eed7640 1 Processor -- start 2023-09-23T03:25:25.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.600+0000 7efd1eed7640 1 -- start start 2023-09-23T03:25:25.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.601+0000 7efd1eed7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd18158480 0x7efd181f1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:25.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.601+0000 7efd1eed7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd1815ba50 0x7efd180c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:25.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.601+0000 7efd1eed7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd1815eef0 0x7efd180c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:25.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.601+0000 7efd1eed7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1815d190 con 0x7efd1815eef0 2023-09-23T03:25:25.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.601+0000 7efd1eed7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd180abe40 con 0x7efd1815ba50 2023-09-23T03:25:25.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.601+0000 7efd1eed7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd1815cbd0 con 0x7efd18158480 2023-09-23T03:25:25.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.602+0000 7efd1e6d6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd1815eef0 0x7efd180c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:25.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.602+0000 7efd1e6d6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd1815eef0 0x7efd180c8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50622/0 (socket says 172.21.15.154:50622) 2023-09-23T03:25:25.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.602+0000 7efd1e6d6640 1 -- 172.21.15.154:0/2621401089 learned_addr learned my addr 172.21.15.154:0/2621401089 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:25.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.602+0000 7efd1d6d4640 1 --2- 172.21.15.154:0/2621401089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd1815ba50 0x7efd180c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:25.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.602+0000 7efd1ded5640 1 --2- 172.21.15.154:0/2621401089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd18158480 0x7efd181f1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:25.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.602+0000 7efd1e6d6640 1 -- 172.21.15.154:0/2621401089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd18158480 msgr2=0x7efd181f1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:25.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.602+0000 7efd1e6d6640 1 --2- 172.21.15.154:0/2621401089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd18158480 0x7efd181f1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:25.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.602+0000 7efd1e6d6640 1 -- 172.21.15.154:0/2621401089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd1815ba50 msgr2=0x7efd180c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:25.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.602+0000 7efd1e6d6640 1 --2- 172.21.15.154:0/2621401089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd1815ba50 0x7efd180c8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:25.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.602+0000 7efd1e6d6640 1 -- 172.21.15.154:0/2621401089 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd08009400 con 0x7efd1815eef0 2023-09-23T03:25:25.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.602+0000 7efd1d6d4640 1 --2- 172.21.15.154:0/2621401089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd1815ba50 0x7efd180c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:25.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.603+0000 7efd1e6d6640 1 --2- 172.21.15.154:0/2621401089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd1815eef0 0x7efd180c8f60 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7efd10007e60 tx=0x7efd1000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:25.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.603+0000 7efd0e7fc640 1 -- 172.21.15.154:0/2621401089 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd10014420 con 0x7efd1815eef0 2023-09-23T03:25:25.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.603+0000 7efd0e7fc640 1 -- 172.21.15.154:0/2621401089 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd100145a0 con 0x7efd1815eef0 2023-09-23T03:25:25.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.603+0000 7efd0e7fc640 1 -- 172.21.15.154:0/2621401089 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd10014950 con 0x7efd1815eef0 2023-09-23T03:25:25.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.603+0000 7efd1eed7640 1 -- 172.21.15.154:0/2621401089 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd1815ceb0 con 0x7efd1815eef0 2023-09-23T03:25:25.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.603+0000 7efd1eed7640 1 -- 172.21.15.154:0/2621401089 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd181f1e70 con 0x7efd1815eef0 2023-09-23T03:25:25.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.604+0000 7efd1eed7640 1 -- 172.21.15.154:0/2621401089 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd1815e3c0 con 0x7efd1815eef0 2023-09-23T03:25:25.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.605+0000 7efd0e7fc640 1 -- 172.21.15.154:0/2621401089 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7efd10014ab0 con 0x7efd1815eef0 2023-09-23T03:25:25.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.606+0000 7efd0e7fc640 1 --2- 172.21.15.154:0/2621401089 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efcfc06af30 0x7efcfc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:25.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.606+0000 7efd0e7fc640 1 -- 172.21.15.154:0/2621401089 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6691+0+0 (secure 0 0 0) 0x7efd1008a490 con 0x7efd1815eef0 2023-09-23T03:25:25.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.606+0000 7efd1ded5640 1 --2- 172.21.15.154:0/2621401089 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efcfc06af30 0x7efcfc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:25.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.607+0000 7efd1ded5640 1 --2- 172.21.15.154:0/2621401089 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efcfc06af30 0x7efcfc06d3f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7efd14000f60 tx=0x7efd14009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:25.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.610+0000 7efd0e7fc640 1 -- 172.21.15.154:0/2621401089 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7efd100550e0 con 0x7efd1815eef0 2023-09-23T03:25:25.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.799+0000 7efd1eed7640 1 -- 172.21.15.154:0/2621401089 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7efd180ab8c0 con 0x7efd1815eef0 2023-09-23T03:25:25.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.800+0000 7efd0e7fc640 1 -- 172.21.15.154:0/2621401089 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v92) v1 ==== 106+0+11 (secure 0 0 0) 0x7efd180ab8c0 con 0x7efd1815eef0 2023-09-23T03:25:25.802 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:25:25.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.803+0000 7efd1eed7640 1 -- 172.21.15.154:0/2621401089 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efcfc06af30 msgr2=0x7efcfc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:25.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.803+0000 7efd1eed7640 1 --2- 172.21.15.154:0/2621401089 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efcfc06af30 0x7efcfc06d3f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7efd14000f60 tx=0x7efd14009040 comp rx=0 tx=0).stop 2023-09-23T03:25:25.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.803+0000 7efd1eed7640 1 -- 172.21.15.154:0/2621401089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd1815eef0 msgr2=0x7efd180c8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:25.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.803+0000 7efd1eed7640 1 --2- 172.21.15.154:0/2621401089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd1815eef0 0x7efd180c8f60 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7efd10007e60 tx=0x7efd1000e4b0 comp rx=0 tx=0).stop 2023-09-23T03:25:25.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.804+0000 7efd1eed7640 1 -- 172.21.15.154:0/2621401089 shutdown_connections 2023-09-23T03:25:25.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.804+0000 7efd1eed7640 1 --2- 172.21.15.154:0/2621401089 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efcfc06af30 0x7efcfc06d3f0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:25.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.804+0000 7efd1eed7640 1 --2- 172.21.15.154:0/2621401089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd1815eef0 0x7efd180c8f60 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:25.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.804+0000 7efd1eed7640 1 --2- 172.21.15.154:0/2621401089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd1815ba50 0x7efd180c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:25.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.804+0000 7efd1eed7640 1 --2- 172.21.15.154:0/2621401089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd18158480 0x7efd181f1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:25.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.804+0000 7efd1eed7640 1 -- 172.21.15.154:0/2621401089 >> 172.21.15.154:0/2621401089 conn(0x7efd180aff20 msgr2=0x7efd18070fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:25.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.804+0000 7efd1eed7640 1 -- 172.21.15.154:0/2621401089 shutdown_connections 2023-09-23T03:25:25.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:25.804+0000 7efd1eed7640 1 -- 172.21.15.154:0/2621401089 wait complete. 2023-09-23T03:25:25.824 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-23T03:25:26.039 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.037+0000 7efdc8ab4640 1 Processor -- start 2023-09-23T03:25:26.039 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.037+0000 7efdc8ab4640 1 -- start start 2023-09-23T03:25:26.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.039+0000 7efdc8ab4640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efdc40acce0 0x7efdc415ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:26.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.039+0000 7efdc8ab4640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efdc40cf040 0x7efdc40ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:26.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.039+0000 7efdc8ab4640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdc415ac30 con 0x7efdc40b1b20 2023-09-23T03:25:26.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.039+0000 7efdc8ab4640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdc40b2e90 con 0x7efdc40cf040 2023-09-23T03:25:26.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.039+0000 7efdc8ab4640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdc40b1f70 con 0x7efdc40acce0 2023-09-23T03:25:26.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.039+0000 7efdc2d9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efdc40cf040 0x7efdc40ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:26.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.039+0000 7efdc1d9b640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7efdc40b1b20 0x7efdc40ce930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40952/0 (socket says 172.21.15.154:40952) 2023-09-23T03:25:26.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.039+0000 7efdc1d9b640 1 -- 172.21.15.154:0/2851735122 learned_addr learned my addr 172.21.15.154:0/2851735122 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:25:26.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.040+0000 7efdc259c640 1 --2- 172.21.15.154:0/2851735122 >> v2:172.21.15.190:3300/0 conn(0x7efdc40acce0 0x7efdc415ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:26.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.040+0000 7efdc2d9d640 1 -- 172.21.15.154:0/2851735122 >> v2:172.21.15.190:3300/0 conn(0x7efdc40acce0 msgr2=0x7efdc415ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:26.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.040+0000 7efdc2d9d640 1 --2- 172.21.15.154:0/2851735122 >> v2:172.21.15.190:3300/0 conn(0x7efdc40acce0 0x7efdc415ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:26.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.040+0000 7efdc2d9d640 1 -- 172.21.15.154:0/2851735122 >> v1:172.21.15.154:6789/0 conn(0x7efdc40b1b20 legacy=0x7efdc40ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:26.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.040+0000 7efdc2d9d640 1 -- 172.21.15.154:0/2851735122 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc415f2a0 con 0x7efdc40cf040 2023-09-23T03:25:26.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.040+0000 7efdc159a640 1 -- 172.21.15.154:0/2851735122 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3470737957 0 0) 0x7efdb8001490 con 0x7efdc40b1b20 2023-09-23T03:25:26.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.040+0000 7efdc259c640 1 --2- 172.21.15.154:0/2851735122 >> v2:172.21.15.190:3300/0 conn(0x7efdc40acce0 0x7efdc415ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:26.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.041+0000 7efdc2d9d640 1 --2- 172.21.15.154:0/2851735122 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdc40cf040 0x7efdc40ab140 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7efdc40cf570 tx=0x7efdbc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=372390bb235842e9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:26.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.041+0000 7efdc159a640 1 -- 172.21.15.154:0/2851735122 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdbc02e070 con 0x7efdc40cf040 2023-09-23T03:25:26.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.041+0000 7efdc159a640 1 -- 172.21.15.154:0/2851735122 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdbc0315a0 con 0x7efdc40cf040 2023-09-23T03:25:26.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.041+0000 7efdc159a640 1 -- 172.21.15.154:0/2851735122 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdbc031930 con 0x7efdc40cf040 2023-09-23T03:25:26.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.042+0000 7efdc8ab4640 1 -- 172.21.15.154:0/2851735122 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdc40cf040 msgr2=0x7efdc40ab140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:26.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.042+0000 7efdc8ab4640 1 --2- 172.21.15.154:0/2851735122 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdc40cf040 0x7efdc40ab140 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7efdc40cf570 tx=0x7efdbc02b840 comp rx=0 tx=0).stop 2023-09-23T03:25:26.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.042+0000 7efdc8ab4640 1 -- 172.21.15.154:0/2851735122 shutdown_connections 2023-09-23T03:25:26.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.042+0000 7efdc8ab4640 1 --2- 172.21.15.154:0/2851735122 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdc40cf040 0x7efdc40ab140 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:26.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.042+0000 7efdc8ab4640 1 --2- 172.21.15.154:0/2851735122 >> v2:172.21.15.190:3300/0 conn(0x7efdc40acce0 0x7efdc415ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:26.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.042+0000 7efdc8ab4640 1 -- 172.21.15.154:0/2851735122 >> 172.21.15.154:0/2851735122 conn(0x7efdc406a0f0 msgr2=0x7efdc406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:26.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.042+0000 7efdc8ab4640 1 -- 172.21.15.154:0/2851735122 shutdown_connections 2023-09-23T03:25:26.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.042+0000 7efdc8ab4640 1 -- 172.21.15.154:0/2851735122 wait complete. 2023-09-23T03:25:26.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.044+0000 7efdc8ab4640 1 Processor -- start 2023-09-23T03:25:26.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.044+0000 7efdc8ab4640 1 -- start start 2023-09-23T03:25:26.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.045+0000 7efdc8ab4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdc40acce0 0x7efdc415d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:26.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.045+0000 7efdc8ab4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdc40b1b20 0x7efdc41f1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:26.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.045+0000 7efdc8ab4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdc40cf040 0x7efdc4159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:26.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.045+0000 7efdc8ab4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc41f2830 con 0x7efdc40b1b20 2023-09-23T03:25:26.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.045+0000 7efdc8ab4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc41f2b10 con 0x7efdc40acce0 2023-09-23T03:25:26.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.045+0000 7efdc8ab4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdc41f2df0 con 0x7efdc40cf040 2023-09-23T03:25:26.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.045+0000 7efdc1d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdc40b1b20 0x7efdc41f1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:26.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.045+0000 7efdc1d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdc40b1b20 0x7efdc41f1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50642/0 (socket says 172.21.15.154:50642) 2023-09-23T03:25:26.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.045+0000 7efdc1d9b640 1 -- 172.21.15.154:0/825632587 learned_addr learned my addr 172.21.15.154:0/825632587 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:26.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.046+0000 7efdc259c640 1 --2- 172.21.15.154:0/825632587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdc40acce0 0x7efdc415d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:26.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.046+0000 7efdc1d9b640 1 -- 172.21.15.154:0/825632587 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdc40cf040 msgr2=0x7efdc4159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:26.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.046+0000 7efdc2d9d640 1 --2- 172.21.15.154:0/825632587 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdc40cf040 0x7efdc4159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:26.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.046+0000 7efdc1d9b640 1 --2- 172.21.15.154:0/825632587 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdc40cf040 0x7efdc4159700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:26.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.046+0000 7efdc1d9b640 1 -- 172.21.15.154:0/825632587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdc40acce0 msgr2=0x7efdc415d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:26.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.046+0000 7efdc1d9b640 1 --2- 172.21.15.154:0/825632587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdc40acce0 0x7efdc415d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:26.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.046+0000 7efdc1d9b640 1 -- 172.21.15.154:0/825632587 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdbc009400 con 0x7efdc40b1b20 2023-09-23T03:25:26.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.046+0000 7efdc259c640 1 --2- 172.21.15.154:0/825632587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdc40acce0 0x7efdc415d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:26.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.046+0000 7efdc1d9b640 1 --2- 172.21.15.154:0/825632587 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdc40b1b20 0x7efdc41f1f70 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7efdb800c890 tx=0x7efdb800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:26.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.046+0000 7efdb2ffd640 1 -- 172.21.15.154:0/825632587 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb800a640 con 0x7efdc40b1b20 2023-09-23T03:25:26.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.046+0000 7efdb2ffd640 1 -- 172.21.15.154:0/825632587 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdb800a7a0 con 0x7efdc40b1b20 2023-09-23T03:25:26.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.046+0000 7efdb2ffd640 1 -- 172.21.15.154:0/825632587 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb800ab30 con 0x7efdc40b1b20 2023-09-23T03:25:26.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.046+0000 7efdc8ab4640 1 -- 172.21.15.154:0/825632587 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdc41efc50 con 0x7efdc40b1b20 2023-09-23T03:25:26.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.047+0000 7efdc8ab4640 1 -- 172.21.15.154:0/825632587 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdc41f24b0 con 0x7efdc40b1b20 2023-09-23T03:25:26.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.048+0000 7efdc8ab4640 1 -- 172.21.15.154:0/825632587 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdc41f0170 con 0x7efdc40b1b20 2023-09-23T03:25:26.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.049+0000 7efdb2ffd640 1 -- 172.21.15.154:0/825632587 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7efdb800ac90 con 0x7efdc40b1b20 2023-09-23T03:25:26.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.050+0000 7efdb2ffd640 1 --2- 172.21.15.154:0/825632587 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efda406af60 0x7efda406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:26.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.050+0000 7efdb2ffd640 1 -- 172.21.15.154:0/825632587 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6691+0+0 (secure 0 0 0) 0x7efdb8088e20 con 0x7efdc40b1b20 2023-09-23T03:25:26.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.050+0000 7efdc259c640 1 --2- 172.21.15.154:0/825632587 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efda406af60 0x7efda406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:26.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.051+0000 7efdc259c640 1 --2- 172.21.15.154:0/825632587 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efda406af60 0x7efda406d420 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7efdb4008870 tx=0x7efdb4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:26.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.054+0000 7efdb2ffd640 1 -- 172.21.15.154:0/825632587 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7efdc41f0170 con 0x7efdc40b1b20 2023-09-23T03:25:26.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.259+0000 7efdc8ab4640 1 -- 172.21.15.154:0/825632587 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7efdc41f0170 con 0x7efdc40b1b20 2023-09-23T03:25:26.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.259+0000 7efdb2ffd640 1 -- 172.21.15.154:0/825632587 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v92) v1 ==== 80+0+372 (secure 0 0 0) 0x7efdc41f0170 con 0x7efdc40b1b20 2023-09-23T03:25:26.261 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.154:6837/1588996498 2023-09-24T03:24:45.398760+0000 2023-09-23T03:25:26.261 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.154:6836/1588996498 2023-09-24T03:24:45.398760+0000 2023-09-23T03:25:26.261 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.154:6837/3812680297 2023-09-24T03:24:38.360723+0000 2023-09-23T03:25:26.261 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.154:6836/3812680297 2023-09-24T03:24:38.360723+0000 2023-09-23T03:25:26.261 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.154:6837/2213283923 2023-09-24T03:24:36.268262+0000 2023-09-23T03:25:26.261 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.154:6836/2213283923 2023-09-24T03:24:36.268262+0000 2023-09-23T03:25:26.262 INFO:teuthology.orchestra.run.smithi154.stderr:listed 6 entries 2023-09-23T03:25:26.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.262+0000 7efdc8ab4640 1 -- 172.21.15.154:0/825632587 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efda406af60 msgr2=0x7efda406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:26.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.262+0000 7efdc8ab4640 1 --2- 172.21.15.154:0/825632587 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efda406af60 0x7efda406d420 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7efdb4008870 tx=0x7efdb4008040 comp rx=0 tx=0).stop 2023-09-23T03:25:26.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.262+0000 7efdc8ab4640 1 -- 172.21.15.154:0/825632587 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdc40b1b20 msgr2=0x7efdc41f1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:26.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.262+0000 7efdc8ab4640 1 --2- 172.21.15.154:0/825632587 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdc40b1b20 0x7efdc41f1f70 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7efdb800c890 tx=0x7efdb800cd50 comp rx=0 tx=0).stop 2023-09-23T03:25:26.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.262+0000 7efdc8ab4640 1 -- 172.21.15.154:0/825632587 shutdown_connections 2023-09-23T03:25:26.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.262+0000 7efdc8ab4640 1 --2- 172.21.15.154:0/825632587 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efda406af60 0x7efda406d420 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:26.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.262+0000 7efdc8ab4640 1 --2- 172.21.15.154:0/825632587 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efdc40cf040 0x7efdc4159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:26.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.262+0000 7efdc8ab4640 1 --2- 172.21.15.154:0/825632587 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdc40b1b20 0x7efdc41f1f70 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:26.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.262+0000 7efdc8ab4640 1 --2- 172.21.15.154:0/825632587 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efdc40acce0 0x7efdc415d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:26.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.262+0000 7efdc8ab4640 1 -- 172.21.15.154:0/825632587 >> 172.21.15.154:0/825632587 conn(0x7efdc406a0f0 msgr2=0x7efdc40726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:26.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.262+0000 7efdc8ab4640 1 -- 172.21.15.154:0/825632587 shutdown_connections 2023-09-23T03:25:26.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:26.262+0000 7efdc8ab4640 1 -- 172.21.15.154:0/825632587 wait complete. 2023-09-23T03:25:26.281 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi154.front.sepia.ceph.com... 2023-09-23T03:25:26.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:25:26.281 DEBUG:teuthology.orchestra.run.smithi154:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-09-23T03:25:26.373 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:ceph-fuse[118311]: fuse finished with error 0 and tester_r 0 2023-09-23T03:25:26.379 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2023-09-23T03:25:26.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:25:26.379 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-09-23T03:25:26.479 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:25:26.479 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:25:26.540 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053603,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248804k,nr_inodes=812201,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-09-23T03:25:26.540 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi190.front.sepia.ceph.com 2023-09-23T03:25:26.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:25:26.540 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-09-23T03:25:26.594 INFO:teuthology.orchestra.run.smithi190.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-09-23T03:25:26.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T03:25:26.596 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:25:26.596 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:25:26.654 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053600,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-09-23T03:25:26.654 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2023-09-23T03:25:26.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:25:26.654 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-09-23T03:25:26.705 INFO:teuthology.orchestra.run.smithi154.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-09-23T03:25:26.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T03:25:26.707 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:25:26.707 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:25:26.765 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053603,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248804k,nr_inodes=812201,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-09-23T03:25:26.765 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi190.front.sepia.ceph.com 2023-09-23T03:25:26.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:25:26.765 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-09-23T03:25:26.815 INFO:teuthology.orchestra.run.smithi190.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-09-23T03:25:26.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T03:25:26.817 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:25:27.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.040+0000 7fdacb8b4640 1 Processor -- start 2023-09-23T03:25:27.042 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.040+0000 7fdacb8b4640 1 -- start start 2023-09-23T03:25:27.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.042+0000 7fdacb8b4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdac4160f90 0x7fdac4167950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:27.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.042+0000 7fdacb8b4640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdac40cd530 con 0x7fdac40cf040 2023-09-23T03:25:27.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.042+0000 7fdacb8b4640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdac40cd810 con 0x7fdac415caf0 2023-09-23T03:25:27.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.042+0000 7fdacb8b4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac4161440 con 0x7fdac4160f90 2023-09-23T03:25:27.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.042+0000 7fdac8e4f640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fdac40cf040 0x7fdac4160880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40968/0 (socket says 172.21.15.154:40968) 2023-09-23T03:25:27.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.042+0000 7fdac8e4f640 1 -- 172.21.15.154:0/1433474757 learned_addr learned my addr 172.21.15.154:0/1433474757 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:25:27.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.042+0000 7fdabbfff640 1 -- 172.21.15.154:0/1433474757 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2158165545 0 0) 0x7fdac40cd530 con 0x7fdac40cf040 2023-09-23T03:25:27.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.042+0000 7fdabbfff640 1 -- 172.21.15.154:0/1433474757 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdab0003520 con 0x7fdac40cf040 2023-09-23T03:25:27.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.042+0000 7fdac9e51640 1 --2- 172.21.15.154:0/1433474757 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdac4160f90 0x7fdac4167950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:27.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.043+0000 7fdabbfff640 1 -- 172.21.15.154:0/1433474757 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1075516592 0 0) 0x7fdab0003520 con 0x7fdac40cf040 2023-09-23T03:25:27.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.043+0000 7fdabbfff640 1 -- 172.21.15.154:0/1433474757 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdac4160f90 msgr2=0x7fdac4167950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:27.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.043+0000 7fdabbfff640 1 --2- 172.21.15.154:0/1433474757 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdac4160f90 0x7fdac4167950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:27.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.043+0000 7fdabbfff640 1 -- 172.21.15.154:0/1433474757 >> v1:172.21.15.190:6789/0 conn(0x7fdac415caf0 legacy=0x7fdac40ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:27.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.043+0000 7fdabbfff640 1 -- 172.21.15.154:0/1433474757 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac40cdaf0 con 0x7fdac40cf040 2023-09-23T03:25:27.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.043+0000 7fdabbfff640 1 -- 172.21.15.154:0/1433474757 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fdab4001670 con 0x7fdac40cf040 2023-09-23T03:25:27.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.043+0000 7fdabbfff640 1 -- 172.21.15.154:0/1433474757 >> v1:172.21.15.154:6789/0 conn(0x7fdac40cf040 legacy=0x7fdac4160880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:27.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.043+0000 7fdabbfff640 1 --2- 172.21.15.154:0/1433474757 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdab00039a0 0x7fdab0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:27.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.043+0000 7fdabbfff640 1 --2- 172.21.15.154:0/1433474757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdab00047c0 0x7fdab0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:27.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.043+0000 7fdabbfff640 1 --2- 172.21.15.154:0/1433474757 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdac4160f90 0x7fdab0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:27.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.043+0000 7fdabbfff640 1 -- 172.21.15.154:0/1433474757 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac4161440 con 0x7fdab00039a0 2023-09-23T03:25:27.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.043+0000 7fdabbfff640 1 -- 172.21.15.154:0/1433474757 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab000bb10 con 0x7fdab00047c0 2023-09-23T03:25:27.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.043+0000 7fdabbfff640 1 -- 172.21.15.154:0/1433474757 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdab000bcd0 con 0x7fdac4160f90 2023-09-23T03:25:27.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.043+0000 7fdac9650640 1 --2- 172.21.15.154:0/1433474757 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdab00039a0 0x7fdab0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:27.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.044+0000 7fdac8e4f640 1 --2- 172.21.15.154:0/1433474757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdab00047c0 0x7fdab0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:27.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.044+0000 7fdac9650640 1 -- 172.21.15.154:0/1433474757 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdac4160f90 msgr2=0x7fdab0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:27.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.044+0000 7fdac9650640 1 --2- 172.21.15.154:0/1433474757 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdac4160f90 0x7fdab0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:27.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.044+0000 7fdac9650640 1 -- 172.21.15.154:0/1433474757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdab00047c0 msgr2=0x7fdab0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:27.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.044+0000 7fdac9650640 1 --2- 172.21.15.154:0/1433474757 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdab00047c0 0x7fdab0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:27.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.044+0000 7fdac9650640 1 -- 172.21.15.154:0/1433474757 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab0003520 con 0x7fdab00039a0 2023-09-23T03:25:27.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.045+0000 7fdac9650640 1 --2- 172.21.15.154:0/1433474757 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdab00039a0 0x7fdab0003dc0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fdac0001770 tx=0x7fdab00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a2f7e8fbfa9a6b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:27.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.045+0000 7fdabbfff640 1 -- 172.21.15.154:0/1433474757 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac00052d0 con 0x7fdab00039a0 2023-09-23T03:25:27.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.045+0000 7fdabbfff640 1 -- 172.21.15.154:0/1433474757 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac002e050 con 0x7fdab00039a0 2023-09-23T03:25:27.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.045+0000 7fdabbfff640 1 -- 172.21.15.154:0/1433474757 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac0003640 con 0x7fdab00039a0 2023-09-23T03:25:27.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.046+0000 7fdacb8b4640 1 -- 172.21.15.154:0/1433474757 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdab00039a0 msgr2=0x7fdab0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:27.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.046+0000 7fdacb8b4640 1 --2- 172.21.15.154:0/1433474757 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdab00039a0 0x7fdab0003dc0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fdac0001770 tx=0x7fdab00057f0 comp rx=0 tx=0).stop 2023-09-23T03:25:27.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.046+0000 7fdac9e51640 1 -- 172.21.15.154:0/1433474757 reap_dead start 2023-09-23T03:25:27.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.046+0000 7fdacb8b4640 1 -- 172.21.15.154:0/1433474757 shutdown_connections 2023-09-23T03:25:27.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.046+0000 7fdacb8b4640 1 -- 172.21.15.154:0/1433474757 >> 172.21.15.154:0/1433474757 conn(0x7fdac406a0f0 msgr2=0x7fdac406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:27.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.046+0000 7fdacb8b4640 1 -- 172.21.15.154:0/1433474757 shutdown_connections 2023-09-23T03:25:27.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.046+0000 7fdacb8b4640 1 -- 172.21.15.154:0/1433474757 wait complete. 2023-09-23T03:25:27.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.048+0000 7fdacb8b4640 1 Processor -- start 2023-09-23T03:25:27.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.048+0000 7fdacb8b4640 1 -- start start 2023-09-23T03:25:27.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.049+0000 7fdacb8b4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdab00039a0 0x7fdac40982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:27.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.049+0000 7fdacb8b4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdac415e940 0x7fdac4155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:27.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.049+0000 7fdacb8b4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdac4158b80 0x7fdac4159030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:27.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.049+0000 7fdacb8b4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac415bc20 con 0x7fdac415e940 2023-09-23T03:25:27.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.049+0000 7fdacb8b4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac40cee00 con 0x7fdab00039a0 2023-09-23T03:25:27.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.049+0000 7fdacb8b4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac415be70 con 0x7fdac4158b80 2023-09-23T03:25:27.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.049+0000 7fdac8e4f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdac415e940 0x7fdac4155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:27.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.049+0000 7fdac8e4f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdac415e940 0x7fdac4155590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50660/0 (socket says 172.21.15.154:50660) 2023-09-23T03:25:27.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.049+0000 7fdac8e4f640 1 -- 172.21.15.154:0/2860866349 learned_addr learned my addr 172.21.15.154:0/2860866349 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:27.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.050+0000 7fdac8e4f640 1 -- 172.21.15.154:0/2860866349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdac4158b80 msgr2=0x7fdac4159030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:27.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.050+0000 7fdac8e4f640 1 --2- 172.21.15.154:0/2860866349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdac4158b80 0x7fdac4159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:27.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.050+0000 7fdac8e4f640 1 -- 172.21.15.154:0/2860866349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdab00039a0 msgr2=0x7fdac40982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:27.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.050+0000 7fdac9650640 1 --2- 172.21.15.154:0/2860866349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdab00039a0 0x7fdac40982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:27.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.050+0000 7fdac8e4f640 1 --2- 172.21.15.154:0/2860866349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdab00039a0 0x7fdac40982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:27.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.050+0000 7fdac8e4f640 1 -- 172.21.15.154:0/2860866349 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac415c150 con 0x7fdac415e940 2023-09-23T03:25:27.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.050+0000 7fdac9650640 1 --2- 172.21.15.154:0/2860866349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdab00039a0 0x7fdac40982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:27.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.050+0000 7fdac8e4f640 1 --2- 172.21.15.154:0/2860866349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdac415e940 0x7fdac4155590 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fdab400e730 tx=0x7fdab400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:27.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.050+0000 7fdab9ffb640 1 -- 172.21.15.154:0/2860866349 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab400ee40 con 0x7fdac415e940 2023-09-23T03:25:27.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.050+0000 7fdab9ffb640 1 -- 172.21.15.154:0/2860866349 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab400c8d0 con 0x7fdac415e940 2023-09-23T03:25:27.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.050+0000 7fdab9ffb640 1 -- 172.21.15.154:0/2860866349 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab400cc80 con 0x7fdac415e940 2023-09-23T03:25:27.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.050+0000 7fdacb8b4640 1 -- 172.21.15.154:0/2860866349 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdac40c4900 con 0x7fdac415e940 2023-09-23T03:25:27.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.050+0000 7fdacb8b4640 1 -- 172.21.15.154:0/2860866349 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac4155ad0 con 0x7fdac415e940 2023-09-23T03:25:27.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.051+0000 7fdacb8b4640 1 -- 172.21.15.154:0/2860866349 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdac4155cf0 con 0x7fdac415e940 2023-09-23T03:25:27.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.052+0000 7fdab9ffb640 1 -- 172.21.15.154:0/2860866349 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fdab401d460 con 0x7fdac415e940 2023-09-23T03:25:27.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.053+0000 7fdab9ffb640 1 --2- 172.21.15.154:0/2860866349 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdaa806af60 0x7fdaa806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:27.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.053+0000 7fdab9ffb640 1 -- 172.21.15.154:0/2860866349 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fdab4089c10 con 0x7fdac415e940 2023-09-23T03:25:27.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.054+0000 7fdac9650640 1 --2- 172.21.15.154:0/2860866349 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdaa806af60 0x7fdaa806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:27.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.054+0000 7fdac9650640 1 --2- 172.21.15.154:0/2860866349 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdaa806af60 0x7fdaa806d420 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fdac0001770 tx=0x7fdac0009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:27.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.058+0000 7fdab9ffb640 1 -- 172.21.15.154:0/2860866349 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fdab4054740 con 0x7fdac415e940 2023-09-23T03:25:27.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.285+0000 7fdacb8b4640 1 -- 172.21.15.154:0/2860866349 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdac40c4760 con 0x7fdac415e940 2023-09-23T03:25:27.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.286+0000 7fdab9ffb640 1 -- 172.21.15.154:0/2860866349 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4925 (secure 0 0 0) 0x7fdac40c4760 con 0x7fdac415e940 2023-09-23T03:25:27.288 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:25:27.288 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":76,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7743,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/31599836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":31599836},{"type":"v1","addr":"172.21.15.154:6836","nonce":31599836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":7752,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/4266451911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":4266451911},{"type":"v1","addr":"172.21.15.154:6837","nonce":4266451911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":7755,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3029600822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3029600822},{"type":"v1","addr":"172.21.15.190:6834","nonce":3029600822}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:24:37.386034+0000","modified":"2023-09-23T03:24:47.439439+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":90,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7758},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7758":{"gid":7758,"name":"b","rank":0,"incarnation":73,"state":"up:active","state_seq":4,"addr":"172.21.15.190:6835/2202597679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2202597679},{"type":"v1","addr":"172.21.15.190:6835","nonce":2202597679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12,13],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-09-23T03:25:27.288 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 76 2023-09-23T03:25:27.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.289+0000 7fdacb8b4640 1 -- 172.21.15.154:0/2860866349 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdaa806af60 msgr2=0x7fdaa806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:27.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.289+0000 7fdacb8b4640 1 --2- 172.21.15.154:0/2860866349 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdaa806af60 0x7fdaa806d420 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fdac0001770 tx=0x7fdac0009030 comp rx=0 tx=0).stop 2023-09-23T03:25:27.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.289+0000 7fdacb8b4640 1 -- 172.21.15.154:0/2860866349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdac415e940 msgr2=0x7fdac4155590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:27.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.289+0000 7fdacb8b4640 1 --2- 172.21.15.154:0/2860866349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdac415e940 0x7fdac4155590 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fdab400e730 tx=0x7fdab400ebf0 comp rx=0 tx=0).stop 2023-09-23T03:25:27.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.289+0000 7fdacb8b4640 1 -- 172.21.15.154:0/2860866349 shutdown_connections 2023-09-23T03:25:27.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.289+0000 7fdacb8b4640 1 --2- 172.21.15.154:0/2860866349 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdaa806af60 0x7fdaa806d420 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:27.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.289+0000 7fdacb8b4640 1 --2- 172.21.15.154:0/2860866349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdac4158b80 0x7fdac4159030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:27.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.289+0000 7fdacb8b4640 1 --2- 172.21.15.154:0/2860866349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdac415e940 0x7fdac4155590 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:27.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.289+0000 7fdacb8b4640 1 --2- 172.21.15.154:0/2860866349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdab00039a0 0x7fdac40982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:27.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.289+0000 7fdacb8b4640 1 -- 172.21.15.154:0/2860866349 >> 172.21.15.154:0/2860866349 conn(0x7fdac406a0f0 msgr2=0x7fdac40716d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:27.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.290+0000 7fdacb8b4640 1 -- 172.21.15.154:0/2860866349 shutdown_connections 2023-09-23T03:25:27.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.290+0000 7fdacb8b4640 1 -- 172.21.15.154:0/2860866349 wait complete. 2023-09-23T03:25:27.310 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:25:27.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.526+0000 7fe44a400640 1 Processor -- start 2023-09-23T03:25:27.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.526+0000 7fe44a400640 1 -- start start 2023-09-23T03:25:27.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.528+0000 7fe44a400640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe4440acce0 0x7fe44415ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:27.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.528+0000 7fe44a400640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe4440cf040 0x7fe4440ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:27.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.528+0000 7fe44a400640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe44415ac30 con 0x7fe4440b1b20 2023-09-23T03:25:27.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.528+0000 7fe44a400640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4440b2e90 con 0x7fe4440cf040 2023-09-23T03:25:27.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.528+0000 7fe44a400640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4440b1f70 con 0x7fe4440acce0 2023-09-23T03:25:27.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.528+0000 7fe44899d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe4440cf040 0x7fe4440ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:27.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.528+0000 7fe43bfff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fe4440b1b20 0x7fe4440ce930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40970/0 (socket says 172.21.15.154:40970) 2023-09-23T03:25:27.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.528+0000 7fe44899d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe4440cf040 0x7fe4440ab140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50662/0 (socket says 172.21.15.154:50662) 2023-09-23T03:25:27.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.528+0000 7fe43bfff640 1 -- 172.21.15.154:0/4238940642 learned_addr learned my addr 172.21.15.154:0/4238940642 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:25:27.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.528+0000 7fe443fff640 1 --2- 172.21.15.154:0/4238940642 >> v2:172.21.15.190:3300/0 conn(0x7fe4440acce0 0x7fe44415ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:27.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.529+0000 7fe44899d640 1 -- 172.21.15.154:0/4238940642 >> v2:172.21.15.190:3300/0 conn(0x7fe4440acce0 msgr2=0x7fe44415ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:27.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.529+0000 7fe44899d640 1 --2- 172.21.15.154:0/4238940642 >> v2:172.21.15.190:3300/0 conn(0x7fe4440acce0 0x7fe44415ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:27.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.529+0000 7fe44899d640 1 -- 172.21.15.154:0/4238940642 >> v1:172.21.15.154:6789/0 conn(0x7fe4440b1b20 legacy=0x7fe4440ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:27.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.529+0000 7fe44899d640 1 -- 172.21.15.154:0/4238940642 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe44415f2a0 con 0x7fe4440cf040 2023-09-23T03:25:27.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.530+0000 7fe44899d640 1 --2- 172.21.15.154:0/4238940642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4440cf040 0x7fe4440ab140 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fe4440cf570 tx=0x7fe43c02be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ffbe23f23974de9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:27.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.530+0000 7fe4437fe640 1 -- 172.21.15.154:0/4238940642 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43c02cbe0 con 0x7fe4440cf040 2023-09-23T03:25:27.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.530+0000 7fe4437fe640 1 -- 172.21.15.154:0/4238940642 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe43c02cd40 con 0x7fe4440cf040 2023-09-23T03:25:27.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.530+0000 7fe4437fe640 1 -- 172.21.15.154:0/4238940642 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43c034660 con 0x7fe4440cf040 2023-09-23T03:25:27.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.531+0000 7fe44a400640 1 -- 172.21.15.154:0/4238940642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4440cf040 msgr2=0x7fe4440ab140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:27.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.531+0000 7fe44a400640 1 --2- 172.21.15.154:0/4238940642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4440cf040 0x7fe4440ab140 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fe4440cf570 tx=0x7fe43c02be30 comp rx=0 tx=0).stop 2023-09-23T03:25:27.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.531+0000 7fe44a400640 1 -- 172.21.15.154:0/4238940642 shutdown_connections 2023-09-23T03:25:27.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.531+0000 7fe44a400640 1 --2- 172.21.15.154:0/4238940642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4440cf040 0x7fe4440ab140 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:27.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.531+0000 7fe44a400640 1 --2- 172.21.15.154:0/4238940642 >> v2:172.21.15.190:3300/0 conn(0x7fe4440acce0 0x7fe44415ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:27.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.531+0000 7fe44a400640 1 -- 172.21.15.154:0/4238940642 >> 172.21.15.154:0/4238940642 conn(0x7fe44406a0f0 msgr2=0x7fe44406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:27.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.531+0000 7fe44a400640 1 -- 172.21.15.154:0/4238940642 shutdown_connections 2023-09-23T03:25:27.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.531+0000 7fe44a400640 1 -- 172.21.15.154:0/4238940642 wait complete. 2023-09-23T03:25:27.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.533+0000 7fe44a400640 1 Processor -- start 2023-09-23T03:25:27.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.533+0000 7fe44a400640 1 -- start start 2023-09-23T03:25:27.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.534+0000 7fe44a400640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4440acce0 0x7fe4440c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:27.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.534+0000 7fe44a400640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4440b1b20 0x7fe4440c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:27.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.534+0000 7fe44a400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4440cf040 0x7fe444152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:27.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.534+0000 7fe44a400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4440c62e0 con 0x7fe4440cf040 2023-09-23T03:25:27.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.534+0000 7fe44a400640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4440c1bd0 con 0x7fe4440acce0 2023-09-23T03:25:27.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.534+0000 7fe44a400640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4440c8130 con 0x7fe4440b1b20 2023-09-23T03:25:27.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.534+0000 7fe44899d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4440cf040 0x7fe444152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:27.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.534+0000 7fe44899d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4440cf040 0x7fe444152d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50674/0 (socket says 172.21.15.154:50674) 2023-09-23T03:25:27.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.534+0000 7fe44899d640 1 -- 172.21.15.154:0/541965729 learned_addr learned my addr 172.21.15.154:0/541965729 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:27.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.534+0000 7fe443fff640 1 --2- 172.21.15.154:0/541965729 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4440acce0 0x7fe4440c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:27.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.534+0000 7fe44899d640 1 -- 172.21.15.154:0/541965729 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4440b1b20 msgr2=0x7fe4440c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:27.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.534+0000 7fe44899d640 1 --2- 172.21.15.154:0/541965729 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4440b1b20 0x7fe4440c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:27.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.534+0000 7fe44899d640 1 -- 172.21.15.154:0/541965729 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4440acce0 msgr2=0x7fe4440c5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:27.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.534+0000 7fe44899d640 1 --2- 172.21.15.154:0/541965729 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4440acce0 0x7fe4440c5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:27.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.534+0000 7fe44899d640 1 -- 172.21.15.154:0/541965729 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe43c009a70 con 0x7fe4440cf040 2023-09-23T03:25:27.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.534+0000 7fe443fff640 1 --2- 172.21.15.154:0/541965729 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4440acce0 0x7fe4440c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:27.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.535+0000 7fe44899d640 1 --2- 172.21.15.154:0/541965729 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4440cf040 0x7fe444152d40 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009e80 tx=0x7fe43c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:27.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.535+0000 7fe4417fa640 1 -- 172.21.15.154:0/541965729 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43c040650 con 0x7fe4440cf040 2023-09-23T03:25:27.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.535+0000 7fe4417fa640 1 -- 172.21.15.154:0/541965729 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe43c0407b0 con 0x7fe4440cf040 2023-09-23T03:25:27.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.535+0000 7fe4417fa640 1 -- 172.21.15.154:0/541965729 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43c040b40 con 0x7fe4440cf040 2023-09-23T03:25:27.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.535+0000 7fe44a400640 1 -- 172.21.15.154:0/541965729 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4440c8410 con 0x7fe4440cf040 2023-09-23T03:25:27.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.535+0000 7fe44a400640 1 -- 172.21.15.154:0/541965729 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe444096b60 con 0x7fe4440cf040 2023-09-23T03:25:27.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.536+0000 7fe44a400640 1 -- 172.21.15.154:0/541965729 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4440cba30 con 0x7fe4440cf040 2023-09-23T03:25:27.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.537+0000 7fe4417fa640 1 -- 172.21.15.154:0/541965729 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fe43c040ca0 con 0x7fe4440cf040 2023-09-23T03:25:27.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.538+0000 7fe4417fa640 1 --2- 172.21.15.154:0/541965729 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe42406af30 0x7fe42406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:27.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.538+0000 7fe4417fa640 1 -- 172.21.15.154:0/541965729 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fe43c0acd90 con 0x7fe4440cf040 2023-09-23T03:25:27.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.538+0000 7fe443fff640 1 --2- 172.21.15.154:0/541965729 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe42406af30 0x7fe42406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:27.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.539+0000 7fe443fff640 1 --2- 172.21.15.154:0/541965729 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe42406af30 0x7fe42406d3f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe434009820 tx=0x7fe434008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:27.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.542+0000 7fe4417fa640 1 -- 172.21.15.154:0/541965729 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fe43c077950 con 0x7fe4440cf040 2023-09-23T03:25:27.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.734+0000 7fe44a400640 1 -- 172.21.15.154:0/541965729 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe444073eb0 con 0x7fe4440cf040 2023-09-23T03:25:27.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.736+0000 7fe4417fa640 1 -- 172.21.15.154:0/541965729 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+18993 (secure 0 0 0) 0x7fe43c039090 con 0x7fe4440cf040 2023-09-23T03:25:27.738 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:25:27.738 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":92,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:24:46.339781+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:24:07.834123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-09-23T03:24:09.058217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data_extra","create_time":"2023-09-23T03:24:19.995906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6837/1588996498":"2023-09-24T03:24:45.398760+0000","172.21.15.154:6836/1588996498":"2023-09-24T03:24:45.398760+0000","172.21.15.154:6837/3812680297":"2023-09-24T03:24:38.360723+0000","172.21.15.154:6836/3812680297":"2023-09-24T03:24:38.360723+0000","172.21.15.154:6837/2213283923":"2023-09-24T03:24:36.268262+0000","172.21.15.154:6836/2213283923":"2023-09-24T03:24:36.268262+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:25:27.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.739+0000 7fe44a400640 1 -- 172.21.15.154:0/541965729 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe42406af30 msgr2=0x7fe42406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:27.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.739+0000 7fe44a400640 1 --2- 172.21.15.154:0/541965729 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe42406af30 0x7fe42406d3f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe434009820 tx=0x7fe434008040 comp rx=0 tx=0).stop 2023-09-23T03:25:27.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.739+0000 7fe44a400640 1 -- 172.21.15.154:0/541965729 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4440cf040 msgr2=0x7fe444152d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:27.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.739+0000 7fe44a400640 1 --2- 172.21.15.154:0/541965729 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4440cf040 0x7fe444152d40 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009e80 tx=0x7fe43c004a40 comp rx=0 tx=0).stop 2023-09-23T03:25:27.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.739+0000 7fe44a400640 1 -- 172.21.15.154:0/541965729 shutdown_connections 2023-09-23T03:25:27.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.739+0000 7fe44a400640 1 --2- 172.21.15.154:0/541965729 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe42406af30 0x7fe42406d3f0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:27.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.739+0000 7fe44a400640 1 --2- 172.21.15.154:0/541965729 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4440cf040 0x7fe444152d40 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:27.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.739+0000 7fe44a400640 1 --2- 172.21.15.154:0/541965729 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4440b1b20 0x7fe4440c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:27.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.739+0000 7fe44a400640 1 --2- 172.21.15.154:0/541965729 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4440acce0 0x7fe4440c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:27.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.739+0000 7fe44a400640 1 -- 172.21.15.154:0/541965729 >> 172.21.15.154:0/541965729 conn(0x7fe44406a0f0 msgr2=0x7fe4440cb320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:27.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.739+0000 7fe44a400640 1 -- 172.21.15.154:0/541965729 shutdown_connections 2023-09-23T03:25:27.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.739+0000 7fe44a400640 1 -- 172.21.15.154:0/541965729 wait complete. 2023-09-23T03:25:27.758 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:24:07.834123+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:24:09.058217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-09-23T03:24:19.995906+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T03:25:27.759 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:25:27.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.966+0000 7fa7502af640 1 Processor -- start 2023-09-23T03:25:27.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.966+0000 7fa7502af640 1 -- start start 2023-09-23T03:25:27.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.968+0000 7fa7502af640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa74815bb30 0x7fa7480caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:27.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.968+0000 7fa7502af640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa74815b6a0 con 0x7fa74815aba0 2023-09-23T03:25:27.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.968+0000 7fa7502af640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa74815f0b0 con 0x7fa7480cb3e0 2023-09-23T03:25:27.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.968+0000 7fa7502af640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7480cb870 con 0x7fa74815bb30 2023-09-23T03:25:27.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.968+0000 7fa74e04b640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fa74815aba0 0x7fa74815e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40980/0 (socket says 172.21.15.154:40980) 2023-09-23T03:25:27.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.968+0000 7fa74e04b640 1 -- 172.21.15.154:0/4278206627 learned_addr learned my addr 172.21.15.154:0/4278206627 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:25:27.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.969+0000 7fa74d84a640 1 --2- 172.21.15.154:0/4278206627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa74815bb30 0x7fa7480caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:27.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.969+0000 7fa74d049640 1 -- 172.21.15.154:0/4278206627 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1456433397 0 0) 0x7fa74815b6a0 con 0x7fa74815aba0 2023-09-23T03:25:27.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.969+0000 7fa74d049640 1 -- 172.21.15.154:0/4278206627 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa734003520 con 0x7fa74815aba0 2023-09-23T03:25:27.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.969+0000 7fa74d049640 1 -- 172.21.15.154:0/4278206627 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4268520640 0 0) 0x7fa734003520 con 0x7fa74815aba0 2023-09-23T03:25:27.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.969+0000 7fa74d049640 1 -- 172.21.15.154:0/4278206627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa74815bb30 msgr2=0x7fa7480caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:27.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.969+0000 7fa74d049640 1 --2- 172.21.15.154:0/4278206627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa74815bb30 0x7fa7480caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:27.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.969+0000 7fa74d049640 1 -- 172.21.15.154:0/4278206627 >> v1:172.21.15.190:6789/0 conn(0x7fa7480cb3e0 legacy=0x7fa7480cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:27.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.969+0000 7fa74d049640 1 -- 172.21.15.154:0/4278206627 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa74815f390 con 0x7fa74815aba0 2023-09-23T03:25:27.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.969+0000 7fa74d049640 1 -- 172.21.15.154:0/4278206627 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fa744001670 con 0x7fa74815aba0 2023-09-23T03:25:27.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.969+0000 7fa74d049640 1 -- 172.21.15.154:0/4278206627 >> v1:172.21.15.154:6789/0 conn(0x7fa74815aba0 legacy=0x7fa74815e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:27.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.969+0000 7fa74d049640 1 --2- 172.21.15.154:0/4278206627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7340039a0 0x7fa734003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:27.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.969+0000 7fa74d049640 1 --2- 172.21.15.154:0/4278206627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa74815bb30 0x7fa7340047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:27.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.969+0000 7fa74d049640 1 --2- 172.21.15.154:0/4278206627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa734004d00 0x7fa734005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:27.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.969+0000 7fa74d049640 1 -- 172.21.15.154:0/4278206627 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7480cb870 con 0x7fa734004d00 2023-09-23T03:25:27.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.969+0000 7fa74d049640 1 -- 172.21.15.154:0/4278206627 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73400ba60 con 0x7fa7340039a0 2023-09-23T03:25:27.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.969+0000 7fa74d049640 1 -- 172.21.15.154:0/4278206627 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa73400bc20 con 0x7fa74815bb30 2023-09-23T03:25:27.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.970+0000 7fa74e84c640 1 --2- 172.21.15.154:0/4278206627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa734004d00 0x7fa734005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:27.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.970+0000 7fa74e04b640 1 --2- 172.21.15.154:0/4278206627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7340039a0 0x7fa734003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:27.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.970+0000 7fa74d84a640 1 --2- 172.21.15.154:0/4278206627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa74815bb30 0x7fa7340047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:27.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.970+0000 7fa74e84c640 1 -- 172.21.15.154:0/4278206627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa74815bb30 msgr2=0x7fa7340047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:27.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.970+0000 7fa74e84c640 1 --2- 172.21.15.154:0/4278206627 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa74815bb30 0x7fa7340047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:27.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.970+0000 7fa74e84c640 1 -- 172.21.15.154:0/4278206627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7340039a0 msgr2=0x7fa734003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:27.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.970+0000 7fa74e84c640 1 --2- 172.21.15.154:0/4278206627 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7340039a0 0x7fa734003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:27.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.970+0000 7fa74e84c640 1 -- 172.21.15.154:0/4278206627 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa734003520 con 0x7fa734004d00 2023-09-23T03:25:27.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.971+0000 7fa74e84c640 1 --2- 172.21.15.154:0/4278206627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa734004d00 0x7fa734005180 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fa740001e20 tx=0x7fa7340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c8713913e4503a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:27.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.971+0000 7fa74d049640 1 -- 172.21.15.154:0/4278206627 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa740003b70 con 0x7fa734004d00 2023-09-23T03:25:27.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.971+0000 7fa74d049640 1 -- 172.21.15.154:0/4278206627 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa740003cd0 con 0x7fa734004d00 2023-09-23T03:25:27.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.971+0000 7fa74d049640 1 -- 172.21.15.154:0/4278206627 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa740005310 con 0x7fa734004d00 2023-09-23T03:25:27.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.972+0000 7fa7502af640 1 -- 172.21.15.154:0/4278206627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa734004d00 msgr2=0x7fa734005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:27.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.972+0000 7fa7502af640 1 --2- 172.21.15.154:0/4278206627 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa734004d00 0x7fa734005180 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fa740001e20 tx=0x7fa7340057f0 comp rx=0 tx=0).stop 2023-09-23T03:25:27.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.972+0000 7fa74e84c640 1 -- 172.21.15.154:0/4278206627 reap_dead start 2023-09-23T03:25:27.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.972+0000 7fa7502af640 1 -- 172.21.15.154:0/4278206627 shutdown_connections 2023-09-23T03:25:27.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.972+0000 7fa7502af640 1 -- 172.21.15.154:0/4278206627 >> 172.21.15.154:0/4278206627 conn(0x7fa74806c010 msgr2=0x7fa74806c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:27.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.972+0000 7fa7502af640 1 -- 172.21.15.154:0/4278206627 shutdown_connections 2023-09-23T03:25:27.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.972+0000 7fa7502af640 1 -- 172.21.15.154:0/4278206627 wait complete. 2023-09-23T03:25:27.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.974+0000 7fa7502af640 1 Processor -- start 2023-09-23T03:25:27.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.974+0000 7fa7502af640 1 -- start start 2023-09-23T03:25:27.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.975+0000 7fa7502af640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa734004d00 0x7fa7480b3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:27.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.975+0000 7fa7502af640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7480ca3f0 0x7fa7480c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:27.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.975+0000 7fa7502af640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7480c83d0 0x7fa748154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:27.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.975+0000 7fa7502af640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7480c8b00 con 0x7fa7480c83d0 2023-09-23T03:25:27.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.975+0000 7fa7502af640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7480c9d30 con 0x7fa7480ca3f0 2023-09-23T03:25:27.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.975+0000 7fa7502af640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7480c9360 con 0x7fa734004d00 2023-09-23T03:25:27.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.975+0000 7fa74e84c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7480c83d0 0x7fa748154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:27.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.975+0000 7fa74e84c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7480c83d0 0x7fa748154960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50688/0 (socket says 172.21.15.154:50688) 2023-09-23T03:25:27.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.975+0000 7fa74e84c640 1 -- 172.21.15.154:0/1645871771 learned_addr learned my addr 172.21.15.154:0/1645871771 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:27.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.976+0000 7fa74e04b640 1 --2- 172.21.15.154:0/1645871771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa734004d00 0x7fa7480b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:27.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.976+0000 7fa74d84a640 1 --2- 172.21.15.154:0/1645871771 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7480ca3f0 0x7fa7480c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:27.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.976+0000 7fa74e84c640 1 -- 172.21.15.154:0/1645871771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa734004d00 msgr2=0x7fa7480b3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:27.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.976+0000 7fa74e84c640 1 --2- 172.21.15.154:0/1645871771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa734004d00 0x7fa7480b3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:27.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.976+0000 7fa74e84c640 1 -- 172.21.15.154:0/1645871771 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7480ca3f0 msgr2=0x7fa7480c97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:27.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.976+0000 7fa74e84c640 1 --2- 172.21.15.154:0/1645871771 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7480ca3f0 0x7fa7480c97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:27.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.976+0000 7fa74e84c640 1 -- 172.21.15.154:0/1645871771 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7480ca960 con 0x7fa7480c83d0 2023-09-23T03:25:27.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.976+0000 7fa74e04b640 1 --2- 172.21.15.154:0/1645871771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa734004d00 0x7fa7480b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:27.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.976+0000 7fa74d84a640 1 --2- 172.21.15.154:0/1645871771 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7480ca3f0 0x7fa7480c97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:27.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.976+0000 7fa74e84c640 1 --2- 172.21.15.154:0/1645871771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7480c83d0 0x7fa748154960 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fa740001f40 tx=0x7fa7400014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:27.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.976+0000 7fa73effd640 1 -- 172.21.15.154:0/1645871771 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74003e070 con 0x7fa7480c83d0 2023-09-23T03:25:27.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.976+0000 7fa73effd640 1 -- 172.21.15.154:0/1645871771 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa740003e40 con 0x7fa7480c83d0 2023-09-23T03:25:27.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.976+0000 7fa7502af640 1 -- 172.21.15.154:0/1645871771 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7480c8d10 con 0x7fa7480c83d0 2023-09-23T03:25:27.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.976+0000 7fa73effd640 1 -- 172.21.15.154:0/1645871771 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa740006d50 con 0x7fa7480c83d0 2023-09-23T03:25:27.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.977+0000 7fa7502af640 1 -- 172.21.15.154:0/1645871771 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7480c50e0 con 0x7fa7480c83d0 2023-09-23T03:25:27.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.978+0000 7fa7502af640 1 -- 172.21.15.154:0/1645871771 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7480c94d0 con 0x7fa7480c83d0 2023-09-23T03:25:27.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.979+0000 7fa73effd640 1 -- 172.21.15.154:0/1645871771 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fa740041070 con 0x7fa7480c83d0 2023-09-23T03:25:27.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.979+0000 7fa73effd640 1 --2- 172.21.15.154:0/1645871771 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa72c06af60 0x7fa72c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:27.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.979+0000 7fa73effd640 1 -- 172.21.15.154:0/1645871771 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fa7400aaa20 con 0x7fa7480c83d0 2023-09-23T03:25:27.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.980+0000 7fa74e04b640 1 --2- 172.21.15.154:0/1645871771 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa72c06af60 0x7fa72c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:27.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.980+0000 7fa74e04b640 1 --2- 172.21.15.154:0/1645871771 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa72c06af60 0x7fa72c06d420 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa744001eb0 tx=0x7fa744000f50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:27.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:27.984+0000 7fa73effd640 1 -- 172.21.15.154:0/1645871771 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fa7400756f0 con 0x7fa7480c83d0 2023-09-23T03:25:28.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.173+0000 7fa7502af640 1 -- 172.21.15.154:0/1645871771 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa74804e920 con 0x7fa7480c83d0 2023-09-23T03:25:28.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.174+0000 7fa73effd640 1 -- 172.21.15.154:0/1645871771 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa7400480a0 con 0x7fa7480c83d0 2023-09-23T03:25:28.176 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:25:28.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.177+0000 7fa7502af640 1 -- 172.21.15.154:0/1645871771 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa72c06af60 msgr2=0x7fa72c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:28.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.177+0000 7fa7502af640 1 --2- 172.21.15.154:0/1645871771 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa72c06af60 0x7fa72c06d420 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa744001eb0 tx=0x7fa744000f50 comp rx=0 tx=0).stop 2023-09-23T03:25:28.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.177+0000 7fa7502af640 1 -- 172.21.15.154:0/1645871771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7480c83d0 msgr2=0x7fa748154960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:28.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.177+0000 7fa7502af640 1 --2- 172.21.15.154:0/1645871771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7480c83d0 0x7fa748154960 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fa740001f40 tx=0x7fa7400014a0 comp rx=0 tx=0).stop 2023-09-23T03:25:28.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.177+0000 7fa7502af640 1 -- 172.21.15.154:0/1645871771 shutdown_connections 2023-09-23T03:25:28.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.177+0000 7fa7502af640 1 --2- 172.21.15.154:0/1645871771 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa72c06af60 0x7fa72c06d420 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:28.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.177+0000 7fa7502af640 1 --2- 172.21.15.154:0/1645871771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa7480c83d0 0x7fa748154960 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:28.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.177+0000 7fa7502af640 1 --2- 172.21.15.154:0/1645871771 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa7480ca3f0 0x7fa7480c97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:28.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.177+0000 7fa7502af640 1 --2- 172.21.15.154:0/1645871771 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa734004d00 0x7fa7480b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:28.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.177+0000 7fa7502af640 1 -- 172.21.15.154:0/1645871771 >> 172.21.15.154:0/1645871771 conn(0x7fa74806c010 msgr2=0x7fa74815e360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:28.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.177+0000 7fa7502af640 1 -- 172.21.15.154:0/1645871771 shutdown_connections 2023-09-23T03:25:28.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.178+0000 7fa7502af640 1 -- 172.21.15.154:0/1645871771 wait complete. 2023-09-23T03:25:28.196 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:25:28.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.418+0000 7fa3c56fc640 1 Processor -- start 2023-09-23T03:25:28.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.418+0000 7fa3c56fc640 1 -- start start 2023-09-23T03:25:28.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.420+0000 7fa3c56fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3c00d0980 0x7fa3c00b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:28.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.420+0000 7fa3c56fc640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa3c00d1890 0x7fa3c00cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:28.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.420+0000 7fa3c56fc640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3c015e330 con 0x7fa3c015df50 2023-09-23T03:25:28.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.420+0000 7fa3c56fc640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3c00cd870 con 0x7fa3c00d1890 2023-09-23T03:25:28.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.420+0000 7fa3c56fc640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c00d0e10 con 0x7fa3c00d0980 2023-09-23T03:25:28.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.420+0000 7fa3bffff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fa3c015df50 0x7fa3c00d1180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40996/0 (socket says 172.21.15.154:40996) 2023-09-23T03:25:28.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.420+0000 7fa3bffff640 1 -- 172.21.15.154:0/2441884342 learned_addr learned my addr 172.21.15.154:0/2441884342 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:25:28.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.421+0000 7fa3beffd640 1 -- 172.21.15.154:0/2441884342 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1411111408 0 0) 0x7fa3c015e330 con 0x7fa3c015df50 2023-09-23T03:25:28.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.421+0000 7fa3beffd640 1 -- 172.21.15.154:0/2441884342 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3ac003520 con 0x7fa3c015df50 2023-09-23T03:25:28.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.421+0000 7fa3c4efb640 1 --2- 172.21.15.154:0/2441884342 >> v2:172.21.15.190:3300/0 conn(0x7fa3c00d1890 0x7fa3c00cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:28.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.421+0000 7fa3bf7fe640 1 --2- 172.21.15.154:0/2441884342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3c00d0980 0x7fa3c00b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:28.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.421+0000 7fa3beffd640 1 -- 172.21.15.154:0/2441884342 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3993927764 0 0) 0x7fa3ac003520 con 0x7fa3c015df50 2023-09-23T03:25:28.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.421+0000 7fa3beffd640 1 -- 172.21.15.154:0/2441884342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3c00d0980 msgr2=0x7fa3c00b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:28.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.421+0000 7fa3beffd640 1 --2- 172.21.15.154:0/2441884342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3c00d0980 0x7fa3c00b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:28.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.421+0000 7fa3beffd640 1 -- 172.21.15.154:0/2441884342 >> v2:172.21.15.190:3300/0 conn(0x7fa3c00d1890 msgr2=0x7fa3c00cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:28.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.421+0000 7fa3beffd640 1 --2- 172.21.15.154:0/2441884342 >> v2:172.21.15.190:3300/0 conn(0x7fa3c00d1890 0x7fa3c00cdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:28.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.421+0000 7fa3beffd640 1 -- 172.21.15.154:0/2441884342 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c00b4c50 con 0x7fa3c015df50 2023-09-23T03:25:28.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.421+0000 7fa3beffd640 1 -- 172.21.15.154:0/2441884342 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fa3b00019f0 con 0x7fa3c015df50 2023-09-23T03:25:28.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.421+0000 7fa3beffd640 1 -- 172.21.15.154:0/2441884342 >> v1:172.21.15.154:6789/0 conn(0x7fa3c015df50 legacy=0x7fa3c00d1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:28.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.421+0000 7fa3beffd640 1 --2- 172.21.15.154:0/2441884342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3ac003970 0x7fa3ac003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:28.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.421+0000 7fa3beffd640 1 --2- 172.21.15.154:0/2441884342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3ac0047c0 0x7fa3ac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:28.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.421+0000 7fa3beffd640 1 --2- 172.21.15.154:0/2441884342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3ac005180 0x7fa3ac00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:28.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.421+0000 7fa3beffd640 1 -- 172.21.15.154:0/2441884342 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c00cd870 con 0x7fa3ac0047c0 2023-09-23T03:25:28.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.421+0000 7fa3beffd640 1 -- 172.21.15.154:0/2441884342 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c00d0e10 con 0x7fa3ac003970 2023-09-23T03:25:28.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.421+0000 7fa3beffd640 1 -- 172.21.15.154:0/2441884342 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac00bec0 con 0x7fa3ac005180 2023-09-23T03:25:28.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.421+0000 7fa3bf7fe640 1 --2- 172.21.15.154:0/2441884342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3ac0047c0 0x7fa3ac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:28.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.422+0000 7fa3c4efb640 1 --2- 172.21.15.154:0/2441884342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3ac005180 0x7fa3ac00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:28.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.422+0000 7fa3bffff640 1 --2- 172.21.15.154:0/2441884342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3ac003970 0x7fa3ac003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:28.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.422+0000 7fa3bf7fe640 1 -- 172.21.15.154:0/2441884342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3ac005180 msgr2=0x7fa3ac00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:28.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.422+0000 7fa3bf7fe640 1 --2- 172.21.15.154:0/2441884342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3ac005180 0x7fa3ac00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:28.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.422+0000 7fa3bf7fe640 1 -- 172.21.15.154:0/2441884342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3ac003970 msgr2=0x7fa3ac003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:28.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.422+0000 7fa3bf7fe640 1 --2- 172.21.15.154:0/2441884342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3ac003970 0x7fa3ac003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:28.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.422+0000 7fa3bf7fe640 1 -- 172.21.15.154:0/2441884342 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ac003520 con 0x7fa3ac0047c0 2023-09-23T03:25:28.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.422+0000 7fa3bffff640 1 --2- 172.21.15.154:0/2441884342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3ac003970 0x7fa3ac003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:28.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.423+0000 7fa3bf7fe640 1 --2- 172.21.15.154:0/2441884342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3ac0047c0 0x7fa3ac004c40 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fa3a800ba30 tx=0x7fa3ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f30af734ddc29b7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:28.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.423+0000 7fa3beffd640 1 -- 172.21.15.154:0/2441884342 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a8032640 con 0x7fa3ac0047c0 2023-09-23T03:25:28.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.423+0000 7fa3beffd640 1 -- 172.21.15.154:0/2441884342 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a80327a0 con 0x7fa3ac0047c0 2023-09-23T03:25:28.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.423+0000 7fa3beffd640 1 -- 172.21.15.154:0/2441884342 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a8032b30 con 0x7fa3ac0047c0 2023-09-23T03:25:28.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.424+0000 7fa3c56fc640 1 -- 172.21.15.154:0/2441884342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3ac0047c0 msgr2=0x7fa3ac004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:28.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.424+0000 7fa3c56fc640 1 --2- 172.21.15.154:0/2441884342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3ac0047c0 0x7fa3ac004c40 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fa3a800ba30 tx=0x7fa3ac0057f0 comp rx=0 tx=0).stop 2023-09-23T03:25:28.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.424+0000 7fa3c4efb640 1 -- 172.21.15.154:0/2441884342 reap_dead start 2023-09-23T03:25:28.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.424+0000 7fa3c56fc640 1 -- 172.21.15.154:0/2441884342 shutdown_connections 2023-09-23T03:25:28.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.424+0000 7fa3c56fc640 1 -- 172.21.15.154:0/2441884342 >> 172.21.15.154:0/2441884342 conn(0x7fa3c009bb20 msgr2=0x7fa3c00d2870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:28.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.424+0000 7fa3c56fc640 1 -- 172.21.15.154:0/2441884342 shutdown_connections 2023-09-23T03:25:28.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.424+0000 7fa3c56fc640 1 -- 172.21.15.154:0/2441884342 wait complete. 2023-09-23T03:25:28.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.426+0000 7fa3c56fc640 1 Processor -- start 2023-09-23T03:25:28.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.426+0000 7fa3c56fc640 1 -- start start 2023-09-23T03:25:28.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.427+0000 7fa3c56fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3ac0047c0 0x7fa3c0159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:28.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.427+0000 7fa3c56fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3c00cbf90 0x7fa3c01e6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:28.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.427+0000 7fa3c56fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3c00cc410 0x7fa3c01e6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:28.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.427+0000 7fa3c56fc640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c00c77b0 con 0x7fa3ac0047c0 2023-09-23T03:25:28.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.427+0000 7fa3c56fc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c015bb80 con 0x7fa3c00cbf90 2023-09-23T03:25:28.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.427+0000 7fa3c56fc640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c015b2e0 con 0x7fa3c00cc410 2023-09-23T03:25:28.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.427+0000 7fa3bffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3ac0047c0 0x7fa3c0159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:28.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.427+0000 7fa3bffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3ac0047c0 0x7fa3c0159c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50706/0 (socket says 172.21.15.154:50706) 2023-09-23T03:25:28.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.427+0000 7fa3bffff640 1 -- 172.21.15.154:0/3721805502 learned_addr learned my addr 172.21.15.154:0/3721805502 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:28.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.428+0000 7fa3bffff640 1 -- 172.21.15.154:0/3721805502 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3c00cc410 msgr2=0x7fa3c01e6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:28.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.428+0000 7fa3bffff640 1 --2- 172.21.15.154:0/3721805502 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3c00cc410 0x7fa3c01e6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:28.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.428+0000 7fa3bffff640 1 -- 172.21.15.154:0/3721805502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3c00cbf90 msgr2=0x7fa3c01e6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:28.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.428+0000 7fa3bf7fe640 1 --2- 172.21.15.154:0/3721805502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3c00cbf90 0x7fa3c01e6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:28.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.428+0000 7fa3bffff640 1 --2- 172.21.15.154:0/3721805502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3c00cbf90 0x7fa3c01e6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:28.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.428+0000 7fa3bffff640 1 -- 172.21.15.154:0/3721805502 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c015b5c0 con 0x7fa3ac0047c0 2023-09-23T03:25:28.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.428+0000 7fa3bffff640 1 --2- 172.21.15.154:0/3721805502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3ac0047c0 0x7fa3c0159c50 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fa3b000de60 tx=0x7fa3b000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:28.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.428+0000 7fa3bcff9640 1 -- 172.21.15.154:0/3721805502 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b0004d60 con 0x7fa3ac0047c0 2023-09-23T03:25:28.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.429+0000 7fa3bcff9640 1 -- 172.21.15.154:0/3721805502 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b000be80 con 0x7fa3ac0047c0 2023-09-23T03:25:28.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.429+0000 7fa3bcff9640 1 -- 172.21.15.154:0/3721805502 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b00072f0 con 0x7fa3ac0047c0 2023-09-23T03:25:28.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.429+0000 7fa3c56fc640 1 -- 172.21.15.154:0/3721805502 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c015b8a0 con 0x7fa3ac0047c0 2023-09-23T03:25:28.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.429+0000 7fa3c56fc640 1 -- 172.21.15.154:0/3721805502 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c015a190 con 0x7fa3ac0047c0 2023-09-23T03:25:28.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.430+0000 7fa3c56fc640 1 -- 172.21.15.154:0/3721805502 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3c004e920 con 0x7fa3ac0047c0 2023-09-23T03:25:28.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.431+0000 7fa3bcff9640 1 -- 172.21.15.154:0/3721805502 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fa3b0007510 con 0x7fa3ac0047c0 2023-09-23T03:25:28.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.432+0000 7fa3bcff9640 1 --2- 172.21.15.154:0/3721805502 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa3a006af30 0x7fa3a006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:28.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.432+0000 7fa3bcff9640 1 -- 172.21.15.154:0/3721805502 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fa3b0089b80 con 0x7fa3ac0047c0 2023-09-23T03:25:28.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.432+0000 7fa3bf7fe640 1 --2- 172.21.15.154:0/3721805502 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa3a006af30 0x7fa3a006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:28.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.433+0000 7fa3bf7fe640 1 --2- 172.21.15.154:0/3721805502 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa3a006af30 0x7fa3a006d3f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fa3a800ba00 tx=0x7fa3a803d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:28.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.436+0000 7fa3bcff9640 1 -- 172.21.15.154:0/3721805502 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fa3b00547f0 con 0x7fa3ac0047c0 2023-09-23T03:25:28.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.624+0000 7fa3c56fc640 1 -- 172.21.15.154:0/3721805502 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa3c00b51e0 con 0x7fa3ac0047c0 2023-09-23T03:25:28.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.626+0000 7fa3bcff9640 1 -- 172.21.15.154:0/3721805502 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa3b00190a0 con 0x7fa3ac0047c0 2023-09-23T03:25:28.627 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:25:28.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.628+0000 7fa3c56fc640 1 -- 172.21.15.154:0/3721805502 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa3a006af30 msgr2=0x7fa3a006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:28.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.628+0000 7fa3c56fc640 1 --2- 172.21.15.154:0/3721805502 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa3a006af30 0x7fa3a006d3f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fa3a800ba00 tx=0x7fa3a803d040 comp rx=0 tx=0).stop 2023-09-23T03:25:28.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.628+0000 7fa3c56fc640 1 -- 172.21.15.154:0/3721805502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3ac0047c0 msgr2=0x7fa3c0159c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:28.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.628+0000 7fa3c56fc640 1 --2- 172.21.15.154:0/3721805502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3ac0047c0 0x7fa3c0159c50 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fa3b000de60 tx=0x7fa3b000b4a0 comp rx=0 tx=0).stop 2023-09-23T03:25:28.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.628+0000 7fa3c56fc640 1 -- 172.21.15.154:0/3721805502 shutdown_connections 2023-09-23T03:25:28.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.628+0000 7fa3c56fc640 1 --2- 172.21.15.154:0/3721805502 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa3a006af30 0x7fa3a006d3f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:28.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.628+0000 7fa3c56fc640 1 --2- 172.21.15.154:0/3721805502 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa3c00cc410 0x7fa3c01e6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:28.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.628+0000 7fa3c56fc640 1 --2- 172.21.15.154:0/3721805502 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3c00cbf90 0x7fa3c01e6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:28.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.628+0000 7fa3c56fc640 1 --2- 172.21.15.154:0/3721805502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3ac0047c0 0x7fa3c0159c50 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:28.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.628+0000 7fa3c56fc640 1 -- 172.21.15.154:0/3721805502 >> 172.21.15.154:0/3721805502 conn(0x7fa3c009bb20 msgr2=0x7fa3c00cdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:28.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.628+0000 7fa3c56fc640 1 -- 172.21.15.154:0/3721805502 shutdown_connections 2023-09-23T03:25:28.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.628+0000 7fa3c56fc640 1 -- 172.21.15.154:0/3721805502 wait complete. 2023-09-23T03:25:28.647 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:25:28.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.861+0000 7f3f22007640 1 Processor -- start 2023-09-23T03:25:28.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.861+0000 7f3f22007640 1 -- start start 2023-09-23T03:25:28.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.862+0000 7f3f22007640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3f1c0c9150 0x7f3f1c15cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:28.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.862+0000 7f3f22007640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3f1c071290 0x7f3f1c0ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:28.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.863+0000 7f3f22007640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f1c0c9d20 con 0x7f3f1c0c8d70 2023-09-23T03:25:28.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.863+0000 7f3f22007640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f1c071850 con 0x7f3f1c071290 2023-09-23T03:25:28.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.863+0000 7f3f22007640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f1c071b30 con 0x7f3f1c0c9150 2023-09-23T03:25:28.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.863+0000 7f3f1bfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3f1c071290 0x7f3f1c0ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:28.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.863+0000 7f3f1bfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3f1c071290 0x7f3f1c0ca1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50722/0 (socket says 172.21.15.154:50722) 2023-09-23T03:25:28.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.863+0000 7f3f1bfff640 1 -- 172.21.15.154:0/2311928446 learned_addr learned my addr 172.21.15.154:0/2311928446 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:28.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.863+0000 7f3f13fff640 1 --2- 172.21.15.154:0/2311928446 >> v2:172.21.15.190:3300/0 conn(0x7f3f1c0c9150 0x7f3f1c15cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:28.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.863+0000 7f3f1bfff640 1 -- 172.21.15.154:0/2311928446 >> v2:172.21.15.190:3300/0 conn(0x7f3f1c0c9150 msgr2=0x7f3f1c15cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:28.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.863+0000 7f3f1bfff640 1 --2- 172.21.15.154:0/2311928446 >> v2:172.21.15.190:3300/0 conn(0x7f3f1c0c9150 0x7f3f1c15cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:28.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.863+0000 7f3f1bfff640 1 -- 172.21.15.154:0/2311928446 >> v1:172.21.15.190:6789/0 conn(0x7f3f1c0c8d70 legacy=0x7f3f1c15c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:28.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.863+0000 7f3f1bfff640 1 -- 172.21.15.154:0/2311928446 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1c071e10 con 0x7f3f1c071290 2023-09-23T03:25:28.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.864+0000 7f3f1bfff640 1 --2- 172.21.15.154:0/2311928446 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f1c071290 0x7f3f1c0ca1b0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c0b3ce0 tx=0x7f3f1402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6fb12743f087098 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:28.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.865+0000 7f3f1affd640 1 -- 172.21.15.154:0/2311928446 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1402ca80 con 0x7f3f1c071290 2023-09-23T03:25:28.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.865+0000 7f3f1affd640 1 -- 172.21.15.154:0/2311928446 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f1402cbe0 con 0x7f3f1c071290 2023-09-23T03:25:28.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.865+0000 7f3f1affd640 1 -- 172.21.15.154:0/2311928446 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f140343f0 con 0x7f3f1c071290 2023-09-23T03:25:28.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.865+0000 7f3f22007640 1 -- 172.21.15.154:0/2311928446 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f1c071290 msgr2=0x7f3f1c0ca1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:28.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.865+0000 7f3f22007640 1 --2- 172.21.15.154:0/2311928446 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f1c071290 0x7f3f1c0ca1b0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c0b3ce0 tx=0x7f3f1402bee0 comp rx=0 tx=0).stop 2023-09-23T03:25:28.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.866+0000 7f3f22007640 1 -- 172.21.15.154:0/2311928446 shutdown_connections 2023-09-23T03:25:28.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.866+0000 7f3f22007640 1 --2- 172.21.15.154:0/2311928446 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f1c071290 0x7f3f1c0ca1b0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:28.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.866+0000 7f3f22007640 1 --2- 172.21.15.154:0/2311928446 >> v2:172.21.15.190:3300/0 conn(0x7f3f1c0c9150 0x7f3f1c15cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:28.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.866+0000 7f3f22007640 1 -- 172.21.15.154:0/2311928446 >> 172.21.15.154:0/2311928446 conn(0x7f3f1c06c010 msgr2=0x7f3f1c06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:28.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.866+0000 7f3f22007640 1 -- 172.21.15.154:0/2311928446 shutdown_connections 2023-09-23T03:25:28.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.866+0000 7f3f22007640 1 -- 172.21.15.154:0/2311928446 wait complete. 2023-09-23T03:25:28.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.867+0000 7f3f22007640 1 Processor -- start 2023-09-23T03:25:28.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.868+0000 7f3f22007640 1 -- start start 2023-09-23T03:25:28.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.869+0000 7f3f22007640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f1c071290 0x7f3f1c0c4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:28.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.869+0000 7f3f22007640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f1c0c8d70 0x7f3f1c158850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:28.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.869+0000 7f3f22007640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f1c0c9150 0x7f3f1c158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:28.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.869+0000 7f3f22007640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c1521c0 con 0x7f3f1c0c9150 2023-09-23T03:25:28.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.869+0000 7f3f22007640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c0c6a60 con 0x7f3f1c071290 2023-09-23T03:25:28.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.869+0000 7f3f22007640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c0c4d50 con 0x7f3f1c0c8d70 2023-09-23T03:25:28.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.869+0000 7f3f1bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f1c0c9150 0x7f3f1c158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:28.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.869+0000 7f3f1bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f1c0c9150 0x7f3f1c158d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50726/0 (socket says 172.21.15.154:50726) 2023-09-23T03:25:28.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.869+0000 7f3f1bfff640 1 -- 172.21.15.154:0/43022555 learned_addr learned my addr 172.21.15.154:0/43022555 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:28.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.869+0000 7f3f1b7fe640 1 --2- 172.21.15.154:0/43022555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f1c071290 0x7f3f1c0c4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:28.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.869+0000 7f3f13fff640 1 --2- 172.21.15.154:0/43022555 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f1c0c8d70 0x7f3f1c158850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:28.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.869+0000 7f3f1bfff640 1 -- 172.21.15.154:0/43022555 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f1c0c8d70 msgr2=0x7f3f1c158850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:28.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.869+0000 7f3f1bfff640 1 --2- 172.21.15.154:0/43022555 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f1c0c8d70 0x7f3f1c158850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:28.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.869+0000 7f3f1bfff640 1 -- 172.21.15.154:0/43022555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f1c071290 msgr2=0x7f3f1c0c4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:28.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.869+0000 7f3f1bfff640 1 --2- 172.21.15.154:0/43022555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f1c071290 0x7f3f1c0c4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:28.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.869+0000 7f3f1bfff640 1 -- 172.21.15.154:0/43022555 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f14009b30 con 0x7f3f1c0c9150 2023-09-23T03:25:28.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.869+0000 7f3f1b7fe640 1 --2- 172.21.15.154:0/43022555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f1c071290 0x7f3f1c0c4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:28.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.870+0000 7f3f1bfff640 1 --2- 172.21.15.154:0/43022555 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f1c0c9150 0x7f3f1c158d90 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f3f14036040 tx=0x7f3f14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:28.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.870+0000 7f3f18ff9640 1 -- 172.21.15.154:0/43022555 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f14009510 con 0x7f3f1c0c9150 2023-09-23T03:25:28.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.870+0000 7f3f18ff9640 1 -- 172.21.15.154:0/43022555 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f1403e450 con 0x7f3f1c0c9150 2023-09-23T03:25:28.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.870+0000 7f3f18ff9640 1 -- 172.21.15.154:0/43022555 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1403e7e0 con 0x7f3f1c0c9150 2023-09-23T03:25:28.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.870+0000 7f3f22007640 1 -- 172.21.15.154:0/43022555 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f1c0c75a0 con 0x7f3f1c0c9150 2023-09-23T03:25:28.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.870+0000 7f3f22007640 1 -- 172.21.15.154:0/43022555 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f1c0c6f50 con 0x7f3f1c0c9150 2023-09-23T03:25:28.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.872+0000 7f3f22007640 1 -- 172.21.15.154:0/43022555 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f1c0c7730 con 0x7f3f1c0c9150 2023-09-23T03:25:28.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.872+0000 7f3f18ff9640 1 -- 172.21.15.154:0/43022555 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f3f1402cd50 con 0x7f3f1c0c9150 2023-09-23T03:25:28.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.874+0000 7f3f18ff9640 1 --2- 172.21.15.154:0/43022555 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3efc06af30 0x7f3efc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:28.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.874+0000 7f3f18ff9640 1 -- 172.21.15.154:0/43022555 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f3f140ac5a0 con 0x7f3f1c0c9150 2023-09-23T03:25:28.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.874+0000 7f3f1b7fe640 1 --2- 172.21.15.154:0/43022555 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3efc06af30 0x7f3efc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:28.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.875+0000 7f3f1b7fe640 1 --2- 172.21.15.154:0/43022555 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3efc06af30 0x7f3efc06d3f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c003eb0 tx=0x7f3f0c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:28.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:28.878+0000 7f3f18ff9640 1 -- 172.21.15.154:0/43022555 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f3f140773b0 con 0x7f3f1c0c9150 2023-09-23T03:25:29.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.066+0000 7f3f22007640 1 -- 172.21.15.154:0/43022555 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3f1c04e920 con 0x7f3f1c0c9150 2023-09-23T03:25:29.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.067+0000 7f3f18ff9640 1 -- 172.21.15.154:0/43022555 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3f1407a9d0 con 0x7f3f1c0c9150 2023-09-23T03:25:29.068 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:25:29.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.071+0000 7f3f22007640 1 -- 172.21.15.154:0/43022555 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3efc06af30 msgr2=0x7f3efc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:29.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.071+0000 7f3f22007640 1 --2- 172.21.15.154:0/43022555 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3efc06af30 0x7f3efc06d3f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c003eb0 tx=0x7f3f0c009040 comp rx=0 tx=0).stop 2023-09-23T03:25:29.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.071+0000 7f3f22007640 1 -- 172.21.15.154:0/43022555 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f1c0c9150 msgr2=0x7f3f1c158d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:29.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.071+0000 7f3f22007640 1 --2- 172.21.15.154:0/43022555 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f1c0c9150 0x7f3f1c158d90 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f3f14036040 tx=0x7f3f14004a40 comp rx=0 tx=0).stop 2023-09-23T03:25:29.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.071+0000 7f3f22007640 1 -- 172.21.15.154:0/43022555 shutdown_connections 2023-09-23T03:25:29.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.071+0000 7f3f22007640 1 --2- 172.21.15.154:0/43022555 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3efc06af30 0x7f3efc06d3f0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:29.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.071+0000 7f3f22007640 1 --2- 172.21.15.154:0/43022555 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3f1c0c9150 0x7f3f1c158d90 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:29.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.071+0000 7f3f22007640 1 --2- 172.21.15.154:0/43022555 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3f1c0c8d70 0x7f3f1c158850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:29.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.071+0000 7f3f22007640 1 --2- 172.21.15.154:0/43022555 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3f1c071290 0x7f3f1c0c4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:29.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.071+0000 7f3f22007640 1 -- 172.21.15.154:0/43022555 >> 172.21.15.154:0/43022555 conn(0x7f3f1c06c010 msgr2=0x7f3f1c153c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:29.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.071+0000 7f3f22007640 1 -- 172.21.15.154:0/43022555 shutdown_connections 2023-09-23T03:25:29.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.071+0000 7f3f22007640 1 -- 172.21.15.154:0/43022555 wait complete. 2023-09-23T03:25:29.091 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-09-23T03:25:29.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.304+0000 7fd80bfa5640 1 Processor -- start 2023-09-23T03:25:29.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.304+0000 7fd80bfa5640 1 -- start start 2023-09-23T03:25:29.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.306+0000 7fd80bfa5640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd804070c60 0x7fd8040caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:29.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.306+0000 7fd80bfa5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd8040cb400 0x7fd8040aba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:29.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.306+0000 7fd80bfa5640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8040712f0 con 0x7fd8040ad830 2023-09-23T03:25:29.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.306+0000 7fd80bfa5640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8040710f0 con 0x7fd8040cb400 2023-09-23T03:25:29.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.306+0000 7fd80bfa5640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8040cb870 con 0x7fd804070c60 2023-09-23T03:25:29.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.306+0000 7fd80a542640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd8040cb400 0x7fd8040aba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:29.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.306+0000 7fd80a542640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd8040cb400 0x7fd8040aba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50732/0 (socket says 172.21.15.154:50732) 2023-09-23T03:25:29.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.306+0000 7fd80a542640 1 -- 172.21.15.154:0/4101394717 learned_addr learned my addr 172.21.15.154:0/4101394717 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:29.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.306+0000 7fd809540640 1 --2- 172.21.15.154:0/4101394717 >> v2:172.21.15.190:3300/0 conn(0x7fd804070c60 0x7fd8040caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:29.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.307+0000 7fd80a542640 1 -- 172.21.15.154:0/4101394717 >> v2:172.21.15.190:3300/0 conn(0x7fd804070c60 msgr2=0x7fd8040caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:29.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.307+0000 7fd80a542640 1 --2- 172.21.15.154:0/4101394717 >> v2:172.21.15.190:3300/0 conn(0x7fd804070c60 0x7fd8040caec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:29.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.307+0000 7fd80a542640 1 -- 172.21.15.154:0/4101394717 >> v1:172.21.15.190:6789/0 conn(0x7fd8040ad830 legacy=0x7fd8040c9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:29.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.307+0000 7fd80a542640 1 -- 172.21.15.154:0/4101394717 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8040715d0 con 0x7fd8040cb400 2023-09-23T03:25:29.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.308+0000 7fd80a542640 1 --2- 172.21.15.154:0/4101394717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8040cb400 0x7fd8040aba70 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fd8040cbb10 tx=0x7fd7fc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee114d33ab8571b8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:29.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.308+0000 7fd808d3f640 1 -- 172.21.15.154:0/4101394717 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc02ca80 con 0x7fd8040cb400 2023-09-23T03:25:29.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.308+0000 7fd808d3f640 1 -- 172.21.15.154:0/4101394717 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7fc02cbe0 con 0x7fd8040cb400 2023-09-23T03:25:29.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.308+0000 7fd808d3f640 1 -- 172.21.15.154:0/4101394717 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc0343f0 con 0x7fd8040cb400 2023-09-23T03:25:29.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.309+0000 7fd80bfa5640 1 -- 172.21.15.154:0/4101394717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8040cb400 msgr2=0x7fd8040aba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:29.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.309+0000 7fd80bfa5640 1 --2- 172.21.15.154:0/4101394717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8040cb400 0x7fd8040aba70 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fd8040cbb10 tx=0x7fd7fc02bee0 comp rx=0 tx=0).stop 2023-09-23T03:25:29.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.309+0000 7fd80bfa5640 1 -- 172.21.15.154:0/4101394717 shutdown_connections 2023-09-23T03:25:29.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.309+0000 7fd80bfa5640 1 --2- 172.21.15.154:0/4101394717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8040cb400 0x7fd8040aba70 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:29.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.309+0000 7fd80bfa5640 1 --2- 172.21.15.154:0/4101394717 >> v2:172.21.15.190:3300/0 conn(0x7fd804070c60 0x7fd8040caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:29.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.309+0000 7fd80bfa5640 1 -- 172.21.15.154:0/4101394717 >> 172.21.15.154:0/4101394717 conn(0x7fd80406c010 msgr2=0x7fd80406c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:29.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.309+0000 7fd80bfa5640 1 -- 172.21.15.154:0/4101394717 shutdown_connections 2023-09-23T03:25:29.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.309+0000 7fd80bfa5640 1 -- 172.21.15.154:0/4101394717 wait complete. 2023-09-23T03:25:29.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.311+0000 7fd80bfa5640 1 Processor -- start 2023-09-23T03:25:29.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.311+0000 7fd80bfa5640 1 -- start start 2023-09-23T03:25:29.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.312+0000 7fd80bfa5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd804070c60 0x7fd8040c33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:29.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.312+0000 7fd80bfa5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8040ad830 0x7fd80415d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:29.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.312+0000 7fd80bfa5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8040cb400 0x7fd8040c7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:29.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.312+0000 7fd80bfa5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8040c4ac0 con 0x7fd804070c60 2023-09-23T03:25:29.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.312+0000 7fd80bfa5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8040b40c0 con 0x7fd8040cb400 2023-09-23T03:25:29.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.312+0000 7fd80bfa5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8040c3a40 con 0x7fd8040ad830 2023-09-23T03:25:29.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.312+0000 7fd809d41640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd804070c60 0x7fd8040c33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:29.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.313+0000 7fd809d41640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd804070c60 0x7fd8040c33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50738/0 (socket says 172.21.15.154:50738) 2023-09-23T03:25:29.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.313+0000 7fd809d41640 1 -- 172.21.15.154:0/1165975277 learned_addr learned my addr 172.21.15.154:0/1165975277 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:29.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.313+0000 7fd809d41640 1 -- 172.21.15.154:0/1165975277 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8040ad830 msgr2=0x7fd80415d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:29.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.313+0000 7fd809d41640 1 --2- 172.21.15.154:0/1165975277 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8040ad830 0x7fd80415d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:29.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.313+0000 7fd809d41640 1 -- 172.21.15.154:0/1165975277 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8040cb400 msgr2=0x7fd8040c7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:29.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.313+0000 7fd809d41640 1 --2- 172.21.15.154:0/1165975277 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8040cb400 0x7fd8040c7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:29.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.313+0000 7fd809d41640 1 -- 172.21.15.154:0/1165975277 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7fc009b30 con 0x7fd804070c60 2023-09-23T03:25:29.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.313+0000 7fd809d41640 1 --2- 172.21.15.154:0/1165975277 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd804070c60 0x7fd8040c33d0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fd80000b3f0 tx=0x7fd80000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:29.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.313+0000 7fd7fa7fc640 1 -- 172.21.15.154:0/1165975277 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd800011830 con 0x7fd804070c60 2023-09-23T03:25:29.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.314+0000 7fd7fa7fc640 1 -- 172.21.15.154:0/1165975277 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd800011990 con 0x7fd804070c60 2023-09-23T03:25:29.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.314+0000 7fd7fa7fc640 1 -- 172.21.15.154:0/1165975277 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd800011d40 con 0x7fd804070c60 2023-09-23T03:25:29.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.314+0000 7fd80bfa5640 1 -- 172.21.15.154:0/1165975277 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8040c3d20 con 0x7fd804070c60 2023-09-23T03:25:29.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.314+0000 7fd80bfa5640 1 -- 172.21.15.154:0/1165975277 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8040c42e0 con 0x7fd804070c60 2023-09-23T03:25:29.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.315+0000 7fd80bfa5640 1 -- 172.21.15.154:0/1165975277 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd80404e920 con 0x7fd804070c60 2023-09-23T03:25:29.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.316+0000 7fd7fa7fc640 1 -- 172.21.15.154:0/1165975277 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fd80001a400 con 0x7fd804070c60 2023-09-23T03:25:29.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.316+0000 7fd7fa7fc640 1 --2- 172.21.15.154:0/1165975277 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd7e806af30 0x7fd7e806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:29.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.317+0000 7fd7fa7fc640 1 -- 172.21.15.154:0/1165975277 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fd800089980 con 0x7fd804070c60 2023-09-23T03:25:29.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.317+0000 7fd809540640 1 --2- 172.21.15.154:0/1165975277 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd7e806af30 0x7fd7e806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:29.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.318+0000 7fd809540640 1 --2- 172.21.15.154:0/1165975277 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd7e806af30 0x7fd7e806d3f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4006fd0 tx=0x7fd7f4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:29.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.321+0000 7fd7fa7fc640 1 -- 172.21.15.154:0/1165975277 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fd8000545d0 con 0x7fd804070c60 2023-09-23T03:25:29.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.516+0000 7fd80bfa5640 1 -- 172.21.15.154:0/1165975277 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7fd8040ad050 con 0x7fd804070c60 2023-09-23T03:25:29.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.517+0000 7fd7fa7fc640 1 -- 172.21.15.154:0/1165975277 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v92) v1 ==== 106+0+11 (secure 0 0 0) 0x7fd800057bf0 con 0x7fd804070c60 2023-09-23T03:25:29.518 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:25:29.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.519+0000 7fd80bfa5640 1 -- 172.21.15.154:0/1165975277 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd7e806af30 msgr2=0x7fd7e806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:29.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.519+0000 7fd80bfa5640 1 --2- 172.21.15.154:0/1165975277 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd7e806af30 0x7fd7e806d3f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4006fd0 tx=0x7fd7f4008040 comp rx=0 tx=0).stop 2023-09-23T03:25:29.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.519+0000 7fd80bfa5640 1 -- 172.21.15.154:0/1165975277 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd804070c60 msgr2=0x7fd8040c33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:29.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.519+0000 7fd80bfa5640 1 --2- 172.21.15.154:0/1165975277 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd804070c60 0x7fd8040c33d0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fd80000b3f0 tx=0x7fd80000b8b0 comp rx=0 tx=0).stop 2023-09-23T03:25:29.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.519+0000 7fd80bfa5640 1 -- 172.21.15.154:0/1165975277 shutdown_connections 2023-09-23T03:25:29.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.519+0000 7fd80bfa5640 1 --2- 172.21.15.154:0/1165975277 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd7e806af30 0x7fd7e806d3f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:29.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.519+0000 7fd80bfa5640 1 --2- 172.21.15.154:0/1165975277 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8040cb400 0x7fd8040c7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:29.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.519+0000 7fd80bfa5640 1 --2- 172.21.15.154:0/1165975277 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8040ad830 0x7fd80415d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:29.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.519+0000 7fd80bfa5640 1 --2- 172.21.15.154:0/1165975277 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd804070c60 0x7fd8040c33d0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:29.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.519+0000 7fd80bfa5640 1 -- 172.21.15.154:0/1165975277 >> 172.21.15.154:0/1165975277 conn(0x7fd80406c010 msgr2=0x7fd80415e690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:29.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.520+0000 7fd80bfa5640 1 -- 172.21.15.154:0/1165975277 shutdown_connections 2023-09-23T03:25:29.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.520+0000 7fd80bfa5640 1 -- 172.21.15.154:0/1165975277 wait complete. 2023-09-23T03:25:29.539 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:25:29.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.758+0000 7f7208c64640 1 Processor -- start 2023-09-23T03:25:29.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.759+0000 7f7208c64640 1 -- start start 2023-09-23T03:25:29.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.760+0000 7f7208c64640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72040cf040 0x7f72041656e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:29.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.760+0000 7f7208c64640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f720415ac30 con 0x7f72040acce0 2023-09-23T03:25:29.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.760+0000 7f7208c64640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72040b1f70 con 0x7f72040b1b20 2023-09-23T03:25:29.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.760+0000 7f7208c64640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72040b2e90 con 0x7f72040cf040 2023-09-23T03:25:29.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.760+0000 7f720259c640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f72040acce0 0x7f720415ed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:41008/0 (socket says 172.21.15.154:41008) 2023-09-23T03:25:29.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.760+0000 7f720259c640 1 -- 172.21.15.154:0/2041909965 learned_addr learned my addr 172.21.15.154:0/2041909965 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:25:29.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.761+0000 7f7202d9d640 1 --2- 172.21.15.154:0/2041909965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72040cf040 0x7f72041656e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:29.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.761+0000 7f720159a640 1 -- 172.21.15.154:0/2041909965 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3741483442 0 0) 0x7f71f4001610 con 0x7f72040acce0 2023-09-23T03:25:29.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.761+0000 7f720159a640 1 -- 172.21.15.154:0/2041909965 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71f0003520 con 0x7f72040acce0 2023-09-23T03:25:29.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.761+0000 7f720159a640 1 -- 172.21.15.154:0/2041909965 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2194928802 0 0) 0x7f71f0003520 con 0x7f72040acce0 2023-09-23T03:25:29.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.761+0000 7f720159a640 1 -- 172.21.15.154:0/2041909965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72040cf040 msgr2=0x7f72041656e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:29.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.761+0000 7f720159a640 1 --2- 172.21.15.154:0/2041909965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72040cf040 0x7f72041656e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:29.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.761+0000 7f720159a640 1 -- 172.21.15.154:0/2041909965 >> v1:172.21.15.190:6789/0 conn(0x7f72040b1b20 legacy=0x7f72040ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:29.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.761+0000 7f720159a640 1 -- 172.21.15.154:0/2041909965 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72040b2210 con 0x7f72040acce0 2023-09-23T03:25:29.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.761+0000 7f720159a640 1 -- 172.21.15.154:0/2041909965 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f71f4001d00 con 0x7f72040acce0 2023-09-23T03:25:29.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.761+0000 7f720159a640 1 -- 172.21.15.154:0/2041909965 >> v1:172.21.15.154:6789/0 conn(0x7f72040acce0 legacy=0x7f720415ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:29.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.761+0000 7f720159a640 1 --2- 172.21.15.154:0/2041909965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72040cf040 0x7f71f00039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:29.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.762+0000 7f720159a640 1 --2- 172.21.15.154:0/2041909965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71f0003f70 0x7f71f00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:29.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.762+0000 7f720159a640 1 --2- 172.21.15.154:0/2041909965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71f0004d00 0x7f71f0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:29.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.762+0000 7f720159a640 1 -- 172.21.15.154:0/2041909965 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72040b2e90 con 0x7f71f0004d00 2023-09-23T03:25:29.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.762+0000 7f720159a640 1 -- 172.21.15.154:0/2041909965 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f000bb70 con 0x7f71f0003f70 2023-09-23T03:25:29.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.762+0000 7f720159a640 1 -- 172.21.15.154:0/2041909965 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f000bd30 con 0x7f72040cf040 2023-09-23T03:25:29.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.762+0000 7f7202d9d640 1 --2- 172.21.15.154:0/2041909965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71f0004d00 0x7f71f0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:29.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.762+0000 7f720259c640 1 --2- 172.21.15.154:0/2041909965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72040cf040 0x7f71f00039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:29.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.762+0000 7f7201d9b640 1 --2- 172.21.15.154:0/2041909965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71f0003f70 0x7f71f00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:29.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.762+0000 7f7202d9d640 1 -- 172.21.15.154:0/2041909965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72040cf040 msgr2=0x7f71f00039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:29.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.762+0000 7f7202d9d640 1 --2- 172.21.15.154:0/2041909965 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f72040cf040 0x7f71f00039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:29.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.762+0000 7f7202d9d640 1 -- 172.21.15.154:0/2041909965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71f0003f70 msgr2=0x7f71f00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:29.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.762+0000 7f7202d9d640 1 --2- 172.21.15.154:0/2041909965 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f71f0003f70 0x7f71f00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:29.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.762+0000 7f7202d9d640 1 -- 172.21.15.154:0/2041909965 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f0003520 con 0x7f71f0004d00 2023-09-23T03:25:29.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.763+0000 7f7202d9d640 1 --2- 172.21.15.154:0/2041909965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71f0004d00 0x7f71f0005180 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009ac0 tx=0x7f71f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c10d296b8a5e5c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:29.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.764+0000 7f720159a640 1 -- 172.21.15.154:0/2041909965 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc02c6b0 con 0x7f71f0004d00 2023-09-23T03:25:29.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.764+0000 7f720159a640 1 -- 172.21.15.154:0/2041909965 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71fc02c810 con 0x7f71f0004d00 2023-09-23T03:25:29.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.764+0000 7f720159a640 1 -- 172.21.15.154:0/2041909965 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc02cba0 con 0x7f71f0004d00 2023-09-23T03:25:29.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.764+0000 7f7208c64640 1 -- 172.21.15.154:0/2041909965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71f0004d00 msgr2=0x7f71f0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:29.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.764+0000 7f7208c64640 1 --2- 172.21.15.154:0/2041909965 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f71f0004d00 0x7f71f0005180 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009ac0 tx=0x7f71f00057f0 comp rx=0 tx=0).stop 2023-09-23T03:25:29.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.764+0000 7f7202d9d640 1 -- 172.21.15.154:0/2041909965 reap_dead start 2023-09-23T03:25:29.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.764+0000 7f7208c64640 1 -- 172.21.15.154:0/2041909965 shutdown_connections 2023-09-23T03:25:29.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.764+0000 7f7208c64640 1 -- 172.21.15.154:0/2041909965 >> 172.21.15.154:0/2041909965 conn(0x7f720406a0f0 msgr2=0x7f720406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:29.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.764+0000 7f7208c64640 1 -- 172.21.15.154:0/2041909965 shutdown_connections 2023-09-23T03:25:29.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.764+0000 7f7208c64640 1 -- 172.21.15.154:0/2041909965 wait complete. 2023-09-23T03:25:29.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.766+0000 7f7208c64640 1 Processor -- start 2023-09-23T03:25:29.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.766+0000 7f7208c64640 1 -- start start 2023-09-23T03:25:29.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.767+0000 7f7208c64640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71f0004d00 0x7f72041f5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:29.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.767+0000 7f7208c64640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72040af490 0x7f72040c4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:29.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.767+0000 7f7208c64640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72041584b0 0x7f72040c4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:29.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.767+0000 7f7208c64640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72040c22f0 con 0x7f72040af490 2023-09-23T03:25:29.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.767+0000 7f7208c64640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72040c4da0 con 0x7f72041584b0 2023-09-23T03:25:29.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.767+0000 7f7208c64640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7204071ef0 con 0x7f71f0004d00 2023-09-23T03:25:29.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.768+0000 7f7201d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72040af490 0x7f72040c4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:29.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.768+0000 7f7201d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72040af490 0x7f72040c4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50768/0 (socket says 172.21.15.154:50768) 2023-09-23T03:25:29.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.768+0000 7f7201d9b640 1 -- 172.21.15.154:0/2215976929 learned_addr learned my addr 172.21.15.154:0/2215976929 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:29.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.768+0000 7f720259c640 1 --2- 172.21.15.154:0/2215976929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71f0004d00 0x7f72041f5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:29.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.768+0000 7f7202d9d640 1 --2- 172.21.15.154:0/2215976929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72041584b0 0x7f72040c4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:29.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.768+0000 7f7201d9b640 1 -- 172.21.15.154:0/2215976929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71f0004d00 msgr2=0x7f72041f5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:29.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.768+0000 7f7201d9b640 1 --2- 172.21.15.154:0/2215976929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71f0004d00 0x7f72041f5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:29.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.768+0000 7f7201d9b640 1 -- 172.21.15.154:0/2215976929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72041584b0 msgr2=0x7f72040c4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:29.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.768+0000 7f7201d9b640 1 --2- 172.21.15.154:0/2215976929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72041584b0 0x7f72040c4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:29.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.768+0000 7f7201d9b640 1 -- 172.21.15.154:0/2215976929 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72040af990 con 0x7f72040af490 2023-09-23T03:25:29.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.768+0000 7f7201d9b640 1 --2- 172.21.15.154:0/2215976929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72040af490 0x7f72040c4320 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f71ec00e490 tx=0x7f71ec00e950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:29.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.768+0000 7f71faffd640 1 -- 172.21.15.154:0/2215976929 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ec0052e0 con 0x7f72040af490 2023-09-23T03:25:29.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.769+0000 7f71faffd640 1 -- 172.21.15.154:0/2215976929 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71ec005440 con 0x7f72040af490 2023-09-23T03:25:29.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.769+0000 7f71faffd640 1 -- 172.21.15.154:0/2215976929 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ec003830 con 0x7f72040af490 2023-09-23T03:25:29.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.769+0000 7f7208c64640 1 -- 172.21.15.154:0/2215976929 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72040c2540 con 0x7f72040af490 2023-09-23T03:25:29.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.769+0000 7f7208c64640 1 -- 172.21.15.154:0/2215976929 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72040c2b00 con 0x7f72040af490 2023-09-23T03:25:29.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.770+0000 7f7208c64640 1 -- 172.21.15.154:0/2215976929 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72040c2c90 con 0x7f72040af490 2023-09-23T03:25:29.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.771+0000 7f71faffd640 1 -- 172.21.15.154:0/2215976929 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f71ec003990 con 0x7f72040af490 2023-09-23T03:25:29.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.772+0000 7f71faffd640 1 --2- 172.21.15.154:0/2215976929 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f71e406af60 0x7f71e406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:29.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.772+0000 7f71faffd640 1 -- 172.21.15.154:0/2215976929 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f71ec089220 con 0x7f72040af490 2023-09-23T03:25:29.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.772+0000 7f720259c640 1 --2- 172.21.15.154:0/2215976929 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f71e406af60 0x7f71e406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:29.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.773+0000 7f720259c640 1 --2- 172.21.15.154:0/2215976929 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f71e406af60 0x7f71e406d420 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f71f4007d80 tx=0x7f71f4001670 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:29.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:29.776+0000 7f71faffd640 1 -- 172.21.15.154:0/2215976929 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f71ec053fc0 con 0x7f72040af490 2023-09-23T03:25:30.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.003+0000 7f7208c64640 1 -- 172.21.15.154:0/2215976929 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7204073eb0 con 0x7f72040af490 2023-09-23T03:25:30.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.005+0000 7f71faffd640 1 -- 172.21.15.154:0/2215976929 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4925 (secure 0 0 0) 0x7f71ec0575e0 con 0x7f72040af490 2023-09-23T03:25:30.007 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:25:30.007 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":76,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7743,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/31599836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":31599836},{"type":"v1","addr":"172.21.15.154:6836","nonce":31599836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":7752,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/4266451911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":4266451911},{"type":"v1","addr":"172.21.15.154:6837","nonce":4266451911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":7755,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3029600822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3029600822},{"type":"v1","addr":"172.21.15.190:6834","nonce":3029600822}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:24:37.386034+0000","modified":"2023-09-23T03:24:47.439439+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":90,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7758},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7758":{"gid":7758,"name":"b","rank":0,"incarnation":73,"state":"up:active","state_seq":4,"addr":"172.21.15.190:6835/2202597679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2202597679},{"type":"v1","addr":"172.21.15.190:6835","nonce":2202597679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12,13],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-09-23T03:25:30.007 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 76 2023-09-23T03:25:30.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.007+0000 7f7208c64640 1 -- 172.21.15.154:0/2215976929 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f71e406af60 msgr2=0x7f71e406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:30.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.007+0000 7f7208c64640 1 --2- 172.21.15.154:0/2215976929 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f71e406af60 0x7f71e406d420 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f71f4007d80 tx=0x7f71f4001670 comp rx=0 tx=0).stop 2023-09-23T03:25:30.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.007+0000 7f7208c64640 1 -- 172.21.15.154:0/2215976929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72040af490 msgr2=0x7f72040c4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:30.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.007+0000 7f7208c64640 1 --2- 172.21.15.154:0/2215976929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72040af490 0x7f72040c4320 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f71ec00e490 tx=0x7f71ec00e950 comp rx=0 tx=0).stop 2023-09-23T03:25:30.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.008+0000 7f7208c64640 1 -- 172.21.15.154:0/2215976929 shutdown_connections 2023-09-23T03:25:30.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.008+0000 7f7208c64640 1 --2- 172.21.15.154:0/2215976929 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f71e406af60 0x7f71e406d420 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:30.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.008+0000 7f7208c64640 1 --2- 172.21.15.154:0/2215976929 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f72041584b0 0x7f72040c4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:30.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.008+0000 7f7208c64640 1 --2- 172.21.15.154:0/2215976929 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f72040af490 0x7f72040c4320 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:30.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.008+0000 7f7208c64640 1 --2- 172.21.15.154:0/2215976929 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f71f0004d00 0x7f72041f5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:30.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.008+0000 7f7208c64640 1 -- 172.21.15.154:0/2215976929 >> 172.21.15.154:0/2215976929 conn(0x7f720406a0f0 msgr2=0x7f72040c9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:30.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.008+0000 7f7208c64640 1 -- 172.21.15.154:0/2215976929 shutdown_connections 2023-09-23T03:25:30.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.008+0000 7f7208c64640 1 -- 172.21.15.154:0/2215976929 wait complete. 2023-09-23T03:25:30.027 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:25:30.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.236+0000 7f992a41a640 1 Processor -- start 2023-09-23T03:25:30.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.236+0000 7f992a41a640 1 -- start start 2023-09-23T03:25:30.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.238+0000 7f992a41a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f992415c840 0x7f99240cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:30.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.238+0000 7f992a41a640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99240cc910 con 0x7f99240cacd0 2023-09-23T03:25:30.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.238+0000 7f992a41a640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99240c89b0 con 0x7f9924160fd0 2023-09-23T03:25:30.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.238+0000 7f992a41a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f992415cc20 con 0x7f992415c840 2023-09-23T03:25:30.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.238+0000 7f9929418640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f992415c840 0x7f99240cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:30.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.238+0000 7f9929c19640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f99240cacd0 0x7f99240ab8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:41020/0 (socket says 172.21.15.154:41020) 2023-09-23T03:25:30.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.238+0000 7f9929c19640 1 -- 172.21.15.154:0/2048051451 learned_addr learned my addr 172.21.15.154:0/2048051451 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:25:30.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.239+0000 7f9929418640 1 -- 172.21.15.154:0/2048051451 >> v1:172.21.15.190:6789/0 conn(0x7f9924160fd0 legacy=0x7f99240c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:30.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.239+0000 7f9929418640 1 -- 172.21.15.154:0/2048051451 >> v1:172.21.15.154:6789/0 conn(0x7f99240cacd0 legacy=0x7f99240ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:30.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.239+0000 7f9929418640 1 -- 172.21.15.154:0/2048051451 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99240c8c90 con 0x7f992415c840 2023-09-23T03:25:30.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.240+0000 7f9929418640 1 --2- 172.21.15.154:0/2048051451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f992415c840 0x7f99240cbfe0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f9920009d90 tx=0x7f992002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a972c8c0b97c15 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:30.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.241+0000 7f991bfff640 1 -- 172.21.15.154:0/2048051451 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992002cc60 con 0x7f992415c840 2023-09-23T03:25:30.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.241+0000 7f991bfff640 1 -- 172.21.15.154:0/2048051451 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992002cdc0 con 0x7f992415c840 2023-09-23T03:25:30.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.241+0000 7f991bfff640 1 -- 172.21.15.154:0/2048051451 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9920034660 con 0x7f992415c840 2023-09-23T03:25:30.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.241+0000 7f992a41a640 1 -- 172.21.15.154:0/2048051451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f992415c840 msgr2=0x7f99240cbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:30.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.241+0000 7f992a41a640 1 --2- 172.21.15.154:0/2048051451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f992415c840 0x7f99240cbfe0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f9920009d90 tx=0x7f992002beb0 comp rx=0 tx=0).stop 2023-09-23T03:25:30.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.241+0000 7f992a41a640 1 -- 172.21.15.154:0/2048051451 shutdown_connections 2023-09-23T03:25:30.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.241+0000 7f992a41a640 1 --2- 172.21.15.154:0/2048051451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f992415c840 0x7f99240cbfe0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:30.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.241+0000 7f992a41a640 1 -- 172.21.15.154:0/2048051451 >> 172.21.15.154:0/2048051451 conn(0x7f992406a390 msgr2=0x7f99240afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:30.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.241+0000 7f992a41a640 1 -- 172.21.15.154:0/2048051451 shutdown_connections 2023-09-23T03:25:30.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.241+0000 7f992a41a640 1 -- 172.21.15.154:0/2048051451 wait complete. 2023-09-23T03:25:30.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.243+0000 7f992a41a640 1 Processor -- start 2023-09-23T03:25:30.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.243+0000 7f992a41a640 1 -- start start 2023-09-23T03:25:30.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.244+0000 7f992a41a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99240cacd0 0x7f99240b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:30.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.244+0000 7f992a41a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f992415c840 0x7f9924157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:30.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.244+0000 7f992a41a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9924160fd0 0x7f99240b3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:30.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.244+0000 7f992a41a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99240b2f00 con 0x7f99240cacd0 2023-09-23T03:25:30.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.244+0000 7f992a41a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99240b16d0 con 0x7f9924160fd0 2023-09-23T03:25:30.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.244+0000 7f992a41a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99240b2530 con 0x7f992415c840 2023-09-23T03:25:30.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.245+0000 7f9929418640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99240cacd0 0x7f99240b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:30.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.245+0000 7f9929418640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99240cacd0 0x7f99240b29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50780/0 (socket says 172.21.15.154:50780) 2023-09-23T03:25:30.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.245+0000 7f9929418640 1 -- 172.21.15.154:0/2037547700 learned_addr learned my addr 172.21.15.154:0/2037547700 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:30.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.245+0000 7f9929418640 1 -- 172.21.15.154:0/2037547700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f992415c840 msgr2=0x7f9924157430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:30.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.245+0000 7f9929418640 1 --2- 172.21.15.154:0/2037547700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f992415c840 0x7f9924157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:30.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.245+0000 7f9929418640 1 -- 172.21.15.154:0/2037547700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9924160fd0 msgr2=0x7f99240b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:30.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.245+0000 7f9929418640 1 --2- 172.21.15.154:0/2037547700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9924160fd0 0x7f99240b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:30.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.245+0000 7f9929418640 1 -- 172.21.15.154:0/2037547700 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9920009a70 con 0x7f99240cacd0 2023-09-23T03:25:30.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.245+0000 7f9929418640 1 --2- 172.21.15.154:0/2037547700 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99240cacd0 0x7f99240b29c0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f9920037040 tx=0x7f9920004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:30.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.246+0000 7f9919ffb640 1 -- 172.21.15.154:0/2037547700 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992003e610 con 0x7f99240cacd0 2023-09-23T03:25:30.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.246+0000 7f9919ffb640 1 -- 172.21.15.154:0/2037547700 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992003e770 con 0x7f99240cacd0 2023-09-23T03:25:30.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.246+0000 7f992a41a640 1 -- 172.21.15.154:0/2037547700 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99240b1920 con 0x7f99240cacd0 2023-09-23T03:25:30.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.246+0000 7f9919ffb640 1 -- 172.21.15.154:0/2037547700 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992003eb20 con 0x7f99240cacd0 2023-09-23T03:25:30.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.246+0000 7f992a41a640 1 -- 172.21.15.154:0/2037547700 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99240b1ee0 con 0x7f99240cacd0 2023-09-23T03:25:30.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.247+0000 7f992a41a640 1 -- 172.21.15.154:0/2037547700 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99240736b0 con 0x7f99240cacd0 2023-09-23T03:25:30.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.248+0000 7f9919ffb640 1 -- 172.21.15.154:0/2037547700 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f992003b050 con 0x7f99240cacd0 2023-09-23T03:25:30.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.249+0000 7f9919ffb640 1 --2- 172.21.15.154:0/2037547700 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f990806af60 0x7f990806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:30.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.250+0000 7f9919ffb640 1 -- 172.21.15.154:0/2037547700 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f99200ac510 con 0x7f99240cacd0 2023-09-23T03:25:30.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.250+0000 7f9928c17640 1 --2- 172.21.15.154:0/2037547700 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f990806af60 0x7f990806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:30.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.250+0000 7f9928c17640 1 --2- 172.21.15.154:0/2037547700 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f990806af60 0x7f990806d420 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f99241f5200 tx=0x7f99140079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:30.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.254+0000 7f9919ffb640 1 -- 172.21.15.154:0/2037547700 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f9920077300 con 0x7f99240cacd0 2023-09-23T03:25:30.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.443+0000 7f992a41a640 1 -- 172.21.15.154:0/2037547700 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f992404e920 con 0x7f99240cacd0 2023-09-23T03:25:30.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.445+0000 7f9919ffb640 1 -- 172.21.15.154:0/2037547700 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+18993 (secure 0 0 0) 0x7f992007a920 con 0x7f99240cacd0 2023-09-23T03:25:30.446 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:25:30.447 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":92,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:24:46.339781+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:24:07.834123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-09-23T03:24:09.058217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data_extra","create_time":"2023-09-23T03:24:19.995906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6837/1588996498":"2023-09-24T03:24:45.398760+0000","172.21.15.154:6836/1588996498":"2023-09-24T03:24:45.398760+0000","172.21.15.154:6837/3812680297":"2023-09-24T03:24:38.360723+0000","172.21.15.154:6836/3812680297":"2023-09-24T03:24:38.360723+0000","172.21.15.154:6837/2213283923":"2023-09-24T03:24:36.268262+0000","172.21.15.154:6836/2213283923":"2023-09-24T03:24:36.268262+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:25:30.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.447+0000 7f992a41a640 1 -- 172.21.15.154:0/2037547700 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f990806af60 msgr2=0x7f990806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:30.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.447+0000 7f992a41a640 1 --2- 172.21.15.154:0/2037547700 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f990806af60 0x7f990806d420 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f99241f5200 tx=0x7f99140079b0 comp rx=0 tx=0).stop 2023-09-23T03:25:30.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.447+0000 7f992a41a640 1 -- 172.21.15.154:0/2037547700 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99240cacd0 msgr2=0x7f99240b29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:30.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.447+0000 7f992a41a640 1 --2- 172.21.15.154:0/2037547700 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99240cacd0 0x7f99240b29c0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f9920037040 tx=0x7f9920004a40 comp rx=0 tx=0).stop 2023-09-23T03:25:30.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.447+0000 7f992a41a640 1 -- 172.21.15.154:0/2037547700 shutdown_connections 2023-09-23T03:25:30.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.447+0000 7f992a41a640 1 --2- 172.21.15.154:0/2037547700 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f990806af60 0x7f990806d420 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:30.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.447+0000 7f992a41a640 1 --2- 172.21.15.154:0/2037547700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9924160fd0 0x7f99240b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:30.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.447+0000 7f992a41a640 1 --2- 172.21.15.154:0/2037547700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f992415c840 0x7f9924157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:30.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.447+0000 7f992a41a640 1 --2- 172.21.15.154:0/2037547700 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99240cacd0 0x7f99240b29c0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:30.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.447+0000 7f992a41a640 1 -- 172.21.15.154:0/2037547700 >> 172.21.15.154:0/2037547700 conn(0x7f992406a390 msgr2=0x7f99240ce110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:30.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.448+0000 7f992a41a640 1 -- 172.21.15.154:0/2037547700 shutdown_connections 2023-09-23T03:25:30.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.448+0000 7f992a41a640 1 -- 172.21.15.154:0/2037547700 wait complete. 2023-09-23T03:25:30.467 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-09-23T03:25:30.467 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:25:30.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.671+0000 7f0a8759e640 1 Processor -- start 2023-09-23T03:25:30.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.671+0000 7f0a8759e640 1 -- start start 2023-09-23T03:25:30.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.672+0000 7f0a8759e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a8815c840 0x7f0a880cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:30.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.672+0000 7f0a8759e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0a88160fd0 0x7f0a880c9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:30.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.672+0000 7f0a8759e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a880cc910 con 0x7f0a880c99d0 2023-09-23T03:25:30.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.672+0000 7f0a8759e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a8815cc20 con 0x7f0a88160fd0 2023-09-23T03:25:30.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.672+0000 7f0a8759e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a880c89b0 con 0x7f0a8815c840 2023-09-23T03:25:30.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.673+0000 7f0a86d9d640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f0a880c99d0 0x7f0a880cacd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:41032/0 (socket says 172.21.15.154:41032) 2023-09-23T03:25:30.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.673+0000 7f0a86d9d640 1 -- 172.21.15.154:0/3890297132 learned_addr learned my addr 172.21.15.154:0/3890297132 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:25:30.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.673+0000 7f0a8659c640 1 --2- 172.21.15.154:0/3890297132 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a8815c840 0x7f0a880cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:30.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.673+0000 7f0a8559a640 1 -- 172.21.15.154:0/3890297132 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3793287742 0 0) 0x7f0a880cc910 con 0x7f0a880c99d0 2023-09-23T03:25:30.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.673+0000 7f0a8559a640 1 -- 172.21.15.154:0/3890297132 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a70003520 con 0x7f0a880c99d0 2023-09-23T03:25:30.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.673+0000 7f0a85d9b640 1 --2- 172.21.15.154:0/3890297132 >> v2:172.21.15.190:3300/0 conn(0x7f0a88160fd0 0x7f0a880c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:30.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.674+0000 7f0a8559a640 1 -- 172.21.15.154:0/3890297132 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 457111756 0 0) 0x7f0a70003520 con 0x7f0a880c99d0 2023-09-23T03:25:30.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.674+0000 7f0a8559a640 1 -- 172.21.15.154:0/3890297132 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a8815c840 msgr2=0x7f0a880cbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:30.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.674+0000 7f0a8559a640 1 --2- 172.21.15.154:0/3890297132 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a8815c840 0x7f0a880cbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:30.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.674+0000 7f0a8559a640 1 -- 172.21.15.154:0/3890297132 >> v2:172.21.15.190:3300/0 conn(0x7f0a88160fd0 msgr2=0x7f0a880c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:30.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.674+0000 7f0a8559a640 1 --2- 172.21.15.154:0/3890297132 >> v2:172.21.15.190:3300/0 conn(0x7f0a88160fd0 0x7f0a880c9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:30.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.674+0000 7f0a8559a640 1 -- 172.21.15.154:0/3890297132 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a880c8c90 con 0x7f0a880c99d0 2023-09-23T03:25:30.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.674+0000 7f0a8559a640 1 -- 172.21.15.154:0/3890297132 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f0a800019f0 con 0x7f0a880c99d0 2023-09-23T03:25:30.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.674+0000 7f0a8559a640 1 -- 172.21.15.154:0/3890297132 >> v1:172.21.15.154:6789/0 conn(0x7f0a880c99d0 legacy=0x7f0a880cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:30.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.674+0000 7f0a8559a640 1 --2- 172.21.15.154:0/3890297132 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a70003970 0x7f0a70003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:30.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.674+0000 7f0a8559a640 1 --2- 172.21.15.154:0/3890297132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a700047c0 0x7f0a70004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:30.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.674+0000 7f0a8559a640 1 --2- 172.21.15.154:0/3890297132 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a8815c840 0x7f0a70005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:30.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.674+0000 7f0a8559a640 1 -- 172.21.15.154:0/3890297132 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8815cc20 con 0x7f0a70003970 2023-09-23T03:25:30.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.674+0000 7f0a8559a640 1 -- 172.21.15.154:0/3890297132 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a880c89b0 con 0x7f0a700047c0 2023-09-23T03:25:30.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.674+0000 7f0a8559a640 1 -- 172.21.15.154:0/3890297132 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7000b980 con 0x7f0a8815c840 2023-09-23T03:25:30.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.674+0000 7f0a8659c640 1 --2- 172.21.15.154:0/3890297132 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a70003970 0x7f0a70003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:30.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.675+0000 7f0a86d9d640 1 --2- 172.21.15.154:0/3890297132 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a8815c840 0x7f0a70005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:30.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.675+0000 7f0a8659c640 1 -- 172.21.15.154:0/3890297132 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a8815c840 msgr2=0x7f0a70005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:30.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.675+0000 7f0a85d9b640 1 --2- 172.21.15.154:0/3890297132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a700047c0 0x7f0a70004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:30.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.675+0000 7f0a8659c640 1 --2- 172.21.15.154:0/3890297132 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a8815c840 0x7f0a70005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:30.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.675+0000 7f0a8659c640 1 -- 172.21.15.154:0/3890297132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a700047c0 msgr2=0x7f0a70004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:30.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.675+0000 7f0a8659c640 1 --2- 172.21.15.154:0/3890297132 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a700047c0 0x7f0a70004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:30.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.675+0000 7f0a8659c640 1 -- 172.21.15.154:0/3890297132 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a70003520 con 0x7f0a70003970 2023-09-23T03:25:30.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.676+0000 7f0a8659c640 1 --2- 172.21.15.154:0/3890297132 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a70003970 0x7f0a70003d90 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f0a78009ac0 tx=0x7f0a700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca9ead43c48fdfb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:30.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.676+0000 7f0a8559a640 1 -- 172.21.15.154:0/3890297132 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7802c6b0 con 0x7f0a70003970 2023-09-23T03:25:30.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.676+0000 7f0a8559a640 1 -- 172.21.15.154:0/3890297132 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a7802c810 con 0x7f0a70003970 2023-09-23T03:25:30.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.676+0000 7f0a8559a640 1 -- 172.21.15.154:0/3890297132 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7802cba0 con 0x7f0a70003970 2023-09-23T03:25:30.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.677+0000 7f0a8759e640 1 -- 172.21.15.154:0/3890297132 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a70003970 msgr2=0x7f0a70003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:30.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.677+0000 7f0a8759e640 1 --2- 172.21.15.154:0/3890297132 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a70003970 0x7f0a70003d90 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f0a78009ac0 tx=0x7f0a700057f0 comp rx=0 tx=0).stop 2023-09-23T03:25:30.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.677+0000 7f0a86d9d640 1 -- 172.21.15.154:0/3890297132 reap_dead start 2023-09-23T03:25:30.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.677+0000 7f0a8759e640 1 -- 172.21.15.154:0/3890297132 shutdown_connections 2023-09-23T03:25:30.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.677+0000 7f0a8759e640 1 -- 172.21.15.154:0/3890297132 >> 172.21.15.154:0/3890297132 conn(0x7f0a8806a390 msgr2=0x7f0a880afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:30.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.677+0000 7f0a8759e640 1 -- 172.21.15.154:0/3890297132 shutdown_connections 2023-09-23T03:25:30.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.677+0000 7f0a8759e640 1 -- 172.21.15.154:0/3890297132 wait complete. 2023-09-23T03:25:30.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.679+0000 7f0a8759e640 1 Processor -- start 2023-09-23T03:25:30.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.679+0000 7f0a8759e640 1 -- start start 2023-09-23T03:25:30.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.680+0000 7f0a8759e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a70003970 0x7f0a881534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:30.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.680+0000 7f0a8759e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a8815cf20 0x7f0a880ce110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:30.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.680+0000 7f0a8759e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a8815d3a0 0x7f0a880ce650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:30.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.680+0000 7f0a8759e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a881f56e0 con 0x7f0a8815d3a0 2023-09-23T03:25:30.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.680+0000 7f0a8759e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a881f59c0 con 0x7f0a8815cf20 2023-09-23T03:25:30.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.680+0000 7f0a8759e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a881f5ca0 con 0x7f0a70003970 2023-09-23T03:25:30.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.681+0000 7f0a86d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a8815d3a0 0x7f0a880ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:30.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.681+0000 7f0a86d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a8815d3a0 0x7f0a880ce650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50802/0 (socket says 172.21.15.154:50802) 2023-09-23T03:25:30.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.681+0000 7f0a86d9d640 1 -- 172.21.15.154:0/1127845273 learned_addr learned my addr 172.21.15.154:0/1127845273 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:30.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.681+0000 7f0a8659c640 1 --2- 172.21.15.154:0/1127845273 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a70003970 0x7f0a881534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:30.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.681+0000 7f0a85d9b640 1 --2- 172.21.15.154:0/1127845273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a8815cf20 0x7f0a880ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:30.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.681+0000 7f0a86d9d640 1 -- 172.21.15.154:0/1127845273 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a70003970 msgr2=0x7f0a881534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:30.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.681+0000 7f0a86d9d640 1 --2- 172.21.15.154:0/1127845273 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a70003970 0x7f0a881534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:30.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.681+0000 7f0a86d9d640 1 -- 172.21.15.154:0/1127845273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a8815cf20 msgr2=0x7f0a880ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:30.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.681+0000 7f0a86d9d640 1 --2- 172.21.15.154:0/1127845273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a8815cf20 0x7f0a880ce110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:30.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.681+0000 7f0a86d9d640 1 -- 172.21.15.154:0/1127845273 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a881f5f80 con 0x7f0a8815d3a0 2023-09-23T03:25:30.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.682+0000 7f0a86d9d640 1 --2- 172.21.15.154:0/1127845273 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a8815d3a0 0x7f0a880ce650 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f0a880cbf40 tx=0x7f0a8000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:30.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.682+0000 7f0a76ffd640 1 -- 172.21.15.154:0/1127845273 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a80004d60 con 0x7f0a8815d3a0 2023-09-23T03:25:30.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.682+0000 7f0a76ffd640 1 -- 172.21.15.154:0/1127845273 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8000bb80 con 0x7f0a8815d3a0 2023-09-23T03:25:30.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.682+0000 7f0a76ffd640 1 -- 172.21.15.154:0/1127845273 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a80007070 con 0x7f0a8815d3a0 2023-09-23T03:25:30.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.682+0000 7f0a8759e640 1 -- 172.21.15.154:0/1127845273 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a881f64a0 con 0x7f0a8815d3a0 2023-09-23T03:25:30.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.682+0000 7f0a8759e640 1 -- 172.21.15.154:0/1127845273 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a8815d970 con 0x7f0a8815d3a0 2023-09-23T03:25:30.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.683+0000 7f0a8759e640 1 -- 172.21.15.154:0/1127845273 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a880736b0 con 0x7f0a8815d3a0 2023-09-23T03:25:30.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.684+0000 7f0a76ffd640 1 -- 172.21.15.154:0/1127845273 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f0a800071d0 con 0x7f0a8815d3a0 2023-09-23T03:25:30.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.686+0000 7f0a76ffd640 1 --2- 172.21.15.154:0/1127845273 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0a6806af30 0x7f0a6806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:30.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.686+0000 7f0a76ffd640 1 -- 172.21.15.154:0/1127845273 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f0a80089ea0 con 0x7f0a8815d3a0 2023-09-23T03:25:30.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.686+0000 7f0a8659c640 1 --2- 172.21.15.154:0/1127845273 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0a6806af30 0x7f0a6806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:30.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.687+0000 7f0a8659c640 1 --2- 172.21.15.154:0/1127845273 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0a6806af30 0x7f0a6806d3f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f0a78036000 tx=0x7f0a78030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:30.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.691+0000 7f0a76ffd640 1 -- 172.21.15.154:0/1127845273 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f0a80054a60 con 0x7f0a8815d3a0 2023-09-23T03:25:30.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.920+0000 7f0a8759e640 1 -- 172.21.15.154:0/1127845273 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a881f5820 con 0x7f0a8815d3a0 2023-09-23T03:25:30.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.921+0000 7f0a76ffd640 1 -- 172.21.15.154:0/1127845273 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4925 (secure 0 0 0) 0x7f0a881f5820 con 0x7f0a8815d3a0 2023-09-23T03:25:30.923 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:25:30.923 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":76,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7743,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/31599836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":31599836},{"type":"v1","addr":"172.21.15.154:6836","nonce":31599836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":7752,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/4266451911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":4266451911},{"type":"v1","addr":"172.21.15.154:6837","nonce":4266451911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":7755,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3029600822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3029600822},{"type":"v1","addr":"172.21.15.190:6834","nonce":3029600822}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:24:37.386034+0000","modified":"2023-09-23T03:24:47.439439+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":90,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7758},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7758":{"gid":7758,"name":"b","rank":0,"incarnation":73,"state":"up:active","state_seq":4,"addr":"172.21.15.190:6835/2202597679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2202597679},{"type":"v1","addr":"172.21.15.190:6835","nonce":2202597679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12,13],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-09-23T03:25:30.923 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 76 2023-09-23T03:25:30.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.924+0000 7f0a8759e640 1 -- 172.21.15.154:0/1127845273 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0a6806af30 msgr2=0x7f0a6806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:30.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.924+0000 7f0a8759e640 1 --2- 172.21.15.154:0/1127845273 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0a6806af30 0x7f0a6806d3f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f0a78036000 tx=0x7f0a78030000 comp rx=0 tx=0).stop 2023-09-23T03:25:30.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.924+0000 7f0a8759e640 1 -- 172.21.15.154:0/1127845273 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a8815d3a0 msgr2=0x7f0a880ce650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:30.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.925+0000 7f0a8759e640 1 --2- 172.21.15.154:0/1127845273 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a8815d3a0 0x7f0a880ce650 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f0a880cbf40 tx=0x7f0a8000b4a0 comp rx=0 tx=0).stop 2023-09-23T03:25:30.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.925+0000 7f0a8759e640 1 -- 172.21.15.154:0/1127845273 shutdown_connections 2023-09-23T03:25:30.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.925+0000 7f0a8759e640 1 --2- 172.21.15.154:0/1127845273 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0a6806af30 0x7f0a6806d3f0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:30.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.925+0000 7f0a8759e640 1 --2- 172.21.15.154:0/1127845273 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0a8815d3a0 0x7f0a880ce650 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:30.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.925+0000 7f0a8759e640 1 --2- 172.21.15.154:0/1127845273 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0a8815cf20 0x7f0a880ce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:30.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.925+0000 7f0a8759e640 1 --2- 172.21.15.154:0/1127845273 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0a70003970 0x7f0a881534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:30.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.925+0000 7f0a8759e640 1 -- 172.21.15.154:0/1127845273 >> 172.21.15.154:0/1127845273 conn(0x7f0a8806a390 msgr2=0x7f0a88071860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:30.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.925+0000 7f0a8759e640 1 -- 172.21.15.154:0/1127845273 shutdown_connections 2023-09-23T03:25:30.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:30.925+0000 7f0a8759e640 1 -- 172.21.15.154:0/1127845273 wait complete. 2023-09-23T03:25:30.943 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:25:31.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.159+0000 7ffbf34b4640 1 Processor -- start 2023-09-23T03:25:31.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.160+0000 7ffbf34b4640 1 -- start start 2023-09-23T03:25:31.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.161+0000 7ffbf34b4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbec161010 0x7ffbec0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:31.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.161+0000 7ffbf34b4640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbec0cb160 con 0x7ffbec0c14e0 2023-09-23T03:25:31.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.161+0000 7ffbf34b4640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbec0ccd20 con 0x7ffbec06c8e0 2023-09-23T03:25:31.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.161+0000 7ffbf34b4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbec06cde0 con 0x7ffbec161010 2023-09-23T03:25:31.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.161+0000 7ffbf0a4f640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ffbec0c14e0 0x7ffbec06c1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:41040/0 (socket says 172.21.15.154:41040) 2023-09-23T03:25:31.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.161+0000 7ffbf0a4f640 1 -- 172.21.15.154:0/1460489411 learned_addr learned my addr 172.21.15.154:0/1460489411 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:25:31.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.162+0000 7ffbf1250640 1 --2- 172.21.15.154:0/1460489411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbec161010 0x7ffbec0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:31.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.162+0000 7ffbdffff640 1 -- 172.21.15.154:0/1460489411 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 961391495 0 0) 0x7ffbec0cb160 con 0x7ffbec0c14e0 2023-09-23T03:25:31.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.162+0000 7ffbdffff640 1 -- 172.21.15.154:0/1460489411 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbd8003520 con 0x7ffbec0c14e0 2023-09-23T03:25:31.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.162+0000 7ffbdffff640 1 -- 172.21.15.154:0/1460489411 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2006766556 0 0) 0x7ffbd8003520 con 0x7ffbec0c14e0 2023-09-23T03:25:31.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.162+0000 7ffbdffff640 1 -- 172.21.15.154:0/1460489411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbec161010 msgr2=0x7ffbec0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:31.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.162+0000 7ffbdffff640 1 --2- 172.21.15.154:0/1460489411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbec161010 0x7ffbec0cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:31.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.162+0000 7ffbdffff640 1 -- 172.21.15.154:0/1460489411 >> v1:172.21.15.190:6789/0 conn(0x7ffbec06c8e0 legacy=0x7ffbec0b1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:31.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.162+0000 7ffbdffff640 1 -- 172.21.15.154:0/1460489411 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbec06d0c0 con 0x7ffbec0c14e0 2023-09-23T03:25:31.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.162+0000 7ffbdffff640 1 -- 172.21.15.154:0/1460489411 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7ffbe0001670 con 0x7ffbec0c14e0 2023-09-23T03:25:31.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.162+0000 7ffbdffff640 1 -- 172.21.15.154:0/1460489411 >> v1:172.21.15.154:6789/0 conn(0x7ffbec0c14e0 legacy=0x7ffbec06c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:31.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.163+0000 7ffbdffff640 1 --2- 172.21.15.154:0/1460489411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbd80039a0 0x7ffbd8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:31.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.163+0000 7ffbdffff640 1 --2- 172.21.15.154:0/1460489411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbec161010 0x7ffbd80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:31.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.163+0000 7ffbdffff640 1 --2- 172.21.15.154:0/1460489411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbd8004d00 0x7ffbd8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:31.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.163+0000 7ffbdffff640 1 -- 172.21.15.154:0/1460489411 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbec06cde0 con 0x7ffbd8004d00 2023-09-23T03:25:31.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.163+0000 7ffbdffff640 1 -- 172.21.15.154:0/1460489411 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd800bb10 con 0x7ffbd80039a0 2023-09-23T03:25:31.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.163+0000 7ffbdffff640 1 -- 172.21.15.154:0/1460489411 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd800bcd0 con 0x7ffbec161010 2023-09-23T03:25:31.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.163+0000 7ffbf1a51640 1 --2- 172.21.15.154:0/1460489411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbd8004d00 0x7ffbd8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:31.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.163+0000 7ffbf1250640 1 --2- 172.21.15.154:0/1460489411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbd80039a0 0x7ffbd8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:31.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.163+0000 7ffbf0a4f640 1 --2- 172.21.15.154:0/1460489411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbec161010 0x7ffbd80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:31.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.163+0000 7ffbf1a51640 1 -- 172.21.15.154:0/1460489411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbec161010 msgr2=0x7ffbd80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:31.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.163+0000 7ffbf1a51640 1 --2- 172.21.15.154:0/1460489411 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbec161010 0x7ffbd80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:31.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.163+0000 7ffbf1a51640 1 -- 172.21.15.154:0/1460489411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbd80039a0 msgr2=0x7ffbd8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:31.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.163+0000 7ffbf1a51640 1 --2- 172.21.15.154:0/1460489411 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbd80039a0 0x7ffbd8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:31.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.163+0000 7ffbf1a51640 1 -- 172.21.15.154:0/1460489411 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd8003520 con 0x7ffbd8004d00 2023-09-23T03:25:31.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.165+0000 7ffbf1a51640 1 --2- 172.21.15.154:0/1460489411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbd8004d00 0x7ffbd8005180 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4001e20 tx=0x7ffbd80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a8604f4ff478d26 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:31.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.165+0000 7ffbdffff640 1 -- 172.21.15.154:0/1460489411 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe4003b70 con 0x7ffbd8004d00 2023-09-23T03:25:31.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.165+0000 7ffbdffff640 1 -- 172.21.15.154:0/1460489411 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbe4003cd0 con 0x7ffbd8004d00 2023-09-23T03:25:31.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.165+0000 7ffbdffff640 1 -- 172.21.15.154:0/1460489411 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe4005310 con 0x7ffbd8004d00 2023-09-23T03:25:31.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.166+0000 7ffbf34b4640 1 -- 172.21.15.154:0/1460489411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbd8004d00 msgr2=0x7ffbd8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:31.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.166+0000 7ffbf34b4640 1 --2- 172.21.15.154:0/1460489411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbd8004d00 0x7ffbd8005180 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4001e20 tx=0x7ffbd80057f0 comp rx=0 tx=0).stop 2023-09-23T03:25:31.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.166+0000 7ffbf1a51640 1 -- 172.21.15.154:0/1460489411 reap_dead start 2023-09-23T03:25:31.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.166+0000 7ffbf34b4640 1 -- 172.21.15.154:0/1460489411 shutdown_connections 2023-09-23T03:25:31.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.166+0000 7ffbf34b4640 1 -- 172.21.15.154:0/1460489411 >> 172.21.15.154:0/1460489411 conn(0x7ffbec06a0f0 msgr2=0x7ffbec06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:31.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.166+0000 7ffbf34b4640 1 -- 172.21.15.154:0/1460489411 shutdown_connections 2023-09-23T03:25:31.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.166+0000 7ffbf34b4640 1 -- 172.21.15.154:0/1460489411 wait complete. 2023-09-23T03:25:31.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.168+0000 7ffbf34b4640 1 Processor -- start 2023-09-23T03:25:31.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.168+0000 7ffbf34b4640 1 -- start start 2023-09-23T03:25:31.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.169+0000 7ffbf34b4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbd8004d00 0x7ffbec0cd120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:31.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.169+0000 7ffbf34b4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbec159fc0 0x7ffbec15a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:31.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.169+0000 7ffbf34b4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbec0c67c0 0x7ffbec0c6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:31.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.169+0000 7ffbf34b4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbec0c81e0 con 0x7ffbec0c67c0 2023-09-23T03:25:31.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.169+0000 7ffbf34b4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbec15a980 con 0x7ffbec159fc0 2023-09-23T03:25:31.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.169+0000 7ffbf34b4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbec0c7560 con 0x7ffbd8004d00 2023-09-23T03:25:31.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.169+0000 7ffbf1a51640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbec0c67c0 0x7ffbec0c6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:31.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.170+0000 7ffbf1a51640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbec0c67c0 0x7ffbec0c6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50812/0 (socket says 172.21.15.154:50812) 2023-09-23T03:25:31.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.170+0000 7ffbf1a51640 1 -- 172.21.15.154:0/3815407642 learned_addr learned my addr 172.21.15.154:0/3815407642 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:31.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.170+0000 7ffbf1250640 1 --2- 172.21.15.154:0/3815407642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbd8004d00 0x7ffbec0cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:31.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.170+0000 7ffbf0a4f640 1 --2- 172.21.15.154:0/3815407642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbec159fc0 0x7ffbec15a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:31.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.170+0000 7ffbf1a51640 1 -- 172.21.15.154:0/3815407642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbd8004d00 msgr2=0x7ffbec0cd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:31.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.170+0000 7ffbf1a51640 1 --2- 172.21.15.154:0/3815407642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbd8004d00 0x7ffbec0cd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:31.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.170+0000 7ffbf1a51640 1 -- 172.21.15.154:0/3815407642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbec159fc0 msgr2=0x7ffbec15a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:31.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.170+0000 7ffbf1a51640 1 --2- 172.21.15.154:0/3815407642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbec159fc0 0x7ffbec15a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:31.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.170+0000 7ffbf1a51640 1 -- 172.21.15.154:0/3815407642 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbec0c7840 con 0x7ffbec0c67c0 2023-09-23T03:25:31.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.170+0000 7ffbf1a51640 1 --2- 172.21.15.154:0/3815407642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbec0c67c0 0x7ffbec0c6c40 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4001f40 tx=0x7ffbe40014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:31.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.171+0000 7ffbddffb640 1 -- 172.21.15.154:0/3815407642 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe4006d50 con 0x7ffbec0c67c0 2023-09-23T03:25:31.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.171+0000 7ffbddffb640 1 -- 172.21.15.154:0/3815407642 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbe4003e40 con 0x7ffbec0c67c0 2023-09-23T03:25:31.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.171+0000 7ffbddffb640 1 -- 172.21.15.154:0/3815407642 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe4007230 con 0x7ffbec0c67c0 2023-09-23T03:25:31.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.171+0000 7ffbf34b4640 1 -- 172.21.15.154:0/3815407642 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbec0c7b20 con 0x7ffbec0c67c0 2023-09-23T03:25:31.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.171+0000 7ffbf34b4640 1 -- 172.21.15.154:0/3815407642 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbec0c7180 con 0x7ffbec0c67c0 2023-09-23T03:25:31.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.172+0000 7ffbf34b4640 1 -- 172.21.15.154:0/3815407642 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbec0c7310 con 0x7ffbec0c67c0 2023-09-23T03:25:31.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.173+0000 7ffbddffb640 1 -- 172.21.15.154:0/3815407642 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7ffbe4007390 con 0x7ffbec0c67c0 2023-09-23T03:25:31.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.174+0000 7ffbddffb640 1 --2- 172.21.15.154:0/3815407642 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffbd006af30 0x7ffbd006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:31.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.174+0000 7ffbddffb640 1 -- 172.21.15.154:0/3815407642 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6691+0+0 (secure 0 0 0) 0x7ffbe40ab520 con 0x7ffbec0c67c0 2023-09-23T03:25:31.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.174+0000 7ffbf1250640 1 --2- 172.21.15.154:0/3815407642 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffbd006af30 0x7ffbd006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:31.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.175+0000 7ffbf1250640 1 --2- 172.21.15.154:0/3815407642 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffbd006af30 0x7ffbd006d3f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ffbe8009a00 tx=0x7ffbe8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:31.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.178+0000 7ffbddffb640 1 -- 172.21.15.154:0/3815407642 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7ffbe4076360 con 0x7ffbec0c67c0 2023-09-23T03:25:31.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.402+0000 7ffbf34b4640 1 -- 172.21.15.154:0/3815407642 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffbec073eb0 con 0x7ffbec0c67c0 2023-09-23T03:25:31.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.404+0000 7ffbddffb640 1 -- 172.21.15.154:0/3815407642 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4925 (secure 0 0 0) 0x7ffbe4047020 con 0x7ffbec0c67c0 2023-09-23T03:25:31.405 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:25:31.405 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":76,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7743,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/31599836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":31599836},{"type":"v1","addr":"172.21.15.154:6836","nonce":31599836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":7752,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/4266451911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":4266451911},{"type":"v1","addr":"172.21.15.154:6837","nonce":4266451911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":7755,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3029600822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3029600822},{"type":"v1","addr":"172.21.15.190:6834","nonce":3029600822}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:24:37.386034+0000","modified":"2023-09-23T03:24:47.439439+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":90,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7758},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7758":{"gid":7758,"name":"b","rank":0,"incarnation":73,"state":"up:active","state_seq":4,"addr":"172.21.15.190:6835/2202597679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2202597679},{"type":"v1","addr":"172.21.15.190:6835","nonce":2202597679}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12,13],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-09-23T03:25:31.405 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 76 2023-09-23T03:25:31.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.406+0000 7ffbf34b4640 1 -- 172.21.15.154:0/3815407642 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffbd006af30 msgr2=0x7ffbd006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:31.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.406+0000 7ffbf34b4640 1 --2- 172.21.15.154:0/3815407642 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffbd006af30 0x7ffbd006d3f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ffbe8009a00 tx=0x7ffbe8008040 comp rx=0 tx=0).stop 2023-09-23T03:25:31.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.406+0000 7ffbf34b4640 1 -- 172.21.15.154:0/3815407642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbec0c67c0 msgr2=0x7ffbec0c6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:31.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.406+0000 7ffbf34b4640 1 --2- 172.21.15.154:0/3815407642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbec0c67c0 0x7ffbec0c6c40 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4001f40 tx=0x7ffbe40014a0 comp rx=0 tx=0).stop 2023-09-23T03:25:31.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.406+0000 7ffbf34b4640 1 -- 172.21.15.154:0/3815407642 shutdown_connections 2023-09-23T03:25:31.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.406+0000 7ffbf34b4640 1 --2- 172.21.15.154:0/3815407642 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffbd006af30 0x7ffbd006d3f0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:31.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.406+0000 7ffbf34b4640 1 --2- 172.21.15.154:0/3815407642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbec0c67c0 0x7ffbec0c6c40 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:31.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.406+0000 7ffbf34b4640 1 --2- 172.21.15.154:0/3815407642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbec159fc0 0x7ffbec15a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:31.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.406+0000 7ffbf34b4640 1 --2- 172.21.15.154:0/3815407642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbd8004d00 0x7ffbec0cd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:31.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.406+0000 7ffbf34b4640 1 -- 172.21.15.154:0/3815407642 >> 172.21.15.154:0/3815407642 conn(0x7ffbec06a0f0 msgr2=0x7ffbec06c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:31.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.406+0000 7ffbf34b4640 1 -- 172.21.15.154:0/3815407642 shutdown_connections 2023-09-23T03:25:31.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.407+0000 7ffbf34b4640 1 -- 172.21.15.154:0/3815407642 wait complete. 2023-09-23T03:25:31.428 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:25:31.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.640+0000 7f323d24d640 1 Processor -- start 2023-09-23T03:25:31.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.640+0000 7f323d24d640 1 -- start start 2023-09-23T03:25:31.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.642+0000 7f323d24d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3238160fd0 0x7f32380c9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:31.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.642+0000 7f323d24d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32380c99d0 0x7f32380cacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:31.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.642+0000 7f323d24d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32380cc910 con 0x7f323815c840 2023-09-23T03:25:31.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.642+0000 7f323d24d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f323815cc20 con 0x7f3238160fd0 2023-09-23T03:25:31.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.642+0000 7f323d24d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32380cc6f0 con 0x7f32380c99d0 2023-09-23T03:25:31.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.642+0000 7f322ffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3238160fd0 0x7f32380c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:31.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.642+0000 7f322ffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f3238160fd0 0x7f32380c9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50818/0 (socket says 172.21.15.154:50818) 2023-09-23T03:25:31.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.642+0000 7f322ffff640 1 -- 172.21.15.154:0/3237181380 learned_addr learned my addr 172.21.15.154:0/3237181380 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:31.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.642+0000 7f323ca4c640 1 --2- 172.21.15.154:0/3237181380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32380c99d0 0x7f32380cacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:31.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.642+0000 7f322ffff640 1 -- 172.21.15.154:0/3237181380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32380c99d0 msgr2=0x7f32380cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:31.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.643+0000 7f322ffff640 1 --2- 172.21.15.154:0/3237181380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32380c99d0 0x7f32380cacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:31.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.643+0000 7f322ffff640 1 -- 172.21.15.154:0/3237181380 >> v1:172.21.15.154:6789/0 conn(0x7f323815c840 legacy=0x7f32380cbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:31.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.643+0000 7f322ffff640 1 -- 172.21.15.154:0/3237181380 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32380c89b0 con 0x7f3238160fd0 2023-09-23T03:25:31.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.644+0000 7f322ffff640 1 --2- 172.21.15.154:0/3237181380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3238160fd0 0x7f32380c9490 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f3220009870 tx=0x7f322002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95812fded2d9585c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:31.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.644+0000 7f32377fe640 1 -- 172.21.15.154:0/3237181380 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3220030840 con 0x7f3238160fd0 2023-09-23T03:25:31.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.644+0000 7f32377fe640 1 -- 172.21.15.154:0/3237181380 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32200309a0 con 0x7f3238160fd0 2023-09-23T03:25:31.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.644+0000 7f32377fe640 1 -- 172.21.15.154:0/3237181380 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3220030d30 con 0x7f3238160fd0 2023-09-23T03:25:31.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.645+0000 7f323d24d640 1 -- 172.21.15.154:0/3237181380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3238160fd0 msgr2=0x7f32380c9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:31.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.645+0000 7f323d24d640 1 --2- 172.21.15.154:0/3237181380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3238160fd0 0x7f32380c9490 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f3220009870 tx=0x7f322002b980 comp rx=0 tx=0).stop 2023-09-23T03:25:31.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.645+0000 7f323d24d640 1 -- 172.21.15.154:0/3237181380 shutdown_connections 2023-09-23T03:25:31.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.645+0000 7f323d24d640 1 --2- 172.21.15.154:0/3237181380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32380c99d0 0x7f32380cacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:31.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.645+0000 7f323d24d640 1 --2- 172.21.15.154:0/3237181380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3238160fd0 0x7f32380c9490 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:31.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.645+0000 7f323d24d640 1 -- 172.21.15.154:0/3237181380 >> 172.21.15.154:0/3237181380 conn(0x7f323806a390 msgr2=0x7f32380afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:31.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.645+0000 7f323d24d640 1 -- 172.21.15.154:0/3237181380 shutdown_connections 2023-09-23T03:25:31.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.645+0000 7f323d24d640 1 -- 172.21.15.154:0/3237181380 wait complete. 2023-09-23T03:25:31.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.647+0000 7f323d24d640 1 Processor -- start 2023-09-23T03:25:31.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.647+0000 7f323d24d640 1 -- start start 2023-09-23T03:25:31.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.648+0000 7f323d24d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32380c99d0 0x7f323814fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:31.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.648+0000 7f323d24d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f323815c840 0x7f323815da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:31.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.648+0000 7f323d24d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3238160fd0 0x7f32380c78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:31.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.648+0000 7f323d24d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323814cf60 con 0x7f323815c840 2023-09-23T03:25:31.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.648+0000 7f323d24d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323815df60 con 0x7f3238160fd0 2023-09-23T03:25:31.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.648+0000 7f323d24d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32380c5d10 con 0x7f32380c99d0 2023-09-23T03:25:31.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.648+0000 7f322ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f323815c840 0x7f323815da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:31.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.648+0000 7f322ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f323815c840 0x7f323815da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50830/0 (socket says 172.21.15.154:50830) 2023-09-23T03:25:31.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.648+0000 7f322ffff640 1 -- 172.21.15.154:0/2324885276 learned_addr learned my addr 172.21.15.154:0/2324885276 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:31.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.649+0000 7f322ffff640 1 -- 172.21.15.154:0/2324885276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32380c99d0 msgr2=0x7f323814fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:31.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.649+0000 7f322ffff640 1 --2- 172.21.15.154:0/2324885276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32380c99d0 0x7f323814fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:31.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.649+0000 7f322ffff640 1 -- 172.21.15.154:0/2324885276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3238160fd0 msgr2=0x7f32380c78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:31.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.649+0000 7f322ffff640 1 --2- 172.21.15.154:0/2324885276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3238160fd0 0x7f32380c78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:31.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.649+0000 7f322ffff640 1 -- 172.21.15.154:0/2324885276 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32200094d0 con 0x7f323815c840 2023-09-23T03:25:31.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.649+0000 7f322ffff640 1 --2- 172.21.15.154:0/2324885276 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f323815c840 0x7f323815da20 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f322002b9c0 tx=0x7f3220004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:31.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.649+0000 7f32357fa640 1 -- 172.21.15.154:0/2324885276 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322003e040 con 0x7f323815c840 2023-09-23T03:25:31.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.649+0000 7f32357fa640 1 -- 172.21.15.154:0/2324885276 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3220030810 con 0x7f323815c840 2023-09-23T03:25:31.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.649+0000 7f32357fa640 1 -- 172.21.15.154:0/2324885276 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3220044610 con 0x7f323815c840 2023-09-23T03:25:31.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.649+0000 7f323d24d640 1 -- 172.21.15.154:0/2324885276 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32380c5ff0 con 0x7f323815c840 2023-09-23T03:25:31.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.649+0000 7f323d24d640 1 -- 172.21.15.154:0/2324885276 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32380b2240 con 0x7f323815c840 2023-09-23T03:25:31.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.650+0000 7f323d24d640 1 -- 172.21.15.154:0/2324885276 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32380736b0 con 0x7f323815c840 2023-09-23T03:25:31.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.651+0000 7f32357fa640 1 -- 172.21.15.154:0/2324885276 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f3220043390 con 0x7f323815c840 2023-09-23T03:25:31.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.652+0000 7f32357fa640 1 --2- 172.21.15.154:0/2324885276 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f321806af30 0x7f321806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:31.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.652+0000 7f32357fa640 1 -- 172.21.15.154:0/2324885276 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f32200ac130 con 0x7f323815c840 2023-09-23T03:25:31.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.652+0000 7f3237fff640 1 --2- 172.21.15.154:0/2324885276 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f321806af30 0x7f321806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:31.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.653+0000 7f3237fff640 1 --2- 172.21.15.154:0/2324885276 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f321806af30 0x7f321806d3f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3228000f60 tx=0x7f32280079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:31.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.657+0000 7f32357fa640 1 -- 172.21.15.154:0/2324885276 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f3220076f20 con 0x7f323815c840 2023-09-23T03:25:31.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.850+0000 7f323d24d640 1 -- 172.21.15.154:0/2324885276 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f323804e920 con 0x7f323815c840 2023-09-23T03:25:31.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.854+0000 7f32357fa640 1 -- 172.21.15.154:0/2324885276 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+18993 (secure 0 0 0) 0x7f322007a540 con 0x7f323815c840 2023-09-23T03:25:31.855 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:25:31.855 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":92,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:24:46.339781+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:24:07.834123+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-09-23T03:24:09.058217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data_extra","create_time":"2023-09-23T03:24:19.995906+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6837/1588996498":"2023-09-24T03:24:45.398760+0000","172.21.15.154:6836/1588996498":"2023-09-24T03:24:45.398760+0000","172.21.15.154:6837/3812680297":"2023-09-24T03:24:38.360723+0000","172.21.15.154:6836/3812680297":"2023-09-24T03:24:38.360723+0000","172.21.15.154:6837/2213283923":"2023-09-24T03:24:36.268262+0000","172.21.15.154:6836/2213283923":"2023-09-24T03:24:36.268262+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:25:31.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.856+0000 7f323d24d640 1 -- 172.21.15.154:0/2324885276 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f321806af30 msgr2=0x7f321806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:31.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.856+0000 7f323d24d640 1 --2- 172.21.15.154:0/2324885276 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f321806af30 0x7f321806d3f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3228000f60 tx=0x7f32280079b0 comp rx=0 tx=0).stop 2023-09-23T03:25:31.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.857+0000 7f323d24d640 1 -- 172.21.15.154:0/2324885276 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f323815c840 msgr2=0x7f323815da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:31.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.857+0000 7f323d24d640 1 --2- 172.21.15.154:0/2324885276 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f323815c840 0x7f323815da20 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f322002b9c0 tx=0x7f3220004730 comp rx=0 tx=0).stop 2023-09-23T03:25:31.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.857+0000 7f323d24d640 1 -- 172.21.15.154:0/2324885276 shutdown_connections 2023-09-23T03:25:31.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.857+0000 7f323d24d640 1 --2- 172.21.15.154:0/2324885276 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f321806af30 0x7f321806d3f0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:31.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.857+0000 7f323d24d640 1 --2- 172.21.15.154:0/2324885276 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3238160fd0 0x7f32380c78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:31.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.857+0000 7f323d24d640 1 --2- 172.21.15.154:0/2324885276 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f323815c840 0x7f323815da20 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:31.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.857+0000 7f323d24d640 1 --2- 172.21.15.154:0/2324885276 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f32380c99d0 0x7f323814fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:31.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.857+0000 7f323d24d640 1 -- 172.21.15.154:0/2324885276 >> 172.21.15.154:0/2324885276 conn(0x7f323806a390 msgr2=0x7f32380cac00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:31.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.857+0000 7f323d24d640 1 -- 172.21.15.154:0/2324885276 shutdown_connections 2023-09-23T03:25:31.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:31.857+0000 7f323d24d640 1 -- 172.21.15.154:0/2324885276 wait complete. 2023-09-23T03:25:31.876 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-09-23T03:25:32.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.091+0000 7fc88bd05640 1 Processor -- start 2023-09-23T03:25:32.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.091+0000 7fc88bd05640 1 -- start start 2023-09-23T03:25:32.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.092+0000 7fc88bd05640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc884160680 0x7fc8840b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:32.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.092+0000 7fc88bd05640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc884161500 con 0x7fc8840d25b0 2023-09-23T03:25:32.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.092+0000 7fc88bd05640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc884160c60 con 0x7fc88415ca80 2023-09-23T03:25:32.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.092+0000 7fc88bd05640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc884160f40 con 0x7fc884160680 2023-09-23T03:25:32.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.092+0000 7fc8892a0640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc884160680 0x7fc8840b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:32.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.092+0000 7fc88a2a2640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fc8840d25b0 0x7fc8840d2a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:41064/0 (socket says 172.21.15.154:41064) 2023-09-23T03:25:32.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.093+0000 7fc88a2a2640 1 -- 172.21.15.154:0/865719099 learned_addr learned my addr 172.21.15.154:0/865719099 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:25:32.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.092+0000 7fc8892a0640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc884160680 0x7fc8840b2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50834/0 (socket says 172.21.15.154:50834) 2023-09-23T03:25:32.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.093+0000 7fc8892a0640 1 -- 172.21.15.154:0/865719099 >> v1:172.21.15.190:6789/0 conn(0x7fc88415ca80 legacy=0x7fc8840d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:32.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.093+0000 7fc8892a0640 1 -- 172.21.15.154:0/865719099 >> v1:172.21.15.154:6789/0 conn(0x7fc8840d25b0 legacy=0x7fc8840d2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:32.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.093+0000 7fc8892a0640 1 -- 172.21.15.154:0/865719099 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc884161220 con 0x7fc884160680 2023-09-23T03:25:32.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.093+0000 7fc888a9f640 1 -- 172.21.15.154:0/865719099 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 585728809 0 0) 0x7fc884161500 con 0x7fc8840d25b0 2023-09-23T03:25:32.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.094+0000 7fc8892a0640 1 --2- 172.21.15.154:0/865719099 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc884160680 0x7fc8840b2620 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fc8780097a0 tx=0x7fc87802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd0d386e69eabfd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:32.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.094+0000 7fc888a9f640 1 -- 172.21.15.154:0/865719099 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc878031830 con 0x7fc884160680 2023-09-23T03:25:32.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.094+0000 7fc888a9f640 1 -- 172.21.15.154:0/865719099 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc878031990 con 0x7fc884160680 2023-09-23T03:25:32.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.094+0000 7fc888a9f640 1 -- 172.21.15.154:0/865719099 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc878031d20 con 0x7fc884160680 2023-09-23T03:25:32.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.095+0000 7fc88bd05640 1 -- 172.21.15.154:0/865719099 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc884160680 msgr2=0x7fc8840b2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:32.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.095+0000 7fc88bd05640 1 --2- 172.21.15.154:0/865719099 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc884160680 0x7fc8840b2620 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fc8780097a0 tx=0x7fc87802b8d0 comp rx=0 tx=0).stop 2023-09-23T03:25:32.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.095+0000 7fc88bd05640 1 -- 172.21.15.154:0/865719099 shutdown_connections 2023-09-23T03:25:32.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.095+0000 7fc88bd05640 1 --2- 172.21.15.154:0/865719099 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc884160680 0x7fc8840b2620 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:32.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.095+0000 7fc88bd05640 1 -- 172.21.15.154:0/865719099 >> 172.21.15.154:0/865719099 conn(0x7fc88409bc60 msgr2=0x7fc884089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:32.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.095+0000 7fc88bd05640 1 -- 172.21.15.154:0/865719099 shutdown_connections 2023-09-23T03:25:32.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.095+0000 7fc88bd05640 1 -- 172.21.15.154:0/865719099 wait complete. 2023-09-23T03:25:32.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.097+0000 7fc88bd05640 1 Processor -- start 2023-09-23T03:25:32.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.097+0000 7fc88bd05640 1 -- start start 2023-09-23T03:25:32.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.098+0000 7fc88bd05640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc8840d25b0 0x7fc88415f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:32.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.098+0000 7fc88bd05640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc88415ca80 0x7fc88415a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:32.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.098+0000 7fc88bd05640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc884160680 0x7fc88415aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:32.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.098+0000 7fc88bd05640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88415e6b0 con 0x7fc8840d25b0 2023-09-23T03:25:32.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.098+0000 7fc88bd05640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88415e990 con 0x7fc884160680 2023-09-23T03:25:32.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.098+0000 7fc88bd05640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc88415ec70 con 0x7fc88415ca80 2023-09-23T03:25:32.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.098+0000 7fc889aa1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc8840d25b0 0x7fc88415f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:32.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.098+0000 7fc889aa1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc8840d25b0 0x7fc88415f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50842/0 (socket says 172.21.15.154:50842) 2023-09-23T03:25:32.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.098+0000 7fc889aa1640 1 -- 172.21.15.154:0/324013453 learned_addr learned my addr 172.21.15.154:0/324013453 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:32.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.099+0000 7fc889aa1640 1 -- 172.21.15.154:0/324013453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc88415ca80 msgr2=0x7fc88415a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:32.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.099+0000 7fc889aa1640 1 --2- 172.21.15.154:0/324013453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc88415ca80 0x7fc88415a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:32.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.099+0000 7fc889aa1640 1 -- 172.21.15.154:0/324013453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc884160680 msgr2=0x7fc88415aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:32.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.099+0000 7fc889aa1640 1 --2- 172.21.15.154:0/324013453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc884160680 0x7fc88415aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:32.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.099+0000 7fc889aa1640 1 -- 172.21.15.154:0/324013453 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc878009400 con 0x7fc8840d25b0 2023-09-23T03:25:32.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.099+0000 7fc889aa1640 1 --2- 172.21.15.154:0/324013453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc8840d25b0 0x7fc88415f2e0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fc87400c730 tx=0x7fc87400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:32.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.099+0000 7fc8827fc640 1 -- 172.21.15.154:0/324013453 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc874003bc0 con 0x7fc8840d25b0 2023-09-23T03:25:32.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.099+0000 7fc8827fc640 1 -- 172.21.15.154:0/324013453 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc874003d40 con 0x7fc8840d25b0 2023-09-23T03:25:32.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.099+0000 7fc8827fc640 1 -- 172.21.15.154:0/324013453 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87400aa90 con 0x7fc8840d25b0 2023-09-23T03:25:32.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.099+0000 7fc88bd05640 1 -- 172.21.15.154:0/324013453 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc88415ef50 con 0x7fc8840d25b0 2023-09-23T03:25:32.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.100+0000 7fc88bd05640 1 -- 172.21.15.154:0/324013453 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc88415f820 con 0x7fc8840d25b0 2023-09-23T03:25:32.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.101+0000 7fc88bd05640 1 -- 172.21.15.154:0/324013453 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc884073eb0 con 0x7fc8840d25b0 2023-09-23T03:25:32.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.102+0000 7fc8827fc640 1 -- 172.21.15.154:0/324013453 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fc87400abf0 con 0x7fc8840d25b0 2023-09-23T03:25:32.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.102+0000 7fc8827fc640 1 --2- 172.21.15.154:0/324013453 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc87006af30 0x7fc87006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:32.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.103+0000 7fc8827fc640 1 -- 172.21.15.154:0/324013453 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fc874089a00 con 0x7fc8840d25b0 2023-09-23T03:25:32.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.103+0000 7fc8892a0640 1 --2- 172.21.15.154:0/324013453 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc87006af30 0x7fc87006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:32.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.103+0000 7fc8892a0640 1 --2- 172.21.15.154:0/324013453 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc87006af30 0x7fc87006d3f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc878036000 tx=0x7fc878030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:32.108 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.107+0000 7fc8827fc640 1 -- 172.21.15.154:0/324013453 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fc874054650 con 0x7fc8840d25b0 2023-09-23T03:25:32.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.318+0000 7fc88bd05640 1 -- 172.21.15.154:0/324013453 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc8840b87f0 con 0x7fc8840d25b0 2023-09-23T03:25:32.321 INFO:tasks.ceph.mon.a.smithi154.stderr:2023-09-23T03:25:32.320+0000 7f3df35a6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-23T03:25:32.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.324+0000 7fc8827fc640 1 -- 172.21.15.154:0/324013453 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v77) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc874057c70 con 0x7fc8840d25b0 2023-09-23T03:25:32.325 INFO:teuthology.orchestra.run.smithi154.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-09-23T03:25:32.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.326+0000 7fc88bd05640 1 -- 172.21.15.154:0/324013453 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc87006af30 msgr2=0x7fc87006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:32.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.326+0000 7fc88bd05640 1 --2- 172.21.15.154:0/324013453 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc87006af30 0x7fc87006d3f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc878036000 tx=0x7fc878030000 comp rx=0 tx=0).stop 2023-09-23T03:25:32.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.326+0000 7fc88bd05640 1 -- 172.21.15.154:0/324013453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc8840d25b0 msgr2=0x7fc88415f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:32.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.326+0000 7fc88bd05640 1 --2- 172.21.15.154:0/324013453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc8840d25b0 0x7fc88415f2e0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fc87400c730 tx=0x7fc87400cbf0 comp rx=0 tx=0).stop 2023-09-23T03:25:32.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.327+0000 7fc88bd05640 1 -- 172.21.15.154:0/324013453 shutdown_connections 2023-09-23T03:25:32.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.327+0000 7fc88bd05640 1 --2- 172.21.15.154:0/324013453 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc87006af30 0x7fc87006d3f0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:32.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.327+0000 7fc88bd05640 1 --2- 172.21.15.154:0/324013453 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc884160680 0x7fc88415aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:32.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.327+0000 7fc88bd05640 1 --2- 172.21.15.154:0/324013453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc88415ca80 0x7fc88415a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:32.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.327+0000 7fc88bd05640 1 --2- 172.21.15.154:0/324013453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc8840d25b0 0x7fc88415f2e0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:32.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.327+0000 7fc88bd05640 1 -- 172.21.15.154:0/324013453 >> 172.21.15.154:0/324013453 conn(0x7fc88409bc60 msgr2=0x7fc8840ce730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:32.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.327+0000 7fc88bd05640 1 -- 172.21.15.154:0/324013453 shutdown_connections 2023-09-23T03:25:32.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.327+0000 7fc88bd05640 1 -- 172.21.15.154:0/324013453 wait complete. 2023-09-23T03:25:32.346 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-09-23T03:25:32.347 INFO:tasks.ceph.mds.b.smithi190.stderr: -4533> 2023-09-23T03:24:45.438+0000 7fce45bea640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-09-23T03:25:32.360 INFO:tasks.ceph.mds.b.smithi190.stderr: -2668> 2023-09-23T03:24:52.362+0000 7fce45bea640 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2023-09-23T03:25:32.361 INFO:tasks.ceph.mds.b.smithi190.stderr: -2505> 2023-09-23T03:24:52.365+0000 7fce45bea640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /subdir/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x5584d4d38580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(11)0x10000000000:[<0x1/subdir v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-09-23T03:25:32.362 INFO:tasks.ceph.mds.b.smithi190.stderr: -2451> 2023-09-23T03:24:52.367+0000 7fce45bea640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/sixmegs auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x5584d4de2680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(13)0x10000000001:[<0x10000000000/sixmegs v8>,<0x1/subdir v10>]//[12]","memoryvalue":"(13)0x10000000001:[<0x10000000000/sixmegs v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:25:32.364 INFO:tasks.ceph.mds.b.smithi190.stderr: -2155> 2023-09-23T03:24:52.376+0000 7fce45bea640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v5 ap=5 snaprealm=0x5584d4d52d80 DIRTYPARENT f(v1 1=0+1) n(v0 rc2023-09-23T03:24:52.077396+0000 1=0+1)/n() (idft sync r=1) (inest sync r=1) (ifile sync->mix r=1) (iversion lock) caps={7818=pLsXs/-@3} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x5584d4d3a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(11)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=0+1)","ondisk_value.rstat":"n(v0 2=0+2)","memory_value.dirstat":"f(v1 1=0+1)","memory_value.rstat":"n(v0 rc2023-09-23T03:24:52.077396+0000 1=0+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-09-23T03:25:32.452 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-09-23T03:25:32.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.547+0000 7fd41bac9640 1 Processor -- start 2023-09-23T03:25:32.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.547+0000 7fd41bac9640 1 -- start start 2023-09-23T03:25:32.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.548+0000 7fd41bac9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd41415bb30 0x7fd4140caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:32.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.548+0000 7fd41bac9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd4140cb3e0 0x7fd4140cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:32.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.548+0000 7fd41bac9640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd41415b6a0 con 0x7fd41415aba0 2023-09-23T03:25:32.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.548+0000 7fd41bac9640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4140cb870 con 0x7fd41415bb30 2023-09-23T03:25:32.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.548+0000 7fd41bac9640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd41415bf70 con 0x7fd4140cb3e0 2023-09-23T03:25:32.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.549+0000 7fd419064640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd41415bb30 0x7fd4140caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:32.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.549+0000 7fd419064640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd41415bb30 0x7fd4140caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50850/0 (socket says 172.21.15.154:50850) 2023-09-23T03:25:32.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.549+0000 7fd419064640 1 -- 172.21.15.154:0/909725104 learned_addr learned my addr 172.21.15.154:0/909725104 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:32.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.549+0000 7fd41a066640 1 --2- 172.21.15.154:0/909725104 >> v2:172.21.15.190:3300/0 conn(0x7fd4140cb3e0 0x7fd4140cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:32.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.549+0000 7fd419064640 1 -- 172.21.15.154:0/909725104 >> v2:172.21.15.190:3300/0 conn(0x7fd4140cb3e0 msgr2=0x7fd4140cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:32.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.549+0000 7fd419064640 1 --2- 172.21.15.154:0/909725104 >> v2:172.21.15.190:3300/0 conn(0x7fd4140cb3e0 0x7fd4140cc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:32.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.549+0000 7fd419064640 1 -- 172.21.15.154:0/909725104 >> v1:172.21.15.190:6789/0 conn(0x7fd41415aba0 legacy=0x7fd41415e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:32.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.549+0000 7fd419064640 1 -- 172.21.15.154:0/909725104 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd41415f0b0 con 0x7fd41415bb30 2023-09-23T03:25:32.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.550+0000 7fd419064640 1 --2- 172.21.15.154:0/909725104 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd41415bb30 0x7fd4140caea0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fd4140c1f80 tx=0x7fd40402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48765fb3053193ab server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:32.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.551+0000 7fd418863640 1 -- 172.21.15.154:0/909725104 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd404031870 con 0x7fd41415bb30 2023-09-23T03:25:32.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.551+0000 7fd418863640 1 -- 172.21.15.154:0/909725104 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4040319d0 con 0x7fd41415bb30 2023-09-23T03:25:32.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.551+0000 7fd418863640 1 -- 172.21.15.154:0/909725104 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd404031d60 con 0x7fd41415bb30 2023-09-23T03:25:32.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.551+0000 7fd41bac9640 1 -- 172.21.15.154:0/909725104 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd41415bb30 msgr2=0x7fd4140caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:32.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.551+0000 7fd41bac9640 1 --2- 172.21.15.154:0/909725104 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd41415bb30 0x7fd4140caea0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fd4140c1f80 tx=0x7fd40402b910 comp rx=0 tx=0).stop 2023-09-23T03:25:32.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.551+0000 7fd41bac9640 1 -- 172.21.15.154:0/909725104 shutdown_connections 2023-09-23T03:25:32.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.551+0000 7fd41bac9640 1 --2- 172.21.15.154:0/909725104 >> v2:172.21.15.190:3300/0 conn(0x7fd4140cb3e0 0x7fd4140cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:32.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.551+0000 7fd41bac9640 1 --2- 172.21.15.154:0/909725104 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd41415bb30 0x7fd4140caea0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:32.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.551+0000 7fd41bac9640 1 -- 172.21.15.154:0/909725104 >> 172.21.15.154:0/909725104 conn(0x7fd41406c010 msgr2=0x7fd41406c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:32.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.552+0000 7fd41bac9640 1 -- 172.21.15.154:0/909725104 shutdown_connections 2023-09-23T03:25:32.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.552+0000 7fd41bac9640 1 -- 172.21.15.154:0/909725104 wait complete. 2023-09-23T03:25:32.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.553+0000 7fd41bac9640 1 Processor -- start 2023-09-23T03:25:32.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.553+0000 7fd41bac9640 1 -- start start 2023-09-23T03:25:32.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.554+0000 7fd41bac9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4140cb3e0 0x7fd4141489f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:32.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.554+0000 7fd41bac9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd41415aba0 0x7fd4141531c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:32.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.554+0000 7fd41bac9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd41415bb30 0x7fd4140ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:32.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.555+0000 7fd41bac9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd414154820 con 0x7fd41415aba0 2023-09-23T03:25:32.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.555+0000 7fd41bac9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd414153c10 con 0x7fd4140cb3e0 2023-09-23T03:25:32.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.555+0000 7fd41bac9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd414153ef0 con 0x7fd41415bb30 2023-09-23T03:25:32.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.555+0000 7fd419064640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd41415aba0 0x7fd4141531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:32.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.555+0000 7fd419064640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd41415aba0 0x7fd4141531c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50854/0 (socket says 172.21.15.154:50854) 2023-09-23T03:25:32.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.555+0000 7fd419064640 1 -- 172.21.15.154:0/3256406003 learned_addr learned my addr 172.21.15.154:0/3256406003 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:32.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.555+0000 7fd419865640 1 --2- 172.21.15.154:0/3256406003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4140cb3e0 0x7fd4141489f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:32.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.555+0000 7fd41a066640 1 --2- 172.21.15.154:0/3256406003 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd41415bb30 0x7fd4140ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:32.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.555+0000 7fd419064640 1 -- 172.21.15.154:0/3256406003 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd41415bb30 msgr2=0x7fd4140ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:32.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.555+0000 7fd419064640 1 --2- 172.21.15.154:0/3256406003 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd41415bb30 0x7fd4140ab8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:32.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.555+0000 7fd419064640 1 -- 172.21.15.154:0/3256406003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4140cb3e0 msgr2=0x7fd4141489f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:32.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.555+0000 7fd419064640 1 --2- 172.21.15.154:0/3256406003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4140cb3e0 0x7fd4141489f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:32.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.555+0000 7fd419064640 1 -- 172.21.15.154:0/3256406003 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4040094d0 con 0x7fd41415aba0 2023-09-23T03:25:32.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.556+0000 7fd419064640 1 --2- 172.21.15.154:0/3256406003 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd41415aba0 0x7fd4141531c0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fd40402b950 tx=0x7fd4040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:32.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.556+0000 7fd40a7fc640 1 -- 172.21.15.154:0/3256406003 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40403bcc0 con 0x7fd41415aba0 2023-09-23T03:25:32.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.556+0000 7fd40a7fc640 1 -- 172.21.15.154:0/3256406003 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd40403be40 con 0x7fd41415aba0 2023-09-23T03:25:32.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.556+0000 7fd41bac9640 1 -- 172.21.15.154:0/3256406003 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4141541d0 con 0x7fd41415aba0 2023-09-23T03:25:32.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.556+0000 7fd40a7fc640 1 -- 172.21.15.154:0/3256406003 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd404045630 con 0x7fd41415aba0 2023-09-23T03:25:32.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.556+0000 7fd41bac9640 1 -- 172.21.15.154:0/3256406003 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd414154cb0 con 0x7fd41415aba0 2023-09-23T03:25:32.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.557+0000 7fd41bac9640 1 -- 172.21.15.154:0/3256406003 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd414154990 con 0x7fd41415aba0 2023-09-23T03:25:32.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.558+0000 7fd40a7fc640 1 -- 172.21.15.154:0/3256406003 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fd40404e450 con 0x7fd41415aba0 2023-09-23T03:25:32.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.560+0000 7fd40a7fc640 1 --2- 172.21.15.154:0/3256406003 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd3f806af60 0x7fd3f806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:32.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.560+0000 7fd40a7fc640 1 -- 172.21.15.154:0/3256406003 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6777+0+0 (secure 0 0 0) 0x7fd40402d040 con 0x7fd41415aba0 2023-09-23T03:25:32.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.560+0000 7fd419865640 1 --2- 172.21.15.154:0/3256406003 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd3f806af60 0x7fd3f806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:32.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.561+0000 7fd419865640 1 --2- 172.21.15.154:0/3256406003 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd3f806af60 0x7fd3f806d420 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd410004ea0 tx=0x7fd410009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:32.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.564+0000 7fd40a7fc640 1 -- 172.21.15.154:0/3256406003 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fd404076db0 con 0x7fd41415aba0 2023-09-23T03:25:32.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:32.773+0000 7fd41bac9640 1 -- 172.21.15.154:0/3256406003 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd41404e920 con 0x7fd41415aba0 2023-09-23T03:25:33.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.422+0000 7fd40a7fc640 1 -- 172.21.15.154:0/3256406003 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v78) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd40407a3d0 con 0x7fd41415aba0 2023-09-23T03:25:33.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.425+0000 7fd41bac9640 1 -- 172.21.15.154:0/3256406003 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd3f806af60 msgr2=0x7fd3f806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:33.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.425+0000 7fd41bac9640 1 --2- 172.21.15.154:0/3256406003 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd3f806af60 0x7fd3f806d420 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd410004ea0 tx=0x7fd410009040 comp rx=0 tx=0).stop 2023-09-23T03:25:33.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.425+0000 7fd41bac9640 1 -- 172.21.15.154:0/3256406003 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd41415aba0 msgr2=0x7fd4141531c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:33.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.425+0000 7fd41bac9640 1 --2- 172.21.15.154:0/3256406003 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd41415aba0 0x7fd4141531c0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fd40402b950 tx=0x7fd4040046c0 comp rx=0 tx=0).stop 2023-09-23T03:25:33.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.426+0000 7fd41bac9640 1 -- 172.21.15.154:0/3256406003 shutdown_connections 2023-09-23T03:25:33.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.426+0000 7fd41bac9640 1 --2- 172.21.15.154:0/3256406003 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd3f806af60 0x7fd3f806d420 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:33.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.426+0000 7fd41bac9640 1 --2- 172.21.15.154:0/3256406003 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd41415bb30 0x7fd4140ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:33.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.426+0000 7fd41bac9640 1 --2- 172.21.15.154:0/3256406003 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd41415aba0 0x7fd4141531c0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:33.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.426+0000 7fd41bac9640 1 --2- 172.21.15.154:0/3256406003 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4140cb3e0 0x7fd4141489f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:33.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.426+0000 7fd41bac9640 1 -- 172.21.15.154:0/3256406003 >> 172.21.15.154:0/3256406003 conn(0x7fd41406c010 msgr2=0x7fd414070780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:33.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.426+0000 7fd41bac9640 1 -- 172.21.15.154:0/3256406003 shutdown_connections 2023-09-23T03:25:33.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.426+0000 7fd41bac9640 1 -- 172.21.15.154:0/3256406003 wait complete. 2023-09-23T03:25:33.450 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-09-23T03:25:33.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.670+0000 7f50c2ead640 1 Processor -- start 2023-09-23T03:25:33.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.670+0000 7f50c2ead640 1 -- start start 2023-09-23T03:25:33.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.672+0000 7f50c2ead640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f50bc06c740 0x7f50bc0ab030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:33.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.672+0000 7f50c2ead640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50bc06c0e0 con 0x7f50bc0b1870 2023-09-23T03:25:33.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.672+0000 7f50c2ead640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50bc0abba0 con 0x7f50bc0acc40 2023-09-23T03:25:33.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.672+0000 7f50c2ead640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50bc0abe80 con 0x7f50bc06c740 2023-09-23T03:25:33.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.672+0000 7f50c26ac640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f50bc0b1870 0x7f50bc0c8960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:41076/0 (socket says 172.21.15.154:41076) 2023-09-23T03:25:33.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.672+0000 7f50c26ac640 1 -- 172.21.15.154:0/449800729 learned_addr learned my addr 172.21.15.154:0/449800729 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:25:33.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.672+0000 7f50c1eab640 1 --2- 172.21.15.154:0/449800729 >> v2:172.21.15.190:3300/0 conn(0x7f50bc06c740 0x7f50bc0ab030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:33.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.673+0000 7f50c0ea9640 1 -- 172.21.15.154:0/449800729 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3347339995 0 0) 0x7f50bc06c0e0 con 0x7f50bc0b1870 2023-09-23T03:25:33.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.673+0000 7f50c0ea9640 1 -- 172.21.15.154:0/449800729 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50a8005730 con 0x7f50bc0b1870 2023-09-23T03:25:33.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.673+0000 7f50c0ea9640 1 -- 172.21.15.154:0/449800729 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2473004818 0 0) 0x7f50a8005730 con 0x7f50bc0b1870 2023-09-23T03:25:33.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.673+0000 7f50c0ea9640 1 -- 172.21.15.154:0/449800729 >> v2:172.21.15.190:3300/0 conn(0x7f50bc06c740 msgr2=0x7f50bc0ab030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:33.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.673+0000 7f50c0ea9640 1 --2- 172.21.15.154:0/449800729 >> v2:172.21.15.190:3300/0 conn(0x7f50bc06c740 0x7f50bc0ab030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:33.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.673+0000 7f50c0ea9640 1 -- 172.21.15.154:0/449800729 >> v1:172.21.15.190:6789/0 conn(0x7f50bc0acc40 legacy=0x7f50bc0b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:33.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.673+0000 7f50c0ea9640 1 -- 172.21.15.154:0/449800729 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50bc0ac160 con 0x7f50bc0b1870 2023-09-23T03:25:33.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.673+0000 7f50c0ea9640 1 -- 172.21.15.154:0/449800729 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f50b40019f0 con 0x7f50bc0b1870 2023-09-23T03:25:33.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.673+0000 7f50c0ea9640 1 -- 172.21.15.154:0/449800729 >> v1:172.21.15.154:6789/0 conn(0x7f50bc0b1870 legacy=0x7f50bc0c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:33.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.673+0000 7f50c0ea9640 1 --2- 172.21.15.154:0/449800729 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50a8005b80 0x7f50a8005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:33.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.673+0000 7f50c0ea9640 1 --2- 172.21.15.154:0/449800729 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50a80069d0 0x7f50a8006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:33.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.673+0000 7f50c0ea9640 1 --2- 172.21.15.154:0/449800729 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50a8007390 0x7f50a800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:33.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.673+0000 7f50c0ea9640 1 -- 172.21.15.154:0/449800729 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc0abe80 con 0x7f50a80069d0 2023-09-23T03:25:33.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.673+0000 7f50c0ea9640 1 -- 172.21.15.154:0/449800729 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a800e230 con 0x7f50a8007390 2023-09-23T03:25:33.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.673+0000 7f50c0ea9640 1 -- 172.21.15.154:0/449800729 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50a800e3f0 con 0x7f50a8005b80 2023-09-23T03:25:33.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.673+0000 7f50c16aa640 1 --2- 172.21.15.154:0/449800729 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50a80069d0 0x7f50a8006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:33.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.674+0000 7f50c1eab640 1 --2- 172.21.15.154:0/449800729 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50a8005b80 0x7f50a8005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:33.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.674+0000 7f50c26ac640 1 --2- 172.21.15.154:0/449800729 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50a8007390 0x7f50a800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:33.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.674+0000 7f50c16aa640 1 -- 172.21.15.154:0/449800729 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50a8005b80 msgr2=0x7f50a8005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:33.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.674+0000 7f50c16aa640 1 --2- 172.21.15.154:0/449800729 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50a8005b80 0x7f50a8005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:33.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.674+0000 7f50c16aa640 1 -- 172.21.15.154:0/449800729 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50a8007390 msgr2=0x7f50a800db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:33.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.674+0000 7f50c16aa640 1 --2- 172.21.15.154:0/449800729 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50a8007390 0x7f50a800db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:33.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.674+0000 7f50c16aa640 1 -- 172.21.15.154:0/449800729 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a8005730 con 0x7f50a80069d0 2023-09-23T03:25:33.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.674+0000 7f50c26ac640 1 -- 172.21.15.154:0/449800729 reap_dead start 2023-09-23T03:25:33.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.675+0000 7f50c16aa640 1 --2- 172.21.15.154:0/449800729 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50a80069d0 0x7f50a8006e50 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f50ac001750 tx=0x7f50a8007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc7385c0d6f0708 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:33.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.675+0000 7f50c0ea9640 1 -- 172.21.15.154:0/449800729 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ac0052d0 con 0x7f50a80069d0 2023-09-23T03:25:33.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.675+0000 7f50c0ea9640 1 -- 172.21.15.154:0/449800729 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50ac02e050 con 0x7f50a80069d0 2023-09-23T03:25:33.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.675+0000 7f50c0ea9640 1 -- 172.21.15.154:0/449800729 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ac003830 con 0x7f50a80069d0 2023-09-23T03:25:33.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.676+0000 7f50c2ead640 1 -- 172.21.15.154:0/449800729 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50a80069d0 msgr2=0x7f50a8006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:33.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.676+0000 7f50c2ead640 1 --2- 172.21.15.154:0/449800729 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50a80069d0 0x7f50a8006e50 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f50ac001750 tx=0x7f50a8007a00 comp rx=0 tx=0).stop 2023-09-23T03:25:33.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.676+0000 7f50c2ead640 1 -- 172.21.15.154:0/449800729 shutdown_connections 2023-09-23T03:25:33.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.676+0000 7f50c2ead640 1 --2- 172.21.15.154:0/449800729 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50a80069d0 0x7f50a8006e50 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:33.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.676+0000 7f50c2ead640 1 -- 172.21.15.154:0/449800729 >> 172.21.15.154:0/449800729 conn(0x7f50bc069eb0 msgr2=0x7f50bc06a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:33.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.676+0000 7f50c2ead640 1 -- 172.21.15.154:0/449800729 shutdown_connections 2023-09-23T03:25:33.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.676+0000 7f50c2ead640 1 -- 172.21.15.154:0/449800729 wait complete. 2023-09-23T03:25:33.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.678+0000 7f50c2ead640 1 Processor -- start 2023-09-23T03:25:33.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.678+0000 7f50c2ead640 1 -- start start 2023-09-23T03:25:33.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.679+0000 7f50c2ead640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50a80069d0 0x7f50bc157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:33.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.679+0000 7f50c2ead640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50bc0c5df0 0x7f50bc09b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:33.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.679+0000 7f50c2ead640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50bc0cd260 0x7f50bc0cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:33.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.679+0000 7f50c2ead640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc157ea0 con 0x7f50bc0cd260 2023-09-23T03:25:33.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.679+0000 7f50c2ead640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc0cdc20 con 0x7f50a80069d0 2023-09-23T03:25:33.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.679+0000 7f50c2ead640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc0c7160 con 0x7f50bc0c5df0 2023-09-23T03:25:33.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.679+0000 7f50c26ac640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50bc0cd260 0x7f50bc0cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:33.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.679+0000 7f50c26ac640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50bc0cd260 0x7f50bc0cd6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50876/0 (socket says 172.21.15.154:50876) 2023-09-23T03:25:33.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.679+0000 7f50c26ac640 1 -- 172.21.15.154:0/2668339816 learned_addr learned my addr 172.21.15.154:0/2668339816 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:33.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.679+0000 7f50c1eab640 1 --2- 172.21.15.154:0/2668339816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50a80069d0 0x7f50bc157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:33.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.679+0000 7f50c16aa640 1 --2- 172.21.15.154:0/2668339816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50bc0c5df0 0x7f50bc09b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:33.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.679+0000 7f50c26ac640 1 -- 172.21.15.154:0/2668339816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50bc0c5df0 msgr2=0x7f50bc09b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:33.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.679+0000 7f50c26ac640 1 --2- 172.21.15.154:0/2668339816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50bc0c5df0 0x7f50bc09b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:33.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.679+0000 7f50c26ac640 1 -- 172.21.15.154:0/2668339816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50a80069d0 msgr2=0x7f50bc157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:33.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.679+0000 7f50c26ac640 1 --2- 172.21.15.154:0/2668339816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50a80069d0 0x7f50bc157800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:33.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.680+0000 7f50c26ac640 1 -- 172.21.15.154:0/2668339816 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50bc09b870 con 0x7f50bc0cd260 2023-09-23T03:25:33.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.680+0000 7f50c1eab640 1 --2- 172.21.15.154:0/2668339816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50a80069d0 0x7f50bc157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:33.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.680+0000 7f50c26ac640 1 --2- 172.21.15.154:0/2668339816 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50bc0cd260 0x7f50bc0cd6e0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f50bc0b33e0 tx=0x7f50b4007af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:33.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.680+0000 7f50b27fc640 1 -- 172.21.15.154:0/2668339816 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b4017070 con 0x7f50bc0cd260 2023-09-23T03:25:33.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.680+0000 7f50b27fc640 1 -- 172.21.15.154:0/2668339816 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b400e9c0 con 0x7f50bc0cd260 2023-09-23T03:25:33.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.680+0000 7f50b27fc640 1 -- 172.21.15.154:0/2668339816 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b400eb50 con 0x7f50bc0cd260 2023-09-23T03:25:33.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.680+0000 7f50c2ead640 1 -- 172.21.15.154:0/2668339816 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50bc0c6830 con 0x7f50bc0cd260 2023-09-23T03:25:33.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.680+0000 7f50c2ead640 1 -- 172.21.15.154:0/2668339816 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50bc0c6df0 con 0x7f50bc0cd260 2023-09-23T03:25:33.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.681+0000 7f50c2ead640 1 -- 172.21.15.154:0/2668339816 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50bc0c72f0 con 0x7f50bc0cd260 2023-09-23T03:25:33.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.682+0000 7f50b27fc640 1 -- 172.21.15.154:0/2668339816 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f50b400ecb0 con 0x7f50bc0cd260 2023-09-23T03:25:33.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.683+0000 7f50b27fc640 1 --2- 172.21.15.154:0/2668339816 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f50a006af30 0x7f50a006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:33.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.683+0000 7f50b27fc640 1 -- 172.21.15.154:0/2668339816 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f50b4089630 con 0x7f50bc0cd260 2023-09-23T03:25:33.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.684+0000 7f50c1eab640 1 --2- 172.21.15.154:0/2668339816 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f50a006af30 0x7f50a006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:33.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.684+0000 7f50c1eab640 1 --2- 172.21.15.154:0/2668339816 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f50a006af30 0x7f50a006d3f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f50b8009a80 tx=0x7f50b8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:33.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.687+0000 7f50b27fc640 1 -- 172.21.15.154:0/2668339816 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f50b4054220 con 0x7f50bc0cd260 2023-09-23T03:25:33.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:33.876+0000 7f50c2ead640 1 -- 172.21.15.154:0/2668339816 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f50bc0ac6f0 con 0x7f50bc0cd260 2023-09-23T03:25:34.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.434+0000 7f50b27fc640 1 -- 172.21.15.154:0/2668339816 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v94) v1 ==== 181+0+0 (secure 0 0 0) 0x7f50b4057840 con 0x7f50bc0cd260 2023-09-23T03:25:34.436 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_metadata' removed 2023-09-23T03:25:34.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.440+0000 7f50c2ead640 1 -- 172.21.15.154:0/2668339816 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f50a006af30 msgr2=0x7f50a006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:34.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.440+0000 7f50c2ead640 1 --2- 172.21.15.154:0/2668339816 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f50a006af30 0x7f50a006d3f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f50b8009a80 tx=0x7f50b8008040 comp rx=0 tx=0).stop 2023-09-23T03:25:34.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.440+0000 7f50c2ead640 1 -- 172.21.15.154:0/2668339816 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50bc0cd260 msgr2=0x7f50bc0cd6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:34.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.440+0000 7f50c2ead640 1 --2- 172.21.15.154:0/2668339816 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50bc0cd260 0x7f50bc0cd6e0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f50bc0b33e0 tx=0x7f50b4007af0 comp rx=0 tx=0).stop 2023-09-23T03:25:34.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.441+0000 7f50c2ead640 1 -- 172.21.15.154:0/2668339816 shutdown_connections 2023-09-23T03:25:34.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.441+0000 7f50c2ead640 1 --2- 172.21.15.154:0/2668339816 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f50a006af30 0x7f50a006d3f0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:34.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.441+0000 7f50c2ead640 1 --2- 172.21.15.154:0/2668339816 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50bc0cd260 0x7f50bc0cd6e0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:34.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.441+0000 7f50c2ead640 1 --2- 172.21.15.154:0/2668339816 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50bc0c5df0 0x7f50bc09b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:34.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.441+0000 7f50c2ead640 1 --2- 172.21.15.154:0/2668339816 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50a80069d0 0x7f50bc157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:34.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.441+0000 7f50c2ead640 1 -- 172.21.15.154:0/2668339816 >> 172.21.15.154:0/2668339816 conn(0x7f50bc069eb0 msgr2=0x7f50bc0af190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:34.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.441+0000 7f50c2ead640 1 -- 172.21.15.154:0/2668339816 shutdown_connections 2023-09-23T03:25:34.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.441+0000 7f50c2ead640 1 -- 172.21.15.154:0/2668339816 wait complete. 2023-09-23T03:25:34.462 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-09-23T03:25:34.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.679+0000 7f98b0d27640 1 Processor -- start 2023-09-23T03:25:34.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.679+0000 7f98b0d27640 1 -- start start 2023-09-23T03:25:34.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.681+0000 7f98b0d27640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f98ac15a860 0x7f98ac0cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:34.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.681+0000 7f98b0d27640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98ac15e350 0x7f98ac0cad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:34.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.681+0000 7f98b0d27640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98ac15ac40 con 0x7f98ac0ce2a0 2023-09-23T03:25:34.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.681+0000 7f98b0d27640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98ac15e860 con 0x7f98ac15a860 2023-09-23T03:25:34.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.681+0000 7f98b0d27640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac15eb20 con 0x7f98ac15e350 2023-09-23T03:25:34.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.681+0000 7f98ab7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f98ac15a860 0x7f98ac0cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:34.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.681+0000 7f98ab7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f98ac15a860 0x7f98ac0cc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49980/0 (socket says 172.21.15.154:49980) 2023-09-23T03:25:34.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.681+0000 7f98abfff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f98ac0ce2a0 0x7f98ac0b25e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46854/0 (socket says 172.21.15.154:46854) 2023-09-23T03:25:34.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.681+0000 7f98ab7fe640 1 -- 172.21.15.154:0/2142601775 learned_addr learned my addr 172.21.15.154:0/2142601775 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:34.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.682+0000 7f98aaffd640 1 --2- 172.21.15.154:0/2142601775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98ac15e350 0x7f98ac0cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:34.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.682+0000 7f98ab7fe640 1 -- 172.21.15.154:0/2142601775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98ac15e350 msgr2=0x7f98ac0cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:34.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.682+0000 7f98ab7fe640 1 --2- 172.21.15.154:0/2142601775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98ac15e350 0x7f98ac0cad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:34.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.682+0000 7f98ab7fe640 1 -- 172.21.15.154:0/2142601775 >> v1:172.21.15.154:6789/0 conn(0x7f98ac0ce2a0 legacy=0x7f98ac0b25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:34.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.682+0000 7f98ab7fe640 1 -- 172.21.15.154:0/2142601775 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ac15ee00 con 0x7f98ac15a860 2023-09-23T03:25:34.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.682+0000 7f98aa7fc640 1 -- 172.21.15.154:0/2142601775 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3718211215 0 0) 0x7f98ac15ac40 con 0x7f98ac0ce2a0 2023-09-23T03:25:34.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.683+0000 7f98ab7fe640 1 --2- 172.21.15.154:0/2142601775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98ac15a860 0x7f98ac0cc210 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f989c009870 tx=0x7f989c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9729eff4282865d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:34.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.683+0000 7f98aa7fc640 1 -- 172.21.15.154:0/2142601775 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c030840 con 0x7f98ac15a860 2023-09-23T03:25:34.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.683+0000 7f98aa7fc640 1 -- 172.21.15.154:0/2142601775 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989c0309a0 con 0x7f98ac15a860 2023-09-23T03:25:34.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.683+0000 7f98aa7fc640 1 -- 172.21.15.154:0/2142601775 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c030d30 con 0x7f98ac15a860 2023-09-23T03:25:34.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.684+0000 7f98b0d27640 1 -- 172.21.15.154:0/2142601775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98ac15a860 msgr2=0x7f98ac0cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:34.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.684+0000 7f98b0d27640 1 --2- 172.21.15.154:0/2142601775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98ac15a860 0x7f98ac0cc210 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f989c009870 tx=0x7f989c02b980 comp rx=0 tx=0).stop 2023-09-23T03:25:34.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.684+0000 7f98b0d27640 1 -- 172.21.15.154:0/2142601775 shutdown_connections 2023-09-23T03:25:34.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.684+0000 7f98b0d27640 1 --2- 172.21.15.154:0/2142601775 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98ac15e350 0x7f98ac0cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:34.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.684+0000 7f98b0d27640 1 --2- 172.21.15.154:0/2142601775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98ac15a860 0x7f98ac0cc210 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:34.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.684+0000 7f98b0d27640 1 -- 172.21.15.154:0/2142601775 >> 172.21.15.154:0/2142601775 conn(0x7f98ac069eb0 msgr2=0x7f98ac06a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:34.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.684+0000 7f98b0d27640 1 -- 172.21.15.154:0/2142601775 shutdown_connections 2023-09-23T03:25:34.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.684+0000 7f98b0d27640 1 -- 172.21.15.154:0/2142601775 wait complete. 2023-09-23T03:25:34.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.686+0000 7f98b0d27640 1 Processor -- start 2023-09-23T03:25:34.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.686+0000 7f98b0d27640 1 -- start start 2023-09-23T03:25:34.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.687+0000 7f98b0d27640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98ac0ce2a0 0x7f98ac0c8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:34.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.687+0000 7f98b0d27640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98ac15a860 0x7f98ac0c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:34.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.687+0000 7f98b0d27640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98ac15e350 0x7f98ac0c3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:34.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.687+0000 7f98b0d27640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac0aeef0 con 0x7f98ac15a860 2023-09-23T03:25:34.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.687+0000 7f98b0d27640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac0c34e0 con 0x7f98ac0ce2a0 2023-09-23T03:25:34.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.687+0000 7f98b0d27640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac0c93e0 con 0x7f98ac15e350 2023-09-23T03:25:34.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.687+0000 7f98aaffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98ac15a860 0x7f98ac0c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:34.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.687+0000 7f98aaffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98ac15a860 0x7f98ac0c8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49988/0 (socket says 172.21.15.154:49988) 2023-09-23T03:25:34.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.687+0000 7f98aaffd640 1 -- 172.21.15.154:0/1672717476 learned_addr learned my addr 172.21.15.154:0/1672717476 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:34.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.688+0000 7f98ab7fe640 1 --2- 172.21.15.154:0/1672717476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98ac0ce2a0 0x7f98ac0c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:34.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.688+0000 7f98aaffd640 1 -- 172.21.15.154:0/1672717476 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98ac15e350 msgr2=0x7f98ac0c3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:34.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.688+0000 7f98abfff640 1 --2- 172.21.15.154:0/1672717476 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98ac15e350 0x7f98ac0c3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:34.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.688+0000 7f98aaffd640 1 --2- 172.21.15.154:0/1672717476 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98ac15e350 0x7f98ac0c3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:34.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.688+0000 7f98aaffd640 1 -- 172.21.15.154:0/1672717476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98ac0ce2a0 msgr2=0x7f98ac0c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:34.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.688+0000 7f98aaffd640 1 --2- 172.21.15.154:0/1672717476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98ac0ce2a0 0x7f98ac0c8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:34.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.688+0000 7f98aaffd640 1 -- 172.21.15.154:0/1672717476 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989c0094d0 con 0x7f98ac15a860 2023-09-23T03:25:34.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.688+0000 7f98ab7fe640 1 --2- 172.21.15.154:0/1672717476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98ac0ce2a0 0x7f98ac0c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:34.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.688+0000 7f98abfff640 1 --2- 172.21.15.154:0/1672717476 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98ac15e350 0x7f98ac0c3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:34.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.688+0000 7f98aaffd640 1 --2- 172.21.15.154:0/1672717476 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98ac15a860 0x7f98ac0c8ea0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f98a000d5f0 tx=0x7f98a000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:34.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.688+0000 7f9893fff640 1 -- 172.21.15.154:0/1672717476 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a00125f0 con 0x7f98ac15a860 2023-09-23T03:25:34.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.688+0000 7f9893fff640 1 -- 172.21.15.154:0/1672717476 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a0012750 con 0x7f98ac15a860 2023-09-23T03:25:34.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.688+0000 7f9893fff640 1 -- 172.21.15.154:0/1672717476 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a0012b00 con 0x7f98ac15a860 2023-09-23T03:25:34.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.688+0000 7f98b0d27640 1 -- 172.21.15.154:0/1672717476 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98ac0c98f0 con 0x7f98ac15a860 2023-09-23T03:25:34.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.689+0000 7f98b0d27640 1 -- 172.21.15.154:0/1672717476 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98ac0c9610 con 0x7f98ac15a860 2023-09-23T03:25:34.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.690+0000 7f98b0d27640 1 -- 172.21.15.154:0/1672717476 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98ac067390 con 0x7f98ac15a860 2023-09-23T03:25:34.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.691+0000 7f9893fff640 1 -- 172.21.15.154:0/1672717476 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f98a0012c60 con 0x7f98ac15a860 2023-09-23T03:25:34.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.692+0000 7f9893fff640 1 --2- 172.21.15.154:0/1672717476 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f989806af30 0x7f989806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:34.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.692+0000 7f9893fff640 1 -- 172.21.15.154:0/1672717476 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6339+0+0 (secure 0 0 0) 0x7f98a0089ee0 con 0x7f98ac15a860 2023-09-23T03:25:34.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.692+0000 7f98ab7fe640 1 --2- 172.21.15.154:0/1672717476 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f989806af30 0x7f989806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:34.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.693+0000 7f98ab7fe640 1 --2- 172.21.15.154:0/1672717476 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f989806af30 0x7f989806d3f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f989c036000 tx=0x7f989c03b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:34.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.696+0000 7f9893fff640 1 -- 172.21.15.154:0/1672717476 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f98a0054c90 con 0x7f98ac15a860 2023-09-23T03:25:34.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:34.897+0000 7f98b0d27640 1 -- 172.21.15.154:0/1672717476 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f98ac06c3e0 con 0x7f98ac15a860 2023-09-23T03:25:35.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.445+0000 7f9893fff640 1 -- 172.21.15.154:0/1672717476 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v95) v1 ==== 169+0+0 (secure 0 0 0) 0x7f98a00582b0 con 0x7f98ac15a860 2023-09-23T03:25:35.447 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_data' removed 2023-09-23T03:25:35.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.450+0000 7f98b0d27640 1 -- 172.21.15.154:0/1672717476 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f989806af30 msgr2=0x7f989806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:35.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.450+0000 7f98b0d27640 1 --2- 172.21.15.154:0/1672717476 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f989806af30 0x7f989806d3f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f989c036000 tx=0x7f989c03b040 comp rx=0 tx=0).stop 2023-09-23T03:25:35.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.450+0000 7f98b0d27640 1 -- 172.21.15.154:0/1672717476 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98ac15a860 msgr2=0x7f98ac0c8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:35.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.450+0000 7f98b0d27640 1 --2- 172.21.15.154:0/1672717476 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98ac15a860 0x7f98ac0c8ea0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f98a000d5f0 tx=0x7f98a000dab0 comp rx=0 tx=0).stop 2023-09-23T03:25:35.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.450+0000 7f98b0d27640 1 -- 172.21.15.154:0/1672717476 shutdown_connections 2023-09-23T03:25:35.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.450+0000 7f98b0d27640 1 --2- 172.21.15.154:0/1672717476 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f989806af30 0x7f989806d3f0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:35.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.450+0000 7f98b0d27640 1 --2- 172.21.15.154:0/1672717476 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98ac15e350 0x7f98ac0c3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:35.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.450+0000 7f98b0d27640 1 --2- 172.21.15.154:0/1672717476 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98ac15a860 0x7f98ac0c8ea0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:35.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.450+0000 7f98b0d27640 1 --2- 172.21.15.154:0/1672717476 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98ac0ce2a0 0x7f98ac0c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:35.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.450+0000 7f98b0d27640 1 -- 172.21.15.154:0/1672717476 >> 172.21.15.154:0/1672717476 conn(0x7f98ac069eb0 msgr2=0x7f98ac0718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:35.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.450+0000 7f98b0d27640 1 -- 172.21.15.154:0/1672717476 shutdown_connections 2023-09-23T03:25:35.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.450+0000 7f98b0d27640 1 -- 172.21.15.154:0/1672717476 wait complete. 2023-09-23T03:25:35.478 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_extra cephfs_data_extra --yes-i-really-really-mean-it 2023-09-23T03:25:35.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.689+0000 7f19e5b61640 1 Processor -- start 2023-09-23T03:25:35.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.689+0000 7f19e5b61640 1 -- start start 2023-09-23T03:25:35.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.690+0000 7f19e5b61640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f19e00700f0 0x7f19e00c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:35.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.691+0000 7f19e5b61640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19e00c95b0 con 0x7f19e00cb460 2023-09-23T03:25:35.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.691+0000 7f19e5b61640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19e00c9890 con 0x7f19e015abd0 2023-09-23T03:25:35.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.691+0000 7f19e5b61640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19e00c9b70 con 0x7f19e00700f0 2023-09-23T03:25:35.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.691+0000 7f19df7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f19e00700f0 0x7f19e00c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:35.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.691+0000 7f19dffff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f19e00cb460 0x7f19e00b1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46860/0 (socket says 172.21.15.154:46860) 2023-09-23T03:25:35.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.691+0000 7f19dffff640 1 -- 172.21.15.154:0/2116360829 learned_addr learned my addr 172.21.15.154:0/2116360829 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:25:35.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.691+0000 7f19df7fe640 1 -- 172.21.15.154:0/2116360829 >> v1:172.21.15.190:6789/0 conn(0x7f19e015abd0 legacy=0x7f19e00704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:35.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.691+0000 7f19df7fe640 1 -- 172.21.15.154:0/2116360829 >> v1:172.21.15.154:6789/0 conn(0x7f19e00cb460 legacy=0x7f19e00b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:35.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.691+0000 7f19df7fe640 1 -- 172.21.15.154:0/2116360829 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e00c9e50 con 0x7f19e00700f0 2023-09-23T03:25:35.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.691+0000 7f19de7fc640 1 -- 172.21.15.154:0/2116360829 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3331396134 0 0) 0x7f19e00c95b0 con 0x7f19e00cb460 2023-09-23T03:25:35.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.693+0000 7f19df7fe640 1 --2- 172.21.15.154:0/2116360829 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19e00700f0 0x7f19e00c8fd0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f19d00097a0 tx=0x7f19d002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ee35400c1b32ca0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:35.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.693+0000 7f19de7fc640 1 -- 172.21.15.154:0/2116360829 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d0031830 con 0x7f19e00700f0 2023-09-23T03:25:35.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.693+0000 7f19de7fc640 1 -- 172.21.15.154:0/2116360829 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d0031990 con 0x7f19e00700f0 2023-09-23T03:25:35.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.693+0000 7f19de7fc640 1 -- 172.21.15.154:0/2116360829 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d0031d20 con 0x7f19e00700f0 2023-09-23T03:25:35.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.693+0000 7f19e5b61640 1 -- 172.21.15.154:0/2116360829 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19e00700f0 msgr2=0x7f19e00c8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:35.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.693+0000 7f19e5b61640 1 --2- 172.21.15.154:0/2116360829 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19e00700f0 0x7f19e00c8fd0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f19d00097a0 tx=0x7f19d002b8d0 comp rx=0 tx=0).stop 2023-09-23T03:25:35.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.693+0000 7f19e5b61640 1 -- 172.21.15.154:0/2116360829 shutdown_connections 2023-09-23T03:25:35.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.693+0000 7f19e5b61640 1 --2- 172.21.15.154:0/2116360829 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19e00700f0 0x7f19e00c8fd0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:35.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.693+0000 7f19e5b61640 1 -- 172.21.15.154:0/2116360829 >> 172.21.15.154:0/2116360829 conn(0x7f19e006a520 msgr2=0x7f19e006bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:35.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.694+0000 7f19e5b61640 1 -- 172.21.15.154:0/2116360829 shutdown_connections 2023-09-23T03:25:35.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.694+0000 7f19e5b61640 1 -- 172.21.15.154:0/2116360829 wait complete. 2023-09-23T03:25:35.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.695+0000 7f19e5b61640 1 Processor -- start 2023-09-23T03:25:35.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.695+0000 7f19e5b61640 1 -- start start 2023-09-23T03:25:35.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.696+0000 7f19e5b61640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19e00700f0 0x7f19e01e5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:35.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.696+0000 7f19e5b61640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19e00cb460 0x7f19e01e6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:35.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.696+0000 7f19e5b61640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19e015abd0 0x7f19e01e7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:35.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.696+0000 7f19e5b61640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e00c3370 con 0x7f19e015abd0 2023-09-23T03:25:35.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.696+0000 7f19e5b61640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e00c7ee0 con 0x7f19e00cb460 2023-09-23T03:25:35.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.696+0000 7f19e5b61640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e00c29d0 con 0x7f19e00700f0 2023-09-23T03:25:35.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.697+0000 7f19dffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19e015abd0 0x7f19e01e7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:35.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.697+0000 7f19dffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19e015abd0 0x7f19e01e7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50000/0 (socket says 172.21.15.154:50000) 2023-09-23T03:25:35.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.697+0000 7f19dffff640 1 -- 172.21.15.154:0/3230433039 learned_addr learned my addr 172.21.15.154:0/3230433039 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:35.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.697+0000 7f19dffff640 1 -- 172.21.15.154:0/3230433039 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19e00700f0 msgr2=0x7f19e01e5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:35.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.697+0000 7f19df7fe640 1 --2- 172.21.15.154:0/3230433039 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19e00700f0 0x7f19e01e5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:35.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.697+0000 7f19dffff640 1 --2- 172.21.15.154:0/3230433039 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19e00700f0 0x7f19e01e5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:35.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.697+0000 7f19dffff640 1 -- 172.21.15.154:0/3230433039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19e00cb460 msgr2=0x7f19e01e6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:35.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.697+0000 7f19deffd640 1 --2- 172.21.15.154:0/3230433039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19e00cb460 0x7f19e01e6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:35.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.697+0000 7f19dffff640 1 --2- 172.21.15.154:0/3230433039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19e00cb460 0x7f19e01e6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:35.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.697+0000 7f19dffff640 1 -- 172.21.15.154:0/3230433039 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d0009400 con 0x7f19e015abd0 2023-09-23T03:25:35.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.697+0000 7f19dffff640 1 --2- 172.21.15.154:0/3230433039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19e015abd0 0x7f19e01e7dd0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f19d800c480 tx=0x7f19d800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:35.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.698+0000 7f19d7fff640 1 -- 172.21.15.154:0/3230433039 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d80074e0 con 0x7f19e015abd0 2023-09-23T03:25:35.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.698+0000 7f19d7fff640 1 -- 172.21.15.154:0/3230433039 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d8007660 con 0x7f19e015abd0 2023-09-23T03:25:35.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.698+0000 7f19e5b61640 1 -- 172.21.15.154:0/3230433039 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e00c2cb0 con 0x7f19e015abd0 2023-09-23T03:25:35.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.698+0000 7f19d7fff640 1 -- 172.21.15.154:0/3230433039 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d8004a20 con 0x7f19e015abd0 2023-09-23T03:25:35.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.698+0000 7f19e5b61640 1 -- 172.21.15.154:0/3230433039 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e01e8310 con 0x7f19e015abd0 2023-09-23T03:25:35.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.699+0000 7f19e5b61640 1 -- 172.21.15.154:0/3230433039 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e00c34b0 con 0x7f19e015abd0 2023-09-23T03:25:35.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.700+0000 7f19d7fff640 1 -- 172.21.15.154:0/3230433039 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f19d8004b80 con 0x7f19e015abd0 2023-09-23T03:25:35.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.701+0000 7f19d7fff640 1 --2- 172.21.15.154:0/3230433039 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f19c006af30 0x7f19c006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:35.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.701+0000 7f19d7fff640 1 -- 172.21.15.154:0/3230433039 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5925+0+0 (secure 0 0 0) 0x7f19d8088db0 con 0x7f19e015abd0 2023-09-23T03:25:35.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.701+0000 7f19df7fe640 1 --2- 172.21.15.154:0/3230433039 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f19c006af30 0x7f19c006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:35.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.702+0000 7f19df7fe640 1 --2- 172.21.15.154:0/3230433039 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f19c006af30 0x7f19c006d3f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f19d0036000 tx=0x7f19d0008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:35.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.705+0000 7f19d7fff640 1 -- 172.21.15.154:0/3230433039 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f19e00c34b0 con 0x7f19e015abd0 2023-09-23T03:25:35.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:35.901+0000 7f19e5b61640 1 -- 172.21.15.154:0/3230433039 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_extra", "pool2": "cephfs_data_extra", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f19e00c34b0 con 0x7f19e015abd0 2023-09-23T03:25:36.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.457+0000 7f19d7fff640 1 -- 172.21.15.154:0/3230433039 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_extra", "pool2": "cephfs_data_extra", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_extra' removed v96) v1 ==== 187+0+0 (secure 0 0 0) 0x7f19e00c34b0 con 0x7f19e015abd0 2023-09-23T03:25:36.458 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_data_extra' removed 2023-09-23T03:25:36.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.461+0000 7f19e5b61640 1 -- 172.21.15.154:0/3230433039 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f19c006af30 msgr2=0x7f19c006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:36.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.461+0000 7f19e5b61640 1 --2- 172.21.15.154:0/3230433039 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f19c006af30 0x7f19c006d3f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f19d0036000 tx=0x7f19d0008030 comp rx=0 tx=0).stop 2023-09-23T03:25:36.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.461+0000 7f19e5b61640 1 -- 172.21.15.154:0/3230433039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19e015abd0 msgr2=0x7f19e01e7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:36.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.461+0000 7f19e5b61640 1 --2- 172.21.15.154:0/3230433039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19e015abd0 0x7f19e01e7dd0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f19d800c480 tx=0x7f19d800c940 comp rx=0 tx=0).stop 2023-09-23T03:25:36.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.461+0000 7f19e5b61640 1 -- 172.21.15.154:0/3230433039 shutdown_connections 2023-09-23T03:25:36.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.461+0000 7f19e5b61640 1 --2- 172.21.15.154:0/3230433039 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f19c006af30 0x7f19c006d3f0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:36.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.461+0000 7f19e5b61640 1 --2- 172.21.15.154:0/3230433039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f19e015abd0 0x7f19e01e7dd0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:36.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.461+0000 7f19e5b61640 1 --2- 172.21.15.154:0/3230433039 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f19e00cb460 0x7f19e01e6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:36.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.461+0000 7f19e5b61640 1 --2- 172.21.15.154:0/3230433039 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f19e00700f0 0x7f19e01e5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:36.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.461+0000 7f19e5b61640 1 -- 172.21.15.154:0/3230433039 >> 172.21.15.154:0/3230433039 conn(0x7f19e006a520 msgr2=0x7f19e00ad270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:36.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.462+0000 7f19e5b61640 1 -- 172.21.15.154:0/3230433039 shutdown_connections 2023-09-23T03:25:36.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.462+0000 7f19e5b61640 1 -- 172.21.15.154:0/3230433039 wait complete. 2023-09-23T03:25:36.486 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-09-23T03:25:36.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.702+0000 7f24d6167640 1 Processor -- start 2023-09-23T03:25:36.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.702+0000 7f24d6167640 1 -- start start 2023-09-23T03:25:36.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.703+0000 7f24d6167640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f24d015c880 0x7f24d01604a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:36.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.703+0000 7f24d6167640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24d0160cf0 con 0x7f24d00ab8c0 2023-09-23T03:25:36.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.703+0000 7f24d6167640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24d0160fd0 con 0x7f24d00cdd00 2023-09-23T03:25:36.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.703+0000 7f24d6167640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24d00cc6e0 con 0x7f24d015c880 2023-09-23T03:25:36.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.704+0000 7f24d5966640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f24d00ab8c0 0x7f24d00afeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46876/0 (socket says 172.21.15.154:46876) 2023-09-23T03:25:36.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.704+0000 7f24d5165640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f24d015c880 0x7f24d01604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:36.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.704+0000 7f24d5966640 1 -- 172.21.15.154:0/602039070 learned_addr learned my addr 172.21.15.154:0/602039070 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:25:36.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.704+0000 7f24c7fff640 1 -- 172.21.15.154:0/602039070 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3574538460 0 0) 0x7f24d0160cf0 con 0x7f24d00ab8c0 2023-09-23T03:25:36.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.704+0000 7f24c7fff640 1 -- 172.21.15.154:0/602039070 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24bc003520 con 0x7f24d00ab8c0 2023-09-23T03:25:36.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.704+0000 7f24d5165640 1 -- 172.21.15.154:0/602039070 >> v1:172.21.15.190:6789/0 conn(0x7f24d00cdd00 legacy=0x7f24d00ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:36.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.704+0000 7f24d5165640 1 -- 172.21.15.154:0/602039070 >> v1:172.21.15.154:6789/0 conn(0x7f24d00ab8c0 legacy=0x7f24d00afeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:36.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.704+0000 7f24d5165640 1 -- 172.21.15.154:0/602039070 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d01612b0 con 0x7f24d015c880 2023-09-23T03:25:36.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.706+0000 7f24d5165640 1 --2- 172.21.15.154:0/602039070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24d015c880 0x7f24d01604a0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f24cc0097a0 tx=0x7f24cc02b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea644a919329c51f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:36.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.706+0000 7f24c7fff640 1 -- 172.21.15.154:0/602039070 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24cc02f050 con 0x7f24d015c880 2023-09-23T03:25:36.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.706+0000 7f24c7fff640 1 -- 172.21.15.154:0/602039070 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24cc031700 con 0x7f24d015c880 2023-09-23T03:25:36.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.706+0000 7f24c7fff640 1 -- 172.21.15.154:0/602039070 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24cc031a70 con 0x7f24d015c880 2023-09-23T03:25:36.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.707+0000 7f24d6167640 1 -- 172.21.15.154:0/602039070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24d015c880 msgr2=0x7f24d01604a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:36.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.707+0000 7f24d6167640 1 --2- 172.21.15.154:0/602039070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24d015c880 0x7f24d01604a0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f24cc0097a0 tx=0x7f24cc02b7a0 comp rx=0 tx=0).stop 2023-09-23T03:25:36.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.707+0000 7f24d6167640 1 -- 172.21.15.154:0/602039070 shutdown_connections 2023-09-23T03:25:36.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.707+0000 7f24d6167640 1 --2- 172.21.15.154:0/602039070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24d015c880 0x7f24d01604a0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:36.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.707+0000 7f24d6167640 1 -- 172.21.15.154:0/602039070 >> 172.21.15.154:0/602039070 conn(0x7f24d006a300 msgr2=0x7f24d00caeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:36.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.707+0000 7f24d6167640 1 -- 172.21.15.154:0/602039070 shutdown_connections 2023-09-23T03:25:36.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.707+0000 7f24d6167640 1 -- 172.21.15.154:0/602039070 wait complete. 2023-09-23T03:25:36.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.709+0000 7f24d6167640 1 Processor -- start 2023-09-23T03:25:36.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.709+0000 7f24d6167640 1 -- start start 2023-09-23T03:25:36.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.710+0000 7f24d6167640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24d00ab8c0 0x7f24d00c5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:36.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.710+0000 7f24d6167640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24d00cdd00 0x7f24d0150980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:36.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.710+0000 7f24d6167640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24d015c880 0x7f24d006fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:36.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.710+0000 7f24d6167640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d00c5a40 con 0x7f24d00ab8c0 2023-09-23T03:25:36.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.710+0000 7f24d6167640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d00c31e0 con 0x7f24d00cdd00 2023-09-23T03:25:36.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.710+0000 7f24d6167640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d015d2f0 con 0x7f24d015c880 2023-09-23T03:25:36.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.710+0000 7f24d5165640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24d00ab8c0 0x7f24d00c5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:36.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.710+0000 7f24d5165640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24d00ab8c0 0x7f24d00c5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50018/0 (socket says 172.21.15.154:50018) 2023-09-23T03:25:36.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.710+0000 7f24d5165640 1 -- 172.21.15.154:0/2468939662 learned_addr learned my addr 172.21.15.154:0/2468939662 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:36.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.710+0000 7f24d5165640 1 -- 172.21.15.154:0/2468939662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24d015c880 msgr2=0x7f24d006fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:36.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.710+0000 7f24d5966640 1 --2- 172.21.15.154:0/2468939662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24d015c880 0x7f24d006fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:36.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.710+0000 7f24d4964640 1 --2- 172.21.15.154:0/2468939662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24d00cdd00 0x7f24d0150980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:36.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.710+0000 7f24d5165640 1 --2- 172.21.15.154:0/2468939662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24d015c880 0x7f24d006fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:36.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.710+0000 7f24d5165640 1 -- 172.21.15.154:0/2468939662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24d00cdd00 msgr2=0x7f24d0150980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:36.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.710+0000 7f24d5165640 1 --2- 172.21.15.154:0/2468939662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24d00cdd00 0x7f24d0150980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:36.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.710+0000 7f24d5165640 1 -- 172.21.15.154:0/2468939662 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24cc009400 con 0x7f24d00ab8c0 2023-09-23T03:25:36.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.711+0000 7f24d5165640 1 --2- 172.21.15.154:0/2468939662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24d00ab8c0 0x7f24d00c5500 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f24cc02b7e0 tx=0x7f24cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:36.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.711+0000 7f24c5ffb640 1 -- 172.21.15.154:0/2468939662 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24cc0339b0 con 0x7f24d00ab8c0 2023-09-23T03:25:36.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.711+0000 7f24c5ffb640 1 -- 172.21.15.154:0/2468939662 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24cc033b10 con 0x7f24d00ab8c0 2023-09-23T03:25:36.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.711+0000 7f24d6167640 1 -- 172.21.15.154:0/2468939662 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d015e0f0 con 0x7f24d00ab8c0 2023-09-23T03:25:36.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.711+0000 7f24c5ffb640 1 -- 172.21.15.154:0/2468939662 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24cc03e480 con 0x7f24d00ab8c0 2023-09-23T03:25:36.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.711+0000 7f24d6167640 1 -- 172.21.15.154:0/2468939662 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d00c2b90 con 0x7f24d00ab8c0 2023-09-23T03:25:36.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.712+0000 7f24d6167640 1 -- 172.21.15.154:0/2468939662 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24d0067390 con 0x7f24d00ab8c0 2023-09-23T03:25:36.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.713+0000 7f24c5ffb640 1 -- 172.21.15.154:0/2468939662 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f24cc03a040 con 0x7f24d00ab8c0 2023-09-23T03:25:36.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.714+0000 7f24c5ffb640 1 --2- 172.21.15.154:0/2468939662 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f24b406af30 0x7f24b406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:36.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.714+0000 7f24c5ffb640 1 -- 172.21.15.154:0/2468939662 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5521+0+0 (secure 0 0 0) 0x7f24cc0ab840 con 0x7f24d00ab8c0 2023-09-23T03:25:36.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.715+0000 7f24d4964640 1 --2- 172.21.15.154:0/2468939662 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f24b406af30 0x7f24b406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:36.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.715+0000 7f24d4964640 1 --2- 172.21.15.154:0/2468939662 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f24b406af30 0x7f24b406d3f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f24d015b2d0 tx=0x7f24c0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:36.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.719+0000 7f24c5ffb640 1 -- 172.21.15.154:0/2468939662 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f24cc076890 con 0x7f24d00ab8c0 2023-09-23T03:25:36.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.910+0000 7f24d6167640 1 -- 172.21.15.154:0/2468939662 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f24d0073f20 con 0x7f24d00ab8c0 2023-09-23T03:25:36.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.917+0000 7f24c5ffb640 1 -- 172.21.15.154:0/2468939662 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f24cc079eb0 con 0x7f24d00ab8c0 2023-09-23T03:25:36.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.920+0000 7f24d6167640 1 -- 172.21.15.154:0/2468939662 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f24b406af30 msgr2=0x7f24b406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:36.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.920+0000 7f24d6167640 1 --2- 172.21.15.154:0/2468939662 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f24b406af30 0x7f24b406d3f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f24d015b2d0 tx=0x7f24c0009040 comp rx=0 tx=0).stop 2023-09-23T03:25:36.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.920+0000 7f24d6167640 1 -- 172.21.15.154:0/2468939662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24d00ab8c0 msgr2=0x7f24d00c5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:36.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.920+0000 7f24d6167640 1 --2- 172.21.15.154:0/2468939662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24d00ab8c0 0x7f24d00c5500 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f24cc02b7e0 tx=0x7f24cc0046c0 comp rx=0 tx=0).stop 2023-09-23T03:25:36.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.920+0000 7f24d6167640 1 -- 172.21.15.154:0/2468939662 shutdown_connections 2023-09-23T03:25:36.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.920+0000 7f24d6167640 1 --2- 172.21.15.154:0/2468939662 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f24b406af30 0x7f24b406d3f0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:36.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.920+0000 7f24d6167640 1 --2- 172.21.15.154:0/2468939662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24d015c880 0x7f24d006fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:36.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.921+0000 7f24d6167640 1 --2- 172.21.15.154:0/2468939662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24d00cdd00 0x7f24d0150980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:36.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.921+0000 7f24d6167640 1 --2- 172.21.15.154:0/2468939662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24d00ab8c0 0x7f24d00c5500 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:36.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.921+0000 7f24d6167640 1 -- 172.21.15.154:0/2468939662 >> 172.21.15.154:0/2468939662 conn(0x7f24d006a300 msgr2=0x7f24d0070d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:36.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.921+0000 7f24d6167640 1 -- 172.21.15.154:0/2468939662 shutdown_connections 2023-09-23T03:25:36.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:36.921+0000 7f24d6167640 1 -- 172.21.15.154:0/2468939662 wait complete. 2023-09-23T03:25:36.941 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_simple' 2023-09-23T03:25:37.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.153+0000 7f5f5a95e640 1 Processor -- start 2023-09-23T03:25:37.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.154+0000 7f5f5a95e640 1 -- start start 2023-09-23T03:25:37.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.155+0000 7f5f5a95e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5f540acce0 0x7f5f5415ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:37.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.155+0000 7f5f5a95e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5f540b1b20 0x7f5f540ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:37.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.155+0000 7f5f5a95e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f5415ac30 con 0x7f5f540cee70 2023-09-23T03:25:37.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.155+0000 7f5f5a95e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f540b2e90 con 0x7f5f540acce0 2023-09-23T03:25:37.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.155+0000 7f5f5a95e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f540cf300 con 0x7f5f540b1b20 2023-09-23T03:25:37.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.155+0000 7f5f53fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5f540acce0 0x7f5f5415ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:37.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.155+0000 7f5f58efb640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f5f540cee70 0x7f5f5406cb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46884/0 (socket says 172.21.15.154:46884) 2023-09-23T03:25:37.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.155+0000 7f5f58efb640 1 -- 172.21.15.154:0/3767590333 learned_addr learned my addr 172.21.15.154:0/3767590333 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:25:37.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.156+0000 7f5f4bfff640 1 --2- 172.21.15.154:0/3767590333 >> v2:172.21.15.190:3300/0 conn(0x7f5f540b1b20 0x7f5f540ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:37.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.156+0000 7f5f537fe640 1 -- 172.21.15.154:0/3767590333 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4130523425 0 0) 0x7f5f4c001490 con 0x7f5f540cee70 2023-09-23T03:25:37.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.156+0000 7f5f537fe640 1 -- 172.21.15.154:0/3767590333 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f40003520 con 0x7f5f540cee70 2023-09-23T03:25:37.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.156+0000 7f5f53fff640 1 -- 172.21.15.154:0/3767590333 >> v2:172.21.15.190:3300/0 conn(0x7f5f540b1b20 msgr2=0x7f5f540ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:37.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.156+0000 7f5f53fff640 1 --2- 172.21.15.154:0/3767590333 >> v2:172.21.15.190:3300/0 conn(0x7f5f540b1b20 0x7f5f540ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:37.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.156+0000 7f5f53fff640 1 -- 172.21.15.154:0/3767590333 >> v1:172.21.15.154:6789/0 conn(0x7f5f540cee70 legacy=0x7f5f5406cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:37.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.156+0000 7f5f53fff640 1 -- 172.21.15.154:0/3767590333 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f540b1f70 con 0x7f5f540acce0 2023-09-23T03:25:37.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.156+0000 7f5f4bfff640 1 --2- 172.21.15.154:0/3767590333 >> v2:172.21.15.190:3300/0 conn(0x7f5f540b1b20 0x7f5f540ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:25:37.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.157+0000 7f5f53fff640 1 --2- 172.21.15.154:0/3767590333 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f540acce0 0x7f5f5415ed60 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f5f540c9430 tx=0x7f5f4402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b32007eb2b5e1bb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:37.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.157+0000 7f5f537fe640 1 -- 172.21.15.154:0/3767590333 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4402e070 con 0x7f5f540acce0 2023-09-23T03:25:37.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.158+0000 7f5f537fe640 1 -- 172.21.15.154:0/3767590333 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f440315a0 con 0x7f5f540acce0 2023-09-23T03:25:37.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.158+0000 7f5f537fe640 1 -- 172.21.15.154:0/3767590333 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f44031910 con 0x7f5f540acce0 2023-09-23T03:25:37.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.158+0000 7f5f5a95e640 1 -- 172.21.15.154:0/3767590333 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f540acce0 msgr2=0x7f5f5415ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:37.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.158+0000 7f5f5a95e640 1 --2- 172.21.15.154:0/3767590333 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f540acce0 0x7f5f5415ed60 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f5f540c9430 tx=0x7f5f4402b840 comp rx=0 tx=0).stop 2023-09-23T03:25:37.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.158+0000 7f5f5a95e640 1 -- 172.21.15.154:0/3767590333 shutdown_connections 2023-09-23T03:25:37.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.158+0000 7f5f5a95e640 1 --2- 172.21.15.154:0/3767590333 >> v2:172.21.15.190:3300/0 conn(0x7f5f540b1b20 0x7f5f540ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:37.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.158+0000 7f5f5a95e640 1 --2- 172.21.15.154:0/3767590333 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f540acce0 0x7f5f5415ed60 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:37.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.158+0000 7f5f5a95e640 1 -- 172.21.15.154:0/3767590333 >> 172.21.15.154:0/3767590333 conn(0x7f5f5406a0f0 msgr2=0x7f5f5406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:37.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.158+0000 7f5f5a95e640 1 -- 172.21.15.154:0/3767590333 shutdown_connections 2023-09-23T03:25:37.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.158+0000 7f5f5a95e640 1 -- 172.21.15.154:0/3767590333 wait complete. 2023-09-23T03:25:37.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.160+0000 7f5f5a95e640 1 Processor -- start 2023-09-23T03:25:37.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.160+0000 7f5f5a95e640 1 -- start start 2023-09-23T03:25:37.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.161+0000 7f5f5a95e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f540acce0 0x7f5f540c6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:37.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.161+0000 7f5f5a95e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f540b1b20 0x7f5f540c6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:37.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.161+0000 7f5f5a95e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f540cee70 0x7f5f540c6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:37.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.161+0000 7f5f5a95e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f54157d30 con 0x7f5f540cee70 2023-09-23T03:25:37.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.161+0000 7f5f5a95e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f540c5680 con 0x7f5f540b1b20 2023-09-23T03:25:37.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.161+0000 7f5f5a95e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f540c5960 con 0x7f5f540acce0 2023-09-23T03:25:37.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.161+0000 7f5f58efb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f540cee70 0x7f5f540c6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:37.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.161+0000 7f5f58efb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f540cee70 0x7f5f540c6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50028/0 (socket says 172.21.15.154:50028) 2023-09-23T03:25:37.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.162+0000 7f5f58efb640 1 -- 172.21.15.154:0/2525699066 learned_addr learned my addr 172.21.15.154:0/2525699066 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:37.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.162+0000 7f5f53fff640 1 --2- 172.21.15.154:0/2525699066 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f540acce0 0x7f5f540c6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:37.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.162+0000 7f5f4bfff640 1 --2- 172.21.15.154:0/2525699066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f540b1b20 0x7f5f540c6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:37.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.162+0000 7f5f58efb640 1 -- 172.21.15.154:0/2525699066 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f540acce0 msgr2=0x7f5f540c6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:37.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.162+0000 7f5f58efb640 1 --2- 172.21.15.154:0/2525699066 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f540acce0 0x7f5f540c6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:37.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.162+0000 7f5f58efb640 1 -- 172.21.15.154:0/2525699066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f540b1b20 msgr2=0x7f5f540c6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:37.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.162+0000 7f5f58efb640 1 --2- 172.21.15.154:0/2525699066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f540b1b20 0x7f5f540c6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:37.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.162+0000 7f5f58efb640 1 -- 172.21.15.154:0/2525699066 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f440093a0 con 0x7f5f540cee70 2023-09-23T03:25:37.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.162+0000 7f5f58efb640 1 --2- 172.21.15.154:0/2525699066 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f540cee70 0x7f5f540c6ec0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c00cae0 tx=0x7f5f4c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:37.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.162+0000 7f5f517fa640 1 -- 172.21.15.154:0/2525699066 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c00ac40 con 0x7f5f540cee70 2023-09-23T03:25:37.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.162+0000 7f5f517fa640 1 -- 172.21.15.154:0/2525699066 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f4c00ada0 con 0x7f5f540cee70 2023-09-23T03:25:37.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.162+0000 7f5f5a95e640 1 -- 172.21.15.154:0/2525699066 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f540c5c40 con 0x7f5f540cee70 2023-09-23T03:25:37.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.163+0000 7f5f517fa640 1 -- 172.21.15.154:0/2525699066 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c012620 con 0x7f5f540cee70 2023-09-23T03:25:37.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.163+0000 7f5f5a95e640 1 -- 172.21.15.154:0/2525699066 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f540c7400 con 0x7f5f540cee70 2023-09-23T03:25:37.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.164+0000 7f5f5a95e640 1 -- 172.21.15.154:0/2525699066 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f540c6160 con 0x7f5f540cee70 2023-09-23T03:25:37.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.165+0000 7f5f517fa640 1 -- 172.21.15.154:0/2525699066 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f5f4c007070 con 0x7f5f540cee70 2023-09-23T03:25:37.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.166+0000 7f5f517fa640 1 --2- 172.21.15.154:0/2525699066 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5f3406af30 0x7f5f3406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:37.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.166+0000 7f5f517fa640 1 -- 172.21.15.154:0/2525699066 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5521+0+0 (secure 0 0 0) 0x7f5f4c089510 con 0x7f5f540cee70 2023-09-23T03:25:37.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.166+0000 7f5f53fff640 1 --2- 172.21.15.154:0/2525699066 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5f3406af30 0x7f5f3406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:37.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.167+0000 7f5f53fff640 1 --2- 172.21.15.154:0/2525699066 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5f3406af30 0x7f5f3406d3f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f5f44036000 tx=0x7f5f4403d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:37.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.170+0000 7f5f517fa640 1 -- 172.21.15.154:0/2525699066 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f5f540c6160 con 0x7f5f540cee70 2023-09-23T03:25:37.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.440+0000 7f5f5a95e640 1 -- 172.21.15.154:0/2525699066 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_simple"]} v 0) v1 -- 0x7f5f540c6160 con 0x7f5f540cee70 2023-09-23T03:25:37.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.461+0000 7f5f517fa640 1 -- 172.21.15.154:0/2525699066 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_simple"]}]=0 v460) v1 ==== 155+0+0 (secure 0 0 0) 0x7f5f540c6160 con 0x7f5f540cee70 2023-09-23T03:25:37.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.463+0000 7f5f5a95e640 1 -- 172.21.15.154:0/2525699066 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5f3406af30 msgr2=0x7f5f3406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:37.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.463+0000 7f5f5a95e640 1 --2- 172.21.15.154:0/2525699066 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5f3406af30 0x7f5f3406d3f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f5f44036000 tx=0x7f5f4403d040 comp rx=0 tx=0).stop 2023-09-23T03:25:37.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.463+0000 7f5f5a95e640 1 -- 172.21.15.154:0/2525699066 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f540cee70 msgr2=0x7f5f540c6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:37.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.463+0000 7f5f5a95e640 1 --2- 172.21.15.154:0/2525699066 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f540cee70 0x7f5f540c6ec0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c00cae0 tx=0x7f5f4c00a060 comp rx=0 tx=0).stop 2023-09-23T03:25:37.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.464+0000 7f5f5a95e640 1 -- 172.21.15.154:0/2525699066 shutdown_connections 2023-09-23T03:25:37.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.464+0000 7f5f5a95e640 1 --2- 172.21.15.154:0/2525699066 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5f3406af30 0x7f5f3406d3f0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:37.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.464+0000 7f5f5a95e640 1 --2- 172.21.15.154:0/2525699066 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5f540cee70 0x7f5f540c6ec0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:37.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.464+0000 7f5f5a95e640 1 --2- 172.21.15.154:0/2525699066 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5f540b1b20 0x7f5f540c6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:37.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.464+0000 7f5f5a95e640 1 --2- 172.21.15.154:0/2525699066 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5f540acce0 0x7f5f540c6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:37.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.464+0000 7f5f5a95e640 1 -- 172.21.15.154:0/2525699066 >> 172.21.15.154:0/2525699066 conn(0x7f5f5406a0f0 msgr2=0x7f5f540affe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:37.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.464+0000 7f5f5a95e640 1 -- 172.21.15.154:0/2525699066 shutdown_connections 2023-09-23T03:25:37.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.464+0000 7f5f5a95e640 1 -- 172.21.15.154:0/2525699066 wait complete. 2023-09-23T03:25:37.483 INFO:tasks.cephfs_test_runner:test_extra_data_pool_rebuild_simple (tasks.cephfs.test_data_scan.TestDataScan) ... ok 2023-09-23T03:25:37.483 INFO:tasks.cephfs_test_runner:Starting test: test_extra_data_pool_stashed_layout (tasks.cephfs.test_data_scan.TestDataScan) 2023-09-23T03:25:37.484 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:25:37.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.694+0000 7f1b80026640 1 Processor -- start 2023-09-23T03:25:37.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.695+0000 7f1b80026640 1 -- start start 2023-09-23T03:25:37.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.696+0000 7f1b80026640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1b780acce0 0x7f1b7815ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:37.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.696+0000 7f1b80026640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1b780cf040 0x7f1b780ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:37.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.696+0000 7f1b80026640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b7815ac30 con 0x7f1b780b1b20 2023-09-23T03:25:37.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.696+0000 7f1b80026640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b780b2e90 con 0x7f1b780cf040 2023-09-23T03:25:37.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.696+0000 7f1b80026640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b780b1f70 con 0x7f1b780acce0 2023-09-23T03:25:37.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.696+0000 7f1b7d5c1640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f1b780b1b20 0x7f1b780ce930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46886/0 (socket says 172.21.15.154:46886) 2023-09-23T03:25:37.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.696+0000 7f1b7d5c1640 1 -- 172.21.15.154:0/3179770379 learned_addr learned my addr 172.21.15.154:0/3179770379 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:25:37.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.696+0000 7f1b7e5c3640 1 --2- 172.21.15.154:0/3179770379 >> v2:172.21.15.154:3300/0 conn(0x7f1b780cf040 0x7f1b780ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:37.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.697+0000 7f1b7ddc2640 1 --2- 172.21.15.154:0/3179770379 >> v2:172.21.15.190:3300/0 conn(0x7f1b780acce0 0x7f1b7815ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:37.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.697+0000 7f1b7cdc0640 1 -- 172.21.15.154:0/3179770379 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4041105068 0 0) 0x7f1b68001110 con 0x7f1b780b1b20 2023-09-23T03:25:37.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.697+0000 7f1b7cdc0640 1 -- 172.21.15.154:0/3179770379 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b64003520 con 0x7f1b780b1b20 2023-09-23T03:25:37.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.697+0000 7f1b7e5c3640 1 -- 172.21.15.154:0/3179770379 >> v2:172.21.15.190:3300/0 conn(0x7f1b780acce0 msgr2=0x7f1b7815ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:37.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.697+0000 7f1b7e5c3640 1 --2- 172.21.15.154:0/3179770379 >> v2:172.21.15.190:3300/0 conn(0x7f1b780acce0 0x7f1b7815ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:37.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.697+0000 7f1b7e5c3640 1 -- 172.21.15.154:0/3179770379 >> v1:172.21.15.154:6789/0 conn(0x7f1b780b1b20 legacy=0x7f1b780ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:37.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.697+0000 7f1b7e5c3640 1 -- 172.21.15.154:0/3179770379 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b7815f2a0 con 0x7f1b780cf040 2023-09-23T03:25:37.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.698+0000 7f1b7e5c3640 1 --2- 172.21.15.154:0/3179770379 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b780cf040 0x7f1b780ab140 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f1b780cf570 tx=0x7f1b7002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=903a689d146cf3de server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:37.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.699+0000 7f1b7cdc0640 1 -- 172.21.15.154:0/3179770379 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7002cbf0 con 0x7f1b780cf040 2023-09-23T03:25:37.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.699+0000 7f1b7cdc0640 1 -- 172.21.15.154:0/3179770379 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b7002cd50 con 0x7f1b780cf040 2023-09-23T03:25:37.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.699+0000 7f1b7cdc0640 1 -- 172.21.15.154:0/3179770379 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b70034640 con 0x7f1b780cf040 2023-09-23T03:25:37.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.699+0000 7f1b80026640 1 -- 172.21.15.154:0/3179770379 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b780cf040 msgr2=0x7f1b780ab140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:37.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.699+0000 7f1b80026640 1 --2- 172.21.15.154:0/3179770379 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b780cf040 0x7f1b780ab140 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f1b780cf570 tx=0x7f1b7002be40 comp rx=0 tx=0).stop 2023-09-23T03:25:37.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.699+0000 7f1b80026640 1 -- 172.21.15.154:0/3179770379 shutdown_connections 2023-09-23T03:25:37.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.699+0000 7f1b80026640 1 --2- 172.21.15.154:0/3179770379 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b780cf040 0x7f1b780ab140 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:37.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.699+0000 7f1b80026640 1 --2- 172.21.15.154:0/3179770379 >> v2:172.21.15.190:3300/0 conn(0x7f1b780acce0 0x7f1b7815ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:37.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.700+0000 7f1b80026640 1 -- 172.21.15.154:0/3179770379 >> 172.21.15.154:0/3179770379 conn(0x7f1b7806a0f0 msgr2=0x7f1b7806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:37.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.700+0000 7f1b80026640 1 -- 172.21.15.154:0/3179770379 shutdown_connections 2023-09-23T03:25:37.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.700+0000 7f1b80026640 1 -- 172.21.15.154:0/3179770379 wait complete. 2023-09-23T03:25:37.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.702+0000 7f1b80026640 1 Processor -- start 2023-09-23T03:25:37.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.702+0000 7f1b80026640 1 -- start start 2023-09-23T03:25:37.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.703+0000 7f1b80026640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b780acce0 0x7f1b78153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:37.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.703+0000 7f1b80026640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b780b1b20 0x7f1b780c4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:37.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.703+0000 7f1b80026640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b780cf040 0x7f1b780cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:37.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.703+0000 7f1b80026640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b780ca170 con 0x7f1b780cf040 2023-09-23T03:25:37.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.703+0000 7f1b80026640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b780c4f50 con 0x7f1b780b1b20 2023-09-23T03:25:37.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.703+0000 7f1b80026640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b780ccaf0 con 0x7f1b780acce0 2023-09-23T03:25:37.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.704+0000 7f1b7e5c3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b780cf040 0x7f1b780cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:37.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.704+0000 7f1b7e5c3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b780cf040 0x7f1b780cd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50048/0 (socket says 172.21.15.154:50048) 2023-09-23T03:25:37.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.704+0000 7f1b7e5c3640 1 -- 172.21.15.154:0/3143276017 learned_addr learned my addr 172.21.15.154:0/3143276017 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:37.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.704+0000 7f1b7d5c1640 1 --2- 172.21.15.154:0/3143276017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b780b1b20 0x7f1b780c4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:37.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.704+0000 7f1b7ddc2640 1 --2- 172.21.15.154:0/3143276017 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b780acce0 0x7f1b78153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:37.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.704+0000 7f1b7e5c3640 1 -- 172.21.15.154:0/3143276017 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b780acce0 msgr2=0x7f1b78153580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:37.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.704+0000 7f1b7e5c3640 1 --2- 172.21.15.154:0/3143276017 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b780acce0 0x7f1b78153580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:37.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.704+0000 7f1b7e5c3640 1 -- 172.21.15.154:0/3143276017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b780b1b20 msgr2=0x7f1b780c4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:37.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.704+0000 7f1b7e5c3640 1 --2- 172.21.15.154:0/3143276017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b780b1b20 0x7f1b780c4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:37.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.704+0000 7f1b7e5c3640 1 -- 172.21.15.154:0/3143276017 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b70009a70 con 0x7f1b780cf040 2023-09-23T03:25:37.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.704+0000 7f1b7ddc2640 1 --2- 172.21.15.154:0/3143276017 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b780acce0 0x7f1b78153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:37.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.704+0000 7f1b7d5c1640 1 --2- 172.21.15.154:0/3143276017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b780b1b20 0x7f1b780c4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:37.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.704+0000 7f1b7e5c3640 1 --2- 172.21.15.154:0/3143276017 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b780cf040 0x7f1b780cd8b0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f1b7002c450 tx=0x7f1b70004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:37.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.705+0000 7f1b6e7fc640 1 -- 172.21.15.154:0/3143276017 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7003e610 con 0x7f1b780cf040 2023-09-23T03:25:37.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.705+0000 7f1b6e7fc640 1 -- 172.21.15.154:0/3143276017 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b7003e770 con 0x7f1b780cf040 2023-09-23T03:25:37.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.705+0000 7f1b6e7fc640 1 -- 172.21.15.154:0/3143276017 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7003eb60 con 0x7f1b780cf040 2023-09-23T03:25:37.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.705+0000 7f1b80026640 1 -- 172.21.15.154:0/3143276017 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b780ccdd0 con 0x7f1b780cf040 2023-09-23T03:25:37.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.705+0000 7f1b80026640 1 -- 172.21.15.154:0/3143276017 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b780cd390 con 0x7f1b780cf040 2023-09-23T03:25:37.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.706+0000 7f1b80026640 1 -- 172.21.15.154:0/3143276017 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b78073eb0 con 0x7f1b780cf040 2023-09-23T03:25:37.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.707+0000 7f1b6e7fc640 1 -- 172.21.15.154:0/3143276017 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f1b7003b050 con 0x7f1b780cf040 2023-09-23T03:25:37.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.708+0000 7f1b6e7fc640 1 --2- 172.21.15.154:0/3143276017 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1b5c06af30 0x7f1b5c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:37.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.708+0000 7f1b6e7fc640 1 -- 172.21.15.154:0/3143276017 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5521+0+0 (secure 0 0 0) 0x7f1b700aba00 con 0x7f1b780cf040 2023-09-23T03:25:37.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.708+0000 7f1b7ddc2640 1 --2- 172.21.15.154:0/3143276017 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1b5c06af30 0x7f1b5c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:37.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.709+0000 7f1b7ddc2640 1 --2- 172.21.15.154:0/3143276017 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1b5c06af30 0x7f1b5c06d3f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f1b74009780 tx=0x7f1b74008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:37.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.712+0000 7f1b6e7fc640 1 -- 172.21.15.154:0/3143276017 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f1b70076a50 con 0x7f1b780cf040 2023-09-23T03:25:37.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.901+0000 7f1b80026640 1 -- 172.21.15.154:0/3143276017 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1b780b27a0 con 0x7f1b780cf040 2023-09-23T03:25:37.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.903+0000 7f1b6e7fc640 1 -- 172.21.15.154:0/3143276017 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+13626 (secure 0 0 0) 0x7f1b700a3e60 con 0x7f1b780cf040 2023-09-23T03:25:37.904 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:25:37.904 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":96,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:25:36.447543+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.190:6835/2202597679":"2023-09-24T03:25:32.321202+0000","172.21.15.154:6837/1588996498":"2023-09-24T03:24:45.398760+0000","172.21.15.154:6836/1588996498":"2023-09-24T03:24:45.398760+0000","172.21.15.154:6837/3812680297":"2023-09-24T03:24:38.360723+0000","172.21.15.190:6833/2202597679":"2023-09-24T03:25:32.321202+0000","172.21.15.154:6836/3812680297":"2023-09-24T03:24:38.360723+0000","172.21.15.154:6837/2213283923":"2023-09-24T03:24:36.268262+0000","172.21.15.154:6836/2213283923":"2023-09-24T03:24:36.268262+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:25:37.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.906+0000 7f1b80026640 1 -- 172.21.15.154:0/3143276017 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1b5c06af30 msgr2=0x7f1b5c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:37.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.906+0000 7f1b80026640 1 --2- 172.21.15.154:0/3143276017 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1b5c06af30 0x7f1b5c06d3f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f1b74009780 tx=0x7f1b74008040 comp rx=0 tx=0).stop 2023-09-23T03:25:37.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.906+0000 7f1b80026640 1 -- 172.21.15.154:0/3143276017 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b780cf040 msgr2=0x7f1b780cd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:37.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.906+0000 7f1b80026640 1 --2- 172.21.15.154:0/3143276017 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b780cf040 0x7f1b780cd8b0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f1b7002c450 tx=0x7f1b70004a40 comp rx=0 tx=0).stop 2023-09-23T03:25:37.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.906+0000 7f1b80026640 1 -- 172.21.15.154:0/3143276017 shutdown_connections 2023-09-23T03:25:37.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.906+0000 7f1b80026640 1 --2- 172.21.15.154:0/3143276017 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1b5c06af30 0x7f1b5c06d3f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:37.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.906+0000 7f1b80026640 1 --2- 172.21.15.154:0/3143276017 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b780cf040 0x7f1b780cd8b0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:37.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.906+0000 7f1b80026640 1 --2- 172.21.15.154:0/3143276017 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b780b1b20 0x7f1b780c4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:37.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.906+0000 7f1b80026640 1 --2- 172.21.15.154:0/3143276017 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b780acce0 0x7f1b78153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:37.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.906+0000 7f1b80026640 1 -- 172.21.15.154:0/3143276017 >> 172.21.15.154:0/3143276017 conn(0x7f1b7806a0f0 msgr2=0x7f1b780ca710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:37.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.906+0000 7f1b80026640 1 -- 172.21.15.154:0/3143276017 shutdown_connections 2023-09-23T03:25:37.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:37.906+0000 7f1b80026640 1 -- 172.21.15.154:0/3143276017 wait complete. 2023-09-23T03:25:37.925 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T03:25:37.925 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:25:38.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.139+0000 7ff06cba9640 1 Processor -- start 2023-09-23T03:25:38.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.139+0000 7ff06cba9640 1 -- start start 2023-09-23T03:25:38.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.141+0000 7ff06cba9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff0680ad830 0x7ff068070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:38.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.141+0000 7ff06cba9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff0680c9680 0x7ff0680caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:38.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.141+0000 7ff06cba9640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0680712f0 con 0x7ff0680cb400 2023-09-23T03:25:38.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.141+0000 7ff06cba9640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0680cb870 con 0x7ff0680ad830 2023-09-23T03:25:38.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.141+0000 7ff06cba9640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0680715d0 con 0x7ff0680c9680 2023-09-23T03:25:38.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.141+0000 7ff06659c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff0680ad830 0x7ff068070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:38.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.141+0000 7ff06659c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff0680ad830 0x7ff068070c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50058/0 (socket says 172.21.15.154:50058) 2023-09-23T03:25:38.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.141+0000 7ff06659c640 1 -- 172.21.15.154:0/1149825146 learned_addr learned my addr 172.21.15.154:0/1149825146 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:38.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.141+0000 7ff06659c640 1 -- 172.21.15.154:0/1149825146 >> v2:172.21.15.190:3300/0 conn(0x7ff0680c9680 msgr2=0x7ff0680caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:38.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.141+0000 7ff06659c640 1 --2- 172.21.15.154:0/1149825146 >> v2:172.21.15.190:3300/0 conn(0x7ff0680c9680 0x7ff0680caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:38.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.142+0000 7ff06659c640 1 -- 172.21.15.154:0/1149825146 >> v1:172.21.15.190:6789/0 conn(0x7ff0680cb400 legacy=0x7ff068160720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:38.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.142+0000 7ff06659c640 1 -- 172.21.15.154:0/1149825146 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0680718b0 con 0x7ff0680ad830 2023-09-23T03:25:38.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.143+0000 7ff06659c640 1 --2- 172.21.15.154:0/1149825146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0680ad830 0x7ff068070c60 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7ff0680cbb10 tx=0x7ff05802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24c901637dd0fa4b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:38.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.143+0000 7ff06559a640 1 -- 172.21.15.154:0/1149825146 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff058031870 con 0x7ff0680ad830 2023-09-23T03:25:38.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.143+0000 7ff06559a640 1 -- 172.21.15.154:0/1149825146 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0580319d0 con 0x7ff0680ad830 2023-09-23T03:25:38.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.143+0000 7ff06559a640 1 -- 172.21.15.154:0/1149825146 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff058031d60 con 0x7ff0680ad830 2023-09-23T03:25:38.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.144+0000 7ff06cba9640 1 -- 172.21.15.154:0/1149825146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0680ad830 msgr2=0x7ff068070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:38.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.144+0000 7ff06cba9640 1 --2- 172.21.15.154:0/1149825146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0680ad830 0x7ff068070c60 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7ff0680cbb10 tx=0x7ff05802b910 comp rx=0 tx=0).stop 2023-09-23T03:25:38.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.144+0000 7ff06cba9640 1 -- 172.21.15.154:0/1149825146 shutdown_connections 2023-09-23T03:25:38.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.144+0000 7ff06cba9640 1 --2- 172.21.15.154:0/1149825146 >> v2:172.21.15.190:3300/0 conn(0x7ff0680c9680 0x7ff0680caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:38.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.144+0000 7ff06cba9640 1 --2- 172.21.15.154:0/1149825146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0680ad830 0x7ff068070c60 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:38.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.144+0000 7ff06cba9640 1 -- 172.21.15.154:0/1149825146 >> 172.21.15.154:0/1149825146 conn(0x7ff06806c010 msgr2=0x7ff06806c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:38.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.144+0000 7ff06cba9640 1 -- 172.21.15.154:0/1149825146 shutdown_connections 2023-09-23T03:25:38.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.144+0000 7ff06cba9640 1 -- 172.21.15.154:0/1149825146 wait complete. 2023-09-23T03:25:38.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.146+0000 7ff06cba9640 1 Processor -- start 2023-09-23T03:25:38.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.146+0000 7ff06cba9640 1 -- start start 2023-09-23T03:25:38.147 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.147+0000 7ff06cba9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0680ad830 0x7ff068154c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:38.147 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.147+0000 7ff06cba9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0680c9680 0x7ff0680c4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:38.147 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.147+0000 7ff06cba9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0680cb400 0x7ff0680c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:38.147 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.147+0000 7ff06cba9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0680c30a0 con 0x7ff0680c9680 2023-09-23T03:25:38.147 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.147+0000 7ff06cba9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0681551a0 con 0x7ff0680ad830 2023-09-23T03:25:38.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.147+0000 7ff06cba9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0680c32f0 con 0x7ff0680cb400 2023-09-23T03:25:38.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.147+0000 7ff065d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0680c9680 0x7ff0680c4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:38.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.147+0000 7ff065d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0680c9680 0x7ff0680c4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50072/0 (socket says 172.21.15.154:50072) 2023-09-23T03:25:38.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.147+0000 7ff065d9b640 1 -- 172.21.15.154:0/1022051171 learned_addr learned my addr 172.21.15.154:0/1022051171 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:38.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.147+0000 7ff065d9b640 1 -- 172.21.15.154:0/1022051171 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0680cb400 msgr2=0x7ff0680c58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:38.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.147+0000 7ff065d9b640 1 --2- 172.21.15.154:0/1022051171 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0680cb400 0x7ff0680c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:38.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.147+0000 7ff065d9b640 1 -- 172.21.15.154:0/1022051171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0680ad830 msgr2=0x7ff068154c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:38.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.147+0000 7ff065d9b640 1 --2- 172.21.15.154:0/1022051171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0680ad830 0x7ff068154c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:38.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.147+0000 7ff065d9b640 1 -- 172.21.15.154:0/1022051171 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0580094d0 con 0x7ff0680c9680 2023-09-23T03:25:38.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.148+0000 7ff065d9b640 1 --2- 172.21.15.154:0/1022051171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0680c9680 0x7ff0680c4390 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7ff05000ba80 tx=0x7ff05000bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:38.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.148+0000 7ff05effd640 1 -- 172.21.15.154:0/1022051171 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05000cc00 con 0x7ff0680c9680 2023-09-23T03:25:38.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.148+0000 7ff05effd640 1 -- 172.21.15.154:0/1022051171 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff05000cd80 con 0x7ff0680c9680 2023-09-23T03:25:38.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.148+0000 7ff05effd640 1 -- 172.21.15.154:0/1022051171 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff050014680 con 0x7ff0680c9680 2023-09-23T03:25:38.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.148+0000 7ff06cba9640 1 -- 172.21.15.154:0/1022051171 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0680c35d0 con 0x7ff0680c9680 2023-09-23T03:25:38.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.148+0000 7ff06cba9640 1 -- 172.21.15.154:0/1022051171 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0680c3b90 con 0x7ff0680c9680 2023-09-23T03:25:38.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.149+0000 7ff06cba9640 1 -- 172.21.15.154:0/1022051171 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff06804e920 con 0x7ff0680c9680 2023-09-23T03:25:38.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.150+0000 7ff05effd640 1 -- 172.21.15.154:0/1022051171 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7ff05001d3b0 con 0x7ff0680c9680 2023-09-23T03:25:38.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.151+0000 7ff05effd640 1 --2- 172.21.15.154:0/1022051171 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff04806af60 0x7ff04806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:38.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.151+0000 7ff05effd640 1 -- 172.21.15.154:0/1022051171 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5521+0+0 (secure 0 0 0) 0x7ff050089630 con 0x7ff0680c9680 2023-09-23T03:25:38.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.152+0000 7ff06659c640 1 --2- 172.21.15.154:0/1022051171 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff04806af60 0x7ff04806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:38.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.152+0000 7ff06659c640 1 --2- 172.21.15.154:0/1022051171 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff04806af60 0x7ff04806d420 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff058036000 tx=0x7ff058030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:38.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.155+0000 7ff05effd640 1 -- 172.21.15.154:0/1022051171 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7ff050054710 con 0x7ff0680c9680 2023-09-23T03:25:38.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.335+0000 7ff06cba9640 1 -- 172.21.15.154:0/1022051171 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff0680cbdc0 con 0x7ff0680c9680 2023-09-23T03:25:38.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.336+0000 7ff05effd640 1 -- 172.21.15.154:0/1022051171 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff050019020 con 0x7ff0680c9680 2023-09-23T03:25:38.336 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:25:38.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.339+0000 7ff06cba9640 1 -- 172.21.15.154:0/1022051171 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff04806af60 msgr2=0x7ff04806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:38.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.339+0000 7ff06cba9640 1 --2- 172.21.15.154:0/1022051171 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff04806af60 0x7ff04806d420 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff058036000 tx=0x7ff058030000 comp rx=0 tx=0).stop 2023-09-23T03:25:38.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.339+0000 7ff06cba9640 1 -- 172.21.15.154:0/1022051171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0680c9680 msgr2=0x7ff0680c4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:38.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.339+0000 7ff06cba9640 1 --2- 172.21.15.154:0/1022051171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0680c9680 0x7ff0680c4390 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7ff05000ba80 tx=0x7ff05000bf40 comp rx=0 tx=0).stop 2023-09-23T03:25:38.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.339+0000 7ff06cba9640 1 -- 172.21.15.154:0/1022051171 shutdown_connections 2023-09-23T03:25:38.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.339+0000 7ff06cba9640 1 --2- 172.21.15.154:0/1022051171 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff04806af60 0x7ff04806d420 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:38.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.339+0000 7ff06cba9640 1 --2- 172.21.15.154:0/1022051171 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff0680cb400 0x7ff0680c58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:38.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.339+0000 7ff06cba9640 1 --2- 172.21.15.154:0/1022051171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0680c9680 0x7ff0680c4390 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:38.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.339+0000 7ff06cba9640 1 --2- 172.21.15.154:0/1022051171 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff0680ad830 0x7ff068154c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:38.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.339+0000 7ff06cba9640 1 -- 172.21.15.154:0/1022051171 >> 172.21.15.154:0/1022051171 conn(0x7ff06806c010 msgr2=0x7ff0680af1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:38.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.339+0000 7ff06cba9640 1 -- 172.21.15.154:0/1022051171 shutdown_connections 2023-09-23T03:25:38.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.339+0000 7ff06cba9640 1 -- 172.21.15.154:0/1022051171 wait complete. 2023-09-23T03:25:38.358 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_stashed_layout' 2023-09-23T03:25:38.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.551+0000 7f595de75640 1 Processor -- start 2023-09-23T03:25:38.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.551+0000 7f595de75640 1 -- start start 2023-09-23T03:25:38.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.552+0000 7f595de75640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59580ada30 0x7f59580ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:38.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.552+0000 7f595de75640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f595806ceb0 con 0x7f59580b2fe0 2023-09-23T03:25:38.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.552+0000 7f595de75640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5958159c90 con 0x7f59580cf130 2023-09-23T03:25:38.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.552+0000 7f595de75640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59580ade10 con 0x7f59580ada30 2023-09-23T03:25:38.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.553+0000 7f594ffff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f59580b2fe0 0x7f59580cea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46896/0 (socket says 172.21.15.154:46896) 2023-09-23T03:25:38.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.553+0000 7f594ffff640 1 -- 172.21.15.154:0/2071719156 learned_addr learned my addr 172.21.15.154:0/2071719156 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:25:38.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.553+0000 7f59577fe640 1 --2- 172.21.15.154:0/2071719156 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59580ada30 0x7f59580ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:38.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.553+0000 7f5956ffd640 1 -- 172.21.15.154:0/2071719156 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 58597645 0 0) 0x7f5940001110 con 0x7f59580b2fe0 2023-09-23T03:25:38.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.553+0000 7f5956ffd640 1 -- 172.21.15.154:0/2071719156 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5944005730 con 0x7f59580b2fe0 2023-09-23T03:25:38.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.553+0000 7f5956ffd640 1 -- 172.21.15.154:0/2071719156 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1645787854 0 0) 0x7f5944005730 con 0x7f59580b2fe0 2023-09-23T03:25:38.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.553+0000 7f5956ffd640 1 -- 172.21.15.154:0/2071719156 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59580ada30 msgr2=0x7f59580ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:38.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.553+0000 7f5956ffd640 1 --2- 172.21.15.154:0/2071719156 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59580ada30 0x7f59580ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:38.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.553+0000 7f5956ffd640 1 -- 172.21.15.154:0/2071719156 >> v1:172.21.15.190:6789/0 conn(0x7f59580cf130 legacy=0x7f595815d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:38.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.553+0000 7f5956ffd640 1 -- 172.21.15.154:0/2071719156 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f595806c110 con 0x7f59580b2fe0 2023-09-23T03:25:38.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.554+0000 7f5956ffd640 1 -- 172.21.15.154:0/2071719156 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f5940001800 con 0x7f59580b2fe0 2023-09-23T03:25:38.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.554+0000 7f5956ffd640 1 -- 172.21.15.154:0/2071719156 >> v1:172.21.15.154:6789/0 conn(0x7f59580b2fe0 legacy=0x7f59580cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:38.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.554+0000 7f5956ffd640 1 --2- 172.21.15.154:0/2071719156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5944005bb0 0x7f5944005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:38.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.554+0000 7f5956ffd640 1 --2- 172.21.15.154:0/2071719156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59440069d0 0x7f5944006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:38.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.554+0000 7f5956ffd640 1 --2- 172.21.15.154:0/2071719156 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59580ada30 0x7f59440073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:38.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.554+0000 7f5956ffd640 1 -- 172.21.15.154:0/2071719156 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59580ade10 con 0x7f5944005bb0 2023-09-23T03:25:38.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.554+0000 7f5956ffd640 1 -- 172.21.15.154:0/2071719156 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594400dcc0 con 0x7f59440069d0 2023-09-23T03:25:38.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.554+0000 7f5956ffd640 1 -- 172.21.15.154:0/2071719156 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f594400de80 con 0x7f59580ada30 2023-09-23T03:25:38.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.554+0000 7f59577fe640 1 --2- 172.21.15.154:0/2071719156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5944005bb0 0x7f5944005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:38.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.554+0000 7f59577fe640 1 -- 172.21.15.154:0/2071719156 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59580ada30 msgr2=0x7f59440073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:38.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.554+0000 7f59577fe640 1 --2- 172.21.15.154:0/2071719156 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f59580ada30 0x7f59440073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:38.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.554+0000 7f59577fe640 1 -- 172.21.15.154:0/2071719156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59440069d0 msgr2=0x7f5944006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:38.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.554+0000 7f59577fe640 1 --2- 172.21.15.154:0/2071719156 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59440069d0 0x7f5944006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:38.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.554+0000 7f59577fe640 1 -- 172.21.15.154:0/2071719156 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5944005730 con 0x7f5944005bb0 2023-09-23T03:25:38.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.555+0000 7f59577fe640 1 --2- 172.21.15.154:0/2071719156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5944005bb0 0x7f5944005fd0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f594800ba10 tx=0x7f5944007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44404a84a0e063f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:38.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.555+0000 7f5956ffd640 1 -- 172.21.15.154:0/2071719156 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5948005b90 con 0x7f5944005bb0 2023-09-23T03:25:38.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.556+0000 7f5956ffd640 1 -- 172.21.15.154:0/2071719156 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5948005cf0 con 0x7f5944005bb0 2023-09-23T03:25:38.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.556+0000 7f5956ffd640 1 -- 172.21.15.154:0/2071719156 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5948006270 con 0x7f5944005bb0 2023-09-23T03:25:38.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.556+0000 7f595de75640 1 -- 172.21.15.154:0/2071719156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5944005bb0 msgr2=0x7f5944005fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:38.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.556+0000 7f595de75640 1 --2- 172.21.15.154:0/2071719156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5944005bb0 0x7f5944005fd0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f594800ba10 tx=0x7f5944007a00 comp rx=0 tx=0).stop 2023-09-23T03:25:38.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.556+0000 7f5957fff640 1 -- 172.21.15.154:0/2071719156 reap_dead start 2023-09-23T03:25:38.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.556+0000 7f595de75640 1 -- 172.21.15.154:0/2071719156 shutdown_connections 2023-09-23T03:25:38.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.556+0000 7f595de75640 1 -- 172.21.15.154:0/2071719156 >> 172.21.15.154:0/2071719156 conn(0x7f595806a0f0 msgr2=0x7f595806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:38.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.556+0000 7f595de75640 1 -- 172.21.15.154:0/2071719156 shutdown_connections 2023-09-23T03:25:38.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.556+0000 7f595de75640 1 -- 172.21.15.154:0/2071719156 wait complete. 2023-09-23T03:25:38.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.558+0000 7f595de75640 1 Processor -- start 2023-09-23T03:25:38.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.558+0000 7f595de75640 1 -- start start 2023-09-23T03:25:38.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.559+0000 7f595de75640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5944005bb0 0x7f59581f65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:38.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.559+0000 7f595de75640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5958158500 0x7f59580c34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:38.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.559+0000 7f595de75640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59580b02d0 0x7f59580b0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:38.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.559+0000 7f595de75640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59581f6ae0 con 0x7f5958158500 2023-09-23T03:25:38.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.559+0000 7f595de75640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59580b0c90 con 0x7f59580b02d0 2023-09-23T03:25:38.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.559+0000 7f595de75640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59580ae3e0 con 0x7f5944005bb0 2023-09-23T03:25:38.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.559+0000 7f594ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5958158500 0x7f59580c34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:38.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.559+0000 7f594ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5958158500 0x7f59580c34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50094/0 (socket says 172.21.15.154:50094) 2023-09-23T03:25:38.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.559+0000 7f594ffff640 1 -- 172.21.15.154:0/3102002092 learned_addr learned my addr 172.21.15.154:0/3102002092 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:38.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.560+0000 7f594ffff640 1 -- 172.21.15.154:0/3102002092 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5944005bb0 msgr2=0x7f59581f65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:38.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.560+0000 7f594ffff640 1 --2- 172.21.15.154:0/3102002092 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5944005bb0 0x7f59581f65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:38.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.560+0000 7f594ffff640 1 -- 172.21.15.154:0/3102002092 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59580b02d0 msgr2=0x7f59580b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:38.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.560+0000 7f594ffff640 1 --2- 172.21.15.154:0/3102002092 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59580b02d0 0x7f59580b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:38.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.560+0000 7f594ffff640 1 -- 172.21.15.154:0/3102002092 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5958158c60 con 0x7f5958158500 2023-09-23T03:25:38.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.560+0000 7f594ffff640 1 --2- 172.21.15.154:0/3102002092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5958158500 0x7f59580c34d0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f594000c790 tx=0x7f594000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:38.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.560+0000 7f5954ff9640 1 -- 172.21.15.154:0/3102002092 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59400074e0 con 0x7f5958158500 2023-09-23T03:25:38.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.560+0000 7f5954ff9640 1 -- 172.21.15.154:0/3102002092 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5940007640 con 0x7f5958158500 2023-09-23T03:25:38.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.560+0000 7f5954ff9640 1 -- 172.21.15.154:0/3102002092 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5940004a00 con 0x7f5958158500 2023-09-23T03:25:38.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.560+0000 7f595de75640 1 -- 172.21.15.154:0/3102002092 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59580ae630 con 0x7f5958158500 2023-09-23T03:25:38.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.560+0000 7f595de75640 1 -- 172.21.15.154:0/3102002092 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59580aebf0 con 0x7f5958158500 2023-09-23T03:25:38.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.561+0000 7f595de75640 1 -- 172.21.15.154:0/3102002092 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5958073eb0 con 0x7f5958158500 2023-09-23T03:25:38.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.562+0000 7f5954ff9640 1 -- 172.21.15.154:0/3102002092 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f5940004b60 con 0x7f5958158500 2023-09-23T03:25:38.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.563+0000 7f5954ff9640 1 --2- 172.21.15.154:0/3102002092 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f593806af30 0x7f593806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:38.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.563+0000 7f5954ff9640 1 -- 172.21.15.154:0/3102002092 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5521+0+0 (secure 0 0 0) 0x7f5940088a90 con 0x7f5958158500 2023-09-23T03:25:38.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.564+0000 7f59577fe640 1 --2- 172.21.15.154:0/3102002092 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f593806af30 0x7f593806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:38.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.564+0000 7f59577fe640 1 --2- 172.21.15.154:0/3102002092 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f593806af30 0x7f593806d3f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5948036000 tx=0x7f5948031000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:38.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.568+0000 7f5954ff9640 1 -- 172.21.15.154:0/3102002092 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f5940053ae0 con 0x7f5958158500 2023-09-23T03:25:38.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:38.808+0000 7f595de75640 1 -- 172.21.15.154:0/3102002092 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_stashed_layout"]} v 0) v1 -- 0x7f595806ceb0 con 0x7f5958158500 2023-09-23T03:25:39.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.473+0000 7f5954ff9640 1 -- 172.21.15.154:0/3102002092 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_stashed_layout"]}]=0 v462) v1 ==== 158+0+0 (secure 0 0 0) 0x7f595806ceb0 con 0x7f5958158500 2023-09-23T03:25:39.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.479+0000 7f595de75640 1 -- 172.21.15.154:0/3102002092 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f593806af30 msgr2=0x7f593806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:39.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.479+0000 7f595de75640 1 --2- 172.21.15.154:0/3102002092 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f593806af30 0x7f593806d3f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5948036000 tx=0x7f5948031000 comp rx=0 tx=0).stop 2023-09-23T03:25:39.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.479+0000 7f595de75640 1 -- 172.21.15.154:0/3102002092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5958158500 msgr2=0x7f59580c34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:39.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.479+0000 7f595de75640 1 --2- 172.21.15.154:0/3102002092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5958158500 0x7f59580c34d0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f594000c790 tx=0x7f594000cc50 comp rx=0 tx=0).stop 2023-09-23T03:25:39.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.479+0000 7f595de75640 1 -- 172.21.15.154:0/3102002092 shutdown_connections 2023-09-23T03:25:39.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.479+0000 7f595de75640 1 --2- 172.21.15.154:0/3102002092 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f593806af30 0x7f593806d3f0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:39.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.479+0000 7f595de75640 1 --2- 172.21.15.154:0/3102002092 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59580b02d0 0x7f59580b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:39.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.479+0000 7f595de75640 1 --2- 172.21.15.154:0/3102002092 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5958158500 0x7f59580c34d0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:39.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.479+0000 7f595de75640 1 --2- 172.21.15.154:0/3102002092 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5944005bb0 0x7f59581f65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:39.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.479+0000 7f595de75640 1 -- 172.21.15.154:0/3102002092 >> 172.21.15.154:0/3102002092 conn(0x7f595806a0f0 msgr2=0x7f59580cc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:39.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.479+0000 7f595de75640 1 -- 172.21.15.154:0/3102002092 shutdown_connections 2023-09-23T03:25:39.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.479+0000 7f595de75640 1 -- 172.21.15.154:0/3102002092 wait complete. 2023-09-23T03:25:39.501 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-09-23T03:25:39.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.717+0000 7f56766d4640 1 Processor -- start 2023-09-23T03:25:39.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.717+0000 7f56766d4640 1 -- start start 2023-09-23T03:25:39.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.719+0000 7f56766d4640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f567015aba0 0x7f567015bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:39.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.719+0000 7f56766d4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f567015e5f0 0x7f567015ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:39.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.719+0000 7f56766d4640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f56700caea0 0x7f56700cb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:39.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.719+0000 7f56766d4640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56700cb870 con 0x7f567015aba0 2023-09-23T03:25:39.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.719+0000 7f56766d4640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f567015b6a0 con 0x7f56700caea0 2023-09-23T03:25:39.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.719+0000 7f56766d4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f567015f0b0 con 0x7f567015e5f0 2023-09-23T03:25:39.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.719+0000 7f566ffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f567015aba0 0x7f567015bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:39.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.719+0000 7f566ffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f567015aba0 0x7f567015bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50108/0 (socket says 172.21.15.154:50108) 2023-09-23T03:25:39.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.719+0000 7f566ffff640 1 -- 172.21.15.154:0/557970206 learned_addr learned my addr 172.21.15.154:0/557970206 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:39.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.719+0000 7f566ffff640 1 -- 172.21.15.154:0/557970206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f567015e5f0 msgr2=0x7f567015ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:39.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.720+0000 7f566ffff640 1 --2- 172.21.15.154:0/557970206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f567015e5f0 0x7f567015ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:39.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.720+0000 7f566ffff640 1 -- 172.21.15.154:0/557970206 >> v2:172.21.15.190:3300/0 conn(0x7f56700caea0 msgr2=0x7f56700cb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:39.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.720+0000 7f566ffff640 1 --2- 172.21.15.154:0/557970206 >> v2:172.21.15.190:3300/0 conn(0x7f56700caea0 0x7f56700cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:39.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.720+0000 7f566ffff640 1 -- 172.21.15.154:0/557970206 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f567015f390 con 0x7f567015aba0 2023-09-23T03:25:39.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.721+0000 7f566ffff640 1 --2- 172.21.15.154:0/557970206 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f567015aba0 0x7f567015bb30 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f56700adf40 tx=0x7f566002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba5ecee4636a195 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:39.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.721+0000 7f566effd640 1 -- 172.21.15.154:0/557970206 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5660031870 con 0x7f567015aba0 2023-09-23T03:25:39.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.721+0000 7f566effd640 1 -- 172.21.15.154:0/557970206 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56600319d0 con 0x7f567015aba0 2023-09-23T03:25:39.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.721+0000 7f566effd640 1 -- 172.21.15.154:0/557970206 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5660031d60 con 0x7f567015aba0 2023-09-23T03:25:39.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.722+0000 7f56766d4640 1 -- 172.21.15.154:0/557970206 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f567015aba0 msgr2=0x7f567015bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:39.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.722+0000 7f56766d4640 1 --2- 172.21.15.154:0/557970206 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f567015aba0 0x7f567015bb30 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f56700adf40 tx=0x7f566002b910 comp rx=0 tx=0).stop 2023-09-23T03:25:39.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.722+0000 7f56766d4640 1 -- 172.21.15.154:0/557970206 shutdown_connections 2023-09-23T03:25:39.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.722+0000 7f56766d4640 1 --2- 172.21.15.154:0/557970206 >> v2:172.21.15.190:3300/0 conn(0x7f56700caea0 0x7f56700cb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:39.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.722+0000 7f56766d4640 1 --2- 172.21.15.154:0/557970206 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f567015e5f0 0x7f567015ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:39.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.722+0000 7f56766d4640 1 --2- 172.21.15.154:0/557970206 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f567015aba0 0x7f567015bb30 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:39.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.722+0000 7f56766d4640 1 -- 172.21.15.154:0/557970206 >> 172.21.15.154:0/557970206 conn(0x7f567006c010 msgr2=0x7f567006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:39.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.722+0000 7f56766d4640 1 -- 172.21.15.154:0/557970206 shutdown_connections 2023-09-23T03:25:39.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.722+0000 7f56766d4640 1 -- 172.21.15.154:0/557970206 wait complete. 2023-09-23T03:25:39.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.724+0000 7f56766d4640 1 Processor -- start 2023-09-23T03:25:39.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.724+0000 7f56766d4640 1 -- start start 2023-09-23T03:25:39.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.726+0000 7f56766d4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56700caea0 0x7f56701469b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:39.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.726+0000 7f56766d4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f567015aba0 0x7f5670155a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:39.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.726+0000 7f56766d4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f567015e5f0 0x7f5670153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:39.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.726+0000 7f56766d4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56700c1df0 con 0x7f56700caea0 2023-09-23T03:25:39.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.726+0000 7f56766d4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56700c8ec0 con 0x7f567015e5f0 2023-09-23T03:25:39.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.726+0000 7f56766d4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5670155f90 con 0x7f567015aba0 2023-09-23T03:25:39.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.726+0000 7f566ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56700caea0 0x7f56701469b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:39.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.726+0000 7f566ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56700caea0 0x7f56701469b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50112/0 (socket says 172.21.15.154:50112) 2023-09-23T03:25:39.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.726+0000 7f566ffff640 1 -- 172.21.15.154:0/228274368 learned_addr learned my addr 172.21.15.154:0/228274368 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:39.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.726+0000 7f566ffff640 1 -- 172.21.15.154:0/228274368 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f567015aba0 msgr2=0x7f5670155a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:39.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.726+0000 7f566ffff640 1 --2- 172.21.15.154:0/228274368 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f567015aba0 0x7f5670155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:39.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.726+0000 7f566ffff640 1 -- 172.21.15.154:0/228274368 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f567015e5f0 msgr2=0x7f5670153000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:39.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.726+0000 7f566ffff640 1 --2- 172.21.15.154:0/228274368 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f567015e5f0 0x7f5670153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:39.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.726+0000 7f566ffff640 1 -- 172.21.15.154:0/228274368 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56600094d0 con 0x7f56700caea0 2023-09-23T03:25:39.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.727+0000 7f566ffff640 1 --2- 172.21.15.154:0/228274368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56700caea0 0x7f56701469b0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f5660000c00 tx=0x7f56600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:39.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.727+0000 7f566cff9640 1 -- 172.21.15.154:0/228274368 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5660031840 con 0x7f56700caea0 2023-09-23T03:25:39.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.727+0000 7f566cff9640 1 -- 172.21.15.154:0/228274368 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56600443f0 con 0x7f56700caea0 2023-09-23T03:25:39.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.727+0000 7f566cff9640 1 -- 172.21.15.154:0/228274368 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56600447a0 con 0x7f56700caea0 2023-09-23T03:25:39.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.727+0000 7f56766d4640 1 -- 172.21.15.154:0/228274368 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56700c9130 con 0x7f56700caea0 2023-09-23T03:25:39.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.727+0000 7f56766d4640 1 -- 172.21.15.154:0/228274368 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56700c96f0 con 0x7f56700caea0 2023-09-23T03:25:39.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.728+0000 7f56766d4640 1 -- 172.21.15.154:0/228274368 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f567004e920 con 0x7f56700caea0 2023-09-23T03:25:39.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.730+0000 7f566cff9640 1 -- 172.21.15.154:0/228274368 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f5660044900 con 0x7f56700caea0 2023-09-23T03:25:39.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.731+0000 7f566cff9640 1 --2- 172.21.15.154:0/228274368 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f565c06af30 0x7f565c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:39.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.731+0000 7f566cff9640 1 -- 172.21.15.154:0/228274368 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5521+0+0 (secure 0 0 0) 0x7f566002d040 con 0x7f56700caea0 2023-09-23T03:25:39.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.731+0000 7f566f7fe640 1 --2- 172.21.15.154:0/228274368 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f565c06af30 0x7f565c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:39.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.732+0000 7f566f7fe640 1 --2- 172.21.15.154:0/228274368 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f565c06af30 0x7f565c06d3f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f56700b4080 tx=0x7f5664006c90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:39.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.736+0000 7f566cff9640 1 -- 172.21.15.154:0/228274368 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f5660077110 con 0x7f56700caea0 2023-09-23T03:25:39.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.924+0000 7f56766d4640 1 -- 172.21.15.154:0/228274368 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f567015c1d0 con 0x7f56700caea0 2023-09-23T03:25:39.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.929+0000 7f566cff9640 1 -- 172.21.15.154:0/228274368 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f566007a730 con 0x7f56700caea0 2023-09-23T03:25:39.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.932+0000 7f56766d4640 1 -- 172.21.15.154:0/228274368 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f565c06af30 msgr2=0x7f565c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:39.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.932+0000 7f56766d4640 1 --2- 172.21.15.154:0/228274368 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f565c06af30 0x7f565c06d3f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f56700b4080 tx=0x7f5664006c90 comp rx=0 tx=0).stop 2023-09-23T03:25:39.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.932+0000 7f56766d4640 1 -- 172.21.15.154:0/228274368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56700caea0 msgr2=0x7f56701469b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:39.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.932+0000 7f56766d4640 1 --2- 172.21.15.154:0/228274368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56700caea0 0x7f56701469b0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f5660000c00 tx=0x7f56600046c0 comp rx=0 tx=0).stop 2023-09-23T03:25:39.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.932+0000 7f56766d4640 1 -- 172.21.15.154:0/228274368 shutdown_connections 2023-09-23T03:25:39.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.932+0000 7f56766d4640 1 --2- 172.21.15.154:0/228274368 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f565c06af30 0x7f565c06d3f0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:39.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.932+0000 7f56766d4640 1 --2- 172.21.15.154:0/228274368 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f567015e5f0 0x7f5670153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:39.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.932+0000 7f56766d4640 1 --2- 172.21.15.154:0/228274368 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f567015aba0 0x7f5670155a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:39.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.932+0000 7f56766d4640 1 --2- 172.21.15.154:0/228274368 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f56700caea0 0x7f56701469b0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:39.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.932+0000 7f56766d4640 1 -- 172.21.15.154:0/228274368 >> 172.21.15.154:0/228274368 conn(0x7f567006c010 msgr2=0x7f56700701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:39.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.932+0000 7f56766d4640 1 -- 172.21.15.154:0/228274368 shutdown_connections 2023-09-23T03:25:39.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:39.932+0000 7f56766d4640 1 -- 172.21.15.154:0/228274368 wait complete. 2023-09-23T03:25:39.951 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-09-23T03:25:39.951 DEBUG:teuthology.orchestra.run.smithi154:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-23T03:25:40.065 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-23T03:25:40.187 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-09-23T03:25:40.188 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:25:40.188 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:25:40.245 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053600,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-09-23T03:25:40.246 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:25:40.246 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:25:40.305 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053603,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248804k,nr_inodes=812201,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-09-23T03:25:40.305 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:25:40.305 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:25:40.365 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053600,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-09-23T03:25:40.366 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:25:40.366 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:25:40.426 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053603,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248804k,nr_inodes=812201,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-09-23T03:25:40.426 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-09-23T03:25:40.426 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:25:40.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.650+0000 7f2efcc72640 1 Processor -- start 2023-09-23T03:25:40.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.651+0000 7f2efcc72640 1 -- start start 2023-09-23T03:25:40.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.652+0000 7f2efcc72640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2ef80cf8e0 0x7f2ef815c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:40.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.652+0000 7f2efcc72640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2ef815ca30 0x7f2ef80b1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:40.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.652+0000 7f2efcc72640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ef80d17f0 con 0x7f2ef80cf500 2023-09-23T03:25:40.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.652+0000 7f2efcc72640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ef806be10 con 0x7f2ef815ca30 2023-09-23T03:25:40.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.652+0000 7f2efcc72640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ef80d0470 con 0x7f2ef80cf8e0 2023-09-23T03:25:40.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.652+0000 7f2ef7fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2ef815ca30 0x7f2ef80b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:40.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.653+0000 7f2ef7fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2ef815ca30 0x7f2ef80b1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50126/0 (socket says 172.21.15.154:50126) 2023-09-23T03:25:40.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.653+0000 7f2ef7fff640 1 -- 172.21.15.154:0/1751063041 learned_addr learned my addr 172.21.15.154:0/1751063041 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:40.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.653+0000 7f2eeffff640 1 --2- 172.21.15.154:0/1751063041 >> v2:172.21.15.190:3300/0 conn(0x7f2ef80cf8e0 0x7f2ef815c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:40.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.653+0000 7f2ef7fff640 1 -- 172.21.15.154:0/1751063041 >> v2:172.21.15.190:3300/0 conn(0x7f2ef80cf8e0 msgr2=0x7f2ef815c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:40.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.653+0000 7f2ef7fff640 1 --2- 172.21.15.154:0/1751063041 >> v2:172.21.15.190:3300/0 conn(0x7f2ef80cf8e0 0x7f2ef815c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:40.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.653+0000 7f2ef7fff640 1 -- 172.21.15.154:0/1751063041 >> v1:172.21.15.154:6789/0 conn(0x7f2ef80cf500 legacy=0x7f2ef80ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:40.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.653+0000 7f2ef7fff640 1 -- 172.21.15.154:0/1751063041 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef80d0750 con 0x7f2ef815ca30 2023-09-23T03:25:40.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.654+0000 7f2ef7fff640 1 --2- 172.21.15.154:0/1751063041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ef815ca30 0x7f2ef80b1430 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0009e50 tx=0x7f2ef002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b2ec59fcaa76461 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:40.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.654+0000 7f2ef6ffd640 1 -- 172.21.15.154:0/1751063041 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef002ca80 con 0x7f2ef815ca30 2023-09-23T03:25:40.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.655+0000 7f2ef6ffd640 1 -- 172.21.15.154:0/1751063041 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ef002cbe0 con 0x7f2ef815ca30 2023-09-23T03:25:40.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.655+0000 7f2ef6ffd640 1 -- 172.21.15.154:0/1751063041 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef00343f0 con 0x7f2ef815ca30 2023-09-23T03:25:40.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.655+0000 7f2efcc72640 1 -- 172.21.15.154:0/1751063041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ef815ca30 msgr2=0x7f2ef80b1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:40.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.655+0000 7f2efcc72640 1 --2- 172.21.15.154:0/1751063041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ef815ca30 0x7f2ef80b1430 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0009e50 tx=0x7f2ef002bee0 comp rx=0 tx=0).stop 2023-09-23T03:25:40.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.655+0000 7f2efcc72640 1 -- 172.21.15.154:0/1751063041 shutdown_connections 2023-09-23T03:25:40.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.655+0000 7f2efcc72640 1 --2- 172.21.15.154:0/1751063041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ef815ca30 0x7f2ef80b1430 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:40.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.655+0000 7f2efcc72640 1 --2- 172.21.15.154:0/1751063041 >> v2:172.21.15.190:3300/0 conn(0x7f2ef80cf8e0 0x7f2ef815c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:40.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.655+0000 7f2efcc72640 1 -- 172.21.15.154:0/1751063041 >> 172.21.15.154:0/1751063041 conn(0x7f2ef809bbb0 msgr2=0x7f2ef80d28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:40.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.655+0000 7f2efcc72640 1 -- 172.21.15.154:0/1751063041 shutdown_connections 2023-09-23T03:25:40.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.655+0000 7f2efcc72640 1 -- 172.21.15.154:0/1751063041 wait complete. 2023-09-23T03:25:40.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.657+0000 7f2efcc72640 1 Processor -- start 2023-09-23T03:25:40.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.657+0000 7f2efcc72640 1 -- start start 2023-09-23T03:25:40.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.658+0000 7f2efcc72640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ef80cf500 0x7f2ef81f4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:40.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.658+0000 7f2efcc72640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ef80cf8e0 0x7f2ef8155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:40.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.658+0000 7f2efcc72640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ef815ca30 0x7f2ef80c6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:40.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.658+0000 7f2efcc72640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef80c70b0 con 0x7f2ef80cf500 2023-09-23T03:25:40.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.658+0000 7f2efcc72640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef8153450 con 0x7f2ef815ca30 2023-09-23T03:25:40.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.658+0000 7f2efcc72640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef80c8740 con 0x7f2ef80cf8e0 2023-09-23T03:25:40.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.659+0000 7f2ef77fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ef80cf500 0x7f2ef81f4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:40.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.659+0000 7f2ef77fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ef80cf500 0x7f2ef81f4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50132/0 (socket says 172.21.15.154:50132) 2023-09-23T03:25:40.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.659+0000 7f2ef77fe640 1 -- 172.21.15.154:0/3270648433 learned_addr learned my addr 172.21.15.154:0/3270648433 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:40.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.659+0000 7f2ef7fff640 1 --2- 172.21.15.154:0/3270648433 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ef815ca30 0x7f2ef80c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:40.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.659+0000 7f2eeffff640 1 --2- 172.21.15.154:0/3270648433 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ef80cf8e0 0x7f2ef8155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:40.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.659+0000 7f2ef77fe640 1 -- 172.21.15.154:0/3270648433 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ef80cf8e0 msgr2=0x7f2ef8155460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:40.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.659+0000 7f2ef77fe640 1 --2- 172.21.15.154:0/3270648433 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ef80cf8e0 0x7f2ef8155460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:40.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.659+0000 7f2ef77fe640 1 -- 172.21.15.154:0/3270648433 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ef815ca30 msgr2=0x7f2ef80c6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:40.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.659+0000 7f2ef77fe640 1 --2- 172.21.15.154:0/3270648433 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ef815ca30 0x7f2ef80c6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:40.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.659+0000 7f2ef77fe640 1 -- 172.21.15.154:0/3270648433 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef0009b30 con 0x7f2ef80cf500 2023-09-23T03:25:40.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.659+0000 7f2ef77fe640 1 --2- 172.21.15.154:0/3270648433 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ef80cf500 0x7f2ef81f4ef0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f2ee800c3c0 tx=0x7f2ee800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:40.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.660+0000 7f2ef4ff9640 1 -- 172.21.15.154:0/3270648433 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee8011790 con 0x7f2ef80cf500 2023-09-23T03:25:40.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.660+0000 7f2ef4ff9640 1 -- 172.21.15.154:0/3270648433 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee80118f0 con 0x7f2ef80cf500 2023-09-23T03:25:40.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.660+0000 7f2ef4ff9640 1 -- 172.21.15.154:0/3270648433 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee8011ce0 con 0x7f2ef80cf500 2023-09-23T03:25:40.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.660+0000 7f2efcc72640 1 -- 172.21.15.154:0/3270648433 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ef80c8a20 con 0x7f2ef80cf500 2023-09-23T03:25:40.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.660+0000 7f2efcc72640 1 -- 172.21.15.154:0/3270648433 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef81530e0 con 0x7f2ef80cf500 2023-09-23T03:25:40.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.661+0000 7f2efcc72640 1 -- 172.21.15.154:0/3270648433 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ef804e920 con 0x7f2ef80cf500 2023-09-23T03:25:40.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.662+0000 7f2ef4ff9640 1 -- 172.21.15.154:0/3270648433 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f2ee801c460 con 0x7f2ef80cf500 2023-09-23T03:25:40.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.663+0000 7f2ef4ff9640 1 --2- 172.21.15.154:0/3270648433 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2ed806af60 0x7f2ed806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:40.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.663+0000 7f2ef4ff9640 1 -- 172.21.15.154:0/3270648433 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5521+0+0 (secure 0 0 0) 0x7f2ee80894d0 con 0x7f2ef80cf500 2023-09-23T03:25:40.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.663+0000 7f2eeffff640 1 --2- 172.21.15.154:0/3270648433 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2ed806af60 0x7f2ed806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:40.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.664+0000 7f2eeffff640 1 --2- 172.21.15.154:0/3270648433 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2ed806af60 0x7f2ed806d420 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8156c70 tx=0x7f2ee0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:40.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.667+0000 7f2ef4ff9640 1 -- 172.21.15.154:0/3270648433 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f2ee80545a0 con 0x7f2ef80cf500 2023-09-23T03:25:40.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.894+0000 7f2efcc72640 1 -- 172.21.15.154:0/3270648433 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ef80d0fc0 con 0x7f2ef80cf500 2023-09-23T03:25:40.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.896+0000 7f2ef4ff9640 1 -- 172.21.15.154:0/3270648433 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f2ee8057bc0 con 0x7f2ef80cf500 2023-09-23T03:25:40.896 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:25:40.896 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":78,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6148,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/845236634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":845236634},{"type":"v1","addr":"172.21.15.190:6835","nonce":845236634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":7743,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/31599836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":31599836},{"type":"v1","addr":"172.21.15.154:6836","nonce":31599836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":7752,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/4266451911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":4266451911},{"type":"v1","addr":"172.21.15.154:6837","nonce":4266451911}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":7755,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/3029600822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":3029600822},{"type":"v1","addr":"172.21.15.190:6834","nonce":3029600822}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[]} 2023-09-23T03:25:40.897 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 78 2023-09-23T03:25:40.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.898+0000 7f2efcc72640 1 -- 172.21.15.154:0/3270648433 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2ed806af60 msgr2=0x7f2ed806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:40.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.898+0000 7f2efcc72640 1 --2- 172.21.15.154:0/3270648433 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2ed806af60 0x7f2ed806d420 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8156c70 tx=0x7f2ee0008040 comp rx=0 tx=0).stop 2023-09-23T03:25:40.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.898+0000 7f2efcc72640 1 -- 172.21.15.154:0/3270648433 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ef80cf500 msgr2=0x7f2ef81f4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:40.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.898+0000 7f2efcc72640 1 --2- 172.21.15.154:0/3270648433 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ef80cf500 0x7f2ef81f4ef0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f2ee800c3c0 tx=0x7f2ee800c880 comp rx=0 tx=0).stop 2023-09-23T03:25:40.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.898+0000 7f2efcc72640 1 -- 172.21.15.154:0/3270648433 shutdown_connections 2023-09-23T03:25:40.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.898+0000 7f2efcc72640 1 --2- 172.21.15.154:0/3270648433 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2ed806af60 0x7f2ed806d420 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:40.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.898+0000 7f2efcc72640 1 --2- 172.21.15.154:0/3270648433 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2ef815ca30 0x7f2ef80c6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:40.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.898+0000 7f2efcc72640 1 --2- 172.21.15.154:0/3270648433 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2ef80cf8e0 0x7f2ef8155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:40.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.898+0000 7f2efcc72640 1 --2- 172.21.15.154:0/3270648433 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2ef80cf500 0x7f2ef81f4ef0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:40.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.898+0000 7f2efcc72640 1 -- 172.21.15.154:0/3270648433 >> 172.21.15.154:0/3270648433 conn(0x7f2ef809bbb0 msgr2=0x7f2ef80b5760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:40.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.898+0000 7f2efcc72640 1 -- 172.21.15.154:0/3270648433 shutdown_connections 2023-09-23T03:25:40.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:40.899+0000 7f2efcc72640 1 -- 172.21.15.154:0/3270648433 wait complete. 2023-09-23T03:25:40.917 INFO:tasks.ceph.mds.a:Restarting daemon 2023-09-23T03:25:40.917 INFO:tasks.ceph.mds.a:Stopping old one... 2023-09-23T03:25:40.917 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-09-23T03:25:40.917 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:25:40.918 INFO:tasks.ceph.mds.c:Restarting daemon 2023-09-23T03:25:40.918 INFO:tasks.ceph.mds.c:Stopping old one... 2023-09-23T03:25:40.918 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-09-23T03:25:40.918 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:25:40.919 INFO:tasks.ceph.mds.b:Restarting daemon 2023-09-23T03:25:40.919 INFO:tasks.ceph.mds.b:Stopping old one... 2023-09-23T03:25:40.919 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-09-23T03:25:40.920 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:25:40.920 INFO:tasks.ceph.mds.d:Restarting daemon 2023-09-23T03:25:40.920 INFO:tasks.ceph.mds.d:Stopping old one... 2023-09-23T03:25:40.920 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-09-23T03:25:40.920 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:25:40.935 INFO:tasks.ceph.mds.a:Stopped 2023-09-23T03:25:40.935 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-09-23T03:25:40.937 INFO:tasks.ceph.mds.c:Stopped 2023-09-23T03:25:40.937 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-09-23T03:25:40.943 INFO:tasks.ceph.mds.b:Stopped 2023-09-23T03:25:40.943 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-09-23T03:25:40.945 INFO:tasks.ceph.mds.d:Stopped 2023-09-23T03:25:40.945 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-09-23T03:25:40.985 INFO:tasks.ceph.mds.a:Started 2023-09-23T03:25:40.985 DEBUG:teuthology.parallel:result is None 2023-09-23T03:25:40.993 INFO:tasks.ceph.mds.c:Started 2023-09-23T03:25:40.993 DEBUG:teuthology.parallel:result is None 2023-09-23T03:25:40.994 INFO:tasks.ceph.mds.b:Started 2023-09-23T03:25:40.994 DEBUG:teuthology.parallel:result is None 2023-09-23T03:25:41.002 INFO:tasks.ceph.mds.d:Started 2023-09-23T03:25:41.003 DEBUG:teuthology.parallel:result is None 2023-09-23T03:25:41.003 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-09-23T03:25:41.146 INFO:tasks.ceph.mds.a.smithi154.stdout:starting mds.a at 2023-09-23T03:25:41.147 INFO:tasks.ceph.mds.c.smithi154.stdout:starting mds.c at 2023-09-23T03:25:41.159 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-09-23T03:25:41.160 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-09-23T03:25:41.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.194+0000 7fe682410640 1 Processor -- start 2023-09-23T03:25:41.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.194+0000 7fe682410640 1 -- start start 2023-09-23T03:25:41.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.195+0000 7fe682410640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe67c160680 0x7fe67c0b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:41.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.195+0000 7fe682410640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe67c161500 con 0x7fe67c15ca80 2023-09-23T03:25:41.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.195+0000 7fe682410640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe67c160c60 con 0x7fe67c0d25b0 2023-09-23T03:25:41.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.195+0000 7fe682410640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe67c160f40 con 0x7fe67c160680 2023-09-23T03:25:41.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.195+0000 7fe67bfff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fe67c15ca80 0x7fe67c0d07b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46910/0 (socket says 172.21.15.154:46910) 2023-09-23T03:25:41.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.195+0000 7fe67bfff640 1 -- 172.21.15.154:0/366828749 learned_addr learned my addr 172.21.15.154:0/366828749 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:25:41.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.195+0000 7fe673fff640 1 --2- 172.21.15.154:0/366828749 >> v2:172.21.15.154:3300/0 conn(0x7fe67c160680 0x7fe67c0b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:41.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.196+0000 7fe67b7fe640 1 -- 172.21.15.154:0/366828749 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4162231212 0 0) 0x7fe67c161500 con 0x7fe67c15ca80 2023-09-23T03:25:41.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.196+0000 7fe67b7fe640 1 -- 172.21.15.154:0/366828749 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe668003520 con 0x7fe67c15ca80 2023-09-23T03:25:41.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.196+0000 7fe673fff640 1 -- 172.21.15.154:0/366828749 >> v1:172.21.15.190:6789/0 conn(0x7fe67c0d25b0 legacy=0x7fe67c0d2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:41.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.196+0000 7fe673fff640 1 -- 172.21.15.154:0/366828749 >> v1:172.21.15.154:6789/0 conn(0x7fe67c15ca80 legacy=0x7fe67c0d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:41.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.196+0000 7fe673fff640 1 -- 172.21.15.154:0/366828749 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe67c161220 con 0x7fe67c160680 2023-09-23T03:25:41.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.197+0000 7fe673fff640 1 --2- 172.21.15.154:0/366828749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe67c160680 0x7fe67c0b2620 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fe6640097a0 tx=0x7fe66402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b02ddcce655ba45 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:41.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.197+0000 7fe67b7fe640 1 -- 172.21.15.154:0/366828749 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe664031830 con 0x7fe67c160680 2023-09-23T03:25:41.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.198+0000 7fe67b7fe640 1 -- 172.21.15.154:0/366828749 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe664031990 con 0x7fe67c160680 2023-09-23T03:25:41.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.198+0000 7fe67b7fe640 1 -- 172.21.15.154:0/366828749 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe664031d20 con 0x7fe67c160680 2023-09-23T03:25:41.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.198+0000 7fe682410640 1 -- 172.21.15.154:0/366828749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe67c160680 msgr2=0x7fe67c0b2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:41.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.198+0000 7fe682410640 1 --2- 172.21.15.154:0/366828749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe67c160680 0x7fe67c0b2620 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fe6640097a0 tx=0x7fe66402b8d0 comp rx=0 tx=0).stop 2023-09-23T03:25:41.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.198+0000 7fe682410640 1 -- 172.21.15.154:0/366828749 shutdown_connections 2023-09-23T03:25:41.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.198+0000 7fe682410640 1 --2- 172.21.15.154:0/366828749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe67c160680 0x7fe67c0b2620 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:41.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.198+0000 7fe682410640 1 -- 172.21.15.154:0/366828749 >> 172.21.15.154:0/366828749 conn(0x7fe67c09bc60 msgr2=0x7fe67c089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:41.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.198+0000 7fe682410640 1 -- 172.21.15.154:0/366828749 shutdown_connections 2023-09-23T03:25:41.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.198+0000 7fe682410640 1 -- 172.21.15.154:0/366828749 wait complete. 2023-09-23T03:25:41.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.200+0000 7fe682410640 1 Processor -- start 2023-09-23T03:25:41.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.200+0000 7fe682410640 1 -- start start 2023-09-23T03:25:41.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.201+0000 7fe682410640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe67c0d25b0 0x7fe67c1f2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:41.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.201+0000 7fe682410640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe67c15ca80 0x7fe67c159030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:41.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.201+0000 7fe682410640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe67c160680 0x7fe67c15eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:41.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.201+0000 7fe682410640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c160050 con 0x7fe67c15ca80 2023-09-23T03:25:41.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.201+0000 7fe682410640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c15f3d0 con 0x7fe67c0d25b0 2023-09-23T03:25:41.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.201+0000 7fe682410640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c15f6b0 con 0x7fe67c160680 2023-09-23T03:25:41.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.201+0000 7fe673fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe67c15ca80 0x7fe67c159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:41.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.201+0000 7fe673fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe67c15ca80 0x7fe67c159030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50184/0 (socket says 172.21.15.154:50184) 2023-09-23T03:25:41.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.201+0000 7fe673fff640 1 -- 172.21.15.154:0/1695027570 learned_addr learned my addr 172.21.15.154:0/1695027570 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:41.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.201+0000 7fe67bfff640 1 --2- 172.21.15.154:0/1695027570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe67c0d25b0 0x7fe67c1f2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:41.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.202+0000 7fe6809ad640 1 --2- 172.21.15.154:0/1695027570 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe67c160680 0x7fe67c15eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:41.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.202+0000 7fe673fff640 1 -- 172.21.15.154:0/1695027570 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe67c160680 msgr2=0x7fe67c15eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:41.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.202+0000 7fe673fff640 1 --2- 172.21.15.154:0/1695027570 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe67c160680 0x7fe67c15eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:41.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.202+0000 7fe673fff640 1 -- 172.21.15.154:0/1695027570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe67c0d25b0 msgr2=0x7fe67c1f2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:41.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.202+0000 7fe673fff640 1 --2- 172.21.15.154:0/1695027570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe67c0d25b0 0x7fe67c1f2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:41.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.202+0000 7fe673fff640 1 -- 172.21.15.154:0/1695027570 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe664009400 con 0x7fe67c15ca80 2023-09-23T03:25:41.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.202+0000 7fe673fff640 1 --2- 172.21.15.154:0/1695027570 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe67c15ca80 0x7fe67c159030 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fe66402bee0 tx=0x7fe6640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:41.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.203+0000 7fe6797fa640 1 -- 172.21.15.154:0/1695027570 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66403a6f0 con 0x7fe67c15ca80 2023-09-23T03:25:41.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.203+0000 7fe6797fa640 1 -- 172.21.15.154:0/1695027570 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe66403a870 con 0x7fe67c15ca80 2023-09-23T03:25:41.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.203+0000 7fe682410640 1 -- 172.21.15.154:0/1695027570 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe67c15f990 con 0x7fe67c15ca80 2023-09-23T03:25:41.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.203+0000 7fe6797fa640 1 -- 172.21.15.154:0/1695027570 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66403ac20 con 0x7fe67c15ca80 2023-09-23T03:25:41.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.203+0000 7fe682410640 1 -- 172.21.15.154:0/1695027570 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe67c159570 con 0x7fe67c15ca80 2023-09-23T03:25:41.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.204+0000 7fe682410640 1 -- 172.21.15.154:0/1695027570 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe67c073eb0 con 0x7fe67c15ca80 2023-09-23T03:25:41.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.205+0000 7fe6797fa640 1 -- 172.21.15.154:0/1695027570 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fe66404c3b0 con 0x7fe67c15ca80 2023-09-23T03:25:41.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.206+0000 7fe6797fa640 1 --2- 172.21.15.154:0/1695027570 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe65c06af30 0x7fe65c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:41.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.206+0000 7fe6797fa640 1 -- 172.21.15.154:0/1695027570 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5521+0+0 (secure 0 0 0) 0x7fe6640ab2e0 con 0x7fe67c15ca80 2023-09-23T03:25:41.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.206+0000 7fe67bfff640 1 --2- 172.21.15.154:0/1695027570 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe65c06af30 0x7fe65c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:41.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.206+0000 7fe67bfff640 1 --2- 172.21.15.154:0/1695027570 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe65c06af30 0x7fe65c06d3f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe66c0010e0 tx=0x7fe66c007be0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:41.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.210+0000 7fe6797fa640 1 -- 172.21.15.154:0/1695027570 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fe6640764b0 con 0x7fe67c15ca80 2023-09-23T03:25:41.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:41.416+0000 7fe682410640 1 -- 172.21.15.154:0/1695027570 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fe67c0d01f0 con 0x7fe67c15ca80 2023-09-23T03:25:42.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.393+0000 7fe6797fa640 1 -- 172.21.15.154:0/1695027570 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v101) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe664079ad0 con 0x7fe67c15ca80 2023-09-23T03:25:42.394 INFO:teuthology.orchestra.run.smithi154.stderr: removed all blocklist entries 2023-09-23T03:25:42.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.397+0000 7fe682410640 1 -- 172.21.15.154:0/1695027570 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe65c06af30 msgr2=0x7fe65c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:42.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.397+0000 7fe682410640 1 --2- 172.21.15.154:0/1695027570 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe65c06af30 0x7fe65c06d3f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe66c0010e0 tx=0x7fe66c007be0 comp rx=0 tx=0).stop 2023-09-23T03:25:42.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.397+0000 7fe682410640 1 -- 172.21.15.154:0/1695027570 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe67c15ca80 msgr2=0x7fe67c159030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:42.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.397+0000 7fe682410640 1 --2- 172.21.15.154:0/1695027570 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe67c15ca80 0x7fe67c159030 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fe66402bee0 tx=0x7fe6640046c0 comp rx=0 tx=0).stop 2023-09-23T03:25:42.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.397+0000 7fe682410640 1 -- 172.21.15.154:0/1695027570 shutdown_connections 2023-09-23T03:25:42.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.397+0000 7fe682410640 1 --2- 172.21.15.154:0/1695027570 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe65c06af30 0x7fe65c06d3f0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:42.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.397+0000 7fe682410640 1 --2- 172.21.15.154:0/1695027570 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe67c160680 0x7fe67c15eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:42.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.397+0000 7fe682410640 1 --2- 172.21.15.154:0/1695027570 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe67c15ca80 0x7fe67c159030 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:42.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.397+0000 7fe682410640 1 --2- 172.21.15.154:0/1695027570 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe67c0d25b0 0x7fe67c1f2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:42.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.397+0000 7fe682410640 1 -- 172.21.15.154:0/1695027570 >> 172.21.15.154:0/1695027570 conn(0x7fe67c09bc60 msgr2=0x7fe67c0d2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:42.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.397+0000 7fe682410640 1 -- 172.21.15.154:0/1695027570 shutdown_connections 2023-09-23T03:25:42.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.398+0000 7fe682410640 1 -- 172.21.15.154:0/1695027570 wait complete. 2023-09-23T03:25:42.418 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-09-23T03:25:42.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.623+0000 7f785b3ce640 1 Processor -- start 2023-09-23T03:25:42.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.623+0000 7f785b3ce640 1 -- start start 2023-09-23T03:25:42.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.625+0000 7f785b3ce640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f78540acce0 0x7f785415ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:42.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.625+0000 7f785b3ce640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78540b1b20 0x7f78540ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:42.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.625+0000 7f785b3ce640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f78540cee70 0x7f785406cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:42.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.625+0000 7f785b3ce640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78540b2e90 con 0x7f78540acce0 2023-09-23T03:25:42.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.625+0000 7f785b3ce640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78540cf320 con 0x7f78540cee70 2023-09-23T03:25:42.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.625+0000 7f785b3ce640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f785415ac30 con 0x7f78540b1b20 2023-09-23T03:25:42.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.625+0000 7f785916a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f78540acce0 0x7f785415ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:42.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.625+0000 7f785916a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f78540acce0 0x7f785415ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50198/0 (socket says 172.21.15.154:50198) 2023-09-23T03:25:42.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.625+0000 7f785916a640 1 -- 172.21.15.154:0/2680024576 learned_addr learned my addr 172.21.15.154:0/2680024576 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:42.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.625+0000 7f7858969640 1 --2- 172.21.15.154:0/2680024576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78540b1b20 0x7f78540ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:42.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.625+0000 7f785996b640 1 --2- 172.21.15.154:0/2680024576 >> v2:172.21.15.190:3300/0 conn(0x7f78540cee70 0x7f785406cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:42.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.625+0000 7f785916a640 1 -- 172.21.15.154:0/2680024576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78540b1b20 msgr2=0x7f78540ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:42.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.625+0000 7f785916a640 1 --2- 172.21.15.154:0/2680024576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78540b1b20 0x7f78540ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:42.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.625+0000 7f785916a640 1 -- 172.21.15.154:0/2680024576 >> v2:172.21.15.190:3300/0 conn(0x7f78540cee70 msgr2=0x7f785406cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:42.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.625+0000 7f785916a640 1 --2- 172.21.15.154:0/2680024576 >> v2:172.21.15.190:3300/0 conn(0x7f78540cee70 0x7f785406cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:42.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.625+0000 7f785916a640 1 -- 172.21.15.154:0/2680024576 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78540b1f70 con 0x7f78540acce0 2023-09-23T03:25:42.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.626+0000 7f7858969640 1 --2- 172.21.15.154:0/2680024576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78540b1b20 0x7f78540ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:42.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.627+0000 7f785916a640 1 --2- 172.21.15.154:0/2680024576 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78540acce0 0x7f785415ed60 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f78540c9150 tx=0x7f785002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ef69adda48ee73 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:42.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.627+0000 7f784bfff640 1 -- 172.21.15.154:0/2680024576 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785002e070 con 0x7f78540acce0 2023-09-23T03:25:42.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.627+0000 7f784bfff640 1 -- 172.21.15.154:0/2680024576 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7850004030 con 0x7f78540acce0 2023-09-23T03:25:42.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.627+0000 7f784bfff640 1 -- 172.21.15.154:0/2680024576 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7850004420 con 0x7f78540acce0 2023-09-23T03:25:42.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.627+0000 7f785b3ce640 1 -- 172.21.15.154:0/2680024576 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78540acce0 msgr2=0x7f785415ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:42.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.627+0000 7f785b3ce640 1 --2- 172.21.15.154:0/2680024576 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78540acce0 0x7f785415ed60 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f78540c9150 tx=0x7f785002b840 comp rx=0 tx=0).stop 2023-09-23T03:25:42.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.628+0000 7f785b3ce640 1 -- 172.21.15.154:0/2680024576 shutdown_connections 2023-09-23T03:25:42.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.628+0000 7f785b3ce640 1 --2- 172.21.15.154:0/2680024576 >> v2:172.21.15.190:3300/0 conn(0x7f78540cee70 0x7f785406cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:42.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.628+0000 7f785b3ce640 1 --2- 172.21.15.154:0/2680024576 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78540b1b20 0x7f78540ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:42.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.628+0000 7f785b3ce640 1 --2- 172.21.15.154:0/2680024576 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78540acce0 0x7f785415ed60 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:42.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.628+0000 7f785b3ce640 1 -- 172.21.15.154:0/2680024576 >> 172.21.15.154:0/2680024576 conn(0x7f785406a0f0 msgr2=0x7f785406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:42.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.628+0000 7f785b3ce640 1 -- 172.21.15.154:0/2680024576 shutdown_connections 2023-09-23T03:25:42.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.628+0000 7f785b3ce640 1 -- 172.21.15.154:0/2680024576 wait complete. 2023-09-23T03:25:42.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.629+0000 7f785b3ce640 1 Processor -- start 2023-09-23T03:25:42.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.629+0000 7f785b3ce640 1 -- start start 2023-09-23T03:25:42.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.630+0000 7f785b3ce640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78540acce0 0x7f785415d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:42.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.630+0000 7f785b3ce640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78540b1b20 0x7f78540c6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:42.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.630+0000 7f785b3ce640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78540cee70 0x7f78541f19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:42.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.630+0000 7f785b3ce640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78541f1f00 con 0x7f78540b1b20 2023-09-23T03:25:42.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.630+0000 7f785b3ce640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7854156ab0 con 0x7f78540acce0 2023-09-23T03:25:42.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.630+0000 7f785b3ce640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78541554c0 con 0x7f78540cee70 2023-09-23T03:25:42.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.631+0000 7f7858969640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78540b1b20 0x7f78540c6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:42.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.631+0000 7f7858969640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78540b1b20 0x7f78540c6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50208/0 (socket says 172.21.15.154:50208) 2023-09-23T03:25:42.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.631+0000 7f7858969640 1 -- 172.21.15.154:0/4196050860 learned_addr learned my addr 172.21.15.154:0/4196050860 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:42.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.631+0000 7f785916a640 1 --2- 172.21.15.154:0/4196050860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78540acce0 0x7f785415d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:42.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.631+0000 7f785996b640 1 --2- 172.21.15.154:0/4196050860 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78540cee70 0x7f78541f19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:42.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.631+0000 7f7858969640 1 -- 172.21.15.154:0/4196050860 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78540cee70 msgr2=0x7f78541f19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:42.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.631+0000 7f7858969640 1 --2- 172.21.15.154:0/4196050860 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78540cee70 0x7f78541f19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:42.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.631+0000 7f7858969640 1 -- 172.21.15.154:0/4196050860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78540acce0 msgr2=0x7f785415d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:42.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.631+0000 7f7858969640 1 --2- 172.21.15.154:0/4196050860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78540acce0 0x7f785415d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:42.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.631+0000 7f7858969640 1 -- 172.21.15.154:0/4196050860 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7850009460 con 0x7f78540b1b20 2023-09-23T03:25:42.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.631+0000 7f7858969640 1 --2- 172.21.15.154:0/4196050860 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78540b1b20 0x7f78540c6460 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f784400c820 tx=0x7f784400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:42.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.632+0000 7f7849ffb640 1 -- 172.21.15.154:0/4196050860 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7844007a30 con 0x7f78540b1b20 2023-09-23T03:25:42.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.632+0000 7f7849ffb640 1 -- 172.21.15.154:0/4196050860 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7844007b90 con 0x7f78540b1b20 2023-09-23T03:25:42.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.632+0000 7f7849ffb640 1 -- 172.21.15.154:0/4196050860 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78440113f0 con 0x7f78540b1b20 2023-09-23T03:25:42.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.632+0000 7f785b3ce640 1 -- 172.21.15.154:0/4196050860 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78541f5d60 con 0x7f78540b1b20 2023-09-23T03:25:42.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.632+0000 7f785b3ce640 1 -- 172.21.15.154:0/4196050860 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78541f6320 con 0x7f78540b1b20 2023-09-23T03:25:42.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.633+0000 7f785b3ce640 1 -- 172.21.15.154:0/4196050860 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7854155670 con 0x7f78540b1b20 2023-09-23T03:25:42.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.634+0000 7f7849ffb640 1 -- 172.21.15.154:0/4196050860 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f7844011550 con 0x7f78540b1b20 2023-09-23T03:25:42.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.635+0000 7f7849ffb640 1 --2- 172.21.15.154:0/4196050860 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f783806af60 0x7f783806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:42.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.635+0000 7f7849ffb640 1 -- 172.21.15.154:0/4196050860 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f7844089740 con 0x7f78540b1b20 2023-09-23T03:25:42.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.635+0000 7f785916a640 1 --2- 172.21.15.154:0/4196050860 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f783806af60 0x7f783806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:42.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.636+0000 7f785916a640 1 --2- 172.21.15.154:0/4196050860 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f783806af60 0x7f783806d420 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f7850034000 tx=0x7f7850008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:42.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.639+0000 7f7849ffb640 1 -- 172.21.15.154:0/4196050860 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f78440548f0 con 0x7f78540b1b20 2023-09-23T03:25:42.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.892+0000 7f785b3ce640 1 -- 172.21.15.154:0/4196050860 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f78541552b0 con 0x7f78540b1b20 2023-09-23T03:25:42.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.894+0000 7f7849ffb640 1 -- 172.21.15.154:0/4196050860 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f7844057f10 con 0x7f78540b1b20 2023-09-23T03:25:42.894 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:25:42.894 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:25:42.894 INFO:teuthology.orchestra.run.smithi154.stdout: "auth_dump": [ 2023-09-23T03:25:42.894 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:25:42.894 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mds.a", 2023-09-23T03:25:42.895 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAGWA5l9biCFhAAhD2xgY9Q/gGqK0pgHRGweQ==", 2023-09-23T03:25:42.895 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:25:42.895 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:25:42.895 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2023-09-23T03:25:42.895 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2023-09-23T03:25:42.895 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:25:42.895 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:25:42.895 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:25:42.895 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:25:42.895 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mds.b", 2023-09-23T03:25:42.895 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAGWA5lsgKCNxAAI+vD2ESh3uNsb5OCO0ydSw==", 2023-09-23T03:25:42.895 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:25:42.896 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:25:42.896 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2023-09-23T03:25:42.896 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2023-09-23T03:25:42.896 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:25:42.896 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:25:42.896 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:25:42.896 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:25:42.896 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mds.c", 2023-09-23T03:25:42.896 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAGWA5los4vJhAA2/ewjMZcc3pnm9ukhnoNgw==", 2023-09-23T03:25:42.896 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:25:42.896 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:25:42.896 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2023-09-23T03:25:42.897 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2023-09-23T03:25:42.897 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:25:42.897 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:25:42.897 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:25:42.897 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:25:42.897 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mds.d", 2023-09-23T03:25:42.897 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAHWA5lml/MDBAAHyXgd5zl/O8CeVE8Da4w5w==", 2023-09-23T03:25:42.897 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:25:42.897 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:25:42.897 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2023-09-23T03:25:42.897 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2023-09-23T03:25:42.897 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:25:42.897 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:25:42.898 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:25:42.898 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:25:42.898 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.0", 2023-09-23T03:25:42.898 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAMWA5lT1R4BBAAI+itqub/FadG3aU6LhnmGg==", 2023-09-23T03:25:42.898 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:25:42.898 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:25:42.898 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:25:42.898 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:25:42.898 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:25:42.898 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:25:42.898 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:25:42.898 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.1", 2023-09-23T03:25:42.899 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAPWA5lSablHRAAmNyr5Zopeihn5ZyoY5pyhA==", 2023-09-23T03:25:42.899 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:25:42.899 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:25:42.899 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:25:42.899 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:25:42.899 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:25:42.899 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:25:42.899 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:25:42.899 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.2", 2023-09-23T03:25:42.899 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQASWA5lSFcdJxAA3UJJ6eSvxcQ31SAm4shV2g==", 2023-09-23T03:25:42.899 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:25:42.899 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:25:42.900 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:25:42.900 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:25:42.900 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:25:42.900 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:25:42.900 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:25:42.900 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.3", 2023-09-23T03:25:42.900 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAVWA5lmSvzMRAA97YDqX3oplHCB+SENIkLYA==", 2023-09-23T03:25:42.900 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:25:42.900 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:25:42.900 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:25:42.900 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:25:42.900 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:25:42.900 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:25:42.900 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:25:42.901 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.4", 2023-09-23T03:25:42.901 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAcWA5lPZqwMBAAW95bBB0PKS/rwhXvrPf+mQ==", 2023-09-23T03:25:42.901 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:25:42.901 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:25:42.901 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:25:42.901 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:25:42.901 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:25:42.901 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:25:42.901 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:25:42.901 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.5", 2023-09-23T03:25:42.901 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAgWA5lmZySDRAAsbnUdWq9esKFSqSDeqsVlA==", 2023-09-23T03:25:42.901 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:25:42.902 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:25:42.902 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:25:42.902 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:25:42.902 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:25:42.902 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:25:42.902 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:25:42.902 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.6", 2023-09-23T03:25:42.902 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAjWA5ll9WqJRAAD9jecVxVJU+yuqANvN9QUQ==", 2023-09-23T03:25:42.902 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:25:42.902 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:25:42.902 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:25:42.902 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:25:42.903 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:25:42.903 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:25:42.903 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:25:42.903 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.7", 2023-09-23T03:25:42.903 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAnWA5lCvexAxAA/QymygbvOt1HQdCGbO8cyg==", 2023-09-23T03:25:42.903 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:25:42.903 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:25:42.903 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:25:42.903 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:25:42.903 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:25:42.903 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:25:42.903 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:25:42.904 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.0", 2023-09-23T03:25:42.904 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAHWA5lYg+qGhAAjaY0fPsmRTz3ipsKbPugMA==", 2023-09-23T03:25:42.904 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:25:42.904 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:25:42.904 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow r", 2023-09-23T03:25:42.904 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-09-23T03:25:42.904 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:25:42.904 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:25:42.904 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:25:42.904 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.1", 2023-09-23T03:25:42.904 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAHWA5lbrrGMBAAn4kvS/1xGjZ+GtRZFqy31w==", 2023-09-23T03:25:42.905 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:25:42.905 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:25:42.905 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow r", 2023-09-23T03:25:42.905 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow rw", 2023-09-23T03:25:42.905 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow rwx" 2023-09-23T03:25:42.905 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:25:42.905 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:25:42.905 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:25:42.905 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.2", 2023-09-23T03:25:42.905 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAHWA5lcLkIJRAAOknGCKwUs78D4atRzLb/Cw==", 2023-09-23T03:25:42.905 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:25:42.905 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:25:42.905 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow r", 2023-09-23T03:25:42.906 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow rw", 2023-09-23T03:25:42.906 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow rwx" 2023-09-23T03:25:42.906 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:25:42.906 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:25:42.906 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:25:42.906 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.3", 2023-09-23T03:25:42.906 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAIWA5lmFUaABAA4EBSJP5IidG2uq4g8lrFKg==", 2023-09-23T03:25:42.906 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:25:42.906 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:25:42.906 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow r", 2023-09-23T03:25:42.906 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow rw", 2023-09-23T03:25:42.906 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow rwx" 2023-09-23T03:25:42.907 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:25:42.907 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:25:42.907 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:25:42.907 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.admin", 2023-09-23T03:25:42.907 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAFWA5l0x+tCxAAGEYW1egctWzycEhM6FjJZw==", 2023-09-23T03:25:42.907 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:25:42.907 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow *", 2023-09-23T03:25:42.907 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2023-09-23T03:25:42.907 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2023-09-23T03:25:42.907 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:25:42.907 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:25:42.907 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:25:42.908 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:25:42.908 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mgr.x", 2023-09-23T03:25:42.908 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAGWA5ljcy5ChAACUOnrxEGqacSxrB9BkZ+lg==", 2023-09-23T03:25:42.908 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:25:42.908 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow *", 2023-09-23T03:25:42.908 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile mgr", 2023-09-23T03:25:42.908 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:25:42.908 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:25:42.908 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:25:42.908 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:25:42.908 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mgr.y", 2023-09-23T03:25:42.908 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAFWA5looGaOhAAdxmefaoLuXesHXCeqeCYXQ==", 2023-09-23T03:25:42.908 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:25:42.909 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow *", 2023-09-23T03:25:42.909 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile mgr", 2023-09-23T03:25:42.909 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:25:42.909 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:25:42.909 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:25:42.909 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2023-09-23T03:25:42.909 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:25:42.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.896+0000 7f785b3ce640 1 -- 172.21.15.154:0/4196050860 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f783806af60 msgr2=0x7f783806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:42.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.896+0000 7f785b3ce640 1 --2- 172.21.15.154:0/4196050860 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f783806af60 0x7f783806d420 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f7850034000 tx=0x7f7850008030 comp rx=0 tx=0).stop 2023-09-23T03:25:42.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.896+0000 7f785b3ce640 1 -- 172.21.15.154:0/4196050860 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78540b1b20 msgr2=0x7f78540c6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:42.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.896+0000 7f785b3ce640 1 --2- 172.21.15.154:0/4196050860 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78540b1b20 0x7f78540c6460 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f784400c820 tx=0x7f784400cce0 comp rx=0 tx=0).stop 2023-09-23T03:25:42.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.896+0000 7f785b3ce640 1 -- 172.21.15.154:0/4196050860 shutdown_connections 2023-09-23T03:25:42.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.896+0000 7f785b3ce640 1 --2- 172.21.15.154:0/4196050860 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f783806af60 0x7f783806d420 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:42.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.896+0000 7f785b3ce640 1 --2- 172.21.15.154:0/4196050860 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f78540cee70 0x7f78541f19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:42.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.896+0000 7f785b3ce640 1 --2- 172.21.15.154:0/4196050860 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78540b1b20 0x7f78540c6460 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:42.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.896+0000 7f785b3ce640 1 --2- 172.21.15.154:0/4196050860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f78540acce0 0x7f785415d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:42.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.896+0000 7f785b3ce640 1 -- 172.21.15.154:0/4196050860 >> 172.21.15.154:0/4196050860 conn(0x7f785406a0f0 msgr2=0x7f785406a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:42.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.896+0000 7f785b3ce640 1 -- 172.21.15.154:0/4196050860 shutdown_connections 2023-09-23T03:25:42.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:42.897+0000 7f785b3ce640 1 -- 172.21.15.154:0/4196050860 wait complete. 2023-09-23T03:25:42.916 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:25:43.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.131+0000 7f50f53c7640 1 Processor -- start 2023-09-23T03:25:43.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.131+0000 7f50f53c7640 1 -- start start 2023-09-23T03:25:43.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.133+0000 7f50f53c7640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f50f00cafb0 0x7f50f00cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:43.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.133+0000 7f50f53c7640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50f015f880 con 0x7f50f015ad20 2023-09-23T03:25:43.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.133+0000 7f50f53c7640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50f00cb400 con 0x7f50f00cd870 2023-09-23T03:25:43.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.133+0000 7f50f53c7640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50f015ed00 con 0x7f50f00cafb0 2023-09-23T03:25:43.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.134+0000 7f50eeffd640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f50f015ad20 0x7f50f00cd160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46926/0 (socket says 172.21.15.154:46926) 2023-09-23T03:25:43.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.134+0000 7f50eeffd640 1 -- 172.21.15.154:0/378696606 learned_addr learned my addr 172.21.15.154:0/378696606 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:25:43.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.134+0000 7f50ef7fe640 1 --2- 172.21.15.154:0/378696606 >> v2:172.21.15.190:3300/0 conn(0x7f50f00cafb0 0x7f50f00cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:43.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.134+0000 7f50edffb640 1 -- 172.21.15.154:0/378696606 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2251999421 0 0) 0x7f50f015f880 con 0x7f50f015ad20 2023-09-23T03:25:43.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.134+0000 7f50edffb640 1 -- 172.21.15.154:0/378696606 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50dc003520 con 0x7f50f015ad20 2023-09-23T03:25:43.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.134+0000 7f50ef7fe640 1 -- 172.21.15.154:0/378696606 >> v1:172.21.15.190:6789/0 conn(0x7f50f00cd870 legacy=0x7f50f00cef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:43.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.134+0000 7f50ef7fe640 1 -- 172.21.15.154:0/378696606 >> v1:172.21.15.154:6789/0 conn(0x7f50f015ad20 legacy=0x7f50f00cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:43.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.134+0000 7f50edffb640 1 -- 172.21.15.154:0/378696606 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2168199632 0 0) 0x7f50dc003520 con 0x7f50f015ad20 2023-09-23T03:25:43.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.135+0000 7f50ef7fe640 1 -- 172.21.15.154:0/378696606 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f015efe0 con 0x7f50f00cafb0 2023-09-23T03:25:43.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.136+0000 7f50ef7fe640 1 --2- 172.21.15.154:0/378696606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50f00cafb0 0x7f50f00cf690 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f50e80097a0 tx=0x7f50e802b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6dbed59c60b274f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:43.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.136+0000 7f50edffb640 1 -- 172.21.15.154:0/378696606 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e802f050 con 0x7f50f00cafb0 2023-09-23T03:25:43.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.136+0000 7f50edffb640 1 -- 172.21.15.154:0/378696606 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e8031500 con 0x7f50f00cafb0 2023-09-23T03:25:43.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.137+0000 7f50edffb640 1 -- 172.21.15.154:0/378696606 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e00019f0 con 0x7f50f00cafb0 2023-09-23T03:25:43.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.137+0000 7f50f53c7640 1 -- 172.21.15.154:0/378696606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50f00cafb0 msgr2=0x7f50f00cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:43.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.137+0000 7f50f53c7640 1 --2- 172.21.15.154:0/378696606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50f00cafb0 0x7f50f00cf690 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f50e80097a0 tx=0x7f50e802b820 comp rx=0 tx=0).stop 2023-09-23T03:25:43.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.137+0000 7f50f53c7640 1 -- 172.21.15.154:0/378696606 shutdown_connections 2023-09-23T03:25:43.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.137+0000 7f50f53c7640 1 --2- 172.21.15.154:0/378696606 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50f00cafb0 0x7f50f00cf690 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:43.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.137+0000 7f50f53c7640 1 -- 172.21.15.154:0/378696606 >> 172.21.15.154:0/378696606 conn(0x7f50f006fc60 msgr2=0x7f50f0070040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:43.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.137+0000 7f50f53c7640 1 -- 172.21.15.154:0/378696606 shutdown_connections 2023-09-23T03:25:43.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.137+0000 7f50f53c7640 1 -- 172.21.15.154:0/378696606 wait complete. 2023-09-23T03:25:43.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.139+0000 7f50f53c7640 1 Processor -- start 2023-09-23T03:25:43.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.139+0000 7f50f53c7640 1 -- start start 2023-09-23T03:25:43.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.140+0000 7f50f53c7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50f00cafb0 0x7f50f0156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:43.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.140+0000 7f50f53c7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50f00cd870 0x7f50f006be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:43.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.140+0000 7f50f53c7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50f015ad20 0x7f50f006c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:43.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.140+0000 7f50f53c7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f01f3c60 con 0x7f50f00cafb0 2023-09-23T03:25:43.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.140+0000 7f50f53c7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f01f5d90 con 0x7f50f015ad20 2023-09-23T03:25:43.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.140+0000 7f50f53c7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f01566f0 con 0x7f50f00cd870 2023-09-23T03:25:43.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.140+0000 7f50eeffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50f00cafb0 0x7f50f0156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:43.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.141+0000 7f50eeffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50f00cafb0 0x7f50f0156080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50218/0 (socket says 172.21.15.154:50218) 2023-09-23T03:25:43.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.141+0000 7f50eeffd640 1 -- 172.21.15.154:0/360619136 learned_addr learned my addr 172.21.15.154:0/360619136 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:43.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.141+0000 7f50ee7fc640 1 --2- 172.21.15.154:0/360619136 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50f00cd870 0x7f50f006be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:43.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.141+0000 7f50ef7fe640 1 --2- 172.21.15.154:0/360619136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50f015ad20 0x7f50f006c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:43.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.141+0000 7f50eeffd640 1 -- 172.21.15.154:0/360619136 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50f00cd870 msgr2=0x7f50f006be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:43.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.141+0000 7f50eeffd640 1 --2- 172.21.15.154:0/360619136 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50f00cd870 0x7f50f006be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:43.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.141+0000 7f50eeffd640 1 -- 172.21.15.154:0/360619136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50f015ad20 msgr2=0x7f50f006c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:43.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.141+0000 7f50eeffd640 1 --2- 172.21.15.154:0/360619136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50f015ad20 0x7f50f006c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:43.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.141+0000 7f50eeffd640 1 -- 172.21.15.154:0/360619136 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e8009400 con 0x7f50f00cafb0 2023-09-23T03:25:43.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.141+0000 7f50eeffd640 1 --2- 172.21.15.154:0/360619136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50f00cafb0 0x7f50f0156080 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f50e000cc70 tx=0x7f50e000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:43.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.141+0000 7f50e77fe640 1 -- 172.21.15.154:0/360619136 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e000ce80 con 0x7f50f00cafb0 2023-09-23T03:25:43.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.142+0000 7f50e77fe640 1 -- 172.21.15.154:0/360619136 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e0012400 con 0x7f50f00cafb0 2023-09-23T03:25:43.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.142+0000 7f50e77fe640 1 -- 172.21.15.154:0/360619136 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e0012810 con 0x7f50f00cafb0 2023-09-23T03:25:43.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.142+0000 7f50f53c7640 1 -- 172.21.15.154:0/360619136 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50f01569d0 con 0x7f50f00cafb0 2023-09-23T03:25:43.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.142+0000 7f50f53c7640 1 -- 172.21.15.154:0/360619136 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f0156f90 con 0x7f50f00cafb0 2023-09-23T03:25:43.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.143+0000 7f50f53c7640 1 -- 172.21.15.154:0/360619136 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50f0073eb0 con 0x7f50f00cafb0 2023-09-23T03:25:43.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.144+0000 7f50e77fe640 1 -- 172.21.15.154:0/360619136 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f50e0007070 con 0x7f50f00cafb0 2023-09-23T03:25:43.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.145+0000 7f50e77fe640 1 --2- 172.21.15.154:0/360619136 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f50d006af60 0x7f50d006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:43.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.145+0000 7f50e77fe640 1 -- 172.21.15.154:0/360619136 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f50e0089640 con 0x7f50f00cafb0 2023-09-23T03:25:43.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.145+0000 7f50ee7fc640 1 --2- 172.21.15.154:0/360619136 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f50d006af60 0x7f50d006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:43.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.146+0000 7f50ee7fc640 1 --2- 172.21.15.154:0/360619136 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f50d006af60 0x7f50d006d420 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f50f00af090 tx=0x7f50d8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:43.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.149+0000 7f50e77fe640 1 -- 172.21.15.154:0/360619136 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f50e0054920 con 0x7f50f00cafb0 2023-09-23T03:25:43.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.342+0000 7f50f53c7640 1 -- 172.21.15.154:0/360619136 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50f00ce500 con 0x7f50f00cafb0 2023-09-23T03:25:43.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.344+0000 7f50e77fe640 1 -- 172.21.15.154:0/360619136 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+13100 (secure 0 0 0) 0x7f50e0057f40 con 0x7f50f00cafb0 2023-09-23T03:25:43.344 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:25:43.345 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":101,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:25:42.390477+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:25:43.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.346+0000 7f50f53c7640 1 -- 172.21.15.154:0/360619136 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f50d006af60 msgr2=0x7f50d006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:43.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.347+0000 7f50f53c7640 1 --2- 172.21.15.154:0/360619136 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f50d006af60 0x7f50d006d420 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f50f00af090 tx=0x7f50d8009040 comp rx=0 tx=0).stop 2023-09-23T03:25:43.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.347+0000 7f50f53c7640 1 -- 172.21.15.154:0/360619136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50f00cafb0 msgr2=0x7f50f0156080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:43.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.347+0000 7f50f53c7640 1 --2- 172.21.15.154:0/360619136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50f00cafb0 0x7f50f0156080 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f50e000cc70 tx=0x7f50e000a430 comp rx=0 tx=0).stop 2023-09-23T03:25:43.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.347+0000 7f50f53c7640 1 -- 172.21.15.154:0/360619136 shutdown_connections 2023-09-23T03:25:43.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.347+0000 7f50f53c7640 1 --2- 172.21.15.154:0/360619136 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f50d006af60 0x7f50d006d420 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:43.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.347+0000 7f50f53c7640 1 --2- 172.21.15.154:0/360619136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f50f015ad20 0x7f50f006c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:43.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.347+0000 7f50f53c7640 1 --2- 172.21.15.154:0/360619136 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f50f00cd870 0x7f50f006be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:43.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.347+0000 7f50f53c7640 1 --2- 172.21.15.154:0/360619136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f50f00cafb0 0x7f50f0156080 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:43.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.347+0000 7f50f53c7640 1 -- 172.21.15.154:0/360619136 >> 172.21.15.154:0/360619136 conn(0x7f50f006fc60 msgr2=0x7f50f00d0110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:43.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.347+0000 7f50f53c7640 1 -- 172.21.15.154:0/360619136 shutdown_connections 2023-09-23T03:25:43.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.347+0000 7f50f53c7640 1 -- 172.21.15.154:0/360619136 wait complete. 2023-09-23T03:25:43.367 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T03:25:43.368 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:25:43.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.581+0000 7f557ef39640 1 Processor -- start 2023-09-23T03:25:43.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.581+0000 7f557ef39640 1 -- start start 2023-09-23T03:25:43.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.583+0000 7f557ef39640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f557815bb30 0x7f55780caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:43.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.583+0000 7f557ef39640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55780cb3e0 0x7f55780cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:43.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.583+0000 7f557ef39640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f557815b6a0 con 0x7f557815aba0 2023-09-23T03:25:43.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.583+0000 7f557ef39640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55780cb870 con 0x7f557815bb30 2023-09-23T03:25:43.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.583+0000 7f557ef39640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f557815bf70 con 0x7f55780cb3e0 2023-09-23T03:25:43.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.583+0000 7f557ccd5640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f557815aba0 0x7f557815e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46942/0 (socket says 172.21.15.154:46942) 2023-09-23T03:25:43.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.583+0000 7f557ccd5640 1 -- 172.21.15.154:0/1606307788 learned_addr learned my addr 172.21.15.154:0/1606307788 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:25:43.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.583+0000 7f5577fff640 1 --2- 172.21.15.154:0/1606307788 >> v2:172.21.15.154:3300/0 conn(0x7f557815bb30 0x7f55780caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:43.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.584+0000 7f557d4d6640 1 --2- 172.21.15.154:0/1606307788 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55780cb3e0 0x7f55780cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:43.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.584+0000 7f55777fe640 1 -- 172.21.15.154:0/1606307788 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3386473317 0 0) 0x7f557815b6a0 con 0x7f557815aba0 2023-09-23T03:25:43.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.584+0000 7f55777fe640 1 -- 172.21.15.154:0/1606307788 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5568003520 con 0x7f557815aba0 2023-09-23T03:25:43.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.584+0000 7f5577fff640 1 -- 172.21.15.154:0/1606307788 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55780cb3e0 msgr2=0x7f55780cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:43.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.584+0000 7f5577fff640 1 --2- 172.21.15.154:0/1606307788 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55780cb3e0 0x7f55780cc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:43.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.584+0000 7f5577fff640 1 -- 172.21.15.154:0/1606307788 >> v1:172.21.15.154:6789/0 conn(0x7f557815aba0 legacy=0x7f557815e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:43.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.584+0000 7f5577fff640 1 -- 172.21.15.154:0/1606307788 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f557815f0b0 con 0x7f557815bb30 2023-09-23T03:25:43.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.585+0000 7f5577fff640 1 --2- 172.21.15.154:0/1606307788 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f557815bb30 0x7f55780caea0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f55640097a0 tx=0x7f556402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e7369593c475c63 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:43.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.585+0000 7f55777fe640 1 -- 172.21.15.154:0/1606307788 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556402d040 con 0x7f557815bb30 2023-09-23T03:25:43.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.585+0000 7f55777fe640 1 -- 172.21.15.154:0/1606307788 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5564031810 con 0x7f557815bb30 2023-09-23T03:25:43.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.585+0000 7f55777fe640 1 -- 172.21.15.154:0/1606307788 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5564031ba0 con 0x7f557815bb30 2023-09-23T03:25:43.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.586+0000 7f557ef39640 1 -- 172.21.15.154:0/1606307788 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f557815bb30 msgr2=0x7f55780caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:43.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.586+0000 7f557ef39640 1 --2- 172.21.15.154:0/1606307788 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f557815bb30 0x7f55780caea0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f55640097a0 tx=0x7f556402b8b0 comp rx=0 tx=0).stop 2023-09-23T03:25:43.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.586+0000 7f557ef39640 1 -- 172.21.15.154:0/1606307788 shutdown_connections 2023-09-23T03:25:43.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.586+0000 7f557ef39640 1 --2- 172.21.15.154:0/1606307788 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f55780cb3e0 0x7f55780cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:43.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.586+0000 7f557ef39640 1 --2- 172.21.15.154:0/1606307788 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f557815bb30 0x7f55780caea0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:43.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.586+0000 7f557ef39640 1 -- 172.21.15.154:0/1606307788 >> 172.21.15.154:0/1606307788 conn(0x7f557806c010 msgr2=0x7f557806c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:43.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.586+0000 7f557ef39640 1 -- 172.21.15.154:0/1606307788 shutdown_connections 2023-09-23T03:25:43.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.586+0000 7f557ef39640 1 -- 172.21.15.154:0/1606307788 wait complete. 2023-09-23T03:25:43.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.588+0000 7f557ef39640 1 Processor -- start 2023-09-23T03:25:43.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.588+0000 7f557ef39640 1 -- start start 2023-09-23T03:25:43.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.589+0000 7f557ef39640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55780cb3e0 0x7f557814a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:43.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.589+0000 7f557ef39640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f557815aba0 0x7f55780acae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:43.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.589+0000 7f557ef39640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f557815bb30 0x7f55780ad020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:43.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.589+0000 7f557ef39640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f557814c710 con 0x7f55780cb3e0 2023-09-23T03:25:43.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.589+0000 7f557ef39640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5578148e60 con 0x7f557815aba0 2023-09-23T03:25:43.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.589+0000 7f557ef39640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5578149140 con 0x7f557815bb30 2023-09-23T03:25:43.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.589+0000 7f557ccd5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55780cb3e0 0x7f557814a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:43.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.589+0000 7f557ccd5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55780cb3e0 0x7f557814a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50238/0 (socket says 172.21.15.154:50238) 2023-09-23T03:25:43.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.589+0000 7f557ccd5640 1 -- 172.21.15.154:0/777499250 learned_addr learned my addr 172.21.15.154:0/777499250 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:43.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.590+0000 7f557ccd5640 1 -- 172.21.15.154:0/777499250 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f557815bb30 msgr2=0x7f55780ad020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:43.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.590+0000 7f557ccd5640 1 --2- 172.21.15.154:0/777499250 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f557815bb30 0x7f55780ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:43.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.590+0000 7f557ccd5640 1 -- 172.21.15.154:0/777499250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f557815aba0 msgr2=0x7f55780acae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:43.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.590+0000 7f557ccd5640 1 --2- 172.21.15.154:0/777499250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f557815aba0 0x7f55780acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:43.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.590+0000 7f557ccd5640 1 -- 172.21.15.154:0/777499250 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5564009400 con 0x7f55780cb3e0 2023-09-23T03:25:43.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.590+0000 7f557ccd5640 1 --2- 172.21.15.154:0/777499250 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55780cb3e0 0x7f557814a740 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f556c00c500 tx=0x7f556c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:43.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.590+0000 7f55757fa640 1 -- 172.21.15.154:0/777499250 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556c005ce0 con 0x7f55780cb3e0 2023-09-23T03:25:43.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.590+0000 7f55757fa640 1 -- 172.21.15.154:0/777499250 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f556c005e40 con 0x7f55780cb3e0 2023-09-23T03:25:43.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.590+0000 7f55757fa640 1 -- 172.21.15.154:0/777499250 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556c007330 con 0x7f55780cb3e0 2023-09-23T03:25:43.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.590+0000 7f557ef39640 1 -- 172.21.15.154:0/777499250 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5578149420 con 0x7f55780cb3e0 2023-09-23T03:25:43.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.591+0000 7f557ef39640 1 -- 172.21.15.154:0/777499250 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55781499e0 con 0x7f55780cb3e0 2023-09-23T03:25:43.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.592+0000 7f557ef39640 1 -- 172.21.15.154:0/777499250 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5578149bd0 con 0x7f55780cb3e0 2023-09-23T03:25:43.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.593+0000 7f55757fa640 1 -- 172.21.15.154:0/777499250 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f556c007490 con 0x7f55780cb3e0 2023-09-23T03:25:43.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.593+0000 7f55757fa640 1 --2- 172.21.15.154:0/777499250 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f555c06af60 0x7f555c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:43.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.593+0000 7f55757fa640 1 -- 172.21.15.154:0/777499250 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f556c088c10 con 0x7f55780cb3e0 2023-09-23T03:25:43.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.594+0000 7f5577fff640 1 --2- 172.21.15.154:0/777499250 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f555c06af60 0x7f555c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:43.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.594+0000 7f5577fff640 1 --2- 172.21.15.154:0/777499250 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f555c06af60 0x7f555c06d420 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5564036000 tx=0x7f5564030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:43.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.598+0000 7f55757fa640 1 -- 172.21.15.154:0/777499250 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f556c053f10 con 0x7f55780cb3e0 2023-09-23T03:25:43.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.784+0000 7f557ef39640 1 -- 172.21.15.154:0/777499250 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f557804e920 con 0x7f55780cb3e0 2023-09-23T03:25:43.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.785+0000 7f55757fa640 1 -- 172.21.15.154:0/777499250 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) v1 ==== 93+0+10 (secure 0 0 0) 0x7f556c017020 con 0x7f55780cb3e0 2023-09-23T03:25:43.786 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:25:43.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.788+0000 7f557ef39640 1 -- 172.21.15.154:0/777499250 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f555c06af60 msgr2=0x7f555c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:43.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.788+0000 7f557ef39640 1 --2- 172.21.15.154:0/777499250 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f555c06af60 0x7f555c06d420 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5564036000 tx=0x7f5564030000 comp rx=0 tx=0).stop 2023-09-23T03:25:43.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.788+0000 7f557ef39640 1 -- 172.21.15.154:0/777499250 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55780cb3e0 msgr2=0x7f557814a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:43.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.788+0000 7f557ef39640 1 --2- 172.21.15.154:0/777499250 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55780cb3e0 0x7f557814a740 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f556c00c500 tx=0x7f556c00c9c0 comp rx=0 tx=0).stop 2023-09-23T03:25:43.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.788+0000 7f557ef39640 1 -- 172.21.15.154:0/777499250 shutdown_connections 2023-09-23T03:25:43.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.788+0000 7f557ef39640 1 --2- 172.21.15.154:0/777499250 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f555c06af60 0x7f555c06d420 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:43.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.788+0000 7f557ef39640 1 --2- 172.21.15.154:0/777499250 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f557815bb30 0x7f55780ad020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:43.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.788+0000 7f557ef39640 1 --2- 172.21.15.154:0/777499250 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f557815aba0 0x7f55780acae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:43.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.788+0000 7f557ef39640 1 --2- 172.21.15.154:0/777499250 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f55780cb3e0 0x7f557814a740 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:43.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.788+0000 7f557ef39640 1 -- 172.21.15.154:0/777499250 >> 172.21.15.154:0/777499250 conn(0x7f557806c010 msgr2=0x7f557815e360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:43.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.788+0000 7f557ef39640 1 -- 172.21.15.154:0/777499250 shutdown_connections 2023-09-23T03:25:43.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:43.788+0000 7f557ef39640 1 -- 172.21.15.154:0/777499250 wait complete. 2023-09-23T03:25:43.805 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-09-23T03:25:43.891 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:25:43.897 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:25:44.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.011+0000 7f25ea3dc640 1 Processor -- start 2023-09-23T03:25:44.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.012+0000 7f25ea3dc640 1 -- start start 2023-09-23T03:25:44.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.013+0000 7f25ea3dc640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f25e40acce0 0x7f25e415ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:44.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.013+0000 7f25ea3dc640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f25e40b1b20 0x7f25e40ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:44.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.013+0000 7f25ea3dc640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25e415ac30 con 0x7f25e40cee70 2023-09-23T03:25:44.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.013+0000 7f25ea3dc640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25e40b2e90 con 0x7f25e40b1b20 2023-09-23T03:25:44.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.013+0000 7f25ea3dc640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25e40cf300 con 0x7f25e40acce0 2023-09-23T03:25:44.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.013+0000 7f25e37fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f25e40b1b20 0x7f25e40ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:44.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.013+0000 7f25e37fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f25e40b1b20 0x7f25e40ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48452/0 (socket says 172.21.15.154:48452) 2023-09-23T03:25:44.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.013+0000 7f25e37fe640 1 -- 172.21.15.154:0/4181686722 learned_addr learned my addr 172.21.15.154:0/4181686722 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:44.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.014+0000 7f25e3fff640 1 --2- 172.21.15.154:0/4181686722 >> v2:172.21.15.190:3300/0 conn(0x7f25e40acce0 0x7f25e415ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:44.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.014+0000 7f25e37fe640 1 -- 172.21.15.154:0/4181686722 >> v2:172.21.15.190:3300/0 conn(0x7f25e40acce0 msgr2=0x7f25e415ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:44.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.014+0000 7f25e37fe640 1 --2- 172.21.15.154:0/4181686722 >> v2:172.21.15.190:3300/0 conn(0x7f25e40acce0 0x7f25e415ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:44.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.014+0000 7f25e37fe640 1 -- 172.21.15.154:0/4181686722 >> v1:172.21.15.190:6789/0 conn(0x7f25e40cee70 legacy=0x7f25e406cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:44.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.014+0000 7f25e37fe640 1 -- 172.21.15.154:0/4181686722 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e40b1f70 con 0x7f25e40b1b20 2023-09-23T03:25:44.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.015+0000 7f25e37fe640 1 --2- 172.21.15.154:0/4181686722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25e40b1b20 0x7f25e40ce930 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f25e40cf570 tx=0x7f25d802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d5611fee61806a3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:44.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.015+0000 7f25e2ffd640 1 -- 172.21.15.154:0/4181686722 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d8031870 con 0x7f25e40b1b20 2023-09-23T03:25:44.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.016+0000 7f25e2ffd640 1 -- 172.21.15.154:0/4181686722 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d80319d0 con 0x7f25e40b1b20 2023-09-23T03:25:44.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.016+0000 7f25e2ffd640 1 -- 172.21.15.154:0/4181686722 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d8031d60 con 0x7f25e40b1b20 2023-09-23T03:25:44.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.016+0000 7f25ea3dc640 1 -- 172.21.15.154:0/4181686722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25e40b1b20 msgr2=0x7f25e40ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:44.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.016+0000 7f25ea3dc640 1 --2- 172.21.15.154:0/4181686722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25e40b1b20 0x7f25e40ce930 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f25e40cf570 tx=0x7f25d802b910 comp rx=0 tx=0).stop 2023-09-23T03:25:44.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.016+0000 7f25ea3dc640 1 -- 172.21.15.154:0/4181686722 shutdown_connections 2023-09-23T03:25:44.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.016+0000 7f25ea3dc640 1 --2- 172.21.15.154:0/4181686722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25e40b1b20 0x7f25e40ce930 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:44.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.016+0000 7f25ea3dc640 1 --2- 172.21.15.154:0/4181686722 >> v2:172.21.15.190:3300/0 conn(0x7f25e40acce0 0x7f25e415ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:44.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.016+0000 7f25ea3dc640 1 -- 172.21.15.154:0/4181686722 >> 172.21.15.154:0/4181686722 conn(0x7f25e406a0f0 msgr2=0x7f25e406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:44.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.016+0000 7f25ea3dc640 1 -- 172.21.15.154:0/4181686722 shutdown_connections 2023-09-23T03:25:44.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.016+0000 7f25ea3dc640 1 -- 172.21.15.154:0/4181686722 wait complete. 2023-09-23T03:25:44.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.018+0000 7f25ea3dc640 1 Processor -- start 2023-09-23T03:25:44.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.018+0000 7f25ea3dc640 1 -- start start 2023-09-23T03:25:44.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.019+0000 7f25ea3dc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25e40acce0 0x7f25e41f65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:44.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.019+0000 7f25ea3dc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25e40b1b20 0x7f25e41ef910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:44.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.020+0000 7f25ea3dc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25e40cee70 0x7f25e40cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:44.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.020+0000 7f25ea3dc640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e41f25a0 con 0x7f25e40acce0 2023-09-23T03:25:44.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.020+0000 7f25ea3dc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e41f6ae0 con 0x7f25e40b1b20 2023-09-23T03:25:44.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.020+0000 7f25ea3dc640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e40cd490 con 0x7f25e40cee70 2023-09-23T03:25:44.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.020+0000 7f25e3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25e40acce0 0x7f25e41f65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:44.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.020+0000 7f25e3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25e40acce0 0x7f25e41f65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48460/0 (socket says 172.21.15.154:48460) 2023-09-23T03:25:44.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.020+0000 7f25e3fff640 1 -- 172.21.15.154:0/3401278655 learned_addr learned my addr 172.21.15.154:0/3401278655 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:44.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.020+0000 7f25e8979640 1 --2- 172.21.15.154:0/3401278655 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25e40cee70 0x7f25e40cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:44.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.020+0000 7f25e37fe640 1 --2- 172.21.15.154:0/3401278655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25e40b1b20 0x7f25e41ef910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:44.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.020+0000 7f25e3fff640 1 -- 172.21.15.154:0/3401278655 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25e40cee70 msgr2=0x7f25e40cadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:44.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.020+0000 7f25e3fff640 1 --2- 172.21.15.154:0/3401278655 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25e40cee70 0x7f25e40cadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:44.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.020+0000 7f25e3fff640 1 -- 172.21.15.154:0/3401278655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25e40b1b20 msgr2=0x7f25e41ef910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:44.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.020+0000 7f25e3fff640 1 --2- 172.21.15.154:0/3401278655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25e40b1b20 0x7f25e41ef910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:44.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.020+0000 7f25e3fff640 1 -- 172.21.15.154:0/3401278655 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d80094d0 con 0x7f25e40acce0 2023-09-23T03:25:44.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.020+0000 7f25e8979640 1 --2- 172.21.15.154:0/3401278655 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25e40cee70 0x7f25e40cadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:44.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.021+0000 7f25e3fff640 1 --2- 172.21.15.154:0/3401278655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25e40acce0 0x7f25e41f65a0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f25d400d830 tx=0x7f25d400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:44.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.021+0000 7f25e0ff9640 1 -- 172.21.15.154:0/3401278655 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d4018070 con 0x7f25e40acce0 2023-09-23T03:25:44.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.021+0000 7f25e0ff9640 1 -- 172.21.15.154:0/3401278655 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d4005990 con 0x7f25e40acce0 2023-09-23T03:25:44.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.021+0000 7f25e0ff9640 1 -- 172.21.15.154:0/3401278655 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d4005b40 con 0x7f25e40acce0 2023-09-23T03:25:44.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.021+0000 7f25ea3dc640 1 -- 172.21.15.154:0/3401278655 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e41f27f0 con 0x7f25e40acce0 2023-09-23T03:25:44.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.021+0000 7f25ea3dc640 1 -- 172.21.15.154:0/3401278655 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e41f2db0 con 0x7f25e40acce0 2023-09-23T03:25:44.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.022+0000 7f25ea3dc640 1 -- 172.21.15.154:0/3401278655 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25e4073eb0 con 0x7f25e40acce0 2023-09-23T03:25:44.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.023+0000 7f25e0ff9640 1 -- 172.21.15.154:0/3401278655 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f25d400b6c0 con 0x7f25e40acce0 2023-09-23T03:25:44.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.024+0000 7f25e0ff9640 1 --2- 172.21.15.154:0/3401278655 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f25d006af30 0x7f25d006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:44.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.024+0000 7f25e0ff9640 1 -- 172.21.15.154:0/3401278655 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f25d4089670 con 0x7f25e40acce0 2023-09-23T03:25:44.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.024+0000 7f25e37fe640 1 --2- 172.21.15.154:0/3401278655 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f25d006af30 0x7f25d006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:44.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.025+0000 7f25e37fe640 1 --2- 172.21.15.154:0/3401278655 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f25d006af30 0x7f25d006d3f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f25d8036000 tx=0x7f25d803c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:44.029 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.028+0000 7f25e0ff9640 1 -- 172.21.15.154:0/3401278655 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f25d40548b0 con 0x7f25e40acce0 2023-09-23T03:25:44.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.220+0000 7f25ea3dc640 1 -- 172.21.15.154:0/3401278655 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f25e406c930 con 0x7f25e40acce0 2023-09-23T03:25:44.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.221+0000 7f25e0ff9640 1 -- 172.21.15.154:0/3401278655 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v101) v1 ==== 84+0+65 (secure 0 0 0) 0x7f25d4057ed0 con 0x7f25e40acce0 2023-09-23T03:25:44.221 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:25:44.222 INFO:teuthology.orchestra.run.smithi154.stdout:[ 2023-09-23T03:25:44.222 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:25:44.222 INFO:teuthology.orchestra.run.smithi154.stdout: "poolnum": 1, 2023-09-23T03:25:44.222 INFO:teuthology.orchestra.run.smithi154.stdout: "poolname": ".mgr" 2023-09-23T03:25:44.222 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:25:44.222 INFO:teuthology.orchestra.run.smithi154.stdout:] 2023-09-23T03:25:44.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.223+0000 7f25ea3dc640 1 -- 172.21.15.154:0/3401278655 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f25d006af30 msgr2=0x7f25d006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:44.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.223+0000 7f25ea3dc640 1 --2- 172.21.15.154:0/3401278655 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f25d006af30 0x7f25d006d3f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f25d8036000 tx=0x7f25d803c040 comp rx=0 tx=0).stop 2023-09-23T03:25:44.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.223+0000 7f25ea3dc640 1 -- 172.21.15.154:0/3401278655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25e40acce0 msgr2=0x7f25e41f65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:44.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.223+0000 7f25ea3dc640 1 --2- 172.21.15.154:0/3401278655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25e40acce0 0x7f25e41f65a0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f25d400d830 tx=0x7f25d400dcf0 comp rx=0 tx=0).stop 2023-09-23T03:25:44.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.223+0000 7f25ea3dc640 1 -- 172.21.15.154:0/3401278655 shutdown_connections 2023-09-23T03:25:44.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.223+0000 7f25ea3dc640 1 --2- 172.21.15.154:0/3401278655 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f25d006af30 0x7f25d006d3f0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:44.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.223+0000 7f25ea3dc640 1 --2- 172.21.15.154:0/3401278655 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f25e40cee70 0x7f25e40cadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:44.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.223+0000 7f25ea3dc640 1 --2- 172.21.15.154:0/3401278655 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f25e40b1b20 0x7f25e41ef910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:44.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.223+0000 7f25ea3dc640 1 --2- 172.21.15.154:0/3401278655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f25e40acce0 0x7f25e41f65a0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:44.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.223+0000 7f25ea3dc640 1 -- 172.21.15.154:0/3401278655 >> 172.21.15.154:0/3401278655 conn(0x7f25e406a0f0 msgr2=0x7f25e40c4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:44.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.224+0000 7f25ea3dc640 1 -- 172.21.15.154:0/3401278655 shutdown_connections 2023-09-23T03:25:44.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.224+0000 7f25ea3dc640 1 -- 172.21.15.154:0/3401278655 wait complete. 2023-09-23T03:25:44.241 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-09-23T03:25:44.241 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-09-23T03:25:44.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.507+0000 7f0ee30b4640 1 Processor -- start 2023-09-23T03:25:44.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.508+0000 7f0ee30b4640 1 -- start start 2023-09-23T03:25:44.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.509+0000 7f0ee30b4640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0edc160090 0x7f0edc0cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:44.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.509+0000 7f0ee30b4640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0edc1606b0 con 0x7f0edc06c890 2023-09-23T03:25:44.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.509+0000 7f0ee30b4640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0edc161550 con 0x7f0edc070710 2023-09-23T03:25:44.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.509+0000 7f0ee30b4640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0edc070eb0 con 0x7f0edc160090 2023-09-23T03:25:44.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.510+0000 7f0ee1651640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f0edc06c890 0x7f0edc0c9360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:49536/0 (socket says 172.21.15.154:49536) 2023-09-23T03:25:44.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.510+0000 7f0ee1651640 1 -- 172.21.15.154:0/2741741736 learned_addr learned my addr 172.21.15.154:0/2741741736 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:25:44.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.510+0000 7f0ee0e50640 1 --2- 172.21.15.154:0/2741741736 >> v2:172.21.15.190:3300/0 conn(0x7f0edc160090 0x7f0edc0cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:44.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.510+0000 7f0edb7fe640 1 -- 172.21.15.154:0/2741741736 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2875621198 0 0) 0x7f0edc1606b0 con 0x7f0edc06c890 2023-09-23T03:25:44.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.510+0000 7f0edb7fe640 1 -- 172.21.15.154:0/2741741736 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ecc003520 con 0x7f0edc06c890 2023-09-23T03:25:44.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.511+0000 7f0edb7fe640 1 -- 172.21.15.154:0/2741741736 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2106832182 0 0) 0x7f0ecc003520 con 0x7f0edc06c890 2023-09-23T03:25:44.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.511+0000 7f0edb7fe640 1 -- 172.21.15.154:0/2741741736 >> v2:172.21.15.190:3300/0 conn(0x7f0edc160090 msgr2=0x7f0edc0cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:44.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.511+0000 7f0edb7fe640 1 --2- 172.21.15.154:0/2741741736 >> v2:172.21.15.190:3300/0 conn(0x7f0edc160090 0x7f0edc0cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:44.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.511+0000 7f0edb7fe640 1 -- 172.21.15.154:0/2741741736 >> v1:172.21.15.190:6789/0 conn(0x7f0edc070710 legacy=0x7f0edc160e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:44.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.511+0000 7f0edb7fe640 1 -- 172.21.15.154:0/2741741736 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0edc071190 con 0x7f0edc06c890 2023-09-23T03:25:44.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.511+0000 7f0edb7fe640 1 -- 172.21.15.154:0/2741741736 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f0ed40019f0 con 0x7f0edc06c890 2023-09-23T03:25:44.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.511+0000 7f0edb7fe640 1 -- 172.21.15.154:0/2741741736 >> v1:172.21.15.154:6789/0 conn(0x7f0edc06c890 legacy=0x7f0edc0c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:44.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.511+0000 7f0edb7fe640 1 --2- 172.21.15.154:0/2741741736 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ecc0039c0 0x7f0ecc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:44.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.511+0000 7f0edb7fe640 1 --2- 172.21.15.154:0/2741741736 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ecc0047c0 0x7f0ecc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:44.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.511+0000 7f0edb7fe640 1 --2- 172.21.15.154:0/2741741736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ecc0051d0 0x7f0ecc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:44.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.511+0000 7f0edb7fe640 1 -- 172.21.15.154:0/2741741736 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc070eb0 con 0x7f0ecc0039c0 2023-09-23T03:25:44.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.511+0000 7f0edb7fe640 1 -- 172.21.15.154:0/2741741736 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc00c120 con 0x7f0ecc0051d0 2023-09-23T03:25:44.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.511+0000 7f0edb7fe640 1 -- 172.21.15.154:0/2741741736 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc00c2e0 con 0x7f0ecc0047c0 2023-09-23T03:25:44.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.511+0000 7f0ee0e50640 1 --2- 172.21.15.154:0/2741741736 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ecc0039c0 0x7f0ecc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:44.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.511+0000 7f0edbfff640 1 --2- 172.21.15.154:0/2741741736 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ecc0047c0 0x7f0ecc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:44.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.511+0000 7f0ee1651640 1 --2- 172.21.15.154:0/2741741736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ecc0051d0 0x7f0ecc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:44.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.511+0000 7f0ee0e50640 1 -- 172.21.15.154:0/2741741736 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ecc0047c0 msgr2=0x7f0ecc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:44.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.511+0000 7f0ee0e50640 1 --2- 172.21.15.154:0/2741741736 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ecc0047c0 0x7f0ecc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:44.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.511+0000 7f0ee0e50640 1 -- 172.21.15.154:0/2741741736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ecc0051d0 msgr2=0x7f0ecc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:44.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.511+0000 7f0ee0e50640 1 --2- 172.21.15.154:0/2741741736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ecc0051d0 0x7f0ecc00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:44.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.511+0000 7f0ee0e50640 1 -- 172.21.15.154:0/2741741736 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ecc003520 con 0x7f0ecc0039c0 2023-09-23T03:25:44.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.511+0000 7f0ee1651640 1 --2- 172.21.15.154:0/2741741736 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0ecc0051d0 0x7f0ecc00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:44.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.511+0000 7f0ee1651640 1 -- 172.21.15.154:0/2741741736 reap_dead start 2023-09-23T03:25:44.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.511+0000 7f0edbfff640 1 --2- 172.21.15.154:0/2741741736 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0ecc0047c0 0x7f0ecc004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:44.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.513+0000 7f0ee0e50640 1 --2- 172.21.15.154:0/2741741736 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ecc0039c0 0x7f0ecc003de0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0009a90 tx=0x7f0ecc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=977dd5092e67622 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:44.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.513+0000 7f0edb7fe640 1 -- 172.21.15.154:0/2741741736 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed002c680 con 0x7f0ecc0039c0 2023-09-23T03:25:44.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.513+0000 7f0edb7fe640 1 -- 172.21.15.154:0/2741741736 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed002c7e0 con 0x7f0ecc0039c0 2023-09-23T03:25:44.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.513+0000 7f0edb7fe640 1 -- 172.21.15.154:0/2741741736 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed002cb70 con 0x7f0ecc0039c0 2023-09-23T03:25:44.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.513+0000 7f0ee30b4640 1 -- 172.21.15.154:0/2741741736 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ecc0039c0 msgr2=0x7f0ecc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:44.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.513+0000 7f0ee30b4640 1 --2- 172.21.15.154:0/2741741736 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ecc0039c0 0x7f0ecc003de0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0009a90 tx=0x7f0ecc0057f0 comp rx=0 tx=0).stop 2023-09-23T03:25:44.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.514+0000 7f0ee30b4640 1 -- 172.21.15.154:0/2741741736 shutdown_connections 2023-09-23T03:25:44.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.514+0000 7f0ee30b4640 1 --2- 172.21.15.154:0/2741741736 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ecc0039c0 0x7f0ecc003de0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:44.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.514+0000 7f0ee30b4640 1 -- 172.21.15.154:0/2741741736 >> 172.21.15.154:0/2741741736 conn(0x7f0edc0ced10 msgr2=0x7f0edc0cf120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:44.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.514+0000 7f0ee30b4640 1 -- 172.21.15.154:0/2741741736 shutdown_connections 2023-09-23T03:25:44.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.514+0000 7f0ee30b4640 1 -- 172.21.15.154:0/2741741736 wait complete. 2023-09-23T03:25:44.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.515+0000 7f0ee30b4640 1 Processor -- start 2023-09-23T03:25:44.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.516+0000 7f0ee30b4640 1 -- start start 2023-09-23T03:25:44.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.516+0000 7f0ee30b4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ecc0039c0 0x7f0edc0ac970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:44.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.517+0000 7f0ee30b4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0edc14bd50 0x7f0edc14c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:44.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.517+0000 7f0ee30b4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0edc0c4aa0 0x7f0edc0c4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:44.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.517+0000 7f0ee30b4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc1f6310 con 0x7f0ecc0039c0 2023-09-23T03:25:44.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.517+0000 7f0ee30b4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc14c710 con 0x7f0edc14bd50 2023-09-23T03:25:44.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.517+0000 7f0ee30b4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc1f5690 con 0x7f0edc0c4aa0 2023-09-23T03:25:44.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.517+0000 7f0ee0e50640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ecc0039c0 0x7f0edc0ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:44.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.517+0000 7f0ee0e50640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ecc0039c0 0x7f0edc0ac970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48482/0 (socket says 172.21.15.154:48482) 2023-09-23T03:25:44.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.517+0000 7f0ee0e50640 1 -- 172.21.15.154:0/488284580 learned_addr learned my addr 172.21.15.154:0/488284580 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:44.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.517+0000 7f0ee0e50640 1 -- 172.21.15.154:0/488284580 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0edc0c4aa0 msgr2=0x7f0edc0c4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:44.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.517+0000 7f0ee0e50640 1 --2- 172.21.15.154:0/488284580 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0edc0c4aa0 0x7f0edc0c4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:44.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.517+0000 7f0ee0e50640 1 -- 172.21.15.154:0/488284580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0edc14bd50 msgr2=0x7f0edc14c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:44.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.517+0000 7f0ee0e50640 1 --2- 172.21.15.154:0/488284580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0edc14bd50 0x7f0edc14c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:44.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.517+0000 7f0ee0e50640 1 -- 172.21.15.154:0/488284580 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0edc1f5970 con 0x7f0ecc0039c0 2023-09-23T03:25:44.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.518+0000 7f0ee0e50640 1 --2- 172.21.15.154:0/488284580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ecc0039c0 0x7f0edc0ac970 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0009a90 tx=0x7f0ed0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:44.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.518+0000 7f0ed97fa640 1 -- 172.21.15.154:0/488284580 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0003810 con 0x7f0ecc0039c0 2023-09-23T03:25:44.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.518+0000 7f0ed97fa640 1 -- 172.21.15.154:0/488284580 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed0003970 con 0x7f0ecc0039c0 2023-09-23T03:25:44.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.518+0000 7f0ed97fa640 1 -- 172.21.15.154:0/488284580 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0003d00 con 0x7f0ecc0039c0 2023-09-23T03:25:44.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.518+0000 7f0ee30b4640 1 -- 172.21.15.154:0/488284580 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0edc1f5c50 con 0x7f0ecc0039c0 2023-09-23T03:25:44.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.518+0000 7f0ee30b4640 1 -- 172.21.15.154:0/488284580 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0edc0c5440 con 0x7f0ecc0039c0 2023-09-23T03:25:44.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.519+0000 7f0ee30b4640 1 -- 172.21.15.154:0/488284580 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0edc04e920 con 0x7f0ecc0039c0 2023-09-23T03:25:44.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.520+0000 7f0ed97fa640 1 -- 172.21.15.154:0/488284580 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f0ed0040450 con 0x7f0ecc0039c0 2023-09-23T03:25:44.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.521+0000 7f0ed97fa640 1 --2- 172.21.15.154:0/488284580 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0ec006af30 0x7f0ec006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:44.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.521+0000 7f0ed97fa640 1 -- 172.21.15.154:0/488284580 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f0ed00ab410 con 0x7f0ecc0039c0 2023-09-23T03:25:44.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.521+0000 7f0edbfff640 1 --2- 172.21.15.154:0/488284580 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0ec006af30 0x7f0ec006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:44.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.522+0000 7f0edbfff640 1 --2- 172.21.15.154:0/488284580 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0ec006af30 0x7f0ec006d3f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f0edc0c84a0 tx=0x7f0ec8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:44.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.525+0000 7f0ed97fa640 1 -- 172.21.15.154:0/488284580 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f0ed0076650 con 0x7f0ecc0039c0 2023-09-23T03:25:44.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:44.714+0000 7f0ee30b4640 1 -- 172.21.15.154:0/488284580 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f0edc06b110 con 0x7f0ecc0039c0 2023-09-23T03:25:45.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.516+0000 7f0ed97fa640 1 -- 172.21.15.154:0/488284580 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v102) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0ed0003e60 con 0x7f0ecc0039c0 2023-09-23T03:25:45.516 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_metadata' created 2023-09-23T03:25:45.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.519+0000 7f0ee30b4640 1 -- 172.21.15.154:0/488284580 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0ec006af30 msgr2=0x7f0ec006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:45.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.519+0000 7f0ee30b4640 1 --2- 172.21.15.154:0/488284580 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0ec006af30 0x7f0ec006d3f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f0edc0c84a0 tx=0x7f0ec8009040 comp rx=0 tx=0).stop 2023-09-23T03:25:45.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.519+0000 7f0ee30b4640 1 -- 172.21.15.154:0/488284580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ecc0039c0 msgr2=0x7f0edc0ac970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:45.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.519+0000 7f0ee30b4640 1 --2- 172.21.15.154:0/488284580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ecc0039c0 0x7f0edc0ac970 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0009a90 tx=0x7f0ed0004730 comp rx=0 tx=0).stop 2023-09-23T03:25:45.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.519+0000 7f0ee30b4640 1 -- 172.21.15.154:0/488284580 shutdown_connections 2023-09-23T03:25:45.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.519+0000 7f0ee30b4640 1 --2- 172.21.15.154:0/488284580 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0ec006af30 0x7f0ec006d3f0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:45.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.519+0000 7f0ee30b4640 1 --2- 172.21.15.154:0/488284580 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0edc0c4aa0 0x7f0edc0c4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:45.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.519+0000 7f0ee30b4640 1 --2- 172.21.15.154:0/488284580 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0edc14bd50 0x7f0edc14c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:45.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.519+0000 7f0ee30b4640 1 --2- 172.21.15.154:0/488284580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ecc0039c0 0x7f0edc0ac970 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:45.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.519+0000 7f0ee30b4640 1 -- 172.21.15.154:0/488284580 >> 172.21.15.154:0/488284580 conn(0x7f0edc0ced10 msgr2=0x7f0edc0ad960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:45.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.520+0000 7f0ee30b4640 1 -- 172.21.15.154:0/488284580 shutdown_connections 2023-09-23T03:25:45.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.520+0000 7f0ee30b4640 1 -- 172.21.15.154:0/488284580 wait complete. 2023-09-23T03:25:45.546 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-09-23T03:25:45.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.761+0000 7f97c93f5640 1 Processor -- start 2023-09-23T03:25:45.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.761+0000 7f97c93f5640 1 -- start start 2023-09-23T03:25:45.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.763+0000 7f97c93f5640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f97c415aba0 0x7f97c415bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:45.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.763+0000 7f97c93f5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97c415e5f0 0x7f97c415ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:45.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.763+0000 7f97c93f5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97c415b6a0 con 0x7f97c40caea0 2023-09-23T03:25:45.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.763+0000 7f97c93f5640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97c415f0b0 con 0x7f97c415aba0 2023-09-23T03:25:45.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.763+0000 7f97c93f5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c415f390 con 0x7f97c415e5f0 2023-09-23T03:25:45.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.763+0000 7f97c37fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f97c40caea0 0x7f97c40cc5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:49540/0 (socket says 172.21.15.154:49540) 2023-09-23T03:25:45.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.763+0000 7f97c37fe640 1 -- 172.21.15.154:0/4185061052 learned_addr learned my addr 172.21.15.154:0/4185061052 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:25:45.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.764+0000 7f97c2ffd640 1 --2- 172.21.15.154:0/4185061052 >> v2:172.21.15.190:3300/0 conn(0x7f97c415aba0 0x7f97c415bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:45.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.764+0000 7f97c27fc640 1 --2- 172.21.15.154:0/4185061052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97c415e5f0 0x7f97c415ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:45.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.764+0000 7f97c1ffb640 1 -- 172.21.15.154:0/4185061052 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3232232924 0 0) 0x7f97c415b6a0 con 0x7f97c40caea0 2023-09-23T03:25:45.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.764+0000 7f97c1ffb640 1 -- 172.21.15.154:0/4185061052 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97ac003520 con 0x7f97c40caea0 2023-09-23T03:25:45.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.764+0000 7f97c1ffb640 1 -- 172.21.15.154:0/4185061052 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2229391722 0 0) 0x7f97ac003520 con 0x7f97c40caea0 2023-09-23T03:25:45.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.764+0000 7f97c1ffb640 1 -- 172.21.15.154:0/4185061052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97c415e5f0 msgr2=0x7f97c415ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:45.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.764+0000 7f97c1ffb640 1 --2- 172.21.15.154:0/4185061052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97c415e5f0 0x7f97c415ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:45.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.764+0000 7f97c1ffb640 1 -- 172.21.15.154:0/4185061052 >> v2:172.21.15.190:3300/0 conn(0x7f97c415aba0 msgr2=0x7f97c415bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:45.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.764+0000 7f97c1ffb640 1 --2- 172.21.15.154:0/4185061052 >> v2:172.21.15.190:3300/0 conn(0x7f97c415aba0 0x7f97c415bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:45.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.764+0000 7f97c1ffb640 1 -- 172.21.15.154:0/4185061052 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c415b330 con 0x7f97c40caea0 2023-09-23T03:25:45.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.764+0000 7f97c1ffb640 1 -- 172.21.15.154:0/4185061052 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f97bc0019f0 con 0x7f97c40caea0 2023-09-23T03:25:45.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.764+0000 7f97c1ffb640 1 -- 172.21.15.154:0/4185061052 >> v1:172.21.15.154:6789/0 conn(0x7f97c40caea0 legacy=0x7f97c40cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:45.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.764+0000 7f97c1ffb640 1 --2- 172.21.15.154:0/4185061052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97ac003970 0x7f97ac003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:45.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.764+0000 7f97c1ffb640 1 --2- 172.21.15.154:0/4185061052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97c415e5f0 0x7f97ac0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:45.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.764+0000 7f97c1ffb640 1 --2- 172.21.15.154:0/4185061052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97ac004d00 0x7f97ac005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:45.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.765+0000 7f97c1ffb640 1 -- 172.21.15.154:0/4185061052 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c415f0b0 con 0x7f97ac003970 2023-09-23T03:25:45.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.765+0000 7f97c1ffb640 1 -- 172.21.15.154:0/4185061052 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c415f390 con 0x7f97ac004d00 2023-09-23T03:25:45.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.765+0000 7f97c1ffb640 1 -- 172.21.15.154:0/4185061052 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac00b980 con 0x7f97c415e5f0 2023-09-23T03:25:45.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.765+0000 7f97c2ffd640 1 --2- 172.21.15.154:0/4185061052 >> v2:172.21.15.190:3300/0 conn(0x7f97c415aba0 0x7f97c415bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:25:45.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.765+0000 7f97c2ffd640 1 --2- 172.21.15.154:0/4185061052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97ac003970 0x7f97ac003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:45.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.765+0000 7f97c27fc640 1 --2- 172.21.15.154:0/4185061052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97c415e5f0 0x7f97ac0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:45.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.765+0000 7f97c37fe640 1 --2- 172.21.15.154:0/4185061052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97ac004d00 0x7f97ac005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:45.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.765+0000 7f97c2ffd640 1 -- 172.21.15.154:0/4185061052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97c415e5f0 msgr2=0x7f97ac0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:45.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.765+0000 7f97c2ffd640 1 --2- 172.21.15.154:0/4185061052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97c415e5f0 0x7f97ac0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:45.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.765+0000 7f97c2ffd640 1 -- 172.21.15.154:0/4185061052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97ac004d00 msgr2=0x7f97ac005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:45.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.765+0000 7f97c2ffd640 1 --2- 172.21.15.154:0/4185061052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97ac004d00 0x7f97ac005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:45.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.765+0000 7f97c2ffd640 1 -- 172.21.15.154:0/4185061052 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ac003520 con 0x7f97ac003970 2023-09-23T03:25:45.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.765+0000 7f97c27fc640 1 --2- 172.21.15.154:0/4185061052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97c415e5f0 0x7f97ac0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:45.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.766+0000 7f97c2ffd640 1 --2- 172.21.15.154:0/4185061052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97ac003970 0x7f97ac003d90 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f97b4009c70 tx=0x7f97ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f034e323ff88dcef server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:45.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.767+0000 7f97c1ffb640 1 -- 172.21.15.154:0/4185061052 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b4004020 con 0x7f97ac003970 2023-09-23T03:25:45.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.767+0000 7f97c1ffb640 1 -- 172.21.15.154:0/4185061052 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b4004180 con 0x7f97ac003970 2023-09-23T03:25:45.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.767+0000 7f97c1ffb640 1 -- 172.21.15.154:0/4185061052 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b402c980 con 0x7f97ac003970 2023-09-23T03:25:45.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.767+0000 7f97c93f5640 1 -- 172.21.15.154:0/4185061052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97ac003970 msgr2=0x7f97ac003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:45.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.767+0000 7f97c93f5640 1 --2- 172.21.15.154:0/4185061052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97ac003970 0x7f97ac003d90 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f97b4009c70 tx=0x7f97ac0057f0 comp rx=0 tx=0).stop 2023-09-23T03:25:45.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.767+0000 7f97c37fe640 1 -- 172.21.15.154:0/4185061052 reap_dead start 2023-09-23T03:25:45.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.767+0000 7f97c93f5640 1 -- 172.21.15.154:0/4185061052 shutdown_connections 2023-09-23T03:25:45.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.767+0000 7f97c93f5640 1 -- 172.21.15.154:0/4185061052 >> 172.21.15.154:0/4185061052 conn(0x7f97c406c010 msgr2=0x7f97c406c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:45.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.767+0000 7f97c93f5640 1 -- 172.21.15.154:0/4185061052 shutdown_connections 2023-09-23T03:25:45.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.767+0000 7f97c93f5640 1 -- 172.21.15.154:0/4185061052 wait complete. 2023-09-23T03:25:45.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.769+0000 7f97c93f5640 1 Processor -- start 2023-09-23T03:25:45.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.769+0000 7f97c93f5640 1 -- start start 2023-09-23T03:25:45.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.770+0000 7f97c93f5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97ac003970 0x7f97c4158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:45.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.770+0000 7f97c93f5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97c415d3e0 0x7f97c40c9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:45.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.770+0000 7f97c93f5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97c415d840 0x7f97c40c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:45.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.770+0000 7f97c93f5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c41587f0 con 0x7f97ac003970 2023-09-23T03:25:45.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.770+0000 7f97c93f5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c40ca2e0 con 0x7f97c415d840 2023-09-23T03:25:45.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.770+0000 7f97c93f5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c4157b70 con 0x7f97c415d3e0 2023-09-23T03:25:45.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.771+0000 7f97c2ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97ac003970 0x7f97c4158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:45.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.771+0000 7f97c2ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97ac003970 0x7f97c4158c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48498/0 (socket says 172.21.15.154:48498) 2023-09-23T03:25:45.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.771+0000 7f97c2ffd640 1 -- 172.21.15.154:0/3392859720 learned_addr learned my addr 172.21.15.154:0/3392859720 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:45.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.771+0000 7f97c37fe640 1 --2- 172.21.15.154:0/3392859720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97c415d840 0x7f97c40c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:45.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.771+0000 7f97c27fc640 1 --2- 172.21.15.154:0/3392859720 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97c415d3e0 0x7f97c40c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:45.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.771+0000 7f97c2ffd640 1 -- 172.21.15.154:0/3392859720 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97c415d3e0 msgr2=0x7f97c40c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:45.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.771+0000 7f97c2ffd640 1 --2- 172.21.15.154:0/3392859720 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97c415d3e0 0x7f97c40c9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:45.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.771+0000 7f97c2ffd640 1 -- 172.21.15.154:0/3392859720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97c415d840 msgr2=0x7f97c40c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:45.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.771+0000 7f97c2ffd640 1 --2- 172.21.15.154:0/3392859720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97c415d840 0x7f97c40c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:45.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.771+0000 7f97c2ffd640 1 -- 172.21.15.154:0/3392859720 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c415b100 con 0x7f97ac003970 2023-09-23T03:25:45.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.771+0000 7f97c27fc640 1 --2- 172.21.15.154:0/3392859720 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97c415d3e0 0x7f97c40c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:45.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.771+0000 7f97c37fe640 1 --2- 172.21.15.154:0/3392859720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97c415d840 0x7f97c40c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:45.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.772+0000 7f97c2ffd640 1 --2- 172.21.15.154:0/3392859720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97ac003970 0x7f97c4158c10 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f97b402c460 tx=0x7f97b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:45.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.772+0000 7f97ab7fe640 1 -- 172.21.15.154:0/3392859720 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b402de10 con 0x7f97ac003970 2023-09-23T03:25:45.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.772+0000 7f97ab7fe640 1 -- 172.21.15.154:0/3392859720 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b400f000 con 0x7f97ac003970 2023-09-23T03:25:45.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.772+0000 7f97ab7fe640 1 -- 172.21.15.154:0/3392859720 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b4006260 con 0x7f97ac003970 2023-09-23T03:25:45.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.772+0000 7f97c93f5640 1 -- 172.21.15.154:0/3392859720 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97c4157e50 con 0x7f97ac003970 2023-09-23T03:25:45.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.772+0000 7f97c93f5640 1 -- 172.21.15.154:0/3392859720 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c4158410 con 0x7f97ac003970 2023-09-23T03:25:45.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.773+0000 7f97c93f5640 1 -- 172.21.15.154:0/3392859720 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97c4159150 con 0x7f97ac003970 2023-09-23T03:25:45.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.774+0000 7f97ab7fe640 1 -- 172.21.15.154:0/3392859720 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f97b40063c0 con 0x7f97ac003970 2023-09-23T03:25:45.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.775+0000 7f97ab7fe640 1 --2- 172.21.15.154:0/3392859720 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f97b006af50 0x7f97b006d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:45.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.775+0000 7f97ab7fe640 1 -- 172.21.15.154:0/3392859720 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f97b40abc50 con 0x7f97ac003970 2023-09-23T03:25:45.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.776+0000 7f97c27fc640 1 --2- 172.21.15.154:0/3392859720 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f97b006af50 0x7f97b006d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:45.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.776+0000 7f97c27fc640 1 --2- 172.21.15.154:0/3392859720 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f97b006af50 0x7f97b006d410 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f97c415a590 tx=0x7f97b8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:45.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.781+0000 7f97ab7fe640 1 -- 172.21.15.154:0/3392859720 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f97b4076c90 con 0x7f97ac003970 2023-09-23T03:25:45.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:45.975+0000 7f97c93f5640 1 -- 172.21.15.154:0/3392859720 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f97c404e920 con 0x7f97ac003970 2023-09-23T03:25:46.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.528+0000 7f97ab7fe640 1 -- 172.21.15.154:0/3392859720 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v103) v1 ==== 170+0+0 (secure 0 0 0) 0x7f97b404d020 con 0x7f97ac003970 2023-09-23T03:25:46.529 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_data' created 2023-09-23T03:25:46.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.532+0000 7f97c93f5640 1 -- 172.21.15.154:0/3392859720 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f97b006af50 msgr2=0x7f97b006d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:46.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.532+0000 7f97c93f5640 1 --2- 172.21.15.154:0/3392859720 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f97b006af50 0x7f97b006d410 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f97c415a590 tx=0x7f97b8008040 comp rx=0 tx=0).stop 2023-09-23T03:25:46.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.532+0000 7f97c93f5640 1 -- 172.21.15.154:0/3392859720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97ac003970 msgr2=0x7f97c4158c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:46.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.532+0000 7f97c93f5640 1 --2- 172.21.15.154:0/3392859720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97ac003970 0x7f97c4158c10 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f97b402c460 tx=0x7f97b4004730 comp rx=0 tx=0).stop 2023-09-23T03:25:46.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.534+0000 7f97c93f5640 1 -- 172.21.15.154:0/3392859720 shutdown_connections 2023-09-23T03:25:46.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.534+0000 7f97c93f5640 1 --2- 172.21.15.154:0/3392859720 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f97b006af50 0x7f97b006d410 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:46.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.534+0000 7f97c93f5640 1 --2- 172.21.15.154:0/3392859720 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f97c415d840 0x7f97c40c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:46.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.534+0000 7f97c93f5640 1 --2- 172.21.15.154:0/3392859720 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f97c415d3e0 0x7f97c40c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:46.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.534+0000 7f97c93f5640 1 --2- 172.21.15.154:0/3392859720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97ac003970 0x7f97c4158c10 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:46.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.534+0000 7f97c93f5640 1 -- 172.21.15.154:0/3392859720 >> 172.21.15.154:0/3392859720 conn(0x7f97c406c010 msgr2=0x7f97c415c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:46.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.535+0000 7f97c93f5640 1 -- 172.21.15.154:0/3392859720 shutdown_connections 2023-09-23T03:25:46.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.535+0000 7f97c93f5640 1 -- 172.21.15.154:0/3392859720 wait complete. 2023-09-23T03:25:46.570 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-09-23T03:25:46.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.756+0000 7f9ff3240640 1 Processor -- start 2023-09-23T03:25:46.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.756+0000 7f9ff3240640 1 -- start start 2023-09-23T03:25:46.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.758+0000 7f9ff3240640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fec0c9410 0x7f9fec165520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:46.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.758+0000 7f9ff3240640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9fec0b0a80 0x7f9fec0cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:46.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.758+0000 7f9ff3240640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9fec0cade0 0x7f9fec0cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:46.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.758+0000 7f9ff3240640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fec0cd660 con 0x7f9fec0b0a80 2023-09-23T03:25:46.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.758+0000 7f9ff3240640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fec0cb7b0 con 0x7f9fec0cade0 2023-09-23T03:25:46.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.758+0000 7f9ff3240640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec0c97f0 con 0x7f9fec0c9410 2023-09-23T03:25:46.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.758+0000 7f9ff1a3d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9fec0b0a80 0x7f9fec0cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:46.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.758+0000 7f9ff1a3d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9fec0b0a80 0x7f9fec0cc7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48514/0 (socket says 172.21.15.154:48514) 2023-09-23T03:25:46.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.758+0000 7f9ff1a3d640 1 -- 172.21.15.154:0/2884655692 learned_addr learned my addr 172.21.15.154:0/2884655692 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:46.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.759+0000 7f9ff2a3f640 1 --2- 172.21.15.154:0/2884655692 >> v2:172.21.15.190:3300/0 conn(0x7f9fec0cade0 0x7f9fec0cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:46.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.759+0000 7f9ff223e640 1 --2- 172.21.15.154:0/2884655692 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fec0c9410 0x7f9fec165520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:46.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.759+0000 7f9ff1a3d640 1 -- 172.21.15.154:0/2884655692 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fec0c9410 msgr2=0x7f9fec165520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:46.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.759+0000 7f9ff1a3d640 1 --2- 172.21.15.154:0/2884655692 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fec0c9410 0x7f9fec165520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:46.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.759+0000 7f9ff1a3d640 1 -- 172.21.15.154:0/2884655692 >> v2:172.21.15.190:3300/0 conn(0x7f9fec0cade0 msgr2=0x7f9fec0cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:46.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.759+0000 7f9ff1a3d640 1 --2- 172.21.15.154:0/2884655692 >> v2:172.21.15.190:3300/0 conn(0x7f9fec0cade0 0x7f9fec0cb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:46.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.759+0000 7f9ff1a3d640 1 -- 172.21.15.154:0/2884655692 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fec161450 con 0x7f9fec0b0a80 2023-09-23T03:25:46.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.759+0000 7f9ff2a3f640 1 --2- 172.21.15.154:0/2884655692 >> v2:172.21.15.190:3300/0 conn(0x7f9fec0cade0 0x7f9fec0cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:46.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.759+0000 7f9ff223e640 1 --2- 172.21.15.154:0/2884655692 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fec0c9410 0x7f9fec165520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:46.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.760+0000 7f9ff1a3d640 1 --2- 172.21.15.154:0/2884655692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fec0b0a80 0x7f9fec0cc7a0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f9fec0c1e30 tx=0x7f9fe002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c44264ce34c9e759 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:46.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.760+0000 7f9ff123c640 1 -- 172.21.15.154:0/2884655692 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0031870 con 0x7f9fec0b0a80 2023-09-23T03:25:46.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.761+0000 7f9ff123c640 1 -- 172.21.15.154:0/2884655692 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe00319d0 con 0x7f9fec0b0a80 2023-09-23T03:25:46.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.761+0000 7f9ff123c640 1 -- 172.21.15.154:0/2884655692 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0031d60 con 0x7f9fec0b0a80 2023-09-23T03:25:46.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.761+0000 7f9ff3240640 1 -- 172.21.15.154:0/2884655692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fec0b0a80 msgr2=0x7f9fec0cc7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:46.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.761+0000 7f9ff3240640 1 --2- 172.21.15.154:0/2884655692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fec0b0a80 0x7f9fec0cc7a0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f9fec0c1e30 tx=0x7f9fe002b910 comp rx=0 tx=0).stop 2023-09-23T03:25:46.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.761+0000 7f9ff3240640 1 -- 172.21.15.154:0/2884655692 shutdown_connections 2023-09-23T03:25:46.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.761+0000 7f9ff3240640 1 --2- 172.21.15.154:0/2884655692 >> v2:172.21.15.190:3300/0 conn(0x7f9fec0cade0 0x7f9fec0cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:46.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.761+0000 7f9ff3240640 1 --2- 172.21.15.154:0/2884655692 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fec0b0a80 0x7f9fec0cc7a0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:46.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.761+0000 7f9ff3240640 1 --2- 172.21.15.154:0/2884655692 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fec0c9410 0x7f9fec165520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:46.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.761+0000 7f9ff3240640 1 -- 172.21.15.154:0/2884655692 >> 172.21.15.154:0/2884655692 conn(0x7f9fec0aff20 msgr2=0x7f9fec0b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:46.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.761+0000 7f9ff3240640 1 -- 172.21.15.154:0/2884655692 shutdown_connections 2023-09-23T03:25:46.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.761+0000 7f9ff3240640 1 -- 172.21.15.154:0/2884655692 wait complete. 2023-09-23T03:25:46.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.763+0000 7f9ff3240640 1 Processor -- start 2023-09-23T03:25:46.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.763+0000 7f9ff3240640 1 -- start start 2023-09-23T03:25:46.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.764+0000 7f9ff3240640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fec0b0a80 0x7f9fec0b3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:46.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.764+0000 7f9ff3240640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fec0c9410 0x7f9fec14d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:46.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.764+0000 7f9ff3240640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fec0cade0 0x7f9fec14d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:46.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.764+0000 7f9ff3240640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec14eaa0 con 0x7f9fec0cade0 2023-09-23T03:25:46.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.764+0000 7f9ff3240640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec14dc00 con 0x7f9fec0b0a80 2023-09-23T03:25:46.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.764+0000 7f9ff3240640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec14de90 con 0x7f9fec0c9410 2023-09-23T03:25:46.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.764+0000 7f9ff2a3f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fec0cade0 0x7f9fec14d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:46.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.764+0000 7f9ff2a3f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fec0cade0 0x7f9fec14d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48522/0 (socket says 172.21.15.154:48522) 2023-09-23T03:25:46.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.765+0000 7f9ff2a3f640 1 -- 172.21.15.154:0/2140873746 learned_addr learned my addr 172.21.15.154:0/2140873746 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:46.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.765+0000 7f9ff223e640 1 --2- 172.21.15.154:0/2140873746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fec0b0a80 0x7f9fec0b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:46.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.765+0000 7f9ff1a3d640 1 --2- 172.21.15.154:0/2140873746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fec0c9410 0x7f9fec14d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:46.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.765+0000 7f9ff2a3f640 1 -- 172.21.15.154:0/2140873746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fec0c9410 msgr2=0x7f9fec14d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:46.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.765+0000 7f9ff2a3f640 1 --2- 172.21.15.154:0/2140873746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fec0c9410 0x7f9fec14d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:46.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.765+0000 7f9ff2a3f640 1 -- 172.21.15.154:0/2140873746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fec0b0a80 msgr2=0x7f9fec0b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:46.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.765+0000 7f9ff2a3f640 1 --2- 172.21.15.154:0/2140873746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fec0b0a80 0x7f9fec0b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:46.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.765+0000 7f9ff2a3f640 1 -- 172.21.15.154:0/2140873746 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe00094d0 con 0x7f9fec0cade0 2023-09-23T03:25:46.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.765+0000 7f9ff1a3d640 1 --2- 172.21.15.154:0/2140873746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fec0c9410 0x7f9fec14d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:46.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.765+0000 7f9ff2a3f640 1 --2- 172.21.15.154:0/2140873746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fec0cade0 0x7f9fec14d6c0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400cd50 tx=0x7f9fe400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:46.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.766+0000 7f9fdeffd640 1 -- 172.21.15.154:0/2140873746 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe400f680 con 0x7f9fec0cade0 2023-09-23T03:25:46.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.766+0000 7f9fdeffd640 1 -- 172.21.15.154:0/2140873746 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe400f800 con 0x7f9fec0cade0 2023-09-23T03:25:46.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.766+0000 7f9fdeffd640 1 -- 172.21.15.154:0/2140873746 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe400fbb0 con 0x7f9fec0cade0 2023-09-23T03:25:46.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.766+0000 7f9ff3240640 1 -- 172.21.15.154:0/2140873746 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fec14e170 con 0x7f9fec0cade0 2023-09-23T03:25:46.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.766+0000 7f9ff3240640 1 -- 172.21.15.154:0/2140873746 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fec14e730 con 0x7f9fec0cade0 2023-09-23T03:25:46.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.767+0000 7f9ff3240640 1 -- 172.21.15.154:0/2140873746 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fec04e920 con 0x7f9fec0cade0 2023-09-23T03:25:46.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.768+0000 7f9fdeffd640 1 -- 172.21.15.154:0/2140873746 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f9fe400b040 con 0x7f9fec0cade0 2023-09-23T03:25:46.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.769+0000 7f9fdeffd640 1 --2- 172.21.15.154:0/2140873746 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9fd006af30 0x7f9fd006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:46.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.769+0000 7f9fdeffd640 1 -- 172.21.15.154:0/2140873746 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f9fe408a260 con 0x7f9fec0cade0 2023-09-23T03:25:46.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.770+0000 7f9ff223e640 1 --2- 172.21.15.154:0/2140873746 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9fd006af30 0x7f9fd006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:46.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.770+0000 7f9ff223e640 1 --2- 172.21.15.154:0/2140873746 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9fd006af30 0x7f9fd006d3f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8006b10 tx=0x7f9fe8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:46.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.775+0000 7f9fdeffd640 1 -- 172.21.15.154:0/2140873746 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f9fe40551c0 con 0x7f9fec0cade0 2023-09-23T03:25:46.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:46.996+0000 7f9ff3240640 1 -- 172.21.15.154:0/2140873746 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9fec0c1390 con 0x7f9fec0cade0 2023-09-23T03:25:47.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.533+0000 7f9fdeffd640 1 -- 172.21.15.154:0/2140873746 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v80) v1 ==== 174+0+0 (secure 0 0 0) 0x7f9fe40587e0 con 0x7f9fec0cade0 2023-09-23T03:25:47.533 INFO:teuthology.orchestra.run.smithi154.stderr:new fs with metadata pool 14 and data pool 15 2023-09-23T03:25:47.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.537+0000 7f9ff3240640 1 -- 172.21.15.154:0/2140873746 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9fd006af30 msgr2=0x7f9fd006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:47.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.537+0000 7f9ff3240640 1 --2- 172.21.15.154:0/2140873746 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9fd006af30 0x7f9fd006d3f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8006b10 tx=0x7f9fe8008040 comp rx=0 tx=0).stop 2023-09-23T03:25:47.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.537+0000 7f9ff3240640 1 -- 172.21.15.154:0/2140873746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fec0cade0 msgr2=0x7f9fec14d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:47.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.537+0000 7f9ff3240640 1 --2- 172.21.15.154:0/2140873746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fec0cade0 0x7f9fec14d6c0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400cd50 tx=0x7f9fe400a650 comp rx=0 tx=0).stop 2023-09-23T03:25:47.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.537+0000 7f9ff3240640 1 -- 172.21.15.154:0/2140873746 shutdown_connections 2023-09-23T03:25:47.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.537+0000 7f9ff3240640 1 --2- 172.21.15.154:0/2140873746 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9fd006af30 0x7f9fd006d3f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:47.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.537+0000 7f9ff3240640 1 --2- 172.21.15.154:0/2140873746 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fec0cade0 0x7f9fec14d6c0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:47.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.537+0000 7f9ff3240640 1 --2- 172.21.15.154:0/2140873746 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9fec0c9410 0x7f9fec14d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:47.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.537+0000 7f9ff3240640 1 --2- 172.21.15.154:0/2140873746 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9fec0b0a80 0x7f9fec0b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:47.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.537+0000 7f9ff3240640 1 -- 172.21.15.154:0/2140873746 >> 172.21.15.154:0/2140873746 conn(0x7f9fec0aff20 msgr2=0x7f9fec06fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:47.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.539+0000 7f9ff3240640 1 -- 172.21.15.154:0/2140873746 shutdown_connections 2023-09-23T03:25:47.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.541+0000 7f9ff3240640 1 -- 172.21.15.154:0/2140873746 wait complete. 2023-09-23T03:25:47.584 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:25:47.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.761+0000 7f7a10505640 1 Processor -- start 2023-09-23T03:25:47.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.761+0000 7f7a10505640 1 -- start start 2023-09-23T03:25:47.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.762+0000 7f7a10505640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a080c14e0 0x7f7a080c1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:47.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.762+0000 7f7a10505640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7a0806c1d0 0x7f7a0806c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:47.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.762+0000 7f7a10505640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a0806cb80 con 0x7f7a08161010 2023-09-23T03:25:47.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.762+0000 7f7a10505640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a080c1ea0 con 0x7f7a0806c1d0 2023-09-23T03:25:47.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.762+0000 7f7a10505640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a080cb160 con 0x7f7a080c14e0 2023-09-23T03:25:47.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.763+0000 7f7a0eaa2640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7a0806c1d0 0x7f7a0806c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:47.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.763+0000 7f7a0daa0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a080c14e0 0x7f7a080c1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:47.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.763+0000 7f7a0eaa2640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7a0806c1d0 0x7f7a0806c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.154:40784/0 (socket says 172.21.15.154:40784) 2023-09-23T03:25:47.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.763+0000 7f7a0eaa2640 1 -- 172.21.15.154:0/2855860609 learned_addr learned my addr 172.21.15.154:0/2855860609 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:47.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.764+0000 7f7a0daa0640 1 -- 172.21.15.154:0/2855860609 >> v2:172.21.15.190:3300/0 conn(0x7f7a0806c1d0 msgr2=0x7f7a0806c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:47.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.764+0000 7f7a0daa0640 1 --2- 172.21.15.154:0/2855860609 >> v2:172.21.15.190:3300/0 conn(0x7f7a0806c1d0 0x7f7a0806c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:47.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.764+0000 7f7a0daa0640 1 -- 172.21.15.154:0/2855860609 >> v1:172.21.15.190:6789/0 conn(0x7f7a08161010 legacy=0x7f7a080cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:47.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.764+0000 7f7a0daa0640 1 -- 172.21.15.154:0/2855860609 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a0806cde0 con 0x7f7a080c14e0 2023-09-23T03:25:47.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.764+0000 7f7a0eaa2640 1 --2- 172.21.15.154:0/2855860609 >> v2:172.21.15.190:3300/0 conn(0x7f7a0806c1d0 0x7f7a0806c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:25:47.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.765+0000 7f7a0daa0640 1 --2- 172.21.15.154:0/2855860609 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a080c14e0 0x7f7a080c1960 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f79fc0097a0 tx=0x7f79fc02b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8096c3d4d1cca85e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:47.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.766+0000 7f7a0d29f640 1 -- 172.21.15.154:0/2855860609 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc031780 con 0x7f7a080c14e0 2023-09-23T03:25:47.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.766+0000 7f7a0d29f640 1 -- 172.21.15.154:0/2855860609 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79fc0318e0 con 0x7f7a080c14e0 2023-09-23T03:25:47.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.766+0000 7f7a0d29f640 1 -- 172.21.15.154:0/2855860609 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc031c70 con 0x7f7a080c14e0 2023-09-23T03:25:47.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.766+0000 7f7a10505640 1 -- 172.21.15.154:0/2855860609 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a080c14e0 msgr2=0x7f7a080c1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:47.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.766+0000 7f7a10505640 1 --2- 172.21.15.154:0/2855860609 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a080c14e0 0x7f7a080c1960 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f79fc0097a0 tx=0x7f79fc02b820 comp rx=0 tx=0).stop 2023-09-23T03:25:47.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.767+0000 7f7a10505640 1 -- 172.21.15.154:0/2855860609 shutdown_connections 2023-09-23T03:25:47.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.767+0000 7f7a10505640 1 --2- 172.21.15.154:0/2855860609 >> v2:172.21.15.190:3300/0 conn(0x7f7a0806c1d0 0x7f7a0806c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:47.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.767+0000 7f7a10505640 1 --2- 172.21.15.154:0/2855860609 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a080c14e0 0x7f7a080c1960 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:47.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.767+0000 7f7a10505640 1 -- 172.21.15.154:0/2855860609 >> 172.21.15.154:0/2855860609 conn(0x7f7a0806a0f0 msgr2=0x7f7a0806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:47.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.767+0000 7f7a10505640 1 -- 172.21.15.154:0/2855860609 shutdown_connections 2023-09-23T03:25:47.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.767+0000 7f7a10505640 1 -- 172.21.15.154:0/2855860609 wait complete. 2023-09-23T03:25:47.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.768+0000 7f7a10505640 1 Processor -- start 2023-09-23T03:25:47.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.768+0000 7f7a10505640 1 -- start start 2023-09-23T03:25:47.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.769+0000 7f7a10505640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a0806c1d0 0x7f7a080708f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:47.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.769+0000 7f7a10505640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a080c14e0 0x7f7a080c4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:47.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.769+0000 7f7a10505640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a08161010 0x7f7a0814f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:47.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.769+0000 7f7a10505640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0814f5e0 con 0x7f7a08161010 2023-09-23T03:25:47.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.769+0000 7f7a10505640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a08071850 con 0x7f7a080c14e0 2023-09-23T03:25:47.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.769+0000 7f7a10505640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a08070fb0 con 0x7f7a0806c1d0 2023-09-23T03:25:47.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.770+0000 7f7a0eaa2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a08161010 0x7f7a0814f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:47.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.770+0000 7f7a0eaa2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a08161010 0x7f7a0814f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48528/0 (socket says 172.21.15.154:48528) 2023-09-23T03:25:47.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.770+0000 7f7a0eaa2640 1 -- 172.21.15.154:0/2831052507 learned_addr learned my addr 172.21.15.154:0/2831052507 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:47.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.770+0000 7f7a0e2a1640 1 --2- 172.21.15.154:0/2831052507 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a0806c1d0 0x7f7a080708f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:47.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.770+0000 7f7a0daa0640 1 --2- 172.21.15.154:0/2831052507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a080c14e0 0x7f7a080c4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:47.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.770+0000 7f7a0eaa2640 1 -- 172.21.15.154:0/2831052507 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a0806c1d0 msgr2=0x7f7a080708f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:47.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.770+0000 7f7a0eaa2640 1 --2- 172.21.15.154:0/2831052507 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a0806c1d0 0x7f7a080708f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:47.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.770+0000 7f7a0eaa2640 1 -- 172.21.15.154:0/2831052507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a080c14e0 msgr2=0x7f7a080c4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:47.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.770+0000 7f7a0eaa2640 1 --2- 172.21.15.154:0/2831052507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a080c14e0 0x7f7a080c4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:47.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.770+0000 7f7a0eaa2640 1 -- 172.21.15.154:0/2831052507 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a000094d0 con 0x7f7a08161010 2023-09-23T03:25:47.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.770+0000 7f7a0eaa2640 1 --2- 172.21.15.154:0/2831052507 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a08161010 0x7f7a0814f0a0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f7a0000e930 tx=0x7f7a0000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:47.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.771+0000 7f7a06ffd640 1 -- 172.21.15.154:0/2831052507 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a00009720 con 0x7f7a08161010 2023-09-23T03:25:47.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.771+0000 7f7a06ffd640 1 -- 172.21.15.154:0/2831052507 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a0000ca00 con 0x7f7a08161010 2023-09-23T03:25:47.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.771+0000 7f7a06ffd640 1 -- 172.21.15.154:0/2831052507 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0000cd90 con 0x7f7a08161010 2023-09-23T03:25:47.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.771+0000 7f7a10505640 1 -- 172.21.15.154:0/2831052507 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79fc009400 con 0x7f7a08161010 2023-09-23T03:25:47.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.771+0000 7f7a10505640 1 -- 172.21.15.154:0/2831052507 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a08071570 con 0x7f7a08161010 2023-09-23T03:25:47.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.772+0000 7f7a10505640 1 -- 172.21.15.154:0/2831052507 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a08073eb0 con 0x7f7a08161010 2023-09-23T03:25:47.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.773+0000 7f7a06ffd640 1 -- 172.21.15.154:0/2831052507 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f7a00013470 con 0x7f7a08161010 2023-09-23T03:25:47.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.774+0000 7f7a06ffd640 1 --2- 172.21.15.154:0/2831052507 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f79f406af30 0x7f79f406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:47.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.774+0000 7f7a06ffd640 1 -- 172.21.15.154:0/2831052507 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7a000894b0 con 0x7f7a08161010 2023-09-23T03:25:47.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.774+0000 7f7a0e2a1640 1 --2- 172.21.15.154:0/2831052507 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f79f406af30 0x7f79f406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:47.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.775+0000 7f7a0e2a1640 1 --2- 172.21.15.154:0/2831052507 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f79f406af30 0x7f79f406d3f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f79f8001330 tx=0x7f79f8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:47.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.778+0000 7f7a06ffd640 1 -- 172.21.15.154:0/2831052507 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f7a00054390 con 0x7f7a08161010 2023-09-23T03:25:47.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.953+0000 7f7a10505640 1 -- 172.21.15.154:0/2831052507 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a080ceda0 con 0x7f7a08161010 2023-09-23T03:25:47.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.956+0000 7f7a06ffd640 1 -- 172.21.15.154:0/2831052507 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+16773 (secure 0 0 0) 0x7f7a000579b0 con 0x7f7a08161010 2023-09-23T03:25:47.957 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:25:47.957 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":104,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:25:46.997336+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:25:44.763842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-09-23T03:25:46.025926+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:25:47.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.958+0000 7f7a10505640 1 -- 172.21.15.154:0/2831052507 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f79f406af30 msgr2=0x7f79f406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:47.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.958+0000 7f7a10505640 1 --2- 172.21.15.154:0/2831052507 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f79f406af30 0x7f79f406d3f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f79f8001330 tx=0x7f79f8009040 comp rx=0 tx=0).stop 2023-09-23T03:25:47.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.958+0000 7f7a10505640 1 -- 172.21.15.154:0/2831052507 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a08161010 msgr2=0x7f7a0814f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:47.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.958+0000 7f7a10505640 1 --2- 172.21.15.154:0/2831052507 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a08161010 0x7f7a0814f0a0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f7a0000e930 tx=0x7f7a0000edf0 comp rx=0 tx=0).stop 2023-09-23T03:25:47.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.958+0000 7f7a10505640 1 -- 172.21.15.154:0/2831052507 shutdown_connections 2023-09-23T03:25:47.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.958+0000 7f7a10505640 1 --2- 172.21.15.154:0/2831052507 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f79f406af30 0x7f79f406d3f0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:47.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.958+0000 7f7a10505640 1 --2- 172.21.15.154:0/2831052507 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a08161010 0x7f7a0814f0a0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:47.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.958+0000 7f7a10505640 1 --2- 172.21.15.154:0/2831052507 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7a080c14e0 0x7f7a080c4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:47.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.958+0000 7f7a10505640 1 --2- 172.21.15.154:0/2831052507 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7a0806c1d0 0x7f7a080708f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:47.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.958+0000 7f7a10505640 1 -- 172.21.15.154:0/2831052507 >> 172.21.15.154:0/2831052507 conn(0x7f7a0806a0f0 msgr2=0x7f7a0806a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:47.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.958+0000 7f7a10505640 1 -- 172.21.15.154:0/2831052507 shutdown_connections 2023-09-23T03:25:47.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:47.958+0000 7f7a10505640 1 -- 172.21.15.154:0/2831052507 wait complete. 2023-09-23T03:25:47.976 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:25:48.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.182+0000 7f9f7e8c9640 1 Processor -- start 2023-09-23T03:25:48.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.182+0000 7f9f7e8c9640 1 -- start start 2023-09-23T03:25:48.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.183+0000 7f9f7e8c9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9f78161010 0x7f9f780cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:48.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.183+0000 7f9f7e8c9640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f780cb160 con 0x7f9f780c14e0 2023-09-23T03:25:48.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.183+0000 7f9f7e8c9640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f780ccd20 con 0x7f9f7806c8e0 2023-09-23T03:25:48.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.183+0000 7f9f7e8c9640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f7806cde0 con 0x7f9f78161010 2023-09-23T03:25:48.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.184+0000 7f9f777fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f9f780c14e0 0x7f9f7806c1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:49550/0 (socket says 172.21.15.154:49550) 2023-09-23T03:25:48.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.184+0000 7f9f777fe640 1 -- 172.21.15.154:0/2345784422 learned_addr learned my addr 172.21.15.154:0/2345784422 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:25:48.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.184+0000 7f9f77fff640 1 --2- 172.21.15.154:0/2345784422 >> v2:172.21.15.190:3300/0 conn(0x7f9f78161010 0x7f9f780cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:48.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.184+0000 7f9f76ffd640 1 -- 172.21.15.154:0/2345784422 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3556285770 0 0) 0x7f9f780cb160 con 0x7f9f780c14e0 2023-09-23T03:25:48.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.184+0000 7f9f76ffd640 1 -- 172.21.15.154:0/2345784422 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f64003520 con 0x7f9f780c14e0 2023-09-23T03:25:48.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.185+0000 7f9f76ffd640 1 -- 172.21.15.154:0/2345784422 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1491087683 0 0) 0x7f9f64003520 con 0x7f9f780c14e0 2023-09-23T03:25:48.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.185+0000 7f9f76ffd640 1 -- 172.21.15.154:0/2345784422 >> v2:172.21.15.190:3300/0 conn(0x7f9f78161010 msgr2=0x7f9f780cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:48.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.185+0000 7f9f76ffd640 1 --2- 172.21.15.154:0/2345784422 >> v2:172.21.15.190:3300/0 conn(0x7f9f78161010 0x7f9f780cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:48.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.185+0000 7f9f76ffd640 1 -- 172.21.15.154:0/2345784422 >> v1:172.21.15.190:6789/0 conn(0x7f9f7806c8e0 legacy=0x7f9f780b1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:48.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.185+0000 7f9f76ffd640 1 -- 172.21.15.154:0/2345784422 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f7806d0c0 con 0x7f9f780c14e0 2023-09-23T03:25:48.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.185+0000 7f9f76ffd640 1 -- 172.21.15.154:0/2345784422 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f9f600019f0 con 0x7f9f780c14e0 2023-09-23T03:25:48.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.185+0000 7f9f76ffd640 1 -- 172.21.15.154:0/2345784422 >> v1:172.21.15.154:6789/0 conn(0x7f9f780c14e0 legacy=0x7f9f7806c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:48.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.185+0000 7f9f76ffd640 1 --2- 172.21.15.154:0/2345784422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f64003970 0x7f9f64003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:48.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.185+0000 7f9f76ffd640 1 --2- 172.21.15.154:0/2345784422 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f640047c0 0x7f9f64004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:48.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.185+0000 7f9f76ffd640 1 --2- 172.21.15.154:0/2345784422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f64005180 0x7f9f6400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:48.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.185+0000 7f9f76ffd640 1 -- 172.21.15.154:0/2345784422 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7806cde0 con 0x7f9f640047c0 2023-09-23T03:25:48.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.185+0000 7f9f76ffd640 1 -- 172.21.15.154:0/2345784422 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6400c120 con 0x7f9f64005180 2023-09-23T03:25:48.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.185+0000 7f9f76ffd640 1 -- 172.21.15.154:0/2345784422 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6400c2e0 con 0x7f9f64003970 2023-09-23T03:25:48.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.185+0000 7f9f777fe640 1 --2- 172.21.15.154:0/2345784422 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f640047c0 0x7f9f64004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:48.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.185+0000 7f9f77fff640 1 --2- 172.21.15.154:0/2345784422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f64003970 0x7f9f64003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:48.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.185+0000 7f9f7ce66640 1 --2- 172.21.15.154:0/2345784422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f64005180 0x7f9f6400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:48.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.185+0000 7f9f777fe640 1 -- 172.21.15.154:0/2345784422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f64003970 msgr2=0x7f9f64003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:48.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.185+0000 7f9f777fe640 1 --2- 172.21.15.154:0/2345784422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f64003970 0x7f9f64003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:48.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.185+0000 7f9f777fe640 1 -- 172.21.15.154:0/2345784422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f64005180 msgr2=0x7f9f6400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:48.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.185+0000 7f9f777fe640 1 --2- 172.21.15.154:0/2345784422 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f64005180 0x7f9f6400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:48.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.185+0000 7f9f777fe640 1 -- 172.21.15.154:0/2345784422 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f64003520 con 0x7f9f640047c0 2023-09-23T03:25:48.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.185+0000 7f9f7ce66640 1 -- 172.21.15.154:0/2345784422 reap_dead start 2023-09-23T03:25:48.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.185+0000 7f9f77fff640 1 --2- 172.21.15.154:0/2345784422 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f64003970 0x7f9f64003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:48.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.187+0000 7f9f777fe640 1 --2- 172.21.15.154:0/2345784422 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f640047c0 0x7f9f64004c40 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f9f6000a830 tx=0x7f9f640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d809be3a6f2c5a80 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:48.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.187+0000 7f9f76ffd640 1 -- 172.21.15.154:0/2345784422 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f600072d0 con 0x7f9f640047c0 2023-09-23T03:25:48.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.187+0000 7f9f76ffd640 1 -- 172.21.15.154:0/2345784422 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f6002f070 con 0x7f9f640047c0 2023-09-23T03:25:48.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.187+0000 7f9f76ffd640 1 -- 172.21.15.154:0/2345784422 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f600076f0 con 0x7f9f640047c0 2023-09-23T03:25:48.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.187+0000 7f9f7e8c9640 1 -- 172.21.15.154:0/2345784422 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f640047c0 msgr2=0x7f9f64004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:48.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.187+0000 7f9f7e8c9640 1 --2- 172.21.15.154:0/2345784422 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f640047c0 0x7f9f64004c40 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f9f6000a830 tx=0x7f9f640057f0 comp rx=0 tx=0).stop 2023-09-23T03:25:48.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.188+0000 7f9f7e8c9640 1 -- 172.21.15.154:0/2345784422 shutdown_connections 2023-09-23T03:25:48.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.188+0000 7f9f7e8c9640 1 --2- 172.21.15.154:0/2345784422 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f640047c0 0x7f9f64004c40 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:48.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.188+0000 7f9f7e8c9640 1 -- 172.21.15.154:0/2345784422 >> 172.21.15.154:0/2345784422 conn(0x7f9f7806a0f0 msgr2=0x7f9f7806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:48.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.188+0000 7f9f7e8c9640 1 -- 172.21.15.154:0/2345784422 shutdown_connections 2023-09-23T03:25:48.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.188+0000 7f9f7e8c9640 1 -- 172.21.15.154:0/2345784422 wait complete. 2023-09-23T03:25:48.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.189+0000 7f9f7e8c9640 1 Processor -- start 2023-09-23T03:25:48.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.190+0000 7f9f7e8c9640 1 -- start start 2023-09-23T03:25:48.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.190+0000 7f9f7e8c9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f640047c0 0x7f9f78159ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:48.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.190+0000 7f9f7e8c9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f7815a160 0x7f9f7815fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:48.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.191+0000 7f9f7e8c9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f781614a0 0x7f9f78161930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:48.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.191+0000 7f9f7e8c9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7815a000 con 0x7f9f781614a0 2023-09-23T03:25:48.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.191+0000 7f9f7e8c9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f78161e70 con 0x7f9f640047c0 2023-09-23T03:25:48.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.191+0000 7f9f7e8c9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f781ea7e0 con 0x7f9f7815a160 2023-09-23T03:25:48.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.191+0000 7f9f7ce66640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f781614a0 0x7f9f78161930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:48.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.191+0000 7f9f7ce66640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f781614a0 0x7f9f78161930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48546/0 (socket says 172.21.15.154:48546) 2023-09-23T03:25:48.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.191+0000 7f9f7ce66640 1 -- 172.21.15.154:0/970164095 learned_addr learned my addr 172.21.15.154:0/970164095 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:48.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.191+0000 7f9f77fff640 1 --2- 172.21.15.154:0/970164095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f640047c0 0x7f9f78159ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:48.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.191+0000 7f9f777fe640 1 --2- 172.21.15.154:0/970164095 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f7815a160 0x7f9f7815fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:48.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.191+0000 7f9f7ce66640 1 -- 172.21.15.154:0/970164095 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f7815a160 msgr2=0x7f9f7815fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:48.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.191+0000 7f9f7ce66640 1 --2- 172.21.15.154:0/970164095 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f7815a160 0x7f9f7815fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:48.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.191+0000 7f9f7ce66640 1 -- 172.21.15.154:0/970164095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f640047c0 msgr2=0x7f9f78159ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:48.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.191+0000 7f9f7ce66640 1 --2- 172.21.15.154:0/970164095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f640047c0 0x7f9f78159ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:48.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.191+0000 7f9f7ce66640 1 -- 172.21.15.154:0/970164095 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f781eaa30 con 0x7f9f781614a0 2023-09-23T03:25:48.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.191+0000 7f9f77fff640 1 --2- 172.21.15.154:0/970164095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f640047c0 0x7f9f78159ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:48.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.191+0000 7f9f777fe640 1 --2- 172.21.15.154:0/970164095 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f7815a160 0x7f9f7815fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:48.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.191+0000 7f9f7ce66640 1 --2- 172.21.15.154:0/970164095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f781614a0 0x7f9f78161930 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f9f781648e0 tx=0x7f9f7000d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:48.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.192+0000 7f9f74ff9640 1 -- 172.21.15.154:0/970164095 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7000b070 con 0x7f9f781614a0 2023-09-23T03:25:48.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.192+0000 7f9f74ff9640 1 -- 172.21.15.154:0/970164095 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f700035f0 con 0x7f9f781614a0 2023-09-23T03:25:48.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.192+0000 7f9f74ff9640 1 -- 172.21.15.154:0/970164095 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f70007ea0 con 0x7f9f781614a0 2023-09-23T03:25:48.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.192+0000 7f9f7e8c9640 1 -- 172.21.15.154:0/970164095 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f781ead10 con 0x7f9f781614a0 2023-09-23T03:25:48.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.192+0000 7f9f7e8c9640 1 -- 172.21.15.154:0/970164095 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f781eb2d0 con 0x7f9f781614a0 2023-09-23T03:25:48.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.193+0000 7f9f7e8c9640 1 -- 172.21.15.154:0/970164095 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f78073eb0 con 0x7f9f781614a0 2023-09-23T03:25:48.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.194+0000 7f9f74ff9640 1 -- 172.21.15.154:0/970164095 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f9f700039a0 con 0x7f9f781614a0 2023-09-23T03:25:48.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.195+0000 7f9f74ff9640 1 --2- 172.21.15.154:0/970164095 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9f5806af30 0x7f9f5806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:48.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.195+0000 7f9f74ff9640 1 -- 172.21.15.154:0/970164095 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f9f70088de0 con 0x7f9f781614a0 2023-09-23T03:25:48.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.195+0000 7f9f77fff640 1 --2- 172.21.15.154:0/970164095 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9f5806af30 0x7f9f5806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:48.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.196+0000 7f9f77fff640 1 --2- 172.21.15.154:0/970164095 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9f5806af30 0x7f9f5806d3f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9f78071210 tx=0x7f9f68008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:48.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.199+0000 7f9f74ff9640 1 -- 172.21.15.154:0/970164095 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f9f70053d70 con 0x7f9f781614a0 2023-09-23T03:25:48.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.388+0000 7f9f7e8c9640 1 -- 172.21.15.154:0/970164095 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f7806bfb0 con 0x7f9f781614a0 2023-09-23T03:25:48.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.390+0000 7f9f74ff9640 1 -- 172.21.15.154:0/970164095 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+16773 (secure 0 0 0) 0x7f9f70018020 con 0x7f9f781614a0 2023-09-23T03:25:48.391 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:25:48.391 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":104,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:25:46.997336+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:25:44.763842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-09-23T03:25:46.025926+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:25:48.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.393+0000 7f9f7e8c9640 1 -- 172.21.15.154:0/970164095 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9f5806af30 msgr2=0x7f9f5806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:48.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.393+0000 7f9f7e8c9640 1 --2- 172.21.15.154:0/970164095 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9f5806af30 0x7f9f5806d3f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9f78071210 tx=0x7f9f68008040 comp rx=0 tx=0).stop 2023-09-23T03:25:48.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.393+0000 7f9f7e8c9640 1 -- 172.21.15.154:0/970164095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f781614a0 msgr2=0x7f9f78161930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:48.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.393+0000 7f9f7e8c9640 1 --2- 172.21.15.154:0/970164095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f781614a0 0x7f9f78161930 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f9f781648e0 tx=0x7f9f7000d890 comp rx=0 tx=0).stop 2023-09-23T03:25:48.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.393+0000 7f9f7e8c9640 1 -- 172.21.15.154:0/970164095 shutdown_connections 2023-09-23T03:25:48.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.393+0000 7f9f7e8c9640 1 --2- 172.21.15.154:0/970164095 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9f5806af30 0x7f9f5806d3f0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:48.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.393+0000 7f9f7e8c9640 1 --2- 172.21.15.154:0/970164095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f781614a0 0x7f9f78161930 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:48.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.393+0000 7f9f7e8c9640 1 --2- 172.21.15.154:0/970164095 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f7815a160 0x7f9f7815fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:48.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.393+0000 7f9f7e8c9640 1 --2- 172.21.15.154:0/970164095 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f640047c0 0x7f9f78159ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:48.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.393+0000 7f9f7e8c9640 1 -- 172.21.15.154:0/970164095 >> 172.21.15.154:0/970164095 conn(0x7f9f7806a0f0 msgr2=0x7f9f78071d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:48.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.393+0000 7f9f7e8c9640 1 -- 172.21.15.154:0/970164095 shutdown_connections 2023-09-23T03:25:48.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.393+0000 7f9f7e8c9640 1 -- 172.21.15.154:0/970164095 wait complete. 2023-09-23T03:25:48.412 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-09-23T03:25:48.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.630+0000 7fc172ff5640 1 Processor -- start 2023-09-23T03:25:48.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.630+0000 7fc172ff5640 1 -- start start 2023-09-23T03:25:48.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.632+0000 7fc172ff5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc16c15aba0 0x7fc16c15bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:48.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.632+0000 7fc172ff5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc16c15e5f0 0x7fc16c15ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:48.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.632+0000 7fc172ff5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc16c15b6a0 con 0x7fc16c0caea0 2023-09-23T03:25:48.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.632+0000 7fc172ff5640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc16c15f0b0 con 0x7fc16c15aba0 2023-09-23T03:25:48.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.632+0000 7fc172ff5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c15f390 con 0x7fc16c15e5f0 2023-09-23T03:25:48.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.632+0000 7fc171592640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fc16c0caea0 0x7fc16c0cc5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:49566/0 (socket says 172.21.15.154:49566) 2023-09-23T03:25:48.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.632+0000 7fc171592640 1 -- 172.21.15.154:0/201972430 learned_addr learned my addr 172.21.15.154:0/201972430 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:25:48.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.633+0000 7fc170d91640 1 --2- 172.21.15.154:0/201972430 >> v2:172.21.15.154:3300/0 conn(0x7fc16c15aba0 0x7fc16c15bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:48.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.633+0000 7fc16bfff640 1 --2- 172.21.15.154:0/201972430 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc16c15e5f0 0x7fc16c15ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:48.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.633+0000 7fc16b7fe640 1 -- 172.21.15.154:0/201972430 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1473459993 0 0) 0x7fc16c15b6a0 con 0x7fc16c0caea0 2023-09-23T03:25:48.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.633+0000 7fc16b7fe640 1 -- 172.21.15.154:0/201972430 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc15c003520 con 0x7fc16c0caea0 2023-09-23T03:25:48.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.633+0000 7fc170d91640 1 -- 172.21.15.154:0/201972430 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc16c15e5f0 msgr2=0x7fc16c15ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:48.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.633+0000 7fc170d91640 1 --2- 172.21.15.154:0/201972430 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc16c15e5f0 0x7fc16c15ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:48.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.633+0000 7fc170d91640 1 -- 172.21.15.154:0/201972430 >> v1:172.21.15.154:6789/0 conn(0x7fc16c0caea0 legacy=0x7fc16c0cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:48.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.633+0000 7fc170d91640 1 -- 172.21.15.154:0/201972430 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc16c15b330 con 0x7fc16c15aba0 2023-09-23T03:25:48.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.634+0000 7fc170d91640 1 --2- 172.21.15.154:0/201972430 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc16c15aba0 0x7fc16c15bb30 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fc1600097a0 tx=0x7fc16002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a54b715765485a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:48.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.635+0000 7fc16b7fe640 1 -- 172.21.15.154:0/201972430 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16002d040 con 0x7fc16c15aba0 2023-09-23T03:25:48.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.635+0000 7fc16b7fe640 1 -- 172.21.15.154:0/201972430 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc160031810 con 0x7fc16c15aba0 2023-09-23T03:25:48.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.635+0000 7fc16b7fe640 1 -- 172.21.15.154:0/201972430 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc160031b80 con 0x7fc16c15aba0 2023-09-23T03:25:48.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.635+0000 7fc172ff5640 1 -- 172.21.15.154:0/201972430 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc16c15aba0 msgr2=0x7fc16c15bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:48.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.635+0000 7fc172ff5640 1 --2- 172.21.15.154:0/201972430 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc16c15aba0 0x7fc16c15bb30 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fc1600097a0 tx=0x7fc16002b8b0 comp rx=0 tx=0).stop 2023-09-23T03:25:48.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.635+0000 7fc172ff5640 1 -- 172.21.15.154:0/201972430 shutdown_connections 2023-09-23T03:25:48.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.635+0000 7fc172ff5640 1 --2- 172.21.15.154:0/201972430 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc16c15e5f0 0x7fc16c15ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:48.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.635+0000 7fc172ff5640 1 --2- 172.21.15.154:0/201972430 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc16c15aba0 0x7fc16c15bb30 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:48.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.635+0000 7fc172ff5640 1 -- 172.21.15.154:0/201972430 >> 172.21.15.154:0/201972430 conn(0x7fc16c06c010 msgr2=0x7fc16c06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:48.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.635+0000 7fc172ff5640 1 -- 172.21.15.154:0/201972430 shutdown_connections 2023-09-23T03:25:48.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.635+0000 7fc172ff5640 1 -- 172.21.15.154:0/201972430 wait complete. 2023-09-23T03:25:48.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.637+0000 7fc172ff5640 1 Processor -- start 2023-09-23T03:25:48.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.637+0000 7fc172ff5640 1 -- start start 2023-09-23T03:25:48.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.638+0000 7fc172ff5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc16c0caea0 0x7fc16c0c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:48.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.638+0000 7fc172ff5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc16c15aba0 0x7fc16c1438a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:48.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.638+0000 7fc172ff5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc16c15e5f0 0x7fc16c143de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:48.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.638+0000 7fc172ff5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c150520 con 0x7fc16c0caea0 2023-09-23T03:25:48.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.638+0000 7fc172ff5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c1529f0 con 0x7fc16c15e5f0 2023-09-23T03:25:48.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.638+0000 7fc172ff5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c152710 con 0x7fc16c15aba0 2023-09-23T03:25:48.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.638+0000 7fc170d91640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc16c0caea0 0x7fc16c0c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:48.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.638+0000 7fc170d91640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc16c0caea0 0x7fc16c0c3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48564/0 (socket says 172.21.15.154:48564) 2023-09-23T03:25:48.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.638+0000 7fc170d91640 1 -- 172.21.15.154:0/1145581922 learned_addr learned my addr 172.21.15.154:0/1145581922 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:48.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.639+0000 7fc170d91640 1 -- 172.21.15.154:0/1145581922 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc16c15aba0 msgr2=0x7fc16c1438a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:48.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.639+0000 7fc170d91640 1 --2- 172.21.15.154:0/1145581922 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc16c15aba0 0x7fc16c1438a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:48.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.639+0000 7fc170d91640 1 -- 172.21.15.154:0/1145581922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc16c15e5f0 msgr2=0x7fc16c143de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:48.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.639+0000 7fc170d91640 1 --2- 172.21.15.154:0/1145581922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc16c15e5f0 0x7fc16c143de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:48.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.639+0000 7fc170d91640 1 -- 172.21.15.154:0/1145581922 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc160009400 con 0x7fc16c0caea0 2023-09-23T03:25:48.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.639+0000 7fc170d91640 1 --2- 172.21.15.154:0/1145581922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc16c0caea0 0x7fc16c0c3fd0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fc16002b8f0 tx=0x7fc1600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:48.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.639+0000 7fc1697fa640 1 -- 172.21.15.154:0/1145581922 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc160003820 con 0x7fc16c0caea0 2023-09-23T03:25:48.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.639+0000 7fc1697fa640 1 -- 172.21.15.154:0/1145581922 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16003f070 con 0x7fc16c0caea0 2023-09-23T03:25:48.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.639+0000 7fc172ff5640 1 -- 172.21.15.154:0/1145581922 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc16c15b100 con 0x7fc16c0caea0 2023-09-23T03:25:48.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.639+0000 7fc1697fa640 1 -- 172.21.15.154:0/1145581922 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc160003ba0 con 0x7fc16c0caea0 2023-09-23T03:25:48.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.639+0000 7fc172ff5640 1 -- 172.21.15.154:0/1145581922 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc16c0c6ac0 con 0x7fc16c0caea0 2023-09-23T03:25:48.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.641+0000 7fc172ff5640 1 -- 172.21.15.154:0/1145581922 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc16c0c6640 con 0x7fc16c0caea0 2023-09-23T03:25:48.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.641+0000 7fc1697fa640 1 -- 172.21.15.154:0/1145581922 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fc160003d00 con 0x7fc16c0caea0 2023-09-23T03:25:48.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.642+0000 7fc1697fa640 1 --2- 172.21.15.154:0/1145581922 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc15006af30 0x7fc15006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:48.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.642+0000 7fc1697fa640 1 -- 172.21.15.154:0/1145581922 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc160041070 con 0x7fc16c0caea0 2023-09-23T03:25:48.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.643+0000 7fc16bfff640 1 --2- 172.21.15.154:0/1145581922 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc15006af30 0x7fc15006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:48.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.643+0000 7fc16bfff640 1 --2- 172.21.15.154:0/1145581922 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc15006af30 0x7fc15006d3f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc158006a90 tx=0x7fc158006a20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:48.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.647+0000 7fc1697fa640 1 -- 172.21.15.154:0/1145581922 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fc16c0c6640 con 0x7fc16c0caea0 2023-09-23T03:25:48.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:48.858+0000 7fc172ff5640 1 -- 172.21.15.154:0/1145581922 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc16c0c6640 con 0x7fc16c0caea0 2023-09-23T03:25:49.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.583+0000 7fc1697fa640 1 -- 172.21.15.154:0/1145581922 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v83) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc16c0c6640 con 0x7fc16c0caea0 2023-09-23T03:25:49.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.587+0000 7fc172ff5640 1 -- 172.21.15.154:0/1145581922 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc15006af30 msgr2=0x7fc15006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:49.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.587+0000 7fc172ff5640 1 --2- 172.21.15.154:0/1145581922 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc15006af30 0x7fc15006d3f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc158006a90 tx=0x7fc158006a20 comp rx=0 tx=0).stop 2023-09-23T03:25:49.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.587+0000 7fc172ff5640 1 -- 172.21.15.154:0/1145581922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc16c0caea0 msgr2=0x7fc16c0c3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:49.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.587+0000 7fc172ff5640 1 --2- 172.21.15.154:0/1145581922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc16c0caea0 0x7fc16c0c3fd0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fc16002b8f0 tx=0x7fc1600046c0 comp rx=0 tx=0).stop 2023-09-23T03:25:49.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.587+0000 7fc172ff5640 1 -- 172.21.15.154:0/1145581922 shutdown_connections 2023-09-23T03:25:49.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.587+0000 7fc172ff5640 1 --2- 172.21.15.154:0/1145581922 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc15006af30 0x7fc15006d3f0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:49.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.587+0000 7fc172ff5640 1 --2- 172.21.15.154:0/1145581922 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc16c15e5f0 0x7fc16c143de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:49.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.587+0000 7fc172ff5640 1 --2- 172.21.15.154:0/1145581922 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc16c15aba0 0x7fc16c1438a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:49.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.587+0000 7fc172ff5640 1 --2- 172.21.15.154:0/1145581922 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc16c0caea0 0x7fc16c0c3fd0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:49.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.587+0000 7fc172ff5640 1 -- 172.21.15.154:0/1145581922 >> 172.21.15.154:0/1145581922 conn(0x7fc16c06c010 msgr2=0x7fc16c0701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:49.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.587+0000 7fc172ff5640 1 -- 172.21.15.154:0/1145581922 shutdown_connections 2023-09-23T03:25:49.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.587+0000 7fc172ff5640 1 -- 172.21.15.154:0/1145581922 wait complete. 2023-09-23T03:25:49.607 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-09-23T03:25:49.607 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-09-23T03:25:49.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.831+0000 7fea448a3640 1 Processor -- start 2023-09-23T03:25:49.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.831+0000 7fea448a3640 1 -- start start 2023-09-23T03:25:49.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.832+0000 7fea448a3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea4015caa0 0x7fea400d0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:49.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.832+0000 7fea448a3640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fea400cf2b0 0x7fea400cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:49.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.832+0000 7fea448a3640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea400cb010 con 0x7fea4006bf30 2023-09-23T03:25:49.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.832+0000 7fea448a3640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea4006c490 con 0x7fea400cf2b0 2023-09-23T03:25:49.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.832+0000 7fea448a3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea4006c770 con 0x7fea4015caa0 2023-09-23T03:25:49.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.833+0000 7fea3effd640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fea4006bf30 0x7fea400ceba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:49574/0 (socket says 172.21.15.154:49574) 2023-09-23T03:25:49.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.833+0000 7fea3effd640 1 -- 172.21.15.154:0/3848809912 learned_addr learned my addr 172.21.15.154:0/3848809912 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:25:49.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.833+0000 7fea3ffff640 1 --2- 172.21.15.154:0/3848809912 >> v2:172.21.15.190:3300/0 conn(0x7fea400cf2b0 0x7fea400cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:49.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.833+0000 7fea3f7fe640 1 --2- 172.21.15.154:0/3848809912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea4015caa0 0x7fea400d0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:49.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.833+0000 7fea3e7fc640 1 -- 172.21.15.154:0/3848809912 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 643414035 0 0) 0x7fea400cb010 con 0x7fea4006bf30 2023-09-23T03:25:49.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.833+0000 7fea3e7fc640 1 -- 172.21.15.154:0/3848809912 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea2c003520 con 0x7fea4006bf30 2023-09-23T03:25:49.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.834+0000 7fea3e7fc640 1 -- 172.21.15.154:0/3848809912 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2317805733 0 0) 0x7fea2c003520 con 0x7fea4006bf30 2023-09-23T03:25:49.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.834+0000 7fea3e7fc640 1 -- 172.21.15.154:0/3848809912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea4015caa0 msgr2=0x7fea400d0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:49.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.834+0000 7fea3e7fc640 1 --2- 172.21.15.154:0/3848809912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea4015caa0 0x7fea400d0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:49.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.834+0000 7fea3e7fc640 1 -- 172.21.15.154:0/3848809912 >> v2:172.21.15.190:3300/0 conn(0x7fea400cf2b0 msgr2=0x7fea400cb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:49.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.834+0000 7fea3e7fc640 1 --2- 172.21.15.154:0/3848809912 >> v2:172.21.15.190:3300/0 conn(0x7fea400cf2b0 0x7fea400cb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:49.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.834+0000 7fea3e7fc640 1 -- 172.21.15.154:0/3848809912 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea4006ca50 con 0x7fea4006bf30 2023-09-23T03:25:49.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.834+0000 7fea3e7fc640 1 -- 172.21.15.154:0/3848809912 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fea28001670 con 0x7fea4006bf30 2023-09-23T03:25:49.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.834+0000 7fea3e7fc640 1 -- 172.21.15.154:0/3848809912 >> v1:172.21.15.154:6789/0 conn(0x7fea4006bf30 legacy=0x7fea400ceba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:49.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.834+0000 7fea3e7fc640 1 --2- 172.21.15.154:0/3848809912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea2c003970 0x7fea2c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:49.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.834+0000 7fea3e7fc640 1 --2- 172.21.15.154:0/3848809912 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea2c0047c0 0x7fea2c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:49.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.834+0000 7fea3e7fc640 1 --2- 172.21.15.154:0/3848809912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea4015caa0 0x7fea2c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:49.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.834+0000 7fea3e7fc640 1 -- 172.21.15.154:0/3848809912 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4006c490 con 0x7fea2c0047c0 2023-09-23T03:25:49.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.834+0000 7fea3e7fc640 1 -- 172.21.15.154:0/3848809912 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4006c770 con 0x7fea2c003970 2023-09-23T03:25:49.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.834+0000 7fea3e7fc640 1 -- 172.21.15.154:0/3848809912 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea2c00b980 con 0x7fea4015caa0 2023-09-23T03:25:49.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.834+0000 7fea3effd640 1 --2- 172.21.15.154:0/3848809912 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea2c0047c0 0x7fea2c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:49.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.834+0000 7fea3f7fe640 1 --2- 172.21.15.154:0/3848809912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea2c003970 0x7fea2c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:49.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.835+0000 7fea3ffff640 1 --2- 172.21.15.154:0/3848809912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea4015caa0 0x7fea2c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:49.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.835+0000 7fea3effd640 1 -- 172.21.15.154:0/3848809912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea4015caa0 msgr2=0x7fea2c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:49.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.835+0000 7fea3effd640 1 --2- 172.21.15.154:0/3848809912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea4015caa0 0x7fea2c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:49.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.835+0000 7fea3effd640 1 -- 172.21.15.154:0/3848809912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea2c003970 msgr2=0x7fea2c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:49.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.835+0000 7fea3effd640 1 --2- 172.21.15.154:0/3848809912 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea2c003970 0x7fea2c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:49.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.835+0000 7fea3effd640 1 -- 172.21.15.154:0/3848809912 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea2c003520 con 0x7fea2c0047c0 2023-09-23T03:25:49.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.835+0000 7fea3ffff640 1 --2- 172.21.15.154:0/3848809912 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea4015caa0 0x7fea2c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:49.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.836+0000 7fea3effd640 1 --2- 172.21.15.154:0/3848809912 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea2c0047c0 0x7fea2c004c40 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fea2800c190 tx=0x7fea2c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8c039cec5286d89 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:49.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.836+0000 7fea3e7fc640 1 -- 172.21.15.154:0/3848809912 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea28005ad0 con 0x7fea2c0047c0 2023-09-23T03:25:49.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.836+0000 7fea3e7fc640 1 -- 172.21.15.154:0/3848809912 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea28005c50 con 0x7fea2c0047c0 2023-09-23T03:25:49.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.836+0000 7fea3e7fc640 1 -- 172.21.15.154:0/3848809912 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2802c810 con 0x7fea2c0047c0 2023-09-23T03:25:49.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.837+0000 7fea448a3640 1 -- 172.21.15.154:0/3848809912 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea2c0047c0 msgr2=0x7fea2c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:49.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.837+0000 7fea448a3640 1 --2- 172.21.15.154:0/3848809912 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea2c0047c0 0x7fea2c004c40 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fea2800c190 tx=0x7fea2c0057f0 comp rx=0 tx=0).stop 2023-09-23T03:25:49.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.837+0000 7fea3ffff640 1 -- 172.21.15.154:0/3848809912 reap_dead start 2023-09-23T03:25:49.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.837+0000 7fea448a3640 1 -- 172.21.15.154:0/3848809912 shutdown_connections 2023-09-23T03:25:49.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.837+0000 7fea448a3640 1 -- 172.21.15.154:0/3848809912 >> 172.21.15.154:0/3848809912 conn(0x7fea400998e0 msgr2=0x7fea400d05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:49.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.837+0000 7fea448a3640 1 -- 172.21.15.154:0/3848809912 shutdown_connections 2023-09-23T03:25:49.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.837+0000 7fea448a3640 1 -- 172.21.15.154:0/3848809912 wait complete. 2023-09-23T03:25:49.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.838+0000 7fea448a3640 1 Processor -- start 2023-09-23T03:25:49.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.839+0000 7fea448a3640 1 -- start start 2023-09-23T03:25:49.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.839+0000 7fea448a3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea2c0047c0 0x7fea4009de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:49.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.839+0000 7fea448a3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea40158660 0x7fea400c7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:49.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.840+0000 7fea448a3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea40156340 0x7fea401567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:49.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.840+0000 7fea448a3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea40156d00 con 0x7fea2c0047c0 2023-09-23T03:25:49.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.840+0000 7fea448a3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea401f1fb0 con 0x7fea40158660 2023-09-23T03:25:49.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.840+0000 7fea448a3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea400c7f80 con 0x7fea40156340 2023-09-23T03:25:49.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.840+0000 7fea3f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea2c0047c0 0x7fea4009de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:49.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.840+0000 7fea3f7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea2c0047c0 0x7fea4009de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48574/0 (socket says 172.21.15.154:48574) 2023-09-23T03:25:49.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.840+0000 7fea3f7fe640 1 -- 172.21.15.154:0/767470974 learned_addr learned my addr 172.21.15.154:0/767470974 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:49.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.840+0000 7fea3f7fe640 1 -- 172.21.15.154:0/767470974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea40156340 msgr2=0x7fea401567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:49.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.840+0000 7fea3f7fe640 1 --2- 172.21.15.154:0/767470974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea40156340 0x7fea401567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:49.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.840+0000 7fea3f7fe640 1 -- 172.21.15.154:0/767470974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea40158660 msgr2=0x7fea400c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:49.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.840+0000 7fea3f7fe640 1 --2- 172.21.15.154:0/767470974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea40158660 0x7fea400c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:49.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.840+0000 7fea3f7fe640 1 -- 172.21.15.154:0/767470974 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea400c8260 con 0x7fea2c0047c0 2023-09-23T03:25:49.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.840+0000 7fea3f7fe640 1 --2- 172.21.15.154:0/767470974 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea2c0047c0 0x7fea4009de00 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fea30007f60 tx=0x7fea30007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:49.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.841+0000 7fea37fff640 1 -- 172.21.15.154:0/767470974 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3000e030 con 0x7fea2c0047c0 2023-09-23T03:25:49.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.841+0000 7fea37fff640 1 -- 172.21.15.154:0/767470974 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea30016470 con 0x7fea2c0047c0 2023-09-23T03:25:49.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.841+0000 7fea37fff640 1 -- 172.21.15.154:0/767470974 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea30016820 con 0x7fea2c0047c0 2023-09-23T03:25:49.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.841+0000 7fea448a3640 1 -- 172.21.15.154:0/767470974 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea400c8540 con 0x7fea2c0047c0 2023-09-23T03:25:49.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.841+0000 7fea448a3640 1 -- 172.21.15.154:0/767470974 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea400c8b00 con 0x7fea2c0047c0 2023-09-23T03:25:49.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.842+0000 7fea448a3640 1 -- 172.21.15.154:0/767470974 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea4004e920 con 0x7fea2c0047c0 2023-09-23T03:25:49.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.843+0000 7fea37fff640 1 -- 172.21.15.154:0/767470974 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fea30016980 con 0x7fea2c0047c0 2023-09-23T03:25:49.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.844+0000 7fea37fff640 1 --2- 172.21.15.154:0/767470974 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fea2006af30 0x7fea2006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:49.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.844+0000 7fea37fff640 1 -- 172.21.15.154:0/767470974 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fea3000f070 con 0x7fea2c0047c0 2023-09-23T03:25:49.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.845+0000 7fea3effd640 1 --2- 172.21.15.154:0/767470974 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fea2006af30 0x7fea2006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:49.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.845+0000 7fea3effd640 1 --2- 172.21.15.154:0/767470974 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fea2006af30 0x7fea2006d3f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fea28034000 tx=0x7fea2802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:49.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:49.849+0000 7fea37fff640 1 -- 172.21.15.154:0/767470974 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fea300555e0 con 0x7fea2c0047c0 2023-09-23T03:25:50.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.057+0000 7fea448a3640 1 -- 172.21.15.154:0/767470974 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fea4008a930 con 0x7fea2c0047c0 2023-09-23T03:25:50.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.587+0000 7fea37fff640 1 -- 172.21.15.154:0/767470974 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v84) v1 ==== 122+0+0 (secure 0 0 0) 0x7fea30088e60 con 0x7fea2c0047c0 2023-09-23T03:25:50.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.590+0000 7fea448a3640 1 -- 172.21.15.154:0/767470974 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fea2006af30 msgr2=0x7fea2006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:50.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.590+0000 7fea448a3640 1 --2- 172.21.15.154:0/767470974 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fea2006af30 0x7fea2006d3f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fea28034000 tx=0x7fea2802e000 comp rx=0 tx=0).stop 2023-09-23T03:25:50.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.590+0000 7fea448a3640 1 -- 172.21.15.154:0/767470974 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea2c0047c0 msgr2=0x7fea4009de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:50.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.590+0000 7fea448a3640 1 --2- 172.21.15.154:0/767470974 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea2c0047c0 0x7fea4009de00 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fea30007f60 tx=0x7fea30007f90 comp rx=0 tx=0).stop 2023-09-23T03:25:50.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.590+0000 7fea448a3640 1 -- 172.21.15.154:0/767470974 shutdown_connections 2023-09-23T03:25:50.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.590+0000 7fea448a3640 1 --2- 172.21.15.154:0/767470974 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fea2006af30 0x7fea2006d3f0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:50.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.590+0000 7fea448a3640 1 --2- 172.21.15.154:0/767470974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea40156340 0x7fea401567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:50.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.590+0000 7fea448a3640 1 --2- 172.21.15.154:0/767470974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea40158660 0x7fea400c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:50.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.590+0000 7fea448a3640 1 --2- 172.21.15.154:0/767470974 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea2c0047c0 0x7fea4009de00 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:50.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.590+0000 7fea448a3640 1 -- 172.21.15.154:0/767470974 >> 172.21.15.154:0/767470974 conn(0x7fea400998e0 msgr2=0x7fea400ad240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:50.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.591+0000 7fea448a3640 1 -- 172.21.15.154:0/767470974 shutdown_connections 2023-09-23T03:25:50.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.591+0000 7fea448a3640 1 -- 172.21.15.154:0/767470974 wait complete. 2023-09-23T03:25:50.612 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-09-23T03:25:50.612 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:25:50.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.825+0000 7f5df0f39640 1 Processor -- start 2023-09-23T03:25:50.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.825+0000 7f5df0f39640 1 -- start start 2023-09-23T03:25:50.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.826+0000 7f5df0f39640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5dec0b2fe0 0x7f5dec0ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:50.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.826+0000 7f5df0f39640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dec06ceb0 con 0x7f5dec0ada30 2023-09-23T03:25:50.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.826+0000 7f5df0f39640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dec159c90 con 0x7f5dec0cf130 2023-09-23T03:25:50.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.826+0000 7f5df0f39640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dec0ade10 con 0x7f5dec0b2fe0 2023-09-23T03:25:50.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.826+0000 7f5dea59c640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f5dec0ada30 0x7f5dec0cea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:49582/0 (socket says 172.21.15.154:49582) 2023-09-23T03:25:50.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.826+0000 7f5dea59c640 1 -- 172.21.15.154:0/702104052 learned_addr learned my addr 172.21.15.154:0/702104052 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:25:50.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.827+0000 7f5de959a640 1 -- 172.21.15.154:0/702104052 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4192782876 0 0) 0x7f5ddc001290 con 0x7f5dec0ada30 2023-09-23T03:25:50.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.827+0000 7f5dead9d640 1 --2- 172.21.15.154:0/702104052 >> v2:172.21.15.190:3300/0 conn(0x7f5dec0b2fe0 0x7f5dec0ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:50.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.827+0000 7f5de959a640 1 -- 172.21.15.154:0/702104052 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5dd8003520 con 0x7f5dec0ada30 2023-09-23T03:25:50.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.827+0000 7f5de959a640 1 -- 172.21.15.154:0/702104052 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2527372692 0 0) 0x7f5dd8003520 con 0x7f5dec0ada30 2023-09-23T03:25:50.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.827+0000 7f5de959a640 1 -- 172.21.15.154:0/702104052 >> v2:172.21.15.190:3300/0 conn(0x7f5dec0b2fe0 msgr2=0x7f5dec0ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:50.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.827+0000 7f5de959a640 1 --2- 172.21.15.154:0/702104052 >> v2:172.21.15.190:3300/0 conn(0x7f5dec0b2fe0 0x7f5dec0ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:50.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.827+0000 7f5de959a640 1 -- 172.21.15.154:0/702104052 >> v1:172.21.15.190:6789/0 conn(0x7f5dec0cf130 legacy=0x7f5dec15d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:50.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.827+0000 7f5de959a640 1 -- 172.21.15.154:0/702104052 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dec06c110 con 0x7f5dec0ada30 2023-09-23T03:25:50.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.827+0000 7f5de959a640 1 -- 172.21.15.154:0/702104052 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f5ddc001980 con 0x7f5dec0ada30 2023-09-23T03:25:50.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.827+0000 7f5de959a640 1 -- 172.21.15.154:0/702104052 >> v1:172.21.15.154:6789/0 conn(0x7f5dec0ada30 legacy=0x7f5dec0cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:50.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.827+0000 7f5de959a640 1 --2- 172.21.15.154:0/702104052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5dd8003970 0x7f5dd8003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:50.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.827+0000 7f5de959a640 1 --2- 172.21.15.154:0/702104052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dd80047c0 0x7f5dd8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:50.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.827+0000 7f5de959a640 1 --2- 172.21.15.154:0/702104052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5dd8005180 0x7f5dd800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:50.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.827+0000 7f5de959a640 1 -- 172.21.15.154:0/702104052 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec0ade10 con 0x7f5dd80047c0 2023-09-23T03:25:50.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.827+0000 7f5de959a640 1 -- 172.21.15.154:0/702104052 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd800c0b0 con 0x7f5dd8003970 2023-09-23T03:25:50.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.827+0000 7f5de959a640 1 -- 172.21.15.154:0/702104052 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd800c270 con 0x7f5dd8005180 2023-09-23T03:25:50.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.828+0000 7f5de9d9b640 1 --2- 172.21.15.154:0/702104052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dd80047c0 0x7f5dd8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:50.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.828+0000 7f5dea59c640 1 --2- 172.21.15.154:0/702104052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5dd8003970 0x7f5dd8003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:50.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.828+0000 7f5dead9d640 1 --2- 172.21.15.154:0/702104052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5dd8005180 0x7f5dd800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:50.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.828+0000 7f5de9d9b640 1 -- 172.21.15.154:0/702104052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5dd8005180 msgr2=0x7f5dd800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:50.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.828+0000 7f5de9d9b640 1 --2- 172.21.15.154:0/702104052 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5dd8005180 0x7f5dd800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:50.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.828+0000 7f5de9d9b640 1 -- 172.21.15.154:0/702104052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5dd8003970 msgr2=0x7f5dd8003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:50.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.828+0000 7f5de9d9b640 1 --2- 172.21.15.154:0/702104052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5dd8003970 0x7f5dd8003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:50.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.828+0000 7f5de9d9b640 1 -- 172.21.15.154:0/702104052 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd8003520 con 0x7f5dd80047c0 2023-09-23T03:25:50.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.828+0000 7f5dea59c640 1 --2- 172.21.15.154:0/702104052 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5dd8003970 0x7f5dd8003d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:50.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.828+0000 7f5dead9d640 1 -- 172.21.15.154:0/702104052 reap_dead start 2023-09-23T03:25:50.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.829+0000 7f5de9d9b640 1 --2- 172.21.15.154:0/702104052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dd80047c0 0x7f5dd8004c40 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f5dec0ca3c0 tx=0x7f5dd80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80594197c65fcb7d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:50.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.829+0000 7f5de959a640 1 -- 172.21.15.154:0/702104052 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4032600 con 0x7f5dd80047c0 2023-09-23T03:25:50.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.830+0000 7f5de959a640 1 -- 172.21.15.154:0/702104052 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd402e050 con 0x7f5dd80047c0 2023-09-23T03:25:50.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.830+0000 7f5de959a640 1 -- 172.21.15.154:0/702104052 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd40329c0 con 0x7f5dd80047c0 2023-09-23T03:25:50.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.830+0000 7f5df0f39640 1 -- 172.21.15.154:0/702104052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dd80047c0 msgr2=0x7f5dd8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:50.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.830+0000 7f5df0f39640 1 --2- 172.21.15.154:0/702104052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dd80047c0 0x7f5dd8004c40 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f5dec0ca3c0 tx=0x7f5dd80057f0 comp rx=0 tx=0).stop 2023-09-23T03:25:50.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.830+0000 7f5df0f39640 1 -- 172.21.15.154:0/702104052 shutdown_connections 2023-09-23T03:25:50.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.830+0000 7f5df0f39640 1 --2- 172.21.15.154:0/702104052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dd80047c0 0x7f5dd8004c40 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:50.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.830+0000 7f5df0f39640 1 -- 172.21.15.154:0/702104052 >> 172.21.15.154:0/702104052 conn(0x7f5dec06a0f0 msgr2=0x7f5dec06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:50.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.830+0000 7f5df0f39640 1 -- 172.21.15.154:0/702104052 shutdown_connections 2023-09-23T03:25:50.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.830+0000 7f5df0f39640 1 -- 172.21.15.154:0/702104052 wait complete. 2023-09-23T03:25:50.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.832+0000 7f5df0f39640 1 Processor -- start 2023-09-23T03:25:50.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.832+0000 7f5df0f39640 1 -- start start 2023-09-23T03:25:50.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.833+0000 7f5df0f39640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5dd80047c0 0x7f5dec168f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:50.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.833+0000 7f5df0f39640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5dec0b0580 0x7f5dec1675f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:50.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.833+0000 7f5df0f39640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dec097660 0x7f5dec167b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:50.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.833+0000 7f5df0f39640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec096f90 con 0x7f5dec097660 2023-09-23T03:25:50.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.833+0000 7f5df0f39640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec0af670 con 0x7f5dd80047c0 2023-09-23T03:25:50.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.833+0000 7f5df0f39640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dec0af930 con 0x7f5dec0b0580 2023-09-23T03:25:50.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.833+0000 7f5dead9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dec097660 0x7f5dec167b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:50.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.833+0000 7f5dead9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dec097660 0x7f5dec167b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48584/0 (socket says 172.21.15.154:48584) 2023-09-23T03:25:50.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.833+0000 7f5dead9d640 1 -- 172.21.15.154:0/3442478954 learned_addr learned my addr 172.21.15.154:0/3442478954 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:50.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.834+0000 7f5dead9d640 1 -- 172.21.15.154:0/3442478954 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5dec0b0580 msgr2=0x7f5dec1675f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:50.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.834+0000 7f5dead9d640 1 --2- 172.21.15.154:0/3442478954 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5dec0b0580 0x7f5dec1675f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:50.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.834+0000 7f5dead9d640 1 -- 172.21.15.154:0/3442478954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5dd80047c0 msgr2=0x7f5dec168f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:50.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.834+0000 7f5dea59c640 1 --2- 172.21.15.154:0/3442478954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5dd80047c0 0x7f5dec168f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:50.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.834+0000 7f5dead9d640 1 --2- 172.21.15.154:0/3442478954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5dd80047c0 0x7f5dec168f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:50.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.834+0000 7f5dead9d640 1 -- 172.21.15.154:0/3442478954 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dec096bb0 con 0x7f5dec097660 2023-09-23T03:25:50.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.834+0000 7f5dead9d640 1 --2- 172.21.15.154:0/3442478954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dec097660 0x7f5dec167b30 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f5dec0b10a0 tx=0x7f5de4009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:50.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.834+0000 7f5de2ffd640 1 -- 172.21.15.154:0/3442478954 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de4007db0 con 0x7f5dec097660 2023-09-23T03:25:50.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.834+0000 7f5de2ffd640 1 -- 172.21.15.154:0/3442478954 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de4014400 con 0x7f5dec097660 2023-09-23T03:25:50.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.834+0000 7f5de2ffd640 1 -- 172.21.15.154:0/3442478954 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de40147b0 con 0x7f5dec097660 2023-09-23T03:25:50.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.834+0000 7f5df0f39640 1 -- 172.21.15.154:0/3442478954 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dec0973b0 con 0x7f5dec097660 2023-09-23T03:25:50.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.834+0000 7f5df0f39640 1 -- 172.21.15.154:0/3442478954 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dec0cc1d0 con 0x7f5dec097660 2023-09-23T03:25:50.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.836+0000 7f5df0f39640 1 -- 172.21.15.154:0/3442478954 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dec0970d0 con 0x7f5dec097660 2023-09-23T03:25:50.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.836+0000 7f5de2ffd640 1 -- 172.21.15.154:0/3442478954 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f5de401d3b0 con 0x7f5dec097660 2023-09-23T03:25:50.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.837+0000 7f5de2ffd640 1 --2- 172.21.15.154:0/3442478954 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5dcc06af30 0x7f5dcc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:50.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.838+0000 7f5de2ffd640 1 -- 172.21.15.154:0/3442478954 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5de4089730 con 0x7f5dec097660 2023-09-23T03:25:50.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.838+0000 7f5dea59c640 1 --2- 172.21.15.154:0/3442478954 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5dcc06af30 0x7f5dcc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:50.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.838+0000 7f5dea59c640 1 --2- 172.21.15.154:0/3442478954 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5dcc06af30 0x7f5dcc06d3f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc000ef0 tx=0x7f5ddc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:50.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:50.841+0000 7f5de2ffd640 1 -- 172.21.15.154:0/3442478954 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f5dec0970d0 con 0x7f5dec097660 2023-09-23T03:25:51.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.067+0000 7f5df0f39640 1 -- 172.21.15.154:0/3442478954 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5dec0970d0 con 0x7f5dec097660 2023-09-23T03:25:51.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.068+0000 7f5de2ffd640 1 -- 172.21.15.154:0/3442478954 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f5dec0970d0 con 0x7f5dec097660 2023-09-23T03:25:51.068 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:25:51.069 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":84,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5603,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/4291187069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":4291187069},{"type":"v1","addr":"172.21.15.190:6834","nonce":4291187069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":8097,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/1240348396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1240348396},{"type":"v1","addr":"172.21.15.154:6836","nonce":1240348396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":8100,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/4233181417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":4233181417},{"type":"v1","addr":"172.21.15.154:6837","nonce":4233181417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:25:46.997893+0000","modified":"2023-09-23T03:25:50.058760+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8115":{"gid":8115,"name":"d","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/2967125458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2967125458},{"type":"v1","addr":"172.21.15.190:6835","nonce":2967125458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-09-23T03:25:51.069 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 84 2023-09-23T03:25:51.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.070+0000 7f5df0f39640 1 -- 172.21.15.154:0/3442478954 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5dcc06af30 msgr2=0x7f5dcc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:51.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.070+0000 7f5df0f39640 1 --2- 172.21.15.154:0/3442478954 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5dcc06af30 0x7f5dcc06d3f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc000ef0 tx=0x7f5ddc009040 comp rx=0 tx=0).stop 2023-09-23T03:25:51.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.070+0000 7f5df0f39640 1 -- 172.21.15.154:0/3442478954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dec097660 msgr2=0x7f5dec167b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:51.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.070+0000 7f5df0f39640 1 --2- 172.21.15.154:0/3442478954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dec097660 0x7f5dec167b30 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f5dec0b10a0 tx=0x7f5de4009dc0 comp rx=0 tx=0).stop 2023-09-23T03:25:51.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.071+0000 7f5df0f39640 1 -- 172.21.15.154:0/3442478954 shutdown_connections 2023-09-23T03:25:51.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.071+0000 7f5df0f39640 1 --2- 172.21.15.154:0/3442478954 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5dcc06af30 0x7f5dcc06d3f0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:51.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.071+0000 7f5df0f39640 1 --2- 172.21.15.154:0/3442478954 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5dec097660 0x7f5dec167b30 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:51.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.071+0000 7f5df0f39640 1 --2- 172.21.15.154:0/3442478954 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5dec0b0580 0x7f5dec1675f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:51.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.071+0000 7f5df0f39640 1 --2- 172.21.15.154:0/3442478954 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5dd80047c0 0x7f5dec168f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:51.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.071+0000 7f5df0f39640 1 -- 172.21.15.154:0/3442478954 >> 172.21.15.154:0/3442478954 conn(0x7f5dec06a0f0 msgr2=0x7f5dec0cb8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:51.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.071+0000 7f5df0f39640 1 -- 172.21.15.154:0/3442478954 shutdown_connections 2023-09-23T03:25:51.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.071+0000 7f5df0f39640 1 -- 172.21.15.154:0/3442478954 wait complete. 2023-09-23T03:25:51.089 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:25:51.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.305+0000 7fe122e07640 1 Processor -- start 2023-09-23T03:25:51.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.305+0000 7fe122e07640 1 -- start start 2023-09-23T03:25:51.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.307+0000 7fe122e07640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe11c159020 0x7fe11c06d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:51.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.307+0000 7fe122e07640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe11c0cac90 0x7fe11c0cb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:51.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.307+0000 7fe122e07640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe11c159400 con 0x7fe11c0cc0e0 2023-09-23T03:25:51.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.307+0000 7fe122e07640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe11c0ada00 con 0x7fe11c0cac90 2023-09-23T03:25:51.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.307+0000 7fe122e07640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe11c0cb660 con 0x7fe11c159020 2023-09-23T03:25:51.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.307+0000 7fe113fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fe11c0cc0e0 0x7fe11c0b1360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:49598/0 (socket says 172.21.15.154:49598) 2023-09-23T03:25:51.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.307+0000 7fe113fff640 1 -- 172.21.15.154:0/155785598 learned_addr learned my addr 172.21.15.154:0/155785598 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:25:51.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.307+0000 7fe1213a4640 1 --2- 172.21.15.154:0/155785598 >> v2:172.21.15.154:3300/0 conn(0x7fe11c0cac90 0x7fe11c0cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:51.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.307+0000 7fe120ba3640 1 --2- 172.21.15.154:0/155785598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe11c159020 0x7fe11c06d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:51.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.307+0000 7fe1137fe640 1 -- 172.21.15.154:0/155785598 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1492486127 0 0) 0x7fe11c159400 con 0x7fe11c0cc0e0 2023-09-23T03:25:51.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.308+0000 7fe1137fe640 1 -- 172.21.15.154:0/155785598 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe104003520 con 0x7fe11c0cc0e0 2023-09-23T03:25:51.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.308+0000 7fe1213a4640 1 -- 172.21.15.154:0/155785598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe11c159020 msgr2=0x7fe11c06d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:51.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.308+0000 7fe1213a4640 1 --2- 172.21.15.154:0/155785598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe11c159020 0x7fe11c06d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:51.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.308+0000 7fe1213a4640 1 -- 172.21.15.154:0/155785598 >> v1:172.21.15.154:6789/0 conn(0x7fe11c0cc0e0 legacy=0x7fe11c0b1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:51.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.308+0000 7fe1213a4640 1 -- 172.21.15.154:0/155785598 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe11c0cdaf0 con 0x7fe11c0cac90 2023-09-23T03:25:51.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.309+0000 7fe1213a4640 1 --2- 172.21.15.154:0/155785598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe11c0cac90 0x7fe11c0cb120 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fe114009d90 tx=0x7fe11402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23ed87d8ef58064 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:51.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.309+0000 7fe1137fe640 1 -- 172.21.15.154:0/155785598 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11402cc50 con 0x7fe11c0cac90 2023-09-23T03:25:51.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.309+0000 7fe1137fe640 1 -- 172.21.15.154:0/155785598 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe11402cdb0 con 0x7fe11c0cac90 2023-09-23T03:25:51.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.309+0000 7fe1137fe640 1 -- 172.21.15.154:0/155785598 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe114034660 con 0x7fe11c0cac90 2023-09-23T03:25:51.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.309+0000 7fe122e07640 1 -- 172.21.15.154:0/155785598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe11c0cac90 msgr2=0x7fe11c0cb120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:51.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.309+0000 7fe122e07640 1 --2- 172.21.15.154:0/155785598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe11c0cac90 0x7fe11c0cb120 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fe114009d90 tx=0x7fe11402bea0 comp rx=0 tx=0).stop 2023-09-23T03:25:51.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.310+0000 7fe122e07640 1 -- 172.21.15.154:0/155785598 shutdown_connections 2023-09-23T03:25:51.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.310+0000 7fe122e07640 1 --2- 172.21.15.154:0/155785598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe11c0cac90 0x7fe11c0cb120 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:51.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.310+0000 7fe122e07640 1 --2- 172.21.15.154:0/155785598 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe11c159020 0x7fe11c06d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:51.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.310+0000 7fe122e07640 1 -- 172.21.15.154:0/155785598 >> 172.21.15.154:0/155785598 conn(0x7fe11c06a0f0 msgr2=0x7fe11c06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:51.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.310+0000 7fe122e07640 1 -- 172.21.15.154:0/155785598 shutdown_connections 2023-09-23T03:25:51.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.310+0000 7fe122e07640 1 -- 172.21.15.154:0/155785598 wait complete. 2023-09-23T03:25:51.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.311+0000 7fe122e07640 1 Processor -- start 2023-09-23T03:25:51.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.312+0000 7fe122e07640 1 -- start start 2023-09-23T03:25:51.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.312+0000 7fe122e07640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe11c0cac90 0x7fe11c1584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:51.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.312+0000 7fe122e07640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe11c0cc0e0 0x7fe11c0c2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:51.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.312+0000 7fe122e07640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe11c159020 0x7fe11c1595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:51.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.312+0000 7fe122e07640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11c0c3770 con 0x7fe11c0cac90 2023-09-23T03:25:51.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.312+0000 7fe122e07640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11c1f1ea0 con 0x7fe11c159020 2023-09-23T03:25:51.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.312+0000 7fe122e07640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe11c1f2460 con 0x7fe11c0cc0e0 2023-09-23T03:25:51.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.313+0000 7fe120ba3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe11c0cac90 0x7fe11c1584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:51.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.313+0000 7fe120ba3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe11c0cac90 0x7fe11c1584d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48598/0 (socket says 172.21.15.154:48598) 2023-09-23T03:25:51.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.313+0000 7fe120ba3640 1 -- 172.21.15.154:0/2560035262 learned_addr learned my addr 172.21.15.154:0/2560035262 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:51.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.313+0000 7fe113fff640 1 --2- 172.21.15.154:0/2560035262 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe11c0cc0e0 0x7fe11c0c2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:51.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.313+0000 7fe120ba3640 1 -- 172.21.15.154:0/2560035262 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe11c0cc0e0 msgr2=0x7fe11c0c2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:51.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.313+0000 7fe120ba3640 1 --2- 172.21.15.154:0/2560035262 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe11c0cc0e0 0x7fe11c0c2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:51.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.313+0000 7fe1213a4640 1 --2- 172.21.15.154:0/2560035262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe11c159020 0x7fe11c1595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:51.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.313+0000 7fe120ba3640 1 -- 172.21.15.154:0/2560035262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe11c159020 msgr2=0x7fe11c1595c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:51.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.313+0000 7fe120ba3640 1 --2- 172.21.15.154:0/2560035262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe11c159020 0x7fe11c1595c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:51.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.313+0000 7fe120ba3640 1 -- 172.21.15.154:0/2560035262 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe114009a70 con 0x7fe11c0cac90 2023-09-23T03:25:51.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.313+0000 7fe120ba3640 1 --2- 172.21.15.154:0/2560035262 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe11c0cac90 0x7fe11c1584d0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fe11800c7f0 tx=0x7fe11800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:51.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.314+0000 7fe1117fa640 1 -- 172.21.15.154:0/2560035262 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe118004220 con 0x7fe11c0cac90 2023-09-23T03:25:51.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.314+0000 7fe1117fa640 1 -- 172.21.15.154:0/2560035262 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe118004380 con 0x7fe11c0cac90 2023-09-23T03:25:51.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.314+0000 7fe1117fa640 1 -- 172.21.15.154:0/2560035262 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe118007840 con 0x7fe11c0cac90 2023-09-23T03:25:51.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.314+0000 7fe122e07640 1 -- 172.21.15.154:0/2560035262 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe11c0c31b0 con 0x7fe11c0cac90 2023-09-23T03:25:51.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.314+0000 7fe122e07640 1 -- 172.21.15.154:0/2560035262 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe11c1f20f0 con 0x7fe11c0cac90 2023-09-23T03:25:51.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.315+0000 7fe122e07640 1 -- 172.21.15.154:0/2560035262 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe11c073eb0 con 0x7fe11c0cac90 2023-09-23T03:25:51.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.316+0000 7fe1117fa640 1 -- 172.21.15.154:0/2560035262 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fe118007bb0 con 0x7fe11c0cac90 2023-09-23T03:25:51.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.317+0000 7fe1117fa640 1 --2- 172.21.15.154:0/2560035262 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe10806af60 0x7fe10806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:51.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.317+0000 7fe1117fa640 1 -- 172.21.15.154:0/2560035262 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe1180897f0 con 0x7fe11c0cac90 2023-09-23T03:25:51.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.317+0000 7fe113fff640 1 --2- 172.21.15.154:0/2560035262 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe10806af60 0x7fe10806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:51.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.318+0000 7fe113fff640 1 --2- 172.21.15.154:0/2560035262 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe10806af60 0x7fe10806d420 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe11c072f00 tx=0x7fe10c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:51.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.321+0000 7fe1117fa640 1 -- 172.21.15.154:0/2560035262 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fe118054650 con 0x7fe11c0cac90 2023-09-23T03:25:51.494 INFO:tasks.ceph.mds.d.smithi190.stderr:2023-09-23T03:25:51.493+0000 7f0a0868a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-23T03:25:51.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.515+0000 7fe122e07640 1 -- 172.21.15.154:0/2560035262 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe11c1f2280 con 0x7fe11c0cac90 2023-09-23T03:25:51.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.517+0000 7fe1117fa640 1 -- 172.21.15.154:0/2560035262 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+16760 (secure 0 0 0) 0x7fe118007e60 con 0x7fe11c0cac90 2023-09-23T03:25:51.517 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:25:51.517 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":105,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:25:48.533206+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:25:44.763842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-09-23T03:25:46.025926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:25:51.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.519+0000 7fe122e07640 1 -- 172.21.15.154:0/2560035262 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe10806af60 msgr2=0x7fe10806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:51.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.519+0000 7fe122e07640 1 --2- 172.21.15.154:0/2560035262 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe10806af60 0x7fe10806d420 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe11c072f00 tx=0x7fe10c009040 comp rx=0 tx=0).stop 2023-09-23T03:25:51.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.519+0000 7fe122e07640 1 -- 172.21.15.154:0/2560035262 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe11c0cac90 msgr2=0x7fe11c1584d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:51.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.519+0000 7fe122e07640 1 --2- 172.21.15.154:0/2560035262 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe11c0cac90 0x7fe11c1584d0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fe11800c7f0 tx=0x7fe11800ccb0 comp rx=0 tx=0).stop 2023-09-23T03:25:51.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.519+0000 7fe122e07640 1 -- 172.21.15.154:0/2560035262 shutdown_connections 2023-09-23T03:25:51.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.519+0000 7fe122e07640 1 --2- 172.21.15.154:0/2560035262 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe10806af60 0x7fe10806d420 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:51.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.519+0000 7fe122e07640 1 --2- 172.21.15.154:0/2560035262 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe11c159020 0x7fe11c1595c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:51.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.519+0000 7fe122e07640 1 --2- 172.21.15.154:0/2560035262 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe11c0cc0e0 0x7fe11c0c2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:51.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.519+0000 7fe122e07640 1 --2- 172.21.15.154:0/2560035262 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe11c0cac90 0x7fe11c1584d0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:51.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.519+0000 7fe122e07640 1 -- 172.21.15.154:0/2560035262 >> 172.21.15.154:0/2560035262 conn(0x7fe11c06a0f0 msgr2=0x7fe11c06a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:51.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.520+0000 7fe122e07640 1 -- 172.21.15.154:0/2560035262 shutdown_connections 2023-09-23T03:25:51.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.520+0000 7fe122e07640 1 -- 172.21.15.154:0/2560035262 wait complete. 2023-09-23T03:25:51.537 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-09-23T03:25:51.537 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-23T03:25:51.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.750+0000 7fb35b7d0640 1 Processor -- start 2023-09-23T03:25:51.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.750+0000 7fb35b7d0640 1 -- start start 2023-09-23T03:25:51.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.751+0000 7fb35b7d0640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb3540cf130 0x7fb3540ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:51.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.751+0000 7fb35b7d0640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb35406ceb0 con 0x7fb3540ada30 2023-09-23T03:25:51.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.751+0000 7fb35b7d0640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb354159c90 con 0x7fb3540b2fe0 2023-09-23T03:25:51.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.751+0000 7fb35b7d0640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3540ade10 con 0x7fb3540cf130 2023-09-23T03:25:51.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.752+0000 7fb35956c640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fb3540ada30 0x7fb3540cea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:49614/0 (socket says 172.21.15.154:49614) 2023-09-23T03:25:51.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.752+0000 7fb35956c640 1 -- 172.21.15.154:0/2151353425 learned_addr learned my addr 172.21.15.154:0/2151353425 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:25:51.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.752+0000 7fb358d6b640 1 --2- 172.21.15.154:0/2151353425 >> v2:172.21.15.190:3300/0 conn(0x7fb3540cf130 0x7fb3540ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:51.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.752+0000 7fb34bfff640 1 -- 172.21.15.154:0/2151353425 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3894584113 0 0) 0x7fb350001290 con 0x7fb3540ada30 2023-09-23T03:25:51.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.752+0000 7fb34bfff640 1 -- 172.21.15.154:0/2151353425 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb340003520 con 0x7fb3540ada30 2023-09-23T03:25:51.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.752+0000 7fb34bfff640 1 -- 172.21.15.154:0/2151353425 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2893957048 0 0) 0x7fb340003520 con 0x7fb3540ada30 2023-09-23T03:25:51.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.752+0000 7fb34bfff640 1 -- 172.21.15.154:0/2151353425 >> v2:172.21.15.190:3300/0 conn(0x7fb3540cf130 msgr2=0x7fb3540ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:51.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.752+0000 7fb34bfff640 1 --2- 172.21.15.154:0/2151353425 >> v2:172.21.15.190:3300/0 conn(0x7fb3540cf130 0x7fb3540ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:51.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.753+0000 7fb34bfff640 1 -- 172.21.15.154:0/2151353425 >> v1:172.21.15.190:6789/0 conn(0x7fb3540b2fe0 legacy=0x7fb35415d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:51.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.753+0000 7fb34bfff640 1 -- 172.21.15.154:0/2151353425 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb35406c110 con 0x7fb3540ada30 2023-09-23T03:25:51.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.753+0000 7fb34bfff640 1 -- 172.21.15.154:0/2151353425 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fb350001980 con 0x7fb3540ada30 2023-09-23T03:25:51.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.753+0000 7fb34bfff640 1 -- 172.21.15.154:0/2151353425 >> v1:172.21.15.154:6789/0 conn(0x7fb3540ada30 legacy=0x7fb3540cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:51.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.753+0000 7fb34bfff640 1 --2- 172.21.15.154:0/2151353425 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb340003970 0x7fb340003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:51.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.753+0000 7fb34bfff640 1 --2- 172.21.15.154:0/2151353425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3400047c0 0x7fb340004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:51.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.753+0000 7fb34bfff640 1 --2- 172.21.15.154:0/2151353425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb340005180 0x7fb34000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:51.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.753+0000 7fb34bfff640 1 -- 172.21.15.154:0/2151353425 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3540ade10 con 0x7fb340005180 2023-09-23T03:25:51.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.753+0000 7fb34bfff640 1 -- 172.21.15.154:0/2151353425 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34000e210 con 0x7fb3400047c0 2023-09-23T03:25:51.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.753+0000 7fb34bfff640 1 -- 172.21.15.154:0/2151353425 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb34000e3d0 con 0x7fb340003970 2023-09-23T03:25:51.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.753+0000 7fb359d6d640 1 --2- 172.21.15.154:0/2151353425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb340005180 0x7fb34000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:51.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.753+0000 7fb35956c640 1 --2- 172.21.15.154:0/2151353425 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb340003970 0x7fb340003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:51.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.753+0000 7fb358d6b640 1 --2- 172.21.15.154:0/2151353425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3400047c0 0x7fb340004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:51.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.754+0000 7fb359d6d640 1 -- 172.21.15.154:0/2151353425 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb340003970 msgr2=0x7fb340003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:51.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.754+0000 7fb359d6d640 1 --2- 172.21.15.154:0/2151353425 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb340003970 0x7fb340003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:51.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.754+0000 7fb359d6d640 1 -- 172.21.15.154:0/2151353425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3400047c0 msgr2=0x7fb340004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:51.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.754+0000 7fb359d6d640 1 --2- 172.21.15.154:0/2151353425 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3400047c0 0x7fb340004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:51.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.754+0000 7fb359d6d640 1 -- 172.21.15.154:0/2151353425 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb340003520 con 0x7fb340005180 2023-09-23T03:25:51.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.755+0000 7fb359d6d640 1 -- 172.21.15.154:0/2151353425 reap_dead start 2023-09-23T03:25:51.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.755+0000 7fb359d6d640 1 --2- 172.21.15.154:0/2151353425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb340005180 0x7fb34000db90 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fb34c004480 tx=0x7fb3400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e785a56750494464 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:51.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.755+0000 7fb34bfff640 1 -- 172.21.15.154:0/2151353425 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34c006b50 con 0x7fb340005180 2023-09-23T03:25:51.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.755+0000 7fb34bfff640 1 -- 172.21.15.154:0/2151353425 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb34c006cd0 con 0x7fb340005180 2023-09-23T03:25:51.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.755+0000 7fb34bfff640 1 -- 172.21.15.154:0/2151353425 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34c007260 con 0x7fb340005180 2023-09-23T03:25:51.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.756+0000 7fb35b7d0640 1 -- 172.21.15.154:0/2151353425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb340005180 msgr2=0x7fb34000db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:51.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.756+0000 7fb35b7d0640 1 --2- 172.21.15.154:0/2151353425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb340005180 0x7fb34000db90 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fb34c004480 tx=0x7fb3400057f0 comp rx=0 tx=0).stop 2023-09-23T03:25:51.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.756+0000 7fb35b7d0640 1 -- 172.21.15.154:0/2151353425 shutdown_connections 2023-09-23T03:25:51.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.756+0000 7fb35b7d0640 1 --2- 172.21.15.154:0/2151353425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb340005180 0x7fb34000db90 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:51.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.756+0000 7fb35b7d0640 1 -- 172.21.15.154:0/2151353425 >> 172.21.15.154:0/2151353425 conn(0x7fb35406a0f0 msgr2=0x7fb35406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:51.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.756+0000 7fb35b7d0640 1 -- 172.21.15.154:0/2151353425 shutdown_connections 2023-09-23T03:25:51.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.756+0000 7fb35b7d0640 1 -- 172.21.15.154:0/2151353425 wait complete. 2023-09-23T03:25:51.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.758+0000 7fb35b7d0640 1 Processor -- start 2023-09-23T03:25:51.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.758+0000 7fb35b7d0640 1 -- start start 2023-09-23T03:25:51.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.759+0000 7fb35b7d0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb340005180 0x7fb3541590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:51.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.759+0000 7fb35b7d0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb35409b400 0x7fb35415c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:51.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.759+0000 7fb35b7d0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3540982c0 0x7fb354098740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:51.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.759+0000 7fb35b7d0640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35409afe0 con 0x7fb340005180 2023-09-23T03:25:51.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.759+0000 7fb35b7d0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb354098c80 con 0x7fb3540982c0 2023-09-23T03:25:51.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.759+0000 7fb35b7d0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb35409a130 con 0x7fb35409b400 2023-09-23T03:25:51.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.759+0000 7fb35956c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb340005180 0x7fb3541590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:51.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.759+0000 7fb35956c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb340005180 0x7fb3541590b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48620/0 (socket says 172.21.15.154:48620) 2023-09-23T03:25:51.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.759+0000 7fb35956c640 1 -- 172.21.15.154:0/3260810938 learned_addr learned my addr 172.21.15.154:0/3260810938 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:51.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.759+0000 7fb359d6d640 1 --2- 172.21.15.154:0/3260810938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3540982c0 0x7fb354098740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:51.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.759+0000 7fb358d6b640 1 --2- 172.21.15.154:0/3260810938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb35409b400 0x7fb35415c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:51.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.759+0000 7fb35956c640 1 -- 172.21.15.154:0/3260810938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb35409b400 msgr2=0x7fb35415c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:51.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.759+0000 7fb35956c640 1 --2- 172.21.15.154:0/3260810938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb35409b400 0x7fb35415c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:51.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.759+0000 7fb35956c640 1 -- 172.21.15.154:0/3260810938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3540982c0 msgr2=0x7fb354098740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:51.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.759+0000 7fb35956c640 1 --2- 172.21.15.154:0/3260810938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3540982c0 0x7fb354098740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:51.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.759+0000 7fb35956c640 1 -- 172.21.15.154:0/3260810938 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb35409a360 con 0x7fb340005180 2023-09-23T03:25:51.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.759+0000 7fb358d6b640 1 --2- 172.21.15.154:0/3260810938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb35409b400 0x7fb35415c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:51.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.759+0000 7fb359d6d640 1 --2- 172.21.15.154:0/3260810938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3540982c0 0x7fb354098740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:51.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.760+0000 7fb35956c640 1 --2- 172.21.15.154:0/3260810938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb340005180 0x7fb3541590b0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fb35000e860 tx=0x7fb35000ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:51.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.760+0000 7fb349ffb640 1 -- 172.21.15.154:0/3260810938 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb350001910 con 0x7fb340005180 2023-09-23T03:25:51.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.760+0000 7fb349ffb640 1 -- 172.21.15.154:0/3260810938 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3500072e0 con 0x7fb340005180 2023-09-23T03:25:51.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.760+0000 7fb35b7d0640 1 -- 172.21.15.154:0/3260810938 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb35409a640 con 0x7fb340005180 2023-09-23T03:25:51.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.760+0000 7fb349ffb640 1 -- 172.21.15.154:0/3260810938 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3500057a0 con 0x7fb340005180 2023-09-23T03:25:51.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.760+0000 7fb35b7d0640 1 -- 172.21.15.154:0/3260810938 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb35409ac00 con 0x7fb340005180 2023-09-23T03:25:51.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.761+0000 7fb35b7d0640 1 -- 172.21.15.154:0/3260810938 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb35409adc0 con 0x7fb340005180 2023-09-23T03:25:51.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.762+0000 7fb349ffb640 1 -- 172.21.15.154:0/3260810938 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fb350005900 con 0x7fb340005180 2023-09-23T03:25:51.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.764+0000 7fb349ffb640 1 --2- 172.21.15.154:0/3260810938 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb33806af30 0x7fb33806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:51.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.764+0000 7fb349ffb640 1 -- 172.21.15.154:0/3260810938 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb350089820 con 0x7fb340005180 2023-09-23T03:25:51.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.764+0000 7fb358d6b640 1 --2- 172.21.15.154:0/3260810938 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb33806af30 0x7fb33806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:51.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.764+0000 7fb358d6b640 1 --2- 172.21.15.154:0/3260810938 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb33806af30 0x7fb33806d3f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb3540cdb80 tx=0x7fb344008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:51.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.768+0000 7fb349ffb640 1 -- 172.21.15.154:0/3260810938 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fb350054710 con 0x7fb340005180 2023-09-23T03:25:51.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.953+0000 7fb35b7d0640 1 -- 172.21.15.154:0/3260810938 --> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb35406c6d0 con 0x7fb33806af30 2023-09-23T03:25:51.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.964+0000 7fb349ffb640 1 -- 172.21.15.154:0/3260810938 <== mgr.4110 v2:172.21.15.154:6832/2422652104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287365 (secure 0 0 0) 0x7fb344012920 con 0x7fb33806af30 2023-09-23T03:25:51.968 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:25:51.969 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2023-09-23T03:25:51.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.970+0000 7fb35b7d0640 1 -- 172.21.15.154:0/3260810938 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb33806af30 msgr2=0x7fb33806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:51.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.970+0000 7fb35b7d0640 1 --2- 172.21.15.154:0/3260810938 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb33806af30 0x7fb33806d3f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb3540cdb80 tx=0x7fb344008040 comp rx=0 tx=0).stop 2023-09-23T03:25:51.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.970+0000 7fb35b7d0640 1 -- 172.21.15.154:0/3260810938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb340005180 msgr2=0x7fb3541590b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:51.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.970+0000 7fb35b7d0640 1 --2- 172.21.15.154:0/3260810938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb340005180 0x7fb3541590b0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fb35000e860 tx=0x7fb35000ed20 comp rx=0 tx=0).stop 2023-09-23T03:25:51.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.970+0000 7fb35b7d0640 1 -- 172.21.15.154:0/3260810938 shutdown_connections 2023-09-23T03:25:51.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.970+0000 7fb35b7d0640 1 --2- 172.21.15.154:0/3260810938 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb33806af30 0x7fb33806d3f0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:51.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.970+0000 7fb35b7d0640 1 --2- 172.21.15.154:0/3260810938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb3540982c0 0x7fb354098740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:51.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.970+0000 7fb35b7d0640 1 --2- 172.21.15.154:0/3260810938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb35409b400 0x7fb35415c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:51.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.970+0000 7fb35b7d0640 1 --2- 172.21.15.154:0/3260810938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb340005180 0x7fb3541590b0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:51.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.970+0000 7fb35b7d0640 1 -- 172.21.15.154:0/3260810938 >> 172.21.15.154:0/3260810938 conn(0x7fb35406a0f0 msgr2=0x7fb3540affe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:51.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.970+0000 7fb35b7d0640 1 -- 172.21.15.154:0/3260810938 shutdown_connections 2023-09-23T03:25:51.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:51.970+0000 7fb35b7d0640 1 -- 172.21.15.154:0/3260810938 wait complete. 2023-09-23T03:25:51.989 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":419,"stamp":"2023-09-23T03:25:51.390602+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":146,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260772,"kb_used_data":11316,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713948,"statfs":{"total":773094113280,"available":772827082752,"internally_reserved":0,"allocated":11587584,"data_stored":6189177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":255315514},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.871181"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.544418+0000","last_change":"2023-09-23T03:25:47.562476+0000","last_active":"2023-09-23T03:25:48.544418+0000","last_peered":"2023-09-23T03:25:48.544418+0000","last_clean":"2023-09-23T03:25:48.544418+0000","last_became_active":"2023-09-23T03:25:47.560804+0000","last_became_peered":"2023-09-23T03:25:47.560804+0000","last_unstale":"2023-09-23T03:25:48.544418+0000","last_undegraded":"2023-09-23T03:25:48.544418+0000","last_fullsized":"2023-09-23T03:25:48.544418+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:08:49.981426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.541564+0000","last_change":"2023-09-23T03:25:47.558003+0000","last_active":"2023-09-23T03:25:48.541564+0000","last_peered":"2023-09-23T03:25:48.541564+0000","last_clean":"2023-09-23T03:25:48.541564+0000","last_became_active":"2023-09-23T03:25:47.557386+0000","last_became_peered":"2023-09-23T03:25:47.557386+0000","last_unstale":"2023-09-23T03:25:48.541564+0000","last_undegraded":"2023-09-23T03:25:48.541564+0000","last_fullsized":"2023-09-23T03:25:48.541564+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:12:15.263365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.544186+0000","last_change":"2023-09-23T03:25:47.562407+0000","last_active":"2023-09-23T03:25:48.544186+0000","last_peered":"2023-09-23T03:25:48.544186+0000","last_clean":"2023-09-23T03:25:48.544186+0000","last_became_active":"2023-09-23T03:25:47.560838+0000","last_became_peered":"2023-09-23T03:25:47.560838+0000","last_unstale":"2023-09-23T03:25:48.544186+0000","last_undegraded":"2023-09-23T03:25:48.544186+0000","last_fullsized":"2023-09-23T03:25:48.544186+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:44:28.405703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.650483+0000","last_change":"2023-09-23T03:25:47.568217+0000","last_active":"2023-09-23T03:25:48.650483+0000","last_peered":"2023-09-23T03:25:48.650483+0000","last_clean":"2023-09-23T03:25:48.650483+0000","last_became_active":"2023-09-23T03:25:47.567960+0000","last_became_peered":"2023-09-23T03:25:47.567960+0000","last_unstale":"2023-09-23T03:25:48.650483+0000","last_undegraded":"2023-09-23T03:25:48.650483+0000","last_fullsized":"2023-09-23T03:25:48.650483+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:36:29.064809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.542453+0000","last_change":"2023-09-23T03:25:47.552407+0000","last_active":"2023-09-23T03:25:48.542453+0000","last_peered":"2023-09-23T03:25:48.542453+0000","last_clean":"2023-09-23T03:25:48.542453+0000","last_became_active":"2023-09-23T03:25:47.552120+0000","last_became_peered":"2023-09-23T03:25:47.552120+0000","last_unstale":"2023-09-23T03:25:48.542453+0000","last_undegraded":"2023-09-23T03:25:48.542453+0000","last_fullsized":"2023-09-23T03:25:48.542453+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:22:29.984556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.652806+0000","last_change":"2023-09-23T03:25:47.565272+0000","last_active":"2023-09-23T03:25:48.652806+0000","last_peered":"2023-09-23T03:25:48.652806+0000","last_clean":"2023-09-23T03:25:48.652806+0000","last_became_active":"2023-09-23T03:25:47.564991+0000","last_became_peered":"2023-09-23T03:25:47.564991+0000","last_unstale":"2023-09-23T03:25:48.652806+0000","last_undegraded":"2023-09-23T03:25:48.652806+0000","last_fullsized":"2023-09-23T03:25:48.652806+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:30:23.312059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.652554+0000","last_change":"2023-09-23T03:25:47.563230+0000","last_active":"2023-09-23T03:25:48.652554+0000","last_peered":"2023-09-23T03:25:48.652554+0000","last_clean":"2023-09-23T03:25:48.652554+0000","last_became_active":"2023-09-23T03:25:47.562893+0000","last_became_peered":"2023-09-23T03:25:47.562893+0000","last_unstale":"2023-09-23T03:25:48.652554+0000","last_undegraded":"2023-09-23T03:25:48.652554+0000","last_fullsized":"2023-09-23T03:25:48.652554+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:11:41.423332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.541193+0000","last_change":"2023-09-23T03:25:47.567792+0000","last_active":"2023-09-23T03:25:48.541193+0000","last_peered":"2023-09-23T03:25:48.541193+0000","last_clean":"2023-09-23T03:25:48.541193+0000","last_became_active":"2023-09-23T03:25:47.564122+0000","last_became_peered":"2023-09-23T03:25:47.564122+0000","last_unstale":"2023-09-23T03:25:48.541193+0000","last_undegraded":"2023-09-23T03:25:48.541193+0000","last_fullsized":"2023-09-23T03:25:48.541193+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:47:17.666939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.653659+0000","last_change":"2023-09-23T03:25:47.567035+0000","last_active":"2023-09-23T03:25:48.653659+0000","last_peered":"2023-09-23T03:25:48.653659+0000","last_clean":"2023-09-23T03:25:48.653659+0000","last_became_active":"2023-09-23T03:25:47.566760+0000","last_became_peered":"2023-09-23T03:25:47.566760+0000","last_unstale":"2023-09-23T03:25:48.653659+0000","last_undegraded":"2023-09-23T03:25:48.653659+0000","last_fullsized":"2023-09-23T03:25:48.653659+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:08:49.981426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.543140+0000","last_change":"2023-09-23T03:25:47.556592+0000","last_active":"2023-09-23T03:25:48.543140+0000","last_peered":"2023-09-23T03:25:48.543140+0000","last_clean":"2023-09-23T03:25:48.543140+0000","last_became_active":"2023-09-23T03:25:47.556291+0000","last_became_peered":"2023-09-23T03:25:47.556291+0000","last_unstale":"2023-09-23T03:25:48.543140+0000","last_undegraded":"2023-09-23T03:25:48.543140+0000","last_fullsized":"2023-09-23T03:25:48.543140+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:40:12.805547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.652737+0000","last_change":"2023-09-23T03:25:47.571607+0000","last_active":"2023-09-23T03:25:48.652737+0000","last_peered":"2023-09-23T03:25:48.652737+0000","last_clean":"2023-09-23T03:25:48.652737+0000","last_became_active":"2023-09-23T03:25:47.570374+0000","last_became_peered":"2023-09-23T03:25:47.570374+0000","last_unstale":"2023-09-23T03:25:48.652737+0000","last_undegraded":"2023-09-23T03:25:48.652737+0000","last_fullsized":"2023-09-23T03:25:48.652737+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:34:22.165335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.654061+0000","last_change":"2023-09-23T03:25:47.559771+0000","last_active":"2023-09-23T03:25:48.654061+0000","last_peered":"2023-09-23T03:25:48.654061+0000","last_clean":"2023-09-23T03:25:48.654061+0000","last_became_active":"2023-09-23T03:25:47.558238+0000","last_became_peered":"2023-09-23T03:25:47.558238+0000","last_unstale":"2023-09-23T03:25:48.654061+0000","last_undegraded":"2023-09-23T03:25:48.654061+0000","last_fullsized":"2023-09-23T03:25:48.654061+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:26:17.879691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.650746+0000","last_change":"2023-09-23T03:25:47.556731+0000","last_active":"2023-09-23T03:25:48.650746+0000","last_peered":"2023-09-23T03:25:48.650746+0000","last_clean":"2023-09-23T03:25:48.650746+0000","last_became_active":"2023-09-23T03:25:47.556305+0000","last_became_peered":"2023-09-23T03:25:47.556305+0000","last_unstale":"2023-09-23T03:25:48.650746+0000","last_undegraded":"2023-09-23T03:25:48.650746+0000","last_fullsized":"2023-09-23T03:25:48.650746+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:11:52.433816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.649996+0000","last_change":"2023-09-23T03:25:47.577118+0000","last_active":"2023-09-23T03:25:48.649996+0000","last_peered":"2023-09-23T03:25:48.649996+0000","last_clean":"2023-09-23T03:25:48.649996+0000","last_became_active":"2023-09-23T03:25:47.576767+0000","last_became_peered":"2023-09-23T03:25:47.576767+0000","last_unstale":"2023-09-23T03:25:48.649996+0000","last_undegraded":"2023-09-23T03:25:48.649996+0000","last_fullsized":"2023-09-23T03:25:48.649996+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:23:53.245942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.649788+0000","last_change":"2023-09-23T03:25:47.564474+0000","last_active":"2023-09-23T03:25:48.649788+0000","last_peered":"2023-09-23T03:25:48.649788+0000","last_clean":"2023-09-23T03:25:48.649788+0000","last_became_active":"2023-09-23T03:25:47.560100+0000","last_became_peered":"2023-09-23T03:25:47.560100+0000","last_unstale":"2023-09-23T03:25:48.649788+0000","last_undegraded":"2023-09-23T03:25:48.649788+0000","last_fullsized":"2023-09-23T03:25:48.649788+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:53:45.800528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.653422+0000","last_change":"2023-09-23T03:25:47.564349+0000","last_active":"2023-09-23T03:25:48.653422+0000","last_peered":"2023-09-23T03:25:48.653422+0000","last_clean":"2023-09-23T03:25:48.653422+0000","last_became_active":"2023-09-23T03:25:47.563370+0000","last_became_peered":"2023-09-23T03:25:47.563370+0000","last_unstale":"2023-09-23T03:25:48.653422+0000","last_undegraded":"2023-09-23T03:25:48.653422+0000","last_fullsized":"2023-09-23T03:25:48.653422+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:14:48.030940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.543302+0000","last_change":"2023-09-23T03:25:47.550598+0000","last_active":"2023-09-23T03:25:48.543302+0000","last_peered":"2023-09-23T03:25:48.543302+0000","last_clean":"2023-09-23T03:25:48.543302+0000","last_became_active":"2023-09-23T03:25:47.550180+0000","last_became_peered":"2023-09-23T03:25:47.550180+0000","last_unstale":"2023-09-23T03:25:48.543302+0000","last_undegraded":"2023-09-23T03:25:48.543302+0000","last_fullsized":"2023-09-23T03:25:48.543302+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:22:11.155654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.650896+0000","last_change":"2023-09-23T03:25:47.581245+0000","last_active":"2023-09-23T03:25:48.650896+0000","last_peered":"2023-09-23T03:25:48.650896+0000","last_clean":"2023-09-23T03:25:48.650896+0000","last_became_active":"2023-09-23T03:25:47.580908+0000","last_became_peered":"2023-09-23T03:25:47.580908+0000","last_unstale":"2023-09-23T03:25:48.650896+0000","last_undegraded":"2023-09-23T03:25:48.650896+0000","last_fullsized":"2023-09-23T03:25:48.650896+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:11:41.423332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.650751+0000","last_change":"2023-09-23T03:25:47.572640+0000","last_active":"2023-09-23T03:25:48.650751+0000","last_peered":"2023-09-23T03:25:48.650751+0000","last_clean":"2023-09-23T03:25:48.650751+0000","last_became_active":"2023-09-23T03:25:47.572387+0000","last_became_peered":"2023-09-23T03:25:47.572387+0000","last_unstale":"2023-09-23T03:25:48.650751+0000","last_undegraded":"2023-09-23T03:25:48.650751+0000","last_fullsized":"2023-09-23T03:25:48.650751+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:19:26.573945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.654444+0000","last_change":"2023-09-23T03:25:47.566060+0000","last_active":"2023-09-23T03:25:48.654444+0000","last_peered":"2023-09-23T03:25:48.654444+0000","last_clean":"2023-09-23T03:25:48.654444+0000","last_became_active":"2023-09-23T03:25:47.565820+0000","last_became_peered":"2023-09-23T03:25:47.565820+0000","last_unstale":"2023-09-23T03:25:48.654444+0000","last_undegraded":"2023-09-23T03:25:48.654444+0000","last_fullsized":"2023-09-23T03:25:48.654444+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:26:10.003436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.650415+0000","last_change":"2023-09-23T03:25:47.570095+0000","last_active":"2023-09-23T03:25:48.650415+0000","last_peered":"2023-09-23T03:25:48.650415+0000","last_clean":"2023-09-23T03:25:48.650415+0000","last_became_active":"2023-09-23T03:25:47.569731+0000","last_became_peered":"2023-09-23T03:25:47.569731+0000","last_unstale":"2023-09-23T03:25:48.650415+0000","last_undegraded":"2023-09-23T03:25:48.650415+0000","last_fullsized":"2023-09-23T03:25:48.650415+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:43:42.824444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.541955+0000","last_change":"2023-09-23T03:25:47.562619+0000","last_active":"2023-09-23T03:25:48.541955+0000","last_peered":"2023-09-23T03:25:48.541955+0000","last_clean":"2023-09-23T03:25:48.541955+0000","last_became_active":"2023-09-23T03:25:47.562323+0000","last_became_peered":"2023-09-23T03:25:47.562323+0000","last_unstale":"2023-09-23T03:25:48.541955+0000","last_undegraded":"2023-09-23T03:25:48.541955+0000","last_fullsized":"2023-09-23T03:25:48.541955+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:23:48.640810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.541510+0000","last_change":"2023-09-23T03:25:47.561642+0000","last_active":"2023-09-23T03:25:48.541510+0000","last_peered":"2023-09-23T03:25:48.541510+0000","last_clean":"2023-09-23T03:25:48.541510+0000","last_became_active":"2023-09-23T03:25:47.561333+0000","last_became_peered":"2023-09-23T03:25:47.561333+0000","last_unstale":"2023-09-23T03:25:48.541510+0000","last_undegraded":"2023-09-23T03:25:48.541510+0000","last_fullsized":"2023-09-23T03:25:48.541510+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:55:49.357259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.652713+0000","last_change":"2023-09-23T03:25:47.564522+0000","last_active":"2023-09-23T03:25:48.652713+0000","last_peered":"2023-09-23T03:25:48.652713+0000","last_clean":"2023-09-23T03:25:48.652713+0000","last_became_active":"2023-09-23T03:25:47.563946+0000","last_became_peered":"2023-09-23T03:25:47.563946+0000","last_unstale":"2023-09-23T03:25:48.652713+0000","last_undegraded":"2023-09-23T03:25:48.652713+0000","last_fullsized":"2023-09-23T03:25:48.652713+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:31:13.203775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.543110+0000","last_change":"2023-09-23T03:25:47.555880+0000","last_active":"2023-09-23T03:25:48.543110+0000","last_peered":"2023-09-23T03:25:48.543110+0000","last_clean":"2023-09-23T03:25:48.543110+0000","last_became_active":"2023-09-23T03:25:47.555573+0000","last_became_peered":"2023-09-23T03:25:47.555573+0000","last_unstale":"2023-09-23T03:25:48.543110+0000","last_undegraded":"2023-09-23T03:25:48.543110+0000","last_fullsized":"2023-09-23T03:25:48.543110+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:45:16.921815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.542791+0000","last_change":"2023-09-23T03:25:47.558174+0000","last_active":"2023-09-23T03:25:48.542791+0000","last_peered":"2023-09-23T03:25:48.542791+0000","last_clean":"2023-09-23T03:25:48.542791+0000","last_became_active":"2023-09-23T03:25:47.557730+0000","last_became_peered":"2023-09-23T03:25:47.557730+0000","last_unstale":"2023-09-23T03:25:48.542791+0000","last_undegraded":"2023-09-23T03:25:48.542791+0000","last_fullsized":"2023-09-23T03:25:48.542791+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:11:28.607821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.653083+0000","last_change":"2023-09-23T03:25:47.575349+0000","last_active":"2023-09-23T03:25:48.653083+0000","last_peered":"2023-09-23T03:25:48.653083+0000","last_clean":"2023-09-23T03:25:48.653083+0000","last_became_active":"2023-09-23T03:25:47.575063+0000","last_became_peered":"2023-09-23T03:25:47.575063+0000","last_unstale":"2023-09-23T03:25:48.653083+0000","last_undegraded":"2023-09-23T03:25:48.653083+0000","last_fullsized":"2023-09-23T03:25:48.653083+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:19:20.436542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.542773+0000","last_change":"2023-09-23T03:25:47.551619+0000","last_active":"2023-09-23T03:25:48.542773+0000","last_peered":"2023-09-23T03:25:48.542773+0000","last_clean":"2023-09-23T03:25:48.542773+0000","last_became_active":"2023-09-23T03:25:47.551329+0000","last_became_peered":"2023-09-23T03:25:47.551329+0000","last_unstale":"2023-09-23T03:25:48.542773+0000","last_undegraded":"2023-09-23T03:25:48.542773+0000","last_fullsized":"2023-09-23T03:25:48.542773+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:15:51.790186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.541808+0000","last_change":"2023-09-23T03:25:47.556821+0000","last_active":"2023-09-23T03:25:48.541808+0000","last_peered":"2023-09-23T03:25:48.541808+0000","last_clean":"2023-09-23T03:25:48.541808+0000","last_became_active":"2023-09-23T03:25:47.556553+0000","last_became_peered":"2023-09-23T03:25:47.556553+0000","last_unstale":"2023-09-23T03:25:48.541808+0000","last_undegraded":"2023-09-23T03:25:48.541808+0000","last_fullsized":"2023-09-23T03:25:48.541808+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:01:43.955510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.650111+0000","last_change":"2023-09-23T03:25:47.573863+0000","last_active":"2023-09-23T03:25:48.650111+0000","last_peered":"2023-09-23T03:25:48.650111+0000","last_clean":"2023-09-23T03:25:48.650111+0000","last_became_active":"2023-09-23T03:25:47.573383+0000","last_became_peered":"2023-09-23T03:25:47.573383+0000","last_unstale":"2023-09-23T03:25:48.650111+0000","last_undegraded":"2023-09-23T03:25:48.650111+0000","last_fullsized":"2023-09-23T03:25:48.650111+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:50:07.532358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.649955+0000","last_change":"2023-09-23T03:25:47.570013+0000","last_active":"2023-09-23T03:25:48.649955+0000","last_peered":"2023-09-23T03:25:48.649955+0000","last_clean":"2023-09-23T03:25:48.649955+0000","last_became_active":"2023-09-23T03:25:47.567042+0000","last_became_peered":"2023-09-23T03:25:47.567042+0000","last_unstale":"2023-09-23T03:25:48.649955+0000","last_undegraded":"2023-09-23T03:25:48.649955+0000","last_fullsized":"2023-09-23T03:25:48.649955+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:58:09.113015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.543032+0000","last_change":"2023-09-23T03:25:47.561072+0000","last_active":"2023-09-23T03:25:48.543032+0000","last_peered":"2023-09-23T03:25:48.543032+0000","last_clean":"2023-09-23T03:25:48.543032+0000","last_became_active":"2023-09-23T03:25:47.560790+0000","last_became_peered":"2023-09-23T03:25:47.560790+0000","last_unstale":"2023-09-23T03:25:48.543032+0000","last_undegraded":"2023-09-23T03:25:48.543032+0000","last_fullsized":"2023-09-23T03:25:48.543032+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:44:06.391679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.650647+0000","last_change":"2023-09-23T03:25:47.574904+0000","last_active":"2023-09-23T03:25:48.650647+0000","last_peered":"2023-09-23T03:25:48.650647+0000","last_clean":"2023-09-23T03:25:48.650647+0000","last_became_active":"2023-09-23T03:25:47.574591+0000","last_became_peered":"2023-09-23T03:25:47.574591+0000","last_unstale":"2023-09-23T03:25:48.650647+0000","last_undegraded":"2023-09-23T03:25:48.650647+0000","last_fullsized":"2023-09-23T03:25:48.650647+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:37:15.720353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"104'4","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.543582+0000","last_change":"2023-09-23T03:25:46.563997+0000","last_active":"2023-09-23T03:25:48.543582+0000","last_peered":"2023-09-23T03:25:48.543582+0000","last_clean":"2023-09-23T03:25:48.543582+0000","last_became_active":"2023-09-23T03:25:46.563366+0000","last_became_peered":"2023-09-23T03:25:46.563366+0000","last_unstale":"2023-09-23T03:25:48.543582+0000","last_undegraded":"2023-09-23T03:25:48.543582+0000","last_fullsized":"2023-09-23T03:25:48.543582+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:05:12.835540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.650583+0000","last_change":"2023-09-23T03:25:47.571243+0000","last_active":"2023-09-23T03:25:48.650583+0000","last_peered":"2023-09-23T03:25:48.650583+0000","last_clean":"2023-09-23T03:25:48.650583+0000","last_became_active":"2023-09-23T03:25:47.570969+0000","last_became_peered":"2023-09-23T03:25:47.570969+0000","last_unstale":"2023-09-23T03:25:48.650583+0000","last_undegraded":"2023-09-23T03:25:48.650583+0000","last_fullsized":"2023-09-23T03:25:48.650583+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:51:49.169984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"104'1","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.654064+0000","last_change":"2023-09-23T03:25:46.565485+0000","last_active":"2023-09-23T03:25:48.654064+0000","last_peered":"2023-09-23T03:25:48.654064+0000","last_clean":"2023-09-23T03:25:48.654064+0000","last_became_active":"2023-09-23T03:25:46.565154+0000","last_became_peered":"2023-09-23T03:25:46.565154+0000","last_unstale":"2023-09-23T03:25:48.654064+0000","last_undegraded":"2023-09-23T03:25:48.654064+0000","last_fullsized":"2023-09-23T03:25:48.654064+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:36:28.050865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.544064+0000","last_change":"2023-09-23T03:25:47.566010+0000","last_active":"2023-09-23T03:25:48.544064+0000","last_peered":"2023-09-23T03:25:48.544064+0000","last_clean":"2023-09-23T03:25:48.544064+0000","last_became_active":"2023-09-23T03:25:47.565750+0000","last_became_peered":"2023-09-23T03:25:47.565750+0000","last_unstale":"2023-09-23T03:25:48.544064+0000","last_undegraded":"2023-09-23T03:25:48.544064+0000","last_fullsized":"2023-09-23T03:25:48.544064+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:05:10.550787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"104'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.542747+0000","last_change":"2023-09-23T03:25:46.560275+0000","last_active":"2023-09-23T03:25:48.542747+0000","last_peered":"2023-09-23T03:25:48.542747+0000","last_clean":"2023-09-23T03:25:48.542747+0000","last_became_active":"2023-09-23T03:25:46.560005+0000","last_became_peered":"2023-09-23T03:25:46.560005+0000","last_unstale":"2023-09-23T03:25:48.542747+0000","last_undegraded":"2023-09-23T03:25:48.542747+0000","last_fullsized":"2023-09-23T03:25:48.542747+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:19:42.909077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.654784+0000","last_change":"2023-09-23T03:25:47.562063+0000","last_active":"2023-09-23T03:25:48.654784+0000","last_peered":"2023-09-23T03:25:48.654784+0000","last_clean":"2023-09-23T03:25:48.654784+0000","last_became_active":"2023-09-23T03:25:47.561784+0000","last_became_peered":"2023-09-23T03:25:47.561784+0000","last_unstale":"2023-09-23T03:25:48.654784+0000","last_undegraded":"2023-09-23T03:25:48.654784+0000","last_fullsized":"2023-09-23T03:25:48.654784+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:52:12.549337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"104'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.541841+0000","last_change":"2023-09-23T03:25:46.558854+0000","last_active":"2023-09-23T03:25:48.541841+0000","last_peered":"2023-09-23T03:25:48.541841+0000","last_clean":"2023-09-23T03:25:48.541841+0000","last_became_active":"2023-09-23T03:25:46.558577+0000","last_became_peered":"2023-09-23T03:25:46.558577+0000","last_unstale":"2023-09-23T03:25:48.541841+0000","last_undegraded":"2023-09-23T03:25:48.541841+0000","last_fullsized":"2023-09-23T03:25:48.541841+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:31:12.189830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.655876+0000","last_change":"2023-09-23T03:25:47.555592+0000","last_active":"2023-09-23T03:25:48.655876+0000","last_peered":"2023-09-23T03:25:48.655876+0000","last_clean":"2023-09-23T03:25:48.655876+0000","last_became_active":"2023-09-23T03:25:47.555277+0000","last_became_peered":"2023-09-23T03:25:47.555277+0000","last_unstale":"2023-09-23T03:25:48.655876+0000","last_undegraded":"2023-09-23T03:25:48.655876+0000","last_fullsized":"2023-09-23T03:25:48.655876+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:57:17.479365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"104'1","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.543891+0000","last_change":"2023-09-23T03:25:46.556929+0000","last_active":"2023-09-23T03:25:48.543891+0000","last_peered":"2023-09-23T03:25:48.543891+0000","last_clean":"2023-09-23T03:25:48.543891+0000","last_became_active":"2023-09-23T03:25:46.556677+0000","last_became_peered":"2023-09-23T03:25:46.556677+0000","last_unstale":"2023-09-23T03:25:48.543891+0000","last_undegraded":"2023-09-23T03:25:48.543891+0000","last_fullsized":"2023-09-23T03:25:48.543891+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:37:14.706408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.652849+0000","last_change":"2023-09-23T03:25:47.571738+0000","last_active":"2023-09-23T03:25:48.652849+0000","last_peered":"2023-09-23T03:25:48.652849+0000","last_clean":"2023-09-23T03:25:48.652849+0000","last_became_active":"2023-09-23T03:25:47.571459+0000","last_became_peered":"2023-09-23T03:25:47.571459+0000","last_unstale":"2023-09-23T03:25:48.652849+0000","last_undegraded":"2023-09-23T03:25:48.652849+0000","last_fullsized":"2023-09-23T03:25:48.652849+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:14:16.668225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"104'1","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.542170+0000","last_change":"2023-09-23T03:25:46.555907+0000","last_active":"2023-09-23T03:25:48.542170+0000","last_peered":"2023-09-23T03:25:48.542170+0000","last_clean":"2023-09-23T03:25:48.542170+0000","last_became_active":"2023-09-23T03:25:46.555650+0000","last_became_peered":"2023-09-23T03:25:46.555650+0000","last_unstale":"2023-09-23T03:25:48.542170+0000","last_undegraded":"2023-09-23T03:25:48.542170+0000","last_fullsized":"2023-09-23T03:25:48.542170+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:14:47.016995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.649794+0000","last_change":"2023-09-23T03:25:47.559681+0000","last_active":"2023-09-23T03:25:48.649794+0000","last_peered":"2023-09-23T03:25:48.649794+0000","last_clean":"2023-09-23T03:25:48.649794+0000","last_became_active":"2023-09-23T03:25:47.559412+0000","last_became_peered":"2023-09-23T03:25:47.559412+0000","last_unstale":"2023-09-23T03:25:48.649794+0000","last_undegraded":"2023-09-23T03:25:48.649794+0000","last_fullsized":"2023-09-23T03:25:48.649794+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:24:59.861432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.542366+0000","last_change":"2023-09-23T03:25:46.567925+0000","last_active":"2023-09-23T03:25:48.542366+0000","last_peered":"2023-09-23T03:25:48.542366+0000","last_clean":"2023-09-23T03:25:48.542366+0000","last_became_active":"2023-09-23T03:25:46.567639+0000","last_became_peered":"2023-09-23T03:25:46.567639+0000","last_unstale":"2023-09-23T03:25:48.542366+0000","last_undegraded":"2023-09-23T03:25:48.542366+0000","last_fullsized":"2023-09-23T03:25:48.542366+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:26:16.865746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.653073+0000","last_change":"2023-09-23T03:25:47.563974+0000","last_active":"2023-09-23T03:25:48.653073+0000","last_peered":"2023-09-23T03:25:48.653073+0000","last_clean":"2023-09-23T03:25:48.653073+0000","last_became_active":"2023-09-23T03:25:47.563703+0000","last_became_peered":"2023-09-23T03:25:47.563703+0000","last_unstale":"2023-09-23T03:25:48.653073+0000","last_undegraded":"2023-09-23T03:25:48.653073+0000","last_fullsized":"2023-09-23T03:25:48.653073+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:44:28.405703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.543309+0000","last_change":"2023-09-23T03:25:46.559639+0000","last_active":"2023-09-23T03:25:48.543309+0000","last_peered":"2023-09-23T03:25:48.543309+0000","last_clean":"2023-09-23T03:25:48.543309+0000","last_became_active":"2023-09-23T03:25:46.559316+0000","last_became_peered":"2023-09-23T03:25:46.559316+0000","last_unstale":"2023-09-23T03:25:48.543309+0000","last_undegraded":"2023-09-23T03:25:48.543309+0000","last_fullsized":"2023-09-23T03:25:48.543309+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:43:41.810500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.542939+0000","last_change":"2023-09-23T03:25:47.552727+0000","last_active":"2023-09-23T03:25:48.542939+0000","last_peered":"2023-09-23T03:25:48.542939+0000","last_clean":"2023-09-23T03:25:48.542939+0000","last_became_active":"2023-09-23T03:25:47.552373+0000","last_became_peered":"2023-09-23T03:25:47.552373+0000","last_unstale":"2023-09-23T03:25:48.542939+0000","last_undegraded":"2023-09-23T03:25:48.542939+0000","last_fullsized":"2023-09-23T03:25:48.542939+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:34:06.551529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.544633+0000","last_change":"2023-09-23T03:25:46.560198+0000","last_active":"2023-09-23T03:25:48.544633+0000","last_peered":"2023-09-23T03:25:48.544633+0000","last_clean":"2023-09-23T03:25:48.544633+0000","last_became_active":"2023-09-23T03:25:46.559391+0000","last_became_peered":"2023-09-23T03:25:46.559391+0000","last_unstale":"2023-09-23T03:25:48.544633+0000","last_undegraded":"2023-09-23T03:25:48.544633+0000","last_fullsized":"2023-09-23T03:25:48.544633+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:19:25.560001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.654003+0000","last_change":"2023-09-23T03:25:47.564201+0000","last_active":"2023-09-23T03:25:48.654003+0000","last_peered":"2023-09-23T03:25:48.654003+0000","last_clean":"2023-09-23T03:25:48.654003+0000","last_became_active":"2023-09-23T03:25:47.563622+0000","last_became_peered":"2023-09-23T03:25:47.563622+0000","last_unstale":"2023-09-23T03:25:48.654003+0000","last_undegraded":"2023-09-23T03:25:48.654003+0000","last_fullsized":"2023-09-23T03:25:48.654003+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:04:29.550642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.650924+0000","last_change":"2023-09-23T03:25:46.550850+0000","last_active":"2023-09-23T03:25:48.650924+0000","last_peered":"2023-09-23T03:25:48.650924+0000","last_clean":"2023-09-23T03:25:48.650924+0000","last_became_active":"2023-09-23T03:25:46.550549+0000","last_became_peered":"2023-09-23T03:25:46.550549+0000","last_unstale":"2023-09-23T03:25:48.650924+0000","last_undegraded":"2023-09-23T03:25:48.650924+0000","last_fullsized":"2023-09-23T03:25:48.650924+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:40:11.791603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.542959+0000","last_change":"2023-09-23T03:25:47.555910+0000","last_active":"2023-09-23T03:25:48.542959+0000","last_peered":"2023-09-23T03:25:48.542959+0000","last_clean":"2023-09-23T03:25:48.542959+0000","last_became_active":"2023-09-23T03:25:47.555613+0000","last_became_peered":"2023-09-23T03:25:47.555613+0000","last_unstale":"2023-09-23T03:25:48.542959+0000","last_undegraded":"2023-09-23T03:25:48.542959+0000","last_fullsized":"2023-09-23T03:25:48.542959+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:34:24.984771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"104'3","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.654196+0000","last_change":"2023-09-23T03:25:46.564782+0000","last_active":"2023-09-23T03:25:48.654196+0000","last_peered":"2023-09-23T03:25:48.654196+0000","last_clean":"2023-09-23T03:25:48.654196+0000","last_became_active":"2023-09-23T03:25:46.564378+0000","last_became_peered":"2023-09-23T03:25:46.564378+0000","last_unstale":"2023-09-23T03:25:48.654196+0000","last_undegraded":"2023-09-23T03:25:48.654196+0000","last_fullsized":"2023-09-23T03:25:48.654196+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:58:08.099071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.653356+0000","last_change":"2023-09-23T03:25:47.561433+0000","last_active":"2023-09-23T03:25:48.653356+0000","last_peered":"2023-09-23T03:25:48.653356+0000","last_clean":"2023-09-23T03:25:48.653356+0000","last_became_active":"2023-09-23T03:25:47.561171+0000","last_became_peered":"2023-09-23T03:25:47.561171+0000","last_unstale":"2023-09-23T03:25:48.653356+0000","last_undegraded":"2023-09-23T03:25:48.653356+0000","last_fullsized":"2023-09-23T03:25:48.653356+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:09:06.345724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"104'6","reported_seq":18,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.655054+0000","last_change":"2023-09-23T03:25:46.563065+0000","last_active":"2023-09-23T03:25:48.655054+0000","last_peered":"2023-09-23T03:25:48.655054+0000","last_clean":"2023-09-23T03:25:48.655054+0000","last_became_active":"2023-09-23T03:25:46.562157+0000","last_became_peered":"2023-09-23T03:25:46.562157+0000","last_unstale":"2023-09-23T03:25:48.655054+0000","last_undegraded":"2023-09-23T03:25:48.655054+0000","last_fullsized":"2023-09-23T03:25:48.655054+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:00:57.861554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.650245+0000","last_change":"2023-09-23T03:25:47.564708+0000","last_active":"2023-09-23T03:25:48.650245+0000","last_peered":"2023-09-23T03:25:48.650245+0000","last_clean":"2023-09-23T03:25:48.650245+0000","last_became_active":"2023-09-23T03:25:47.564414+0000","last_became_peered":"2023-09-23T03:25:47.564414+0000","last_unstale":"2023-09-23T03:25:48.650245+0000","last_undegraded":"2023-09-23T03:25:48.650245+0000","last_fullsized":"2023-09-23T03:25:48.650245+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:53:32.559094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"104'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.650604+0000","last_change":"2023-09-23T03:25:46.560666+0000","last_active":"2023-09-23T03:25:48.650604+0000","last_peered":"2023-09-23T03:25:48.650604+0000","last_clean":"2023-09-23T03:25:48.650604+0000","last_became_active":"2023-09-23T03:25:46.559945+0000","last_became_peered":"2023-09-23T03:25:46.559945+0000","last_unstale":"2023-09-23T03:25:48.650604+0000","last_undegraded":"2023-09-23T03:25:48.650604+0000","last_fullsized":"2023-09-23T03:25:48.650604+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:47:22.988221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.543856+0000","last_change":"2023-09-23T03:25:47.576710+0000","last_active":"2023-09-23T03:25:48.543856+0000","last_peered":"2023-09-23T03:25:48.543856+0000","last_clean":"2023-09-23T03:25:48.543856+0000","last_became_active":"2023-09-23T03:25:47.576417+0000","last_became_peered":"2023-09-23T03:25:47.576417+0000","last_unstale":"2023-09-23T03:25:48.543856+0000","last_undegraded":"2023-09-23T03:25:48.543856+0000","last_fullsized":"2023-09-23T03:25:48.543856+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:00:58.875499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.650368+0000","last_change":"2023-09-23T03:25:46.557090+0000","last_active":"2023-09-23T03:25:48.650368+0000","last_peered":"2023-09-23T03:25:48.650368+0000","last_clean":"2023-09-23T03:25:48.650368+0000","last_became_active":"2023-09-23T03:25:46.556821+0000","last_became_peered":"2023-09-23T03:25:46.556821+0000","last_unstale":"2023-09-23T03:25:48.650368+0000","last_undegraded":"2023-09-23T03:25:48.650368+0000","last_fullsized":"2023-09-23T03:25:48.650368+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:46:13.084887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.652555+0000","last_change":"2023-09-23T03:25:47.561739+0000","last_active":"2023-09-23T03:25:48.652555+0000","last_peered":"2023-09-23T03:25:48.652555+0000","last_clean":"2023-09-23T03:25:48.652555+0000","last_became_active":"2023-09-23T03:25:47.561443+0000","last_became_peered":"2023-09-23T03:25:47.561443+0000","last_unstale":"2023-09-23T03:25:48.652555+0000","last_undegraded":"2023-09-23T03:25:48.652555+0000","last_fullsized":"2023-09-23T03:25:48.652555+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:48:16.248495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.653066+0000","last_change":"2023-09-23T03:25:46.557594+0000","last_active":"2023-09-23T03:25:48.653066+0000","last_peered":"2023-09-23T03:25:48.653066+0000","last_clean":"2023-09-23T03:25:48.653066+0000","last_became_active":"2023-09-23T03:25:46.557188+0000","last_became_peered":"2023-09-23T03:25:46.557188+0000","last_unstale":"2023-09-23T03:25:48.653066+0000","last_undegraded":"2023-09-23T03:25:48.653066+0000","last_fullsized":"2023-09-23T03:25:48.653066+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:43:41.810500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.650029+0000","last_change":"2023-09-23T03:25:47.572410+0000","last_active":"2023-09-23T03:25:48.650029+0000","last_peered":"2023-09-23T03:25:48.650029+0000","last_clean":"2023-09-23T03:25:48.650029+0000","last_became_active":"2023-09-23T03:25:47.572099+0000","last_became_peered":"2023-09-23T03:25:47.572099+0000","last_unstale":"2023-09-23T03:25:48.650029+0000","last_undegraded":"2023-09-23T03:25:48.650029+0000","last_fullsized":"2023-09-23T03:25:48.650029+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:37:41.517191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"104'8","reported_seq":20,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.543487+0000","last_change":"2023-09-23T03:25:46.565386+0000","last_active":"2023-09-23T03:25:48.543487+0000","last_peered":"2023-09-23T03:25:48.543487+0000","last_clean":"2023-09-23T03:25:48.543487+0000","last_became_active":"2023-09-23T03:25:46.564730+0000","last_became_peered":"2023-09-23T03:25:46.564730+0000","last_unstale":"2023-09-23T03:25:48.543487+0000","last_undegraded":"2023-09-23T03:25:48.543487+0000","last_fullsized":"2023-09-23T03:25:48.543487+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:25:32.261618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.543342+0000","last_change":"2023-09-23T03:25:47.555158+0000","last_active":"2023-09-23T03:25:48.543342+0000","last_peered":"2023-09-23T03:25:48.543342+0000","last_clean":"2023-09-23T03:25:48.543342+0000","last_became_active":"2023-09-23T03:25:47.554873+0000","last_became_peered":"2023-09-23T03:25:47.554873+0000","last_unstale":"2023-09-23T03:25:48.543342+0000","last_undegraded":"2023-09-23T03:25:48.543342+0000","last_fullsized":"2023-09-23T03:25:48.543342+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:53:45.800528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"104'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.654978+0000","last_change":"2023-09-23T03:25:46.559412+0000","last_active":"2023-09-23T03:25:48.654978+0000","last_peered":"2023-09-23T03:25:48.654978+0000","last_clean":"2023-09-23T03:25:48.654978+0000","last_became_active":"2023-09-23T03:25:46.559082+0000","last_became_peered":"2023-09-23T03:25:46.559082+0000","last_unstale":"2023-09-23T03:25:48.654978+0000","last_undegraded":"2023-09-23T03:25:48.654978+0000","last_fullsized":"2023-09-23T03:25:48.654978+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:25:32.261618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.655563+0000","last_change":"2023-09-23T03:25:47.567520+0000","last_active":"2023-09-23T03:25:48.655563+0000","last_peered":"2023-09-23T03:25:48.655563+0000","last_clean":"2023-09-23T03:25:48.655563+0000","last_became_active":"2023-09-23T03:25:47.567131+0000","last_became_peered":"2023-09-23T03:25:47.567131+0000","last_unstale":"2023-09-23T03:25:48.655563+0000","last_undegraded":"2023-09-23T03:25:48.655563+0000","last_fullsized":"2023-09-23T03:25:48.655563+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:24:57.796899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"104'1","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.542752+0000","last_change":"2023-09-23T03:25:46.560175+0000","last_active":"2023-09-23T03:25:48.542752+0000","last_peered":"2023-09-23T03:25:48.542752+0000","last_clean":"2023-09-23T03:25:48.542752+0000","last_became_active":"2023-09-23T03:25:46.558277+0000","last_became_peered":"2023-09-23T03:25:46.558277+0000","last_unstale":"2023-09-23T03:25:48.542752+0000","last_undegraded":"2023-09-23T03:25:48.542752+0000","last_fullsized":"2023-09-23T03:25:48.542752+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:39:44.467126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.653904+0000","last_change":"2023-09-23T03:25:47.571139+0000","last_active":"2023-09-23T03:25:48.653904+0000","last_peered":"2023-09-23T03:25:48.653904+0000","last_clean":"2023-09-23T03:25:48.653904+0000","last_became_active":"2023-09-23T03:25:47.570861+0000","last_became_peered":"2023-09-23T03:25:47.570861+0000","last_unstale":"2023-09-23T03:25:48.653904+0000","last_undegraded":"2023-09-23T03:25:48.653904+0000","last_fullsized":"2023-09-23T03:25:48.653904+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:19:43.923022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.542040+0000","last_change":"2023-09-23T03:25:46.561312+0000","last_active":"2023-09-23T03:25:48.542040+0000","last_peered":"2023-09-23T03:25:48.542040+0000","last_clean":"2023-09-23T03:25:48.542040+0000","last_became_active":"2023-09-23T03:25:46.561052+0000","last_became_peered":"2023-09-23T03:25:46.561052+0000","last_unstale":"2023-09-23T03:25:48.542040+0000","last_undegraded":"2023-09-23T03:25:48.542040+0000","last_fullsized":"2023-09-23T03:25:48.542040+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:14:15.654280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.650231+0000","last_change":"2023-09-23T03:25:47.573902+0000","last_active":"2023-09-23T03:25:48.650231+0000","last_peered":"2023-09-23T03:25:48.650231+0000","last_clean":"2023-09-23T03:25:48.650231+0000","last_became_active":"2023-09-23T03:25:47.573600+0000","last_became_peered":"2023-09-23T03:25:47.573600+0000","last_unstale":"2023-09-23T03:25:48.650231+0000","last_undegraded":"2023-09-23T03:25:48.650231+0000","last_fullsized":"2023-09-23T03:25:48.650231+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:45:07.332713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"104'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.542033+0000","last_change":"2023-09-23T03:25:46.559615+0000","last_active":"2023-09-23T03:25:48.542033+0000","last_peered":"2023-09-23T03:25:48.542033+0000","last_clean":"2023-09-23T03:25:48.542033+0000","last_became_active":"2023-09-23T03:25:46.559329+0000","last_became_peered":"2023-09-23T03:25:46.559329+0000","last_unstale":"2023-09-23T03:25:48.542033+0000","last_undegraded":"2023-09-23T03:25:48.542033+0000","last_fullsized":"2023-09-23T03:25:48.542033+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:32:10.143445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.650552+0000","last_change":"2023-09-23T03:25:47.570065+0000","last_active":"2023-09-23T03:25:48.650552+0000","last_peered":"2023-09-23T03:25:48.650552+0000","last_clean":"2023-09-23T03:25:48.650552+0000","last_became_active":"2023-09-23T03:25:47.569696+0000","last_became_peered":"2023-09-23T03:25:47.569696+0000","last_unstale":"2023-09-23T03:25:48.650552+0000","last_undegraded":"2023-09-23T03:25:48.650552+0000","last_fullsized":"2023-09-23T03:25:48.650552+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:25:33.275563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.655783+0000","last_change":"2023-09-23T03:25:46.565437+0000","last_active":"2023-09-23T03:25:48.655783+0000","last_peered":"2023-09-23T03:25:48.655783+0000","last_clean":"2023-09-23T03:25:48.655783+0000","last_became_active":"2023-09-23T03:25:46.565087+0000","last_became_peered":"2023-09-23T03:25:46.565087+0000","last_unstale":"2023-09-23T03:25:48.655783+0000","last_undegraded":"2023-09-23T03:25:48.655783+0000","last_fullsized":"2023-09-23T03:25:48.655783+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:30:43.602868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.649979+0000","last_change":"2023-09-23T03:25:47.574312+0000","last_active":"2023-09-23T03:25:48.649979+0000","last_peered":"2023-09-23T03:25:48.649979+0000","last_clean":"2023-09-23T03:25:48.649979+0000","last_became_active":"2023-09-23T03:25:47.574019+0000","last_became_peered":"2023-09-23T03:25:47.574019+0000","last_unstale":"2023-09-23T03:25:48.649979+0000","last_undegraded":"2023-09-23T03:25:48.649979+0000","last_fullsized":"2023-09-23T03:25:48.649979+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:12:04.699882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.655383+0000","last_change":"2023-09-23T03:25:46.559604+0000","last_active":"2023-09-23T03:25:48.655383+0000","last_peered":"2023-09-23T03:25:48.655383+0000","last_clean":"2023-09-23T03:25:48.655383+0000","last_became_active":"2023-09-23T03:25:46.559315+0000","last_became_peered":"2023-09-23T03:25:46.559315+0000","last_unstale":"2023-09-23T03:25:48.655383+0000","last_undegraded":"2023-09-23T03:25:48.655383+0000","last_fullsized":"2023-09-23T03:25:48.655383+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:39:44.467126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.541903+0000","last_change":"2023-09-23T03:25:47.560277+0000","last_active":"2023-09-23T03:25:48.541903+0000","last_peered":"2023-09-23T03:25:48.541903+0000","last_clean":"2023-09-23T03:25:48.541903+0000","last_became_active":"2023-09-23T03:25:47.556974+0000","last_became_peered":"2023-09-23T03:25:47.556974+0000","last_unstale":"2023-09-23T03:25:48.541903+0000","last_undegraded":"2023-09-23T03:25:48.541903+0000","last_fullsized":"2023-09-23T03:25:48.541903+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:19:31.149437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.541197+0000","last_change":"2023-09-23T03:25:46.555302+0000","last_active":"2023-09-23T03:25:48.541197+0000","last_peered":"2023-09-23T03:25:48.541197+0000","last_clean":"2023-09-23T03:25:48.541197+0000","last_became_active":"2023-09-23T03:25:46.554954+0000","last_became_peered":"2023-09-23T03:25:46.554954+0000","last_unstale":"2023-09-23T03:25:48.541197+0000","last_undegraded":"2023-09-23T03:25:48.541197+0000","last_fullsized":"2023-09-23T03:25:48.541197+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:19:19.422597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.543142+0000","last_change":"2023-09-23T03:25:47.560873+0000","last_active":"2023-09-23T03:25:48.543142+0000","last_peered":"2023-09-23T03:25:48.543142+0000","last_clean":"2023-09-23T03:25:48.543142+0000","last_became_active":"2023-09-23T03:25:47.560601+0000","last_became_peered":"2023-09-23T03:25:47.560601+0000","last_unstale":"2023-09-23T03:25:48.543142+0000","last_undegraded":"2023-09-23T03:25:48.543142+0000","last_fullsized":"2023-09-23T03:25:48.543142+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:51:34.682928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"104'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.656104+0000","last_change":"2023-09-23T03:25:46.560272+0000","last_active":"2023-09-23T03:25:48.656104+0000","last_peered":"2023-09-23T03:25:48.656104+0000","last_clean":"2023-09-23T03:25:48.656104+0000","last_became_active":"2023-09-23T03:25:46.559990+0000","last_became_peered":"2023-09-23T03:25:46.559990+0000","last_unstale":"2023-09-23T03:25:48.656104+0000","last_undegraded":"2023-09-23T03:25:48.656104+0000","last_fullsized":"2023-09-23T03:25:48.656104+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:52:11.535392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.544432+0000","last_change":"2023-09-23T03:25:47.576834+0000","last_active":"2023-09-23T03:25:48.544432+0000","last_peered":"2023-09-23T03:25:48.544432+0000","last_clean":"2023-09-23T03:25:48.544432+0000","last_became_active":"2023-09-23T03:25:47.576583+0000","last_became_peered":"2023-09-23T03:25:47.576583+0000","last_unstale":"2023-09-23T03:25:48.544432+0000","last_undegraded":"2023-09-23T03:25:48.544432+0000","last_fullsized":"2023-09-23T03:25:48.544432+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:10:44.788203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.543111+0000","last_change":"2023-09-23T03:25:46.559387+0000","last_active":"2023-09-23T03:25:48.543111+0000","last_peered":"2023-09-23T03:25:48.543111+0000","last_clean":"2023-09-23T03:25:48.543111+0000","last_became_active":"2023-09-23T03:25:46.559119+0000","last_became_peered":"2023-09-23T03:25:46.559119+0000","last_unstale":"2023-09-23T03:25:48.543111+0000","last_undegraded":"2023-09-23T03:25:48.543111+0000","last_fullsized":"2023-09-23T03:25:48.543111+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:47:09.875520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.544148+0000","last_change":"2023-09-23T03:25:47.569213+0000","last_active":"2023-09-23T03:25:48.544148+0000","last_peered":"2023-09-23T03:25:48.544148+0000","last_clean":"2023-09-23T03:25:48.544148+0000","last_became_active":"2023-09-23T03:25:47.568796+0000","last_became_peered":"2023-09-23T03:25:47.568796+0000","last_unstale":"2023-09-23T03:25:48.544148+0000","last_undegraded":"2023-09-23T03:25:48.544148+0000","last_fullsized":"2023-09-23T03:25:48.544148+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:48:16.248495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.542491+0000","last_change":"2023-09-23T03:25:46.558796+0000","last_active":"2023-09-23T03:25:48.542491+0000","last_peered":"2023-09-23T03:25:48.542491+0000","last_clean":"2023-09-23T03:25:48.542491+0000","last_became_active":"2023-09-23T03:25:46.558505+0000","last_became_peered":"2023-09-23T03:25:46.558505+0000","last_unstale":"2023-09-23T03:25:48.542491+0000","last_undegraded":"2023-09-23T03:25:48.542491+0000","last_fullsized":"2023-09-23T03:25:48.542491+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:19:45.116812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.542495+0000","last_change":"2023-09-23T03:25:47.551566+0000","last_active":"2023-09-23T03:25:48.542495+0000","last_peered":"2023-09-23T03:25:48.542495+0000","last_clean":"2023-09-23T03:25:48.542495+0000","last_became_active":"2023-09-23T03:25:47.551253+0000","last_became_peered":"2023-09-23T03:25:47.551253+0000","last_unstale":"2023-09-23T03:25:48.542495+0000","last_undegraded":"2023-09-23T03:25:48.542495+0000","last_fullsized":"2023-09-23T03:25:48.542495+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:05:49.332342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.543004+0000","last_change":"2023-09-23T03:25:46.563687+0000","last_active":"2023-09-23T03:25:48.543004+0000","last_peered":"2023-09-23T03:25:48.543004+0000","last_clean":"2023-09-23T03:25:48.543004+0000","last_became_active":"2023-09-23T03:25:46.563287+0000","last_became_peered":"2023-09-23T03:25:46.563287+0000","last_unstale":"2023-09-23T03:25:48.543004+0000","last_undegraded":"2023-09-23T03:25:48.543004+0000","last_fullsized":"2023-09-23T03:25:48.543004+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:22:24.282357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.650349+0000","last_change":"2023-09-23T03:25:47.573670+0000","last_active":"2023-09-23T03:25:48.650349+0000","last_peered":"2023-09-23T03:25:48.650349+0000","last_clean":"2023-09-23T03:25:48.650349+0000","last_became_active":"2023-09-23T03:25:47.573098+0000","last_became_peered":"2023-09-23T03:25:47.573098+0000","last_unstale":"2023-09-23T03:25:48.650349+0000","last_undegraded":"2023-09-23T03:25:48.650349+0000","last_fullsized":"2023-09-23T03:25:48.650349+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:05:29.456594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.542235+0000","last_change":"2023-09-23T03:25:46.545088+0000","last_active":"2023-09-23T03:25:48.542235+0000","last_peered":"2023-09-23T03:25:48.542235+0000","last_clean":"2023-09-23T03:25:48.542235+0000","last_became_active":"2023-09-23T03:25:46.544372+0000","last_became_peered":"2023-09-23T03:25:46.544372+0000","last_unstale":"2023-09-23T03:25:48.542235+0000","last_undegraded":"2023-09-23T03:25:48.542235+0000","last_fullsized":"2023-09-23T03:25:48.542235+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:18:56.396735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.653850+0000","last_change":"2023-09-23T03:25:47.549323+0000","last_active":"2023-09-23T03:25:48.653850+0000","last_peered":"2023-09-23T03:25:48.653850+0000","last_clean":"2023-09-23T03:25:48.653850+0000","last_became_active":"2023-09-23T03:25:47.548909+0000","last_became_peered":"2023-09-23T03:25:47.548909+0000","last_unstale":"2023-09-23T03:25:48.653850+0000","last_undegraded":"2023-09-23T03:25:48.653850+0000","last_fullsized":"2023-09-23T03:25:48.653850+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:10:44.788203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.542050+0000","last_change":"2023-09-23T03:25:46.555570+0000","last_active":"2023-09-23T03:25:48.542050+0000","last_peered":"2023-09-23T03:25:48.542050+0000","last_clean":"2023-09-23T03:25:48.542050+0000","last_became_active":"2023-09-23T03:25:46.555285+0000","last_became_peered":"2023-09-23T03:25:46.555285+0000","last_unstale":"2023-09-23T03:25:48.542050+0000","last_undegraded":"2023-09-23T03:25:48.542050+0000","last_fullsized":"2023-09-23T03:25:48.542050+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:10:43.774259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.650087+0000","last_change":"2023-09-23T03:25:47.574240+0000","last_active":"2023-09-23T03:25:48.650087+0000","last_peered":"2023-09-23T03:25:48.650087+0000","last_clean":"2023-09-23T03:25:48.650087+0000","last_became_active":"2023-09-23T03:25:47.573682+0000","last_became_peered":"2023-09-23T03:25:47.573682+0000","last_unstale":"2023-09-23T03:25:48.650087+0000","last_undegraded":"2023-09-23T03:25:48.650087+0000","last_fullsized":"2023-09-23T03:25:48.650087+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:05:14.106449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.649925+0000","last_change":"2023-09-23T03:25:46.557313+0000","last_active":"2023-09-23T03:25:48.649925+0000","last_peered":"2023-09-23T03:25:48.649925+0000","last_clean":"2023-09-23T03:25:48.649925+0000","last_became_active":"2023-09-23T03:25:46.557025+0000","last_became_peered":"2023-09-23T03:25:46.557025+0000","last_unstale":"2023-09-23T03:25:48.649925+0000","last_undegraded":"2023-09-23T03:25:48.649925+0000","last_fullsized":"2023-09-23T03:25:48.649925+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:34:23.970826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.542142+0000","last_change":"2023-09-23T03:25:47.552785+0000","last_active":"2023-09-23T03:25:48.542142+0000","last_peered":"2023-09-23T03:25:48.542142+0000","last_clean":"2023-09-23T03:25:48.542142+0000","last_became_active":"2023-09-23T03:25:47.552497+0000","last_became_peered":"2023-09-23T03:25:47.552497+0000","last_unstale":"2023-09-23T03:25:48.542142+0000","last_undegraded":"2023-09-23T03:25:48.542142+0000","last_fullsized":"2023-09-23T03:25:48.542142+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:17:45.061241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.650067+0000","last_change":"2023-09-23T03:25:46.557092+0000","last_active":"2023-09-23T03:25:48.650067+0000","last_peered":"2023-09-23T03:25:48.650067+0000","last_clean":"2023-09-23T03:25:48.650067+0000","last_became_active":"2023-09-23T03:25:46.556860+0000","last_became_peered":"2023-09-23T03:25:46.556860+0000","last_unstale":"2023-09-23T03:25:48.650067+0000","last_undegraded":"2023-09-23T03:25:48.650067+0000","last_fullsized":"2023-09-23T03:25:48.650067+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:19:58.683118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.650775+0000","last_change":"2023-09-23T03:25:47.574919+0000","last_active":"2023-09-23T03:25:48.650775+0000","last_peered":"2023-09-23T03:25:48.650775+0000","last_clean":"2023-09-23T03:25:48.650775+0000","last_became_active":"2023-09-23T03:25:47.574591+0000","last_became_peered":"2023-09-23T03:25:47.574591+0000","last_unstale":"2023-09-23T03:25:48.650775+0000","last_undegraded":"2023-09-23T03:25:48.650775+0000","last_fullsized":"2023-09-23T03:25:48.650775+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:44:06.391679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"104'1","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.654354+0000","last_change":"2023-09-23T03:25:46.567250+0000","last_active":"2023-09-23T03:25:48.654354+0000","last_peered":"2023-09-23T03:25:48.654354+0000","last_clean":"2023-09-23T03:25:48.654354+0000","last_became_active":"2023-09-23T03:25:46.566955+0000","last_became_peered":"2023-09-23T03:25:46.566955+0000","last_unstale":"2023-09-23T03:25:48.654354+0000","last_undegraded":"2023-09-23T03:25:48.654354+0000","last_fullsized":"2023-09-23T03:25:48.654354+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:51:48.156040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":238,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-09-23T03:25:47.553458+0000","last_change":"2023-09-23T03:15:11.042447+0000","last_active":"2023-09-23T03:25:47.553458+0000","last_peered":"2023-09-23T03:25:47.553458+0000","last_clean":"2023-09-23T03:25:47.553458+0000","last_became_active":"2023-09-23T03:15:11.040322+0000","last_became_peered":"2023-09-23T03:15:11.040322+0000","last_unstale":"2023-09-23T03:25:47.553458+0000","last_undegraded":"2023-09-23T03:25:47.553458+0000","last_fullsized":"2023-09-23T03:25:47.553458+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_clean_scrub_stamp":"2023-09-23T03:15:09.474842+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:18:36.331645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574978,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33024,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338816,"statfs":{"total":96636764160,"available":96602947584,"internally_reserved":0,"allocated":1888256,"data_stored":1212750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:25:31 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.036,"15min":1.0049999999999999},"min":{"1min":0.751,"5min":0.57999999999999996,"15min":0.442},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.149},"last":0.68000000000000005},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.0660000000000001,"15min":1.0549999999999999},"min":{"1min":0.84099999999999997,"5min":0.67200000000000004,"15min":0.52000000000000002},"max":{"1min":1.9910000000000001,"5min":1.9910000000000001,"15min":2.1480000000000001},"last":0.93100000000000005}]},{"osd":1,"last update":"Sat Sep 23 03:25:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.23,"15min":1.2170000000000001},"min":{"1min":0.76500000000000001,"5min":0.66400000000000003,"15min":0.504},"max":{"1min":2.6400000000000001,"5min":2.6400000000000001,"15min":3.1859999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.2370000000000001,"15min":1.3080000000000001},"min":{"1min":0.85499999999999998,"5min":0.71299999999999997,"15min":0.67900000000000005},"max":{"1min":2.3149999999999999,"5min":2.3149999999999999,"15min":2.613},"last":1.101}]},{"osd":2,"last update":"Sat Sep 23 03:25:31 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.1180000000000001,"15min":1.1339999999999999},"min":{"1min":0.70399999999999996,"5min":0.61199999999999999,"15min":0.42399999999999999},"max":{"1min":1.4850000000000001,"5min":1.6439999999999999,"15min":2.331},"last":1.577},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.222,"15min":1.2490000000000001},"min":{"1min":1.0449999999999999,"5min":0.63200000000000001,"15min":0.55200000000000005},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":3.093},"last":1.054}]},{"osd":3,"last update":"Sat Sep 23 03:25:31 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.2010000000000001,"15min":1.2509999999999999},"min":{"1min":0.86899999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.9530000000000001,"5min":1.9530000000000001,"15min":2.532},"last":1.25},{"interface":"front","average":{"1min":1.514,"5min":1.3360000000000001,"15min":1.379},"min":{"1min":1.2310000000000001,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.4289999999999998,"5min":2.4289999999999998,"15min":3.2349999999999999},"last":0.89700000000000002}]},{"osd":4,"last update":"Sat Sep 23 03:25:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.2749999999999999,"15min":1.4099999999999999},"min":{"1min":0.879,"5min":0.57999999999999996,"15min":0.45400000000000001},"max":{"1min":2.214,"5min":2.214,"15min":2.7210000000000001},"last":1.198},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.4370000000000001,"15min":1.492},"min":{"1min":1.2270000000000001,"5min":0.73599999999999999,"15min":0.48099999999999998},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":3.7719999999999998},"last":1.6639999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:25:31 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.3879999999999999,"15min":1.427},"min":{"1min":0.98399999999999999,"5min":0.749,"15min":0.47999999999999998},"max":{"1min":2.5390000000000001,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":1.151},{"interface":"front","average":{"1min":1.375,"5min":1.371,"15min":1.4610000000000001},"min":{"1min":0.96999999999999997,"5min":0.77000000000000002,"15min":0.497},"max":{"1min":2.089,"5min":2.089,"15min":2.4689999999999999},"last":1.3320000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:25:31 2023","interfaces":[{"interface":"back","average":{"1min":1.522,"5min":1.47,"15min":1.5660000000000001},"min":{"1min":1.121,"5min":0.85499999999999998,"15min":0.53800000000000003},"max":{"1min":2.395,"5min":2.395,"15min":2.8420000000000001},"last":1.419},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.4330000000000001,"15min":1.514},"min":{"1min":1.111,"5min":0.78900000000000003,"15min":0.45900000000000002},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":3.6739999999999999},"last":1.468}]}]},{"osd":6,"up_from":13,"seq":55834574978,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32460,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339380,"statfs":{"total":96636764160,"available":96603525120,"internally_reserved":0,"allocated":1306624,"data_stored":629542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:25:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.1439999999999999,"15min":1.0780000000000001},"min":{"1min":0.90600000000000003,"5min":0.55000000000000004,"15min":0.38},"max":{"1min":1.722,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.292},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1319999999999999,"15min":1.0800000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.46100000000000002},"max":{"1min":1.8149999999999999,"5min":3.7629999999999999,"15min":3.7629999999999999},"last":0.82999999999999996}]},{"osd":1,"last update":"Sat Sep 23 03:25:22 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.149,"15min":1.119},"min":{"1min":0.68999999999999995,"5min":0.59099999999999997,"15min":0.438},"max":{"1min":2.2400000000000002,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1759999999999999,"15min":1.2030000000000001},"min":{"1min":0.56599999999999995,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.7829999999999999,"5min":2.972,"15min":2.972},"last":0.72399999999999998}]},{"osd":2,"last update":"Sat Sep 23 03:25:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.1719999999999999,"15min":1.1279999999999999},"min":{"1min":0.70799999999999996,"5min":0.64700000000000002,"15min":0.51300000000000001},"max":{"1min":1.76,"5min":3.4390000000000001,"15min":3.4390000000000001},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.2290000000000001,"15min":1.246},"min":{"1min":0.84099999999999997,"5min":0.66200000000000003,"15min":0.64300000000000002},"max":{"1min":2.177,"5min":2.177,"15min":2.177},"last":1.627}]},{"osd":3,"last update":"Sat Sep 23 03:25:22 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.331,"15min":1.3420000000000001},"min":{"1min":1.0549999999999999,"5min":0.72399999999999998,"15min":0.54600000000000004},"max":{"1min":2.129,"5min":2.129,"15min":2.6509999999999998},"last":1.7250000000000001},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.3740000000000001,"15min":1.4039999999999999},"min":{"1min":1.155,"5min":0.84599999999999997,"15min":0.74299999999999999},"max":{"1min":1.853,"5min":3.0910000000000002,"15min":3.0910000000000002},"last":1.1160000000000001}]},{"osd":4,"last update":"Sat Sep 23 03:25:22 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.375,"15min":1.381},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.498},"max":{"1min":2.024,"5min":3.1789999999999998,"15min":3.1789999999999998},"last":1.494},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.3520000000000001,"15min":1.304},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.56000000000000005},"max":{"1min":1.9590000000000001,"5min":3.6480000000000001,"15min":3.6480000000000001},"last":1.212}]},{"osd":5,"last update":"Sat Sep 23 03:25:22 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.456,"15min":1.413},"min":{"1min":1.081,"5min":1.081,"15min":0.69199999999999995},"max":{"1min":2.1579999999999999,"5min":4.093,"15min":4.093},"last":1.421},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.4330000000000001,"15min":1.458},"min":{"1min":1.099,"5min":0.86699999999999999,"15min":0.59899999999999998},"max":{"1min":1.9710000000000001,"5min":1.996,"15min":4.758},"last":1.9379999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:25:22 2023","interfaces":[{"interface":"back","average":{"1min":1.5780000000000001,"5min":1.5449999999999999,"15min":1.538},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.63500000000000001},"max":{"1min":2.1379999999999999,"5min":3.536,"15min":3.536},"last":1.748},{"interface":"front","average":{"1min":1.575,"5min":1.534,"15min":1.5449999999999999},"min":{"1min":1.2310000000000001,"5min":0.82799999999999996,"15min":0.67300000000000004},"max":{"1min":1.9950000000000001,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":1.843}]}]},{"osd":5,"up_from":13,"seq":55834574980,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1290240,"data_stored":622340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:25:19 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.026,"15min":1.0309999999999999},"min":{"1min":0.75,"5min":0.61199999999999999,"15min":0.57199999999999995},"max":{"1min":1.4910000000000001,"5min":1.732,"15min":1.732},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.163,"5min":1.1120000000000001,"15min":1.0669999999999999},"min":{"1min":0.83899999999999997,"5min":0.624,"15min":0.61399999999999999},"max":{"1min":1.5600000000000001,"5min":2.2320000000000002,"15min":2.2320000000000002},"last":1.278}]},{"osd":1,"last update":"Sat Sep 23 03:25:19 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.2350000000000001,"15min":1.2210000000000001},"min":{"1min":0.71899999999999997,"5min":0.68300000000000005,"15min":0.62},"max":{"1min":1.484,"5min":1.657,"15min":1.849},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.2729999999999999,"15min":1.294},"min":{"1min":0.92900000000000005,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.6160000000000001,"5min":1.724,"15min":1.893},"last":1.456}]},{"osd":2,"last update":"Sat Sep 23 03:25:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.1659999999999999,"15min":1.2},"min":{"1min":0.88700000000000001,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.641,"5min":1.726,"15min":1.726},"last":1.478},{"interface":"front","average":{"1min":1.107,"5min":1.171,"15min":1.24},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.71599999999999997},"max":{"1min":1.516,"5min":1.5940000000000001,"15min":1.7629999999999999},"last":1.0369999999999999}]},{"osd":3,"last update":"Sat Sep 23 03:25:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.24,"15min":1.288},"min":{"1min":0.86599999999999999,"5min":0.67200000000000004,"15min":0.66400000000000003},"max":{"1min":1.5549999999999999,"5min":1.696,"15min":1.905},"last":1.349},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.3100000000000001,"15min":1.387},"min":{"1min":0.88900000000000001,"5min":0.71699999999999997,"15min":0.69799999999999995},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":3.6110000000000002},"last":0.97599999999999998}]},{"osd":4,"last update":"Sat Sep 23 03:25:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.3640000000000001,"15min":1.401},"min":{"1min":1.0309999999999999,"5min":0.96599999999999997,"15min":0.54100000000000004},"max":{"1min":1.9430000000000001,"5min":1.9430000000000001,"15min":2.069},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.268,"5min":1.2689999999999999,"15min":1.375},"min":{"1min":0.95899999999999996,"5min":0.60299999999999998,"15min":0.48899999999999999},"max":{"1min":1.6739999999999999,"5min":1.6899999999999999,"15min":2.319},"last":1.3109999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:25:19 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.4850000000000001,"15min":1.514},"min":{"1min":1.0649999999999999,"5min":1.0289999999999999,"15min":0.59199999999999997},"max":{"1min":2.1339999999999999,"5min":2.1339999999999999,"15min":2.1509999999999998},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.343,"5min":1.5029999999999999,"15min":1.552},"min":{"1min":0.91400000000000003,"5min":0.78700000000000003,"15min":0.55100000000000005},"max":{"1min":1.681,"5min":9.6560000000000006,"15min":9.6560000000000006},"last":1.6060000000000001}]},{"osd":7,"last update":"Sat Sep 23 03:25:19 2023","interfaces":[{"interface":"back","average":{"1min":1.468,"5min":1.4099999999999999,"15min":1.466},"min":{"1min":1.05,"5min":0.89100000000000001,"15min":0.626},"max":{"1min":1.7430000000000001,"5min":1.8560000000000001,"15min":2.0350000000000001},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.4830000000000001,"15min":1.5700000000000001},"min":{"1min":1.232,"5min":0.85999999999999999,"15min":0.60799999999999998},"max":{"1min":1.7909999999999999,"5min":1.8140000000000001,"15min":2.1960000000000002},"last":1.4219999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574979,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1318912,"data_stored":630626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:25:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.048,"15min":1.1040000000000001},"min":{"1min":0.68400000000000005,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.488,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.5529999999999999},{"interface":"front","average":{"1min":1.23,"5min":1.169,"15min":1.1619999999999999},"min":{"1min":0.78600000000000003,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.5960000000000001,"5min":2.004,"15min":2.004},"last":1.0600000000000001}]},{"osd":1,"last update":"Sat Sep 23 03:25:22 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.91500000000000004,"5min":0.64000000000000001,"15min":0.61799999999999999},"max":{"1min":1.7969999999999999,"5min":1.8200000000000001,"15min":2.1110000000000002},"last":1.296},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.056,"15min":1.151},"min":{"1min":0.68300000000000005,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.508,"5min":1.718,"15min":3.2189999999999999},"last":1.409}]},{"osd":2,"last update":"Sat Sep 23 03:25:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.157,"15min":1.1930000000000001},"min":{"1min":0.81100000000000005,"5min":0.75600000000000001,"15min":0.61599999999999999},"max":{"1min":1.619,"5min":1.7450000000000001,"15min":2.5920000000000001},"last":1.694},{"interface":"front","average":{"1min":1.216,"5min":1.2490000000000001,"15min":1.2350000000000001},"min":{"1min":0.79200000000000004,"5min":0.69999999999999996,"15min":0.68300000000000005},"max":{"1min":1.7110000000000001,"5min":1.9530000000000001,"15min":1.9530000000000001},"last":1.179}]},{"osd":3,"last update":"Sat Sep 23 03:25:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1850000000000001,"15min":1.1699999999999999},"min":{"1min":0.91600000000000004,"5min":0.71599999999999997,"15min":0.69199999999999995},"max":{"1min":1.429,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.254,"5min":1.284,"15min":1.3089999999999999},"min":{"1min":0.88300000000000001,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.5920000000000001,"5min":2.0609999999999999,"15min":2.4969999999999999},"last":1.347}]},{"osd":5,"last update":"Sat Sep 23 03:25:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3620000000000001,"15min":1.359},"min":{"1min":0.88900000000000001,"5min":0.83099999999999996,"15min":0.70199999999999996},"max":{"1min":1.669,"5min":1.756,"15min":2.8050000000000002},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.343,"15min":1.3360000000000001},"min":{"1min":1.012,"5min":0.878,"15min":0.81799999999999995},"max":{"1min":1.663,"5min":2.0539999999999998,"15min":2.887},"last":1.607}]},{"osd":6,"last update":"Sat Sep 23 03:25:22 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.379,"15min":1.383},"min":{"1min":1.119,"5min":0.81299999999999994,"15min":0.66400000000000003},"max":{"1min":1.6519999999999999,"5min":2.113,"15min":2.4249999999999998},"last":1.7769999999999999},{"interface":"front","average":{"1min":1.478,"5min":1.488,"15min":1.4910000000000001},"min":{"1min":1.107,"5min":0.97199999999999998,"15min":0.89900000000000002},"max":{"1min":1.8779999999999999,"5min":2.0800000000000001,"15min":2.1779999999999999},"last":1.6519999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:25:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.452,"15min":1.4650000000000001},"min":{"1min":1.1539999999999999,"5min":0.98099999999999998,"15min":0.81399999999999995},"max":{"1min":1.7929999999999999,"5min":2.1970000000000001,"15min":2.5680000000000001},"last":1.7370000000000001},{"interface":"front","average":{"1min":1.5800000000000001,"5min":1.556,"15min":1.5589999999999999},"min":{"1min":1.3859999999999999,"5min":1.1200000000000001,"15min":0.84199999999999997},"max":{"1min":1.786,"5min":2.2400000000000002,"15min":2.4769999999999999},"last":1.8280000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574979,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1314816,"data_stored":630524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:25:35 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.88800000000000001,"15min":0.873},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.254,"5min":2.952,"15min":2.952},"last":1.208},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.90400000000000003,"15min":0.89400000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.29299999999999998},"max":{"1min":1.4710000000000001,"5min":1.7430000000000001,"15min":1.8979999999999999},"last":1.2709999999999999}]},{"osd":1,"last update":"Sat Sep 23 03:25:35 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.82899999999999996,"15min":0.89800000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.419,"5min":1.603,"15min":1.6839999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.02,"15min":0.98899999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.45400000000000001},"max":{"1min":1.7010000000000001,"5min":2.903,"15min":2.903},"last":1.351}]},{"osd":2,"last update":"Sat Sep 23 03:25:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.024,"15min":1.0680000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.4830000000000001,"5min":1.9850000000000001,"15min":2.2240000000000002},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1160000000000001,"15min":1.109},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.7190000000000001,"5min":3.129,"15min":3.129},"last":1.506}]},{"osd":4,"last update":"Sat Sep 23 03:25:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.167,"15min":1.25},"min":{"1min":0.82299999999999995,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.5900000000000001,"5min":1.671,"15min":2.698},"last":0.998},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.2769999999999999,"15min":1.371},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.752},"max":{"1min":1.661,"5min":1.6990000000000001,"15min":2.419},"last":1.141}]},{"osd":5,"last update":"Sat Sep 23 03:25:35 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.349,"15min":1.349},"min":{"1min":0.82599999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.8799999999999999,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":1.6200000000000001},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.25,"15min":1.3029999999999999},"min":{"1min":0.77800000000000002,"5min":0.624,"15min":0.624},"max":{"1min":1.607,"5min":1.841,"15min":2.0070000000000001},"last":1.401}]},{"osd":6,"last update":"Sat Sep 23 03:25:35 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.327,"15min":1.3759999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.69799999999999995},"max":{"1min":1.9530000000000001,"5min":1.9530000000000001,"15min":2.6440000000000001},"last":1.319},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4239999999999999,"15min":1.4470000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.67800000000000005},"max":{"1min":1.95,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":2.177}]},{"osd":7,"last update":"Sat Sep 23 03:25:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.339,"15min":1.4139999999999999},"min":{"1min":0.875,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.7879999999999998},"last":1.591},{"interface":"front","average":{"1min":1.464,"5min":1.3779999999999999,"15min":1.4550000000000001},"min":{"1min":0.81000000000000005,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.0270000000000001,"5min":2.0270000000000001,"15min":2.5379999999999998},"last":1.5549999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574980,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1290240,"data_stored":622304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.86399999999999999,"15min":0.84099999999999997},"min":{"1min":0.40799999999999997,"5min":0.371,"15min":0.30299999999999999},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.0569999999999999,"5min":0.91800000000000004,"15min":0.87},"min":{"1min":0.46999999999999997,"5min":0.39300000000000002,"15min":0.36299999999999999},"max":{"1min":2.613,"5min":2.613,"15min":2.613},"last":0.71899999999999997}]},{"osd":1,"last update":"Sat Sep 23 03:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.85999999999999999,"15min":0.85899999999999999},"min":{"1min":0.628,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.0249999999999999,"15min":0.96899999999999997},"min":{"1min":0.59399999999999997,"5min":0.49399999999999999,"15min":0.38},"max":{"1min":2.9809999999999999,"5min":2.9809999999999999,"15min":2.9809999999999999},"last":0.78700000000000003}]},{"osd":3,"last update":"Sat Sep 23 03:25:19 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.032,"15min":0.98199999999999998},"min":{"1min":0.64300000000000002,"5min":0.51600000000000001,"15min":0.40699999999999997},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":1.115},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.0429999999999999,"15min":1.0049999999999999},"min":{"1min":0.52800000000000002,"5min":0.441,"15min":0.34999999999999998},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.86499999999999999}]},{"osd":4,"last update":"Sat Sep 23 03:25:19 2023","interfaces":[{"interface":"back","average":{"1min":1.5580000000000001,"5min":1.3080000000000001,"15min":1.3},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.55800000000000005},"max":{"1min":5.6669999999999998,"5min":5.6669999999999998,"15min":5.6669999999999998},"last":1.214},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.2809999999999999,"15min":1.3100000000000001},"min":{"1min":0.82099999999999995,"5min":0.77200000000000002,"15min":0.76100000000000001},"max":{"1min":2.6909999999999998,"5min":2.6909999999999998,"15min":2.6909999999999998},"last":1.044}]},{"osd":5,"last update":"Sat Sep 23 03:25:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.3400000000000001,"15min":1.3600000000000001},"min":{"1min":0.92800000000000005,"5min":0.78600000000000003,"15min":0.71499999999999997},"max":{"1min":2.8969999999999998,"5min":2.8969999999999998,"15min":3.9409999999999998},"last":1.071},{"interface":"front","average":{"1min":1.506,"5min":1.389,"15min":1.448},"min":{"1min":1.0049999999999999,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":3.415,"5min":3.415,"15min":4.9199999999999999},"last":1.292}]},{"osd":6,"last update":"Sat Sep 23 03:25:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.353,"15min":1.379},"min":{"1min":1.1180000000000001,"5min":0.86499999999999999,"15min":0.755},"max":{"1min":2.7839999999999998,"5min":2.7839999999999998,"15min":3.0009999999999999},"last":1.349},{"interface":"front","average":{"1min":1.73,"5min":1.474,"15min":1.4490000000000001},"min":{"1min":0.76600000000000001,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":7.8300000000000001,"5min":7.8300000000000001,"15min":7.8300000000000001},"last":1.1659999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:25:19 2023","interfaces":[{"interface":"back","average":{"1min":1.5409999999999999,"5min":1.4379999999999999,"15min":1.4019999999999999},"min":{"1min":1.0389999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":4.0330000000000004,"5min":4.0330000000000004,"15min":4.0330000000000004},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.548,"5min":1.4259999999999999,"15min":1.4370000000000001},"min":{"1min":1.097,"5min":0.82599999999999996,"15min":0.74399999999999999},"max":{"1min":4.2199999999999998,"5min":4.2199999999999998,"15min":4.2199999999999998},"last":1.2470000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574981,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1298432,"data_stored":628890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:25:32 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.78700000000000003,"15min":0.79800000000000004},"min":{"1min":0.56699999999999995,"5min":0.41299999999999998,"15min":0.32300000000000001},"max":{"1min":1.2490000000000001,"5min":1.377,"15min":2.8279999999999998},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.86399999999999999,"15min":0.84799999999999998},"min":{"1min":0.67300000000000004,"5min":0.43099999999999999,"15min":0.312},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.8919999999999999},"last":0.64300000000000002}]},{"osd":2,"last update":"Sat Sep 23 03:25:32 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.88400000000000001,"15min":0.88200000000000001},"min":{"1min":0.64200000000000002,"5min":0.503,"15min":0.36699999999999999},"max":{"1min":1.581,"5min":1.581,"15min":2.3340000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.048,"15min":1.032},"min":{"1min":0.60999999999999999,"5min":0.48899999999999999,"15min":0.45600000000000002},"max":{"1min":1.4730000000000001,"5min":1.6499999999999999,"15min":1.9570000000000001},"last":0.45600000000000002}]},{"osd":3,"last update":"Sat Sep 23 03:25:32 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.0700000000000001,"15min":1.0609999999999999},"min":{"1min":0.82299999999999995,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":2.2370000000000001},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.111,"15min":1.1259999999999999},"min":{"1min":0.85599999999999998,"5min":0.50800000000000001,"15min":0.45900000000000002},"max":{"1min":1.659,"5min":1.659,"15min":3.2130000000000001},"last":0.84399999999999997}]},{"osd":4,"last update":"Sat Sep 23 03:25:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.276,"15min":1.2629999999999999},"min":{"1min":1.2390000000000001,"5min":0.81200000000000006,"15min":0.70699999999999996},"max":{"1min":1.7709999999999999,"5min":1.885,"15min":3.2629999999999999},"last":1.333},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.2549999999999999,"15min":1.2689999999999999},"min":{"1min":1.0529999999999999,"5min":0.77400000000000002,"15min":0.66500000000000004},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":2.4689999999999999},"last":1.0409999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:25:32 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.296,"15min":1.2849999999999999},"min":{"1min":1.1200000000000001,"5min":0.76700000000000002,"15min":0.64600000000000002},"max":{"1min":1.577,"5min":1.788,"15min":3.161},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.3520000000000001,"15min":1.337},"min":{"1min":1.208,"5min":0.81399999999999995,"15min":0.72399999999999998},"max":{"1min":1.746,"5min":1.8260000000000001,"15min":2.4289999999999998},"last":1.153}]},{"osd":6,"last update":"Sat Sep 23 03:25:32 2023","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":1.365,"15min":1.337},"min":{"1min":1.1499999999999999,"5min":0.83599999999999997,"15min":0.747},"max":{"1min":1.7050000000000001,"5min":1.915,"15min":3.0179999999999998},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.482,"5min":1.375,"15min":1.379},"min":{"1min":1.1160000000000001,"5min":0.82499999999999996,"15min":0.72799999999999998},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":3.3580000000000001},"last":1.228}]},{"osd":7,"last update":"Sat Sep 23 03:25:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.357,"15min":1.383},"min":{"1min":0.82799999999999996,"5min":0.79100000000000004,"15min":0.68600000000000005},"max":{"1min":1.6950000000000001,"5min":1.8300000000000001,"15min":3.1099999999999999},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.464,"5min":1.3999999999999999,"15min":1.4390000000000001},"min":{"1min":0.88900000000000001,"5min":0.82899999999999996,"15min":0.78500000000000003},"max":{"1min":1.802,"5min":1.917,"15min":3.306},"last":1.1779999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574981,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33016,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338824,"statfs":{"total":96636764160,"available":96602955776,"internally_reserved":0,"allocated":1880064,"data_stored":1212201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 23 03:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.85799999999999998,"15min":0.83899999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.30299999999999999},"max":{"1min":1.383,"5min":1.919,"15min":1.919},"last":1.1200000000000001},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.90000000000000002,"15min":0.89300000000000002},"min":{"1min":0.42599999999999999,"5min":0.41299999999999998,"15min":0.38500000000000001},"max":{"1min":1.3740000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":1.1739999999999999}]},{"osd":2,"last update":"Sat Sep 23 03:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.88600000000000001,"15min":0.83799999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.5249999999999999,"5min":3.089,"15min":3.089},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.004,"5min":1.0389999999999999,"15min":1.006},"min":{"1min":0.54100000000000004,"5min":0.49199999999999999,"15min":0.42499999999999999},"max":{"1min":1.4379999999999999,"5min":2.633,"15min":2.633},"last":0.92800000000000005}]},{"osd":3,"last update":"Sat Sep 23 03:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":1.016,"15min":0.96299999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.387,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.038,"5min":1.018,"15min":1.008},"min":{"1min":0.52900000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.802,"5min":2.012,"15min":2.012},"last":1.069}]},{"osd":4,"last update":"Sat Sep 23 03:25:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.3009999999999999,"15min":1.2529999999999999},"min":{"1min":0.78800000000000003,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7330000000000001,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":1.387},{"interface":"front","average":{"1min":1.175,"5min":1.27,"15min":1.2749999999999999},"min":{"1min":0.79600000000000004,"5min":0.79500000000000004,"15min":0.65300000000000002},"max":{"1min":1.7430000000000001,"5min":3.0230000000000001,"15min":3.0230000000000001},"last":1.427}]},{"osd":5,"last update":"Sat Sep 23 03:25:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.319,"15min":1.2609999999999999},"min":{"1min":0.81499999999999995,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.9550000000000001,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.379,"15min":1.3260000000000001},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.71999999999999997},"max":{"1min":1.9159999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.454}]},{"osd":6,"last update":"Sat Sep 23 03:25:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.371,"15min":1.3280000000000001},"min":{"1min":0.77700000000000002,"5min":0.77400000000000002,"15min":0.75600000000000001},"max":{"1min":2.0870000000000002,"5min":2.9489999999999998,"15min":2.9489999999999998},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.4470000000000001,"15min":1.3720000000000001},"min":{"1min":0.81100000000000005,"5min":0.71899999999999997,"15min":0.68999999999999995},"max":{"1min":1.8540000000000001,"5min":2.2429999999999999,"15min":2.3239999999999998},"last":1.5309999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:25:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.3839999999999999,"15min":1.3600000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.747},"max":{"1min":1.9390000000000001,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":1.21},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.458,"15min":1.4390000000000001},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.81699999999999995},"max":{"1min":1.968,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":1.3680000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-23T03:25:51.995 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-23T03:25:52.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.251+0000 7f5983bce640 1 Processor -- start 2023-09-23T03:25:52.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.251+0000 7f5983bce640 1 -- start start 2023-09-23T03:25:52.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.253+0000 7f5983bce640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f597c161010 0x7f597c0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:52.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.253+0000 7f5983bce640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f597c0c14e0 0x7f597c0c1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:52.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.253+0000 7f5983bce640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f597c0cb160 con 0x7f597c06c1d0 2023-09-23T03:25:52.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.253+0000 7f5983bce640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f597c0c1ea0 con 0x7f597c0c14e0 2023-09-23T03:25:52.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.253+0000 7f5983bce640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f597c0ccd20 con 0x7f597c161010 2023-09-23T03:25:52.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.253+0000 7f598216b640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f597c06c1d0 0x7f597c06c620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:49624/0 (socket says 172.21.15.154:49624) 2023-09-23T03:25:52.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.253+0000 7f598216b640 1 -- 172.21.15.154:0/1114285522 learned_addr learned my addr 172.21.15.154:0/1114285522 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:25:52.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.253+0000 7f5981169640 1 --2- 172.21.15.154:0/1114285522 >> v2:172.21.15.154:3300/0 conn(0x7f597c0c14e0 0x7f597c0c1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:52.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.254+0000 7f598196a640 1 --2- 172.21.15.154:0/1114285522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f597c161010 0x7f597c0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:52.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.254+0000 7f5981169640 1 -- 172.21.15.154:0/1114285522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f597c161010 msgr2=0x7f597c0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:52.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.254+0000 7f5981169640 1 --2- 172.21.15.154:0/1114285522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f597c161010 0x7f597c0cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:52.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.254+0000 7f5981169640 1 -- 172.21.15.154:0/1114285522 >> v1:172.21.15.154:6789/0 conn(0x7f597c06c1d0 legacy=0x7f597c06c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:52.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.254+0000 7f5980968640 1 -- 172.21.15.154:0/1114285522 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2332392093 0 0) 0x7f597c0cb160 con 0x7f597c06c1d0 2023-09-23T03:25:52.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.254+0000 7f5981169640 1 -- 172.21.15.154:0/1114285522 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f597c06cde0 con 0x7f597c0c14e0 2023-09-23T03:25:52.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.255+0000 7f5981169640 1 --2- 172.21.15.154:0/1114285522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f597c0c14e0 0x7f597c0c1960 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f596c0097a0 tx=0x7f596c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=302ec6055cdac2d0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:52.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.255+0000 7f5980968640 1 -- 172.21.15.154:0/1114285522 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c031780 con 0x7f597c0c14e0 2023-09-23T03:25:52.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.255+0000 7f5980968640 1 -- 172.21.15.154:0/1114285522 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596c0318e0 con 0x7f597c0c14e0 2023-09-23T03:25:52.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.255+0000 7f5980968640 1 -- 172.21.15.154:0/1114285522 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c031c70 con 0x7f597c0c14e0 2023-09-23T03:25:52.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.256+0000 7f5983bce640 1 -- 172.21.15.154:0/1114285522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f597c0c14e0 msgr2=0x7f597c0c1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:52.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.256+0000 7f5983bce640 1 --2- 172.21.15.154:0/1114285522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f597c0c14e0 0x7f597c0c1960 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f596c0097a0 tx=0x7f596c02b820 comp rx=0 tx=0).stop 2023-09-23T03:25:52.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.256+0000 7f5983bce640 1 -- 172.21.15.154:0/1114285522 shutdown_connections 2023-09-23T03:25:52.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.256+0000 7f5983bce640 1 --2- 172.21.15.154:0/1114285522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f597c0c14e0 0x7f597c0c1960 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:52.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.256+0000 7f5983bce640 1 --2- 172.21.15.154:0/1114285522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f597c161010 0x7f597c0cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:52.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.256+0000 7f5983bce640 1 -- 172.21.15.154:0/1114285522 >> 172.21.15.154:0/1114285522 conn(0x7f597c06a0f0 msgr2=0x7f597c06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:52.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.256+0000 7f5983bce640 1 -- 172.21.15.154:0/1114285522 shutdown_connections 2023-09-23T03:25:52.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.256+0000 7f5983bce640 1 -- 172.21.15.154:0/1114285522 wait complete. 2023-09-23T03:25:52.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.258+0000 7f5983bce640 1 Processor -- start 2023-09-23T03:25:52.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.258+0000 7f5983bce640 1 -- start start 2023-09-23T03:25:52.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.259+0000 7f5983bce640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f597c06c1d0 0x7f597c163f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:52.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.259+0000 7f5983bce640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f597c0c14e0 0x7f597c0ac190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:52.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.259+0000 7f5983bce640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f597c161010 0x7f597c0ac6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:52.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.259+0000 7f5983bce640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597c1619a0 con 0x7f597c161010 2023-09-23T03:25:52.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.259+0000 7f5983bce640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597c0c8450 con 0x7f597c0c14e0 2023-09-23T03:25:52.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.259+0000 7f5983bce640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f597c164460 con 0x7f597c06c1d0 2023-09-23T03:25:52.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.259+0000 7f598216b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f597c161010 0x7f597c0ac6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:52.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.260+0000 7f598216b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f597c161010 0x7f597c0ac6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48640/0 (socket says 172.21.15.154:48640) 2023-09-23T03:25:52.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.260+0000 7f598216b640 1 -- 172.21.15.154:0/2769386918 learned_addr learned my addr 172.21.15.154:0/2769386918 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:52.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.260+0000 7f5981169640 1 --2- 172.21.15.154:0/2769386918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f597c0c14e0 0x7f597c0ac190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:52.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.260+0000 7f598196a640 1 --2- 172.21.15.154:0/2769386918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f597c06c1d0 0x7f597c163f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:52.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.260+0000 7f598216b640 1 -- 172.21.15.154:0/2769386918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f597c06c1d0 msgr2=0x7f597c163f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:52.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.260+0000 7f598216b640 1 --2- 172.21.15.154:0/2769386918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f597c06c1d0 0x7f597c163f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:52.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.260+0000 7f598216b640 1 -- 172.21.15.154:0/2769386918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f597c0c14e0 msgr2=0x7f597c0ac190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:52.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.260+0000 7f598216b640 1 --2- 172.21.15.154:0/2769386918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f597c0c14e0 0x7f597c0ac190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:52.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.260+0000 7f598216b640 1 -- 172.21.15.154:0/2769386918 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596c009400 con 0x7f597c161010 2023-09-23T03:25:52.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.260+0000 7f598196a640 1 --2- 172.21.15.154:0/2769386918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f597c06c1d0 0x7f597c163f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:52.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.260+0000 7f598216b640 1 --2- 172.21.15.154:0/2769386918 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f597c161010 0x7f597c0ac6d0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f597400c7e0 tx=0x7f597400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:52.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.261+0000 7f59727fc640 1 -- 172.21.15.154:0/2769386918 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597400a640 con 0x7f597c161010 2023-09-23T03:25:52.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.261+0000 7f59727fc640 1 -- 172.21.15.154:0/2769386918 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f597400a7c0 con 0x7f597c161010 2023-09-23T03:25:52.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.261+0000 7f59727fc640 1 -- 172.21.15.154:0/2769386918 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597400ab70 con 0x7f597c161010 2023-09-23T03:25:52.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.261+0000 7f5983bce640 1 -- 172.21.15.154:0/2769386918 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f597c0c7e90 con 0x7f597c161010 2023-09-23T03:25:52.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.261+0000 7f5983bce640 1 -- 172.21.15.154:0/2769386918 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f597c163a00 con 0x7f597c161010 2023-09-23T03:25:52.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.262+0000 7f5983bce640 1 -- 172.21.15.154:0/2769386918 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f597c163bf0 con 0x7f597c161010 2023-09-23T03:25:52.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.263+0000 7f59727fc640 1 -- 172.21.15.154:0/2769386918 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f597400acd0 con 0x7f597c161010 2023-09-23T03:25:52.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.264+0000 7f59727fc640 1 --2- 172.21.15.154:0/2769386918 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f596006af30 0x7f596006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:52.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.264+0000 7f59727fc640 1 -- 172.21.15.154:0/2769386918 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5974088ee0 con 0x7f597c161010 2023-09-23T03:25:52.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.264+0000 7f598196a640 1 --2- 172.21.15.154:0/2769386918 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f596006af30 0x7f596006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:52.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.265+0000 7f598196a640 1 --2- 172.21.15.154:0/2769386918 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f596006af30 0x7f596006d3f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f5978009780 tx=0x7f5978008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:52.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.268+0000 7f59727fc640 1 -- 172.21.15.154:0/2769386918 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f5974053dd0 con 0x7f597c161010 2023-09-23T03:25:52.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.456+0000 7f5983bce640 1 -- 172.21.15.154:0/2769386918 --> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f597c163880 con 0x7f596006af30 2023-09-23T03:25:52.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.467+0000 7f59727fc640 1 -- 172.21.15.154:0/2769386918 <== mgr.4110 v2:172.21.15.154:6832/2422652104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287365 (secure 0 0 0) 0x7f5978009d60 con 0x7f596006af30 2023-09-23T03:25:52.471 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:25:52.473 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2023-09-23T03:25:52.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.474+0000 7f5983bce640 1 -- 172.21.15.154:0/2769386918 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f596006af30 msgr2=0x7f596006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:52.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.474+0000 7f5983bce640 1 --2- 172.21.15.154:0/2769386918 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f596006af30 0x7f596006d3f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f5978009780 tx=0x7f5978008040 comp rx=0 tx=0).stop 2023-09-23T03:25:52.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.474+0000 7f5983bce640 1 -- 172.21.15.154:0/2769386918 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f597c161010 msgr2=0x7f597c0ac6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:52.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.474+0000 7f5983bce640 1 --2- 172.21.15.154:0/2769386918 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f597c161010 0x7f597c0ac6d0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f597400c7e0 tx=0x7f597400cca0 comp rx=0 tx=0).stop 2023-09-23T03:25:52.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.474+0000 7f5983bce640 1 -- 172.21.15.154:0/2769386918 shutdown_connections 2023-09-23T03:25:52.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.474+0000 7f5983bce640 1 --2- 172.21.15.154:0/2769386918 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f596006af30 0x7f596006d3f0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:52.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.474+0000 7f5983bce640 1 --2- 172.21.15.154:0/2769386918 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f597c161010 0x7f597c0ac6d0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:52.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.474+0000 7f5983bce640 1 --2- 172.21.15.154:0/2769386918 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f597c0c14e0 0x7f597c0ac190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:52.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.474+0000 7f5983bce640 1 --2- 172.21.15.154:0/2769386918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f597c06c1d0 0x7f597c163f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:52.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.474+0000 7f5983bce640 1 -- 172.21.15.154:0/2769386918 >> 172.21.15.154:0/2769386918 conn(0x7f597c06a0f0 msgr2=0x7f597c164910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:52.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.475+0000 7f5983bce640 1 -- 172.21.15.154:0/2769386918 shutdown_connections 2023-09-23T03:25:52.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.475+0000 7f5983bce640 1 -- 172.21.15.154:0/2769386918 wait complete. 2023-09-23T03:25:52.492 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":419,"stamp":"2023-09-23T03:25:51.390602+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":146,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260772,"kb_used_data":11316,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713948,"statfs":{"total":773094113280,"available":772827082752,"internally_reserved":0,"allocated":11587584,"data_stored":6189177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":255315514},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.871181"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.544418+0000","last_change":"2023-09-23T03:25:47.562476+0000","last_active":"2023-09-23T03:25:48.544418+0000","last_peered":"2023-09-23T03:25:48.544418+0000","last_clean":"2023-09-23T03:25:48.544418+0000","last_became_active":"2023-09-23T03:25:47.560804+0000","last_became_peered":"2023-09-23T03:25:47.560804+0000","last_unstale":"2023-09-23T03:25:48.544418+0000","last_undegraded":"2023-09-23T03:25:48.544418+0000","last_fullsized":"2023-09-23T03:25:48.544418+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:08:49.981426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.541564+0000","last_change":"2023-09-23T03:25:47.558003+0000","last_active":"2023-09-23T03:25:48.541564+0000","last_peered":"2023-09-23T03:25:48.541564+0000","last_clean":"2023-09-23T03:25:48.541564+0000","last_became_active":"2023-09-23T03:25:47.557386+0000","last_became_peered":"2023-09-23T03:25:47.557386+0000","last_unstale":"2023-09-23T03:25:48.541564+0000","last_undegraded":"2023-09-23T03:25:48.541564+0000","last_fullsized":"2023-09-23T03:25:48.541564+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:12:15.263365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.544186+0000","last_change":"2023-09-23T03:25:47.562407+0000","last_active":"2023-09-23T03:25:48.544186+0000","last_peered":"2023-09-23T03:25:48.544186+0000","last_clean":"2023-09-23T03:25:48.544186+0000","last_became_active":"2023-09-23T03:25:47.560838+0000","last_became_peered":"2023-09-23T03:25:47.560838+0000","last_unstale":"2023-09-23T03:25:48.544186+0000","last_undegraded":"2023-09-23T03:25:48.544186+0000","last_fullsized":"2023-09-23T03:25:48.544186+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:44:28.405703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.650483+0000","last_change":"2023-09-23T03:25:47.568217+0000","last_active":"2023-09-23T03:25:48.650483+0000","last_peered":"2023-09-23T03:25:48.650483+0000","last_clean":"2023-09-23T03:25:48.650483+0000","last_became_active":"2023-09-23T03:25:47.567960+0000","last_became_peered":"2023-09-23T03:25:47.567960+0000","last_unstale":"2023-09-23T03:25:48.650483+0000","last_undegraded":"2023-09-23T03:25:48.650483+0000","last_fullsized":"2023-09-23T03:25:48.650483+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:36:29.064809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.542453+0000","last_change":"2023-09-23T03:25:47.552407+0000","last_active":"2023-09-23T03:25:48.542453+0000","last_peered":"2023-09-23T03:25:48.542453+0000","last_clean":"2023-09-23T03:25:48.542453+0000","last_became_active":"2023-09-23T03:25:47.552120+0000","last_became_peered":"2023-09-23T03:25:47.552120+0000","last_unstale":"2023-09-23T03:25:48.542453+0000","last_undegraded":"2023-09-23T03:25:48.542453+0000","last_fullsized":"2023-09-23T03:25:48.542453+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:22:29.984556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.652806+0000","last_change":"2023-09-23T03:25:47.565272+0000","last_active":"2023-09-23T03:25:48.652806+0000","last_peered":"2023-09-23T03:25:48.652806+0000","last_clean":"2023-09-23T03:25:48.652806+0000","last_became_active":"2023-09-23T03:25:47.564991+0000","last_became_peered":"2023-09-23T03:25:47.564991+0000","last_unstale":"2023-09-23T03:25:48.652806+0000","last_undegraded":"2023-09-23T03:25:48.652806+0000","last_fullsized":"2023-09-23T03:25:48.652806+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:30:23.312059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.652554+0000","last_change":"2023-09-23T03:25:47.563230+0000","last_active":"2023-09-23T03:25:48.652554+0000","last_peered":"2023-09-23T03:25:48.652554+0000","last_clean":"2023-09-23T03:25:48.652554+0000","last_became_active":"2023-09-23T03:25:47.562893+0000","last_became_peered":"2023-09-23T03:25:47.562893+0000","last_unstale":"2023-09-23T03:25:48.652554+0000","last_undegraded":"2023-09-23T03:25:48.652554+0000","last_fullsized":"2023-09-23T03:25:48.652554+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:11:41.423332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.541193+0000","last_change":"2023-09-23T03:25:47.567792+0000","last_active":"2023-09-23T03:25:48.541193+0000","last_peered":"2023-09-23T03:25:48.541193+0000","last_clean":"2023-09-23T03:25:48.541193+0000","last_became_active":"2023-09-23T03:25:47.564122+0000","last_became_peered":"2023-09-23T03:25:47.564122+0000","last_unstale":"2023-09-23T03:25:48.541193+0000","last_undegraded":"2023-09-23T03:25:48.541193+0000","last_fullsized":"2023-09-23T03:25:48.541193+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:47:17.666939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.653659+0000","last_change":"2023-09-23T03:25:47.567035+0000","last_active":"2023-09-23T03:25:48.653659+0000","last_peered":"2023-09-23T03:25:48.653659+0000","last_clean":"2023-09-23T03:25:48.653659+0000","last_became_active":"2023-09-23T03:25:47.566760+0000","last_became_peered":"2023-09-23T03:25:47.566760+0000","last_unstale":"2023-09-23T03:25:48.653659+0000","last_undegraded":"2023-09-23T03:25:48.653659+0000","last_fullsized":"2023-09-23T03:25:48.653659+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:08:49.981426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.543140+0000","last_change":"2023-09-23T03:25:47.556592+0000","last_active":"2023-09-23T03:25:48.543140+0000","last_peered":"2023-09-23T03:25:48.543140+0000","last_clean":"2023-09-23T03:25:48.543140+0000","last_became_active":"2023-09-23T03:25:47.556291+0000","last_became_peered":"2023-09-23T03:25:47.556291+0000","last_unstale":"2023-09-23T03:25:48.543140+0000","last_undegraded":"2023-09-23T03:25:48.543140+0000","last_fullsized":"2023-09-23T03:25:48.543140+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:40:12.805547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.652737+0000","last_change":"2023-09-23T03:25:47.571607+0000","last_active":"2023-09-23T03:25:48.652737+0000","last_peered":"2023-09-23T03:25:48.652737+0000","last_clean":"2023-09-23T03:25:48.652737+0000","last_became_active":"2023-09-23T03:25:47.570374+0000","last_became_peered":"2023-09-23T03:25:47.570374+0000","last_unstale":"2023-09-23T03:25:48.652737+0000","last_undegraded":"2023-09-23T03:25:48.652737+0000","last_fullsized":"2023-09-23T03:25:48.652737+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:34:22.165335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.654061+0000","last_change":"2023-09-23T03:25:47.559771+0000","last_active":"2023-09-23T03:25:48.654061+0000","last_peered":"2023-09-23T03:25:48.654061+0000","last_clean":"2023-09-23T03:25:48.654061+0000","last_became_active":"2023-09-23T03:25:47.558238+0000","last_became_peered":"2023-09-23T03:25:47.558238+0000","last_unstale":"2023-09-23T03:25:48.654061+0000","last_undegraded":"2023-09-23T03:25:48.654061+0000","last_fullsized":"2023-09-23T03:25:48.654061+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:26:17.879691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.650746+0000","last_change":"2023-09-23T03:25:47.556731+0000","last_active":"2023-09-23T03:25:48.650746+0000","last_peered":"2023-09-23T03:25:48.650746+0000","last_clean":"2023-09-23T03:25:48.650746+0000","last_became_active":"2023-09-23T03:25:47.556305+0000","last_became_peered":"2023-09-23T03:25:47.556305+0000","last_unstale":"2023-09-23T03:25:48.650746+0000","last_undegraded":"2023-09-23T03:25:48.650746+0000","last_fullsized":"2023-09-23T03:25:48.650746+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:11:52.433816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.649996+0000","last_change":"2023-09-23T03:25:47.577118+0000","last_active":"2023-09-23T03:25:48.649996+0000","last_peered":"2023-09-23T03:25:48.649996+0000","last_clean":"2023-09-23T03:25:48.649996+0000","last_became_active":"2023-09-23T03:25:47.576767+0000","last_became_peered":"2023-09-23T03:25:47.576767+0000","last_unstale":"2023-09-23T03:25:48.649996+0000","last_undegraded":"2023-09-23T03:25:48.649996+0000","last_fullsized":"2023-09-23T03:25:48.649996+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:23:53.245942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.649788+0000","last_change":"2023-09-23T03:25:47.564474+0000","last_active":"2023-09-23T03:25:48.649788+0000","last_peered":"2023-09-23T03:25:48.649788+0000","last_clean":"2023-09-23T03:25:48.649788+0000","last_became_active":"2023-09-23T03:25:47.560100+0000","last_became_peered":"2023-09-23T03:25:47.560100+0000","last_unstale":"2023-09-23T03:25:48.649788+0000","last_undegraded":"2023-09-23T03:25:48.649788+0000","last_fullsized":"2023-09-23T03:25:48.649788+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:53:45.800528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.653422+0000","last_change":"2023-09-23T03:25:47.564349+0000","last_active":"2023-09-23T03:25:48.653422+0000","last_peered":"2023-09-23T03:25:48.653422+0000","last_clean":"2023-09-23T03:25:48.653422+0000","last_became_active":"2023-09-23T03:25:47.563370+0000","last_became_peered":"2023-09-23T03:25:47.563370+0000","last_unstale":"2023-09-23T03:25:48.653422+0000","last_undegraded":"2023-09-23T03:25:48.653422+0000","last_fullsized":"2023-09-23T03:25:48.653422+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:14:48.030940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.543302+0000","last_change":"2023-09-23T03:25:47.550598+0000","last_active":"2023-09-23T03:25:48.543302+0000","last_peered":"2023-09-23T03:25:48.543302+0000","last_clean":"2023-09-23T03:25:48.543302+0000","last_became_active":"2023-09-23T03:25:47.550180+0000","last_became_peered":"2023-09-23T03:25:47.550180+0000","last_unstale":"2023-09-23T03:25:48.543302+0000","last_undegraded":"2023-09-23T03:25:48.543302+0000","last_fullsized":"2023-09-23T03:25:48.543302+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:22:11.155654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.650896+0000","last_change":"2023-09-23T03:25:47.581245+0000","last_active":"2023-09-23T03:25:48.650896+0000","last_peered":"2023-09-23T03:25:48.650896+0000","last_clean":"2023-09-23T03:25:48.650896+0000","last_became_active":"2023-09-23T03:25:47.580908+0000","last_became_peered":"2023-09-23T03:25:47.580908+0000","last_unstale":"2023-09-23T03:25:48.650896+0000","last_undegraded":"2023-09-23T03:25:48.650896+0000","last_fullsized":"2023-09-23T03:25:48.650896+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:11:41.423332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.650751+0000","last_change":"2023-09-23T03:25:47.572640+0000","last_active":"2023-09-23T03:25:48.650751+0000","last_peered":"2023-09-23T03:25:48.650751+0000","last_clean":"2023-09-23T03:25:48.650751+0000","last_became_active":"2023-09-23T03:25:47.572387+0000","last_became_peered":"2023-09-23T03:25:47.572387+0000","last_unstale":"2023-09-23T03:25:48.650751+0000","last_undegraded":"2023-09-23T03:25:48.650751+0000","last_fullsized":"2023-09-23T03:25:48.650751+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:19:26.573945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.654444+0000","last_change":"2023-09-23T03:25:47.566060+0000","last_active":"2023-09-23T03:25:48.654444+0000","last_peered":"2023-09-23T03:25:48.654444+0000","last_clean":"2023-09-23T03:25:48.654444+0000","last_became_active":"2023-09-23T03:25:47.565820+0000","last_became_peered":"2023-09-23T03:25:47.565820+0000","last_unstale":"2023-09-23T03:25:48.654444+0000","last_undegraded":"2023-09-23T03:25:48.654444+0000","last_fullsized":"2023-09-23T03:25:48.654444+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:26:10.003436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.650415+0000","last_change":"2023-09-23T03:25:47.570095+0000","last_active":"2023-09-23T03:25:48.650415+0000","last_peered":"2023-09-23T03:25:48.650415+0000","last_clean":"2023-09-23T03:25:48.650415+0000","last_became_active":"2023-09-23T03:25:47.569731+0000","last_became_peered":"2023-09-23T03:25:47.569731+0000","last_unstale":"2023-09-23T03:25:48.650415+0000","last_undegraded":"2023-09-23T03:25:48.650415+0000","last_fullsized":"2023-09-23T03:25:48.650415+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:43:42.824444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.541955+0000","last_change":"2023-09-23T03:25:47.562619+0000","last_active":"2023-09-23T03:25:48.541955+0000","last_peered":"2023-09-23T03:25:48.541955+0000","last_clean":"2023-09-23T03:25:48.541955+0000","last_became_active":"2023-09-23T03:25:47.562323+0000","last_became_peered":"2023-09-23T03:25:47.562323+0000","last_unstale":"2023-09-23T03:25:48.541955+0000","last_undegraded":"2023-09-23T03:25:48.541955+0000","last_fullsized":"2023-09-23T03:25:48.541955+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:23:48.640810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.541510+0000","last_change":"2023-09-23T03:25:47.561642+0000","last_active":"2023-09-23T03:25:48.541510+0000","last_peered":"2023-09-23T03:25:48.541510+0000","last_clean":"2023-09-23T03:25:48.541510+0000","last_became_active":"2023-09-23T03:25:47.561333+0000","last_became_peered":"2023-09-23T03:25:47.561333+0000","last_unstale":"2023-09-23T03:25:48.541510+0000","last_undegraded":"2023-09-23T03:25:48.541510+0000","last_fullsized":"2023-09-23T03:25:48.541510+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:55:49.357259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.652713+0000","last_change":"2023-09-23T03:25:47.564522+0000","last_active":"2023-09-23T03:25:48.652713+0000","last_peered":"2023-09-23T03:25:48.652713+0000","last_clean":"2023-09-23T03:25:48.652713+0000","last_became_active":"2023-09-23T03:25:47.563946+0000","last_became_peered":"2023-09-23T03:25:47.563946+0000","last_unstale":"2023-09-23T03:25:48.652713+0000","last_undegraded":"2023-09-23T03:25:48.652713+0000","last_fullsized":"2023-09-23T03:25:48.652713+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:31:13.203775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.543110+0000","last_change":"2023-09-23T03:25:47.555880+0000","last_active":"2023-09-23T03:25:48.543110+0000","last_peered":"2023-09-23T03:25:48.543110+0000","last_clean":"2023-09-23T03:25:48.543110+0000","last_became_active":"2023-09-23T03:25:47.555573+0000","last_became_peered":"2023-09-23T03:25:47.555573+0000","last_unstale":"2023-09-23T03:25:48.543110+0000","last_undegraded":"2023-09-23T03:25:48.543110+0000","last_fullsized":"2023-09-23T03:25:48.543110+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:45:16.921815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.542791+0000","last_change":"2023-09-23T03:25:47.558174+0000","last_active":"2023-09-23T03:25:48.542791+0000","last_peered":"2023-09-23T03:25:48.542791+0000","last_clean":"2023-09-23T03:25:48.542791+0000","last_became_active":"2023-09-23T03:25:47.557730+0000","last_became_peered":"2023-09-23T03:25:47.557730+0000","last_unstale":"2023-09-23T03:25:48.542791+0000","last_undegraded":"2023-09-23T03:25:48.542791+0000","last_fullsized":"2023-09-23T03:25:48.542791+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:11:28.607821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.653083+0000","last_change":"2023-09-23T03:25:47.575349+0000","last_active":"2023-09-23T03:25:48.653083+0000","last_peered":"2023-09-23T03:25:48.653083+0000","last_clean":"2023-09-23T03:25:48.653083+0000","last_became_active":"2023-09-23T03:25:47.575063+0000","last_became_peered":"2023-09-23T03:25:47.575063+0000","last_unstale":"2023-09-23T03:25:48.653083+0000","last_undegraded":"2023-09-23T03:25:48.653083+0000","last_fullsized":"2023-09-23T03:25:48.653083+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:19:20.436542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.542773+0000","last_change":"2023-09-23T03:25:47.551619+0000","last_active":"2023-09-23T03:25:48.542773+0000","last_peered":"2023-09-23T03:25:48.542773+0000","last_clean":"2023-09-23T03:25:48.542773+0000","last_became_active":"2023-09-23T03:25:47.551329+0000","last_became_peered":"2023-09-23T03:25:47.551329+0000","last_unstale":"2023-09-23T03:25:48.542773+0000","last_undegraded":"2023-09-23T03:25:48.542773+0000","last_fullsized":"2023-09-23T03:25:48.542773+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:15:51.790186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.541808+0000","last_change":"2023-09-23T03:25:47.556821+0000","last_active":"2023-09-23T03:25:48.541808+0000","last_peered":"2023-09-23T03:25:48.541808+0000","last_clean":"2023-09-23T03:25:48.541808+0000","last_became_active":"2023-09-23T03:25:47.556553+0000","last_became_peered":"2023-09-23T03:25:47.556553+0000","last_unstale":"2023-09-23T03:25:48.541808+0000","last_undegraded":"2023-09-23T03:25:48.541808+0000","last_fullsized":"2023-09-23T03:25:48.541808+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:01:43.955510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.650111+0000","last_change":"2023-09-23T03:25:47.573863+0000","last_active":"2023-09-23T03:25:48.650111+0000","last_peered":"2023-09-23T03:25:48.650111+0000","last_clean":"2023-09-23T03:25:48.650111+0000","last_became_active":"2023-09-23T03:25:47.573383+0000","last_became_peered":"2023-09-23T03:25:47.573383+0000","last_unstale":"2023-09-23T03:25:48.650111+0000","last_undegraded":"2023-09-23T03:25:48.650111+0000","last_fullsized":"2023-09-23T03:25:48.650111+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:50:07.532358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.649955+0000","last_change":"2023-09-23T03:25:47.570013+0000","last_active":"2023-09-23T03:25:48.649955+0000","last_peered":"2023-09-23T03:25:48.649955+0000","last_clean":"2023-09-23T03:25:48.649955+0000","last_became_active":"2023-09-23T03:25:47.567042+0000","last_became_peered":"2023-09-23T03:25:47.567042+0000","last_unstale":"2023-09-23T03:25:48.649955+0000","last_undegraded":"2023-09-23T03:25:48.649955+0000","last_fullsized":"2023-09-23T03:25:48.649955+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:58:09.113015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.543032+0000","last_change":"2023-09-23T03:25:47.561072+0000","last_active":"2023-09-23T03:25:48.543032+0000","last_peered":"2023-09-23T03:25:48.543032+0000","last_clean":"2023-09-23T03:25:48.543032+0000","last_became_active":"2023-09-23T03:25:47.560790+0000","last_became_peered":"2023-09-23T03:25:47.560790+0000","last_unstale":"2023-09-23T03:25:48.543032+0000","last_undegraded":"2023-09-23T03:25:48.543032+0000","last_fullsized":"2023-09-23T03:25:48.543032+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:44:06.391679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.650647+0000","last_change":"2023-09-23T03:25:47.574904+0000","last_active":"2023-09-23T03:25:48.650647+0000","last_peered":"2023-09-23T03:25:48.650647+0000","last_clean":"2023-09-23T03:25:48.650647+0000","last_became_active":"2023-09-23T03:25:47.574591+0000","last_became_peered":"2023-09-23T03:25:47.574591+0000","last_unstale":"2023-09-23T03:25:48.650647+0000","last_undegraded":"2023-09-23T03:25:48.650647+0000","last_fullsized":"2023-09-23T03:25:48.650647+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:37:15.720353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"104'4","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.543582+0000","last_change":"2023-09-23T03:25:46.563997+0000","last_active":"2023-09-23T03:25:48.543582+0000","last_peered":"2023-09-23T03:25:48.543582+0000","last_clean":"2023-09-23T03:25:48.543582+0000","last_became_active":"2023-09-23T03:25:46.563366+0000","last_became_peered":"2023-09-23T03:25:46.563366+0000","last_unstale":"2023-09-23T03:25:48.543582+0000","last_undegraded":"2023-09-23T03:25:48.543582+0000","last_fullsized":"2023-09-23T03:25:48.543582+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:05:12.835540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.650583+0000","last_change":"2023-09-23T03:25:47.571243+0000","last_active":"2023-09-23T03:25:48.650583+0000","last_peered":"2023-09-23T03:25:48.650583+0000","last_clean":"2023-09-23T03:25:48.650583+0000","last_became_active":"2023-09-23T03:25:47.570969+0000","last_became_peered":"2023-09-23T03:25:47.570969+0000","last_unstale":"2023-09-23T03:25:48.650583+0000","last_undegraded":"2023-09-23T03:25:48.650583+0000","last_fullsized":"2023-09-23T03:25:48.650583+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:51:49.169984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"104'1","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.654064+0000","last_change":"2023-09-23T03:25:46.565485+0000","last_active":"2023-09-23T03:25:48.654064+0000","last_peered":"2023-09-23T03:25:48.654064+0000","last_clean":"2023-09-23T03:25:48.654064+0000","last_became_active":"2023-09-23T03:25:46.565154+0000","last_became_peered":"2023-09-23T03:25:46.565154+0000","last_unstale":"2023-09-23T03:25:48.654064+0000","last_undegraded":"2023-09-23T03:25:48.654064+0000","last_fullsized":"2023-09-23T03:25:48.654064+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:36:28.050865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.544064+0000","last_change":"2023-09-23T03:25:47.566010+0000","last_active":"2023-09-23T03:25:48.544064+0000","last_peered":"2023-09-23T03:25:48.544064+0000","last_clean":"2023-09-23T03:25:48.544064+0000","last_became_active":"2023-09-23T03:25:47.565750+0000","last_became_peered":"2023-09-23T03:25:47.565750+0000","last_unstale":"2023-09-23T03:25:48.544064+0000","last_undegraded":"2023-09-23T03:25:48.544064+0000","last_fullsized":"2023-09-23T03:25:48.544064+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:05:10.550787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"104'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.542747+0000","last_change":"2023-09-23T03:25:46.560275+0000","last_active":"2023-09-23T03:25:48.542747+0000","last_peered":"2023-09-23T03:25:48.542747+0000","last_clean":"2023-09-23T03:25:48.542747+0000","last_became_active":"2023-09-23T03:25:46.560005+0000","last_became_peered":"2023-09-23T03:25:46.560005+0000","last_unstale":"2023-09-23T03:25:48.542747+0000","last_undegraded":"2023-09-23T03:25:48.542747+0000","last_fullsized":"2023-09-23T03:25:48.542747+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:19:42.909077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.654784+0000","last_change":"2023-09-23T03:25:47.562063+0000","last_active":"2023-09-23T03:25:48.654784+0000","last_peered":"2023-09-23T03:25:48.654784+0000","last_clean":"2023-09-23T03:25:48.654784+0000","last_became_active":"2023-09-23T03:25:47.561784+0000","last_became_peered":"2023-09-23T03:25:47.561784+0000","last_unstale":"2023-09-23T03:25:48.654784+0000","last_undegraded":"2023-09-23T03:25:48.654784+0000","last_fullsized":"2023-09-23T03:25:48.654784+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:52:12.549337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"104'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.541841+0000","last_change":"2023-09-23T03:25:46.558854+0000","last_active":"2023-09-23T03:25:48.541841+0000","last_peered":"2023-09-23T03:25:48.541841+0000","last_clean":"2023-09-23T03:25:48.541841+0000","last_became_active":"2023-09-23T03:25:46.558577+0000","last_became_peered":"2023-09-23T03:25:46.558577+0000","last_unstale":"2023-09-23T03:25:48.541841+0000","last_undegraded":"2023-09-23T03:25:48.541841+0000","last_fullsized":"2023-09-23T03:25:48.541841+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:31:12.189830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.655876+0000","last_change":"2023-09-23T03:25:47.555592+0000","last_active":"2023-09-23T03:25:48.655876+0000","last_peered":"2023-09-23T03:25:48.655876+0000","last_clean":"2023-09-23T03:25:48.655876+0000","last_became_active":"2023-09-23T03:25:47.555277+0000","last_became_peered":"2023-09-23T03:25:47.555277+0000","last_unstale":"2023-09-23T03:25:48.655876+0000","last_undegraded":"2023-09-23T03:25:48.655876+0000","last_fullsized":"2023-09-23T03:25:48.655876+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:57:17.479365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"104'1","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.543891+0000","last_change":"2023-09-23T03:25:46.556929+0000","last_active":"2023-09-23T03:25:48.543891+0000","last_peered":"2023-09-23T03:25:48.543891+0000","last_clean":"2023-09-23T03:25:48.543891+0000","last_became_active":"2023-09-23T03:25:46.556677+0000","last_became_peered":"2023-09-23T03:25:46.556677+0000","last_unstale":"2023-09-23T03:25:48.543891+0000","last_undegraded":"2023-09-23T03:25:48.543891+0000","last_fullsized":"2023-09-23T03:25:48.543891+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:37:14.706408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.652849+0000","last_change":"2023-09-23T03:25:47.571738+0000","last_active":"2023-09-23T03:25:48.652849+0000","last_peered":"2023-09-23T03:25:48.652849+0000","last_clean":"2023-09-23T03:25:48.652849+0000","last_became_active":"2023-09-23T03:25:47.571459+0000","last_became_peered":"2023-09-23T03:25:47.571459+0000","last_unstale":"2023-09-23T03:25:48.652849+0000","last_undegraded":"2023-09-23T03:25:48.652849+0000","last_fullsized":"2023-09-23T03:25:48.652849+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:14:16.668225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"104'1","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.542170+0000","last_change":"2023-09-23T03:25:46.555907+0000","last_active":"2023-09-23T03:25:48.542170+0000","last_peered":"2023-09-23T03:25:48.542170+0000","last_clean":"2023-09-23T03:25:48.542170+0000","last_became_active":"2023-09-23T03:25:46.555650+0000","last_became_peered":"2023-09-23T03:25:46.555650+0000","last_unstale":"2023-09-23T03:25:48.542170+0000","last_undegraded":"2023-09-23T03:25:48.542170+0000","last_fullsized":"2023-09-23T03:25:48.542170+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:14:47.016995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.649794+0000","last_change":"2023-09-23T03:25:47.559681+0000","last_active":"2023-09-23T03:25:48.649794+0000","last_peered":"2023-09-23T03:25:48.649794+0000","last_clean":"2023-09-23T03:25:48.649794+0000","last_became_active":"2023-09-23T03:25:47.559412+0000","last_became_peered":"2023-09-23T03:25:47.559412+0000","last_unstale":"2023-09-23T03:25:48.649794+0000","last_undegraded":"2023-09-23T03:25:48.649794+0000","last_fullsized":"2023-09-23T03:25:48.649794+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:24:59.861432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.542366+0000","last_change":"2023-09-23T03:25:46.567925+0000","last_active":"2023-09-23T03:25:48.542366+0000","last_peered":"2023-09-23T03:25:48.542366+0000","last_clean":"2023-09-23T03:25:48.542366+0000","last_became_active":"2023-09-23T03:25:46.567639+0000","last_became_peered":"2023-09-23T03:25:46.567639+0000","last_unstale":"2023-09-23T03:25:48.542366+0000","last_undegraded":"2023-09-23T03:25:48.542366+0000","last_fullsized":"2023-09-23T03:25:48.542366+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:26:16.865746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.653073+0000","last_change":"2023-09-23T03:25:47.563974+0000","last_active":"2023-09-23T03:25:48.653073+0000","last_peered":"2023-09-23T03:25:48.653073+0000","last_clean":"2023-09-23T03:25:48.653073+0000","last_became_active":"2023-09-23T03:25:47.563703+0000","last_became_peered":"2023-09-23T03:25:47.563703+0000","last_unstale":"2023-09-23T03:25:48.653073+0000","last_undegraded":"2023-09-23T03:25:48.653073+0000","last_fullsized":"2023-09-23T03:25:48.653073+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:44:28.405703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.543309+0000","last_change":"2023-09-23T03:25:46.559639+0000","last_active":"2023-09-23T03:25:48.543309+0000","last_peered":"2023-09-23T03:25:48.543309+0000","last_clean":"2023-09-23T03:25:48.543309+0000","last_became_active":"2023-09-23T03:25:46.559316+0000","last_became_peered":"2023-09-23T03:25:46.559316+0000","last_unstale":"2023-09-23T03:25:48.543309+0000","last_undegraded":"2023-09-23T03:25:48.543309+0000","last_fullsized":"2023-09-23T03:25:48.543309+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:43:41.810500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.542939+0000","last_change":"2023-09-23T03:25:47.552727+0000","last_active":"2023-09-23T03:25:48.542939+0000","last_peered":"2023-09-23T03:25:48.542939+0000","last_clean":"2023-09-23T03:25:48.542939+0000","last_became_active":"2023-09-23T03:25:47.552373+0000","last_became_peered":"2023-09-23T03:25:47.552373+0000","last_unstale":"2023-09-23T03:25:48.542939+0000","last_undegraded":"2023-09-23T03:25:48.542939+0000","last_fullsized":"2023-09-23T03:25:48.542939+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:34:06.551529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.544633+0000","last_change":"2023-09-23T03:25:46.560198+0000","last_active":"2023-09-23T03:25:48.544633+0000","last_peered":"2023-09-23T03:25:48.544633+0000","last_clean":"2023-09-23T03:25:48.544633+0000","last_became_active":"2023-09-23T03:25:46.559391+0000","last_became_peered":"2023-09-23T03:25:46.559391+0000","last_unstale":"2023-09-23T03:25:48.544633+0000","last_undegraded":"2023-09-23T03:25:48.544633+0000","last_fullsized":"2023-09-23T03:25:48.544633+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:19:25.560001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.654003+0000","last_change":"2023-09-23T03:25:47.564201+0000","last_active":"2023-09-23T03:25:48.654003+0000","last_peered":"2023-09-23T03:25:48.654003+0000","last_clean":"2023-09-23T03:25:48.654003+0000","last_became_active":"2023-09-23T03:25:47.563622+0000","last_became_peered":"2023-09-23T03:25:47.563622+0000","last_unstale":"2023-09-23T03:25:48.654003+0000","last_undegraded":"2023-09-23T03:25:48.654003+0000","last_fullsized":"2023-09-23T03:25:48.654003+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:04:29.550642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.650924+0000","last_change":"2023-09-23T03:25:46.550850+0000","last_active":"2023-09-23T03:25:48.650924+0000","last_peered":"2023-09-23T03:25:48.650924+0000","last_clean":"2023-09-23T03:25:48.650924+0000","last_became_active":"2023-09-23T03:25:46.550549+0000","last_became_peered":"2023-09-23T03:25:46.550549+0000","last_unstale":"2023-09-23T03:25:48.650924+0000","last_undegraded":"2023-09-23T03:25:48.650924+0000","last_fullsized":"2023-09-23T03:25:48.650924+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:40:11.791603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.542959+0000","last_change":"2023-09-23T03:25:47.555910+0000","last_active":"2023-09-23T03:25:48.542959+0000","last_peered":"2023-09-23T03:25:48.542959+0000","last_clean":"2023-09-23T03:25:48.542959+0000","last_became_active":"2023-09-23T03:25:47.555613+0000","last_became_peered":"2023-09-23T03:25:47.555613+0000","last_unstale":"2023-09-23T03:25:48.542959+0000","last_undegraded":"2023-09-23T03:25:48.542959+0000","last_fullsized":"2023-09-23T03:25:48.542959+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:34:24.984771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"104'3","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.654196+0000","last_change":"2023-09-23T03:25:46.564782+0000","last_active":"2023-09-23T03:25:48.654196+0000","last_peered":"2023-09-23T03:25:48.654196+0000","last_clean":"2023-09-23T03:25:48.654196+0000","last_became_active":"2023-09-23T03:25:46.564378+0000","last_became_peered":"2023-09-23T03:25:46.564378+0000","last_unstale":"2023-09-23T03:25:48.654196+0000","last_undegraded":"2023-09-23T03:25:48.654196+0000","last_fullsized":"2023-09-23T03:25:48.654196+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:58:08.099071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.653356+0000","last_change":"2023-09-23T03:25:47.561433+0000","last_active":"2023-09-23T03:25:48.653356+0000","last_peered":"2023-09-23T03:25:48.653356+0000","last_clean":"2023-09-23T03:25:48.653356+0000","last_became_active":"2023-09-23T03:25:47.561171+0000","last_became_peered":"2023-09-23T03:25:47.561171+0000","last_unstale":"2023-09-23T03:25:48.653356+0000","last_undegraded":"2023-09-23T03:25:48.653356+0000","last_fullsized":"2023-09-23T03:25:48.653356+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:09:06.345724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"104'6","reported_seq":18,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.655054+0000","last_change":"2023-09-23T03:25:46.563065+0000","last_active":"2023-09-23T03:25:48.655054+0000","last_peered":"2023-09-23T03:25:48.655054+0000","last_clean":"2023-09-23T03:25:48.655054+0000","last_became_active":"2023-09-23T03:25:46.562157+0000","last_became_peered":"2023-09-23T03:25:46.562157+0000","last_unstale":"2023-09-23T03:25:48.655054+0000","last_undegraded":"2023-09-23T03:25:48.655054+0000","last_fullsized":"2023-09-23T03:25:48.655054+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:00:57.861554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.650245+0000","last_change":"2023-09-23T03:25:47.564708+0000","last_active":"2023-09-23T03:25:48.650245+0000","last_peered":"2023-09-23T03:25:48.650245+0000","last_clean":"2023-09-23T03:25:48.650245+0000","last_became_active":"2023-09-23T03:25:47.564414+0000","last_became_peered":"2023-09-23T03:25:47.564414+0000","last_unstale":"2023-09-23T03:25:48.650245+0000","last_undegraded":"2023-09-23T03:25:48.650245+0000","last_fullsized":"2023-09-23T03:25:48.650245+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:53:32.559094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"104'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.650604+0000","last_change":"2023-09-23T03:25:46.560666+0000","last_active":"2023-09-23T03:25:48.650604+0000","last_peered":"2023-09-23T03:25:48.650604+0000","last_clean":"2023-09-23T03:25:48.650604+0000","last_became_active":"2023-09-23T03:25:46.559945+0000","last_became_peered":"2023-09-23T03:25:46.559945+0000","last_unstale":"2023-09-23T03:25:48.650604+0000","last_undegraded":"2023-09-23T03:25:48.650604+0000","last_fullsized":"2023-09-23T03:25:48.650604+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:47:22.988221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.543856+0000","last_change":"2023-09-23T03:25:47.576710+0000","last_active":"2023-09-23T03:25:48.543856+0000","last_peered":"2023-09-23T03:25:48.543856+0000","last_clean":"2023-09-23T03:25:48.543856+0000","last_became_active":"2023-09-23T03:25:47.576417+0000","last_became_peered":"2023-09-23T03:25:47.576417+0000","last_unstale":"2023-09-23T03:25:48.543856+0000","last_undegraded":"2023-09-23T03:25:48.543856+0000","last_fullsized":"2023-09-23T03:25:48.543856+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:00:58.875499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.650368+0000","last_change":"2023-09-23T03:25:46.557090+0000","last_active":"2023-09-23T03:25:48.650368+0000","last_peered":"2023-09-23T03:25:48.650368+0000","last_clean":"2023-09-23T03:25:48.650368+0000","last_became_active":"2023-09-23T03:25:46.556821+0000","last_became_peered":"2023-09-23T03:25:46.556821+0000","last_unstale":"2023-09-23T03:25:48.650368+0000","last_undegraded":"2023-09-23T03:25:48.650368+0000","last_fullsized":"2023-09-23T03:25:48.650368+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:46:13.084887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.652555+0000","last_change":"2023-09-23T03:25:47.561739+0000","last_active":"2023-09-23T03:25:48.652555+0000","last_peered":"2023-09-23T03:25:48.652555+0000","last_clean":"2023-09-23T03:25:48.652555+0000","last_became_active":"2023-09-23T03:25:47.561443+0000","last_became_peered":"2023-09-23T03:25:47.561443+0000","last_unstale":"2023-09-23T03:25:48.652555+0000","last_undegraded":"2023-09-23T03:25:48.652555+0000","last_fullsized":"2023-09-23T03:25:48.652555+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:48:16.248495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.653066+0000","last_change":"2023-09-23T03:25:46.557594+0000","last_active":"2023-09-23T03:25:48.653066+0000","last_peered":"2023-09-23T03:25:48.653066+0000","last_clean":"2023-09-23T03:25:48.653066+0000","last_became_active":"2023-09-23T03:25:46.557188+0000","last_became_peered":"2023-09-23T03:25:46.557188+0000","last_unstale":"2023-09-23T03:25:48.653066+0000","last_undegraded":"2023-09-23T03:25:48.653066+0000","last_fullsized":"2023-09-23T03:25:48.653066+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:43:41.810500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.650029+0000","last_change":"2023-09-23T03:25:47.572410+0000","last_active":"2023-09-23T03:25:48.650029+0000","last_peered":"2023-09-23T03:25:48.650029+0000","last_clean":"2023-09-23T03:25:48.650029+0000","last_became_active":"2023-09-23T03:25:47.572099+0000","last_became_peered":"2023-09-23T03:25:47.572099+0000","last_unstale":"2023-09-23T03:25:48.650029+0000","last_undegraded":"2023-09-23T03:25:48.650029+0000","last_fullsized":"2023-09-23T03:25:48.650029+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:37:41.517191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"104'8","reported_seq":20,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.543487+0000","last_change":"2023-09-23T03:25:46.565386+0000","last_active":"2023-09-23T03:25:48.543487+0000","last_peered":"2023-09-23T03:25:48.543487+0000","last_clean":"2023-09-23T03:25:48.543487+0000","last_became_active":"2023-09-23T03:25:46.564730+0000","last_became_peered":"2023-09-23T03:25:46.564730+0000","last_unstale":"2023-09-23T03:25:48.543487+0000","last_undegraded":"2023-09-23T03:25:48.543487+0000","last_fullsized":"2023-09-23T03:25:48.543487+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:25:32.261618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.543342+0000","last_change":"2023-09-23T03:25:47.555158+0000","last_active":"2023-09-23T03:25:48.543342+0000","last_peered":"2023-09-23T03:25:48.543342+0000","last_clean":"2023-09-23T03:25:48.543342+0000","last_became_active":"2023-09-23T03:25:47.554873+0000","last_became_peered":"2023-09-23T03:25:47.554873+0000","last_unstale":"2023-09-23T03:25:48.543342+0000","last_undegraded":"2023-09-23T03:25:48.543342+0000","last_fullsized":"2023-09-23T03:25:48.543342+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:53:45.800528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"104'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.654978+0000","last_change":"2023-09-23T03:25:46.559412+0000","last_active":"2023-09-23T03:25:48.654978+0000","last_peered":"2023-09-23T03:25:48.654978+0000","last_clean":"2023-09-23T03:25:48.654978+0000","last_became_active":"2023-09-23T03:25:46.559082+0000","last_became_peered":"2023-09-23T03:25:46.559082+0000","last_unstale":"2023-09-23T03:25:48.654978+0000","last_undegraded":"2023-09-23T03:25:48.654978+0000","last_fullsized":"2023-09-23T03:25:48.654978+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:25:32.261618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.655563+0000","last_change":"2023-09-23T03:25:47.567520+0000","last_active":"2023-09-23T03:25:48.655563+0000","last_peered":"2023-09-23T03:25:48.655563+0000","last_clean":"2023-09-23T03:25:48.655563+0000","last_became_active":"2023-09-23T03:25:47.567131+0000","last_became_peered":"2023-09-23T03:25:47.567131+0000","last_unstale":"2023-09-23T03:25:48.655563+0000","last_undegraded":"2023-09-23T03:25:48.655563+0000","last_fullsized":"2023-09-23T03:25:48.655563+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:24:57.796899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"104'1","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.542752+0000","last_change":"2023-09-23T03:25:46.560175+0000","last_active":"2023-09-23T03:25:48.542752+0000","last_peered":"2023-09-23T03:25:48.542752+0000","last_clean":"2023-09-23T03:25:48.542752+0000","last_became_active":"2023-09-23T03:25:46.558277+0000","last_became_peered":"2023-09-23T03:25:46.558277+0000","last_unstale":"2023-09-23T03:25:48.542752+0000","last_undegraded":"2023-09-23T03:25:48.542752+0000","last_fullsized":"2023-09-23T03:25:48.542752+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:39:44.467126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.653904+0000","last_change":"2023-09-23T03:25:47.571139+0000","last_active":"2023-09-23T03:25:48.653904+0000","last_peered":"2023-09-23T03:25:48.653904+0000","last_clean":"2023-09-23T03:25:48.653904+0000","last_became_active":"2023-09-23T03:25:47.570861+0000","last_became_peered":"2023-09-23T03:25:47.570861+0000","last_unstale":"2023-09-23T03:25:48.653904+0000","last_undegraded":"2023-09-23T03:25:48.653904+0000","last_fullsized":"2023-09-23T03:25:48.653904+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:19:43.923022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.542040+0000","last_change":"2023-09-23T03:25:46.561312+0000","last_active":"2023-09-23T03:25:48.542040+0000","last_peered":"2023-09-23T03:25:48.542040+0000","last_clean":"2023-09-23T03:25:48.542040+0000","last_became_active":"2023-09-23T03:25:46.561052+0000","last_became_peered":"2023-09-23T03:25:46.561052+0000","last_unstale":"2023-09-23T03:25:48.542040+0000","last_undegraded":"2023-09-23T03:25:48.542040+0000","last_fullsized":"2023-09-23T03:25:48.542040+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:14:15.654280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.650231+0000","last_change":"2023-09-23T03:25:47.573902+0000","last_active":"2023-09-23T03:25:48.650231+0000","last_peered":"2023-09-23T03:25:48.650231+0000","last_clean":"2023-09-23T03:25:48.650231+0000","last_became_active":"2023-09-23T03:25:47.573600+0000","last_became_peered":"2023-09-23T03:25:47.573600+0000","last_unstale":"2023-09-23T03:25:48.650231+0000","last_undegraded":"2023-09-23T03:25:48.650231+0000","last_fullsized":"2023-09-23T03:25:48.650231+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:45:07.332713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"104'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.542033+0000","last_change":"2023-09-23T03:25:46.559615+0000","last_active":"2023-09-23T03:25:48.542033+0000","last_peered":"2023-09-23T03:25:48.542033+0000","last_clean":"2023-09-23T03:25:48.542033+0000","last_became_active":"2023-09-23T03:25:46.559329+0000","last_became_peered":"2023-09-23T03:25:46.559329+0000","last_unstale":"2023-09-23T03:25:48.542033+0000","last_undegraded":"2023-09-23T03:25:48.542033+0000","last_fullsized":"2023-09-23T03:25:48.542033+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:32:10.143445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.650552+0000","last_change":"2023-09-23T03:25:47.570065+0000","last_active":"2023-09-23T03:25:48.650552+0000","last_peered":"2023-09-23T03:25:48.650552+0000","last_clean":"2023-09-23T03:25:48.650552+0000","last_became_active":"2023-09-23T03:25:47.569696+0000","last_became_peered":"2023-09-23T03:25:47.569696+0000","last_unstale":"2023-09-23T03:25:48.650552+0000","last_undegraded":"2023-09-23T03:25:48.650552+0000","last_fullsized":"2023-09-23T03:25:48.650552+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:25:33.275563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.655783+0000","last_change":"2023-09-23T03:25:46.565437+0000","last_active":"2023-09-23T03:25:48.655783+0000","last_peered":"2023-09-23T03:25:48.655783+0000","last_clean":"2023-09-23T03:25:48.655783+0000","last_became_active":"2023-09-23T03:25:46.565087+0000","last_became_peered":"2023-09-23T03:25:46.565087+0000","last_unstale":"2023-09-23T03:25:48.655783+0000","last_undegraded":"2023-09-23T03:25:48.655783+0000","last_fullsized":"2023-09-23T03:25:48.655783+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:30:43.602868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.649979+0000","last_change":"2023-09-23T03:25:47.574312+0000","last_active":"2023-09-23T03:25:48.649979+0000","last_peered":"2023-09-23T03:25:48.649979+0000","last_clean":"2023-09-23T03:25:48.649979+0000","last_became_active":"2023-09-23T03:25:47.574019+0000","last_became_peered":"2023-09-23T03:25:47.574019+0000","last_unstale":"2023-09-23T03:25:48.649979+0000","last_undegraded":"2023-09-23T03:25:48.649979+0000","last_fullsized":"2023-09-23T03:25:48.649979+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:12:04.699882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.655383+0000","last_change":"2023-09-23T03:25:46.559604+0000","last_active":"2023-09-23T03:25:48.655383+0000","last_peered":"2023-09-23T03:25:48.655383+0000","last_clean":"2023-09-23T03:25:48.655383+0000","last_became_active":"2023-09-23T03:25:46.559315+0000","last_became_peered":"2023-09-23T03:25:46.559315+0000","last_unstale":"2023-09-23T03:25:48.655383+0000","last_undegraded":"2023-09-23T03:25:48.655383+0000","last_fullsized":"2023-09-23T03:25:48.655383+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:39:44.467126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.541903+0000","last_change":"2023-09-23T03:25:47.560277+0000","last_active":"2023-09-23T03:25:48.541903+0000","last_peered":"2023-09-23T03:25:48.541903+0000","last_clean":"2023-09-23T03:25:48.541903+0000","last_became_active":"2023-09-23T03:25:47.556974+0000","last_became_peered":"2023-09-23T03:25:47.556974+0000","last_unstale":"2023-09-23T03:25:48.541903+0000","last_undegraded":"2023-09-23T03:25:48.541903+0000","last_fullsized":"2023-09-23T03:25:48.541903+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:19:31.149437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.541197+0000","last_change":"2023-09-23T03:25:46.555302+0000","last_active":"2023-09-23T03:25:48.541197+0000","last_peered":"2023-09-23T03:25:48.541197+0000","last_clean":"2023-09-23T03:25:48.541197+0000","last_became_active":"2023-09-23T03:25:46.554954+0000","last_became_peered":"2023-09-23T03:25:46.554954+0000","last_unstale":"2023-09-23T03:25:48.541197+0000","last_undegraded":"2023-09-23T03:25:48.541197+0000","last_fullsized":"2023-09-23T03:25:48.541197+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:19:19.422597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.543142+0000","last_change":"2023-09-23T03:25:47.560873+0000","last_active":"2023-09-23T03:25:48.543142+0000","last_peered":"2023-09-23T03:25:48.543142+0000","last_clean":"2023-09-23T03:25:48.543142+0000","last_became_active":"2023-09-23T03:25:47.560601+0000","last_became_peered":"2023-09-23T03:25:47.560601+0000","last_unstale":"2023-09-23T03:25:48.543142+0000","last_undegraded":"2023-09-23T03:25:48.543142+0000","last_fullsized":"2023-09-23T03:25:48.543142+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:51:34.682928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"104'2","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.656104+0000","last_change":"2023-09-23T03:25:46.560272+0000","last_active":"2023-09-23T03:25:48.656104+0000","last_peered":"2023-09-23T03:25:48.656104+0000","last_clean":"2023-09-23T03:25:48.656104+0000","last_became_active":"2023-09-23T03:25:46.559990+0000","last_became_peered":"2023-09-23T03:25:46.559990+0000","last_unstale":"2023-09-23T03:25:48.656104+0000","last_undegraded":"2023-09-23T03:25:48.656104+0000","last_fullsized":"2023-09-23T03:25:48.656104+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:52:11.535392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.544432+0000","last_change":"2023-09-23T03:25:47.576834+0000","last_active":"2023-09-23T03:25:48.544432+0000","last_peered":"2023-09-23T03:25:48.544432+0000","last_clean":"2023-09-23T03:25:48.544432+0000","last_became_active":"2023-09-23T03:25:47.576583+0000","last_became_peered":"2023-09-23T03:25:47.576583+0000","last_unstale":"2023-09-23T03:25:48.544432+0000","last_undegraded":"2023-09-23T03:25:48.544432+0000","last_fullsized":"2023-09-23T03:25:48.544432+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:10:44.788203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.543111+0000","last_change":"2023-09-23T03:25:46.559387+0000","last_active":"2023-09-23T03:25:48.543111+0000","last_peered":"2023-09-23T03:25:48.543111+0000","last_clean":"2023-09-23T03:25:48.543111+0000","last_became_active":"2023-09-23T03:25:46.559119+0000","last_became_peered":"2023-09-23T03:25:46.559119+0000","last_unstale":"2023-09-23T03:25:48.543111+0000","last_undegraded":"2023-09-23T03:25:48.543111+0000","last_fullsized":"2023-09-23T03:25:48.543111+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:47:09.875520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.544148+0000","last_change":"2023-09-23T03:25:47.569213+0000","last_active":"2023-09-23T03:25:48.544148+0000","last_peered":"2023-09-23T03:25:48.544148+0000","last_clean":"2023-09-23T03:25:48.544148+0000","last_became_active":"2023-09-23T03:25:47.568796+0000","last_became_peered":"2023-09-23T03:25:47.568796+0000","last_unstale":"2023-09-23T03:25:48.544148+0000","last_undegraded":"2023-09-23T03:25:48.544148+0000","last_fullsized":"2023-09-23T03:25:48.544148+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:48:16.248495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.542491+0000","last_change":"2023-09-23T03:25:46.558796+0000","last_active":"2023-09-23T03:25:48.542491+0000","last_peered":"2023-09-23T03:25:48.542491+0000","last_clean":"2023-09-23T03:25:48.542491+0000","last_became_active":"2023-09-23T03:25:46.558505+0000","last_became_peered":"2023-09-23T03:25:46.558505+0000","last_unstale":"2023-09-23T03:25:48.542491+0000","last_undegraded":"2023-09-23T03:25:48.542491+0000","last_fullsized":"2023-09-23T03:25:48.542491+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:19:45.116812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.542495+0000","last_change":"2023-09-23T03:25:47.551566+0000","last_active":"2023-09-23T03:25:48.542495+0000","last_peered":"2023-09-23T03:25:48.542495+0000","last_clean":"2023-09-23T03:25:48.542495+0000","last_became_active":"2023-09-23T03:25:47.551253+0000","last_became_peered":"2023-09-23T03:25:47.551253+0000","last_unstale":"2023-09-23T03:25:48.542495+0000","last_undegraded":"2023-09-23T03:25:48.542495+0000","last_fullsized":"2023-09-23T03:25:48.542495+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:05:49.332342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.543004+0000","last_change":"2023-09-23T03:25:46.563687+0000","last_active":"2023-09-23T03:25:48.543004+0000","last_peered":"2023-09-23T03:25:48.543004+0000","last_clean":"2023-09-23T03:25:48.543004+0000","last_became_active":"2023-09-23T03:25:46.563287+0000","last_became_peered":"2023-09-23T03:25:46.563287+0000","last_unstale":"2023-09-23T03:25:48.543004+0000","last_undegraded":"2023-09-23T03:25:48.543004+0000","last_fullsized":"2023-09-23T03:25:48.543004+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:22:24.282357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.650349+0000","last_change":"2023-09-23T03:25:47.573670+0000","last_active":"2023-09-23T03:25:48.650349+0000","last_peered":"2023-09-23T03:25:48.650349+0000","last_clean":"2023-09-23T03:25:48.650349+0000","last_became_active":"2023-09-23T03:25:47.573098+0000","last_became_peered":"2023-09-23T03:25:47.573098+0000","last_unstale":"2023-09-23T03:25:48.650349+0000","last_undegraded":"2023-09-23T03:25:48.650349+0000","last_fullsized":"2023-09-23T03:25:48.650349+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:05:29.456594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.542235+0000","last_change":"2023-09-23T03:25:46.545088+0000","last_active":"2023-09-23T03:25:48.542235+0000","last_peered":"2023-09-23T03:25:48.542235+0000","last_clean":"2023-09-23T03:25:48.542235+0000","last_became_active":"2023-09-23T03:25:46.544372+0000","last_became_peered":"2023-09-23T03:25:46.544372+0000","last_unstale":"2023-09-23T03:25:48.542235+0000","last_undegraded":"2023-09-23T03:25:48.542235+0000","last_fullsized":"2023-09-23T03:25:48.542235+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:18:56.396735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.653850+0000","last_change":"2023-09-23T03:25:47.549323+0000","last_active":"2023-09-23T03:25:48.653850+0000","last_peered":"2023-09-23T03:25:48.653850+0000","last_clean":"2023-09-23T03:25:48.653850+0000","last_became_active":"2023-09-23T03:25:47.548909+0000","last_became_peered":"2023-09-23T03:25:47.548909+0000","last_unstale":"2023-09-23T03:25:48.653850+0000","last_undegraded":"2023-09-23T03:25:48.653850+0000","last_fullsized":"2023-09-23T03:25:48.653850+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:10:44.788203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.542050+0000","last_change":"2023-09-23T03:25:46.555570+0000","last_active":"2023-09-23T03:25:48.542050+0000","last_peered":"2023-09-23T03:25:48.542050+0000","last_clean":"2023-09-23T03:25:48.542050+0000","last_became_active":"2023-09-23T03:25:46.555285+0000","last_became_peered":"2023-09-23T03:25:46.555285+0000","last_unstale":"2023-09-23T03:25:48.542050+0000","last_undegraded":"2023-09-23T03:25:48.542050+0000","last_fullsized":"2023-09-23T03:25:48.542050+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:10:43.774259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.650087+0000","last_change":"2023-09-23T03:25:47.574240+0000","last_active":"2023-09-23T03:25:48.650087+0000","last_peered":"2023-09-23T03:25:48.650087+0000","last_clean":"2023-09-23T03:25:48.650087+0000","last_became_active":"2023-09-23T03:25:47.573682+0000","last_became_peered":"2023-09-23T03:25:47.573682+0000","last_unstale":"2023-09-23T03:25:48.650087+0000","last_undegraded":"2023-09-23T03:25:48.650087+0000","last_fullsized":"2023-09-23T03:25:48.650087+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:05:14.106449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.649925+0000","last_change":"2023-09-23T03:25:46.557313+0000","last_active":"2023-09-23T03:25:48.649925+0000","last_peered":"2023-09-23T03:25:48.649925+0000","last_clean":"2023-09-23T03:25:48.649925+0000","last_became_active":"2023-09-23T03:25:46.557025+0000","last_became_peered":"2023-09-23T03:25:46.557025+0000","last_unstale":"2023-09-23T03:25:48.649925+0000","last_undegraded":"2023-09-23T03:25:48.649925+0000","last_fullsized":"2023-09-23T03:25:48.649925+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:34:23.970826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.542142+0000","last_change":"2023-09-23T03:25:47.552785+0000","last_active":"2023-09-23T03:25:48.542142+0000","last_peered":"2023-09-23T03:25:48.542142+0000","last_clean":"2023-09-23T03:25:48.542142+0000","last_became_active":"2023-09-23T03:25:47.552497+0000","last_became_peered":"2023-09-23T03:25:47.552497+0000","last_unstale":"2023-09-23T03:25:48.542142+0000","last_undegraded":"2023-09-23T03:25:48.542142+0000","last_fullsized":"2023-09-23T03:25:48.542142+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:17:45.061241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.650067+0000","last_change":"2023-09-23T03:25:46.557092+0000","last_active":"2023-09-23T03:25:48.650067+0000","last_peered":"2023-09-23T03:25:48.650067+0000","last_clean":"2023-09-23T03:25:48.650067+0000","last_became_active":"2023-09-23T03:25:46.556860+0000","last_became_peered":"2023-09-23T03:25:46.556860+0000","last_unstale":"2023-09-23T03:25:48.650067+0000","last_undegraded":"2023-09-23T03:25:48.650067+0000","last_fullsized":"2023-09-23T03:25:48.650067+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:19:58.683118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":11,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.650775+0000","last_change":"2023-09-23T03:25:47.574919+0000","last_active":"2023-09-23T03:25:48.650775+0000","last_peered":"2023-09-23T03:25:48.650775+0000","last_clean":"2023-09-23T03:25:48.650775+0000","last_became_active":"2023-09-23T03:25:47.574591+0000","last_became_peered":"2023-09-23T03:25:47.574591+0000","last_unstale":"2023-09-23T03:25:48.650775+0000","last_undegraded":"2023-09-23T03:25:48.650775+0000","last_fullsized":"2023-09-23T03:25:48.650775+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:46.516976+0000","last_clean_scrub_stamp":"2023-09-23T03:25:46.516976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:44:06.391679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"104'1","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-09-23T03:25:48.654354+0000","last_change":"2023-09-23T03:25:46.567250+0000","last_active":"2023-09-23T03:25:48.654354+0000","last_peered":"2023-09-23T03:25:48.654354+0000","last_clean":"2023-09-23T03:25:48.654354+0000","last_became_active":"2023-09-23T03:25:46.566955+0000","last_became_peered":"2023-09-23T03:25:46.566955+0000","last_unstale":"2023-09-23T03:25:48.654354+0000","last_undegraded":"2023-09-23T03:25:48.654354+0000","last_fullsized":"2023-09-23T03:25:48.654354+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:25:45.503032+0000","last_clean_scrub_stamp":"2023-09-23T03:25:45.503032+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:51:48.156040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":238,"reported_epoch":104,"state":"active+clean","last_fresh":"2023-09-23T03:25:47.553458+0000","last_change":"2023-09-23T03:15:11.042447+0000","last_active":"2023-09-23T03:25:47.553458+0000","last_peered":"2023-09-23T03:25:47.553458+0000","last_clean":"2023-09-23T03:25:47.553458+0000","last_became_active":"2023-09-23T03:15:11.040322+0000","last_became_peered":"2023-09-23T03:15:11.040322+0000","last_unstale":"2023-09-23T03:25:47.553458+0000","last_undegraded":"2023-09-23T03:25:47.553458+0000","last_fullsized":"2023-09-23T03:25:47.553458+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_clean_scrub_stamp":"2023-09-23T03:15:09.474842+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:18:36.331645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574978,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33024,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338816,"statfs":{"total":96636764160,"available":96602947584,"internally_reserved":0,"allocated":1888256,"data_stored":1212750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:25:31 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.036,"15min":1.0049999999999999},"min":{"1min":0.751,"5min":0.57999999999999996,"15min":0.442},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.149},"last":0.68000000000000005},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.0660000000000001,"15min":1.0549999999999999},"min":{"1min":0.84099999999999997,"5min":0.67200000000000004,"15min":0.52000000000000002},"max":{"1min":1.9910000000000001,"5min":1.9910000000000001,"15min":2.1480000000000001},"last":0.93100000000000005}]},{"osd":1,"last update":"Sat Sep 23 03:25:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.23,"15min":1.2170000000000001},"min":{"1min":0.76500000000000001,"5min":0.66400000000000003,"15min":0.504},"max":{"1min":2.6400000000000001,"5min":2.6400000000000001,"15min":3.1859999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.2370000000000001,"15min":1.3080000000000001},"min":{"1min":0.85499999999999998,"5min":0.71299999999999997,"15min":0.67900000000000005},"max":{"1min":2.3149999999999999,"5min":2.3149999999999999,"15min":2.613},"last":1.101}]},{"osd":2,"last update":"Sat Sep 23 03:25:31 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.1180000000000001,"15min":1.1339999999999999},"min":{"1min":0.70399999999999996,"5min":0.61199999999999999,"15min":0.42399999999999999},"max":{"1min":1.4850000000000001,"5min":1.6439999999999999,"15min":2.331},"last":1.577},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.222,"15min":1.2490000000000001},"min":{"1min":1.0449999999999999,"5min":0.63200000000000001,"15min":0.55200000000000005},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":3.093},"last":1.054}]},{"osd":3,"last update":"Sat Sep 23 03:25:31 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.2010000000000001,"15min":1.2509999999999999},"min":{"1min":0.86899999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.9530000000000001,"5min":1.9530000000000001,"15min":2.532},"last":1.25},{"interface":"front","average":{"1min":1.514,"5min":1.3360000000000001,"15min":1.379},"min":{"1min":1.2310000000000001,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.4289999999999998,"5min":2.4289999999999998,"15min":3.2349999999999999},"last":0.89700000000000002}]},{"osd":4,"last update":"Sat Sep 23 03:25:31 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.2749999999999999,"15min":1.4099999999999999},"min":{"1min":0.879,"5min":0.57999999999999996,"15min":0.45400000000000001},"max":{"1min":2.214,"5min":2.214,"15min":2.7210000000000001},"last":1.198},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.4370000000000001,"15min":1.492},"min":{"1min":1.2270000000000001,"5min":0.73599999999999999,"15min":0.48099999999999998},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":3.7719999999999998},"last":1.6639999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:25:31 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.3879999999999999,"15min":1.427},"min":{"1min":0.98399999999999999,"5min":0.749,"15min":0.47999999999999998},"max":{"1min":2.5390000000000001,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":1.151},{"interface":"front","average":{"1min":1.375,"5min":1.371,"15min":1.4610000000000001},"min":{"1min":0.96999999999999997,"5min":0.77000000000000002,"15min":0.497},"max":{"1min":2.089,"5min":2.089,"15min":2.4689999999999999},"last":1.3320000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:25:31 2023","interfaces":[{"interface":"back","average":{"1min":1.522,"5min":1.47,"15min":1.5660000000000001},"min":{"1min":1.121,"5min":0.85499999999999998,"15min":0.53800000000000003},"max":{"1min":2.395,"5min":2.395,"15min":2.8420000000000001},"last":1.419},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.4330000000000001,"15min":1.514},"min":{"1min":1.111,"5min":0.78900000000000003,"15min":0.45900000000000002},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":3.6739999999999999},"last":1.468}]}]},{"osd":6,"up_from":13,"seq":55834574978,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32460,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339380,"statfs":{"total":96636764160,"available":96603525120,"internally_reserved":0,"allocated":1306624,"data_stored":629542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:25:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.1439999999999999,"15min":1.0780000000000001},"min":{"1min":0.90600000000000003,"5min":0.55000000000000004,"15min":0.38},"max":{"1min":1.722,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.292},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1319999999999999,"15min":1.0800000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.46100000000000002},"max":{"1min":1.8149999999999999,"5min":3.7629999999999999,"15min":3.7629999999999999},"last":0.82999999999999996}]},{"osd":1,"last update":"Sat Sep 23 03:25:22 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.149,"15min":1.119},"min":{"1min":0.68999999999999995,"5min":0.59099999999999997,"15min":0.438},"max":{"1min":2.2400000000000002,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1759999999999999,"15min":1.2030000000000001},"min":{"1min":0.56599999999999995,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.7829999999999999,"5min":2.972,"15min":2.972},"last":0.72399999999999998}]},{"osd":2,"last update":"Sat Sep 23 03:25:22 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.1719999999999999,"15min":1.1279999999999999},"min":{"1min":0.70799999999999996,"5min":0.64700000000000002,"15min":0.51300000000000001},"max":{"1min":1.76,"5min":3.4390000000000001,"15min":3.4390000000000001},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.2290000000000001,"15min":1.246},"min":{"1min":0.84099999999999997,"5min":0.66200000000000003,"15min":0.64300000000000002},"max":{"1min":2.177,"5min":2.177,"15min":2.177},"last":1.627}]},{"osd":3,"last update":"Sat Sep 23 03:25:22 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.331,"15min":1.3420000000000001},"min":{"1min":1.0549999999999999,"5min":0.72399999999999998,"15min":0.54600000000000004},"max":{"1min":2.129,"5min":2.129,"15min":2.6509999999999998},"last":1.7250000000000001},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.3740000000000001,"15min":1.4039999999999999},"min":{"1min":1.155,"5min":0.84599999999999997,"15min":0.74299999999999999},"max":{"1min":1.853,"5min":3.0910000000000002,"15min":3.0910000000000002},"last":1.1160000000000001}]},{"osd":4,"last update":"Sat Sep 23 03:25:22 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.375,"15min":1.381},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.498},"max":{"1min":2.024,"5min":3.1789999999999998,"15min":3.1789999999999998},"last":1.494},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.3520000000000001,"15min":1.304},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.56000000000000005},"max":{"1min":1.9590000000000001,"5min":3.6480000000000001,"15min":3.6480000000000001},"last":1.212}]},{"osd":5,"last update":"Sat Sep 23 03:25:22 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.456,"15min":1.413},"min":{"1min":1.081,"5min":1.081,"15min":0.69199999999999995},"max":{"1min":2.1579999999999999,"5min":4.093,"15min":4.093},"last":1.421},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.4330000000000001,"15min":1.458},"min":{"1min":1.099,"5min":0.86699999999999999,"15min":0.59899999999999998},"max":{"1min":1.9710000000000001,"5min":1.996,"15min":4.758},"last":1.9379999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:25:22 2023","interfaces":[{"interface":"back","average":{"1min":1.5780000000000001,"5min":1.5449999999999999,"15min":1.538},"min":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.63500000000000001},"max":{"1min":2.1379999999999999,"5min":3.536,"15min":3.536},"last":1.748},{"interface":"front","average":{"1min":1.575,"5min":1.534,"15min":1.5449999999999999},"min":{"1min":1.2310000000000001,"5min":0.82799999999999996,"15min":0.67300000000000004},"max":{"1min":1.9950000000000001,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":1.843}]}]},{"osd":5,"up_from":13,"seq":55834574980,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1290240,"data_stored":622340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:25:19 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.026,"15min":1.0309999999999999},"min":{"1min":0.75,"5min":0.61199999999999999,"15min":0.57199999999999995},"max":{"1min":1.4910000000000001,"5min":1.732,"15min":1.732},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.163,"5min":1.1120000000000001,"15min":1.0669999999999999},"min":{"1min":0.83899999999999997,"5min":0.624,"15min":0.61399999999999999},"max":{"1min":1.5600000000000001,"5min":2.2320000000000002,"15min":2.2320000000000002},"last":1.278}]},{"osd":1,"last update":"Sat Sep 23 03:25:19 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.2350000000000001,"15min":1.2210000000000001},"min":{"1min":0.71899999999999997,"5min":0.68300000000000005,"15min":0.62},"max":{"1min":1.484,"5min":1.657,"15min":1.849},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.2729999999999999,"15min":1.294},"min":{"1min":0.92900000000000005,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.6160000000000001,"5min":1.724,"15min":1.893},"last":1.456}]},{"osd":2,"last update":"Sat Sep 23 03:25:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.1659999999999999,"15min":1.2},"min":{"1min":0.88700000000000001,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.641,"5min":1.726,"15min":1.726},"last":1.478},{"interface":"front","average":{"1min":1.107,"5min":1.171,"15min":1.24},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.71599999999999997},"max":{"1min":1.516,"5min":1.5940000000000001,"15min":1.7629999999999999},"last":1.0369999999999999}]},{"osd":3,"last update":"Sat Sep 23 03:25:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.24,"15min":1.288},"min":{"1min":0.86599999999999999,"5min":0.67200000000000004,"15min":0.66400000000000003},"max":{"1min":1.5549999999999999,"5min":1.696,"15min":1.905},"last":1.349},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.3100000000000001,"15min":1.387},"min":{"1min":0.88900000000000001,"5min":0.71699999999999997,"15min":0.69799999999999995},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":3.6110000000000002},"last":0.97599999999999998}]},{"osd":4,"last update":"Sat Sep 23 03:25:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.3640000000000001,"15min":1.401},"min":{"1min":1.0309999999999999,"5min":0.96599999999999997,"15min":0.54100000000000004},"max":{"1min":1.9430000000000001,"5min":1.9430000000000001,"15min":2.069},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.268,"5min":1.2689999999999999,"15min":1.375},"min":{"1min":0.95899999999999996,"5min":0.60299999999999998,"15min":0.48899999999999999},"max":{"1min":1.6739999999999999,"5min":1.6899999999999999,"15min":2.319},"last":1.3109999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:25:19 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.4850000000000001,"15min":1.514},"min":{"1min":1.0649999999999999,"5min":1.0289999999999999,"15min":0.59199999999999997},"max":{"1min":2.1339999999999999,"5min":2.1339999999999999,"15min":2.1509999999999998},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.343,"5min":1.5029999999999999,"15min":1.552},"min":{"1min":0.91400000000000003,"5min":0.78700000000000003,"15min":0.55100000000000005},"max":{"1min":1.681,"5min":9.6560000000000006,"15min":9.6560000000000006},"last":1.6060000000000001}]},{"osd":7,"last update":"Sat Sep 23 03:25:19 2023","interfaces":[{"interface":"back","average":{"1min":1.468,"5min":1.4099999999999999,"15min":1.466},"min":{"1min":1.05,"5min":0.89100000000000001,"15min":0.626},"max":{"1min":1.7430000000000001,"5min":1.8560000000000001,"15min":2.0350000000000001},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.4830000000000001,"15min":1.5700000000000001},"min":{"1min":1.232,"5min":0.85999999999999999,"15min":0.60799999999999998},"max":{"1min":1.7909999999999999,"5min":1.8140000000000001,"15min":2.1960000000000002},"last":1.4219999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574979,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1318912,"data_stored":630626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:25:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.048,"15min":1.1040000000000001},"min":{"1min":0.68400000000000005,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.488,"5min":1.9770000000000001,"15min":1.9770000000000001},"last":1.5529999999999999},{"interface":"front","average":{"1min":1.23,"5min":1.169,"15min":1.1619999999999999},"min":{"1min":0.78600000000000003,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.5960000000000001,"5min":2.004,"15min":2.004},"last":1.0600000000000001}]},{"osd":1,"last update":"Sat Sep 23 03:25:22 2023","interfaces":[{"interface":"back","average":{"1min":1.294,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.91500000000000004,"5min":0.64000000000000001,"15min":0.61799999999999999},"max":{"1min":1.7969999999999999,"5min":1.8200000000000001,"15min":2.1110000000000002},"last":1.296},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.056,"15min":1.151},"min":{"1min":0.68300000000000005,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.508,"5min":1.718,"15min":3.2189999999999999},"last":1.409}]},{"osd":2,"last update":"Sat Sep 23 03:25:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.157,"15min":1.1930000000000001},"min":{"1min":0.81100000000000005,"5min":0.75600000000000001,"15min":0.61599999999999999},"max":{"1min":1.619,"5min":1.7450000000000001,"15min":2.5920000000000001},"last":1.694},{"interface":"front","average":{"1min":1.216,"5min":1.2490000000000001,"15min":1.2350000000000001},"min":{"1min":0.79200000000000004,"5min":0.69999999999999996,"15min":0.68300000000000005},"max":{"1min":1.7110000000000001,"5min":1.9530000000000001,"15min":1.9530000000000001},"last":1.179}]},{"osd":3,"last update":"Sat Sep 23 03:25:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1850000000000001,"15min":1.1699999999999999},"min":{"1min":0.91600000000000004,"5min":0.71599999999999997,"15min":0.69199999999999995},"max":{"1min":1.429,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.254,"5min":1.284,"15min":1.3089999999999999},"min":{"1min":0.88300000000000001,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.5920000000000001,"5min":2.0609999999999999,"15min":2.4969999999999999},"last":1.347}]},{"osd":5,"last update":"Sat Sep 23 03:25:22 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3620000000000001,"15min":1.359},"min":{"1min":0.88900000000000001,"5min":0.83099999999999996,"15min":0.70199999999999996},"max":{"1min":1.669,"5min":1.756,"15min":2.8050000000000002},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.343,"15min":1.3360000000000001},"min":{"1min":1.012,"5min":0.878,"15min":0.81799999999999995},"max":{"1min":1.663,"5min":2.0539999999999998,"15min":2.887},"last":1.607}]},{"osd":6,"last update":"Sat Sep 23 03:25:22 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.379,"15min":1.383},"min":{"1min":1.119,"5min":0.81299999999999994,"15min":0.66400000000000003},"max":{"1min":1.6519999999999999,"5min":2.113,"15min":2.4249999999999998},"last":1.7769999999999999},{"interface":"front","average":{"1min":1.478,"5min":1.488,"15min":1.4910000000000001},"min":{"1min":1.107,"5min":0.97199999999999998,"15min":0.89900000000000002},"max":{"1min":1.8779999999999999,"5min":2.0800000000000001,"15min":2.1779999999999999},"last":1.6519999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:25:22 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.452,"15min":1.4650000000000001},"min":{"1min":1.1539999999999999,"5min":0.98099999999999998,"15min":0.81399999999999995},"max":{"1min":1.7929999999999999,"5min":2.1970000000000001,"15min":2.5680000000000001},"last":1.7370000000000001},{"interface":"front","average":{"1min":1.5800000000000001,"5min":1.556,"15min":1.5589999999999999},"min":{"1min":1.3859999999999999,"5min":1.1200000000000001,"15min":0.84199999999999997},"max":{"1min":1.786,"5min":2.2400000000000002,"15min":2.4769999999999999},"last":1.8280000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574979,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1314816,"data_stored":630524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:25:35 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.88800000000000001,"15min":0.873},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.254,"5min":2.952,"15min":2.952},"last":1.208},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.90400000000000003,"15min":0.89400000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.29299999999999998},"max":{"1min":1.4710000000000001,"5min":1.7430000000000001,"15min":1.8979999999999999},"last":1.2709999999999999}]},{"osd":1,"last update":"Sat Sep 23 03:25:35 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.82899999999999996,"15min":0.89800000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.419,"5min":1.603,"15min":1.6839999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.02,"15min":0.98899999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.45400000000000001},"max":{"1min":1.7010000000000001,"5min":2.903,"15min":2.903},"last":1.351}]},{"osd":2,"last update":"Sat Sep 23 03:25:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.024,"15min":1.0680000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.4830000000000001,"5min":1.9850000000000001,"15min":2.2240000000000002},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1160000000000001,"15min":1.109},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.7190000000000001,"5min":3.129,"15min":3.129},"last":1.506}]},{"osd":4,"last update":"Sat Sep 23 03:25:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.167,"15min":1.25},"min":{"1min":0.82299999999999995,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.5900000000000001,"5min":1.671,"15min":2.698},"last":0.998},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.2769999999999999,"15min":1.371},"min":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.752},"max":{"1min":1.661,"5min":1.6990000000000001,"15min":2.419},"last":1.141}]},{"osd":5,"last update":"Sat Sep 23 03:25:35 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.349,"15min":1.349},"min":{"1min":0.82599999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.8799999999999999,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":1.6200000000000001},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.25,"15min":1.3029999999999999},"min":{"1min":0.77800000000000002,"5min":0.624,"15min":0.624},"max":{"1min":1.607,"5min":1.841,"15min":2.0070000000000001},"last":1.401}]},{"osd":6,"last update":"Sat Sep 23 03:25:35 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.327,"15min":1.3759999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.69799999999999995},"max":{"1min":1.9530000000000001,"5min":1.9530000000000001,"15min":2.6440000000000001},"last":1.319},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4239999999999999,"15min":1.4470000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.67800000000000005},"max":{"1min":1.95,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":2.177}]},{"osd":7,"last update":"Sat Sep 23 03:25:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.339,"15min":1.4139999999999999},"min":{"1min":0.875,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.0859999999999999,"5min":2.0859999999999999,"15min":2.7879999999999998},"last":1.591},{"interface":"front","average":{"1min":1.464,"5min":1.3779999999999999,"15min":1.4550000000000001},"min":{"1min":0.81000000000000005,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.0270000000000001,"5min":2.0270000000000001,"15min":2.5379999999999998},"last":1.5549999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574980,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1290240,"data_stored":622304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":0.86399999999999999,"15min":0.84099999999999997},"min":{"1min":0.40799999999999997,"5min":0.371,"15min":0.30299999999999999},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.0569999999999999,"5min":0.91800000000000004,"15min":0.87},"min":{"1min":0.46999999999999997,"5min":0.39300000000000002,"15min":0.36299999999999999},"max":{"1min":2.613,"5min":2.613,"15min":2.613},"last":0.71899999999999997}]},{"osd":1,"last update":"Sat Sep 23 03:25:19 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.85999999999999999,"15min":0.85899999999999999},"min":{"1min":0.628,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.0249999999999999,"15min":0.96899999999999997},"min":{"1min":0.59399999999999997,"5min":0.49399999999999999,"15min":0.38},"max":{"1min":2.9809999999999999,"5min":2.9809999999999999,"15min":2.9809999999999999},"last":0.78700000000000003}]},{"osd":3,"last update":"Sat Sep 23 03:25:19 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.032,"15min":0.98199999999999998},"min":{"1min":0.64300000000000002,"5min":0.51600000000000001,"15min":0.40699999999999997},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":1.115},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.0429999999999999,"15min":1.0049999999999999},"min":{"1min":0.52800000000000002,"5min":0.441,"15min":0.34999999999999998},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.86499999999999999}]},{"osd":4,"last update":"Sat Sep 23 03:25:19 2023","interfaces":[{"interface":"back","average":{"1min":1.5580000000000001,"5min":1.3080000000000001,"15min":1.3},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.55800000000000005},"max":{"1min":5.6669999999999998,"5min":5.6669999999999998,"15min":5.6669999999999998},"last":1.214},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.2809999999999999,"15min":1.3100000000000001},"min":{"1min":0.82099999999999995,"5min":0.77200000000000002,"15min":0.76100000000000001},"max":{"1min":2.6909999999999998,"5min":2.6909999999999998,"15min":2.6909999999999998},"last":1.044}]},{"osd":5,"last update":"Sat Sep 23 03:25:19 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.3400000000000001,"15min":1.3600000000000001},"min":{"1min":0.92800000000000005,"5min":0.78600000000000003,"15min":0.71499999999999997},"max":{"1min":2.8969999999999998,"5min":2.8969999999999998,"15min":3.9409999999999998},"last":1.071},{"interface":"front","average":{"1min":1.506,"5min":1.389,"15min":1.448},"min":{"1min":1.0049999999999999,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":3.415,"5min":3.415,"15min":4.9199999999999999},"last":1.292}]},{"osd":6,"last update":"Sat Sep 23 03:25:19 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.353,"15min":1.379},"min":{"1min":1.1180000000000001,"5min":0.86499999999999999,"15min":0.755},"max":{"1min":2.7839999999999998,"5min":2.7839999999999998,"15min":3.0009999999999999},"last":1.349},{"interface":"front","average":{"1min":1.73,"5min":1.474,"15min":1.4490000000000001},"min":{"1min":0.76600000000000001,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":7.8300000000000001,"5min":7.8300000000000001,"15min":7.8300000000000001},"last":1.1659999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:25:19 2023","interfaces":[{"interface":"back","average":{"1min":1.5409999999999999,"5min":1.4379999999999999,"15min":1.4019999999999999},"min":{"1min":1.0389999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":4.0330000000000004,"5min":4.0330000000000004,"15min":4.0330000000000004},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.548,"5min":1.4259999999999999,"15min":1.4370000000000001},"min":{"1min":1.097,"5min":0.82599999999999996,"15min":0.74399999999999999},"max":{"1min":4.2199999999999998,"5min":4.2199999999999998,"15min":4.2199999999999998},"last":1.2470000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574981,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1298432,"data_stored":628890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:25:32 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.78700000000000003,"15min":0.79800000000000004},"min":{"1min":0.56699999999999995,"5min":0.41299999999999998,"15min":0.32300000000000001},"max":{"1min":1.2490000000000001,"5min":1.377,"15min":2.8279999999999998},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.86399999999999999,"15min":0.84799999999999998},"min":{"1min":0.67300000000000004,"5min":0.43099999999999999,"15min":0.312},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.8919999999999999},"last":0.64300000000000002}]},{"osd":2,"last update":"Sat Sep 23 03:25:32 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.88400000000000001,"15min":0.88200000000000001},"min":{"1min":0.64200000000000002,"5min":0.503,"15min":0.36699999999999999},"max":{"1min":1.581,"5min":1.581,"15min":2.3340000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.048,"15min":1.032},"min":{"1min":0.60999999999999999,"5min":0.48899999999999999,"15min":0.45600000000000002},"max":{"1min":1.4730000000000001,"5min":1.6499999999999999,"15min":1.9570000000000001},"last":0.45600000000000002}]},{"osd":3,"last update":"Sat Sep 23 03:25:32 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.0700000000000001,"15min":1.0609999999999999},"min":{"1min":0.82299999999999995,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":2.2370000000000001},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.111,"15min":1.1259999999999999},"min":{"1min":0.85599999999999998,"5min":0.50800000000000001,"15min":0.45900000000000002},"max":{"1min":1.659,"5min":1.659,"15min":3.2130000000000001},"last":0.84399999999999997}]},{"osd":4,"last update":"Sat Sep 23 03:25:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.276,"15min":1.2629999999999999},"min":{"1min":1.2390000000000001,"5min":0.81200000000000006,"15min":0.70699999999999996},"max":{"1min":1.7709999999999999,"5min":1.885,"15min":3.2629999999999999},"last":1.333},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.2549999999999999,"15min":1.2689999999999999},"min":{"1min":1.0529999999999999,"5min":0.77400000000000002,"15min":0.66500000000000004},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":2.4689999999999999},"last":1.0409999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:25:32 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.296,"15min":1.2849999999999999},"min":{"1min":1.1200000000000001,"5min":0.76700000000000002,"15min":0.64600000000000002},"max":{"1min":1.577,"5min":1.788,"15min":3.161},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.3520000000000001,"15min":1.337},"min":{"1min":1.208,"5min":0.81399999999999995,"15min":0.72399999999999998},"max":{"1min":1.746,"5min":1.8260000000000001,"15min":2.4289999999999998},"last":1.153}]},{"osd":6,"last update":"Sat Sep 23 03:25:32 2023","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":1.365,"15min":1.337},"min":{"1min":1.1499999999999999,"5min":0.83599999999999997,"15min":0.747},"max":{"1min":1.7050000000000001,"5min":1.915,"15min":3.0179999999999998},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.482,"5min":1.375,"15min":1.379},"min":{"1min":1.1160000000000001,"5min":0.82499999999999996,"15min":0.72799999999999998},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":3.3580000000000001},"last":1.228}]},{"osd":7,"last update":"Sat Sep 23 03:25:32 2023","interfaces":[{"interface":"back","average":{"1min":1.4179999999999999,"5min":1.357,"15min":1.383},"min":{"1min":0.82799999999999996,"5min":0.79100000000000004,"15min":0.68600000000000005},"max":{"1min":1.6950000000000001,"5min":1.8300000000000001,"15min":3.1099999999999999},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.464,"5min":1.3999999999999999,"15min":1.4390000000000001},"min":{"1min":0.88900000000000001,"5min":0.82899999999999996,"15min":0.78500000000000003},"max":{"1min":1.802,"5min":1.917,"15min":3.306},"last":1.1779999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574981,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33016,"kb_used_data":1836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338824,"statfs":{"total":96636764160,"available":96602955776,"internally_reserved":0,"allocated":1880064,"data_stored":1212201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 23 03:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.85799999999999998,"15min":0.83899999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.30299999999999999},"max":{"1min":1.383,"5min":1.919,"15min":1.919},"last":1.1200000000000001},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.90000000000000002,"15min":0.89300000000000002},"min":{"1min":0.42599999999999999,"5min":0.41299999999999998,"15min":0.38500000000000001},"max":{"1min":1.3740000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":1.1739999999999999}]},{"osd":2,"last update":"Sat Sep 23 03:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.88600000000000001,"15min":0.83799999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.5249999999999999,"5min":3.089,"15min":3.089},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.004,"5min":1.0389999999999999,"15min":1.006},"min":{"1min":0.54100000000000004,"5min":0.49199999999999999,"15min":0.42499999999999999},"max":{"1min":1.4379999999999999,"5min":2.633,"15min":2.633},"last":0.92800000000000005}]},{"osd":3,"last update":"Sat Sep 23 03:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":1.016,"15min":0.96299999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.387,"5min":2.8660000000000001,"15min":2.8660000000000001},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.038,"5min":1.018,"15min":1.008},"min":{"1min":0.52900000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.802,"5min":2.012,"15min":2.012},"last":1.069}]},{"osd":4,"last update":"Sat Sep 23 03:25:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.3009999999999999,"15min":1.2529999999999999},"min":{"1min":0.78800000000000003,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7330000000000001,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":1.387},{"interface":"front","average":{"1min":1.175,"5min":1.27,"15min":1.2749999999999999},"min":{"1min":0.79600000000000004,"5min":0.79500000000000004,"15min":0.65300000000000002},"max":{"1min":1.7430000000000001,"5min":3.0230000000000001,"15min":3.0230000000000001},"last":1.427}]},{"osd":5,"last update":"Sat Sep 23 03:25:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.319,"15min":1.2609999999999999},"min":{"1min":0.81499999999999995,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.9550000000000001,"5min":2.5489999999999999,"15min":2.5489999999999999},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.379,"15min":1.3260000000000001},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.71999999999999997},"max":{"1min":1.9159999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.454}]},{"osd":6,"last update":"Sat Sep 23 03:25:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.371,"15min":1.3280000000000001},"min":{"1min":0.77700000000000002,"5min":0.77400000000000002,"15min":0.75600000000000001},"max":{"1min":2.0870000000000002,"5min":2.9489999999999998,"15min":2.9489999999999998},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.4470000000000001,"15min":1.3720000000000001},"min":{"1min":0.81100000000000005,"5min":0.71899999999999997,"15min":0.68999999999999995},"max":{"1min":1.8540000000000001,"5min":2.2429999999999999,"15min":2.3239999999999998},"last":1.5309999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:25:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.3839999999999999,"15min":1.3600000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.747},"max":{"1min":1.9390000000000001,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":1.21},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.458,"15min":1.4390000000000001},"min":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.81699999999999995},"max":{"1min":1.968,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":1.3680000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-23T03:25:52.499 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-09-23T03:25:52.500 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-09-23T03:25:52.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.762+0000 7f7ba3207640 1 Processor -- start 2023-09-23T03:25:52.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.762+0000 7f7ba3207640 1 -- start start 2023-09-23T03:25:52.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.764+0000 7f7ba3207640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7b9c15aba0 0x7f7b9c15bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:52.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.764+0000 7f7ba3207640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7b9c15e5f0 0x7f7b9c15ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:52.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.764+0000 7f7ba3207640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b9c15b6a0 con 0x7f7b9c0caea0 2023-09-23T03:25:52.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.764+0000 7f7ba3207640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b9c15f0b0 con 0x7f7b9c15e5f0 2023-09-23T03:25:52.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.764+0000 7f7ba3207640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b9c15f390 con 0x7f7b9c15aba0 2023-09-23T03:25:52.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.764+0000 7f7b9bfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7b9c15e5f0 0x7f7b9c15ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:52.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.764+0000 7f7b9bfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7b9c15e5f0 0x7f7b9c15ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48654/0 (socket says 172.21.15.154:48654) 2023-09-23T03:25:52.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.764+0000 7f7b9bfff640 1 -- 172.21.15.154:0/2433011959 learned_addr learned my addr 172.21.15.154:0/2433011959 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:52.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.764+0000 7f7ba0fa3640 1 --2- 172.21.15.154:0/2433011959 >> v2:172.21.15.190:3300/0 conn(0x7f7b9c15aba0 0x7f7b9c15bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:52.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.765+0000 7f7b9bfff640 1 -- 172.21.15.154:0/2433011959 >> v2:172.21.15.190:3300/0 conn(0x7f7b9c15aba0 msgr2=0x7f7b9c15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:52.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.765+0000 7f7b9bfff640 1 --2- 172.21.15.154:0/2433011959 >> v2:172.21.15.190:3300/0 conn(0x7f7b9c15aba0 0x7f7b9c15bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:52.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.765+0000 7f7b9bfff640 1 -- 172.21.15.154:0/2433011959 >> v1:172.21.15.190:6789/0 conn(0x7f7b9c0caea0 legacy=0x7f7b9c0cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:52.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.765+0000 7f7b9bfff640 1 -- 172.21.15.154:0/2433011959 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b9c15b330 con 0x7f7b9c15e5f0 2023-09-23T03:25:52.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.766+0000 7f7b9bfff640 1 --2- 172.21.15.154:0/2433011959 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b9c15e5f0 0x7f7b9c15ea30 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c06e1f0 tx=0x7f7b8402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34de42d5047410c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:52.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.767+0000 7f7b9b7fe640 1 -- 172.21.15.154:0/2433011959 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b84031870 con 0x7f7b9c15e5f0 2023-09-23T03:25:52.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.767+0000 7f7b9b7fe640 1 -- 172.21.15.154:0/2433011959 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b840319d0 con 0x7f7b9c15e5f0 2023-09-23T03:25:52.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.767+0000 7f7b9b7fe640 1 -- 172.21.15.154:0/2433011959 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b84031d60 con 0x7f7b9c15e5f0 2023-09-23T03:25:52.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.767+0000 7f7ba3207640 1 -- 172.21.15.154:0/2433011959 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b9c15e5f0 msgr2=0x7f7b9c15ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:52.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.767+0000 7f7ba3207640 1 --2- 172.21.15.154:0/2433011959 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b9c15e5f0 0x7f7b9c15ea30 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c06e1f0 tx=0x7f7b8402b910 comp rx=0 tx=0).stop 2023-09-23T03:25:52.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.768+0000 7f7ba3207640 1 -- 172.21.15.154:0/2433011959 shutdown_connections 2023-09-23T03:25:52.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.768+0000 7f7ba3207640 1 --2- 172.21.15.154:0/2433011959 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b9c15e5f0 0x7f7b9c15ea30 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:52.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.768+0000 7f7ba3207640 1 --2- 172.21.15.154:0/2433011959 >> v2:172.21.15.190:3300/0 conn(0x7f7b9c15aba0 0x7f7b9c15bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:52.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.768+0000 7f7ba3207640 1 -- 172.21.15.154:0/2433011959 >> 172.21.15.154:0/2433011959 conn(0x7f7b9c06c010 msgr2=0x7f7b9c06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:52.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.768+0000 7f7ba3207640 1 -- 172.21.15.154:0/2433011959 shutdown_connections 2023-09-23T03:25:52.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.768+0000 7f7ba3207640 1 -- 172.21.15.154:0/2433011959 wait complete. 2023-09-23T03:25:52.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.769+0000 7f7ba3207640 1 Processor -- start 2023-09-23T03:25:52.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.770+0000 7f7ba3207640 1 -- start start 2023-09-23T03:25:52.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.771+0000 7f7ba3207640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b9c0caea0 0x7f7b9c153570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:52.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.771+0000 7f7ba3207640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b9c15aba0 0x7f7b9c157480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:52.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.771+0000 7f7ba3207640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b9c15e5f0 0x7f7b9c1579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:52.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.771+0000 7f7ba3207640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c0ac770 con 0x7f7b9c15e5f0 2023-09-23T03:25:52.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.771+0000 7f7ba3207640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c14aa90 con 0x7f7b9c0caea0 2023-09-23T03:25:52.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.771+0000 7f7ba3207640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c0ab8c0 con 0x7f7b9c15aba0 2023-09-23T03:25:52.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.771+0000 7f7ba17a4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b9c15e5f0 0x7f7b9c1579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:52.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.771+0000 7f7ba17a4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b9c15e5f0 0x7f7b9c1579c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48656/0 (socket says 172.21.15.154:48656) 2023-09-23T03:25:52.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.771+0000 7f7ba17a4640 1 -- 172.21.15.154:0/2875307259 learned_addr learned my addr 172.21.15.154:0/2875307259 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:52.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.771+0000 7f7b9bfff640 1 --2- 172.21.15.154:0/2875307259 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b9c15aba0 0x7f7b9c157480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:52.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.771+0000 7f7ba0fa3640 1 --2- 172.21.15.154:0/2875307259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b9c0caea0 0x7f7b9c153570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:52.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.771+0000 7f7ba17a4640 1 -- 172.21.15.154:0/2875307259 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b9c15aba0 msgr2=0x7f7b9c157480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:52.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.771+0000 7f7ba17a4640 1 --2- 172.21.15.154:0/2875307259 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b9c15aba0 0x7f7b9c157480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:52.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.771+0000 7f7ba17a4640 1 -- 172.21.15.154:0/2875307259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b9c0caea0 msgr2=0x7f7b9c153570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:52.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.771+0000 7f7ba17a4640 1 --2- 172.21.15.154:0/2875307259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b9c0caea0 0x7f7b9c153570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:52.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.771+0000 7f7ba17a4640 1 -- 172.21.15.154:0/2875307259 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b840094d0 con 0x7f7b9c15e5f0 2023-09-23T03:25:52.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.772+0000 7f7ba17a4640 1 --2- 172.21.15.154:0/2875307259 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b9c15e5f0 0x7f7b9c1579c0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f7b9400c410 tx=0x7f7b9400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:52.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.772+0000 7f7b997fa640 1 -- 172.21.15.154:0/2875307259 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b94011790 con 0x7f7b9c15e5f0 2023-09-23T03:25:52.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.772+0000 7f7b997fa640 1 -- 172.21.15.154:0/2875307259 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b940118f0 con 0x7f7b9c15e5f0 2023-09-23T03:25:52.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.772+0000 7f7b997fa640 1 -- 172.21.15.154:0/2875307259 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b94011ce0 con 0x7f7b9c15e5f0 2023-09-23T03:25:52.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.772+0000 7f7ba3207640 1 -- 172.21.15.154:0/2875307259 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b9c15b100 con 0x7f7b9c15e5f0 2023-09-23T03:25:52.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.772+0000 7f7ba3207640 1 -- 172.21.15.154:0/2875307259 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b9c0abe40 con 0x7f7b9c15e5f0 2023-09-23T03:25:52.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.774+0000 7f7ba3207640 1 -- 172.21.15.154:0/2875307259 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b9c04e920 con 0x7f7b9c15e5f0 2023-09-23T03:25:52.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.774+0000 7f7b997fa640 1 -- 172.21.15.154:0/2875307259 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f7b9401c460 con 0x7f7b9c15e5f0 2023-09-23T03:25:52.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.775+0000 7f7b997fa640 1 --2- 172.21.15.154:0/2875307259 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7b7c06af60 0x7f7b7c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:52.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.775+0000 7f7b997fa640 1 -- 172.21.15.154:0/2875307259 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f7b94089580 con 0x7f7b9c15e5f0 2023-09-23T03:25:52.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.776+0000 7f7ba0fa3640 1 --2- 172.21.15.154:0/2875307259 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7b7c06af60 0x7f7b7c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:52.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.776+0000 7f7ba0fa3640 1 --2- 172.21.15.154:0/2875307259 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7b7c06af60 0x7f7b7c06d420 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c009a60 tx=0x7f7b8c003990 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:52.782 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:52.781+0000 7f7b997fa640 1 -- 172.21.15.154:0/2875307259 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f7b94054480 con 0x7f7b9c15e5f0 2023-09-23T03:25:53.038 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.038+0000 7f7ba3207640 1 -- 172.21.15.154:0/2875307259 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f7b9c0cbc10 con 0x7f7b9c15e5f0 2023-09-23T03:25:53.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.043+0000 7f7b997fa640 1 -- 172.21.15.154:0/2875307259 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+147 (secure 0 0 0) 0x7f7b94057aa0 con 0x7f7b9c15e5f0 2023-09-23T03:25:53.044 INFO:teuthology.orchestra.run.smithi154.stdout:[client.0] 2023-09-23T03:25:53.044 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQAHWA5lYg+qGhAAjaY0fPsmRTz3ipsKbPugMA== 2023-09-23T03:25:53.044 INFO:teuthology.orchestra.run.smithi154.stdout: caps mds = "allow" 2023-09-23T03:25:53.044 INFO:teuthology.orchestra.run.smithi154.stdout: caps mon = "allow r" 2023-09-23T03:25:53.044 INFO:teuthology.orchestra.run.smithi154.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-09-23T03:25:53.044 INFO:teuthology.orchestra.run.smithi154.stderr:updated caps for client.0 2023-09-23T03:25:53.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.046+0000 7f7ba3207640 1 -- 172.21.15.154:0/2875307259 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7b7c06af60 msgr2=0x7f7b7c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:53.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.046+0000 7f7ba3207640 1 --2- 172.21.15.154:0/2875307259 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7b7c06af60 0x7f7b7c06d420 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c009a60 tx=0x7f7b8c003990 comp rx=0 tx=0).stop 2023-09-23T03:25:53.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.046+0000 7f7ba3207640 1 -- 172.21.15.154:0/2875307259 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b9c15e5f0 msgr2=0x7f7b9c1579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:53.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.046+0000 7f7ba3207640 1 --2- 172.21.15.154:0/2875307259 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b9c15e5f0 0x7f7b9c1579c0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f7b9400c410 tx=0x7f7b9400c8d0 comp rx=0 tx=0).stop 2023-09-23T03:25:53.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.046+0000 7f7ba3207640 1 -- 172.21.15.154:0/2875307259 shutdown_connections 2023-09-23T03:25:53.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.046+0000 7f7ba3207640 1 --2- 172.21.15.154:0/2875307259 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7b7c06af60 0x7f7b7c06d420 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:53.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.046+0000 7f7ba3207640 1 --2- 172.21.15.154:0/2875307259 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b9c15e5f0 0x7f7b9c1579c0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:53.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.046+0000 7f7ba3207640 1 --2- 172.21.15.154:0/2875307259 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7b9c15aba0 0x7f7b9c157480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:53.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.046+0000 7f7ba3207640 1 --2- 172.21.15.154:0/2875307259 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7b9c0caea0 0x7f7b9c153570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:53.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.046+0000 7f7ba3207640 1 -- 172.21.15.154:0/2875307259 >> 172.21.15.154:0/2875307259 conn(0x7f7b9c06c010 msgr2=0x7f7b9c0728b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:53.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.046+0000 7f7ba3207640 1 -- 172.21.15.154:0/2875307259 shutdown_connections 2023-09-23T03:25:53.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.046+0000 7f7ba3207640 1 -- 172.21.15.154:0/2875307259 wait complete. 2023-09-23T03:25:53.065 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:25:53.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.278+0000 7ffbd139a640 1 Processor -- start 2023-09-23T03:25:53.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.278+0000 7ffbd139a640 1 -- start start 2023-09-23T03:25:53.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.280+0000 7ffbd139a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ffbcc15c840 0x7ffbcc0cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:53.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.280+0000 7ffbd139a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbcc0cacd0 0x7ffbcc0cb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:53.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.280+0000 7ffbd139a640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbcc0cc910 con 0x7ffbcc160fd0 2023-09-23T03:25:53.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.280+0000 7ffbd139a640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbcc0cb6c0 con 0x7ffbcc15c840 2023-09-23T03:25:53.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.280+0000 7ffbd139a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbcc15cc20 con 0x7ffbcc0cacd0 2023-09-23T03:25:53.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.280+0000 7ffbcb7fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ffbcc160fd0 0x7ffbcc0c9490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:49632/0 (socket says 172.21.15.154:49632) 2023-09-23T03:25:53.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.280+0000 7ffbcb7fe640 1 -- 172.21.15.154:0/3453131349 learned_addr learned my addr 172.21.15.154:0/3453131349 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:25:53.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.280+0000 7ffbd0b99640 1 --2- 172.21.15.154:0/3453131349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbcc0cacd0 0x7ffbcc0cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:53.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.280+0000 7ffbcbfff640 1 --2- 172.21.15.154:0/3453131349 >> v2:172.21.15.190:3300/0 conn(0x7ffbcc15c840 0x7ffbcc0cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:53.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.281+0000 7ffbcaffd640 1 -- 172.21.15.154:0/3453131349 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 871937177 0 0) 0x7ffbcc0cc910 con 0x7ffbcc160fd0 2023-09-23T03:25:53.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.281+0000 7ffbcaffd640 1 -- 172.21.15.154:0/3453131349 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbb4003520 con 0x7ffbcc160fd0 2023-09-23T03:25:53.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.281+0000 7ffbcaffd640 1 -- 172.21.15.154:0/3453131349 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2028721502 0 0) 0x7ffbb4003520 con 0x7ffbcc160fd0 2023-09-23T03:25:53.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.281+0000 7ffbcaffd640 1 -- 172.21.15.154:0/3453131349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbcc0cacd0 msgr2=0x7ffbcc0cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:53.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.281+0000 7ffbcaffd640 1 --2- 172.21.15.154:0/3453131349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbcc0cacd0 0x7ffbcc0cb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:53.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.281+0000 7ffbcaffd640 1 -- 172.21.15.154:0/3453131349 >> v2:172.21.15.190:3300/0 conn(0x7ffbcc15c840 msgr2=0x7ffbcc0cbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:53.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.281+0000 7ffbcaffd640 1 --2- 172.21.15.154:0/3453131349 >> v2:172.21.15.190:3300/0 conn(0x7ffbcc15c840 0x7ffbcc0cbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:53.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.281+0000 7ffbcaffd640 1 -- 172.21.15.154:0/3453131349 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbcc0c89b0 con 0x7ffbcc160fd0 2023-09-23T03:25:53.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.281+0000 7ffbcaffd640 1 -- 172.21.15.154:0/3453131349 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7ffbc0001670 con 0x7ffbcc160fd0 2023-09-23T03:25:53.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.281+0000 7ffbcaffd640 1 -- 172.21.15.154:0/3453131349 >> v1:172.21.15.154:6789/0 conn(0x7ffbcc160fd0 legacy=0x7ffbcc0c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:53.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.281+0000 7ffbcaffd640 1 --2- 172.21.15.154:0/3453131349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbb4003970 0x7ffbb4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:53.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.281+0000 7ffbcaffd640 1 --2- 172.21.15.154:0/3453131349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbcc0cacd0 0x7ffbb40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:53.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.281+0000 7ffbcaffd640 1 --2- 172.21.15.154:0/3453131349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbb4004d00 0x7ffbb4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:53.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.281+0000 7ffbcaffd640 1 -- 172.21.15.154:0/3453131349 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbcc0cb6c0 con 0x7ffbb4003970 2023-09-23T03:25:53.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.281+0000 7ffbcaffd640 1 -- 172.21.15.154:0/3453131349 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbcc15cc20 con 0x7ffbb4004d00 2023-09-23T03:25:53.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.281+0000 7ffbcaffd640 1 -- 172.21.15.154:0/3453131349 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb400b980 con 0x7ffbcc0cacd0 2023-09-23T03:25:53.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.282+0000 7ffbcbfff640 1 --2- 172.21.15.154:0/3453131349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbb4003970 0x7ffbb4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:53.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.282+0000 7ffbcb7fe640 1 --2- 172.21.15.154:0/3453131349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbcc0cacd0 0x7ffbb40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:53.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.282+0000 7ffbd0b99640 1 --2- 172.21.15.154:0/3453131349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbb4004d00 0x7ffbb4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:53.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.282+0000 7ffbcbfff640 1 -- 172.21.15.154:0/3453131349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbcc0cacd0 msgr2=0x7ffbb40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:53.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.282+0000 7ffbcbfff640 1 --2- 172.21.15.154:0/3453131349 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbcc0cacd0 0x7ffbb40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:53.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.282+0000 7ffbcbfff640 1 -- 172.21.15.154:0/3453131349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbb4004d00 msgr2=0x7ffbb4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:53.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.282+0000 7ffbcbfff640 1 --2- 172.21.15.154:0/3453131349 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbb4004d00 0x7ffbb4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:53.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.282+0000 7ffbcbfff640 1 -- 172.21.15.154:0/3453131349 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb4003520 con 0x7ffbb4003970 2023-09-23T03:25:53.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.284+0000 7ffbcbfff640 1 --2- 172.21.15.154:0/3453131349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbb4003970 0x7ffbb4003d90 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc009ac0 tx=0x7ffbb40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e72edf33cfddac7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:53.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.284+0000 7ffbcaffd640 1 -- 172.21.15.154:0/3453131349 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc02c6d0 con 0x7ffbb4003970 2023-09-23T03:25:53.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.285+0000 7ffbcaffd640 1 -- 172.21.15.154:0/3453131349 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbbc02c830 con 0x7ffbb4003970 2023-09-23T03:25:53.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.285+0000 7ffbcaffd640 1 -- 172.21.15.154:0/3453131349 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc02cbc0 con 0x7ffbb4003970 2023-09-23T03:25:53.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.285+0000 7ffbd139a640 1 -- 172.21.15.154:0/3453131349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbb4003970 msgr2=0x7ffbb4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:53.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.285+0000 7ffbd139a640 1 --2- 172.21.15.154:0/3453131349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbb4003970 0x7ffbb4003d90 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc009ac0 tx=0x7ffbb40057f0 comp rx=0 tx=0).stop 2023-09-23T03:25:53.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.285+0000 7ffbd0b99640 1 -- 172.21.15.154:0/3453131349 reap_dead start 2023-09-23T03:25:53.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.285+0000 7ffbd139a640 1 -- 172.21.15.154:0/3453131349 shutdown_connections 2023-09-23T03:25:53.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.285+0000 7ffbd139a640 1 -- 172.21.15.154:0/3453131349 >> 172.21.15.154:0/3453131349 conn(0x7ffbcc06a390 msgr2=0x7ffbcc0afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:53.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.285+0000 7ffbd139a640 1 -- 172.21.15.154:0/3453131349 shutdown_connections 2023-09-23T03:25:53.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.285+0000 7ffbd139a640 1 -- 172.21.15.154:0/3453131349 wait complete. 2023-09-23T03:25:53.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.287+0000 7ffbd139a640 1 Processor -- start 2023-09-23T03:25:53.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.287+0000 7ffbd139a640 1 -- start start 2023-09-23T03:25:53.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.288+0000 7ffbd139a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbb4003970 0x7ffbcc148670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:53.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.288+0000 7ffbd139a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbcc15da50 0x7ffbcc0b2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:53.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.288+0000 7ffbd139a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbcc0b3120 0x7ffbcc0c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:53.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.288+0000 7ffbd139a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbcc0b35d0 con 0x7ffbcc0b3120 2023-09-23T03:25:53.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.288+0000 7ffbd139a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbcc0b0f10 con 0x7ffbcc15da50 2023-09-23T03:25:53.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.288+0000 7ffbd139a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbcc0c6a90 con 0x7ffbb4003970 2023-09-23T03:25:53.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.289+0000 7ffbd0b99640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbcc0b3120 0x7ffbcc0c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:53.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.289+0000 7ffbd0b99640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbcc0b3120 0x7ffbcc0c5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:48662/0 (socket says 172.21.15.154:48662) 2023-09-23T03:25:53.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.289+0000 7ffbd0b99640 1 -- 172.21.15.154:0/508048699 learned_addr learned my addr 172.21.15.154:0/508048699 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:53.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.289+0000 7ffbcb7fe640 1 --2- 172.21.15.154:0/508048699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbcc15da50 0x7ffbcc0b2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:53.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.289+0000 7ffbcbfff640 1 --2- 172.21.15.154:0/508048699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbb4003970 0x7ffbcc148670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:53.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.289+0000 7ffbd0b99640 1 -- 172.21.15.154:0/508048699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbb4003970 msgr2=0x7ffbcc148670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:53.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.289+0000 7ffbd0b99640 1 --2- 172.21.15.154:0/508048699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbb4003970 0x7ffbcc148670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:53.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.289+0000 7ffbd0b99640 1 -- 172.21.15.154:0/508048699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbcc15da50 msgr2=0x7ffbcc0b2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:53.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.289+0000 7ffbd0b99640 1 --2- 172.21.15.154:0/508048699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbcc15da50 0x7ffbcc0b2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:53.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.289+0000 7ffbd0b99640 1 -- 172.21.15.154:0/508048699 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbcc0b1180 con 0x7ffbcc0b3120 2023-09-23T03:25:53.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.290+0000 7ffbd0b99640 1 --2- 172.21.15.154:0/508048699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbcc0b3120 0x7ffbcc0c5da0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc0cbf40 tx=0x7ffbc400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:53.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.290+0000 7ffbc8ff9640 1 -- 172.21.15.154:0/508048699 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc400e040 con 0x7ffbcc0b3120 2023-09-23T03:25:53.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.290+0000 7ffbc8ff9640 1 -- 172.21.15.154:0/508048699 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc400f040 con 0x7ffbcc0b3120 2023-09-23T03:25:53.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.290+0000 7ffbc8ff9640 1 -- 172.21.15.154:0/508048699 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4015430 con 0x7ffbcc0b3120 2023-09-23T03:25:53.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.290+0000 7ffbd139a640 1 -- 172.21.15.154:0/508048699 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbcc0b1460 con 0x7ffbcc0b3120 2023-09-23T03:25:53.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.290+0000 7ffbd139a640 1 -- 172.21.15.154:0/508048699 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbcc0c6d70 con 0x7ffbcc0b3120 2023-09-23T03:25:53.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.291+0000 7ffbd139a640 1 -- 172.21.15.154:0/508048699 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbcc0736b0 con 0x7ffbcc0b3120 2023-09-23T03:25:53.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.292+0000 7ffbc8ff9640 1 -- 172.21.15.154:0/508048699 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7ffbc4015590 con 0x7ffbcc0b3120 2023-09-23T03:25:53.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.293+0000 7ffbc8ff9640 1 --2- 172.21.15.154:0/508048699 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffbb806af30 0x7ffbb806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:53.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.293+0000 7ffbc8ff9640 1 -- 172.21.15.154:0/508048699 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ffbc408a0d0 con 0x7ffbcc0b3120 2023-09-23T03:25:53.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.293+0000 7ffbcbfff640 1 --2- 172.21.15.154:0/508048699 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffbb806af30 0x7ffbb806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:53.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.294+0000 7ffbcbfff640 1 --2- 172.21.15.154:0/508048699 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffbb806af30 0x7ffbb806d3f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc036000 tx=0x7ffbbc030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:53.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.298+0000 7ffbc8ff9640 1 -- 172.21.15.154:0/508048699 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7ffbc4054f30 con 0x7ffbcc0b3120 2023-09-23T03:25:53.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.530+0000 7ffbd139a640 1 -- 172.21.15.154:0/508048699 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffbcc0b1650 con 0x7ffbcc0b3120 2023-09-23T03:25:53.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.531+0000 7ffbc8ff9640 1 -- 172.21.15.154:0/508048699 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+4927 (secure 0 0 0) 0x7ffbc4058550 con 0x7ffbcc0b3120 2023-09-23T03:25:53.532 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:25:53.532 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":84,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5603,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/4291187069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":4291187069},{"type":"v1","addr":"172.21.15.190:6834","nonce":4291187069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":8097,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/1240348396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1240348396},{"type":"v1","addr":"172.21.15.154:6836","nonce":1240348396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":8100,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/4233181417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":4233181417},{"type":"v1","addr":"172.21.15.154:6837","nonce":4233181417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:25:46.997893+0000","modified":"2023-09-23T03:25:50.058760+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8115":{"gid":8115,"name":"d","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/2967125458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2967125458},{"type":"v1","addr":"172.21.15.190:6835","nonce":2967125458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-09-23T03:25:53.532 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 84 2023-09-23T03:25:53.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.534+0000 7ffbd139a640 1 -- 172.21.15.154:0/508048699 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffbb806af30 msgr2=0x7ffbb806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:53.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.534+0000 7ffbd139a640 1 --2- 172.21.15.154:0/508048699 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffbb806af30 0x7ffbb806d3f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc036000 tx=0x7ffbbc030000 comp rx=0 tx=0).stop 2023-09-23T03:25:53.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.534+0000 7ffbd139a640 1 -- 172.21.15.154:0/508048699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbcc0b3120 msgr2=0x7ffbcc0c5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:53.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.534+0000 7ffbd139a640 1 --2- 172.21.15.154:0/508048699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbcc0b3120 0x7ffbcc0c5da0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc0cbf40 tx=0x7ffbc400f580 comp rx=0 tx=0).stop 2023-09-23T03:25:53.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.534+0000 7ffbd139a640 1 -- 172.21.15.154:0/508048699 shutdown_connections 2023-09-23T03:25:53.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.534+0000 7ffbd139a640 1 --2- 172.21.15.154:0/508048699 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffbb806af30 0x7ffbb806d3f0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:53.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.534+0000 7ffbd139a640 1 --2- 172.21.15.154:0/508048699 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffbcc0b3120 0x7ffbcc0c5da0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:53.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.534+0000 7ffbd139a640 1 --2- 172.21.15.154:0/508048699 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffbcc15da50 0x7ffbcc0b2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:53.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.534+0000 7ffbd139a640 1 --2- 172.21.15.154:0/508048699 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffbb4003970 0x7ffbcc148670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:53.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.534+0000 7ffbd139a640 1 -- 172.21.15.154:0/508048699 >> 172.21.15.154:0/508048699 conn(0x7ffbcc06a390 msgr2=0x7ffbcc0ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:53.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.534+0000 7ffbd139a640 1 -- 172.21.15.154:0/508048699 shutdown_connections 2023-09-23T03:25:53.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:53.534+0000 7ffbd139a640 1 -- 172.21.15.154:0/508048699 wait complete. 2023-09-23T03:25:53.955 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 84, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-23T03:25:46.997893+0000', 'modified': '2023-09-23T03:25:50.058760+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8115}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8115': {'gid': 8115, 'name': 'd', 'rank': 0, 'incarnation': 80, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.190:6835/2967125458', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6833', 'nonce': 2967125458}, {'type': 'v1', 'addr': '172.21.15.190:6835', 'nonce': 2967125458}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-23T03:25:53.955 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-23T03:25:53.955 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-09-23T03:25:54.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.180+0000 7f197253c640 1 Processor -- start 2023-09-23T03:25:54.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.181+0000 7f197253c640 1 -- start start 2023-09-23T03:25:54.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.182+0000 7f197253c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f196c15ad20 0x7f196c0cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:54.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.182+0000 7f197253c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f196c0cd160 0x7f196c0cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:54.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.182+0000 7f197253c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f196c0cdb00 con 0x7f196c0b21d0 2023-09-23T03:25:54.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.182+0000 7f197253c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f196c15f880 con 0x7f196c0cd160 2023-09-23T03:25:54.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.182+0000 7f197253c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196c15ed00 con 0x7f196c15ad20 2023-09-23T03:25:54.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.183+0000 7f196b7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f196c0cd160 0x7f196c0cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:54.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.183+0000 7f1970ad9640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f196c0b21d0 0x7f196c0b2660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:60370/0 (socket says 172.21.15.154:60370) 2023-09-23T03:25:54.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.183+0000 7f1970ad9640 1 -- 172.21.15.154:0/3874734282 learned_addr learned my addr 172.21.15.154:0/3874734282 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:25:54.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.183+0000 7f196bfff640 1 --2- 172.21.15.154:0/3874734282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f196c15ad20 0x7f196c0cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:54.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.183+0000 7f196b7fe640 1 -- 172.21.15.154:0/3874734282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f196c15ad20 msgr2=0x7f196c0cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:54.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.183+0000 7f196b7fe640 1 --2- 172.21.15.154:0/3874734282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f196c15ad20 0x7f196c0cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.183+0000 7f196b7fe640 1 -- 172.21.15.154:0/3874734282 >> v1:172.21.15.154:6789/0 conn(0x7f196c0b21d0 legacy=0x7f196c0b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:54.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.183+0000 7f196affd640 1 -- 172.21.15.154:0/3874734282 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4118030745 0 0) 0x7f196c0cdb00 con 0x7f196c0b21d0 2023-09-23T03:25:54.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.183+0000 7f196b7fe640 1 -- 172.21.15.154:0/3874734282 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196c15efe0 con 0x7f196c0cd160 2023-09-23T03:25:54.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.184+0000 7f196b7fe640 1 --2- 172.21.15.154:0/3874734282 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f196c0cd160 0x7f196c0cd5c0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f19540097a0 tx=0x7f195402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77562a5096912f6c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:54.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.185+0000 7f196affd640 1 -- 172.21.15.154:0/3874734282 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1954031830 con 0x7f196c0cd160 2023-09-23T03:25:54.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.185+0000 7f196affd640 1 -- 172.21.15.154:0/3874734282 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1954031990 con 0x7f196c0cd160 2023-09-23T03:25:54.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.185+0000 7f196affd640 1 -- 172.21.15.154:0/3874734282 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1954031d20 con 0x7f196c0cd160 2023-09-23T03:25:54.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.185+0000 7f197253c640 1 -- 172.21.15.154:0/3874734282 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f196c0cd160 msgr2=0x7f196c0cd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:54.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.185+0000 7f197253c640 1 --2- 172.21.15.154:0/3874734282 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f196c0cd160 0x7f196c0cd5c0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f19540097a0 tx=0x7f195402b8d0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.185+0000 7f197253c640 1 -- 172.21.15.154:0/3874734282 shutdown_connections 2023-09-23T03:25:54.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.185+0000 7f197253c640 1 --2- 172.21.15.154:0/3874734282 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f196c0cd160 0x7f196c0cd5c0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.185+0000 7f197253c640 1 --2- 172.21.15.154:0/3874734282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f196c15ad20 0x7f196c0cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.185+0000 7f197253c640 1 -- 172.21.15.154:0/3874734282 >> 172.21.15.154:0/3874734282 conn(0x7f196c06fc60 msgr2=0x7f196c070040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:54.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.186+0000 7f197253c640 1 -- 172.21.15.154:0/3874734282 shutdown_connections 2023-09-23T03:25:54.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.186+0000 7f197253c640 1 -- 172.21.15.154:0/3874734282 wait complete. 2023-09-23T03:25:54.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.187+0000 7f197253c640 1 Processor -- start 2023-09-23T03:25:54.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.188+0000 7f197253c640 1 -- start start 2023-09-23T03:25:54.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.188+0000 7f197253c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f196c0b21d0 0x7f196c0af050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:54.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.188+0000 7f197253c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f196c0cd160 0x7f196c1547e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:54.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.189+0000 7f197253c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f196c15ad20 0x7f196c0c4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:54.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.189+0000 7f197253c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c0c4840 con 0x7f196c0b21d0 2023-09-23T03:25:54.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.189+0000 7f197253c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196c154d20 con 0x7f196c15ad20 2023-09-23T03:25:54.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.189+0000 7f197253c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196c155210 con 0x7f196c0cd160 2023-09-23T03:25:54.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.189+0000 7f196bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f196c0b21d0 0x7f196c0af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:54.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.189+0000 7f196bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f196c0b21d0 0x7f196c0af050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50716/0 (socket says 172.21.15.154:50716) 2023-09-23T03:25:54.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.189+0000 7f196bfff640 1 -- 172.21.15.154:0/132780136 learned_addr learned my addr 172.21.15.154:0/132780136 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:54.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.189+0000 7f196b7fe640 1 --2- 172.21.15.154:0/132780136 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f196c0cd160 0x7f196c1547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:54.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.189+0000 7f196bfff640 1 -- 172.21.15.154:0/132780136 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f196c0cd160 msgr2=0x7f196c1547e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:54.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.189+0000 7f196bfff640 1 --2- 172.21.15.154:0/132780136 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f196c0cd160 0x7f196c1547e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.189+0000 7f1970ad9640 1 --2- 172.21.15.154:0/132780136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f196c15ad20 0x7f196c0c4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:54.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.189+0000 7f196bfff640 1 -- 172.21.15.154:0/132780136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f196c15ad20 msgr2=0x7f196c0c4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:54.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.189+0000 7f196bfff640 1 --2- 172.21.15.154:0/132780136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f196c15ad20 0x7f196c0c4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.189+0000 7f196bfff640 1 -- 172.21.15.154:0/132780136 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1954009400 con 0x7f196c0b21d0 2023-09-23T03:25:54.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.189+0000 7f1970ad9640 1 --2- 172.21.15.154:0/132780136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f196c15ad20 0x7f196c0c4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:54.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.190+0000 7f196bfff640 1 --2- 172.21.15.154:0/132780136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f196c0b21d0 0x7f196c0af050 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f195c00d5d0 tx=0x7f195c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:54.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.190+0000 7f1968ff9640 1 -- 172.21.15.154:0/132780136 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c0125f0 con 0x7f196c0b21d0 2023-09-23T03:25:54.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.190+0000 7f1968ff9640 1 -- 172.21.15.154:0/132780136 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195c012770 con 0x7f196c0b21d0 2023-09-23T03:25:54.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.190+0000 7f1968ff9640 1 -- 172.21.15.154:0/132780136 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c012b20 con 0x7f196c0b21d0 2023-09-23T03:25:54.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.190+0000 7f197253c640 1 -- 172.21.15.154:0/132780136 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f196c06b920 con 0x7f196c0b21d0 2023-09-23T03:25:54.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.190+0000 7f197253c640 1 -- 172.21.15.154:0/132780136 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f196c06bee0 con 0x7f196c0b21d0 2023-09-23T03:25:54.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.192+0000 7f1968ff9640 1 -- 172.21.15.154:0/132780136 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f195c012c80 con 0x7f196c0b21d0 2023-09-23T03:25:54.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.193+0000 7f1968ff9640 1 --2- 172.21.15.154:0/132780136 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f194c06af30 0x7f194c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:54.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.193+0000 7f1968ff9640 1 -- 172.21.15.154:0/132780136 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f195c089e00 con 0x7f196c0b21d0 2023-09-23T03:25:54.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.193+0000 7f196b7fe640 1 --2- 172.21.15.154:0/132780136 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f194c06af30 0x7f194c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:54.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.194+0000 7f19751ab740 1 Processor -- start 2023-09-23T03:25:54.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.194+0000 7f19751ab740 1 -- start start 2023-09-23T03:25:54.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.194+0000 7f196b7fe640 1 --2- 172.21.15.154:0/132780136 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f194c06af30 0x7f194c06d3f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f1954036000 tx=0x7f1954008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:54.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.195+0000 7f19751ab740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a302ce1c00 0x55a302cb4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:54.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.195+0000 7f19751ab740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a302ce1fe0 0x55a302ce4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:54.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.195+0000 7f19751ab740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a302cb5040 0x55a302ce4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:54.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.195+0000 7f19751ab740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a302c286d0 con 0x55a302ce1c00 2023-09-23T03:25:54.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.195+0000 7f19751ab740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a302c25dc0 con 0x55a302ce1fe0 2023-09-23T03:25:54.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.195+0000 7f19751ab740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a302c216b0 con 0x55a302cb5040 2023-09-23T03:25:54.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.195+0000 7f196bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a302ce1c00 0x55a302cb4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:54.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.195+0000 7f196bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a302ce1c00 0x55a302cb4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50728/0 (socket says 172.21.15.154:50728) 2023-09-23T03:25:54.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.195+0000 7f196bfff640 1 -- 172.21.15.154:0/889150279 learned_addr learned my addr 172.21.15.154:0/889150279 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:54.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.196+0000 7f196bfff640 1 -- 172.21.15.154:0/889150279 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a302cb5040 msgr2=0x55a302ce4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:54.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.196+0000 7f196bfff640 1 --2- 172.21.15.154:0/889150279 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a302cb5040 0x55a302ce4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.196+0000 7f196b7fe640 1 --2- 172.21.15.154:0/889150279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a302ce1fe0 0x55a302ce4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:54.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.196+0000 7f196bfff640 1 -- 172.21.15.154:0/889150279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a302ce1fe0 msgr2=0x55a302ce4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:54.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.196+0000 7f196bfff640 1 --2- 172.21.15.154:0/889150279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a302ce1fe0 0x55a302ce4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.196+0000 7f196bfff640 1 -- 172.21.15.154:0/889150279 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a302be58e0 con 0x55a302ce1c00 2023-09-23T03:25:54.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.196+0000 7f196bfff640 1 --2- 172.21.15.154:0/889150279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a302ce1c00 0x55a302cb4b00 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f195c088150 tx=0x7f195c0590b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2da5f4211e8525 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:54.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.196+0000 7f1961ffb640 1 -- 172.21.15.154:0/889150279 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c05cb00 con 0x55a302ce1c00 2023-09-23T03:25:54.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.196+0000 7f1961ffb640 1 -- 172.21.15.154:0/889150279 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195c060cb0 con 0x55a302ce1c00 2023-09-23T03:25:54.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.196+0000 7f1961ffb640 1 -- 172.21.15.154:0/889150279 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c0550c0 con 0x55a302ce1c00 2023-09-23T03:25:54.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.197+0000 7f19751ab740 1 -- 172.21.15.154:0/889150279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a302ce1c00 msgr2=0x55a302cb4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:54.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.197+0000 7f19751ab740 1 --2- 172.21.15.154:0/889150279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a302ce1c00 0x55a302cb4b00 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f195c088150 tx=0x7f195c0590b0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.197+0000 7f19751ab740 1 -- 172.21.15.154:0/889150279 shutdown_connections 2023-09-23T03:25:54.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.197+0000 7f19751ab740 1 --2- 172.21.15.154:0/889150279 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a302cb5040 0x55a302ce4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.197+0000 7f19751ab740 1 --2- 172.21.15.154:0/889150279 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a302ce1fe0 0x55a302ce4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.197+0000 7f19751ab740 1 --2- 172.21.15.154:0/889150279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a302ce1c00 0x55a302cb4b00 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.197+0000 7f19751ab740 1 -- 172.21.15.154:0/889150279 >> 172.21.15.154:0/889150279 conn(0x55a302cb6e10 msgr2=0x55a302cff990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:54.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.197+0000 7f19751ab740 1 -- 172.21.15.154:0/889150279 shutdown_connections 2023-09-23T03:25:54.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.197+0000 7f19751ab740 1 -- 172.21.15.154:0/889150279 wait complete. 2023-09-23T03:25:54.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.198+0000 7f19751ab740 1 -- start start 2023-09-23T03:25:54.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.198+0000 7f19751ab740 1 Processor -- start 2023-09-23T03:25:54.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.199+0000 7f19751ab740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a302ce1c00 0x55a302cd5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:54.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.199+0000 7f19751ab740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a302ce1fe0 0x55a302cf9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:54.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.199+0000 7f19751ab740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a302d12d60 0x55a302d131e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:54.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.199+0000 7f19751ab740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a302bbf940 con 0x55a302ce1fe0 2023-09-23T03:25:54.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.199+0000 7f19751ab740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a302bbf5e0 con 0x55a302ce1c00 2023-09-23T03:25:54.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.199+0000 7f19751ab740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a302985f20 con 0x55a302d12d60 2023-09-23T03:25:54.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.199+0000 7f19751ab740 10 client.0 _ll_register_callbacks cb 0x55a302bfd5e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:25:54.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.199+0000 7f196b7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a302ce1fe0 0x55a302cf9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:54.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.199+0000 7f196b7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a302ce1fe0 0x55a302cf9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50740/0 (socket says 172.21.15.154:50740) 2023-09-23T03:25:54.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.199+0000 7f196b7fe640 1 -- 172.21.15.154:0/1571829489 learned_addr learned my addr 172.21.15.154:0/1571829489 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:54.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.200+0000 7f1970ad9640 1 --2- 172.21.15.154:0/1571829489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a302d12d60 0x55a302d131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:54.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.200+0000 7f196bfff640 1 --2- 172.21.15.154:0/1571829489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a302ce1c00 0x55a302cd5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:54.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.200+0000 7f196b7fe640 1 -- 172.21.15.154:0/1571829489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a302d12d60 msgr2=0x55a302d131e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:54.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.200+0000 7f196b7fe640 1 --2- 172.21.15.154:0/1571829489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a302d12d60 0x55a302d131e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.200+0000 7f196b7fe640 1 -- 172.21.15.154:0/1571829489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a302ce1c00 msgr2=0x55a302cd5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:54.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.200+0000 7f196b7fe640 1 --2- 172.21.15.154:0/1571829489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a302ce1c00 0x55a302cd5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.200+0000 7f196b7fe640 1 -- 172.21.15.154:0/1571829489 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f196c06bee0 con 0x55a302ce1fe0 2023-09-23T03:25:54.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.200+0000 7f196bfff640 1 --2- 172.21.15.154:0/1571829489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a302ce1c00 0x55a302cd5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:54.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.200+0000 7f196b7fe640 1 --2- 172.21.15.154:0/1571829489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a302ce1fe0 0x55a302cf9810 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f1954009890 tx=0x7f1954004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3d574161628093 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:54.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.200+0000 7f194affd640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:25:54.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.201+0000 7f194affd640 1 -- 172.21.15.154:0/1571829489 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195403f070 con 0x55a302ce1fe0 2023-09-23T03:25:54.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.201+0000 7f19751ab740 1 -- 172.21.15.154:0/1571829489 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a302c1a690 con 0x55a302ce1fe0 2023-09-23T03:25:54.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.201+0000 7f194affd640 1 -- 172.21.15.154:0/1571829489 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1954045af0 con 0x55a302ce1fe0 2023-09-23T03:25:54.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.201+0000 7f194affd640 1 -- 172.21.15.154:0/1571829489 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1954045cb0 con 0x55a302ce1fe0 2023-09-23T03:25:54.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.201+0000 7f19751ab740 10 client.8250 fetch_fsmap learned FSMap version 84 2023-09-23T03:25:54.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.201+0000 7f19751ab740 1 -- 172.21.15.154:0/1571829489 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) v3 -- 0x55a302c1fd30 con 0x55a302ce1fe0 2023-09-23T03:25:54.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.201+0000 7f194affd640 1 -- 172.21.15.154:0/1571829489 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a302c1a690 con 0x55a302ce1fe0 2023-09-23T03:25:54.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.201+0000 7f194affd640 1 -- 172.21.15.154:0/1571829489 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 84) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f195404e070 con 0x55a302ce1fe0 2023-09-23T03:25:54.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.201+0000 7f19751ab740 10 client.8250 fetch_fsmap finished waiting for FSMap version 84 2023-09-23T03:25:54.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.201+0000 7f19751ab740 10 client.8250 resolve_mds: resolved name 'd' to daemon mds.d 2023-09-23T03:25:54.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.201+0000 7f19751ab740 20 client.8250 populate_metadata read hostname 'smithi154' 2023-09-23T03:25:54.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.201+0000 7f19751ab740 1 --2- 172.21.15.154:0/1571829489 >> [v2:172.21.15.190:6833/2967125458,v1:172.21.15.190:6835/2967125458] conn(0x55a302ce7420 0x55a302cf7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:54.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.201+0000 7f19751ab740 4 client.8250 mds_command: new command op to 8115 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T03:25:54.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.201+0000 7f19751ab740 1 -- 172.21.15.154:0/1571829489 --> [v2:172.21.15.190:6833/2967125458,v1:172.21.15.190:6835/2967125458] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a302c1a3b0 con 0x55a302ce7420 2023-09-23T03:25:54.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.202+0000 7f196bfff640 1 --2- 172.21.15.154:0/1571829489 >> [v2:172.21.15.190:6833/2967125458,v1:172.21.15.190:6835/2967125458] conn(0x55a302ce7420 0x55a302cf7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:54.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.203+0000 7f196bfff640 1 --2- 172.21.15.154:0/1571829489 >> [v2:172.21.15.190:6833/2967125458,v1:172.21.15.190:6835/2967125458] conn(0x55a302ce7420 0x55a302cf7c50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=403ed261b7d1fd2e server_cookie=d08676eb15bc0750 in_seq=0 out_seq=0 2023-09-23T03:25:54.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.203+0000 7f194affd640 10 client.8250 ms_handle_connect on v2:172.21.15.190:6833/2967125458 2023-09-23T03:25:54.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.205+0000 7f194affd640 1 -- 172.21.15.154:0/1571829489 <== mds.0 v2:172.21.15.190:6833/2967125458 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55a302c1a3b0 con 0x55a302ce7420 2023-09-23T03:25:54.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.205+0000 7f194affd640 10 client.8250 handle_command_reply: tid=0 2023-09-23T03:25:54.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.205+0000 7f19751ab740 1 client.8250 shutdown 2023-09-23T03:25:54.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.205+0000 7f19751ab740 1 -- 172.21.15.154:0/1571829489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a302ce1fe0 msgr2=0x55a302cf9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:54.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.205+0000 7f19751ab740 1 --2- 172.21.15.154:0/1571829489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a302ce1fe0 0x55a302cf9810 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f1954009890 tx=0x7f1954004730 comp rx=0 tx=0).stop 2023-09-23T03:25:54.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.205+0000 7f19751ab740 1 -- 172.21.15.154:0/1571829489 shutdown_connections 2023-09-23T03:25:54.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.205+0000 7f19751ab740 1 --2- 172.21.15.154:0/1571829489 >> [v2:172.21.15.190:6833/2967125458,v1:172.21.15.190:6835/2967125458] conn(0x55a302ce7420 0x55a302cf7c50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.205+0000 7f19751ab740 1 --2- 172.21.15.154:0/1571829489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a302d12d60 0x55a302d131e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.205+0000 7f19751ab740 1 --2- 172.21.15.154:0/1571829489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a302ce1fe0 0x55a302cf9810 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.205+0000 7f19751ab740 1 --2- 172.21.15.154:0/1571829489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a302ce1c00 0x55a302cd5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.205+0000 7f194affd640 0 client.8250 ms_handle_reset on v2:172.21.15.190:6833/2967125458 2023-09-23T03:25:54.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.205+0000 7f19751ab740 1 -- 172.21.15.154:0/1571829489 >> 172.21.15.154:0/1571829489 conn(0x55a302cb5040 msgr2=0x55a302cb8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:54.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.205+0000 7f19751ab740 1 -- 172.21.15.154:0/1571829489 shutdown_connections 2023-09-23T03:25:54.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.205+0000 7f19751ab740 1 -- 172.21.15.154:0/1571829489 wait complete. 2023-09-23T03:25:54.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.205+0000 7f19751ab740 20 client.8250 trim_cache size 0 max 16384 2023-09-23T03:25:54.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.222+0000 7f19751ab740 1 Processor -- start 2023-09-23T03:25:54.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.222+0000 7f19751ab740 1 -- start start 2023-09-23T03:25:54.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.223+0000 7f19751ab740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a302ce1c00 0x55a302cf8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:54.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.223+0000 7f19751ab740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a302ce1fe0 0x55a302cb3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:54.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.223+0000 7f19751ab740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a302d12d60 0x55a302cdfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:54.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.223+0000 7f19751ab740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a302d02cd0 con 0x55a302ce1fe0 2023-09-23T03:25:54.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.223+0000 7f19751ab740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a302cb64d0 con 0x55a302ce1c00 2023-09-23T03:25:54.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.223+0000 7f19751ab740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a302ce51f0 con 0x55a302d12d60 2023-09-23T03:25:54.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.224+0000 7f196b7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a302ce1fe0 0x55a302cb3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:54.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.224+0000 7f196b7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a302ce1fe0 0x55a302cb3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50748/0 (socket says 172.21.15.154:50748) 2023-09-23T03:25:54.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.224+0000 7f196b7fe640 1 -- 172.21.15.154:0/2016937947 learned_addr learned my addr 172.21.15.154:0/2016937947 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:54.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.224+0000 7f196bfff640 1 --2- 172.21.15.154:0/2016937947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a302ce1c00 0x55a302cf8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:54.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.224+0000 7f1970ad9640 1 --2- 172.21.15.154:0/2016937947 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a302d12d60 0x55a302cdfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:54.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.224+0000 7f196b7fe640 1 -- 172.21.15.154:0/2016937947 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a302d12d60 msgr2=0x55a302cdfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:54.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.224+0000 7f196b7fe640 1 --2- 172.21.15.154:0/2016937947 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a302d12d60 0x55a302cdfe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.224+0000 7f196b7fe640 1 -- 172.21.15.154:0/2016937947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a302ce1c00 msgr2=0x55a302cf8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:54.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.224+0000 7f196b7fe640 1 --2- 172.21.15.154:0/2016937947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a302ce1c00 0x55a302cf8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.224+0000 7f196b7fe640 1 -- 172.21.15.154:0/2016937947 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a302bbeb50 con 0x55a302ce1fe0 2023-09-23T03:25:54.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.224+0000 7f196bfff640 1 --2- 172.21.15.154:0/2016937947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a302ce1c00 0x55a302cf8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:54.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.224+0000 7f196b7fe640 1 --2- 172.21.15.154:0/2016937947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a302ce1fe0 0x55a302cb3bc0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f19540098c0 tx=0x7f1954047070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78aaffb26248051e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:54.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.225+0000 7f19627fc640 1 -- 172.21.15.154:0/2016937947 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1954031b00 con 0x55a302ce1fe0 2023-09-23T03:25:54.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.225+0000 7f19627fc640 1 -- 172.21.15.154:0/2016937947 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1954031e90 con 0x55a302ce1fe0 2023-09-23T03:25:54.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.225+0000 7f19627fc640 1 -- 172.21.15.154:0/2016937947 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1954004240 con 0x55a302ce1fe0 2023-09-23T03:25:54.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.225+0000 7f19751ab740 1 -- 172.21.15.154:0/2016937947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a302ce1fe0 msgr2=0x55a302cb3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:54.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.225+0000 7f19751ab740 1 --2- 172.21.15.154:0/2016937947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a302ce1fe0 0x55a302cb3bc0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f19540098c0 tx=0x7f1954047070 comp rx=0 tx=0).stop 2023-09-23T03:25:54.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.225+0000 7f19751ab740 1 -- 172.21.15.154:0/2016937947 shutdown_connections 2023-09-23T03:25:54.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.225+0000 7f19751ab740 1 --2- 172.21.15.154:0/2016937947 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a302d12d60 0x55a302cdfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.225+0000 7f19751ab740 1 --2- 172.21.15.154:0/2016937947 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a302ce1fe0 0x55a302cb3bc0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.225+0000 7f19751ab740 1 --2- 172.21.15.154:0/2016937947 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a302ce1c00 0x55a302cf8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.225+0000 7f19751ab740 1 -- 172.21.15.154:0/2016937947 >> 172.21.15.154:0/2016937947 conn(0x55a302cb5040 msgr2=0x55a302d03330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:54.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.225+0000 7f19751ab740 1 -- 172.21.15.154:0/2016937947 shutdown_connections 2023-09-23T03:25:54.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.225+0000 7f19751ab740 1 -- 172.21.15.154:0/2016937947 wait complete. 2023-09-23T03:25:54.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.226+0000 7f19751ab740 1 -- start start 2023-09-23T03:25:54.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.226+0000 7f19751ab740 1 Processor -- start 2023-09-23T03:25:54.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.227+0000 7f19751ab740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a302ce1c00 0x55a302ce47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:54.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.227+0000 7f19751ab740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a302ce1fe0 0x55a302d11970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:54.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.227+0000 7f19751ab740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a302d12d60 0x55a302d51770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:54.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.227+0000 7f19751ab740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a302d00bc0 con 0x55a302ce1fe0 2023-09-23T03:25:54.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.227+0000 7f19751ab740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a302d12450 con 0x55a302ce1c00 2023-09-23T03:25:54.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.227+0000 7f19751ab740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a302cf5320 con 0x55a302d12d60 2023-09-23T03:25:54.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.227+0000 7f19751ab740 10 client.0 _ll_register_callbacks cb 0x7f1954009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:25:54.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.227+0000 7f196b7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a302ce1fe0 0x55a302d11970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:54.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.227+0000 7f196b7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a302ce1fe0 0x55a302d11970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50752/0 (socket says 172.21.15.154:50752) 2023-09-23T03:25:54.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.227+0000 7f196b7fe640 1 -- 172.21.15.154:0/417734045 learned_addr learned my addr 172.21.15.154:0/417734045 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:54.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.228+0000 7f196bfff640 1 --2- 172.21.15.154:0/417734045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a302ce1c00 0x55a302ce47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:54.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.228+0000 7f1970ad9640 1 --2- 172.21.15.154:0/417734045 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a302d12d60 0x55a302d51770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:54.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.228+0000 7f196b7fe640 1 -- 172.21.15.154:0/417734045 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a302d12d60 msgr2=0x55a302d51770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:54.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.228+0000 7f196b7fe640 1 --2- 172.21.15.154:0/417734045 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a302d12d60 0x55a302d51770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.228+0000 7f196b7fe640 1 -- 172.21.15.154:0/417734045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a302ce1c00 msgr2=0x55a302ce47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:54.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.228+0000 7f196b7fe640 1 --2- 172.21.15.154:0/417734045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a302ce1c00 0x55a302ce47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.228+0000 7f196b7fe640 1 -- 172.21.15.154:0/417734045 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a302c1fd30 con 0x55a302ce1fe0 2023-09-23T03:25:54.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.228+0000 7f196b7fe640 1 --2- 172.21.15.154:0/417734045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a302ce1fe0 0x55a302d11970 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f195402b910 tx=0x7f1954009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d97ae1b63f0911d3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:54.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.228+0000 7f194affd640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:25:54.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.229+0000 7f194affd640 1 -- 172.21.15.154:0/417734045 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195404f650 con 0x55a302ce1fe0 2023-09-23T03:25:54.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.229+0000 7f19751ab740 1 -- 172.21.15.154:0/417734045 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f195c060cb0 con 0x55a302ce1fe0 2023-09-23T03:25:54.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.229+0000 7f194affd640 1 -- 172.21.15.154:0/417734045 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f195404f7b0 con 0x55a302ce1fe0 2023-09-23T03:25:54.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.229+0000 7f194affd640 1 -- 172.21.15.154:0/417734045 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f195404f990 con 0x55a302ce1fe0 2023-09-23T03:25:54.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.229+0000 7f19751ab740 10 client.8256 fetch_fsmap learned FSMap version 84 2023-09-23T03:25:54.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.229+0000 7f19751ab740 1 -- 172.21.15.154:0/417734045 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) v3 -- 0x55a302be4140 con 0x55a302ce1fe0 2023-09-23T03:25:54.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.229+0000 7f194affd640 1 -- 172.21.15.154:0/417734045 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195c060cb0 con 0x55a302ce1fe0 2023-09-23T03:25:54.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.229+0000 7f194affd640 1 -- 172.21.15.154:0/417734045 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 84) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f195404e040 con 0x55a302ce1fe0 2023-09-23T03:25:54.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.229+0000 7f19751ab740 10 client.8256 fetch_fsmap finished waiting for FSMap version 84 2023-09-23T03:25:54.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.229+0000 7f19751ab740 10 client.8256 resolve_mds: resolved name 'd' to daemon mds.d 2023-09-23T03:25:54.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.229+0000 7f19751ab740 20 client.8256 populate_metadata read hostname 'smithi154' 2023-09-23T03:25:54.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.229+0000 7f19751ab740 1 --2- 172.21.15.154:0/417734045 >> [v2:172.21.15.190:6833/2967125458,v1:172.21.15.190:6835/2967125458] conn(0x55a302cd44f0 0x55a302cd48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:54.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.230+0000 7f19751ab740 4 client.8256 mds_command: new command op to 8115 tid=0[{"prefix": "status"}] 2023-09-23T03:25:54.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.230+0000 7f19751ab740 1 -- 172.21.15.154:0/417734045 --> [v2:172.21.15.190:6833/2967125458,v1:172.21.15.190:6835/2967125458] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a3029236c0 con 0x55a302cd44f0 2023-09-23T03:25:54.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.230+0000 7f196bfff640 1 --2- 172.21.15.154:0/417734045 >> [v2:172.21.15.190:6833/2967125458,v1:172.21.15.190:6835/2967125458] conn(0x55a302cd44f0 0x55a302cd48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:54.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.231+0000 7f196bfff640 1 --2- 172.21.15.154:0/417734045 >> [v2:172.21.15.190:6833/2967125458,v1:172.21.15.190:6835/2967125458] conn(0x55a302cd44f0 0x55a302cd48d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5046feb208dc0e65 server_cookie=7d042475c1dcb497 in_seq=0 out_seq=0 2023-09-23T03:25:54.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.231+0000 7f194affd640 10 client.8256 ms_handle_connect on v2:172.21.15.190:6833/2967125458 2023-09-23T03:25:54.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.231+0000 7f194affd640 1 -- 172.21.15.154:0/417734045 <== mds.0 v2:172.21.15.190:6833/2967125458 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55a3029236c0 con 0x55a302cd44f0 2023-09-23T03:25:54.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.231+0000 7f194affd640 10 client.8256 handle_command_reply: tid=0 2023-09-23T03:25:54.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.232+0000 7f19751ab740 1 client.8256 shutdown 2023-09-23T03:25:54.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.232+0000 7f19751ab740 1 -- 172.21.15.154:0/417734045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a302ce1fe0 msgr2=0x55a302d11970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:54.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.232+0000 7f19751ab740 1 --2- 172.21.15.154:0/417734045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a302ce1fe0 0x55a302d11970 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f195402b910 tx=0x7f1954009190 comp rx=0 tx=0).stop 2023-09-23T03:25:54.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.232+0000 7f19751ab740 1 -- 172.21.15.154:0/417734045 shutdown_connections 2023-09-23T03:25:54.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.232+0000 7f19751ab740 1 --2- 172.21.15.154:0/417734045 >> [v2:172.21.15.190:6833/2967125458,v1:172.21.15.190:6835/2967125458] conn(0x55a302cd44f0 0x55a302cd48d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.232+0000 7f19751ab740 1 --2- 172.21.15.154:0/417734045 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55a302d12d60 0x55a302d51770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.232+0000 7f19751ab740 1 --2- 172.21.15.154:0/417734045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a302ce1fe0 0x55a302d11970 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.232+0000 7f194affd640 0 client.8256 ms_handle_reset on v2:172.21.15.190:6833/2967125458 2023-09-23T03:25:54.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.232+0000 7f19751ab740 1 --2- 172.21.15.154:0/417734045 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55a302ce1c00 0x55a302ce47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.232+0000 7f19751ab740 1 -- 172.21.15.154:0/417734045 >> 172.21.15.154:0/417734045 conn(0x55a302cb5040 msgr2=0x55a302ce79f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:54.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.232+0000 7f19751ab740 1 -- 172.21.15.154:0/417734045 shutdown_connections 2023-09-23T03:25:54.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.232+0000 7f19751ab740 1 -- 172.21.15.154:0/417734045 wait complete. 2023-09-23T03:25:54.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.232+0000 7f19751ab740 20 client.8256 trim_cache size 0 max 16384 2023-09-23T03:25:54.235 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:25:54.235 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_fsid": "8c8039d3-6c84-43db-8808-85e0dbdd47c8", 2023-09-23T03:25:54.236 INFO:teuthology.orchestra.run.smithi154.stdout: "whoami": 0, 2023-09-23T03:25:54.236 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 8115, 2023-09-23T03:25:54.236 INFO:teuthology.orchestra.run.smithi154.stdout: "want_state": "up:active", 2023-09-23T03:25:54.236 INFO:teuthology.orchestra.run.smithi154.stdout: "state": "up:active", 2023-09-23T03:25:54.236 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs", 2023-09-23T03:25:54.236 INFO:teuthology.orchestra.run.smithi154.stdout: "rank_uptime": 6.6963205109999997, 2023-09-23T03:25:54.236 INFO:teuthology.orchestra.run.smithi154.stdout: "mdsmap_epoch": 84, 2023-09-23T03:25:54.236 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_epoch": 105, 2023-09-23T03:25:54.236 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_epoch_barrier": 105, 2023-09-23T03:25:54.236 INFO:teuthology.orchestra.run.smithi154.stdout: "uptime": 13.066889993 2023-09-23T03:25:54.236 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:25:54.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.233+0000 7f197253c640 1 -- 172.21.15.154:0/132780136 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f194c06af30 msgr2=0x7f194c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:54.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.233+0000 7f197253c640 1 --2- 172.21.15.154:0/132780136 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f194c06af30 0x7f194c06d3f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f1954036000 tx=0x7f1954008030 comp rx=0 tx=0).stop 2023-09-23T03:25:54.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.233+0000 7f197253c640 1 -- 172.21.15.154:0/132780136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f196c0b21d0 msgr2=0x7f196c0af050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:54.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.233+0000 7f197253c640 1 --2- 172.21.15.154:0/132780136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f196c0b21d0 0x7f196c0af050 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f195c00d5d0 tx=0x7f195c00da90 comp rx=0 tx=0).stop 2023-09-23T03:25:54.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.233+0000 7f197253c640 1 -- 172.21.15.154:0/132780136 shutdown_connections 2023-09-23T03:25:54.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.233+0000 7f197253c640 1 --2- 172.21.15.154:0/132780136 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f194c06af30 0x7f194c06d3f0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.233+0000 7f197253c640 1 --2- 172.21.15.154:0/132780136 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f196c15ad20 0x7f196c0c4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.233+0000 7f197253c640 1 --2- 172.21.15.154:0/132780136 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f196c0cd160 0x7f196c1547e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.233+0000 7f197253c640 1 --2- 172.21.15.154:0/132780136 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f196c0b21d0 0x7f196c0af050 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.233+0000 7f197253c640 1 -- 172.21.15.154:0/132780136 >> 172.21.15.154:0/132780136 conn(0x7f196c06fc60 msgr2=0x7f196c0b17c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:54.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.233+0000 7f197253c640 1 -- 172.21.15.154:0/132780136 shutdown_connections 2023-09-23T03:25:54.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.233+0000 7f197253c640 1 -- 172.21.15.154:0/132780136 wait complete. 2023-09-23T03:25:54.250 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-23T03:25:54.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.463+0000 7ff902f05640 1 Processor -- start 2023-09-23T03:25:54.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.463+0000 7ff902f05640 1 -- start start 2023-09-23T03:25:54.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.464+0000 7ff902f05640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff8fc0acce0 0x7ff8fc15ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:54.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.465+0000 7ff902f05640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8fc15ac30 con 0x7ff8fc0cf040 2023-09-23T03:25:54.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.465+0000 7ff902f05640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8fc0b1f70 con 0x7ff8fc0b1b20 2023-09-23T03:25:54.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.465+0000 7ff902f05640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8fc0b2e90 con 0x7ff8fc0acce0 2023-09-23T03:25:54.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.465+0000 7ff9014a2640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff8fc0cf040 0x7ff8fc0ab140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:60378/0 (socket says 172.21.15.154:60378) 2023-09-23T03:25:54.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.465+0000 7ff9014a2640 1 -- 172.21.15.154:0/3493532890 learned_addr learned my addr 172.21.15.154:0/3493532890 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:25:54.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.465+0000 7ff900ca1640 1 --2- 172.21.15.154:0/3493532890 >> v2:172.21.15.154:3300/0 conn(0x7ff8fc0acce0 0x7ff8fc15ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:54.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.465+0000 7ff8fb7fe640 1 -- 172.21.15.154:0/3493532890 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4171250095 0 0) 0x7ff8f4001490 con 0x7ff8fc0cf040 2023-09-23T03:25:54.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.465+0000 7ff8fb7fe640 1 -- 172.21.15.154:0/3493532890 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8e4003520 con 0x7ff8fc0cf040 2023-09-23T03:25:54.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.466+0000 7ff900ca1640 1 -- 172.21.15.154:0/3493532890 >> v1:172.21.15.190:6789/0 conn(0x7ff8fc0b1b20 legacy=0x7ff8fc0ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:54.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.466+0000 7ff900ca1640 1 -- 172.21.15.154:0/3493532890 >> v1:172.21.15.154:6789/0 conn(0x7ff8fc0cf040 legacy=0x7ff8fc0ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:54.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.466+0000 7ff900ca1640 1 -- 172.21.15.154:0/3493532890 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8fc15f2a0 con 0x7ff8fc0acce0 2023-09-23T03:25:54.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.467+0000 7ff900ca1640 1 --2- 172.21.15.154:0/3493532890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8fc0acce0 0x7ff8fc15ed60 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0009790 tx=0x7ff8f002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe6edf4648755bd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:54.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.467+0000 7ff8fb7fe640 1 -- 172.21.15.154:0/3493532890 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f002d040 con 0x7ff8fc0acce0 2023-09-23T03:25:54.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.467+0000 7ff8fb7fe640 1 -- 172.21.15.154:0/3493532890 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f0031810 con 0x7ff8fc0acce0 2023-09-23T03:25:54.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.467+0000 7ff8fb7fe640 1 -- 172.21.15.154:0/3493532890 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0031ba0 con 0x7ff8fc0acce0 2023-09-23T03:25:54.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.468+0000 7ff902f05640 1 -- 172.21.15.154:0/3493532890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8fc0acce0 msgr2=0x7ff8fc15ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:54.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.468+0000 7ff902f05640 1 --2- 172.21.15.154:0/3493532890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8fc0acce0 0x7ff8fc15ed60 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0009790 tx=0x7ff8f002b8b0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.468+0000 7ff902f05640 1 -- 172.21.15.154:0/3493532890 shutdown_connections 2023-09-23T03:25:54.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.468+0000 7ff902f05640 1 --2- 172.21.15.154:0/3493532890 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8fc0acce0 0x7ff8fc15ed60 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.468+0000 7ff902f05640 1 -- 172.21.15.154:0/3493532890 >> 172.21.15.154:0/3493532890 conn(0x7ff8fc06a0f0 msgr2=0x7ff8fc06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:54.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.468+0000 7ff902f05640 1 -- 172.21.15.154:0/3493532890 shutdown_connections 2023-09-23T03:25:54.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.468+0000 7ff902f05640 1 -- 172.21.15.154:0/3493532890 wait complete. 2023-09-23T03:25:54.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.469+0000 7ff902f05640 1 Processor -- start 2023-09-23T03:25:54.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.470+0000 7ff902f05640 1 -- start start 2023-09-23T03:25:54.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.470+0000 7ff902f05640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8fc0acce0 0x7ff8fc155960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:54.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.471+0000 7ff902f05640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8fc0b1b20 0x7ff8fc0c3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:54.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.471+0000 7ff902f05640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8fc0cf040 0x7ff8fc158e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:54.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.471+0000 7ff902f05640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc0c3d50 con 0x7ff8fc0b1b20 2023-09-23T03:25:54.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.471+0000 7ff902f05640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc0c4030 con 0x7ff8fc0acce0 2023-09-23T03:25:54.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.471+0000 7ff902f05640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc0c4310 con 0x7ff8fc0cf040 2023-09-23T03:25:54.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.471+0000 7ff8fbfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8fc0b1b20 0x7ff8fc0c3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:54.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.471+0000 7ff8fbfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8fc0b1b20 0x7ff8fc0c3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50766/0 (socket says 172.21.15.154:50766) 2023-09-23T03:25:54.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.471+0000 7ff8fbfff640 1 -- 172.21.15.154:0/1317085526 learned_addr learned my addr 172.21.15.154:0/1317085526 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:54.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.471+0000 7ff900ca1640 1 --2- 172.21.15.154:0/1317085526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8fc0acce0 0x7ff8fc155960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:54.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.471+0000 7ff9014a2640 1 --2- 172.21.15.154:0/1317085526 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8fc0cf040 0x7ff8fc158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:54.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.471+0000 7ff8fbfff640 1 -- 172.21.15.154:0/1317085526 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8fc0cf040 msgr2=0x7ff8fc158e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:54.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.471+0000 7ff8fbfff640 1 --2- 172.21.15.154:0/1317085526 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8fc0cf040 0x7ff8fc158e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.471+0000 7ff8fbfff640 1 -- 172.21.15.154:0/1317085526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8fc0acce0 msgr2=0x7ff8fc155960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:54.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.471+0000 7ff8fbfff640 1 --2- 172.21.15.154:0/1317085526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8fc0acce0 0x7ff8fc155960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.471+0000 7ff8fbfff640 1 -- 172.21.15.154:0/1317085526 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f00093a0 con 0x7ff8fc0b1b20 2023-09-23T03:25:54.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.472+0000 7ff8fbfff640 1 --2- 172.21.15.154:0/1317085526 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8fc0b1b20 0x7ff8fc0c3350 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec00c450 tx=0x7ff8ec00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:54.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.472+0000 7ff8f97fa640 1 -- 172.21.15.154:0/1317085526 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec0052d0 con 0x7ff8fc0b1b20 2023-09-23T03:25:54.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.472+0000 7ff8f97fa640 1 -- 172.21.15.154:0/1317085526 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8ec005430 con 0x7ff8fc0b1b20 2023-09-23T03:25:54.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.472+0000 7ff8f97fa640 1 -- 172.21.15.154:0/1317085526 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec012870 con 0x7ff8fc0b1b20 2023-09-23T03:25:54.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.472+0000 7ff902f05640 1 -- 172.21.15.154:0/1317085526 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8fc0c45f0 con 0x7ff8fc0b1b20 2023-09-23T03:25:54.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.472+0000 7ff902f05640 1 -- 172.21.15.154:0/1317085526 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8fc155ea0 con 0x7ff8fc0b1b20 2023-09-23T03:25:54.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.473+0000 7ff902f05640 1 -- 172.21.15.154:0/1317085526 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8fc073eb0 con 0x7ff8fc0b1b20 2023-09-23T03:25:54.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.474+0000 7ff8f97fa640 1 -- 172.21.15.154:0/1317085526 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7ff8ec0129d0 con 0x7ff8fc0b1b20 2023-09-23T03:25:54.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.475+0000 7ff8f97fa640 1 --2- 172.21.15.154:0/1317085526 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff8e806af30 0x7ff8e806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:54.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.475+0000 7ff8f97fa640 1 -- 172.21.15.154:0/1317085526 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff8ec089530 con 0x7ff8fc0b1b20 2023-09-23T03:25:54.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.475+0000 7ff900ca1640 1 --2- 172.21.15.154:0/1317085526 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff8e806af30 0x7ff8e806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:54.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.476+0000 7ff900ca1640 1 --2- 172.21.15.154:0/1317085526 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff8e806af30 0x7ff8e806d3f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0036000 tx=0x7ff8f003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:54.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.479+0000 7ff8f97fa640 1 -- 172.21.15.154:0/1317085526 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7ff8ec054300 con 0x7ff8fc0b1b20 2023-09-23T03:25:54.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.713+0000 7ff902f05640 1 -- 172.21.15.154:0/1317085526 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff8fc0ac850 con 0x7ff8fc0b1b20 2023-09-23T03:25:54.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.714+0000 7ff8f97fa640 1 -- 172.21.15.154:0/1317085526 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v84) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff8ec057920 con 0x7ff8fc0b1b20 2023-09-23T03:25:54.714 INFO:teuthology.orchestra.run.smithi154.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-09-23T03:25:54.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.717+0000 7ff902f05640 1 -- 172.21.15.154:0/1317085526 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff8e806af30 msgr2=0x7ff8e806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:54.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.717+0000 7ff902f05640 1 --2- 172.21.15.154:0/1317085526 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff8e806af30 0x7ff8e806d3f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0036000 tx=0x7ff8f003c040 comp rx=0 tx=0).stop 2023-09-23T03:25:54.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.717+0000 7ff902f05640 1 -- 172.21.15.154:0/1317085526 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8fc0b1b20 msgr2=0x7ff8fc0c3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:54.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.717+0000 7ff902f05640 1 --2- 172.21.15.154:0/1317085526 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8fc0b1b20 0x7ff8fc0c3350 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec00c450 tx=0x7ff8ec00c910 comp rx=0 tx=0).stop 2023-09-23T03:25:54.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.717+0000 7ff902f05640 1 -- 172.21.15.154:0/1317085526 shutdown_connections 2023-09-23T03:25:54.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.717+0000 7ff902f05640 1 --2- 172.21.15.154:0/1317085526 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff8e806af30 0x7ff8e806d3f0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.717+0000 7ff902f05640 1 --2- 172.21.15.154:0/1317085526 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff8fc0cf040 0x7ff8fc158e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.717+0000 7ff902f05640 1 --2- 172.21.15.154:0/1317085526 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8fc0b1b20 0x7ff8fc0c3350 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.717+0000 7ff902f05640 1 --2- 172.21.15.154:0/1317085526 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff8fc0acce0 0x7ff8fc155960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:54.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.717+0000 7ff902f05640 1 -- 172.21.15.154:0/1317085526 >> 172.21.15.154:0/1317085526 conn(0x7ff8fc06a0f0 msgr2=0x7ff8fc15a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:54.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.717+0000 7ff902f05640 1 -- 172.21.15.154:0/1317085526 shutdown_connections 2023-09-23T03:25:54.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:54.717+0000 7ff902f05640 1 -- 172.21.15.154:0/1317085526 wait complete. 2023-09-23T03:25:54.735 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-23T03:25:54.735 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-23T03:25:54.735 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi154.front.sepia.ceph.com 2023-09-23T03:25:54.735 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-09-23T03:25:54.735 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-23T03:25:54.735 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-23T03:25:54.736 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-23T03:25:54.736 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-23T03:25:54.736 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-23T03:25:54.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:25:54.736 DEBUG:teuthology.orchestra.run.smithi154:> ip addr 2023-09-23T03:25:54.789 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-23T03:25:54.789 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-23T03:25:54.789 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2023-09-23T03:25:54.790 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:25:54.790 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2023-09-23T03:25:54.790 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:25:54.790 INFO:teuthology.orchestra.run.smithi154.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:25:54.790 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:54 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:25:54.790 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f0 2023-09-23T03:25:54.790 INFO:teuthology.orchestra.run.smithi154.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:25:54.790 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:55 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:25:54.790 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f1 2023-09-23T03:25:54.790 INFO:teuthology.orchestra.run.smithi154.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:25:54.790 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:60 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:25:54.791 INFO:teuthology.orchestra.run.smithi154.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-23T03:25:54.791 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:61 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:25:54.791 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.21.15.154/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-09-23T03:25:54.791 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft 41770sec preferred_lft 41770sec 2023-09-23T03:25:54.791 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::ec4:7aff:fe88:7761/64 scope link 2023-09-23T03:25:54.791 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:25:54.791 INFO:teuthology.orchestra.run.smithi154.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-23T03:25:54.791 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 5a:c6:b2:96:fb:dc brd ff:ff:ff:ff:ff:ff 2023-09-23T03:25:54.791 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-09-23T03:25:54.791 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:25:54.791 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::a801:22ff:febf:2833/64 scope link 2023-09-23T03:25:54.791 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:25:54.791 INFO:teuthology.orchestra.run.smithi154.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-23T03:25:54.792 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 5a:c6:b2:96:fb:dc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:25:54.792 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::58c6:b2ff:fe96:fbdc/64 scope link 2023-09-23T03:25:54.792 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:25:54.792 INFO:teuthology.orchestra.run.smithi154.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-23T03:25:54.792 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether b2:23:b0:e4:c2:ab brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:25:54.792 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::b023:b0ff:fee4:c2ab/64 scope link 2023-09-23T03:25:54.792 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:25:54.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:25:54.793 DEBUG:teuthology.orchestra.run.smithi154:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-23T03:25:54.881 INFO:teuthology.orchestra.run.smithi154.stdout:1 2023-09-23T03:25:54.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:25:54.886 DEBUG:teuthology.orchestra.run.smithi154:> route 2023-09-23T03:25:54.939 INFO:teuthology.orchestra.run.smithi154.stdout:Kernel IP routing table 2023-09-23T03:25:54.939 INFO:teuthology.orchestra.run.smithi154.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-23T03:25:54.939 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-23T03:25:54.939 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-09-23T03:25:54.939 INFO:teuthology.orchestra.run.smithi154.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-23T03:25:54.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:25:54.940 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-09-23T03:25:54.940 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-09-23T03:25:54.940 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-23T03:25:54.940 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-23T03:25:54.941 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-23T03:25:54.941 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-09-23T03:25:55.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:25:55.122 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2023-09-23T03:25:55.171 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-09-23T03:25:55.171 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-09-23T03:25:55.172 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-23T03:25:55.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:25:55.172 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set brx.0 up 2023-09-23T03:25:55.262 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-09-23T03:25:55.263 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-23T03:25:55.263 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-09-23T03:25:55.315 INFO:teuthology.orchestra.run.smithi154.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-09-23T03:25:55.315 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-23T03:25:55.315 DEBUG:teuthology.orchestra.run.smithi154:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-09-23T03:25:55.362 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-09-23T03:25:55.507 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2023-09-23T03:25:55.565 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2023-09-23T03:25:55.565 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2023-09-23T03:25:55.565 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2023-09-23T03:25:55.566 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2023-09-23T03:25:55.566 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2023-09-23T03:25:55.566 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2023-09-23T03:25:55.566 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2023-09-23T03:25:55.566 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2023-09-23T03:25:55.566 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2023-09-23T03:25:55.566 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2023-09-23T03:25:55.566 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2023-09-23T03:25:55.566 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T03:25:55.566 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2023-09-23T03:25:55.566 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2023-09-23T03:25:55.567 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2023-09-23T03:25:55.567 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2023-09-23T03:25:55.567 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2023-09-23T03:25:55.567 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2023-09-23T03:25:55.567 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2023-09-23T03:25:55.567 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-09-23T03:25:55.567 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-09-23T03:25:55.567 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2023-09-23T03:25:55.567 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2023-09-23T03:25:55.567 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2023-09-23T03:25:55.567 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-0 2023-09-23T03:25:55.567 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-1 2023-09-23T03:25:55.567 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-2 2023-09-23T03:25:55.568 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-3 2023-09-23T03:25:55.568 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2023-09-23T03:25:55.568 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:25:55.568 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:25:55.568 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:25:55.568 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:25:55.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:25:55.569 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-09-23T03:25:55.623 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-23T03:25:55.623 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-09-23T03:25:55.628 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-09-23T03:25:55.709 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2023-09-23T03:25:55.755 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2023-09-23T03:25:55.755 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2023-09-23T03:25:55.756 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2023-09-23T03:25:55.756 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2023-09-23T03:25:55.756 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2023-09-23T03:25:55.756 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2023-09-23T03:25:55.756 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2023-09-23T03:25:55.756 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2023-09-23T03:25:55.756 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2023-09-23T03:25:55.756 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2023-09-23T03:25:55.756 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2023-09-23T03:25:55.756 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T03:25:55.756 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2023-09-23T03:25:55.756 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2023-09-23T03:25:55.757 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2023-09-23T03:25:55.757 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2023-09-23T03:25:55.757 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2023-09-23T03:25:55.757 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2023-09-23T03:25:55.757 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2023-09-23T03:25:55.757 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-09-23T03:25:55.757 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-09-23T03:25:55.757 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2023-09-23T03:25:55.757 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2023-09-23T03:25:55.757 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2023-09-23T03:25:55.757 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-0 2023-09-23T03:25:55.757 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-1 2023-09-23T03:25:55.758 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-2 2023-09-23T03:25:55.758 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-3 2023-09-23T03:25:55.758 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2023-09-23T03:25:55.758 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:25:55.758 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:25:55.758 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:25:55.758 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:25:55.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:25:55.759 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-09-23T03:25:55.819 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:2023-09-23T03:25:55.819+0000 7f7c5603a480 -1 init, newargv = 0x560362d8fc60 newargc=15 2023-09-23T03:25:55.820 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:ceph-fuse[121281]: starting ceph client 2023-09-23T03:25:55.829 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:ceph-fuse[121281]: starting fuse 2023-09-23T03:25:55.843 INFO:teuthology.orchestra.run.smithi154.stdout:39 2023-09-23T03:25:55.844 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-09-23T03:25:55.844 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:25:55.844 DEBUG:teuthology.orchestra.run.smithi154:> import glob 2023-09-23T03:25:55.844 DEBUG:teuthology.orchestra.run.smithi154:> import re 2023-09-23T03:25:55.844 DEBUG:teuthology.orchestra.run.smithi154:> import os 2023-09-23T03:25:55.844 DEBUG:teuthology.orchestra.run.smithi154:> import subprocess 2023-09-23T03:25:55.844 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:25:55.844 DEBUG:teuthology.orchestra.run.smithi154:> def _find_admin_socket(client_name): 2023-09-23T03:25:55.844 DEBUG:teuthology.orchestra.run.smithi154:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-09-23T03:25:55.845 DEBUG:teuthology.orchestra.run.smithi154:> files = glob.glob(asok_path) 2023-09-23T03:25:55.845 DEBUG:teuthology.orchestra.run.smithi154:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-09-23T03:25:55.845 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:25:55.845 DEBUG:teuthology.orchestra.run.smithi154:> # Given a non-glob path, it better be there 2023-09-23T03:25:55.845 DEBUG:teuthology.orchestra.run.smithi154:> if "*" not in asok_path: 2023-09-23T03:25:55.845 DEBUG:teuthology.orchestra.run.smithi154:> assert(len(files) == 1) 2023-09-23T03:25:55.845 DEBUG:teuthology.orchestra.run.smithi154:> return files[0] 2023-09-23T03:25:55.845 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:25:55.845 DEBUG:teuthology.orchestra.run.smithi154:> for f in files: 2023-09-23T03:25:55.845 DEBUG:teuthology.orchestra.run.smithi154:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-23T03:25:55.845 DEBUG:teuthology.orchestra.run.smithi154:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-23T03:25:55.845 DEBUG:teuthology.orchestra.run.smithi154:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-23T03:25:55.845 DEBUG:teuthology.orchestra.run.smithi154:> contents = proc_f.read() 2023-09-23T03:25:55.846 DEBUG:teuthology.orchestra.run.smithi154:> if mountpoint in contents: 2023-09-23T03:25:55.846 DEBUG:teuthology.orchestra.run.smithi154:> return f 2023-09-23T03:25:55.846 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-23T03:25:55.846 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:25:55.846 DEBUG:teuthology.orchestra.run.smithi154:> print(_find_admin_socket("client.0")) 2023-09-23T03:25:55.846 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:25:56.054 INFO:teuthology.orchestra.run.smithi154.stdout:/var/run/ceph/ceph-client.0.121281.asok 2023-09-23T03:25:56.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:25:56.072 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.121281.asok 2023-09-23T03:25:56.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:25:56.072 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.121281.asok status 2023-09-23T03:25:56.313 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:25:56.313 INFO:teuthology.orchestra.run.smithi154.stdout: "metadata": { 2023-09-23T03:25:56.313 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_sha1": "862663913a74a99e1c52d50e6a7e59dd6705749a", 2023-09-23T03:25:56.313 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_version": "ceph version 18.0.0-6336-g86266391 (862663913a74a99e1c52d50e6a7e59dd6705749a) reef (dev)", 2023-09-23T03:25:56.313 INFO:teuthology.orchestra.run.smithi154.stdout: "entity_id": "0", 2023-09-23T03:25:56.313 INFO:teuthology.orchestra.run.smithi154.stdout: "hostname": "smithi154", 2023-09-23T03:25:56.313 INFO:teuthology.orchestra.run.smithi154.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-09-23T03:25:56.313 INFO:teuthology.orchestra.run.smithi154.stdout: "pid": "121281", 2023-09-23T03:25:56.313 INFO:teuthology.orchestra.run.smithi154.stdout: "root": "/" 2023-09-23T03:25:56.314 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:25:56.314 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_count": 0, 2023-09-23T03:25:56.314 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_pinned_count": 0, 2023-09-23T03:25:56.314 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 8271, 2023-09-23T03:25:56.314 INFO:teuthology.orchestra.run.smithi154.stdout: "inst": { 2023-09-23T03:25:56.314 INFO:teuthology.orchestra.run.smithi154.stdout: "name": { 2023-09-23T03:25:56.314 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "client", 2023-09-23T03:25:56.314 INFO:teuthology.orchestra.run.smithi154.stdout: "num": 8271 2023-09-23T03:25:56.314 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:25:56.314 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2023-09-23T03:25:56.314 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2023-09-23T03:25:56.315 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.1:0", 2023-09-23T03:25:56.315 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3071245840 2023-09-23T03:25:56.315 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:25:56.315 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:25:56.315 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2023-09-23T03:25:56.315 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2023-09-23T03:25:56.315 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.1:0", 2023-09-23T03:25:56.315 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3071245840 2023-09-23T03:25:56.315 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:25:56.315 INFO:teuthology.orchestra.run.smithi154.stdout: "inst_str": "client.8271 192.168.0.1:0/3071245840", 2023-09-23T03:25:56.315 INFO:teuthology.orchestra.run.smithi154.stdout: "addr_str": "192.168.0.1:0/3071245840", 2023-09-23T03:25:56.315 INFO:teuthology.orchestra.run.smithi154.stdout: "inode_count": 1, 2023-09-23T03:25:56.316 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_epoch": 84, 2023-09-23T03:25:56.316 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch": 105, 2023-09-23T03:25:56.316 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch_barrier": 0, 2023-09-23T03:25:56.316 INFO:teuthology.orchestra.run.smithi154.stdout: "blocklisted": false, 2023-09-23T03:25:56.316 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs" 2023-09-23T03:25:56.316 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:25:56.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:25:56.328 DEBUG:teuthology.orchestra.run.smithi154:> stat --file-system '--printf=%T 2023-09-23T03:25:56.328 DEBUG:teuthology.orchestra.run.smithi154:> ' -- /home/ubuntu/cephtest/mnt.0 2023-09-23T03:25:56.382 INFO:teuthology.orchestra.run.smithi154.stdout:fuseblk 2023-09-23T03:25:56.383 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-09-23T03:25:56.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:25:56.383 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-09-23T03:25:56.478 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_extra --pg_num_min 64 2023-09-23T03:25:56.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.701+0000 7f8e2c802640 1 Processor -- start 2023-09-23T03:25:56.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.701+0000 7f8e2c802640 1 -- start start 2023-09-23T03:25:56.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.703+0000 7f8e2c802640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8e2815fdf0 0x7f8e280d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:56.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.703+0000 7f8e2c802640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e280cf550 0x7f8e2806b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:56.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.703+0000 7f8e2c802640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e280b1ee0 con 0x7f8e281601d0 2023-09-23T03:25:56.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.703+0000 7f8e2c802640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e280d3620 con 0x7f8e2815fdf0 2023-09-23T03:25:56.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.703+0000 7f8e2c802640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e280ce770 con 0x7f8e280cf550 2023-09-23T03:25:56.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.703+0000 7f8e26ffd640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f8e281601d0 0x7f8e280cee40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:60388/0 (socket says 172.21.15.154:60388) 2023-09-23T03:25:56.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.703+0000 7f8e26ffd640 1 -- 172.21.15.154:0/4198080716 learned_addr learned my addr 172.21.15.154:0/4198080716 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:25:56.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.703+0000 7f8e277fe640 1 --2- 172.21.15.154:0/4198080716 >> v2:172.21.15.190:3300/0 conn(0x7f8e2815fdf0 0x7f8e280d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:56.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.703+0000 7f8e267fc640 1 -- 172.21.15.154:0/4198080716 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4067679237 0 0) 0x7f8e280b1ee0 con 0x7f8e281601d0 2023-09-23T03:25:56.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.704+0000 7f8e27fff640 1 --2- 172.21.15.154:0/4198080716 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e280cf550 0x7f8e2806b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:56.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.704+0000 7f8e267fc640 1 -- 172.21.15.154:0/4198080716 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e14003520 con 0x7f8e281601d0 2023-09-23T03:25:56.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.704+0000 7f8e267fc640 1 -- 172.21.15.154:0/4198080716 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2408961203 0 0) 0x7f8e14003520 con 0x7f8e281601d0 2023-09-23T03:25:56.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.704+0000 7f8e267fc640 1 -- 172.21.15.154:0/4198080716 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e280cf550 msgr2=0x7f8e2806b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:56.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.704+0000 7f8e267fc640 1 --2- 172.21.15.154:0/4198080716 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e280cf550 0x7f8e2806b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:56.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.704+0000 7f8e267fc640 1 -- 172.21.15.154:0/4198080716 >> v2:172.21.15.190:3300/0 conn(0x7f8e2815fdf0 msgr2=0x7f8e280d30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:56.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.704+0000 7f8e267fc640 1 --2- 172.21.15.154:0/4198080716 >> v2:172.21.15.190:3300/0 conn(0x7f8e2815fdf0 0x7f8e280d30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:56.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.704+0000 7f8e267fc640 1 -- 172.21.15.154:0/4198080716 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e2806ae20 con 0x7f8e281601d0 2023-09-23T03:25:56.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.704+0000 7f8e267fc640 1 -- 172.21.15.154:0/4198080716 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f8e10001670 con 0x7f8e281601d0 2023-09-23T03:25:56.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.704+0000 7f8e267fc640 1 -- 172.21.15.154:0/4198080716 >> v1:172.21.15.154:6789/0 conn(0x7f8e281601d0 legacy=0x7f8e280cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:56.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.704+0000 7f8e267fc640 1 --2- 172.21.15.154:0/4198080716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e14003970 0x7f8e14003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:56.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.704+0000 7f8e267fc640 1 --2- 172.21.15.154:0/4198080716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e140047c0 0x7f8e14004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:56.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.704+0000 7f8e267fc640 1 --2- 172.21.15.154:0/4198080716 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e280cf550 0x7f8e14005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:56.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.704+0000 7f8e267fc640 1 -- 172.21.15.154:0/4198080716 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e280d3620 con 0x7f8e140047c0 2023-09-23T03:25:56.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.704+0000 7f8e267fc640 1 -- 172.21.15.154:0/4198080716 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e280ce770 con 0x7f8e14003970 2023-09-23T03:25:56.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.704+0000 7f8e267fc640 1 -- 172.21.15.154:0/4198080716 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1400b9d0 con 0x7f8e280cf550 2023-09-23T03:25:56.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.704+0000 7f8e26ffd640 1 --2- 172.21.15.154:0/4198080716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e140047c0 0x7f8e14004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:56.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.705+0000 7f8e277fe640 1 --2- 172.21.15.154:0/4198080716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e14003970 0x7f8e14003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:56.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.705+0000 7f8e26ffd640 1 -- 172.21.15.154:0/4198080716 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e280cf550 msgr2=0x7f8e14005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:56.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.705+0000 7f8e26ffd640 1 --2- 172.21.15.154:0/4198080716 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e280cf550 0x7f8e14005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:56.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.705+0000 7f8e26ffd640 1 -- 172.21.15.154:0/4198080716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e14003970 msgr2=0x7f8e14003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:56.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.705+0000 7f8e26ffd640 1 --2- 172.21.15.154:0/4198080716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e14003970 0x7f8e14003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:56.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.705+0000 7f8e26ffd640 1 -- 172.21.15.154:0/4198080716 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e14003520 con 0x7f8e140047c0 2023-09-23T03:25:56.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.705+0000 7f8e277fe640 1 --2- 172.21.15.154:0/4198080716 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e14003970 0x7f8e14003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:56.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.706+0000 7f8e26ffd640 1 --2- 172.21.15.154:0/4198080716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e140047c0 0x7f8e14004c40 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f8e1000c210 tx=0x7f8e140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8494b10ae995e012 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:56.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.706+0000 7f8e267fc640 1 -- 172.21.15.154:0/4198080716 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1002cae0 con 0x7f8e140047c0 2023-09-23T03:25:56.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.706+0000 7f8e267fc640 1 -- 172.21.15.154:0/4198080716 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e1002cc40 con 0x7f8e140047c0 2023-09-23T03:25:56.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.706+0000 7f8e267fc640 1 -- 172.21.15.154:0/4198080716 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e100058d0 con 0x7f8e140047c0 2023-09-23T03:25:56.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.707+0000 7f8e2c802640 1 -- 172.21.15.154:0/4198080716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e140047c0 msgr2=0x7f8e14004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:56.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.707+0000 7f8e2c802640 1 --2- 172.21.15.154:0/4198080716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e140047c0 0x7f8e14004c40 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f8e1000c210 tx=0x7f8e140057f0 comp rx=0 tx=0).stop 2023-09-23T03:25:56.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.707+0000 7f8e27fff640 1 -- 172.21.15.154:0/4198080716 reap_dead start 2023-09-23T03:25:56.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.707+0000 7f8e2c802640 1 -- 172.21.15.154:0/4198080716 shutdown_connections 2023-09-23T03:25:56.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.707+0000 7f8e2c802640 1 -- 172.21.15.154:0/4198080716 >> 172.21.15.154:0/4198080716 conn(0x7f8e2809b9e0 msgr2=0x7f8e280d2660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:56.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.707+0000 7f8e2c802640 1 -- 172.21.15.154:0/4198080716 shutdown_connections 2023-09-23T03:25:56.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.707+0000 7f8e2c802640 1 -- 172.21.15.154:0/4198080716 wait complete. 2023-09-23T03:25:56.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.709+0000 7f8e2c802640 1 Processor -- start 2023-09-23T03:25:56.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.709+0000 7f8e2c802640 1 -- start start 2023-09-23T03:25:56.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.710+0000 7f8e2c802640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e140047c0 0x7f8e2815b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:56.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.710+0000 7f8e2c802640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e2815c470 0x7f8e2815c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:56.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.710+0000 7f8e2c802640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e281e6310 0x7f8e281e6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:56.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.710+0000 7f8e2c802640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2815d030 con 0x7f8e281e6310 2023-09-23T03:25:56.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.710+0000 7f8e2c802640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2815ce30 con 0x7f8e140047c0 2023-09-23T03:25:56.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.710+0000 7f8e2c802640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2814f370 con 0x7f8e2815c470 2023-09-23T03:25:56.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.710+0000 7f8e27fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e281e6310 0x7f8e281e6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:56.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.710+0000 7f8e27fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e281e6310 0x7f8e281e6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50784/0 (socket says 172.21.15.154:50784) 2023-09-23T03:25:56.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.710+0000 7f8e27fff640 1 -- 172.21.15.154:0/1459979534 learned_addr learned my addr 172.21.15.154:0/1459979534 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:56.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.710+0000 7f8e277fe640 1 --2- 172.21.15.154:0/1459979534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e140047c0 0x7f8e2815b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:56.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.710+0000 7f8e27fff640 1 -- 172.21.15.154:0/1459979534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e2815c470 msgr2=0x7f8e2815c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:56.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.710+0000 7f8e27fff640 1 --2- 172.21.15.154:0/1459979534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e2815c470 0x7f8e2815c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:56.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.710+0000 7f8e27fff640 1 -- 172.21.15.154:0/1459979534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e140047c0 msgr2=0x7f8e2815b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:56.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.710+0000 7f8e27fff640 1 --2- 172.21.15.154:0/1459979534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e140047c0 0x7f8e2815b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:56.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.710+0000 7f8e27fff640 1 -- 172.21.15.154:0/1459979534 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e281f5f00 con 0x7f8e281e6310 2023-09-23T03:25:56.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.711+0000 7f8e27fff640 1 --2- 172.21.15.154:0/1459979534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e281e6310 0x7f8e281e6790 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f8e2808bef0 tx=0x7f8e2000d820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:56.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.711+0000 7f8e1ffff640 1 -- 172.21.15.154:0/1459979534 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e20004030 con 0x7f8e281e6310 2023-09-23T03:25:56.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.711+0000 7f8e1ffff640 1 -- 172.21.15.154:0/1459979534 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e200041b0 con 0x7f8e281e6310 2023-09-23T03:25:56.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.711+0000 7f8e1ffff640 1 -- 172.21.15.154:0/1459979534 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e20003610 con 0x7f8e281e6310 2023-09-23T03:25:56.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.711+0000 7f8e2c802640 1 -- 172.21.15.154:0/1459979534 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e281f55d0 con 0x7f8e281e6310 2023-09-23T03:25:56.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.711+0000 7f8e2c802640 1 -- 172.21.15.154:0/1459979534 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e281f5b90 con 0x7f8e281e6310 2023-09-23T03:25:56.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.712+0000 7f8e2c802640 1 -- 172.21.15.154:0/1459979534 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e281f6370 con 0x7f8e281e6310 2023-09-23T03:25:56.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.713+0000 7f8e1ffff640 1 -- 172.21.15.154:0/1459979534 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f8e20003770 con 0x7f8e281e6310 2023-09-23T03:25:56.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.715+0000 7f8e1ffff640 1 --2- 172.21.15.154:0/1459979534 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8e0806af30 0x7f8e0806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:56.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.715+0000 7f8e1ffff640 1 -- 172.21.15.154:0/1459979534 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8e20089450 con 0x7f8e281e6310 2023-09-23T03:25:56.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.715+0000 7f8e277fe640 1 --2- 172.21.15.154:0/1459979534 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8e0806af30 0x7f8e0806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:56.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.716+0000 7f8e277fe640 1 --2- 172.21.15.154:0/1459979534 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8e0806af30 0x7f8e0806d3f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8e281e70c0 tx=0x7f8e1800b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:56.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.720+0000 7f8e1ffff640 1 -- 172.21.15.154:0/1459979534 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f8e281f6370 con 0x7f8e281e6310 2023-09-23T03:25:56.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:56.911+0000 7f8e2c802640 1 -- 172.21.15.154:0/1459979534 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_extra", "pg_num_min": 64} v 0) v1 -- 0x7f8e281f6370 con 0x7f8e281e6310 2023-09-23T03:25:57.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.390+0000 7f8e1ffff640 1 -- 172.21.15.154:0/1459979534 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_extra", "pg_num_min": 64}]=0 pool 'cephfs_data_extra' created v106) v1 ==== 142+0+0 (secure 0 0 0) 0x7f8e281f6370 con 0x7f8e281e6310 2023-09-23T03:25:57.392 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_data_extra' created 2023-09-23T03:25:57.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.395+0000 7f8e2c802640 1 -- 172.21.15.154:0/1459979534 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8e0806af30 msgr2=0x7f8e0806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:57.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.395+0000 7f8e2c802640 1 --2- 172.21.15.154:0/1459979534 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8e0806af30 0x7f8e0806d3f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8e281e70c0 tx=0x7f8e1800b040 comp rx=0 tx=0).stop 2023-09-23T03:25:57.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.395+0000 7f8e2c802640 1 -- 172.21.15.154:0/1459979534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e281e6310 msgr2=0x7f8e281e6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:57.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.395+0000 7f8e2c802640 1 --2- 172.21.15.154:0/1459979534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e281e6310 0x7f8e281e6790 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f8e2808bef0 tx=0x7f8e2000d820 comp rx=0 tx=0).stop 2023-09-23T03:25:57.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.396+0000 7f8e2c802640 1 -- 172.21.15.154:0/1459979534 shutdown_connections 2023-09-23T03:25:57.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.396+0000 7f8e2c802640 1 --2- 172.21.15.154:0/1459979534 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8e0806af30 0x7f8e0806d3f0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:57.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.396+0000 7f8e2c802640 1 --2- 172.21.15.154:0/1459979534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8e281e6310 0x7f8e281e6790 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:57.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.396+0000 7f8e2c802640 1 --2- 172.21.15.154:0/1459979534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8e2815c470 0x7f8e2815c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:57.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.396+0000 7f8e2c802640 1 --2- 172.21.15.154:0/1459979534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8e140047c0 0x7f8e2815b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:57.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.396+0000 7f8e2c802640 1 -- 172.21.15.154:0/1459979534 >> 172.21.15.154:0/1459979534 conn(0x7f8e2809b9e0 msgr2=0x7f8e280d2530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:57.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.397+0000 7f8e2c802640 1 -- 172.21.15.154:0/1459979534 shutdown_connections 2023-09-23T03:25:57.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.397+0000 7f8e2c802640 1 -- 172.21.15.154:0/1459979534 wait complete. 2023-09-23T03:25:57.426 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_extra 2023-09-23T03:25:57.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.617+0000 7f489a064640 1 Processor -- start 2023-09-23T03:25:57.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.617+0000 7f489a064640 1 -- start start 2023-09-23T03:25:57.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.618+0000 7f489a064640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f489415aba0 0x7f489415bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:57.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.618+0000 7f489a064640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f48940cb5b0 0x7f489415ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:57.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.618+0000 7f489a064640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f489415b6a0 con 0x7f489415e5f0 2023-09-23T03:25:57.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.618+0000 7f489a064640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f489415f0b0 con 0x7f48940cb5b0 2023-09-23T03:25:57.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.618+0000 7f489a064640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f489415f390 con 0x7f489415aba0 2023-09-23T03:25:57.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.619+0000 7f4893fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f48940cb5b0 0x7f489415ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:57.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.619+0000 7f4893fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f48940cb5b0 0x7f489415ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50790/0 (socket says 172.21.15.154:50790) 2023-09-23T03:25:57.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.619+0000 7f4893fff640 1 -- 172.21.15.154:0/70840030 learned_addr learned my addr 172.21.15.154:0/70840030 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:57.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.619+0000 7f488bfff640 1 --1- 172.21.15.154:0/70840030 >> v1:172.21.15.154:6789/0 conn(0x7f489415e5f0 0x7f48940caea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:60394/0 (socket says 172.21.15.154:60394) 2023-09-23T03:25:57.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.619+0000 7f48937fe640 1 --2- 172.21.15.154:0/70840030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f489415aba0 0x7f489415bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:57.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.619+0000 7f4893fff640 1 -- 172.21.15.154:0/70840030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f489415aba0 msgr2=0x7f489415bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:57.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.619+0000 7f4893fff640 1 --2- 172.21.15.154:0/70840030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f489415aba0 0x7f489415bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:57.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.620+0000 7f4893fff640 1 -- 172.21.15.154:0/70840030 >> v1:172.21.15.154:6789/0 conn(0x7f489415e5f0 legacy=0x7f48940caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:57.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.620+0000 7f4892ffd640 1 -- 172.21.15.154:0/70840030 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4081334830 0 0) 0x7f489415b6a0 con 0x7f489415e5f0 2023-09-23T03:25:57.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.620+0000 7f4893fff640 1 -- 172.21.15.154:0/70840030 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f489415b330 con 0x7f48940cb5b0 2023-09-23T03:25:57.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.621+0000 7f4893fff640 1 --2- 172.21.15.154:0/70840030 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48940cb5b0 0x7f489415ea30 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f488c009e50 tx=0x7f488c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b663b6873f3ce1f2 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:57.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.621+0000 7f4892ffd640 1 -- 172.21.15.154:0/70840030 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c02cb00 con 0x7f48940cb5b0 2023-09-23T03:25:57.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.621+0000 7f4892ffd640 1 -- 172.21.15.154:0/70840030 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488c02cc60 con 0x7f48940cb5b0 2023-09-23T03:25:57.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.621+0000 7f4892ffd640 1 -- 172.21.15.154:0/70840030 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c0343f0 con 0x7f48940cb5b0 2023-09-23T03:25:57.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.622+0000 7f489a064640 1 -- 172.21.15.154:0/70840030 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48940cb5b0 msgr2=0x7f489415ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:57.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.622+0000 7f489a064640 1 --2- 172.21.15.154:0/70840030 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48940cb5b0 0x7f489415ea30 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f488c009e50 tx=0x7f488c02bf60 comp rx=0 tx=0).stop 2023-09-23T03:25:57.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.622+0000 7f489a064640 1 -- 172.21.15.154:0/70840030 shutdown_connections 2023-09-23T03:25:57.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.622+0000 7f489a064640 1 --2- 172.21.15.154:0/70840030 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48940cb5b0 0x7f489415ea30 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:57.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.622+0000 7f489a064640 1 --2- 172.21.15.154:0/70840030 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f489415aba0 0x7f489415bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:57.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.622+0000 7f489a064640 1 -- 172.21.15.154:0/70840030 >> 172.21.15.154:0/70840030 conn(0x7f489406c010 msgr2=0x7f489406c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:57.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.622+0000 7f489a064640 1 -- 172.21.15.154:0/70840030 shutdown_connections 2023-09-23T03:25:57.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.622+0000 7f489a064640 1 -- 172.21.15.154:0/70840030 wait complete. 2023-09-23T03:25:57.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.624+0000 7f489a064640 1 Processor -- start 2023-09-23T03:25:57.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.624+0000 7f489a064640 1 -- start start 2023-09-23T03:25:57.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.625+0000 7f489a064640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48940cb5b0 0x7f48940b3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:57.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.625+0000 7f489a064640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f489415aba0 0x7f4894157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:57.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.625+0000 7f489a064640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f489415e5f0 0x7f48940a7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:57.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.625+0000 7f489a064640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48940b3b60 con 0x7f48940cb5b0 2023-09-23T03:25:57.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.625+0000 7f489a064640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4894157d60 con 0x7f489415aba0 2023-09-23T03:25:57.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.625+0000 7f489a064640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48940b2580 con 0x7f489415e5f0 2023-09-23T03:25:57.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.625+0000 7f48937fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48940cb5b0 0x7f48940b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:57.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.625+0000 7f48937fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48940cb5b0 0x7f48940b3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50804/0 (socket says 172.21.15.154:50804) 2023-09-23T03:25:57.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.625+0000 7f48937fe640 1 -- 172.21.15.154:0/4014906725 learned_addr learned my addr 172.21.15.154:0/4014906725 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:57.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.625+0000 7f4893fff640 1 --2- 172.21.15.154:0/4014906725 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f489415e5f0 0x7f48940a7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:57.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.625+0000 7f48937fe640 1 -- 172.21.15.154:0/4014906725 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f489415e5f0 msgr2=0x7f48940a7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:57.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.625+0000 7f48937fe640 1 --2- 172.21.15.154:0/4014906725 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f489415e5f0 0x7f48940a7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:57.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.625+0000 7f48937fe640 1 -- 172.21.15.154:0/4014906725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f489415aba0 msgr2=0x7f4894157820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:57.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.625+0000 7f488bfff640 1 --2- 172.21.15.154:0/4014906725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f489415aba0 0x7f4894157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:57.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.625+0000 7f48937fe640 1 --2- 172.21.15.154:0/4014906725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f489415aba0 0x7f4894157820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:57.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.625+0000 7f48937fe640 1 -- 172.21.15.154:0/4014906725 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f488c009b30 con 0x7f48940cb5b0 2023-09-23T03:25:57.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.626+0000 7f48937fe640 1 --2- 172.21.15.154:0/4014906725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48940cb5b0 0x7f48940b3620 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f488400d830 tx=0x7f488400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:57.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.626+0000 7f4890ff9640 1 -- 172.21.15.154:0/4014906725 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488400b6c0 con 0x7f48940cb5b0 2023-09-23T03:25:57.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.626+0000 7f4890ff9640 1 -- 172.21.15.154:0/4014906725 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488400b840 con 0x7f48940cb5b0 2023-09-23T03:25:57.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.626+0000 7f4890ff9640 1 -- 172.21.15.154:0/4014906725 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488400bbf0 con 0x7f48940cb5b0 2023-09-23T03:25:57.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.626+0000 7f489a064640 1 -- 172.21.15.154:0/4014906725 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f489415b100 con 0x7f48940cb5b0 2023-09-23T03:25:57.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.626+0000 7f489a064640 1 -- 172.21.15.154:0/4014906725 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48940b2b40 con 0x7f48940cb5b0 2023-09-23T03:25:57.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.627+0000 7f489a064640 1 -- 172.21.15.154:0/4014906725 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4894071710 con 0x7f48940cb5b0 2023-09-23T03:25:57.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.628+0000 7f4890ff9640 1 -- 172.21.15.154:0/4014906725 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f488401e460 con 0x7f48940cb5b0 2023-09-23T03:25:57.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.629+0000 7f4890ff9640 1 --2- 172.21.15.154:0/4014906725 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f487406af30 0x7f487406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:57.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.629+0000 7f4890ff9640 1 -- 172.21.15.154:0/4014906725 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6401+0+0 (secure 0 0 0) 0x7f488408a2d0 con 0x7f48940cb5b0 2023-09-23T03:25:57.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.630+0000 7f488bfff640 1 --2- 172.21.15.154:0/4014906725 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f487406af30 0x7f487406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:57.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.631+0000 7f488bfff640 1 --2- 172.21.15.154:0/4014906725 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f487406af30 0x7f487406d3f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f487c007fd0 tx=0x7f487c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:57.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.634+0000 7f4890ff9640 1 -- 172.21.15.154:0/4014906725 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f4884054fb0 con 0x7f48940cb5b0 2023-09-23T03:25:57.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:57.864+0000 7f489a064640 1 -- 172.21.15.154:0/4014906725 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_extra"} v 0) v1 -- 0x7f489404e920 con 0x7f48940cb5b0 2023-09-23T03:25:58.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.423+0000 7f4890ff9640 1 -- 172.21.15.154:0/4014906725 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_extra"}]=0 added data pool 16 to fsmap v85) v1 ==== 141+0+0 (secure 0 0 0) 0x7f48840585d0 con 0x7f48940cb5b0 2023-09-23T03:25:58.427 INFO:teuthology.orchestra.run.smithi154.stderr:added data pool 16 to fsmap 2023-09-23T03:25:58.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.430+0000 7f489a064640 1 -- 172.21.15.154:0/4014906725 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f487406af30 msgr2=0x7f487406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:58.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.430+0000 7f489a064640 1 --2- 172.21.15.154:0/4014906725 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f487406af30 0x7f487406d3f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f487c007fd0 tx=0x7f487c009040 comp rx=0 tx=0).stop 2023-09-23T03:25:58.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.430+0000 7f489a064640 1 -- 172.21.15.154:0/4014906725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48940cb5b0 msgr2=0x7f48940b3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:58.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.430+0000 7f489a064640 1 --2- 172.21.15.154:0/4014906725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48940cb5b0 0x7f48940b3620 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f488400d830 tx=0x7f488400dcf0 comp rx=0 tx=0).stop 2023-09-23T03:25:58.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.430+0000 7f489a064640 1 -- 172.21.15.154:0/4014906725 shutdown_connections 2023-09-23T03:25:58.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.430+0000 7f489a064640 1 --2- 172.21.15.154:0/4014906725 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f487406af30 0x7f487406d3f0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:58.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.430+0000 7f489a064640 1 --2- 172.21.15.154:0/4014906725 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f489415e5f0 0x7f48940a7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:58.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.430+0000 7f489a064640 1 --2- 172.21.15.154:0/4014906725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f489415aba0 0x7f4894157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:58.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.430+0000 7f489a064640 1 --2- 172.21.15.154:0/4014906725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48940cb5b0 0x7f48940b3620 secure :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f488400d830 tx=0x7f488400dcf0 comp rx=0 tx=0).stop 2023-09-23T03:25:58.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.430+0000 7f489a064640 1 -- 172.21.15.154:0/4014906725 >> 172.21.15.154:0/4014906725 conn(0x7f489406c010 msgr2=0x7f48940701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:58.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.431+0000 7f489a064640 1 -- 172.21.15.154:0/4014906725 shutdown_connections 2023-09-23T03:25:58.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.431+0000 7f489a064640 1 -- 172.21.15.154:0/4014906725 wait complete. 2023-09-23T03:25:58.452 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:25:58.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.639+0000 7f152380e640 1 Processor -- start 2023-09-23T03:25:58.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.639+0000 7f152380e640 1 -- start start 2023-09-23T03:25:58.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.640+0000 7f152380e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f151c0acce0 0x7f151c15ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:58.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.640+0000 7f152380e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f151c0cf040 0x7f151c0ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:58.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.640+0000 7f152380e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f151c15ac30 con 0x7f151c0b1b20 2023-09-23T03:25:58.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.640+0000 7f152380e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f151c0b2e90 con 0x7f151c0cf040 2023-09-23T03:25:58.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.640+0000 7f152380e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f151c0b1f70 con 0x7f151c0acce0 2023-09-23T03:25:58.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.641+0000 7f15215aa640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f151c0acce0 0x7f151c15ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:58.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.641+0000 7f1521dab640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f151c0cf040 0x7f151c0ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:58.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.641+0000 7f15215aa640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f151c0acce0 0x7f151c15ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:38378/0 (socket says 172.21.15.154:38378) 2023-09-23T03:25:58.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.641+0000 7f15215aa640 1 -- 172.21.15.154:0/1930362687 learned_addr learned my addr 172.21.15.154:0/1930362687 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:58.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.642+0000 7f1521dab640 1 -- 172.21.15.154:0/1930362687 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f151c0acce0 msgr2=0x7f151c15ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:58.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.642+0000 7f1521dab640 1 --2- 172.21.15.154:0/1930362687 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f151c0acce0 0x7f151c15ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:58.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.642+0000 7f1521dab640 1 -- 172.21.15.154:0/1930362687 >> v1:172.21.15.190:6789/0 conn(0x7f151c0b1b20 legacy=0x7f151c0ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:58.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.642+0000 7f1521dab640 1 -- 172.21.15.154:0/1930362687 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f151c15f2a0 con 0x7f151c0cf040 2023-09-23T03:25:58.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.644+0000 7f1521dab640 1 --2- 172.21.15.154:0/1930362687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f151c0cf040 0x7f151c0ab140 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1514009d90 tx=0x7f151402be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1914ade9d4f95f05 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:58.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.644+0000 7f1513fff640 1 -- 172.21.15.154:0/1930362687 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151402cbd0 con 0x7f151c0cf040 2023-09-23T03:25:58.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.644+0000 7f1513fff640 1 -- 172.21.15.154:0/1930362687 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f151402cd30 con 0x7f151c0cf040 2023-09-23T03:25:58.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.644+0000 7f1513fff640 1 -- 172.21.15.154:0/1930362687 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151402cea0 con 0x7f151c0cf040 2023-09-23T03:25:58.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.645+0000 7f152380e640 1 -- 172.21.15.154:0/1930362687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f151c0cf040 msgr2=0x7f151c0ab140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:58.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.645+0000 7f152380e640 1 --2- 172.21.15.154:0/1930362687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f151c0cf040 0x7f151c0ab140 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1514009d90 tx=0x7f151402be20 comp rx=0 tx=0).stop 2023-09-23T03:25:58.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.645+0000 7f152380e640 1 -- 172.21.15.154:0/1930362687 shutdown_connections 2023-09-23T03:25:58.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.645+0000 7f152380e640 1 --2- 172.21.15.154:0/1930362687 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f151c0cf040 0x7f151c0ab140 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:58.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.645+0000 7f152380e640 1 --2- 172.21.15.154:0/1930362687 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f151c0acce0 0x7f151c15ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:58.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.645+0000 7f152380e640 1 -- 172.21.15.154:0/1930362687 >> 172.21.15.154:0/1930362687 conn(0x7f151c06a0f0 msgr2=0x7f151c06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:58.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.645+0000 7f152380e640 1 -- 172.21.15.154:0/1930362687 shutdown_connections 2023-09-23T03:25:58.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.645+0000 7f152380e640 1 -- 172.21.15.154:0/1930362687 wait complete. 2023-09-23T03:25:58.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.647+0000 7f152380e640 1 Processor -- start 2023-09-23T03:25:58.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.647+0000 7f152380e640 1 -- start start 2023-09-23T03:25:58.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.648+0000 7f152380e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f151c0acce0 0x7f151c0cda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:58.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.649+0000 7f152380e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f151c0b1b20 0x7f151c0c3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:58.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.649+0000 7f152380e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f151c0cf040 0x7f151c0c1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:58.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.649+0000 7f152380e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151c0c24e0 con 0x7f151c0b1b20 2023-09-23T03:25:58.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.649+0000 7f152380e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151c0c2760 con 0x7f151c0acce0 2023-09-23T03:25:58.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.649+0000 7f152380e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f151c0c2a40 con 0x7f151c0cf040 2023-09-23T03:25:58.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.649+0000 7f1520da9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f151c0b1b20 0x7f151c0c3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:58.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.649+0000 7f1520da9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f151c0b1b20 0x7f151c0c3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50820/0 (socket says 172.21.15.154:50820) 2023-09-23T03:25:58.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.649+0000 7f1520da9640 1 -- 172.21.15.154:0/3762920782 learned_addr learned my addr 172.21.15.154:0/3762920782 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:58.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.649+0000 7f15215aa640 1 --2- 172.21.15.154:0/3762920782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f151c0acce0 0x7f151c0cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:58.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.649+0000 7f1521dab640 1 --2- 172.21.15.154:0/3762920782 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f151c0cf040 0x7f151c0c1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:58.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.649+0000 7f1520da9640 1 -- 172.21.15.154:0/3762920782 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f151c0cf040 msgr2=0x7f151c0c1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:58.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.649+0000 7f1520da9640 1 --2- 172.21.15.154:0/3762920782 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f151c0cf040 0x7f151c0c1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:58.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.649+0000 7f1520da9640 1 -- 172.21.15.154:0/3762920782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f151c0acce0 msgr2=0x7f151c0cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:58.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.649+0000 7f1520da9640 1 --2- 172.21.15.154:0/3762920782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f151c0acce0 0x7f151c0cda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:58.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.650+0000 7f1520da9640 1 -- 172.21.15.154:0/3762920782 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1514009a70 con 0x7f151c0b1b20 2023-09-23T03:25:58.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.650+0000 7f15215aa640 1 --2- 172.21.15.154:0/3762920782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f151c0acce0 0x7f151c0cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:25:58.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.650+0000 7f1520da9640 1 --2- 172.21.15.154:0/3762920782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f151c0b1b20 0x7f151c0c3800 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f150c00c400 tx=0x7f150c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:58.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.650+0000 7f1511ffb640 1 -- 172.21.15.154:0/3762920782 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150c0052d0 con 0x7f151c0b1b20 2023-09-23T03:25:58.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.650+0000 7f1511ffb640 1 -- 172.21.15.154:0/3762920782 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f150c005430 con 0x7f151c0b1b20 2023-09-23T03:25:58.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.650+0000 7f1511ffb640 1 -- 172.21.15.154:0/3762920782 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150c003830 con 0x7f151c0b1b20 2023-09-23T03:25:58.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.650+0000 7f152380e640 1 -- 172.21.15.154:0/3762920782 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f151c0c2d20 con 0x7f151c0b1b20 2023-09-23T03:25:58.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.650+0000 7f152380e640 1 -- 172.21.15.154:0/3762920782 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f151c0c32e0 con 0x7f151c0b1b20 2023-09-23T03:25:58.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.651+0000 7f152380e640 1 -- 172.21.15.154:0/3762920782 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f151c0c3500 con 0x7f151c0b1b20 2023-09-23T03:25:58.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.652+0000 7f1511ffb640 1 -- 172.21.15.154:0/3762920782 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f150c003990 con 0x7f151c0b1b20 2023-09-23T03:25:58.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.653+0000 7f1511ffb640 1 --2- 172.21.15.154:0/3762920782 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f150006af30 0x7f150006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:58.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.653+0000 7f1511ffb640 1 -- 172.21.15.154:0/3762920782 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f150c088fd0 con 0x7f151c0b1b20 2023-09-23T03:25:58.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.654+0000 7f15215aa640 1 --2- 172.21.15.154:0/3762920782 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f150006af30 0x7f150006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:58.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.654+0000 7f15215aa640 1 --2- 172.21.15.154:0/3762920782 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f150006af30 0x7f150006d3f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f1518009b80 tx=0x7f1518008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:58.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.658+0000 7f1511ffb640 1 -- 172.21.15.154:0/3762920782 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f150c053c00 con 0x7f151c0b1b20 2023-09-23T03:25:58.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.885+0000 7f152380e640 1 -- 172.21.15.154:0/3762920782 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f151c15abd0 con 0x7f151c0b1b20 2023-09-23T03:25:58.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.885+0000 7f1511ffb640 1 -- 172.21.15.154:0/3762920782 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4930 (secure 0 0 0) 0x7f150c017020 con 0x7f151c0b1b20 2023-09-23T03:25:58.887 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:25:58.887 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":85,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5603,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/4291187069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":4291187069},{"type":"v1","addr":"172.21.15.190:6834","nonce":4291187069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":8097,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/1240348396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1240348396},{"type":"v1","addr":"172.21.15.154:6836","nonce":1240348396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":8100,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/4233181417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":4233181417},{"type":"v1","addr":"172.21.15.154:6837","nonce":4233181417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:25:46.997893+0000","modified":"2023-09-23T03:25:57.867830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8115":{"gid":8115,"name":"d","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/2967125458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2967125458},{"type":"v1","addr":"172.21.15.190:6835","nonce":2967125458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15,16],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-09-23T03:25:58.887 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 85 2023-09-23T03:25:58.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.888+0000 7f152380e640 1 -- 172.21.15.154:0/3762920782 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f150006af30 msgr2=0x7f150006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:58.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.888+0000 7f152380e640 1 --2- 172.21.15.154:0/3762920782 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f150006af30 0x7f150006d3f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f1518009b80 tx=0x7f1518008040 comp rx=0 tx=0).stop 2023-09-23T03:25:58.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.888+0000 7f152380e640 1 -- 172.21.15.154:0/3762920782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f151c0b1b20 msgr2=0x7f151c0c3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:58.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.888+0000 7f152380e640 1 --2- 172.21.15.154:0/3762920782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f151c0b1b20 0x7f151c0c3800 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f150c00c400 tx=0x7f150c00c8c0 comp rx=0 tx=0).stop 2023-09-23T03:25:58.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.888+0000 7f152380e640 1 -- 172.21.15.154:0/3762920782 shutdown_connections 2023-09-23T03:25:58.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.888+0000 7f152380e640 1 --2- 172.21.15.154:0/3762920782 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f150006af30 0x7f150006d3f0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:58.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.888+0000 7f152380e640 1 --2- 172.21.15.154:0/3762920782 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f151c0cf040 0x7f151c0c1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:58.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.888+0000 7f152380e640 1 --2- 172.21.15.154:0/3762920782 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f151c0b1b20 0x7f151c0c3800 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:58.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.888+0000 7f152380e640 1 --2- 172.21.15.154:0/3762920782 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f151c0acce0 0x7f151c0cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:58.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.888+0000 7f152380e640 1 -- 172.21.15.154:0/3762920782 >> 172.21.15.154:0/3762920782 conn(0x7f151c06a0f0 msgr2=0x7f151c0b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:58.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.888+0000 7f152380e640 1 -- 172.21.15.154:0/3762920782 shutdown_connections 2023-09-23T03:25:58.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:58.888+0000 7f152380e640 1 -- 172.21.15.154:0/3762920782 wait complete. 2023-09-23T03:25:58.908 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:25:59.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.122+0000 7fd832c6f640 1 Processor -- start 2023-09-23T03:25:59.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.123+0000 7fd832c6f640 1 -- start start 2023-09-23T03:25:59.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.124+0000 7fd832c6f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd82c160680 0x7fd82c0b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:59.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.124+0000 7fd832c6f640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd82c0d25b0 0x7fd82c0d2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:59.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.124+0000 7fd832c6f640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd82c161500 con 0x7fd82c15ca80 2023-09-23T03:25:59.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.124+0000 7fd832c6f640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd82c160c60 con 0x7fd82c160680 2023-09-23T03:25:59.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.124+0000 7fd832c6f640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd82c160f40 con 0x7fd82c0d25b0 2023-09-23T03:25:59.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.125+0000 7fd82bfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd82c160680 0x7fd82c0b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:59.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.125+0000 7fd82bfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd82c160680 0x7fd82c0b2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50832/0 (socket says 172.21.15.154:50832) 2023-09-23T03:25:59.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.125+0000 7fd82bfff640 1 -- 172.21.15.154:0/2955892188 learned_addr learned my addr 172.21.15.154:0/2955892188 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:59.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.125+0000 7fd83120c640 1 --2- 172.21.15.154:0/2955892188 >> v2:172.21.15.190:3300/0 conn(0x7fd82c0d25b0 0x7fd82c0d2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:59.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.125+0000 7fd82bfff640 1 -- 172.21.15.154:0/2955892188 >> v2:172.21.15.190:3300/0 conn(0x7fd82c0d25b0 msgr2=0x7fd82c0d2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:59.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.125+0000 7fd82bfff640 1 --2- 172.21.15.154:0/2955892188 >> v2:172.21.15.190:3300/0 conn(0x7fd82c0d25b0 0x7fd82c0d2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:59.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.125+0000 7fd82bfff640 1 -- 172.21.15.154:0/2955892188 >> v1:172.21.15.190:6789/0 conn(0x7fd82c15ca80 legacy=0x7fd82c0d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:25:59.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.125+0000 7fd82bfff640 1 -- 172.21.15.154:0/2955892188 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd82c161220 con 0x7fd82c160680 2023-09-23T03:25:59.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.125+0000 7fd83120c640 1 --2- 172.21.15.154:0/2955892188 >> v2:172.21.15.190:3300/0 conn(0x7fd82c0d25b0 0x7fd82c0d2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:25:59.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.126+0000 7fd82bfff640 1 --2- 172.21.15.154:0/2955892188 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd82c160680 0x7fd82c0b2620 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fd82c0c9a00 tx=0x7fd81c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e60b30a9b9268b99 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:59.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.126+0000 7fd82b7fe640 1 -- 172.21.15.154:0/2955892188 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81c0317e0 con 0x7fd82c160680 2023-09-23T03:25:59.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.127+0000 7fd82b7fe640 1 -- 172.21.15.154:0/2955892188 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd81c031940 con 0x7fd82c160680 2023-09-23T03:25:59.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.127+0000 7fd82b7fe640 1 -- 172.21.15.154:0/2955892188 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81c031cd0 con 0x7fd82c160680 2023-09-23T03:25:59.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.127+0000 7fd832c6f640 1 -- 172.21.15.154:0/2955892188 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd82c160680 msgr2=0x7fd82c0b2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:59.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.127+0000 7fd832c6f640 1 --2- 172.21.15.154:0/2955892188 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd82c160680 0x7fd82c0b2620 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fd82c0c9a00 tx=0x7fd81c02b880 comp rx=0 tx=0).stop 2023-09-23T03:25:59.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.127+0000 7fd832c6f640 1 -- 172.21.15.154:0/2955892188 shutdown_connections 2023-09-23T03:25:59.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.127+0000 7fd832c6f640 1 --2- 172.21.15.154:0/2955892188 >> v2:172.21.15.190:3300/0 conn(0x7fd82c0d25b0 0x7fd82c0d2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:59.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.127+0000 7fd832c6f640 1 --2- 172.21.15.154:0/2955892188 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd82c160680 0x7fd82c0b2620 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:59.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.127+0000 7fd832c6f640 1 -- 172.21.15.154:0/2955892188 >> 172.21.15.154:0/2955892188 conn(0x7fd82c09bc60 msgr2=0x7fd82c089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:59.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.127+0000 7fd832c6f640 1 -- 172.21.15.154:0/2955892188 shutdown_connections 2023-09-23T03:25:59.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.127+0000 7fd832c6f640 1 -- 172.21.15.154:0/2955892188 wait complete. 2023-09-23T03:25:59.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.129+0000 7fd832c6f640 1 Processor -- start 2023-09-23T03:25:59.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.129+0000 7fd832c6f640 1 -- start start 2023-09-23T03:25:59.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.130+0000 7fd832c6f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd82c0d25b0 0x7fd82c1f1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:59.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.130+0000 7fd832c6f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd82c15ca80 0x7fd82c0b7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:59.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.130+0000 7fd832c6f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd82c160680 0x7fd82c15f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:59.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.130+0000 7fd832c6f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82c0cfe00 con 0x7fd82c160680 2023-09-23T03:25:59.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.130+0000 7fd832c6f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82c0cf560 con 0x7fd82c15ca80 2023-09-23T03:25:59.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.130+0000 7fd832c6f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd82c0cf840 con 0x7fd82c0d25b0 2023-09-23T03:25:59.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.130+0000 7fd83120c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd82c160680 0x7fd82c15f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:59.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.131+0000 7fd83120c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd82c160680 0x7fd82c15f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50836/0 (socket says 172.21.15.154:50836) 2023-09-23T03:25:59.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.131+0000 7fd83120c640 1 -- 172.21.15.154:0/3765487803 learned_addr learned my addr 172.21.15.154:0/3765487803 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:25:59.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.131+0000 7fd82bfff640 1 --2- 172.21.15.154:0/3765487803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd82c15ca80 0x7fd82c0b7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:59.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.131+0000 7fd830a0b640 1 --2- 172.21.15.154:0/3765487803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd82c0d25b0 0x7fd82c1f1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:59.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.131+0000 7fd83120c640 1 -- 172.21.15.154:0/3765487803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd82c0d25b0 msgr2=0x7fd82c1f1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:59.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.131+0000 7fd83120c640 1 --2- 172.21.15.154:0/3765487803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd82c0d25b0 0x7fd82c1f1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:59.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.131+0000 7fd83120c640 1 -- 172.21.15.154:0/3765487803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd82c15ca80 msgr2=0x7fd82c0b7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:59.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.131+0000 7fd83120c640 1 --2- 172.21.15.154:0/3765487803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd82c15ca80 0x7fd82c0b7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:59.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.131+0000 7fd83120c640 1 -- 172.21.15.154:0/3765487803 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd81c0094d0 con 0x7fd82c160680 2023-09-23T03:25:59.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.131+0000 7fd83120c640 1 --2- 172.21.15.154:0/3765487803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd82c160680 0x7fd82c15f730 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fd82400cd50 tx=0x7fd82400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:59.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.132+0000 7fd8297fa640 1 -- 172.21.15.154:0/3765487803 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82400f680 con 0x7fd82c160680 2023-09-23T03:25:59.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.132+0000 7fd8297fa640 1 -- 172.21.15.154:0/3765487803 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82400f800 con 0x7fd82c160680 2023-09-23T03:25:59.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.132+0000 7fd8297fa640 1 -- 172.21.15.154:0/3765487803 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82400fbb0 con 0x7fd82c160680 2023-09-23T03:25:59.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.132+0000 7fd832c6f640 1 -- 172.21.15.154:0/3765487803 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd82c0cfb20 con 0x7fd82c160680 2023-09-23T03:25:59.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.132+0000 7fd832c6f640 1 -- 172.21.15.154:0/3765487803 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd82c0c9c50 con 0x7fd82c160680 2023-09-23T03:25:59.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.133+0000 7fd832c6f640 1 -- 172.21.15.154:0/3765487803 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd82c0d2f10 con 0x7fd82c160680 2023-09-23T03:25:59.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.134+0000 7fd8297fa640 1 -- 172.21.15.154:0/3765487803 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fd82400b040 con 0x7fd82c160680 2023-09-23T03:25:59.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.135+0000 7fd8297fa640 1 --2- 172.21.15.154:0/3765487803 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd80c06af30 0x7fd80c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:25:59.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.135+0000 7fd8297fa640 1 -- 172.21.15.154:0/3765487803 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6433+0+0 (secure 0 0 0) 0x7fd82408a500 con 0x7fd82c160680 2023-09-23T03:25:59.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.135+0000 7fd830a0b640 1 --2- 172.21.15.154:0/3765487803 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd80c06af30 0x7fd80c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:25:59.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.136+0000 7fd830a0b640 1 --2- 172.21.15.154:0/3765487803 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd80c06af30 0x7fd80c06d3f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd81400b7f0 tx=0x7fd814008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:25:59.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.139+0000 7fd8297fa640 1 -- 172.21.15.154:0/3765487803 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fd8240551c0 con 0x7fd82c160680 2023-09-23T03:25:59.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.329+0000 7fd832c6f640 1 -- 172.21.15.154:0/3765487803 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd82c073eb0 con 0x7fd82c160680 2023-09-23T03:25:59.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.331+0000 7fd8297fa640 1 -- 172.21.15.154:0/3765487803 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+18563 (secure 0 0 0) 0x7fd8240587e0 con 0x7fd82c160680 2023-09-23T03:25:59.333 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:25:59.333 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":107,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:25:57.867243+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:25:44.763842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-09-23T03:25:46.025926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data_extra","create_time":"2023-09-23T03:25:56.961772+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:25:59.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.334+0000 7fd832c6f640 1 -- 172.21.15.154:0/3765487803 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd80c06af30 msgr2=0x7fd80c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:59.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.334+0000 7fd832c6f640 1 --2- 172.21.15.154:0/3765487803 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd80c06af30 0x7fd80c06d3f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd81400b7f0 tx=0x7fd814008040 comp rx=0 tx=0).stop 2023-09-23T03:25:59.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.334+0000 7fd832c6f640 1 -- 172.21.15.154:0/3765487803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd82c160680 msgr2=0x7fd82c15f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:25:59.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.334+0000 7fd832c6f640 1 --2- 172.21.15.154:0/3765487803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd82c160680 0x7fd82c15f730 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fd82400cd50 tx=0x7fd82400a650 comp rx=0 tx=0).stop 2023-09-23T03:25:59.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.334+0000 7fd832c6f640 1 -- 172.21.15.154:0/3765487803 shutdown_connections 2023-09-23T03:25:59.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.334+0000 7fd832c6f640 1 --2- 172.21.15.154:0/3765487803 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd80c06af30 0x7fd80c06d3f0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:59.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.334+0000 7fd832c6f640 1 --2- 172.21.15.154:0/3765487803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd82c160680 0x7fd82c15f730 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:59.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.334+0000 7fd832c6f640 1 --2- 172.21.15.154:0/3765487803 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd82c15ca80 0x7fd82c0b7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:59.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.334+0000 7fd832c6f640 1 --2- 172.21.15.154:0/3765487803 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd82c0d25b0 0x7fd82c1f1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:25:59.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.334+0000 7fd832c6f640 1 -- 172.21.15.154:0/3765487803 >> 172.21.15.154:0/3765487803 conn(0x7fd82c09bc60 msgr2=0x7fd82c0b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:25:59.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.334+0000 7fd832c6f640 1 -- 172.21.15.154:0/3765487803 shutdown_connections 2023-09-23T03:25:59.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:25:59.334+0000 7fd832c6f640 1 -- 172.21.15.154:0/3765487803 wait complete. 2023-09-23T03:25:59.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:25:59.356 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir stripey) 2023-09-23T03:25:59.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:25:59.424 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout -v 'stripe_unit=65536 stripe_count=4 object_size=262144 pool=cephfs_data_extra' ./stripey) 2023-09-23T03:25:59.492 INFO:tasks.cephfs.mount:Writing 1048576 bytes to stripey/flushed_file_0 2023-09-23T03:25:59.492 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:25:59.492 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:25:59.493 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_0" 2023-09-23T03:25:59.493 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'w'"'"') as f: 2023-09-23T03:25:59.493 DEBUG:teuthology.orchestra.run.smithi154:> for i in range(0, 1048576): 2023-09-23T03:25:59.493 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:25:59.493 DEBUG:teuthology.orchestra.run.smithi154:> f.write(chr(val)) 2023-09-23T03:25:59.493 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:26:00.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:26:00.280 INFO:tasks.cephfs.mount:Validating 1048576 bytes from stripey/flushed_file_0 2023-09-23T03:26:00.280 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:26:00.280 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:26:00.281 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_0" 2023-09-23T03:26:00.281 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:26:00.281 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:26:00.281 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 1048576: 2023-09-23T03:26:00.281 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:26:00.281 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 1048576 2023-09-23T03:26:00.281 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:26:00.281 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:26:00.281 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:26:00.281 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:26:00.281 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:26:00.281 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:26:01.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:26:01.097 INFO:tasks.cephfs.mount:Writing 524288 bytes to stripey/flushed_file_1 2023-09-23T03:26:01.097 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:26:01.098 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:26:01.098 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_1" 2023-09-23T03:26:01.098 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'w'"'"') as f: 2023-09-23T03:26:01.098 DEBUG:teuthology.orchestra.run.smithi154:> for i in range(0, 524288): 2023-09-23T03:26:01.098 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:26:01.098 DEBUG:teuthology.orchestra.run.smithi154:> f.write(chr(val)) 2023-09-23T03:26:01.098 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:26:01.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:26:01.616 INFO:tasks.cephfs.mount:Validating 524288 bytes from stripey/flushed_file_1 2023-09-23T03:26:01.616 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:26:01.616 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:26:01.616 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_1" 2023-09-23T03:26:01.617 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:26:01.617 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:26:01.617 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 524288: 2023-09-23T03:26:01.617 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:26:01.617 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 524288 2023-09-23T03:26:01.617 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:26:01.617 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:26:01.617 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:26:01.617 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:26:01.617 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:26:01.617 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:26:02.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:26:02.149 INFO:tasks.cephfs.mount:Writing 917504 bytes to stripey/flushed_file_2 2023-09-23T03:26:02.150 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:26:02.150 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:26:02.150 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_2" 2023-09-23T03:26:02.150 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'w'"'"') as f: 2023-09-23T03:26:02.150 DEBUG:teuthology.orchestra.run.smithi154:> for i in range(0, 917504): 2023-09-23T03:26:02.150 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:26:02.150 DEBUG:teuthology.orchestra.run.smithi154:> f.write(chr(val)) 2023-09-23T03:26:02.150 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:26:02.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:26:02.893 INFO:tasks.cephfs.mount:Validating 917504 bytes from stripey/flushed_file_2 2023-09-23T03:26:02.893 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:26:02.894 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:26:02.894 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_2" 2023-09-23T03:26:02.894 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:26:02.894 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:26:02.894 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 917504: 2023-09-23T03:26:02.894 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:26:02.894 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 917504 2023-09-23T03:26:02.894 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:26:02.894 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:26:02.894 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:26:02.894 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:26:02.894 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:26:02.895 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:26:03.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:26:03.641 INFO:tasks.cephfs.mount:Writing 917503 bytes to stripey/flushed_file_3 2023-09-23T03:26:03.641 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:26:03.641 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:26:03.641 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_3" 2023-09-23T03:26:03.641 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'w'"'"') as f: 2023-09-23T03:26:03.642 DEBUG:teuthology.orchestra.run.smithi154:> for i in range(0, 917503): 2023-09-23T03:26:03.642 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:26:03.642 DEBUG:teuthology.orchestra.run.smithi154:> f.write(chr(val)) 2023-09-23T03:26:03.642 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:26:04.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:26:04.378 INFO:tasks.cephfs.mount:Validating 917503 bytes from stripey/flushed_file_3 2023-09-23T03:26:04.378 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:26:04.378 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:26:04.379 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_3" 2023-09-23T03:26:04.379 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:26:04.379 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:26:04.379 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 917503: 2023-09-23T03:26:04.379 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:26:04.379 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 917503 2023-09-23T03:26:04.379 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:26:04.379 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:26:04.379 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:26:04.379 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:26:04.379 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:26:04.379 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:26:05.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:26:05.117 INFO:tasks.cephfs.mount:Writing 1441792 bytes to stripey/flushed_file_4 2023-09-23T03:26:05.117 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:26:05.117 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:26:05.117 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_4" 2023-09-23T03:26:05.117 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'w'"'"') as f: 2023-09-23T03:26:05.117 DEBUG:teuthology.orchestra.run.smithi154:> for i in range(0, 1441792): 2023-09-23T03:26:05.118 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:26:05.118 DEBUG:teuthology.orchestra.run.smithi154:> f.write(chr(val)) 2023-09-23T03:26:05.118 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:26:06.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:26:06.196 INFO:tasks.cephfs.mount:Validating 1441792 bytes from stripey/flushed_file_4 2023-09-23T03:26:06.197 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:26:06.197 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:26:06.197 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_4" 2023-09-23T03:26:06.197 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:26:06.197 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:26:06.197 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 1441792: 2023-09-23T03:26:06.197 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:26:06.197 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 1441792 2023-09-23T03:26:06.197 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:26:06.197 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:26:06.198 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:26:06.198 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:26:06.198 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:26:06.198 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:26:07.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:26:07.214 INFO:tasks.cephfs.mount:Writing 1441793 bytes to stripey/flushed_file_5 2023-09-23T03:26:07.215 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:26:07.215 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:26:07.215 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_5" 2023-09-23T03:26:07.215 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'w'"'"') as f: 2023-09-23T03:26:07.215 DEBUG:teuthology.orchestra.run.smithi154:> for i in range(0, 1441793): 2023-09-23T03:26:07.215 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:26:07.215 DEBUG:teuthology.orchestra.run.smithi154:> f.write(chr(val)) 2023-09-23T03:26:07.215 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:26:08.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:26:08.265 INFO:tasks.cephfs.mount:Validating 1441793 bytes from stripey/flushed_file_5 2023-09-23T03:26:08.265 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:26:08.265 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:26:08.265 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_5" 2023-09-23T03:26:08.265 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:26:08.265 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:26:08.265 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 1441793: 2023-09-23T03:26:08.266 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:26:08.266 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 1441793 2023-09-23T03:26:08.266 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:26:08.266 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:26:08.266 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:26:08.266 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:26:08.266 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:26:08.266 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:26:09.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:26:09.288 INFO:tasks.cephfs.mount:Writing 1572864 bytes to stripey/flushed_file_6 2023-09-23T03:26:09.288 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:26:09.289 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:26:09.289 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_6" 2023-09-23T03:26:09.289 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'w'"'"') as f: 2023-09-23T03:26:09.289 DEBUG:teuthology.orchestra.run.smithi154:> for i in range(0, 1572864): 2023-09-23T03:26:09.289 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:26:09.289 DEBUG:teuthology.orchestra.run.smithi154:> f.write(chr(val)) 2023-09-23T03:26:09.289 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:26:10.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:26:10.396 INFO:tasks.cephfs.mount:Validating 1572864 bytes from stripey/flushed_file_6 2023-09-23T03:26:10.397 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:26:10.397 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:26:10.397 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_6" 2023-09-23T03:26:10.397 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:26:10.397 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:26:10.397 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 1572864: 2023-09-23T03:26:10.397 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:26:10.397 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 1572864 2023-09-23T03:26:10.397 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:26:10.398 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:26:10.398 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:26:10.398 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:26:10.398 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:26:10.398 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:26:11.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:26:11.463 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:26:11.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.721+0000 7f9cc279e640 1 Processor -- start 2023-09-23T03:26:11.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.721+0000 7f9cc279e640 1 -- start start 2023-09-23T03:26:11.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.722+0000 7f9cc279e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9cbc0cf130 0x7f9cbc0ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:11.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.722+0000 7f9cc279e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cbc06ceb0 con 0x7f9cbc0b2fe0 2023-09-23T03:26:11.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.722+0000 7f9cc279e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cbc159c90 con 0x7f9cbc0ada30 2023-09-23T03:26:11.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.722+0000 7f9cc279e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cbc0ade10 con 0x7f9cbc0cf130 2023-09-23T03:26:11.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.723+0000 7f9cc0d3b640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f9cbc0b2fe0 0x7f9cbc15d380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:59410/0 (socket says 172.21.15.154:59410) 2023-09-23T03:26:11.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.723+0000 7f9cc0d3b640 1 -- 172.21.15.154:0/4077687706 learned_addr learned my addr 172.21.15.154:0/4077687706 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:26:11.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.723+0000 7f9cbb7fe640 1 --2- 172.21.15.154:0/4077687706 >> v2:172.21.15.190:3300/0 conn(0x7f9cbc0cf130 0x7f9cbc0ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:11.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.723+0000 7f9cbaffd640 1 -- 172.21.15.154:0/4077687706 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1340693744 0 0) 0x7f9cb4001110 con 0x7f9cbc0b2fe0 2023-09-23T03:26:11.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.723+0000 7f9cbaffd640 1 -- 172.21.15.154:0/4077687706 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ca8003520 con 0x7f9cbc0b2fe0 2023-09-23T03:26:11.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.724+0000 7f9cbaffd640 1 -- 172.21.15.154:0/4077687706 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3493976103 0 0) 0x7f9ca8003520 con 0x7f9cbc0b2fe0 2023-09-23T03:26:11.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.724+0000 7f9cbaffd640 1 -- 172.21.15.154:0/4077687706 >> v2:172.21.15.190:3300/0 conn(0x7f9cbc0cf130 msgr2=0x7f9cbc0ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:11.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.724+0000 7f9cbaffd640 1 --2- 172.21.15.154:0/4077687706 >> v2:172.21.15.190:3300/0 conn(0x7f9cbc0cf130 0x7f9cbc0ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:11.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.724+0000 7f9cbaffd640 1 -- 172.21.15.154:0/4077687706 >> v1:172.21.15.190:6789/0 conn(0x7f9cbc0ada30 legacy=0x7f9cbc0cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:11.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.724+0000 7f9cbaffd640 1 -- 172.21.15.154:0/4077687706 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cbc06c110 con 0x7f9cbc0b2fe0 2023-09-23T03:26:11.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.724+0000 7f9cbaffd640 1 -- 172.21.15.154:0/4077687706 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f9cb4001800 con 0x7f9cbc0b2fe0 2023-09-23T03:26:11.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.724+0000 7f9cbaffd640 1 -- 172.21.15.154:0/4077687706 >> v1:172.21.15.154:6789/0 conn(0x7f9cbc0b2fe0 legacy=0x7f9cbc15d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:11.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.724+0000 7f9cbaffd640 1 --2- 172.21.15.154:0/4077687706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ca80039c0 0x7f9ca8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:11.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.724+0000 7f9cbaffd640 1 --2- 172.21.15.154:0/4077687706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ca80047c0 0x7f9ca8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:11.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.724+0000 7f9cbaffd640 1 --2- 172.21.15.154:0/4077687706 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ca8005180 0x7f9ca800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:11.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.724+0000 7f9cbaffd640 1 -- 172.21.15.154:0/4077687706 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cbc0ade10 con 0x7f9ca8005180 2023-09-23T03:26:11.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.724+0000 7f9cbaffd640 1 -- 172.21.15.154:0/4077687706 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca800e290 con 0x7f9ca80039c0 2023-09-23T03:26:11.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.724+0000 7f9cbaffd640 1 -- 172.21.15.154:0/4077687706 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca800e450 con 0x7f9ca80047c0 2023-09-23T03:26:11.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.724+0000 7f9cc0d3b640 1 --2- 172.21.15.154:0/4077687706 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ca8005180 0x7f9ca800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:11.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.724+0000 7f9cbbfff640 1 --2- 172.21.15.154:0/4077687706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ca80039c0 0x7f9ca8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:11.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.724+0000 7f9cbb7fe640 1 --2- 172.21.15.154:0/4077687706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ca80047c0 0x7f9ca8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:11.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.725+0000 7f9cc0d3b640 1 -- 172.21.15.154:0/4077687706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ca80047c0 msgr2=0x7f9ca8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:11.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.725+0000 7f9cc0d3b640 1 --2- 172.21.15.154:0/4077687706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9ca80047c0 0x7f9ca8004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:11.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.725+0000 7f9cc0d3b640 1 -- 172.21.15.154:0/4077687706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ca80039c0 msgr2=0x7f9ca8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:11.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.725+0000 7f9cc0d3b640 1 --2- 172.21.15.154:0/4077687706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ca80039c0 0x7f9ca8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:11.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.725+0000 7f9cc0d3b640 1 -- 172.21.15.154:0/4077687706 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca8003520 con 0x7f9ca8005180 2023-09-23T03:26:11.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.725+0000 7f9cbbfff640 1 --2- 172.21.15.154:0/4077687706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ca80039c0 0x7f9ca8003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:26:11.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.725+0000 7f9cc0d3b640 1 -- 172.21.15.154:0/4077687706 reap_dead start 2023-09-23T03:26:11.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.726+0000 7f9cc0d3b640 1 --2- 172.21.15.154:0/4077687706 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ca8005180 0x7f9ca800db90 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc0b3410 tx=0x7f9ca80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b68ee5b3e71076 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:11.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.726+0000 7f9cbaffd640 1 -- 172.21.15.154:0/4077687706 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4005ad0 con 0x7f9ca8005180 2023-09-23T03:26:11.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.726+0000 7f9cbaffd640 1 -- 172.21.15.154:0/4077687706 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cb4005c70 con 0x7f9ca8005180 2023-09-23T03:26:11.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.726+0000 7f9cbaffd640 1 -- 172.21.15.154:0/4077687706 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb402c7c0 con 0x7f9ca8005180 2023-09-23T03:26:11.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.727+0000 7f9cc279e640 1 -- 172.21.15.154:0/4077687706 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ca8005180 msgr2=0x7f9ca800db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:11.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.727+0000 7f9cc279e640 1 --2- 172.21.15.154:0/4077687706 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ca8005180 0x7f9ca800db90 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc0b3410 tx=0x7f9ca80057f0 comp rx=0 tx=0).stop 2023-09-23T03:26:11.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.727+0000 7f9cc279e640 1 -- 172.21.15.154:0/4077687706 shutdown_connections 2023-09-23T03:26:11.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.727+0000 7f9cc279e640 1 --2- 172.21.15.154:0/4077687706 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9ca8005180 0x7f9ca800db90 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:11.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.727+0000 7f9cc279e640 1 -- 172.21.15.154:0/4077687706 >> 172.21.15.154:0/4077687706 conn(0x7f9cbc06a0f0 msgr2=0x7f9cbc06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:11.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.727+0000 7f9cc279e640 1 -- 172.21.15.154:0/4077687706 shutdown_connections 2023-09-23T03:26:11.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.727+0000 7f9cc279e640 1 -- 172.21.15.154:0/4077687706 wait complete. 2023-09-23T03:26:11.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.728+0000 7f9cc279e640 1 Processor -- start 2023-09-23T03:26:11.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.729+0000 7f9cc279e640 1 -- start start 2023-09-23T03:26:11.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.730+0000 7f9cc279e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ca8005180 0x7f9cbc1588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:11.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.730+0000 7f9cc279e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cbc1f1b80 0x7f9cbc0c3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:11.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.730+0000 7f9cc279e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9cbc0c42c0 0x7f9cbc0c4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:11.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.730+0000 7f9cc279e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cbc15b320 con 0x7f9cbc0c42c0 2023-09-23T03:26:11.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.730+0000 7f9cc279e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cbc0c4c80 con 0x7f9ca8005180 2023-09-23T03:26:11.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.730+0000 7f9cc279e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cbc0c3b40 con 0x7f9cbc1f1b80 2023-09-23T03:26:11.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.730+0000 7f9cc0d3b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9cbc0c42c0 0x7f9cbc0c4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:11.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.730+0000 7f9cc0d3b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9cbc0c42c0 0x7f9cbc0c4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53526/0 (socket says 172.21.15.154:53526) 2023-09-23T03:26:11.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.730+0000 7f9cc0d3b640 1 -- 172.21.15.154:0/1794420283 learned_addr learned my addr 172.21.15.154:0/1794420283 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:11.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.730+0000 7f9cbbfff640 1 --2- 172.21.15.154:0/1794420283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ca8005180 0x7f9cbc1588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:11.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.730+0000 7f9cbb7fe640 1 --2- 172.21.15.154:0/1794420283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cbc1f1b80 0x7f9cbc0c3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:11.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.730+0000 7f9cc0d3b640 1 -- 172.21.15.154:0/1794420283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cbc1f1b80 msgr2=0x7f9cbc0c3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:11.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.730+0000 7f9cc0d3b640 1 --2- 172.21.15.154:0/1794420283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cbc1f1b80 0x7f9cbc0c3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:11.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.730+0000 7f9cc0d3b640 1 -- 172.21.15.154:0/1794420283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ca8005180 msgr2=0x7f9cbc1588c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:11.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.730+0000 7f9cc0d3b640 1 --2- 172.21.15.154:0/1794420283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ca8005180 0x7f9cbc1588c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:11.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.730+0000 7f9cc0d3b640 1 -- 172.21.15.154:0/1794420283 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cbc158f50 con 0x7f9cbc0c42c0 2023-09-23T03:26:11.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.730+0000 7f9cbbfff640 1 --2- 172.21.15.154:0/1794420283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ca8005180 0x7f9cbc1588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:26:11.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.730+0000 7f9cbb7fe640 1 --2- 172.21.15.154:0/1794420283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cbc1f1b80 0x7f9cbc0c3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:26:11.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.731+0000 7f9cc0d3b640 1 --2- 172.21.15.154:0/1794420283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9cbc0c42c0 0x7f9cbc0c4740 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f9cb400c1d0 tx=0x7f9cb4001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:11.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.731+0000 7f9cb8ff9640 1 -- 172.21.15.154:0/1794420283 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb402c9c0 con 0x7f9cbc0c42c0 2023-09-23T03:26:11.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.731+0000 7f9cb8ff9640 1 -- 172.21.15.154:0/1794420283 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cb402cb20 con 0x7f9cbc0c42c0 2023-09-23T03:26:11.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.731+0000 7f9cb8ff9640 1 -- 172.21.15.154:0/1794420283 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4001110 con 0x7f9cbc0c42c0 2023-09-23T03:26:11.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.731+0000 7f9cc279e640 1 -- 172.21.15.154:0/1794420283 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cbc15a9c0 con 0x7f9cbc0c42c0 2023-09-23T03:26:11.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.731+0000 7f9cc279e640 1 -- 172.21.15.154:0/1794420283 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cbc15ac60 con 0x7f9cbc0c42c0 2023-09-23T03:26:11.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.732+0000 7f9cc279e640 1 -- 172.21.15.154:0/1794420283 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cbc06d050 con 0x7f9cbc0c42c0 2023-09-23T03:26:11.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.733+0000 7f9cb8ff9640 1 -- 172.21.15.154:0/1794420283 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f9cb4032070 con 0x7f9cbc0c42c0 2023-09-23T03:26:11.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.734+0000 7f9cb8ff9640 1 --2- 172.21.15.154:0/1794420283 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9c9c06af60 0x7f9c9c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:11.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.734+0000 7f9cb8ff9640 1 -- 172.21.15.154:0/1794420283 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f9cb403c070 con 0x7f9cbc0c42c0 2023-09-23T03:26:11.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.734+0000 7f9cbbfff640 1 --2- 172.21.15.154:0/1794420283 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9c9c06af60 0x7f9c9c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:11.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.735+0000 7f9cbbfff640 1 --2- 172.21.15.154:0/1794420283 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9c9c06af60 0x7f9c9c06d420 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f9ca40016e0 tx=0x7f9ca4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:11.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.739+0000 7f9cb8ff9640 1 -- 172.21.15.154:0/1794420283 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f9cb4075570 con 0x7f9cbc0c42c0 2023-09-23T03:26:11.958 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.957+0000 7f9cc279e640 1 -- 172.21.15.154:0/1794420283 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9cbc06ceb0 con 0x7f9cbc0c42c0 2023-09-23T03:26:11.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.959+0000 7f9cb8ff9640 1 -- 172.21.15.154:0/1794420283 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4930 (secure 0 0 0) 0x7f9cbc06ceb0 con 0x7f9cbc0c42c0 2023-09-23T03:26:11.960 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:11.960 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":85,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5603,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/4291187069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":4291187069},{"type":"v1","addr":"172.21.15.190:6834","nonce":4291187069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":8097,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/1240348396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1240348396},{"type":"v1","addr":"172.21.15.154:6836","nonce":1240348396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":8100,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/4233181417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":4233181417},{"type":"v1","addr":"172.21.15.154:6837","nonce":4233181417}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:25:46.997893+0000","modified":"2023-09-23T03:25:57.867830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8115":{"gid":8115,"name":"d","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/2967125458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":2967125458},{"type":"v1","addr":"172.21.15.190:6835","nonce":2967125458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15,16],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-09-23T03:26:11.960 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 85 2023-09-23T03:26:11.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.961+0000 7f9cc279e640 1 -- 172.21.15.154:0/1794420283 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9c9c06af60 msgr2=0x7f9c9c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:11.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.961+0000 7f9cc279e640 1 --2- 172.21.15.154:0/1794420283 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9c9c06af60 0x7f9c9c06d420 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f9ca40016e0 tx=0x7f9ca4009040 comp rx=0 tx=0).stop 2023-09-23T03:26:11.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.961+0000 7f9cc279e640 1 -- 172.21.15.154:0/1794420283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9cbc0c42c0 msgr2=0x7f9cbc0c4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:11.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.961+0000 7f9cc279e640 1 --2- 172.21.15.154:0/1794420283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9cbc0c42c0 0x7f9cbc0c4740 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f9cb400c1d0 tx=0x7f9cb4001ac0 comp rx=0 tx=0).stop 2023-09-23T03:26:11.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.961+0000 7f9cc279e640 1 -- 172.21.15.154:0/1794420283 shutdown_connections 2023-09-23T03:26:11.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.961+0000 7f9cc279e640 1 --2- 172.21.15.154:0/1794420283 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9c9c06af60 0x7f9c9c06d420 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:11.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.961+0000 7f9cc279e640 1 --2- 172.21.15.154:0/1794420283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9cbc0c42c0 0x7f9cbc0c4740 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:11.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.961+0000 7f9cc279e640 1 --2- 172.21.15.154:0/1794420283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9cbc1f1b80 0x7f9cbc0c3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:11.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.961+0000 7f9cc279e640 1 --2- 172.21.15.154:0/1794420283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9ca8005180 0x7f9cbc1588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:11.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.961+0000 7f9cc279e640 1 -- 172.21.15.154:0/1794420283 >> 172.21.15.154:0/1794420283 conn(0x7f9cbc06a0f0 msgr2=0x7f9cbc0b1670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:11.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.961+0000 7f9cc279e640 1 -- 172.21.15.154:0/1794420283 shutdown_connections 2023-09-23T03:26:11.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:11.961+0000 7f9cc279e640 1 -- 172.21.15.154:0/1794420283 wait complete. 2023-09-23T03:26:11.980 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2023-09-23T03:26:12.248 INFO:teuthology.orchestra.run.smithi190.stdout:{"message":"","return_code":0} 2023-09-23T03:26:12.248 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-09-23T03:26:12.248 INFO:tasks.cephfs.mount:Writing 524288 bytes to stripey/unflushed_file 2023-09-23T03:26:12.248 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:26:12.248 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:26:12.248 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/stripey/unflushed_file" 2023-09-23T03:26:12.249 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'w'"'"') as f: 2023-09-23T03:26:12.249 DEBUG:teuthology.orchestra.run.smithi154:> for i in range(0, 524288): 2023-09-23T03:26:12.249 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:26:12.249 DEBUG:teuthology.orchestra.run.smithi154:> f.write(chr(val)) 2023-09-23T03:26:12.249 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:26:12.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:26:12.738 INFO:tasks.cephfs.mount:Validating 524288 bytes from stripey/unflushed_file 2023-09-23T03:26:12.738 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:26:12.738 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:26:12.738 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/stripey/unflushed_file" 2023-09-23T03:26:12.738 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:26:12.738 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:26:12.739 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 524288: 2023-09-23T03:26:12.739 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:26:12.739 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 524288 2023-09-23T03:26:12.739 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:26:12.739 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:26:12.739 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:26:12.739 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:26:12.739 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:26:12.739 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:26:13.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:26:13.272 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:26:13.272 DEBUG:teuthology.orchestra.run.smithi154:> import os 2023-09-23T03:26:13.272 DEBUG:teuthology.orchestra.run.smithi154:> import stat 2023-09-23T03:26:13.272 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:26:13.272 DEBUG:teuthology.orchestra.run.smithi154:> print(os.stat("/home/ubuntu/cephtest/mnt.0/stripey/unflushed_file").st_ino) 2023-09-23T03:26:13.272 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:26:13.448 INFO:teuthology.orchestra.run.smithi154.stdout:1099511628284 2023-09-23T03:26:13.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:26:13.462 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:26:13.462 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:26:13.562 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:26:13.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.779+0000 7f1f5db78640 1 Processor -- start 2023-09-23T03:26:13.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.779+0000 7f1f5db78640 1 -- start start 2023-09-23T03:26:13.782 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.781+0000 7f1f5db78640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1f580b1b20 0x7f1f580ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:13.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.781+0000 7f1f5db78640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1f580cee70 0x7f1f580ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:13.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.781+0000 7f1f5db78640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f5815ac30 con 0x7f1f580acce0 2023-09-23T03:26:13.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.781+0000 7f1f5db78640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f580b2e90 con 0x7f1f580b1b20 2023-09-23T03:26:13.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.781+0000 7f1f5db78640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f580cf320 con 0x7f1f580cee70 2023-09-23T03:26:13.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.781+0000 7f1f4ffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1f580b1b20 0x7f1f580ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:13.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.781+0000 7f1f4ffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1f580b1b20 0x7f1f580ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53536/0 (socket says 172.21.15.154:53536) 2023-09-23T03:26:13.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.781+0000 7f1f4ffff640 1 -- 172.21.15.154:0/3608417819 learned_addr learned my addr 172.21.15.154:0/3608417819 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:13.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.782+0000 7f1f4ffff640 1 -- 172.21.15.154:0/3608417819 >> v2:172.21.15.190:3300/0 conn(0x7f1f580cee70 msgr2=0x7f1f580ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:13.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.782+0000 7f1f4ffff640 1 --2- 172.21.15.154:0/3608417819 >> v2:172.21.15.190:3300/0 conn(0x7f1f580cee70 0x7f1f580ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:13.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.782+0000 7f1f4ffff640 1 -- 172.21.15.154:0/3608417819 >> v1:172.21.15.154:6789/0 conn(0x7f1f580acce0 legacy=0x7f1f5815ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:13.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.782+0000 7f1f56ffd640 1 -- 172.21.15.154:0/3608417819 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1674025664 0 0) 0x7f1f48001180 con 0x7f1f580acce0 2023-09-23T03:26:13.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.782+0000 7f1f4ffff640 1 -- 172.21.15.154:0/3608417819 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f580b1f70 con 0x7f1f580b1b20 2023-09-23T03:26:13.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.783+0000 7f1f4ffff640 1 --2- 172.21.15.154:0/3608417819 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f580b1b20 0x7f1f580ce930 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f1f580c9430 tx=0x7f1f4002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c8f0de795782c6c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:13.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.783+0000 7f1f56ffd640 1 -- 172.21.15.154:0/3608417819 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f40031870 con 0x7f1f580b1b20 2023-09-23T03:26:13.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.783+0000 7f1f56ffd640 1 -- 172.21.15.154:0/3608417819 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f400319d0 con 0x7f1f580b1b20 2023-09-23T03:26:13.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.783+0000 7f1f56ffd640 1 -- 172.21.15.154:0/3608417819 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f40031d60 con 0x7f1f580b1b20 2023-09-23T03:26:13.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.784+0000 7f1f5db78640 1 -- 172.21.15.154:0/3608417819 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f580b1b20 msgr2=0x7f1f580ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:13.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.784+0000 7f1f5db78640 1 --2- 172.21.15.154:0/3608417819 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f580b1b20 0x7f1f580ce930 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f1f580c9430 tx=0x7f1f4002b910 comp rx=0 tx=0).stop 2023-09-23T03:26:13.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.784+0000 7f1f5db78640 1 -- 172.21.15.154:0/3608417819 shutdown_connections 2023-09-23T03:26:13.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.784+0000 7f1f5db78640 1 --2- 172.21.15.154:0/3608417819 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f580b1b20 0x7f1f580ce930 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:13.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.784+0000 7f1f5db78640 1 --2- 172.21.15.154:0/3608417819 >> v2:172.21.15.190:3300/0 conn(0x7f1f580cee70 0x7f1f580ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:13.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.784+0000 7f1f5db78640 1 -- 172.21.15.154:0/3608417819 >> 172.21.15.154:0/3608417819 conn(0x7f1f5806a0f0 msgr2=0x7f1f5806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:13.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.784+0000 7f1f5db78640 1 -- 172.21.15.154:0/3608417819 shutdown_connections 2023-09-23T03:26:13.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.784+0000 7f1f5db78640 1 -- 172.21.15.154:0/3608417819 wait complete. 2023-09-23T03:26:13.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.786+0000 7f1f5db78640 1 Processor -- start 2023-09-23T03:26:13.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.786+0000 7f1f5db78640 1 -- start start 2023-09-23T03:26:13.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.787+0000 7f1f5db78640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f580acce0 0x7f1f58097990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:13.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.787+0000 7f1f5db78640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f580b1b20 0x7f1f5815d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:13.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.787+0000 7f1f5db78640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f580cee70 0x7f1f5815d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:13.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.787+0000 7f1f5db78640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5815db20 con 0x7f1f580b1b20 2023-09-23T03:26:13.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.787+0000 7f1f5db78640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f58097ed0 con 0x7f1f580cee70 2023-09-23T03:26:13.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.787+0000 7f1f5db78640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f58144ae0 con 0x7f1f580acce0 2023-09-23T03:26:13.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.787+0000 7f1f4ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f580b1b20 0x7f1f5815d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:13.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.787+0000 7f1f4ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f580b1b20 0x7f1f5815d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53540/0 (socket says 172.21.15.154:53540) 2023-09-23T03:26:13.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.787+0000 7f1f4ffff640 1 -- 172.21.15.154:0/1540546374 learned_addr learned my addr 172.21.15.154:0/1540546374 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:13.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.787+0000 7f1f4ffff640 1 -- 172.21.15.154:0/1540546374 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f580acce0 msgr2=0x7f1f58097990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:13.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.787+0000 7f1f4ffff640 1 --2- 172.21.15.154:0/1540546374 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f580acce0 0x7f1f58097990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:13.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.787+0000 7f1f4ffff640 1 -- 172.21.15.154:0/1540546374 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f580cee70 msgr2=0x7f1f5815d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:13.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.787+0000 7f1f4ffff640 1 --2- 172.21.15.154:0/1540546374 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f580cee70 0x7f1f5815d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:13.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.787+0000 7f1f4ffff640 1 -- 172.21.15.154:0/1540546374 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f400094d0 con 0x7f1f580b1b20 2023-09-23T03:26:13.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.788+0000 7f1f4ffff640 1 --2- 172.21.15.154:0/1540546374 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f580b1b20 0x7f1f5815d0a0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f1f40000c00 tx=0x7f1f400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:13.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.788+0000 7f1f54ff9640 1 -- 172.21.15.154:0/1540546374 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4003ad10 con 0x7f1f580b1b20 2023-09-23T03:26:13.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.788+0000 7f1f54ff9640 1 -- 172.21.15.154:0/1540546374 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f4003ae70 con 0x7f1f580b1b20 2023-09-23T03:26:13.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.788+0000 7f1f5db78640 1 -- 172.21.15.154:0/1540546374 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f58144dc0 con 0x7f1f580b1b20 2023-09-23T03:26:13.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.788+0000 7f1f54ff9640 1 -- 172.21.15.154:0/1540546374 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f40044610 con 0x7f1f580b1b20 2023-09-23T03:26:13.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.788+0000 7f1f5db78640 1 -- 172.21.15.154:0/1540546374 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f58145380 con 0x7f1f580b1b20 2023-09-23T03:26:13.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.789+0000 7f1f5db78640 1 -- 172.21.15.154:0/1540546374 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f58073eb0 con 0x7f1f580b1b20 2023-09-23T03:26:13.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.791+0000 7f1f54ff9640 1 -- 172.21.15.154:0/1540546374 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f1f40044770 con 0x7f1f580b1b20 2023-09-23T03:26:13.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.791+0000 7f1f54ff9640 1 --2- 172.21.15.154:0/1540546374 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1f3806af60 0x7f1f3806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:13.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.792+0000 7f1f54ff9640 1 -- 172.21.15.154:0/1540546374 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f1f4002d040 con 0x7f1f580b1b20 2023-09-23T03:26:13.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.792+0000 7f1f577fe640 1 --2- 172.21.15.154:0/1540546374 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1f3806af60 0x7f1f3806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:13.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.793+0000 7f1f577fe640 1 --2- 172.21.15.154:0/1540546374 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1f3806af60 0x7f1f3806d420 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f1f480015e0 tx=0x7f1f48001340 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:13.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.796+0000 7f1f54ff9640 1 -- 172.21.15.154:0/1540546374 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f1f40076ee0 con 0x7f1f580b1b20 2023-09-23T03:26:13.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.988+0000 7f1f5db78640 1 -- 172.21.15.154:0/1540546374 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f5806c8f0 con 0x7f1f580b1b20 2023-09-23T03:26:13.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.989+0000 7f1f54ff9640 1 -- 172.21.15.154:0/1540546374 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+18550 (secure 0 0 0) 0x7f1f4007a500 con 0x7f1f580b1b20 2023-09-23T03:26:13.991 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:13.992 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":108,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:25:59.416373+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:25:44.763842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-09-23T03:25:46.025926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data_extra","create_time":"2023-09-23T03:25:56.961772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:26:13.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.992+0000 7f1f5db78640 1 -- 172.21.15.154:0/1540546374 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1f3806af60 msgr2=0x7f1f3806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:13.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.992+0000 7f1f5db78640 1 --2- 172.21.15.154:0/1540546374 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1f3806af60 0x7f1f3806d420 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f1f480015e0 tx=0x7f1f48001340 comp rx=0 tx=0).stop 2023-09-23T03:26:13.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.992+0000 7f1f5db78640 1 -- 172.21.15.154:0/1540546374 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f580b1b20 msgr2=0x7f1f5815d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:13.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.992+0000 7f1f5db78640 1 --2- 172.21.15.154:0/1540546374 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f580b1b20 0x7f1f5815d0a0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f1f40000c00 tx=0x7f1f400046c0 comp rx=0 tx=0).stop 2023-09-23T03:26:13.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.992+0000 7f1f5db78640 1 -- 172.21.15.154:0/1540546374 shutdown_connections 2023-09-23T03:26:13.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.992+0000 7f1f5db78640 1 --2- 172.21.15.154:0/1540546374 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1f3806af60 0x7f1f3806d420 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:13.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.992+0000 7f1f5db78640 1 --2- 172.21.15.154:0/1540546374 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1f580cee70 0x7f1f5815d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:13.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.992+0000 7f1f5db78640 1 --2- 172.21.15.154:0/1540546374 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1f580b1b20 0x7f1f5815d0a0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:13.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.992+0000 7f1f5db78640 1 --2- 172.21.15.154:0/1540546374 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1f580acce0 0x7f1f58097990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:13.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.992+0000 7f1f5db78640 1 -- 172.21.15.154:0/1540546374 >> 172.21.15.154:0/1540546374 conn(0x7f1f5806a0f0 msgr2=0x7f1f580af7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:13.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.992+0000 7f1f5db78640 1 -- 172.21.15.154:0/1540546374 shutdown_connections 2023-09-23T03:26:13.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:13.993+0000 7f1f5db78640 1 -- 172.21.15.154:0/1540546374 wait complete. 2023-09-23T03:26:14.013 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:25:44.763842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:25:46.025926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-09-23T03:25:56.961772+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T03:26:14.014 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:26:14.042 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:26:14.068 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:26:14.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.223+0000 7f20b9d3c640 1 Processor -- start 2023-09-23T03:26:14.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.223+0000 7f20b9d3c640 1 -- start start 2023-09-23T03:26:14.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.225+0000 7f20b9d3c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f20b40cb5d0 0x7f20b4167ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:14.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.225+0000 7f20b9d3c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20b40712f0 con 0x7f20b4070c60 2023-09-23T03:26:14.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.225+0000 7f20b9d3c640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20b40715d0 con 0x7f20b40ad830 2023-09-23T03:26:14.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.225+0000 7f20b9d3c640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20b40710f0 con 0x7f20b40cb5d0 2023-09-23T03:26:14.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.225+0000 7f20b2ffd640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f20b4070c60 0x7f20b40caec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:44608/0 (socket says 172.21.15.154:44608) 2023-09-23T03:26:14.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.225+0000 7f20b2ffd640 1 -- 172.21.15.154:0/2813219870 learned_addr learned my addr 172.21.15.154:0/2813219870 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:26:14.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.225+0000 7f20b3fff640 1 --2- 172.21.15.154:0/2813219870 >> v2:172.21.15.190:3300/0 conn(0x7f20b40cb5d0 0x7f20b4167ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:14.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.226+0000 7f20b27fc640 1 -- 172.21.15.154:0/2813219870 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 354847503 0 0) 0x7f20b40712f0 con 0x7f20b4070c60 2023-09-23T03:26:14.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.226+0000 7f20b27fc640 1 -- 172.21.15.154:0/2813219870 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f209c003520 con 0x7f20b4070c60 2023-09-23T03:26:14.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.226+0000 7f20b27fc640 1 -- 172.21.15.154:0/2813219870 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 39594465 0 0) 0x7f209c003520 con 0x7f20b4070c60 2023-09-23T03:26:14.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.226+0000 7f20b27fc640 1 -- 172.21.15.154:0/2813219870 >> v2:172.21.15.190:3300/0 conn(0x7f20b40cb5d0 msgr2=0x7f20b4167ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:14.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.226+0000 7f20b27fc640 1 --2- 172.21.15.154:0/2813219870 >> v2:172.21.15.190:3300/0 conn(0x7f20b40cb5d0 0x7f20b4167ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:14.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.226+0000 7f20b27fc640 1 -- 172.21.15.154:0/2813219870 >> v1:172.21.15.190:6789/0 conn(0x7f20b40ad830 legacy=0x7f20b40c9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:14.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.226+0000 7f20b27fc640 1 -- 172.21.15.154:0/2813219870 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b40718b0 con 0x7f20b4070c60 2023-09-23T03:26:14.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.226+0000 7f20b27fc640 1 -- 172.21.15.154:0/2813219870 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f20a8001670 con 0x7f20b4070c60 2023-09-23T03:26:14.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.226+0000 7f20b27fc640 1 -- 172.21.15.154:0/2813219870 >> v1:172.21.15.154:6789/0 conn(0x7f20b4070c60 legacy=0x7f20b40caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:14.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.226+0000 7f20b27fc640 1 --2- 172.21.15.154:0/2813219870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f209c003970 0x7f209c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:14.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.226+0000 7f20b27fc640 1 --2- 172.21.15.154:0/2813219870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f209c0047c0 0x7f209c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:14.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.226+0000 7f20b27fc640 1 --2- 172.21.15.154:0/2813219870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f209c005180 0x7f209c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:14.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.226+0000 7f20b27fc640 1 -- 172.21.15.154:0/2813219870 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b40710f0 con 0x7f209c005180 2023-09-23T03:26:14.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.226+0000 7f20b27fc640 1 -- 172.21.15.154:0/2813219870 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209c00c020 con 0x7f209c003970 2023-09-23T03:26:14.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.226+0000 7f20b27fc640 1 -- 172.21.15.154:0/2813219870 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f209c00c1e0 con 0x7f209c0047c0 2023-09-23T03:26:14.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.226+0000 7f20b3fff640 1 --2- 172.21.15.154:0/2813219870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f209c005180 0x7f209c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:14.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.227+0000 7f20b37fe640 1 --2- 172.21.15.154:0/2813219870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f209c003970 0x7f209c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:14.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.227+0000 7f20b3fff640 1 -- 172.21.15.154:0/2813219870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f209c0047c0 msgr2=0x7f209c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:14.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.227+0000 7f20b3fff640 1 --2- 172.21.15.154:0/2813219870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f209c0047c0 0x7f209c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:14.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.227+0000 7f20b3fff640 1 -- 172.21.15.154:0/2813219870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f209c003970 msgr2=0x7f209c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:14.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.227+0000 7f20b3fff640 1 --2- 172.21.15.154:0/2813219870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f209c003970 0x7f209c003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:14.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.227+0000 7f20b3fff640 1 -- 172.21.15.154:0/2813219870 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f209c003520 con 0x7f209c005180 2023-09-23T03:26:14.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.228+0000 7f20b3fff640 1 -- 172.21.15.154:0/2813219870 reap_dead start 2023-09-23T03:26:14.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.228+0000 7f20b3fff640 1 --2- 172.21.15.154:0/2813219870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f209c005180 0x7f209c00b980 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f20b41617e0 tx=0x7f209c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1088f31a5bfd27c6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:14.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.228+0000 7f20b27fc640 1 -- 172.21.15.154:0/2813219870 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ac02cbf0 con 0x7f209c005180 2023-09-23T03:26:14.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.228+0000 7f20b27fc640 1 -- 172.21.15.154:0/2813219870 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20ac00f040 con 0x7f209c005180 2023-09-23T03:26:14.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.228+0000 7f20b27fc640 1 -- 172.21.15.154:0/2813219870 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ac034440 con 0x7f209c005180 2023-09-23T03:26:14.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.229+0000 7f20b9d3c640 1 -- 172.21.15.154:0/2813219870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f209c005180 msgr2=0x7f209c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:14.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.229+0000 7f20b9d3c640 1 --2- 172.21.15.154:0/2813219870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f209c005180 0x7f209c00b980 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f20b41617e0 tx=0x7f209c0057f0 comp rx=0 tx=0).stop 2023-09-23T03:26:14.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.229+0000 7f20b9d3c640 1 -- 172.21.15.154:0/2813219870 shutdown_connections 2023-09-23T03:26:14.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.229+0000 7f20b9d3c640 1 --2- 172.21.15.154:0/2813219870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f209c005180 0x7f209c00b980 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:14.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.229+0000 7f20b9d3c640 1 -- 172.21.15.154:0/2813219870 >> 172.21.15.154:0/2813219870 conn(0x7f20b406c010 msgr2=0x7f20b406c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:14.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.229+0000 7f20b9d3c640 1 -- 172.21.15.154:0/2813219870 shutdown_connections 2023-09-23T03:26:14.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.229+0000 7f20b9d3c640 1 -- 172.21.15.154:0/2813219870 wait complete. 2023-09-23T03:26:14.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.231+0000 7f20b9d3c640 1 Processor -- start 2023-09-23T03:26:14.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.231+0000 7f20b9d3c640 1 -- start start 2023-09-23T03:26:14.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.232+0000 7f20b9d3c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f209c005180 0x7f20b415b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:14.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.232+0000 7f20b9d3c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20b40c8240 0x7f20b41f5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:14.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.232+0000 7f20b9d3c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20b40c5f70 0x7f20b40c63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:14.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.232+0000 7f20b9d3c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b40c6930 con 0x7f20b40c5f70 2023-09-23T03:26:14.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.232+0000 7f20b9d3c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b415b8a0 con 0x7f209c005180 2023-09-23T03:26:14.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.232+0000 7f20b9d3c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b40b2200 con 0x7f20b40c8240 2023-09-23T03:26:14.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.232+0000 7f20b3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20b40c5f70 0x7f20b40c63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:14.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.232+0000 7f20b3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20b40c5f70 0x7f20b40c63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35352/0 (socket says 172.21.15.154:35352) 2023-09-23T03:26:14.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.232+0000 7f20b3fff640 1 -- 172.21.15.154:0/3695317436 learned_addr learned my addr 172.21.15.154:0/3695317436 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:14.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.232+0000 7f20b37fe640 1 --2- 172.21.15.154:0/3695317436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f209c005180 0x7f20b415b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:14.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.232+0000 7f20b2ffd640 1 --2- 172.21.15.154:0/3695317436 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20b40c8240 0x7f20b41f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:14.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.232+0000 7f20b3fff640 1 -- 172.21.15.154:0/3695317436 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20b40c8240 msgr2=0x7f20b41f5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:14.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.232+0000 7f20b3fff640 1 --2- 172.21.15.154:0/3695317436 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20b40c8240 0x7f20b41f5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:14.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.232+0000 7f20b3fff640 1 -- 172.21.15.154:0/3695317436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f209c005180 msgr2=0x7f20b415b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:14.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.232+0000 7f20b3fff640 1 --2- 172.21.15.154:0/3695317436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f209c005180 0x7f20b415b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:14.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.232+0000 7f20b3fff640 1 -- 172.21.15.154:0/3695317436 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b41f6160 con 0x7f20b40c5f70 2023-09-23T03:26:14.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.233+0000 7f20b3fff640 1 --2- 172.21.15.154:0/3695317436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20b40c5f70 0x7f20b40c63f0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f20ac02c660 tx=0x7f20ac009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:14.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.233+0000 7f209bfff640 1 -- 172.21.15.154:0/3695317436 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ac035d10 con 0x7f20b40c5f70 2023-09-23T03:26:14.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.233+0000 7f209bfff640 1 -- 172.21.15.154:0/3695317436 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20ac035e70 con 0x7f20b40c5f70 2023-09-23T03:26:14.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.233+0000 7f20b9d3c640 1 -- 172.21.15.154:0/3695317436 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20b41f6440 con 0x7f20b40c5f70 2023-09-23T03:26:14.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.233+0000 7f209bfff640 1 -- 172.21.15.154:0/3695317436 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ac03f640 con 0x7f20b40c5f70 2023-09-23T03:26:14.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.233+0000 7f20b9d3c640 1 -- 172.21.15.154:0/3695317436 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20b40b24a0 con 0x7f20b40c5f70 2023-09-23T03:26:14.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.234+0000 7f20b9d3c640 1 -- 172.21.15.154:0/3695317436 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20b404e920 con 0x7f20b40c5f70 2023-09-23T03:26:14.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.235+0000 7f209bfff640 1 -- 172.21.15.154:0/3695317436 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f20ac03b040 con 0x7f20b40c5f70 2023-09-23T03:26:14.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.236+0000 7f209bfff640 1 --2- 172.21.15.154:0/3695317436 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f20a006af30 0x7f20a006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:14.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.236+0000 7f209bfff640 1 -- 172.21.15.154:0/3695317436 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f20ac0aba00 con 0x7f20b40c5f70 2023-09-23T03:26:14.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.237+0000 7f20b37fe640 1 --2- 172.21.15.154:0/3695317436 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f20a006af30 0x7f20a006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:14.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.237+0000 7f20b37fe640 1 --2- 172.21.15.154:0/3695317436 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f20a006af30 0x7f20a006d3f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f20b415ee00 tx=0x7f20a4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:14.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.241+0000 7f209bfff640 1 -- 172.21.15.154:0/3695317436 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f20ac076770 con 0x7f20b40c5f70 2023-09-23T03:26:14.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.438+0000 7f20b9d3c640 1 -- 172.21.15.154:0/3695317436 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f20b40cbdc0 con 0x7f20b40c5f70 2023-09-23T03:26:14.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.439+0000 7f209bfff640 1 -- 172.21.15.154:0/3695317436 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v108) v1 ==== 93+0+10 (secure 0 0 0) 0x7f20ac079d90 con 0x7f20b40c5f70 2023-09-23T03:26:14.440 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:26:14.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.442+0000 7f20b9d3c640 1 -- 172.21.15.154:0/3695317436 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f20a006af30 msgr2=0x7f20a006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:14.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.442+0000 7f20b9d3c640 1 --2- 172.21.15.154:0/3695317436 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f20a006af30 0x7f20a006d3f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f20b415ee00 tx=0x7f20a4009040 comp rx=0 tx=0).stop 2023-09-23T03:26:14.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.442+0000 7f20b9d3c640 1 -- 172.21.15.154:0/3695317436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20b40c5f70 msgr2=0x7f20b40c63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:14.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.442+0000 7f20b9d3c640 1 --2- 172.21.15.154:0/3695317436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20b40c5f70 0x7f20b40c63f0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f20ac02c660 tx=0x7f20ac009750 comp rx=0 tx=0).stop 2023-09-23T03:26:14.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.442+0000 7f20b9d3c640 1 -- 172.21.15.154:0/3695317436 shutdown_connections 2023-09-23T03:26:14.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.442+0000 7f20b9d3c640 1 --2- 172.21.15.154:0/3695317436 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f20a006af30 0x7f20a006d3f0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:14.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.442+0000 7f20b9d3c640 1 --2- 172.21.15.154:0/3695317436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20b40c5f70 0x7f20b40c63f0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:14.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.442+0000 7f20b9d3c640 1 --2- 172.21.15.154:0/3695317436 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20b40c8240 0x7f20b41f5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:14.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.442+0000 7f20b9d3c640 1 --2- 172.21.15.154:0/3695317436 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f209c005180 0x7f20b415b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:14.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.442+0000 7f20b9d3c640 1 -- 172.21.15.154:0/3695317436 >> 172.21.15.154:0/3695317436 conn(0x7f20b406c010 msgr2=0x7f20b40ae310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:14.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.442+0000 7f20b9d3c640 1 -- 172.21.15.154:0/3695317436 shutdown_connections 2023-09-23T03:26:14.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.442+0000 7f20b9d3c640 1 -- 172.21.15.154:0/3695317436 wait complete. 2023-09-23T03:26:14.462 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:26:14.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.672+0000 7fc7dec51640 1 Processor -- start 2023-09-23T03:26:14.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.672+0000 7fc7dec51640 1 -- start start 2023-09-23T03:26:14.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.674+0000 7fc7dec51640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc7d80ad830 0x7fc7d8070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:14.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.674+0000 7fc7dec51640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7d80c9680 0x7fc7d80caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:14.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.674+0000 7fc7dec51640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7d80712f0 con 0x7fc7d80cb400 2023-09-23T03:26:14.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.674+0000 7fc7dec51640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7d80cb890 con 0x7fc7d80ad830 2023-09-23T03:26:14.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.674+0000 7fc7dec51640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d80715d0 con 0x7fc7d80c9680 2023-09-23T03:26:14.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.674+0000 7fc7dd1ee640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fc7d80cb400 0x7fc7d8160720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:44620/0 (socket says 172.21.15.154:44620) 2023-09-23T03:26:14.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.674+0000 7fc7dd1ee640 1 -- 172.21.15.154:0/779613520 learned_addr learned my addr 172.21.15.154:0/779613520 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:26:14.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.675+0000 7fc7d7fff640 1 --2- 172.21.15.154:0/779613520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7d80c9680 0x7fc7d80caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:14.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.675+0000 7fc7dc9ed640 1 --2- 172.21.15.154:0/779613520 >> v2:172.21.15.190:3300/0 conn(0x7fc7d80ad830 0x7fc7d8070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:14.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.675+0000 7fc7d77fe640 1 -- 172.21.15.154:0/779613520 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2550461281 0 0) 0x7fc7d80712f0 con 0x7fc7d80cb400 2023-09-23T03:26:14.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.675+0000 7fc7d77fe640 1 -- 172.21.15.154:0/779613520 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7c8003520 con 0x7fc7d80cb400 2023-09-23T03:26:14.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.675+0000 7fc7d77fe640 1 -- 172.21.15.154:0/779613520 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2156721551 0 0) 0x7fc7c8003520 con 0x7fc7d80cb400 2023-09-23T03:26:14.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.675+0000 7fc7d77fe640 1 -- 172.21.15.154:0/779613520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7d80c9680 msgr2=0x7fc7d80caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:14.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.675+0000 7fc7d77fe640 1 --2- 172.21.15.154:0/779613520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7d80c9680 0x7fc7d80caec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:14.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.675+0000 7fc7d77fe640 1 -- 172.21.15.154:0/779613520 >> v2:172.21.15.190:3300/0 conn(0x7fc7d80ad830 msgr2=0x7fc7d8070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:14.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.675+0000 7fc7d77fe640 1 --2- 172.21.15.154:0/779613520 >> v2:172.21.15.190:3300/0 conn(0x7fc7d80ad830 0x7fc7d8070c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:14.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.675+0000 7fc7d77fe640 1 -- 172.21.15.154:0/779613520 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d80718b0 con 0x7fc7d80cb400 2023-09-23T03:26:14.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.675+0000 7fc7d77fe640 1 -- 172.21.15.154:0/779613520 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fc7d00019f0 con 0x7fc7d80cb400 2023-09-23T03:26:14.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.675+0000 7fc7d77fe640 1 -- 172.21.15.154:0/779613520 >> v1:172.21.15.154:6789/0 conn(0x7fc7d80cb400 legacy=0x7fc7d8160720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:14.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.675+0000 7fc7d77fe640 1 --2- 172.21.15.154:0/779613520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7d80c9680 0x7fc7c8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:14.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.675+0000 7fc7d77fe640 1 --2- 172.21.15.154:0/779613520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7c8003ef0 0x7fc7c80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:14.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.675+0000 7fc7d77fe640 1 --2- 172.21.15.154:0/779613520 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7c8004d00 0x7fc7c8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:14.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.675+0000 7fc7d77fe640 1 -- 172.21.15.154:0/779613520 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d80cb890 con 0x7fc7c8003ef0 2023-09-23T03:26:14.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.675+0000 7fc7d77fe640 1 -- 172.21.15.154:0/779613520 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d80715d0 con 0x7fc7c8004d00 2023-09-23T03:26:14.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.675+0000 7fc7d77fe640 1 -- 172.21.15.154:0/779613520 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c800b980 con 0x7fc7d80c9680 2023-09-23T03:26:14.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.676+0000 7fc7d7fff640 1 --2- 172.21.15.154:0/779613520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7c8003ef0 0x7fc7c80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:14.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.676+0000 7fc7dc9ed640 1 --2- 172.21.15.154:0/779613520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7d80c9680 0x7fc7c8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:14.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.676+0000 7fc7d7fff640 1 -- 172.21.15.154:0/779613520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7d80c9680 msgr2=0x7fc7c8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:14.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.676+0000 7fc7d7fff640 1 --2- 172.21.15.154:0/779613520 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7d80c9680 0x7fc7c8003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:14.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.676+0000 7fc7d7fff640 1 -- 172.21.15.154:0/779613520 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7c8004d00 msgr2=0x7fc7c8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:14.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.676+0000 7fc7dd1ee640 1 --2- 172.21.15.154:0/779613520 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7c8004d00 0x7fc7c8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:14.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.676+0000 7fc7d7fff640 1 --2- 172.21.15.154:0/779613520 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7c8004d00 0x7fc7c8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:14.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.676+0000 7fc7d7fff640 1 -- 172.21.15.154:0/779613520 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c8003520 con 0x7fc7c8003ef0 2023-09-23T03:26:14.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.676+0000 7fc7dd1ee640 1 --2- 172.21.15.154:0/779613520 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7c8004d00 0x7fc7c8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:26:14.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.677+0000 7fc7d7fff640 1 --2- 172.21.15.154:0/779613520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7c8003ef0 0x7fc7c80047c0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4009ac0 tx=0x7fc7c80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6206c373cc4c57 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:14.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.678+0000 7fc7d77fe640 1 -- 172.21.15.154:0/779613520 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c402c6b0 con 0x7fc7c8003ef0 2023-09-23T03:26:14.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.678+0000 7fc7d77fe640 1 -- 172.21.15.154:0/779613520 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c402c810 con 0x7fc7c8003ef0 2023-09-23T03:26:14.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.678+0000 7fc7d77fe640 1 -- 172.21.15.154:0/779613520 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c402cba0 con 0x7fc7c8003ef0 2023-09-23T03:26:14.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.678+0000 7fc7dec51640 1 -- 172.21.15.154:0/779613520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7c8003ef0 msgr2=0x7fc7c80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:14.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.678+0000 7fc7dec51640 1 --2- 172.21.15.154:0/779613520 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7c8003ef0 0x7fc7c80047c0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4009ac0 tx=0x7fc7c80057f0 comp rx=0 tx=0).stop 2023-09-23T03:26:14.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.678+0000 7fc7dd1ee640 1 -- 172.21.15.154:0/779613520 reap_dead start 2023-09-23T03:26:14.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.678+0000 7fc7dec51640 1 -- 172.21.15.154:0/779613520 shutdown_connections 2023-09-23T03:26:14.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.678+0000 7fc7dec51640 1 -- 172.21.15.154:0/779613520 >> 172.21.15.154:0/779613520 conn(0x7fc7d806c010 msgr2=0x7fc7d806c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:14.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.678+0000 7fc7dec51640 1 -- 172.21.15.154:0/779613520 shutdown_connections 2023-09-23T03:26:14.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.678+0000 7fc7dec51640 1 -- 172.21.15.154:0/779613520 wait complete. 2023-09-23T03:26:14.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.680+0000 7fc7dec51640 1 Processor -- start 2023-09-23T03:26:14.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.680+0000 7fc7dec51640 1 -- start start 2023-09-23T03:26:14.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.681+0000 7fc7dec51640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7c8003ef0 0x7fc7d8151cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:14.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.681+0000 7fc7dec51640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7d815e1b0 0x7fc7d815e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:14.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.681+0000 7fc7dec51640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7d815f660 0x7fc7d815fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:14.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.681+0000 7fc7dec51640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d815eb70 con 0x7fc7d815e1b0 2023-09-23T03:26:14.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.681+0000 7fc7dec51640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d8160020 con 0x7fc7d815f660 2023-09-23T03:26:14.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.681+0000 7fc7dec51640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d80c6f70 con 0x7fc7c8003ef0 2023-09-23T03:26:14.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.682+0000 7fc7d7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7d815e1b0 0x7fc7d815e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:14.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.682+0000 7fc7d7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7d815e1b0 0x7fc7d815e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35374/0 (socket says 172.21.15.154:35374) 2023-09-23T03:26:14.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.682+0000 7fc7d7fff640 1 -- 172.21.15.154:0/931088896 learned_addr learned my addr 172.21.15.154:0/931088896 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:14.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.682+0000 7fc7dc9ed640 1 --2- 172.21.15.154:0/931088896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7c8003ef0 0x7fc7d8151cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:14.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.682+0000 7fc7dd1ee640 1 --2- 172.21.15.154:0/931088896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7d815f660 0x7fc7d815fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:14.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.682+0000 7fc7d7fff640 1 -- 172.21.15.154:0/931088896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7c8003ef0 msgr2=0x7fc7d8151cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:14.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.682+0000 7fc7d7fff640 1 --2- 172.21.15.154:0/931088896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7c8003ef0 0x7fc7d8151cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:14.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.682+0000 7fc7d7fff640 1 -- 172.21.15.154:0/931088896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7d815f660 msgr2=0x7fc7d815fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:14.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.682+0000 7fc7d7fff640 1 --2- 172.21.15.154:0/931088896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7d815f660 0x7fc7d815fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:14.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.682+0000 7fc7d7fff640 1 -- 172.21.15.154:0/931088896 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d80c6360 con 0x7fc7d815e1b0 2023-09-23T03:26:14.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.682+0000 7fc7dd1ee640 1 --2- 172.21.15.154:0/931088896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7d815f660 0x7fc7d815fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:26:14.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.682+0000 7fc7d7fff640 1 --2- 172.21.15.154:0/931088896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7d815e1b0 0x7fc7d815e630 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4037040 tx=0x7fc7c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:14.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.683+0000 7fc7d57fa640 1 -- 172.21.15.154:0/931088896 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4003810 con 0x7fc7d815e1b0 2023-09-23T03:26:14.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.683+0000 7fc7d57fa640 1 -- 172.21.15.154:0/931088896 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c4003970 con 0x7fc7d815e1b0 2023-09-23T03:26:14.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.683+0000 7fc7dec51640 1 -- 172.21.15.154:0/931088896 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7d80c6640 con 0x7fc7d815e1b0 2023-09-23T03:26:14.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.683+0000 7fc7d57fa640 1 -- 172.21.15.154:0/931088896 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4003d20 con 0x7fc7d815e1b0 2023-09-23T03:26:14.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.683+0000 7fc7dec51640 1 -- 172.21.15.154:0/931088896 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7d80c6c00 con 0x7fc7d815e1b0 2023-09-23T03:26:14.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.684+0000 7fc7dec51640 1 -- 172.21.15.154:0/931088896 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7d804e920 con 0x7fc7d815e1b0 2023-09-23T03:26:14.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.685+0000 7fc7d57fa640 1 -- 172.21.15.154:0/931088896 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fc7c4040450 con 0x7fc7d815e1b0 2023-09-23T03:26:14.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.686+0000 7fc7d57fa640 1 --2- 172.21.15.154:0/931088896 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc7bc06af60 0x7fc7bc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:14.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.686+0000 7fc7d57fa640 1 -- 172.21.15.154:0/931088896 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6433+0+0 (secure 0 0 0) 0x7fc7c40ab8a0 con 0x7fc7d815e1b0 2023-09-23T03:26:14.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.686+0000 7fc7dc9ed640 1 --2- 172.21.15.154:0/931088896 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc7bc06af60 0x7fc7bc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:14.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.687+0000 7fc7dc9ed640 1 --2- 172.21.15.154:0/931088896 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc7bc06af60 0x7fc7bc06d420 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc009a90 tx=0x7fc7cc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:14.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.691+0000 7fc7d57fa640 1 -- 172.21.15.154:0/931088896 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fc7c4076600 con 0x7fc7d815e1b0 2023-09-23T03:26:14.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.882+0000 7fc7dec51640 1 -- 172.21.15.154:0/931088896 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc7d8072140 con 0x7fc7d815e1b0 2023-09-23T03:26:14.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.883+0000 7fc7d57fa640 1 -- 172.21.15.154:0/931088896 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v108) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc7c4079c20 con 0x7fc7d815e1b0 2023-09-23T03:26:14.884 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:26:14.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.885+0000 7fc7dec51640 1 -- 172.21.15.154:0/931088896 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc7bc06af60 msgr2=0x7fc7bc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:14.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.885+0000 7fc7dec51640 1 --2- 172.21.15.154:0/931088896 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc7bc06af60 0x7fc7bc06d420 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc009a90 tx=0x7fc7cc008040 comp rx=0 tx=0).stop 2023-09-23T03:26:14.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.885+0000 7fc7dec51640 1 -- 172.21.15.154:0/931088896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7d815e1b0 msgr2=0x7fc7d815e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:14.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.885+0000 7fc7dec51640 1 --2- 172.21.15.154:0/931088896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7d815e1b0 0x7fc7d815e630 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4037040 tx=0x7fc7c4004730 comp rx=0 tx=0).stop 2023-09-23T03:26:14.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.885+0000 7fc7dec51640 1 -- 172.21.15.154:0/931088896 shutdown_connections 2023-09-23T03:26:14.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.885+0000 7fc7dec51640 1 --2- 172.21.15.154:0/931088896 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc7bc06af60 0x7fc7bc06d420 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:14.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.885+0000 7fc7dec51640 1 --2- 172.21.15.154:0/931088896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc7d815f660 0x7fc7d815fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:14.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.885+0000 7fc7dec51640 1 --2- 172.21.15.154:0/931088896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc7d815e1b0 0x7fc7d815e630 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:14.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.885+0000 7fc7dec51640 1 --2- 172.21.15.154:0/931088896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc7c8003ef0 0x7fc7d8151cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:14.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.885+0000 7fc7dec51640 1 -- 172.21.15.154:0/931088896 >> 172.21.15.154:0/931088896 conn(0x7fc7d806c010 msgr2=0x7fc7d8070110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:14.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.885+0000 7fc7dec51640 1 -- 172.21.15.154:0/931088896 shutdown_connections 2023-09-23T03:26:14.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:14.885+0000 7fc7dec51640 1 -- 172.21.15.154:0/931088896 wait complete. 2023-09-23T03:26:14.904 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:26:15.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.113+0000 7fdbd6933640 1 Processor -- start 2023-09-23T03:26:15.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.113+0000 7fdbd6933640 1 -- start start 2023-09-23T03:26:15.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.114+0000 7fdbd6933640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdbd0070c60 0x7fdbd00caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:15.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.114+0000 7fdbd6933640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdbd00cb400 0x7fdbd00aba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:15.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.114+0000 7fdbd6933640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbd00712f0 con 0x7fdbd00ad830 2023-09-23T03:26:15.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.114+0000 7fdbd6933640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbd00710f0 con 0x7fdbd00cb400 2023-09-23T03:26:15.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.114+0000 7fdbd6933640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbd00cb870 con 0x7fdbd0070c60 2023-09-23T03:26:15.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.115+0000 7fdbcffff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fdbd00ad830 0x7fdbd00c9680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:44624/0 (socket says 172.21.15.154:44624) 2023-09-23T03:26:15.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.115+0000 7fdbcffff640 1 -- 172.21.15.154:0/3118499161 learned_addr learned my addr 172.21.15.154:0/3118499161 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:26:15.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.115+0000 7fdbd4ed0640 1 --2- 172.21.15.154:0/3118499161 >> v2:172.21.15.154:3300/0 conn(0x7fdbd00cb400 0x7fdbd00aba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:15.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.115+0000 7fdbc7fff640 1 --2- 172.21.15.154:0/3118499161 >> v2:172.21.15.190:3300/0 conn(0x7fdbd0070c60 0x7fdbd00caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:15.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.115+0000 7fdbcf7fe640 1 -- 172.21.15.154:0/3118499161 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 439146418 0 0) 0x7fdbd00712f0 con 0x7fdbd00ad830 2023-09-23T03:26:15.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.115+0000 7fdbd4ed0640 1 -- 172.21.15.154:0/3118499161 >> v2:172.21.15.190:3300/0 conn(0x7fdbd0070c60 msgr2=0x7fdbd00caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:15.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.115+0000 7fdbd4ed0640 1 --2- 172.21.15.154:0/3118499161 >> v2:172.21.15.190:3300/0 conn(0x7fdbd0070c60 0x7fdbd00caec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:15.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.115+0000 7fdbd4ed0640 1 -- 172.21.15.154:0/3118499161 >> v1:172.21.15.154:6789/0 conn(0x7fdbd00ad830 legacy=0x7fdbd00c9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:15.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.115+0000 7fdbd4ed0640 1 -- 172.21.15.154:0/3118499161 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd00715d0 con 0x7fdbd00cb400 2023-09-23T03:26:15.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.115+0000 7fdbc7fff640 1 --2- 172.21.15.154:0/3118499161 >> v2:172.21.15.190:3300/0 conn(0x7fdbd0070c60 0x7fdbd00caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:26:15.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.117+0000 7fdbd4ed0640 1 --2- 172.21.15.154:0/3118499161 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbd00cb400 0x7fdbd00aba70 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fdbd00cbb10 tx=0x7fdbc802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4cc84402ace6544 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:15.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.117+0000 7fdbcf7fe640 1 -- 172.21.15.154:0/3118499161 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc802cc70 con 0x7fdbd00cb400 2023-09-23T03:26:15.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.117+0000 7fdbcf7fe640 1 -- 172.21.15.154:0/3118499161 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc802cdd0 con 0x7fdbd00cb400 2023-09-23T03:26:15.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.117+0000 7fdbcf7fe640 1 -- 172.21.15.154:0/3118499161 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8034660 con 0x7fdbd00cb400 2023-09-23T03:26:15.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.117+0000 7fdbd6933640 1 -- 172.21.15.154:0/3118499161 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbd00cb400 msgr2=0x7fdbd00aba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:15.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.117+0000 7fdbd6933640 1 --2- 172.21.15.154:0/3118499161 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbd00cb400 0x7fdbd00aba70 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fdbd00cbb10 tx=0x7fdbc802bec0 comp rx=0 tx=0).stop 2023-09-23T03:26:15.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.118+0000 7fdbd6933640 1 -- 172.21.15.154:0/3118499161 shutdown_connections 2023-09-23T03:26:15.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.118+0000 7fdbd6933640 1 --2- 172.21.15.154:0/3118499161 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbd00cb400 0x7fdbd00aba70 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:15.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.118+0000 7fdbd6933640 1 --2- 172.21.15.154:0/3118499161 >> v2:172.21.15.190:3300/0 conn(0x7fdbd0070c60 0x7fdbd00caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:15.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.118+0000 7fdbd6933640 1 -- 172.21.15.154:0/3118499161 >> 172.21.15.154:0/3118499161 conn(0x7fdbd006c010 msgr2=0x7fdbd006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:15.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.118+0000 7fdbd6933640 1 -- 172.21.15.154:0/3118499161 shutdown_connections 2023-09-23T03:26:15.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.118+0000 7fdbd6933640 1 -- 172.21.15.154:0/3118499161 wait complete. 2023-09-23T03:26:15.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.119+0000 7fdbd6933640 1 Processor -- start 2023-09-23T03:26:15.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.119+0000 7fdbd6933640 1 -- start start 2023-09-23T03:26:15.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.120+0000 7fdbd6933640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbd0070c60 0x7fdbd01540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:15.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.120+0000 7fdbd6933640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdbd00ad830 0x7fdbd00b1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:15.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.120+0000 7fdbd6933640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbd00cb400 0x7fdbd00ae830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:15.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.120+0000 7fdbd6933640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd00b3cf0 con 0x7fdbd00cb400 2023-09-23T03:26:15.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.120+0000 7fdbd6933640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd00c3070 con 0x7fdbd0070c60 2023-09-23T03:26:15.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.120+0000 7fdbd6933640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd0154ca0 con 0x7fdbd00ad830 2023-09-23T03:26:15.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.121+0000 7fdbd4ed0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbd00cb400 0x7fdbd00ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:15.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.121+0000 7fdbd4ed0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbd00cb400 0x7fdbd00ae830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35398/0 (socket says 172.21.15.154:35398) 2023-09-23T03:26:15.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.121+0000 7fdbd4ed0640 1 -- 172.21.15.154:0/2673793089 learned_addr learned my addr 172.21.15.154:0/2673793089 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:15.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.121+0000 7fdbcffff640 1 --2- 172.21.15.154:0/2673793089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbd0070c60 0x7fdbd01540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:15.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.121+0000 7fdbc7fff640 1 --2- 172.21.15.154:0/2673793089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdbd00ad830 0x7fdbd00b1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:15.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.121+0000 7fdbd4ed0640 1 -- 172.21.15.154:0/2673793089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdbd00ad830 msgr2=0x7fdbd00b1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:15.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.121+0000 7fdbd4ed0640 1 --2- 172.21.15.154:0/2673793089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdbd00ad830 0x7fdbd00b1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:15.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.121+0000 7fdbd4ed0640 1 -- 172.21.15.154:0/2673793089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbd0070c60 msgr2=0x7fdbd01540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:15.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.121+0000 7fdbd4ed0640 1 --2- 172.21.15.154:0/2673793089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbd0070c60 0x7fdbd01540c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:15.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.121+0000 7fdbd4ed0640 1 -- 172.21.15.154:0/2673793089 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc8009a70 con 0x7fdbd00cb400 2023-09-23T03:26:15.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.122+0000 7fdbd4ed0640 1 --2- 172.21.15.154:0/2673793089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbd00cb400 0x7fdbd00ae830 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009d90 tx=0x7fdbc8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:15.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.122+0000 7fdbcd7fa640 1 -- 172.21.15.154:0/2673793089 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8040650 con 0x7fdbd00cb400 2023-09-23T03:26:15.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.122+0000 7fdbcd7fa640 1 -- 172.21.15.154:0/2673793089 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc80407d0 con 0x7fdbd00cb400 2023-09-23T03:26:15.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.122+0000 7fdbcd7fa640 1 -- 172.21.15.154:0/2673793089 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8040b80 con 0x7fdbd00cb400 2023-09-23T03:26:15.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.122+0000 7fdbd6933640 1 -- 172.21.15.154:0/2673793089 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd0154f80 con 0x7fdbd00cb400 2023-09-23T03:26:15.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.122+0000 7fdbd6933640 1 -- 172.21.15.154:0/2673793089 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd00c2d00 con 0x7fdbd00cb400 2023-09-23T03:26:15.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.123+0000 7fdbd6933640 1 -- 172.21.15.154:0/2673793089 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbd00c3200 con 0x7fdbd00cb400 2023-09-23T03:26:15.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.124+0000 7fdbcd7fa640 1 -- 172.21.15.154:0/2673793089 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fdbc8040ce0 con 0x7fdbd00cb400 2023-09-23T03:26:15.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.126+0000 7fdbcd7fa640 1 --2- 172.21.15.154:0/2673793089 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdbb006af60 0x7fdbb006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:15.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.126+0000 7fdbcd7fa640 1 -- 172.21.15.154:0/2673793089 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6433+0+0 (secure 0 0 0) 0x7fdbc80ac6e0 con 0x7fdbd00cb400 2023-09-23T03:26:15.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.126+0000 7fdbcffff640 1 --2- 172.21.15.154:0/2673793089 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdbb006af60 0x7fdbb006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:15.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.127+0000 7fdbcffff640 1 --2- 172.21.15.154:0/2673793089 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdbb006af60 0x7fdbb006d420 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0007fd0 tx=0x7fdbc0007de0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:15.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.130+0000 7fdbcd7fa640 1 -- 172.21.15.154:0/2673793089 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fdbc8077420 con 0x7fdbd00cb400 2023-09-23T03:26:15.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.320+0000 7fdbd6933640 1 -- 172.21.15.154:0/2673793089 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdbd004e920 con 0x7fdbd00cb400 2023-09-23T03:26:15.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.321+0000 7fdbcd7fa640 1 -- 172.21.15.154:0/2673793089 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v108) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdbc807aa40 con 0x7fdbd00cb400 2023-09-23T03:26:15.322 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:26:15.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.324+0000 7fdbd6933640 1 -- 172.21.15.154:0/2673793089 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdbb006af60 msgr2=0x7fdbb006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:15.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.324+0000 7fdbd6933640 1 --2- 172.21.15.154:0/2673793089 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdbb006af60 0x7fdbb006d420 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0007fd0 tx=0x7fdbc0007de0 comp rx=0 tx=0).stop 2023-09-23T03:26:15.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.324+0000 7fdbd6933640 1 -- 172.21.15.154:0/2673793089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbd00cb400 msgr2=0x7fdbd00ae830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:15.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.324+0000 7fdbd6933640 1 --2- 172.21.15.154:0/2673793089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbd00cb400 0x7fdbd00ae830 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009d90 tx=0x7fdbc8004a40 comp rx=0 tx=0).stop 2023-09-23T03:26:15.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.324+0000 7fdbd6933640 1 -- 172.21.15.154:0/2673793089 shutdown_connections 2023-09-23T03:26:15.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.324+0000 7fdbd6933640 1 --2- 172.21.15.154:0/2673793089 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdbb006af60 0x7fdbb006d420 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:15.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.324+0000 7fdbd6933640 1 --2- 172.21.15.154:0/2673793089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbd00cb400 0x7fdbd00ae830 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:15.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.324+0000 7fdbd6933640 1 --2- 172.21.15.154:0/2673793089 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdbd00ad830 0x7fdbd00b1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:15.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.324+0000 7fdbd6933640 1 --2- 172.21.15.154:0/2673793089 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdbd0070c60 0x7fdbd01540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:15.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.324+0000 7fdbd6933640 1 -- 172.21.15.154:0/2673793089 >> 172.21.15.154:0/2673793089 conn(0x7fdbd006c010 msgr2=0x7fdbd00c8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:15.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.324+0000 7fdbd6933640 1 -- 172.21.15.154:0/2673793089 shutdown_connections 2023-09-23T03:26:15.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.324+0000 7fdbd6933640 1 -- 172.21.15.154:0/2673793089 wait complete. 2023-09-23T03:26:15.343 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-09-23T03:26:15.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.562+0000 7f5ca4487640 1 Processor -- start 2023-09-23T03:26:15.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.562+0000 7f5ca4487640 1 -- start start 2023-09-23T03:26:15.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.563+0000 7f5ca4487640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c9c15aba0 0x7f5c9c15bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:15.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.563+0000 7f5ca4487640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5c9c15e5f0 0x7f5c9c15ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:15.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.563+0000 7f5ca4487640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5c9c0caea0 0x7f5c9c0cb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:15.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.563+0000 7f5ca4487640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c9c0cb870 con 0x7f5c9c0caea0 2023-09-23T03:26:15.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.563+0000 7f5ca4487640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c9c15b6a0 con 0x7f5c9c15e5f0 2023-09-23T03:26:15.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.563+0000 7f5ca4487640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c15f0b0 con 0x7f5c9c15aba0 2023-09-23T03:26:15.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.564+0000 7f5ca2a24640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5c9c0caea0 0x7f5c9c0cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:15.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.564+0000 7f5ca2a24640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5c9c0caea0 0x7f5c9c0cb330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35406/0 (socket says 172.21.15.154:35406) 2023-09-23T03:26:15.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.564+0000 7f5ca2a24640 1 -- 172.21.15.154:0/2618899453 learned_addr learned my addr 172.21.15.154:0/2618899453 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:15.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.564+0000 7f5ca1a22640 1 --2- 172.21.15.154:0/2618899453 >> v2:172.21.15.190:3300/0 conn(0x7f5c9c15e5f0 0x7f5c9c15ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:15.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.564+0000 7f5ca2223640 1 --2- 172.21.15.154:0/2618899453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c9c15aba0 0x7f5c9c15bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:15.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.565+0000 7f5ca2a24640 1 -- 172.21.15.154:0/2618899453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c9c15aba0 msgr2=0x7f5c9c15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:15.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.565+0000 7f5ca2a24640 1 --2- 172.21.15.154:0/2618899453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c9c15aba0 0x7f5c9c15bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:15.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.565+0000 7f5ca2a24640 1 -- 172.21.15.154:0/2618899453 >> v2:172.21.15.190:3300/0 conn(0x7f5c9c15e5f0 msgr2=0x7f5c9c15ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:15.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.565+0000 7f5ca2a24640 1 --2- 172.21.15.154:0/2618899453 >> v2:172.21.15.190:3300/0 conn(0x7f5c9c15e5f0 0x7f5c9c15ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:15.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.565+0000 7f5ca2a24640 1 -- 172.21.15.154:0/2618899453 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9c15f390 con 0x7f5c9c0caea0 2023-09-23T03:26:15.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.566+0000 7f5ca2a24640 1 --2- 172.21.15.154:0/2618899453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c9c0caea0 0x7f5c9c0cb330 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c0cadd0 tx=0x7f5c9402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4c3765ae64a7f2 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:15.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.566+0000 7f5ca1221640 1 -- 172.21.15.154:0/2618899453 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9402ca70 con 0x7f5c9c0caea0 2023-09-23T03:26:15.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.566+0000 7f5ca1221640 1 -- 172.21.15.154:0/2618899453 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c9402cbd0 con 0x7f5c9c0caea0 2023-09-23T03:26:15.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.566+0000 7f5ca1221640 1 -- 172.21.15.154:0/2618899453 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c940343f0 con 0x7f5c9c0caea0 2023-09-23T03:26:15.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.567+0000 7f5ca4487640 1 -- 172.21.15.154:0/2618899453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c9c0caea0 msgr2=0x7f5c9c0cb330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:15.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.567+0000 7f5ca4487640 1 --2- 172.21.15.154:0/2618899453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c9c0caea0 0x7f5c9c0cb330 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c0cadd0 tx=0x7f5c9402bed0 comp rx=0 tx=0).stop 2023-09-23T03:26:15.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.567+0000 7f5ca4487640 1 -- 172.21.15.154:0/2618899453 shutdown_connections 2023-09-23T03:26:15.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.567+0000 7f5ca4487640 1 --2- 172.21.15.154:0/2618899453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c9c0caea0 0x7f5c9c0cb330 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:15.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.567+0000 7f5ca4487640 1 --2- 172.21.15.154:0/2618899453 >> v2:172.21.15.190:3300/0 conn(0x7f5c9c15e5f0 0x7f5c9c15ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:15.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.567+0000 7f5ca4487640 1 --2- 172.21.15.154:0/2618899453 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c9c15aba0 0x7f5c9c15bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:15.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.567+0000 7f5ca4487640 1 -- 172.21.15.154:0/2618899453 >> 172.21.15.154:0/2618899453 conn(0x7f5c9c06c010 msgr2=0x7f5c9c06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:15.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.567+0000 7f5ca4487640 1 -- 172.21.15.154:0/2618899453 shutdown_connections 2023-09-23T03:26:15.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.567+0000 7f5ca4487640 1 -- 172.21.15.154:0/2618899453 wait complete. 2023-09-23T03:26:15.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.569+0000 7f5ca4487640 1 Processor -- start 2023-09-23T03:26:15.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.569+0000 7f5ca4487640 1 -- start start 2023-09-23T03:26:15.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.570+0000 7f5ca4487640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c9c0caea0 0x7f5c9c145790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:15.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.570+0000 7f5ca4487640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c9c15aba0 0x7f5c9c152330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:15.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.570+0000 7f5ca4487640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c9c15e5f0 0x7f5c9c1544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:15.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.570+0000 7f5ca4487640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c154080 con 0x7f5c9c15e5f0 2023-09-23T03:26:15.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.570+0000 7f5ca4487640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c152e80 con 0x7f5c9c15aba0 2023-09-23T03:26:15.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.570+0000 7f5ca4487640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c145cd0 con 0x7f5c9c0caea0 2023-09-23T03:26:15.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.571+0000 7f5ca2a24640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c9c15e5f0 0x7f5c9c1544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:15.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.571+0000 7f5ca2a24640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c9c15e5f0 0x7f5c9c1544a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35416/0 (socket says 172.21.15.154:35416) 2023-09-23T03:26:15.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.571+0000 7f5ca2a24640 1 -- 172.21.15.154:0/911409361 learned_addr learned my addr 172.21.15.154:0/911409361 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:15.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.571+0000 7f5ca2223640 1 --2- 172.21.15.154:0/911409361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c9c0caea0 0x7f5c9c145790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:15.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.571+0000 7f5ca1a22640 1 --2- 172.21.15.154:0/911409361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c9c15aba0 0x7f5c9c152330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:15.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.571+0000 7f5ca2a24640 1 -- 172.21.15.154:0/911409361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c9c0caea0 msgr2=0x7f5c9c145790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:15.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.571+0000 7f5ca2a24640 1 --2- 172.21.15.154:0/911409361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c9c0caea0 0x7f5c9c145790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:15.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.571+0000 7f5ca2a24640 1 -- 172.21.15.154:0/911409361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c9c15aba0 msgr2=0x7f5c9c152330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:15.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.571+0000 7f5ca2a24640 1 --2- 172.21.15.154:0/911409361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c9c15aba0 0x7f5c9c152330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:15.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.571+0000 7f5ca2a24640 1 -- 172.21.15.154:0/911409361 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c94009b30 con 0x7f5c9c15e5f0 2023-09-23T03:26:15.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.571+0000 7f5ca2a24640 1 --2- 172.21.15.154:0/911409361 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c9c15e5f0 0x7f5c9c1544a0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f5c9402c4e0 tx=0x7f5c94004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:15.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.572+0000 7f5c92ffd640 1 -- 172.21.15.154:0/911409361 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c94009510 con 0x7f5c9c15e5f0 2023-09-23T03:26:15.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.572+0000 7f5c92ffd640 1 -- 172.21.15.154:0/911409361 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c9403e450 con 0x7f5c9c15e5f0 2023-09-23T03:26:15.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.572+0000 7f5c92ffd640 1 -- 172.21.15.154:0/911409361 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9403e7e0 con 0x7f5c9c15e5f0 2023-09-23T03:26:15.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.572+0000 7f5ca4487640 1 -- 172.21.15.154:0/911409361 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c9c152ba0 con 0x7f5c9c15e5f0 2023-09-23T03:26:15.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.572+0000 7f5ca4487640 1 -- 172.21.15.154:0/911409361 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c9c153ca0 con 0x7f5c9c15e5f0 2023-09-23T03:26:15.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.573+0000 7f5ca4487640 1 -- 172.21.15.154:0/911409361 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c9c0711e0 con 0x7f5c9c15e5f0 2023-09-23T03:26:15.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.574+0000 7f5c92ffd640 1 -- 172.21.15.154:0/911409361 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f5c9403e940 con 0x7f5c9c15e5f0 2023-09-23T03:26:15.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.575+0000 7f5c92ffd640 1 --2- 172.21.15.154:0/911409361 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5c8006af30 0x7f5c8006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:15.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.575+0000 7f5c92ffd640 1 -- 172.21.15.154:0/911409361 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f5c940acc30 con 0x7f5c9c15e5f0 2023-09-23T03:26:15.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.575+0000 7f5ca2223640 1 --2- 172.21.15.154:0/911409361 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5c8006af30 0x7f5c8006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:15.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.576+0000 7f5ca2223640 1 --2- 172.21.15.154:0/911409361 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5c8006af30 0x7f5c8006d3f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5c98006fd0 tx=0x7f5c98008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:15.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.579+0000 7f5c92ffd640 1 -- 172.21.15.154:0/911409361 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f5c94077aa0 con 0x7f5c9c15e5f0 2023-09-23T03:26:15.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.770+0000 7f5ca4487640 1 -- 172.21.15.154:0/911409361 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7f5c9c04e920 con 0x7f5c9c15e5f0 2023-09-23T03:26:15.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.771+0000 7f5c92ffd640 1 -- 172.21.15.154:0/911409361 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v108) v1 ==== 106+0+11 (secure 0 0 0) 0x7f5c94043020 con 0x7f5c9c15e5f0 2023-09-23T03:26:15.772 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:26:15.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.774+0000 7f5ca4487640 1 -- 172.21.15.154:0/911409361 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5c8006af30 msgr2=0x7f5c8006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:15.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.774+0000 7f5ca4487640 1 --2- 172.21.15.154:0/911409361 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5c8006af30 0x7f5c8006d3f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5c98006fd0 tx=0x7f5c98008040 comp rx=0 tx=0).stop 2023-09-23T03:26:15.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.774+0000 7f5ca4487640 1 -- 172.21.15.154:0/911409361 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c9c15e5f0 msgr2=0x7f5c9c1544a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:15.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.774+0000 7f5ca4487640 1 --2- 172.21.15.154:0/911409361 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c9c15e5f0 0x7f5c9c1544a0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f5c9402c4e0 tx=0x7f5c94004a40 comp rx=0 tx=0).stop 2023-09-23T03:26:15.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.774+0000 7f5ca4487640 1 -- 172.21.15.154:0/911409361 shutdown_connections 2023-09-23T03:26:15.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.774+0000 7f5ca4487640 1 --2- 172.21.15.154:0/911409361 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5c8006af30 0x7f5c8006d3f0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:15.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.774+0000 7f5ca4487640 1 --2- 172.21.15.154:0/911409361 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c9c15e5f0 0x7f5c9c1544a0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:15.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.774+0000 7f5ca4487640 1 --2- 172.21.15.154:0/911409361 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c9c15aba0 0x7f5c9c152330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:15.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.774+0000 7f5ca4487640 1 --2- 172.21.15.154:0/911409361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c9c0caea0 0x7f5c9c145790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:15.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.774+0000 7f5ca4487640 1 -- 172.21.15.154:0/911409361 >> 172.21.15.154:0/911409361 conn(0x7f5c9c06c010 msgr2=0x7f5c9c0c1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:15.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.774+0000 7f5ca4487640 1 -- 172.21.15.154:0/911409361 shutdown_connections 2023-09-23T03:26:15.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:15.774+0000 7f5ca4487640 1 -- 172.21.15.154:0/911409361 wait complete. 2023-09-23T03:26:15.792 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-23T03:26:16.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.008+0000 7f61cf851640 1 Processor -- start 2023-09-23T03:26:16.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.009+0000 7f61cf851640 1 -- start start 2023-09-23T03:26:16.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.010+0000 7f61cf851640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f61c80b1400 0x7f61c80ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:16.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.010+0000 7f61cf851640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f61c815cf80 0x7f61c806c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:16.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.010+0000 7f61cf851640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c80ce8c0 con 0x7f61c80cdfa0 2023-09-23T03:26:16.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.010+0000 7f61cf851640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c815d550 con 0x7f61c815cf80 2023-09-23T03:26:16.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.010+0000 7f61cf851640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c815d830 con 0x7f61c80b1400 2023-09-23T03:26:16.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.010+0000 7f61cddee640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f61c815cf80 0x7f61c806c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:16.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.010+0000 7f61cd5ed640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f61c80cdfa0 0x7f61c80659f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:44638/0 (socket says 172.21.15.154:44638) 2023-09-23T03:26:16.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.010+0000 7f61cd5ed640 1 -- 172.21.15.154:0/1265128764 learned_addr learned my addr 172.21.15.154:0/1265128764 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:26:16.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.010+0000 7f61cddee640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f61c815cf80 0x7f61c806c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35422/0 (socket says 172.21.15.154:35422) 2023-09-23T03:26:16.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.011+0000 7f61ccdec640 1 --2- 172.21.15.154:0/1265128764 >> v2:172.21.15.190:3300/0 conn(0x7f61c80b1400 0x7f61c80ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:16.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.011+0000 7f61cddee640 1 -- 172.21.15.154:0/1265128764 >> v2:172.21.15.190:3300/0 conn(0x7f61c80b1400 msgr2=0x7f61c80ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:16.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.011+0000 7f61cddee640 1 --2- 172.21.15.154:0/1265128764 >> v2:172.21.15.190:3300/0 conn(0x7f61c80b1400 0x7f61c80ce380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:16.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.011+0000 7f61cddee640 1 -- 172.21.15.154:0/1265128764 >> v1:172.21.15.154:6789/0 conn(0x7f61c80cdfa0 legacy=0x7f61c80659f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:16.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.011+0000 7f61cddee640 1 -- 172.21.15.154:0/1265128764 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c815db10 con 0x7f61c815cf80 2023-09-23T03:26:16.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.012+0000 7f61cddee640 1 --2- 172.21.15.154:0/1265128764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61c815cf80 0x7f61c806c870 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f61c80724c0 tx=0x7f61c002b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9cc1480e37f602f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:16.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.012+0000 7f61c7fff640 1 -- 172.21.15.154:0/1265128764 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c0031520 con 0x7f61c815cf80 2023-09-23T03:26:16.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.012+0000 7f61c7fff640 1 -- 172.21.15.154:0/1265128764 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c0031680 con 0x7f61c815cf80 2023-09-23T03:26:16.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.012+0000 7f61c7fff640 1 -- 172.21.15.154:0/1265128764 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c0031a10 con 0x7f61c815cf80 2023-09-23T03:26:16.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.013+0000 7f61cf851640 1 -- 172.21.15.154:0/1265128764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61c815cf80 msgr2=0x7f61c806c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:16.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.013+0000 7f61cf851640 1 --2- 172.21.15.154:0/1265128764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61c815cf80 0x7f61c806c870 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f61c80724c0 tx=0x7f61c002b7c0 comp rx=0 tx=0).stop 2023-09-23T03:26:16.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.013+0000 7f61cf851640 1 -- 172.21.15.154:0/1265128764 shutdown_connections 2023-09-23T03:26:16.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.013+0000 7f61cf851640 1 --2- 172.21.15.154:0/1265128764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61c815cf80 0x7f61c806c870 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:16.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.013+0000 7f61cf851640 1 --2- 172.21.15.154:0/1265128764 >> v2:172.21.15.190:3300/0 conn(0x7f61c80b1400 0x7f61c80ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:16.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.013+0000 7f61cf851640 1 -- 172.21.15.154:0/1265128764 >> 172.21.15.154:0/1265128764 conn(0x7f61c80b08e0 msgr2=0x7f61c80b0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:16.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.013+0000 7f61cf851640 1 -- 172.21.15.154:0/1265128764 shutdown_connections 2023-09-23T03:26:16.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.013+0000 7f61cf851640 1 -- 172.21.15.154:0/1265128764 wait complete. 2023-09-23T03:26:16.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.015+0000 7f61cf851640 1 Processor -- start 2023-09-23T03:26:16.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.015+0000 7f61cf851640 1 -- start start 2023-09-23T03:26:16.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.016+0000 7f61cf851640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61c80b1400 0x7f61c8155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:16.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.016+0000 7f61cf851640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61c80cdfa0 0x7f61c80b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:16.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.016+0000 7f61cf851640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61c815cf80 0x7f61c80b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:16.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.016+0000 7f61cf851640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c8156470 con 0x7f61c80cdfa0 2023-09-23T03:26:16.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.016+0000 7f61cf851640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c80b3070 con 0x7f61c80b1400 2023-09-23T03:26:16.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.016+0000 7f61cf851640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c80b2460 con 0x7f61c815cf80 2023-09-23T03:26:16.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.016+0000 7f61ccdec640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61c80cdfa0 0x7f61c80b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:16.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.016+0000 7f61ccdec640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61c80cdfa0 0x7f61c80b1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35434/0 (socket says 172.21.15.154:35434) 2023-09-23T03:26:16.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.016+0000 7f61ccdec640 1 -- 172.21.15.154:0/1111440829 learned_addr learned my addr 172.21.15.154:0/1111440829 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:16.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.017+0000 7f61cd5ed640 1 --2- 172.21.15.154:0/1111440829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61c80b1400 0x7f61c8155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:16.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.017+0000 7f61ccdec640 1 -- 172.21.15.154:0/1111440829 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61c815cf80 msgr2=0x7f61c80b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:16.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.017+0000 7f61cddee640 1 --2- 172.21.15.154:0/1111440829 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61c815cf80 0x7f61c80b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:16.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.017+0000 7f61ccdec640 1 --2- 172.21.15.154:0/1111440829 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61c815cf80 0x7f61c80b3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:16.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.017+0000 7f61ccdec640 1 -- 172.21.15.154:0/1111440829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61c80b1400 msgr2=0x7f61c8155f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:16.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.017+0000 7f61ccdec640 1 --2- 172.21.15.154:0/1111440829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61c80b1400 0x7f61c8155f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:16.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.017+0000 7f61ccdec640 1 -- 172.21.15.154:0/1111440829 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c0009400 con 0x7f61c80cdfa0 2023-09-23T03:26:16.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.017+0000 7f61cd5ed640 1 --2- 172.21.15.154:0/1111440829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61c80b1400 0x7f61c8155f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:26:16.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.017+0000 7f61ccdec640 1 --2- 172.21.15.154:0/1111440829 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61c80cdfa0 0x7f61c80b1a10 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f61bc00c7f0 tx=0x7f61bc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:16.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.017+0000 7f61c5ffb640 1 -- 172.21.15.154:0/1111440829 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61bc004220 con 0x7f61c80cdfa0 2023-09-23T03:26:16.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.018+0000 7f61c5ffb640 1 -- 172.21.15.154:0/1111440829 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61bc0043a0 con 0x7f61c80cdfa0 2023-09-23T03:26:16.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.018+0000 7f61c5ffb640 1 -- 172.21.15.154:0/1111440829 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61bc007860 con 0x7f61c80cdfa0 2023-09-23T03:26:16.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.018+0000 7f61cf851640 1 -- 172.21.15.154:0/1111440829 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c80b2740 con 0x7f61c80cdfa0 2023-09-23T03:26:16.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.018+0000 7f61cf851640 1 -- 172.21.15.154:0/1111440829 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c80b2d00 con 0x7f61c80cdfa0 2023-09-23T03:26:16.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.019+0000 7f61cf851640 1 -- 172.21.15.154:0/1111440829 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61c80b3200 con 0x7f61c80cdfa0 2023-09-23T03:26:16.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.020+0000 7f61c5ffb640 1 -- 172.21.15.154:0/1111440829 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f61bc007bd0 con 0x7f61c80cdfa0 2023-09-23T03:26:16.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.021+0000 7f61c5ffb640 1 --2- 172.21.15.154:0/1111440829 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f61b406af30 0x7f61b406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:16.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.021+0000 7f61c5ffb640 1 -- 172.21.15.154:0/1111440829 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f61bc0899b0 con 0x7f61c80cdfa0 2023-09-23T03:26:16.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.021+0000 7f61cd5ed640 1 --2- 172.21.15.154:0/1111440829 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f61b406af30 0x7f61b406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:16.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.022+0000 7f61cd5ed640 1 --2- 172.21.15.154:0/1111440829 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f61b406af30 0x7f61b406d3f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f61b80012e0 tx=0x7f61b8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:16.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.025+0000 7f61c5ffb640 1 -- 172.21.15.154:0/1111440829 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f61bc054700 con 0x7f61c80cdfa0 2023-09-23T03:26:16.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.227+0000 7f61cf851640 1 -- 172.21.15.154:0/1111440829 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f61c8073eb0 con 0x7f61c80cdfa0 2023-09-23T03:26:16.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.228+0000 7f61c5ffb640 1 -- 172.21.15.154:0/1111440829 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v108) v1 ==== 80+0+0 (secure 0 0 0) 0x7f61bc01e080 con 0x7f61c80cdfa0 2023-09-23T03:26:16.229 INFO:teuthology.orchestra.run.smithi154.stderr:listed 0 entries 2023-09-23T03:26:16.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.231+0000 7f61cf851640 1 -- 172.21.15.154:0/1111440829 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f61b406af30 msgr2=0x7f61b406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:16.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.231+0000 7f61cf851640 1 --2- 172.21.15.154:0/1111440829 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f61b406af30 0x7f61b406d3f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f61b80012e0 tx=0x7f61b8009040 comp rx=0 tx=0).stop 2023-09-23T03:26:16.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.231+0000 7f61cf851640 1 -- 172.21.15.154:0/1111440829 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61c80cdfa0 msgr2=0x7f61c80b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:16.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.231+0000 7f61cf851640 1 --2- 172.21.15.154:0/1111440829 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61c80cdfa0 0x7f61c80b1a10 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f61bc00c7f0 tx=0x7f61bc00ccb0 comp rx=0 tx=0).stop 2023-09-23T03:26:16.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.231+0000 7f61cf851640 1 -- 172.21.15.154:0/1111440829 shutdown_connections 2023-09-23T03:26:16.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.231+0000 7f61cf851640 1 --2- 172.21.15.154:0/1111440829 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f61b406af30 0x7f61b406d3f0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:16.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.231+0000 7f61cf851640 1 --2- 172.21.15.154:0/1111440829 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f61c815cf80 0x7f61c80b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:16.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.231+0000 7f61cf851640 1 --2- 172.21.15.154:0/1111440829 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61c80cdfa0 0x7f61c80b1a10 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:16.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.231+0000 7f61cf851640 1 --2- 172.21.15.154:0/1111440829 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61c80b1400 0x7f61c8155f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:16.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.231+0000 7f61cf851640 1 -- 172.21.15.154:0/1111440829 >> 172.21.15.154:0/1111440829 conn(0x7f61c80b08e0 msgr2=0x7f61c80b0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:16.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.231+0000 7f61cf851640 1 -- 172.21.15.154:0/1111440829 shutdown_connections 2023-09-23T03:26:16.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.231+0000 7f61cf851640 1 -- 172.21.15.154:0/1111440829 wait complete. 2023-09-23T03:26:16.250 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:26:16.251 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:26:16.303 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:26:16.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.522+0000 7f0aff373640 1 Processor -- start 2023-09-23T03:26:16.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.522+0000 7f0aff373640 1 -- start start 2023-09-23T03:26:16.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.523+0000 7f0aff373640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0af80b1b20 0x7f0af80ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:16.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.523+0000 7f0aff373640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0af815ac30 con 0x7f0af80cee70 2023-09-23T03:26:16.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.523+0000 7f0aff373640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0af80b1f70 con 0x7f0af80acce0 2023-09-23T03:26:16.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.523+0000 7f0aff373640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0af80b2e90 con 0x7f0af80b1b20 2023-09-23T03:26:16.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.524+0000 7f0afc90e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0af80b1b20 0x7f0af80ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:16.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.524+0000 7f0afc90e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f0af80b1b20 0x7f0af80ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35440/0 (socket says 172.21.15.154:35440) 2023-09-23T03:26:16.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.524+0000 7f0afd910640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f0af80cee70 0x7f0af80ab140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:44652/0 (socket says 172.21.15.154:44652) 2023-09-23T03:26:16.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.524+0000 7f0afc90e640 1 -- 172.21.15.154:0/1998656003 learned_addr learned my addr 172.21.15.154:0/1998656003 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:16.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.524+0000 7f0afc90e640 1 -- 172.21.15.154:0/1998656003 >> v1:172.21.15.190:6789/0 conn(0x7f0af80acce0 legacy=0x7f0af815ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:16.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.524+0000 7f0afc90e640 1 -- 172.21.15.154:0/1998656003 >> v1:172.21.15.154:6789/0 conn(0x7f0af80cee70 legacy=0x7f0af80ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:16.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.524+0000 7f0afc90e640 1 -- 172.21.15.154:0/1998656003 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af80b2210 con 0x7f0af80b1b20 2023-09-23T03:26:16.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.525+0000 7f0afc90e640 1 --2- 172.21.15.154:0/1998656003 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0af80b1b20 0x7f0af80ce930 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8009870 tx=0x7f0ae802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce305259d41f4001 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:16.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.525+0000 7f0aeffff640 1 -- 172.21.15.154:0/1998656003 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8031870 con 0x7f0af80b1b20 2023-09-23T03:26:16.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.526+0000 7f0aeffff640 1 -- 172.21.15.154:0/1998656003 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae80319d0 con 0x7f0af80b1b20 2023-09-23T03:26:16.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.526+0000 7f0aeffff640 1 -- 172.21.15.154:0/1998656003 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8031d60 con 0x7f0af80b1b20 2023-09-23T03:26:16.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.526+0000 7f0aff373640 1 -- 172.21.15.154:0/1998656003 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0af80b1b20 msgr2=0x7f0af80ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:16.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.526+0000 7f0aff373640 1 --2- 172.21.15.154:0/1998656003 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0af80b1b20 0x7f0af80ce930 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8009870 tx=0x7f0ae802b910 comp rx=0 tx=0).stop 2023-09-23T03:26:16.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.526+0000 7f0aff373640 1 -- 172.21.15.154:0/1998656003 shutdown_connections 2023-09-23T03:26:16.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.526+0000 7f0aff373640 1 --2- 172.21.15.154:0/1998656003 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0af80b1b20 0x7f0af80ce930 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:16.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.526+0000 7f0aff373640 1 -- 172.21.15.154:0/1998656003 >> 172.21.15.154:0/1998656003 conn(0x7f0af806a0f0 msgr2=0x7f0af806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:16.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.526+0000 7f0aff373640 1 -- 172.21.15.154:0/1998656003 shutdown_connections 2023-09-23T03:26:16.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.526+0000 7f0aff373640 1 -- 172.21.15.154:0/1998656003 wait complete. 2023-09-23T03:26:16.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.528+0000 7f0aff373640 1 Processor -- start 2023-09-23T03:26:16.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.528+0000 7f0aff373640 1 -- start start 2023-09-23T03:26:16.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.529+0000 7f0aff373640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0af80acce0 0x7f0af815ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:16.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.529+0000 7f0aff373640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0af80b1b20 0x7f0af80c7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:16.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.529+0000 7f0aff373640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0af80cee70 0x7f0af8158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:16.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.529+0000 7f0aff373640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af80cf320 con 0x7f0af80acce0 2023-09-23T03:26:16.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.529+0000 7f0aff373640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af80c7d90 con 0x7f0af80cee70 2023-09-23T03:26:16.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.529+0000 7f0aff373640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af815ba50 con 0x7f0af80b1b20 2023-09-23T03:26:16.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.529+0000 7f0afd10f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0af80acce0 0x7f0af815ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:16.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.529+0000 7f0afd10f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0af80acce0 0x7f0af815ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35444/0 (socket says 172.21.15.154:35444) 2023-09-23T03:26:16.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.529+0000 7f0afd10f640 1 -- 172.21.15.154:0/3372022560 learned_addr learned my addr 172.21.15.154:0/3372022560 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:16.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.530+0000 7f0afd10f640 1 -- 172.21.15.154:0/3372022560 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0af80b1b20 msgr2=0x7f0af80c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:16.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.530+0000 7f0afd10f640 1 --2- 172.21.15.154:0/3372022560 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0af80b1b20 0x7f0af80c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:16.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.530+0000 7f0afd910640 1 --2- 172.21.15.154:0/3372022560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0af80cee70 0x7f0af8158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:16.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.530+0000 7f0afd10f640 1 -- 172.21.15.154:0/3372022560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0af80cee70 msgr2=0x7f0af8158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:16.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.530+0000 7f0afd10f640 1 --2- 172.21.15.154:0/3372022560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0af80cee70 0x7f0af8158bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:16.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.530+0000 7f0afd10f640 1 -- 172.21.15.154:0/3372022560 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae80094d0 con 0x7f0af80acce0 2023-09-23T03:26:16.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.530+0000 7f0afd10f640 1 --2- 172.21.15.154:0/3372022560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0af80acce0 0x7f0af815ae80 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f0af4004640 tx=0x7f0af400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:16.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.530+0000 7f0aedffb640 1 -- 172.21.15.154:0/3372022560 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af4011790 con 0x7f0af80acce0 2023-09-23T03:26:16.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.530+0000 7f0aedffb640 1 -- 172.21.15.154:0/3372022560 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af40118f0 con 0x7f0af80acce0 2023-09-23T03:26:16.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.530+0000 7f0aedffb640 1 -- 172.21.15.154:0/3372022560 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af4011ca0 con 0x7f0af80acce0 2023-09-23T03:26:16.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.530+0000 7f0aff373640 1 -- 172.21.15.154:0/3372022560 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af80c8030 con 0x7f0af80acce0 2023-09-23T03:26:16.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.531+0000 7f0aff373640 1 -- 172.21.15.154:0/3372022560 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af815bd10 con 0x7f0af80acce0 2023-09-23T03:26:16.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.532+0000 7f0aff373640 1 -- 172.21.15.154:0/3372022560 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af8073eb0 con 0x7f0af80acce0 2023-09-23T03:26:16.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.533+0000 7f0aedffb640 1 -- 172.21.15.154:0/3372022560 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f0af401c460 con 0x7f0af80acce0 2023-09-23T03:26:16.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.534+0000 7f0aedffb640 1 --2- 172.21.15.154:0/3372022560 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0adc06af30 0x7f0adc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:16.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.534+0000 7f0aedffb640 1 -- 172.21.15.154:0/3372022560 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f0af4089a10 con 0x7f0af80acce0 2023-09-23T03:26:16.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.534+0000 7f0afc90e640 1 --2- 172.21.15.154:0/3372022560 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0adc06af30 0x7f0adc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:16.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.535+0000 7f0afc90e640 1 --2- 172.21.15.154:0/3372022560 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0adc06af30 0x7f0adc06d3f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8036000 tx=0x7f0ae8030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:16.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.538+0000 7f0aedffb640 1 -- 172.21.15.154:0/3372022560 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f0af4054650 con 0x7f0af80acce0 2023-09-23T03:26:16.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.729+0000 7f0aff373640 1 -- 172.21.15.154:0/3372022560 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0af806c930 con 0x7f0af80acce0 2023-09-23T03:26:16.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.730+0000 7f0aedffb640 1 -- 172.21.15.154:0/3372022560 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+18550 (secure 0 0 0) 0x7f0af4057c70 con 0x7f0af80acce0 2023-09-23T03:26:16.731 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:16.731 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":108,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:25:59.416373+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:25:44.763842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-09-23T03:25:46.025926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data_extra","create_time":"2023-09-23T03:25:56.961772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:26:16.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.733+0000 7f0aff373640 1 -- 172.21.15.154:0/3372022560 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0adc06af30 msgr2=0x7f0adc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:16.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.733+0000 7f0aff373640 1 --2- 172.21.15.154:0/3372022560 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0adc06af30 0x7f0adc06d3f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8036000 tx=0x7f0ae8030000 comp rx=0 tx=0).stop 2023-09-23T03:26:16.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.733+0000 7f0aff373640 1 -- 172.21.15.154:0/3372022560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0af80acce0 msgr2=0x7f0af815ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:16.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.733+0000 7f0aff373640 1 --2- 172.21.15.154:0/3372022560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0af80acce0 0x7f0af815ae80 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f0af4004640 tx=0x7f0af400c8e0 comp rx=0 tx=0).stop 2023-09-23T03:26:16.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.733+0000 7f0aff373640 1 -- 172.21.15.154:0/3372022560 shutdown_connections 2023-09-23T03:26:16.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.733+0000 7f0aff373640 1 --2- 172.21.15.154:0/3372022560 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0adc06af30 0x7f0adc06d3f0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:16.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.733+0000 7f0aff373640 1 --2- 172.21.15.154:0/3372022560 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0af80cee70 0x7f0af8158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:16.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.733+0000 7f0aff373640 1 --2- 172.21.15.154:0/3372022560 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0af80b1b20 0x7f0af80c7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:16.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.733+0000 7f0aff373640 1 --2- 172.21.15.154:0/3372022560 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0af80acce0 0x7f0af815ae80 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:16.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.733+0000 7f0aff373640 1 -- 172.21.15.154:0/3372022560 >> 172.21.15.154:0/3372022560 conn(0x7f0af806a0f0 msgr2=0x7f0af815a170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:16.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.733+0000 7f0aff373640 1 -- 172.21.15.154:0/3372022560 shutdown_connections 2023-09-23T03:26:16.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.733+0000 7f0aff373640 1 -- 172.21.15.154:0/3372022560 wait complete. 2023-09-23T03:26:16.753 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:25:44.763842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:25:46.025926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-09-23T03:25:56.961772+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T03:26:16.753 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:26:16.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.972+0000 7f4793fff640 1 Processor -- start 2023-09-23T03:26:16.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.972+0000 7f4793fff640 1 -- start start 2023-09-23T03:26:16.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.974+0000 7f4793fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f4794158160 0x7f479415ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:16.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.974+0000 7f4793fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f479415e130 0x7f47940c9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:16.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.974+0000 7f4793fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47940c9a80 0x7f479406fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:16.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.974+0000 7f4793fff640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4794158540 con 0x7f479415e130 2023-09-23T03:26:16.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.974+0000 7f4793fff640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f479415b3e0 con 0x7f4794158160 2023-09-23T03:26:16.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.974+0000 7f4793fff640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f479415b6c0 con 0x7f47940c9a80 2023-09-23T03:26:16.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.974+0000 7f47927fc640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f479415e130 0x7f47940c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:16.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.974+0000 7f47927fc640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f479415e130 0x7f47940c9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35446/0 (socket says 172.21.15.154:35446) 2023-09-23T03:26:16.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.974+0000 7f47927fc640 1 -- 172.21.15.154:0/76419878 learned_addr learned my addr 172.21.15.154:0/76419878 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:16.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.974+0000 7f47937fe640 1 --2- 172.21.15.154:0/76419878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47940c9a80 0x7f479406fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:16.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.974+0000 7f4792ffd640 1 --2- 172.21.15.154:0/76419878 >> v2:172.21.15.190:3300/0 conn(0x7f4794158160 0x7f479415ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:16.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.974+0000 7f47927fc640 1 -- 172.21.15.154:0/76419878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47940c9a80 msgr2=0x7f479406fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:16.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.974+0000 7f47927fc640 1 --2- 172.21.15.154:0/76419878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47940c9a80 0x7f479406fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:16.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.974+0000 7f47927fc640 1 -- 172.21.15.154:0/76419878 >> v2:172.21.15.190:3300/0 conn(0x7f4794158160 msgr2=0x7f479415ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:16.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.974+0000 7f47927fc640 1 --2- 172.21.15.154:0/76419878 >> v2:172.21.15.190:3300/0 conn(0x7f4794158160 0x7f479415ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:16.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.974+0000 7f47927fc640 1 -- 172.21.15.154:0/76419878 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479415b9a0 con 0x7f479415e130 2023-09-23T03:26:16.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.975+0000 7f4792ffd640 1 --2- 172.21.15.154:0/76419878 >> v2:172.21.15.190:3300/0 conn(0x7f4794158160 0x7f479415ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:26:16.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.976+0000 7f47927fc640 1 --2- 172.21.15.154:0/76419878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f479415e130 0x7f47940c9540 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f479406ce80 tx=0x7f477c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37e57768949b10f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:16.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.976+0000 7f4791ffb640 1 -- 172.21.15.154:0/76419878 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477c031870 con 0x7f479415e130 2023-09-23T03:26:16.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.976+0000 7f4791ffb640 1 -- 172.21.15.154:0/76419878 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f477c0319d0 con 0x7f479415e130 2023-09-23T03:26:16.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.976+0000 7f4791ffb640 1 -- 172.21.15.154:0/76419878 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477c031d60 con 0x7f479415e130 2023-09-23T03:26:16.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.977+0000 7f4793fff640 1 -- 172.21.15.154:0/76419878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f479415e130 msgr2=0x7f47940c9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:16.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.977+0000 7f4793fff640 1 --2- 172.21.15.154:0/76419878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f479415e130 0x7f47940c9540 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f479406ce80 tx=0x7f477c02b910 comp rx=0 tx=0).stop 2023-09-23T03:26:16.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.977+0000 7f4793fff640 1 -- 172.21.15.154:0/76419878 shutdown_connections 2023-09-23T03:26:16.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.977+0000 7f4793fff640 1 --2- 172.21.15.154:0/76419878 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47940c9a80 0x7f479406fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:16.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.977+0000 7f4793fff640 1 --2- 172.21.15.154:0/76419878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f479415e130 0x7f47940c9540 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:16.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.977+0000 7f4793fff640 1 --2- 172.21.15.154:0/76419878 >> v2:172.21.15.190:3300/0 conn(0x7f4794158160 0x7f479415ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:16.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.977+0000 7f4793fff640 1 -- 172.21.15.154:0/76419878 >> 172.21.15.154:0/76419878 conn(0x7f479406a300 msgr2=0x7f47940afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:16.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.977+0000 7f4793fff640 1 -- 172.21.15.154:0/76419878 shutdown_connections 2023-09-23T03:26:16.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.977+0000 7f4793fff640 1 -- 172.21.15.154:0/76419878 wait complete. 2023-09-23T03:26:16.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.978+0000 7f4793fff640 1 Processor -- start 2023-09-23T03:26:16.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.979+0000 7f4793fff640 1 -- start start 2023-09-23T03:26:16.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.980+0000 7f4793fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4794158160 0x7f47940afe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:16.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.980+0000 7f4793fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f479415e130 0x7f47940ae740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:16.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.980+0000 7f4793fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47941f2580 0x7f479414e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:16.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.980+0000 7f4793fff640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47940af810 con 0x7f4794158160 2023-09-23T03:26:16.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.980+0000 7f4793fff640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479414c400 con 0x7f47941f2580 2023-09-23T03:26:16.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.980+0000 7f4793fff640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47940aef70 con 0x7f479415e130 2023-09-23T03:26:16.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.980+0000 7f4792ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4794158160 0x7f47940afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:16.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.980+0000 7f4792ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4794158160 0x7f47940afe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35456/0 (socket says 172.21.15.154:35456) 2023-09-23T03:26:16.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.980+0000 7f4792ffd640 1 -- 172.21.15.154:0/1584514870 learned_addr learned my addr 172.21.15.154:0/1584514870 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:16.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.980+0000 7f47927fc640 1 --2- 172.21.15.154:0/1584514870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f479415e130 0x7f47940ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:16.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.980+0000 7f4792ffd640 1 -- 172.21.15.154:0/1584514870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f479415e130 msgr2=0x7f47940ae740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:16.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.980+0000 7f4792ffd640 1 --2- 172.21.15.154:0/1584514870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f479415e130 0x7f47940ae740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:16.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.980+0000 7f47937fe640 1 --2- 172.21.15.154:0/1584514870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47941f2580 0x7f479414e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:16.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.980+0000 7f4792ffd640 1 -- 172.21.15.154:0/1584514870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47941f2580 msgr2=0x7f479414e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:16.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.980+0000 7f4792ffd640 1 --2- 172.21.15.154:0/1584514870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47941f2580 0x7f479414e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:16.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.980+0000 7f4792ffd640 1 -- 172.21.15.154:0/1584514870 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477c0094d0 con 0x7f4794158160 2023-09-23T03:26:16.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.981+0000 7f4792ffd640 1 --2- 172.21.15.154:0/1584514870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4794158160 0x7f47940afe10 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f478400cf60 tx=0x7f478400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:16.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.981+0000 7f478bfff640 1 -- 172.21.15.154:0/1584514870 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478400d840 con 0x7f4794158160 2023-09-23T03:26:16.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.981+0000 7f478bfff640 1 -- 172.21.15.154:0/1584514870 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f478400d9a0 con 0x7f4794158160 2023-09-23T03:26:16.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.981+0000 7f478bfff640 1 -- 172.21.15.154:0/1584514870 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478400dd30 con 0x7f4794158160 2023-09-23T03:26:16.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.981+0000 7f4793fff640 1 -- 172.21.15.154:0/1584514870 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47940af250 con 0x7f4794158160 2023-09-23T03:26:16.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.981+0000 7f4793fff640 1 -- 172.21.15.154:0/1584514870 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47940aec80 con 0x7f4794158160 2023-09-23T03:26:16.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.982+0000 7f4793fff640 1 -- 172.21.15.154:0/1584514870 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47940736b0 con 0x7f4794158160 2023-09-23T03:26:16.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.983+0000 7f478bfff640 1 -- 172.21.15.154:0/1584514870 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f4784015440 con 0x7f4794158160 2023-09-23T03:26:16.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.984+0000 7f478bfff640 1 --2- 172.21.15.154:0/1584514870 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f477406af60 0x7f477406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:16.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.984+0000 7f478bfff640 1 -- 172.21.15.154:0/1584514870 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f478408a320 con 0x7f4794158160 2023-09-23T03:26:16.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.984+0000 7f47927fc640 1 --2- 172.21.15.154:0/1584514870 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f477406af60 0x7f477406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:16.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.985+0000 7f47927fc640 1 --2- 172.21.15.154:0/1584514870 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f477406af60 0x7f477406d420 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f477c036000 tx=0x7f477c03c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:16.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:16.989+0000 7f478bfff640 1 -- 172.21.15.154:0/1584514870 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f47840551c0 con 0x7f4794158160 2023-09-23T03:26:17.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.180+0000 7f4793fff640 1 -- 172.21.15.154:0/1584514870 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f479414c5c0 con 0x7f4794158160 2023-09-23T03:26:17.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.181+0000 7f478bfff640 1 -- 172.21.15.154:0/1584514870 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v108) v1 ==== 93+0+10 (secure 0 0 0) 0x7f47840587e0 con 0x7f4794158160 2023-09-23T03:26:17.181 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:26:17.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.183+0000 7f4793fff640 1 -- 172.21.15.154:0/1584514870 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f477406af60 msgr2=0x7f477406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:17.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.183+0000 7f4793fff640 1 --2- 172.21.15.154:0/1584514870 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f477406af60 0x7f477406d420 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f477c036000 tx=0x7f477c03c040 comp rx=0 tx=0).stop 2023-09-23T03:26:17.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.183+0000 7f4793fff640 1 -- 172.21.15.154:0/1584514870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4794158160 msgr2=0x7f47940afe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:17.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.184+0000 7f4793fff640 1 --2- 172.21.15.154:0/1584514870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4794158160 0x7f47940afe10 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f478400cf60 tx=0x7f478400cf90 comp rx=0 tx=0).stop 2023-09-23T03:26:17.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.184+0000 7f4793fff640 1 -- 172.21.15.154:0/1584514870 shutdown_connections 2023-09-23T03:26:17.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.184+0000 7f4793fff640 1 --2- 172.21.15.154:0/1584514870 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f477406af60 0x7f477406d420 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:17.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.184+0000 7f4793fff640 1 --2- 172.21.15.154:0/1584514870 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47941f2580 0x7f479414e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:17.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.184+0000 7f4793fff640 1 --2- 172.21.15.154:0/1584514870 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f479415e130 0x7f47940ae740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:17.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.184+0000 7f4793fff640 1 --2- 172.21.15.154:0/1584514870 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4794158160 0x7f47940afe10 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:17.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.184+0000 7f4793fff640 1 -- 172.21.15.154:0/1584514870 >> 172.21.15.154:0/1584514870 conn(0x7f479406a300 msgr2=0x7f479415d290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:17.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.184+0000 7f4793fff640 1 -- 172.21.15.154:0/1584514870 shutdown_connections 2023-09-23T03:26:17.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.184+0000 7f4793fff640 1 -- 172.21.15.154:0/1584514870 wait complete. 2023-09-23T03:26:17.201 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:26:17.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.420+0000 7f6e51199640 1 Processor -- start 2023-09-23T03:26:17.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.420+0000 7f6e51199640 1 -- start start 2023-09-23T03:26:17.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.422+0000 7f6e51199640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e4c15cb30 0x7f6e4c070fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:17.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.422+0000 7f6e51199640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6e4c0714e0 0x7f6e4c069cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:17.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.422+0000 7f6e51199640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e4c0c6590 con 0x7f6e4c06a210 2023-09-23T03:26:17.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.422+0000 7f6e51199640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e4c06a640 con 0x7f6e4c0714e0 2023-09-23T03:26:17.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.422+0000 7f6e51199640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c06f840 con 0x7f6e4c15cb30 2023-09-23T03:26:17.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.422+0000 7f6e437fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6e4c0714e0 0x7f6e4c069cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:17.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.423+0000 7f6e4bfff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e4c15cb30 0x7f6e4c070fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:17.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.423+0000 7f6e437fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6e4c0714e0 0x7f6e4c069cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.154:35594/0 (socket says 172.21.15.154:35594) 2023-09-23T03:26:17.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.423+0000 7f6e437fe640 1 -- 172.21.15.154:0/2330988642 learned_addr learned my addr 172.21.15.154:0/2330988642 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:17.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.423+0000 7f6e437fe640 1 -- 172.21.15.154:0/2330988642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e4c15cb30 msgr2=0x7f6e4c070fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:17.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.423+0000 7f6e437fe640 1 --2- 172.21.15.154:0/2330988642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e4c15cb30 0x7f6e4c070fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:17.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.423+0000 7f6e437fe640 1 -- 172.21.15.154:0/2330988642 >> v1:172.21.15.190:6789/0 conn(0x7f6e4c06a210 legacy=0x7f6e4c1604d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:17.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.423+0000 7f6e437fe640 1 -- 172.21.15.154:0/2330988642 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e4c071b10 con 0x7f6e4c0714e0 2023-09-23T03:26:17.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.423+0000 7f6e4bfff640 1 --2- 172.21.15.154:0/2330988642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e4c15cb30 0x7f6e4c070fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:26:17.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.425+0000 7f6e437fe640 1 --2- 172.21.15.154:0/2330988642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e4c0714e0 0x7f6e4c069cd0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6e34009870 tx=0x7f6e3402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f642e11f6893b1d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:17.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.425+0000 7f6e4b7fe640 1 -- 172.21.15.154:0/2330988642 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e34030840 con 0x7f6e4c0714e0 2023-09-23T03:26:17.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.425+0000 7f6e4b7fe640 1 -- 172.21.15.154:0/2330988642 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e340309a0 con 0x7f6e4c0714e0 2023-09-23T03:26:17.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.425+0000 7f6e4b7fe640 1 -- 172.21.15.154:0/2330988642 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e34030d30 con 0x7f6e4c0714e0 2023-09-23T03:26:17.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.426+0000 7f6e51199640 1 -- 172.21.15.154:0/2330988642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e4c0714e0 msgr2=0x7f6e4c069cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:17.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.426+0000 7f6e51199640 1 --2- 172.21.15.154:0/2330988642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e4c0714e0 0x7f6e4c069cd0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6e34009870 tx=0x7f6e3402b980 comp rx=0 tx=0).stop 2023-09-23T03:26:17.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.426+0000 7f6e51199640 1 -- 172.21.15.154:0/2330988642 shutdown_connections 2023-09-23T03:26:17.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.426+0000 7f6e51199640 1 --2- 172.21.15.154:0/2330988642 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e4c0714e0 0x7f6e4c069cd0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:17.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.426+0000 7f6e51199640 1 --2- 172.21.15.154:0/2330988642 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e4c15cb30 0x7f6e4c070fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:17.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.426+0000 7f6e51199640 1 -- 172.21.15.154:0/2330988642 >> 172.21.15.154:0/2330988642 conn(0x7f6e4c0b1030 msgr2=0x7f6e4c0b1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:17.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.426+0000 7f6e51199640 1 -- 172.21.15.154:0/2330988642 shutdown_connections 2023-09-23T03:26:17.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.426+0000 7f6e51199640 1 -- 172.21.15.154:0/2330988642 wait complete. 2023-09-23T03:26:17.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.428+0000 7f6e51199640 1 Processor -- start 2023-09-23T03:26:17.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.428+0000 7f6e51199640 1 -- start start 2023-09-23T03:26:17.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.429+0000 7f6e51199640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e4c06a210 0x7f6e4c15b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:17.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.429+0000 7f6e51199640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e4c0714e0 0x7f6e4c0adbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:17.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.429+0000 7f6e51199640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e4c15cb30 0x7f6e4c0c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:17.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.429+0000 7f6e51199640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c0aec40 con 0x7f6e4c0714e0 2023-09-23T03:26:17.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.429+0000 7f6e51199640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c0c9e40 con 0x7f6e4c15cb30 2023-09-23T03:26:17.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.429+0000 7f6e51199640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c15b980 con 0x7f6e4c06a210 2023-09-23T03:26:17.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.429+0000 7f6e437fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e4c0714e0 0x7f6e4c0adbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:17.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.429+0000 7f6e437fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e4c0714e0 0x7f6e4c0adbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35464/0 (socket says 172.21.15.154:35464) 2023-09-23T03:26:17.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.429+0000 7f6e437fe640 1 -- 172.21.15.154:0/3238648579 learned_addr learned my addr 172.21.15.154:0/3238648579 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:17.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.430+0000 7f6e4bfff640 1 --2- 172.21.15.154:0/3238648579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e4c06a210 0x7f6e4c15b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:17.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.430+0000 7f6e50998640 1 --2- 172.21.15.154:0/3238648579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e4c15cb30 0x7f6e4c0c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:17.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.430+0000 7f6e437fe640 1 -- 172.21.15.154:0/3238648579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e4c06a210 msgr2=0x7f6e4c15b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:17.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.430+0000 7f6e437fe640 1 --2- 172.21.15.154:0/3238648579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e4c06a210 0x7f6e4c15b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:17.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.430+0000 7f6e437fe640 1 -- 172.21.15.154:0/3238648579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e4c15cb30 msgr2=0x7f6e4c0c8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:17.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.430+0000 7f6e437fe640 1 --2- 172.21.15.154:0/3238648579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e4c15cb30 0x7f6e4c0c8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:17.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.430+0000 7f6e437fe640 1 -- 172.21.15.154:0/3238648579 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e3c009400 con 0x7f6e4c0714e0 2023-09-23T03:26:17.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.430+0000 7f6e4bfff640 1 --2- 172.21.15.154:0/3238648579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e4c06a210 0x7f6e4c15b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:26:17.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.430+0000 7f6e50998640 1 --2- 172.21.15.154:0/3238648579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e4c15cb30 0x7f6e4c0c8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:26:17.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.430+0000 7f6e437fe640 1 --2- 172.21.15.154:0/3238648579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e4c0714e0 0x7f6e4c0adbc0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f6e3402b9c0 tx=0x7f6e34004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:17.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.430+0000 7f6e497fa640 1 -- 172.21.15.154:0/3238648579 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e34039c90 con 0x7f6e4c0714e0 2023-09-23T03:26:17.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.430+0000 7f6e497fa640 1 -- 172.21.15.154:0/3238648579 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e34039df0 con 0x7f6e4c0714e0 2023-09-23T03:26:17.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.430+0000 7f6e51199640 1 -- 172.21.15.154:0/3238648579 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e340094d0 con 0x7f6e4c0714e0 2023-09-23T03:26:17.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.430+0000 7f6e497fa640 1 -- 172.21.15.154:0/3238648579 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e340446c0 con 0x7f6e4c0714e0 2023-09-23T03:26:17.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.431+0000 7f6e51199640 1 -- 172.21.15.154:0/3238648579 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e4c0ae960 con 0x7f6e4c0714e0 2023-09-23T03:26:17.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.432+0000 7f6e51199640 1 -- 172.21.15.154:0/3238648579 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e4c04e920 con 0x7f6e4c0714e0 2023-09-23T03:26:17.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.433+0000 7f6e497fa640 1 -- 172.21.15.154:0/3238648579 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f6e3404d3b0 con 0x7f6e4c0714e0 2023-09-23T03:26:17.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.433+0000 7f6e497fa640 1 --2- 172.21.15.154:0/3238648579 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6e2c06af30 0x7f6e2c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:17.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.434+0000 7f6e497fa640 1 -- 172.21.15.154:0/3238648579 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f6e3403e070 con 0x7f6e4c0714e0 2023-09-23T03:26:17.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.434+0000 7f6e4bfff640 1 --2- 172.21.15.154:0/3238648579 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6e2c06af30 0x7f6e2c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:17.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.434+0000 7f6e4bfff640 1 --2- 172.21.15.154:0/3238648579 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6e2c06af30 0x7f6e2c06d3f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009c50 tx=0x7f6e3c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:17.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.438+0000 7f6e497fa640 1 -- 172.21.15.154:0/3238648579 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f6e34077460 con 0x7f6e4c0714e0 2023-09-23T03:26:17.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.629+0000 7f6e51199640 1 -- 172.21.15.154:0/3238648579 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6e4c0c9f80 con 0x7f6e4c0714e0 2023-09-23T03:26:17.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.630+0000 7f6e497fa640 1 -- 172.21.15.154:0/3238648579 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v108) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6e3407aa80 con 0x7f6e4c0714e0 2023-09-23T03:26:17.631 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:26:17.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.633+0000 7f6e51199640 1 -- 172.21.15.154:0/3238648579 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6e2c06af30 msgr2=0x7f6e2c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:17.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.633+0000 7f6e51199640 1 --2- 172.21.15.154:0/3238648579 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6e2c06af30 0x7f6e2c06d3f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009c50 tx=0x7f6e3c008040 comp rx=0 tx=0).stop 2023-09-23T03:26:17.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.633+0000 7f6e51199640 1 -- 172.21.15.154:0/3238648579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e4c0714e0 msgr2=0x7f6e4c0adbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:17.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.633+0000 7f6e51199640 1 --2- 172.21.15.154:0/3238648579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e4c0714e0 0x7f6e4c0adbc0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f6e3402b9c0 tx=0x7f6e34004730 comp rx=0 tx=0).stop 2023-09-23T03:26:17.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.633+0000 7f6e51199640 1 -- 172.21.15.154:0/3238648579 shutdown_connections 2023-09-23T03:26:17.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.633+0000 7f6e51199640 1 --2- 172.21.15.154:0/3238648579 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6e2c06af30 0x7f6e2c06d3f0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:17.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.633+0000 7f6e51199640 1 --2- 172.21.15.154:0/3238648579 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e4c15cb30 0x7f6e4c0c8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:17.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.633+0000 7f6e51199640 1 --2- 172.21.15.154:0/3238648579 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e4c0714e0 0x7f6e4c0adbc0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:17.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.633+0000 7f6e51199640 1 --2- 172.21.15.154:0/3238648579 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e4c06a210 0x7f6e4c15b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:17.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.633+0000 7f6e51199640 1 -- 172.21.15.154:0/3238648579 >> 172.21.15.154:0/3238648579 conn(0x7f6e4c0b1030 msgr2=0x7f6e4c0cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:17.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.633+0000 7f6e51199640 1 -- 172.21.15.154:0/3238648579 shutdown_connections 2023-09-23T03:26:17.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.633+0000 7f6e51199640 1 -- 172.21.15.154:0/3238648579 wait complete. 2023-09-23T03:26:17.652 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:26:17.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.871+0000 7f4993c90640 1 Processor -- start 2023-09-23T03:26:17.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.871+0000 7f4993c90640 1 -- start start 2023-09-23T03:26:17.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.872+0000 7f4993c90640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f498c0c9510 0x7f498c1636c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:17.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.872+0000 7f4993c90640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f498c0c9980 0x7f498c163c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:17.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.872+0000 7f4993c90640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f498c0c1a40 con 0x7f498c0cae90 2023-09-23T03:26:17.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.873+0000 7f4993c90640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f498c1624f0 con 0x7f498c0c9510 2023-09-23T03:26:17.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.873+0000 7f4993c90640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f498c162be0 con 0x7f498c0c9980 2023-09-23T03:26:17.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.873+0000 7f499122b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f498c0c9510 0x7f498c1636c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:17.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.873+0000 7f4991a2c640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f498c0cae90 0x7f498c0cb270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:44656/0 (socket says 172.21.15.154:44656) 2023-09-23T03:26:17.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.873+0000 7f499122b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f498c0c9510 0x7f498c1636c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35474/0 (socket says 172.21.15.154:35474) 2023-09-23T03:26:17.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.873+0000 7f4991a2c640 1 -- 172.21.15.154:0/1155247716 learned_addr learned my addr 172.21.15.154:0/1155247716 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:26:17.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.873+0000 7f499222d640 1 --2- 172.21.15.154:0/1155247716 >> v2:172.21.15.190:3300/0 conn(0x7f498c0c9980 0x7f498c163c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:17.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.873+0000 7f499122b640 1 -- 172.21.15.154:0/1155247716 >> v2:172.21.15.190:3300/0 conn(0x7f498c0c9980 msgr2=0x7f498c163c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:17.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.873+0000 7f499122b640 1 --2- 172.21.15.154:0/1155247716 >> v2:172.21.15.190:3300/0 conn(0x7f498c0c9980 0x7f498c163c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:17.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.873+0000 7f499122b640 1 -- 172.21.15.154:0/1155247716 >> v1:172.21.15.154:6789/0 conn(0x7f498c0cae90 legacy=0x7f498c0cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:17.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.873+0000 7f499122b640 1 -- 172.21.15.154:0/1155247716 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498c162ec0 con 0x7f498c0c9510 2023-09-23T03:26:17.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.874+0000 7f499122b640 1 --2- 172.21.15.154:0/1155247716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f498c0c9510 0x7f498c1636c0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f498c0cc560 tx=0x7f497c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f9662c55604621d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:17.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.875+0000 7f4990a2a640 1 -- 172.21.15.154:0/1155247716 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c030610 con 0x7f498c0c9510 2023-09-23T03:26:17.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.875+0000 7f4990a2a640 1 -- 172.21.15.154:0/1155247716 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497c030770 con 0x7f498c0c9510 2023-09-23T03:26:17.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.875+0000 7f4990a2a640 1 -- 172.21.15.154:0/1155247716 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c030b00 con 0x7f498c0c9510 2023-09-23T03:26:17.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.875+0000 7f4993c90640 1 -- 172.21.15.154:0/1155247716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f498c0c9510 msgr2=0x7f498c1636c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:17.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.875+0000 7f4993c90640 1 --2- 172.21.15.154:0/1155247716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f498c0c9510 0x7f498c1636c0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f498c0cc560 tx=0x7f497c02b960 comp rx=0 tx=0).stop 2023-09-23T03:26:17.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.875+0000 7f4993c90640 1 -- 172.21.15.154:0/1155247716 shutdown_connections 2023-09-23T03:26:17.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.875+0000 7f4993c90640 1 --2- 172.21.15.154:0/1155247716 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f498c0c9510 0x7f498c1636c0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:17.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.875+0000 7f4993c90640 1 --2- 172.21.15.154:0/1155247716 >> v2:172.21.15.190:3300/0 conn(0x7f498c0c9980 0x7f498c163c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:17.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.875+0000 7f4993c90640 1 -- 172.21.15.154:0/1155247716 >> 172.21.15.154:0/1155247716 conn(0x7f498c06c010 msgr2=0x7f498c06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:17.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.876+0000 7f4993c90640 1 -- 172.21.15.154:0/1155247716 shutdown_connections 2023-09-23T03:26:17.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.876+0000 7f4993c90640 1 -- 172.21.15.154:0/1155247716 wait complete. 2023-09-23T03:26:17.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.877+0000 7f4993c90640 1 Processor -- start 2023-09-23T03:26:17.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.877+0000 7f4993c90640 1 -- start start 2023-09-23T03:26:17.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.878+0000 7f4993c90640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f498c0c9510 0x7f498c0b2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:17.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.878+0000 7f4993c90640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f498c0c9980 0x7f498c1646e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:17.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.878+0000 7f4993c90640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f498c0cae90 0x7f498c164c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:17.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.878+0000 7f4993c90640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498c1f7b40 con 0x7f498c0cae90 2023-09-23T03:26:17.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.878+0000 7f4993c90640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498c1f7860 con 0x7f498c0c9510 2023-09-23T03:26:17.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.878+0000 7f4993c90640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498c0c5720 con 0x7f498c0c9980 2023-09-23T03:26:17.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.879+0000 7f499222d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f498c0cae90 0x7f498c164c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:17.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.879+0000 7f499222d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f498c0cae90 0x7f498c164c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35478/0 (socket says 172.21.15.154:35478) 2023-09-23T03:26:17.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.879+0000 7f499222d640 1 -- 172.21.15.154:0/182067392 learned_addr learned my addr 172.21.15.154:0/182067392 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:17.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.879+0000 7f4991a2c640 1 --2- 172.21.15.154:0/182067392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f498c0c9510 0x7f498c0b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:17.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.879+0000 7f499222d640 1 -- 172.21.15.154:0/182067392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f498c0c9980 msgr2=0x7f498c1646e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:17.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.879+0000 7f499222d640 1 --2- 172.21.15.154:0/182067392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f498c0c9980 0x7f498c1646e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:17.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.879+0000 7f499222d640 1 -- 172.21.15.154:0/182067392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f498c0c9510 msgr2=0x7f498c0b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:17.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.879+0000 7f499222d640 1 --2- 172.21.15.154:0/182067392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f498c0c9510 0x7f498c0b2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:17.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.879+0000 7f499222d640 1 -- 172.21.15.154:0/182067392 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f497c009400 con 0x7f498c0cae90 2023-09-23T03:26:17.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.879+0000 7f499222d640 1 --2- 172.21.15.154:0/182067392 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f498c0cae90 0x7f498c164c20 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f498400cd70 tx=0x7f498400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:17.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.880+0000 7f49827fc640 1 -- 172.21.15.154:0/182067392 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4984003b90 con 0x7f498c0cae90 2023-09-23T03:26:17.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.880+0000 7f49827fc640 1 -- 172.21.15.154:0/182067392 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4984003cf0 con 0x7f498c0cae90 2023-09-23T03:26:17.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.880+0000 7f4993c90640 1 -- 172.21.15.154:0/182067392 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f498c0c5a00 con 0x7f498c0cae90 2023-09-23T03:26:17.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.880+0000 7f49827fc640 1 -- 172.21.15.154:0/182067392 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4984004240 con 0x7f498c0cae90 2023-09-23T03:26:17.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.880+0000 7f4993c90640 1 -- 172.21.15.154:0/182067392 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f498c0b2a70 con 0x7f498c0cae90 2023-09-23T03:26:17.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.881+0000 7f4993c90640 1 -- 172.21.15.154:0/182067392 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f498c0c5580 con 0x7f498c0cae90 2023-09-23T03:26:17.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.882+0000 7f49827fc640 1 -- 172.21.15.154:0/182067392 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f49840043a0 con 0x7f498c0cae90 2023-09-23T03:26:17.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.883+0000 7f49827fc640 1 --2- 172.21.15.154:0/182067392 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f497006af30 0x7f497006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:17.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.883+0000 7f49827fc640 1 -- 172.21.15.154:0/182067392 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f4984012070 con 0x7f498c0cae90 2023-09-23T03:26:17.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.883+0000 7f4991a2c640 1 --2- 172.21.15.154:0/182067392 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f497006af30 0x7f497006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:17.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.884+0000 7f4991a2c640 1 --2- 172.21.15.154:0/182067392 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f497006af30 0x7f497006d3f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f4988000f60 tx=0x7f4988009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:17.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:17.887+0000 7f49827fc640 1 -- 172.21.15.154:0/182067392 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f498c0c5580 con 0x7f498c0cae90 2023-09-23T03:26:18.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.077+0000 7f4993c90640 1 -- 172.21.15.154:0/182067392 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f498c0c5f20 con 0x7f498c0cae90 2023-09-23T03:26:18.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.079+0000 7f49827fc640 1 -- 172.21.15.154:0/182067392 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v108) v1 ==== 100+0+11 (secure 0 0 0) 0x7f498c0c5f20 con 0x7f498c0cae90 2023-09-23T03:26:18.079 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:26:18.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.081+0000 7f4993c90640 1 -- 172.21.15.154:0/182067392 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f497006af30 msgr2=0x7f497006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:18.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.081+0000 7f4993c90640 1 --2- 172.21.15.154:0/182067392 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f497006af30 0x7f497006d3f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f4988000f60 tx=0x7f4988009040 comp rx=0 tx=0).stop 2023-09-23T03:26:18.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.081+0000 7f4993c90640 1 -- 172.21.15.154:0/182067392 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f498c0cae90 msgr2=0x7f498c164c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:18.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.081+0000 7f4993c90640 1 --2- 172.21.15.154:0/182067392 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f498c0cae90 0x7f498c164c20 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f498400cd70 tx=0x7f498400a660 comp rx=0 tx=0).stop 2023-09-23T03:26:18.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.081+0000 7f4993c90640 1 -- 172.21.15.154:0/182067392 shutdown_connections 2023-09-23T03:26:18.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.081+0000 7f4993c90640 1 --2- 172.21.15.154:0/182067392 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f497006af30 0x7f497006d3f0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:18.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.081+0000 7f4993c90640 1 --2- 172.21.15.154:0/182067392 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f498c0cae90 0x7f498c164c20 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:18.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.081+0000 7f4993c90640 1 --2- 172.21.15.154:0/182067392 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f498c0c9980 0x7f498c1646e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:18.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.081+0000 7f4993c90640 1 --2- 172.21.15.154:0/182067392 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f498c0c9510 0x7f498c0b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:18.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.081+0000 7f4993c90640 1 -- 172.21.15.154:0/182067392 >> 172.21.15.154:0/182067392 conn(0x7f498c06c010 msgr2=0x7f498c071730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:18.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.081+0000 7f4993c90640 1 -- 172.21.15.154:0/182067392 shutdown_connections 2023-09-23T03:26:18.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.081+0000 7f4993c90640 1 -- 172.21.15.154:0/182067392 wait complete. 2023-09-23T03:26:18.100 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-09-23T03:26:18.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.319+0000 7f116a525640 1 Processor -- start 2023-09-23T03:26:18.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.319+0000 7f116a525640 1 -- start start 2023-09-23T03:26:18.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.320+0000 7f116a525640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f11640cc9b0 0x7f11640b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:18.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.321+0000 7f116a525640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f11640cce40 0x7f11640b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:18.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.321+0000 7f116a525640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11640ce510 con 0x7f116415abe0 2023-09-23T03:26:18.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.321+0000 7f116a525640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11640cd390 con 0x7f11640cc9b0 2023-09-23T03:26:18.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.321+0000 7f116a525640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11640cd670 con 0x7f11640cce40 2023-09-23T03:26:18.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.321+0000 7f11637fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f11640cc9b0 0x7f11640b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:18.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.321+0000 7f1168ac2640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f11640cce40 0x7f11640b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:18.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.321+0000 7f1168ac2640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f11640cce40 0x7f11640b26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.154:35628/0 (socket says 172.21.15.154:35628) 2023-09-23T03:26:18.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.321+0000 7f1168ac2640 1 -- 172.21.15.154:0/2899515304 learned_addr learned my addr 172.21.15.154:0/2899515304 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:18.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.321+0000 7f11637fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f11640cc9b0 0x7f11640b21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35482/0 (socket says 172.21.15.154:35482) 2023-09-23T03:26:18.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.322+0000 7f1168ac2640 1 -- 172.21.15.154:0/2899515304 >> v2:172.21.15.154:3300/0 conn(0x7f11640cc9b0 msgr2=0x7f11640b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:18.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.322+0000 7f1168ac2640 1 --2- 172.21.15.154:0/2899515304 >> v2:172.21.15.154:3300/0 conn(0x7f11640cc9b0 0x7f11640b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:18.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.322+0000 7f1168ac2640 1 -- 172.21.15.154:0/2899515304 >> v1:172.21.15.190:6789/0 conn(0x7f116415abe0 legacy=0x7f11640cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:18.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.322+0000 7f1168ac2640 1 -- 172.21.15.154:0/2899515304 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11640cd950 con 0x7f11640cce40 2023-09-23T03:26:18.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.324+0000 7f1168ac2640 1 --2- 172.21.15.154:0/2899515304 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11640cce40 0x7f11640b26f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f11640676f0 tx=0x7f115c02b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3728551215318914 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:18.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.324+0000 7f1162ffd640 1 -- 172.21.15.154:0/2899515304 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115c031870 con 0x7f11640cce40 2023-09-23T03:26:18.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.324+0000 7f1162ffd640 1 -- 172.21.15.154:0/2899515304 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f115c0319d0 con 0x7f11640cce40 2023-09-23T03:26:18.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.324+0000 7f1162ffd640 1 -- 172.21.15.154:0/2899515304 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115c031d60 con 0x7f11640cce40 2023-09-23T03:26:18.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.325+0000 7f116a525640 1 -- 172.21.15.154:0/2899515304 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11640cce40 msgr2=0x7f11640b26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:18.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.325+0000 7f116a525640 1 --2- 172.21.15.154:0/2899515304 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11640cce40 0x7f11640b26f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f11640676f0 tx=0x7f115c02b910 comp rx=0 tx=0).stop 2023-09-23T03:26:18.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.325+0000 7f116a525640 1 -- 172.21.15.154:0/2899515304 shutdown_connections 2023-09-23T03:26:18.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.325+0000 7f116a525640 1 --2- 172.21.15.154:0/2899515304 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f11640cce40 0x7f11640b26f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:18.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.325+0000 7f116a525640 1 --2- 172.21.15.154:0/2899515304 >> v2:172.21.15.154:3300/0 conn(0x7f11640cc9b0 0x7f11640b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:18.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.325+0000 7f116a525640 1 -- 172.21.15.154:0/2899515304 >> 172.21.15.154:0/2899515304 conn(0x7f116406fc40 msgr2=0x7f1164070020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:18.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.325+0000 7f116a525640 1 -- 172.21.15.154:0/2899515304 shutdown_connections 2023-09-23T03:26:18.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.325+0000 7f116a525640 1 -- 172.21.15.154:0/2899515304 wait complete. 2023-09-23T03:26:18.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.327+0000 7f116a525640 1 Processor -- start 2023-09-23T03:26:18.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.327+0000 7f116a525640 1 -- start start 2023-09-23T03:26:18.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.328+0000 7f116a525640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11640cc9b0 0x7f1164159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:18.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.328+0000 7f116a525640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11640cce40 0x7f11640d0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:18.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.328+0000 7f116a525640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f116415abe0 0x7f11640d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:18.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.328+0000 7f116a525640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116415a1f0 con 0x7f11640cce40 2023-09-23T03:26:18.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.328+0000 7f116a525640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11640c7050 con 0x7f116415abe0 2023-09-23T03:26:18.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.328+0000 7f116a525640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11640b3480 con 0x7f11640cc9b0 2023-09-23T03:26:18.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.328+0000 7f11637fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11640cce40 0x7f11640d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:18.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.328+0000 7f11637fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11640cce40 0x7f11640d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35484/0 (socket says 172.21.15.154:35484) 2023-09-23T03:26:18.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.328+0000 7f11637fe640 1 -- 172.21.15.154:0/3191045599 learned_addr learned my addr 172.21.15.154:0/3191045599 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:18.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.329+0000 7f1168ac2640 1 --2- 172.21.15.154:0/3191045599 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f116415abe0 0x7f11640d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:18.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.329+0000 7f11637fe640 1 -- 172.21.15.154:0/3191045599 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11640cc9b0 msgr2=0x7f1164159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:18.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.329+0000 7f11637fe640 1 --2- 172.21.15.154:0/3191045599 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11640cc9b0 0x7f1164159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:18.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.329+0000 7f11637fe640 1 -- 172.21.15.154:0/3191045599 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f116415abe0 msgr2=0x7f11640d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:18.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.329+0000 7f11637fe640 1 --2- 172.21.15.154:0/3191045599 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f116415abe0 0x7f11640d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:18.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.329+0000 7f11637fe640 1 -- 172.21.15.154:0/3191045599 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f115c0094d0 con 0x7f11640cce40 2023-09-23T03:26:18.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.329+0000 7f11637fe640 1 --2- 172.21.15.154:0/3191045599 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11640cce40 0x7f11640d0910 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f114c00e9b0 tx=0x7f114c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:18.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.329+0000 7f1160ff9640 1 -- 172.21.15.154:0/3191045599 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114c009670 con 0x7f11640cce40 2023-09-23T03:26:18.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.329+0000 7f1160ff9640 1 -- 172.21.15.154:0/3191045599 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f114c00c9a0 con 0x7f11640cce40 2023-09-23T03:26:18.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.329+0000 7f1160ff9640 1 -- 172.21.15.154:0/3191045599 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114c00cd50 con 0x7f11640cce40 2023-09-23T03:26:18.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.329+0000 7f116a525640 1 -- 172.21.15.154:0/3191045599 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11640b3760 con 0x7f11640cce40 2023-09-23T03:26:18.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.329+0000 7f116a525640 1 -- 172.21.15.154:0/3191045599 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11640b3d20 con 0x7f11640cce40 2023-09-23T03:26:18.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.330+0000 7f116a525640 1 -- 172.21.15.154:0/3191045599 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f116404e920 con 0x7f11640cce40 2023-09-23T03:26:18.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.331+0000 7f1160ff9640 1 -- 172.21.15.154:0/3191045599 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f114c013470 con 0x7f11640cce40 2023-09-23T03:26:18.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.332+0000 7f1160ff9640 1 --2- 172.21.15.154:0/3191045599 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f114406af60 0x7f114406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:18.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.332+0000 7f1160ff9640 1 -- 172.21.15.154:0/3191045599 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f114c089ce0 con 0x7f11640cce40 2023-09-23T03:26:18.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.333+0000 7f1163fff640 1 --2- 172.21.15.154:0/3191045599 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f114406af60 0x7f114406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:18.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.333+0000 7f1163fff640 1 --2- 172.21.15.154:0/3191045599 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f114406af60 0x7f114406d420 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1154001330 tx=0x7f1154007d20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:18.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.337+0000 7f1160ff9640 1 -- 172.21.15.154:0/3191045599 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f114c054a30 con 0x7f11640cce40 2023-09-23T03:26:18.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.526+0000 7f116a525640 1 -- 172.21.15.154:0/3191045599 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7f11640afe70 con 0x7f11640cce40 2023-09-23T03:26:18.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.527+0000 7f1160ff9640 1 -- 172.21.15.154:0/3191045599 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v108) v1 ==== 106+0+11 (secure 0 0 0) 0x7f114c0170b0 con 0x7f11640cce40 2023-09-23T03:26:18.528 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:26:18.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.530+0000 7f116a525640 1 -- 172.21.15.154:0/3191045599 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f114406af60 msgr2=0x7f114406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:18.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.530+0000 7f116a525640 1 --2- 172.21.15.154:0/3191045599 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f114406af60 0x7f114406d420 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1154001330 tx=0x7f1154007d20 comp rx=0 tx=0).stop 2023-09-23T03:26:18.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.530+0000 7f116a525640 1 -- 172.21.15.154:0/3191045599 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11640cce40 msgr2=0x7f11640d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:18.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.530+0000 7f116a525640 1 --2- 172.21.15.154:0/3191045599 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11640cce40 0x7f11640d0910 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f114c00e9b0 tx=0x7f114c00ee70 comp rx=0 tx=0).stop 2023-09-23T03:26:18.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.530+0000 7f116a525640 1 -- 172.21.15.154:0/3191045599 shutdown_connections 2023-09-23T03:26:18.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.530+0000 7f116a525640 1 --2- 172.21.15.154:0/3191045599 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f114406af60 0x7f114406d420 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:18.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.530+0000 7f116a525640 1 --2- 172.21.15.154:0/3191045599 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f116415abe0 0x7f11640d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:18.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.530+0000 7f116a525640 1 --2- 172.21.15.154:0/3191045599 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11640cce40 0x7f11640d0910 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:18.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.530+0000 7f116a525640 1 --2- 172.21.15.154:0/3191045599 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11640cc9b0 0x7f1164159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:18.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.530+0000 7f116a525640 1 -- 172.21.15.154:0/3191045599 >> 172.21.15.154:0/3191045599 conn(0x7f116406fc40 msgr2=0x7f116415afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:18.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.530+0000 7f116a525640 1 -- 172.21.15.154:0/3191045599 shutdown_connections 2023-09-23T03:26:18.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.530+0000 7f116a525640 1 -- 172.21.15.154:0/3191045599 wait complete. 2023-09-23T03:26:18.548 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-23T03:26:18.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.749+0000 7f1bc056c640 1 Processor -- start 2023-09-23T03:26:18.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.749+0000 7f1bc056c640 1 -- start start 2023-09-23T03:26:18.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.751+0000 7f1bc056c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bb80ada30 0x7f1bb80ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:18.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.751+0000 7f1bc056c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1bb80cf130 0x7f1bb815d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:18.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.751+0000 7f1bc056c640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bb806ceb0 con 0x7f1bb80b2fe0 2023-09-23T03:26:18.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.751+0000 7f1bc056c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bb80ade10 con 0x7f1bb80cf130 2023-09-23T03:26:18.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.751+0000 7f1bc056c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb8159c90 con 0x7f1bb80ada30 2023-09-23T03:26:18.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.751+0000 7f1bbeb09640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1bb80cf130 0x7f1bb815d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:18.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.751+0000 7f1bbeb09640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1bb80cf130 0x7f1bb815d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35500/0 (socket says 172.21.15.154:35500) 2023-09-23T03:26:18.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.751+0000 7f1bbeb09640 1 -- 172.21.15.154:0/1203981010 learned_addr learned my addr 172.21.15.154:0/1203981010 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:18.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.752+0000 7f1bbe308640 1 --2- 172.21.15.154:0/1203981010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bb80ada30 0x7f1bb80ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:18.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.752+0000 7f1bbeb09640 1 -- 172.21.15.154:0/1203981010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bb80ada30 msgr2=0x7f1bb80ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:18.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.752+0000 7f1bbeb09640 1 --2- 172.21.15.154:0/1203981010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bb80ada30 0x7f1bb80ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:18.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.752+0000 7f1bbeb09640 1 -- 172.21.15.154:0/1203981010 >> v1:172.21.15.190:6789/0 conn(0x7f1bb80b2fe0 legacy=0x7f1bb80cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:18.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.752+0000 7f1bbeb09640 1 -- 172.21.15.154:0/1203981010 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb806c110 con 0x7f1bb80cf130 2023-09-23T03:26:18.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.752+0000 7f1bbe308640 1 --2- 172.21.15.154:0/1203981010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bb80ada30 0x7f1bb80ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:26:18.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.753+0000 7f1bbeb09640 1 --2- 172.21.15.154:0/1203981010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bb80cf130 0x7f1bb815d380 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0009e50 tx=0x7f1bb002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c222ebf9fe36ae server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:18.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.753+0000 7f1bbd306640 1 -- 172.21.15.154:0/1203981010 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb002ca70 con 0x7f1bb80cf130 2023-09-23T03:26:18.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.753+0000 7f1bbd306640 1 -- 172.21.15.154:0/1203981010 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb002cbd0 con 0x7f1bb80cf130 2023-09-23T03:26:18.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.753+0000 7f1bbd306640 1 -- 172.21.15.154:0/1203981010 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb00343f0 con 0x7f1bb80cf130 2023-09-23T03:26:18.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.754+0000 7f1bc056c640 1 -- 172.21.15.154:0/1203981010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bb80cf130 msgr2=0x7f1bb815d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:18.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.754+0000 7f1bc056c640 1 --2- 172.21.15.154:0/1203981010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bb80cf130 0x7f1bb815d380 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0009e50 tx=0x7f1bb002bed0 comp rx=0 tx=0).stop 2023-09-23T03:26:18.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.754+0000 7f1bc056c640 1 -- 172.21.15.154:0/1203981010 shutdown_connections 2023-09-23T03:26:18.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.754+0000 7f1bc056c640 1 --2- 172.21.15.154:0/1203981010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bb80cf130 0x7f1bb815d380 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:18.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.754+0000 7f1bc056c640 1 --2- 172.21.15.154:0/1203981010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bb80ada30 0x7f1bb80ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:18.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.754+0000 7f1bc056c640 1 -- 172.21.15.154:0/1203981010 >> 172.21.15.154:0/1203981010 conn(0x7f1bb806a0f0 msgr2=0x7f1bb806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:18.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.754+0000 7f1bc056c640 1 -- 172.21.15.154:0/1203981010 shutdown_connections 2023-09-23T03:26:18.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.754+0000 7f1bc056c640 1 -- 172.21.15.154:0/1203981010 wait complete. 2023-09-23T03:26:18.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.755+0000 7f1bc056c640 1 Processor -- start 2023-09-23T03:26:18.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.755+0000 7f1bc056c640 1 -- start start 2023-09-23T03:26:18.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.756+0000 7f1bc056c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bb80ada30 0x7f1bb80c4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:18.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.756+0000 7f1bc056c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bb80b2fe0 0x7f1bb81ef580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:18.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.756+0000 7f1bc056c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bb80cf130 0x7f1bb81efac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:18.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.756+0000 7f1bc056c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb81f0000 con 0x7f1bb80b2fe0 2023-09-23T03:26:18.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.756+0000 7f1bc056c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb815b300 con 0x7f1bb80ada30 2023-09-23T03:26:18.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.756+0000 7f1bc056c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb815b5e0 con 0x7f1bb80cf130 2023-09-23T03:26:18.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.757+0000 7f1bbdb07640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bb80b2fe0 0x7f1bb81ef580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:18.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.757+0000 7f1bbdb07640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bb80b2fe0 0x7f1bb81ef580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:35506/0 (socket says 172.21.15.154:35506) 2023-09-23T03:26:18.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.757+0000 7f1bbdb07640 1 -- 172.21.15.154:0/3001773827 learned_addr learned my addr 172.21.15.154:0/3001773827 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:18.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.757+0000 7f1bbdb07640 1 -- 172.21.15.154:0/3001773827 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bb80cf130 msgr2=0x7f1bb81efac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:18.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.757+0000 7f1bbdb07640 1 --2- 172.21.15.154:0/3001773827 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bb80cf130 0x7f1bb81efac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:18.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.757+0000 7f1bbdb07640 1 -- 172.21.15.154:0/3001773827 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bb80ada30 msgr2=0x7f1bb80c4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:18.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.757+0000 7f1bbdb07640 1 --2- 172.21.15.154:0/3001773827 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bb80ada30 0x7f1bb80c4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:18.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.757+0000 7f1bbdb07640 1 -- 172.21.15.154:0/3001773827 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb0009b30 con 0x7f1bb80b2fe0 2023-09-23T03:26:18.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.757+0000 7f1bbdb07640 1 --2- 172.21.15.154:0/3001773827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bb80b2fe0 0x7f1bb81ef580 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f1bac00b3f0 tx=0x7f1bac00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:18.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.757+0000 7f1baaffd640 1 -- 172.21.15.154:0/3001773827 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bac011830 con 0x7f1bb80b2fe0 2023-09-23T03:26:18.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.758+0000 7f1baaffd640 1 -- 172.21.15.154:0/3001773827 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bac011990 con 0x7f1bb80b2fe0 2023-09-23T03:26:18.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.758+0000 7f1baaffd640 1 -- 172.21.15.154:0/3001773827 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bac011d40 con 0x7f1bb80b2fe0 2023-09-23T03:26:18.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.758+0000 7f1bc056c640 1 -- 172.21.15.154:0/3001773827 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bb815b8c0 con 0x7f1bb80b2fe0 2023-09-23T03:26:18.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.758+0000 7f1bc056c640 1 -- 172.21.15.154:0/3001773827 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb8099910 con 0x7f1bb80b2fe0 2023-09-23T03:26:18.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.759+0000 7f1bc056c640 1 -- 172.21.15.154:0/3001773827 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bb8073eb0 con 0x7f1bb80b2fe0 2023-09-23T03:26:18.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.760+0000 7f1baaffd640 1 -- 172.21.15.154:0/3001773827 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f1bac01a400 con 0x7f1bb80b2fe0 2023-09-23T03:26:18.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.761+0000 7f1baaffd640 1 --2- 172.21.15.154:0/3001773827 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1b9c06af30 0x7f1b9c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:18.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.761+0000 7f1baaffd640 1 -- 172.21.15.154:0/3001773827 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f1bac089940 con 0x7f1bb80b2fe0 2023-09-23T03:26:18.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.761+0000 7f1bbe308640 1 --2- 172.21.15.154:0/3001773827 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1b9c06af30 0x7f1b9c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:18.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.762+0000 7f1bbe308640 1 --2- 172.21.15.154:0/3001773827 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1b9c06af30 0x7f1b9c06d3f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1bb400a870 tx=0x7f1bb4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:18.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.765+0000 7f1baaffd640 1 -- 172.21.15.154:0/3001773827 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f1bac054600 con 0x7f1bb80b2fe0 2023-09-23T03:26:18.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.950+0000 7f1bc056c640 1 -- 172.21.15.154:0/3001773827 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1bb8099b30 con 0x7f1bb80b2fe0 2023-09-23T03:26:18.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.951+0000 7f1baaffd640 1 -- 172.21.15.154:0/3001773827 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v108) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1bac057c20 con 0x7f1bb80b2fe0 2023-09-23T03:26:18.951 INFO:teuthology.orchestra.run.smithi154.stderr:listed 0 entries 2023-09-23T03:26:18.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.953+0000 7f1bc056c640 1 -- 172.21.15.154:0/3001773827 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1b9c06af30 msgr2=0x7f1b9c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:18.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.953+0000 7f1bc056c640 1 --2- 172.21.15.154:0/3001773827 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1b9c06af30 0x7f1b9c06d3f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1bb400a870 tx=0x7f1bb4008040 comp rx=0 tx=0).stop 2023-09-23T03:26:18.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.953+0000 7f1bc056c640 1 -- 172.21.15.154:0/3001773827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bb80b2fe0 msgr2=0x7f1bb81ef580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:18.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.953+0000 7f1bc056c640 1 --2- 172.21.15.154:0/3001773827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bb80b2fe0 0x7f1bb81ef580 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f1bac00b3f0 tx=0x7f1bac00b8b0 comp rx=0 tx=0).stop 2023-09-23T03:26:18.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.954+0000 7f1bc056c640 1 -- 172.21.15.154:0/3001773827 shutdown_connections 2023-09-23T03:26:18.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.954+0000 7f1bc056c640 1 --2- 172.21.15.154:0/3001773827 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1b9c06af30 0x7f1b9c06d3f0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:18.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.954+0000 7f1bc056c640 1 --2- 172.21.15.154:0/3001773827 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1bb80cf130 0x7f1bb81efac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:18.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.954+0000 7f1bc056c640 1 --2- 172.21.15.154:0/3001773827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1bb80b2fe0 0x7f1bb81ef580 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:18.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.954+0000 7f1bc056c640 1 --2- 172.21.15.154:0/3001773827 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1bb80ada30 0x7f1bb80c4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:18.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.954+0000 7f1bc056c640 1 -- 172.21.15.154:0/3001773827 >> 172.21.15.154:0/3001773827 conn(0x7f1bb806a0f0 msgr2=0x7f1bb8158580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:18.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.954+0000 7f1bc056c640 1 -- 172.21.15.154:0/3001773827 shutdown_connections 2023-09-23T03:26:18.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:18.954+0000 7f1bc056c640 1 -- 172.21.15.154:0/3001773827 wait complete. 2023-09-23T03:26:18.971 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi154.front.sepia.ceph.com... 2023-09-23T03:26:18.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:26:18.972 DEBUG:teuthology.orchestra.run.smithi154:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-09-23T03:26:19.054 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:ceph-fuse[121281]: fuse finished with error 0 and tester_r 0 2023-09-23T03:26:19.061 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:26:25.162 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2023-09-23T03:26:25.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:26:25.163 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-09-23T03:26:25.226 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-09-23T03:26:25.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.446+0000 7f38abd05640 1 Processor -- start 2023-09-23T03:26:25.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.447+0000 7f38abd05640 1 -- start start 2023-09-23T03:26:25.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.448+0000 7f38abd05640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f38a415d010 0x7f38a406c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:25.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.448+0000 7f38abd05640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f38a40ca900 0x7f38a40cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:25.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.448+0000 7f38abd05640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a415d3f0 con 0x7f38a40cb070 2023-09-23T03:26:25.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.448+0000 7f38abd05640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38a406ade0 con 0x7f38a40ca900 2023-09-23T03:26:25.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.448+0000 7f38abd05640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38a406b4e0 con 0x7f38a415d010 2023-09-23T03:26:25.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.448+0000 7f38aa2a2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f38a40ca900 0x7f38a40cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:25.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.449+0000 7f38aa2a2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f38a40ca900 0x7f38a40cc9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45706/0 (socket says 172.21.15.154:45706) 2023-09-23T03:26:25.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.449+0000 7f38aa2a2640 1 -- 172.21.15.154:0/1473410639 learned_addr learned my addr 172.21.15.154:0/1473410639 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:25.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.449+0000 7f38a9aa1640 1 --2- 172.21.15.154:0/1473410639 >> v2:172.21.15.190:3300/0 conn(0x7f38a415d010 0x7f38a406c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:25.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.449+0000 7f38aa2a2640 1 -- 172.21.15.154:0/1473410639 >> v2:172.21.15.190:3300/0 conn(0x7f38a415d010 msgr2=0x7f38a406c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:25.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.449+0000 7f38aa2a2640 1 --2- 172.21.15.154:0/1473410639 >> v2:172.21.15.190:3300/0 conn(0x7f38a415d010 0x7f38a406c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:25.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.449+0000 7f38aa2a2640 1 -- 172.21.15.154:0/1473410639 >> v1:172.21.15.190:6789/0 conn(0x7f38a40cb070 legacy=0x7f38a40ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:25.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.449+0000 7f38aa2a2640 1 -- 172.21.15.154:0/1473410639 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a40c9650 con 0x7f38a40ca900 2023-09-23T03:26:25.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.450+0000 7f38aa2a2640 1 --2- 172.21.15.154:0/1473410639 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38a40ca900 0x7f38a40cc9c0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f38a40668a0 tx=0x7f389c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bbd0b49a16d3408 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:25.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.450+0000 7f38a8a9f640 1 -- 172.21.15.154:0/1473410639 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c030860 con 0x7f38a40ca900 2023-09-23T03:26:25.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.451+0000 7f38a8a9f640 1 -- 172.21.15.154:0/1473410639 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389c0309c0 con 0x7f38a40ca900 2023-09-23T03:26:25.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.451+0000 7f38a8a9f640 1 -- 172.21.15.154:0/1473410639 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c030d50 con 0x7f38a40ca900 2023-09-23T03:26:25.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.451+0000 7f38abd05640 1 -- 172.21.15.154:0/1473410639 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38a40ca900 msgr2=0x7f38a40cc9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:25.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.451+0000 7f38abd05640 1 --2- 172.21.15.154:0/1473410639 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38a40ca900 0x7f38a40cc9c0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f38a40668a0 tx=0x7f389c02b9a0 comp rx=0 tx=0).stop 2023-09-23T03:26:25.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.451+0000 7f38abd05640 1 -- 172.21.15.154:0/1473410639 shutdown_connections 2023-09-23T03:26:25.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.451+0000 7f38abd05640 1 --2- 172.21.15.154:0/1473410639 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38a40ca900 0x7f38a40cc9c0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:25.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.451+0000 7f38abd05640 1 --2- 172.21.15.154:0/1473410639 >> v2:172.21.15.190:3300/0 conn(0x7f38a415d010 0x7f38a406c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:25.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.451+0000 7f38abd05640 1 -- 172.21.15.154:0/1473410639 >> 172.21.15.154:0/1473410639 conn(0x7f38a40b08e0 msgr2=0x7f38a40b0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:25.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.451+0000 7f38abd05640 1 -- 172.21.15.154:0/1473410639 shutdown_connections 2023-09-23T03:26:25.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.451+0000 7f38abd05640 1 -- 172.21.15.154:0/1473410639 wait complete. 2023-09-23T03:26:25.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.453+0000 7f38abd05640 1 Processor -- start 2023-09-23T03:26:25.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.453+0000 7f38abd05640 1 -- start start 2023-09-23T03:26:25.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.455+0000 7f38abd05640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38a40ca900 0x7f38a4159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:25.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.455+0000 7f38abd05640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38a40cb070 0x7f38a409e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:25.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.455+0000 7f38abd05640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38a415d010 0x7f38a41f2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:25.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.455+0000 7f38abd05640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a415a0d0 con 0x7f38a40cb070 2023-09-23T03:26:25.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.455+0000 7f38abd05640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a415a7b0 con 0x7f38a415d010 2023-09-23T03:26:25.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.455+0000 7f38abd05640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a4157da0 con 0x7f38a40ca900 2023-09-23T03:26:25.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.455+0000 7f38a92a0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38a40cb070 0x7f38a409e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:25.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.455+0000 7f38a92a0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38a40cb070 0x7f38a409e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45710/0 (socket says 172.21.15.154:45710) 2023-09-23T03:26:25.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.455+0000 7f38a92a0640 1 -- 172.21.15.154:0/157953559 learned_addr learned my addr 172.21.15.154:0/157953559 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:25.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.456+0000 7f38a92a0640 1 -- 172.21.15.154:0/157953559 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38a40ca900 msgr2=0x7f38a4159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:25.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.456+0000 7f38a92a0640 1 --2- 172.21.15.154:0/157953559 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38a40ca900 0x7f38a4159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:25.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.456+0000 7f38aa2a2640 1 --2- 172.21.15.154:0/157953559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38a415d010 0x7f38a41f2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:25.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.456+0000 7f38a92a0640 1 -- 172.21.15.154:0/157953559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38a415d010 msgr2=0x7f38a41f2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:25.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.456+0000 7f38a92a0640 1 --2- 172.21.15.154:0/157953559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38a415d010 0x7f38a41f2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:25.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.456+0000 7f38a92a0640 1 -- 172.21.15.154:0/157953559 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f389c0094d0 con 0x7f38a40cb070 2023-09-23T03:26:25.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.456+0000 7f38a92a0640 1 --2- 172.21.15.154:0/157953559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38a40cb070 0x7f38a409e9d0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f389400c400 tx=0x7f389400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:25.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.456+0000 7f389a7fc640 1 -- 172.21.15.154:0/157953559 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3894011790 con 0x7f38a40cb070 2023-09-23T03:26:25.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.456+0000 7f389a7fc640 1 -- 172.21.15.154:0/157953559 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3894011930 con 0x7f38a40cb070 2023-09-23T03:26:25.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.456+0000 7f389a7fc640 1 -- 172.21.15.154:0/157953559 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3894011ce0 con 0x7f38a40cb070 2023-09-23T03:26:25.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.456+0000 7f38abd05640 1 -- 172.21.15.154:0/157953559 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a415aa20 con 0x7f38a40cb070 2023-09-23T03:26:25.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.456+0000 7f38abd05640 1 -- 172.21.15.154:0/157953559 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a415afe0 con 0x7f38a40cb070 2023-09-23T03:26:25.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.457+0000 7f38abd05640 1 -- 172.21.15.154:0/157953559 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a4073eb0 con 0x7f38a40cb070 2023-09-23T03:26:25.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.459+0000 7f389a7fc640 1 -- 172.21.15.154:0/157953559 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f389401c460 con 0x7f38a40cb070 2023-09-23T03:26:25.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.460+0000 7f389a7fc640 1 --2- 172.21.15.154:0/157953559 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f388806af30 0x7f388806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:25.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.460+0000 7f389a7fc640 1 -- 172.21.15.154:0/157953559 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6433+0+0 (secure 0 0 0) 0x7f38940899e0 con 0x7f38a40cb070 2023-09-23T03:26:25.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.460+0000 7f38a9aa1640 1 --2- 172.21.15.154:0/157953559 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f388806af30 0x7f388806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:25.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.461+0000 7f38a9aa1640 1 --2- 172.21.15.154:0/157953559 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f388806af30 0x7f388806d3f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f38a00070b0 tx=0x7f38a0007040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:25.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.464+0000 7f389a7fc640 1 -- 172.21.15.154:0/157953559 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f3894054730 con 0x7f38a40cb070 2023-09-23T03:26:25.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.687+0000 7f38abd05640 1 -- 172.21.15.154:0/157953559 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f38a4071a20 con 0x7f38a40cb070 2023-09-23T03:26:25.689 INFO:tasks.ceph.mon.a.smithi154.stderr:2023-09-23T03:26:25.688+0000 7f3df35a6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-23T03:26:25.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.692+0000 7f389a7fc640 1 -- 172.21.15.154:0/157953559 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v86) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3894057d50 con 0x7f38a40cb070 2023-09-23T03:26:25.693 INFO:teuthology.orchestra.run.smithi154.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-09-23T03:26:25.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.695+0000 7f38abd05640 1 -- 172.21.15.154:0/157953559 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f388806af30 msgr2=0x7f388806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:25.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.695+0000 7f38abd05640 1 --2- 172.21.15.154:0/157953559 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f388806af30 0x7f388806d3f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f38a00070b0 tx=0x7f38a0007040 comp rx=0 tx=0).stop 2023-09-23T03:26:25.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.695+0000 7f38abd05640 1 -- 172.21.15.154:0/157953559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38a40cb070 msgr2=0x7f38a409e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:25.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.695+0000 7f38abd05640 1 --2- 172.21.15.154:0/157953559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38a40cb070 0x7f38a409e9d0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f389400c400 tx=0x7f389400c8c0 comp rx=0 tx=0).stop 2023-09-23T03:26:25.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.695+0000 7f38abd05640 1 -- 172.21.15.154:0/157953559 shutdown_connections 2023-09-23T03:26:25.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.695+0000 7f38abd05640 1 --2- 172.21.15.154:0/157953559 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f388806af30 0x7f388806d3f0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:25.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.695+0000 7f38abd05640 1 --2- 172.21.15.154:0/157953559 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f38a415d010 0x7f38a41f2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:25.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.695+0000 7f38abd05640 1 --2- 172.21.15.154:0/157953559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f38a40cb070 0x7f38a409e9d0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:25.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.695+0000 7f38abd05640 1 --2- 172.21.15.154:0/157953559 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f38a40ca900 0x7f38a4159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:25.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.695+0000 7f38abd05640 1 -- 172.21.15.154:0/157953559 >> 172.21.15.154:0/157953559 conn(0x7f38a40b08e0 msgr2=0x7f38a415d740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:25.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.695+0000 7f38abd05640 1 -- 172.21.15.154:0/157953559 shutdown_connections 2023-09-23T03:26:25.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:25.695+0000 7f38abd05640 1 -- 172.21.15.154:0/157953559 wait complete. 2023-09-23T03:26:25.716 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-23T03:26:25.722 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-23T03:26:25.809 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-09-23T03:26:25.880 INFO:teuthology.orchestra.run.smithi154.stdout:[global] 2023-09-23T03:26:25.880 INFO:teuthology.orchestra.run.smithi154.stdout: chdir = "" 2023-09-23T03:26:25.880 INFO:teuthology.orchestra.run.smithi154.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-23T03:26:25.880 INFO:teuthology.orchestra.run.smithi154.stdout: auth supported = cephx 2023-09-23T03:26:25.880 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:25.880 INFO:teuthology.orchestra.run.smithi154.stdout: filestore xattr use omap = true 2023-09-23T03:26:25.880 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:25.880 INFO:teuthology.orchestra.run.smithi154.stdout: mon clock drift allowed = 1.000 2023-09-23T03:26:25.881 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:25.881 INFO:teuthology.orchestra.run.smithi154.stdout: osd crush chooseleaf type = 0 2023-09-23T03:26:25.881 INFO:teuthology.orchestra.run.smithi154.stdout: auth debug = true 2023-09-23T03:26:25.881 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:25.881 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on old message = true 2023-09-23T03:26:25.881 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on bug = true 2023-09-23T03:26:25.881 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:25.881 INFO:teuthology.orchestra.run.smithi154.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-23T03:26:25.881 INFO:teuthology.orchestra.run.smithi154.stdout: mon pg warn max object skew = 0 2023-09-23T03:26:25.881 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:25.881 INFO:teuthology.orchestra.run.smithi154.stdout: # disable pg_autoscaler by default for new pools 2023-09-23T03:26:25.881 INFO:teuthology.orchestra.run.smithi154.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-23T03:26:25.882 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:25.882 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default size = 2 2023-09-23T03:26:25.882 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:25.882 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow primary affinity = true 2023-09-23T03:26:25.882 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow pg remap = true 2023-09-23T03:26:25.882 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on legacy crush tunables = false 2023-09-23T03:26:25.882 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on crush straw calc version zero = false 2023-09-23T03:26:25.882 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on no sortbitwise = false 2023-09-23T03:26:25.882 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on osd down out interval zero = false 2023-09-23T03:26:25.882 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on too few osds = false 2023-09-23T03:26:25.882 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-23T03:26:25.883 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-23T03:26:25.883 INFO:teuthology.orchestra.run.smithi154.stdout: mon_allow_pool_size_one = true 2023-09-23T03:26:25.883 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:25.883 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-23T03:26:25.883 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:25.883 INFO:teuthology.orchestra.run.smithi154.stdout: osd default data pool replay window = 5 2023-09-23T03:26:25.883 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:25.883 INFO:teuthology.orchestra.run.smithi154.stdout: mon allow pool delete = true 2023-09-23T03:26:25.883 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:25.883 INFO:teuthology.orchestra.run.smithi154.stdout: mon cluster log file level = debug 2023-09-23T03:26:25.883 INFO:teuthology.orchestra.run.smithi154.stdout: debug asserts on shutdown = true 2023-09-23T03:26:25.883 INFO:teuthology.orchestra.run.smithi154.stdout: mon health detail to clog = false 2023-09-23T03:26:25.884 INFO:teuthology.orchestra.run.smithi154.stdout: mon host = "172.21.15.154,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]" 2023-09-23T03:26:25.884 INFO:teuthology.orchestra.run.smithi154.stdout: fsid = 8c8039d3-6c84-43db-8808-85e0dbdd47c8 2023-09-23T03:26:25.884 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:25.884 INFO:teuthology.orchestra.run.smithi154.stdout:[osd] 2023-09-23T03:26:25.884 INFO:teuthology.orchestra.run.smithi154.stdout: osd journal size = 100 2023-09-23T03:26:25.884 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:25.884 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub load threshold = 5.0 2023-09-23T03:26:25.884 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub max interval = 600 2023-09-23T03:26:25.884 INFO:teuthology.orchestra.run.smithi154.stdout: osd mclock profile = high_recovery_ops 2023-09-23T03:26:25.884 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:25.884 INFO:teuthology.orchestra.run.smithi154.stdout: osd recover clone overlap = true 2023-09-23T03:26:25.884 INFO:teuthology.orchestra.run.smithi154.stdout: osd recovery max chunk = 1048576 2023-09-23T03:26:25.885 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:25.885 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug shutdown = true 2023-09-23T03:26:25.885 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug op order = true 2023-09-23T03:26:25.885 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug verify stray on activate = true 2023-09-23T03:26:25.885 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:25.885 INFO:teuthology.orchestra.run.smithi154.stdout: osd open classes on start = true 2023-09-23T03:26:25.885 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug pg log writeout = true 2023-09-23T03:26:25.885 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:25.885 INFO:teuthology.orchestra.run.smithi154.stdout: osd deep scrub update digest min age = 30 2023-09-23T03:26:25.885 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:25.885 INFO:teuthology.orchestra.run.smithi154.stdout: osd map max advance = 10 2023-09-23T03:26:25.885 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:25.886 INFO:teuthology.orchestra.run.smithi154.stdout: journal zero on create = true 2023-09-23T03:26:25.886 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:25.886 INFO:teuthology.orchestra.run.smithi154.stdout: filestore ondisk finisher threads = 3 2023-09-23T03:26:25.886 INFO:teuthology.orchestra.run.smithi154.stdout: filestore apply finisher threads = 3 2023-09-23T03:26:25.886 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:25.886 INFO:teuthology.orchestra.run.smithi154.stdout: bdev debug aio = true 2023-09-23T03:26:25.886 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug misdirected ops = true 2023-09-23T03:26:25.886 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore block size = 96636764160 2023-09-23T03:26:25.886 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore fsck on mount = True 2023-09-23T03:26:25.886 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluefs = 20 2023-09-23T03:26:25.886 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluestore = 20 2023-09-23T03:26:25.887 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:26:25.887 INFO:teuthology.orchestra.run.smithi154.stdout: debug osd = 20 2023-09-23T03:26:25.887 INFO:teuthology.orchestra.run.smithi154.stdout: debug rocksdb = 10 2023-09-23T03:26:25.887 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd backfillfull_ratio = 0.85 2023-09-23T03:26:25.887 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd full ratio = 0.9 2023-09-23T03:26:25.887 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd nearfull ratio = 0.8 2023-09-23T03:26:25.887 INFO:teuthology.orchestra.run.smithi154.stdout: osd failsafe full ratio = 0.95 2023-09-23T03:26:25.887 INFO:teuthology.orchestra.run.smithi154.stdout: osd objectstore = bluestore 2023-09-23T03:26:25.887 INFO:teuthology.orchestra.run.smithi154.stdout: osd op complaint time = 180 2023-09-23T03:26:25.887 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:25.887 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr] 2023-09-23T03:26:25.887 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:26:25.888 INFO:teuthology.orchestra.run.smithi154.stdout: debug mgr = 20 2023-09-23T03:26:25.888 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2023-09-23T03:26:25.888 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2023-09-23T03:26:25.888 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:26:25.888 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:26:25.888 INFO:teuthology.orchestra.run.smithi154.stdout: mgr/telemetry/nag = false 2023-09-23T03:26:25.888 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:25.888 INFO:teuthology.orchestra.run.smithi154.stdout:[mon] 2023-09-23T03:26:25.888 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:26:25.888 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2023-09-23T03:26:25.888 INFO:teuthology.orchestra.run.smithi154.stdout: debug paxos = 20 2023-09-23T03:26:25.888 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2023-09-23T03:26:25.888 INFO:teuthology.orchestra.run.smithi154.stdout: mon data avail warn = 5 2023-09-23T03:26:25.889 INFO:teuthology.orchestra.run.smithi154.stdout: mon mgr mkfs grace = 240 2023-09-23T03:26:25.889 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:26:25.889 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd reporter subtree level = osd 2023-09-23T03:26:25.889 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd prime pg temp = true 2023-09-23T03:26:25.889 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:26:25.889 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:25.889 INFO:teuthology.orchestra.run.smithi154.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-23T03:26:25.889 INFO:teuthology.orchestra.run.smithi154.stdout: auth mon ticket ttl = 660 # 11m 2023-09-23T03:26:25.889 INFO:teuthology.orchestra.run.smithi154.stdout: auth service ticket ttl = 240 # 4m 2023-09-23T03:26:25.889 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:25.889 INFO:teuthology.orchestra.run.smithi154.stdout: # don't complain about insecure global_id in the test suite 2023-09-23T03:26:25.889 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-23T03:26:25.890 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-23T03:26:25.890 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:25.890 INFO:teuthology.orchestra.run.smithi154.stdout: # 1m isn't quite enough 2023-09-23T03:26:25.890 INFO:teuthology.orchestra.run.smithi154.stdout: mon_down_mkfs_grace = 2m 2023-09-23T03:26:25.890 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:25.890 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_filestore_osds = false 2023-09-23T03:26:25.890 INFO:teuthology.orchestra.run.smithi154.stdout: mon op complaint time = 120 2023-09-23T03:26:25.890 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:25.890 INFO:teuthology.orchestra.run.smithi154.stdout:[client] 2023-09-23T03:26:25.890 INFO:teuthology.orchestra.run.smithi154.stdout: rgw cache enabled = true 2023-09-23T03:26:25.890 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable ops log = true 2023-09-23T03:26:25.891 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable usage log = true 2023-09-23T03:26:25.891 INFO:teuthology.orchestra.run.smithi154.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-23T03:26:25.891 INFO:teuthology.orchestra.run.smithi154.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-23T03:26:25.891 INFO:teuthology.orchestra.run.smithi154.stdout: client mount timeout = 600 2023-09-23T03:26:25.891 INFO:teuthology.orchestra.run.smithi154.stdout: debug client = 20 2023-09-23T03:26:25.891 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:26:25.891 INFO:teuthology.orchestra.run.smithi154.stdout: pid file = "" 2023-09-23T03:26:25.891 INFO:teuthology.orchestra.run.smithi154.stdout: rados mon op timeout = 900 2023-09-23T03:26:25.891 INFO:teuthology.orchestra.run.smithi154.stdout: rados osd op timeout = 900 2023-09-23T03:26:25.891 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.a] 2023-09-23T03:26:25.891 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.b] 2023-09-23T03:26:25.891 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.c] 2023-09-23T03:26:25.892 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.a] 2023-09-23T03:26:25.892 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.c] 2023-09-23T03:26:25.892 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.b] 2023-09-23T03:26:25.892 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.d] 2023-09-23T03:26:25.892 INFO:teuthology.orchestra.run.smithi154.stdout:[mds] 2023-09-23T03:26:25.892 INFO:teuthology.orchestra.run.smithi154.stdout: debug mds = 20 2023-09-23T03:26:25.892 INFO:teuthology.orchestra.run.smithi154.stdout: debug mds balancer = 20 2023-09-23T03:26:25.892 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:26:25.892 INFO:teuthology.orchestra.run.smithi154.stdout: mds debug frag = True 2023-09-23T03:26:25.892 INFO:teuthology.orchestra.run.smithi154.stdout: mds debug scatterstat = False 2023-09-23T03:26:25.892 INFO:teuthology.orchestra.run.smithi154.stdout: mds op complaint time = 180 2023-09-23T03:26:25.892 INFO:teuthology.orchestra.run.smithi154.stdout: mds verify scatter = False 2023-09-23T03:26:25.892 INFO:teuthology.orchestra.run.smithi154.stdout: osd op complaint time = 180 2023-09-23T03:26:25.893 INFO:teuthology.orchestra.run.smithi154.stdout: rados mon op timeout = 900 2023-09-23T03:26:25.893 INFO:teuthology.orchestra.run.smithi154.stdout: rados osd op timeout = 900 2023-09-23T03:26:25.893 INFO:teuthology.orchestra.run.smithi190.stdout:[global] 2023-09-23T03:26:25.893 INFO:teuthology.orchestra.run.smithi190.stdout: chdir = "" 2023-09-23T03:26:25.893 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-23T03:26:25.893 INFO:teuthology.orchestra.run.smithi190.stdout: auth supported = cephx 2023-09-23T03:26:25.893 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:25.894 INFO:teuthology.orchestra.run.smithi190.stdout: filestore xattr use omap = true 2023-09-23T03:26:25.894 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:25.894 INFO:teuthology.orchestra.run.smithi190.stdout: mon clock drift allowed = 1.000 2023-09-23T03:26:25.894 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:25.894 INFO:teuthology.orchestra.run.smithi190.stdout: osd crush chooseleaf type = 0 2023-09-23T03:26:25.894 INFO:teuthology.orchestra.run.smithi190.stdout: auth debug = true 2023-09-23T03:26:25.894 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:25.894 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on old message = true 2023-09-23T03:26:25.894 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on bug = true 2023-09-23T03:26:25.894 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:25.894 INFO:teuthology.orchestra.run.smithi190.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-23T03:26:25.895 INFO:teuthology.orchestra.run.smithi190.stdout: mon pg warn max object skew = 0 2023-09-23T03:26:25.895 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:25.895 INFO:teuthology.orchestra.run.smithi190.stdout: # disable pg_autoscaler by default for new pools 2023-09-23T03:26:25.895 INFO:teuthology.orchestra.run.smithi190.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-23T03:26:25.895 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:25.895 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default size = 2 2023-09-23T03:26:25.895 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:25.895 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow primary affinity = true 2023-09-23T03:26:25.895 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow pg remap = true 2023-09-23T03:26:25.895 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on legacy crush tunables = false 2023-09-23T03:26:25.895 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on crush straw calc version zero = false 2023-09-23T03:26:25.895 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on no sortbitwise = false 2023-09-23T03:26:25.896 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on osd down out interval zero = false 2023-09-23T03:26:25.896 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on too few osds = false 2023-09-23T03:26:25.896 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-23T03:26:25.896 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-23T03:26:25.896 INFO:teuthology.orchestra.run.smithi190.stdout: mon_allow_pool_size_one = true 2023-09-23T03:26:25.896 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:25.896 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-23T03:26:25.896 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:25.896 INFO:teuthology.orchestra.run.smithi190.stdout: osd default data pool replay window = 5 2023-09-23T03:26:25.896 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:25.896 INFO:teuthology.orchestra.run.smithi190.stdout: mon allow pool delete = true 2023-09-23T03:26:25.896 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:25.897 INFO:teuthology.orchestra.run.smithi190.stdout: mon cluster log file level = debug 2023-09-23T03:26:25.897 INFO:teuthology.orchestra.run.smithi190.stdout: debug asserts on shutdown = true 2023-09-23T03:26:25.897 INFO:teuthology.orchestra.run.smithi190.stdout: mon health detail to clog = false 2023-09-23T03:26:25.897 INFO:teuthology.orchestra.run.smithi190.stdout: mon host = "172.21.15.154,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]" 2023-09-23T03:26:25.897 INFO:teuthology.orchestra.run.smithi190.stdout: fsid = 8c8039d3-6c84-43db-8808-85e0dbdd47c8 2023-09-23T03:26:25.897 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:25.897 INFO:teuthology.orchestra.run.smithi190.stdout:[osd] 2023-09-23T03:26:25.897 INFO:teuthology.orchestra.run.smithi190.stdout: osd journal size = 100 2023-09-23T03:26:25.897 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:25.897 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub load threshold = 5.0 2023-09-23T03:26:25.897 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub max interval = 600 2023-09-23T03:26:25.897 INFO:teuthology.orchestra.run.smithi190.stdout: osd mclock profile = high_recovery_ops 2023-09-23T03:26:25.898 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:25.898 INFO:teuthology.orchestra.run.smithi190.stdout: osd recover clone overlap = true 2023-09-23T03:26:25.898 INFO:teuthology.orchestra.run.smithi190.stdout: osd recovery max chunk = 1048576 2023-09-23T03:26:25.898 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:25.898 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug shutdown = true 2023-09-23T03:26:25.898 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug op order = true 2023-09-23T03:26:25.898 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug verify stray on activate = true 2023-09-23T03:26:25.898 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:25.898 INFO:teuthology.orchestra.run.smithi190.stdout: osd open classes on start = true 2023-09-23T03:26:25.898 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug pg log writeout = true 2023-09-23T03:26:25.898 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:25.898 INFO:teuthology.orchestra.run.smithi190.stdout: osd deep scrub update digest min age = 30 2023-09-23T03:26:25.899 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:25.899 INFO:teuthology.orchestra.run.smithi190.stdout: osd map max advance = 10 2023-09-23T03:26:25.899 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:25.899 INFO:teuthology.orchestra.run.smithi190.stdout: journal zero on create = true 2023-09-23T03:26:25.899 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:25.899 INFO:teuthology.orchestra.run.smithi190.stdout: filestore ondisk finisher threads = 3 2023-09-23T03:26:25.899 INFO:teuthology.orchestra.run.smithi190.stdout: filestore apply finisher threads = 3 2023-09-23T03:26:25.899 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:25.899 INFO:teuthology.orchestra.run.smithi190.stdout: bdev debug aio = true 2023-09-23T03:26:25.899 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug misdirected ops = true 2023-09-23T03:26:25.899 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore block size = 96636764160 2023-09-23T03:26:25.899 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore fsck on mount = True 2023-09-23T03:26:25.900 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluefs = 20 2023-09-23T03:26:25.900 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluestore = 20 2023-09-23T03:26:25.900 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:26:25.900 INFO:teuthology.orchestra.run.smithi190.stdout: debug osd = 20 2023-09-23T03:26:25.900 INFO:teuthology.orchestra.run.smithi190.stdout: debug rocksdb = 10 2023-09-23T03:26:25.900 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd backfillfull_ratio = 0.85 2023-09-23T03:26:25.900 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd full ratio = 0.9 2023-09-23T03:26:25.900 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd nearfull ratio = 0.8 2023-09-23T03:26:25.900 INFO:teuthology.orchestra.run.smithi190.stdout: osd failsafe full ratio = 0.95 2023-09-23T03:26:25.900 INFO:teuthology.orchestra.run.smithi190.stdout: osd objectstore = bluestore 2023-09-23T03:26:25.900 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2023-09-23T03:26:25.900 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:25.901 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr] 2023-09-23T03:26:25.901 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:26:25.901 INFO:teuthology.orchestra.run.smithi190.stdout: debug mgr = 20 2023-09-23T03:26:25.901 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2023-09-23T03:26:25.901 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2023-09-23T03:26:25.901 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:26:25.901 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:26:25.901 INFO:teuthology.orchestra.run.smithi190.stdout: mgr/telemetry/nag = false 2023-09-23T03:26:25.901 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:25.901 INFO:teuthology.orchestra.run.smithi190.stdout:[mon] 2023-09-23T03:26:25.901 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:26:25.901 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2023-09-23T03:26:25.902 INFO:teuthology.orchestra.run.smithi190.stdout: debug paxos = 20 2023-09-23T03:26:25.902 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2023-09-23T03:26:25.902 INFO:teuthology.orchestra.run.smithi190.stdout: mon data avail warn = 5 2023-09-23T03:26:25.902 INFO:teuthology.orchestra.run.smithi190.stdout: mon mgr mkfs grace = 240 2023-09-23T03:26:25.902 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:26:25.902 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd reporter subtree level = osd 2023-09-23T03:26:25.902 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd prime pg temp = true 2023-09-23T03:26:25.902 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:26:25.902 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:25.902 INFO:teuthology.orchestra.run.smithi190.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-23T03:26:25.902 INFO:teuthology.orchestra.run.smithi190.stdout: auth mon ticket ttl = 660 # 11m 2023-09-23T03:26:25.902 INFO:teuthology.orchestra.run.smithi190.stdout: auth service ticket ttl = 240 # 4m 2023-09-23T03:26:25.903 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:25.903 INFO:teuthology.orchestra.run.smithi190.stdout: # don't complain about insecure global_id in the test suite 2023-09-23T03:26:25.903 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-23T03:26:25.903 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-23T03:26:25.903 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:25.903 INFO:teuthology.orchestra.run.smithi190.stdout: # 1m isn't quite enough 2023-09-23T03:26:25.903 INFO:teuthology.orchestra.run.smithi190.stdout: mon_down_mkfs_grace = 2m 2023-09-23T03:26:25.903 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:25.903 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_filestore_osds = false 2023-09-23T03:26:25.903 INFO:teuthology.orchestra.run.smithi190.stdout: mon op complaint time = 120 2023-09-23T03:26:25.903 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:25.903 INFO:teuthology.orchestra.run.smithi190.stdout:[client] 2023-09-23T03:26:25.903 INFO:teuthology.orchestra.run.smithi190.stdout: rgw cache enabled = true 2023-09-23T03:26:25.904 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable ops log = true 2023-09-23T03:26:25.904 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable usage log = true 2023-09-23T03:26:25.904 INFO:teuthology.orchestra.run.smithi190.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-23T03:26:25.904 INFO:teuthology.orchestra.run.smithi190.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-23T03:26:25.904 INFO:teuthology.orchestra.run.smithi190.stdout: client mount timeout = 600 2023-09-23T03:26:25.904 INFO:teuthology.orchestra.run.smithi190.stdout: debug client = 20 2023-09-23T03:26:25.904 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:26:25.904 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = "" 2023-09-23T03:26:25.904 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 900 2023-09-23T03:26:25.904 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 900 2023-09-23T03:26:25.904 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.a] 2023-09-23T03:26:25.905 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.b] 2023-09-23T03:26:25.905 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.c] 2023-09-23T03:26:25.905 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.a] 2023-09-23T03:26:25.905 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.c] 2023-09-23T03:26:25.905 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.b] 2023-09-23T03:26:25.905 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.d] 2023-09-23T03:26:25.905 INFO:teuthology.orchestra.run.smithi190.stdout:[mds] 2023-09-23T03:26:25.905 INFO:teuthology.orchestra.run.smithi190.stdout: debug mds = 20 2023-09-23T03:26:25.905 INFO:teuthology.orchestra.run.smithi190.stdout: debug mds balancer = 20 2023-09-23T03:26:25.905 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:26:25.905 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug frag = True 2023-09-23T03:26:25.905 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug scatterstat = False 2023-09-23T03:26:25.906 INFO:teuthology.orchestra.run.smithi190.stdout: mds op complaint time = 180 2023-09-23T03:26:25.906 INFO:teuthology.orchestra.run.smithi190.stdout: mds verify scatter = False 2023-09-23T03:26:25.906 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2023-09-23T03:26:25.906 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 900 2023-09-23T03:26:25.906 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 900 2023-09-23T03:26:25.927 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-23T03:26:25.932 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-23T03:26:26.090 INFO:teuthology.orchestra.run.smithi154.stdout:[global] 2023-09-23T03:26:26.090 INFO:teuthology.orchestra.run.smithi154.stdout: chdir = "" 2023-09-23T03:26:26.090 INFO:teuthology.orchestra.run.smithi154.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-23T03:26:26.090 INFO:teuthology.orchestra.run.smithi154.stdout: auth supported = cephx 2023-09-23T03:26:26.090 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:26.090 INFO:teuthology.orchestra.run.smithi154.stdout: filestore xattr use omap = true 2023-09-23T03:26:26.090 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:26.090 INFO:teuthology.orchestra.run.smithi154.stdout: mon clock drift allowed = 1.000 2023-09-23T03:26:26.090 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:26.091 INFO:teuthology.orchestra.run.smithi154.stdout: osd crush chooseleaf type = 0 2023-09-23T03:26:26.091 INFO:teuthology.orchestra.run.smithi154.stdout: auth debug = true 2023-09-23T03:26:26.091 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:26.091 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on old message = true 2023-09-23T03:26:26.091 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on bug = true 2023-09-23T03:26:26.091 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:26.091 INFO:teuthology.orchestra.run.smithi154.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-23T03:26:26.091 INFO:teuthology.orchestra.run.smithi154.stdout: mon pg warn max object skew = 0 2023-09-23T03:26:26.091 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:26.091 INFO:teuthology.orchestra.run.smithi154.stdout: # disable pg_autoscaler by default for new pools 2023-09-23T03:26:26.091 INFO:teuthology.orchestra.run.smithi154.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-23T03:26:26.092 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:26.092 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default size = 2 2023-09-23T03:26:26.092 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:26.092 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow primary affinity = true 2023-09-23T03:26:26.092 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow pg remap = true 2023-09-23T03:26:26.092 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on legacy crush tunables = false 2023-09-23T03:26:26.092 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on crush straw calc version zero = false 2023-09-23T03:26:26.092 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on no sortbitwise = false 2023-09-23T03:26:26.092 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on osd down out interval zero = false 2023-09-23T03:26:26.092 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on too few osds = false 2023-09-23T03:26:26.092 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-23T03:26:26.092 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-23T03:26:26.093 INFO:teuthology.orchestra.run.smithi154.stdout: mon_allow_pool_size_one = true 2023-09-23T03:26:26.093 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:26.093 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-23T03:26:26.093 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:26.093 INFO:teuthology.orchestra.run.smithi154.stdout: osd default data pool replay window = 5 2023-09-23T03:26:26.093 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:26.093 INFO:teuthology.orchestra.run.smithi154.stdout: mon allow pool delete = true 2023-09-23T03:26:26.093 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:26.093 INFO:teuthology.orchestra.run.smithi154.stdout: mon cluster log file level = debug 2023-09-23T03:26:26.093 INFO:teuthology.orchestra.run.smithi154.stdout: debug asserts on shutdown = true 2023-09-23T03:26:26.093 INFO:teuthology.orchestra.run.smithi154.stdout: mon health detail to clog = false 2023-09-23T03:26:26.093 INFO:teuthology.orchestra.run.smithi154.stdout: mon host = "172.21.15.154,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]" 2023-09-23T03:26:26.094 INFO:teuthology.orchestra.run.smithi154.stdout: fsid = 8c8039d3-6c84-43db-8808-85e0dbdd47c8 2023-09-23T03:26:26.094 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:26.094 INFO:teuthology.orchestra.run.smithi154.stdout:[osd] 2023-09-23T03:26:26.094 INFO:teuthology.orchestra.run.smithi154.stdout: osd journal size = 100 2023-09-23T03:26:26.094 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:26.094 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub load threshold = 5.0 2023-09-23T03:26:26.094 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub max interval = 600 2023-09-23T03:26:26.094 INFO:teuthology.orchestra.run.smithi154.stdout: osd mclock profile = high_recovery_ops 2023-09-23T03:26:26.094 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:26.094 INFO:teuthology.orchestra.run.smithi154.stdout: osd recover clone overlap = true 2023-09-23T03:26:26.094 INFO:teuthology.orchestra.run.smithi154.stdout: osd recovery max chunk = 1048576 2023-09-23T03:26:26.094 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:26.095 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug shutdown = true 2023-09-23T03:26:26.095 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug op order = true 2023-09-23T03:26:26.095 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug verify stray on activate = true 2023-09-23T03:26:26.095 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:26.095 INFO:teuthology.orchestra.run.smithi154.stdout: osd open classes on start = true 2023-09-23T03:26:26.095 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug pg log writeout = true 2023-09-23T03:26:26.095 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:26.095 INFO:teuthology.orchestra.run.smithi154.stdout: osd deep scrub update digest min age = 30 2023-09-23T03:26:26.095 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:26.095 INFO:teuthology.orchestra.run.smithi154.stdout: osd map max advance = 10 2023-09-23T03:26:26.095 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:26.096 INFO:teuthology.orchestra.run.smithi154.stdout: journal zero on create = true 2023-09-23T03:26:26.096 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:26.096 INFO:teuthology.orchestra.run.smithi154.stdout: filestore ondisk finisher threads = 3 2023-09-23T03:26:26.096 INFO:teuthology.orchestra.run.smithi154.stdout: filestore apply finisher threads = 3 2023-09-23T03:26:26.096 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:26.096 INFO:teuthology.orchestra.run.smithi154.stdout: bdev debug aio = true 2023-09-23T03:26:26.096 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug misdirected ops = true 2023-09-23T03:26:26.096 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore block size = 96636764160 2023-09-23T03:26:26.096 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore fsck on mount = True 2023-09-23T03:26:26.096 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluefs = 20 2023-09-23T03:26:26.096 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluestore = 20 2023-09-23T03:26:26.097 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:26:26.097 INFO:teuthology.orchestra.run.smithi154.stdout: debug osd = 20 2023-09-23T03:26:26.097 INFO:teuthology.orchestra.run.smithi154.stdout: debug rocksdb = 10 2023-09-23T03:26:26.097 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd backfillfull_ratio = 0.85 2023-09-23T03:26:26.097 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd full ratio = 0.9 2023-09-23T03:26:26.097 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd nearfull ratio = 0.8 2023-09-23T03:26:26.097 INFO:teuthology.orchestra.run.smithi154.stdout: osd failsafe full ratio = 0.95 2023-09-23T03:26:26.097 INFO:teuthology.orchestra.run.smithi154.stdout: osd objectstore = bluestore 2023-09-23T03:26:26.097 INFO:teuthology.orchestra.run.smithi154.stdout: osd op complaint time = 180 2023-09-23T03:26:26.097 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:26.097 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr] 2023-09-23T03:26:26.097 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:26:26.098 INFO:teuthology.orchestra.run.smithi154.stdout: debug mgr = 20 2023-09-23T03:26:26.098 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2023-09-23T03:26:26.098 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2023-09-23T03:26:26.098 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:26:26.098 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:26:26.098 INFO:teuthology.orchestra.run.smithi154.stdout: mgr/telemetry/nag = false 2023-09-23T03:26:26.098 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:26.098 INFO:teuthology.orchestra.run.smithi154.stdout:[mon] 2023-09-23T03:26:26.098 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:26:26.098 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2023-09-23T03:26:26.098 INFO:teuthology.orchestra.run.smithi154.stdout: debug paxos = 20 2023-09-23T03:26:26.098 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2023-09-23T03:26:26.099 INFO:teuthology.orchestra.run.smithi154.stdout: mon data avail warn = 5 2023-09-23T03:26:26.099 INFO:teuthology.orchestra.run.smithi154.stdout: mon mgr mkfs grace = 240 2023-09-23T03:26:26.099 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:26:26.099 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd reporter subtree level = osd 2023-09-23T03:26:26.099 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd prime pg temp = true 2023-09-23T03:26:26.099 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:26:26.099 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:26.099 INFO:teuthology.orchestra.run.smithi154.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-23T03:26:26.099 INFO:teuthology.orchestra.run.smithi154.stdout: auth mon ticket ttl = 660 # 11m 2023-09-23T03:26:26.099 INFO:teuthology.orchestra.run.smithi154.stdout: auth service ticket ttl = 240 # 4m 2023-09-23T03:26:26.099 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:26.100 INFO:teuthology.orchestra.run.smithi154.stdout: # don't complain about insecure global_id in the test suite 2023-09-23T03:26:26.100 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-23T03:26:26.100 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-23T03:26:26.100 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:26.100 INFO:teuthology.orchestra.run.smithi154.stdout: # 1m isn't quite enough 2023-09-23T03:26:26.100 INFO:teuthology.orchestra.run.smithi154.stdout: mon_down_mkfs_grace = 2m 2023-09-23T03:26:26.100 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:26.100 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_filestore_osds = false 2023-09-23T03:26:26.100 INFO:teuthology.orchestra.run.smithi154.stdout: mon op complaint time = 120 2023-09-23T03:26:26.100 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:26.100 INFO:teuthology.orchestra.run.smithi154.stdout:[client] 2023-09-23T03:26:26.100 INFO:teuthology.orchestra.run.smithi154.stdout: rgw cache enabled = true 2023-09-23T03:26:26.101 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable ops log = true 2023-09-23T03:26:26.101 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable usage log = true 2023-09-23T03:26:26.101 INFO:teuthology.orchestra.run.smithi154.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-23T03:26:26.101 INFO:teuthology.orchestra.run.smithi154.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-23T03:26:26.101 INFO:teuthology.orchestra.run.smithi154.stdout: client mount timeout = 600 2023-09-23T03:26:26.101 INFO:teuthology.orchestra.run.smithi154.stdout: debug client = 20 2023-09-23T03:26:26.101 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:26:26.101 INFO:teuthology.orchestra.run.smithi154.stdout: pid file = "" 2023-09-23T03:26:26.101 INFO:teuthology.orchestra.run.smithi154.stdout: rados mon op timeout = 900 2023-09-23T03:26:26.101 INFO:teuthology.orchestra.run.smithi154.stdout: rados osd op timeout = 900 2023-09-23T03:26:26.101 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.a] 2023-09-23T03:26:26.101 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.b] 2023-09-23T03:26:26.101 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.c] 2023-09-23T03:26:26.102 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.a] 2023-09-23T03:26:26.102 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.c] 2023-09-23T03:26:26.102 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.b] 2023-09-23T03:26:26.102 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.d] 2023-09-23T03:26:26.102 INFO:teuthology.orchestra.run.smithi154.stdout:[mds] 2023-09-23T03:26:26.102 INFO:teuthology.orchestra.run.smithi154.stdout: debug mds = 20 2023-09-23T03:26:26.102 INFO:teuthology.orchestra.run.smithi154.stdout: debug mds balancer = 20 2023-09-23T03:26:26.102 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:26:26.102 INFO:teuthology.orchestra.run.smithi154.stdout: mds debug frag = True 2023-09-23T03:26:26.102 INFO:teuthology.orchestra.run.smithi154.stdout: mds debug scatterstat = False 2023-09-23T03:26:26.102 INFO:teuthology.orchestra.run.smithi154.stdout: mds op complaint time = 180 2023-09-23T03:26:26.102 INFO:teuthology.orchestra.run.smithi154.stdout: mds verify scatter = False 2023-09-23T03:26:26.103 INFO:teuthology.orchestra.run.smithi154.stdout: osd op complaint time = 180 2023-09-23T03:26:26.103 INFO:teuthology.orchestra.run.smithi154.stdout: rados mon op timeout = 900 2023-09-23T03:26:26.103 INFO:teuthology.orchestra.run.smithi154.stdout: rados osd op timeout = 900 2023-09-23T03:26:26.103 INFO:teuthology.orchestra.run.smithi190.stdout:[global] 2023-09-23T03:26:26.103 INFO:teuthology.orchestra.run.smithi190.stdout: chdir = "" 2023-09-23T03:26:26.103 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-23T03:26:26.103 INFO:teuthology.orchestra.run.smithi190.stdout: auth supported = cephx 2023-09-23T03:26:26.103 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:26.104 INFO:teuthology.orchestra.run.smithi190.stdout: filestore xattr use omap = true 2023-09-23T03:26:26.104 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:26.104 INFO:teuthology.orchestra.run.smithi190.stdout: mon clock drift allowed = 1.000 2023-09-23T03:26:26.104 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:26.104 INFO:teuthology.orchestra.run.smithi190.stdout: osd crush chooseleaf type = 0 2023-09-23T03:26:26.104 INFO:teuthology.orchestra.run.smithi190.stdout: auth debug = true 2023-09-23T03:26:26.104 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:26.104 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on old message = true 2023-09-23T03:26:26.104 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on bug = true 2023-09-23T03:26:26.104 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:26.104 INFO:teuthology.orchestra.run.smithi190.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-23T03:26:26.104 INFO:teuthology.orchestra.run.smithi190.stdout: mon pg warn max object skew = 0 2023-09-23T03:26:26.105 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:26.105 INFO:teuthology.orchestra.run.smithi190.stdout: # disable pg_autoscaler by default for new pools 2023-09-23T03:26:26.105 INFO:teuthology.orchestra.run.smithi190.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-23T03:26:26.105 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:26.105 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default size = 2 2023-09-23T03:26:26.105 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:26.105 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow primary affinity = true 2023-09-23T03:26:26.105 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow pg remap = true 2023-09-23T03:26:26.105 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on legacy crush tunables = false 2023-09-23T03:26:26.105 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on crush straw calc version zero = false 2023-09-23T03:26:26.105 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on no sortbitwise = false 2023-09-23T03:26:26.106 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on osd down out interval zero = false 2023-09-23T03:26:26.106 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on too few osds = false 2023-09-23T03:26:26.106 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-23T03:26:26.106 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-23T03:26:26.106 INFO:teuthology.orchestra.run.smithi190.stdout: mon_allow_pool_size_one = true 2023-09-23T03:26:26.106 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:26.106 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-23T03:26:26.106 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:26.106 INFO:teuthology.orchestra.run.smithi190.stdout: osd default data pool replay window = 5 2023-09-23T03:26:26.106 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:26.106 INFO:teuthology.orchestra.run.smithi190.stdout: mon allow pool delete = true 2023-09-23T03:26:26.107 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:26.107 INFO:teuthology.orchestra.run.smithi190.stdout: mon cluster log file level = debug 2023-09-23T03:26:26.107 INFO:teuthology.orchestra.run.smithi190.stdout: debug asserts on shutdown = true 2023-09-23T03:26:26.107 INFO:teuthology.orchestra.run.smithi190.stdout: mon health detail to clog = false 2023-09-23T03:26:26.107 INFO:teuthology.orchestra.run.smithi190.stdout: mon host = "172.21.15.154,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]" 2023-09-23T03:26:26.107 INFO:teuthology.orchestra.run.smithi190.stdout: fsid = 8c8039d3-6c84-43db-8808-85e0dbdd47c8 2023-09-23T03:26:26.107 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:26.107 INFO:teuthology.orchestra.run.smithi190.stdout:[osd] 2023-09-23T03:26:26.107 INFO:teuthology.orchestra.run.smithi190.stdout: osd journal size = 100 2023-09-23T03:26:26.107 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:26.107 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub load threshold = 5.0 2023-09-23T03:26:26.107 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub max interval = 600 2023-09-23T03:26:26.108 INFO:teuthology.orchestra.run.smithi190.stdout: osd mclock profile = high_recovery_ops 2023-09-23T03:26:26.108 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:26.108 INFO:teuthology.orchestra.run.smithi190.stdout: osd recover clone overlap = true 2023-09-23T03:26:26.108 INFO:teuthology.orchestra.run.smithi190.stdout: osd recovery max chunk = 1048576 2023-09-23T03:26:26.108 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:26.108 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug shutdown = true 2023-09-23T03:26:26.108 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug op order = true 2023-09-23T03:26:26.108 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug verify stray on activate = true 2023-09-23T03:26:26.108 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:26.108 INFO:teuthology.orchestra.run.smithi190.stdout: osd open classes on start = true 2023-09-23T03:26:26.108 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug pg log writeout = true 2023-09-23T03:26:26.108 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:26.109 INFO:teuthology.orchestra.run.smithi190.stdout: osd deep scrub update digest min age = 30 2023-09-23T03:26:26.109 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:26.109 INFO:teuthology.orchestra.run.smithi190.stdout: osd map max advance = 10 2023-09-23T03:26:26.109 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:26.109 INFO:teuthology.orchestra.run.smithi190.stdout: journal zero on create = true 2023-09-23T03:26:26.109 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:26.109 INFO:teuthology.orchestra.run.smithi190.stdout: filestore ondisk finisher threads = 3 2023-09-23T03:26:26.109 INFO:teuthology.orchestra.run.smithi190.stdout: filestore apply finisher threads = 3 2023-09-23T03:26:26.109 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:26.109 INFO:teuthology.orchestra.run.smithi190.stdout: bdev debug aio = true 2023-09-23T03:26:26.109 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug misdirected ops = true 2023-09-23T03:26:26.110 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore block size = 96636764160 2023-09-23T03:26:26.110 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore fsck on mount = True 2023-09-23T03:26:26.110 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluefs = 20 2023-09-23T03:26:26.110 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluestore = 20 2023-09-23T03:26:26.110 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:26:26.110 INFO:teuthology.orchestra.run.smithi190.stdout: debug osd = 20 2023-09-23T03:26:26.110 INFO:teuthology.orchestra.run.smithi190.stdout: debug rocksdb = 10 2023-09-23T03:26:26.110 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd backfillfull_ratio = 0.85 2023-09-23T03:26:26.110 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd full ratio = 0.9 2023-09-23T03:26:26.110 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd nearfull ratio = 0.8 2023-09-23T03:26:26.110 INFO:teuthology.orchestra.run.smithi190.stdout: osd failsafe full ratio = 0.95 2023-09-23T03:26:26.111 INFO:teuthology.orchestra.run.smithi190.stdout: osd objectstore = bluestore 2023-09-23T03:26:26.111 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2023-09-23T03:26:26.111 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:26.111 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr] 2023-09-23T03:26:26.111 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:26:26.111 INFO:teuthology.orchestra.run.smithi190.stdout: debug mgr = 20 2023-09-23T03:26:26.111 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2023-09-23T03:26:26.111 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2023-09-23T03:26:26.111 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:26:26.111 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:26:26.111 INFO:teuthology.orchestra.run.smithi190.stdout: mgr/telemetry/nag = false 2023-09-23T03:26:26.112 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:26.112 INFO:teuthology.orchestra.run.smithi190.stdout:[mon] 2023-09-23T03:26:26.112 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:26:26.112 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2023-09-23T03:26:26.112 INFO:teuthology.orchestra.run.smithi190.stdout: debug paxos = 20 2023-09-23T03:26:26.112 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2023-09-23T03:26:26.112 INFO:teuthology.orchestra.run.smithi190.stdout: mon data avail warn = 5 2023-09-23T03:26:26.112 INFO:teuthology.orchestra.run.smithi190.stdout: mon mgr mkfs grace = 240 2023-09-23T03:26:26.112 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:26:26.112 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd reporter subtree level = osd 2023-09-23T03:26:26.112 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd prime pg temp = true 2023-09-23T03:26:26.113 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:26:26.113 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:26.113 INFO:teuthology.orchestra.run.smithi190.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-23T03:26:26.113 INFO:teuthology.orchestra.run.smithi190.stdout: auth mon ticket ttl = 660 # 11m 2023-09-23T03:26:26.113 INFO:teuthology.orchestra.run.smithi190.stdout: auth service ticket ttl = 240 # 4m 2023-09-23T03:26:26.113 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:26.113 INFO:teuthology.orchestra.run.smithi190.stdout: # don't complain about insecure global_id in the test suite 2023-09-23T03:26:26.113 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-23T03:26:26.113 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-23T03:26:26.113 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:26.113 INFO:teuthology.orchestra.run.smithi190.stdout: # 1m isn't quite enough 2023-09-23T03:26:26.114 INFO:teuthology.orchestra.run.smithi190.stdout: mon_down_mkfs_grace = 2m 2023-09-23T03:26:26.114 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:26.114 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_filestore_osds = false 2023-09-23T03:26:26.114 INFO:teuthology.orchestra.run.smithi190.stdout: mon op complaint time = 120 2023-09-23T03:26:26.114 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:26:26.114 INFO:teuthology.orchestra.run.smithi190.stdout:[client] 2023-09-23T03:26:26.114 INFO:teuthology.orchestra.run.smithi190.stdout: rgw cache enabled = true 2023-09-23T03:26:26.114 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable ops log = true 2023-09-23T03:26:26.114 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable usage log = true 2023-09-23T03:26:26.114 INFO:teuthology.orchestra.run.smithi190.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-23T03:26:26.114 INFO:teuthology.orchestra.run.smithi190.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-23T03:26:26.115 INFO:teuthology.orchestra.run.smithi190.stdout: client mount timeout = 600 2023-09-23T03:26:26.115 INFO:teuthology.orchestra.run.smithi190.stdout: debug client = 20 2023-09-23T03:26:26.115 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:26:26.115 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = "" 2023-09-23T03:26:26.115 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 900 2023-09-23T03:26:26.115 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 900 2023-09-23T03:26:26.115 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.a] 2023-09-23T03:26:26.115 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.b] 2023-09-23T03:26:26.115 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.c] 2023-09-23T03:26:26.115 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.a] 2023-09-23T03:26:26.115 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.c] 2023-09-23T03:26:26.116 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.b] 2023-09-23T03:26:26.116 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.d] 2023-09-23T03:26:26.116 INFO:teuthology.orchestra.run.smithi190.stdout:[mds] 2023-09-23T03:26:26.116 INFO:teuthology.orchestra.run.smithi190.stdout: debug mds = 20 2023-09-23T03:26:26.116 INFO:teuthology.orchestra.run.smithi190.stdout: debug mds balancer = 20 2023-09-23T03:26:26.116 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:26:26.116 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug frag = True 2023-09-23T03:26:26.116 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug scatterstat = False 2023-09-23T03:26:26.116 INFO:teuthology.orchestra.run.smithi190.stdout: mds op complaint time = 180 2023-09-23T03:26:26.116 INFO:teuthology.orchestra.run.smithi190.stdout: mds verify scatter = False 2023-09-23T03:26:26.116 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2023-09-23T03:26:26.117 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 900 2023-09-23T03:26:26.117 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 900 2023-09-23T03:26:26.137 DEBUG:teuthology.orchestra.run.smithi154:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph purge cephfs_metadata --yes-i-really-really-mean-it 2023-09-23T03:26:26.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.243+0000 7fdfc8640f40 1 Processor -- start 2023-09-23T03:26:26.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.243+0000 7fdfc8640f40 1 -- start start 2023-09-23T03:26:26.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.244+0000 7fdfc8640f40 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b0e594410 0x558b0e622510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:26.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.244+0000 7fdfc8640f40 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558b0e5947f0 0x558b0e625760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:26.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.244+0000 7fdfc8640f40 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558b0e595e20 0x558b0e625ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:26.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.244+0000 7fdfc8640f40 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b0e498800 con 0x558b0e5947f0 2023-09-23T03:26:26.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.244+0000 7fdfc8640f40 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b0e4984a0 con 0x558b0e594410 2023-09-23T03:26:26.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.244+0000 7fdfc8640f40 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x558b0e48c4f0 con 0x558b0e595e20 2023-09-23T03:26:26.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.244+0000 7fdfbd12d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558b0e5947f0 0x558b0e625760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:26.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.244+0000 7fdfbd12d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558b0e5947f0 0x558b0e625760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45732/0 (socket says 172.21.15.154:45732) 2023-09-23T03:26:26.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.244+0000 7fdfbd12d640 1 -- 172.21.15.154:0/2754747309 learned_addr learned my addr 172.21.15.154:0/2754747309 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:26.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.244+0000 7fdfc592e640 1 --2- 172.21.15.154:0/2754747309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b0e594410 0x558b0e622510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:26.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.244+0000 7fdfc612f640 1 --2- 172.21.15.154:0/2754747309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558b0e595e20 0x558b0e625ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:26.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.244+0000 7fdfbd12d640 1 -- 172.21.15.154:0/2754747309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558b0e595e20 msgr2=0x558b0e625ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:26.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.244+0000 7fdfbd12d640 1 --2- 172.21.15.154:0/2754747309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558b0e595e20 0x558b0e625ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:26.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.244+0000 7fdfbd12d640 1 -- 172.21.15.154:0/2754747309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b0e594410 msgr2=0x558b0e622510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:26.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.244+0000 7fdfbd12d640 1 --2- 172.21.15.154:0/2754747309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b0e594410 0x558b0e622510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:26.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.244+0000 7fdfbd12d640 1 -- 172.21.15.154:0/2754747309 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb0005de0 con 0x558b0e5947f0 2023-09-23T03:26:26.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.244+0000 7fdfc612f640 1 --2- 172.21.15.154:0/2754747309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558b0e595e20 0x558b0e625ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:26:26.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.245+0000 7fdfbd12d640 1 --2- 172.21.15.154:0/2754747309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558b0e5947f0 0x558b0e625760 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fdfb000a5a0 tx=0x7fdfb004c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e18a5eee809735ed server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:26.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.245+0000 7fdfc492c640 1 -- 172.21.15.154:0/2754747309 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0052440 con 0x558b0e5947f0 2023-09-23T03:26:26.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.245+0000 7fdfc492c640 1 -- 172.21.15.154:0/2754747309 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb00525a0 con 0x558b0e5947f0 2023-09-23T03:26:26.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.245+0000 7fdfc492c640 1 -- 172.21.15.154:0/2754747309 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0052930 con 0x558b0e5947f0 2023-09-23T03:26:26.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.246+0000 7fdfc8640f40 1 -- 172.21.15.154:0/2754747309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558b0e5947f0 msgr2=0x558b0e625760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:26.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.246+0000 7fdfc8640f40 1 --2- 172.21.15.154:0/2754747309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558b0e5947f0 0x558b0e625760 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fdfb000a5a0 tx=0x7fdfb004c000 comp rx=0 tx=0).stop 2023-09-23T03:26:26.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.246+0000 7fdfc8640f40 1 -- 172.21.15.154:0/2754747309 shutdown_connections 2023-09-23T03:26:26.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.246+0000 7fdfc8640f40 1 --2- 172.21.15.154:0/2754747309 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558b0e595e20 0x558b0e625ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:26.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.246+0000 7fdfc8640f40 1 --2- 172.21.15.154:0/2754747309 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558b0e5947f0 0x558b0e625760 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:26.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.246+0000 7fdfc8640f40 1 --2- 172.21.15.154:0/2754747309 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b0e594410 0x558b0e622510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:26.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.246+0000 7fdfc8640f40 1 -- 172.21.15.154:0/2754747309 >> 172.21.15.154:0/2754747309 conn(0x558b0e598c50 msgr2=0x558b0e623db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:26.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.246+0000 7fdfc8640f40 1 -- 172.21.15.154:0/2754747309 shutdown_connections 2023-09-23T03:26:26.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.246+0000 7fdfc8640f40 1 -- 172.21.15.154:0/2754747309 wait complete. 2023-09-23T03:26:26.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.246+0000 7fdfc8640f40 1 Processor -- start 2023-09-23T03:26:26.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.247+0000 7fdfc8640f40 1 -- start start 2023-09-23T03:26:26.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.247+0000 7fdfc8640f40 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558b0e594410 0x558b0e60e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:26.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.248+0000 7fdfc8640f40 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558b0e5947f0 0x558b0e579cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:26.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.248+0000 7fdfc8640f40 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b0e595e20 0x558b0e61caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:26.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.248+0000 7fdfc8640f40 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b0e57a600 con 0x558b0e5947f0 2023-09-23T03:26:26.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.248+0000 7fdfc8640f40 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b0e615520 con 0x558b0e595e20 2023-09-23T03:26:26.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.248+0000 7fdfc8640f40 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x558b0e57a320 con 0x558b0e594410 2023-09-23T03:26:26.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.248+0000 7fdfbd12d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558b0e5947f0 0x558b0e579cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:26.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.248+0000 7fdfbd12d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558b0e5947f0 0x558b0e579cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45738/0 (socket says 172.21.15.154:45738) 2023-09-23T03:26:26.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.248+0000 7fdfbd12d640 1 -- 172.21.15.154:0/3015864293 learned_addr learned my addr 172.21.15.154:0/3015864293 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:26.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.248+0000 7fdfc592e640 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558b0e594410 0x558b0e60e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:26.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.248+0000 7fdfc612f640 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b0e595e20 0x558b0e61caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:26.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.248+0000 7fdfbd12d640 1 -- 172.21.15.154:0/3015864293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558b0e594410 msgr2=0x558b0e60e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:26.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.248+0000 7fdfbd12d640 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558b0e594410 0x558b0e60e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:26.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.248+0000 7fdfbd12d640 1 -- 172.21.15.154:0/3015864293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b0e595e20 msgr2=0x558b0e61caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:26.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.248+0000 7fdfbd12d640 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558b0e595e20 0x558b0e61caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:26.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.248+0000 7fdfbd12d640 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b0e4be640 con 0x558b0e5947f0 2023-09-23T03:26:26.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.249+0000 7fdfbd12d640 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558b0e5947f0 0x558b0e579cd0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fdfb000a5a0 tx=0x7fdfb0002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:26.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.249+0000 7fdfbf7fe640 1 -- 172.21.15.154:0/3015864293 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb005c410 con 0x558b0e5947f0 2023-09-23T03:26:26.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.249+0000 7fdfbf7fe640 1 -- 172.21.15.154:0/3015864293 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb0060040 con 0x558b0e5947f0 2023-09-23T03:26:26.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.249+0000 7fdfbf7fe640 1 -- 172.21.15.154:0/3015864293 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb005c790 con 0x558b0e5947f0 2023-09-23T03:26:26.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.249+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558b0e4a4650 con 0x558b0e5947f0 2023-09-23T03:26:26.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.249+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b0e497800 con 0x558b0e5947f0 2023-09-23T03:26:26.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.251+0000 7fdfbf7fe640 1 -- 172.21.15.154:0/3015864293 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fdfb0059050 con 0x558b0e5947f0 2023-09-23T03:26:26.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.252+0000 7fdfbf7fe640 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdfa80698e0 0x7fdfa806bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:26.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.252+0000 7fdfbf7fe640 1 -- 172.21.15.154:0/3015864293 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6519+0+0 (secure 0 0 0) 0x7fdfb00cb180 con 0x558b0e5947f0 2023-09-23T03:26:26.254 INFO:teuthology.orchestra.run.smithi154.stdout:Warning: using slow linear search 2023-09-23T03:26:26.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.252+0000 7fdfc8640f40 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x558b0e625f00 0x558b0e6ca680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:26.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.252+0000 7fdfc592e640 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdfa80698e0 0x7fdfa806bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:26.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.252+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:1 14.0 14:00000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x558b0e6cabc0 con 0x558b0e625f00 2023-09-23T03:26:26.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.253+0000 7fdfc612f640 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x558b0e625f00 0x558b0e6ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:26.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.253+0000 7fdfc592e640 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdfa80698e0 0x7fdfa806bda0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fdfb800d4a0 tx=0x7fdfb8007040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:26.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.253+0000 7fdfc612f640 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x558b0e625f00 0x558b0e6ca680 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:26.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.254+0000 7fdfc612f640 1 -- 172.21.15.154:0/3015864293 <== osd.2 v2:172.21.15.154:6822/2884407243 1 ==== osd_op_reply(1 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdfc000f3d0 con 0x558b0e625f00 2023-09-23T03:26:26.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.254+0000 7fdfc612f640 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x7fdfc000e870 0x7fdfc000d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:26.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.254+0000 7fdfc612f640 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:2 14.10 14:08000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x558b0e6cabc0 con 0x7fdfc000e870 2023-09-23T03:26:26.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.254+0000 7fdfc592e640 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x7fdfc000e870 0x7fdfc000d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:26.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.255+0000 7fdfc592e640 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x7fdfc000e870 0x7fdfc000d350 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:26.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.255+0000 7fdfc592e640 1 -- 172.21.15.154:0/3015864293 <== osd.1 v2:172.21.15.154:6816/1267710005 1 ==== osd_op_reply(2 [pgnls start_epoch 109 out=177b] v108'10 uv10 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x7fdfb8014810 con 0x7fdfc000e870 2023-09-23T03:26:26.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.255+0000 7fdfc592e640 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x7fdfb801f3d0 0x7fdfb801f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:26.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.255+0000 7fdfc592e640 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:3 14.8 14:10000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x558b0e6cabc0 con 0x7fdfb801f3d0 2023-09-23T03:26:26.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.256+0000 7fdfbd12d640 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x7fdfb801f3d0 0x7fdfb801f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:26.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.256+0000 7fdfbd12d640 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x7fdfb801f3d0 0x7fdfb801f830 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:26.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.256+0000 7fdfbd12d640 1 -- 172.21.15.154:0/3015864293 <== osd.0 v2:172.21.15.154:6800/2920938138 1 ==== osd_op_reply(3 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdfb006e020 con 0x7fdfb801f3d0 2023-09-23T03:26:26.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.256+0000 7fdfbd12d640 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:4 14.18 14:18000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x558b0e6cabc0 con 0x7fdfc000e870 2023-09-23T03:26:26.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.257+0000 7fdfc592e640 1 -- 172.21.15.154:0/3015864293 <== osd.1 v2:172.21.15.154:6816/1267710005 2 ==== osd_op_reply(4 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdfb8014810 con 0x7fdfc000e870 2023-09-23T03:26:26.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.257+0000 7fdfc592e640 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x7fdfb80229e0 0x7fdfb80133d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:26.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.257+0000 7fdfc592e640 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:5 14.4 14:20000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x558b0e6cabc0 con 0x7fdfb80229e0 2023-09-23T03:26:26.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.257+0000 7fdfc612f640 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x7fdfb80229e0 0x7fdfb80133d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:26.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.258+0000 7fdfc612f640 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x7fdfb80229e0 0x7fdfb80133d0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:26.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.259+0000 7fdfc612f640 1 -- 172.21.15.154:0/3015864293 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== osd_op_reply(5 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdfc000f3d0 con 0x7fdfb80229e0 2023-09-23T03:26:26.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.259+0000 7fdfc612f640 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x7fdfc0012960 0x7fdfc000b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:26.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.259+0000 7fdfc612f640 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:6 14.14 14:28000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x558b0e6cabc0 con 0x7fdfc0012960 2023-09-23T03:26:26.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.260+0000 7fdfc592e640 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x7fdfc0012960 0x7fdfc000b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:26.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.261+0000 7fdfc592e640 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x7fdfc0012960 0x7fdfc000b2d0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:26.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.262+0000 7fdfc592e640 1 -- 172.21.15.154:0/3015864293 <== osd.4 v2:172.21.15.190:6808/717302641 1 ==== osd_op_reply(6 [pgnls start_epoch 109 out=149b] v108'42 uv42 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x7fdfb8014810 con 0x7fdfc0012960 2023-09-23T03:26:26.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.262+0000 7fdfc592e640 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:7 14.c 14:30000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x558b0e6cabc0 con 0x7fdfb801f3d0 2023-09-23T03:26:26.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.262+0000 7fdfbd12d640 1 -- 172.21.15.154:0/3015864293 <== osd.0 v2:172.21.15.154:6800/2920938138 2 ==== osd_op_reply(7 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdfb006e020 con 0x7fdfb801f3d0 2023-09-23T03:26:26.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.262+0000 7fdfbd12d640 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:8 14.1c 14:38000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x558b0e6cabc0 con 0x7fdfb801f3d0 2023-09-23T03:26:26.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.263+0000 7fdfbd12d640 1 -- 172.21.15.154:0/3015864293 <== osd.0 v2:172.21.15.154:6800/2920938138 3 ==== osd_op_reply(8 [pgnls start_epoch 109 out=73b] v104'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fdfb006e020 con 0x7fdfb801f3d0 2023-09-23T03:26:26.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.263+0000 7fdfbd12d640 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:9 14.2 14:40000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x558b0e6cabc0 con 0x7fdfb80229e0 2023-09-23T03:26:26.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.263+0000 7fdfc612f640 1 -- 172.21.15.154:0/3015864293 <== osd.6 v2:172.21.15.190:6816/1964294001 2 ==== osd_op_reply(9 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdfc000f3d0 con 0x7fdfb80229e0 2023-09-23T03:26:26.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.263+0000 7fdfc612f640 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:10 14.12 14:48000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x558b0e6cabc0 con 0x558b0e625f00 2023-09-23T03:26:26.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.264+0000 7fdfc612f640 1 -- 172.21.15.154:0/3015864293 <== osd.2 v2:172.21.15.154:6822/2884407243 2 ==== osd_op_reply(10 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdfc000f3d0 con 0x558b0e625f00 2023-09-23T03:26:26.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.264+0000 7fdfc612f640 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x7fdfc00212d0 0x7fdfc0021750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:26.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.264+0000 7fdfc612f640 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:11 14.a 14:50000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x558b0e6cabc0 con 0x7fdfc00212d0 2023-09-23T03:26:26.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.265+0000 7fdfbd12d640 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x7fdfc00212d0 0x7fdfc0021750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:26.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.266+0000 7fdfbd12d640 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x7fdfc00212d0 0x7fdfc0021750 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:26.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.266+0000 7fdfbd12d640 1 -- 172.21.15.154:0/3015864293 <== osd.5 v2:172.21.15.190:6800/882536885 1 ==== osd_op_reply(11 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdfb006e020 con 0x7fdfc00212d0 2023-09-23T03:26:26.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.267+0000 7fdfbd12d640 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:12 14.1a 14:58000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x558b0e6cabc0 con 0x7fdfb801f3d0 2023-09-23T03:26:26.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.267+0000 7fdfbd12d640 1 -- 172.21.15.154:0/3015864293 <== osd.0 v2:172.21.15.154:6800/2920938138 4 ==== osd_op_reply(12 [pgnls start_epoch 109 out=49b] v104'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdfb006e020 con 0x7fdfb801f3d0 2023-09-23T03:26:26.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.267+0000 7fdfbd12d640 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:13 14.6 14:60000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x558b0e6cabc0 con 0x7fdfb801f3d0 2023-09-23T03:26:26.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.268+0000 7fdfbd12d640 1 -- 172.21.15.154:0/3015864293 <== osd.0 v2:172.21.15.154:6800/2920938138 5 ==== osd_op_reply(13 [pgnls start_epoch 109 out=77b] v108'7 uv7 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x7fdfb006e020 con 0x7fdfb801f3d0 2023-09-23T03:26:26.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.268+0000 7fdfbd12d640 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x7fdfb0099ec0 0x7fdfb0098bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:26.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.268+0000 7fdfbd12d640 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:14 14.16 14:68000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x558b0e6cabc0 con 0x7fdfb0099ec0 2023-09-23T03:26:26.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.268+0000 7fdfc612f640 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x7fdfb0099ec0 0x7fdfb0098bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:26.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.269+0000 7fdfc612f640 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x7fdfb0099ec0 0x7fdfb0098bc0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:26.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.269+0000 7fdfc612f640 1 -- 172.21.15.154:0/3015864293 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(14 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdfc000f3d0 con 0x7fdfb0099ec0 2023-09-23T03:26:26.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.269+0000 7fdfc612f640 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:15 14.e 14:70000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x558b0e6cabc0 con 0x7fdfc0012960 2023-09-23T03:26:26.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.270+0000 7fdfc592e640 1 -- 172.21.15.154:0/3015864293 <== osd.4 v2:172.21.15.190:6808/717302641 2 ==== osd_op_reply(15 [pgnls start_epoch 109 out=73b] v104'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fdfb8014810 con 0x7fdfc0012960 2023-09-23T03:26:26.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.270+0000 7fdfc592e640 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x7fdfb802f390 0x7fdfb802f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:26.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.270+0000 7fdfc592e640 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:16 14.1e 14:78000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x558b0e6cabc0 con 0x7fdfb802f390 2023-09-23T03:26:26.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.271+0000 7fdfc592e640 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x7fdfb802f390 0x7fdfb802f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:26.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.272+0000 7fdfc592e640 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x7fdfb802f390 0x7fdfb802f820 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:26.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.273+0000 7fdfc592e640 1 -- 172.21.15.154:0/3015864293 <== osd.7 v2:172.21.15.190:6824/3460136272 1 ==== osd_op_reply(16 [pgnls start_epoch 109 out=49b] v104'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdfb8014810 con 0x7fdfb802f390 2023-09-23T03:26:26.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.273+0000 7fdfc592e640 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:17 14.1 14:80000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x558b0e6cabc0 con 0x7fdfb0099ec0 2023-09-23T03:26:26.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.274+0000 7fdfc612f640 1 -- 172.21.15.154:0/3015864293 <== osd.3 v2:172.21.15.154:6808/1133688468 2 ==== osd_op_reply(17 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdfc000f3d0 con 0x7fdfb0099ec0 2023-09-23T03:26:26.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.274+0000 7fdfc612f640 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:18 14.11 14:88000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x558b0e6cabc0 con 0x7fdfc0012960 2023-09-23T03:26:26.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.275+0000 7fdfc592e640 1 -- 172.21.15.154:0/3015864293 <== osd.4 v2:172.21.15.190:6808/717302641 3 ==== osd_op_reply(18 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdfb8014810 con 0x7fdfc0012960 2023-09-23T03:26:26.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.275+0000 7fdfc592e640 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:19 14.9 14:90000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x558b0e6cabc0 con 0x7fdfc00212d0 2023-09-23T03:26:26.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.276+0000 7fdfbd12d640 1 -- 172.21.15.154:0/3015864293 <== osd.5 v2:172.21.15.190:6800/882536885 2 ==== osd_op_reply(19 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdfb006e020 con 0x7fdfc00212d0 2023-09-23T03:26:26.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.276+0000 7fdfbd12d640 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:20 14.19 14:98000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x558b0e6cabc0 con 0x7fdfb80229e0 2023-09-23T03:26:26.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.277+0000 7fdfc612f640 1 -- 172.21.15.154:0/3015864293 <== osd.6 v2:172.21.15.190:6816/1964294001 3 ==== osd_op_reply(20 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdfc000f3d0 con 0x7fdfb80229e0 2023-09-23T03:26:26.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.277+0000 7fdfc612f640 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:21 14.5 14:a0000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x558b0e6cabc0 con 0x7fdfb801f3d0 2023-09-23T03:26:26.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.278+0000 7fdfbd12d640 1 -- 172.21.15.154:0/3015864293 <== osd.0 v2:172.21.15.154:6800/2920938138 6 ==== osd_op_reply(21 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdfb00a4040 con 0x7fdfb801f3d0 2023-09-23T03:26:26.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.278+0000 7fdfbd12d640 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:22 14.15 14:a8000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x558b0e6cabc0 con 0x7fdfb802f390 2023-09-23T03:26:26.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.279+0000 7fdfc592e640 1 -- 172.21.15.154:0/3015864293 <== osd.7 v2:172.21.15.190:6824/3460136272 2 ==== osd_op_reply(22 [pgnls start_epoch 109 out=97b] v104'3 uv3 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x7fdfb8014810 con 0x7fdfb802f390 2023-09-23T03:26:26.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.279+0000 7fdfc592e640 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:23 14.d 14:b0000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x558b0e6cabc0 con 0x7fdfb80229e0 2023-09-23T03:26:26.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.280+0000 7fdfc612f640 1 -- 172.21.15.154:0/3015864293 <== osd.6 v2:172.21.15.190:6816/1964294001 4 ==== osd_op_reply(23 [pgnls start_epoch 109 out=74b] v104'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fdfc000f3d0 con 0x7fdfb80229e0 2023-09-23T03:26:26.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.280+0000 7fdfc612f640 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:24 14.1d 14:b8000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x558b0e6cabc0 con 0x7fdfb801f3d0 2023-09-23T03:26:26.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.281+0000 7fdfbd12d640 1 -- 172.21.15.154:0/3015864293 <== osd.0 v2:172.21.15.154:6800/2920938138 7 ==== osd_op_reply(24 [pgnls start_epoch 109 out=73b] v104'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fdfb00a4040 con 0x7fdfb801f3d0 2023-09-23T03:26:26.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.281+0000 7fdfbd12d640 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:25 14.3 14:c0000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x558b0e6cabc0 con 0x7fdfb801f3d0 2023-09-23T03:26:26.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.282+0000 7fdfbd12d640 1 -- 172.21.15.154:0/3015864293 <== osd.0 v2:172.21.15.154:6800/2920938138 8 ==== osd_op_reply(25 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdfb00a4040 con 0x7fdfb801f3d0 2023-09-23T03:26:26.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.282+0000 7fdfbd12d640 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:26 14.13 14:c8000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x558b0e6cabc0 con 0x558b0e625f00 2023-09-23T03:26:26.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.283+0000 7fdfc612f640 1 -- 172.21.15.154:0/3015864293 <== osd.2 v2:172.21.15.154:6822/2884407243 3 ==== osd_op_reply(26 [pgnls start_epoch 109 out=98b] v108'3 uv3 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x7fdfc000f3d0 con 0x558b0e625f00 2023-09-23T03:26:26.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.283+0000 7fdfc612f640 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:27 14.b 14:d0000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x558b0e6cabc0 con 0x7fdfb801f3d0 2023-09-23T03:26:26.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.284+0000 7fdfbd12d640 1 -- 172.21.15.154:0/3015864293 <== osd.0 v2:172.21.15.154:6800/2920938138 9 ==== osd_op_reply(27 [pgnls start_epoch 109 out=76b] v108'3 uv3 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x7fdfb00a4040 con 0x7fdfb801f3d0 2023-09-23T03:26:26.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.284+0000 7fdfbd12d640 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:28 14.1b 14:d8000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x558b0e6cabc0 con 0x7fdfc000e870 2023-09-23T03:26:26.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.285+0000 7fdfc592e640 1 -- 172.21.15.154:0/3015864293 <== osd.1 v2:172.21.15.154:6816/1267710005 3 ==== osd_op_reply(28 [pgnls start_epoch 109 out=49b] v104'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdfb8014810 con 0x7fdfc000e870 2023-09-23T03:26:26.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.285+0000 7fdfc592e640 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:29 14.7 14:e0000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x558b0e6cabc0 con 0x7fdfc00212d0 2023-09-23T03:26:26.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.286+0000 7fdfbd12d640 1 -- 172.21.15.154:0/3015864293 <== osd.5 v2:172.21.15.190:6800/882536885 3 ==== osd_op_reply(29 [pgnls start_epoch 109 out=73b] v104'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fdfb00a4040 con 0x7fdfc00212d0 2023-09-23T03:26:26.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.286+0000 7fdfbd12d640 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:30 14.17 14:e8000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x558b0e6cabc0 con 0x7fdfc000e870 2023-09-23T03:26:26.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.286+0000 7fdfc592e640 1 -- 172.21.15.154:0/3015864293 <== osd.1 v2:172.21.15.154:6816/1267710005 4 ==== osd_op_reply(30 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fdfb8014810 con 0x7fdfc000e870 2023-09-23T03:26:26.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.286+0000 7fdfc592e640 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:31 14.f 14:f0000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x558b0e6cabc0 con 0x7fdfb802f390 2023-09-23T03:26:26.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.287+0000 7fdfc592e640 1 -- 172.21.15.154:0/3015864293 <== osd.7 v2:172.21.15.190:6824/3460136272 3 ==== osd_op_reply(31 [pgnls start_epoch 109 out=73b] v104'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fdfb8014810 con 0x7fdfb802f390 2023-09-23T03:26:26.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.287+0000 7fdfc592e640 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:32 14.1f 14:f8000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x558b0e6cabc0 con 0x7fdfb801f3d0 2023-09-23T03:26:26.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.288+0000 7fdfbd12d640 1 -- 172.21.15.154:0/3015864293 <== osd.0 v2:172.21.15.154:6800/2920938138 10 ==== osd_op_reply(32 [pgnls start_epoch 109 out=101b] v108'6 uv6 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x7fdfb00a4040 con 0x7fdfb801f3d0 2023-09-23T03:26:26.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.288+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:33 14.10 14:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x558b0e6be090 con 0x7fdfc000e870 2023-09-23T03:26:26.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.288+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:34 14.10 14:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x558b0e6be8c0 con 0x7fdfc000e870 2023-09-23T03:26:26.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.289+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:35 14.10 14:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x558b0e6cb3c0 con 0x7fdfc000e870 2023-09-23T03:26:26.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.289+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:36 14.10 14:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x558b0e6cbbc0 con 0x7fdfc000e870 2023-09-23T03:26:26.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.289+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:37 14.10 14:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x558b0e6cc3c0 con 0x7fdfc000e870 2023-09-23T03:26:26.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.289+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:38 14.14 14:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x558b0e6ccbc0 con 0x7fdfc0012960 2023-09-23T03:26:26.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.289+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:39 14.14 14:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x558b0e6cd3c0 con 0x7fdfc0012960 2023-09-23T03:26:26.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.289+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:40 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x558b0e6cdbc0 con 0x7fdfc0012960 2023-09-23T03:26:26.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.289+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:41 14.14 14:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x558b0e6ce3c0 con 0x7fdfc0012960 2023-09-23T03:26:26.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.289+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:42 14.1c 14:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x558b0e6cebc0 con 0x7fdfb801f3d0 2023-09-23T03:26:26.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.289+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:43 14.6 14:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x558b0e6cd3c0 con 0x7fdfb801f3d0 2023-09-23T03:26:26.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.289+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:44 14.e 14:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x558b0e6cdbc0 con 0x7fdfc0012960 2023-09-23T03:26:26.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.289+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:45 14.15 14:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x558b0e6cfd00 con 0x7fdfb802f390 2023-09-23T03:26:26.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.289+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:46 14.15 14:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x558b0e6d0560 con 0x7fdfb802f390 2023-09-23T03:26:26.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.289+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:47 14.d 14:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x558b0e6d1020 con 0x7fdfb80229e0 2023-09-23T03:26:26.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.289+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:48 14.1d 14:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x558b0e6cfd00 con 0x7fdfb801f3d0 2023-09-23T03:26:26.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.293+0000 7fdfbd12d640 1 -- 172.21.15.154:0/3015864293 <== osd.0 v2:172.21.15.154:6800/2920938138 11 ==== osd_op_reply(43 mds0_openfiles.0 [delete] v109'8 uv8 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fdfb00a4040 con 0x7fdfb801f3d0 2023-09-23T03:26:26.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.293+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:49 14.13 14:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x558b0e6d0560 con 0x558b0e625f00 2023-09-23T03:26:26.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.293+0000 7fdfbd12d640 1 -- 172.21.15.154:0/3015864293 <== osd.0 v2:172.21.15.154:6800/2920938138 12 ==== osd_op_reply(48 605.00000000 [delete] v109'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fdfb00a4040 con 0x7fdfb801f3d0 2023-09-23T03:26:26.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.294+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:50 14.13 14:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x558b0e6d18e0 con 0x558b0e625f00 2023-09-23T03:26:26.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.294+0000 7fdfc592e640 1 -- 172.21.15.154:0/3015864293 <== osd.4 v2:172.21.15.190:6808/717302641 4 ==== osd_op_reply(44 608.00000000 [delete] v109'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fdfb8014810 con 0x7fdfc0012960 2023-09-23T03:26:26.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.294+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:51 14.b 14:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x558b0e6cf4c0 con 0x7fdfb801f3d0 2023-09-23T03:26:26.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.296+0000 7fdfc592e640 1 -- 172.21.15.154:0/3015864293 <== osd.1 v2:172.21.15.154:6816/1267710005 5 ==== osd_op_reply(33 601.00000000 [delete] v109'11 uv11 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fdfb8014810 con 0x7fdfc000e870 2023-09-23T03:26:26.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.296+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:52 14.7 14:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x558b0e6d21b0 con 0x7fdfc00212d0 2023-09-23T03:26:26.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.296+0000 7fdfc612f640 1 -- 172.21.15.154:0/3015864293 <== osd.6 v2:172.21.15.190:6816/1964294001 5 ==== osd_op_reply(47 mds_snaptable [delete] v109'2 uv2 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fdfc000f3d0 con 0x7fdfb80229e0 2023-09-23T03:26:26.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.296+0000 7fdfc592e640 1 -- 172.21.15.154:0/3015864293 <== osd.1 v2:172.21.15.154:6816/1267710005 6 ==== osd_op_reply(34 602.00000000 [delete] v109'12 uv12 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fdfb8014810 con 0x7fdfc000e870 2023-09-23T03:26:26.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.296+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:53 14.f 14:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x558b0e6be480 con 0x7fdfb802f390 2023-09-23T03:26:26.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.296+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:54 14.1f 14:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x558b0e6d2ae0 con 0x7fdfb801f3d0 2023-09-23T03:26:26.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.296+0000 7fdfc592e640 1 -- 172.21.15.154:0/3015864293 <== osd.4 v2:172.21.15.190:6808/717302641 5 ==== osd_op_reply(38 1.00000000.inode [delete] v109'43 uv43 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fdfb8014810 con 0x7fdfc0012960 2023-09-23T03:26:26.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.297+0000 7fdfc592e640 1 -- 172.21.15.154:0/3015864293 <== osd.4 v2:172.21.15.190:6808/717302641 6 ==== osd_op_reply(39 200.00000000 [delete] v109'44 uv44 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fdfb8014810 con 0x7fdfc0012960 2023-09-23T03:26:26.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.297+0000 7fdfc592e640 1 -- 172.21.15.154:0/3015864293 <== osd.7 v2:172.21.15.190:6824/3460136272 4 ==== osd_op_reply(45 604.00000000 [delete] v109'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fdfb8014810 con 0x7fdfb802f390 2023-09-23T03:26:26.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.297+0000 7fdfc592e640 1 -- 172.21.15.154:0/3015864293 <== osd.1 v2:172.21.15.154:6816/1267710005 7 ==== osd_op_reply(35 10000000000.00000000 [delete] v109'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fdfb8014810 con 0x7fdfc000e870 2023-09-23T03:26:26.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.297+0000 7fdfbd12d640 1 -- 172.21.15.154:0/3015864293 <== osd.0 v2:172.21.15.154:6800/2920938138 13 ==== osd_op_reply(51 mds0_sessionmap [delete] v109'4 uv4 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fdfb00a4040 con 0x7fdfb801f3d0 2023-09-23T03:26:26.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.297+0000 7fdfbd12d640 1 -- 172.21.15.154:0/3015864293 <== osd.0 v2:172.21.15.154:6800/2920938138 14 ==== osd_op_reply(42 607.00000000 [delete] v109'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fdfb00a4040 con 0x7fdfb801f3d0 2023-09-23T03:26:26.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.297+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:55 14.1f 14:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x558b0e6be480 con 0x7fdfb801f3d0 2023-09-23T03:26:26.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.298+0000 7fdfc592e640 1 -- 172.21.15.154:0/3015864293 <== osd.4 v2:172.21.15.190:6808/717302641 7 ==== osd_op_reply(40 200.00000001 [delete] v109'45 uv45 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fdfb8014810 con 0x7fdfc0012960 2023-09-23T03:26:26.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.298+0000 7fdfc592e640 1 -- 172.21.15.154:0/3015864293 <== osd.1 v2:172.21.15.154:6816/1267710005 8 ==== osd_op_reply(36 600.00000000 [delete] v109'14 uv14 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fdfb8014810 con 0x7fdfc000e870 2023-09-23T03:26:26.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.298+0000 7fdfc592e640 1 -- 172.21.15.154:0/3015864293 <== osd.1 v2:172.21.15.154:6816/1267710005 9 ==== osd_op_reply(37 603.00000000 [delete] v109'15 uv15 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fdfb8014810 con 0x7fdfc000e870 2023-09-23T03:26:26.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.299+0000 7fdfc592e640 1 -- 172.21.15.154:0/3015864293 <== osd.7 v2:172.21.15.190:6824/3460136272 5 ==== osd_op_reply(46 500.00000000 [delete] v109'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fdfb8014810 con 0x7fdfb802f390 2023-09-23T03:26:26.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.300+0000 7fdfc592e640 1 -- 172.21.15.154:0/3015864293 <== osd.4 v2:172.21.15.190:6808/717302641 8 ==== osd_op_reply(41 606.00000000 [delete] v109'46 uv46 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fdfb8014810 con 0x7fdfc0012960 2023-09-23T03:26:26.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.300+0000 7fdfc612f640 1 -- 172.21.15.154:0/3015864293 <== osd.2 v2:172.21.15.154:6822/2884407243 4 ==== osd_op_reply(49 mds0_inotable [delete] v109'4 uv4 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fdfc000f3d0 con 0x558b0e625f00 2023-09-23T03:26:26.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.300+0000 7fdfc612f640 1 -- 172.21.15.154:0/3015864293 <== osd.2 v2:172.21.15.154:6822/2884407243 5 ==== osd_op_reply(50 100.00000000 [delete] v109'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fdfc000f3d0 con 0x558b0e625f00 2023-09-23T03:26:26.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.300+0000 7fdfbd12d640 1 -- 172.21.15.154:0/3015864293 <== osd.0 v2:172.21.15.154:6800/2920938138 15 ==== osd_op_reply(54 100.00000000.inode [delete] v109'7 uv7 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fdfb00a4040 con 0x7fdfb801f3d0 2023-09-23T03:26:26.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.301+0000 7fdfc592e640 1 -- 172.21.15.154:0/3015864293 <== osd.7 v2:172.21.15.190:6824/3460136272 6 ==== osd_op_reply(53 400.00000000 [delete] v109'2 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fdfb8014810 con 0x7fdfb802f390 2023-09-23T03:26:26.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.302+0000 7fdfbd12d640 1 -- 172.21.15.154:0/3015864293 <== osd.5 v2:172.21.15.190:6800/882536885 4 ==== osd_op_reply(52 609.00000000 [delete] v109'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fdfb00a4040 con 0x7fdfc00212d0 2023-09-23T03:26:26.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.302+0000 7fdfbd12d640 1 -- 172.21.15.154:0/3015864293 <== osd.0 v2:172.21.15.154:6800/2920938138 16 ==== osd_op_reply(55 1.00000000 [delete] v109'8 uv8 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fdfb00a4040 con 0x7fdfb801f3d0 2023-09-23T03:26:26.302 INFO:teuthology.orchestra.run.smithi154.stdout:Removed 23 objects 2023-09-23T03:26:26.303 INFO:teuthology.orchestra.run.smithi154.stdout:successfully purged pool cephfs_metadata 2023-09-23T03:26:26.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.302+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x7fdfb801f3d0 msgr2=0x7fdfb801f830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:26.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.302+0000 7fdfc8640f40 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x7fdfb801f3d0 0x7fdfb801f830 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:26.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.302+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x7fdfc000e870 msgr2=0x7fdfc000d350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:26.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.302+0000 7fdfc8640f40 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x7fdfc000e870 0x7fdfc000d350 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:26.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.302+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x558b0e625f00 msgr2=0x558b0e6ca680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:26.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.302+0000 7fdfc8640f40 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x558b0e625f00 0x558b0e6ca680 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:26.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.302+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x7fdfb0099ec0 msgr2=0x7fdfb0098bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:26.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.302+0000 7fdfc8640f40 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x7fdfb0099ec0 0x7fdfb0098bc0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:26.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.302+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x7fdfc0012960 msgr2=0x7fdfc000b2d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:26.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.302+0000 7fdfc8640f40 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x7fdfc0012960 0x7fdfc000b2d0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:26.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.302+0000 7fdfc612f640 1 -- 172.21.15.154:0/3015864293 reap_dead start 2023-09-23T03:26:26.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.302+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x7fdfc00212d0 msgr2=0x7fdfc0021750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:26.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.302+0000 7fdfc8640f40 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x7fdfc00212d0 0x7fdfc0021750 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:26.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.302+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x7fdfb80229e0 msgr2=0x7fdfb80133d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:26.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.302+0000 7fdfc8640f40 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x7fdfb80229e0 0x7fdfb80133d0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:26.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.302+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x7fdfb802f390 msgr2=0x7fdfb802f820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:26.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.302+0000 7fdfc8640f40 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x7fdfb802f390 0x7fdfb802f820 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:26.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.302+0000 7fdfc612f640 1 -- 172.21.15.154:0/3015864293 reap_dead start 2023-09-23T03:26:26.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.302+0000 7fdfc612f640 1 -- 172.21.15.154:0/3015864293 reap_dead start 2023-09-23T03:26:26.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.302+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdfa80698e0 msgr2=0x7fdfa806bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:26.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.302+0000 7fdfc8640f40 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdfa80698e0 0x7fdfa806bda0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fdfb800d4a0 tx=0x7fdfb8007040 comp rx=0 tx=0).stop 2023-09-23T03:26:26.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.302+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558b0e5947f0 msgr2=0x558b0e579cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:26.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.302+0000 7fdfc8640f40 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558b0e5947f0 0x558b0e579cd0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fdfb000a5a0 tx=0x7fdfb0002fb0 comp rx=0 tx=0).stop 2023-09-23T03:26:26.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.303+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 shutdown_connections 2023-09-23T03:26:26.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.303+0000 7fdfc8640f40 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdfa80698e0 0x7fdfa806bda0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:26.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.303+0000 7fdfc8640f40 1 --2- 172.21.15.154:0/3015864293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558b0e5947f0 0x558b0e579cd0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:26.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.303+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 >> 172.21.15.154:0/3015864293 conn(0x558b0e598c50 msgr2=0x558b0e625760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:26.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.303+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 shutdown_connections 2023-09-23T03:26:26.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.303+0000 7fdfc8640f40 1 -- 172.21.15.154:0/3015864293 wait complete. 2023-09-23T03:26:26.312 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs reset cephfs --yes-i-really-mean-it 2023-09-23T03:26:26.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.521+0000 7efe704e0640 1 Processor -- start 2023-09-23T03:26:26.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.522+0000 7efe704e0640 1 -- start start 2023-09-23T03:26:26.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.523+0000 7efe704e0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe6815aba0 0x7efe6815bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:26.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.523+0000 7efe704e0640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efe680cb5b0 0x7efe6815ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:26.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.523+0000 7efe704e0640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe6815b6a0 con 0x7efe6815e5f0 2023-09-23T03:26:26.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.523+0000 7efe704e0640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe6815f0b0 con 0x7efe680cb5b0 2023-09-23T03:26:26.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.523+0000 7efe704e0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe6815f390 con 0x7efe6815aba0 2023-09-23T03:26:26.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.524+0000 7efe6ea7d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efe680cb5b0 0x7efe6815ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:26.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.524+0000 7efe6ea7d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efe680cb5b0 0x7efe6815ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45752/0 (socket says 172.21.15.154:45752) 2023-09-23T03:26:26.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.524+0000 7efe6ea7d640 1 -- 172.21.15.154:0/3725401166 learned_addr learned my addr 172.21.15.154:0/3725401166 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:26.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.524+0000 7efe6e27c640 1 --2- 172.21.15.154:0/3725401166 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe6815aba0 0x7efe6815bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:26.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.524+0000 7efe6ea7d640 1 -- 172.21.15.154:0/3725401166 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe6815aba0 msgr2=0x7efe6815bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:26.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.524+0000 7efe6ea7d640 1 --2- 172.21.15.154:0/3725401166 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe6815aba0 0x7efe6815bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:26.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.524+0000 7efe6ea7d640 1 -- 172.21.15.154:0/3725401166 >> v1:172.21.15.154:6789/0 conn(0x7efe6815e5f0 legacy=0x7efe680caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:26.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.524+0000 7efe6ea7d640 1 -- 172.21.15.154:0/3725401166 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe6815b330 con 0x7efe680cb5b0 2023-09-23T03:26:26.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.526+0000 7efe6ea7d640 1 --2- 172.21.15.154:0/3725401166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe680cb5b0 0x7efe6815ea30 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7efe60009870 tx=0x7efe6002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67e8ac9019914474 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:26.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.526+0000 7efe6d27a640 1 -- 172.21.15.154:0/3725401166 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe60030840 con 0x7efe680cb5b0 2023-09-23T03:26:26.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.526+0000 7efe6d27a640 1 -- 172.21.15.154:0/3725401166 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe600309a0 con 0x7efe680cb5b0 2023-09-23T03:26:26.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.526+0000 7efe6d27a640 1 -- 172.21.15.154:0/3725401166 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe60030d30 con 0x7efe680cb5b0 2023-09-23T03:26:26.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.526+0000 7efe704e0640 1 -- 172.21.15.154:0/3725401166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe680cb5b0 msgr2=0x7efe6815ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:26.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.526+0000 7efe704e0640 1 --2- 172.21.15.154:0/3725401166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe680cb5b0 0x7efe6815ea30 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7efe60009870 tx=0x7efe6002b980 comp rx=0 tx=0).stop 2023-09-23T03:26:26.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.527+0000 7efe704e0640 1 -- 172.21.15.154:0/3725401166 shutdown_connections 2023-09-23T03:26:26.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.527+0000 7efe704e0640 1 --2- 172.21.15.154:0/3725401166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe680cb5b0 0x7efe6815ea30 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:26.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.527+0000 7efe704e0640 1 --2- 172.21.15.154:0/3725401166 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe6815aba0 0x7efe6815bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:26.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.527+0000 7efe704e0640 1 -- 172.21.15.154:0/3725401166 >> 172.21.15.154:0/3725401166 conn(0x7efe6806c010 msgr2=0x7efe6806c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:26.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.527+0000 7efe704e0640 1 -- 172.21.15.154:0/3725401166 shutdown_connections 2023-09-23T03:26:26.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.527+0000 7efe704e0640 1 -- 172.21.15.154:0/3725401166 wait complete. 2023-09-23T03:26:26.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.529+0000 7efe704e0640 1 Processor -- start 2023-09-23T03:26:26.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.529+0000 7efe704e0640 1 -- start start 2023-09-23T03:26:26.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.530+0000 7efe704e0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe680cb5b0 0x7efe68152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:26.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.530+0000 7efe704e0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe6815aba0 0x7efe680c5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:26.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.530+0000 7efe704e0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe6815e5f0 0x7efe680c6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:26.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.530+0000 7efe704e0640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe681545e0 con 0x7efe6815aba0 2023-09-23T03:26:26.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.530+0000 7efe704e0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe68158080 con 0x7efe6815e5f0 2023-09-23T03:26:26.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.530+0000 7efe704e0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe681547f0 con 0x7efe680cb5b0 2023-09-23T03:26:26.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.530+0000 7efe6da7b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe6815aba0 0x7efe680c5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:26.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.530+0000 7efe6da7b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe6815aba0 0x7efe680c5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45754/0 (socket says 172.21.15.154:45754) 2023-09-23T03:26:26.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.530+0000 7efe6da7b640 1 -- 172.21.15.154:0/2307184486 learned_addr learned my addr 172.21.15.154:0/2307184486 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:26.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.530+0000 7efe6e27c640 1 --2- 172.21.15.154:0/2307184486 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe680cb5b0 0x7efe68152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:26.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.531+0000 7efe6ea7d640 1 --2- 172.21.15.154:0/2307184486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe6815e5f0 0x7efe680c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:26.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.531+0000 7efe6da7b640 1 -- 172.21.15.154:0/2307184486 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe680cb5b0 msgr2=0x7efe68152b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:26.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.531+0000 7efe6da7b640 1 --2- 172.21.15.154:0/2307184486 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe680cb5b0 0x7efe68152b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:26.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.531+0000 7efe6da7b640 1 -- 172.21.15.154:0/2307184486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe6815e5f0 msgr2=0x7efe680c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:26.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.531+0000 7efe6da7b640 1 --2- 172.21.15.154:0/2307184486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe6815e5f0 0x7efe680c6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:26.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.531+0000 7efe6da7b640 1 -- 172.21.15.154:0/2307184486 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe600094d0 con 0x7efe6815aba0 2023-09-23T03:26:26.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.531+0000 7efe6ea7d640 1 --2- 172.21.15.154:0/2307184486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe6815e5f0 0x7efe680c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:26:26.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.531+0000 7efe6da7b640 1 --2- 172.21.15.154:0/2307184486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe6815aba0 0x7efe680c5b20 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7efe5800c400 tx=0x7efe5800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:26.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.531+0000 7efe5effd640 1 -- 172.21.15.154:0/2307184486 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe580052d0 con 0x7efe6815aba0 2023-09-23T03:26:26.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.531+0000 7efe5effd640 1 -- 172.21.15.154:0/2307184486 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe58005430 con 0x7efe6815aba0 2023-09-23T03:26:26.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.531+0000 7efe704e0640 1 -- 172.21.15.154:0/2307184486 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe6815b100 con 0x7efe6815aba0 2023-09-23T03:26:26.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.531+0000 7efe5effd640 1 -- 172.21.15.154:0/2307184486 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe58003830 con 0x7efe6815aba0 2023-09-23T03:26:26.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.532+0000 7efe704e0640 1 -- 172.21.15.154:0/2307184486 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe68154a90 con 0x7efe6815aba0 2023-09-23T03:26:26.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.533+0000 7efe704e0640 1 -- 172.21.15.154:0/2307184486 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe6804e920 con 0x7efe6815aba0 2023-09-23T03:26:26.534 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.534+0000 7efe5effd640 1 -- 172.21.15.154:0/2307184486 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7efe58003990 con 0x7efe6815aba0 2023-09-23T03:26:26.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.535+0000 7efe5effd640 1 --2- 172.21.15.154:0/2307184486 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efe4c06af30 0x7efe4c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:26.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.535+0000 7efe5effd640 1 -- 172.21.15.154:0/2307184486 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6519+0+0 (secure 0 0 0) 0x7efe580890e0 con 0x7efe6815aba0 2023-09-23T03:26:26.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.535+0000 7efe6e27c640 1 --2- 172.21.15.154:0/2307184486 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efe4c06af30 0x7efe4c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:26.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.536+0000 7efe6e27c640 1 --2- 172.21.15.154:0/2307184486 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efe4c06af30 0x7efe4c06d3f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7efe64007330 tx=0x7efe6400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:26.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.540+0000 7efe5effd640 1 -- 172.21.15.154:0/2307184486 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7efe58053d50 con 0x7efe6815aba0 2023-09-23T03:26:26.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:26.772+0000 7efe704e0640 1 -- 172.21.15.154:0/2307184486 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs reset", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7efe680cbc10 con 0x7efe6815aba0 2023-09-23T03:26:27.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.699+0000 7efe5effd640 1 -- 172.21.15.154:0/2307184486 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs reset", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v88) v1 ==== 107+0+0 (secure 0 0 0) 0x7efe58057370 con 0x7efe6815aba0 2023-09-23T03:26:27.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.702+0000 7efe704e0640 1 -- 172.21.15.154:0/2307184486 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efe4c06af30 msgr2=0x7efe4c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:27.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.702+0000 7efe704e0640 1 --2- 172.21.15.154:0/2307184486 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efe4c06af30 0x7efe4c06d3f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7efe64007330 tx=0x7efe6400b040 comp rx=0 tx=0).stop 2023-09-23T03:26:27.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.702+0000 7efe704e0640 1 -- 172.21.15.154:0/2307184486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe6815aba0 msgr2=0x7efe680c5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:27.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.702+0000 7efe704e0640 1 --2- 172.21.15.154:0/2307184486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe6815aba0 0x7efe680c5b20 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7efe5800c400 tx=0x7efe5800c8c0 comp rx=0 tx=0).stop 2023-09-23T03:26:27.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.702+0000 7efe704e0640 1 -- 172.21.15.154:0/2307184486 shutdown_connections 2023-09-23T03:26:27.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.702+0000 7efe704e0640 1 --2- 172.21.15.154:0/2307184486 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efe4c06af30 0x7efe4c06d3f0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:27.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.702+0000 7efe704e0640 1 --2- 172.21.15.154:0/2307184486 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe6815e5f0 0x7efe680c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:27.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.702+0000 7efe704e0640 1 --2- 172.21.15.154:0/2307184486 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe6815aba0 0x7efe680c5b20 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:27.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.703+0000 7efe704e0640 1 --2- 172.21.15.154:0/2307184486 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe680cb5b0 0x7efe68152b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:27.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.703+0000 7efe704e0640 1 -- 172.21.15.154:0/2307184486 >> 172.21.15.154:0/2307184486 conn(0x7efe6806c010 msgr2=0x7efe681585c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:27.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.703+0000 7efe704e0640 1 -- 172.21.15.154:0/2307184486 shutdown_connections 2023-09-23T03:26:27.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.703+0000 7efe704e0640 1 -- 172.21.15.154:0/2307184486 wait complete. 2023-09-23T03:26:27.714 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:26:27.713+0000 7f5459987640 -1 mds.0.inotable: load_2 could not read table: -2 2023-09-23T03:26:27.714 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:26:27.713+0000 7f5459987640 -1 log_channel(cluster) log [ERR] : error reading table object 'mds0_inotable' -2 ((2) No such file or directory) 2023-09-23T03:26:27.714 INFO:tasks.ceph.mon.a.smithi154.stderr:2023-09-23T03:26:27.714+0000 7f3df35a6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-23T03:26:27.714 INFO:tasks.ceph.mon.a.smithi154.stderr:2023-09-23T03:26:27.714+0000 7f3df35a6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-09-23T03:26:27.725 INFO:tasks.ceph.mds.a.smithi154.stderr: -32> 2023-09-23T03:26:27.713+0000 7f5459987640 -1 mds.0.inotable: load_2 could not read table: -2 2023-09-23T03:26:27.725 INFO:tasks.ceph.mds.a.smithi154.stderr: -31> 2023-09-23T03:26:27.713+0000 7f5459987640 -1 log_channel(cluster) log [ERR] : error reading table object 'mds0_inotable' -2 ((2) No such file or directory) 2023-09-23T03:26:27.726 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-09-23T03:26:27.768 INFO:tasks.ceph.mds.a.smithi154.stdout:starting mds.a at 2023-09-23T03:26:27.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.922+0000 7f84edfa9640 1 Processor -- start 2023-09-23T03:26:27.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.922+0000 7f84edfa9640 1 -- start start 2023-09-23T03:26:27.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.923+0000 7f84edfa9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f84e80cae90 0x7f84e80cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:27.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.923+0000 7f84edfa9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84e80c9510 0x7f84e8164c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:27.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.923+0000 7f84edfa9640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84e80c1a40 con 0x7f84e80c9980 2023-09-23T03:26:27.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.923+0000 7f84edfa9640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84e81624f0 con 0x7f84e80cae90 2023-09-23T03:26:27.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.923+0000 7f84edfa9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84e8162be0 con 0x7f84e80c9510 2023-09-23T03:26:27.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.924+0000 7f84e77fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f84e80cae90 0x7f84e80cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:27.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.924+0000 7f84e77fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f84e80cae90 0x7f84e80cb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45762/0 (socket says 172.21.15.154:45762) 2023-09-23T03:26:27.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.924+0000 7f84e77fe640 1 -- 172.21.15.154:0/1087733187 learned_addr learned my addr 172.21.15.154:0/1087733187 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:27.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.924+0000 7f84dffff640 1 --2- 172.21.15.154:0/1087733187 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84e80c9510 0x7f84e8164c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:27.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.924+0000 7f84e77fe640 1 -- 172.21.15.154:0/1087733187 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84e80c9510 msgr2=0x7f84e8164c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:27.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.924+0000 7f84e77fe640 1 --2- 172.21.15.154:0/1087733187 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84e80c9510 0x7f84e8164c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:27.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.924+0000 7f84e77fe640 1 -- 172.21.15.154:0/1087733187 >> v1:172.21.15.190:6789/0 conn(0x7f84e80c9980 legacy=0x7f84e8165180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:27.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.924+0000 7f84e77fe640 1 -- 172.21.15.154:0/1087733187 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e8162ec0 con 0x7f84e80cae90 2023-09-23T03:26:27.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.925+0000 7f84e77fe640 1 --2- 172.21.15.154:0/1087733187 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84e80cae90 0x7f84e80cb270 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f84d8009870 tx=0x7f84d802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4aaeed971a5c201 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:27.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.926+0000 7f84e6ffd640 1 -- 172.21.15.154:0/1087733187 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d8030840 con 0x7f84e80cae90 2023-09-23T03:26:27.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.926+0000 7f84e6ffd640 1 -- 172.21.15.154:0/1087733187 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84d80309a0 con 0x7f84e80cae90 2023-09-23T03:26:27.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.926+0000 7f84e6ffd640 1 -- 172.21.15.154:0/1087733187 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d8030d30 con 0x7f84e80cae90 2023-09-23T03:26:27.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.926+0000 7f84edfa9640 1 -- 172.21.15.154:0/1087733187 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84e80cae90 msgr2=0x7f84e80cb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:27.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.926+0000 7f84edfa9640 1 --2- 172.21.15.154:0/1087733187 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84e80cae90 0x7f84e80cb270 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f84d8009870 tx=0x7f84d802b980 comp rx=0 tx=0).stop 2023-09-23T03:26:27.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.926+0000 7f84edfa9640 1 -- 172.21.15.154:0/1087733187 shutdown_connections 2023-09-23T03:26:27.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.926+0000 7f84edfa9640 1 --2- 172.21.15.154:0/1087733187 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84e80c9510 0x7f84e8164c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:27.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.926+0000 7f84edfa9640 1 --2- 172.21.15.154:0/1087733187 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84e80cae90 0x7f84e80cb270 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:27.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.926+0000 7f84edfa9640 1 -- 172.21.15.154:0/1087733187 >> 172.21.15.154:0/1087733187 conn(0x7f84e806c010 msgr2=0x7f84e806c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:27.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.927+0000 7f84edfa9640 1 -- 172.21.15.154:0/1087733187 shutdown_connections 2023-09-23T03:26:27.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.927+0000 7f84edfa9640 1 -- 172.21.15.154:0/1087733187 wait complete. 2023-09-23T03:26:27.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.928+0000 7f84edfa9640 1 Processor -- start 2023-09-23T03:26:27.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.928+0000 7f84edfa9640 1 -- start start 2023-09-23T03:26:27.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.929+0000 7f84edfa9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84e80c9510 0x7f84e815b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:27.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.929+0000 7f84edfa9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84e80c9980 0x7f84e814ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:27.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.929+0000 7f84edfa9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84e80cae90 0x7f84e80c1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:27.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.929+0000 7f84edfa9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e80b46c0 con 0x7f84e80c9980 2023-09-23T03:26:27.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.929+0000 7f84edfa9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e80c23c0 con 0x7f84e80cae90 2023-09-23T03:26:27.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.929+0000 7f84edfa9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84e80b1250 con 0x7f84e80c9510 2023-09-23T03:26:27.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.930+0000 7f84dffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84e80c9980 0x7f84e814ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:27.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.930+0000 7f84dffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84e80c9980 0x7f84e814ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45776/0 (socket says 172.21.15.154:45776) 2023-09-23T03:26:27.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.930+0000 7f84dffff640 1 -- 172.21.15.154:0/3584820459 learned_addr learned my addr 172.21.15.154:0/3584820459 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:27.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.930+0000 7f84e7fff640 1 --2- 172.21.15.154:0/3584820459 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84e80cae90 0x7f84e80c1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:27.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.930+0000 7f84e77fe640 1 --2- 172.21.15.154:0/3584820459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84e80c9510 0x7f84e815b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:27.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.930+0000 7f84dffff640 1 -- 172.21.15.154:0/3584820459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84e80c9510 msgr2=0x7f84e815b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:27.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.930+0000 7f84dffff640 1 --2- 172.21.15.154:0/3584820459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84e80c9510 0x7f84e815b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:27.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.930+0000 7f84dffff640 1 -- 172.21.15.154:0/3584820459 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84e80cae90 msgr2=0x7f84e80c1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:27.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.930+0000 7f84dffff640 1 --2- 172.21.15.154:0/3584820459 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84e80cae90 0x7f84e80c1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:27.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.930+0000 7f84dffff640 1 -- 172.21.15.154:0/3584820459 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d80094d0 con 0x7f84e80c9980 2023-09-23T03:26:27.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.930+0000 7f84e77fe640 1 --2- 172.21.15.154:0/3584820459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84e80c9510 0x7f84e815b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:26:27.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.930+0000 7f84dffff640 1 --2- 172.21.15.154:0/3584820459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84e80c9980 0x7f84e814ea70 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f84d000d620 tx=0x7f84d000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:27.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.931+0000 7f84e4ff9640 1 -- 172.21.15.154:0/3584820459 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d000b620 con 0x7f84e80c9980 2023-09-23T03:26:27.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.931+0000 7f84e4ff9640 1 -- 172.21.15.154:0/3584820459 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84d000b780 con 0x7f84e80c9980 2023-09-23T03:26:27.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.931+0000 7f84e4ff9640 1 -- 172.21.15.154:0/3584820459 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d000bb30 con 0x7f84e80c9980 2023-09-23T03:26:27.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.931+0000 7f84edfa9640 1 -- 172.21.15.154:0/3584820459 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84e80c2e90 con 0x7f84e80c9980 2023-09-23T03:26:27.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.931+0000 7f84edfa9640 1 -- 172.21.15.154:0/3584820459 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e80c28d0 con 0x7f84e80c9980 2023-09-23T03:26:27.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.932+0000 7f84edfa9640 1 -- 172.21.15.154:0/3584820459 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84e80b13c0 con 0x7f84e80c9980 2023-09-23T03:26:27.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.933+0000 7f84e4ff9640 1 -- 172.21.15.154:0/3584820459 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f84d000bc90 con 0x7f84e80c9980 2023-09-23T03:26:27.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.934+0000 7f84e4ff9640 1 --2- 172.21.15.154:0/3584820459 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f84c806af30 0x7f84c806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:27.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.934+0000 7f84e4ff9640 1 -- 172.21.15.154:0/3584820459 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6605+0+0 (secure 0 0 0) 0x7f84d0089f40 con 0x7f84e80c9980 2023-09-23T03:26:27.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.934+0000 7f84e77fe640 1 --2- 172.21.15.154:0/3584820459 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f84c806af30 0x7f84c806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:27.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.935+0000 7f84e77fe640 1 --2- 172.21.15.154:0/3584820459 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f84c806af30 0x7f84c806d3f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f84d8036000 tx=0x7f84d803b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:27.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:27.938+0000 7f84e4ff9640 1 -- 172.21.15.154:0/3584820459 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f84d0054c90 con 0x7f84e80c9980 2023-09-23T03:26:28.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:28.162+0000 7f84edfa9640 1 -- 172.21.15.154:0/3584820459 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f84e80ccf80 con 0x7f84e80c9980 2023-09-23T03:26:28.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:28.721+0000 7f84e4ff9640 1 -- 172.21.15.154:0/3584820459 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v91) v1 ==== 162+0+0 (secure 0 0 0) 0x7f84d00582b0 con 0x7f84e80c9980 2023-09-23T03:26:28.722 INFO:teuthology.orchestra.run.smithi154.stderr:cephfs marked joinable; MDS may join as newly active. 2023-09-23T03:26:28.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:28.724+0000 7f84edfa9640 1 -- 172.21.15.154:0/3584820459 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f84c806af30 msgr2=0x7f84c806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:28.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:28.724+0000 7f84edfa9640 1 --2- 172.21.15.154:0/3584820459 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f84c806af30 0x7f84c806d3f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f84d8036000 tx=0x7f84d803b040 comp rx=0 tx=0).stop 2023-09-23T03:26:28.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:28.724+0000 7f84edfa9640 1 -- 172.21.15.154:0/3584820459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84e80c9980 msgr2=0x7f84e814ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:28.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:28.724+0000 7f84edfa9640 1 --2- 172.21.15.154:0/3584820459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84e80c9980 0x7f84e814ea70 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f84d000d620 tx=0x7f84d000dae0 comp rx=0 tx=0).stop 2023-09-23T03:26:28.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:28.725+0000 7f84edfa9640 1 -- 172.21.15.154:0/3584820459 shutdown_connections 2023-09-23T03:26:28.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:28.725+0000 7f84edfa9640 1 --2- 172.21.15.154:0/3584820459 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f84c806af30 0x7f84c806d3f0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:28.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:28.725+0000 7f84edfa9640 1 --2- 172.21.15.154:0/3584820459 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84e80cae90 0x7f84e80c1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:28.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:28.725+0000 7f84edfa9640 1 --2- 172.21.15.154:0/3584820459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84e80c9980 0x7f84e814ea70 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:28.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:28.725+0000 7f84edfa9640 1 --2- 172.21.15.154:0/3584820459 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84e80c9510 0x7f84e815b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:28.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:28.725+0000 7f84edfa9640 1 -- 172.21.15.154:0/3584820459 >> 172.21.15.154:0/3584820459 conn(0x7f84e806c010 msgr2=0x7f84e8070150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:28.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:28.725+0000 7f84edfa9640 1 -- 172.21.15.154:0/3584820459 shutdown_connections 2023-09-23T03:26:28.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:28.725+0000 7f84edfa9640 1 -- 172.21.15.154:0/3584820459 wait complete. 2023-09-23T03:26:28.745 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:26:29.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.007+0000 7effe471d640 1 Processor -- start 2023-09-23T03:26:29.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.008+0000 7effe471d640 1 -- start start 2023-09-23T03:26:29.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.009+0000 7effe471d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7effdc0b1b20 0x7effdc0ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:29.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.009+0000 7effe471d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effdc15ac30 con 0x7effdc0cee70 2023-09-23T03:26:29.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.009+0000 7effe471d640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effdc0b1f70 con 0x7effdc0acce0 2023-09-23T03:26:29.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.009+0000 7effe471d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effdc0b2e90 con 0x7effdc0b1b20 2023-09-23T03:26:29.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.009+0000 7effe2cba640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7effdc0cee70 0x7effdc0ab140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:50528/0 (socket says 172.21.15.154:50528) 2023-09-23T03:26:29.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.009+0000 7effe1cb8640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7effdc0b1b20 0x7effdc0ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:29.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.009+0000 7effe2cba640 1 -- 172.21.15.154:0/3056583426 learned_addr learned my addr 172.21.15.154:0/3056583426 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:26:29.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.010+0000 7effe1cb8640 1 -- 172.21.15.154:0/3056583426 >> v1:172.21.15.190:6789/0 conn(0x7effdc0acce0 legacy=0x7effdc15ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:29.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.010+0000 7effe14b7640 1 -- 172.21.15.154:0/3056583426 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 746777288 0 0) 0x7effd4001490 con 0x7effdc0cee70 2023-09-23T03:26:29.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.010+0000 7effe1cb8640 1 -- 172.21.15.154:0/3056583426 >> v1:172.21.15.154:6789/0 conn(0x7effdc0cee70 legacy=0x7effdc0ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:29.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.010+0000 7effe1cb8640 1 -- 172.21.15.154:0/3056583426 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effdc0b2210 con 0x7effdc0b1b20 2023-09-23T03:26:29.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.011+0000 7effe1cb8640 1 --2- 172.21.15.154:0/3056583426 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effdc0b1b20 0x7effdc0ce930 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7effcc0097a0 tx=0x7effcc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c09f0e7112625c6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:29.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.011+0000 7effe14b7640 1 -- 172.21.15.154:0/3056583426 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effcc02e070 con 0x7effdc0b1b20 2023-09-23T03:26:29.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.011+0000 7effe14b7640 1 -- 172.21.15.154:0/3056583426 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effcc0315a0 con 0x7effdc0b1b20 2023-09-23T03:26:29.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.011+0000 7effe14b7640 1 -- 172.21.15.154:0/3056583426 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effcc031930 con 0x7effdc0b1b20 2023-09-23T03:26:29.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.012+0000 7effe471d640 1 -- 172.21.15.154:0/3056583426 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effdc0b1b20 msgr2=0x7effdc0ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:29.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.012+0000 7effe471d640 1 --2- 172.21.15.154:0/3056583426 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effdc0b1b20 0x7effdc0ce930 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7effcc0097a0 tx=0x7effcc02b840 comp rx=0 tx=0).stop 2023-09-23T03:26:29.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.012+0000 7effe471d640 1 -- 172.21.15.154:0/3056583426 shutdown_connections 2023-09-23T03:26:29.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.012+0000 7effe471d640 1 --2- 172.21.15.154:0/3056583426 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effdc0b1b20 0x7effdc0ce930 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:29.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.012+0000 7effe471d640 1 -- 172.21.15.154:0/3056583426 >> 172.21.15.154:0/3056583426 conn(0x7effdc06a0f0 msgr2=0x7effdc06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:29.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.012+0000 7effe471d640 1 -- 172.21.15.154:0/3056583426 shutdown_connections 2023-09-23T03:26:29.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.012+0000 7effe471d640 1 -- 172.21.15.154:0/3056583426 wait complete. 2023-09-23T03:26:29.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.014+0000 7effe471d640 1 Processor -- start 2023-09-23T03:26:29.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.014+0000 7effe471d640 1 -- start start 2023-09-23T03:26:29.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.015+0000 7effe471d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effdc0acce0 0x7effdc0c2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:29.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.015+0000 7effe471d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effdc0b1b20 0x7effdc15add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:29.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.015+0000 7effe471d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effdc0cee70 0x7effdc06fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:29.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.015+0000 7effe471d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effdc0cf320 con 0x7effdc0b1b20 2023-09-23T03:26:29.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.015+0000 7effe471d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effdc15b890 con 0x7effdc0cee70 2023-09-23T03:26:29.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.015+0000 7effe471d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effdc15bb70 con 0x7effdc0acce0 2023-09-23T03:26:29.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.015+0000 7effe1cb8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effdc0b1b20 0x7effdc15add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:29.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.015+0000 7effe1cb8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effdc0b1b20 0x7effdc15add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45790/0 (socket says 172.21.15.154:45790) 2023-09-23T03:26:29.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.015+0000 7effe1cb8640 1 -- 172.21.15.154:0/1948879772 learned_addr learned my addr 172.21.15.154:0/1948879772 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:29.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.015+0000 7effe24b9640 1 --2- 172.21.15.154:0/1948879772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effdc0acce0 0x7effdc0c2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:29.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.015+0000 7effe1cb8640 1 -- 172.21.15.154:0/1948879772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effdc0acce0 msgr2=0x7effdc0c2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:29.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.015+0000 7effe1cb8640 1 --2- 172.21.15.154:0/1948879772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effdc0acce0 0x7effdc0c2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:29.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.015+0000 7effe1cb8640 1 -- 172.21.15.154:0/1948879772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effdc0cee70 msgr2=0x7effdc06fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:29.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.015+0000 7effe1cb8640 1 --2- 172.21.15.154:0/1948879772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effdc0cee70 0x7effdc06fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:29.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.015+0000 7effe1cb8640 1 -- 172.21.15.154:0/1948879772 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effcc009400 con 0x7effdc0b1b20 2023-09-23T03:26:29.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.016+0000 7effe1cb8640 1 --2- 172.21.15.154:0/1948879772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effdc0b1b20 0x7effdc15add0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7effcc034040 tx=0x7effcc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:29.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.016+0000 7effd2ffd640 1 -- 172.21.15.154:0/1948879772 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effcc032900 con 0x7effdc0b1b20 2023-09-23T03:26:29.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.016+0000 7effd2ffd640 1 -- 172.21.15.154:0/1948879772 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effcc032a60 con 0x7effdc0b1b20 2023-09-23T03:26:29.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.016+0000 7effd2ffd640 1 -- 172.21.15.154:0/1948879772 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effcc032e10 con 0x7effdc0b1b20 2023-09-23T03:26:29.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.016+0000 7effe471d640 1 -- 172.21.15.154:0/1948879772 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effdc15be50 con 0x7effdc0b1b20 2023-09-23T03:26:29.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.016+0000 7effe471d640 1 -- 172.21.15.154:0/1948879772 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effdc15c410 con 0x7effdc0b1b20 2023-09-23T03:26:29.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.017+0000 7effe471d640 1 -- 172.21.15.154:0/1948879772 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effdc15c600 con 0x7effdc0b1b20 2023-09-23T03:26:29.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.018+0000 7effd2ffd640 1 -- 172.21.15.154:0/1948879772 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7effcc03b3f0 con 0x7effdc0b1b20 2023-09-23T03:26:29.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.019+0000 7effd2ffd640 1 --2- 172.21.15.154:0/1948879772 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7effc006af30 0x7effc006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:29.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.019+0000 7effd2ffd640 1 -- 172.21.15.154:0/1948879772 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6605+0+0 (secure 0 0 0) 0x7effcc0abc50 con 0x7effdc0b1b20 2023-09-23T03:26:29.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.019+0000 7effe24b9640 1 --2- 172.21.15.154:0/1948879772 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7effc006af30 0x7effc006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:29.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.020+0000 7effe24b9640 1 --2- 172.21.15.154:0/1948879772 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7effc006af30 0x7effc006d3f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7effdc09bf00 tx=0x7effd8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:29.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.023+0000 7effd2ffd640 1 -- 172.21.15.154:0/1948879772 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7effcc076870 con 0x7effdc0b1b20 2023-09-23T03:26:29.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.231+0000 7effe471d640 1 -- 172.21.15.154:0/1948879772 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7effdc073eb0 con 0x7effdc0b1b20 2023-09-23T03:26:29.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.232+0000 7effd2ffd640 1 -- 172.21.15.154:0/1948879772 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4920 (secure 0 0 0) 0x7effcc079e90 con 0x7effdc0b1b20 2023-09-23T03:26:29.233 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:29.233 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":91,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5603,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/4291187069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":4291187069},{"type":"v1","addr":"172.21.15.190:6834","nonce":4291187069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":6337,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/585518074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":585518074},{"type":"v1","addr":"172.21.15.190:6835","nonce":585518074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":8097,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/1240348396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1240348396},{"type":"v1","addr":"172.21.15.154:6836","nonce":1240348396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":8427,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/3181099748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":3181099748},{"type":"v1","addr":"172.21.15.154:6837","nonce":3181099748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:26:26.774375+0000","modified":"2023-09-23T03:26:28.163572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15,16],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-09-23T03:26:29.233 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 91 2023-09-23T03:26:29.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.235+0000 7effe471d640 1 -- 172.21.15.154:0/1948879772 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7effc006af30 msgr2=0x7effc006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:29.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.235+0000 7effe471d640 1 --2- 172.21.15.154:0/1948879772 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7effc006af30 0x7effc006d3f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7effdc09bf00 tx=0x7effd8009040 comp rx=0 tx=0).stop 2023-09-23T03:26:29.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.235+0000 7effe471d640 1 -- 172.21.15.154:0/1948879772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effdc0b1b20 msgr2=0x7effdc15add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:29.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.235+0000 7effe471d640 1 --2- 172.21.15.154:0/1948879772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effdc0b1b20 0x7effdc15add0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7effcc034040 tx=0x7effcc0046c0 comp rx=0 tx=0).stop 2023-09-23T03:26:29.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.235+0000 7effe471d640 1 -- 172.21.15.154:0/1948879772 shutdown_connections 2023-09-23T03:26:29.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.235+0000 7effe471d640 1 --2- 172.21.15.154:0/1948879772 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7effc006af30 0x7effc006d3f0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:29.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.235+0000 7effe471d640 1 --2- 172.21.15.154:0/1948879772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7effdc0cee70 0x7effdc06fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:29.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.235+0000 7effe471d640 1 --2- 172.21.15.154:0/1948879772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effdc0b1b20 0x7effdc15add0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:29.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.235+0000 7effe471d640 1 --2- 172.21.15.154:0/1948879772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7effdc0acce0 0x7effdc0c2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:29.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.235+0000 7effe471d640 1 -- 172.21.15.154:0/1948879772 >> 172.21.15.154:0/1948879772 conn(0x7effdc06a0f0 msgr2=0x7effdc0c75f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:29.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.235+0000 7effe471d640 1 -- 172.21.15.154:0/1948879772 shutdown_connections 2023-09-23T03:26:29.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.235+0000 7effe471d640 1 -- 172.21.15.154:0/1948879772 wait complete. 2023-09-23T03:26:29.253 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-09-23T03:26:29.253 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:26:29.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.465+0000 7f7ae80f3640 1 Processor -- start 2023-09-23T03:26:29.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.465+0000 7f7ae80f3640 1 -- start start 2023-09-23T03:26:29.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.467+0000 7f7ae80f3640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7ae015cb30 0x7f7ae0065ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:29.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.467+0000 7f7ae80f3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ae0160f10 0x7f7ae00c3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:29.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.467+0000 7f7ae80f3640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ae00cd1d0 con 0x7f7ae006e150 2023-09-23T03:26:29.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.467+0000 7f7ae80f3640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ae0069d70 con 0x7f7ae015cb30 2023-09-23T03:26:29.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.467+0000 7f7ae80f3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae006a050 con 0x7f7ae0160f10 2023-09-23T03:26:29.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.467+0000 7f7ae5e8f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7ae015cb30 0x7f7ae0065ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:29.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.468+0000 7f7ae5e8f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f7ae015cb30 0x7f7ae0065ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45804/0 (socket says 172.21.15.154:45804) 2023-09-23T03:26:29.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.468+0000 7f7ae568e640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f7ae006e150 0x7f7ae00ccac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:50538/0 (socket says 172.21.15.154:50538) 2023-09-23T03:26:29.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.468+0000 7f7ae5e8f640 1 -- 172.21.15.154:0/1267520934 learned_addr learned my addr 172.21.15.154:0/1267520934 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:29.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.468+0000 7f7ae6690640 1 --2- 172.21.15.154:0/1267520934 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ae0160f10 0x7f7ae00c3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:29.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.468+0000 7f7ae5e8f640 1 -- 172.21.15.154:0/1267520934 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ae0160f10 msgr2=0x7f7ae00c3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:29.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.468+0000 7f7ae5e8f640 1 --2- 172.21.15.154:0/1267520934 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ae0160f10 0x7f7ae00c3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:29.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.468+0000 7f7ae5e8f640 1 -- 172.21.15.154:0/1267520934 >> v1:172.21.15.154:6789/0 conn(0x7f7ae006e150 legacy=0x7f7ae00ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:29.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.468+0000 7f7ae5e8f640 1 -- 172.21.15.154:0/1267520934 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae006a330 con 0x7f7ae015cb30 2023-09-23T03:26:29.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.468+0000 7f7ae6690640 1 --2- 172.21.15.154:0/1267520934 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ae0160f10 0x7f7ae00c3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:26:29.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.468+0000 7f7ae4e8d640 1 -- 172.21.15.154:0/1267520934 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3200873851 0 0) 0x7f7ae00cd1d0 con 0x7f7ae006e150 2023-09-23T03:26:29.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.469+0000 7f7ae5e8f640 1 --2- 172.21.15.154:0/1267520934 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ae015cb30 0x7f7ae0065ce0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009e50 tx=0x7f7adc02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d453508e050a5e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:29.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.470+0000 7f7ae4e8d640 1 -- 172.21.15.154:0/1267520934 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7adc02cb00 con 0x7f7ae015cb30 2023-09-23T03:26:29.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.470+0000 7f7ae4e8d640 1 -- 172.21.15.154:0/1267520934 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7adc02cc60 con 0x7f7ae015cb30 2023-09-23T03:26:29.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.470+0000 7f7ae4e8d640 1 -- 172.21.15.154:0/1267520934 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7adc0343f0 con 0x7f7ae015cb30 2023-09-23T03:26:29.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.470+0000 7f7ae80f3640 1 -- 172.21.15.154:0/1267520934 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ae015cb30 msgr2=0x7f7ae0065ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:29.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.470+0000 7f7ae80f3640 1 --2- 172.21.15.154:0/1267520934 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ae015cb30 0x7f7ae0065ce0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009e50 tx=0x7f7adc02bf60 comp rx=0 tx=0).stop 2023-09-23T03:26:29.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.470+0000 7f7ae80f3640 1 -- 172.21.15.154:0/1267520934 shutdown_connections 2023-09-23T03:26:29.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.470+0000 7f7ae80f3640 1 --2- 172.21.15.154:0/1267520934 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ae0160f10 0x7f7ae00c3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:29.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.470+0000 7f7ae80f3640 1 --2- 172.21.15.154:0/1267520934 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ae015cb30 0x7f7ae0065ce0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:29.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.470+0000 7f7ae80f3640 1 -- 172.21.15.154:0/1267520934 >> 172.21.15.154:0/1267520934 conn(0x7f7ae00cb790 msgr2=0x7f7ae00cbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:29.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.470+0000 7f7ae80f3640 1 -- 172.21.15.154:0/1267520934 shutdown_connections 2023-09-23T03:26:29.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.470+0000 7f7ae80f3640 1 -- 172.21.15.154:0/1267520934 wait complete. 2023-09-23T03:26:29.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.472+0000 7f7ae80f3640 1 Processor -- start 2023-09-23T03:26:29.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.472+0000 7f7ae80f3640 1 -- start start 2023-09-23T03:26:29.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.473+0000 7f7ae80f3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ae006e150 0x7f7ae014ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:29.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.473+0000 7f7ae80f3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ae015cb30 0x7f7ae00b2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:29.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.473+0000 7f7ae80f3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ae00b2560 0x7f7ae00c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:29.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.473+0000 7f7ae80f3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae00abef0 con 0x7f7ae015cb30 2023-09-23T03:26:29.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.473+0000 7f7ae80f3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae00ac1d0 con 0x7f7ae00b2560 2023-09-23T03:26:29.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.473+0000 7f7ae80f3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae014d3a0 con 0x7f7ae006e150 2023-09-23T03:26:29.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.473+0000 7f7ae568e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ae015cb30 0x7f7ae00b2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:29.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.474+0000 7f7ae568e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ae015cb30 0x7f7ae00b2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45812/0 (socket says 172.21.15.154:45812) 2023-09-23T03:26:29.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.474+0000 7f7ae568e640 1 -- 172.21.15.154:0/408854085 learned_addr learned my addr 172.21.15.154:0/408854085 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:29.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.474+0000 7f7ae5e8f640 1 --2- 172.21.15.154:0/408854085 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ae006e150 0x7f7ae014ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:29.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.474+0000 7f7ae6690640 1 --2- 172.21.15.154:0/408854085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ae00b2560 0x7f7ae00c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:29.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.474+0000 7f7ae568e640 1 -- 172.21.15.154:0/408854085 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ae006e150 msgr2=0x7f7ae014ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:29.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.474+0000 7f7ae568e640 1 --2- 172.21.15.154:0/408854085 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ae006e150 0x7f7ae014ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:29.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.474+0000 7f7ae568e640 1 -- 172.21.15.154:0/408854085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ae00b2560 msgr2=0x7f7ae00c9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:29.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.474+0000 7f7ae568e640 1 --2- 172.21.15.154:0/408854085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ae00b2560 0x7f7ae00c9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:29.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.474+0000 7f7ae568e640 1 -- 172.21.15.154:0/408854085 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7adc009b30 con 0x7f7ae015cb30 2023-09-23T03:26:29.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.474+0000 7f7ae568e640 1 --2- 172.21.15.154:0/408854085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ae015cb30 0x7f7ae00b2020 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f7ad000c4d0 tx=0x7f7ad000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:29.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.474+0000 7f7ad67fc640 1 -- 172.21.15.154:0/408854085 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad00074e0 con 0x7f7ae015cb30 2023-09-23T03:26:29.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.475+0000 7f7ad67fc640 1 -- 172.21.15.154:0/408854085 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad0007660 con 0x7f7ae015cb30 2023-09-23T03:26:29.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.475+0000 7f7ae80f3640 1 -- 172.21.15.154:0/408854085 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ae00b2a00 con 0x7f7ae015cb30 2023-09-23T03:26:29.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.475+0000 7f7ad67fc640 1 -- 172.21.15.154:0/408854085 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad0004a20 con 0x7f7ae015cb30 2023-09-23T03:26:29.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.475+0000 7f7ae80f3640 1 -- 172.21.15.154:0/408854085 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae00b2fc0 con 0x7f7ae015cb30 2023-09-23T03:26:29.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.476+0000 7f7ae80f3640 1 -- 172.21.15.154:0/408854085 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ae0066d30 con 0x7f7ae015cb30 2023-09-23T03:26:29.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.477+0000 7f7ad67fc640 1 -- 172.21.15.154:0/408854085 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f7ad0004b80 con 0x7f7ae015cb30 2023-09-23T03:26:29.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.478+0000 7f7ad67fc640 1 --2- 172.21.15.154:0/408854085 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7ac406af60 0x7f7ac406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:29.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.478+0000 7f7ad67fc640 1 -- 172.21.15.154:0/408854085 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6605+0+0 (secure 0 0 0) 0x7f7ad00890c0 con 0x7f7ae015cb30 2023-09-23T03:26:29.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.478+0000 7f7ae5e8f640 1 --2- 172.21.15.154:0/408854085 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7ac406af60 0x7f7ac406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:29.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.479+0000 7f7ae5e8f640 1 --2- 172.21.15.154:0/408854085 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7ac406af60 0x7f7ac406d420 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f7adc035000 tx=0x7f7adc030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:29.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.482+0000 7f7ad67fc640 1 -- 172.21.15.154:0/408854085 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f7ad0053ce0 con 0x7f7ae015cb30 2023-09-23T03:26:29.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.708+0000 7f7ae80f3640 1 -- 172.21.15.154:0/408854085 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ae00b31e0 con 0x7f7ae015cb30 2023-09-23T03:26:29.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.710+0000 7f7ad67fc640 1 -- 172.21.15.154:0/408854085 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f7ad0018090 con 0x7f7ae015cb30 2023-09-23T03:26:29.710 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:29.710 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":91,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5603,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/4291187069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":4291187069},{"type":"v1","addr":"172.21.15.190:6834","nonce":4291187069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":6337,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/585518074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":585518074},{"type":"v1","addr":"172.21.15.190:6835","nonce":585518074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":8097,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/1240348396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1240348396},{"type":"v1","addr":"172.21.15.154:6836","nonce":1240348396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":8427,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/3181099748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":3181099748},{"type":"v1","addr":"172.21.15.154:6837","nonce":3181099748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:26:26.774375+0000","modified":"2023-09-23T03:26:28.163572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15,16],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-09-23T03:26:29.711 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 91 2023-09-23T03:26:29.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.712+0000 7f7ae80f3640 1 -- 172.21.15.154:0/408854085 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7ac406af60 msgr2=0x7f7ac406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:29.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.712+0000 7f7ae80f3640 1 --2- 172.21.15.154:0/408854085 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7ac406af60 0x7f7ac406d420 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f7adc035000 tx=0x7f7adc030000 comp rx=0 tx=0).stop 2023-09-23T03:26:29.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.712+0000 7f7ae80f3640 1 -- 172.21.15.154:0/408854085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ae015cb30 msgr2=0x7f7ae00b2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:29.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.712+0000 7f7ae80f3640 1 --2- 172.21.15.154:0/408854085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ae015cb30 0x7f7ae00b2020 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f7ad000c4d0 tx=0x7f7ad000c990 comp rx=0 tx=0).stop 2023-09-23T03:26:29.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.712+0000 7f7ae80f3640 1 -- 172.21.15.154:0/408854085 shutdown_connections 2023-09-23T03:26:29.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.712+0000 7f7ae80f3640 1 --2- 172.21.15.154:0/408854085 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7ac406af60 0x7f7ac406d420 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:29.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.712+0000 7f7ae80f3640 1 --2- 172.21.15.154:0/408854085 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7ae00b2560 0x7f7ae00c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:29.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.712+0000 7f7ae80f3640 1 --2- 172.21.15.154:0/408854085 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7ae015cb30 0x7f7ae00b2020 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:29.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.712+0000 7f7ae80f3640 1 --2- 172.21.15.154:0/408854085 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7ae006e150 0x7f7ae014ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:29.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.712+0000 7f7ae80f3640 1 -- 172.21.15.154:0/408854085 >> 172.21.15.154:0/408854085 conn(0x7f7ae00cb790 msgr2=0x7f7ae006aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:29.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.713+0000 7f7ae80f3640 1 -- 172.21.15.154:0/408854085 shutdown_connections 2023-09-23T03:26:29.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.713+0000 7f7ae80f3640 1 -- 172.21.15.154:0/408854085 wait complete. 2023-09-23T03:26:29.734 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:26:29.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.943+0000 7f05cab05640 1 Processor -- start 2023-09-23T03:26:29.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.943+0000 7f05cab05640 1 -- start start 2023-09-23T03:26:29.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.944+0000 7f05cab05640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f05c40acce0 0x7f05c415ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:29.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.944+0000 7f05cab05640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f05c40b1b20 0x7f05c40ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:29.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.944+0000 7f05cab05640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05c40cee70 0x7f05c406cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:29.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.944+0000 7f05cab05640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05c40b2e90 con 0x7f05c40acce0 2023-09-23T03:26:29.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.944+0000 7f05cab05640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05c40cf300 con 0x7f05c40b1b20 2023-09-23T03:26:29.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.944+0000 7f05cab05640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05c415ac30 con 0x7f05c40cee70 2023-09-23T03:26:29.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.944+0000 7f05c88a1640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f05c40acce0 0x7f05c415ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:29.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.944+0000 7f05c88a1640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f05c40acce0 0x7f05c415ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45822/0 (socket says 172.21.15.154:45822) 2023-09-23T03:26:29.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.944+0000 7f05c88a1640 1 -- 172.21.15.154:0/2765009421 learned_addr learned my addr 172.21.15.154:0/2765009421 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:29.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.945+0000 7f05c88a1640 1 -- 172.21.15.154:0/2765009421 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05c40cee70 msgr2=0x7f05c406cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:29.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.945+0000 7f05c90a2640 1 --2- 172.21.15.154:0/2765009421 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05c40cee70 0x7f05c406cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:29.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.945+0000 7f05c3fff640 1 --2- 172.21.15.154:0/2765009421 >> v2:172.21.15.190:3300/0 conn(0x7f05c40b1b20 0x7f05c40ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:29.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.945+0000 7f05c88a1640 1 --2- 172.21.15.154:0/2765009421 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05c40cee70 0x7f05c406cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:29.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.945+0000 7f05c88a1640 1 -- 172.21.15.154:0/2765009421 >> v2:172.21.15.190:3300/0 conn(0x7f05c40b1b20 msgr2=0x7f05c40ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:29.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.945+0000 7f05c88a1640 1 --2- 172.21.15.154:0/2765009421 >> v2:172.21.15.190:3300/0 conn(0x7f05c40b1b20 0x7f05c40ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:29.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.945+0000 7f05c88a1640 1 -- 172.21.15.154:0/2765009421 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c40b1f70 con 0x7f05c40acce0 2023-09-23T03:26:29.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.945+0000 7f05c90a2640 1 --2- 172.21.15.154:0/2765009421 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05c40cee70 0x7f05c406cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:26:29.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.945+0000 7f05c3fff640 1 --2- 172.21.15.154:0/2765009421 >> v2:172.21.15.190:3300/0 conn(0x7f05c40b1b20 0x7f05c40ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:26:29.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.946+0000 7f05c88a1640 1 --2- 172.21.15.154:0/2765009421 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05c40acce0 0x7f05c415ed60 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f05c40c9430 tx=0x7f05b802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c4940ba094f878 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:29.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.946+0000 7f05c37fe640 1 -- 172.21.15.154:0/2765009421 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b80317c0 con 0x7f05c40acce0 2023-09-23T03:26:29.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.947+0000 7f05c37fe640 1 -- 172.21.15.154:0/2765009421 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b8031920 con 0x7f05c40acce0 2023-09-23T03:26:29.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.947+0000 7f05c37fe640 1 -- 172.21.15.154:0/2765009421 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b8031cb0 con 0x7f05c40acce0 2023-09-23T03:26:29.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.947+0000 7f05cab05640 1 -- 172.21.15.154:0/2765009421 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05c40acce0 msgr2=0x7f05c415ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:29.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.947+0000 7f05cab05640 1 --2- 172.21.15.154:0/2765009421 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05c40acce0 0x7f05c415ed60 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f05c40c9430 tx=0x7f05b802b860 comp rx=0 tx=0).stop 2023-09-23T03:26:29.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.947+0000 7f05cab05640 1 -- 172.21.15.154:0/2765009421 shutdown_connections 2023-09-23T03:26:29.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.947+0000 7f05cab05640 1 --2- 172.21.15.154:0/2765009421 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05c40cee70 0x7f05c406cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:29.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.947+0000 7f05cab05640 1 --2- 172.21.15.154:0/2765009421 >> v2:172.21.15.190:3300/0 conn(0x7f05c40b1b20 0x7f05c40ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:29.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.947+0000 7f05cab05640 1 --2- 172.21.15.154:0/2765009421 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05c40acce0 0x7f05c415ed60 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:29.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.947+0000 7f05cab05640 1 -- 172.21.15.154:0/2765009421 >> 172.21.15.154:0/2765009421 conn(0x7f05c406a0f0 msgr2=0x7f05c406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:29.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.947+0000 7f05cab05640 1 -- 172.21.15.154:0/2765009421 shutdown_connections 2023-09-23T03:26:29.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.947+0000 7f05cab05640 1 -- 172.21.15.154:0/2765009421 wait complete. 2023-09-23T03:26:29.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.949+0000 7f05cab05640 1 Processor -- start 2023-09-23T03:26:29.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.949+0000 7f05cab05640 1 -- start start 2023-09-23T03:26:29.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.950+0000 7f05cab05640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05c40acce0 0x7f05c40cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:29.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.950+0000 7f05cab05640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05c40b1b20 0x7f05c40c3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:29.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.950+0000 7f05cab05640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05c40cee70 0x7f05c4157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:29.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.950+0000 7f05cab05640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c40c3f60 con 0x7f05c40b1b20 2023-09-23T03:26:29.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.950+0000 7f05cab05640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c41f5f40 con 0x7f05c40cee70 2023-09-23T03:26:29.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.950+0000 7f05cab05640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05c40c4170 con 0x7f05c40acce0 2023-09-23T03:26:29.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.950+0000 7f05c3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05c40b1b20 0x7f05c40c3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:29.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.951+0000 7f05c3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05c40b1b20 0x7f05c40c3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45832/0 (socket says 172.21.15.154:45832) 2023-09-23T03:26:29.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.951+0000 7f05c3fff640 1 -- 172.21.15.154:0/1918508938 learned_addr learned my addr 172.21.15.154:0/1918508938 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:29.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.951+0000 7f05c88a1640 1 --2- 172.21.15.154:0/1918508938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05c40acce0 0x7f05c40cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:29.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.951+0000 7f05c90a2640 1 --2- 172.21.15.154:0/1918508938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05c40cee70 0x7f05c4157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:29.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.951+0000 7f05c3fff640 1 -- 172.21.15.154:0/1918508938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05c40acce0 msgr2=0x7f05c40cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:29.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.951+0000 7f05c3fff640 1 --2- 172.21.15.154:0/1918508938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05c40acce0 0x7f05c40cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:29.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.951+0000 7f05c3fff640 1 -- 172.21.15.154:0/1918508938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05c40cee70 msgr2=0x7f05c4157550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:29.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.951+0000 7f05c3fff640 1 --2- 172.21.15.154:0/1918508938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05c40cee70 0x7f05c4157550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:29.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.951+0000 7f05c3fff640 1 -- 172.21.15.154:0/1918508938 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b8009470 con 0x7f05c40b1b20 2023-09-23T03:26:29.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.951+0000 7f05c3fff640 1 --2- 172.21.15.154:0/1918508938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05c40b1b20 0x7f05c40c3a20 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f05b000c7f0 tx=0x7f05b000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:29.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.951+0000 7f05c17fa640 1 -- 172.21.15.154:0/1918508938 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b0007a30 con 0x7f05c40b1b20 2023-09-23T03:26:29.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.952+0000 7f05c17fa640 1 -- 172.21.15.154:0/1918508938 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b0007b90 con 0x7f05c40b1b20 2023-09-23T03:26:29.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.952+0000 7f05c17fa640 1 -- 172.21.15.154:0/1918508938 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b000ce40 con 0x7f05c40b1b20 2023-09-23T03:26:29.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.952+0000 7f05cab05640 1 -- 172.21.15.154:0/1918508938 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05c40c4450 con 0x7f05c40b1b20 2023-09-23T03:26:29.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.952+0000 7f05cab05640 1 -- 172.21.15.154:0/1918508938 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05c41f5bd0 con 0x7f05c40b1b20 2023-09-23T03:26:29.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.953+0000 7f05cab05640 1 -- 172.21.15.154:0/1918508938 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05c41f60f0 con 0x7f05c40b1b20 2023-09-23T03:26:29.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.954+0000 7f05c17fa640 1 -- 172.21.15.154:0/1918508938 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f05b00113f0 con 0x7f05c40b1b20 2023-09-23T03:26:29.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.955+0000 7f05c17fa640 1 --2- 172.21.15.154:0/1918508938 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f05a806af60 0x7f05a806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:29.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.955+0000 7f05c17fa640 1 -- 172.21.15.154:0/1918508938 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6605+0+0 (secure 0 0 0) 0x7f05b0089cd0 con 0x7f05c40b1b20 2023-09-23T03:26:29.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.955+0000 7f05c88a1640 1 --2- 172.21.15.154:0/1918508938 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f05a806af60 0x7f05a806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:29.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.956+0000 7f05c88a1640 1 --2- 172.21.15.154:0/1918508938 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f05a806af60 0x7f05a806d420 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f05b8036000 tx=0x7f05b803d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:29.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:29.959+0000 7f05c17fa640 1 -- 172.21.15.154:0/1918508938 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f05c41f60f0 con 0x7f05c40b1b20 2023-09-23T03:26:30.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.187+0000 7f05cab05640 1 -- 172.21.15.154:0/1918508938 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05c41f60b0 con 0x7f05c40b1b20 2023-09-23T03:26:30.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.188+0000 7f05c17fa640 1 -- 172.21.15.154:0/1918508938 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4920 (secure 0 0 0) 0x7f05b001f090 con 0x7f05c40b1b20 2023-09-23T03:26:30.189 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:30.189 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":91,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5603,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/4291187069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":4291187069},{"type":"v1","addr":"172.21.15.190:6834","nonce":4291187069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":6337,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/585518074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":585518074},{"type":"v1","addr":"172.21.15.190:6835","nonce":585518074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":8097,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/1240348396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1240348396},{"type":"v1","addr":"172.21.15.154:6836","nonce":1240348396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":8427,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/3181099748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":3181099748},{"type":"v1","addr":"172.21.15.154:6837","nonce":3181099748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:26:26.774375+0000","modified":"2023-09-23T03:26:28.163572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15,16],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-09-23T03:26:30.190 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 91 2023-09-23T03:26:30.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.191+0000 7f05cab05640 1 -- 172.21.15.154:0/1918508938 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f05a806af60 msgr2=0x7f05a806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:30.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.192+0000 7f05cab05640 1 --2- 172.21.15.154:0/1918508938 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f05a806af60 0x7f05a806d420 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f05b8036000 tx=0x7f05b803d040 comp rx=0 tx=0).stop 2023-09-23T03:26:30.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.192+0000 7f05cab05640 1 -- 172.21.15.154:0/1918508938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05c40b1b20 msgr2=0x7f05c40c3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:30.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.192+0000 7f05cab05640 1 --2- 172.21.15.154:0/1918508938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05c40b1b20 0x7f05c40c3a20 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f05b000c7f0 tx=0x7f05b000cbf0 comp rx=0 tx=0).stop 2023-09-23T03:26:30.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.192+0000 7f05cab05640 1 -- 172.21.15.154:0/1918508938 shutdown_connections 2023-09-23T03:26:30.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.192+0000 7f05cab05640 1 --2- 172.21.15.154:0/1918508938 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f05a806af60 0x7f05a806d420 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:30.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.192+0000 7f05cab05640 1 --2- 172.21.15.154:0/1918508938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05c40cee70 0x7f05c4157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:30.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.192+0000 7f05cab05640 1 --2- 172.21.15.154:0/1918508938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05c40b1b20 0x7f05c40c3a20 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:30.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.192+0000 7f05cab05640 1 --2- 172.21.15.154:0/1918508938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05c40acce0 0x7f05c40cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:30.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.192+0000 7f05cab05640 1 -- 172.21.15.154:0/1918508938 >> 172.21.15.154:0/1918508938 conn(0x7f05c406a0f0 msgr2=0x7f05c406a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:30.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.192+0000 7f05cab05640 1 -- 172.21.15.154:0/1918508938 shutdown_connections 2023-09-23T03:26:30.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.192+0000 7f05cab05640 1 -- 172.21.15.154:0/1918508938 wait complete. 2023-09-23T03:26:30.211 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:26:30.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.426+0000 7fdb25927640 1 Processor -- start 2023-09-23T03:26:30.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.427+0000 7fdb25927640 1 -- start start 2023-09-23T03:26:30.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.428+0000 7fdb25927640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdb200cf500 0x7fdb200ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:30.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.428+0000 7fdb25927640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb200cf8e0 0x7fdb200ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:30.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.428+0000 7fdb25927640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb200d17f0 con 0x7fdb2015c4f0 2023-09-23T03:26:30.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.428+0000 7fdb25927640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb2006be10 con 0x7fdb200cf500 2023-09-23T03:26:30.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.428+0000 7fdb25927640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb200d0470 con 0x7fdb200cf8e0 2023-09-23T03:26:30.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.428+0000 7fdb25126640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fdb2015c4f0 0x7fdb2015c980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:50554/0 (socket says 172.21.15.154:50554) 2023-09-23T03:26:30.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.429+0000 7fdb25126640 1 -- 172.21.15.154:0/3397171019 learned_addr learned my addr 172.21.15.154:0/3397171019 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:26:30.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.429+0000 7fdb24925640 1 --2- 172.21.15.154:0/3397171019 >> v2:172.21.15.190:3300/0 conn(0x7fdb200cf500 0x7fdb200ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:30.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.429+0000 7fdb17fff640 1 --2- 172.21.15.154:0/3397171019 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb200cf8e0 0x7fdb200ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:30.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.429+0000 7fdb177fe640 1 -- 172.21.15.154:0/3397171019 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2363059862 0 0) 0x7fdb200d17f0 con 0x7fdb2015c4f0 2023-09-23T03:26:30.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.429+0000 7fdb177fe640 1 -- 172.21.15.154:0/3397171019 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb08003520 con 0x7fdb2015c4f0 2023-09-23T03:26:30.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.430+0000 7fdb177fe640 1 -- 172.21.15.154:0/3397171019 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 7547156 0 0) 0x7fdb08003520 con 0x7fdb2015c4f0 2023-09-23T03:26:30.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.430+0000 7fdb177fe640 1 -- 172.21.15.154:0/3397171019 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb200cf8e0 msgr2=0x7fdb200ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:30.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.430+0000 7fdb177fe640 1 --2- 172.21.15.154:0/3397171019 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb200cf8e0 0x7fdb200ce590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:30.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.430+0000 7fdb177fe640 1 -- 172.21.15.154:0/3397171019 >> v2:172.21.15.190:3300/0 conn(0x7fdb200cf500 msgr2=0x7fdb200ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:30.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.430+0000 7fdb177fe640 1 --2- 172.21.15.154:0/3397171019 >> v2:172.21.15.190:3300/0 conn(0x7fdb200cf500 0x7fdb200ce050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:30.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.430+0000 7fdb177fe640 1 -- 172.21.15.154:0/3397171019 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb200d0750 con 0x7fdb2015c4f0 2023-09-23T03:26:30.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.430+0000 7fdb177fe640 1 -- 172.21.15.154:0/3397171019 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fdb180019f0 con 0x7fdb2015c4f0 2023-09-23T03:26:30.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.430+0000 7fdb177fe640 1 -- 172.21.15.154:0/3397171019 >> v1:172.21.15.154:6789/0 conn(0x7fdb2015c4f0 legacy=0x7fdb2015c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:30.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.430+0000 7fdb177fe640 1 --2- 172.21.15.154:0/3397171019 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb200cf8e0 0x7fdb08003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:30.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.430+0000 7fdb177fe640 1 --2- 172.21.15.154:0/3397171019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb08003f40 0x7fdb080047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:30.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.430+0000 7fdb177fe640 1 --2- 172.21.15.154:0/3397171019 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb08004d00 0x7fdb08005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:30.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.430+0000 7fdb177fe640 1 -- 172.21.15.154:0/3397171019 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2006be10 con 0x7fdb08003f40 2023-09-23T03:26:30.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.430+0000 7fdb177fe640 1 -- 172.21.15.154:0/3397171019 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb200d0470 con 0x7fdb08004d00 2023-09-23T03:26:30.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.430+0000 7fdb177fe640 1 -- 172.21.15.154:0/3397171019 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0800ba30 con 0x7fdb200cf8e0 2023-09-23T03:26:30.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.430+0000 7fdb17fff640 1 --2- 172.21.15.154:0/3397171019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb08003f40 0x7fdb080047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:30.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.431+0000 7fdb24925640 1 --2- 172.21.15.154:0/3397171019 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb200cf8e0 0x7fdb08003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:30.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.431+0000 7fdb17fff640 1 -- 172.21.15.154:0/3397171019 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb200cf8e0 msgr2=0x7fdb08003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:30.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.431+0000 7fdb17fff640 1 --2- 172.21.15.154:0/3397171019 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb200cf8e0 0x7fdb08003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:30.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.431+0000 7fdb17fff640 1 -- 172.21.15.154:0/3397171019 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb08004d00 msgr2=0x7fdb08005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:30.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.431+0000 7fdb17fff640 1 --2- 172.21.15.154:0/3397171019 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb08004d00 0x7fdb08005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:30.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.431+0000 7fdb17fff640 1 -- 172.21.15.154:0/3397171019 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb08003520 con 0x7fdb08003f40 2023-09-23T03:26:30.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.431+0000 7fdb24925640 1 --2- 172.21.15.154:0/3397171019 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb200cf8e0 0x7fdb08003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:26:30.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.432+0000 7fdb17fff640 1 --2- 172.21.15.154:0/3397171019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb08003f40 0x7fdb080047c0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fdb10009ac0 tx=0x7fdb080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df8c7bd301fc7b6f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:30.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.432+0000 7fdb177fe640 1 -- 172.21.15.154:0/3397171019 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1002c6b0 con 0x7fdb08003f40 2023-09-23T03:26:30.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.432+0000 7fdb177fe640 1 -- 172.21.15.154:0/3397171019 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb1002c810 con 0x7fdb08003f40 2023-09-23T03:26:30.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.432+0000 7fdb177fe640 1 -- 172.21.15.154:0/3397171019 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1002cba0 con 0x7fdb08003f40 2023-09-23T03:26:30.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.433+0000 7fdb25927640 1 -- 172.21.15.154:0/3397171019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb08003f40 msgr2=0x7fdb080047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:30.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.433+0000 7fdb25927640 1 --2- 172.21.15.154:0/3397171019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb08003f40 0x7fdb080047c0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fdb10009ac0 tx=0x7fdb080057f0 comp rx=0 tx=0).stop 2023-09-23T03:26:30.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.433+0000 7fdb25126640 1 -- 172.21.15.154:0/3397171019 reap_dead start 2023-09-23T03:26:30.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.433+0000 7fdb25927640 1 -- 172.21.15.154:0/3397171019 shutdown_connections 2023-09-23T03:26:30.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.433+0000 7fdb25927640 1 -- 172.21.15.154:0/3397171019 >> 172.21.15.154:0/3397171019 conn(0x7fdb2009bbb0 msgr2=0x7fdb200d28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:30.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.433+0000 7fdb25927640 1 -- 172.21.15.154:0/3397171019 shutdown_connections 2023-09-23T03:26:30.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.433+0000 7fdb25927640 1 -- 172.21.15.154:0/3397171019 wait complete. 2023-09-23T03:26:30.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.435+0000 7fdb25927640 1 Processor -- start 2023-09-23T03:26:30.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.435+0000 7fdb25927640 1 -- start start 2023-09-23T03:26:30.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.436+0000 7fdb25927640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb08003f40 0x7fdb200c68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:30.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.436+0000 7fdb25927640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb200cb520 0x7fdb200cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:30.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.436+0000 7fdb25927640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb200c8270 0x7fdb200c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:30.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.436+0000 7fdb25927640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb200cbee0 con 0x7fdb08003f40 2023-09-23T03:26:30.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.436+0000 7fdb25927640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2014ae60 con 0x7fdb200cb520 2023-09-23T03:26:30.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.436+0000 7fdb25927640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2014a560 con 0x7fdb200c8270 2023-09-23T03:26:30.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.436+0000 7fdb24925640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb08003f40 0x7fdb200c68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:30.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.436+0000 7fdb24925640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb08003f40 0x7fdb200c68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45844/0 (socket says 172.21.15.154:45844) 2023-09-23T03:26:30.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.436+0000 7fdb24925640 1 -- 172.21.15.154:0/635355550 learned_addr learned my addr 172.21.15.154:0/635355550 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:30.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.437+0000 7fdb24925640 1 -- 172.21.15.154:0/635355550 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb200c8270 msgr2=0x7fdb200c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:30.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.437+0000 7fdb24925640 1 --2- 172.21.15.154:0/635355550 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb200c8270 0x7fdb200c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:30.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.437+0000 7fdb24925640 1 -- 172.21.15.154:0/635355550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb200cb520 msgr2=0x7fdb200cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:30.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.437+0000 7fdb17fff640 1 --2- 172.21.15.154:0/635355550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb200cb520 0x7fdb200cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:30.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.437+0000 7fdb24925640 1 --2- 172.21.15.154:0/635355550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb200cb520 0x7fdb200cb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:30.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.437+0000 7fdb24925640 1 -- 172.21.15.154:0/635355550 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb200c6e00 con 0x7fdb08003f40 2023-09-23T03:26:30.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.437+0000 7fdb17fff640 1 --2- 172.21.15.154:0/635355550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb200cb520 0x7fdb200cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:26:30.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.437+0000 7fdb24925640 1 --2- 172.21.15.154:0/635355550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb08003f40 0x7fdb200c68c0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c010950 tx=0x7fdb1c010e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:30.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.437+0000 7fdb157fa640 1 -- 172.21.15.154:0/635355550 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c004220 con 0x7fdb08003f40 2023-09-23T03:26:30.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.437+0000 7fdb157fa640 1 -- 172.21.15.154:0/635355550 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb1c004380 con 0x7fdb08003f40 2023-09-23T03:26:30.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.437+0000 7fdb157fa640 1 -- 172.21.15.154:0/635355550 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c00ebc0 con 0x7fdb08003f40 2023-09-23T03:26:30.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.438+0000 7fdb25927640 1 -- 172.21.15.154:0/635355550 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb200c9080 con 0x7fdb08003f40 2023-09-23T03:26:30.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.438+0000 7fdb25927640 1 -- 172.21.15.154:0/635355550 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb200c9640 con 0x7fdb08003f40 2023-09-23T03:26:30.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.439+0000 7fdb25927640 1 -- 172.21.15.154:0/635355550 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb2004e920 con 0x7fdb08003f40 2023-09-23T03:26:30.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.440+0000 7fdb157fa640 1 -- 172.21.15.154:0/635355550 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fdb1c00ed20 con 0x7fdb08003f40 2023-09-23T03:26:30.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.441+0000 7fdb157fa640 1 --2- 172.21.15.154:0/635355550 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdb0c06af30 0x7fdb0c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:30.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.441+0000 7fdb157fa640 1 -- 172.21.15.154:0/635355550 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6605+0+0 (secure 0 0 0) 0x7fdb1c089e50 con 0x7fdb08003f40 2023-09-23T03:26:30.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.441+0000 7fdb17fff640 1 --2- 172.21.15.154:0/635355550 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdb0c06af30 0x7fdb0c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:30.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.442+0000 7fdb17fff640 1 --2- 172.21.15.154:0/635355550 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdb0c06af30 0x7fdb0c06d3f0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fdb10036000 tx=0x7fdb10030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:30.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.445+0000 7fdb157fa640 1 -- 172.21.15.154:0/635355550 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fdb1c054b10 con 0x7fdb08003f40 2023-09-23T03:26:30.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.670+0000 7fdb25927640 1 -- 172.21.15.154:0/635355550 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb200d0000 con 0x7fdb08003f40 2023-09-23T03:26:30.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.671+0000 7fdb157fa640 1 -- 172.21.15.154:0/635355550 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4920 (secure 0 0 0) 0x7fdb1c058130 con 0x7fdb08003f40 2023-09-23T03:26:30.671 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:30.671 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":91,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5603,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/4291187069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":4291187069},{"type":"v1","addr":"172.21.15.190:6834","nonce":4291187069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":6337,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/585518074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":585518074},{"type":"v1","addr":"172.21.15.190:6835","nonce":585518074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":8097,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/1240348396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1240348396},{"type":"v1","addr":"172.21.15.154:6836","nonce":1240348396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":8427,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/3181099748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":3181099748},{"type":"v1","addr":"172.21.15.154:6837","nonce":3181099748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:26:26.774375+0000","modified":"2023-09-23T03:26:28.163572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15,16],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-09-23T03:26:30.672 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 91 2023-09-23T03:26:30.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.673+0000 7fdb25927640 1 -- 172.21.15.154:0/635355550 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdb0c06af30 msgr2=0x7fdb0c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:30.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.673+0000 7fdb25927640 1 --2- 172.21.15.154:0/635355550 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdb0c06af30 0x7fdb0c06d3f0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fdb10036000 tx=0x7fdb10030000 comp rx=0 tx=0).stop 2023-09-23T03:26:30.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.673+0000 7fdb25927640 1 -- 172.21.15.154:0/635355550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb08003f40 msgr2=0x7fdb200c68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:30.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.673+0000 7fdb25927640 1 --2- 172.21.15.154:0/635355550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb08003f40 0x7fdb200c68c0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c010950 tx=0x7fdb1c010e10 comp rx=0 tx=0).stop 2023-09-23T03:26:30.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.673+0000 7fdb25927640 1 -- 172.21.15.154:0/635355550 shutdown_connections 2023-09-23T03:26:30.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.673+0000 7fdb25927640 1 --2- 172.21.15.154:0/635355550 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdb0c06af30 0x7fdb0c06d3f0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:30.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.673+0000 7fdb25927640 1 --2- 172.21.15.154:0/635355550 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdb200c8270 0x7fdb200c86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:30.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.673+0000 7fdb25927640 1 --2- 172.21.15.154:0/635355550 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdb200cb520 0x7fdb200cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:30.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.673+0000 7fdb25927640 1 --2- 172.21.15.154:0/635355550 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdb08003f40 0x7fdb200c68c0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:30.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.673+0000 7fdb25927640 1 -- 172.21.15.154:0/635355550 >> 172.21.15.154:0/635355550 conn(0x7fdb2009bbb0 msgr2=0x7fdb201e8040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:30.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.674+0000 7fdb25927640 1 -- 172.21.15.154:0/635355550 shutdown_connections 2023-09-23T03:26:30.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.674+0000 7fdb25927640 1 -- 172.21.15.154:0/635355550 wait complete. 2023-09-23T03:26:30.693 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:26:30.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.905+0000 7fba6b239640 1 Processor -- start 2023-09-23T03:26:30.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.905+0000 7fba6b239640 1 -- start start 2023-09-23T03:26:30.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.906+0000 7fba6b239640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fba640acce0 0x7fba6415ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:30.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.906+0000 7fba6b239640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba6415ac30 con 0x7fba640b1b20 2023-09-23T03:26:30.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.906+0000 7fba6b239640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba640b1f70 con 0x7fba640cf040 2023-09-23T03:26:30.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.906+0000 7fba6b239640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba640b2e90 con 0x7fba640acce0 2023-09-23T03:26:30.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.907+0000 7fba68fd5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fba640acce0 0x7fba6415ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:30.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.907+0000 7fba63fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fba640b1b20 0x7fba640ce930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:50566/0 (socket says 172.21.15.154:50566) 2023-09-23T03:26:30.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.907+0000 7fba63fff640 1 -- 172.21.15.154:0/377221434 learned_addr learned my addr 172.21.15.154:0/377221434 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:26:30.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.907+0000 7fba68fd5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fba640acce0 0x7fba6415ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45848/0 (socket says 172.21.15.154:45848) 2023-09-23T03:26:30.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.907+0000 7fba68fd5640 1 -- 172.21.15.154:0/377221434 >> v1:172.21.15.190:6789/0 conn(0x7fba640cf040 legacy=0x7fba640ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:30.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.907+0000 7fba68fd5640 1 -- 172.21.15.154:0/377221434 >> v1:172.21.15.154:6789/0 conn(0x7fba640b1b20 legacy=0x7fba640ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:30.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.907+0000 7fba68fd5640 1 -- 172.21.15.154:0/377221434 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba6415f2a0 con 0x7fba640acce0 2023-09-23T03:26:30.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.908+0000 7fba68fd5640 1 --2- 172.21.15.154:0/377221434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba640acce0 0x7fba6415ed60 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fba58009790 tx=0x7fba5802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=182ce345f546deaf server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:30.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.909+0000 7fba637fe640 1 -- 172.21.15.154:0/377221434 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba58031780 con 0x7fba640acce0 2023-09-23T03:26:30.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.909+0000 7fba637fe640 1 -- 172.21.15.154:0/377221434 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba580318e0 con 0x7fba640acce0 2023-09-23T03:26:30.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.909+0000 7fba637fe640 1 -- 172.21.15.154:0/377221434 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba58031c70 con 0x7fba640acce0 2023-09-23T03:26:30.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.909+0000 7fba6b239640 1 -- 172.21.15.154:0/377221434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba640acce0 msgr2=0x7fba6415ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:30.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.909+0000 7fba6b239640 1 --2- 172.21.15.154:0/377221434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba640acce0 0x7fba6415ed60 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fba58009790 tx=0x7fba5802b820 comp rx=0 tx=0).stop 2023-09-23T03:26:30.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.909+0000 7fba6b239640 1 -- 172.21.15.154:0/377221434 shutdown_connections 2023-09-23T03:26:30.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.909+0000 7fba6b239640 1 --2- 172.21.15.154:0/377221434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba640acce0 0x7fba6415ed60 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:30.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.909+0000 7fba6b239640 1 -- 172.21.15.154:0/377221434 >> 172.21.15.154:0/377221434 conn(0x7fba6406a0f0 msgr2=0x7fba6406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:30.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.909+0000 7fba6b239640 1 -- 172.21.15.154:0/377221434 shutdown_connections 2023-09-23T03:26:30.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.910+0000 7fba6b239640 1 -- 172.21.15.154:0/377221434 wait complete. 2023-09-23T03:26:30.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.911+0000 7fba6b239640 1 Processor -- start 2023-09-23T03:26:30.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.911+0000 7fba6b239640 1 -- start start 2023-09-23T03:26:30.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.912+0000 7fba6b239640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba640acce0 0x7fba64149d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:30.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.912+0000 7fba6b239640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba640b1b20 0x7fba641f1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:30.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.912+0000 7fba6b239640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba640cf040 0x7fba64070f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:30.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.912+0000 7fba6b239640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba640c3ce0 con 0x7fba640cf040 2023-09-23T03:26:30.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.912+0000 7fba6b239640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba641f22d0 con 0x7fba640acce0 2023-09-23T03:26:30.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.912+0000 7fba6b239640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba6415b240 con 0x7fba640b1b20 2023-09-23T03:26:30.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.913+0000 7fba697d6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba640cf040 0x7fba64070f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:30.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.913+0000 7fba697d6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba640cf040 0x7fba64070f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45856/0 (socket says 172.21.15.154:45856) 2023-09-23T03:26:30.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.913+0000 7fba697d6640 1 -- 172.21.15.154:0/1380425103 learned_addr learned my addr 172.21.15.154:0/1380425103 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:30.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.913+0000 7fba68fd5640 1 --2- 172.21.15.154:0/1380425103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba640acce0 0x7fba64149d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:30.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.913+0000 7fba63fff640 1 --2- 172.21.15.154:0/1380425103 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba640b1b20 0x7fba641f1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:30.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.913+0000 7fba697d6640 1 -- 172.21.15.154:0/1380425103 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba640b1b20 msgr2=0x7fba641f1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:30.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.913+0000 7fba697d6640 1 --2- 172.21.15.154:0/1380425103 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba640b1b20 0x7fba641f1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:30.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.913+0000 7fba697d6640 1 -- 172.21.15.154:0/1380425103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba640acce0 msgr2=0x7fba64149d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:30.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.913+0000 7fba697d6640 1 --2- 172.21.15.154:0/1380425103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba640acce0 0x7fba64149d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:30.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.913+0000 7fba697d6640 1 -- 172.21.15.154:0/1380425103 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba580093a0 con 0x7fba640cf040 2023-09-23T03:26:30.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.913+0000 7fba63fff640 1 --2- 172.21.15.154:0/1380425103 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba640b1b20 0x7fba641f1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:26:30.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.913+0000 7fba697d6640 1 --2- 172.21.15.154:0/1380425103 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba640cf040 0x7fba64070f30 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fba5c00b9e0 tx=0x7fba5c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:30.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.914+0000 7fba617fa640 1 -- 172.21.15.154:0/1380425103 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c003b70 con 0x7fba640cf040 2023-09-23T03:26:30.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.914+0000 7fba617fa640 1 -- 172.21.15.154:0/1380425103 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba5c003cd0 con 0x7fba640cf040 2023-09-23T03:26:30.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.914+0000 7fba617fa640 1 -- 172.21.15.154:0/1380425103 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c00cb00 con 0x7fba640cf040 2023-09-23T03:26:30.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.914+0000 7fba6b239640 1 -- 172.21.15.154:0/1380425103 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba6415b520 con 0x7fba640cf040 2023-09-23T03:26:30.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.914+0000 7fba6b239640 1 -- 172.21.15.154:0/1380425103 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba640c3900 con 0x7fba640cf040 2023-09-23T03:26:30.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.915+0000 7fba6b239640 1 -- 172.21.15.154:0/1380425103 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba640c3e20 con 0x7fba640cf040 2023-09-23T03:26:30.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.916+0000 7fba617fa640 1 -- 172.21.15.154:0/1380425103 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fba5c00cc60 con 0x7fba640cf040 2023-09-23T03:26:30.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.917+0000 7fba617fa640 1 --2- 172.21.15.154:0/1380425103 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fba4806af30 0x7fba4806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:30.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.917+0000 7fba617fa640 1 -- 172.21.15.154:0/1380425103 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6605+0+0 (secure 0 0 0) 0x7fba5c0897f0 con 0x7fba640cf040 2023-09-23T03:26:30.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.917+0000 7fba68fd5640 1 --2- 172.21.15.154:0/1380425103 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fba4806af30 0x7fba4806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:30.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.918+0000 7fba68fd5640 1 --2- 172.21.15.154:0/1380425103 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fba4806af30 0x7fba4806d3f0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fba58036000 tx=0x7fba5803d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:30.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:30.921+0000 7fba617fa640 1 -- 172.21.15.154:0/1380425103 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fba640c3e20 con 0x7fba640cf040 2023-09-23T03:26:31.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.131+0000 7fba6b239640 1 -- 172.21.15.154:0/1380425103 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba640c3e20 con 0x7fba640cf040 2023-09-23T03:26:31.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.132+0000 7fba617fa640 1 -- 172.21.15.154:0/1380425103 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4920 (secure 0 0 0) 0x7fba640c3e20 con 0x7fba640cf040 2023-09-23T03:26:31.133 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:31.133 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":91,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5603,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/4291187069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":4291187069},{"type":"v1","addr":"172.21.15.190:6834","nonce":4291187069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":6337,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/585518074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":585518074},{"type":"v1","addr":"172.21.15.190:6835","nonce":585518074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":8097,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/1240348396","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1240348396},{"type":"v1","addr":"172.21.15.154:6836","nonce":1240348396}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":79},{"gid":8427,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/3181099748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":3181099748},{"type":"v1","addr":"172.21.15.154:6837","nonce":3181099748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:26:26.774375+0000","modified":"2023-09-23T03:26:28.163572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[15,16],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-09-23T03:26:31.133 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 91 2023-09-23T03:26:31.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.135+0000 7fba6b239640 1 -- 172.21.15.154:0/1380425103 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fba4806af30 msgr2=0x7fba4806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.135+0000 7fba6b239640 1 --2- 172.21.15.154:0/1380425103 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fba4806af30 0x7fba4806d3f0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fba58036000 tx=0x7fba5803d040 comp rx=0 tx=0).stop 2023-09-23T03:26:31.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.135+0000 7fba6b239640 1 -- 172.21.15.154:0/1380425103 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba640cf040 msgr2=0x7fba64070f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.135+0000 7fba6b239640 1 --2- 172.21.15.154:0/1380425103 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba640cf040 0x7fba64070f30 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fba5c00b9e0 tx=0x7fba5c00bea0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.135+0000 7fba6b239640 1 -- 172.21.15.154:0/1380425103 shutdown_connections 2023-09-23T03:26:31.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.135+0000 7fba6b239640 1 --2- 172.21.15.154:0/1380425103 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fba4806af30 0x7fba4806d3f0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.135+0000 7fba6b239640 1 --2- 172.21.15.154:0/1380425103 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba640cf040 0x7fba64070f30 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.135+0000 7fba6b239640 1 --2- 172.21.15.154:0/1380425103 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fba640b1b20 0x7fba641f1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.135+0000 7fba6b239640 1 --2- 172.21.15.154:0/1380425103 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fba640acce0 0x7fba64149d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.135+0000 7fba6b239640 1 -- 172.21.15.154:0/1380425103 >> 172.21.15.154:0/1380425103 conn(0x7fba6406a0f0 msgr2=0x7fba640726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:31.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.135+0000 7fba6b239640 1 -- 172.21.15.154:0/1380425103 shutdown_connections 2023-09-23T03:26:31.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.135+0000 7fba6b239640 1 -- 172.21.15.154:0/1380425103 wait complete. 2023-09-23T03:26:31.153 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset session 2023-09-23T03:26:31.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.254+0000 7f7a9be42100 1 -- start start 2023-09-23T03:26:31.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.254+0000 7f7a9be42100 1 Processor -- start 2023-09-23T03:26:31.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.255+0000 7f7a9be42100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556400700400 0x5563ff9a9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.255+0000 7f7a9be42100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563ff939c00 0x5563ff9a9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.255+0000 7f7a9be42100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556400700800 0x5563ff9a9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.255+0000 7f7a9be42100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563ff985180 con 0x556400700400 2023-09-23T03:26:31.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.255+0000 7f7a9be42100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563ff984b40 con 0x5563ff939c00 2023-09-23T03:26:31.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.255+0000 7f7a9be42100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5563ff984000 con 0x556400700800 2023-09-23T03:26:31.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.256+0000 7f7a9a5ea640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556400700400 0x5563ff9a9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.256+0000 7f7a9a5ea640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556400700400 0x5563ff9a9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45866/0 (socket says 172.21.15.154:45866) 2023-09-23T03:26:31.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.256+0000 7f7a9a5ea640 1 -- 172.21.15.154:0/2699459843 learned_addr learned my addr 172.21.15.154:0/2699459843 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:31.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.256+0000 7f7a9a5ea640 1 -- 172.21.15.154:0/2699459843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556400700800 msgr2=0x5563ff9a9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:31.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.256+0000 7f7a9a5ea640 1 --2- 172.21.15.154:0/2699459843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556400700800 0x5563ff9a9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.256+0000 7f7a9a5ea640 1 -- 172.21.15.154:0/2699459843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563ff939c00 msgr2=0x5563ff9a9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:31.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.256+0000 7f7a9a5ea640 1 --2- 172.21.15.154:0/2699459843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563ff939c00 0x5563ff9a9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.256+0000 7f7a9a5ea640 1 -- 172.21.15.154:0/2699459843 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563ff96cea0 con 0x556400700400 2023-09-23T03:26:31.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.256+0000 7f7a9a5ea640 1 --2- 172.21.15.154:0/2699459843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556400700400 0x5563ff9a9600 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x55640075e9f0 tx=0x5564007fa120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f1669e094de03a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.257+0000 7f7a97de5640 1 -- 172.21.15.154:0/2699459843 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5563ff983600 con 0x556400700400 2023-09-23T03:26:31.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.257+0000 7f7a97de5640 1 -- 172.21.15.154:0/2699459843 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5563ff95f800 con 0x556400700400 2023-09-23T03:26:31.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.257+0000 7f7a97de5640 1 -- 172.21.15.154:0/2699459843 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5563ff983080 con 0x556400700400 2023-09-23T03:26:31.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.257+0000 7f7a9be42100 1 -- 172.21.15.154:0/2699459843 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5563ff96d520 con 0x556400700400 2023-09-23T03:26:31.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.257+0000 7f7a9be42100 1 -- 172.21.15.154:0/2699459843 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5563ff96d6c0 con 0x556400700400 2023-09-23T03:26:31.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.257+0000 7f7a9be42100 4 waiting for MDS map... 2023-09-23T03:26:31.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.257+0000 7f7a97de5640 1 -- 172.21.15.154:0/2699459843 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(110..110 src has 1..110) v4 ==== 6605+0+0 (secure 0 0 0) 0x5563ff964380 con 0x556400700400 2023-09-23T03:26:31.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.257+0000 7f7a97de5640 1 -- 172.21.15.154:0/2699459843 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 91) v1 ==== 3306+0+0 (secure 0 0 0) 0x5563ff98cf00 con 0x556400700400 2023-09-23T03:26:31.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.257+0000 7f7a9be42100 4 Got MDS map 91 2023-09-23T03:26:31.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.257+0000 7f7a9be42100 10 main: main 2023-09-23T03:26:31.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.257+0000 7f7a9be42100 4 main: connecting to RADOS... 2023-09-23T03:26:31.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.258+0000 7f7a9be42100 1 Processor -- start 2023-09-23T03:26:31.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.258+0000 7f7a9be42100 1 -- start start 2023-09-23T03:26:31.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.259+0000 7f7a9be42100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556400800c00 0x5563ff9a8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.259+0000 7f7a9be42100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556400801000 0x5563ff9aa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.259+0000 7f7a9be42100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556400801400 0x5563ff9aa680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.259+0000 7f7a9be42100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563ff984a00 con 0x556400800c00 2023-09-23T03:26:31.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.259+0000 7f7a9be42100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563ffa06dc0 con 0x556400801400 2023-09-23T03:26:31.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.259+0000 7f7a9be42100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5563ffa07180 con 0x556400801000 2023-09-23T03:26:31.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.259+0000 7f7a9adeb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556400800c00 0x5563ff9a8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.259+0000 7f7a9adeb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556400800c00 0x5563ff9a8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45870/0 (socket says 172.21.15.154:45870) 2023-09-23T03:26:31.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.259+0000 7f7a9adeb640 1 -- 172.21.15.154:0/2337512291 learned_addr learned my addr 172.21.15.154:0/2337512291 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:31.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.260+0000 7f7a9adeb640 1 -- 172.21.15.154:0/2337512291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556400801000 msgr2=0x5563ff9aa100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:31.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.260+0000 7f7a9adeb640 1 --2- 172.21.15.154:0/2337512291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556400801000 0x5563ff9aa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.260+0000 7f7a9adeb640 1 -- 172.21.15.154:0/2337512291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556400801400 msgr2=0x5563ff9aa680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:31.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.260+0000 7f7a9adeb640 1 --2- 172.21.15.154:0/2337512291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556400801400 0x5563ff9aa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.260+0000 7f7a9adeb640 1 -- 172.21.15.154:0/2337512291 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563ff96d860 con 0x556400800c00 2023-09-23T03:26:31.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.260+0000 7f7a9adeb640 1 --2- 172.21.15.154:0/2337512291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556400800c00 0x5563ff9a8b00 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x55640075ff50 tx=0x556400818c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7bbc0846fcce194 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.260+0000 7f7a95de1640 1 -- 172.21.15.154:0/2337512291 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5563ff983600 con 0x556400800c00 2023-09-23T03:26:31.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.260+0000 7f7a95de1640 1 -- 172.21.15.154:0/2337512291 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55640080a300 con 0x556400800c00 2023-09-23T03:26:31.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.260+0000 7f7a95de1640 1 -- 172.21.15.154:0/2337512291 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5563ff983080 con 0x556400800c00 2023-09-23T03:26:31.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.261+0000 7f7a9be42100 1 -- 172.21.15.154:0/2337512291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556400800c00 msgr2=0x5563ff9a8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.261+0000 7f7a9be42100 1 --2- 172.21.15.154:0/2337512291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556400800c00 0x5563ff9a8b00 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x55640075ff50 tx=0x556400818c00 comp rx=0 tx=0).stop 2023-09-23T03:26:31.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.261+0000 7f7a9be42100 1 -- 172.21.15.154:0/2337512291 shutdown_connections 2023-09-23T03:26:31.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.261+0000 7f7a9be42100 1 --2- 172.21.15.154:0/2337512291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556400801400 0x5563ff9aa680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.261+0000 7f7a9be42100 1 --2- 172.21.15.154:0/2337512291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556400801000 0x5563ff9aa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.261+0000 7f7a9be42100 1 --2- 172.21.15.154:0/2337512291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556400800c00 0x5563ff9a8b00 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.261+0000 7f7a9be42100 1 -- 172.21.15.154:0/2337512291 >> 172.21.15.154:0/2337512291 conn(0x556400700000 msgr2=0x5564006b1800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:31.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.261+0000 7f7a9be42100 1 -- 172.21.15.154:0/2337512291 shutdown_connections 2023-09-23T03:26:31.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.261+0000 7f7a9be42100 1 -- 172.21.15.154:0/2337512291 wait complete. 2023-09-23T03:26:31.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.261+0000 7f7a9be42100 1 Processor -- start 2023-09-23T03:26:31.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.262+0000 7f7a9be42100 1 -- start start 2023-09-23T03:26:31.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.262+0000 7f7a9be42100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556400800c00 0x5563ff9a8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.262+0000 7f7a9be42100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556400801000 0x5563ff9aa100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.262+0000 7f7a9be42100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556400801400 0x5563ff9aa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.262+0000 7f7a9be42100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563ff984b40 con 0x556400801400 2023-09-23T03:26:31.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.262+0000 7f7a9be42100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563ff985180 con 0x556400801000 2023-09-23T03:26:31.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.262+0000 7f7a9be42100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5563ff984140 con 0x556400800c00 2023-09-23T03:26:31.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.263+0000 7f7a99de9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556400801400 0x5563ff9aa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.263+0000 7f7a99de9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556400801400 0x5563ff9aa680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45884/0 (socket says 172.21.15.154:45884) 2023-09-23T03:26:31.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.263+0000 7f7a99de9640 1 -- 172.21.15.154:0/234609238 learned_addr learned my addr 172.21.15.154:0/234609238 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:31.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.263+0000 7f7a9a5ea640 1 --2- 172.21.15.154:0/234609238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556400801000 0x5563ff9aa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.263+0000 7f7a9adeb640 1 --2- 172.21.15.154:0/234609238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556400800c00 0x5563ff9a8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.263+0000 7f7a99de9640 1 -- 172.21.15.154:0/234609238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556400800c00 msgr2=0x5563ff9a8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.263+0000 7f7a99de9640 1 --2- 172.21.15.154:0/234609238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556400800c00 0x5563ff9a8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.263+0000 7f7a99de9640 1 -- 172.21.15.154:0/234609238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556400801000 msgr2=0x5563ff9aa100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.263+0000 7f7a99de9640 1 --2- 172.21.15.154:0/234609238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556400801000 0x5563ff9aa100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.263+0000 7f7a99de9640 1 -- 172.21.15.154:0/234609238 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563ff96c340 con 0x556400801400 2023-09-23T03:26:31.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.263+0000 7f7a9a5ea640 1 --2- 172.21.15.154:0/234609238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x556400801000 0x5563ff9aa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:26:31.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.263+0000 7f7a99de9640 1 --2- 172.21.15.154:0/234609238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556400801400 0x5563ff9aa680 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x556400870bd0 tx=0x556400818e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.264+0000 7f7a945de640 1 -- 172.21.15.154:0/234609238 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5563ff9831e0 con 0x556400801400 2023-09-23T03:26:31.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.264+0000 7f7a945de640 1 -- 172.21.15.154:0/234609238 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55640080b380 con 0x556400801400 2023-09-23T03:26:31.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.264+0000 7f7a9be42100 1 -- 172.21.15.154:0/234609238 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5563ff96d1e0 con 0x556400801400 2023-09-23T03:26:31.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.264+0000 7f7a945de640 1 -- 172.21.15.154:0/234609238 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5563ff983600 con 0x556400801400 2023-09-23T03:26:31.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.264+0000 7f7a9be42100 1 -- 172.21.15.154:0/234609238 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5563ff96cea0 con 0x556400801400 2023-09-23T03:26:31.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.264+0000 7f7a9be42100 4 main: resolving pool 14 2023-09-23T03:26:31.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.265+0000 7f7a945de640 1 -- 172.21.15.154:0/234609238 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x5563ff96b080 con 0x556400801400 2023-09-23T03:26:31.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.266+0000 7f7a945de640 1 --2- 172.21.15.154:0/234609238 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5563ffc8c000 0x5563ff9ab700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.266+0000 7f7a945de640 1 -- 172.21.15.154:0/234609238 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6605+0+0 (secure 0 0 0) 0x5563ff964540 con 0x556400801400 2023-09-23T03:26:31.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.266+0000 7f7a9be42100 4 main: creating IoCtx.. 2023-09-23T03:26:31.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.266+0000 7f7a9be42100 1 --2- 172.21.15.154:0/234609238 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x556400801800 0x5563ff9aac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.266+0000 7f7a9adeb640 1 --2- 172.21.15.154:0/234609238 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5563ffc8c000 0x5563ff9ab700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.266+0000 7f7a9be42100 1 -- 172.21.15.154:0/234609238 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:1 14.b 14:d0630e4c:::mds0_sessionmap:head [omap-clear,omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5563ff995880 con 0x556400801800 2023-09-23T03:26:31.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.266+0000 7f7a9a5ea640 1 --2- 172.21.15.154:0/234609238 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x556400801800 0x5563ff9aac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.267+0000 7f7a9adeb640 1 --2- 172.21.15.154:0/234609238 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5563ffc8c000 0x5563ff9ab700 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x55640075ff50 tx=0x5564006c9800 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.267+0000 7f7a9a5ea640 1 --2- 172.21.15.154:0/234609238 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x556400801800 0x5563ff9aac00 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.268+0000 7f7a9a5ea640 1 -- 172.21.15.154:0/234609238 <== osd.0 v2:172.21.15.154:6800/2920938138 1 ==== osd_op_reply(1 mds0_sessionmap [omap-clear,omap-set-header] v110'5 uv5 ondisk = 0) v8 ==== 201+0+0 (crc 0 0 0) 0x5564007f4b40 con 0x556400801800 2023-09-23T03:26:31.275 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:26:31.275 INFO:teuthology.orchestra.run.smithi154.stdout: "0": { 2023-09-23T03:26:31.275 INFO:teuthology.orchestra.run.smithi154.stdout: "data": {}, 2023-09-23T03:26:31.275 INFO:teuthology.orchestra.run.smithi154.stdout: "result": 0 2023-09-23T03:26:31.275 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:26:31.275 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:26:31.275 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:31.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.269+0000 7f7a9be42100 1 -- 172.21.15.154:0/234609238 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x556400801800 msgr2=0x5563ff9aac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.269+0000 7f7a9be42100 1 --2- 172.21.15.154:0/234609238 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x556400801800 0x5563ff9aac00 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.269+0000 7f7a9be42100 1 -- 172.21.15.154:0/234609238 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5563ffc8c000 msgr2=0x5563ff9ab700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.269+0000 7f7a9be42100 1 --2- 172.21.15.154:0/234609238 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5563ffc8c000 0x5563ff9ab700 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x55640075ff50 tx=0x5564006c9800 comp rx=0 tx=0).stop 2023-09-23T03:26:31.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.269+0000 7f7a9be42100 1 -- 172.21.15.154:0/234609238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556400801400 msgr2=0x5563ff9aa680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.269+0000 7f7a9be42100 1 --2- 172.21.15.154:0/234609238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556400801400 0x5563ff9aa680 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x556400870bd0 tx=0x556400818e40 comp rx=0 tx=0).stop 2023-09-23T03:26:31.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.269+0000 7f7a99de9640 1 -- 172.21.15.154:0/234609238 reap_dead start 2023-09-23T03:26:31.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.269+0000 7f7a9be42100 1 -- 172.21.15.154:0/234609238 shutdown_connections 2023-09-23T03:26:31.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.269+0000 7f7a9be42100 1 -- 172.21.15.154:0/234609238 >> 172.21.15.154:0/234609238 conn(0x556400700000 msgr2=0x5564006b1800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:31.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.269+0000 7f7a9be42100 1 -- 172.21.15.154:0/234609238 shutdown_connections 2023-09-23T03:26:31.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.269+0000 7f7a9be42100 1 -- 172.21.15.154:0/234609238 wait complete. 2023-09-23T03:26:31.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.269+0000 7f7a9be42100 1 -- 172.21.15.154:0/2699459843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556400700400 msgr2=0x5563ff9a9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.269+0000 7f7a9be42100 1 --2- 172.21.15.154:0/2699459843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556400700400 0x5563ff9a9600 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x55640075e9f0 tx=0x5564007fa120 comp rx=0 tx=0).stop 2023-09-23T03:26:31.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.269+0000 7f7a9be42100 1 -- 172.21.15.154:0/2699459843 shutdown_connections 2023-09-23T03:26:31.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.269+0000 7f7a9be42100 1 --2- 172.21.15.154:0/2699459843 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x556400700800 0x5563ff9a9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.269+0000 7f7a9be42100 1 --2- 172.21.15.154:0/2699459843 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5563ff939c00 0x5563ff9a9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.269+0000 7f7a9be42100 1 --2- 172.21.15.154:0/2699459843 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556400700400 0x5563ff9a9600 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.269+0000 7f7a9be42100 1 -- 172.21.15.154:0/2699459843 >> 172.21.15.154:0/2699459843 conn(0x5563ff939000 msgr2=0x5564006b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:31.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.270+0000 7f7a9be42100 1 -- 172.21.15.154:0/2699459843 shutdown_connections 2023-09-23T03:26:31.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.270+0000 7f7a9be42100 1 -- 172.21.15.154:0/2699459843 wait complete. 2023-09-23T03:26:31.280 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'session'] in time 0:00:00.127240, result: { "0": { "data": {}, "result": 0 } } 2023-09-23T03:26:31.280 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset snap 2023-09-23T03:26:31.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.362+0000 7fbfa923a100 1 -- start start 2023-09-23T03:26:31.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.362+0000 7fbfa923a100 1 Processor -- start 2023-09-23T03:26:31.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.362+0000 7fbfa923a100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f0846e2400 0x55f08397f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.362+0000 7fbfa923a100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f0846e2800 0x55f08397eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.362+0000 7fbfa923a100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f0846e3c00 0x55f08397fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.362+0000 7fbfa923a100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f08395a140 con 0x55f0846e3c00 2023-09-23T03:26:31.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.362+0000 7fbfa923a100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f08395b180 con 0x55f0846e2400 2023-09-23T03:26:31.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.362+0000 7fbfa923a100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f0839dcdc0 con 0x55f0846e2800 2023-09-23T03:26:31.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.363+0000 7fbfa79ad640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f0846e3c00 0x55f08397fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.363+0000 7fbfa79ad640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f0846e3c00 0x55f08397fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45892/0 (socket says 172.21.15.154:45892) 2023-09-23T03:26:31.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.363+0000 7fbfa79ad640 1 -- 172.21.15.154:0/2644481505 learned_addr learned my addr 172.21.15.154:0/2644481505 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:31.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.363+0000 7fbfa79ad640 1 -- 172.21.15.154:0/2644481505 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f0846e2800 msgr2=0x55f08397eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:31.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.363+0000 7fbfa79ad640 1 --2- 172.21.15.154:0/2644481505 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f0846e2800 0x55f08397eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.363+0000 7fbfa71ac640 1 --2- 172.21.15.154:0/2644481505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f0846e2400 0x55f08397f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.363+0000 7fbfa79ad640 1 -- 172.21.15.154:0/2644481505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f0846e2400 msgr2=0x55f08397f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:31.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.363+0000 7fbfa79ad640 1 --2- 172.21.15.154:0/2644481505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f0846e2400 0x55f08397f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.363+0000 7fbfa79ad640 1 -- 172.21.15.154:0/2644481505 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f083942000 con 0x55f0846e3c00 2023-09-23T03:26:31.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.363+0000 7fbfa79ad640 1 --2- 172.21.15.154:0/2644481505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f0846e3c00 0x55f08397fb80 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x55f0839dae10 tx=0x55f083bea120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdbf8a2d5d16097a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.363+0000 7fbfa49a7640 1 -- 172.21.15.154:0/2644481505 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f083959080 con 0x55f0846e3c00 2023-09-23T03:26:31.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.364+0000 7fbfa49a7640 1 -- 172.21.15.154:0/2644481505 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f083bf8000 con 0x55f0846e3c00 2023-09-23T03:26:31.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.364+0000 7fbfa49a7640 1 -- 172.21.15.154:0/2644481505 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f083959600 con 0x55f0846e3c00 2023-09-23T03:26:31.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.364+0000 7fbfa923a100 1 -- 172.21.15.154:0/2644481505 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f083942340 con 0x55f0846e3c00 2023-09-23T03:26:31.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.364+0000 7fbfa923a100 1 -- 172.21.15.154:0/2644481505 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55f0839431e0 con 0x55f0846e3c00 2023-09-23T03:26:31.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.364+0000 7fbfa923a100 4 waiting for MDS map... 2023-09-23T03:26:31.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.364+0000 7fbfa49a7640 1 -- 172.21.15.154:0/2644481505 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(110..110 src has 1..110) v4 ==== 6605+0+0 (secure 0 0 0) 0x55f08393a380 con 0x55f0846e3c00 2023-09-23T03:26:31.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.364+0000 7fbfa49a7640 1 -- 172.21.15.154:0/2644481505 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 91) v1 ==== 3306+0+0 (secure 0 0 0) 0x55f083963200 con 0x55f0846e3c00 2023-09-23T03:26:31.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.364+0000 7fbfa923a100 4 Got MDS map 91 2023-09-23T03:26:31.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.364+0000 7fbfa923a100 10 main: main 2023-09-23T03:26:31.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.364+0000 7fbfa923a100 4 main: connecting to RADOS... 2023-09-23T03:26:31.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.365+0000 7fbfa923a100 1 Processor -- start 2023-09-23T03:26:31.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.365+0000 7fbfa923a100 1 -- start start 2023-09-23T03:26:31.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.366+0000 7fbfa923a100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f0846e3800 0x55f08397f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.366+0000 7fbfa923a100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f083c56000 0x55f08397e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.366+0000 7fbfa923a100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f083c56400 0x55f083980100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.366+0000 7fbfa923a100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0839dd180 con 0x55f083c56400 2023-09-23T03:26:31.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.366+0000 7fbfa923a100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f08395ab40 con 0x55f083c56000 2023-09-23T03:26:31.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.366+0000 7fbfa923a100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f08395aa00 con 0x55f0846e3800 2023-09-23T03:26:31.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.366+0000 7fbfa69ab640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f083c56400 0x55f083980100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.366+0000 7fbfa69ab640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f083c56400 0x55f083980100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45904/0 (socket says 172.21.15.154:45904) 2023-09-23T03:26:31.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.366+0000 7fbfa69ab640 1 -- 172.21.15.154:0/2818943446 learned_addr learned my addr 172.21.15.154:0/2818943446 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:31.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.366+0000 7fbfa71ac640 1 --2- 172.21.15.154:0/2818943446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f083c56000 0x55f08397e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.366+0000 7fbfa69ab640 1 --2- 172.21.15.154:0/2818943446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f0846e3800 0x55f08397f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.366+0000 7fbfa69ab640 1 -- 172.21.15.154:0/2818943446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f0846e3800 msgr2=0x55f08397f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:31.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.366+0000 7fbfa69ab640 1 --2- 172.21.15.154:0/2818943446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f0846e3800 0x55f08397f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.366+0000 7fbfa69ab640 1 -- 172.21.15.154:0/2818943446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f083c56000 msgr2=0x55f08397e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:31.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.366+0000 7fbfa69ab640 1 --2- 172.21.15.154:0/2818943446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f083c56000 0x55f08397e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.366+0000 7fbfa69ab640 1 -- 172.21.15.154:0/2818943446 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0839429c0 con 0x55f083c56400 2023-09-23T03:26:31.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.366+0000 7fbfa71ac640 1 --2- 172.21.15.154:0/2818943446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f083c56000 0x55f08397e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:26:31.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.367+0000 7fbfa69ab640 1 --2- 172.21.15.154:0/2818943446 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f083c56400 0x55f083980100 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x55f083c68000 tx=0x55f083c0ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb65954d86c7947f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.367+0000 7fbfa29a3640 1 -- 172.21.15.154:0/2818943446 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f083959080 con 0x55f083c56400 2023-09-23T03:26:31.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.367+0000 7fbfa29a3640 1 -- 172.21.15.154:0/2818943446 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f083bf8780 con 0x55f083c56400 2023-09-23T03:26:31.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.367+0000 7fbfa29a3640 1 -- 172.21.15.154:0/2818943446 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f083959600 con 0x55f083c56400 2023-09-23T03:26:31.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.367+0000 7fbfa923a100 1 -- 172.21.15.154:0/2818943446 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f083c56400 msgr2=0x55f083980100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.367+0000 7fbfa923a100 1 --2- 172.21.15.154:0/2818943446 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f083c56400 0x55f083980100 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x55f083c68000 tx=0x55f083c0ccc0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.367+0000 7fbfa923a100 1 -- 172.21.15.154:0/2818943446 shutdown_connections 2023-09-23T03:26:31.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.367+0000 7fbfa923a100 1 --2- 172.21.15.154:0/2818943446 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f083c56400 0x55f083980100 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.367+0000 7fbfa923a100 1 --2- 172.21.15.154:0/2818943446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f083c56000 0x55f08397e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.367+0000 7fbfa923a100 1 --2- 172.21.15.154:0/2818943446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f0846e3800 0x55f08397f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.367+0000 7fbfa923a100 1 -- 172.21.15.154:0/2818943446 >> 172.21.15.154:0/2818943446 conn(0x55f0846e3400 msgr2=0x55f084699800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:31.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.367+0000 7fbfa923a100 1 -- 172.21.15.154:0/2818943446 shutdown_connections 2023-09-23T03:26:31.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.368+0000 7fbfa923a100 1 -- 172.21.15.154:0/2818943446 wait complete. 2023-09-23T03:26:31.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.368+0000 7fbfa923a100 1 Processor -- start 2023-09-23T03:26:31.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.368+0000 7fbfa923a100 1 -- start start 2023-09-23T03:26:31.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.369+0000 7fbfa923a100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f0846e3400 0x55f083981180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.369+0000 7fbfa923a100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f083c57000 0x55f083980c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.369+0000 7fbfa923a100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f083c56c00 0x55f083980680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.369+0000 7fbfa923a100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0839dcdc0 con 0x55f083c57000 2023-09-23T03:26:31.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.369+0000 7fbfa923a100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f08395aa00 con 0x55f0846e3400 2023-09-23T03:26:31.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.369+0000 7fbfa923a100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f08395ab40 con 0x55f083c56c00 2023-09-23T03:26:31.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.369+0000 7fbfa71ac640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f083c57000 0x55f083980c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.369+0000 7fbfa71ac640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f083c57000 0x55f083980c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45910/0 (socket says 172.21.15.154:45910) 2023-09-23T03:26:31.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.369+0000 7fbfa71ac640 1 -- 172.21.15.154:0/1477080283 learned_addr learned my addr 172.21.15.154:0/1477080283 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:31.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.370+0000 7fbfa71ac640 1 -- 172.21.15.154:0/1477080283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f083c56c00 msgr2=0x55f083980680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.370+0000 7fbfa71ac640 1 --2- 172.21.15.154:0/1477080283 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f083c56c00 0x55f083980680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.370+0000 7fbfa71ac640 1 -- 172.21.15.154:0/1477080283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f0846e3400 msgr2=0x55f083981180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.370+0000 7fbfa71ac640 1 --2- 172.21.15.154:0/1477080283 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f0846e3400 0x55f083981180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.370+0000 7fbfa71ac640 1 -- 172.21.15.154:0/1477080283 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f083942000 con 0x55f083c57000 2023-09-23T03:26:31.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.370+0000 7fbfa71ac640 1 --2- 172.21.15.154:0/1477080283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f083c57000 0x55f083980c00 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x55f083c68db0 tx=0x55f083c0cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.370+0000 7fbfa11a0640 1 -- 172.21.15.154:0/1477080283 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f083959080 con 0x55f083c57000 2023-09-23T03:26:31.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.370+0000 7fbfa11a0640 1 -- 172.21.15.154:0/1477080283 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f083935b00 con 0x55f083c57000 2023-09-23T03:26:31.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.370+0000 7fbfa11a0640 1 -- 172.21.15.154:0/1477080283 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f083959600 con 0x55f083c57000 2023-09-23T03:26:31.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.370+0000 7fbfa923a100 1 -- 172.21.15.154:0/1477080283 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f083942ea0 con 0x55f083c57000 2023-09-23T03:26:31.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.370+0000 7fbfa923a100 1 -- 172.21.15.154:0/1477080283 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f083943520 con 0x55f083c57000 2023-09-23T03:26:31.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.370+0000 7fbfa923a100 4 main: resolving pool 14 2023-09-23T03:26:31.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.372+0000 7fbfa11a0640 1 -- 172.21.15.154:0/1477080283 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x55f083940b00 con 0x55f083c57000 2023-09-23T03:26:31.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.373+0000 7fbfa11a0640 1 --2- 172.21.15.154:0/1477080283 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55f083c70c00 0x55f083981700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.373+0000 7fbfa11a0640 1 -- 172.21.15.154:0/1477080283 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6605+0+0 (secure 0 0 0) 0x55f08393a540 con 0x55f083c57000 2023-09-23T03:26:31.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.373+0000 7fbfa923a100 4 main: creating IoCtx.. 2023-09-23T03:26:31.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.373+0000 7fbfa923a100 1 --2- 172.21.15.154:0/1477080283 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55f083c56800 0x55f08397f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.373+0000 7fbfa923a100 1 -- 172.21.15.154:0/1477080283 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:1 14.d 14:b50e409b:::mds_snaptable:head [writefull 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f08396b880 con 0x55f083c56800 2023-09-23T03:26:31.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.373+0000 7fbfa69ab640 1 --2- 172.21.15.154:0/1477080283 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55f083c70c00 0x55f083981700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.374+0000 7fbfa69ab640 1 --2- 172.21.15.154:0/1477080283 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55f083c56800 0x55f08397f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.374+0000 7fbfa69ab640 1 --2- 172.21.15.154:0/1477080283 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55f083c70c00 0x55f083981700 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x55f083c69f50 tx=0x55f083c0cc00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.374+0000 7fbfa69ab640 1 --2- 172.21.15.154:0/1477080283 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55f083c56800 0x55f08397f080 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.378+0000 7fbfa69ab640 1 -- 172.21.15.154:0/1477080283 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== osd_op_reply(1 mds_snaptable [writefull 0~70] v110'3 uv3 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x55f0839e8b40 con 0x55f083c56800 2023-09-23T03:26:31.379 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:26:31.379 INFO:teuthology.orchestra.run.smithi154.stdout: "result": 0 2023-09-23T03:26:31.379 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:26:31.379 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:31.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.378+0000 7fbfa923a100 1 -- 172.21.15.154:0/1477080283 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55f083c56800 msgr2=0x55f08397f080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.378+0000 7fbfa923a100 1 --2- 172.21.15.154:0/1477080283 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55f083c56800 0x55f08397f080 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.379+0000 7fbfa923a100 1 -- 172.21.15.154:0/1477080283 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55f083c70c00 msgr2=0x55f083981700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.379+0000 7fbfa923a100 1 --2- 172.21.15.154:0/1477080283 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55f083c70c00 0x55f083981700 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x55f083c69f50 tx=0x55f083c0cc00 comp rx=0 tx=0).stop 2023-09-23T03:26:31.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.379+0000 7fbfa923a100 1 -- 172.21.15.154:0/1477080283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f083c57000 msgr2=0x55f083980c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.379+0000 7fbfa923a100 1 --2- 172.21.15.154:0/1477080283 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f083c57000 0x55f083980c00 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x55f083c68db0 tx=0x55f083c0cde0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.379+0000 7fbfa71ac640 1 -- 172.21.15.154:0/1477080283 reap_dead start 2023-09-23T03:26:31.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.379+0000 7fbfa923a100 1 -- 172.21.15.154:0/1477080283 shutdown_connections 2023-09-23T03:26:31.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.379+0000 7fbfa923a100 1 -- 172.21.15.154:0/1477080283 >> 172.21.15.154:0/1477080283 conn(0x55f0846e3800 msgr2=0x55f084699800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:31.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.379+0000 7fbfa923a100 1 -- 172.21.15.154:0/1477080283 shutdown_connections 2023-09-23T03:26:31.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.379+0000 7fbfa923a100 1 -- 172.21.15.154:0/1477080283 wait complete. 2023-09-23T03:26:31.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.379+0000 7fbfa923a100 1 -- 172.21.15.154:0/2644481505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f0846e3c00 msgr2=0x55f08397fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.379+0000 7fbfa923a100 1 --2- 172.21.15.154:0/2644481505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f0846e3c00 0x55f08397fb80 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x55f0839dae10 tx=0x55f083bea120 comp rx=0 tx=0).stop 2023-09-23T03:26:31.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.379+0000 7fbfa923a100 1 -- 172.21.15.154:0/2644481505 shutdown_connections 2023-09-23T03:26:31.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.379+0000 7fbfa923a100 1 --2- 172.21.15.154:0/2644481505 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f0846e3c00 0x55f08397fb80 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.379+0000 7fbfa923a100 1 --2- 172.21.15.154:0/2644481505 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f0846e2800 0x55f08397eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.379+0000 7fbfa923a100 1 --2- 172.21.15.154:0/2644481505 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f0846e2400 0x55f08397f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.379+0000 7fbfa923a100 1 -- 172.21.15.154:0/2644481505 >> 172.21.15.154:0/2644481505 conn(0x55f08390f000 msgr2=0x55f084698000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:31.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.379+0000 7fbfa923a100 1 -- 172.21.15.154:0/2644481505 shutdown_connections 2023-09-23T03:26:31.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.379+0000 7fbfa923a100 1 -- 172.21.15.154:0/2644481505 wait complete. 2023-09-23T03:26:31.385 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'snap'] in time 0:00:00.104536, result: { "result": 0 } 2023-09-23T03:26:31.385 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset inode 2023-09-23T03:26:31.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.468+0000 7f9f99441100 1 -- start start 2023-09-23T03:26:31.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.468+0000 7f9f99441100 1 Processor -- start 2023-09-23T03:26:31.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.469+0000 7f9f99441100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56236a246c00 0x56236a2b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.469+0000 7f9f99441100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56236a247800 0x56236a2b7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.469+0000 7f9f99441100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56236a247c00 0x56236a2b7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.469+0000 7f9f99441100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236a315180 con 0x56236a247800 2023-09-23T03:26:31.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.469+0000 7f9f99441100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236a314dc0 con 0x56236a246c00 2023-09-23T03:26:31.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.469+0000 7f9f99441100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56236a292a00 con 0x56236a247c00 2023-09-23T03:26:31.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.470+0000 7f9f96cd0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56236a247800 0x56236a2b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.470+0000 7f9f96cd0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56236a247800 0x56236a2b7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45936/0 (socket says 172.21.15.154:45936) 2023-09-23T03:26:31.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.470+0000 7f9f96cd0640 1 -- 172.21.15.154:0/670346812 learned_addr learned my addr 172.21.15.154:0/670346812 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:31.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.470+0000 7f9f974d1640 1 --2- 172.21.15.154:0/670346812 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56236a246c00 0x56236a2b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.470+0000 7f9f96cd0640 1 -- 172.21.15.154:0/670346812 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56236a247c00 msgr2=0x56236a2b7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:31.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.470+0000 7f9f97cd2640 1 --2- 172.21.15.154:0/670346812 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56236a247c00 0x56236a2b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.470+0000 7f9f96cd0640 1 --2- 172.21.15.154:0/670346812 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56236a247c00 0x56236a2b7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.470+0000 7f9f96cd0640 1 -- 172.21.15.154:0/670346812 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56236a246c00 msgr2=0x56236a2b7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:31.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.470+0000 7f9f96cd0640 1 --2- 172.21.15.154:0/670346812 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56236a246c00 0x56236a2b7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.470+0000 7f9f96cd0640 1 -- 172.21.15.154:0/670346812 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56236a27ab60 con 0x56236a247800 2023-09-23T03:26:31.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.470+0000 7f9f96cd0640 1 --2- 172.21.15.154:0/670346812 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56236a247800 0x56236a2b7600 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x56236afd4c60 tx=0x56236a52c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92a4f360cc64252 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.470+0000 7f9f94ccc640 1 -- 172.21.15.154:0/670346812 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56236a291600 con 0x56236a247800 2023-09-23T03:26:31.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.471+0000 7f9f94ccc640 1 -- 172.21.15.154:0/670346812 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56236a534000 con 0x56236a247800 2023-09-23T03:26:31.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.471+0000 7f9f94ccc640 1 -- 172.21.15.154:0/670346812 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56236a2911e0 con 0x56236a247800 2023-09-23T03:26:31.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.471+0000 7f9f99441100 1 -- 172.21.15.154:0/670346812 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56236a27a340 con 0x56236a247800 2023-09-23T03:26:31.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.471+0000 7f9f99441100 1 -- 172.21.15.154:0/670346812 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x56236a27a000 con 0x56236a247800 2023-09-23T03:26:31.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.471+0000 7f9f99441100 4 waiting for MDS map... 2023-09-23T03:26:31.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.471+0000 7f9f94ccc640 1 -- 172.21.15.154:0/670346812 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(110..110 src has 1..110) v4 ==== 6605+0+0 (secure 0 0 0) 0x56236a272380 con 0x56236a247800 2023-09-23T03:26:31.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.471+0000 7f9f94ccc640 1 -- 172.21.15.154:0/670346812 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 91) v1 ==== 3306+0+0 (secure 0 0 0) 0x56236a29b200 con 0x56236a247800 2023-09-23T03:26:31.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.471+0000 7f9f99441100 4 Got MDS map 91 2023-09-23T03:26:31.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.471+0000 7f9f99441100 10 main: main 2023-09-23T03:26:31.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.471+0000 7f9f99441100 4 main: connecting to RADOS... 2023-09-23T03:26:31.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.472+0000 7f9f99441100 1 Processor -- start 2023-09-23T03:26:31.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.472+0000 7f9f99441100 1 -- start start 2023-09-23T03:26:31.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.473+0000 7f9f99441100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56236a533000 0x56236a2b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.473+0000 7f9f99441100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56236a533400 0x56236a2b6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.473+0000 7f9f99441100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56236a533800 0x56236a2b8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.473+0000 7f9f99441100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236a292b40 con 0x56236a533800 2023-09-23T03:26:31.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.473+0000 7f9f99441100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236a314dc0 con 0x56236a533000 2023-09-23T03:26:31.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.473+0000 7f9f99441100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56236a314c80 con 0x56236a533400 2023-09-23T03:26:31.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.473+0000 7f9f96cd0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56236a533800 0x56236a2b8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.473+0000 7f9f96cd0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56236a533800 0x56236a2b8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45948/0 (socket says 172.21.15.154:45948) 2023-09-23T03:26:31.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.473+0000 7f9f96cd0640 1 -- 172.21.15.154:0/4170334896 learned_addr learned my addr 172.21.15.154:0/4170334896 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:31.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.473+0000 7f9f97cd2640 1 --2- 172.21.15.154:0/4170334896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56236a533000 0x56236a2b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.473+0000 7f9f96cd0640 1 -- 172.21.15.154:0/4170334896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56236a533400 msgr2=0x56236a2b6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:31.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.473+0000 7f9f96cd0640 1 --2- 172.21.15.154:0/4170334896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56236a533400 0x56236a2b6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.473+0000 7f9f96cd0640 1 -- 172.21.15.154:0/4170334896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56236a533000 msgr2=0x56236a2b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:31.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.473+0000 7f9f96cd0640 1 --2- 172.21.15.154:0/4170334896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56236a533000 0x56236a2b6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.473+0000 7f9f96cd0640 1 -- 172.21.15.154:0/4170334896 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56236a27aea0 con 0x56236a533800 2023-09-23T03:26:31.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.474+0000 7f9f96cd0640 1 --2- 172.21.15.154:0/4170334896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56236a533800 0x56236a2b8100 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x56236a596120 tx=0x56236afc7e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48eb99c2a3ca4b1e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.474+0000 7f9f92cc8640 1 -- 172.21.15.154:0/4170334896 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56236a290f20 con 0x56236a533800 2023-09-23T03:26:31.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.474+0000 7f9f92cc8640 1 -- 172.21.15.154:0/4170334896 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56236a534480 con 0x56236a533800 2023-09-23T03:26:31.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.474+0000 7f9f92cc8640 1 -- 172.21.15.154:0/4170334896 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56236a291600 con 0x56236a533800 2023-09-23T03:26:31.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.474+0000 7f9f99441100 1 -- 172.21.15.154:0/4170334896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56236a533800 msgr2=0x56236a2b8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.474+0000 7f9f99441100 1 --2- 172.21.15.154:0/4170334896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56236a533800 0x56236a2b8100 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x56236a596120 tx=0x56236afc7e60 comp rx=0 tx=0).stop 2023-09-23T03:26:31.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.474+0000 7f9f99441100 1 -- 172.21.15.154:0/4170334896 shutdown_connections 2023-09-23T03:26:31.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.474+0000 7f9f99441100 1 --2- 172.21.15.154:0/4170334896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56236a533800 0x56236a2b8100 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.474+0000 7f9f99441100 1 --2- 172.21.15.154:0/4170334896 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56236a533400 0x56236a2b6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.474+0000 7f9f99441100 1 --2- 172.21.15.154:0/4170334896 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56236a533000 0x56236a2b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.474+0000 7f9f99441100 1 -- 172.21.15.154:0/4170334896 >> 172.21.15.154:0/4170334896 conn(0x56236a532c00 msgr2=0x56236a325000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:31.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.474+0000 7f9f99441100 1 -- 172.21.15.154:0/4170334896 shutdown_connections 2023-09-23T03:26:31.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.474+0000 7f9f99441100 1 -- 172.21.15.154:0/4170334896 wait complete. 2023-09-23T03:26:31.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.475+0000 7f9f99441100 1 Processor -- start 2023-09-23T03:26:31.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.475+0000 7f9f99441100 1 -- start start 2023-09-23T03:26:31.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.476+0000 7f9f99441100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56236a533000 0x56236a2b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.476+0000 7f9f99441100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56236a533400 0x56236a2b6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.476+0000 7f9f99441100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56236a533800 0x56236a2b8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.476+0000 7f9f99441100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236a314b40 con 0x56236a533400 2023-09-23T03:26:31.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.476+0000 7f9f99441100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56236a314a00 con 0x56236a533800 2023-09-23T03:26:31.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.476+0000 7f9f99441100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56236a3148c0 con 0x56236a533000 2023-09-23T03:26:31.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.476+0000 7f9f974d1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56236a533400 0x56236a2b6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.476+0000 7f9f974d1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56236a533400 0x56236a2b6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45962/0 (socket says 172.21.15.154:45962) 2023-09-23T03:26:31.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.476+0000 7f9f974d1640 1 -- 172.21.15.154:0/3095837860 learned_addr learned my addr 172.21.15.154:0/3095837860 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:31.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.476+0000 7f9f974d1640 1 -- 172.21.15.154:0/3095837860 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56236a533000 msgr2=0x56236a2b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.476+0000 7f9f974d1640 1 --2- 172.21.15.154:0/3095837860 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56236a533000 0x56236a2b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.476+0000 7f9f974d1640 1 -- 172.21.15.154:0/3095837860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56236a533800 msgr2=0x56236a2b8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.476+0000 7f9f974d1640 1 --2- 172.21.15.154:0/3095837860 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56236a533800 0x56236a2b8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.477+0000 7f9f974d1640 1 -- 172.21.15.154:0/3095837860 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56236a27ab60 con 0x56236a533400 2023-09-23T03:26:31.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.477+0000 7f9f974d1640 1 --2- 172.21.15.154:0/3095837860 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56236a533400 0x56236a2b6580 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x56236a596d20 tx=0x56236a52c120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.477+0000 7f9f914c5640 1 -- 172.21.15.154:0/3095837860 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56236a291080 con 0x56236a533400 2023-09-23T03:26:31.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.477+0000 7f9f914c5640 1 -- 172.21.15.154:0/3095837860 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56236a26ca80 con 0x56236a533400 2023-09-23T03:26:31.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.477+0000 7f9f99441100 1 -- 172.21.15.154:0/3095837860 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56236a27a340 con 0x56236a533400 2023-09-23T03:26:31.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.477+0000 7f9f914c5640 1 -- 172.21.15.154:0/3095837860 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56236a290f20 con 0x56236a533400 2023-09-23T03:26:31.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.477+0000 7f9f99441100 1 -- 172.21.15.154:0/3095837860 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56236a27b520 con 0x56236a533400 2023-09-23T03:26:31.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.477+0000 7f9f99441100 4 main: resolving pool 14 2023-09-23T03:26:31.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.479+0000 7f9f914c5640 1 -- 172.21.15.154:0/3095837860 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x56236a279080 con 0x56236a533400 2023-09-23T03:26:31.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.480+0000 7f9f914c5640 1 --2- 172.21.15.154:0/3095837860 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x56236a533c00 0x56236a690000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.480+0000 7f9f914c5640 1 -- 172.21.15.154:0/3095837860 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6605+0+0 (secure 0 0 0) 0x56236a272540 con 0x56236a533400 2023-09-23T03:26:31.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.480+0000 7f9f99441100 4 main: creating IoCtx.. 2023-09-23T03:26:31.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.480+0000 7f9f97cd2640 1 --2- 172.21.15.154:0/3095837860 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x56236a533c00 0x56236a690000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.480+0000 7f9f99441100 1 --2- 172.21.15.154:0/3095837860 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x56236a698000 0x56236a2b8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.480+0000 7f9f99441100 1 -- 172.21.15.154:0/3095837860 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x56236a2a3880 con 0x56236a698000 2023-09-23T03:26:31.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.480+0000 7f9f96cd0640 1 --2- 172.21.15.154:0/3095837860 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x56236a698000 0x56236a2b8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.481+0000 7f9f97cd2640 1 --2- 172.21.15.154:0/3095837860 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x56236a533c00 0x56236a690000 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x56236afa0030 tx=0x56236a545f20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.481+0000 7f9f96cd0640 1 --2- 172.21.15.154:0/3095837860 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x56236a698000 0x56236a2b8680 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.484+0000 7f9f96cd0640 1 -- 172.21.15.154:0/3095837860 <== osd.2 v2:172.21.15.154:6822/2884407243 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v110'6 uv6 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x56236b010b40 con 0x56236a698000 2023-09-23T03:26:31.485 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:26:31.486 INFO:teuthology.orchestra.run.smithi154.stdout: "0": { 2023-09-23T03:26:31.486 INFO:teuthology.orchestra.run.smithi154.stdout: "data": {}, 2023-09-23T03:26:31.486 INFO:teuthology.orchestra.run.smithi154.stdout: "result": 0 2023-09-23T03:26:31.486 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:26:31.486 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:26:31.486 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:31.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.484+0000 7f9f99441100 1 -- 172.21.15.154:0/3095837860 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x56236a698000 msgr2=0x56236a2b8680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.484+0000 7f9f99441100 1 --2- 172.21.15.154:0/3095837860 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x56236a698000 0x56236a2b8680 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.484+0000 7f9f99441100 1 -- 172.21.15.154:0/3095837860 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x56236a533c00 msgr2=0x56236a690000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.484+0000 7f9f99441100 1 --2- 172.21.15.154:0/3095837860 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x56236a533c00 0x56236a690000 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x56236afa0030 tx=0x56236a545f20 comp rx=0 tx=0).stop 2023-09-23T03:26:31.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.484+0000 7f9f99441100 1 -- 172.21.15.154:0/3095837860 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56236a533400 msgr2=0x56236a2b6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.484+0000 7f9f99441100 1 --2- 172.21.15.154:0/3095837860 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56236a533400 0x56236a2b6580 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x56236a596d20 tx=0x56236a52c120 comp rx=0 tx=0).stop 2023-09-23T03:26:31.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.485+0000 7f9f974d1640 1 -- 172.21.15.154:0/3095837860 reap_dead start 2023-09-23T03:26:31.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.485+0000 7f9f99441100 1 -- 172.21.15.154:0/3095837860 shutdown_connections 2023-09-23T03:26:31.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.485+0000 7f9f99441100 1 -- 172.21.15.154:0/3095837860 >> 172.21.15.154:0/3095837860 conn(0x56236a532c00 msgr2=0x56236a325000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:31.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.485+0000 7f9f99441100 1 -- 172.21.15.154:0/3095837860 shutdown_connections 2023-09-23T03:26:31.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.485+0000 7f9f99441100 1 -- 172.21.15.154:0/3095837860 wait complete. 2023-09-23T03:26:31.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.485+0000 7f9f99441100 1 -- 172.21.15.154:0/670346812 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56236a247800 msgr2=0x56236a2b7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.485+0000 7f9f99441100 1 --2- 172.21.15.154:0/670346812 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56236a247800 0x56236a2b7600 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x56236afd4c60 tx=0x56236a52c300 comp rx=0 tx=0).stop 2023-09-23T03:26:31.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.485+0000 7f9f99441100 1 -- 172.21.15.154:0/670346812 shutdown_connections 2023-09-23T03:26:31.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.485+0000 7f9f99441100 1 --2- 172.21.15.154:0/670346812 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56236a247c00 0x56236a2b7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.485+0000 7f9f99441100 1 --2- 172.21.15.154:0/670346812 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56236a247800 0x56236a2b7600 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.485+0000 7f9f99441100 1 --2- 172.21.15.154:0/670346812 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56236a246c00 0x56236a2b7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.485+0000 7f9f99441100 1 -- 172.21.15.154:0/670346812 >> 172.21.15.154:0/670346812 conn(0x56236a247000 msgr2=0x56236a324800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:31.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.485+0000 7f9f99441100 1 -- 172.21.15.154:0/670346812 shutdown_connections 2023-09-23T03:26:31.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.485+0000 7f9f99441100 1 -- 172.21.15.154:0/670346812 wait complete. 2023-09-23T03:26:31.490 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'inode'] in time 0:00:00.105224, result: { "0": { "data": {}, "result": 0 } } 2023-09-23T03:26:31.490 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2023-09-23T03:26:31.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.575+0000 7f901384a100 1 -- start start 2023-09-23T03:26:31.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.575+0000 7f901384a100 1 Processor -- start 2023-09-23T03:26:31.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.576+0000 7f901384a100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5592ae7d8c00 0x5592ae848580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.576+0000 7f901384a100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592ae7d9800 0x5592ae848b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.576+0000 7f901384a100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5592ae7d9c00 0x5592ae849080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.576+0000 7f901384a100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592ae825180 con 0x5592ae7d9800 2023-09-23T03:26:31.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.576+0000 7f901384a100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592ae824000 con 0x5592ae7d9c00 2023-09-23T03:26:31.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.576+0000 7f901384a100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5592ae824140 con 0x5592ae7d8c00 2023-09-23T03:26:31.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.576+0000 7f9011051640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592ae7d9800 0x5592ae848b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.577+0000 7f9011051640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592ae7d9800 0x5592ae848b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45964/0 (socket says 172.21.15.154:45964) 2023-09-23T03:26:31.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.577+0000 7f9011051640 1 -- 172.21.15.154:0/3821944935 learned_addr learned my addr 172.21.15.154:0/3821944935 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:31.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.577+0000 7f9012053640 1 --2- 172.21.15.154:0/3821944935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5592ae7d9c00 0x5592ae849080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.577+0000 7f9011852640 1 --2- 172.21.15.154:0/3821944935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5592ae7d8c00 0x5592ae848580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.577+0000 7f9011051640 1 -- 172.21.15.154:0/3821944935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5592ae7d8c00 msgr2=0x5592ae848580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:31.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.577+0000 7f9011051640 1 --2- 172.21.15.154:0/3821944935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5592ae7d8c00 0x5592ae848580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.577+0000 7f9011051640 1 -- 172.21.15.154:0/3821944935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5592ae7d9c00 msgr2=0x5592ae849080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:31.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.577+0000 7f9011051640 1 --2- 172.21.15.154:0/3821944935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5592ae7d9c00 0x5592ae849080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.577+0000 7f9011051640 1 -- 172.21.15.154:0/3821944935 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592ae80cb60 con 0x5592ae7d9800 2023-09-23T03:26:31.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.577+0000 7f9012053640 1 --2- 172.21.15.154:0/3821944935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5592ae7d9c00 0x5592ae849080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:26:31.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.577+0000 7f9011051640 1 --2- 172.21.15.154:0/3821944935 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592ae7d9800 0x5592ae848b00 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x5592af5eadb0 tx=0x5592aeaba120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0fe9ea8f4cd1a98 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.577+0000 7f900f04d640 1 -- 172.21.15.154:0/3821944935 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5592ae823080 con 0x5592ae7d9800 2023-09-23T03:26:31.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.577+0000 7f900f04d640 1 -- 172.21.15.154:0/3821944935 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5592af5ae300 con 0x5592ae7d9800 2023-09-23T03:26:31.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.577+0000 7f900f04d640 1 -- 172.21.15.154:0/3821944935 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5592ae823600 con 0x5592ae7d9800 2023-09-23T03:26:31.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.577+0000 7f901384a100 1 -- 172.21.15.154:0/3821944935 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5592ae80d1e0 con 0x5592ae7d9800 2023-09-23T03:26:31.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.577+0000 7f901384a100 1 -- 172.21.15.154:0/3821944935 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5592ae80cb60 con 0x5592ae7d9800 2023-09-23T03:26:31.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.577+0000 7f901384a100 4 waiting for MDS map... 2023-09-23T03:26:31.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.578+0000 7f900f04d640 1 -- 172.21.15.154:0/3821944935 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(110..110 src has 1..110) v4 ==== 6605+0+0 (secure 0 0 0) 0x5592ae804380 con 0x5592ae7d9800 2023-09-23T03:26:31.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.578+0000 7f900f04d640 1 -- 172.21.15.154:0/3821944935 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 91) v1 ==== 3306+0+0 (secure 0 0 0) 0x5592ae82cf00 con 0x5592ae7d9800 2023-09-23T03:26:31.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.578+0000 7f901384a100 4 Got MDS map 91 2023-09-23T03:26:31.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.578+0000 7f901384a100 10 main: JournalTool::main 2023-09-23T03:26:31.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.578+0000 7f901384a100 4 main: JournalTool: connecting to RADOS... 2023-09-23T03:26:31.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.579+0000 7f901384a100 1 Processor -- start 2023-09-23T03:26:31.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.579+0000 7f901384a100 1 -- start start 2023-09-23T03:26:31.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.580+0000 7f901384a100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592af59dc00 0x5592ae849600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.580+0000 7f901384a100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5592aeb20000 0x5592ae849b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.580+0000 7f901384a100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5592aeb20400 0x5592ae84a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.580+0000 7f901384a100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592ae824a00 con 0x5592af59dc00 2023-09-23T03:26:31.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.580+0000 7f901384a100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592ae8a7180 con 0x5592aeb20000 2023-09-23T03:26:31.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.580+0000 7f901384a100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5592ae8a6dc0 con 0x5592aeb20400 2023-09-23T03:26:31.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.580+0000 7f9012053640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592af59dc00 0x5592ae849600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.580+0000 7f9012053640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592af59dc00 0x5592ae849600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45966/0 (socket says 172.21.15.154:45966) 2023-09-23T03:26:31.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.580+0000 7f9012053640 1 -- 172.21.15.154:0/1478049640 learned_addr learned my addr 172.21.15.154:0/1478049640 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:31.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.580+0000 7f9012053640 1 -- 172.21.15.154:0/1478049640 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5592aeb20400 msgr2=0x5592ae84a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:31.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.580+0000 7f9012053640 1 --2- 172.21.15.154:0/1478049640 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5592aeb20400 0x5592ae84a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.580+0000 7f9012053640 1 -- 172.21.15.154:0/1478049640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5592aeb20000 msgr2=0x5592ae849b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:31.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.580+0000 7f9012053640 1 --2- 172.21.15.154:0/1478049640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5592aeb20000 0x5592ae849b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.580+0000 7f9012053640 1 -- 172.21.15.154:0/1478049640 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592ae80c340 con 0x5592af59dc00 2023-09-23T03:26:31.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.581+0000 7f9012053640 1 --2- 172.21.15.154:0/1478049640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592af59dc00 0x5592ae849600 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x5592aeb1c660 tx=0x5592aead0cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=888f724e60f78dd0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.581+0000 7f900d049640 1 -- 172.21.15.154:0/1478049640 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5592ae823080 con 0x5592af59dc00 2023-09-23T03:26:31.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.581+0000 7f900d049640 1 -- 172.21.15.154:0/1478049640 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5592ae7fea80 con 0x5592af59dc00 2023-09-23T03:26:31.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.581+0000 7f900d049640 1 -- 172.21.15.154:0/1478049640 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5592ae823600 con 0x5592af59dc00 2023-09-23T03:26:31.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.581+0000 7f901384a100 1 -- 172.21.15.154:0/1478049640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592af59dc00 msgr2=0x5592ae849600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.581+0000 7f901384a100 1 --2- 172.21.15.154:0/1478049640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592af59dc00 0x5592ae849600 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x5592aeb1c660 tx=0x5592aead0cc0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.581+0000 7f901384a100 1 -- 172.21.15.154:0/1478049640 shutdown_connections 2023-09-23T03:26:31.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.581+0000 7f901384a100 1 --2- 172.21.15.154:0/1478049640 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5592aeb20400 0x5592ae84a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.581+0000 7f901384a100 1 --2- 172.21.15.154:0/1478049640 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5592aeb20000 0x5592ae849b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.581+0000 7f901384a100 1 --2- 172.21.15.154:0/1478049640 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592af59dc00 0x5592ae849600 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.581+0000 7f901384a100 1 -- 172.21.15.154:0/1478049640 >> 172.21.15.154:0/1478049640 conn(0x5592af59d800 msgr2=0x5592af551000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:31.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.581+0000 7f901384a100 1 -- 172.21.15.154:0/1478049640 shutdown_connections 2023-09-23T03:26:31.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.581+0000 7f901384a100 1 -- 172.21.15.154:0/1478049640 wait complete. 2023-09-23T03:26:31.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.582+0000 7f901384a100 1 Processor -- start 2023-09-23T03:26:31.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.582+0000 7f901384a100 1 -- start start 2023-09-23T03:26:31.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.583+0000 7f901384a100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592af59d800 0x5592ae84b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.583+0000 7f901384a100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5592aeb21000 0x5592ae84ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.583+0000 7f901384a100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5592aeb20c00 0x5592ae84a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.583+0000 7f901384a100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592ae824000 con 0x5592af59d800 2023-09-23T03:26:31.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.583+0000 7f901384a100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592ae825180 con 0x5592aeb20c00 2023-09-23T03:26:31.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.583+0000 7f901384a100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5592ae824b40 con 0x5592aeb21000 2023-09-23T03:26:31.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.583+0000 7f9012053640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592af59d800 0x5592ae84b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.583+0000 7f9012053640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592af59d800 0x5592ae84b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45972/0 (socket says 172.21.15.154:45972) 2023-09-23T03:26:31.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.583+0000 7f9012053640 1 -- 172.21.15.154:0/1876854252 learned_addr learned my addr 172.21.15.154:0/1876854252 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:31.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.583+0000 7f9012053640 1 -- 172.21.15.154:0/1876854252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5592aeb21000 msgr2=0x5592ae84ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.583+0000 7f9012053640 1 --2- 172.21.15.154:0/1876854252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5592aeb21000 0x5592ae84ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.583+0000 7f9012053640 1 -- 172.21.15.154:0/1876854252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5592aeb20c00 msgr2=0x5592ae84a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.583+0000 7f9012053640 1 --2- 172.21.15.154:0/1876854252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5592aeb20c00 0x5592ae84a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.584+0000 7f9012053640 1 -- 172.21.15.154:0/1876854252 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592ae80c000 con 0x5592af59d800 2023-09-23T03:26:31.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.584+0000 7f9012053640 1 --2- 172.21.15.154:0/1876854252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592af59d800 0x5592ae84b180 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x5592aeb1d320 tx=0x5592aead0c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.584+0000 7f900b846640 1 -- 172.21.15.154:0/1876854252 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5592ae822dc0 con 0x5592af59d800 2023-09-23T03:26:31.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.584+0000 7f900b846640 1 -- 172.21.15.154:0/1876854252 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5592af5ae600 con 0x5592af59d800 2023-09-23T03:26:31.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.584+0000 7f900b846640 1 -- 172.21.15.154:0/1876854252 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5592ae823080 con 0x5592af59d800 2023-09-23T03:26:31.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.584+0000 7f901384a100 1 -- 172.21.15.154:0/1876854252 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5592ae80cea0 con 0x5592af59d800 2023-09-23T03:26:31.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.584+0000 7f901384a100 1 -- 172.21.15.154:0/1876854252 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5592ae80d520 con 0x5592af59d800 2023-09-23T03:26:31.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.584+0000 7f901384a100 4 main: JournalTool: resolving pool 14 2023-09-23T03:26:31.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.586+0000 7f900b846640 1 -- 172.21.15.154:0/1876854252 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x5592ae80a840 con 0x5592af59d800 2023-09-23T03:26:31.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.587+0000 7f900b846640 1 --2- 172.21.15.154:0/1876854252 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5592aec1a000 0x5592ae84b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.587+0000 7f900b846640 1 -- 172.21.15.154:0/1876854252 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6605+0+0 (secure 0 0 0) 0x5592ae804540 con 0x5592af59d800 2023-09-23T03:26:31.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.587+0000 7f901384a100 4 main: JournalTool: creating IoCtx.. 2023-09-23T03:26:31.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.587+0000 7f901384a100 4 main: Executing for rank 0 2023-09-23T03:26:31.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.587+0000 7f9011852640 1 --2- 172.21.15.154:0/1876854252 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5592aec1a000 0x5592ae84b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.588+0000 7f9011852640 1 --2- 172.21.15.154:0/1876854252 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5592aec1a000 0x5592ae84b700 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x5592ae8b40f0 tx=0x5592aec2b800 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.588+0000 7f901384a100 1 -- start start 2023-09-23T03:26:31.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.588+0000 7f901384a100 1 Processor -- start 2023-09-23T03:26:31.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.589+0000 7f901384a100 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5592aeb20400 0x5592ae849600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.589+0000 7f901384a100 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5592aeb20000 0x5592ae84a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.589+0000 7f901384a100 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592aec1b000 0x5592ae849b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.589+0000 7f901384a100 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592ae8a6dc0 con 0x5592aec1b000 2023-09-23T03:26:31.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.589+0000 7f901384a100 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592ae8a7180 con 0x5592aeb20000 2023-09-23T03:26:31.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.589+0000 7f901384a100 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5592ae8a6c80 con 0x5592aeb20400 2023-09-23T03:26:31.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.589+0000 7f9011051640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592aec1b000 0x5592ae849b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.589+0000 7f9011051640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592aec1b000 0x5592ae849b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45976/0 (socket says 172.21.15.154:45976) 2023-09-23T03:26:31.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.589+0000 7f9011051640 1 -- 172.21.15.154:0/991059960 learned_addr learned my addr 172.21.15.154:0/991059960 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:31.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.590+0000 7f9011852640 1 --2- 172.21.15.154:0/991059960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5592aeb20000 0x5592ae84a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.590+0000 7f9011051640 1 -- 172.21.15.154:0/991059960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5592aeb20400 msgr2=0x5592ae849600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:31.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.590+0000 7f9011051640 1 --2- 172.21.15.154:0/991059960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5592aeb20400 0x5592ae849600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.590+0000 7f9011051640 1 -- 172.21.15.154:0/991059960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5592aeb20000 msgr2=0x5592ae84a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:31.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.590+0000 7f9011051640 1 --2- 172.21.15.154:0/991059960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5592aeb20000 0x5592ae84a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.590+0000 7f9011051640 1 -- 172.21.15.154:0/991059960 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592aeb609c0 con 0x5592aec1b000 2023-09-23T03:26:31.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.590+0000 7f9011852640 1 --2- 172.21.15.154:0/991059960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5592aeb20000 0x5592ae84a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:26:31.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.590+0000 7f9011051640 1 --2- 172.21.15.154:0/991059960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592aec1b000 0x5592ae849b80 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x5592af5eb230 tx=0x5592aeaba1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64da765071b31477 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.590+0000 7f9008840640 1 -- 172.21.15.154:0/991059960 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5592ae8231e0 con 0x5592aec1b000 2023-09-23T03:26:31.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.590+0000 7f9008840640 1 -- 172.21.15.154:0/991059960 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5592af5aec00 con 0x5592aec1b000 2023-09-23T03:26:31.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.590+0000 7f901384a100 1 -- 172.21.15.154:0/991059960 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5592aeb60820 con 0x5592aec1b000 2023-09-23T03:26:31.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.590+0000 7f901384a100 1 -- 172.21.15.154:0/991059960 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5592aeb60680 con 0x5592aec1b000 2023-09-23T03:26:31.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.590+0000 7f901384a100 4 waiting for MDS map... 2023-09-23T03:26:31.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.590+0000 7f9008840640 1 -- 172.21.15.154:0/991059960 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5592ae822f20 con 0x5592aec1b000 2023-09-23T03:26:31.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.591+0000 7f9008840640 1 -- 172.21.15.154:0/991059960 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(110..110 src has 1..110) v4 ==== 6605+0+0 (secure 0 0 0) 0x5592ae804a80 con 0x5592aec1b000 2023-09-23T03:26:31.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.591+0000 7f9008840640 1 -- 172.21.15.154:0/991059960 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 91) v1 ==== 3306+0+0 (secure 0 0 0) 0x5592aec98000 con 0x5592aec1b000 2023-09-23T03:26:31.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.591+0000 7f901384a100 4 Got MDS map 91 2023-09-23T03:26:31.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.591+0000 7f901384a100 1 --2- 172.21.15.154:0/991059960 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x5592aec1b400 0x5592aeca6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.591+0000 7f901384a100 1 -- 172.21.15.154:0/991059960 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:1 14.f 14:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5592aeca2380 con 0x5592aec1b400 2023-09-23T03:26:31.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.592+0000 7f9012053640 1 --2- 172.21.15.154:0/991059960 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x5592aec1b400 0x5592aeca6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.593+0000 7f9012053640 1 --2- 172.21.15.154:0/991059960 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x5592aec1b400 0x5592aeca6000 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=5b3dd53283f9dc49 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.596+0000 7f9012053640 1 -- 172.21.15.154:0/991059960 <== osd.7 v2:172.21.15.190:6824/3460136272 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v110'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x5592af67cb40 con 0x5592aec1b400 2023-09-23T03:26:31.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.596+0000 7f901384a100 1 --2- 172.21.15.154:0/991059960 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5592aec1b800 0x5592aeca6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.596+0000 7f901384a100 1 -- 172.21.15.154:0/991059960 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:2 14.14 14:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5592aeca2700 con 0x5592aec1b800 2023-09-23T03:26:31.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.596+0000 7f9011852640 1 --2- 172.21.15.154:0/991059960 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5592aec1b800 0x5592aeca6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.597+0000 7f9011852640 1 --2- 172.21.15.154:0/991059960 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5592aec1b800 0x5592aeca6580 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=36fdfecc5ecf0852 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.600+0000 7f9011852640 1 -- 172.21.15.154:0/991059960 <== osd.4 v2:172.21.15.190:6808/717302641 1 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v110'47 uv47 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x5592af67cd80 con 0x5592aec1b800 2023-09-23T03:26:31.600 INFO:teuthology.orchestra.run.smithi154.stdout:writing EResetJournal entry 2023-09-23T03:26:31.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.600+0000 7f901384a100 1 -- 172.21.15.154:0/991059960 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:3 14.14 14:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5592aeca2a80 con 0x5592aec1b800 2023-09-23T03:26:31.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.600+0000 7f901384a100 1 --2- 172.21.15.154:0/991059960 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5592aec1bc00 0x5592aeca6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.600+0000 7f901384a100 1 -- 172.21.15.154:0/991059960 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:4 14.1b 14:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5592aeca2e00 con 0x5592aec1bc00 2023-09-23T03:26:31.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.600+0000 7f901384a100 1 --2- 172.21.15.154:0/991059960 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5592aec8d000 0x5592aeca7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.600+0000 7f901384a100 1 -- 172.21.15.154:0/991059960 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:5 14.b 14:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5592aeca3180 con 0x5592aec8d000 2023-09-23T03:26:31.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.600+0000 7f901384a100 1 -- 172.21.15.154:0/991059960 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:6 14.1f 14:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5592aeca3500 con 0x5592aec8d000 2023-09-23T03:26:31.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.600+0000 7f901384a100 1 -- 172.21.15.154:0/991059960 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:7 14.1a 14:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5592aeca3880 con 0x5592aec8d000 2023-09-23T03:26:31.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.600+0000 7f901384a100 1 -- 172.21.15.154:0/991059960 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:8 14.1e 14:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5592aeca2700 con 0x5592aec1b400 2023-09-23T03:26:31.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.600+0000 7f9011852640 1 --2- 172.21.15.154:0/991059960 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5592aec8d000 0x5592aeca7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.600+0000 7f9012053640 1 --2- 172.21.15.154:0/991059960 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5592aec1bc00 0x5592aeca6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.600+0000 7f9012053640 1 --2- 172.21.15.154:0/991059960 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5592aec1bc00 0x5592aeca6b00 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=fa7795e018470773 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.601+0000 7f9011852640 1 --2- 172.21.15.154:0/991059960 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5592aec8d000 0x5592aeca7080 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=e2f2f6a9ee77c46c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.602+0000 7f9011852640 1 -- 172.21.15.154:0/991059960 <== osd.0 v2:172.21.15.154:6800/2920938138 1 ==== osd_op_reply(5 200.00000003 [delete] v110'6 uv5 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5592af67cd80 con 0x5592aec8d000 2023-09-23T03:26:31.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.602+0000 7f9011852640 1 -- 172.21.15.154:0/991059960 <== osd.0 v2:172.21.15.154:6800/2920938138 2 ==== osd_op_reply(7 200.00000005 [delete] v110'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5592af67cd80 con 0x5592aec8d000 2023-09-23T03:26:31.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.603+0000 7f9011852640 1 -- 172.21.15.154:0/991059960 <== osd.4 v2:172.21.15.190:6808/717302641 2 ==== osd_op_reply(3 200.00000001 [delete] v110'48 uv47 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5592af67cd80 con 0x5592aec1b800 2023-09-23T03:26:31.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.603+0000 7f9012053640 1 -- 172.21.15.154:0/991059960 <== osd.7 v2:172.21.15.190:6824/3460136272 2 ==== osd_op_reply(8 200.00000006 [delete] v110'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5592af67cb40 con 0x5592aec1b400 2023-09-23T03:26:31.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.603+0000 7f9012053640 1 -- 172.21.15.154:0/991059960 <== osd.1 v2:172.21.15.154:6816/1267710005 1 ==== osd_op_reply(4 200.00000002 [delete] v110'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5592af67cb40 con 0x5592aec1bc00 2023-09-23T03:26:31.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.603+0000 7f900703d640 1 -- 172.21.15.154:0/991059960 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:9 14.14 14:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5592aeca3180 con 0x5592aec1b800 2023-09-23T03:26:31.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.605+0000 7f9011852640 1 -- 172.21.15.154:0/991059960 <== osd.0 v2:172.21.15.154:6800/2920938138 3 ==== osd_op_reply(6 200.00000004 [delete] v110'9 uv8 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5592af67cd80 con 0x5592aec8d000 2023-09-23T03:26:31.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.606+0000 7f9011852640 1 -- 172.21.15.154:0/991059960 <== osd.4 v2:172.21.15.190:6808/717302641 3 ==== osd_op_reply(9 200.00000001 [write 0~48] v110'49 uv49 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x5592af67cd80 con 0x5592aec1b800 2023-09-23T03:26:31.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.606+0000 7f901384a100 4 Successfully wrote new journal pointer and header for rank 5:0 2023-09-23T03:26:31.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.606+0000 7f901384a100 1 -- 172.21.15.154:0/991059960 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5592aec8d000 msgr2=0x5592aeca7080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.606+0000 7f901384a100 1 --2- 172.21.15.154:0/991059960 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5592aec8d000 0x5592aeca7080 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.606+0000 7f901384a100 1 -- 172.21.15.154:0/991059960 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5592aec1bc00 msgr2=0x5592aeca6b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.606+0000 7f901384a100 1 --2- 172.21.15.154:0/991059960 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5592aec1bc00 0x5592aeca6b00 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.606+0000 7f901384a100 1 -- 172.21.15.154:0/991059960 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5592aec1b800 msgr2=0x5592aeca6580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.606+0000 7f901384a100 1 --2- 172.21.15.154:0/991059960 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5592aec1b800 0x5592aeca6580 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.606+0000 7f901384a100 1 -- 172.21.15.154:0/991059960 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x5592aec1b400 msgr2=0x5592aeca6000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.606+0000 7f901384a100 1 --2- 172.21.15.154:0/991059960 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x5592aec1b400 0x5592aeca6000 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.606+0000 7f9011051640 1 -- 172.21.15.154:0/991059960 reap_dead start 2023-09-23T03:26:31.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.606+0000 7f901384a100 1 -- 172.21.15.154:0/991059960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592aec1b000 msgr2=0x5592ae849b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.606+0000 7f901384a100 1 --2- 172.21.15.154:0/991059960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592aec1b000 0x5592ae849b80 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x5592af5eb230 tx=0x5592aeaba1e0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.606+0000 7f901384a100 1 -- 172.21.15.154:0/991059960 shutdown_connections 2023-09-23T03:26:31.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.606+0000 7f901384a100 1 --2- 172.21.15.154:0/991059960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592aec1b000 0x5592ae849b80 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.606+0000 7f901384a100 1 -- 172.21.15.154:0/991059960 >> 172.21.15.154:0/991059960 conn(0x5592aeb20800 msgr2=0x5592af551800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:31.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.606+0000 7f901384a100 1 -- 172.21.15.154:0/991059960 shutdown_connections 2023-09-23T03:26:31.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.606+0000 7f901384a100 1 -- 172.21.15.154:0/991059960 wait complete. 2023-09-23T03:26:31.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.606+0000 7f901384a100 1 -- 172.21.15.154:0/1876854252 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5592aec1a000 msgr2=0x5592ae84b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.606+0000 7f901384a100 1 --2- 172.21.15.154:0/1876854252 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5592aec1a000 0x5592ae84b700 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x5592ae8b40f0 tx=0x5592aec2b800 comp rx=0 tx=0).stop 2023-09-23T03:26:31.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.606+0000 7f901384a100 1 -- 172.21.15.154:0/1876854252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592af59d800 msgr2=0x5592ae84b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.606+0000 7f901384a100 1 --2- 172.21.15.154:0/1876854252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592af59d800 0x5592ae84b180 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x5592aeb1d320 tx=0x5592aead0c00 comp rx=0 tx=0).stop 2023-09-23T03:26:31.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.606+0000 7f901384a100 1 -- 172.21.15.154:0/1876854252 shutdown_connections 2023-09-23T03:26:31.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.606+0000 7f901384a100 1 --2- 172.21.15.154:0/1876854252 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5592aec1a000 0x5592ae84b700 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.606+0000 7f901384a100 1 --2- 172.21.15.154:0/1876854252 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5592aeb20c00 0x5592ae84a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.606+0000 7f901384a100 1 --2- 172.21.15.154:0/1876854252 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5592aeb21000 0x5592ae84ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.606+0000 7f901384a100 1 --2- 172.21.15.154:0/1876854252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592af59d800 0x5592ae84b180 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.607+0000 7f901384a100 1 -- 172.21.15.154:0/1876854252 >> 172.21.15.154:0/1876854252 conn(0x5592af59dc00 msgr2=0x5592af551000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:31.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.607+0000 7f901384a100 1 -- 172.21.15.154:0/1876854252 shutdown_connections 2023-09-23T03:26:31.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.607+0000 7f901384a100 1 -- 172.21.15.154:0/1876854252 wait complete. 2023-09-23T03:26:31.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.607+0000 7f901384a100 1 -- 172.21.15.154:0/3821944935 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592ae7d9800 msgr2=0x5592ae848b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.607+0000 7f901384a100 1 --2- 172.21.15.154:0/3821944935 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592ae7d9800 0x5592ae848b00 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x5592af5eadb0 tx=0x5592aeaba120 comp rx=0 tx=0).stop 2023-09-23T03:26:31.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.607+0000 7f901384a100 1 -- 172.21.15.154:0/3821944935 shutdown_connections 2023-09-23T03:26:31.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.607+0000 7f901384a100 1 --2- 172.21.15.154:0/3821944935 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5592ae7d9c00 0x5592ae849080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.607+0000 7f901384a100 1 --2- 172.21.15.154:0/3821944935 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592ae7d9800 0x5592ae848b00 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.607+0000 7f901384a100 1 --2- 172.21.15.154:0/3821944935 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5592ae7d8c00 0x5592ae848580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.607+0000 7f901384a100 1 -- 172.21.15.154:0/3821944935 >> 172.21.15.154:0/3821944935 conn(0x5592ae7d9000 msgr2=0x5592af550800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:31.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.607+0000 7f901384a100 1 -- 172.21.15.154:0/3821944935 shutdown_connections 2023-09-23T03:26:31.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.607+0000 7f901384a100 1 -- 172.21.15.154:0/3821944935 wait complete. 2023-09-23T03:26:31.612 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.121718, result: writing EResetJournal entry 2023-09-23T03:26:31.612 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 init 2023-09-23T03:26:31.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.696+0000 7f02531805c0 1 -- start start 2023-09-23T03:26:31.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.696+0000 7f02531805c0 1 Processor -- start 2023-09-23T03:26:31.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.697+0000 7f02531805c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560af962e800 0x560af95b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.697+0000 7f02531805c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560af962fc00 0x560af95b9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.697+0000 7f02531805c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560af962f400 0x560af95b9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.697+0000 7f02531805c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af9599180 con 0x560af962fc00 2023-09-23T03:26:31.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.697+0000 7f02531805c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af961d180 con 0x560af962f400 2023-09-23T03:26:31.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.697+0000 7f02531805c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x560af9598b40 con 0x560af962e800 2023-09-23T03:26:31.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.697+0000 7f0251127640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560af962fc00 0x560af95b9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.697+0000 7f0251127640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560af962fc00 0x560af95b9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46008/0 (socket says 172.21.15.154:46008) 2023-09-23T03:26:31.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.697+0000 7f0251127640 1 -- 172.21.15.154:0/3927438139 learned_addr learned my addr 172.21.15.154:0/3927438139 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:31.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.697+0000 7f0251127640 1 -- 172.21.15.154:0/3927438139 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560af962e800 msgr2=0x560af95b8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:31.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.697+0000 7f0251127640 1 --2- 172.21.15.154:0/3927438139 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560af962e800 0x560af95b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.697+0000 7f0251127640 1 -- 172.21.15.154:0/3927438139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560af962f400 msgr2=0x560af95b9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:31.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.697+0000 7f0251127640 1 --2- 172.21.15.154:0/3927438139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560af962f400 0x560af95b9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.697+0000 7f0251127640 1 -- 172.21.15.154:0/3927438139 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560af9582000 con 0x560af962fc00 2023-09-23T03:26:31.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.698+0000 7f0251127640 1 --2- 172.21.15.154:0/3927438139 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560af962fc00 0x560af95b9b80 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x560afa3613e0 tx=0x560afa32a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=accf2b4ff70f3393 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.698+0000 7f024f123640 1 -- 172.21.15.154:0/3927438139 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560af9597080 con 0x560af962fc00 2023-09-23T03:26:31.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.698+0000 7f024f123640 1 -- 172.21.15.154:0/3927438139 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560afa41a000 con 0x560af962fc00 2023-09-23T03:26:31.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.698+0000 7f024f123640 1 -- 172.21.15.154:0/3927438139 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560af9597600 con 0x560af962fc00 2023-09-23T03:26:31.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.698+0000 7f02531805c0 1 -- 172.21.15.154:0/3927438139 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560af9582ea0 con 0x560af962fc00 2023-09-23T03:26:31.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.698+0000 7f02531805c0 1 -- 172.21.15.154:0/3927438139 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x560af9583520 con 0x560af962fc00 2023-09-23T03:26:31.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.698+0000 7f02531805c0 4 waiting for MDS map... 2023-09-23T03:26:31.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.698+0000 7f024f123640 1 -- 172.21.15.154:0/3927438139 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(110..110 src has 1..110) v4 ==== 6605+0+0 (secure 0 0 0) 0x560af960ec40 con 0x560af962fc00 2023-09-23T03:26:31.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.698+0000 7f024f123640 1 -- 172.21.15.154:0/3927438139 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 91) v1 ==== 3306+0+0 (secure 0 0 0) 0x560af95a1800 con 0x560af962fc00 2023-09-23T03:26:31.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.698+0000 7f02531805c0 4 Got MDS map 91 2023-09-23T03:26:31.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.699+0000 7f02531805c0 4 datascan.main: Using metadata pool output 2023-09-23T03:26:31.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.699+0000 7f02531805c0 4 datascan.main: connecting to RADOS... 2023-09-23T03:26:31.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.699+0000 7f02531805c0 1 Processor -- start 2023-09-23T03:26:31.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.699+0000 7f02531805c0 1 -- start start 2023-09-23T03:26:31.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.700+0000 7f02531805c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560afa414c00 0x560af95b8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.700+0000 7f02531805c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560afa415000 0x560af95ba100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.700+0000 7f02531805c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560afa415400 0x560af95ba680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.700+0000 7f02531805c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af9598a00 con 0x560afa414c00 2023-09-23T03:26:31.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.700+0000 7f02531805c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af9598000 con 0x560afa415400 2023-09-23T03:26:31.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.700+0000 7f02531805c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x560af961d180 con 0x560afa415000 2023-09-23T03:26:31.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.700+0000 7f0252129640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560afa414c00 0x560af95b8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.700+0000 7f0252129640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560afa414c00 0x560af95b8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46014/0 (socket says 172.21.15.154:46014) 2023-09-23T03:26:31.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.700+0000 7f0252129640 1 -- 172.21.15.154:0/2380481662 learned_addr learned my addr 172.21.15.154:0/2380481662 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:31.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.701+0000 7f0252129640 1 -- 172.21.15.154:0/2380481662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560afa415000 msgr2=0x560af95ba100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:31.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.701+0000 7f0252129640 1 --2- 172.21.15.154:0/2380481662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560afa415000 0x560af95ba100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.701+0000 7f0252129640 1 -- 172.21.15.154:0/2380481662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560afa415400 msgr2=0x560af95ba680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:31.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.701+0000 7f0252129640 1 --2- 172.21.15.154:0/2380481662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560afa415400 0x560af95ba680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.701+0000 7f0252129640 1 -- 172.21.15.154:0/2380481662 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560af95836c0 con 0x560afa414c00 2023-09-23T03:26:31.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.701+0000 7f0252129640 1 --2- 172.21.15.154:0/2380481662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560afa414c00 0x560af95b8580 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x560afa2ca3f0 tx=0x560afa42cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f79b5efa1492bc9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.701+0000 7f024d11f640 1 -- 172.21.15.154:0/2380481662 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560af9597080 con 0x560afa414c00 2023-09-23T03:26:31.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.701+0000 7f024d11f640 1 -- 172.21.15.154:0/2380481662 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560afa41a180 con 0x560afa414c00 2023-09-23T03:26:31.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.701+0000 7f024d11f640 1 -- 172.21.15.154:0/2380481662 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560af9597600 con 0x560afa414c00 2023-09-23T03:26:31.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.702+0000 7f02531805c0 1 -- 172.21.15.154:0/2380481662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560afa414c00 msgr2=0x560af95b8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.702+0000 7f02531805c0 1 --2- 172.21.15.154:0/2380481662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560afa414c00 0x560af95b8580 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x560afa2ca3f0 tx=0x560afa42cd20 comp rx=0 tx=0).stop 2023-09-23T03:26:31.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.702+0000 7f02531805c0 1 -- 172.21.15.154:0/2380481662 shutdown_connections 2023-09-23T03:26:31.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.702+0000 7f02531805c0 1 --2- 172.21.15.154:0/2380481662 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560afa415400 0x560af95ba680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.702+0000 7f02531805c0 1 --2- 172.21.15.154:0/2380481662 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560afa415000 0x560af95ba100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.702+0000 7f02531805c0 1 --2- 172.21.15.154:0/2380481662 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560afa414c00 0x560af95b8580 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.702+0000 7f02531805c0 1 -- 172.21.15.154:0/2380481662 >> 172.21.15.154:0/2380481662 conn(0x560af962f800 msgr2=0x560afa2c8800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:31.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.702+0000 7f02531805c0 1 -- 172.21.15.154:0/2380481662 shutdown_connections 2023-09-23T03:26:31.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.702+0000 7f02531805c0 1 -- 172.21.15.154:0/2380481662 wait complete. 2023-09-23T03:26:31.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.702+0000 7f02531805c0 1 Processor -- start 2023-09-23T03:26:31.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.703+0000 7f02531805c0 1 -- start start 2023-09-23T03:26:31.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.703+0000 7f02531805c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560afa414c00 0x560af95b8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.703+0000 7f02531805c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560afa415000 0x560af95ba100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.703+0000 7f02531805c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560afa415400 0x560af95ba680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.703+0000 7f02531805c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af961cdc0 con 0x560afa415000 2023-09-23T03:26:31.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.703+0000 7f02531805c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560af961cc80 con 0x560afa414c00 2023-09-23T03:26:31.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.703+0000 7f02531805c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x560af961cb40 con 0x560afa415400 2023-09-23T03:26:31.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.704+0000 7f0251928640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560afa415000 0x560af95ba100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.704+0000 7f0251928640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560afa415000 0x560af95ba100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46028/0 (socket says 172.21.15.154:46028) 2023-09-23T03:26:31.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.704+0000 7f0251928640 1 -- 172.21.15.154:0/3828019763 learned_addr learned my addr 172.21.15.154:0/3828019763 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:31.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.704+0000 7f0252129640 1 --2- 172.21.15.154:0/3828019763 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560afa414c00 0x560af95b8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.704+0000 7f0251928640 1 -- 172.21.15.154:0/3828019763 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560afa415400 msgr2=0x560af95ba680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.704+0000 7f0251928640 1 --2- 172.21.15.154:0/3828019763 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560afa415400 0x560af95ba680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.704+0000 7f0251928640 1 -- 172.21.15.154:0/3828019763 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560afa414c00 msgr2=0x560af95b8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.704+0000 7f0251928640 1 --2- 172.21.15.154:0/3828019763 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560afa414c00 0x560af95b8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.704+0000 7f0251928640 1 -- 172.21.15.154:0/3828019763 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560af9582340 con 0x560afa415000 2023-09-23T03:26:31.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.704+0000 7f0252129640 1 --2- 172.21.15.154:0/3828019763 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560afa414c00 0x560af95b8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:26:31.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.704+0000 7f0251928640 1 --2- 172.21.15.154:0/3828019763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560afa415000 0x560af95ba100 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x560afa429620 tx=0x560afa42cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.704+0000 7f024b91c640 1 -- 172.21.15.154:0/3828019763 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560af9597080 con 0x560afa415000 2023-09-23T03:26:31.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.705+0000 7f024b91c640 1 -- 172.21.15.154:0/3828019763 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560af9575800 con 0x560afa415000 2023-09-23T03:26:31.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.705+0000 7f024b91c640 1 -- 172.21.15.154:0/3828019763 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560af9597600 con 0x560afa415000 2023-09-23T03:26:31.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.705+0000 7f02531805c0 1 -- 172.21.15.154:0/3828019763 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560af95831e0 con 0x560afa415000 2023-09-23T03:26:31.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.705+0000 7f02531805c0 1 -- 172.21.15.154:0/3828019763 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560af9582b60 con 0x560afa415000 2023-09-23T03:26:31.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.705+0000 7f02531805c0 4 datascan.init: resolving metadata pool 14 2023-09-23T03:26:31.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.706+0000 7f024b91c640 1 -- 172.21.15.154:0/3828019763 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x560af9581340 con 0x560afa415000 2023-09-23T03:26:31.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.707+0000 7f024b91c640 1 --2- 172.21.15.154:0/3828019763 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x560afa48ac00 0x560af95bb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.707+0000 7f024b91c640 1 -- 172.21.15.154:0/3828019763 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6605+0+0 (secure 0 0 0) 0x560af960ea80 con 0x560afa415000 2023-09-23T03:26:31.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.707+0000 7f02531805c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-09-23T03:26:31.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.707+0000 7f02531805c0 1 --2- 172.21.15.154:0/3828019763 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x560afa415800 0x560af95bac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.707+0000 7f0252129640 1 --2- 172.21.15.154:0/3828019763 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x560afa48ac00 0x560af95bb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.707+0000 7f02531805c0 1 -- 172.21.15.154:0/3828019763 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:1 14.14 14:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x560af95a9880 con 0x560afa415800 2023-09-23T03:26:31.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.708+0000 7f0252129640 1 --2- 172.21.15.154:0/3828019763 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x560afa48ac00 0x560af95bb700 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x560afa2ca3f0 tx=0x560afa42cc00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.708+0000 7f0251127640 1 --2- 172.21.15.154:0/3828019763 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x560afa415800 0x560af95bac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.709+0000 7f0251127640 1 --2- 172.21.15.154:0/3828019763 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x560afa415800 0x560af95bac00 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.709+0000 7f0251127640 1 -- 172.21.15.154:0/3828019763 <== osd.4 v2:172.21.15.190:6808/717302641 1 ==== osd_op_reply(1 1.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 160+0+0 (crc 0 0 0) 0x560afa2a2b40 con 0x560afa415800 2023-09-23T03:26:31.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.709+0000 7f02531805c0 1 -- 172.21.15.154:0/3828019763 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:2 14.14 14:287034c4:::1.00000000.inode:head [writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x560af95a9c00 con 0x560afa415800 2023-09-23T03:26:31.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.712+0000 7f0251127640 1 -- 172.21.15.154:0/3828019763 <== osd.4 v2:172.21.15.190:6808/717302641 2 ==== osd_op_reply(2 1.00000000.inode [writefull 0~582] v110'50 uv50 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x560afa2a2b40 con 0x560afa415800 2023-09-23T03:26:31.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.712+0000 7f02531805c0 1 --2- 172.21.15.154:0/3828019763 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x560afa415c00 0x560af95bb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.712+0000 7f02531805c0 1 -- 172.21.15.154:0/3828019763 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:3 14.1f 14:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x560af98b8000 con 0x560afa415c00 2023-09-23T03:26:31.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.712+0000 7f0252129640 1 --2- 172.21.15.154:0/3828019763 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x560afa415c00 0x560af95bb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.713+0000 7f0252129640 1 --2- 172.21.15.154:0/3828019763 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x560afa415c00 0x560af95bb180 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.713+0000 7f0252129640 1 -- 172.21.15.154:0/3828019763 <== osd.0 v2:172.21.15.154:6800/2920938138 1 ==== osd_op_reply(3 100.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 162+0+0 (crc 0 0 0) 0x560afa2a3200 con 0x560afa415c00 2023-09-23T03:26:31.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.713+0000 7f02531805c0 1 -- 172.21.15.154:0/3828019763 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:4 14.1f 14:fe07bba1:::100.00000000.inode:head [writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x560af98b8380 con 0x560afa415c00 2023-09-23T03:26:31.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.715+0000 7f0252129640 1 -- 172.21.15.154:0/3828019763 <== osd.0 v2:172.21.15.154:6800/2920938138 2 ==== osd_op_reply(4 100.00000000.inode [writefull 0~582] v110'10 uv10 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x560afa2a3200 con 0x560afa415c00 2023-09-23T03:26:31.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.715+0000 7f02531805c0 1 --2- 172.21.15.154:0/3828019763 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x560afa48b000 0x560af98c2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.715+0000 7f02531805c0 1 -- 172.21.15.154:0/3828019763 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:5 14.13 14:cd5a64a3:::100.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x560af98b8700 con 0x560afa48b000 2023-09-23T03:26:31.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.716+0000 7f0251928640 1 --2- 172.21.15.154:0/3828019763 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x560afa48b000 0x560af98c2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.716+0000 7f0251928640 1 --2- 172.21.15.154:0/3828019763 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x560afa48b000 0x560af98c2000 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.717+0000 7f0251928640 1 -- 172.21.15.154:0/3828019763 <== osd.2 v2:172.21.15.154:6822/2884407243 1 ==== osd_op_reply(5 100.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x560afa2a38c0 con 0x560afa48b000 2023-09-23T03:26:31.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.717+0000 7f02531805c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-09-23T03:26:31.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.717+0000 7f02531805c0 1 -- 172.21.15.154:0/3828019763 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:6 14.13 14:cd5a64a3:::100.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x560af98b8a80 con 0x560afa48b000 2023-09-23T03:26:31.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.719+0000 7f0251928640 1 -- 172.21.15.154:0/3828019763 <== osd.2 v2:172.21.15.154:6822/2884407243 2 ==== osd_op_reply(6 100.00000000 [create,omap-set-header] v110'7 uv7 ondisk = 0) v8 ==== 198+0+0 (crc 0 0 0) 0x560afa2a38c0 con 0x560afa48b000 2023-09-23T03:26:31.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.719+0000 7f02531805c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x100 2023-09-23T03:26:31.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.719+0000 7f02531805c0 1 -- 172.21.15.154:0/3828019763 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x560afa415c00 msgr2=0x560af95bb180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.719+0000 7f02531805c0 1 --2- 172.21.15.154:0/3828019763 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x560afa415c00 0x560af95bb180 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.719+0000 7f02531805c0 1 -- 172.21.15.154:0/3828019763 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x560afa48b000 msgr2=0x560af98c2000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.719+0000 7f02531805c0 1 --2- 172.21.15.154:0/3828019763 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x560afa48b000 0x560af98c2000 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.719+0000 7f02531805c0 1 -- 172.21.15.154:0/3828019763 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x560afa415800 msgr2=0x560af95bac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.719+0000 7f02531805c0 1 --2- 172.21.15.154:0/3828019763 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x560afa415800 0x560af95bac00 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.719+0000 7f0251928640 1 -- 172.21.15.154:0/3828019763 reap_dead start 2023-09-23T03:26:31.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.719+0000 7f02531805c0 1 -- 172.21.15.154:0/3828019763 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x560afa48ac00 msgr2=0x560af95bb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.719+0000 7f02531805c0 1 --2- 172.21.15.154:0/3828019763 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x560afa48ac00 0x560af95bb700 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x560afa2ca3f0 tx=0x560afa42cc00 comp rx=0 tx=0).stop 2023-09-23T03:26:31.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.719+0000 7f02531805c0 1 -- 172.21.15.154:0/3828019763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560afa415000 msgr2=0x560af95ba100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.719+0000 7f02531805c0 1 --2- 172.21.15.154:0/3828019763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560afa415000 0x560af95ba100 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x560afa429620 tx=0x560afa42cf00 comp rx=0 tx=0).stop 2023-09-23T03:26:31.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.719+0000 7f02531805c0 1 -- 172.21.15.154:0/3828019763 shutdown_connections 2023-09-23T03:26:31.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.719+0000 7f02531805c0 1 --2- 172.21.15.154:0/3828019763 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x560afa48ac00 0x560af95bb700 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.719+0000 7f02531805c0 1 --2- 172.21.15.154:0/3828019763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560afa415000 0x560af95ba100 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.719+0000 7f02531805c0 1 -- 172.21.15.154:0/3828019763 >> 172.21.15.154:0/3828019763 conn(0x560af962f800 msgr2=0x560afa2c8800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:31.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.719+0000 7f02531805c0 1 -- 172.21.15.154:0/3828019763 shutdown_connections 2023-09-23T03:26:31.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.719+0000 7f02531805c0 1 -- 172.21.15.154:0/3828019763 wait complete. 2023-09-23T03:26:31.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.720+0000 7f02531805c0 1 -- 172.21.15.154:0/3927438139 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560af962fc00 msgr2=0x560af95b9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.720+0000 7f02531805c0 1 --2- 172.21.15.154:0/3927438139 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560af962fc00 0x560af95b9b80 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x560afa3613e0 tx=0x560afa32a1e0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.720+0000 7f02531805c0 1 -- 172.21.15.154:0/3927438139 shutdown_connections 2023-09-23T03:26:31.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.720+0000 7f02531805c0 1 --2- 172.21.15.154:0/3927438139 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560af962f400 0x560af95b9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.720+0000 7f02531805c0 1 --2- 172.21.15.154:0/3927438139 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560af962fc00 0x560af95b9b80 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.720+0000 7f02531805c0 1 --2- 172.21.15.154:0/3927438139 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560af962e800 0x560af95b8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.720+0000 7f02531805c0 1 -- 172.21.15.154:0/3927438139 >> 172.21.15.154:0/3927438139 conn(0x560af954f000 msgr2=0x560afa2c9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:31.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.720+0000 7f02531805c0 1 -- 172.21.15.154:0/3927438139 shutdown_connections 2023-09-23T03:26:31.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.720+0000 7f02531805c0 1 -- 172.21.15.154:0/3927438139 wait complete. 2023-09-23T03:26:31.725 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'init'] in time 0:00:00.112708, result: 2023-09-23T03:26:31.725 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents 2023-09-23T03:26:31.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.810+0000 7fe6558495c0 1 -- start start 2023-09-23T03:26:31.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.810+0000 7fe6558495c0 1 Processor -- start 2023-09-23T03:26:31.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.811+0000 7fe6558495c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f9cb90a400 0x55f9caba9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.811+0000 7fe6558495c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f9cb90b000 0x55f9caba8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.811+0000 7fe6558495c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f9cb90b400 0x55f9caba9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.811+0000 7fe6558495c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9cab89180 con 0x55f9cb90b400 2023-09-23T03:26:31.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.811+0000 7fe6558495c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9cab88140 con 0x55f9cb90b000 2023-09-23T03:26:31.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.811+0000 7fe6558495c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f9cab88b40 con 0x55f9cb90a400 2023-09-23T03:26:31.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.811+0000 7fe653fdf640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f9cb90b400 0x55f9caba9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.811+0000 7fe653fdf640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f9cb90b400 0x55f9caba9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46036/0 (socket says 172.21.15.154:46036) 2023-09-23T03:26:31.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.811+0000 7fe653fdf640 1 -- 172.21.15.154:0/3100203787 learned_addr learned my addr 172.21.15.154:0/3100203787 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:31.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.811+0000 7fe653fdf640 1 -- 172.21.15.154:0/3100203787 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f9cb90a400 msgr2=0x55f9caba9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:31.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.811+0000 7fe653fdf640 1 --2- 172.21.15.154:0/3100203787 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f9cb90a400 0x55f9caba9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.811+0000 7fe652fdd640 1 --2- 172.21.15.154:0/3100203787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f9cb90b000 0x55f9caba8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.811+0000 7fe653fdf640 1 -- 172.21.15.154:0/3100203787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f9cb90b000 msgr2=0x55f9caba8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:31.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.811+0000 7fe653fdf640 1 --2- 172.21.15.154:0/3100203787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f9cb90b000 0x55f9caba8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.811+0000 7fe653fdf640 1 -- 172.21.15.154:0/3100203787 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9cab72ea0 con 0x55f9cb90b400 2023-09-23T03:26:31.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.811+0000 7fe652fdd640 1 --2- 172.21.15.154:0/3100203787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f9cb90b000 0x55f9caba8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:26:31.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.812+0000 7fe653fdf640 1 --2- 172.21.15.154:0/3100203787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f9cb90b400 0x55f9caba9b80 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x55f9cb945500 tx=0x55f9cae36300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c7d29aec8a8693 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.812+0000 7fe650fd9640 1 -- 172.21.15.154:0/3100203787 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f9cab87080 con 0x55f9cb90b400 2023-09-23T03:26:31.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.812+0000 7fe650fd9640 1 -- 172.21.15.154:0/3100203787 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f9cab64a80 con 0x55f9cb90b400 2023-09-23T03:26:31.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.812+0000 7fe650fd9640 1 -- 172.21.15.154:0/3100203787 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f9cab87600 con 0x55f9cb90b400 2023-09-23T03:26:31.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.812+0000 7fe6558495c0 1 -- 172.21.15.154:0/3100203787 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f9cab73520 con 0x55f9cb90b400 2023-09-23T03:26:31.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.812+0000 7fe6558495c0 1 -- 172.21.15.154:0/3100203787 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55f9cab736c0 con 0x55f9cb90b400 2023-09-23T03:26:31.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.812+0000 7fe6558495c0 4 waiting for MDS map... 2023-09-23T03:26:31.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.812+0000 7fe650fd9640 1 -- 172.21.15.154:0/3100203787 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(110..110 src has 1..110) v4 ==== 6605+0+0 (secure 0 0 0) 0x55f9cabfec40 con 0x55f9cb90b400 2023-09-23T03:26:31.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.812+0000 7fe650fd9640 1 -- 172.21.15.154:0/3100203787 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 91) v1 ==== 3306+0+0 (secure 0 0 0) 0x55f9cab91200 con 0x55f9cb90b400 2023-09-23T03:26:31.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.812+0000 7fe6558495c0 4 Got MDS map 91 2023-09-23T03:26:31.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.812+0000 7fe6558495c0 4 datascan.main: Using metadata pool output 2023-09-23T03:26:31.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.812+0000 7fe6558495c0 4 datascan.main: connecting to RADOS... 2023-09-23T03:26:31.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.813+0000 7fe6558495c0 1 Processor -- start 2023-09-23T03:26:31.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.813+0000 7fe6558495c0 1 -- start start 2023-09-23T03:26:31.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.814+0000 7fe6558495c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f9cb90bc00 0x55f9caba9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.814+0000 7fe6558495c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f9cae2cc00 0x55f9cabaa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.814+0000 7fe6558495c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f9cae2d000 0x55f9cabaa680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.814+0000 7fe6558495c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9cab88a00 con 0x55f9cae2d000 2023-09-23T03:26:31.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.814+0000 7fe6558495c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9cab88000 con 0x55f9cae2cc00 2023-09-23T03:26:31.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.814+0000 7fe6558495c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f9cac0cdc0 con 0x55f9cb90bc00 2023-09-23T03:26:31.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.814+0000 7fe652fdd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f9cae2d000 0x55f9cabaa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.814+0000 7fe652fdd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f9cae2d000 0x55f9cabaa680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46038/0 (socket says 172.21.15.154:46038) 2023-09-23T03:26:31.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.814+0000 7fe652fdd640 1 -- 172.21.15.154:0/2222084274 learned_addr learned my addr 172.21.15.154:0/2222084274 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:31.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.815+0000 7fe6537de640 1 --2- 172.21.15.154:0/2222084274 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f9cae2cc00 0x55f9cabaa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.815+0000 7fe652fdd640 1 --2- 172.21.15.154:0/2222084274 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f9cb90bc00 0x55f9caba9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.815+0000 7fe652fdd640 1 -- 172.21.15.154:0/2222084274 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f9cb90bc00 msgr2=0x55f9caba9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:31.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.815+0000 7fe652fdd640 1 --2- 172.21.15.154:0/2222084274 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f9cb90bc00 0x55f9caba9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.815+0000 7fe652fdd640 1 -- 172.21.15.154:0/2222084274 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f9cae2cc00 msgr2=0x55f9cabaa100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:31.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.815+0000 7fe652fdd640 1 --2- 172.21.15.154:0/2222084274 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f9cae2cc00 0x55f9cabaa100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.815+0000 7fe652fdd640 1 -- 172.21.15.154:0/2222084274 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9cab73860 con 0x55f9cae2d000 2023-09-23T03:26:31.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.815+0000 7fe652fdd640 1 --2- 172.21.15.154:0/2222084274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f9cae2d000 0x55f9cabaa680 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x55f9cb944930 tx=0x55f9cae36120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107434bcb8940684 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.815+0000 7fe64efd5640 1 -- 172.21.15.154:0/2222084274 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f9cab86f20 con 0x55f9cae2d000 2023-09-23T03:26:31.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.815+0000 7fe64efd5640 1 -- 172.21.15.154:0/2222084274 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f9cae28180 con 0x55f9cae2d000 2023-09-23T03:26:31.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.815+0000 7fe64efd5640 1 -- 172.21.15.154:0/2222084274 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f9cab87080 con 0x55f9cae2d000 2023-09-23T03:26:31.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.816+0000 7fe6558495c0 1 -- 172.21.15.154:0/2222084274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f9cae2d000 msgr2=0x55f9cabaa680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.816+0000 7fe6558495c0 1 --2- 172.21.15.154:0/2222084274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f9cae2d000 0x55f9cabaa680 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x55f9cb944930 tx=0x55f9cae36120 comp rx=0 tx=0).stop 2023-09-23T03:26:31.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.816+0000 7fe6558495c0 1 -- 172.21.15.154:0/2222084274 shutdown_connections 2023-09-23T03:26:31.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.816+0000 7fe6558495c0 1 --2- 172.21.15.154:0/2222084274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f9cae2d000 0x55f9cabaa680 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.816+0000 7fe6558495c0 1 --2- 172.21.15.154:0/2222084274 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f9cae2cc00 0x55f9cabaa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.816+0000 7fe6558495c0 1 --2- 172.21.15.154:0/2222084274 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f9cb90bc00 0x55f9caba9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.816+0000 7fe6558495c0 1 -- 172.21.15.154:0/2222084274 >> 172.21.15.154:0/2222084274 conn(0x55f9cb90b800 msgr2=0x55f9cb8b9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:31.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.816+0000 7fe6558495c0 1 -- 172.21.15.154:0/2222084274 shutdown_connections 2023-09-23T03:26:31.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.816+0000 7fe6558495c0 1 -- 172.21.15.154:0/2222084274 wait complete. 2023-09-23T03:26:31.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.817+0000 7fe6558495c0 1 Processor -- start 2023-09-23T03:26:31.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.817+0000 7fe6558495c0 1 -- start start 2023-09-23T03:26:31.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.817+0000 7fe6558495c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f9cb90b800 0x55f9cabab700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.817+0000 7fe6558495c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f9cae2dc00 0x55f9cabab180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.817+0000 7fe6558495c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f9cae2d800 0x55f9cabaac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.817+0000 7fe6558495c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9cac0d180 con 0x55f9cae2d800 2023-09-23T03:26:31.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.817+0000 7fe6558495c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9cac0cc80 con 0x55f9cb90b800 2023-09-23T03:26:31.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.817+0000 7fe6558495c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f9cac0cb40 con 0x55f9cae2dc00 2023-09-23T03:26:31.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.818+0000 7fe652fdd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f9cae2d800 0x55f9cabaac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.818+0000 7fe652fdd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f9cae2d800 0x55f9cabaac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46054/0 (socket says 172.21.15.154:46054) 2023-09-23T03:26:31.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.818+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 learned_addr learned my addr 172.21.15.154:0/524486974 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:31.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.818+0000 7fe652fdd640 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f9cb90b800 0x55f9cabab700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.818+0000 7fe6537de640 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f9cae2dc00 0x55f9cabab180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.818+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f9cae2dc00 msgr2=0x55f9cabab180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.818+0000 7fe652fdd640 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f9cae2dc00 0x55f9cabab180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.818+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f9cb90b800 msgr2=0x55f9cabab700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:31.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.818+0000 7fe652fdd640 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f9cb90b800 0x55f9cabab700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:31.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.818+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9cab72000 con 0x55f9cae2d800 2023-09-23T03:26:31.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.818+0000 7fe6537de640 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f9cae2dc00 0x55f9cabab180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:26:31.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.818+0000 7fe652fdd640 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f9cae2d800 0x55f9cabaac00 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x55f9cae3f860 tx=0x55f9cae360c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.819+0000 7fe64d7d2640 1 -- 172.21.15.154:0/524486974 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f9cab87600 con 0x55f9cae2d800 2023-09-23T03:26:31.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.819+0000 7fe64d7d2640 1 -- 172.21.15.154:0/524486974 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f9cab64a80 con 0x55f9cae2d800 2023-09-23T03:26:31.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.819+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f9cab729c0 con 0x55f9cae2d800 2023-09-23T03:26:31.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.819+0000 7fe64d7d2640 1 -- 172.21.15.154:0/524486974 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f9cab86dc0 con 0x55f9cae2d800 2023-09-23T03:26:31.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.819+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f9cab73860 con 0x55f9cae2d800 2023-09-23T03:26:31.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.819+0000 7fe6558495c0 4 datascan.init: resolving metadata pool 14 2023-09-23T03:26:31.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.821+0000 7fe64d7d2640 1 -- 172.21.15.154:0/524486974 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x55f9cab71080 con 0x55f9cae2d800 2023-09-23T03:26:31.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.821+0000 7fe64d7d2640 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55f9caf92000 0x55f9caf98000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.821+0000 7fe64d7d2640 1 -- 172.21.15.154:0/524486974 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6605+0+0 (secure 0 0 0) 0x55f9cabfea80 con 0x55f9cae2d800 2023-09-23T03:26:31.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.821+0000 7fe6558495c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-09-23T03:26:31.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.822+0000 7fe6558495c0 4 datascan.main: data pool 'cephfs_data' has ID 15 2023-09-23T03:26:31.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.822+0000 7fe6558495c0 4 datascan.main: opening data pool 'cephfs_data' 2023-09-23T03:26:31.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.822+0000 7fe6558495c0 4 datascan.main: data pool 'cephfs_data_extra' has ID 16 2023-09-23T03:26:31.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.822+0000 7fe6558495c0 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-09-23T03:26:31.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.822+0000 7fe6558495c0 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55f9cae2d400 0x55f9caba9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.822+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:1 15.0 15:00000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cab99880 con 0x55f9cae2d400 2023-09-23T03:26:31.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.822+0000 7fe6537de640 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55f9caf92000 0x55f9caf98000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.822+0000 7fe652fdd640 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55f9cae2d400 0x55f9caba9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.822+0000 7fe6537de640 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55f9caf92000 0x55f9caf98000 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x55f9cac30930 tx=0x55f9cae4bce0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.822+0000 7fe652fdd640 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55f9cae2d400 0x55f9caba9080 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.823+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.2 v2:172.21.15.154:6822/2884407243 1 ==== osd_op_reply(1 [pgnls start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9cae2d400 2023-09-23T03:26:31.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.823+0000 7fe64e7d4640 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55f9caf93000 0x55f9caf98580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.823+0000 7fe64e7d4640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:2 15.20 15:04000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1e000 con 0x55f9caf93000 2023-09-23T03:26:31.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.823+0000 7fe653fdf640 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55f9caf93000 0x55f9caf98580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.824+0000 7fe653fdf640 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55f9caf93000 0x55f9caf98580 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.828 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.824+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 1 ==== osd_op_reply(2 [pgnls start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:31.828 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.824+0000 7fe64efd5640 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55f9caf93400 0x55f9caf98b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.828 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.824+0000 7fe64efd5640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:3 15.10 15:08000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1e380 con 0x55f9caf93400 2023-09-23T03:26:31.828 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.825+0000 7fe6537de640 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55f9caf93400 0x55f9caf98b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.828 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.826+0000 7fe6537de640 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55f9caf93400 0x55f9caf98b00 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.826+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.4 v2:172.21.15.190:6808/717302641 1 ==== osd_op_reply(3 [pgnls start_epoch 110 out=49b] v108'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93400 2023-09-23T03:26:31.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.826+0000 7fe64e7d4640 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55f9caf93800 0x55f9caf99080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.826+0000 7fe64e7d4640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:4 15.30 15:0c000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1e700 con 0x55f9caf93800 2023-09-23T03:26:31.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.827+0000 7fe652fdd640 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55f9caf93800 0x55f9caf99080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.828+0000 7fe652fdd640 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55f9caf93800 0x55f9caf99080 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.829+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.7 v2:172.21.15.190:6824/3460136272 1 ==== osd_op_reply(4 [pgnls start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf93800 2023-09-23T03:26:31.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.829+0000 7fe64efd5640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:5 15.8 15:10000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1e700 con 0x55f9cae2d400 2023-09-23T03:26:31.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.829+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.2 v2:172.21.15.154:6822/2884407243 2 ==== osd_op_reply(5 [pgnls start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9cae2d400 2023-09-23T03:26:31.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.829+0000 7fe64e7d4640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:6 15.28 15:14000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1ea80 con 0x55f9caf93800 2023-09-23T03:26:31.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.830+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.7 v2:172.21.15.190:6824/3460136272 2 ==== osd_op_reply(6 [pgnls start_epoch 110 out=49b] v108'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf93800 2023-09-23T03:26:31.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.830+0000 7fe64efd5640 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55f9caf06000 0x55f9caf99b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.830+0000 7fe64efd5640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:7 15.18 15:18000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cab99880 con 0x55f9caf06000 2023-09-23T03:26:31.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.830+0000 7fe653fdf640 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55f9caf06000 0x55f9caf99b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.831+0000 7fe653fdf640 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55f9caf06000 0x55f9caf99b80 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.831+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(7 [pgnls start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06000 2023-09-23T03:26:31.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.831+0000 7fe64e7d4640 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55f9caf93c00 0x55f9caf99600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.831+0000 7fe64e7d4640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:8 15.38 15:1c000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1ee00 con 0x55f9caf93c00 2023-09-23T03:26:31.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.832+0000 7fe6537de640 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55f9caf93c00 0x55f9caf99600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.832+0000 7fe6537de640 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55f9caf93c00 0x55f9caf99600 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.833+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.0 v2:172.21.15.154:6800/2920938138 1 ==== osd_op_reply(8 [pgnls start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93c00 2023-09-23T03:26:31.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.833+0000 7fe64efd5640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:9 15.4 15:20000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1e380 con 0x55f9caf93000 2023-09-23T03:26:31.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.833+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 2 ==== osd_op_reply(9 [pgnls start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:31.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.833+0000 7fe64e7d4640 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55f9caf06800 0x55f9caf9a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.833+0000 7fe64e7d4640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:10 15.24 15:24000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1f180 con 0x55f9caf06800 2023-09-23T03:26:31.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.834+0000 7fe652fdd640 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55f9caf06800 0x55f9caf9a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.835+0000 7fe652fdd640 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55f9caf06800 0x55f9caf9a680 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.835+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== osd_op_reply(10 [pgnls start_epoch 110 out=49b] v108'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf06800 2023-09-23T03:26:31.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.835+0000 7fe64efd5640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:11 15.14 15:28000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1ee00 con 0x55f9caf06800 2023-09-23T03:26:31.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.836+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.6 v2:172.21.15.190:6816/1964294001 2 ==== osd_op_reply(11 [pgnls start_epoch 110 out=49b] v108'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf06800 2023-09-23T03:26:31.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.836+0000 7fe64e7d4640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:12 15.34 15:2c000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1f500 con 0x55f9caf93400 2023-09-23T03:26:31.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.837+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.4 v2:172.21.15.190:6808/717302641 2 ==== osd_op_reply(12 [pgnls start_epoch 110 out=49b] v108'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93400 2023-09-23T03:26:31.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.837+0000 7fe64efd5640 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55f9caf06400 0x55f9caf9a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:31.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.837+0000 7fe64efd5640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:13 15.c 15:30000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1f880 con 0x55f9caf06400 2023-09-23T03:26:31.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.837+0000 7fe653fdf640 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55f9caf06400 0x55f9caf9a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:31.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.838+0000 7fe653fdf640 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55f9caf06400 0x55f9caf9a100 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:31.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.839+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.5 v2:172.21.15.190:6800/882536885 1 ==== osd_op_reply(13 [pgnls start_epoch 110 out=49b] v108'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06400 2023-09-23T03:26:31.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.839+0000 7fe64e7d4640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:14 15.2c 15:34000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1e700 con 0x55f9caf93400 2023-09-23T03:26:31.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.840+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.4 v2:172.21.15.190:6808/717302641 3 ==== osd_op_reply(14 [pgnls start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93400 2023-09-23T03:26:31.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.840+0000 7fe64efd5640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:15 15.1c 15:38000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1e000 con 0x55f9caf93000 2023-09-23T03:26:31.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.841+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 3 ==== osd_op_reply(15 [pgnls start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:31.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.841+0000 7fe64e7d4640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:16 15.3c 15:3c000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1ea80 con 0x55f9caf06000 2023-09-23T03:26:31.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.841+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.3 v2:172.21.15.154:6808/1133688468 2 ==== osd_op_reply(16 [pgnls start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06000 2023-09-23T03:26:31.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.841+0000 7fe64efd5640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:17 15.2 15:40000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1e380 con 0x55f9cae2d400 2023-09-23T03:26:31.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.842+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.2 v2:172.21.15.154:6822/2884407243 3 ==== osd_op_reply(17 [pgnls start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9cae2d400 2023-09-23T03:26:31.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.842+0000 7fe64e7d4640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:18 15.22 15:44000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1f180 con 0x55f9cae2d400 2023-09-23T03:26:31.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.842+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.2 v2:172.21.15.154:6822/2884407243 4 ==== osd_op_reply(18 [pgnls start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9cae2d400 2023-09-23T03:26:31.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.842+0000 7fe64efd5640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:19 15.12 15:48000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1fc00 con 0x55f9caf06000 2023-09-23T03:26:31.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.842+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.3 v2:172.21.15.154:6808/1133688468 3 ==== osd_op_reply(19 [pgnls start_epoch 110 out=49b] v108'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06000 2023-09-23T03:26:31.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.842+0000 7fe64e7d4640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:20 15.32 15:4c000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cab99880 con 0x55f9cae2d400 2023-09-23T03:26:31.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.843+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.2 v2:172.21.15.154:6822/2884407243 5 ==== osd_op_reply(20 [pgnls start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9cae2d400 2023-09-23T03:26:31.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.843+0000 7fe64efd5640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:21 15.a 15:50000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1ee00 con 0x55f9cae2d400 2023-09-23T03:26:31.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.843+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.2 v2:172.21.15.154:6822/2884407243 6 ==== osd_op_reply(21 [pgnls start_epoch 110 out=49b] v108'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9cae2d400 2023-09-23T03:26:31.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.843+0000 7fe64e7d4640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:22 15.2a 15:54000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1e000 con 0x55f9caf93c00 2023-09-23T03:26:31.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.843+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.0 v2:172.21.15.154:6800/2920938138 2 ==== osd_op_reply(22 [pgnls start_epoch 110 out=81b] v108'10 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93c00 2023-09-23T03:26:31.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.843+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:23 15.2a 15:57fe7535:::100000001f6.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cab99c00 con 0x55f9caf93c00 2023-09-23T03:26:31.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.844+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.0 v2:172.21.15.154:6800/2920938138 3 ==== osd_op_reply(23 100000001f6.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93c00 2023-09-23T03:26:31.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.844+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628278.0 2023-09-23T03:26:31.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.844+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:24 15.2a 15:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1f500 con 0x55f9caf93c00 2023-09-23T03:26:31.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.847+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.0 v2:172.21.15.154:6800/2920938138 4 ==== osd_op_reply(24 100000001f6.00000000 [call] v110'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93c00 2023-09-23T03:26:31.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.847+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:25 15.1a 15:58000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1e700 con 0x55f9caf06400 2023-09-23T03:26:31.848 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.848+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.5 v2:172.21.15.190:6800/882536885 2 ==== osd_op_reply(25 [pgnls start_epoch 110 out=49b] v108'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06400 2023-09-23T03:26:31.848 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.848+0000 7fe64e7d4640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:26 15.3a 15:5c000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1ea80 con 0x55f9caf93800 2023-09-23T03:26:31.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.849+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.7 v2:172.21.15.190:6824/3460136272 3 ==== osd_op_reply(26 [pgnls start_epoch 110 out=49b] v108'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf93800 2023-09-23T03:26:31.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.849+0000 7fe64efd5640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:27 15.6 15:60000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1e380 con 0x55f9caf93c00 2023-09-23T03:26:31.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.849+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.0 v2:172.21.15.154:6800/2920938138 5 ==== osd_op_reply(27 [pgnls start_epoch 110 out=49b] v108'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93c00 2023-09-23T03:26:31.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.849+0000 7fe64e7d4640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:28 15.26 15:64000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1fc00 con 0x55f9caf93c00 2023-09-23T03:26:31.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.850+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.0 v2:172.21.15.154:6800/2920938138 6 ==== osd_op_reply(28 [pgnls start_epoch 110 out=49b] v108'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93c00 2023-09-23T03:26:31.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.850+0000 7fe64efd5640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:29 15.16 15:68000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1f180 con 0x55f9caf93c00 2023-09-23T03:26:31.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.850+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.0 v2:172.21.15.154:6800/2920938138 7 ==== osd_op_reply(29 [pgnls start_epoch 110 out=49b] v108'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93c00 2023-09-23T03:26:31.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.850+0000 7fe64e7d4640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:30 15.36 15:6c000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1f500 con 0x55f9caf06800 2023-09-23T03:26:31.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.851+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.6 v2:172.21.15.190:6816/1964294001 3 ==== osd_op_reply(30 [pgnls start_epoch 110 out=101b] v108'17 uv2 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf06800 2023-09-23T03:26:31.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.851+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:31 15.36 15:6ee5e2c4:::100000001f9.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1e000 con 0x55f9caf06800 2023-09-23T03:26:31.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.852+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.6 v2:172.21.15.190:6816/1964294001 4 ==== osd_op_reply(31 100000001f9.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf06800 2023-09-23T03:26:31.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.852+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628281.0 2023-09-23T03:26:31.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.852+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:32 15.36 15:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd6000 con 0x55f9caf06800 2023-09-23T03:26:31.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.855+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.6 v2:172.21.15.190:6816/1964294001 5 ==== osd_op_reply(32 100000001f9.00000000 [call] v110'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf06800 2023-09-23T03:26:31.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.855+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:33 15.36 15:6fe38dc2::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd6380 con 0x55f9caf06800 2023-09-23T03:26:31.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.856+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.6 v2:172.21.15.190:6816/1964294001 6 ==== osd_op_reply(33 [pgnls start_epoch 110 out=81b] v110'18 uv18 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf06800 2023-09-23T03:26:31.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.856+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:34 15.36 15:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd6700 con 0x55f9caf06800 2023-09-23T03:26:31.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.857+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.6 v2:172.21.15.190:6816/1964294001 7 ==== osd_op_reply(34 100000001f7.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf06800 2023-09-23T03:26:31.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.857+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628279.0 2023-09-23T03:26:31.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.857+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:35 15.36 15:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd6a80 con 0x55f9caf06800 2023-09-23T03:26:31.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.860+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.6 v2:172.21.15.190:6816/1964294001 8 ==== osd_op_reply(35 100000001f7.00000000 [call] v110'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf06800 2023-09-23T03:26:31.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.860+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:36 15.e 15:70000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd6e00 con 0x55f9cae2d400 2023-09-23T03:26:31.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.861+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.2 v2:172.21.15.154:6822/2884407243 7 ==== osd_op_reply(36 [pgnls start_epoch 110 out=49b] v108'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9cae2d400 2023-09-23T03:26:31.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.861+0000 7fe64efd5640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:37 15.2e 15:74000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1f880 con 0x55f9caf06000 2023-09-23T03:26:31.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.862+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.3 v2:172.21.15.154:6808/1133688468 4 ==== osd_op_reply(37 [pgnls start_epoch 110 out=49b] v108'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06000 2023-09-23T03:26:31.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.862+0000 7fe64e7d4640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:38 15.1e 15:78000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1e380 con 0x55f9cae2d400 2023-09-23T03:26:31.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.862+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.2 v2:172.21.15.154:6822/2884407243 8 ==== osd_op_reply(38 [pgnls start_epoch 110 out=49b] v108'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9cae2d400 2023-09-23T03:26:31.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.862+0000 7fe64efd5640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:39 15.3e 15:7c000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cab99880 con 0x55f9caf93c00 2023-09-23T03:26:31.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.863+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.0 v2:172.21.15.154:6800/2920938138 8 ==== osd_op_reply(39 [pgnls start_epoch 110 out=49b] v108'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93c00 2023-09-23T03:26:31.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.863+0000 7fe64e7d4640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:40 15.1 15:80000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1fc00 con 0x55f9caf06800 2023-09-23T03:26:31.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.864+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.6 v2:172.21.15.190:6816/1964294001 9 ==== osd_op_reply(40 [pgnls start_epoch 110 out=49b] v108'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf06800 2023-09-23T03:26:31.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.864+0000 7fe64efd5640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:41 15.21 15:84000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1e380 con 0x55f9caf06000 2023-09-23T03:26:31.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.864+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.3 v2:172.21.15.154:6808/1133688468 5 ==== osd_op_reply(41 [pgnls start_epoch 110 out=49b] v108'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06000 2023-09-23T03:26:31.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.864+0000 7fe64e7d4640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:42 15.11 15:88000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cab99c00 con 0x55f9cae2d400 2023-09-23T03:26:31.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.865+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.2 v2:172.21.15.154:6822/2884407243 9 ==== osd_op_reply(42 [pgnls start_epoch 110 out=49b] v108'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9cae2d400 2023-09-23T03:26:31.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.865+0000 7fe64efd5640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:43 15.31 15:8c000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1fc00 con 0x55f9caf06000 2023-09-23T03:26:31.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.865+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.3 v2:172.21.15.154:6808/1133688468 6 ==== osd_op_reply(43 [pgnls start_epoch 110 out=49b] v108'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06000 2023-09-23T03:26:31.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.866+0000 7fe64e7d4640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:44 15.9 15:90000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe0000 con 0x55f9caf06800 2023-09-23T03:26:31.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.866+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.6 v2:172.21.15.190:6816/1964294001 10 ==== osd_op_reply(44 [pgnls start_epoch 110 out=49b] v108'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf06800 2023-09-23T03:26:31.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.867+0000 7fe64efd5640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:45 15.29 15:94000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1ee00 con 0x55f9caf93c00 2023-09-23T03:26:31.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.867+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.0 v2:172.21.15.154:6800/2920938138 9 ==== osd_op_reply(45 [pgnls start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93c00 2023-09-23T03:26:31.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.867+0000 7fe64e7d4640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:46 15.19 15:98000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe0380 con 0x55f9caf93400 2023-09-23T03:26:31.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.868+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.4 v2:172.21.15.190:6808/717302641 4 ==== osd_op_reply(46 [pgnls start_epoch 110 out=81b] v108'11 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93400 2023-09-23T03:26:31.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.868+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:47 15.19 15:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe1880 con 0x55f9caf93400 2023-09-23T03:26:31.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.869+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.4 v2:172.21.15.190:6808/717302641 5 ==== osd_op_reply(47 100000001f8.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93400 2023-09-23T03:26:31.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.869+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628280.0 2023-09-23T03:26:31.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.869+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:48 15.19 15:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe1c00 con 0x55f9caf93400 2023-09-23T03:26:31.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.874+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.4 v2:172.21.15.190:6808/717302641 6 ==== osd_op_reply(48 100000001f8.00000000 [call] v110'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93400 2023-09-23T03:26:31.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.874+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:49 15.39 15:9c000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe0380 con 0x55f9caf93800 2023-09-23T03:26:31.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.875+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.7 v2:172.21.15.190:6824/3460136272 4 ==== osd_op_reply(49 [pgnls start_epoch 110 out=49b] v108'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf93800 2023-09-23T03:26:31.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.875+0000 7fe64e7d4640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:50 15.5 15:a0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe0700 con 0x55f9caf06800 2023-09-23T03:26:31.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.876+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.6 v2:172.21.15.190:6816/1964294001 11 ==== osd_op_reply(50 [pgnls start_epoch 110 out=49b] v108'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf06800 2023-09-23T03:26:31.876 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.876+0000 7fe64efd5640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:51 15.25 15:a4000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1ea80 con 0x55f9caf93800 2023-09-23T03:26:31.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.877+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.7 v2:172.21.15.190:6824/3460136272 5 ==== osd_op_reply(51 [pgnls start_epoch 110 out=49b] v108'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf93800 2023-09-23T03:26:31.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.877+0000 7fe64e7d4640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:52 15.15 15:a8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe0a80 con 0x55f9caf93800 2023-09-23T03:26:31.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.878+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.7 v2:172.21.15.190:6824/3460136272 6 ==== osd_op_reply(52 [pgnls start_epoch 110 out=49b] v108'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf93800 2023-09-23T03:26:31.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.878+0000 7fe64efd5640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:53 15.35 15:ac000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1f500 con 0x55f9caf93800 2023-09-23T03:26:31.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.879+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.7 v2:172.21.15.190:6824/3460136272 7 ==== osd_op_reply(53 [pgnls start_epoch 110 out=49b] v108'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf93800 2023-09-23T03:26:31.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.879+0000 7fe64e7d4640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:54 15.d 15:b0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe0e00 con 0x55f9caf93800 2023-09-23T03:26:31.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.880+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.7 v2:172.21.15.190:6824/3460136272 8 ==== osd_op_reply(54 [pgnls start_epoch 110 out=49b] v108'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf93800 2023-09-23T03:26:31.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.880+0000 7fe64efd5640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:55 15.2d 15:b4000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1e000 con 0x55f9cae2d400 2023-09-23T03:26:31.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.881+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.2 v2:172.21.15.154:6822/2884407243 10 ==== osd_op_reply(55 [pgnls start_epoch 110 out=49b] v108'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9cae2d400 2023-09-23T03:26:31.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.881+0000 7fe64e7d4640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:56 15.1d 15:b8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe1180 con 0x55f9caf93400 2023-09-23T03:26:31.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.882+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.4 v2:172.21.15.190:6808/717302641 7 ==== osd_op_reply(56 [pgnls start_epoch 110 out=49b] v108'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93400 2023-09-23T03:26:31.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.882+0000 7fe64efd5640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:57 15.3d 15:bc000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1ea80 con 0x55f9caf93000 2023-09-23T03:26:31.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.883+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 4 ==== osd_op_reply(57 [pgnls start_epoch 110 out=49b] v108'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:31.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.883+0000 7fe64e7d4640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:58 15.3 15:c0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe1500 con 0x55f9caf93400 2023-09-23T03:26:31.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.884+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.4 v2:172.21.15.190:6808/717302641 8 ==== osd_op_reply(58 [pgnls start_epoch 110 out=49b] v108'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93400 2023-09-23T03:26:31.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.884+0000 7fe64efd5640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:59 15.23 15:c4000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1f500 con 0x55f9caf93c00 2023-09-23T03:26:31.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.885+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.0 v2:172.21.15.154:6800/2920938138 10 ==== osd_op_reply(59 [pgnls start_epoch 110 out=49b] v108'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93c00 2023-09-23T03:26:31.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.885+0000 7fe64e7d4640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:60 15.13 15:c8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1e700 con 0x55f9caf93000 2023-09-23T03:26:31.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.885+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 5 ==== osd_op_reply(60 [pgnls start_epoch 110 out=81b] v108'16 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:31.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.886+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:61 15.13 15:ca8fdbba:::100000001fb.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe1880 con 0x55f9caf93000 2023-09-23T03:26:31.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.886+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 6 ==== osd_op_reply(61 100000001fb.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:31.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.886+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628283.0 2023-09-23T03:26:31.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.886+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:62 15.13 15:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1f180 con 0x55f9caf93000 2023-09-23T03:26:31.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.889+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 7 ==== osd_op_reply(62 100000001fb.00000000 [call] v110'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:31.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.889+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:63 15.33 15:cc000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1ee00 con 0x55f9caf06000 2023-09-23T03:26:31.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.889+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.3 v2:172.21.15.154:6808/1133688468 7 ==== osd_op_reply(63 [pgnls start_epoch 110 out=49b] v108'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06000 2023-09-23T03:26:31.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.890+0000 7fe64e7d4640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:64 15.b 15:d0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1f880 con 0x55f9cae2d400 2023-09-23T03:26:31.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.890+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.2 v2:172.21.15.154:6822/2884407243 11 ==== osd_op_reply(64 [pgnls start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9cae2d400 2023-09-23T03:26:31.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.890+0000 7fe64efd5640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:65 15.2b 15:d4000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe0000 con 0x55f9cae2d400 2023-09-23T03:26:31.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.891+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.2 v2:172.21.15.154:6822/2884407243 12 ==== osd_op_reply(65 [pgnls start_epoch 110 out=49b] v108'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9cae2d400 2023-09-23T03:26:31.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.891+0000 7fe64e7d4640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:66 15.1b 15:d8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1e380 con 0x55f9caf93800 2023-09-23T03:26:31.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.892+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.7 v2:172.21.15.190:6824/3460136272 9 ==== osd_op_reply(66 [pgnls start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf93800 2023-09-23T03:26:31.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.892+0000 7fe64efd5640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:67 15.3b 15:dc000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe0380 con 0x55f9caf06800 2023-09-23T03:26:31.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.894+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.6 v2:172.21.15.190:6816/1964294001 12 ==== osd_op_reply(67 [pgnls start_epoch 110 out=49b] v108'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf06800 2023-09-23T03:26:31.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.894+0000 7fe64e7d4640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:68 15.7 15:e0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1fc00 con 0x55f9caf93000 2023-09-23T03:26:31.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.894+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 8 ==== osd_op_reply(68 [pgnls start_epoch 110 out=81b] v108'6 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:31.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.894+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:69 15.7 15:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cab99880 con 0x55f9caf93000 2023-09-23T03:26:31.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.895+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 9 ==== osd_op_reply(69 10000000001.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:31.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.895+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511627777.0 2023-09-23T03:26:31.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.895+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:70 15.7 15:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cab99880 con 0x55f9caf93000 2023-09-23T03:26:31.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.898+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 10 ==== osd_op_reply(70 10000000001.00000000 [call] v110'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:31.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.898+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:71 15.27 15:e4000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1e700 con 0x55f9caf06800 2023-09-23T03:26:31.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.899+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.6 v2:172.21.15.190:6816/1964294001 13 ==== osd_op_reply(71 [pgnls start_epoch 110 out=49b] v108'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf06800 2023-09-23T03:26:31.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.899+0000 7fe64e7d4640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:72 15.17 15:e8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1ea80 con 0x55f9caf93400 2023-09-23T03:26:31.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.901+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.4 v2:172.21.15.190:6808/717302641 9 ==== osd_op_reply(72 [pgnls start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93400 2023-09-23T03:26:31.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.901+0000 7fe64efd5640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:73 15.37 15:ec000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe0700 con 0x55f9caf93400 2023-09-23T03:26:31.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.902+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.4 v2:172.21.15.190:6808/717302641 10 ==== osd_op_reply(73 [pgnls start_epoch 110 out=49b] v108'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93400 2023-09-23T03:26:31.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.902+0000 7fe64e7d4640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:74 15.f 15:f0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe1c00 con 0x55f9caf06800 2023-09-23T03:26:31.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.903+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.6 v2:172.21.15.190:6816/1964294001 14 ==== osd_op_reply(74 [pgnls start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf06800 2023-09-23T03:26:31.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.903+0000 7fe64efd5640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:75 15.2f 15:f4000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe0a80 con 0x55f9caf93c00 2023-09-23T03:26:31.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.904+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.0 v2:172.21.15.154:6800/2920938138 11 ==== osd_op_reply(75 [pgnls start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93c00 2023-09-23T03:26:31.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.904+0000 7fe64e7d4640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:76 15.1f 15:f8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe1180 con 0x55f9caf06000 2023-09-23T03:26:31.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.905+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.3 v2:172.21.15.154:6808/1133688468 8 ==== osd_op_reply(76 [pgnls start_epoch 110 out=49b] v108'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06000 2023-09-23T03:26:31.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.905+0000 7fe64efd5640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:77 15.3f 15:fc000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cab99c00 con 0x55f9caf93000 2023-09-23T03:26:31.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.906+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 11 ==== osd_op_reply(77 [pgnls start_epoch 110 out=81b] v108'9 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:31.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.906+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:78 15.3f 15:feb1c803:::100000001fa.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1f180 con 0x55f9caf93000 2023-09-23T03:26:31.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.907+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 12 ==== osd_op_reply(78 100000001fa.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:31.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.907+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628282.0 2023-09-23T03:26:31.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.907+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:79 15.3f 15:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1ee00 con 0x55f9caf93000 2023-09-23T03:26:31.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.910+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 13 ==== osd_op_reply(79 100000001fa.00000000 [call] v110'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:31.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.910+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:80 16.0 16:00000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1fc00 con 0x55f9caf93c00 2023-09-23T03:26:31.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.911+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.0 v2:172.21.15.154:6800/2920938138 12 ==== osd_op_reply(80 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93c00 2023-09-23T03:26:31.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.912+0000 7fe64efd5640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:81 16.10 16:08000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe0e00 con 0x55f9cae2d400 2023-09-23T03:26:31.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.912+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.2 v2:172.21.15.154:6822/2884407243 13 ==== osd_op_reply(81 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9cae2d400 2023-09-23T03:26:31.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.913+0000 7fe64e7d4640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:82 16.8 16:10000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe1500 con 0x55f9caf93c00 2023-09-23T03:26:31.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.914+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.0 v2:172.21.15.154:6800/2920938138 13 ==== osd_op_reply(82 [pgnls start_epoch 110 out=81b] v108'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93c00 2023-09-23T03:26:31.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.914+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:83 16.8 16:1247e94b:::100000001f9.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe1880 con 0x55f9caf93c00 2023-09-23T03:26:31.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.915+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.0 v2:172.21.15.154:6800/2920938138 14 ==== osd_op_reply(83 100000001f9.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93c00 2023-09-23T03:26:31.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.915+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628281.1 2023-09-23T03:26:31.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.915+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:84 15.36 15:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd6000 con 0x55f9caf06800 2023-09-23T03:26:31.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.919+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.6 v2:172.21.15.190:6816/1964294001 15 ==== osd_op_reply(84 100000001f9.00000000 [call,setxattr (8)] v110'20 uv20 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf06800 2023-09-23T03:26:31.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.919+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:85 16.18 16:18000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd6380 con 0x55f9caf93c00 2023-09-23T03:26:31.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.920+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.0 v2:172.21.15.154:6800/2920938138 15 ==== osd_op_reply(85 [pgnls start_epoch 110 out=81b] v108'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93c00 2023-09-23T03:26:31.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.921+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:86 16.18 16:1cd44203:::100000001fb.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd6700 con 0x55f9caf93c00 2023-09-23T03:26:31.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.921+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.0 v2:172.21.15.154:6800/2920938138 16 ==== osd_op_reply(86 100000001fb.00000002 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93c00 2023-09-23T03:26:31.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.922+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628283.2 2023-09-23T03:26:31.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.922+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:87 15.13 15:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd6a80 con 0x55f9caf93000 2023-09-23T03:26:31.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.925+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 14 ==== osd_op_reply(87 100000001fb.00000000 [call,setxattr (8)] v110'18 uv18 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:31.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.925+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:88 16.4 16:20000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd6e00 con 0x55f9caf06000 2023-09-23T03:26:31.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.926+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.3 v2:172.21.15.154:6808/1133688468 9 ==== osd_op_reply(88 [pgnls start_epoch 110 out=81b] v108'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06000 2023-09-23T03:26:31.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.926+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:89 16.4 16:24ae92ec:::100000001fb.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd7180 con 0x55f9caf06000 2023-09-23T03:26:31.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.927+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.3 v2:172.21.15.154:6808/1133688468 10 ==== osd_op_reply(89 100000001fb.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06000 2023-09-23T03:26:31.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.927+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628283.1 2023-09-23T03:26:31.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.927+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:90 15.13 15:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd7500 con 0x55f9caf93000 2023-09-23T03:26:31.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.930+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 15 ==== osd_op_reply(90 100000001fb.00000000 [call,setxattr (8)] v110'19 uv19 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:31.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.930+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:91 16.14 16:28000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd7880 con 0x55f9caf93c00 2023-09-23T03:26:31.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.931+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.0 v2:172.21.15.154:6800/2920938138 17 ==== osd_op_reply(91 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93c00 2023-09-23T03:26:31.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.931+0000 7fe64e7d4640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:92 16.c 16:30000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe0700 con 0x55f9caf06400 2023-09-23T03:26:31.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.932+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.5 v2:172.21.15.190:6800/882536885 3 ==== osd_op_reply(92 [pgnls start_epoch 110 out=81b] v108'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06400 2023-09-23T03:26:31.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.932+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:93 16.c 16:30604137:::100000001fc.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd7c00 con 0x55f9caf06400 2023-09-23T03:26:31.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.933+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.5 v2:172.21.15.190:6800/882536885 4 ==== osd_op_reply(93 100000001fc.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06400 2023-09-23T03:26:31.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.934+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628284.0 2023-09-23T03:26:31.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.934+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:94 15.c 15:30604137:::100000001fc.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd6380 con 0x55f9caf06400 2023-09-23T03:26:31.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.938+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.5 v2:172.21.15.190:6800/882536885 5 ==== osd_op_reply(94 100000001fc.00000000 [call,setxattr (8)] v110'7 uv7 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06400 2023-09-23T03:26:31.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.938+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:95 16.1c 16:38000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1fc00 con 0x55f9cae2d400 2023-09-23T03:26:31.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.939+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.2 v2:172.21.15.154:6822/2884407243 14 ==== osd_op_reply(95 [pgnls start_epoch 110 out=101b] v108'3 uv3 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9cae2d400 2023-09-23T03:26:31.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.939+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:96 16.1c 16:398da958:::100000001f7.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1f180 con 0x55f9cae2d400 2023-09-23T03:26:31.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.940+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.2 v2:172.21.15.154:6822/2884407243 15 ==== osd_op_reply(96 100000001f7.00000002 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9cae2d400 2023-09-23T03:26:31.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.940+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628279.2 2023-09-23T03:26:31.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.940+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:97 15.36 15:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1ee00 con 0x55f9caf06800 2023-09-23T03:26:31.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.944+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.6 v2:172.21.15.190:6816/1964294001 16 ==== osd_op_reply(97 100000001f7.00000000 [call,setxattr (8)] v110'21 uv21 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf06800 2023-09-23T03:26:31.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.944+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:98 16.1c 16:3c0114ac::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cab99880 con 0x55f9cae2d400 2023-09-23T03:26:31.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.945+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.2 v2:172.21.15.154:6822/2884407243 16 ==== osd_op_reply(98 [pgnls start_epoch 110 out=101b] v108'3 uv3 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9cae2d400 2023-09-23T03:26:31.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.945+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:99 16.1c 16:3c0114ac:::100000001fa.00000004:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cab99c00 con 0x55f9cae2d400 2023-09-23T03:26:31.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.947+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.2 v2:172.21.15.154:6822/2884407243 17 ==== osd_op_reply(99 100000001fa.00000004 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9cae2d400 2023-09-23T03:26:31.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.947+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628282.4 2023-09-23T03:26:31.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.947+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:100 15.3f 15:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe0a80 con 0x55f9caf93000 2023-09-23T03:26:31.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.950+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 16 ==== osd_op_reply(100 100000001fa.00000000 [call,setxattr (8)] v110'11 uv11 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:31.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.950+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:101 16.1c 16:3d6db04b::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe1500 con 0x55f9cae2d400 2023-09-23T03:26:31.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.951+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.2 v2:172.21.15.154:6822/2884407243 18 ==== osd_op_reply(101 [pgnls start_epoch 110 out=81b] v108'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9cae2d400 2023-09-23T03:26:31.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.951+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:102 16.1c 16:3d6db04b:::100000001f6.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe1880 con 0x55f9cae2d400 2023-09-23T03:26:31.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.952+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.2 v2:172.21.15.154:6822/2884407243 19 ==== osd_op_reply(102 100000001f6.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9cae2d400 2023-09-23T03:26:31.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.952+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628278.1 2023-09-23T03:26:31.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.953+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:103 15.2a 15:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe1180 con 0x55f9caf93c00 2023-09-23T03:26:31.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.956+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.0 v2:172.21.15.154:6800/2920938138 18 ==== osd_op_reply(103 100000001f6.00000000 [call,setxattr (8)] v110'12 uv12 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93c00 2023-09-23T03:26:31.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.956+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:104 16.2 16:40000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd6000 con 0x55f9caf93400 2023-09-23T03:26:31.958 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.957+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.4 v2:172.21.15.190:6808/717302641 11 ==== osd_op_reply(104 [pgnls start_epoch 110 out=101b] v108'2 uv2 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93400 2023-09-23T03:26:31.958 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.957+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:105 16.2 16:40d56c48:::100000001fc.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd6700 con 0x55f9caf93400 2023-09-23T03:26:31.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.959+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.4 v2:172.21.15.190:6808/717302641 12 ==== osd_op_reply(105 100000001fc.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93400 2023-09-23T03:26:31.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.959+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628284.1 2023-09-23T03:26:31.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.959+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:106 15.c 15:30604137:::100000001fc.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd7880 con 0x55f9caf06400 2023-09-23T03:26:31.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.963+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.5 v2:172.21.15.190:6800/882536885 6 ==== osd_op_reply(106 100000001fc.00000000 [call,setxattr (8)] v110'8 uv8 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06400 2023-09-23T03:26:31.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.963+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:107 16.2 16:46f89f8a::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe0e00 con 0x55f9caf93400 2023-09-23T03:26:31.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.964+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.4 v2:172.21.15.190:6808/717302641 13 ==== osd_op_reply(107 [pgnls start_epoch 110 out=81b] v108'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93400 2023-09-23T03:26:31.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.964+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:108 16.2 16:46f89f8a:::100000001f9.00000005:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe1500 con 0x55f9caf93400 2023-09-23T03:26:31.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.965+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.4 v2:172.21.15.190:6808/717302641 14 ==== osd_op_reply(108 100000001f9.00000005 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93400 2023-09-23T03:26:31.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.965+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628281.5 2023-09-23T03:26:31.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.965+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:109 15.36 15:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe1880 con 0x55f9caf06800 2023-09-23T03:26:31.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.969+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.6 v2:172.21.15.190:6816/1964294001 17 ==== osd_op_reply(109 100000001f9.00000000 [call,setxattr (8)] v110'22 uv22 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf06800 2023-09-23T03:26:31.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.969+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:110 16.12 16:48000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cab99880 con 0x55f9caf06800 2023-09-23T03:26:31.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.970+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.6 v2:172.21.15.190:6816/1964294001 18 ==== osd_op_reply(110 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf06800 2023-09-23T03:26:31.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.970+0000 7fe64efd5640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:111 16.a 16:50000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe0000 con 0x55f9cae2d400 2023-09-23T03:26:31.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.971+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.2 v2:172.21.15.154:6822/2884407243 20 ==== osd_op_reply(111 [pgnls start_epoch 110 out=101b] v108'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9cae2d400 2023-09-23T03:26:31.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.971+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:112 16.a 16:544c5ef0:::100000001fc.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cab99c00 con 0x55f9cae2d400 2023-09-23T03:26:31.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.972+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.2 v2:172.21.15.154:6822/2884407243 21 ==== osd_op_reply(112 100000001fc.00000003 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9cae2d400 2023-09-23T03:26:31.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.972+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628284.3 2023-09-23T03:26:31.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.972+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:113 15.c 15:30604137:::100000001fc.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1fc00 con 0x55f9caf06400 2023-09-23T03:26:31.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.977+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.5 v2:172.21.15.190:6800/882536885 7 ==== osd_op_reply(113 100000001fc.00000000 [call,setxattr (8)] v110'9 uv9 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06400 2023-09-23T03:26:31.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.977+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:114 16.a 16:56c65d28::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1f180 con 0x55f9cae2d400 2023-09-23T03:26:31.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.977+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.2 v2:172.21.15.154:6822/2884407243 22 ==== osd_op_reply(114 [pgnls start_epoch 110 out=101b] v108'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9cae2d400 2023-09-23T03:26:31.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.978+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:115 16.a 16:56c65d28:::100000001fa.00000006:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1ee00 con 0x55f9cae2d400 2023-09-23T03:26:31.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.979+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.2 v2:172.21.15.154:6822/2884407243 23 ==== osd_op_reply(115 100000001fa.00000006 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9cae2d400 2023-09-23T03:26:31.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.979+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628282.6 2023-09-23T03:26:31.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.979+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:116 15.3f 15:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd7180 con 0x55f9caf93000 2023-09-23T03:26:31.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.982+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 17 ==== osd_op_reply(116 100000001fa.00000000 [call,setxattr (8)] v110'12 uv12 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:31.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.982+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:117 16.a 16:57fe7535::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd7500 con 0x55f9cae2d400 2023-09-23T03:26:31.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.983+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.2 v2:172.21.15.154:6822/2884407243 24 ==== osd_op_reply(117 [pgnls start_epoch 110 out=81b] v108'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9cae2d400 2023-09-23T03:26:31.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.983+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:118 16.a 16:57fe7535:::100000001f6.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd7c00 con 0x55f9cae2d400 2023-09-23T03:26:31.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.985+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.2 v2:172.21.15.154:6822/2884407243 25 ==== osd_op_reply(118 100000001f6.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9cae2d400 2023-09-23T03:26:31.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.985+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628278.0 2023-09-23T03:26:31.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.985+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:119 15.2a 15:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd6380 con 0x55f9caf93c00 2023-09-23T03:26:31.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.988+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.0 v2:172.21.15.154:6800/2920938138 19 ==== osd_op_reply(119 100000001f6.00000000 [call,setxattr (8)] v110'13 uv13 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93c00 2023-09-23T03:26:31.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.988+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:120 16.1a 16:58000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd7880 con 0x55f9caf06000 2023-09-23T03:26:31.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.989+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.3 v2:172.21.15.154:6808/1133688468 11 ==== osd_op_reply(120 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06000 2023-09-23T03:26:31.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.989+0000 7fe64efd5640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:121 16.6 16:60000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe0380 con 0x55f9caf06800 2023-09-23T03:26:31.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.991+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.6 v2:172.21.15.190:6816/1964294001 19 ==== osd_op_reply(121 [pgnls start_epoch 110 out=81b] v108'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf06800 2023-09-23T03:26:31.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.991+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:122 16.6 16:613bf181:::100000001f7.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd6a80 con 0x55f9caf06800 2023-09-23T03:26:31.992 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.992+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.6 v2:172.21.15.190:6816/1964294001 20 ==== osd_op_reply(122 100000001f7.00000003 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf06800 2023-09-23T03:26:31.992 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.992+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628279.3 2023-09-23T03:26:31.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.992+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:123 15.36 15:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd6e00 con 0x55f9caf06800 2023-09-23T03:26:31.997 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.996+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.6 v2:172.21.15.190:6816/1964294001 21 ==== osd_op_reply(123 100000001f7.00000000 [call,setxattr (8)] v110'23 uv23 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf06800 2023-09-23T03:26:31.997 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.996+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:124 16.16 16:68000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1fc00 con 0x55f9caf06000 2023-09-23T03:26:31.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.997+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.3 v2:172.21.15.154:6808/1133688468 12 ==== osd_op_reply(124 [pgnls start_epoch 110 out=101b] v108'5 uv5 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06000 2023-09-23T03:26:31.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.997+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:125 16.16 16:6df493a7:::100000001f9.00000006:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe0700 con 0x55f9caf06000 2023-09-23T03:26:31.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.998+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.3 v2:172.21.15.154:6808/1133688468 13 ==== osd_op_reply(125 100000001f9.00000006 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06000 2023-09-23T03:26:31.999 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.998+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628281.6 2023-09-23T03:26:31.999 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:31.998+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:126 15.36 15:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe0a80 con 0x55f9caf06800 2023-09-23T03:26:32.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.002+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.6 v2:172.21.15.190:6816/1964294001 22 ==== osd_op_reply(126 100000001f9.00000000 [call,setxattr (8)] v110'24 uv24 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf06800 2023-09-23T03:26:32.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.002+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:127 16.16 16:6ee5e2c4::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd7500 con 0x55f9caf06000 2023-09-23T03:26:32.003 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.003+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.3 v2:172.21.15.154:6808/1133688468 14 ==== osd_op_reply(127 [pgnls start_epoch 110 out=101b] v108'5 uv5 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06000 2023-09-23T03:26:32.003 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.003+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:128 16.16 16:6ee5e2c4:::100000001f9.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd7c00 con 0x55f9caf06000 2023-09-23T03:26:32.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.004+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.3 v2:172.21.15.154:6808/1133688468 15 ==== osd_op_reply(128 100000001f9.00000000 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06000 2023-09-23T03:26:32.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.004+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628281.0 2023-09-23T03:26:32.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.004+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:129 15.36 15:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd6a80 con 0x55f9caf06800 2023-09-23T03:26:32.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.008+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.6 v2:172.21.15.190:6816/1964294001 23 ==== osd_op_reply(129 100000001f9.00000000 [call,setxattr (8)] v110'25 uv25 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf06800 2023-09-23T03:26:32.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.008+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:130 16.16 16:6fe38dc2::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe1880 con 0x55f9caf06000 2023-09-23T03:26:32.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.009+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.3 v2:172.21.15.154:6808/1133688468 16 ==== osd_op_reply(130 [pgnls start_epoch 110 out=81b] v108'5 uv5 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06000 2023-09-23T03:26:32.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.009+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:131 16.16 16:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe0000 con 0x55f9caf06000 2023-09-23T03:26:32.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.010+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.3 v2:172.21.15.154:6808/1133688468 17 ==== osd_op_reply(131 100000001f7.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06000 2023-09-23T03:26:32.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.010+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628279.0 2023-09-23T03:26:32.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.010+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:132 15.36 15:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe0380 con 0x55f9caf06800 2023-09-23T03:26:32.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.015+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.6 v2:172.21.15.190:6816/1964294001 24 ==== osd_op_reply(132 100000001f7.00000000 [call,setxattr (8)] v110'26 uv26 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf06800 2023-09-23T03:26:32.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.015+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:133 16.e 16:70000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1f180 con 0x55f9cae2d400 2023-09-23T03:26:32.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.016+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.2 v2:172.21.15.154:6822/2884407243 26 ==== osd_op_reply(133 [pgnls start_epoch 110 out=101b] v108'3 uv3 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9cae2d400 2023-09-23T03:26:32.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.016+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:134 16.e 16:72395991:::10000000001.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1ee00 con 0x55f9cae2d400 2023-09-23T03:26:32.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.017+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.2 v2:172.21.15.154:6822/2884407243 27 ==== osd_op_reply(134 10000000001.00000003 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9cae2d400 2023-09-23T03:26:32.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.017+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511627777.3 2023-09-23T03:26:32.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.017+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:135 15.7 15:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cab99880 con 0x55f9caf93000 2023-09-23T03:26:32.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.020+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 18 ==== osd_op_reply(135 10000000001.00000000 [call,setxattr (8)] v110'8 uv8 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:32.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.021+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:136 16.e 16:728aa123::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cab99c00 con 0x55f9cae2d400 2023-09-23T03:26:32.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.022+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.2 v2:172.21.15.154:6822/2884407243 28 ==== osd_op_reply(136 [pgnls start_epoch 110 out=101b] v108'3 uv3 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9cae2d400 2023-09-23T03:26:32.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.022+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:137 16.e 16:728aa123:::100000001f6.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cab99880 con 0x55f9cae2d400 2023-09-23T03:26:32.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.023+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.2 v2:172.21.15.154:6822/2884407243 29 ==== osd_op_reply(137 100000001f6.00000003 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9cae2d400 2023-09-23T03:26:32.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.023+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628278.3 2023-09-23T03:26:32.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.023+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:138 15.2a 15:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe0700 con 0x55f9caf93c00 2023-09-23T03:26:32.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.027+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.0 v2:172.21.15.154:6800/2920938138 20 ==== osd_op_reply(138 100000001f6.00000000 [call,setxattr (8)] v110'14 uv14 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93c00 2023-09-23T03:26:32.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.027+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:139 16.e 16:7780a862::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe1880 con 0x55f9cae2d400 2023-09-23T03:26:32.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.028+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.2 v2:172.21.15.154:6822/2884407243 30 ==== osd_op_reply(139 [pgnls start_epoch 110 out=81b] v108'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9cae2d400 2023-09-23T03:26:32.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.028+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:140 16.e 16:7780a862:::100000001fa.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe0000 con 0x55f9cae2d400 2023-09-23T03:26:32.029 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.029+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.2 v2:172.21.15.154:6822/2884407243 31 ==== osd_op_reply(140 100000001fa.00000003 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9cae2d400 2023-09-23T03:26:32.029 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.029+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628282.3 2023-09-23T03:26:32.029 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.029+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:141 15.3f 15:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd7180 con 0x55f9caf93000 2023-09-23T03:26:32.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.032+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 19 ==== osd_op_reply(141 100000001fa.00000000 [call,setxattr (8)] v110'13 uv13 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:32.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.032+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:142 16.1e 16:78000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd7880 con 0x55f9caf93800 2023-09-23T03:26:32.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.034+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.7 v2:172.21.15.190:6824/3460136272 10 ==== osd_op_reply(142 [pgnls start_epoch 110 out=101b] v108'2 uv2 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf93800 2023-09-23T03:26:32.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.034+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:143 16.1e 16:7e4d8c81:::10000000001.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd7500 con 0x55f9caf93800 2023-09-23T03:26:32.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.035+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.7 v2:172.21.15.190:6824/3460136272 11 ==== osd_op_reply(143 10000000001.00000002 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf93800 2023-09-23T03:26:32.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.035+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511627777.2 2023-09-23T03:26:32.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.035+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:144 15.7 15:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd7c00 con 0x55f9caf93000 2023-09-23T03:26:32.039 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.038+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 20 ==== osd_op_reply(144 10000000001.00000000 [call,setxattr (8)] v110'9 uv9 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:32.039 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.038+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:145 16.1e 16:7e8133fb::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1fc00 con 0x55f9caf93800 2023-09-23T03:26:32.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.039+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.7 v2:172.21.15.190:6824/3460136272 12 ==== osd_op_reply(145 [pgnls start_epoch 110 out=81b] v108'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf93800 2023-09-23T03:26:32.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.040+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:146 16.1e 16:7e8133fb:::100000001fb.00000006:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd6e00 con 0x55f9caf93800 2023-09-23T03:26:32.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.040+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.7 v2:172.21.15.190:6824/3460136272 13 ==== osd_op_reply(146 100000001fb.00000006 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf93800 2023-09-23T03:26:32.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.041+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628283.6 2023-09-23T03:26:32.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.041+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:147 15.13 15:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd6a80 con 0x55f9caf93000 2023-09-23T03:26:32.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.044+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 21 ==== osd_op_reply(147 100000001fb.00000000 [call,setxattr (8)] v110'20 uv20 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:32.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.044+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:148 16.1 16:80000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd7880 con 0x55f9cae2d400 2023-09-23T03:26:32.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.045+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.2 v2:172.21.15.154:6822/2884407243 32 ==== osd_op_reply(148 [pgnls start_epoch 110 out=81b] v108'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9cae2d400 2023-09-23T03:26:32.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.045+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:149 16.1 16:848ebb0b:::100000001f7.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe0a80 con 0x55f9cae2d400 2023-09-23T03:26:32.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.046+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.2 v2:172.21.15.154:6822/2884407243 33 ==== osd_op_reply(149 100000001f7.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9cae2d400 2023-09-23T03:26:32.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.046+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628279.1 2023-09-23T03:26:32.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.046+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:150 15.36 15:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe0380 con 0x55f9caf06800 2023-09-23T03:26:32.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.050+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.6 v2:172.21.15.190:6816/1964294001 25 ==== osd_op_reply(150 100000001f7.00000000 [call,setxattr (8)] v110'27 uv27 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf06800 2023-09-23T03:26:32.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.050+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:151 16.11 16:88000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe1880 con 0x55f9caf06400 2023-09-23T03:26:32.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.052+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.5 v2:172.21.15.190:6800/882536885 8 ==== osd_op_reply(151 [pgnls start_epoch 110 out=81b] v108'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06400 2023-09-23T03:26:32.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.052+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:152 16.11 16:893be98d:::100000001f8.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe0000 con 0x55f9caf06400 2023-09-23T03:26:32.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.053+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.5 v2:172.21.15.190:6800/882536885 9 ==== osd_op_reply(152 100000001f8.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06400 2023-09-23T03:26:32.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.053+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628280.1 2023-09-23T03:26:32.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.053+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:153 15.19 15:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cab99c00 con 0x55f9caf93400 2023-09-23T03:26:32.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.056+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.4 v2:172.21.15.190:6808/717302641 15 ==== osd_op_reply(153 100000001f8.00000000 [call,setxattr (8)] v110'13 uv13 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93400 2023-09-23T03:26:32.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.056+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:154 16.9 16:90000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cab99880 con 0x55f9caf93000 2023-09-23T03:26:32.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.058+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 22 ==== osd_op_reply(154 [pgnls start_epoch 110 out=81b] v108'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:32.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.058+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:155 16.9 16:94f3e7b4:::100000001fa.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1f180 con 0x55f9caf93000 2023-09-23T03:26:32.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.059+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 23 ==== osd_op_reply(155 100000001fa.00000002 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:32.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.059+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628282.2 2023-09-23T03:26:32.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.059+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:156 15.3f 15:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1ee00 con 0x55f9caf93000 2023-09-23T03:26:32.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.062+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 24 ==== osd_op_reply(156 100000001fa.00000000 [call,setxattr (8)] v110'14 uv14 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:32.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.063+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:157 16.19 16:98000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cab99c00 con 0x55f9caf93400 2023-09-23T03:26:32.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.064+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.4 v2:172.21.15.190:6808/717302641 16 ==== osd_op_reply(157 [pgnls start_epoch 110 out=81b] v108'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93400 2023-09-23T03:26:32.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.064+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:158 16.19 16:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe1180 con 0x55f9caf93400 2023-09-23T03:26:32.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.065+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.4 v2:172.21.15.190:6808/717302641 17 ==== osd_op_reply(158 100000001f8.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93400 2023-09-23T03:26:32.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.065+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628280.0 2023-09-23T03:26:32.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.065+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:159 15.19 15:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe0e00 con 0x55f9caf93400 2023-09-23T03:26:32.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.069+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.4 v2:172.21.15.190:6808/717302641 18 ==== osd_op_reply(159 100000001f8.00000000 [call,setxattr (8)] v110'14 uv14 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93400 2023-09-23T03:26:32.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.069+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:160 16.5 16:a0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe1500 con 0x55f9caf93c00 2023-09-23T03:26:32.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.070+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.0 v2:172.21.15.154:6800/2920938138 21 ==== osd_op_reply(160 [pgnls start_epoch 110 out=81b] v108'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93c00 2023-09-23T03:26:32.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.070+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:161 16.5 16:a238cdfc:::100000001f9.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe0700 con 0x55f9caf93c00 2023-09-23T03:26:32.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.071+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.0 v2:172.21.15.154:6800/2920938138 22 ==== osd_op_reply(161 100000001f9.00000002 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93c00 2023-09-23T03:26:32.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.071+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628281.2 2023-09-23T03:26:32.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.071+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:162 15.36 15:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd6000 con 0x55f9caf06800 2023-09-23T03:26:32.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.076+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.6 v2:172.21.15.190:6816/1964294001 26 ==== osd_op_reply(162 100000001f9.00000000 [call,setxattr (8)] v110'28 uv28 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf06800 2023-09-23T03:26:32.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.076+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:163 16.15 16:a8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd6700 con 0x55f9caf93c00 2023-09-23T03:26:32.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.077+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.0 v2:172.21.15.154:6800/2920938138 23 ==== osd_op_reply(163 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93c00 2023-09-23T03:26:32.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.077+0000 7fe64e7d4640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:164 16.d 16:b0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1f500 con 0x55f9caf06800 2023-09-23T03:26:32.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.078+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.6 v2:172.21.15.190:6816/1964294001 27 ==== osd_op_reply(164 [pgnls start_epoch 110 out=81b] v108'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf06800 2023-09-23T03:26:32.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.078+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:165 16.d 16:b0a6c804:::100000001fc.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd6380 con 0x55f9caf06800 2023-09-23T03:26:32.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.079+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.6 v2:172.21.15.190:6816/1964294001 28 ==== osd_op_reply(165 100000001fc.00000002 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf06800 2023-09-23T03:26:32.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.079+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628284.2 2023-09-23T03:26:32.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.079+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:166 15.c 15:30604137:::100000001fc.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cb010000 con 0x55f9caf06400 2023-09-23T03:26:32.084 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.083+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.5 v2:172.21.15.190:6800/882536885 10 ==== osd_op_reply(166 100000001fc.00000000 [call,setxattr (8)] v110'10 uv10 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06400 2023-09-23T03:26:32.084 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.084+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:167 16.1d 16:b8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cb010380 con 0x55f9cae2d400 2023-09-23T03:26:32.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.084+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.2 v2:172.21.15.154:6822/2884407243 34 ==== osd_op_reply(167 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9cae2d400 2023-09-23T03:26:32.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.085+0000 7fe64e7d4640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:168 16.3 16:c0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1ea80 con 0x55f9caf06400 2023-09-23T03:26:32.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.086+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.5 v2:172.21.15.190:6800/882536885 11 ==== osd_op_reply(168 [pgnls start_epoch 110 out=101b] v108'2 uv2 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06400 2023-09-23T03:26:32.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.086+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:169 16.3 16:c0957981:::100000001fa.00000005:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cb010700 con 0x55f9caf06400 2023-09-23T03:26:32.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.087+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.5 v2:172.21.15.190:6800/882536885 12 ==== osd_op_reply(169 100000001fa.00000005 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06400 2023-09-23T03:26:32.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.087+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628282.5 2023-09-23T03:26:32.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.087+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:170 15.3f 15:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cb010a80 con 0x55f9caf93000 2023-09-23T03:26:32.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.091+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 25 ==== osd_op_reply(170 100000001fa.00000000 [call,setxattr (8)] v110'15 uv15 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:32.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.091+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:171 16.3 16:c5d8c51d::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cb010e00 con 0x55f9caf06400 2023-09-23T03:26:32.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.093+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.5 v2:172.21.15.190:6800/882536885 13 ==== osd_op_reply(171 [pgnls start_epoch 110 out=81b] v108'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06400 2023-09-23T03:26:32.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.093+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:172 16.3 16:c5d8c51d:::100000001fb.00000004:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cb011180 con 0x55f9caf06400 2023-09-23T03:26:32.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.094+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.5 v2:172.21.15.190:6800/882536885 14 ==== osd_op_reply(172 100000001fb.00000004 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06400 2023-09-23T03:26:32.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.094+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628283.4 2023-09-23T03:26:32.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.094+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:173 15.13 15:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cb011500 con 0x55f9caf93000 2023-09-23T03:26:32.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.098+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 26 ==== osd_op_reply(173 100000001fb.00000000 [call,setxattr (8)] v110'21 uv21 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:32.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.098+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:174 16.13 16:c8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cb011880 con 0x55f9caf93000 2023-09-23T03:26:32.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.099+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 27 ==== osd_op_reply(174 [pgnls start_epoch 110 out=101b] v108'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:32.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.099+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:175 16.13 16:c9e7fe79:::100000001f9.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cb011c00 con 0x55f9caf93000 2023-09-23T03:26:32.101 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.100+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 28 ==== osd_op_reply(175 100000001f9.00000003 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:32.101 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.100+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628281.3 2023-09-23T03:26:32.101 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.100+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:176 15.36 15:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1f180 con 0x55f9caf06800 2023-09-23T03:26:32.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.105+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.6 v2:172.21.15.190:6816/1964294001 29 ==== osd_op_reply(176 100000001f9.00000000 [call,setxattr (8)] v110'29 uv29 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf06800 2023-09-23T03:26:32.106 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.105+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:177 16.13 16:ca8fdbba::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1ee00 con 0x55f9caf93000 2023-09-23T03:26:32.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.106+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 29 ==== osd_op_reply(177 [pgnls start_epoch 110 out=101b] v108'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:32.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.106+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:178 16.13 16:ca8fdbba:::100000001fb.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1ea80 con 0x55f9caf93000 2023-09-23T03:26:32.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.107+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 30 ==== osd_op_reply(178 100000001fb.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:32.108 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.107+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628283.0 2023-09-23T03:26:32.108 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.107+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:179 15.13 15:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cb010000 con 0x55f9caf93000 2023-09-23T03:26:32.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.110+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 31 ==== osd_op_reply(179 100000001fb.00000000 [call,setxattr (8)] v110'22 uv22 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:32.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.111+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:180 16.13 16:cb24bee2::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cab99880 con 0x55f9caf93000 2023-09-23T03:26:32.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.111+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 32 ==== osd_op_reply(180 [pgnls start_epoch 110 out=81b] v108'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:32.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.111+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:181 16.13 16:cb24bee2:::100000001fa.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe1880 con 0x55f9caf93000 2023-09-23T03:26:32.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.112+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 33 ==== osd_op_reply(181 100000001fa.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:32.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.112+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628282.1 2023-09-23T03:26:32.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.112+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:182 15.3f 15:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe0000 con 0x55f9caf93000 2023-09-23T03:26:32.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.116+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 34 ==== osd_op_reply(182 100000001fa.00000000 [call,setxattr (8)] v110'16 uv16 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:32.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.117+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:183 16.b 16:d0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd7180 con 0x55f9caf93000 2023-09-23T03:26:32.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.117+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 35 ==== osd_op_reply(183 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:32.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.117+0000 7fe64e7d4640 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:184 16.1b 16:d8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cb010700 con 0x55f9caf06400 2023-09-23T03:26:32.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.119+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.5 v2:172.21.15.190:6800/882536885 15 ==== osd_op_reply(184 [pgnls start_epoch 110 out=81b] v108'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06400 2023-09-23T03:26:32.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.119+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:185 16.1b 16:de1cf3c0:::100000001fb.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd7c00 con 0x55f9caf06400 2023-09-23T03:26:32.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.120+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.5 v2:172.21.15.190:6800/882536885 16 ==== osd_op_reply(185 100000001fb.00000003 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06400 2023-09-23T03:26:32.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.120+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628283.3 2023-09-23T03:26:32.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.120+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:186 15.13 15:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd6a80 con 0x55f9caf93000 2023-09-23T03:26:32.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.124+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 36 ==== osd_op_reply(186 100000001fb.00000000 [call,setxattr (8)] v110'23 uv23 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:32.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.124+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:187 16.7 16:e0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cab99880 con 0x55f9caf06000 2023-09-23T03:26:32.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.125+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.3 v2:172.21.15.154:6808/1133688468 18 ==== osd_op_reply(187 [pgnls start_epoch 110 out=101b] v108'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06000 2023-09-23T03:26:32.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.125+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:188 16.7 16:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cb01e000 con 0x55f9caf06000 2023-09-23T03:26:32.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.126+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.3 v2:172.21.15.154:6808/1133688468 19 ==== osd_op_reply(188 10000000001.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06000 2023-09-23T03:26:32.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.126+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511627777.0 2023-09-23T03:26:32.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.126+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:189 15.7 15:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cb01e380 con 0x55f9caf93000 2023-09-23T03:26:32.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.130+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 37 ==== osd_op_reply(189 10000000001.00000000 [call,setxattr (8)] v110'10 uv10 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:32.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.130+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:190 16.7 16:e7ecf421::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cb01e700 con 0x55f9caf06000 2023-09-23T03:26:32.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.131+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.3 v2:172.21.15.154:6808/1133688468 20 ==== osd_op_reply(190 [pgnls start_epoch 110 out=81b] v108'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06000 2023-09-23T03:26:32.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.131+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:191 16.7 16:e7ecf421:::100000001f9.00000004:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cb01ea80 con 0x55f9caf06000 2023-09-23T03:26:32.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.131+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.3 v2:172.21.15.154:6808/1133688468 21 ==== osd_op_reply(191 100000001f9.00000004 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06000 2023-09-23T03:26:32.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.131+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628281.4 2023-09-23T03:26:32.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.131+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:192 15.36 15:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cb01ee00 con 0x55f9caf06800 2023-09-23T03:26:32.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.136+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.6 v2:172.21.15.190:6816/1964294001 30 ==== osd_op_reply(192 100000001f9.00000000 [call,setxattr (8)] v110'30 uv30 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf06800 2023-09-23T03:26:32.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.136+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:193 16.17 16:e8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cb01f180 con 0x55f9cae2d400 2023-09-23T03:26:32.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.137+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.2 v2:172.21.15.154:6822/2884407243 35 ==== osd_op_reply(193 [pgnls start_epoch 110 out=101b] v108'2 uv2 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9cae2d400 2023-09-23T03:26:32.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.137+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:194 16.17 16:ee7e3069:::100000001f9.00000007:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cb01f500 con 0x55f9cae2d400 2023-09-23T03:26:32.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.138+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.2 v2:172.21.15.154:6822/2884407243 36 ==== osd_op_reply(194 100000001f9.00000007 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9cae2d400 2023-09-23T03:26:32.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.138+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628281.7 2023-09-23T03:26:32.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.138+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:195 15.36 15:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cb01f880 con 0x55f9caf06800 2023-09-23T03:26:32.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.142+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.6 v2:172.21.15.190:6816/1964294001 31 ==== osd_op_reply(195 100000001f9.00000000 [call,setxattr (8)] v110'31 uv31 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9caf06800 2023-09-23T03:26:32.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.142+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:196 16.17 16:eff9002a::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cb01fc00 con 0x55f9cae2d400 2023-09-23T03:26:32.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.143+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.2 v2:172.21.15.154:6822/2884407243 37 ==== osd_op_reply(196 [pgnls start_epoch 110 out=81b] v108'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9cae2d400 2023-09-23T03:26:32.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.143+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:197 16.17 16:eff9002a:::100000001f8.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cb01ee00 con 0x55f9cae2d400 2023-09-23T03:26:32.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.144+0000 7fe652fdd640 1 -- 172.21.15.154:0/524486974 <== osd.2 v2:172.21.15.154:6822/2884407243 38 ==== osd_op_reply(197 100000001f8.00000003 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2b40 con 0x55f9cae2d400 2023-09-23T03:26:32.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.144+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628280.3 2023-09-23T03:26:32.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.144+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:198 15.19 15:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1fc00 con 0x55f9caf93400 2023-09-23T03:26:32.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.147+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.4 v2:172.21.15.190:6808/717302641 19 ==== osd_op_reply(198 100000001f8.00000000 [call,setxattr (8)] v110'15 uv15 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93400 2023-09-23T03:26:32.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.148+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:199 16.f 16:f0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1f500 con 0x55f9caf93c00 2023-09-23T03:26:32.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.148+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.0 v2:172.21.15.154:6800/2920938138 24 ==== osd_op_reply(199 [pgnls start_epoch 110 out=101b] v108'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93c00 2023-09-23T03:26:32.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.149+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:200 16.f 16:f26fdfa9:::100000001f8.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1f180 con 0x55f9caf93c00 2023-09-23T03:26:32.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.149+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.0 v2:172.21.15.154:6800/2920938138 25 ==== osd_op_reply(200 100000001f8.00000002 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93c00 2023-09-23T03:26:32.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.149+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628280.2 2023-09-23T03:26:32.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.150+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:201 15.19 15:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cb010380 con 0x55f9caf93400 2023-09-23T03:26:32.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.153+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.4 v2:172.21.15.190:6808/717302641 20 ==== osd_op_reply(201 100000001f8.00000000 [call,setxattr (8)] v110'16 uv16 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93400 2023-09-23T03:26:32.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.153+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:202 16.f 16:f376f0dc::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd7500 con 0x55f9caf93c00 2023-09-23T03:26:32.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.154+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.0 v2:172.21.15.154:6800/2920938138 26 ==== osd_op_reply(202 [pgnls start_epoch 110 out=101b] v108'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93c00 2023-09-23T03:26:32.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.154+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:203 16.f 16:f376f0dc:::100000001fb.00000005:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd6e00 con 0x55f9caf93c00 2023-09-23T03:26:32.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.155+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.0 v2:172.21.15.154:6800/2920938138 27 ==== osd_op_reply(203 100000001fb.00000005 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93c00 2023-09-23T03:26:32.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.155+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628283.5 2023-09-23T03:26:32.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.155+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:204 15.13 15:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd7880 con 0x55f9caf93000 2023-09-23T03:26:32.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.159+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 38 ==== osd_op_reply(204 100000001fb.00000000 [call,setxattr (8)] v110'24 uv24 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:32.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.159+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:205 16.f 16:f526278d::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd6000 con 0x55f9caf93c00 2023-09-23T03:26:32.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.160+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.0 v2:172.21.15.154:6800/2920938138 28 ==== osd_op_reply(205 [pgnls start_epoch 110 out=101b] v108'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93c00 2023-09-23T03:26:32.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.160+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:206 16.f 16:f526278d:::10000000001.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd6380 con 0x55f9caf93c00 2023-09-23T03:26:32.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.161+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.0 v2:172.21.15.154:6800/2920938138 29 ==== osd_op_reply(206 10000000001.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93c00 2023-09-23T03:26:32.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.161+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511627777.1 2023-09-23T03:26:32.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.161+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:207 15.7 15:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe0a80 con 0x55f9caf93000 2023-09-23T03:26:32.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.164+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 39 ==== osd_op_reply(207 10000000001.00000000 [call,setxattr (8)] v110'11 uv11 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:32.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.164+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:208 16.f 16:f776b802::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe0380 con 0x55f9caf93c00 2023-09-23T03:26:32.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.165+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.0 v2:172.21.15.154:6800/2920938138 30 ==== osd_op_reply(208 [pgnls start_epoch 110 out=81b] v108'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93c00 2023-09-23T03:26:32.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.165+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:209 16.f 16:f776b802:::100000001fb.00000007:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1fc00 con 0x55f9caf93c00 2023-09-23T03:26:32.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.166+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.0 v2:172.21.15.154:6800/2920938138 31 ==== osd_op_reply(209 100000001fb.00000007 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93c00 2023-09-23T03:26:32.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.166+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628283.7 2023-09-23T03:26:32.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.166+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:210 15.13 15:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1f500 con 0x55f9caf93000 2023-09-23T03:26:32.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.169+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 40 ==== osd_op_reply(210 100000001fb.00000000 [call,setxattr (8)] v110'25 uv25 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:32.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.169+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:211 16.1f 16:f8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9caf1f180 con 0x55f9caf06400 2023-09-23T03:26:32.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.170+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.5 v2:172.21.15.190:6800/882536885 17 ==== osd_op_reply(211 [pgnls start_epoch 110 out=101b] v108'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06400 2023-09-23T03:26:32.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.171+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:212 16.1f 16:faedc8eb:::100000001f6.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd7180 con 0x55f9caf06400 2023-09-23T03:26:32.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.171+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.5 v2:172.21.15.190:6800/882536885 18 ==== osd_op_reply(212 100000001f6.00000002 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06400 2023-09-23T03:26:32.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.172+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628278.2 2023-09-23T03:26:32.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.172+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:213 15.2a 15:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd7c00 con 0x55f9caf93c00 2023-09-23T03:26:32.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.175+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 <== osd.0 v2:172.21.15.154:6800/2920938138 32 ==== osd_op_reply(213 100000001f6.00000000 [call,setxattr (8)] v110'15 uv15 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2fc0 con 0x55f9caf93c00 2023-09-23T03:26:32.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.175+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:214 16.1f 16:fb032386::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd6a80 con 0x55f9caf06400 2023-09-23T03:26:32.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.176+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.5 v2:172.21.15.190:6800/882536885 19 ==== osd_op_reply(214 [pgnls start_epoch 110 out=101b] v108'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06400 2023-09-23T03:26:32.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.176+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:215 16.1f 16:fb032386:::100000001fa.00000007:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd7880 con 0x55f9caf06400 2023-09-23T03:26:32.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.177+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.5 v2:172.21.15.190:6800/882536885 20 ==== osd_op_reply(215 100000001fa.00000007 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06400 2023-09-23T03:26:32.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.177+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628282.7 2023-09-23T03:26:32.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.177+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:216 15.3f 15:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafd6700 con 0x55f9caf93000 2023-09-23T03:26:32.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.182+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 41 ==== osd_op_reply(216 100000001fa.00000000 [call,setxattr (8)] v110'17 uv17 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:32.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.182+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:217 16.1f 16:feb1c803::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe1880 con 0x55f9caf06400 2023-09-23T03:26:32.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.183+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.5 v2:172.21.15.190:6800/882536885 21 ==== osd_op_reply(217 [pgnls start_epoch 110 out=81b] v108'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06400 2023-09-23T03:26:32.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.183+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:218 16.1f 16:feb1c803:::100000001fa.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe0000 con 0x55f9caf06400 2023-09-23T03:26:32.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.184+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.5 v2:172.21.15.190:6800/882536885 22 ==== osd_op_reply(218 100000001fa.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf06400 2023-09-23T03:26:32.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.184+0000 7fe6558495c0 10 datascan.operator(): handling object 1099511628282.0 2023-09-23T03:26:32.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.184+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:219 15.3f 15:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55f9cafe1180 con 0x55f9caf93000 2023-09-23T03:26:32.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.188+0000 7fe653fdf640 1 -- 172.21.15.154:0/524486974 <== osd.1 v2:172.21.15.154:6816/1267710005 42 ==== osd_op_reply(219 100000001fa.00000000 [call,setxattr (8)] v110'18 uv18 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55f9cb9e2d80 con 0x55f9caf93000 2023-09-23T03:26:32.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.188+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55f9caf93c00 msgr2=0x55f9caf99600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.188+0000 7fe6558495c0 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55f9caf93c00 0x55f9caf99600 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.188+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55f9caf93000 msgr2=0x55f9caf98580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.188+0000 7fe6558495c0 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55f9caf93000 0x55f9caf98580 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.188+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55f9cae2d400 msgr2=0x55f9caba9080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.188+0000 7fe6558495c0 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55f9cae2d400 0x55f9caba9080 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.188+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55f9caf06000 msgr2=0x55f9caf99b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.188+0000 7fe6558495c0 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55f9caf06000 0x55f9caf99b80 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.188+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55f9caf93400 msgr2=0x55f9caf98b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.188+0000 7fe6558495c0 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55f9caf93400 0x55f9caf98b00 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.188+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55f9caf06400 msgr2=0x55f9caf9a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.188+0000 7fe6558495c0 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55f9caf06400 0x55f9caf9a100 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.188+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55f9caf06800 msgr2=0x55f9caf9a680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.188+0000 7fe6558495c0 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55f9caf06800 0x55f9caf9a680 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.188+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 reap_dead start 2023-09-23T03:26:32.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.189+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55f9caf93800 msgr2=0x55f9caf99080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.189+0000 7fe6558495c0 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55f9caf93800 0x55f9caf99080 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.189+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 reap_dead start 2023-09-23T03:26:32.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.189+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 reap_dead start 2023-09-23T03:26:32.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.189+0000 7fe6537de640 1 -- 172.21.15.154:0/524486974 reap_dead start 2023-09-23T03:26:32.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.189+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55f9caf92000 msgr2=0x55f9caf98000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.189+0000 7fe6558495c0 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55f9caf92000 0x55f9caf98000 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x55f9cac30930 tx=0x55f9cae4bce0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.189+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f9cae2d800 msgr2=0x55f9cabaac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.189+0000 7fe6558495c0 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f9cae2d800 0x55f9cabaac00 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x55f9cae3f860 tx=0x55f9cae360c0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.189+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 shutdown_connections 2023-09-23T03:26:32.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.189+0000 7fe6558495c0 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55f9caf92000 0x55f9caf98000 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.189+0000 7fe6558495c0 1 --2- 172.21.15.154:0/524486974 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f9cae2d800 0x55f9cabaac00 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.189+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 >> 172.21.15.154:0/524486974 conn(0x55f9cb90bc00 msgr2=0x55f9cb8b9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:32.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.190+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 shutdown_connections 2023-09-23T03:26:32.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.190+0000 7fe6558495c0 1 -- 172.21.15.154:0/524486974 wait complete. 2023-09-23T03:26:32.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.191+0000 7fe6558495c0 1 -- 172.21.15.154:0/3100203787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f9cb90b400 msgr2=0x55f9caba9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.191+0000 7fe6558495c0 1 --2- 172.21.15.154:0/3100203787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f9cb90b400 0x55f9caba9b80 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x55f9cb945500 tx=0x55f9cae36300 comp rx=0 tx=0).stop 2023-09-23T03:26:32.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.191+0000 7fe6558495c0 1 -- 172.21.15.154:0/3100203787 shutdown_connections 2023-09-23T03:26:32.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.191+0000 7fe6558495c0 1 --2- 172.21.15.154:0/3100203787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f9cb90b400 0x55f9caba9b80 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.191+0000 7fe6558495c0 1 --2- 172.21.15.154:0/3100203787 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55f9cb90b000 0x55f9caba8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.191+0000 7fe6558495c0 1 --2- 172.21.15.154:0/3100203787 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55f9cb90a400 0x55f9caba9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.191+0000 7fe6558495c0 1 -- 172.21.15.154:0/3100203787 >> 172.21.15.154:0/3100203787 conn(0x55f9cab3f000 msgr2=0x55f9cb8b8000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:32.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.191+0000 7fe6558495c0 1 -- 172.21.15.154:0/3100203787 shutdown_connections 2023-09-23T03:26:32.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.192+0000 7fe6558495c0 1 -- 172.21.15.154:0/3100203787 wait complete. 2023-09-23T03:26:32.203 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents'] in time 0:00:00.478454, result: 2023-09-23T03:26:32.204 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes 2023-09-23T03:26:32.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.311+0000 7fd3793915c0 1 -- start start 2023-09-23T03:26:32.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.311+0000 7fd3793915c0 1 Processor -- start 2023-09-23T03:26:32.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.312+0000 7fd3793915c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649a534ac00 0x5649a53b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:32.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.312+0000 7fd3793915c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5649a534b800 0x5649a53b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:32.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.312+0000 7fd3793915c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649a534bc00 0x5649a53b5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:32.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.312+0000 7fd3793915c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649a5395180 con 0x5649a534bc00 2023-09-23T03:26:32.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.312+0000 7fd3793915c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649a5394140 con 0x5649a534ac00 2023-09-23T03:26:32.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.312+0000 7fd3793915c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5649a5394000 con 0x5649a534b800 2023-09-23T03:26:32.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.312+0000 7fd37833a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649a534bc00 0x5649a53b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:32.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.312+0000 7fd37833a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649a534bc00 0x5649a53b5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46074/0 (socket says 172.21.15.154:46074) 2023-09-23T03:26:32.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.312+0000 7fd37833a640 1 -- 172.21.15.154:0/980133284 learned_addr learned my addr 172.21.15.154:0/980133284 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:32.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.313+0000 7fd377b39640 1 --2- 172.21.15.154:0/980133284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649a534ac00 0x5649a53b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:32.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.313+0000 7fd377338640 1 --2- 172.21.15.154:0/980133284 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5649a534b800 0x5649a53b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:32.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.313+0000 7fd37833a640 1 -- 172.21.15.154:0/980133284 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5649a534b800 msgr2=0x5649a53b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:32.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.313+0000 7fd37833a640 1 --2- 172.21.15.154:0/980133284 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5649a534b800 0x5649a53b4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.313+0000 7fd37833a640 1 -- 172.21.15.154:0/980133284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649a534ac00 msgr2=0x5649a53b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:32.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.313+0000 7fd37833a640 1 --2- 172.21.15.154:0/980133284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649a534ac00 0x5649a53b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.313+0000 7fd37833a640 1 -- 172.21.15.154:0/980133284 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649a537eb60 con 0x5649a534bc00 2023-09-23T03:26:32.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.313+0000 7fd37833a640 1 --2- 172.21.15.154:0/980133284 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649a534bc00 0x5649a53b5080 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x5649a61533e0 tx=0x5649a5628420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a2fe45633b0b16e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:32.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.313+0000 7fd375334640 1 -- 172.21.15.154:0/980133284 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5649a5393600 con 0x5649a534bc00 2023-09-23T03:26:32.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.313+0000 7fd375334640 1 -- 172.21.15.154:0/980133284 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5649a5371800 con 0x5649a534bc00 2023-09-23T03:26:32.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.313+0000 7fd375334640 1 -- 172.21.15.154:0/980133284 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5649a53931e0 con 0x5649a534bc00 2023-09-23T03:26:32.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.313+0000 7fd3793915c0 1 -- 172.21.15.154:0/980133284 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5649a537e340 con 0x5649a534bc00 2023-09-23T03:26:32.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.313+0000 7fd3793915c0 1 -- 172.21.15.154:0/980133284 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5649a537e000 con 0x5649a534bc00 2023-09-23T03:26:32.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.313+0000 7fd3793915c0 4 waiting for MDS map... 2023-09-23T03:26:32.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.314+0000 7fd375334640 1 -- 172.21.15.154:0/980133284 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(110..110 src has 1..110) v4 ==== 6605+0+0 (secure 0 0 0) 0x5649a540ac40 con 0x5649a534bc00 2023-09-23T03:26:32.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.314+0000 7fd375334640 1 -- 172.21.15.154:0/980133284 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 91) v1 ==== 3306+0+0 (secure 0 0 0) 0x5649a539d200 con 0x5649a534bc00 2023-09-23T03:26:32.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.314+0000 7fd3793915c0 4 Got MDS map 91 2023-09-23T03:26:32.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.314+0000 7fd3793915c0 4 datascan.main: Using metadata pool output 2023-09-23T03:26:32.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.314+0000 7fd3793915c0 4 datascan.main: connecting to RADOS... 2023-09-23T03:26:32.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.315+0000 7fd3793915c0 1 Processor -- start 2023-09-23T03:26:32.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.315+0000 7fd3793915c0 1 -- start start 2023-09-23T03:26:32.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.316+0000 7fd3793915c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649a5639000 0x5649a53b5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:32.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.316+0000 7fd3793915c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649a5639400 0x5649a53b5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:32.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.316+0000 7fd3793915c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5649a5639800 0x5649a53b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:32.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.316+0000 7fd3793915c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649a5394a00 con 0x5649a5639000 2023-09-23T03:26:32.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.316+0000 7fd3793915c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649a5419180 con 0x5649a5639400 2023-09-23T03:26:32.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.316+0000 7fd3793915c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5649a5418dc0 con 0x5649a5639800 2023-09-23T03:26:32.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.316+0000 7fd377338640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649a5639000 0x5649a53b5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:32.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.316+0000 7fd377338640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649a5639000 0x5649a53b5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46084/0 (socket says 172.21.15.154:46084) 2023-09-23T03:26:32.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.316+0000 7fd377338640 1 -- 172.21.15.154:0/633170070 learned_addr learned my addr 172.21.15.154:0/633170070 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:32.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.316+0000 7fd377b39640 1 --2- 172.21.15.154:0/633170070 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649a5639400 0x5649a53b5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:32.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.316+0000 7fd377338640 1 -- 172.21.15.154:0/633170070 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5649a5639800 msgr2=0x5649a53b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:32.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.316+0000 7fd377338640 1 --2- 172.21.15.154:0/633170070 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5649a5639800 0x5649a53b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.316+0000 7fd377338640 1 -- 172.21.15.154:0/633170070 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649a5639400 msgr2=0x5649a53b5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:32.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.316+0000 7fd377338640 1 --2- 172.21.15.154:0/633170070 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649a5639400 0x5649a53b5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.316+0000 7fd377338640 1 -- 172.21.15.154:0/633170070 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649a537eea0 con 0x5649a5639000 2023-09-23T03:26:32.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.317+0000 7fd377338640 1 --2- 172.21.15.154:0/633170070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649a5639000 0x5649a53b5600 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x5649a56446c0 tx=0x5649a5648cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=262ac90a6d3f26fc server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:32.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.317+0000 7fd373330640 1 -- 172.21.15.154:0/633170070 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5649a5393600 con 0x5649a5639000 2023-09-23T03:26:32.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.317+0000 7fd373330640 1 -- 172.21.15.154:0/633170070 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5649a5370a80 con 0x5649a5639000 2023-09-23T03:26:32.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.317+0000 7fd373330640 1 -- 172.21.15.154:0/633170070 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5649a53931e0 con 0x5649a5639000 2023-09-23T03:26:32.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.318+0000 7fd3793915c0 1 -- 172.21.15.154:0/633170070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649a5639000 msgr2=0x5649a53b5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.318+0000 7fd3793915c0 1 --2- 172.21.15.154:0/633170070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649a5639000 0x5649a53b5600 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x5649a56446c0 tx=0x5649a5648cc0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.318+0000 7fd3793915c0 1 -- 172.21.15.154:0/633170070 shutdown_connections 2023-09-23T03:26:32.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.318+0000 7fd3793915c0 1 --2- 172.21.15.154:0/633170070 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5649a5639800 0x5649a53b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.318+0000 7fd3793915c0 1 --2- 172.21.15.154:0/633170070 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649a5639400 0x5649a53b5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.318+0000 7fd3793915c0 1 --2- 172.21.15.154:0/633170070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649a5639000 0x5649a53b5600 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.318+0000 7fd3793915c0 1 -- 172.21.15.154:0/633170070 >> 172.21.15.154:0/633170070 conn(0x5649a5638c00 msgr2=0x5649a60c5000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:32.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.318+0000 7fd3793915c0 1 -- 172.21.15.154:0/633170070 shutdown_connections 2023-09-23T03:26:32.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.318+0000 7fd3793915c0 1 -- 172.21.15.154:0/633170070 wait complete. 2023-09-23T03:26:32.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.318+0000 7fd3793915c0 1 Processor -- start 2023-09-23T03:26:32.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.319+0000 7fd3793915c0 1 -- start start 2023-09-23T03:26:32.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.319+0000 7fd3793915c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5649a5639000 0x5649a53b7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:32.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.319+0000 7fd3793915c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649a5639400 0x5649a53b6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:32.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.319+0000 7fd3793915c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649a5639800 0x5649a53b6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:32.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.319+0000 7fd3793915c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649a5418c80 con 0x5649a5639400 2023-09-23T03:26:32.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.319+0000 7fd3793915c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649a5394140 con 0x5649a5639800 2023-09-23T03:26:32.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.319+0000 7fd3793915c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5649a5394b40 con 0x5649a5639000 2023-09-23T03:26:32.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.320+0000 7fd377b39640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649a5639400 0x5649a53b6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:32.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.320+0000 7fd377b39640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649a5639400 0x5649a53b6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46100/0 (socket says 172.21.15.154:46100) 2023-09-23T03:26:32.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.320+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 learned_addr learned my addr 172.21.15.154:0/361050481 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:32.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.320+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5649a5639000 msgr2=0x5649a53b7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.320+0000 7fd377b39640 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5649a5639000 0x5649a53b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.320+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649a5639800 msgr2=0x5649a53b6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.320+0000 7fd377b39640 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649a5639800 0x5649a53b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.320+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649a537eb60 con 0x5649a5639400 2023-09-23T03:26:32.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.320+0000 7fd377b39640 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649a5639400 0x5649a53b6c00 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x5649a6152a50 tx=0x5649a56281e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:32.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.320+0000 7fd371b2d640 1 -- 172.21.15.154:0/361050481 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5649a5393080 con 0x5649a5639400 2023-09-23T03:26:32.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.321+0000 7fd371b2d640 1 -- 172.21.15.154:0/361050481 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5649a54c4300 con 0x5649a5639400 2023-09-23T03:26:32.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.321+0000 7fd371b2d640 1 -- 172.21.15.154:0/361050481 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5649a5393600 con 0x5649a5639400 2023-09-23T03:26:32.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.321+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5649a537e340 con 0x5649a5639400 2023-09-23T03:26:32.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.321+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5649a537f520 con 0x5649a5639400 2023-09-23T03:26:32.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.321+0000 7fd3793915c0 4 datascan.init: resolving metadata pool 14 2023-09-23T03:26:32.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.322+0000 7fd371b2d640 1 -- 172.21.15.154:0/361050481 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x5649a537d080 con 0x5649a5639400 2023-09-23T03:26:32.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.323+0000 7fd371b2d640 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5649a56a2800 0x5649a53b7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:32.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.323+0000 7fd371b2d640 1 -- 172.21.15.154:0/361050481 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6605+0+0 (secure 0 0 0) 0x5649a540aa80 con 0x5649a5639400 2023-09-23T03:26:32.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.323+0000 7fd3793915c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-09-23T03:26:32.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.323+0000 7fd3793915c0 4 datascan.main: data pool 'cephfs_data' has ID 15 2023-09-23T03:26:32.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.323+0000 7fd3793915c0 4 datascan.main: opening data pool 'cephfs_data' 2023-09-23T03:26:32.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.323+0000 7fd3793915c0 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5649a56a2c00 0x5649a53b6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:32.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.323+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:1 14.14 14:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a53a5880 con 0x5649a56a2c00 2023-09-23T03:26:32.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.323+0000 7fd377338640 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5649a56a2800 0x5649a53b7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:32.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.324+0000 7fd377338640 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5649a56a2c00 0x5649a53b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:32.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.324+0000 7fd377338640 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5649a56a2800 0x5649a53b7700 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x5649a5644810 tx=0x5649a5648c00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:32.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.325+0000 7fd377338640 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5649a56a2c00 0x5649a53b6100 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:32.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.326+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.4 v2:172.21.15.190:6808/717302641 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv50 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x5649a609eb40 con 0x5649a56a2c00 2023-09-23T03:26:32.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.326+0000 7fd3793915c0 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5649a56a3000 0x5649a53b5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:32.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.326+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:2 14.1f 14:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a53a5c00 con 0x5649a56a3000 2023-09-23T03:26:32.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.326+0000 7fd37833a640 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5649a56a3000 0x5649a53b5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:32.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.327+0000 7fd37833a640 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5649a56a3000 0x5649a53b5b80 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:32.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.327+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.0 v2:172.21.15.154:6800/2920938138 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x5649a609ed80 con 0x5649a56a3000 2023-09-23T03:26:32.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.327+0000 7fd3793915c0 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5649a56a3400 0x5649a53b5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:32.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.327+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:3 15.0 15:00000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b6000 con 0x5649a56a3400 2023-09-23T03:26:32.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.328+0000 7fd377b39640 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5649a56a3400 0x5649a53b5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:32.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.328+0000 7fd377b39640 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5649a56a3400 0x5649a53b5600 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:32.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.328+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.2 v2:172.21.15.154:6822/2884407243 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609efc0 con 0x5649a56a3400 2023-09-23T03:26:32.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.329+0000 7fd372b2f640 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5649a57c4000 0x5649a57ca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:32.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.329+0000 7fd372b2f640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:4 15.20 15:04000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b6e00 con 0x5649a57c4000 2023-09-23T03:26:32.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.329+0000 7fd377338640 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5649a57c4000 0x5649a57ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:32.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.329+0000 7fd377338640 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5649a57c4000 0x5649a57ca000 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:32.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.330+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.330+0000 7fd373330640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:5 15.10 15:08000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a53a5880 con 0x5649a56a2c00 2023-09-23T03:26:32.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.331+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.4 v2:172.21.15.190:6808/717302641 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 110 out=49b] v108'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609eb40 con 0x5649a56a2c00 2023-09-23T03:26:32.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.331+0000 7fd372b2f640 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x5649a57c4400 0x5649a57ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:32.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.331+0000 7fd372b2f640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:6 15.30 15:0c000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b6e00 con 0x5649a57c4400 2023-09-23T03:26:32.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.332+0000 7fd37833a640 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x5649a57c4400 0x5649a57ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:32.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.333+0000 7fd37833a640 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x5649a57c4400 0x5649a57ca580 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:32.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.334+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.7 v2:172.21.15.190:6824/3460136272 1 ==== osd_op_reply(6 [pgnls-filter start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609ed80 con 0x5649a57c4400 2023-09-23T03:26:32.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.334+0000 7fd373330640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:7 15.8 15:10000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b7500 con 0x5649a56a3400 2023-09-23T03:26:32.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.334+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.2 v2:172.21.15.154:6822/2884407243 2 ==== osd_op_reply(7 [pgnls-filter start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609efc0 con 0x5649a56a3400 2023-09-23T03:26:32.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.334+0000 7fd372b2f640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:8 15.28 15:14000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b7180 con 0x5649a57c4400 2023-09-23T03:26:32.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.335+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.7 v2:172.21.15.190:6824/3460136272 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 110 out=49b] v108'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609ed80 con 0x5649a57c4400 2023-09-23T03:26:32.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.335+0000 7fd373330640 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5649a57c4c00 0x5649a57cb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:32.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.335+0000 7fd373330640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:9 15.18 15:18000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b7880 con 0x5649a57c4c00 2023-09-23T03:26:32.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.336+0000 7fd377b39640 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5649a57c4c00 0x5649a57cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:32.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.336+0000 7fd377b39640 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5649a57c4c00 0x5649a57cb080 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:32.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.336+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(9 [pgnls-filter start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609efc0 con 0x5649a57c4c00 2023-09-23T03:26:32.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.336+0000 7fd372b2f640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:10 15.38 15:1c000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b6e00 con 0x5649a56a3000 2023-09-23T03:26:32.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.337+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.0 v2:172.21.15.154:6800/2920938138 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609ed80 con 0x5649a56a3000 2023-09-23T03:26:32.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.337+0000 7fd373330640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:11 15.4 15:20000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b6000 con 0x5649a57c4000 2023-09-23T03:26:32.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.337+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 2 ==== osd_op_reply(11 [pgnls-filter start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.337+0000 7fd372b2f640 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x5649a57c4800 0x5649a57cab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:32.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.337+0000 7fd372b2f640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:12 15.24 15:24000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b7180 con 0x5649a57c4800 2023-09-23T03:26:32.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.338+0000 7fd377338640 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x5649a57c4800 0x5649a57cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:32.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.339+0000 7fd377338640 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x5649a57c4800 0x5649a57cab00 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:32.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.340+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== osd_op_reply(12 [pgnls-filter start_epoch 110 out=49b] v108'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4800 2023-09-23T03:26:32.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.340+0000 7fd373330640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:13 15.14 15:28000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b7500 con 0x5649a57c4800 2023-09-23T03:26:32.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.341+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.6 v2:172.21.15.190:6816/1964294001 2 ==== osd_op_reply(13 [pgnls-filter start_epoch 110 out=49b] v108'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4800 2023-09-23T03:26:32.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.341+0000 7fd372b2f640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:14 15.34 15:2c000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a53a5c00 con 0x5649a56a2c00 2023-09-23T03:26:32.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.341+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.4 v2:172.21.15.190:6808/717302641 3 ==== osd_op_reply(14 [pgnls-filter start_epoch 110 out=49b] v108'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609eb40 con 0x5649a56a2c00 2023-09-23T03:26:32.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.342+0000 7fd373330640 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x5649a57c5000 0x5649a57cb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:32.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.342+0000 7fd373330640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:15 15.c 15:30000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b7c00 con 0x5649a57c5000 2023-09-23T03:26:32.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.342+0000 7fd37833a640 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x5649a57c5000 0x5649a57cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:32.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.343+0000 7fd37833a640 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x5649a57c5000 0x5649a57cb600 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:32.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.344+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.5 v2:172.21.15.190:6800/882536885 1 ==== osd_op_reply(15 [pgnls-filter start_epoch 110 out=81b] v110'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5649a609ed80 con 0x5649a57c5000 2023-09-23T03:26:32.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.344+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:16 15.0 15:00000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b6380 con 0x5649a56a3400 2023-09-23T03:26:32.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.345+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.2 v2:172.21.15.154:6822/2884407243 3 ==== osd_op_reply(16 [pgnls-filter start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609efc0 con 0x5649a56a3400 2023-09-23T03:26:32.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.345+0000 7fd373330640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:17 15.20 15:04000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b6000 con 0x5649a57c4000 2023-09-23T03:26:32.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.345+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 3 ==== osd_op_reply(17 [pgnls-filter start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.345+0000 7fd372b2f640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:18 15.10 15:08000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57ea000 con 0x5649a56a2c00 2023-09-23T03:26:32.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.346+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.4 v2:172.21.15.190:6808/717302641 4 ==== osd_op_reply(18 [pgnls-filter start_epoch 110 out=49b] v108'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609eb40 con 0x5649a56a2c00 2023-09-23T03:26:32.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.346+0000 7fd373330640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:19 15.30 15:0c000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b7180 con 0x5649a57c4400 2023-09-23T03:26:32.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.347+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.7 v2:172.21.15.190:6824/3460136272 3 ==== osd_op_reply(19 [pgnls-filter start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609ed80 con 0x5649a57c4400 2023-09-23T03:26:32.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.347+0000 7fd372b2f640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:20 15.8 15:10000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57ea380 con 0x5649a56a3400 2023-09-23T03:26:32.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.348+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.2 v2:172.21.15.154:6822/2884407243 4 ==== osd_op_reply(20 [pgnls-filter start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609efc0 con 0x5649a56a3400 2023-09-23T03:26:32.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.348+0000 7fd373330640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:21 15.28 15:14000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a53a5880 con 0x5649a57c4400 2023-09-23T03:26:32.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.349+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.7 v2:172.21.15.190:6824/3460136272 4 ==== osd_op_reply(21 [pgnls-filter start_epoch 110 out=49b] v108'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609ed80 con 0x5649a57c4400 2023-09-23T03:26:32.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.349+0000 7fd372b2f640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:22 15.18 15:18000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57ea700 con 0x5649a57c4c00 2023-09-23T03:26:32.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.349+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.3 v2:172.21.15.154:6808/1133688468 2 ==== osd_op_reply(22 [pgnls-filter start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609efc0 con 0x5649a57c4c00 2023-09-23T03:26:32.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.349+0000 7fd373330640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:23 15.38 15:1c000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b6e00 con 0x5649a56a3000 2023-09-23T03:26:32.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.350+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.0 v2:172.21.15.154:6800/2920938138 3 ==== osd_op_reply(23 [pgnls-filter start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609ed80 con 0x5649a56a3000 2023-09-23T03:26:32.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.350+0000 7fd372b2f640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:24 15.4 15:20000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57eaa80 con 0x5649a57c4000 2023-09-23T03:26:32.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.351+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.351+0000 7fd373330640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:25 15.24 15:24000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b7c00 con 0x5649a57c4800 2023-09-23T03:26:32.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.352+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.6 v2:172.21.15.190:6816/1964294001 3 ==== osd_op_reply(25 [pgnls-filter start_epoch 110 out=49b] v108'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4800 2023-09-23T03:26:32.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.352+0000 7fd372b2f640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:26 15.14 15:28000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57ea380 con 0x5649a57c4800 2023-09-23T03:26:32.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.353+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.6 v2:172.21.15.190:6816/1964294001 4 ==== osd_op_reply(26 [pgnls-filter start_epoch 110 out=49b] v108'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4800 2023-09-23T03:26:32.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.353+0000 7fd373330640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:27 15.34 15:2c000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b7180 con 0x5649a56a2c00 2023-09-23T03:26:32.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.354+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.4 v2:172.21.15.190:6808/717302641 5 ==== osd_op_reply(27 [pgnls-filter start_epoch 110 out=49b] v108'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609eb40 con 0x5649a56a2c00 2023-09-23T03:26:32.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.354+0000 7fd372b2f640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:28 15.c 15:30000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b7880 con 0x5649a57c5000 2023-09-23T03:26:32.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.355+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.5 v2:172.21.15.190:6800/882536885 2 ==== osd_op_reply(28 [pgnls-filter start_epoch 110 out=81b] v110'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5649a609ed80 con 0x5649a57c5000 2023-09-23T03:26:32.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.355+0000 7fd3793915c0 20 datascan.forall_objects: OSD matched oid 100000001fc.00000000 2023-09-23T03:26:32.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.355+0000 7fd3793915c0 10 datascan.operator(): handling object 100000001fc.0 2023-09-23T03:26:32.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.356+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:29 15.c 15:30604137:::100000001fc.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b6700 con 0x5649a57c5000 2023-09-23T03:26:32.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.357+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.5 v2:172.21.15.190:6800/882536885 3 ==== osd_op_reply(29 100000001fc.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr,getxattr] v0'0 uv10 ondisk = 0) v8 ==== 416+0+46 (crc 0 0 0) 0x5649a609ed80 con 0x5649a57c5000 2023-09-23T03:26:32.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.357+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:30 16.c 16:30604137:::100000001fc.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57ea000 con 0x5649a57c5000 2023-09-23T03:26:32.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.358+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.5 v2:172.21.15.190:6800/882536885 4 ==== osd_op_reply(30 100000001fc.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x5649a609ed80 con 0x5649a57c5000 2023-09-23T03:26:32.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.358+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:31 14.1f 14:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57eae00 con 0x5649a56a3000 2023-09-23T03:26:32.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.359+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.0 v2:172.21.15.154:6800/2920938138 4 ==== osd_op_reply(31 1.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x5649a609ed80 con 0x5649a56a3000 2023-09-23T03:26:32.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.359+0000 7fd3793915c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-09-23T03:26:32.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.359+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:32 14.1f 14:ff5b34d6:::1.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57eb180 con 0x5649a56a3000 2023-09-23T03:26:32.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.362+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.0 v2:172.21.15.154:6800/2920938138 5 ==== osd_op_reply(32 1.00000000 [create,omap-set-header] v110'11 uv11 ondisk = 0) v8 ==== 196+0+0 (crc 0 0 0) 0x5649a609ed80 con 0x5649a56a3000 2023-09-23T03:26:32.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.362+0000 7fd3793915c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x1 2023-09-23T03:26:32.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.362+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:33 14.1f 14:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=23b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57eb500 con 0x5649a56a3000 2023-09-23T03:26:32.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.363+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.0 v2:172.21.15.154:6800/2920938138 6 ==== osd_op_reply(33 1.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv11 ondisk = 0) v8 ==== 154+0+4 (crc 0 0 0) 0x5649a609ed80 con 0x5649a56a3000 2023-09-23T03:26:32.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.363+0000 7fd3793915c0 20 datascan.read_dentry: oid=1.00000000 dname=lost+found frag=*, r=0 2023-09-23T03:26:32.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.363+0000 7fd3793915c0 20 datascan.read_dentry: lost+found_head not found in result 2023-09-23T03:26:32.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.363+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:34 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57eb880 con 0x5649a56a3000 2023-09-23T03:26:32.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.366+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.0 v2:172.21.15.154:6800/2920938138 7 ==== osd_op_reply(34 1.00000000 [omap-set-vals] v110'12 uv12 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x5649a609ed80 con 0x5649a56a3000 2023-09-23T03:26:32.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.366+0000 7fd3793915c0 20 datascan.inject_linkage: Injected dentry 0x0x1/lost+found pointing to 0x0x4 2023-09-23T03:26:32.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.366+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:35 14.3 14:c0ce80d4:::4.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57ea000 con 0x5649a56a3000 2023-09-23T03:26:32.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.366+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.0 v2:172.21.15.154:6800/2920938138 8 ==== osd_op_reply(35 4.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x5649a609ed80 con 0x5649a56a3000 2023-09-23T03:26:32.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.366+0000 7fd3793915c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-09-23T03:26:32.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.366+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:36 14.3 14:c0ce80d4:::4.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57ebc00 con 0x5649a56a3000 2023-09-23T03:26:32.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.369+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.0 v2:172.21.15.154:6800/2920938138 9 ==== osd_op_reply(36 4.00000000 [create,omap-set-header] v110'1 uv1 ondisk = 0) v8 ==== 196+0+0 (crc 0 0 0) 0x5649a609ed80 con 0x5649a56a3000 2023-09-23T03:26:32.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.369+0000 7fd3793915c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x4 2023-09-23T03:26:32.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.369+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:37 14.3 14:c0ce80d4:::4.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b6e00 con 0x5649a56a3000 2023-09-23T03:26:32.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.371+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.0 v2:172.21.15.154:6800/2920938138 10 ==== osd_op_reply(37 4.00000000 [omap-set-vals] v110'2 uv2 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x5649a609ed80 con 0x5649a56a3000 2023-09-23T03:26:32.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.371+0000 7fd3793915c0 20 datascan.inject_linkage: Injected dentry 0x0x4/100000001fc pointing to 0x0x100000001fc 2023-09-23T03:26:32.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.371+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:38 15.2c 15:34000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b7880 con 0x5649a56a2c00 2023-09-23T03:26:32.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.372+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.4 v2:172.21.15.190:6808/717302641 6 ==== osd_op_reply(38 [pgnls-filter start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609eb40 con 0x5649a56a2c00 2023-09-23T03:26:32.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.372+0000 7fd372b2f640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:39 15.1c 15:38000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b6380 con 0x5649a57c4000 2023-09-23T03:26:32.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.373+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 5 ==== osd_op_reply(39 [pgnls-filter start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.373+0000 7fd373330640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:40 15.3c 15:3c000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a53a5880 con 0x5649a57c4c00 2023-09-23T03:26:32.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.374+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.3 v2:172.21.15.154:6808/1133688468 3 ==== osd_op_reply(40 [pgnls-filter start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609efc0 con 0x5649a57c4c00 2023-09-23T03:26:32.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.374+0000 7fd372b2f640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:41 15.2 15:40000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b7500 con 0x5649a56a3400 2023-09-23T03:26:32.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.374+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.2 v2:172.21.15.154:6822/2884407243 5 ==== osd_op_reply(41 [pgnls-filter start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609efc0 con 0x5649a56a3400 2023-09-23T03:26:32.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.374+0000 7fd373330640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:42 15.22 15:44000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a53a5c00 con 0x5649a56a3400 2023-09-23T03:26:32.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.375+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.2 v2:172.21.15.154:6822/2884407243 6 ==== osd_op_reply(42 [pgnls-filter start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609efc0 con 0x5649a56a3400 2023-09-23T03:26:32.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.375+0000 7fd372b2f640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:43 15.12 15:48000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b6000 con 0x5649a57c4c00 2023-09-23T03:26:32.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.376+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.3 v2:172.21.15.154:6808/1133688468 4 ==== osd_op_reply(43 [pgnls-filter start_epoch 110 out=49b] v108'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609efc0 con 0x5649a57c4c00 2023-09-23T03:26:32.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.376+0000 7fd373330640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:44 15.32 15:4c000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b7500 con 0x5649a56a3400 2023-09-23T03:26:32.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.376+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.2 v2:172.21.15.154:6822/2884407243 7 ==== osd_op_reply(44 [pgnls-filter start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609efc0 con 0x5649a56a3400 2023-09-23T03:26:32.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.376+0000 7fd372b2f640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:45 15.a 15:50000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57ea000 con 0x5649a56a3400 2023-09-23T03:26:32.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.377+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.2 v2:172.21.15.154:6822/2884407243 8 ==== osd_op_reply(45 [pgnls-filter start_epoch 110 out=49b] v108'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609efc0 con 0x5649a56a3400 2023-09-23T03:26:32.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.377+0000 7fd373330640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:46 15.2a 15:54000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b7c00 con 0x5649a56a3000 2023-09-23T03:26:32.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.378+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.0 v2:172.21.15.154:6800/2920938138 11 ==== osd_op_reply(46 [pgnls-filter start_epoch 110 out=81b] v110'15 uv15 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5649a609ed80 con 0x5649a56a3000 2023-09-23T03:26:32.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.378+0000 7fd3793915c0 20 datascan.forall_objects: OSD matched oid 100000001f6.00000000 2023-09-23T03:26:32.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.378+0000 7fd3793915c0 10 datascan.operator(): handling object 100000001f6.0 2023-09-23T03:26:32.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.378+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:47 15.2a 15:57fe7535:::100000001f6.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b6a80 con 0x5649a56a3000 2023-09-23T03:26:32.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.378+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.0 v2:172.21.15.154:6800/2920938138 12 ==== osd_op_reply(47 100000001f6.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (111) out=111b,getxattr,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x5649a609ed80 con 0x5649a56a3000 2023-09-23T03:26:32.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.379+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:48 16.a 16:57fe7535:::100000001f6.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57eb500 con 0x5649a56a3400 2023-09-23T03:26:32.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.379+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.2 v2:172.21.15.154:6822/2884407243 9 ==== osd_op_reply(48 100000001f6.00000000 [getxattr (30) out=30b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x5649a609efc0 con 0x5649a56a3400 2023-09-23T03:26:32.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.379+0000 7fd3793915c0 20 datascan.operator(): loaded layout from xattr: pi: 16 os: 262144 sc: 4 su: 65536 2023-09-23T03:26:32.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.379+0000 7fd3793915c0 10 datascan.operator(): calculating striped size from complete objs: 0, partial objs: 4 2023-09-23T03:26:32.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.379+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:49 16.a 16:57fe7535:::100000001f6.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57eb880 con 0x5649a56a3400 2023-09-23T03:26:32.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.380+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.2 v2:172.21.15.154:6822/2884407243 10 ==== osd_op_reply(49 100000001f6.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5649a609efc0 con 0x5649a56a3400 2023-09-23T03:26:32.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.380+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:50 16.1c 16:3d6db04b:::100000001f6.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57ebc00 con 0x5649a56a3400 2023-09-23T03:26:32.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.380+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.2 v2:172.21.15.154:6822/2884407243 11 ==== osd_op_reply(50 100000001f6.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5649a609efc0 con 0x5649a56a3400 2023-09-23T03:26:32.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.380+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:51 16.1f 16:faedc8eb:::100000001f6.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57ea000 con 0x5649a57c5000 2023-09-23T03:26:32.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.381+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.5 v2:172.21.15.190:6800/882536885 5 ==== osd_op_reply(51 100000001f6.00000002 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5649a609ed80 con 0x5649a57c5000 2023-09-23T03:26:32.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.381+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:52 16.e 16:728aa123:::100000001f6.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57eb500 con 0x5649a56a3400 2023-09-23T03:26:32.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.382+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.2 v2:172.21.15.154:6822/2884407243 12 ==== osd_op_reply(52 100000001f6.00000003 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5649a609efc0 con 0x5649a56a3400 2023-09-23T03:26:32.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.382+0000 7fd3793915c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f6 2023-09-23T03:26:32.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.382+0000 7fd3793915c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/flushed_file_1 2023-09-23T03:26:32.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.382+0000 7fd3793915c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=flushed_file_1 2023-09-23T03:26:32.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.382+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:53 14.10 14:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57eb880 con 0x5649a57c4000 2023-09-23T03:26:32.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.382+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 6 ==== osd_op_reply(53 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.382+0000 7fd3793915c0 4 datascan.get_frag_of: Unexpected error on '10000000000.00000000': (2) No such file or directory 2023-09-23T03:26:32.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.383+0000 7fd3793915c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/flushed_file_1, will insert to root 2023-09-23T03:26:32.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.383+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:54 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b6000 con 0x5649a57c4000 2023-09-23T03:26:32.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.383+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 7 ==== osd_op_reply(54 10000000000.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.383+0000 7fd3793915c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-09-23T03:26:32.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.383+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:55 14.10 14:0cf6ad0f:::10000000000.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b7500 con 0x5649a57c4000 2023-09-23T03:26:32.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.387+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 8 ==== osd_op_reply(55 10000000000.00000000 [create,omap-set-header] v110'16 uv16 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.387+0000 7fd3793915c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x10000000000 2023-09-23T03:26:32.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.387+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:56 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57e4000 con 0x5649a57c4000 2023-09-23T03:26:32.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.388+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 9 ==== osd_op_reply(56 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.388+0000 7fd3793915c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_1 frag=*, r=0 2023-09-23T03:26:32.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.388+0000 7fd3793915c0 20 datascan.read_dentry: flushed_file_1_head not found in result 2023-09-23T03:26:32.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.388+0000 7fd3793915c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f6 at 0x0x10000000000/flushed_file_1 with size=524288 bytes 2023-09-23T03:26:32.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.388+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:57 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57e4380 con 0x5649a57c4000 2023-09-23T03:26:32.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.391+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 10 ==== osd_op_reply(57 10000000000.00000000 [omap-set-vals] v110'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.391+0000 7fd3793915c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_1 pointing to 0x0x100000001f6 2023-09-23T03:26:32.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.391+0000 7fd3793915c0 10 datascan.inject_with_backtrace: backptr: 0x0x1/stripey 2023-09-23T03:26:32.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.391+0000 7fd3793915c0 20 datascan.get_frag_of: dirino=0x1 target_dname=stripey 2023-09-23T03:26:32.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.391+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:58 14.1f 14:ff5b34d6:::1.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57e4700 con 0x5649a56a3000 2023-09-23T03:26:32.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.391+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.0 v2:172.21.15.154:6800/2920938138 13 ==== osd_op_reply(58 1.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 154+0+0 (crc 0 0 0) 0x5649a609ed80 con 0x5649a56a3000 2023-09-23T03:26:32.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.391+0000 7fd3793915c0 10 datascan.get_frag_of: No backtrace on '1.00000000' 2023-09-23T03:26:32.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.391+0000 7fd3793915c0 4 datascan.get_frag_of: No backtrace on '1.00000000': cannot determine fragtree 2023-09-23T03:26:32.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.391+0000 7fd3793915c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x1/stripey, will insert to root 2023-09-23T03:26:32.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.391+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:59 14.1f 14:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57e4a80 con 0x5649a56a3000 2023-09-23T03:26:32.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.392+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.0 v2:172.21.15.154:6800/2920938138 14 ==== osd_op_reply(59 1.00000000 [omap-get-header out=274b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+274 (crc 0 0 0) 0x5649a609ed80 con 0x5649a56a3000 2023-09-23T03:26:32.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.392+0000 7fd3793915c0 10 datascan.find_or_create_dirfrag: read_version = 12 2023-09-23T03:26:32.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.392+0000 7fd3793915c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x1 * 2023-09-23T03:26:32.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.392+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:60 14.1f 14:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57e4e00 con 0x5649a56a3000 2023-09-23T03:26:32.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.392+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.0 v2:172.21.15.154:6800/2920938138 15 ==== osd_op_reply(60 1.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+4 (crc 0 0 0) 0x5649a609ed80 con 0x5649a56a3000 2023-09-23T03:26:32.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.392+0000 7fd3793915c0 20 datascan.read_dentry: oid=1.00000000 dname=stripey frag=*, r=0 2023-09-23T03:26:32.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.392+0000 7fd3793915c0 20 datascan.read_dentry: stripey_head not found in result 2023-09-23T03:26:32.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.392+0000 7fd3793915c0 10 datascan.inject_with_backtrace: Linking ancestor directory of inode 0x0x10000000000 at 0x0x1:stripey 2023-09-23T03:26:32.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.392+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:61 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=508b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57e5180 con 0x5649a56a3000 2023-09-23T03:26:32.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.395+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.0 v2:172.21.15.154:6800/2920938138 16 ==== osd_op_reply(61 1.00000000 [omap-set-vals] v110'13 uv13 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x5649a609ed80 con 0x5649a56a3000 2023-09-23T03:26:32.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.396+0000 7fd3793915c0 20 datascan.inject_linkage: Injected dentry 0x0x1/stripey pointing to 0x0x10000000000 2023-09-23T03:26:32.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.396+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:62 15.1a 15:58000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57e5500 con 0x5649a57c5000 2023-09-23T03:26:32.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.397+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.5 v2:172.21.15.190:6800/882536885 6 ==== osd_op_reply(62 [pgnls-filter start_epoch 110 out=49b] v108'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609ed80 con 0x5649a57c5000 2023-09-23T03:26:32.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.397+0000 7fd373330640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:63 15.3a 15:5c000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b7180 con 0x5649a57c4400 2023-09-23T03:26:32.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.398+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.7 v2:172.21.15.190:6824/3460136272 5 ==== osd_op_reply(63 [pgnls-filter start_epoch 110 out=49b] v108'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609ed80 con 0x5649a57c4400 2023-09-23T03:26:32.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.398+0000 7fd372b2f640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:64 15.6 15:60000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57ea380 con 0x5649a56a3000 2023-09-23T03:26:32.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.399+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.0 v2:172.21.15.154:6800/2920938138 17 ==== osd_op_reply(64 [pgnls-filter start_epoch 110 out=49b] v108'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609ed80 con 0x5649a56a3000 2023-09-23T03:26:32.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.399+0000 7fd373330640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:65 15.26 15:64000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b7880 con 0x5649a56a3000 2023-09-23T03:26:32.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.399+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.0 v2:172.21.15.154:6800/2920938138 18 ==== osd_op_reply(65 [pgnls-filter start_epoch 110 out=49b] v108'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609ed80 con 0x5649a56a3000 2023-09-23T03:26:32.400 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.399+0000 7fd372b2f640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:66 15.16 15:68000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57ea700 con 0x5649a56a3000 2023-09-23T03:26:32.400 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.400+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.0 v2:172.21.15.154:6800/2920938138 19 ==== osd_op_reply(66 [pgnls-filter start_epoch 110 out=49b] v108'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609ed80 con 0x5649a56a3000 2023-09-23T03:26:32.400 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.400+0000 7fd373330640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:67 15.36 15:6c000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a53a5880 con 0x5649a57c4800 2023-09-23T03:26:32.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.401+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.6 v2:172.21.15.190:6816/1964294001 5 ==== osd_op_reply(67 [pgnls-filter start_epoch 110 out=101b] v110'31 uv31 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4800 2023-09-23T03:26:32.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.401+0000 7fd3793915c0 20 datascan.forall_objects: OSD matched oid 100000001f9.00000000 2023-09-23T03:26:32.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.401+0000 7fd3793915c0 10 datascan.operator(): handling object 100000001f9.0 2023-09-23T03:26:32.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.402+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:68 15.36 15:6ee5e2c4:::100000001f9.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57e5880 con 0x5649a57c4800 2023-09-23T03:26:32.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.403+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.6 v2:172.21.15.190:6816/1964294001 6 ==== osd_op_reply(68 100000001f9.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (111) out=111b,getxattr,getxattr] v0'0 uv31 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4800 2023-09-23T03:26:32.403 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.403+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:69 16.16 16:6ee5e2c4:::100000001f9.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57e4700 con 0x5649a57c4c00 2023-09-23T03:26:32.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.404+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.3 v2:172.21.15.154:6808/1133688468 5 ==== osd_op_reply(69 100000001f9.00000000 [getxattr (30) out=30b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x5649a609efc0 con 0x5649a57c4c00 2023-09-23T03:26:32.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.404+0000 7fd3793915c0 20 datascan.operator(): loaded layout from xattr: pi: 16 os: 262144 sc: 4 su: 65536 2023-09-23T03:26:32.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.404+0000 7fd3793915c0 10 datascan.operator(): calculating striped size from complete objs: 4, partial objs: 4 2023-09-23T03:26:32.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.404+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:70 16.7 16:e7ecf421:::100000001f9.00000004:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b6e00 con 0x5649a57c4c00 2023-09-23T03:26:32.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.404+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.3 v2:172.21.15.154:6808/1133688468 6 ==== osd_op_reply(70 100000001f9.00000004 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5649a609efc0 con 0x5649a57c4c00 2023-09-23T03:26:32.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.405+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:71 16.2 16:46f89f8a:::100000001f9.00000005:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b7c00 con 0x5649a56a2c00 2023-09-23T03:26:32.406 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.406+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.4 v2:172.21.15.190:6808/717302641 7 ==== osd_op_reply(71 100000001f9.00000005 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5649a609eb40 con 0x5649a56a2c00 2023-09-23T03:26:32.406 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.406+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:72 16.16 16:6df493a7:::100000001f9.00000006:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b6700 con 0x5649a57c4c00 2023-09-23T03:26:32.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.406+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.3 v2:172.21.15.154:6808/1133688468 7 ==== osd_op_reply(72 100000001f9.00000006 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5649a609efc0 con 0x5649a57c4c00 2023-09-23T03:26:32.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.406+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:73 16.17 16:ee7e3069:::100000001f9.00000007:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b6a80 con 0x5649a56a3400 2023-09-23T03:26:32.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.407+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.2 v2:172.21.15.154:6822/2884407243 13 ==== osd_op_reply(73 100000001f9.00000007 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5649a609efc0 con 0x5649a56a3400 2023-09-23T03:26:32.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.407+0000 7fd3793915c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f9 2023-09-23T03:26:32.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.407+0000 7fd3793915c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/flushed_file_4 2023-09-23T03:26:32.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.407+0000 7fd3793915c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=flushed_file_4 2023-09-23T03:26:32.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.407+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:74 14.10 14:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b6380 con 0x5649a57c4000 2023-09-23T03:26:32.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.408+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 11 ==== osd_op_reply(74 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.408+0000 7fd3793915c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-09-23T03:26:32.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.408+0000 7fd3793915c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-09-23T03:26:32.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.408+0000 7fd3793915c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/flushed_file_4, will insert to root 2023-09-23T03:26:32.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.408+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:75 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b6000 con 0x5649a57c4000 2023-09-23T03:26:32.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.409+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 12 ==== osd_op_reply(75 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.409+0000 7fd3793915c0 10 datascan.find_or_create_dirfrag: read_version = 17 2023-09-23T03:26:32.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.409+0000 7fd3793915c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-09-23T03:26:32.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.409+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:76 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b7500 con 0x5649a57c4000 2023-09-23T03:26:32.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.410+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 13 ==== osd_op_reply(76 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.410+0000 7fd3793915c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_4 frag=*, r=0 2023-09-23T03:26:32.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.410+0000 7fd3793915c0 20 datascan.read_dentry: flushed_file_4_head not found in result 2023-09-23T03:26:32.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.410+0000 7fd3793915c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f9 at 0x0x10000000000/flushed_file_4 with size=1441792 bytes 2023-09-23T03:26:32.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.410+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:77 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57ea000 con 0x5649a57c4000 2023-09-23T03:26:32.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.413+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 14 ==== osd_op_reply(77 10000000000.00000000 [omap-set-vals] v110'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.413+0000 7fd3793915c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_4 pointing to 0x0x100000001f9 2023-09-23T03:26:32.414 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.413+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:78 15.36 15:6fe38dc2::::head [pgnls-filter start_epoch 110 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57eb880 con 0x5649a57c4800 2023-09-23T03:26:32.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.414+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.6 v2:172.21.15.190:6816/1964294001 7 ==== osd_op_reply(78 [pgnls-filter start_epoch 110 out=81b] v110'31 uv31 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4800 2023-09-23T03:26:32.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.415+0000 7fd3793915c0 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2023-09-23T03:26:32.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.415+0000 7fd3793915c0 10 datascan.operator(): handling object 100000001f7.0 2023-09-23T03:26:32.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.415+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:79 15.36 15:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57e5c00 con 0x5649a57c4800 2023-09-23T03:26:32.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.416+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.6 v2:172.21.15.190:6816/1964294001 8 ==== osd_op_reply(79 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (111) out=111b,getxattr,getxattr] v0'0 uv27 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4800 2023-09-23T03:26:32.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.416+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:80 16.16 16:6fe38dc2:::100000001f7.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b6380 con 0x5649a57c4c00 2023-09-23T03:26:32.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.417+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.3 v2:172.21.15.154:6808/1133688468 8 ==== osd_op_reply(80 100000001f7.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x5649a609efc0 con 0x5649a57c4c00 2023-09-23T03:26:32.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.417+0000 7fd3793915c0 20 datascan.operator(): loaded layout from xattr: pi: 16 os: 262144 sc: 4 su: 65536 2023-09-23T03:26:32.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.417+0000 7fd3793915c0 10 datascan.operator(): calculating striped size from complete objs: 0, partial objs: 4 2023-09-23T03:26:32.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.417+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:81 16.16 16:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b6000 con 0x5649a57c4c00 2023-09-23T03:26:32.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.418+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.3 v2:172.21.15.154:6808/1133688468 9 ==== osd_op_reply(81 100000001f7.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5649a609efc0 con 0x5649a57c4c00 2023-09-23T03:26:32.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.418+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:82 16.1 16:848ebb0b:::100000001f7.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b7500 con 0x5649a56a3400 2023-09-23T03:26:32.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.419+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.2 v2:172.21.15.154:6822/2884407243 14 ==== osd_op_reply(82 100000001f7.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5649a609efc0 con 0x5649a56a3400 2023-09-23T03:26:32.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.419+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:83 16.1c 16:398da958:::100000001f7.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b6e00 con 0x5649a56a3400 2023-09-23T03:26:32.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.420+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.2 v2:172.21.15.154:6822/2884407243 15 ==== osd_op_reply(83 100000001f7.00000002 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5649a609efc0 con 0x5649a56a3400 2023-09-23T03:26:32.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.420+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:84 16.6 16:613bf181:::100000001f7.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b6700 con 0x5649a57c4800 2023-09-23T03:26:32.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.421+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.6 v2:172.21.15.190:6816/1964294001 9 ==== osd_op_reply(84 100000001f7.00000003 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4800 2023-09-23T03:26:32.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.421+0000 7fd3793915c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2023-09-23T03:26:32.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.421+0000 7fd3793915c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/flushed_file_2 2023-09-23T03:26:32.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.421+0000 7fd3793915c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=flushed_file_2 2023-09-23T03:26:32.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.421+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:85 14.10 14:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b6a80 con 0x5649a57c4000 2023-09-23T03:26:32.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.422+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 15 ==== osd_op_reply(85 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.422+0000 7fd3793915c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-09-23T03:26:32.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.422+0000 7fd3793915c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-09-23T03:26:32.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.422+0000 7fd3793915c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/flushed_file_2, will insert to root 2023-09-23T03:26:32.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.422+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:86 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57ebc00 con 0x5649a57c4000 2023-09-23T03:26:32.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.423+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 16 ==== osd_op_reply(86 10000000000.00000000 [omap-get-header out=274b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.423+0000 7fd3793915c0 10 datascan.find_or_create_dirfrag: read_version = 18 2023-09-23T03:26:32.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.423+0000 7fd3793915c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-09-23T03:26:32.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.423+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:87 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57eb500 con 0x5649a57c4000 2023-09-23T03:26:32.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.424+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 17 ==== osd_op_reply(87 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.424+0000 7fd3793915c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_2 frag=*, r=0 2023-09-23T03:26:32.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.424+0000 7fd3793915c0 20 datascan.read_dentry: flushed_file_2_head not found in result 2023-09-23T03:26:32.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.424+0000 7fd3793915c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000000/flushed_file_2 with size=917504 bytes 2023-09-23T03:26:32.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.424+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:88 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a53a5880 con 0x5649a57c4000 2023-09-23T03:26:32.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.428+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 18 ==== osd_op_reply(88 10000000000.00000000 [omap-set-vals] v110'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.428+0000 7fd3793915c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_2 pointing to 0x0x100000001f7 2023-09-23T03:26:32.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.428+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:89 15.e 15:70000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57e5c00 con 0x5649a56a3400 2023-09-23T03:26:32.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.429+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.2 v2:172.21.15.154:6822/2884407243 16 ==== osd_op_reply(89 [pgnls-filter start_epoch 110 out=49b] v108'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609efc0 con 0x5649a56a3400 2023-09-23T03:26:32.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.429+0000 7fd372b2f640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:90 15.2e 15:74000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57eaa80 con 0x5649a57c4c00 2023-09-23T03:26:32.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.430+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.3 v2:172.21.15.154:6808/1133688468 10 ==== osd_op_reply(90 [pgnls-filter start_epoch 110 out=49b] v108'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609efc0 con 0x5649a57c4c00 2023-09-23T03:26:32.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.430+0000 7fd373330640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:91 15.1e 15:78000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a53a5c00 con 0x5649a56a3400 2023-09-23T03:26:32.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.431+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.2 v2:172.21.15.154:6822/2884407243 17 ==== osd_op_reply(91 [pgnls-filter start_epoch 110 out=49b] v108'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609efc0 con 0x5649a56a3400 2023-09-23T03:26:32.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.431+0000 7fd372b2f640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:92 15.3e 15:7c000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57eae00 con 0x5649a56a3000 2023-09-23T03:26:32.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.432+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.0 v2:172.21.15.154:6800/2920938138 20 ==== osd_op_reply(92 [pgnls-filter start_epoch 110 out=49b] v108'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609ed80 con 0x5649a56a3000 2023-09-23T03:26:32.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.432+0000 7fd373330640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:93 15.1 15:80000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57ea380 con 0x5649a57c4800 2023-09-23T03:26:32.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.433+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.6 v2:172.21.15.190:6816/1964294001 10 ==== osd_op_reply(93 [pgnls-filter start_epoch 110 out=49b] v108'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4800 2023-09-23T03:26:32.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.433+0000 7fd372b2f640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:94 15.21 15:84000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57eb180 con 0x5649a57c4c00 2023-09-23T03:26:32.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.434+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.3 v2:172.21.15.154:6808/1133688468 11 ==== osd_op_reply(94 [pgnls-filter start_epoch 110 out=49b] v108'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609efc0 con 0x5649a57c4c00 2023-09-23T03:26:32.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.434+0000 7fd373330640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:95 15.11 15:88000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b7180 con 0x5649a56a3400 2023-09-23T03:26:32.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.435+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.2 v2:172.21.15.154:6822/2884407243 18 ==== osd_op_reply(95 [pgnls-filter start_epoch 110 out=49b] v108'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609efc0 con 0x5649a56a3400 2023-09-23T03:26:32.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.435+0000 7fd372b2f640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:96 15.31 15:8c000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57e4e00 con 0x5649a57c4c00 2023-09-23T03:26:32.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.436+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.3 v2:172.21.15.154:6808/1133688468 12 ==== osd_op_reply(96 [pgnls-filter start_epoch 110 out=49b] v108'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609efc0 con 0x5649a57c4c00 2023-09-23T03:26:32.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.436+0000 7fd373330640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:97 15.9 15:90000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b7880 con 0x5649a57c4800 2023-09-23T03:26:32.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.437+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.6 v2:172.21.15.190:6816/1964294001 11 ==== osd_op_reply(97 [pgnls-filter start_epoch 110 out=49b] v108'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4800 2023-09-23T03:26:32.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.437+0000 7fd372b2f640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:98 15.29 15:94000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57e5180 con 0x5649a56a3000 2023-09-23T03:26:32.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.438+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.0 v2:172.21.15.154:6800/2920938138 21 ==== osd_op_reply(98 [pgnls-filter start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609ed80 con 0x5649a56a3000 2023-09-23T03:26:32.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.439+0000 7fd373330640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:99 15.19 15:98000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b6380 con 0x5649a56a2c00 2023-09-23T03:26:32.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.440+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.4 v2:172.21.15.190:6808/717302641 8 ==== osd_op_reply(99 [pgnls-filter start_epoch 110 out=81b] v110'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5649a609eb40 con 0x5649a56a2c00 2023-09-23T03:26:32.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.440+0000 7fd3793915c0 20 datascan.forall_objects: OSD matched oid 100000001f8.00000000 2023-09-23T03:26:32.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.440+0000 7fd3793915c0 10 datascan.operator(): handling object 100000001f8.0 2023-09-23T03:26:32.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.440+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:100 15.19 15:997b7129:::100000001f8.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57e5880 con 0x5649a56a2c00 2023-09-23T03:26:32.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.441+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.4 v2:172.21.15.190:6808/717302641 9 ==== osd_op_reply(100 100000001f8.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (111) out=111b,getxattr,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x5649a609eb40 con 0x5649a56a2c00 2023-09-23T03:26:32.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.441+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:101 16.19 16:997b7129:::100000001f8.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57e4a80 con 0x5649a56a2c00 2023-09-23T03:26:32.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.443+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.4 v2:172.21.15.190:6808/717302641 10 ==== osd_op_reply(101 100000001f8.00000000 [getxattr (30) out=30b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x5649a609eb40 con 0x5649a56a2c00 2023-09-23T03:26:32.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.443+0000 7fd3793915c0 20 datascan.operator(): loaded layout from xattr: pi: 16 os: 262144 sc: 4 su: 65536 2023-09-23T03:26:32.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.443+0000 7fd3793915c0 10 datascan.operator(): calculating striped size from complete objs: 0, partial objs: 4 2023-09-23T03:26:32.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.443+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:102 16.19 16:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57e4000 con 0x5649a56a2c00 2023-09-23T03:26:32.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.444+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.4 v2:172.21.15.190:6808/717302641 11 ==== osd_op_reply(102 100000001f8.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5649a609eb40 con 0x5649a56a2c00 2023-09-23T03:26:32.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.444+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:103 16.11 16:893be98d:::100000001f8.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57ea000 con 0x5649a57c5000 2023-09-23T03:26:32.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.445+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.5 v2:172.21.15.190:6800/882536885 7 ==== osd_op_reply(103 100000001f8.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5649a609ed80 con 0x5649a57c5000 2023-09-23T03:26:32.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.445+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:104 16.f 16:f26fdfa9:::100000001f8.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57eb880 con 0x5649a56a3000 2023-09-23T03:26:32.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.446+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.0 v2:172.21.15.154:6800/2920938138 22 ==== osd_op_reply(104 100000001f8.00000002 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5649a609ed80 con 0x5649a56a3000 2023-09-23T03:26:32.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.446+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:105 16.17 16:eff9002a:::100000001f8.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57ebc00 con 0x5649a56a3400 2023-09-23T03:26:32.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.447+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.2 v2:172.21.15.154:6822/2884407243 19 ==== osd_op_reply(105 100000001f8.00000003 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5649a609efc0 con 0x5649a56a3400 2023-09-23T03:26:32.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.447+0000 7fd3793915c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f8 2023-09-23T03:26:32.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.447+0000 7fd3793915c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/flushed_file_3 2023-09-23T03:26:32.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.447+0000 7fd3793915c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=flushed_file_3 2023-09-23T03:26:32.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.447+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:106 14.10 14:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57eb500 con 0x5649a57c4000 2023-09-23T03:26:32.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.448+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 19 ==== osd_op_reply(106 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.448+0000 7fd3793915c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-09-23T03:26:32.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.448+0000 7fd3793915c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-09-23T03:26:32.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.448+0000 7fd3793915c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/flushed_file_3, will insert to root 2023-09-23T03:26:32.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.448+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:107 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b7c00 con 0x5649a57c4000 2023-09-23T03:26:32.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.449+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 20 ==== osd_op_reply(107 10000000000.00000000 [omap-get-header out=274b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.449+0000 7fd3793915c0 10 datascan.find_or_create_dirfrag: read_version = 19 2023-09-23T03:26:32.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.449+0000 7fd3793915c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-09-23T03:26:32.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.449+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:108 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b6700 con 0x5649a57c4000 2023-09-23T03:26:32.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.450+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 21 ==== osd_op_reply(108 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.450+0000 7fd3793915c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_3 frag=*, r=0 2023-09-23T03:26:32.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.450+0000 7fd3793915c0 20 datascan.read_dentry: flushed_file_3_head not found in result 2023-09-23T03:26:32.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.450+0000 7fd3793915c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f8 at 0x0x10000000000/flushed_file_3 with size=917503 bytes 2023-09-23T03:26:32.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.450+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:109 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b6a80 con 0x5649a57c4000 2023-09-23T03:26:32.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.454+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 22 ==== osd_op_reply(109 10000000000.00000000 [omap-set-vals] v110'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.454+0000 7fd3793915c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_3 pointing to 0x0x100000001f8 2023-09-23T03:26:32.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.454+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:110 15.39 15:9c000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57e4700 con 0x5649a57c4400 2023-09-23T03:26:32.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.455+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.7 v2:172.21.15.190:6824/3460136272 6 ==== osd_op_reply(110 [pgnls-filter start_epoch 110 out=49b] v108'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609ed80 con 0x5649a57c4400 2023-09-23T03:26:32.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.455+0000 7fd373330640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:111 15.5 15:a0000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b6000 con 0x5649a57c4800 2023-09-23T03:26:32.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.456+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.6 v2:172.21.15.190:6816/1964294001 12 ==== osd_op_reply(111 [pgnls-filter start_epoch 110 out=49b] v108'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4800 2023-09-23T03:26:32.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.456+0000 7fd372b2f640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:112 15.25 15:a4000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57e5500 con 0x5649a57c4400 2023-09-23T03:26:32.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.457+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.7 v2:172.21.15.190:6824/3460136272 7 ==== osd_op_reply(112 [pgnls-filter start_epoch 110 out=49b] v108'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609ed80 con 0x5649a57c4400 2023-09-23T03:26:32.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.457+0000 7fd373330640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:113 15.15 15:a8000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b7500 con 0x5649a57c4400 2023-09-23T03:26:32.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.459+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.7 v2:172.21.15.190:6824/3460136272 8 ==== osd_op_reply(113 [pgnls-filter start_epoch 110 out=49b] v108'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609ed80 con 0x5649a57c4400 2023-09-23T03:26:32.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.459+0000 7fd372b2f640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:114 15.35 15:ac000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a53a5c00 con 0x5649a57c4400 2023-09-23T03:26:32.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.460+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.7 v2:172.21.15.190:6824/3460136272 9 ==== osd_op_reply(114 [pgnls-filter start_epoch 110 out=49b] v108'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609ed80 con 0x5649a57c4400 2023-09-23T03:26:32.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.460+0000 7fd373330640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:115 15.d 15:b0000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b6e00 con 0x5649a57c4400 2023-09-23T03:26:32.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.461+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.7 v2:172.21.15.190:6824/3460136272 10 ==== osd_op_reply(115 [pgnls-filter start_epoch 110 out=49b] v108'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609ed80 con 0x5649a57c4400 2023-09-23T03:26:32.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.461+0000 7fd372b2f640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:116 15.2d 15:b4000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57ea700 con 0x5649a56a3400 2023-09-23T03:26:32.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.462+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.2 v2:172.21.15.154:6822/2884407243 20 ==== osd_op_reply(116 [pgnls-filter start_epoch 110 out=49b] v108'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609efc0 con 0x5649a56a3400 2023-09-23T03:26:32.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.462+0000 7fd373330640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:117 15.1d 15:b8000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57fe000 con 0x5649a56a2c00 2023-09-23T03:26:32.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.463+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.4 v2:172.21.15.190:6808/717302641 12 ==== osd_op_reply(117 [pgnls-filter start_epoch 110 out=49b] v108'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609eb40 con 0x5649a56a2c00 2023-09-23T03:26:32.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.463+0000 7fd372b2f640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:118 15.3d 15:bc000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57eae00 con 0x5649a57c4000 2023-09-23T03:26:32.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.464+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 23 ==== osd_op_reply(118 [pgnls-filter start_epoch 110 out=49b] v108'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.464+0000 7fd373330640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:119 15.3 15:c0000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57fe380 con 0x5649a56a2c00 2023-09-23T03:26:32.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.465+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.4 v2:172.21.15.190:6808/717302641 13 ==== osd_op_reply(119 [pgnls-filter start_epoch 110 out=49b] v108'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609eb40 con 0x5649a56a2c00 2023-09-23T03:26:32.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.465+0000 7fd372b2f640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:120 15.23 15:c4000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57eaa80 con 0x5649a56a3000 2023-09-23T03:26:32.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.466+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.0 v2:172.21.15.154:6800/2920938138 23 ==== osd_op_reply(120 [pgnls-filter start_epoch 110 out=49b] v108'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609ed80 con 0x5649a56a3000 2023-09-23T03:26:32.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.466+0000 7fd373330640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:121 15.13 15:c8000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57fe700 con 0x5649a57c4000 2023-09-23T03:26:32.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.467+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 24 ==== osd_op_reply(121 [pgnls-filter start_epoch 110 out=81b] v110'25 uv25 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.468+0000 7fd3793915c0 20 datascan.forall_objects: OSD matched oid 100000001fb.00000000 2023-09-23T03:26:32.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.468+0000 7fd3793915c0 10 datascan.operator(): handling object 100000001fb.0 2023-09-23T03:26:32.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.468+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:122 15.13 15:ca8fdbba:::100000001fb.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57e5880 con 0x5649a57c4000 2023-09-23T03:26:32.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.469+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 25 ==== osd_op_reply(122 100000001fb.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (111) out=111b,getxattr,getxattr] v0'0 uv25 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.469+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:123 16.13 16:ca8fdbba:::100000001fb.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57e4000 con 0x5649a57c4000 2023-09-23T03:26:32.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.470+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 26 ==== osd_op_reply(123 100000001fb.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.470+0000 7fd3793915c0 20 datascan.operator(): loaded layout from xattr: pi: 16 os: 262144 sc: 4 su: 65536 2023-09-23T03:26:32.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.470+0000 7fd3793915c0 10 datascan.operator(): calculating striped size from complete objs: 4, partial objs: 4 2023-09-23T03:26:32.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.470+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:124 16.3 16:c5d8c51d:::100000001fb.00000004:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57e5c00 con 0x5649a57c5000 2023-09-23T03:26:32.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.471+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.5 v2:172.21.15.190:6800/882536885 8 ==== osd_op_reply(124 100000001fb.00000004 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5649a609ed80 con 0x5649a57c5000 2023-09-23T03:26:32.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.471+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:125 16.f 16:f376f0dc:::100000001fb.00000005:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57e4380 con 0x5649a56a3000 2023-09-23T03:26:32.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.472+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.0 v2:172.21.15.154:6800/2920938138 24 ==== osd_op_reply(125 100000001fb.00000005 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5649a609ed80 con 0x5649a56a3000 2023-09-23T03:26:32.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.472+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:126 16.1e 16:7e8133fb:::100000001fb.00000006:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57ea380 con 0x5649a57c4400 2023-09-23T03:26:32.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.473+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.7 v2:172.21.15.190:6824/3460136272 11 ==== osd_op_reply(126 100000001fb.00000006 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5649a609ed80 con 0x5649a57c4400 2023-09-23T03:26:32.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.473+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:127 16.f 16:f776b802:::100000001fb.00000007:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57eb500 con 0x5649a56a3000 2023-09-23T03:26:32.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.474+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.0 v2:172.21.15.154:6800/2920938138 25 ==== osd_op_reply(127 100000001fb.00000007 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5649a609ed80 con 0x5649a56a3000 2023-09-23T03:26:32.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.474+0000 7fd3793915c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001fb 2023-09-23T03:26:32.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.474+0000 7fd3793915c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/flushed_file_6 2023-09-23T03:26:32.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.474+0000 7fd3793915c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=flushed_file_6 2023-09-23T03:26:32.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.474+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:128 14.10 14:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b7880 con 0x5649a57c4000 2023-09-23T03:26:32.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.475+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 27 ==== osd_op_reply(128 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.475+0000 7fd3793915c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-09-23T03:26:32.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.475+0000 7fd3793915c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-09-23T03:26:32.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.475+0000 7fd3793915c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/flushed_file_6, will insert to root 2023-09-23T03:26:32.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.475+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:129 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b6380 con 0x5649a57c4000 2023-09-23T03:26:32.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.476+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 28 ==== osd_op_reply(129 10000000000.00000000 [omap-get-header out=274b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.476+0000 7fd3793915c0 10 datascan.find_or_create_dirfrag: read_version = 20 2023-09-23T03:26:32.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.476+0000 7fd3793915c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-09-23T03:26:32.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.476+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:130 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a53a5880 con 0x5649a57c4000 2023-09-23T03:26:32.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.477+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 29 ==== osd_op_reply(130 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.477+0000 7fd3793915c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_6 frag=*, r=0 2023-09-23T03:26:32.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.477+0000 7fd3793915c0 20 datascan.read_dentry: flushed_file_6_head not found in result 2023-09-23T03:26:32.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.477+0000 7fd3793915c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fb at 0x0x10000000000/flushed_file_6 with size=1572864 bytes 2023-09-23T03:26:32.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.477+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:131 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b6000 con 0x5649a57c4000 2023-09-23T03:26:32.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.481+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 30 ==== osd_op_reply(131 10000000000.00000000 [omap-set-vals] v110'21 uv21 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.481+0000 7fd3793915c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_6 pointing to 0x0x100000001fb 2023-09-23T03:26:32.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.481+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:132 15.33 15:cc000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b6a80 con 0x5649a57c4c00 2023-09-23T03:26:32.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.482+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.3 v2:172.21.15.154:6808/1133688468 13 ==== osd_op_reply(132 [pgnls-filter start_epoch 110 out=49b] v108'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609efc0 con 0x5649a57c4c00 2023-09-23T03:26:32.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.482+0000 7fd373330640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:133 15.b 15:d0000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57fea80 con 0x5649a56a3400 2023-09-23T03:26:32.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.483+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.2 v2:172.21.15.154:6822/2884407243 21 ==== osd_op_reply(133 [pgnls-filter start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609efc0 con 0x5649a56a3400 2023-09-23T03:26:32.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.483+0000 7fd372b2f640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:134 15.2b 15:d4000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57fe000 con 0x5649a56a3400 2023-09-23T03:26:32.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.484+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.2 v2:172.21.15.154:6822/2884407243 22 ==== osd_op_reply(134 [pgnls-filter start_epoch 110 out=49b] v108'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609efc0 con 0x5649a56a3400 2023-09-23T03:26:32.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.484+0000 7fd373330640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:135 15.1b 15:d8000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57fee00 con 0x5649a57c4400 2023-09-23T03:26:32.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.485+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.7 v2:172.21.15.190:6824/3460136272 12 ==== osd_op_reply(135 [pgnls-filter start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609ed80 con 0x5649a57c4400 2023-09-23T03:26:32.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.485+0000 7fd372b2f640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:136 15.3b 15:dc000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57fe380 con 0x5649a57c4800 2023-09-23T03:26:32.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.486+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.6 v2:172.21.15.190:6816/1964294001 13 ==== osd_op_reply(136 [pgnls-filter start_epoch 110 out=49b] v108'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4800 2023-09-23T03:26:32.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.487+0000 7fd373330640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:137 15.7 15:e0000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57ff180 con 0x5649a57c4000 2023-09-23T03:26:32.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.488+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 31 ==== osd_op_reply(137 [pgnls-filter start_epoch 110 out=81b] v110'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.488+0000 7fd3793915c0 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2023-09-23T03:26:32.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.488+0000 7fd3793915c0 10 datascan.operator(): handling object 10000000001.0 2023-09-23T03:26:32.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.488+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:138 15.7 15:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b6700 con 0x5649a57c4000 2023-09-23T03:26:32.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.489+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 32 ==== osd_op_reply(138 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (111) out=111b,getxattr,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.489+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:139 16.7 16:e2e07930:::10000000001.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b7880 con 0x5649a57c4c00 2023-09-23T03:26:32.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.490+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.3 v2:172.21.15.154:6808/1133688468 14 ==== osd_op_reply(139 10000000001.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x5649a609efc0 con 0x5649a57c4c00 2023-09-23T03:26:32.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.490+0000 7fd3793915c0 20 datascan.operator(): loaded layout from xattr: pi: 16 os: 262144 sc: 4 su: 65536 2023-09-23T03:26:32.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.490+0000 7fd3793915c0 10 datascan.operator(): calculating striped size from complete objs: 0, partial objs: 4 2023-09-23T03:26:32.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.490+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:140 16.7 16:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b7500 con 0x5649a57c4c00 2023-09-23T03:26:32.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.491+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.3 v2:172.21.15.154:6808/1133688468 15 ==== osd_op_reply(140 10000000001.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5649a609efc0 con 0x5649a57c4c00 2023-09-23T03:26:32.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.491+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:141 16.f 16:f526278d:::10000000001.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b6e00 con 0x5649a56a3000 2023-09-23T03:26:32.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.492+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.0 v2:172.21.15.154:6800/2920938138 26 ==== osd_op_reply(141 10000000001.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5649a609ed80 con 0x5649a56a3000 2023-09-23T03:26:32.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.492+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:142 16.1e 16:7e4d8c81:::10000000001.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57e5500 con 0x5649a57c4400 2023-09-23T03:26:32.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.493+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.7 v2:172.21.15.190:6824/3460136272 13 ==== osd_op_reply(142 10000000001.00000002 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5649a609ed80 con 0x5649a57c4400 2023-09-23T03:26:32.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.493+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:143 16.e 16:72395991:::10000000001.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57e4700 con 0x5649a56a3400 2023-09-23T03:26:32.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.494+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.2 v2:172.21.15.154:6822/2884407243 23 ==== osd_op_reply(143 10000000001.00000003 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5649a609efc0 con 0x5649a56a3400 2023-09-23T03:26:32.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.494+0000 7fd3793915c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2023-09-23T03:26:32.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.494+0000 7fd3793915c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/flushed_file_0 2023-09-23T03:26:32.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.494+0000 7fd3793915c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=flushed_file_0 2023-09-23T03:26:32.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.494+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:144 14.10 14:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57e5180 con 0x5649a57c4000 2023-09-23T03:26:32.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.495+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 33 ==== osd_op_reply(144 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.495+0000 7fd3793915c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-09-23T03:26:32.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.495+0000 7fd3793915c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-09-23T03:26:32.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.495+0000 7fd3793915c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/flushed_file_0, will insert to root 2023-09-23T03:26:32.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.495+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:145 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57e5880 con 0x5649a57c4000 2023-09-23T03:26:32.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.496+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 34 ==== osd_op_reply(145 10000000000.00000000 [omap-get-header out=274b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.496+0000 7fd3793915c0 10 datascan.find_or_create_dirfrag: read_version = 21 2023-09-23T03:26:32.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.496+0000 7fd3793915c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-09-23T03:26:32.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.496+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:146 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57e4a80 con 0x5649a57c4000 2023-09-23T03:26:32.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.497+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 35 ==== osd_op_reply(146 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.497+0000 7fd3793915c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_0 frag=*, r=0 2023-09-23T03:26:32.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.497+0000 7fd3793915c0 20 datascan.read_dentry: flushed_file_0_head not found in result 2023-09-23T03:26:32.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.497+0000 7fd3793915c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/flushed_file_0 with size=1048576 bytes 2023-09-23T03:26:32.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.497+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:147 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57e4000 con 0x5649a57c4000 2023-09-23T03:26:32.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.500+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 36 ==== osd_op_reply(147 10000000000.00000000 [omap-set-vals] v110'22 uv22 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.500+0000 7fd3793915c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_0 pointing to 0x0x10000000001 2023-09-23T03:26:32.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.500+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:148 15.27 15:e4000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a53a5c00 con 0x5649a57c4800 2023-09-23T03:26:32.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.501+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.6 v2:172.21.15.190:6816/1964294001 14 ==== osd_op_reply(148 [pgnls-filter start_epoch 110 out=49b] v108'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4800 2023-09-23T03:26:32.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.501+0000 7fd373330640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:149 15.17 15:e8000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57ff500 con 0x5649a56a2c00 2023-09-23T03:26:32.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.503+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.4 v2:172.21.15.190:6808/717302641 14 ==== osd_op_reply(149 [pgnls-filter start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609eb40 con 0x5649a56a2c00 2023-09-23T03:26:32.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.503+0000 7fd372b2f640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:150 15.37 15:ec000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57fe700 con 0x5649a56a2c00 2023-09-23T03:26:32.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.504+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.4 v2:172.21.15.190:6808/717302641 15 ==== osd_op_reply(150 [pgnls-filter start_epoch 110 out=49b] v108'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609eb40 con 0x5649a56a2c00 2023-09-23T03:26:32.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.504+0000 7fd373330640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:151 15.f 15:f0000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57ff880 con 0x5649a57c4800 2023-09-23T03:26:32.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.505+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.6 v2:172.21.15.190:6816/1964294001 15 ==== osd_op_reply(151 [pgnls-filter start_epoch 110 out=49b] v108'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4800 2023-09-23T03:26:32.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.505+0000 7fd372b2f640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:152 15.2f 15:f4000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57ffc00 con 0x5649a56a3000 2023-09-23T03:26:32.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.506+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.0 v2:172.21.15.154:6800/2920938138 27 ==== osd_op_reply(152 [pgnls-filter start_epoch 110 out=49b] v108'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609ed80 con 0x5649a56a3000 2023-09-23T03:26:32.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.506+0000 7fd373330640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:153 15.1f 15:f8000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57eb180 con 0x5649a57c4c00 2023-09-23T03:26:32.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.507+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.3 v2:172.21.15.154:6808/1133688468 16 ==== osd_op_reply(153 [pgnls-filter start_epoch 110 out=49b] v108'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5649a609efc0 con 0x5649a57c4c00 2023-09-23T03:26:32.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.507+0000 7fd372b2f640 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:154 15.3f 15:fc000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57eae00 con 0x5649a57c4000 2023-09-23T03:26:32.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.508+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 37 ==== osd_op_reply(154 [pgnls-filter start_epoch 110 out=81b] v110'18 uv18 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.508+0000 7fd3793915c0 20 datascan.forall_objects: OSD matched oid 100000001fa.00000000 2023-09-23T03:26:32.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.508+0000 7fd3793915c0 10 datascan.operator(): handling object 100000001fa.0 2023-09-23T03:26:32.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.509+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:155 15.3f 15:feb1c803:::100000001fa.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57fe380 con 0x5649a57c4000 2023-09-23T03:26:32.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.510+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 38 ==== osd_op_reply(155 100000001fa.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (111) out=111b,getxattr,getxattr] v0'0 uv18 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.510+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:156 16.1f 16:feb1c803:::100000001fa.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57ff500 con 0x5649a57c5000 2023-09-23T03:26:32.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.511+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.5 v2:172.21.15.190:6800/882536885 9 ==== osd_op_reply(156 100000001fa.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x5649a609ed80 con 0x5649a57c5000 2023-09-23T03:26:32.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.511+0000 7fd3793915c0 20 datascan.operator(): loaded layout from xattr: pi: 16 os: 262144 sc: 4 su: 65536 2023-09-23T03:26:32.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.511+0000 7fd3793915c0 10 datascan.operator(): calculating striped size from complete objs: 4, partial objs: 4 2023-09-23T03:26:32.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.511+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:157 16.1c 16:3c0114ac:::100000001fa.00000004:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57fea80 con 0x5649a56a3400 2023-09-23T03:26:32.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.512+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.2 v2:172.21.15.154:6822/2884407243 24 ==== osd_op_reply(157 100000001fa.00000004 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5649a609efc0 con 0x5649a56a3400 2023-09-23T03:26:32.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.512+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:158 16.3 16:c0957981:::100000001fa.00000005:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57fe000 con 0x5649a57c5000 2023-09-23T03:26:32.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.513+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.5 v2:172.21.15.190:6800/882536885 10 ==== osd_op_reply(158 100000001fa.00000005 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5649a609ed80 con 0x5649a57c5000 2023-09-23T03:26:32.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.513+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:159 16.a 16:56c65d28:::100000001fa.00000006:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b6380 con 0x5649a56a3400 2023-09-23T03:26:32.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.514+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 <== osd.2 v2:172.21.15.154:6822/2884407243 25 ==== osd_op_reply(159 100000001fa.00000006 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5649a609efc0 con 0x5649a56a3400 2023-09-23T03:26:32.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.514+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:160 16.1f 16:fb032386:::100000001fa.00000007:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b6000 con 0x5649a57c5000 2023-09-23T03:26:32.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.515+0000 7fd37833a640 1 -- 172.21.15.154:0/361050481 <== osd.5 v2:172.21.15.190:6800/882536885 11 ==== osd_op_reply(160 100000001fa.00000007 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5649a609ed80 con 0x5649a57c5000 2023-09-23T03:26:32.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.515+0000 7fd3793915c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001fa 2023-09-23T03:26:32.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.515+0000 7fd3793915c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/flushed_file_5 2023-09-23T03:26:32.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.515+0000 7fd3793915c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=flushed_file_5 2023-09-23T03:26:32.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.516+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:161 14.10 14:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b7c00 con 0x5649a57c4000 2023-09-23T03:26:32.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.516+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 39 ==== osd_op_reply(161 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.516+0000 7fd3793915c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-09-23T03:26:32.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.516+0000 7fd3793915c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-09-23T03:26:32.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.516+0000 7fd3793915c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/flushed_file_5, will insert to root 2023-09-23T03:26:32.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.516+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:162 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b6700 con 0x5649a57c4000 2023-09-23T03:26:32.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.517+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 40 ==== osd_op_reply(162 10000000000.00000000 [omap-get-header out=274b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.517+0000 7fd3793915c0 10 datascan.find_or_create_dirfrag: read_version = 22 2023-09-23T03:26:32.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.517+0000 7fd3793915c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-09-23T03:26:32.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.517+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:163 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b7180 con 0x5649a57c4000 2023-09-23T03:26:32.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.518+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 41 ==== osd_op_reply(163 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.518+0000 7fd3793915c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_5 frag=*, r=0 2023-09-23T03:26:32.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.518+0000 7fd3793915c0 20 datascan.read_dentry: flushed_file_5_head not found in result 2023-09-23T03:26:32.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.518+0000 7fd3793915c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fa at 0x0x10000000000/flushed_file_5 with size=1441793 bytes 2023-09-23T03:26:32.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.518+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:164 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x5649a57b6a80 con 0x5649a57c4000 2023-09-23T03:26:32.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.521+0000 7fd377338640 1 -- 172.21.15.154:0/361050481 <== osd.1 v2:172.21.15.154:6816/1267710005 42 ==== osd_op_reply(164 10000000000.00000000 [omap-set-vals] v110'23 uv23 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5649a609eb40 con 0x5649a57c4000 2023-09-23T03:26:32.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.522+0000 7fd3793915c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_5 pointing to 0x0x100000001fa 2023-09-23T03:26:32.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.522+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5649a56a3000 msgr2=0x5649a53b5b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.522+0000 7fd3793915c0 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x5649a56a3000 0x5649a53b5b80 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.522+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5649a57c4000 msgr2=0x5649a57ca000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.522+0000 7fd3793915c0 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5649a57c4000 0x5649a57ca000 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.522+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5649a56a3400 msgr2=0x5649a53b5600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.522+0000 7fd3793915c0 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x5649a56a3400 0x5649a53b5600 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.522+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5649a57c4c00 msgr2=0x5649a57cb080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.522+0000 7fd3793915c0 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x5649a57c4c00 0x5649a57cb080 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.522+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5649a56a2c00 msgr2=0x5649a53b6100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.522+0000 7fd3793915c0 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x5649a56a2c00 0x5649a53b6100 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.522+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x5649a57c5000 msgr2=0x5649a57cb600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.522+0000 7fd3793915c0 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x5649a57c5000 0x5649a57cb600 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.522+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 reap_dead start 2023-09-23T03:26:32.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.522+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x5649a57c4800 msgr2=0x5649a57cab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.522+0000 7fd3793915c0 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x5649a57c4800 0x5649a57cab00 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.522+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x5649a57c4400 msgr2=0x5649a57ca580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.522+0000 7fd3793915c0 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x5649a57c4400 0x5649a57ca580 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.522+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5649a56a2800 msgr2=0x5649a53b7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.522+0000 7fd3793915c0 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x5649a56a2800 0x5649a53b7700 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x5649a5644810 tx=0x5649a5648c00 comp rx=0 tx=0).stop 2023-09-23T03:26:32.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.522+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649a5639400 msgr2=0x5649a53b6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.522+0000 7fd3793915c0 1 --2- 172.21.15.154:0/361050481 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649a5639400 0x5649a53b6c00 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x5649a6152a50 tx=0x5649a56281e0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.522+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 reap_dead start 2023-09-23T03:26:32.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.523+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 shutdown_connections 2023-09-23T03:26:32.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.523+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 >> 172.21.15.154:0/361050481 conn(0x5649a5638c00 msgr2=0x5649a60c5000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:32.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.523+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 reap_dead start 2023-09-23T03:26:32.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.523+0000 7fd377b39640 1 -- 172.21.15.154:0/361050481 reap_dead start 2023-09-23T03:26:32.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.523+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 shutdown_connections 2023-09-23T03:26:32.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.523+0000 7fd3793915c0 1 -- 172.21.15.154:0/361050481 wait complete. 2023-09-23T03:26:32.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.524+0000 7fd3793915c0 1 -- 172.21.15.154:0/980133284 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649a534bc00 msgr2=0x5649a53b5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.524+0000 7fd3793915c0 1 --2- 172.21.15.154:0/980133284 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649a534bc00 0x5649a53b5080 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x5649a61533e0 tx=0x5649a5628420 comp rx=0 tx=0).stop 2023-09-23T03:26:32.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.525+0000 7fd3793915c0 1 -- 172.21.15.154:0/980133284 shutdown_connections 2023-09-23T03:26:32.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.525+0000 7fd3793915c0 1 --2- 172.21.15.154:0/980133284 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5649a534bc00 0x5649a53b5080 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.525+0000 7fd3793915c0 1 --2- 172.21.15.154:0/980133284 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5649a534b800 0x5649a53b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.525+0000 7fd3793915c0 1 --2- 172.21.15.154:0/980133284 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5649a534ac00 0x5649a53b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.525+0000 7fd3793915c0 1 -- 172.21.15.154:0/980133284 >> 172.21.15.154:0/980133284 conn(0x5649a534b000 msgr2=0x5649a60c4800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:32.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.525+0000 7fd3793915c0 1 -- 172.21.15.154:0/980133284 shutdown_connections 2023-09-23T03:26:32.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.525+0000 7fd3793915c0 1 -- 172.21.15.154:0/980133284 wait complete. 2023-09-23T03:26:32.532 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes'] in time 0:00:00.328286, result: 2023-09-23T03:26:32.532 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2023-09-23T03:26:32.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.637+0000 7fa7514355c0 1 -- start start 2023-09-23T03:26:32.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.637+0000 7fa7514355c0 1 Processor -- start 2023-09-23T03:26:32.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.638+0000 7fa7514355c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5605077ee400 0x560506a8d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:32.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.638+0000 7fa7514355c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5605077efc00 0x560506a8cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:32.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.638+0000 7fa7514355c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560506d0a000 0x560506a8db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:32.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.638+0000 7fa7514355c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x560506a6d180 con 0x5605077efc00 2023-09-23T03:26:32.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.638+0000 7fa7514355c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560506af0dc0 con 0x560506d0a000 2023-09-23T03:26:32.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.638+0000 7fa7514355c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x560506a6c140 con 0x5605077ee400 2023-09-23T03:26:32.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.638+0000 7fa74eba9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5605077efc00 0x560506a8cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:32.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.638+0000 7fa74eba9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5605077efc00 0x560506a8cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46130/0 (socket says 172.21.15.154:46130) 2023-09-23T03:26:32.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.638+0000 7fa74eba9640 1 -- 172.21.15.154:0/1139200614 learned_addr learned my addr 172.21.15.154:0/1139200614 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:32.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.639+0000 7fa74eba9640 1 -- 172.21.15.154:0/1139200614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5605077ee400 msgr2=0x560506a8d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:32.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.639+0000 7fa74eba9640 1 --2- 172.21.15.154:0/1139200614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5605077ee400 0x560506a8d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.639+0000 7fa74eba9640 1 -- 172.21.15.154:0/1139200614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560506d0a000 msgr2=0x560506a8db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:32.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.639+0000 7fa74eba9640 1 --2- 172.21.15.154:0/1139200614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560506d0a000 0x560506a8db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.639+0000 7fa74eba9640 1 -- 172.21.15.154:0/1139200614 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560506a56000 con 0x5605077efc00 2023-09-23T03:26:32.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.639+0000 7fa74eba9640 1 --2- 172.21.15.154:0/1139200614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5605077efc00 0x560506a8cb00 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x560507834750 tx=0x560506d14120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45864b91d17e9a78 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:32.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.639+0000 7fa74cba5640 1 -- 172.21.15.154:0/1139200614 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560506a6b080 con 0x5605077efc00 2023-09-23T03:26:32.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.639+0000 7fa74cba5640 1 -- 172.21.15.154:0/1139200614 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5605077f0000 con 0x5605077efc00 2023-09-23T03:26:32.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.639+0000 7fa74cba5640 1 -- 172.21.15.154:0/1139200614 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560506a6b600 con 0x5605077efc00 2023-09-23T03:26:32.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.639+0000 7fa7514355c0 1 -- 172.21.15.154:0/1139200614 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560506a56340 con 0x5605077efc00 2023-09-23T03:26:32.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.639+0000 7fa7514355c0 1 -- 172.21.15.154:0/1139200614 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x560506a571e0 con 0x5605077efc00 2023-09-23T03:26:32.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.639+0000 7fa7514355c0 4 waiting for MDS map... 2023-09-23T03:26:32.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.639+0000 7fa74cba5640 1 -- 172.21.15.154:0/1139200614 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(110..110 src has 1..110) v4 ==== 6605+0+0 (secure 0 0 0) 0x560506ae2c40 con 0x5605077efc00 2023-09-23T03:26:32.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.640+0000 7fa74cba5640 1 -- 172.21.15.154:0/1139200614 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 91) v1 ==== 3306+0+0 (secure 0 0 0) 0x560506a75200 con 0x5605077efc00 2023-09-23T03:26:32.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.640+0000 7fa7514355c0 4 Got MDS map 91 2023-09-23T03:26:32.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.640+0000 7fa7514355c0 4 datascan.main: Using metadata pool output 2023-09-23T03:26:32.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.640+0000 7fa7514355c0 4 datascan.main: connecting to RADOS... 2023-09-23T03:26:32.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.641+0000 7fa7514355c0 1 Processor -- start 2023-09-23T03:26:32.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.641+0000 7fa7514355c0 1 -- start start 2023-09-23T03:26:32.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.641+0000 7fa7514355c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560506d0a800 0x560506a8c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:32.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.642+0000 7fa7514355c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560506d0ac00 0x560506a8e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:32.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.642+0000 7fa7514355c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560506d0b000 0x560506a8e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:32.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.642+0000 7fa7514355c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x560506a6cb40 con 0x560506d0a800 2023-09-23T03:26:32.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.642+0000 7fa7514355c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560506a6ca00 con 0x560506d0ac00 2023-09-23T03:26:32.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.642+0000 7fa7514355c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x560506af0dc0 con 0x560506d0b000 2023-09-23T03:26:32.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.642+0000 7fa74fbab640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560506d0a800 0x560506a8c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:32.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.642+0000 7fa74fbab640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560506d0a800 0x560506a8c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46138/0 (socket says 172.21.15.154:46138) 2023-09-23T03:26:32.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.642+0000 7fa74fbab640 1 -- 172.21.15.154:0/1496822798 learned_addr learned my addr 172.21.15.154:0/1496822798 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:32.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.642+0000 7fa74f3aa640 1 --2- 172.21.15.154:0/1496822798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560506d0ac00 0x560506a8e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:32.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.642+0000 7fa74fbab640 1 -- 172.21.15.154:0/1496822798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560506d0b000 msgr2=0x560506a8e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:32.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.642+0000 7fa74eba9640 1 --2- 172.21.15.154:0/1496822798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560506d0b000 0x560506a8e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:32.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.642+0000 7fa74fbab640 1 --2- 172.21.15.154:0/1496822798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560506d0b000 0x560506a8e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.642+0000 7fa74fbab640 1 -- 172.21.15.154:0/1496822798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560506d0ac00 msgr2=0x560506a8e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:32.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.642+0000 7fa74fbab640 1 --2- 172.21.15.154:0/1496822798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560506d0ac00 0x560506a8e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.642+0000 7fa74fbab640 1 -- 172.21.15.154:0/1496822798 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560506a569c0 con 0x560506d0a800 2023-09-23T03:26:32.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.642+0000 7fa74f3aa640 1 --2- 172.21.15.154:0/1496822798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560506d0ac00 0x560506a8e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:26:32.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.643+0000 7fa74fbab640 1 --2- 172.21.15.154:0/1496822798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560506d0a800 0x560506a8c580 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x560506d22840 tx=0x560506d2ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=704aef394e154e8d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:32.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.643+0000 7fa74aba1640 1 -- 172.21.15.154:0/1496822798 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560506a6b080 con 0x560506d0a800 2023-09-23T03:26:32.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.643+0000 7fa74aba1640 1 -- 172.21.15.154:0/1496822798 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5605077f1680 con 0x560506d0a800 2023-09-23T03:26:32.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.643+0000 7fa74aba1640 1 -- 172.21.15.154:0/1496822798 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560506a6b600 con 0x560506d0a800 2023-09-23T03:26:32.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.643+0000 7fa7514355c0 1 -- 172.21.15.154:0/1496822798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560506d0a800 msgr2=0x560506a8c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.643+0000 7fa7514355c0 1 --2- 172.21.15.154:0/1496822798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560506d0a800 0x560506a8c580 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x560506d22840 tx=0x560506d2ecc0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.644+0000 7fa7514355c0 1 -- 172.21.15.154:0/1496822798 shutdown_connections 2023-09-23T03:26:32.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.644+0000 7fa7514355c0 1 --2- 172.21.15.154:0/1496822798 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560506d0b000 0x560506a8e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.644+0000 7fa7514355c0 1 --2- 172.21.15.154:0/1496822798 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560506d0ac00 0x560506a8e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.644+0000 7fa7514355c0 1 --2- 172.21.15.154:0/1496822798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560506d0a800 0x560506a8c580 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.644+0000 7fa7514355c0 1 -- 172.21.15.154:0/1496822798 >> 172.21.15.154:0/1496822798 conn(0x560506d0a400 msgr2=0x56050779d000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:32.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.644+0000 7fa7514355c0 1 -- 172.21.15.154:0/1496822798 shutdown_connections 2023-09-23T03:26:32.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.644+0000 7fa7514355c0 1 -- 172.21.15.154:0/1496822798 wait complete. 2023-09-23T03:26:32.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.644+0000 7fa7514355c0 1 Processor -- start 2023-09-23T03:26:32.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.645+0000 7fa7514355c0 1 -- start start 2023-09-23T03:26:32.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.645+0000 7fa7514355c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560506d0a800 0x560506a8f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:32.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.645+0000 7fa7514355c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560506d0ac00 0x560506a8f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:32.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.645+0000 7fa7514355c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560506d0b000 0x560506a8ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:32.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.645+0000 7fa7514355c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x560506af1180 con 0x560506d0b000 2023-09-23T03:26:32.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.645+0000 7fa7514355c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560506af0c80 con 0x560506d0a800 2023-09-23T03:26:32.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.645+0000 7fa7514355c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x560506a6c140 con 0x560506d0ac00 2023-09-23T03:26:32.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.646+0000 7fa74eba9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560506d0b000 0x560506a8ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:32.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.646+0000 7fa74eba9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560506d0b000 0x560506a8ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46152/0 (socket says 172.21.15.154:46152) 2023-09-23T03:26:32.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.646+0000 7fa74eba9640 1 -- 172.21.15.154:0/2089902380 learned_addr learned my addr 172.21.15.154:0/2089902380 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:32.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.646+0000 7fa74fbab640 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560506d0a800 0x560506a8f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:32.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.646+0000 7fa74f3aa640 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560506d0ac00 0x560506a8f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:32.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.646+0000 7fa74eba9640 1 -- 172.21.15.154:0/2089902380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560506d0ac00 msgr2=0x560506a8f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.646+0000 7fa74eba9640 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560506d0ac00 0x560506a8f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.646+0000 7fa74eba9640 1 -- 172.21.15.154:0/2089902380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560506d0a800 msgr2=0x560506a8f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.646+0000 7fa74eba9640 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560506d0a800 0x560506a8f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.646+0000 7fa74eba9640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560506a56000 con 0x560506d0b000 2023-09-23T03:26:32.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.646+0000 7fa74eba9640 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560506d0b000 0x560506a8ec00 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x560507835650 tx=0x560506d141e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:32.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.647+0000 7fa74939e640 1 -- 172.21.15.154:0/2089902380 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560506a6b1e0 con 0x560506d0b000 2023-09-23T03:26:32.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.647+0000 7fa74939e640 1 -- 172.21.15.154:0/2089902380 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5605077f0c00 con 0x560506d0b000 2023-09-23T03:26:32.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.647+0000 7fa74939e640 1 -- 172.21.15.154:0/2089902380 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x560506a6b080 con 0x560506d0b000 2023-09-23T03:26:32.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.647+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560506a56ea0 con 0x560506d0b000 2023-09-23T03:26:32.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.647+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560506a57520 con 0x560506d0b000 2023-09-23T03:26:32.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.647+0000 7fa7514355c0 4 datascan.init: resolving metadata pool 14 2023-09-23T03:26:32.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.649+0000 7fa74939e640 1 -- 172.21.15.154:0/2089902380 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x560506a55080 con 0x560506d0b000 2023-09-23T03:26:32.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.649+0000 7fa74939e640 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x560506e72000 0x560506e78000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:32.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.650+0000 7fa74939e640 1 -- 172.21.15.154:0/2089902380 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6605+0+0 (secure 0 0 0) 0x560506ae2a80 con 0x560506d0b000 2023-09-23T03:26:32.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.650+0000 7fa7514355c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-09-23T03:26:32.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.650+0000 7fa7514355c0 4 datascan.main: resolving metadata pool 14 2023-09-23T03:26:32.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.650+0000 7fa7514355c0 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x560506e72400 0x560506a8e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:32.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.650+0000 7fa74fbab640 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x560506e72000 0x560506e78000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:32.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.650+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:1 14.0 14:00000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7d880 con 0x560506e72400 2023-09-23T03:26:32.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.650+0000 7fa74f3aa640 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x560506e72400 0x560506a8e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:32.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.650+0000 7fa74fbab640 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x560506e72000 0x560506e78000 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x560506d22840 tx=0x560506d2ec00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:32.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.651+0000 7fa74f3aa640 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x560506e72400 0x560506a8e680 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:32.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.651+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.2 v2:172.21.15.154:6822/2884407243 1 ==== osd_op_reply(1 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04b40 con 0x560506e72400 2023-09-23T03:26:32.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.651+0000 7fa74f3aa640 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5605077ee000 0x560506a8d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:32.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.651+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:2 14.10 14:08000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7d880 con 0x5605077ee000 2023-09-23T03:26:32.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.652+0000 7fa74fbab640 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5605077ee000 0x560506a8d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:32.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.652+0000 7fa74fbab640 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5605077ee000 0x560506a8d080 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:32.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.652+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 <== osd.1 v2:172.21.15.154:6816/1267710005 1 ==== osd_op_reply(2 [pgnls start_epoch 110 out=81b] v110'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x560506d04d80 con 0x5605077ee000 2023-09-23T03:26:32.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.652+0000 7fa74fbab640 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x560506e73c00 0x560506e78580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:32.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.652+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:3 14.8 14:10000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7d880 con 0x560506e73c00 2023-09-23T03:26:32.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.653+0000 7fa74f3aa640 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x560506e73c00 0x560506e78580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:32.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.653+0000 7fa74f3aa640 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x560506e73c00 0x560506e78580 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:32.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.654+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.0 v2:172.21.15.154:6800/2920938138 1 ==== osd_op_reply(3 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04b40 con 0x560506e73c00 2023-09-23T03:26:32.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.654+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:4 14.18 14:18000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7d880 con 0x5605077ee000 2023-09-23T03:26:32.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.654+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 <== osd.1 v2:172.21.15.154:6816/1267710005 2 ==== osd_op_reply(4 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04d80 con 0x5605077ee000 2023-09-23T03:26:32.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.654+0000 7fa74fbab640 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x560506e88000 0x560506e78b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:32.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.654+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:5 14.4 14:20000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7d880 con 0x560506e88000 2023-09-23T03:26:32.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.655+0000 7fa74eba9640 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x560506e88000 0x560506e78b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:32.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.656+0000 7fa74eba9640 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x560506e88000 0x560506e78b00 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:32.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.656+0000 7fa74eba9640 1 -- 172.21.15.154:0/2089902380 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== osd_op_reply(5 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04fc0 con 0x560506e88000 2023-09-23T03:26:32.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.656+0000 7fa74eba9640 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x560506d0bc00 0x560506e79080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:32.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.656+0000 7fa74eba9640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:6 14.14 14:28000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7d880 con 0x560506d0bc00 2023-09-23T03:26:32.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.657+0000 7fa74fbab640 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x560506d0bc00 0x560506e79080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:32.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.658+0000 7fa74fbab640 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x560506d0bc00 0x560506e79080 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:32.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.658+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 <== osd.4 v2:172.21.15.190:6808/717302641 1 ==== osd_op_reply(6 [pgnls start_epoch 110 out=125b] v110'50 uv50 ondisk = 1) v8 ==== 144+0+125 (crc 0 0 0) 0x560506d04d80 con 0x560506d0bc00 2023-09-23T03:26:32.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.658+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:7 14.c 14:30000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7d880 con 0x560506e73c00 2023-09-23T03:26:32.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.659+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.0 v2:172.21.15.154:6800/2920938138 2 ==== osd_op_reply(7 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04b40 con 0x560506e73c00 2023-09-23T03:26:32.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.659+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:8 14.1c 14:38000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7d880 con 0x560506e73c00 2023-09-23T03:26:32.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.659+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.0 v2:172.21.15.154:6800/2920938138 3 ==== osd_op_reply(8 [pgnls start_epoch 110 out=49b] v109'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04b40 con 0x560506e73c00 2023-09-23T03:26:32.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.659+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:9 14.2 14:40000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7d880 con 0x560506e88000 2023-09-23T03:26:32.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.660+0000 7fa74eba9640 1 -- 172.21.15.154:0/2089902380 <== osd.6 v2:172.21.15.190:6816/1964294001 2 ==== osd_op_reply(9 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04fc0 con 0x560506e88000 2023-09-23T03:26:32.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.660+0000 7fa74eba9640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:10 14.12 14:48000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7d880 con 0x560506e72400 2023-09-23T03:26:32.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.660+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.2 v2:172.21.15.154:6822/2884407243 2 ==== osd_op_reply(10 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04b40 con 0x560506e72400 2023-09-23T03:26:32.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.660+0000 7fa74f3aa640 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x560506e88c00 0x560506e79600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:32.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.660+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:11 14.a 14:50000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7d880 con 0x560506e88c00 2023-09-23T03:26:32.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.661+0000 7fa74f3aa640 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x560506e88c00 0x560506e79600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:32.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.662+0000 7fa74f3aa640 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x560506e88c00 0x560506e79600 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:32.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.663+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.5 v2:172.21.15.190:6800/882536885 1 ==== osd_op_reply(11 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04b40 con 0x560506e88c00 2023-09-23T03:26:32.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.663+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:12 14.1a 14:58000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7d880 con 0x560506e73c00 2023-09-23T03:26:32.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.664+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.0 v2:172.21.15.154:6800/2920938138 4 ==== osd_op_reply(12 [pgnls start_epoch 110 out=49b] v110'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04b40 con 0x560506e73c00 2023-09-23T03:26:32.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.664+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:13 14.6 14:60000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7d880 con 0x560506e73c00 2023-09-23T03:26:32.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.664+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.0 v2:172.21.15.154:6800/2920938138 5 ==== osd_op_reply(13 [pgnls start_epoch 110 out=49b] v109'8 uv8 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04b40 con 0x560506e73c00 2023-09-23T03:26:32.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.664+0000 7fa74f3aa640 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x560506e89000 0x560506e79b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:32.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.664+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:14 14.16 14:68000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7d880 con 0x560506e89000 2023-09-23T03:26:32.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.665+0000 7fa74eba9640 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x560506e89000 0x560506e79b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:32.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.665+0000 7fa74eba9640 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x560506e89000 0x560506e79b80 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:32.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.666+0000 7fa74eba9640 1 -- 172.21.15.154:0/2089902380 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(14 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04fc0 con 0x560506e89000 2023-09-23T03:26:32.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.666+0000 7fa74eba9640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:15 14.e 14:70000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7d880 con 0x560506d0bc00 2023-09-23T03:26:32.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.667+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 <== osd.4 v2:172.21.15.190:6808/717302641 2 ==== osd_op_reply(15 [pgnls start_epoch 110 out=49b] v109'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04d80 con 0x560506d0bc00 2023-09-23T03:26:32.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.667+0000 7fa74fbab640 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x560506e88400 0x560506e7a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:32.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.667+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:16 14.1e 14:78000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7d880 con 0x560506e88400 2023-09-23T03:26:32.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.667+0000 7fa74fbab640 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x560506e88400 0x560506e7a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:32.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.668+0000 7fa74fbab640 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x560506e88400 0x560506e7a680 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:32.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.669+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 <== osd.7 v2:172.21.15.190:6824/3460136272 1 ==== osd_op_reply(16 [pgnls start_epoch 110 out=49b] v110'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04d80 con 0x560506e88400 2023-09-23T03:26:32.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.669+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:17 14.1 14:80000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7d880 con 0x560506e89000 2023-09-23T03:26:32.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.670+0000 7fa74eba9640 1 -- 172.21.15.154:0/2089902380 <== osd.3 v2:172.21.15.154:6808/1133688468 2 ==== osd_op_reply(17 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04fc0 con 0x560506e89000 2023-09-23T03:26:32.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.670+0000 7fa74eba9640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:18 14.11 14:88000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7d880 con 0x560506d0bc00 2023-09-23T03:26:32.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.671+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 <== osd.4 v2:172.21.15.190:6808/717302641 3 ==== osd_op_reply(18 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04d80 con 0x560506d0bc00 2023-09-23T03:26:32.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.671+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:19 14.9 14:90000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7d880 con 0x560506e88c00 2023-09-23T03:26:32.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.672+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.5 v2:172.21.15.190:6800/882536885 2 ==== osd_op_reply(19 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04b40 con 0x560506e88c00 2023-09-23T03:26:32.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.672+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:20 14.19 14:98000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7d880 con 0x560506e88000 2023-09-23T03:26:32.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.673+0000 7fa74eba9640 1 -- 172.21.15.154:0/2089902380 <== osd.6 v2:172.21.15.190:6816/1964294001 3 ==== osd_op_reply(20 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04fc0 con 0x560506e88000 2023-09-23T03:26:32.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.673+0000 7fa74eba9640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:21 14.5 14:a0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7d880 con 0x560506e73c00 2023-09-23T03:26:32.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.674+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.0 v2:172.21.15.154:6800/2920938138 6 ==== osd_op_reply(21 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04b40 con 0x560506e73c00 2023-09-23T03:26:32.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.674+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:22 14.15 14:a8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7d880 con 0x560506e88400 2023-09-23T03:26:32.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.675+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 <== osd.7 v2:172.21.15.190:6824/3460136272 2 ==== osd_op_reply(22 [pgnls start_epoch 110 out=73b] v109'6 uv6 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x560506d04d80 con 0x560506e88400 2023-09-23T03:26:32.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.675+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:23 14.d 14:b0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7d880 con 0x560506e88000 2023-09-23T03:26:32.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.676+0000 7fa74eba9640 1 -- 172.21.15.154:0/2089902380 <== osd.6 v2:172.21.15.190:6816/1964294001 4 ==== osd_op_reply(23 [pgnls start_epoch 110 out=74b] v110'3 uv3 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x560506d04fc0 con 0x560506e88000 2023-09-23T03:26:32.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.676+0000 7fa74eba9640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:24 14.1d 14:b8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7d880 con 0x560506e73c00 2023-09-23T03:26:32.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.676+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.0 v2:172.21.15.154:6800/2920938138 7 ==== osd_op_reply(24 [pgnls start_epoch 110 out=49b] v109'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04b40 con 0x560506e73c00 2023-09-23T03:26:32.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.676+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:25 14.3 14:c0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7d880 con 0x560506e73c00 2023-09-23T03:26:32.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.677+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.0 v2:172.21.15.154:6800/2920938138 8 ==== osd_op_reply(25 [pgnls start_epoch 110 out=71b] v110'2 uv2 ondisk = 1) v8 ==== 144+0+71 (crc 0 0 0) 0x560506d04b40 con 0x560506e73c00 2023-09-23T03:26:32.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.677+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:26 14.13 14:c8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7d880 con 0x560506e72400 2023-09-23T03:26:32.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.677+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.2 v2:172.21.15.154:6822/2884407243 3 ==== osd_op_reply(26 [pgnls start_epoch 110 out=98b] v110'7 uv7 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x560506d04b40 con 0x560506e72400 2023-09-23T03:26:32.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.677+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:27 14.b 14:d0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7d880 con 0x560506e73c00 2023-09-23T03:26:32.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.678+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.0 v2:172.21.15.154:6800/2920938138 9 ==== osd_op_reply(27 [pgnls start_epoch 110 out=76b] v110'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x560506d04b40 con 0x560506e73c00 2023-09-23T03:26:32.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.678+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:28 14.1b 14:d8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7d880 con 0x5605077ee000 2023-09-23T03:26:32.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.678+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 <== osd.1 v2:172.21.15.154:6816/1267710005 3 ==== osd_op_reply(28 [pgnls start_epoch 110 out=49b] v110'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04d80 con 0x5605077ee000 2023-09-23T03:26:32.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.678+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:29 14.7 14:e0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7d880 con 0x560506e88c00 2023-09-23T03:26:32.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.679+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.5 v2:172.21.15.190:6800/882536885 3 ==== osd_op_reply(29 [pgnls start_epoch 110 out=49b] v109'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04b40 con 0x560506e88c00 2023-09-23T03:26:32.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.679+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:30 14.17 14:e8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7d880 con 0x5605077ee000 2023-09-23T03:26:32.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.680+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 <== osd.1 v2:172.21.15.154:6816/1267710005 4 ==== osd_op_reply(30 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04d80 con 0x5605077ee000 2023-09-23T03:26:32.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.680+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:31 14.f 14:f0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7d880 con 0x560506e88400 2023-09-23T03:26:32.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.681+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 <== osd.7 v2:172.21.15.190:6824/3460136272 3 ==== osd_op_reply(31 [pgnls start_epoch 110 out=73b] v110'4 uv4 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x560506d04d80 con 0x560506e88400 2023-09-23T03:26:32.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.681+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:32 14.1f 14:f8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7d880 con 0x560506e73c00 2023-09-23T03:26:32.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.681+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.0 v2:172.21.15.154:6800/2920938138 10 ==== osd_op_reply(32 [pgnls start_epoch 110 out=101b] v110'13 uv13 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x560506d04b40 con 0x560506e73c00 2023-09-23T03:26:32.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.682+0000 7fa7514355c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2023-09-23T03:26:32.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.682+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:33 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7dc00 con 0x5605077ee000 2023-09-23T03:26:32.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.682+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 <== osd.1 v2:172.21.15.154:6816/1267710005 5 ==== osd_op_reply(33 10000000000.00000000 [omap-get-vals out=3582b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+3582 (crc 0 0 0) 0x560506d04d80 con 0x5605077ee000 2023-09-23T03:26:32.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.682+0000 7fa7514355c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:26:32.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.682+0000 7fa7514355c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:26:32.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.682+0000 7fa7514355c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:26:32.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.682+0000 7fa7514355c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:26:32.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.682+0000 7fa7514355c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:26:32.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.682+0000 7fa7514355c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:26:32.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.682+0000 7fa7514355c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:26:32.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.682+0000 7fa7514355c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2023-09-23T03:26:32.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.682+0000 7fa7514355c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-09-23T03:26:32.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.682+0000 7fa7514355c0 10 datascan.scan_links: step 1: handling object 200.00000000 2023-09-23T03:26:32.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.682+0000 7fa7514355c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-09-23T03:26:32.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.682+0000 7fa7514355c0 10 datascan.scan_links: step 1: handling object 200.00000001 2023-09-23T03:26:32.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.682+0000 7fa7514355c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-09-23T03:26:32.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.682+0000 7fa7514355c0 10 datascan.scan_links: step 1: handling object 500.00000000 2023-09-23T03:26:32.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.682+0000 7fa7514355c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-09-23T03:26:32.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.682+0000 7fa7514355c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2023-09-23T03:26:32.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.682+0000 7fa7514355c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-09-23T03:26:32.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.682+0000 7fa7514355c0 10 datascan.scan_links: step 1: handling object 4.00000000 2023-09-23T03:26:32.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.683+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:34 14.3 14:c0ce80d4:::4.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506eae000 con 0x560506e73c00 2023-09-23T03:26:32.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.683+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.0 v2:172.21.15.154:6800/2920938138 11 ==== osd_op_reply(34 4.00000000 [omap-get-vals out=513b] v0'0 uv2 ondisk = 0) v8 ==== 154+0+513 (crc 0 0 0) 0x560506d04b40 con 0x560506e73c00 2023-09-23T03:26:32.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.683+0000 7fa7514355c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:26:32.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.683+0000 7fa7514355c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2023-09-23T03:26:32.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.683+0000 7fa7514355c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-09-23T03:26:32.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.683+0000 7fa7514355c0 10 datascan.scan_links: step 1: handling object 100.00000000 2023-09-23T03:26:32.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.683+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:35 14.13 14:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506eae380 con 0x560506e72400 2023-09-23T03:26:32.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.684+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.2 v2:172.21.15.154:6822/2884407243 4 ==== osd_op_reply(35 100.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x560506d04b40 con 0x560506e72400 2023-09-23T03:26:32.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.684+0000 7fa7514355c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2023-09-23T03:26:32.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.684+0000 7fa7514355c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-09-23T03:26:32.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.684+0000 7fa7514355c0 10 datascan.scan_links: step 1: handling object 400.00000000 2023-09-23T03:26:32.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.684+0000 7fa7514355c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-09-23T03:26:32.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.684+0000 7fa7514355c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2023-09-23T03:26:32.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.684+0000 7fa7514355c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-09-23T03:26:32.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.684+0000 7fa7514355c0 10 datascan.scan_links: step 1: handling object 1.00000000 2023-09-23T03:26:32.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.684+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:36 14.1f 14:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506eae700 con 0x560506e73c00 2023-09-23T03:26:32.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.684+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.0 v2:172.21.15.154:6800/2920938138 12 ==== osd_op_reply(36 1.00000000 [omap-get-vals out=1016b] v0'0 uv13 ondisk = 0) v8 ==== 154+0+1016 (crc 0 0 0) 0x560506d04b40 con 0x560506e73c00 2023-09-23T03:26:32.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.684+0000 7fa7514355c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:26:32.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.684+0000 7fa7514355c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:26:32.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.684+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:37 14.0 14:00000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506eaea80 con 0x560506e72400 2023-09-23T03:26:32.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.685+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.2 v2:172.21.15.154:6822/2884407243 5 ==== osd_op_reply(37 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04b40 con 0x560506e72400 2023-09-23T03:26:32.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.685+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:38 14.10 14:08000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7d880 con 0x5605077ee000 2023-09-23T03:26:32.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.685+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 <== osd.1 v2:172.21.15.154:6816/1267710005 6 ==== osd_op_reply(38 [pgnls start_epoch 110 out=81b] v110'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x560506d04d80 con 0x5605077ee000 2023-09-23T03:26:32.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.685+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:39 14.8 14:10000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7dc00 con 0x560506e73c00 2023-09-23T03:26:32.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.686+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.0 v2:172.21.15.154:6800/2920938138 13 ==== osd_op_reply(39 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04b40 con 0x560506e73c00 2023-09-23T03:26:32.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.686+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:40 14.18 14:18000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7dc00 con 0x5605077ee000 2023-09-23T03:26:32.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.686+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 <== osd.1 v2:172.21.15.154:6816/1267710005 7 ==== osd_op_reply(40 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04d80 con 0x5605077ee000 2023-09-23T03:26:32.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.686+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:41 14.4 14:20000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7dc00 con 0x560506e88000 2023-09-23T03:26:32.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.687+0000 7fa74eba9640 1 -- 172.21.15.154:0/2089902380 <== osd.6 v2:172.21.15.190:6816/1964294001 5 ==== osd_op_reply(41 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04fc0 con 0x560506e88000 2023-09-23T03:26:32.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.687+0000 7fa74eba9640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:42 14.14 14:28000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7dc00 con 0x560506d0bc00 2023-09-23T03:26:32.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.689+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 <== osd.4 v2:172.21.15.190:6808/717302641 4 ==== osd_op_reply(42 [pgnls start_epoch 110 out=125b] v110'50 uv50 ondisk = 1) v8 ==== 144+0+125 (crc 0 0 0) 0x560506d04d80 con 0x560506d0bc00 2023-09-23T03:26:32.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.689+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:43 14.c 14:30000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7dc00 con 0x560506e73c00 2023-09-23T03:26:32.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.689+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.0 v2:172.21.15.154:6800/2920938138 14 ==== osd_op_reply(43 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04b40 con 0x560506e73c00 2023-09-23T03:26:32.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.689+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:44 14.1c 14:38000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7dc00 con 0x560506e73c00 2023-09-23T03:26:32.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.690+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.0 v2:172.21.15.154:6800/2920938138 15 ==== osd_op_reply(44 [pgnls start_epoch 110 out=49b] v109'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04b40 con 0x560506e73c00 2023-09-23T03:26:32.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.690+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:45 14.2 14:40000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7dc00 con 0x560506e88000 2023-09-23T03:26:32.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.691+0000 7fa74eba9640 1 -- 172.21.15.154:0/2089902380 <== osd.6 v2:172.21.15.190:6816/1964294001 6 ==== osd_op_reply(45 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04fc0 con 0x560506e88000 2023-09-23T03:26:32.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.691+0000 7fa74eba9640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:46 14.12 14:48000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7dc00 con 0x560506e72400 2023-09-23T03:26:32.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.691+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.2 v2:172.21.15.154:6822/2884407243 6 ==== osd_op_reply(46 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04b40 con 0x560506e72400 2023-09-23T03:26:32.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.691+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:47 14.a 14:50000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7dc00 con 0x560506e88c00 2023-09-23T03:26:32.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.692+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.5 v2:172.21.15.190:6800/882536885 4 ==== osd_op_reply(47 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04b40 con 0x560506e88c00 2023-09-23T03:26:32.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.692+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:48 14.1a 14:58000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7dc00 con 0x560506e73c00 2023-09-23T03:26:32.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.693+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.0 v2:172.21.15.154:6800/2920938138 16 ==== osd_op_reply(48 [pgnls start_epoch 110 out=49b] v110'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04b40 con 0x560506e73c00 2023-09-23T03:26:32.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.693+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:49 14.6 14:60000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7dc00 con 0x560506e73c00 2023-09-23T03:26:32.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.693+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.0 v2:172.21.15.154:6800/2920938138 17 ==== osd_op_reply(49 [pgnls start_epoch 110 out=49b] v109'8 uv8 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04b40 con 0x560506e73c00 2023-09-23T03:26:32.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.693+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:50 14.16 14:68000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7dc00 con 0x560506e89000 2023-09-23T03:26:32.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.694+0000 7fa74eba9640 1 -- 172.21.15.154:0/2089902380 <== osd.3 v2:172.21.15.154:6808/1133688468 3 ==== osd_op_reply(50 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04fc0 con 0x560506e89000 2023-09-23T03:26:32.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.694+0000 7fa74eba9640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:51 14.e 14:70000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7dc00 con 0x560506d0bc00 2023-09-23T03:26:32.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.695+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 <== osd.4 v2:172.21.15.190:6808/717302641 5 ==== osd_op_reply(51 [pgnls start_epoch 110 out=49b] v109'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04d80 con 0x560506d0bc00 2023-09-23T03:26:32.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.695+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:52 14.1e 14:78000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7dc00 con 0x560506e88400 2023-09-23T03:26:32.696 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.696+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 <== osd.7 v2:172.21.15.190:6824/3460136272 4 ==== osd_op_reply(52 [pgnls start_epoch 110 out=49b] v110'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04d80 con 0x560506e88400 2023-09-23T03:26:32.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.696+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:53 14.1 14:80000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7dc00 con 0x560506e89000 2023-09-23T03:26:32.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.697+0000 7fa74eba9640 1 -- 172.21.15.154:0/2089902380 <== osd.3 v2:172.21.15.154:6808/1133688468 4 ==== osd_op_reply(53 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04fc0 con 0x560506e89000 2023-09-23T03:26:32.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.697+0000 7fa74eba9640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:54 14.11 14:88000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7dc00 con 0x560506d0bc00 2023-09-23T03:26:32.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.698+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 <== osd.4 v2:172.21.15.190:6808/717302641 6 ==== osd_op_reply(54 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04d80 con 0x560506d0bc00 2023-09-23T03:26:32.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.698+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:55 14.9 14:90000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7dc00 con 0x560506e88c00 2023-09-23T03:26:32.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.699+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.5 v2:172.21.15.190:6800/882536885 5 ==== osd_op_reply(55 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04b40 con 0x560506e88c00 2023-09-23T03:26:32.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.699+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:56 14.19 14:98000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7dc00 con 0x560506e88000 2023-09-23T03:26:32.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.701+0000 7fa74eba9640 1 -- 172.21.15.154:0/2089902380 <== osd.6 v2:172.21.15.190:6816/1964294001 7 ==== osd_op_reply(56 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04fc0 con 0x560506e88000 2023-09-23T03:26:32.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.701+0000 7fa74eba9640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:57 14.5 14:a0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7dc00 con 0x560506e73c00 2023-09-23T03:26:32.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.702+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.0 v2:172.21.15.154:6800/2920938138 18 ==== osd_op_reply(57 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04b40 con 0x560506e73c00 2023-09-23T03:26:32.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.702+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:58 14.15 14:a8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7dc00 con 0x560506e88400 2023-09-23T03:26:32.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.703+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 <== osd.7 v2:172.21.15.190:6824/3460136272 5 ==== osd_op_reply(58 [pgnls start_epoch 110 out=73b] v109'6 uv6 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x560506d04d80 con 0x560506e88400 2023-09-23T03:26:32.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.703+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:59 14.d 14:b0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7dc00 con 0x560506e88000 2023-09-23T03:26:32.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.704+0000 7fa74eba9640 1 -- 172.21.15.154:0/2089902380 <== osd.6 v2:172.21.15.190:6816/1964294001 8 ==== osd_op_reply(59 [pgnls start_epoch 110 out=74b] v110'3 uv3 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x560506d04fc0 con 0x560506e88000 2023-09-23T03:26:32.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.704+0000 7fa74eba9640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:60 14.1d 14:b8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7dc00 con 0x560506e73c00 2023-09-23T03:26:32.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.705+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.0 v2:172.21.15.154:6800/2920938138 19 ==== osd_op_reply(60 [pgnls start_epoch 110 out=49b] v109'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04b40 con 0x560506e73c00 2023-09-23T03:26:32.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.705+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:61 14.3 14:c0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7dc00 con 0x560506e73c00 2023-09-23T03:26:32.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.705+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.0 v2:172.21.15.154:6800/2920938138 20 ==== osd_op_reply(61 [pgnls start_epoch 110 out=71b] v110'2 uv2 ondisk = 1) v8 ==== 144+0+71 (crc 0 0 0) 0x560506d04b40 con 0x560506e73c00 2023-09-23T03:26:32.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.705+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:62 14.13 14:c8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7dc00 con 0x560506e72400 2023-09-23T03:26:32.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.706+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.2 v2:172.21.15.154:6822/2884407243 7 ==== osd_op_reply(62 [pgnls start_epoch 110 out=98b] v110'7 uv7 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x560506d04b40 con 0x560506e72400 2023-09-23T03:26:32.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.706+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:63 14.b 14:d0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7dc00 con 0x560506e73c00 2023-09-23T03:26:32.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.707+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.0 v2:172.21.15.154:6800/2920938138 21 ==== osd_op_reply(63 [pgnls start_epoch 110 out=76b] v110'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x560506d04b40 con 0x560506e73c00 2023-09-23T03:26:32.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.707+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:64 14.1b 14:d8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7dc00 con 0x5605077ee000 2023-09-23T03:26:32.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.708+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 <== osd.1 v2:172.21.15.154:6816/1267710005 8 ==== osd_op_reply(64 [pgnls start_epoch 110 out=49b] v110'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04d80 con 0x5605077ee000 2023-09-23T03:26:32.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.708+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:65 14.7 14:e0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7dc00 con 0x560506e88c00 2023-09-23T03:26:32.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.709+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.5 v2:172.21.15.190:6800/882536885 6 ==== osd_op_reply(65 [pgnls start_epoch 110 out=49b] v109'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04b40 con 0x560506e88c00 2023-09-23T03:26:32.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.709+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:66 14.17 14:e8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7dc00 con 0x5605077ee000 2023-09-23T03:26:32.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.710+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 <== osd.1 v2:172.21.15.154:6816/1267710005 9 ==== osd_op_reply(66 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560506d04d80 con 0x5605077ee000 2023-09-23T03:26:32.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.710+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:67 14.f 14:f0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7dc00 con 0x560506e88400 2023-09-23T03:26:32.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.711+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 <== osd.7 v2:172.21.15.190:6824/3460136272 6 ==== osd_op_reply(67 [pgnls start_epoch 110 out=73b] v110'4 uv4 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x560506d04d80 con 0x560506e88400 2023-09-23T03:26:32.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.711+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:68 14.1f 14:f8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7dc00 con 0x560506e73c00 2023-09-23T03:26:32.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.712+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.0 v2:172.21.15.154:6800/2920938138 22 ==== osd_op_reply(68 [pgnls start_epoch 110 out=101b] v110'13 uv13 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x560506d04b40 con 0x560506e73c00 2023-09-23T03:26:32.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.712+0000 7fa7514355c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2023-09-23T03:26:32.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.712+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:69 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506eae000 con 0x5605077ee000 2023-09-23T03:26:32.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.713+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 <== osd.1 v2:172.21.15.154:6816/1267710005 10 ==== osd_op_reply(69 10000000000.00000000 [omap-get-vals out=3582b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+3582 (crc 0 0 0) 0x560506d04d80 con 0x5605077ee000 2023-09-23T03:26:32.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.713+0000 7fa7514355c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:26:32.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.713+0000 7fa7514355c0 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2023-09-23T03:26:32.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.713+0000 7fa7514355c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:26:32.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.713+0000 7fa7514355c0 20 datascan.scan_links: adding 0x100000001f6 for future processing to fix dnfirst 2023-09-23T03:26:32.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.713+0000 7fa7514355c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:26:32.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.713+0000 7fa7514355c0 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2023-09-23T03:26:32.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.713+0000 7fa7514355c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:26:32.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.713+0000 7fa7514355c0 20 datascan.scan_links: adding 0x100000001f8 for future processing to fix dnfirst 2023-09-23T03:26:32.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.713+0000 7fa7514355c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:26:32.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.713+0000 7fa7514355c0 20 datascan.scan_links: adding 0x100000001f9 for future processing to fix dnfirst 2023-09-23T03:26:32.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.713+0000 7fa7514355c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:26:32.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.713+0000 7fa7514355c0 20 datascan.scan_links: adding 0x100000001fa for future processing to fix dnfirst 2023-09-23T03:26:32.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.713+0000 7fa7514355c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:26:32.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.713+0000 7fa7514355c0 20 datascan.scan_links: adding 0x100000001fb for future processing to fix dnfirst 2023-09-23T03:26:32.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.713+0000 7fa7514355c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2023-09-23T03:26:32.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.713+0000 7fa7514355c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-09-23T03:26:32.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.713+0000 7fa7514355c0 10 datascan.scan_links: step 2: handling object 200.00000000 2023-09-23T03:26:32.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.713+0000 7fa7514355c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-09-23T03:26:32.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.713+0000 7fa7514355c0 10 datascan.scan_links: step 2: handling object 200.00000001 2023-09-23T03:26:32.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.713+0000 7fa7514355c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-09-23T03:26:32.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.713+0000 7fa7514355c0 10 datascan.scan_links: step 2: handling object 500.00000000 2023-09-23T03:26:32.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.713+0000 7fa7514355c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-09-23T03:26:32.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.713+0000 7fa7514355c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2023-09-23T03:26:32.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.713+0000 7fa7514355c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-09-23T03:26:32.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.713+0000 7fa7514355c0 10 datascan.scan_links: step 2: handling object 4.00000000 2023-09-23T03:26:32.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.713+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:70 14.3 14:c0ce80d4:::4.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506eae380 con 0x560506e73c00 2023-09-23T03:26:32.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.714+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.0 v2:172.21.15.154:6800/2920938138 23 ==== osd_op_reply(70 4.00000000 [omap-get-vals out=513b] v0'0 uv2 ondisk = 0) v8 ==== 154+0+513 (crc 0 0 0) 0x560506d04b40 con 0x560506e73c00 2023-09-23T03:26:32.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.714+0000 7fa7514355c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:26:32.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.714+0000 7fa7514355c0 20 datascan.scan_links: adding 0x100000001fc for future processing to fix dnfirst 2023-09-23T03:26:32.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.714+0000 7fa7514355c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2023-09-23T03:26:32.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.714+0000 7fa7514355c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-09-23T03:26:32.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.714+0000 7fa7514355c0 10 datascan.scan_links: step 2: handling object 100.00000000 2023-09-23T03:26:32.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.714+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:71 14.13 14:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506eae700 con 0x560506e72400 2023-09-23T03:26:32.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.715+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.2 v2:172.21.15.154:6822/2884407243 8 ==== osd_op_reply(71 100.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x560506d04b40 con 0x560506e72400 2023-09-23T03:26:32.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.715+0000 7fa7514355c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2023-09-23T03:26:32.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.715+0000 7fa7514355c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-09-23T03:26:32.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.715+0000 7fa7514355c0 10 datascan.scan_links: step 2: handling object 400.00000000 2023-09-23T03:26:32.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.715+0000 7fa7514355c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-09-23T03:26:32.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.715+0000 7fa7514355c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2023-09-23T03:26:32.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.715+0000 7fa7514355c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-09-23T03:26:32.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.715+0000 7fa7514355c0 10 datascan.scan_links: step 2: handling object 1.00000000 2023-09-23T03:26:32.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.715+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:72 14.1f 14:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506eaea80 con 0x560506e73c00 2023-09-23T03:26:32.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.716+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.0 v2:172.21.15.154:6800/2920938138 24 ==== osd_op_reply(72 1.00000000 [omap-get-vals out=1016b] v0'0 uv13 ondisk = 0) v8 ==== 154+0+1016 (crc 0 0 0) 0x560506d04b40 con 0x560506e73c00 2023-09-23T03:26:32.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.716+0000 7fa7514355c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:26:32.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.716+0000 7fa7514355c0 20 datascan.scan_links: adding 0x4 for future processing to fix dnfirst 2023-09-23T03:26:32.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.716+0000 7fa7514355c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:26:32.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.716+0000 7fa7514355c0 20 datascan.scan_links: adding 0x10000000000 for future processing to fix dnfirst 2023-09-23T03:26:32.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.716+0000 7fa7514355c0 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2023-09-23T03:26:32.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.716+0000 7fa7514355c0 10 datascan.scan_links: removing dup dentries from 0 objects 2023-09-23T03:26:32.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.716+0000 7fa7514355c0 10 datascan.scan_links: processing 0 bad_nlink_inos 2023-09-23T03:26:32.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.716+0000 7fa7514355c0 10 datascan.scan_links: processing 10 injected_inos 2023-09-23T03:26:32.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.716+0000 7fa7514355c0 10 datascan.scan_links: handling injected_ino 0x4 2023-09-23T03:26:32.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.716+0000 7fa7514355c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x1:lost+found 2023-09-23T03:26:32.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.716+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:73 14.1f 14:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=23b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506eaee00 con 0x560506e73c00 2023-09-23T03:26:32.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.716+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.0 v2:172.21.15.154:6800/2920938138 25 ==== osd_op_reply(73 1.00000000 [omap-get-vals-by-keys out=511b] v0'0 uv13 ondisk = 0) v8 ==== 154+0+511 (crc 0 0 0) 0x560506d04b40 con 0x560506e73c00 2023-09-23T03:26:32.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.716+0000 7fa7514355c0 20 datascan.read_dentry: oid=1.00000000 dname=lost+found frag=*, r=0 2023-09-23T03:26:32.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.716+0000 7fa7514355c0 20 datascan.scan_links: first is now 2 2023-09-23T03:26:32.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.716+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:74 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506eae380 con 0x560506e73c00 2023-09-23T03:26:32.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.720+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.0 v2:172.21.15.154:6800/2920938138 26 ==== osd_op_reply(74 1.00000000 [omap-set-vals] v110'14 uv14 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x560506d04b40 con 0x560506e73c00 2023-09-23T03:26:32.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.720+0000 7fa7514355c0 20 datascan.inject_linkage: Injected dentry 0x0x1/lost+found pointing to 0x0x4 2023-09-23T03:26:32.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.720+0000 7fa7514355c0 10 datascan.scan_links: handling injected_ino 0x10000000000 2023-09-23T03:26:32.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.720+0000 7fa7514355c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x1:stripey 2023-09-23T03:26:32.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.720+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:75 14.1f 14:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506eae700 con 0x560506e73c00 2023-09-23T03:26:32.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.721+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.0 v2:172.21.15.154:6800/2920938138 27 ==== osd_op_reply(75 1.00000000 [omap-get-vals-by-keys out=508b] v0'0 uv14 ondisk = 0) v8 ==== 154+0+508 (crc 0 0 0) 0x560506d04b40 con 0x560506e73c00 2023-09-23T03:26:32.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.721+0000 7fa7514355c0 20 datascan.read_dentry: oid=1.00000000 dname=stripey frag=*, r=0 2023-09-23T03:26:32.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.721+0000 7fa7514355c0 20 datascan.scan_links: first is now 2 2023-09-23T03:26:32.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.721+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:76 14.1f 14:ff5b34d6:::1.00000000:head [omap-set-vals in=508b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506eaea80 con 0x560506e73c00 2023-09-23T03:26:32.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.724+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.0 v2:172.21.15.154:6800/2920938138 28 ==== osd_op_reply(76 1.00000000 [omap-set-vals] v110'15 uv15 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x560506d04b40 con 0x560506e73c00 2023-09-23T03:26:32.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.724+0000 7fa7514355c0 20 datascan.inject_linkage: Injected dentry 0x0x1/stripey pointing to 0x0x10000000000 2023-09-23T03:26:32.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.724+0000 7fa7514355c0 10 datascan.scan_links: handling injected_ino 0x10000000001 2023-09-23T03:26:32.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.724+0000 7fa7514355c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:flushed_file_0 2023-09-23T03:26:32.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.724+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:77 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506eaee00 con 0x5605077ee000 2023-09-23T03:26:32.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.725+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 <== osd.1 v2:172.21.15.154:6816/1267710005 11 ==== osd_op_reply(77 10000000000.00000000 [omap-get-vals-by-keys out=515b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+515 (crc 0 0 0) 0x560506d04d80 con 0x5605077ee000 2023-09-23T03:26:32.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.725+0000 7fa7514355c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_0 frag=*, r=0 2023-09-23T03:26:32.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.725+0000 7fa7514355c0 20 datascan.scan_links: first is now 2 2023-09-23T03:26:32.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.725+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:78 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506eaf180 con 0x5605077ee000 2023-09-23T03:26:32.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.729+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 <== osd.1 v2:172.21.15.154:6816/1267710005 12 ==== osd_op_reply(78 10000000000.00000000 [omap-set-vals] v110'24 uv24 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x560506d04d80 con 0x5605077ee000 2023-09-23T03:26:32.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.729+0000 7fa7514355c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_0 pointing to 0x0x10000000001 2023-09-23T03:26:32.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.729+0000 7fa7514355c0 10 datascan.scan_links: handling injected_ino 0x100000001f6 2023-09-23T03:26:32.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.729+0000 7fa7514355c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:flushed_file_1 2023-09-23T03:26:32.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.729+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:79 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7dc00 con 0x5605077ee000 2023-09-23T03:26:32.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.730+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 <== osd.1 v2:172.21.15.154:6816/1267710005 13 ==== osd_op_reply(79 10000000000.00000000 [omap-get-vals-by-keys out=515b] v0'0 uv24 ondisk = 0) v8 ==== 164+0+515 (crc 0 0 0) 0x560506d04d80 con 0x5605077ee000 2023-09-23T03:26:32.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.730+0000 7fa7514355c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_1 frag=*, r=0 2023-09-23T03:26:32.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.730+0000 7fa7514355c0 20 datascan.scan_links: first is now 2 2023-09-23T03:26:32.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.730+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:80 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506eae000 con 0x5605077ee000 2023-09-23T03:26:32.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.733+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 <== osd.1 v2:172.21.15.154:6816/1267710005 14 ==== osd_op_reply(80 10000000000.00000000 [omap-set-vals] v110'25 uv25 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x560506d04d80 con 0x5605077ee000 2023-09-23T03:26:32.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.733+0000 7fa7514355c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_1 pointing to 0x0x100000001f6 2023-09-23T03:26:32.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.733+0000 7fa7514355c0 10 datascan.scan_links: handling injected_ino 0x100000001f7 2023-09-23T03:26:32.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.733+0000 7fa7514355c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:flushed_file_2 2023-09-23T03:26:32.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.733+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:81 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506eaee00 con 0x5605077ee000 2023-09-23T03:26:32.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.734+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 <== osd.1 v2:172.21.15.154:6816/1267710005 15 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals-by-keys out=515b] v0'0 uv25 ondisk = 0) v8 ==== 164+0+515 (crc 0 0 0) 0x560506d04d80 con 0x5605077ee000 2023-09-23T03:26:32.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.734+0000 7fa7514355c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_2 frag=*, r=0 2023-09-23T03:26:32.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.734+0000 7fa7514355c0 20 datascan.scan_links: first is now 2 2023-09-23T03:26:32.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.734+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:82 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506eaf180 con 0x5605077ee000 2023-09-23T03:26:32.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.737+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 <== osd.1 v2:172.21.15.154:6816/1267710005 16 ==== osd_op_reply(82 10000000000.00000000 [omap-set-vals] v110'26 uv26 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x560506d04d80 con 0x5605077ee000 2023-09-23T03:26:32.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.737+0000 7fa7514355c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_2 pointing to 0x0x100000001f7 2023-09-23T03:26:32.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.737+0000 7fa7514355c0 10 datascan.scan_links: handling injected_ino 0x100000001f8 2023-09-23T03:26:32.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.737+0000 7fa7514355c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:flushed_file_3 2023-09-23T03:26:32.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.737+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:83 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506eaf500 con 0x5605077ee000 2023-09-23T03:26:32.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.738+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 <== osd.1 v2:172.21.15.154:6816/1267710005 17 ==== osd_op_reply(83 10000000000.00000000 [omap-get-vals-by-keys out=515b] v0'0 uv26 ondisk = 0) v8 ==== 164+0+515 (crc 0 0 0) 0x560506d04d80 con 0x5605077ee000 2023-09-23T03:26:32.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.738+0000 7fa7514355c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_3 frag=*, r=0 2023-09-23T03:26:32.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.738+0000 7fa7514355c0 20 datascan.scan_links: first is now 2 2023-09-23T03:26:32.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.738+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:84 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506eaf880 con 0x5605077ee000 2023-09-23T03:26:32.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.742+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 <== osd.1 v2:172.21.15.154:6816/1267710005 18 ==== osd_op_reply(84 10000000000.00000000 [omap-set-vals] v110'27 uv27 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x560506d04d80 con 0x5605077ee000 2023-09-23T03:26:32.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.742+0000 7fa7514355c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_3 pointing to 0x0x100000001f8 2023-09-23T03:26:32.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.742+0000 7fa7514355c0 10 datascan.scan_links: handling injected_ino 0x100000001f9 2023-09-23T03:26:32.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.742+0000 7fa7514355c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:flushed_file_4 2023-09-23T03:26:32.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.742+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:85 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506eafc00 con 0x5605077ee000 2023-09-23T03:26:32.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.743+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 <== osd.1 v2:172.21.15.154:6816/1267710005 19 ==== osd_op_reply(85 10000000000.00000000 [omap-get-vals-by-keys out=515b] v0'0 uv27 ondisk = 0) v8 ==== 164+0+515 (crc 0 0 0) 0x560506d04d80 con 0x5605077ee000 2023-09-23T03:26:32.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.743+0000 7fa7514355c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_4 frag=*, r=0 2023-09-23T03:26:32.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.743+0000 7fa7514355c0 20 datascan.scan_links: first is now 2 2023-09-23T03:26:32.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.743+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:86 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7d880 con 0x5605077ee000 2023-09-23T03:26:32.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.746+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 <== osd.1 v2:172.21.15.154:6816/1267710005 20 ==== osd_op_reply(86 10000000000.00000000 [omap-set-vals] v110'28 uv28 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x560506d04d80 con 0x5605077ee000 2023-09-23T03:26:32.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.746+0000 7fa7514355c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_4 pointing to 0x0x100000001f9 2023-09-23T03:26:32.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.746+0000 7fa7514355c0 10 datascan.scan_links: handling injected_ino 0x100000001fa 2023-09-23T03:26:32.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.746+0000 7fa7514355c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:flushed_file_5 2023-09-23T03:26:32.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.746+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:87 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506eae000 con 0x5605077ee000 2023-09-23T03:26:32.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.747+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 <== osd.1 v2:172.21.15.154:6816/1267710005 21 ==== osd_op_reply(87 10000000000.00000000 [omap-get-vals-by-keys out=515b] v0'0 uv28 ondisk = 0) v8 ==== 164+0+515 (crc 0 0 0) 0x560506d04d80 con 0x5605077ee000 2023-09-23T03:26:32.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.747+0000 7fa7514355c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_5 frag=*, r=0 2023-09-23T03:26:32.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.747+0000 7fa7514355c0 20 datascan.scan_links: first is now 2 2023-09-23T03:26:32.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.747+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:88 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506eaee00 con 0x5605077ee000 2023-09-23T03:26:32.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.751+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 <== osd.1 v2:172.21.15.154:6816/1267710005 22 ==== osd_op_reply(88 10000000000.00000000 [omap-set-vals] v110'29 uv29 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x560506d04d80 con 0x5605077ee000 2023-09-23T03:26:32.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.751+0000 7fa7514355c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_5 pointing to 0x0x100000001fa 2023-09-23T03:26:32.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.751+0000 7fa7514355c0 10 datascan.scan_links: handling injected_ino 0x100000001fb 2023-09-23T03:26:32.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.751+0000 7fa7514355c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:flushed_file_6 2023-09-23T03:26:32.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.751+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:89 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506eaf180 con 0x5605077ee000 2023-09-23T03:26:32.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.752+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 <== osd.1 v2:172.21.15.154:6816/1267710005 23 ==== osd_op_reply(89 10000000000.00000000 [omap-get-vals-by-keys out=515b] v0'0 uv29 ondisk = 0) v8 ==== 164+0+515 (crc 0 0 0) 0x560506d04d80 con 0x5605077ee000 2023-09-23T03:26:32.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.752+0000 7fa7514355c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_6 frag=*, r=0 2023-09-23T03:26:32.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.752+0000 7fa7514355c0 20 datascan.scan_links: first is now 2 2023-09-23T03:26:32.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.752+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:90 14.10 14:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506eaf500 con 0x5605077ee000 2023-09-23T03:26:32.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.756+0000 7fa74fbab640 1 -- 172.21.15.154:0/2089902380 <== osd.1 v2:172.21.15.154:6816/1267710005 24 ==== osd_op_reply(90 10000000000.00000000 [omap-set-vals] v110'30 uv30 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x560506d04d80 con 0x5605077ee000 2023-09-23T03:26:32.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.756+0000 7fa7514355c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_6 pointing to 0x0x100000001fb 2023-09-23T03:26:32.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.756+0000 7fa7514355c0 10 datascan.scan_links: handling injected_ino 0x100000001fc 2023-09-23T03:26:32.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.756+0000 7fa7514355c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x4:100000001fc 2023-09-23T03:26:32.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.756+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:91 14.3 14:c0ce80d4:::4.00000000:head [omap-get-vals-by-keys in=24b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506a7dc00 con 0x560506e73c00 2023-09-23T03:26:32.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.757+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.0 v2:172.21.15.154:6800/2920938138 29 ==== osd_op_reply(91 4.00000000 [omap-get-vals-by-keys out=512b] v0'0 uv2 ondisk = 0) v8 ==== 154+0+512 (crc 0 0 0) 0x560506d04b40 con 0x560506e73c00 2023-09-23T03:26:32.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.757+0000 7fa7514355c0 20 datascan.read_dentry: oid=4.00000000 dname=100000001fc frag=*, r=0 2023-09-23T03:26:32.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.757+0000 7fa7514355c0 20 datascan.scan_links: first is now 2 2023-09-23T03:26:32.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.757+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:92 14.3 14:c0ce80d4:::4.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506eaf880 con 0x560506e73c00 2023-09-23T03:26:32.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.761+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.0 v2:172.21.15.154:6800/2920938138 30 ==== osd_op_reply(92 4.00000000 [omap-set-vals] v110'3 uv3 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x560506d04b40 con 0x560506e73c00 2023-09-23T03:26:32.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.761+0000 7fa7514355c0 20 datascan.inject_linkage: Injected dentry 0x0x4/100000001fc pointing to 0x0x100000001fc 2023-09-23T03:26:32.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.761+0000 7fa7514355c0 10 datascan.scan_links: updating inotable 2023-09-23T03:26:32.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.761+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:93 14.13 14:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506eafc00 con 0x560506e72400 2023-09-23T03:26:32.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.762+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.2 v2:172.21.15.154:6822/2884407243 9 ==== osd_op_reply(93 mds0_inotable [read 0~34 out=34b] v0'0 uv6 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x560506d04b40 con 0x560506e72400 2023-09-23T03:26:32.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.762+0000 7fa7514355c0 10 mds.0.inotable: skip_inos was [0x10000000000~0x10000000000] 2023-09-23T03:26:32.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.762+0000 7fa7514355c0 10 mds.0.inotable: skip_inos now [0x100000001fd~0xfffffffe03] 2023-09-23T03:26:32.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.762+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:94 14.13 14:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506eae000 con 0x560506e72400 2023-09-23T03:26:32.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.766+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 <== osd.2 v2:172.21.15.154:6822/2884407243 10 ==== osd_op_reply(94 mds0_inotable [writefull 0~34] v110'8 uv8 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x560506d04b40 con 0x560506e72400 2023-09-23T03:26:32.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.766+0000 7fa7514355c0 10 datascan.scan_links: updating snaptable 2023-09-23T03:26:32.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.766+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:95 14.d 14:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x560506eaee00 con 0x560506e88000 2023-09-23T03:26:32.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.768+0000 7fa74eba9640 1 -- 172.21.15.154:0/2089902380 <== osd.6 v2:172.21.15.190:6816/1964294001 9 ==== osd_op_reply(95 mds_snaptable [read 0~70 out=70b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x560506d04fc0 con 0x560506e88000 2023-09-23T03:26:32.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.768+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x560506e73c00 msgr2=0x560506e78580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.768+0000 7fa7514355c0 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x560506e73c00 0x560506e78580 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.768+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5605077ee000 msgr2=0x560506a8d080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.768+0000 7fa7514355c0 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x5605077ee000 0x560506a8d080 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.768+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x560506e72400 msgr2=0x560506a8e680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.768+0000 7fa7514355c0 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x560506e72400 0x560506a8e680 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.768+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x560506e89000 msgr2=0x560506e79b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.768+0000 7fa7514355c0 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x560506e89000 0x560506e79b80 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.768+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x560506d0bc00 msgr2=0x560506e79080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.768+0000 7fa7514355c0 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x560506d0bc00 0x560506e79080 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.768+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x560506e88c00 msgr2=0x560506e79600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.768+0000 7fa7514355c0 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x560506e88c00 0x560506e79600 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.768+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x560506e88000 msgr2=0x560506e78b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.768+0000 7fa7514355c0 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x560506e88000 0x560506e78b00 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.768+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x560506e88400 msgr2=0x560506e7a680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.768+0000 7fa7514355c0 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x560506e88400 0x560506e7a680 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.768+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 reap_dead start 2023-09-23T03:26:32.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.768+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 reap_dead start 2023-09-23T03:26:32.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.768+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 reap_dead start 2023-09-23T03:26:32.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.768+0000 7fa74f3aa640 1 -- 172.21.15.154:0/2089902380 reap_dead start 2023-09-23T03:26:32.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.769+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x560506e72000 msgr2=0x560506e78000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.769+0000 7fa7514355c0 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x560506e72000 0x560506e78000 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x560506d22840 tx=0x560506d2ec00 comp rx=0 tx=0).stop 2023-09-23T03:26:32.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.769+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560506d0b000 msgr2=0x560506a8ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.769+0000 7fa7514355c0 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560506d0b000 0x560506a8ec00 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x560507835650 tx=0x560506d141e0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.769+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 shutdown_connections 2023-09-23T03:26:32.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.769+0000 7fa7514355c0 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x560506e72000 0x560506e78000 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.769+0000 7fa7514355c0 1 --2- 172.21.15.154:0/2089902380 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560506d0b000 0x560506a8ec00 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.769+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 >> 172.21.15.154:0/2089902380 conn(0x560506d0a400 msgr2=0x56050779d000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:32.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.769+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 shutdown_connections 2023-09-23T03:26:32.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.769+0000 7fa7514355c0 1 -- 172.21.15.154:0/2089902380 wait complete. 2023-09-23T03:26:32.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.770+0000 7fa7514355c0 1 -- 172.21.15.154:0/1139200614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5605077efc00 msgr2=0x560506a8cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:32.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.770+0000 7fa7514355c0 1 --2- 172.21.15.154:0/1139200614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5605077efc00 0x560506a8cb00 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x560507834750 tx=0x560506d14120 comp rx=0 tx=0).stop 2023-09-23T03:26:32.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.771+0000 7fa7514355c0 1 -- 172.21.15.154:0/1139200614 shutdown_connections 2023-09-23T03:26:32.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.771+0000 7fa7514355c0 1 --2- 172.21.15.154:0/1139200614 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560506d0a000 0x560506a8db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.771+0000 7fa7514355c0 1 --2- 172.21.15.154:0/1139200614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5605077efc00 0x560506a8cb00 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.771+0000 7fa7514355c0 1 --2- 172.21.15.154:0/1139200614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5605077ee400 0x560506a8d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:32.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.771+0000 7fa7514355c0 1 -- 172.21.15.154:0/1139200614 >> 172.21.15.154:0/1139200614 conn(0x560506a23000 msgr2=0x56050779c000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:32.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.771+0000 7fa7514355c0 1 -- 172.21.15.154:0/1139200614 shutdown_connections 2023-09-23T03:26:32.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:32.771+0000 7fa7514355c0 1 -- 172.21.15.154:0/1139200614 wait complete. 2023-09-23T03:26:32.781 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.248571, result: 2023-09-23T03:26:32.781 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-09-23T03:26:33.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.002+0000 7f4784bbb640 1 Processor -- start 2023-09-23T03:26:33.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.002+0000 7f4784bbb640 1 -- start start 2023-09-23T03:26:33.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.003+0000 7f4784bbb640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47800acce0 0x7f478015ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:33.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.003+0000 7f4784bbb640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f47800cf040 0x7f47800ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:33.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.003+0000 7f4784bbb640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f478015ac30 con 0x7f47800b1b20 2023-09-23T03:26:33.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.003+0000 7f4784bbb640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47800b2e90 con 0x7f47800cf040 2023-09-23T03:26:33.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.003+0000 7f4784bbb640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47800b1f70 con 0x7f47800acce0 2023-09-23T03:26:33.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.003+0000 7f477ed9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f47800cf040 0x7f47800ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:33.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.003+0000 7f477ed9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f47800cf040 0x7f47800ab140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46166/0 (socket says 172.21.15.154:46166) 2023-09-23T03:26:33.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.003+0000 7f477ed9d640 1 -- 172.21.15.154:0/1901532124 learned_addr learned my addr 172.21.15.154:0/1901532124 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:33.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.003+0000 7f477dd9b640 1 --1- 172.21.15.154:0/1901532124 >> v1:172.21.15.154:6789/0 conn(0x7f47800b1b20 0x7f47800ce930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:50642/0 (socket says 172.21.15.154:50642) 2023-09-23T03:26:33.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.004+0000 7f477e59c640 1 --2- 172.21.15.154:0/1901532124 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47800acce0 0x7f478015ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:33.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.004+0000 7f477ed9d640 1 -- 172.21.15.154:0/1901532124 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47800acce0 msgr2=0x7f478015ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:33.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.004+0000 7f477ed9d640 1 --2- 172.21.15.154:0/1901532124 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47800acce0 0x7f478015ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:33.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.004+0000 7f477ed9d640 1 -- 172.21.15.154:0/1901532124 >> v1:172.21.15.154:6789/0 conn(0x7f47800b1b20 legacy=0x7f47800ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:33.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.004+0000 7f477ed9d640 1 -- 172.21.15.154:0/1901532124 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f478015f2a0 con 0x7f47800cf040 2023-09-23T03:26:33.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.004+0000 7f477d59a640 1 -- 172.21.15.154:0/1901532124 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4088244819 0 0) 0x7f4768001490 con 0x7f47800b1b20 2023-09-23T03:26:33.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.005+0000 7f477ed9d640 1 --2- 172.21.15.154:0/1901532124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47800cf040 0x7f47800ab140 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f4778009870 tx=0x7f477802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d46b40404b1865 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:33.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.005+0000 7f477d59a640 1 -- 172.21.15.154:0/1901532124 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4778031640 con 0x7f47800cf040 2023-09-23T03:26:33.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.005+0000 7f477d59a640 1 -- 172.21.15.154:0/1901532124 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47780317a0 con 0x7f47800cf040 2023-09-23T03:26:33.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.005+0000 7f477d59a640 1 -- 172.21.15.154:0/1901532124 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4778031b30 con 0x7f47800cf040 2023-09-23T03:26:33.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.006+0000 7f4784bbb640 1 -- 172.21.15.154:0/1901532124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47800cf040 msgr2=0x7f47800ab140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:33.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.006+0000 7f4784bbb640 1 --2- 172.21.15.154:0/1901532124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47800cf040 0x7f47800ab140 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f4778009870 tx=0x7f477802b8f0 comp rx=0 tx=0).stop 2023-09-23T03:26:33.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.006+0000 7f4784bbb640 1 -- 172.21.15.154:0/1901532124 shutdown_connections 2023-09-23T03:26:33.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.006+0000 7f4784bbb640 1 --2- 172.21.15.154:0/1901532124 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47800cf040 0x7f47800ab140 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:33.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.006+0000 7f4784bbb640 1 --2- 172.21.15.154:0/1901532124 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47800acce0 0x7f478015ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:33.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.006+0000 7f4784bbb640 1 -- 172.21.15.154:0/1901532124 >> 172.21.15.154:0/1901532124 conn(0x7f478006a0f0 msgr2=0x7f478006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:33.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.006+0000 7f4784bbb640 1 -- 172.21.15.154:0/1901532124 shutdown_connections 2023-09-23T03:26:33.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.006+0000 7f4784bbb640 1 -- 172.21.15.154:0/1901532124 wait complete. 2023-09-23T03:26:33.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.008+0000 7f4784bbb640 1 Processor -- start 2023-09-23T03:26:33.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.008+0000 7f4784bbb640 1 -- start start 2023-09-23T03:26:33.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.009+0000 7f4784bbb640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47800acce0 0x7f47800c2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:33.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.009+0000 7f4784bbb640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47800b1b20 0x7f47800c3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:33.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.009+0000 7f4784bbb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47800cf040 0x7f478015b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:33.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.009+0000 7f4784bbb640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47800c15e0 con 0x7f47800cf040 2023-09-23T03:26:33.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.009+0000 7f4784bbb640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47800c73b0 con 0x7f47800acce0 2023-09-23T03:26:33.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.009+0000 7f4784bbb640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47800c7690 con 0x7f47800b1b20 2023-09-23T03:26:33.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.009+0000 7f477ed9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47800cf040 0x7f478015b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:33.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.009+0000 7f477ed9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47800cf040 0x7f478015b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46180/0 (socket says 172.21.15.154:46180) 2023-09-23T03:26:33.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.009+0000 7f477ed9d640 1 -- 172.21.15.154:0/4096469664 learned_addr learned my addr 172.21.15.154:0/4096469664 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:33.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.009+0000 7f477ed9d640 1 -- 172.21.15.154:0/4096469664 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47800b1b20 msgr2=0x7f47800c3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:33.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.010+0000 7f477ed9d640 1 --2- 172.21.15.154:0/4096469664 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47800b1b20 0x7f47800c3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:33.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.010+0000 7f477ed9d640 1 -- 172.21.15.154:0/4096469664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47800acce0 msgr2=0x7f47800c2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:33.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.010+0000 7f477ed9d640 1 --2- 172.21.15.154:0/4096469664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47800acce0 0x7f47800c2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:33.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.010+0000 7f477ed9d640 1 -- 172.21.15.154:0/4096469664 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47780094d0 con 0x7f47800cf040 2023-09-23T03:26:33.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.010+0000 7f477ed9d640 1 --2- 172.21.15.154:0/4096469664 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47800cf040 0x7f478015b0b0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f477802b930 tx=0x7f47780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:33.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.010+0000 7f4776ffd640 1 -- 172.21.15.154:0/4096469664 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477803ac60 con 0x7f47800cf040 2023-09-23T03:26:33.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.010+0000 7f4776ffd640 1 -- 172.21.15.154:0/4096469664 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f477803ade0 con 0x7f47800cf040 2023-09-23T03:26:33.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.010+0000 7f4776ffd640 1 -- 172.21.15.154:0/4096469664 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4778044630 con 0x7f47800cf040 2023-09-23T03:26:33.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.010+0000 7f4784bbb640 1 -- 172.21.15.154:0/4096469664 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47800c7970 con 0x7f47800cf040 2023-09-23T03:26:33.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.010+0000 7f4784bbb640 1 -- 172.21.15.154:0/4096469664 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47800c7f30 con 0x7f47800cf040 2023-09-23T03:26:33.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.011+0000 7f4784bbb640 1 -- 172.21.15.154:0/4096469664 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f478015add0 con 0x7f47800cf040 2023-09-23T03:26:33.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.013+0000 7f4776ffd640 1 -- 172.21.15.154:0/4096469664 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f477802f050 con 0x7f47800cf040 2023-09-23T03:26:33.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.013+0000 7f4776ffd640 1 --2- 172.21.15.154:0/4096469664 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f476006af60 0x7f476006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:33.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.014+0000 7f4776ffd640 1 -- 172.21.15.154:0/4096469664 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6605+0+0 (secure 0 0 0) 0x7f47780abdf0 con 0x7f47800cf040 2023-09-23T03:26:33.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.014+0000 7f477e59c640 1 --2- 172.21.15.154:0/4096469664 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f476006af60 0x7f476006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:33.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.015+0000 7f477e59c640 1 --2- 172.21.15.154:0/4096469664 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f476006af60 0x7f476006d420 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f477000a890 tx=0x7f4770008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:33.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.017+0000 7f4776ffd640 1 -- 172.21.15.154:0/4096469664 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f4778076b20 con 0x7f47800cf040 2023-09-23T03:26:33.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.281+0000 7f4784bbb640 1 -- 172.21.15.154:0/4096469664 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f478015add0 con 0x7f47800cf040 2023-09-23T03:26:33.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.390+0000 7f4776ffd640 1 -- 172.21.15.154:0/4096469664 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 5:0 v92) v1 ==== 101+0+0 (secure 0 0 0) 0x7f478015add0 con 0x7f47800cf040 2023-09-23T03:26:33.391 INFO:teuthology.orchestra.run.smithi154.stderr:repaired: restoring rank 5:0 2023-09-23T03:26:33.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.394+0000 7f4784bbb640 1 -- 172.21.15.154:0/4096469664 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f476006af60 msgr2=0x7f476006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:33.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.394+0000 7f4784bbb640 1 --2- 172.21.15.154:0/4096469664 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f476006af60 0x7f476006d420 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f477000a890 tx=0x7f4770008040 comp rx=0 tx=0).stop 2023-09-23T03:26:33.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.394+0000 7f4784bbb640 1 -- 172.21.15.154:0/4096469664 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47800cf040 msgr2=0x7f478015b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:33.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.394+0000 7f4784bbb640 1 --2- 172.21.15.154:0/4096469664 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47800cf040 0x7f478015b0b0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f477802b930 tx=0x7f47780046c0 comp rx=0 tx=0).stop 2023-09-23T03:26:33.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.394+0000 7f4784bbb640 1 -- 172.21.15.154:0/4096469664 shutdown_connections 2023-09-23T03:26:33.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.394+0000 7f4784bbb640 1 --2- 172.21.15.154:0/4096469664 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f476006af60 0x7f476006d420 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:33.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.394+0000 7f4784bbb640 1 --2- 172.21.15.154:0/4096469664 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f47800cf040 0x7f478015b0b0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:33.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.394+0000 7f4784bbb640 1 --2- 172.21.15.154:0/4096469664 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f47800b1b20 0x7f47800c3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:33.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.394+0000 7f4784bbb640 1 --2- 172.21.15.154:0/4096469664 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47800acce0 0x7f47800c2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:33.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.394+0000 7f4784bbb640 1 -- 172.21.15.154:0/4096469664 >> 172.21.15.154:0/4096469664 conn(0x7f478006a0f0 msgr2=0x7f47800b0870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:33.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.394+0000 7f4784bbb640 1 -- 172.21.15.154:0/4096469664 shutdown_connections 2023-09-23T03:26:33.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:33.395+0000 7f4784bbb640 1 -- 172.21.15.154:0/4096469664 wait complete. 2023-09-23T03:26:33.402 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:26:33.402+0000 7f8c87dc1640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-09-23T03:26:33.417 INFO:tasks.ceph.mds.a:Restarting daemon 2023-09-23T03:26:33.417 INFO:tasks.ceph.mds.a:Stopping old one... 2023-09-23T03:26:33.417 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-09-23T03:26:33.417 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:26:33.418 INFO:tasks.ceph.mds.c:Restarting daemon 2023-09-23T03:26:33.418 INFO:tasks.ceph.mds.c:Stopping old one... 2023-09-23T03:26:33.418 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-09-23T03:26:33.418 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:26:33.418 INFO:tasks.ceph.mds.b:Restarting daemon 2023-09-23T03:26:33.418 INFO:tasks.ceph.mds.b:Stopping old one... 2023-09-23T03:26:33.418 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-09-23T03:26:33.419 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:26:33.419 INFO:tasks.ceph.mds.d:Restarting daemon 2023-09-23T03:26:33.419 INFO:tasks.ceph.mds.d:Stopping old one... 2023-09-23T03:26:33.419 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-09-23T03:26:33.419 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:26:33.434 INFO:tasks.ceph.mds.a:Stopped 2023-09-23T03:26:33.434 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-09-23T03:26:33.435 INFO:tasks.ceph.mds.c:Stopped 2023-09-23T03:26:33.435 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-09-23T03:26:33.439 INFO:tasks.ceph.mds.b:Stopped 2023-09-23T03:26:33.439 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-09-23T03:26:33.483 INFO:tasks.ceph.mds.a:Started 2023-09-23T03:26:33.483 DEBUG:teuthology.parallel:result is None 2023-09-23T03:26:33.490 INFO:tasks.ceph.mds.b:Started 2023-09-23T03:26:33.490 DEBUG:teuthology.parallel:result is None 2023-09-23T03:26:33.491 INFO:tasks.ceph.mds.c:Started 2023-09-23T03:26:33.491 DEBUG:teuthology.parallel:result is None 2023-09-23T03:26:33.649 INFO:tasks.ceph.mds.c.smithi154.stdout:starting mds.c at 2023-09-23T03:26:33.650 INFO:tasks.ceph.mds.a.smithi154.stdout:starting mds.a at 2023-09-23T03:26:33.678 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-09-23T03:26:34.396 INFO:tasks.ceph.mon.a.smithi154.stderr:2023-09-23T03:26:34.396+0000 7f3df0da1640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-23T03:26:34.434 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:26:34.434+0000 7fd09cd76640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-09-23T03:26:37.075 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-09-23T03:26:39.439 INFO:tasks.ceph.mds.d:Stopped 2023-09-23T03:26:39.439 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-09-23T03:26:39.449 INFO:tasks.ceph.mds.d:Started 2023-09-23T03:26:39.449 DEBUG:teuthology.parallel:result is None 2023-09-23T03:26:39.449 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:26:39.634 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-09-23T03:26:39.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.675+0000 7facb2275640 1 Processor -- start 2023-09-23T03:26:39.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.675+0000 7facb2275640 1 -- start start 2023-09-23T03:26:39.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.677+0000 7facb2275640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7facac15caf0 0x7facac0ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:39.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.677+0000 7facb2275640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facac0cee70 0x7facac160880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:39.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.677+0000 7facb2275640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facac0cd530 con 0x7facac160dc0 2023-09-23T03:26:39.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.677+0000 7facb2275640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facac15ced0 con 0x7facac15caf0 2023-09-23T03:26:39.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.677+0000 7facb2275640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facac0cf2f0 con 0x7facac0cee70 2023-09-23T03:26:39.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.677+0000 7facabfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7facac15caf0 0x7facac0ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:39.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.677+0000 7facabfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7facac15caf0 0x7facac0ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:38726/0 (socket says 172.21.15.154:38726) 2023-09-23T03:26:39.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.677+0000 7facabfff640 1 -- 172.21.15.154:0/1377679907 learned_addr learned my addr 172.21.15.154:0/1377679907 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:39.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.677+0000 7faca3fff640 1 --2- 172.21.15.154:0/1377679907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facac0cee70 0x7facac160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:39.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.678+0000 7facabfff640 1 -- 172.21.15.154:0/1377679907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facac0cee70 msgr2=0x7facac160880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:39.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.678+0000 7facabfff640 1 --2- 172.21.15.154:0/1377679907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facac0cee70 0x7facac160880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:39.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.678+0000 7facabfff640 1 -- 172.21.15.154:0/1377679907 >> v1:172.21.15.154:6789/0 conn(0x7facac160dc0 legacy=0x7facac06cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:39.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.678+0000 7facabfff640 1 -- 172.21.15.154:0/1377679907 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facac0cd810 con 0x7facac15caf0 2023-09-23T03:26:39.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.678+0000 7faca3fff640 1 --2- 172.21.15.154:0/1377679907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facac0cee70 0x7facac160880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:26:39.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.679+0000 7facabfff640 1 --2- 172.21.15.154:0/1377679907 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facac15caf0 0x7facac0ce930 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009850 tx=0x7fac9c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc44e5d9adc4380 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:39.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.679+0000 7facab7fe640 1 -- 172.21.15.154:0/1377679907 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c004230 con 0x7facac15caf0 2023-09-23T03:26:39.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.679+0000 7facab7fe640 1 -- 172.21.15.154:0/1377679907 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9c004390 con 0x7facac15caf0 2023-09-23T03:26:39.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.679+0000 7facab7fe640 1 -- 172.21.15.154:0/1377679907 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c003810 con 0x7facac15caf0 2023-09-23T03:26:39.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.680+0000 7facb2275640 1 -- 172.21.15.154:0/1377679907 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facac15caf0 msgr2=0x7facac0ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:39.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.680+0000 7facb2275640 1 --2- 172.21.15.154:0/1377679907 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facac15caf0 0x7facac0ce930 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009850 tx=0x7fac9c02b960 comp rx=0 tx=0).stop 2023-09-23T03:26:39.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.680+0000 7facb2275640 1 -- 172.21.15.154:0/1377679907 shutdown_connections 2023-09-23T03:26:39.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.680+0000 7facb2275640 1 --2- 172.21.15.154:0/1377679907 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facac0cee70 0x7facac160880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:39.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.680+0000 7facb2275640 1 --2- 172.21.15.154:0/1377679907 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facac15caf0 0x7facac0ce930 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:39.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.680+0000 7facb2275640 1 -- 172.21.15.154:0/1377679907 >> 172.21.15.154:0/1377679907 conn(0x7facac06a0f0 msgr2=0x7facac06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:39.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.680+0000 7facb2275640 1 -- 172.21.15.154:0/1377679907 shutdown_connections 2023-09-23T03:26:39.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.680+0000 7facb2275640 1 -- 172.21.15.154:0/1377679907 wait complete. 2023-09-23T03:26:39.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.682+0000 7facb2275640 1 Processor -- start 2023-09-23T03:26:39.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.682+0000 7facb2275640 1 -- start start 2023-09-23T03:26:39.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.683+0000 7facb2275640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facac0cee70 0x7facac097a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:39.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.683+0000 7facb2275640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facac15caf0 0x7facac0c41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:39.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.683+0000 7facb2275640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facac160dc0 0x7facac0c4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:39.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.683+0000 7facb2275640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facac15e090 con 0x7facac15caf0 2023-09-23T03:26:39.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.683+0000 7facb2275640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facac1585f0 con 0x7facac0cee70 2023-09-23T03:26:39.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.683+0000 7facb2275640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facac15d480 con 0x7facac160dc0 2023-09-23T03:26:39.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.683+0000 7faca3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facac15caf0 0x7facac0c41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:39.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.683+0000 7faca3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facac15caf0 0x7facac0c41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:38730/0 (socket says 172.21.15.154:38730) 2023-09-23T03:26:39.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.683+0000 7faca3fff640 1 -- 172.21.15.154:0/852840287 learned_addr learned my addr 172.21.15.154:0/852840287 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:39.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.684+0000 7facabfff640 1 --2- 172.21.15.154:0/852840287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facac0cee70 0x7facac097a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:39.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.684+0000 7facb0812640 1 --2- 172.21.15.154:0/852840287 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facac160dc0 0x7facac0c4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:39.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.684+0000 7faca3fff640 1 -- 172.21.15.154:0/852840287 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facac160dc0 msgr2=0x7facac0c4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:39.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.684+0000 7faca3fff640 1 --2- 172.21.15.154:0/852840287 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facac160dc0 0x7facac0c4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:39.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.684+0000 7faca3fff640 1 -- 172.21.15.154:0/852840287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facac0cee70 msgr2=0x7facac097a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:39.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.684+0000 7faca3fff640 1 --2- 172.21.15.154:0/852840287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facac0cee70 0x7facac097a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:39.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.684+0000 7faca3fff640 1 -- 172.21.15.154:0/852840287 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac9c009460 con 0x7facac15caf0 2023-09-23T03:26:39.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.684+0000 7facabfff640 1 --2- 172.21.15.154:0/852840287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facac0cee70 0x7facac097a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:26:39.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.684+0000 7faca3fff640 1 --2- 172.21.15.154:0/852840287 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facac15caf0 0x7facac0c41c0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fac9400e7e0 tx=0x7fac9400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:39.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.684+0000 7faca97fa640 1 -- 172.21.15.154:0/852840287 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac94004030 con 0x7facac15caf0 2023-09-23T03:26:39.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.684+0000 7faca97fa640 1 -- 172.21.15.154:0/852840287 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac94004190 con 0x7facac15caf0 2023-09-23T03:26:39.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.684+0000 7facb2275640 1 -- 172.21.15.154:0/852840287 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facac15d760 con 0x7facac15caf0 2023-09-23T03:26:39.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.684+0000 7faca97fa640 1 -- 172.21.15.154:0/852840287 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac94004520 con 0x7facac15caf0 2023-09-23T03:26:39.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.685+0000 7facb2275640 1 -- 172.21.15.154:0/852840287 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facac15dd20 con 0x7facac15caf0 2023-09-23T03:26:39.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.686+0000 7facb2275640 1 -- 172.21.15.154:0/852840287 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facac15e200 con 0x7facac15caf0 2023-09-23T03:26:39.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.686+0000 7faca97fa640 1 -- 172.21.15.154:0/852840287 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fac9400c6d0 con 0x7facac15caf0 2023-09-23T03:26:39.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.688+0000 7faca97fa640 1 --2- 172.21.15.154:0/852840287 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fac8c06af30 0x7fac8c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:39.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.688+0000 7faca97fa640 1 -- 172.21.15.154:0/852840287 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fac940898a0 con 0x7facac15caf0 2023-09-23T03:26:39.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.688+0000 7facabfff640 1 --2- 172.21.15.154:0/852840287 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fac8c06af30 0x7fac8c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:39.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.689+0000 7facabfff640 1 --2- 172.21.15.154:0/852840287 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fac8c06af30 0x7fac8c06d3f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fac9c035000 tx=0x7fac9c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:39.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.693+0000 7faca97fa640 1 -- 172.21.15.154:0/852840287 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fac94054460 con 0x7facac15caf0 2023-09-23T03:26:39.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.919+0000 7facb2275640 1 -- 172.21.15.154:0/852840287 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7facac073eb0 con 0x7facac15caf0 2023-09-23T03:26:39.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.920+0000 7faca97fa640 1 -- 172.21.15.154:0/852840287 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+4923 (secure 0 0 0) 0x7fac94057a80 con 0x7facac15caf0 2023-09-23T03:26:39.922 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:39.922 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":98,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5753,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/60017546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":60017546},{"type":"v1","addr":"172.21.15.190:6833","nonce":60017546}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95},{"gid":6337,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/585518074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":585518074},{"type":"v1","addr":"172.21.15.190:6835","nonce":585518074}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":8592,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/1846547753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1846547753},{"type":"v1","addr":"172.21.15.154:6836","nonce":1846547753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:26:26.774375+0000","modified":"2023-09-23T03:26:36.432946+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8595":{"gid":8595,"name":"a","rank":0,"incarnation":95,"state":"up:active","state_seq":4,"addr":"172.21.15.154:6837/1758434763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1758434763},{"type":"v1","addr":"172.21.15.154:6837","nonce":1758434763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15,16],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-09-23T03:26:39.922 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 98 2023-09-23T03:26:39.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.923+0000 7facb2275640 1 -- 172.21.15.154:0/852840287 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fac8c06af30 msgr2=0x7fac8c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:39.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.923+0000 7facb2275640 1 --2- 172.21.15.154:0/852840287 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fac8c06af30 0x7fac8c06d3f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fac9c035000 tx=0x7fac9c008030 comp rx=0 tx=0).stop 2023-09-23T03:26:39.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.923+0000 7facb2275640 1 -- 172.21.15.154:0/852840287 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facac15caf0 msgr2=0x7facac0c41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:39.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.923+0000 7facb2275640 1 --2- 172.21.15.154:0/852840287 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facac15caf0 0x7facac0c41c0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fac9400e7e0 tx=0x7fac9400eca0 comp rx=0 tx=0).stop 2023-09-23T03:26:39.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.923+0000 7facb2275640 1 -- 172.21.15.154:0/852840287 shutdown_connections 2023-09-23T03:26:39.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.923+0000 7facb2275640 1 --2- 172.21.15.154:0/852840287 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fac8c06af30 0x7fac8c06d3f0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:39.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.923+0000 7facb2275640 1 --2- 172.21.15.154:0/852840287 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7facac160dc0 0x7facac0c4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:39.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.923+0000 7facb2275640 1 --2- 172.21.15.154:0/852840287 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facac15caf0 0x7facac0c41c0 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:39.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.923+0000 7facb2275640 1 --2- 172.21.15.154:0/852840287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7facac0cee70 0x7facac097a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:39.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.923+0000 7facb2275640 1 -- 172.21.15.154:0/852840287 >> 172.21.15.154:0/852840287 conn(0x7facac06a0f0 msgr2=0x7facac0b2090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:39.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.924+0000 7facb2275640 1 -- 172.21.15.154:0/852840287 shutdown_connections 2023-09-23T03:26:39.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:39.924+0000 7facb2275640 1 -- 172.21.15.154:0/852840287 wait complete. 2023-09-23T03:26:40.346 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 98, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-23T03:26:26.774375+0000', 'modified': '2023-09-23T03:26:36.432946+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 112, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8595}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8595': {'gid': 8595, 'name': 'a', 'rank': 0, 'incarnation': 95, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.154:6837/1758434763', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6835', 'nonce': 1758434763}, {'type': 'v1', 'addr': '172.21.15.154:6837', 'nonce': 1758434763}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [15, 16], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-23T03:26:40.347 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-23T03:26:40.347 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-09-23T03:26:40.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.576+0000 7f86aa355640 1 Processor -- start 2023-09-23T03:26:40.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.577+0000 7f86aa355640 1 -- start start 2023-09-23T03:26:40.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.578+0000 7f86aa355640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f86a415c8e0 0x7f86a406c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:40.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.578+0000 7f86aa355640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86a40cc690 0x7f86a40ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:40.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.578+0000 7f86aa355640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86a406c5b0 con 0x7f86a415ccc0 2023-09-23T03:26:40.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.578+0000 7f86aa355640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86a40cd040 con 0x7f86a415c8e0 2023-09-23T03:26:40.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.578+0000 7f86aa355640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a4161550 con 0x7f86a40cc690 2023-09-23T03:26:40.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.579+0000 7f86a3fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f86a415c8e0 0x7f86a406c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:40.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.579+0000 7f86a3fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f86a415c8e0 0x7f86a406c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.154:35156/0 (socket says 172.21.15.154:35156) 2023-09-23T03:26:40.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.579+0000 7f86a3fff640 1 -- 172.21.15.154:0/3730629273 learned_addr learned my addr 172.21.15.154:0/3730629273 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:40.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.579+0000 7f86a88f2640 1 --2- 172.21.15.154:0/3730629273 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86a40cc690 0x7f86a40ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:40.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.580+0000 7f86a88f2640 1 -- 172.21.15.154:0/3730629273 >> v2:172.21.15.190:3300/0 conn(0x7f86a415c8e0 msgr2=0x7f86a406c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:40.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.580+0000 7f86a88f2640 1 --2- 172.21.15.154:0/3730629273 >> v2:172.21.15.190:3300/0 conn(0x7f86a415c8e0 0x7f86a406c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:40.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.580+0000 7f86a88f2640 1 -- 172.21.15.154:0/3730629273 >> v1:172.21.15.190:6789/0 conn(0x7f86a415ccc0 legacy=0x7f86a406d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:40.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.580+0000 7f86a88f2640 1 -- 172.21.15.154:0/3730629273 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a41609d0 con 0x7f86a40cc690 2023-09-23T03:26:40.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.581+0000 7f86a88f2640 1 --2- 172.21.15.154:0/3730629273 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86a40cc690 0x7f86a40ccb00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f869c0097a0 tx=0x7f869c02b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39aa33a0c39f0dfd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:40.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.582+0000 7f86a2ffd640 1 -- 172.21.15.154:0/3730629273 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869c031780 con 0x7f86a40cc690 2023-09-23T03:26:40.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.582+0000 7f86a2ffd640 1 -- 172.21.15.154:0/3730629273 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f869c0318e0 con 0x7f86a40cc690 2023-09-23T03:26:40.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.582+0000 7f86a2ffd640 1 -- 172.21.15.154:0/3730629273 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869c031c70 con 0x7f86a40cc690 2023-09-23T03:26:40.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.583+0000 7f86aa355640 1 -- 172.21.15.154:0/3730629273 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86a40cc690 msgr2=0x7f86a40ccb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:40.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.583+0000 7f86aa355640 1 --2- 172.21.15.154:0/3730629273 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86a40cc690 0x7f86a40ccb00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f869c0097a0 tx=0x7f869c02b820 comp rx=0 tx=0).stop 2023-09-23T03:26:40.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.584+0000 7f86aa355640 1 -- 172.21.15.154:0/3730629273 shutdown_connections 2023-09-23T03:26:40.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.584+0000 7f86aa355640 1 --2- 172.21.15.154:0/3730629273 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86a40cc690 0x7f86a40ccb00 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:40.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.584+0000 7f86aa355640 1 --2- 172.21.15.154:0/3730629273 >> v2:172.21.15.190:3300/0 conn(0x7f86a415c8e0 0x7f86a406c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:40.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.584+0000 7f86aa355640 1 -- 172.21.15.154:0/3730629273 >> 172.21.15.154:0/3730629273 conn(0x7f86a40cf060 msgr2=0x7f86a40cb3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:40.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.584+0000 7f86aa355640 1 -- 172.21.15.154:0/3730629273 shutdown_connections 2023-09-23T03:26:40.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.584+0000 7f86aa355640 1 -- 172.21.15.154:0/3730629273 wait complete. 2023-09-23T03:26:40.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.587+0000 7f86aa355640 1 Processor -- start 2023-09-23T03:26:40.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.587+0000 7f86aa355640 1 -- start start 2023-09-23T03:26:40.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.588+0000 7f86aa355640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86a40cc690 0x7f86a41f3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:40.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.588+0000 7f86aa355640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86a415c8e0 0x7f86a40b00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:40.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.588+0000 7f86aa355640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86a415ccc0 0x7f86a40c0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:40.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.588+0000 7f86aa355640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a41f42f0 con 0x7f86a415c8e0 2023-09-23T03:26:40.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.588+0000 7f86aa355640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a40b0610 con 0x7f86a40cc690 2023-09-23T03:26:40.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.588+0000 7f86aa355640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a40c1030 con 0x7f86a415ccc0 2023-09-23T03:26:40.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.589+0000 7f86a37fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86a415c8e0 0x7f86a40b00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:40.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.589+0000 7f86a37fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86a415c8e0 0x7f86a40b00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:38742/0 (socket says 172.21.15.154:38742) 2023-09-23T03:26:40.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.589+0000 7f86a37fe640 1 -- 172.21.15.154:0/3850365777 learned_addr learned my addr 172.21.15.154:0/3850365777 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:40.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.589+0000 7f86a3fff640 1 --2- 172.21.15.154:0/3850365777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86a40cc690 0x7f86a41f3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:40.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.589+0000 7f86a88f2640 1 --2- 172.21.15.154:0/3850365777 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86a415ccc0 0x7f86a40c0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:40.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.589+0000 7f86a37fe640 1 -- 172.21.15.154:0/3850365777 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86a415ccc0 msgr2=0x7f86a40c0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:40.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.589+0000 7f86a37fe640 1 --2- 172.21.15.154:0/3850365777 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86a415ccc0 0x7f86a40c0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:40.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.589+0000 7f86a37fe640 1 -- 172.21.15.154:0/3850365777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86a40cc690 msgr2=0x7f86a41f3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:40.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.589+0000 7f86a37fe640 1 --2- 172.21.15.154:0/3850365777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86a40cc690 0x7f86a41f3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:40.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.589+0000 7f86a37fe640 1 -- 172.21.15.154:0/3850365777 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869c009400 con 0x7f86a415c8e0 2023-09-23T03:26:40.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.590+0000 7f86a37fe640 1 --2- 172.21.15.154:0/3850365777 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86a415c8e0 0x7f86a40b00d0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f868c00c450 tx=0x7f868c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:40.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.590+0000 7f86a0ff9640 1 -- 172.21.15.154:0/3850365777 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c0052d0 con 0x7f86a415c8e0 2023-09-23T03:26:40.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.590+0000 7f86a0ff9640 1 -- 172.21.15.154:0/3850365777 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868c005430 con 0x7f86a415c8e0 2023-09-23T03:26:40.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.590+0000 7f86a0ff9640 1 -- 172.21.15.154:0/3850365777 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c012870 con 0x7f86a415c8e0 2023-09-23T03:26:40.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.590+0000 7f86aa355640 1 -- 172.21.15.154:0/3850365777 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86a41f4510 con 0x7f86a415c8e0 2023-09-23T03:26:40.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.590+0000 7f86aa355640 1 -- 172.21.15.154:0/3850365777 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86a40ae3a0 con 0x7f86a415c8e0 2023-09-23T03:26:40.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.592+0000 7f86a0ff9640 1 -- 172.21.15.154:0/3850365777 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f868c0129d0 con 0x7f86a415c8e0 2023-09-23T03:26:40.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.593+0000 7f86a0ff9640 1 --2- 172.21.15.154:0/3850365777 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f868406af30 0x7f868406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:40.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.593+0000 7f86a0ff9640 1 -- 172.21.15.154:0/3850365777 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f868c089640 con 0x7f86a415c8e0 2023-09-23T03:26:40.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.593+0000 7f86a3fff640 1 --2- 172.21.15.154:0/3850365777 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f868406af30 0x7f868406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:40.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.594+0000 7f86acfbe740 1 Processor -- start 2023-09-23T03:26:40.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.594+0000 7f86acfbe740 1 -- start start 2023-09-23T03:26:40.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.594+0000 7f86a3fff640 1 --2- 172.21.15.154:0/3850365777 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f868406af30 0x7f868406d3f0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f8694009ce0 tx=0x7f869400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:40.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.595+0000 7f86acfbe740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5592423f6c00 0x5592423c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:40.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.595+0000 7f86acfbe740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5592423f6fe0 0x5592423f9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:40.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.595+0000 7f86acfbe740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592423ca040 0x5592423f9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:40.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.595+0000 7f86acfbe740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55924233d6d0 con 0x5592423ca040 2023-09-23T03:26:40.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.595+0000 7f86acfbe740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55924233adc0 con 0x5592423f6fe0 2023-09-23T03:26:40.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.595+0000 7f86acfbe740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5592423366b0 con 0x5592423f6c00 2023-09-23T03:26:40.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.595+0000 7f86a88f2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592423ca040 0x5592423f9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:40.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.595+0000 7f86a88f2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592423ca040 0x5592423f9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:38754/0 (socket says 172.21.15.154:38754) 2023-09-23T03:26:40.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.595+0000 7f86a88f2640 1 -- 172.21.15.154:0/1628356793 learned_addr learned my addr 172.21.15.154:0/1628356793 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:40.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.596+0000 7f86a37fe640 1 --2- 172.21.15.154:0/1628356793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5592423f6fe0 0x5592423f9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:40.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.596+0000 7f86a3fff640 1 --2- 172.21.15.154:0/1628356793 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5592423f6c00 0x5592423c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:40.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.596+0000 7f86a88f2640 1 -- 172.21.15.154:0/1628356793 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5592423f6c00 msgr2=0x5592423c9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:40.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.596+0000 7f86a88f2640 1 --2- 172.21.15.154:0/1628356793 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5592423f6c00 0x5592423c9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:40.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.596+0000 7f86a88f2640 1 -- 172.21.15.154:0/1628356793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5592423f6fe0 msgr2=0x5592423f9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:40.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.596+0000 7f86a88f2640 1 --2- 172.21.15.154:0/1628356793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5592423f6fe0 0x5592423f9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:40.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.596+0000 7f86a88f2640 1 -- 172.21.15.154:0/1628356793 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592422fa8e0 con 0x5592423ca040 2023-09-23T03:26:40.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.596+0000 7f86a37fe640 1 --2- 172.21.15.154:0/1628356793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5592423f6fe0 0x5592423f9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:26:40.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.596+0000 7f86a3fff640 1 --2- 172.21.15.154:0/1628356793 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5592423f6c00 0x5592423c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:26:40.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.596+0000 7f86a88f2640 1 --2- 172.21.15.154:0/1628356793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592423ca040 0x5592423f9ed0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f869c0098c0 tx=0x7f869c008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5bf766882f27b53 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:40.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.596+0000 7f8699ffb640 1 -- 172.21.15.154:0/1628356793 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869c031c50 con 0x5592423ca040 2023-09-23T03:26:40.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.596+0000 7f8699ffb640 1 -- 172.21.15.154:0/1628356793 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f869c031db0 con 0x5592423ca040 2023-09-23T03:26:40.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.596+0000 7f8699ffb640 1 -- 172.21.15.154:0/1628356793 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869c046680 con 0x5592423ca040 2023-09-23T03:26:40.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.597+0000 7f86acfbe740 1 -- 172.21.15.154:0/1628356793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592423ca040 msgr2=0x5592423f9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:40.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.597+0000 7f86acfbe740 1 --2- 172.21.15.154:0/1628356793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592423ca040 0x5592423f9ed0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f869c0098c0 tx=0x7f869c008030 comp rx=0 tx=0).stop 2023-09-23T03:26:40.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.597+0000 7f86acfbe740 1 -- 172.21.15.154:0/1628356793 shutdown_connections 2023-09-23T03:26:40.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.597+0000 7f86acfbe740 1 --2- 172.21.15.154:0/1628356793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592423ca040 0x5592423f9ed0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:40.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.597+0000 7f86acfbe740 1 --2- 172.21.15.154:0/1628356793 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5592423f6fe0 0x5592423f9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:40.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.597+0000 7f86acfbe740 1 --2- 172.21.15.154:0/1628356793 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5592423f6c00 0x5592423c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:40.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.597+0000 7f86acfbe740 1 -- 172.21.15.154:0/1628356793 >> 172.21.15.154:0/1628356793 conn(0x5592423cbe10 msgr2=0x559242414990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:40.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.597+0000 7f86acfbe740 1 -- 172.21.15.154:0/1628356793 shutdown_connections 2023-09-23T03:26:40.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.597+0000 7f86acfbe740 1 -- 172.21.15.154:0/1628356793 wait complete. 2023-09-23T03:26:40.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.598+0000 7f86acfbe740 1 -- start start 2023-09-23T03:26:40.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.598+0000 7f86acfbe740 1 Processor -- start 2023-09-23T03:26:40.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.599+0000 7f86acfbe740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5592423f6c00 0x5592423e3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:40.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.599+0000 7f86acfbe740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592423f6fe0 0x5592423ef650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:40.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.599+0000 7f86acfbe740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559242427cc0 0x559242428140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:40.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.599+0000 7f86acfbe740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592422d4940 con 0x5592423f6fe0 2023-09-23T03:26:40.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.599+0000 7f86acfbe740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592422d45e0 con 0x559242427cc0 2023-09-23T03:26:40.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.599+0000 7f86acfbe740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55924209af20 con 0x5592423f6c00 2023-09-23T03:26:40.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.599+0000 7f86acfbe740 10 client.0 _ll_register_callbacks cb 0x5592423125e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:26:40.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.600+0000 7f86a37fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592423f6fe0 0x5592423ef650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:40.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.600+0000 7f86a37fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592423f6fe0 0x5592423ef650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:38770/0 (socket says 172.21.15.154:38770) 2023-09-23T03:26:40.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.600+0000 7f86a37fe640 1 -- 172.21.15.154:0/1432764173 learned_addr learned my addr 172.21.15.154:0/1432764173 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:40.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.600+0000 7f86a88f2640 1 --2- 172.21.15.154:0/1432764173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559242427cc0 0x559242428140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:40.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.600+0000 7f86a3fff640 1 --2- 172.21.15.154:0/1432764173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5592423f6c00 0x5592423e3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:40.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.600+0000 7f86a37fe640 1 -- 172.21.15.154:0/1432764173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5592423f6c00 msgr2=0x5592423e3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:40.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.600+0000 7f86a37fe640 1 --2- 172.21.15.154:0/1432764173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5592423f6c00 0x5592423e3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:40.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.600+0000 7f86a37fe640 1 -- 172.21.15.154:0/1432764173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559242427cc0 msgr2=0x559242428140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:40.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.600+0000 7f86a37fe640 1 --2- 172.21.15.154:0/1432764173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559242427cc0 0x559242428140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:40.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.600+0000 7f86a37fe640 1 -- 172.21.15.154:0/1432764173 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86a41609d0 con 0x5592423f6fe0 2023-09-23T03:26:40.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.600+0000 7f86a88f2640 1 --2- 172.21.15.154:0/1432764173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559242427cc0 0x559242428140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:26:40.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.600+0000 7f86a37fe640 1 --2- 172.21.15.154:0/1432764173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592423f6fe0 0x5592423ef650 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f868c0876f0 tx=0x7f868c058790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cea144fe41369b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:40.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.600+0000 7f8682ffd640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:26:40.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.600+0000 7f8682ffd640 1 -- 172.21.15.154:0/1432764173 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c060250 con 0x5592423f6fe0 2023-09-23T03:26:40.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.601+0000 7f86acfbe740 1 -- 172.21.15.154:0/1432764173 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55924232f690 con 0x5592423f6fe0 2023-09-23T03:26:40.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.601+0000 7f8682ffd640 1 -- 172.21.15.154:0/1432764173 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f868c054660 con 0x5592423f6fe0 2023-09-23T03:26:40.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.601+0000 7f8682ffd640 1 -- 172.21.15.154:0/1432764173 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f868c054820 con 0x5592423f6fe0 2023-09-23T03:26:40.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.601+0000 7f86acfbe740 10 client.8616 fetch_fsmap learned FSMap version 99 2023-09-23T03:26:40.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.601+0000 7f86acfbe740 1 -- 172.21.15.154:0/1432764173 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x559242334d30 con 0x5592423f6fe0 2023-09-23T03:26:40.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.601+0000 7f8682ffd640 1 -- 172.21.15.154:0/1432764173 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868c073080 con 0x5592423f6fe0 2023-09-23T03:26:40.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.601+0000 7f8682ffd640 1 -- 172.21.15.154:0/1432764173 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 99) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f868c019020 con 0x5592423f6fe0 2023-09-23T03:26:40.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.601+0000 7f86acfbe740 10 client.8616 fetch_fsmap finished waiting for FSMap version 99 2023-09-23T03:26:40.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.601+0000 7f86acfbe740 10 client.8616 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-23T03:26:40.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.601+0000 7f86acfbe740 20 client.8616 populate_metadata read hostname 'smithi154' 2023-09-23T03:26:40.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.601+0000 7f86acfbe740 1 --2- 172.21.15.154:0/1432764173 >> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] conn(0x5592423f8dd0 0x559242431f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:40.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.601+0000 7f86acfbe740 4 client.8616 mds_command: new command op to 8595 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T03:26:40.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.601+0000 7f86acfbe740 1 -- 172.21.15.154:0/1432764173 --> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55924232f3b0 con 0x5592423f8dd0 2023-09-23T03:26:40.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.601+0000 7f86a3fff640 1 --2- 172.21.15.154:0/1432764173 >> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] conn(0x5592423f8dd0 0x559242431f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:40.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.602+0000 7f86a3fff640 1 --2- 172.21.15.154:0/1432764173 >> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] conn(0x5592423f8dd0 0x559242431f90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3da2fd6ab8cbc474 server_cookie=7c7c53489a90360a in_seq=0 out_seq=0 2023-09-23T03:26:40.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.602+0000 7f8682ffd640 10 client.8616 ms_handle_connect on v2:172.21.15.154:6835/1758434763 2023-09-23T03:26:40.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.603+0000 7f8682ffd640 1 -- 172.21.15.154:0/1432764173 <== mds.0 v2:172.21.15.154:6835/1758434763 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55924232f3b0 con 0x5592423f8dd0 2023-09-23T03:26:40.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.603+0000 7f8682ffd640 10 client.8616 handle_command_reply: tid=0 2023-09-23T03:26:40.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.603+0000 7f86acfbe740 1 client.8616 shutdown 2023-09-23T03:26:40.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.603+0000 7f86acfbe740 1 -- 172.21.15.154:0/1432764173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592423f6fe0 msgr2=0x5592423ef650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:40.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.603+0000 7f86acfbe740 1 --2- 172.21.15.154:0/1432764173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592423f6fe0 0x5592423ef650 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f868c0876f0 tx=0x7f868c058790 comp rx=0 tx=0).stop 2023-09-23T03:26:40.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.603+0000 7f86acfbe740 1 -- 172.21.15.154:0/1432764173 shutdown_connections 2023-09-23T03:26:40.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.603+0000 7f86acfbe740 1 --2- 172.21.15.154:0/1432764173 >> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] conn(0x5592423f8dd0 0x559242431f90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:40.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.603+0000 7f86acfbe740 1 --2- 172.21.15.154:0/1432764173 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x559242427cc0 0x559242428140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:40.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.603+0000 7f86acfbe740 1 --2- 172.21.15.154:0/1432764173 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592423f6fe0 0x5592423ef650 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:40.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.603+0000 7f86acfbe740 1 --2- 172.21.15.154:0/1432764173 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5592423f6c00 0x5592423e3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:40.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.603+0000 7f86acfbe740 1 -- 172.21.15.154:0/1432764173 >> 172.21.15.154:0/1432764173 conn(0x5592423ca040 msgr2=0x5592423cdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:40.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.603+0000 7f8682ffd640 0 client.8616 ms_handle_reset on v2:172.21.15.154:6835/1758434763 2023-09-23T03:26:40.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.603+0000 7f86acfbe740 1 -- 172.21.15.154:0/1432764173 shutdown_connections 2023-09-23T03:26:40.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.603+0000 7f86acfbe740 1 -- 172.21.15.154:0/1432764173 wait complete. 2023-09-23T03:26:40.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.604+0000 7f86acfbe740 20 client.8616 trim_cache size 0 max 16384 2023-09-23T03:26:40.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.620+0000 7f86acfbe740 1 Processor -- start 2023-09-23T03:26:40.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.620+0000 7f86acfbe740 1 -- start start 2023-09-23T03:26:40.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.621+0000 7f86acfbe740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5592423f6c00 0x5592423e2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:40.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.621+0000 7f86acfbe740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592423f6fe0 0x5592423e3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:40.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.621+0000 7f86acfbe740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559242427cc0 0x5592423cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:40.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.621+0000 7f86acfbe740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x559242436950 con 0x5592423f6fe0 2023-09-23T03:26:40.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.621+0000 7f86acfbe740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x559242436400 con 0x5592423f6c00 2023-09-23T03:26:40.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.621+0000 7f86acfbe740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5592423f2230 con 0x559242427cc0 2023-09-23T03:26:40.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.621+0000 7f86a37fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592423f6fe0 0x5592423e3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:40.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.621+0000 7f86a37fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592423f6fe0 0x5592423e3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:38786/0 (socket says 172.21.15.154:38786) 2023-09-23T03:26:40.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.621+0000 7f86a37fe640 1 -- 172.21.15.154:0/2700475258 learned_addr learned my addr 172.21.15.154:0/2700475258 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:40.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.622+0000 7f86a3fff640 1 --2- 172.21.15.154:0/2700475258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5592423f6c00 0x5592423e2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:40.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.622+0000 7f86a37fe640 1 -- 172.21.15.154:0/2700475258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559242427cc0 msgr2=0x5592423cd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:40.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.622+0000 7f86a88f2640 1 --2- 172.21.15.154:0/2700475258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559242427cc0 0x5592423cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:40.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.622+0000 7f86a37fe640 1 --2- 172.21.15.154:0/2700475258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559242427cc0 0x5592423cd7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:40.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.622+0000 7f86a37fe640 1 -- 172.21.15.154:0/2700475258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5592423f6c00 msgr2=0x5592423e2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:40.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.622+0000 7f86a37fe640 1 --2- 172.21.15.154:0/2700475258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5592423f6c00 0x5592423e2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:40.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.622+0000 7f86a37fe640 1 -- 172.21.15.154:0/2700475258 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592422d3b50 con 0x5592423f6fe0 2023-09-23T03:26:40.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.622+0000 7f86a3fff640 1 --2- 172.21.15.154:0/2700475258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5592423f6c00 0x5592423e2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:26:40.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.622+0000 7f86a37fe640 1 --2- 172.21.15.154:0/2700475258 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592423f6fe0 0x5592423e3190 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f868c00a040 tx=0x7f868c05a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3219cb99aea51d8f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:40.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.622+0000 7f869a7fc640 1 -- 172.21.15.154:0/2700475258 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c012c80 con 0x5592423f6fe0 2023-09-23T03:26:40.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.622+0000 7f869a7fc640 1 -- 172.21.15.154:0/2700475258 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55924232f690 con 0x5592423f6fe0 2023-09-23T03:26:40.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.622+0000 7f869a7fc640 1 -- 172.21.15.154:0/2700475258 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c012e60 con 0x5592423f6fe0 2023-09-23T03:26:40.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.623+0000 7f86acfbe740 1 -- 172.21.15.154:0/2700475258 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592423f6fe0 msgr2=0x5592423e3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:40.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.623+0000 7f86acfbe740 1 --2- 172.21.15.154:0/2700475258 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592423f6fe0 0x5592423e3190 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f868c00a040 tx=0x7f868c05a000 comp rx=0 tx=0).stop 2023-09-23T03:26:40.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.623+0000 7f86acfbe740 1 -- 172.21.15.154:0/2700475258 shutdown_connections 2023-09-23T03:26:40.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.623+0000 7f86acfbe740 1 --2- 172.21.15.154:0/2700475258 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559242427cc0 0x5592423cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:40.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.623+0000 7f86acfbe740 1 --2- 172.21.15.154:0/2700475258 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592423f6fe0 0x5592423e3190 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:40.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.623+0000 7f86acfbe740 1 --2- 172.21.15.154:0/2700475258 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5592423f6c00 0x5592423e2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:40.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.623+0000 7f86acfbe740 1 -- 172.21.15.154:0/2700475258 >> 172.21.15.154:0/2700475258 conn(0x5592423ca040 msgr2=0x559242417ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:40.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.623+0000 7f86acfbe740 1 -- 172.21.15.154:0/2700475258 shutdown_connections 2023-09-23T03:26:40.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.623+0000 7f86acfbe740 1 -- 172.21.15.154:0/2700475258 wait complete. 2023-09-23T03:26:40.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.624+0000 7f86acfbe740 1 -- start start 2023-09-23T03:26:40.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.624+0000 7f86acfbe740 1 Processor -- start 2023-09-23T03:26:40.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.625+0000 7f86acfbe740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592423f6c00 0x5592423e2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:40.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.625+0000 7f86acfbe740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5592423f6fe0 0x5592423cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:40.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.625+0000 7f86acfbe740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559242427cc0 0x5592423cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:40.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.625+0000 7f86acfbe740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55924247dcf0 con 0x5592423f6c00 2023-09-23T03:26:40.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.625+0000 7f86acfbe740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592423e3300 con 0x5592423f6fe0 2023-09-23T03:26:40.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.625+0000 7f86acfbe740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5592423e3550 con 0x559242427cc0 2023-09-23T03:26:40.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.625+0000 7f86acfbe740 10 client.0 _ll_register_callbacks cb 0x7f86940096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:26:40.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.625+0000 7f86a3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592423f6c00 0x5592423e2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:40.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.625+0000 7f86a3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592423f6c00 0x5592423e2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:38790/0 (socket says 172.21.15.154:38790) 2023-09-23T03:26:40.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.625+0000 7f86a3fff640 1 -- 172.21.15.154:0/239590944 learned_addr learned my addr 172.21.15.154:0/239590944 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:40.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.626+0000 7f86a3fff640 1 -- 172.21.15.154:0/239590944 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559242427cc0 msgr2=0x5592423cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:40.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.626+0000 7f86a3fff640 1 --2- 172.21.15.154:0/239590944 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559242427cc0 0x5592423cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:40.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.626+0000 7f86a3fff640 1 -- 172.21.15.154:0/239590944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5592423f6fe0 msgr2=0x5592423cd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:40.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.626+0000 7f86a3fff640 1 --2- 172.21.15.154:0/239590944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5592423f6fe0 0x5592423cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:40.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.626+0000 7f86a3fff640 1 -- 172.21.15.154:0/239590944 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559242334d30 con 0x5592423f6c00 2023-09-23T03:26:40.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.626+0000 7f86a3fff640 1 --2- 172.21.15.154:0/239590944 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592423f6c00 0x5592423e2d00 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x5592423f88e0 tx=0x7f8694004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=812ae8c94b887523 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:40.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.626+0000 7f8682ffd640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:26:40.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.626+0000 7f8682ffd640 1 -- 172.21.15.154:0/239590944 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869400e050 con 0x5592423f6c00 2023-09-23T03:26:40.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.626+0000 7f8682ffd640 1 -- 172.21.15.154:0/239590944 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f8694024990 con 0x5592423f6c00 2023-09-23T03:26:40.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.626+0000 7f86acfbe740 1 -- 172.21.15.154:0/239590944 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f869c031db0 con 0x5592423f6c00 2023-09-23T03:26:40.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.626+0000 7f8682ffd640 1 -- 172.21.15.154:0/239590944 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8694015e80 con 0x5592423f6c00 2023-09-23T03:26:40.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.626+0000 7f86acfbe740 10 client.8622 fetch_fsmap learned FSMap version 99 2023-09-23T03:26:40.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.626+0000 7f86acfbe740 1 -- 172.21.15.154:0/239590944 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x5592422f9140 con 0x5592423f6c00 2023-09-23T03:26:40.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.627+0000 7f8682ffd640 1 -- 172.21.15.154:0/239590944 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f869c031db0 con 0x5592423f6c00 2023-09-23T03:26:40.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.627+0000 7f8682ffd640 1 -- 172.21.15.154:0/239590944 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 99) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f8694022040 con 0x5592423f6c00 2023-09-23T03:26:40.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.627+0000 7f86acfbe740 10 client.8622 fetch_fsmap finished waiting for FSMap version 99 2023-09-23T03:26:40.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.627+0000 7f86acfbe740 10 client.8622 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-23T03:26:40.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.627+0000 7f86acfbe740 20 client.8622 populate_metadata read hostname 'smithi154' 2023-09-23T03:26:40.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.627+0000 7f86acfbe740 1 --2- 172.21.15.154:0/239590944 >> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] conn(0x55924247c320 0x55924247c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:40.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.627+0000 7f86acfbe740 4 client.8622 mds_command: new command op to 8595 tid=0[{"prefix": "status"}] 2023-09-23T03:26:40.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.627+0000 7f86acfbe740 1 -- 172.21.15.154:0/239590944 --> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5592420386c0 con 0x55924247c320 2023-09-23T03:26:40.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.627+0000 7f86a37fe640 1 --2- 172.21.15.154:0/239590944 >> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] conn(0x55924247c320 0x55924247c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:40.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.627+0000 7f86a37fe640 1 --2- 172.21.15.154:0/239590944 >> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] conn(0x55924247c320 0x55924247c700 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adf6212e3c22e0ed server_cookie=661b551d8076641 in_seq=0 out_seq=0 2023-09-23T03:26:40.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.627+0000 7f8682ffd640 10 client.8622 ms_handle_connect on v2:172.21.15.154:6835/1758434763 2023-09-23T03:26:40.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.628+0000 7f8682ffd640 1 -- 172.21.15.154:0/239590944 <== mds.0 v2:172.21.15.154:6835/1758434763 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x5592420386c0 con 0x55924247c320 2023-09-23T03:26:40.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.628+0000 7f8682ffd640 10 client.8622 handle_command_reply: tid=0 2023-09-23T03:26:40.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.628+0000 7f86acfbe740 1 client.8622 shutdown 2023-09-23T03:26:40.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.628+0000 7f86acfbe740 1 -- 172.21.15.154:0/239590944 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592423f6c00 msgr2=0x5592423e2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:40.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.628+0000 7f86acfbe740 1 --2- 172.21.15.154:0/239590944 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592423f6c00 0x5592423e2d00 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x5592423f88e0 tx=0x7f8694004a40 comp rx=0 tx=0).stop 2023-09-23T03:26:40.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.628+0000 7f86acfbe740 1 -- 172.21.15.154:0/239590944 shutdown_connections 2023-09-23T03:26:40.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.628+0000 7f86acfbe740 1 --2- 172.21.15.154:0/239590944 >> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] conn(0x55924247c320 0x55924247c700 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:40.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.628+0000 7f86acfbe740 1 --2- 172.21.15.154:0/239590944 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x559242427cc0 0x5592423cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:40.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.628+0000 7f86acfbe740 1 --2- 172.21.15.154:0/239590944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5592423f6fe0 0x5592423cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:40.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.628+0000 7f86acfbe740 1 --2- 172.21.15.154:0/239590944 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5592423f6c00 0x5592423e2d00 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:40.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.628+0000 7f8682ffd640 0 client.8622 ms_handle_reset on v2:172.21.15.154:6835/1758434763 2023-09-23T03:26:40.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.628+0000 7f86acfbe740 1 -- 172.21.15.154:0/239590944 >> 172.21.15.154:0/239590944 conn(0x5592423ca040 msgr2=0x5592423fb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:40.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.628+0000 7f86acfbe740 1 -- 172.21.15.154:0/239590944 shutdown_connections 2023-09-23T03:26:40.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.628+0000 7f86acfbe740 1 -- 172.21.15.154:0/239590944 wait complete. 2023-09-23T03:26:40.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.628+0000 7f86acfbe740 20 client.8622 trim_cache size 0 max 16384 2023-09-23T03:26:40.634 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:26:40.634 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_fsid": "8c8039d3-6c84-43db-8808-85e0dbdd47c8", 2023-09-23T03:26:40.634 INFO:teuthology.orchestra.run.smithi154.stdout: "whoami": 0, 2023-09-23T03:26:40.634 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 8595, 2023-09-23T03:26:40.634 INFO:teuthology.orchestra.run.smithi154.stdout: "want_state": "up:active", 2023-09-23T03:26:40.634 INFO:teuthology.orchestra.run.smithi154.stdout: "state": "up:active", 2023-09-23T03:26:40.634 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs", 2023-09-23T03:26:40.634 INFO:teuthology.orchestra.run.smithi154.stdout: "rank_uptime": 6.206091647, 2023-09-23T03:26:40.634 INFO:teuthology.orchestra.run.smithi154.stdout: "mdsmap_epoch": 98, 2023-09-23T03:26:40.634 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_epoch": 114, 2023-09-23T03:26:40.635 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_epoch_barrier": 113, 2023-09-23T03:26:40.635 INFO:teuthology.orchestra.run.smithi154.stdout: "uptime": 6.9739808979999998 2023-09-23T03:26:40.635 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:26:40.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.629+0000 7f86aa355640 1 -- 172.21.15.154:0/3850365777 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f868406af30 msgr2=0x7f868406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:40.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.629+0000 7f86aa355640 1 --2- 172.21.15.154:0/3850365777 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f868406af30 0x7f868406d3f0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f8694009ce0 tx=0x7f869400b040 comp rx=0 tx=0).stop 2023-09-23T03:26:40.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.629+0000 7f86aa355640 1 -- 172.21.15.154:0/3850365777 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86a415c8e0 msgr2=0x7f86a40b00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:40.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.629+0000 7f86aa355640 1 --2- 172.21.15.154:0/3850365777 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86a415c8e0 0x7f86a40b00d0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f868c00c450 tx=0x7f868c00c910 comp rx=0 tx=0).stop 2023-09-23T03:26:40.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.629+0000 7f86aa355640 1 -- 172.21.15.154:0/3850365777 shutdown_connections 2023-09-23T03:26:40.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.629+0000 7f86aa355640 1 --2- 172.21.15.154:0/3850365777 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f868406af30 0x7f868406d3f0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:40.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.629+0000 7f86aa355640 1 --2- 172.21.15.154:0/3850365777 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f86a415ccc0 0x7f86a40c0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:40.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.629+0000 7f86aa355640 1 --2- 172.21.15.154:0/3850365777 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86a415c8e0 0x7f86a40b00d0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:40.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.629+0000 7f86aa355640 1 --2- 172.21.15.154:0/3850365777 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f86a40cc690 0x7f86a41f3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:40.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.629+0000 7f86aa355640 1 -- 172.21.15.154:0/3850365777 >> 172.21.15.154:0/3850365777 conn(0x7f86a40cf060 msgr2=0x7f86a4070290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:40.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.629+0000 7f86aa355640 1 -- 172.21.15.154:0/3850365777 shutdown_connections 2023-09-23T03:26:40.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.629+0000 7f86aa355640 1 -- 172.21.15.154:0/3850365777 wait complete. 2023-09-23T03:26:40.647 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:26:40.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.852+0000 7fa970844640 1 Processor -- start 2023-09-23T03:26:40.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.852+0000 7fa970844640 1 -- start start 2023-09-23T03:26:40.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.853+0000 7fa970844640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fa96c0b1b20 0x7fa96c0ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:40.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.854+0000 7fa970844640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa96c15ac30 con 0x7fa96c0cee70 2023-09-23T03:26:40.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.854+0000 7fa970844640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa96c0b1f70 con 0x7fa96c0acce0 2023-09-23T03:26:40.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.854+0000 7fa970844640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa96c0b2e90 con 0x7fa96c0b1b20 2023-09-23T03:26:40.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.854+0000 7fa96ad9d640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fa96c0cee70 0x7fa96c0ab140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:57596/0 (socket says 172.21.15.154:57596) 2023-09-23T03:26:40.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.854+0000 7fa96ad9d640 1 -- 172.21.15.154:0/165127282 learned_addr learned my addr 172.21.15.154:0/165127282 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:26:40.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.854+0000 7fa969d9b640 1 --2- 172.21.15.154:0/165127282 >> v2:172.21.15.190:3300/0 conn(0x7fa96c0b1b20 0x7fa96c0ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:40.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.854+0000 7fa96959a640 1 -- 172.21.15.154:0/165127282 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1122951838 0 0) 0x7fa964001490 con 0x7fa96c0cee70 2023-09-23T03:26:40.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.854+0000 7fa96959a640 1 -- 172.21.15.154:0/165127282 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa954003520 con 0x7fa96c0cee70 2023-09-23T03:26:40.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.855+0000 7fa96959a640 1 -- 172.21.15.154:0/165127282 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 629053764 0 0) 0x7fa954003520 con 0x7fa96c0cee70 2023-09-23T03:26:40.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.855+0000 7fa96959a640 1 -- 172.21.15.154:0/165127282 >> v2:172.21.15.190:3300/0 conn(0x7fa96c0b1b20 msgr2=0x7fa96c0ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:40.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.855+0000 7fa96959a640 1 --2- 172.21.15.154:0/165127282 >> v2:172.21.15.190:3300/0 conn(0x7fa96c0b1b20 0x7fa96c0ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:40.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.855+0000 7fa96959a640 1 -- 172.21.15.154:0/165127282 >> v1:172.21.15.190:6789/0 conn(0x7fa96c0acce0 legacy=0x7fa96c15ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:40.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.855+0000 7fa96959a640 1 -- 172.21.15.154:0/165127282 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa96c0b2210 con 0x7fa96c0cee70 2023-09-23T03:26:40.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.855+0000 7fa96959a640 1 -- 172.21.15.154:0/165127282 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fa964001b80 con 0x7fa96c0cee70 2023-09-23T03:26:40.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.855+0000 7fa96959a640 1 -- 172.21.15.154:0/165127282 >> v1:172.21.15.154:6789/0 conn(0x7fa96c0cee70 legacy=0x7fa96c0ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:40.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.855+0000 7fa96959a640 1 --2- 172.21.15.154:0/165127282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa954003970 0x7fa954003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:40.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.855+0000 7fa96959a640 1 --2- 172.21.15.154:0/165127282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9540047c0 0x7fa954004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:40.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.855+0000 7fa96959a640 1 --2- 172.21.15.154:0/165127282 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa954005180 0x7fa95400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:40.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.855+0000 7fa96959a640 1 -- 172.21.15.154:0/165127282 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c0b2e90 con 0x7fa954005180 2023-09-23T03:26:40.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.855+0000 7fa96959a640 1 -- 172.21.15.154:0/165127282 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95400c120 con 0x7fa954003970 2023-09-23T03:26:40.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.855+0000 7fa96959a640 1 -- 172.21.15.154:0/165127282 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa95400c2e0 con 0x7fa9540047c0 2023-09-23T03:26:40.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.855+0000 7fa96ad9d640 1 --2- 172.21.15.154:0/165127282 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa954005180 0x7fa95400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:40.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.855+0000 7fa96a59c640 1 --2- 172.21.15.154:0/165127282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa954003970 0x7fa954003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:40.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.855+0000 7fa96ad9d640 1 -- 172.21.15.154:0/165127282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9540047c0 msgr2=0x7fa954004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:40.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.855+0000 7fa96ad9d640 1 --2- 172.21.15.154:0/165127282 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa9540047c0 0x7fa954004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:40.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.855+0000 7fa96ad9d640 1 -- 172.21.15.154:0/165127282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa954003970 msgr2=0x7fa954003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:40.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.855+0000 7fa96ad9d640 1 --2- 172.21.15.154:0/165127282 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa954003970 0x7fa954003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:40.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.855+0000 7fa96ad9d640 1 -- 172.21.15.154:0/165127282 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa954003520 con 0x7fa954005180 2023-09-23T03:26:40.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.856+0000 7fa96ad9d640 1 -- 172.21.15.154:0/165127282 reap_dead start 2023-09-23T03:26:40.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.857+0000 7fa96ad9d640 1 --2- 172.21.15.154:0/165127282 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa954005180 0x7fa95400b980 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fa96400a830 tx=0x7fa9540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e3d2b96ab1e647 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:40.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.857+0000 7fa96959a640 1 -- 172.21.15.154:0/165127282 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa964034630 con 0x7fa954005180 2023-09-23T03:26:40.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.857+0000 7fa96959a640 1 -- 172.21.15.154:0/165127282 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa96402f070 con 0x7fa954005180 2023-09-23T03:26:40.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.857+0000 7fa96959a640 1 -- 172.21.15.154:0/165127282 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9640349f0 con 0x7fa954005180 2023-09-23T03:26:40.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.857+0000 7fa970844640 1 -- 172.21.15.154:0/165127282 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa954005180 msgr2=0x7fa95400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:40.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.857+0000 7fa970844640 1 --2- 172.21.15.154:0/165127282 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa954005180 0x7fa95400b980 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fa96400a830 tx=0x7fa9540057f0 comp rx=0 tx=0).stop 2023-09-23T03:26:40.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.857+0000 7fa970844640 1 -- 172.21.15.154:0/165127282 shutdown_connections 2023-09-23T03:26:40.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.857+0000 7fa970844640 1 --2- 172.21.15.154:0/165127282 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa954005180 0x7fa95400b980 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:40.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.857+0000 7fa970844640 1 -- 172.21.15.154:0/165127282 >> 172.21.15.154:0/165127282 conn(0x7fa96c06a0f0 msgr2=0x7fa96c06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:40.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.858+0000 7fa970844640 1 -- 172.21.15.154:0/165127282 shutdown_connections 2023-09-23T03:26:40.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.858+0000 7fa970844640 1 -- 172.21.15.154:0/165127282 wait complete. 2023-09-23T03:26:40.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.859+0000 7fa970844640 1 Processor -- start 2023-09-23T03:26:40.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.859+0000 7fa970844640 1 -- start start 2023-09-23T03:26:40.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.860+0000 7fa970844640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa954005180 0x7fa96c1537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:40.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.860+0000 7fa970844640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa96c15c520 0x7fa96c1f04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:40.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.860+0000 7fa970844640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa96c070cf0 0x7fa96c15b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:40.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.860+0000 7fa970844640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c0cf320 con 0x7fa954005180 2023-09-23T03:26:40.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.860+0000 7fa970844640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c154370 con 0x7fa96c15c520 2023-09-23T03:26:40.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.860+0000 7fa970844640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa96c154090 con 0x7fa96c070cf0 2023-09-23T03:26:40.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.861+0000 7fa96a59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa954005180 0x7fa96c1537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:40.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.861+0000 7fa96a59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa954005180 0x7fa96c1537a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:38806/0 (socket says 172.21.15.154:38806) 2023-09-23T03:26:40.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.861+0000 7fa96a59c640 1 -- 172.21.15.154:0/635506818 learned_addr learned my addr 172.21.15.154:0/635506818 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:40.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.861+0000 7fa96a59c640 1 -- 172.21.15.154:0/635506818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa96c070cf0 msgr2=0x7fa96c15b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:40.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.861+0000 7fa96a59c640 1 --2- 172.21.15.154:0/635506818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa96c070cf0 0x7fa96c15b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:40.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.861+0000 7fa96a59c640 1 -- 172.21.15.154:0/635506818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa96c15c520 msgr2=0x7fa96c1f04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:40.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.861+0000 7fa96a59c640 1 --2- 172.21.15.154:0/635506818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa96c15c520 0x7fa96c1f04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:40.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.861+0000 7fa96a59c640 1 -- 172.21.15.154:0/635506818 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa96c0c3aa0 con 0x7fa954005180 2023-09-23T03:26:40.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.861+0000 7fa96a59c640 1 --2- 172.21.15.154:0/635506818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa954005180 0x7fa96c1537a0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fa95c00d570 tx=0x7fa95c00da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:40.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.861+0000 7fa95affd640 1 -- 172.21.15.154:0/635506818 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95c005500 con 0x7fa954005180 2023-09-23T03:26:40.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.861+0000 7fa95affd640 1 -- 172.21.15.154:0/635506818 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa95c003660 con 0x7fa954005180 2023-09-23T03:26:40.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.861+0000 7fa95affd640 1 -- 172.21.15.154:0/635506818 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95c0039d0 con 0x7fa954005180 2023-09-23T03:26:40.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.861+0000 7fa970844640 1 -- 172.21.15.154:0/635506818 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa96c0c3d80 con 0x7fa954005180 2023-09-23T03:26:40.864 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.862+0000 7fa970844640 1 -- 172.21.15.154:0/635506818 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa96c153ce0 con 0x7fa954005180 2023-09-23T03:26:40.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.863+0000 7fa970844640 1 -- 172.21.15.154:0/635506818 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa96c073eb0 con 0x7fa954005180 2023-09-23T03:26:40.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.864+0000 7fa95affd640 1 -- 172.21.15.154:0/635506818 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fa95c003b30 con 0x7fa954005180 2023-09-23T03:26:40.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.864+0000 7fa95affd640 1 --2- 172.21.15.154:0/635506818 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa94c06af60 0x7fa94c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:40.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.865+0000 7fa95affd640 1 -- 172.21.15.154:0/635506818 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fa95c089920 con 0x7fa954005180 2023-09-23T03:26:40.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.865+0000 7fa969d9b640 1 --2- 172.21.15.154:0/635506818 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa94c06af60 0x7fa94c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:40.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.866+0000 7fa969d9b640 1 --2- 172.21.15.154:0/635506818 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa94c06af60 0x7fa94c06d420 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa960006a90 tx=0x7fa960006a20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:40.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:40.869+0000 7fa95affd640 1 -- 172.21.15.154:0/635506818 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fa95c0545f0 con 0x7fa954005180 2023-09-23T03:26:41.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.096+0000 7fa970844640 1 -- 172.21.15.154:0/635506818 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa96c06c930 con 0x7fa954005180 2023-09-23T03:26:41.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.097+0000 7fa95affd640 1 -- 172.21.15.154:0/635506818 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4926 (secure 0 0 0) 0x7fa95c057c10 con 0x7fa954005180 2023-09-23T03:26:41.098 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:26:41.099 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":99,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5753,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/60017546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":60017546},{"type":"v1","addr":"172.21.15.190:6833","nonce":60017546}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95},{"gid":6415,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2708068523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2708068523},{"type":"v1","addr":"172.21.15.190:6835","nonce":2708068523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":99},{"gid":8592,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/1846547753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1846547753},{"type":"v1","addr":"172.21.15.154:6836","nonce":1846547753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:26:26.774375+0000","modified":"2023-09-23T03:26:36.432946+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8595":{"gid":8595,"name":"a","rank":0,"incarnation":95,"state":"up:active","state_seq":4,"addr":"172.21.15.154:6837/1758434763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1758434763},{"type":"v1","addr":"172.21.15.154:6837","nonce":1758434763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15,16],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-09-23T03:26:41.099 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 99 2023-09-23T03:26:41.101 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.099+0000 7fa970844640 1 -- 172.21.15.154:0/635506818 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa94c06af60 msgr2=0x7fa94c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:41.101 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.099+0000 7fa970844640 1 --2- 172.21.15.154:0/635506818 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa94c06af60 0x7fa94c06d420 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa960006a90 tx=0x7fa960006a20 comp rx=0 tx=0).stop 2023-09-23T03:26:41.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.100+0000 7fa970844640 1 -- 172.21.15.154:0/635506818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa954005180 msgr2=0x7fa96c1537a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:41.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.100+0000 7fa970844640 1 --2- 172.21.15.154:0/635506818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa954005180 0x7fa96c1537a0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fa95c00d570 tx=0x7fa95c00da30 comp rx=0 tx=0).stop 2023-09-23T03:26:41.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.100+0000 7fa970844640 1 -- 172.21.15.154:0/635506818 shutdown_connections 2023-09-23T03:26:41.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.100+0000 7fa970844640 1 --2- 172.21.15.154:0/635506818 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa94c06af60 0x7fa94c06d420 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:41.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.100+0000 7fa970844640 1 --2- 172.21.15.154:0/635506818 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa96c070cf0 0x7fa96c15b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:41.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.100+0000 7fa970844640 1 --2- 172.21.15.154:0/635506818 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa96c15c520 0x7fa96c1f04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:41.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.100+0000 7fa970844640 1 --2- 172.21.15.154:0/635506818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa954005180 0x7fa96c1537a0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:41.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.100+0000 7fa970844640 1 -- 172.21.15.154:0/635506818 >> 172.21.15.154:0/635506818 conn(0x7fa96c06a0f0 msgr2=0x7fa96c0705a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:41.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.100+0000 7fa970844640 1 -- 172.21.15.154:0/635506818 shutdown_connections 2023-09-23T03:26:41.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.100+0000 7fa970844640 1 -- 172.21.15.154:0/635506818 wait complete. 2023-09-23T03:26:41.120 INFO:tasks.cephfs.test_data_scan:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 5, "epoch": 99, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 5, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-09-23T03:26:26.774375+0000", "damaged": [], "data_pools": [ 15, 16 ], "enabled": true, "epoch": 98, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_8595": { "addr": "172.21.15.154:6837/1758434763", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6835", "nonce": 1758434763, "type": "v2" }, { "addr": "172.21.15.154:6837", "nonce": 1758434763, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 8595, "incarnation": 95, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 4 } }, "last_failure": 0, "last_failure_osd_epoch": 112, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 14, "modified": "2023-09-23T03:26:36.432946+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 8595 } } } ], "standbys": [ { "addr": "172.21.15.190:6833/60017546", "addrs": { "addrvec": [ { "addr": "172.21.15.190:6832", "nonce": 60017546, "type": "v2" }, { "addr": "172.21.15.190:6833", "nonce": 60017546, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 95, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5753, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.190:6835/2708068523", "addrs": { "addrvec": [ { "addr": "172.21.15.190:6834", "nonce": 2708068523, "type": "v2" }, { "addr": "172.21.15.190:6835", "nonce": 2708068523, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 99, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6415, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.154:6836/1846547753", "addrs": { "addrvec": [ { "addr": "172.21.15.154:6834", "nonce": 1846547753, "type": "v2" }, { "addr": "172.21.15.154:6836", "nonce": 1846547753, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 94, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 8592, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-09-23T03:26:41.121 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-23T03:26:41.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.331+0000 7f6c58df5640 1 Processor -- start 2023-09-23T03:26:41.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.331+0000 7f6c58df5640 1 -- start start 2023-09-23T03:26:41.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.332+0000 7f6c58df5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6c540b1b20 0x7f6c540ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:41.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.332+0000 7f6c58df5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c540cee70 0x7f6c540ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:41.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.332+0000 7f6c58df5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c5415ac30 con 0x7f6c540acce0 2023-09-23T03:26:41.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.332+0000 7f6c58df5640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c540b2e90 con 0x7f6c540b1b20 2023-09-23T03:26:41.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.332+0000 7f6c58df5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c540cf320 con 0x7f6c540cee70 2023-09-23T03:26:41.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.333+0000 7f6c4bfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6c540b1b20 0x7f6c540ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:41.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.333+0000 7f6c5259c640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f6c540acce0 0x7f6c5415ed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:57608/0 (socket says 172.21.15.154:57608) 2023-09-23T03:26:41.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.333+0000 7f6c5259c640 1 -- 172.21.15.154:0/2436748918 learned_addr learned my addr 172.21.15.154:0/2436748918 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:26:41.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.333+0000 7f6c4bfff640 1 -- 172.21.15.154:0/2436748918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c540cee70 msgr2=0x7f6c540ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:41.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.333+0000 7f6c52d9d640 1 --2- 172.21.15.154:0/2436748918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c540cee70 0x7f6c540ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:41.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.333+0000 7f6c51d9b640 1 -- 172.21.15.154:0/2436748918 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1140005361 0 0) 0x7f6c44001290 con 0x7f6c540acce0 2023-09-23T03:26:41.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.333+0000 7f6c4bfff640 1 --2- 172.21.15.154:0/2436748918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c540cee70 0x7f6c540ab140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:41.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.333+0000 7f6c4bfff640 1 -- 172.21.15.154:0/2436748918 >> v1:172.21.15.154:6789/0 conn(0x7f6c540acce0 legacy=0x7f6c5415ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:41.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.333+0000 7f6c4bfff640 1 -- 172.21.15.154:0/2436748918 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c540b1f70 con 0x7f6c540b1b20 2023-09-23T03:26:41.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.333+0000 7f6c52d9d640 1 --2- 172.21.15.154:0/2436748918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c540cee70 0x7f6c540ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:26:41.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.335+0000 7f6c4bfff640 1 --2- 172.21.15.154:0/2436748918 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c540b1b20 0x7f6c540ce930 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c0097a0 tx=0x7f6c3c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f0f9c43d77b4f5a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:41.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.335+0000 7f6c51d9b640 1 -- 172.21.15.154:0/2436748918 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c031780 con 0x7f6c540b1b20 2023-09-23T03:26:41.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.335+0000 7f6c51d9b640 1 -- 172.21.15.154:0/2436748918 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3c0318e0 con 0x7f6c540b1b20 2023-09-23T03:26:41.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.335+0000 7f6c51d9b640 1 -- 172.21.15.154:0/2436748918 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c031c70 con 0x7f6c540b1b20 2023-09-23T03:26:41.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.335+0000 7f6c58df5640 1 -- 172.21.15.154:0/2436748918 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c540b1b20 msgr2=0x7f6c540ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:41.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.335+0000 7f6c58df5640 1 --2- 172.21.15.154:0/2436748918 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c540b1b20 0x7f6c540ce930 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c0097a0 tx=0x7f6c3c02b820 comp rx=0 tx=0).stop 2023-09-23T03:26:41.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.335+0000 7f6c58df5640 1 -- 172.21.15.154:0/2436748918 shutdown_connections 2023-09-23T03:26:41.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.335+0000 7f6c58df5640 1 --2- 172.21.15.154:0/2436748918 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c540cee70 0x7f6c540ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:41.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.335+0000 7f6c58df5640 1 --2- 172.21.15.154:0/2436748918 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c540b1b20 0x7f6c540ce930 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:41.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.335+0000 7f6c58df5640 1 -- 172.21.15.154:0/2436748918 >> 172.21.15.154:0/2436748918 conn(0x7f6c5406a0f0 msgr2=0x7f6c5406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:41.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.336+0000 7f6c58df5640 1 -- 172.21.15.154:0/2436748918 shutdown_connections 2023-09-23T03:26:41.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.336+0000 7f6c58df5640 1 -- 172.21.15.154:0/2436748918 wait complete. 2023-09-23T03:26:41.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.337+0000 7f6c58df5640 1 Processor -- start 2023-09-23T03:26:41.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.337+0000 7f6c58df5640 1 -- start start 2023-09-23T03:26:41.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.338+0000 7f6c58df5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c540acce0 0x7f6c54158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:41.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.338+0000 7f6c58df5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c540b1b20 0x7f6c541533a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:41.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.338+0000 7f6c58df5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c540cee70 0x7f6c540c5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:41.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.338+0000 7f6c58df5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c54158bd0 con 0x7f6c540cee70 2023-09-23T03:26:41.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.338+0000 7f6c58df5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c541538e0 con 0x7f6c540acce0 2023-09-23T03:26:41.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.338+0000 7f6c58df5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c54157f80 con 0x7f6c540b1b20 2023-09-23T03:26:41.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.339+0000 7f6c52d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c540cee70 0x7f6c540c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:41.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.339+0000 7f6c52d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c540cee70 0x7f6c540c5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:38824/0 (socket says 172.21.15.154:38824) 2023-09-23T03:26:41.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.339+0000 7f6c52d9d640 1 -- 172.21.15.154:0/3675742011 learned_addr learned my addr 172.21.15.154:0/3675742011 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:41.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.339+0000 7f6c4bfff640 1 --2- 172.21.15.154:0/3675742011 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c540b1b20 0x7f6c541533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:41.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.339+0000 7f6c52d9d640 1 -- 172.21.15.154:0/3675742011 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c540b1b20 msgr2=0x7f6c541533a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:41.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.339+0000 7f6c52d9d640 1 --2- 172.21.15.154:0/3675742011 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c540b1b20 0x7f6c541533a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:41.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.339+0000 7f6c52d9d640 1 -- 172.21.15.154:0/3675742011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c540acce0 msgr2=0x7f6c54158690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:41.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.339+0000 7f6c52d9d640 1 --2- 172.21.15.154:0/3675742011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c540acce0 0x7f6c54158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:41.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.339+0000 7f6c52d9d640 1 -- 172.21.15.154:0/3675742011 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3c009400 con 0x7f6c540cee70 2023-09-23T03:26:41.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.339+0000 7f6c52d9d640 1 --2- 172.21.15.154:0/3675742011 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c540cee70 0x7f6c540c5850 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00cd50 tx=0x7f6c4c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:41.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.340+0000 7f6c4affd640 1 -- 172.21.15.154:0/3675742011 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c00f680 con 0x7f6c540cee70 2023-09-23T03:26:41.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.340+0000 7f6c4affd640 1 -- 172.21.15.154:0/3675742011 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4c00f800 con 0x7f6c540cee70 2023-09-23T03:26:41.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.340+0000 7f6c4affd640 1 -- 172.21.15.154:0/3675742011 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c00fbb0 con 0x7f6c540cee70 2023-09-23T03:26:41.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.340+0000 7f6c58df5640 1 -- 172.21.15.154:0/3675742011 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c540c7cf0 con 0x7f6c540cee70 2023-09-23T03:26:41.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.340+0000 7f6c58df5640 1 -- 172.21.15.154:0/3675742011 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c540c82b0 con 0x7f6c540cee70 2023-09-23T03:26:41.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.341+0000 7f6c58df5640 1 -- 172.21.15.154:0/3675742011 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c5406c640 con 0x7f6c540cee70 2023-09-23T03:26:41.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.342+0000 7f6c4affd640 1 -- 172.21.15.154:0/3675742011 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f6c4c00b040 con 0x7f6c540cee70 2023-09-23T03:26:41.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.343+0000 7f6c4affd640 1 --2- 172.21.15.154:0/3675742011 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6c3406af30 0x7f6c3406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:41.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.343+0000 7f6c4affd640 1 -- 172.21.15.154:0/3675742011 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f6c4c08a580 con 0x7f6c540cee70 2023-09-23T03:26:41.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.343+0000 7f6c5259c640 1 --2- 172.21.15.154:0/3675742011 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6c3406af30 0x7f6c3406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:41.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.344+0000 7f6c5259c640 1 --2- 172.21.15.154:0/3675742011 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6c3406af30 0x7f6c3406d3f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f6c440015c0 tx=0x7f6c44001350 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:41.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.347+0000 7f6c4affd640 1 -- 172.21.15.154:0/3675742011 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f6c4c0551d0 con 0x7f6c540cee70 2023-09-23T03:26:41.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.561+0000 7f6c58df5640 1 -- 172.21.15.154:0/3675742011 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6c54073eb0 con 0x7f6c540cee70 2023-09-23T03:26:41.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.562+0000 7f6c4affd640 1 -- 172.21.15.154:0/3675742011 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v99) v1 ==== 53+0+91 (secure 0 0 0) 0x7f6c4c0587f0 con 0x7f6c540cee70 2023-09-23T03:26:41.563 INFO:teuthology.orchestra.run.smithi154.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_extra ] 2023-09-23T03:26:41.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.564+0000 7f6c58df5640 1 -- 172.21.15.154:0/3675742011 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6c3406af30 msgr2=0x7f6c3406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:41.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.564+0000 7f6c58df5640 1 --2- 172.21.15.154:0/3675742011 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6c3406af30 0x7f6c3406d3f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f6c440015c0 tx=0x7f6c44001350 comp rx=0 tx=0).stop 2023-09-23T03:26:41.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.564+0000 7f6c58df5640 1 -- 172.21.15.154:0/3675742011 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c540cee70 msgr2=0x7f6c540c5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:41.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.564+0000 7f6c58df5640 1 --2- 172.21.15.154:0/3675742011 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c540cee70 0x7f6c540c5850 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00cd50 tx=0x7f6c4c00a650 comp rx=0 tx=0).stop 2023-09-23T03:26:41.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.564+0000 7f6c58df5640 1 -- 172.21.15.154:0/3675742011 shutdown_connections 2023-09-23T03:26:41.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.564+0000 7f6c58df5640 1 --2- 172.21.15.154:0/3675742011 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6c3406af30 0x7f6c3406d3f0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:41.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.564+0000 7f6c58df5640 1 --2- 172.21.15.154:0/3675742011 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c540cee70 0x7f6c540c5850 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:41.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.564+0000 7f6c58df5640 1 --2- 172.21.15.154:0/3675742011 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c540b1b20 0x7f6c541533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:41.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.564+0000 7f6c58df5640 1 --2- 172.21.15.154:0/3675742011 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c540acce0 0x7f6c54158690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:41.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.564+0000 7f6c58df5640 1 -- 172.21.15.154:0/3675742011 >> 172.21.15.154:0/3675742011 conn(0x7f6c5406a0f0 msgr2=0x7f6c5415a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:41.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.564+0000 7f6c58df5640 1 -- 172.21.15.154:0/3675742011 shutdown_connections 2023-09-23T03:26:41.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:41.564+0000 7f6c58df5640 1 -- 172.21.15.154:0/3675742011 wait complete. 2023-09-23T03:26:41.583 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-23T03:26:41.584 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-23T03:26:41.584 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi154.front.sepia.ceph.com 2023-09-23T03:26:41.584 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-09-23T03:26:41.584 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-23T03:26:41.584 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-23T03:26:41.584 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-23T03:26:41.584 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-23T03:26:41.584 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-23T03:26:41.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:26:41.584 DEBUG:teuthology.orchestra.run.smithi154:> ip addr 2023-09-23T03:26:41.634 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-23T03:26:41.635 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-23T03:26:41.635 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2023-09-23T03:26:41.635 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:26:41.635 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2023-09-23T03:26:41.635 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:26:41.635 INFO:teuthology.orchestra.run.smithi154.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:26:41.635 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:54 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:26:41.635 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f0 2023-09-23T03:26:41.635 INFO:teuthology.orchestra.run.smithi154.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:26:41.635 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:55 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:26:41.636 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f1 2023-09-23T03:26:41.636 INFO:teuthology.orchestra.run.smithi154.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:26:41.636 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:60 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:26:41.636 INFO:teuthology.orchestra.run.smithi154.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-23T03:26:41.636 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:61 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:26:41.636 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.21.15.154/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-09-23T03:26:41.636 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft 41723sec preferred_lft 41723sec 2023-09-23T03:26:41.636 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::ec4:7aff:fe88:7761/64 scope link 2023-09-23T03:26:41.636 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:26:41.636 INFO:teuthology.orchestra.run.smithi154.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-23T03:26:41.636 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 5a:c6:b2:96:fb:dc brd ff:ff:ff:ff:ff:ff 2023-09-23T03:26:41.636 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-09-23T03:26:41.637 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:26:41.637 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::a801:22ff:febf:2833/64 scope link 2023-09-23T03:26:41.637 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:26:41.637 INFO:teuthology.orchestra.run.smithi154.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-23T03:26:41.637 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 5a:c6:b2:96:fb:dc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:26:41.637 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::58c6:b2ff:fe96:fbdc/64 scope link 2023-09-23T03:26:41.637 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:26:41.637 INFO:teuthology.orchestra.run.smithi154.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-23T03:26:41.637 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether b2:23:b0:e4:c2:ab brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:26:41.637 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::b023:b0ff:fee4:c2ab/64 scope link 2023-09-23T03:26:41.637 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:26:41.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:26:41.638 DEBUG:teuthology.orchestra.run.smithi154:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-23T03:26:41.722 INFO:teuthology.orchestra.run.smithi154.stdout:1 2023-09-23T03:26:41.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:26:41.726 DEBUG:teuthology.orchestra.run.smithi154:> route 2023-09-23T03:26:41.780 INFO:teuthology.orchestra.run.smithi154.stdout:Kernel IP routing table 2023-09-23T03:26:41.780 INFO:teuthology.orchestra.run.smithi154.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-23T03:26:41.780 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-23T03:26:41.780 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-09-23T03:26:41.780 INFO:teuthology.orchestra.run.smithi154.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-23T03:26:41.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:26:41.781 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-09-23T03:26:41.781 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-09-23T03:26:41.781 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-23T03:26:41.781 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-23T03:26:41.781 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-23T03:26:41.781 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-09-23T03:26:41.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:26:41.961 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2023-09-23T03:26:42.012 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-09-23T03:26:42.012 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-09-23T03:26:42.013 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-23T03:26:42.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:26:42.013 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set brx.0 up 2023-09-23T03:26:42.108 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-09-23T03:26:42.108 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-23T03:26:42.108 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-09-23T03:26:42.163 INFO:teuthology.orchestra.run.smithi154.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-09-23T03:26:42.163 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-23T03:26:42.163 DEBUG:teuthology.orchestra.run.smithi154:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-09-23T03:26:42.215 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-09-23T03:26:42.315 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2023-09-23T03:26:42.373 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2023-09-23T03:26:42.373 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2023-09-23T03:26:42.373 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2023-09-23T03:26:42.374 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2023-09-23T03:26:42.374 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2023-09-23T03:26:42.374 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2023-09-23T03:26:42.374 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2023-09-23T03:26:42.374 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2023-09-23T03:26:42.374 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2023-09-23T03:26:42.374 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2023-09-23T03:26:42.374 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2023-09-23T03:26:42.374 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T03:26:42.374 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2023-09-23T03:26:42.374 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2023-09-23T03:26:42.375 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2023-09-23T03:26:42.375 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2023-09-23T03:26:42.375 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2023-09-23T03:26:42.375 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2023-09-23T03:26:42.375 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2023-09-23T03:26:42.375 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-09-23T03:26:42.375 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-09-23T03:26:42.375 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2023-09-23T03:26:42.375 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2023-09-23T03:26:42.375 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2023-09-23T03:26:42.375 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-0 2023-09-23T03:26:42.375 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-1 2023-09-23T03:26:42.376 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-2 2023-09-23T03:26:42.376 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-3 2023-09-23T03:26:42.376 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2023-09-23T03:26:42.376 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:26:42.376 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:26:42.376 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:26:42.376 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:26:42.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:26:42.377 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-09-23T03:26:42.429 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-23T03:26:42.430 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-09-23T03:26:42.434 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-09-23T03:26:42.519 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2023-09-23T03:26:42.566 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2023-09-23T03:26:42.566 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2023-09-23T03:26:42.566 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2023-09-23T03:26:42.566 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2023-09-23T03:26:42.566 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2023-09-23T03:26:42.566 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2023-09-23T03:26:42.566 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2023-09-23T03:26:42.566 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2023-09-23T03:26:42.566 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2023-09-23T03:26:42.566 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2023-09-23T03:26:42.567 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2023-09-23T03:26:42.567 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T03:26:42.567 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2023-09-23T03:26:42.567 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2023-09-23T03:26:42.567 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2023-09-23T03:26:42.567 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2023-09-23T03:26:42.567 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2023-09-23T03:26:42.567 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2023-09-23T03:26:42.567 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2023-09-23T03:26:42.567 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-09-23T03:26:42.567 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-09-23T03:26:42.567 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2023-09-23T03:26:42.568 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2023-09-23T03:26:42.568 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2023-09-23T03:26:42.568 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-0 2023-09-23T03:26:42.568 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-1 2023-09-23T03:26:42.568 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-2 2023-09-23T03:26:42.568 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-3 2023-09-23T03:26:42.568 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2023-09-23T03:26:42.568 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:26:42.568 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:26:42.568 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:26:42.568 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:26:42.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:26:42.569 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-09-23T03:26:42.630 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:2023-09-23T03:26:42.629+0000 7f597b44b480 -1 init, newargv = 0x55795544abc0 newargc=15 2023-09-23T03:26:42.630 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:ceph-fuse[123940]: starting ceph client 2023-09-23T03:26:42.638 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:ceph-fuse[123940]: starting fuse 2023-09-23T03:26:43.711 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-09-23T03:26:43.788 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-09-23T03:26:43.817 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2023-09-23T03:26:43.875 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2023-09-23T03:26:43.875 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2023-09-23T03:26:43.875 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2023-09-23T03:26:43.875 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2023-09-23T03:26:43.875 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2023-09-23T03:26:43.875 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2023-09-23T03:26:43.875 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2023-09-23T03:26:43.875 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2023-09-23T03:26:43.875 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2023-09-23T03:26:43.876 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2023-09-23T03:26:43.876 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2023-09-23T03:26:43.876 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T03:26:43.876 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2023-09-23T03:26:43.876 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2023-09-23T03:26:43.876 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2023-09-23T03:26:43.876 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2023-09-23T03:26:43.876 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2023-09-23T03:26:43.876 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2023-09-23T03:26:43.876 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2023-09-23T03:26:43.876 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-09-23T03:26:43.876 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-09-23T03:26:43.876 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2023-09-23T03:26:43.877 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2023-09-23T03:26:43.877 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2023-09-23T03:26:43.877 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-0 2023-09-23T03:26:43.877 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-1 2023-09-23T03:26:43.877 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-2 2023-09-23T03:26:43.877 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-3 2023-09-23T03:26:43.877 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2023-09-23T03:26:43.877 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:26:43.877 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:26:43.877 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:26:43.877 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:26:43.877 INFO:teuthology.orchestra.run.smithi154.stdout:/home/ubuntu/cephtest/mnt.0 2023-09-23T03:26:43.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:26:43.878 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-09-23T03:26:43.929 INFO:teuthology.orchestra.run.smithi154.stdout:39 2023-09-23T03:26:43.930 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-09-23T03:26:43.930 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:26:43.930 DEBUG:teuthology.orchestra.run.smithi154:> import glob 2023-09-23T03:26:43.930 DEBUG:teuthology.orchestra.run.smithi154:> import re 2023-09-23T03:26:43.930 DEBUG:teuthology.orchestra.run.smithi154:> import os 2023-09-23T03:26:43.930 DEBUG:teuthology.orchestra.run.smithi154:> import subprocess 2023-09-23T03:26:43.930 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:26:43.930 DEBUG:teuthology.orchestra.run.smithi154:> def _find_admin_socket(client_name): 2023-09-23T03:26:43.930 DEBUG:teuthology.orchestra.run.smithi154:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-09-23T03:26:43.930 DEBUG:teuthology.orchestra.run.smithi154:> files = glob.glob(asok_path) 2023-09-23T03:26:43.931 DEBUG:teuthology.orchestra.run.smithi154:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-09-23T03:26:43.931 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:26:43.931 DEBUG:teuthology.orchestra.run.smithi154:> # Given a non-glob path, it better be there 2023-09-23T03:26:43.931 DEBUG:teuthology.orchestra.run.smithi154:> if "*" not in asok_path: 2023-09-23T03:26:43.931 DEBUG:teuthology.orchestra.run.smithi154:> assert(len(files) == 1) 2023-09-23T03:26:43.931 DEBUG:teuthology.orchestra.run.smithi154:> return files[0] 2023-09-23T03:26:43.931 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:26:43.931 DEBUG:teuthology.orchestra.run.smithi154:> for f in files: 2023-09-23T03:26:43.931 DEBUG:teuthology.orchestra.run.smithi154:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-23T03:26:43.931 DEBUG:teuthology.orchestra.run.smithi154:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-23T03:26:43.931 DEBUG:teuthology.orchestra.run.smithi154:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-23T03:26:43.931 DEBUG:teuthology.orchestra.run.smithi154:> contents = proc_f.read() 2023-09-23T03:26:43.931 DEBUG:teuthology.orchestra.run.smithi154:> if mountpoint in contents: 2023-09-23T03:26:43.932 DEBUG:teuthology.orchestra.run.smithi154:> return f 2023-09-23T03:26:43.932 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-23T03:26:43.932 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:26:43.932 DEBUG:teuthology.orchestra.run.smithi154:> print(_find_admin_socket("client.0")) 2023-09-23T03:26:43.932 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:26:44.110 INFO:teuthology.orchestra.run.smithi154.stdout:/var/run/ceph/ceph-client.0.123940.asok 2023-09-23T03:26:44.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:26:44.128 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.123940.asok 2023-09-23T03:26:44.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:26:44.128 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.123940.asok status 2023-09-23T03:26:44.196 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:26:44.224 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:26:44.371 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:26:44.371 INFO:teuthology.orchestra.run.smithi154.stdout: "metadata": { 2023-09-23T03:26:44.371 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_sha1": "862663913a74a99e1c52d50e6a7e59dd6705749a", 2023-09-23T03:26:44.371 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_version": "ceph version 18.0.0-6336-g86266391 (862663913a74a99e1c52d50e6a7e59dd6705749a) reef (dev)", 2023-09-23T03:26:44.371 INFO:teuthology.orchestra.run.smithi154.stdout: "entity_id": "0", 2023-09-23T03:26:44.371 INFO:teuthology.orchestra.run.smithi154.stdout: "hostname": "smithi154", 2023-09-23T03:26:44.371 INFO:teuthology.orchestra.run.smithi154.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-09-23T03:26:44.371 INFO:teuthology.orchestra.run.smithi154.stdout: "pid": "123940", 2023-09-23T03:26:44.371 INFO:teuthology.orchestra.run.smithi154.stdout: "root": "/" 2023-09-23T03:26:44.371 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:26:44.371 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_count": 0, 2023-09-23T03:26:44.372 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_pinned_count": 0, 2023-09-23T03:26:44.372 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 8643, 2023-09-23T03:26:44.372 INFO:teuthology.orchestra.run.smithi154.stdout: "inst": { 2023-09-23T03:26:44.372 INFO:teuthology.orchestra.run.smithi154.stdout: "name": { 2023-09-23T03:26:44.372 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "client", 2023-09-23T03:26:44.372 INFO:teuthology.orchestra.run.smithi154.stdout: "num": 8643 2023-09-23T03:26:44.372 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:26:44.372 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2023-09-23T03:26:44.372 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2023-09-23T03:26:44.372 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.1:0", 2023-09-23T03:26:44.372 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 4067852711 2023-09-23T03:26:44.372 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:26:44.373 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:26:44.373 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2023-09-23T03:26:44.373 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2023-09-23T03:26:44.373 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.1:0", 2023-09-23T03:26:44.373 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 4067852711 2023-09-23T03:26:44.373 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:26:44.373 INFO:teuthology.orchestra.run.smithi154.stdout: "inst_str": "client.8643 192.168.0.1:0/4067852711", 2023-09-23T03:26:44.373 INFO:teuthology.orchestra.run.smithi154.stdout: "addr_str": "192.168.0.1:0/4067852711", 2023-09-23T03:26:44.373 INFO:teuthology.orchestra.run.smithi154.stdout: "inode_count": 1, 2023-09-23T03:26:44.373 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_epoch": 98, 2023-09-23T03:26:44.373 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch": 114, 2023-09-23T03:26:44.373 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch_barrier": 0, 2023-09-23T03:26:44.374 INFO:teuthology.orchestra.run.smithi154.stdout: "blocklisted": false, 2023-09-23T03:26:44.374 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs" 2023-09-23T03:26:44.374 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:26:44.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:26:44.385 DEBUG:teuthology.orchestra.run.smithi154:> stat --file-system '--printf=%T 2023-09-23T03:26:44.385 DEBUG:teuthology.orchestra.run.smithi154:> ' -- /home/ubuntu/cephtest/mnt.0 2023-09-23T03:26:44.485 INFO:teuthology.orchestra.run.smithi154.stdout:fuseblk 2023-09-23T03:26:44.485 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-09-23T03:26:44.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:26:44.486 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-09-23T03:26:44.581 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 scrub start / repair,recursive 2023-09-23T03:26:44.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.806+0000 7fce7df59640 1 Processor -- start 2023-09-23T03:26:44.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.806+0000 7fce7df59640 1 -- start start 2023-09-23T03:26:44.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.807+0000 7fce7df59640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce7815a410 0x7fce780cc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:44.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.807+0000 7fce7df59640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fce780ce270 0x7fce780ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:44.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.807+0000 7fce7df59640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce780cec00 con 0x7fce7815dc00 2023-09-23T03:26:44.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.807+0000 7fce7df59640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce7806cbf0 con 0x7fce780ce270 2023-09-23T03:26:44.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.807+0000 7fce7df59640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce780b2bc0 con 0x7fce7815a410 2023-09-23T03:26:44.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.808+0000 7fce77fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fce7815dc00 0x7fce780b2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:55426/0 (socket says 172.21.15.154:55426) 2023-09-23T03:26:44.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.808+0000 7fce77fff640 1 -- 172.21.15.154:0/443432364 learned_addr learned my addr 172.21.15.154:0/443432364 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:26:44.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.808+0000 7fce7d758640 1 --2- 172.21.15.154:0/443432364 >> v2:172.21.15.190:3300/0 conn(0x7fce780ce270 0x7fce780ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:44.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.808+0000 7fce7cf57640 1 --2- 172.21.15.154:0/443432364 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce7815a410 0x7fce780cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:44.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.808+0000 7fce777fe640 1 -- 172.21.15.154:0/443432364 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1915920819 0 0) 0x7fce780cec00 con 0x7fce7815dc00 2023-09-23T03:26:44.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.808+0000 7fce777fe640 1 -- 172.21.15.154:0/443432364 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce68003520 con 0x7fce7815dc00 2023-09-23T03:26:44.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.809+0000 7fce777fe640 1 -- 172.21.15.154:0/443432364 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 502456137 0 0) 0x7fce68003520 con 0x7fce7815dc00 2023-09-23T03:26:44.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.809+0000 7fce777fe640 1 -- 172.21.15.154:0/443432364 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce7815a410 msgr2=0x7fce780cc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:44.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.809+0000 7fce777fe640 1 --2- 172.21.15.154:0/443432364 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce7815a410 0x7fce780cc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:44.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.809+0000 7fce777fe640 1 -- 172.21.15.154:0/443432364 >> v2:172.21.15.190:3300/0 conn(0x7fce780ce270 msgr2=0x7fce780ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:44.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.809+0000 7fce777fe640 1 --2- 172.21.15.154:0/443432364 >> v2:172.21.15.190:3300/0 conn(0x7fce780ce270 0x7fce780ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:44.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.809+0000 7fce777fe640 1 -- 172.21.15.154:0/443432364 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce780b2ea0 con 0x7fce7815dc00 2023-09-23T03:26:44.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.809+0000 7fce777fe640 1 -- 172.21.15.154:0/443432364 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fce64001670 con 0x7fce7815dc00 2023-09-23T03:26:44.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.809+0000 7fce777fe640 1 -- 172.21.15.154:0/443432364 >> v1:172.21.15.154:6789/0 conn(0x7fce7815dc00 legacy=0x7fce780b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:44.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.809+0000 7fce777fe640 1 --2- 172.21.15.154:0/443432364 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce7815a410 0x7fce68003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:44.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.809+0000 7fce777fe640 1 --2- 172.21.15.154:0/443432364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce68003ef0 0x7fce680047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:44.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.809+0000 7fce777fe640 1 --2- 172.21.15.154:0/443432364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce68004d00 0x7fce68005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:44.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.809+0000 7fce777fe640 1 -- 172.21.15.154:0/443432364 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7806cbf0 con 0x7fce68004d00 2023-09-23T03:26:44.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.809+0000 7fce777fe640 1 -- 172.21.15.154:0/443432364 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce780b2bc0 con 0x7fce68003ef0 2023-09-23T03:26:44.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.809+0000 7fce777fe640 1 -- 172.21.15.154:0/443432364 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce6800b9d0 con 0x7fce7815a410 2023-09-23T03:26:44.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.809+0000 7fce7d758640 1 --2- 172.21.15.154:0/443432364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce68004d00 0x7fce68005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:44.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.809+0000 7fce7d758640 1 -- 172.21.15.154:0/443432364 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce7815a410 msgr2=0x7fce68003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:44.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.809+0000 7fce7d758640 1 --2- 172.21.15.154:0/443432364 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce7815a410 0x7fce68003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:44.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.809+0000 7fce7d758640 1 -- 172.21.15.154:0/443432364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce68003ef0 msgr2=0x7fce680047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:44.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.809+0000 7fce7d758640 1 --2- 172.21.15.154:0/443432364 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce68003ef0 0x7fce680047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:44.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.809+0000 7fce7d758640 1 -- 172.21.15.154:0/443432364 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce68003520 con 0x7fce68004d00 2023-09-23T03:26:44.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.811+0000 7fce7d758640 1 --2- 172.21.15.154:0/443432364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce68004d00 0x7fce68005180 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fce7000e170 tx=0x7fce680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad72ea82713b5054 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:44.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.811+0000 7fce777fe640 1 -- 172.21.15.154:0/443432364 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7002cc20 con 0x7fce68004d00 2023-09-23T03:26:44.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.811+0000 7fce777fe640 1 -- 172.21.15.154:0/443432364 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7000f040 con 0x7fce68004d00 2023-09-23T03:26:44.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.811+0000 7fce777fe640 1 -- 172.21.15.154:0/443432364 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce70034440 con 0x7fce68004d00 2023-09-23T03:26:44.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.812+0000 7fce7df59640 1 -- 172.21.15.154:0/443432364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce68004d00 msgr2=0x7fce68005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:44.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.812+0000 7fce7df59640 1 --2- 172.21.15.154:0/443432364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce68004d00 0x7fce68005180 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fce7000e170 tx=0x7fce680057f0 comp rx=0 tx=0).stop 2023-09-23T03:26:44.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.812+0000 7fce7d758640 1 -- 172.21.15.154:0/443432364 reap_dead start 2023-09-23T03:26:44.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.812+0000 7fce7df59640 1 -- 172.21.15.154:0/443432364 shutdown_connections 2023-09-23T03:26:44.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.812+0000 7fce7df59640 1 -- 172.21.15.154:0/443432364 >> 172.21.15.154:0/443432364 conn(0x7fce78069eb0 msgr2=0x7fce7806a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:44.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.812+0000 7fce7df59640 1 -- 172.21.15.154:0/443432364 shutdown_connections 2023-09-23T03:26:44.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.812+0000 7fce7df59640 1 -- 172.21.15.154:0/443432364 wait complete. 2023-09-23T03:26:44.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.814+0000 7fce7df59640 1 Processor -- start 2023-09-23T03:26:44.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.814+0000 7fce7df59640 1 -- start start 2023-09-23T03:26:44.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.815+0000 7fce7df59640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce68004d00 0x7fce781f4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:44.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.815+0000 7fce7df59640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce7815c6e0 0x7fce78160a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:44.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.815+0000 7fce7df59640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce780adc70 0x7fce780ae710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:44.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.815+0000 7fce7df59640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7815fa60 con 0x7fce780adc70 2023-09-23T03:26:44.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.815+0000 7fce7df59640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce781605e0 con 0x7fce7815c6e0 2023-09-23T03:26:44.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.815+0000 7fce7df59640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7815fcb0 con 0x7fce68004d00 2023-09-23T03:26:44.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.815+0000 7fce7d758640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce780adc70 0x7fce780ae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:44.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.815+0000 7fce7d758640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce780adc70 0x7fce780ae710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52722/0 (socket says 172.21.15.154:52722) 2023-09-23T03:26:44.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.815+0000 7fce7d758640 1 -- 172.21.15.154:0/2900872489 learned_addr learned my addr 172.21.15.154:0/2900872489 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:44.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.815+0000 7fce7cf57640 1 --2- 172.21.15.154:0/2900872489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce68004d00 0x7fce781f4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:44.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.815+0000 7fce77fff640 1 --2- 172.21.15.154:0/2900872489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce7815c6e0 0x7fce78160a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:44.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.816+0000 7fce7d758640 1 -- 172.21.15.154:0/2900872489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce68004d00 msgr2=0x7fce781f4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:44.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.816+0000 7fce7d758640 1 --2- 172.21.15.154:0/2900872489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce68004d00 0x7fce781f4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:44.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.816+0000 7fce7d758640 1 -- 172.21.15.154:0/2900872489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce7815c6e0 msgr2=0x7fce78160a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:44.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.816+0000 7fce7d758640 1 --2- 172.21.15.154:0/2900872489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce7815c6e0 0x7fce78160a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:44.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.816+0000 7fce7d758640 1 -- 172.21.15.154:0/2900872489 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce7815ff90 con 0x7fce780adc70 2023-09-23T03:26:44.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.816+0000 7fce7d758640 1 --2- 172.21.15.154:0/2900872489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce780adc70 0x7fce780ae710 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fce7000e170 tx=0x7fce70009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:44.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.816+0000 7fce757fa640 1 -- 172.21.15.154:0/2900872489 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce70034b50 con 0x7fce780adc70 2023-09-23T03:26:44.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.816+0000 7fce757fa640 1 -- 172.21.15.154:0/2900872489 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce70034cd0 con 0x7fce780adc70 2023-09-23T03:26:44.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.816+0000 7fce7df59640 1 -- 172.21.15.154:0/2900872489 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce78160270 con 0x7fce780adc70 2023-09-23T03:26:44.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.816+0000 7fce757fa640 1 -- 172.21.15.154:0/2900872489 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce70003bd0 con 0x7fce780adc70 2023-09-23T03:26:44.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.816+0000 7fce7df59640 1 -- 172.21.15.154:0/2900872489 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce780ae2d0 con 0x7fce780adc70 2023-09-23T03:26:44.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.819+0000 7fce757fa640 1 -- 172.21.15.154:0/2900872489 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fce7003e440 con 0x7fce780adc70 2023-09-23T03:26:44.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.820+0000 7fce757fa640 1 --2- 172.21.15.154:0/2900872489 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fce5c06af30 0x7fce5c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:44.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.820+0000 7fce757fa640 1 -- 172.21.15.154:0/2900872489 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fce700ab9f0 con 0x7fce780adc70 2023-09-23T03:26:44.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.820+0000 7fce7cf57640 1 --2- 172.21.15.154:0/2900872489 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fce5c06af30 0x7fce5c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:44.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.820+0000 7fce81e2a740 1 Processor -- start 2023-09-23T03:26:44.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.820+0000 7fce81e2a740 1 -- start start 2023-09-23T03:26:44.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.821+0000 7fce7cf57640 1 --2- 172.21.15.154:0/2900872489 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fce5c06af30 0x7fce5c06d3f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fce6c004770 tx=0x7fce6c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:44.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.821+0000 7fce81e2a740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563d0ceb8060 0x563d0ce8af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:44.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.821+0000 7fce81e2a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0ceb8440 0x563d0cebadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:44.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.821+0000 7fce81e2a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d0ce8b4a0 0x563d0cebb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:44.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.821+0000 7fce81e2a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d0cdfe4e0 con 0x563d0ce8b4a0 2023-09-23T03:26:44.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.821+0000 7fce81e2a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d0cdfbbd0 con 0x563d0ceb8440 2023-09-23T03:26:44.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.821+0000 7fce81e2a740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d0cdf74c0 con 0x563d0ceb8060 2023-09-23T03:26:44.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.822+0000 7fce7d758640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d0ce8b4a0 0x563d0cebb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:44.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.822+0000 7fce7d758640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d0ce8b4a0 0x563d0cebb330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52732/0 (socket says 172.21.15.154:52732) 2023-09-23T03:26:44.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.822+0000 7fce7d758640 1 -- 172.21.15.154:0/3733661365 learned_addr learned my addr 172.21.15.154:0/3733661365 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:44.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.822+0000 7fce7d758640 1 -- 172.21.15.154:0/3733661365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563d0ceb8060 msgr2=0x563d0ce8af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:44.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.822+0000 7fce7d758640 1 --2- 172.21.15.154:0/3733661365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563d0ceb8060 0x563d0ce8af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:44.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.822+0000 7fce7d758640 1 -- 172.21.15.154:0/3733661365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0ceb8440 msgr2=0x563d0cebadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:44.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.822+0000 7fce77fff640 1 --2- 172.21.15.154:0/3733661365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0ceb8440 0x563d0cebadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:44.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.822+0000 7fce7d758640 1 --2- 172.21.15.154:0/3733661365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0ceb8440 0x563d0cebadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:44.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.822+0000 7fce7d758640 1 -- 172.21.15.154:0/3733661365 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d0cd94cb0 con 0x563d0ce8b4a0 2023-09-23T03:26:44.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.822+0000 7fce7d758640 1 --2- 172.21.15.154:0/3733661365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d0ce8b4a0 0x563d0cebb330 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fce700a9bd0 tx=0x7fce7002df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a00eb825cc7f1cb5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:44.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.823+0000 7fce627fc640 1 -- 172.21.15.154:0/3733661365 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7007e4f0 con 0x563d0ce8b4a0 2023-09-23T03:26:44.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.823+0000 7fce627fc640 1 -- 172.21.15.154:0/3733661365 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce700826a0 con 0x563d0ce8b4a0 2023-09-23T03:26:44.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.823+0000 7fce627fc640 1 -- 172.21.15.154:0/3733661365 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce70009c10 con 0x563d0ce8b4a0 2023-09-23T03:26:44.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.823+0000 7fce81e2a740 1 -- 172.21.15.154:0/3733661365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d0ce8b4a0 msgr2=0x563d0cebb330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:44.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.823+0000 7fce81e2a740 1 --2- 172.21.15.154:0/3733661365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d0ce8b4a0 0x563d0cebb330 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fce700a9bd0 tx=0x7fce7002df90 comp rx=0 tx=0).stop 2023-09-23T03:26:44.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.823+0000 7fce81e2a740 1 -- 172.21.15.154:0/3733661365 shutdown_connections 2023-09-23T03:26:44.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.823+0000 7fce81e2a740 1 --2- 172.21.15.154:0/3733661365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d0ce8b4a0 0x563d0cebb330 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:44.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.823+0000 7fce81e2a740 1 --2- 172.21.15.154:0/3733661365 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0ceb8440 0x563d0cebadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:44.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.823+0000 7fce81e2a740 1 --2- 172.21.15.154:0/3733661365 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563d0ceb8060 0x563d0ce8af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:44.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.823+0000 7fce81e2a740 1 -- 172.21.15.154:0/3733661365 >> 172.21.15.154:0/3733661365 conn(0x563d0ce8d270 msgr2=0x563d0ced5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:44.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.823+0000 7fce81e2a740 1 -- 172.21.15.154:0/3733661365 shutdown_connections 2023-09-23T03:26:44.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.823+0000 7fce81e2a740 1 -- 172.21.15.154:0/3733661365 wait complete. 2023-09-23T03:26:44.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.824+0000 7fce81e2a740 1 -- start start 2023-09-23T03:26:44.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.824+0000 7fce81e2a740 1 Processor -- start 2023-09-23T03:26:44.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.825+0000 7fce81e2a740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563d0ceb8060 0x563d0cee6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:44.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.825+0000 7fce81e2a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0ceb8440 0x563d0cebdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:44.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.825+0000 7fce81e2a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d0cebeaa0 0x563d0ceb3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:44.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.825+0000 7fce81e2a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d0cd951e0 con 0x563d0cebeaa0 2023-09-23T03:26:44.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.825+0000 7fce81e2a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d0cd94e80 con 0x563d0ceb8440 2023-09-23T03:26:44.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.825+0000 7fce81e2a740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d0cb5c0c0 con 0x563d0ceb8060 2023-09-23T03:26:44.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.825+0000 7fce81e2a740 10 client.0 _ll_register_callbacks cb 0x563d0cdecd10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:26:44.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.826+0000 7fce7d758640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d0cebeaa0 0x563d0ceb3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:44.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.826+0000 7fce7d758640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d0cebeaa0 0x563d0ceb3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52736/0 (socket says 172.21.15.154:52736) 2023-09-23T03:26:44.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.826+0000 7fce7d758640 1 -- 172.21.15.154:0/1290012100 learned_addr learned my addr 172.21.15.154:0/1290012100 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:44.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.826+0000 7fce7d758640 1 -- 172.21.15.154:0/1290012100 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563d0ceb8060 msgr2=0x563d0cee6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:44.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.826+0000 7fce7d758640 1 --2- 172.21.15.154:0/1290012100 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563d0ceb8060 0x563d0cee6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:44.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.826+0000 7fce7d758640 1 -- 172.21.15.154:0/1290012100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0ceb8440 msgr2=0x563d0cebdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:44.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.826+0000 7fce7d758640 1 --2- 172.21.15.154:0/1290012100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0ceb8440 0x563d0cebdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:44.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.826+0000 7fce7d758640 1 -- 172.21.15.154:0/1290012100 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce780ae2d0 con 0x563d0cebeaa0 2023-09-23T03:26:44.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.826+0000 7fce7d758640 1 --2- 172.21.15.154:0/1290012100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d0cebeaa0 0x563d0ceb3650 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fce700a9c20 tx=0x7fce70000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25801f256730fd9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:44.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.826+0000 7fce577fe640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:26:44.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.826+0000 7fce577fe640 1 -- 172.21.15.154:0/1290012100 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7007e4f0 con 0x563d0cebeaa0 2023-09-23T03:26:44.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.827+0000 7fce577fe640 1 -- 172.21.15.154:0/1290012100 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fce7007e070 con 0x563d0cebeaa0 2023-09-23T03:26:44.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.827+0000 7fce81e2a740 1 -- 172.21.15.154:0/1290012100 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563d0cdf04a0 con 0x563d0cebeaa0 2023-09-23T03:26:44.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.827+0000 7fce577fe640 1 -- 172.21.15.154:0/1290012100 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce7007e230 con 0x563d0cebeaa0 2023-09-23T03:26:44.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.827+0000 7fce81e2a740 10 client.8655 fetch_fsmap learned FSMap version 99 2023-09-23T03:26:44.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.827+0000 7fce81e2a740 1 -- 172.21.15.154:0/1290012100 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x563d0cdf5b40 con 0x563d0cebeaa0 2023-09-23T03:26:44.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.827+0000 7fce577fe640 1 -- 172.21.15.154:0/1290012100 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563d0cdf04a0 con 0x563d0cebeaa0 2023-09-23T03:26:44.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.827+0000 7fce577fe640 1 -- 172.21.15.154:0/1290012100 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 99) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fce70091d20 con 0x563d0cebeaa0 2023-09-23T03:26:44.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.827+0000 7fce81e2a740 10 client.8655 fetch_fsmap finished waiting for FSMap version 99 2023-09-23T03:26:44.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.827+0000 7fce81e2a740 10 client.8655 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2023-09-23T03:26:44.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.827+0000 7fce81e2a740 20 client.8655 populate_metadata read hostname 'smithi154' 2023-09-23T03:26:44.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.827+0000 7fce81e2a740 1 --2- 172.21.15.154:0/1290012100 >> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] conn(0x563d0ceb41d0 0x563d0ceccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:44.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.827+0000 7fce81e2a740 4 client.8655 mds_command: new command op to 8595 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T03:26:44.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.827+0000 7fce81e2a740 1 -- 172.21.15.154:0/1290012100 --> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563d0cdf01c0 con 0x563d0ceb41d0 2023-09-23T03:26:44.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.828+0000 7fce7cf57640 1 --2- 172.21.15.154:0/1290012100 >> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] conn(0x563d0ceb41d0 0x563d0ceccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:44.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.828+0000 7fce7cf57640 1 --2- 172.21.15.154:0/1290012100 >> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] conn(0x563d0ceb41d0 0x563d0ceccbc0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c999b6efef0f10a8 server_cookie=6aa302453d49ac92 in_seq=0 out_seq=0 2023-09-23T03:26:44.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.828+0000 7fce577fe640 10 client.8655 ms_handle_connect on v2:172.21.15.154:6835/1758434763 2023-09-23T03:26:44.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.829+0000 7fce577fe640 1 -- 172.21.15.154:0/1290012100 <== mds.0 v2:172.21.15.154:6835/1758434763 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563d0cdf01c0 con 0x563d0ceb41d0 2023-09-23T03:26:44.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.829+0000 7fce577fe640 10 client.8655 handle_command_reply: tid=0 2023-09-23T03:26:44.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.829+0000 7fce81e2a740 1 client.8655 shutdown 2023-09-23T03:26:44.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.829+0000 7fce81e2a740 1 -- 172.21.15.154:0/1290012100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d0cebeaa0 msgr2=0x563d0ceb3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:44.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.829+0000 7fce81e2a740 1 --2- 172.21.15.154:0/1290012100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d0cebeaa0 0x563d0ceb3650 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fce700a9c20 tx=0x7fce70000f40 comp rx=0 tx=0).stop 2023-09-23T03:26:44.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.829+0000 7fce81e2a740 1 -- 172.21.15.154:0/1290012100 shutdown_connections 2023-09-23T03:26:44.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.829+0000 7fce81e2a740 1 --2- 172.21.15.154:0/1290012100 >> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] conn(0x563d0ceb41d0 0x563d0ceccbc0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:44.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.829+0000 7fce81e2a740 1 --2- 172.21.15.154:0/1290012100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d0cebeaa0 0x563d0ceb3650 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:44.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.829+0000 7fce81e2a740 1 --2- 172.21.15.154:0/1290012100 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0ceb8440 0x563d0cebdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:44.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.829+0000 7fce81e2a740 1 --2- 172.21.15.154:0/1290012100 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563d0ceb8060 0x563d0cee6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:44.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.829+0000 7fce577fe640 0 client.8655 ms_handle_reset on v2:172.21.15.154:6835/1758434763 2023-09-23T03:26:44.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.829+0000 7fce81e2a740 1 -- 172.21.15.154:0/1290012100 >> 172.21.15.154:0/1290012100 conn(0x563d0ce8b4a0 msgr2=0x563d0ced9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:44.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.829+0000 7fce81e2a740 1 -- 172.21.15.154:0/1290012100 shutdown_connections 2023-09-23T03:26:44.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.829+0000 7fce81e2a740 1 -- 172.21.15.154:0/1290012100 wait complete. 2023-09-23T03:26:44.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.830+0000 7fce81e2a740 20 client.8655 trim_cache size 0 max 16384 2023-09-23T03:26:44.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.845+0000 7fce81e2a740 1 Processor -- start 2023-09-23T03:26:44.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.845+0000 7fce81e2a740 1 -- start start 2023-09-23T03:26:44.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.846+0000 7fce81e2a740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563d0ceb8060 0x563d0ce89d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:44.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.846+0000 7fce81e2a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0ceb8440 0x563d0ceb2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:44.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.846+0000 7fce81e2a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d0cebeaa0 0x563d0cebafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:44.848 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.846+0000 7fce81e2a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d0cebcfb0 con 0x563d0cebeaa0 2023-09-23T03:26:44.848 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.846+0000 7fce81e2a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d0cee6560 con 0x563d0ceb8440 2023-09-23T03:26:44.848 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.846+0000 7fce81e2a740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d0ce8bf40 con 0x563d0ceb8060 2023-09-23T03:26:44.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.847+0000 7fce7d758640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d0cebeaa0 0x563d0cebafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:44.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.847+0000 7fce7d758640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d0cebeaa0 0x563d0cebafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52750/0 (socket says 172.21.15.154:52750) 2023-09-23T03:26:44.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.847+0000 7fce7d758640 1 -- 172.21.15.154:0/4031936877 learned_addr learned my addr 172.21.15.154:0/4031936877 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:44.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.847+0000 7fce77fff640 1 --2- 172.21.15.154:0/4031936877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0ceb8440 0x563d0ceb2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:44.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.847+0000 7fce7cf57640 1 --2- 172.21.15.154:0/4031936877 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563d0ceb8060 0x563d0ce89d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:44.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.847+0000 7fce7d758640 1 -- 172.21.15.154:0/4031936877 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563d0ceb8060 msgr2=0x563d0ce89d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:44.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.847+0000 7fce7d758640 1 --2- 172.21.15.154:0/4031936877 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563d0ceb8060 0x563d0ce89d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:44.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.847+0000 7fce7d758640 1 -- 172.21.15.154:0/4031936877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0ceb8440 msgr2=0x563d0ceb2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:44.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.847+0000 7fce7d758640 1 --2- 172.21.15.154:0/4031936877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0ceb8440 0x563d0ceb2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:44.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.847+0000 7fce7d758640 1 -- 172.21.15.154:0/4031936877 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d0cd94540 con 0x563d0cebeaa0 2023-09-23T03:26:44.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.847+0000 7fce7d758640 1 --2- 172.21.15.154:0/4031936877 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d0cebeaa0 0x563d0cebafd0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fce7002c5c0 tx=0x7fce7008f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1249d0708f00266e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:44.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.847+0000 7fce62ffd640 1 -- 172.21.15.154:0/4031936877 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7007e4f0 con 0x563d0cebeaa0 2023-09-23T03:26:44.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.848+0000 7fce62ffd640 1 -- 172.21.15.154:0/4031936877 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7007a170 con 0x563d0cebeaa0 2023-09-23T03:26:44.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.848+0000 7fce62ffd640 1 -- 172.21.15.154:0/4031936877 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7007a600 con 0x563d0cebeaa0 2023-09-23T03:26:44.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.848+0000 7fce81e2a740 1 -- 172.21.15.154:0/4031936877 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d0cebeaa0 msgr2=0x563d0cebafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:44.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.848+0000 7fce81e2a740 1 --2- 172.21.15.154:0/4031936877 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d0cebeaa0 0x563d0cebafd0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fce7002c5c0 tx=0x7fce7008f040 comp rx=0 tx=0).stop 2023-09-23T03:26:44.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.848+0000 7fce81e2a740 1 -- 172.21.15.154:0/4031936877 shutdown_connections 2023-09-23T03:26:44.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.848+0000 7fce81e2a740 1 --2- 172.21.15.154:0/4031936877 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d0cebeaa0 0x563d0cebafd0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:44.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.848+0000 7fce81e2a740 1 --2- 172.21.15.154:0/4031936877 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0ceb8440 0x563d0ceb2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:44.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.848+0000 7fce81e2a740 1 --2- 172.21.15.154:0/4031936877 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563d0ceb8060 0x563d0ce89d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:44.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.848+0000 7fce81e2a740 1 -- 172.21.15.154:0/4031936877 >> 172.21.15.154:0/4031936877 conn(0x563d0ce8b4a0 msgr2=0x563d0ceb4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:44.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.848+0000 7fce81e2a740 1 -- 172.21.15.154:0/4031936877 shutdown_connections 2023-09-23T03:26:44.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.848+0000 7fce81e2a740 1 -- 172.21.15.154:0/4031936877 wait complete. 2023-09-23T03:26:44.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.849+0000 7fce81e2a740 1 -- start start 2023-09-23T03:26:44.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.849+0000 7fce81e2a740 1 Processor -- start 2023-09-23T03:26:44.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.850+0000 7fce81e2a740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563d0ceb8060 0x563d0cef83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:44.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.850+0000 7fce81e2a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0ceb8440 0x563d0cef37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:44.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.850+0000 7fce81e2a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d0cebeaa0 0x563d0cee5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:44.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.850+0000 7fce81e2a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d0cebbda0 con 0x563d0cebeaa0 2023-09-23T03:26:44.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.850+0000 7fce81e2a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d0cef4250 con 0x563d0ceb8440 2023-09-23T03:26:44.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.850+0000 7fce81e2a740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d0cebbb50 con 0x563d0ceb8060 2023-09-23T03:26:44.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.850+0000 7fce81e2a740 10 client.0 _ll_register_callbacks cb 0x7fce7008df50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:26:44.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.850+0000 7fce7d758640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d0cebeaa0 0x563d0cee5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:44.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.851+0000 7fce7d758640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d0cebeaa0 0x563d0cee5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52766/0 (socket says 172.21.15.154:52766) 2023-09-23T03:26:44.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.851+0000 7fce7d758640 1 -- 172.21.15.154:0/3659231633 learned_addr learned my addr 172.21.15.154:0/3659231633 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:44.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.851+0000 7fce7cf57640 1 --2- 172.21.15.154:0/3659231633 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563d0ceb8060 0x563d0cef83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:44.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.851+0000 7fce7d758640 1 -- 172.21.15.154:0/3659231633 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563d0ceb8060 msgr2=0x563d0cef83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:44.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.851+0000 7fce77fff640 1 --2- 172.21.15.154:0/3659231633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0ceb8440 0x563d0cef37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:44.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.851+0000 7fce7d758640 1 --2- 172.21.15.154:0/3659231633 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563d0ceb8060 0x563d0cef83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:44.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.851+0000 7fce7d758640 1 -- 172.21.15.154:0/3659231633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0ceb8440 msgr2=0x563d0cef37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:44.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.851+0000 7fce7d758640 1 --2- 172.21.15.154:0/3659231633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0ceb8440 0x563d0cef37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:44.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.851+0000 7fce7d758640 1 -- 172.21.15.154:0/3659231633 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce780ae2d0 con 0x563d0cebeaa0 2023-09-23T03:26:44.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.851+0000 7fce7d758640 1 --2- 172.21.15.154:0/3659231633 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d0cebeaa0 0x563d0cee5970 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fce70078990 tx=0x7fce7000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d2329792a6d270 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:44.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.851+0000 7fce577fe640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:26:44.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.851+0000 7fce577fe640 1 -- 172.21.15.154:0/3659231633 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7007a8c0 con 0x563d0cebeaa0 2023-09-23T03:26:44.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.851+0000 7fce81e2a740 1 -- 172.21.15.154:0/3659231633 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce700826a0 con 0x563d0cebeaa0 2023-09-23T03:26:44.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.851+0000 7fce577fe640 1 -- 172.21.15.154:0/3659231633 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fce700a2e90 con 0x563d0cebeaa0 2023-09-23T03:26:44.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.852+0000 7fce577fe640 1 -- 172.21.15.154:0/3659231633 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce70078740 con 0x563d0cebeaa0 2023-09-23T03:26:44.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.852+0000 7fce81e2a740 10 client.8661 fetch_fsmap learned FSMap version 99 2023-09-23T03:26:44.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.852+0000 7fce81e2a740 1 -- 172.21.15.154:0/3659231633 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x563d0cdb9bf0 con 0x563d0cebeaa0 2023-09-23T03:26:44.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.852+0000 7fce577fe640 1 -- 172.21.15.154:0/3659231633 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce700826a0 con 0x563d0cebeaa0 2023-09-23T03:26:44.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.852+0000 7fce577fe640 1 -- 172.21.15.154:0/3659231633 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 99) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fce70091d20 con 0x563d0cebeaa0 2023-09-23T03:26:44.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.852+0000 7fce81e2a740 10 client.8661 fetch_fsmap finished waiting for FSMap version 99 2023-09-23T03:26:44.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.852+0000 7fce81e2a740 10 client.8661 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2023-09-23T03:26:44.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.852+0000 7fce81e2a740 20 client.8661 populate_metadata read hostname 'smithi154' 2023-09-23T03:26:44.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.852+0000 7fce81e2a740 1 --2- 172.21.15.154:0/3659231633 >> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] conn(0x563d0cf35390 0x563d0cf42960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:44.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.852+0000 7fce81e2a740 4 client.8661 mds_command: new command op to 8595 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2023-09-23T03:26:44.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.852+0000 7fce81e2a740 1 -- 172.21.15.154:0/3659231633 --> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x563d0caf9860 con 0x563d0cf35390 2023-09-23T03:26:44.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.852+0000 7fce7cf57640 1 --2- 172.21.15.154:0/3659231633 >> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] conn(0x563d0cf35390 0x563d0cf42960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:44.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.853+0000 7fce7cf57640 1 --2- 172.21.15.154:0/3659231633 >> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] conn(0x563d0cf35390 0x563d0cf42960 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=625027b36f7f082 server_cookie=93e7c948dd02ed5c in_seq=0 out_seq=0 2023-09-23T03:26:44.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.853+0000 7fce577fe640 10 client.8661 ms_handle_connect on v2:172.21.15.154:6835/1758434763 2023-09-23T03:26:44.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.854+0000 7fce577fe640 1 -- 172.21.15.154:0/3659231633 <== mds.0 v2:172.21.15.154:6835/1758434763 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563d0caf9860 con 0x563d0cf35390 2023-09-23T03:26:44.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.854+0000 7fce577fe640 10 client.8661 handle_command_reply: tid=0 2023-09-23T03:26:44.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.854+0000 7fce81e2a740 1 client.8661 shutdown 2023-09-23T03:26:44.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.854+0000 7fce81e2a740 1 -- 172.21.15.154:0/3659231633 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d0cebeaa0 msgr2=0x563d0cee5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:44.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.854+0000 7fce81e2a740 1 --2- 172.21.15.154:0/3659231633 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d0cebeaa0 0x563d0cee5970 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fce70078990 tx=0x7fce7000b040 comp rx=0 tx=0).stop 2023-09-23T03:26:44.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.854+0000 7fce81e2a740 1 -- 172.21.15.154:0/3659231633 shutdown_connections 2023-09-23T03:26:44.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.854+0000 7fce81e2a740 1 --2- 172.21.15.154:0/3659231633 >> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] conn(0x563d0cf35390 0x563d0cf42960 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:44.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.854+0000 7fce577fe640 0 client.8661 ms_handle_reset on v2:172.21.15.154:6835/1758434763 2023-09-23T03:26:44.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.854+0000 7fce81e2a740 1 --2- 172.21.15.154:0/3659231633 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563d0cebeaa0 0x563d0cee5970 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:44.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.854+0000 7fce81e2a740 1 --2- 172.21.15.154:0/3659231633 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563d0ceb8440 0x563d0cef37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:44.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.854+0000 7fce81e2a740 1 --2- 172.21.15.154:0/3659231633 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563d0ceb8060 0x563d0cef83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:44.859 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:26:44.854+0000 7fd09cd76640 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2023-09-23T03:26:44.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.854+0000 7fce81e2a740 1 -- 172.21.15.154:0/3659231633 >> 172.21.15.154:0/3659231633 conn(0x563d0ce8b4a0 msgr2=0x563d0cee6810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:44.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.855+0000 7fce81e2a740 1 -- 172.21.15.154:0/3659231633 shutdown_connections 2023-09-23T03:26:44.860 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:26:44.855+0000 7fd09cd76640 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x4 2023-09-23T03:26:44.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.855+0000 7fce81e2a740 1 -- 172.21.15.154:0/3659231633 wait complete. 2023-09-23T03:26:44.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.855+0000 7fce81e2a740 20 client.8661 trim_cache size 0 max 16384 2023-09-23T03:26:44.861 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:26:44.861 INFO:teuthology.orchestra.run.smithi154.stdout: "return_code": 0, 2023-09-23T03:26:44.861 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_tag": "f3c89d87-c258-444b-91fe-b9c64274a2a5", 2023-09-23T03:26:44.861 INFO:teuthology.orchestra.run.smithi154.stdout: "mode": "asynchronous" 2023-09-23T03:26:44.861 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:26:44.862 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:26:44.856+0000 7fd09cd76640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /stripey/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55dfe04e8100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000000:[<0x1/stripey v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-09-23T03:26:44.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.856+0000 7fce7df59640 1 -- 172.21.15.154:0/2900872489 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fce5c06af30 msgr2=0x7fce5c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:44.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.856+0000 7fce7df59640 1 --2- 172.21.15.154:0/2900872489 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fce5c06af30 0x7fce5c06d3f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fce6c004770 tx=0x7fce6c008040 comp rx=0 tx=0).stop 2023-09-23T03:26:44.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.856+0000 7fce7df59640 1 -- 172.21.15.154:0/2900872489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce780adc70 msgr2=0x7fce780ae710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:44.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.856+0000 7fce7df59640 1 --2- 172.21.15.154:0/2900872489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce780adc70 0x7fce780ae710 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fce7000e170 tx=0x7fce70009750 comp rx=0 tx=0).stop 2023-09-23T03:26:44.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.856+0000 7fce7df59640 1 -- 172.21.15.154:0/2900872489 shutdown_connections 2023-09-23T03:26:44.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.856+0000 7fce7df59640 1 --2- 172.21.15.154:0/2900872489 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fce5c06af30 0x7fce5c06d3f0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:44.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.856+0000 7fce7df59640 1 --2- 172.21.15.154:0/2900872489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce780adc70 0x7fce780ae710 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:44.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.856+0000 7fce7df59640 1 --2- 172.21.15.154:0/2900872489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce7815c6e0 0x7fce78160a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:44.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.856+0000 7fce7df59640 1 --2- 172.21.15.154:0/2900872489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce68004d00 0x7fce781f4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:44.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.856+0000 7fce7df59640 1 -- 172.21.15.154:0/2900872489 >> 172.21.15.154:0/2900872489 conn(0x7fce78069eb0 msgr2=0x7fce7815afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:44.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.856+0000 7fce7df59640 1 -- 172.21.15.154:0/2900872489 shutdown_connections 2023-09-23T03:26:44.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:44.857+0000 7fce7df59640 1 -- 172.21.15.154:0/2900872489 wait complete. 2023-09-23T03:26:44.864 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:26:44.857+0000 7fd09cd76640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x4 [...2,head] /lost+found/ auth v1 pv6 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55dfe04e8680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x4:[<0x1/lost+found v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-09-23T03:26:44.864 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:26:44.857+0000 7fd09cd76640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fb [2,head] /stripey/flushed_file_6 auth v1 ap=1 DIRTYPARENT s=1572864 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55dfe052ac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(16)0x100000001fb:[<0x10000000000/flushed_file_6 v68>,<0x1/stripey v75>]//[15]","memoryvalue":"(16)0x100000001fb:[<0x10000000000/flushed_file_6 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:26:44.864 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:26:44.857+0000 7fd09cd76640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f9 [2,head] /stripey/flushed_file_4 auth v1 ap=1 DIRTYPARENT s=1441792 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55dfe052b700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(16)0x100000001f9:[<0x10000000000/flushed_file_4 v70>,<0x1/stripey v75>]//[15]","memoryvalue":"(16)0x100000001f9:[<0x10000000000/flushed_file_4 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:26:44.864 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:26:44.857+0000 7fd09cd76640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f7 [2,head] /stripey/flushed_file_2 auth v1 ap=1 DIRTYPARENT s=917504 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55dfe0564100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(16)0x100000001f7:[<0x10000000000/flushed_file_2 v56>,<0x1/stripey v75>]//[15]","memoryvalue":"(16)0x100000001f7:[<0x10000000000/flushed_file_2 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:26:44.865 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:26:44.858+0000 7fd09cd76640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f6 [2,head] /stripey/flushed_file_1 auth v1 ap=1 DIRTYPARENT s=524288 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55dfe0564680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(16)0x100000001f6:[<0x10000000000/flushed_file_1 v50>,<0x1/stripey v75>]//[15]","memoryvalue":"(16)0x100000001f6:[<0x10000000000/flushed_file_1 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:26:44.865 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:26:44.858+0000 7fd09cd76640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /stripey/flushed_file_0 auth v1 ap=1 DIRTYPARENT s=1048576 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55dfe0564c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(16)0x10000000001:[<0x10000000000/flushed_file_0 v38>,<0x1/stripey v75>]//[15]","memoryvalue":"(16)0x10000000001:[<0x10000000000/flushed_file_0 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:26:44.865 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:26:44.858+0000 7fd09cd76640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f8 [2,head] /stripey/flushed_file_3 auth v1 ap=1 DIRTYPARENT s=917503 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55dfe0563b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(16)0x100000001f8:[<0x10000000000/flushed_file_3 v64>,<0x1/stripey v75>]//[15]","memoryvalue":"(16)0x100000001f8:[<0x10000000000/flushed_file_3 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:26:44.865 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:26:44.860+0000 7fd09cd76640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fa [2,head] /stripey/flushed_file_5 auth v1 ap=1 DIRTYPARENT s=1441793 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55dfe052b180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(16)0x100000001fa:[<0x10000000000/flushed_file_5 v54>,<0x1/stripey v75>]//[15]","memoryvalue":"(16)0x100000001fa:[<0x10000000000/flushed_file_5 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:26:44.865 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:26:44.860+0000 7fd09cd76640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fc [2,head] /lost+found/100000001fc auth v1 ap=1 DIRTYPARENT s=524288 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55dfe0565180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(16)0x100000001fc:[<0x4/100000001fc v1>,<0x1/lost+found v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2023-09-23T03:26:44.869 INFO:tasks.ceph.mds.a.smithi154.stderr:2023-09-23T03:26:44.867+0000 7fd09cd76640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v7 ap=5 snaprealm=0x55dfe047a900 DIRTYPARENT f(v1 2=0+2) n(v0 rc2023-09-23T03:26:44.576922+0000 2=0+2)/n() (idft sync r=1) (inest sync r=1) (ifile sync->mix r=1) (iversion lock) caps={8643=pLsXs/-@3} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x55dfe04e7600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 2=0+2)","ondisk_value.rstat":"n(v0 3=0+3)","memory_value.dirstat":"f(v1 2=0+2)","memory_value.rstat":"n(v0 rc2023-09-23T03:26:44.576922+0000 2=0+2)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-09-23T03:26:44.877 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 scrub status 2023-09-23T03:26:45.084 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.082+0000 7fbe081e2640 1 Processor -- start 2023-09-23T03:26:45.084 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.083+0000 7fbe081e2640 1 -- start start 2023-09-23T03:26:45.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.084+0000 7fbe081e2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe000ad830 0x7fbe00070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:45.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.084+0000 7fbe081e2640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbe000c9680 0x7fbe000caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:45.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.084+0000 7fbe081e2640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe000712f0 con 0x7fbe000cb400 2023-09-23T03:26:45.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.084+0000 7fbe081e2640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe000cb890 con 0x7fbe000c9680 2023-09-23T03:26:45.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.084+0000 7fbe081e2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe000715d0 con 0x7fbe000ad830 2023-09-23T03:26:45.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.085+0000 7fbe0677f640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fbe000cb400 0x7fbe00160720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:55440/0 (socket says 172.21.15.154:55440) 2023-09-23T03:26:45.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.085+0000 7fbe0677f640 1 -- 172.21.15.154:0/1184974123 learned_addr learned my addr 172.21.15.154:0/1184974123 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:26:45.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.085+0000 7fbe0577d640 1 --2- 172.21.15.154:0/1184974123 >> v2:172.21.15.190:3300/0 conn(0x7fbe000c9680 0x7fbe000caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:45.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.085+0000 7fbe05f7e640 1 --2- 172.21.15.154:0/1184974123 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe000ad830 0x7fbe00070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:45.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.085+0000 7fbe04f7c640 1 -- 172.21.15.154:0/1184974123 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4002060901 0 0) 0x7fbe000712f0 con 0x7fbe000cb400 2023-09-23T03:26:45.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.085+0000 7fbe04f7c640 1 -- 172.21.15.154:0/1184974123 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbde8003520 con 0x7fbe000cb400 2023-09-23T03:26:45.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.086+0000 7fbe04f7c640 1 -- 172.21.15.154:0/1184974123 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2484870699 0 0) 0x7fbde8003520 con 0x7fbe000cb400 2023-09-23T03:26:45.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.086+0000 7fbe04f7c640 1 -- 172.21.15.154:0/1184974123 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe000ad830 msgr2=0x7fbe00070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:45.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.086+0000 7fbe04f7c640 1 --2- 172.21.15.154:0/1184974123 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe000ad830 0x7fbe00070c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:45.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.086+0000 7fbe04f7c640 1 -- 172.21.15.154:0/1184974123 >> v2:172.21.15.190:3300/0 conn(0x7fbe000c9680 msgr2=0x7fbe000caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:45.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.086+0000 7fbe04f7c640 1 --2- 172.21.15.154:0/1184974123 >> v2:172.21.15.190:3300/0 conn(0x7fbe000c9680 0x7fbe000caec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:45.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.086+0000 7fbe04f7c640 1 -- 172.21.15.154:0/1184974123 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe000718b0 con 0x7fbe000cb400 2023-09-23T03:26:45.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.086+0000 7fbe04f7c640 1 -- 172.21.15.154:0/1184974123 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fbdf8001670 con 0x7fbe000cb400 2023-09-23T03:26:45.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.086+0000 7fbe04f7c640 1 -- 172.21.15.154:0/1184974123 >> v1:172.21.15.154:6789/0 conn(0x7fbe000cb400 legacy=0x7fbe00160720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:45.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.086+0000 7fbe04f7c640 1 --2- 172.21.15.154:0/1184974123 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe000ad830 0x7fbde8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:45.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.086+0000 7fbe04f7c640 1 --2- 172.21.15.154:0/1184974123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbde8003ef0 0x7fbde80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:45.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.086+0000 7fbe04f7c640 1 --2- 172.21.15.154:0/1184974123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbde8004d00 0x7fbde8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:45.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.086+0000 7fbe04f7c640 1 -- 172.21.15.154:0/1184974123 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe000cb890 con 0x7fbde8003ef0 2023-09-23T03:26:45.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.086+0000 7fbe04f7c640 1 -- 172.21.15.154:0/1184974123 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe000715d0 con 0x7fbde8004d00 2023-09-23T03:26:45.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.086+0000 7fbe04f7c640 1 -- 172.21.15.154:0/1184974123 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde800b980 con 0x7fbe000ad830 2023-09-23T03:26:45.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.086+0000 7fbe0577d640 1 --2- 172.21.15.154:0/1184974123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbde8003ef0 0x7fbde80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:45.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.086+0000 7fbe05f7e640 1 --2- 172.21.15.154:0/1184974123 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe000ad830 0x7fbde8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:45.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.086+0000 7fbe0577d640 1 -- 172.21.15.154:0/1184974123 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe000ad830 msgr2=0x7fbde8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:45.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.086+0000 7fbe0677f640 1 --2- 172.21.15.154:0/1184974123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbde8004d00 0x7fbde8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:45.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.087+0000 7fbe0577d640 1 --2- 172.21.15.154:0/1184974123 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe000ad830 0x7fbde8003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:45.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.087+0000 7fbe0577d640 1 -- 172.21.15.154:0/1184974123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbde8004d00 msgr2=0x7fbde8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:45.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.087+0000 7fbe0577d640 1 --2- 172.21.15.154:0/1184974123 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbde8004d00 0x7fbde8005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:45.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.087+0000 7fbe0577d640 1 -- 172.21.15.154:0/1184974123 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde8003520 con 0x7fbde8003ef0 2023-09-23T03:26:45.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.088+0000 7fbe0577d640 1 --2- 172.21.15.154:0/1184974123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbde8003ef0 0x7fbde80047c0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4009ac0 tx=0x7fbde80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b10a39c368972b0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:45.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.088+0000 7fbe04f7c640 1 -- 172.21.15.154:0/1184974123 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf402c6d0 con 0x7fbde8003ef0 2023-09-23T03:26:45.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.088+0000 7fbe04f7c640 1 -- 172.21.15.154:0/1184974123 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf402c830 con 0x7fbde8003ef0 2023-09-23T03:26:45.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.088+0000 7fbe04f7c640 1 -- 172.21.15.154:0/1184974123 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf402cbc0 con 0x7fbde8003ef0 2023-09-23T03:26:45.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.088+0000 7fbe081e2640 1 -- 172.21.15.154:0/1184974123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbde8003ef0 msgr2=0x7fbde80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:45.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.088+0000 7fbe081e2640 1 --2- 172.21.15.154:0/1184974123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbde8003ef0 0x7fbde80047c0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4009ac0 tx=0x7fbde80057f0 comp rx=0 tx=0).stop 2023-09-23T03:26:45.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.089+0000 7fbe0677f640 1 -- 172.21.15.154:0/1184974123 reap_dead start 2023-09-23T03:26:45.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.089+0000 7fbe081e2640 1 -- 172.21.15.154:0/1184974123 shutdown_connections 2023-09-23T03:26:45.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.089+0000 7fbe081e2640 1 -- 172.21.15.154:0/1184974123 >> 172.21.15.154:0/1184974123 conn(0x7fbe0006c010 msgr2=0x7fbe0006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:45.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.089+0000 7fbe081e2640 1 -- 172.21.15.154:0/1184974123 shutdown_connections 2023-09-23T03:26:45.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.089+0000 7fbe081e2640 1 -- 172.21.15.154:0/1184974123 wait complete. 2023-09-23T03:26:45.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.091+0000 7fbe081e2640 1 Processor -- start 2023-09-23T03:26:45.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.091+0000 7fbe081e2640 1 -- start start 2023-09-23T03:26:45.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.092+0000 7fbe081e2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbde8003ef0 0x7fbe001547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:45.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.092+0000 7fbe081e2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe000c68a0 0x7fbe000c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:45.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.092+0000 7fbe081e2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe000b1fd0 0x7fbe000cd910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:45.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.092+0000 7fbe081e2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe000c6d30 con 0x7fbe000b1fd0 2023-09-23T03:26:45.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.092+0000 7fbe081e2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe000c7010 con 0x7fbde8003ef0 2023-09-23T03:26:45.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.092+0000 7fbe081e2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe000c72f0 con 0x7fbe000c68a0 2023-09-23T03:26:45.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.092+0000 7fbe0677f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe000b1fd0 0x7fbe000cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:45.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.092+0000 7fbe0677f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe000b1fd0 0x7fbe000cd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52786/0 (socket says 172.21.15.154:52786) 2023-09-23T03:26:45.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.092+0000 7fbe0677f640 1 -- 172.21.15.154:0/3128618869 learned_addr learned my addr 172.21.15.154:0/3128618869 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:45.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.092+0000 7fbe0577d640 1 --2- 172.21.15.154:0/3128618869 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe000c68a0 0x7fbe000c7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:45.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.092+0000 7fbe05f7e640 1 --2- 172.21.15.154:0/3128618869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbde8003ef0 0x7fbe001547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:45.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.092+0000 7fbe0677f640 1 -- 172.21.15.154:0/3128618869 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe000c68a0 msgr2=0x7fbe000c7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:45.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.092+0000 7fbe0677f640 1 --2- 172.21.15.154:0/3128618869 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe000c68a0 0x7fbe000c7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:45.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.092+0000 7fbe0677f640 1 -- 172.21.15.154:0/3128618869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbde8003ef0 msgr2=0x7fbe001547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:45.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.092+0000 7fbe0677f640 1 --2- 172.21.15.154:0/3128618869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbde8003ef0 0x7fbe001547f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:45.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.093+0000 7fbe0677f640 1 -- 172.21.15.154:0/3128618869 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe000c75d0 con 0x7fbe000b1fd0 2023-09-23T03:26:45.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.093+0000 7fbe0677f640 1 --2- 172.21.15.154:0/3128618869 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe000b1fd0 0x7fbe000cd910 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fbe000cb7e0 tx=0x7fbdf800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:45.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.093+0000 7fbdfe7fc640 1 -- 172.21.15.154:0/3128618869 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8007230 con 0x7fbe000b1fd0 2023-09-23T03:26:45.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.093+0000 7fbdfe7fc640 1 -- 172.21.15.154:0/3128618869 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf8007390 con 0x7fbe000b1fd0 2023-09-23T03:26:45.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.093+0000 7fbdfe7fc640 1 -- 172.21.15.154:0/3128618869 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf80047e0 con 0x7fbe000b1fd0 2023-09-23T03:26:45.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.093+0000 7fbe081e2640 1 -- 172.21.15.154:0/3128618869 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe000c78b0 con 0x7fbe000b1fd0 2023-09-23T03:26:45.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.093+0000 7fbe081e2640 1 -- 172.21.15.154:0/3128618869 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe000c8310 con 0x7fbe000b1fd0 2023-09-23T03:26:45.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.095+0000 7fbdfe7fc640 1 -- 172.21.15.154:0/3128618869 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fbdf8004940 con 0x7fbe000b1fd0 2023-09-23T03:26:45.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.097+0000 7fbdfe7fc640 1 --2- 172.21.15.154:0/3128618869 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbdec06afb0 0x7fbdec06d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:45.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.097+0000 7fbdfe7fc640 1 -- 172.21.15.154:0/3128618869 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fbdf80899b0 con 0x7fbe000b1fd0 2023-09-23T03:26:45.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.097+0000 7fbe05f7e640 1 --2- 172.21.15.154:0/3128618869 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbdec06afb0 0x7fbdec06d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:45.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.098+0000 7fbe0ad71740 1 Processor -- start 2023-09-23T03:26:45.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.098+0000 7fbe05f7e640 1 --2- 172.21.15.154:0/3128618869 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbdec06afb0 0x7fbdec06d470 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0003b80 tx=0x7fbdf000b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:45.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.098+0000 7fbe0ad71740 1 -- start start 2023-09-23T03:26:45.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.099+0000 7fbe0ad71740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558923300690 0x5589232d3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:45.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.099+0000 7fbe0ad71740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558923300a70 0x558923303420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:45.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.099+0000 7fbe0ad71740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5589232d3ad0 0x558923303960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:45.101 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.099+0000 7fbe0ad71740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x558923245a50 con 0x558923300a70 2023-09-23T03:26:45.101 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.099+0000 7fbe0ad71740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558923243140 con 0x558923300690 2023-09-23T03:26:45.101 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.099+0000 7fbe0ad71740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55892323ea30 con 0x5589232d3ad0 2023-09-23T03:26:45.101 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.099+0000 7fbe0577d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558923300a70 0x558923303420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:45.101 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.099+0000 7fbe0577d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558923300a70 0x558923303420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52792/0 (socket says 172.21.15.154:52792) 2023-09-23T03:26:45.101 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.099+0000 7fbe0577d640 1 -- 172.21.15.154:0/3613341344 learned_addr learned my addr 172.21.15.154:0/3613341344 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:45.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.100+0000 7fbe0577d640 1 -- 172.21.15.154:0/3613341344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5589232d3ad0 msgr2=0x558923303960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:45.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.100+0000 7fbe0577d640 1 --2- 172.21.15.154:0/3613341344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5589232d3ad0 0x558923303960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:45.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.100+0000 7fbe0577d640 1 -- 172.21.15.154:0/3613341344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558923300690 msgr2=0x5589232d3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:45.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.100+0000 7fbe0577d640 1 --2- 172.21.15.154:0/3613341344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558923300690 0x5589232d3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:45.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.100+0000 7fbe0577d640 1 -- 172.21.15.154:0/3613341344 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558923202a70 con 0x558923300a70 2023-09-23T03:26:45.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.100+0000 7fbe0577d640 1 --2- 172.21.15.154:0/3613341344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558923300a70 0x558923303420 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4000c00 tx=0x7fbdf402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff15679bb03cfaf server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:45.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.100+0000 7fbde77fe640 1 -- 172.21.15.154:0/3613341344 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf4004240 con 0x558923300a70 2023-09-23T03:26:45.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.100+0000 7fbde77fe640 1 -- 172.21.15.154:0/3613341344 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf40043a0 con 0x558923300a70 2023-09-23T03:26:45.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.100+0000 7fbde77fe640 1 -- 172.21.15.154:0/3613341344 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf402cbc0 con 0x558923300a70 2023-09-23T03:26:45.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.101+0000 7fbe0ad71740 1 -- 172.21.15.154:0/3613341344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558923300a70 msgr2=0x558923303420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:45.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.101+0000 7fbe0ad71740 1 --2- 172.21.15.154:0/3613341344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558923300a70 0x558923303420 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4000c00 tx=0x7fbdf402e000 comp rx=0 tx=0).stop 2023-09-23T03:26:45.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.101+0000 7fbe0ad71740 1 -- 172.21.15.154:0/3613341344 shutdown_connections 2023-09-23T03:26:45.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.101+0000 7fbe0ad71740 1 --2- 172.21.15.154:0/3613341344 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5589232d3ad0 0x558923303960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:45.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.101+0000 7fbe0ad71740 1 --2- 172.21.15.154:0/3613341344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558923300a70 0x558923303420 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:45.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.101+0000 7fbe0ad71740 1 --2- 172.21.15.154:0/3613341344 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558923300690 0x5589232d3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:45.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.101+0000 7fbe0ad71740 1 -- 172.21.15.154:0/3613341344 >> 172.21.15.154:0/3613341344 conn(0x5589232d58a0 msgr2=0x55892331e420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:45.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.101+0000 7fbe0ad71740 1 -- 172.21.15.154:0/3613341344 shutdown_connections 2023-09-23T03:26:45.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.101+0000 7fbe0ad71740 1 -- 172.21.15.154:0/3613341344 wait complete. 2023-09-23T03:26:45.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.102+0000 7fbe0ad71740 1 -- start start 2023-09-23T03:26:45.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.102+0000 7fbe0ad71740 1 Processor -- start 2023-09-23T03:26:45.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.103+0000 7fbe0ad71740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558923300690 0x5589232f19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:45.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.103+0000 7fbe0ad71740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558923300a70 0x558923318a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:45.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.103+0000 7fbe0ad71740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558923317a20 0x558923316370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:45.108 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.103+0000 7fbe0ad71740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589231dcad0 con 0x558923300a70 2023-09-23T03:26:45.108 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.103+0000 7fbe0ad71740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589231dc770 con 0x558923317a20 2023-09-23T03:26:45.108 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.103+0000 7fbe0ad71740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x558922f38f90 con 0x558923300690 2023-09-23T03:26:45.108 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.103+0000 7fbe0ad71740 10 client.0 _ll_register_callbacks cb 0x558923200220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:26:45.108 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.104+0000 7fbe0577d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558923300a70 0x558923318a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:45.108 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.104+0000 7fbe0577d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558923300a70 0x558923318a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52802/0 (socket says 172.21.15.154:52802) 2023-09-23T03:26:45.108 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.104+0000 7fbe0577d640 1 -- 172.21.15.154:0/835181767 learned_addr learned my addr 172.21.15.154:0/835181767 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:45.108 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.104+0000 7fbe05f7e640 1 --2- 172.21.15.154:0/835181767 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558923300690 0x5589232f19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:45.108 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.104+0000 7fbe0677f640 1 --2- 172.21.15.154:0/835181767 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558923317a20 0x558923316370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:45.108 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.104+0000 7fbe0577d640 1 -- 172.21.15.154:0/835181767 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558923300690 msgr2=0x5589232f19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:45.108 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.104+0000 7fbe0577d640 1 --2- 172.21.15.154:0/835181767 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558923300690 0x5589232f19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:45.109 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.104+0000 7fbe0577d640 1 -- 172.21.15.154:0/835181767 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558923317a20 msgr2=0x558923316370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:45.109 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.104+0000 7fbe0577d640 1 --2- 172.21.15.154:0/835181767 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558923317a20 0x558923316370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:45.109 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.104+0000 7fbe0577d640 1 -- 172.21.15.154:0/835181767 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde8003520 con 0x558923300a70 2023-09-23T03:26:45.109 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.104+0000 7fbe0577d640 1 --2- 172.21.15.154:0/835181767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558923300a70 0x558923318a50 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4009be0 tx=0x7fbdf40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c1ea64650141a6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:45.109 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.104+0000 7fbde4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:26:45.109 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.105+0000 7fbde4ff9640 1 -- 172.21.15.154:0/835181767 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf4034850 con 0x558923300a70 2023-09-23T03:26:45.109 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.105+0000 7fbde4ff9640 1 -- 172.21.15.154:0/835181767 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fbdf4040ac0 con 0x558923300a70 2023-09-23T03:26:45.109 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.105+0000 7fbe0ad71740 1 -- 172.21.15.154:0/835181767 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5589232654d0 con 0x558923300a70 2023-09-23T03:26:45.109 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.105+0000 7fbde4ff9640 1 -- 172.21.15.154:0/835181767 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbdf4040c80 con 0x558923300a70 2023-09-23T03:26:45.109 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.105+0000 7fbe0ad71740 10 client.8673 fetch_fsmap learned FSMap version 99 2023-09-23T03:26:45.109 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.105+0000 7fbe0ad71740 1 -- 172.21.15.154:0/835181767 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x55892323d0b0 con 0x558923300a70 2023-09-23T03:26:45.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.105+0000 7fbde4ff9640 1 -- 172.21.15.154:0/835181767 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5589232654d0 con 0x558923300a70 2023-09-23T03:26:45.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.105+0000 7fbde4ff9640 1 -- 172.21.15.154:0/835181767 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 99) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fbdf40349b0 con 0x558923300a70 2023-09-23T03:26:45.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.105+0000 7fbe0ad71740 10 client.8673 fetch_fsmap finished waiting for FSMap version 99 2023-09-23T03:26:45.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.105+0000 7fbe0ad71740 10 client.8673 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2023-09-23T03:26:45.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.105+0000 7fbe0ad71740 20 client.8673 populate_metadata read hostname 'smithi154' 2023-09-23T03:26:45.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.105+0000 7fbe0ad71740 1 --2- 172.21.15.154:0/835181767 >> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] conn(0x5589232f25b0 0x558923305460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:45.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.105+0000 7fbe0ad71740 4 client.8673 mds_command: new command op to 8595 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T03:26:45.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.105+0000 7fbe0ad71740 1 -- 172.21.15.154:0/835181767 --> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558923211ab0 con 0x5589232f25b0 2023-09-23T03:26:45.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.106+0000 7fbe05f7e640 1 --2- 172.21.15.154:0/835181767 >> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] conn(0x5589232f25b0 0x558923305460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:45.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.106+0000 7fbe05f7e640 1 --2- 172.21.15.154:0/835181767 >> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] conn(0x5589232f25b0 0x558923305460 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46c0cddf9d163047 server_cookie=959336bd9192ad0 in_seq=0 out_seq=0 2023-09-23T03:26:45.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.106+0000 7fbde4ff9640 10 client.8673 ms_handle_connect on v2:172.21.15.154:6835/1758434763 2023-09-23T03:26:45.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.107+0000 7fbde4ff9640 1 -- 172.21.15.154:0/835181767 <== mds.0 v2:172.21.15.154:6835/1758434763 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x558923211ab0 con 0x5589232f25b0 2023-09-23T03:26:45.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.107+0000 7fbde4ff9640 10 client.8673 handle_command_reply: tid=0 2023-09-23T03:26:45.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.107+0000 7fbe0ad71740 1 client.8673 shutdown 2023-09-23T03:26:45.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.107+0000 7fbe0ad71740 1 -- 172.21.15.154:0/835181767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558923300a70 msgr2=0x558923318a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:45.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.107+0000 7fbe0ad71740 1 --2- 172.21.15.154:0/835181767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558923300a70 0x558923318a50 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4009be0 tx=0x7fbdf40046c0 comp rx=0 tx=0).stop 2023-09-23T03:26:45.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.107+0000 7fbe0ad71740 1 -- 172.21.15.154:0/835181767 shutdown_connections 2023-09-23T03:26:45.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.107+0000 7fbe0ad71740 1 --2- 172.21.15.154:0/835181767 >> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] conn(0x5589232f25b0 0x558923305460 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:45.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.107+0000 7fbe0ad71740 1 --2- 172.21.15.154:0/835181767 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558923317a20 0x558923316370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:45.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.107+0000 7fbe0ad71740 1 --2- 172.21.15.154:0/835181767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558923300a70 0x558923318a50 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:45.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.107+0000 7fbde4ff9640 0 client.8673 ms_handle_reset on v2:172.21.15.154:6835/1758434763 2023-09-23T03:26:45.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.107+0000 7fbe0ad71740 1 --2- 172.21.15.154:0/835181767 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558923300690 0x5589232f19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:45.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.107+0000 7fbe0ad71740 1 -- 172.21.15.154:0/835181767 >> 172.21.15.154:0/835181767 conn(0x5589232d3ad0 msgr2=0x5589232d7510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:45.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.107+0000 7fbe0ad71740 1 -- 172.21.15.154:0/835181767 shutdown_connections 2023-09-23T03:26:45.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.107+0000 7fbe0ad71740 1 -- 172.21.15.154:0/835181767 wait complete. 2023-09-23T03:26:45.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.108+0000 7fbe0ad71740 20 client.8673 trim_cache size 0 max 16384 2023-09-23T03:26:45.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.123+0000 7fbe0ad71740 1 Processor -- start 2023-09-23T03:26:45.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.123+0000 7fbe0ad71740 1 -- start start 2023-09-23T03:26:45.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.124+0000 7fbe0ad71740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558923300690 0x558923303020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:45.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.124+0000 7fbe0ad71740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558923300a70 0x55892331e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:45.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.124+0000 7fbe0ad71740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558923317a20 0x558923314f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:45.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.124+0000 7fbe0ad71740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x558923303560 con 0x558923300690 2023-09-23T03:26:45.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.124+0000 7fbe0ad71740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558923233e30 con 0x558923300a70 2023-09-23T03:26:45.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.124+0000 7fbe0ad71740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55892331ed40 con 0x558923317a20 2023-09-23T03:26:45.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.125+0000 7fbe05f7e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558923300690 0x558923303020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:45.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.125+0000 7fbe05f7e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558923300690 0x558923303020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52818/0 (socket says 172.21.15.154:52818) 2023-09-23T03:26:45.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.125+0000 7fbe05f7e640 1 -- 172.21.15.154:0/1838707536 learned_addr learned my addr 172.21.15.154:0/1838707536 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:45.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.125+0000 7fbe05f7e640 1 -- 172.21.15.154:0/1838707536 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558923317a20 msgr2=0x558923314f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:45.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.125+0000 7fbe05f7e640 1 --2- 172.21.15.154:0/1838707536 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558923317a20 0x558923314f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:45.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.125+0000 7fbe05f7e640 1 -- 172.21.15.154:0/1838707536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558923300a70 msgr2=0x55892331e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:45.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.125+0000 7fbe05f7e640 1 --2- 172.21.15.154:0/1838707536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558923300a70 0x55892331e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:45.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.125+0000 7fbe05f7e640 1 -- 172.21.15.154:0/1838707536 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55892326eda0 con 0x558923300690 2023-09-23T03:26:45.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.125+0000 7fbe05f7e640 1 --2- 172.21.15.154:0/1838707536 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558923300690 0x558923303020 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0027c30 tx=0x7fbdf0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d5ba3d4416dd3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:45.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.125+0000 7fbde7fff640 1 -- 172.21.15.154:0/1838707536 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf0027e60 con 0x558923300690 2023-09-23T03:26:45.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.125+0000 7fbde7fff640 1 -- 172.21.15.154:0/1838707536 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf0020040 con 0x558923300690 2023-09-23T03:26:45.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.125+0000 7fbde7fff640 1 -- 172.21.15.154:0/1838707536 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf0027280 con 0x558923300690 2023-09-23T03:26:45.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.126+0000 7fbe0ad71740 1 -- 172.21.15.154:0/1838707536 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558923300690 msgr2=0x558923303020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:45.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.126+0000 7fbe0ad71740 1 --2- 172.21.15.154:0/1838707536 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558923300690 0x558923303020 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0027c30 tx=0x7fbdf0013000 comp rx=0 tx=0).stop 2023-09-23T03:26:45.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.126+0000 7fbe0ad71740 1 -- 172.21.15.154:0/1838707536 shutdown_connections 2023-09-23T03:26:45.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.126+0000 7fbe0ad71740 1 --2- 172.21.15.154:0/1838707536 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558923317a20 0x558923314f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:45.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.126+0000 7fbe0ad71740 1 --2- 172.21.15.154:0/1838707536 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558923300a70 0x55892331e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:45.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.126+0000 7fbe0ad71740 1 --2- 172.21.15.154:0/1838707536 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558923300690 0x558923303020 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:45.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.126+0000 7fbe0ad71740 1 -- 172.21.15.154:0/1838707536 >> 172.21.15.154:0/1838707536 conn(0x5589232d3ad0 msgr2=0x5589232ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:45.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.126+0000 7fbe0ad71740 1 -- 172.21.15.154:0/1838707536 shutdown_connections 2023-09-23T03:26:45.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.126+0000 7fbe0ad71740 1 -- 172.21.15.154:0/1838707536 wait complete. 2023-09-23T03:26:45.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.127+0000 7fbe0ad71740 1 -- start start 2023-09-23T03:26:45.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.127+0000 7fbe0ad71740 1 Processor -- start 2023-09-23T03:26:45.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.128+0000 7fbe0ad71740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558923300690 0x558923378330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:45.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.128+0000 7fbe0ad71740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558923300a70 0x558923317e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:45.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.128+0000 7fbe0ad71740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558923317a20 0x558923318340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:45.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.128+0000 7fbe0ad71740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55892338ab60 con 0x558923300690 2023-09-23T03:26:45.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.128+0000 7fbe0ad71740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558923388c60 con 0x558923300a70 2023-09-23T03:26:45.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.128+0000 7fbe0ad71740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5589232d2d60 con 0x558923317a20 2023-09-23T03:26:45.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.128+0000 7fbe0ad71740 10 client.0 _ll_register_callbacks cb 0x7fbdf4037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:26:45.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.128+0000 7fbe05f7e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558923300690 0x558923378330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:45.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.128+0000 7fbe05f7e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558923300690 0x558923378330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52832/0 (socket says 172.21.15.154:52832) 2023-09-23T03:26:45.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.128+0000 7fbe05f7e640 1 -- 172.21.15.154:0/4061658820 learned_addr learned my addr 172.21.15.154:0/4061658820 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:26:45.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.129+0000 7fbe05f7e640 1 -- 172.21.15.154:0/4061658820 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558923317a20 msgr2=0x558923318340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:45.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.129+0000 7fbe05f7e640 1 --2- 172.21.15.154:0/4061658820 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558923317a20 0x558923318340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:45.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.129+0000 7fbe05f7e640 1 -- 172.21.15.154:0/4061658820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558923300a70 msgr2=0x558923317e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:26:45.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.129+0000 7fbe05f7e640 1 --2- 172.21.15.154:0/4061658820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558923300a70 0x558923317e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:45.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.129+0000 7fbe05f7e640 1 -- 172.21.15.154:0/4061658820 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdf00279f0 con 0x558923300690 2023-09-23T03:26:45.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.129+0000 7fbe05f7e640 1 --2- 172.21.15.154:0/4061658820 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558923300690 0x558923378330 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0018c80 tx=0x7fbdf0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=534267240bfa9868 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:26:45.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.129+0000 7fbde4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:26:45.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.129+0000 7fbde4ff9640 1 -- 172.21.15.154:0/4061658820 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf0025820 con 0x558923300690 2023-09-23T03:26:45.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.129+0000 7fbde4ff9640 1 -- 172.21.15.154:0/4061658820 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fbdf0025980 con 0x558923300690 2023-09-23T03:26:45.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.129+0000 7fbe0ad71740 1 -- 172.21.15.154:0/4061658820 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbdf40043a0 con 0x558923300690 2023-09-23T03:26:45.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.129+0000 7fbde4ff9640 1 -- 172.21.15.154:0/4061658820 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbdf0025b40 con 0x558923300690 2023-09-23T03:26:45.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.129+0000 7fbe0ad71740 10 client.8679 fetch_fsmap learned FSMap version 99 2023-09-23T03:26:45.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.129+0000 7fbe0ad71740 1 -- 172.21.15.154:0/4061658820 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x5589231dbce0 con 0x558923300690 2023-09-23T03:26:45.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.130+0000 7fbde4ff9640 1 -- 172.21.15.154:0/4061658820 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf0039070 con 0x558923300690 2023-09-23T03:26:45.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.130+0000 7fbde4ff9640 1 -- 172.21.15.154:0/4061658820 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 99) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fbdf0030020 con 0x558923300690 2023-09-23T03:26:45.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.130+0000 7fbe0ad71740 10 client.8679 fetch_fsmap finished waiting for FSMap version 99 2023-09-23T03:26:45.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.130+0000 7fbe0ad71740 10 client.8679 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2023-09-23T03:26:45.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.130+0000 7fbe0ad71740 20 client.8679 populate_metadata read hostname 'smithi154' 2023-09-23T03:26:45.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.130+0000 7fbe0ad71740 1 --2- 172.21.15.154:0/4061658820 >> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] conn(0x558923303020 0x5589232ee1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:26:45.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.130+0000 7fbe0ad71740 4 client.8679 mds_command: new command op to 8595 tid=0[{"prefix": "scrub status"}] 2023-09-23T03:26:45.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.130+0000 7fbe0ad71740 1 -- 172.21.15.154:0/4061658820 --> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558923237a10 con 0x558923303020 2023-09-23T03:26:45.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.130+0000 7fbe0577d640 1 --2- 172.21.15.154:0/4061658820 >> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] conn(0x558923303020 0x5589232ee1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:26:45.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.130+0000 7fbe0577d640 1 --2- 172.21.15.154:0/4061658820 >> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] conn(0x558923303020 0x5589232ee1a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6feb1b6328fb4ea9 server_cookie=246d77430bac6b08 in_seq=0 out_seq=0 2023-09-23T03:26:45.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.130+0000 7fbde4ff9640 10 client.8679 ms_handle_connect on v2:172.21.15.154:6835/1758434763 2023-09-23T03:26:45.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.131+0000 7fbde4ff9640 1 -- 172.21.15.154:0/4061658820 <== mds.0 v2:172.21.15.154:6835/1758434763 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x558923237a10 con 0x558923303020 2023-09-23T03:26:45.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.131+0000 7fbde4ff9640 10 client.8679 handle_command_reply: tid=0 2023-09-23T03:26:45.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.131+0000 7fbe0ad71740 1 client.8679 shutdown 2023-09-23T03:26:45.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.131+0000 7fbe0ad71740 1 -- 172.21.15.154:0/4061658820 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558923300690 msgr2=0x558923378330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:45.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.131+0000 7fbe0ad71740 1 --2- 172.21.15.154:0/4061658820 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558923300690 0x558923378330 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0018c80 tx=0x7fbdf0004ab0 comp rx=0 tx=0).stop 2023-09-23T03:26:45.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.131+0000 7fbe0ad71740 1 -- 172.21.15.154:0/4061658820 shutdown_connections 2023-09-23T03:26:45.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.131+0000 7fbe0ad71740 1 --2- 172.21.15.154:0/4061658820 >> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] conn(0x558923303020 0x5589232ee1a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:45.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.131+0000 7fbe0ad71740 1 --2- 172.21.15.154:0/4061658820 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x558923317a20 0x558923318340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:45.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.131+0000 7fbe0ad71740 1 --2- 172.21.15.154:0/4061658820 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558923300a70 0x558923317e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:45.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.131+0000 7fbde4ff9640 0 client.8679 ms_handle_reset on v2:172.21.15.154:6835/1758434763 2023-09-23T03:26:45.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.131+0000 7fbe0ad71740 1 --2- 172.21.15.154:0/4061658820 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558923300690 0x558923378330 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:45.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.131+0000 7fbe0ad71740 1 -- 172.21.15.154:0/4061658820 >> 172.21.15.154:0/4061658820 conn(0x5589232d3ad0 msgr2=0x5589232f2990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:45.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.131+0000 7fbe0ad71740 1 -- 172.21.15.154:0/4061658820 shutdown_connections 2023-09-23T03:26:45.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.131+0000 7fbe0ad71740 1 -- 172.21.15.154:0/4061658820 wait complete. 2023-09-23T03:26:45.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.131+0000 7fbe0ad71740 20 client.8679 trim_cache size 0 max 16384 2023-09-23T03:26:45.137 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:26:45.137 INFO:teuthology.orchestra.run.smithi154.stdout: "status": "IDLE (waiting for more scrubs)", 2023-09-23T03:26:45.137 INFO:teuthology.orchestra.run.smithi154.stdout: "scrubs": { 2023-09-23T03:26:45.138 INFO:teuthology.orchestra.run.smithi154.stdout: "f3c89d87-c258-444b-91fe-b9c64274a2a5": { 2023-09-23T03:26:45.138 INFO:teuthology.orchestra.run.smithi154.stdout: "path": "/", 2023-09-23T03:26:45.138 INFO:teuthology.orchestra.run.smithi154.stdout: "tag": "f3c89d87-c258-444b-91fe-b9c64274a2a5", 2023-09-23T03:26:45.138 INFO:teuthology.orchestra.run.smithi154.stdout: "options": "recursive,repair" 2023-09-23T03:26:45.138 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:26:45.138 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:26:45.138 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:26:45.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.132+0000 7fbe081e2640 1 -- 172.21.15.154:0/3128618869 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbdec06afb0 msgr2=0x7fbdec06d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:45.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.132+0000 7fbe081e2640 1 --2- 172.21.15.154:0/3128618869 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbdec06afb0 0x7fbdec06d470 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fbdf0003b80 tx=0x7fbdf000b040 comp rx=0 tx=0).stop 2023-09-23T03:26:45.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.132+0000 7fbe081e2640 1 -- 172.21.15.154:0/3128618869 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe000b1fd0 msgr2=0x7fbe000cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:26:45.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.132+0000 7fbe081e2640 1 --2- 172.21.15.154:0/3128618869 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe000b1fd0 0x7fbe000cd910 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fbe000cb7e0 tx=0x7fbdf800dc00 comp rx=0 tx=0).stop 2023-09-23T03:26:45.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.132+0000 7fbe081e2640 1 -- 172.21.15.154:0/3128618869 shutdown_connections 2023-09-23T03:26:45.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.132+0000 7fbe081e2640 1 --2- 172.21.15.154:0/3128618869 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbdec06afb0 0x7fbdec06d470 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:45.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.132+0000 7fbe081e2640 1 --2- 172.21.15.154:0/3128618869 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbe000b1fd0 0x7fbe000cd910 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:45.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.132+0000 7fbe081e2640 1 --2- 172.21.15.154:0/3128618869 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbe000c68a0 0x7fbe000c7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:45.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.132+0000 7fbe081e2640 1 --2- 172.21.15.154:0/3128618869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbde8003ef0 0x7fbe001547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:26:45.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.132+0000 7fbe081e2640 1 -- 172.21.15.154:0/3128618869 >> 172.21.15.154:0/3128618869 conn(0x7fbe0006c010 msgr2=0x7fbe000aeeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:26:45.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.132+0000 7fbe081e2640 1 -- 172.21.15.154:0/3128618869 shutdown_connections 2023-09-23T03:26:45.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:26:45.132+0000 7fbe081e2640 1 -- 172.21.15.154:0/3128618869 wait complete. 2023-09-23T03:26:45.149 INFO:tasks.cephfs.filesystem:scrub status for tag:f3c89d87-c258-444b-91fe-b9c64274a2a5 - {'path': '/', 'tag': 'f3c89d87-c258-444b-91fe-b9c64274a2a5', 'options': 'recursive,repair'} 2023-09-23T03:27:14.354 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:27:14.364 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:27:15.150 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 scrub status 2023-09-23T03:27:15.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.373+0000 7fb7eed5e640 1 Processor -- start 2023-09-23T03:27:15.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.373+0000 7fb7eed5e640 1 -- start start 2023-09-23T03:27:15.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.374+0000 7fb7eed5e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb7e80c8d70 0x7fb7e815c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:15.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.374+0000 7fb7eed5e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7e80c9150 0x7fb7e815cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:15.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.374+0000 7fb7eed5e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb7e8071290 0x7fb7e80ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:15.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.374+0000 7fb7eed5e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7e80c9d20 con 0x7fb7e8071290 2023-09-23T03:27:15.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.375+0000 7fb7eed5e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7e8071850 con 0x7fb7e80c8d70 2023-09-23T03:27:15.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.375+0000 7fb7eed5e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e8071b30 con 0x7fb7e80c9150 2023-09-23T03:27:15.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.375+0000 7fb7ed2fb640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb7e8071290 0x7fb7e80ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:15.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.375+0000 7fb7ed2fb640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb7e8071290 0x7fb7e80ca1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46496/0 (socket says 172.21.15.154:46496) 2023-09-23T03:27:15.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.375+0000 7fb7ed2fb640 1 -- 172.21.15.154:0/652592121 learned_addr learned my addr 172.21.15.154:0/652592121 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:15.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.375+0000 7fb7ecafa640 1 --2- 172.21.15.154:0/652592121 >> v2:172.21.15.190:3300/0 conn(0x7fb7e80c8d70 0x7fb7e815c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:15.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.375+0000 7fb7ed2fb640 1 -- 172.21.15.154:0/652592121 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7e80c9150 msgr2=0x7fb7e815cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:15.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.375+0000 7fb7dffff640 1 --2- 172.21.15.154:0/652592121 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7e80c9150 0x7fb7e815cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:15.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.375+0000 7fb7ed2fb640 1 --2- 172.21.15.154:0/652592121 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7e80c9150 0x7fb7e815cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:15.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.375+0000 7fb7ed2fb640 1 -- 172.21.15.154:0/652592121 >> v2:172.21.15.190:3300/0 conn(0x7fb7e80c8d70 msgr2=0x7fb7e815c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:15.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.375+0000 7fb7ed2fb640 1 --2- 172.21.15.154:0/652592121 >> v2:172.21.15.190:3300/0 conn(0x7fb7e80c8d70 0x7fb7e815c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:15.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.375+0000 7fb7ed2fb640 1 -- 172.21.15.154:0/652592121 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e8071e10 con 0x7fb7e8071290 2023-09-23T03:27:15.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.375+0000 7fb7dffff640 1 --2- 172.21.15.154:0/652592121 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7e80c9150 0x7fb7e815cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:15.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.375+0000 7fb7ecafa640 1 --2- 172.21.15.154:0/652592121 >> v2:172.21.15.190:3300/0 conn(0x7fb7e80c8d70 0x7fb7e815c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:15.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.376+0000 7fb7ed2fb640 1 --2- 172.21.15.154:0/652592121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7e8071290 0x7fb7e80ca1b0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fb7e806b790 tx=0x7fb7e002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7396b9320a57dd43 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:15.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.377+0000 7fb7df7fe640 1 -- 172.21.15.154:0/652592121 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e002c9f0 con 0x7fb7e8071290 2023-09-23T03:27:15.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.377+0000 7fb7df7fe640 1 -- 172.21.15.154:0/652592121 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e002cb50 con 0x7fb7e8071290 2023-09-23T03:27:15.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.377+0000 7fb7df7fe640 1 -- 172.21.15.154:0/652592121 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e00343f0 con 0x7fb7e8071290 2023-09-23T03:27:15.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.377+0000 7fb7eed5e640 1 -- 172.21.15.154:0/652592121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7e8071290 msgr2=0x7fb7e80ca1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:15.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.377+0000 7fb7eed5e640 1 --2- 172.21.15.154:0/652592121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7e8071290 0x7fb7e80ca1b0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fb7e806b790 tx=0x7fb7e002be50 comp rx=0 tx=0).stop 2023-09-23T03:27:15.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.377+0000 7fb7eed5e640 1 -- 172.21.15.154:0/652592121 shutdown_connections 2023-09-23T03:27:15.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.377+0000 7fb7eed5e640 1 --2- 172.21.15.154:0/652592121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7e8071290 0x7fb7e80ca1b0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:15.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.377+0000 7fb7eed5e640 1 --2- 172.21.15.154:0/652592121 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7e80c9150 0x7fb7e815cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:15.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.377+0000 7fb7eed5e640 1 --2- 172.21.15.154:0/652592121 >> v2:172.21.15.190:3300/0 conn(0x7fb7e80c8d70 0x7fb7e815c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:15.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.377+0000 7fb7eed5e640 1 -- 172.21.15.154:0/652592121 >> 172.21.15.154:0/652592121 conn(0x7fb7e806c010 msgr2=0x7fb7e806c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:15.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.378+0000 7fb7eed5e640 1 -- 172.21.15.154:0/652592121 shutdown_connections 2023-09-23T03:27:15.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.378+0000 7fb7eed5e640 1 -- 172.21.15.154:0/652592121 wait complete. 2023-09-23T03:27:15.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.379+0000 7fb7eed5e640 1 Processor -- start 2023-09-23T03:27:15.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.379+0000 7fb7eed5e640 1 -- start start 2023-09-23T03:27:15.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.380+0000 7fb7eed5e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7e8071290 0x7fb7e80c5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:15.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.380+0000 7fb7eed5e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7e80c8d70 0x7fb7e81f59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:15.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.380+0000 7fb7eed5e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7e80c9150 0x7fb7e80b2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:15.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.380+0000 7fb7eed5e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e80c6ce0 con 0x7fb7e8071290 2023-09-23T03:27:15.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.380+0000 7fb7eed5e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e80c7030 con 0x7fb7e80c8d70 2023-09-23T03:27:15.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.380+0000 7fb7eed5e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e81f6420 con 0x7fb7e80c9150 2023-09-23T03:27:15.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.381+0000 7fb7ecafa640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7e8071290 0x7fb7e80c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:15.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.381+0000 7fb7ecafa640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7e8071290 0x7fb7e80c5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46502/0 (socket says 172.21.15.154:46502) 2023-09-23T03:27:15.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.381+0000 7fb7ecafa640 1 -- 172.21.15.154:0/2827190532 learned_addr learned my addr 172.21.15.154:0/2827190532 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:15.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.381+0000 7fb7ed2fb640 1 --2- 172.21.15.154:0/2827190532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7e80c9150 0x7fb7e80b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:15.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.381+0000 7fb7dffff640 1 --2- 172.21.15.154:0/2827190532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7e80c8d70 0x7fb7e81f59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:15.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.381+0000 7fb7ecafa640 1 -- 172.21.15.154:0/2827190532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7e80c9150 msgr2=0x7fb7e80b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:15.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.381+0000 7fb7ecafa640 1 --2- 172.21.15.154:0/2827190532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7e80c9150 0x7fb7e80b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:15.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.381+0000 7fb7ecafa640 1 -- 172.21.15.154:0/2827190532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7e80c8d70 msgr2=0x7fb7e81f59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:15.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.381+0000 7fb7ecafa640 1 --2- 172.21.15.154:0/2827190532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7e80c8d70 0x7fb7e81f59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:15.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.381+0000 7fb7ecafa640 1 -- 172.21.15.154:0/2827190532 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e0009b30 con 0x7fb7e8071290 2023-09-23T03:27:15.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.382+0000 7fb7ecafa640 1 --2- 172.21.15.154:0/2827190532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7e8071290 0x7fb7e80c5580 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fb7e400c7f0 tx=0x7fb7e400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:15.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.382+0000 7fb7dd7fa640 1 -- 172.21.15.154:0/2827190532 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4007820 con 0x7fb7e8071290 2023-09-23T03:27:15.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.382+0000 7fb7dd7fa640 1 -- 172.21.15.154:0/2827190532 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e4007980 con 0x7fb7e8071290 2023-09-23T03:27:15.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.382+0000 7fb7dd7fa640 1 -- 172.21.15.154:0/2827190532 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4007d30 con 0x7fb7e8071290 2023-09-23T03:27:15.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.382+0000 7fb7eed5e640 1 -- 172.21.15.154:0/2827190532 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7e81f6700 con 0x7fb7e8071290 2023-09-23T03:27:15.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.382+0000 7fb7eed5e640 1 -- 172.21.15.154:0/2827190532 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7e80c7560 con 0x7fb7e8071290 2023-09-23T03:27:15.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.384+0000 7fb7dd7fa640 1 -- 172.21.15.154:0/2827190532 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fb7e40113f0 con 0x7fb7e8071290 2023-09-23T03:27:15.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.386+0000 7fb7dd7fa640 1 --2- 172.21.15.154:0/2827190532 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb7d406af30 0x7fb7d406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:15.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.386+0000 7fb7dd7fa640 1 -- 172.21.15.154:0/2827190532 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fb7e4089e90 con 0x7fb7e8071290 2023-09-23T03:27:15.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.386+0000 7fb7dffff640 1 --2- 172.21.15.154:0/2827190532 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb7d406af30 0x7fb7d406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:15.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.387+0000 7fb7f19c9740 1 Processor -- start 2023-09-23T03:27:15.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.387+0000 7fb7dffff640 1 --2- 172.21.15.154:0/2827190532 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb7d406af30 0x7fb7d406d3f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb7e80cd380 tx=0x7fb7d8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:15.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.387+0000 7fb7f19c9740 1 -- start start 2023-09-23T03:27:15.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.388+0000 7fb7f19c9740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563061495690 0x563061468590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:15.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.388+0000 7fb7f19c9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563061495a70 0x563061498420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:15.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.388+0000 7fb7f19c9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563061468ad0 0x563061498960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:15.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.388+0000 7fb7f19c9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630613daa50 con 0x563061468ad0 2023-09-23T03:27:15.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.388+0000 7fb7f19c9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630613d8140 con 0x563061495a70 2023-09-23T03:27:15.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.388+0000 7fb7f19c9740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5630613d3a30 con 0x563061495690 2023-09-23T03:27:15.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.388+0000 7fb7ed2fb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563061468ad0 0x563061498960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:15.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.388+0000 7fb7ed2fb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563061468ad0 0x563061498960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46504/0 (socket says 172.21.15.154:46504) 2023-09-23T03:27:15.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.388+0000 7fb7ed2fb640 1 -- 172.21.15.154:0/2950205697 learned_addr learned my addr 172.21.15.154:0/2950205697 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:15.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.389+0000 7fb7dffff640 1 --2- 172.21.15.154:0/2950205697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563061495a70 0x563061498420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:15.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.389+0000 7fb7ecafa640 1 --2- 172.21.15.154:0/2950205697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563061495690 0x563061468590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:15.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.389+0000 7fb7ed2fb640 1 -- 172.21.15.154:0/2950205697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563061495690 msgr2=0x563061468590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:15.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.389+0000 7fb7ed2fb640 1 --2- 172.21.15.154:0/2950205697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563061495690 0x563061468590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:15.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.389+0000 7fb7ed2fb640 1 -- 172.21.15.154:0/2950205697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563061495a70 msgr2=0x563061498420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:15.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.389+0000 7fb7ed2fb640 1 --2- 172.21.15.154:0/2950205697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563061495a70 0x563061498420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:15.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.389+0000 7fb7ed2fb640 1 -- 172.21.15.154:0/2950205697 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563061397a70 con 0x563061468ad0 2023-09-23T03:27:15.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.389+0000 7fb7ecafa640 1 --2- 172.21.15.154:0/2950205697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563061495690 0x563061468590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:15.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.389+0000 7fb7dffff640 1 --2- 172.21.15.154:0/2950205697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563061495a70 0x563061498420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:15.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.389+0000 7fb7ed2fb640 1 --2- 172.21.15.154:0/2950205697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563061468ad0 0x563061498960 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0009f40 tx=0x7fb7e002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56018f26a8b5b518 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:15.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.389+0000 7fb7ce7fc640 1 -- 172.21.15.154:0/2950205697 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e003e630 con 0x563061468ad0 2023-09-23T03:27:15.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.389+0000 7fb7ce7fc640 1 -- 172.21.15.154:0/2950205697 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e003e790 con 0x563061468ad0 2023-09-23T03:27:15.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.389+0000 7fb7ce7fc640 1 -- 172.21.15.154:0/2950205697 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e003eb00 con 0x563061468ad0 2023-09-23T03:27:15.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.390+0000 7fb7f19c9740 1 -- 172.21.15.154:0/2950205697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563061468ad0 msgr2=0x563061498960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:15.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.390+0000 7fb7f19c9740 1 --2- 172.21.15.154:0/2950205697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563061468ad0 0x563061498960 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0009f40 tx=0x7fb7e002e000 comp rx=0 tx=0).stop 2023-09-23T03:27:15.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.390+0000 7fb7f19c9740 1 -- 172.21.15.154:0/2950205697 shutdown_connections 2023-09-23T03:27:15.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.390+0000 7fb7f19c9740 1 --2- 172.21.15.154:0/2950205697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563061468ad0 0x563061498960 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:15.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.390+0000 7fb7f19c9740 1 --2- 172.21.15.154:0/2950205697 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563061495a70 0x563061498420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:15.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.390+0000 7fb7f19c9740 1 --2- 172.21.15.154:0/2950205697 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563061495690 0x563061468590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:15.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.390+0000 7fb7f19c9740 1 -- 172.21.15.154:0/2950205697 >> 172.21.15.154:0/2950205697 conn(0x56306146a8a0 msgr2=0x5630614b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:15.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.390+0000 7fb7f19c9740 1 -- 172.21.15.154:0/2950205697 shutdown_connections 2023-09-23T03:27:15.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.390+0000 7fb7f19c9740 1 -- 172.21.15.154:0/2950205697 wait complete. 2023-09-23T03:27:15.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.391+0000 7fb7f19c9740 1 -- start start 2023-09-23T03:27:15.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.391+0000 7fb7f19c9740 1 Processor -- start 2023-09-23T03:27:15.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.392+0000 7fb7f19c9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563061495690 0x5630614b5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:15.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.392+0000 7fb7f19c9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563061495a70 0x5630614b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:15.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.392+0000 7fb7f19c9740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56306149c0e0 0x5630614c3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:15.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.392+0000 7fb7f19c9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x563061371ad0 con 0x563061495690 2023-09-23T03:27:15.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.392+0000 7fb7f19c9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563061371770 con 0x563061495a70 2023-09-23T03:27:15.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.392+0000 7fb7f19c9740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5630610cdf90 con 0x56306149c0e0 2023-09-23T03:27:15.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.392+0000 7fb7f19c9740 10 client.0 _ll_register_callbacks cb 0x563061395220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:27:15.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.392+0000 7fb7ecafa640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563061495690 0x5630614b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:15.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.392+0000 7fb7ecafa640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563061495690 0x5630614b5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46516/0 (socket says 172.21.15.154:46516) 2023-09-23T03:27:15.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.393+0000 7fb7ecafa640 1 -- 172.21.15.154:0/2985069835 learned_addr learned my addr 172.21.15.154:0/2985069835 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:15.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.393+0000 7fb7ecafa640 1 -- 172.21.15.154:0/2985069835 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56306149c0e0 msgr2=0x5630614c3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:15.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.393+0000 7fb7ecafa640 1 --2- 172.21.15.154:0/2985069835 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56306149c0e0 0x5630614c3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:15.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.393+0000 7fb7ecafa640 1 -- 172.21.15.154:0/2985069835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563061495a70 msgr2=0x5630614b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:15.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.393+0000 7fb7dffff640 1 --2- 172.21.15.154:0/2985069835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563061495a70 0x5630614b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:15.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.393+0000 7fb7ecafa640 1 --2- 172.21.15.154:0/2985069835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563061495a70 0x5630614b55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:15.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.393+0000 7fb7ecafa640 1 -- 172.21.15.154:0/2985069835 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7e8071e10 con 0x563061495690 2023-09-23T03:27:15.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.393+0000 7fb7dffff640 1 --2- 172.21.15.154:0/2985069835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563061495a70 0x5630614b55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:15.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.393+0000 7fb7ecafa640 1 --2- 172.21.15.154:0/2985069835 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563061495690 0x5630614b5080 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4004770 tx=0x7fb7e40587e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=150593926e20f710 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:15.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.393+0000 7fb7c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:27:15.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.393+0000 7fb7c77fe640 1 -- 172.21.15.154:0/2985069835 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4057c10 con 0x563061495690 2023-09-23T03:27:15.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.393+0000 7fb7c77fe640 1 -- 172.21.15.154:0/2985069835 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fb7e4057d70 con 0x563061495690 2023-09-23T03:27:15.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.393+0000 7fb7f19c9740 1 -- 172.21.15.154:0/2985069835 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5630613fa4d0 con 0x563061495690 2023-09-23T03:27:15.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.394+0000 7fb7c77fe640 1 -- 172.21.15.154:0/2985069835 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7e4060590 con 0x563061495690 2023-09-23T03:27:15.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.394+0000 7fb7f19c9740 10 client.8691 fetch_fsmap learned FSMap version 99 2023-09-23T03:27:15.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.394+0000 7fb7f19c9740 1 -- 172.21.15.154:0/2985069835 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x5630613d20b0 con 0x563061495690 2023-09-23T03:27:15.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.394+0000 7fb7c77fe640 1 -- 172.21.15.154:0/2985069835 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e4085080 con 0x563061495690 2023-09-23T03:27:15.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.394+0000 7fb7c77fe640 1 -- 172.21.15.154:0/2985069835 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 99) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fb7e4054510 con 0x563061495690 2023-09-23T03:27:15.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.394+0000 7fb7f19c9740 10 client.8691 fetch_fsmap finished waiting for FSMap version 99 2023-09-23T03:27:15.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.394+0000 7fb7f19c9740 10 client.8691 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2023-09-23T03:27:15.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.394+0000 7fb7f19c9740 20 client.8691 populate_metadata read hostname 'smithi154' 2023-09-23T03:27:15.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.394+0000 7fb7f19c9740 1 --2- 172.21.15.154:0/2985069835 >> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] conn(0x5630614c38c0 0x56306149a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:15.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.394+0000 7fb7f19c9740 4 client.8691 mds_command: new command op to 8595 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T03:27:15.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.394+0000 7fb7f19c9740 1 -- 172.21.15.154:0/2985069835 --> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5630613a6ab0 con 0x5630614c38c0 2023-09-23T03:27:15.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.394+0000 7fb7dffff640 1 --2- 172.21.15.154:0/2985069835 >> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] conn(0x5630614c38c0 0x56306149a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:15.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.395+0000 7fb7dffff640 1 --2- 172.21.15.154:0/2985069835 >> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] conn(0x5630614c38c0 0x56306149a460 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fe00f085e78217e server_cookie=4a0e27c6267323cb in_seq=0 out_seq=0 2023-09-23T03:27:15.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.395+0000 7fb7c77fe640 10 client.8691 ms_handle_connect on v2:172.21.15.154:6835/1758434763 2023-09-23T03:27:15.400 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.396+0000 7fb7c77fe640 1 -- 172.21.15.154:0/2985069835 <== mds.0 v2:172.21.15.154:6835/1758434763 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5630613a6ab0 con 0x5630614c38c0 2023-09-23T03:27:15.400 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.396+0000 7fb7c77fe640 10 client.8691 handle_command_reply: tid=0 2023-09-23T03:27:15.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.396+0000 7fb7f19c9740 1 client.8691 shutdown 2023-09-23T03:27:15.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.396+0000 7fb7f19c9740 1 -- 172.21.15.154:0/2985069835 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563061495690 msgr2=0x5630614b5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:15.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.396+0000 7fb7f19c9740 1 --2- 172.21.15.154:0/2985069835 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563061495690 0x5630614b5080 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4004770 tx=0x7fb7e40587e0 comp rx=0 tx=0).stop 2023-09-23T03:27:15.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.396+0000 7fb7f19c9740 1 -- 172.21.15.154:0/2985069835 shutdown_connections 2023-09-23T03:27:15.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.396+0000 7fb7f19c9740 1 --2- 172.21.15.154:0/2985069835 >> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] conn(0x5630614c38c0 0x56306149a460 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:15.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.396+0000 7fb7f19c9740 1 --2- 172.21.15.154:0/2985069835 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56306149c0e0 0x5630614c3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:15.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.396+0000 7fb7f19c9740 1 --2- 172.21.15.154:0/2985069835 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563061495a70 0x5630614b55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:15.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.396+0000 7fb7f19c9740 1 --2- 172.21.15.154:0/2985069835 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563061495690 0x5630614b5080 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:15.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.396+0000 7fb7c77fe640 0 client.8691 ms_handle_reset on v2:172.21.15.154:6835/1758434763 2023-09-23T03:27:15.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.396+0000 7fb7f19c9740 1 -- 172.21.15.154:0/2985069835 >> 172.21.15.154:0/2985069835 conn(0x563061468ad0 msgr2=0x5630614b6930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:15.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.396+0000 7fb7f19c9740 1 -- 172.21.15.154:0/2985069835 shutdown_connections 2023-09-23T03:27:15.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.396+0000 7fb7f19c9740 1 -- 172.21.15.154:0/2985069835 wait complete. 2023-09-23T03:27:15.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.396+0000 7fb7f19c9740 20 client.8691 trim_cache size 0 max 16384 2023-09-23T03:27:15.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.412+0000 7fb7f19c9740 1 Processor -- start 2023-09-23T03:27:15.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.412+0000 7fb7f19c9740 1 -- start start 2023-09-23T03:27:15.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.413+0000 7fb7f19c9740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563061495690 0x5630614b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:15.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.413+0000 7fb7f19c9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563061495a70 0x5630614ab930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:15.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.413+0000 7fb7f19c9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56306149c0e0 0x563061491f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:15.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.413+0000 7fb7f19c9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630614d5e30 con 0x56306149c0e0 2023-09-23T03:27:15.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.413+0000 7fb7f19c9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x563061498de0 con 0x563061495a70 2023-09-23T03:27:15.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.413+0000 7fb7f19c9740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5630614adf80 con 0x563061495690 2023-09-23T03:27:15.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.413+0000 7fb7ed2fb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56306149c0e0 0x563061491f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:15.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.413+0000 7fb7ed2fb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56306149c0e0 0x563061491f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46522/0 (socket says 172.21.15.154:46522) 2023-09-23T03:27:15.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.413+0000 7fb7ed2fb640 1 -- 172.21.15.154:0/1601978020 learned_addr learned my addr 172.21.15.154:0/1601978020 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:15.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.414+0000 7fb7ed2fb640 1 -- 172.21.15.154:0/1601978020 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563061495690 msgr2=0x5630614b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:15.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.414+0000 7fb7ed2fb640 1 --2- 172.21.15.154:0/1601978020 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563061495690 0x5630614b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:15.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.414+0000 7fb7ed2fb640 1 -- 172.21.15.154:0/1601978020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563061495a70 msgr2=0x5630614ab930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:15.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.414+0000 7fb7ed2fb640 1 --2- 172.21.15.154:0/1601978020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563061495a70 0x5630614ab930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:15.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.414+0000 7fb7ed2fb640 1 -- 172.21.15.154:0/1601978020 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563061403da0 con 0x56306149c0e0 2023-09-23T03:27:15.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.414+0000 7fb7ed2fb640 1 --2- 172.21.15.154:0/1601978020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56306149c0e0 0x563061491f80 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0009f10 tx=0x7fb7e002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eebb46bad75dd00 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:15.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.414+0000 7fb7ceffd640 1 -- 172.21.15.154:0/1601978020 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e003e630 con 0x56306149c0e0 2023-09-23T03:27:15.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.414+0000 7fb7ceffd640 1 -- 172.21.15.154:0/1601978020 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e0003990 con 0x56306149c0e0 2023-09-23T03:27:15.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.414+0000 7fb7ceffd640 1 -- 172.21.15.154:0/1601978020 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e0003d00 con 0x56306149c0e0 2023-09-23T03:27:15.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.415+0000 7fb7f19c9740 1 -- 172.21.15.154:0/1601978020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56306149c0e0 msgr2=0x563061491f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:15.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.415+0000 7fb7f19c9740 1 --2- 172.21.15.154:0/1601978020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56306149c0e0 0x563061491f80 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0009f10 tx=0x7fb7e002e070 comp rx=0 tx=0).stop 2023-09-23T03:27:15.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.415+0000 7fb7f19c9740 1 -- 172.21.15.154:0/1601978020 shutdown_connections 2023-09-23T03:27:15.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.415+0000 7fb7f19c9740 1 --2- 172.21.15.154:0/1601978020 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56306149c0e0 0x563061491f80 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:15.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.415+0000 7fb7f19c9740 1 --2- 172.21.15.154:0/1601978020 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x563061495a70 0x5630614ab930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:15.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.415+0000 7fb7f19c9740 1 --2- 172.21.15.154:0/1601978020 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563061495690 0x5630614b4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:15.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.415+0000 7fb7f19c9740 1 -- 172.21.15.154:0/1601978020 >> 172.21.15.154:0/1601978020 conn(0x563061468ad0 msgr2=0x5630614b5e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:15.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.415+0000 7fb7f19c9740 1 -- 172.21.15.154:0/1601978020 shutdown_connections 2023-09-23T03:27:15.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.415+0000 7fb7f19c9740 1 -- 172.21.15.154:0/1601978020 wait complete. 2023-09-23T03:27:15.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.416+0000 7fb7f19c9740 1 -- start start 2023-09-23T03:27:15.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.416+0000 7fb7f19c9740 1 Processor -- start 2023-09-23T03:27:15.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.417+0000 7fb7f19c9740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563061495690 0x5630614ab980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:15.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.417+0000 7fb7f19c9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563061495a70 0x5630614b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:15.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.417+0000 7fb7f19c9740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56306149c0e0 0x5630614c2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:15.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.417+0000 7fb7f19c9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630614b70a0 con 0x563061495a70 2023-09-23T03:27:15.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.417+0000 7fb7f19c9740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630614ac400 con 0x56306149c0e0 2023-09-23T03:27:15.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.417+0000 7fb7f19c9740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5630614b6e50 con 0x563061495690 2023-09-23T03:27:15.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.417+0000 7fb7f19c9740 10 client.0 _ll_register_callbacks cb 0x7fb7e0009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:27:15.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.417+0000 7fb7dffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563061495a70 0x5630614b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:15.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.417+0000 7fb7dffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563061495a70 0x5630614b3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46536/0 (socket says 172.21.15.154:46536) 2023-09-23T03:27:15.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.417+0000 7fb7dffff640 1 -- 172.21.15.154:0/2402837088 learned_addr learned my addr 172.21.15.154:0/2402837088 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:15.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.418+0000 7fb7dffff640 1 -- 172.21.15.154:0/2402837088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563061495690 msgr2=0x5630614ab980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:15.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.418+0000 7fb7dffff640 1 --2- 172.21.15.154:0/2402837088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563061495690 0x5630614ab980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:15.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.418+0000 7fb7dffff640 1 -- 172.21.15.154:0/2402837088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56306149c0e0 msgr2=0x5630614c2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:15.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.418+0000 7fb7dffff640 1 --2- 172.21.15.154:0/2402837088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56306149c0e0 0x5630614c2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:15.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.418+0000 7fb7dffff640 1 -- 172.21.15.154:0/2402837088 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563061397a70 con 0x563061495a70 2023-09-23T03:27:15.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.418+0000 7fb7dffff640 1 --2- 172.21.15.154:0/2402837088 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563061495a70 0x5630614b3c30 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fb7e80cd330 tx=0x7fb7d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1258c6e8bb76645 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:15.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.418+0000 7fb7c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:27:15.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.418+0000 7fb7c77fe640 1 -- 172.21.15.154:0/2402837088 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d801b040 con 0x563061495a70 2023-09-23T03:27:15.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.418+0000 7fb7c77fe640 1 -- 172.21.15.154:0/2402837088 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fb7d80144c0 con 0x563061495a70 2023-09-23T03:27:15.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.418+0000 7fb7f19c9740 1 -- 172.21.15.154:0/2402837088 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb7e003e790 con 0x563061495a70 2023-09-23T03:27:15.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.419+0000 7fb7c77fe640 1 -- 172.21.15.154:0/2402837088 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7d8014680 con 0x563061495a70 2023-09-23T03:27:15.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.419+0000 7fb7f19c9740 10 client.8697 fetch_fsmap learned FSMap version 99 2023-09-23T03:27:15.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.419+0000 7fb7f19c9740 1 -- 172.21.15.154:0/2402837088 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) v3 -- 0x563061370ce0 con 0x563061495a70 2023-09-23T03:27:15.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.419+0000 7fb7c77fe640 1 -- 172.21.15.154:0/2402837088 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e003e790 con 0x563061495a70 2023-09-23T03:27:15.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.419+0000 7fb7c77fe640 1 -- 172.21.15.154:0/2402837088 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 99) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fb7d80147e0 con 0x563061495a70 2023-09-23T03:27:15.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.419+0000 7fb7f19c9740 10 client.8697 fetch_fsmap finished waiting for FSMap version 99 2023-09-23T03:27:15.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.419+0000 7fb7f19c9740 10 client.8697 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2023-09-23T03:27:15.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.419+0000 7fb7f19c9740 20 client.8697 populate_metadata read hostname 'smithi154' 2023-09-23T03:27:15.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.419+0000 7fb7f19c9740 1 --2- 172.21.15.154:0/2402837088 >> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] conn(0x5630614b0370 0x5630614c31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:15.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.419+0000 7fb7f19c9740 4 client.8697 mds_command: new command op to 8595 tid=0[{"prefix": "scrub status"}] 2023-09-23T03:27:15.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.419+0000 7fb7f19c9740 1 -- 172.21.15.154:0/2402837088 --> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5630613cca10 con 0x5630614b0370 2023-09-23T03:27:15.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.419+0000 7fb7ecafa640 1 --2- 172.21.15.154:0/2402837088 >> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] conn(0x5630614b0370 0x5630614c31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:15.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.420+0000 7fb7ecafa640 1 --2- 172.21.15.154:0/2402837088 >> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] conn(0x5630614b0370 0x5630614c31b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bed8349fcd6fe733 server_cookie=9e5c7444163b1749 in_seq=0 out_seq=0 2023-09-23T03:27:15.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.420+0000 7fb7c77fe640 10 client.8697 ms_handle_connect on v2:172.21.15.154:6835/1758434763 2023-09-23T03:27:15.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.420+0000 7fb7c77fe640 1 -- 172.21.15.154:0/2402837088 <== mds.0 v2:172.21.15.154:6835/1758434763 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5630613cca10 con 0x5630614b0370 2023-09-23T03:27:15.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.420+0000 7fb7c77fe640 10 client.8697 handle_command_reply: tid=0 2023-09-23T03:27:15.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.420+0000 7fb7f19c9740 1 client.8697 shutdown 2023-09-23T03:27:15.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.420+0000 7fb7f19c9740 1 -- 172.21.15.154:0/2402837088 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563061495a70 msgr2=0x5630614b3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:15.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.420+0000 7fb7f19c9740 1 --2- 172.21.15.154:0/2402837088 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563061495a70 0x5630614b3c30 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fb7e80cd330 tx=0x7fb7d80046c0 comp rx=0 tx=0).stop 2023-09-23T03:27:15.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.420+0000 7fb7f19c9740 1 -- 172.21.15.154:0/2402837088 shutdown_connections 2023-09-23T03:27:15.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.420+0000 7fb7f19c9740 1 --2- 172.21.15.154:0/2402837088 >> [v2:172.21.15.154:6835/1758434763,v1:172.21.15.154:6837/1758434763] conn(0x5630614b0370 0x5630614c31b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:15.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.420+0000 7fb7f19c9740 1 --2- 172.21.15.154:0/2402837088 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56306149c0e0 0x5630614c2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:15.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.420+0000 7fb7f19c9740 1 --2- 172.21.15.154:0/2402837088 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563061495a70 0x5630614b3c30 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:15.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.420+0000 7fb7f19c9740 1 --2- 172.21.15.154:0/2402837088 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x563061495690 0x5630614ab980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:15.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.420+0000 7fb7c77fe640 0 client.8697 ms_handle_reset on v2:172.21.15.154:6835/1758434763 2023-09-23T03:27:15.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.420+0000 7fb7f19c9740 1 -- 172.21.15.154:0/2402837088 >> 172.21.15.154:0/2402837088 conn(0x563061468ad0 msgr2=0x563061491f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:15.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.421+0000 7fb7f19c9740 1 -- 172.21.15.154:0/2402837088 shutdown_connections 2023-09-23T03:27:15.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.421+0000 7fb7f19c9740 1 -- 172.21.15.154:0/2402837088 wait complete. 2023-09-23T03:27:15.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.421+0000 7fb7f19c9740 20 client.8697 trim_cache size 0 max 16384 2023-09-23T03:27:15.425 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:27:15.425 INFO:teuthology.orchestra.run.smithi154.stdout: "status": "no active scrubs running", 2023-09-23T03:27:15.425 INFO:teuthology.orchestra.run.smithi154.stdout: "scrubs": {} 2023-09-23T03:27:15.426 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:27:15.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.421+0000 7fb7eed5e640 1 -- 172.21.15.154:0/2827190532 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb7d406af30 msgr2=0x7fb7d406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:15.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.421+0000 7fb7eed5e640 1 --2- 172.21.15.154:0/2827190532 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb7d406af30 0x7fb7d406d3f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fb7e80cd380 tx=0x7fb7d8008040 comp rx=0 tx=0).stop 2023-09-23T03:27:15.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.421+0000 7fb7eed5e640 1 -- 172.21.15.154:0/2827190532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7e8071290 msgr2=0x7fb7e80c5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:15.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.421+0000 7fb7eed5e640 1 --2- 172.21.15.154:0/2827190532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7e8071290 0x7fb7e80c5580 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fb7e400c7f0 tx=0x7fb7e400ccb0 comp rx=0 tx=0).stop 2023-09-23T03:27:15.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.421+0000 7fb7eed5e640 1 -- 172.21.15.154:0/2827190532 shutdown_connections 2023-09-23T03:27:15.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.421+0000 7fb7eed5e640 1 --2- 172.21.15.154:0/2827190532 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb7d406af30 0x7fb7d406d3f0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:15.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.421+0000 7fb7eed5e640 1 --2- 172.21.15.154:0/2827190532 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7e80c9150 0x7fb7e80b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:15.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.421+0000 7fb7eed5e640 1 --2- 172.21.15.154:0/2827190532 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7e80c8d70 0x7fb7e81f59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:15.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.421+0000 7fb7eed5e640 1 --2- 172.21.15.154:0/2827190532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7e8071290 0x7fb7e80c5580 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:15.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.421+0000 7fb7eed5e640 1 -- 172.21.15.154:0/2827190532 >> 172.21.15.154:0/2827190532 conn(0x7fb7e806c010 msgr2=0x7fb7e80cc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:15.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.422+0000 7fb7eed5e640 1 -- 172.21.15.154:0/2827190532 shutdown_connections 2023-09-23T03:27:15.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:15.422+0000 7fb7eed5e640 1 -- 172.21.15.154:0/2827190532 wait complete. 2023-09-23T03:27:15.439 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-09-23T03:27:15.439 INFO:tasks.cephfs.mount:Validating 1048576 bytes from stripey/flushed_file_0 2023-09-23T03:27:15.439 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:27:15.439 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:27:15.439 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_0" 2023-09-23T03:27:15.439 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:27:15.439 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:27:15.439 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 1048576: 2023-09-23T03:27:15.439 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:27:15.440 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 1048576 2023-09-23T03:27:15.440 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:27:15.440 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:27:15.440 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:27:15.440 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:27:15.440 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:27:15.440 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:27:16.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:27:16.224 INFO:tasks.cephfs.mount:Validating 524288 bytes from stripey/flushed_file_1 2023-09-23T03:27:16.224 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:27:16.224 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:27:16.224 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_1" 2023-09-23T03:27:16.224 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:27:16.224 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:27:16.224 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 524288: 2023-09-23T03:27:16.224 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:27:16.225 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 524288 2023-09-23T03:27:16.225 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:27:16.225 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:27:16.225 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:27:16.225 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:27:16.225 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:27:16.225 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:27:16.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:27:16.766 INFO:tasks.cephfs.mount:Validating 917504 bytes from stripey/flushed_file_2 2023-09-23T03:27:16.766 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:27:16.766 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:27:16.767 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_2" 2023-09-23T03:27:16.767 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:27:16.767 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:27:16.767 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 917504: 2023-09-23T03:27:16.767 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:27:16.767 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 917504 2023-09-23T03:27:16.767 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:27:16.767 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:27:16.767 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:27:16.767 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:27:16.767 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:27:16.767 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:27:17.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:27:17.511 INFO:tasks.cephfs.mount:Validating 917503 bytes from stripey/flushed_file_3 2023-09-23T03:27:17.511 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:27:17.511 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:27:17.511 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_3" 2023-09-23T03:27:17.511 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:27:17.511 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:27:17.511 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 917503: 2023-09-23T03:27:17.511 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:27:17.512 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 917503 2023-09-23T03:27:17.512 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:27:17.512 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:27:17.512 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:27:17.512 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:27:17.512 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:27:17.512 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:27:18.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:27:18.279 INFO:tasks.cephfs.mount:Validating 1441792 bytes from stripey/flushed_file_4 2023-09-23T03:27:18.279 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:27:18.279 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:27:18.280 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_4" 2023-09-23T03:27:18.280 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:27:18.280 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:27:18.280 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 1441792: 2023-09-23T03:27:18.280 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:27:18.280 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 1441792 2023-09-23T03:27:18.280 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:27:18.280 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:27:18.280 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:27:18.280 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:27:18.280 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:27:18.280 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:27:19.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:27:19.279 INFO:tasks.cephfs.mount:Validating 1441793 bytes from stripey/flushed_file_5 2023-09-23T03:27:19.280 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:27:19.280 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:27:19.280 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_5" 2023-09-23T03:27:19.280 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:27:19.280 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:27:19.280 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 1441793: 2023-09-23T03:27:19.280 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:27:19.280 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 1441793 2023-09-23T03:27:19.280 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:27:19.280 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:27:19.280 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:27:19.280 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:27:19.281 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:27:19.281 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:27:20.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:27:20.322 INFO:tasks.cephfs.mount:Validating 1572864 bytes from stripey/flushed_file_6 2023-09-23T03:27:20.322 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:27:20.322 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:27:20.322 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_6" 2023-09-23T03:27:20.322 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:27:20.322 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:27:20.322 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 1572864: 2023-09-23T03:27:20.322 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:27:20.323 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 1572864 2023-09-23T03:27:20.323 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:27:20.323 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:27:20.323 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:27:20.323 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:27:20.323 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:27:20.323 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:27:21.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:27:21.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:27:21.380 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo ls lost+found) 2023-09-23T03:27:21.523 INFO:teuthology.orchestra.run.smithi154.stdout:100000001fc 2023-09-23T03:27:21.527 INFO:tasks.cephfs.mount:Validating 524288 bytes from lost+found/100000001fc 2023-09-23T03:27:21.528 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:27:21.528 DEBUG:teuthology.orchestra.run.smithi154:> import zlib 2023-09-23T03:27:21.528 DEBUG:teuthology.orchestra.run.smithi154:> path = "/home/ubuntu/cephtest/mnt.0/lost+found/100000001fc" 2023-09-23T03:27:21.528 DEBUG:teuthology.orchestra.run.smithi154:> with open(path, '"'"'r'"'"') as f: 2023-09-23T03:27:21.528 DEBUG:teuthology.orchestra.run.smithi154:> bytes = f.read() 2023-09-23T03:27:21.528 DEBUG:teuthology.orchestra.run.smithi154:> if len(bytes) != 524288: 2023-09-23T03:27:21.528 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-09-23T03:27:21.528 DEBUG:teuthology.orchestra.run.smithi154:> len(bytes), 524288 2023-09-23T03:27:21.528 DEBUG:teuthology.orchestra.run.smithi154:> )) 2023-09-23T03:27:21.528 DEBUG:teuthology.orchestra.run.smithi154:> for i, b in enumerate(bytes): 2023-09-23T03:27:21.528 DEBUG:teuthology.orchestra.run.smithi154:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-09-23T03:27:21.528 DEBUG:teuthology.orchestra.run.smithi154:> if b != chr(val): 2023-09-23T03:27:21.529 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-09-23T03:27:21.529 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:27:21.751 INFO:teuthology.orchestra.run.smithi154.stderr:Traceback (most recent call last): 2023-09-23T03:27:21.751 INFO:teuthology.orchestra.run.smithi154.stderr: File "", line 13, in 2023-09-23T03:27:21.751 INFO:teuthology.orchestra.run.smithi154.stderr:RuntimeError: Bad data at offset 65536 2023-09-23T03:27:21.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:21 stdin-killer INFO: command exited with status 1: exiting normally with same code! 2023-09-23T03:27:21.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T03:27:21.768 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-09-23T03:27:21.768 DEBUG:teuthology.orchestra.run.smithi154:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-23T03:27:21.928 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-23T03:27:22.053 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-09-23T03:27:22.053 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:27:22.053 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:27:22.113 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:27:22.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.331+0000 7f734655e640 1 Processor -- start 2023-09-23T03:27:22.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.332+0000 7f734655e640 1 -- start start 2023-09-23T03:27:22.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.333+0000 7f734655e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73400b1b20 0x7f73400ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:22.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.333+0000 7f734655e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f73400cee70 0x7f73400ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:22.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.333+0000 7f734655e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f734015ac30 con 0x7f73400acce0 2023-09-23T03:27:22.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.333+0000 7f734655e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73400b2e90 con 0x7f73400cee70 2023-09-23T03:27:22.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.333+0000 7f734655e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73400cf320 con 0x7f73400b1b20 2023-09-23T03:27:22.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.333+0000 7f733ffff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f73400acce0 0x7f734015ed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:60786/0 (socket says 172.21.15.154:60786) 2023-09-23T03:27:22.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.333+0000 7f733ffff640 1 -- 172.21.15.154:0/1932832478 learned_addr learned my addr 172.21.15.154:0/1932832478 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:27:22.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.334+0000 7f7344afb640 1 --2- 172.21.15.154:0/1932832478 >> v2:172.21.15.190:3300/0 conn(0x7f73400cee70 0x7f73400ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:22.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.334+0000 7f7337fff640 1 --2- 172.21.15.154:0/1932832478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73400b1b20 0x7f73400ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:22.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.334+0000 7f733f7fe640 1 -- 172.21.15.154:0/1932832478 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2361211116 0 0) 0x7f7330001290 con 0x7f73400acce0 2023-09-23T03:27:22.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.334+0000 7f733f7fe640 1 -- 172.21.15.154:0/1932832478 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f732c003520 con 0x7f73400acce0 2023-09-23T03:27:22.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.334+0000 7f733f7fe640 1 -- 172.21.15.154:0/1932832478 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1280837760 0 0) 0x7f732c003520 con 0x7f73400acce0 2023-09-23T03:27:22.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.334+0000 7f733f7fe640 1 -- 172.21.15.154:0/1932832478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73400b1b20 msgr2=0x7f73400ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:22.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.334+0000 7f733f7fe640 1 --2- 172.21.15.154:0/1932832478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73400b1b20 0x7f73400ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:22.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.334+0000 7f733f7fe640 1 -- 172.21.15.154:0/1932832478 >> v2:172.21.15.190:3300/0 conn(0x7f73400cee70 msgr2=0x7f73400ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:22.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.334+0000 7f733f7fe640 1 --2- 172.21.15.154:0/1932832478 >> v2:172.21.15.190:3300/0 conn(0x7f73400cee70 0x7f73400ab140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:22.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.334+0000 7f733f7fe640 1 -- 172.21.15.154:0/1932832478 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73400b1f70 con 0x7f73400acce0 2023-09-23T03:27:22.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.334+0000 7f733f7fe640 1 -- 172.21.15.154:0/1932832478 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f7330001980 con 0x7f73400acce0 2023-09-23T03:27:22.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.334+0000 7f733f7fe640 1 -- 172.21.15.154:0/1932832478 >> v1:172.21.15.154:6789/0 conn(0x7f73400acce0 legacy=0x7f734015ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:22.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.334+0000 7f733f7fe640 1 --2- 172.21.15.154:0/1932832478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f732c003970 0x7f732c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:22.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.334+0000 7f733f7fe640 1 --2- 172.21.15.154:0/1932832478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73400b1b20 0x7f732c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:22.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.334+0000 7f733f7fe640 1 --2- 172.21.15.154:0/1932832478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f732c004d00 0x7f732c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:22.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.334+0000 7f733f7fe640 1 -- 172.21.15.154:0/1932832478 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73400b2e90 con 0x7f732c003970 2023-09-23T03:27:22.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.334+0000 7f733f7fe640 1 -- 172.21.15.154:0/1932832478 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73400cf320 con 0x7f732c004d00 2023-09-23T03:27:22.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.334+0000 7f733f7fe640 1 -- 172.21.15.154:0/1932832478 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f732c00b980 con 0x7f73400b1b20 2023-09-23T03:27:22.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.335+0000 7f733ffff640 1 --2- 172.21.15.154:0/1932832478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f732c003970 0x7f732c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:22.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.335+0000 7f7344afb640 1 --2- 172.21.15.154:0/1932832478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f732c004d00 0x7f732c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:22.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.335+0000 7f7337fff640 1 --2- 172.21.15.154:0/1932832478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73400b1b20 0x7f732c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:22.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.335+0000 7f733ffff640 1 -- 172.21.15.154:0/1932832478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73400b1b20 msgr2=0x7f732c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:22.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.335+0000 7f733ffff640 1 --2- 172.21.15.154:0/1932832478 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73400b1b20 0x7f732c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:22.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.335+0000 7f733ffff640 1 -- 172.21.15.154:0/1932832478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f732c004d00 msgr2=0x7f732c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:22.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.335+0000 7f733ffff640 1 --2- 172.21.15.154:0/1932832478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f732c004d00 0x7f732c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:22.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.335+0000 7f733ffff640 1 -- 172.21.15.154:0/1932832478 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f732c003520 con 0x7f732c003970 2023-09-23T03:27:22.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.335+0000 7f7344afb640 1 --2- 172.21.15.154:0/1932832478 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f732c004d00 0x7f732c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:22.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.336+0000 7f733ffff640 1 --2- 172.21.15.154:0/1932832478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f732c003970 0x7f732c003d90 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f733000c210 tx=0x7f732c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d3191400dc3ad5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:22.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.337+0000 7f733f7fe640 1 -- 172.21.15.154:0/1932832478 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733002cad0 con 0x7f732c003970 2023-09-23T03:27:22.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.337+0000 7f733f7fe640 1 -- 172.21.15.154:0/1932832478 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f733002cc50 con 0x7f732c003970 2023-09-23T03:27:22.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.337+0000 7f733f7fe640 1 -- 172.21.15.154:0/1932832478 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7330007070 con 0x7f732c003970 2023-09-23T03:27:22.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.337+0000 7f734655e640 1 -- 172.21.15.154:0/1932832478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f732c003970 msgr2=0x7f732c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:22.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.337+0000 7f734655e640 1 --2- 172.21.15.154:0/1932832478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f732c003970 0x7f732c003d90 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f733000c210 tx=0x7f732c0057f0 comp rx=0 tx=0).stop 2023-09-23T03:27:22.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.337+0000 7f7344afb640 1 -- 172.21.15.154:0/1932832478 reap_dead start 2023-09-23T03:27:22.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.337+0000 7f734655e640 1 -- 172.21.15.154:0/1932832478 shutdown_connections 2023-09-23T03:27:22.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.337+0000 7f734655e640 1 -- 172.21.15.154:0/1932832478 >> 172.21.15.154:0/1932832478 conn(0x7f734006a0f0 msgr2=0x7f734006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:22.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.337+0000 7f734655e640 1 -- 172.21.15.154:0/1932832478 shutdown_connections 2023-09-23T03:27:22.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.337+0000 7f734655e640 1 -- 172.21.15.154:0/1932832478 wait complete. 2023-09-23T03:27:22.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.339+0000 7f734655e640 1 Processor -- start 2023-09-23T03:27:22.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.339+0000 7f734655e640 1 -- start start 2023-09-23T03:27:22.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.340+0000 7f734655e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f732c003970 0x7f73400ab650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:22.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.340+0000 7f734655e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f734015c790 0x7f73401f5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:22.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.340+0000 7f734655e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73400c5dd0 0x7f73400c6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:22.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.340+0000 7f734655e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734015d9c0 con 0x7f734015c790 2023-09-23T03:27:22.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.340+0000 7f734655e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73400c6790 con 0x7f732c003970 2023-09-23T03:27:22.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.340+0000 7f734655e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73400abb90 con 0x7f73400c5dd0 2023-09-23T03:27:22.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.341+0000 7f7337fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f734015c790 0x7f73401f5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:22.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.341+0000 7f7337fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f734015c790 0x7f73401f5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46560/0 (socket says 172.21.15.154:46560) 2023-09-23T03:27:22.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.341+0000 7f7337fff640 1 -- 172.21.15.154:0/3977432851 learned_addr learned my addr 172.21.15.154:0/3977432851 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:22.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.341+0000 7f733ffff640 1 --2- 172.21.15.154:0/3977432851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f732c003970 0x7f73400ab650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:22.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.341+0000 7f7337fff640 1 -- 172.21.15.154:0/3977432851 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73400c5dd0 msgr2=0x7f73400c6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:22.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.341+0000 7f7337fff640 1 --2- 172.21.15.154:0/3977432851 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73400c5dd0 0x7f73400c6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:22.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.341+0000 7f7337fff640 1 -- 172.21.15.154:0/3977432851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f732c003970 msgr2=0x7f73400ab650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:22.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.341+0000 7f7337fff640 1 --2- 172.21.15.154:0/3977432851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f732c003970 0x7f73400ab650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:22.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.341+0000 7f7337fff640 1 -- 172.21.15.154:0/3977432851 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f734015cd40 con 0x7f734015c790 2023-09-23T03:27:22.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.341+0000 7f7337fff640 1 --2- 172.21.15.154:0/3977432851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f734015c790 0x7f73401f5480 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f7328010890 tx=0x7f7328010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:22.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.342+0000 7f733d7fa640 1 -- 172.21.15.154:0/3977432851 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73280095b0 con 0x7f734015c790 2023-09-23T03:27:22.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.342+0000 7f733d7fa640 1 -- 172.21.15.154:0/3977432851 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732800eb00 con 0x7f734015c790 2023-09-23T03:27:22.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.342+0000 7f734655e640 1 -- 172.21.15.154:0/3977432851 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f734015d020 con 0x7f734015c790 2023-09-23T03:27:22.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.342+0000 7f733d7fa640 1 -- 172.21.15.154:0/3977432851 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732800eeb0 con 0x7f734015c790 2023-09-23T03:27:22.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.342+0000 7f734655e640 1 -- 172.21.15.154:0/3977432851 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f734015d5e0 con 0x7f734015c790 2023-09-23T03:27:22.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.343+0000 7f734655e640 1 -- 172.21.15.154:0/3977432851 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7340073eb0 con 0x7f734015c790 2023-09-23T03:27:22.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.344+0000 7f733d7fa640 1 -- 172.21.15.154:0/3977432851 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f73280253f0 con 0x7f734015c790 2023-09-23T03:27:22.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.345+0000 7f733d7fa640 1 --2- 172.21.15.154:0/3977432851 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f732006af30 0x7f732006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:22.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.345+0000 7f733d7fa640 1 -- 172.21.15.154:0/3977432851 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f732808a0a0 con 0x7f734015c790 2023-09-23T03:27:22.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.345+0000 7f733ffff640 1 --2- 172.21.15.154:0/3977432851 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f732006af30 0x7f732006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:22.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.346+0000 7f733ffff640 1 --2- 172.21.15.154:0/3977432851 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f732006af30 0x7f732006d3f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7330034000 tx=0x7f7330007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:22.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.350+0000 7f733d7fa640 1 -- 172.21.15.154:0/3977432851 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f7328054cf0 con 0x7f734015c790 2023-09-23T03:27:22.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.528+0000 7f734655e640 1 -- 172.21.15.154:0/3977432851 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f73400abd00 con 0x7f734015c790 2023-09-23T03:27:22.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.532+0000 7f733d7fa640 1 -- 172.21.15.154:0/3977432851 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+18945 (secure 0 0 0) 0x7f73400abd00 con 0x7f734015c790 2023-09-23T03:27:22.534 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:22.534 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":114,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:26:39.641972+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:25:44.763842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-09-23T03:25:46.025926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data_extra","create_time":"2023-09-23T03:25:56.961772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6837/3181099748":"2023-09-24T03:26:34.396308+0000","172.21.15.154:6835/3181099748":"2023-09-24T03:26:34.396308+0000","172.21.15.154:6835/4233181417":"2023-09-24T03:26:27.714769+0000","172.21.15.154:6837/4233181417":"2023-09-24T03:26:27.714769+0000","172.21.15.190:6835/2967125458":"2023-09-24T03:26:25.688784+0000","172.21.15.190:6833/2967125458":"2023-09-24T03:26:25.688784+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:27:22.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.534+0000 7f734655e640 1 -- 172.21.15.154:0/3977432851 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f732006af30 msgr2=0x7f732006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:22.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.534+0000 7f734655e640 1 --2- 172.21.15.154:0/3977432851 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f732006af30 0x7f732006d3f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7330034000 tx=0x7f7330007000 comp rx=0 tx=0).stop 2023-09-23T03:27:22.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.534+0000 7f734655e640 1 -- 172.21.15.154:0/3977432851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f734015c790 msgr2=0x7f73401f5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:22.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.534+0000 7f734655e640 1 --2- 172.21.15.154:0/3977432851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f734015c790 0x7f73401f5480 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f7328010890 tx=0x7f7328010d50 comp rx=0 tx=0).stop 2023-09-23T03:27:22.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.534+0000 7f734655e640 1 -- 172.21.15.154:0/3977432851 shutdown_connections 2023-09-23T03:27:22.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.534+0000 7f734655e640 1 --2- 172.21.15.154:0/3977432851 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f732006af30 0x7f732006d3f0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:22.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.534+0000 7f734655e640 1 --2- 172.21.15.154:0/3977432851 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f73400c5dd0 0x7f73400c6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:22.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.534+0000 7f734655e640 1 --2- 172.21.15.154:0/3977432851 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f734015c790 0x7f73401f5480 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:22.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.534+0000 7f734655e640 1 --2- 172.21.15.154:0/3977432851 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f732c003970 0x7f73400ab650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:22.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.535+0000 7f734655e640 1 -- 172.21.15.154:0/3977432851 >> 172.21.15.154:0/3977432851 conn(0x7f734006a0f0 msgr2=0x7f73400affe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:22.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.535+0000 7f734655e640 1 -- 172.21.15.154:0/3977432851 shutdown_connections 2023-09-23T03:27:22.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.535+0000 7f734655e640 1 -- 172.21.15.154:0/3977432851 wait complete. 2023-09-23T03:27:22.555 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:25:44.763842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:25:46.025926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-09-23T03:25:56.961772+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T03:27:22.556 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:27:22.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.770+0000 7fb9898ff640 1 Processor -- start 2023-09-23T03:27:22.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.770+0000 7fb9898ff640 1 -- start start 2023-09-23T03:27:22.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.772+0000 7fb9898ff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb9840b1530 0x7fb98406acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:22.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.772+0000 7fb9898ff640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9840cdd90 con 0x7fb9840d2eb0 2023-09-23T03:27:22.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.772+0000 7fb9898ff640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9840cd460 con 0x7fb98415c9b0 2023-09-23T03:27:22.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.772+0000 7fb9898ff640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9840cd740 con 0x7fb9840b1530 2023-09-23T03:27:22.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.772+0000 7fb9890fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fb9840d2eb0 0x7fb9840ce220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:60788/0 (socket says 172.21.15.154:60788) 2023-09-23T03:27:22.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.772+0000 7fb9890fe640 1 -- 172.21.15.154:0/3619948010 learned_addr learned my addr 172.21.15.154:0/3619948010 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:27:22.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.773+0000 7fb9837fe640 1 -- 172.21.15.154:0/3619948010 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2596706512 0 0) 0x7fb9840cdd90 con 0x7fb9840d2eb0 2023-09-23T03:27:22.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.773+0000 7fb983fff640 1 --2- 172.21.15.154:0/3619948010 >> v2:172.21.15.190:3300/0 conn(0x7fb9840b1530 0x7fb98406acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:22.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.773+0000 7fb9837fe640 1 -- 172.21.15.154:0/3619948010 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb974003520 con 0x7fb9840d2eb0 2023-09-23T03:27:22.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.773+0000 7fb9837fe640 1 -- 172.21.15.154:0/3619948010 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 316696713 0 0) 0x7fb974003520 con 0x7fb9840d2eb0 2023-09-23T03:27:22.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.773+0000 7fb9837fe640 1 -- 172.21.15.154:0/3619948010 >> v2:172.21.15.190:3300/0 conn(0x7fb9840b1530 msgr2=0x7fb98406acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:22.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.773+0000 7fb9837fe640 1 --2- 172.21.15.154:0/3619948010 >> v2:172.21.15.190:3300/0 conn(0x7fb9840b1530 0x7fb98406acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:22.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.773+0000 7fb9837fe640 1 -- 172.21.15.154:0/3619948010 >> v1:172.21.15.190:6789/0 conn(0x7fb98415c9b0 legacy=0x7fb9840b41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:22.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.773+0000 7fb9837fe640 1 -- 172.21.15.154:0/3619948010 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9840cda20 con 0x7fb9840d2eb0 2023-09-23T03:27:22.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.773+0000 7fb9837fe640 1 -- 172.21.15.154:0/3619948010 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fb97c0019f0 con 0x7fb9840d2eb0 2023-09-23T03:27:22.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.773+0000 7fb9837fe640 1 -- 172.21.15.154:0/3619948010 >> v1:172.21.15.154:6789/0 conn(0x7fb9840d2eb0 legacy=0x7fb9840ce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:22.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.773+0000 7fb9837fe640 1 --2- 172.21.15.154:0/3619948010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9740039c0 0x7fb974003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:22.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.773+0000 7fb9837fe640 1 --2- 172.21.15.154:0/3619948010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9740047c0 0x7fb974004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:22.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.773+0000 7fb9837fe640 1 --2- 172.21.15.154:0/3619948010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9740051d0 0x7fb97400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:22.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.773+0000 7fb9837fe640 1 -- 172.21.15.154:0/3619948010 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9840cd740 con 0x7fb9740047c0 2023-09-23T03:27:22.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.774+0000 7fb9837fe640 1 -- 172.21.15.154:0/3619948010 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97400c120 con 0x7fb9740051d0 2023-09-23T03:27:22.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.774+0000 7fb9837fe640 1 -- 172.21.15.154:0/3619948010 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb97400c2e0 con 0x7fb9740039c0 2023-09-23T03:27:22.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.774+0000 7fb983fff640 1 --2- 172.21.15.154:0/3619948010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9740047c0 0x7fb974004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:22.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.774+0000 7fb983fff640 1 -- 172.21.15.154:0/3619948010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9740039c0 msgr2=0x7fb974003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:22.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.774+0000 7fb983fff640 1 --2- 172.21.15.154:0/3619948010 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9740039c0 0x7fb974003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:22.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.774+0000 7fb983fff640 1 -- 172.21.15.154:0/3619948010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9740051d0 msgr2=0x7fb97400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:22.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.774+0000 7fb9890fe640 1 --2- 172.21.15.154:0/3619948010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9740051d0 0x7fb97400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:22.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.774+0000 7fb983fff640 1 --2- 172.21.15.154:0/3619948010 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9740051d0 0x7fb97400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:22.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.774+0000 7fb983fff640 1 -- 172.21.15.154:0/3619948010 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb974003520 con 0x7fb9740047c0 2023-09-23T03:27:22.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.774+0000 7fb9890fe640 1 -- 172.21.15.154:0/3619948010 reap_dead start 2023-09-23T03:27:22.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.775+0000 7fb983fff640 1 --2- 172.21.15.154:0/3619948010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9740047c0 0x7fb974004c40 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fb97000ba10 tx=0x7fb9740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5beb9c88a336bf7d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:22.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.776+0000 7fb9837fe640 1 -- 172.21.15.154:0/3619948010 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb970032630 con 0x7fb9740047c0 2023-09-23T03:27:22.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.776+0000 7fb9837fe640 1 -- 172.21.15.154:0/3619948010 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9700327b0 con 0x7fb9740047c0 2023-09-23T03:27:22.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.776+0000 7fb9837fe640 1 -- 172.21.15.154:0/3619948010 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb970032b60 con 0x7fb9740047c0 2023-09-23T03:27:22.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.776+0000 7fb9898ff640 1 -- 172.21.15.154:0/3619948010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9740047c0 msgr2=0x7fb974004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:22.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.776+0000 7fb9898ff640 1 --2- 172.21.15.154:0/3619948010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9740047c0 0x7fb974004c40 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fb97000ba10 tx=0x7fb9740057f0 comp rx=0 tx=0).stop 2023-09-23T03:27:22.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.776+0000 7fb9898ff640 1 -- 172.21.15.154:0/3619948010 shutdown_connections 2023-09-23T03:27:22.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.776+0000 7fb9898ff640 1 --2- 172.21.15.154:0/3619948010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9740047c0 0x7fb974004c40 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:22.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.776+0000 7fb9898ff640 1 -- 172.21.15.154:0/3619948010 >> 172.21.15.154:0/3619948010 conn(0x7fb98409ba70 msgr2=0x7fb9840d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:22.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.776+0000 7fb9898ff640 1 -- 172.21.15.154:0/3619948010 shutdown_connections 2023-09-23T03:27:22.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.776+0000 7fb9898ff640 1 -- 172.21.15.154:0/3619948010 wait complete. 2023-09-23T03:27:22.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.778+0000 7fb9898ff640 1 Processor -- start 2023-09-23T03:27:22.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.778+0000 7fb9898ff640 1 -- start start 2023-09-23T03:27:22.782 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.779+0000 7fb9898ff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9740047c0 0x7fb9841f3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:22.782 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.779+0000 7fb9898ff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9841f5930 0x7fb9841f5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:22.782 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.779+0000 7fb9898ff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9840cbd70 0x7fb9840cc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:22.782 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.779+0000 7fb9898ff640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9841f62f0 con 0x7fb9840cbd70 2023-09-23T03:27:22.782 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.779+0000 7fb9898ff640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98415eb50 con 0x7fb9841f5930 2023-09-23T03:27:22.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.779+0000 7fb9898ff640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9840cca10 con 0x7fb9740047c0 2023-09-23T03:27:22.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.779+0000 7fb9890fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9840cbd70 0x7fb9840cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:22.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.780+0000 7fb9890fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9840cbd70 0x7fb9840cc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46568/0 (socket says 172.21.15.154:46568) 2023-09-23T03:27:22.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.780+0000 7fb9890fe640 1 -- 172.21.15.154:0/2964551324 learned_addr learned my addr 172.21.15.154:0/2964551324 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:22.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.780+0000 7fb9888fd640 1 --2- 172.21.15.154:0/2964551324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9740047c0 0x7fb9841f3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:22.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.780+0000 7fb983fff640 1 --2- 172.21.15.154:0/2964551324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9841f5930 0x7fb9841f5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:22.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.780+0000 7fb9890fe640 1 -- 172.21.15.154:0/2964551324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9740047c0 msgr2=0x7fb9841f3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:22.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.780+0000 7fb9890fe640 1 --2- 172.21.15.154:0/2964551324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9740047c0 0x7fb9841f3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:22.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.780+0000 7fb9890fe640 1 -- 172.21.15.154:0/2964551324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9841f5930 msgr2=0x7fb9841f5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:22.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.780+0000 7fb9890fe640 1 --2- 172.21.15.154:0/2964551324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9841f5930 0x7fb9841f5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:22.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.780+0000 7fb9890fe640 1 -- 172.21.15.154:0/2964551324 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb98415cf10 con 0x7fb9840cbd70 2023-09-23T03:27:22.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.780+0000 7fb9888fd640 1 --2- 172.21.15.154:0/2964551324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9740047c0 0x7fb9841f3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:22.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.780+0000 7fb983fff640 1 --2- 172.21.15.154:0/2964551324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9841f5930 0x7fb9841f5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:22.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.780+0000 7fb9890fe640 1 --2- 172.21.15.154:0/2964551324 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9840cbd70 0x7fb9840cc220 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fb98408bba0 tx=0x7fb97c00df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:22.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.780+0000 7fb9817fa640 1 -- 172.21.15.154:0/2964551324 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97c00b7a0 con 0x7fb9840cbd70 2023-09-23T03:27:22.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.781+0000 7fb9817fa640 1 -- 172.21.15.154:0/2964551324 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb97c00b900 con 0x7fb9840cbd70 2023-09-23T03:27:22.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.781+0000 7fb9817fa640 1 -- 172.21.15.154:0/2964551324 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97c00bcb0 con 0x7fb9840cbd70 2023-09-23T03:27:22.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.781+0000 7fb9898ff640 1 -- 172.21.15.154:0/2964551324 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9840cc760 con 0x7fb9840cbd70 2023-09-23T03:27:22.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.781+0000 7fb9898ff640 1 -- 172.21.15.154:0/2964551324 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb98415e220 con 0x7fb9840cbd70 2023-09-23T03:27:22.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.782+0000 7fb9898ff640 1 -- 172.21.15.154:0/2964551324 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb984067390 con 0x7fb9840cbd70 2023-09-23T03:27:22.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.783+0000 7fb9817fa640 1 -- 172.21.15.154:0/2964551324 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fb97c007110 con 0x7fb9840cbd70 2023-09-23T03:27:22.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.783+0000 7fb9817fa640 1 --2- 172.21.15.154:0/2964551324 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb96806af60 0x7fb96806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:22.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.784+0000 7fb9817fa640 1 -- 172.21.15.154:0/2964551324 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fb97c089d60 con 0x7fb9840cbd70 2023-09-23T03:27:22.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.784+0000 7fb9888fd640 1 --2- 172.21.15.154:0/2964551324 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb96806af60 0x7fb96806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:22.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.784+0000 7fb9888fd640 1 --2- 172.21.15.154:0/2964551324 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb96806af60 0x7fb96806d420 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fb98415fd20 tx=0x7fb978009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:22.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.788+0000 7fb9817fa640 1 -- 172.21.15.154:0/2964551324 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fb97c0584e0 con 0x7fb9840cbd70 2023-09-23T03:27:22.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.976+0000 7fb9898ff640 1 -- 172.21.15.154:0/2964551324 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb98415ec90 con 0x7fb9840cbd70 2023-09-23T03:27:22.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.977+0000 7fb9817fa640 1 -- 172.21.15.154:0/2964551324 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v114) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb97c058070 con 0x7fb9840cbd70 2023-09-23T03:27:22.978 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:27:22.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.980+0000 7fb9898ff640 1 -- 172.21.15.154:0/2964551324 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb96806af60 msgr2=0x7fb96806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:22.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.980+0000 7fb9898ff640 1 --2- 172.21.15.154:0/2964551324 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb96806af60 0x7fb96806d420 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fb98415fd20 tx=0x7fb978009040 comp rx=0 tx=0).stop 2023-09-23T03:27:22.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.980+0000 7fb9898ff640 1 -- 172.21.15.154:0/2964551324 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9840cbd70 msgr2=0x7fb9840cc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:22.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.980+0000 7fb9898ff640 1 --2- 172.21.15.154:0/2964551324 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9840cbd70 0x7fb9840cc220 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fb98408bba0 tx=0x7fb97c00df70 comp rx=0 tx=0).stop 2023-09-23T03:27:22.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.980+0000 7fb9898ff640 1 -- 172.21.15.154:0/2964551324 shutdown_connections 2023-09-23T03:27:22.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.980+0000 7fb9898ff640 1 --2- 172.21.15.154:0/2964551324 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb96806af60 0x7fb96806d420 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:22.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.980+0000 7fb9898ff640 1 --2- 172.21.15.154:0/2964551324 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9840cbd70 0x7fb9840cc220 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:22.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.980+0000 7fb9898ff640 1 --2- 172.21.15.154:0/2964551324 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9841f5930 0x7fb9841f5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:22.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.980+0000 7fb9898ff640 1 --2- 172.21.15.154:0/2964551324 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9740047c0 0x7fb9841f3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:22.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.980+0000 7fb9898ff640 1 -- 172.21.15.154:0/2964551324 >> 172.21.15.154:0/2964551324 conn(0x7fb98409ba70 msgr2=0x7fb98406b970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:22.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.980+0000 7fb9898ff640 1 -- 172.21.15.154:0/2964551324 shutdown_connections 2023-09-23T03:27:22.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:22.980+0000 7fb9898ff640 1 -- 172.21.15.154:0/2964551324 wait complete. 2023-09-23T03:27:23.001 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:27:23.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.214+0000 7fc4b73cc640 1 Processor -- start 2023-09-23T03:27:23.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.214+0000 7fc4b73cc640 1 -- start start 2023-09-23T03:27:23.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.216+0000 7fc4b73cc640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc4b00c8d70 0x7fc4b015c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:23.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.216+0000 7fc4b73cc640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4b00c9d20 con 0x7fc4b0071290 2023-09-23T03:27:23.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.216+0000 7fc4b73cc640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4b0071850 con 0x7fc4b00c9150 2023-09-23T03:27:23.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.216+0000 7fc4b73cc640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4b0071b30 con 0x7fc4b00c8d70 2023-09-23T03:27:23.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.216+0000 7fc4b5969640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fc4b0071290 0x7fc4b00ca1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:60800/0 (socket says 172.21.15.154:60800) 2023-09-23T03:27:23.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.216+0000 7fc4b5969640 1 -- 172.21.15.154:0/402123702 learned_addr learned my addr 172.21.15.154:0/402123702 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:27:23.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.217+0000 7fc4b5168640 1 --2- 172.21.15.154:0/402123702 >> v2:172.21.15.190:3300/0 conn(0x7fc4b00c8d70 0x7fc4b015c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:23.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.217+0000 7fc4a7fff640 1 -- 172.21.15.154:0/402123702 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3390026816 0 0) 0x7fc4b00c9d20 con 0x7fc4b0071290 2023-09-23T03:27:23.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.217+0000 7fc4a7fff640 1 -- 172.21.15.154:0/402123702 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc49c003520 con 0x7fc4b0071290 2023-09-23T03:27:23.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.217+0000 7fc4a7fff640 1 -- 172.21.15.154:0/402123702 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3258533945 0 0) 0x7fc49c003520 con 0x7fc4b0071290 2023-09-23T03:27:23.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.217+0000 7fc4a7fff640 1 -- 172.21.15.154:0/402123702 >> v2:172.21.15.190:3300/0 conn(0x7fc4b00c8d70 msgr2=0x7fc4b015c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:23.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.217+0000 7fc4a7fff640 1 --2- 172.21.15.154:0/402123702 >> v2:172.21.15.190:3300/0 conn(0x7fc4b00c8d70 0x7fc4b015c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:23.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.217+0000 7fc4a7fff640 1 -- 172.21.15.154:0/402123702 >> v1:172.21.15.190:6789/0 conn(0x7fc4b00c9150 legacy=0x7fc4b015cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:23.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.217+0000 7fc4a7fff640 1 -- 172.21.15.154:0/402123702 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b0071e10 con 0x7fc4b0071290 2023-09-23T03:27:23.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.217+0000 7fc4a7fff640 1 -- 172.21.15.154:0/402123702 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fc4a80019f0 con 0x7fc4b0071290 2023-09-23T03:27:23.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.217+0000 7fc4a7fff640 1 -- 172.21.15.154:0/402123702 >> v1:172.21.15.154:6789/0 conn(0x7fc4b0071290 legacy=0x7fc4b00ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:23.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.217+0000 7fc4a7fff640 1 --2- 172.21.15.154:0/402123702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc49c0039c0 0x7fc49c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:23.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.217+0000 7fc4a7fff640 1 --2- 172.21.15.154:0/402123702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc49c0047c0 0x7fc49c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:23.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.217+0000 7fc4a7fff640 1 --2- 172.21.15.154:0/402123702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc49c005180 0x7fc49c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:23.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.217+0000 7fc4a7fff640 1 -- 172.21.15.154:0/402123702 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b0071b30 con 0x7fc49c0039c0 2023-09-23T03:27:23.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.217+0000 7fc4a7fff640 1 -- 172.21.15.154:0/402123702 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c00c080 con 0x7fc49c005180 2023-09-23T03:27:23.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.217+0000 7fc4a7fff640 1 -- 172.21.15.154:0/402123702 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c00c240 con 0x7fc49c0047c0 2023-09-23T03:27:23.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.218+0000 7fc4b5168640 1 --2- 172.21.15.154:0/402123702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc49c0039c0 0x7fc49c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:23.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.218+0000 7fc4b5168640 1 -- 172.21.15.154:0/402123702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc49c0047c0 msgr2=0x7fc49c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:23.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.218+0000 7fc4b5168640 1 --2- 172.21.15.154:0/402123702 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc49c0047c0 0x7fc49c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:23.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.218+0000 7fc4b5168640 1 -- 172.21.15.154:0/402123702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc49c005180 msgr2=0x7fc49c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:23.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.218+0000 7fc4b5969640 1 --2- 172.21.15.154:0/402123702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc49c005180 0x7fc49c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:23.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.218+0000 7fc4b5168640 1 --2- 172.21.15.154:0/402123702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc49c005180 0x7fc49c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:23.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.218+0000 7fc4b5168640 1 -- 172.21.15.154:0/402123702 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49c003520 con 0x7fc49c0039c0 2023-09-23T03:27:23.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.218+0000 7fc4b5969640 1 --2- 172.21.15.154:0/402123702 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc49c005180 0x7fc49c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:23.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.218+0000 7fc4b5969640 1 -- 172.21.15.154:0/402123702 reap_dead start 2023-09-23T03:27:23.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.219+0000 7fc4b5168640 1 --2- 172.21.15.154:0/402123702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc49c0039c0 0x7fc49c003de0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac009ac0 tx=0x7fc49c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=199c0949bfdbe950 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:23.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.219+0000 7fc4a7fff640 1 -- 172.21.15.154:0/402123702 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac02c6b0 con 0x7fc49c0039c0 2023-09-23T03:27:23.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.219+0000 7fc4a7fff640 1 -- 172.21.15.154:0/402123702 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4ac02c810 con 0x7fc49c0039c0 2023-09-23T03:27:23.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.219+0000 7fc4a7fff640 1 -- 172.21.15.154:0/402123702 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac02cba0 con 0x7fc49c0039c0 2023-09-23T03:27:23.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.220+0000 7fc4b73cc640 1 -- 172.21.15.154:0/402123702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc49c0039c0 msgr2=0x7fc49c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:23.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.220+0000 7fc4b73cc640 1 --2- 172.21.15.154:0/402123702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc49c0039c0 0x7fc49c003de0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac009ac0 tx=0x7fc49c0057f0 comp rx=0 tx=0).stop 2023-09-23T03:27:23.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.220+0000 7fc4b73cc640 1 -- 172.21.15.154:0/402123702 shutdown_connections 2023-09-23T03:27:23.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.220+0000 7fc4b73cc640 1 --2- 172.21.15.154:0/402123702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc49c0039c0 0x7fc49c003de0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:23.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.220+0000 7fc4b73cc640 1 -- 172.21.15.154:0/402123702 >> 172.21.15.154:0/402123702 conn(0x7fc4b006c010 msgr2=0x7fc4b006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:23.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.220+0000 7fc4b73cc640 1 -- 172.21.15.154:0/402123702 shutdown_connections 2023-09-23T03:27:23.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.220+0000 7fc4b73cc640 1 -- 172.21.15.154:0/402123702 wait complete. 2023-09-23T03:27:23.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.222+0000 7fc4b73cc640 1 Processor -- start 2023-09-23T03:27:23.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.222+0000 7fc4b73cc640 1 -- start start 2023-09-23T03:27:23.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.223+0000 7fc4b73cc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc49c0039c0 0x7fc4b00c7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:23.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.223+0000 7fc4b73cc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4b0153cf0 0x7fc4b00c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:23.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.223+0000 7fc4b73cc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4b0153100 0x7fc4b00c3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:23.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.223+0000 7fc4b73cc640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b00b1bf0 con 0x7fc4b0153cf0 2023-09-23T03:27:23.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.223+0000 7fc4b73cc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b00b1ed0 con 0x7fc4b0153100 2023-09-23T03:27:23.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.223+0000 7fc4b73cc640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b00b21b0 con 0x7fc49c0039c0 2023-09-23T03:27:23.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.223+0000 7fc4b4967640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4b0153cf0 0x7fc4b00c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:23.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.223+0000 7fc4b4967640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4b0153cf0 0x7fc4b00c7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46574/0 (socket says 172.21.15.154:46574) 2023-09-23T03:27:23.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.223+0000 7fc4b4967640 1 -- 172.21.15.154:0/2748193137 learned_addr learned my addr 172.21.15.154:0/2748193137 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:23.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.224+0000 7fc4b5969640 1 --2- 172.21.15.154:0/2748193137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4b0153100 0x7fc4b00c3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:23.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.224+0000 7fc4b4967640 1 -- 172.21.15.154:0/2748193137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc49c0039c0 msgr2=0x7fc4b00c7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:23.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.224+0000 7fc4b5168640 1 --2- 172.21.15.154:0/2748193137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc49c0039c0 0x7fc4b00c7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:23.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.224+0000 7fc4b4967640 1 --2- 172.21.15.154:0/2748193137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc49c0039c0 0x7fc4b00c7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:23.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.224+0000 7fc4b4967640 1 -- 172.21.15.154:0/2748193137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4b0153100 msgr2=0x7fc4b00c3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:23.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.224+0000 7fc4b4967640 1 --2- 172.21.15.154:0/2748193137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4b0153100 0x7fc4b00c3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:23.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.224+0000 7fc4b4967640 1 -- 172.21.15.154:0/2748193137 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b00b2490 con 0x7fc4b0153cf0 2023-09-23T03:27:23.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.224+0000 7fc4b4967640 1 --2- 172.21.15.154:0/2748193137 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4b0153cf0 0x7fc4b00c7ad0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fc4a000c500 tx=0x7fc4a000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:23.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.224+0000 7fc4a5ffb640 1 -- 172.21.15.154:0/2748193137 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a00052d0 con 0x7fc4b0153cf0 2023-09-23T03:27:23.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.224+0000 7fc4a5ffb640 1 -- 172.21.15.154:0/2748193137 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4a0005430 con 0x7fc4b0153cf0 2023-09-23T03:27:23.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.224+0000 7fc4a5ffb640 1 -- 172.21.15.154:0/2748193137 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0012870 con 0x7fc4b0153cf0 2023-09-23T03:27:23.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.224+0000 7fc4b73cc640 1 -- 172.21.15.154:0/2748193137 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4b00b2770 con 0x7fc4b0153cf0 2023-09-23T03:27:23.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.225+0000 7fc4b73cc640 1 -- 172.21.15.154:0/2748193137 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4b00b1810 con 0x7fc4b0153cf0 2023-09-23T03:27:23.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.226+0000 7fc4b73cc640 1 -- 172.21.15.154:0/2748193137 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4b00b2e70 con 0x7fc4b0153cf0 2023-09-23T03:27:23.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.227+0000 7fc4a5ffb640 1 -- 172.21.15.154:0/2748193137 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fc4a00129d0 con 0x7fc4b0153cf0 2023-09-23T03:27:23.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.228+0000 7fc4a5ffb640 1 --2- 172.21.15.154:0/2748193137 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc49406af60 0x7fc49406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:23.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.228+0000 7fc4a5ffb640 1 -- 172.21.15.154:0/2748193137 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fc4a0089730 con 0x7fc4b0153cf0 2023-09-23T03:27:23.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.228+0000 7fc4b5168640 1 --2- 172.21.15.154:0/2748193137 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc49406af60 0x7fc49406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:23.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.229+0000 7fc4b5168640 1 --2- 172.21.15.154:0/2748193137 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc49406af60 0x7fc49406d420 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac036000 tx=0x7fc4ac030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:23.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.232+0000 7fc4a5ffb640 1 -- 172.21.15.154:0/2748193137 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fc4a0054380 con 0x7fc4b0153cf0 2023-09-23T03:27:23.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.423+0000 7fc4b73cc640 1 -- 172.21.15.154:0/2748193137 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc4b004e920 con 0x7fc4b0153cf0 2023-09-23T03:27:23.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.424+0000 7fc4a5ffb640 1 -- 172.21.15.154:0/2748193137 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v114) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc4a00579a0 con 0x7fc4b0153cf0 2023-09-23T03:27:23.425 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:27:23.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.427+0000 7fc4b73cc640 1 -- 172.21.15.154:0/2748193137 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc49406af60 msgr2=0x7fc49406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:23.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.427+0000 7fc4b73cc640 1 --2- 172.21.15.154:0/2748193137 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc49406af60 0x7fc49406d420 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac036000 tx=0x7fc4ac030000 comp rx=0 tx=0).stop 2023-09-23T03:27:23.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.427+0000 7fc4b73cc640 1 -- 172.21.15.154:0/2748193137 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4b0153cf0 msgr2=0x7fc4b00c7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:23.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.427+0000 7fc4b73cc640 1 --2- 172.21.15.154:0/2748193137 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4b0153cf0 0x7fc4b00c7ad0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fc4a000c500 tx=0x7fc4a000c9c0 comp rx=0 tx=0).stop 2023-09-23T03:27:23.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.427+0000 7fc4b73cc640 1 -- 172.21.15.154:0/2748193137 shutdown_connections 2023-09-23T03:27:23.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.427+0000 7fc4b73cc640 1 --2- 172.21.15.154:0/2748193137 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc49406af60 0x7fc49406d420 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:23.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.427+0000 7fc4b73cc640 1 --2- 172.21.15.154:0/2748193137 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc4b0153100 0x7fc4b00c3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:23.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.427+0000 7fc4b73cc640 1 --2- 172.21.15.154:0/2748193137 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc4b0153cf0 0x7fc4b00c7ad0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:23.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.427+0000 7fc4b73cc640 1 --2- 172.21.15.154:0/2748193137 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc49c0039c0 0x7fc4b00c7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:23.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.427+0000 7fc4b73cc640 1 -- 172.21.15.154:0/2748193137 >> 172.21.15.154:0/2748193137 conn(0x7fc4b006c010 msgr2=0x7fc4b015b810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:23.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.427+0000 7fc4b73cc640 1 -- 172.21.15.154:0/2748193137 shutdown_connections 2023-09-23T03:27:23.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.428+0000 7fc4b73cc640 1 -- 172.21.15.154:0/2748193137 wait complete. 2023-09-23T03:27:23.447 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:27:23.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.657+0000 7f9f83c34640 1 Processor -- start 2023-09-23T03:27:23.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.658+0000 7f9f83c34640 1 -- start start 2023-09-23T03:27:23.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.659+0000 7f9f83c34640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f7c15c9c0 0x7f9f7c0d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:23.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.659+0000 7f9f83c34640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9f7c15cda0 0x7f9f7c0d2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:23.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.659+0000 7f9f83c34640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f7c0cd450 con 0x7f9f7c0d2b60 2023-09-23T03:27:23.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.659+0000 7f9f83c34640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f7c0d0df0 con 0x7f9f7c15cda0 2023-09-23T03:27:23.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.659+0000 7f9f83c34640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7c0d10d0 con 0x7f9f7c15c9c0 2023-09-23T03:27:23.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.660+0000 7f9f821d1640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f9f7c0d2b60 0x7f9f7c0cf2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:60802/0 (socket says 172.21.15.154:60802) 2023-09-23T03:27:23.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.660+0000 7f9f821d1640 1 -- 172.21.15.154:0/409860234 learned_addr learned my addr 172.21.15.154:0/409860234 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:27:23.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.660+0000 7f9f811cf640 1 --2- 172.21.15.154:0/409860234 >> v2:172.21.15.190:3300/0 conn(0x7f9f7c15cda0 0x7f9f7c0d2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:23.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.660+0000 7f9f819d0640 1 --2- 172.21.15.154:0/409860234 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f7c15c9c0 0x7f9f7c0d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:23.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.660+0000 7f9f809ce640 1 -- 172.21.15.154:0/409860234 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2500621523 0 0) 0x7f9f7c0cd450 con 0x7f9f7c0d2b60 2023-09-23T03:27:23.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.660+0000 7f9f809ce640 1 -- 172.21.15.154:0/409860234 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f68003520 con 0x7f9f7c0d2b60 2023-09-23T03:27:23.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.660+0000 7f9f809ce640 1 -- 172.21.15.154:0/409860234 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1101208069 0 0) 0x7f9f68003520 con 0x7f9f7c0d2b60 2023-09-23T03:27:23.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.660+0000 7f9f809ce640 1 -- 172.21.15.154:0/409860234 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f7c15c9c0 msgr2=0x7f9f7c0d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:23.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.660+0000 7f9f809ce640 1 --2- 172.21.15.154:0/409860234 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f7c15c9c0 0x7f9f7c0d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:23.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.660+0000 7f9f809ce640 1 -- 172.21.15.154:0/409860234 >> v2:172.21.15.190:3300/0 conn(0x7f9f7c15cda0 msgr2=0x7f9f7c0d2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:23.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.660+0000 7f9f809ce640 1 --2- 172.21.15.154:0/409860234 >> v2:172.21.15.190:3300/0 conn(0x7f9f7c15cda0 0x7f9f7c0d2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:23.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.660+0000 7f9f809ce640 1 -- 172.21.15.154:0/409860234 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f7c0d13b0 con 0x7f9f7c0d2b60 2023-09-23T03:27:23.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.660+0000 7f9f809ce640 1 -- 172.21.15.154:0/409860234 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f9f740019f0 con 0x7f9f7c0d2b60 2023-09-23T03:27:23.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.661+0000 7f9f809ce640 1 -- 172.21.15.154:0/409860234 >> v1:172.21.15.154:6789/0 conn(0x7f9f7c0d2b60 legacy=0x7f9f7c0cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:23.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.661+0000 7f9f809ce640 1 --2- 172.21.15.154:0/409860234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f68003970 0x7f9f68003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:23.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.661+0000 7f9f809ce640 1 --2- 172.21.15.154:0/409860234 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f680047c0 0x7f9f68004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:23.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.661+0000 7f9f809ce640 1 --2- 172.21.15.154:0/409860234 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f7c15c9c0 0x7f9f68005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:23.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.661+0000 7f9f809ce640 1 -- 172.21.15.154:0/409860234 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7c0d0df0 con 0x7f9f680047c0 2023-09-23T03:27:23.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.661+0000 7f9f809ce640 1 -- 172.21.15.154:0/409860234 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7c0d10d0 con 0x7f9f68003970 2023-09-23T03:27:23.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.661+0000 7f9f809ce640 1 -- 172.21.15.154:0/409860234 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6800b980 con 0x7f9f7c15c9c0 2023-09-23T03:27:23.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.661+0000 7f9f811cf640 1 --2- 172.21.15.154:0/409860234 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f680047c0 0x7f9f68004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:23.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.661+0000 7f9f819d0640 1 --2- 172.21.15.154:0/409860234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f68003970 0x7f9f68003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:23.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.661+0000 7f9f821d1640 1 --2- 172.21.15.154:0/409860234 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f7c15c9c0 0x7f9f68005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:23.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.661+0000 7f9f811cf640 1 -- 172.21.15.154:0/409860234 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f7c15c9c0 msgr2=0x7f9f68005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:23.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.661+0000 7f9f811cf640 1 --2- 172.21.15.154:0/409860234 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f7c15c9c0 0x7f9f68005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:23.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.661+0000 7f9f811cf640 1 -- 172.21.15.154:0/409860234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f68003970 msgr2=0x7f9f68003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:23.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.661+0000 7f9f811cf640 1 --2- 172.21.15.154:0/409860234 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f68003970 0x7f9f68003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:23.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.661+0000 7f9f811cf640 1 -- 172.21.15.154:0/409860234 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f68003520 con 0x7f9f680047c0 2023-09-23T03:27:23.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.662+0000 7f9f811cf640 1 --2- 172.21.15.154:0/409860234 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f680047c0 0x7f9f68004c40 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c009ac0 tx=0x7f9f680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beacb0a57c52911d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:23.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.663+0000 7f9f809ce640 1 -- 172.21.15.154:0/409860234 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c02c710 con 0x7f9f680047c0 2023-09-23T03:27:23.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.663+0000 7f9f809ce640 1 -- 172.21.15.154:0/409860234 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f6c02c870 con 0x7f9f680047c0 2023-09-23T03:27:23.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.663+0000 7f9f809ce640 1 -- 172.21.15.154:0/409860234 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c02cc00 con 0x7f9f680047c0 2023-09-23T03:27:23.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.663+0000 7f9f83c34640 1 -- 172.21.15.154:0/409860234 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f680047c0 msgr2=0x7f9f68004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:23.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.663+0000 7f9f83c34640 1 --2- 172.21.15.154:0/409860234 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f680047c0 0x7f9f68004c40 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c009ac0 tx=0x7f9f680057f0 comp rx=0 tx=0).stop 2023-09-23T03:27:23.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.663+0000 7f9f821d1640 1 -- 172.21.15.154:0/409860234 reap_dead start 2023-09-23T03:27:23.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.663+0000 7f9f83c34640 1 -- 172.21.15.154:0/409860234 shutdown_connections 2023-09-23T03:27:23.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.663+0000 7f9f83c34640 1 -- 172.21.15.154:0/409860234 >> 172.21.15.154:0/409860234 conn(0x7f9f7c09bc20 msgr2=0x7f9f7c089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:23.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.663+0000 7f9f83c34640 1 -- 172.21.15.154:0/409860234 shutdown_connections 2023-09-23T03:27:23.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.663+0000 7f9f83c34640 1 -- 172.21.15.154:0/409860234 wait complete. 2023-09-23T03:27:23.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.665+0000 7f9f83c34640 1 Processor -- start 2023-09-23T03:27:23.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.665+0000 7f9f83c34640 1 -- start start 2023-09-23T03:27:23.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.666+0000 7f9f83c34640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f680047c0 0x7f9f7c157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:23.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.666+0000 7f9f83c34640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f7c0c6400 0x7f9f7c0c7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:23.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.666+0000 7f9f83c34640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f7c15b470 0x7f9f7c0c76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:23.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.666+0000 7f9f83c34640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7c160150 con 0x7f9f7c0c6400 2023-09-23T03:27:23.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.666+0000 7f9f83c34640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7c1f66c0 con 0x7f9f7c15b470 2023-09-23T03:27:23.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.666+0000 7f9f83c34640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7c1f5ab0 con 0x7f9f680047c0 2023-09-23T03:27:23.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.666+0000 7f9f811cf640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f7c0c6400 0x7f9f7c0c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:23.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.666+0000 7f9f811cf640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f7c0c6400 0x7f9f7c0c7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46590/0 (socket says 172.21.15.154:46590) 2023-09-23T03:27:23.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.666+0000 7f9f811cf640 1 -- 172.21.15.154:0/274538143 learned_addr learned my addr 172.21.15.154:0/274538143 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:23.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.667+0000 7f9f811cf640 1 -- 172.21.15.154:0/274538143 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f680047c0 msgr2=0x7f9f7c157140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:23.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.667+0000 7f9f811cf640 1 --2- 172.21.15.154:0/274538143 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f680047c0 0x7f9f7c157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:23.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.667+0000 7f9f811cf640 1 -- 172.21.15.154:0/274538143 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f7c15b470 msgr2=0x7f9f7c0c76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:23.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.667+0000 7f9f811cf640 1 --2- 172.21.15.154:0/274538143 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f7c15b470 0x7f9f7c0c76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:23.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.667+0000 7f9f811cf640 1 -- 172.21.15.154:0/274538143 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f7c1f5d90 con 0x7f9f7c0c6400 2023-09-23T03:27:23.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.667+0000 7f9f811cf640 1 --2- 172.21.15.154:0/274538143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f7c0c6400 0x7f9f7c0c7190 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c02c2b0 tx=0x7f9f6c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:23.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.667+0000 7f9f727fc640 1 -- 172.21.15.154:0/274538143 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c003810 con 0x7f9f7c0c6400 2023-09-23T03:27:23.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.667+0000 7f9f727fc640 1 -- 172.21.15.154:0/274538143 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f6c02de90 con 0x7f9f7c0c6400 2023-09-23T03:27:23.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.667+0000 7f9f727fc640 1 -- 172.21.15.154:0/274538143 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c003b70 con 0x7f9f7c0c6400 2023-09-23T03:27:23.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.667+0000 7f9f83c34640 1 -- 172.21.15.154:0/274538143 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f7c1f6070 con 0x7f9f7c0c6400 2023-09-23T03:27:23.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.667+0000 7f9f83c34640 1 -- 172.21.15.154:0/274538143 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f7c157680 con 0x7f9f7c0c6400 2023-09-23T03:27:23.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.668+0000 7f9f83c34640 1 -- 172.21.15.154:0/274538143 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f7c04e920 con 0x7f9f7c0c6400 2023-09-23T03:27:23.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.669+0000 7f9f727fc640 1 -- 172.21.15.154:0/274538143 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f9f6c003cd0 con 0x7f9f7c0c6400 2023-09-23T03:27:23.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.670+0000 7f9f727fc640 1 --2- 172.21.15.154:0/274538143 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9f6006af60 0x7f9f6006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:23.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.670+0000 7f9f727fc640 1 -- 172.21.15.154:0/274538143 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f9f6c0aba60 con 0x7f9f7c0c6400 2023-09-23T03:27:23.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.670+0000 7f9f819d0640 1 --2- 172.21.15.154:0/274538143 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9f6006af60 0x7f9f6006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:23.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.671+0000 7f9f819d0640 1 --2- 172.21.15.154:0/274538143 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9f6006af60 0x7f9f6006d420 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009820 tx=0x7f9f78008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:23.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.674+0000 7f9f727fc640 1 -- 172.21.15.154:0/274538143 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f9f6c0766c0 con 0x7f9f7c0c6400 2023-09-23T03:27:23.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.861+0000 7f9f83c34640 1 -- 172.21.15.154:0/274538143 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9f7c0b22c0 con 0x7f9f7c0c6400 2023-09-23T03:27:23.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.862+0000 7f9f727fc640 1 -- 172.21.15.154:0/274538143 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v114) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9f6c079ce0 con 0x7f9f7c0c6400 2023-09-23T03:27:23.866 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:27:23.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.865+0000 7f9f83c34640 1 -- 172.21.15.154:0/274538143 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9f6006af60 msgr2=0x7f9f6006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:23.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.865+0000 7f9f83c34640 1 --2- 172.21.15.154:0/274538143 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9f6006af60 0x7f9f6006d420 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009820 tx=0x7f9f78008040 comp rx=0 tx=0).stop 2023-09-23T03:27:23.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.865+0000 7f9f83c34640 1 -- 172.21.15.154:0/274538143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f7c0c6400 msgr2=0x7f9f7c0c7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:23.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.865+0000 7f9f83c34640 1 --2- 172.21.15.154:0/274538143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f7c0c6400 0x7f9f7c0c7190 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c02c2b0 tx=0x7f9f6c004730 comp rx=0 tx=0).stop 2023-09-23T03:27:23.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.865+0000 7f9f83c34640 1 -- 172.21.15.154:0/274538143 shutdown_connections 2023-09-23T03:27:23.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.865+0000 7f9f83c34640 1 --2- 172.21.15.154:0/274538143 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9f6006af60 0x7f9f6006d420 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:23.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.865+0000 7f9f83c34640 1 --2- 172.21.15.154:0/274538143 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9f7c15b470 0x7f9f7c0c76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:23.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.865+0000 7f9f83c34640 1 --2- 172.21.15.154:0/274538143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f7c0c6400 0x7f9f7c0c7190 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:23.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.865+0000 7f9f83c34640 1 --2- 172.21.15.154:0/274538143 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9f680047c0 0x7f9f7c157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:23.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.865+0000 7f9f83c34640 1 -- 172.21.15.154:0/274538143 >> 172.21.15.154:0/274538143 conn(0x7f9f7c09bc20 msgr2=0x7f9f7c0b4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:23.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.865+0000 7f9f83c34640 1 -- 172.21.15.154:0/274538143 shutdown_connections 2023-09-23T03:27:23.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:23.865+0000 7f9f83c34640 1 -- 172.21.15.154:0/274538143 wait complete. 2023-09-23T03:27:23.886 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-09-23T03:27:24.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.101+0000 7fa346eda640 1 Processor -- start 2023-09-23T03:27:24.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.101+0000 7fa346eda640 1 -- start start 2023-09-23T03:27:24.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.103+0000 7fa346eda640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa340071290 0x7fa34015cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:24.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.103+0000 7fa346eda640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3400c9d20 con 0x7fa3400c8d70 2023-09-23T03:27:24.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.103+0000 7fa346eda640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa340071850 con 0x7fa3400c9150 2023-09-23T03:27:24.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.103+0000 7fa346eda640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa340071b30 con 0x7fa340071290 2023-09-23T03:27:24.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.103+0000 7fa344c76640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fa3400c8d70 0x7fa34015c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:35464/0 (socket says 172.21.15.154:35464) 2023-09-23T03:27:24.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.103+0000 7fa344c76640 1 -- 172.21.15.154:0/1129459715 learned_addr learned my addr 172.21.15.154:0/1129459715 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:27:24.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.104+0000 7fa345477640 1 --2- 172.21.15.154:0/1129459715 >> v2:172.21.15.154:3300/0 conn(0x7fa340071290 0x7fa34015cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:24.106 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.104+0000 7fa33f7fe640 1 -- 172.21.15.154:0/1129459715 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 338650997 0 0) 0x7fa3400c9d20 con 0x7fa3400c8d70 2023-09-23T03:27:24.106 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.104+0000 7fa33f7fe640 1 -- 172.21.15.154:0/1129459715 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa330005730 con 0x7fa3400c8d70 2023-09-23T03:27:24.106 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.104+0000 7fa345477640 1 -- 172.21.15.154:0/1129459715 >> v1:172.21.15.190:6789/0 conn(0x7fa3400c9150 legacy=0x7fa3400ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:24.106 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.104+0000 7fa345477640 1 -- 172.21.15.154:0/1129459715 >> v1:172.21.15.154:6789/0 conn(0x7fa3400c8d70 legacy=0x7fa34015c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:24.106 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.104+0000 7fa345477640 1 -- 172.21.15.154:0/1129459715 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa340071e10 con 0x7fa340071290 2023-09-23T03:27:24.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.105+0000 7fa345477640 1 --2- 172.21.15.154:0/1129459715 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa340071290 0x7fa34015cde0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fa338009d90 tx=0x7fa33802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6678155711a36180 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:24.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.106+0000 7fa33f7fe640 1 -- 172.21.15.154:0/1129459715 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33802cd00 con 0x7fa340071290 2023-09-23T03:27:24.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.106+0000 7fa33f7fe640 1 -- 172.21.15.154:0/1129459715 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa33802ce60 con 0x7fa340071290 2023-09-23T03:27:24.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.106+0000 7fa33f7fe640 1 -- 172.21.15.154:0/1129459715 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa338034640 con 0x7fa340071290 2023-09-23T03:27:24.108 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.106+0000 7fa346eda640 1 -- 172.21.15.154:0/1129459715 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa340071290 msgr2=0x7fa34015cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:24.108 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.106+0000 7fa346eda640 1 --2- 172.21.15.154:0/1129459715 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa340071290 0x7fa34015cde0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fa338009d90 tx=0x7fa33802bf50 comp rx=0 tx=0).stop 2023-09-23T03:27:24.108 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.107+0000 7fa346eda640 1 -- 172.21.15.154:0/1129459715 shutdown_connections 2023-09-23T03:27:24.108 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.107+0000 7fa346eda640 1 --2- 172.21.15.154:0/1129459715 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa340071290 0x7fa34015cde0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:24.108 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.107+0000 7fa346eda640 1 -- 172.21.15.154:0/1129459715 >> 172.21.15.154:0/1129459715 conn(0x7fa34006c010 msgr2=0x7fa34006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:24.108 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.107+0000 7fa346eda640 1 -- 172.21.15.154:0/1129459715 shutdown_connections 2023-09-23T03:27:24.109 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.107+0000 7fa346eda640 1 -- 172.21.15.154:0/1129459715 wait complete. 2023-09-23T03:27:24.109 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.108+0000 7fa346eda640 1 Processor -- start 2023-09-23T03:27:24.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.108+0000 7fa346eda640 1 -- start start 2023-09-23T03:27:24.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.109+0000 7fa346eda640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa340071290 0x7fa3400b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:24.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.109+0000 7fa346eda640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3400c8d70 0x7fa3401526f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:24.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.109+0000 7fa346eda640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3400c9150 0x7fa3400c7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:24.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.109+0000 7fa346eda640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3400b2dd0 con 0x7fa3400c8d70 2023-09-23T03:27:24.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.109+0000 7fa346eda640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa340152c30 con 0x7fa3400c9150 2023-09-23T03:27:24.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.109+0000 7fa346eda640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3400b21c0 con 0x7fa340071290 2023-09-23T03:27:24.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.110+0000 7fa33ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3400c8d70 0x7fa3401526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:24.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.110+0000 7fa33ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3400c8d70 0x7fa3401526f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46786/0 (socket says 172.21.15.154:46786) 2023-09-23T03:27:24.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.110+0000 7fa33ffff640 1 -- 172.21.15.154:0/2749661455 learned_addr learned my addr 172.21.15.154:0/2749661455 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:24.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.110+0000 7fa345477640 1 --2- 172.21.15.154:0/2749661455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3400c9150 0x7fa3400c7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:24.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.110+0000 7fa344c76640 1 --2- 172.21.15.154:0/2749661455 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa340071290 0x7fa3400b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:24.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.110+0000 7fa33ffff640 1 -- 172.21.15.154:0/2749661455 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa340071290 msgr2=0x7fa3400b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:24.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.110+0000 7fa33ffff640 1 --2- 172.21.15.154:0/2749661455 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa340071290 0x7fa3400b1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:24.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.110+0000 7fa33ffff640 1 -- 172.21.15.154:0/2749661455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3400c9150 msgr2=0x7fa3400c7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:24.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.110+0000 7fa33ffff640 1 --2- 172.21.15.154:0/2749661455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3400c9150 0x7fa3400c7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:24.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.110+0000 7fa33ffff640 1 -- 172.21.15.154:0/2749661455 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa338009a70 con 0x7fa3400c8d70 2023-09-23T03:27:24.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.110+0000 7fa345477640 1 --2- 172.21.15.154:0/2749661455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3400c9150 0x7fa3400c7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:24.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.111+0000 7fa33ffff640 1 --2- 172.21.15.154:0/2749661455 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3400c8d70 0x7fa3401526f0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fa32c00c400 tx=0x7fa32c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:24.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.111+0000 7fa33d7fa640 1 -- 172.21.15.154:0/2749661455 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32c0052d0 con 0x7fa3400c8d70 2023-09-23T03:27:24.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.111+0000 7fa33d7fa640 1 -- 172.21.15.154:0/2749661455 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa32c005430 con 0x7fa3400c8d70 2023-09-23T03:27:24.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.111+0000 7fa33d7fa640 1 -- 172.21.15.154:0/2749661455 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32c003830 con 0x7fa3400c8d70 2023-09-23T03:27:24.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.111+0000 7fa346eda640 1 -- 172.21.15.154:0/2749661455 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3400b24a0 con 0x7fa3400c8d70 2023-09-23T03:27:24.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.111+0000 7fa346eda640 1 -- 172.21.15.154:0/2749661455 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3400b2a60 con 0x7fa3400c8d70 2023-09-23T03:27:24.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.112+0000 7fa346eda640 1 -- 172.21.15.154:0/2749661455 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa34004e920 con 0x7fa3400c8d70 2023-09-23T03:27:24.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.113+0000 7fa33d7fa640 1 -- 172.21.15.154:0/2749661455 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fa32c003990 con 0x7fa3400c8d70 2023-09-23T03:27:24.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.114+0000 7fa33d7fa640 1 --2- 172.21.15.154:0/2749661455 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa32406af60 0x7fa32406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:24.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.114+0000 7fa33d7fa640 1 -- 172.21.15.154:0/2749661455 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fa32c0890c0 con 0x7fa3400c8d70 2023-09-23T03:27:24.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.114+0000 7fa344c76640 1 --2- 172.21.15.154:0/2749661455 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa32406af60 0x7fa32406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:24.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.115+0000 7fa344c76640 1 --2- 172.21.15.154:0/2749661455 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa32406af60 0x7fa32406d420 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fa3340011e0 tx=0x7fa334009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:24.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.118+0000 7fa33d7fa640 1 -- 172.21.15.154:0/2749661455 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fa32c053d10 con 0x7fa3400c8d70 2023-09-23T03:27:24.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.306+0000 7fa346eda640 1 -- 172.21.15.154:0/2749661455 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7fa34006cbc0 con 0x7fa3400c8d70 2023-09-23T03:27:24.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.307+0000 7fa33d7fa640 1 -- 172.21.15.154:0/2749661455 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v114) v1 ==== 106+0+11 (secure 0 0 0) 0x7fa32c057330 con 0x7fa3400c8d70 2023-09-23T03:27:24.309 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:27:24.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.310+0000 7fa346eda640 1 -- 172.21.15.154:0/2749661455 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa32406af60 msgr2=0x7fa32406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:24.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.310+0000 7fa346eda640 1 --2- 172.21.15.154:0/2749661455 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa32406af60 0x7fa32406d420 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fa3340011e0 tx=0x7fa334009040 comp rx=0 tx=0).stop 2023-09-23T03:27:24.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.310+0000 7fa346eda640 1 -- 172.21.15.154:0/2749661455 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3400c8d70 msgr2=0x7fa3401526f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:24.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.310+0000 7fa346eda640 1 --2- 172.21.15.154:0/2749661455 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3400c8d70 0x7fa3401526f0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fa32c00c400 tx=0x7fa32c00c8c0 comp rx=0 tx=0).stop 2023-09-23T03:27:24.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.310+0000 7fa346eda640 1 -- 172.21.15.154:0/2749661455 shutdown_connections 2023-09-23T03:27:24.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.310+0000 7fa346eda640 1 --2- 172.21.15.154:0/2749661455 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa32406af60 0x7fa32406d420 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:24.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.310+0000 7fa346eda640 1 --2- 172.21.15.154:0/2749661455 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa3400c9150 0x7fa3400c7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:24.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.310+0000 7fa346eda640 1 --2- 172.21.15.154:0/2749661455 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa3400c8d70 0x7fa3401526f0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:24.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.310+0000 7fa346eda640 1 --2- 172.21.15.154:0/2749661455 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa340071290 0x7fa3400b1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:24.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.310+0000 7fa346eda640 1 -- 172.21.15.154:0/2749661455 >> 172.21.15.154:0/2749661455 conn(0x7fa34006c010 msgr2=0x7fa3400cc440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:24.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.310+0000 7fa346eda640 1 -- 172.21.15.154:0/2749661455 shutdown_connections 2023-09-23T03:27:24.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.310+0000 7fa346eda640 1 -- 172.21.15.154:0/2749661455 wait complete. 2023-09-23T03:27:24.331 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-23T03:27:24.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.546+0000 7f0526955640 1 Processor -- start 2023-09-23T03:27:24.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.546+0000 7f0526955640 1 -- start start 2023-09-23T03:27:24.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.547+0000 7f0526955640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05200acce0 0x7f052015ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:24.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.547+0000 7f0526955640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f05200b1b20 0x7f05200ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:24.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.548+0000 7f0526955640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f052015ac30 con 0x7f05200cee70 2023-09-23T03:27:24.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.548+0000 7f0526955640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05200b2e90 con 0x7f05200b1b20 2023-09-23T03:27:24.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.548+0000 7f0526955640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05200cf320 con 0x7f05200acce0 2023-09-23T03:27:24.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.548+0000 7f051f7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f05200b1b20 0x7f05200ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:24.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.548+0000 7f0524ef2640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f05200cee70 0x7f052006cb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:35478/0 (socket says 172.21.15.154:35478) 2023-09-23T03:27:24.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.548+0000 7f0524ef2640 1 -- 172.21.15.154:0/3720906614 learned_addr learned my addr 172.21.15.154:0/3720906614 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:27:24.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.548+0000 7f051ffff640 1 --2- 172.21.15.154:0/3720906614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05200acce0 0x7f052015ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:24.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.548+0000 7f051f7fe640 1 -- 172.21.15.154:0/3720906614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05200acce0 msgr2=0x7f052015ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:24.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.548+0000 7f051f7fe640 1 --2- 172.21.15.154:0/3720906614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05200acce0 0x7f052015ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:24.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.548+0000 7f051f7fe640 1 -- 172.21.15.154:0/3720906614 >> v1:172.21.15.154:6789/0 conn(0x7f05200cee70 legacy=0x7f052006cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:24.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.548+0000 7f051f7fe640 1 -- 172.21.15.154:0/3720906614 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05200b1f70 con 0x7f05200b1b20 2023-09-23T03:27:24.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.548+0000 7f051effd640 1 -- 172.21.15.154:0/3720906614 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 656539983 0 0) 0x7f0518001490 con 0x7f05200cee70 2023-09-23T03:27:24.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.550+0000 7f051f7fe640 1 --2- 172.21.15.154:0/3720906614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05200b1b20 0x7f05200ce930 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f05140097a0 tx=0x7f051402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1bb8613f5d41358 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:24.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.550+0000 7f051effd640 1 -- 172.21.15.154:0/3720906614 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0514031780 con 0x7f05200b1b20 2023-09-23T03:27:24.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.550+0000 7f051effd640 1 -- 172.21.15.154:0/3720906614 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05140318e0 con 0x7f05200b1b20 2023-09-23T03:27:24.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.550+0000 7f051effd640 1 -- 172.21.15.154:0/3720906614 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0514031c70 con 0x7f05200b1b20 2023-09-23T03:27:24.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.550+0000 7f0526955640 1 -- 172.21.15.154:0/3720906614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05200b1b20 msgr2=0x7f05200ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:24.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.550+0000 7f0526955640 1 --2- 172.21.15.154:0/3720906614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05200b1b20 0x7f05200ce930 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f05140097a0 tx=0x7f051402b820 comp rx=0 tx=0).stop 2023-09-23T03:27:24.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.551+0000 7f0526955640 1 -- 172.21.15.154:0/3720906614 shutdown_connections 2023-09-23T03:27:24.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.551+0000 7f0526955640 1 --2- 172.21.15.154:0/3720906614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05200b1b20 0x7f05200ce930 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:24.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.551+0000 7f0526955640 1 --2- 172.21.15.154:0/3720906614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05200acce0 0x7f052015ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:24.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.551+0000 7f0526955640 1 -- 172.21.15.154:0/3720906614 >> 172.21.15.154:0/3720906614 conn(0x7f052006a0f0 msgr2=0x7f052006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:24.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.551+0000 7f0526955640 1 -- 172.21.15.154:0/3720906614 shutdown_connections 2023-09-23T03:27:24.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.551+0000 7f0526955640 1 -- 172.21.15.154:0/3720906614 wait complete. 2023-09-23T03:27:24.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.552+0000 7f0526955640 1 Processor -- start 2023-09-23T03:27:24.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.553+0000 7f0526955640 1 -- start start 2023-09-23T03:27:24.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.554+0000 7f0526955640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05200acce0 0x7f0520159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:24.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.554+0000 7f0526955640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05200b1b20 0x7f05200cc350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:24.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.554+0000 7f0526955640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05200cee70 0x7f05200cc890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:24.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.554+0000 7f0526955640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0520157c70 con 0x7f05200acce0 2023-09-23T03:27:24.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.554+0000 7f0526955640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05200c55e0 con 0x7f05200b1b20 2023-09-23T03:27:24.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.554+0000 7f0526955640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05201f1fd0 con 0x7f05200cee70 2023-09-23T03:27:24.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.554+0000 7f051ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05200acce0 0x7f0520159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:24.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.554+0000 7f051ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05200acce0 0x7f0520159420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46802/0 (socket says 172.21.15.154:46802) 2023-09-23T03:27:24.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.554+0000 7f051ffff640 1 -- 172.21.15.154:0/3521578725 learned_addr learned my addr 172.21.15.154:0/3521578725 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:24.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.554+0000 7f051ffff640 1 -- 172.21.15.154:0/3521578725 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05200cee70 msgr2=0x7f05200cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:24.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.554+0000 7f051ffff640 1 --2- 172.21.15.154:0/3521578725 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05200cee70 0x7f05200cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:24.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.554+0000 7f051f7fe640 1 --2- 172.21.15.154:0/3521578725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05200b1b20 0x7f05200cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:24.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.554+0000 7f051ffff640 1 -- 172.21.15.154:0/3521578725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05200b1b20 msgr2=0x7f05200cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:24.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.554+0000 7f051ffff640 1 --2- 172.21.15.154:0/3521578725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05200b1b20 0x7f05200cc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:24.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.554+0000 7f051ffff640 1 -- 172.21.15.154:0/3521578725 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0514009400 con 0x7f05200acce0 2023-09-23T03:27:24.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.555+0000 7f051ffff640 1 --2- 172.21.15.154:0/3521578725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05200acce0 0x7f0520159420 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f051000d800 tx=0x7f051000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:24.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.555+0000 7f051cff9640 1 -- 172.21.15.154:0/3521578725 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051000b6c0 con 0x7f05200acce0 2023-09-23T03:27:24.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.555+0000 7f051cff9640 1 -- 172.21.15.154:0/3521578725 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051000b820 con 0x7f05200acce0 2023-09-23T03:27:24.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.555+0000 7f0526955640 1 -- 172.21.15.154:0/3521578725 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0520159e00 con 0x7f05200acce0 2023-09-23T03:27:24.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.555+0000 7f051cff9640 1 -- 172.21.15.154:0/3521578725 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051000bbd0 con 0x7f05200acce0 2023-09-23T03:27:24.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.555+0000 7f0526955640 1 -- 172.21.15.154:0/3521578725 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05200c5b10 con 0x7f05200acce0 2023-09-23T03:27:24.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.556+0000 7f0526955640 1 -- 172.21.15.154:0/3521578725 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0520073eb0 con 0x7f05200acce0 2023-09-23T03:27:24.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.557+0000 7f051cff9640 1 -- 172.21.15.154:0/3521578725 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f051000bd30 con 0x7f05200acce0 2023-09-23T03:27:24.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.558+0000 7f051cff9640 1 --2- 172.21.15.154:0/3521578725 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f050c06af30 0x7f050c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:24.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.558+0000 7f051cff9640 1 -- 172.21.15.154:0/3521578725 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f051008a170 con 0x7f05200acce0 2023-09-23T03:27:24.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.558+0000 7f051f7fe640 1 --2- 172.21.15.154:0/3521578725 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f050c06af30 0x7f050c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:24.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.559+0000 7f051f7fe640 1 --2- 172.21.15.154:0/3521578725 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f050c06af30 0x7f050c06d3f0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f0514036000 tx=0x7f051403d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:24.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.562+0000 7f051cff9640 1 -- 172.21.15.154:0/3521578725 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f0510054d30 con 0x7f05200acce0 2023-09-23T03:27:24.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.768+0000 7f0526955640 1 -- 172.21.15.154:0/3521578725 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f052006c930 con 0x7f05200acce0 2023-09-23T03:27:24.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.769+0000 7f051cff9640 1 -- 172.21.15.154:0/3521578725 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v114) v1 ==== 80+0+372 (secure 0 0 0) 0x7f0510058350 con 0x7f05200acce0 2023-09-23T03:27:24.770 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.154:6837/3181099748 2023-09-24T03:26:34.396308+0000 2023-09-23T03:27:24.770 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.154:6835/3181099748 2023-09-24T03:26:34.396308+0000 2023-09-23T03:27:24.770 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.154:6835/4233181417 2023-09-24T03:26:27.714769+0000 2023-09-23T03:27:24.770 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.154:6837/4233181417 2023-09-24T03:26:27.714769+0000 2023-09-23T03:27:24.770 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.190:6835/2967125458 2023-09-24T03:26:25.688784+0000 2023-09-23T03:27:24.770 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.190:6833/2967125458 2023-09-24T03:26:25.688784+0000 2023-09-23T03:27:24.771 INFO:teuthology.orchestra.run.smithi154.stderr:listed 6 entries 2023-09-23T03:27:24.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.771+0000 7f0526955640 1 -- 172.21.15.154:0/3521578725 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f050c06af30 msgr2=0x7f050c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:24.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.771+0000 7f0526955640 1 --2- 172.21.15.154:0/3521578725 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f050c06af30 0x7f050c06d3f0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f0514036000 tx=0x7f051403d040 comp rx=0 tx=0).stop 2023-09-23T03:27:24.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.771+0000 7f0526955640 1 -- 172.21.15.154:0/3521578725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05200acce0 msgr2=0x7f0520159420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:24.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.771+0000 7f0526955640 1 --2- 172.21.15.154:0/3521578725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05200acce0 0x7f0520159420 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f051000d800 tx=0x7f051000dcc0 comp rx=0 tx=0).stop 2023-09-23T03:27:24.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.772+0000 7f0526955640 1 -- 172.21.15.154:0/3521578725 shutdown_connections 2023-09-23T03:27:24.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.772+0000 7f0526955640 1 --2- 172.21.15.154:0/3521578725 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f050c06af30 0x7f050c06d3f0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:24.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.772+0000 7f0526955640 1 --2- 172.21.15.154:0/3521578725 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f05200cee70 0x7f05200cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:24.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.772+0000 7f0526955640 1 --2- 172.21.15.154:0/3521578725 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f05200b1b20 0x7f05200cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:24.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.772+0000 7f0526955640 1 --2- 172.21.15.154:0/3521578725 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05200acce0 0x7f0520159420 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:24.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.772+0000 7f0526955640 1 -- 172.21.15.154:0/3521578725 >> 172.21.15.154:0/3521578725 conn(0x7f052006a0f0 msgr2=0x7f05200cb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:24.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.772+0000 7f0526955640 1 -- 172.21.15.154:0/3521578725 shutdown_connections 2023-09-23T03:27:24.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:24.772+0000 7f0526955640 1 -- 172.21.15.154:0/3521578725 wait complete. 2023-09-23T03:27:24.792 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi154.front.sepia.ceph.com... 2023-09-23T03:27:24.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:27:24.792 DEBUG:teuthology.orchestra.run.smithi154:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-09-23T03:27:24.883 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:ceph-fuse[123940]: fuse finished with error 0 and tester_r 0 2023-09-23T03:27:24.889 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2023-09-23T03:27:24.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:27:24.889 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-09-23T03:27:25.068 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:27:25.068 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:27:25.127 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053603,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248804k,nr_inodes=812201,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-09-23T03:27:25.127 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi190.front.sepia.ceph.com 2023-09-23T03:27:25.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:27:25.127 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-09-23T03:27:25.179 INFO:teuthology.orchestra.run.smithi190.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-09-23T03:27:25.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T03:27:25.180 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:27:25.180 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:27:25.241 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053600,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-09-23T03:27:25.241 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2023-09-23T03:27:25.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:27:25.241 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-09-23T03:27:25.293 INFO:teuthology.orchestra.run.smithi154.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-09-23T03:27:25.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T03:27:25.295 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:27:25.295 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:27:25.354 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053603,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248804k,nr_inodes=812201,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-09-23T03:27:25.354 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi190.front.sepia.ceph.com 2023-09-23T03:27:25.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:27:25.354 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-09-23T03:27:25.407 INFO:teuthology.orchestra.run.smithi190.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-09-23T03:27:25.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T03:27:25.408 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:27:25.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.622+0000 7f87e7fff640 1 Processor -- start 2023-09-23T03:27:25.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.622+0000 7f87e7fff640 1 -- start start 2023-09-23T03:27:25.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.624+0000 7f87e7fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f87e80cf500 0x7f87e80ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:25.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.624+0000 7f87e7fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f87e815cc00 0x7f87e80ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:25.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.624+0000 7f87e7fff640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87e80d17f0 con 0x7f87e80cf8e0 2023-09-23T03:27:25.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.624+0000 7f87e7fff640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87e806be10 con 0x7f87e80cf500 2023-09-23T03:27:25.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.624+0000 7f87e7fff640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87e80d0470 con 0x7f87e815cc00 2023-09-23T03:27:25.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.624+0000 7f87e6ffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f87e80cf500 0x7f87e80ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:25.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.624+0000 7f87e6ffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f87e80cf500 0x7f87e80ce050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46818/0 (socket says 172.21.15.154:46818) 2023-09-23T03:27:25.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.624+0000 7f87e6ffd640 1 -- 172.21.15.154:0/2167555854 learned_addr learned my addr 172.21.15.154:0/2167555854 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:25.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.625+0000 7f87e77fe640 1 --2- 172.21.15.154:0/2167555854 >> v2:172.21.15.190:3300/0 conn(0x7f87e815cc00 0x7f87e80ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:25.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.625+0000 7f87e6ffd640 1 -- 172.21.15.154:0/2167555854 >> v2:172.21.15.190:3300/0 conn(0x7f87e815cc00 msgr2=0x7f87e80ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:25.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.625+0000 7f87e6ffd640 1 --2- 172.21.15.154:0/2167555854 >> v2:172.21.15.190:3300/0 conn(0x7f87e815cc00 0x7f87e80ce590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:25.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.625+0000 7f87e6ffd640 1 -- 172.21.15.154:0/2167555854 >> v1:172.21.15.190:6789/0 conn(0x7f87e80cf8e0 legacy=0x7f87e815c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:25.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.625+0000 7f87e6ffd640 1 -- 172.21.15.154:0/2167555854 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e80d0750 con 0x7f87e80cf500 2023-09-23T03:27:25.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.626+0000 7f87e6ffd640 1 --2- 172.21.15.154:0/2167555854 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87e80cf500 0x7f87e80ce050 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f87d8009850 tx=0x7f87d802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804b535f2172059a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:25.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.627+0000 7f87e67fc640 1 -- 172.21.15.154:0/2167555854 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d8004230 con 0x7f87e80cf500 2023-09-23T03:27:25.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.627+0000 7f87e67fc640 1 -- 172.21.15.154:0/2167555854 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d8004390 con 0x7f87e80cf500 2023-09-23T03:27:25.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.627+0000 7f87e67fc640 1 -- 172.21.15.154:0/2167555854 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d8003810 con 0x7f87e80cf500 2023-09-23T03:27:25.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.627+0000 7f87e7fff640 1 -- 172.21.15.154:0/2167555854 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87e80cf500 msgr2=0x7f87e80ce050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:25.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.627+0000 7f87e7fff640 1 --2- 172.21.15.154:0/2167555854 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87e80cf500 0x7f87e80ce050 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f87d8009850 tx=0x7f87d802b860 comp rx=0 tx=0).stop 2023-09-23T03:27:25.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.628+0000 7f87e7fff640 1 -- 172.21.15.154:0/2167555854 shutdown_connections 2023-09-23T03:27:25.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.628+0000 7f87e7fff640 1 --2- 172.21.15.154:0/2167555854 >> v2:172.21.15.190:3300/0 conn(0x7f87e815cc00 0x7f87e80ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:25.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.628+0000 7f87e7fff640 1 --2- 172.21.15.154:0/2167555854 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87e80cf500 0x7f87e80ce050 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:25.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.628+0000 7f87e7fff640 1 -- 172.21.15.154:0/2167555854 >> 172.21.15.154:0/2167555854 conn(0x7f87e809bbb0 msgr2=0x7f87e80d28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:25.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.628+0000 7f87e7fff640 1 -- 172.21.15.154:0/2167555854 shutdown_connections 2023-09-23T03:27:25.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.628+0000 7f87e7fff640 1 -- 172.21.15.154:0/2167555854 wait complete. 2023-09-23T03:27:25.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.629+0000 7f87e7fff640 1 Processor -- start 2023-09-23T03:27:25.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.630+0000 7f87e7fff640 1 -- start start 2023-09-23T03:27:25.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.630+0000 7f87e7fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87e80cf500 0x7f87e80cb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:25.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.630+0000 7f87e7fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87e80cf8e0 0x7f87e8147b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:25.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.630+0000 7f87e7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87e815cc00 0x7f87e8153ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:25.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.630+0000 7f87e7fff640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e80cc840 con 0x7f87e815cc00 2023-09-23T03:27:25.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.630+0000 7f87e7fff640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e80cbbc0 con 0x7f87e80cf500 2023-09-23T03:27:25.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.630+0000 7f87e7fff640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87e80cbea0 con 0x7f87e80cf8e0 2023-09-23T03:27:25.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.631+0000 7f87e77fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87e815cc00 0x7f87e8153ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:25.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.631+0000 7f87e77fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87e815cc00 0x7f87e8153ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46834/0 (socket says 172.21.15.154:46834) 2023-09-23T03:27:25.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.631+0000 7f87e77fe640 1 -- 172.21.15.154:0/2465661591 learned_addr learned my addr 172.21.15.154:0/2465661591 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:25.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.631+0000 7f87e6ffd640 1 --2- 172.21.15.154:0/2465661591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87e80cf500 0x7f87e80cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:25.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.631+0000 7f87dffff640 1 --2- 172.21.15.154:0/2465661591 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87e80cf8e0 0x7f87e8147b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:25.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.631+0000 7f87e77fe640 1 -- 172.21.15.154:0/2465661591 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87e80cf8e0 msgr2=0x7f87e8147b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:25.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.631+0000 7f87e77fe640 1 --2- 172.21.15.154:0/2465661591 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87e80cf8e0 0x7f87e8147b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:25.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.631+0000 7f87e77fe640 1 -- 172.21.15.154:0/2465661591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87e80cf500 msgr2=0x7f87e80cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:25.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.631+0000 7f87e77fe640 1 --2- 172.21.15.154:0/2465661591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87e80cf500 0x7f87e80cb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:25.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.631+0000 7f87e77fe640 1 -- 172.21.15.154:0/2465661591 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d8009460 con 0x7f87e815cc00 2023-09-23T03:27:25.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.631+0000 7f87e6ffd640 1 --2- 172.21.15.154:0/2465661591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87e80cf500 0x7f87e80cb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:25.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.631+0000 7f87dffff640 1 --2- 172.21.15.154:0/2465661591 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87e80cf8e0 0x7f87e8147b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:25.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.631+0000 7f87e77fe640 1 --2- 172.21.15.154:0/2465661591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87e815cc00 0x7f87e8153ec0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f87e0007e40 tx=0x7f87e000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:25.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.632+0000 7f87e4ff9640 1 -- 172.21.15.154:0/2465661591 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e0014620 con 0x7f87e815cc00 2023-09-23T03:27:25.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.632+0000 7f87e4ff9640 1 -- 172.21.15.154:0/2465661591 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e000ee90 con 0x7f87e815cc00 2023-09-23T03:27:25.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.632+0000 7f87e4ff9640 1 -- 172.21.15.154:0/2465661591 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e00149e0 con 0x7f87e815cc00 2023-09-23T03:27:25.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.632+0000 7f87e7fff640 1 -- 172.21.15.154:0/2465661591 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87e80cc180 con 0x7f87e815cc00 2023-09-23T03:27:25.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.632+0000 7f87e7fff640 1 -- 172.21.15.154:0/2465661591 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e80cb8d0 con 0x7f87e815cc00 2023-09-23T03:27:25.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.633+0000 7f87e7fff640 1 -- 172.21.15.154:0/2465661591 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87e80cc980 con 0x7f87e815cc00 2023-09-23T03:27:25.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.634+0000 7f87e4ff9640 1 -- 172.21.15.154:0/2465661591 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f87e0014b40 con 0x7f87e815cc00 2023-09-23T03:27:25.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.635+0000 7f87e4ff9640 1 --2- 172.21.15.154:0/2465661591 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f87c806af30 0x7f87c806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:25.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.635+0000 7f87e4ff9640 1 -- 172.21.15.154:0/2465661591 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f87e008a610 con 0x7f87e815cc00 2023-09-23T03:27:25.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.635+0000 7f87e6ffd640 1 --2- 172.21.15.154:0/2465661591 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f87c806af30 0x7f87c806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:25.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.636+0000 7f87e6ffd640 1 --2- 172.21.15.154:0/2465661591 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f87c806af30 0x7f87c806d3f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f87d8034000 tx=0x7f87d8008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:25.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.639+0000 7f87e4ff9640 1 -- 172.21.15.154:0/2465661591 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f87e80cc980 con 0x7f87e815cc00 2023-09-23T03:27:25.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.864+0000 7f87e7fff640 1 -- 172.21.15.154:0/2465661591 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87e80cc980 con 0x7f87e815cc00 2023-09-23T03:27:25.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.865+0000 7f87e4ff9640 1 -- 172.21.15.154:0/2465661591 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4926 (secure 0 0 0) 0x7f87e80cc980 con 0x7f87e815cc00 2023-09-23T03:27:25.866 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:25.867 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":99,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5753,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/60017546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":60017546},{"type":"v1","addr":"172.21.15.190:6833","nonce":60017546}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95},{"gid":6415,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2708068523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2708068523},{"type":"v1","addr":"172.21.15.190:6835","nonce":2708068523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":99},{"gid":8592,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/1846547753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1846547753},{"type":"v1","addr":"172.21.15.154:6836","nonce":1846547753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:26:26.774375+0000","modified":"2023-09-23T03:26:36.432946+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8595":{"gid":8595,"name":"a","rank":0,"incarnation":95,"state":"up:active","state_seq":4,"addr":"172.21.15.154:6837/1758434763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1758434763},{"type":"v1","addr":"172.21.15.154:6837","nonce":1758434763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15,16],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-09-23T03:27:25.867 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 99 2023-09-23T03:27:25.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.868+0000 7f87e7fff640 1 -- 172.21.15.154:0/2465661591 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f87c806af30 msgr2=0x7f87c806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:25.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.868+0000 7f87e7fff640 1 --2- 172.21.15.154:0/2465661591 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f87c806af30 0x7f87c806d3f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f87d8034000 tx=0x7f87d8008030 comp rx=0 tx=0).stop 2023-09-23T03:27:25.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.868+0000 7f87e7fff640 1 -- 172.21.15.154:0/2465661591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87e815cc00 msgr2=0x7f87e8153ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:25.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.868+0000 7f87e7fff640 1 --2- 172.21.15.154:0/2465661591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87e815cc00 0x7f87e8153ec0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f87e0007e40 tx=0x7f87e000e4b0 comp rx=0 tx=0).stop 2023-09-23T03:27:25.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.868+0000 7f87e7fff640 1 -- 172.21.15.154:0/2465661591 shutdown_connections 2023-09-23T03:27:25.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.868+0000 7f87e7fff640 1 --2- 172.21.15.154:0/2465661591 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f87c806af30 0x7f87c806d3f0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:25.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.868+0000 7f87e7fff640 1 --2- 172.21.15.154:0/2465661591 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f87e815cc00 0x7f87e8153ec0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:25.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.868+0000 7f87e7fff640 1 --2- 172.21.15.154:0/2465661591 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f87e80cf8e0 0x7f87e8147b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:25.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.868+0000 7f87e7fff640 1 --2- 172.21.15.154:0/2465661591 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f87e80cf500 0x7f87e80cb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:25.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.868+0000 7f87e7fff640 1 -- 172.21.15.154:0/2465661591 >> 172.21.15.154:0/2465661591 conn(0x7f87e809bbb0 msgr2=0x7f87e815c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:25.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.869+0000 7f87e7fff640 1 -- 172.21.15.154:0/2465661591 shutdown_connections 2023-09-23T03:27:25.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:25.869+0000 7f87e7fff640 1 -- 172.21.15.154:0/2465661591 wait complete. 2023-09-23T03:27:25.887 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:27:26.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.104+0000 7f900db6e640 1 Processor -- start 2023-09-23T03:27:26.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.104+0000 7f900db6e640 1 -- start start 2023-09-23T03:27:26.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.105+0000 7f900db6e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90080b2fe0 0x7f90080ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:26.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.105+0000 7f900db6e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f900806ceb0 con 0x7f90080ada30 2023-09-23T03:27:26.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.106+0000 7f900db6e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9008159c90 con 0x7f90080cf130 2023-09-23T03:27:26.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.106+0000 7f900db6e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90080ade10 con 0x7f90080b2fe0 2023-09-23T03:27:26.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.106+0000 7f90077fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f90080ada30 0x7f90080cea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:35480/0 (socket says 172.21.15.154:35480) 2023-09-23T03:27:26.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.106+0000 7f90077fe640 1 -- 172.21.15.154:0/711201135 learned_addr learned my addr 172.21.15.154:0/711201135 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:27:26.109 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.106+0000 7f90067fc640 1 -- 172.21.15.154:0/711201135 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1510100695 0 0) 0x7f8ff8001290 con 0x7f90080ada30 2023-09-23T03:27:26.109 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.106+0000 7f9007fff640 1 --2- 172.21.15.154:0/711201135 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90080b2fe0 0x7f90080ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:26.109 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.106+0000 7f90067fc640 1 -- 172.21.15.154:0/711201135 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ff4003520 con 0x7f90080ada30 2023-09-23T03:27:26.109 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.107+0000 7f90067fc640 1 -- 172.21.15.154:0/711201135 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3839603942 0 0) 0x7f8ff4003520 con 0x7f90080ada30 2023-09-23T03:27:26.109 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.107+0000 7f90067fc640 1 -- 172.21.15.154:0/711201135 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90080b2fe0 msgr2=0x7f90080ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:26.109 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.107+0000 7f90067fc640 1 --2- 172.21.15.154:0/711201135 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90080b2fe0 0x7f90080ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:26.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.107+0000 7f90067fc640 1 -- 172.21.15.154:0/711201135 >> v1:172.21.15.190:6789/0 conn(0x7f90080cf130 legacy=0x7f900815d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:26.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.107+0000 7f90067fc640 1 -- 172.21.15.154:0/711201135 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f900806c110 con 0x7f90080ada30 2023-09-23T03:27:26.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.107+0000 7f90067fc640 1 -- 172.21.15.154:0/711201135 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f8ff8001980 con 0x7f90080ada30 2023-09-23T03:27:26.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.107+0000 7f90067fc640 1 -- 172.21.15.154:0/711201135 >> v1:172.21.15.154:6789/0 conn(0x7f90080ada30 legacy=0x7f90080cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:26.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.107+0000 7f90067fc640 1 --2- 172.21.15.154:0/711201135 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90080b2fe0 0x7f8ff40039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:26.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.107+0000 7f90067fc640 1 --2- 172.21.15.154:0/711201135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ff4003fd0 0x7f8ff40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:26.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.107+0000 7f90067fc640 1 --2- 172.21.15.154:0/711201135 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ff4004d00 0x7f8ff4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:26.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.107+0000 7f90067fc640 1 -- 172.21.15.154:0/711201135 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90080ade10 con 0x7f8ff4004d00 2023-09-23T03:27:26.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.107+0000 7f90067fc640 1 -- 172.21.15.154:0/711201135 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff400dd50 con 0x7f8ff4003fd0 2023-09-23T03:27:26.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.107+0000 7f90067fc640 1 -- 172.21.15.154:0/711201135 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff40059d0 con 0x7f90080b2fe0 2023-09-23T03:27:26.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.107+0000 7f9007fff640 1 --2- 172.21.15.154:0/711201135 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ff4004d00 0x7f8ff4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:26.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.107+0000 7f9006ffd640 1 --2- 172.21.15.154:0/711201135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ff4003fd0 0x7f8ff40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:26.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.107+0000 7f9007fff640 1 -- 172.21.15.154:0/711201135 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90080b2fe0 msgr2=0x7f8ff40039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:26.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.107+0000 7f90077fe640 1 --2- 172.21.15.154:0/711201135 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90080b2fe0 0x7f8ff40039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:26.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.107+0000 7f9007fff640 1 --2- 172.21.15.154:0/711201135 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90080b2fe0 0x7f8ff40039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:26.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.107+0000 7f9007fff640 1 -- 172.21.15.154:0/711201135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ff4003fd0 msgr2=0x7f8ff40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:26.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.107+0000 7f9007fff640 1 --2- 172.21.15.154:0/711201135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ff4003fd0 0x7f8ff40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:26.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.107+0000 7f9007fff640 1 -- 172.21.15.154:0/711201135 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff4003520 con 0x7f8ff4004d00 2023-09-23T03:27:26.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.108+0000 7f9006ffd640 1 --2- 172.21.15.154:0/711201135 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ff4003fd0 0x7f8ff40047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:26.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.108+0000 7f90077fe640 1 --2- 172.21.15.154:0/711201135 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90080b2fe0 0x7f8ff40039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:26.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.109+0000 7f9007fff640 1 --2- 172.21.15.154:0/711201135 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ff4004d00 0x7f8ff4005180 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f90080b3410 tx=0x7f8ff40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf6d32d0b69474f6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:26.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.109+0000 7f90067fc640 1 -- 172.21.15.154:0/711201135 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9000033680 con 0x7f8ff4004d00 2023-09-23T03:27:26.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.109+0000 7f90067fc640 1 -- 172.21.15.154:0/711201135 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90000337e0 con 0x7f8ff4004d00 2023-09-23T03:27:26.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.109+0000 7f90067fc640 1 -- 172.21.15.154:0/711201135 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9000033b70 con 0x7f8ff4004d00 2023-09-23T03:27:26.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.109+0000 7f900db6e640 1 -- 172.21.15.154:0/711201135 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ff4004d00 msgr2=0x7f8ff4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:26.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.109+0000 7f900db6e640 1 --2- 172.21.15.154:0/711201135 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ff4004d00 0x7f8ff4005180 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f90080b3410 tx=0x7f8ff40057f0 comp rx=0 tx=0).stop 2023-09-23T03:27:26.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.109+0000 7f9007fff640 1 -- 172.21.15.154:0/711201135 reap_dead start 2023-09-23T03:27:26.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.109+0000 7f900db6e640 1 -- 172.21.15.154:0/711201135 shutdown_connections 2023-09-23T03:27:26.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.109+0000 7f900db6e640 1 -- 172.21.15.154:0/711201135 >> 172.21.15.154:0/711201135 conn(0x7f900806a0f0 msgr2=0x7f900806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:26.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.110+0000 7f900db6e640 1 -- 172.21.15.154:0/711201135 shutdown_connections 2023-09-23T03:27:26.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.110+0000 7f900db6e640 1 -- 172.21.15.154:0/711201135 wait complete. 2023-09-23T03:27:26.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.111+0000 7f900db6e640 1 Processor -- start 2023-09-23T03:27:26.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.111+0000 7f900db6e640 1 -- start start 2023-09-23T03:27:26.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.112+0000 7f900db6e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ff4004d00 0x7f90081f2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:26.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.112+0000 7f900db6e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90080b09f0 0x7f90080af020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:26.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.112+0000 7f900db6e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90081f16f0 0x7f900815c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:26.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.112+0000 7f900db6e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90080b0560 con 0x7f90080b09f0 2023-09-23T03:27:26.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.112+0000 7f900db6e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90080b0e70 con 0x7f8ff4004d00 2023-09-23T03:27:26.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.112+0000 7f900db6e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90080af950 con 0x7f90081f16f0 2023-09-23T03:27:26.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.113+0000 7f9006ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90080b09f0 0x7f90080af020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:26.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.113+0000 7f9006ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90080b09f0 0x7f90080af020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46840/0 (socket says 172.21.15.154:46840) 2023-09-23T03:27:26.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.113+0000 7f9006ffd640 1 -- 172.21.15.154:0/3177433760 learned_addr learned my addr 172.21.15.154:0/3177433760 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:26.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.113+0000 7f90077fe640 1 --2- 172.21.15.154:0/3177433760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ff4004d00 0x7f90081f2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:26.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.113+0000 7f9007fff640 1 --2- 172.21.15.154:0/3177433760 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90081f16f0 0x7f900815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:26.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.113+0000 7f9006ffd640 1 -- 172.21.15.154:0/3177433760 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90081f16f0 msgr2=0x7f900815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:26.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.113+0000 7f9006ffd640 1 --2- 172.21.15.154:0/3177433760 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90081f16f0 0x7f900815c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:26.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.113+0000 7f9006ffd640 1 -- 172.21.15.154:0/3177433760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ff4004d00 msgr2=0x7f90081f2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:26.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.113+0000 7f9006ffd640 1 --2- 172.21.15.154:0/3177433760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ff4004d00 0x7f90081f2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:26.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.113+0000 7f9006ffd640 1 -- 172.21.15.154:0/3177433760 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90080afc30 con 0x7f90080b09f0 2023-09-23T03:27:26.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.113+0000 7f90077fe640 1 --2- 172.21.15.154:0/3177433760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ff4004d00 0x7f90081f2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:27:26.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.114+0000 7f9006ffd640 1 --2- 172.21.15.154:0/3177433760 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90080b09f0 0x7f90080af020 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f8ff000db10 tx=0x7f8ff0002060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:26.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.114+0000 7f8ffffff640 1 -- 172.21.15.154:0/3177433760 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0017070 con 0x7f90080b09f0 2023-09-23T03:27:26.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.114+0000 7f8ffffff640 1 -- 172.21.15.154:0/3177433760 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ff0003ba0 con 0x7f90080b09f0 2023-09-23T03:27:26.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.114+0000 7f8ffffff640 1 -- 172.21.15.154:0/3177433760 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff0006b50 con 0x7f90080b09f0 2023-09-23T03:27:26.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.114+0000 7f900db6e640 1 -- 172.21.15.154:0/3177433760 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90080aff10 con 0x7f90080b09f0 2023-09-23T03:27:26.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.114+0000 7f900db6e640 1 -- 172.21.15.154:0/3177433760 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90080af560 con 0x7f90080b09f0 2023-09-23T03:27:26.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.115+0000 7f900db6e640 1 -- 172.21.15.154:0/3177433760 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9008073eb0 con 0x7f90080b09f0 2023-09-23T03:27:26.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.116+0000 7f8ffffff640 1 -- 172.21.15.154:0/3177433760 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f8ff0006cb0 con 0x7f90080b09f0 2023-09-23T03:27:26.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.117+0000 7f8ffffff640 1 --2- 172.21.15.154:0/3177433760 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8fe806af30 0x7f8fe806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:26.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.117+0000 7f8ffffff640 1 -- 172.21.15.154:0/3177433760 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f8ff0089810 con 0x7f90080b09f0 2023-09-23T03:27:26.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.117+0000 7f90077fe640 1 --2- 172.21.15.154:0/3177433760 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8fe806af30 0x7f8fe806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:26.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.118+0000 7f90077fe640 1 --2- 172.21.15.154:0/3177433760 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8fe806af30 0x7f8fe806d3f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f8ff8001ab0 tx=0x7f8ff8001070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:26.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.121+0000 7f8ffffff640 1 -- 172.21.15.154:0/3177433760 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f8ff00543d0 con 0x7f90080b09f0 2023-09-23T03:27:26.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.311+0000 7f900db6e640 1 -- 172.21.15.154:0/3177433760 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90080b06a0 con 0x7f90080b09f0 2023-09-23T03:27:26.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.314+0000 7f8ffffff640 1 -- 172.21.15.154:0/3177433760 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+18945 (secure 0 0 0) 0x7f8ff00579f0 con 0x7f90080b09f0 2023-09-23T03:27:26.315 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:26.316 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":114,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:26:39.641972+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:25:44.763842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-09-23T03:25:46.025926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data_extra","create_time":"2023-09-23T03:25:56.961772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6837/3181099748":"2023-09-24T03:26:34.396308+0000","172.21.15.154:6835/3181099748":"2023-09-24T03:26:34.396308+0000","172.21.15.154:6835/4233181417":"2023-09-24T03:26:27.714769+0000","172.21.15.154:6837/4233181417":"2023-09-24T03:26:27.714769+0000","172.21.15.190:6835/2967125458":"2023-09-24T03:26:25.688784+0000","172.21.15.190:6833/2967125458":"2023-09-24T03:26:25.688784+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:27:26.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.316+0000 7f900db6e640 1 -- 172.21.15.154:0/3177433760 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8fe806af30 msgr2=0x7f8fe806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:26.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.317+0000 7f900db6e640 1 --2- 172.21.15.154:0/3177433760 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8fe806af30 0x7f8fe806d3f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f8ff8001ab0 tx=0x7f8ff8001070 comp rx=0 tx=0).stop 2023-09-23T03:27:26.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.317+0000 7f900db6e640 1 -- 172.21.15.154:0/3177433760 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90080b09f0 msgr2=0x7f90080af020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:26.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.317+0000 7f900db6e640 1 --2- 172.21.15.154:0/3177433760 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90080b09f0 0x7f90080af020 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f8ff000db10 tx=0x7f8ff0002060 comp rx=0 tx=0).stop 2023-09-23T03:27:26.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.317+0000 7f900db6e640 1 -- 172.21.15.154:0/3177433760 shutdown_connections 2023-09-23T03:27:26.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.317+0000 7f900db6e640 1 --2- 172.21.15.154:0/3177433760 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8fe806af30 0x7f8fe806d3f0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:26.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.317+0000 7f900db6e640 1 --2- 172.21.15.154:0/3177433760 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f90081f16f0 0x7f900815c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:26.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.317+0000 7f900db6e640 1 --2- 172.21.15.154:0/3177433760 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f90080b09f0 0x7f90080af020 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:26.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.317+0000 7f900db6e640 1 --2- 172.21.15.154:0/3177433760 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8ff4004d00 0x7f90081f2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:26.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.317+0000 7f900db6e640 1 -- 172.21.15.154:0/3177433760 >> 172.21.15.154:0/3177433760 conn(0x7f900806a0f0 msgr2=0x7f90080ab570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:26.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.317+0000 7f900db6e640 1 -- 172.21.15.154:0/3177433760 shutdown_connections 2023-09-23T03:27:26.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.317+0000 7f900db6e640 1 -- 172.21.15.154:0/3177433760 wait complete. 2023-09-23T03:27:26.336 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:25:44.763842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:25:46.025926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-09-23T03:25:56.961772+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T03:27:26.337 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:27:26.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.555+0000 7f43d2dbf640 1 Processor -- start 2023-09-23T03:27:26.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.555+0000 7f43d2dbf640 1 -- start start 2023-09-23T03:27:26.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.557+0000 7f43d2dbf640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f43cc06c470 0x7f43cc15d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:26.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.557+0000 7f43d2dbf640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f43cc0c88f0 0x7f43cc159f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:26.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.557+0000 7f43d2dbf640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43cc070cd0 con 0x7f43cc159b50 2023-09-23T03:27:26.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.557+0000 7f43d2dbf640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43cc15e050 con 0x7f43cc0c88f0 2023-09-23T03:27:26.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.557+0000 7f43d2dbf640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43cc0c8d40 con 0x7f43cc06c470 2023-09-23T03:27:26.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.557+0000 7f43d25be640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f43cc0c88f0 0x7f43cc159f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:26.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.557+0000 7f43d25be640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f43cc0c88f0 0x7f43cc159f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46842/0 (socket says 172.21.15.154:46842) 2023-09-23T03:27:26.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.557+0000 7f43d25be640 1 -- 172.21.15.154:0/3338040768 learned_addr learned my addr 172.21.15.154:0/3338040768 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:26.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.558+0000 7f43d15bc640 1 --2- 172.21.15.154:0/3338040768 >> v2:172.21.15.190:3300/0 conn(0x7f43cc06c470 0x7f43cc15d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:26.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.558+0000 7f43d25be640 1 -- 172.21.15.154:0/3338040768 >> v2:172.21.15.190:3300/0 conn(0x7f43cc06c470 msgr2=0x7f43cc15d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:26.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.558+0000 7f43d25be640 1 --2- 172.21.15.154:0/3338040768 >> v2:172.21.15.190:3300/0 conn(0x7f43cc06c470 0x7f43cc15d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:26.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.558+0000 7f43d25be640 1 -- 172.21.15.154:0/3338040768 >> v1:172.21.15.190:6789/0 conn(0x7f43cc159b50 legacy=0x7f43cc06bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:26.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.558+0000 7f43d25be640 1 -- 172.21.15.154:0/3338040768 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43cc15d7b0 con 0x7f43cc0c88f0 2023-09-23T03:27:26.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.558+0000 7f43d15bc640 1 --2- 172.21.15.154:0/3338040768 >> v2:172.21.15.190:3300/0 conn(0x7f43cc06c470 0x7f43cc15d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:26.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.559+0000 7f43d25be640 1 --2- 172.21.15.154:0/3338040768 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43cc0c88f0 0x7f43cc159f30 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f43cc159ae0 tx=0x7f43c402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ad25395e1a8d9a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:26.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.559+0000 7f43d0dbb640 1 -- 172.21.15.154:0/3338040768 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c402ca80 con 0x7f43cc0c88f0 2023-09-23T03:27:26.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.559+0000 7f43d0dbb640 1 -- 172.21.15.154:0/3338040768 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c402cbe0 con 0x7f43cc0c88f0 2023-09-23T03:27:26.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.559+0000 7f43d0dbb640 1 -- 172.21.15.154:0/3338040768 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c40343f0 con 0x7f43cc0c88f0 2023-09-23T03:27:26.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.560+0000 7f43d2dbf640 1 -- 172.21.15.154:0/3338040768 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43cc0c88f0 msgr2=0x7f43cc159f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:26.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.560+0000 7f43d2dbf640 1 --2- 172.21.15.154:0/3338040768 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43cc0c88f0 0x7f43cc159f30 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f43cc159ae0 tx=0x7f43c402bee0 comp rx=0 tx=0).stop 2023-09-23T03:27:26.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.560+0000 7f43d2dbf640 1 -- 172.21.15.154:0/3338040768 shutdown_connections 2023-09-23T03:27:26.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.560+0000 7f43d2dbf640 1 --2- 172.21.15.154:0/3338040768 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43cc0c88f0 0x7f43cc159f30 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:26.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.560+0000 7f43d2dbf640 1 --2- 172.21.15.154:0/3338040768 >> v2:172.21.15.190:3300/0 conn(0x7f43cc06c470 0x7f43cc15d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:26.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.560+0000 7f43d2dbf640 1 -- 172.21.15.154:0/3338040768 >> 172.21.15.154:0/3338040768 conn(0x7f43cc06a300 msgr2=0x7f43cc0afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:26.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.560+0000 7f43d2dbf640 1 -- 172.21.15.154:0/3338040768 shutdown_connections 2023-09-23T03:27:26.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.560+0000 7f43d2dbf640 1 -- 172.21.15.154:0/3338040768 wait complete. 2023-09-23T03:27:26.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.562+0000 7f43d2dbf640 1 Processor -- start 2023-09-23T03:27:26.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.563+0000 7f43d2dbf640 1 -- start start 2023-09-23T03:27:26.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.564+0000 7f43d2dbf640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43cc06c470 0x7f43cc148750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:26.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.564+0000 7f43d2dbf640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43cc0c88f0 0x7f43cc15a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:26.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.564+0000 7f43d2dbf640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43cc159b50 0x7f43cc0c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:26.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.564+0000 7f43d2dbf640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc159f30 con 0x7f43cc159b50 2023-09-23T03:27:26.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.564+0000 7f43d2dbf640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc0c7e60 con 0x7f43cc06c470 2023-09-23T03:27:26.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.564+0000 7f43d2dbf640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc0c8140 con 0x7f43cc0c88f0 2023-09-23T03:27:26.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.564+0000 7f43d25be640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43cc159b50 0x7f43cc0c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:26.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.564+0000 7f43d25be640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43cc159b50 0x7f43cc0c75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46850/0 (socket says 172.21.15.154:46850) 2023-09-23T03:27:26.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.564+0000 7f43d25be640 1 -- 172.21.15.154:0/1011817473 learned_addr learned my addr 172.21.15.154:0/1011817473 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:26.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.565+0000 7f43d1dbd640 1 --2- 172.21.15.154:0/1011817473 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43cc06c470 0x7f43cc148750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:26.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.565+0000 7f43d15bc640 1 --2- 172.21.15.154:0/1011817473 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43cc0c88f0 0x7f43cc15a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:26.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.565+0000 7f43d25be640 1 -- 172.21.15.154:0/1011817473 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43cc0c88f0 msgr2=0x7f43cc15a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:26.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.565+0000 7f43d25be640 1 --2- 172.21.15.154:0/1011817473 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43cc0c88f0 0x7f43cc15a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:26.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.565+0000 7f43d25be640 1 -- 172.21.15.154:0/1011817473 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43cc06c470 msgr2=0x7f43cc148750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:26.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.565+0000 7f43d25be640 1 --2- 172.21.15.154:0/1011817473 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43cc06c470 0x7f43cc148750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:26.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.565+0000 7f43d25be640 1 -- 172.21.15.154:0/1011817473 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c4009b30 con 0x7f43cc159b50 2023-09-23T03:27:26.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.565+0000 7f43d15bc640 1 --2- 172.21.15.154:0/1011817473 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43cc0c88f0 0x7f43cc15a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:26.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.565+0000 7f43d25be640 1 --2- 172.21.15.154:0/1011817473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43cc159b50 0x7f43cc0c75a0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f43c4036040 tx=0x7f43c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:26.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.565+0000 7f43c27fc640 1 -- 172.21.15.154:0/1011817473 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c4009510 con 0x7f43cc159b50 2023-09-23T03:27:26.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.565+0000 7f43c27fc640 1 -- 172.21.15.154:0/1011817473 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c403e450 con 0x7f43cc159b50 2023-09-23T03:27:26.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.565+0000 7f43c27fc640 1 -- 172.21.15.154:0/1011817473 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c403e800 con 0x7f43cc159b50 2023-09-23T03:27:26.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.565+0000 7f43d2dbf640 1 -- 172.21.15.154:0/1011817473 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43cc15a180 con 0x7f43cc159b50 2023-09-23T03:27:26.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.566+0000 7f43d2dbf640 1 -- 172.21.15.154:0/1011817473 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43cc15aec0 con 0x7f43cc159b50 2023-09-23T03:27:26.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.567+0000 7f43d2dbf640 1 -- 172.21.15.154:0/1011817473 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43cc0736b0 con 0x7f43cc159b50 2023-09-23T03:27:26.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.568+0000 7f43c27fc640 1 -- 172.21.15.154:0/1011817473 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f43c402cd50 con 0x7f43cc159b50 2023-09-23T03:27:26.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.569+0000 7f43c27fc640 1 --2- 172.21.15.154:0/1011817473 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f43b006af30 0x7f43b006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:26.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.569+0000 7f43c27fc640 1 -- 172.21.15.154:0/1011817473 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f43c40ac710 con 0x7f43cc159b50 2023-09-23T03:27:26.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.569+0000 7f43d1dbd640 1 --2- 172.21.15.154:0/1011817473 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f43b006af30 0x7f43b006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:26.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.570+0000 7f43d1dbd640 1 --2- 172.21.15.154:0/1011817473 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f43b006af30 0x7f43b006d3f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f43c8006fd0 tx=0x7f43c8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:26.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.573+0000 7f43c27fc640 1 -- 172.21.15.154:0/1011817473 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f43c4077360 con 0x7f43cc159b50 2023-09-23T03:27:26.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.766+0000 7f43d2dbf640 1 -- 172.21.15.154:0/1011817473 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f43cc15a640 con 0x7f43cc159b50 2023-09-23T03:27:26.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.767+0000 7f43c27fc640 1 -- 172.21.15.154:0/1011817473 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v114) v1 ==== 93+0+10 (secure 0 0 0) 0x7f43c403e960 con 0x7f43cc159b50 2023-09-23T03:27:26.769 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:27:26.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.770+0000 7f43d2dbf640 1 -- 172.21.15.154:0/1011817473 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f43b006af30 msgr2=0x7f43b006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:26.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.770+0000 7f43d2dbf640 1 --2- 172.21.15.154:0/1011817473 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f43b006af30 0x7f43b006d3f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f43c8006fd0 tx=0x7f43c8008040 comp rx=0 tx=0).stop 2023-09-23T03:27:26.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.770+0000 7f43d2dbf640 1 -- 172.21.15.154:0/1011817473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43cc159b50 msgr2=0x7f43cc0c75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:26.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.770+0000 7f43d2dbf640 1 --2- 172.21.15.154:0/1011817473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43cc159b50 0x7f43cc0c75a0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f43c4036040 tx=0x7f43c4004a40 comp rx=0 tx=0).stop 2023-09-23T03:27:26.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.770+0000 7f43d2dbf640 1 -- 172.21.15.154:0/1011817473 shutdown_connections 2023-09-23T03:27:26.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.770+0000 7f43d2dbf640 1 --2- 172.21.15.154:0/1011817473 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f43b006af30 0x7f43b006d3f0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:26.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.770+0000 7f43d2dbf640 1 --2- 172.21.15.154:0/1011817473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f43cc159b50 0x7f43cc0c75a0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:26.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.770+0000 7f43d2dbf640 1 --2- 172.21.15.154:0/1011817473 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f43cc0c88f0 0x7f43cc15a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:26.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.770+0000 7f43d2dbf640 1 --2- 172.21.15.154:0/1011817473 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f43cc06c470 0x7f43cc148750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:26.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.770+0000 7f43d2dbf640 1 -- 172.21.15.154:0/1011817473 >> 172.21.15.154:0/1011817473 conn(0x7f43cc06a300 msgr2=0x7f43cc0aca60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:26.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.770+0000 7f43d2dbf640 1 -- 172.21.15.154:0/1011817473 shutdown_connections 2023-09-23T03:27:26.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:26.770+0000 7f43d2dbf640 1 -- 172.21.15.154:0/1011817473 wait complete. 2023-09-23T03:27:26.791 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:27:27.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.010+0000 7fd387215640 1 Processor -- start 2023-09-23T03:27:27.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.010+0000 7fd387215640 1 -- start start 2023-09-23T03:27:27.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.012+0000 7fd387215640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd38015cb90 0x7fd380160ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:27.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.012+0000 7fd387215640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd380071ea0 con 0x7fd3800caea0 2023-09-23T03:27:27.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.012+0000 7fd387215640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd38015cf70 con 0x7fd380069cd0 2023-09-23T03:27:27.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.012+0000 7fd387215640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3800cd320 con 0x7fd38015cb90 2023-09-23T03:27:27.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.012+0000 7fd384fb1640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd38015cb90 0x7fd380160ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:27.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.012+0000 7fd384fb1640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd38015cb90 0x7fd380160ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46858/0 (socket says 172.21.15.154:46858) 2023-09-23T03:27:27.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.012+0000 7fd384fb1640 1 -- 172.21.15.154:0/3365038604 learned_addr learned my addr 172.21.15.154:0/3365038604 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:27.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.013+0000 7fd384fb1640 1 -- 172.21.15.154:0/3365038604 >> v1:172.21.15.190:6789/0 conn(0x7fd380069cd0 legacy=0x7fd380071080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:27.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.013+0000 7fd384fb1640 1 -- 172.21.15.154:0/3365038604 >> v1:172.21.15.154:6789/0 conn(0x7fd3800caea0 legacy=0x7fd380071790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:27.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.013+0000 7fd37f7fe640 1 -- 172.21.15.154:0/3365038604 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1403172442 0 0) 0x7fd380071ea0 con 0x7fd3800caea0 2023-09-23T03:27:27.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.013+0000 7fd384fb1640 1 -- 172.21.15.154:0/3365038604 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3800cd600 con 0x7fd38015cb90 2023-09-23T03:27:27.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.014+0000 7fd384fb1640 1 --2- 172.21.15.154:0/3365038604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd38015cb90 0x7fd380160ee0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fd370009e50 tx=0x7fd37002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fde7e0a40cfef3a2 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:27.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.014+0000 7fd37f7fe640 1 -- 172.21.15.154:0/3365038604 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37002cb20 con 0x7fd38015cb90 2023-09-23T03:27:27.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.014+0000 7fd37f7fe640 1 -- 172.21.15.154:0/3365038604 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37002cc80 con 0x7fd38015cb90 2023-09-23T03:27:27.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.014+0000 7fd37f7fe640 1 -- 172.21.15.154:0/3365038604 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3700343f0 con 0x7fd38015cb90 2023-09-23T03:27:27.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.015+0000 7fd387215640 1 -- 172.21.15.154:0/3365038604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd38015cb90 msgr2=0x7fd380160ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:27.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.015+0000 7fd387215640 1 --2- 172.21.15.154:0/3365038604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd38015cb90 0x7fd380160ee0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fd370009e50 tx=0x7fd37002bf80 comp rx=0 tx=0).stop 2023-09-23T03:27:27.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.015+0000 7fd387215640 1 -- 172.21.15.154:0/3365038604 shutdown_connections 2023-09-23T03:27:27.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.015+0000 7fd387215640 1 --2- 172.21.15.154:0/3365038604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd38015cb90 0x7fd380160ee0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:27.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.015+0000 7fd387215640 1 -- 172.21.15.154:0/3365038604 >> 172.21.15.154:0/3365038604 conn(0x7fd38006d120 msgr2=0x7fd38006d500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:27.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.015+0000 7fd387215640 1 -- 172.21.15.154:0/3365038604 shutdown_connections 2023-09-23T03:27:27.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.015+0000 7fd387215640 1 -- 172.21.15.154:0/3365038604 wait complete. 2023-09-23T03:27:27.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.017+0000 7fd387215640 1 Processor -- start 2023-09-23T03:27:27.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.017+0000 7fd387215640 1 -- start start 2023-09-23T03:27:27.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.018+0000 7fd387215640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd380069cd0 0x7fd3801f1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:27.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.018+0000 7fd387215640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3800caea0 0x7fd3800c5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:27.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.018+0000 7fd387215640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd38015cb90 0x7fd3800c5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:27.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.018+0000 7fd387215640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3801604a0 con 0x7fd38015cb90 2023-09-23T03:27:27.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.018+0000 7fd387215640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38015fee0 con 0x7fd380069cd0 2023-09-23T03:27:27.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.018+0000 7fd387215640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3801601c0 con 0x7fd3800caea0 2023-09-23T03:27:27.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.018+0000 7fd3857b2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd38015cb90 0x7fd3800c5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:27.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.018+0000 7fd3857b2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd38015cb90 0x7fd3800c5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46870/0 (socket says 172.21.15.154:46870) 2023-09-23T03:27:27.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.018+0000 7fd3857b2640 1 -- 172.21.15.154:0/2759318004 learned_addr learned my addr 172.21.15.154:0/2759318004 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:27.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.019+0000 7fd384fb1640 1 --2- 172.21.15.154:0/2759318004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd380069cd0 0x7fd3801f1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:27.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.019+0000 7fd3857b2640 1 -- 172.21.15.154:0/2759318004 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3800caea0 msgr2=0x7fd3800c5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:27.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.019+0000 7fd3857b2640 1 --2- 172.21.15.154:0/2759318004 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3800caea0 0x7fd3800c5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:27.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.019+0000 7fd3857b2640 1 -- 172.21.15.154:0/2759318004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd380069cd0 msgr2=0x7fd3801f1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:27.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.019+0000 7fd3857b2640 1 --2- 172.21.15.154:0/2759318004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd380069cd0 0x7fd3801f1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:27.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.019+0000 7fd3857b2640 1 -- 172.21.15.154:0/2759318004 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd370009b30 con 0x7fd38015cb90 2023-09-23T03:27:27.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.019+0000 7fd3857b2640 1 --2- 172.21.15.154:0/2759318004 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd38015cb90 0x7fd3800c5840 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fd37800c3e0 tx=0x7fd37800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:27.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.019+0000 7fd37d7fa640 1 -- 172.21.15.154:0/2759318004 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3780035f0 con 0x7fd38015cb90 2023-09-23T03:27:27.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.019+0000 7fd37d7fa640 1 -- 172.21.15.154:0/2759318004 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd378003750 con 0x7fd38015cb90 2023-09-23T03:27:27.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.019+0000 7fd37d7fa640 1 -- 172.21.15.154:0/2759318004 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd378003b00 con 0x7fd38015cb90 2023-09-23T03:27:27.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.019+0000 7fd387215640 1 -- 172.21.15.154:0/2759318004 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd380145950 con 0x7fd38015cb90 2023-09-23T03:27:27.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.020+0000 7fd387215640 1 -- 172.21.15.154:0/2759318004 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd38015f1b0 con 0x7fd38015cb90 2023-09-23T03:27:27.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.021+0000 7fd387215640 1 -- 172.21.15.154:0/2759318004 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3801457b0 con 0x7fd38015cb90 2023-09-23T03:27:27.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.022+0000 7fd37d7fa640 1 -- 172.21.15.154:0/2759318004 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fd378003c60 con 0x7fd38015cb90 2023-09-23T03:27:27.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.023+0000 7fd37d7fa640 1 --2- 172.21.15.154:0/2759318004 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd36006af30 0x7fd36006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:27.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.023+0000 7fd37d7fa640 1 -- 172.21.15.154:0/2759318004 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fd3780890a0 con 0x7fd38015cb90 2023-09-23T03:27:27.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.023+0000 7fd384fb1640 1 --2- 172.21.15.154:0/2759318004 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd36006af30 0x7fd36006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:27.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.024+0000 7fd384fb1640 1 --2- 172.21.15.154:0/2759318004 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd36006af30 0x7fd36006d3f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fd370035000 tx=0x7fd370030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:27.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.027+0000 7fd37d7fa640 1 -- 172.21.15.154:0/2759318004 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fd3801457b0 con 0x7fd38015cb90 2023-09-23T03:27:27.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.215+0000 7fd387215640 1 -- 172.21.15.154:0/2759318004 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd380145e70 con 0x7fd38015cb90 2023-09-23T03:27:27.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.216+0000 7fd37d7fa640 1 -- 172.21.15.154:0/2759318004 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v114) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd380145e70 con 0x7fd38015cb90 2023-09-23T03:27:27.217 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:27:27.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.218+0000 7fd387215640 1 -- 172.21.15.154:0/2759318004 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd36006af30 msgr2=0x7fd36006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:27.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.218+0000 7fd387215640 1 --2- 172.21.15.154:0/2759318004 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd36006af30 0x7fd36006d3f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fd370035000 tx=0x7fd370030000 comp rx=0 tx=0).stop 2023-09-23T03:27:27.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.218+0000 7fd387215640 1 -- 172.21.15.154:0/2759318004 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd38015cb90 msgr2=0x7fd3800c5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:27.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.218+0000 7fd387215640 1 --2- 172.21.15.154:0/2759318004 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd38015cb90 0x7fd3800c5840 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fd37800c3e0 tx=0x7fd37800c8a0 comp rx=0 tx=0).stop 2023-09-23T03:27:27.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.218+0000 7fd387215640 1 -- 172.21.15.154:0/2759318004 shutdown_connections 2023-09-23T03:27:27.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.218+0000 7fd387215640 1 --2- 172.21.15.154:0/2759318004 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd36006af30 0x7fd36006d3f0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:27.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.218+0000 7fd387215640 1 --2- 172.21.15.154:0/2759318004 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd38015cb90 0x7fd3800c5840 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:27.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.218+0000 7fd387215640 1 --2- 172.21.15.154:0/2759318004 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd3800caea0 0x7fd3800c5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:27.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.218+0000 7fd387215640 1 --2- 172.21.15.154:0/2759318004 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd380069cd0 0x7fd3801f1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:27.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.218+0000 7fd387215640 1 -- 172.21.15.154:0/2759318004 >> 172.21.15.154:0/2759318004 conn(0x7fd38006d120 msgr2=0x7fd380070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:27.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.219+0000 7fd387215640 1 -- 172.21.15.154:0/2759318004 shutdown_connections 2023-09-23T03:27:27.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.219+0000 7fd387215640 1 -- 172.21.15.154:0/2759318004 wait complete. 2023-09-23T03:27:27.239 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:27:27.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.451+0000 7f45c46a9640 1 Processor -- start 2023-09-23T03:27:27.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.452+0000 7f45c46a9640 1 -- start start 2023-09-23T03:27:27.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.453+0000 7f45c46a9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45bc0c8d70 0x7f45bc15c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:27.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.453+0000 7f45c46a9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f45bc0c9150 0x7f45bc15cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:27.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.453+0000 7f45c46a9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f45bc071290 0x7f45bc0ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:27.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.453+0000 7f45c46a9640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45bc0c9d20 con 0x7f45bc071290 2023-09-23T03:27:27.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.453+0000 7f45c46a9640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45bc071850 con 0x7f45bc0c9150 2023-09-23T03:27:27.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.453+0000 7f45c46a9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc071b30 con 0x7f45bc0c8d70 2023-09-23T03:27:27.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.454+0000 7f45c2c46640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f45bc071290 0x7f45bc0ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:27.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.454+0000 7f45c2c46640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f45bc071290 0x7f45bc0ca1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46874/0 (socket says 172.21.15.154:46874) 2023-09-23T03:27:27.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.454+0000 7f45c2c46640 1 -- 172.21.15.154:0/4285400618 learned_addr learned my addr 172.21.15.154:0/4285400618 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:27.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.454+0000 7f45c2445640 1 --2- 172.21.15.154:0/4285400618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45bc0c8d70 0x7f45bc15c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:27.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.454+0000 7f45c1c44640 1 --2- 172.21.15.154:0/4285400618 >> v2:172.21.15.190:3300/0 conn(0x7f45bc0c9150 0x7f45bc15cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:27.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.454+0000 7f45c2c46640 1 -- 172.21.15.154:0/4285400618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45bc0c8d70 msgr2=0x7f45bc15c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:27.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.454+0000 7f45c2c46640 1 --2- 172.21.15.154:0/4285400618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45bc0c8d70 0x7f45bc15c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:27.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.454+0000 7f45c2c46640 1 -- 172.21.15.154:0/4285400618 >> v2:172.21.15.190:3300/0 conn(0x7f45bc0c9150 msgr2=0x7f45bc15cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:27.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.454+0000 7f45c2c46640 1 --2- 172.21.15.154:0/4285400618 >> v2:172.21.15.190:3300/0 conn(0x7f45bc0c9150 0x7f45bc15cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:27.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.454+0000 7f45c2c46640 1 -- 172.21.15.154:0/4285400618 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45bc071e10 con 0x7f45bc071290 2023-09-23T03:27:27.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.456+0000 7f45c2c46640 1 --2- 172.21.15.154:0/4285400618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45bc071290 0x7f45bc0ca1b0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f45bc06b7c0 tx=0x7f45b402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2aaa12555cb37bd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:27.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.456+0000 7f45c1443640 1 -- 172.21.15.154:0/4285400618 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b402c9e0 con 0x7f45bc071290 2023-09-23T03:27:27.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.456+0000 7f45c1443640 1 -- 172.21.15.154:0/4285400618 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b402cb40 con 0x7f45bc071290 2023-09-23T03:27:27.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.456+0000 7f45c1443640 1 -- 172.21.15.154:0/4285400618 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b40343f0 con 0x7f45bc071290 2023-09-23T03:27:27.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.457+0000 7f45c46a9640 1 -- 172.21.15.154:0/4285400618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45bc071290 msgr2=0x7f45bc0ca1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:27.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.457+0000 7f45c46a9640 1 --2- 172.21.15.154:0/4285400618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45bc071290 0x7f45bc0ca1b0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f45bc06b7c0 tx=0x7f45b402be40 comp rx=0 tx=0).stop 2023-09-23T03:27:27.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.457+0000 7f45c46a9640 1 -- 172.21.15.154:0/4285400618 shutdown_connections 2023-09-23T03:27:27.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.457+0000 7f45c46a9640 1 --2- 172.21.15.154:0/4285400618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45bc071290 0x7f45bc0ca1b0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:27.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.457+0000 7f45c46a9640 1 --2- 172.21.15.154:0/4285400618 >> v2:172.21.15.190:3300/0 conn(0x7f45bc0c9150 0x7f45bc15cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:27.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.457+0000 7f45c46a9640 1 --2- 172.21.15.154:0/4285400618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45bc0c8d70 0x7f45bc15c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:27.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.457+0000 7f45c46a9640 1 -- 172.21.15.154:0/4285400618 >> 172.21.15.154:0/4285400618 conn(0x7f45bc06c010 msgr2=0x7f45bc06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:27.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.457+0000 7f45c46a9640 1 -- 172.21.15.154:0/4285400618 shutdown_connections 2023-09-23T03:27:27.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.457+0000 7f45c46a9640 1 -- 172.21.15.154:0/4285400618 wait complete. 2023-09-23T03:27:27.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.459+0000 7f45c46a9640 1 Processor -- start 2023-09-23T03:27:27.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.459+0000 7f45c46a9640 1 -- start start 2023-09-23T03:27:27.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.460+0000 7f45c46a9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45bc071290 0x7f45bc0c4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:27.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.460+0000 7f45c46a9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45bc0c8d70 0x7f45bc14dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:27.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.460+0000 7f45c46a9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45bc0c9150 0x7f45bc0c5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:27.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.460+0000 7f45c46a9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc157cc0 con 0x7f45bc0c8d70 2023-09-23T03:27:27.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.460+0000 7f45c46a9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc157fa0 con 0x7f45bc0c9150 2023-09-23T03:27:27.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.460+0000 7f45c46a9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45bc158280 con 0x7f45bc071290 2023-09-23T03:27:27.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.460+0000 7f45c1c44640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45bc0c8d70 0x7f45bc14dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:27.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.460+0000 7f45c1c44640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45bc0c8d70 0x7f45bc14dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46878/0 (socket says 172.21.15.154:46878) 2023-09-23T03:27:27.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.460+0000 7f45c1c44640 1 -- 172.21.15.154:0/3026764931 learned_addr learned my addr 172.21.15.154:0/3026764931 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:27.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.461+0000 7f45c1c44640 1 -- 172.21.15.154:0/3026764931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45bc071290 msgr2=0x7f45bc0c4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:27.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.461+0000 7f45c1c44640 1 --2- 172.21.15.154:0/3026764931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45bc071290 0x7f45bc0c4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:27.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.461+0000 7f45c1c44640 1 -- 172.21.15.154:0/3026764931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45bc0c9150 msgr2=0x7f45bc0c5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:27.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.461+0000 7f45c2c46640 1 --2- 172.21.15.154:0/3026764931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45bc0c9150 0x7f45bc0c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:27.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.461+0000 7f45c1c44640 1 --2- 172.21.15.154:0/3026764931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45bc0c9150 0x7f45bc0c5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:27.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.461+0000 7f45c1c44640 1 -- 172.21.15.154:0/3026764931 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b4009b30 con 0x7f45bc0c8d70 2023-09-23T03:27:27.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.461+0000 7f45c1c44640 1 --2- 172.21.15.154:0/3026764931 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45bc0c8d70 0x7f45bc14dad0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f45b000d800 tx=0x7f45b000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:27.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.461+0000 7f45aeffd640 1 -- 172.21.15.154:0/3026764931 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b000b6c0 con 0x7f45bc0c8d70 2023-09-23T03:27:27.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.461+0000 7f45aeffd640 1 -- 172.21.15.154:0/3026764931 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b000b820 con 0x7f45bc0c8d70 2023-09-23T03:27:27.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.461+0000 7f45c46a9640 1 -- 172.21.15.154:0/3026764931 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45bc158560 con 0x7f45bc0c8d70 2023-09-23T03:27:27.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.461+0000 7f45aeffd640 1 -- 172.21.15.154:0/3026764931 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b000bbb0 con 0x7f45bc0c8d70 2023-09-23T03:27:27.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.461+0000 7f45c46a9640 1 -- 172.21.15.154:0/3026764931 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45bc14e010 con 0x7f45bc0c8d70 2023-09-23T03:27:27.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.462+0000 7f45c46a9640 1 -- 172.21.15.154:0/3026764931 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45bc04e920 con 0x7f45bc0c8d70 2023-09-23T03:27:27.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.464+0000 7f45aeffd640 1 -- 172.21.15.154:0/3026764931 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f45b000bd10 con 0x7f45bc0c8d70 2023-09-23T03:27:27.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.464+0000 7f45aeffd640 1 --2- 172.21.15.154:0/3026764931 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f45a006af30 0x7f45a006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:27.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.465+0000 7f45aeffd640 1 -- 172.21.15.154:0/3026764931 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f45b0089ff0 con 0x7f45bc0c8d70 2023-09-23T03:27:27.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.465+0000 7f45c2445640 1 --2- 172.21.15.154:0/3026764931 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f45a006af30 0x7f45a006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:27.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.465+0000 7f45c2445640 1 --2- 172.21.15.154:0/3026764931 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f45a006af30 0x7f45a006d3f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f45b800b7f0 tx=0x7f45b8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:27.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.469+0000 7f45aeffd640 1 -- 172.21.15.154:0/3026764931 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f45b0054ce0 con 0x7f45bc0c8d70 2023-09-23T03:27:27.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.659+0000 7f45c46a9640 1 -- 172.21.15.154:0/3026764931 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f45bc06cbc0 con 0x7f45bc0c8d70 2023-09-23T03:27:27.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.660+0000 7f45aeffd640 1 -- 172.21.15.154:0/3026764931 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v114) v1 ==== 100+0+11 (secure 0 0 0) 0x7f45b0058300 con 0x7f45bc0c8d70 2023-09-23T03:27:27.661 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:27:27.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.663+0000 7f45c46a9640 1 -- 172.21.15.154:0/3026764931 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f45a006af30 msgr2=0x7f45a006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:27.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.663+0000 7f45c46a9640 1 --2- 172.21.15.154:0/3026764931 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f45a006af30 0x7f45a006d3f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f45b800b7f0 tx=0x7f45b8008040 comp rx=0 tx=0).stop 2023-09-23T03:27:27.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.663+0000 7f45c46a9640 1 -- 172.21.15.154:0/3026764931 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45bc0c8d70 msgr2=0x7f45bc14dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:27.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.663+0000 7f45c46a9640 1 --2- 172.21.15.154:0/3026764931 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45bc0c8d70 0x7f45bc14dad0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f45b000d800 tx=0x7f45b000dcc0 comp rx=0 tx=0).stop 2023-09-23T03:27:27.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.663+0000 7f45c46a9640 1 -- 172.21.15.154:0/3026764931 shutdown_connections 2023-09-23T03:27:27.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.663+0000 7f45c46a9640 1 --2- 172.21.15.154:0/3026764931 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f45a006af30 0x7f45a006d3f0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:27.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.663+0000 7f45c46a9640 1 --2- 172.21.15.154:0/3026764931 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f45bc0c9150 0x7f45bc0c5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:27.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.663+0000 7f45c46a9640 1 --2- 172.21.15.154:0/3026764931 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f45bc0c8d70 0x7f45bc14dad0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:27.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.663+0000 7f45c46a9640 1 --2- 172.21.15.154:0/3026764931 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f45bc071290 0x7f45bc0c4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:27.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.663+0000 7f45c46a9640 1 -- 172.21.15.154:0/3026764931 >> 172.21.15.154:0/3026764931 conn(0x7f45bc06c010 msgr2=0x7f45bc15bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:27.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.663+0000 7f45c46a9640 1 -- 172.21.15.154:0/3026764931 shutdown_connections 2023-09-23T03:27:27.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.663+0000 7f45c46a9640 1 -- 172.21.15.154:0/3026764931 wait complete. 2023-09-23T03:27:27.682 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-09-23T03:27:27.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.893+0000 7ffb8453c640 1 Processor -- start 2023-09-23T03:27:27.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.894+0000 7ffb8453c640 1 -- start start 2023-09-23T03:27:27.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.895+0000 7ffb8453c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ffb7c15bb30 0x7ffb7c0caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:27.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.895+0000 7ffb8453c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb7c0cb3e0 0x7ffb7c0cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:27.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.895+0000 7ffb8453c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb7c15b6a0 con 0x7ffb7c15aba0 2023-09-23T03:27:27.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.895+0000 7ffb8453c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb7c0cb870 con 0x7ffb7c15bb30 2023-09-23T03:27:27.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.895+0000 7ffb8453c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7c15bf70 con 0x7ffb7c0cb3e0 2023-09-23T03:27:27.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.895+0000 7ffb81ad7640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ffb7c15bb30 0x7ffb7c0caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:27.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.895+0000 7ffb81ad7640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ffb7c15bb30 0x7ffb7c0caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46884/0 (socket says 172.21.15.154:46884) 2023-09-23T03:27:27.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.895+0000 7ffb81ad7640 1 -- 172.21.15.154:0/3025649427 learned_addr learned my addr 172.21.15.154:0/3025649427 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:27.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.896+0000 7ffb82ad9640 1 --2- 172.21.15.154:0/3025649427 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb7c0cb3e0 0x7ffb7c0cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:27.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.896+0000 7ffb81ad7640 1 -- 172.21.15.154:0/3025649427 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb7c0cb3e0 msgr2=0x7ffb7c0cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:27.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.896+0000 7ffb81ad7640 1 --2- 172.21.15.154:0/3025649427 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb7c0cb3e0 0x7ffb7c0cc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:27.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.896+0000 7ffb81ad7640 1 -- 172.21.15.154:0/3025649427 >> v1:172.21.15.154:6789/0 conn(0x7ffb7c15aba0 legacy=0x7ffb7c15e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:27.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.896+0000 7ffb81ad7640 1 -- 172.21.15.154:0/3025649427 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb7c15f0b0 con 0x7ffb7c15bb30 2023-09-23T03:27:27.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.896+0000 7ffb82ad9640 1 --2- 172.21.15.154:0/3025649427 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb7c0cb3e0 0x7ffb7c0cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:27.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.897+0000 7ffb81ad7640 1 --2- 172.21.15.154:0/3025649427 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb7c15bb30 0x7ffb7c0caea0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c009870 tx=0x7ffb6c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e16efd7cf5f39bb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:27.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.897+0000 7ffb812d6640 1 -- 172.21.15.154:0/3025649427 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c030840 con 0x7ffb7c15bb30 2023-09-23T03:27:27.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.897+0000 7ffb812d6640 1 -- 172.21.15.154:0/3025649427 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb6c0309a0 con 0x7ffb7c15bb30 2023-09-23T03:27:27.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.897+0000 7ffb812d6640 1 -- 172.21.15.154:0/3025649427 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c030d30 con 0x7ffb7c15bb30 2023-09-23T03:27:27.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.898+0000 7ffb8453c640 1 -- 172.21.15.154:0/3025649427 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb7c15bb30 msgr2=0x7ffb7c0caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:27.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.898+0000 7ffb8453c640 1 --2- 172.21.15.154:0/3025649427 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb7c15bb30 0x7ffb7c0caea0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c009870 tx=0x7ffb6c02b980 comp rx=0 tx=0).stop 2023-09-23T03:27:27.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.898+0000 7ffb8453c640 1 -- 172.21.15.154:0/3025649427 shutdown_connections 2023-09-23T03:27:27.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.898+0000 7ffb8453c640 1 --2- 172.21.15.154:0/3025649427 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb7c0cb3e0 0x7ffb7c0cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:27.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.898+0000 7ffb8453c640 1 --2- 172.21.15.154:0/3025649427 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb7c15bb30 0x7ffb7c0caea0 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:27.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.898+0000 7ffb8453c640 1 -- 172.21.15.154:0/3025649427 >> 172.21.15.154:0/3025649427 conn(0x7ffb7c06c010 msgr2=0x7ffb7c06c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:27.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.898+0000 7ffb8453c640 1 -- 172.21.15.154:0/3025649427 shutdown_connections 2023-09-23T03:27:27.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.898+0000 7ffb8453c640 1 -- 172.21.15.154:0/3025649427 wait complete. 2023-09-23T03:27:27.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.900+0000 7ffb8453c640 1 Processor -- start 2023-09-23T03:27:27.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.900+0000 7ffb8453c640 1 -- start start 2023-09-23T03:27:27.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.901+0000 7ffb8453c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb7c0cb3e0 0x7ffb7c0be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:27.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.901+0000 7ffb8453c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb7c15aba0 0x7ffb7c159510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:27.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.901+0000 7ffb8453c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb7c15bb30 0x7ffb7c158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:27.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.901+0000 7ffb8453c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7c0a74b0 con 0x7ffb7c15aba0 2023-09-23T03:27:27.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.901+0000 7ffb8453c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7c1592c0 con 0x7ffb7c0cb3e0 2023-09-23T03:27:27.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.901+0000 7ffb8453c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7c0befd0 con 0x7ffb7c15bb30 2023-09-23T03:27:27.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.901+0000 7ffb81ad7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb7c15aba0 0x7ffb7c159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:27.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.901+0000 7ffb81ad7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb7c15aba0 0x7ffb7c159510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46892/0 (socket says 172.21.15.154:46892) 2023-09-23T03:27:27.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.901+0000 7ffb81ad7640 1 -- 172.21.15.154:0/3969227127 learned_addr learned my addr 172.21.15.154:0/3969227127 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:27.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.901+0000 7ffb82ad9640 1 --2- 172.21.15.154:0/3969227127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb7c15bb30 0x7ffb7c158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:27.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.901+0000 7ffb81ad7640 1 -- 172.21.15.154:0/3969227127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb7c15bb30 msgr2=0x7ffb7c158d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:27.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.901+0000 7ffb81ad7640 1 --2- 172.21.15.154:0/3969227127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb7c15bb30 0x7ffb7c158d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:27.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.901+0000 7ffb81ad7640 1 -- 172.21.15.154:0/3969227127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb7c0cb3e0 msgr2=0x7ffb7c0be9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:27.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.901+0000 7ffb822d8640 1 --2- 172.21.15.154:0/3969227127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb7c0cb3e0 0x7ffb7c0be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:27.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.902+0000 7ffb81ad7640 1 --2- 172.21.15.154:0/3969227127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb7c0cb3e0 0x7ffb7c0be9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:27.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.902+0000 7ffb81ad7640 1 -- 172.21.15.154:0/3969227127 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb6c0094d0 con 0x7ffb7c15aba0 2023-09-23T03:27:27.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.902+0000 7ffb81ad7640 1 --2- 172.21.15.154:0/3969227127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb7c15aba0 0x7ffb7c159510 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c02b9c0 tx=0x7ffb6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:27.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.902+0000 7ffb72ffd640 1 -- 172.21.15.154:0/3969227127 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c03e070 con 0x7ffb7c15aba0 2023-09-23T03:27:27.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.902+0000 7ffb72ffd640 1 -- 172.21.15.154:0/3969227127 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb6c030810 con 0x7ffb7c15aba0 2023-09-23T03:27:27.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.902+0000 7ffb72ffd640 1 -- 172.21.15.154:0/3969227127 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c0423f0 con 0x7ffb7c15aba0 2023-09-23T03:27:27.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.902+0000 7ffb8453c640 1 -- 172.21.15.154:0/3969227127 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb7c0bf2b0 con 0x7ffb7c15aba0 2023-09-23T03:27:27.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.902+0000 7ffb8453c640 1 -- 172.21.15.154:0/3969227127 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb7c0bf870 con 0x7ffb7c15aba0 2023-09-23T03:27:27.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.904+0000 7ffb8453c640 1 -- 172.21.15.154:0/3969227127 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb7c0bfa60 con 0x7ffb7c15aba0 2023-09-23T03:27:27.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.904+0000 7ffb72ffd640 1 -- 172.21.15.154:0/3969227127 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7ffb6c042550 con 0x7ffb7c15aba0 2023-09-23T03:27:27.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.905+0000 7ffb72ffd640 1 --2- 172.21.15.154:0/3969227127 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffb6006af30 0x7ffb6006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:27.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.905+0000 7ffb72ffd640 1 -- 172.21.15.154:0/3969227127 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6691+0+0 (secure 0 0 0) 0x7ffb6c0ac350 con 0x7ffb7c15aba0 2023-09-23T03:27:27.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.905+0000 7ffb822d8640 1 --2- 172.21.15.154:0/3969227127 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffb6006af30 0x7ffb6006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:27.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.906+0000 7ffb822d8640 1 --2- 172.21.15.154:0/3969227127 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffb6006af30 0x7ffb6006d3f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0c5260 tx=0x7ffb78009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:27.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:27.909+0000 7ffb72ffd640 1 -- 172.21.15.154:0/3969227127 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7ffb6c076fa0 con 0x7ffb7c15aba0 2023-09-23T03:27:28.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.099+0000 7ffb8453c640 1 -- 172.21.15.154:0/3969227127 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7ffb7c04e920 con 0x7ffb7c15aba0 2023-09-23T03:27:28.101 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.100+0000 7ffb72ffd640 1 -- 172.21.15.154:0/3969227127 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v114) v1 ==== 106+0+11 (secure 0 0 0) 0x7ffb6c034020 con 0x7ffb7c15aba0 2023-09-23T03:27:28.102 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:27:28.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.103+0000 7ffb8453c640 1 -- 172.21.15.154:0/3969227127 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffb6006af30 msgr2=0x7ffb6006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:28.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.103+0000 7ffb8453c640 1 --2- 172.21.15.154:0/3969227127 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffb6006af30 0x7ffb6006d3f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0c5260 tx=0x7ffb78009040 comp rx=0 tx=0).stop 2023-09-23T03:27:28.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.103+0000 7ffb8453c640 1 -- 172.21.15.154:0/3969227127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb7c15aba0 msgr2=0x7ffb7c159510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:28.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.103+0000 7ffb8453c640 1 --2- 172.21.15.154:0/3969227127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb7c15aba0 0x7ffb7c159510 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c02b9c0 tx=0x7ffb6c0046c0 comp rx=0 tx=0).stop 2023-09-23T03:27:28.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.103+0000 7ffb8453c640 1 -- 172.21.15.154:0/3969227127 shutdown_connections 2023-09-23T03:27:28.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.103+0000 7ffb8453c640 1 --2- 172.21.15.154:0/3969227127 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffb6006af30 0x7ffb6006d3f0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:28.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.103+0000 7ffb8453c640 1 --2- 172.21.15.154:0/3969227127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb7c15bb30 0x7ffb7c158d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:28.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.103+0000 7ffb8453c640 1 --2- 172.21.15.154:0/3969227127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb7c15aba0 0x7ffb7c159510 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:28.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.103+0000 7ffb8453c640 1 --2- 172.21.15.154:0/3969227127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb7c0cb3e0 0x7ffb7c0be9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:28.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.103+0000 7ffb8453c640 1 -- 172.21.15.154:0/3969227127 >> 172.21.15.154:0/3969227127 conn(0x7ffb7c06c010 msgr2=0x7ffb7c15c130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:28.106 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.104+0000 7ffb8453c640 1 -- 172.21.15.154:0/3969227127 shutdown_connections 2023-09-23T03:27:28.106 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.104+0000 7ffb8453c640 1 -- 172.21.15.154:0/3969227127 wait complete. 2023-09-23T03:27:28.124 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:27:28.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.330+0000 7f132bb00640 1 Processor -- start 2023-09-23T03:27:28.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.330+0000 7f132bb00640 1 -- start start 2023-09-23T03:27:28.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.331+0000 7f132bb00640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13240b1b20 0x7f13240ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:28.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.331+0000 7f132bb00640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f13240cee70 0x7f13240ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:28.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.331+0000 7f132bb00640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f132415ac30 con 0x7f13240acce0 2023-09-23T03:27:28.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.331+0000 7f132bb00640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13240b2e90 con 0x7f13240cee70 2023-09-23T03:27:28.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.331+0000 7f132bb00640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13240cf320 con 0x7f13240b1b20 2023-09-23T03:27:28.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.332+0000 7f132909b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13240b1b20 0x7f13240ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:28.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.332+0000 7f132909b640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13240b1b20 0x7f13240ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:49228/0 (socket says 172.21.15.154:49228) 2023-09-23T03:27:28.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.332+0000 7f132909b640 1 -- 172.21.15.154:0/3698452178 learned_addr learned my addr 172.21.15.154:0/3698452178 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:28.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.332+0000 7f132a09d640 1 --2- 172.21.15.154:0/3698452178 >> v2:172.21.15.190:3300/0 conn(0x7f13240cee70 0x7f13240ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:28.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.333+0000 7f132909b640 1 -- 172.21.15.154:0/3698452178 >> v2:172.21.15.190:3300/0 conn(0x7f13240cee70 msgr2=0x7f13240ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:28.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.333+0000 7f132909b640 1 --2- 172.21.15.154:0/3698452178 >> v2:172.21.15.190:3300/0 conn(0x7f13240cee70 0x7f13240ab140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:28.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.333+0000 7f132909b640 1 -- 172.21.15.154:0/3698452178 >> v1:172.21.15.190:6789/0 conn(0x7f13240acce0 legacy=0x7f132415ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:28.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.333+0000 7f132909b640 1 -- 172.21.15.154:0/3698452178 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13240b1f70 con 0x7f13240b1b20 2023-09-23T03:27:28.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.334+0000 7f132909b640 1 --2- 172.21.15.154:0/3698452178 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13240b1b20 0x7f13240ce930 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1318009870 tx=0x7f131802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=178a5b8b9ad7329f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:28.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.334+0000 7f132889a640 1 -- 172.21.15.154:0/3698452178 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1318031640 con 0x7f13240b1b20 2023-09-23T03:27:28.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.334+0000 7f132889a640 1 -- 172.21.15.154:0/3698452178 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13180317a0 con 0x7f13240b1b20 2023-09-23T03:27:28.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.335+0000 7f132889a640 1 -- 172.21.15.154:0/3698452178 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1318031b30 con 0x7f13240b1b20 2023-09-23T03:27:28.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.335+0000 7f132bb00640 1 -- 172.21.15.154:0/3698452178 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13240b1b20 msgr2=0x7f13240ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:28.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.335+0000 7f132bb00640 1 --2- 172.21.15.154:0/3698452178 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13240b1b20 0x7f13240ce930 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1318009870 tx=0x7f131802b8f0 comp rx=0 tx=0).stop 2023-09-23T03:27:28.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.335+0000 7f132bb00640 1 -- 172.21.15.154:0/3698452178 shutdown_connections 2023-09-23T03:27:28.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.335+0000 7f132bb00640 1 --2- 172.21.15.154:0/3698452178 >> v2:172.21.15.190:3300/0 conn(0x7f13240cee70 0x7f13240ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:28.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.335+0000 7f132bb00640 1 --2- 172.21.15.154:0/3698452178 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13240b1b20 0x7f13240ce930 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:28.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.335+0000 7f132bb00640 1 -- 172.21.15.154:0/3698452178 >> 172.21.15.154:0/3698452178 conn(0x7f132406a0f0 msgr2=0x7f132406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:28.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.335+0000 7f132bb00640 1 -- 172.21.15.154:0/3698452178 shutdown_connections 2023-09-23T03:27:28.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.335+0000 7f132bb00640 1 -- 172.21.15.154:0/3698452178 wait complete. 2023-09-23T03:27:28.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.337+0000 7f132bb00640 1 Processor -- start 2023-09-23T03:27:28.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.337+0000 7f132bb00640 1 -- start start 2023-09-23T03:27:28.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.338+0000 7f132bb00640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13240acce0 0x7f132415d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:28.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.338+0000 7f132bb00640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13240b1b20 0x7f132415add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:28.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.338+0000 7f132bb00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13240cee70 0x7f1324159f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:28.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.338+0000 7f132bb00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132415b7b0 con 0x7f13240cee70 2023-09-23T03:27:28.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.338+0000 7f132bb00640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132415ba90 con 0x7f13240acce0 2023-09-23T03:27:28.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.338+0000 7f132bb00640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f132415bd70 con 0x7f13240b1b20 2023-09-23T03:27:28.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.338+0000 7f132a09d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13240cee70 0x7f1324159f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:28.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.338+0000 7f132a09d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13240cee70 0x7f1324159f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46904/0 (socket says 172.21.15.154:46904) 2023-09-23T03:27:28.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.338+0000 7f132a09d640 1 -- 172.21.15.154:0/149039093 learned_addr learned my addr 172.21.15.154:0/149039093 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:28.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.339+0000 7f132989c640 1 --2- 172.21.15.154:0/149039093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13240acce0 0x7f132415d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:28.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.339+0000 7f132909b640 1 --2- 172.21.15.154:0/149039093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13240b1b20 0x7f132415add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:28.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.339+0000 7f132a09d640 1 -- 172.21.15.154:0/149039093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13240b1b20 msgr2=0x7f132415add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:28.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.339+0000 7f132a09d640 1 --2- 172.21.15.154:0/149039093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13240b1b20 0x7f132415add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:28.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.339+0000 7f132a09d640 1 -- 172.21.15.154:0/149039093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13240acce0 msgr2=0x7f132415d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:28.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.339+0000 7f132a09d640 1 --2- 172.21.15.154:0/149039093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13240acce0 0x7f132415d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:28.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.339+0000 7f132a09d640 1 -- 172.21.15.154:0/149039093 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13180094d0 con 0x7f13240cee70 2023-09-23T03:27:28.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.339+0000 7f132a09d640 1 --2- 172.21.15.154:0/149039093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13240cee70 0x7f1324159f30 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f131c00e7f0 tx=0x7f131c00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:28.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.339+0000 7f13227fc640 1 -- 172.21.15.154:0/149039093 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131c009510 con 0x7f13240cee70 2023-09-23T03:27:28.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.339+0000 7f13227fc640 1 -- 172.21.15.154:0/149039093 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f131c00ee40 con 0x7f13240cee70 2023-09-23T03:27:28.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.339+0000 7f132bb00640 1 -- 172.21.15.154:0/149039093 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f132415c050 con 0x7f13240cee70 2023-09-23T03:27:28.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.340+0000 7f13227fc640 1 -- 172.21.15.154:0/149039093 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131c00cb80 con 0x7f13240cee70 2023-09-23T03:27:28.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.340+0000 7f132bb00640 1 -- 172.21.15.154:0/149039093 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f132415c850 con 0x7f13240cee70 2023-09-23T03:27:28.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.341+0000 7f132bb00640 1 -- 172.21.15.154:0/149039093 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f132415c570 con 0x7f13240cee70 2023-09-23T03:27:28.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.342+0000 7f13227fc640 1 -- 172.21.15.154:0/149039093 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f131c00cce0 con 0x7f13240cee70 2023-09-23T03:27:28.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.343+0000 7f13227fc640 1 --2- 172.21.15.154:0/149039093 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f131006af30 0x7f131006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:28.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.343+0000 7f13227fc640 1 -- 172.21.15.154:0/149039093 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f131c089bc0 con 0x7f13240cee70 2023-09-23T03:27:28.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.343+0000 7f132989c640 1 --2- 172.21.15.154:0/149039093 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f131006af30 0x7f131006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:28.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.344+0000 7f132989c640 1 --2- 172.21.15.154:0/149039093 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f131006af30 0x7f131006d3f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f1314001330 tx=0x7f1314009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:28.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.347+0000 7f13227fc640 1 -- 172.21.15.154:0/149039093 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f131c054810 con 0x7f13240cee70 2023-09-23T03:27:28.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.580+0000 7f132bb00640 1 -- 172.21.15.154:0/149039093 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1324073eb0 con 0x7f13240cee70 2023-09-23T03:27:28.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.581+0000 7f13227fc640 1 -- 172.21.15.154:0/149039093 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4926 (secure 0 0 0) 0x7f131c017090 con 0x7f13240cee70 2023-09-23T03:27:28.583 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:28.583 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":99,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5753,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/60017546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":60017546},{"type":"v1","addr":"172.21.15.190:6833","nonce":60017546}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95},{"gid":6415,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2708068523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2708068523},{"type":"v1","addr":"172.21.15.190:6835","nonce":2708068523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":99},{"gid":8592,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/1846547753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1846547753},{"type":"v1","addr":"172.21.15.154:6836","nonce":1846547753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:26:26.774375+0000","modified":"2023-09-23T03:26:36.432946+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8595":{"gid":8595,"name":"a","rank":0,"incarnation":95,"state":"up:active","state_seq":4,"addr":"172.21.15.154:6837/1758434763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1758434763},{"type":"v1","addr":"172.21.15.154:6837","nonce":1758434763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15,16],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-09-23T03:27:28.583 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 99 2023-09-23T03:27:28.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.584+0000 7f132bb00640 1 -- 172.21.15.154:0/149039093 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f131006af30 msgr2=0x7f131006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:28.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.584+0000 7f132bb00640 1 --2- 172.21.15.154:0/149039093 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f131006af30 0x7f131006d3f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f1314001330 tx=0x7f1314009040 comp rx=0 tx=0).stop 2023-09-23T03:27:28.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.584+0000 7f132bb00640 1 -- 172.21.15.154:0/149039093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13240cee70 msgr2=0x7f1324159f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:28.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.584+0000 7f132bb00640 1 --2- 172.21.15.154:0/149039093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13240cee70 0x7f1324159f30 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f131c00e7f0 tx=0x7f131c00ebf0 comp rx=0 tx=0).stop 2023-09-23T03:27:28.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.584+0000 7f132bb00640 1 -- 172.21.15.154:0/149039093 shutdown_connections 2023-09-23T03:27:28.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.584+0000 7f132bb00640 1 --2- 172.21.15.154:0/149039093 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f131006af30 0x7f131006d3f0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:28.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.584+0000 7f132bb00640 1 --2- 172.21.15.154:0/149039093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13240cee70 0x7f1324159f30 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:28.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.584+0000 7f132bb00640 1 --2- 172.21.15.154:0/149039093 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f13240b1b20 0x7f132415add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:28.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.584+0000 7f132bb00640 1 --2- 172.21.15.154:0/149039093 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f13240acce0 0x7f132415d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:28.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.584+0000 7f132bb00640 1 -- 172.21.15.154:0/149039093 >> 172.21.15.154:0/149039093 conn(0x7f132406a0f0 msgr2=0x7f13240c9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:28.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.584+0000 7f132bb00640 1 -- 172.21.15.154:0/149039093 shutdown_connections 2023-09-23T03:27:28.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.584+0000 7f132bb00640 1 -- 172.21.15.154:0/149039093 wait complete. 2023-09-23T03:27:28.604 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:27:28.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.813+0000 7fe4b13e9640 1 Processor -- start 2023-09-23T03:27:28.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.813+0000 7fe4b13e9640 1 -- start start 2023-09-23T03:27:28.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.815+0000 7fe4b13e9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe4ac0c4970 0x7fe4ac071870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:28.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.815+0000 7fe4b13e9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4ac06c470 0x7fe4ac0ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:28.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.815+0000 7fe4b13e9640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4ac159a00 con 0x7fe4ac0ca260 2023-09-23T03:27:28.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.815+0000 7fe4b13e9640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4ac0cbe90 con 0x7fe4ac0c4970 2023-09-23T03:27:28.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.815+0000 7fe4b13e9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac071f70 con 0x7fe4ac06c470 2023-09-23T03:27:28.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.815+0000 7fe4a3fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fe4ac0ca260 0x7fe4ac06bd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:35512/0 (socket says 172.21.15.154:35512) 2023-09-23T03:27:28.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.815+0000 7fe4a3fff640 1 -- 172.21.15.154:0/1411126795 learned_addr learned my addr 172.21.15.154:0/1411126795 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:27:28.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.816+0000 7fe4abfff640 1 --2- 172.21.15.154:0/1411126795 >> v2:172.21.15.190:3300/0 conn(0x7fe4ac0c4970 0x7fe4ac071870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:28.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.816+0000 7fe4b0be8640 1 --2- 172.21.15.154:0/1411126795 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4ac06c470 0x7fe4ac0ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:28.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.816+0000 7fe4ab7fe640 1 -- 172.21.15.154:0/1411126795 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4097659628 0 0) 0x7fe4ac159a00 con 0x7fe4ac0ca260 2023-09-23T03:27:28.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.816+0000 7fe4ab7fe640 1 -- 172.21.15.154:0/1411126795 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe498005730 con 0x7fe4ac0ca260 2023-09-23T03:27:28.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.816+0000 7fe4ab7fe640 1 -- 172.21.15.154:0/1411126795 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1717282060 0 0) 0x7fe498005730 con 0x7fe4ac0ca260 2023-09-23T03:27:28.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.816+0000 7fe4ab7fe640 1 -- 172.21.15.154:0/1411126795 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4ac06c470 msgr2=0x7fe4ac0ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:28.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.816+0000 7fe4ab7fe640 1 --2- 172.21.15.154:0/1411126795 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4ac06c470 0x7fe4ac0ccc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:28.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.816+0000 7fe4ab7fe640 1 -- 172.21.15.154:0/1411126795 >> v2:172.21.15.190:3300/0 conn(0x7fe4ac0c4970 msgr2=0x7fe4ac071870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:28.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.816+0000 7fe4ab7fe640 1 --2- 172.21.15.154:0/1411126795 >> v2:172.21.15.190:3300/0 conn(0x7fe4ac0c4970 0x7fe4ac071870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:28.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.816+0000 7fe4ab7fe640 1 -- 172.21.15.154:0/1411126795 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4ac0cc150 con 0x7fe4ac0ca260 2023-09-23T03:27:28.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.816+0000 7fe4ab7fe640 1 -- 172.21.15.154:0/1411126795 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fe494001670 con 0x7fe4ac0ca260 2023-09-23T03:27:28.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.816+0000 7fe4ab7fe640 1 -- 172.21.15.154:0/1411126795 >> v1:172.21.15.154:6789/0 conn(0x7fe4ac0ca260 legacy=0x7fe4ac06bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:28.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.816+0000 7fe4ab7fe640 1 --2- 172.21.15.154:0/1411126795 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4ac06c470 0x7fe498005b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:28.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.816+0000 7fe4ab7fe640 1 --2- 172.21.15.154:0/1411126795 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe498006150 0x7fe4980069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:28.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.816+0000 7fe4ab7fe640 1 --2- 172.21.15.154:0/1411126795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe498006f10 0x7fe498007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:28.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.816+0000 7fe4ab7fe640 1 -- 172.21.15.154:0/1411126795 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac0cbe90 con 0x7fe498006150 2023-09-23T03:27:28.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.816+0000 7fe4ab7fe640 1 -- 172.21.15.154:0/1411126795 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac071f70 con 0x7fe498006f10 2023-09-23T03:27:28.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.816+0000 7fe4ab7fe640 1 -- 172.21.15.154:0/1411126795 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe49800db90 con 0x7fe4ac06c470 2023-09-23T03:27:28.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.817+0000 7fe4a3fff640 1 --2- 172.21.15.154:0/1411126795 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe498006150 0x7fe4980069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:28.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.817+0000 7fe4b0be8640 1 --2- 172.21.15.154:0/1411126795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe498006f10 0x7fe498007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:28.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.817+0000 7fe4abfff640 1 --2- 172.21.15.154:0/1411126795 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4ac06c470 0x7fe498005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:28.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.817+0000 7fe4a3fff640 1 -- 172.21.15.154:0/1411126795 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4ac06c470 msgr2=0x7fe498005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:28.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.817+0000 7fe4a3fff640 1 --2- 172.21.15.154:0/1411126795 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4ac06c470 0x7fe498005b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:28.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.817+0000 7fe4a3fff640 1 -- 172.21.15.154:0/1411126795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe498006f10 msgr2=0x7fe498007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:28.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.817+0000 7fe4a3fff640 1 --2- 172.21.15.154:0/1411126795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe498006f10 0x7fe498007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:28.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.817+0000 7fe4a3fff640 1 -- 172.21.15.154:0/1411126795 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe498005730 con 0x7fe498006150 2023-09-23T03:27:28.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.817+0000 7fe4b0be8640 1 --2- 172.21.15.154:0/1411126795 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe498006f10 0x7fe498007390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:28.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.818+0000 7fe4a3fff640 1 --2- 172.21.15.154:0/1411126795 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe498006150 0x7fe4980069d0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fe49400c140 tx=0x7fe498007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d01092daa3715a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:28.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.818+0000 7fe4ab7fe640 1 -- 172.21.15.154:0/1411126795 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe494005ad0 con 0x7fe498006150 2023-09-23T03:27:28.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.818+0000 7fe4ab7fe640 1 -- 172.21.15.154:0/1411126795 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe494005c50 con 0x7fe498006150 2023-09-23T03:27:28.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.818+0000 7fe4ab7fe640 1 -- 172.21.15.154:0/1411126795 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49402c810 con 0x7fe498006150 2023-09-23T03:27:28.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.819+0000 7fe4b13e9640 1 -- 172.21.15.154:0/1411126795 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe498006150 msgr2=0x7fe4980069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:28.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.819+0000 7fe4b13e9640 1 --2- 172.21.15.154:0/1411126795 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe498006150 0x7fe4980069d0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fe49400c140 tx=0x7fe498007a00 comp rx=0 tx=0).stop 2023-09-23T03:27:28.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.819+0000 7fe4b0be8640 1 -- 172.21.15.154:0/1411126795 reap_dead start 2023-09-23T03:27:28.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.819+0000 7fe4b13e9640 1 -- 172.21.15.154:0/1411126795 shutdown_connections 2023-09-23T03:27:28.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.819+0000 7fe4b13e9640 1 -- 172.21.15.154:0/1411126795 >> 172.21.15.154:0/1411126795 conn(0x7fe4ac06a390 msgr2=0x7fe4ac0afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:28.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.819+0000 7fe4b13e9640 1 -- 172.21.15.154:0/1411126795 shutdown_connections 2023-09-23T03:27:28.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.819+0000 7fe4b13e9640 1 -- 172.21.15.154:0/1411126795 wait complete. 2023-09-23T03:27:28.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.821+0000 7fe4b13e9640 1 Processor -- start 2023-09-23T03:27:28.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.821+0000 7fe4b13e9640 1 -- start start 2023-09-23T03:27:28.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.822+0000 7fe4b13e9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe498006150 0x7fe4ac0b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:28.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.822+0000 7fe4b13e9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4ac150710 0x7fe4ac15a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:28.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.822+0000 7fe4b13e9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4ac15abb0 0x7fe4ac0ac360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:28.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.822+0000 7fe4b13e9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac0c7250 con 0x7fe4ac150710 2023-09-23T03:27:28.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.822+0000 7fe4b13e9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac15b030 con 0x7fe498006150 2023-09-23T03:27:28.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.822+0000 7fe4b13e9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac0b1230 con 0x7fe4ac15abb0 2023-09-23T03:27:28.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.822+0000 7fe4a3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4ac150710 0x7fe4ac15a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:28.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.822+0000 7fe4a3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4ac150710 0x7fe4ac15a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46914/0 (socket says 172.21.15.154:46914) 2023-09-23T03:27:28.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.822+0000 7fe4a3fff640 1 -- 172.21.15.154:0/29710496 learned_addr learned my addr 172.21.15.154:0/29710496 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:28.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.823+0000 7fe4abfff640 1 --2- 172.21.15.154:0/29710496 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe498006150 0x7fe4ac0b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:28.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.823+0000 7fe4b0be8640 1 --2- 172.21.15.154:0/29710496 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4ac15abb0 0x7fe4ac0ac360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:28.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.823+0000 7fe4a3fff640 1 -- 172.21.15.154:0/29710496 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4ac15abb0 msgr2=0x7fe4ac0ac360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:28.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.823+0000 7fe4a3fff640 1 --2- 172.21.15.154:0/29710496 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4ac15abb0 0x7fe4ac0ac360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:28.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.823+0000 7fe4a3fff640 1 -- 172.21.15.154:0/29710496 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe498006150 msgr2=0x7fe4ac0b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:28.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.823+0000 7fe4a3fff640 1 --2- 172.21.15.154:0/29710496 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe498006150 0x7fe4ac0b0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:28.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.823+0000 7fe4a3fff640 1 -- 172.21.15.154:0/29710496 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4ac0ad120 con 0x7fe4ac150710 2023-09-23T03:27:28.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.823+0000 7fe4a3fff640 1 --2- 172.21.15.154:0/29710496 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4ac150710 0x7fe4ac15a670 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fe49400c0e0 tx=0x7fe494001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:28.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.823+0000 7fe4a97fa640 1 -- 172.21.15.154:0/29710496 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49403c040 con 0x7fe4ac150710 2023-09-23T03:27:28.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.823+0000 7fe4a97fa640 1 -- 172.21.15.154:0/29710496 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe49402cc20 con 0x7fe4ac150710 2023-09-23T03:27:28.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.823+0000 7fe4a97fa640 1 -- 172.21.15.154:0/29710496 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe494007070 con 0x7fe4ac150710 2023-09-23T03:27:28.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.823+0000 7fe4b13e9640 1 -- 172.21.15.154:0/29710496 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4ac0c68b0 con 0x7fe4ac150710 2023-09-23T03:27:28.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.824+0000 7fe4b13e9640 1 -- 172.21.15.154:0/29710496 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4ac0c6e70 con 0x7fe4ac150710 2023-09-23T03:27:28.828 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.825+0000 7fe4b13e9640 1 -- 172.21.15.154:0/29710496 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4ac0736b0 con 0x7fe4ac150710 2023-09-23T03:27:28.828 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.826+0000 7fe4a97fa640 1 -- 172.21.15.154:0/29710496 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fe494032070 con 0x7fe4ac150710 2023-09-23T03:27:28.828 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.827+0000 7fe4a97fa640 1 --2- 172.21.15.154:0/29710496 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe48c06af30 0x7fe48c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:28.828 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.827+0000 7fe4a97fa640 1 -- 172.21.15.154:0/29710496 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fe4940aac90 con 0x7fe4ac150710 2023-09-23T03:27:28.828 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.827+0000 7fe4abfff640 1 --2- 172.21.15.154:0/29710496 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe48c06af30 0x7fe48c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:28.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.828+0000 7fe4abfff640 1 --2- 172.21.15.154:0/29710496 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe48c06af30 0x7fe48c06d3f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe49c009990 tx=0x7fe49c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:28.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:28.831+0000 7fe4a97fa640 1 -- 172.21.15.154:0/29710496 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fe4940758e0 con 0x7fe4ac150710 2023-09-23T03:27:29.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.008+0000 7fe4b13e9640 1 -- 172.21.15.154:0/29710496 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe4ac0cc400 con 0x7fe4ac150710 2023-09-23T03:27:29.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.010+0000 7fe4a97fa640 1 -- 172.21.15.154:0/29710496 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+18945 (secure 0 0 0) 0x7fe494046070 con 0x7fe4ac150710 2023-09-23T03:27:29.012 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:29.012 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":114,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:26:39.641972+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:25:44.763842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-09-23T03:25:46.025926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data_extra","create_time":"2023-09-23T03:25:56.961772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6837/3181099748":"2023-09-24T03:26:34.396308+0000","172.21.15.154:6835/3181099748":"2023-09-24T03:26:34.396308+0000","172.21.15.154:6835/4233181417":"2023-09-24T03:26:27.714769+0000","172.21.15.154:6837/4233181417":"2023-09-24T03:26:27.714769+0000","172.21.15.190:6835/2967125458":"2023-09-24T03:26:25.688784+0000","172.21.15.190:6833/2967125458":"2023-09-24T03:26:25.688784+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:27:29.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.013+0000 7fe4b13e9640 1 -- 172.21.15.154:0/29710496 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe48c06af30 msgr2=0x7fe48c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:29.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.013+0000 7fe4b13e9640 1 --2- 172.21.15.154:0/29710496 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe48c06af30 0x7fe48c06d3f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe49c009990 tx=0x7fe49c008040 comp rx=0 tx=0).stop 2023-09-23T03:27:29.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.013+0000 7fe4b13e9640 1 -- 172.21.15.154:0/29710496 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4ac150710 msgr2=0x7fe4ac15a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:29.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.013+0000 7fe4b13e9640 1 --2- 172.21.15.154:0/29710496 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4ac150710 0x7fe4ac15a670 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fe49400c0e0 tx=0x7fe494001930 comp rx=0 tx=0).stop 2023-09-23T03:27:29.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.013+0000 7fe4b13e9640 1 -- 172.21.15.154:0/29710496 shutdown_connections 2023-09-23T03:27:29.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.013+0000 7fe4b13e9640 1 --2- 172.21.15.154:0/29710496 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe48c06af30 0x7fe48c06d3f0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:29.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.013+0000 7fe4b13e9640 1 --2- 172.21.15.154:0/29710496 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4ac15abb0 0x7fe4ac0ac360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:29.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.013+0000 7fe4b13e9640 1 --2- 172.21.15.154:0/29710496 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4ac150710 0x7fe4ac15a670 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:29.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.013+0000 7fe4b13e9640 1 --2- 172.21.15.154:0/29710496 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe498006150 0x7fe4ac0b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:29.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.013+0000 7fe4b13e9640 1 -- 172.21.15.154:0/29710496 >> 172.21.15.154:0/29710496 conn(0x7fe4ac06a390 msgr2=0x7fe4ac0cda20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:29.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.013+0000 7fe4b13e9640 1 -- 172.21.15.154:0/29710496 shutdown_connections 2023-09-23T03:27:29.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.013+0000 7fe4b13e9640 1 -- 172.21.15.154:0/29710496 wait complete. 2023-09-23T03:27:29.032 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-09-23T03:27:29.032 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:27:29.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.241+0000 7f74133f4640 1 Processor -- start 2023-09-23T03:27:29.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.241+0000 7f74133f4640 1 -- start start 2023-09-23T03:27:29.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.243+0000 7f74133f4640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f740c0cf500 0x7f740c0ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:29.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.243+0000 7f74133f4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f740c0cf8e0 0x7f740c0ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:29.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.243+0000 7f74133f4640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f740c15c4f0 0x7f740c15c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:29.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.243+0000 7f74133f4640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f740c0d17f0 con 0x7f740c0cf500 2023-09-23T03:27:29.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.243+0000 7f74133f4640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f740c0d0470 con 0x7f740c15c4f0 2023-09-23T03:27:29.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.243+0000 7f74133f4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f740c0d0750 con 0x7f740c0cf8e0 2023-09-23T03:27:29.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.243+0000 7f74123f2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f740c0cf500 0x7f740c0ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:29.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.243+0000 7f74123f2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f740c0cf500 0x7f740c0ce050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46930/0 (socket says 172.21.15.154:46930) 2023-09-23T03:27:29.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.243+0000 7f74123f2640 1 -- 172.21.15.154:0/2603376891 learned_addr learned my addr 172.21.15.154:0/2603376891 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:29.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.244+0000 7f74123f2640 1 -- 172.21.15.154:0/2603376891 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f740c0cf8e0 msgr2=0x7f740c0ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:29.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.244+0000 7f74123f2640 1 --2- 172.21.15.154:0/2603376891 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f740c0cf8e0 0x7f740c0ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:29.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.244+0000 7f74123f2640 1 -- 172.21.15.154:0/2603376891 >> v2:172.21.15.190:3300/0 conn(0x7f740c15c4f0 msgr2=0x7f740c15c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:29.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.244+0000 7f74123f2640 1 --2- 172.21.15.154:0/2603376891 >> v2:172.21.15.190:3300/0 conn(0x7f740c15c4f0 0x7f740c15c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:29.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.244+0000 7f74123f2640 1 -- 172.21.15.154:0/2603376891 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f740c0d0a30 con 0x7f740c0cf500 2023-09-23T03:27:29.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.245+0000 7f74123f2640 1 --2- 172.21.15.154:0/2603376891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f740c0cf500 0x7f740c0ce050 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f740c0c6160 tx=0x7f740802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=306895eb82f5592 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:29.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.245+0000 7f74113f0640 1 -- 172.21.15.154:0/2603376891 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7408004230 con 0x7f740c0cf500 2023-09-23T03:27:29.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.245+0000 7f74113f0640 1 -- 172.21.15.154:0/2603376891 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7408004390 con 0x7f740c0cf500 2023-09-23T03:27:29.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.245+0000 7f74113f0640 1 -- 172.21.15.154:0/2603376891 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7408003810 con 0x7f740c0cf500 2023-09-23T03:27:29.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.246+0000 7f74133f4640 1 -- 172.21.15.154:0/2603376891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f740c0cf500 msgr2=0x7f740c0ce050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:29.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.246+0000 7f74133f4640 1 --2- 172.21.15.154:0/2603376891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f740c0cf500 0x7f740c0ce050 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f740c0c6160 tx=0x7f740802b860 comp rx=0 tx=0).stop 2023-09-23T03:27:29.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.246+0000 7f74133f4640 1 -- 172.21.15.154:0/2603376891 shutdown_connections 2023-09-23T03:27:29.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.246+0000 7f74133f4640 1 --2- 172.21.15.154:0/2603376891 >> v2:172.21.15.190:3300/0 conn(0x7f740c15c4f0 0x7f740c15c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:29.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.246+0000 7f74133f4640 1 --2- 172.21.15.154:0/2603376891 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f740c0cf8e0 0x7f740c0ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:29.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.246+0000 7f74133f4640 1 --2- 172.21.15.154:0/2603376891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f740c0cf500 0x7f740c0ce050 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:29.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.246+0000 7f74133f4640 1 -- 172.21.15.154:0/2603376891 >> 172.21.15.154:0/2603376891 conn(0x7f740c09bbb0 msgr2=0x7f740c0d28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:29.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.246+0000 7f74133f4640 1 -- 172.21.15.154:0/2603376891 shutdown_connections 2023-09-23T03:27:29.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.246+0000 7f74133f4640 1 -- 172.21.15.154:0/2603376891 wait complete. 2023-09-23T03:27:29.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.248+0000 7f74133f4640 1 Processor -- start 2023-09-23T03:27:29.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.248+0000 7f74133f4640 1 -- start start 2023-09-23T03:27:29.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.249+0000 7f74133f4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f740c0cf500 0x7f740c0af410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:29.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.249+0000 7f74133f4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f740c0cf8e0 0x7f740c14aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:29.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.249+0000 7f74133f4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f740c15c4f0 0x7f740c0c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:29.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.249+0000 7f74133f4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c14e2b0 con 0x7f740c0cf500 2023-09-23T03:27:29.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.249+0000 7f74133f4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c14af40 con 0x7f740c0cf8e0 2023-09-23T03:27:29.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.249+0000 7f74133f4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f740c14dfd0 con 0x7f740c15c4f0 2023-09-23T03:27:29.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.249+0000 7f74123f2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f740c0cf500 0x7f740c0af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:29.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.249+0000 7f74123f2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f740c0cf500 0x7f740c0af410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46946/0 (socket says 172.21.15.154:46946) 2023-09-23T03:27:29.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.249+0000 7f74123f2640 1 -- 172.21.15.154:0/541902330 learned_addr learned my addr 172.21.15.154:0/541902330 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:29.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.249+0000 7f74123f2640 1 -- 172.21.15.154:0/541902330 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f740c15c4f0 msgr2=0x7f740c0c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:29.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.249+0000 7f74123f2640 1 --2- 172.21.15.154:0/541902330 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f740c15c4f0 0x7f740c0c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:29.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.249+0000 7f74123f2640 1 -- 172.21.15.154:0/541902330 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f740c0cf8e0 msgr2=0x7f740c14aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:29.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.249+0000 7f74123f2640 1 --2- 172.21.15.154:0/541902330 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f740c0cf8e0 0x7f740c14aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:29.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.249+0000 7f74123f2640 1 -- 172.21.15.154:0/541902330 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7408009460 con 0x7f740c0cf500 2023-09-23T03:27:29.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.250+0000 7f74123f2640 1 --2- 172.21.15.154:0/541902330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f740c0cf500 0x7f740c0af410 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f7408035040 tx=0x7f74080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:29.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.250+0000 7f7402ffd640 1 -- 172.21.15.154:0/541902330 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7408003e50 con 0x7f740c0cf500 2023-09-23T03:27:29.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.250+0000 7f7402ffd640 1 -- 172.21.15.154:0/541902330 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7408004500 con 0x7f740c0cf500 2023-09-23T03:27:29.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.250+0000 7f7402ffd640 1 -- 172.21.15.154:0/541902330 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7408006240 con 0x7f740c0cf500 2023-09-23T03:27:29.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.250+0000 7f74133f4640 1 -- 172.21.15.154:0/541902330 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f740c1f5af0 con 0x7f740c0cf500 2023-09-23T03:27:29.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.250+0000 7f74133f4640 1 -- 172.21.15.154:0/541902330 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f740c1f60b0 con 0x7f740c0cf500 2023-09-23T03:27:29.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.251+0000 7f74133f4640 1 -- 172.21.15.154:0/541902330 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f740c1f5950 con 0x7f740c0cf500 2023-09-23T03:27:29.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.252+0000 7f7402ffd640 1 -- 172.21.15.154:0/541902330 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f7408039050 con 0x7f740c0cf500 2023-09-23T03:27:29.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.253+0000 7f7402ffd640 1 --2- 172.21.15.154:0/541902330 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f73f006af30 0x7f73f006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:29.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.253+0000 7f7402ffd640 1 -- 172.21.15.154:0/541902330 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f74080aba70 con 0x7f740c0cf500 2023-09-23T03:27:29.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.254+0000 7f7411bf1640 1 --2- 172.21.15.154:0/541902330 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f73f006af30 0x7f73f006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:29.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.254+0000 7f7411bf1640 1 --2- 172.21.15.154:0/541902330 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f73f006af30 0x7f73f006d3f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f740c0cc540 tx=0x7f73fc006c90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:29.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.258+0000 7f7402ffd640 1 -- 172.21.15.154:0/541902330 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f740c1f5950 con 0x7f740c0cf500 2023-09-23T03:27:29.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.488+0000 7f74133f4640 1 -- 172.21.15.154:0/541902330 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f740c1f5950 con 0x7f740c0cf500 2023-09-23T03:27:29.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.489+0000 7f7402ffd640 1 -- 172.21.15.154:0/541902330 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4926 (secure 0 0 0) 0x7f740c1f5950 con 0x7f740c0cf500 2023-09-23T03:27:29.490 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:29.490 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":99,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5753,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/60017546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":60017546},{"type":"v1","addr":"172.21.15.190:6833","nonce":60017546}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95},{"gid":6415,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2708068523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2708068523},{"type":"v1","addr":"172.21.15.190:6835","nonce":2708068523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":99},{"gid":8592,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/1846547753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1846547753},{"type":"v1","addr":"172.21.15.154:6836","nonce":1846547753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:26:26.774375+0000","modified":"2023-09-23T03:26:36.432946+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8595":{"gid":8595,"name":"a","rank":0,"incarnation":95,"state":"up:active","state_seq":4,"addr":"172.21.15.154:6837/1758434763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1758434763},{"type":"v1","addr":"172.21.15.154:6837","nonce":1758434763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15,16],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-09-23T03:27:29.491 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 99 2023-09-23T03:27:29.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.491+0000 7f74133f4640 1 -- 172.21.15.154:0/541902330 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f73f006af30 msgr2=0x7f73f006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:29.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.491+0000 7f74133f4640 1 --2- 172.21.15.154:0/541902330 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f73f006af30 0x7f73f006d3f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f740c0cc540 tx=0x7f73fc006c90 comp rx=0 tx=0).stop 2023-09-23T03:27:29.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.491+0000 7f74133f4640 1 -- 172.21.15.154:0/541902330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f740c0cf500 msgr2=0x7f740c0af410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:29.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.491+0000 7f74133f4640 1 --2- 172.21.15.154:0/541902330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f740c0cf500 0x7f740c0af410 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f7408035040 tx=0x7f74080046c0 comp rx=0 tx=0).stop 2023-09-23T03:27:29.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.492+0000 7f74133f4640 1 -- 172.21.15.154:0/541902330 shutdown_connections 2023-09-23T03:27:29.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.492+0000 7f74133f4640 1 --2- 172.21.15.154:0/541902330 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f73f006af30 0x7f73f006d3f0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:29.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.492+0000 7f74133f4640 1 --2- 172.21.15.154:0/541902330 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f740c15c4f0 0x7f740c0c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:29.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.492+0000 7f74133f4640 1 --2- 172.21.15.154:0/541902330 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f740c0cf8e0 0x7f740c14aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:29.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.492+0000 7f74133f4640 1 --2- 172.21.15.154:0/541902330 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f740c0cf500 0x7f740c0af410 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:29.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.492+0000 7f74133f4640 1 -- 172.21.15.154:0/541902330 >> 172.21.15.154:0/541902330 conn(0x7f740c09bbb0 msgr2=0x7f740c0726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:29.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.492+0000 7f74133f4640 1 -- 172.21.15.154:0/541902330 shutdown_connections 2023-09-23T03:27:29.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.492+0000 7f74133f4640 1 -- 172.21.15.154:0/541902330 wait complete. 2023-09-23T03:27:29.513 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:27:29.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.721+0000 7f43638ec640 1 Processor -- start 2023-09-23T03:27:29.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.721+0000 7f43638ec640 1 -- start start 2023-09-23T03:27:29.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.723+0000 7f43638ec640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f435c0acce0 0x7f435c15ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:29.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.723+0000 7f43638ec640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f435c0b1b20 0x7f435c0ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:29.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.723+0000 7f43638ec640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f435c15ac30 con 0x7f435c0cee70 2023-09-23T03:27:29.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.723+0000 7f43638ec640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f435c0b2e90 con 0x7f435c0acce0 2023-09-23T03:27:29.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.723+0000 7f43638ec640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f435c0cf320 con 0x7f435c0b1b20 2023-09-23T03:27:29.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.723+0000 7f4361688640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f435c0acce0 0x7f435c15ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:29.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.723+0000 7f4361688640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f435c0acce0 0x7f435c15ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46962/0 (socket says 172.21.15.154:46962) 2023-09-23T03:27:29.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.723+0000 7f4361688640 1 -- 172.21.15.154:0/86740822 learned_addr learned my addr 172.21.15.154:0/86740822 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:29.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.724+0000 7f4360e87640 1 --2- 172.21.15.154:0/86740822 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f435c0b1b20 0x7f435c0ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:29.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.724+0000 7f4361688640 1 -- 172.21.15.154:0/86740822 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f435c0b1b20 msgr2=0x7f435c0ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:29.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.724+0000 7f4361688640 1 --2- 172.21.15.154:0/86740822 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f435c0b1b20 0x7f435c0ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:29.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.724+0000 7f4361688640 1 -- 172.21.15.154:0/86740822 >> v1:172.21.15.190:6789/0 conn(0x7f435c0cee70 legacy=0x7f435c06cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:29.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.724+0000 7f4361688640 1 -- 172.21.15.154:0/86740822 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f435c0b1f70 con 0x7f435c0acce0 2023-09-23T03:27:29.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.726+0000 7f4361688640 1 --2- 172.21.15.154:0/86740822 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f435c0acce0 0x7f435c15ed60 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f4358009860 tx=0x7f435802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513433ddafa2b06b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:29.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.726+0000 7f4353fff640 1 -- 172.21.15.154:0/86740822 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43580317c0 con 0x7f435c0acce0 2023-09-23T03:27:29.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.726+0000 7f4353fff640 1 -- 172.21.15.154:0/86740822 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4358031920 con 0x7f435c0acce0 2023-09-23T03:27:29.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.726+0000 7f4353fff640 1 -- 172.21.15.154:0/86740822 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4358031cb0 con 0x7f435c0acce0 2023-09-23T03:27:29.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.727+0000 7f43638ec640 1 -- 172.21.15.154:0/86740822 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f435c0acce0 msgr2=0x7f435c15ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:29.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.727+0000 7f43638ec640 1 --2- 172.21.15.154:0/86740822 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f435c0acce0 0x7f435c15ed60 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f4358009860 tx=0x7f435802b860 comp rx=0 tx=0).stop 2023-09-23T03:27:29.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.727+0000 7f43638ec640 1 -- 172.21.15.154:0/86740822 shutdown_connections 2023-09-23T03:27:29.728 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.727+0000 7f43638ec640 1 --2- 172.21.15.154:0/86740822 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f435c0b1b20 0x7f435c0ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:29.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.727+0000 7f43638ec640 1 --2- 172.21.15.154:0/86740822 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f435c0acce0 0x7f435c15ed60 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:29.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.727+0000 7f43638ec640 1 -- 172.21.15.154:0/86740822 >> 172.21.15.154:0/86740822 conn(0x7f435c06a0f0 msgr2=0x7f435c06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:29.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.727+0000 7f43638ec640 1 -- 172.21.15.154:0/86740822 shutdown_connections 2023-09-23T03:27:29.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.727+0000 7f43638ec640 1 -- 172.21.15.154:0/86740822 wait complete. 2023-09-23T03:27:29.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.729+0000 7f43638ec640 1 Processor -- start 2023-09-23T03:27:29.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.729+0000 7f43638ec640 1 -- start start 2023-09-23T03:27:29.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.730+0000 7f43638ec640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f435c0acce0 0x7f435c1f6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:29.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.730+0000 7f43638ec640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f435c0b1b20 0x7f435c15b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:29.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.730+0000 7f43638ec640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f435c0cee70 0x7f435c159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:29.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.730+0000 7f43638ec640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435c1f58e0 con 0x7f435c0acce0 2023-09-23T03:27:29.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.730+0000 7f43638ec640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435c15bb20 con 0x7f435c0cee70 2023-09-23T03:27:29.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.730+0000 7f43638ec640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f435c15a190 con 0x7f435c0b1b20 2023-09-23T03:27:29.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.730+0000 7f4361688640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f435c0acce0 0x7f435c1f6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:29.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.730+0000 7f4361688640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f435c0acce0 0x7f435c1f6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46968/0 (socket says 172.21.15.154:46968) 2023-09-23T03:27:29.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.730+0000 7f4361688640 1 -- 172.21.15.154:0/2526079681 learned_addr learned my addr 172.21.15.154:0/2526079681 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:29.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.730+0000 7f4361688640 1 -- 172.21.15.154:0/2526079681 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f435c0b1b20 msgr2=0x7f435c15b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:29.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.730+0000 7f4361688640 1 --2- 172.21.15.154:0/2526079681 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f435c0b1b20 0x7f435c15b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:29.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.730+0000 7f4361688640 1 -- 172.21.15.154:0/2526079681 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f435c0cee70 msgr2=0x7f435c159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:29.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.730+0000 7f4361688640 1 --2- 172.21.15.154:0/2526079681 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f435c0cee70 0x7f435c159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:29.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.731+0000 7f4361688640 1 -- 172.21.15.154:0/2526079681 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4358009470 con 0x7f435c0acce0 2023-09-23T03:27:29.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.731+0000 7f4361688640 1 --2- 172.21.15.154:0/2526079681 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f435c0acce0 0x7f435c1f6330 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f4358000c00 tx=0x7f43580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:29.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.731+0000 7f4351ffb640 1 -- 172.21.15.154:0/2526079681 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4358032860 con 0x7f435c0acce0 2023-09-23T03:27:29.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.731+0000 7f4351ffb640 1 -- 172.21.15.154:0/2526079681 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43580329c0 con 0x7f435c0acce0 2023-09-23T03:27:29.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.731+0000 7f4351ffb640 1 -- 172.21.15.154:0/2526079681 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4358032d50 con 0x7f435c0acce0 2023-09-23T03:27:29.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.731+0000 7f43638ec640 1 -- 172.21.15.154:0/2526079681 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f435c15bda0 con 0x7f435c0acce0 2023-09-23T03:27:29.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.731+0000 7f43638ec640 1 -- 172.21.15.154:0/2526079681 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f435c1f5b30 con 0x7f435c0acce0 2023-09-23T03:27:29.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.732+0000 7f43638ec640 1 -- 172.21.15.154:0/2526079681 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f435c15c2c0 con 0x7f435c0acce0 2023-09-23T03:27:29.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.733+0000 7f4351ffb640 1 -- 172.21.15.154:0/2526079681 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f435804e450 con 0x7f435c0acce0 2023-09-23T03:27:29.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.734+0000 7f4351ffb640 1 --2- 172.21.15.154:0/2526079681 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f434006af60 0x7f434006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:29.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.734+0000 7f4351ffb640 1 -- 172.21.15.154:0/2526079681 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f43580abe10 con 0x7f435c0acce0 2023-09-23T03:27:29.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.735+0000 7f4360e87640 1 --2- 172.21.15.154:0/2526079681 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f434006af60 0x7f434006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:29.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.735+0000 7f4360e87640 1 --2- 172.21.15.154:0/2526079681 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f434006af60 0x7f434006d420 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f435c0c19b0 tx=0x7f434c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:29.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.739+0000 7f4351ffb640 1 -- 172.21.15.154:0/2526079681 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f43580af030 con 0x7f435c0acce0 2023-09-23T03:27:29.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.963+0000 7f43638ec640 1 -- 172.21.15.154:0/2526079681 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f435c073eb0 con 0x7f435c0acce0 2023-09-23T03:27:29.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.964+0000 7f4351ffb640 1 -- 172.21.15.154:0/2526079681 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4926 (secure 0 0 0) 0x7f4358076c30 con 0x7f435c0acce0 2023-09-23T03:27:29.965 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:29.965 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":99,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5753,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/60017546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":60017546},{"type":"v1","addr":"172.21.15.190:6833","nonce":60017546}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95},{"gid":6415,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2708068523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2708068523},{"type":"v1","addr":"172.21.15.190:6835","nonce":2708068523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":99},{"gid":8592,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/1846547753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1846547753},{"type":"v1","addr":"172.21.15.154:6836","nonce":1846547753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:26:26.774375+0000","modified":"2023-09-23T03:26:36.432946+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8595},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8595":{"gid":8595,"name":"a","rank":0,"incarnation":95,"state":"up:active","state_seq":4,"addr":"172.21.15.154:6837/1758434763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1758434763},{"type":"v1","addr":"172.21.15.154:6837","nonce":1758434763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15,16],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-09-23T03:27:29.965 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 99 2023-09-23T03:27:29.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.967+0000 7f43638ec640 1 -- 172.21.15.154:0/2526079681 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f434006af60 msgr2=0x7f434006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:29.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.967+0000 7f43638ec640 1 --2- 172.21.15.154:0/2526079681 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f434006af60 0x7f434006d420 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f435c0c19b0 tx=0x7f434c008040 comp rx=0 tx=0).stop 2023-09-23T03:27:29.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.967+0000 7f43638ec640 1 -- 172.21.15.154:0/2526079681 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f435c0acce0 msgr2=0x7f435c1f6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:29.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.967+0000 7f43638ec640 1 --2- 172.21.15.154:0/2526079681 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f435c0acce0 0x7f435c1f6330 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f4358000c00 tx=0x7f43580046c0 comp rx=0 tx=0).stop 2023-09-23T03:27:29.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.967+0000 7f43638ec640 1 -- 172.21.15.154:0/2526079681 shutdown_connections 2023-09-23T03:27:29.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.967+0000 7f43638ec640 1 --2- 172.21.15.154:0/2526079681 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f434006af60 0x7f434006d420 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:29.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.967+0000 7f43638ec640 1 --2- 172.21.15.154:0/2526079681 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f435c0cee70 0x7f435c159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:29.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.967+0000 7f43638ec640 1 --2- 172.21.15.154:0/2526079681 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f435c0b1b20 0x7f435c15b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:29.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.967+0000 7f43638ec640 1 --2- 172.21.15.154:0/2526079681 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f435c0acce0 0x7f435c1f6330 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:29.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.967+0000 7f43638ec640 1 -- 172.21.15.154:0/2526079681 >> 172.21.15.154:0/2526079681 conn(0x7f435c06a0f0 msgr2=0x7f435c0b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:29.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.967+0000 7f43638ec640 1 -- 172.21.15.154:0/2526079681 shutdown_connections 2023-09-23T03:27:29.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:29.967+0000 7f43638ec640 1 -- 172.21.15.154:0/2526079681 wait complete. 2023-09-23T03:27:29.987 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:27:30.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.199+0000 7f3ce4389640 1 Processor -- start 2023-09-23T03:27:30.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.200+0000 7f3ce4389640 1 -- start start 2023-09-23T03:27:30.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.201+0000 7f3ce4389640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3cdc0cf040 0x7f3cdc1656e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:30.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.201+0000 7f3ce4389640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cdc15ac30 con 0x7f3cdc0acce0 2023-09-23T03:27:30.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.201+0000 7f3ce4389640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cdc0b1f70 con 0x7f3cdc0b1b20 2023-09-23T03:27:30.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.201+0000 7f3ce4389640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cdc0b2e90 con 0x7f3cdc0cf040 2023-09-23T03:27:30.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.201+0000 7f3ce2125640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f3cdc0acce0 0x7f3cdc15ed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:35524/0 (socket says 172.21.15.154:35524) 2023-09-23T03:27:30.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.201+0000 7f3ce2125640 1 -- 172.21.15.154:0/2172933130 learned_addr learned my addr 172.21.15.154:0/2172933130 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:27:30.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.201+0000 7f3ce2926640 1 --2- 172.21.15.154:0/2172933130 >> v2:172.21.15.190:3300/0 conn(0x7f3cdc0cf040 0x7f3cdc1656e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:30.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.201+0000 7f3ce1123640 1 -- 172.21.15.154:0/2172933130 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1429727445 0 0) 0x7f3cd8001290 con 0x7f3cdc0acce0 2023-09-23T03:27:30.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.202+0000 7f3ce1123640 1 -- 172.21.15.154:0/2172933130 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cc8003520 con 0x7f3cdc0acce0 2023-09-23T03:27:30.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.202+0000 7f3ce1123640 1 -- 172.21.15.154:0/2172933130 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1892373797 0 0) 0x7f3cc8003520 con 0x7f3cdc0acce0 2023-09-23T03:27:30.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.202+0000 7f3ce1123640 1 -- 172.21.15.154:0/2172933130 >> v2:172.21.15.190:3300/0 conn(0x7f3cdc0cf040 msgr2=0x7f3cdc1656e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:30.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.202+0000 7f3ce1123640 1 --2- 172.21.15.154:0/2172933130 >> v2:172.21.15.190:3300/0 conn(0x7f3cdc0cf040 0x7f3cdc1656e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:30.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.202+0000 7f3ce1123640 1 -- 172.21.15.154:0/2172933130 >> v1:172.21.15.190:6789/0 conn(0x7f3cdc0b1b20 legacy=0x7f3cdc0ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:30.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.202+0000 7f3ce1123640 1 -- 172.21.15.154:0/2172933130 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cdc0b2210 con 0x7f3cdc0acce0 2023-09-23T03:27:30.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.202+0000 7f3ce1123640 1 -- 172.21.15.154:0/2172933130 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f3cd8001980 con 0x7f3cdc0acce0 2023-09-23T03:27:30.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.202+0000 7f3ce1123640 1 -- 172.21.15.154:0/2172933130 >> v1:172.21.15.154:6789/0 conn(0x7f3cdc0acce0 legacy=0x7f3cdc15ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:30.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.202+0000 7f3ce1123640 1 --2- 172.21.15.154:0/2172933130 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cc80039c0 0x7f3cc8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:30.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.202+0000 7f3ce1123640 1 --2- 172.21.15.154:0/2172933130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cc80047c0 0x7f3cc8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:30.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.202+0000 7f3ce1123640 1 --2- 172.21.15.154:0/2172933130 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cc80051d0 0x7f3cc800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:30.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.202+0000 7f3ce1123640 1 -- 172.21.15.154:0/2172933130 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cdc0b2e90 con 0x7f3cc80051d0 2023-09-23T03:27:30.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.202+0000 7f3ce1123640 1 -- 172.21.15.154:0/2172933130 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc800c120 con 0x7f3cc80047c0 2023-09-23T03:27:30.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.202+0000 7f3ce1123640 1 -- 172.21.15.154:0/2172933130 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc800c2e0 con 0x7f3cc80039c0 2023-09-23T03:27:30.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.202+0000 7f3ce2926640 1 --2- 172.21.15.154:0/2172933130 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cc80051d0 0x7f3cc800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:30.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.203+0000 7f3ce1924640 1 --2- 172.21.15.154:0/2172933130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cc80047c0 0x7f3cc8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:30.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.203+0000 7f3ce2125640 1 --2- 172.21.15.154:0/2172933130 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cc80039c0 0x7f3cc8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:30.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.203+0000 7f3ce2926640 1 -- 172.21.15.154:0/2172933130 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cc80039c0 msgr2=0x7f3cc8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:30.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.203+0000 7f3ce2926640 1 --2- 172.21.15.154:0/2172933130 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cc80039c0 0x7f3cc8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:30.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.203+0000 7f3ce2926640 1 -- 172.21.15.154:0/2172933130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cc80047c0 msgr2=0x7f3cc8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:30.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.203+0000 7f3ce2926640 1 --2- 172.21.15.154:0/2172933130 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cc80047c0 0x7f3cc8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:30.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.203+0000 7f3ce2926640 1 -- 172.21.15.154:0/2172933130 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc8003520 con 0x7f3cc80051d0 2023-09-23T03:27:30.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.204+0000 7f3ce2926640 1 -- 172.21.15.154:0/2172933130 reap_dead start 2023-09-23T03:27:30.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.204+0000 7f3ce2926640 1 --2- 172.21.15.154:0/2172933130 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cc80051d0 0x7f3cc800b980 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4009f90 tx=0x7f3cc80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e4bab74dabeda7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:30.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.204+0000 7f3ce1123640 1 -- 172.21.15.154:0/2172933130 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd402cc20 con 0x7f3cc80051d0 2023-09-23T03:27:30.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.204+0000 7f3ce1123640 1 -- 172.21.15.154:0/2172933130 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd400f040 con 0x7f3cc80051d0 2023-09-23T03:27:30.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.204+0000 7f3ce1123640 1 -- 172.21.15.154:0/2172933130 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4003990 con 0x7f3cc80051d0 2023-09-23T03:27:30.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.205+0000 7f3ce4389640 1 -- 172.21.15.154:0/2172933130 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cc80051d0 msgr2=0x7f3cc800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:30.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.205+0000 7f3ce4389640 1 --2- 172.21.15.154:0/2172933130 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cc80051d0 0x7f3cc800b980 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4009f90 tx=0x7f3cc80057f0 comp rx=0 tx=0).stop 2023-09-23T03:27:30.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.205+0000 7f3ce4389640 1 -- 172.21.15.154:0/2172933130 shutdown_connections 2023-09-23T03:27:30.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.205+0000 7f3ce4389640 1 --2- 172.21.15.154:0/2172933130 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cc80051d0 0x7f3cc800b980 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:30.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.205+0000 7f3ce4389640 1 -- 172.21.15.154:0/2172933130 >> 172.21.15.154:0/2172933130 conn(0x7f3cdc06a0f0 msgr2=0x7f3cdc06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:30.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.205+0000 7f3ce4389640 1 -- 172.21.15.154:0/2172933130 shutdown_connections 2023-09-23T03:27:30.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.205+0000 7f3ce4389640 1 -- 172.21.15.154:0/2172933130 wait complete. 2023-09-23T03:27:30.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.207+0000 7f3ce4389640 1 Processor -- start 2023-09-23T03:27:30.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.207+0000 7f3ce4389640 1 -- start start 2023-09-23T03:27:30.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.208+0000 7f3ce4389640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cc80051d0 0x7f3cdc159110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:30.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.208+0000 7f3ce4389640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cdc0c4c20 0x7f3cdc0c2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:30.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.208+0000 7f3ce4389640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cdc1ef260 0x7f3cdc1ef710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:30.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.208+0000 7f3ce4389640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cdc144a50 con 0x7f3cdc1ef260 2023-09-23T03:27:30.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.208+0000 7f3ce4389640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cdc1efc50 con 0x7f3cc80051d0 2023-09-23T03:27:30.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.208+0000 7f3ce4389640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cdc159a10 con 0x7f3cdc0c4c20 2023-09-23T03:27:30.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.208+0000 7f3ce2926640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cdc1ef260 0x7f3cdc1ef710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:30.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.208+0000 7f3ce2926640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cdc1ef260 0x7f3cdc1ef710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:46992/0 (socket says 172.21.15.154:46992) 2023-09-23T03:27:30.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.208+0000 7f3ce2926640 1 -- 172.21.15.154:0/576127115 learned_addr learned my addr 172.21.15.154:0/576127115 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:30.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.209+0000 7f3ce1924640 1 --2- 172.21.15.154:0/576127115 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cdc0c4c20 0x7f3cdc0c2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:30.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.209+0000 7f3ce2125640 1 --2- 172.21.15.154:0/576127115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cc80051d0 0x7f3cdc159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:30.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.209+0000 7f3ce2926640 1 -- 172.21.15.154:0/576127115 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cdc0c4c20 msgr2=0x7f3cdc0c2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:30.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.209+0000 7f3ce2926640 1 --2- 172.21.15.154:0/576127115 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cdc0c4c20 0x7f3cdc0c2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:30.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.209+0000 7f3ce2926640 1 -- 172.21.15.154:0/576127115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cc80051d0 msgr2=0x7f3cdc159110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:30.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.209+0000 7f3ce2926640 1 --2- 172.21.15.154:0/576127115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cc80051d0 0x7f3cdc159110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:30.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.209+0000 7f3ce2926640 1 -- 172.21.15.154:0/576127115 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cdc159cf0 con 0x7f3cdc1ef260 2023-09-23T03:27:30.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.209+0000 7f3ce2125640 1 --2- 172.21.15.154:0/576127115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cc80051d0 0x7f3cdc159110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:30.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.209+0000 7f3ce1924640 1 --2- 172.21.15.154:0/576127115 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cdc0c4c20 0x7f3cdc0c2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:30.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.209+0000 7f3ce2926640 1 --2- 172.21.15.154:0/576127115 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cdc1ef260 0x7f3cdc1ef710 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f3cd402c690 tx=0x7f3cd4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:30.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.209+0000 7f3cd2ffd640 1 -- 172.21.15.154:0/576127115 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4035ee0 con 0x7f3cdc1ef260 2023-09-23T03:27:30.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.209+0000 7f3cd2ffd640 1 -- 172.21.15.154:0/576127115 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd4040470 con 0x7f3cdc1ef260 2023-09-23T03:27:30.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.209+0000 7f3cd2ffd640 1 -- 172.21.15.154:0/576127115 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4040800 con 0x7f3cdc1ef260 2023-09-23T03:27:30.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.209+0000 7f3ce4389640 1 -- 172.21.15.154:0/576127115 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cdc159fd0 con 0x7f3cdc1ef260 2023-09-23T03:27:30.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.210+0000 7f3ce4389640 1 -- 172.21.15.154:0/576127115 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cdc159650 con 0x7f3cdc1ef260 2023-09-23T03:27:30.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.211+0000 7f3ce4389640 1 -- 172.21.15.154:0/576127115 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cdc073eb0 con 0x7f3cdc1ef260 2023-09-23T03:27:30.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.211+0000 7f3cd2ffd640 1 -- 172.21.15.154:0/576127115 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f3cd4040960 con 0x7f3cdc1ef260 2023-09-23T03:27:30.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.212+0000 7f3cd2ffd640 1 --2- 172.21.15.154:0/576127115 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3cc006af30 0x7f3cc006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:30.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.212+0000 7f3cd2ffd640 1 -- 172.21.15.154:0/576127115 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f3cd40ac920 con 0x7f3cdc1ef260 2023-09-23T03:27:30.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.213+0000 7f3ce2125640 1 --2- 172.21.15.154:0/576127115 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3cc006af30 0x7f3cc006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:30.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.213+0000 7f3ce2125640 1 --2- 172.21.15.154:0/576127115 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3cc006af30 0x7f3cc006d3f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f3cd80013f0 tx=0x7f3cd8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:30.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.216+0000 7f3cd2ffd640 1 -- 172.21.15.154:0/576127115 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f3cd4077690 con 0x7f3cdc1ef260 2023-09-23T03:27:30.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.405+0000 7f3ce4389640 1 -- 172.21.15.154:0/576127115 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3cdc06c930 con 0x7f3cdc1ef260 2023-09-23T03:27:30.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.407+0000 7f3cd2ffd640 1 -- 172.21.15.154:0/576127115 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+18945 (secure 0 0 0) 0x7f3cd407acb0 con 0x7f3cdc1ef260 2023-09-23T03:27:30.409 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:30.409 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":114,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:26:39.641972+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:25:44.763842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-09-23T03:25:46.025926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data_extra","create_time":"2023-09-23T03:25:56.961772+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6837/3181099748":"2023-09-24T03:26:34.396308+0000","172.21.15.154:6835/3181099748":"2023-09-24T03:26:34.396308+0000","172.21.15.154:6835/4233181417":"2023-09-24T03:26:27.714769+0000","172.21.15.154:6837/4233181417":"2023-09-24T03:26:27.714769+0000","172.21.15.190:6835/2967125458":"2023-09-24T03:26:25.688784+0000","172.21.15.190:6833/2967125458":"2023-09-24T03:26:25.688784+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:27:30.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.410+0000 7f3ce4389640 1 -- 172.21.15.154:0/576127115 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3cc006af30 msgr2=0x7f3cc006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:30.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.410+0000 7f3ce4389640 1 --2- 172.21.15.154:0/576127115 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3cc006af30 0x7f3cc006d3f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f3cd80013f0 tx=0x7f3cd8009040 comp rx=0 tx=0).stop 2023-09-23T03:27:30.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.410+0000 7f3ce4389640 1 -- 172.21.15.154:0/576127115 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cdc1ef260 msgr2=0x7f3cdc1ef710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:30.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.410+0000 7f3ce4389640 1 --2- 172.21.15.154:0/576127115 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cdc1ef260 0x7f3cdc1ef710 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f3cd402c690 tx=0x7f3cd4009750 comp rx=0 tx=0).stop 2023-09-23T03:27:30.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.410+0000 7f3ce4389640 1 -- 172.21.15.154:0/576127115 shutdown_connections 2023-09-23T03:27:30.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.410+0000 7f3ce4389640 1 --2- 172.21.15.154:0/576127115 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3cc006af30 0x7f3cc006d3f0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:30.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.410+0000 7f3ce4389640 1 --2- 172.21.15.154:0/576127115 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cdc1ef260 0x7f3cdc1ef710 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:30.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.410+0000 7f3ce4389640 1 --2- 172.21.15.154:0/576127115 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3cdc0c4c20 0x7f3cdc0c2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:30.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.410+0000 7f3ce4389640 1 --2- 172.21.15.154:0/576127115 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3cc80051d0 0x7f3cdc159110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:30.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.410+0000 7f3ce4389640 1 -- 172.21.15.154:0/576127115 >> 172.21.15.154:0/576127115 conn(0x7f3cdc06a0f0 msgr2=0x7f3cdc0b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:30.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.410+0000 7f3ce4389640 1 -- 172.21.15.154:0/576127115 shutdown_connections 2023-09-23T03:27:30.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.411+0000 7f3ce4389640 1 -- 172.21.15.154:0/576127115 wait complete. 2023-09-23T03:27:30.430 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-09-23T03:27:30.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.646+0000 7f400dee0640 1 Processor -- start 2023-09-23T03:27:30.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.646+0000 7f400dee0640 1 -- start start 2023-09-23T03:27:30.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.647+0000 7f400dee0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40080acce0 0x7f400815ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:30.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.647+0000 7f400dee0640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f40080cf040 0x7f40080ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:30.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.647+0000 7f400dee0640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f400815ac30 con 0x7f40080b1b20 2023-09-23T03:27:30.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.647+0000 7f400dee0640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40080b2e90 con 0x7f40080cf040 2023-09-23T03:27:30.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.647+0000 7f400dee0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40080b1f70 con 0x7f40080acce0 2023-09-23T03:27:30.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.648+0000 7f4007fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f40080cf040 0x7f40080ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:30.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.648+0000 7f4006ffd640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f40080b1b20 0x7f40080ce930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.154:35422/0 (socket says 172.21.15.154:35422) 2023-09-23T03:27:30.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.648+0000 7f4006ffd640 1 -- 172.21.15.154:0/2991815659 learned_addr learned my addr 172.21.15.154:0/2991815659 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:27:30.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.648+0000 7f40077fe640 1 --2- 172.21.15.154:0/2991815659 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40080acce0 0x7f400815ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:30.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.649+0000 7f40077fe640 1 -- 172.21.15.154:0/2991815659 >> v2:172.21.15.190:3300/0 conn(0x7f40080cf040 msgr2=0x7f40080ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:30.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.649+0000 7f40077fe640 1 --2- 172.21.15.154:0/2991815659 >> v2:172.21.15.190:3300/0 conn(0x7f40080cf040 0x7f40080ab140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:30.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.649+0000 7f40077fe640 1 -- 172.21.15.154:0/2991815659 >> v1:172.21.15.190:6789/0 conn(0x7f40080b1b20 legacy=0x7f40080ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:30.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.649+0000 7f40077fe640 1 -- 172.21.15.154:0/2991815659 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f400815f2a0 con 0x7f40080acce0 2023-09-23T03:27:30.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.649+0000 7f4007fff640 1 --2- 172.21.15.154:0/2991815659 >> v2:172.21.15.190:3300/0 conn(0x7f40080cf040 0x7f40080ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:27:30.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.650+0000 7f40077fe640 1 --2- 172.21.15.154:0/2991815659 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40080acce0 0x7f400815ed60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8009de0 tx=0x7f3ff802be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e10727a5b5689df server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:30.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.651+0000 7f40067fc640 1 -- 172.21.15.154:0/2991815659 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff802cc00 con 0x7f40080acce0 2023-09-23T03:27:30.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.651+0000 7f40067fc640 1 -- 172.21.15.154:0/2991815659 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff802cd60 con 0x7f40080acce0 2023-09-23T03:27:30.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.651+0000 7f40067fc640 1 -- 172.21.15.154:0/2991815659 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8034660 con 0x7f40080acce0 2023-09-23T03:27:30.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.651+0000 7f400dee0640 1 -- 172.21.15.154:0/2991815659 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40080acce0 msgr2=0x7f400815ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:30.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.651+0000 7f400dee0640 1 --2- 172.21.15.154:0/2991815659 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40080acce0 0x7f400815ed60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8009de0 tx=0x7f3ff802be50 comp rx=0 tx=0).stop 2023-09-23T03:27:30.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.652+0000 7f400dee0640 1 -- 172.21.15.154:0/2991815659 shutdown_connections 2023-09-23T03:27:30.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.652+0000 7f400dee0640 1 --2- 172.21.15.154:0/2991815659 >> v2:172.21.15.190:3300/0 conn(0x7f40080cf040 0x7f40080ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:30.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.652+0000 7f400dee0640 1 --2- 172.21.15.154:0/2991815659 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40080acce0 0x7f400815ed60 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:30.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.652+0000 7f400dee0640 1 -- 172.21.15.154:0/2991815659 >> 172.21.15.154:0/2991815659 conn(0x7f400806a0f0 msgr2=0x7f400806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:30.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.652+0000 7f400dee0640 1 -- 172.21.15.154:0/2991815659 shutdown_connections 2023-09-23T03:27:30.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.652+0000 7f400dee0640 1 -- 172.21.15.154:0/2991815659 wait complete. 2023-09-23T03:27:30.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.654+0000 7f400dee0640 1 Processor -- start 2023-09-23T03:27:30.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.654+0000 7f400dee0640 1 -- start start 2023-09-23T03:27:30.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.655+0000 7f400dee0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40080acce0 0x7f40080c7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:30.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.655+0000 7f400dee0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40080b1b20 0x7f400815a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:30.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.655+0000 7f400dee0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40080cf040 0x7f40080abea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:30.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.655+0000 7f400dee0640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40080c89c0 con 0x7f40080b1b20 2023-09-23T03:27:30.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.655+0000 7f400dee0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40080c7ed0 con 0x7f40080cf040 2023-09-23T03:27:30.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.655+0000 7f400dee0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40080c8120 con 0x7f40080acce0 2023-09-23T03:27:30.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.655+0000 7f4006ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40080b1b20 0x7f400815a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:30.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.655+0000 7f4006ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40080b1b20 0x7f400815a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47004/0 (socket says 172.21.15.154:47004) 2023-09-23T03:27:30.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.655+0000 7f4006ffd640 1 -- 172.21.15.154:0/882159317 learned_addr learned my addr 172.21.15.154:0/882159317 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:30.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.656+0000 7f4007fff640 1 --2- 172.21.15.154:0/882159317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40080cf040 0x7f40080abea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:30.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.656+0000 7f40077fe640 1 --2- 172.21.15.154:0/882159317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40080acce0 0x7f40080c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:30.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.656+0000 7f4006ffd640 1 -- 172.21.15.154:0/882159317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40080acce0 msgr2=0x7f40080c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:30.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.656+0000 7f4006ffd640 1 --2- 172.21.15.154:0/882159317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40080acce0 0x7f40080c7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:30.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.656+0000 7f4006ffd640 1 -- 172.21.15.154:0/882159317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40080cf040 msgr2=0x7f40080abea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:30.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.656+0000 7f4006ffd640 1 --2- 172.21.15.154:0/882159317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40080cf040 0x7f40080abea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:30.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.656+0000 7f4006ffd640 1 -- 172.21.15.154:0/882159317 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4000009400 con 0x7f40080b1b20 2023-09-23T03:27:30.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.656+0000 7f4007fff640 1 --2- 172.21.15.154:0/882159317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40080cf040 0x7f40080abea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:30.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.656+0000 7f4006ffd640 1 --2- 172.21.15.154:0/882159317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40080b1b20 0x7f400815a2c0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000c500 tx=0x7f3ff000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:30.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.656+0000 7f3ffffff640 1 -- 172.21.15.154:0/882159317 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0005ce0 con 0x7f40080b1b20 2023-09-23T03:27:30.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.656+0000 7f3ffffff640 1 -- 172.21.15.154:0/882159317 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff0005e40 con 0x7f40080b1b20 2023-09-23T03:27:30.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.656+0000 7f400dee0640 1 -- 172.21.15.154:0/882159317 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff8009a70 con 0x7f40080b1b20 2023-09-23T03:27:30.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.656+0000 7f3ffffff640 1 -- 172.21.15.154:0/882159317 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff00072f0 con 0x7f40080b1b20 2023-09-23T03:27:30.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.657+0000 7f400dee0640 1 -- 172.21.15.154:0/882159317 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40080c86e0 con 0x7f40080b1b20 2023-09-23T03:27:30.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.658+0000 7f400dee0640 1 -- 172.21.15.154:0/882159317 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4008073eb0 con 0x7f40080b1b20 2023-09-23T03:27:30.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.659+0000 7f3ffffff640 1 -- 172.21.15.154:0/882159317 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f3ff0007450 con 0x7f40080b1b20 2023-09-23T03:27:30.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.659+0000 7f3ffffff640 1 --2- 172.21.15.154:0/882159317 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3fe806af30 0x7f3fe806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:30.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.660+0000 7f3ffffff640 1 -- 172.21.15.154:0/882159317 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f3ff0088e50 con 0x7f40080b1b20 2023-09-23T03:27:30.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.660+0000 7f40077fe640 1 --2- 172.21.15.154:0/882159317 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3fe806af30 0x7f3fe806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:30.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.661+0000 7f40077fe640 1 --2- 172.21.15.154:0/882159317 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3fe806af30 0x7f3fe806d3f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8036000 tx=0x7f3ff8030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:30.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.664+0000 7f3ffffff640 1 -- 172.21.15.154:0/882159317 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f3ff0053b50 con 0x7f40080b1b20 2023-09-23T03:27:30.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.891+0000 7f400dee0640 1 -- 172.21.15.154:0/882159317 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f400815ac10 con 0x7f40080b1b20 2023-09-23T03:27:30.893 INFO:tasks.ceph.mon.a.smithi154.stderr:2023-09-23T03:27:30.892+0000 7f3df35a6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-23T03:27:30.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.895+0000 7f3ffffff640 1 -- 172.21.15.154:0/882159317 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v100) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3ff0017020 con 0x7f40080b1b20 2023-09-23T03:27:30.897 INFO:teuthology.orchestra.run.smithi154.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-09-23T03:27:30.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.899+0000 7f400dee0640 1 -- 172.21.15.154:0/882159317 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3fe806af30 msgr2=0x7f3fe806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:30.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.899+0000 7f400dee0640 1 --2- 172.21.15.154:0/882159317 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3fe806af30 0x7f3fe806d3f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8036000 tx=0x7f3ff8030000 comp rx=0 tx=0).stop 2023-09-23T03:27:30.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.900+0000 7f400dee0640 1 -- 172.21.15.154:0/882159317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40080b1b20 msgr2=0x7f400815a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:30.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.900+0000 7f400dee0640 1 --2- 172.21.15.154:0/882159317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40080b1b20 0x7f400815a2c0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000c500 tx=0x7f3ff000c9c0 comp rx=0 tx=0).stop 2023-09-23T03:27:30.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.900+0000 7f400dee0640 1 -- 172.21.15.154:0/882159317 shutdown_connections 2023-09-23T03:27:30.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.900+0000 7f400dee0640 1 --2- 172.21.15.154:0/882159317 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3fe806af30 0x7f3fe806d3f0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:30.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.900+0000 7f400dee0640 1 --2- 172.21.15.154:0/882159317 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f40080cf040 0x7f40080abea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:30.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.900+0000 7f400dee0640 1 --2- 172.21.15.154:0/882159317 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f40080b1b20 0x7f400815a2c0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:30.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.900+0000 7f400dee0640 1 --2- 172.21.15.154:0/882159317 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f40080acce0 0x7f40080c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:30.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.900+0000 7f400dee0640 1 -- 172.21.15.154:0/882159317 >> 172.21.15.154:0/882159317 conn(0x7f400806a0f0 msgr2=0x7f40080aef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:30.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.900+0000 7f400dee0640 1 -- 172.21.15.154:0/882159317 shutdown_connections 2023-09-23T03:27:30.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:30.901+0000 7f400dee0640 1 -- 172.21.15.154:0/882159317 wait complete. 2023-09-23T03:27:30.913 INFO:tasks.ceph.mds.a.smithi154.stderr: -6614> 2023-09-23T03:26:34.434+0000 7fd09cd76640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-09-23T03:27:30.920 INFO:tasks.ceph.mds.a.smithi154.stderr: -4650> 2023-09-23T03:26:44.854+0000 7fd09cd76640 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2023-09-23T03:27:30.921 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-09-23T03:27:30.922 INFO:tasks.ceph.mds.a.smithi154.stderr: -4604> 2023-09-23T03:26:44.855+0000 7fd09cd76640 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x4 2023-09-23T03:27:30.922 INFO:tasks.ceph.mds.a.smithi154.stderr: -4419> 2023-09-23T03:26:44.856+0000 7fd09cd76640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /stripey/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55dfe04e8100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x10000000000:[<0x1/stripey v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-09-23T03:27:30.922 INFO:tasks.ceph.mds.a.smithi154.stderr: -4155> 2023-09-23T03:26:44.857+0000 7fd09cd76640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x4 [...2,head] /lost+found/ auth v1 pv6 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55dfe04e8680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x4:[<0x1/lost+found v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-09-23T03:27:30.922 INFO:tasks.ceph.mds.a.smithi154.stderr: -4120> 2023-09-23T03:26:44.857+0000 7fd09cd76640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fb [2,head] /stripey/flushed_file_6 auth v1 ap=1 DIRTYPARENT s=1572864 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55dfe052ac00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(16)0x100000001fb:[<0x10000000000/flushed_file_6 v68>,<0x1/stripey v75>]//[15]","memoryvalue":"(16)0x100000001fb:[<0x10000000000/flushed_file_6 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:27:30.923 INFO:tasks.ceph.mds.a.smithi154.stderr: -4072> 2023-09-23T03:26:44.857+0000 7fd09cd76640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f9 [2,head] /stripey/flushed_file_4 auth v1 ap=1 DIRTYPARENT s=1441792 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55dfe052b700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(16)0x100000001f9:[<0x10000000000/flushed_file_4 v70>,<0x1/stripey v75>]//[15]","memoryvalue":"(16)0x100000001f9:[<0x10000000000/flushed_file_4 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:27:30.923 INFO:tasks.ceph.mds.a.smithi154.stderr: -4048> 2023-09-23T03:26:44.857+0000 7fd09cd76640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f7 [2,head] /stripey/flushed_file_2 auth v1 ap=1 DIRTYPARENT s=917504 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55dfe0564100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(16)0x100000001f7:[<0x10000000000/flushed_file_2 v56>,<0x1/stripey v75>]//[15]","memoryvalue":"(16)0x100000001f7:[<0x10000000000/flushed_file_2 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:27:30.923 INFO:tasks.ceph.mds.a.smithi154.stderr: -4025> 2023-09-23T03:26:44.858+0000 7fd09cd76640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f6 [2,head] /stripey/flushed_file_1 auth v1 ap=1 DIRTYPARENT s=524288 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55dfe0564680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(16)0x100000001f6:[<0x10000000000/flushed_file_1 v50>,<0x1/stripey v75>]//[15]","memoryvalue":"(16)0x100000001f6:[<0x10000000000/flushed_file_1 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:27:30.923 INFO:tasks.ceph.mds.a.smithi154.stderr: -4000> 2023-09-23T03:26:44.858+0000 7fd09cd76640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /stripey/flushed_file_0 auth v1 ap=1 DIRTYPARENT s=1048576 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55dfe0564c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(16)0x10000000001:[<0x10000000000/flushed_file_0 v38>,<0x1/stripey v75>]//[15]","memoryvalue":"(16)0x10000000001:[<0x10000000000/flushed_file_0 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:27:30.923 INFO:tasks.ceph.mds.a.smithi154.stderr: -3983> 2023-09-23T03:26:44.858+0000 7fd09cd76640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f8 [2,head] /stripey/flushed_file_3 auth v1 ap=1 DIRTYPARENT s=917503 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55dfe0563b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(16)0x100000001f8:[<0x10000000000/flushed_file_3 v64>,<0x1/stripey v75>]//[15]","memoryvalue":"(16)0x100000001f8:[<0x10000000000/flushed_file_3 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:27:30.923 INFO:tasks.ceph.mds.a.smithi154.stderr: -3964> 2023-09-23T03:26:44.860+0000 7fd09cd76640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fa [2,head] /stripey/flushed_file_5 auth v1 ap=1 DIRTYPARENT s=1441793 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55dfe052b180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(16)0x100000001fa:[<0x10000000000/flushed_file_5 v54>,<0x1/stripey v75>]//[15]","memoryvalue":"(16)0x100000001fa:[<0x10000000000/flushed_file_5 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-09-23T03:27:30.923 INFO:tasks.ceph.mds.a.smithi154.stderr: -3949> 2023-09-23T03:26:44.860+0000 7fd09cd76640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fc [2,head] /lost+found/100000001fc auth v1 ap=1 DIRTYPARENT s=524288 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55dfe0565180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(16)0x100000001fc:[<0x4/100000001fc v1>,<0x1/lost+found v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2023-09-23T03:27:30.924 INFO:tasks.ceph.mds.a.smithi154.stderr: -3460> 2023-09-23T03:26:44.867+0000 7fd09cd76640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v7 ap=5 snaprealm=0x55dfe047a900 DIRTYPARENT f(v1 2=0+2) n(v0 rc2023-09-23T03:26:44.576922+0000 2=0+2)/n() (idft sync r=1) (inest sync r=1) (ifile sync->mix r=1) (iversion lock) caps={8643=pLsXs/-@3} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x55dfe04e7600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(14)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 2=0+2)","ondisk_value.rstat":"n(v0 3=0+3)","memory_value.dirstat":"f(v1 2=0+2)","memory_value.rstat":"n(v0 rc2023-09-23T03:26:44.576922+0000 2=0+2)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-09-23T03:27:30.974 INFO:tasks.ceph.mds.a.smithi154.stdout:starting mds.a at 2023-09-23T03:27:31.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.117+0000 7ff124f55640 1 Processor -- start 2023-09-23T03:27:31.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.117+0000 7ff124f55640 1 -- start start 2023-09-23T03:27:31.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.120+0000 7ff124f55640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff12015bb30 0x7ff1200caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:31.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.120+0000 7ff124f55640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff12015b6a0 con 0x7ff12015aba0 2023-09-23T03:27:31.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.120+0000 7ff124f55640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff12015f0b0 con 0x7ff1200cb3e0 2023-09-23T03:27:31.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.120+0000 7ff124f55640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1200cb870 con 0x7ff12015bb30 2023-09-23T03:27:31.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.120+0000 7ff117fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff12015bb30 0x7ff1200caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:31.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.120+0000 7ff11e59c640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff12015aba0 0x7ff12015e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:35544/0 (socket says 172.21.15.154:35544) 2023-09-23T03:27:31.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.120+0000 7ff11e59c640 1 -- 172.21.15.154:0/1288774698 learned_addr learned my addr 172.21.15.154:0/1288774698 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:27:31.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.120+0000 7ff117fff640 1 -- 172.21.15.154:0/1288774698 >> v1:172.21.15.190:6789/0 conn(0x7ff1200cb3e0 legacy=0x7ff1200cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:31.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.120+0000 7ff117fff640 1 -- 172.21.15.154:0/1288774698 >> v1:172.21.15.154:6789/0 conn(0x7ff12015aba0 legacy=0x7ff12015e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:31.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.120+0000 7ff117fff640 1 -- 172.21.15.154:0/1288774698 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff12015f390 con 0x7ff12015bb30 2023-09-23T03:27:31.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.121+0000 7ff117fff640 1 --2- 172.21.15.154:0/1288774698 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff12015bb30 0x7ff1200caea0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7ff1080097a0 tx=0x7ff10802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae99c9212a7be86 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:31.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.121+0000 7ff11dd9b640 1 -- 172.21.15.154:0/1288774698 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff108031830 con 0x7ff12015bb30 2023-09-23T03:27:31.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.121+0000 7ff11dd9b640 1 -- 172.21.15.154:0/1288774698 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff108031990 con 0x7ff12015bb30 2023-09-23T03:27:31.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.121+0000 7ff11dd9b640 1 -- 172.21.15.154:0/1288774698 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff108031d20 con 0x7ff12015bb30 2023-09-23T03:27:31.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.122+0000 7ff124f55640 1 -- 172.21.15.154:0/1288774698 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff12015bb30 msgr2=0x7ff1200caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:31.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.122+0000 7ff124f55640 1 --2- 172.21.15.154:0/1288774698 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff12015bb30 0x7ff1200caea0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7ff1080097a0 tx=0x7ff10802b8d0 comp rx=0 tx=0).stop 2023-09-23T03:27:31.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.122+0000 7ff124f55640 1 -- 172.21.15.154:0/1288774698 shutdown_connections 2023-09-23T03:27:31.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.122+0000 7ff124f55640 1 --2- 172.21.15.154:0/1288774698 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff12015bb30 0x7ff1200caea0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:31.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.122+0000 7ff124f55640 1 -- 172.21.15.154:0/1288774698 >> 172.21.15.154:0/1288774698 conn(0x7ff12006c010 msgr2=0x7ff12006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:31.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.122+0000 7ff124f55640 1 -- 172.21.15.154:0/1288774698 shutdown_connections 2023-09-23T03:27:31.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.122+0000 7ff124f55640 1 -- 172.21.15.154:0/1288774698 wait complete. 2023-09-23T03:27:31.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.124+0000 7ff124f55640 1 Processor -- start 2023-09-23T03:27:31.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.124+0000 7ff124f55640 1 -- start start 2023-09-23T03:27:31.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.125+0000 7ff124f55640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1200cb3e0 0x7ff1201429b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:31.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.125+0000 7ff124f55640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff12015aba0 0x7ff120152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:31.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.125+0000 7ff124f55640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff120152ab0 0x7ff1200701b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:31.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.125+0000 7ff124f55640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff120153c00 con 0x7ff1200cb3e0 2023-09-23T03:27:31.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.125+0000 7ff124f55640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff120142ef0 con 0x7ff12015aba0 2023-09-23T03:27:31.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.125+0000 7ff124f55640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff120152ff0 con 0x7ff120152ab0 2023-09-23T03:27:31.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.125+0000 7ff11e59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1200cb3e0 0x7ff1201429b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:31.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.125+0000 7ff11e59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1200cb3e0 0x7ff1201429b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47040/0 (socket says 172.21.15.154:47040) 2023-09-23T03:27:31.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.125+0000 7ff11e59c640 1 -- 172.21.15.154:0/1304463398 learned_addr learned my addr 172.21.15.154:0/1304463398 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:31.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.126+0000 7ff117fff640 1 --2- 172.21.15.154:0/1304463398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff12015aba0 0x7ff120152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:31.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.126+0000 7ff11e59c640 1 -- 172.21.15.154:0/1304463398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff120152ab0 msgr2=0x7ff1200701b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:31.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.126+0000 7ff11e59c640 1 --2- 172.21.15.154:0/1304463398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff120152ab0 0x7ff1200701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:31.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.126+0000 7ff11e59c640 1 -- 172.21.15.154:0/1304463398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff12015aba0 msgr2=0x7ff120152570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:31.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.126+0000 7ff11e59c640 1 --2- 172.21.15.154:0/1304463398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff12015aba0 0x7ff120152570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:31.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.126+0000 7ff11e59c640 1 -- 172.21.15.154:0/1304463398 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff108009400 con 0x7ff1200cb3e0 2023-09-23T03:27:31.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.126+0000 7ff11e59c640 1 --2- 172.21.15.154:0/1304463398 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1200cb3e0 0x7ff1201429b0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7ff11000c480 tx=0x7ff11000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:31.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.126+0000 7ff116ffd640 1 -- 172.21.15.154:0/1304463398 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff110005ce0 con 0x7ff1200cb3e0 2023-09-23T03:27:31.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.126+0000 7ff116ffd640 1 -- 172.21.15.154:0/1304463398 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff110005e40 con 0x7ff1200cb3e0 2023-09-23T03:27:31.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.126+0000 7ff116ffd640 1 -- 172.21.15.154:0/1304463398 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1100072f0 con 0x7ff1200cb3e0 2023-09-23T03:27:31.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.126+0000 7ff124f55640 1 -- 172.21.15.154:0/1304463398 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1201532d0 con 0x7ff1200cb3e0 2023-09-23T03:27:31.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.126+0000 7ff124f55640 1 -- 172.21.15.154:0/1304463398 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff120153890 con 0x7ff1200cb3e0 2023-09-23T03:27:31.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.127+0000 7ff124f55640 1 -- 172.21.15.154:0/1304463398 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff12004e920 con 0x7ff1200cb3e0 2023-09-23T03:27:31.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.128+0000 7ff116ffd640 1 -- 172.21.15.154:0/1304463398 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7ff110007450 con 0x7ff1200cb3e0 2023-09-23T03:27:31.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.129+0000 7ff116ffd640 1 --2- 172.21.15.154:0/1304463398 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff10006af60 0x7ff10006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:31.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.130+0000 7ff116ffd640 1 -- 172.21.15.154:0/1304463398 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6777+0+0 (secure 0 0 0) 0x7ff110088ff0 con 0x7ff1200cb3e0 2023-09-23T03:27:31.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.130+0000 7ff117fff640 1 --2- 172.21.15.154:0/1304463398 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff10006af60 0x7ff10006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:31.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.130+0000 7ff117fff640 1 --2- 172.21.15.154:0/1304463398 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff10006af60 0x7ff10006d420 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff108036000 tx=0x7ff108008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:31.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.134+0000 7ff116ffd640 1 -- 172.21.15.154:0/1304463398 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7ff1100bba50 con 0x7ff1200cb3e0 2023-09-23T03:27:31.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.362+0000 7ff124f55640 1 -- 172.21.15.154:0/1304463398 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff1200cbc10 con 0x7ff1200cb3e0 2023-09-23T03:27:31.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.900+0000 7ff116ffd640 1 -- 172.21.15.154:0/1304463398 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v101) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff1100170a0 con 0x7ff1200cb3e0 2023-09-23T03:27:31.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.904+0000 7ff124f55640 1 -- 172.21.15.154:0/1304463398 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff10006af60 msgr2=0x7ff10006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:31.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.904+0000 7ff124f55640 1 --2- 172.21.15.154:0/1304463398 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff10006af60 0x7ff10006d420 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff108036000 tx=0x7ff108008030 comp rx=0 tx=0).stop 2023-09-23T03:27:31.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.904+0000 7ff124f55640 1 -- 172.21.15.154:0/1304463398 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1200cb3e0 msgr2=0x7ff1201429b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:31.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.904+0000 7ff124f55640 1 --2- 172.21.15.154:0/1304463398 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1200cb3e0 0x7ff1201429b0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7ff11000c480 tx=0x7ff11000c940 comp rx=0 tx=0).stop 2023-09-23T03:27:31.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.904+0000 7ff124f55640 1 -- 172.21.15.154:0/1304463398 shutdown_connections 2023-09-23T03:27:31.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.904+0000 7ff124f55640 1 --2- 172.21.15.154:0/1304463398 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff10006af60 0x7ff10006d420 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:31.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.904+0000 7ff124f55640 1 --2- 172.21.15.154:0/1304463398 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff120152ab0 0x7ff1200701b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:31.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.904+0000 7ff124f55640 1 --2- 172.21.15.154:0/1304463398 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff12015aba0 0x7ff120152570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:31.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.904+0000 7ff124f55640 1 --2- 172.21.15.154:0/1304463398 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1200cb3e0 0x7ff1201429b0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:31.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.905+0000 7ff124f55640 1 -- 172.21.15.154:0/1304463398 >> 172.21.15.154:0/1304463398 conn(0x7ff12006c010 msgr2=0x7ff12015d870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:31.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.905+0000 7ff124f55640 1 -- 172.21.15.154:0/1304463398 shutdown_connections 2023-09-23T03:27:31.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:31.905+0000 7ff124f55640 1 -- 172.21.15.154:0/1304463398 wait complete. 2023-09-23T03:27:31.926 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-09-23T03:27:32.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.149+0000 7f6a7f59e640 1 Processor -- start 2023-09-23T03:27:32.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.149+0000 7f6a7f59e640 1 -- start start 2023-09-23T03:27:32.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.150+0000 7f6a7f59e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a8015c940 0x7f6a800d1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:32.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.150+0000 7f6a7f59e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6a800cde90 0x7f6a800ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:32.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.150+0000 7f6a7f59e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a800d0290 con 0x7f6a800cf020 2023-09-23T03:27:32.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.150+0000 7f6a7f59e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a800d1870 con 0x7f6a800cde90 2023-09-23T03:27:32.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.150+0000 7f6a7f59e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a800d0570 con 0x7f6a8015c940 2023-09-23T03:27:32.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.151+0000 7f6a7dd9b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6a800cde90 0x7f6a800ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:32.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.151+0000 7f6a7ed9d640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f6a800cf020 0x7f6a8006ad50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:35558/0 (socket says 172.21.15.154:35558) 2023-09-23T03:27:32.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.151+0000 7f6a7ed9d640 1 -- 172.21.15.154:0/3364781900 learned_addr learned my addr 172.21.15.154:0/3364781900 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:27:32.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.151+0000 7f6a7dd9b640 1 --2- 172.21.15.154:0/3364781900 >> v2:172.21.15.154:3300/0 conn(0x7f6a800cde90 0x7f6a800ce2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47052/0 (socket says 172.21.15.154:47052) 2023-09-23T03:27:32.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.151+0000 7f6a7e59c640 1 --2- 172.21.15.154:0/3364781900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a8015c940 0x7f6a800d1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:32.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.151+0000 7f6a7dd9b640 1 -- 172.21.15.154:0/3364781900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a8015c940 msgr2=0x7f6a800d1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:32.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.151+0000 7f6a7dd9b640 1 --2- 172.21.15.154:0/3364781900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a8015c940 0x7f6a800d1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:32.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.151+0000 7f6a7dd9b640 1 -- 172.21.15.154:0/3364781900 >> v1:172.21.15.154:6789/0 conn(0x7f6a800cf020 legacy=0x7f6a8006ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:32.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.151+0000 7f6a7dd9b640 1 -- 172.21.15.154:0/3364781900 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a800d0850 con 0x7f6a800cde90 2023-09-23T03:27:32.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.152+0000 7f6a7dd9b640 1 --2- 172.21.15.154:0/3364781900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a800cde90 0x7f6a800ce2f0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f6a680097a0 tx=0x7f6a6802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f997af68c71bff server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:32.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.153+0000 7f6a7d59a640 1 -- 172.21.15.154:0/3364781900 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6802d040 con 0x7f6a800cde90 2023-09-23T03:27:32.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.153+0000 7f6a7d59a640 1 -- 172.21.15.154:0/3364781900 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a68031810 con 0x7f6a800cde90 2023-09-23T03:27:32.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.153+0000 7f6a7d59a640 1 -- 172.21.15.154:0/3364781900 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a68031ba0 con 0x7f6a800cde90 2023-09-23T03:27:32.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.153+0000 7f6a7f59e640 1 -- 172.21.15.154:0/3364781900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a800cde90 msgr2=0x7f6a800ce2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:32.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.153+0000 7f6a7f59e640 1 --2- 172.21.15.154:0/3364781900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a800cde90 0x7f6a800ce2f0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f6a680097a0 tx=0x7f6a6802b8b0 comp rx=0 tx=0).stop 2023-09-23T03:27:32.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.153+0000 7f6a7f59e640 1 -- 172.21.15.154:0/3364781900 shutdown_connections 2023-09-23T03:27:32.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.153+0000 7f6a7f59e640 1 --2- 172.21.15.154:0/3364781900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a800cde90 0x7f6a800ce2f0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:32.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.153+0000 7f6a7f59e640 1 --2- 172.21.15.154:0/3364781900 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a8015c940 0x7f6a800d1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:32.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.153+0000 7f6a7f59e640 1 -- 172.21.15.154:0/3364781900 >> 172.21.15.154:0/3364781900 conn(0x7f6a8009ba90 msgr2=0x7f6a800d26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:32.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.153+0000 7f6a7f59e640 1 -- 172.21.15.154:0/3364781900 shutdown_connections 2023-09-23T03:27:32.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.153+0000 7f6a7f59e640 1 -- 172.21.15.154:0/3364781900 wait complete. 2023-09-23T03:27:32.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.155+0000 7f6a7f59e640 1 Processor -- start 2023-09-23T03:27:32.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.155+0000 7f6a7f59e640 1 -- start start 2023-09-23T03:27:32.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.156+0000 7f6a7f59e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a800cde90 0x7f6a801f5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:32.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.156+0000 7f6a7f59e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a800cf020 0x7f6a800b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:32.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.156+0000 7f6a7f59e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a8015c940 0x7f6a801e70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:32.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.156+0000 7f6a7f59e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a801f6270 con 0x7f6a800cf020 2023-09-23T03:27:32.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.156+0000 7f6a7f59e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a801f6510 con 0x7f6a800cde90 2023-09-23T03:27:32.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.156+0000 7f6a7f59e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a801f67f0 con 0x7f6a8015c940 2023-09-23T03:27:32.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.157+0000 7f6a7dd9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a800cf020 0x7f6a800b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:32.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.157+0000 7f6a7dd9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a800cf020 0x7f6a800b05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47058/0 (socket says 172.21.15.154:47058) 2023-09-23T03:27:32.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.157+0000 7f6a7dd9b640 1 -- 172.21.15.154:0/1236587484 learned_addr learned my addr 172.21.15.154:0/1236587484 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:32.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.157+0000 7f6a7dd9b640 1 -- 172.21.15.154:0/1236587484 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a8015c940 msgr2=0x7f6a801e70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:32.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.157+0000 7f6a7dd9b640 1 --2- 172.21.15.154:0/1236587484 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a8015c940 0x7f6a801e70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:32.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.157+0000 7f6a7dd9b640 1 -- 172.21.15.154:0/1236587484 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a800cde90 msgr2=0x7f6a801f5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:32.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.157+0000 7f6a7e59c640 1 --2- 172.21.15.154:0/1236587484 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a800cde90 0x7f6a801f5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:32.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.157+0000 7f6a7dd9b640 1 --2- 172.21.15.154:0/1236587484 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a800cde90 0x7f6a801f5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:32.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.157+0000 7f6a7dd9b640 1 -- 172.21.15.154:0/1236587484 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a68009400 con 0x7f6a800cf020 2023-09-23T03:27:32.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.157+0000 7f6a7dd9b640 1 --2- 172.21.15.154:0/1236587484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a800cf020 0x7f6a800b05b0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f6a68000c00 tx=0x7f6a680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:32.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.158+0000 7f6a76ffd640 1 -- 172.21.15.154:0/1236587484 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6803ad60 con 0x7f6a800cf020 2023-09-23T03:27:32.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.158+0000 7f6a76ffd640 1 -- 172.21.15.154:0/1236587484 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a680443f0 con 0x7f6a800cf020 2023-09-23T03:27:32.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.158+0000 7f6a76ffd640 1 -- 172.21.15.154:0/1236587484 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a680447a0 con 0x7f6a800cf020 2023-09-23T03:27:32.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.158+0000 7f6a7f59e640 1 -- 172.21.15.154:0/1236587484 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a801f6ad0 con 0x7f6a800cf020 2023-09-23T03:27:32.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.158+0000 7f6a7f59e640 1 -- 172.21.15.154:0/1236587484 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a801f2780 con 0x7f6a800cf020 2023-09-23T03:27:32.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.159+0000 7f6a7f59e640 1 -- 172.21.15.154:0/1236587484 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a800d1d80 con 0x7f6a800cf020 2023-09-23T03:27:32.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.160+0000 7f6a76ffd640 1 -- 172.21.15.154:0/1236587484 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f6a6804d450 con 0x7f6a800cf020 2023-09-23T03:27:32.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.161+0000 7f6a76ffd640 1 --2- 172.21.15.154:0/1236587484 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6a6006af30 0x7f6a6006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:32.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.161+0000 7f6a76ffd640 1 -- 172.21.15.154:0/1236587484 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f6a680abe30 con 0x7f6a800cf020 2023-09-23T03:27:32.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.161+0000 7f6a7e59c640 1 --2- 172.21.15.154:0/1236587484 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6a6006af30 0x7f6a6006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:32.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.162+0000 7f6a7e59c640 1 --2- 172.21.15.154:0/1236587484 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6a6006af30 0x7f6a6006d3f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f6a70009f80 tx=0x7f6a7000b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:32.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.166+0000 7f6a76ffd640 1 -- 172.21.15.154:0/1236587484 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f6a68076a30 con 0x7f6a800cf020 2023-09-23T03:27:32.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.358+0000 7f6a7f59e640 1 -- 172.21.15.154:0/1236587484 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6a801f2cc0 con 0x7f6a800cf020 2023-09-23T03:27:32.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.904+0000 7f6a76ffd640 1 -- 172.21.15.154:0/1236587484 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v116) v1 ==== 181+0+0 (secure 0 0 0) 0x7f6a801f2cc0 con 0x7f6a800cf020 2023-09-23T03:27:32.904 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_metadata' removed 2023-09-23T03:27:32.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.908+0000 7f6a7f59e640 1 -- 172.21.15.154:0/1236587484 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6a6006af30 msgr2=0x7f6a6006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:32.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.908+0000 7f6a7f59e640 1 --2- 172.21.15.154:0/1236587484 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6a6006af30 0x7f6a6006d3f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f6a70009f80 tx=0x7f6a7000b040 comp rx=0 tx=0).stop 2023-09-23T03:27:32.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.908+0000 7f6a7f59e640 1 -- 172.21.15.154:0/1236587484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a800cf020 msgr2=0x7f6a800b05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:32.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.908+0000 7f6a7f59e640 1 --2- 172.21.15.154:0/1236587484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a800cf020 0x7f6a800b05b0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f6a68000c00 tx=0x7f6a680046c0 comp rx=0 tx=0).stop 2023-09-23T03:27:32.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.908+0000 7f6a7f59e640 1 -- 172.21.15.154:0/1236587484 shutdown_connections 2023-09-23T03:27:32.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.908+0000 7f6a7f59e640 1 --2- 172.21.15.154:0/1236587484 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6a6006af30 0x7f6a6006d3f0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:32.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.908+0000 7f6a7f59e640 1 --2- 172.21.15.154:0/1236587484 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a8015c940 0x7f6a801e70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:32.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.908+0000 7f6a7f59e640 1 --2- 172.21.15.154:0/1236587484 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a800cf020 0x7f6a800b05b0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:32.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.908+0000 7f6a7f59e640 1 --2- 172.21.15.154:0/1236587484 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a800cde90 0x7f6a801f5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:32.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.908+0000 7f6a7f59e640 1 -- 172.21.15.154:0/1236587484 >> 172.21.15.154:0/1236587484 conn(0x7f6a8009ba90 msgr2=0x7f6a800b56f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:32.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.909+0000 7f6a7f59e640 1 -- 172.21.15.154:0/1236587484 shutdown_connections 2023-09-23T03:27:32.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:32.909+0000 7f6a7f59e640 1 -- 172.21.15.154:0/1236587484 wait complete. 2023-09-23T03:27:32.933 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-09-23T03:27:33.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.150+0000 7f0c53fff640 1 Processor -- start 2023-09-23T03:27:33.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.151+0000 7f0c53fff640 1 -- start start 2023-09-23T03:27:33.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.152+0000 7f0c53fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f0c5415e350 0x7f0c540cad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:33.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.152+0000 7f0c53fff640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c5415ac40 con 0x7f0c540ce2a0 2023-09-23T03:27:33.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.152+0000 7f0c53fff640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c5415e860 con 0x7f0c5415a860 2023-09-23T03:27:33.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.152+0000 7f0c53fff640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c5415eb20 con 0x7f0c5415e350 2023-09-23T03:27:33.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.153+0000 7f0c537fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f0c540ce2a0 0x7f0c540b25e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:35562/0 (socket says 172.21.15.154:35562) 2023-09-23T03:27:33.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.153+0000 7f0c537fe640 1 -- 172.21.15.154:0/3041024727 learned_addr learned my addr 172.21.15.154:0/3041024727 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:27:33.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.153+0000 7f0c4bfff640 1 --2- 172.21.15.154:0/3041024727 >> v2:172.21.15.190:3300/0 conn(0x7f0c5415e350 0x7f0c540cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:33.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.153+0000 7f0c527fc640 1 -- 172.21.15.154:0/3041024727 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3078846225 0 0) 0x7f0c5415ac40 con 0x7f0c540ce2a0 2023-09-23T03:27:33.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.153+0000 7f0c527fc640 1 -- 172.21.15.154:0/3041024727 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c40003520 con 0x7f0c540ce2a0 2023-09-23T03:27:33.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.153+0000 7f0c527fc640 1 -- 172.21.15.154:0/3041024727 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3610200247 0 0) 0x7f0c40003520 con 0x7f0c540ce2a0 2023-09-23T03:27:33.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.153+0000 7f0c527fc640 1 -- 172.21.15.154:0/3041024727 >> v2:172.21.15.190:3300/0 conn(0x7f0c5415e350 msgr2=0x7f0c540cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:33.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.153+0000 7f0c527fc640 1 --2- 172.21.15.154:0/3041024727 >> v2:172.21.15.190:3300/0 conn(0x7f0c5415e350 0x7f0c540cad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:33.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.153+0000 7f0c527fc640 1 -- 172.21.15.154:0/3041024727 >> v1:172.21.15.190:6789/0 conn(0x7f0c5415a860 legacy=0x7f0c540cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:33.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.153+0000 7f0c527fc640 1 -- 172.21.15.154:0/3041024727 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c5415ee00 con 0x7f0c540ce2a0 2023-09-23T03:27:33.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.154+0000 7f0c527fc640 1 -- 172.21.15.154:0/3041024727 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f0c4c0019f0 con 0x7f0c540ce2a0 2023-09-23T03:27:33.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.154+0000 7f0c527fc640 1 -- 172.21.15.154:0/3041024727 >> v1:172.21.15.154:6789/0 conn(0x7f0c540ce2a0 legacy=0x7f0c540b25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:33.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.154+0000 7f0c527fc640 1 --2- 172.21.15.154:0/3041024727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c400039c0 0x7f0c40003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:33.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.154+0000 7f0c527fc640 1 --2- 172.21.15.154:0/3041024727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c400047c0 0x7f0c40004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:33.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.154+0000 7f0c527fc640 1 --2- 172.21.15.154:0/3041024727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c40005180 0x7f0c4000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:33.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.154+0000 7f0c527fc640 1 -- 172.21.15.154:0/3041024727 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5415eb20 con 0x7f0c40005180 2023-09-23T03:27:33.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.154+0000 7f0c527fc640 1 -- 172.21.15.154:0/3041024727 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4000c080 con 0x7f0c400047c0 2023-09-23T03:27:33.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.154+0000 7f0c527fc640 1 -- 172.21.15.154:0/3041024727 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4000c240 con 0x7f0c400039c0 2023-09-23T03:27:33.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.154+0000 7f0c537fe640 1 --2- 172.21.15.154:0/3041024727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c40005180 0x7f0c4000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:33.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.154+0000 7f0c52ffd640 1 --2- 172.21.15.154:0/3041024727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c400039c0 0x7f0c40003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:33.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.154+0000 7f0c4bfff640 1 --2- 172.21.15.154:0/3041024727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c400047c0 0x7f0c40004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:33.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.154+0000 7f0c537fe640 1 -- 172.21.15.154:0/3041024727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c400039c0 msgr2=0x7f0c40003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:33.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.154+0000 7f0c537fe640 1 --2- 172.21.15.154:0/3041024727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c400039c0 0x7f0c40003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:33.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.154+0000 7f0c537fe640 1 -- 172.21.15.154:0/3041024727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c400047c0 msgr2=0x7f0c40004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:33.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.154+0000 7f0c537fe640 1 --2- 172.21.15.154:0/3041024727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c400047c0 0x7f0c40004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:33.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.154+0000 7f0c537fe640 1 -- 172.21.15.154:0/3041024727 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c40003520 con 0x7f0c40005180 2023-09-23T03:27:33.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.154+0000 7f0c4bfff640 1 --2- 172.21.15.154:0/3041024727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c400047c0 0x7f0c40004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:33.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.155+0000 7f0c537fe640 1 -- 172.21.15.154:0/3041024727 reap_dead start 2023-09-23T03:27:33.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.156+0000 7f0c537fe640 1 --2- 172.21.15.154:0/3041024727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c40005180 0x7f0c4000b980 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c00a750 tx=0x7f0c400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f3ab28fd06c541 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:33.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.156+0000 7f0c527fc640 1 -- 172.21.15.154:0/3041024727 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c034630 con 0x7f0c40005180 2023-09-23T03:27:33.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.156+0000 7f0c527fc640 1 -- 172.21.15.154:0/3041024727 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c4c02f070 con 0x7f0c40005180 2023-09-23T03:27:33.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.156+0000 7f0c527fc640 1 -- 172.21.15.154:0/3041024727 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c0349d0 con 0x7f0c40005180 2023-09-23T03:27:33.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.156+0000 7f0c53fff640 1 -- 172.21.15.154:0/3041024727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c40005180 msgr2=0x7f0c4000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:33.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.156+0000 7f0c53fff640 1 --2- 172.21.15.154:0/3041024727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c40005180 0x7f0c4000b980 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c00a750 tx=0x7f0c400057f0 comp rx=0 tx=0).stop 2023-09-23T03:27:33.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.157+0000 7f0c53fff640 1 -- 172.21.15.154:0/3041024727 shutdown_connections 2023-09-23T03:27:33.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.157+0000 7f0c53fff640 1 --2- 172.21.15.154:0/3041024727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c40005180 0x7f0c4000b980 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:33.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.157+0000 7f0c53fff640 1 -- 172.21.15.154:0/3041024727 >> 172.21.15.154:0/3041024727 conn(0x7f0c54069eb0 msgr2=0x7f0c5406a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:33.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.157+0000 7f0c53fff640 1 -- 172.21.15.154:0/3041024727 shutdown_connections 2023-09-23T03:27:33.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.157+0000 7f0c53fff640 1 -- 172.21.15.154:0/3041024727 wait complete. 2023-09-23T03:27:33.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.158+0000 7f0c53fff640 1 Processor -- start 2023-09-23T03:27:33.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.158+0000 7f0c53fff640 1 -- start start 2023-09-23T03:27:33.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.159+0000 7f0c53fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c40005180 0x7f0c540c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:33.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.159+0000 7f0c53fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c540c0de0 0x7f0c540b2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:33.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.159+0000 7f0c53fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c540b2970 0x7f0c541f1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:33.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.159+0000 7f0c53fff640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c540b2df0 con 0x7f0c540b2970 2023-09-23T03:27:33.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.159+0000 7f0c53fff640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c540c7790 con 0x7f0c540c0de0 2023-09-23T03:27:33.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.159+0000 7f0c53fff640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c540c7a70 con 0x7f0c40005180 2023-09-23T03:27:33.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.160+0000 7f0c537fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c540b2970 0x7f0c541f1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:33.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.160+0000 7f0c537fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c540b2970 0x7f0c541f1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:47080/0 (socket says 172.21.15.154:47080) 2023-09-23T03:27:33.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.160+0000 7f0c537fe640 1 -- 172.21.15.154:0/818367363 learned_addr learned my addr 172.21.15.154:0/818367363 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:33.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.160+0000 7f0c52ffd640 1 --2- 172.21.15.154:0/818367363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c40005180 0x7f0c540c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:33.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.160+0000 7f0c4bfff640 1 --2- 172.21.15.154:0/818367363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c540c0de0 0x7f0c540b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:33.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.160+0000 7f0c537fe640 1 -- 172.21.15.154:0/818367363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c40005180 msgr2=0x7f0c540c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:33.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.160+0000 7f0c537fe640 1 --2- 172.21.15.154:0/818367363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c40005180 0x7f0c540c9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:33.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.160+0000 7f0c537fe640 1 -- 172.21.15.154:0/818367363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c540c0de0 msgr2=0x7f0c540b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:33.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.160+0000 7f0c537fe640 1 --2- 172.21.15.154:0/818367363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c540c0de0 0x7f0c540b2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:33.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.160+0000 7f0c537fe640 1 -- 172.21.15.154:0/818367363 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c540c7d50 con 0x7f0c540b2970 2023-09-23T03:27:33.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.161+0000 7f0c537fe640 1 --2- 172.21.15.154:0/818367363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c540b2970 0x7f0c541f1f00 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c02cb70 tx=0x7f0c540b1e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:33.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.161+0000 7f0c4b7fe640 1 -- 172.21.15.154:0/818367363 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c034430 con 0x7f0c540b2970 2023-09-23T03:27:33.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.161+0000 7f0c4b7fe640 1 -- 172.21.15.154:0/818367363 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c4c034590 con 0x7f0c540b2970 2023-09-23T03:27:33.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.161+0000 7f0c4b7fe640 1 -- 172.21.15.154:0/818367363 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c034920 con 0x7f0c540b2970 2023-09-23T03:27:33.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.161+0000 7f0c53fff640 1 -- 172.21.15.154:0/818367363 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c540c8030 con 0x7f0c540b2970 2023-09-23T03:27:33.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.161+0000 7f0c53fff640 1 -- 172.21.15.154:0/818367363 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c540c73f0 con 0x7f0c540b2970 2023-09-23T03:27:33.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.162+0000 7f0c53fff640 1 -- 172.21.15.154:0/818367363 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c54067390 con 0x7f0c540b2970 2023-09-23T03:27:33.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.163+0000 7f0c4b7fe640 1 -- 172.21.15.154:0/818367363 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f0c4c031030 con 0x7f0c540b2970 2023-09-23T03:27:33.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.164+0000 7f0c4b7fe640 1 --2- 172.21.15.154:0/818367363 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0c3406af30 0x7f0c3406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:33.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.164+0000 7f0c4b7fe640 1 -- 172.21.15.154:0/818367363 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6339+0+0 (secure 0 0 0) 0x7f0c4c0ab4d0 con 0x7f0c540b2970 2023-09-23T03:27:33.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.164+0000 7f0c52ffd640 1 --2- 172.21.15.154:0/818367363 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0c3406af30 0x7f0c3406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:33.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.165+0000 7f0c52ffd640 1 --2- 172.21.15.154:0/818367363 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0c3406af30 0x7f0c3406d3f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f0c44005410 tx=0x7f0c44009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:33.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.168+0000 7f0c4b7fe640 1 -- 172.21.15.154:0/818367363 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f0c4c0761f0 con 0x7f0c540b2970 2023-09-23T03:27:33.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.356+0000 7f0c53fff640 1 -- 172.21.15.154:0/818367363 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0c54071120 con 0x7f0c540b2970 2023-09-23T03:27:33.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.916+0000 7f0c4b7fe640 1 -- 172.21.15.154:0/818367363 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v117) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0c4c079810 con 0x7f0c540b2970 2023-09-23T03:27:33.918 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_data' removed 2023-09-23T03:27:33.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.920+0000 7f0c53fff640 1 -- 172.21.15.154:0/818367363 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0c3406af30 msgr2=0x7f0c3406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:33.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.920+0000 7f0c53fff640 1 --2- 172.21.15.154:0/818367363 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0c3406af30 0x7f0c3406d3f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f0c44005410 tx=0x7f0c44009040 comp rx=0 tx=0).stop 2023-09-23T03:27:33.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.920+0000 7f0c53fff640 1 -- 172.21.15.154:0/818367363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c540b2970 msgr2=0x7f0c541f1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:33.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.920+0000 7f0c53fff640 1 --2- 172.21.15.154:0/818367363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c540b2970 0x7f0c541f1f00 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c02cb70 tx=0x7f0c540b1e00 comp rx=0 tx=0).stop 2023-09-23T03:27:33.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.920+0000 7f0c53fff640 1 -- 172.21.15.154:0/818367363 shutdown_connections 2023-09-23T03:27:33.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.920+0000 7f0c53fff640 1 --2- 172.21.15.154:0/818367363 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f0c3406af30 0x7f0c3406d3f0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:33.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.920+0000 7f0c53fff640 1 --2- 172.21.15.154:0/818367363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0c540b2970 0x7f0c541f1f00 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:33.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.920+0000 7f0c53fff640 1 --2- 172.21.15.154:0/818367363 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f0c540c0de0 0x7f0c540b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:33.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.920+0000 7f0c53fff640 1 --2- 172.21.15.154:0/818367363 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f0c40005180 0x7f0c540c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:33.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.920+0000 7f0c53fff640 1 -- 172.21.15.154:0/818367363 >> 172.21.15.154:0/818367363 conn(0x7f0c54069eb0 msgr2=0x7f0c540cad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:33.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.921+0000 7f0c53fff640 1 -- 172.21.15.154:0/818367363 shutdown_connections 2023-09-23T03:27:33.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:33.921+0000 7f0c53fff640 1 -- 172.21.15.154:0/818367363 wait complete. 2023-09-23T03:27:33.951 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_extra cephfs_data_extra --yes-i-really-really-mean-it 2023-09-23T03:27:34.147 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.147+0000 7f10b8eaf640 1 Processor -- start 2023-09-23T03:27:34.147 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.147+0000 7f10b8eaf640 1 -- start start 2023-09-23T03:27:34.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.148+0000 7f10b8eaf640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10b41590b0 0x7f10b4071a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:34.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.148+0000 7f10b8eaf640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f10b40ca230 0x7f10b40cc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:34.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.148+0000 7f10b8eaf640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10b40c9190 con 0x7f10b4071f50 2023-09-23T03:27:34.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.149+0000 7f10b8eaf640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10b40c9e10 con 0x7f10b40ca230 2023-09-23T03:27:34.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.149+0000 7f10b8eaf640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10b40c9470 con 0x7f10b41590b0 2023-09-23T03:27:34.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.149+0000 7f10b2d9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f10b40ca230 0x7f10b40cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:34.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.149+0000 7f10b2d9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f10b40ca230 0x7f10b40cc020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54592/0 (socket says 172.21.15.154:54592) 2023-09-23T03:27:34.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.149+0000 7f10b1d9b640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f10b4071f50 0x7f10b40cac00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40730/0 (socket says 172.21.15.154:40730) 2023-09-23T03:27:34.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.149+0000 7f10b2d9d640 1 -- 172.21.15.154:0/442840468 learned_addr learned my addr 172.21.15.154:0/442840468 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:34.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.149+0000 7f10b259c640 1 --2- 172.21.15.154:0/442840468 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10b41590b0 0x7f10b4071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:34.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.149+0000 7f10b2d9d640 1 -- 172.21.15.154:0/442840468 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10b41590b0 msgr2=0x7f10b4071a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:34.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.149+0000 7f10b2d9d640 1 --2- 172.21.15.154:0/442840468 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10b41590b0 0x7f10b4071a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:34.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.149+0000 7f10b2d9d640 1 -- 172.21.15.154:0/442840468 >> v1:172.21.15.154:6789/0 conn(0x7f10b4071f50 legacy=0x7f10b40cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:34.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.149+0000 7f10b2d9d640 1 -- 172.21.15.154:0/442840468 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b40c9750 con 0x7f10b40ca230 2023-09-23T03:27:34.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.149+0000 7f10b159a640 1 -- 172.21.15.154:0/442840468 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2659918790 0 0) 0x7f10b40c9190 con 0x7f10b4071f50 2023-09-23T03:27:34.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.151+0000 7f10b2d9d640 1 --2- 172.21.15.154:0/442840468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10b40ca230 0x7f10b40cc020 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f10ac009870 tx=0x7f10ac02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7228f661f616e86 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:34.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.151+0000 7f10b159a640 1 -- 172.21.15.154:0/442840468 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ac030840 con 0x7f10b40ca230 2023-09-23T03:27:34.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.151+0000 7f10b159a640 1 -- 172.21.15.154:0/442840468 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10ac0309a0 con 0x7f10b40ca230 2023-09-23T03:27:34.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.151+0000 7f10b159a640 1 -- 172.21.15.154:0/442840468 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ac030d30 con 0x7f10b40ca230 2023-09-23T03:27:34.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.151+0000 7f10b8eaf640 1 -- 172.21.15.154:0/442840468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10b40ca230 msgr2=0x7f10b40cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:34.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.151+0000 7f10b8eaf640 1 --2- 172.21.15.154:0/442840468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10b40ca230 0x7f10b40cc020 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f10ac009870 tx=0x7f10ac02b980 comp rx=0 tx=0).stop 2023-09-23T03:27:34.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.152+0000 7f10b8eaf640 1 -- 172.21.15.154:0/442840468 shutdown_connections 2023-09-23T03:27:34.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.152+0000 7f10b8eaf640 1 --2- 172.21.15.154:0/442840468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10b40ca230 0x7f10b40cc020 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:34.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.152+0000 7f10b8eaf640 1 --2- 172.21.15.154:0/442840468 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10b41590b0 0x7f10b4071a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:34.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.152+0000 7f10b8eaf640 1 -- 172.21.15.154:0/442840468 >> 172.21.15.154:0/442840468 conn(0x7f10b406a520 msgr2=0x7f10b406bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:34.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.152+0000 7f10b8eaf640 1 -- 172.21.15.154:0/442840468 shutdown_connections 2023-09-23T03:27:34.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.152+0000 7f10b8eaf640 1 -- 172.21.15.154:0/442840468 wait complete. 2023-09-23T03:27:34.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.153+0000 7f10b8eaf640 1 Processor -- start 2023-09-23T03:27:34.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.154+0000 7f10b8eaf640 1 -- start start 2023-09-23T03:27:34.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.154+0000 7f10b8eaf640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10b40ca230 0x7f10b4156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:34.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.154+0000 7f10b8eaf640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10b41590b0 0x7f10b40c7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:34.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.154+0000 7f10b8eaf640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10b40c7c70 0x7f10b41e3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:34.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.154+0000 7f10b8eaf640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b41e5110 con 0x7f10b41590b0 2023-09-23T03:27:34.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.154+0000 7f10b8eaf640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b40c8120 con 0x7f10b40c7c70 2023-09-23T03:27:34.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.154+0000 7f10b8eaf640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10b41e2c90 con 0x7f10b40ca230 2023-09-23T03:27:34.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.155+0000 7f10b1d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10b41590b0 0x7f10b40c7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:34.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.155+0000 7f10b1d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10b41590b0 0x7f10b40c7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54596/0 (socket says 172.21.15.154:54596) 2023-09-23T03:27:34.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.155+0000 7f10b1d9b640 1 -- 172.21.15.154:0/1189853479 learned_addr learned my addr 172.21.15.154:0/1189853479 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:34.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.155+0000 7f10b2d9d640 1 --2- 172.21.15.154:0/1189853479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10b40c7c70 0x7f10b41e3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:34.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.155+0000 7f10b259c640 1 --2- 172.21.15.154:0/1189853479 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10b40ca230 0x7f10b4156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:34.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.155+0000 7f10b1d9b640 1 -- 172.21.15.154:0/1189853479 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10b40ca230 msgr2=0x7f10b4156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:34.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.155+0000 7f10b1d9b640 1 --2- 172.21.15.154:0/1189853479 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10b40ca230 0x7f10b4156f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:34.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.155+0000 7f10b1d9b640 1 -- 172.21.15.154:0/1189853479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10b40c7c70 msgr2=0x7f10b41e3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:34.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.155+0000 7f10b1d9b640 1 --2- 172.21.15.154:0/1189853479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10b40c7c70 0x7f10b41e3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:34.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.155+0000 7f10b1d9b640 1 -- 172.21.15.154:0/1189853479 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10ac0094d0 con 0x7f10b41590b0 2023-09-23T03:27:34.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.156+0000 7f10b1d9b640 1 --2- 172.21.15.154:0/1189853479 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10b41590b0 0x7f10b40c7730 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f109c00c4d0 tx=0x7f109c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:34.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.156+0000 7f10aaffd640 1 -- 172.21.15.154:0/1189853479 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c0074e0 con 0x7f10b41590b0 2023-09-23T03:27:34.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.156+0000 7f10aaffd640 1 -- 172.21.15.154:0/1189853479 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109c007660 con 0x7f10b41590b0 2023-09-23T03:27:34.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.156+0000 7f10aaffd640 1 -- 172.21.15.154:0/1189853479 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c004a20 con 0x7f10b41590b0 2023-09-23T03:27:34.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.156+0000 7f10b8eaf640 1 -- 172.21.15.154:0/1189853479 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10b41e4490 con 0x7f10b41590b0 2023-09-23T03:27:34.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.156+0000 7f10b8eaf640 1 -- 172.21.15.154:0/1189853479 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10b41e4a50 con 0x7f10b41590b0 2023-09-23T03:27:34.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.157+0000 7f10b8eaf640 1 -- 172.21.15.154:0/1189853479 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10b41e5250 con 0x7f10b41590b0 2023-09-23T03:27:34.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.158+0000 7f10aaffd640 1 -- 172.21.15.154:0/1189853479 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f109c004b80 con 0x7f10b41590b0 2023-09-23T03:27:34.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.159+0000 7f10aaffd640 1 --2- 172.21.15.154:0/1189853479 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f109406af30 0x7f109406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:34.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.159+0000 7f10aaffd640 1 -- 172.21.15.154:0/1189853479 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5925+0+0 (secure 0 0 0) 0x7f109c0889b0 con 0x7f10b41590b0 2023-09-23T03:27:34.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.159+0000 7f10b259c640 1 --2- 172.21.15.154:0/1189853479 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f109406af30 0x7f109406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:34.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.160+0000 7f10b259c640 1 --2- 172.21.15.154:0/1189853479 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f109406af30 0x7f109406d3f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f10a4009e90 tx=0x7f10a400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:34.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.163+0000 7f10aaffd640 1 -- 172.21.15.154:0/1189853479 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f10b41e5250 con 0x7f10b41590b0 2023-09-23T03:27:34.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.363+0000 7f10b8eaf640 1 -- 172.21.15.154:0/1189853479 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_extra", "pool2": "cephfs_data_extra", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f10b41e5250 con 0x7f10b41590b0 2023-09-23T03:27:34.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.928+0000 7f10aaffd640 1 -- 172.21.15.154:0/1189853479 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_extra", "pool2": "cephfs_data_extra", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_extra' removed v118) v1 ==== 187+0+0 (secure 0 0 0) 0x7f10b41e5250 con 0x7f10b41590b0 2023-09-23T03:27:34.929 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_data_extra' removed 2023-09-23T03:27:34.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.936+0000 7f10b8eaf640 1 -- 172.21.15.154:0/1189853479 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f109406af30 msgr2=0x7f109406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:34.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.936+0000 7f10b8eaf640 1 --2- 172.21.15.154:0/1189853479 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f109406af30 0x7f109406d3f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f10a4009e90 tx=0x7f10a400b040 comp rx=0 tx=0).stop 2023-09-23T03:27:34.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.936+0000 7f10b8eaf640 1 -- 172.21.15.154:0/1189853479 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10b41590b0 msgr2=0x7f10b40c7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:34.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.936+0000 7f10b8eaf640 1 --2- 172.21.15.154:0/1189853479 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10b41590b0 0x7f10b40c7730 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f109c00c4d0 tx=0x7f109c00c990 comp rx=0 tx=0).stop 2023-09-23T03:27:34.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.937+0000 7f10b8eaf640 1 -- 172.21.15.154:0/1189853479 shutdown_connections 2023-09-23T03:27:34.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.937+0000 7f10b8eaf640 1 --2- 172.21.15.154:0/1189853479 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f109406af30 0x7f109406d3f0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:34.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.937+0000 7f10b8eaf640 1 --2- 172.21.15.154:0/1189853479 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10b40c7c70 0x7f10b41e3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:34.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.937+0000 7f10b8eaf640 1 --2- 172.21.15.154:0/1189853479 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10b41590b0 0x7f10b40c7730 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:34.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.937+0000 7f10b8eaf640 1 --2- 172.21.15.154:0/1189853479 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10b40ca230 0x7f10b4156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:34.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.937+0000 7f10b8eaf640 1 -- 172.21.15.154:0/1189853479 >> 172.21.15.154:0/1189853479 conn(0x7f10b406a520 msgr2=0x7f10b415bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:34.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.938+0000 7f10b8eaf640 1 -- 172.21.15.154:0/1189853479 shutdown_connections 2023-09-23T03:27:34.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:34.938+0000 7f10b8eaf640 1 -- 172.21.15.154:0/1189853479 wait complete. 2023-09-23T03:27:34.962 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-09-23T03:27:35.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.164+0000 7f9b25a90640 1 Processor -- start 2023-09-23T03:27:35.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.164+0000 7f9b25a90640 1 -- start start 2023-09-23T03:27:35.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.166+0000 7f9b25a90640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f9b20160f80 0x7f9b20069cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:35.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.166+0000 7f9b25a90640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9b200cd990 0x7f9b2006a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:35.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.166+0000 7f9b25a90640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b200b10a0 con 0x7f9b200b1dd0 2023-09-23T03:27:35.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.166+0000 7f9b25a90640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b200cdde0 con 0x7f9b200cd990 2023-09-23T03:27:35.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.166+0000 7f9b25a90640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b200b1940 con 0x7f9b20160f80 2023-09-23T03:27:35.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.166+0000 7f9b1ffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9b200cd990 0x7f9b2006a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:35.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.166+0000 7f9b1ffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9b200cd990 0x7f9b2006a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54610/0 (socket says 172.21.15.154:54610) 2023-09-23T03:27:35.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.166+0000 7f9b1ffff640 1 -- 172.21.15.154:0/4156673936 learned_addr learned my addr 172.21.15.154:0/4156673936 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:35.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.166+0000 7f9b1f7fe640 1 --2- 172.21.15.154:0/4156673936 >> v2:172.21.15.190:3300/0 conn(0x7f9b20160f80 0x7f9b20069cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:35.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.167+0000 7f9b1ffff640 1 -- 172.21.15.154:0/4156673936 >> v2:172.21.15.190:3300/0 conn(0x7f9b20160f80 msgr2=0x7f9b20069cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:35.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.167+0000 7f9b1ffff640 1 --2- 172.21.15.154:0/4156673936 >> v2:172.21.15.190:3300/0 conn(0x7f9b20160f80 0x7f9b20069cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:35.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.167+0000 7f9b1ffff640 1 -- 172.21.15.154:0/4156673936 >> v1:172.21.15.190:6789/0 conn(0x7f9b200b1dd0 legacy=0x7f9b200cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:35.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.167+0000 7f9b1ffff640 1 -- 172.21.15.154:0/4156673936 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b200ce000 con 0x7f9b200cd990 2023-09-23T03:27:35.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.168+0000 7f9b1ffff640 1 --2- 172.21.15.154:0/4156673936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b200cd990 0x7f9b2006a210 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f9b2006ed10 tx=0x7f9b1802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac969b5a750af5e2 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:35.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.168+0000 7f9b1e7fc640 1 -- 172.21.15.154:0/4156673936 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b18031870 con 0x7f9b200cd990 2023-09-23T03:27:35.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.168+0000 7f9b1e7fc640 1 -- 172.21.15.154:0/4156673936 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b180319d0 con 0x7f9b200cd990 2023-09-23T03:27:35.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.168+0000 7f9b1e7fc640 1 -- 172.21.15.154:0/4156673936 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b18031d60 con 0x7f9b200cd990 2023-09-23T03:27:35.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.169+0000 7f9b25a90640 1 -- 172.21.15.154:0/4156673936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b200cd990 msgr2=0x7f9b2006a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:35.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.169+0000 7f9b25a90640 1 --2- 172.21.15.154:0/4156673936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b200cd990 0x7f9b2006a210 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f9b2006ed10 tx=0x7f9b1802b910 comp rx=0 tx=0).stop 2023-09-23T03:27:35.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.169+0000 7f9b25a90640 1 -- 172.21.15.154:0/4156673936 shutdown_connections 2023-09-23T03:27:35.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.169+0000 7f9b25a90640 1 --2- 172.21.15.154:0/4156673936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b200cd990 0x7f9b2006a210 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:35.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.169+0000 7f9b25a90640 1 --2- 172.21.15.154:0/4156673936 >> v2:172.21.15.190:3300/0 conn(0x7f9b20160f80 0x7f9b20069cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:35.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.169+0000 7f9b25a90640 1 -- 172.21.15.154:0/4156673936 >> 172.21.15.154:0/4156673936 conn(0x7f9b2006d120 msgr2=0x7f9b2006d500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:35.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.169+0000 7f9b25a90640 1 -- 172.21.15.154:0/4156673936 shutdown_connections 2023-09-23T03:27:35.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.169+0000 7f9b25a90640 1 -- 172.21.15.154:0/4156673936 wait complete. 2023-09-23T03:27:35.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.171+0000 7f9b25a90640 1 Processor -- start 2023-09-23T03:27:35.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.171+0000 7f9b25a90640 1 -- start start 2023-09-23T03:27:35.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.172+0000 7f9b25a90640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b200b1dd0 0x7f9b200c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:35.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.172+0000 7f9b25a90640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b200cd990 0x7f9b2015da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:35.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.172+0000 7f9b25a90640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b20160f80 0x7f9b200c3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:35.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.172+0000 7f9b25a90640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b200c66f0 con 0x7f9b200cd990 2023-09-23T03:27:35.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.172+0000 7f9b25a90640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b200c6f90 con 0x7f9b20160f80 2023-09-23T03:27:35.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.172+0000 7f9b25a90640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2015df50 con 0x7f9b200b1dd0 2023-09-23T03:27:35.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.172+0000 7f9b1effd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b200cd990 0x7f9b2015da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:35.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.172+0000 7f9b1effd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b200cd990 0x7f9b2015da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54620/0 (socket says 172.21.15.154:54620) 2023-09-23T03:27:35.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.172+0000 7f9b1effd640 1 -- 172.21.15.154:0/3540246271 learned_addr learned my addr 172.21.15.154:0/3540246271 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:35.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.173+0000 7f9b1ffff640 1 --2- 172.21.15.154:0/3540246271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b20160f80 0x7f9b200c3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:35.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.173+0000 7f9b1effd640 1 -- 172.21.15.154:0/3540246271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b200b1dd0 msgr2=0x7f9b200c92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:35.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.173+0000 7f9b1effd640 1 --2- 172.21.15.154:0/3540246271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b200b1dd0 0x7f9b200c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:35.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.173+0000 7f9b1effd640 1 -- 172.21.15.154:0/3540246271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b20160f80 msgr2=0x7f9b200c3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:35.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.173+0000 7f9b1effd640 1 --2- 172.21.15.154:0/3540246271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b20160f80 0x7f9b200c3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:35.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.173+0000 7f9b1effd640 1 -- 172.21.15.154:0/3540246271 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b180094d0 con 0x7f9b200cd990 2023-09-23T03:27:35.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.173+0000 7f9b1effd640 1 --2- 172.21.15.154:0/3540246271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b200cd990 0x7f9b2015da10 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800b3f0 tx=0x7f9b0800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:35.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.173+0000 7f9b17fff640 1 -- 172.21.15.154:0/3540246271 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b08011830 con 0x7f9b200cd990 2023-09-23T03:27:35.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.173+0000 7f9b17fff640 1 -- 172.21.15.154:0/3540246271 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b08011990 con 0x7f9b200cd990 2023-09-23T03:27:35.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.173+0000 7f9b17fff640 1 -- 172.21.15.154:0/3540246271 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b08011d40 con 0x7f9b200cd990 2023-09-23T03:27:35.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.173+0000 7f9b25a90640 1 -- 172.21.15.154:0/3540246271 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b200c4310 con 0x7f9b200cd990 2023-09-23T03:27:35.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.173+0000 7f9b25a90640 1 -- 172.21.15.154:0/3540246271 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b200c6c20 con 0x7f9b200cd990 2023-09-23T03:27:35.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.174+0000 7f9b25a90640 1 -- 172.21.15.154:0/3540246271 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b2004e920 con 0x7f9b200cd990 2023-09-23T03:27:35.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.175+0000 7f9b17fff640 1 -- 172.21.15.154:0/3540246271 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f9b0801a400 con 0x7f9b200cd990 2023-09-23T03:27:35.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.176+0000 7f9b17fff640 1 --2- 172.21.15.154:0/3540246271 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9b0006af30 0x7f9b0006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:35.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.176+0000 7f9b17fff640 1 -- 172.21.15.154:0/3540246271 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5521+0+0 (secure 0 0 0) 0x7f9b08089520 con 0x7f9b200cd990 2023-09-23T03:27:35.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.177+0000 7f9b1f7fe640 1 --2- 172.21.15.154:0/3540246271 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9b0006af30 0x7f9b0006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:35.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.177+0000 7f9b1f7fe640 1 --2- 172.21.15.154:0/3540246271 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9b0006af30 0x7f9b0006d3f0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f9b100070b0 tx=0x7f9b10007040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:35.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.181+0000 7f9b17fff640 1 -- 172.21.15.154:0/3540246271 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f9b08054600 con 0x7f9b200cd990 2023-09-23T03:27:35.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.377+0000 7f9b25a90640 1 -- 172.21.15.154:0/3540246271 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f9b200c7100 con 0x7f9b200cd990 2023-09-23T03:27:35.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.382+0000 7f9b17fff640 1 -- 172.21.15.154:0/3540246271 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9b08057c20 con 0x7f9b200cd990 2023-09-23T03:27:35.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.386+0000 7f9b25a90640 1 -- 172.21.15.154:0/3540246271 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9b0006af30 msgr2=0x7f9b0006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:35.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.386+0000 7f9b25a90640 1 --2- 172.21.15.154:0/3540246271 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9b0006af30 0x7f9b0006d3f0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f9b100070b0 tx=0x7f9b10007040 comp rx=0 tx=0).stop 2023-09-23T03:27:35.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.386+0000 7f9b25a90640 1 -- 172.21.15.154:0/3540246271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b200cd990 msgr2=0x7f9b2015da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:35.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.386+0000 7f9b25a90640 1 --2- 172.21.15.154:0/3540246271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b200cd990 0x7f9b2015da10 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800b3f0 tx=0x7f9b0800b8b0 comp rx=0 tx=0).stop 2023-09-23T03:27:35.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.386+0000 7f9b25a90640 1 -- 172.21.15.154:0/3540246271 shutdown_connections 2023-09-23T03:27:35.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.386+0000 7f9b25a90640 1 --2- 172.21.15.154:0/3540246271 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9b0006af30 0x7f9b0006d3f0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:35.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.386+0000 7f9b25a90640 1 --2- 172.21.15.154:0/3540246271 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9b20160f80 0x7f9b200c3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:35.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.386+0000 7f9b25a90640 1 --2- 172.21.15.154:0/3540246271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b200cd990 0x7f9b2015da10 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:35.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.386+0000 7f9b25a90640 1 --2- 172.21.15.154:0/3540246271 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9b200b1dd0 0x7f9b200c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:35.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.386+0000 7f9b25a90640 1 -- 172.21.15.154:0/3540246271 >> 172.21.15.154:0/3540246271 conn(0x7f9b2006d120 msgr2=0x7f9b200b2470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:35.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.386+0000 7f9b25a90640 1 -- 172.21.15.154:0/3540246271 shutdown_connections 2023-09-23T03:27:35.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.386+0000 7f9b25a90640 1 -- 172.21.15.154:0/3540246271 wait complete. 2023-09-23T03:27:35.405 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_stashed_layout' 2023-09-23T03:27:35.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.614+0000 7ff16a426640 1 Processor -- start 2023-09-23T03:27:35.615 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.614+0000 7ff16a426640 1 -- start start 2023-09-23T03:27:35.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.615+0000 7ff16a426640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1640ada30 0x7ff1640ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:35.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.616+0000 7ff16a426640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff1640cf130 0x7ff16415d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:35.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.616+0000 7ff16a426640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff16406ceb0 con 0x7ff1640b2fe0 2023-09-23T03:27:35.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.616+0000 7ff16a426640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1640ade10 con 0x7ff1640cf130 2023-09-23T03:27:35.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.616+0000 7ff16a426640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff164159c90 con 0x7ff1640ada30 2023-09-23T03:27:35.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.616+0000 7ff15bfff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff1640b2fe0 0x7ff1640cea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40740/0 (socket says 172.21.15.154:40740) 2023-09-23T03:27:35.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.616+0000 7ff15bfff640 1 -- 172.21.15.154:0/975046814 learned_addr learned my addr 172.21.15.154:0/975046814 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:27:35.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.616+0000 7ff1689c3640 1 --2- 172.21.15.154:0/975046814 >> v2:172.21.15.190:3300/0 conn(0x7ff1640cf130 0x7ff16415d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:35.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.616+0000 7ff163fff640 1 --2- 172.21.15.154:0/975046814 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1640ada30 0x7ff1640ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:35.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.616+0000 7ff1637fe640 1 -- 172.21.15.154:0/975046814 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2575413540 0 0) 0x7ff14c001110 con 0x7ff1640b2fe0 2023-09-23T03:27:35.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.616+0000 7ff1637fe640 1 -- 172.21.15.154:0/975046814 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff150005730 con 0x7ff1640b2fe0 2023-09-23T03:27:35.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.617+0000 7ff1637fe640 1 -- 172.21.15.154:0/975046814 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2560549394 0 0) 0x7ff150005730 con 0x7ff1640b2fe0 2023-09-23T03:27:35.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.617+0000 7ff1637fe640 1 -- 172.21.15.154:0/975046814 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1640ada30 msgr2=0x7ff1640ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:35.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.617+0000 7ff1637fe640 1 --2- 172.21.15.154:0/975046814 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1640ada30 0x7ff1640ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:35.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.617+0000 7ff1637fe640 1 -- 172.21.15.154:0/975046814 >> v2:172.21.15.190:3300/0 conn(0x7ff1640cf130 msgr2=0x7ff16415d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:35.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.617+0000 7ff1637fe640 1 --2- 172.21.15.154:0/975046814 >> v2:172.21.15.190:3300/0 conn(0x7ff1640cf130 0x7ff16415d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:35.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.617+0000 7ff1637fe640 1 -- 172.21.15.154:0/975046814 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff16406c110 con 0x7ff1640b2fe0 2023-09-23T03:27:35.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.617+0000 7ff1637fe640 1 -- 172.21.15.154:0/975046814 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7ff14c001800 con 0x7ff1640b2fe0 2023-09-23T03:27:35.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.617+0000 7ff1689c3640 1 --2- 172.21.15.154:0/975046814 >> v2:172.21.15.190:3300/0 conn(0x7ff1640cf130 0x7ff16415d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:27:35.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.617+0000 7ff1637fe640 1 -- 172.21.15.154:0/975046814 >> v1:172.21.15.154:6789/0 conn(0x7ff1640b2fe0 legacy=0x7ff1640cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:35.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.617+0000 7ff1637fe640 1 --2- 172.21.15.154:0/975046814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff150005b80 0x7ff150005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:35.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.617+0000 7ff1637fe640 1 --2- 172.21.15.154:0/975046814 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1500069d0 0x7ff150006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:35.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.617+0000 7ff1637fe640 1 --2- 172.21.15.154:0/975046814 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1640ada30 0x7ff150007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:35.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.617+0000 7ff1637fe640 1 -- 172.21.15.154:0/975046814 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1640ade10 con 0x7ff1500069d0 2023-09-23T03:27:35.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.617+0000 7ff1637fe640 1 -- 172.21.15.154:0/975046814 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff164159c90 con 0x7ff150005b80 2023-09-23T03:27:35.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.617+0000 7ff1637fe640 1 -- 172.21.15.154:0/975046814 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff15000dbe0 con 0x7ff1640ada30 2023-09-23T03:27:35.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.617+0000 7ff15bfff640 1 --2- 172.21.15.154:0/975046814 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1500069d0 0x7ff150006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:35.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.618+0000 7ff163fff640 1 --2- 172.21.15.154:0/975046814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff150005b80 0x7ff150005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:35.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.618+0000 7ff15bfff640 1 -- 172.21.15.154:0/975046814 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1640ada30 msgr2=0x7ff150007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:35.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.618+0000 7ff15bfff640 1 --2- 172.21.15.154:0/975046814 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1640ada30 0x7ff150007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:35.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.618+0000 7ff15bfff640 1 -- 172.21.15.154:0/975046814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff150005b80 msgr2=0x7ff150005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:35.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.618+0000 7ff15bfff640 1 --2- 172.21.15.154:0/975046814 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff150005b80 0x7ff150005fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:35.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.618+0000 7ff15bfff640 1 -- 172.21.15.154:0/975046814 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff150005730 con 0x7ff1500069d0 2023-09-23T03:27:35.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.619+0000 7ff15bfff640 1 --2- 172.21.15.154:0/975046814 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1500069d0 0x7ff150006e50 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7ff14c00c110 tx=0x7ff150007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=690288d926c44daf server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:35.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.619+0000 7ff1637fe640 1 -- 172.21.15.154:0/975046814 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14c005ad0 con 0x7ff1500069d0 2023-09-23T03:27:35.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.619+0000 7ff1637fe640 1 -- 172.21.15.154:0/975046814 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff14c005c50 con 0x7ff1500069d0 2023-09-23T03:27:35.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.619+0000 7ff1637fe640 1 -- 172.21.15.154:0/975046814 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14c02c810 con 0x7ff1500069d0 2023-09-23T03:27:35.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.620+0000 7ff16a426640 1 -- 172.21.15.154:0/975046814 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1500069d0 msgr2=0x7ff150006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:35.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.620+0000 7ff16a426640 1 --2- 172.21.15.154:0/975046814 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1500069d0 0x7ff150006e50 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7ff14c00c110 tx=0x7ff150007a00 comp rx=0 tx=0).stop 2023-09-23T03:27:35.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.620+0000 7ff1689c3640 1 -- 172.21.15.154:0/975046814 reap_dead start 2023-09-23T03:27:35.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.620+0000 7ff16a426640 1 -- 172.21.15.154:0/975046814 shutdown_connections 2023-09-23T03:27:35.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.620+0000 7ff16a426640 1 -- 172.21.15.154:0/975046814 >> 172.21.15.154:0/975046814 conn(0x7ff16406a0f0 msgr2=0x7ff16406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:35.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.620+0000 7ff16a426640 1 -- 172.21.15.154:0/975046814 shutdown_connections 2023-09-23T03:27:35.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.620+0000 7ff16a426640 1 -- 172.21.15.154:0/975046814 wait complete. 2023-09-23T03:27:35.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.622+0000 7ff16a426640 1 Processor -- start 2023-09-23T03:27:35.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.622+0000 7ff16a426640 1 -- start start 2023-09-23T03:27:35.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.623+0000 7ff16a426640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1500069d0 0x7ff16415af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:35.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.623+0000 7ff16a426640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1641449b0 0x7ff16415c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:35.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.623+0000 7ff16a426640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1641f1f20 0x7ff1640b1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:35.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.623+0000 7ff16a426640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff164144e30 con 0x7ff1500069d0 2023-09-23T03:27:35.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.623+0000 7ff16a426640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1640c3c70 con 0x7ff1641f1f20 2023-09-23T03:27:35.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.623+0000 7ff16a426640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1640c3340 con 0x7ff1641449b0 2023-09-23T03:27:35.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.623+0000 7ff163fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1500069d0 0x7ff16415af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:35.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.623+0000 7ff163fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1500069d0 0x7ff16415af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54644/0 (socket says 172.21.15.154:54644) 2023-09-23T03:27:35.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.623+0000 7ff163fff640 1 -- 172.21.15.154:0/1152124970 learned_addr learned my addr 172.21.15.154:0/1152124970 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:35.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.623+0000 7ff15bfff640 1 --2- 172.21.15.154:0/1152124970 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1641449b0 0x7ff16415c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:35.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.623+0000 7ff163fff640 1 -- 172.21.15.154:0/1152124970 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1641449b0 msgr2=0x7ff16415c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:35.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.623+0000 7ff163fff640 1 --2- 172.21.15.154:0/1152124970 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1641449b0 0x7ff16415c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:35.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.623+0000 7ff163fff640 1 -- 172.21.15.154:0/1152124970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1641f1f20 msgr2=0x7ff1640b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:35.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.623+0000 7ff163fff640 1 --2- 172.21.15.154:0/1152124970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1641f1f20 0x7ff1640b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:35.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.623+0000 7ff163fff640 1 -- 172.21.15.154:0/1152124970 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1640c3620 con 0x7ff1500069d0 2023-09-23T03:27:35.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.624+0000 7ff163fff640 1 --2- 172.21.15.154:0/1152124970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1500069d0 0x7ff16415af30 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7ff154010890 tx=0x7ff154010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:35.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.624+0000 7ff1617fa640 1 -- 172.21.15.154:0/1152124970 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff154009600 con 0x7ff1500069d0 2023-09-23T03:27:35.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.624+0000 7ff1617fa640 1 -- 172.21.15.154:0/1152124970 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff154005b90 con 0x7ff1500069d0 2023-09-23T03:27:35.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.624+0000 7ff1617fa640 1 -- 172.21.15.154:0/1152124970 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15400e8e0 con 0x7ff1500069d0 2023-09-23T03:27:35.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.624+0000 7ff16a426640 1 -- 172.21.15.154:0/1152124970 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1640c3900 con 0x7ff1500069d0 2023-09-23T03:27:35.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.624+0000 7ff16a426640 1 -- 172.21.15.154:0/1152124970 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1641538d0 con 0x7ff1500069d0 2023-09-23T03:27:35.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.625+0000 7ff16a426640 1 -- 172.21.15.154:0/1152124970 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff164073eb0 con 0x7ff1500069d0 2023-09-23T03:27:35.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.626+0000 7ff1617fa640 1 -- 172.21.15.154:0/1152124970 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7ff15400ea40 con 0x7ff1500069d0 2023-09-23T03:27:35.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.627+0000 7ff1617fa640 1 --2- 172.21.15.154:0/1152124970 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff14406af60 0x7ff14406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:35.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.627+0000 7ff1617fa640 1 -- 172.21.15.154:0/1152124970 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5521+0+0 (secure 0 0 0) 0x7ff154089950 con 0x7ff1500069d0 2023-09-23T03:27:35.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.627+0000 7ff15bfff640 1 --2- 172.21.15.154:0/1152124970 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff14406af60 0x7ff14406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:35.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.628+0000 7ff15bfff640 1 --2- 172.21.15.154:0/1152124970 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff14406af60 0x7ff14406d420 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff14c034000 tx=0x7ff14c007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:35.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.632+0000 7ff1617fa640 1 -- 172.21.15.154:0/1152124970 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7ff154054910 con 0x7ff1500069d0 2023-09-23T03:27:35.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.889+0000 7ff16a426640 1 -- 172.21.15.154:0/1152124970 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_stashed_layout"]} v 0) v1 -- 0x7ff16406dc60 con 0x7ff1500069d0 2023-09-23T03:27:35.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.931+0000 7ff1617fa640 1 -- 172.21.15.154:0/1152124970 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_stashed_layout"]}]=0 v544) v1 ==== 155+0+0 (secure 0 0 0) 0x7ff15400ecf0 con 0x7ff1500069d0 2023-09-23T03:27:35.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.934+0000 7ff16a426640 1 -- 172.21.15.154:0/1152124970 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff14406af60 msgr2=0x7ff14406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:35.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.934+0000 7ff16a426640 1 --2- 172.21.15.154:0/1152124970 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff14406af60 0x7ff14406d420 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff14c034000 tx=0x7ff14c007000 comp rx=0 tx=0).stop 2023-09-23T03:27:35.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.934+0000 7ff16a426640 1 -- 172.21.15.154:0/1152124970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1500069d0 msgr2=0x7ff16415af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:35.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.934+0000 7ff16a426640 1 --2- 172.21.15.154:0/1152124970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1500069d0 0x7ff16415af30 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7ff154010890 tx=0x7ff154010d50 comp rx=0 tx=0).stop 2023-09-23T03:27:35.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.934+0000 7ff16a426640 1 -- 172.21.15.154:0/1152124970 shutdown_connections 2023-09-23T03:27:35.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.934+0000 7ff16a426640 1 --2- 172.21.15.154:0/1152124970 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff14406af60 0x7ff14406d420 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:35.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.934+0000 7ff16a426640 1 --2- 172.21.15.154:0/1152124970 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff1641f1f20 0x7ff1640b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:35.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.934+0000 7ff16a426640 1 --2- 172.21.15.154:0/1152124970 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff1641449b0 0x7ff16415c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:35.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.934+0000 7ff16a426640 1 --2- 172.21.15.154:0/1152124970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1500069d0 0x7ff16415af30 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:35.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.934+0000 7ff16a426640 1 -- 172.21.15.154:0/1152124970 >> 172.21.15.154:0/1152124970 conn(0x7ff16406a0f0 msgr2=0x7ff1640aeb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:35.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.934+0000 7ff16a426640 1 -- 172.21.15.154:0/1152124970 shutdown_connections 2023-09-23T03:27:35.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:35.934+0000 7ff16a426640 1 -- 172.21.15.154:0/1152124970 wait complete. 2023-09-23T03:27:35.951 INFO:tasks.cephfs_test_runner:test_extra_data_pool_stashed_layout (tasks.cephfs.test_data_scan.TestDataScan) ... ok 2023-09-23T03:27:35.952 INFO:tasks.cephfs_test_runner:Starting test: test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan) That when injecting a dentry into a fragmented directory, we put it in the right fragment. 2023-09-23T03:27:35.952 INFO:tasks.cephfs_test_runner:test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan) 2023-09-23T03:27:35.952 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:27:36.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.164+0000 7f156f2f3640 1 Processor -- start 2023-09-23T03:27:36.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.164+0000 7f156f2f3640 1 -- start start 2023-09-23T03:27:36.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.166+0000 7f156f2f3640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f15680704d0 0x7f15680ca500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:36.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.166+0000 7f156f2f3640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f156806bff0 con 0x7f156815eb00 2023-09-23T03:27:36.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.166+0000 7f156f2f3640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f156815efb0 con 0x7f156806c7a0 2023-09-23T03:27:36.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.166+0000 7f156f2f3640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1568155070 con 0x7f15680704d0 2023-09-23T03:27:36.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.166+0000 7f156daf0640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f156815eb00 0x7f156806fdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40750/0 (socket says 172.21.15.154:40750) 2023-09-23T03:27:36.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.166+0000 7f156daf0640 1 -- 172.21.15.154:0/2810945493 learned_addr learned my addr 172.21.15.154:0/2810945493 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:27:36.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.167+0000 7f156eaf2640 1 --2- 172.21.15.154:0/2810945493 >> v2:172.21.15.190:3300/0 conn(0x7f15680704d0 0x7f15680ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:36.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.167+0000 7f156d2ef640 1 -- 172.21.15.154:0/2810945493 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4058197737 0 0) 0x7f156806bff0 con 0x7f156815eb00 2023-09-23T03:27:36.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.167+0000 7f156d2ef640 1 -- 172.21.15.154:0/2810945493 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1554003520 con 0x7f156815eb00 2023-09-23T03:27:36.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.167+0000 7f156d2ef640 1 -- 172.21.15.154:0/2810945493 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4193968780 0 0) 0x7f1554003520 con 0x7f156815eb00 2023-09-23T03:27:36.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.167+0000 7f156d2ef640 1 -- 172.21.15.154:0/2810945493 >> v2:172.21.15.190:3300/0 conn(0x7f15680704d0 msgr2=0x7f15680ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:36.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.167+0000 7f156d2ef640 1 --2- 172.21.15.154:0/2810945493 >> v2:172.21.15.190:3300/0 conn(0x7f15680704d0 0x7f15680ca500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:36.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.167+0000 7f156d2ef640 1 -- 172.21.15.154:0/2810945493 >> v1:172.21.15.190:6789/0 conn(0x7f156806c7a0 legacy=0x7f156815ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:36.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.167+0000 7f156d2ef640 1 -- 172.21.15.154:0/2810945493 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156806cb80 con 0x7f156815eb00 2023-09-23T03:27:36.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.167+0000 7f156d2ef640 1 -- 172.21.15.154:0/2810945493 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f1558001670 con 0x7f156815eb00 2023-09-23T03:27:36.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.167+0000 7f156d2ef640 1 -- 172.21.15.154:0/2810945493 >> v1:172.21.15.154:6789/0 conn(0x7f156815eb00 legacy=0x7f156806fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:36.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.167+0000 7f156d2ef640 1 --2- 172.21.15.154:0/2810945493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15540039c0 0x7f1554003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:36.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.167+0000 7f156d2ef640 1 --2- 172.21.15.154:0/2810945493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15540047c0 0x7f1554004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:36.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.167+0000 7f156d2ef640 1 --2- 172.21.15.154:0/2810945493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1554005180 0x7f155400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:36.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.167+0000 7f156d2ef640 1 -- 172.21.15.154:0/2810945493 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155400c050 con 0x7f1554005180 2023-09-23T03:27:36.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.167+0000 7f156d2ef640 1 -- 172.21.15.154:0/2810945493 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155400c190 con 0x7f15540039c0 2023-09-23T03:27:36.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.167+0000 7f156d2ef640 1 -- 172.21.15.154:0/2810945493 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f155400c350 con 0x7f15540047c0 2023-09-23T03:27:36.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.168+0000 7f156eaf2640 1 --2- 172.21.15.154:0/2810945493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1554005180 0x7f155400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:36.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.168+0000 7f156daf0640 1 --2- 172.21.15.154:0/2810945493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15540047c0 0x7f1554004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:36.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.168+0000 7f156e2f1640 1 --2- 172.21.15.154:0/2810945493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15540039c0 0x7f1554003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:36.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.168+0000 7f156eaf2640 1 -- 172.21.15.154:0/2810945493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15540047c0 msgr2=0x7f1554004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:36.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.168+0000 7f156eaf2640 1 --2- 172.21.15.154:0/2810945493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15540047c0 0x7f1554004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:36.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.168+0000 7f156eaf2640 1 -- 172.21.15.154:0/2810945493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15540039c0 msgr2=0x7f1554003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:36.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.168+0000 7f156eaf2640 1 --2- 172.21.15.154:0/2810945493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15540039c0 0x7f1554003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:36.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.168+0000 7f156eaf2640 1 -- 172.21.15.154:0/2810945493 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1554003520 con 0x7f1554005180 2023-09-23T03:27:36.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.168+0000 7f156e2f1640 1 --2- 172.21.15.154:0/2810945493 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15540039c0 0x7f1554003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:36.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.168+0000 7f156daf0640 1 --2- 172.21.15.154:0/2810945493 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15540047c0 0x7f1554004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:36.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.169+0000 7f156eaf2640 1 -- 172.21.15.154:0/2810945493 reap_dead start 2023-09-23T03:27:36.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.169+0000 7f156eaf2640 1 --2- 172.21.15.154:0/2810945493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1554005180 0x7f155400b980 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f1560009fc0 tx=0x7f15540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d27c82b28ef332b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:36.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.169+0000 7f156d2ef640 1 -- 172.21.15.154:0/2810945493 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156002cc30 con 0x7f1554005180 2023-09-23T03:27:36.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.169+0000 7f156d2ef640 1 -- 172.21.15.154:0/2810945493 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156000f040 con 0x7f1554005180 2023-09-23T03:27:36.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.169+0000 7f156d2ef640 1 -- 172.21.15.154:0/2810945493 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1560003990 con 0x7f1554005180 2023-09-23T03:27:36.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.170+0000 7f156f2f3640 1 -- 172.21.15.154:0/2810945493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1554005180 msgr2=0x7f155400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:36.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.170+0000 7f156f2f3640 1 --2- 172.21.15.154:0/2810945493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1554005180 0x7f155400b980 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f1560009fc0 tx=0x7f15540057f0 comp rx=0 tx=0).stop 2023-09-23T03:27:36.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.170+0000 7f156f2f3640 1 -- 172.21.15.154:0/2810945493 shutdown_connections 2023-09-23T03:27:36.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.170+0000 7f156f2f3640 1 --2- 172.21.15.154:0/2810945493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1554005180 0x7f155400b980 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:36.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.170+0000 7f156f2f3640 1 -- 172.21.15.154:0/2810945493 >> 172.21.15.154:0/2810945493 conn(0x7f156806a390 msgr2=0x7f15680afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:36.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.170+0000 7f156f2f3640 1 -- 172.21.15.154:0/2810945493 shutdown_connections 2023-09-23T03:27:36.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.170+0000 7f156f2f3640 1 -- 172.21.15.154:0/2810945493 wait complete. 2023-09-23T03:27:36.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.172+0000 7f156f2f3640 1 Processor -- start 2023-09-23T03:27:36.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.172+0000 7f156f2f3640 1 -- start start 2023-09-23T03:27:36.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.173+0000 7f156f2f3640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1554005180 0x7f15680c7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:36.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.173+0000 7f156f2f3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f156815d100 0x7f15680c8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:36.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.173+0000 7f156f2f3640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15680c9360 0x7f15680b1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:36.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.173+0000 7f156f2f3640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156815cce0 con 0x7f156815d100 2023-09-23T03:27:36.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.173+0000 7f156f2f3640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15680c97e0 con 0x7f15680c9360 2023-09-23T03:27:36.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.173+0000 7f156f2f3640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156815c060 con 0x7f1554005180 2023-09-23T03:27:36.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.173+0000 7f156daf0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f156815d100 0x7f15680c8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:36.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.173+0000 7f156daf0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f156815d100 0x7f15680c8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54654/0 (socket says 172.21.15.154:54654) 2023-09-23T03:27:36.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.173+0000 7f156daf0640 1 -- 172.21.15.154:0/1238801038 learned_addr learned my addr 172.21.15.154:0/1238801038 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:36.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.174+0000 7f156e2f1640 1 --2- 172.21.15.154:0/1238801038 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1554005180 0x7f15680c7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:36.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.174+0000 7f156daf0640 1 -- 172.21.15.154:0/1238801038 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1554005180 msgr2=0x7f15680c7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:36.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.174+0000 7f156daf0640 1 --2- 172.21.15.154:0/1238801038 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1554005180 0x7f15680c7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:36.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.174+0000 7f156daf0640 1 -- 172.21.15.154:0/1238801038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15680c9360 msgr2=0x7f15680b1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:36.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.174+0000 7f156daf0640 1 --2- 172.21.15.154:0/1238801038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15680c9360 0x7f15680b1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:36.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.174+0000 7f156daf0640 1 -- 172.21.15.154:0/1238801038 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156815c340 con 0x7f156815d100 2023-09-23T03:27:36.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.174+0000 7f156daf0640 1 --2- 172.21.15.154:0/1238801038 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f156815d100 0x7f15680c8e20 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f155800d770 tx=0x7f155800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:36.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.174+0000 7f155effd640 1 -- 172.21.15.154:0/1238801038 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15580074e0 con 0x7f156815d100 2023-09-23T03:27:36.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.174+0000 7f155effd640 1 -- 172.21.15.154:0/1238801038 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1558007640 con 0x7f156815d100 2023-09-23T03:27:36.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.174+0000 7f156f2f3640 1 -- 172.21.15.154:0/1238801038 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f156815c620 con 0x7f156815d100 2023-09-23T03:27:36.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.174+0000 7f155effd640 1 -- 172.21.15.154:0/1238801038 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1558004a00 con 0x7f156815d100 2023-09-23T03:27:36.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.174+0000 7f156f2f3640 1 -- 172.21.15.154:0/1238801038 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15680c79b0 con 0x7f156815d100 2023-09-23T03:27:36.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.176+0000 7f156f2f3640 1 -- 172.21.15.154:0/1238801038 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15680736b0 con 0x7f156815d100 2023-09-23T03:27:36.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.176+0000 7f155effd640 1 -- 172.21.15.154:0/1238801038 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f1558004b60 con 0x7f156815d100 2023-09-23T03:27:36.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.177+0000 7f155effd640 1 --2- 172.21.15.154:0/1238801038 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f154c06af30 0x7f154c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:36.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.177+0000 7f155effd640 1 -- 172.21.15.154:0/1238801038 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5521+0+0 (secure 0 0 0) 0x7f1558089300 con 0x7f156815d100 2023-09-23T03:27:36.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.178+0000 7f156e2f1640 1 --2- 172.21.15.154:0/1238801038 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f154c06af30 0x7f154c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:36.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.178+0000 7f156e2f1640 1 --2- 172.21.15.154:0/1238801038 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f154c06af30 0x7f154c06d3f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f1564004e70 tx=0x7f1564009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:36.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.182+0000 7f155effd640 1 -- 172.21.15.154:0/1238801038 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f1558054360 con 0x7f156815d100 2023-09-23T03:27:36.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.373+0000 7f156f2f3640 1 -- 172.21.15.154:0/1238801038 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f156815ce20 con 0x7f156815d100 2023-09-23T03:27:36.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.374+0000 7f155effd640 1 -- 172.21.15.154:0/1238801038 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+13635 (secure 0 0 0) 0x7f156815ce20 con 0x7f156815d100 2023-09-23T03:27:36.375 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:36.375 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":118,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:27:34.917572+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6837/1758434763":"2023-09-24T03:27:30.893904+0000","172.21.15.154:6835/1758434763":"2023-09-24T03:27:30.893904+0000","172.21.15.154:6837/3181099748":"2023-09-24T03:26:34.396308+0000","172.21.15.154:6835/3181099748":"2023-09-24T03:26:34.396308+0000","172.21.15.154:6835/4233181417":"2023-09-24T03:26:27.714769+0000","172.21.15.154:6837/4233181417":"2023-09-24T03:26:27.714769+0000","172.21.15.190:6835/2967125458":"2023-09-24T03:26:25.688784+0000","172.21.15.190:6833/2967125458":"2023-09-24T03:26:25.688784+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:27:36.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.377+0000 7f156f2f3640 1 -- 172.21.15.154:0/1238801038 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f154c06af30 msgr2=0x7f154c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:36.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.377+0000 7f156f2f3640 1 --2- 172.21.15.154:0/1238801038 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f154c06af30 0x7f154c06d3f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f1564004e70 tx=0x7f1564009040 comp rx=0 tx=0).stop 2023-09-23T03:27:36.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.377+0000 7f156f2f3640 1 -- 172.21.15.154:0/1238801038 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f156815d100 msgr2=0x7f15680c8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:36.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.377+0000 7f156f2f3640 1 --2- 172.21.15.154:0/1238801038 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f156815d100 0x7f15680c8e20 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f155800d770 tx=0x7f155800dc30 comp rx=0 tx=0).stop 2023-09-23T03:27:36.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.377+0000 7f156f2f3640 1 -- 172.21.15.154:0/1238801038 shutdown_connections 2023-09-23T03:27:36.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.377+0000 7f156f2f3640 1 --2- 172.21.15.154:0/1238801038 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f154c06af30 0x7f154c06d3f0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:36.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.377+0000 7f156f2f3640 1 --2- 172.21.15.154:0/1238801038 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15680c9360 0x7f15680b1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:36.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.377+0000 7f156f2f3640 1 --2- 172.21.15.154:0/1238801038 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f156815d100 0x7f15680c8e20 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:36.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.377+0000 7f156f2f3640 1 --2- 172.21.15.154:0/1238801038 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1554005180 0x7f15680c7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:36.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.377+0000 7f156f2f3640 1 -- 172.21.15.154:0/1238801038 >> 172.21.15.154:0/1238801038 conn(0x7f156806a390 msgr2=0x7f1568153610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:36.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.377+0000 7f156f2f3640 1 -- 172.21.15.154:0/1238801038 shutdown_connections 2023-09-23T03:27:36.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.378+0000 7f156f2f3640 1 -- 172.21.15.154:0/1238801038 wait complete. 2023-09-23T03:27:36.398 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T03:27:36.398 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:27:36.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.616+0000 7f4483ff5640 1 Processor -- start 2023-09-23T03:27:36.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.616+0000 7f4483ff5640 1 -- start start 2023-09-23T03:27:36.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.618+0000 7f4483ff5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f447c15c9c0 0x7f447c0d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:36.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.618+0000 7f4483ff5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f447c15cda0 0x7f447c0d2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:36.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.618+0000 7f4483ff5640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f447c0cd450 con 0x7f447c0d2b60 2023-09-23T03:27:36.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.618+0000 7f4483ff5640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f447c0d0df0 con 0x7f447c15cda0 2023-09-23T03:27:36.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.618+0000 7f4483ff5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447c0d10d0 con 0x7f447c15c9c0 2023-09-23T03:27:36.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.618+0000 7f4481590640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f447c15cda0 0x7f447c0d2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:36.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.618+0000 7f4481590640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f447c15cda0 0x7f447c0d2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54668/0 (socket says 172.21.15.154:54668) 2023-09-23T03:27:36.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.619+0000 7f4481590640 1 -- 172.21.15.154:0/2093386809 learned_addr learned my addr 172.21.15.154:0/2093386809 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:36.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.619+0000 7f4481d91640 1 --2- 172.21.15.154:0/2093386809 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f447c15c9c0 0x7f447c0d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:36.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.619+0000 7f4481590640 1 -- 172.21.15.154:0/2093386809 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f447c15c9c0 msgr2=0x7f447c0d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:36.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.619+0000 7f4481590640 1 --2- 172.21.15.154:0/2093386809 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f447c15c9c0 0x7f447c0d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:36.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.619+0000 7f4481590640 1 -- 172.21.15.154:0/2093386809 >> v1:172.21.15.190:6789/0 conn(0x7f447c0d2b60 legacy=0x7f447c0cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:36.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.619+0000 7f4481590640 1 -- 172.21.15.154:0/2093386809 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447c0d13b0 con 0x7f447c15cda0 2023-09-23T03:27:36.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.620+0000 7f4481590640 1 --2- 172.21.15.154:0/2093386809 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f447c15cda0 0x7f447c0d2620 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f446c009870 tx=0x7f446c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=851eb93e33ae9a03 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:36.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.621+0000 7f4480d8f640 1 -- 172.21.15.154:0/2093386809 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c031640 con 0x7f447c15cda0 2023-09-23T03:27:36.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.621+0000 7f4480d8f640 1 -- 172.21.15.154:0/2093386809 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446c0317a0 con 0x7f447c15cda0 2023-09-23T03:27:36.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.621+0000 7f4480d8f640 1 -- 172.21.15.154:0/2093386809 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c031b30 con 0x7f447c15cda0 2023-09-23T03:27:36.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.621+0000 7f4483ff5640 1 -- 172.21.15.154:0/2093386809 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f447c15cda0 msgr2=0x7f447c0d2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:36.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.621+0000 7f4483ff5640 1 --2- 172.21.15.154:0/2093386809 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f447c15cda0 0x7f447c0d2620 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f446c009870 tx=0x7f446c02b8f0 comp rx=0 tx=0).stop 2023-09-23T03:27:36.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.621+0000 7f4483ff5640 1 -- 172.21.15.154:0/2093386809 shutdown_connections 2023-09-23T03:27:36.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.621+0000 7f4483ff5640 1 --2- 172.21.15.154:0/2093386809 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f447c15cda0 0x7f447c0d2620 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:36.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.621+0000 7f4483ff5640 1 --2- 172.21.15.154:0/2093386809 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f447c15c9c0 0x7f447c0d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:36.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.621+0000 7f4483ff5640 1 -- 172.21.15.154:0/2093386809 >> 172.21.15.154:0/2093386809 conn(0x7f447c09bc20 msgr2=0x7f447c089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:36.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.622+0000 7f4483ff5640 1 -- 172.21.15.154:0/2093386809 shutdown_connections 2023-09-23T03:27:36.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.622+0000 7f4483ff5640 1 -- 172.21.15.154:0/2093386809 wait complete. 2023-09-23T03:27:36.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.623+0000 7f4483ff5640 1 Processor -- start 2023-09-23T03:27:36.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.624+0000 7f4483ff5640 1 -- start start 2023-09-23T03:27:36.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.624+0000 7f4483ff5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f447c0d2b60 0x7f447c0c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:36.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.624+0000 7f4483ff5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f447c15c9c0 0x7f447c158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:36.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.624+0000 7f4483ff5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f447c15cda0 0x7f447c0cae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:36.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.624+0000 7f4483ff5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c0cb8c0 con 0x7f447c0d2b60 2023-09-23T03:27:36.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.624+0000 7f4483ff5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c0cbba0 con 0x7f447c15c9c0 2023-09-23T03:27:36.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.624+0000 7f4483ff5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447c0c8320 con 0x7f447c15cda0 2023-09-23T03:27:36.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.625+0000 7f4481d91640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f447c0d2b60 0x7f447c0c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:36.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.625+0000 7f4481d91640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f447c0d2b60 0x7f447c0c8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54672/0 (socket says 172.21.15.154:54672) 2023-09-23T03:27:36.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.625+0000 7f4481d91640 1 -- 172.21.15.154:0/56825683 learned_addr learned my addr 172.21.15.154:0/56825683 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:36.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.625+0000 7f4481d91640 1 -- 172.21.15.154:0/56825683 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f447c15cda0 msgr2=0x7f447c0cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:36.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.625+0000 7f4482592640 1 --2- 172.21.15.154:0/56825683 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f447c15cda0 0x7f447c0cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:36.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.625+0000 7f4481590640 1 --2- 172.21.15.154:0/56825683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f447c15c9c0 0x7f447c158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:36.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.625+0000 7f4481d91640 1 --2- 172.21.15.154:0/56825683 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f447c15cda0 0x7f447c0cae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:36.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.625+0000 7f4481d91640 1 -- 172.21.15.154:0/56825683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f447c15c9c0 msgr2=0x7f447c158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:36.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.625+0000 7f4481d91640 1 --2- 172.21.15.154:0/56825683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f447c15c9c0 0x7f447c158150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:36.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.625+0000 7f4481d91640 1 -- 172.21.15.154:0/56825683 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446c0094d0 con 0x7f447c0d2b60 2023-09-23T03:27:36.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.625+0000 7f4481d91640 1 --2- 172.21.15.154:0/56825683 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f447c0d2b60 0x7f447c0c8e00 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f447800d800 tx=0x7f447800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:36.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.626+0000 7f44727fc640 1 -- 172.21.15.154:0/56825683 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447800b6c0 con 0x7f447c0d2b60 2023-09-23T03:27:36.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.626+0000 7f44727fc640 1 -- 172.21.15.154:0/56825683 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447800b820 con 0x7f447c0d2b60 2023-09-23T03:27:36.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.626+0000 7f44727fc640 1 -- 172.21.15.154:0/56825683 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447800bbd0 con 0x7f447c0d2b60 2023-09-23T03:27:36.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.626+0000 7f4483ff5640 1 -- 172.21.15.154:0/56825683 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f447c0c8600 con 0x7f447c0d2b60 2023-09-23T03:27:36.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.626+0000 7f4483ff5640 1 -- 172.21.15.154:0/56825683 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f447c158690 con 0x7f447c0d2b60 2023-09-23T03:27:36.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.627+0000 7f4483ff5640 1 -- 172.21.15.154:0/56825683 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f447c04e920 con 0x7f447c0d2b60 2023-09-23T03:27:36.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.628+0000 7f44727fc640 1 -- 172.21.15.154:0/56825683 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f447800bd30 con 0x7f447c0d2b60 2023-09-23T03:27:36.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.629+0000 7f44727fc640 1 --2- 172.21.15.154:0/56825683 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f446006af30 0x7f446006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:36.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.629+0000 7f44727fc640 1 -- 172.21.15.154:0/56825683 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5521+0+0 (secure 0 0 0) 0x7f4478089bb0 con 0x7f447c0d2b60 2023-09-23T03:27:36.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.629+0000 7f4481590640 1 --2- 172.21.15.154:0/56825683 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f446006af30 0x7f446006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:36.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.630+0000 7f4481590640 1 --2- 172.21.15.154:0/56825683 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f446006af30 0x7f446006d3f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f446c036000 tx=0x7f446c03c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:36.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.634+0000 7f44727fc640 1 -- 172.21.15.154:0/56825683 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f4478054d30 con 0x7f447c0d2b60 2023-09-23T03:27:36.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.824+0000 7f4483ff5640 1 -- 172.21.15.154:0/56825683 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f447c0b22c0 con 0x7f447c0d2b60 2023-09-23T03:27:36.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.825+0000 7f44727fc640 1 -- 172.21.15.154:0/56825683 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v118) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4478058350 con 0x7f447c0d2b60 2023-09-23T03:27:36.826 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:27:36.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.828+0000 7f4483ff5640 1 -- 172.21.15.154:0/56825683 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f446006af30 msgr2=0x7f446006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:36.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.828+0000 7f4483ff5640 1 --2- 172.21.15.154:0/56825683 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f446006af30 0x7f446006d3f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f446c036000 tx=0x7f446c03c040 comp rx=0 tx=0).stop 2023-09-23T03:27:36.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.828+0000 7f4483ff5640 1 -- 172.21.15.154:0/56825683 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f447c0d2b60 msgr2=0x7f447c0c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:36.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.828+0000 7f4483ff5640 1 --2- 172.21.15.154:0/56825683 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f447c0d2b60 0x7f447c0c8e00 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f447800d800 tx=0x7f447800dcc0 comp rx=0 tx=0).stop 2023-09-23T03:27:36.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.828+0000 7f4483ff5640 1 -- 172.21.15.154:0/56825683 shutdown_connections 2023-09-23T03:27:36.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.828+0000 7f4483ff5640 1 --2- 172.21.15.154:0/56825683 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f446006af30 0x7f446006d3f0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:36.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.828+0000 7f4483ff5640 1 --2- 172.21.15.154:0/56825683 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f447c15cda0 0x7f447c0cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:36.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.828+0000 7f4483ff5640 1 --2- 172.21.15.154:0/56825683 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f447c15c9c0 0x7f447c158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:36.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.828+0000 7f4483ff5640 1 --2- 172.21.15.154:0/56825683 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f447c0d2b60 0x7f447c0c8e00 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:36.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.828+0000 7f4483ff5640 1 -- 172.21.15.154:0/56825683 >> 172.21.15.154:0/56825683 conn(0x7f447c09bc20 msgr2=0x7f447c15bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:36.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.828+0000 7f4483ff5640 1 -- 172.21.15.154:0/56825683 shutdown_connections 2023-09-23T03:27:36.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:36.828+0000 7f4483ff5640 1 -- 172.21.15.154:0/56825683 wait complete. 2023-09-23T03:27:36.846 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_data_scan.TestDataScan.test_fragmented_injection' 2023-09-23T03:27:37.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.060+0000 7fd465f55640 1 Processor -- start 2023-09-23T03:27:37.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.060+0000 7fd465f55640 1 -- start start 2023-09-23T03:27:37.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.061+0000 7fd465f55640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd4600acce0 0x7fd46015ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:37.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.061+0000 7fd465f55640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd4600b1b20 0x7fd4600ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:37.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.061+0000 7fd465f55640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd46015ac30 con 0x7fd4600cee70 2023-09-23T03:27:37.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.061+0000 7fd465f55640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4600b2e90 con 0x7fd4600b1b20 2023-09-23T03:27:37.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.061+0000 7fd465f55640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4600cf300 con 0x7fd4600acce0 2023-09-23T03:27:37.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.062+0000 7fd457fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd4600b1b20 0x7fd4600ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:37.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.062+0000 7fd457fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd4600b1b20 0x7fd4600ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54688/0 (socket says 172.21.15.154:54688) 2023-09-23T03:27:37.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.062+0000 7fd457fff640 1 -- 172.21.15.154:0/3623537917 learned_addr learned my addr 172.21.15.154:0/3623537917 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:37.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.062+0000 7fd45f7fe640 1 --2- 172.21.15.154:0/3623537917 >> v2:172.21.15.190:3300/0 conn(0x7fd4600acce0 0x7fd46015ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:37.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.062+0000 7fd457fff640 1 -- 172.21.15.154:0/3623537917 >> v2:172.21.15.190:3300/0 conn(0x7fd4600acce0 msgr2=0x7fd46015ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:37.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.062+0000 7fd457fff640 1 --2- 172.21.15.154:0/3623537917 >> v2:172.21.15.190:3300/0 conn(0x7fd4600acce0 0x7fd46015ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:37.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.062+0000 7fd457fff640 1 -- 172.21.15.154:0/3623537917 >> v1:172.21.15.190:6789/0 conn(0x7fd4600cee70 legacy=0x7fd46006cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:37.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.062+0000 7fd457fff640 1 -- 172.21.15.154:0/3623537917 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4600b1f70 con 0x7fd4600b1b20 2023-09-23T03:27:37.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.063+0000 7fd457fff640 1 --2- 172.21.15.154:0/3623537917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4600b1b20 0x7fd4600ce930 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fd4600cf570 tx=0x7fd44802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54bb1be6bd5ea349 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:37.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.063+0000 7fd45effd640 1 -- 172.21.15.154:0/3623537917 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4480317e0 con 0x7fd4600b1b20 2023-09-23T03:27:37.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.064+0000 7fd45effd640 1 -- 172.21.15.154:0/3623537917 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd448031940 con 0x7fd4600b1b20 2023-09-23T03:27:37.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.064+0000 7fd45effd640 1 -- 172.21.15.154:0/3623537917 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd448031cd0 con 0x7fd4600b1b20 2023-09-23T03:27:37.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.064+0000 7fd465f55640 1 -- 172.21.15.154:0/3623537917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4600b1b20 msgr2=0x7fd4600ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:37.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.064+0000 7fd465f55640 1 --2- 172.21.15.154:0/3623537917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4600b1b20 0x7fd4600ce930 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fd4600cf570 tx=0x7fd44802b880 comp rx=0 tx=0).stop 2023-09-23T03:27:37.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.064+0000 7fd465f55640 1 -- 172.21.15.154:0/3623537917 shutdown_connections 2023-09-23T03:27:37.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.064+0000 7fd465f55640 1 --2- 172.21.15.154:0/3623537917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4600b1b20 0x7fd4600ce930 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:37.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.064+0000 7fd465f55640 1 --2- 172.21.15.154:0/3623537917 >> v2:172.21.15.190:3300/0 conn(0x7fd4600acce0 0x7fd46015ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:37.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.064+0000 7fd465f55640 1 -- 172.21.15.154:0/3623537917 >> 172.21.15.154:0/3623537917 conn(0x7fd46006a0f0 msgr2=0x7fd46006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:37.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.064+0000 7fd465f55640 1 -- 172.21.15.154:0/3623537917 shutdown_connections 2023-09-23T03:27:37.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.064+0000 7fd465f55640 1 -- 172.21.15.154:0/3623537917 wait complete. 2023-09-23T03:27:37.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.066+0000 7fd465f55640 1 Processor -- start 2023-09-23T03:27:37.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.066+0000 7fd465f55640 1 -- start start 2023-09-23T03:27:37.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.067+0000 7fd465f55640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4600acce0 0x7fd460159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:37.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.067+0000 7fd465f55640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4600b1b20 0x7fd4600cd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:37.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.067+0000 7fd465f55640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4600cee70 0x7fd4600c5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:37.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.067+0000 7fd465f55640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4600cc110 con 0x7fd4600b1b20 2023-09-23T03:27:37.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.067+0000 7fd465f55640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46015b1d0 con 0x7fd4600acce0 2023-09-23T03:27:37.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.067+0000 7fd465f55640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4600cc360 con 0x7fd4600cee70 2023-09-23T03:27:37.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.067+0000 7fd457fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4600b1b20 0x7fd4600cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:37.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.067+0000 7fd457fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4600b1b20 0x7fd4600cd400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54694/0 (socket says 172.21.15.154:54694) 2023-09-23T03:27:37.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.067+0000 7fd457fff640 1 -- 172.21.15.154:0/367261405 learned_addr learned my addr 172.21.15.154:0/367261405 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:37.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.068+0000 7fd457fff640 1 -- 172.21.15.154:0/367261405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4600cee70 msgr2=0x7fd4600c5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:37.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.068+0000 7fd457fff640 1 --2- 172.21.15.154:0/367261405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4600cee70 0x7fd4600c5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:37.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.068+0000 7fd45f7fe640 1 --2- 172.21.15.154:0/367261405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4600acce0 0x7fd460159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:37.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.068+0000 7fd457fff640 1 -- 172.21.15.154:0/367261405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4600acce0 msgr2=0x7fd460159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:37.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.068+0000 7fd457fff640 1 --2- 172.21.15.154:0/367261405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4600acce0 0x7fd460159b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:37.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.068+0000 7fd457fff640 1 -- 172.21.15.154:0/367261405 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4480094d0 con 0x7fd4600b1b20 2023-09-23T03:27:37.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.068+0000 7fd457fff640 1 --2- 172.21.15.154:0/367261405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4600b1b20 0x7fd4600cd400 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fd448000c00 tx=0x7fd4480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:37.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.068+0000 7fd45cff9640 1 -- 172.21.15.154:0/367261405 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd448032880 con 0x7fd4600b1b20 2023-09-23T03:27:37.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.069+0000 7fd45cff9640 1 -- 172.21.15.154:0/367261405 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4480329e0 con 0x7fd4600b1b20 2023-09-23T03:27:37.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.069+0000 7fd45cff9640 1 -- 172.21.15.154:0/367261405 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd448032d90 con 0x7fd4600b1b20 2023-09-23T03:27:37.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.069+0000 7fd465f55640 1 -- 172.21.15.154:0/367261405 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4600cc640 con 0x7fd4600b1b20 2023-09-23T03:27:37.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.069+0000 7fd465f55640 1 -- 172.21.15.154:0/367261405 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4600ccc00 con 0x7fd4600b1b20 2023-09-23T03:27:37.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.070+0000 7fd465f55640 1 -- 172.21.15.154:0/367261405 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd460073eb0 con 0x7fd4600b1b20 2023-09-23T03:27:37.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.071+0000 7fd45cff9640 1 -- 172.21.15.154:0/367261405 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fd44804d3f0 con 0x7fd4600b1b20 2023-09-23T03:27:37.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.072+0000 7fd45cff9640 1 --2- 172.21.15.154:0/367261405 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd44006af30 0x7fd44006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:37.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.072+0000 7fd45cff9640 1 -- 172.21.15.154:0/367261405 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5521+0+0 (secure 0 0 0) 0x7fd4480ab3d0 con 0x7fd4600b1b20 2023-09-23T03:27:37.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.072+0000 7fd45f7fe640 1 --2- 172.21.15.154:0/367261405 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd44006af30 0x7fd44006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:37.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.073+0000 7fd45f7fe640 1 --2- 172.21.15.154:0/367261405 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd44006af30 0x7fd44006d3f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fd4500097f0 tx=0x7fd450008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:37.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.076+0000 7fd45cff9640 1 -- 172.21.15.154:0/367261405 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fd448076420 con 0x7fd4600b1b20 2023-09-23T03:27:37.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.334+0000 7fd465f55640 1 -- 172.21.15.154:0/367261405 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_fragmented_injection"]} v 0) v1 -- 0x7fd46015ac10 con 0x7fd4600b1b20 2023-09-23T03:27:37.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.937+0000 7fd45cff9640 1 -- 172.21.15.154:0/367261405 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_fragmented_injection"]}]=0 v546) v1 ==== 148+0+0 (secure 0 0 0) 0x7fd448079a40 con 0x7fd4600b1b20 2023-09-23T03:27:37.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.940+0000 7fd465f55640 1 -- 172.21.15.154:0/367261405 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd44006af30 msgr2=0x7fd44006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:37.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.940+0000 7fd465f55640 1 --2- 172.21.15.154:0/367261405 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd44006af30 0x7fd44006d3f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fd4500097f0 tx=0x7fd450008040 comp rx=0 tx=0).stop 2023-09-23T03:27:37.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.940+0000 7fd465f55640 1 -- 172.21.15.154:0/367261405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4600b1b20 msgr2=0x7fd4600cd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:37.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.940+0000 7fd465f55640 1 --2- 172.21.15.154:0/367261405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4600b1b20 0x7fd4600cd400 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fd448000c00 tx=0x7fd4480046c0 comp rx=0 tx=0).stop 2023-09-23T03:27:37.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.940+0000 7fd465f55640 1 -- 172.21.15.154:0/367261405 shutdown_connections 2023-09-23T03:27:37.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.940+0000 7fd465f55640 1 --2- 172.21.15.154:0/367261405 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd44006af30 0x7fd44006d3f0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:37.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.940+0000 7fd465f55640 1 --2- 172.21.15.154:0/367261405 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd4600cee70 0x7fd4600c5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:37.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.940+0000 7fd465f55640 1 --2- 172.21.15.154:0/367261405 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd4600b1b20 0x7fd4600cd400 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:37.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.940+0000 7fd465f55640 1 --2- 172.21.15.154:0/367261405 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd4600acce0 0x7fd460159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:37.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.940+0000 7fd465f55640 1 -- 172.21.15.154:0/367261405 >> 172.21.15.154:0/367261405 conn(0x7fd46006a0f0 msgr2=0x7fd4600b05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:37.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.940+0000 7fd465f55640 1 -- 172.21.15.154:0/367261405 shutdown_connections 2023-09-23T03:27:37.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:37.940+0000 7fd465f55640 1 -- 172.21.15.154:0/367261405 wait complete. 2023-09-23T03:27:37.960 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-09-23T03:27:38.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.170+0000 7f6a2b097640 1 Processor -- start 2023-09-23T03:27:38.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.170+0000 7f6a2b097640 1 -- start start 2023-09-23T03:27:38.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.172+0000 7f6a2b097640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6a2415aba0 0x7f6a2415bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:38.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.172+0000 7f6a2b097640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a240cb5b0 0x7f6a2415ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:38.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.172+0000 7f6a2b097640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a2415b6a0 con 0x7f6a2415e5f0 2023-09-23T03:27:38.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.172+0000 7f6a2b097640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a2415f0b0 con 0x7f6a2415aba0 2023-09-23T03:27:38.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.172+0000 7f6a2b097640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2415f390 con 0x7f6a240cb5b0 2023-09-23T03:27:38.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.172+0000 7f6a28e33640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6a2415aba0 0x7f6a2415bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:38.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.172+0000 7f6a29634640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a240cb5b0 0x7f6a2415ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:38.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.173+0000 7f6a28e33640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6a2415aba0 0x7f6a2415bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.154:59786/0 (socket says 172.21.15.154:59786) 2023-09-23T03:27:38.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.173+0000 7f6a28e33640 1 -- 172.21.15.154:0/1721048174 learned_addr learned my addr 172.21.15.154:0/1721048174 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:38.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.173+0000 7f6a28e33640 1 -- 172.21.15.154:0/1721048174 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a240cb5b0 msgr2=0x7f6a2415ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:38.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.173+0000 7f6a28e33640 1 --2- 172.21.15.154:0/1721048174 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a240cb5b0 0x7f6a2415ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:38.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.173+0000 7f6a28e33640 1 -- 172.21.15.154:0/1721048174 >> v1:172.21.15.190:6789/0 conn(0x7f6a2415e5f0 legacy=0x7f6a240caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:38.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.173+0000 7f6a28e33640 1 -- 172.21.15.154:0/1721048174 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a2415b330 con 0x7f6a2415aba0 2023-09-23T03:27:38.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.174+0000 7f6a28e33640 1 --2- 172.21.15.154:0/1721048174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a2415aba0 0x7f6a2415bb30 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6a18009870 tx=0x7f6a1802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d87dd01af5f5f87 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:38.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.175+0000 7f6a237fe640 1 -- 172.21.15.154:0/1721048174 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a18030840 con 0x7f6a2415aba0 2023-09-23T03:27:38.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.175+0000 7f6a237fe640 1 -- 172.21.15.154:0/1721048174 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a180309a0 con 0x7f6a2415aba0 2023-09-23T03:27:38.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.175+0000 7f6a237fe640 1 -- 172.21.15.154:0/1721048174 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a18030d30 con 0x7f6a2415aba0 2023-09-23T03:27:38.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.175+0000 7f6a2b097640 1 -- 172.21.15.154:0/1721048174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a2415aba0 msgr2=0x7f6a2415bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:38.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.175+0000 7f6a2b097640 1 --2- 172.21.15.154:0/1721048174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a2415aba0 0x7f6a2415bb30 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6a18009870 tx=0x7f6a1802b980 comp rx=0 tx=0).stop 2023-09-23T03:27:38.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.175+0000 7f6a2b097640 1 -- 172.21.15.154:0/1721048174 shutdown_connections 2023-09-23T03:27:38.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.175+0000 7f6a2b097640 1 --2- 172.21.15.154:0/1721048174 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a240cb5b0 0x7f6a2415ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:38.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.175+0000 7f6a2b097640 1 --2- 172.21.15.154:0/1721048174 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a2415aba0 0x7f6a2415bb30 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:38.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.175+0000 7f6a2b097640 1 -- 172.21.15.154:0/1721048174 >> 172.21.15.154:0/1721048174 conn(0x7f6a2406c010 msgr2=0x7f6a2406c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:38.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.176+0000 7f6a2b097640 1 -- 172.21.15.154:0/1721048174 shutdown_connections 2023-09-23T03:27:38.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.176+0000 7f6a2b097640 1 -- 172.21.15.154:0/1721048174 wait complete. 2023-09-23T03:27:38.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.177+0000 7f6a2b097640 1 Processor -- start 2023-09-23T03:27:38.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.177+0000 7f6a2b097640 1 -- start start 2023-09-23T03:27:38.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.178+0000 7f6a2b097640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a240cb5b0 0x7f6a241f3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:38.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.178+0000 7f6a2b097640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a2415aba0 0x7f6a241570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:38.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.178+0000 7f6a2b097640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a2415e5f0 0x7f6a241f2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:38.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.178+0000 7f6a2b097640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a241f5270 con 0x7f6a2415e5f0 2023-09-23T03:27:38.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.178+0000 7f6a2b097640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a241575f0 con 0x7f6a240cb5b0 2023-09-23T03:27:38.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.178+0000 7f6a2b097640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a241f4660 con 0x7f6a2415aba0 2023-09-23T03:27:38.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.179+0000 7f6a29634640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a2415e5f0 0x7f6a241f2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:38.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.179+0000 7f6a29634640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a2415e5f0 0x7f6a241f2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54698/0 (socket says 172.21.15.154:54698) 2023-09-23T03:27:38.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.179+0000 7f6a29634640 1 -- 172.21.15.154:0/3210083629 learned_addr learned my addr 172.21.15.154:0/3210083629 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:38.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.179+0000 7f6a28e33640 1 --2- 172.21.15.154:0/3210083629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a240cb5b0 0x7f6a241f3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:38.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.179+0000 7f6a23fff640 1 --2- 172.21.15.154:0/3210083629 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a2415aba0 0x7f6a241570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:38.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.179+0000 7f6a29634640 1 -- 172.21.15.154:0/3210083629 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a2415aba0 msgr2=0x7f6a241570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:38.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.179+0000 7f6a29634640 1 --2- 172.21.15.154:0/3210083629 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a2415aba0 0x7f6a241570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:38.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.179+0000 7f6a29634640 1 -- 172.21.15.154:0/3210083629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a240cb5b0 msgr2=0x7f6a241f3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:38.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.179+0000 7f6a29634640 1 --2- 172.21.15.154:0/3210083629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a240cb5b0 0x7f6a241f3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:38.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.179+0000 7f6a29634640 1 -- 172.21.15.154:0/3210083629 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a180094d0 con 0x7f6a2415e5f0 2023-09-23T03:27:38.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.180+0000 7f6a29634640 1 --2- 172.21.15.154:0/3210083629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a2415e5f0 0x7f6a241f2530 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c00efc0 tx=0x7f6a1c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:38.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.180+0000 7f6a217fa640 1 -- 172.21.15.154:0/3210083629 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c014480 con 0x7f6a2415e5f0 2023-09-23T03:27:38.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.180+0000 7f6a217fa640 1 -- 172.21.15.154:0/3210083629 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a1c00f040 con 0x7f6a2415e5f0 2023-09-23T03:27:38.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.180+0000 7f6a217fa640 1 -- 172.21.15.154:0/3210083629 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c014820 con 0x7f6a2415e5f0 2023-09-23T03:27:38.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.180+0000 7f6a2b097640 1 -- 172.21.15.154:0/3210083629 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a2415b100 con 0x7f6a2415e5f0 2023-09-23T03:27:38.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.180+0000 7f6a2b097640 1 -- 172.21.15.154:0/3210083629 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a241f4c20 con 0x7f6a2415e5f0 2023-09-23T03:27:38.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.181+0000 7f6a2b097640 1 -- 172.21.15.154:0/3210083629 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a241f53e0 con 0x7f6a2415e5f0 2023-09-23T03:27:38.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.182+0000 7f6a217fa640 1 -- 172.21.15.154:0/3210083629 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f6a1c014980 con 0x7f6a2415e5f0 2023-09-23T03:27:38.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.183+0000 7f6a217fa640 1 --2- 172.21.15.154:0/3210083629 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6a0806af30 0x7f6a0806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:38.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.183+0000 7f6a217fa640 1 -- 172.21.15.154:0/3210083629 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5521+0+0 (secure 0 0 0) 0x7f6a1c08a2e0 con 0x7f6a2415e5f0 2023-09-23T03:27:38.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.183+0000 7f6a28e33640 1 --2- 172.21.15.154:0/3210083629 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6a0806af30 0x7f6a0806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:38.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.184+0000 7f6a28e33640 1 --2- 172.21.15.154:0/3210083629 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6a0806af30 0x7f6a0806d3f0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f6a18036000 tx=0x7f6a18008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:38.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.187+0000 7f6a217fa640 1 -- 172.21.15.154:0/3210083629 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f6a1c0553d0 con 0x7f6a2415e5f0 2023-09-23T03:27:38.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.365+0000 7f6a2b097640 1 -- 172.21.15.154:0/3210083629 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6a2404e920 con 0x7f6a2415e5f0 2023-09-23T03:27:38.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.370+0000 7f6a217fa640 1 -- 172.21.15.154:0/3210083629 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6a1c0589f0 con 0x7f6a2415e5f0 2023-09-23T03:27:38.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.373+0000 7f6a2b097640 1 -- 172.21.15.154:0/3210083629 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6a0806af30 msgr2=0x7f6a0806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:38.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.373+0000 7f6a2b097640 1 --2- 172.21.15.154:0/3210083629 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6a0806af30 0x7f6a0806d3f0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f6a18036000 tx=0x7f6a18008030 comp rx=0 tx=0).stop 2023-09-23T03:27:38.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.373+0000 7f6a2b097640 1 -- 172.21.15.154:0/3210083629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a2415e5f0 msgr2=0x7f6a241f2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:38.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.373+0000 7f6a2b097640 1 --2- 172.21.15.154:0/3210083629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a2415e5f0 0x7f6a241f2530 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c00efc0 tx=0x7f6a1c00c490 comp rx=0 tx=0).stop 2023-09-23T03:27:38.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.373+0000 7f6a2b097640 1 -- 172.21.15.154:0/3210083629 shutdown_connections 2023-09-23T03:27:38.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.373+0000 7f6a2b097640 1 --2- 172.21.15.154:0/3210083629 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6a0806af30 0x7f6a0806d3f0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:38.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.373+0000 7f6a2b097640 1 --2- 172.21.15.154:0/3210083629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a2415e5f0 0x7f6a241f2530 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:38.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.373+0000 7f6a2b097640 1 --2- 172.21.15.154:0/3210083629 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6a2415aba0 0x7f6a241570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:38.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.373+0000 7f6a2b097640 1 --2- 172.21.15.154:0/3210083629 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6a240cb5b0 0x7f6a241f3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:38.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.373+0000 7f6a2b097640 1 -- 172.21.15.154:0/3210083629 >> 172.21.15.154:0/3210083629 conn(0x7f6a2406c010 msgr2=0x7f6a240701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:38.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.373+0000 7f6a2b097640 1 -- 172.21.15.154:0/3210083629 shutdown_connections 2023-09-23T03:27:38.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:38.373+0000 7f6a2b097640 1 -- 172.21.15.154:0/3210083629 wait complete. 2023-09-23T03:27:38.390 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-09-23T03:27:38.390 DEBUG:teuthology.orchestra.run.smithi154:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-23T03:27:38.506 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-23T03:27:38.628 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-09-23T03:27:38.628 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:27:38.628 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:27:38.688 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053600,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-09-23T03:27:38.688 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:27:38.688 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:27:38.746 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053603,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248804k,nr_inodes=812201,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-09-23T03:27:38.746 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:27:38.746 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:27:38.805 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053600,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-09-23T03:27:38.805 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:27:38.805 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:27:38.864 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053603,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248804k,nr_inodes=812201,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-09-23T03:27:38.864 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-09-23T03:27:38.865 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:27:39.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.088+0000 7f6e8ff1d640 1 Processor -- start 2023-09-23T03:27:39.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.088+0000 7f6e8ff1d640 1 -- start start 2023-09-23T03:27:39.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.089+0000 7f6e8ff1d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6e880ada30 0x7f6e880ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:39.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.089+0000 7f6e8ff1d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f6e880cf130 0x7f6e8815d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:39.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.089+0000 7f6e8ff1d640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e8806ceb0 con 0x7f6e880b2fe0 2023-09-23T03:27:39.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.089+0000 7f6e8ff1d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e880ade10 con 0x7f6e880ada30 2023-09-23T03:27:39.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.089+0000 7f6e8ff1d640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e88159c90 con 0x7f6e880cf130 2023-09-23T03:27:39.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.090+0000 7f6e8dcb9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6e880ada30 0x7f6e880ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:39.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.090+0000 7f6e8dcb9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6e880ada30 0x7f6e880ccaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54710/0 (socket says 172.21.15.154:54710) 2023-09-23T03:27:39.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.090+0000 7f6e8dcb9640 1 -- 172.21.15.154:0/1402548369 learned_addr learned my addr 172.21.15.154:0/1402548369 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:39.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.090+0000 7f6e8e4ba640 1 --2- 172.21.15.154:0/1402548369 >> v2:172.21.15.190:3300/0 conn(0x7f6e880cf130 0x7f6e8815d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:39.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.090+0000 7f6e8dcb9640 1 -- 172.21.15.154:0/1402548369 >> v2:172.21.15.190:3300/0 conn(0x7f6e880cf130 msgr2=0x7f6e8815d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:39.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.090+0000 7f6e8dcb9640 1 --2- 172.21.15.154:0/1402548369 >> v2:172.21.15.190:3300/0 conn(0x7f6e880cf130 0x7f6e8815d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:39.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.090+0000 7f6e8dcb9640 1 -- 172.21.15.154:0/1402548369 >> v1:172.21.15.190:6789/0 conn(0x7f6e880b2fe0 legacy=0x7f6e880cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:39.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.090+0000 7f6e8dcb9640 1 -- 172.21.15.154:0/1402548369 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e8806c110 con 0x7f6e880ada30 2023-09-23T03:27:39.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.091+0000 7f6e8dcb9640 1 --2- 172.21.15.154:0/1402548369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e880ada30 0x7f6e880ccaf0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f6e880ce4e0 tx=0x7f6e8402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae540ab8e5ca847c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:39.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.092+0000 7f6e8ccb7640 1 -- 172.21.15.154:0/1402548369 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e84031870 con 0x7f6e880ada30 2023-09-23T03:27:39.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.092+0000 7f6e8ccb7640 1 -- 172.21.15.154:0/1402548369 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e840319d0 con 0x7f6e880ada30 2023-09-23T03:27:39.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.092+0000 7f6e8ccb7640 1 -- 172.21.15.154:0/1402548369 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e84031d60 con 0x7f6e880ada30 2023-09-23T03:27:39.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.092+0000 7f6e8ff1d640 1 -- 172.21.15.154:0/1402548369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e880ada30 msgr2=0x7f6e880ccaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:39.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.092+0000 7f6e8ff1d640 1 --2- 172.21.15.154:0/1402548369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e880ada30 0x7f6e880ccaf0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f6e880ce4e0 tx=0x7f6e8402b910 comp rx=0 tx=0).stop 2023-09-23T03:27:39.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.092+0000 7f6e8ff1d640 1 -- 172.21.15.154:0/1402548369 shutdown_connections 2023-09-23T03:27:39.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.092+0000 7f6e8ff1d640 1 --2- 172.21.15.154:0/1402548369 >> v2:172.21.15.190:3300/0 conn(0x7f6e880cf130 0x7f6e8815d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:39.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.092+0000 7f6e8ff1d640 1 --2- 172.21.15.154:0/1402548369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e880ada30 0x7f6e880ccaf0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:39.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.092+0000 7f6e8ff1d640 1 -- 172.21.15.154:0/1402548369 >> 172.21.15.154:0/1402548369 conn(0x7f6e8806a0f0 msgr2=0x7f6e8806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:39.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.092+0000 7f6e8ff1d640 1 -- 172.21.15.154:0/1402548369 shutdown_connections 2023-09-23T03:27:39.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.092+0000 7f6e8ff1d640 1 -- 172.21.15.154:0/1402548369 wait complete. 2023-09-23T03:27:39.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.094+0000 7f6e8ff1d640 1 Processor -- start 2023-09-23T03:27:39.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.094+0000 7f6e8ff1d640 1 -- start start 2023-09-23T03:27:39.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.095+0000 7f6e8ff1d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e880ada30 0x7f6e880aca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:39.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.095+0000 7f6e8ff1d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e880b2fe0 0x7f6e881f33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:39.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.095+0000 7f6e8ff1d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e880cf130 0x7f6e88159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:39.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.095+0000 7f6e8ff1d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e880acfc0 con 0x7f6e880ada30 2023-09-23T03:27:39.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.095+0000 7f6e8ff1d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e881f3910 con 0x7f6e880cf130 2023-09-23T03:27:39.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.095+0000 7f6e8ff1d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e881f3bc0 con 0x7f6e880b2fe0 2023-09-23T03:27:39.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.096+0000 7f6e8dcb9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e880ada30 0x7f6e880aca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:39.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.096+0000 7f6e8dcb9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e880ada30 0x7f6e880aca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54716/0 (socket says 172.21.15.154:54716) 2023-09-23T03:27:39.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.096+0000 7f6e8dcb9640 1 -- 172.21.15.154:0/1543463972 learned_addr learned my addr 172.21.15.154:0/1543463972 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:39.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.096+0000 7f6e8dcb9640 1 -- 172.21.15.154:0/1543463972 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e880b2fe0 msgr2=0x7f6e881f33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:39.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.096+0000 7f6e8dcb9640 1 --2- 172.21.15.154:0/1543463972 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e880b2fe0 0x7f6e881f33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:39.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.096+0000 7f6e8dcb9640 1 -- 172.21.15.154:0/1543463972 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e880cf130 msgr2=0x7f6e88159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:39.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.096+0000 7f6e8dcb9640 1 --2- 172.21.15.154:0/1543463972 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e880cf130 0x7f6e88159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:39.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.096+0000 7f6e8dcb9640 1 -- 172.21.15.154:0/1543463972 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e84009470 con 0x7f6e880ada30 2023-09-23T03:27:39.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.096+0000 7f6e8dcb9640 1 --2- 172.21.15.154:0/1543463972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e880ada30 0x7f6e880aca80 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f6e84000c00 tx=0x7f6e840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:39.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.097+0000 7f6e7e7fc640 1 -- 172.21.15.154:0/1543463972 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8403abe0 con 0x7f6e880ada30 2023-09-23T03:27:39.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.097+0000 7f6e7e7fc640 1 -- 172.21.15.154:0/1543463972 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e8403ad40 con 0x7f6e880ada30 2023-09-23T03:27:39.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.097+0000 7f6e7e7fc640 1 -- 172.21.15.154:0/1543463972 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e84044610 con 0x7f6e880ada30 2023-09-23T03:27:39.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.097+0000 7f6e8ff1d640 1 -- 172.21.15.154:0/1543463972 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e881f3ea0 con 0x7f6e880ada30 2023-09-23T03:27:39.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.097+0000 7f6e8ff1d640 1 -- 172.21.15.154:0/1543463972 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e881f4460 con 0x7f6e880ada30 2023-09-23T03:27:39.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.098+0000 7f6e8ff1d640 1 -- 172.21.15.154:0/1543463972 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e88066fd0 con 0x7f6e880ada30 2023-09-23T03:27:39.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.099+0000 7f6e7e7fc640 1 -- 172.21.15.154:0/1543463972 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f6e8404d450 con 0x7f6e880ada30 2023-09-23T03:27:39.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.100+0000 7f6e7e7fc640 1 --2- 172.21.15.154:0/1543463972 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6e6c06af60 0x7f6e6c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:39.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.100+0000 7f6e7e7fc640 1 -- 172.21.15.154:0/1543463972 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5521+0+0 (secure 0 0 0) 0x7f6e8402d040 con 0x7f6e880ada30 2023-09-23T03:27:39.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.100+0000 7f6e8d4b8640 1 --2- 172.21.15.154:0/1543463972 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6e6c06af60 0x7f6e6c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:39.101 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.101+0000 7f6e8d4b8640 1 --2- 172.21.15.154:0/1543463972 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6e6c06af60 0x7f6e6c06d420 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6e8815cf10 tx=0x7f6e78007990 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:39.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.104+0000 7f6e7e7fc640 1 -- 172.21.15.154:0/1543463972 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f6e84076c10 con 0x7f6e880ada30 2023-09-23T03:27:39.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.329+0000 7f6e8ff1d640 1 -- 172.21.15.154:0/1543463972 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e8806ceb0 con 0x7f6e880ada30 2023-09-23T03:27:39.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.331+0000 7f6e7e7fc640 1 -- 172.21.15.154:0/1543463972 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+3707 (secure 0 0 0) 0x7f6e8806ceb0 con 0x7f6e880ada30 2023-09-23T03:27:39.331 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:39.331 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":101,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5753,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6833/60017546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":60017546},{"type":"v1","addr":"172.21.15.190:6833","nonce":60017546}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95},{"gid":6415,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/2708068523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6834","nonce":2708068523},{"type":"v1","addr":"172.21.15.190:6835","nonce":2708068523}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":99},{"gid":8592,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6836/1846547753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":1846547753},{"type":"v1","addr":"172.21.15.154:6836","nonce":1846547753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":94},{"gid":8817,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/1130200913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6835","nonce":1130200913},{"type":"v1","addr":"172.21.15.154:6837","nonce":1130200913}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":101}],"filesystems":[]} 2023-09-23T03:27:39.332 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 101 2023-09-23T03:27:39.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.333+0000 7f6e8ff1d640 1 -- 172.21.15.154:0/1543463972 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6e6c06af60 msgr2=0x7f6e6c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:39.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.333+0000 7f6e8ff1d640 1 --2- 172.21.15.154:0/1543463972 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6e6c06af60 0x7f6e6c06d420 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6e8815cf10 tx=0x7f6e78007990 comp rx=0 tx=0).stop 2023-09-23T03:27:39.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.333+0000 7f6e8ff1d640 1 -- 172.21.15.154:0/1543463972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e880ada30 msgr2=0x7f6e880aca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:39.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.333+0000 7f6e8ff1d640 1 --2- 172.21.15.154:0/1543463972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e880ada30 0x7f6e880aca80 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f6e84000c00 tx=0x7f6e840046c0 comp rx=0 tx=0).stop 2023-09-23T03:27:39.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.333+0000 7f6e8ff1d640 1 -- 172.21.15.154:0/1543463972 shutdown_connections 2023-09-23T03:27:39.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.333+0000 7f6e8ff1d640 1 --2- 172.21.15.154:0/1543463972 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6e6c06af60 0x7f6e6c06d420 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:39.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.333+0000 7f6e8ff1d640 1 --2- 172.21.15.154:0/1543463972 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6e880cf130 0x7f6e88159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:39.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.333+0000 7f6e8ff1d640 1 --2- 172.21.15.154:0/1543463972 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6e880b2fe0 0x7f6e881f33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:39.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.333+0000 7f6e8ff1d640 1 --2- 172.21.15.154:0/1543463972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6e880ada30 0x7f6e880aca80 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:39.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.333+0000 7f6e8ff1d640 1 -- 172.21.15.154:0/1543463972 >> 172.21.15.154:0/1543463972 conn(0x7f6e8806a0f0 msgr2=0x7f6e880af0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:39.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.334+0000 7f6e8ff1d640 1 -- 172.21.15.154:0/1543463972 shutdown_connections 2023-09-23T03:27:39.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.334+0000 7f6e8ff1d640 1 -- 172.21.15.154:0/1543463972 wait complete. 2023-09-23T03:27:39.353 INFO:tasks.ceph.mds.a:Restarting daemon 2023-09-23T03:27:39.353 INFO:tasks.ceph.mds.a:Stopping old one... 2023-09-23T03:27:39.353 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-09-23T03:27:39.354 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:27:39.354 INFO:tasks.ceph.mds.c:Restarting daemon 2023-09-23T03:27:39.354 INFO:tasks.ceph.mds.c:Stopping old one... 2023-09-23T03:27:39.354 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-09-23T03:27:39.354 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:27:39.354 INFO:tasks.ceph.mds.b:Restarting daemon 2023-09-23T03:27:39.354 INFO:tasks.ceph.mds.b:Stopping old one... 2023-09-23T03:27:39.355 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-09-23T03:27:39.355 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:27:39.355 INFO:tasks.ceph.mds.d:Restarting daemon 2023-09-23T03:27:39.355 INFO:tasks.ceph.mds.d:Stopping old one... 2023-09-23T03:27:39.355 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-09-23T03:27:39.355 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:27:39.371 INFO:tasks.ceph.mds.a:Stopped 2023-09-23T03:27:39.371 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-09-23T03:27:39.374 INFO:tasks.ceph.mds.d:Stopped 2023-09-23T03:27:39.374 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-09-23T03:27:39.375 INFO:tasks.ceph.mds.c:Stopped 2023-09-23T03:27:39.375 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-09-23T03:27:39.379 INFO:tasks.ceph.mds.b:Stopped 2023-09-23T03:27:39.379 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-09-23T03:27:39.424 INFO:tasks.ceph.mds.a:Started 2023-09-23T03:27:39.424 DEBUG:teuthology.parallel:result is None 2023-09-23T03:27:39.428 INFO:tasks.ceph.mds.d:Started 2023-09-23T03:27:39.428 DEBUG:teuthology.parallel:result is None 2023-09-23T03:27:39.432 INFO:tasks.ceph.mds.c:Started 2023-09-23T03:27:39.432 DEBUG:teuthology.parallel:result is None 2023-09-23T03:27:39.435 INFO:tasks.ceph.mds.b:Started 2023-09-23T03:27:39.436 DEBUG:teuthology.parallel:result is None 2023-09-23T03:27:39.436 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-09-23T03:27:39.583 INFO:tasks.ceph.mds.a.smithi154.stdout:starting mds.a at 2023-09-23T03:27:39.591 INFO:tasks.ceph.mds.c.smithi154.stdout:starting mds.c at 2023-09-23T03:27:39.602 INFO:tasks.ceph.mds.d.smithi190.stdout:starting mds.d at 2023-09-23T03:27:39.603 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-09-23T03:27:39.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.626+0000 7fa16ee7f640 1 Processor -- start 2023-09-23T03:27:39.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.626+0000 7fa16ee7f640 1 -- start start 2023-09-23T03:27:39.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.627+0000 7fa16ee7f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1680cf8e0 0x7fa16815c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:39.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.627+0000 7fa16ee7f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa16815ca30 0x7fa1680b1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:39.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.627+0000 7fa16ee7f640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1680d17f0 con 0x7fa1680cf500 2023-09-23T03:27:39.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.627+0000 7fa16ee7f640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa16806be10 con 0x7fa16815ca30 2023-09-23T03:27:39.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.627+0000 7fa16ee7f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1680d0470 con 0x7fa1680cf8e0 2023-09-23T03:27:39.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.628+0000 7fa16e67e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa16815ca30 0x7fa1680b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:39.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.628+0000 7fa16de7d640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fa1680cf500 0x7fa1680ce050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.154:55262/0 (socket says 172.21.15.154:55262) 2023-09-23T03:27:39.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.628+0000 7fa16de7d640 1 -- 172.21.15.154:0/1297173614 learned_addr learned my addr 172.21.15.154:0/1297173614 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:27:39.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.628+0000 7fa16d67c640 1 --2- 172.21.15.154:0/1297173614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1680cf8e0 0x7fa16815c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:39.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.628+0000 7fa16ce7b640 1 -- 172.21.15.154:0/1297173614 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3360481923 0 0) 0x7fa1680d17f0 con 0x7fa1680cf500 2023-09-23T03:27:39.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.628+0000 7fa16ce7b640 1 -- 172.21.15.154:0/1297173614 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa154003520 con 0x7fa1680cf500 2023-09-23T03:27:39.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.628+0000 7fa16e67e640 1 -- 172.21.15.154:0/1297173614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1680cf8e0 msgr2=0x7fa16815c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:39.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.628+0000 7fa16e67e640 1 --2- 172.21.15.154:0/1297173614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1680cf8e0 0x7fa16815c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:39.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.628+0000 7fa16e67e640 1 -- 172.21.15.154:0/1297173614 >> v1:172.21.15.190:6789/0 conn(0x7fa1680cf500 legacy=0x7fa1680ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:39.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.628+0000 7fa16e67e640 1 -- 172.21.15.154:0/1297173614 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1680d0750 con 0x7fa16815ca30 2023-09-23T03:27:39.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.629+0000 7fa16e67e640 1 --2- 172.21.15.154:0/1297173614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa16815ca30 0x7fa1680b1430 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fa160009d90 tx=0x7fa16002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f81e614544b4dd server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:39.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.630+0000 7fa16ce7b640 1 -- 172.21.15.154:0/1297173614 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16002cbd0 con 0x7fa16815ca30 2023-09-23T03:27:39.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.630+0000 7fa16ce7b640 1 -- 172.21.15.154:0/1297173614 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa16002cd30 con 0x7fa16815ca30 2023-09-23T03:27:39.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.630+0000 7fa16ce7b640 1 -- 172.21.15.154:0/1297173614 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16002cea0 con 0x7fa16815ca30 2023-09-23T03:27:39.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.630+0000 7fa16ee7f640 1 -- 172.21.15.154:0/1297173614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa16815ca30 msgr2=0x7fa1680b1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:39.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.630+0000 7fa16ee7f640 1 --2- 172.21.15.154:0/1297173614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa16815ca30 0x7fa1680b1430 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fa160009d90 tx=0x7fa16002be20 comp rx=0 tx=0).stop 2023-09-23T03:27:39.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.630+0000 7fa16ee7f640 1 -- 172.21.15.154:0/1297173614 shutdown_connections 2023-09-23T03:27:39.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.630+0000 7fa16ee7f640 1 --2- 172.21.15.154:0/1297173614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa16815ca30 0x7fa1680b1430 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:39.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.630+0000 7fa16ee7f640 1 --2- 172.21.15.154:0/1297173614 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa1680cf8e0 0x7fa16815c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:39.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.630+0000 7fa16ee7f640 1 -- 172.21.15.154:0/1297173614 >> 172.21.15.154:0/1297173614 conn(0x7fa16809bbb0 msgr2=0x7fa1680d28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:39.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.630+0000 7fa16ee7f640 1 -- 172.21.15.154:0/1297173614 shutdown_connections 2023-09-23T03:27:39.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.630+0000 7fa16ee7f640 1 -- 172.21.15.154:0/1297173614 wait complete. 2023-09-23T03:27:39.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.632+0000 7fa16ee7f640 1 Processor -- start 2023-09-23T03:27:39.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.632+0000 7fa16ee7f640 1 -- start start 2023-09-23T03:27:39.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.633+0000 7fa16ee7f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1680cf500 0x7fa168148590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:39.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.633+0000 7fa16ee7f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1680cf8e0 0x7fa16809f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:39.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.633+0000 7fa16ee7f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa16815ca30 0x7fa16809fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:39.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.633+0000 7fa16ee7f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1680a0280 con 0x7fa1680cf500 2023-09-23T03:27:39.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.633+0000 7fa16ee7f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16809c5c0 con 0x7fa1680cf8e0 2023-09-23T03:27:39.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.633+0000 7fa16ee7f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa16809c830 con 0x7fa16815ca30 2023-09-23T03:27:39.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.633+0000 7fa16de7d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1680cf500 0x7fa168148590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:39.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.633+0000 7fa16de7d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1680cf500 0x7fa168148590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54772/0 (socket says 172.21.15.154:54772) 2023-09-23T03:27:39.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.633+0000 7fa16de7d640 1 -- 172.21.15.154:0/81937291 learned_addr learned my addr 172.21.15.154:0/81937291 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:39.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.634+0000 7fa16d67c640 1 --2- 172.21.15.154:0/81937291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1680cf8e0 0x7fa16809f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:39.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.634+0000 7fa16e67e640 1 --2- 172.21.15.154:0/81937291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa16815ca30 0x7fa16809fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:39.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.634+0000 7fa16de7d640 1 -- 172.21.15.154:0/81937291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa16815ca30 msgr2=0x7fa16809fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:39.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.634+0000 7fa16de7d640 1 --2- 172.21.15.154:0/81937291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa16815ca30 0x7fa16809fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:39.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.634+0000 7fa16de7d640 1 -- 172.21.15.154:0/81937291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1680cf8e0 msgr2=0x7fa16809f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:39.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.634+0000 7fa16de7d640 1 --2- 172.21.15.154:0/81937291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1680cf8e0 0x7fa16809f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:39.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.634+0000 7fa16de7d640 1 -- 172.21.15.154:0/81937291 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa160009a70 con 0x7fa1680cf500 2023-09-23T03:27:39.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.634+0000 7fa16de7d640 1 --2- 172.21.15.154:0/81937291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1680cf500 0x7fa168148590 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fa164001130 tx=0x7fa16400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:39.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.634+0000 7fa15e7fc640 1 -- 172.21.15.154:0/81937291 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa164005d30 con 0x7fa1680cf500 2023-09-23T03:27:39.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.634+0000 7fa15e7fc640 1 -- 172.21.15.154:0/81937291 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa164005e90 con 0x7fa1680cf500 2023-09-23T03:27:39.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.635+0000 7fa15e7fc640 1 -- 172.21.15.154:0/81937291 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa164007310 con 0x7fa1680cf500 2023-09-23T03:27:39.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.635+0000 7fa16ee7f640 1 -- 172.21.15.154:0/81937291 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa16809cb10 con 0x7fa1680cf500 2023-09-23T03:27:39.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.635+0000 7fa16ee7f640 1 -- 172.21.15.154:0/81937291 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1681f2f50 con 0x7fa1680cf500 2023-09-23T03:27:39.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.636+0000 7fa16ee7f640 1 -- 172.21.15.154:0/81937291 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa16804e920 con 0x7fa1680cf500 2023-09-23T03:27:39.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.637+0000 7fa15e7fc640 1 -- 172.21.15.154:0/81937291 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fa164007470 con 0x7fa1680cf500 2023-09-23T03:27:39.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.638+0000 7fa15e7fc640 1 --2- 172.21.15.154:0/81937291 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa14c06af30 0x7fa14c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:39.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.638+0000 7fa15e7fc640 1 -- 172.21.15.154:0/81937291 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5521+0+0 (secure 0 0 0) 0x7fa164088c30 con 0x7fa1680cf500 2023-09-23T03:27:39.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.638+0000 7fa16d67c640 1 --2- 172.21.15.154:0/81937291 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa14c06af30 0x7fa14c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:39.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.639+0000 7fa16d67c640 1 --2- 172.21.15.154:0/81937291 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa14c06af30 0x7fa14c06d3f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa16815ad70 tx=0x7fa158009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:39.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.642+0000 7fa15e7fc640 1 -- 172.21.15.154:0/81937291 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fa164053dd0 con 0x7fa1680cf500 2023-09-23T03:27:39.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.841+0000 7fa16ee7f640 1 -- 172.21.15.154:0/81937291 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa168072730 con 0x7fa1680cf500 2023-09-23T03:27:39.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.960+0000 7fa15e7fc640 1 -- 172.21.15.154:0/81937291 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v120) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa164017020 con 0x7fa1680cf500 2023-09-23T03:27:39.961 INFO:teuthology.orchestra.run.smithi154.stderr: removed all blocklist entries 2023-09-23T03:27:39.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.963+0000 7fa16ee7f640 1 -- 172.21.15.154:0/81937291 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa14c06af30 msgr2=0x7fa14c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:39.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.963+0000 7fa16ee7f640 1 --2- 172.21.15.154:0/81937291 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa14c06af30 0x7fa14c06d3f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa16815ad70 tx=0x7fa158009190 comp rx=0 tx=0).stop 2023-09-23T03:27:39.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.963+0000 7fa16ee7f640 1 -- 172.21.15.154:0/81937291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1680cf500 msgr2=0x7fa168148590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:39.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.963+0000 7fa16ee7f640 1 --2- 172.21.15.154:0/81937291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1680cf500 0x7fa168148590 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fa164001130 tx=0x7fa16400c8f0 comp rx=0 tx=0).stop 2023-09-23T03:27:39.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.963+0000 7fa16ee7f640 1 -- 172.21.15.154:0/81937291 shutdown_connections 2023-09-23T03:27:39.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.963+0000 7fa16ee7f640 1 --2- 172.21.15.154:0/81937291 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa14c06af30 0x7fa14c06d3f0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:39.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.963+0000 7fa16ee7f640 1 --2- 172.21.15.154:0/81937291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa16815ca30 0x7fa16809fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:39.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.963+0000 7fa16ee7f640 1 --2- 172.21.15.154:0/81937291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa1680cf8e0 0x7fa16809f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:39.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.963+0000 7fa16ee7f640 1 --2- 172.21.15.154:0/81937291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa1680cf500 0x7fa168148590 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:39.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.963+0000 7fa16ee7f640 1 -- 172.21.15.154:0/81937291 >> 172.21.15.154:0/81937291 conn(0x7fa16809bbb0 msgr2=0x7fa1680c63d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:39.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.964+0000 7fa16ee7f640 1 -- 172.21.15.154:0/81937291 shutdown_connections 2023-09-23T03:27:39.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:39.964+0000 7fa16ee7f640 1 -- 172.21.15.154:0/81937291 wait complete. 2023-09-23T03:27:39.984 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-09-23T03:27:40.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.176+0000 7f99ddd90640 1 Processor -- start 2023-09-23T03:27:40.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.176+0000 7f99ddd90640 1 -- start start 2023-09-23T03:27:40.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.177+0000 7f99ddd90640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99d80cf130 0x7f99d80ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:40.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.177+0000 7f99ddd90640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f99d80b2fe0 0x7f99d815d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:40.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.177+0000 7f99ddd90640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99d806ceb0 con 0x7f99d80ada30 2023-09-23T03:27:40.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.177+0000 7f99ddd90640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99d80ade10 con 0x7f99d80b2fe0 2023-09-23T03:27:40.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.177+0000 7f99ddd90640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99d8159c90 con 0x7f99d80cf130 2023-09-23T03:27:40.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.178+0000 7f99d7fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f99d80b2fe0 0x7f99d815d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:40.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.178+0000 7f99d7fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f99d80b2fe0 0x7f99d815d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54786/0 (socket says 172.21.15.154:54786) 2023-09-23T03:27:40.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.178+0000 7f99d7fff640 1 -- 172.21.15.154:0/915892804 learned_addr learned my addr 172.21.15.154:0/915892804 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:40.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.178+0000 7f99d7fff640 1 -- 172.21.15.154:0/915892804 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99d80cf130 msgr2=0x7f99d80ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:40.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.178+0000 7f99d7fff640 1 --2- 172.21.15.154:0/915892804 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99d80cf130 0x7f99d80ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:40.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.178+0000 7f99d7fff640 1 -- 172.21.15.154:0/915892804 >> v1:172.21.15.190:6789/0 conn(0x7f99d80ada30 legacy=0x7f99d80cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:40.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.178+0000 7f99d7fff640 1 -- 172.21.15.154:0/915892804 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d806c110 con 0x7f99d80b2fe0 2023-09-23T03:27:40.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.179+0000 7f99d7fff640 1 --2- 172.21.15.154:0/915892804 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99d80b2fe0 0x7f99d815d380 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f99d0009e50 tx=0x7f99d002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b653674b62207c3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:40.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.179+0000 7f99d6ffd640 1 -- 172.21.15.154:0/915892804 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d002ca70 con 0x7f99d80b2fe0 2023-09-23T03:27:40.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.180+0000 7f99d6ffd640 1 -- 172.21.15.154:0/915892804 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d002cbd0 con 0x7f99d80b2fe0 2023-09-23T03:27:40.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.180+0000 7f99d6ffd640 1 -- 172.21.15.154:0/915892804 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d00343f0 con 0x7f99d80b2fe0 2023-09-23T03:27:40.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.180+0000 7f99ddd90640 1 -- 172.21.15.154:0/915892804 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99d80b2fe0 msgr2=0x7f99d815d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:40.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.180+0000 7f99ddd90640 1 --2- 172.21.15.154:0/915892804 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99d80b2fe0 0x7f99d815d380 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f99d0009e50 tx=0x7f99d002bed0 comp rx=0 tx=0).stop 2023-09-23T03:27:40.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.180+0000 7f99ddd90640 1 -- 172.21.15.154:0/915892804 shutdown_connections 2023-09-23T03:27:40.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.180+0000 7f99ddd90640 1 --2- 172.21.15.154:0/915892804 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99d80b2fe0 0x7f99d815d380 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:40.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.180+0000 7f99ddd90640 1 --2- 172.21.15.154:0/915892804 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99d80cf130 0x7f99d80ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:40.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.180+0000 7f99ddd90640 1 -- 172.21.15.154:0/915892804 >> 172.21.15.154:0/915892804 conn(0x7f99d806a0f0 msgr2=0x7f99d806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:40.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.180+0000 7f99ddd90640 1 -- 172.21.15.154:0/915892804 shutdown_connections 2023-09-23T03:27:40.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.180+0000 7f99ddd90640 1 -- 172.21.15.154:0/915892804 wait complete. 2023-09-23T03:27:40.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.182+0000 7f99ddd90640 1 Processor -- start 2023-09-23T03:27:40.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.183+0000 7f99ddd90640 1 -- start start 2023-09-23T03:27:40.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.184+0000 7f99ddd90640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99d80ada30 0x7f99d8154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:40.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.184+0000 7f99ddd90640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99d80b2fe0 0x7f99d80976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:40.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.184+0000 7f99ddd90640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99d80cf130 0x7f99d8161580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:40.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.184+0000 7f99ddd90640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d81532e0 con 0x7f99d80cf130 2023-09-23T03:27:40.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.184+0000 7f99ddd90640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d81538a0 con 0x7f99d80b2fe0 2023-09-23T03:27:40.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.184+0000 7f99ddd90640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99d81550f0 con 0x7f99d80ada30 2023-09-23T03:27:40.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.184+0000 7f99d7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99d80cf130 0x7f99d8161580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:40.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.184+0000 7f99d7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99d80cf130 0x7f99d8161580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54796/0 (socket says 172.21.15.154:54796) 2023-09-23T03:27:40.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.184+0000 7f99d7fff640 1 -- 172.21.15.154:0/1565301884 learned_addr learned my addr 172.21.15.154:0/1565301884 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:40.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.184+0000 7f99d77fe640 1 --2- 172.21.15.154:0/1565301884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99d80ada30 0x7f99d8154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:40.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.184+0000 7f99d7fff640 1 -- 172.21.15.154:0/1565301884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99d80ada30 msgr2=0x7f99d8154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:40.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.184+0000 7f99d7fff640 1 --2- 172.21.15.154:0/1565301884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99d80ada30 0x7f99d8154b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:40.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.184+0000 7f99cffff640 1 --2- 172.21.15.154:0/1565301884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99d80b2fe0 0x7f99d80976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:40.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.184+0000 7f99d7fff640 1 -- 172.21.15.154:0/1565301884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99d80b2fe0 msgr2=0x7f99d80976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:40.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.184+0000 7f99d7fff640 1 --2- 172.21.15.154:0/1565301884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99d80b2fe0 0x7f99d80976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:40.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.184+0000 7f99d7fff640 1 -- 172.21.15.154:0/1565301884 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d0009b30 con 0x7f99d80cf130 2023-09-23T03:27:40.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.184+0000 7f99cffff640 1 --2- 172.21.15.154:0/1565301884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99d80b2fe0 0x7f99d80976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:40.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.185+0000 7f99d7fff640 1 --2- 172.21.15.154:0/1565301884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99d80cf130 0x7f99d8161580 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f99d0036040 tx=0x7f99d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:40.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.185+0000 7f99d4ff9640 1 -- 172.21.15.154:0/1565301884 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d003e450 con 0x7f99d80cf130 2023-09-23T03:27:40.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.185+0000 7f99d4ff9640 1 -- 172.21.15.154:0/1565301884 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d003e5d0 con 0x7f99d80cf130 2023-09-23T03:27:40.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.185+0000 7f99d4ff9640 1 -- 172.21.15.154:0/1565301884 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d003e980 con 0x7f99d80cf130 2023-09-23T03:27:40.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.185+0000 7f99ddd90640 1 -- 172.21.15.154:0/1565301884 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99d81553d0 con 0x7f99d80cf130 2023-09-23T03:27:40.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.185+0000 7f99ddd90640 1 -- 172.21.15.154:0/1565301884 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99d81659c0 con 0x7f99d80cf130 2023-09-23T03:27:40.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.186+0000 7f99ddd90640 1 -- 172.21.15.154:0/1565301884 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99d8153a10 con 0x7f99d80cf130 2023-09-23T03:27:40.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.187+0000 7f99d4ff9640 1 -- 172.21.15.154:0/1565301884 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f99d003eae0 con 0x7f99d80cf130 2023-09-23T03:27:40.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.188+0000 7f99d4ff9640 1 --2- 172.21.15.154:0/1565301884 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f99b806af30 0x7f99b806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:40.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.188+0000 7f99d4ff9640 1 -- 172.21.15.154:0/1565301884 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f99d00ac360 con 0x7f99d80cf130 2023-09-23T03:27:40.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.188+0000 7f99d77fe640 1 --2- 172.21.15.154:0/1565301884 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f99b806af30 0x7f99b806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:40.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.189+0000 7f99d77fe640 1 --2- 172.21.15.154:0/1565301884 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f99b806af30 0x7f99b806d3f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f99c800b7f0 tx=0x7f99c8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:40.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.193+0000 7f99d4ff9640 1 -- 172.21.15.154:0/1565301884 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f99d0077510 con 0x7f99d80cf130 2023-09-23T03:27:40.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.429+0000 7f99ddd90640 1 -- 172.21.15.154:0/1565301884 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f99d806ceb0 con 0x7f99d80cf130 2023-09-23T03:27:40.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.430+0000 7f99d4ff9640 1 -- 172.21.15.154:0/1565301884 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f99d806ceb0 con 0x7f99d80cf130 2023-09-23T03:27:40.431 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:40.431 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:27:40.431 INFO:teuthology.orchestra.run.smithi154.stdout: "auth_dump": [ 2023-09-23T03:27:40.431 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:27:40.431 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mds.a", 2023-09-23T03:27:40.431 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAGWA5l9biCFhAAhD2xgY9Q/gGqK0pgHRGweQ==", 2023-09-23T03:27:40.431 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:27:40.431 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:27:40.432 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2023-09-23T03:27:40.432 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2023-09-23T03:27:40.432 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:27:40.432 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:27:40.432 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:27:40.432 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:27:40.432 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mds.b", 2023-09-23T03:27:40.432 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAGWA5lsgKCNxAAI+vD2ESh3uNsb5OCO0ydSw==", 2023-09-23T03:27:40.432 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:27:40.432 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:27:40.432 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2023-09-23T03:27:40.432 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2023-09-23T03:27:40.432 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:27:40.433 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:27:40.433 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:27:40.433 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:27:40.433 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mds.c", 2023-09-23T03:27:40.433 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAGWA5los4vJhAA2/ewjMZcc3pnm9ukhnoNgw==", 2023-09-23T03:27:40.433 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:27:40.433 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:27:40.433 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2023-09-23T03:27:40.433 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2023-09-23T03:27:40.433 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:27:40.433 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:27:40.433 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:27:40.433 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:27:40.434 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mds.d", 2023-09-23T03:27:40.434 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAHWA5lml/MDBAAHyXgd5zl/O8CeVE8Da4w5w==", 2023-09-23T03:27:40.434 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:27:40.434 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:27:40.434 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2023-09-23T03:27:40.434 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2023-09-23T03:27:40.434 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:27:40.434 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:27:40.434 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:27:40.434 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:27:40.434 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.0", 2023-09-23T03:27:40.434 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAMWA5lT1R4BBAAI+itqub/FadG3aU6LhnmGg==", 2023-09-23T03:27:40.434 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:27:40.435 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:27:40.435 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:27:40.435 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:27:40.435 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:27:40.435 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:27:40.435 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:27:40.435 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.1", 2023-09-23T03:27:40.435 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAPWA5lSablHRAAmNyr5Zopeihn5ZyoY5pyhA==", 2023-09-23T03:27:40.435 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:27:40.435 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:27:40.435 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:27:40.435 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:27:40.436 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:27:40.436 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:27:40.436 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:27:40.436 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.2", 2023-09-23T03:27:40.436 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQASWA5lSFcdJxAA3UJJ6eSvxcQ31SAm4shV2g==", 2023-09-23T03:27:40.436 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:27:40.436 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:27:40.436 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:27:40.436 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:27:40.436 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:27:40.436 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:27:40.436 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:27:40.436 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.3", 2023-09-23T03:27:40.437 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAVWA5lmSvzMRAA97YDqX3oplHCB+SENIkLYA==", 2023-09-23T03:27:40.437 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:27:40.437 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:27:40.437 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:27:40.437 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:27:40.437 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:27:40.437 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:27:40.437 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:27:40.437 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.4", 2023-09-23T03:27:40.437 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAcWA5lPZqwMBAAW95bBB0PKS/rwhXvrPf+mQ==", 2023-09-23T03:27:40.437 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:27:40.437 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:27:40.437 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:27:40.437 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:27:40.438 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:27:40.438 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:27:40.438 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:27:40.438 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.5", 2023-09-23T03:27:40.438 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAgWA5lmZySDRAAsbnUdWq9esKFSqSDeqsVlA==", 2023-09-23T03:27:40.438 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:27:40.438 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:27:40.438 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:27:40.438 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:27:40.438 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:27:40.438 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:27:40.438 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:27:40.438 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.6", 2023-09-23T03:27:40.439 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAjWA5ll9WqJRAAD9jecVxVJU+yuqANvN9QUQ==", 2023-09-23T03:27:40.439 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:27:40.439 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:27:40.439 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:27:40.439 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:27:40.439 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:27:40.439 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:27:40.439 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:27:40.439 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "osd.7", 2023-09-23T03:27:40.439 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAnWA5lCvexAxAA/QymygbvOt1HQdCGbO8cyg==", 2023-09-23T03:27:40.439 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:27:40.439 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow profile osd", 2023-09-23T03:27:40.439 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile osd", 2023-09-23T03:27:40.440 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:27:40.440 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:27:40.440 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:27:40.440 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:27:40.440 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.0", 2023-09-23T03:27:40.440 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAHWA5lYg+qGhAAjaY0fPsmRTz3ipsKbPugMA==", 2023-09-23T03:27:40.440 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:27:40.440 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:27:40.440 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow r", 2023-09-23T03:27:40.440 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-09-23T03:27:40.440 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:27:40.440 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:27:40.440 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:27:40.441 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.1", 2023-09-23T03:27:40.441 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAHWA5lbrrGMBAAn4kvS/1xGjZ+GtRZFqy31w==", 2023-09-23T03:27:40.441 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:27:40.441 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:27:40.441 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow r", 2023-09-23T03:27:40.441 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow rw", 2023-09-23T03:27:40.441 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow rwx" 2023-09-23T03:27:40.441 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:27:40.441 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:27:40.441 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:27:40.441 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.2", 2023-09-23T03:27:40.441 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAHWA5lcLkIJRAAOknGCKwUs78D4atRzLb/Cw==", 2023-09-23T03:27:40.441 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:27:40.441 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:27:40.442 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow r", 2023-09-23T03:27:40.442 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow rw", 2023-09-23T03:27:40.442 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow rwx" 2023-09-23T03:27:40.442 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:27:40.442 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:27:40.442 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:27:40.442 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.3", 2023-09-23T03:27:40.442 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAIWA5lmFUaABAA4EBSJP5IidG2uq4g8lrFKg==", 2023-09-23T03:27:40.442 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:27:40.442 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow", 2023-09-23T03:27:40.442 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow r", 2023-09-23T03:27:40.442 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow rw", 2023-09-23T03:27:40.443 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow rwx" 2023-09-23T03:27:40.443 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:27:40.443 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:27:40.443 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:27:40.443 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "client.admin", 2023-09-23T03:27:40.443 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAFWA5l0x+tCxAAGEYW1egctWzycEhM6FjJZw==", 2023-09-23T03:27:40.443 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:27:40.443 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow *", 2023-09-23T03:27:40.443 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": "allow *", 2023-09-23T03:27:40.443 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow *", 2023-09-23T03:27:40.443 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:27:40.443 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:27:40.443 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:27:40.443 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:27:40.444 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mgr.x", 2023-09-23T03:27:40.444 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAGWA5ljcy5ChAACUOnrxEGqacSxrB9BkZ+lg==", 2023-09-23T03:27:40.444 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:27:40.444 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow *", 2023-09-23T03:27:40.444 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile mgr", 2023-09-23T03:27:40.444 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:27:40.444 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:27:40.444 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:27:40.444 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:27:40.444 INFO:teuthology.orchestra.run.smithi154.stdout: "entity": "mgr.y", 2023-09-23T03:27:40.444 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "AQAFWA5looGaOhAAdxmefaoLuXesHXCeqeCYXQ==", 2023-09-23T03:27:40.445 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": { 2023-09-23T03:27:40.445 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": "allow *", 2023-09-23T03:27:40.445 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": "allow profile mgr", 2023-09-23T03:27:40.445 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": "allow *" 2023-09-23T03:27:40.445 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:27:40.445 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:27:40.445 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2023-09-23T03:27:40.445 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:27:40.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.433+0000 7f99ddd90640 1 -- 172.21.15.154:0/1565301884 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f99b806af30 msgr2=0x7f99b806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:40.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.433+0000 7f99ddd90640 1 --2- 172.21.15.154:0/1565301884 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f99b806af30 0x7f99b806d3f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f99c800b7f0 tx=0x7f99c8008040 comp rx=0 tx=0).stop 2023-09-23T03:27:40.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.433+0000 7f99ddd90640 1 -- 172.21.15.154:0/1565301884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99d80cf130 msgr2=0x7f99d8161580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:40.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.433+0000 7f99ddd90640 1 --2- 172.21.15.154:0/1565301884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99d80cf130 0x7f99d8161580 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f99d0036040 tx=0x7f99d0004a40 comp rx=0 tx=0).stop 2023-09-23T03:27:40.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.433+0000 7f99ddd90640 1 -- 172.21.15.154:0/1565301884 shutdown_connections 2023-09-23T03:27:40.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.433+0000 7f99ddd90640 1 --2- 172.21.15.154:0/1565301884 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f99b806af30 0x7f99b806d3f0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:40.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.433+0000 7f99ddd90640 1 --2- 172.21.15.154:0/1565301884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99d80cf130 0x7f99d8161580 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:40.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.433+0000 7f99ddd90640 1 --2- 172.21.15.154:0/1565301884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99d80b2fe0 0x7f99d80976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:40.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.433+0000 7f99ddd90640 1 --2- 172.21.15.154:0/1565301884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99d80ada30 0x7f99d8154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:40.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.433+0000 7f99ddd90640 1 -- 172.21.15.154:0/1565301884 >> 172.21.15.154:0/1565301884 conn(0x7f99d806a0f0 msgr2=0x7f99d80af610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:40.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.433+0000 7f99ddd90640 1 -- 172.21.15.154:0/1565301884 shutdown_connections 2023-09-23T03:27:40.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.433+0000 7f99ddd90640 1 -- 172.21.15.154:0/1565301884 wait complete. 2023-09-23T03:27:40.451 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:27:40.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.658+0000 7ffb12832640 1 Processor -- start 2023-09-23T03:27:40.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.658+0000 7ffb12832640 1 -- start start 2023-09-23T03:27:40.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.660+0000 7ffb12832640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb0c0acce0 0x7ffb0c15ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:40.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.660+0000 7ffb12832640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ffb0c0cf040 0x7ffb0c0ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:40.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.660+0000 7ffb12832640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb0c15ac30 con 0x7ffb0c0b1b20 2023-09-23T03:27:40.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.660+0000 7ffb12832640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb0c0b2e90 con 0x7ffb0c0cf040 2023-09-23T03:27:40.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.660+0000 7ffb12832640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c0b1f70 con 0x7ffb0c0acce0 2023-09-23T03:27:40.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.660+0000 7ffb0bfff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb0c0acce0 0x7ffb0c15ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:40.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.660+0000 7ffb10dcf640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ffb0c0cf040 0x7ffb0c0ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:40.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.661+0000 7ffb0bfff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb0c0acce0 0x7ffb0c15ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:57400/0 (socket says 172.21.15.154:57400) 2023-09-23T03:27:40.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.661+0000 7ffb0bfff640 1 -- 172.21.15.154:0/241508086 learned_addr learned my addr 172.21.15.154:0/241508086 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:40.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.661+0000 7ffb0bfff640 1 -- 172.21.15.154:0/241508086 >> v2:172.21.15.190:3300/0 conn(0x7ffb0c0cf040 msgr2=0x7ffb0c0ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:40.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.661+0000 7ffb0bfff640 1 --2- 172.21.15.154:0/241508086 >> v2:172.21.15.190:3300/0 conn(0x7ffb0c0cf040 0x7ffb0c0ab140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:40.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.661+0000 7ffb0bfff640 1 -- 172.21.15.154:0/241508086 >> v1:172.21.15.190:6789/0 conn(0x7ffb0c0b1b20 legacy=0x7ffb0c0ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:40.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.661+0000 7ffb0bfff640 1 -- 172.21.15.154:0/241508086 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb0c15f2a0 con 0x7ffb0c0acce0 2023-09-23T03:27:40.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.662+0000 7ffb0bfff640 1 --2- 172.21.15.154:0/241508086 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb0c0acce0 0x7ffb0c15ed60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ffafc009850 tx=0x7ffafc02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dffe5f886a6ded35 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:40.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.663+0000 7ffb0b7fe640 1 -- 172.21.15.154:0/241508086 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc02e070 con 0x7ffb0c0acce0 2023-09-23T03:27:40.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.663+0000 7ffb0b7fe640 1 -- 172.21.15.154:0/241508086 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffafc004030 con 0x7ffb0c0acce0 2023-09-23T03:27:40.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.663+0000 7ffb0b7fe640 1 -- 172.21.15.154:0/241508086 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc004420 con 0x7ffb0c0acce0 2023-09-23T03:27:40.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.664+0000 7ffb12832640 1 -- 172.21.15.154:0/241508086 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb0c0acce0 msgr2=0x7ffb0c15ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:40.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.664+0000 7ffb12832640 1 --2- 172.21.15.154:0/241508086 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb0c0acce0 0x7ffb0c15ed60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ffafc009850 tx=0x7ffafc02b840 comp rx=0 tx=0).stop 2023-09-23T03:27:40.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.664+0000 7ffb12832640 1 -- 172.21.15.154:0/241508086 shutdown_connections 2023-09-23T03:27:40.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.664+0000 7ffb12832640 1 --2- 172.21.15.154:0/241508086 >> v2:172.21.15.190:3300/0 conn(0x7ffb0c0cf040 0x7ffb0c0ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:40.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.664+0000 7ffb12832640 1 --2- 172.21.15.154:0/241508086 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb0c0acce0 0x7ffb0c15ed60 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:40.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.664+0000 7ffb12832640 1 -- 172.21.15.154:0/241508086 >> 172.21.15.154:0/241508086 conn(0x7ffb0c06a0f0 msgr2=0x7ffb0c06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:40.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.664+0000 7ffb12832640 1 -- 172.21.15.154:0/241508086 shutdown_connections 2023-09-23T03:27:40.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.664+0000 7ffb12832640 1 -- 172.21.15.154:0/241508086 wait complete. 2023-09-23T03:27:40.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.666+0000 7ffb12832640 1 Processor -- start 2023-09-23T03:27:40.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.666+0000 7ffb12832640 1 -- start start 2023-09-23T03:27:40.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.667+0000 7ffb12832640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb0c0acce0 0x7ffb0c0c1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:40.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.667+0000 7ffb12832640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb0c0b1b20 0x7ffb0c0c24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:40.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.667+0000 7ffb12832640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb0c0cf040 0x7ffb0c0c3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:40.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.668+0000 7ffb12832640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c0aedd0 con 0x7ffb0c0acce0 2023-09-23T03:27:40.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.668+0000 7ffb12832640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c0cc0e0 con 0x7ffb0c0b1b20 2023-09-23T03:27:40.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.668+0000 7ffb12832640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c1f68c0 con 0x7ffb0c0cf040 2023-09-23T03:27:40.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.668+0000 7ffb0bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb0c0acce0 0x7ffb0c0c1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:40.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.668+0000 7ffb0bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb0c0acce0 0x7ffb0c0c1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54800/0 (socket says 172.21.15.154:54800) 2023-09-23T03:27:40.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.668+0000 7ffb0bfff640 1 -- 172.21.15.154:0/4011327290 learned_addr learned my addr 172.21.15.154:0/4011327290 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:40.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.668+0000 7ffb03fff640 1 --2- 172.21.15.154:0/4011327290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb0c0b1b20 0x7ffb0c0c24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:40.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.668+0000 7ffb10dcf640 1 --2- 172.21.15.154:0/4011327290 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb0c0cf040 0x7ffb0c0c3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:40.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.668+0000 7ffb0bfff640 1 -- 172.21.15.154:0/4011327290 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb0c0cf040 msgr2=0x7ffb0c0c3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:40.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.668+0000 7ffb0bfff640 1 --2- 172.21.15.154:0/4011327290 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb0c0cf040 0x7ffb0c0c3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:40.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.668+0000 7ffb0bfff640 1 -- 172.21.15.154:0/4011327290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb0c0b1b20 msgr2=0x7ffb0c0c24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:40.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.668+0000 7ffb0bfff640 1 --2- 172.21.15.154:0/4011327290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb0c0b1b20 0x7ffb0c0c24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:40.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.668+0000 7ffb0bfff640 1 -- 172.21.15.154:0/4011327290 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffafc009460 con 0x7ffb0c0acce0 2023-09-23T03:27:40.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.669+0000 7ffb0bfff640 1 --2- 172.21.15.154:0/4011327290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb0c0acce0 0x7ffb0c0c1fa0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7ffafc009970 tx=0x7ffafc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:40.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.669+0000 7ffb097fa640 1 -- 172.21.15.154:0/4011327290 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc003960 con 0x7ffb0c0acce0 2023-09-23T03:27:40.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.669+0000 7ffb097fa640 1 -- 172.21.15.154:0/4011327290 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffafc003ac0 con 0x7ffb0c0acce0 2023-09-23T03:27:40.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.669+0000 7ffb097fa640 1 -- 172.21.15.154:0/4011327290 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc003e70 con 0x7ffb0c0acce0 2023-09-23T03:27:40.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.669+0000 7ffb12832640 1 -- 172.21.15.154:0/4011327290 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb0c0af020 con 0x7ffb0c0acce0 2023-09-23T03:27:40.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.669+0000 7ffb12832640 1 -- 172.21.15.154:0/4011327290 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb0c1f6520 con 0x7ffb0c0acce0 2023-09-23T03:27:40.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.670+0000 7ffb12832640 1 -- 172.21.15.154:0/4011327290 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb0c073eb0 con 0x7ffb0c0acce0 2023-09-23T03:27:40.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.671+0000 7ffb097fa640 1 -- 172.21.15.154:0/4011327290 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7ffafc039070 con 0x7ffb0c0acce0 2023-09-23T03:27:40.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.672+0000 7ffb097fa640 1 --2- 172.21.15.154:0/4011327290 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffaec06af60 0x7ffaec06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:40.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.673+0000 7ffb097fa640 1 -- 172.21.15.154:0/4011327290 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5177+0+0 (secure 0 0 0) 0x7ffafc0aaa00 con 0x7ffb0c0acce0 2023-09-23T03:27:40.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.673+0000 7ffb03fff640 1 --2- 172.21.15.154:0/4011327290 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffaec06af60 0x7ffaec06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:40.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.673+0000 7ffb03fff640 1 --2- 172.21.15.154:0/4011327290 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffaec06af60 0x7ffaec06d420 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c072710 tx=0x7ffaf4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:40.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.677+0000 7ffb097fa640 1 -- 172.21.15.154:0/4011327290 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7ffafc075c40 con 0x7ffb0c0acce0 2023-09-23T03:27:40.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.855+0000 7ffb12832640 1 -- 172.21.15.154:0/4011327290 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb0c15ac10 con 0x7ffb0c0acce0 2023-09-23T03:27:40.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.857+0000 7ffb097fa640 1 -- 172.21.15.154:0/4011327290 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+13108 (secure 0 0 0) 0x7ffafc040020 con 0x7ffb0c0acce0 2023-09-23T03:27:40.858 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:40.858 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":122,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:27:39.975068+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:27:40.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.860+0000 7ffb12832640 1 -- 172.21.15.154:0/4011327290 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffaec06af60 msgr2=0x7ffaec06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:40.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.860+0000 7ffb12832640 1 --2- 172.21.15.154:0/4011327290 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffaec06af60 0x7ffaec06d420 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c072710 tx=0x7ffaf4009040 comp rx=0 tx=0).stop 2023-09-23T03:27:40.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.860+0000 7ffb12832640 1 -- 172.21.15.154:0/4011327290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb0c0acce0 msgr2=0x7ffb0c0c1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:40.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.860+0000 7ffb12832640 1 --2- 172.21.15.154:0/4011327290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb0c0acce0 0x7ffb0c0c1fa0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7ffafc009970 tx=0x7ffafc0046c0 comp rx=0 tx=0).stop 2023-09-23T03:27:40.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.860+0000 7ffb12832640 1 -- 172.21.15.154:0/4011327290 shutdown_connections 2023-09-23T03:27:40.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.860+0000 7ffb12832640 1 --2- 172.21.15.154:0/4011327290 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ffaec06af60 0x7ffaec06d420 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:40.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.860+0000 7ffb12832640 1 --2- 172.21.15.154:0/4011327290 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ffb0c0cf040 0x7ffb0c0c3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:40.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.860+0000 7ffb12832640 1 --2- 172.21.15.154:0/4011327290 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ffb0c0b1b20 0x7ffb0c0c24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:40.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.860+0000 7ffb12832640 1 --2- 172.21.15.154:0/4011327290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ffb0c0acce0 0x7ffb0c0c1fa0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:40.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.860+0000 7ffb12832640 1 -- 172.21.15.154:0/4011327290 >> 172.21.15.154:0/4011327290 conn(0x7ffb0c06a0f0 msgr2=0x7ffb0c0b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:40.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.861+0000 7ffb12832640 1 -- 172.21.15.154:0/4011327290 shutdown_connections 2023-09-23T03:27:40.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:40.861+0000 7ffb12832640 1 -- 172.21.15.154:0/4011327290 wait complete. 2023-09-23T03:27:40.879 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T03:27:40.879 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:27:41.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.090+0000 7f84cd72c640 1 Processor -- start 2023-09-23T03:27:41.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.090+0000 7f84cd72c640 1 -- start start 2023-09-23T03:27:41.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.091+0000 7f84cd72c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f84c80c9150 0x7f84c815cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:41.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.091+0000 7f84cd72c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f84c8071290 0x7f84c80ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:41.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.091+0000 7f84cd72c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84c80c9d20 con 0x7f84c80c8d70 2023-09-23T03:27:41.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.091+0000 7f84cd72c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84c8071850 con 0x7f84c80c9150 2023-09-23T03:27:41.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.091+0000 7f84cd72c640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84c8071b30 con 0x7f84c8071290 2023-09-23T03:27:41.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.092+0000 7f84c6ffd640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f84c80c8d70 0x7f84c815c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40780/0 (socket says 172.21.15.154:40780) 2023-09-23T03:27:41.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.092+0000 7f84c6ffd640 1 -- 172.21.15.154:0/4090829166 learned_addr learned my addr 172.21.15.154:0/4090829166 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:27:41.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.092+0000 7f84c67fc640 1 --2- 172.21.15.154:0/4090829166 >> v2:172.21.15.154:3300/0 conn(0x7f84c80c9150 0x7f84c815cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:41.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.092+0000 7f84c77fe640 1 --2- 172.21.15.154:0/4090829166 >> v2:172.21.15.190:3300/0 conn(0x7f84c8071290 0x7f84c80ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:41.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.092+0000 7f84c67fc640 1 -- 172.21.15.154:0/4090829166 >> v2:172.21.15.190:3300/0 conn(0x7f84c8071290 msgr2=0x7f84c80ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:41.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.092+0000 7f84c67fc640 1 --2- 172.21.15.154:0/4090829166 >> v2:172.21.15.190:3300/0 conn(0x7f84c8071290 0x7f84c80ca1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:41.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.092+0000 7f84c67fc640 1 -- 172.21.15.154:0/4090829166 >> v1:172.21.15.154:6789/0 conn(0x7f84c80c8d70 legacy=0x7f84c815c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:41.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.092+0000 7f84c5ffb640 1 -- 172.21.15.154:0/4090829166 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3713595051 0 0) 0x7f84c80c9d20 con 0x7f84c80c8d70 2023-09-23T03:27:41.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.092+0000 7f84c67fc640 1 -- 172.21.15.154:0/4090829166 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c8071e10 con 0x7f84c80c9150 2023-09-23T03:27:41.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.092+0000 7f84c77fe640 1 --2- 172.21.15.154:0/4090829166 >> v2:172.21.15.190:3300/0 conn(0x7f84c8071290 0x7f84c80ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:41.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.093+0000 7f84c67fc640 1 --2- 172.21.15.154:0/4090829166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84c80c9150 0x7f84c815cde0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f84c80b3820 tx=0x7f84bc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c354c7dcd99756 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:41.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.094+0000 7f84c5ffb640 1 -- 172.21.15.154:0/4090829166 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84bc031830 con 0x7f84c80c9150 2023-09-23T03:27:41.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.094+0000 7f84c5ffb640 1 -- 172.21.15.154:0/4090829166 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84bc031990 con 0x7f84c80c9150 2023-09-23T03:27:41.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.094+0000 7f84c5ffb640 1 -- 172.21.15.154:0/4090829166 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84bc031d20 con 0x7f84c80c9150 2023-09-23T03:27:41.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.094+0000 7f84cd72c640 1 -- 172.21.15.154:0/4090829166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84c80c9150 msgr2=0x7f84c815cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:41.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.094+0000 7f84cd72c640 1 --2- 172.21.15.154:0/4090829166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84c80c9150 0x7f84c815cde0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f84c80b3820 tx=0x7f84bc02b8d0 comp rx=0 tx=0).stop 2023-09-23T03:27:41.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.094+0000 7f84cd72c640 1 -- 172.21.15.154:0/4090829166 shutdown_connections 2023-09-23T03:27:41.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.094+0000 7f84cd72c640 1 --2- 172.21.15.154:0/4090829166 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84c80c9150 0x7f84c815cde0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:41.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.094+0000 7f84cd72c640 1 --2- 172.21.15.154:0/4090829166 >> v2:172.21.15.190:3300/0 conn(0x7f84c8071290 0x7f84c80ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:41.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.094+0000 7f84cd72c640 1 -- 172.21.15.154:0/4090829166 >> 172.21.15.154:0/4090829166 conn(0x7f84c806c010 msgr2=0x7f84c806c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:41.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.094+0000 7f84cd72c640 1 -- 172.21.15.154:0/4090829166 shutdown_connections 2023-09-23T03:27:41.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.094+0000 7f84cd72c640 1 -- 172.21.15.154:0/4090829166 wait complete. 2023-09-23T03:27:41.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.096+0000 7f84cd72c640 1 Processor -- start 2023-09-23T03:27:41.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.096+0000 7f84cd72c640 1 -- start start 2023-09-23T03:27:41.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.097+0000 7f84cd72c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84c8071290 0x7f84c814daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:41.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.097+0000 7f84cd72c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84c80c8d70 0x7f84c814d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:41.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.097+0000 7f84cd72c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84c80c9150 0x7f84c80cdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:41.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.097+0000 7f84cd72c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c814e1f0 con 0x7f84c8071290 2023-09-23T03:27:41.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.097+0000 7f84cd72c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c814e4d0 con 0x7f84c80c8d70 2023-09-23T03:27:41.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.097+0000 7f84cd72c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c814e7b0 con 0x7f84c80c9150 2023-09-23T03:27:41.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.097+0000 7f84c6ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84c8071290 0x7f84c814daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:41.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.097+0000 7f84c6ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84c8071290 0x7f84c814daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54820/0 (socket says 172.21.15.154:54820) 2023-09-23T03:27:41.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.097+0000 7f84c6ffd640 1 -- 172.21.15.154:0/3192528938 learned_addr learned my addr 172.21.15.154:0/3192528938 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:41.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.098+0000 7f84c6ffd640 1 -- 172.21.15.154:0/3192528938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84c80c9150 msgr2=0x7f84c80cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:41.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.098+0000 7f84c67fc640 1 --2- 172.21.15.154:0/3192528938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84c80c8d70 0x7f84c814d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:41.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.098+0000 7f84c6ffd640 1 --2- 172.21.15.154:0/3192528938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84c80c9150 0x7f84c80cdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:41.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.098+0000 7f84c6ffd640 1 -- 172.21.15.154:0/3192528938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84c80c8d70 msgr2=0x7f84c814d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:41.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.098+0000 7f84c6ffd640 1 --2- 172.21.15.154:0/3192528938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84c80c8d70 0x7f84c814d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:41.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.098+0000 7f84c6ffd640 1 -- 172.21.15.154:0/3192528938 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84bc009400 con 0x7f84c8071290 2023-09-23T03:27:41.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.098+0000 7f84c6ffd640 1 --2- 172.21.15.154:0/3192528938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84c8071290 0x7f84c814daf0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f84b800c4d0 tx=0x7f84b800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:41.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.098+0000 7f84af7fe640 1 -- 172.21.15.154:0/3192528938 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b80074e0 con 0x7f84c8071290 2023-09-23T03:27:41.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.098+0000 7f84af7fe640 1 -- 172.21.15.154:0/3192528938 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b8007660 con 0x7f84c8071290 2023-09-23T03:27:41.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.098+0000 7f84cd72c640 1 -- 172.21.15.154:0/3192528938 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c814ea90 con 0x7f84c8071290 2023-09-23T03:27:41.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.099+0000 7f84af7fe640 1 -- 172.21.15.154:0/3192528938 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b8004a20 con 0x7f84c8071290 2023-09-23T03:27:41.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.099+0000 7f84cd72c640 1 -- 172.21.15.154:0/3192528938 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c80c6d00 con 0x7f84c8071290 2023-09-23T03:27:41.101 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.100+0000 7f84cd72c640 1 -- 172.21.15.154:0/3192528938 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c80c33c0 con 0x7f84c8071290 2023-09-23T03:27:41.101 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.101+0000 7f84af7fe640 1 -- 172.21.15.154:0/3192528938 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f84b8004b80 con 0x7f84c8071290 2023-09-23T03:27:41.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.102+0000 7f84af7fe640 1 --2- 172.21.15.154:0/3192528938 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f84b406af30 0x7f84b406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:41.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.102+0000 7f84af7fe640 1 -- 172.21.15.154:0/3192528938 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f84b8088b30 con 0x7f84c8071290 2023-09-23T03:27:41.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.102+0000 7f84c67fc640 1 --2- 172.21.15.154:0/3192528938 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f84b406af30 0x7f84b406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:41.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.103+0000 7f84c67fc640 1 --2- 172.21.15.154:0/3192528938 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f84b406af30 0x7f84b406d3f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f84bc036000 tx=0x7f84bc008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:41.106 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.106+0000 7f84af7fe640 1 -- 172.21.15.154:0/3192528938 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f84c80c33c0 con 0x7f84c8071290 2023-09-23T03:27:41.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.297+0000 7f84cd72c640 1 -- 172.21.15.154:0/3192528938 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f84c80c33c0 con 0x7f84c8071290 2023-09-23T03:27:41.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.298+0000 7f84af7fe640 1 -- 172.21.15.154:0/3192528938 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v122) v1 ==== 93+0+10 (secure 0 0 0) 0x7f84b8018020 con 0x7f84c8071290 2023-09-23T03:27:41.298 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:27:41.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.300+0000 7f84cd72c640 1 -- 172.21.15.154:0/3192528938 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f84b406af30 msgr2=0x7f84b406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:41.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.300+0000 7f84cd72c640 1 --2- 172.21.15.154:0/3192528938 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f84b406af30 0x7f84b406d3f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f84bc036000 tx=0x7f84bc008030 comp rx=0 tx=0).stop 2023-09-23T03:27:41.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.300+0000 7f84cd72c640 1 -- 172.21.15.154:0/3192528938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84c8071290 msgr2=0x7f84c814daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:41.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.300+0000 7f84cd72c640 1 --2- 172.21.15.154:0/3192528938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84c8071290 0x7f84c814daf0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f84b800c4d0 tx=0x7f84b800c990 comp rx=0 tx=0).stop 2023-09-23T03:27:41.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.301+0000 7f84cd72c640 1 -- 172.21.15.154:0/3192528938 shutdown_connections 2023-09-23T03:27:41.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.301+0000 7f84cd72c640 1 --2- 172.21.15.154:0/3192528938 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f84b406af30 0x7f84b406d3f0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:41.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.301+0000 7f84cd72c640 1 --2- 172.21.15.154:0/3192528938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f84c80c9150 0x7f84c80cdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:41.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.301+0000 7f84cd72c640 1 --2- 172.21.15.154:0/3192528938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f84c80c8d70 0x7f84c814d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:41.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.301+0000 7f84cd72c640 1 --2- 172.21.15.154:0/3192528938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f84c8071290 0x7f84c814daf0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:41.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.301+0000 7f84cd72c640 1 -- 172.21.15.154:0/3192528938 >> 172.21.15.154:0/3192528938 conn(0x7f84c806c010 msgr2=0x7f84c8150b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:41.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.301+0000 7f84cd72c640 1 -- 172.21.15.154:0/3192528938 shutdown_connections 2023-09-23T03:27:41.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.301+0000 7f84cd72c640 1 -- 172.21.15.154:0/3192528938 wait complete. 2023-09-23T03:27:41.319 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-09-23T03:27:41.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.520+0000 7fb7bc2d4640 1 Processor -- start 2023-09-23T03:27:41.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.520+0000 7fb7bc2d4640 1 -- start start 2023-09-23T03:27:41.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.521+0000 7fb7bc2d4640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb7b40acce0 0x7fb7b415ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:41.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.521+0000 7fb7bc2d4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7b40cf040 0x7fb7b40ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:41.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.521+0000 7fb7bc2d4640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7b415ac30 con 0x7fb7b40b1b20 2023-09-23T03:27:41.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.521+0000 7fb7bc2d4640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7b40b2e90 con 0x7fb7b40acce0 2023-09-23T03:27:41.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.521+0000 7fb7bc2d4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b40b1f70 con 0x7fb7b40cf040 2023-09-23T03:27:41.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.522+0000 7fb7b986f640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fb7b40b1b20 0x7fb7b40ce930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:40790/0 (socket says 172.21.15.154:40790) 2023-09-23T03:27:41.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.522+0000 7fb7b986f640 1 -- 172.21.15.154:0/1007965593 learned_addr learned my addr 172.21.15.154:0/1007965593 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:27:41.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.522+0000 7fb7ba871640 1 --2- 172.21.15.154:0/1007965593 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7b40cf040 0x7fb7b40ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:41.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.522+0000 7fb7ba070640 1 --2- 172.21.15.154:0/1007965593 >> v2:172.21.15.190:3300/0 conn(0x7fb7b40acce0 0x7fb7b415ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:41.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.522+0000 7fb7b906e640 1 -- 172.21.15.154:0/1007965593 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3499128024 0 0) 0x7fb7a4001110 con 0x7fb7b40b1b20 2023-09-23T03:27:41.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.522+0000 7fb7b906e640 1 -- 172.21.15.154:0/1007965593 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7a0003520 con 0x7fb7b40b1b20 2023-09-23T03:27:41.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.523+0000 7fb7ba070640 1 -- 172.21.15.154:0/1007965593 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7b40cf040 msgr2=0x7fb7b40ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:41.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.523+0000 7fb7ba070640 1 --2- 172.21.15.154:0/1007965593 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7b40cf040 0x7fb7b40ab140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:41.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.523+0000 7fb7ba070640 1 -- 172.21.15.154:0/1007965593 >> v1:172.21.15.154:6789/0 conn(0x7fb7b40b1b20 legacy=0x7fb7b40ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:41.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.523+0000 7fb7b906e640 1 -- 172.21.15.154:0/1007965593 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3850456977 0 0) 0x7fb7a0003520 con 0x7fb7b40b1b20 2023-09-23T03:27:41.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.523+0000 7fb7ba070640 1 -- 172.21.15.154:0/1007965593 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b415f2a0 con 0x7fb7b40acce0 2023-09-23T03:27:41.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.524+0000 7fb7ba070640 1 --2- 172.21.15.154:0/1007965593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7b40acce0 0x7fb7b415ed60 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0009790 tx=0x7fb7b002b6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86a05a13787459c6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:41.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.524+0000 7fb7b906e640 1 -- 172.21.15.154:0/1007965593 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a4001800 con 0x7fb7b40acce0 2023-09-23T03:27:41.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.524+0000 7fb7b906e640 1 -- 172.21.15.154:0/1007965593 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b002e050 con 0x7fb7b40acce0 2023-09-23T03:27:41.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.524+0000 7fb7b906e640 1 -- 172.21.15.154:0/1007965593 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b0030730 con 0x7fb7b40acce0 2023-09-23T03:27:41.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.525+0000 7fb7bc2d4640 1 -- 172.21.15.154:0/1007965593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7b40acce0 msgr2=0x7fb7b415ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:41.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.525+0000 7fb7bc2d4640 1 --2- 172.21.15.154:0/1007965593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7b40acce0 0x7fb7b415ed60 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0009790 tx=0x7fb7b002b6e0 comp rx=0 tx=0).stop 2023-09-23T03:27:41.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.525+0000 7fb7bc2d4640 1 -- 172.21.15.154:0/1007965593 shutdown_connections 2023-09-23T03:27:41.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.525+0000 7fb7bc2d4640 1 --2- 172.21.15.154:0/1007965593 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7b40cf040 0x7fb7b40ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:41.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.525+0000 7fb7bc2d4640 1 --2- 172.21.15.154:0/1007965593 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7b40acce0 0x7fb7b415ed60 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:41.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.525+0000 7fb7bc2d4640 1 -- 172.21.15.154:0/1007965593 >> 172.21.15.154:0/1007965593 conn(0x7fb7b406a0f0 msgr2=0x7fb7b406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:41.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.525+0000 7fb7bc2d4640 1 -- 172.21.15.154:0/1007965593 shutdown_connections 2023-09-23T03:27:41.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.525+0000 7fb7bc2d4640 1 -- 172.21.15.154:0/1007965593 wait complete. 2023-09-23T03:27:41.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.527+0000 7fb7bc2d4640 1 Processor -- start 2023-09-23T03:27:41.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.527+0000 7fb7bc2d4640 1 -- start start 2023-09-23T03:27:41.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.528+0000 7fb7bc2d4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7b40acce0 0x7fb7b415add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:41.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.528+0000 7fb7bc2d4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7b40b1b20 0x7fb7b4158e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:41.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.528+0000 7fb7bc2d4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7b40cf040 0x7fb7b4159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:41.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.528+0000 7fb7bc2d4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b4157c70 con 0x7fb7b40cf040 2023-09-23T03:27:41.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.528+0000 7fb7bc2d4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b41f3820 con 0x7fb7b40b1b20 2023-09-23T03:27:41.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.528+0000 7fb7bc2d4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b41f3a90 con 0x7fb7b40acce0 2023-09-23T03:27:41.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.528+0000 7fb7ba871640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7b40cf040 0x7fb7b4159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:41.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.528+0000 7fb7ba871640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7b40cf040 0x7fb7b4159370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54828/0 (socket says 172.21.15.154:54828) 2023-09-23T03:27:41.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.528+0000 7fb7ba871640 1 -- 172.21.15.154:0/320108389 learned_addr learned my addr 172.21.15.154:0/320108389 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:41.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.528+0000 7fb7ba070640 1 --2- 172.21.15.154:0/320108389 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7b40acce0 0x7fb7b415add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:41.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.528+0000 7fb7b986f640 1 --2- 172.21.15.154:0/320108389 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7b40b1b20 0x7fb7b4158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:41.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.528+0000 7fb7ba871640 1 -- 172.21.15.154:0/320108389 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7b40acce0 msgr2=0x7fb7b415add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:41.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.528+0000 7fb7ba871640 1 --2- 172.21.15.154:0/320108389 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7b40acce0 0x7fb7b415add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:41.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.528+0000 7fb7ba871640 1 -- 172.21.15.154:0/320108389 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7b40b1b20 msgr2=0x7fb7b4158e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:41.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.528+0000 7fb7ba871640 1 --2- 172.21.15.154:0/320108389 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7b40b1b20 0x7fb7b4158e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:41.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.528+0000 7fb7ba871640 1 -- 172.21.15.154:0/320108389 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b00093a0 con 0x7fb7b40cf040 2023-09-23T03:27:41.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.529+0000 7fb7ba871640 1 --2- 172.21.15.154:0/320108389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7b40cf040 0x7fb7b4159370 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac00efc0 tx=0x7fb7ac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:41.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.529+0000 7fb7aaffd640 1 -- 172.21.15.154:0/320108389 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac009ba0 con 0x7fb7b40cf040 2023-09-23T03:27:41.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.529+0000 7fb7aaffd640 1 -- 172.21.15.154:0/320108389 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7ac00f040 con 0x7fb7b40cf040 2023-09-23T03:27:41.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.529+0000 7fb7aaffd640 1 -- 172.21.15.154:0/320108389 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac0146c0 con 0x7fb7b40cf040 2023-09-23T03:27:41.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.529+0000 7fb7bc2d4640 1 -- 172.21.15.154:0/320108389 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7b41f3d70 con 0x7fb7b40cf040 2023-09-23T03:27:41.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.529+0000 7fb7bc2d4640 1 -- 172.21.15.154:0/320108389 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7b41f4330 con 0x7fb7b40cf040 2023-09-23T03:27:41.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.530+0000 7fb7bc2d4640 1 -- 172.21.15.154:0/320108389 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7b41f4550 con 0x7fb7b40cf040 2023-09-23T03:27:41.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.532+0000 7fb7aaffd640 1 -- 172.21.15.154:0/320108389 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fb7ac014820 con 0x7fb7b40cf040 2023-09-23T03:27:41.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.532+0000 7fb7aaffd640 1 --2- 172.21.15.154:0/320108389 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb79806af30 0x7fb79806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:41.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.532+0000 7fb7aaffd640 1 -- 172.21.15.154:0/320108389 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fb7ac08a330 con 0x7fb7b40cf040 2023-09-23T03:27:41.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.533+0000 7fb7ba070640 1 --2- 172.21.15.154:0/320108389 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb79806af30 0x7fb79806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:41.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.533+0000 7fb7ba070640 1 --2- 172.21.15.154:0/320108389 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb79806af30 0x7fb79806d3f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0034000 tx=0x7fb7b0008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:41.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.537+0000 7fb7aaffd640 1 -- 172.21.15.154:0/320108389 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fb7ac055580 con 0x7fb7b40cf040 2023-09-23T03:27:41.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.715+0000 7fb7bc2d4640 1 -- 172.21.15.154:0/320108389 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb7b4073eb0 con 0x7fb7b40cf040 2023-09-23T03:27:41.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.715+0000 7fb7aaffd640 1 -- 172.21.15.154:0/320108389 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v122) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb7ac018060 con 0x7fb7b40cf040 2023-09-23T03:27:41.716 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:41.716 INFO:teuthology.orchestra.run.smithi154.stdout:[ 2023-09-23T03:27:41.716 INFO:teuthology.orchestra.run.smithi154.stdout: { 2023-09-23T03:27:41.716 INFO:teuthology.orchestra.run.smithi154.stdout: "poolnum": 1, 2023-09-23T03:27:41.716 INFO:teuthology.orchestra.run.smithi154.stdout: "poolname": ".mgr" 2023-09-23T03:27:41.716 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:27:41.716 INFO:teuthology.orchestra.run.smithi154.stdout:] 2023-09-23T03:27:41.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.718+0000 7fb7bc2d4640 1 -- 172.21.15.154:0/320108389 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb79806af30 msgr2=0x7fb79806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:41.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.718+0000 7fb7bc2d4640 1 --2- 172.21.15.154:0/320108389 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb79806af30 0x7fb79806d3f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0034000 tx=0x7fb7b0008030 comp rx=0 tx=0).stop 2023-09-23T03:27:41.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.718+0000 7fb7bc2d4640 1 -- 172.21.15.154:0/320108389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7b40cf040 msgr2=0x7fb7b4159370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:41.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.718+0000 7fb7bc2d4640 1 --2- 172.21.15.154:0/320108389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7b40cf040 0x7fb7b4159370 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac00efc0 tx=0x7fb7ac00c490 comp rx=0 tx=0).stop 2023-09-23T03:27:41.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.718+0000 7fb7bc2d4640 1 -- 172.21.15.154:0/320108389 shutdown_connections 2023-09-23T03:27:41.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.718+0000 7fb7bc2d4640 1 --2- 172.21.15.154:0/320108389 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb79806af30 0x7fb79806d3f0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:41.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.718+0000 7fb7bc2d4640 1 --2- 172.21.15.154:0/320108389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7b40cf040 0x7fb7b4159370 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:41.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.718+0000 7fb7bc2d4640 1 --2- 172.21.15.154:0/320108389 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb7b40b1b20 0x7fb7b4158e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:41.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.718+0000 7fb7bc2d4640 1 --2- 172.21.15.154:0/320108389 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb7b40acce0 0x7fb7b415add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:41.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.718+0000 7fb7bc2d4640 1 -- 172.21.15.154:0/320108389 >> 172.21.15.154:0/320108389 conn(0x7fb7b406a0f0 msgr2=0x7fb7b40726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:41.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.718+0000 7fb7bc2d4640 1 -- 172.21.15.154:0/320108389 shutdown_connections 2023-09-23T03:27:41.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.718+0000 7fb7bc2d4640 1 -- 172.21.15.154:0/320108389 wait complete. 2023-09-23T03:27:41.736 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-09-23T03:27:41.737 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-09-23T03:27:41.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.948+0000 7fce79d5e640 1 Processor -- start 2023-09-23T03:27:41.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.949+0000 7fce79d5e640 1 -- start start 2023-09-23T03:27:41.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.950+0000 7fce79d5e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fce740700f0 0x7fce740c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:41.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.950+0000 7fce79d5e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fce740cb460 0x7fce740b1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:41.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.950+0000 7fce79d5e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce740c95b0 con 0x7fce7415abd0 2023-09-23T03:27:41.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.950+0000 7fce79d5e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce740c9890 con 0x7fce740cb460 2023-09-23T03:27:41.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.950+0000 7fce79d5e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce740c9b70 con 0x7fce740700f0 2023-09-23T03:27:41.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.951+0000 7fce73fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fce740cb460 0x7fce740b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:41.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.951+0000 7fce73fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fce740cb460 0x7fce740b1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54836/0 (socket says 172.21.15.154:54836) 2023-09-23T03:27:41.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.951+0000 7fce73fff640 1 -- 172.21.15.154:0/3112203466 learned_addr learned my addr 172.21.15.154:0/3112203466 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:41.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.951+0000 7fce737fe640 1 --2- 172.21.15.154:0/3112203466 >> v2:172.21.15.190:3300/0 conn(0x7fce740700f0 0x7fce740c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:41.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.951+0000 7fce73fff640 1 -- 172.21.15.154:0/3112203466 >> v2:172.21.15.190:3300/0 conn(0x7fce740700f0 msgr2=0x7fce740c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:41.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.951+0000 7fce73fff640 1 --2- 172.21.15.154:0/3112203466 >> v2:172.21.15.190:3300/0 conn(0x7fce740700f0 0x7fce740c8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:41.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.951+0000 7fce73fff640 1 -- 172.21.15.154:0/3112203466 >> v1:172.21.15.190:6789/0 conn(0x7fce7415abd0 legacy=0x7fce740704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:41.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.951+0000 7fce73fff640 1 -- 172.21.15.154:0/3112203466 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce740c9e50 con 0x7fce740cb460 2023-09-23T03:27:41.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.952+0000 7fce73fff640 1 --2- 172.21.15.154:0/3112203466 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce740cb460 0x7fce740b1420 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fce740cbc40 tx=0x7fce6c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e980251a14b52c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:41.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.953+0000 7fce72ffd640 1 -- 172.21.15.154:0/3112203466 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c031870 con 0x7fce740cb460 2023-09-23T03:27:41.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.953+0000 7fce72ffd640 1 -- 172.21.15.154:0/3112203466 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6c0319d0 con 0x7fce740cb460 2023-09-23T03:27:41.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.953+0000 7fce72ffd640 1 -- 172.21.15.154:0/3112203466 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c031d60 con 0x7fce740cb460 2023-09-23T03:27:41.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.953+0000 7fce79d5e640 1 -- 172.21.15.154:0/3112203466 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce740cb460 msgr2=0x7fce740b1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:41.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.953+0000 7fce79d5e640 1 --2- 172.21.15.154:0/3112203466 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce740cb460 0x7fce740b1420 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fce740cbc40 tx=0x7fce6c02b910 comp rx=0 tx=0).stop 2023-09-23T03:27:41.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.953+0000 7fce79d5e640 1 -- 172.21.15.154:0/3112203466 shutdown_connections 2023-09-23T03:27:41.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.953+0000 7fce79d5e640 1 --2- 172.21.15.154:0/3112203466 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce740cb460 0x7fce740b1420 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:41.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.953+0000 7fce79d5e640 1 --2- 172.21.15.154:0/3112203466 >> v2:172.21.15.190:3300/0 conn(0x7fce740700f0 0x7fce740c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:41.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.953+0000 7fce79d5e640 1 -- 172.21.15.154:0/3112203466 >> 172.21.15.154:0/3112203466 conn(0x7fce7406a520 msgr2=0x7fce7406bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:41.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.953+0000 7fce79d5e640 1 -- 172.21.15.154:0/3112203466 shutdown_connections 2023-09-23T03:27:41.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.953+0000 7fce79d5e640 1 -- 172.21.15.154:0/3112203466 wait complete. 2023-09-23T03:27:41.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.955+0000 7fce79d5e640 1 Processor -- start 2023-09-23T03:27:41.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.955+0000 7fce79d5e640 1 -- start start 2023-09-23T03:27:41.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.956+0000 7fce79d5e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce740700f0 0x7fce740c7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:41.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.956+0000 7fce79d5e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce740cb460 0x7fce740c6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:41.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.956+0000 7fce79d5e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce7415abd0 0x7fce740c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:41.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.956+0000 7fce79d5e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce74146660 con 0x7fce740700f0 2023-09-23T03:27:41.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.956+0000 7fce79d5e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce74143a90 con 0x7fce7415abd0 2023-09-23T03:27:41.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.956+0000 7fce79d5e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce74143d70 con 0x7fce740cb460 2023-09-23T03:27:41.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.956+0000 7fce737fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce740700f0 0x7fce740c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:41.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.957+0000 7fce737fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce740700f0 0x7fce740c7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54848/0 (socket says 172.21.15.154:54848) 2023-09-23T03:27:41.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.957+0000 7fce737fe640 1 -- 172.21.15.154:0/278737381 learned_addr learned my addr 172.21.15.154:0/278737381 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:41.958 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.957+0000 7fce73fff640 1 --2- 172.21.15.154:0/278737381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce7415abd0 0x7fce740c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:41.958 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.957+0000 7fce6affd640 1 --2- 172.21.15.154:0/278737381 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce740cb460 0x7fce740c6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:41.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.957+0000 7fce737fe640 1 -- 172.21.15.154:0/278737381 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce740cb460 msgr2=0x7fce740c6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:41.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.957+0000 7fce737fe640 1 --2- 172.21.15.154:0/278737381 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce740cb460 0x7fce740c6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:41.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.957+0000 7fce737fe640 1 -- 172.21.15.154:0/278737381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce7415abd0 msgr2=0x7fce740c6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:41.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.957+0000 7fce737fe640 1 --2- 172.21.15.154:0/278737381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce7415abd0 0x7fce740c6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:41.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.957+0000 7fce737fe640 1 -- 172.21.15.154:0/278737381 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6c0094d0 con 0x7fce740700f0 2023-09-23T03:27:41.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.957+0000 7fce73fff640 1 --2- 172.21.15.154:0/278737381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce7415abd0 0x7fce740c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:41.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.957+0000 7fce737fe640 1 --2- 172.21.15.154:0/278737381 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce740700f0 0x7fce740c7740 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fce64007ef0 tx=0x7fce6400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:41.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.958+0000 7fce70ff9640 1 -- 172.21.15.154:0/278737381 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce64014620 con 0x7fce740700f0 2023-09-23T03:27:41.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.958+0000 7fce70ff9640 1 -- 172.21.15.154:0/278737381 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce64014780 con 0x7fce740700f0 2023-09-23T03:27:41.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.958+0000 7fce70ff9640 1 -- 172.21.15.154:0/278737381 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce64014b30 con 0x7fce740700f0 2023-09-23T03:27:41.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.958+0000 7fce79d5e640 1 -- 172.21.15.154:0/278737381 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce74144050 con 0x7fce740700f0 2023-09-23T03:27:41.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.958+0000 7fce79d5e640 1 -- 172.21.15.154:0/278737381 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce74144610 con 0x7fce740700f0 2023-09-23T03:27:41.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.959+0000 7fce79d5e640 1 -- 172.21.15.154:0/278737381 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce741447d0 con 0x7fce740700f0 2023-09-23T03:27:41.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.960+0000 7fce70ff9640 1 -- 172.21.15.154:0/278737381 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fce64014c90 con 0x7fce740700f0 2023-09-23T03:27:41.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.961+0000 7fce70ff9640 1 --2- 172.21.15.154:0/278737381 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fce5406af30 0x7fce5406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:41.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.961+0000 7fce70ff9640 1 -- 172.21.15.154:0/278737381 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fce64089fd0 con 0x7fce740700f0 2023-09-23T03:27:41.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.961+0000 7fce6affd640 1 --2- 172.21.15.154:0/278737381 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fce5406af30 0x7fce5406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:41.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.962+0000 7fce6affd640 1 --2- 172.21.15.154:0/278737381 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fce5406af30 0x7fce5406d3f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fce740c8190 tx=0x7fce5c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:41.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:41.965+0000 7fce70ff9640 1 -- 172.21.15.154:0/278737381 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fce64055180 con 0x7fce740700f0 2023-09-23T03:27:42.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:42.160+0000 7fce79d5e640 1 -- 172.21.15.154:0/278737381 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fce740c1db0 con 0x7fce740700f0 2023-09-23T03:27:42.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:42.979+0000 7fce70ff9640 1 -- 172.21.15.154:0/278737381 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v123) v1 ==== 138+0+0 (secure 0 0 0) 0x7fce640587a0 con 0x7fce740700f0 2023-09-23T03:27:42.980 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_metadata' created 2023-09-23T03:27:42.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:42.983+0000 7fce79d5e640 1 -- 172.21.15.154:0/278737381 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fce5406af30 msgr2=0x7fce5406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:42.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:42.983+0000 7fce79d5e640 1 --2- 172.21.15.154:0/278737381 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fce5406af30 0x7fce5406d3f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fce740c8190 tx=0x7fce5c008040 comp rx=0 tx=0).stop 2023-09-23T03:27:42.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:42.983+0000 7fce79d5e640 1 -- 172.21.15.154:0/278737381 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce740700f0 msgr2=0x7fce740c7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:42.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:42.983+0000 7fce79d5e640 1 --2- 172.21.15.154:0/278737381 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce740700f0 0x7fce740c7740 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fce64007ef0 tx=0x7fce6400e4b0 comp rx=0 tx=0).stop 2023-09-23T03:27:42.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:42.983+0000 7fce79d5e640 1 -- 172.21.15.154:0/278737381 shutdown_connections 2023-09-23T03:27:42.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:42.983+0000 7fce79d5e640 1 --2- 172.21.15.154:0/278737381 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fce5406af30 0x7fce5406d3f0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:42.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:42.983+0000 7fce79d5e640 1 --2- 172.21.15.154:0/278737381 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fce7415abd0 0x7fce740c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:42.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:42.983+0000 7fce79d5e640 1 --2- 172.21.15.154:0/278737381 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fce740cb460 0x7fce740c6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:42.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:42.983+0000 7fce79d5e640 1 --2- 172.21.15.154:0/278737381 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fce740700f0 0x7fce740c7740 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:42.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:42.983+0000 7fce79d5e640 1 -- 172.21.15.154:0/278737381 >> 172.21.15.154:0/278737381 conn(0x7fce7406a520 msgr2=0x7fce740ce340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:42.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:42.984+0000 7fce79d5e640 1 -- 172.21.15.154:0/278737381 shutdown_connections 2023-09-23T03:27:42.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:42.984+0000 7fce79d5e640 1 -- 172.21.15.154:0/278737381 wait complete. 2023-09-23T03:27:43.009 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-09-23T03:27:43.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.221+0000 7f11994f2640 1 Processor -- start 2023-09-23T03:27:43.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.222+0000 7f11994f2640 1 -- start start 2023-09-23T03:27:43.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.223+0000 7f11994f2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f11940c9150 0x7f119415cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:43.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.223+0000 7f11994f2640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11940c9d20 con 0x7f1194071290 2023-09-23T03:27:43.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.223+0000 7f11994f2640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1194071850 con 0x7f11940c8d70 2023-09-23T03:27:43.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.223+0000 7f11994f2640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1194071b30 con 0x7f11940c9150 2023-09-23T03:27:43.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.224+0000 7f11927fc640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f11940c9150 0x7f119415cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:43.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.224+0000 7f11927fc640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f11940c9150 0x7f119415cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54850/0 (socket says 172.21.15.154:54850) 2023-09-23T03:27:43.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.224+0000 7f11927fc640 1 -- 172.21.15.154:0/2213400808 learned_addr learned my addr 172.21.15.154:0/2213400808 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:43.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.224+0000 7f11927fc640 1 -- 172.21.15.154:0/2213400808 >> v1:172.21.15.190:6789/0 conn(0x7f11940c8d70 legacy=0x7f119415c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:43.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.224+0000 7f11927fc640 1 -- 172.21.15.154:0/2213400808 >> v1:172.21.15.154:6789/0 conn(0x7f1194071290 legacy=0x7f11940ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:43.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.224+0000 7f11927fc640 1 -- 172.21.15.154:0/2213400808 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1194071e10 con 0x7f11940c9150 2023-09-23T03:27:43.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.224+0000 7f1191ffb640 1 -- 172.21.15.154:0/2213400808 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3424075694 0 0) 0x7f11940c9d20 con 0x7f1194071290 2023-09-23T03:27:43.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.225+0000 7f11927fc640 1 --2- 172.21.15.154:0/2213400808 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11940c9150 0x7f119415cde0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f117c009870 tx=0x7f117c02ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad2956c6355d260b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:43.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.225+0000 7f1191ffb640 1 -- 172.21.15.154:0/2213400808 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c02f840 con 0x7f11940c9150 2023-09-23T03:27:43.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.226+0000 7f1191ffb640 1 -- 172.21.15.154:0/2213400808 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117c02f9a0 con 0x7f11940c9150 2023-09-23T03:27:43.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.226+0000 7f1191ffb640 1 -- 172.21.15.154:0/2213400808 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c02fd30 con 0x7f11940c9150 2023-09-23T03:27:43.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.226+0000 7f11994f2640 1 -- 172.21.15.154:0/2213400808 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11940c9150 msgr2=0x7f119415cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:43.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.226+0000 7f11994f2640 1 --2- 172.21.15.154:0/2213400808 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11940c9150 0x7f119415cde0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f117c009870 tx=0x7f117c02ba20 comp rx=0 tx=0).stop 2023-09-23T03:27:43.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.226+0000 7f11994f2640 1 -- 172.21.15.154:0/2213400808 shutdown_connections 2023-09-23T03:27:43.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.226+0000 7f11994f2640 1 --2- 172.21.15.154:0/2213400808 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11940c9150 0x7f119415cde0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:43.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.226+0000 7f11994f2640 1 -- 172.21.15.154:0/2213400808 >> 172.21.15.154:0/2213400808 conn(0x7f119406c010 msgr2=0x7f119406c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:43.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.226+0000 7f11994f2640 1 -- 172.21.15.154:0/2213400808 shutdown_connections 2023-09-23T03:27:43.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.226+0000 7f11994f2640 1 -- 172.21.15.154:0/2213400808 wait complete. 2023-09-23T03:27:43.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.228+0000 7f11994f2640 1 Processor -- start 2023-09-23T03:27:43.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.228+0000 7f11994f2640 1 -- start start 2023-09-23T03:27:43.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.229+0000 7f11994f2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1194071290 0x7f11941527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:43.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.229+0000 7f11994f2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11940c8d70 0x7f1194144990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:43.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.229+0000 7f11994f2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11940c9150 0x7f11940aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:43.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.229+0000 7f11994f2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11940b1bb0 con 0x7f11940c9150 2023-09-23T03:27:43.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.229+0000 7f11994f2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11940b1e90 con 0x7f1194071290 2023-09-23T03:27:43.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.229+0000 7f11994f2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11940b2170 con 0x7f11940c8d70 2023-09-23T03:27:43.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.230+0000 7f11937fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11940c9150 0x7f11940aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:43.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.230+0000 7f11937fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11940c9150 0x7f11940aef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:54864/0 (socket says 172.21.15.154:54864) 2023-09-23T03:27:43.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.230+0000 7f11937fe640 1 -- 172.21.15.154:0/733380799 learned_addr learned my addr 172.21.15.154:0/733380799 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:43.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.230+0000 7f1192ffd640 1 --2- 172.21.15.154:0/733380799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1194071290 0x7f11941527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:43.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.230+0000 7f11937fe640 1 -- 172.21.15.154:0/733380799 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11940c8d70 msgr2=0x7f1194144990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:43.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.230+0000 7f11937fe640 1 --2- 172.21.15.154:0/733380799 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11940c8d70 0x7f1194144990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:43.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.230+0000 7f11937fe640 1 -- 172.21.15.154:0/733380799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1194071290 msgr2=0x7f11941527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:43.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.230+0000 7f11937fe640 1 --2- 172.21.15.154:0/733380799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1194071290 0x7f11941527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:43.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.230+0000 7f11937fe640 1 -- 172.21.15.154:0/733380799 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117c0094d0 con 0x7f11940c9150 2023-09-23T03:27:43.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.230+0000 7f1192ffd640 1 --2- 172.21.15.154:0/733380799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1194071290 0x7f11941527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:43.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.230+0000 7f11937fe640 1 --2- 172.21.15.154:0/733380799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11940c9150 0x7f11940aef90 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f118c00c760 tx=0x7f118c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:43.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.231+0000 7f118b7fe640 1 -- 172.21.15.154:0/733380799 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118c003da0 con 0x7f11940c9150 2023-09-23T03:27:43.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.231+0000 7f118b7fe640 1 -- 172.21.15.154:0/733380799 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f118c00ce70 con 0x7f11940c9150 2023-09-23T03:27:43.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.231+0000 7f118b7fe640 1 -- 172.21.15.154:0/733380799 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118c005290 con 0x7f11940c9150 2023-09-23T03:27:43.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.231+0000 7f11994f2640 1 -- 172.21.15.154:0/733380799 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11940b2450 con 0x7f11940c9150 2023-09-23T03:27:43.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.231+0000 7f11994f2640 1 -- 172.21.15.154:0/733380799 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11940b2c50 con 0x7f11940c9150 2023-09-23T03:27:43.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.232+0000 7f11994f2640 1 -- 172.21.15.154:0/733380799 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11940b2970 con 0x7f11940c9150 2023-09-23T03:27:43.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.233+0000 7f118b7fe640 1 -- 172.21.15.154:0/733380799 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f118c0053f0 con 0x7f11940c9150 2023-09-23T03:27:43.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.234+0000 7f118b7fe640 1 --2- 172.21.15.154:0/733380799 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f117406af30 0x7f117406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:43.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.234+0000 7f118b7fe640 1 -- 172.21.15.154:0/733380799 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f118c0895a0 con 0x7f11940c9150 2023-09-23T03:27:43.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.235+0000 7f1192ffd640 1 --2- 172.21.15.154:0/733380799 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f117406af30 0x7f117406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:43.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.235+0000 7f1192ffd640 1 --2- 172.21.15.154:0/733380799 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f117406af30 0x7f117406d3f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f11940c8900 tx=0x7f1184009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:43.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.239+0000 7f118b7fe640 1 -- 172.21.15.154:0/733380799 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f11940b2970 con 0x7f11940c9150 2023-09-23T03:27:43.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.420+0000 7f11994f2640 1 -- 172.21.15.154:0/733380799 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f11940b2970 con 0x7f11940c9150 2023-09-23T03:27:43.997 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:43.997+0000 7f118b7fe640 1 -- 172.21.15.154:0/733380799 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v124) v1 ==== 170+0+0 (secure 0 0 0) 0x7f11940b2970 con 0x7f11940c9150 2023-09-23T03:27:43.997 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_data' created 2023-09-23T03:27:44.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.000+0000 7f11994f2640 1 -- 172.21.15.154:0/733380799 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f117406af30 msgr2=0x7f117406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:44.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.000+0000 7f11994f2640 1 --2- 172.21.15.154:0/733380799 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f117406af30 0x7f117406d3f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f11940c8900 tx=0x7f1184009040 comp rx=0 tx=0).stop 2023-09-23T03:27:44.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.000+0000 7f11994f2640 1 -- 172.21.15.154:0/733380799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11940c9150 msgr2=0x7f11940aef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:44.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.000+0000 7f11994f2640 1 --2- 172.21.15.154:0/733380799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11940c9150 0x7f11940aef90 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f118c00c760 tx=0x7f118c00cc20 comp rx=0 tx=0).stop 2023-09-23T03:27:44.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.000+0000 7f11994f2640 1 -- 172.21.15.154:0/733380799 shutdown_connections 2023-09-23T03:27:44.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.000+0000 7f11994f2640 1 --2- 172.21.15.154:0/733380799 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f117406af30 0x7f117406d3f0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:44.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.001+0000 7f11994f2640 1 --2- 172.21.15.154:0/733380799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f11940c9150 0x7f11940aef90 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:44.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.001+0000 7f11994f2640 1 --2- 172.21.15.154:0/733380799 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f11940c8d70 0x7f1194144990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:44.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.001+0000 7f11994f2640 1 --2- 172.21.15.154:0/733380799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1194071290 0x7f11941527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:44.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.001+0000 7f11994f2640 1 -- 172.21.15.154:0/733380799 >> 172.21.15.154:0/733380799 conn(0x7f119406c010 msgr2=0x7f11940726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:44.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.001+0000 7f11994f2640 1 -- 172.21.15.154:0/733380799 shutdown_connections 2023-09-23T03:27:44.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.001+0000 7f11994f2640 1 -- 172.21.15.154:0/733380799 wait complete. 2023-09-23T03:27:44.038 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-09-23T03:27:44.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.221+0000 7fea60cda640 1 Processor -- start 2023-09-23T03:27:44.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.222+0000 7fea60cda640 1 -- start start 2023-09-23T03:27:44.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.223+0000 7fea60cda640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea5c158480 0x7fea5c0cade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:44.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.223+0000 7fea60cda640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fea5c15ba50 0x7fea5c06fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:44.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.223+0000 7fea60cda640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea5c0cb6c0 con 0x7fea5c15eef0 2023-09-23T03:27:44.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.223+0000 7fea60cda640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea5c15ad80 con 0x7fea5c15ba50 2023-09-23T03:27:44.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.223+0000 7fea60cda640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea5c15af70 con 0x7fea5c158480 2023-09-23T03:27:44.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.223+0000 7fea5bfff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fea5c15eef0 0x7fea5c06bf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:47878/0 (socket says 172.21.15.154:47878) 2023-09-23T03:27:44.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.223+0000 7fea5bfff640 1 -- 172.21.15.154:0/1660511127 learned_addr learned my addr 172.21.15.154:0/1660511127 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:27:44.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.224+0000 7fea5b7fe640 1 --2- 172.21.15.154:0/1660511127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea5c158480 0x7fea5c0cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:44.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.224+0000 7fea5affd640 1 --2- 172.21.15.154:0/1660511127 >> v2:172.21.15.190:3300/0 conn(0x7fea5c15ba50 0x7fea5c06fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:44.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.224+0000 7fea5a7fc640 1 -- 172.21.15.154:0/1660511127 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1872701974 0 0) 0x7fea5c0cb6c0 con 0x7fea5c15eef0 2023-09-23T03:27:44.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.224+0000 7fea5a7fc640 1 -- 172.21.15.154:0/1660511127 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea44003520 con 0x7fea5c15eef0 2023-09-23T03:27:44.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.224+0000 7fea5a7fc640 1 -- 172.21.15.154:0/1660511127 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2624402116 0 0) 0x7fea44003520 con 0x7fea5c15eef0 2023-09-23T03:27:44.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.224+0000 7fea5a7fc640 1 -- 172.21.15.154:0/1660511127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea5c158480 msgr2=0x7fea5c0cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:44.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.224+0000 7fea5a7fc640 1 --2- 172.21.15.154:0/1660511127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea5c158480 0x7fea5c0cade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:44.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.224+0000 7fea5a7fc640 1 -- 172.21.15.154:0/1660511127 >> v2:172.21.15.190:3300/0 conn(0x7fea5c15ba50 msgr2=0x7fea5c06fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:44.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.224+0000 7fea5a7fc640 1 --2- 172.21.15.154:0/1660511127 >> v2:172.21.15.190:3300/0 conn(0x7fea5c15ba50 0x7fea5c06fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:44.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.224+0000 7fea5a7fc640 1 -- 172.21.15.154:0/1660511127 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea5c15b250 con 0x7fea5c15eef0 2023-09-23T03:27:44.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.224+0000 7fea5a7fc640 1 -- 172.21.15.154:0/1660511127 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fea540019f0 con 0x7fea5c15eef0 2023-09-23T03:27:44.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.224+0000 7fea5a7fc640 1 -- 172.21.15.154:0/1660511127 >> v1:172.21.15.154:6789/0 conn(0x7fea5c15eef0 legacy=0x7fea5c06bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:44.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.224+0000 7fea5a7fc640 1 --2- 172.21.15.154:0/1660511127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea44003970 0x7fea44003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:44.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.225+0000 7fea5a7fc640 1 --2- 172.21.15.154:0/1660511127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea440047c0 0x7fea44004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:44.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.225+0000 7fea5a7fc640 1 --2- 172.21.15.154:0/1660511127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea5c158480 0x7fea44005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:44.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.225+0000 7fea5a7fc640 1 -- 172.21.15.154:0/1660511127 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5c15af70 con 0x7fea44003970 2023-09-23T03:27:44.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.225+0000 7fea5a7fc640 1 -- 172.21.15.154:0/1660511127 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4400b980 con 0x7fea440047c0 2023-09-23T03:27:44.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.225+0000 7fea5a7fc640 1 -- 172.21.15.154:0/1660511127 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea4400bac0 con 0x7fea5c158480 2023-09-23T03:27:44.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.225+0000 7fea5b7fe640 1 --2- 172.21.15.154:0/1660511127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea44003970 0x7fea44003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:44.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.225+0000 7fea5b7fe640 1 -- 172.21.15.154:0/1660511127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea5c158480 msgr2=0x7fea44005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:44.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.225+0000 7fea5bfff640 1 --2- 172.21.15.154:0/1660511127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea5c158480 0x7fea44005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:44.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.225+0000 7fea5affd640 1 --2- 172.21.15.154:0/1660511127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea440047c0 0x7fea44004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:44.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.225+0000 7fea5b7fe640 1 --2- 172.21.15.154:0/1660511127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea5c158480 0x7fea44005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:44.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.225+0000 7fea5b7fe640 1 -- 172.21.15.154:0/1660511127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea440047c0 msgr2=0x7fea44004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:44.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.225+0000 7fea5b7fe640 1 --2- 172.21.15.154:0/1660511127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea440047c0 0x7fea44004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:44.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.225+0000 7fea5b7fe640 1 -- 172.21.15.154:0/1660511127 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea44003520 con 0x7fea44003970 2023-09-23T03:27:44.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.225+0000 7fea5bfff640 1 --2- 172.21.15.154:0/1660511127 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea5c158480 0x7fea44005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:44.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.225+0000 7fea5affd640 1 --2- 172.21.15.154:0/1660511127 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea440047c0 0x7fea44004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:44.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.226+0000 7fea5b7fe640 1 --2- 172.21.15.154:0/1660511127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea44003970 0x7fea44003d90 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fea4c009af0 tx=0x7fea440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a621750d2538d7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:44.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.227+0000 7fea5a7fc640 1 -- 172.21.15.154:0/1660511127 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4c02c6b0 con 0x7fea44003970 2023-09-23T03:27:44.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.227+0000 7fea5a7fc640 1 -- 172.21.15.154:0/1660511127 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea4c02c810 con 0x7fea44003970 2023-09-23T03:27:44.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.227+0000 7fea5a7fc640 1 -- 172.21.15.154:0/1660511127 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4c02cba0 con 0x7fea44003970 2023-09-23T03:27:44.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.227+0000 7fea60cda640 1 -- 172.21.15.154:0/1660511127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea44003970 msgr2=0x7fea44003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:44.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.227+0000 7fea60cda640 1 --2- 172.21.15.154:0/1660511127 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea44003970 0x7fea44003d90 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fea4c009af0 tx=0x7fea440057f0 comp rx=0 tx=0).stop 2023-09-23T03:27:44.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.227+0000 7fea5bfff640 1 -- 172.21.15.154:0/1660511127 reap_dead start 2023-09-23T03:27:44.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.227+0000 7fea60cda640 1 -- 172.21.15.154:0/1660511127 shutdown_connections 2023-09-23T03:27:44.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.227+0000 7fea60cda640 1 -- 172.21.15.154:0/1660511127 >> 172.21.15.154:0/1660511127 conn(0x7fea5c0aff20 msgr2=0x7fea5c0b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:44.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.227+0000 7fea60cda640 1 -- 172.21.15.154:0/1660511127 shutdown_connections 2023-09-23T03:27:44.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.227+0000 7fea60cda640 1 -- 172.21.15.154:0/1660511127 wait complete. 2023-09-23T03:27:44.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.229+0000 7fea60cda640 1 Processor -- start 2023-09-23T03:27:44.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.229+0000 7fea60cda640 1 -- start start 2023-09-23T03:27:44.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.230+0000 7fea60cda640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea44003970 0x7fea5c1570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:44.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.230+0000 7fea60cda640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea5c0ae810 0x7fea5c0aec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:44.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.230+0000 7fea60cda640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea5c0c5660 0x7fea5c0c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:44.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.230+0000 7fea60cda640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5c0af1d0 con 0x7fea5c0c5660 2023-09-23T03:27:44.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.230+0000 7fea60cda640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5c1575f0 con 0x7fea44003970 2023-09-23T03:27:44.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.230+0000 7fea60cda640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea5c0c6390 con 0x7fea5c0ae810 2023-09-23T03:27:44.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.231+0000 7fea5bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea5c0c5660 0x7fea5c0c5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:44.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.231+0000 7fea5bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea5c0c5660 0x7fea5c0c5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50338/0 (socket says 172.21.15.154:50338) 2023-09-23T03:27:44.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.231+0000 7fea5bfff640 1 -- 172.21.15.154:0/1270538960 learned_addr learned my addr 172.21.15.154:0/1270538960 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:44.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.231+0000 7fea5b7fe640 1 --2- 172.21.15.154:0/1270538960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea44003970 0x7fea5c1570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:44.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.231+0000 7fea5affd640 1 --2- 172.21.15.154:0/1270538960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea5c0ae810 0x7fea5c0aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:44.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.231+0000 7fea5bfff640 1 -- 172.21.15.154:0/1270538960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea5c0ae810 msgr2=0x7fea5c0aec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:44.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.231+0000 7fea5bfff640 1 --2- 172.21.15.154:0/1270538960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea5c0ae810 0x7fea5c0aec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:44.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.231+0000 7fea5bfff640 1 -- 172.21.15.154:0/1270538960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea44003970 msgr2=0x7fea5c1570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:44.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.231+0000 7fea5bfff640 1 --2- 172.21.15.154:0/1270538960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea44003970 0x7fea5c1570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:44.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.231+0000 7fea5bfff640 1 -- 172.21.15.154:0/1270538960 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea5c0c6670 con 0x7fea5c0c5660 2023-09-23T03:27:44.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.231+0000 7fea5b7fe640 1 --2- 172.21.15.154:0/1270538960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea44003970 0x7fea5c1570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:44.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.231+0000 7fea5bfff640 1 --2- 172.21.15.154:0/1270538960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea5c0c5660 0x7fea5c0c5ae0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fea5c15f2d0 tx=0x7fea5400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:44.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.232+0000 7fea43fff640 1 -- 172.21.15.154:0/1270538960 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea54004d60 con 0x7fea5c0c5660 2023-09-23T03:27:44.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.232+0000 7fea43fff640 1 -- 172.21.15.154:0/1270538960 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea5400bb80 con 0x7fea5c0c5660 2023-09-23T03:27:44.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.232+0000 7fea43fff640 1 -- 172.21.15.154:0/1270538960 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea54007070 con 0x7fea5c0c5660 2023-09-23T03:27:44.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.232+0000 7fea60cda640 1 -- 172.21.15.154:0/1270538960 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea5c0c6950 con 0x7fea5c0c5660 2023-09-23T03:27:44.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.232+0000 7fea60cda640 1 -- 172.21.15.154:0/1270538960 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea5c0c6f10 con 0x7fea5c0c5660 2023-09-23T03:27:44.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.233+0000 7fea60cda640 1 -- 172.21.15.154:0/1270538960 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea5c04e920 con 0x7fea5c0c5660 2023-09-23T03:27:44.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.234+0000 7fea43fff640 1 -- 172.21.15.154:0/1270538960 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fea540071d0 con 0x7fea5c0c5660 2023-09-23T03:27:44.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.235+0000 7fea43fff640 1 --2- 172.21.15.154:0/1270538960 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fea4806af30 0x7fea4806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:44.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.235+0000 7fea43fff640 1 -- 172.21.15.154:0/1270538960 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5929+0+0 (secure 0 0 0) 0x7fea54089bc0 con 0x7fea5c0c5660 2023-09-23T03:27:44.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.235+0000 7fea5b7fe640 1 --2- 172.21.15.154:0/1270538960 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fea4806af30 0x7fea4806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:44.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.236+0000 7fea5b7fe640 1 --2- 172.21.15.154:0/1270538960 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fea4806af30 0x7fea4806d3f0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fea4c036000 tx=0x7fea4c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:44.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.240+0000 7fea43fff640 1 -- 172.21.15.154:0/1270538960 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fea54054a80 con 0x7fea5c0c5660 2023-09-23T03:27:44.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:44.468+0000 7fea60cda640 1 -- 172.21.15.154:0/1270538960 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fea5c0cc870 con 0x7fea5c0c5660 2023-09-23T03:27:44.501 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:27:44.510 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:27:45.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.001+0000 7fea43fff640 1 -- 172.21.15.154:0/1270538960 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v104) v1 ==== 174+0+0 (secure 0 0 0) 0x7fea540580a0 con 0x7fea5c0c5660 2023-09-23T03:27:45.002 INFO:teuthology.orchestra.run.smithi154.stderr:new fs with metadata pool 17 and data pool 18 2023-09-23T03:27:45.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.005+0000 7fea60cda640 1 -- 172.21.15.154:0/1270538960 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fea4806af30 msgr2=0x7fea4806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:45.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.005+0000 7fea60cda640 1 --2- 172.21.15.154:0/1270538960 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fea4806af30 0x7fea4806d3f0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fea4c036000 tx=0x7fea4c030000 comp rx=0 tx=0).stop 2023-09-23T03:27:45.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.005+0000 7fea60cda640 1 -- 172.21.15.154:0/1270538960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea5c0c5660 msgr2=0x7fea5c0c5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:45.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.005+0000 7fea60cda640 1 --2- 172.21.15.154:0/1270538960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea5c0c5660 0x7fea5c0c5ae0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fea5c15f2d0 tx=0x7fea5400b4a0 comp rx=0 tx=0).stop 2023-09-23T03:27:45.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.006+0000 7fea60cda640 1 -- 172.21.15.154:0/1270538960 shutdown_connections 2023-09-23T03:27:45.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.006+0000 7fea60cda640 1 --2- 172.21.15.154:0/1270538960 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fea4806af30 0x7fea4806d3f0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:45.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.006+0000 7fea60cda640 1 --2- 172.21.15.154:0/1270538960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fea5c0c5660 0x7fea5c0c5ae0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:45.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.006+0000 7fea60cda640 1 --2- 172.21.15.154:0/1270538960 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fea5c0ae810 0x7fea5c0aec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:45.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.006+0000 7fea60cda640 1 --2- 172.21.15.154:0/1270538960 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fea44003970 0x7fea5c1570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:45.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.006+0000 7fea60cda640 1 -- 172.21.15.154:0/1270538960 >> 172.21.15.154:0/1270538960 conn(0x7fea5c0aff20 msgr2=0x7fea5c0af5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:45.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.014+0000 7fea60cda640 1 -- 172.21.15.154:0/1270538960 shutdown_connections 2023-09-23T03:27:45.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.015+0000 7fea60cda640 1 -- 172.21.15.154:0/1270538960 wait complete. 2023-09-23T03:27:45.052 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:27:45.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.235+0000 7f53ce5b6640 1 Processor -- start 2023-09-23T03:27:45.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.235+0000 7f53ce5b6640 1 -- start start 2023-09-23T03:27:45.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.236+0000 7f53ce5b6640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f53c80ada30 0x7f53c80ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:45.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.236+0000 7f53ce5b6640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f53c80b2fe0 0x7f53c80cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:45.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.236+0000 7f53ce5b6640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53c80cef60 0x7f53c80b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:45.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.236+0000 7f53ce5b6640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c80cf410 con 0x7f53c80b2fe0 2023-09-23T03:27:45.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.236+0000 7f53ce5b6640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c80ade10 con 0x7f53c80ada30 2023-09-23T03:27:45.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.236+0000 7f53ce5b6640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c806ceb0 con 0x7f53c80cef60 2023-09-23T03:27:45.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.236+0000 7f53c77fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f53c80b2fe0 0x7f53c80cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:45.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.237+0000 7f53c77fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f53c80b2fe0 0x7f53c80cea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50340/0 (socket says 172.21.15.154:50340) 2023-09-23T03:27:45.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.237+0000 7f53c77fe640 1 -- 172.21.15.154:0/2519966580 learned_addr learned my addr 172.21.15.154:0/2519966580 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:45.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.237+0000 7f53c7fff640 1 --2- 172.21.15.154:0/2519966580 >> v2:172.21.15.190:3300/0 conn(0x7f53c80ada30 0x7f53c80ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:45.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.237+0000 7f53ccb53640 1 --2- 172.21.15.154:0/2519966580 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53c80cef60 0x7f53c80b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:45.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.237+0000 7f53c77fe640 1 -- 172.21.15.154:0/2519966580 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53c80cef60 msgr2=0x7f53c80b18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:45.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.237+0000 7f53c77fe640 1 --2- 172.21.15.154:0/2519966580 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53c80cef60 0x7f53c80b18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:45.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.237+0000 7f53c77fe640 1 -- 172.21.15.154:0/2519966580 >> v2:172.21.15.190:3300/0 conn(0x7f53c80ada30 msgr2=0x7f53c80ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:45.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.237+0000 7f53c77fe640 1 --2- 172.21.15.154:0/2519966580 >> v2:172.21.15.190:3300/0 conn(0x7f53c80ada30 0x7f53c80ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:45.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.237+0000 7f53c77fe640 1 -- 172.21.15.154:0/2519966580 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c8159c90 con 0x7f53c80b2fe0 2023-09-23T03:27:45.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.237+0000 7f53ccb53640 1 --2- 172.21.15.154:0/2519966580 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53c80cef60 0x7f53c80b18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:45.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.238+0000 7f53c77fe640 1 --2- 172.21.15.154:0/2519966580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53c80b2fe0 0x7f53c80cea20 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f53c806d990 tx=0x7f53b002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34d6fbfdc9188aec server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:45.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.239+0000 7f53c6ffd640 1 -- 172.21.15.154:0/2519966580 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b0031640 con 0x7f53c80b2fe0 2023-09-23T03:27:45.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.239+0000 7f53c6ffd640 1 -- 172.21.15.154:0/2519966580 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b00317a0 con 0x7f53c80b2fe0 2023-09-23T03:27:45.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.239+0000 7f53c6ffd640 1 -- 172.21.15.154:0/2519966580 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b0031b30 con 0x7f53c80b2fe0 2023-09-23T03:27:45.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.239+0000 7f53ce5b6640 1 -- 172.21.15.154:0/2519966580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53c80b2fe0 msgr2=0x7f53c80cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:45.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.239+0000 7f53ce5b6640 1 --2- 172.21.15.154:0/2519966580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53c80b2fe0 0x7f53c80cea20 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f53c806d990 tx=0x7f53b002b8f0 comp rx=0 tx=0).stop 2023-09-23T03:27:45.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.239+0000 7f53ce5b6640 1 -- 172.21.15.154:0/2519966580 shutdown_connections 2023-09-23T03:27:45.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.239+0000 7f53ce5b6640 1 --2- 172.21.15.154:0/2519966580 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53c80cef60 0x7f53c80b18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:45.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.239+0000 7f53ce5b6640 1 --2- 172.21.15.154:0/2519966580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53c80b2fe0 0x7f53c80cea20 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:45.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.239+0000 7f53ce5b6640 1 --2- 172.21.15.154:0/2519966580 >> v2:172.21.15.190:3300/0 conn(0x7f53c80ada30 0x7f53c80ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:45.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.239+0000 7f53ce5b6640 1 -- 172.21.15.154:0/2519966580 >> 172.21.15.154:0/2519966580 conn(0x7f53c806a0f0 msgr2=0x7f53c806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:45.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.239+0000 7f53ce5b6640 1 -- 172.21.15.154:0/2519966580 shutdown_connections 2023-09-23T03:27:45.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.239+0000 7f53ce5b6640 1 -- 172.21.15.154:0/2519966580 wait complete. 2023-09-23T03:27:45.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.241+0000 7f53ce5b6640 1 Processor -- start 2023-09-23T03:27:45.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.241+0000 7f53ce5b6640 1 -- start start 2023-09-23T03:27:45.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.242+0000 7f53ce5b6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53c80ada30 0x7f53c815c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:45.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.242+0000 7f53ce5b6640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53c80b2fe0 0x7f53c80979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:45.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.242+0000 7f53ce5b6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53c80cef60 0x7f53c815a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:45.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.242+0000 7f53ce5b6640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c815ca40 con 0x7f53c80ada30 2023-09-23T03:27:45.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.242+0000 7f53ce5b6640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c8097f20 con 0x7f53c80cef60 2023-09-23T03:27:45.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.242+0000 7f53ce5b6640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c80ad5b0 con 0x7f53c80b2fe0 2023-09-23T03:27:45.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.243+0000 7f53c7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53c80ada30 0x7f53c815c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:45.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.243+0000 7f53c7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53c80ada30 0x7f53c815c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50356/0 (socket says 172.21.15.154:50356) 2023-09-23T03:27:45.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.243+0000 7f53c7fff640 1 -- 172.21.15.154:0/2770162489 learned_addr learned my addr 172.21.15.154:0/2770162489 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:45.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.243+0000 7f53c77fe640 1 --2- 172.21.15.154:0/2770162489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53c80b2fe0 0x7f53c80979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:45.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.243+0000 7f53ccb53640 1 --2- 172.21.15.154:0/2770162489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53c80cef60 0x7f53c815a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:45.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.243+0000 7f53c7fff640 1 -- 172.21.15.154:0/2770162489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53c80b2fe0 msgr2=0x7f53c80979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:45.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.243+0000 7f53c7fff640 1 --2- 172.21.15.154:0/2770162489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53c80b2fe0 0x7f53c80979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:45.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.243+0000 7f53c7fff640 1 -- 172.21.15.154:0/2770162489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53c80cef60 msgr2=0x7f53c815a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:45.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.243+0000 7f53c7fff640 1 --2- 172.21.15.154:0/2770162489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53c80cef60 0x7f53c815a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:45.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.243+0000 7f53c7fff640 1 -- 172.21.15.154:0/2770162489 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b00094d0 con 0x7f53c80ada30 2023-09-23T03:27:45.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.243+0000 7f53ccb53640 1 --2- 172.21.15.154:0/2770162489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53c80cef60 0x7f53c815a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:45.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.244+0000 7f53c7fff640 1 --2- 172.21.15.154:0/2770162489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53c80ada30 0x7f53c815c500 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f53b8007e10 tx=0x7f53b800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:45.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.244+0000 7f53c4ff9640 1 -- 172.21.15.154:0/2770162489 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b8014620 con 0x7f53c80ada30 2023-09-23T03:27:45.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.244+0000 7f53c4ff9640 1 -- 172.21.15.154:0/2770162489 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b800ee90 con 0x7f53c80ada30 2023-09-23T03:27:45.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.244+0000 7f53ce5b6640 1 -- 172.21.15.154:0/2770162489 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c815cc50 con 0x7f53c80ada30 2023-09-23T03:27:45.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.244+0000 7f53c4ff9640 1 -- 172.21.15.154:0/2770162489 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b80149c0 con 0x7f53c80ada30 2023-09-23T03:27:45.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.244+0000 7f53ce5b6640 1 -- 172.21.15.154:0/2770162489 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c80c2eb0 con 0x7f53c80ada30 2023-09-23T03:27:45.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.245+0000 7f53ce5b6640 1 -- 172.21.15.154:0/2770162489 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53c8073eb0 con 0x7f53c80ada30 2023-09-23T03:27:45.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.246+0000 7f53c4ff9640 1 -- 172.21.15.154:0/2770162489 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f53b8014b20 con 0x7f53c80ada30 2023-09-23T03:27:45.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.247+0000 7f53c4ff9640 1 --2- 172.21.15.154:0/2770162489 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f53a806af60 0x7f53a806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:45.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.247+0000 7f53c4ff9640 1 -- 172.21.15.154:0/2770162489 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f53b808a240 con 0x7f53c80ada30 2023-09-23T03:27:45.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.247+0000 7f53c77fe640 1 --2- 172.21.15.154:0/2770162489 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f53a806af60 0x7f53a806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:45.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.248+0000 7f53c77fe640 1 --2- 172.21.15.154:0/2770162489 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f53a806af60 0x7f53a806d420 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f53b0036000 tx=0x7f53b003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:45.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.252+0000 7f53c4ff9640 1 -- 172.21.15.154:0/2770162489 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f53b80550a0 con 0x7f53c80ada30 2023-09-23T03:27:45.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.443+0000 7f53ce5b6640 1 -- 172.21.15.154:0/2770162489 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53c8159950 con 0x7f53c80ada30 2023-09-23T03:27:45.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.445+0000 7f53c4ff9640 1 -- 172.21.15.154:0/2770162489 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+16833 (secure 0 0 0) 0x7f53b80586c0 con 0x7f53c80ada30 2023-09-23T03:27:45.445 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:45.446 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":125,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:27:44.469883+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:27:42.207617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-09-23T03:27:43.473867+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:27:45.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.447+0000 7f53ce5b6640 1 -- 172.21.15.154:0/2770162489 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f53a806af60 msgr2=0x7f53a806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:45.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.447+0000 7f53ce5b6640 1 --2- 172.21.15.154:0/2770162489 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f53a806af60 0x7f53a806d420 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f53b0036000 tx=0x7f53b003c040 comp rx=0 tx=0).stop 2023-09-23T03:27:45.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.447+0000 7f53ce5b6640 1 -- 172.21.15.154:0/2770162489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53c80ada30 msgr2=0x7f53c815c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:45.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.447+0000 7f53ce5b6640 1 --2- 172.21.15.154:0/2770162489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53c80ada30 0x7f53c815c500 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f53b8007e10 tx=0x7f53b800e4b0 comp rx=0 tx=0).stop 2023-09-23T03:27:45.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.447+0000 7f53ce5b6640 1 -- 172.21.15.154:0/2770162489 shutdown_connections 2023-09-23T03:27:45.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.447+0000 7f53ce5b6640 1 --2- 172.21.15.154:0/2770162489 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f53a806af60 0x7f53a806d420 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:45.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.447+0000 7f53ce5b6640 1 --2- 172.21.15.154:0/2770162489 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f53c80cef60 0x7f53c815a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:45.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.447+0000 7f53ce5b6640 1 --2- 172.21.15.154:0/2770162489 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f53c80b2fe0 0x7f53c80979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:45.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.447+0000 7f53ce5b6640 1 --2- 172.21.15.154:0/2770162489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f53c80ada30 0x7f53c815c500 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:45.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.448+0000 7f53ce5b6640 1 -- 172.21.15.154:0/2770162489 >> 172.21.15.154:0/2770162489 conn(0x7f53c806a0f0 msgr2=0x7f53c8072880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:45.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.448+0000 7f53ce5b6640 1 -- 172.21.15.154:0/2770162489 shutdown_connections 2023-09-23T03:27:45.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.448+0000 7f53ce5b6640 1 -- 172.21.15.154:0/2770162489 wait complete. 2023-09-23T03:27:45.466 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:27:45.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.681+0000 7f2088075640 1 Processor -- start 2023-09-23T03:27:45.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.681+0000 7f2088075640 1 -- start start 2023-09-23T03:27:45.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.683+0000 7f2088075640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20800b1b20 0x7f20800ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:45.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.683+0000 7f2088075640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f20800cee70 0x7f20800ab140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:45.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.683+0000 7f2088075640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f208015ac30 con 0x7f20800acce0 2023-09-23T03:27:45.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.683+0000 7f2088075640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20800b2e90 con 0x7f20800cee70 2023-09-23T03:27:45.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.683+0000 7f2088075640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20800cf320 con 0x7f20800b1b20 2023-09-23T03:27:45.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.683+0000 7f2086612640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f20800cee70 0x7f20800ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:45.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.683+0000 7f2086612640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f20800cee70 0x7f20800ab140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50372/0 (socket says 172.21.15.154:50372) 2023-09-23T03:27:45.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.683+0000 7f2086612640 1 -- 172.21.15.154:0/1025766618 learned_addr learned my addr 172.21.15.154:0/1025766618 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:45.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.683+0000 7f2085610640 1 --2- 172.21.15.154:0/1025766618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20800b1b20 0x7f20800ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:45.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.684+0000 7f2086612640 1 -- 172.21.15.154:0/1025766618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20800b1b20 msgr2=0x7f20800ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:45.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.684+0000 7f2086612640 1 --2- 172.21.15.154:0/1025766618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20800b1b20 0x7f20800ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:45.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.684+0000 7f2086612640 1 -- 172.21.15.154:0/1025766618 >> v1:172.21.15.154:6789/0 conn(0x7f20800acce0 legacy=0x7f208015ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:45.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.684+0000 7f2086612640 1 -- 172.21.15.154:0/1025766618 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20800b1f70 con 0x7f20800cee70 2023-09-23T03:27:45.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.685+0000 7f2086612640 1 --2- 172.21.15.154:0/1025766618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20800cee70 0x7f20800ab140 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f2078009e50 tx=0x7f207802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ada90159f006 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:45.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.685+0000 7f2084e0f640 1 -- 172.21.15.154:0/1025766618 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207802ca80 con 0x7f20800cee70 2023-09-23T03:27:45.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.685+0000 7f2084e0f640 1 -- 172.21.15.154:0/1025766618 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f207802cbe0 con 0x7f20800cee70 2023-09-23T03:27:45.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.685+0000 7f2084e0f640 1 -- 172.21.15.154:0/1025766618 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20780343f0 con 0x7f20800cee70 2023-09-23T03:27:45.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.686+0000 7f2088075640 1 -- 172.21.15.154:0/1025766618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20800cee70 msgr2=0x7f20800ab140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:45.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.686+0000 7f2088075640 1 --2- 172.21.15.154:0/1025766618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20800cee70 0x7f20800ab140 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f2078009e50 tx=0x7f207802bee0 comp rx=0 tx=0).stop 2023-09-23T03:27:45.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.686+0000 7f2088075640 1 -- 172.21.15.154:0/1025766618 shutdown_connections 2023-09-23T03:27:45.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.686+0000 7f2088075640 1 --2- 172.21.15.154:0/1025766618 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20800cee70 0x7f20800ab140 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:45.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.686+0000 7f2088075640 1 --2- 172.21.15.154:0/1025766618 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20800b1b20 0x7f20800ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:45.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.686+0000 7f2088075640 1 -- 172.21.15.154:0/1025766618 >> 172.21.15.154:0/1025766618 conn(0x7f208006a0f0 msgr2=0x7f208006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:45.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.686+0000 7f2088075640 1 -- 172.21.15.154:0/1025766618 shutdown_connections 2023-09-23T03:27:45.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.686+0000 7f2088075640 1 -- 172.21.15.154:0/1025766618 wait complete. 2023-09-23T03:27:45.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.687+0000 7f2088075640 1 Processor -- start 2023-09-23T03:27:45.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.688+0000 7f2088075640 1 -- start start 2023-09-23T03:27:45.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.689+0000 7f2088075640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20800acce0 0x7f20800cd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:45.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.689+0000 7f2088075640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20800b1b20 0x7f20800cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:45.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.689+0000 7f2088075640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20800cee70 0x7f2080150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:45.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.689+0000 7f2088075640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20800c1fa0 con 0x7f20800b1b20 2023-09-23T03:27:45.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.689+0000 7f2088075640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2080152d80 con 0x7f20800cee70 2023-09-23T03:27:45.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.689+0000 7f2088075640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2080153210 con 0x7f20800acce0 2023-09-23T03:27:45.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.689+0000 7f2085610640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20800b1b20 0x7f20800cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:45.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.689+0000 7f2085610640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20800b1b20 0x7f20800cd9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50374/0 (socket says 172.21.15.154:50374) 2023-09-23T03:27:45.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.689+0000 7f2085610640 1 -- 172.21.15.154:0/2866345959 learned_addr learned my addr 172.21.15.154:0/2866345959 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:45.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.689+0000 7f2085e11640 1 --2- 172.21.15.154:0/2866345959 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20800acce0 0x7f20800cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:45.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.689+0000 7f2085610640 1 -- 172.21.15.154:0/2866345959 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20800acce0 msgr2=0x7f20800cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:45.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.689+0000 7f2086612640 1 --2- 172.21.15.154:0/2866345959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20800cee70 0x7f2080150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:45.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.689+0000 7f2085610640 1 --2- 172.21.15.154:0/2866345959 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20800acce0 0x7f20800cd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:45.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.689+0000 7f2085610640 1 -- 172.21.15.154:0/2866345959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20800cee70 msgr2=0x7f2080150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:45.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.689+0000 7f2085610640 1 --2- 172.21.15.154:0/2866345959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20800cee70 0x7f2080150d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:45.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.689+0000 7f2085610640 1 -- 172.21.15.154:0/2866345959 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2078009b30 con 0x7f20800b1b20 2023-09-23T03:27:45.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.690+0000 7f2085610640 1 --2- 172.21.15.154:0/2866345959 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20800b1b20 0x7f20800cd9d0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f207000d7d0 tx=0x7f207000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:45.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.690+0000 7f20767fc640 1 -- 172.21.15.154:0/2866345959 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207000b6c0 con 0x7f20800b1b20 2023-09-23T03:27:45.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.690+0000 7f20767fc640 1 -- 172.21.15.154:0/2866345959 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f207000b820 con 0x7f20800b1b20 2023-09-23T03:27:45.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.690+0000 7f20767fc640 1 -- 172.21.15.154:0/2866345959 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207000bbb0 con 0x7f20800b1b20 2023-09-23T03:27:45.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.690+0000 7f2088075640 1 -- 172.21.15.154:0/2866345959 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20800c2180 con 0x7f20800b1b20 2023-09-23T03:27:45.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.690+0000 7f2088075640 1 -- 172.21.15.154:0/2866345959 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20800c2460 con 0x7f20800b1b20 2023-09-23T03:27:45.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.691+0000 7f2088075640 1 -- 172.21.15.154:0/2866345959 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2080152ef0 con 0x7f20800b1b20 2023-09-23T03:27:45.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.692+0000 7f20767fc640 1 -- 172.21.15.154:0/2866345959 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f207000bd10 con 0x7f20800b1b20 2023-09-23T03:27:45.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.693+0000 7f20767fc640 1 --2- 172.21.15.154:0/2866345959 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f206406af30 0x7f206406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:45.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.693+0000 7f20767fc640 1 -- 172.21.15.154:0/2866345959 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2070089ec0 con 0x7f20800b1b20 2023-09-23T03:27:45.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.693+0000 7f2085e11640 1 --2- 172.21.15.154:0/2866345959 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f206406af30 0x7f206406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:45.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.694+0000 7f2085e11640 1 --2- 172.21.15.154:0/2866345959 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f206406af30 0x7f206406d3f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f207c0036f0 tx=0x7f207c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:45.697 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.697+0000 7f20767fc640 1 -- 172.21.15.154:0/2866345959 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f2070054d20 con 0x7f20800b1b20 2023-09-23T03:27:45.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.888+0000 7f2088075640 1 -- 172.21.15.154:0/2866345959 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2080073eb0 con 0x7f20800b1b20 2023-09-23T03:27:45.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.890+0000 7f20767fc640 1 -- 172.21.15.154:0/2866345959 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+16833 (secure 0 0 0) 0x7f2070058340 con 0x7f20800b1b20 2023-09-23T03:27:45.891 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:45.891 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":125,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:27:44.469883+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:27:42.207617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-09-23T03:27:43.473867+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:27:45.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.893+0000 7f2088075640 1 -- 172.21.15.154:0/2866345959 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f206406af30 msgr2=0x7f206406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:45.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.893+0000 7f2088075640 1 --2- 172.21.15.154:0/2866345959 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f206406af30 0x7f206406d3f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f207c0036f0 tx=0x7f207c009040 comp rx=0 tx=0).stop 2023-09-23T03:27:45.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.893+0000 7f2088075640 1 -- 172.21.15.154:0/2866345959 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20800b1b20 msgr2=0x7f20800cd9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:45.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.893+0000 7f2088075640 1 --2- 172.21.15.154:0/2866345959 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20800b1b20 0x7f20800cd9d0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f207000d7d0 tx=0x7f207000dc90 comp rx=0 tx=0).stop 2023-09-23T03:27:45.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.893+0000 7f2088075640 1 -- 172.21.15.154:0/2866345959 shutdown_connections 2023-09-23T03:27:45.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.893+0000 7f2088075640 1 --2- 172.21.15.154:0/2866345959 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f206406af30 0x7f206406d3f0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:45.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.893+0000 7f2088075640 1 --2- 172.21.15.154:0/2866345959 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f20800cee70 0x7f2080150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:45.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.893+0000 7f2088075640 1 --2- 172.21.15.154:0/2866345959 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20800b1b20 0x7f20800cd9d0 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:45.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.893+0000 7f2088075640 1 --2- 172.21.15.154:0/2866345959 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f20800acce0 0x7f20800cd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:45.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.893+0000 7f2088075640 1 -- 172.21.15.154:0/2866345959 >> 172.21.15.154:0/2866345959 conn(0x7f208006a0f0 msgr2=0x7f20800cb330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:45.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.893+0000 7f2088075640 1 -- 172.21.15.154:0/2866345959 shutdown_connections 2023-09-23T03:27:45.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:45.893+0000 7f2088075640 1 -- 172.21.15.154:0/2866345959 wait complete. 2023-09-23T03:27:45.912 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-09-23T03:27:46.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.137+0000 7f8f7eda9640 1 Processor -- start 2023-09-23T03:27:46.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.137+0000 7f8f7eda9640 1 -- start start 2023-09-23T03:27:46.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.139+0000 7f8f7eda9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f8f7806a490 0x7f8f78071080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:46.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.139+0000 7f8f7eda9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8f780cb6b0 0x7f8f780cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:46.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.139+0000 7f8f7eda9640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f7815e090 con 0x7f8f780cbb30 2023-09-23T03:27:46.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.139+0000 7f8f7eda9640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f7815d240 con 0x7f8f780cb6b0 2023-09-23T03:27:46.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.139+0000 7f8f7eda9640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f7815d480 con 0x7f8f7806a490 2023-09-23T03:27:46.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.139+0000 7f8f77fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8f780cb6b0 0x7f8f780cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:46.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.139+0000 7f8f77fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f8f780cb6b0 0x7f8f780cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50386/0 (socket says 172.21.15.154:50386) 2023-09-23T03:27:46.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.139+0000 7f8f77fff640 1 -- 172.21.15.154:0/1842793103 learned_addr learned my addr 172.21.15.154:0/1842793103 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:46.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.140+0000 7f8f7cb45640 1 --2- 172.21.15.154:0/1842793103 >> v2:172.21.15.190:3300/0 conn(0x7f8f7806a490 0x7f8f78071080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:46.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.140+0000 7f8f77fff640 1 -- 172.21.15.154:0/1842793103 >> v2:172.21.15.190:3300/0 conn(0x7f8f7806a490 msgr2=0x7f8f78071080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:46.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.140+0000 7f8f77fff640 1 --2- 172.21.15.154:0/1842793103 >> v2:172.21.15.190:3300/0 conn(0x7f8f7806a490 0x7f8f78071080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:46.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.140+0000 7f8f77fff640 1 -- 172.21.15.154:0/1842793103 >> v1:172.21.15.190:6789/0 conn(0x7f8f780cbb30 legacy=0x7f8f78160950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:46.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.140+0000 7f8f77fff640 1 -- 172.21.15.154:0/1842793103 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f7815d760 con 0x7f8f780cb6b0 2023-09-23T03:27:46.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.141+0000 7f8f77fff640 1 --2- 172.21.15.154:0/1842793103 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f780cb6b0 0x7f8f780cd390 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f8f78070cf0 tx=0x7f8f6c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a352142b360e8a25 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:46.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.141+0000 7f8f777fe640 1 -- 172.21.15.154:0/1842793103 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c0317e0 con 0x7f8f780cb6b0 2023-09-23T03:27:46.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.141+0000 7f8f777fe640 1 -- 172.21.15.154:0/1842793103 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6c031940 con 0x7f8f780cb6b0 2023-09-23T03:27:46.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.141+0000 7f8f777fe640 1 -- 172.21.15.154:0/1842793103 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c031cd0 con 0x7f8f780cb6b0 2023-09-23T03:27:46.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.142+0000 7f8f7eda9640 1 -- 172.21.15.154:0/1842793103 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f780cb6b0 msgr2=0x7f8f780cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:46.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.142+0000 7f8f7eda9640 1 --2- 172.21.15.154:0/1842793103 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f780cb6b0 0x7f8f780cd390 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f8f78070cf0 tx=0x7f8f6c02b880 comp rx=0 tx=0).stop 2023-09-23T03:27:46.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.142+0000 7f8f7eda9640 1 -- 172.21.15.154:0/1842793103 shutdown_connections 2023-09-23T03:27:46.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.142+0000 7f8f7eda9640 1 --2- 172.21.15.154:0/1842793103 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f780cb6b0 0x7f8f780cd390 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:46.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.142+0000 7f8f7eda9640 1 --2- 172.21.15.154:0/1842793103 >> v2:172.21.15.190:3300/0 conn(0x7f8f7806a490 0x7f8f78071080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:46.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.142+0000 7f8f7eda9640 1 -- 172.21.15.154:0/1842793103 >> 172.21.15.154:0/1842793103 conn(0x7f8f780cc030 msgr2=0x7f8f780cc410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:46.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.142+0000 7f8f7eda9640 1 -- 172.21.15.154:0/1842793103 shutdown_connections 2023-09-23T03:27:46.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.142+0000 7f8f7eda9640 1 -- 172.21.15.154:0/1842793103 wait complete. 2023-09-23T03:27:46.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.144+0000 7f8f7eda9640 1 Processor -- start 2023-09-23T03:27:46.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.144+0000 7f8f7eda9640 1 -- start start 2023-09-23T03:27:46.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.145+0000 7f8f7eda9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f7806a490 0x7f8f780c7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:46.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.145+0000 7f8f7eda9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f780cb6b0 0x7f8f7815bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:46.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.145+0000 7f8f7eda9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f780cbb30 0x7f8f780b2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:46.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.145+0000 7f8f7eda9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f780ac8b0 con 0x7f8f780cbb30 2023-09-23T03:27:46.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.145+0000 7f8f7eda9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f780b31e0 con 0x7f8f7806a490 2023-09-23T03:27:46.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.145+0000 7f8f7eda9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f780c7720 con 0x7f8f780cb6b0 2023-09-23T03:27:46.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.145+0000 7f8f7d346640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f780cbb30 0x7f8f780b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:46.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.145+0000 7f8f7d346640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f780cbb30 0x7f8f780b2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50400/0 (socket says 172.21.15.154:50400) 2023-09-23T03:27:46.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.145+0000 7f8f7d346640 1 -- 172.21.15.154:0/3510518108 learned_addr learned my addr 172.21.15.154:0/3510518108 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:46.147 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.146+0000 7f8f7cb45640 1 --2- 172.21.15.154:0/3510518108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f7806a490 0x7f8f780c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:46.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.146+0000 7f8f77fff640 1 --2- 172.21.15.154:0/3510518108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f780cb6b0 0x7f8f7815bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:46.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.146+0000 7f8f7d346640 1 -- 172.21.15.154:0/3510518108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f780cb6b0 msgr2=0x7f8f7815bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:46.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.146+0000 7f8f7d346640 1 --2- 172.21.15.154:0/3510518108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f780cb6b0 0x7f8f7815bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:46.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.146+0000 7f8f7d346640 1 -- 172.21.15.154:0/3510518108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f7806a490 msgr2=0x7f8f780c7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:46.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.146+0000 7f8f7d346640 1 --2- 172.21.15.154:0/3510518108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f7806a490 0x7f8f780c7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:46.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.146+0000 7f8f7d346640 1 -- 172.21.15.154:0/3510518108 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f6c0094d0 con 0x7f8f780cbb30 2023-09-23T03:27:46.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.146+0000 7f8f7cb45640 1 --2- 172.21.15.154:0/3510518108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f7806a490 0x7f8f780c7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:46.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.146+0000 7f8f77fff640 1 --2- 172.21.15.154:0/3510518108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f780cb6b0 0x7f8f7815bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:46.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.146+0000 7f8f7d346640 1 --2- 172.21.15.154:0/3510518108 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f780cbb30 0x7f8f780b2400 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f8f7000c780 tx=0x7f8f7000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:46.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.146+0000 7f8f757fa640 1 -- 172.21.15.154:0/3510518108 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f70003d80 con 0x7f8f780cbb30 2023-09-23T03:27:46.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.146+0000 7f8f757fa640 1 -- 172.21.15.154:0/3510518108 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f7000ce90 con 0x7f8f780cbb30 2023-09-23T03:27:46.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.147+0000 7f8f7eda9640 1 -- 172.21.15.154:0/3510518108 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f780c7a00 con 0x7f8f780cbb30 2023-09-23T03:27:46.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.147+0000 7f8f757fa640 1 -- 172.21.15.154:0/3510518108 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f700052b0 con 0x7f8f780cbb30 2023-09-23T03:27:46.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.147+0000 7f8f7eda9640 1 -- 172.21.15.154:0/3510518108 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f780c7fc0 con 0x7f8f780cbb30 2023-09-23T03:27:46.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.148+0000 7f8f7eda9640 1 -- 172.21.15.154:0/3510518108 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f780ac9f0 con 0x7f8f780cbb30 2023-09-23T03:27:46.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.149+0000 7f8f757fa640 1 -- 172.21.15.154:0/3510518108 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f8f70005410 con 0x7f8f780cbb30 2023-09-23T03:27:46.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.150+0000 7f8f757fa640 1 --2- 172.21.15.154:0/3510518108 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8f6406af30 0x7f8f6406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:46.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.150+0000 7f8f757fa640 1 -- 172.21.15.154:0/3510518108 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f8f700891d0 con 0x7f8f780cbb30 2023-09-23T03:27:46.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.150+0000 7f8f7cb45640 1 --2- 172.21.15.154:0/3510518108 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8f6406af30 0x7f8f6406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:46.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.151+0000 7f8f7cb45640 1 --2- 172.21.15.154:0/3510518108 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8f6406af30 0x7f8f6406d3f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f8f6800b820 tx=0x7f8f68008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:46.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.154+0000 7f8f757fa640 1 -- 172.21.15.154:0/3510518108 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f8f780ac9f0 con 0x7f8f780cbb30 2023-09-23T03:27:46.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:46.380+0000 7f8f7eda9640 1 -- 172.21.15.154:0/3510518108 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8f780ac9f0 con 0x7f8f780cbb30 2023-09-23T03:27:47.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.040+0000 7f8f757fa640 1 -- 172.21.15.154:0/3510518108 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v107) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8f780ac9f0 con 0x7f8f780cbb30 2023-09-23T03:27:47.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.043+0000 7f8f7eda9640 1 -- 172.21.15.154:0/3510518108 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8f6406af30 msgr2=0x7f8f6406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:47.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.043+0000 7f8f7eda9640 1 --2- 172.21.15.154:0/3510518108 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8f6406af30 0x7f8f6406d3f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f8f6800b820 tx=0x7f8f68008040 comp rx=0 tx=0).stop 2023-09-23T03:27:47.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.043+0000 7f8f7eda9640 1 -- 172.21.15.154:0/3510518108 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f780cbb30 msgr2=0x7f8f780b2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:47.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.043+0000 7f8f7eda9640 1 --2- 172.21.15.154:0/3510518108 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f780cbb30 0x7f8f780b2400 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f8f7000c780 tx=0x7f8f7000cc40 comp rx=0 tx=0).stop 2023-09-23T03:27:47.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.043+0000 7f8f7eda9640 1 -- 172.21.15.154:0/3510518108 shutdown_connections 2023-09-23T03:27:47.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.044+0000 7f8f7eda9640 1 --2- 172.21.15.154:0/3510518108 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f8f6406af30 0x7f8f6406d3f0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:47.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.044+0000 7f8f7eda9640 1 --2- 172.21.15.154:0/3510518108 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8f780cbb30 0x7f8f780b2400 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:47.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.044+0000 7f8f7eda9640 1 --2- 172.21.15.154:0/3510518108 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f8f780cb6b0 0x7f8f7815bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:47.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.044+0000 7f8f7eda9640 1 --2- 172.21.15.154:0/3510518108 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f8f7806a490 0x7f8f780c7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:47.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.044+0000 7f8f7eda9640 1 -- 172.21.15.154:0/3510518108 >> 172.21.15.154:0/3510518108 conn(0x7f8f780cc030 msgr2=0x7f8f780c9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:47.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.044+0000 7f8f7eda9640 1 -- 172.21.15.154:0/3510518108 shutdown_connections 2023-09-23T03:27:47.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.044+0000 7f8f7eda9640 1 -- 172.21.15.154:0/3510518108 wait complete. 2023-09-23T03:27:47.063 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-09-23T03:27:47.065 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-09-23T03:27:47.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.269+0000 7fad84c94640 1 Processor -- start 2023-09-23T03:27:47.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.269+0000 7fad84c94640 1 -- start start 2023-09-23T03:27:47.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.271+0000 7fad84c94640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fad800c9150 0x7fad8015cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:47.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.271+0000 7fad84c94640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad80071290 0x7fad800ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:47.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.271+0000 7fad84c94640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad800c9d20 con 0x7fad800c8d70 2023-09-23T03:27:47.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.271+0000 7fad84c94640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad80071850 con 0x7fad800c9150 2023-09-23T03:27:47.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.271+0000 7fad84c94640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad80071b30 con 0x7fad80071290 2023-09-23T03:27:47.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.271+0000 7fad77fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fad800c9150 0x7fad8015cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:47.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.271+0000 7fad77fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fad800c9150 0x7fad8015cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50414/0 (socket says 172.21.15.154:50414) 2023-09-23T03:27:47.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.271+0000 7fad77fff640 1 -- 172.21.15.154:0/4277525917 learned_addr learned my addr 172.21.15.154:0/4277525917 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:47.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.272+0000 7fad7ed9d640 1 --2- 172.21.15.154:0/4277525917 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad80071290 0x7fad800ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:47.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.272+0000 7fad77fff640 1 -- 172.21.15.154:0/4277525917 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad80071290 msgr2=0x7fad800ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:47.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.272+0000 7fad77fff640 1 --2- 172.21.15.154:0/4277525917 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad80071290 0x7fad800ca1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:47.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.272+0000 7fad77fff640 1 -- 172.21.15.154:0/4277525917 >> v1:172.21.15.190:6789/0 conn(0x7fad800c8d70 legacy=0x7fad8015c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:47.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.272+0000 7fad77fff640 1 -- 172.21.15.154:0/4277525917 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad80071e10 con 0x7fad800c9150 2023-09-23T03:27:47.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.272+0000 7fad7ed9d640 1 --2- 172.21.15.154:0/4277525917 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad80071290 0x7fad800ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:47.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.273+0000 7fad77fff640 1 --2- 172.21.15.154:0/4277525917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad800c9150 0x7fad8015cde0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fad800b3ce0 tx=0x7fad6802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d86f2bc773fdc1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:47.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.273+0000 7fad7dd9b640 1 -- 172.21.15.154:0/4277525917 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad68031830 con 0x7fad800c9150 2023-09-23T03:27:47.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.273+0000 7fad7dd9b640 1 -- 172.21.15.154:0/4277525917 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad68031990 con 0x7fad800c9150 2023-09-23T03:27:47.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.273+0000 7fad7dd9b640 1 -- 172.21.15.154:0/4277525917 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad68031d20 con 0x7fad800c9150 2023-09-23T03:27:47.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.274+0000 7fad84c94640 1 -- 172.21.15.154:0/4277525917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad800c9150 msgr2=0x7fad8015cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:47.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.274+0000 7fad84c94640 1 --2- 172.21.15.154:0/4277525917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad800c9150 0x7fad8015cde0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fad800b3ce0 tx=0x7fad6802b8d0 comp rx=0 tx=0).stop 2023-09-23T03:27:47.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.274+0000 7fad84c94640 1 -- 172.21.15.154:0/4277525917 shutdown_connections 2023-09-23T03:27:47.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.274+0000 7fad84c94640 1 --2- 172.21.15.154:0/4277525917 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad80071290 0x7fad800ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:47.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.274+0000 7fad84c94640 1 --2- 172.21.15.154:0/4277525917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad800c9150 0x7fad8015cde0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:47.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.274+0000 7fad84c94640 1 -- 172.21.15.154:0/4277525917 >> 172.21.15.154:0/4277525917 conn(0x7fad8006c010 msgr2=0x7fad8006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:47.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.274+0000 7fad84c94640 1 -- 172.21.15.154:0/4277525917 shutdown_connections 2023-09-23T03:27:47.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.274+0000 7fad84c94640 1 -- 172.21.15.154:0/4277525917 wait complete. 2023-09-23T03:27:47.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.276+0000 7fad84c94640 1 Processor -- start 2023-09-23T03:27:47.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.276+0000 7fad84c94640 1 -- start start 2023-09-23T03:27:47.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.277+0000 7fad84c94640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad80071290 0x7fad800b2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:47.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.277+0000 7fad84c94640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad800c8d70 0x7fad80153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:47.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.277+0000 7fad84c94640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad800c9150 0x7fad8015ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:47.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.277+0000 7fad84c94640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad800c65d0 con 0x7fad800c9150 2023-09-23T03:27:47.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.277+0000 7fad84c94640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8015d380 con 0x7fad80071290 2023-09-23T03:27:47.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.277+0000 7fad84c94640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad800b2ee0 con 0x7fad800c8d70 2023-09-23T03:27:47.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.277+0000 7fad7ed9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad800c9150 0x7fad8015ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:47.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.277+0000 7fad7ed9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad800c9150 0x7fad8015ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50424/0 (socket says 172.21.15.154:50424) 2023-09-23T03:27:47.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.277+0000 7fad7ed9d640 1 -- 172.21.15.154:0/1385397739 learned_addr learned my addr 172.21.15.154:0/1385397739 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:47.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.277+0000 7fad7e59c640 1 --2- 172.21.15.154:0/1385397739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad80071290 0x7fad800b2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:47.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.277+0000 7fad77fff640 1 --2- 172.21.15.154:0/1385397739 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad800c8d70 0x7fad80153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:47.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.277+0000 7fad7ed9d640 1 -- 172.21.15.154:0/1385397739 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad800c8d70 msgr2=0x7fad80153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:47.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.277+0000 7fad7ed9d640 1 --2- 172.21.15.154:0/1385397739 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad800c8d70 0x7fad80153a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:47.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.277+0000 7fad7ed9d640 1 -- 172.21.15.154:0/1385397739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad80071290 msgr2=0x7fad800b2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:47.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.278+0000 7fad7ed9d640 1 --2- 172.21.15.154:0/1385397739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad80071290 0x7fad800b2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:47.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.278+0000 7fad7ed9d640 1 -- 172.21.15.154:0/1385397739 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad680094d0 con 0x7fad800c9150 2023-09-23T03:27:47.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.278+0000 7fad7e59c640 1 --2- 172.21.15.154:0/1385397739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad80071290 0x7fad800b2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:47.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.278+0000 7fad77fff640 1 --2- 172.21.15.154:0/1385397739 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad800c8d70 0x7fad80153a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:47.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.278+0000 7fad7ed9d640 1 --2- 172.21.15.154:0/1385397739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad800c9150 0x7fad8015ce40 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fad7800cef0 tx=0x7fad7800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:47.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.278+0000 7fad76ffd640 1 -- 172.21.15.154:0/1385397739 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7800d840 con 0x7fad800c9150 2023-09-23T03:27:47.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.278+0000 7fad76ffd640 1 -- 172.21.15.154:0/1385397739 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7800d9c0 con 0x7fad800c9150 2023-09-23T03:27:47.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.278+0000 7fad76ffd640 1 -- 172.21.15.154:0/1385397739 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7800dd70 con 0x7fad800c9150 2023-09-23T03:27:47.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.278+0000 7fad84c94640 1 -- 172.21.15.154:0/1385397739 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad800b31c0 con 0x7fad800c9150 2023-09-23T03:27:47.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.278+0000 7fad84c94640 1 -- 172.21.15.154:0/1385397739 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad800c6820 con 0x7fad800c9150 2023-09-23T03:27:47.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.279+0000 7fad84c94640 1 -- 172.21.15.154:0/1385397739 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad8004e920 con 0x7fad800c9150 2023-09-23T03:27:47.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.281+0000 7fad76ffd640 1 -- 172.21.15.154:0/1385397739 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fad7801e3b0 con 0x7fad800c9150 2023-09-23T03:27:47.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.282+0000 7fad76ffd640 1 --2- 172.21.15.154:0/1385397739 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fad6006af30 0x7fad6006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:47.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.282+0000 7fad76ffd640 1 -- 172.21.15.154:0/1385397739 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fad7808a3b0 con 0x7fad800c9150 2023-09-23T03:27:47.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.282+0000 7fad7e59c640 1 --2- 172.21.15.154:0/1385397739 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fad6006af30 0x7fad6006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:47.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.283+0000 7fad7e59c640 1 --2- 172.21.15.154:0/1385397739 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fad6006af30 0x7fad6006d3f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fad70003eb0 tx=0x7fad70009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:47.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.286+0000 7fad76ffd640 1 -- 172.21.15.154:0/1385397739 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fad78055360 con 0x7fad800c9150 2023-09-23T03:27:47.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:47.517+0000 7fad84c94640 1 -- 172.21.15.154:0/1385397739 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fad800c6a10 con 0x7fad800c9150 2023-09-23T03:27:48.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.053+0000 7fad76ffd640 1 -- 172.21.15.154:0/1385397739 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v108) v1 ==== 122+0+0 (secure 0 0 0) 0x7fad78058980 con 0x7fad800c9150 2023-09-23T03:27:48.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.056+0000 7fad84c94640 1 -- 172.21.15.154:0/1385397739 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fad6006af30 msgr2=0x7fad6006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:48.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.056+0000 7fad84c94640 1 --2- 172.21.15.154:0/1385397739 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fad6006af30 0x7fad6006d3f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fad70003eb0 tx=0x7fad70009040 comp rx=0 tx=0).stop 2023-09-23T03:27:48.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.056+0000 7fad84c94640 1 -- 172.21.15.154:0/1385397739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad800c9150 msgr2=0x7fad8015ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:48.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.056+0000 7fad84c94640 1 --2- 172.21.15.154:0/1385397739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad800c9150 0x7fad8015ce40 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fad7800cef0 tx=0x7fad7800a710 comp rx=0 tx=0).stop 2023-09-23T03:27:48.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.057+0000 7fad84c94640 1 -- 172.21.15.154:0/1385397739 shutdown_connections 2023-09-23T03:27:48.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.057+0000 7fad84c94640 1 --2- 172.21.15.154:0/1385397739 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fad6006af30 0x7fad6006d3f0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:48.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.057+0000 7fad84c94640 1 --2- 172.21.15.154:0/1385397739 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fad800c9150 0x7fad8015ce40 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:48.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.057+0000 7fad84c94640 1 --2- 172.21.15.154:0/1385397739 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fad800c8d70 0x7fad80153a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:48.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.057+0000 7fad84c94640 1 --2- 172.21.15.154:0/1385397739 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fad80071290 0x7fad800b2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:48.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.057+0000 7fad84c94640 1 -- 172.21.15.154:0/1385397739 >> 172.21.15.154:0/1385397739 conn(0x7fad8006c010 msgr2=0x7fad800c25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:48.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.057+0000 7fad84c94640 1 -- 172.21.15.154:0/1385397739 shutdown_connections 2023-09-23T03:27:48.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.057+0000 7fad84c94640 1 -- 172.21.15.154:0/1385397739 wait complete. 2023-09-23T03:27:48.077 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-09-23T03:27:48.078 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:27:48.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.294+0000 7fae0a81a640 1 Processor -- start 2023-09-23T03:27:48.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.294+0000 7fae0a81a640 1 -- start start 2023-09-23T03:27:48.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.296+0000 7fae0a81a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fae040cdfa0 0x7fae040659f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:48.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.296+0000 7fae0a81a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae040b1400 0x7fae040ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:48.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.296+0000 7fae0a81a640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae04065f30 con 0x7fae0415cf80 2023-09-23T03:27:48.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.296+0000 7fae0a81a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae040ce8c0 con 0x7fae040cdfa0 2023-09-23T03:27:48.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.296+0000 7fae0a81a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae0415d550 con 0x7fae040b1400 2023-09-23T03:27:48.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.296+0000 7fae03fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fae040cdfa0 0x7fae040659f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:48.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.296+0000 7fae08db7640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fae0415cf80 0x7fae04160770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:47890/0 (socket says 172.21.15.154:47890) 2023-09-23T03:27:48.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.296+0000 7fae08db7640 1 -- 172.21.15.154:0/2304071538 learned_addr learned my addr 172.21.15.154:0/2304071538 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:27:48.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.296+0000 7fae03fff640 1 -- 172.21.15.154:0/2304071538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae040b1400 msgr2=0x7fae040ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:48.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.296+0000 7fae037fe640 1 --2- 172.21.15.154:0/2304071538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae040b1400 0x7fae040ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:48.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.296+0000 7fae03fff640 1 --2- 172.21.15.154:0/2304071538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae040b1400 0x7fae040ce380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:48.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.296+0000 7fae03fff640 1 -- 172.21.15.154:0/2304071538 >> v1:172.21.15.154:6789/0 conn(0x7fae0415cf80 legacy=0x7fae04160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:48.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.297+0000 7fae03fff640 1 -- 172.21.15.154:0/2304071538 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0415d830 con 0x7fae040cdfa0 2023-09-23T03:27:48.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.296+0000 7fae02ffd640 1 -- 172.21.15.154:0/2304071538 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3022140127 0 0) 0x7fae04065f30 con 0x7fae0415cf80 2023-09-23T03:27:48.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.297+0000 7fae037fe640 1 --2- 172.21.15.154:0/2304071538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae040b1400 0x7fae040ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:48.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.298+0000 7fae03fff640 1 --2- 172.21.15.154:0/2304071538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae040cdfa0 0x7fae040659f0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fadf40097a0 tx=0x7fadf402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b383a311c0821e88 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:48.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.298+0000 7fae02ffd640 1 -- 172.21.15.154:0/2304071538 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf4031780 con 0x7fae040cdfa0 2023-09-23T03:27:48.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.298+0000 7fae02ffd640 1 -- 172.21.15.154:0/2304071538 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf40318e0 con 0x7fae040cdfa0 2023-09-23T03:27:48.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.298+0000 7fae02ffd640 1 -- 172.21.15.154:0/2304071538 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf4031c70 con 0x7fae040cdfa0 2023-09-23T03:27:48.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.298+0000 7fae0a81a640 1 -- 172.21.15.154:0/2304071538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae040cdfa0 msgr2=0x7fae040659f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:48.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.298+0000 7fae0a81a640 1 --2- 172.21.15.154:0/2304071538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae040cdfa0 0x7fae040659f0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fadf40097a0 tx=0x7fadf402b820 comp rx=0 tx=0).stop 2023-09-23T03:27:48.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.299+0000 7fae0a81a640 1 -- 172.21.15.154:0/2304071538 shutdown_connections 2023-09-23T03:27:48.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.299+0000 7fae0a81a640 1 --2- 172.21.15.154:0/2304071538 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae040b1400 0x7fae040ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:48.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.299+0000 7fae0a81a640 1 --2- 172.21.15.154:0/2304071538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae040cdfa0 0x7fae040659f0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:48.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.299+0000 7fae0a81a640 1 -- 172.21.15.154:0/2304071538 >> 172.21.15.154:0/2304071538 conn(0x7fae040b08e0 msgr2=0x7fae040b0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:48.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.299+0000 7fae0a81a640 1 -- 172.21.15.154:0/2304071538 shutdown_connections 2023-09-23T03:27:48.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.299+0000 7fae0a81a640 1 -- 172.21.15.154:0/2304071538 wait complete. 2023-09-23T03:27:48.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.300+0000 7fae0a81a640 1 Processor -- start 2023-09-23T03:27:48.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.300+0000 7fae0a81a640 1 -- start start 2023-09-23T03:27:48.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.301+0000 7fae0a81a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae040b1400 0x7fae040c5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:48.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.301+0000 7fae0a81a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae040cdfa0 0x7fae0409eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:48.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.301+0000 7fae0a81a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae0415cf80 0x7fae0409f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:48.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.301+0000 7fae0a81a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0409e5b0 con 0x7fae040b1400 2023-09-23T03:27:48.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.301+0000 7fae0a81a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0409da30 con 0x7fae040cdfa0 2023-09-23T03:27:48.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.301+0000 7fae0a81a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae0409dd10 con 0x7fae0415cf80 2023-09-23T03:27:48.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.302+0000 7fae03fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae040b1400 0x7fae040c5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:48.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.302+0000 7fae03fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae040b1400 0x7fae040c5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50452/0 (socket says 172.21.15.154:50452) 2023-09-23T03:27:48.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.302+0000 7fae03fff640 1 -- 172.21.15.154:0/1077964131 learned_addr learned my addr 172.21.15.154:0/1077964131 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:48.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.302+0000 7fae03fff640 1 -- 172.21.15.154:0/1077964131 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae0415cf80 msgr2=0x7fae0409f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:48.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.302+0000 7fae03fff640 1 --2- 172.21.15.154:0/1077964131 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae0415cf80 0x7fae0409f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:48.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.302+0000 7fae03fff640 1 -- 172.21.15.154:0/1077964131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae040cdfa0 msgr2=0x7fae0409eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:48.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.302+0000 7fae037fe640 1 --2- 172.21.15.154:0/1077964131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae040cdfa0 0x7fae0409eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:48.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.302+0000 7fae03fff640 1 --2- 172.21.15.154:0/1077964131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae040cdfa0 0x7fae0409eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:48.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.302+0000 7fae03fff640 1 -- 172.21.15.154:0/1077964131 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf4009400 con 0x7fae040b1400 2023-09-23T03:27:48.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.302+0000 7fae03fff640 1 --2- 172.21.15.154:0/1077964131 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae040b1400 0x7fae040c5ec0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fadf4034040 tx=0x7fadf40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:48.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.303+0000 7fae00ff9640 1 -- 172.21.15.154:0/1077964131 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf4032960 con 0x7fae040b1400 2023-09-23T03:27:48.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.303+0000 7fae00ff9640 1 -- 172.21.15.154:0/1077964131 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf4032ac0 con 0x7fae040b1400 2023-09-23T03:27:48.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.303+0000 7fae00ff9640 1 -- 172.21.15.154:0/1077964131 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf4032e70 con 0x7fae040b1400 2023-09-23T03:27:48.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.303+0000 7fae0a81a640 1 -- 172.21.15.154:0/1077964131 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae040b2a50 con 0x7fae040b1400 2023-09-23T03:27:48.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.303+0000 7fae0a81a640 1 -- 172.21.15.154:0/1077964131 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae0409dff0 con 0x7fae040b1400 2023-09-23T03:27:48.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.304+0000 7fae0a81a640 1 -- 172.21.15.154:0/1077964131 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae040b28b0 con 0x7fae040b1400 2023-09-23T03:27:48.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.305+0000 7fae00ff9640 1 -- 172.21.15.154:0/1077964131 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fadf403b3f0 con 0x7fae040b1400 2023-09-23T03:27:48.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.306+0000 7fae00ff9640 1 --2- 172.21.15.154:0/1077964131 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fadf006af30 0x7fadf006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:48.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.306+0000 7fae00ff9640 1 -- 172.21.15.154:0/1077964131 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fadf40abdb0 con 0x7fae040b1400 2023-09-23T03:27:48.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.306+0000 7fae037fe640 1 --2- 172.21.15.154:0/1077964131 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fadf006af30 0x7fadf006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:48.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.307+0000 7fae037fe640 1 --2- 172.21.15.154:0/1077964131 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fadf006af30 0x7fadf006d3f0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fae04065d80 tx=0x7fadf8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:48.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.310+0000 7fae00ff9640 1 -- 172.21.15.154:0/1077964131 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fae040b28b0 con 0x7fae040b1400 2023-09-23T03:27:48.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.539+0000 7fae0a81a640 1 -- 172.21.15.154:0/1077964131 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae040b2f70 con 0x7fae040b1400 2023-09-23T03:27:48.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.540+0000 7fae00ff9640 1 -- 172.21.15.154:0/1077964131 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) v1 ==== 95+0+4933 (secure 0 0 0) 0x7fae040b2f70 con 0x7fae040b1400 2023-09-23T03:27:48.933 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:48.933 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":108,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/2759187683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2759187683},{"type":"v1","addr":"172.21.15.154:6835","nonce":2759187683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":6544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/4147378242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":4147378242},{"type":"v1","addr":"172.21.15.190:6834","nonce":4147378242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103},{"gid":8907,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/4006549651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4006549651},{"type":"v1","addr":"172.21.15.154:6837","nonce":4006549651}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:27:44.470312+0000","modified":"2023-09-23T03:27:47.518558+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8919},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8919":{"gid":8919,"name":"b","rank":0,"incarnation":104,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/3558096060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3558096060},{"type":"v1","addr":"172.21.15.190:6835","nonce":3558096060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-09-23T03:27:48.936 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 108 2023-09-23T03:27:48.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.542+0000 7fae0a81a640 1 -- 172.21.15.154:0/1077964131 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fadf006af30 msgr2=0x7fadf006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:48.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.542+0000 7fae0a81a640 1 --2- 172.21.15.154:0/1077964131 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fadf006af30 0x7fadf006d3f0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fae04065d80 tx=0x7fadf8008040 comp rx=0 tx=0).stop 2023-09-23T03:27:48.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.542+0000 7fae0a81a640 1 -- 172.21.15.154:0/1077964131 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae040b1400 msgr2=0x7fae040c5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:48.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.542+0000 7fae0a81a640 1 --2- 172.21.15.154:0/1077964131 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae040b1400 0x7fae040c5ec0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fadf4034040 tx=0x7fadf40046c0 comp rx=0 tx=0).stop 2023-09-23T03:27:48.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.543+0000 7fae0a81a640 1 -- 172.21.15.154:0/1077964131 shutdown_connections 2023-09-23T03:27:48.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.543+0000 7fae0a81a640 1 --2- 172.21.15.154:0/1077964131 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fadf006af30 0x7fadf006d3f0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:48.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.543+0000 7fae0a81a640 1 --2- 172.21.15.154:0/1077964131 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fae0415cf80 0x7fae0409f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:48.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.543+0000 7fae0a81a640 1 --2- 172.21.15.154:0/1077964131 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fae040cdfa0 0x7fae0409eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:48.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.543+0000 7fae0a81a640 1 --2- 172.21.15.154:0/1077964131 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fae040b1400 0x7fae040c5ec0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:48.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.543+0000 7fae0a81a640 1 -- 172.21.15.154:0/1077964131 >> 172.21.15.154:0/1077964131 conn(0x7fae040b08e0 msgr2=0x7fae040cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:48.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.543+0000 7fae0a81a640 1 -- 172.21.15.154:0/1077964131 shutdown_connections 2023-09-23T03:27:48.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:48.543+0000 7fae0a81a640 1 -- 172.21.15.154:0/1077964131 wait complete. 2023-09-23T03:27:48.938 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:27:49.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.171+0000 7f2b929e2640 1 Processor -- start 2023-09-23T03:27:49.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.171+0000 7f2b929e2640 1 -- start start 2023-09-23T03:27:49.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.172+0000 7f2b929e2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2b8c0cdfa0 0x7f2b8c0659f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:49.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.172+0000 7f2b929e2640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2b8c0b1400 0x7f2b8c0ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:49.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.172+0000 7f2b929e2640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b8c065f30 con 0x7f2b8c15cf80 2023-09-23T03:27:49.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.172+0000 7f2b929e2640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b8c0ce8c0 con 0x7f2b8c0cdfa0 2023-09-23T03:27:49.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.172+0000 7f2b929e2640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b8c15d550 con 0x7f2b8c0b1400 2023-09-23T03:27:49.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.172+0000 7f2b8bfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2b8c0cdfa0 0x7f2b8c0659f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:49.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.172+0000 7f2b90f7f640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f2b8c15cf80 0x7f2b8c160770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:47902/0 (socket says 172.21.15.154:47902) 2023-09-23T03:27:49.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.172+0000 7f2b90f7f640 1 -- 172.21.15.154:0/2119451334 learned_addr learned my addr 172.21.15.154:0/2119451334 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:27:49.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.172+0000 7f2b8bfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2b8c0cdfa0 0x7f2b8c0659f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50462/0 (socket says 172.21.15.154:50462) 2023-09-23T03:27:49.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.173+0000 7f2b8b7fe640 1 --2- 172.21.15.154:0/2119451334 >> v2:172.21.15.190:3300/0 conn(0x7f2b8c0b1400 0x7f2b8c0ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:49.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.173+0000 7f2b8bfff640 1 -- 172.21.15.154:0/2119451334 >> v2:172.21.15.190:3300/0 conn(0x7f2b8c0b1400 msgr2=0x7f2b8c0ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:49.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.173+0000 7f2b8bfff640 1 --2- 172.21.15.154:0/2119451334 >> v2:172.21.15.190:3300/0 conn(0x7f2b8c0b1400 0x7f2b8c0ce380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:49.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.173+0000 7f2b8bfff640 1 -- 172.21.15.154:0/2119451334 >> v1:172.21.15.154:6789/0 conn(0x7f2b8c15cf80 legacy=0x7f2b8c160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:49.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.173+0000 7f2b8affd640 1 -- 172.21.15.154:0/2119451334 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 689375405 0 0) 0x7f2b8c065f30 con 0x7f2b8c15cf80 2023-09-23T03:27:49.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.173+0000 7f2b8bfff640 1 -- 172.21.15.154:0/2119451334 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b8c15d830 con 0x7f2b8c0cdfa0 2023-09-23T03:27:49.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.174+0000 7f2b8bfff640 1 --2- 172.21.15.154:0/2119451334 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b8c0cdfa0 0x7f2b8c0659f0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c0724c0 tx=0x7f2b7c02bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53add1c022953dee server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:49.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.174+0000 7f2b8affd640 1 -- 172.21.15.154:0/2119451334 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c02cb60 con 0x7f2b8c0cdfa0 2023-09-23T03:27:49.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.174+0000 7f2b8affd640 1 -- 172.21.15.154:0/2119451334 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b7c02ccc0 con 0x7f2b8c0cdfa0 2023-09-23T03:27:49.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.174+0000 7f2b8affd640 1 -- 172.21.15.154:0/2119451334 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c034660 con 0x7f2b8c0cdfa0 2023-09-23T03:27:49.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.175+0000 7f2b929e2640 1 -- 172.21.15.154:0/2119451334 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b8c0cdfa0 msgr2=0x7f2b8c0659f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:49.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.175+0000 7f2b929e2640 1 --2- 172.21.15.154:0/2119451334 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b8c0cdfa0 0x7f2b8c0659f0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c0724c0 tx=0x7f2b7c02bdb0 comp rx=0 tx=0).stop 2023-09-23T03:27:49.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.175+0000 7f2b929e2640 1 -- 172.21.15.154:0/2119451334 shutdown_connections 2023-09-23T03:27:49.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.175+0000 7f2b929e2640 1 --2- 172.21.15.154:0/2119451334 >> v2:172.21.15.190:3300/0 conn(0x7f2b8c0b1400 0x7f2b8c0ce380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:49.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.175+0000 7f2b929e2640 1 --2- 172.21.15.154:0/2119451334 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b8c0cdfa0 0x7f2b8c0659f0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:49.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.175+0000 7f2b929e2640 1 -- 172.21.15.154:0/2119451334 >> 172.21.15.154:0/2119451334 conn(0x7f2b8c0b08e0 msgr2=0x7f2b8c0b0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:49.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.175+0000 7f2b929e2640 1 -- 172.21.15.154:0/2119451334 shutdown_connections 2023-09-23T03:27:49.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.175+0000 7f2b929e2640 1 -- 172.21.15.154:0/2119451334 wait complete. 2023-09-23T03:27:49.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.177+0000 7f2b929e2640 1 Processor -- start 2023-09-23T03:27:49.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.177+0000 7f2b929e2640 1 -- start start 2023-09-23T03:27:49.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.178+0000 7f2b929e2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b8c0b1400 0x7f2b8c0c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:49.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.178+0000 7f2b929e2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b8c0cdfa0 0x7f2b8c1f1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:49.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.178+0000 7f2b929e2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b8c15cf80 0x7f2b8c0c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:49.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.178+0000 7f2b929e2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c1f2800 con 0x7f2b8c15cf80 2023-09-23T03:27:49.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.178+0000 7f2b929e2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c1f2ae0 con 0x7f2b8c0cdfa0 2023-09-23T03:27:49.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.178+0000 7f2b929e2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c1f2dc0 con 0x7f2b8c0b1400 2023-09-23T03:27:49.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.178+0000 7f2b90f7f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b8c15cf80 0x7f2b8c0c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:49.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.178+0000 7f2b90f7f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b8c15cf80 0x7f2b8c0c84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50474/0 (socket says 172.21.15.154:50474) 2023-09-23T03:27:49.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.178+0000 7f2b90f7f640 1 -- 172.21.15.154:0/3371830762 learned_addr learned my addr 172.21.15.154:0/3371830762 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:49.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.179+0000 7f2b8b7fe640 1 --2- 172.21.15.154:0/3371830762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b8c0cdfa0 0x7f2b8c1f1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:49.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.179+0000 7f2b90f7f640 1 -- 172.21.15.154:0/3371830762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b8c0b1400 msgr2=0x7f2b8c0c6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:49.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.179+0000 7f2b90f7f640 1 --2- 172.21.15.154:0/3371830762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b8c0b1400 0x7f2b8c0c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:49.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.179+0000 7f2b90f7f640 1 -- 172.21.15.154:0/3371830762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b8c0cdfa0 msgr2=0x7f2b8c1f1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:49.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.179+0000 7f2b90f7f640 1 --2- 172.21.15.154:0/3371830762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b8c0cdfa0 0x7f2b8c1f1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:49.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.179+0000 7f2b90f7f640 1 -- 172.21.15.154:0/3371830762 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b7c009a70 con 0x7f2b8c15cf80 2023-09-23T03:27:49.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.179+0000 7f2b90f7f640 1 --2- 172.21.15.154:0/3371830762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b8c15cf80 0x7f2b8c0c84a0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f2b8400c450 tx=0x7f2b8400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:49.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.179+0000 7f2b88ff9640 1 -- 172.21.15.154:0/3371830762 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b840074e0 con 0x7f2b8c15cf80 2023-09-23T03:27:49.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.179+0000 7f2b88ff9640 1 -- 172.21.15.154:0/3371830762 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b84007660 con 0x7f2b8c15cf80 2023-09-23T03:27:49.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.179+0000 7f2b88ff9640 1 -- 172.21.15.154:0/3371830762 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b84004a20 con 0x7f2b8c15cf80 2023-09-23T03:27:49.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.179+0000 7f2b929e2640 1 -- 172.21.15.154:0/3371830762 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b8c1f30a0 con 0x7f2b8c15cf80 2023-09-23T03:27:49.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.179+0000 7f2b929e2640 1 -- 172.21.15.154:0/3371830762 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b8c1f22b0 con 0x7f2b8c15cf80 2023-09-23T03:27:49.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.180+0000 7f2b929e2640 1 -- 172.21.15.154:0/3371830762 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b8c1f2590 con 0x7f2b8c15cf80 2023-09-23T03:27:49.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.181+0000 7f2b88ff9640 1 -- 172.21.15.154:0/3371830762 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f2b84004b80 con 0x7f2b8c15cf80 2023-09-23T03:27:49.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.182+0000 7f2b88ff9640 1 --2- 172.21.15.154:0/3371830762 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2b6c06af30 0x7f2b6c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:49.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.182+0000 7f2b88ff9640 1 -- 172.21.15.154:0/3371830762 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2b84088a50 con 0x7f2b8c15cf80 2023-09-23T03:27:49.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.182+0000 7f2b8bfff640 1 --2- 172.21.15.154:0/3371830762 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2b6c06af30 0x7f2b6c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:49.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.183+0000 7f2b8bfff640 1 --2- 172.21.15.154:0/3371830762 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2b6c06af30 0x7f2b6c06d3f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c036000 tx=0x7f2b7c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:49.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.187+0000 7f2b88ff9640 1 -- 172.21.15.154:0/3371830762 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f2b84053a10 con 0x7f2b8c15cf80 2023-09-23T03:27:49.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.378+0000 7f2b929e2640 1 -- 172.21.15.154:0/3371830762 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b8c1f3230 con 0x7f2b8c15cf80 2023-09-23T03:27:49.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.379+0000 7f2b88ff9640 1 -- 172.21.15.154:0/3371830762 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+16820 (secure 0 0 0) 0x7f2b84018020 con 0x7f2b8c15cf80 2023-09-23T03:27:49.380 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:49.380 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":126,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:27:46.016186+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:27:42.207617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-09-23T03:27:43.473867+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:27:49.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.382+0000 7f2b929e2640 1 -- 172.21.15.154:0/3371830762 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2b6c06af30 msgr2=0x7f2b6c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:49.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.382+0000 7f2b929e2640 1 --2- 172.21.15.154:0/3371830762 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2b6c06af30 0x7f2b6c06d3f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c036000 tx=0x7f2b7c02e000 comp rx=0 tx=0).stop 2023-09-23T03:27:49.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.382+0000 7f2b929e2640 1 -- 172.21.15.154:0/3371830762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b8c15cf80 msgr2=0x7f2b8c0c84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:49.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.382+0000 7f2b929e2640 1 --2- 172.21.15.154:0/3371830762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b8c15cf80 0x7f2b8c0c84a0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f2b8400c450 tx=0x7f2b8400c910 comp rx=0 tx=0).stop 2023-09-23T03:27:49.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.382+0000 7f2b929e2640 1 -- 172.21.15.154:0/3371830762 shutdown_connections 2023-09-23T03:27:49.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.382+0000 7f2b929e2640 1 --2- 172.21.15.154:0/3371830762 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2b6c06af30 0x7f2b6c06d3f0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:49.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.382+0000 7f2b929e2640 1 --2- 172.21.15.154:0/3371830762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2b8c15cf80 0x7f2b8c0c84a0 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:49.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.382+0000 7f2b929e2640 1 --2- 172.21.15.154:0/3371830762 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2b8c0cdfa0 0x7f2b8c1f1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:49.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.382+0000 7f2b929e2640 1 --2- 172.21.15.154:0/3371830762 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2b8c0b1400 0x7f2b8c0c6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:49.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.382+0000 7f2b929e2640 1 -- 172.21.15.154:0/3371830762 >> 172.21.15.154:0/3371830762 conn(0x7f2b8c0b08e0 msgr2=0x7f2b8c069cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:49.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.382+0000 7f2b929e2640 1 -- 172.21.15.154:0/3371830762 shutdown_connections 2023-09-23T03:27:49.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.382+0000 7f2b929e2640 1 -- 172.21.15.154:0/3371830762 wait complete. 2023-09-23T03:27:49.403 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-09-23T03:27:49.403 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-23T03:27:49.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.610+0000 7fbb478a9640 1 Processor -- start 2023-09-23T03:27:49.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.610+0000 7fbb478a9640 1 -- start start 2023-09-23T03:27:49.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.612+0000 7fbb478a9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbb400cf040 0x7fbb401656e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:49.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.612+0000 7fbb478a9640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb4015ac30 con 0x7fbb400acce0 2023-09-23T03:27:49.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.612+0000 7fbb478a9640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb400b1f70 con 0x7fbb400b1b20 2023-09-23T03:27:49.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.612+0000 7fbb478a9640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb400b2e90 con 0x7fbb400cf040 2023-09-23T03:27:49.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.612+0000 7fbb45e46640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbb400cf040 0x7fbb401656e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:49.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.612+0000 7fbb45e46640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fbb400cf040 0x7fbb401656e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50484/0 (socket says 172.21.15.154:50484) 2023-09-23T03:27:49.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.612+0000 7fbb45e46640 1 -- 172.21.15.154:0/3003578018 learned_addr learned my addr 172.21.15.154:0/3003578018 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:49.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.612+0000 7fbb45645640 1 --1- 172.21.15.154:0/3003578018 >> v1:172.21.15.154:6789/0 conn(0x7fbb400acce0 0x7fbb4015ed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:47914/0 (socket says 172.21.15.154:47914) 2023-09-23T03:27:49.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.613+0000 7fbb45e46640 1 -- 172.21.15.154:0/3003578018 >> v1:172.21.15.190:6789/0 conn(0x7fbb400b1b20 legacy=0x7fbb400ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:49.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.613+0000 7fbb45e46640 1 -- 172.21.15.154:0/3003578018 >> v1:172.21.15.154:6789/0 conn(0x7fbb400acce0 legacy=0x7fbb4015ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:49.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.613+0000 7fbb45e46640 1 -- 172.21.15.154:0/3003578018 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb400b2210 con 0x7fbb400cf040 2023-09-23T03:27:49.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.613+0000 7fbb37fff640 1 -- 172.21.15.154:0/3003578018 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2757103706 0 0) 0x7fbb3c0011d0 con 0x7fbb400acce0 2023-09-23T03:27:49.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.614+0000 7fbb45e46640 1 --2- 172.21.15.154:0/3003578018 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb400cf040 0x7fbb401656e0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fbb38009e50 tx=0x7fbb3802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db2ed56052bec90 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:49.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.614+0000 7fbb37fff640 1 -- 172.21.15.154:0/3003578018 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3802cb20 con 0x7fbb400cf040 2023-09-23T03:27:49.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.614+0000 7fbb37fff640 1 -- 172.21.15.154:0/3003578018 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb3802cc80 con 0x7fbb400cf040 2023-09-23T03:27:49.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.614+0000 7fbb37fff640 1 -- 172.21.15.154:0/3003578018 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb380343f0 con 0x7fbb400cf040 2023-09-23T03:27:49.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.615+0000 7fbb478a9640 1 -- 172.21.15.154:0/3003578018 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb400cf040 msgr2=0x7fbb401656e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:49.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.615+0000 7fbb478a9640 1 --2- 172.21.15.154:0/3003578018 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb400cf040 0x7fbb401656e0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fbb38009e50 tx=0x7fbb3802bf80 comp rx=0 tx=0).stop 2023-09-23T03:27:49.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.615+0000 7fbb478a9640 1 -- 172.21.15.154:0/3003578018 shutdown_connections 2023-09-23T03:27:49.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.615+0000 7fbb478a9640 1 --2- 172.21.15.154:0/3003578018 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb400cf040 0x7fbb401656e0 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:49.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.615+0000 7fbb478a9640 1 -- 172.21.15.154:0/3003578018 >> 172.21.15.154:0/3003578018 conn(0x7fbb4006a0f0 msgr2=0x7fbb4006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:49.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.615+0000 7fbb478a9640 1 -- 172.21.15.154:0/3003578018 shutdown_connections 2023-09-23T03:27:49.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.615+0000 7fbb478a9640 1 -- 172.21.15.154:0/3003578018 wait complete. 2023-09-23T03:27:49.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.617+0000 7fbb478a9640 1 Processor -- start 2023-09-23T03:27:49.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.617+0000 7fbb478a9640 1 -- start start 2023-09-23T03:27:49.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.618+0000 7fbb478a9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb400acce0 0x7fbb400cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:49.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.618+0000 7fbb478a9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb400b1b20 0x7fbb401e65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:49.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.618+0000 7fbb478a9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb400cf040 0x7fbb4015add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:49.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.618+0000 7fbb478a9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb400ce110 con 0x7fbb400acce0 2023-09-23T03:27:49.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.618+0000 7fbb478a9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb400caa30 con 0x7fbb400cf040 2023-09-23T03:27:49.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.618+0000 7fbb478a9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb400cacd0 con 0x7fbb400b1b20 2023-09-23T03:27:49.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.618+0000 7fbb45645640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb400acce0 0x7fbb400cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:49.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.618+0000 7fbb45645640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb400acce0 0x7fbb400cdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50490/0 (socket says 172.21.15.154:50490) 2023-09-23T03:27:49.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.618+0000 7fbb45645640 1 -- 172.21.15.154:0/1306898049 learned_addr learned my addr 172.21.15.154:0/1306898049 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:49.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.619+0000 7fbb45e46640 1 --2- 172.21.15.154:0/1306898049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb400cf040 0x7fbb4015add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:49.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.619+0000 7fbb44e44640 1 --2- 172.21.15.154:0/1306898049 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb400b1b20 0x7fbb401e65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:49.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.619+0000 7fbb45645640 1 -- 172.21.15.154:0/1306898049 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb400b1b20 msgr2=0x7fbb401e65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:49.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.619+0000 7fbb45645640 1 --2- 172.21.15.154:0/1306898049 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb400b1b20 0x7fbb401e65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:49.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.619+0000 7fbb45645640 1 -- 172.21.15.154:0/1306898049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb400cf040 msgr2=0x7fbb4015add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:49.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.619+0000 7fbb45645640 1 --2- 172.21.15.154:0/1306898049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb400cf040 0x7fbb4015add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:49.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.619+0000 7fbb45645640 1 -- 172.21.15.154:0/1306898049 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb38009b30 con 0x7fbb400acce0 2023-09-23T03:27:49.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.619+0000 7fbb45645640 1 --2- 172.21.15.154:0/1306898049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb400acce0 0x7fbb400cdbd0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00c700 tx=0x7fbb3c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:49.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.620+0000 7fbb35ffb640 1 -- 172.21.15.154:0/1306898049 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c007270 con 0x7fbb400acce0 2023-09-23T03:27:49.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.620+0000 7fbb35ffb640 1 -- 172.21.15.154:0/1306898049 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb3c0073d0 con 0x7fbb400acce0 2023-09-23T03:27:49.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.620+0000 7fbb478a9640 1 -- 172.21.15.154:0/1306898049 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb400cd3d0 con 0x7fbb400acce0 2023-09-23T03:27:49.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.620+0000 7fbb35ffb640 1 -- 172.21.15.154:0/1306898049 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c0057a0 con 0x7fbb400acce0 2023-09-23T03:27:49.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.620+0000 7fbb478a9640 1 -- 172.21.15.154:0/1306898049 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb400c2af0 con 0x7fbb400acce0 2023-09-23T03:27:49.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.621+0000 7fbb478a9640 1 -- 172.21.15.154:0/1306898049 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb400cd8f0 con 0x7fbb400acce0 2023-09-23T03:27:49.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.622+0000 7fbb35ffb640 1 -- 172.21.15.154:0/1306898049 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fbb3c005900 con 0x7fbb400acce0 2023-09-23T03:27:49.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.623+0000 7fbb35ffb640 1 --2- 172.21.15.154:0/1306898049 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbb2406af30 0x7fbb2406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:49.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.623+0000 7fbb35ffb640 1 -- 172.21.15.154:0/1306898049 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fbb3c089140 con 0x7fbb400acce0 2023-09-23T03:27:49.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.623+0000 7fbb44e44640 1 --2- 172.21.15.154:0/1306898049 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbb2406af30 0x7fbb2406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:49.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.624+0000 7fbb44e44640 1 --2- 172.21.15.154:0/1306898049 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbb2406af30 0x7fbb2406d3f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fbb400c1180 tx=0x7fbb30009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:49.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.627+0000 7fbb35ffb640 1 -- 172.21.15.154:0/1306898049 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fbb400cd8f0 con 0x7fbb400acce0 2023-09-23T03:27:49.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.814+0000 7fbb478a9640 1 -- 172.21.15.154:0/1306898049 --> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbb400b27d0 con 0x7fbb2406af30 2023-09-23T03:27:49.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.837+0000 7fbb35ffb640 1 -- 172.21.15.154:0/1306898049 <== mgr.4110 v2:172.21.15.154:6832/2422652104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287051 (secure 0 0 0) 0x7fbb30001c70 con 0x7fbb2406af30 2023-09-23T03:27:49.841 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:49.842 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2023-09-23T03:27:49.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.842+0000 7fbb478a9640 1 -- 172.21.15.154:0/1306898049 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbb2406af30 msgr2=0x7fbb2406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:49.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.842+0000 7fbb478a9640 1 --2- 172.21.15.154:0/1306898049 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbb2406af30 0x7fbb2406d3f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fbb400c1180 tx=0x7fbb30009040 comp rx=0 tx=0).stop 2023-09-23T03:27:49.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.842+0000 7fbb478a9640 1 -- 172.21.15.154:0/1306898049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb400acce0 msgr2=0x7fbb400cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:49.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.842+0000 7fbb478a9640 1 --2- 172.21.15.154:0/1306898049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb400acce0 0x7fbb400cdbd0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00c700 tx=0x7fbb3c00cbc0 comp rx=0 tx=0).stop 2023-09-23T03:27:49.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.842+0000 7fbb478a9640 1 -- 172.21.15.154:0/1306898049 shutdown_connections 2023-09-23T03:27:49.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.842+0000 7fbb478a9640 1 --2- 172.21.15.154:0/1306898049 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbb2406af30 0x7fbb2406d3f0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:49.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.842+0000 7fbb478a9640 1 --2- 172.21.15.154:0/1306898049 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbb400cf040 0x7fbb4015add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:49.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.842+0000 7fbb478a9640 1 --2- 172.21.15.154:0/1306898049 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbb400b1b20 0x7fbb401e65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:49.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.842+0000 7fbb478a9640 1 --2- 172.21.15.154:0/1306898049 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb400acce0 0x7fbb400cdbd0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:49.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.842+0000 7fbb478a9640 1 -- 172.21.15.154:0/1306898049 >> 172.21.15.154:0/1306898049 conn(0x7fbb4006a0f0 msgr2=0x7fbb400aed20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:49.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.843+0000 7fbb478a9640 1 -- 172.21.15.154:0/1306898049 shutdown_connections 2023-09-23T03:27:49.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:49.843+0000 7fbb478a9640 1 -- 172.21.15.154:0/1306898049 wait complete. 2023-09-23T03:27:49.862 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":499,"stamp":"2023-09-23T03:27:49.433961+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":190,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262936,"kb_used_data":13476,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711784,"statfs":{"total":773094113280,"available":772824866816,"internally_reserved":0,"allocated":13799424,"data_stored":7316777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":255315514},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001624"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.381444+0000","last_change":"2023-09-23T03:27:45.025920+0000","last_active":"2023-09-23T03:27:47.381444+0000","last_peered":"2023-09-23T03:27:47.381444+0000","last_clean":"2023-09-23T03:27:47.381444+0000","last_became_active":"2023-09-23T03:27:45.025588+0000","last_became_peered":"2023-09-23T03:27:45.025588+0000","last_unstale":"2023-09-23T03:27:47.381444+0000","last_undegraded":"2023-09-23T03:27:47.381444+0000","last_fullsized":"2023-09-23T03:27:47.381444+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:58:00.149687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.383647+0000","last_change":"2023-09-23T03:27:45.047955+0000","last_active":"2023-09-23T03:27:47.383647+0000","last_peered":"2023-09-23T03:27:47.383647+0000","last_clean":"2023-09-23T03:27:47.383647+0000","last_became_active":"2023-09-23T03:27:45.047592+0000","last_became_peered":"2023-09-23T03:27:45.047592+0000","last_unstale":"2023-09-23T03:27:47.383647+0000","last_undegraded":"2023-09-23T03:27:47.383647+0000","last_fullsized":"2023-09-23T03:27:47.383647+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:26:50.402279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.384169+0000","last_change":"2023-09-23T03:27:45.046402+0000","last_active":"2023-09-23T03:27:47.384169+0000","last_peered":"2023-09-23T03:27:47.384169+0000","last_clean":"2023-09-23T03:27:47.384169+0000","last_became_active":"2023-09-23T03:27:45.046113+0000","last_became_peered":"2023-09-23T03:27:45.046113+0000","last_unstale":"2023-09-23T03:27:47.384169+0000","last_undegraded":"2023-09-23T03:27:47.384169+0000","last_fullsized":"2023-09-23T03:27:47.384169+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:15:12.188323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:46.033055+0000","last_change":"2023-09-23T03:27:45.036372+0000","last_active":"2023-09-23T03:27:46.033055+0000","last_peered":"2023-09-23T03:27:46.033055+0000","last_clean":"2023-09-23T03:27:46.033055+0000","last_became_active":"2023-09-23T03:27:45.036014+0000","last_became_peered":"2023-09-23T03:27:45.036014+0000","last_unstale":"2023-09-23T03:27:46.033055+0000","last_undegraded":"2023-09-23T03:27:46.033055+0000","last_fullsized":"2023-09-23T03:27:46.033055+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:04:35.006829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.381864+0000","last_change":"2023-09-23T03:27:45.017104+0000","last_active":"2023-09-23T03:27:47.381864+0000","last_peered":"2023-09-23T03:27:47.381864+0000","last_clean":"2023-09-23T03:27:47.381864+0000","last_became_active":"2023-09-23T03:27:45.016482+0000","last_became_peered":"2023-09-23T03:27:45.016482+0000","last_unstale":"2023-09-23T03:27:47.381864+0000","last_undegraded":"2023-09-23T03:27:47.381864+0000","last_fullsized":"2023-09-23T03:27:47.381864+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:45:05.020375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.380975+0000","last_change":"2023-09-23T03:27:45.026003+0000","last_active":"2023-09-23T03:27:47.380975+0000","last_peered":"2023-09-23T03:27:47.380975+0000","last_clean":"2023-09-23T03:27:47.380975+0000","last_became_active":"2023-09-23T03:27:45.025738+0000","last_became_peered":"2023-09-23T03:27:45.025738+0000","last_unstale":"2023-09-23T03:27:47.380975+0000","last_undegraded":"2023-09-23T03:27:47.380975+0000","last_fullsized":"2023-09-23T03:27:47.380975+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:41:47.146262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.381445+0000","last_change":"2023-09-23T03:27:45.044087+0000","last_active":"2023-09-23T03:27:47.381445+0000","last_peered":"2023-09-23T03:27:47.381445+0000","last_clean":"2023-09-23T03:27:47.381445+0000","last_became_active":"2023-09-23T03:27:45.043704+0000","last_became_peered":"2023-09-23T03:27:45.043704+0000","last_unstale":"2023-09-23T03:27:47.381445+0000","last_undegraded":"2023-09-23T03:27:47.381445+0000","last_fullsized":"2023-09-23T03:27:47.381445+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:26:03.298065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.381739+0000","last_change":"2023-09-23T03:27:45.042385+0000","last_active":"2023-09-23T03:27:47.381739+0000","last_peered":"2023-09-23T03:27:47.381739+0000","last_clean":"2023-09-23T03:27:47.381739+0000","last_became_active":"2023-09-23T03:27:45.042100+0000","last_became_peered":"2023-09-23T03:27:45.042100+0000","last_unstale":"2023-09-23T03:27:47.381739+0000","last_undegraded":"2023-09-23T03:27:47.381739+0000","last_fullsized":"2023-09-23T03:27:47.381739+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:04:35.006829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:46.034085+0000","last_change":"2023-09-23T03:27:45.047530+0000","last_active":"2023-09-23T03:27:46.034085+0000","last_peered":"2023-09-23T03:27:46.034085+0000","last_clean":"2023-09-23T03:27:46.034085+0000","last_became_active":"2023-09-23T03:27:45.047264+0000","last_became_peered":"2023-09-23T03:27:45.047264+0000","last_unstale":"2023-09-23T03:27:46.034085+0000","last_undegraded":"2023-09-23T03:27:46.034085+0000","last_fullsized":"2023-09-23T03:27:46.034085+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:51:53.201001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.383259+0000","last_change":"2023-09-23T03:27:45.029426+0000","last_active":"2023-09-23T03:27:47.383259+0000","last_peered":"2023-09-23T03:27:47.383259+0000","last_clean":"2023-09-23T03:27:47.383259+0000","last_became_active":"2023-09-23T03:27:45.029159+0000","last_became_peered":"2023-09-23T03:27:45.029159+0000","last_unstale":"2023-09-23T03:27:47.383259+0000","last_undegraded":"2023-09-23T03:27:47.383259+0000","last_fullsized":"2023-09-23T03:27:47.383259+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:35:45.037564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.382242+0000","last_change":"2023-09-23T03:27:45.030013+0000","last_active":"2023-09-23T03:27:47.382242+0000","last_peered":"2023-09-23T03:27:47.382242+0000","last_clean":"2023-09-23T03:27:47.382242+0000","last_became_active":"2023-09-23T03:27:45.029744+0000","last_became_peered":"2023-09-23T03:27:45.029744+0000","last_unstale":"2023-09-23T03:27:47.382242+0000","last_undegraded":"2023-09-23T03:27:47.382242+0000","last_fullsized":"2023-09-23T03:27:47.382242+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:35:45.037564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.382843+0000","last_change":"2023-09-23T03:27:45.028794+0000","last_active":"2023-09-23T03:27:47.382843+0000","last_peered":"2023-09-23T03:27:47.382843+0000","last_clean":"2023-09-23T03:27:47.382843+0000","last_became_active":"2023-09-23T03:27:45.028516+0000","last_became_peered":"2023-09-23T03:27:45.028516+0000","last_unstale":"2023-09-23T03:27:47.382843+0000","last_undegraded":"2023-09-23T03:27:47.382843+0000","last_fullsized":"2023-09-23T03:27:47.382843+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:44:27.854290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.382283+0000","last_change":"2023-09-23T03:27:45.030916+0000","last_active":"2023-09-23T03:27:47.382283+0000","last_peered":"2023-09-23T03:27:47.382283+0000","last_clean":"2023-09-23T03:27:47.382283+0000","last_became_active":"2023-09-23T03:27:45.030657+0000","last_became_peered":"2023-09-23T03:27:45.030657+0000","last_unstale":"2023-09-23T03:27:47.382283+0000","last_undegraded":"2023-09-23T03:27:47.382283+0000","last_fullsized":"2023-09-23T03:27:47.382283+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:14:40.320939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.381361+0000","last_change":"2023-09-23T03:27:45.041728+0000","last_active":"2023-09-23T03:27:47.381361+0000","last_peered":"2023-09-23T03:27:47.381361+0000","last_clean":"2023-09-23T03:27:47.381361+0000","last_became_active":"2023-09-23T03:27:45.041167+0000","last_became_peered":"2023-09-23T03:27:45.041167+0000","last_unstale":"2023-09-23T03:27:47.381361+0000","last_undegraded":"2023-09-23T03:27:47.381361+0000","last_fullsized":"2023-09-23T03:27:47.381361+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:11:36.520222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.380615+0000","last_change":"2023-09-23T03:27:45.031107+0000","last_active":"2023-09-23T03:27:47.380615+0000","last_peered":"2023-09-23T03:27:47.380615+0000","last_clean":"2023-09-23T03:27:47.380615+0000","last_became_active":"2023-09-23T03:27:45.030361+0000","last_became_peered":"2023-09-23T03:27:45.030361+0000","last_unstale":"2023-09-23T03:27:47.380615+0000","last_undegraded":"2023-09-23T03:27:47.380615+0000","last_fullsized":"2023-09-23T03:27:47.380615+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:14:40.320939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.380336+0000","last_change":"2023-09-23T03:27:45.021810+0000","last_active":"2023-09-23T03:27:47.380336+0000","last_peered":"2023-09-23T03:27:47.380336+0000","last_clean":"2023-09-23T03:27:47.380336+0000","last_became_active":"2023-09-23T03:27:45.021146+0000","last_became_peered":"2023-09-23T03:27:45.021146+0000","last_unstale":"2023-09-23T03:27:47.380336+0000","last_undegraded":"2023-09-23T03:27:47.380336+0000","last_fullsized":"2023-09-23T03:27:47.380336+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:04:56.120276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.383982+0000","last_change":"2023-09-23T03:27:45.031602+0000","last_active":"2023-09-23T03:27:47.383982+0000","last_peered":"2023-09-23T03:27:47.383982+0000","last_clean":"2023-09-23T03:27:47.383982+0000","last_became_active":"2023-09-23T03:27:45.031331+0000","last_became_peered":"2023-09-23T03:27:45.031331+0000","last_unstale":"2023-09-23T03:27:47.383982+0000","last_undegraded":"2023-09-23T03:27:47.383982+0000","last_fullsized":"2023-09-23T03:27:47.383982+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:39:04.634154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.383794+0000","last_change":"2023-09-23T03:27:45.047390+0000","last_active":"2023-09-23T03:27:47.383794+0000","last_peered":"2023-09-23T03:27:47.383794+0000","last_clean":"2023-09-23T03:27:47.383794+0000","last_became_active":"2023-09-23T03:27:45.047111+0000","last_became_peered":"2023-09-23T03:27:45.047111+0000","last_unstale":"2023-09-23T03:27:47.383794+0000","last_undegraded":"2023-09-23T03:27:47.383794+0000","last_fullsized":"2023-09-23T03:27:47.383794+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:24:25.148743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:46.035546+0000","last_change":"2023-09-23T03:27:45.038744+0000","last_active":"2023-09-23T03:27:46.035546+0000","last_peered":"2023-09-23T03:27:46.035546+0000","last_clean":"2023-09-23T03:27:46.035546+0000","last_became_active":"2023-09-23T03:27:45.038370+0000","last_became_peered":"2023-09-23T03:27:45.038370+0000","last_unstale":"2023-09-23T03:27:46.035546+0000","last_undegraded":"2023-09-23T03:27:46.035546+0000","last_fullsized":"2023-09-23T03:27:46.035546+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:16:05.766908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:46.032159+0000","last_change":"2023-09-23T03:27:45.022548+0000","last_active":"2023-09-23T03:27:46.032159+0000","last_peered":"2023-09-23T03:27:46.032159+0000","last_clean":"2023-09-23T03:27:46.032159+0000","last_became_active":"2023-09-23T03:27:45.022234+0000","last_became_peered":"2023-09-23T03:27:45.022234+0000","last_unstale":"2023-09-23T03:27:46.032159+0000","last_undegraded":"2023-09-23T03:27:46.032159+0000","last_fullsized":"2023-09-23T03:27:46.032159+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:39:38.063635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:46.033666+0000","last_change":"2023-09-23T03:27:45.036794+0000","last_active":"2023-09-23T03:27:46.033666+0000","last_peered":"2023-09-23T03:27:46.033666+0000","last_clean":"2023-09-23T03:27:46.033666+0000","last_became_active":"2023-09-23T03:27:45.036308+0000","last_became_peered":"2023-09-23T03:27:45.036308+0000","last_unstale":"2023-09-23T03:27:46.033666+0000","last_undegraded":"2023-09-23T03:27:46.033666+0000","last_fullsized":"2023-09-23T03:27:46.033666+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:16:08.067236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.381445+0000","last_change":"2023-09-23T03:27:45.026645+0000","last_active":"2023-09-23T03:27:47.381445+0000","last_peered":"2023-09-23T03:27:47.381445+0000","last_clean":"2023-09-23T03:27:47.381445+0000","last_became_active":"2023-09-23T03:27:45.026306+0000","last_became_peered":"2023-09-23T03:27:45.026306+0000","last_unstale":"2023-09-23T03:27:47.381445+0000","last_undegraded":"2023-09-23T03:27:47.381445+0000","last_fullsized":"2023-09-23T03:27:47.381445+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:41:47.631539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:46.032642+0000","last_change":"2023-09-23T03:27:45.047533+0000","last_active":"2023-09-23T03:27:46.032642+0000","last_peered":"2023-09-23T03:27:46.032642+0000","last_clean":"2023-09-23T03:27:46.032642+0000","last_became_active":"2023-09-23T03:27:45.047218+0000","last_became_peered":"2023-09-23T03:27:45.047218+0000","last_unstale":"2023-09-23T03:27:46.032642+0000","last_undegraded":"2023-09-23T03:27:46.032642+0000","last_fullsized":"2023-09-23T03:27:46.032642+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:54:18.955362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.380442+0000","last_change":"2023-09-23T03:27:45.032574+0000","last_active":"2023-09-23T03:27:47.380442+0000","last_peered":"2023-09-23T03:27:47.380442+0000","last_clean":"2023-09-23T03:27:47.380442+0000","last_became_active":"2023-09-23T03:27:45.032285+0000","last_became_peered":"2023-09-23T03:27:45.032285+0000","last_unstale":"2023-09-23T03:27:47.380442+0000","last_undegraded":"2023-09-23T03:27:47.380442+0000","last_fullsized":"2023-09-23T03:27:47.380442+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:54:34.616945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.382266+0000","last_change":"2023-09-23T03:27:45.038242+0000","last_active":"2023-09-23T03:27:47.382266+0000","last_peered":"2023-09-23T03:27:47.382266+0000","last_clean":"2023-09-23T03:27:47.382266+0000","last_became_active":"2023-09-23T03:27:45.037720+0000","last_became_peered":"2023-09-23T03:27:45.037720+0000","last_unstale":"2023-09-23T03:27:47.382266+0000","last_undegraded":"2023-09-23T03:27:47.382266+0000","last_fullsized":"2023-09-23T03:27:47.382266+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:52:03.205421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:46.032485+0000","last_change":"2023-09-23T03:27:45.017554+0000","last_active":"2023-09-23T03:27:46.032485+0000","last_peered":"2023-09-23T03:27:46.032485+0000","last_clean":"2023-09-23T03:27:46.032485+0000","last_became_active":"2023-09-23T03:27:45.017263+0000","last_became_peered":"2023-09-23T03:27:45.017263+0000","last_unstale":"2023-09-23T03:27:46.032485+0000","last_undegraded":"2023-09-23T03:27:46.032485+0000","last_fullsized":"2023-09-23T03:27:46.032485+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:10:03.525856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:46.035677+0000","last_change":"2023-09-23T03:27:45.041958+0000","last_active":"2023-09-23T03:27:46.035677+0000","last_peered":"2023-09-23T03:27:46.035677+0000","last_clean":"2023-09-23T03:27:46.035677+0000","last_became_active":"2023-09-23T03:27:45.041682+0000","last_became_peered":"2023-09-23T03:27:45.041682+0000","last_unstale":"2023-09-23T03:27:46.035677+0000","last_undegraded":"2023-09-23T03:27:46.035677+0000","last_fullsized":"2023-09-23T03:27:46.035677+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:30:18.768868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.383146+0000","last_change":"2023-09-23T03:27:45.035003+0000","last_active":"2023-09-23T03:27:47.383146+0000","last_peered":"2023-09-23T03:27:47.383146+0000","last_clean":"2023-09-23T03:27:47.383146+0000","last_became_active":"2023-09-23T03:27:45.034666+0000","last_became_peered":"2023-09-23T03:27:45.034666+0000","last_unstale":"2023-09-23T03:27:47.383146+0000","last_undegraded":"2023-09-23T03:27:47.383146+0000","last_fullsized":"2023-09-23T03:27:47.383146+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:05:54.457634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:46.032509+0000","last_change":"2023-09-23T03:27:45.026019+0000","last_active":"2023-09-23T03:27:46.032509+0000","last_peered":"2023-09-23T03:27:46.032509+0000","last_clean":"2023-09-23T03:27:46.032509+0000","last_became_active":"2023-09-23T03:27:45.025765+0000","last_became_peered":"2023-09-23T03:27:45.025765+0000","last_unstale":"2023-09-23T03:27:46.032509+0000","last_undegraded":"2023-09-23T03:27:46.032509+0000","last_fullsized":"2023-09-23T03:27:46.032509+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:07:33.355203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.381719+0000","last_change":"2023-09-23T03:27:45.027482+0000","last_active":"2023-09-23T03:27:47.381719+0000","last_peered":"2023-09-23T03:27:47.381719+0000","last_clean":"2023-09-23T03:27:47.381719+0000","last_became_active":"2023-09-23T03:27:45.027218+0000","last_became_peered":"2023-09-23T03:27:45.027218+0000","last_unstale":"2023-09-23T03:27:47.381719+0000","last_undegraded":"2023-09-23T03:27:47.381719+0000","last_fullsized":"2023-09-23T03:27:47.381719+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:40:45.625728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.381155+0000","last_change":"2023-09-23T03:27:45.029377+0000","last_active":"2023-09-23T03:27:47.381155+0000","last_peered":"2023-09-23T03:27:47.381155+0000","last_clean":"2023-09-23T03:27:47.381155+0000","last_became_active":"2023-09-23T03:27:45.029058+0000","last_became_peered":"2023-09-23T03:27:45.029058+0000","last_unstale":"2023-09-23T03:27:47.381155+0000","last_undegraded":"2023-09-23T03:27:47.381155+0000","last_fullsized":"2023-09-23T03:27:47.381155+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:46:09.395570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.380453+0000","last_change":"2023-09-23T03:27:45.033139+0000","last_active":"2023-09-23T03:27:47.380453+0000","last_peered":"2023-09-23T03:27:47.380453+0000","last_clean":"2023-09-23T03:27:47.380453+0000","last_became_active":"2023-09-23T03:27:45.032841+0000","last_became_peered":"2023-09-23T03:27:45.032841+0000","last_unstale":"2023-09-23T03:27:47.380453+0000","last_undegraded":"2023-09-23T03:27:47.380453+0000","last_fullsized":"2023-09-23T03:27:47.380453+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:00:15.928606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.383420+0000","last_change":"2023-09-23T03:27:45.023987+0000","last_active":"2023-09-23T03:27:47.383420+0000","last_peered":"2023-09-23T03:27:47.383420+0000","last_clean":"2023-09-23T03:27:47.383420+0000","last_became_active":"2023-09-23T03:27:45.023687+0000","last_became_peered":"2023-09-23T03:27:45.023687+0000","last_unstale":"2023-09-23T03:27:47.383420+0000","last_undegraded":"2023-09-23T03:27:47.383420+0000","last_fullsized":"2023-09-23T03:27:47.383420+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:07:46.010795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"125'4","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.045895+0000","last_change":"2023-09-23T03:27:44.026019+0000","last_active":"2023-09-23T03:27:45.045895+0000","last_peered":"2023-09-23T03:27:45.045895+0000","last_clean":"2023-09-23T03:27:45.045895+0000","last_became_active":"2023-09-23T03:27:44.025591+0000","last_became_peered":"2023-09-23T03:27:44.025591+0000","last_unstale":"2023-09-23T03:27:45.045895+0000","last_undegraded":"2023-09-23T03:27:45.045895+0000","last_fullsized":"2023-09-23T03:27:45.045895+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:23:07.728424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.381758+0000","last_change":"2023-09-23T03:27:45.025775+0000","last_active":"2023-09-23T03:27:47.381758+0000","last_peered":"2023-09-23T03:27:47.381758+0000","last_clean":"2023-09-23T03:27:47.381758+0000","last_became_active":"2023-09-23T03:27:45.025480+0000","last_became_peered":"2023-09-23T03:27:45.025480+0000","last_unstale":"2023-09-23T03:27:47.381758+0000","last_undegraded":"2023-09-23T03:27:47.381758+0000","last_fullsized":"2023-09-23T03:27:47.381758+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:14:02.931595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"125'1","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.040299+0000","last_change":"2023-09-23T03:27:44.030761+0000","last_active":"2023-09-23T03:27:45.040299+0000","last_peered":"2023-09-23T03:27:45.040299+0000","last_clean":"2023-09-23T03:27:45.040299+0000","last_became_active":"2023-09-23T03:27:44.030488+0000","last_became_peered":"2023-09-23T03:27:44.030488+0000","last_unstale":"2023-09-23T03:27:45.040299+0000","last_undegraded":"2023-09-23T03:27:45.040299+0000","last_fullsized":"2023-09-23T03:27:45.040299+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:39:37.051847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.382607+0000","last_change":"2023-09-23T03:27:45.040938+0000","last_active":"2023-09-23T03:27:47.382607+0000","last_peered":"2023-09-23T03:27:47.382607+0000","last_clean":"2023-09-23T03:27:47.382607+0000","last_became_active":"2023-09-23T03:27:45.040674+0000","last_became_peered":"2023-09-23T03:27:45.040674+0000","last_unstale":"2023-09-23T03:27:47.382607+0000","last_undegraded":"2023-09-23T03:27:47.382607+0000","last_fullsized":"2023-09-23T03:27:47.382607+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:00:23.239974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"125'2","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.043944+0000","last_change":"2023-09-23T03:27:44.037191+0000","last_active":"2023-09-23T03:27:45.043944+0000","last_peered":"2023-09-23T03:27:45.043944+0000","last_clean":"2023-09-23T03:27:45.043944+0000","last_became_active":"2023-09-23T03:27:44.036801+0000","last_became_peered":"2023-09-23T03:27:44.036801+0000","last_unstale":"2023-09-23T03:27:45.043944+0000","last_undegraded":"2023-09-23T03:27:45.043944+0000","last_fullsized":"2023-09-23T03:27:45.043944+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:14:15.913622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:46.034402+0000","last_change":"2023-09-23T03:27:45.035267+0000","last_active":"2023-09-23T03:27:46.034402+0000","last_peered":"2023-09-23T03:27:46.034402+0000","last_clean":"2023-09-23T03:27:46.034402+0000","last_became_active":"2023-09-23T03:27:45.034952+0000","last_became_peered":"2023-09-23T03:27:45.034952+0000","last_unstale":"2023-09-23T03:27:46.034402+0000","last_undegraded":"2023-09-23T03:27:46.034402+0000","last_fullsized":"2023-09-23T03:27:46.034402+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:07:35.140412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"125'2","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.044236+0000","last_change":"2023-09-23T03:27:44.039286+0000","last_active":"2023-09-23T03:27:45.044236+0000","last_peered":"2023-09-23T03:27:45.044236+0000","last_clean":"2023-09-23T03:27:45.044236+0000","last_became_active":"2023-09-23T03:27:44.038866+0000","last_became_peered":"2023-09-23T03:27:44.038866+0000","last_unstale":"2023-09-23T03:27:45.044236+0000","last_undegraded":"2023-09-23T03:27:45.044236+0000","last_fullsized":"2023-09-23T03:27:45.044236+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:03:40.407589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.382200+0000","last_change":"2023-09-23T03:27:45.034553+0000","last_active":"2023-09-23T03:27:47.382200+0000","last_peered":"2023-09-23T03:27:47.382200+0000","last_clean":"2023-09-23T03:27:47.382200+0000","last_became_active":"2023-09-23T03:27:45.034228+0000","last_became_peered":"2023-09-23T03:27:45.034228+0000","last_unstale":"2023-09-23T03:27:47.382200+0000","last_undegraded":"2023-09-23T03:27:47.382200+0000","last_fullsized":"2023-09-23T03:27:47.382200+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:48:13.711973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"125'1","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.042990+0000","last_change":"2023-09-23T03:27:44.036216+0000","last_active":"2023-09-23T03:27:45.042990+0000","last_peered":"2023-09-23T03:27:45.042990+0000","last_clean":"2023-09-23T03:27:45.042990+0000","last_became_active":"2023-09-23T03:27:44.035843+0000","last_became_peered":"2023-09-23T03:27:44.035843+0000","last_unstale":"2023-09-23T03:27:45.042990+0000","last_undegraded":"2023-09-23T03:27:45.042990+0000","last_fullsized":"2023-09-23T03:27:45.042990+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:53:31.135007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.380965+0000","last_change":"2023-09-23T03:27:45.038771+0000","last_active":"2023-09-23T03:27:47.380965+0000","last_peered":"2023-09-23T03:27:47.380965+0000","last_clean":"2023-09-23T03:27:47.380965+0000","last_became_active":"2023-09-23T03:27:45.038473+0000","last_became_peered":"2023-09-23T03:27:45.038473+0000","last_unstale":"2023-09-23T03:27:47.380965+0000","last_undegraded":"2023-09-23T03:27:47.380965+0000","last_fullsized":"2023-09-23T03:27:47.380965+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:56:03.506431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"125'1","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.048899+0000","last_change":"2023-09-23T03:27:44.032435+0000","last_active":"2023-09-23T03:27:45.048899+0000","last_peered":"2023-09-23T03:27:45.048899+0000","last_clean":"2023-09-23T03:27:45.048899+0000","last_became_active":"2023-09-23T03:27:44.032184+0000","last_became_peered":"2023-09-23T03:27:44.032184+0000","last_unstale":"2023-09-23T03:27:45.048899+0000","last_undegraded":"2023-09-23T03:27:45.048899+0000","last_fullsized":"2023-09-23T03:27:45.048899+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:32:20.075247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.382060+0000","last_change":"2023-09-23T03:27:45.026770+0000","last_active":"2023-09-23T03:27:47.382060+0000","last_peered":"2023-09-23T03:27:47.382060+0000","last_clean":"2023-09-23T03:27:47.382060+0000","last_became_active":"2023-09-23T03:27:45.026469+0000","last_became_peered":"2023-09-23T03:27:45.026469+0000","last_unstale":"2023-09-23T03:27:47.382060+0000","last_undegraded":"2023-09-23T03:27:47.382060+0000","last_fullsized":"2023-09-23T03:27:47.382060+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:16:08.067236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.015222+0000","last_change":"2023-09-23T03:27:44.025904+0000","last_active":"2023-09-23T03:27:45.015222+0000","last_peered":"2023-09-23T03:27:45.015222+0000","last_clean":"2023-09-23T03:27:45.015222+0000","last_became_active":"2023-09-23T03:27:44.025435+0000","last_became_peered":"2023-09-23T03:27:44.025435+0000","last_unstale":"2023-09-23T03:27:45.015222+0000","last_undegraded":"2023-09-23T03:27:45.015222+0000","last_fullsized":"2023-09-23T03:27:45.015222+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:35:44.025775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.382670+0000","last_change":"2023-09-23T03:27:45.025083+0000","last_active":"2023-09-23T03:27:47.382670+0000","last_peered":"2023-09-23T03:27:47.382670+0000","last_clean":"2023-09-23T03:27:47.382670+0000","last_became_active":"2023-09-23T03:27:45.024766+0000","last_became_peered":"2023-09-23T03:27:45.024766+0000","last_unstale":"2023-09-23T03:27:47.382670+0000","last_undegraded":"2023-09-23T03:27:47.382670+0000","last_fullsized":"2023-09-23T03:27:47.382670+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:41:47.631539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.008645+0000","last_change":"2023-09-23T03:27:44.039337+0000","last_active":"2023-09-23T03:27:45.008645+0000","last_peered":"2023-09-23T03:27:45.008645+0000","last_clean":"2023-09-23T03:27:45.008645+0000","last_became_active":"2023-09-23T03:27:44.039053+0000","last_became_peered":"2023-09-23T03:27:44.039053+0000","last_unstale":"2023-09-23T03:27:45.008645+0000","last_undegraded":"2023-09-23T03:27:45.008645+0000","last_fullsized":"2023-09-23T03:27:45.008645+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:54:00.914666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:46.035160+0000","last_change":"2023-09-23T03:27:45.040008+0000","last_active":"2023-09-23T03:27:46.035160+0000","last_peered":"2023-09-23T03:27:46.035160+0000","last_clean":"2023-09-23T03:27:46.035160+0000","last_became_active":"2023-09-23T03:27:45.039684+0000","last_became_peered":"2023-09-23T03:27:45.039684+0000","last_unstale":"2023-09-23T03:27:46.035160+0000","last_undegraded":"2023-09-23T03:27:46.035160+0000","last_fullsized":"2023-09-23T03:27:46.035160+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:00:15.928606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.010215+0000","last_change":"2023-09-23T03:27:44.031054+0000","last_active":"2023-09-23T03:27:45.010215+0000","last_peered":"2023-09-23T03:27:45.010215+0000","last_clean":"2023-09-23T03:27:45.010215+0000","last_became_active":"2023-09-23T03:27:44.030699+0000","last_became_peered":"2023-09-23T03:27:44.030699+0000","last_unstale":"2023-09-23T03:27:45.010215+0000","last_undegraded":"2023-09-23T03:27:45.010215+0000","last_fullsized":"2023-09-23T03:27:45.010215+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:38:50.544329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.381722+0000","last_change":"2023-09-23T03:27:45.028306+0000","last_active":"2023-09-23T03:27:47.381722+0000","last_peered":"2023-09-23T03:27:47.381722+0000","last_clean":"2023-09-23T03:27:47.381722+0000","last_became_active":"2023-09-23T03:27:45.027966+0000","last_became_peered":"2023-09-23T03:27:45.027966+0000","last_unstale":"2023-09-23T03:27:47.381722+0000","last_undegraded":"2023-09-23T03:27:47.381722+0000","last_fullsized":"2023-09-23T03:27:47.381722+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:32:38.141959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.021373+0000","last_change":"2023-09-23T03:27:44.031597+0000","last_active":"2023-09-23T03:27:45.021373+0000","last_peered":"2023-09-23T03:27:45.021373+0000","last_clean":"2023-09-23T03:27:45.021373+0000","last_became_active":"2023-09-23T03:27:44.030611+0000","last_became_peered":"2023-09-23T03:27:44.030611+0000","last_unstale":"2023-09-23T03:27:45.021373+0000","last_undegraded":"2023-09-23T03:27:45.021373+0000","last_fullsized":"2023-09-23T03:27:45.021373+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:14:09.639441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.384022+0000","last_change":"2023-09-23T03:27:45.046861+0000","last_active":"2023-09-23T03:27:47.384022+0000","last_peered":"2023-09-23T03:27:47.384022+0000","last_clean":"2023-09-23T03:27:47.384022+0000","last_became_active":"2023-09-23T03:27:45.046591+0000","last_became_peered":"2023-09-23T03:27:45.046591+0000","last_unstale":"2023-09-23T03:27:47.384022+0000","last_undegraded":"2023-09-23T03:27:47.384022+0000","last_fullsized":"2023-09-23T03:27:47.384022+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:26:00.555159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"125'3","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.050842+0000","last_change":"2023-09-23T03:27:44.028308+0000","last_active":"2023-09-23T03:27:45.050842+0000","last_peered":"2023-09-23T03:27:45.050842+0000","last_clean":"2023-09-23T03:27:45.050842+0000","last_became_active":"2023-09-23T03:27:44.026049+0000","last_became_peered":"2023-09-23T03:27:44.026049+0000","last_unstale":"2023-09-23T03:27:45.050842+0000","last_undegraded":"2023-09-23T03:27:45.050842+0000","last_fullsized":"2023-09-23T03:27:45.050842+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:38:50.544329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.382391+0000","last_change":"2023-09-23T03:27:45.032354+0000","last_active":"2023-09-23T03:27:47.382391+0000","last_peered":"2023-09-23T03:27:47.382391+0000","last_clean":"2023-09-23T03:27:47.382391+0000","last_became_active":"2023-09-23T03:27:45.032076+0000","last_became_peered":"2023-09-23T03:27:45.032076+0000","last_unstale":"2023-09-23T03:27:47.382391+0000","last_undegraded":"2023-09-23T03:27:47.382391+0000","last_fullsized":"2023-09-23T03:27:47.382391+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:28:20.892702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"125'6","reported_seq":18,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.050048+0000","last_change":"2023-09-23T03:27:44.030573+0000","last_active":"2023-09-23T03:27:45.050048+0000","last_peered":"2023-09-23T03:27:45.050048+0000","last_clean":"2023-09-23T03:27:45.050048+0000","last_became_active":"2023-09-23T03:27:44.030296+0000","last_became_peered":"2023-09-23T03:27:44.030296+0000","last_unstale":"2023-09-23T03:27:45.050048+0000","last_undegraded":"2023-09-23T03:27:45.050048+0000","last_fullsized":"2023-09-23T03:27:45.050048+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:42:53.815632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:46.035459+0000","last_change":"2023-09-23T03:27:45.029851+0000","last_active":"2023-09-23T03:27:46.035459+0000","last_peered":"2023-09-23T03:27:46.035459+0000","last_clean":"2023-09-23T03:27:46.035459+0000","last_became_active":"2023-09-23T03:27:45.029562+0000","last_became_peered":"2023-09-23T03:27:45.029562+0000","last_unstale":"2023-09-23T03:27:46.035459+0000","last_undegraded":"2023-09-23T03:27:46.035459+0000","last_fullsized":"2023-09-23T03:27:46.035459+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:46:09.395570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"125'2","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.051392+0000","last_change":"2023-09-23T03:27:44.034874+0000","last_active":"2023-09-23T03:27:45.051392+0000","last_peered":"2023-09-23T03:27:45.051392+0000","last_clean":"2023-09-23T03:27:45.051392+0000","last_became_active":"2023-09-23T03:27:44.034585+0000","last_became_peered":"2023-09-23T03:27:44.034585+0000","last_unstale":"2023-09-23T03:27:45.051392+0000","last_undegraded":"2023-09-23T03:27:45.051392+0000","last_fullsized":"2023-09-23T03:27:45.051392+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:40:35.511461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.382427+0000","last_change":"2023-09-23T03:27:45.038478+0000","last_active":"2023-09-23T03:27:47.382427+0000","last_peered":"2023-09-23T03:27:47.382427+0000","last_clean":"2023-09-23T03:27:47.382427+0000","last_became_active":"2023-09-23T03:27:45.038185+0000","last_became_peered":"2023-09-23T03:27:45.038185+0000","last_unstale":"2023-09-23T03:27:47.382427+0000","last_undegraded":"2023-09-23T03:27:47.382427+0000","last_fullsized":"2023-09-23T03:27:47.382427+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:56:03.506431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.018205+0000","last_change":"2023-09-23T03:27:44.032000+0000","last_active":"2023-09-23T03:27:45.018205+0000","last_peered":"2023-09-23T03:27:45.018205+0000","last_clean":"2023-09-23T03:27:45.018205+0000","last_became_active":"2023-09-23T03:27:44.031230+0000","last_became_peered":"2023-09-23T03:27:44.031230+0000","last_unstale":"2023-09-23T03:27:45.018205+0000","last_undegraded":"2023-09-23T03:27:45.018205+0000","last_fullsized":"2023-09-23T03:27:45.018205+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:48:36.568201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:46.032158+0000","last_change":"2023-09-23T03:27:45.021451+0000","last_active":"2023-09-23T03:27:46.032158+0000","last_peered":"2023-09-23T03:27:46.032158+0000","last_clean":"2023-09-23T03:27:46.032158+0000","last_became_active":"2023-09-23T03:27:45.021179+0000","last_became_peered":"2023-09-23T03:27:45.021179+0000","last_unstale":"2023-09-23T03:27:46.032158+0000","last_undegraded":"2023-09-23T03:27:46.032158+0000","last_fullsized":"2023-09-23T03:27:46.032158+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:02:06.668404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.016083+0000","last_change":"2023-09-23T03:27:44.032754+0000","last_active":"2023-09-23T03:27:45.016083+0000","last_peered":"2023-09-23T03:27:45.016083+0000","last_clean":"2023-09-23T03:27:45.016083+0000","last_became_active":"2023-09-23T03:27:44.032391+0000","last_became_peered":"2023-09-23T03:27:44.032391+0000","last_unstale":"2023-09-23T03:27:45.016083+0000","last_undegraded":"2023-09-23T03:27:45.016083+0000","last_fullsized":"2023-09-23T03:27:45.016083+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:34:44.915726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.381543+0000","last_change":"2023-09-23T03:27:45.042388+0000","last_active":"2023-09-23T03:27:47.381543+0000","last_peered":"2023-09-23T03:27:47.381543+0000","last_clean":"2023-09-23T03:27:47.381543+0000","last_became_active":"2023-09-23T03:27:45.041630+0000","last_became_peered":"2023-09-23T03:27:45.041630+0000","last_unstale":"2023-09-23T03:27:47.381543+0000","last_undegraded":"2023-09-23T03:27:47.381543+0000","last_fullsized":"2023-09-23T03:27:47.381543+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:30:18.768868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"125'1","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.042729+0000","last_change":"2023-09-23T03:27:44.042829+0000","last_active":"2023-09-23T03:27:45.042729+0000","last_peered":"2023-09-23T03:27:45.042729+0000","last_clean":"2023-09-23T03:27:45.042729+0000","last_became_active":"2023-09-23T03:27:44.042547+0000","last_became_peered":"2023-09-23T03:27:44.042547+0000","last_unstale":"2023-09-23T03:27:45.042729+0000","last_undegraded":"2023-09-23T03:27:45.042729+0000","last_fullsized":"2023-09-23T03:27:45.042729+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:53:31.135007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.382446+0000","last_change":"2023-09-23T03:27:45.026059+0000","last_active":"2023-09-23T03:27:47.382446+0000","last_peered":"2023-09-23T03:27:47.382446+0000","last_clean":"2023-09-23T03:27:47.382446+0000","last_became_active":"2023-09-23T03:27:45.025620+0000","last_became_peered":"2023-09-23T03:27:45.025620+0000","last_unstale":"2023-09-23T03:27:47.382446+0000","last_undegraded":"2023-09-23T03:27:47.382446+0000","last_fullsized":"2023-09-23T03:27:47.382446+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:07:46.010795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"125'2","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.045073+0000","last_change":"2023-09-23T03:27:44.028974+0000","last_active":"2023-09-23T03:27:45.045073+0000","last_peered":"2023-09-23T03:27:45.045073+0000","last_clean":"2023-09-23T03:27:45.045073+0000","last_became_active":"2023-09-23T03:27:44.028112+0000","last_became_peered":"2023-09-23T03:27:44.028112+0000","last_unstale":"2023-09-23T03:27:45.045073+0000","last_undegraded":"2023-09-23T03:27:45.045073+0000","last_fullsized":"2023-09-23T03:27:45.045073+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:53:31.135007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.381049+0000","last_change":"2023-09-23T03:27:45.031971+0000","last_active":"2023-09-23T03:27:47.381049+0000","last_peered":"2023-09-23T03:27:47.381049+0000","last_clean":"2023-09-23T03:27:47.381049+0000","last_became_active":"2023-09-23T03:27:45.031685+0000","last_became_peered":"2023-09-23T03:27:45.031685+0000","last_unstale":"2023-09-23T03:27:47.381049+0000","last_undegraded":"2023-09-23T03:27:47.381049+0000","last_fullsized":"2023-09-23T03:27:47.381049+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:20:18.634153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"125'1","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.044083+0000","last_change":"2023-09-23T03:27:44.032617+0000","last_active":"2023-09-23T03:27:45.044083+0000","last_peered":"2023-09-23T03:27:45.044083+0000","last_clean":"2023-09-23T03:27:45.044083+0000","last_became_active":"2023-09-23T03:27:44.031990+0000","last_became_peered":"2023-09-23T03:27:44.031990+0000","last_unstale":"2023-09-23T03:27:45.044083+0000","last_undegraded":"2023-09-23T03:27:45.044083+0000","last_fullsized":"2023-09-23T03:27:45.044083+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:00:14.855143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.382625+0000","last_change":"2023-09-23T03:27:45.038013+0000","last_active":"2023-09-23T03:27:47.382625+0000","last_peered":"2023-09-23T03:27:47.382625+0000","last_clean":"2023-09-23T03:27:47.382625+0000","last_became_active":"2023-09-23T03:27:45.037340+0000","last_became_peered":"2023-09-23T03:27:45.037340+0000","last_unstale":"2023-09-23T03:27:47.382625+0000","last_undegraded":"2023-09-23T03:27:47.382625+0000","last_fullsized":"2023-09-23T03:27:47.382625+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:18:20.232803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.027210+0000","last_change":"2023-09-23T03:27:44.036175+0000","last_active":"2023-09-23T03:27:45.027210+0000","last_peered":"2023-09-23T03:27:45.027210+0000","last_clean":"2023-09-23T03:27:45.027210+0000","last_became_active":"2023-09-23T03:27:44.035896+0000","last_became_peered":"2023-09-23T03:27:44.035896+0000","last_unstale":"2023-09-23T03:27:45.027210+0000","last_undegraded":"2023-09-23T03:27:45.027210+0000","last_fullsized":"2023-09-23T03:27:45.027210+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:48:36.568201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.380498+0000","last_change":"2023-09-23T03:27:45.023094+0000","last_active":"2023-09-23T03:27:47.380498+0000","last_peered":"2023-09-23T03:27:47.380498+0000","last_clean":"2023-09-23T03:27:47.380498+0000","last_became_active":"2023-09-23T03:27:45.019872+0000","last_became_peered":"2023-09-23T03:27:45.019872+0000","last_unstale":"2023-09-23T03:27:47.380498+0000","last_undegraded":"2023-09-23T03:27:47.380498+0000","last_fullsized":"2023-09-23T03:27:47.380498+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:07:16.989269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"125'2","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.050347+0000","last_change":"2023-09-23T03:27:44.027098+0000","last_active":"2023-09-23T03:27:45.050347+0000","last_peered":"2023-09-23T03:27:45.050347+0000","last_clean":"2023-09-23T03:27:45.050347+0000","last_became_active":"2023-09-23T03:27:44.026793+0000","last_became_peered":"2023-09-23T03:27:44.026793+0000","last_unstale":"2023-09-23T03:27:45.050347+0000","last_undegraded":"2023-09-23T03:27:45.050347+0000","last_fullsized":"2023-09-23T03:27:45.050347+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:08:30.847042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.382067+0000","last_change":"2023-09-23T03:27:45.031167+0000","last_active":"2023-09-23T03:27:47.382067+0000","last_peered":"2023-09-23T03:27:47.382067+0000","last_clean":"2023-09-23T03:27:47.382067+0000","last_became_active":"2023-09-23T03:27:45.030906+0000","last_became_peered":"2023-09-23T03:27:45.030906+0000","last_unstale":"2023-09-23T03:27:47.382067+0000","last_undegraded":"2023-09-23T03:27:47.382067+0000","last_fullsized":"2023-09-23T03:27:47.382067+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:59:59.766177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.015945+0000","last_change":"2023-09-23T03:27:44.038407+0000","last_active":"2023-09-23T03:27:45.015945+0000","last_peered":"2023-09-23T03:27:45.015945+0000","last_clean":"2023-09-23T03:27:45.015945+0000","last_became_active":"2023-09-23T03:27:44.038137+0000","last_became_peered":"2023-09-23T03:27:44.038137+0000","last_unstale":"2023-09-23T03:27:45.015945+0000","last_undegraded":"2023-09-23T03:27:45.015945+0000","last_fullsized":"2023-09-23T03:27:45.015945+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:14:43.859672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.382389+0000","last_change":"2023-09-23T03:27:45.040671+0000","last_active":"2023-09-23T03:27:47.382389+0000","last_peered":"2023-09-23T03:27:47.382389+0000","last_clean":"2023-09-23T03:27:47.382389+0000","last_became_active":"2023-09-23T03:27:45.040373+0000","last_became_peered":"2023-09-23T03:27:45.040373+0000","last_unstale":"2023-09-23T03:27:47.382389+0000","last_undegraded":"2023-09-23T03:27:47.382389+0000","last_fullsized":"2023-09-23T03:27:47.382389+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:51:53.201001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.016301+0000","last_change":"2023-09-23T03:27:44.028929+0000","last_active":"2023-09-23T03:27:45.016301+0000","last_peered":"2023-09-23T03:27:45.016301+0000","last_clean":"2023-09-23T03:27:45.016301+0000","last_became_active":"2023-09-23T03:27:44.028642+0000","last_became_peered":"2023-09-23T03:27:44.028642+0000","last_unstale":"2023-09-23T03:27:45.016301+0000","last_undegraded":"2023-09-23T03:27:45.016301+0000","last_fullsized":"2023-09-23T03:27:45.016301+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:06:02.129656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.381182+0000","last_change":"2023-09-23T03:27:45.031216+0000","last_active":"2023-09-23T03:27:47.381182+0000","last_peered":"2023-09-23T03:27:47.381182+0000","last_clean":"2023-09-23T03:27:47.381182+0000","last_became_active":"2023-09-23T03:27:45.030947+0000","last_became_peered":"2023-09-23T03:27:45.030947+0000","last_unstale":"2023-09-23T03:27:47.381182+0000","last_undegraded":"2023-09-23T03:27:47.381182+0000","last_fullsized":"2023-09-23T03:27:47.381182+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:16:05.766908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.019142+0000","last_change":"2023-09-23T03:27:44.036596+0000","last_active":"2023-09-23T03:27:45.019142+0000","last_peered":"2023-09-23T03:27:45.019142+0000","last_clean":"2023-09-23T03:27:45.019142+0000","last_became_active":"2023-09-23T03:27:44.035882+0000","last_became_peered":"2023-09-23T03:27:44.035882+0000","last_unstale":"2023-09-23T03:27:45.019142+0000","last_undegraded":"2023-09-23T03:27:45.019142+0000","last_fullsized":"2023-09-23T03:27:45.019142+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:26:31.860786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.383264+0000","last_change":"2023-09-23T03:27:45.030275+0000","last_active":"2023-09-23T03:27:47.383264+0000","last_peered":"2023-09-23T03:27:47.383264+0000","last_clean":"2023-09-23T03:27:47.383264+0000","last_became_active":"2023-09-23T03:27:45.030016+0000","last_became_peered":"2023-09-23T03:27:45.030016+0000","last_unstale":"2023-09-23T03:27:47.383264+0000","last_undegraded":"2023-09-23T03:27:47.383264+0000","last_fullsized":"2023-09-23T03:27:47.383264+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:13:09.353030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"125'2","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.045622+0000","last_change":"2023-09-23T03:27:44.037493+0000","last_active":"2023-09-23T03:27:45.045622+0000","last_peered":"2023-09-23T03:27:45.045622+0000","last_clean":"2023-09-23T03:27:45.045622+0000","last_became_active":"2023-09-23T03:27:44.037196+0000","last_became_peered":"2023-09-23T03:27:44.037196+0000","last_unstale":"2023-09-23T03:27:45.045622+0000","last_undegraded":"2023-09-23T03:27:45.045622+0000","last_fullsized":"2023-09-23T03:27:45.045622+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:53:45.660506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.382970+0000","last_change":"2023-09-23T03:27:45.034757+0000","last_active":"2023-09-23T03:27:47.382970+0000","last_peered":"2023-09-23T03:27:47.382970+0000","last_clean":"2023-09-23T03:27:47.382970+0000","last_became_active":"2023-09-23T03:27:45.034484+0000","last_became_peered":"2023-09-23T03:27:45.034484+0000","last_unstale":"2023-09-23T03:27:47.382970+0000","last_undegraded":"2023-09-23T03:27:47.382970+0000","last_fullsized":"2023-09-23T03:27:47.382970+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:45:05.020375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.009841+0000","last_change":"2023-09-23T03:27:44.031734+0000","last_active":"2023-09-23T03:27:45.009841+0000","last_peered":"2023-09-23T03:27:45.009841+0000","last_clean":"2023-09-23T03:27:45.009841+0000","last_became_active":"2023-09-23T03:27:44.031447+0000","last_became_peered":"2023-09-23T03:27:44.031447+0000","last_unstale":"2023-09-23T03:27:45.009841+0000","last_undegraded":"2023-09-23T03:27:45.009841+0000","last_fullsized":"2023-09-23T03:27:45.009841+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:14:09.639441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:46.033335+0000","last_change":"2023-09-23T03:27:45.041957+0000","last_active":"2023-09-23T03:27:46.033335+0000","last_peered":"2023-09-23T03:27:46.033335+0000","last_clean":"2023-09-23T03:27:46.033335+0000","last_became_active":"2023-09-23T03:27:45.041690+0000","last_became_peered":"2023-09-23T03:27:45.041690+0000","last_unstale":"2023-09-23T03:27:46.033335+0000","last_undegraded":"2023-09-23T03:27:46.033335+0000","last_fullsized":"2023-09-23T03:27:46.033335+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:40:45.625728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.010370+0000","last_change":"2023-09-23T03:27:44.032549+0000","last_active":"2023-09-23T03:27:45.010370+0000","last_peered":"2023-09-23T03:27:45.010370+0000","last_clean":"2023-09-23T03:27:45.010370+0000","last_became_active":"2023-09-23T03:27:44.032241+0000","last_became_peered":"2023-09-23T03:27:44.032241+0000","last_unstale":"2023-09-23T03:27:45.010370+0000","last_undegraded":"2023-09-23T03:27:45.010370+0000","last_fullsized":"2023-09-23T03:27:45.010370+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:19:10.042252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:46.032658+0000","last_change":"2023-09-23T03:27:45.029784+0000","last_active":"2023-09-23T03:27:46.032658+0000","last_peered":"2023-09-23T03:27:46.032658+0000","last_clean":"2023-09-23T03:27:46.032658+0000","last_became_active":"2023-09-23T03:27:45.029468+0000","last_became_peered":"2023-09-23T03:27:45.029468+0000","last_unstale":"2023-09-23T03:27:46.032658+0000","last_undegraded":"2023-09-23T03:27:46.032658+0000","last_fullsized":"2023-09-23T03:27:46.032658+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:48:13.711973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.012979+0000","last_change":"2023-09-23T03:27:44.039309+0000","last_active":"2023-09-23T03:27:45.012979+0000","last_peered":"2023-09-23T03:27:45.012979+0000","last_clean":"2023-09-23T03:27:45.012979+0000","last_became_active":"2023-09-23T03:27:44.039035+0000","last_became_peered":"2023-09-23T03:27:44.039035+0000","last_unstale":"2023-09-23T03:27:45.012979+0000","last_undegraded":"2023-09-23T03:27:45.012979+0000","last_fullsized":"2023-09-23T03:27:45.012979+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:40:41.002962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.380647+0000","last_change":"2023-09-23T03:27:45.023059+0000","last_active":"2023-09-23T03:27:47.380647+0000","last_peered":"2023-09-23T03:27:47.380647+0000","last_clean":"2023-09-23T03:27:47.380647+0000","last_became_active":"2023-09-23T03:27:45.021471+0000","last_became_peered":"2023-09-23T03:27:45.021471+0000","last_unstale":"2023-09-23T03:27:47.380647+0000","last_undegraded":"2023-09-23T03:27:47.380647+0000","last_fullsized":"2023-09-23T03:27:47.380647+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:37:39.211653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.012969+0000","last_change":"2023-09-23T03:27:44.032738+0000","last_active":"2023-09-23T03:27:45.012969+0000","last_peered":"2023-09-23T03:27:45.012969+0000","last_clean":"2023-09-23T03:27:45.012969+0000","last_became_active":"2023-09-23T03:27:44.032424+0000","last_became_peered":"2023-09-23T03:27:44.032424+0000","last_unstale":"2023-09-23T03:27:45.012969+0000","last_undegraded":"2023-09-23T03:27:45.012969+0000","last_fullsized":"2023-09-23T03:27:45.012969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:24:07.607733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.382411+0000","last_change":"2023-09-23T03:27:45.043661+0000","last_active":"2023-09-23T03:27:47.382411+0000","last_peered":"2023-09-23T03:27:47.382411+0000","last_clean":"2023-09-23T03:27:47.382411+0000","last_became_active":"2023-09-23T03:27:45.043378+0000","last_became_peered":"2023-09-23T03:27:45.043378+0000","last_unstale":"2023-09-23T03:27:47.382411+0000","last_undegraded":"2023-09-23T03:27:47.382411+0000","last_fullsized":"2023-09-23T03:27:47.382411+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:07:35.140412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.014929+0000","last_change":"2023-09-23T03:27:44.034445+0000","last_active":"2023-09-23T03:27:45.014929+0000","last_peered":"2023-09-23T03:27:45.014929+0000","last_clean":"2023-09-23T03:27:45.014929+0000","last_became_active":"2023-09-23T03:27:44.033651+0000","last_became_peered":"2023-09-23T03:27:44.033651+0000","last_unstale":"2023-09-23T03:27:45.014929+0000","last_undegraded":"2023-09-23T03:27:45.014929+0000","last_fullsized":"2023-09-23T03:27:45.014929+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:48:12.700185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:46.032713+0000","last_change":"2023-09-23T03:27:45.035388+0000","last_active":"2023-09-23T03:27:46.032713+0000","last_peered":"2023-09-23T03:27:46.032713+0000","last_clean":"2023-09-23T03:27:46.032713+0000","last_became_active":"2023-09-23T03:27:45.035126+0000","last_became_peered":"2023-09-23T03:27:45.035126+0000","last_unstale":"2023-09-23T03:27:46.032713+0000","last_undegraded":"2023-09-23T03:27:46.032713+0000","last_fullsized":"2023-09-23T03:27:46.032713+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:11:36.520222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.009779+0000","last_change":"2023-09-23T03:27:44.038626+0000","last_active":"2023-09-23T03:27:45.009779+0000","last_peered":"2023-09-23T03:27:45.009779+0000","last_clean":"2023-09-23T03:27:45.009779+0000","last_became_active":"2023-09-23T03:27:44.038353+0000","last_became_peered":"2023-09-23T03:27:44.038353+0000","last_unstale":"2023-09-23T03:27:45.009779+0000","last_undegraded":"2023-09-23T03:27:45.009779+0000","last_fullsized":"2023-09-23T03:27:45.009779+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:06:02.927861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.382719+0000","last_change":"2023-09-23T03:27:45.038565+0000","last_active":"2023-09-23T03:27:47.382719+0000","last_peered":"2023-09-23T03:27:47.382719+0000","last_clean":"2023-09-23T03:27:47.382719+0000","last_became_active":"2023-09-23T03:27:45.038247+0000","last_became_peered":"2023-09-23T03:27:45.038247+0000","last_unstale":"2023-09-23T03:27:47.382719+0000","last_undegraded":"2023-09-23T03:27:47.382719+0000","last_fullsized":"2023-09-23T03:27:47.382719+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:58:00.149687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.019286+0000","last_change":"2023-09-23T03:27:44.036392+0000","last_active":"2023-09-23T03:27:45.019286+0000","last_peered":"2023-09-23T03:27:45.019286+0000","last_clean":"2023-09-23T03:27:45.019286+0000","last_became_active":"2023-09-23T03:27:44.035456+0000","last_became_peered":"2023-09-23T03:27:44.035456+0000","last_unstale":"2023-09-23T03:27:45.019286+0000","last_undegraded":"2023-09-23T03:27:45.019286+0000","last_fullsized":"2023-09-23T03:27:45.019286+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:43:59.744799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:46.033738+0000","last_change":"2023-09-23T03:27:45.043461+0000","last_active":"2023-09-23T03:27:46.033738+0000","last_peered":"2023-09-23T03:27:46.033738+0000","last_clean":"2023-09-23T03:27:46.033738+0000","last_became_active":"2023-09-23T03:27:45.043112+0000","last_became_peered":"2023-09-23T03:27:45.043112+0000","last_unstale":"2023-09-23T03:27:46.033738+0000","last_undegraded":"2023-09-23T03:27:46.033738+0000","last_fullsized":"2023-09-23T03:27:46.033738+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:59:59.766177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"125'8","reported_seq":20,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.051953+0000","last_change":"2023-09-23T03:27:44.034373+0000","last_active":"2023-09-23T03:27:45.051953+0000","last_peered":"2023-09-23T03:27:45.051953+0000","last_clean":"2023-09-23T03:27:45.051953+0000","last_became_active":"2023-09-23T03:27:44.033040+0000","last_became_peered":"2023-09-23T03:27:44.033040+0000","last_unstale":"2023-09-23T03:27:45.051953+0000","last_undegraded":"2023-09-23T03:27:45.051953+0000","last_fullsized":"2023-09-23T03:27:45.051953+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:00:14.855143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":257,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.019485+0000","last_change":"2023-09-23T03:15:11.042447+0000","last_active":"2023-09-23T03:27:45.019485+0000","last_peered":"2023-09-23T03:27:45.019485+0000","last_clean":"2023-09-23T03:27:45.019485+0000","last_became_active":"2023-09-23T03:15:11.040322+0000","last_became_peered":"2023-09-23T03:15:11.040322+0000","last_unstale":"2023-09-23T03:27:45.019485+0000","last_undegraded":"2023-09-23T03:27:45.019485+0000","last_fullsized":"2023-09-23T03:27:45.019485+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_clean_scrub_stamp":"2023-09-23T03:15:09.474842+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:18:36.331645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575002,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33312,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338528,"statfs":{"total":96636764160,"available":96602652672,"internally_reserved":0,"allocated":2179072,"data_stored":1357674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:27:33 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.0509999999999999,"15min":1.022},"min":{"1min":0.81699999999999995,"5min":0.57999999999999996,"15min":0.442},"max":{"1min":1.736,"5min":2.0299999999999998,"15min":2.149},"last":1.5880000000000001},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.0720000000000001,"15min":1.0620000000000001},"min":{"1min":0.74199999999999999,"5min":0.63100000000000001,"15min":0.52000000000000002},"max":{"1min":1.617,"5min":1.9910000000000001,"15min":2.1480000000000001},"last":0.91800000000000004}]},{"osd":1,"last update":"Sat Sep 23 03:27:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.214,"15min":1.214},"min":{"1min":0.80600000000000005,"5min":0.66400000000000003,"15min":0.504},"max":{"1min":1.478,"5min":2.6400000000000001,"15min":3.1859999999999999},"last":1.524},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.2729999999999999,"15min":1.294},"min":{"1min":0.79900000000000004,"5min":0.71299999999999997,"15min":0.67900000000000005},"max":{"1min":1.8089999999999999,"5min":2.3149999999999999,"15min":2.613},"last":1.1919999999999999}]},{"osd":2,"last update":"Sat Sep 23 03:27:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.80400000000000005,"5min":0.61199999999999999,"15min":0.42399999999999999},"max":{"1min":1.516,"5min":1.6439999999999999,"15min":2.331},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.248,"5min":1.2509999999999999,"15min":1.2410000000000001},"min":{"1min":1.004,"5min":0.749,"15min":0.55200000000000005},"max":{"1min":1.651,"5min":2.2719999999999998,"15min":3.093},"last":1.2490000000000001}]},{"osd":3,"last update":"Sat Sep 23 03:27:33 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.236,"15min":1.2450000000000001},"min":{"1min":0.89000000000000001,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.464,"5min":2.0840000000000001,"15min":2.532},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.93799999999999994,"5min":0.71899999999999997,"15min":0.69799999999999995},"max":{"1min":1.8089999999999999,"5min":2.4289999999999998,"15min":3.2349999999999999},"last":0.99299999999999999}]},{"osd":4,"last update":"Sat Sep 23 03:27:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.292,"15min":1.375},"min":{"1min":0.82599999999999996,"5min":0.57999999999999996,"15min":0.45400000000000001},"max":{"1min":1.605,"5min":2.3260000000000001,"15min":2.7210000000000001},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.4490000000000001,"15min":1.476},"min":{"1min":1.0880000000000001,"5min":0.73599999999999999,"15min":0.48099999999999998},"max":{"1min":1.75,"5min":2.2240000000000002,"15min":3.7719999999999998},"last":1.478}]},{"osd":5,"last update":"Sat Sep 23 03:27:33 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.3959999999999999,"15min":1.4139999999999999},"min":{"1min":0.96999999999999997,"5min":0.749,"15min":0.47999999999999998},"max":{"1min":1.871,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":1.409},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.413,"15min":1.4450000000000001},"min":{"1min":0.95099999999999996,"5min":0.77000000000000002,"15min":0.497},"max":{"1min":1.74,"5min":2.5070000000000001,"15min":2.5070000000000001},"last":1.3089999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:27:33 2023","interfaces":[{"interface":"back","average":{"1min":1.516,"5min":1.5049999999999999,"15min":1.5469999999999999},"min":{"1min":1.1850000000000001,"5min":0.85499999999999998,"15min":0.53800000000000003},"max":{"1min":1.835,"5min":2.649,"15min":2.8420000000000001},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.383,"5min":1.4299999999999999,"15min":1.484},"min":{"1min":0.92100000000000004,"5min":0.78900000000000003,"15min":0.45900000000000002},"max":{"1min":1.667,"5min":2.4500000000000002,"15min":3.6739999999999999},"last":1.1319999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575002,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32720,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1572864,"data_stored":767112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:27:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.107,"15min":1.0840000000000001},"min":{"1min":0.66500000000000004,"5min":0.53600000000000003,"15min":0.38},"max":{"1min":1.7789999999999999,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.3360000000000001},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.091,"15min":1.0720000000000001},"min":{"1min":0.68999999999999995,"5min":0.60999999999999999,"15min":0.46100000000000002},"max":{"1min":1.403,"5min":3.7629999999999999,"15min":3.7629999999999999},"last":0.70099999999999996}]},{"osd":1,"last update":"Sat Sep 23 03:27:25 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.1359999999999999,"15min":1.119},"min":{"1min":0.66500000000000004,"5min":0.63800000000000001,"15min":0.438},"max":{"1min":1.4910000000000001,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.242,"5min":1.1659999999999999,"15min":1.1910000000000001},"min":{"1min":0.70499999999999996,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.593,"5min":2.972,"15min":2.972},"last":1.7350000000000001}]},{"osd":2,"last update":"Sat Sep 23 03:27:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.1579999999999999,"15min":1.1319999999999999},"min":{"1min":0.80100000000000005,"5min":0.56499999999999995,"15min":0.51300000000000001},"max":{"1min":1.4810000000000001,"5min":3.4390000000000001,"15min":3.4390000000000001},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.214,"5min":1.246,"15min":1.2390000000000001},"min":{"1min":0.80700000000000005,"5min":0.68000000000000005,"15min":0.64300000000000002},"max":{"1min":1.5549999999999999,"5min":2.177,"15min":2.177},"last":0.97399999999999998}]},{"osd":3,"last update":"Sat Sep 23 03:27:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.355,"15min":1.337},"min":{"1min":1.01,"5min":0.78500000000000003,"15min":0.54600000000000004},"max":{"1min":1.583,"5min":2.129,"15min":2.6509999999999998},"last":1.6890000000000001},{"interface":"front","average":{"1min":1.294,"5min":1.327,"15min":1.373},"min":{"1min":0.98499999999999999,"5min":0.86199999999999999,"15min":0.74299999999999999},"max":{"1min":1.7190000000000001,"5min":3.0910000000000002,"15min":3.0910000000000002},"last":1.5620000000000001}]},{"osd":4,"last update":"Sat Sep 23 03:27:25 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.3440000000000001,"15min":1.369},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.498},"max":{"1min":1.657,"5min":3.1789999999999998,"15min":3.1789999999999998},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.226,"5min":1.3240000000000001,"15min":1.2989999999999999},"min":{"1min":0.76000000000000001,"5min":0.72399999999999998,"15min":0.56000000000000005},"max":{"1min":1.6200000000000001,"5min":3.6480000000000001,"15min":3.6480000000000001},"last":1.3959999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:27:25 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.429,"15min":1.4099999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6739999999999999,"5min":4.093,"15min":4.093},"last":1.639},{"interface":"front","average":{"1min":1.391,"5min":1.45,"15min":1.4450000000000001},"min":{"1min":0.70599999999999996,"5min":0.69899999999999995,"15min":0.59899999999999998},"max":{"1min":1.669,"5min":2.0459999999999998,"15min":4.758},"last":1.472}]},{"osd":7,"last update":"Sat Sep 23 03:27:25 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.5189999999999999,"15min":1.52},"min":{"1min":0.81100000000000005,"5min":0.76500000000000001,"15min":0.63500000000000001},"max":{"1min":1.712,"5min":3.536,"15min":3.536},"last":1.9570000000000001},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.546,"15min":1.536},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.8720000000000001,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":1.788}]}]},{"osd":5,"up_from":13,"seq":55834575004,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":766690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:27:26 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.054,"15min":1.046},"min":{"1min":0.66000000000000003,"5min":0.61199999999999999,"15min":0.57199999999999995},"max":{"1min":1.5860000000000001,"5min":2.9740000000000002,"15min":2.9740000000000002},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.171,"5min":1.131,"15min":1.081},"min":{"1min":0.73299999999999998,"5min":0.624,"15min":0.61399999999999999},"max":{"1min":1.619,"5min":1.962,"15min":2.2320000000000002},"last":1.383}]},{"osd":1,"last update":"Sat Sep 23 03:27:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.21,"15min":1.2210000000000001},"min":{"1min":0.84799999999999998,"5min":0.68300000000000005,"15min":0.62},"max":{"1min":1.5569999999999999,"5min":2.4369999999999998,"15min":2.4369999999999998},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.274,"5min":1.2869999999999999,"15min":1.288},"min":{"1min":0.76800000000000002,"5min":0.68100000000000005,"15min":0.67200000000000004},"max":{"1min":1.5860000000000001,"5min":3.5859999999999999,"15min":3.5859999999999999},"last":1.1990000000000001}]},{"osd":2,"last update":"Sat Sep 23 03:27:26 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.196,"15min":1.2030000000000001},"min":{"1min":0.92800000000000005,"5min":0.65300000000000002,"15min":0.64300000000000002},"max":{"1min":1.593,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":1.129},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.1399999999999999,"15min":1.218},"min":{"1min":0.79500000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.6970000000000001,"5min":2.8500000000000001,"15min":2.8500000000000001},"last":1.0489999999999999}]},{"osd":3,"last update":"Sat Sep 23 03:27:26 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.266,"15min":1.2669999999999999},"min":{"1min":0.96999999999999997,"5min":0.67200000000000004,"15min":0.66400000000000003},"max":{"1min":1.5640000000000001,"5min":3.4359999999999999,"15min":3.4359999999999999},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.3029999999999999,"15min":1.3580000000000001},"min":{"1min":0.98699999999999999,"5min":0.85999999999999999,"15min":0.69799999999999995},"max":{"1min":1.726,"5min":1.9159999999999999,"15min":3.6110000000000002},"last":1.2629999999999999}]},{"osd":4,"last update":"Sat Sep 23 03:27:26 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.401,"15min":1.395},"min":{"1min":1.0429999999999999,"5min":0.68600000000000005,"15min":0.54100000000000004},"max":{"1min":1.7789999999999999,"5min":4.2750000000000004,"15min":4.2750000000000004},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.2929999999999999,"15min":1.3540000000000001},"min":{"1min":0.79200000000000004,"5min":0.54100000000000004,"15min":0.48899999999999999},"max":{"1min":1.7989999999999999,"5min":3.3090000000000002,"15min":3.3090000000000002},"last":1.3300000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:27:26 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.52,"15min":1.5149999999999999},"min":{"1min":1.272,"5min":1.0329999999999999,"15min":0.59199999999999997},"max":{"1min":1.784,"5min":4.6040000000000001,"15min":4.6040000000000001},"last":1.5389999999999999},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4139999999999999,"15min":1.5249999999999999},"min":{"1min":1.0700000000000001,"5min":0.81200000000000006,"15min":0.55100000000000005},"max":{"1min":1.978,"5min":3.169,"15min":9.6560000000000006},"last":1.2330000000000001}]},{"osd":7,"last update":"Sat Sep 23 03:27:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.431,"15min":1.444},"min":{"1min":0.93899999999999995,"5min":0.77100000000000002,"15min":0.626},"max":{"1min":1.819,"5min":2.077,"15min":2.077},"last":1.304},{"interface":"front","average":{"1min":1.51,"5min":1.494,"15min":1.5509999999999999},"min":{"1min":1.2430000000000001,"5min":0.92200000000000004,"15min":0.60799999999999998},"max":{"1min":1.893,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":1.429}]}]},{"osd":4,"up_from":13,"seq":55834575003,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1581056,"data_stored":768142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:27:23 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0640000000000001,"15min":1.1020000000000001},"min":{"1min":0.58499999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.546,"5min":1.6699999999999999,"15min":1.9770000000000001},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1619999999999999,"15min":1.1699999999999999},"min":{"1min":0.68200000000000005,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.724,"5min":2.004,"15min":2.004},"last":0.871}]},{"osd":1,"last update":"Sat Sep 23 03:27:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2250000000000001,"15min":1.2150000000000001},"min":{"1min":0.69699999999999995,"5min":0.65400000000000003,"15min":0.61799999999999999},"max":{"1min":1.8149999999999999,"5min":1.8200000000000001,"15min":2.1110000000000002},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.139,"5min":1.0840000000000001,"15min":1.137},"min":{"1min":0.71599999999999997,"5min":0.67100000000000004,"15min":0.60299999999999998},"max":{"1min":1.581,"5min":1.718,"15min":3.2189999999999999},"last":1.4610000000000001}]},{"osd":2,"last update":"Sat Sep 23 03:27:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.167,"15min":1.177},"min":{"1min":0.90400000000000003,"5min":0.72899999999999998,"15min":0.61599999999999999},"max":{"1min":1.5880000000000001,"5min":1.694,"15min":2.5920000000000001},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.306,"5min":1.268,"15min":1.248},"min":{"1min":0.89900000000000002,"5min":0.73499999999999999,"15min":0.68300000000000005},"max":{"1min":1.734,"5min":1.99,"15min":1.99},"last":1.2569999999999999}]},{"osd":3,"last update":"Sat Sep 23 03:27:23 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.161,"15min":1.169},"min":{"1min":0.73399999999999999,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.702,"5min":1.702,"15min":1.9450000000000001},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.2549999999999999,"15min":1.294},"min":{"1min":0.82499999999999996,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.7689999999999999,"5min":1.833,"15min":2.4969999999999999},"last":1.3999999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:27:23 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.355,"15min":1.361},"min":{"1min":0.93200000000000005,"5min":0.84899999999999998,"15min":0.70199999999999996},"max":{"1min":2.0070000000000001,"5min":2.0409999999999999,"15min":2.8050000000000002},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.292,"5min":1.3480000000000001,"15min":1.335},"min":{"1min":0.90900000000000003,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":2.887},"last":1.5349999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:27:23 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.3580000000000001,"15min":1.3720000000000001},"min":{"1min":0.93200000000000005,"5min":0.84699999999999998,"15min":0.66400000000000003},"max":{"1min":1.863,"5min":1.9490000000000001,"15min":2.4249999999999998},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.4750000000000001,"15min":1.4910000000000001},"min":{"1min":1.111,"5min":0.91500000000000004,"15min":0.89900000000000002},"max":{"1min":1.7969999999999999,"5min":2.0939999999999999,"15min":2.1779999999999999},"last":1.3560000000000001}]},{"osd":7,"last update":"Sat Sep 23 03:27:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4370000000000001,"15min":1.4590000000000001},"min":{"1min":1.069,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.5680000000000001},"last":1.597},{"interface":"front","average":{"1min":1.514,"5min":1.538,"15min":1.5509999999999999},"min":{"1min":1.268,"5min":0.98899999999999999,"15min":0.84199999999999997},"max":{"1min":1.931,"5min":2.0129999999999999,"15min":2.4769999999999999},"last":1.579}]}]},{"osd":3,"up_from":13,"seq":55834575003,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1585152,"data_stored":768176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.84999999999999998,"15min":0.85799999999999998},"min":{"1min":0.31900000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.121,"5min":1.7050000000000001,"15min":2.952},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.88,"5min":0.89000000000000001,"15min":0.89100000000000001},"min":{"1min":0.54700000000000004,"5min":0.377,"15min":0.29299999999999998},"max":{"1min":1.361,"5min":1.7430000000000001,"15min":1.8979999999999999},"last":0.52700000000000002}]},{"osd":1,"last update":"Sat Sep 23 03:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.81200000000000006,"15min":0.873},"min":{"1min":0.45500000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.359,"5min":1.603,"15min":1.6839999999999999},"last":1.218},{"interface":"front","average":{"1min":0.94299999999999995,"5min":1.002,"15min":0.98799999999999999},"min":{"1min":0.42499999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.5089999999999999,"5min":1.7010000000000001,"15min":2.903},"last":1.482}]},{"osd":2,"last update":"Sat Sep 23 03:27:36 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.022,"15min":1.0509999999999999},"min":{"1min":0.52000000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.669,"5min":1.9850000000000001,"15min":2.2240000000000002},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.099,"5min":1.0860000000000001,"15min":1.103},"min":{"1min":0.61699999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.665,"5min":1.7190000000000001,"15min":3.129},"last":1.0009999999999999}]},{"osd":4,"last update":"Sat Sep 23 03:27:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1519999999999999,"15min":1.218},"min":{"1min":0.72799999999999998,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":2.698},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.2549999999999999,"15min":1.339},"min":{"1min":0.876,"5min":0.81200000000000006,"15min":0.752},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":2.419},"last":1.3979999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:27:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.2889999999999999,"15min":1.329},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.7989999999999999,"5min":2.04,"15min":2.9950000000000001},"last":1.143},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.236,"15min":1.278},"min":{"1min":0.68700000000000006,"5min":0.624,"15min":0.624},"max":{"1min":1.6499999999999999,"5min":1.9830000000000001,"15min":2.0070000000000001},"last":1.1699999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:27:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.3149999999999999,"15min":1.349},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.7190000000000001,"5min":1.9530000000000001,"15min":2.6440000000000001},"last":1.542},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3899999999999999,"15min":1.4279999999999999},"min":{"1min":0.81100000000000005,"5min":0.73299999999999998,"15min":0.67800000000000005},"max":{"1min":1.8700000000000001,"5min":2.177,"15min":2.9750000000000001},"last":1.266}]},{"osd":7,"last update":"Sat Sep 23 03:27:36 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.331,"15min":1.379},"min":{"1min":0.80800000000000005,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.597,"5min":2.0859999999999999,"15min":2.7879999999999998},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.3600000000000001,"15min":1.421},"min":{"1min":0.86299999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.891,"5min":2.0270000000000001,"15min":2.5379999999999998},"last":1.3049999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575004,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32716,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339124,"statfs":{"total":96636764160,"available":96603262976,"internally_reserved":0,"allocated":1568768,"data_stored":766530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.83099999999999996,"15min":0.83899999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.30299999999999999},"max":{"1min":1.4890000000000001,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":1.077,"5min":0.93000000000000005,"15min":0.88600000000000001},"min":{"1min":0.57499999999999996,"5min":0.37,"15min":0.36299999999999999},"max":{"1min":1.901,"5min":2.613,"15min":2.613},"last":0.84299999999999997}]},{"osd":1,"last update":"Sat Sep 23 03:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.585,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":1.234},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.0189999999999999,"15min":0.97899999999999998},"min":{"1min":0.501,"5min":0.501,"15min":0.38},"max":{"1min":1.498,"5min":2.9809999999999999,"15min":2.9809999999999999},"last":1.367}]},{"osd":3,"last update":"Sat Sep 23 03:27:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.0700000000000001,"15min":1.002},"min":{"1min":0.64000000000000001,"5min":0.52100000000000002,"15min":0.40699999999999997},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.119,"5min":1.038,"15min":1.012},"min":{"1min":0.47799999999999998,"5min":0.441,"15min":0.34999999999999998},"max":{"1min":1.7230000000000001,"5min":1.827,"15min":1.827},"last":0.628}]},{"osd":4,"last update":"Sat Sep 23 03:27:25 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.3280000000000001,"15min":1.2989999999999999},"min":{"1min":0.84099999999999997,"5min":0.79400000000000004,"15min":0.55800000000000005},"max":{"1min":1.8260000000000001,"5min":5.6669999999999998,"15min":5.6669999999999998},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.2889999999999999,"15min":1.3},"min":{"1min":0.83899999999999997,"5min":0.82099999999999995,"15min":0.76100000000000001},"max":{"1min":1.6339999999999999,"5min":2.6909999999999998,"15min":2.6909999999999998},"last":1.4910000000000001}]},{"osd":5,"last update":"Sat Sep 23 03:27:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.345,"15min":1.3480000000000001},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.9390000000000001,"5min":2.8969999999999998,"15min":3.9409999999999998},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.403,"15min":1.4199999999999999},"min":{"1min":1.0469999999999999,"5min":0.81299999999999994,"15min":0.56399999999999995},"max":{"1min":1.778,"5min":3.415,"15min":4.9199999999999999},"last":1.2769999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:27:25 2023","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.363,"15min":1.369},"min":{"1min":0.99199999999999999,"5min":0.88,"15min":0.755},"max":{"1min":1.794,"5min":2.7839999999999998,"15min":3.0009999999999999},"last":1.413},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.4730000000000001,"15min":1.4350000000000001},"min":{"1min":0.77100000000000002,"5min":0.76600000000000001,"15min":0.65100000000000002},"max":{"1min":1.839,"5min":7.8300000000000001,"15min":7.8300000000000001},"last":1.4630000000000001}]},{"osd":7,"last update":"Sat Sep 23 03:27:25 2023","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.446,"15min":1.403},"min":{"1min":1.0780000000000001,"5min":0.82899999999999996,"15min":0.61399999999999999},"max":{"1min":1.8180000000000001,"5min":4.0330000000000004,"15min":4.0330000000000004},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.458,"5min":1.4299999999999999,"15min":1.431},"min":{"1min":0.93300000000000005,"5min":0.78600000000000003,"15min":0.74399999999999999},"max":{"1min":1.883,"5min":4.2199999999999998,"15min":4.2199999999999998},"last":1.329}]}]},{"osd":1,"up_from":13,"seq":55834575005,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32716,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339124,"statfs":{"total":96636764160,"available":96603262976,"internally_reserved":0,"allocated":1572864,"data_stored":766552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.80700000000000005,"15min":0.80000000000000004},"min":{"1min":0.56399999999999995,"5min":0.432,"15min":0.32300000000000001},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":2.8279999999999998},"last":0.877},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86399999999999999,"15min":0.84699999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.312},"max":{"1min":1.619,"5min":1.8129999999999999,"15min":1.8919999999999999},"last":1.171}]},{"osd":2,"last update":"Sat Sep 23 03:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.84699999999999998,"15min":0.871},"min":{"1min":0.47799999999999998,"5min":0.46300000000000002,"15min":0.36699999999999999},"max":{"1min":1.2629999999999999,"5min":1.607,"15min":2.3340000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.044,"5min":1.0429999999999999,"15min":1.032},"min":{"1min":0.47599999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5149999999999999,"5min":1.5600000000000001,"15min":1.9570000000000001},"last":1.2749999999999999}]},{"osd":3,"last update":"Sat Sep 23 03:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":1.036,"15min":1.042},"min":{"1min":0.59299999999999997,"5min":0.504,"15min":0.47299999999999998},"max":{"1min":1.325,"5min":1.7250000000000001,"15min":2.2370000000000001},"last":1.379},{"interface":"front","average":{"1min":1.109,"5min":1.121,"15min":1.115},"min":{"1min":0.54400000000000004,"5min":0.50800000000000001,"15min":0.45900000000000002},"max":{"1min":1.6080000000000001,"5min":1.758,"15min":3.2130000000000001},"last":1.2290000000000001}]},{"osd":4,"last update":"Sat Sep 23 03:27:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.278,"15min":1.2569999999999999},"min":{"1min":0.81000000000000005,"5min":0.79900000000000004,"15min":0.70699999999999996},"max":{"1min":1.5640000000000001,"5min":1.885,"15min":3.2629999999999999},"last":1.482},{"interface":"front","average":{"1min":1.224,"5min":1.244,"15min":1.256},"min":{"1min":0.75,"5min":0.75,"15min":0.66500000000000004},"max":{"1min":1.5469999999999999,"5min":1.8460000000000001,"15min":2.4689999999999999},"last":1.333}]},{"osd":5,"last update":"Sat Sep 23 03:27:36 2023","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.288,"15min":1.282},"min":{"1min":0.90200000000000002,"5min":0.76300000000000001,"15min":0.64600000000000002},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":3.161},"last":1.712},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.327,"15min":1.3280000000000001},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.72399999999999998},"max":{"1min":1.6639999999999999,"5min":1.7929999999999999,"15min":2.4289999999999998},"last":1.423}]},{"osd":6,"last update":"Sat Sep 23 03:27:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3300000000000001},"min":{"1min":0.79400000000000004,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.728,"5min":1.823,"15min":3.0179999999999998},"last":1.613},{"interface":"front","average":{"1min":1.369,"5min":1.375,"15min":1.371},"min":{"1min":0.91800000000000004,"5min":0.82499999999999996,"15min":0.72799999999999998},"max":{"1min":1.706,"5min":1.8340000000000001,"15min":3.3580000000000001},"last":1.569}]},{"osd":7,"last update":"Sat Sep 23 03:27:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.359,"15min":1.371},"min":{"1min":0.88,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.663,"5min":1.931,"15min":3.1099999999999999},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.397,"5min":1.397,"15min":1.425},"min":{"1min":0.85899999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.73,"5min":1.9299999999999999,"15min":3.306},"last":1.6759999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575005,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33292,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338548,"statfs":{"total":96636764160,"available":96602673152,"internally_reserved":0,"allocated":2162688,"data_stored":1355901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 23 03:27:29 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.81899999999999995,"15min":0.83599999999999997},"min":{"1min":0.56000000000000005,"5min":0.38,"15min":0.30299999999999999},"max":{"1min":1.256,"5min":1.383,"15min":1.919},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.86699999999999999,"15min":0.88800000000000001},"min":{"1min":0.45200000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":2.1360000000000001},"last":0.75}]},{"osd":2,"last update":"Sat Sep 23 03:27:29 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.88100000000000001,"15min":0.84899999999999998},"min":{"1min":0.46999999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.2130000000000001,"5min":1.5249999999999999,"15min":3.089},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.96599999999999997,"15min":1.002},"min":{"1min":0.45700000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.3959999999999999,"5min":1.5760000000000001,"15min":2.633},"last":0.91000000000000003}]},{"osd":3,"last update":"Sat Sep 23 03:27:29 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":0.998,"15min":0.97399999999999998},"min":{"1min":0.65500000000000003,"5min":0.433,"15min":0.433},"max":{"1min":1.4019999999999999,"5min":1.615,"15min":2.8660000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.0129999999999999,"15min":1.014},"min":{"1min":0.57399999999999995,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.591,"5min":1.802,"15min":2.012},"last":0.96699999999999997}]},{"osd":4,"last update":"Sat Sep 23 03:27:29 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.2629999999999999,"15min":1.2609999999999999},"min":{"1min":0.70899999999999996,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.6950000000000001,"5min":1.885,"15min":2.4140000000000001},"last":0.999},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.226,"15min":1.2649999999999999},"min":{"1min":0.76100000000000001,"5min":0.73999999999999999,"15min":0.65300000000000002},"max":{"1min":1.845,"5min":1.976,"15min":3.0230000000000001},"last":0.83199999999999996}]},{"osd":5,"last update":"Sat Sep 23 03:27:29 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.304,"15min":1.27},"min":{"1min":0.94299999999999995,"5min":0.81499999999999995,"15min":0.65600000000000003},"max":{"1min":1.6870000000000001,"5min":2.1240000000000001,"15min":2.5489999999999999},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.425,"5min":1.341,"15min":1.3340000000000001},"min":{"1min":0.77500000000000002,"5min":0.73099999999999998,"15min":0.71999999999999997},"max":{"1min":1.7769999999999999,"5min":1.9159999999999999,"15min":2.1539999999999999},"last":1.079}]},{"osd":6,"last update":"Sat Sep 23 03:27:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3320000000000001,"15min":1.3240000000000001},"min":{"1min":0.86799999999999999,"5min":0.77700000000000002,"15min":0.75600000000000001},"max":{"1min":1.7869999999999999,"5min":2.0870000000000002,"15min":2.9489999999999998},"last":1.151},{"interface":"front","average":{"1min":1.446,"5min":1.4039999999999999,"15min":1.383},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.901,"5min":1.9670000000000001,"15min":2.3239999999999998},"last":1.1279999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:27:29 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.363,"15min":1.3640000000000001},"min":{"1min":0.83699999999999997,"5min":0.752,"15min":0.747},"max":{"1min":2.052,"5min":2.052,"15min":2.2330000000000001},"last":1.02},{"interface":"front","average":{"1min":1.478,"5min":1.419,"15min":1.4330000000000001},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.1419999999999999,"5min":2.1419999999999999,"15min":2.3330000000000002},"last":1.0960000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-23T03:27:49.869 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-23T03:27:50.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.128+0000 7fb236fee640 1 Processor -- start 2023-09-23T03:27:50.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.128+0000 7fb236fee640 1 -- start start 2023-09-23T03:27:50.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.130+0000 7fb236fee640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb2300acce0 0x7fb23015ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:50.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.130+0000 7fb236fee640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2300b1b20 0x7fb2300ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:50.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.130+0000 7fb236fee640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb23015ac30 con 0x7fb2300cee70 2023-09-23T03:27:50.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.130+0000 7fb236fee640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2300b2e90 con 0x7fb2300acce0 2023-09-23T03:27:50.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.130+0000 7fb236fee640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2300cf320 con 0x7fb2300b1b20 2023-09-23T03:27:50.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.130+0000 7fb234d8a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb2300acce0 0x7fb23015ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:50.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.130+0000 7fb23558b640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fb2300cee70 0x7fb23006cb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:47924/0 (socket says 172.21.15.154:47924) 2023-09-23T03:27:50.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.130+0000 7fb23558b640 1 -- 172.21.15.154:0/602569910 learned_addr learned my addr 172.21.15.154:0/602569910 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:27:50.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.130+0000 7fb22ffff640 1 --2- 172.21.15.154:0/602569910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2300b1b20 0x7fb2300ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:50.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.131+0000 7fb234d8a640 1 -- 172.21.15.154:0/602569910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2300b1b20 msgr2=0x7fb2300ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:50.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.131+0000 7fb234d8a640 1 --2- 172.21.15.154:0/602569910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2300b1b20 0x7fb2300ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:50.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.131+0000 7fb234d8a640 1 -- 172.21.15.154:0/602569910 >> v1:172.21.15.154:6789/0 conn(0x7fb2300cee70 legacy=0x7fb23006cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:50.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.131+0000 7fb234d8a640 1 -- 172.21.15.154:0/602569910 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2300b1f70 con 0x7fb2300acce0 2023-09-23T03:27:50.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.131+0000 7fb22f7fe640 1 -- 172.21.15.154:0/602569910 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3763576768 0 0) 0x7fb228001490 con 0x7fb2300cee70 2023-09-23T03:27:50.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.132+0000 7fb234d8a640 1 --2- 172.21.15.154:0/602569910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2300acce0 0x7fb23015ed60 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fb224009790 tx=0x7fb22402b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291eda05ac7c8391 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:50.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.132+0000 7fb22f7fe640 1 -- 172.21.15.154:0/602569910 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb224031770 con 0x7fb2300acce0 2023-09-23T03:27:50.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.132+0000 7fb22f7fe640 1 -- 172.21.15.154:0/602569910 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2240318d0 con 0x7fb2300acce0 2023-09-23T03:27:50.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.132+0000 7fb22f7fe640 1 -- 172.21.15.154:0/602569910 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb224031c60 con 0x7fb2300acce0 2023-09-23T03:27:50.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.133+0000 7fb236fee640 1 -- 172.21.15.154:0/602569910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2300acce0 msgr2=0x7fb23015ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:50.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.133+0000 7fb236fee640 1 --2- 172.21.15.154:0/602569910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2300acce0 0x7fb23015ed60 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fb224009790 tx=0x7fb22402b810 comp rx=0 tx=0).stop 2023-09-23T03:27:50.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.133+0000 7fb236fee640 1 -- 172.21.15.154:0/602569910 shutdown_connections 2023-09-23T03:27:50.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.133+0000 7fb236fee640 1 --2- 172.21.15.154:0/602569910 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2300b1b20 0x7fb2300ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:50.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.133+0000 7fb236fee640 1 --2- 172.21.15.154:0/602569910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2300acce0 0x7fb23015ed60 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:50.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.133+0000 7fb236fee640 1 -- 172.21.15.154:0/602569910 >> 172.21.15.154:0/602569910 conn(0x7fb23006a0f0 msgr2=0x7fb23006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:50.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.133+0000 7fb236fee640 1 -- 172.21.15.154:0/602569910 shutdown_connections 2023-09-23T03:27:50.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.133+0000 7fb236fee640 1 -- 172.21.15.154:0/602569910 wait complete. 2023-09-23T03:27:50.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.135+0000 7fb236fee640 1 Processor -- start 2023-09-23T03:27:50.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.135+0000 7fb236fee640 1 -- start start 2023-09-23T03:27:50.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.136+0000 7fb236fee640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2300acce0 0x7fb2300c3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:50.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.136+0000 7fb236fee640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2300b1b20 0x7fb230159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:50.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.136+0000 7fb236fee640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2300cee70 0x7fb23015cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:50.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.136+0000 7fb236fee640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23015dc60 con 0x7fb2300cee70 2023-09-23T03:27:50.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.136+0000 7fb236fee640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2300c49a0 con 0x7fb2300acce0 2023-09-23T03:27:50.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.136+0000 7fb236fee640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2300c6840 con 0x7fb2300b1b20 2023-09-23T03:27:50.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.136+0000 7fb23558b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2300cee70 0x7fb23015cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:50.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.136+0000 7fb23558b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2300cee70 0x7fb23015cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50508/0 (socket says 172.21.15.154:50508) 2023-09-23T03:27:50.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.136+0000 7fb23558b640 1 -- 172.21.15.154:0/3032478118 learned_addr learned my addr 172.21.15.154:0/3032478118 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:50.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.136+0000 7fb234d8a640 1 --2- 172.21.15.154:0/3032478118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2300acce0 0x7fb2300c3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:50.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.136+0000 7fb22ffff640 1 --2- 172.21.15.154:0/3032478118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2300b1b20 0x7fb230159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:50.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.136+0000 7fb23558b640 1 -- 172.21.15.154:0/3032478118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2300b1b20 msgr2=0x7fb230159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:50.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.136+0000 7fb23558b640 1 --2- 172.21.15.154:0/3032478118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2300b1b20 0x7fb230159b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:50.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.136+0000 7fb23558b640 1 -- 172.21.15.154:0/3032478118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2300acce0 msgr2=0x7fb2300c3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:50.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.136+0000 7fb23558b640 1 --2- 172.21.15.154:0/3032478118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2300acce0 0x7fb2300c3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:50.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.137+0000 7fb23558b640 1 -- 172.21.15.154:0/3032478118 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2240093a0 con 0x7fb2300cee70 2023-09-23T03:27:50.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.137+0000 7fb23558b640 1 --2- 172.21.15.154:0/3032478118 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2300cee70 0x7fb23015cea0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fb22800c780 tx=0x7fb22800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:50.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.137+0000 7fb22d7fa640 1 -- 172.21.15.154:0/3032478118 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22800ce90 con 0x7fb2300cee70 2023-09-23T03:27:50.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.137+0000 7fb22d7fa640 1 -- 172.21.15.154:0/3032478118 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb22800a640 con 0x7fb2300cee70 2023-09-23T03:27:50.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.137+0000 7fb22d7fa640 1 -- 172.21.15.154:0/3032478118 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22800a9d0 con 0x7fb2300cee70 2023-09-23T03:27:50.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.137+0000 7fb236fee640 1 -- 172.21.15.154:0/3032478118 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2300c4100 con 0x7fb2300cee70 2023-09-23T03:27:50.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.137+0000 7fb236fee640 1 -- 172.21.15.154:0/3032478118 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2300c46c0 con 0x7fb2300cee70 2023-09-23T03:27:50.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.138+0000 7fb236fee640 1 -- 172.21.15.154:0/3032478118 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb230073eb0 con 0x7fb2300cee70 2023-09-23T03:27:50.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.139+0000 7fb22d7fa640 1 -- 172.21.15.154:0/3032478118 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fb22800ab30 con 0x7fb2300cee70 2023-09-23T03:27:50.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.140+0000 7fb22d7fa640 1 --2- 172.21.15.154:0/3032478118 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb21406af30 0x7fb21406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:50.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.140+0000 7fb22d7fa640 1 -- 172.21.15.154:0/3032478118 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fb228088b60 con 0x7fb2300cee70 2023-09-23T03:27:50.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.140+0000 7fb234d8a640 1 --2- 172.21.15.154:0/3032478118 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb21406af30 0x7fb21406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:50.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.141+0000 7fb234d8a640 1 --2- 172.21.15.154:0/3032478118 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb21406af30 0x7fb21406d3f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb224036000 tx=0x7fb22403d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:50.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.144+0000 7fb22d7fa640 1 -- 172.21.15.154:0/3032478118 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fb228053a50 con 0x7fb2300cee70 2023-09-23T03:27:50.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.329+0000 7fb236fee640 1 -- 172.21.15.154:0/3032478118 --> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb2300b2210 con 0x7fb21406af30 2023-09-23T03:27:50.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.351+0000 7fb22d7fa640 1 -- 172.21.15.154:0/3032478118 <== mgr.4110 v2:172.21.15.154:6832/2422652104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287051 (secure 0 0 0) 0x7fb224031a40 con 0x7fb21406af30 2023-09-23T03:27:50.355 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:50.357 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2023-09-23T03:27:50.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.358+0000 7fb236fee640 1 -- 172.21.15.154:0/3032478118 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb21406af30 msgr2=0x7fb21406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:50.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.358+0000 7fb236fee640 1 --2- 172.21.15.154:0/3032478118 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb21406af30 0x7fb21406d3f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb224036000 tx=0x7fb22403d040 comp rx=0 tx=0).stop 2023-09-23T03:27:50.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.358+0000 7fb236fee640 1 -- 172.21.15.154:0/3032478118 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2300cee70 msgr2=0x7fb23015cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:50.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.358+0000 7fb236fee640 1 --2- 172.21.15.154:0/3032478118 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2300cee70 0x7fb23015cea0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fb22800c780 tx=0x7fb22800cc40 comp rx=0 tx=0).stop 2023-09-23T03:27:50.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.358+0000 7fb236fee640 1 -- 172.21.15.154:0/3032478118 shutdown_connections 2023-09-23T03:27:50.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.358+0000 7fb236fee640 1 --2- 172.21.15.154:0/3032478118 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb21406af30 0x7fb21406d3f0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:50.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.358+0000 7fb236fee640 1 --2- 172.21.15.154:0/3032478118 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb2300cee70 0x7fb23015cea0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:50.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.358+0000 7fb236fee640 1 --2- 172.21.15.154:0/3032478118 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb2300b1b20 0x7fb230159b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:50.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.358+0000 7fb236fee640 1 --2- 172.21.15.154:0/3032478118 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb2300acce0 0x7fb2300c3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:50.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.358+0000 7fb236fee640 1 -- 172.21.15.154:0/3032478118 >> 172.21.15.154:0/3032478118 conn(0x7fb23006a0f0 msgr2=0x7fb2300b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:50.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.359+0000 7fb236fee640 1 -- 172.21.15.154:0/3032478118 shutdown_connections 2023-09-23T03:27:50.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.359+0000 7fb236fee640 1 -- 172.21.15.154:0/3032478118 wait complete. 2023-09-23T03:27:50.379 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":499,"stamp":"2023-09-23T03:27:49.433961+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":190,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262936,"kb_used_data":13476,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711784,"statfs":{"total":773094113280,"available":772824866816,"internally_reserved":0,"allocated":13799424,"data_stored":7316777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":255315514},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001624"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.381444+0000","last_change":"2023-09-23T03:27:45.025920+0000","last_active":"2023-09-23T03:27:47.381444+0000","last_peered":"2023-09-23T03:27:47.381444+0000","last_clean":"2023-09-23T03:27:47.381444+0000","last_became_active":"2023-09-23T03:27:45.025588+0000","last_became_peered":"2023-09-23T03:27:45.025588+0000","last_unstale":"2023-09-23T03:27:47.381444+0000","last_undegraded":"2023-09-23T03:27:47.381444+0000","last_fullsized":"2023-09-23T03:27:47.381444+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:58:00.149687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.383647+0000","last_change":"2023-09-23T03:27:45.047955+0000","last_active":"2023-09-23T03:27:47.383647+0000","last_peered":"2023-09-23T03:27:47.383647+0000","last_clean":"2023-09-23T03:27:47.383647+0000","last_became_active":"2023-09-23T03:27:45.047592+0000","last_became_peered":"2023-09-23T03:27:45.047592+0000","last_unstale":"2023-09-23T03:27:47.383647+0000","last_undegraded":"2023-09-23T03:27:47.383647+0000","last_fullsized":"2023-09-23T03:27:47.383647+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:26:50.402279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.384169+0000","last_change":"2023-09-23T03:27:45.046402+0000","last_active":"2023-09-23T03:27:47.384169+0000","last_peered":"2023-09-23T03:27:47.384169+0000","last_clean":"2023-09-23T03:27:47.384169+0000","last_became_active":"2023-09-23T03:27:45.046113+0000","last_became_peered":"2023-09-23T03:27:45.046113+0000","last_unstale":"2023-09-23T03:27:47.384169+0000","last_undegraded":"2023-09-23T03:27:47.384169+0000","last_fullsized":"2023-09-23T03:27:47.384169+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:15:12.188323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:46.033055+0000","last_change":"2023-09-23T03:27:45.036372+0000","last_active":"2023-09-23T03:27:46.033055+0000","last_peered":"2023-09-23T03:27:46.033055+0000","last_clean":"2023-09-23T03:27:46.033055+0000","last_became_active":"2023-09-23T03:27:45.036014+0000","last_became_peered":"2023-09-23T03:27:45.036014+0000","last_unstale":"2023-09-23T03:27:46.033055+0000","last_undegraded":"2023-09-23T03:27:46.033055+0000","last_fullsized":"2023-09-23T03:27:46.033055+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:04:35.006829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.381864+0000","last_change":"2023-09-23T03:27:45.017104+0000","last_active":"2023-09-23T03:27:47.381864+0000","last_peered":"2023-09-23T03:27:47.381864+0000","last_clean":"2023-09-23T03:27:47.381864+0000","last_became_active":"2023-09-23T03:27:45.016482+0000","last_became_peered":"2023-09-23T03:27:45.016482+0000","last_unstale":"2023-09-23T03:27:47.381864+0000","last_undegraded":"2023-09-23T03:27:47.381864+0000","last_fullsized":"2023-09-23T03:27:47.381864+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:45:05.020375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.380975+0000","last_change":"2023-09-23T03:27:45.026003+0000","last_active":"2023-09-23T03:27:47.380975+0000","last_peered":"2023-09-23T03:27:47.380975+0000","last_clean":"2023-09-23T03:27:47.380975+0000","last_became_active":"2023-09-23T03:27:45.025738+0000","last_became_peered":"2023-09-23T03:27:45.025738+0000","last_unstale":"2023-09-23T03:27:47.380975+0000","last_undegraded":"2023-09-23T03:27:47.380975+0000","last_fullsized":"2023-09-23T03:27:47.380975+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:41:47.146262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.381445+0000","last_change":"2023-09-23T03:27:45.044087+0000","last_active":"2023-09-23T03:27:47.381445+0000","last_peered":"2023-09-23T03:27:47.381445+0000","last_clean":"2023-09-23T03:27:47.381445+0000","last_became_active":"2023-09-23T03:27:45.043704+0000","last_became_peered":"2023-09-23T03:27:45.043704+0000","last_unstale":"2023-09-23T03:27:47.381445+0000","last_undegraded":"2023-09-23T03:27:47.381445+0000","last_fullsized":"2023-09-23T03:27:47.381445+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:26:03.298065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.381739+0000","last_change":"2023-09-23T03:27:45.042385+0000","last_active":"2023-09-23T03:27:47.381739+0000","last_peered":"2023-09-23T03:27:47.381739+0000","last_clean":"2023-09-23T03:27:47.381739+0000","last_became_active":"2023-09-23T03:27:45.042100+0000","last_became_peered":"2023-09-23T03:27:45.042100+0000","last_unstale":"2023-09-23T03:27:47.381739+0000","last_undegraded":"2023-09-23T03:27:47.381739+0000","last_fullsized":"2023-09-23T03:27:47.381739+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:04:35.006829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:46.034085+0000","last_change":"2023-09-23T03:27:45.047530+0000","last_active":"2023-09-23T03:27:46.034085+0000","last_peered":"2023-09-23T03:27:46.034085+0000","last_clean":"2023-09-23T03:27:46.034085+0000","last_became_active":"2023-09-23T03:27:45.047264+0000","last_became_peered":"2023-09-23T03:27:45.047264+0000","last_unstale":"2023-09-23T03:27:46.034085+0000","last_undegraded":"2023-09-23T03:27:46.034085+0000","last_fullsized":"2023-09-23T03:27:46.034085+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:51:53.201001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.383259+0000","last_change":"2023-09-23T03:27:45.029426+0000","last_active":"2023-09-23T03:27:47.383259+0000","last_peered":"2023-09-23T03:27:47.383259+0000","last_clean":"2023-09-23T03:27:47.383259+0000","last_became_active":"2023-09-23T03:27:45.029159+0000","last_became_peered":"2023-09-23T03:27:45.029159+0000","last_unstale":"2023-09-23T03:27:47.383259+0000","last_undegraded":"2023-09-23T03:27:47.383259+0000","last_fullsized":"2023-09-23T03:27:47.383259+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:35:45.037564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.382242+0000","last_change":"2023-09-23T03:27:45.030013+0000","last_active":"2023-09-23T03:27:47.382242+0000","last_peered":"2023-09-23T03:27:47.382242+0000","last_clean":"2023-09-23T03:27:47.382242+0000","last_became_active":"2023-09-23T03:27:45.029744+0000","last_became_peered":"2023-09-23T03:27:45.029744+0000","last_unstale":"2023-09-23T03:27:47.382242+0000","last_undegraded":"2023-09-23T03:27:47.382242+0000","last_fullsized":"2023-09-23T03:27:47.382242+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:35:45.037564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.382843+0000","last_change":"2023-09-23T03:27:45.028794+0000","last_active":"2023-09-23T03:27:47.382843+0000","last_peered":"2023-09-23T03:27:47.382843+0000","last_clean":"2023-09-23T03:27:47.382843+0000","last_became_active":"2023-09-23T03:27:45.028516+0000","last_became_peered":"2023-09-23T03:27:45.028516+0000","last_unstale":"2023-09-23T03:27:47.382843+0000","last_undegraded":"2023-09-23T03:27:47.382843+0000","last_fullsized":"2023-09-23T03:27:47.382843+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:44:27.854290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.382283+0000","last_change":"2023-09-23T03:27:45.030916+0000","last_active":"2023-09-23T03:27:47.382283+0000","last_peered":"2023-09-23T03:27:47.382283+0000","last_clean":"2023-09-23T03:27:47.382283+0000","last_became_active":"2023-09-23T03:27:45.030657+0000","last_became_peered":"2023-09-23T03:27:45.030657+0000","last_unstale":"2023-09-23T03:27:47.382283+0000","last_undegraded":"2023-09-23T03:27:47.382283+0000","last_fullsized":"2023-09-23T03:27:47.382283+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:14:40.320939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.381361+0000","last_change":"2023-09-23T03:27:45.041728+0000","last_active":"2023-09-23T03:27:47.381361+0000","last_peered":"2023-09-23T03:27:47.381361+0000","last_clean":"2023-09-23T03:27:47.381361+0000","last_became_active":"2023-09-23T03:27:45.041167+0000","last_became_peered":"2023-09-23T03:27:45.041167+0000","last_unstale":"2023-09-23T03:27:47.381361+0000","last_undegraded":"2023-09-23T03:27:47.381361+0000","last_fullsized":"2023-09-23T03:27:47.381361+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:11:36.520222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.380615+0000","last_change":"2023-09-23T03:27:45.031107+0000","last_active":"2023-09-23T03:27:47.380615+0000","last_peered":"2023-09-23T03:27:47.380615+0000","last_clean":"2023-09-23T03:27:47.380615+0000","last_became_active":"2023-09-23T03:27:45.030361+0000","last_became_peered":"2023-09-23T03:27:45.030361+0000","last_unstale":"2023-09-23T03:27:47.380615+0000","last_undegraded":"2023-09-23T03:27:47.380615+0000","last_fullsized":"2023-09-23T03:27:47.380615+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:14:40.320939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.380336+0000","last_change":"2023-09-23T03:27:45.021810+0000","last_active":"2023-09-23T03:27:47.380336+0000","last_peered":"2023-09-23T03:27:47.380336+0000","last_clean":"2023-09-23T03:27:47.380336+0000","last_became_active":"2023-09-23T03:27:45.021146+0000","last_became_peered":"2023-09-23T03:27:45.021146+0000","last_unstale":"2023-09-23T03:27:47.380336+0000","last_undegraded":"2023-09-23T03:27:47.380336+0000","last_fullsized":"2023-09-23T03:27:47.380336+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:04:56.120276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.383982+0000","last_change":"2023-09-23T03:27:45.031602+0000","last_active":"2023-09-23T03:27:47.383982+0000","last_peered":"2023-09-23T03:27:47.383982+0000","last_clean":"2023-09-23T03:27:47.383982+0000","last_became_active":"2023-09-23T03:27:45.031331+0000","last_became_peered":"2023-09-23T03:27:45.031331+0000","last_unstale":"2023-09-23T03:27:47.383982+0000","last_undegraded":"2023-09-23T03:27:47.383982+0000","last_fullsized":"2023-09-23T03:27:47.383982+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:39:04.634154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.383794+0000","last_change":"2023-09-23T03:27:45.047390+0000","last_active":"2023-09-23T03:27:47.383794+0000","last_peered":"2023-09-23T03:27:47.383794+0000","last_clean":"2023-09-23T03:27:47.383794+0000","last_became_active":"2023-09-23T03:27:45.047111+0000","last_became_peered":"2023-09-23T03:27:45.047111+0000","last_unstale":"2023-09-23T03:27:47.383794+0000","last_undegraded":"2023-09-23T03:27:47.383794+0000","last_fullsized":"2023-09-23T03:27:47.383794+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:24:25.148743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:46.035546+0000","last_change":"2023-09-23T03:27:45.038744+0000","last_active":"2023-09-23T03:27:46.035546+0000","last_peered":"2023-09-23T03:27:46.035546+0000","last_clean":"2023-09-23T03:27:46.035546+0000","last_became_active":"2023-09-23T03:27:45.038370+0000","last_became_peered":"2023-09-23T03:27:45.038370+0000","last_unstale":"2023-09-23T03:27:46.035546+0000","last_undegraded":"2023-09-23T03:27:46.035546+0000","last_fullsized":"2023-09-23T03:27:46.035546+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:16:05.766908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:46.032159+0000","last_change":"2023-09-23T03:27:45.022548+0000","last_active":"2023-09-23T03:27:46.032159+0000","last_peered":"2023-09-23T03:27:46.032159+0000","last_clean":"2023-09-23T03:27:46.032159+0000","last_became_active":"2023-09-23T03:27:45.022234+0000","last_became_peered":"2023-09-23T03:27:45.022234+0000","last_unstale":"2023-09-23T03:27:46.032159+0000","last_undegraded":"2023-09-23T03:27:46.032159+0000","last_fullsized":"2023-09-23T03:27:46.032159+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:39:38.063635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:46.033666+0000","last_change":"2023-09-23T03:27:45.036794+0000","last_active":"2023-09-23T03:27:46.033666+0000","last_peered":"2023-09-23T03:27:46.033666+0000","last_clean":"2023-09-23T03:27:46.033666+0000","last_became_active":"2023-09-23T03:27:45.036308+0000","last_became_peered":"2023-09-23T03:27:45.036308+0000","last_unstale":"2023-09-23T03:27:46.033666+0000","last_undegraded":"2023-09-23T03:27:46.033666+0000","last_fullsized":"2023-09-23T03:27:46.033666+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:16:08.067236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.381445+0000","last_change":"2023-09-23T03:27:45.026645+0000","last_active":"2023-09-23T03:27:47.381445+0000","last_peered":"2023-09-23T03:27:47.381445+0000","last_clean":"2023-09-23T03:27:47.381445+0000","last_became_active":"2023-09-23T03:27:45.026306+0000","last_became_peered":"2023-09-23T03:27:45.026306+0000","last_unstale":"2023-09-23T03:27:47.381445+0000","last_undegraded":"2023-09-23T03:27:47.381445+0000","last_fullsized":"2023-09-23T03:27:47.381445+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:41:47.631539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:46.032642+0000","last_change":"2023-09-23T03:27:45.047533+0000","last_active":"2023-09-23T03:27:46.032642+0000","last_peered":"2023-09-23T03:27:46.032642+0000","last_clean":"2023-09-23T03:27:46.032642+0000","last_became_active":"2023-09-23T03:27:45.047218+0000","last_became_peered":"2023-09-23T03:27:45.047218+0000","last_unstale":"2023-09-23T03:27:46.032642+0000","last_undegraded":"2023-09-23T03:27:46.032642+0000","last_fullsized":"2023-09-23T03:27:46.032642+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:54:18.955362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.380442+0000","last_change":"2023-09-23T03:27:45.032574+0000","last_active":"2023-09-23T03:27:47.380442+0000","last_peered":"2023-09-23T03:27:47.380442+0000","last_clean":"2023-09-23T03:27:47.380442+0000","last_became_active":"2023-09-23T03:27:45.032285+0000","last_became_peered":"2023-09-23T03:27:45.032285+0000","last_unstale":"2023-09-23T03:27:47.380442+0000","last_undegraded":"2023-09-23T03:27:47.380442+0000","last_fullsized":"2023-09-23T03:27:47.380442+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:54:34.616945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.382266+0000","last_change":"2023-09-23T03:27:45.038242+0000","last_active":"2023-09-23T03:27:47.382266+0000","last_peered":"2023-09-23T03:27:47.382266+0000","last_clean":"2023-09-23T03:27:47.382266+0000","last_became_active":"2023-09-23T03:27:45.037720+0000","last_became_peered":"2023-09-23T03:27:45.037720+0000","last_unstale":"2023-09-23T03:27:47.382266+0000","last_undegraded":"2023-09-23T03:27:47.382266+0000","last_fullsized":"2023-09-23T03:27:47.382266+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:52:03.205421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:46.032485+0000","last_change":"2023-09-23T03:27:45.017554+0000","last_active":"2023-09-23T03:27:46.032485+0000","last_peered":"2023-09-23T03:27:46.032485+0000","last_clean":"2023-09-23T03:27:46.032485+0000","last_became_active":"2023-09-23T03:27:45.017263+0000","last_became_peered":"2023-09-23T03:27:45.017263+0000","last_unstale":"2023-09-23T03:27:46.032485+0000","last_undegraded":"2023-09-23T03:27:46.032485+0000","last_fullsized":"2023-09-23T03:27:46.032485+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:10:03.525856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:46.035677+0000","last_change":"2023-09-23T03:27:45.041958+0000","last_active":"2023-09-23T03:27:46.035677+0000","last_peered":"2023-09-23T03:27:46.035677+0000","last_clean":"2023-09-23T03:27:46.035677+0000","last_became_active":"2023-09-23T03:27:45.041682+0000","last_became_peered":"2023-09-23T03:27:45.041682+0000","last_unstale":"2023-09-23T03:27:46.035677+0000","last_undegraded":"2023-09-23T03:27:46.035677+0000","last_fullsized":"2023-09-23T03:27:46.035677+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:30:18.768868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.383146+0000","last_change":"2023-09-23T03:27:45.035003+0000","last_active":"2023-09-23T03:27:47.383146+0000","last_peered":"2023-09-23T03:27:47.383146+0000","last_clean":"2023-09-23T03:27:47.383146+0000","last_became_active":"2023-09-23T03:27:45.034666+0000","last_became_peered":"2023-09-23T03:27:45.034666+0000","last_unstale":"2023-09-23T03:27:47.383146+0000","last_undegraded":"2023-09-23T03:27:47.383146+0000","last_fullsized":"2023-09-23T03:27:47.383146+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:05:54.457634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:46.032509+0000","last_change":"2023-09-23T03:27:45.026019+0000","last_active":"2023-09-23T03:27:46.032509+0000","last_peered":"2023-09-23T03:27:46.032509+0000","last_clean":"2023-09-23T03:27:46.032509+0000","last_became_active":"2023-09-23T03:27:45.025765+0000","last_became_peered":"2023-09-23T03:27:45.025765+0000","last_unstale":"2023-09-23T03:27:46.032509+0000","last_undegraded":"2023-09-23T03:27:46.032509+0000","last_fullsized":"2023-09-23T03:27:46.032509+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:07:33.355203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.381719+0000","last_change":"2023-09-23T03:27:45.027482+0000","last_active":"2023-09-23T03:27:47.381719+0000","last_peered":"2023-09-23T03:27:47.381719+0000","last_clean":"2023-09-23T03:27:47.381719+0000","last_became_active":"2023-09-23T03:27:45.027218+0000","last_became_peered":"2023-09-23T03:27:45.027218+0000","last_unstale":"2023-09-23T03:27:47.381719+0000","last_undegraded":"2023-09-23T03:27:47.381719+0000","last_fullsized":"2023-09-23T03:27:47.381719+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:40:45.625728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.381155+0000","last_change":"2023-09-23T03:27:45.029377+0000","last_active":"2023-09-23T03:27:47.381155+0000","last_peered":"2023-09-23T03:27:47.381155+0000","last_clean":"2023-09-23T03:27:47.381155+0000","last_became_active":"2023-09-23T03:27:45.029058+0000","last_became_peered":"2023-09-23T03:27:45.029058+0000","last_unstale":"2023-09-23T03:27:47.381155+0000","last_undegraded":"2023-09-23T03:27:47.381155+0000","last_fullsized":"2023-09-23T03:27:47.381155+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:46:09.395570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.380453+0000","last_change":"2023-09-23T03:27:45.033139+0000","last_active":"2023-09-23T03:27:47.380453+0000","last_peered":"2023-09-23T03:27:47.380453+0000","last_clean":"2023-09-23T03:27:47.380453+0000","last_became_active":"2023-09-23T03:27:45.032841+0000","last_became_peered":"2023-09-23T03:27:45.032841+0000","last_unstale":"2023-09-23T03:27:47.380453+0000","last_undegraded":"2023-09-23T03:27:47.380453+0000","last_fullsized":"2023-09-23T03:27:47.380453+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:00:15.928606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.383420+0000","last_change":"2023-09-23T03:27:45.023987+0000","last_active":"2023-09-23T03:27:47.383420+0000","last_peered":"2023-09-23T03:27:47.383420+0000","last_clean":"2023-09-23T03:27:47.383420+0000","last_became_active":"2023-09-23T03:27:45.023687+0000","last_became_peered":"2023-09-23T03:27:45.023687+0000","last_unstale":"2023-09-23T03:27:47.383420+0000","last_undegraded":"2023-09-23T03:27:47.383420+0000","last_fullsized":"2023-09-23T03:27:47.383420+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:07:46.010795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"125'4","reported_seq":16,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.045895+0000","last_change":"2023-09-23T03:27:44.026019+0000","last_active":"2023-09-23T03:27:45.045895+0000","last_peered":"2023-09-23T03:27:45.045895+0000","last_clean":"2023-09-23T03:27:45.045895+0000","last_became_active":"2023-09-23T03:27:44.025591+0000","last_became_peered":"2023-09-23T03:27:44.025591+0000","last_unstale":"2023-09-23T03:27:45.045895+0000","last_undegraded":"2023-09-23T03:27:45.045895+0000","last_fullsized":"2023-09-23T03:27:45.045895+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:23:07.728424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.381758+0000","last_change":"2023-09-23T03:27:45.025775+0000","last_active":"2023-09-23T03:27:47.381758+0000","last_peered":"2023-09-23T03:27:47.381758+0000","last_clean":"2023-09-23T03:27:47.381758+0000","last_became_active":"2023-09-23T03:27:45.025480+0000","last_became_peered":"2023-09-23T03:27:45.025480+0000","last_unstale":"2023-09-23T03:27:47.381758+0000","last_undegraded":"2023-09-23T03:27:47.381758+0000","last_fullsized":"2023-09-23T03:27:47.381758+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:14:02.931595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"125'1","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.040299+0000","last_change":"2023-09-23T03:27:44.030761+0000","last_active":"2023-09-23T03:27:45.040299+0000","last_peered":"2023-09-23T03:27:45.040299+0000","last_clean":"2023-09-23T03:27:45.040299+0000","last_became_active":"2023-09-23T03:27:44.030488+0000","last_became_peered":"2023-09-23T03:27:44.030488+0000","last_unstale":"2023-09-23T03:27:45.040299+0000","last_undegraded":"2023-09-23T03:27:45.040299+0000","last_fullsized":"2023-09-23T03:27:45.040299+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:39:37.051847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.382607+0000","last_change":"2023-09-23T03:27:45.040938+0000","last_active":"2023-09-23T03:27:47.382607+0000","last_peered":"2023-09-23T03:27:47.382607+0000","last_clean":"2023-09-23T03:27:47.382607+0000","last_became_active":"2023-09-23T03:27:45.040674+0000","last_became_peered":"2023-09-23T03:27:45.040674+0000","last_unstale":"2023-09-23T03:27:47.382607+0000","last_undegraded":"2023-09-23T03:27:47.382607+0000","last_fullsized":"2023-09-23T03:27:47.382607+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:00:23.239974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"125'2","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.043944+0000","last_change":"2023-09-23T03:27:44.037191+0000","last_active":"2023-09-23T03:27:45.043944+0000","last_peered":"2023-09-23T03:27:45.043944+0000","last_clean":"2023-09-23T03:27:45.043944+0000","last_became_active":"2023-09-23T03:27:44.036801+0000","last_became_peered":"2023-09-23T03:27:44.036801+0000","last_unstale":"2023-09-23T03:27:45.043944+0000","last_undegraded":"2023-09-23T03:27:45.043944+0000","last_fullsized":"2023-09-23T03:27:45.043944+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:14:15.913622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:46.034402+0000","last_change":"2023-09-23T03:27:45.035267+0000","last_active":"2023-09-23T03:27:46.034402+0000","last_peered":"2023-09-23T03:27:46.034402+0000","last_clean":"2023-09-23T03:27:46.034402+0000","last_became_active":"2023-09-23T03:27:45.034952+0000","last_became_peered":"2023-09-23T03:27:45.034952+0000","last_unstale":"2023-09-23T03:27:46.034402+0000","last_undegraded":"2023-09-23T03:27:46.034402+0000","last_fullsized":"2023-09-23T03:27:46.034402+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:07:35.140412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"125'2","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.044236+0000","last_change":"2023-09-23T03:27:44.039286+0000","last_active":"2023-09-23T03:27:45.044236+0000","last_peered":"2023-09-23T03:27:45.044236+0000","last_clean":"2023-09-23T03:27:45.044236+0000","last_became_active":"2023-09-23T03:27:44.038866+0000","last_became_peered":"2023-09-23T03:27:44.038866+0000","last_unstale":"2023-09-23T03:27:45.044236+0000","last_undegraded":"2023-09-23T03:27:45.044236+0000","last_fullsized":"2023-09-23T03:27:45.044236+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:03:40.407589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.382200+0000","last_change":"2023-09-23T03:27:45.034553+0000","last_active":"2023-09-23T03:27:47.382200+0000","last_peered":"2023-09-23T03:27:47.382200+0000","last_clean":"2023-09-23T03:27:47.382200+0000","last_became_active":"2023-09-23T03:27:45.034228+0000","last_became_peered":"2023-09-23T03:27:45.034228+0000","last_unstale":"2023-09-23T03:27:47.382200+0000","last_undegraded":"2023-09-23T03:27:47.382200+0000","last_fullsized":"2023-09-23T03:27:47.382200+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:48:13.711973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"125'1","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.042990+0000","last_change":"2023-09-23T03:27:44.036216+0000","last_active":"2023-09-23T03:27:45.042990+0000","last_peered":"2023-09-23T03:27:45.042990+0000","last_clean":"2023-09-23T03:27:45.042990+0000","last_became_active":"2023-09-23T03:27:44.035843+0000","last_became_peered":"2023-09-23T03:27:44.035843+0000","last_unstale":"2023-09-23T03:27:45.042990+0000","last_undegraded":"2023-09-23T03:27:45.042990+0000","last_fullsized":"2023-09-23T03:27:45.042990+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:53:31.135007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.380965+0000","last_change":"2023-09-23T03:27:45.038771+0000","last_active":"2023-09-23T03:27:47.380965+0000","last_peered":"2023-09-23T03:27:47.380965+0000","last_clean":"2023-09-23T03:27:47.380965+0000","last_became_active":"2023-09-23T03:27:45.038473+0000","last_became_peered":"2023-09-23T03:27:45.038473+0000","last_unstale":"2023-09-23T03:27:47.380965+0000","last_undegraded":"2023-09-23T03:27:47.380965+0000","last_fullsized":"2023-09-23T03:27:47.380965+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:56:03.506431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"125'1","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.048899+0000","last_change":"2023-09-23T03:27:44.032435+0000","last_active":"2023-09-23T03:27:45.048899+0000","last_peered":"2023-09-23T03:27:45.048899+0000","last_clean":"2023-09-23T03:27:45.048899+0000","last_became_active":"2023-09-23T03:27:44.032184+0000","last_became_peered":"2023-09-23T03:27:44.032184+0000","last_unstale":"2023-09-23T03:27:45.048899+0000","last_undegraded":"2023-09-23T03:27:45.048899+0000","last_fullsized":"2023-09-23T03:27:45.048899+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:32:20.075247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.382060+0000","last_change":"2023-09-23T03:27:45.026770+0000","last_active":"2023-09-23T03:27:47.382060+0000","last_peered":"2023-09-23T03:27:47.382060+0000","last_clean":"2023-09-23T03:27:47.382060+0000","last_became_active":"2023-09-23T03:27:45.026469+0000","last_became_peered":"2023-09-23T03:27:45.026469+0000","last_unstale":"2023-09-23T03:27:47.382060+0000","last_undegraded":"2023-09-23T03:27:47.382060+0000","last_fullsized":"2023-09-23T03:27:47.382060+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:16:08.067236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.015222+0000","last_change":"2023-09-23T03:27:44.025904+0000","last_active":"2023-09-23T03:27:45.015222+0000","last_peered":"2023-09-23T03:27:45.015222+0000","last_clean":"2023-09-23T03:27:45.015222+0000","last_became_active":"2023-09-23T03:27:44.025435+0000","last_became_peered":"2023-09-23T03:27:44.025435+0000","last_unstale":"2023-09-23T03:27:45.015222+0000","last_undegraded":"2023-09-23T03:27:45.015222+0000","last_fullsized":"2023-09-23T03:27:45.015222+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:35:44.025775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.382670+0000","last_change":"2023-09-23T03:27:45.025083+0000","last_active":"2023-09-23T03:27:47.382670+0000","last_peered":"2023-09-23T03:27:47.382670+0000","last_clean":"2023-09-23T03:27:47.382670+0000","last_became_active":"2023-09-23T03:27:45.024766+0000","last_became_peered":"2023-09-23T03:27:45.024766+0000","last_unstale":"2023-09-23T03:27:47.382670+0000","last_undegraded":"2023-09-23T03:27:47.382670+0000","last_fullsized":"2023-09-23T03:27:47.382670+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:41:47.631539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.008645+0000","last_change":"2023-09-23T03:27:44.039337+0000","last_active":"2023-09-23T03:27:45.008645+0000","last_peered":"2023-09-23T03:27:45.008645+0000","last_clean":"2023-09-23T03:27:45.008645+0000","last_became_active":"2023-09-23T03:27:44.039053+0000","last_became_peered":"2023-09-23T03:27:44.039053+0000","last_unstale":"2023-09-23T03:27:45.008645+0000","last_undegraded":"2023-09-23T03:27:45.008645+0000","last_fullsized":"2023-09-23T03:27:45.008645+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:54:00.914666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:46.035160+0000","last_change":"2023-09-23T03:27:45.040008+0000","last_active":"2023-09-23T03:27:46.035160+0000","last_peered":"2023-09-23T03:27:46.035160+0000","last_clean":"2023-09-23T03:27:46.035160+0000","last_became_active":"2023-09-23T03:27:45.039684+0000","last_became_peered":"2023-09-23T03:27:45.039684+0000","last_unstale":"2023-09-23T03:27:46.035160+0000","last_undegraded":"2023-09-23T03:27:46.035160+0000","last_fullsized":"2023-09-23T03:27:46.035160+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:00:15.928606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.010215+0000","last_change":"2023-09-23T03:27:44.031054+0000","last_active":"2023-09-23T03:27:45.010215+0000","last_peered":"2023-09-23T03:27:45.010215+0000","last_clean":"2023-09-23T03:27:45.010215+0000","last_became_active":"2023-09-23T03:27:44.030699+0000","last_became_peered":"2023-09-23T03:27:44.030699+0000","last_unstale":"2023-09-23T03:27:45.010215+0000","last_undegraded":"2023-09-23T03:27:45.010215+0000","last_fullsized":"2023-09-23T03:27:45.010215+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:38:50.544329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.381722+0000","last_change":"2023-09-23T03:27:45.028306+0000","last_active":"2023-09-23T03:27:47.381722+0000","last_peered":"2023-09-23T03:27:47.381722+0000","last_clean":"2023-09-23T03:27:47.381722+0000","last_became_active":"2023-09-23T03:27:45.027966+0000","last_became_peered":"2023-09-23T03:27:45.027966+0000","last_unstale":"2023-09-23T03:27:47.381722+0000","last_undegraded":"2023-09-23T03:27:47.381722+0000","last_fullsized":"2023-09-23T03:27:47.381722+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:32:38.141959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.021373+0000","last_change":"2023-09-23T03:27:44.031597+0000","last_active":"2023-09-23T03:27:45.021373+0000","last_peered":"2023-09-23T03:27:45.021373+0000","last_clean":"2023-09-23T03:27:45.021373+0000","last_became_active":"2023-09-23T03:27:44.030611+0000","last_became_peered":"2023-09-23T03:27:44.030611+0000","last_unstale":"2023-09-23T03:27:45.021373+0000","last_undegraded":"2023-09-23T03:27:45.021373+0000","last_fullsized":"2023-09-23T03:27:45.021373+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:14:09.639441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.384022+0000","last_change":"2023-09-23T03:27:45.046861+0000","last_active":"2023-09-23T03:27:47.384022+0000","last_peered":"2023-09-23T03:27:47.384022+0000","last_clean":"2023-09-23T03:27:47.384022+0000","last_became_active":"2023-09-23T03:27:45.046591+0000","last_became_peered":"2023-09-23T03:27:45.046591+0000","last_unstale":"2023-09-23T03:27:47.384022+0000","last_undegraded":"2023-09-23T03:27:47.384022+0000","last_fullsized":"2023-09-23T03:27:47.384022+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:26:00.555159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"125'3","reported_seq":15,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.050842+0000","last_change":"2023-09-23T03:27:44.028308+0000","last_active":"2023-09-23T03:27:45.050842+0000","last_peered":"2023-09-23T03:27:45.050842+0000","last_clean":"2023-09-23T03:27:45.050842+0000","last_became_active":"2023-09-23T03:27:44.026049+0000","last_became_peered":"2023-09-23T03:27:44.026049+0000","last_unstale":"2023-09-23T03:27:45.050842+0000","last_undegraded":"2023-09-23T03:27:45.050842+0000","last_fullsized":"2023-09-23T03:27:45.050842+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:38:50.544329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.382391+0000","last_change":"2023-09-23T03:27:45.032354+0000","last_active":"2023-09-23T03:27:47.382391+0000","last_peered":"2023-09-23T03:27:47.382391+0000","last_clean":"2023-09-23T03:27:47.382391+0000","last_became_active":"2023-09-23T03:27:45.032076+0000","last_became_peered":"2023-09-23T03:27:45.032076+0000","last_unstale":"2023-09-23T03:27:47.382391+0000","last_undegraded":"2023-09-23T03:27:47.382391+0000","last_fullsized":"2023-09-23T03:27:47.382391+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:28:20.892702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"125'6","reported_seq":18,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.050048+0000","last_change":"2023-09-23T03:27:44.030573+0000","last_active":"2023-09-23T03:27:45.050048+0000","last_peered":"2023-09-23T03:27:45.050048+0000","last_clean":"2023-09-23T03:27:45.050048+0000","last_became_active":"2023-09-23T03:27:44.030296+0000","last_became_peered":"2023-09-23T03:27:44.030296+0000","last_unstale":"2023-09-23T03:27:45.050048+0000","last_undegraded":"2023-09-23T03:27:45.050048+0000","last_fullsized":"2023-09-23T03:27:45.050048+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:42:53.815632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:46.035459+0000","last_change":"2023-09-23T03:27:45.029851+0000","last_active":"2023-09-23T03:27:46.035459+0000","last_peered":"2023-09-23T03:27:46.035459+0000","last_clean":"2023-09-23T03:27:46.035459+0000","last_became_active":"2023-09-23T03:27:45.029562+0000","last_became_peered":"2023-09-23T03:27:45.029562+0000","last_unstale":"2023-09-23T03:27:46.035459+0000","last_undegraded":"2023-09-23T03:27:46.035459+0000","last_fullsized":"2023-09-23T03:27:46.035459+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:46:09.395570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"125'2","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.051392+0000","last_change":"2023-09-23T03:27:44.034874+0000","last_active":"2023-09-23T03:27:45.051392+0000","last_peered":"2023-09-23T03:27:45.051392+0000","last_clean":"2023-09-23T03:27:45.051392+0000","last_became_active":"2023-09-23T03:27:44.034585+0000","last_became_peered":"2023-09-23T03:27:44.034585+0000","last_unstale":"2023-09-23T03:27:45.051392+0000","last_undegraded":"2023-09-23T03:27:45.051392+0000","last_fullsized":"2023-09-23T03:27:45.051392+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:40:35.511461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.382427+0000","last_change":"2023-09-23T03:27:45.038478+0000","last_active":"2023-09-23T03:27:47.382427+0000","last_peered":"2023-09-23T03:27:47.382427+0000","last_clean":"2023-09-23T03:27:47.382427+0000","last_became_active":"2023-09-23T03:27:45.038185+0000","last_became_peered":"2023-09-23T03:27:45.038185+0000","last_unstale":"2023-09-23T03:27:47.382427+0000","last_undegraded":"2023-09-23T03:27:47.382427+0000","last_fullsized":"2023-09-23T03:27:47.382427+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:56:03.506431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.018205+0000","last_change":"2023-09-23T03:27:44.032000+0000","last_active":"2023-09-23T03:27:45.018205+0000","last_peered":"2023-09-23T03:27:45.018205+0000","last_clean":"2023-09-23T03:27:45.018205+0000","last_became_active":"2023-09-23T03:27:44.031230+0000","last_became_peered":"2023-09-23T03:27:44.031230+0000","last_unstale":"2023-09-23T03:27:45.018205+0000","last_undegraded":"2023-09-23T03:27:45.018205+0000","last_fullsized":"2023-09-23T03:27:45.018205+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:48:36.568201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:46.032158+0000","last_change":"2023-09-23T03:27:45.021451+0000","last_active":"2023-09-23T03:27:46.032158+0000","last_peered":"2023-09-23T03:27:46.032158+0000","last_clean":"2023-09-23T03:27:46.032158+0000","last_became_active":"2023-09-23T03:27:45.021179+0000","last_became_peered":"2023-09-23T03:27:45.021179+0000","last_unstale":"2023-09-23T03:27:46.032158+0000","last_undegraded":"2023-09-23T03:27:46.032158+0000","last_fullsized":"2023-09-23T03:27:46.032158+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T13:02:06.668404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.016083+0000","last_change":"2023-09-23T03:27:44.032754+0000","last_active":"2023-09-23T03:27:45.016083+0000","last_peered":"2023-09-23T03:27:45.016083+0000","last_clean":"2023-09-23T03:27:45.016083+0000","last_became_active":"2023-09-23T03:27:44.032391+0000","last_became_peered":"2023-09-23T03:27:44.032391+0000","last_unstale":"2023-09-23T03:27:45.016083+0000","last_undegraded":"2023-09-23T03:27:45.016083+0000","last_fullsized":"2023-09-23T03:27:45.016083+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:34:44.915726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.381543+0000","last_change":"2023-09-23T03:27:45.042388+0000","last_active":"2023-09-23T03:27:47.381543+0000","last_peered":"2023-09-23T03:27:47.381543+0000","last_clean":"2023-09-23T03:27:47.381543+0000","last_became_active":"2023-09-23T03:27:45.041630+0000","last_became_peered":"2023-09-23T03:27:45.041630+0000","last_unstale":"2023-09-23T03:27:47.381543+0000","last_undegraded":"2023-09-23T03:27:47.381543+0000","last_fullsized":"2023-09-23T03:27:47.381543+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:30:18.768868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"125'1","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.042729+0000","last_change":"2023-09-23T03:27:44.042829+0000","last_active":"2023-09-23T03:27:45.042729+0000","last_peered":"2023-09-23T03:27:45.042729+0000","last_clean":"2023-09-23T03:27:45.042729+0000","last_became_active":"2023-09-23T03:27:44.042547+0000","last_became_peered":"2023-09-23T03:27:44.042547+0000","last_unstale":"2023-09-23T03:27:45.042729+0000","last_undegraded":"2023-09-23T03:27:45.042729+0000","last_fullsized":"2023-09-23T03:27:45.042729+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:53:31.135007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.382446+0000","last_change":"2023-09-23T03:27:45.026059+0000","last_active":"2023-09-23T03:27:47.382446+0000","last_peered":"2023-09-23T03:27:47.382446+0000","last_clean":"2023-09-23T03:27:47.382446+0000","last_became_active":"2023-09-23T03:27:45.025620+0000","last_became_peered":"2023-09-23T03:27:45.025620+0000","last_unstale":"2023-09-23T03:27:47.382446+0000","last_undegraded":"2023-09-23T03:27:47.382446+0000","last_fullsized":"2023-09-23T03:27:47.382446+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:07:46.010795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"125'2","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.045073+0000","last_change":"2023-09-23T03:27:44.028974+0000","last_active":"2023-09-23T03:27:45.045073+0000","last_peered":"2023-09-23T03:27:45.045073+0000","last_clean":"2023-09-23T03:27:45.045073+0000","last_became_active":"2023-09-23T03:27:44.028112+0000","last_became_peered":"2023-09-23T03:27:44.028112+0000","last_unstale":"2023-09-23T03:27:45.045073+0000","last_undegraded":"2023-09-23T03:27:45.045073+0000","last_fullsized":"2023-09-23T03:27:45.045073+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:53:31.135007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.381049+0000","last_change":"2023-09-23T03:27:45.031971+0000","last_active":"2023-09-23T03:27:47.381049+0000","last_peered":"2023-09-23T03:27:47.381049+0000","last_clean":"2023-09-23T03:27:47.381049+0000","last_became_active":"2023-09-23T03:27:45.031685+0000","last_became_peered":"2023-09-23T03:27:45.031685+0000","last_unstale":"2023-09-23T03:27:47.381049+0000","last_undegraded":"2023-09-23T03:27:47.381049+0000","last_fullsized":"2023-09-23T03:27:47.381049+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:20:18.634153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"125'1","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.044083+0000","last_change":"2023-09-23T03:27:44.032617+0000","last_active":"2023-09-23T03:27:45.044083+0000","last_peered":"2023-09-23T03:27:45.044083+0000","last_clean":"2023-09-23T03:27:45.044083+0000","last_became_active":"2023-09-23T03:27:44.031990+0000","last_became_peered":"2023-09-23T03:27:44.031990+0000","last_unstale":"2023-09-23T03:27:45.044083+0000","last_undegraded":"2023-09-23T03:27:45.044083+0000","last_fullsized":"2023-09-23T03:27:45.044083+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:00:14.855143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.382625+0000","last_change":"2023-09-23T03:27:45.038013+0000","last_active":"2023-09-23T03:27:47.382625+0000","last_peered":"2023-09-23T03:27:47.382625+0000","last_clean":"2023-09-23T03:27:47.382625+0000","last_became_active":"2023-09-23T03:27:45.037340+0000","last_became_peered":"2023-09-23T03:27:45.037340+0000","last_unstale":"2023-09-23T03:27:47.382625+0000","last_undegraded":"2023-09-23T03:27:47.382625+0000","last_fullsized":"2023-09-23T03:27:47.382625+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:18:20.232803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.027210+0000","last_change":"2023-09-23T03:27:44.036175+0000","last_active":"2023-09-23T03:27:45.027210+0000","last_peered":"2023-09-23T03:27:45.027210+0000","last_clean":"2023-09-23T03:27:45.027210+0000","last_became_active":"2023-09-23T03:27:44.035896+0000","last_became_peered":"2023-09-23T03:27:44.035896+0000","last_unstale":"2023-09-23T03:27:45.027210+0000","last_undegraded":"2023-09-23T03:27:45.027210+0000","last_fullsized":"2023-09-23T03:27:45.027210+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:48:36.568201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.380498+0000","last_change":"2023-09-23T03:27:45.023094+0000","last_active":"2023-09-23T03:27:47.380498+0000","last_peered":"2023-09-23T03:27:47.380498+0000","last_clean":"2023-09-23T03:27:47.380498+0000","last_became_active":"2023-09-23T03:27:45.019872+0000","last_became_peered":"2023-09-23T03:27:45.019872+0000","last_unstale":"2023-09-23T03:27:47.380498+0000","last_undegraded":"2023-09-23T03:27:47.380498+0000","last_fullsized":"2023-09-23T03:27:47.380498+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:07:16.989269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"125'2","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.050347+0000","last_change":"2023-09-23T03:27:44.027098+0000","last_active":"2023-09-23T03:27:45.050347+0000","last_peered":"2023-09-23T03:27:45.050347+0000","last_clean":"2023-09-23T03:27:45.050347+0000","last_became_active":"2023-09-23T03:27:44.026793+0000","last_became_peered":"2023-09-23T03:27:44.026793+0000","last_unstale":"2023-09-23T03:27:45.050347+0000","last_undegraded":"2023-09-23T03:27:45.050347+0000","last_fullsized":"2023-09-23T03:27:45.050347+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:08:30.847042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.382067+0000","last_change":"2023-09-23T03:27:45.031167+0000","last_active":"2023-09-23T03:27:47.382067+0000","last_peered":"2023-09-23T03:27:47.382067+0000","last_clean":"2023-09-23T03:27:47.382067+0000","last_became_active":"2023-09-23T03:27:45.030906+0000","last_became_peered":"2023-09-23T03:27:45.030906+0000","last_unstale":"2023-09-23T03:27:47.382067+0000","last_undegraded":"2023-09-23T03:27:47.382067+0000","last_fullsized":"2023-09-23T03:27:47.382067+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:59:59.766177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.015945+0000","last_change":"2023-09-23T03:27:44.038407+0000","last_active":"2023-09-23T03:27:45.015945+0000","last_peered":"2023-09-23T03:27:45.015945+0000","last_clean":"2023-09-23T03:27:45.015945+0000","last_became_active":"2023-09-23T03:27:44.038137+0000","last_became_peered":"2023-09-23T03:27:44.038137+0000","last_unstale":"2023-09-23T03:27:45.015945+0000","last_undegraded":"2023-09-23T03:27:45.015945+0000","last_fullsized":"2023-09-23T03:27:45.015945+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T09:14:43.859672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.382389+0000","last_change":"2023-09-23T03:27:45.040671+0000","last_active":"2023-09-23T03:27:47.382389+0000","last_peered":"2023-09-23T03:27:47.382389+0000","last_clean":"2023-09-23T03:27:47.382389+0000","last_became_active":"2023-09-23T03:27:45.040373+0000","last_became_peered":"2023-09-23T03:27:45.040373+0000","last_unstale":"2023-09-23T03:27:47.382389+0000","last_undegraded":"2023-09-23T03:27:47.382389+0000","last_fullsized":"2023-09-23T03:27:47.382389+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:51:53.201001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.016301+0000","last_change":"2023-09-23T03:27:44.028929+0000","last_active":"2023-09-23T03:27:45.016301+0000","last_peered":"2023-09-23T03:27:45.016301+0000","last_clean":"2023-09-23T03:27:45.016301+0000","last_became_active":"2023-09-23T03:27:44.028642+0000","last_became_peered":"2023-09-23T03:27:44.028642+0000","last_unstale":"2023-09-23T03:27:45.016301+0000","last_undegraded":"2023-09-23T03:27:45.016301+0000","last_fullsized":"2023-09-23T03:27:45.016301+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:06:02.129656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.381182+0000","last_change":"2023-09-23T03:27:45.031216+0000","last_active":"2023-09-23T03:27:47.381182+0000","last_peered":"2023-09-23T03:27:47.381182+0000","last_clean":"2023-09-23T03:27:47.381182+0000","last_became_active":"2023-09-23T03:27:45.030947+0000","last_became_peered":"2023-09-23T03:27:45.030947+0000","last_unstale":"2023-09-23T03:27:47.381182+0000","last_undegraded":"2023-09-23T03:27:47.381182+0000","last_fullsized":"2023-09-23T03:27:47.381182+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T15:16:05.766908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.019142+0000","last_change":"2023-09-23T03:27:44.036596+0000","last_active":"2023-09-23T03:27:45.019142+0000","last_peered":"2023-09-23T03:27:45.019142+0000","last_clean":"2023-09-23T03:27:45.019142+0000","last_became_active":"2023-09-23T03:27:44.035882+0000","last_became_peered":"2023-09-23T03:27:44.035882+0000","last_unstale":"2023-09-23T03:27:45.019142+0000","last_undegraded":"2023-09-23T03:27:45.019142+0000","last_fullsized":"2023-09-23T03:27:45.019142+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:26:31.860786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.383264+0000","last_change":"2023-09-23T03:27:45.030275+0000","last_active":"2023-09-23T03:27:47.383264+0000","last_peered":"2023-09-23T03:27:47.383264+0000","last_clean":"2023-09-23T03:27:47.383264+0000","last_became_active":"2023-09-23T03:27:45.030016+0000","last_became_peered":"2023-09-23T03:27:45.030016+0000","last_unstale":"2023-09-23T03:27:47.383264+0000","last_undegraded":"2023-09-23T03:27:47.383264+0000","last_fullsized":"2023-09-23T03:27:47.383264+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:13:09.353030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"125'2","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.045622+0000","last_change":"2023-09-23T03:27:44.037493+0000","last_active":"2023-09-23T03:27:45.045622+0000","last_peered":"2023-09-23T03:27:45.045622+0000","last_clean":"2023-09-23T03:27:45.045622+0000","last_became_active":"2023-09-23T03:27:44.037196+0000","last_became_peered":"2023-09-23T03:27:44.037196+0000","last_unstale":"2023-09-23T03:27:45.045622+0000","last_undegraded":"2023-09-23T03:27:45.045622+0000","last_fullsized":"2023-09-23T03:27:45.045622+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:53:45.660506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.382970+0000","last_change":"2023-09-23T03:27:45.034757+0000","last_active":"2023-09-23T03:27:47.382970+0000","last_peered":"2023-09-23T03:27:47.382970+0000","last_clean":"2023-09-23T03:27:47.382970+0000","last_became_active":"2023-09-23T03:27:45.034484+0000","last_became_peered":"2023-09-23T03:27:45.034484+0000","last_unstale":"2023-09-23T03:27:47.382970+0000","last_undegraded":"2023-09-23T03:27:47.382970+0000","last_fullsized":"2023-09-23T03:27:47.382970+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:45:05.020375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.009841+0000","last_change":"2023-09-23T03:27:44.031734+0000","last_active":"2023-09-23T03:27:45.009841+0000","last_peered":"2023-09-23T03:27:45.009841+0000","last_clean":"2023-09-23T03:27:45.009841+0000","last_became_active":"2023-09-23T03:27:44.031447+0000","last_became_peered":"2023-09-23T03:27:44.031447+0000","last_unstale":"2023-09-23T03:27:45.009841+0000","last_undegraded":"2023-09-23T03:27:45.009841+0000","last_fullsized":"2023-09-23T03:27:45.009841+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:14:09.639441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:46.033335+0000","last_change":"2023-09-23T03:27:45.041957+0000","last_active":"2023-09-23T03:27:46.033335+0000","last_peered":"2023-09-23T03:27:46.033335+0000","last_clean":"2023-09-23T03:27:46.033335+0000","last_became_active":"2023-09-23T03:27:45.041690+0000","last_became_peered":"2023-09-23T03:27:45.041690+0000","last_unstale":"2023-09-23T03:27:46.033335+0000","last_undegraded":"2023-09-23T03:27:46.033335+0000","last_fullsized":"2023-09-23T03:27:46.033335+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T10:40:45.625728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.010370+0000","last_change":"2023-09-23T03:27:44.032549+0000","last_active":"2023-09-23T03:27:45.010370+0000","last_peered":"2023-09-23T03:27:45.010370+0000","last_clean":"2023-09-23T03:27:45.010370+0000","last_became_active":"2023-09-23T03:27:44.032241+0000","last_became_peered":"2023-09-23T03:27:44.032241+0000","last_unstale":"2023-09-23T03:27:45.010370+0000","last_undegraded":"2023-09-23T03:27:45.010370+0000","last_fullsized":"2023-09-23T03:27:45.010370+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:19:10.042252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:46.032658+0000","last_change":"2023-09-23T03:27:45.029784+0000","last_active":"2023-09-23T03:27:46.032658+0000","last_peered":"2023-09-23T03:27:46.032658+0000","last_clean":"2023-09-23T03:27:46.032658+0000","last_became_active":"2023-09-23T03:27:45.029468+0000","last_became_peered":"2023-09-23T03:27:45.029468+0000","last_unstale":"2023-09-23T03:27:46.032658+0000","last_undegraded":"2023-09-23T03:27:46.032658+0000","last_fullsized":"2023-09-23T03:27:46.032658+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:48:13.711973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.012979+0000","last_change":"2023-09-23T03:27:44.039309+0000","last_active":"2023-09-23T03:27:45.012979+0000","last_peered":"2023-09-23T03:27:45.012979+0000","last_clean":"2023-09-23T03:27:45.012979+0000","last_became_active":"2023-09-23T03:27:44.039035+0000","last_became_peered":"2023-09-23T03:27:44.039035+0000","last_unstale":"2023-09-23T03:27:45.012979+0000","last_undegraded":"2023-09-23T03:27:45.012979+0000","last_fullsized":"2023-09-23T03:27:45.012979+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:40:41.002962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.380647+0000","last_change":"2023-09-23T03:27:45.023059+0000","last_active":"2023-09-23T03:27:47.380647+0000","last_peered":"2023-09-23T03:27:47.380647+0000","last_clean":"2023-09-23T03:27:47.380647+0000","last_became_active":"2023-09-23T03:27:45.021471+0000","last_became_peered":"2023-09-23T03:27:45.021471+0000","last_unstale":"2023-09-23T03:27:47.380647+0000","last_undegraded":"2023-09-23T03:27:47.380647+0000","last_fullsized":"2023-09-23T03:27:47.380647+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:37:39.211653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.012969+0000","last_change":"2023-09-23T03:27:44.032738+0000","last_active":"2023-09-23T03:27:45.012969+0000","last_peered":"2023-09-23T03:27:45.012969+0000","last_clean":"2023-09-23T03:27:45.012969+0000","last_became_active":"2023-09-23T03:27:44.032424+0000","last_became_peered":"2023-09-23T03:27:44.032424+0000","last_unstale":"2023-09-23T03:27:45.012969+0000","last_undegraded":"2023-09-23T03:27:45.012969+0000","last_fullsized":"2023-09-23T03:27:45.012969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:24:07.607733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.382411+0000","last_change":"2023-09-23T03:27:45.043661+0000","last_active":"2023-09-23T03:27:47.382411+0000","last_peered":"2023-09-23T03:27:47.382411+0000","last_clean":"2023-09-23T03:27:47.382411+0000","last_became_active":"2023-09-23T03:27:45.043378+0000","last_became_peered":"2023-09-23T03:27:45.043378+0000","last_unstale":"2023-09-23T03:27:47.382411+0000","last_undegraded":"2023-09-23T03:27:47.382411+0000","last_fullsized":"2023-09-23T03:27:47.382411+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T06:07:35.140412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.014929+0000","last_change":"2023-09-23T03:27:44.034445+0000","last_active":"2023-09-23T03:27:45.014929+0000","last_peered":"2023-09-23T03:27:45.014929+0000","last_clean":"2023-09-23T03:27:45.014929+0000","last_became_active":"2023-09-23T03:27:44.033651+0000","last_became_peered":"2023-09-23T03:27:44.033651+0000","last_unstale":"2023-09-23T03:27:45.014929+0000","last_undegraded":"2023-09-23T03:27:45.014929+0000","last_fullsized":"2023-09-23T03:27:45.014929+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T03:48:12.700185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:46.032713+0000","last_change":"2023-09-23T03:27:45.035388+0000","last_active":"2023-09-23T03:27:46.032713+0000","last_peered":"2023-09-23T03:27:46.032713+0000","last_clean":"2023-09-23T03:27:46.032713+0000","last_became_active":"2023-09-23T03:27:45.035126+0000","last_became_peered":"2023-09-23T03:27:45.035126+0000","last_unstale":"2023-09-23T03:27:46.032713+0000","last_undegraded":"2023-09-23T03:27:46.032713+0000","last_fullsized":"2023-09-23T03:27:46.032713+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T07:11:36.520222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.009779+0000","last_change":"2023-09-23T03:27:44.038626+0000","last_active":"2023-09-23T03:27:45.009779+0000","last_peered":"2023-09-23T03:27:45.009779+0000","last_clean":"2023-09-23T03:27:45.009779+0000","last_became_active":"2023-09-23T03:27:44.038353+0000","last_became_peered":"2023-09-23T03:27:44.038353+0000","last_unstale":"2023-09-23T03:27:45.009779+0000","last_undegraded":"2023-09-23T03:27:45.009779+0000","last_fullsized":"2023-09-23T03:27:45.009779+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T12:06:02.927861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:47.382719+0000","last_change":"2023-09-23T03:27:45.038565+0000","last_active":"2023-09-23T03:27:47.382719+0000","last_peered":"2023-09-23T03:27:47.382719+0000","last_clean":"2023-09-23T03:27:47.382719+0000","last_became_active":"2023-09-23T03:27:45.038247+0000","last_became_peered":"2023-09-23T03:27:45.038247+0000","last_unstale":"2023-09-23T03:27:47.382719+0000","last_undegraded":"2023-09-23T03:27:47.382719+0000","last_fullsized":"2023-09-23T03:27:47.382719+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:58:00.149687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.019286+0000","last_change":"2023-09-23T03:27:44.036392+0000","last_active":"2023-09-23T03:27:45.019286+0000","last_peered":"2023-09-23T03:27:45.019286+0000","last_clean":"2023-09-23T03:27:45.019286+0000","last_became_active":"2023-09-23T03:27:44.035456+0000","last_became_peered":"2023-09-23T03:27:44.035456+0000","last_unstale":"2023-09-23T03:27:45.019286+0000","last_undegraded":"2023-09-23T03:27:45.019286+0000","last_fullsized":"2023-09-23T03:27:45.019286+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T14:43:59.744799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-09-23T03:27:46.033738+0000","last_change":"2023-09-23T03:27:45.043461+0000","last_active":"2023-09-23T03:27:46.033738+0000","last_peered":"2023-09-23T03:27:46.033738+0000","last_clean":"2023-09-23T03:27:46.033738+0000","last_became_active":"2023-09-23T03:27:45.043112+0000","last_became_peered":"2023-09-23T03:27:45.043112+0000","last_unstale":"2023-09-23T03:27:46.033738+0000","last_undegraded":"2023-09-23T03:27:46.033738+0000","last_fullsized":"2023-09-23T03:27:46.033738+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:43.980844+0000","last_clean_scrub_stamp":"2023-09-23T03:27:43.980844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T05:59:59.766177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"125'8","reported_seq":20,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.051953+0000","last_change":"2023-09-23T03:27:44.034373+0000","last_active":"2023-09-23T03:27:45.051953+0000","last_peered":"2023-09-23T03:27:45.051953+0000","last_clean":"2023-09-23T03:27:45.051953+0000","last_became_active":"2023-09-23T03:27:44.033040+0000","last_became_peered":"2023-09-23T03:27:44.033040+0000","last_unstale":"2023-09-23T03:27:45.051953+0000","last_undegraded":"2023-09-23T03:27:45.051953+0000","last_fullsized":"2023-09-23T03:27:45.051953+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:27:42.969055+0000","last_clean_scrub_stamp":"2023-09-23T03:27:42.969055+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T08:00:14.855143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":257,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-09-23T03:27:45.019485+0000","last_change":"2023-09-23T03:15:11.042447+0000","last_active":"2023-09-23T03:27:45.019485+0000","last_peered":"2023-09-23T03:27:45.019485+0000","last_clean":"2023-09-23T03:27:45.019485+0000","last_became_active":"2023-09-23T03:15:11.040322+0000","last_became_peered":"2023-09-23T03:15:11.040322+0000","last_unstale":"2023-09-23T03:27:45.019485+0000","last_undegraded":"2023-09-23T03:27:45.019485+0000","last_fullsized":"2023-09-23T03:27:45.019485+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_clean_scrub_stamp":"2023-09-23T03:15:09.474842+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:18:36.331645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575002,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33312,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338528,"statfs":{"total":96636764160,"available":96602652672,"internally_reserved":0,"allocated":2179072,"data_stored":1357674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:27:33 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.0509999999999999,"15min":1.022},"min":{"1min":0.81699999999999995,"5min":0.57999999999999996,"15min":0.442},"max":{"1min":1.736,"5min":2.0299999999999998,"15min":2.149},"last":1.5880000000000001},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.0720000000000001,"15min":1.0620000000000001},"min":{"1min":0.74199999999999999,"5min":0.63100000000000001,"15min":0.52000000000000002},"max":{"1min":1.617,"5min":1.9910000000000001,"15min":2.1480000000000001},"last":0.91800000000000004}]},{"osd":1,"last update":"Sat Sep 23 03:27:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.214,"15min":1.214},"min":{"1min":0.80600000000000005,"5min":0.66400000000000003,"15min":0.504},"max":{"1min":1.478,"5min":2.6400000000000001,"15min":3.1859999999999999},"last":1.524},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.2729999999999999,"15min":1.294},"min":{"1min":0.79900000000000004,"5min":0.71299999999999997,"15min":0.67900000000000005},"max":{"1min":1.8089999999999999,"5min":2.3149999999999999,"15min":2.613},"last":1.1919999999999999}]},{"osd":2,"last update":"Sat Sep 23 03:27:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.80400000000000005,"5min":0.61199999999999999,"15min":0.42399999999999999},"max":{"1min":1.516,"5min":1.6439999999999999,"15min":2.331},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.248,"5min":1.2509999999999999,"15min":1.2410000000000001},"min":{"1min":1.004,"5min":0.749,"15min":0.55200000000000005},"max":{"1min":1.651,"5min":2.2719999999999998,"15min":3.093},"last":1.2490000000000001}]},{"osd":3,"last update":"Sat Sep 23 03:27:33 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.236,"15min":1.2450000000000001},"min":{"1min":0.89000000000000001,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.464,"5min":2.0840000000000001,"15min":2.532},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.93799999999999994,"5min":0.71899999999999997,"15min":0.69799999999999995},"max":{"1min":1.8089999999999999,"5min":2.4289999999999998,"15min":3.2349999999999999},"last":0.99299999999999999}]},{"osd":4,"last update":"Sat Sep 23 03:27:33 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.292,"15min":1.375},"min":{"1min":0.82599999999999996,"5min":0.57999999999999996,"15min":0.45400000000000001},"max":{"1min":1.605,"5min":2.3260000000000001,"15min":2.7210000000000001},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.4490000000000001,"15min":1.476},"min":{"1min":1.0880000000000001,"5min":0.73599999999999999,"15min":0.48099999999999998},"max":{"1min":1.75,"5min":2.2240000000000002,"15min":3.7719999999999998},"last":1.478}]},{"osd":5,"last update":"Sat Sep 23 03:27:33 2023","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.3959999999999999,"15min":1.4139999999999999},"min":{"1min":0.96999999999999997,"5min":0.749,"15min":0.47999999999999998},"max":{"1min":1.871,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":1.409},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.413,"15min":1.4450000000000001},"min":{"1min":0.95099999999999996,"5min":0.77000000000000002,"15min":0.497},"max":{"1min":1.74,"5min":2.5070000000000001,"15min":2.5070000000000001},"last":1.3089999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:27:33 2023","interfaces":[{"interface":"back","average":{"1min":1.516,"5min":1.5049999999999999,"15min":1.5469999999999999},"min":{"1min":1.1850000000000001,"5min":0.85499999999999998,"15min":0.53800000000000003},"max":{"1min":1.835,"5min":2.649,"15min":2.8420000000000001},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.383,"5min":1.4299999999999999,"15min":1.484},"min":{"1min":0.92100000000000004,"5min":0.78900000000000003,"15min":0.45900000000000002},"max":{"1min":1.667,"5min":2.4500000000000002,"15min":3.6739999999999999},"last":1.1319999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575002,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32720,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1572864,"data_stored":767112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:27:25 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.107,"15min":1.0840000000000001},"min":{"1min":0.66500000000000004,"5min":0.53600000000000003,"15min":0.38},"max":{"1min":1.7789999999999999,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.3360000000000001},{"interface":"front","average":{"1min":0.98399999999999999,"5min":1.091,"15min":1.0720000000000001},"min":{"1min":0.68999999999999995,"5min":0.60999999999999999,"15min":0.46100000000000002},"max":{"1min":1.403,"5min":3.7629999999999999,"15min":3.7629999999999999},"last":0.70099999999999996}]},{"osd":1,"last update":"Sat Sep 23 03:27:25 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.1359999999999999,"15min":1.119},"min":{"1min":0.66500000000000004,"5min":0.63800000000000001,"15min":0.438},"max":{"1min":1.4910000000000001,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.242,"5min":1.1659999999999999,"15min":1.1910000000000001},"min":{"1min":0.70499999999999996,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.593,"5min":2.972,"15min":2.972},"last":1.7350000000000001}]},{"osd":2,"last update":"Sat Sep 23 03:27:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.1579999999999999,"15min":1.1319999999999999},"min":{"1min":0.80100000000000005,"5min":0.56499999999999995,"15min":0.51300000000000001},"max":{"1min":1.4810000000000001,"5min":3.4390000000000001,"15min":3.4390000000000001},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.214,"5min":1.246,"15min":1.2390000000000001},"min":{"1min":0.80700000000000005,"5min":0.68000000000000005,"15min":0.64300000000000002},"max":{"1min":1.5549999999999999,"5min":2.177,"15min":2.177},"last":0.97399999999999998}]},{"osd":3,"last update":"Sat Sep 23 03:27:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.355,"15min":1.337},"min":{"1min":1.01,"5min":0.78500000000000003,"15min":0.54600000000000004},"max":{"1min":1.583,"5min":2.129,"15min":2.6509999999999998},"last":1.6890000000000001},{"interface":"front","average":{"1min":1.294,"5min":1.327,"15min":1.373},"min":{"1min":0.98499999999999999,"5min":0.86199999999999999,"15min":0.74299999999999999},"max":{"1min":1.7190000000000001,"5min":3.0910000000000002,"15min":3.0910000000000002},"last":1.5620000000000001}]},{"osd":4,"last update":"Sat Sep 23 03:27:25 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.3440000000000001,"15min":1.369},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.498},"max":{"1min":1.657,"5min":3.1789999999999998,"15min":3.1789999999999998},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.226,"5min":1.3240000000000001,"15min":1.2989999999999999},"min":{"1min":0.76000000000000001,"5min":0.72399999999999998,"15min":0.56000000000000005},"max":{"1min":1.6200000000000001,"5min":3.6480000000000001,"15min":3.6480000000000001},"last":1.3959999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:27:25 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.429,"15min":1.4099999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6739999999999999,"5min":4.093,"15min":4.093},"last":1.639},{"interface":"front","average":{"1min":1.391,"5min":1.45,"15min":1.4450000000000001},"min":{"1min":0.70599999999999996,"5min":0.69899999999999995,"15min":0.59899999999999998},"max":{"1min":1.669,"5min":2.0459999999999998,"15min":4.758},"last":1.472}]},{"osd":7,"last update":"Sat Sep 23 03:27:25 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.5189999999999999,"15min":1.52},"min":{"1min":0.81100000000000005,"5min":0.76500000000000001,"15min":0.63500000000000001},"max":{"1min":1.712,"5min":3.536,"15min":3.536},"last":1.9570000000000001},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.546,"15min":1.536},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.8720000000000001,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":1.788}]}]},{"osd":5,"up_from":13,"seq":55834575004,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":766690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:27:26 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.054,"15min":1.046},"min":{"1min":0.66000000000000003,"5min":0.61199999999999999,"15min":0.57199999999999995},"max":{"1min":1.5860000000000001,"5min":2.9740000000000002,"15min":2.9740000000000002},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.171,"5min":1.131,"15min":1.081},"min":{"1min":0.73299999999999998,"5min":0.624,"15min":0.61399999999999999},"max":{"1min":1.619,"5min":1.962,"15min":2.2320000000000002},"last":1.383}]},{"osd":1,"last update":"Sat Sep 23 03:27:26 2023","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.21,"15min":1.2210000000000001},"min":{"1min":0.84799999999999998,"5min":0.68300000000000005,"15min":0.62},"max":{"1min":1.5569999999999999,"5min":2.4369999999999998,"15min":2.4369999999999998},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.274,"5min":1.2869999999999999,"15min":1.288},"min":{"1min":0.76800000000000002,"5min":0.68100000000000005,"15min":0.67200000000000004},"max":{"1min":1.5860000000000001,"5min":3.5859999999999999,"15min":3.5859999999999999},"last":1.1990000000000001}]},{"osd":2,"last update":"Sat Sep 23 03:27:26 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.196,"15min":1.2030000000000001},"min":{"1min":0.92800000000000005,"5min":0.65300000000000002,"15min":0.64300000000000002},"max":{"1min":1.593,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":1.129},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.1399999999999999,"15min":1.218},"min":{"1min":0.79500000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.6970000000000001,"5min":2.8500000000000001,"15min":2.8500000000000001},"last":1.0489999999999999}]},{"osd":3,"last update":"Sat Sep 23 03:27:26 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.266,"15min":1.2669999999999999},"min":{"1min":0.96999999999999997,"5min":0.67200000000000004,"15min":0.66400000000000003},"max":{"1min":1.5640000000000001,"5min":3.4359999999999999,"15min":3.4359999999999999},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.3029999999999999,"15min":1.3580000000000001},"min":{"1min":0.98699999999999999,"5min":0.85999999999999999,"15min":0.69799999999999995},"max":{"1min":1.726,"5min":1.9159999999999999,"15min":3.6110000000000002},"last":1.2629999999999999}]},{"osd":4,"last update":"Sat Sep 23 03:27:26 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.401,"15min":1.395},"min":{"1min":1.0429999999999999,"5min":0.68600000000000005,"15min":0.54100000000000004},"max":{"1min":1.7789999999999999,"5min":4.2750000000000004,"15min":4.2750000000000004},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.2929999999999999,"15min":1.3540000000000001},"min":{"1min":0.79200000000000004,"5min":0.54100000000000004,"15min":0.48899999999999999},"max":{"1min":1.7989999999999999,"5min":3.3090000000000002,"15min":3.3090000000000002},"last":1.3300000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:27:26 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.52,"15min":1.5149999999999999},"min":{"1min":1.272,"5min":1.0329999999999999,"15min":0.59199999999999997},"max":{"1min":1.784,"5min":4.6040000000000001,"15min":4.6040000000000001},"last":1.5389999999999999},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4139999999999999,"15min":1.5249999999999999},"min":{"1min":1.0700000000000001,"5min":0.81200000000000006,"15min":0.55100000000000005},"max":{"1min":1.978,"5min":3.169,"15min":9.6560000000000006},"last":1.2330000000000001}]},{"osd":7,"last update":"Sat Sep 23 03:27:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.431,"15min":1.444},"min":{"1min":0.93899999999999995,"5min":0.77100000000000002,"15min":0.626},"max":{"1min":1.819,"5min":2.077,"15min":2.077},"last":1.304},{"interface":"front","average":{"1min":1.51,"5min":1.494,"15min":1.5509999999999999},"min":{"1min":1.2430000000000001,"5min":0.92200000000000004,"15min":0.60799999999999998},"max":{"1min":1.893,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":1.429}]}]},{"osd":4,"up_from":13,"seq":55834575003,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1581056,"data_stored":768142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:27:23 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0640000000000001,"15min":1.1020000000000001},"min":{"1min":0.58499999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.546,"5min":1.6699999999999999,"15min":1.9770000000000001},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1619999999999999,"15min":1.1699999999999999},"min":{"1min":0.68200000000000005,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.724,"5min":2.004,"15min":2.004},"last":0.871}]},{"osd":1,"last update":"Sat Sep 23 03:27:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2250000000000001,"15min":1.2150000000000001},"min":{"1min":0.69699999999999995,"5min":0.65400000000000003,"15min":0.61799999999999999},"max":{"1min":1.8149999999999999,"5min":1.8200000000000001,"15min":2.1110000000000002},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.139,"5min":1.0840000000000001,"15min":1.137},"min":{"1min":0.71599999999999997,"5min":0.67100000000000004,"15min":0.60299999999999998},"max":{"1min":1.581,"5min":1.718,"15min":3.2189999999999999},"last":1.4610000000000001}]},{"osd":2,"last update":"Sat Sep 23 03:27:23 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.167,"15min":1.177},"min":{"1min":0.90400000000000003,"5min":0.72899999999999998,"15min":0.61599999999999999},"max":{"1min":1.5880000000000001,"5min":1.694,"15min":2.5920000000000001},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.306,"5min":1.268,"15min":1.248},"min":{"1min":0.89900000000000002,"5min":0.73499999999999999,"15min":0.68300000000000005},"max":{"1min":1.734,"5min":1.99,"15min":1.99},"last":1.2569999999999999}]},{"osd":3,"last update":"Sat Sep 23 03:27:23 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.161,"15min":1.169},"min":{"1min":0.73399999999999999,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.702,"5min":1.702,"15min":1.9450000000000001},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.2549999999999999,"15min":1.294},"min":{"1min":0.82499999999999996,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.7689999999999999,"5min":1.833,"15min":2.4969999999999999},"last":1.3999999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:27:23 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.355,"15min":1.361},"min":{"1min":0.93200000000000005,"5min":0.84899999999999998,"15min":0.70199999999999996},"max":{"1min":2.0070000000000001,"5min":2.0409999999999999,"15min":2.8050000000000002},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.292,"5min":1.3480000000000001,"15min":1.335},"min":{"1min":0.90900000000000003,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":2.887},"last":1.5349999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:27:23 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.3580000000000001,"15min":1.3720000000000001},"min":{"1min":0.93200000000000005,"5min":0.84699999999999998,"15min":0.66400000000000003},"max":{"1min":1.863,"5min":1.9490000000000001,"15min":2.4249999999999998},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.4750000000000001,"15min":1.4910000000000001},"min":{"1min":1.111,"5min":0.91500000000000004,"15min":0.89900000000000002},"max":{"1min":1.7969999999999999,"5min":2.0939999999999999,"15min":2.1779999999999999},"last":1.3560000000000001}]},{"osd":7,"last update":"Sat Sep 23 03:27:23 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4370000000000001,"15min":1.4590000000000001},"min":{"1min":1.069,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.5680000000000001},"last":1.597},{"interface":"front","average":{"1min":1.514,"5min":1.538,"15min":1.5509999999999999},"min":{"1min":1.268,"5min":0.98899999999999999,"15min":0.84199999999999997},"max":{"1min":1.931,"5min":2.0129999999999999,"15min":2.4769999999999999},"last":1.579}]}]},{"osd":3,"up_from":13,"seq":55834575003,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1585152,"data_stored":768176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.84999999999999998,"15min":0.85799999999999998},"min":{"1min":0.31900000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.121,"5min":1.7050000000000001,"15min":2.952},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.88,"5min":0.89000000000000001,"15min":0.89100000000000001},"min":{"1min":0.54700000000000004,"5min":0.377,"15min":0.29299999999999998},"max":{"1min":1.361,"5min":1.7430000000000001,"15min":1.8979999999999999},"last":0.52700000000000002}]},{"osd":1,"last update":"Sat Sep 23 03:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.81200000000000006,"15min":0.873},"min":{"1min":0.45500000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.359,"5min":1.603,"15min":1.6839999999999999},"last":1.218},{"interface":"front","average":{"1min":0.94299999999999995,"5min":1.002,"15min":0.98799999999999999},"min":{"1min":0.42499999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.5089999999999999,"5min":1.7010000000000001,"15min":2.903},"last":1.482}]},{"osd":2,"last update":"Sat Sep 23 03:27:36 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.022,"15min":1.0509999999999999},"min":{"1min":0.52000000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.669,"5min":1.9850000000000001,"15min":2.2240000000000002},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.099,"5min":1.0860000000000001,"15min":1.103},"min":{"1min":0.61699999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.665,"5min":1.7190000000000001,"15min":3.129},"last":1.0009999999999999}]},{"osd":4,"last update":"Sat Sep 23 03:27:36 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1519999999999999,"15min":1.218},"min":{"1min":0.72799999999999998,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":2.698},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.2549999999999999,"15min":1.339},"min":{"1min":0.876,"5min":0.81200000000000006,"15min":0.752},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":2.419},"last":1.3979999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:27:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.2889999999999999,"15min":1.329},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.7989999999999999,"5min":2.04,"15min":2.9950000000000001},"last":1.143},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.236,"15min":1.278},"min":{"1min":0.68700000000000006,"5min":0.624,"15min":0.624},"max":{"1min":1.6499999999999999,"5min":1.9830000000000001,"15min":2.0070000000000001},"last":1.1699999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:27:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.3149999999999999,"15min":1.349},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.7190000000000001,"5min":1.9530000000000001,"15min":2.6440000000000001},"last":1.542},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3899999999999999,"15min":1.4279999999999999},"min":{"1min":0.81100000000000005,"5min":0.73299999999999998,"15min":0.67800000000000005},"max":{"1min":1.8700000000000001,"5min":2.177,"15min":2.9750000000000001},"last":1.266}]},{"osd":7,"last update":"Sat Sep 23 03:27:36 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.331,"15min":1.379},"min":{"1min":0.80800000000000005,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.597,"5min":2.0859999999999999,"15min":2.7879999999999998},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.3600000000000001,"15min":1.421},"min":{"1min":0.86299999999999999,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.891,"5min":2.0270000000000001,"15min":2.5379999999999998},"last":1.3049999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575004,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32716,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339124,"statfs":{"total":96636764160,"available":96603262976,"internally_reserved":0,"allocated":1568768,"data_stored":766530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.83099999999999996,"15min":0.83899999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.30299999999999999},"max":{"1min":1.4890000000000001,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":1.077,"5min":0.93000000000000005,"15min":0.88600000000000001},"min":{"1min":0.57499999999999996,"5min":0.37,"15min":0.36299999999999999},"max":{"1min":1.901,"5min":2.613,"15min":2.613},"last":0.84299999999999997}]},{"osd":1,"last update":"Sat Sep 23 03:27:25 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.585,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":1.234},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.0189999999999999,"15min":0.97899999999999998},"min":{"1min":0.501,"5min":0.501,"15min":0.38},"max":{"1min":1.498,"5min":2.9809999999999999,"15min":2.9809999999999999},"last":1.367}]},{"osd":3,"last update":"Sat Sep 23 03:27:25 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.0700000000000001,"15min":1.002},"min":{"1min":0.64000000000000001,"5min":0.52100000000000002,"15min":0.40699999999999997},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.119,"5min":1.038,"15min":1.012},"min":{"1min":0.47799999999999998,"5min":0.441,"15min":0.34999999999999998},"max":{"1min":1.7230000000000001,"5min":1.827,"15min":1.827},"last":0.628}]},{"osd":4,"last update":"Sat Sep 23 03:27:25 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.3280000000000001,"15min":1.2989999999999999},"min":{"1min":0.84099999999999997,"5min":0.79400000000000004,"15min":0.55800000000000005},"max":{"1min":1.8260000000000001,"5min":5.6669999999999998,"15min":5.6669999999999998},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.2889999999999999,"15min":1.3},"min":{"1min":0.83899999999999997,"5min":0.82099999999999995,"15min":0.76100000000000001},"max":{"1min":1.6339999999999999,"5min":2.6909999999999998,"15min":2.6909999999999998},"last":1.4910000000000001}]},{"osd":5,"last update":"Sat Sep 23 03:27:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.345,"15min":1.3480000000000001},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.9390000000000001,"5min":2.8969999999999998,"15min":3.9409999999999998},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.403,"15min":1.4199999999999999},"min":{"1min":1.0469999999999999,"5min":0.81299999999999994,"15min":0.56399999999999995},"max":{"1min":1.778,"5min":3.415,"15min":4.9199999999999999},"last":1.2769999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:27:25 2023","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.363,"15min":1.369},"min":{"1min":0.99199999999999999,"5min":0.88,"15min":0.755},"max":{"1min":1.794,"5min":2.7839999999999998,"15min":3.0009999999999999},"last":1.413},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.4730000000000001,"15min":1.4350000000000001},"min":{"1min":0.77100000000000002,"5min":0.76600000000000001,"15min":0.65100000000000002},"max":{"1min":1.839,"5min":7.8300000000000001,"15min":7.8300000000000001},"last":1.4630000000000001}]},{"osd":7,"last update":"Sat Sep 23 03:27:25 2023","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.446,"15min":1.403},"min":{"1min":1.0780000000000001,"5min":0.82899999999999996,"15min":0.61399999999999999},"max":{"1min":1.8180000000000001,"5min":4.0330000000000004,"15min":4.0330000000000004},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.458,"5min":1.4299999999999999,"15min":1.431},"min":{"1min":0.93300000000000005,"5min":0.78600000000000003,"15min":0.74399999999999999},"max":{"1min":1.883,"5min":4.2199999999999998,"15min":4.2199999999999998},"last":1.329}]}]},{"osd":1,"up_from":13,"seq":55834575005,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32716,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339124,"statfs":{"total":96636764160,"available":96603262976,"internally_reserved":0,"allocated":1572864,"data_stored":766552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.80700000000000005,"15min":0.80000000000000004},"min":{"1min":0.56399999999999995,"5min":0.432,"15min":0.32300000000000001},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":2.8279999999999998},"last":0.877},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86399999999999999,"15min":0.84699999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.312},"max":{"1min":1.619,"5min":1.8129999999999999,"15min":1.8919999999999999},"last":1.171}]},{"osd":2,"last update":"Sat Sep 23 03:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.84699999999999998,"15min":0.871},"min":{"1min":0.47799999999999998,"5min":0.46300000000000002,"15min":0.36699999999999999},"max":{"1min":1.2629999999999999,"5min":1.607,"15min":2.3340000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.044,"5min":1.0429999999999999,"15min":1.032},"min":{"1min":0.47599999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5149999999999999,"5min":1.5600000000000001,"15min":1.9570000000000001},"last":1.2749999999999999}]},{"osd":3,"last update":"Sat Sep 23 03:27:36 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":1.036,"15min":1.042},"min":{"1min":0.59299999999999997,"5min":0.504,"15min":0.47299999999999998},"max":{"1min":1.325,"5min":1.7250000000000001,"15min":2.2370000000000001},"last":1.379},{"interface":"front","average":{"1min":1.109,"5min":1.121,"15min":1.115},"min":{"1min":0.54400000000000004,"5min":0.50800000000000001,"15min":0.45900000000000002},"max":{"1min":1.6080000000000001,"5min":1.758,"15min":3.2130000000000001},"last":1.2290000000000001}]},{"osd":4,"last update":"Sat Sep 23 03:27:36 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.278,"15min":1.2569999999999999},"min":{"1min":0.81000000000000005,"5min":0.79900000000000004,"15min":0.70699999999999996},"max":{"1min":1.5640000000000001,"5min":1.885,"15min":3.2629999999999999},"last":1.482},{"interface":"front","average":{"1min":1.224,"5min":1.244,"15min":1.256},"min":{"1min":0.75,"5min":0.75,"15min":0.66500000000000004},"max":{"1min":1.5469999999999999,"5min":1.8460000000000001,"15min":2.4689999999999999},"last":1.333}]},{"osd":5,"last update":"Sat Sep 23 03:27:36 2023","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.288,"15min":1.282},"min":{"1min":0.90200000000000002,"5min":0.76300000000000001,"15min":0.64600000000000002},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":3.161},"last":1.712},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.327,"15min":1.3280000000000001},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.72399999999999998},"max":{"1min":1.6639999999999999,"5min":1.7929999999999999,"15min":2.4289999999999998},"last":1.423}]},{"osd":6,"last update":"Sat Sep 23 03:27:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3300000000000001},"min":{"1min":0.79400000000000004,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.728,"5min":1.823,"15min":3.0179999999999998},"last":1.613},{"interface":"front","average":{"1min":1.369,"5min":1.375,"15min":1.371},"min":{"1min":0.91800000000000004,"5min":0.82499999999999996,"15min":0.72799999999999998},"max":{"1min":1.706,"5min":1.8340000000000001,"15min":3.3580000000000001},"last":1.569}]},{"osd":7,"last update":"Sat Sep 23 03:27:36 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.359,"15min":1.371},"min":{"1min":0.88,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.663,"5min":1.931,"15min":3.1099999999999999},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.397,"5min":1.397,"15min":1.425},"min":{"1min":0.85899999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.73,"5min":1.9299999999999999,"15min":3.306},"last":1.6759999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575005,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33292,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338548,"statfs":{"total":96636764160,"available":96602673152,"internally_reserved":0,"allocated":2162688,"data_stored":1355901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 23 03:27:29 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.81899999999999995,"15min":0.83599999999999997},"min":{"1min":0.56000000000000005,"5min":0.38,"15min":0.30299999999999999},"max":{"1min":1.256,"5min":1.383,"15min":1.919},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.86699999999999999,"15min":0.88800000000000001},"min":{"1min":0.45200000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":2.1360000000000001},"last":0.75}]},{"osd":2,"last update":"Sat Sep 23 03:27:29 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.88100000000000001,"15min":0.84899999999999998},"min":{"1min":0.46999999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.2130000000000001,"5min":1.5249999999999999,"15min":3.089},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.96599999999999997,"15min":1.002},"min":{"1min":0.45700000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.3959999999999999,"5min":1.5760000000000001,"15min":2.633},"last":0.91000000000000003}]},{"osd":3,"last update":"Sat Sep 23 03:27:29 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":0.998,"15min":0.97399999999999998},"min":{"1min":0.65500000000000003,"5min":0.433,"15min":0.433},"max":{"1min":1.4019999999999999,"5min":1.615,"15min":2.8660000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.0129999999999999,"15min":1.014},"min":{"1min":0.57399999999999995,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.591,"5min":1.802,"15min":2.012},"last":0.96699999999999997}]},{"osd":4,"last update":"Sat Sep 23 03:27:29 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.2629999999999999,"15min":1.2609999999999999},"min":{"1min":0.70899999999999996,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.6950000000000001,"5min":1.885,"15min":2.4140000000000001},"last":0.999},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.226,"15min":1.2649999999999999},"min":{"1min":0.76100000000000001,"5min":0.73999999999999999,"15min":0.65300000000000002},"max":{"1min":1.845,"5min":1.976,"15min":3.0230000000000001},"last":0.83199999999999996}]},{"osd":5,"last update":"Sat Sep 23 03:27:29 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.304,"15min":1.27},"min":{"1min":0.94299999999999995,"5min":0.81499999999999995,"15min":0.65600000000000003},"max":{"1min":1.6870000000000001,"5min":2.1240000000000001,"15min":2.5489999999999999},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.425,"5min":1.341,"15min":1.3340000000000001},"min":{"1min":0.77500000000000002,"5min":0.73099999999999998,"15min":0.71999999999999997},"max":{"1min":1.7769999999999999,"5min":1.9159999999999999,"15min":2.1539999999999999},"last":1.079}]},{"osd":6,"last update":"Sat Sep 23 03:27:29 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.3320000000000001,"15min":1.3240000000000001},"min":{"1min":0.86799999999999999,"5min":0.77700000000000002,"15min":0.75600000000000001},"max":{"1min":1.7869999999999999,"5min":2.0870000000000002,"15min":2.9489999999999998},"last":1.151},{"interface":"front","average":{"1min":1.446,"5min":1.4039999999999999,"15min":1.383},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.901,"5min":1.9670000000000001,"15min":2.3239999999999998},"last":1.1279999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:27:29 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.363,"15min":1.3640000000000001},"min":{"1min":0.83699999999999997,"5min":0.752,"15min":0.747},"max":{"1min":2.052,"5min":2.052,"15min":2.2330000000000001},"last":1.02},{"interface":"front","average":{"1min":1.478,"5min":1.419,"15min":1.4330000000000001},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.1419999999999999,"5min":2.1419999999999999,"15min":2.3330000000000002},"last":1.0960000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-23T03:27:50.385 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-09-23T03:27:50.385 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-09-23T03:27:50.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.648+0000 7fe498a1a640 1 Processor -- start 2023-09-23T03:27:50.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.649+0000 7fe498a1a640 1 -- start start 2023-09-23T03:27:50.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.650+0000 7fe498a1a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe4940ad830 0x7fe494070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:50.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.650+0000 7fe498a1a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe4940c9680 0x7fe4940caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:50.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.650+0000 7fe498a1a640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4940712f0 con 0x7fe4940cb400 2023-09-23T03:27:50.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.650+0000 7fe498a1a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4940cb870 con 0x7fe4940ad830 2023-09-23T03:27:50.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.650+0000 7fe498a1a640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4940715d0 con 0x7fe4940c9680 2023-09-23T03:27:50.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.651+0000 7fe49259c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe4940ad830 0x7fe494070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:50.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.651+0000 7fe49259c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fe4940ad830 0x7fe494070c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50512/0 (socket says 172.21.15.154:50512) 2023-09-23T03:27:50.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.651+0000 7fe49259c640 1 -- 172.21.15.154:0/1026309433 learned_addr learned my addr 172.21.15.154:0/1026309433 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:50.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.651+0000 7fe48bfff640 1 --2- 172.21.15.154:0/1026309433 >> v2:172.21.15.190:3300/0 conn(0x7fe4940c9680 0x7fe4940caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:50.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.651+0000 7fe49259c640 1 -- 172.21.15.154:0/1026309433 >> v2:172.21.15.190:3300/0 conn(0x7fe4940c9680 msgr2=0x7fe4940caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:50.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.651+0000 7fe49259c640 1 --2- 172.21.15.154:0/1026309433 >> v2:172.21.15.190:3300/0 conn(0x7fe4940c9680 0x7fe4940caec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:50.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.651+0000 7fe49259c640 1 -- 172.21.15.154:0/1026309433 >> v1:172.21.15.190:6789/0 conn(0x7fe4940cb400 legacy=0x7fe494160720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:50.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.651+0000 7fe49259c640 1 -- 172.21.15.154:0/1026309433 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4940718b0 con 0x7fe4940ad830 2023-09-23T03:27:50.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.653+0000 7fe49259c640 1 --2- 172.21.15.154:0/1026309433 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4940ad830 0x7fe494070c60 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fe4940cbb10 tx=0x7fe48402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72430755d422de4f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:50.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.653+0000 7fe491d9b640 1 -- 172.21.15.154:0/1026309433 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe484031870 con 0x7fe4940ad830 2023-09-23T03:27:50.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.653+0000 7fe491d9b640 1 -- 172.21.15.154:0/1026309433 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4840319d0 con 0x7fe4940ad830 2023-09-23T03:27:50.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.653+0000 7fe491d9b640 1 -- 172.21.15.154:0/1026309433 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe484031d60 con 0x7fe4940ad830 2023-09-23T03:27:50.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.654+0000 7fe498a1a640 1 -- 172.21.15.154:0/1026309433 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4940ad830 msgr2=0x7fe494070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:50.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.654+0000 7fe498a1a640 1 --2- 172.21.15.154:0/1026309433 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4940ad830 0x7fe494070c60 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fe4940cbb10 tx=0x7fe48402b910 comp rx=0 tx=0).stop 2023-09-23T03:27:50.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.654+0000 7fe498a1a640 1 -- 172.21.15.154:0/1026309433 shutdown_connections 2023-09-23T03:27:50.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.654+0000 7fe498a1a640 1 --2- 172.21.15.154:0/1026309433 >> v2:172.21.15.190:3300/0 conn(0x7fe4940c9680 0x7fe4940caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:50.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.654+0000 7fe498a1a640 1 --2- 172.21.15.154:0/1026309433 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4940ad830 0x7fe494070c60 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:50.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.654+0000 7fe498a1a640 1 -- 172.21.15.154:0/1026309433 >> 172.21.15.154:0/1026309433 conn(0x7fe49406c010 msgr2=0x7fe49406c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:50.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.654+0000 7fe498a1a640 1 -- 172.21.15.154:0/1026309433 shutdown_connections 2023-09-23T03:27:50.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.654+0000 7fe498a1a640 1 -- 172.21.15.154:0/1026309433 wait complete. 2023-09-23T03:27:50.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.656+0000 7fe498a1a640 1 Processor -- start 2023-09-23T03:27:50.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.656+0000 7fe498a1a640 1 -- start start 2023-09-23T03:27:50.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.657+0000 7fe498a1a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4940ad830 0x7fe4940b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:50.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.657+0000 7fe498a1a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4940c9680 0x7fe4941545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:50.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.657+0000 7fe498a1a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4940cb400 0x7fe4940cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:50.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.657+0000 7fe498a1a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4940b2370 con 0x7fe4940cb400 2023-09-23T03:27:50.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.657+0000 7fe498a1a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4940c2860 con 0x7fe4940c9680 2023-09-23T03:27:50.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.657+0000 7fe498a1a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4940c2b40 con 0x7fe4940ad830 2023-09-23T03:27:50.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.658+0000 7fe492d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4940cb400 0x7fe4940cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:50.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.658+0000 7fe492d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4940cb400 0x7fe4940cdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50526/0 (socket says 172.21.15.154:50526) 2023-09-23T03:27:50.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.658+0000 7fe492d9d640 1 -- 172.21.15.154:0/1131618456 learned_addr learned my addr 172.21.15.154:0/1131618456 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:50.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.658+0000 7fe49259c640 1 --2- 172.21.15.154:0/1131618456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4940ad830 0x7fe4940b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:50.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.658+0000 7fe492d9d640 1 -- 172.21.15.154:0/1131618456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4940ad830 msgr2=0x7fe4940b1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:50.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.658+0000 7fe48bfff640 1 --2- 172.21.15.154:0/1131618456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4940c9680 0x7fe4941545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:50.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.658+0000 7fe492d9d640 1 --2- 172.21.15.154:0/1131618456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4940ad830 0x7fe4940b1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:50.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.658+0000 7fe492d9d640 1 -- 172.21.15.154:0/1131618456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4940c9680 msgr2=0x7fe4941545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:50.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.658+0000 7fe492d9d640 1 --2- 172.21.15.154:0/1131618456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4940c9680 0x7fe4941545f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:50.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.658+0000 7fe492d9d640 1 -- 172.21.15.154:0/1131618456 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4840094d0 con 0x7fe4940cb400 2023-09-23T03:27:50.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.658+0000 7fe492d9d640 1 --2- 172.21.15.154:0/1131618456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4940cb400 0x7fe4940cdaf0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fe48c00c780 tx=0x7fe48c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:50.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.659+0000 7fe48affd640 1 -- 172.21.15.154:0/1131618456 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48c003d80 con 0x7fe4940cb400 2023-09-23T03:27:50.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.659+0000 7fe48affd640 1 -- 172.21.15.154:0/1131618456 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe48c00ce90 con 0x7fe4940cb400 2023-09-23T03:27:50.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.659+0000 7fe48affd640 1 -- 172.21.15.154:0/1131618456 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48c0052b0 con 0x7fe4940cb400 2023-09-23T03:27:50.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.659+0000 7fe498a1a640 1 -- 172.21.15.154:0/1131618456 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4940c2e20 con 0x7fe4940cb400 2023-09-23T03:27:50.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.659+0000 7fe498a1a640 1 -- 172.21.15.154:0/1131618456 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4940b1fe0 con 0x7fe4940cb400 2023-09-23T03:27:50.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.660+0000 7fe498a1a640 1 -- 172.21.15.154:0/1131618456 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe49404e920 con 0x7fe4940cb400 2023-09-23T03:27:50.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.661+0000 7fe48affd640 1 -- 172.21.15.154:0/1131618456 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fe48c005410 con 0x7fe4940cb400 2023-09-23T03:27:50.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.663+0000 7fe48affd640 1 --2- 172.21.15.154:0/1131618456 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe47406af30 0x7fe47406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:50.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.663+0000 7fe48affd640 1 -- 172.21.15.154:0/1131618456 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe48c088cf0 con 0x7fe4940cb400 2023-09-23T03:27:50.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.663+0000 7fe49259c640 1 --2- 172.21.15.154:0/1131618456 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe47406af30 0x7fe47406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:50.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.664+0000 7fe49259c640 1 --2- 172.21.15.154:0/1131618456 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe47406af30 0x7fe47406d3f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fe484036000 tx=0x7fe48403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:50.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.667+0000 7fe48affd640 1 -- 172.21.15.154:0/1131618456 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fe48c08d070 con 0x7fe4940cb400 2023-09-23T03:27:50.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.922+0000 7fe498a1a640 1 -- 172.21.15.154:0/1131618456 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe49415f240 con 0x7fe4940cb400 2023-09-23T03:27:50.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.926+0000 7fe48affd640 1 -- 172.21.15.154:0/1131618456 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7fe48c017020 con 0x7fe4940cb400 2023-09-23T03:27:50.926 INFO:teuthology.orchestra.run.smithi154.stdout:[client.0] 2023-09-23T03:27:50.927 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQAHWA5lYg+qGhAAjaY0fPsmRTz3ipsKbPugMA== 2023-09-23T03:27:50.927 INFO:teuthology.orchestra.run.smithi154.stdout: caps mds = "allow" 2023-09-23T03:27:50.927 INFO:teuthology.orchestra.run.smithi154.stdout: caps mon = "allow r" 2023-09-23T03:27:50.927 INFO:teuthology.orchestra.run.smithi154.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-09-23T03:27:50.927 INFO:teuthology.orchestra.run.smithi154.stderr:updated caps for client.0 2023-09-23T03:27:50.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.929+0000 7fe498a1a640 1 -- 172.21.15.154:0/1131618456 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe47406af30 msgr2=0x7fe47406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:50.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.929+0000 7fe498a1a640 1 --2- 172.21.15.154:0/1131618456 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe47406af30 0x7fe47406d3f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fe484036000 tx=0x7fe48403c040 comp rx=0 tx=0).stop 2023-09-23T03:27:50.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.929+0000 7fe498a1a640 1 -- 172.21.15.154:0/1131618456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4940cb400 msgr2=0x7fe4940cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:50.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.929+0000 7fe498a1a640 1 --2- 172.21.15.154:0/1131618456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4940cb400 0x7fe4940cdaf0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fe48c00c780 tx=0x7fe48c00cc40 comp rx=0 tx=0).stop 2023-09-23T03:27:50.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.930+0000 7fe498a1a640 1 -- 172.21.15.154:0/1131618456 shutdown_connections 2023-09-23T03:27:50.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.930+0000 7fe498a1a640 1 --2- 172.21.15.154:0/1131618456 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe47406af30 0x7fe47406d3f0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:50.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.930+0000 7fe498a1a640 1 --2- 172.21.15.154:0/1131618456 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe4940cb400 0x7fe4940cdaf0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:50.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.930+0000 7fe498a1a640 1 --2- 172.21.15.154:0/1131618456 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe4940c9680 0x7fe4941545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:50.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.930+0000 7fe498a1a640 1 --2- 172.21.15.154:0/1131618456 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe4940ad830 0x7fe4940b1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:50.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.930+0000 7fe498a1a640 1 -- 172.21.15.154:0/1131618456 >> 172.21.15.154:0/1131618456 conn(0x7fe49406c010 msgr2=0x7fe494160520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:50.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.930+0000 7fe498a1a640 1 -- 172.21.15.154:0/1131618456 shutdown_connections 2023-09-23T03:27:50.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:50.930+0000 7fe498a1a640 1 -- 172.21.15.154:0/1131618456 wait complete. 2023-09-23T03:27:50.955 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:27:50.974 INFO:tasks.ceph.mds.b.smithi190.stderr:2023-09-23T03:27:50.974+0000 7fa1b4941640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-23T03:27:51.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.161+0000 7f1b96eec640 1 Processor -- start 2023-09-23T03:27:51.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.161+0000 7f1b96eec640 1 -- start start 2023-09-23T03:27:51.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.162+0000 7f1b96eec640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1b900acce0 0x7f1b9015ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:51.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.162+0000 7f1b96eec640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1b900b1b20 0x7f1b900ce930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:51.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.162+0000 7f1b96eec640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b9015ac30 con 0x7f1b900cee70 2023-09-23T03:27:51.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.162+0000 7f1b96eec640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b900b2e90 con 0x7f1b900b1b20 2023-09-23T03:27:51.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.162+0000 7f1b96eec640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b900cf300 con 0x7f1b900acce0 2023-09-23T03:27:51.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.163+0000 7f1b87fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1b900b1b20 0x7f1b900ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:51.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.163+0000 7f1b87fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1b900b1b20 0x7f1b900ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50536/0 (socket says 172.21.15.154:50536) 2023-09-23T03:27:51.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.163+0000 7f1b94c88640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f1b900acce0 0x7f1b9015ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:51.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.163+0000 7f1b87fff640 1 -- 172.21.15.154:0/2657774882 learned_addr learned my addr 172.21.15.154:0/2657774882 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:51.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.163+0000 7f1b87fff640 1 -- 172.21.15.154:0/2657774882 >> v2:172.21.15.190:3300/0 conn(0x7f1b900acce0 msgr2=0x7f1b9015ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:51.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.163+0000 7f1b87fff640 1 --2- 172.21.15.154:0/2657774882 >> v2:172.21.15.190:3300/0 conn(0x7f1b900acce0 0x7f1b9015ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:51.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.163+0000 7f1b87fff640 1 -- 172.21.15.154:0/2657774882 >> v1:172.21.15.190:6789/0 conn(0x7f1b900cee70 legacy=0x7f1b9006cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:51.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.163+0000 7f1b87fff640 1 -- 172.21.15.154:0/2657774882 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b900b1f70 con 0x7f1b900b1b20 2023-09-23T03:27:51.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.165+0000 7f1b87fff640 1 --2- 172.21.15.154:0/2657774882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b900b1b20 0x7f1b900ce930 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f1b900cf570 tx=0x7f1b8002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=175a4f42f6aa8bf9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:51.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.165+0000 7f1b877fe640 1 -- 172.21.15.154:0/2657774882 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b80031870 con 0x7f1b900b1b20 2023-09-23T03:27:51.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.165+0000 7f1b877fe640 1 -- 172.21.15.154:0/2657774882 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b800319d0 con 0x7f1b900b1b20 2023-09-23T03:27:51.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.165+0000 7f1b877fe640 1 -- 172.21.15.154:0/2657774882 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b80031d60 con 0x7f1b900b1b20 2023-09-23T03:27:51.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.166+0000 7f1b96eec640 1 -- 172.21.15.154:0/2657774882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b900b1b20 msgr2=0x7f1b900ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:51.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.166+0000 7f1b96eec640 1 --2- 172.21.15.154:0/2657774882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b900b1b20 0x7f1b900ce930 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f1b900cf570 tx=0x7f1b8002b910 comp rx=0 tx=0).stop 2023-09-23T03:27:51.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.166+0000 7f1b96eec640 1 -- 172.21.15.154:0/2657774882 shutdown_connections 2023-09-23T03:27:51.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.166+0000 7f1b96eec640 1 --2- 172.21.15.154:0/2657774882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b900b1b20 0x7f1b900ce930 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:51.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.166+0000 7f1b96eec640 1 --2- 172.21.15.154:0/2657774882 >> v2:172.21.15.190:3300/0 conn(0x7f1b900acce0 0x7f1b9015ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:51.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.166+0000 7f1b96eec640 1 -- 172.21.15.154:0/2657774882 >> 172.21.15.154:0/2657774882 conn(0x7f1b9006a0f0 msgr2=0x7f1b9006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:51.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.166+0000 7f1b96eec640 1 -- 172.21.15.154:0/2657774882 shutdown_connections 2023-09-23T03:27:51.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.166+0000 7f1b96eec640 1 -- 172.21.15.154:0/2657774882 wait complete. 2023-09-23T03:27:51.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.167+0000 7f1b96eec640 1 Processor -- start 2023-09-23T03:27:51.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.168+0000 7f1b96eec640 1 -- start start 2023-09-23T03:27:51.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.168+0000 7f1b96eec640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b900acce0 0x7f1b900c4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:51.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.169+0000 7f1b96eec640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b900b1b20 0x7f1b901f1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:51.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.169+0000 7f1b96eec640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b900cee70 0x7f1b900caef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:51.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.169+0000 7f1b96eec640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b900c46d0 con 0x7f1b900cee70 2023-09-23T03:27:51.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.169+0000 7f1b96eec640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b901f16d0 con 0x7f1b900acce0 2023-09-23T03:27:51.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.169+0000 7f1b96eec640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b900c5680 con 0x7f1b900b1b20 2023-09-23T03:27:51.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.169+0000 7f1b95489640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b900cee70 0x7f1b900caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:51.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.169+0000 7f1b95489640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b900cee70 0x7f1b900caef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50552/0 (socket says 172.21.15.154:50552) 2023-09-23T03:27:51.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.169+0000 7f1b95489640 1 -- 172.21.15.154:0/1554387055 learned_addr learned my addr 172.21.15.154:0/1554387055 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:51.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.169+0000 7f1b87fff640 1 --2- 172.21.15.154:0/1554387055 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b900b1b20 0x7f1b901f1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:51.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.169+0000 7f1b94c88640 1 --2- 172.21.15.154:0/1554387055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b900acce0 0x7f1b900c4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:51.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.169+0000 7f1b95489640 1 -- 172.21.15.154:0/1554387055 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b900b1b20 msgr2=0x7f1b901f1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:51.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.169+0000 7f1b95489640 1 --2- 172.21.15.154:0/1554387055 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b900b1b20 0x7f1b901f1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:51.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.169+0000 7f1b95489640 1 -- 172.21.15.154:0/1554387055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b900acce0 msgr2=0x7f1b900c4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:51.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.169+0000 7f1b95489640 1 --2- 172.21.15.154:0/1554387055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b900acce0 0x7f1b900c4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:51.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.169+0000 7f1b95489640 1 -- 172.21.15.154:0/1554387055 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b800094d0 con 0x7f1b900cee70 2023-09-23T03:27:51.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.170+0000 7f1b95489640 1 --2- 172.21.15.154:0/1554387055 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b900cee70 0x7f1b900caef0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f1b8800c6e0 tx=0x7f1b8800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:51.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.170+0000 7f1b857fa640 1 -- 172.21.15.154:0/1554387055 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b88003b70 con 0x7f1b900cee70 2023-09-23T03:27:51.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.170+0000 7f1b857fa640 1 -- 172.21.15.154:0/1554387055 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b88003cf0 con 0x7f1b900cee70 2023-09-23T03:27:51.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.170+0000 7f1b857fa640 1 -- 172.21.15.154:0/1554387055 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b88005310 con 0x7f1b900cee70 2023-09-23T03:27:51.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.170+0000 7f1b96eec640 1 -- 172.21.15.154:0/1554387055 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b900c5960 con 0x7f1b900cee70 2023-09-23T03:27:51.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.170+0000 7f1b96eec640 1 -- 172.21.15.154:0/1554387055 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b900c5f20 con 0x7f1b900cee70 2023-09-23T03:27:51.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.171+0000 7f1b96eec640 1 -- 172.21.15.154:0/1554387055 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b90073eb0 con 0x7f1b900cee70 2023-09-23T03:27:51.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.172+0000 7f1b857fa640 1 -- 172.21.15.154:0/1554387055 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f1b88005470 con 0x7f1b900cee70 2023-09-23T03:27:51.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.173+0000 7f1b857fa640 1 --2- 172.21.15.154:0/1554387055 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1b7c06af30 0x7f1b7c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:51.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.173+0000 7f1b857fa640 1 -- 172.21.15.154:0/1554387055 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1b88089520 con 0x7f1b900cee70 2023-09-23T03:27:51.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.173+0000 7f1b94c88640 1 --2- 172.21.15.154:0/1554387055 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1b7c06af30 0x7f1b7c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:51.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.174+0000 7f1b94c88640 1 --2- 172.21.15.154:0/1554387055 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1b7c06af30 0x7f1b7c06d3f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f1b900c8910 tx=0x7f1b8c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:51.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.177+0000 7f1b857fa640 1 -- 172.21.15.154:0/1554387055 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f1b88054380 con 0x7f1b900cee70 2023-09-23T03:27:51.404 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.404+0000 7f1b96eec640 1 -- 172.21.15.154:0/1554387055 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b9015ac10 con 0x7f1b900cee70 2023-09-23T03:27:51.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.405+0000 7f1b857fa640 1 -- 172.21.15.154:0/1554387055 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) v1 ==== 95+0+4933 (secure 0 0 0) 0x7f1b88003e60 con 0x7f1b900cee70 2023-09-23T03:27:51.405 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:51.406 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":108,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/2759187683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2759187683},{"type":"v1","addr":"172.21.15.154:6835","nonce":2759187683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":6544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/4147378242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":4147378242},{"type":"v1","addr":"172.21.15.190:6834","nonce":4147378242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103},{"gid":8907,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/4006549651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4006549651},{"type":"v1","addr":"172.21.15.154:6837","nonce":4006549651}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:27:44.470312+0000","modified":"2023-09-23T03:27:47.518558+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8919},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8919":{"gid":8919,"name":"b","rank":0,"incarnation":104,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/3558096060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3558096060},{"type":"v1","addr":"172.21.15.190:6835","nonce":3558096060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-09-23T03:27:51.406 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 108 2023-09-23T03:27:51.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.408+0000 7f1b96eec640 1 -- 172.21.15.154:0/1554387055 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1b7c06af30 msgr2=0x7f1b7c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:51.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.408+0000 7f1b96eec640 1 --2- 172.21.15.154:0/1554387055 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1b7c06af30 0x7f1b7c06d3f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f1b900c8910 tx=0x7f1b8c008040 comp rx=0 tx=0).stop 2023-09-23T03:27:51.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.408+0000 7f1b96eec640 1 -- 172.21.15.154:0/1554387055 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b900cee70 msgr2=0x7f1b900caef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:51.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.408+0000 7f1b96eec640 1 --2- 172.21.15.154:0/1554387055 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b900cee70 0x7f1b900caef0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f1b8800c6e0 tx=0x7f1b8800cba0 comp rx=0 tx=0).stop 2023-09-23T03:27:51.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.408+0000 7f1b96eec640 1 -- 172.21.15.154:0/1554387055 shutdown_connections 2023-09-23T03:27:51.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.408+0000 7f1b96eec640 1 --2- 172.21.15.154:0/1554387055 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1b7c06af30 0x7f1b7c06d3f0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:51.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.408+0000 7f1b96eec640 1 --2- 172.21.15.154:0/1554387055 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b900cee70 0x7f1b900caef0 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:51.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.408+0000 7f1b96eec640 1 --2- 172.21.15.154:0/1554387055 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1b900b1b20 0x7f1b901f1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:51.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.408+0000 7f1b96eec640 1 --2- 172.21.15.154:0/1554387055 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1b900acce0 0x7f1b900c4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:51.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.408+0000 7f1b96eec640 1 -- 172.21.15.154:0/1554387055 >> 172.21.15.154:0/1554387055 conn(0x7f1b9006a0f0 msgr2=0x7f1b900af5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:51.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.408+0000 7f1b96eec640 1 -- 172.21.15.154:0/1554387055 shutdown_connections 2023-09-23T03:27:51.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:51.408+0000 7f1b96eec640 1 -- 172.21.15.154:0/1554387055 wait complete. 2023-09-23T03:27:51.830 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 108, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-23T03:27:44.470312+0000', 'modified': '2023-09-23T03:27:47.518558+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8919}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8919': {'gid': 8919, 'name': 'b', 'rank': 0, 'incarnation': 104, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.190:6835/3558096060', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.190:6833', 'nonce': 3558096060}, {'type': 'v1', 'addr': '172.21.15.190:6835', 'nonce': 3558096060}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-23T03:27:51.830 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-23T03:27:51.830 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-09-23T03:27:52.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.058+0000 7fde8b5e7640 1 Processor -- start 2023-09-23T03:27:52.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.058+0000 7fde8b5e7640 1 -- start start 2023-09-23T03:27:52.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.059+0000 7fde8b5e7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde840b2fe0 0x7fde840ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:52.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.059+0000 7fde8b5e7640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde8406ceb0 con 0x7fde840ada30 2023-09-23T03:27:52.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.059+0000 7fde8b5e7640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde84159c90 con 0x7fde840cf130 2023-09-23T03:27:52.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.059+0000 7fde8b5e7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde840ade10 con 0x7fde840b2fe0 2023-09-23T03:27:52.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.059+0000 7fde89383640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fde840ada30 0x7fde840cea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:47928/0 (socket says 172.21.15.154:47928) 2023-09-23T03:27:52.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.060+0000 7fde89383640 1 -- 172.21.15.154:0/194768706 learned_addr learned my addr 172.21.15.154:0/194768706 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:27:52.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.060+0000 7fde89b84640 1 --2- 172.21.15.154:0/194768706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde840b2fe0 0x7fde840ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:52.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.060+0000 7fde7bfff640 1 -- 172.21.15.154:0/194768706 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3639940843 0 0) 0x7fde80001290 con 0x7fde840ada30 2023-09-23T03:27:52.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.060+0000 7fde7bfff640 1 -- 172.21.15.154:0/194768706 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde70005730 con 0x7fde840ada30 2023-09-23T03:27:52.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.060+0000 7fde7bfff640 1 -- 172.21.15.154:0/194768706 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2327529600 0 0) 0x7fde70005730 con 0x7fde840ada30 2023-09-23T03:27:52.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.060+0000 7fde7bfff640 1 -- 172.21.15.154:0/194768706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde840b2fe0 msgr2=0x7fde840ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:52.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.060+0000 7fde7bfff640 1 --2- 172.21.15.154:0/194768706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde840b2fe0 0x7fde840ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.060+0000 7fde7bfff640 1 -- 172.21.15.154:0/194768706 >> v1:172.21.15.190:6789/0 conn(0x7fde840cf130 legacy=0x7fde8415d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:52.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.060+0000 7fde7bfff640 1 -- 172.21.15.154:0/194768706 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde8406c110 con 0x7fde840ada30 2023-09-23T03:27:52.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.060+0000 7fde7bfff640 1 -- 172.21.15.154:0/194768706 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fde80001980 con 0x7fde840ada30 2023-09-23T03:27:52.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.060+0000 7fde7bfff640 1 -- 172.21.15.154:0/194768706 >> v1:172.21.15.154:6789/0 conn(0x7fde840ada30 legacy=0x7fde840cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:52.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.061+0000 7fde7bfff640 1 --2- 172.21.15.154:0/194768706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde70005bb0 0x7fde70005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:52.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.061+0000 7fde7bfff640 1 --2- 172.21.15.154:0/194768706 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde700069d0 0x7fde70006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:52.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.061+0000 7fde7bfff640 1 --2- 172.21.15.154:0/194768706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde840b2fe0 0x7fde70007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:52.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.061+0000 7fde7bfff640 1 -- 172.21.15.154:0/194768706 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde840ade10 con 0x7fde700069d0 2023-09-23T03:27:52.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.061+0000 7fde7bfff640 1 -- 172.21.15.154:0/194768706 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7000dc70 con 0x7fde70005bb0 2023-09-23T03:27:52.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.061+0000 7fde7bfff640 1 -- 172.21.15.154:0/194768706 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde7000de30 con 0x7fde840b2fe0 2023-09-23T03:27:52.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.061+0000 7fde88b82640 1 --2- 172.21.15.154:0/194768706 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde700069d0 0x7fde70006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:52.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.061+0000 7fde88b82640 1 -- 172.21.15.154:0/194768706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde840b2fe0 msgr2=0x7fde70007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:52.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.061+0000 7fde88b82640 1 --2- 172.21.15.154:0/194768706 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde840b2fe0 0x7fde70007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.061+0000 7fde89383640 1 --2- 172.21.15.154:0/194768706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde70005bb0 0x7fde70005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:52.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.061+0000 7fde88b82640 1 -- 172.21.15.154:0/194768706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde70005bb0 msgr2=0x7fde70005fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:52.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.061+0000 7fde88b82640 1 --2- 172.21.15.154:0/194768706 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde70005bb0 0x7fde70005fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.061+0000 7fde88b82640 1 -- 172.21.15.154:0/194768706 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde70005730 con 0x7fde700069d0 2023-09-23T03:27:52.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.062+0000 7fde88b82640 1 --2- 172.21.15.154:0/194768706 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde700069d0 0x7fde70006e50 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fde840ca3c0 tx=0x7fde70007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2215abff7147daf8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:52.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.063+0000 7fde7bfff640 1 -- 172.21.15.154:0/194768706 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde740052d0 con 0x7fde700069d0 2023-09-23T03:27:52.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.063+0000 7fde7bfff640 1 -- 172.21.15.154:0/194768706 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde7402e050 con 0x7fde700069d0 2023-09-23T03:27:52.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.063+0000 7fde7bfff640 1 -- 172.21.15.154:0/194768706 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde740035f0 con 0x7fde700069d0 2023-09-23T03:27:52.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.063+0000 7fde8b5e7640 1 -- 172.21.15.154:0/194768706 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde700069d0 msgr2=0x7fde70006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:52.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.063+0000 7fde8b5e7640 1 --2- 172.21.15.154:0/194768706 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde700069d0 0x7fde70006e50 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fde840ca3c0 tx=0x7fde70007a00 comp rx=0 tx=0).stop 2023-09-23T03:27:52.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.063+0000 7fde89b84640 1 -- 172.21.15.154:0/194768706 reap_dead start 2023-09-23T03:27:52.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.063+0000 7fde8b5e7640 1 -- 172.21.15.154:0/194768706 shutdown_connections 2023-09-23T03:27:52.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.063+0000 7fde8b5e7640 1 -- 172.21.15.154:0/194768706 >> 172.21.15.154:0/194768706 conn(0x7fde8406a0f0 msgr2=0x7fde8406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:52.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.063+0000 7fde8b5e7640 1 -- 172.21.15.154:0/194768706 shutdown_connections 2023-09-23T03:27:52.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.063+0000 7fde8b5e7640 1 -- 172.21.15.154:0/194768706 wait complete. 2023-09-23T03:27:52.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.065+0000 7fde8b5e7640 1 Processor -- start 2023-09-23T03:27:52.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.065+0000 7fde8b5e7640 1 -- start start 2023-09-23T03:27:52.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.066+0000 7fde8b5e7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde700069d0 0x7fde840c1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:52.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.066+0000 7fde8b5e7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde840c99f0 0x7fde841f1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:52.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.066+0000 7fde8b5e7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde840c4980 0x7fde840c4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:52.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.066+0000 7fde8b5e7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde840c39d0 con 0x7fde840c99f0 2023-09-23T03:27:52.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.066+0000 7fde8b5e7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde840c3c00 con 0x7fde700069d0 2023-09-23T03:27:52.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.066+0000 7fde8b5e7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde840c3ea0 con 0x7fde840c4980 2023-09-23T03:27:52.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.066+0000 7fde88b82640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde840c99f0 0x7fde841f1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:52.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.067+0000 7fde88b82640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde840c99f0 0x7fde841f1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50564/0 (socket says 172.21.15.154:50564) 2023-09-23T03:27:52.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.067+0000 7fde88b82640 1 -- 172.21.15.154:0/3511260001 learned_addr learned my addr 172.21.15.154:0/3511260001 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:52.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.067+0000 7fde89383640 1 --2- 172.21.15.154:0/3511260001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde700069d0 0x7fde840c1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:52.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.067+0000 7fde89b84640 1 --2- 172.21.15.154:0/3511260001 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde840c4980 0x7fde840c4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:52.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.067+0000 7fde88b82640 1 -- 172.21.15.154:0/3511260001 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde840c4980 msgr2=0x7fde840c4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:52.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.067+0000 7fde88b82640 1 --2- 172.21.15.154:0/3511260001 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde840c4980 0x7fde840c4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.067+0000 7fde88b82640 1 -- 172.21.15.154:0/3511260001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde700069d0 msgr2=0x7fde840c1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:52.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.067+0000 7fde88b82640 1 --2- 172.21.15.154:0/3511260001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde700069d0 0x7fde840c1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.067+0000 7fde88b82640 1 -- 172.21.15.154:0/3511260001 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde840c4180 con 0x7fde840c99f0 2023-09-23T03:27:52.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.067+0000 7fde88b82640 1 --2- 172.21.15.154:0/3511260001 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde840c99f0 0x7fde841f1f20 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fde7402bb80 tx=0x7fde74001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:52.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.068+0000 7fde79ffb640 1 -- 172.21.15.154:0/3511260001 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde740076c0 con 0x7fde840c99f0 2023-09-23T03:27:52.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.068+0000 7fde79ffb640 1 -- 172.21.15.154:0/3511260001 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde740069e0 con 0x7fde840c99f0 2023-09-23T03:27:52.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.068+0000 7fde8b5e7640 1 -- 172.21.15.154:0/3511260001 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde840c4460 con 0x7fde840c99f0 2023-09-23T03:27:52.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.068+0000 7fde79ffb640 1 -- 172.21.15.154:0/3511260001 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde74006d50 con 0x7fde840c99f0 2023-09-23T03:27:52.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.068+0000 7fde8b5e7640 1 -- 172.21.15.154:0/3511260001 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde840c24c0 con 0x7fde840c99f0 2023-09-23T03:27:52.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.070+0000 7fde79ffb640 1 -- 172.21.15.154:0/3511260001 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fde740035f0 con 0x7fde840c99f0 2023-09-23T03:27:52.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.071+0000 7fde79ffb640 1 --2- 172.21.15.154:0/3511260001 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fde6806af60 0x7fde6806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:52.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.071+0000 7fde79ffb640 1 -- 172.21.15.154:0/3511260001 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fde740ab530 con 0x7fde840c99f0 2023-09-23T03:27:52.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.072+0000 7fde89383640 1 --2- 172.21.15.154:0/3511260001 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fde6806af60 0x7fde6806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:52.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.072+0000 7fde89383640 1 --2- 172.21.15.154:0/3511260001 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fde6806af60 0x7fde6806d420 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fde80001eb0 tx=0x7fde80009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:52.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.072+0000 7fde8e17a740 1 Processor -- start 2023-09-23T03:27:52.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.073+0000 7fde8e17a740 1 -- start start 2023-09-23T03:27:52.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.073+0000 7fde8e17a740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e374418c00 0x55e3743ebb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:52.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.073+0000 7fde8e17a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e374418fe0 0x55e37441b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:52.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.073+0000 7fde8e17a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3743ec040 0x55e37441bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:52.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.073+0000 7fde8e17a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e37435f6d0 con 0x55e374418fe0 2023-09-23T03:27:52.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.073+0000 7fde8e17a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e37435cdc0 con 0x55e3743ec040 2023-09-23T03:27:52.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.073+0000 7fde8e17a740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e3743586b0 con 0x55e374418c00 2023-09-23T03:27:52.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.074+0000 7fde88b82640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e374418fe0 0x55e37441b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:52.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.074+0000 7fde88b82640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e374418fe0 0x55e37441b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50576/0 (socket says 172.21.15.154:50576) 2023-09-23T03:27:52.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.074+0000 7fde88b82640 1 -- 172.21.15.154:0/2423249650 learned_addr learned my addr 172.21.15.154:0/2423249650 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:52.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.074+0000 7fde88b82640 1 -- 172.21.15.154:0/2423249650 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e374418c00 msgr2=0x55e3743ebb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:52.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.074+0000 7fde88b82640 1 --2- 172.21.15.154:0/2423249650 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e374418c00 0x55e3743ebb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.074+0000 7fde88b82640 1 -- 172.21.15.154:0/2423249650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3743ec040 msgr2=0x55e37441bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:52.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.074+0000 7fde88b82640 1 --2- 172.21.15.154:0/2423249650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3743ec040 0x55e37441bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.074+0000 7fde88b82640 1 -- 172.21.15.154:0/2423249650 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e37431c8e0 con 0x55e374418fe0 2023-09-23T03:27:52.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.074+0000 7fde88b82640 1 --2- 172.21.15.154:0/2423249650 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e374418fe0 0x55e37441b990 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fde74008fd0 tx=0x7fde74079fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167554a4883c87b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:52.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.075+0000 7fde6effd640 1 -- 172.21.15.154:0/2423249650 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7407e020 con 0x55e374418fe0 2023-09-23T03:27:52.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.075+0000 7fde6effd640 1 -- 172.21.15.154:0/2423249650 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde7407e220 con 0x55e374418fe0 2023-09-23T03:27:52.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.075+0000 7fde6effd640 1 -- 172.21.15.154:0/2423249650 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde740823d0 con 0x55e374418fe0 2023-09-23T03:27:52.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.076+0000 7fde8e17a740 1 -- 172.21.15.154:0/2423249650 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e374418fe0 msgr2=0x55e37441b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:52.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.076+0000 7fde8e17a740 1 --2- 172.21.15.154:0/2423249650 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e374418fe0 0x55e37441b990 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fde74008fd0 tx=0x7fde74079fd0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.076+0000 7fde8e17a740 1 -- 172.21.15.154:0/2423249650 shutdown_connections 2023-09-23T03:27:52.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.076+0000 7fde8e17a740 1 --2- 172.21.15.154:0/2423249650 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e3743ec040 0x55e37441bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.076+0000 7fde8e17a740 1 --2- 172.21.15.154:0/2423249650 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e374418fe0 0x55e37441b990 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.076+0000 7fde8e17a740 1 --2- 172.21.15.154:0/2423249650 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e374418c00 0x55e3743ebb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.076+0000 7fde8e17a740 1 -- 172.21.15.154:0/2423249650 >> 172.21.15.154:0/2423249650 conn(0x55e3743ede10 msgr2=0x55e374436990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:52.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.076+0000 7fde8e17a740 1 -- 172.21.15.154:0/2423249650 shutdown_connections 2023-09-23T03:27:52.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.076+0000 7fde8e17a740 1 -- 172.21.15.154:0/2423249650 wait complete. 2023-09-23T03:27:52.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.077+0000 7fde8e17a740 1 -- start start 2023-09-23T03:27:52.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.077+0000 7fde8e17a740 1 Processor -- start 2023-09-23T03:27:52.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.078+0000 7fde8e17a740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e374418c00 0x55e37440cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:52.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.078+0000 7fde8e17a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e374418fe0 0x55e37441e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:52.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.078+0000 7fde8e17a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e37441ee00 0x55e374430e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:52.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.078+0000 7fde8e17a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3742f6940 con 0x55e37441ee00 2023-09-23T03:27:52.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.078+0000 7fde8e17a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3742f65e0 con 0x55e374418fe0 2023-09-23T03:27:52.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.078+0000 7fde8e17a740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e3740bcf20 con 0x55e374418c00 2023-09-23T03:27:52.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.078+0000 7fde8e17a740 10 client.0 _ll_register_callbacks cb 0x55e3743345e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:27:52.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.078+0000 7fde89b84640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e37441ee00 0x55e374430e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:52.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.078+0000 7fde89b84640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e37441ee00 0x55e374430e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50592/0 (socket says 172.21.15.154:50592) 2023-09-23T03:27:52.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.078+0000 7fde89b84640 1 -- 172.21.15.154:0/3996757646 learned_addr learned my addr 172.21.15.154:0/3996757646 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:52.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.078+0000 7fde88b82640 1 --2- 172.21.15.154:0/3996757646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e374418fe0 0x55e37441e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:52.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.079+0000 7fde89b84640 1 -- 172.21.15.154:0/3996757646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e374418c00 msgr2=0x55e37440cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:52.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.079+0000 7fde89383640 1 --2- 172.21.15.154:0/3996757646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e374418c00 0x55e37440cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:52.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.079+0000 7fde89b84640 1 --2- 172.21.15.154:0/3996757646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e374418c00 0x55e37440cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.079+0000 7fde89b84640 1 -- 172.21.15.154:0/3996757646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e374418fe0 msgr2=0x55e37441e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:52.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.079+0000 7fde89b84640 1 --2- 172.21.15.154:0/3996757646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e374418fe0 0x55e37441e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.079+0000 7fde89b84640 1 -- 172.21.15.154:0/3996757646 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde840c24c0 con 0x55e37441ee00 2023-09-23T03:27:52.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.079+0000 7fde89b84640 1 --2- 172.21.15.154:0/3996757646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e37441ee00 0x55e374430e70 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fde840b10a0 tx=0x7fde7c007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=182cafdb5ae88266 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:52.084 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.079+0000 7fde63fff640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:27:52.084 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.079+0000 7fde63fff640 1 -- 172.21.15.154:0/3996757646 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7c003b90 con 0x55e37441ee00 2023-09-23T03:27:52.084 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.079+0000 7fde63fff640 1 -- 172.21.15.154:0/3996757646 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fde7c003cf0 con 0x55e37441ee00 2023-09-23T03:27:52.084 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.079+0000 7fde8e17a740 1 -- 172.21.15.154:0/3996757646 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e374351690 con 0x55e37441ee00 2023-09-23T03:27:52.084 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.079+0000 7fde63fff640 1 -- 172.21.15.154:0/3996757646 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde7c00f890 con 0x55e37441ee00 2023-09-23T03:27:52.084 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.079+0000 7fde8e17a740 10 client.9057 fetch_fsmap learned FSMap version 108 2023-09-23T03:27:52.084 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.079+0000 7fde8e17a740 1 -- 172.21.15.154:0/3996757646 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) v3 -- 0x55e374356d30 con 0x55e37441ee00 2023-09-23T03:27:52.084 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.080+0000 7fde63fff640 1 -- 172.21.15.154:0/3996757646 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e374351690 con 0x55e37441ee00 2023-09-23T03:27:52.084 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.080+0000 7fde63fff640 1 -- 172.21.15.154:0/3996757646 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 108) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fde7c014070 con 0x55e37441ee00 2023-09-23T03:27:52.084 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.080+0000 7fde8e17a740 10 client.9057 fetch_fsmap finished waiting for FSMap version 108 2023-09-23T03:27:52.084 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.080+0000 7fde8e17a740 10 client.9057 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-23T03:27:52.084 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.080+0000 7fde8e17a740 20 client.9057 populate_metadata read hostname 'smithi154' 2023-09-23T03:27:52.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.080+0000 7fde8e17a740 1 --2- 172.21.15.154:0/3996757646 >> [v2:172.21.15.190:6833/3558096060,v1:172.21.15.190:6835/3558096060] conn(0x55e37444a600 0x55e374453ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:52.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.080+0000 7fde8e17a740 4 client.9057 mds_command: new command op to 8919 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T03:27:52.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.080+0000 7fde8e17a740 1 -- 172.21.15.154:0/3996757646 --> [v2:172.21.15.190:6833/3558096060,v1:172.21.15.190:6835/3558096060] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e3743513b0 con 0x55e37444a600 2023-09-23T03:27:52.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.081+0000 7fde89383640 1 --2- 172.21.15.154:0/3996757646 >> [v2:172.21.15.190:6833/3558096060,v1:172.21.15.190:6835/3558096060] conn(0x55e37444a600 0x55e374453ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:52.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.081+0000 7fde89383640 1 --2- 172.21.15.154:0/3996757646 >> [v2:172.21.15.190:6833/3558096060,v1:172.21.15.190:6835/3558096060] conn(0x55e37444a600 0x55e374453ea0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbd1f037f89f8a28 server_cookie=881f1cb5c4b18422 in_seq=0 out_seq=0 2023-09-23T03:27:52.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.081+0000 7fde63fff640 10 client.9057 ms_handle_connect on v2:172.21.15.190:6833/3558096060 2023-09-23T03:27:52.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.083+0000 7fde63fff640 1 -- 172.21.15.154:0/3996757646 <== mds.0 v2:172.21.15.190:6833/3558096060 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e3743513b0 con 0x55e37444a600 2023-09-23T03:27:52.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.083+0000 7fde63fff640 10 client.9057 handle_command_reply: tid=0 2023-09-23T03:27:52.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.084+0000 7fde8e17a740 1 client.9057 shutdown 2023-09-23T03:27:52.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.084+0000 7fde8e17a740 1 -- 172.21.15.154:0/3996757646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e37441ee00 msgr2=0x55e374430e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:52.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.084+0000 7fde8e17a740 1 --2- 172.21.15.154:0/3996757646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e37441ee00 0x55e374430e70 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fde840b10a0 tx=0x7fde7c007f30 comp rx=0 tx=0).stop 2023-09-23T03:27:52.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.084+0000 7fde8e17a740 1 -- 172.21.15.154:0/3996757646 shutdown_connections 2023-09-23T03:27:52.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.084+0000 7fde8e17a740 1 --2- 172.21.15.154:0/3996757646 >> [v2:172.21.15.190:6833/3558096060,v1:172.21.15.190:6835/3558096060] conn(0x55e37444a600 0x55e374453ea0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.084+0000 7fde8e17a740 1 --2- 172.21.15.154:0/3996757646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e37441ee00 0x55e374430e70 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.084+0000 7fde8e17a740 1 --2- 172.21.15.154:0/3996757646 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e374418fe0 0x55e37441e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.084+0000 7fde8e17a740 1 --2- 172.21.15.154:0/3996757646 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e374418c00 0x55e37440cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.084+0000 7fde63fff640 0 client.9057 ms_handle_reset on v2:172.21.15.190:6833/3558096060 2023-09-23T03:27:52.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.084+0000 7fde8e17a740 1 -- 172.21.15.154:0/3996757646 >> 172.21.15.154:0/3996757646 conn(0x55e3743ec040 msgr2=0x55e37441bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:52.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.084+0000 7fde8e17a740 1 -- 172.21.15.154:0/3996757646 shutdown_connections 2023-09-23T03:27:52.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.084+0000 7fde8e17a740 1 -- 172.21.15.154:0/3996757646 wait complete. 2023-09-23T03:27:52.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.084+0000 7fde8e17a740 20 client.9057 trim_cache size 0 max 16384 2023-09-23T03:27:52.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.101+0000 7fde8e17a740 1 Processor -- start 2023-09-23T03:27:52.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.101+0000 7fde8e17a740 1 -- start start 2023-09-23T03:27:52.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.102+0000 7fde8e17a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e374418c00 0x55e3743ef8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:52.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.102+0000 7fde8e17a740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e374418fe0 0x55e374448660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:52.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.102+0000 7fde8e17a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e37441ee00 0x55e374448ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:52.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.102+0000 7fde8e17a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3743eccf0 con 0x55e37441ee00 2023-09-23T03:27:52.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.102+0000 7fde8e17a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e37443a1c0 con 0x55e374418c00 2023-09-23T03:27:52.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.102+0000 7fde8e17a740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e3744394e0 con 0x55e374418fe0 2023-09-23T03:27:52.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.102+0000 7fde89b84640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e37441ee00 0x55e374448ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:52.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.102+0000 7fde89b84640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e37441ee00 0x55e374448ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50604/0 (socket says 172.21.15.154:50604) 2023-09-23T03:27:52.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.102+0000 7fde89b84640 1 -- 172.21.15.154:0/1710363306 learned_addr learned my addr 172.21.15.154:0/1710363306 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:52.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.102+0000 7fde89b84640 1 -- 172.21.15.154:0/1710363306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e374418fe0 msgr2=0x55e374448660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:52.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.102+0000 7fde89b84640 1 --2- 172.21.15.154:0/1710363306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e374418fe0 0x55e374448660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.102+0000 7fde89b84640 1 -- 172.21.15.154:0/1710363306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e374418c00 msgr2=0x55e3743ef8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:52.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.102+0000 7fde89b84640 1 --2- 172.21.15.154:0/1710363306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e374418c00 0x55e3743ef8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.102+0000 7fde89b84640 1 -- 172.21.15.154:0/1710363306 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e37443a530 con 0x55e37441ee00 2023-09-23T03:27:52.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.103+0000 7fde89b84640 1 --2- 172.21.15.154:0/1710363306 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e37441ee00 0x55e374448ba0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fde7c000f80 tx=0x7fde7c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51dcdbcfae7e631 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:52.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.103+0000 7fde6f7fe640 1 -- 172.21.15.154:0/1710363306 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7c00fd50 con 0x55e37441ee00 2023-09-23T03:27:52.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.103+0000 7fde6f7fe640 1 -- 172.21.15.154:0/1710363306 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde7c029070 con 0x55e37441ee00 2023-09-23T03:27:52.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.103+0000 7fde6f7fe640 1 -- 172.21.15.154:0/1710363306 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7c003bd0 con 0x55e37441ee00 2023-09-23T03:27:52.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.103+0000 7fde8e17a740 1 -- 172.21.15.154:0/1710363306 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e37441ee00 msgr2=0x55e374448ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:52.106 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.103+0000 7fde8e17a740 1 --2- 172.21.15.154:0/1710363306 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e37441ee00 0x55e374448ba0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fde7c000f80 tx=0x7fde7c012000 comp rx=0 tx=0).stop 2023-09-23T03:27:52.106 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.103+0000 7fde8e17a740 1 -- 172.21.15.154:0/1710363306 shutdown_connections 2023-09-23T03:27:52.106 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.103+0000 7fde8e17a740 1 --2- 172.21.15.154:0/1710363306 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e37441ee00 0x55e374448ba0 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.106 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.103+0000 7fde8e17a740 1 --2- 172.21.15.154:0/1710363306 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e374418fe0 0x55e374448660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.106 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.103+0000 7fde8e17a740 1 --2- 172.21.15.154:0/1710363306 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e374418c00 0x55e3743ef8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.106 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.103+0000 7fde8e17a740 1 -- 172.21.15.154:0/1710363306 >> 172.21.15.154:0/1710363306 conn(0x55e3743ec040 msgr2=0x55e374432300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:52.106 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.104+0000 7fde8e17a740 1 -- 172.21.15.154:0/1710363306 shutdown_connections 2023-09-23T03:27:52.106 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.104+0000 7fde8e17a740 1 -- 172.21.15.154:0/1710363306 wait complete. 2023-09-23T03:27:52.109 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.104+0000 7fde8e17a740 1 -- start start 2023-09-23T03:27:52.109 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.104+0000 7fde8e17a740 1 Processor -- start 2023-09-23T03:27:52.109 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.105+0000 7fde8e17a740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e374418c00 0x55e3744a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:52.109 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.105+0000 7fde8e17a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e374418fe0 0x55e37440ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:52.109 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.105+0000 7fde8e17a740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e37441ee00 0x55e37440b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:52.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.105+0000 7fde8e17a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3744a1ee0 con 0x55e374418fe0 2023-09-23T03:27:52.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.105+0000 7fde8e17a740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3744a8930 con 0x55e37441ee00 2023-09-23T03:27:52.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.105+0000 7fde8e17a740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e374437800 con 0x55e374418c00 2023-09-23T03:27:52.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.105+0000 7fde8e17a740 10 client.0 _ll_register_callbacks cb 0x7fde74006eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:27:52.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.106+0000 7fde88b82640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e374418fe0 0x55e37440ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:52.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.106+0000 7fde88b82640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e374418fe0 0x55e37440ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50610/0 (socket says 172.21.15.154:50610) 2023-09-23T03:27:52.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.106+0000 7fde88b82640 1 -- 172.21.15.154:0/788255943 learned_addr learned my addr 172.21.15.154:0/788255943 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:52.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.106+0000 7fde89383640 1 --2- 172.21.15.154:0/788255943 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e374418c00 0x55e3744a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:52.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.106+0000 7fde88b82640 1 -- 172.21.15.154:0/788255943 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e374418c00 msgr2=0x55e3744a19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:52.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.106+0000 7fde88b82640 1 --2- 172.21.15.154:0/788255943 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e374418c00 0x55e3744a19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.106+0000 7fde89b84640 1 --2- 172.21.15.154:0/788255943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e37441ee00 0x55e37440b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:52.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.106+0000 7fde88b82640 1 -- 172.21.15.154:0/788255943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e37441ee00 msgr2=0x55e37440b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:52.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.106+0000 7fde88b82640 1 --2- 172.21.15.154:0/788255943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e37441ee00 0x55e37440b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.106+0000 7fde88b82640 1 -- 172.21.15.154:0/788255943 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e374356d30 con 0x55e374418fe0 2023-09-23T03:27:52.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.106+0000 7fde89383640 1 --2- 172.21.15.154:0/788255943 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e374418c00 0x55e3744a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:52.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.106+0000 7fde88b82640 1 --2- 172.21.15.154:0/788255943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e374418fe0 0x55e37440ac80 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fde7407a060 tx=0x7fde740a97d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd69c112de625e91 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:52.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.106+0000 7fde63fff640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:27:52.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.107+0000 7fde63fff640 1 -- 172.21.15.154:0/788255943 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde740823d0 con 0x55e374418fe0 2023-09-23T03:27:52.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.107+0000 7fde63fff640 1 -- 172.21.15.154:0/788255943 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fde740313a0 con 0x55e374418fe0 2023-09-23T03:27:52.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.107+0000 7fde8e17a740 1 -- 172.21.15.154:0/788255943 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde7407e220 con 0x55e374418fe0 2023-09-23T03:27:52.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.107+0000 7fde63fff640 1 -- 172.21.15.154:0/788255943 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde74031580 con 0x55e374418fe0 2023-09-23T03:27:52.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.107+0000 7fde8e17a740 10 client.9063 fetch_fsmap learned FSMap version 108 2023-09-23T03:27:52.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.107+0000 7fde8e17a740 1 -- 172.21.15.154:0/788255943 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) v3 -- 0x55e3742f5b50 con 0x55e374418fe0 2023-09-23T03:27:52.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.107+0000 7fde63fff640 1 -- 172.21.15.154:0/788255943 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde7407e220 con 0x55e374418fe0 2023-09-23T03:27:52.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.107+0000 7fde63fff640 1 -- 172.21.15.154:0/788255943 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 108) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fde740a9d20 con 0x55e374418fe0 2023-09-23T03:27:52.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.107+0000 7fde8e17a740 10 client.9063 fetch_fsmap finished waiting for FSMap version 108 2023-09-23T03:27:52.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.107+0000 7fde8e17a740 10 client.9063 resolve_mds: resolved name 'b' to daemon mds.b 2023-09-23T03:27:52.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.107+0000 7fde8e17a740 20 client.9063 populate_metadata read hostname 'smithi154' 2023-09-23T03:27:52.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.107+0000 7fde8e17a740 1 --2- 172.21.15.154:0/788255943 >> [v2:172.21.15.190:6833/3558096060,v1:172.21.15.190:6835/3558096060] conn(0x55e3744a79c0 0x55e3744a7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:52.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.107+0000 7fde8e17a740 4 client.9063 mds_command: new command op to 8919 tid=0[{"prefix": "status"}] 2023-09-23T03:27:52.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.107+0000 7fde8e17a740 1 -- 172.21.15.154:0/788255943 --> [v2:172.21.15.190:6833/3558096060,v1:172.21.15.190:6835/3558096060] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e37405a6c0 con 0x55e3744a79c0 2023-09-23T03:27:52.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.108+0000 7fde89383640 1 --2- 172.21.15.154:0/788255943 >> [v2:172.21.15.190:6833/3558096060,v1:172.21.15.190:6835/3558096060] conn(0x55e3744a79c0 0x55e3744a7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:52.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.109+0000 7fde89383640 1 --2- 172.21.15.154:0/788255943 >> [v2:172.21.15.190:6833/3558096060,v1:172.21.15.190:6835/3558096060] conn(0x55e3744a79c0 0x55e3744a7da0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=241182df630899c1 server_cookie=58c54ea52048cdff in_seq=0 out_seq=0 2023-09-23T03:27:52.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.109+0000 7fde63fff640 10 client.9063 ms_handle_connect on v2:172.21.15.190:6833/3558096060 2023-09-23T03:27:52.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.109+0000 7fde63fff640 1 -- 172.21.15.154:0/788255943 <== mds.0 v2:172.21.15.190:6833/3558096060 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x7fde8001b160 con 0x55e3744a79c0 2023-09-23T03:27:52.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.109+0000 7fde63fff640 10 client.9063 handle_command_reply: tid=0 2023-09-23T03:27:52.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.110+0000 7fde8e17a740 1 client.9063 shutdown 2023-09-23T03:27:52.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.110+0000 7fde8e17a740 1 -- 172.21.15.154:0/788255943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e374418fe0 msgr2=0x55e37440ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:52.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.110+0000 7fde8e17a740 1 --2- 172.21.15.154:0/788255943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e374418fe0 0x55e37440ac80 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fde7407a060 tx=0x7fde740a97d0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.110+0000 7fde8e17a740 1 -- 172.21.15.154:0/788255943 shutdown_connections 2023-09-23T03:27:52.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.110+0000 7fde8e17a740 1 --2- 172.21.15.154:0/788255943 >> [v2:172.21.15.190:6833/3558096060,v1:172.21.15.190:6835/3558096060] conn(0x55e3744a79c0 0x55e3744a7da0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.110+0000 7fde8e17a740 1 --2- 172.21.15.154:0/788255943 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e37441ee00 0x55e37440b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.110+0000 7fde8e17a740 1 --2- 172.21.15.154:0/788255943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e374418fe0 0x55e37440ac80 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.110+0000 7fde8e17a740 1 --2- 172.21.15.154:0/788255943 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e374418c00 0x55e3744a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.110+0000 7fde63fff640 0 client.9063 ms_handle_reset on v2:172.21.15.190:6833/3558096060 2023-09-23T03:27:52.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.110+0000 7fde8e17a740 1 -- 172.21.15.154:0/788255943 >> 172.21.15.154:0/788255943 conn(0x55e3743ec040 msgr2=0x55e37441b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:52.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.110+0000 7fde8e17a740 1 -- 172.21.15.154:0/788255943 shutdown_connections 2023-09-23T03:27:52.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.110+0000 7fde8e17a740 1 -- 172.21.15.154:0/788255943 wait complete. 2023-09-23T03:27:52.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.110+0000 7fde8e17a740 20 client.9063 trim_cache size 0 max 16384 2023-09-23T03:27:52.115 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:27:52.115 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_fsid": "8c8039d3-6c84-43db-8808-85e0dbdd47c8", 2023-09-23T03:27:52.115 INFO:teuthology.orchestra.run.smithi154.stdout: "whoami": 0, 2023-09-23T03:27:52.116 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 8919, 2023-09-23T03:27:52.116 INFO:teuthology.orchestra.run.smithi154.stdout: "want_state": "up:active", 2023-09-23T03:27:52.116 INFO:teuthology.orchestra.run.smithi154.stdout: "state": "up:active", 2023-09-23T03:27:52.116 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs", 2023-09-23T03:27:52.116 INFO:teuthology.orchestra.run.smithi154.stdout: "rank_uptime": 7.1074957550000004, 2023-09-23T03:27:52.116 INFO:teuthology.orchestra.run.smithi154.stdout: "mdsmap_epoch": 108, 2023-09-23T03:27:52.116 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_epoch": 126, 2023-09-23T03:27:52.116 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_epoch_barrier": 126, 2023-09-23T03:27:52.116 INFO:teuthology.orchestra.run.smithi154.stdout: "uptime": 12.502598109999999 2023-09-23T03:27:52.116 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:27:52.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.111+0000 7fde8b5e7640 1 -- 172.21.15.154:0/3511260001 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fde6806af60 msgr2=0x7fde6806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:52.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.111+0000 7fde8b5e7640 1 --2- 172.21.15.154:0/3511260001 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fde6806af60 0x7fde6806d420 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fde80001eb0 tx=0x7fde80009040 comp rx=0 tx=0).stop 2023-09-23T03:27:52.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.111+0000 7fde8b5e7640 1 -- 172.21.15.154:0/3511260001 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde840c99f0 msgr2=0x7fde841f1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:52.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.111+0000 7fde8b5e7640 1 --2- 172.21.15.154:0/3511260001 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde840c99f0 0x7fde841f1f20 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fde7402bb80 tx=0x7fde74001120 comp rx=0 tx=0).stop 2023-09-23T03:27:52.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.111+0000 7fde8b5e7640 1 -- 172.21.15.154:0/3511260001 shutdown_connections 2023-09-23T03:27:52.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.111+0000 7fde8b5e7640 1 --2- 172.21.15.154:0/3511260001 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fde6806af60 0x7fde6806d420 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.111+0000 7fde8b5e7640 1 --2- 172.21.15.154:0/3511260001 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fde840c4980 0x7fde840c4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.111+0000 7fde8b5e7640 1 --2- 172.21.15.154:0/3511260001 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde840c99f0 0x7fde841f1f20 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.111+0000 7fde8b5e7640 1 --2- 172.21.15.154:0/3511260001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fde700069d0 0x7fde840c1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.111+0000 7fde8b5e7640 1 -- 172.21.15.154:0/3511260001 >> 172.21.15.154:0/3511260001 conn(0x7fde8406a0f0 msgr2=0x7fde840ab140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:52.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.111+0000 7fde8b5e7640 1 -- 172.21.15.154:0/3511260001 shutdown_connections 2023-09-23T03:27:52.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.111+0000 7fde8b5e7640 1 -- 172.21.15.154:0/3511260001 wait complete. 2023-09-23T03:27:52.129 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-23T03:27:52.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.337+0000 7f4c0a226640 1 Processor -- start 2023-09-23T03:27:52.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.338+0000 7f4c0a226640 1 -- start start 2023-09-23T03:27:52.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.339+0000 7f4c0a226640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4c040acce0 0x7f4c0415ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:52.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.339+0000 7f4c0a226640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c0415ac30 con 0x7f4c040b1b20 2023-09-23T03:27:52.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.339+0000 7f4c0a226640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c040b1f70 con 0x7f4c040cf040 2023-09-23T03:27:52.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.339+0000 7f4c0a226640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c040b2e90 con 0x7f4c040acce0 2023-09-23T03:27:52.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.339+0000 7f4c037fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4c040acce0 0x7f4c0415ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:52.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.340+0000 7f4c037fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f4c040acce0 0x7f4c0415ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50620/0 (socket says 172.21.15.154:50620) 2023-09-23T03:27:52.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.340+0000 7f4c037fe640 1 -- 172.21.15.154:0/512103990 learned_addr learned my addr 172.21.15.154:0/512103990 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:52.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.340+0000 7f4c037fe640 1 -- 172.21.15.154:0/512103990 >> v1:172.21.15.190:6789/0 conn(0x7f4c040cf040 legacy=0x7f4c040ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:52.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.340+0000 7f4c037fe640 1 -- 172.21.15.154:0/512103990 >> v1:172.21.15.154:6789/0 conn(0x7f4c040b1b20 legacy=0x7f4c040ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:52.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.340+0000 7f4c037fe640 1 -- 172.21.15.154:0/512103990 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c0415f2a0 con 0x7f4c040acce0 2023-09-23T03:27:52.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.341+0000 7f4c037fe640 1 --2- 172.21.15.154:0/512103990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c040acce0 0x7f4c0415ed60 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4009850 tx=0x7f4bf402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2669c5a1bab56fe2 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:52.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.341+0000 7f4c02ffd640 1 -- 172.21.15.154:0/512103990 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf4004230 con 0x7f4c040acce0 2023-09-23T03:27:52.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.341+0000 7f4c02ffd640 1 -- 172.21.15.154:0/512103990 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf4004390 con 0x7f4c040acce0 2023-09-23T03:27:52.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.341+0000 7f4c02ffd640 1 -- 172.21.15.154:0/512103990 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf4003810 con 0x7f4c040acce0 2023-09-23T03:27:52.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.342+0000 7f4c0a226640 1 -- 172.21.15.154:0/512103990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c040acce0 msgr2=0x7f4c0415ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:52.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.342+0000 7f4c0a226640 1 --2- 172.21.15.154:0/512103990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c040acce0 0x7f4c0415ed60 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4009850 tx=0x7f4bf402b8f0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.342+0000 7f4c0a226640 1 -- 172.21.15.154:0/512103990 shutdown_connections 2023-09-23T03:27:52.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.342+0000 7f4c0a226640 1 --2- 172.21.15.154:0/512103990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c040acce0 0x7f4c0415ed60 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.342+0000 7f4c0a226640 1 -- 172.21.15.154:0/512103990 >> 172.21.15.154:0/512103990 conn(0x7f4c0406a0f0 msgr2=0x7f4c0406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:52.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.342+0000 7f4c0a226640 1 -- 172.21.15.154:0/512103990 shutdown_connections 2023-09-23T03:27:52.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.342+0000 7f4c0a226640 1 -- 172.21.15.154:0/512103990 wait complete. 2023-09-23T03:27:52.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.344+0000 7f4c0a226640 1 Processor -- start 2023-09-23T03:27:52.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.344+0000 7f4c0a226640 1 -- start start 2023-09-23T03:27:52.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.345+0000 7f4c0a226640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c040acce0 0x7f4c041577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:52.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.345+0000 7f4c0a226640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c040b1b20 0x7f4c0415b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:52.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.345+0000 7f4c0a226640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c040cf040 0x7f4c0415b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:52.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.345+0000 7f4c0a226640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c04157d30 con 0x7f4c040b1b20 2023-09-23T03:27:52.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.345+0000 7f4c0a226640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c040c3780 con 0x7f4c040cf040 2023-09-23T03:27:52.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.345+0000 7f4c0a226640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c040c2e50 con 0x7f4c040acce0 2023-09-23T03:27:52.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.345+0000 7f4bfbfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c040b1b20 0x7f4c0415b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:52.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.345+0000 7f4bfbfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c040b1b20 0x7f4c0415b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:50632/0 (socket says 172.21.15.154:50632) 2023-09-23T03:27:52.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.345+0000 7f4bfbfff640 1 -- 172.21.15.154:0/3924913217 learned_addr learned my addr 172.21.15.154:0/3924913217 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:52.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.346+0000 7f4c03fff640 1 --2- 172.21.15.154:0/3924913217 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c040cf040 0x7f4c0415b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:52.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.346+0000 7f4c037fe640 1 --2- 172.21.15.154:0/3924913217 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c040acce0 0x7f4c041577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:52.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.346+0000 7f4bfbfff640 1 -- 172.21.15.154:0/3924913217 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c040acce0 msgr2=0x7f4c041577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:52.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.346+0000 7f4bfbfff640 1 --2- 172.21.15.154:0/3924913217 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c040acce0 0x7f4c041577f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.346+0000 7f4bfbfff640 1 -- 172.21.15.154:0/3924913217 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c040cf040 msgr2=0x7f4c0415b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:52.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.346+0000 7f4bfbfff640 1 --2- 172.21.15.154:0/3924913217 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c040cf040 0x7f4c0415b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.346+0000 7f4bfbfff640 1 -- 172.21.15.154:0/3924913217 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf4009460 con 0x7f4c040b1b20 2023-09-23T03:27:52.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.346+0000 7f4c03fff640 1 --2- 172.21.15.154:0/3924913217 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c040cf040 0x7f4c0415b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:52.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.346+0000 7f4bfbfff640 1 --2- 172.21.15.154:0/3924913217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c040b1b20 0x7f4c0415b060 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f4bec00c450 tx=0x7f4bec00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:52.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.346+0000 7f4c00ff9640 1 -- 172.21.15.154:0/3924913217 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec0052d0 con 0x7f4c040b1b20 2023-09-23T03:27:52.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.346+0000 7f4c00ff9640 1 -- 172.21.15.154:0/3924913217 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bec005430 con 0x7f4c040b1b20 2023-09-23T03:27:52.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.346+0000 7f4c00ff9640 1 -- 172.21.15.154:0/3924913217 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec012870 con 0x7f4c040b1b20 2023-09-23T03:27:52.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.347+0000 7f4c0a226640 1 -- 172.21.15.154:0/3924913217 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c040c3130 con 0x7f4c040b1b20 2023-09-23T03:27:52.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.347+0000 7f4c0a226640 1 -- 172.21.15.154:0/3924913217 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c040c2ab0 con 0x7f4c040b1b20 2023-09-23T03:27:52.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.348+0000 7f4c0a226640 1 -- 172.21.15.154:0/3924913217 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c040c3910 con 0x7f4c040b1b20 2023-09-23T03:27:52.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.349+0000 7f4c00ff9640 1 -- 172.21.15.154:0/3924913217 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f4bec0129d0 con 0x7f4c040b1b20 2023-09-23T03:27:52.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.350+0000 7f4c00ff9640 1 --2- 172.21.15.154:0/3924913217 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4be406af60 0x7f4be406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:52.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.350+0000 7f4c00ff9640 1 -- 172.21.15.154:0/3924913217 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f4bec089230 con 0x7f4c040b1b20 2023-09-23T03:27:52.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.350+0000 7f4c037fe640 1 --2- 172.21.15.154:0/3924913217 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4be406af60 0x7f4be406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:52.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.351+0000 7f4c037fe640 1 --2- 172.21.15.154:0/3924913217 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4be406af60 0x7f4be406d420 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4035000 tx=0x7f4bf4008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:52.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.354+0000 7f4c00ff9640 1 -- 172.21.15.154:0/3924913217 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f4bec054130 con 0x7f4c040b1b20 2023-09-23T03:27:52.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.577+0000 7f4c0a226640 1 -- 172.21.15.154:0/3924913217 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4c0415ac10 con 0x7f4c040b1b20 2023-09-23T03:27:52.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.578+0000 7f4c00ff9640 1 -- 172.21.15.154:0/3924913217 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v108) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4bec057750 con 0x7f4c040b1b20 2023-09-23T03:27:52.580 INFO:teuthology.orchestra.run.smithi154.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-09-23T03:27:52.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.580+0000 7f4c0a226640 1 -- 172.21.15.154:0/3924913217 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4be406af60 msgr2=0x7f4be406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:52.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.580+0000 7f4c0a226640 1 --2- 172.21.15.154:0/3924913217 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4be406af60 0x7f4be406d420 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4035000 tx=0x7f4bf4008030 comp rx=0 tx=0).stop 2023-09-23T03:27:52.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.580+0000 7f4c0a226640 1 -- 172.21.15.154:0/3924913217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c040b1b20 msgr2=0x7f4c0415b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:52.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.580+0000 7f4c0a226640 1 --2- 172.21.15.154:0/3924913217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c040b1b20 0x7f4c0415b060 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f4bec00c450 tx=0x7f4bec00c910 comp rx=0 tx=0).stop 2023-09-23T03:27:52.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.581+0000 7f4c0a226640 1 -- 172.21.15.154:0/3924913217 shutdown_connections 2023-09-23T03:27:52.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.581+0000 7f4c0a226640 1 --2- 172.21.15.154:0/3924913217 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4be406af60 0x7f4be406d420 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.581+0000 7f4c0a226640 1 --2- 172.21.15.154:0/3924913217 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4c040cf040 0x7f4c0415b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.581+0000 7f4c0a226640 1 --2- 172.21.15.154:0/3924913217 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4c040b1b20 0x7f4c0415b060 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.581+0000 7f4c0a226640 1 --2- 172.21.15.154:0/3924913217 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4c040acce0 0x7f4c041577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:52.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.581+0000 7f4c0a226640 1 -- 172.21.15.154:0/3924913217 >> 172.21.15.154:0/3924913217 conn(0x7f4c0406a0f0 msgr2=0x7f4c040b0870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:52.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.581+0000 7f4c0a226640 1 -- 172.21.15.154:0/3924913217 shutdown_connections 2023-09-23T03:27:52.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:52.581+0000 7f4c0a226640 1 -- 172.21.15.154:0/3924913217 wait complete. 2023-09-23T03:27:52.601 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-23T03:27:52.601 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-23T03:27:52.601 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi154.front.sepia.ceph.com 2023-09-23T03:27:52.601 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-09-23T03:27:52.601 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-23T03:27:52.601 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-23T03:27:52.601 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-23T03:27:52.601 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-23T03:27:52.601 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-23T03:27:52.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:27:52.602 DEBUG:teuthology.orchestra.run.smithi154:> ip addr 2023-09-23T03:27:52.654 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-23T03:27:52.654 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-23T03:27:52.654 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2023-09-23T03:27:52.654 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:27:52.654 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2023-09-23T03:27:52.654 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:27:52.655 INFO:teuthology.orchestra.run.smithi154.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:27:52.655 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:54 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:27:52.655 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f0 2023-09-23T03:27:52.655 INFO:teuthology.orchestra.run.smithi154.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:27:52.655 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:55 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:27:52.655 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f1 2023-09-23T03:27:52.655 INFO:teuthology.orchestra.run.smithi154.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:27:52.655 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:60 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:27:52.655 INFO:teuthology.orchestra.run.smithi154.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-23T03:27:52.655 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:61 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:27:52.655 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.21.15.154/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-09-23T03:27:52.655 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft 41652sec preferred_lft 41652sec 2023-09-23T03:27:52.656 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::ec4:7aff:fe88:7761/64 scope link 2023-09-23T03:27:52.656 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:27:52.656 INFO:teuthology.orchestra.run.smithi154.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-23T03:27:52.656 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 5a:c6:b2:96:fb:dc brd ff:ff:ff:ff:ff:ff 2023-09-23T03:27:52.656 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-09-23T03:27:52.656 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:27:52.656 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::a801:22ff:febf:2833/64 scope link 2023-09-23T03:27:52.656 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:27:52.656 INFO:teuthology.orchestra.run.smithi154.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-23T03:27:52.656 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 5a:c6:b2:96:fb:dc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:27:52.656 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::58c6:b2ff:fe96:fbdc/64 scope link 2023-09-23T03:27:52.657 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:27:52.657 INFO:teuthology.orchestra.run.smithi154.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-23T03:27:52.657 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether b2:23:b0:e4:c2:ab brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:27:52.657 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::b023:b0ff:fee4:c2ab/64 scope link 2023-09-23T03:27:52.657 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:27:52.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:27:52.658 DEBUG:teuthology.orchestra.run.smithi154:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-23T03:27:52.745 INFO:teuthology.orchestra.run.smithi154.stdout:1 2023-09-23T03:27:52.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:27:52.750 DEBUG:teuthology.orchestra.run.smithi154:> route 2023-09-23T03:27:52.802 INFO:teuthology.orchestra.run.smithi154.stdout:Kernel IP routing table 2023-09-23T03:27:52.802 INFO:teuthology.orchestra.run.smithi154.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-23T03:27:52.802 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-23T03:27:52.803 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-09-23T03:27:52.803 INFO:teuthology.orchestra.run.smithi154.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-23T03:27:52.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:27:52.804 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-09-23T03:27:52.804 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-09-23T03:27:52.804 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-23T03:27:52.804 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-23T03:27:52.804 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-23T03:27:52.804 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-09-23T03:27:52.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:27:52.977 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2023-09-23T03:27:53.029 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-09-23T03:27:53.029 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-09-23T03:27:53.030 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-23T03:27:53.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:27:53.030 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set brx.0 up 2023-09-23T03:27:53.113 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-09-23T03:27:53.113 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-23T03:27:53.114 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-09-23T03:27:53.168 INFO:teuthology.orchestra.run.smithi154.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-09-23T03:27:53.168 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-23T03:27:53.168 DEBUG:teuthology.orchestra.run.smithi154:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-09-23T03:27:53.219 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-09-23T03:27:53.316 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2023-09-23T03:27:53.374 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2023-09-23T03:27:53.374 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2023-09-23T03:27:53.374 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2023-09-23T03:27:53.374 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2023-09-23T03:27:53.374 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2023-09-23T03:27:53.374 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2023-09-23T03:27:53.374 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2023-09-23T03:27:53.374 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2023-09-23T03:27:53.375 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2023-09-23T03:27:53.375 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2023-09-23T03:27:53.375 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2023-09-23T03:27:53.375 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T03:27:53.375 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2023-09-23T03:27:53.375 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2023-09-23T03:27:53.375 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2023-09-23T03:27:53.375 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2023-09-23T03:27:53.375 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2023-09-23T03:27:53.376 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2023-09-23T03:27:53.376 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2023-09-23T03:27:53.376 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-09-23T03:27:53.376 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-09-23T03:27:53.376 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2023-09-23T03:27:53.376 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2023-09-23T03:27:53.376 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2023-09-23T03:27:53.376 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-0 2023-09-23T03:27:53.376 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-1 2023-09-23T03:27:53.376 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-2 2023-09-23T03:27:53.377 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-3 2023-09-23T03:27:53.377 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2023-09-23T03:27:53.377 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:27:53.377 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:27:53.377 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:27:53.377 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:27:53.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:27:53.378 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-09-23T03:27:53.430 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-23T03:27:53.431 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-09-23T03:27:53.436 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-09-23T03:27:53.517 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2023-09-23T03:27:53.561 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2023-09-23T03:27:53.561 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2023-09-23T03:27:53.561 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2023-09-23T03:27:53.561 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2023-09-23T03:27:53.562 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2023-09-23T03:27:53.562 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2023-09-23T03:27:53.562 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2023-09-23T03:27:53.562 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2023-09-23T03:27:53.562 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2023-09-23T03:27:53.562 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2023-09-23T03:27:53.562 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2023-09-23T03:27:53.562 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T03:27:53.562 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2023-09-23T03:27:53.562 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2023-09-23T03:27:53.562 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2023-09-23T03:27:53.563 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2023-09-23T03:27:53.563 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2023-09-23T03:27:53.563 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2023-09-23T03:27:53.563 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2023-09-23T03:27:53.563 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-09-23T03:27:53.563 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-09-23T03:27:53.563 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2023-09-23T03:27:53.563 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2023-09-23T03:27:53.563 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2023-09-23T03:27:53.563 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-0 2023-09-23T03:27:53.563 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-1 2023-09-23T03:27:53.564 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-2 2023-09-23T03:27:53.564 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-3 2023-09-23T03:27:53.564 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2023-09-23T03:27:53.564 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:27:53.564 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:27:53.564 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:27:53.564 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:27:53.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:27:53.565 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-09-23T03:27:53.632 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:2023-09-23T03:27:53.631+0000 7f6b20f33480 -1 init, newargv = 0x55e099009030 newargc=15 2023-09-23T03:27:53.633 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:ceph-fuse[127122]: starting ceph client 2023-09-23T03:27:53.643 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:ceph-fuse[127122]: starting fuse 2023-09-23T03:27:53.654 INFO:teuthology.orchestra.run.smithi154.stdout:39 2023-09-23T03:27:53.655 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-09-23T03:27:53.655 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:27:53.655 DEBUG:teuthology.orchestra.run.smithi154:> import glob 2023-09-23T03:27:53.655 DEBUG:teuthology.orchestra.run.smithi154:> import re 2023-09-23T03:27:53.655 DEBUG:teuthology.orchestra.run.smithi154:> import os 2023-09-23T03:27:53.655 DEBUG:teuthology.orchestra.run.smithi154:> import subprocess 2023-09-23T03:27:53.655 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:27:53.655 DEBUG:teuthology.orchestra.run.smithi154:> def _find_admin_socket(client_name): 2023-09-23T03:27:53.655 DEBUG:teuthology.orchestra.run.smithi154:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-09-23T03:27:53.656 DEBUG:teuthology.orchestra.run.smithi154:> files = glob.glob(asok_path) 2023-09-23T03:27:53.656 DEBUG:teuthology.orchestra.run.smithi154:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-09-23T03:27:53.656 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:27:53.656 DEBUG:teuthology.orchestra.run.smithi154:> # Given a non-glob path, it better be there 2023-09-23T03:27:53.656 DEBUG:teuthology.orchestra.run.smithi154:> if "*" not in asok_path: 2023-09-23T03:27:53.656 DEBUG:teuthology.orchestra.run.smithi154:> assert(len(files) == 1) 2023-09-23T03:27:53.656 DEBUG:teuthology.orchestra.run.smithi154:> return files[0] 2023-09-23T03:27:53.656 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:27:53.656 DEBUG:teuthology.orchestra.run.smithi154:> for f in files: 2023-09-23T03:27:53.656 DEBUG:teuthology.orchestra.run.smithi154:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-23T03:27:53.656 DEBUG:teuthology.orchestra.run.smithi154:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-23T03:27:53.656 DEBUG:teuthology.orchestra.run.smithi154:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-23T03:27:53.656 DEBUG:teuthology.orchestra.run.smithi154:> contents = proc_f.read() 2023-09-23T03:27:53.657 DEBUG:teuthology.orchestra.run.smithi154:> if mountpoint in contents: 2023-09-23T03:27:53.657 DEBUG:teuthology.orchestra.run.smithi154:> return f 2023-09-23T03:27:53.657 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-23T03:27:53.657 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:27:53.657 DEBUG:teuthology.orchestra.run.smithi154:> print(_find_admin_socket("client.0")) 2023-09-23T03:27:53.657 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:27:53.855 INFO:teuthology.orchestra.run.smithi154.stdout:/var/run/ceph/ceph-client.0.127122.asok 2023-09-23T03:27:53.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:27:53.871 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.127122.asok 2023-09-23T03:27:53.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:27:53.872 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.127122.asok status 2023-09-23T03:27:54.117 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:27:54.117 INFO:teuthology.orchestra.run.smithi154.stdout: "metadata": { 2023-09-23T03:27:54.118 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_sha1": "862663913a74a99e1c52d50e6a7e59dd6705749a", 2023-09-23T03:27:54.118 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_version": "ceph version 18.0.0-6336-g86266391 (862663913a74a99e1c52d50e6a7e59dd6705749a) reef (dev)", 2023-09-23T03:27:54.118 INFO:teuthology.orchestra.run.smithi154.stdout: "entity_id": "0", 2023-09-23T03:27:54.118 INFO:teuthology.orchestra.run.smithi154.stdout: "hostname": "smithi154", 2023-09-23T03:27:54.118 INFO:teuthology.orchestra.run.smithi154.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-09-23T03:27:54.118 INFO:teuthology.orchestra.run.smithi154.stdout: "pid": "127122", 2023-09-23T03:27:54.118 INFO:teuthology.orchestra.run.smithi154.stdout: "root": "/" 2023-09-23T03:27:54.118 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:27:54.118 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_count": 0, 2023-09-23T03:27:54.118 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_pinned_count": 0, 2023-09-23T03:27:54.119 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 9078, 2023-09-23T03:27:54.119 INFO:teuthology.orchestra.run.smithi154.stdout: "inst": { 2023-09-23T03:27:54.119 INFO:teuthology.orchestra.run.smithi154.stdout: "name": { 2023-09-23T03:27:54.119 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "client", 2023-09-23T03:27:54.119 INFO:teuthology.orchestra.run.smithi154.stdout: "num": 9078 2023-09-23T03:27:54.119 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:27:54.119 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2023-09-23T03:27:54.119 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2023-09-23T03:27:54.119 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.1:0", 2023-09-23T03:27:54.119 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2754240987 2023-09-23T03:27:54.119 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:27:54.120 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:27:54.120 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2023-09-23T03:27:54.120 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2023-09-23T03:27:54.120 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.1:0", 2023-09-23T03:27:54.120 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2754240987 2023-09-23T03:27:54.120 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:27:54.120 INFO:teuthology.orchestra.run.smithi154.stdout: "inst_str": "client.9078 192.168.0.1:0/2754240987", 2023-09-23T03:27:54.120 INFO:teuthology.orchestra.run.smithi154.stdout: "addr_str": "192.168.0.1:0/2754240987", 2023-09-23T03:27:54.120 INFO:teuthology.orchestra.run.smithi154.stdout: "inode_count": 1, 2023-09-23T03:27:54.120 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_epoch": 108, 2023-09-23T03:27:54.120 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch": 126, 2023-09-23T03:27:54.120 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch_barrier": 0, 2023-09-23T03:27:54.121 INFO:teuthology.orchestra.run.smithi154.stdout: "blocklisted": false, 2023-09-23T03:27:54.121 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs" 2023-09-23T03:27:54.121 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:27:54.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:27:54.133 DEBUG:teuthology.orchestra.run.smithi154:> stat --file-system '--printf=%T 2023-09-23T03:27:54.133 DEBUG:teuthology.orchestra.run.smithi154:> ' -- /home/ubuntu/cephtest/mnt.0 2023-09-23T03:27:54.188 INFO:teuthology.orchestra.run.smithi154.stdout:fuseblk 2023-09-23T03:27:54.188 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-09-23T03:27:54.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:27:54.189 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-09-23T03:27:54.279 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-23T03:27:54.288 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-23T03:27:54.446 INFO:teuthology.orchestra.run.smithi154.stdout:[global] 2023-09-23T03:27:54.446 INFO:teuthology.orchestra.run.smithi154.stdout: chdir = "" 2023-09-23T03:27:54.446 INFO:teuthology.orchestra.run.smithi154.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-23T03:27:54.446 INFO:teuthology.orchestra.run.smithi154.stdout: auth supported = cephx 2023-09-23T03:27:54.447 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.447 INFO:teuthology.orchestra.run.smithi154.stdout: filestore xattr use omap = true 2023-09-23T03:27:54.447 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.447 INFO:teuthology.orchestra.run.smithi154.stdout: mon clock drift allowed = 1.000 2023-09-23T03:27:54.447 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.447 INFO:teuthology.orchestra.run.smithi154.stdout: osd crush chooseleaf type = 0 2023-09-23T03:27:54.447 INFO:teuthology.orchestra.run.smithi154.stdout: auth debug = true 2023-09-23T03:27:54.447 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.447 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on old message = true 2023-09-23T03:27:54.447 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on bug = true 2023-09-23T03:27:54.447 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.447 INFO:teuthology.orchestra.run.smithi154.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-23T03:27:54.447 INFO:teuthology.orchestra.run.smithi154.stdout: mon pg warn max object skew = 0 2023-09-23T03:27:54.448 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.448 INFO:teuthology.orchestra.run.smithi154.stdout: # disable pg_autoscaler by default for new pools 2023-09-23T03:27:54.448 INFO:teuthology.orchestra.run.smithi154.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-23T03:27:54.448 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.448 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default size = 2 2023-09-23T03:27:54.448 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.448 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow primary affinity = true 2023-09-23T03:27:54.448 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow pg remap = true 2023-09-23T03:27:54.448 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on legacy crush tunables = false 2023-09-23T03:27:54.448 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on crush straw calc version zero = false 2023-09-23T03:27:54.448 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on no sortbitwise = false 2023-09-23T03:27:54.448 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on osd down out interval zero = false 2023-09-23T03:27:54.449 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on too few osds = false 2023-09-23T03:27:54.449 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-23T03:27:54.449 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-23T03:27:54.449 INFO:teuthology.orchestra.run.smithi154.stdout: mon_allow_pool_size_one = true 2023-09-23T03:27:54.449 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.449 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-23T03:27:54.449 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.449 INFO:teuthology.orchestra.run.smithi154.stdout: osd default data pool replay window = 5 2023-09-23T03:27:54.449 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.449 INFO:teuthology.orchestra.run.smithi154.stdout: mon allow pool delete = true 2023-09-23T03:27:54.449 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.449 INFO:teuthology.orchestra.run.smithi154.stdout: mon cluster log file level = debug 2023-09-23T03:27:54.450 INFO:teuthology.orchestra.run.smithi154.stdout: debug asserts on shutdown = true 2023-09-23T03:27:54.450 INFO:teuthology.orchestra.run.smithi154.stdout: mon health detail to clog = false 2023-09-23T03:27:54.450 INFO:teuthology.orchestra.run.smithi154.stdout: mon host = "172.21.15.154,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]" 2023-09-23T03:27:54.450 INFO:teuthology.orchestra.run.smithi154.stdout: fsid = 8c8039d3-6c84-43db-8808-85e0dbdd47c8 2023-09-23T03:27:54.450 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.450 INFO:teuthology.orchestra.run.smithi154.stdout:[osd] 2023-09-23T03:27:54.450 INFO:teuthology.orchestra.run.smithi154.stdout: osd journal size = 100 2023-09-23T03:27:54.450 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.450 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub load threshold = 5.0 2023-09-23T03:27:54.450 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub max interval = 600 2023-09-23T03:27:54.450 INFO:teuthology.orchestra.run.smithi154.stdout: osd mclock profile = high_recovery_ops 2023-09-23T03:27:54.450 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.451 INFO:teuthology.orchestra.run.smithi154.stdout: osd recover clone overlap = true 2023-09-23T03:27:54.451 INFO:teuthology.orchestra.run.smithi154.stdout: osd recovery max chunk = 1048576 2023-09-23T03:27:54.451 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.451 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug shutdown = true 2023-09-23T03:27:54.451 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug op order = true 2023-09-23T03:27:54.451 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug verify stray on activate = true 2023-09-23T03:27:54.451 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.451 INFO:teuthology.orchestra.run.smithi154.stdout: osd open classes on start = true 2023-09-23T03:27:54.451 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug pg log writeout = true 2023-09-23T03:27:54.451 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.451 INFO:teuthology.orchestra.run.smithi154.stdout: osd deep scrub update digest min age = 30 2023-09-23T03:27:54.451 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.452 INFO:teuthology.orchestra.run.smithi154.stdout: osd map max advance = 10 2023-09-23T03:27:54.452 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.452 INFO:teuthology.orchestra.run.smithi154.stdout: journal zero on create = true 2023-09-23T03:27:54.452 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.452 INFO:teuthology.orchestra.run.smithi154.stdout: filestore ondisk finisher threads = 3 2023-09-23T03:27:54.452 INFO:teuthology.orchestra.run.smithi154.stdout: filestore apply finisher threads = 3 2023-09-23T03:27:54.452 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.452 INFO:teuthology.orchestra.run.smithi154.stdout: bdev debug aio = true 2023-09-23T03:27:54.452 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug misdirected ops = true 2023-09-23T03:27:54.452 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore block size = 96636764160 2023-09-23T03:27:54.452 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore fsck on mount = True 2023-09-23T03:27:54.452 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluefs = 20 2023-09-23T03:27:54.453 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluestore = 20 2023-09-23T03:27:54.453 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:27:54.453 INFO:teuthology.orchestra.run.smithi154.stdout: debug osd = 20 2023-09-23T03:27:54.453 INFO:teuthology.orchestra.run.smithi154.stdout: debug rocksdb = 10 2023-09-23T03:27:54.453 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd backfillfull_ratio = 0.85 2023-09-23T03:27:54.453 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd full ratio = 0.9 2023-09-23T03:27:54.453 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd nearfull ratio = 0.8 2023-09-23T03:27:54.453 INFO:teuthology.orchestra.run.smithi154.stdout: osd failsafe full ratio = 0.95 2023-09-23T03:27:54.453 INFO:teuthology.orchestra.run.smithi154.stdout: osd objectstore = bluestore 2023-09-23T03:27:54.453 INFO:teuthology.orchestra.run.smithi154.stdout: osd op complaint time = 180 2023-09-23T03:27:54.453 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.453 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr] 2023-09-23T03:27:54.454 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:27:54.454 INFO:teuthology.orchestra.run.smithi154.stdout: debug mgr = 20 2023-09-23T03:27:54.454 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2023-09-23T03:27:54.454 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2023-09-23T03:27:54.454 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:27:54.454 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:27:54.454 INFO:teuthology.orchestra.run.smithi154.stdout: mgr/telemetry/nag = false 2023-09-23T03:27:54.454 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.454 INFO:teuthology.orchestra.run.smithi154.stdout:[mon] 2023-09-23T03:27:54.454 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:27:54.454 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2023-09-23T03:27:54.455 INFO:teuthology.orchestra.run.smithi154.stdout: debug paxos = 20 2023-09-23T03:27:54.455 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2023-09-23T03:27:54.455 INFO:teuthology.orchestra.run.smithi154.stdout: mon data avail warn = 5 2023-09-23T03:27:54.455 INFO:teuthology.orchestra.run.smithi154.stdout: mon mgr mkfs grace = 240 2023-09-23T03:27:54.455 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:27:54.455 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd reporter subtree level = osd 2023-09-23T03:27:54.455 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd prime pg temp = true 2023-09-23T03:27:54.455 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:27:54.455 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.455 INFO:teuthology.orchestra.run.smithi154.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-23T03:27:54.455 INFO:teuthology.orchestra.run.smithi154.stdout: auth mon ticket ttl = 660 # 11m 2023-09-23T03:27:54.455 INFO:teuthology.orchestra.run.smithi154.stdout: auth service ticket ttl = 240 # 4m 2023-09-23T03:27:54.455 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.456 INFO:teuthology.orchestra.run.smithi154.stdout: # don't complain about insecure global_id in the test suite 2023-09-23T03:27:54.456 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-23T03:27:54.456 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-23T03:27:54.456 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.456 INFO:teuthology.orchestra.run.smithi154.stdout: # 1m isn't quite enough 2023-09-23T03:27:54.456 INFO:teuthology.orchestra.run.smithi154.stdout: mon_down_mkfs_grace = 2m 2023-09-23T03:27:54.456 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.456 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_filestore_osds = false 2023-09-23T03:27:54.456 INFO:teuthology.orchestra.run.smithi154.stdout: mon op complaint time = 120 2023-09-23T03:27:54.456 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.456 INFO:teuthology.orchestra.run.smithi154.stdout:[client] 2023-09-23T03:27:54.456 INFO:teuthology.orchestra.run.smithi154.stdout: rgw cache enabled = true 2023-09-23T03:27:54.457 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable ops log = true 2023-09-23T03:27:54.457 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable usage log = true 2023-09-23T03:27:54.457 INFO:teuthology.orchestra.run.smithi154.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-23T03:27:54.457 INFO:teuthology.orchestra.run.smithi154.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-23T03:27:54.457 INFO:teuthology.orchestra.run.smithi154.stdout: client mount timeout = 600 2023-09-23T03:27:54.457 INFO:teuthology.orchestra.run.smithi154.stdout: debug client = 20 2023-09-23T03:27:54.457 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:27:54.457 INFO:teuthology.orchestra.run.smithi154.stdout: pid file = "" 2023-09-23T03:27:54.457 INFO:teuthology.orchestra.run.smithi154.stdout: rados mon op timeout = 900 2023-09-23T03:27:54.457 INFO:teuthology.orchestra.run.smithi154.stdout: rados osd op timeout = 900 2023-09-23T03:27:54.457 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.a] 2023-09-23T03:27:54.457 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.b] 2023-09-23T03:27:54.458 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.c] 2023-09-23T03:27:54.458 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.a] 2023-09-23T03:27:54.458 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.c] 2023-09-23T03:27:54.458 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.b] 2023-09-23T03:27:54.458 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.d] 2023-09-23T03:27:54.458 INFO:teuthology.orchestra.run.smithi154.stdout:[mds] 2023-09-23T03:27:54.458 INFO:teuthology.orchestra.run.smithi154.stdout: debug mds = 20 2023-09-23T03:27:54.458 INFO:teuthology.orchestra.run.smithi154.stdout: debug mds balancer = 20 2023-09-23T03:27:54.458 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:27:54.458 INFO:teuthology.orchestra.run.smithi154.stdout: mds debug frag = True 2023-09-23T03:27:54.458 INFO:teuthology.orchestra.run.smithi154.stdout: mds debug scatterstat = False 2023-09-23T03:27:54.458 INFO:teuthology.orchestra.run.smithi154.stdout: mds op complaint time = 180 2023-09-23T03:27:54.459 INFO:teuthology.orchestra.run.smithi154.stdout: mds verify scatter = False 2023-09-23T03:27:54.459 INFO:teuthology.orchestra.run.smithi154.stdout: osd op complaint time = 180 2023-09-23T03:27:54.459 INFO:teuthology.orchestra.run.smithi154.stdout: rados mon op timeout = 900 2023-09-23T03:27:54.459 INFO:teuthology.orchestra.run.smithi154.stdout: rados osd op timeout = 900 2023-09-23T03:27:54.459 INFO:teuthology.orchestra.run.smithi154.stdout: mds bal merge size = 0 2023-09-23T03:27:54.469 INFO:teuthology.orchestra.run.smithi190.stdout:[global] 2023-09-23T03:27:54.470 INFO:teuthology.orchestra.run.smithi190.stdout: chdir = "" 2023-09-23T03:27:54.470 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-23T03:27:54.470 INFO:teuthology.orchestra.run.smithi190.stdout: auth supported = cephx 2023-09-23T03:27:54.470 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.470 INFO:teuthology.orchestra.run.smithi190.stdout: filestore xattr use omap = true 2023-09-23T03:27:54.470 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.470 INFO:teuthology.orchestra.run.smithi190.stdout: mon clock drift allowed = 1.000 2023-09-23T03:27:54.470 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.470 INFO:teuthology.orchestra.run.smithi190.stdout: osd crush chooseleaf type = 0 2023-09-23T03:27:54.470 INFO:teuthology.orchestra.run.smithi190.stdout: auth debug = true 2023-09-23T03:27:54.471 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.471 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on old message = true 2023-09-23T03:27:54.471 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on bug = true 2023-09-23T03:27:54.471 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.471 INFO:teuthology.orchestra.run.smithi190.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-23T03:27:54.471 INFO:teuthology.orchestra.run.smithi190.stdout: mon pg warn max object skew = 0 2023-09-23T03:27:54.471 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.471 INFO:teuthology.orchestra.run.smithi190.stdout: # disable pg_autoscaler by default for new pools 2023-09-23T03:27:54.471 INFO:teuthology.orchestra.run.smithi190.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-23T03:27:54.471 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.471 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default size = 2 2023-09-23T03:27:54.471 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.472 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow primary affinity = true 2023-09-23T03:27:54.472 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow pg remap = true 2023-09-23T03:27:54.472 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on legacy crush tunables = false 2023-09-23T03:27:54.472 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on crush straw calc version zero = false 2023-09-23T03:27:54.472 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on no sortbitwise = false 2023-09-23T03:27:54.472 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on osd down out interval zero = false 2023-09-23T03:27:54.472 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on too few osds = false 2023-09-23T03:27:54.472 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-23T03:27:54.472 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-23T03:27:54.472 INFO:teuthology.orchestra.run.smithi190.stdout: mon_allow_pool_size_one = true 2023-09-23T03:27:54.472 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.472 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-23T03:27:54.473 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.473 INFO:teuthology.orchestra.run.smithi190.stdout: osd default data pool replay window = 5 2023-09-23T03:27:54.473 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.473 INFO:teuthology.orchestra.run.smithi190.stdout: mon allow pool delete = true 2023-09-23T03:27:54.473 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.473 INFO:teuthology.orchestra.run.smithi190.stdout: mon cluster log file level = debug 2023-09-23T03:27:54.473 INFO:teuthology.orchestra.run.smithi190.stdout: debug asserts on shutdown = true 2023-09-23T03:27:54.473 INFO:teuthology.orchestra.run.smithi190.stdout: mon health detail to clog = false 2023-09-23T03:27:54.473 INFO:teuthology.orchestra.run.smithi190.stdout: mon host = "172.21.15.154,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]" 2023-09-23T03:27:54.473 INFO:teuthology.orchestra.run.smithi190.stdout: fsid = 8c8039d3-6c84-43db-8808-85e0dbdd47c8 2023-09-23T03:27:54.473 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.473 INFO:teuthology.orchestra.run.smithi190.stdout:[osd] 2023-09-23T03:27:54.474 INFO:teuthology.orchestra.run.smithi190.stdout: osd journal size = 100 2023-09-23T03:27:54.474 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.474 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub load threshold = 5.0 2023-09-23T03:27:54.474 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub max interval = 600 2023-09-23T03:27:54.474 INFO:teuthology.orchestra.run.smithi190.stdout: osd mclock profile = high_recovery_ops 2023-09-23T03:27:54.474 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.474 INFO:teuthology.orchestra.run.smithi190.stdout: osd recover clone overlap = true 2023-09-23T03:27:54.474 INFO:teuthology.orchestra.run.smithi190.stdout: osd recovery max chunk = 1048576 2023-09-23T03:27:54.474 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.474 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug shutdown = true 2023-09-23T03:27:54.474 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug op order = true 2023-09-23T03:27:54.475 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug verify stray on activate = true 2023-09-23T03:27:54.475 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.475 INFO:teuthology.orchestra.run.smithi190.stdout: osd open classes on start = true 2023-09-23T03:27:54.475 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug pg log writeout = true 2023-09-23T03:27:54.475 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.475 INFO:teuthology.orchestra.run.smithi190.stdout: osd deep scrub update digest min age = 30 2023-09-23T03:27:54.475 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.475 INFO:teuthology.orchestra.run.smithi190.stdout: osd map max advance = 10 2023-09-23T03:27:54.475 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.475 INFO:teuthology.orchestra.run.smithi190.stdout: journal zero on create = true 2023-09-23T03:27:54.475 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.475 INFO:teuthology.orchestra.run.smithi190.stdout: filestore ondisk finisher threads = 3 2023-09-23T03:27:54.475 INFO:teuthology.orchestra.run.smithi190.stdout: filestore apply finisher threads = 3 2023-09-23T03:27:54.476 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.476 INFO:teuthology.orchestra.run.smithi190.stdout: bdev debug aio = true 2023-09-23T03:27:54.476 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug misdirected ops = true 2023-09-23T03:27:54.476 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore block size = 96636764160 2023-09-23T03:27:54.476 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore fsck on mount = True 2023-09-23T03:27:54.476 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluefs = 20 2023-09-23T03:27:54.476 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluestore = 20 2023-09-23T03:27:54.476 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:27:54.476 INFO:teuthology.orchestra.run.smithi190.stdout: debug osd = 20 2023-09-23T03:27:54.476 INFO:teuthology.orchestra.run.smithi190.stdout: debug rocksdb = 10 2023-09-23T03:27:54.476 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd backfillfull_ratio = 0.85 2023-09-23T03:27:54.476 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd full ratio = 0.9 2023-09-23T03:27:54.477 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd nearfull ratio = 0.8 2023-09-23T03:27:54.477 INFO:teuthology.orchestra.run.smithi190.stdout: osd failsafe full ratio = 0.95 2023-09-23T03:27:54.477 INFO:teuthology.orchestra.run.smithi190.stdout: osd objectstore = bluestore 2023-09-23T03:27:54.477 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2023-09-23T03:27:54.477 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.477 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr] 2023-09-23T03:27:54.477 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:27:54.477 INFO:teuthology.orchestra.run.smithi190.stdout: debug mgr = 20 2023-09-23T03:27:54.477 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2023-09-23T03:27:54.477 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2023-09-23T03:27:54.477 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:27:54.478 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:27:54.478 INFO:teuthology.orchestra.run.smithi190.stdout: mgr/telemetry/nag = false 2023-09-23T03:27:54.478 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.478 INFO:teuthology.orchestra.run.smithi190.stdout:[mon] 2023-09-23T03:27:54.478 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:27:54.478 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2023-09-23T03:27:54.478 INFO:teuthology.orchestra.run.smithi190.stdout: debug paxos = 20 2023-09-23T03:27:54.478 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2023-09-23T03:27:54.478 INFO:teuthology.orchestra.run.smithi190.stdout: mon data avail warn = 5 2023-09-23T03:27:54.478 INFO:teuthology.orchestra.run.smithi190.stdout: mon mgr mkfs grace = 240 2023-09-23T03:27:54.478 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:27:54.478 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd reporter subtree level = osd 2023-09-23T03:27:54.479 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd prime pg temp = true 2023-09-23T03:27:54.479 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:27:54.479 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.479 INFO:teuthology.orchestra.run.smithi190.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-23T03:27:54.479 INFO:teuthology.orchestra.run.smithi190.stdout: auth mon ticket ttl = 660 # 11m 2023-09-23T03:27:54.479 INFO:teuthology.orchestra.run.smithi190.stdout: auth service ticket ttl = 240 # 4m 2023-09-23T03:27:54.479 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.479 INFO:teuthology.orchestra.run.smithi190.stdout: # don't complain about insecure global_id in the test suite 2023-09-23T03:27:54.479 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-23T03:27:54.479 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-23T03:27:54.479 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.479 INFO:teuthology.orchestra.run.smithi190.stdout: # 1m isn't quite enough 2023-09-23T03:27:54.480 INFO:teuthology.orchestra.run.smithi190.stdout: mon_down_mkfs_grace = 2m 2023-09-23T03:27:54.480 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.480 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_filestore_osds = false 2023-09-23T03:27:54.480 INFO:teuthology.orchestra.run.smithi190.stdout: mon op complaint time = 120 2023-09-23T03:27:54.480 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.480 INFO:teuthology.orchestra.run.smithi190.stdout:[client] 2023-09-23T03:27:54.480 INFO:teuthology.orchestra.run.smithi190.stdout: rgw cache enabled = true 2023-09-23T03:27:54.480 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable ops log = true 2023-09-23T03:27:54.480 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable usage log = true 2023-09-23T03:27:54.480 INFO:teuthology.orchestra.run.smithi190.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-23T03:27:54.480 INFO:teuthology.orchestra.run.smithi190.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-23T03:27:54.480 INFO:teuthology.orchestra.run.smithi190.stdout: client mount timeout = 600 2023-09-23T03:27:54.481 INFO:teuthology.orchestra.run.smithi190.stdout: debug client = 20 2023-09-23T03:27:54.481 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:27:54.481 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = "" 2023-09-23T03:27:54.481 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 900 2023-09-23T03:27:54.481 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 900 2023-09-23T03:27:54.481 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.a] 2023-09-23T03:27:54.481 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.b] 2023-09-23T03:27:54.481 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.c] 2023-09-23T03:27:54.481 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.a] 2023-09-23T03:27:54.481 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.c] 2023-09-23T03:27:54.481 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.b] 2023-09-23T03:27:54.481 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.d] 2023-09-23T03:27:54.481 INFO:teuthology.orchestra.run.smithi190.stdout:[mds] 2023-09-23T03:27:54.482 INFO:teuthology.orchestra.run.smithi190.stdout: debug mds = 20 2023-09-23T03:27:54.482 INFO:teuthology.orchestra.run.smithi190.stdout: debug mds balancer = 20 2023-09-23T03:27:54.482 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:27:54.482 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug frag = True 2023-09-23T03:27:54.482 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug scatterstat = False 2023-09-23T03:27:54.482 INFO:teuthology.orchestra.run.smithi190.stdout: mds op complaint time = 180 2023-09-23T03:27:54.482 INFO:teuthology.orchestra.run.smithi190.stdout: mds verify scatter = False 2023-09-23T03:27:54.482 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2023-09-23T03:27:54.482 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 900 2023-09-23T03:27:54.482 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 900 2023-09-23T03:27:54.482 INFO:teuthology.orchestra.run.smithi190.stdout: mds bal merge size = 0 2023-09-23T03:27:54.515 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-23T03:27:54.521 DEBUG:teuthology.orchestra.run.smithi190:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-23T03:27:54.692 INFO:teuthology.orchestra.run.smithi190.stdout:[global] 2023-09-23T03:27:54.692 INFO:teuthology.orchestra.run.smithi190.stdout: chdir = "" 2023-09-23T03:27:54.692 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-23T03:27:54.692 INFO:teuthology.orchestra.run.smithi190.stdout: auth supported = cephx 2023-09-23T03:27:54.692 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.692 INFO:teuthology.orchestra.run.smithi190.stdout: filestore xattr use omap = true 2023-09-23T03:27:54.692 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.692 INFO:teuthology.orchestra.run.smithi190.stdout: mon clock drift allowed = 1.000 2023-09-23T03:27:54.692 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.692 INFO:teuthology.orchestra.run.smithi190.stdout: osd crush chooseleaf type = 0 2023-09-23T03:27:54.693 INFO:teuthology.orchestra.run.smithi190.stdout: auth debug = true 2023-09-23T03:27:54.693 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.693 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on old message = true 2023-09-23T03:27:54.693 INFO:teuthology.orchestra.run.smithi190.stdout: ms die on bug = true 2023-09-23T03:27:54.693 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.693 INFO:teuthology.orchestra.run.smithi190.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-23T03:27:54.693 INFO:teuthology.orchestra.run.smithi190.stdout: mon pg warn max object skew = 0 2023-09-23T03:27:54.693 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.693 INFO:teuthology.orchestra.run.smithi190.stdout: # disable pg_autoscaler by default for new pools 2023-09-23T03:27:54.693 INFO:teuthology.orchestra.run.smithi190.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-23T03:27:54.693 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.693 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default size = 2 2023-09-23T03:27:54.694 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.694 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow primary affinity = true 2023-09-23T03:27:54.694 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd allow pg remap = true 2023-09-23T03:27:54.694 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on legacy crush tunables = false 2023-09-23T03:27:54.694 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on crush straw calc version zero = false 2023-09-23T03:27:54.694 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on no sortbitwise = false 2023-09-23T03:27:54.694 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on osd down out interval zero = false 2023-09-23T03:27:54.694 INFO:teuthology.orchestra.run.smithi190.stdout: mon warn on too few osds = false 2023-09-23T03:27:54.694 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-23T03:27:54.694 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-23T03:27:54.694 INFO:teuthology.orchestra.run.smithi190.stdout: mon_allow_pool_size_one = true 2023-09-23T03:27:54.695 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.695 INFO:teuthology.orchestra.run.smithi190.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-23T03:27:54.695 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.695 INFO:teuthology.orchestra.run.smithi190.stdout: osd default data pool replay window = 5 2023-09-23T03:27:54.695 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.695 INFO:teuthology.orchestra.run.smithi190.stdout: mon allow pool delete = true 2023-09-23T03:27:54.695 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.695 INFO:teuthology.orchestra.run.smithi190.stdout: mon cluster log file level = debug 2023-09-23T03:27:54.695 INFO:teuthology.orchestra.run.smithi190.stdout: debug asserts on shutdown = true 2023-09-23T03:27:54.695 INFO:teuthology.orchestra.run.smithi190.stdout: mon health detail to clog = false 2023-09-23T03:27:54.695 INFO:teuthology.orchestra.run.smithi190.stdout: mon host = "172.21.15.154,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]" 2023-09-23T03:27:54.695 INFO:teuthology.orchestra.run.smithi190.stdout: fsid = 8c8039d3-6c84-43db-8808-85e0dbdd47c8 2023-09-23T03:27:54.695 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.696 INFO:teuthology.orchestra.run.smithi190.stdout:[osd] 2023-09-23T03:27:54.696 INFO:teuthology.orchestra.run.smithi190.stdout: osd journal size = 100 2023-09-23T03:27:54.696 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.696 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub load threshold = 5.0 2023-09-23T03:27:54.696 INFO:teuthology.orchestra.run.smithi190.stdout: osd scrub max interval = 600 2023-09-23T03:27:54.696 INFO:teuthology.orchestra.run.smithi190.stdout: osd mclock profile = high_recovery_ops 2023-09-23T03:27:54.696 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.696 INFO:teuthology.orchestra.run.smithi190.stdout: osd recover clone overlap = true 2023-09-23T03:27:54.696 INFO:teuthology.orchestra.run.smithi190.stdout: osd recovery max chunk = 1048576 2023-09-23T03:27:54.696 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.696 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug shutdown = true 2023-09-23T03:27:54.696 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug op order = true 2023-09-23T03:27:54.697 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug verify stray on activate = true 2023-09-23T03:27:54.697 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.697 INFO:teuthology.orchestra.run.smithi190.stdout: osd open classes on start = true 2023-09-23T03:27:54.697 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug pg log writeout = true 2023-09-23T03:27:54.697 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.697 INFO:teuthology.orchestra.run.smithi190.stdout: osd deep scrub update digest min age = 30 2023-09-23T03:27:54.697 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.697 INFO:teuthology.orchestra.run.smithi190.stdout: osd map max advance = 10 2023-09-23T03:27:54.697 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.697 INFO:teuthology.orchestra.run.smithi190.stdout: journal zero on create = true 2023-09-23T03:27:54.697 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.697 INFO:teuthology.orchestra.run.smithi190.stdout: filestore ondisk finisher threads = 3 2023-09-23T03:27:54.698 INFO:teuthology.orchestra.run.smithi190.stdout: filestore apply finisher threads = 3 2023-09-23T03:27:54.698 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.698 INFO:teuthology.orchestra.run.smithi190.stdout: bdev debug aio = true 2023-09-23T03:27:54.698 INFO:teuthology.orchestra.run.smithi190.stdout: osd debug misdirected ops = true 2023-09-23T03:27:54.698 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore block size = 96636764160 2023-09-23T03:27:54.698 INFO:teuthology.orchestra.run.smithi190.stdout: bluestore fsck on mount = True 2023-09-23T03:27:54.698 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluefs = 20 2023-09-23T03:27:54.698 INFO:teuthology.orchestra.run.smithi190.stdout: debug bluestore = 20 2023-09-23T03:27:54.698 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:27:54.698 INFO:teuthology.orchestra.run.smithi190.stdout: debug osd = 20 2023-09-23T03:27:54.698 INFO:teuthology.orchestra.run.smithi190.stdout: debug rocksdb = 10 2023-09-23T03:27:54.699 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd backfillfull_ratio = 0.85 2023-09-23T03:27:54.699 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd full ratio = 0.9 2023-09-23T03:27:54.699 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd nearfull ratio = 0.8 2023-09-23T03:27:54.699 INFO:teuthology.orchestra.run.smithi190.stdout: osd failsafe full ratio = 0.95 2023-09-23T03:27:54.699 INFO:teuthology.orchestra.run.smithi190.stdout: osd objectstore = bluestore 2023-09-23T03:27:54.699 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2023-09-23T03:27:54.699 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.699 INFO:teuthology.orchestra.run.smithi190.stdout:[mgr] 2023-09-23T03:27:54.699 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:27:54.699 INFO:teuthology.orchestra.run.smithi190.stdout: debug mgr = 20 2023-09-23T03:27:54.700 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2023-09-23T03:27:54.700 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2023-09-23T03:27:54.700 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:27:54.700 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:27:54.700 INFO:teuthology.orchestra.run.smithi190.stdout: mgr/telemetry/nag = false 2023-09-23T03:27:54.700 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.700 INFO:teuthology.orchestra.run.smithi190.stdout:[mon] 2023-09-23T03:27:54.700 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:27:54.700 INFO:teuthology.orchestra.run.smithi190.stdout: debug mon = 20 2023-09-23T03:27:54.700 INFO:teuthology.orchestra.run.smithi190.stdout: debug paxos = 20 2023-09-23T03:27:54.701 INFO:teuthology.orchestra.run.smithi190.stdout: debug auth = 20 2023-09-23T03:27:54.701 INFO:teuthology.orchestra.run.smithi190.stdout: mon data avail warn = 5 2023-09-23T03:27:54.701 INFO:teuthology.orchestra.run.smithi190.stdout: mon mgr mkfs grace = 240 2023-09-23T03:27:54.701 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:27:54.701 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd reporter subtree level = osd 2023-09-23T03:27:54.702 INFO:teuthology.orchestra.run.smithi190.stdout: mon osd prime pg temp = true 2023-09-23T03:27:54.702 INFO:teuthology.orchestra.run.smithi190.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:27:54.702 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.702 INFO:teuthology.orchestra.run.smithi190.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-23T03:27:54.702 INFO:teuthology.orchestra.run.smithi190.stdout: auth mon ticket ttl = 660 # 11m 2023-09-23T03:27:54.702 INFO:teuthology.orchestra.run.smithi190.stdout: auth service ticket ttl = 240 # 4m 2023-09-23T03:27:54.703 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.703 INFO:teuthology.orchestra.run.smithi190.stdout: # don't complain about insecure global_id in the test suite 2023-09-23T03:27:54.703 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-23T03:27:54.703 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-23T03:27:54.703 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.703 INFO:teuthology.orchestra.run.smithi190.stdout: # 1m isn't quite enough 2023-09-23T03:27:54.704 INFO:teuthology.orchestra.run.smithi190.stdout: mon_down_mkfs_grace = 2m 2023-09-23T03:27:54.704 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.704 INFO:teuthology.orchestra.run.smithi190.stdout: mon_warn_on_filestore_osds = false 2023-09-23T03:27:54.704 INFO:teuthology.orchestra.run.smithi190.stdout: mon op complaint time = 120 2023-09-23T03:27:54.704 INFO:teuthology.orchestra.run.smithi190.stdout: 2023-09-23T03:27:54.704 INFO:teuthology.orchestra.run.smithi190.stdout:[client] 2023-09-23T03:27:54.705 INFO:teuthology.orchestra.run.smithi190.stdout: rgw cache enabled = true 2023-09-23T03:27:54.705 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable ops log = true 2023-09-23T03:27:54.705 INFO:teuthology.orchestra.run.smithi190.stdout: rgw enable usage log = true 2023-09-23T03:27:54.705 INFO:teuthology.orchestra.run.smithi190.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-23T03:27:54.705 INFO:teuthology.orchestra.run.smithi190.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-23T03:27:54.705 INFO:teuthology.orchestra.run.smithi190.stdout: client mount timeout = 600 2023-09-23T03:27:54.706 INFO:teuthology.orchestra.run.smithi190.stdout: debug client = 20 2023-09-23T03:27:54.706 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:27:54.706 INFO:teuthology.orchestra.run.smithi190.stdout: pid file = "" 2023-09-23T03:27:54.706 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 900 2023-09-23T03:27:54.706 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 900 2023-09-23T03:27:54.707 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.a] 2023-09-23T03:27:54.707 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.b] 2023-09-23T03:27:54.707 INFO:teuthology.orchestra.run.smithi190.stdout:[mon.c] 2023-09-23T03:27:54.707 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.a] 2023-09-23T03:27:54.707 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.c] 2023-09-23T03:27:54.708 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.b] 2023-09-23T03:27:54.708 INFO:teuthology.orchestra.run.smithi190.stdout:[mds.d] 2023-09-23T03:27:54.708 INFO:teuthology.orchestra.run.smithi190.stdout:[mds] 2023-09-23T03:27:54.708 INFO:teuthology.orchestra.run.smithi190.stdout: debug mds = 20 2023-09-23T03:27:54.708 INFO:teuthology.orchestra.run.smithi190.stdout: debug mds balancer = 20 2023-09-23T03:27:54.708 INFO:teuthology.orchestra.run.smithi190.stdout: debug ms = 1 2023-09-23T03:27:54.708 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug frag = True 2023-09-23T03:27:54.709 INFO:teuthology.orchestra.run.smithi190.stdout: mds debug scatterstat = False 2023-09-23T03:27:54.709 INFO:teuthology.orchestra.run.smithi190.stdout: mds op complaint time = 180 2023-09-23T03:27:54.709 INFO:teuthology.orchestra.run.smithi190.stdout: mds verify scatter = False 2023-09-23T03:27:54.709 INFO:teuthology.orchestra.run.smithi190.stdout: osd op complaint time = 180 2023-09-23T03:27:54.709 INFO:teuthology.orchestra.run.smithi190.stdout: rados mon op timeout = 900 2023-09-23T03:27:54.710 INFO:teuthology.orchestra.run.smithi190.stdout: rados osd op timeout = 900 2023-09-23T03:27:54.710 INFO:teuthology.orchestra.run.smithi190.stdout: mds bal merge size = 0 2023-09-23T03:27:54.710 INFO:teuthology.orchestra.run.smithi190.stdout: mds bal split size = 10000 2023-09-23T03:27:54.710 INFO:teuthology.orchestra.run.smithi154.stdout:[global] 2023-09-23T03:27:54.710 INFO:teuthology.orchestra.run.smithi154.stdout: chdir = "" 2023-09-23T03:27:54.711 INFO:teuthology.orchestra.run.smithi154.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-23T03:27:54.711 INFO:teuthology.orchestra.run.smithi154.stdout: auth supported = cephx 2023-09-23T03:27:54.711 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.711 INFO:teuthology.orchestra.run.smithi154.stdout: filestore xattr use omap = true 2023-09-23T03:27:54.711 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.712 INFO:teuthology.orchestra.run.smithi154.stdout: mon clock drift allowed = 1.000 2023-09-23T03:27:54.712 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.712 INFO:teuthology.orchestra.run.smithi154.stdout: osd crush chooseleaf type = 0 2023-09-23T03:27:54.712 INFO:teuthology.orchestra.run.smithi154.stdout: auth debug = true 2023-09-23T03:27:54.712 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.712 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on old message = true 2023-09-23T03:27:54.713 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on bug = true 2023-09-23T03:27:54.713 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.713 INFO:teuthology.orchestra.run.smithi154.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-23T03:27:54.713 INFO:teuthology.orchestra.run.smithi154.stdout: mon pg warn max object skew = 0 2023-09-23T03:27:54.713 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.713 INFO:teuthology.orchestra.run.smithi154.stdout: # disable pg_autoscaler by default for new pools 2023-09-23T03:27:54.714 INFO:teuthology.orchestra.run.smithi154.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-23T03:27:54.714 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.714 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default size = 2 2023-09-23T03:27:54.714 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.714 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow primary affinity = true 2023-09-23T03:27:54.715 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow pg remap = true 2023-09-23T03:27:54.715 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on legacy crush tunables = false 2023-09-23T03:27:54.715 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on crush straw calc version zero = false 2023-09-23T03:27:54.715 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on no sortbitwise = false 2023-09-23T03:27:54.715 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on osd down out interval zero = false 2023-09-23T03:27:54.715 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on too few osds = false 2023-09-23T03:27:54.716 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-23T03:27:54.716 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-23T03:27:54.716 INFO:teuthology.orchestra.run.smithi154.stdout: mon_allow_pool_size_one = true 2023-09-23T03:27:54.716 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.716 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-23T03:27:54.716 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.717 INFO:teuthology.orchestra.run.smithi154.stdout: osd default data pool replay window = 5 2023-09-23T03:27:54.717 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.717 INFO:teuthology.orchestra.run.smithi154.stdout: mon allow pool delete = true 2023-09-23T03:27:54.717 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.717 INFO:teuthology.orchestra.run.smithi154.stdout: mon cluster log file level = debug 2023-09-23T03:27:54.718 INFO:teuthology.orchestra.run.smithi154.stdout: debug asserts on shutdown = true 2023-09-23T03:27:54.718 INFO:teuthology.orchestra.run.smithi154.stdout: mon health detail to clog = false 2023-09-23T03:27:54.718 INFO:teuthology.orchestra.run.smithi154.stdout: mon host = "172.21.15.154,172.21.15.190,[v2:172.21.15.190:3301,v1:172.21.15.190:6790]" 2023-09-23T03:27:54.718 INFO:teuthology.orchestra.run.smithi154.stdout: fsid = 8c8039d3-6c84-43db-8808-85e0dbdd47c8 2023-09-23T03:27:54.718 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.718 INFO:teuthology.orchestra.run.smithi154.stdout:[osd] 2023-09-23T03:27:54.719 INFO:teuthology.orchestra.run.smithi154.stdout: osd journal size = 100 2023-09-23T03:27:54.719 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.719 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub load threshold = 5.0 2023-09-23T03:27:54.719 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub max interval = 600 2023-09-23T03:27:54.719 INFO:teuthology.orchestra.run.smithi154.stdout: osd mclock profile = high_recovery_ops 2023-09-23T03:27:54.720 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.720 INFO:teuthology.orchestra.run.smithi154.stdout: osd recover clone overlap = true 2023-09-23T03:27:54.720 INFO:teuthology.orchestra.run.smithi154.stdout: osd recovery max chunk = 1048576 2023-09-23T03:27:54.720 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.720 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug shutdown = true 2023-09-23T03:27:54.720 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug op order = true 2023-09-23T03:27:54.721 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug verify stray on activate = true 2023-09-23T03:27:54.721 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.721 INFO:teuthology.orchestra.run.smithi154.stdout: osd open classes on start = true 2023-09-23T03:27:54.721 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug pg log writeout = true 2023-09-23T03:27:54.721 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.721 INFO:teuthology.orchestra.run.smithi154.stdout: osd deep scrub update digest min age = 30 2023-09-23T03:27:54.722 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.722 INFO:teuthology.orchestra.run.smithi154.stdout: osd map max advance = 10 2023-09-23T03:27:54.722 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.722 INFO:teuthology.orchestra.run.smithi154.stdout: journal zero on create = true 2023-09-23T03:27:54.722 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.722 INFO:teuthology.orchestra.run.smithi154.stdout: filestore ondisk finisher threads = 3 2023-09-23T03:27:54.722 INFO:teuthology.orchestra.run.smithi154.stdout: filestore apply finisher threads = 3 2023-09-23T03:27:54.722 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.723 INFO:teuthology.orchestra.run.smithi154.stdout: bdev debug aio = true 2023-09-23T03:27:54.723 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug misdirected ops = true 2023-09-23T03:27:54.723 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore block size = 96636764160 2023-09-23T03:27:54.723 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore fsck on mount = True 2023-09-23T03:27:54.723 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluefs = 20 2023-09-23T03:27:54.723 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluestore = 20 2023-09-23T03:27:54.723 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:27:54.724 INFO:teuthology.orchestra.run.smithi154.stdout: debug osd = 20 2023-09-23T03:27:54.724 INFO:teuthology.orchestra.run.smithi154.stdout: debug rocksdb = 10 2023-09-23T03:27:54.724 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd backfillfull_ratio = 0.85 2023-09-23T03:27:54.724 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd full ratio = 0.9 2023-09-23T03:27:54.724 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd nearfull ratio = 0.8 2023-09-23T03:27:54.724 INFO:teuthology.orchestra.run.smithi154.stdout: osd failsafe full ratio = 0.95 2023-09-23T03:27:54.724 INFO:teuthology.orchestra.run.smithi154.stdout: osd objectstore = bluestore 2023-09-23T03:27:54.725 INFO:teuthology.orchestra.run.smithi154.stdout: osd op complaint time = 180 2023-09-23T03:27:54.725 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.725 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr] 2023-09-23T03:27:54.725 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:27:54.725 INFO:teuthology.orchestra.run.smithi154.stdout: debug mgr = 20 2023-09-23T03:27:54.725 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2023-09-23T03:27:54.725 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2023-09-23T03:27:54.725 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:27:54.725 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:27:54.725 INFO:teuthology.orchestra.run.smithi154.stdout: mgr/telemetry/nag = false 2023-09-23T03:27:54.725 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.725 INFO:teuthology.orchestra.run.smithi154.stdout:[mon] 2023-09-23T03:27:54.725 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:27:54.726 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2023-09-23T03:27:54.726 INFO:teuthology.orchestra.run.smithi154.stdout: debug paxos = 20 2023-09-23T03:27:54.726 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2023-09-23T03:27:54.726 INFO:teuthology.orchestra.run.smithi154.stdout: mon data avail warn = 5 2023-09-23T03:27:54.726 INFO:teuthology.orchestra.run.smithi154.stdout: mon mgr mkfs grace = 240 2023-09-23T03:27:54.726 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2023-09-23T03:27:54.726 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd reporter subtree level = osd 2023-09-23T03:27:54.726 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd prime pg temp = true 2023-09-23T03:27:54.726 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2023-09-23T03:27:54.726 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.726 INFO:teuthology.orchestra.run.smithi154.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-23T03:27:54.727 INFO:teuthology.orchestra.run.smithi154.stdout: auth mon ticket ttl = 660 # 11m 2023-09-23T03:27:54.727 INFO:teuthology.orchestra.run.smithi154.stdout: auth service ticket ttl = 240 # 4m 2023-09-23T03:27:54.727 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.727 INFO:teuthology.orchestra.run.smithi154.stdout: # don't complain about insecure global_id in the test suite 2023-09-23T03:27:54.727 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-23T03:27:54.727 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-23T03:27:54.727 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.727 INFO:teuthology.orchestra.run.smithi154.stdout: # 1m isn't quite enough 2023-09-23T03:27:54.727 INFO:teuthology.orchestra.run.smithi154.stdout: mon_down_mkfs_grace = 2m 2023-09-23T03:27:54.727 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.727 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_filestore_osds = false 2023-09-23T03:27:54.727 INFO:teuthology.orchestra.run.smithi154.stdout: mon op complaint time = 120 2023-09-23T03:27:54.727 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:54.728 INFO:teuthology.orchestra.run.smithi154.stdout:[client] 2023-09-23T03:27:54.728 INFO:teuthology.orchestra.run.smithi154.stdout: rgw cache enabled = true 2023-09-23T03:27:54.728 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable ops log = true 2023-09-23T03:27:54.728 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable usage log = true 2023-09-23T03:27:54.728 INFO:teuthology.orchestra.run.smithi154.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-23T03:27:54.728 INFO:teuthology.orchestra.run.smithi154.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-23T03:27:54.728 INFO:teuthology.orchestra.run.smithi154.stdout: client mount timeout = 600 2023-09-23T03:27:54.728 INFO:teuthology.orchestra.run.smithi154.stdout: debug client = 20 2023-09-23T03:27:54.728 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:27:54.728 INFO:teuthology.orchestra.run.smithi154.stdout: pid file = "" 2023-09-23T03:27:54.728 INFO:teuthology.orchestra.run.smithi154.stdout: rados mon op timeout = 900 2023-09-23T03:27:54.728 INFO:teuthology.orchestra.run.smithi154.stdout: rados osd op timeout = 900 2023-09-23T03:27:54.729 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.a] 2023-09-23T03:27:54.729 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.b] 2023-09-23T03:27:54.729 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.c] 2023-09-23T03:27:54.729 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.a] 2023-09-23T03:27:54.729 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.c] 2023-09-23T03:27:54.729 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.b] 2023-09-23T03:27:54.729 INFO:teuthology.orchestra.run.smithi154.stdout:[mds.d] 2023-09-23T03:27:54.729 INFO:teuthology.orchestra.run.smithi154.stdout:[mds] 2023-09-23T03:27:54.729 INFO:teuthology.orchestra.run.smithi154.stdout: debug mds = 20 2023-09-23T03:27:54.729 INFO:teuthology.orchestra.run.smithi154.stdout: debug mds balancer = 20 2023-09-23T03:27:54.730 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2023-09-23T03:27:54.730 INFO:teuthology.orchestra.run.smithi154.stdout: mds debug frag = True 2023-09-23T03:27:54.730 INFO:teuthology.orchestra.run.smithi154.stdout: mds debug scatterstat = False 2023-09-23T03:27:54.730 INFO:teuthology.orchestra.run.smithi154.stdout: mds op complaint time = 180 2023-09-23T03:27:54.730 INFO:teuthology.orchestra.run.smithi154.stdout: mds verify scatter = False 2023-09-23T03:27:54.730 INFO:teuthology.orchestra.run.smithi154.stdout: osd op complaint time = 180 2023-09-23T03:27:54.730 INFO:teuthology.orchestra.run.smithi154.stdout: rados mon op timeout = 900 2023-09-23T03:27:54.730 INFO:teuthology.orchestra.run.smithi154.stdout: rados osd op timeout = 900 2023-09-23T03:27:54.730 INFO:teuthology.orchestra.run.smithi154.stdout: mds bal merge size = 0 2023-09-23T03:27:54.730 INFO:teuthology.orchestra.run.smithi154.stdout: mds bal split size = 10000 2023-09-23T03:27:54.739 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:27:54.739 DEBUG:teuthology.orchestra.run.smithi154:> import os 2023-09-23T03:27:54.739 DEBUG:teuthology.orchestra.run.smithi154:> path = os.path.join("/home/ubuntu/cephtest/mnt.0", "subdir") 2023-09-23T03:27:54.739 DEBUG:teuthology.orchestra.run.smithi154:> os.mkdir(path) 2023-09-23T03:27:54.739 DEBUG:teuthology.orchestra.run.smithi154:> for n in range(0, 100): 2023-09-23T03:27:54.739 DEBUG:teuthology.orchestra.run.smithi154:> open(os.path.join(path, "%s" % n), '"'"'w'"'"').write("%s" % n) 2023-09-23T03:27:54.739 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:27:55.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:27:55.490 DEBUG:teuthology.orchestra.run.smithi154:> stdin-killer -- python3 -c ' 2023-09-23T03:27:55.490 DEBUG:teuthology.orchestra.run.smithi154:> import os 2023-09-23T03:27:55.490 DEBUG:teuthology.orchestra.run.smithi154:> import stat 2023-09-23T03:27:55.491 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:27:55.491 DEBUG:teuthology.orchestra.run.smithi154:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2023-09-23T03:27:55.491 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:27:55.677 INFO:teuthology.orchestra.run.smithi154.stdout:1099511627776 2023-09-23T03:27:55.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:27:55.690 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:27:55.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.954+0000 7f57f724d640 1 Processor -- start 2023-09-23T03:27:55.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.954+0000 7f57f724d640 1 -- start start 2023-09-23T03:27:55.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.955+0000 7f57f724d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f57f015cbe0 0x7f57f0160520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:55.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.955+0000 7f57f724d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f0161430 con 0x7f57f00cacb0 2023-09-23T03:27:55.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.955+0000 7f57f724d640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f0160b90 con 0x7f57f00cdc80 2023-09-23T03:27:55.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.955+0000 7f57f724d640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57f0160e70 con 0x7f57f015cbe0 2023-09-23T03:27:55.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.956+0000 7f57f624b640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f57f00cacb0 0x7f57f00cb090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46622/0 (socket says 172.21.15.154:46622) 2023-09-23T03:27:55.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.956+0000 7f57f624b640 1 -- 172.21.15.154:0/3694740243 learned_addr learned my addr 172.21.15.154:0/3694740243 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:27:55.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.956+0000 7f57f5a4a640 1 --2- 172.21.15.154:0/3694740243 >> v2:172.21.15.190:3300/0 conn(0x7f57f015cbe0 0x7f57f0160520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:55.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.956+0000 7f57f5249640 1 -- 172.21.15.154:0/3694740243 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 328294180 0 0) 0x7f57f0161430 con 0x7f57f00cacb0 2023-09-23T03:27:55.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.956+0000 7f57f5249640 1 -- 172.21.15.154:0/3694740243 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57dc003520 con 0x7f57f00cacb0 2023-09-23T03:27:55.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.957+0000 7f57f5249640 1 -- 172.21.15.154:0/3694740243 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1485147856 0 0) 0x7f57dc003520 con 0x7f57f00cacb0 2023-09-23T03:27:55.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.957+0000 7f57f5249640 1 -- 172.21.15.154:0/3694740243 >> v2:172.21.15.190:3300/0 conn(0x7f57f015cbe0 msgr2=0x7f57f0160520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:55.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.957+0000 7f57f5249640 1 --2- 172.21.15.154:0/3694740243 >> v2:172.21.15.190:3300/0 conn(0x7f57f015cbe0 0x7f57f0160520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:55.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.957+0000 7f57f5249640 1 -- 172.21.15.154:0/3694740243 >> v1:172.21.15.190:6789/0 conn(0x7f57f00cdc80 legacy=0x7f57f00cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:55.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.957+0000 7f57f5249640 1 -- 172.21.15.154:0/3694740243 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f0161150 con 0x7f57f00cacb0 2023-09-23T03:27:55.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.957+0000 7f57f5249640 1 -- 172.21.15.154:0/3694740243 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f57ec001670 con 0x7f57f00cacb0 2023-09-23T03:27:55.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.957+0000 7f57f5249640 1 -- 172.21.15.154:0/3694740243 >> v1:172.21.15.154:6789/0 conn(0x7f57f00cacb0 legacy=0x7f57f00cb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:55.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.957+0000 7f57f5249640 1 --2- 172.21.15.154:0/3694740243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57dc003970 0x7f57dc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:55.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.957+0000 7f57f5249640 1 --2- 172.21.15.154:0/3694740243 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57dc0047c0 0x7f57dc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:55.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.957+0000 7f57f5249640 1 --2- 172.21.15.154:0/3694740243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57dc0052a0 0x7f57dc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:55.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.957+0000 7f57f5249640 1 -- 172.21.15.154:0/3694740243 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f0160e70 con 0x7f57dc003970 2023-09-23T03:27:55.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.957+0000 7f57f5249640 1 -- 172.21.15.154:0/3694740243 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc00c170 con 0x7f57dc0052a0 2023-09-23T03:27:55.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.957+0000 7f57f5249640 1 -- 172.21.15.154:0/3694740243 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc00c330 con 0x7f57dc0047c0 2023-09-23T03:27:55.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.957+0000 7f57f624b640 1 --2- 172.21.15.154:0/3694740243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57dc003970 0x7f57dc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:55.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.957+0000 7f57f6a4c640 1 --2- 172.21.15.154:0/3694740243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57dc0052a0 0x7f57dc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:55.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.958+0000 7f57f5a4a640 1 --2- 172.21.15.154:0/3694740243 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57dc0047c0 0x7f57dc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:55.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.958+0000 7f57f624b640 1 -- 172.21.15.154:0/3694740243 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57dc0047c0 msgr2=0x7f57dc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:55.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.958+0000 7f57f624b640 1 --2- 172.21.15.154:0/3694740243 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57dc0047c0 0x7f57dc004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:55.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.958+0000 7f57f624b640 1 -- 172.21.15.154:0/3694740243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57dc0052a0 msgr2=0x7f57dc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:55.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.958+0000 7f57f624b640 1 --2- 172.21.15.154:0/3694740243 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57dc0052a0 0x7f57dc00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:55.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.958+0000 7f57f624b640 1 -- 172.21.15.154:0/3694740243 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57dc003520 con 0x7f57dc003970 2023-09-23T03:27:55.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.958+0000 7f57f6a4c640 1 -- 172.21.15.154:0/3694740243 reap_dead start 2023-09-23T03:27:55.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.959+0000 7f57f624b640 1 --2- 172.21.15.154:0/3694740243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57dc003970 0x7f57dc003d90 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f57ec00c210 tx=0x7f57dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b0cc43fba1e8c6f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:55.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.959+0000 7f57f5249640 1 -- 172.21.15.154:0/3694740243 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec005ad0 con 0x7f57dc003970 2023-09-23T03:27:55.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.959+0000 7f57f5249640 1 -- 172.21.15.154:0/3694740243 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ec005c30 con 0x7f57dc003970 2023-09-23T03:27:55.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.959+0000 7f57f5249640 1 -- 172.21.15.154:0/3694740243 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec02cad0 con 0x7f57dc003970 2023-09-23T03:27:55.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.960+0000 7f57f724d640 1 -- 172.21.15.154:0/3694740243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57dc003970 msgr2=0x7f57dc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:55.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.960+0000 7f57f724d640 1 --2- 172.21.15.154:0/3694740243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57dc003970 0x7f57dc003d90 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f57ec00c210 tx=0x7f57dc0057f0 comp rx=0 tx=0).stop 2023-09-23T03:27:55.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.960+0000 7f57f724d640 1 -- 172.21.15.154:0/3694740243 shutdown_connections 2023-09-23T03:27:55.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.960+0000 7f57f724d640 1 --2- 172.21.15.154:0/3694740243 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57dc003970 0x7f57dc003d90 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:55.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.960+0000 7f57f724d640 1 -- 172.21.15.154:0/3694740243 >> 172.21.15.154:0/3694740243 conn(0x7f57f006a4c0 msgr2=0x7f57f00afe50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:55.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.960+0000 7f57f724d640 1 -- 172.21.15.154:0/3694740243 shutdown_connections 2023-09-23T03:27:55.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.960+0000 7f57f724d640 1 -- 172.21.15.154:0/3694740243 wait complete. 2023-09-23T03:27:55.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.962+0000 7f57f724d640 1 Processor -- start 2023-09-23T03:27:55.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.962+0000 7f57f724d640 1 -- start start 2023-09-23T03:27:55.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.963+0000 7f57f724d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57dc003970 0x7f57f0156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:55.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.963+0000 7f57f724d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57f00b0f60 0x7f57f00c6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:55.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.963+0000 7f57f724d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57f00ad870 0x7f57f00adcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:55.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.963+0000 7f57f724d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f015dbd0 con 0x7f57dc003970 2023-09-23T03:27:55.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.963+0000 7f57f724d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f00ae230 con 0x7f57f00b0f60 2023-09-23T03:27:55.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.963+0000 7f57f724d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f00b2130 con 0x7f57f00ad870 2023-09-23T03:27:55.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.963+0000 7f57f624b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57dc003970 0x7f57f0156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:55.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.963+0000 7f57f624b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57dc003970 0x7f57f0156410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49742/0 (socket says 172.21.15.154:49742) 2023-09-23T03:27:55.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.963+0000 7f57f624b640 1 -- 172.21.15.154:0/3638111710 learned_addr learned my addr 172.21.15.154:0/3638111710 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:55.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.963+0000 7f57f5a4a640 1 --2- 172.21.15.154:0/3638111710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57f00b0f60 0x7f57f00c6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:55.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.963+0000 7f57f624b640 1 -- 172.21.15.154:0/3638111710 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57f00ad870 msgr2=0x7f57f00adcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:55.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.963+0000 7f57f6a4c640 1 --2- 172.21.15.154:0/3638111710 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57f00ad870 0x7f57f00adcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:55.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.963+0000 7f57f624b640 1 --2- 172.21.15.154:0/3638111710 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57f00ad870 0x7f57f00adcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:55.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.963+0000 7f57f624b640 1 -- 172.21.15.154:0/3638111710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57f00b0f60 msgr2=0x7f57f00c6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:55.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.963+0000 7f57f624b640 1 --2- 172.21.15.154:0/3638111710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57f00b0f60 0x7f57f00c6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:55.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.963+0000 7f57f624b640 1 -- 172.21.15.154:0/3638111710 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f0156950 con 0x7f57dc003970 2023-09-23T03:27:55.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.963+0000 7f57f5a4a640 1 --2- 172.21.15.154:0/3638111710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57f00b0f60 0x7f57f00c6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:55.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.964+0000 7f57f624b640 1 --2- 172.21.15.154:0/3638111710 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57dc003970 0x7f57f0156410 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f57ec02c430 tx=0x7f57ec02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:55.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.964+0000 7f57e6ffd640 1 -- 172.21.15.154:0/3638111710 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec03c040 con 0x7f57dc003970 2023-09-23T03:27:55.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.964+0000 7f57e6ffd640 1 -- 172.21.15.154:0/3638111710 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ec02cca0 con 0x7f57dc003970 2023-09-23T03:27:55.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.964+0000 7f57f724d640 1 -- 172.21.15.154:0/3638111710 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f00b1520 con 0x7f57dc003970 2023-09-23T03:27:55.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.964+0000 7f57e6ffd640 1 -- 172.21.15.154:0/3638111710 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec007290 con 0x7f57dc003970 2023-09-23T03:27:55.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.964+0000 7f57f724d640 1 -- 172.21.15.154:0/3638111710 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f00b1ae0 con 0x7f57dc003970 2023-09-23T03:27:55.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.966+0000 7f57f724d640 1 -- 172.21.15.154:0/3638111710 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57f0066730 con 0x7f57dc003970 2023-09-23T03:27:55.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.967+0000 7f57e6ffd640 1 -- 172.21.15.154:0/3638111710 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f57ec032070 con 0x7f57dc003970 2023-09-23T03:27:55.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.968+0000 7f57e6ffd640 1 --2- 172.21.15.154:0/3638111710 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f57d406af30 0x7f57d406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:55.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.968+0000 7f57e6ffd640 1 -- 172.21.15.154:0/3638111710 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f57ec0aad00 con 0x7f57dc003970 2023-09-23T03:27:55.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.968+0000 7f57f5a4a640 1 --2- 172.21.15.154:0/3638111710 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f57d406af30 0x7f57d406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:55.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.969+0000 7f57f5a4a640 1 --2- 172.21.15.154:0/3638111710 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f57d406af30 0x7f57d406d3f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f57f00c7a40 tx=0x7f57e0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:55.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:55.973+0000 7f57e6ffd640 1 -- 172.21.15.154:0/3638111710 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f57ec075ca0 con 0x7f57dc003970 2023-09-23T03:27:56.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.200+0000 7f57f724d640 1 -- 172.21.15.154:0/3638111710 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57f004e920 con 0x7f57dc003970 2023-09-23T03:27:56.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.201+0000 7f57e6ffd640 1 -- 172.21.15.154:0/3638111710 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) v1 ==== 95+0+4933 (secure 0 0 0) 0x7f57ec046020 con 0x7f57dc003970 2023-09-23T03:27:56.202 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:56.202 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":108,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/2759187683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2759187683},{"type":"v1","addr":"172.21.15.154:6835","nonce":2759187683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":6544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/4147378242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":4147378242},{"type":"v1","addr":"172.21.15.190:6834","nonce":4147378242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103},{"gid":8907,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/4006549651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4006549651},{"type":"v1","addr":"172.21.15.154:6837","nonce":4006549651}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:27:44.470312+0000","modified":"2023-09-23T03:27:47.518558+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8919},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8919":{"gid":8919,"name":"b","rank":0,"incarnation":104,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/3558096060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3558096060},{"type":"v1","addr":"172.21.15.190:6835","nonce":3558096060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-09-23T03:27:56.202 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 108 2023-09-23T03:27:56.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.203+0000 7f57f724d640 1 -- 172.21.15.154:0/3638111710 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f57d406af30 msgr2=0x7f57d406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:56.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.203+0000 7f57f724d640 1 --2- 172.21.15.154:0/3638111710 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f57d406af30 0x7f57d406d3f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f57f00c7a40 tx=0x7f57e0008040 comp rx=0 tx=0).stop 2023-09-23T03:27:56.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.203+0000 7f57f724d640 1 -- 172.21.15.154:0/3638111710 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57dc003970 msgr2=0x7f57f0156410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:56.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.203+0000 7f57f724d640 1 --2- 172.21.15.154:0/3638111710 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57dc003970 0x7f57f0156410 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f57ec02c430 tx=0x7f57ec02e070 comp rx=0 tx=0).stop 2023-09-23T03:27:56.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.203+0000 7f57f724d640 1 -- 172.21.15.154:0/3638111710 shutdown_connections 2023-09-23T03:27:56.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.203+0000 7f57f724d640 1 --2- 172.21.15.154:0/3638111710 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f57d406af30 0x7f57d406d3f0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:56.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.203+0000 7f57f724d640 1 --2- 172.21.15.154:0/3638111710 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f57f00ad870 0x7f57f00adcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:56.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.203+0000 7f57f724d640 1 --2- 172.21.15.154:0/3638111710 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f57f00b0f60 0x7f57f00c6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:56.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.203+0000 7f57f724d640 1 --2- 172.21.15.154:0/3638111710 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57dc003970 0x7f57f0156410 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:56.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.203+0000 7f57f724d640 1 -- 172.21.15.154:0/3638111710 >> 172.21.15.154:0/3638111710 conn(0x7f57f006a4c0 msgr2=0x7f57f0158000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:56.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.204+0000 7f57f724d640 1 -- 172.21.15.154:0/3638111710 shutdown_connections 2023-09-23T03:27:56.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.204+0000 7f57f724d640 1 -- 172.21.15.154:0/3638111710 wait complete. 2023-09-23T03:27:56.223 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:27:56.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.434+0000 7fcfd2e1a640 1 Processor -- start 2023-09-23T03:27:56.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.434+0000 7fcfd2e1a640 1 -- start start 2023-09-23T03:27:56.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.435+0000 7fcfd2e1a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcfcc0ad9c0 0x7fcfcc0c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:56.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.435+0000 7fcfd2e1a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcfcc0b0780 0x7fcfcc06c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:56.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.435+0000 7fcfd2e1a640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfcc070b40 con 0x7fcfcc15ad70 2023-09-23T03:27:56.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.435+0000 7fcfd2e1a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfcc071100 con 0x7fcfcc0ad9c0 2023-09-23T03:27:56.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.435+0000 7fcfd2e1a640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfcc0c9390 con 0x7fcfcc0b0780 2023-09-23T03:27:56.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.435+0000 7fcfcbfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcfcc0ad9c0 0x7fcfcc0c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:56.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.435+0000 7fcfcbfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcfcc0ad9c0 0x7fcfcc0c8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49756/0 (socket says 172.21.15.154:49756) 2023-09-23T03:27:56.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.435+0000 7fcfcbfff640 1 -- 172.21.15.154:0/2967482482 learned_addr learned my addr 172.21.15.154:0/2967482482 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:56.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.436+0000 7fcfd13b7640 1 --2- 172.21.15.154:0/2967482482 >> v2:172.21.15.190:3300/0 conn(0x7fcfcc0b0780 0x7fcfcc06c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:56.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.436+0000 7fcfcbfff640 1 -- 172.21.15.154:0/2967482482 >> v2:172.21.15.190:3300/0 conn(0x7fcfcc0b0780 msgr2=0x7fcfcc06c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:56.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.436+0000 7fcfcbfff640 1 --2- 172.21.15.154:0/2967482482 >> v2:172.21.15.190:3300/0 conn(0x7fcfcc0b0780 0x7fcfcc06c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:56.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.436+0000 7fcfcbfff640 1 -- 172.21.15.154:0/2967482482 >> v1:172.21.15.190:6789/0 conn(0x7fcfcc15ad70 legacy=0x7fcfcc0b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:56.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.436+0000 7fcfcbfff640 1 -- 172.21.15.154:0/2967482482 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfcc070e20 con 0x7fcfcc0ad9c0 2023-09-23T03:27:56.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.437+0000 7fcfcbfff640 1 --2- 172.21.15.154:0/2967482482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcfcc0ad9c0 0x7fcfcc0c8e50 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8009870 tx=0x7fcfb802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df17e2bdb5faf25 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:56.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.437+0000 7fcfcb7fe640 1 -- 172.21.15.154:0/2967482482 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8031870 con 0x7fcfcc0ad9c0 2023-09-23T03:27:56.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.437+0000 7fcfcb7fe640 1 -- 172.21.15.154:0/2967482482 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb80319d0 con 0x7fcfcc0ad9c0 2023-09-23T03:27:56.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.437+0000 7fcfcb7fe640 1 -- 172.21.15.154:0/2967482482 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8031d60 con 0x7fcfcc0ad9c0 2023-09-23T03:27:56.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.438+0000 7fcfd2e1a640 1 -- 172.21.15.154:0/2967482482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcfcc0ad9c0 msgr2=0x7fcfcc0c8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:56.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.438+0000 7fcfd2e1a640 1 --2- 172.21.15.154:0/2967482482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcfcc0ad9c0 0x7fcfcc0c8e50 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8009870 tx=0x7fcfb802b910 comp rx=0 tx=0).stop 2023-09-23T03:27:56.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.438+0000 7fcfd2e1a640 1 -- 172.21.15.154:0/2967482482 shutdown_connections 2023-09-23T03:27:56.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.438+0000 7fcfd2e1a640 1 --2- 172.21.15.154:0/2967482482 >> v2:172.21.15.190:3300/0 conn(0x7fcfcc0b0780 0x7fcfcc06c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:56.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.438+0000 7fcfd2e1a640 1 --2- 172.21.15.154:0/2967482482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcfcc0ad9c0 0x7fcfcc0c8e50 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:56.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.438+0000 7fcfd2e1a640 1 -- 172.21.15.154:0/2967482482 >> 172.21.15.154:0/2967482482 conn(0x7fcfcc0cb070 msgr2=0x7fcfcc0cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:56.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.438+0000 7fcfd2e1a640 1 -- 172.21.15.154:0/2967482482 shutdown_connections 2023-09-23T03:27:56.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.438+0000 7fcfd2e1a640 1 -- 172.21.15.154:0/2967482482 wait complete. 2023-09-23T03:27:56.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.440+0000 7fcfd2e1a640 1 Processor -- start 2023-09-23T03:27:56.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.440+0000 7fcfd2e1a640 1 -- start start 2023-09-23T03:27:56.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.441+0000 7fcfd2e1a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfcc0ad9c0 0x7fcfcc0c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:56.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.441+0000 7fcfd2e1a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfcc0b0780 0x7fcfcc0cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:56.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.441+0000 7fcfd2e1a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcfcc15ad70 0x7fcfcc1f1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:56.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.441+0000 7fcfd2e1a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc147210 con 0x7fcfcc15ad70 2023-09-23T03:27:56.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.441+0000 7fcfd2e1a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc0c7670 con 0x7fcfcc0b0780 2023-09-23T03:27:56.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.441+0000 7fcfd2e1a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc146f30 con 0x7fcfcc0ad9c0 2023-09-23T03:27:56.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.441+0000 7fcfd13b7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcfcc15ad70 0x7fcfcc1f1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:56.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.442+0000 7fcfd13b7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcfcc15ad70 0x7fcfcc1f1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49764/0 (socket says 172.21.15.154:49764) 2023-09-23T03:27:56.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.442+0000 7fcfd13b7640 1 -- 172.21.15.154:0/3453300490 learned_addr learned my addr 172.21.15.154:0/3453300490 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:56.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.442+0000 7fcfcbfff640 1 --2- 172.21.15.154:0/3453300490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfcc0b0780 0x7fcfcc0cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:56.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.442+0000 7fcfd0bb6640 1 --2- 172.21.15.154:0/3453300490 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfcc0ad9c0 0x7fcfcc0c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:56.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.442+0000 7fcfd13b7640 1 -- 172.21.15.154:0/3453300490 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfcc0ad9c0 msgr2=0x7fcfcc0c83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:56.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.442+0000 7fcfd13b7640 1 --2- 172.21.15.154:0/3453300490 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfcc0ad9c0 0x7fcfcc0c83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:56.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.442+0000 7fcfd13b7640 1 -- 172.21.15.154:0/3453300490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfcc0b0780 msgr2=0x7fcfcc0cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:56.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.442+0000 7fcfd13b7640 1 --2- 172.21.15.154:0/3453300490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfcc0b0780 0x7fcfcc0cd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:56.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.442+0000 7fcfd13b7640 1 -- 172.21.15.154:0/3453300490 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb80094d0 con 0x7fcfcc15ad70 2023-09-23T03:27:56.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.442+0000 7fcfd13b7640 1 --2- 172.21.15.154:0/3453300490 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcfcc15ad70 0x7fcfcc1f1870 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4007e30 tx=0x7fcfc400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:56.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.443+0000 7fcfc97fa640 1 -- 172.21.15.154:0/3453300490 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4014420 con 0x7fcfcc15ad70 2023-09-23T03:27:56.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.443+0000 7fcfc97fa640 1 -- 172.21.15.154:0/3453300490 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc40145a0 con 0x7fcfcc15ad70 2023-09-23T03:27:56.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.443+0000 7fcfc97fa640 1 -- 172.21.15.154:0/3453300490 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4014950 con 0x7fcfcc15ad70 2023-09-23T03:27:56.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.443+0000 7fcfd2e1a640 1 -- 172.21.15.154:0/3453300490 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfcc0c7910 con 0x7fcfcc15ad70 2023-09-23T03:27:56.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.443+0000 7fcfd2e1a640 1 -- 172.21.15.154:0/3453300490 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfcc0c7ed0 con 0x7fcfcc15ad70 2023-09-23T03:27:56.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.444+0000 7fcfd2e1a640 1 -- 172.21.15.154:0/3453300490 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfcc067870 con 0x7fcfcc15ad70 2023-09-23T03:27:56.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.445+0000 7fcfc97fa640 1 -- 172.21.15.154:0/3453300490 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fcfc4014ab0 con 0x7fcfcc15ad70 2023-09-23T03:27:56.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.446+0000 7fcfc97fa640 1 --2- 172.21.15.154:0/3453300490 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcfb006af30 0x7fcfb006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:56.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.446+0000 7fcfc97fa640 1 -- 172.21.15.154:0/3453300490 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fcfc408a3b0 con 0x7fcfcc15ad70 2023-09-23T03:27:56.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.446+0000 7fcfd0bb6640 1 --2- 172.21.15.154:0/3453300490 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcfb006af30 0x7fcfb006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:56.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.447+0000 7fcfd0bb6640 1 --2- 172.21.15.154:0/3453300490 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcfb006af30 0x7fcfb006d3f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0006fd0 tx=0x7fcfc0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:56.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.450+0000 7fcfc97fa640 1 -- 172.21.15.154:0/3453300490 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fcfc4055180 con 0x7fcfcc15ad70 2023-09-23T03:27:56.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.678+0000 7fcfd2e1a640 1 -- 172.21.15.154:0/3453300490 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcfcc073eb0 con 0x7fcfcc15ad70 2023-09-23T03:27:56.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.679+0000 7fcfc97fa640 1 -- 172.21.15.154:0/3453300490 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) v1 ==== 95+0+4933 (secure 0 0 0) 0x7fcfc405c360 con 0x7fcfcc15ad70 2023-09-23T03:27:56.681 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:56.681 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":108,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/2759187683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2759187683},{"type":"v1","addr":"172.21.15.154:6835","nonce":2759187683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":6544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/4147378242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":4147378242},{"type":"v1","addr":"172.21.15.190:6834","nonce":4147378242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103},{"gid":8907,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/4006549651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4006549651},{"type":"v1","addr":"172.21.15.154:6837","nonce":4006549651}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:27:44.470312+0000","modified":"2023-09-23T03:27:47.518558+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8919},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8919":{"gid":8919,"name":"b","rank":0,"incarnation":104,"state":"up:active","state_seq":3,"addr":"172.21.15.190:6835/3558096060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3558096060},{"type":"v1","addr":"172.21.15.190:6835","nonce":3558096060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-09-23T03:27:56.681 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 108 2023-09-23T03:27:56.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.682+0000 7fcfd2e1a640 1 -- 172.21.15.154:0/3453300490 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcfb006af30 msgr2=0x7fcfb006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:56.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.682+0000 7fcfd2e1a640 1 --2- 172.21.15.154:0/3453300490 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcfb006af30 0x7fcfb006d3f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0006fd0 tx=0x7fcfc0008040 comp rx=0 tx=0).stop 2023-09-23T03:27:56.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.682+0000 7fcfd2e1a640 1 -- 172.21.15.154:0/3453300490 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcfcc15ad70 msgr2=0x7fcfcc1f1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:56.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.682+0000 7fcfd2e1a640 1 --2- 172.21.15.154:0/3453300490 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcfcc15ad70 0x7fcfcc1f1870 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4007e30 tx=0x7fcfc400e4b0 comp rx=0 tx=0).stop 2023-09-23T03:27:56.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.682+0000 7fcfd2e1a640 1 -- 172.21.15.154:0/3453300490 shutdown_connections 2023-09-23T03:27:56.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.682+0000 7fcfd2e1a640 1 --2- 172.21.15.154:0/3453300490 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcfb006af30 0x7fcfb006d3f0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:56.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.682+0000 7fcfd2e1a640 1 --2- 172.21.15.154:0/3453300490 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcfcc15ad70 0x7fcfcc1f1870 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:56.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.682+0000 7fcfd2e1a640 1 --2- 172.21.15.154:0/3453300490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcfcc0b0780 0x7fcfcc0cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:56.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.682+0000 7fcfd2e1a640 1 --2- 172.21.15.154:0/3453300490 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcfcc0ad9c0 0x7fcfcc0c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:56.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.682+0000 7fcfd2e1a640 1 -- 172.21.15.154:0/3453300490 >> 172.21.15.154:0/3453300490 conn(0x7fcfcc0cb070 msgr2=0x7fcfcc0afff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:56.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.682+0000 7fcfd2e1a640 1 -- 172.21.15.154:0/3453300490 shutdown_connections 2023-09-23T03:27:56.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:56.683+0000 7fcfd2e1a640 1 -- 172.21.15.154:0/3453300490 wait complete. 2023-09-23T03:27:56.703 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json dirfrag split /subdir 0/0 1 2023-09-23T03:27:56.943 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2023-09-23T03:27:56.943 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:27:56.943 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:27:57.003 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:27:57.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.219+0000 7fee82a4f640 1 Processor -- start 2023-09-23T03:27:57.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.220+0000 7fee82a4f640 1 -- start start 2023-09-23T03:27:57.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.221+0000 7fee82a4f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee7c06a6b0 0x7fee7c15ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:57.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.221+0000 7fee82a4f640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fee7c0cb980 0x7fee7c0cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:57.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.221+0000 7fee82a4f640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee7c15b4a0 con 0x7fee7c0b0640 2023-09-23T03:27:57.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.221+0000 7fee82a4f640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee7c15b780 con 0x7fee7c0cb980 2023-09-23T03:27:57.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.221+0000 7fee82a4f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c15ba60 con 0x7fee7c06a6b0 2023-09-23T03:27:57.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.222+0000 7fee80fec640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fee7c0cb980 0x7fee7c0cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:57.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.222+0000 7fee73fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee7c06a6b0 0x7fee7c15ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:57.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.222+0000 7fee80fec640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fee7c0cb980 0x7fee7c0cbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.154:57950/0 (socket says 172.21.15.154:57950) 2023-09-23T03:27:57.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.222+0000 7fee80fec640 1 -- 172.21.15.154:0/3739550869 learned_addr learned my addr 172.21.15.154:0/3739550869 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:57.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.223+0000 7fee80fec640 1 -- 172.21.15.154:0/3739550869 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee7c06a6b0 msgr2=0x7fee7c15ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:57.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.223+0000 7fee80fec640 1 --2- 172.21.15.154:0/3739550869 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee7c06a6b0 0x7fee7c15ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:57.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.223+0000 7fee80fec640 1 -- 172.21.15.154:0/3739550869 >> v1:172.21.15.190:6789/0 conn(0x7fee7c0b0640 legacy=0x7fee7c0b0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:57.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.223+0000 7fee80fec640 1 -- 172.21.15.154:0/3739550869 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee7c15bd40 con 0x7fee7c0cb980 2023-09-23T03:27:57.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.224+0000 7fee80fec640 1 --2- 172.21.15.154:0/3739550869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee7c0cb980 0x7fee7c0cbe10 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fee74009e50 tx=0x7fee7402bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae28dec7c1d33c00 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:57.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.224+0000 7fee7b7fe640 1 -- 172.21.15.154:0/3739550869 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7402cb00 con 0x7fee7c0cb980 2023-09-23T03:27:57.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.225+0000 7fee7b7fe640 1 -- 172.21.15.154:0/3739550869 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee7402cc60 con 0x7fee7c0cb980 2023-09-23T03:27:57.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.225+0000 7fee7b7fe640 1 -- 172.21.15.154:0/3739550869 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee740343f0 con 0x7fee7c0cb980 2023-09-23T03:27:57.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.225+0000 7fee82a4f640 1 -- 172.21.15.154:0/3739550869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee7c0cb980 msgr2=0x7fee7c0cbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:57.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.225+0000 7fee82a4f640 1 --2- 172.21.15.154:0/3739550869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee7c0cb980 0x7fee7c0cbe10 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fee74009e50 tx=0x7fee7402bf60 comp rx=0 tx=0).stop 2023-09-23T03:27:57.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.225+0000 7fee82a4f640 1 -- 172.21.15.154:0/3739550869 shutdown_connections 2023-09-23T03:27:57.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.225+0000 7fee82a4f640 1 --2- 172.21.15.154:0/3739550869 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee7c0cb980 0x7fee7c0cbe10 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:57.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.225+0000 7fee82a4f640 1 --2- 172.21.15.154:0/3739550869 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee7c06a6b0 0x7fee7c15ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:57.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.225+0000 7fee82a4f640 1 -- 172.21.15.154:0/3739550869 >> 172.21.15.154:0/3739550869 conn(0x7fee7c098a90 msgr2=0x7fee7c0cf700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:57.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.225+0000 7fee82a4f640 1 -- 172.21.15.154:0/3739550869 shutdown_connections 2023-09-23T03:27:57.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.225+0000 7fee82a4f640 1 -- 172.21.15.154:0/3739550869 wait complete. 2023-09-23T03:27:57.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.227+0000 7fee82a4f640 1 Processor -- start 2023-09-23T03:27:57.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.227+0000 7fee82a4f640 1 -- start start 2023-09-23T03:27:57.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.228+0000 7fee82a4f640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee7c06a6b0 0x7fee7c1f50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:57.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.228+0000 7fee82a4f640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee7c0b0640 0x7fee7c1f32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:57.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.228+0000 7fee82a4f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee7c0cb980 0x7fee7c0c7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:57.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.228+0000 7fee82a4f640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c0c7450 con 0x7fee7c0cb980 2023-09-23T03:27:57.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.228+0000 7fee82a4f640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c0c90c0 con 0x7fee7c0b0640 2023-09-23T03:27:57.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.228+0000 7fee82a4f640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c0c67d0 con 0x7fee7c06a6b0 2023-09-23T03:27:57.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.229+0000 7fee80fec640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee7c0cb980 0x7fee7c0c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:57.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.229+0000 7fee80fec640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee7c0cb980 0x7fee7c0c7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49780/0 (socket says 172.21.15.154:49780) 2023-09-23T03:27:57.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.229+0000 7fee80fec640 1 -- 172.21.15.154:0/2556834661 learned_addr learned my addr 172.21.15.154:0/2556834661 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:57.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.229+0000 7fee7bfff640 1 --2- 172.21.15.154:0/2556834661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee7c06a6b0 0x7fee7c1f50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:57.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.229+0000 7fee73fff640 1 --2- 172.21.15.154:0/2556834661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee7c0b0640 0x7fee7c1f32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:57.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.229+0000 7fee80fec640 1 -- 172.21.15.154:0/2556834661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee7c06a6b0 msgr2=0x7fee7c1f50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:57.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.229+0000 7fee80fec640 1 --2- 172.21.15.154:0/2556834661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee7c06a6b0 0x7fee7c1f50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:57.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.229+0000 7fee80fec640 1 -- 172.21.15.154:0/2556834661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee7c0b0640 msgr2=0x7fee7c1f32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:57.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.229+0000 7fee80fec640 1 --2- 172.21.15.154:0/2556834661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee7c0b0640 0x7fee7c1f32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:57.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.229+0000 7fee80fec640 1 -- 172.21.15.154:0/2556834661 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee74009b30 con 0x7fee7c0cb980 2023-09-23T03:27:57.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.229+0000 7fee73fff640 1 --2- 172.21.15.154:0/2556834661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee7c0b0640 0x7fee7c1f32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:57.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.230+0000 7fee80fec640 1 --2- 172.21.15.154:0/2556834661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee7c0cb980 0x7fee7c0c7870 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fee74036040 tx=0x7fee74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:57.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.230+0000 7fee797fa640 1 -- 172.21.15.154:0/2556834661 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee74009510 con 0x7fee7c0cb980 2023-09-23T03:27:57.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.230+0000 7fee797fa640 1 -- 172.21.15.154:0/2556834661 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee7403e450 con 0x7fee7c0cb980 2023-09-23T03:27:57.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.230+0000 7fee82a4f640 1 -- 172.21.15.154:0/2556834661 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee7c0c6ab0 con 0x7fee7c0cb980 2023-09-23T03:27:57.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.230+0000 7fee797fa640 1 -- 172.21.15.154:0/2556834661 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7403e7e0 con 0x7fee7c0cb980 2023-09-23T03:27:57.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.230+0000 7fee82a4f640 1 -- 172.21.15.154:0/2556834661 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee7c0c7070 con 0x7fee7c0cb980 2023-09-23T03:27:57.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.231+0000 7fee82a4f640 1 -- 172.21.15.154:0/2556834661 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee7c067870 con 0x7fee7c0cb980 2023-09-23T03:27:57.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.233+0000 7fee797fa640 1 -- 172.21.15.154:0/2556834661 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fee7403e940 con 0x7fee7c0cb980 2023-09-23T03:27:57.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.233+0000 7fee797fa640 1 --2- 172.21.15.154:0/2556834661 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fee5c06af30 0x7fee5c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:57.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.234+0000 7fee797fa640 1 -- 172.21.15.154:0/2556834661 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fee740ac620 con 0x7fee7c0cb980 2023-09-23T03:27:57.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.234+0000 7fee7bfff640 1 --2- 172.21.15.154:0/2556834661 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fee5c06af30 0x7fee5c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:57.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.234+0000 7fee7bfff640 1 --2- 172.21.15.154:0/2556834661 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fee5c06af30 0x7fee5c06d3f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fee6c000fb0 tx=0x7fee6c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:57.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.238+0000 7fee797fa640 1 -- 172.21.15.154:0/2556834661 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fee74077510 con 0x7fee7c0cb980 2023-09-23T03:27:57.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.428+0000 7fee82a4f640 1 -- 172.21.15.154:0/2556834661 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee7c0c7590 con 0x7fee7c0cb980 2023-09-23T03:27:57.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.430+0000 7fee797fa640 1 -- 172.21.15.154:0/2556834661 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+16820 (secure 0 0 0) 0x7fee7c0c7590 con 0x7fee7c0cb980 2023-09-23T03:27:57.432 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:57.432 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":126,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:27:46.016186+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:27:42.207617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-09-23T03:27:43.473867+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:27:57.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.432+0000 7fee82a4f640 1 -- 172.21.15.154:0/2556834661 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fee5c06af30 msgr2=0x7fee5c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:57.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.432+0000 7fee82a4f640 1 --2- 172.21.15.154:0/2556834661 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fee5c06af30 0x7fee5c06d3f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fee6c000fb0 tx=0x7fee6c009040 comp rx=0 tx=0).stop 2023-09-23T03:27:57.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.432+0000 7fee82a4f640 1 -- 172.21.15.154:0/2556834661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee7c0cb980 msgr2=0x7fee7c0c7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:57.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.432+0000 7fee82a4f640 1 --2- 172.21.15.154:0/2556834661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee7c0cb980 0x7fee7c0c7870 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fee74036040 tx=0x7fee74004a40 comp rx=0 tx=0).stop 2023-09-23T03:27:57.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.432+0000 7fee82a4f640 1 -- 172.21.15.154:0/2556834661 shutdown_connections 2023-09-23T03:27:57.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.432+0000 7fee82a4f640 1 --2- 172.21.15.154:0/2556834661 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fee5c06af30 0x7fee5c06d3f0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:57.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.432+0000 7fee82a4f640 1 --2- 172.21.15.154:0/2556834661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee7c0cb980 0x7fee7c0c7870 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:57.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.432+0000 7fee82a4f640 1 --2- 172.21.15.154:0/2556834661 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fee7c0b0640 0x7fee7c1f32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:57.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.432+0000 7fee82a4f640 1 --2- 172.21.15.154:0/2556834661 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fee7c06a6b0 0x7fee7c1f50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:57.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.432+0000 7fee82a4f640 1 -- 172.21.15.154:0/2556834661 >> 172.21.15.154:0/2556834661 conn(0x7fee7c098a90 msgr2=0x7fee7c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:57.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.433+0000 7fee82a4f640 1 -- 172.21.15.154:0/2556834661 shutdown_connections 2023-09-23T03:27:57.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.433+0000 7fee82a4f640 1 -- 172.21.15.154:0/2556834661 wait complete. 2023-09-23T03:27:57.452 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:27:42.207617+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:27:43.473867+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T03:27:57.453 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:27:57.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.669+0000 7ff6cc2af640 1 Processor -- start 2023-09-23T03:27:57.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.669+0000 7ff6cc2af640 1 -- start start 2023-09-23T03:27:57.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.671+0000 7ff6cc2af640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6c4161190 0x7ff6c40b0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:57.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.671+0000 7ff6cc2af640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff6c40c18d0 0x7ff6c40b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:57.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.671+0000 7ff6cc2af640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff6c4165720 0x7ff6c4070130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:57.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.671+0000 7ff6cc2af640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6c4165c20 con 0x7ff6c40c18d0 2023-09-23T03:27:57.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.671+0000 7ff6cc2af640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6c40cd3a0 con 0x7ff6c4165720 2023-09-23T03:27:57.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.671+0000 7ff6cc2af640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c40ada80 con 0x7ff6c4161190 2023-09-23T03:27:57.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.671+0000 7ff6c984a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff6c40c18d0 0x7ff6c40b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:57.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.671+0000 7ff6c984a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7ff6c40c18d0 0x7ff6c40b05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49788/0 (socket says 172.21.15.154:49788) 2023-09-23T03:27:57.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.671+0000 7ff6c984a640 1 -- 172.21.15.154:0/284147361 learned_addr learned my addr 172.21.15.154:0/284147361 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:57.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.671+0000 7ff6ca04b640 1 --2- 172.21.15.154:0/284147361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6c4161190 0x7ff6c40b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:57.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.671+0000 7ff6ca84c640 1 --2- 172.21.15.154:0/284147361 >> v2:172.21.15.190:3300/0 conn(0x7ff6c4165720 0x7ff6c4070130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:57.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.671+0000 7ff6c984a640 1 -- 172.21.15.154:0/284147361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6c4161190 msgr2=0x7ff6c40b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:57.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.672+0000 7ff6c984a640 1 --2- 172.21.15.154:0/284147361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6c4161190 0x7ff6c40b0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:57.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.672+0000 7ff6c984a640 1 -- 172.21.15.154:0/284147361 >> v2:172.21.15.190:3300/0 conn(0x7ff6c4165720 msgr2=0x7ff6c4070130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:57.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.672+0000 7ff6c984a640 1 --2- 172.21.15.154:0/284147361 >> v2:172.21.15.190:3300/0 conn(0x7ff6c4165720 0x7ff6c4070130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:57.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.672+0000 7ff6c984a640 1 -- 172.21.15.154:0/284147361 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c40add60 con 0x7ff6c40c18d0 2023-09-23T03:27:57.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.672+0000 7ff6ca84c640 1 --2- 172.21.15.154:0/284147361 >> v2:172.21.15.190:3300/0 conn(0x7ff6c4165720 0x7ff6c4070130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:57.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.673+0000 7ff6c984a640 1 --2- 172.21.15.154:0/284147361 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6c40c18d0 0x7ff6c40b05b0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7ff6c406e140 tx=0x7ff6b402b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5286e4f562292ab7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:57.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.673+0000 7ff6c9049640 1 -- 172.21.15.154:0/284147361 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4031740 con 0x7ff6c40c18d0 2023-09-23T03:27:57.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.673+0000 7ff6c9049640 1 -- 172.21.15.154:0/284147361 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b40318a0 con 0x7ff6c40c18d0 2023-09-23T03:27:57.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.673+0000 7ff6c9049640 1 -- 172.21.15.154:0/284147361 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4031c30 con 0x7ff6c40c18d0 2023-09-23T03:27:57.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.674+0000 7ff6cc2af640 1 -- 172.21.15.154:0/284147361 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6c40c18d0 msgr2=0x7ff6c40b05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:57.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.674+0000 7ff6cc2af640 1 --2- 172.21.15.154:0/284147361 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6c40c18d0 0x7ff6c40b05b0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7ff6c406e140 tx=0x7ff6b402b7e0 comp rx=0 tx=0).stop 2023-09-23T03:27:57.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.674+0000 7ff6cc2af640 1 -- 172.21.15.154:0/284147361 shutdown_connections 2023-09-23T03:27:57.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.674+0000 7ff6cc2af640 1 --2- 172.21.15.154:0/284147361 >> v2:172.21.15.190:3300/0 conn(0x7ff6c4165720 0x7ff6c4070130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:57.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.674+0000 7ff6cc2af640 1 --2- 172.21.15.154:0/284147361 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6c40c18d0 0x7ff6c40b05b0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:57.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.674+0000 7ff6cc2af640 1 --2- 172.21.15.154:0/284147361 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6c4161190 0x7ff6c40b0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:57.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.674+0000 7ff6cc2af640 1 -- 172.21.15.154:0/284147361 >> 172.21.15.154:0/284147361 conn(0x7ff6c40cb070 msgr2=0x7ff6c40cb480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:57.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.674+0000 7ff6cc2af640 1 -- 172.21.15.154:0/284147361 shutdown_connections 2023-09-23T03:27:57.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.674+0000 7ff6cc2af640 1 -- 172.21.15.154:0/284147361 wait complete. 2023-09-23T03:27:57.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.676+0000 7ff6cc2af640 1 Processor -- start 2023-09-23T03:27:57.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.676+0000 7ff6cc2af640 1 -- start start 2023-09-23T03:27:57.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.677+0000 7ff6cc2af640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6c40c18d0 0x7ff6c41f6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:57.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.677+0000 7ff6cc2af640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6c4161190 0x7ff6c415d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:57.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.677+0000 7ff6cc2af640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6c4165720 0x7ff6c415d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:57.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.677+0000 7ff6cc2af640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c414e740 con 0x7ff6c4165720 2023-09-23T03:27:57.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.677+0000 7ff6cc2af640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c414b390 con 0x7ff6c40c18d0 2023-09-23T03:27:57.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.677+0000 7ff6cc2af640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c40b08d0 con 0x7ff6c4161190 2023-09-23T03:27:57.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.678+0000 7ff6ca84c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6c4165720 0x7ff6c415d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:57.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.678+0000 7ff6ca84c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6c4165720 0x7ff6c415d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49792/0 (socket says 172.21.15.154:49792) 2023-09-23T03:27:57.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.678+0000 7ff6ca84c640 1 -- 172.21.15.154:0/3180953304 learned_addr learned my addr 172.21.15.154:0/3180953304 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:57.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.678+0000 7ff6c984a640 1 --2- 172.21.15.154:0/3180953304 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6c4161190 0x7ff6c415d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:57.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.678+0000 7ff6ca04b640 1 --2- 172.21.15.154:0/3180953304 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6c40c18d0 0x7ff6c41f6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:57.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.678+0000 7ff6ca84c640 1 -- 172.21.15.154:0/3180953304 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6c4161190 msgr2=0x7ff6c415d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:57.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.678+0000 7ff6ca84c640 1 --2- 172.21.15.154:0/3180953304 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6c4161190 0x7ff6c415d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:57.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.678+0000 7ff6ca84c640 1 -- 172.21.15.154:0/3180953304 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6c40c18d0 msgr2=0x7ff6c41f6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:57.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.678+0000 7ff6ca84c640 1 --2- 172.21.15.154:0/3180953304 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6c40c18d0 0x7ff6c41f6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:57.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.678+0000 7ff6ca84c640 1 -- 172.21.15.154:0/3180953304 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b40094d0 con 0x7ff6c4165720 2023-09-23T03:27:57.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.678+0000 7ff6c984a640 1 --2- 172.21.15.154:0/3180953304 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6c4161190 0x7ff6c415d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:27:57.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.678+0000 7ff6ca04b640 1 --2- 172.21.15.154:0/3180953304 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6c40c18d0 0x7ff6c41f6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:27:57.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.679+0000 7ff6ca84c640 1 --2- 172.21.15.154:0/3180953304 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6c4165720 0x7ff6c415d640 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc00ccf0 tx=0x7ff6bc00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:57.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.679+0000 7ff6baffd640 1 -- 172.21.15.154:0/3180953304 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc00f680 con 0x7ff6c4165720 2023-09-23T03:27:57.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.679+0000 7ff6baffd640 1 -- 172.21.15.154:0/3180953304 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6bc00f800 con 0x7ff6c4165720 2023-09-23T03:27:57.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.679+0000 7ff6baffd640 1 -- 172.21.15.154:0/3180953304 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc00fbb0 con 0x7ff6c4165720 2023-09-23T03:27:57.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.679+0000 7ff6cc2af640 1 -- 172.21.15.154:0/3180953304 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6c40c5880 con 0x7ff6c4165720 2023-09-23T03:27:57.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.679+0000 7ff6cc2af640 1 -- 172.21.15.154:0/3180953304 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6c41f68a0 con 0x7ff6c4165720 2023-09-23T03:27:57.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.680+0000 7ff6cc2af640 1 -- 172.21.15.154:0/3180953304 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6c4067870 con 0x7ff6c4165720 2023-09-23T03:27:57.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.681+0000 7ff6baffd640 1 -- 172.21.15.154:0/3180953304 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7ff6bc00b040 con 0x7ff6c4165720 2023-09-23T03:27:57.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.683+0000 7ff6baffd640 1 --2- 172.21.15.154:0/3180953304 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff6a806af30 0x7ff6a806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:57.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.683+0000 7ff6baffd640 1 -- 172.21.15.154:0/3180953304 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff6bc08a150 con 0x7ff6c4165720 2023-09-23T03:27:57.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.683+0000 7ff6ca04b640 1 --2- 172.21.15.154:0/3180953304 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff6a806af30 0x7ff6a806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:57.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.684+0000 7ff6ca04b640 1 --2- 172.21.15.154:0/3180953304 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff6a806af30 0x7ff6a806d3f0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0006fd0 tx=0x7ff6c0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:57.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.687+0000 7ff6baffd640 1 -- 172.21.15.154:0/3180953304 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7ff6bc055040 con 0x7ff6c4165720 2023-09-23T03:27:57.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.877+0000 7ff6cc2af640 1 -- 172.21.15.154:0/3180953304 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff6c40c5a10 con 0x7ff6c4165720 2023-09-23T03:27:57.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.878+0000 7ff6baffd640 1 -- 172.21.15.154:0/3180953304 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v126) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff6bc058660 con 0x7ff6c4165720 2023-09-23T03:27:57.879 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:27:57.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.880+0000 7ff6cc2af640 1 -- 172.21.15.154:0/3180953304 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff6a806af30 msgr2=0x7ff6a806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:57.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.880+0000 7ff6cc2af640 1 --2- 172.21.15.154:0/3180953304 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff6a806af30 0x7ff6a806d3f0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0006fd0 tx=0x7ff6c0008040 comp rx=0 tx=0).stop 2023-09-23T03:27:57.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.880+0000 7ff6cc2af640 1 -- 172.21.15.154:0/3180953304 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6c4165720 msgr2=0x7ff6c415d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:57.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.880+0000 7ff6cc2af640 1 --2- 172.21.15.154:0/3180953304 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6c4165720 0x7ff6c415d640 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc00ccf0 tx=0x7ff6bc00a650 comp rx=0 tx=0).stop 2023-09-23T03:27:57.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.880+0000 7ff6cc2af640 1 -- 172.21.15.154:0/3180953304 shutdown_connections 2023-09-23T03:27:57.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.880+0000 7ff6cc2af640 1 --2- 172.21.15.154:0/3180953304 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff6a806af30 0x7ff6a806d3f0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:57.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.880+0000 7ff6cc2af640 1 --2- 172.21.15.154:0/3180953304 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6c4165720 0x7ff6c415d640 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:57.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.880+0000 7ff6cc2af640 1 --2- 172.21.15.154:0/3180953304 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff6c4161190 0x7ff6c415d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:57.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.880+0000 7ff6cc2af640 1 --2- 172.21.15.154:0/3180953304 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff6c40c18d0 0x7ff6c41f6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:57.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.880+0000 7ff6cc2af640 1 -- 172.21.15.154:0/3180953304 >> 172.21.15.154:0/3180953304 conn(0x7ff6c40cb070 msgr2=0x7ff6c406c930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:57.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.881+0000 7ff6cc2af640 1 -- 172.21.15.154:0/3180953304 shutdown_connections 2023-09-23T03:27:57.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:57.881+0000 7ff6cc2af640 1 -- 172.21.15.154:0/3180953304 wait complete. 2023-09-23T03:27:57.901 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:27:58.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.119+0000 7f95d5861640 1 Processor -- start 2023-09-23T03:27:58.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.119+0000 7f95d5861640 1 -- start start 2023-09-23T03:27:58.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.120+0000 7f95d5861640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95d0144130 0x7f95d00b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:58.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.120+0000 7f95d5861640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f95d00caf80 0x7f95d00cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:58.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.120+0000 7f95d5861640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95d00cb920 con 0x7f95d015e520 2023-09-23T03:27:58.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.120+0000 7f95d5861640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95d0070fb0 con 0x7f95d00caf80 2023-09-23T03:27:58.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.120+0000 7f95d5861640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d00700e0 con 0x7f95d0144130 2023-09-23T03:27:58.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.121+0000 7f95cf7fe640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f95d015e520 0x7f95d015e9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.154:40318/0 (socket says 172.21.15.154:40318) 2023-09-23T03:27:58.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.121+0000 7f95ce7fc640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f95d00caf80 0x7f95d00cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:58.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.121+0000 7f95cf7fe640 1 -- 172.21.15.154:0/620204292 learned_addr learned my addr 172.21.15.154:0/620204292 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:27:58.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.121+0000 7f95ceffd640 1 --2- 172.21.15.154:0/620204292 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95d0144130 0x7f95d00b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:58.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.122+0000 7f95ce7fc640 1 -- 172.21.15.154:0/620204292 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95d0144130 msgr2=0x7f95d00b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:58.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.122+0000 7f95ce7fc640 1 --2- 172.21.15.154:0/620204292 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95d0144130 0x7f95d00b1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:58.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.122+0000 7f95ce7fc640 1 -- 172.21.15.154:0/620204292 >> v1:172.21.15.190:6789/0 conn(0x7f95d015e520 legacy=0x7f95d015e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:58.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.122+0000 7f95ce7fc640 1 -- 172.21.15.154:0/620204292 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d0070330 con 0x7f95d00caf80 2023-09-23T03:27:58.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.123+0000 7f95ce7fc640 1 --2- 172.21.15.154:0/620204292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95d00caf80 0x7f95d00cb3e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f95c40097a0 tx=0x7f95c402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bb8307e04104592 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:58.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.124+0000 7f95cdffb640 1 -- 172.21.15.154:0/620204292 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c4031830 con 0x7f95d00caf80 2023-09-23T03:27:58.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.124+0000 7f95cdffb640 1 -- 172.21.15.154:0/620204292 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c4031990 con 0x7f95d00caf80 2023-09-23T03:27:58.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.124+0000 7f95cdffb640 1 -- 172.21.15.154:0/620204292 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c4031d20 con 0x7f95d00caf80 2023-09-23T03:27:58.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.125+0000 7f95d5861640 1 -- 172.21.15.154:0/620204292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95d00caf80 msgr2=0x7f95d00cb3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:58.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.125+0000 7f95d5861640 1 --2- 172.21.15.154:0/620204292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95d00caf80 0x7f95d00cb3e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f95c40097a0 tx=0x7f95c402b8d0 comp rx=0 tx=0).stop 2023-09-23T03:27:58.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.125+0000 7f95d5861640 1 -- 172.21.15.154:0/620204292 shutdown_connections 2023-09-23T03:27:58.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.125+0000 7f95d5861640 1 --2- 172.21.15.154:0/620204292 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95d00caf80 0x7f95d00cb3e0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:58.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.125+0000 7f95d5861640 1 --2- 172.21.15.154:0/620204292 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95d0144130 0x7f95d00b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:58.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.125+0000 7f95d5861640 1 -- 172.21.15.154:0/620204292 >> 172.21.15.154:0/620204292 conn(0x7f95d006a440 msgr2=0x7f95d00b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:58.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.125+0000 7f95d5861640 1 -- 172.21.15.154:0/620204292 shutdown_connections 2023-09-23T03:27:58.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.125+0000 7f95d5861640 1 -- 172.21.15.154:0/620204292 wait complete. 2023-09-23T03:27:58.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.126+0000 7f95d5861640 1 Processor -- start 2023-09-23T03:27:58.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.127+0000 7f95d5861640 1 -- start start 2023-09-23T03:27:58.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.127+0000 7f95d5861640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95d00caf80 0x7f95d00c4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:58.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.128+0000 7f95d5861640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95d0144130 0x7f95d00c34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:58.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.128+0000 7f95d5861640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95d015e520 0x7f95d006c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:58.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.128+0000 7f95d5861640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d00c39f0 con 0x7f95d015e520 2023-09-23T03:27:58.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.128+0000 7f95d5861640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d00c3c20 con 0x7f95d00caf80 2023-09-23T03:27:58.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.128+0000 7f95d5861640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d00c3f00 con 0x7f95d0144130 2023-09-23T03:27:58.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.128+0000 7f95cf7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95d015e520 0x7f95d006c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:58.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.128+0000 7f95cf7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95d015e520 0x7f95d006c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49800/0 (socket says 172.21.15.154:49800) 2023-09-23T03:27:58.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.128+0000 7f95cf7fe640 1 -- 172.21.15.154:0/274442285 learned_addr learned my addr 172.21.15.154:0/274442285 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:58.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.128+0000 7f95ceffd640 1 --2- 172.21.15.154:0/274442285 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95d00caf80 0x7f95d00c4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:58.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.128+0000 7f95ce7fc640 1 --2- 172.21.15.154:0/274442285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95d0144130 0x7f95d00c34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:58.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.128+0000 7f95cf7fe640 1 -- 172.21.15.154:0/274442285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95d0144130 msgr2=0x7f95d00c34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:58.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.128+0000 7f95cf7fe640 1 --2- 172.21.15.154:0/274442285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95d0144130 0x7f95d00c34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:58.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.128+0000 7f95cf7fe640 1 -- 172.21.15.154:0/274442285 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95d00caf80 msgr2=0x7f95d00c4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:58.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.128+0000 7f95cf7fe640 1 --2- 172.21.15.154:0/274442285 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95d00caf80 0x7f95d00c4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:58.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.128+0000 7f95cf7fe640 1 -- 172.21.15.154:0/274442285 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c4009400 con 0x7f95d015e520 2023-09-23T03:27:58.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.129+0000 7f95cf7fe640 1 --2- 172.21.15.154:0/274442285 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95d015e520 0x7f95d006c1b0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f95c800ca80 tx=0x7f95c800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:58.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.129+0000 7f95b7fff640 1 -- 172.21.15.154:0/274442285 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c800ac30 con 0x7f95d015e520 2023-09-23T03:27:58.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.129+0000 7f95b7fff640 1 -- 172.21.15.154:0/274442285 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c800ad90 con 0x7f95d015e520 2023-09-23T03:27:58.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.129+0000 7f95d5861640 1 -- 172.21.15.154:0/274442285 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d00c41e0 con 0x7f95d015e520 2023-09-23T03:27:58.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.129+0000 7f95b7fff640 1 -- 172.21.15.154:0/274442285 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c8012600 con 0x7f95d015e520 2023-09-23T03:27:58.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.129+0000 7f95d5861640 1 -- 172.21.15.154:0/274442285 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d00afa10 con 0x7f95d015e520 2023-09-23T03:27:58.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.130+0000 7f95d5861640 1 -- 172.21.15.154:0/274442285 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95d00c4420 con 0x7f95d015e520 2023-09-23T03:27:58.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.131+0000 7f95b7fff640 1 -- 172.21.15.154:0/274442285 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f95c8007070 con 0x7f95d015e520 2023-09-23T03:27:58.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.132+0000 7f95b7fff640 1 --2- 172.21.15.154:0/274442285 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f95bc06af30 0x7f95bc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:58.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.132+0000 7f95b7fff640 1 -- 172.21.15.154:0/274442285 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f95c8089710 con 0x7f95d015e520 2023-09-23T03:27:58.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.132+0000 7f95ceffd640 1 --2- 172.21.15.154:0/274442285 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f95bc06af30 0x7f95bc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:58.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.133+0000 7f95ceffd640 1 --2- 172.21.15.154:0/274442285 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f95bc06af30 0x7f95bc06d3f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f95c0009ad0 tx=0x7f95c0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:58.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.137+0000 7f95b7fff640 1 -- 172.21.15.154:0/274442285 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f95d00c4420 con 0x7f95d015e520 2023-09-23T03:27:58.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.312+0000 7f95d5861640 1 -- 172.21.15.154:0/274442285 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f95d012ed90 con 0x7f95d015e520 2023-09-23T03:27:58.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.313+0000 7f95b7fff640 1 -- 172.21.15.154:0/274442285 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v126) v1 ==== 104+0+11 (secure 0 0 0) 0x7f95d012ed90 con 0x7f95d015e520 2023-09-23T03:27:58.314 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:27:58.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.315+0000 7f95d5861640 1 -- 172.21.15.154:0/274442285 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f95bc06af30 msgr2=0x7f95bc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:58.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.315+0000 7f95d5861640 1 --2- 172.21.15.154:0/274442285 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f95bc06af30 0x7f95bc06d3f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f95c0009ad0 tx=0x7f95c0008040 comp rx=0 tx=0).stop 2023-09-23T03:27:58.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.315+0000 7f95d5861640 1 -- 172.21.15.154:0/274442285 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95d015e520 msgr2=0x7f95d006c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:58.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.315+0000 7f95d5861640 1 --2- 172.21.15.154:0/274442285 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95d015e520 0x7f95d006c1b0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f95c800ca80 tx=0x7f95c800cf40 comp rx=0 tx=0).stop 2023-09-23T03:27:58.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.316+0000 7f95d5861640 1 -- 172.21.15.154:0/274442285 shutdown_connections 2023-09-23T03:27:58.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.316+0000 7f95d5861640 1 --2- 172.21.15.154:0/274442285 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f95bc06af30 0x7f95bc06d3f0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:58.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.316+0000 7f95d5861640 1 --2- 172.21.15.154:0/274442285 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95d015e520 0x7f95d006c1b0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:58.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.316+0000 7f95d5861640 1 --2- 172.21.15.154:0/274442285 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f95d0144130 0x7f95d00c34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:58.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.316+0000 7f95d5861640 1 --2- 172.21.15.154:0/274442285 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f95d00caf80 0x7f95d00c4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:58.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.316+0000 7f95d5861640 1 -- 172.21.15.154:0/274442285 >> 172.21.15.154:0/274442285 conn(0x7f95d006a440 msgr2=0x7f95d015ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:58.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.316+0000 7f95d5861640 1 -- 172.21.15.154:0/274442285 shutdown_connections 2023-09-23T03:27:58.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.316+0000 7f95d5861640 1 -- 172.21.15.154:0/274442285 wait complete. 2023-09-23T03:27:58.336 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:27:58.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.546+0000 7f76df1a9640 1 Processor -- start 2023-09-23T03:27:58.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.546+0000 7f76df1a9640 1 -- start start 2023-09-23T03:27:58.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.548+0000 7f76df1a9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f76d80708e0 0x7f76d8070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:58.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.548+0000 7f76df1a9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f76d80c9b10 0x7f76d815d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:58.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.548+0000 7f76df1a9640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76d8071200 con 0x7f76d815d5f0 2023-09-23T03:27:58.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.548+0000 7f76df1a9640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76d806c1b0 con 0x7f76d80708e0 2023-09-23T03:27:58.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.548+0000 7f76df1a9640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76d815dde0 con 0x7f76d80c9b10 2023-09-23T03:27:58.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.548+0000 7f76dcf45640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f76d80708e0 0x7f76d8070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:58.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.548+0000 7f76dcf45640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f76d80708e0 0x7f76d8070cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49806/0 (socket says 172.21.15.154:49806) 2023-09-23T03:27:58.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.548+0000 7f76dcf45640 1 -- 172.21.15.154:0/3261140629 learned_addr learned my addr 172.21.15.154:0/3261140629 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:58.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.549+0000 7f76d7fff640 1 --2- 172.21.15.154:0/3261140629 >> v2:172.21.15.190:3300/0 conn(0x7f76d80c9b10 0x7f76d815d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:58.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.549+0000 7f76dcf45640 1 -- 172.21.15.154:0/3261140629 >> v2:172.21.15.190:3300/0 conn(0x7f76d80c9b10 msgr2=0x7f76d815d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:58.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.549+0000 7f76dcf45640 1 --2- 172.21.15.154:0/3261140629 >> v2:172.21.15.190:3300/0 conn(0x7f76d80c9b10 0x7f76d815d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:58.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.549+0000 7f76dcf45640 1 -- 172.21.15.154:0/3261140629 >> v1:172.21.15.154:6789/0 conn(0x7f76d815d5f0 legacy=0x7f76d806cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:58.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.549+0000 7f76dcf45640 1 -- 172.21.15.154:0/3261140629 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d806c3c0 con 0x7f76d80708e0 2023-09-23T03:27:58.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.550+0000 7f76dcf45640 1 --2- 172.21.15.154:0/3261140629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76d80708e0 0x7f76d8070cc0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f76d806e5f0 tx=0x7f76cc02b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb12d34395905f46 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:58.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.550+0000 7f76d77fe640 1 -- 172.21.15.154:0/3261140629 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76cc030870 con 0x7f76d80708e0 2023-09-23T03:27:58.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.550+0000 7f76d77fe640 1 -- 172.21.15.154:0/3261140629 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76cc0309d0 con 0x7f76d80708e0 2023-09-23T03:27:58.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.550+0000 7f76d77fe640 1 -- 172.21.15.154:0/3261140629 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76cc030d60 con 0x7f76d80708e0 2023-09-23T03:27:58.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.551+0000 7f76df1a9640 1 -- 172.21.15.154:0/3261140629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76d80708e0 msgr2=0x7f76d8070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:58.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.551+0000 7f76df1a9640 1 --2- 172.21.15.154:0/3261140629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76d80708e0 0x7f76d8070cc0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f76d806e5f0 tx=0x7f76cc02b9b0 comp rx=0 tx=0).stop 2023-09-23T03:27:58.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.551+0000 7f76df1a9640 1 -- 172.21.15.154:0/3261140629 shutdown_connections 2023-09-23T03:27:58.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.551+0000 7f76df1a9640 1 --2- 172.21.15.154:0/3261140629 >> v2:172.21.15.190:3300/0 conn(0x7f76d80c9b10 0x7f76d815d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:58.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.551+0000 7f76df1a9640 1 --2- 172.21.15.154:0/3261140629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76d80708e0 0x7f76d8070cc0 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:58.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.551+0000 7f76df1a9640 1 -- 172.21.15.154:0/3261140629 >> 172.21.15.154:0/3261140629 conn(0x7f76d806a440 msgr2=0x7f76d80b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:58.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.551+0000 7f76df1a9640 1 -- 172.21.15.154:0/3261140629 shutdown_connections 2023-09-23T03:27:58.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.551+0000 7f76df1a9640 1 -- 172.21.15.154:0/3261140629 wait complete. 2023-09-23T03:27:58.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.553+0000 7f76df1a9640 1 Processor -- start 2023-09-23T03:27:58.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.553+0000 7f76df1a9640 1 -- start start 2023-09-23T03:27:58.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.554+0000 7f76df1a9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76d80708e0 0x7f76d8151170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:58.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.554+0000 7f76df1a9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76d80c9b10 0x7f76d80cc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:58.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.554+0000 7f76df1a9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76d815d5f0 0x7f76d81572f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:58.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.554+0000 7f76df1a9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d80b2470 con 0x7f76d815d5f0 2023-09-23T03:27:58.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.554+0000 7f76df1a9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d80b17f0 con 0x7f76d80c9b10 2023-09-23T03:27:58.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.554+0000 7f76df1a9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76d80b1ad0 con 0x7f76d80708e0 2023-09-23T03:27:58.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.554+0000 7f76dd746640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76d815d5f0 0x7f76d81572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:58.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.554+0000 7f76dd746640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76d815d5f0 0x7f76d81572f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49822/0 (socket says 172.21.15.154:49822) 2023-09-23T03:27:58.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.554+0000 7f76dd746640 1 -- 172.21.15.154:0/258522266 learned_addr learned my addr 172.21.15.154:0/258522266 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:58.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.555+0000 7f76dcf45640 1 --2- 172.21.15.154:0/258522266 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76d80708e0 0x7f76d8151170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:58.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.555+0000 7f76d7fff640 1 --2- 172.21.15.154:0/258522266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76d80c9b10 0x7f76d80cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:58.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.555+0000 7f76dd746640 1 -- 172.21.15.154:0/258522266 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76d80708e0 msgr2=0x7f76d8151170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:58.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.555+0000 7f76dd746640 1 --2- 172.21.15.154:0/258522266 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76d80708e0 0x7f76d8151170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:58.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.555+0000 7f76dd746640 1 -- 172.21.15.154:0/258522266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76d80c9b10 msgr2=0x7f76d80cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:58.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.555+0000 7f76dd746640 1 --2- 172.21.15.154:0/258522266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76d80c9b10 0x7f76d80cc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:58.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.555+0000 7f76dd746640 1 -- 172.21.15.154:0/258522266 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76cc0094d0 con 0x7f76d815d5f0 2023-09-23T03:27:58.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.555+0000 7f76d7fff640 1 --2- 172.21.15.154:0/258522266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76d80c9b10 0x7f76d80cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:58.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.555+0000 7f76dd746640 1 --2- 172.21.15.154:0/258522266 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76d815d5f0 0x7f76d81572f0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f76d000c6b0 tx=0x7f76d000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:58.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.555+0000 7f76d57fa640 1 -- 172.21.15.154:0/258522266 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d0003b70 con 0x7f76d815d5f0 2023-09-23T03:27:58.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.555+0000 7f76d57fa640 1 -- 172.21.15.154:0/258522266 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d0003cf0 con 0x7f76d815d5f0 2023-09-23T03:27:58.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.555+0000 7f76df1a9640 1 -- 172.21.15.154:0/258522266 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76d80b1db0 con 0x7f76d815d5f0 2023-09-23T03:27:58.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.555+0000 7f76d57fa640 1 -- 172.21.15.154:0/258522266 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d0003ea0 con 0x7f76d815d5f0 2023-09-23T03:27:58.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.555+0000 7f76df1a9640 1 -- 172.21.15.154:0/258522266 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76d80ccec0 con 0x7f76d815d5f0 2023-09-23T03:27:58.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.556+0000 7f76df1a9640 1 -- 172.21.15.154:0/258522266 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76d80b25b0 con 0x7f76d815d5f0 2023-09-23T03:27:58.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.557+0000 7f76d57fa640 1 -- 172.21.15.154:0/258522266 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f76d0005310 con 0x7f76d815d5f0 2023-09-23T03:27:58.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.558+0000 7f76d57fa640 1 --2- 172.21.15.154:0/258522266 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f76bc06af30 0x7f76bc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:58.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.558+0000 7f76d57fa640 1 -- 172.21.15.154:0/258522266 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f76d0089640 con 0x7f76d815d5f0 2023-09-23T03:27:58.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.559+0000 7f76dcf45640 1 --2- 172.21.15.154:0/258522266 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f76bc06af30 0x7f76bc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:58.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.559+0000 7f76dcf45640 1 --2- 172.21.15.154:0/258522266 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f76bc06af30 0x7f76bc06d3f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f76cc036000 tx=0x7f76cc03b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:58.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.563+0000 7f76d57fa640 1 -- 172.21.15.154:0/258522266 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f76d80b25b0 con 0x7f76d815d5f0 2023-09-23T03:27:58.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.737+0000 7f76df1a9640 1 -- 172.21.15.154:0/258522266 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f76d80b25b0 con 0x7f76d815d5f0 2023-09-23T03:27:58.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.738+0000 7f76d57fa640 1 -- 172.21.15.154:0/258522266 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v126) v1 ==== 100+0+11 (secure 0 0 0) 0x7f76d80b25b0 con 0x7f76d815d5f0 2023-09-23T03:27:58.739 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:27:58.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.740+0000 7f76df1a9640 1 -- 172.21.15.154:0/258522266 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f76bc06af30 msgr2=0x7f76bc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:58.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.740+0000 7f76df1a9640 1 --2- 172.21.15.154:0/258522266 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f76bc06af30 0x7f76bc06d3f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f76cc036000 tx=0x7f76cc03b040 comp rx=0 tx=0).stop 2023-09-23T03:27:58.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.740+0000 7f76df1a9640 1 -- 172.21.15.154:0/258522266 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76d815d5f0 msgr2=0x7f76d81572f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:58.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.740+0000 7f76df1a9640 1 --2- 172.21.15.154:0/258522266 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76d815d5f0 0x7f76d81572f0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f76d000c6b0 tx=0x7f76d000cb70 comp rx=0 tx=0).stop 2023-09-23T03:27:58.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.740+0000 7f76df1a9640 1 -- 172.21.15.154:0/258522266 shutdown_connections 2023-09-23T03:27:58.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.740+0000 7f76df1a9640 1 --2- 172.21.15.154:0/258522266 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f76bc06af30 0x7f76bc06d3f0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:58.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.740+0000 7f76df1a9640 1 --2- 172.21.15.154:0/258522266 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76d815d5f0 0x7f76d81572f0 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:58.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.740+0000 7f76df1a9640 1 --2- 172.21.15.154:0/258522266 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f76d80c9b10 0x7f76d80cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:58.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.740+0000 7f76df1a9640 1 --2- 172.21.15.154:0/258522266 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f76d80708e0 0x7f76d8151170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:58.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.740+0000 7f76df1a9640 1 -- 172.21.15.154:0/258522266 >> 172.21.15.154:0/258522266 conn(0x7f76d806a440 msgr2=0x7f76d80cd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:58.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.740+0000 7f76df1a9640 1 -- 172.21.15.154:0/258522266 shutdown_connections 2023-09-23T03:27:58.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.740+0000 7f76df1a9640 1 -- 172.21.15.154:0/258522266 wait complete. 2023-09-23T03:27:58.760 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-23T03:27:58.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.970+0000 7fec9ae7e640 1 Processor -- start 2023-09-23T03:27:58.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.971+0000 7fec9ae7e640 1 -- start start 2023-09-23T03:27:58.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.972+0000 7fec9ae7e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fec940d10e0 0x7fec940b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:58.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.972+0000 7fec9ae7e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec940b3d20 con 0x7fec940ce230 2023-09-23T03:27:58.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.972+0000 7fec9ae7e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec940b3a40 con 0x7fec941600b0 2023-09-23T03:27:58.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.972+0000 7fec9ae7e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec940ce950 con 0x7fec940d10e0 2023-09-23T03:27:58.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.972+0000 7fec9967b640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fec940ce230 0x7fec940b5830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46640/0 (socket says 172.21.15.154:46640) 2023-09-23T03:27:58.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.972+0000 7fec9967b640 1 -- 172.21.15.154:0/3484375367 learned_addr learned my addr 172.21.15.154:0/3484375367 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:27:58.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.972+0000 7fec9a67d640 1 --2- 172.21.15.154:0/3484375367 >> v2:172.21.15.154:3300/0 conn(0x7fec940d10e0 0x7fec940b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:58.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.973+0000 7fec98e7a640 1 -- 172.21.15.154:0/3484375367 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 615854507 0 0) 0x7fec940b3d20 con 0x7fec940ce230 2023-09-23T03:27:58.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.973+0000 7fec98e7a640 1 -- 172.21.15.154:0/3484375367 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec80003520 con 0x7fec940ce230 2023-09-23T03:27:58.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.973+0000 7fec9a67d640 1 -- 172.21.15.154:0/3484375367 >> v1:172.21.15.190:6789/0 conn(0x7fec941600b0 legacy=0x7fec940d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:58.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.973+0000 7fec9a67d640 1 -- 172.21.15.154:0/3484375367 >> v1:172.21.15.154:6789/0 conn(0x7fec940ce230 legacy=0x7fec940b5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:58.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.973+0000 7fec9a67d640 1 -- 172.21.15.154:0/3484375367 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec940cec30 con 0x7fec940d10e0 2023-09-23T03:27:58.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.974+0000 7fec9a67d640 1 --2- 172.21.15.154:0/3484375367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec940d10e0 0x7fec940b5f40 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fec8c009d90 tx=0x7fec8c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3846fb18973d952 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:58.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.975+0000 7fec98e7a640 1 -- 172.21.15.154:0/3484375367 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8c02cc60 con 0x7fec940d10e0 2023-09-23T03:27:58.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.975+0000 7fec98e7a640 1 -- 172.21.15.154:0/3484375367 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec8c02cdc0 con 0x7fec940d10e0 2023-09-23T03:27:58.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.975+0000 7fec98e7a640 1 -- 172.21.15.154:0/3484375367 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8c034640 con 0x7fec940d10e0 2023-09-23T03:27:58.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.975+0000 7fec9ae7e640 1 -- 172.21.15.154:0/3484375367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec940d10e0 msgr2=0x7fec940b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:58.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.975+0000 7fec9ae7e640 1 --2- 172.21.15.154:0/3484375367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec940d10e0 0x7fec940b5f40 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fec8c009d90 tx=0x7fec8c02beb0 comp rx=0 tx=0).stop 2023-09-23T03:27:58.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.975+0000 7fec9ae7e640 1 -- 172.21.15.154:0/3484375367 shutdown_connections 2023-09-23T03:27:58.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.975+0000 7fec9ae7e640 1 --2- 172.21.15.154:0/3484375367 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec940d10e0 0x7fec940b5f40 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:58.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.976+0000 7fec9ae7e640 1 -- 172.21.15.154:0/3484375367 >> 172.21.15.154:0/3484375367 conn(0x7fec9409bb30 msgr2=0x7fec940d27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:58.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.976+0000 7fec9ae7e640 1 -- 172.21.15.154:0/3484375367 shutdown_connections 2023-09-23T03:27:58.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.976+0000 7fec9ae7e640 1 -- 172.21.15.154:0/3484375367 wait complete. 2023-09-23T03:27:58.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.977+0000 7fec9ae7e640 1 Processor -- start 2023-09-23T03:27:58.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.978+0000 7fec9ae7e640 1 -- start start 2023-09-23T03:27:58.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.978+0000 7fec9ae7e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec940ce230 0x7fec940cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:58.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.978+0000 7fec9ae7e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec940d10e0 0x7fec940c6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:58.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.978+0000 7fec9ae7e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec941600b0 0x7fec94156810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:58.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.978+0000 7fec9ae7e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec940c75f0 con 0x7fec940ce230 2023-09-23T03:27:58.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.978+0000 7fec9ae7e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec940c6780 con 0x7fec941600b0 2023-09-23T03:27:58.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.978+0000 7fec9ae7e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec940c69e0 con 0x7fec940d10e0 2023-09-23T03:27:58.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.979+0000 7fec99e7c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec940ce230 0x7fec940cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:58.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.979+0000 7fec99e7c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec940ce230 0x7fec940cb810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49836/0 (socket says 172.21.15.154:49836) 2023-09-23T03:27:58.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.979+0000 7fec99e7c640 1 -- 172.21.15.154:0/3309776737 learned_addr learned my addr 172.21.15.154:0/3309776737 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:58.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.979+0000 7fec99e7c640 1 -- 172.21.15.154:0/3309776737 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec940d10e0 msgr2=0x7fec940c6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:58.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.979+0000 7fec99e7c640 1 --2- 172.21.15.154:0/3309776737 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec940d10e0 0x7fec940c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:58.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.979+0000 7fec99e7c640 1 -- 172.21.15.154:0/3309776737 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec941600b0 msgr2=0x7fec94156810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:58.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.979+0000 7fec99e7c640 1 --2- 172.21.15.154:0/3309776737 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec941600b0 0x7fec94156810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:58.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.979+0000 7fec99e7c640 1 -- 172.21.15.154:0/3309776737 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec8c009a70 con 0x7fec940ce230 2023-09-23T03:27:58.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.979+0000 7fec99e7c640 1 --2- 172.21.15.154:0/3309776737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec940ce230 0x7fec940cb810 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fec9000c3c0 tx=0x7fec9000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:58.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.980+0000 7fec8a7fc640 1 -- 172.21.15.154:0/3309776737 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec900052d0 con 0x7fec940ce230 2023-09-23T03:27:58.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.980+0000 7fec8a7fc640 1 -- 172.21.15.154:0/3309776737 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec90005450 con 0x7fec940ce230 2023-09-23T03:27:58.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.980+0000 7fec9ae7e640 1 -- 172.21.15.154:0/3309776737 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec940c6cc0 con 0x7fec940ce230 2023-09-23T03:27:58.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.980+0000 7fec8a7fc640 1 -- 172.21.15.154:0/3309776737 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec90007260 con 0x7fec940ce230 2023-09-23T03:27:58.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.980+0000 7fec9ae7e640 1 -- 172.21.15.154:0/3309776737 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec940c7280 con 0x7fec940ce230 2023-09-23T03:27:58.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.981+0000 7fec9ae7e640 1 -- 172.21.15.154:0/3309776737 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec9404e920 con 0x7fec940ce230 2023-09-23T03:27:58.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.982+0000 7fec8a7fc640 1 -- 172.21.15.154:0/3309776737 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fec900073c0 con 0x7fec940ce230 2023-09-23T03:27:58.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.983+0000 7fec8a7fc640 1 --2- 172.21.15.154:0/3309776737 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fec7806af30 0x7fec7806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:58.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.983+0000 7fec8a7fc640 1 -- 172.21.15.154:0/3309776737 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fec90088c60 con 0x7fec940ce230 2023-09-23T03:27:58.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.983+0000 7fec9967b640 1 --2- 172.21.15.154:0/3309776737 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fec7806af30 0x7fec7806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:58.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.984+0000 7fec9967b640 1 --2- 172.21.15.154:0/3309776737 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fec7806af30 0x7fec7806d3f0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fec94157a20 tx=0x7fec84007930 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:58.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:58.988+0000 7fec8a7fc640 1 -- 172.21.15.154:0/3309776737 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fec90053b50 con 0x7fec940ce230 2023-09-23T03:27:59.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.193+0000 7fec9ae7e640 1 -- 172.21.15.154:0/3309776737 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fec94156d50 con 0x7fec940ce230 2023-09-23T03:27:59.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.194+0000 7fec8a7fc640 1 -- 172.21.15.154:0/3309776737 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v126) v1 ==== 80+0+0 (secure 0 0 0) 0x7fec90057170 con 0x7fec940ce230 2023-09-23T03:27:59.195 INFO:teuthology.orchestra.run.smithi154.stderr:listed 0 entries 2023-09-23T03:27:59.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.196+0000 7fec9ae7e640 1 -- 172.21.15.154:0/3309776737 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fec7806af30 msgr2=0x7fec7806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:59.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.196+0000 7fec9ae7e640 1 --2- 172.21.15.154:0/3309776737 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fec7806af30 0x7fec7806d3f0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fec94157a20 tx=0x7fec84007930 comp rx=0 tx=0).stop 2023-09-23T03:27:59.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.196+0000 7fec9ae7e640 1 -- 172.21.15.154:0/3309776737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec940ce230 msgr2=0x7fec940cb810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:59.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.196+0000 7fec9ae7e640 1 --2- 172.21.15.154:0/3309776737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec940ce230 0x7fec940cb810 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fec9000c3c0 tx=0x7fec9000c880 comp rx=0 tx=0).stop 2023-09-23T03:27:59.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.197+0000 7fec9ae7e640 1 -- 172.21.15.154:0/3309776737 shutdown_connections 2023-09-23T03:27:59.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.197+0000 7fec9ae7e640 1 --2- 172.21.15.154:0/3309776737 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fec7806af30 0x7fec7806d3f0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:59.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.197+0000 7fec9ae7e640 1 --2- 172.21.15.154:0/3309776737 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fec941600b0 0x7fec94156810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:59.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.197+0000 7fec9ae7e640 1 --2- 172.21.15.154:0/3309776737 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fec940d10e0 0x7fec940c6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:59.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.197+0000 7fec9ae7e640 1 --2- 172.21.15.154:0/3309776737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fec940ce230 0x7fec940cb810 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:59.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.197+0000 7fec9ae7e640 1 -- 172.21.15.154:0/3309776737 >> 172.21.15.154:0/3309776737 conn(0x7fec9409bb30 msgr2=0x7fec9406aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:59.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.197+0000 7fec9ae7e640 1 -- 172.21.15.154:0/3309776737 shutdown_connections 2023-09-23T03:27:59.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.197+0000 7fec9ae7e640 1 -- 172.21.15.154:0/3309776737 wait complete. 2023-09-23T03:27:59.217 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:27:59.217 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:27:59.271 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:27:59.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.481+0000 7f67ea57d640 1 Processor -- start 2023-09-23T03:27:59.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.481+0000 7f67ea57d640 1 -- start start 2023-09-23T03:27:59.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.483+0000 7f67ea57d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f67e4160280 0x7f67e406d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:59.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.483+0000 7f67ea57d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f67e4160970 0x7f67e40b0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:59.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.483+0000 7f67ea57d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67e41613c0 con 0x7f67e4160df0 2023-09-23T03:27:59.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.483+0000 7f67ea57d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67e406c810 con 0x7f67e4160970 2023-09-23T03:27:59.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.483+0000 7f67ea57d640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67e41616a0 con 0x7f67e4160280 2023-09-23T03:27:59.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.483+0000 7f67e8b1a640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f67e4160df0 0x7f67e40c8f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46652/0 (socket says 172.21.15.154:46652) 2023-09-23T03:27:59.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.483+0000 7f67e8b1a640 1 -- 172.21.15.154:0/1875594933 learned_addr learned my addr 172.21.15.154:0/1875594933 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:27:59.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.483+0000 7f67e37fe640 1 --2- 172.21.15.154:0/1875594933 >> v2:172.21.15.154:3300/0 conn(0x7f67e4160970 0x7f67e40b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:59.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.484+0000 7f67e3fff640 1 --2- 172.21.15.154:0/1875594933 >> v2:172.21.15.190:3300/0 conn(0x7f67e4160280 0x7f67e406d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:59.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.484+0000 7f67e37fe640 1 -- 172.21.15.154:0/1875594933 >> v2:172.21.15.190:3300/0 conn(0x7f67e4160280 msgr2=0x7f67e406d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:59.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.484+0000 7f67e37fe640 1 --2- 172.21.15.154:0/1875594933 >> v2:172.21.15.190:3300/0 conn(0x7f67e4160280 0x7f67e406d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:59.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.484+0000 7f67e37fe640 1 -- 172.21.15.154:0/1875594933 >> v1:172.21.15.154:6789/0 conn(0x7f67e4160df0 legacy=0x7f67e40c8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:59.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.484+0000 7f67e37fe640 1 -- 172.21.15.154:0/1875594933 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e406ca20 con 0x7f67e4160970 2023-09-23T03:27:59.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.484+0000 7f67e2ffd640 1 -- 172.21.15.154:0/1875594933 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 593022979 0 0) 0x7f67e41613c0 con 0x7f67e4160df0 2023-09-23T03:27:59.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.484+0000 7f67e3fff640 1 --2- 172.21.15.154:0/1875594933 >> v2:172.21.15.190:3300/0 conn(0x7f67e4160280 0x7f67e406d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:59.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.485+0000 7f67e37fe640 1 --2- 172.21.15.154:0/1875594933 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67e4160970 0x7f67e40b0070 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f67e40cbfc0 tx=0x7f67d802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f4799c4ae64b0df server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:59.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.485+0000 7f67e2ffd640 1 -- 172.21.15.154:0/1875594933 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d8031830 con 0x7f67e4160970 2023-09-23T03:27:59.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.485+0000 7f67e2ffd640 1 -- 172.21.15.154:0/1875594933 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d8031990 con 0x7f67e4160970 2023-09-23T03:27:59.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.485+0000 7f67e2ffd640 1 -- 172.21.15.154:0/1875594933 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d8031d20 con 0x7f67e4160970 2023-09-23T03:27:59.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.486+0000 7f67ea57d640 1 -- 172.21.15.154:0/1875594933 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67e4160970 msgr2=0x7f67e40b0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:59.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.486+0000 7f67ea57d640 1 --2- 172.21.15.154:0/1875594933 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67e4160970 0x7f67e40b0070 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f67e40cbfc0 tx=0x7f67d802b8d0 comp rx=0 tx=0).stop 2023-09-23T03:27:59.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.486+0000 7f67ea57d640 1 -- 172.21.15.154:0/1875594933 shutdown_connections 2023-09-23T03:27:59.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.486+0000 7f67ea57d640 1 --2- 172.21.15.154:0/1875594933 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67e4160970 0x7f67e40b0070 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:59.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.486+0000 7f67ea57d640 1 --2- 172.21.15.154:0/1875594933 >> v2:172.21.15.190:3300/0 conn(0x7f67e4160280 0x7f67e406d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:59.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.486+0000 7f67ea57d640 1 -- 172.21.15.154:0/1875594933 >> 172.21.15.154:0/1875594933 conn(0x7f67e40cb070 msgr2=0x7f67e40cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:59.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.486+0000 7f67ea57d640 1 -- 172.21.15.154:0/1875594933 shutdown_connections 2023-09-23T03:27:59.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.486+0000 7f67ea57d640 1 -- 172.21.15.154:0/1875594933 wait complete. 2023-09-23T03:27:59.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.488+0000 7f67ea57d640 1 Processor -- start 2023-09-23T03:27:59.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.488+0000 7f67ea57d640 1 -- start start 2023-09-23T03:27:59.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.489+0000 7f67ea57d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67e4160280 0x7f67e40c60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:59.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.489+0000 7f67ea57d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67e4160970 0x7f67e41e7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:59.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.489+0000 7f67ea57d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67e4160df0 0x7f67e40c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:59.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.489+0000 7f67ea57d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e40b1c50 con 0x7f67e4160df0 2023-09-23T03:27:59.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.489+0000 7f67ea57d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e40b12b0 con 0x7f67e4160970 2023-09-23T03:27:59.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.489+0000 7f67ea57d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e40b1590 con 0x7f67e4160280 2023-09-23T03:27:59.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.489+0000 7f67e8b1a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67e4160df0 0x7f67e40c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:59.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.489+0000 7f67e8b1a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67e4160df0 0x7f67e40c7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49846/0 (socket says 172.21.15.154:49846) 2023-09-23T03:27:59.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.489+0000 7f67e8b1a640 1 -- 172.21.15.154:0/741264578 learned_addr learned my addr 172.21.15.154:0/741264578 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:59.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.490+0000 7f67e37fe640 1 --2- 172.21.15.154:0/741264578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67e4160970 0x7f67e41e7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:59.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.490+0000 7f67e8b1a640 1 -- 172.21.15.154:0/741264578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67e4160280 msgr2=0x7f67e40c60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:59.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.490+0000 7f67e3fff640 1 --2- 172.21.15.154:0/741264578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67e4160280 0x7f67e40c60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:59.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.490+0000 7f67e8b1a640 1 --2- 172.21.15.154:0/741264578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67e4160280 0x7f67e40c60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:59.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.490+0000 7f67e8b1a640 1 -- 172.21.15.154:0/741264578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67e4160970 msgr2=0x7f67e41e7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:59.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.490+0000 7f67e8b1a640 1 --2- 172.21.15.154:0/741264578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67e4160970 0x7f67e41e7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:59.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.490+0000 7f67e8b1a640 1 -- 172.21.15.154:0/741264578 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d8009400 con 0x7f67e4160df0 2023-09-23T03:27:59.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.490+0000 7f67e37fe640 1 --2- 172.21.15.154:0/741264578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67e4160970 0x7f67e41e7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:59.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.490+0000 7f67e8b1a640 1 --2- 172.21.15.154:0/741264578 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67e4160df0 0x7f67e40c7b90 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00c700 tx=0x7f67dc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:59.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.490+0000 7f67e0ff9640 1 -- 172.21.15.154:0/741264578 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67dc00ce10 con 0x7f67e4160df0 2023-09-23T03:27:59.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.490+0000 7f67e0ff9640 1 -- 172.21.15.154:0/741264578 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67dc00a640 con 0x7f67e4160df0 2023-09-23T03:27:59.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.490+0000 7f67e0ff9640 1 -- 172.21.15.154:0/741264578 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67dc00aa70 con 0x7f67e4160df0 2023-09-23T03:27:59.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.490+0000 7f67ea57d640 1 -- 172.21.15.154:0/741264578 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e40b1870 con 0x7f67e4160df0 2023-09-23T03:27:59.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.490+0000 7f67ea57d640 1 -- 172.21.15.154:0/741264578 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e40c6920 con 0x7f67e4160df0 2023-09-23T03:27:59.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.492+0000 7f67ea57d640 1 -- 172.21.15.154:0/741264578 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67e4067870 con 0x7f67e4160df0 2023-09-23T03:27:59.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.492+0000 7f67e0ff9640 1 -- 172.21.15.154:0/741264578 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f67dc00abd0 con 0x7f67e4160df0 2023-09-23T03:27:59.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.493+0000 7f67e0ff9640 1 --2- 172.21.15.154:0/741264578 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f67d006af30 0x7f67d006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:59.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.493+0000 7f67e0ff9640 1 -- 172.21.15.154:0/741264578 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f67dc088df0 con 0x7f67e4160df0 2023-09-23T03:27:59.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.494+0000 7f67e3fff640 1 --2- 172.21.15.154:0/741264578 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f67d006af30 0x7f67d006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:59.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.494+0000 7f67e3fff640 1 --2- 172.21.15.154:0/741264578 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f67d006af30 0x7f67d006d3f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f67d4008850 tx=0x7f67d4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:59.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.498+0000 7f67e0ff9640 1 -- 172.21.15.154:0/741264578 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f67dc053ce0 con 0x7f67e4160df0 2023-09-23T03:27:59.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.692+0000 7f67ea57d640 1 -- 172.21.15.154:0/741264578 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67e40b1d90 con 0x7f67e4160df0 2023-09-23T03:27:59.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.694+0000 7f67e0ff9640 1 -- 172.21.15.154:0/741264578 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+16820 (secure 0 0 0) 0x7f67e40b1d90 con 0x7f67e4160df0 2023-09-23T03:27:59.695 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:27:59.696 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":126,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:27:46.016186+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:27:42.207617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-09-23T03:27:43.473867+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:27:59.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.696+0000 7f67ea57d640 1 -- 172.21.15.154:0/741264578 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f67d006af30 msgr2=0x7f67d006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:59.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.696+0000 7f67ea57d640 1 --2- 172.21.15.154:0/741264578 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f67d006af30 0x7f67d006d3f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f67d4008850 tx=0x7f67d4008040 comp rx=0 tx=0).stop 2023-09-23T03:27:59.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.696+0000 7f67ea57d640 1 -- 172.21.15.154:0/741264578 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67e4160df0 msgr2=0x7f67e40c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:59.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.696+0000 7f67ea57d640 1 --2- 172.21.15.154:0/741264578 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67e4160df0 0x7f67e40c7b90 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00c700 tx=0x7f67dc00cbc0 comp rx=0 tx=0).stop 2023-09-23T03:27:59.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.697+0000 7f67ea57d640 1 -- 172.21.15.154:0/741264578 shutdown_connections 2023-09-23T03:27:59.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.697+0000 7f67ea57d640 1 --2- 172.21.15.154:0/741264578 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f67d006af30 0x7f67d006d3f0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:59.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.697+0000 7f67ea57d640 1 --2- 172.21.15.154:0/741264578 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f67e4160df0 0x7f67e40c7b90 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:59.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.697+0000 7f67ea57d640 1 --2- 172.21.15.154:0/741264578 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f67e4160970 0x7f67e41e7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:59.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.697+0000 7f67ea57d640 1 --2- 172.21.15.154:0/741264578 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f67e4160280 0x7f67e40c60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:59.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.697+0000 7f67ea57d640 1 -- 172.21.15.154:0/741264578 >> 172.21.15.154:0/741264578 conn(0x7f67e40cb070 msgr2=0x7f67e415dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:59.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.697+0000 7f67ea57d640 1 -- 172.21.15.154:0/741264578 shutdown_connections 2023-09-23T03:27:59.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.697+0000 7f67ea57d640 1 -- 172.21.15.154:0/741264578 wait complete. 2023-09-23T03:27:59.718 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:27:42.207617+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:27:43.473867+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T03:27:59.718 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:27:59.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.929+0000 7f14d7b33640 1 Processor -- start 2023-09-23T03:27:59.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.929+0000 7f14d7b33640 1 -- start start 2023-09-23T03:27:59.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.930+0000 7f14d7b33640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f14d015cc50 0x7f14d00ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:59.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.930+0000 7f14d7b33640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f14d0161300 0x7f14d00b0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:59.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.930+0000 7f14d7b33640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14d015dcd0 con 0x7f14d00b05b0 2023-09-23T03:27:59.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.930+0000 7f14d7b33640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14d00cd0a0 con 0x7f14d0161300 2023-09-23T03:27:59.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.930+0000 7f14d7b33640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14d00b09e0 con 0x7f14d015cc50 2023-09-23T03:27:59.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.931+0000 7f14d50ce640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f14d0161300 0x7f14d00b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:59.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.931+0000 7f14d50ce640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f14d0161300 0x7f14d00b0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49862/0 (socket says 172.21.15.154:49862) 2023-09-23T03:27:59.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.931+0000 7f14d50ce640 1 -- 172.21.15.154:0/3611966850 learned_addr learned my addr 172.21.15.154:0/3611966850 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:59.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.931+0000 7f14d58cf640 1 --2- 172.21.15.154:0/3611966850 >> v2:172.21.15.190:3300/0 conn(0x7f14d015cc50 0x7f14d00ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:59.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.931+0000 7f14d50ce640 1 -- 172.21.15.154:0/3611966850 >> v2:172.21.15.190:3300/0 conn(0x7f14d015cc50 msgr2=0x7f14d00ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:59.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.931+0000 7f14d50ce640 1 --2- 172.21.15.154:0/3611966850 >> v2:172.21.15.190:3300/0 conn(0x7f14d015cc50 0x7f14d00ccb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:59.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.931+0000 7f14d50ce640 1 -- 172.21.15.154:0/3611966850 >> v1:172.21.15.190:6789/0 conn(0x7f14d00b05b0 legacy=0x7f14d0070130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:27:59.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.931+0000 7f14d50ce640 1 -- 172.21.15.154:0/3611966850 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d01617b0 con 0x7f14d0161300 2023-09-23T03:27:59.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.931+0000 7f14d58cf640 1 --2- 172.21.15.154:0/3611966850 >> v2:172.21.15.190:3300/0 conn(0x7f14d015cc50 0x7f14d00ccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:59.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.932+0000 7f14d50ce640 1 --2- 172.21.15.154:0/3611966850 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14d0161300 0x7f14d00b0070 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f14d006c310 tx=0x7f14c002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc3fc4f60c3b9f65 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:59.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.933+0000 7f14d48cd640 1 -- 172.21.15.154:0/3611966850 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c0031870 con 0x7f14d0161300 2023-09-23T03:27:59.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.933+0000 7f14d48cd640 1 -- 172.21.15.154:0/3611966850 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c00319d0 con 0x7f14d0161300 2023-09-23T03:27:59.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.933+0000 7f14d48cd640 1 -- 172.21.15.154:0/3611966850 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c0031d60 con 0x7f14d0161300 2023-09-23T03:27:59.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.933+0000 7f14d7b33640 1 -- 172.21.15.154:0/3611966850 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14d0161300 msgr2=0x7f14d00b0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:59.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.933+0000 7f14d7b33640 1 --2- 172.21.15.154:0/3611966850 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14d0161300 0x7f14d00b0070 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f14d006c310 tx=0x7f14c002b910 comp rx=0 tx=0).stop 2023-09-23T03:27:59.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.933+0000 7f14d7b33640 1 -- 172.21.15.154:0/3611966850 shutdown_connections 2023-09-23T03:27:59.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.933+0000 7f14d7b33640 1 --2- 172.21.15.154:0/3611966850 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14d0161300 0x7f14d00b0070 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:59.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.933+0000 7f14d7b33640 1 --2- 172.21.15.154:0/3611966850 >> v2:172.21.15.190:3300/0 conn(0x7f14d015cc50 0x7f14d00ccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:59.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.933+0000 7f14d7b33640 1 -- 172.21.15.154:0/3611966850 >> 172.21.15.154:0/3611966850 conn(0x7f14d00cb070 msgr2=0x7f14d00cb480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:27:59.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.933+0000 7f14d7b33640 1 -- 172.21.15.154:0/3611966850 shutdown_connections 2023-09-23T03:27:59.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.934+0000 7f14d7b33640 1 -- 172.21.15.154:0/3611966850 wait complete. 2023-09-23T03:27:59.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.935+0000 7f14d7b33640 1 Processor -- start 2023-09-23T03:27:59.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.935+0000 7f14d7b33640 1 -- start start 2023-09-23T03:27:59.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.936+0000 7f14d7b33640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14d00b05b0 0x7f14d00c43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:59.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.936+0000 7f14d7b33640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14d015cc50 0x7f14d01f3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:59.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.936+0000 7f14d7b33640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14d0161300 0x7f14d01efe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:59.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.936+0000 7f14d7b33640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d01f3990 con 0x7f14d015cc50 2023-09-23T03:27:59.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.936+0000 7f14d7b33640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d01f2990 con 0x7f14d0161300 2023-09-23T03:27:59.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.936+0000 7f14d7b33640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d00b3c50 con 0x7f14d00b05b0 2023-09-23T03:27:59.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.937+0000 7f14d50ce640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14d015cc50 0x7f14d01f3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:59.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.937+0000 7f14d50ce640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14d015cc50 0x7f14d01f3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49870/0 (socket says 172.21.15.154:49870) 2023-09-23T03:27:59.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.937+0000 7f14d50ce640 1 -- 172.21.15.154:0/1924093882 learned_addr learned my addr 172.21.15.154:0/1924093882 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:27:59.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.937+0000 7f14d60d0640 1 --2- 172.21.15.154:0/1924093882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14d0161300 0x7f14d01efe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:59.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.937+0000 7f14d50ce640 1 -- 172.21.15.154:0/1924093882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14d00b05b0 msgr2=0x7f14d00c43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:59.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.937+0000 7f14d58cf640 1 --2- 172.21.15.154:0/1924093882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14d00b05b0 0x7f14d00c43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:59.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.937+0000 7f14d50ce640 1 --2- 172.21.15.154:0/1924093882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14d00b05b0 0x7f14d00c43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:59.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.937+0000 7f14d50ce640 1 -- 172.21.15.154:0/1924093882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14d0161300 msgr2=0x7f14d01efe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:27:59.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.937+0000 7f14d50ce640 1 --2- 172.21.15.154:0/1924093882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14d0161300 0x7f14d01efe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:27:59.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.937+0000 7f14d50ce640 1 -- 172.21.15.154:0/1924093882 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c00094d0 con 0x7f14d015cc50 2023-09-23T03:27:59.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.937+0000 7f14d60d0640 1 --2- 172.21.15.154:0/1924093882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14d0161300 0x7f14d01efe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:27:59.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.937+0000 7f14d50ce640 1 --2- 172.21.15.154:0/1924093882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14d015cc50 0x7f14d01f3450 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f14c0000c00 tx=0x7f14c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:59.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.938+0000 7f14c67fc640 1 -- 172.21.15.154:0/1924093882 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c0031840 con 0x7f14d015cc50 2023-09-23T03:27:59.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.938+0000 7f14c67fc640 1 -- 172.21.15.154:0/1924093882 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c00443f0 con 0x7f14d015cc50 2023-09-23T03:27:59.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.938+0000 7f14c67fc640 1 -- 172.21.15.154:0/1924093882 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c00447a0 con 0x7f14d015cc50 2023-09-23T03:27:59.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.938+0000 7f14d7b33640 1 -- 172.21.15.154:0/1924093882 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14d00b3f30 con 0x7f14d015cc50 2023-09-23T03:27:59.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.938+0000 7f14d7b33640 1 -- 172.21.15.154:0/1924093882 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d01f2f30 con 0x7f14d015cc50 2023-09-23T03:27:59.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.939+0000 7f14d7b33640 1 -- 172.21.15.154:0/1924093882 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14d0067870 con 0x7f14d015cc50 2023-09-23T03:27:59.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.940+0000 7f14c67fc640 1 -- 172.21.15.154:0/1924093882 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f14c004d450 con 0x7f14d015cc50 2023-09-23T03:27:59.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.941+0000 7f14c67fc640 1 --2- 172.21.15.154:0/1924093882 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f14b406af30 0x7f14b406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:27:59.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.941+0000 7f14c67fc640 1 -- 172.21.15.154:0/1924093882 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f14c002d040 con 0x7f14d015cc50 2023-09-23T03:27:59.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.941+0000 7f14d58cf640 1 --2- 172.21.15.154:0/1924093882 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f14b406af30 0x7f14b406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:27:59.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.942+0000 7f14d58cf640 1 --2- 172.21.15.154:0/1924093882 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f14b406af30 0x7f14b406d3f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f14d00ad090 tx=0x7f14cc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:27:59.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:27:59.945+0000 7f14c67fc640 1 -- 172.21.15.154:0/1924093882 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f14c0076f90 con 0x7f14d015cc50 2023-09-23T03:28:00.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.138+0000 7f14d7b33640 1 -- 172.21.15.154:0/1924093882 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f14d0073eb0 con 0x7f14d015cc50 2023-09-23T03:28:00.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.139+0000 7f14c67fc640 1 -- 172.21.15.154:0/1924093882 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v126) v1 ==== 93+0+10 (secure 0 0 0) 0x7f14c00350a0 con 0x7f14d015cc50 2023-09-23T03:28:00.140 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:28:00.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.141+0000 7f14d7b33640 1 -- 172.21.15.154:0/1924093882 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f14b406af30 msgr2=0x7f14b406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:00.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.141+0000 7f14d7b33640 1 --2- 172.21.15.154:0/1924093882 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f14b406af30 0x7f14b406d3f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f14d00ad090 tx=0x7f14cc008040 comp rx=0 tx=0).stop 2023-09-23T03:28:00.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.141+0000 7f14d7b33640 1 -- 172.21.15.154:0/1924093882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14d015cc50 msgr2=0x7f14d01f3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:00.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.141+0000 7f14d7b33640 1 --2- 172.21.15.154:0/1924093882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14d015cc50 0x7f14d01f3450 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f14c0000c00 tx=0x7f14c00046c0 comp rx=0 tx=0).stop 2023-09-23T03:28:00.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.142+0000 7f14d7b33640 1 -- 172.21.15.154:0/1924093882 shutdown_connections 2023-09-23T03:28:00.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.142+0000 7f14d7b33640 1 --2- 172.21.15.154:0/1924093882 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f14b406af30 0x7f14b406d3f0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:00.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.142+0000 7f14d7b33640 1 --2- 172.21.15.154:0/1924093882 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f14d0161300 0x7f14d01efe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:00.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.142+0000 7f14d7b33640 1 --2- 172.21.15.154:0/1924093882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14d015cc50 0x7f14d01f3450 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:00.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.142+0000 7f14d7b33640 1 --2- 172.21.15.154:0/1924093882 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f14d00b05b0 0x7f14d00c43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:00.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.142+0000 7f14d7b33640 1 -- 172.21.15.154:0/1924093882 >> 172.21.15.154:0/1924093882 conn(0x7f14d00cb070 msgr2=0x7f14d015f210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:00.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.142+0000 7f14d7b33640 1 -- 172.21.15.154:0/1924093882 shutdown_connections 2023-09-23T03:28:00.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.142+0000 7f14d7b33640 1 -- 172.21.15.154:0/1924093882 wait complete. 2023-09-23T03:28:00.162 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:28:00.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.381+0000 7f2a9d613640 1 Processor -- start 2023-09-23T03:28:00.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.381+0000 7f2a9d613640 1 -- start start 2023-09-23T03:28:00.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.383+0000 7f2a9d613640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a9815cb00 0x7f2a98070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:00.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.383+0000 7f2a9d613640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2a980711a0 0x7f2a980c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:00.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.383+0000 7f2a9d613640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2a980c9190 0x7f2a980ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:00.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.383+0000 7f2a9d613640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a980c99d0 con 0x7f2a980c9190 2023-09-23T03:28:00.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.383+0000 7f2a9d613640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a980c96f0 con 0x7f2a980711a0 2023-09-23T03:28:00.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.383+0000 7f2a9d613640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a98070180 con 0x7f2a9815cb00 2023-09-23T03:28:00.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.383+0000 7f2a9ce12640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2a980c9190 0x7f2a980ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:00.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.383+0000 7f2a9ce12640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2a980c9190 0x7f2a980ce380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49876/0 (socket says 172.21.15.154:49876) 2023-09-23T03:28:00.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.383+0000 7f2a9ce12640 1 -- 172.21.15.154:0/1267485289 learned_addr learned my addr 172.21.15.154:0/1267485289 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:00.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.384+0000 7f2a8ffff640 1 --2- 172.21.15.154:0/1267485289 >> v2:172.21.15.190:3300/0 conn(0x7f2a980711a0 0x7f2a980c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:00.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.384+0000 7f2a9ce12640 1 -- 172.21.15.154:0/1267485289 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a9815cb00 msgr2=0x7f2a98070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:00.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.384+0000 7f2a97fff640 1 --2- 172.21.15.154:0/1267485289 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a9815cb00 0x7f2a98070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:00.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.384+0000 7f2a9ce12640 1 --2- 172.21.15.154:0/1267485289 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a9815cb00 0x7f2a98070c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:00.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.384+0000 7f2a9ce12640 1 -- 172.21.15.154:0/1267485289 >> v2:172.21.15.190:3300/0 conn(0x7f2a980711a0 msgr2=0x7f2a980c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:00.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.384+0000 7f2a9ce12640 1 --2- 172.21.15.154:0/1267485289 >> v2:172.21.15.190:3300/0 conn(0x7f2a980711a0 0x7f2a980c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:00.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.384+0000 7f2a9ce12640 1 -- 172.21.15.154:0/1267485289 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a98070460 con 0x7f2a980c9190 2023-09-23T03:28:00.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.385+0000 7f2a9ce12640 1 --2- 172.21.15.154:0/1267485289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2a980c9190 0x7f2a980ce380 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f2a980cbf30 tx=0x7f2a9002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e73a4b45ab750bf server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:00.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.385+0000 7f2a977fe640 1 -- 172.21.15.154:0/1267485289 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9002ca80 con 0x7f2a980c9190 2023-09-23T03:28:00.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.385+0000 7f2a977fe640 1 -- 172.21.15.154:0/1267485289 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a9002cbe0 con 0x7f2a980c9190 2023-09-23T03:28:00.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.385+0000 7f2a977fe640 1 -- 172.21.15.154:0/1267485289 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a900343f0 con 0x7f2a980c9190 2023-09-23T03:28:00.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.386+0000 7f2a9d613640 1 -- 172.21.15.154:0/1267485289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2a980c9190 msgr2=0x7f2a980ce380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:00.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.386+0000 7f2a9d613640 1 --2- 172.21.15.154:0/1267485289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2a980c9190 0x7f2a980ce380 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f2a980cbf30 tx=0x7f2a9002bee0 comp rx=0 tx=0).stop 2023-09-23T03:28:00.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.386+0000 7f2a9d613640 1 -- 172.21.15.154:0/1267485289 shutdown_connections 2023-09-23T03:28:00.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.386+0000 7f2a9d613640 1 --2- 172.21.15.154:0/1267485289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2a980c9190 0x7f2a980ce380 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:00.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.386+0000 7f2a9d613640 1 --2- 172.21.15.154:0/1267485289 >> v2:172.21.15.190:3300/0 conn(0x7f2a980711a0 0x7f2a980c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:00.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.386+0000 7f2a9d613640 1 --2- 172.21.15.154:0/1267485289 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a9815cb00 0x7f2a98070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:00.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.386+0000 7f2a9d613640 1 -- 172.21.15.154:0/1267485289 >> 172.21.15.154:0/1267485289 conn(0x7f2a9806a300 msgr2=0x7f2a9806bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:00.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.386+0000 7f2a9d613640 1 -- 172.21.15.154:0/1267485289 shutdown_connections 2023-09-23T03:28:00.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.386+0000 7f2a9d613640 1 -- 172.21.15.154:0/1267485289 wait complete. 2023-09-23T03:28:00.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.388+0000 7f2a9d613640 1 Processor -- start 2023-09-23T03:28:00.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.388+0000 7f2a9d613640 1 -- start start 2023-09-23T03:28:00.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.389+0000 7f2a9d613640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a980c9190 0x7f2a980ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:00.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.389+0000 7f2a9d613640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2a9815cb00 0x7f2a980c54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:00.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.389+0000 7f2a9d613640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a980c5a00 0x7f2a980c4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:00.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.389+0000 7f2a9d613640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9815f5a0 con 0x7f2a9815cb00 2023-09-23T03:28:00.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.389+0000 7f2a9d613640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9815ea20 con 0x7f2a980c9190 2023-09-23T03:28:00.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.389+0000 7f2a9d613640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9815ed00 con 0x7f2a980c5a00 2023-09-23T03:28:00.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.389+0000 7f2a8ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2a9815cb00 0x7f2a980c54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:00.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.389+0000 7f2a8ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2a9815cb00 0x7f2a980c54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49890/0 (socket says 172.21.15.154:49890) 2023-09-23T03:28:00.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.389+0000 7f2a8ffff640 1 -- 172.21.15.154:0/44356940 learned_addr learned my addr 172.21.15.154:0/44356940 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:00.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.390+0000 7f2a97fff640 1 --2- 172.21.15.154:0/44356940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a980c9190 0x7f2a980ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:00.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.390+0000 7f2a8ffff640 1 -- 172.21.15.154:0/44356940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a980c5a00 msgr2=0x7f2a980c4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:00.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.390+0000 7f2a8ffff640 1 --2- 172.21.15.154:0/44356940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a980c5a00 0x7f2a980c4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:00.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.390+0000 7f2a8ffff640 1 -- 172.21.15.154:0/44356940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a980c9190 msgr2=0x7f2a980ce5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:00.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.390+0000 7f2a8ffff640 1 --2- 172.21.15.154:0/44356940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a980c9190 0x7f2a980ce5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:00.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.390+0000 7f2a8ffff640 1 -- 172.21.15.154:0/44356940 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a90009b30 con 0x7f2a9815cb00 2023-09-23T03:28:00.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.390+0000 7f2a8ffff640 1 --2- 172.21.15.154:0/44356940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2a9815cb00 0x7f2a980c54c0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f2a8000d5f0 tx=0x7f2a8000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:00.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.390+0000 7f2a957fa640 1 -- 172.21.15.154:0/44356940 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a800125f0 con 0x7f2a9815cb00 2023-09-23T03:28:00.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.390+0000 7f2a957fa640 1 -- 172.21.15.154:0/44356940 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a80012750 con 0x7f2a9815cb00 2023-09-23T03:28:00.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.391+0000 7f2a957fa640 1 -- 172.21.15.154:0/44356940 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a80012b00 con 0x7f2a9815cb00 2023-09-23T03:28:00.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.391+0000 7f2a9d613640 1 -- 172.21.15.154:0/44356940 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a9815efe0 con 0x7f2a9815cb00 2023-09-23T03:28:00.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.391+0000 7f2a9d613640 1 -- 172.21.15.154:0/44356940 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a980cc090 con 0x7f2a9815cb00 2023-09-23T03:28:00.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.392+0000 7f2a9d613640 1 -- 172.21.15.154:0/44356940 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a980c82c0 con 0x7f2a9815cb00 2023-09-23T03:28:00.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.393+0000 7f2a957fa640 1 -- 172.21.15.154:0/44356940 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f2a80012c60 con 0x7f2a9815cb00 2023-09-23T03:28:00.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.394+0000 7f2a957fa640 1 --2- 172.21.15.154:0/44356940 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2a7806af30 0x7f2a7806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:00.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.394+0000 7f2a957fa640 1 -- 172.21.15.154:0/44356940 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2a80089d00 con 0x7f2a9815cb00 2023-09-23T03:28:00.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.394+0000 7f2a97fff640 1 --2- 172.21.15.154:0/44356940 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2a7806af30 0x7f2a7806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:00.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.395+0000 7f2a97fff640 1 --2- 172.21.15.154:0/44356940 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2a7806af30 0x7f2a7806d3f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2a8800b7e0 tx=0x7f2a88008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:00.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.398+0000 7f2a957fa640 1 -- 172.21.15.154:0/44356940 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f2a80054c90 con 0x7f2a9815cb00 2023-09-23T03:28:00.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.588+0000 7f2a9d613640 1 -- 172.21.15.154:0/44356940 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2a98073eb0 con 0x7f2a9815cb00 2023-09-23T03:28:00.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.589+0000 7f2a957fa640 1 -- 172.21.15.154:0/44356940 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v126) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2a800582b0 con 0x7f2a9815cb00 2023-09-23T03:28:00.591 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:28:00.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.592+0000 7f2a9d613640 1 -- 172.21.15.154:0/44356940 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2a7806af30 msgr2=0x7f2a7806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:00.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.592+0000 7f2a9d613640 1 --2- 172.21.15.154:0/44356940 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2a7806af30 0x7f2a7806d3f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2a8800b7e0 tx=0x7f2a88008040 comp rx=0 tx=0).stop 2023-09-23T03:28:00.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.592+0000 7f2a9d613640 1 -- 172.21.15.154:0/44356940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2a9815cb00 msgr2=0x7f2a980c54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:00.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.592+0000 7f2a9d613640 1 --2- 172.21.15.154:0/44356940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2a9815cb00 0x7f2a980c54c0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f2a8000d5f0 tx=0x7f2a8000dab0 comp rx=0 tx=0).stop 2023-09-23T03:28:00.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.592+0000 7f2a9d613640 1 -- 172.21.15.154:0/44356940 shutdown_connections 2023-09-23T03:28:00.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.592+0000 7f2a9d613640 1 --2- 172.21.15.154:0/44356940 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2a7806af30 0x7f2a7806d3f0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:00.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.592+0000 7f2a9d613640 1 --2- 172.21.15.154:0/44356940 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2a980c5a00 0x7f2a980c4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:00.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.592+0000 7f2a9d613640 1 --2- 172.21.15.154:0/44356940 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2a9815cb00 0x7f2a980c54c0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:00.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.592+0000 7f2a9d613640 1 --2- 172.21.15.154:0/44356940 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2a980c9190 0x7f2a980ce5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:00.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.592+0000 7f2a9d613640 1 -- 172.21.15.154:0/44356940 >> 172.21.15.154:0/44356940 conn(0x7f2a9806a300 msgr2=0x7f2a9815f990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:00.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.593+0000 7f2a9d613640 1 -- 172.21.15.154:0/44356940 shutdown_connections 2023-09-23T03:28:00.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.593+0000 7f2a9d613640 1 -- 172.21.15.154:0/44356940 wait complete. 2023-09-23T03:28:00.613 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:28:00.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.812+0000 7fcb1f739640 1 Processor -- start 2023-09-23T03:28:00.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.813+0000 7fcb1f739640 1 -- start start 2023-09-23T03:28:00.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.814+0000 7fcb1f739640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb180caf80 0x7fcb180cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:00.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.814+0000 7fcb1f739640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcb1806c320 0x7fcb1806c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:00.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.814+0000 7fcb1f739640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb180cb920 con 0x7fcb18144130 2023-09-23T03:28:00.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.814+0000 7fcb1f739640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb18070fb0 con 0x7fcb1806c320 2023-09-23T03:28:00.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.814+0000 7fcb1f739640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb180b2280 con 0x7fcb180caf80 2023-09-23T03:28:00.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.815+0000 7fcb1d4d5640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fcb18144130 0x7fcb180b1b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46658/0 (socket says 172.21.15.154:46658) 2023-09-23T03:28:00.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.815+0000 7fcb1d4d5640 1 -- 172.21.15.154:0/1142051189 learned_addr learned my addr 172.21.15.154:0/1142051189 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:28:00.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.815+0000 7fcb0ffff640 1 -- 172.21.15.154:0/1142051189 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2709698922 0 0) 0x7fcb180cb920 con 0x7fcb18144130 2023-09-23T03:28:00.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.815+0000 7fcb1dcd6640 1 --2- 172.21.15.154:0/1142051189 >> v2:172.21.15.190:3300/0 conn(0x7fcb1806c320 0x7fcb1806c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:00.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.815+0000 7fcb0ffff640 1 -- 172.21.15.154:0/1142051189 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb04003520 con 0x7fcb18144130 2023-09-23T03:28:00.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.815+0000 7fcb1ccd4640 1 --2- 172.21.15.154:0/1142051189 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb180caf80 0x7fcb180cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:00.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.815+0000 7fcb0ffff640 1 -- 172.21.15.154:0/1142051189 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1809525097 0 0) 0x7fcb04003520 con 0x7fcb18144130 2023-09-23T03:28:00.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.815+0000 7fcb0ffff640 1 -- 172.21.15.154:0/1142051189 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb180caf80 msgr2=0x7fcb180cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:00.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.815+0000 7fcb0ffff640 1 --2- 172.21.15.154:0/1142051189 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb180caf80 0x7fcb180cb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:00.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.815+0000 7fcb0ffff640 1 -- 172.21.15.154:0/1142051189 >> v2:172.21.15.190:3300/0 conn(0x7fcb1806c320 msgr2=0x7fcb1806c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:00.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.815+0000 7fcb0ffff640 1 --2- 172.21.15.154:0/1142051189 >> v2:172.21.15.190:3300/0 conn(0x7fcb1806c320 0x7fcb1806c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:00.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.815+0000 7fcb0ffff640 1 -- 172.21.15.154:0/1142051189 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb180700e0 con 0x7fcb18144130 2023-09-23T03:28:00.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.815+0000 7fcb0ffff640 1 -- 172.21.15.154:0/1142051189 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fcb14001670 con 0x7fcb18144130 2023-09-23T03:28:00.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.816+0000 7fcb1dcd6640 1 --2- 172.21.15.154:0/1142051189 >> v2:172.21.15.190:3300/0 conn(0x7fcb1806c320 0x7fcb1806c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:28:00.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.816+0000 7fcb0ffff640 1 -- 172.21.15.154:0/1142051189 >> v1:172.21.15.154:6789/0 conn(0x7fcb18144130 legacy=0x7fcb180b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:00.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.816+0000 7fcb0ffff640 1 --2- 172.21.15.154:0/1142051189 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb180caf80 0x7fcb04003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:00.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.816+0000 7fcb0ffff640 1 --2- 172.21.15.154:0/1142051189 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb04003fa0 0x7fcb040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:00.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.816+0000 7fcb0ffff640 1 --2- 172.21.15.154:0/1142051189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb04004d00 0x7fcb04005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:00.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.816+0000 7fcb0ffff640 1 -- 172.21.15.154:0/1142051189 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb18070fb0 con 0x7fcb04003fa0 2023-09-23T03:28:00.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.816+0000 7fcb0ffff640 1 -- 172.21.15.154:0/1142051189 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb180b2280 con 0x7fcb04004d00 2023-09-23T03:28:00.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.816+0000 7fcb0ffff640 1 -- 172.21.15.154:0/1142051189 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0400ba80 con 0x7fcb180caf80 2023-09-23T03:28:00.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.816+0000 7fcb1ccd4640 1 --2- 172.21.15.154:0/1142051189 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb04003fa0 0x7fcb040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:00.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.816+0000 7fcb1d4d5640 1 --2- 172.21.15.154:0/1142051189 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb180caf80 0x7fcb04003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:00.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.816+0000 7fcb1dcd6640 1 --2- 172.21.15.154:0/1142051189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb04004d00 0x7fcb04005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:00.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.816+0000 7fcb1ccd4640 1 -- 172.21.15.154:0/1142051189 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb180caf80 msgr2=0x7fcb04003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:00.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.816+0000 7fcb1ccd4640 1 --2- 172.21.15.154:0/1142051189 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb180caf80 0x7fcb04003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:00.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.816+0000 7fcb1ccd4640 1 -- 172.21.15.154:0/1142051189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb04004d00 msgr2=0x7fcb04005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:00.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.816+0000 7fcb1ccd4640 1 --2- 172.21.15.154:0/1142051189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb04004d00 0x7fcb04005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:00.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.816+0000 7fcb1ccd4640 1 -- 172.21.15.154:0/1142051189 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb04003520 con 0x7fcb04003fa0 2023-09-23T03:28:00.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.816+0000 7fcb1dcd6640 1 --2- 172.21.15.154:0/1142051189 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb04004d00 0x7fcb04005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:00.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.817+0000 7fcb1ccd4640 1 --2- 172.21.15.154:0/1142051189 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb04003fa0 0x7fcb040047c0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800ba30 tx=0x7fcb040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26314550c0b6e53e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:00.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.818+0000 7fcb0ffff640 1 -- 172.21.15.154:0/1142051189 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb08032640 con 0x7fcb04003fa0 2023-09-23T03:28:00.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.818+0000 7fcb0ffff640 1 -- 172.21.15.154:0/1142051189 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb080327a0 con 0x7fcb04003fa0 2023-09-23T03:28:00.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.818+0000 7fcb0ffff640 1 -- 172.21.15.154:0/1142051189 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb08032b30 con 0x7fcb04003fa0 2023-09-23T03:28:00.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.818+0000 7fcb1f739640 1 -- 172.21.15.154:0/1142051189 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb04003fa0 msgr2=0x7fcb040047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:00.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.818+0000 7fcb1f739640 1 --2- 172.21.15.154:0/1142051189 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb04003fa0 0x7fcb040047c0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800ba30 tx=0x7fcb040057f0 comp rx=0 tx=0).stop 2023-09-23T03:28:00.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.818+0000 7fcb1dcd6640 1 -- 172.21.15.154:0/1142051189 reap_dead start 2023-09-23T03:28:00.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.818+0000 7fcb1f739640 1 -- 172.21.15.154:0/1142051189 shutdown_connections 2023-09-23T03:28:00.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.818+0000 7fcb1f739640 1 -- 172.21.15.154:0/1142051189 >> 172.21.15.154:0/1142051189 conn(0x7fcb1806a440 msgr2=0x7fcb180b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:00.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.818+0000 7fcb1f739640 1 -- 172.21.15.154:0/1142051189 shutdown_connections 2023-09-23T03:28:00.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.818+0000 7fcb1f739640 1 -- 172.21.15.154:0/1142051189 wait complete. 2023-09-23T03:28:00.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.820+0000 7fcb1f739640 1 Processor -- start 2023-09-23T03:28:00.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.820+0000 7fcb1f739640 1 -- start start 2023-09-23T03:28:00.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.821+0000 7fcb1f739640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb04003fa0 0x7fcb18159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:00.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.821+0000 7fcb1f739640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb180c7a00 0x7fcb180c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:00.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.821+0000 7fcb1f739640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb180a9b20 0x7fcb180a9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:00.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.821+0000 7fcb1f739640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb180aa4f0 con 0x7fcb180a9b20 2023-09-23T03:28:00.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.821+0000 7fcb1f739640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb180a77b0 con 0x7fcb04003fa0 2023-09-23T03:28:00.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.821+0000 7fcb1f739640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb180a7a90 con 0x7fcb180c7a00 2023-09-23T03:28:00.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.821+0000 7fcb1dcd6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb180a9b20 0x7fcb180a9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:00.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.821+0000 7fcb1dcd6640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb180a9b20 0x7fcb180a9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49904/0 (socket says 172.21.15.154:49904) 2023-09-23T03:28:00.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.821+0000 7fcb1dcd6640 1 -- 172.21.15.154:0/3971855142 learned_addr learned my addr 172.21.15.154:0/3971855142 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:00.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.822+0000 7fcb1dcd6640 1 -- 172.21.15.154:0/3971855142 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb180c7a00 msgr2=0x7fcb180c7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:00.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.822+0000 7fcb1dcd6640 1 --2- 172.21.15.154:0/3971855142 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb180c7a00 0x7fcb180c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:00.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.822+0000 7fcb1dcd6640 1 -- 172.21.15.154:0/3971855142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb04003fa0 msgr2=0x7fcb18159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:00.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.822+0000 7fcb1dcd6640 1 --2- 172.21.15.154:0/3971855142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb04003fa0 0x7fcb18159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:00.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.822+0000 7fcb1dcd6640 1 -- 172.21.15.154:0/3971855142 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb180a7d70 con 0x7fcb180a9b20 2023-09-23T03:28:00.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.822+0000 7fcb1dcd6640 1 --2- 172.21.15.154:0/3971855142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb180a9b20 0x7fcb180a9fb0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fcb180caed0 tx=0x7fcb10004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:00.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.822+0000 7fcb0dffb640 1 -- 172.21.15.154:0/3971855142 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb10007cb0 con 0x7fcb180a9b20 2023-09-23T03:28:00.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.822+0000 7fcb0dffb640 1 -- 172.21.15.154:0/3971855142 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb10007e10 con 0x7fcb180a9b20 2023-09-23T03:28:00.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.822+0000 7fcb0dffb640 1 -- 172.21.15.154:0/3971855142 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb10014640 con 0x7fcb180a9b20 2023-09-23T03:28:00.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.822+0000 7fcb1f739640 1 -- 172.21.15.154:0/3971855142 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb180a8050 con 0x7fcb180a9b20 2023-09-23T03:28:00.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.823+0000 7fcb1f739640 1 -- 172.21.15.154:0/3971855142 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb1815a520 con 0x7fcb180a9b20 2023-09-23T03:28:00.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.823+0000 7fcb1f739640 1 -- 172.21.15.154:0/3971855142 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb180a8570 con 0x7fcb180a9b20 2023-09-23T03:28:00.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.824+0000 7fcb0dffb640 1 -- 172.21.15.154:0/3971855142 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fcb1001d3b0 con 0x7fcb180a9b20 2023-09-23T03:28:00.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.825+0000 7fcb0dffb640 1 --2- 172.21.15.154:0/3971855142 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcafc06af60 0x7fcafc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:00.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.825+0000 7fcb0dffb640 1 -- 172.21.15.154:0/3971855142 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fcb10089720 con 0x7fcb180a9b20 2023-09-23T03:28:00.828 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.826+0000 7fcb1d4d5640 1 --2- 172.21.15.154:0/3971855142 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcafc06af60 0x7fcafc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:00.828 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.826+0000 7fcb1d4d5640 1 --2- 172.21.15.154:0/3971855142 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcafc06af60 0x7fcafc06d420 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fcb14000ff0 tx=0x7fcb14009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:00.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:00.830+0000 7fcb0dffb640 1 -- 172.21.15.154:0/3971855142 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fcb180a8570 con 0x7fcb180a9b20 2023-09-23T03:28:01.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.021+0000 7fcb1f739640 1 -- 172.21.15.154:0/3971855142 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcb180a8570 con 0x7fcb180a9b20 2023-09-23T03:28:01.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.021+0000 7fcb0dffb640 1 -- 172.21.15.154:0/3971855142 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v126) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcb180a8570 con 0x7fcb180a9b20 2023-09-23T03:28:01.023 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:28:01.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.024+0000 7fcb1f739640 1 -- 172.21.15.154:0/3971855142 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcafc06af60 msgr2=0x7fcafc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:01.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.024+0000 7fcb1f739640 1 --2- 172.21.15.154:0/3971855142 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcafc06af60 0x7fcafc06d420 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fcb14000ff0 tx=0x7fcb14009040 comp rx=0 tx=0).stop 2023-09-23T03:28:01.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.024+0000 7fcb1f739640 1 -- 172.21.15.154:0/3971855142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb180a9b20 msgr2=0x7fcb180a9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:01.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.024+0000 7fcb1f739640 1 --2- 172.21.15.154:0/3971855142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb180a9b20 0x7fcb180a9fb0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fcb180caed0 tx=0x7fcb10004840 comp rx=0 tx=0).stop 2023-09-23T03:28:01.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.024+0000 7fcb1f739640 1 -- 172.21.15.154:0/3971855142 shutdown_connections 2023-09-23T03:28:01.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.024+0000 7fcb1f739640 1 --2- 172.21.15.154:0/3971855142 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcafc06af60 0x7fcafc06d420 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:01.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.024+0000 7fcb1f739640 1 --2- 172.21.15.154:0/3971855142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb180a9b20 0x7fcb180a9fb0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:01.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.024+0000 7fcb1f739640 1 --2- 172.21.15.154:0/3971855142 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcb180c7a00 0x7fcb180c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:01.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.024+0000 7fcb1f739640 1 --2- 172.21.15.154:0/3971855142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcb04003fa0 0x7fcb18159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:01.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.024+0000 7fcb1f739640 1 -- 172.21.15.154:0/3971855142 >> 172.21.15.154:0/3971855142 conn(0x7fcb1806a440 msgr2=0x7fcb1815ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:01.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.025+0000 7fcb1f739640 1 -- 172.21.15.154:0/3971855142 shutdown_connections 2023-09-23T03:28:01.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.025+0000 7fcb1f739640 1 -- 172.21.15.154:0/3971855142 wait complete. 2023-09-23T03:28:01.045 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-23T03:28:01.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.249+0000 7f9907f61640 1 Processor -- start 2023-09-23T03:28:01.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.249+0000 7f9907f61640 1 -- start start 2023-09-23T03:28:01.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.251+0000 7f9907f61640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99000cae40 0x7f99000b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:01.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.251+0000 7f9907f61640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f99000b2700 0x7f990006b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:01.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.251+0000 7f9907f61640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f990006cdd0 con 0x7f990015cc80 2023-09-23T03:28:01.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.251+0000 7f9907f61640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f990015d060 con 0x7f99000b2700 2023-09-23T03:28:01.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.251+0000 7f9907f61640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99000b2b90 con 0x7f99000cae40 2023-09-23T03:28:01.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.251+0000 7f9905cfd640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f990015cc80 0x7f990006c6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:46670/0 (socket says 172.21.15.154:46670) 2023-09-23T03:28:01.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.251+0000 7f9905cfd640 1 -- 172.21.15.154:0/4257657864 learned_addr learned my addr 172.21.15.154:0/4257657864 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:28:01.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.251+0000 7f99054fc640 1 --2- 172.21.15.154:0/4257657864 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99000cae40 0x7f99000b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:01.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.251+0000 7f99064fe640 1 --2- 172.21.15.154:0/4257657864 >> v2:172.21.15.190:3300/0 conn(0x7f99000b2700 0x7f990006b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:01.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.251+0000 7f9904cfb640 1 -- 172.21.15.154:0/4257657864 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2646001634 0 0) 0x7f990006cdd0 con 0x7f990015cc80 2023-09-23T03:28:01.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.251+0000 7f9904cfb640 1 -- 172.21.15.154:0/4257657864 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98e8003520 con 0x7f990015cc80 2023-09-23T03:28:01.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.252+0000 7f9904cfb640 1 -- 172.21.15.154:0/4257657864 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 295718353 0 0) 0x7f98e8003520 con 0x7f990015cc80 2023-09-23T03:28:01.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.252+0000 7f9904cfb640 1 -- 172.21.15.154:0/4257657864 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99000cae40 msgr2=0x7f99000b21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:01.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.252+0000 7f9904cfb640 1 --2- 172.21.15.154:0/4257657864 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99000cae40 0x7f99000b21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:01.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.252+0000 7f9904cfb640 1 -- 172.21.15.154:0/4257657864 >> v2:172.21.15.190:3300/0 conn(0x7f99000b2700 msgr2=0x7f990006b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:01.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.252+0000 7f9904cfb640 1 --2- 172.21.15.154:0/4257657864 >> v2:172.21.15.190:3300/0 conn(0x7f99000b2700 0x7f990006b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:01.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.252+0000 7f9904cfb640 1 -- 172.21.15.154:0/4257657864 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99000ce440 con 0x7f990015cc80 2023-09-23T03:28:01.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.252+0000 7f9904cfb640 1 -- 172.21.15.154:0/4257657864 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f98f00019f0 con 0x7f990015cc80 2023-09-23T03:28:01.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.252+0000 7f9904cfb640 1 -- 172.21.15.154:0/4257657864 >> v1:172.21.15.154:6789/0 conn(0x7f990015cc80 legacy=0x7f990006c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:01.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.252+0000 7f9904cfb640 1 --2- 172.21.15.154:0/4257657864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98e8003a10 0x7f98e8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:01.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.252+0000 7f9904cfb640 1 --2- 172.21.15.154:0/4257657864 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98e80047c0 0x7f98e8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:01.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.252+0000 7f9904cfb640 1 --2- 172.21.15.154:0/4257657864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98e8005180 0x7f98e800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:01.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.252+0000 7f9904cfb640 1 -- 172.21.15.154:0/4257657864 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99000b2b90 con 0x7f98e8005180 2023-09-23T03:28:01.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.252+0000 7f9904cfb640 1 -- 172.21.15.154:0/4257657864 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e800bf20 con 0x7f98e8003a10 2023-09-23T03:28:01.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.252+0000 7f9904cfb640 1 -- 172.21.15.154:0/4257657864 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e800c0e0 con 0x7f98e80047c0 2023-09-23T03:28:01.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.252+0000 7f99054fc640 1 --2- 172.21.15.154:0/4257657864 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f99000cae40 0x7f99000b21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:28:01.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.252+0000 7f99064fe640 1 --2- 172.21.15.154:0/4257657864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98e8005180 0x7f98e800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:01.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.253+0000 7f9905cfd640 1 --2- 172.21.15.154:0/4257657864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98e8003a10 0x7f98e8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:01.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.253+0000 7f99064fe640 1 -- 172.21.15.154:0/4257657864 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98e80047c0 msgr2=0x7f98e8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:01.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.253+0000 7f99064fe640 1 --2- 172.21.15.154:0/4257657864 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98e80047c0 0x7f98e8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:01.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.253+0000 7f99064fe640 1 -- 172.21.15.154:0/4257657864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98e8003a10 msgr2=0x7f98e8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:01.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.253+0000 7f99064fe640 1 --2- 172.21.15.154:0/4257657864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98e8003a10 0x7f98e8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:01.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.253+0000 7f99064fe640 1 -- 172.21.15.154:0/4257657864 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e8003520 con 0x7f98e8005180 2023-09-23T03:28:01.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.253+0000 7f9905cfd640 1 --2- 172.21.15.154:0/4257657864 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f98e8003a10 0x7f98e8003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:01.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.254+0000 7f99064fe640 1 --2- 172.21.15.154:0/4257657864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98e8005180 0x7f98e800b980 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f98f8009ac0 tx=0x7f98e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a3048f6c74bd8ea server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:01.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.254+0000 7f9904cfb640 1 -- 172.21.15.154:0/4257657864 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f802c710 con 0x7f98e8005180 2023-09-23T03:28:01.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.254+0000 7f9904cfb640 1 -- 172.21.15.154:0/4257657864 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f802c870 con 0x7f98e8005180 2023-09-23T03:28:01.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.254+0000 7f9904cfb640 1 -- 172.21.15.154:0/4257657864 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f802cc00 con 0x7f98e8005180 2023-09-23T03:28:01.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.255+0000 7f9907f61640 1 -- 172.21.15.154:0/4257657864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98e8005180 msgr2=0x7f98e800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:01.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.255+0000 7f9907f61640 1 --2- 172.21.15.154:0/4257657864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f98e8005180 0x7f98e800b980 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f98f8009ac0 tx=0x7f98e80057f0 comp rx=0 tx=0).stop 2023-09-23T03:28:01.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.255+0000 7f99064fe640 1 -- 172.21.15.154:0/4257657864 reap_dead start 2023-09-23T03:28:01.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.255+0000 7f9907f61640 1 -- 172.21.15.154:0/4257657864 shutdown_connections 2023-09-23T03:28:01.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.255+0000 7f9907f61640 1 -- 172.21.15.154:0/4257657864 >> 172.21.15.154:0/4257657864 conn(0x7f99000cd1c0 msgr2=0x7f99000cd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:01.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.255+0000 7f9907f61640 1 -- 172.21.15.154:0/4257657864 shutdown_connections 2023-09-23T03:28:01.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.255+0000 7f9907f61640 1 -- 172.21.15.154:0/4257657864 wait complete. 2023-09-23T03:28:01.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.257+0000 7f9907f61640 1 Processor -- start 2023-09-23T03:28:01.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.257+0000 7f9907f61640 1 -- start start 2023-09-23T03:28:01.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.258+0000 7f9907f61640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98e8005180 0x7f99000c6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:01.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.258+0000 7f9907f61640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99000c4c60 0x7f990014ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:01.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.258+0000 7f9907f61640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99000c8e90 0x7f99000c9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:01.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.258+0000 7f9907f61640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99000c9850 con 0x7f99000c4c60 2023-09-23T03:28:01.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.258+0000 7f9907f61640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990014f200 con 0x7f99000c8e90 2023-09-23T03:28:01.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.258+0000 7f9907f61640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99000c50a0 con 0x7f98e8005180 2023-09-23T03:28:01.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.258+0000 7f99054fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99000c4c60 0x7f990014ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:01.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.258+0000 7f99054fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99000c4c60 0x7f990014ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49914/0 (socket says 172.21.15.154:49914) 2023-09-23T03:28:01.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.258+0000 7f99054fc640 1 -- 172.21.15.154:0/1972266905 learned_addr learned my addr 172.21.15.154:0/1972266905 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:01.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.258+0000 7f99054fc640 1 -- 172.21.15.154:0/1972266905 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98e8005180 msgr2=0x7f99000c6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:01.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.258+0000 7f99054fc640 1 --2- 172.21.15.154:0/1972266905 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98e8005180 0x7f99000c6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:01.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.258+0000 7f99054fc640 1 -- 172.21.15.154:0/1972266905 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99000c8e90 msgr2=0x7f99000c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:01.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.258+0000 7f99054fc640 1 --2- 172.21.15.154:0/1972266905 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99000c8e90 0x7f99000c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:01.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.258+0000 7f99054fc640 1 -- 172.21.15.154:0/1972266905 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99000c5320 con 0x7f99000c4c60 2023-09-23T03:28:01.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.259+0000 7f99054fc640 1 --2- 172.21.15.154:0/1972266905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99000c4c60 0x7f990014ecc0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f98f400e560 tx=0x7f98f400ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:01.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.259+0000 7f98fe7fc640 1 -- 172.21.15.154:0/1972266905 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f4004230 con 0x7f99000c4c60 2023-09-23T03:28:01.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.259+0000 7f98fe7fc640 1 -- 172.21.15.154:0/1972266905 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f40043b0 con 0x7f99000c4c60 2023-09-23T03:28:01.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.259+0000 7f98fe7fc640 1 -- 172.21.15.154:0/1972266905 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f4004520 con 0x7f99000c4c60 2023-09-23T03:28:01.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.259+0000 7f9907f61640 1 -- 172.21.15.154:0/1972266905 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99000c5600 con 0x7f99000c4c60 2023-09-23T03:28:01.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.259+0000 7f9907f61640 1 -- 172.21.15.154:0/1972266905 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99000c5bc0 con 0x7f99000c4c60 2023-09-23T03:28:01.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.260+0000 7f9907f61640 1 -- 172.21.15.154:0/1972266905 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9900073eb0 con 0x7f99000c4c60 2023-09-23T03:28:01.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.261+0000 7f98fe7fc640 1 -- 172.21.15.154:0/1972266905 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f98f4003850 con 0x7f99000c4c60 2023-09-23T03:28:01.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.262+0000 7f98fe7fc640 1 --2- 172.21.15.154:0/1972266905 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f98ec06af30 0x7f98ec06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:01.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.262+0000 7f98fe7fc640 1 -- 172.21.15.154:0/1972266905 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f98f40893b0 con 0x7f99000c4c60 2023-09-23T03:28:01.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.262+0000 7f9905cfd640 1 --2- 172.21.15.154:0/1972266905 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f98ec06af30 0x7f98ec06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:01.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.263+0000 7f9905cfd640 1 --2- 172.21.15.154:0/1972266905 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f98ec06af30 0x7f98ec06d3f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f98f0001b20 tx=0x7f98f0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:01.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.266+0000 7f98fe7fc640 1 -- 172.21.15.154:0/1972266905 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f98f40542a0 con 0x7f99000c4c60 2023-09-23T03:28:01.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.469+0000 7f9907f61640 1 -- 172.21.15.154:0/1972266905 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9900152300 con 0x7f99000c4c60 2023-09-23T03:28:01.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.470+0000 7f98fe7fc640 1 -- 172.21.15.154:0/1972266905 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v126) v1 ==== 80+0+0 (secure 0 0 0) 0x7f98f40578c0 con 0x7f99000c4c60 2023-09-23T03:28:01.471 INFO:teuthology.orchestra.run.smithi154.stderr:listed 0 entries 2023-09-23T03:28:01.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.472+0000 7f9907f61640 1 -- 172.21.15.154:0/1972266905 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f98ec06af30 msgr2=0x7f98ec06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:01.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.472+0000 7f9907f61640 1 --2- 172.21.15.154:0/1972266905 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f98ec06af30 0x7f98ec06d3f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f98f0001b20 tx=0x7f98f0009040 comp rx=0 tx=0).stop 2023-09-23T03:28:01.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.472+0000 7f9907f61640 1 -- 172.21.15.154:0/1972266905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99000c4c60 msgr2=0x7f990014ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:01.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.472+0000 7f9907f61640 1 --2- 172.21.15.154:0/1972266905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99000c4c60 0x7f990014ecc0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f98f400e560 tx=0x7f98f400ea20 comp rx=0 tx=0).stop 2023-09-23T03:28:01.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.473+0000 7f9907f61640 1 -- 172.21.15.154:0/1972266905 shutdown_connections 2023-09-23T03:28:01.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.473+0000 7f9907f61640 1 --2- 172.21.15.154:0/1972266905 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f98ec06af30 0x7f98ec06d3f0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:01.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.473+0000 7f9907f61640 1 --2- 172.21.15.154:0/1972266905 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f99000c8e90 0x7f99000c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:01.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.473+0000 7f9907f61640 1 --2- 172.21.15.154:0/1972266905 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99000c4c60 0x7f990014ecc0 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:01.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.473+0000 7f9907f61640 1 --2- 172.21.15.154:0/1972266905 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f98e8005180 0x7f99000c6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:01.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.473+0000 7f9907f61640 1 -- 172.21.15.154:0/1972266905 >> 172.21.15.154:0/1972266905 conn(0x7f99000cd1c0 msgr2=0x7f99000b0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:01.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.473+0000 7f9907f61640 1 -- 172.21.15.154:0/1972266905 shutdown_connections 2023-09-23T03:28:01.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:01.473+0000 7f9907f61640 1 -- 172.21.15.154:0/1972266905 wait complete. 2023-09-23T03:28:01.492 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi154.front.sepia.ceph.com... 2023-09-23T03:28:01.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:01.492 DEBUG:teuthology.orchestra.run.smithi154:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-09-23T03:28:01.585 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:ceph-fuse[127122]: fuse finished with error 0 and tester_r 0 2023-09-23T03:28:01.591 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:28:07.692 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2023-09-23T03:28:07.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:07.693 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-09-23T03:28:07.756 DEBUG:teuthology.orchestra.run.smithi190:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2023-09-23T03:28:08.029 INFO:teuthology.orchestra.run.smithi190.stdout:{"message":"","return_code":0} 2023-09-23T03:28:08.029 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-09-23T03:28:08.029 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-09-23T03:28:08.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.222+0000 7fa606a9a640 1 Processor -- start 2023-09-23T03:28:08.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.222+0000 7fa606a9a640 1 -- start start 2023-09-23T03:28:08.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.224+0000 7fa606a9a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa60015e2d0 0x7fa60006c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:08.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.224+0000 7fa606a9a640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6000b4270 con 0x7fa60015b310 2023-09-23T03:28:08.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.224+0000 7fa606a9a640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa60015f150 con 0x7fa600158200 2023-09-23T03:28:08.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.224+0000 7fa606a9a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa60015e8b0 con 0x7fa60015e2d0 2023-09-23T03:28:08.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.224+0000 7fa605297640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa60015e2d0 0x7fa60006c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:08.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.224+0000 7fa606299640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fa60015b310 0x7fa60006fe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:42828/0 (socket says 172.21.15.154:42828) 2023-09-23T03:28:08.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.224+0000 7fa606299640 1 -- 172.21.15.154:0/2033211076 learned_addr learned my addr 172.21.15.154:0/2033211076 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:28:08.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.224+0000 7fa605297640 1 -- 172.21.15.154:0/2033211076 >> v1:172.21.15.190:6789/0 conn(0x7fa600158200 legacy=0x7fa6000c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:08.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.224+0000 7fa605297640 1 -- 172.21.15.154:0/2033211076 >> v1:172.21.15.154:6789/0 conn(0x7fa60015b310 legacy=0x7fa60006fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:08.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.225+0000 7fa605297640 1 -- 172.21.15.154:0/2033211076 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa60015eb90 con 0x7fa60015e2d0 2023-09-23T03:28:08.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.226+0000 7fa605297640 1 --2- 172.21.15.154:0/2033211076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa60015e2d0 0x7fa60006c820 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fa5f00097a0 tx=0x7fa5f002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f10a8b3a132a7b server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:08.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.226+0000 7fa604a96640 1 -- 172.21.15.154:0/2033211076 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0031830 con 0x7fa60015e2d0 2023-09-23T03:28:08.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.226+0000 7fa604a96640 1 -- 172.21.15.154:0/2033211076 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f0031990 con 0x7fa60015e2d0 2023-09-23T03:28:08.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.226+0000 7fa604a96640 1 -- 172.21.15.154:0/2033211076 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0031d20 con 0x7fa60015e2d0 2023-09-23T03:28:08.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.227+0000 7fa606a9a640 1 -- 172.21.15.154:0/2033211076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa60015e2d0 msgr2=0x7fa60006c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:08.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.227+0000 7fa606a9a640 1 --2- 172.21.15.154:0/2033211076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa60015e2d0 0x7fa60006c820 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fa5f00097a0 tx=0x7fa5f002b8d0 comp rx=0 tx=0).stop 2023-09-23T03:28:08.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.227+0000 7fa606a9a640 1 -- 172.21.15.154:0/2033211076 shutdown_connections 2023-09-23T03:28:08.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.227+0000 7fa606a9a640 1 --2- 172.21.15.154:0/2033211076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa60015e2d0 0x7fa60006c820 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:08.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.227+0000 7fa606a9a640 1 -- 172.21.15.154:0/2033211076 >> 172.21.15.154:0/2033211076 conn(0x7fa60006a4c0 msgr2=0x7fa6000afe50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:08.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.227+0000 7fa606a9a640 1 -- 172.21.15.154:0/2033211076 shutdown_connections 2023-09-23T03:28:08.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.227+0000 7fa606a9a640 1 -- 172.21.15.154:0/2033211076 wait complete. 2023-09-23T03:28:08.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.228+0000 7fa606a9a640 1 Processor -- start 2023-09-23T03:28:08.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.229+0000 7fa606a9a640 1 -- start start 2023-09-23T03:28:08.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.229+0000 7fa606a9a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa600158200 0x7fa6001f22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:08.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.230+0000 7fa606a9a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa60015b310 0x7fa6000c4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:08.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.230+0000 7fa606a9a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa60015e2d0 0x7fa6000c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:08.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.230+0000 7fa606a9a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6000c4c50 con 0x7fa60015e2d0 2023-09-23T03:28:08.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.230+0000 7fa606a9a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6000c9d60 con 0x7fa600158200 2023-09-23T03:28:08.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.230+0000 7fa606a9a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6001f28a0 con 0x7fa60015b310 2023-09-23T03:28:08.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.230+0000 7fa606299640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa60015e2d0 0x7fa6000c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:08.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.230+0000 7fa606299640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa60015e2d0 0x7fa6000c9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44900/0 (socket says 172.21.15.154:44900) 2023-09-23T03:28:08.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.230+0000 7fa606299640 1 -- 172.21.15.154:0/4090256755 learned_addr learned my addr 172.21.15.154:0/4090256755 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:08.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.230+0000 7fa605297640 1 --2- 172.21.15.154:0/4090256755 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa60015b310 0x7fa6000c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:08.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.230+0000 7fa605a98640 1 --2- 172.21.15.154:0/4090256755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa600158200 0x7fa6001f22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:08.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.230+0000 7fa606299640 1 -- 172.21.15.154:0/4090256755 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa60015b310 msgr2=0x7fa6000c4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:08.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.230+0000 7fa606299640 1 --2- 172.21.15.154:0/4090256755 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa60015b310 0x7fa6000c4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:08.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.230+0000 7fa606299640 1 -- 172.21.15.154:0/4090256755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa600158200 msgr2=0x7fa6001f22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:08.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.230+0000 7fa606299640 1 --2- 172.21.15.154:0/4090256755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa600158200 0x7fa6001f22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:08.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.230+0000 7fa606299640 1 -- 172.21.15.154:0/4090256755 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f0009400 con 0x7fa60015e2d0 2023-09-23T03:28:08.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.230+0000 7fa605a98640 1 --2- 172.21.15.154:0/4090256755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa600158200 0x7fa6001f22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:08.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.231+0000 7fa606299640 1 --2- 172.21.15.154:0/4090256755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa60015e2d0 0x7fa6000c9820 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fa5f800cbf0 tx=0x7fa5f800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:08.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.231+0000 7fa5f67fc640 1 -- 172.21.15.154:0/4090256755 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f800ad40 con 0x7fa60015e2d0 2023-09-23T03:28:08.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.231+0000 7fa5f67fc640 1 -- 172.21.15.154:0/4090256755 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f8012400 con 0x7fa60015e2d0 2023-09-23T03:28:08.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.231+0000 7fa5f67fc640 1 -- 172.21.15.154:0/4090256755 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f800aea0 con 0x7fa60015e2d0 2023-09-23T03:28:08.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.231+0000 7fa606a9a640 1 -- 172.21.15.154:0/4090256755 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6001f2b80 con 0x7fa60015e2d0 2023-09-23T03:28:08.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.231+0000 7fa606a9a640 1 -- 172.21.15.154:0/4090256755 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6001f3140 con 0x7fa60015e2d0 2023-09-23T03:28:08.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.232+0000 7fa606a9a640 1 -- 172.21.15.154:0/4090256755 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa600066730 con 0x7fa60015e2d0 2023-09-23T03:28:08.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.233+0000 7fa5f67fc640 1 -- 172.21.15.154:0/4090256755 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fa5f8007070 con 0x7fa60015e2d0 2023-09-23T03:28:08.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.234+0000 7fa5f67fc640 1 --2- 172.21.15.154:0/4090256755 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa5e406af30 0x7fa5e406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:08.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.234+0000 7fa5f67fc640 1 -- 172.21.15.154:0/4090256755 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fa5f8089700 con 0x7fa60015e2d0 2023-09-23T03:28:08.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.234+0000 7fa605a98640 1 --2- 172.21.15.154:0/4090256755 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa5e406af30 0x7fa5e406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:08.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.235+0000 7fa605a98640 1 --2- 172.21.15.154:0/4090256755 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa5e406af30 0x7fa5e406d3f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc00a830 tx=0x7fa5fc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:08.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.239+0000 7fa5f67fc640 1 -- 172.21.15.154:0/4090256755 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fa5f80546b0 con 0x7fa60015e2d0 2023-09-23T03:28:08.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.466+0000 7fa606a9a640 1 -- 172.21.15.154:0/4090256755 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa60004e920 con 0x7fa60015e2d0 2023-09-23T03:28:08.468 INFO:tasks.ceph.mon.a.smithi154.stderr:2023-09-23T03:28:08.467+0000 7f3df35a6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-23T03:28:08.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.470+0000 7fa5f67fc640 1 -- 172.21.15.154:0/4090256755 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v109) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa5f8057cd0 con 0x7fa60015e2d0 2023-09-23T03:28:08.471 INFO:teuthology.orchestra.run.smithi154.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-09-23T03:28:08.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.473+0000 7fa606a9a640 1 -- 172.21.15.154:0/4090256755 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa5e406af30 msgr2=0x7fa5e406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:08.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.473+0000 7fa606a9a640 1 --2- 172.21.15.154:0/4090256755 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa5e406af30 0x7fa5e406d3f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc00a830 tx=0x7fa5fc008040 comp rx=0 tx=0).stop 2023-09-23T03:28:08.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.473+0000 7fa606a9a640 1 -- 172.21.15.154:0/4090256755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa60015e2d0 msgr2=0x7fa6000c9820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:08.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.473+0000 7fa606a9a640 1 --2- 172.21.15.154:0/4090256755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa60015e2d0 0x7fa6000c9820 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fa5f800cbf0 tx=0x7fa5f800a430 comp rx=0 tx=0).stop 2023-09-23T03:28:08.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.473+0000 7fa606a9a640 1 -- 172.21.15.154:0/4090256755 shutdown_connections 2023-09-23T03:28:08.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.473+0000 7fa606a9a640 1 --2- 172.21.15.154:0/4090256755 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa5e406af30 0x7fa5e406d3f0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:08.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.473+0000 7fa606a9a640 1 --2- 172.21.15.154:0/4090256755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa60015e2d0 0x7fa6000c9820 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:08.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.473+0000 7fa606a9a640 1 --2- 172.21.15.154:0/4090256755 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa60015b310 0x7fa6000c4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:08.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.473+0000 7fa606a9a640 1 --2- 172.21.15.154:0/4090256755 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa600158200 0x7fa6001f22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:08.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.473+0000 7fa606a9a640 1 -- 172.21.15.154:0/4090256755 >> 172.21.15.154:0/4090256755 conn(0x7fa60006a4c0 msgr2=0x7fa60006c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:08.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.474+0000 7fa606a9a640 1 -- 172.21.15.154:0/4090256755 shutdown_connections 2023-09-23T03:28:08.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.474+0000 7fa606a9a640 1 -- 172.21.15.154:0/4090256755 wait complete. 2023-09-23T03:28:08.494 DEBUG:teuthology.orchestra.run.smithi154:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 10000000000.01000000 2023-09-23T03:28:08.598 INFO:tasks.ceph.mds.b.smithi190.stdout:starting mds.b at 2023-09-23T03:28:08.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.600+0000 7fbefdc3df40 1 Processor -- start 2023-09-23T03:28:08.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.601+0000 7fbefdc3df40 1 -- start start 2023-09-23T03:28:08.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.602+0000 7fbefdc3df40 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56431a0fdce0 0x56431a147080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:08.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.602+0000 7fbefdc3df40 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56431a147700 0x56431a1d53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:08.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.602+0000 7fbefdc3df40 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56431a1d69c0 0x56431a12d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:08.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.602+0000 7fbefdc3df40 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431a049800 con 0x56431a1d69c0 2023-09-23T03:28:08.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.602+0000 7fbefdc3df40 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431a0494a0 con 0x56431a147700 2023-09-23T03:28:08.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.602+0000 7fbefdc3df40 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56431a03d4f0 con 0x56431a0fdce0 2023-09-23T03:28:08.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.602+0000 7fbefaec7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56431a0fdce0 0x56431a147080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:08.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.602+0000 7fbefb6c8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56431a1d69c0 0x56431a12d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:08.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.602+0000 7fbef26c6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56431a147700 0x56431a1d53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:08.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.602+0000 7fbefaec7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56431a0fdce0 0x56431a147080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:46308/0 (socket says 172.21.15.154:46308) 2023-09-23T03:28:08.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.602+0000 7fbefaec7640 1 -- 172.21.15.154:0/1567716188 learned_addr learned my addr 172.21.15.154:0/1567716188 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:08.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.602+0000 7fbef26c6640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56431a147700 0x56431a1d53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.154:42490/0 (socket says 172.21.15.154:42490) 2023-09-23T03:28:08.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.602+0000 7fbefb6c8640 1 -- 172.21.15.154:0/1567716188 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56431a0fdce0 msgr2=0x56431a147080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:08.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.602+0000 7fbefb6c8640 1 --2- 172.21.15.154:0/1567716188 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56431a0fdce0 0x56431a147080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:08.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.602+0000 7fbefb6c8640 1 -- 172.21.15.154:0/1567716188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56431a147700 msgr2=0x56431a1d53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:08.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.602+0000 7fbefb6c8640 1 --2- 172.21.15.154:0/1567716188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56431a147700 0x56431a1d53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:08.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.602+0000 7fbefb6c8640 1 -- 172.21.15.154:0/1567716188 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee4005d20 con 0x56431a1d69c0 2023-09-23T03:28:08.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.603+0000 7fbefb6c8640 1 --2- 172.21.15.154:0/1567716188 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56431a1d69c0 0x56431a12d940 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fbef400b650 tx=0x7fbef400bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba396829351aa98 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:08.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.603+0000 7fbef9ec5640 1 -- 172.21.15.154:0/1567716188 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef400d7b0 con 0x56431a1d69c0 2023-09-23T03:28:08.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.603+0000 7fbef9ec5640 1 -- 172.21.15.154:0/1567716188 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef400d910 con 0x56431a1d69c0 2023-09-23T03:28:08.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.603+0000 7fbef9ec5640 1 -- 172.21.15.154:0/1567716188 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef400dac0 con 0x56431a1d69c0 2023-09-23T03:28:08.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.604+0000 7fbefdc3df40 1 -- 172.21.15.154:0/1567716188 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56431a1d69c0 msgr2=0x56431a12d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:08.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.604+0000 7fbefdc3df40 1 --2- 172.21.15.154:0/1567716188 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56431a1d69c0 0x56431a12d940 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fbef400b650 tx=0x7fbef400bb10 comp rx=0 tx=0).stop 2023-09-23T03:28:08.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.604+0000 7fbefdc3df40 1 -- 172.21.15.154:0/1567716188 shutdown_connections 2023-09-23T03:28:08.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.604+0000 7fbefdc3df40 1 --2- 172.21.15.154:0/1567716188 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56431a1d69c0 0x56431a12d940 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:08.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.604+0000 7fbefdc3df40 1 --2- 172.21.15.154:0/1567716188 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56431a147700 0x56431a1d53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:08.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.604+0000 7fbefdc3df40 1 --2- 172.21.15.154:0/1567716188 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56431a0fdce0 0x56431a147080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:08.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.604+0000 7fbefdc3df40 1 -- 172.21.15.154:0/1567716188 >> 172.21.15.154:0/1567716188 conn(0x56431a14d0b0 msgr2=0x56431a1427e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:08.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.604+0000 7fbefdc3df40 1 -- 172.21.15.154:0/1567716188 shutdown_connections 2023-09-23T03:28:08.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.604+0000 7fbefdc3df40 1 -- 172.21.15.154:0/1567716188 wait complete. 2023-09-23T03:28:08.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.605+0000 7fbefdc3df40 1 Processor -- start 2023-09-23T03:28:08.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.605+0000 7fbefdc3df40 1 -- start start 2023-09-23T03:28:08.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.606+0000 7fbefdc3df40 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56431a0fdce0 0x56431a1d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:08.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.606+0000 7fbefdc3df40 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56431a147700 0x56431a1322c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:08.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.606+0000 7fbefdc3df40 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56431a1d69c0 0x56431a12db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:08.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.606+0000 7fbefdc3df40 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431a1d9db0 con 0x56431a0fdce0 2023-09-23T03:28:08.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.606+0000 7fbefdc3df40 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431a1d9ad0 con 0x56431a1d69c0 2023-09-23T03:28:08.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.606+0000 7fbefdc3df40 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56431a12d200 con 0x56431a147700 2023-09-23T03:28:08.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.606+0000 7fbefaec7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56431a0fdce0 0x56431a1d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:08.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.606+0000 7fbefaec7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56431a0fdce0 0x56431a1d01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44926/0 (socket says 172.21.15.154:44926) 2023-09-23T03:28:08.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.606+0000 7fbefaec7640 1 -- 172.21.15.154:0/428260944 learned_addr learned my addr 172.21.15.154:0/428260944 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:08.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.606+0000 7fbefb6c8640 1 --2- 172.21.15.154:0/428260944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56431a1d69c0 0x56431a12db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:08.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.606+0000 7fbef26c6640 1 --2- 172.21.15.154:0/428260944 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56431a147700 0x56431a1322c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:08.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.607+0000 7fbefaec7640 1 -- 172.21.15.154:0/428260944 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56431a147700 msgr2=0x56431a1322c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:08.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.607+0000 7fbefaec7640 1 --2- 172.21.15.154:0/428260944 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56431a147700 0x56431a1322c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:08.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.607+0000 7fbefaec7640 1 -- 172.21.15.154:0/428260944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56431a1d69c0 msgr2=0x56431a12db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:08.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.607+0000 7fbefaec7640 1 --2- 172.21.15.154:0/428260944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56431a1d69c0 0x56431a12db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:08.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.607+0000 7fbefaec7640 1 -- 172.21.15.154:0/428260944 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef400b300 con 0x56431a0fdce0 2023-09-23T03:28:08.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.607+0000 7fbefb6c8640 1 --2- 172.21.15.154:0/428260944 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56431a1d69c0 0x56431a12db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:28:08.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.607+0000 7fbef26c6640 1 --2- 172.21.15.154:0/428260944 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56431a147700 0x56431a1322c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:08.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.607+0000 7fbefaec7640 1 --2- 172.21.15.154:0/428260944 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56431a0fdce0 0x56431a1d01e0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fbeec00c660 tx=0x7fbeec00cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:08.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.607+0000 7fbef8ec3640 1 -- 172.21.15.154:0/428260944 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec006c50 con 0x56431a0fdce0 2023-09-23T03:28:08.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.607+0000 7fbef8ec3640 1 -- 172.21.15.154:0/428260944 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeec00bb20 con 0x56431a0fdce0 2023-09-23T03:28:08.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.607+0000 7fbefdc3df40 1 -- 172.21.15.154:0/428260944 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56431a055650 con 0x56431a0fdce0 2023-09-23T03:28:08.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.607+0000 7fbef8ec3640 1 -- 172.21.15.154:0/428260944 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec00ce60 con 0x56431a0fdce0 2023-09-23T03:28:08.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.607+0000 7fbefdc3df40 1 -- 172.21.15.154:0/428260944 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56431a048800 con 0x56431a0fdce0 2023-09-23T03:28:08.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.610+0000 7fbef8ec3640 1 -- 172.21.15.154:0/428260944 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fbeec020440 con 0x56431a0fdce0 2023-09-23T03:28:08.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.611+0000 7fbef8ec3640 1 --2- 172.21.15.154:0/428260944 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbedc0698e0 0x7fbedc06bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:08.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.611+0000 7fbef8ec3640 1 -- 172.21.15.154:0/428260944 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fbeec089140 con 0x56431a0fdce0 2023-09-23T03:28:08.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.611+0000 7fbef26c6640 1 --2- 172.21.15.154:0/428260944 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbedc0698e0 0x7fbedc06bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:08.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.611+0000 7fbefdc3df40 1 --2- 172.21.15.154:0/428260944 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x56431a12e070 0x56431a1ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:08.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.611+0000 7fbefdc3df40 1 -- 172.21.15.154:0/428260944 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:1 17.5 17:a40620de:::10000000000.01000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v8 -- 0x56431a132800 con 0x56431a12e070 2023-09-23T03:28:08.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.612+0000 7fbefb6c8640 1 --2- 172.21.15.154:0/428260944 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x56431a12e070 0x56431a1ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:08.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.612+0000 7fbef26c6640 1 --2- 172.21.15.154:0/428260944 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbedc0698e0 0x7fbedc06bda0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fbee4054000 tx=0x7fbee404e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:08.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.612+0000 7fbefb6c8640 1 --2- 172.21.15.154:0/428260944 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x56431a12e070 0x56431a1ce0f0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:08.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.613+0000 7fbefb6c8640 1 -- 172.21.15.154:0/428260944 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== osd_op_reply(1 10000000000.01000000 [omap-get-keys out=560b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+560 (crc 0 0 0) 0x7fbef4012470 con 0x56431a12e070 2023-09-23T03:28:08.614 INFO:teuthology.orchestra.run.smithi154.stdout:10_head 2023-09-23T03:28:08.615 INFO:teuthology.orchestra.run.smithi154.stdout:11_head 2023-09-23T03:28:08.615 INFO:teuthology.orchestra.run.smithi154.stdout:12_head 2023-09-23T03:28:08.615 INFO:teuthology.orchestra.run.smithi154.stdout:15_head 2023-09-23T03:28:08.615 INFO:teuthology.orchestra.run.smithi154.stdout:19_head 2023-09-23T03:28:08.615 INFO:teuthology.orchestra.run.smithi154.stdout:1_head 2023-09-23T03:28:08.615 INFO:teuthology.orchestra.run.smithi154.stdout:20_head 2023-09-23T03:28:08.615 INFO:teuthology.orchestra.run.smithi154.stdout:21_head 2023-09-23T03:28:08.615 INFO:teuthology.orchestra.run.smithi154.stdout:27_head 2023-09-23T03:28:08.615 INFO:teuthology.orchestra.run.smithi154.stdout:28_head 2023-09-23T03:28:08.615 INFO:teuthology.orchestra.run.smithi154.stdout:2_head 2023-09-23T03:28:08.615 INFO:teuthology.orchestra.run.smithi154.stdout:31_head 2023-09-23T03:28:08.616 INFO:teuthology.orchestra.run.smithi154.stdout:34_head 2023-09-23T03:28:08.616 INFO:teuthology.orchestra.run.smithi154.stdout:36_head 2023-09-23T03:28:08.616 INFO:teuthology.orchestra.run.smithi154.stdout:38_head 2023-09-23T03:28:08.616 INFO:teuthology.orchestra.run.smithi154.stdout:39_head 2023-09-23T03:28:08.616 INFO:teuthology.orchestra.run.smithi154.stdout:3_head 2023-09-23T03:28:08.616 INFO:teuthology.orchestra.run.smithi154.stdout:42_head 2023-09-23T03:28:08.616 INFO:teuthology.orchestra.run.smithi154.stdout:43_head 2023-09-23T03:28:08.616 INFO:teuthology.orchestra.run.smithi154.stdout:46_head 2023-09-23T03:28:08.616 INFO:teuthology.orchestra.run.smithi154.stdout:4_head 2023-09-23T03:28:08.616 INFO:teuthology.orchestra.run.smithi154.stdout:50_head 2023-09-23T03:28:08.616 INFO:teuthology.orchestra.run.smithi154.stdout:52_head 2023-09-23T03:28:08.616 INFO:teuthology.orchestra.run.smithi154.stdout:53_head 2023-09-23T03:28:08.617 INFO:teuthology.orchestra.run.smithi154.stdout:55_head 2023-09-23T03:28:08.617 INFO:teuthology.orchestra.run.smithi154.stdout:59_head 2023-09-23T03:28:08.617 INFO:teuthology.orchestra.run.smithi154.stdout:61_head 2023-09-23T03:28:08.617 INFO:teuthology.orchestra.run.smithi154.stdout:63_head 2023-09-23T03:28:08.617 INFO:teuthology.orchestra.run.smithi154.stdout:64_head 2023-09-23T03:28:08.617 INFO:teuthology.orchestra.run.smithi154.stdout:65_head 2023-09-23T03:28:08.617 INFO:teuthology.orchestra.run.smithi154.stdout:66_head 2023-09-23T03:28:08.617 INFO:teuthology.orchestra.run.smithi154.stdout:69_head 2023-09-23T03:28:08.617 INFO:teuthology.orchestra.run.smithi154.stdout:70_head 2023-09-23T03:28:08.617 INFO:teuthology.orchestra.run.smithi154.stdout:73_head 2023-09-23T03:28:08.617 INFO:teuthology.orchestra.run.smithi154.stdout:74_head 2023-09-23T03:28:08.617 INFO:teuthology.orchestra.run.smithi154.stdout:75_head 2023-09-23T03:28:08.618 INFO:teuthology.orchestra.run.smithi154.stdout:77_head 2023-09-23T03:28:08.618 INFO:teuthology.orchestra.run.smithi154.stdout:80_head 2023-09-23T03:28:08.618 INFO:teuthology.orchestra.run.smithi154.stdout:83_head 2023-09-23T03:28:08.618 INFO:teuthology.orchestra.run.smithi154.stdout:84_head 2023-09-23T03:28:08.618 INFO:teuthology.orchestra.run.smithi154.stdout:85_head 2023-09-23T03:28:08.618 INFO:teuthology.orchestra.run.smithi154.stdout:86_head 2023-09-23T03:28:08.618 INFO:teuthology.orchestra.run.smithi154.stdout:87_head 2023-09-23T03:28:08.618 INFO:teuthology.orchestra.run.smithi154.stdout:88_head 2023-09-23T03:28:08.618 INFO:teuthology.orchestra.run.smithi154.stdout:8_head 2023-09-23T03:28:08.618 INFO:teuthology.orchestra.run.smithi154.stdout:90_head 2023-09-23T03:28:08.618 INFO:teuthology.orchestra.run.smithi154.stdout:92_head 2023-09-23T03:28:08.618 INFO:teuthology.orchestra.run.smithi154.stdout:95_head 2023-09-23T03:28:08.619 INFO:teuthology.orchestra.run.smithi154.stdout:96_head 2023-09-23T03:28:08.619 INFO:teuthology.orchestra.run.smithi154.stdout:97_head 2023-09-23T03:28:08.619 INFO:teuthology.orchestra.run.smithi154.stdout:9_head 2023-09-23T03:28:08.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.613+0000 7fbefdc3df40 1 -- 172.21.15.154:0/428260944 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x56431a12e070 msgr2=0x56431a1ce0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:08.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.613+0000 7fbefdc3df40 1 --2- 172.21.15.154:0/428260944 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x56431a12e070 0x56431a1ce0f0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:08.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.613+0000 7fbefdc3df40 1 -- 172.21.15.154:0/428260944 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbedc0698e0 msgr2=0x7fbedc06bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:08.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.613+0000 7fbefdc3df40 1 --2- 172.21.15.154:0/428260944 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbedc0698e0 0x7fbedc06bda0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fbee4054000 tx=0x7fbee404e000 comp rx=0 tx=0).stop 2023-09-23T03:28:08.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.613+0000 7fbefdc3df40 1 -- 172.21.15.154:0/428260944 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56431a0fdce0 msgr2=0x56431a1d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:08.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.613+0000 7fbefdc3df40 1 --2- 172.21.15.154:0/428260944 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56431a0fdce0 0x56431a1d01e0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fbeec00c660 tx=0x7fbeec00cb20 comp rx=0 tx=0).stop 2023-09-23T03:28:08.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.613+0000 7fbefb6c8640 1 -- 172.21.15.154:0/428260944 reap_dead start 2023-09-23T03:28:08.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.613+0000 7fbefdc3df40 1 -- 172.21.15.154:0/428260944 shutdown_connections 2023-09-23T03:28:08.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.613+0000 7fbefdc3df40 1 -- 172.21.15.154:0/428260944 >> 172.21.15.154:0/428260944 conn(0x56431a14d0b0 msgr2=0x56431a1cf4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:08.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.613+0000 7fbefdc3df40 1 -- 172.21.15.154:0/428260944 shutdown_connections 2023-09-23T03:28:08.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.613+0000 7fbefdc3df40 1 -- 172.21.15.154:0/428260944 wait complete. 2023-09-23T03:28:08.621 INFO:tasks.cephfs.test_data_scan:victim_key=21_head 2023-09-23T03:28:08.621 DEBUG:teuthology.orchestra.run.smithi154:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000000.01000000 21_head 2023-09-23T03:28:08.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.721+0000 7f212ce40f40 1 Processor -- start 2023-09-23T03:28:08.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.721+0000 7f212ce40f40 1 -- start start 2023-09-23T03:28:08.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.722+0000 7f212ce40f40 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2114003a70 0x558fca23c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:08.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.722+0000 7f212ce40f40 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558fca1b3a40 0x558fca1b8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:08.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.722+0000 7f212ce40f40 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558fca1b9050 0x558fca1b7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:08.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.722+0000 7f212ce40f40 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fca120170 con 0x558fca1b3a40 2023-09-23T03:28:08.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.722+0000 7f212ce40f40 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fca0bb800 con 0x558fca1b9050 2023-09-23T03:28:08.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.722+0000 7f212ce40f40 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x558fca0bb4a0 con 0x7f2114003a70 2023-09-23T03:28:08.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.722+0000 7f212992d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558fca1b3a40 0x558fca1b8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:08.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.722+0000 7f212992d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558fca1b3a40 0x558fca1b8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44946/0 (socket says 172.21.15.154:44946) 2023-09-23T03:28:08.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.722+0000 7f212992d640 1 -- 172.21.15.154:0/3883142552 learned_addr learned my addr 172.21.15.154:0/3883142552 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:08.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.723+0000 7f212992d640 1 -- 172.21.15.154:0/3883142552 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2114003a70 msgr2=0x558fca23c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:08.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.723+0000 7f212992d640 1 --2- 172.21.15.154:0/3883142552 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2114003a70 0x558fca23c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:08.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.723+0000 7f212992d640 1 -- 172.21.15.154:0/3883142552 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558fca1b9050 msgr2=0x558fca1b7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:08.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.723+0000 7f212992d640 1 --2- 172.21.15.154:0/3883142552 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558fca1b9050 0x558fca1b7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:08.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.723+0000 7f212992d640 1 -- 172.21.15.154:0/3883142552 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fca0ba800 con 0x558fca1b3a40 2023-09-23T03:28:08.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.723+0000 7f212992d640 1 --2- 172.21.15.154:0/3883142552 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558fca1b3a40 0x558fca1b8b10 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f212000f4d0 tx=0x7f212000f8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943837ee1ce15860 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:08.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.723+0000 7f212892b640 1 -- 172.21.15.154:0/3883142552 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212000d690 con 0x558fca1b3a40 2023-09-23T03:28:08.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.723+0000 7f212892b640 1 -- 172.21.15.154:0/3883142552 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f212000d810 con 0x558fca1b3a40 2023-09-23T03:28:08.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.723+0000 7f212892b640 1 -- 172.21.15.154:0/3883142552 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212000dbc0 con 0x558fca1b3a40 2023-09-23T03:28:08.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.724+0000 7f212ce40f40 1 -- 172.21.15.154:0/3883142552 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558fca1b3a40 msgr2=0x558fca1b8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:08.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.724+0000 7f212ce40f40 1 --2- 172.21.15.154:0/3883142552 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558fca1b3a40 0x558fca1b8b10 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f212000f4d0 tx=0x7f212000f8d0 comp rx=0 tx=0).stop 2023-09-23T03:28:08.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.724+0000 7f212ce40f40 1 -- 172.21.15.154:0/3883142552 shutdown_connections 2023-09-23T03:28:08.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.724+0000 7f212ce40f40 1 --2- 172.21.15.154:0/3883142552 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558fca1b9050 0x558fca1b7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:08.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.724+0000 7f212ce40f40 1 --2- 172.21.15.154:0/3883142552 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558fca1b3a40 0x558fca1b8b10 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:08.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.724+0000 7f212ce40f40 1 --2- 172.21.15.154:0/3883142552 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2114003a70 0x558fca23c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:08.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.724+0000 7f212ce40f40 1 -- 172.21.15.154:0/3883142552 >> 172.21.15.154:0/3883142552 conn(0x558fca1b9850 msgr2=0x558fca19cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:08.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.724+0000 7f212ce40f40 1 -- 172.21.15.154:0/3883142552 shutdown_connections 2023-09-23T03:28:08.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.724+0000 7f212ce40f40 1 -- 172.21.15.154:0/3883142552 wait complete. 2023-09-23T03:28:08.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.725+0000 7f212ce40f40 1 Processor -- start 2023-09-23T03:28:08.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.725+0000 7f212ce40f40 1 -- start start 2023-09-23T03:28:08.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.726+0000 7f212ce40f40 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2114003a70 0x558fca23b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:08.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.726+0000 7f212ce40f40 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558fca1714b0 0x558fca171930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:08.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.726+0000 7f212ce40f40 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558fca1b52f0 0x558fca1b5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:08.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.726+0000 7f212ce40f40 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fca0af4f0 con 0x558fca1b52f0 2023-09-23T03:28:08.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.726+0000 7f212ce40f40 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fca171e70 con 0x558fca1714b0 2023-09-23T03:28:08.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.726+0000 7f212ce40f40 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x558fca1b5cb0 con 0x7f2114003a70 2023-09-23T03:28:08.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.726+0000 7f212a92f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558fca1b52f0 0x558fca1b5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:08.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.726+0000 7f212a92f640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558fca1b52f0 0x558fca1b5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44950/0 (socket says 172.21.15.154:44950) 2023-09-23T03:28:08.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.726+0000 7f212a92f640 1 -- 172.21.15.154:0/3338755239 learned_addr learned my addr 172.21.15.154:0/3338755239 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:08.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.726+0000 7f212992d640 1 --2- 172.21.15.154:0/3338755239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558fca1714b0 0x558fca171930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:08.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.726+0000 7f212a12e640 1 --2- 172.21.15.154:0/3338755239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2114003a70 0x558fca23b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:08.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.726+0000 7f212a92f640 1 -- 172.21.15.154:0/3338755239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2114003a70 msgr2=0x558fca23b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:08.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.726+0000 7f212a92f640 1 --2- 172.21.15.154:0/3338755239 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2114003a70 0x558fca23b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:08.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.726+0000 7f212a92f640 1 -- 172.21.15.154:0/3338755239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558fca1714b0 msgr2=0x558fca171930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:08.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.726+0000 7f212a92f640 1 --2- 172.21.15.154:0/3338755239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558fca1714b0 0x558fca171930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:08.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.726+0000 7f212a92f640 1 -- 172.21.15.154:0/3338755239 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fca0c7650 con 0x558fca1b52f0 2023-09-23T03:28:08.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.726+0000 7f212992d640 1 --2- 172.21.15.154:0/3338755239 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x558fca1714b0 0x558fca171930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:08.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.727+0000 7f212a92f640 1 --2- 172.21.15.154:0/3338755239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558fca1b52f0 0x558fca1b5770 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f2124044400 tx=0x7f2124052910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:08.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.727+0000 7f211b7fe640 1 -- 172.21.15.154:0/3338755239 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2124006230 con 0x558fca1b52f0 2023-09-23T03:28:08.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.727+0000 7f211b7fe640 1 -- 172.21.15.154:0/3338755239 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2124001250 con 0x558fca1b52f0 2023-09-23T03:28:08.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.727+0000 7f211b7fe640 1 -- 172.21.15.154:0/3338755239 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2124052c40 con 0x558fca1b52f0 2023-09-23T03:28:08.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.727+0000 7f212ce40f40 1 -- 172.21.15.154:0/3338755239 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558fca0ba260 con 0x558fca1b52f0 2023-09-23T03:28:08.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.727+0000 7f212ce40f40 1 -- 172.21.15.154:0/3338755239 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558fca1a0700 con 0x558fca1b52f0 2023-09-23T03:28:08.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.729+0000 7f211b7fe640 1 -- 172.21.15.154:0/3338755239 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f212405b480 con 0x558fca1b52f0 2023-09-23T03:28:08.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.730+0000 7f211b7fe640 1 --2- 172.21.15.154:0/3338755239 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f210c069910 0x7f210c06bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:08.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.730+0000 7f211b7fe640 1 -- 172.21.15.154:0/3338755239 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f21240c93e0 con 0x558fca1b52f0 2023-09-23T03:28:08.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.730+0000 7f212a12e640 1 --2- 172.21.15.154:0/3338755239 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f210c069910 0x7f210c06bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:08.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.730+0000 7f212ce40f40 1 --2- 172.21.15.154:0/3338755239 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x558fca1703a0 0x558fca1a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:08.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.730+0000 7f212ce40f40 1 -- 172.21.15.154:0/3338755239 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:1 17.5 17:a40620de:::10000000000.01000000:head [omap-rm-keys in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v8 -- 0x558fca23be10 con 0x558fca1703a0 2023-09-23T03:28:08.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.731+0000 7f212a12e640 1 --2- 172.21.15.154:0/3338755239 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f210c069910 0x7f210c06bdd0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f211c009600 tx=0x7f211c002fb0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:08.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.731+0000 7f212992d640 1 --2- 172.21.15.154:0/3338755239 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x558fca1703a0 0x558fca1a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:08.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.732+0000 7f212992d640 1 --2- 172.21.15.154:0/3338755239 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x558fca1703a0 0x558fca1a0b70 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:08.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.734+0000 7f212992d640 1 -- 172.21.15.154:0/3338755239 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== osd_op_reply(1 10000000000.01000000 [omap-rm-keys] v127'3 uv3 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f2120022040 con 0x558fca1703a0 2023-09-23T03:28:08.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.734+0000 7f212ce40f40 1 -- 172.21.15.154:0/3338755239 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x558fca1703a0 msgr2=0x558fca1a0b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:08.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.734+0000 7f212ce40f40 1 --2- 172.21.15.154:0/3338755239 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x558fca1703a0 0x558fca1a0b70 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:08.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.734+0000 7f212ce40f40 1 -- 172.21.15.154:0/3338755239 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f210c069910 msgr2=0x7f210c06bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:08.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.734+0000 7f212ce40f40 1 --2- 172.21.15.154:0/3338755239 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f210c069910 0x7f210c06bdd0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f211c009600 tx=0x7f211c002fb0 comp rx=0 tx=0).stop 2023-09-23T03:28:08.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.734+0000 7f212ce40f40 1 -- 172.21.15.154:0/3338755239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558fca1b52f0 msgr2=0x558fca1b5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:08.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.734+0000 7f212ce40f40 1 --2- 172.21.15.154:0/3338755239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558fca1b52f0 0x558fca1b5770 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f2124044400 tx=0x7f2124052910 comp rx=0 tx=0).stop 2023-09-23T03:28:08.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.734+0000 7f212a92f640 1 -- 172.21.15.154:0/3338755239 reap_dead start 2023-09-23T03:28:08.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.735+0000 7f212ce40f40 1 -- 172.21.15.154:0/3338755239 shutdown_connections 2023-09-23T03:28:08.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.735+0000 7f212ce40f40 1 -- 172.21.15.154:0/3338755239 >> 172.21.15.154:0/3338755239 conn(0x558fca1b9850 msgr2=0x558fca23c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:08.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.735+0000 7f212ce40f40 1 -- 172.21.15.154:0/3338755239 shutdown_connections 2023-09-23T03:28:08.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.735+0000 7f212ce40f40 1 -- 172.21.15.154:0/3338755239 wait complete. 2023-09-23T03:28:08.741 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-09-23T03:28:08.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.925+0000 7f6489d6c640 1 Processor -- start 2023-09-23T03:28:08.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.925+0000 7f6489d6c640 1 -- start start 2023-09-23T03:28:08.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.926+0000 7f6489d6c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6484144130 0x7f64840b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:08.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.926+0000 7f6489d6c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f64840caf80 0x7f64840cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:08.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.926+0000 7f6489d6c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f648415e520 0x7f648415e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:08.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.926+0000 7f6489d6c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6484070fb0 con 0x7f64840caf80 2023-09-23T03:28:08.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.926+0000 7f6489d6c640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64840cb920 con 0x7f648415e520 2023-09-23T03:28:08.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.926+0000 7f6489d6c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64840700e0 con 0x7f6484144130 2023-09-23T03:28:08.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.927+0000 7f647affd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f64840caf80 0x7f64840cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:08.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.927+0000 7f647affd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f64840caf80 0x7f64840cb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44958/0 (socket says 172.21.15.154:44958) 2023-09-23T03:28:08.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.927+0000 7f647affd640 1 -- 172.21.15.154:0/4172553990 learned_addr learned my addr 172.21.15.154:0/4172553990 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:08.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.927+0000 7f647affd640 1 -- 172.21.15.154:0/4172553990 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6484144130 msgr2=0x7f64840b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:08.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.927+0000 7f647affd640 1 --2- 172.21.15.154:0/4172553990 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6484144130 0x7f64840b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:08.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.927+0000 7f6483fff640 1 --2- 172.21.15.154:0/4172553990 >> v2:172.21.15.190:3300/0 conn(0x7f648415e520 0x7f648415e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:08.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.927+0000 7f647affd640 1 -- 172.21.15.154:0/4172553990 >> v2:172.21.15.190:3300/0 conn(0x7f648415e520 msgr2=0x7f648415e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:08.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.927+0000 7f647affd640 1 --2- 172.21.15.154:0/4172553990 >> v2:172.21.15.190:3300/0 conn(0x7f648415e520 0x7f648415e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:08.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.927+0000 7f647affd640 1 -- 172.21.15.154:0/4172553990 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6484070330 con 0x7f64840caf80 2023-09-23T03:28:08.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.928+0000 7f647affd640 1 --2- 172.21.15.154:0/4172553990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64840caf80 0x7f64840cb3e0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f64840cbce0 tx=0x7f646c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d537f11543cdeb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:08.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.928+0000 7f6482ffd640 1 -- 172.21.15.154:0/4172553990 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646c031870 con 0x7f64840caf80 2023-09-23T03:28:08.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.928+0000 7f6482ffd640 1 -- 172.21.15.154:0/4172553990 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f646c0319d0 con 0x7f64840caf80 2023-09-23T03:28:08.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.929+0000 7f6482ffd640 1 -- 172.21.15.154:0/4172553990 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646c031d60 con 0x7f64840caf80 2023-09-23T03:28:08.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.929+0000 7f6489d6c640 1 -- 172.21.15.154:0/4172553990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64840caf80 msgr2=0x7f64840cb3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:08.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.929+0000 7f6489d6c640 1 --2- 172.21.15.154:0/4172553990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64840caf80 0x7f64840cb3e0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f64840cbce0 tx=0x7f646c02b910 comp rx=0 tx=0).stop 2023-09-23T03:28:08.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.929+0000 7f6489d6c640 1 -- 172.21.15.154:0/4172553990 shutdown_connections 2023-09-23T03:28:08.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.929+0000 7f6489d6c640 1 --2- 172.21.15.154:0/4172553990 >> v2:172.21.15.190:3300/0 conn(0x7f648415e520 0x7f648415e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:08.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.929+0000 7f6489d6c640 1 --2- 172.21.15.154:0/4172553990 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f64840caf80 0x7f64840cb3e0 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:08.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.929+0000 7f6489d6c640 1 --2- 172.21.15.154:0/4172553990 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6484144130 0x7f64840b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:08.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.929+0000 7f6489d6c640 1 -- 172.21.15.154:0/4172553990 >> 172.21.15.154:0/4172553990 conn(0x7f648406a440 msgr2=0x7f64840b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:08.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.929+0000 7f6489d6c640 1 -- 172.21.15.154:0/4172553990 shutdown_connections 2023-09-23T03:28:08.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.929+0000 7f6489d6c640 1 -- 172.21.15.154:0/4172553990 wait complete. 2023-09-23T03:28:08.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.931+0000 7f6489d6c640 1 Processor -- start 2023-09-23T03:28:08.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.931+0000 7f6489d6c640 1 -- start start 2023-09-23T03:28:08.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.932+0000 7f6489d6c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64840caf80 0x7f6484159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:08.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.932+0000 7f6489d6c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6484144130 0x7f64840c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:08.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.932+0000 7f6489d6c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f648415e520 0x7f64840c4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:08.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.932+0000 7f6489d6c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64840c6420 con 0x7f648415e520 2023-09-23T03:28:08.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.932+0000 7f6489d6c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64840c5810 con 0x7f6484144130 2023-09-23T03:28:08.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.932+0000 7f6489d6c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64840c5af0 con 0x7f64840caf80 2023-09-23T03:28:08.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.932+0000 7f6483fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f648415e520 0x7f64840c4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:08.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.932+0000 7f6483fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f648415e520 0x7f64840c4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44964/0 (socket says 172.21.15.154:44964) 2023-09-23T03:28:08.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.932+0000 7f6483fff640 1 -- 172.21.15.154:0/1744222966 learned_addr learned my addr 172.21.15.154:0/1744222966 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:08.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.933+0000 7f64837fe640 1 --2- 172.21.15.154:0/1744222966 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64840caf80 0x7f6484159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:08.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.933+0000 7f647affd640 1 --2- 172.21.15.154:0/1744222966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6484144130 0x7f64840c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:08.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.933+0000 7f6483fff640 1 -- 172.21.15.154:0/1744222966 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64840caf80 msgr2=0x7f6484159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:08.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.933+0000 7f6483fff640 1 --2- 172.21.15.154:0/1744222966 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64840caf80 0x7f6484159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:08.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.933+0000 7f6483fff640 1 -- 172.21.15.154:0/1744222966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6484144130 msgr2=0x7f64840c68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:08.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.933+0000 7f6483fff640 1 --2- 172.21.15.154:0/1744222966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6484144130 0x7f64840c68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:08.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.933+0000 7f6483fff640 1 -- 172.21.15.154:0/1744222966 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f646c0094d0 con 0x7f648415e520 2023-09-23T03:28:08.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.933+0000 7f64837fe640 1 --2- 172.21.15.154:0/1744222966 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64840caf80 0x7f6484159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:08.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.933+0000 7f6483fff640 1 --2- 172.21.15.154:0/1744222966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f648415e520 0x7f64840c4d70 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f647c00cc30 tx=0x7f647c00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:08.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.933+0000 7f6480ff9640 1 -- 172.21.15.154:0/1744222966 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647c00ce40 con 0x7f648415e520 2023-09-23T03:28:08.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.933+0000 7f6480ff9640 1 -- 172.21.15.154:0/1744222966 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f647c003990 con 0x7f648415e520 2023-09-23T03:28:08.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.933+0000 7f6489d6c640 1 -- 172.21.15.154:0/1744222966 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64840c5dd0 con 0x7f648415e520 2023-09-23T03:28:08.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.933+0000 7f6480ff9640 1 -- 172.21.15.154:0/1744222966 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647c003d20 con 0x7f648415e520 2023-09-23T03:28:08.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.934+0000 7f6489d6c640 1 -- 172.21.15.154:0/1744222966 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64841415f0 con 0x7f648415e520 2023-09-23T03:28:08.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.935+0000 7f6489d6c640 1 -- 172.21.15.154:0/1744222966 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64840c6590 con 0x7f648415e520 2023-09-23T03:28:08.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.936+0000 7f6480ff9640 1 -- 172.21.15.154:0/1744222966 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f647c003e80 con 0x7f648415e520 2023-09-23T03:28:08.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.937+0000 7f6480ff9640 1 --2- 172.21.15.154:0/1744222966 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f646406af30 0x7f646406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:08.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.937+0000 7f6480ff9640 1 -- 172.21.15.154:0/1744222966 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f647c089d80 con 0x7f648415e520 2023-09-23T03:28:08.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.937+0000 7f64837fe640 1 --2- 172.21.15.154:0/1744222966 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f646406af30 0x7f646406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:08.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.937+0000 7f64837fe640 1 --2- 172.21.15.154:0/1744222966 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f646406af30 0x7f646406d3f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f6474006fd0 tx=0x7f6474008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:08.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:08.941+0000 7f6480ff9640 1 -- 172.21.15.154:0/1744222966 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f647c054b80 con 0x7f648415e520 2023-09-23T03:28:09.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.157+0000 7f6489d6c640 1 -- 172.21.15.154:0/1744222966 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f6484073eb0 con 0x7f648415e520 2023-09-23T03:28:09.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.510+0000 7f6480ff9640 1 -- 172.21.15.154:0/1744222966 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v110) v1 ==== 162+0+0 (secure 0 0 0) 0x7f647c01d070 con 0x7f648415e520 2023-09-23T03:28:09.512 INFO:teuthology.orchestra.run.smithi154.stderr:cephfs marked joinable; MDS may join as newly active. 2023-09-23T03:28:09.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.514+0000 7f6489d6c640 1 -- 172.21.15.154:0/1744222966 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f646406af30 msgr2=0x7f646406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:09.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.514+0000 7f6489d6c640 1 --2- 172.21.15.154:0/1744222966 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f646406af30 0x7f646406d3f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f6474006fd0 tx=0x7f6474008040 comp rx=0 tx=0).stop 2023-09-23T03:28:09.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.514+0000 7f6489d6c640 1 -- 172.21.15.154:0/1744222966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f648415e520 msgr2=0x7f64840c4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:09.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.514+0000 7f6489d6c640 1 --2- 172.21.15.154:0/1744222966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f648415e520 0x7f64840c4d70 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f647c00cc30 tx=0x7f647c00a5c0 comp rx=0 tx=0).stop 2023-09-23T03:28:09.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.514+0000 7f6489d6c640 1 -- 172.21.15.154:0/1744222966 shutdown_connections 2023-09-23T03:28:09.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.514+0000 7f6489d6c640 1 --2- 172.21.15.154:0/1744222966 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f646406af30 0x7f646406d3f0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:09.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.514+0000 7f6489d6c640 1 --2- 172.21.15.154:0/1744222966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f648415e520 0x7f64840c4d70 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:09.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.514+0000 7f6489d6c640 1 --2- 172.21.15.154:0/1744222966 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6484144130 0x7f64840c68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:09.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.514+0000 7f6489d6c640 1 --2- 172.21.15.154:0/1744222966 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f64840caf80 0x7f6484159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:09.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.514+0000 7f6489d6c640 1 -- 172.21.15.154:0/1744222966 >> 172.21.15.154:0/1744222966 conn(0x7f648406a440 msgr2=0x7f648415abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:09.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.514+0000 7f6489d6c640 1 -- 172.21.15.154:0/1744222966 shutdown_connections 2023-09-23T03:28:09.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.514+0000 7f6489d6c640 1 -- 172.21.15.154:0/1744222966 wait complete. 2023-09-23T03:28:09.538 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:28:09.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.733+0000 7f24c520e640 1 Processor -- start 2023-09-23T03:28:09.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.733+0000 7f24c520e640 1 -- start start 2023-09-23T03:28:09.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.734+0000 7f24c520e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24c015ad70 0x7f24c00c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:09.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.734+0000 7f24c520e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f24c00b0780 0x7f24c006c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:09.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.734+0000 7f24c520e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24c0070b40 con 0x7f24c00ad9c0 2023-09-23T03:28:09.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.734+0000 7f24c520e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24c0071100 con 0x7f24c00b0780 2023-09-23T03:28:09.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.734+0000 7f24c520e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c00c9390 con 0x7f24c015ad70 2023-09-23T03:28:09.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.735+0000 7f24bf59e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f24c00b0780 0x7f24c006c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:09.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.735+0000 7f24bed9d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24c015ad70 0x7f24c00c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:09.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.735+0000 7f24bf59e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f24c00b0780 0x7f24c006c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.154:42548/0 (socket says 172.21.15.154:42548) 2023-09-23T03:28:09.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.735+0000 7f24bf59e640 1 -- 172.21.15.154:0/1013434747 learned_addr learned my addr 172.21.15.154:0/1013434747 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:09.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.735+0000 7f24bed9d640 1 -- 172.21.15.154:0/1013434747 >> v2:172.21.15.190:3300/0 conn(0x7f24c00b0780 msgr2=0x7f24c006c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:09.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.735+0000 7f24bed9d640 1 --2- 172.21.15.154:0/1013434747 >> v2:172.21.15.190:3300/0 conn(0x7f24c00b0780 0x7f24c006c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:09.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.735+0000 7f24bed9d640 1 -- 172.21.15.154:0/1013434747 >> v1:172.21.15.190:6789/0 conn(0x7f24c00ad9c0 legacy=0x7f24c00b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:09.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.735+0000 7f24bed9d640 1 -- 172.21.15.154:0/1013434747 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c0070e20 con 0x7f24c015ad70 2023-09-23T03:28:09.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.735+0000 7f24bf59e640 1 --2- 172.21.15.154:0/1013434747 >> v2:172.21.15.190:3300/0 conn(0x7f24c00b0780 0x7f24c006c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:28:09.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.737+0000 7f24bed9d640 1 --2- 172.21.15.154:0/1013434747 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24c015ad70 0x7f24c00c8e50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f24b0009d90 tx=0x7f24b002be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f53266612af908af server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:09.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.737+0000 7f24be59c640 1 -- 172.21.15.154:0/1013434747 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b002cbd0 con 0x7f24c015ad70 2023-09-23T03:28:09.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.737+0000 7f24be59c640 1 -- 172.21.15.154:0/1013434747 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b002cd30 con 0x7f24c015ad70 2023-09-23T03:28:09.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.737+0000 7f24be59c640 1 -- 172.21.15.154:0/1013434747 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b002cea0 con 0x7f24c015ad70 2023-09-23T03:28:09.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.738+0000 7f24c520e640 1 -- 172.21.15.154:0/1013434747 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24c015ad70 msgr2=0x7f24c00c8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:09.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.738+0000 7f24c520e640 1 --2- 172.21.15.154:0/1013434747 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24c015ad70 0x7f24c00c8e50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f24b0009d90 tx=0x7f24b002be20 comp rx=0 tx=0).stop 2023-09-23T03:28:09.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.738+0000 7f24c520e640 1 -- 172.21.15.154:0/1013434747 shutdown_connections 2023-09-23T03:28:09.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.738+0000 7f24c520e640 1 --2- 172.21.15.154:0/1013434747 >> v2:172.21.15.190:3300/0 conn(0x7f24c00b0780 0x7f24c006c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:09.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.738+0000 7f24c520e640 1 --2- 172.21.15.154:0/1013434747 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24c015ad70 0x7f24c00c8e50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:09.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.738+0000 7f24c520e640 1 -- 172.21.15.154:0/1013434747 >> 172.21.15.154:0/1013434747 conn(0x7f24c00cb070 msgr2=0x7f24c00cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:09.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.738+0000 7f24c520e640 1 -- 172.21.15.154:0/1013434747 shutdown_connections 2023-09-23T03:28:09.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.738+0000 7f24c520e640 1 -- 172.21.15.154:0/1013434747 wait complete. 2023-09-23T03:28:09.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.741+0000 7f24c520e640 1 Processor -- start 2023-09-23T03:28:09.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.741+0000 7f24c520e640 1 -- start start 2023-09-23T03:28:09.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.743+0000 7f24c520e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24c00ad9c0 0x7f24c015b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:09.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.743+0000 7f24c520e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24c00b0780 0x7f24c00c6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:09.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.743+0000 7f24c520e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24c015ad70 0x7f24c014d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:09.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.743+0000 7f24c520e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c00c6940 con 0x7f24c015ad70 2023-09-23T03:28:09.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.743+0000 7f24c520e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c015ba80 con 0x7f24c00b0780 2023-09-23T03:28:09.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.743+0000 7f24c520e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c015bd60 con 0x7f24c00ad9c0 2023-09-23T03:28:09.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.743+0000 7f24bf59e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24c015ad70 0x7f24c014d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:09.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.743+0000 7f24bf59e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24c015ad70 0x7f24c014d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44966/0 (socket says 172.21.15.154:44966) 2023-09-23T03:28:09.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.743+0000 7f24bf59e640 1 -- 172.21.15.154:0/3715423916 learned_addr learned my addr 172.21.15.154:0/3715423916 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:09.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.743+0000 7f24bed9d640 1 --2- 172.21.15.154:0/3715423916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24c00ad9c0 0x7f24c015b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:09.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.743+0000 7f24b659c640 1 --2- 172.21.15.154:0/3715423916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24c00b0780 0x7f24c00c6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:09.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.743+0000 7f24bf59e640 1 -- 172.21.15.154:0/3715423916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24c00ad9c0 msgr2=0x7f24c015b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:09.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.743+0000 7f24bf59e640 1 --2- 172.21.15.154:0/3715423916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24c00ad9c0 0x7f24c015b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:09.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.743+0000 7f24bf59e640 1 -- 172.21.15.154:0/3715423916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24c00b0780 msgr2=0x7f24c00c6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:09.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.743+0000 7f24bf59e640 1 --2- 172.21.15.154:0/3715423916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24c00b0780 0x7f24c00c6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:09.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.743+0000 7f24bf59e640 1 -- 172.21.15.154:0/3715423916 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b80094d0 con 0x7f24c015ad70 2023-09-23T03:28:09.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.743+0000 7f24bf59e640 1 --2- 172.21.15.154:0/3715423916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24c015ad70 0x7f24c014d930 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f24b800e930 tx=0x7f24b800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:09.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.743+0000 7f24b7fff640 1 -- 172.21.15.154:0/3715423916 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b8009720 con 0x7f24c015ad70 2023-09-23T03:28:09.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.743+0000 7f24b7fff640 1 -- 172.21.15.154:0/3715423916 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b800ca00 con 0x7f24c015ad70 2023-09-23T03:28:09.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.743+0000 7f24b7fff640 1 -- 172.21.15.154:0/3715423916 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b800cd90 con 0x7f24c015ad70 2023-09-23T03:28:09.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.743+0000 7f24c520e640 1 -- 172.21.15.154:0/3715423916 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24b0009a70 con 0x7f24c015ad70 2023-09-23T03:28:09.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.743+0000 7f24c520e640 1 -- 172.21.15.154:0/3715423916 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24c015c320 con 0x7f24c015ad70 2023-09-23T03:28:09.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.745+0000 7f24c520e640 1 -- 172.21.15.154:0/3715423916 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24c0067870 con 0x7f24c015ad70 2023-09-23T03:28:09.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.745+0000 7f24b7fff640 1 -- 172.21.15.154:0/3715423916 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f24b8013470 con 0x7f24c015ad70 2023-09-23T03:28:09.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.746+0000 7f24b7fff640 1 --2- 172.21.15.154:0/3715423916 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f24a006af60 0x7f24a006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:09.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.746+0000 7f24b7fff640 1 -- 172.21.15.154:0/3715423916 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f24b80894e0 con 0x7f24c015ad70 2023-09-23T03:28:09.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.747+0000 7f24bed9d640 1 --2- 172.21.15.154:0/3715423916 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f24a006af60 0x7f24a006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:09.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.747+0000 7f24bed9d640 1 --2- 172.21.15.154:0/3715423916 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f24a006af60 0x7f24a006d420 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f24b0036000 tx=0x7f24b0030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:09.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.751+0000 7f24b7fff640 1 -- 172.21.15.154:0/3715423916 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f24b8054380 con 0x7f24c015ad70 2023-09-23T03:28:09.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.980+0000 7f24c520e640 1 -- 172.21.15.154:0/3715423916 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24c0073eb0 con 0x7f24c015ad70 2023-09-23T03:28:09.983 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:28:09.983 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":111,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/2759187683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2759187683},{"type":"v1","addr":"172.21.15.154:6835","nonce":2759187683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":5924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3712825179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3712825179},{"type":"v1","addr":"172.21.15.190:6835","nonce":3712825179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":110},{"gid":6544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/4147378242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":4147378242},{"type":"v1","addr":"172.21.15.190:6834","nonce":4147378242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:27:44.470312+0000","modified":"2023-09-23T03:28:09.516686+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8907},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8907":{"gid":8907,"name":"c","rank":0,"incarnation":111,"state":"up:replay","state_seq":1,"addr":"172.21.15.154:6837/4006549651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4006549651},{"type":"v1","addr":"172.21.15.154:6837","nonce":4006549651}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-09-23T03:28:09.983 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 111 2023-09-23T03:28:10.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.982+0000 7f24b7fff640 1 -- 172.21.15.154:0/3715423916 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+4935 (secure 0 0 0) 0x7f24b80579a0 con 0x7f24c015ad70 2023-09-23T03:28:10.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.984+0000 7f24c520e640 1 -- 172.21.15.154:0/3715423916 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f24a006af60 msgr2=0x7f24a006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:10.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.984+0000 7f24c520e640 1 --2- 172.21.15.154:0/3715423916 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f24a006af60 0x7f24a006d420 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f24b0036000 tx=0x7f24b0030000 comp rx=0 tx=0).stop 2023-09-23T03:28:10.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.984+0000 7f24c520e640 1 -- 172.21.15.154:0/3715423916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24c015ad70 msgr2=0x7f24c014d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:10.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.984+0000 7f24c520e640 1 --2- 172.21.15.154:0/3715423916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24c015ad70 0x7f24c014d930 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f24b800e930 tx=0x7f24b800edf0 comp rx=0 tx=0).stop 2023-09-23T03:28:10.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.984+0000 7f24c520e640 1 -- 172.21.15.154:0/3715423916 shutdown_connections 2023-09-23T03:28:10.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.984+0000 7f24c520e640 1 --2- 172.21.15.154:0/3715423916 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f24a006af60 0x7f24a006d420 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:10.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.984+0000 7f24c520e640 1 --2- 172.21.15.154:0/3715423916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f24c015ad70 0x7f24c014d930 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:10.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.984+0000 7f24c520e640 1 --2- 172.21.15.154:0/3715423916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f24c00b0780 0x7f24c00c6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:10.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.984+0000 7f24c520e640 1 --2- 172.21.15.154:0/3715423916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f24c00ad9c0 0x7f24c015b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:10.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.984+0000 7f24c520e640 1 -- 172.21.15.154:0/3715423916 >> 172.21.15.154:0/3715423916 conn(0x7f24c00cb070 msgr2=0x7f24c00b2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:10.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.984+0000 7f24c520e640 1 -- 172.21.15.154:0/3715423916 shutdown_connections 2023-09-23T03:28:10.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:09.984+0000 7f24c520e640 1 -- 172.21.15.154:0/3715423916 wait complete. 2023-09-23T03:28:10.423 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 111, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-23T03:27:44.470312+0000', 'modified': '2023-09-23T03:28:09.516686+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 127, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8907}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8907': {'gid': 8907, 'name': 'c', 'rank': 0, 'incarnation': 111, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.154:6837/4006549651', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6836', 'nonce': 4006549651}, {'type': 'v1', 'addr': '172.21.15.154:6837', 'nonce': 4006549651}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-23T03:28:10.423 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8907:up:replay 2023-09-23T03:28:11.425 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:28:11.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.650+0000 7f022e716640 1 Processor -- start 2023-09-23T03:28:11.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.650+0000 7f022e716640 1 -- start start 2023-09-23T03:28:11.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.652+0000 7f022e716640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f02280ad9c0 0x7f02280c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:11.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.652+0000 7f022e716640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0228070b40 con 0x7f022815ad70 2023-09-23T03:28:11.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.652+0000 7f022e716640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0228070e20 con 0x7f02280b0780 2023-09-23T03:28:11.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.652+0000 7f022e716640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0228071100 con 0x7f02280ad9c0 2023-09-23T03:28:11.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.652+0000 7f021ffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f02280ad9c0 0x7f02280c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:11.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.652+0000 7f0227fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f022815ad70 0x7f02280b0070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:42844/0 (socket says 172.21.15.154:42844) 2023-09-23T03:28:11.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.652+0000 7f0227fff640 1 -- 172.21.15.154:0/326016316 learned_addr learned my addr 172.21.15.154:0/326016316 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:28:11.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.653+0000 7f021ffff640 1 -- 172.21.15.154:0/326016316 >> v1:172.21.15.190:6789/0 conn(0x7f02280b0780 legacy=0x7f022806c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:11.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.653+0000 7f021ffff640 1 -- 172.21.15.154:0/326016316 >> v1:172.21.15.154:6789/0 conn(0x7f022815ad70 legacy=0x7f02280b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:11.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.653+0000 7f021ffff640 1 -- 172.21.15.154:0/326016316 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02280c96d0 con 0x7f02280ad9c0 2023-09-23T03:28:11.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.654+0000 7f021ffff640 1 --2- 172.21.15.154:0/326016316 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02280ad9c0 0x7f02280c8e50 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f02100097d0 tx=0x7f021002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d243b3376c5dd86 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:11.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.654+0000 7f02277fe640 1 -- 172.21.15.154:0/326016316 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021002d040 con 0x7f02280ad9c0 2023-09-23T03:28:11.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.654+0000 7f02277fe640 1 -- 172.21.15.154:0/326016316 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0210031810 con 0x7f02280ad9c0 2023-09-23T03:28:11.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.654+0000 7f02277fe640 1 -- 172.21.15.154:0/326016316 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0210031ba0 con 0x7f02280ad9c0 2023-09-23T03:28:11.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.655+0000 7f022e716640 1 -- 172.21.15.154:0/326016316 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02280ad9c0 msgr2=0x7f02280c8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:11.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.655+0000 7f022e716640 1 --2- 172.21.15.154:0/326016316 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02280ad9c0 0x7f02280c8e50 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f02100097d0 tx=0x7f021002b8b0 comp rx=0 tx=0).stop 2023-09-23T03:28:11.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.655+0000 7f022e716640 1 -- 172.21.15.154:0/326016316 shutdown_connections 2023-09-23T03:28:11.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.655+0000 7f022e716640 1 --2- 172.21.15.154:0/326016316 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02280ad9c0 0x7f02280c8e50 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:11.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.655+0000 7f022e716640 1 -- 172.21.15.154:0/326016316 >> 172.21.15.154:0/326016316 conn(0x7f02280cb070 msgr2=0x7f02280cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:11.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.655+0000 7f022e716640 1 -- 172.21.15.154:0/326016316 shutdown_connections 2023-09-23T03:28:11.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.655+0000 7f022e716640 1 -- 172.21.15.154:0/326016316 wait complete. 2023-09-23T03:28:11.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.657+0000 7f022e716640 1 Processor -- start 2023-09-23T03:28:11.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.657+0000 7f022e716640 1 -- start start 2023-09-23T03:28:11.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.658+0000 7f022e716640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02280ad9c0 0x7f0228153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:11.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.658+0000 7f022e716640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02280b0780 0x7f02280c46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:11.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.658+0000 7f022e716640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f022815ad70 0x7f02280c2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:11.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.658+0000 7f022e716640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022815b2d0 con 0x7f02280ad9c0 2023-09-23T03:28:11.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.658+0000 7f022e716640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02280ae140 con 0x7f022815ad70 2023-09-23T03:28:11.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.658+0000 7f022e716640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02280ae3e0 con 0x7f02280b0780 2023-09-23T03:28:11.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.658+0000 7f0227fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02280ad9c0 0x7f0228153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:11.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.658+0000 7f0227fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02280ad9c0 0x7f0228153fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44986/0 (socket says 172.21.15.154:44986) 2023-09-23T03:28:11.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.658+0000 7f0227fff640 1 -- 172.21.15.154:0/2238706794 learned_addr learned my addr 172.21.15.154:0/2238706794 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:11.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.659+0000 7f022ccb3640 1 --2- 172.21.15.154:0/2238706794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f022815ad70 0x7f02280c2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:11.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.659+0000 7f0227fff640 1 -- 172.21.15.154:0/2238706794 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02280b0780 msgr2=0x7f02280c46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:11.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.659+0000 7f0227fff640 1 --2- 172.21.15.154:0/2238706794 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02280b0780 0x7f02280c46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:11.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.659+0000 7f0227fff640 1 -- 172.21.15.154:0/2238706794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f022815ad70 msgr2=0x7f02280c2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:11.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.659+0000 7f0227fff640 1 --2- 172.21.15.154:0/2238706794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f022815ad70 0x7f02280c2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:11.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.659+0000 7f0227fff640 1 -- 172.21.15.154:0/2238706794 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0210009400 con 0x7f02280ad9c0 2023-09-23T03:28:11.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.659+0000 7f0227fff640 1 --2- 172.21.15.154:0/2238706794 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02280ad9c0 0x7f0228153fa0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f021800c480 tx=0x7f021800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:11.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.659+0000 7f02257fa640 1 -- 172.21.15.154:0/2238706794 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0218005ce0 con 0x7f02280ad9c0 2023-09-23T03:28:11.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.659+0000 7f02257fa640 1 -- 172.21.15.154:0/2238706794 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0218005e40 con 0x7f02280ad9c0 2023-09-23T03:28:11.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.659+0000 7f02257fa640 1 -- 172.21.15.154:0/2238706794 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02180072f0 con 0x7f02280ad9c0 2023-09-23T03:28:11.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.659+0000 7f022e716640 1 -- 172.21.15.154:0/2238706794 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02280ae6c0 con 0x7f02280ad9c0 2023-09-23T03:28:11.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.660+0000 7f022e716640 1 -- 172.21.15.154:0/2238706794 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02280aec80 con 0x7f02280ad9c0 2023-09-23T03:28:11.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.661+0000 7f022e716640 1 -- 172.21.15.154:0/2238706794 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0228067870 con 0x7f02280ad9c0 2023-09-23T03:28:11.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.662+0000 7f02257fa640 1 -- 172.21.15.154:0/2238706794 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f0218007510 con 0x7f02280ad9c0 2023-09-23T03:28:11.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.662+0000 7f02257fa640 1 --2- 172.21.15.154:0/2238706794 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f020806af30 0x7f020806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:11.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.663+0000 7f02257fa640 1 -- 172.21.15.154:0/2238706794 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f0218088e20 con 0x7f02280ad9c0 2023-09-23T03:28:11.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.663+0000 7f021ffff640 1 --2- 172.21.15.154:0/2238706794 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f020806af30 0x7f020806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:11.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.663+0000 7f021ffff640 1 --2- 172.21.15.154:0/2238706794 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f020806af30 0x7f020806d3f0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f0210036000 tx=0x7f0210030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:11.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.667+0000 7f02257fa640 1 -- 172.21.15.154:0/2238706794 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f0218053d60 con 0x7f02280ad9c0 2023-09-23T03:28:11.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.892+0000 7f022e716640 1 -- 172.21.15.154:0/2238706794 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0228073eb0 con 0x7f02280ad9c0 2023-09-23T03:28:11.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.893+0000 7f02257fa640 1 -- 172.21.15.154:0/2238706794 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+4936 (secure 0 0 0) 0x7f0218017020 con 0x7f02280ad9c0 2023-09-23T03:28:11.894 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:28:11.894 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":113,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/2759187683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2759187683},{"type":"v1","addr":"172.21.15.154:6835","nonce":2759187683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":5924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3712825179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3712825179},{"type":"v1","addr":"172.21.15.190:6835","nonce":3712825179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":110},{"gid":6544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/4147378242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":4147378242},{"type":"v1","addr":"172.21.15.190:6834","nonce":4147378242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:27:44.470312+0000","modified":"2023-09-23T03:28:10.536761+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8907},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8907":{"gid":8907,"name":"c","rank":0,"incarnation":111,"state":"up:rejoin","state_seq":10,"addr":"172.21.15.154:6837/4006549651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4006549651},{"type":"v1","addr":"172.21.15.154:6837","nonce":4006549651}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-09-23T03:28:11.894 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 113 2023-09-23T03:28:11.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.896+0000 7f022e716640 1 -- 172.21.15.154:0/2238706794 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f020806af30 msgr2=0x7f020806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:11.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.896+0000 7f022e716640 1 --2- 172.21.15.154:0/2238706794 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f020806af30 0x7f020806d3f0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f0210036000 tx=0x7f0210030000 comp rx=0 tx=0).stop 2023-09-23T03:28:11.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.896+0000 7f022e716640 1 -- 172.21.15.154:0/2238706794 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02280ad9c0 msgr2=0x7f0228153fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:11.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.896+0000 7f022e716640 1 --2- 172.21.15.154:0/2238706794 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02280ad9c0 0x7f0228153fa0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f021800c480 tx=0x7f021800c940 comp rx=0 tx=0).stop 2023-09-23T03:28:11.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.896+0000 7f022e716640 1 -- 172.21.15.154:0/2238706794 shutdown_connections 2023-09-23T03:28:11.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.896+0000 7f022e716640 1 --2- 172.21.15.154:0/2238706794 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f020806af30 0x7f020806d3f0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:11.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.896+0000 7f022e716640 1 --2- 172.21.15.154:0/2238706794 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f022815ad70 0x7f02280c2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:11.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.896+0000 7f022e716640 1 --2- 172.21.15.154:0/2238706794 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f02280b0780 0x7f02280c46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:11.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.896+0000 7f022e716640 1 --2- 172.21.15.154:0/2238706794 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02280ad9c0 0x7f0228153fa0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:11.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.896+0000 7f022e716640 1 -- 172.21.15.154:0/2238706794 >> 172.21.15.154:0/2238706794 conn(0x7f02280cb070 msgr2=0x7f02280cd790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:11.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.896+0000 7f022e716640 1 -- 172.21.15.154:0/2238706794 shutdown_connections 2023-09-23T03:28:11.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:11.896+0000 7f022e716640 1 -- 172.21.15.154:0/2238706794 wait complete. 2023-09-23T03:28:12.317 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 113, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-23T03:27:44.470312+0000', 'modified': '2023-09-23T03:28:10.536761+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 127, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8907}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8907': {'gid': 8907, 'name': 'c', 'rank': 0, 'incarnation': 111, 'state': 'up:rejoin', 'state_seq': 10, 'addr': '172.21.15.154:6837/4006549651', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6836', 'nonce': 4006549651}, {'type': 'v1', 'addr': '172.21.15.154:6837', 'nonce': 4006549651}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-23T03:28:12.318 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8907:up:rejoin 2023-09-23T03:28:13.318 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:28:13.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.548+0000 7f89f7c90640 1 Processor -- start 2023-09-23T03:28:13.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.548+0000 7f89f7c90640 1 -- start start 2023-09-23T03:28:13.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.550+0000 7f89f7c90640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89f015ad70 0x7f89f00c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:13.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.550+0000 7f89f7c90640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f89f00b0780 0x7f89f006c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:13.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.550+0000 7f89f7c90640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89f0070b40 con 0x7f89f00ad9c0 2023-09-23T03:28:13.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.550+0000 7f89f7c90640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89f0071100 con 0x7f89f00b0780 2023-09-23T03:28:13.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.550+0000 7f89f7c90640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f00c9390 con 0x7f89f015ad70 2023-09-23T03:28:13.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.550+0000 7f89f522b640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f89f00ad9c0 0x7f89f00b0070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:42854/0 (socket says 172.21.15.154:42854) 2023-09-23T03:28:13.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.550+0000 7f89f522b640 1 -- 172.21.15.154:0/738657481 learned_addr learned my addr 172.21.15.154:0/738657481 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:28:13.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.551+0000 7f89f5a2c640 1 --2- 172.21.15.154:0/738657481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89f015ad70 0x7f89f00c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:13.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.551+0000 7f89f622d640 1 --2- 172.21.15.154:0/738657481 >> v2:172.21.15.190:3300/0 conn(0x7f89f00b0780 0x7f89f006c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:13.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.551+0000 7f89f622d640 1 -- 172.21.15.154:0/738657481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89f015ad70 msgr2=0x7f89f00c8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:13.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.551+0000 7f89f622d640 1 --2- 172.21.15.154:0/738657481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89f015ad70 0x7f89f00c8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:13.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.551+0000 7f89f622d640 1 -- 172.21.15.154:0/738657481 >> v1:172.21.15.154:6789/0 conn(0x7f89f00ad9c0 legacy=0x7f89f00b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:13.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.552+0000 7f89f622d640 1 -- 172.21.15.154:0/738657481 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f0070e20 con 0x7f89f00b0780 2023-09-23T03:28:13.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.552+0000 7f89f5a2c640 1 --2- 172.21.15.154:0/738657481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89f015ad70 0x7f89f00c8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:28:13.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.553+0000 7f89f622d640 1 --2- 172.21.15.154:0/738657481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89f00b0780 0x7f89f006c8c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f89e8009d90 tx=0x7f89e802bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32e2e831d2696bf3 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:13.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.557+0000 7f89f4a2a640 1 -- 172.21.15.154:0/738657481 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e802cc50 con 0x7f89f00b0780 2023-09-23T03:28:13.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.558+0000 7f89f4a2a640 1 -- 172.21.15.154:0/738657481 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e802cdb0 con 0x7f89f00b0780 2023-09-23T03:28:13.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.558+0000 7f89f4a2a640 1 -- 172.21.15.154:0/738657481 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e8034660 con 0x7f89f00b0780 2023-09-23T03:28:13.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.558+0000 7f89f7c90640 1 -- 172.21.15.154:0/738657481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89f00b0780 msgr2=0x7f89f006c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:13.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.558+0000 7f89f7c90640 1 --2- 172.21.15.154:0/738657481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89f00b0780 0x7f89f006c8c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f89e8009d90 tx=0x7f89e802bea0 comp rx=0 tx=0).stop 2023-09-23T03:28:13.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.558+0000 7f89f7c90640 1 -- 172.21.15.154:0/738657481 shutdown_connections 2023-09-23T03:28:13.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.558+0000 7f89f7c90640 1 --2- 172.21.15.154:0/738657481 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89f00b0780 0x7f89f006c8c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:13.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.558+0000 7f89f7c90640 1 --2- 172.21.15.154:0/738657481 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89f015ad70 0x7f89f00c8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:13.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.558+0000 7f89f7c90640 1 -- 172.21.15.154:0/738657481 >> 172.21.15.154:0/738657481 conn(0x7f89f00cb070 msgr2=0x7f89f00cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:13.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.558+0000 7f89f7c90640 1 -- 172.21.15.154:0/738657481 shutdown_connections 2023-09-23T03:28:13.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.559+0000 7f89f7c90640 1 -- 172.21.15.154:0/738657481 wait complete. 2023-09-23T03:28:13.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.560+0000 7f89f7c90640 1 Processor -- start 2023-09-23T03:28:13.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.561+0000 7f89f7c90640 1 -- start start 2023-09-23T03:28:13.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.561+0000 7f89f7c90640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89f00ad9c0 0x7f89f0155c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:13.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.561+0000 7f89f7c90640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89f00b0780 0x7f89f00c3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:13.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.562+0000 7f89f7c90640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89f015ad70 0x7f89f0156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:13.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.562+0000 7f89f7c90640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f00c3860 con 0x7f89f015ad70 2023-09-23T03:28:13.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.562+0000 7f89f7c90640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f00c6340 con 0x7f89f00ad9c0 2023-09-23T03:28:13.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.562+0000 7f89f7c90640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f00c3210 con 0x7f89f00b0780 2023-09-23T03:28:13.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.562+0000 7f89f622d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89f015ad70 0x7f89f0156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:13.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.562+0000 7f89f622d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89f015ad70 0x7f89f0156950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45000/0 (socket says 172.21.15.154:45000) 2023-09-23T03:28:13.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.562+0000 7f89f622d640 1 -- 172.21.15.154:0/3411700408 learned_addr learned my addr 172.21.15.154:0/3411700408 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:13.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.562+0000 7f89f5a2c640 1 --2- 172.21.15.154:0/3411700408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89f00ad9c0 0x7f89f0155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:13.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.562+0000 7f89f522b640 1 --2- 172.21.15.154:0/3411700408 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89f00b0780 0x7f89f00c3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:13.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.562+0000 7f89f622d640 1 -- 172.21.15.154:0/3411700408 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89f00b0780 msgr2=0x7f89f00c3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:13.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.562+0000 7f89f622d640 1 --2- 172.21.15.154:0/3411700408 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89f00b0780 0x7f89f00c3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:13.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.562+0000 7f89f622d640 1 -- 172.21.15.154:0/3411700408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89f00ad9c0 msgr2=0x7f89f0155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:13.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.562+0000 7f89f622d640 1 --2- 172.21.15.154:0/3411700408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89f00ad9c0 0x7f89f0155c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:13.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.562+0000 7f89f622d640 1 -- 172.21.15.154:0/3411700408 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89ec009400 con 0x7f89f015ad70 2023-09-23T03:28:13.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.562+0000 7f89f522b640 1 --2- 172.21.15.154:0/3411700408 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89f00b0780 0x7f89f00c3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:13.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.563+0000 7f89f622d640 1 --2- 172.21.15.154:0/3411700408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89f015ad70 0x7f89f0156950 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f89e8009d90 tx=0x7f89e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:13.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.563+0000 7f89e67fc640 1 -- 172.21.15.154:0/3411700408 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e8040650 con 0x7f89f015ad70 2023-09-23T03:28:13.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.563+0000 7f89e67fc640 1 -- 172.21.15.154:0/3411700408 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e80407b0 con 0x7f89f015ad70 2023-09-23T03:28:13.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.563+0000 7f89e67fc640 1 -- 172.21.15.154:0/3411700408 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e8040b60 con 0x7f89f015ad70 2023-09-23T03:28:13.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.563+0000 7f89f7c90640 1 -- 172.21.15.154:0/3411700408 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e8009a70 con 0x7f89f015ad70 2023-09-23T03:28:13.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.563+0000 7f89f7c90640 1 -- 172.21.15.154:0/3411700408 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f0156140 con 0x7f89f015ad70 2023-09-23T03:28:13.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.564+0000 7f89f7c90640 1 -- 172.21.15.154:0/3411700408 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89f0067870 con 0x7f89f015ad70 2023-09-23T03:28:13.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.565+0000 7f89e67fc640 1 -- 172.21.15.154:0/3411700408 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f89e8040cc0 con 0x7f89f015ad70 2023-09-23T03:28:13.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.566+0000 7f89e67fc640 1 --2- 172.21.15.154:0/3411700408 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f89d406af30 0x7f89d406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:13.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.566+0000 7f89e67fc640 1 -- 172.21.15.154:0/3411700408 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f89e80ac6f0 con 0x7f89f015ad70 2023-09-23T03:28:13.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.566+0000 7f89f5a2c640 1 --2- 172.21.15.154:0/3411700408 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f89d406af30 0x7f89d406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:13.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.567+0000 7f89f5a2c640 1 --2- 172.21.15.154:0/3411700408 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f89d406af30 0x7f89d406d3f0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f89ec009b80 tx=0x7f89ec008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:13.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.570+0000 7f89e67fc640 1 -- 172.21.15.154:0/3411700408 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f89e80774f0 con 0x7f89f015ad70 2023-09-23T03:28:13.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.795+0000 7f89f7c90640 1 -- 172.21.15.154:0/3411700408 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89f00c9960 con 0x7f89f015ad70 2023-09-23T03:28:13.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.796+0000 7f89e67fc640 1 -- 172.21.15.154:0/3411700408 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+4936 (secure 0 0 0) 0x7f89e8009450 con 0x7f89f015ad70 2023-09-23T03:28:13.796 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:28:13.797 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":114,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/2759187683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2759187683},{"type":"v1","addr":"172.21.15.154:6835","nonce":2759187683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":5924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3712825179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3712825179},{"type":"v1","addr":"172.21.15.190:6835","nonce":3712825179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":110},{"gid":6544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/4147378242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":4147378242},{"type":"v1","addr":"172.21.15.190:6834","nonce":4147378242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:27:44.470312+0000","modified":"2023-09-23T03:28:11.546387+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8907},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8907":{"gid":8907,"name":"c","rank":0,"incarnation":111,"state":"up:active","state_seq":11,"addr":"172.21.15.154:6837/4006549651","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":4006549651},{"type":"v1","addr":"172.21.15.154:6837","nonce":4006549651}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-09-23T03:28:13.797 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 114 2023-09-23T03:28:13.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.798+0000 7f89f7c90640 1 -- 172.21.15.154:0/3411700408 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f89d406af30 msgr2=0x7f89d406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:13.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.798+0000 7f89f7c90640 1 --2- 172.21.15.154:0/3411700408 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f89d406af30 0x7f89d406d3f0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f89ec009b80 tx=0x7f89ec008040 comp rx=0 tx=0).stop 2023-09-23T03:28:13.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.798+0000 7f89f7c90640 1 -- 172.21.15.154:0/3411700408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89f015ad70 msgr2=0x7f89f0156950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:13.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.798+0000 7f89f7c90640 1 --2- 172.21.15.154:0/3411700408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89f015ad70 0x7f89f0156950 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f89e8009d90 tx=0x7f89e8004a40 comp rx=0 tx=0).stop 2023-09-23T03:28:13.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.799+0000 7f89f7c90640 1 -- 172.21.15.154:0/3411700408 shutdown_connections 2023-09-23T03:28:13.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.799+0000 7f89f7c90640 1 --2- 172.21.15.154:0/3411700408 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f89d406af30 0x7f89d406d3f0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:13.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.799+0000 7f89f7c90640 1 --2- 172.21.15.154:0/3411700408 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89f015ad70 0x7f89f0156950 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:13.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.799+0000 7f89f7c90640 1 --2- 172.21.15.154:0/3411700408 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f89f00b0780 0x7f89f00c3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:13.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.799+0000 7f89f7c90640 1 --2- 172.21.15.154:0/3411700408 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f89f00ad9c0 0x7f89f0155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:13.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.799+0000 7f89f7c90640 1 -- 172.21.15.154:0/3411700408 >> 172.21.15.154:0/3411700408 conn(0x7f89f00cb070 msgr2=0x7f89f00af120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:13.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.799+0000 7f89f7c90640 1 -- 172.21.15.154:0/3411700408 shutdown_connections 2023-09-23T03:28:13.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:13.799+0000 7f89f7c90640 1 -- 172.21.15.154:0/3411700408 wait complete. 2023-09-23T03:28:14.220 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 114, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-23T03:27:44.470312+0000', 'modified': '2023-09-23T03:28:11.546387+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 127, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8907}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8907': {'gid': 8907, 'name': 'c', 'rank': 0, 'incarnation': 111, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.154:6837/4006549651', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6836', 'nonce': 4006549651}, {'type': 'v1', 'addr': '172.21.15.154:6837', 'nonce': 4006549651}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-23T03:28:14.220 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-23T03:28:14.220 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-09-23T03:28:14.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.440+0000 7f3704dae640 1 Processor -- start 2023-09-23T03:28:14.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.440+0000 7f3704dae640 1 -- start start 2023-09-23T03:28:14.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.441+0000 7f3704dae640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37000ce230 0x7f37000b5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:14.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.441+0000 7f3704dae640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f37000d10e0 0x7f37000b5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:14.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.441+0000 7f3704dae640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37000b3d20 con 0x7f37001600b0 2023-09-23T03:28:14.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.441+0000 7f3704dae640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37000b3a40 con 0x7f37000d10e0 2023-09-23T03:28:14.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.441+0000 7f3704dae640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37000ce950 con 0x7f37000ce230 2023-09-23T03:28:14.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.442+0000 7f36ff7fe640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f37001600b0 0x7f37000d09d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.154:59888/0 (socket says 172.21.15.154:59888) 2023-09-23T03:28:14.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.442+0000 7f36ff7fe640 1 -- 172.21.15.154:0/2719544172 learned_addr learned my addr 172.21.15.154:0/2719544172 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:28:14.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.442+0000 7f36fffff640 1 --2- 172.21.15.154:0/2719544172 >> v2:172.21.15.190:3300/0 conn(0x7f37000d10e0 0x7f37000b5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:14.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.442+0000 7f36feffd640 1 --2- 172.21.15.154:0/2719544172 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37000ce230 0x7f37000b5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:14.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.442+0000 7f36fe7fc640 1 -- 172.21.15.154:0/2719544172 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3999010233 0 0) 0x7f37000b3d20 con 0x7f37001600b0 2023-09-23T03:28:14.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.442+0000 7f36fe7fc640 1 -- 172.21.15.154:0/2719544172 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36ec003520 con 0x7f37001600b0 2023-09-23T03:28:14.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.443+0000 7f36fffff640 1 -- 172.21.15.154:0/2719544172 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37000ce230 msgr2=0x7f37000b5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:14.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.443+0000 7f36fffff640 1 --2- 172.21.15.154:0/2719544172 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37000ce230 0x7f37000b5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.443+0000 7f36fffff640 1 -- 172.21.15.154:0/2719544172 >> v1:172.21.15.190:6789/0 conn(0x7f37001600b0 legacy=0x7f37000d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:14.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.443+0000 7f36fffff640 1 -- 172.21.15.154:0/2719544172 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37000cec30 con 0x7f37000d10e0 2023-09-23T03:28:14.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.444+0000 7f36fffff640 1 --2- 172.21.15.154:0/2719544172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37000d10e0 0x7f37000b5d70 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f36f80097a0 tx=0x7f36f802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d12f8b20cbf3819a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:14.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.444+0000 7f36fe7fc640 1 -- 172.21.15.154:0/2719544172 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f802e070 con 0x7f37000d10e0 2023-09-23T03:28:14.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.444+0000 7f36fe7fc640 1 -- 172.21.15.154:0/2719544172 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f80315a0 con 0x7f37000d10e0 2023-09-23T03:28:14.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.444+0000 7f36fe7fc640 1 -- 172.21.15.154:0/2719544172 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f8031910 con 0x7f37000d10e0 2023-09-23T03:28:14.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.445+0000 7f3704dae640 1 -- 172.21.15.154:0/2719544172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37000d10e0 msgr2=0x7f37000b5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:14.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.445+0000 7f3704dae640 1 --2- 172.21.15.154:0/2719544172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37000d10e0 0x7f37000b5d70 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f36f80097a0 tx=0x7f36f802b840 comp rx=0 tx=0).stop 2023-09-23T03:28:14.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.445+0000 7f3704dae640 1 -- 172.21.15.154:0/2719544172 shutdown_connections 2023-09-23T03:28:14.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.445+0000 7f3704dae640 1 --2- 172.21.15.154:0/2719544172 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37000d10e0 0x7f37000b5d70 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.445+0000 7f3704dae640 1 --2- 172.21.15.154:0/2719544172 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37000ce230 0x7f37000b5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.445+0000 7f3704dae640 1 -- 172.21.15.154:0/2719544172 >> 172.21.15.154:0/2719544172 conn(0x7f370009bb30 msgr2=0x7f37000d27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:14.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.445+0000 7f3704dae640 1 -- 172.21.15.154:0/2719544172 shutdown_connections 2023-09-23T03:28:14.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.445+0000 7f3704dae640 1 -- 172.21.15.154:0/2719544172 wait complete. 2023-09-23T03:28:14.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.447+0000 7f3704dae640 1 Processor -- start 2023-09-23T03:28:14.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.447+0000 7f3704dae640 1 -- start start 2023-09-23T03:28:14.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.448+0000 7f3704dae640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37000ce230 0x7f3700152b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:14.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.448+0000 7f3704dae640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37000d10e0 0x7f37001f1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:14.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.448+0000 7f3704dae640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37001600b0 0x7f37001546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:14.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.448+0000 7f3704dae640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37000b58d0 con 0x7f37000d10e0 2023-09-23T03:28:14.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.448+0000 7f3704dae640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37001f2e80 con 0x7f37001600b0 2023-09-23T03:28:14.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.448+0000 7f3704dae640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37001f23c0 con 0x7f37000ce230 2023-09-23T03:28:14.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.449+0000 7f36feffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37000d10e0 0x7f37001f1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:14.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.449+0000 7f36feffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37000d10e0 0x7f37001f1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59550/0 (socket says 172.21.15.154:59550) 2023-09-23T03:28:14.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.449+0000 7f36feffd640 1 -- 172.21.15.154:0/730577678 learned_addr learned my addr 172.21.15.154:0/730577678 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:14.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.449+0000 7f36ff7fe640 1 --2- 172.21.15.154:0/730577678 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37000ce230 0x7f3700152b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:14.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.449+0000 7f36fffff640 1 --2- 172.21.15.154:0/730577678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37001600b0 0x7f37001546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:14.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.449+0000 7f36feffd640 1 -- 172.21.15.154:0/730577678 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37000ce230 msgr2=0x7f3700152b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:14.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.449+0000 7f36feffd640 1 --2- 172.21.15.154:0/730577678 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37000ce230 0x7f3700152b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.449+0000 7f36feffd640 1 -- 172.21.15.154:0/730577678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37001600b0 msgr2=0x7f37001546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:14.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.449+0000 7f36feffd640 1 --2- 172.21.15.154:0/730577678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37001600b0 0x7f37001546a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.449+0000 7f36feffd640 1 -- 172.21.15.154:0/730577678 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f8009400 con 0x7f37000d10e0 2023-09-23T03:28:14.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.449+0000 7f36feffd640 1 --2- 172.21.15.154:0/730577678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37000d10e0 0x7f37001f1e80 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f36f000efc0 tx=0x7f36f000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:14.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.450+0000 7f36f7fff640 1 -- 172.21.15.154:0/730577678 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f0014480 con 0x7f37000d10e0 2023-09-23T03:28:14.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.450+0000 7f36f7fff640 1 -- 172.21.15.154:0/730577678 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f000f040 con 0x7f37000d10e0 2023-09-23T03:28:14.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.450+0000 7f3704dae640 1 -- 172.21.15.154:0/730577678 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3700154be0 con 0x7f37000d10e0 2023-09-23T03:28:14.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.450+0000 7f36f7fff640 1 -- 172.21.15.154:0/730577678 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f0014800 con 0x7f37000d10e0 2023-09-23T03:28:14.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.450+0000 7f3704dae640 1 -- 172.21.15.154:0/730577678 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37001f28c0 con 0x7f37000d10e0 2023-09-23T03:28:14.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.453+0000 7f36f7fff640 1 -- 172.21.15.154:0/730577678 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f36f0014960 con 0x7f37000d10e0 2023-09-23T03:28:14.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.454+0000 7f36f7fff640 1 --2- 172.21.15.154:0/730577678 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f36e006af60 0x7f36e006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:14.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.454+0000 7f36f7fff640 1 -- 172.21.15.154:0/730577678 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f36f008a420 con 0x7f37000d10e0 2023-09-23T03:28:14.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.455+0000 7f36ff7fe640 1 --2- 172.21.15.154:0/730577678 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f36e006af60 0x7f36e006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:14.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.455+0000 7f3708ba1740 1 Processor -- start 2023-09-23T03:28:14.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.455+0000 7f3708ba1740 1 -- start start 2023-09-23T03:28:14.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.455+0000 7f36ff7fe640 1 --2- 172.21.15.154:0/730577678 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f36e006af60 0x7f36e006d420 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f36e80011e0 tx=0x7f36e8007d30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:14.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.456+0000 7f3708ba1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56465b142da0 0x56465b115ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:14.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.456+0000 7f3708ba1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56465b143180 0x56465b145b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:14.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.456+0000 7f3708ba1740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56465b1161e0 0x56465b146070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:14.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.456+0000 7f3708ba1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x56465b0896d0 con 0x56465b142da0 2023-09-23T03:28:14.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.456+0000 7f3708ba1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56465b086dc0 con 0x56465b143180 2023-09-23T03:28:14.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.456+0000 7f3708ba1740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56465b0826b0 con 0x56465b1161e0 2023-09-23T03:28:14.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.456+0000 7f36ff7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56465b142da0 0x56465b115ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:14.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.456+0000 7f36ff7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56465b142da0 0x56465b115ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59560/0 (socket says 172.21.15.154:59560) 2023-09-23T03:28:14.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.456+0000 7f36ff7fe640 1 -- 172.21.15.154:0/113594884 learned_addr learned my addr 172.21.15.154:0/113594884 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:14.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.457+0000 7f36ff7fe640 1 -- 172.21.15.154:0/113594884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56465b1161e0 msgr2=0x56465b146070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:14.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.457+0000 7f36fffff640 1 --2- 172.21.15.154:0/113594884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56465b1161e0 0x56465b146070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:14.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.457+0000 7f36feffd640 1 --2- 172.21.15.154:0/113594884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56465b143180 0x56465b145b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:14.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.457+0000 7f36ff7fe640 1 --2- 172.21.15.154:0/113594884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56465b1161e0 0x56465b146070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.457+0000 7f36ff7fe640 1 -- 172.21.15.154:0/113594884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56465b143180 msgr2=0x56465b145b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:14.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.457+0000 7f36ff7fe640 1 --2- 172.21.15.154:0/113594884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56465b143180 0x56465b145b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.457+0000 7f36ff7fe640 1 -- 172.21.15.154:0/113594884 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56465b0468e0 con 0x56465b142da0 2023-09-23T03:28:14.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.457+0000 7f36feffd640 1 --2- 172.21.15.154:0/113594884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56465b143180 0x56465b145b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:14.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.457+0000 7f36fffff640 1 --2- 172.21.15.154:0/113594884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56465b1161e0 0x56465b146070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:14.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.457+0000 7f36ff7fe640 1 --2- 172.21.15.154:0/113594884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56465b142da0 0x56465b115ca0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f36e8005a80 tx=0x7f36e8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db30c69151ba10e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:14.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.457+0000 7f36f57fa640 1 -- 172.21.15.154:0/113594884 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e8001410 con 0x56465b142da0 2023-09-23T03:28:14.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.457+0000 7f36f57fa640 1 -- 172.21.15.154:0/113594884 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e8001570 con 0x56465b142da0 2023-09-23T03:28:14.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.457+0000 7f36f57fa640 1 -- 172.21.15.154:0/113594884 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e80018e0 con 0x56465b142da0 2023-09-23T03:28:14.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.458+0000 7f3708ba1740 1 -- 172.21.15.154:0/113594884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56465b142da0 msgr2=0x56465b115ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:14.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.458+0000 7f3708ba1740 1 --2- 172.21.15.154:0/113594884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56465b142da0 0x56465b115ca0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f36e8005a80 tx=0x7f36e8012000 comp rx=0 tx=0).stop 2023-09-23T03:28:14.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.458+0000 7f3708ba1740 1 -- 172.21.15.154:0/113594884 shutdown_connections 2023-09-23T03:28:14.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.458+0000 7f3708ba1740 1 --2- 172.21.15.154:0/113594884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56465b1161e0 0x56465b146070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.458+0000 7f3708ba1740 1 --2- 172.21.15.154:0/113594884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56465b143180 0x56465b145b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.458+0000 7f3708ba1740 1 --2- 172.21.15.154:0/113594884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56465b142da0 0x56465b115ca0 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.458+0000 7f3708ba1740 1 -- 172.21.15.154:0/113594884 >> 172.21.15.154:0/113594884 conn(0x56465b117fb0 msgr2=0x56465b160b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:14.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.458+0000 7f3708ba1740 1 -- 172.21.15.154:0/113594884 shutdown_connections 2023-09-23T03:28:14.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.458+0000 7f3708ba1740 1 -- 172.21.15.154:0/113594884 wait complete. 2023-09-23T03:28:14.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.459+0000 7f3708ba1740 1 -- start start 2023-09-23T03:28:14.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.459+0000 7f3708ba1740 1 Processor -- start 2023-09-23T03:28:14.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.460+0000 7f3708ba1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56465b142da0 0x56465b15e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:14.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.460+0000 7f3708ba1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56465b143180 0x56465b15a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:14.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.460+0000 7f3708ba1740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56465b15ee90 0x56465b15aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:14.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.460+0000 7f3708ba1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x56465b020940 con 0x56465b142da0 2023-09-23T03:28:14.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.460+0000 7f3708ba1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56465b0205e0 con 0x56465b143180 2023-09-23T03:28:14.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.460+0000 7f3708ba1740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56465ade6f20 con 0x56465b15ee90 2023-09-23T03:28:14.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.460+0000 7f3708ba1740 10 client.0 _ll_register_callbacks cb 0x56465b05e5e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:28:14.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.460+0000 7f36ff7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56465b142da0 0x56465b15e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:14.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.461+0000 7f36ff7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56465b142da0 0x56465b15e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59566/0 (socket says 172.21.15.154:59566) 2023-09-23T03:28:14.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.461+0000 7f36ff7fe640 1 -- 172.21.15.154:0/2365793057 learned_addr learned my addr 172.21.15.154:0/2365793057 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:14.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.461+0000 7f36feffd640 1 --2- 172.21.15.154:0/2365793057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56465b143180 0x56465b15a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:14.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.461+0000 7f36fffff640 1 --2- 172.21.15.154:0/2365793057 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56465b15ee90 0x56465b15aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:14.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.461+0000 7f36ff7fe640 1 -- 172.21.15.154:0/2365793057 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56465b15ee90 msgr2=0x56465b15aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:14.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.461+0000 7f36ff7fe640 1 --2- 172.21.15.154:0/2365793057 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56465b15ee90 0x56465b15aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.461+0000 7f36ff7fe640 1 -- 172.21.15.154:0/2365793057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56465b143180 msgr2=0x56465b15a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:14.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.461+0000 7f36ff7fe640 1 --2- 172.21.15.154:0/2365793057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56465b143180 0x56465b15a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.461+0000 7f36ff7fe640 1 -- 172.21.15.154:0/2365793057 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36ec003520 con 0x56465b142da0 2023-09-23T03:28:14.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.461+0000 7f36ff7fe640 1 --2- 172.21.15.154:0/2365793057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56465b142da0 0x56465b15e950 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f36e8006400 tx=0x7f36e8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83ec6fed9fa13604 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:14.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.461+0000 7f36de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:28:14.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.461+0000 7f36de7fc640 1 -- 172.21.15.154:0/2365793057 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e8001410 con 0x56465b142da0 2023-09-23T03:28:14.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.462+0000 7f3708ba1740 1 -- 172.21.15.154:0/2365793057 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56465b07b690 con 0x56465b142da0 2023-09-23T03:28:14.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.462+0000 7f36de7fc640 1 -- 172.21.15.154:0/2365793057 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f36e801b050 con 0x56465b142da0 2023-09-23T03:28:14.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.462+0000 7f36de7fc640 1 -- 172.21.15.154:0/2365793057 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36e80079d0 con 0x56465b142da0 2023-09-23T03:28:14.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.462+0000 7f3708ba1740 10 client.9213 fetch_fsmap learned FSMap version 114 2023-09-23T03:28:14.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.462+0000 7f3708ba1740 1 -- 172.21.15.154:0/2365793057 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=2+}) v3 -- 0x56465b080d30 con 0x56465b142da0 2023-09-23T03:28:14.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.462+0000 7f36de7fc640 1 -- 172.21.15.154:0/2365793057 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56465b07b690 con 0x56465b142da0 2023-09-23T03:28:14.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.462+0000 7f36de7fc640 1 -- 172.21.15.154:0/2365793057 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 114) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f36e8028070 con 0x56465b142da0 2023-09-23T03:28:14.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.462+0000 7f3708ba1740 10 client.9213 fetch_fsmap finished waiting for FSMap version 114 2023-09-23T03:28:14.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.462+0000 7f3708ba1740 10 client.9213 resolve_mds: resolved name 'c' to daemon mds.c 2023-09-23T03:28:14.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.462+0000 7f3708ba1740 20 client.9213 populate_metadata read hostname 'smithi154' 2023-09-23T03:28:14.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.462+0000 7f3708ba1740 1 --2- 172.21.15.154:0/2365793057 >> [v2:172.21.15.154:6836/4006549651,v1:172.21.15.154:6837/4006549651] conn(0x56465b1611a0 0x56465b174180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:14.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.462+0000 7f3708ba1740 4 client.9213 mds_command: new command op to 8907 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T03:28:14.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.462+0000 7f3708ba1740 1 -- 172.21.15.154:0/2365793057 --> [v2:172.21.15.154:6836/4006549651,v1:172.21.15.154:6837/4006549651] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56465b07b3b0 con 0x56465b1611a0 2023-09-23T03:28:14.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.463+0000 7f36feffd640 1 --2- 172.21.15.154:0/2365793057 >> [v2:172.21.15.154:6836/4006549651,v1:172.21.15.154:6837/4006549651] conn(0x56465b1611a0 0x56465b174180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:14.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.463+0000 7f36feffd640 1 --2- 172.21.15.154:0/2365793057 >> [v2:172.21.15.154:6836/4006549651,v1:172.21.15.154:6837/4006549651] conn(0x56465b1611a0 0x56465b174180 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=266cfd1661041512 server_cookie=197744fb64c6fef9 in_seq=0 out_seq=0 2023-09-23T03:28:14.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.463+0000 7f36de7fc640 10 client.9213 ms_handle_connect on v2:172.21.15.154:6836/4006549651 2023-09-23T03:28:14.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.464+0000 7f36de7fc640 1 -- 172.21.15.154:0/2365793057 <== mds.0 v2:172.21.15.154:6836/4006549651 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56465b07b3b0 con 0x56465b1611a0 2023-09-23T03:28:14.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.464+0000 7f36de7fc640 10 client.9213 handle_command_reply: tid=0 2023-09-23T03:28:14.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.464+0000 7f3708ba1740 1 client.9213 shutdown 2023-09-23T03:28:14.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.464+0000 7f3708ba1740 1 -- 172.21.15.154:0/2365793057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56465b142da0 msgr2=0x56465b15e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:14.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.464+0000 7f3708ba1740 1 --2- 172.21.15.154:0/2365793057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56465b142da0 0x56465b15e950 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f36e8006400 tx=0x7f36e8000ed0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.464+0000 7f3708ba1740 1 -- 172.21.15.154:0/2365793057 shutdown_connections 2023-09-23T03:28:14.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.464+0000 7f3708ba1740 1 --2- 172.21.15.154:0/2365793057 >> [v2:172.21.15.154:6836/4006549651,v1:172.21.15.154:6837/4006549651] conn(0x56465b1611a0 0x56465b174180 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.464+0000 7f3708ba1740 1 --2- 172.21.15.154:0/2365793057 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56465b15ee90 0x56465b15aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.464+0000 7f3708ba1740 1 --2- 172.21.15.154:0/2365793057 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56465b143180 0x56465b15a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.464+0000 7f3708ba1740 1 --2- 172.21.15.154:0/2365793057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56465b142da0 0x56465b15e950 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.464+0000 7f36de7fc640 0 client.9213 ms_handle_reset on v2:172.21.15.154:6836/4006549651 2023-09-23T03:28:14.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.464+0000 7f3708ba1740 1 -- 172.21.15.154:0/2365793057 >> 172.21.15.154:0/2365793057 conn(0x56465b1161e0 msgr2=0x56465b145b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:14.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.464+0000 7f3708ba1740 1 -- 172.21.15.154:0/2365793057 shutdown_connections 2023-09-23T03:28:14.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.464+0000 7f3708ba1740 1 -- 172.21.15.154:0/2365793057 wait complete. 2023-09-23T03:28:14.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.464+0000 7f3708ba1740 20 client.9213 trim_cache size 0 max 16384 2023-09-23T03:28:14.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.481+0000 7f3708ba1740 1 Processor -- start 2023-09-23T03:28:14.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.481+0000 7f3708ba1740 1 -- start start 2023-09-23T03:28:14.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.482+0000 7f3708ba1740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56465b142da0 0x56465b11bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:14.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.482+0000 7f3708ba1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56465b143180 0x56465b172bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:14.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.482+0000 7f3708ba1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56465b15ee90 0x56465b173100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:14.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.482+0000 7f3708ba1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x56465b115670 con 0x56465b143180 2023-09-23T03:28:14.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.482+0000 7f3708ba1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56465b116760 con 0x56465b15ee90 2023-09-23T03:28:14.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.482+0000 7f3708ba1740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56465b11a210 con 0x56465b142da0 2023-09-23T03:28:14.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.482+0000 7f36feffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56465b143180 0x56465b172bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:14.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.482+0000 7f36feffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56465b143180 0x56465b172bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59574/0 (socket says 172.21.15.154:59574) 2023-09-23T03:28:14.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.482+0000 7f36feffd640 1 -- 172.21.15.154:0/1127867845 learned_addr learned my addr 172.21.15.154:0/1127867845 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:14.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.483+0000 7f36feffd640 1 -- 172.21.15.154:0/1127867845 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56465b142da0 msgr2=0x56465b11bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:14.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.483+0000 7f36feffd640 1 --2- 172.21.15.154:0/1127867845 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56465b142da0 0x56465b11bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.483+0000 7f36feffd640 1 -- 172.21.15.154:0/1127867845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56465b15ee90 msgr2=0x56465b173100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:14.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.483+0000 7f36feffd640 1 --2- 172.21.15.154:0/1127867845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56465b15ee90 0x56465b173100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.483+0000 7f36feffd640 1 -- 172.21.15.154:0/1127867845 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56465b01fb50 con 0x56465b143180 2023-09-23T03:28:14.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.483+0000 7f36feffd640 1 --2- 172.21.15.154:0/1127867845 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56465b143180 0x56465b172bc0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f36f000c990 tx=0x7f36f00885e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9f361bfcb7505d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:14.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.483+0000 7f36f5ffb640 1 -- 172.21.15.154:0/1127867845 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f0088820 con 0x56465b143180 2023-09-23T03:28:14.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.483+0000 7f36f5ffb640 1 -- 172.21.15.154:0/1127867845 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f0057330 con 0x56465b143180 2023-09-23T03:28:14.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.483+0000 7f36f5ffb640 1 -- 172.21.15.154:0/1127867845 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f00556a0 con 0x56465b143180 2023-09-23T03:28:14.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.484+0000 7f3708ba1740 1 -- 172.21.15.154:0/1127867845 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56465b143180 msgr2=0x56465b172bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:14.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.484+0000 7f3708ba1740 1 --2- 172.21.15.154:0/1127867845 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56465b143180 0x56465b172bc0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f36f000c990 tx=0x7f36f00885e0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.484+0000 7f3708ba1740 1 -- 172.21.15.154:0/1127867845 shutdown_connections 2023-09-23T03:28:14.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.484+0000 7f3708ba1740 1 --2- 172.21.15.154:0/1127867845 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56465b15ee90 0x56465b173100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.484+0000 7f3708ba1740 1 --2- 172.21.15.154:0/1127867845 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56465b143180 0x56465b172bc0 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.484+0000 7f3708ba1740 1 --2- 172.21.15.154:0/1127867845 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56465b142da0 0x56465b11bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.484+0000 7f3708ba1740 1 -- 172.21.15.154:0/1127867845 >> 172.21.15.154:0/1127867845 conn(0x56465b1161e0 msgr2=0x56465b145a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:14.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.484+0000 7f3708ba1740 1 -- 172.21.15.154:0/1127867845 shutdown_connections 2023-09-23T03:28:14.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.484+0000 7f3708ba1740 1 -- 172.21.15.154:0/1127867845 wait complete. 2023-09-23T03:28:14.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.485+0000 7f3708ba1740 1 -- start start 2023-09-23T03:28:14.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.485+0000 7f3708ba1740 1 Processor -- start 2023-09-23T03:28:14.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.486+0000 7f3708ba1740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56465b142da0 0x56465b1c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:14.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.486+0000 7f3708ba1740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56465b143180 0x56465b1469f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:14.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.486+0000 7f3708ba1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56465b15ee90 0x56465b146f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:14.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.486+0000 7f3708ba1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x56465b172c30 con 0x56465b15ee90 2023-09-23T03:28:14.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.486+0000 7f3708ba1740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56465b147890 con 0x56465b142da0 2023-09-23T03:28:14.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.486+0000 7f3708ba1740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56465b170100 con 0x56465b143180 2023-09-23T03:28:14.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.486+0000 7f3708ba1740 10 client.0 _ll_register_callbacks cb 0x7f36e8006c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:28:14.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.486+0000 7f36fffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56465b15ee90 0x56465b146f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:14.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.486+0000 7f36fffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56465b15ee90 0x56465b146f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59584/0 (socket says 172.21.15.154:59584) 2023-09-23T03:28:14.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.486+0000 7f36fffff640 1 -- 172.21.15.154:0/2191920659 learned_addr learned my addr 172.21.15.154:0/2191920659 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:14.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.486+0000 7f36ff7fe640 1 --2- 172.21.15.154:0/2191920659 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56465b142da0 0x56465b1c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:14.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.486+0000 7f36feffd640 1 --2- 172.21.15.154:0/2191920659 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56465b143180 0x56465b1469f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:14.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.487+0000 7f36fffff640 1 -- 172.21.15.154:0/2191920659 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56465b143180 msgr2=0x56465b1469f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:14.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.487+0000 7f36fffff640 1 --2- 172.21.15.154:0/2191920659 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56465b143180 0x56465b1469f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.487+0000 7f36fffff640 1 -- 172.21.15.154:0/2191920659 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56465b142da0 msgr2=0x56465b1c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:14.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.487+0000 7f36fffff640 1 --2- 172.21.15.154:0/2191920659 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56465b142da0 0x56465b1c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.487+0000 7f36fffff640 1 -- 172.21.15.154:0/2191920659 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37001f28c0 con 0x56465b15ee90 2023-09-23T03:28:14.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.487+0000 7f36fffff640 1 --2- 172.21.15.154:0/2191920659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56465b15ee90 0x56465b146f30 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f36f802b880 tx=0x7f36f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2128c3eef20c53c0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:14.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.487+0000 7f36de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:28:14.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.487+0000 7f36de7fc640 1 -- 172.21.15.154:0/2191920659 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f8046850 con 0x56465b15ee90 2023-09-23T03:28:14.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.487+0000 7f3708ba1740 1 -- 172.21.15.154:0/2191920659 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f36e8001570 con 0x56465b15ee90 2023-09-23T03:28:14.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.487+0000 7f36de7fc640 1 -- 172.21.15.154:0/2191920659 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f36f8040040 con 0x56465b15ee90 2023-09-23T03:28:14.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.488+0000 7f36de7fc640 1 -- 172.21.15.154:0/2191920659 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=114) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36f8006e10 con 0x56465b15ee90 2023-09-23T03:28:14.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.488+0000 7f3708ba1740 10 client.9219 fetch_fsmap learned FSMap version 114 2023-09-23T03:28:14.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.488+0000 7f3708ba1740 1 -- 172.21.15.154:0/2191920659 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=114,monmap=2+}) v3 -- 0x56465b045140 con 0x56465b15ee90 2023-09-23T03:28:14.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.488+0000 7f36de7fc640 1 -- 172.21.15.154:0/2191920659 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e8001570 con 0x56465b15ee90 2023-09-23T03:28:14.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.488+0000 7f36de7fc640 1 -- 172.21.15.154:0/2191920659 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 114) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f36f80469b0 con 0x56465b15ee90 2023-09-23T03:28:14.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.488+0000 7f3708ba1740 10 client.9219 fetch_fsmap finished waiting for FSMap version 114 2023-09-23T03:28:14.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.488+0000 7f3708ba1740 10 client.9219 resolve_mds: resolved name 'c' to daemon mds.c 2023-09-23T03:28:14.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.488+0000 7f3708ba1740 20 client.9219 populate_metadata read hostname 'smithi154' 2023-09-23T03:28:14.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.488+0000 7f3708ba1740 1 --2- 172.21.15.154:0/2191920659 >> [v2:172.21.15.154:6836/4006549651,v1:172.21.15.154:6837/4006549651] conn(0x56465b16f550 0x56465b182080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:14.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.488+0000 7f3708ba1740 4 client.9219 mds_command: new command op to 8907 tid=0[{"prefix": "status"}] 2023-09-23T03:28:14.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.488+0000 7f3708ba1740 1 -- 172.21.15.154:0/2191920659 --> [v2:172.21.15.154:6836/4006549651,v1:172.21.15.154:6837/4006549651] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56465ad846c0 con 0x56465b16f550 2023-09-23T03:28:14.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.488+0000 7f36ff7fe640 1 --2- 172.21.15.154:0/2191920659 >> [v2:172.21.15.154:6836/4006549651,v1:172.21.15.154:6837/4006549651] conn(0x56465b16f550 0x56465b182080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:14.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.489+0000 7f36ff7fe640 1 --2- 172.21.15.154:0/2191920659 >> [v2:172.21.15.154:6836/4006549651,v1:172.21.15.154:6837/4006549651] conn(0x56465b16f550 0x56465b182080 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=117ff10c07552512 server_cookie=dbd4c83a6dfb4264 in_seq=0 out_seq=0 2023-09-23T03:28:14.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.489+0000 7f36de7fc640 10 client.9219 ms_handle_connect on v2:172.21.15.154:6836/4006549651 2023-09-23T03:28:14.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.489+0000 7f36de7fc640 1 -- 172.21.15.154:0/2191920659 <== mds.0 v2:172.21.15.154:6836/4006549651 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x56465ad846c0 con 0x56465b16f550 2023-09-23T03:28:14.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.489+0000 7f36de7fc640 10 client.9219 handle_command_reply: tid=0 2023-09-23T03:28:14.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.489+0000 7f3708ba1740 1 client.9219 shutdown 2023-09-23T03:28:14.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.489+0000 7f3708ba1740 1 -- 172.21.15.154:0/2191920659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56465b15ee90 msgr2=0x56465b146f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:14.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.489+0000 7f3708ba1740 1 --2- 172.21.15.154:0/2191920659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56465b15ee90 0x56465b146f30 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f36f802b880 tx=0x7f36f80046c0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.489+0000 7f3708ba1740 1 -- 172.21.15.154:0/2191920659 shutdown_connections 2023-09-23T03:28:14.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.489+0000 7f3708ba1740 1 --2- 172.21.15.154:0/2191920659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56465b15ee90 0x56465b146f30 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.489+0000 7f3708ba1740 1 --2- 172.21.15.154:0/2191920659 >> [v2:172.21.15.154:6836/4006549651,v1:172.21.15.154:6837/4006549651] conn(0x56465b16f550 0x56465b182080 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.489+0000 7f3708ba1740 1 --2- 172.21.15.154:0/2191920659 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56465b143180 0x56465b1469f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.489+0000 7f3708ba1740 1 --2- 172.21.15.154:0/2191920659 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56465b142da0 0x56465b1c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.489+0000 7f36de7fc640 0 client.9219 ms_handle_reset on v2:172.21.15.154:6836/4006549651 2023-09-23T03:28:14.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.489+0000 7f3708ba1740 1 -- 172.21.15.154:0/2191920659 >> 172.21.15.154:0/2191920659 conn(0x56465b1161e0 msgr2=0x56465b114dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:14.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.490+0000 7f3708ba1740 1 -- 172.21.15.154:0/2191920659 shutdown_connections 2023-09-23T03:28:14.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.490+0000 7f3708ba1740 1 -- 172.21.15.154:0/2191920659 wait complete. 2023-09-23T03:28:14.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.490+0000 7f3708ba1740 20 client.9219 trim_cache size 0 max 16384 2023-09-23T03:28:14.495 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:28:14.495 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_fsid": "8c8039d3-6c84-43db-8808-85e0dbdd47c8", 2023-09-23T03:28:14.495 INFO:teuthology.orchestra.run.smithi154.stdout: "whoami": 0, 2023-09-23T03:28:14.495 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 8907, 2023-09-23T03:28:14.495 INFO:teuthology.orchestra.run.smithi154.stdout: "want_state": "up:active", 2023-09-23T03:28:14.495 INFO:teuthology.orchestra.run.smithi154.stdout: "state": "up:active", 2023-09-23T03:28:14.495 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs", 2023-09-23T03:28:14.495 INFO:teuthology.orchestra.run.smithi154.stdout: "rank_uptime": 4.9690802569999999, 2023-09-23T03:28:14.495 INFO:teuthology.orchestra.run.smithi154.stdout: "mdsmap_epoch": 114, 2023-09-23T03:28:14.495 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_epoch": 127, 2023-09-23T03:28:14.496 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_epoch_barrier": 127, 2023-09-23T03:28:14.496 INFO:teuthology.orchestra.run.smithi154.stdout: "uptime": 34.893890837000001 2023-09-23T03:28:14.496 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:28:14.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.490+0000 7f3704dae640 1 -- 172.21.15.154:0/730577678 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f36e006af60 msgr2=0x7f36e006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:14.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.490+0000 7f3704dae640 1 --2- 172.21.15.154:0/730577678 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f36e006af60 0x7f36e006d420 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f36e80011e0 tx=0x7f36e8007d30 comp rx=0 tx=0).stop 2023-09-23T03:28:14.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.490+0000 7f3704dae640 1 -- 172.21.15.154:0/730577678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37000d10e0 msgr2=0x7f37001f1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:14.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.490+0000 7f3704dae640 1 --2- 172.21.15.154:0/730577678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37000d10e0 0x7f37001f1e80 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f36f000efc0 tx=0x7f36f000c490 comp rx=0 tx=0).stop 2023-09-23T03:28:14.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.490+0000 7f3704dae640 1 -- 172.21.15.154:0/730577678 shutdown_connections 2023-09-23T03:28:14.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.490+0000 7f3704dae640 1 --2- 172.21.15.154:0/730577678 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f36e006af60 0x7f36e006d420 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.490+0000 7f3704dae640 1 --2- 172.21.15.154:0/730577678 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f37001600b0 0x7f37001546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.490+0000 7f3704dae640 1 --2- 172.21.15.154:0/730577678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37000d10e0 0x7f37001f1e80 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.490+0000 7f3704dae640 1 --2- 172.21.15.154:0/730577678 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f37000ce230 0x7f3700152b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.490+0000 7f3704dae640 1 -- 172.21.15.154:0/730577678 >> 172.21.15.154:0/730577678 conn(0x7f370009bb30 msgr2=0x7f370015f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:14.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.491+0000 7f3704dae640 1 -- 172.21.15.154:0/730577678 shutdown_connections 2023-09-23T03:28:14.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.491+0000 7f3704dae640 1 -- 172.21.15.154:0/730577678 wait complete. 2023-09-23T03:28:14.507 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-23T03:28:14.640 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:28:14.649 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:28:14.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.705+0000 7f237ff39640 1 Processor -- start 2023-09-23T03:28:14.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.705+0000 7f237ff39640 1 -- start start 2023-09-23T03:28:14.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.706+0000 7f237ff39640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f23780cf750 0x7f23780cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:14.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.706+0000 7f237ff39640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f23780cfb30 0x7f23781615d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:14.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.706+0000 7f237ff39640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23780ce480 con 0x7f23780b5ee0 2023-09-23T03:28:14.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.706+0000 7f237ff39640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23780ce1a0 con 0x7f23780cf750 2023-09-23T03:28:14.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.706+0000 7f237ff39640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23780d0df0 con 0x7f23780cfb30 2023-09-23T03:28:14.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.706+0000 7f237dcd5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f23780cf750 0x7f23780cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:14.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.706+0000 7f237dcd5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f23780cf750 0x7f23780cdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59596/0 (socket says 172.21.15.154:59596) 2023-09-23T03:28:14.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.706+0000 7f237dcd5640 1 -- 172.21.15.154:0/471026962 learned_addr learned my addr 172.21.15.154:0/471026962 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:14.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.707+0000 7f237d4d4640 1 --2- 172.21.15.154:0/471026962 >> v2:172.21.15.190:3300/0 conn(0x7f23780cfb30 0x7f23781615d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:14.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.707+0000 7f237dcd5640 1 -- 172.21.15.154:0/471026962 >> v2:172.21.15.190:3300/0 conn(0x7f23780cfb30 msgr2=0x7f23781615d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:14.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.707+0000 7f237dcd5640 1 --2- 172.21.15.154:0/471026962 >> v2:172.21.15.190:3300/0 conn(0x7f23780cfb30 0x7f23781615d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.707+0000 7f237dcd5640 1 -- 172.21.15.154:0/471026962 >> v1:172.21.15.190:6789/0 conn(0x7f23780b5ee0 legacy=0x7f237815c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:14.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.707+0000 7f237dcd5640 1 -- 172.21.15.154:0/471026962 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23780d10d0 con 0x7f23780cf750 2023-09-23T03:28:14.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.708+0000 7f237dcd5640 1 --2- 172.21.15.154:0/471026962 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23780cf750 0x7f23780cdaa0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f23780cffc0 tx=0x7f237402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=902a5c92d9a05aea server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:14.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.708+0000 7f237ccd3640 1 -- 172.21.15.154:0/471026962 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23740317e0 con 0x7f23780cf750 2023-09-23T03:28:14.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.708+0000 7f237ccd3640 1 -- 172.21.15.154:0/471026962 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2374031940 con 0x7f23780cf750 2023-09-23T03:28:14.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.708+0000 7f237ccd3640 1 -- 172.21.15.154:0/471026962 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2374031cd0 con 0x7f23780cf750 2023-09-23T03:28:14.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.709+0000 7f237ff39640 1 -- 172.21.15.154:0/471026962 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23780cf750 msgr2=0x7f23780cdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:14.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.709+0000 7f237ff39640 1 --2- 172.21.15.154:0/471026962 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23780cf750 0x7f23780cdaa0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f23780cffc0 tx=0x7f237402b880 comp rx=0 tx=0).stop 2023-09-23T03:28:14.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.709+0000 7f237ff39640 1 -- 172.21.15.154:0/471026962 shutdown_connections 2023-09-23T03:28:14.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.709+0000 7f237ff39640 1 --2- 172.21.15.154:0/471026962 >> v2:172.21.15.190:3300/0 conn(0x7f23780cfb30 0x7f23781615d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.709+0000 7f237ff39640 1 --2- 172.21.15.154:0/471026962 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23780cf750 0x7f23780cdaa0 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.709+0000 7f237ff39640 1 -- 172.21.15.154:0/471026962 >> 172.21.15.154:0/471026962 conn(0x7f2378071a30 msgr2=0x7f237806af80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:14.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.709+0000 7f237ff39640 1 -- 172.21.15.154:0/471026962 shutdown_connections 2023-09-23T03:28:14.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.709+0000 7f237ff39640 1 -- 172.21.15.154:0/471026962 wait complete. 2023-09-23T03:28:14.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.712+0000 7f237ff39640 1 Processor -- start 2023-09-23T03:28:14.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.712+0000 7f237ff39640 1 -- start start 2023-09-23T03:28:14.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.714+0000 7f237ff39640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23780b5ee0 0x7f2378158950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:14.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.714+0000 7f237ff39640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23780cf750 0x7f2378144d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:14.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.714+0000 7f237ff39640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23780cfb30 0x7f2378151bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:14.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.714+0000 7f237ff39640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237815b080 con 0x7f23780b5ee0 2023-09-23T03:28:14.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.714+0000 7f237ff39640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2378145240 con 0x7f23780cfb30 2023-09-23T03:28:14.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.714+0000 7f237ff39640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f237815b8e0 con 0x7f23780cf750 2023-09-23T03:28:14.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.714+0000 7f237dcd5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23780b5ee0 0x7f2378158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:14.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.714+0000 7f237dcd5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23780b5ee0 0x7f2378158950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59610/0 (socket says 172.21.15.154:59610) 2023-09-23T03:28:14.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.714+0000 7f237dcd5640 1 -- 172.21.15.154:0/3449999190 learned_addr learned my addr 172.21.15.154:0/3449999190 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:14.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.714+0000 7f237d4d4640 1 --2- 172.21.15.154:0/3449999190 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23780cf750 0x7f2378144d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:14.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.714+0000 7f237e4d6640 1 --2- 172.21.15.154:0/3449999190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23780cfb30 0x7f2378151bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:14.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.714+0000 7f237dcd5640 1 -- 172.21.15.154:0/3449999190 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23780cf750 msgr2=0x7f2378144d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:14.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.714+0000 7f237dcd5640 1 --2- 172.21.15.154:0/3449999190 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23780cf750 0x7f2378144d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.714+0000 7f237dcd5640 1 -- 172.21.15.154:0/3449999190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23780cfb30 msgr2=0x7f2378151bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:14.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.714+0000 7f237dcd5640 1 --2- 172.21.15.154:0/3449999190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23780cfb30 0x7f2378151bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.714+0000 7f237dcd5640 1 -- 172.21.15.154:0/3449999190 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23740094d0 con 0x7f23780b5ee0 2023-09-23T03:28:14.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.715+0000 7f237dcd5640 1 --2- 172.21.15.154:0/3449999190 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23780b5ee0 0x7f2378158950 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f237402b8c0 tx=0x7f23740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:14.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.715+0000 7f236e7fc640 1 -- 172.21.15.154:0/3449999190 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2374032880 con 0x7f23780b5ee0 2023-09-23T03:28:14.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.715+0000 7f236e7fc640 1 -- 172.21.15.154:0/3449999190 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23740329e0 con 0x7f23780b5ee0 2023-09-23T03:28:14.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.715+0000 7f236e7fc640 1 -- 172.21.15.154:0/3449999190 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2374032d90 con 0x7f23780b5ee0 2023-09-23T03:28:14.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.715+0000 7f237ff39640 1 -- 172.21.15.154:0/3449999190 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f237815b320 con 0x7f23780b5ee0 2023-09-23T03:28:14.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.716+0000 7f237ff39640 1 -- 172.21.15.154:0/3449999190 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2378152110 con 0x7f23780b5ee0 2023-09-23T03:28:14.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.717+0000 7f237ff39640 1 -- 172.21.15.154:0/3449999190 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2378073eb0 con 0x7f23780b5ee0 2023-09-23T03:28:14.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.718+0000 7f236e7fc640 1 -- 172.21.15.154:0/3449999190 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f237404e450 con 0x7f23780b5ee0 2023-09-23T03:28:14.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.719+0000 7f236e7fc640 1 --2- 172.21.15.154:0/3449999190 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f235c06af30 0x7f235c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:14.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.719+0000 7f236e7fc640 1 -- 172.21.15.154:0/3449999190 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f23740ab8f0 con 0x7f23780b5ee0 2023-09-23T03:28:14.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.719+0000 7f237d4d4640 1 --2- 172.21.15.154:0/3449999190 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f235c06af30 0x7f235c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:14.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.720+0000 7f237d4d4640 1 --2- 172.21.15.154:0/3449999190 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f235c06af30 0x7f235c06d3f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f23780d3240 tx=0x7f2368008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:14.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.723+0000 7f236e7fc640 1 -- 172.21.15.154:0/3449999190 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f2374076830 con 0x7f23780b5ee0 2023-09-23T03:28:14.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.951+0000 7f237ff39640 1 -- 172.21.15.154:0/3449999190 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f23781568b0 con 0x7f23780b5ee0 2023-09-23T03:28:14.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.952+0000 7f236e7fc640 1 -- 172.21.15.154:0/3449999190 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v114) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2374079e50 con 0x7f23780b5ee0 2023-09-23T03:28:14.952 INFO:teuthology.orchestra.run.smithi154.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-09-23T03:28:14.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.955+0000 7f237ff39640 1 -- 172.21.15.154:0/3449999190 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f235c06af30 msgr2=0x7f235c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:14.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.955+0000 7f237ff39640 1 --2- 172.21.15.154:0/3449999190 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f235c06af30 0x7f235c06d3f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f23780d3240 tx=0x7f2368008040 comp rx=0 tx=0).stop 2023-09-23T03:28:14.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.955+0000 7f237ff39640 1 -- 172.21.15.154:0/3449999190 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23780b5ee0 msgr2=0x7f2378158950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:14.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.955+0000 7f237ff39640 1 --2- 172.21.15.154:0/3449999190 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23780b5ee0 0x7f2378158950 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f237402b8c0 tx=0x7f23740046c0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.956+0000 7f237ff39640 1 -- 172.21.15.154:0/3449999190 shutdown_connections 2023-09-23T03:28:14.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.956+0000 7f237ff39640 1 --2- 172.21.15.154:0/3449999190 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f235c06af30 0x7f235c06d3f0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.956+0000 7f237ff39640 1 --2- 172.21.15.154:0/3449999190 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f23780cfb30 0x7f2378151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.956+0000 7f237ff39640 1 --2- 172.21.15.154:0/3449999190 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f23780cf750 0x7f2378144d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.956+0000 7f237ff39640 1 --2- 172.21.15.154:0/3449999190 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23780b5ee0 0x7f2378158950 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:14.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.956+0000 7f237ff39640 1 -- 172.21.15.154:0/3449999190 >> 172.21.15.154:0/3449999190 conn(0x7f2378071a30 msgr2=0x7f23780c9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:14.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.956+0000 7f237ff39640 1 -- 172.21.15.154:0/3449999190 shutdown_connections 2023-09-23T03:28:14.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:14.956+0000 7f237ff39640 1 -- 172.21.15.154:0/3449999190 wait complete. 2023-09-23T03:28:14.974 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-23T03:28:14.974 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-23T03:28:14.974 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi154.front.sepia.ceph.com 2023-09-23T03:28:14.974 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-09-23T03:28:14.974 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-23T03:28:14.974 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-23T03:28:14.974 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-23T03:28:14.974 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-23T03:28:14.974 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-23T03:28:14.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:14.975 DEBUG:teuthology.orchestra.run.smithi154:> ip addr 2023-09-23T03:28:15.029 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-23T03:28:15.029 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-23T03:28:15.029 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2023-09-23T03:28:15.029 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:28:15.029 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2023-09-23T03:28:15.029 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:28:15.030 INFO:teuthology.orchestra.run.smithi154.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:28:15.030 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:54 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:28:15.030 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f0 2023-09-23T03:28:15.030 INFO:teuthology.orchestra.run.smithi154.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:28:15.030 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:55 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:28:15.030 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f1 2023-09-23T03:28:15.030 INFO:teuthology.orchestra.run.smithi154.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:28:15.030 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:60 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:28:15.030 INFO:teuthology.orchestra.run.smithi154.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-23T03:28:15.030 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:61 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:28:15.030 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.21.15.154/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-09-23T03:28:15.030 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft 41630sec preferred_lft 41630sec 2023-09-23T03:28:15.031 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::ec4:7aff:fe88:7761/64 scope link 2023-09-23T03:28:15.031 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:28:15.031 INFO:teuthology.orchestra.run.smithi154.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-23T03:28:15.031 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 5a:c6:b2:96:fb:dc brd ff:ff:ff:ff:ff:ff 2023-09-23T03:28:15.031 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-09-23T03:28:15.031 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:28:15.031 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::a801:22ff:febf:2833/64 scope link 2023-09-23T03:28:15.031 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:28:15.031 INFO:teuthology.orchestra.run.smithi154.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-23T03:28:15.031 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 5a:c6:b2:96:fb:dc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:28:15.031 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::58c6:b2ff:fe96:fbdc/64 scope link 2023-09-23T03:28:15.031 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:28:15.032 INFO:teuthology.orchestra.run.smithi154.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-23T03:28:15.032 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether b2:23:b0:e4:c2:ab brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:28:15.032 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::b023:b0ff:fee4:c2ab/64 scope link 2023-09-23T03:28:15.032 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:28:15.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:15.033 DEBUG:teuthology.orchestra.run.smithi154:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-23T03:28:15.124 INFO:teuthology.orchestra.run.smithi154.stdout:1 2023-09-23T03:28:15.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:15.128 DEBUG:teuthology.orchestra.run.smithi154:> route 2023-09-23T03:28:15.182 INFO:teuthology.orchestra.run.smithi154.stdout:Kernel IP routing table 2023-09-23T03:28:15.182 INFO:teuthology.orchestra.run.smithi154.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-23T03:28:15.183 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-23T03:28:15.183 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-09-23T03:28:15.183 INFO:teuthology.orchestra.run.smithi154.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-23T03:28:15.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:15.183 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-09-23T03:28:15.183 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-09-23T03:28:15.183 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-23T03:28:15.183 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-23T03:28:15.183 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-23T03:28:15.184 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-09-23T03:28:15.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:15.375 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2023-09-23T03:28:15.425 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-09-23T03:28:15.426 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-09-23T03:28:15.427 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-23T03:28:15.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:15.427 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set brx.0 up 2023-09-23T03:28:15.523 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-09-23T03:28:15.523 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-23T03:28:15.523 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-09-23T03:28:15.577 INFO:teuthology.orchestra.run.smithi154.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-09-23T03:28:15.578 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-23T03:28:15.578 DEBUG:teuthology.orchestra.run.smithi154:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-09-23T03:28:15.628 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-09-23T03:28:15.719 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2023-09-23T03:28:15.774 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2023-09-23T03:28:15.774 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2023-09-23T03:28:15.774 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2023-09-23T03:28:15.774 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2023-09-23T03:28:15.774 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2023-09-23T03:28:15.774 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2023-09-23T03:28:15.775 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2023-09-23T03:28:15.775 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2023-09-23T03:28:15.775 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2023-09-23T03:28:15.775 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2023-09-23T03:28:15.775 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2023-09-23T03:28:15.775 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T03:28:15.775 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2023-09-23T03:28:15.775 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2023-09-23T03:28:15.775 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2023-09-23T03:28:15.775 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2023-09-23T03:28:15.775 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2023-09-23T03:28:15.775 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2023-09-23T03:28:15.776 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2023-09-23T03:28:15.776 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-09-23T03:28:15.776 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-09-23T03:28:15.776 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2023-09-23T03:28:15.776 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2023-09-23T03:28:15.776 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2023-09-23T03:28:15.776 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-0 2023-09-23T03:28:15.776 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-1 2023-09-23T03:28:15.776 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-2 2023-09-23T03:28:15.776 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-3 2023-09-23T03:28:15.776 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2023-09-23T03:28:15.777 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:28:15.777 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:28:15.777 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:28:15.777 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:28:15.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:15.778 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-09-23T03:28:15.834 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-23T03:28:15.834 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-09-23T03:28:15.842 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-09-23T03:28:15.924 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2023-09-23T03:28:15.967 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2023-09-23T03:28:15.967 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2023-09-23T03:28:15.967 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2023-09-23T03:28:15.967 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2023-09-23T03:28:15.967 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2023-09-23T03:28:15.968 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2023-09-23T03:28:15.968 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2023-09-23T03:28:15.968 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2023-09-23T03:28:15.968 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2023-09-23T03:28:15.968 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2023-09-23T03:28:15.968 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2023-09-23T03:28:15.968 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T03:28:15.968 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2023-09-23T03:28:15.968 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2023-09-23T03:28:15.968 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2023-09-23T03:28:15.968 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2023-09-23T03:28:15.969 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2023-09-23T03:28:15.969 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2023-09-23T03:28:15.969 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2023-09-23T03:28:15.969 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-09-23T03:28:15.969 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-09-23T03:28:15.969 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2023-09-23T03:28:15.969 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2023-09-23T03:28:15.969 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2023-09-23T03:28:15.969 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-0 2023-09-23T03:28:15.970 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-1 2023-09-23T03:28:15.970 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-2 2023-09-23T03:28:15.970 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-3 2023-09-23T03:28:15.970 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2023-09-23T03:28:15.970 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:28:15.970 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:28:15.970 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:28:15.970 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:28:15.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:15.971 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-09-23T03:28:15.977 INFO:tasks.ceph.mds.c.smithi154.stderr:2023-09-23T03:28:15.974+0000 7f391cc56640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-23T03:28:16.032 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:2023-09-23T03:28:16.032+0000 7f7e87635480 -1 init, newargv = 0x5575dbbecc50 newargc=15 2023-09-23T03:28:16.033 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:ceph-fuse[128588]: starting ceph client 2023-09-23T03:28:16.041 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:ceph-fuse[128588]: starting fuse 2023-09-23T03:28:17.119 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-09-23T03:28:17.222 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2023-09-23T03:28:17.281 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2023-09-23T03:28:17.281 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2023-09-23T03:28:17.281 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2023-09-23T03:28:17.281 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2023-09-23T03:28:17.282 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2023-09-23T03:28:17.282 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2023-09-23T03:28:17.282 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2023-09-23T03:28:17.282 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2023-09-23T03:28:17.282 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2023-09-23T03:28:17.282 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2023-09-23T03:28:17.282 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2023-09-23T03:28:17.282 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T03:28:17.282 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2023-09-23T03:28:17.282 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2023-09-23T03:28:17.283 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2023-09-23T03:28:17.283 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2023-09-23T03:28:17.283 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2023-09-23T03:28:17.283 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2023-09-23T03:28:17.283 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2023-09-23T03:28:17.283 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-09-23T03:28:17.283 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-09-23T03:28:17.283 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2023-09-23T03:28:17.283 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2023-09-23T03:28:17.283 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2023-09-23T03:28:17.283 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-0 2023-09-23T03:28:17.283 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-1 2023-09-23T03:28:17.284 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-2 2023-09-23T03:28:17.284 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-3 2023-09-23T03:28:17.284 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2023-09-23T03:28:17.284 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:28:17.284 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:28:17.284 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:28:17.284 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:28:17.284 INFO:teuthology.orchestra.run.smithi154.stdout:/home/ubuntu/cephtest/mnt.0 2023-09-23T03:28:17.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:17.285 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-09-23T03:28:17.336 INFO:teuthology.orchestra.run.smithi154.stdout:39 2023-09-23T03:28:17.336 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-09-23T03:28:17.336 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:28:17.336 DEBUG:teuthology.orchestra.run.smithi154:> import glob 2023-09-23T03:28:17.336 DEBUG:teuthology.orchestra.run.smithi154:> import re 2023-09-23T03:28:17.337 DEBUG:teuthology.orchestra.run.smithi154:> import os 2023-09-23T03:28:17.337 DEBUG:teuthology.orchestra.run.smithi154:> import subprocess 2023-09-23T03:28:17.337 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:28:17.337 DEBUG:teuthology.orchestra.run.smithi154:> def _find_admin_socket(client_name): 2023-09-23T03:28:17.337 DEBUG:teuthology.orchestra.run.smithi154:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-09-23T03:28:17.337 DEBUG:teuthology.orchestra.run.smithi154:> files = glob.glob(asok_path) 2023-09-23T03:28:17.337 DEBUG:teuthology.orchestra.run.smithi154:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-09-23T03:28:17.337 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:28:17.337 DEBUG:teuthology.orchestra.run.smithi154:> # Given a non-glob path, it better be there 2023-09-23T03:28:17.337 DEBUG:teuthology.orchestra.run.smithi154:> if "*" not in asok_path: 2023-09-23T03:28:17.337 DEBUG:teuthology.orchestra.run.smithi154:> assert(len(files) == 1) 2023-09-23T03:28:17.337 DEBUG:teuthology.orchestra.run.smithi154:> return files[0] 2023-09-23T03:28:17.338 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:28:17.338 DEBUG:teuthology.orchestra.run.smithi154:> for f in files: 2023-09-23T03:28:17.338 DEBUG:teuthology.orchestra.run.smithi154:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-23T03:28:17.338 DEBUG:teuthology.orchestra.run.smithi154:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-23T03:28:17.338 DEBUG:teuthology.orchestra.run.smithi154:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-23T03:28:17.338 DEBUG:teuthology.orchestra.run.smithi154:> contents = proc_f.read() 2023-09-23T03:28:17.338 DEBUG:teuthology.orchestra.run.smithi154:> if mountpoint in contents: 2023-09-23T03:28:17.338 DEBUG:teuthology.orchestra.run.smithi154:> return f 2023-09-23T03:28:17.338 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-23T03:28:17.338 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:28:17.338 DEBUG:teuthology.orchestra.run.smithi154:> print(_find_admin_socket("client.0")) 2023-09-23T03:28:17.338 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:28:17.516 INFO:teuthology.orchestra.run.smithi154.stdout:/var/run/ceph/ceph-client.0.128588.asok 2023-09-23T03:28:17.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:28:17.534 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.128588.asok 2023-09-23T03:28:17.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:17.535 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.128588.asok status 2023-09-23T03:28:17.775 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:28:17.775 INFO:teuthology.orchestra.run.smithi154.stdout: "metadata": { 2023-09-23T03:28:17.775 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_sha1": "862663913a74a99e1c52d50e6a7e59dd6705749a", 2023-09-23T03:28:17.775 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_version": "ceph version 18.0.0-6336-g86266391 (862663913a74a99e1c52d50e6a7e59dd6705749a) reef (dev)", 2023-09-23T03:28:17.775 INFO:teuthology.orchestra.run.smithi154.stdout: "entity_id": "0", 2023-09-23T03:28:17.775 INFO:teuthology.orchestra.run.smithi154.stdout: "hostname": "smithi154", 2023-09-23T03:28:17.776 INFO:teuthology.orchestra.run.smithi154.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-09-23T03:28:17.776 INFO:teuthology.orchestra.run.smithi154.stdout: "pid": "128588", 2023-09-23T03:28:17.776 INFO:teuthology.orchestra.run.smithi154.stdout: "root": "/" 2023-09-23T03:28:17.776 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:28:17.776 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_count": 0, 2023-09-23T03:28:17.776 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_pinned_count": 0, 2023-09-23T03:28:17.776 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 9234, 2023-09-23T03:28:17.776 INFO:teuthology.orchestra.run.smithi154.stdout: "inst": { 2023-09-23T03:28:17.776 INFO:teuthology.orchestra.run.smithi154.stdout: "name": { 2023-09-23T03:28:17.776 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "client", 2023-09-23T03:28:17.776 INFO:teuthology.orchestra.run.smithi154.stdout: "num": 9234 2023-09-23T03:28:17.776 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:28:17.777 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2023-09-23T03:28:17.777 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2023-09-23T03:28:17.777 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.1:0", 2023-09-23T03:28:17.777 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2780174287 2023-09-23T03:28:17.777 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:28:17.777 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:28:17.777 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2023-09-23T03:28:17.777 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2023-09-23T03:28:17.777 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.1:0", 2023-09-23T03:28:17.777 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2780174287 2023-09-23T03:28:17.777 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:28:17.777 INFO:teuthology.orchestra.run.smithi154.stdout: "inst_str": "client.9234 192.168.0.1:0/2780174287", 2023-09-23T03:28:17.778 INFO:teuthology.orchestra.run.smithi154.stdout: "addr_str": "192.168.0.1:0/2780174287", 2023-09-23T03:28:17.778 INFO:teuthology.orchestra.run.smithi154.stdout: "inode_count": 1, 2023-09-23T03:28:17.778 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_epoch": 114, 2023-09-23T03:28:17.778 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch": 127, 2023-09-23T03:28:17.778 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch_barrier": 0, 2023-09-23T03:28:17.778 INFO:teuthology.orchestra.run.smithi154.stdout: "blocklisted": false, 2023-09-23T03:28:17.778 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs" 2023-09-23T03:28:17.778 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:28:17.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:17.790 DEBUG:teuthology.orchestra.run.smithi154:> stat --file-system '--printf=%T 2023-09-23T03:28:17.790 DEBUG:teuthology.orchestra.run.smithi154:> ' -- /home/ubuntu/cephtest/mnt.0 2023-09-23T03:28:17.842 INFO:teuthology.orchestra.run.smithi154.stdout:fuseblk 2023-09-23T03:28:17.843 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-09-23T03:28:17.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:17.843 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-09-23T03:28:17.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:17.927 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2023-09-23T03:28:18.017 INFO:teuthology.orchestra.run.smithi154.stdout:0 2023-09-23T03:28:18.017 INFO:teuthology.orchestra.run.smithi154.stdout:1 2023-09-23T03:28:18.017 INFO:teuthology.orchestra.run.smithi154.stdout:10 2023-09-23T03:28:18.017 INFO:teuthology.orchestra.run.smithi154.stdout:11 2023-09-23T03:28:18.017 INFO:teuthology.orchestra.run.smithi154.stdout:12 2023-09-23T03:28:18.017 INFO:teuthology.orchestra.run.smithi154.stdout:13 2023-09-23T03:28:18.017 INFO:teuthology.orchestra.run.smithi154.stdout:14 2023-09-23T03:28:18.017 INFO:teuthology.orchestra.run.smithi154.stdout:15 2023-09-23T03:28:18.018 INFO:teuthology.orchestra.run.smithi154.stdout:16 2023-09-23T03:28:18.018 INFO:teuthology.orchestra.run.smithi154.stdout:17 2023-09-23T03:28:18.018 INFO:teuthology.orchestra.run.smithi154.stdout:18 2023-09-23T03:28:18.018 INFO:teuthology.orchestra.run.smithi154.stdout:19 2023-09-23T03:28:18.018 INFO:teuthology.orchestra.run.smithi154.stdout:2 2023-09-23T03:28:18.018 INFO:teuthology.orchestra.run.smithi154.stdout:20 2023-09-23T03:28:18.018 INFO:teuthology.orchestra.run.smithi154.stdout:22 2023-09-23T03:28:18.018 INFO:teuthology.orchestra.run.smithi154.stdout:23 2023-09-23T03:28:18.018 INFO:teuthology.orchestra.run.smithi154.stdout:24 2023-09-23T03:28:18.018 INFO:teuthology.orchestra.run.smithi154.stdout:25 2023-09-23T03:28:18.018 INFO:teuthology.orchestra.run.smithi154.stdout:26 2023-09-23T03:28:18.019 INFO:teuthology.orchestra.run.smithi154.stdout:27 2023-09-23T03:28:18.019 INFO:teuthology.orchestra.run.smithi154.stdout:28 2023-09-23T03:28:18.019 INFO:teuthology.orchestra.run.smithi154.stdout:29 2023-09-23T03:28:18.019 INFO:teuthology.orchestra.run.smithi154.stdout:3 2023-09-23T03:28:18.019 INFO:teuthology.orchestra.run.smithi154.stdout:30 2023-09-23T03:28:18.019 INFO:teuthology.orchestra.run.smithi154.stdout:31 2023-09-23T03:28:18.019 INFO:teuthology.orchestra.run.smithi154.stdout:32 2023-09-23T03:28:18.019 INFO:teuthology.orchestra.run.smithi154.stdout:33 2023-09-23T03:28:18.019 INFO:teuthology.orchestra.run.smithi154.stdout:34 2023-09-23T03:28:18.019 INFO:teuthology.orchestra.run.smithi154.stdout:35 2023-09-23T03:28:18.019 INFO:teuthology.orchestra.run.smithi154.stdout:36 2023-09-23T03:28:18.019 INFO:teuthology.orchestra.run.smithi154.stdout:37 2023-09-23T03:28:18.019 INFO:teuthology.orchestra.run.smithi154.stdout:38 2023-09-23T03:28:18.020 INFO:teuthology.orchestra.run.smithi154.stdout:39 2023-09-23T03:28:18.020 INFO:teuthology.orchestra.run.smithi154.stdout:4 2023-09-23T03:28:18.020 INFO:teuthology.orchestra.run.smithi154.stdout:40 2023-09-23T03:28:18.020 INFO:teuthology.orchestra.run.smithi154.stdout:41 2023-09-23T03:28:18.020 INFO:teuthology.orchestra.run.smithi154.stdout:42 2023-09-23T03:28:18.020 INFO:teuthology.orchestra.run.smithi154.stdout:43 2023-09-23T03:28:18.020 INFO:teuthology.orchestra.run.smithi154.stdout:44 2023-09-23T03:28:18.020 INFO:teuthology.orchestra.run.smithi154.stdout:45 2023-09-23T03:28:18.020 INFO:teuthology.orchestra.run.smithi154.stdout:46 2023-09-23T03:28:18.020 INFO:teuthology.orchestra.run.smithi154.stdout:47 2023-09-23T03:28:18.020 INFO:teuthology.orchestra.run.smithi154.stdout:48 2023-09-23T03:28:18.020 INFO:teuthology.orchestra.run.smithi154.stdout:49 2023-09-23T03:28:18.021 INFO:teuthology.orchestra.run.smithi154.stdout:5 2023-09-23T03:28:18.021 INFO:teuthology.orchestra.run.smithi154.stdout:50 2023-09-23T03:28:18.021 INFO:teuthology.orchestra.run.smithi154.stdout:51 2023-09-23T03:28:18.021 INFO:teuthology.orchestra.run.smithi154.stdout:52 2023-09-23T03:28:18.021 INFO:teuthology.orchestra.run.smithi154.stdout:53 2023-09-23T03:28:18.021 INFO:teuthology.orchestra.run.smithi154.stdout:54 2023-09-23T03:28:18.021 INFO:teuthology.orchestra.run.smithi154.stdout:55 2023-09-23T03:28:18.021 INFO:teuthology.orchestra.run.smithi154.stdout:56 2023-09-23T03:28:18.021 INFO:teuthology.orchestra.run.smithi154.stdout:57 2023-09-23T03:28:18.021 INFO:teuthology.orchestra.run.smithi154.stdout:58 2023-09-23T03:28:18.021 INFO:teuthology.orchestra.run.smithi154.stdout:59 2023-09-23T03:28:18.021 INFO:teuthology.orchestra.run.smithi154.stdout:6 2023-09-23T03:28:18.021 INFO:teuthology.orchestra.run.smithi154.stdout:60 2023-09-23T03:28:18.022 INFO:teuthology.orchestra.run.smithi154.stdout:61 2023-09-23T03:28:18.022 INFO:teuthology.orchestra.run.smithi154.stdout:62 2023-09-23T03:28:18.022 INFO:teuthology.orchestra.run.smithi154.stdout:63 2023-09-23T03:28:18.022 INFO:teuthology.orchestra.run.smithi154.stdout:64 2023-09-23T03:28:18.022 INFO:teuthology.orchestra.run.smithi154.stdout:65 2023-09-23T03:28:18.022 INFO:teuthology.orchestra.run.smithi154.stdout:66 2023-09-23T03:28:18.022 INFO:teuthology.orchestra.run.smithi154.stdout:67 2023-09-23T03:28:18.022 INFO:teuthology.orchestra.run.smithi154.stdout:68 2023-09-23T03:28:18.022 INFO:teuthology.orchestra.run.smithi154.stdout:69 2023-09-23T03:28:18.022 INFO:teuthology.orchestra.run.smithi154.stdout:7 2023-09-23T03:28:18.022 INFO:teuthology.orchestra.run.smithi154.stdout:70 2023-09-23T03:28:18.022 INFO:teuthology.orchestra.run.smithi154.stdout:71 2023-09-23T03:28:18.023 INFO:teuthology.orchestra.run.smithi154.stdout:72 2023-09-23T03:28:18.023 INFO:teuthology.orchestra.run.smithi154.stdout:73 2023-09-23T03:28:18.023 INFO:teuthology.orchestra.run.smithi154.stdout:74 2023-09-23T03:28:18.023 INFO:teuthology.orchestra.run.smithi154.stdout:75 2023-09-23T03:28:18.023 INFO:teuthology.orchestra.run.smithi154.stdout:76 2023-09-23T03:28:18.023 INFO:teuthology.orchestra.run.smithi154.stdout:77 2023-09-23T03:28:18.023 INFO:teuthology.orchestra.run.smithi154.stdout:78 2023-09-23T03:28:18.023 INFO:teuthology.orchestra.run.smithi154.stdout:79 2023-09-23T03:28:18.023 INFO:teuthology.orchestra.run.smithi154.stdout:8 2023-09-23T03:28:18.023 INFO:teuthology.orchestra.run.smithi154.stdout:80 2023-09-23T03:28:18.023 INFO:teuthology.orchestra.run.smithi154.stdout:81 2023-09-23T03:28:18.023 INFO:teuthology.orchestra.run.smithi154.stdout:82 2023-09-23T03:28:18.024 INFO:teuthology.orchestra.run.smithi154.stdout:83 2023-09-23T03:28:18.024 INFO:teuthology.orchestra.run.smithi154.stdout:84 2023-09-23T03:28:18.024 INFO:teuthology.orchestra.run.smithi154.stdout:85 2023-09-23T03:28:18.024 INFO:teuthology.orchestra.run.smithi154.stdout:86 2023-09-23T03:28:18.024 INFO:teuthology.orchestra.run.smithi154.stdout:87 2023-09-23T03:28:18.024 INFO:teuthology.orchestra.run.smithi154.stdout:88 2023-09-23T03:28:18.024 INFO:teuthology.orchestra.run.smithi154.stdout:89 2023-09-23T03:28:18.024 INFO:teuthology.orchestra.run.smithi154.stdout:9 2023-09-23T03:28:18.024 INFO:teuthology.orchestra.run.smithi154.stdout:90 2023-09-23T03:28:18.024 INFO:teuthology.orchestra.run.smithi154.stdout:91 2023-09-23T03:28:18.024 INFO:teuthology.orchestra.run.smithi154.stdout:92 2023-09-23T03:28:18.024 INFO:teuthology.orchestra.run.smithi154.stdout:93 2023-09-23T03:28:18.024 INFO:teuthology.orchestra.run.smithi154.stdout:94 2023-09-23T03:28:18.025 INFO:teuthology.orchestra.run.smithi154.stdout:95 2023-09-23T03:28:18.025 INFO:teuthology.orchestra.run.smithi154.stdout:96 2023-09-23T03:28:18.025 INFO:teuthology.orchestra.run.smithi154.stdout:97 2023-09-23T03:28:18.025 INFO:teuthology.orchestra.run.smithi154.stdout:98 2023-09-23T03:28:18.025 INFO:teuthology.orchestra.run.smithi154.stdout:99 2023-09-23T03:28:18.026 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:28:18.026 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:28:18.081 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:28:18.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.297+0000 7fe182dae640 1 Processor -- start 2023-09-23T03:28:18.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.297+0000 7fe182dae640 1 -- start start 2023-09-23T03:28:18.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.298+0000 7fe182dae640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe17c15e2d0 0x7fe17c06c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:18.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.299+0000 7fe182dae640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17c15b310 0x7fe17c15b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:18.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.299+0000 7fe182dae640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe17c0b4270 con 0x7fe17c158200 2023-09-23T03:28:18.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.299+0000 7fe182dae640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe17c15f150 con 0x7fe17c15e2d0 2023-09-23T03:28:18.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.299+0000 7fe182dae640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c15e8b0 con 0x7fe17c15b310 2023-09-23T03:28:18.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.299+0000 7fe1825ad640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17c15b310 0x7fe17c15b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:18.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.299+0000 7fe1815ab640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe17c15e2d0 0x7fe17c06c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:18.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.299+0000 7fe1825ad640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17c15b310 0x7fe17c15b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:57504/0 (socket says 172.21.15.154:57504) 2023-09-23T03:28:18.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.299+0000 7fe1825ad640 1 -- 172.21.15.154:0/1545474558 learned_addr learned my addr 172.21.15.154:0/1545474558 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:18.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.300+0000 7fe1825ad640 1 -- 172.21.15.154:0/1545474558 >> v2:172.21.15.190:3300/0 conn(0x7fe17c15e2d0 msgr2=0x7fe17c06c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:18.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.300+0000 7fe1825ad640 1 --2- 172.21.15.154:0/1545474558 >> v2:172.21.15.190:3300/0 conn(0x7fe17c15e2d0 0x7fe17c06c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:18.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.300+0000 7fe1825ad640 1 -- 172.21.15.154:0/1545474558 >> v1:172.21.15.190:6789/0 conn(0x7fe17c158200 legacy=0x7fe17c0c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:18.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.300+0000 7fe1825ad640 1 -- 172.21.15.154:0/1545474558 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe17c15eb90 con 0x7fe17c15b310 2023-09-23T03:28:18.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.301+0000 7fe1825ad640 1 --2- 172.21.15.154:0/1545474558 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17c15b310 0x7fe17c15b760 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe174009e50 tx=0x7fe17402bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1aef149220d81ac server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:18.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.302+0000 7fe180daa640 1 -- 172.21.15.154:0/1545474558 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17402cb00 con 0x7fe17c15b310 2023-09-23T03:28:18.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.302+0000 7fe180daa640 1 -- 172.21.15.154:0/1545474558 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17402cc60 con 0x7fe17c15b310 2023-09-23T03:28:18.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.302+0000 7fe180daa640 1 -- 172.21.15.154:0/1545474558 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1740343f0 con 0x7fe17c15b310 2023-09-23T03:28:18.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.302+0000 7fe182dae640 1 -- 172.21.15.154:0/1545474558 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17c15b310 msgr2=0x7fe17c15b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:18.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.302+0000 7fe182dae640 1 --2- 172.21.15.154:0/1545474558 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17c15b310 0x7fe17c15b760 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe174009e50 tx=0x7fe17402bf60 comp rx=0 tx=0).stop 2023-09-23T03:28:18.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.303+0000 7fe182dae640 1 -- 172.21.15.154:0/1545474558 shutdown_connections 2023-09-23T03:28:18.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.303+0000 7fe182dae640 1 --2- 172.21.15.154:0/1545474558 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17c15b310 0x7fe17c15b760 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:18.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.303+0000 7fe182dae640 1 --2- 172.21.15.154:0/1545474558 >> v2:172.21.15.190:3300/0 conn(0x7fe17c15e2d0 0x7fe17c06c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:18.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.303+0000 7fe182dae640 1 -- 172.21.15.154:0/1545474558 >> 172.21.15.154:0/1545474558 conn(0x7fe17c06a4c0 msgr2=0x7fe17c0afe50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:18.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.303+0000 7fe182dae640 1 -- 172.21.15.154:0/1545474558 shutdown_connections 2023-09-23T03:28:18.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.303+0000 7fe182dae640 1 -- 172.21.15.154:0/1545474558 wait complete. 2023-09-23T03:28:18.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.304+0000 7fe182dae640 1 Processor -- start 2023-09-23T03:28:18.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.305+0000 7fe182dae640 1 -- start start 2023-09-23T03:28:18.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.305+0000 7fe182dae640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe17c158200 0x7fe17c14ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:18.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.305+0000 7fe182dae640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe17c15b310 0x7fe17c0c35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:18.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.305+0000 7fe182dae640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17c15e2d0 0x7fe17c1599f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:18.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.305+0000 7fe182dae640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c0af5a0 con 0x7fe17c15b310 2023-09-23T03:28:18.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.305+0000 7fe182dae640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c0ae680 con 0x7fe17c158200 2023-09-23T03:28:18.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.305+0000 7fe182dae640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c14b370 con 0x7fe17c15e2d0 2023-09-23T03:28:18.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.306+0000 7fe1815ab640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe17c15b310 0x7fe17c0c35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:18.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.306+0000 7fe1815ab640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe17c15b310 0x7fe17c0c35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59624/0 (socket says 172.21.15.154:59624) 2023-09-23T03:28:18.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.306+0000 7fe1815ab640 1 -- 172.21.15.154:0/2212247522 learned_addr learned my addr 172.21.15.154:0/2212247522 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:18.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.306+0000 7fe1825ad640 1 --2- 172.21.15.154:0/2212247522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17c15e2d0 0x7fe17c1599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:18.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.306+0000 7fe181dac640 1 --2- 172.21.15.154:0/2212247522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe17c158200 0x7fe17c14ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:18.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.306+0000 7fe1815ab640 1 -- 172.21.15.154:0/2212247522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17c15e2d0 msgr2=0x7fe17c1599f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:18.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.306+0000 7fe1815ab640 1 --2- 172.21.15.154:0/2212247522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17c15e2d0 0x7fe17c1599f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:18.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.306+0000 7fe1815ab640 1 -- 172.21.15.154:0/2212247522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe17c158200 msgr2=0x7fe17c14ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:18.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.306+0000 7fe1815ab640 1 --2- 172.21.15.154:0/2212247522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe17c158200 0x7fe17c14ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:18.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.306+0000 7fe1815ab640 1 -- 172.21.15.154:0/2212247522 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe174009b30 con 0x7fe17c15b310 2023-09-23T03:28:18.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.306+0000 7fe181dac640 1 --2- 172.21.15.154:0/2212247522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe17c158200 0x7fe17c14ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:18.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.307+0000 7fe1815ab640 1 --2- 172.21.15.154:0/2212247522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe17c15b310 0x7fe17c0c35e0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fe16c00e870 tx=0x7fe16c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:18.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.307+0000 7fe1727fc640 1 -- 172.21.15.154:0/2212247522 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16c0094f0 con 0x7fe17c15b310 2023-09-23T03:28:18.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.307+0000 7fe1727fc640 1 -- 172.21.15.154:0/2212247522 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe16c00c940 con 0x7fe17c15b310 2023-09-23T03:28:18.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.307+0000 7fe182dae640 1 -- 172.21.15.154:0/2212247522 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe17c0ae920 con 0x7fe17c15b310 2023-09-23T03:28:18.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.307+0000 7fe1727fc640 1 -- 172.21.15.154:0/2212247522 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16c00ccf0 con 0x7fe17c15b310 2023-09-23T03:28:18.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.307+0000 7fe182dae640 1 -- 172.21.15.154:0/2212247522 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe17c0aeee0 con 0x7fe17c15b310 2023-09-23T03:28:18.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.308+0000 7fe182dae640 1 -- 172.21.15.154:0/2212247522 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe17c066730 con 0x7fe17c15b310 2023-09-23T03:28:18.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.310+0000 7fe1727fc640 1 -- 172.21.15.154:0/2212247522 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fe16c013470 con 0x7fe17c15b310 2023-09-23T03:28:18.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.311+0000 7fe1727fc640 1 --2- 172.21.15.154:0/2212247522 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe16006af30 0x7fe16006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:18.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.311+0000 7fe1727fc640 1 -- 172.21.15.154:0/2212247522 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fe16c089aa0 con 0x7fe17c15b310 2023-09-23T03:28:18.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.311+0000 7fe181dac640 1 --2- 172.21.15.154:0/2212247522 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe16006af30 0x7fe16006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:18.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.311+0000 7fe181dac640 1 --2- 172.21.15.154:0/2212247522 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe16006af30 0x7fe16006d3f0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fe178000f60 tx=0x7fe178009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:18.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.315+0000 7fe1727fc640 1 -- 172.21.15.154:0/2212247522 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fe16c0548a0 con 0x7fe17c15b310 2023-09-23T03:28:18.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.508+0000 7fe182dae640 1 -- 172.21.15.154:0/2212247522 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe17c04e920 con 0x7fe17c15b310 2023-09-23T03:28:18.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.510+0000 7fe1727fc640 1 -- 172.21.15.154:0/2212247522 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+16951 (secure 0 0 0) 0x7fe16c017090 con 0x7fe17c15b310 2023-09-23T03:28:18.511 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:28:18.512 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":127,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:28:08.468660+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:27:42.207617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-09-23T03:27:43.473867+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.190:6835/3558096060":"2023-09-24T03:28:08.468628+0000","172.21.15.190:6833/3558096060":"2023-09-24T03:28:08.468628+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:28:18.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.513+0000 7fe182dae640 1 -- 172.21.15.154:0/2212247522 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe16006af30 msgr2=0x7fe16006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:18.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.513+0000 7fe182dae640 1 --2- 172.21.15.154:0/2212247522 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe16006af30 0x7fe16006d3f0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fe178000f60 tx=0x7fe178009040 comp rx=0 tx=0).stop 2023-09-23T03:28:18.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.513+0000 7fe182dae640 1 -- 172.21.15.154:0/2212247522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe17c15b310 msgr2=0x7fe17c0c35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:18.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.513+0000 7fe182dae640 1 --2- 172.21.15.154:0/2212247522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe17c15b310 0x7fe17c0c35e0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fe16c00e870 tx=0x7fe16c00ed30 comp rx=0 tx=0).stop 2023-09-23T03:28:18.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.514+0000 7fe182dae640 1 -- 172.21.15.154:0/2212247522 shutdown_connections 2023-09-23T03:28:18.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.514+0000 7fe182dae640 1 --2- 172.21.15.154:0/2212247522 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe16006af30 0x7fe16006d3f0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:18.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.514+0000 7fe182dae640 1 --2- 172.21.15.154:0/2212247522 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe17c15e2d0 0x7fe17c1599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:18.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.514+0000 7fe182dae640 1 --2- 172.21.15.154:0/2212247522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe17c15b310 0x7fe17c0c35e0 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:18.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.514+0000 7fe182dae640 1 --2- 172.21.15.154:0/2212247522 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe17c158200 0x7fe17c14ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:18.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.514+0000 7fe182dae640 1 -- 172.21.15.154:0/2212247522 >> 172.21.15.154:0/2212247522 conn(0x7fe17c06a4c0 msgr2=0x7fe17c0ab860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:18.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.514+0000 7fe182dae640 1 -- 172.21.15.154:0/2212247522 shutdown_connections 2023-09-23T03:28:18.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.514+0000 7fe182dae640 1 -- 172.21.15.154:0/2212247522 wait complete. 2023-09-23T03:28:18.533 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:27:42.207617+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:27:43.473867+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T03:28:18.533 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:28:18.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.752+0000 7f5c89dd7640 1 Processor -- start 2023-09-23T03:28:18.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.753+0000 7f5c89dd7640 1 -- start start 2023-09-23T03:28:18.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.754+0000 7f5c89dd7640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5c8415cb20 0x7f5c840b2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:18.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.754+0000 7f5c89dd7640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f5c840cc200 0x7f5c840b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:18.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.754+0000 7f5c89dd7640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c8406b4f0 con 0x7f5c8406aff0 2023-09-23T03:28:18.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.754+0000 7f5c89dd7640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c8406b7d0 con 0x7f5c8415cb20 2023-09-23T03:28:18.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.754+0000 7f5c89dd7640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c8406bab0 con 0x7f5c840cc200 2023-09-23T03:28:18.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.754+0000 7f5c837fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5c8415cb20 0x7f5c840b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:18.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.755+0000 7f5c837fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5c8415cb20 0x7f5c840b2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59632/0 (socket says 172.21.15.154:59632) 2023-09-23T03:28:18.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.755+0000 7f5c82ffd640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f5c8406aff0 0x7f5c8406fc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:35260/0 (socket says 172.21.15.154:35260) 2023-09-23T03:28:18.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.755+0000 7f5c837fe640 1 -- 172.21.15.154:0/1449237904 learned_addr learned my addr 172.21.15.154:0/1449237904 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:18.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.755+0000 7f5c83fff640 1 --2- 172.21.15.154:0/1449237904 >> v2:172.21.15.190:3300/0 conn(0x7f5c840cc200 0x7f5c840b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:18.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.755+0000 7f5c837fe640 1 -- 172.21.15.154:0/1449237904 >> v2:172.21.15.190:3300/0 conn(0x7f5c840cc200 msgr2=0x7f5c840b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:18.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.755+0000 7f5c837fe640 1 --2- 172.21.15.154:0/1449237904 >> v2:172.21.15.190:3300/0 conn(0x7f5c840cc200 0x7f5c840b26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:18.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.755+0000 7f5c837fe640 1 -- 172.21.15.154:0/1449237904 >> v1:172.21.15.154:6789/0 conn(0x7f5c8406aff0 legacy=0x7f5c8406fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:18.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.755+0000 7f5c837fe640 1 -- 172.21.15.154:0/1449237904 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c8406bd90 con 0x7f5c8415cb20 2023-09-23T03:28:18.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.755+0000 7f5c827fc640 1 -- 172.21.15.154:0/1449237904 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1850117659 0 0) 0x7f5c8406b4f0 con 0x7f5c8406aff0 2023-09-23T03:28:18.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.756+0000 7f5c837fe640 1 --2- 172.21.15.154:0/1449237904 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c8415cb20 0x7f5c840b2190 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f5c840afa70 tx=0x7f5c7402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc422fc909991d85 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:18.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.756+0000 7f5c827fc640 1 -- 172.21.15.154:0/1449237904 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7402cba0 con 0x7f5c8415cb20 2023-09-23T03:28:18.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.756+0000 7f5c827fc640 1 -- 172.21.15.154:0/1449237904 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7402cd00 con 0x7f5c8415cb20 2023-09-23T03:28:18.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.756+0000 7f5c827fc640 1 -- 172.21.15.154:0/1449237904 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c74034610 con 0x7f5c8415cb20 2023-09-23T03:28:18.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.757+0000 7f5c89dd7640 1 -- 172.21.15.154:0/1449237904 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c8415cb20 msgr2=0x7f5c840b2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:18.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.757+0000 7f5c89dd7640 1 --2- 172.21.15.154:0/1449237904 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c8415cb20 0x7f5c840b2190 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f5c840afa70 tx=0x7f5c7402c000 comp rx=0 tx=0).stop 2023-09-23T03:28:18.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.757+0000 7f5c89dd7640 1 -- 172.21.15.154:0/1449237904 shutdown_connections 2023-09-23T03:28:18.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.757+0000 7f5c89dd7640 1 --2- 172.21.15.154:0/1449237904 >> v2:172.21.15.190:3300/0 conn(0x7f5c840cc200 0x7f5c840b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:18.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.757+0000 7f5c89dd7640 1 --2- 172.21.15.154:0/1449237904 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c8415cb20 0x7f5c840b2190 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:18.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.757+0000 7f5c89dd7640 1 -- 172.21.15.154:0/1449237904 >> 172.21.15.154:0/1449237904 conn(0x7f5c840cd190 msgr2=0x7f5c840cd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:18.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.757+0000 7f5c89dd7640 1 -- 172.21.15.154:0/1449237904 shutdown_connections 2023-09-23T03:28:18.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.757+0000 7f5c89dd7640 1 -- 172.21.15.154:0/1449237904 wait complete. 2023-09-23T03:28:18.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.759+0000 7f5c89dd7640 1 Processor -- start 2023-09-23T03:28:18.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.759+0000 7f5c89dd7640 1 -- start start 2023-09-23T03:28:18.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.760+0000 7f5c89dd7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c8406aff0 0x7f5c840c9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:18.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.760+0000 7f5c89dd7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c840cc200 0x7f5c841f1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:18.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.760+0000 7f5c89dd7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c8415cb20 0x7f5c841f21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:18.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.760+0000 7f5c89dd7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c840c8610 con 0x7f5c8415cb20 2023-09-23T03:28:18.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.760+0000 7f5c89dd7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c840c88f0 con 0x7f5c8406aff0 2023-09-23T03:28:18.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.760+0000 7f5c89dd7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c840c8bd0 con 0x7f5c840cc200 2023-09-23T03:28:18.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.760+0000 7f5c83fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c8415cb20 0x7f5c841f21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:18.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.760+0000 7f5c83fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c8415cb20 0x7f5c841f21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59634/0 (socket says 172.21.15.154:59634) 2023-09-23T03:28:18.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.760+0000 7f5c83fff640 1 -- 172.21.15.154:0/166464540 learned_addr learned my addr 172.21.15.154:0/166464540 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:18.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.761+0000 7f5c837fe640 1 --2- 172.21.15.154:0/166464540 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c8406aff0 0x7f5c840c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:18.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.761+0000 7f5c83fff640 1 -- 172.21.15.154:0/166464540 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c840cc200 msgr2=0x7f5c841f1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:18.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.761+0000 7f5c83fff640 1 --2- 172.21.15.154:0/166464540 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c840cc200 0x7f5c841f1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:18.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.761+0000 7f5c83fff640 1 -- 172.21.15.154:0/166464540 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c8406aff0 msgr2=0x7f5c840c9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:18.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.761+0000 7f5c83fff640 1 --2- 172.21.15.154:0/166464540 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c8406aff0 0x7f5c840c9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:18.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.761+0000 7f5c83fff640 1 -- 172.21.15.154:0/166464540 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c74009b30 con 0x7f5c8415cb20 2023-09-23T03:28:18.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.761+0000 7f5c83fff640 1 --2- 172.21.15.154:0/166464540 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c8415cb20 0x7f5c841f21c0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c00c7f0 tx=0x7f5c7c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:18.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.761+0000 7f5c7bfff640 1 -- 172.21.15.154:0/166464540 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c004220 con 0x7f5c8415cb20 2023-09-23T03:28:18.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.761+0000 7f5c7bfff640 1 -- 172.21.15.154:0/166464540 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7c004380 con 0x7f5c8415cb20 2023-09-23T03:28:18.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.761+0000 7f5c7bfff640 1 -- 172.21.15.154:0/166464540 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c007860 con 0x7f5c8415cb20 2023-09-23T03:28:18.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.761+0000 7f5c89dd7640 1 -- 172.21.15.154:0/166464540 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c840c8eb0 con 0x7f5c8415cb20 2023-09-23T03:28:18.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.762+0000 7f5c89dd7640 1 -- 172.21.15.154:0/166464540 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c8415efe0 con 0x7f5c8415cb20 2023-09-23T03:28:18.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.763+0000 7f5c89dd7640 1 -- 172.21.15.154:0/166464540 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c84073eb0 con 0x7f5c8415cb20 2023-09-23T03:28:18.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.764+0000 7f5c7bfff640 1 -- 172.21.15.154:0/166464540 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f5c7c007bd0 con 0x7f5c8415cb20 2023-09-23T03:28:18.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.764+0000 7f5c7bfff640 1 --2- 172.21.15.154:0/166464540 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5c6406af30 0x7f5c6406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:18.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.765+0000 7f5c7bfff640 1 -- 172.21.15.154:0/166464540 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f5c7c089840 con 0x7f5c8415cb20 2023-09-23T03:28:18.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.765+0000 7f5c837fe640 1 --2- 172.21.15.154:0/166464540 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5c6406af30 0x7f5c6406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:18.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.765+0000 7f5c837fe640 1 --2- 172.21.15.154:0/166464540 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5c6406af30 0x7f5c6406d3f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5c74035000 tx=0x7f5c74030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:18.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.770+0000 7f5c7bfff640 1 -- 172.21.15.154:0/166464540 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f5c7c054770 con 0x7f5c8415cb20 2023-09-23T03:28:18.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.951+0000 7f5c89dd7640 1 -- 172.21.15.154:0/166464540 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5c840708a0 con 0x7f5c8415cb20 2023-09-23T03:28:18.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.952+0000 7f5c7bfff640 1 -- 172.21.15.154:0/166464540 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5c7c01e020 con 0x7f5c8415cb20 2023-09-23T03:28:18.953 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:28:18.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.955+0000 7f5c89dd7640 1 -- 172.21.15.154:0/166464540 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5c6406af30 msgr2=0x7f5c6406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:18.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.955+0000 7f5c89dd7640 1 --2- 172.21.15.154:0/166464540 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5c6406af30 0x7f5c6406d3f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5c74035000 tx=0x7f5c74030000 comp rx=0 tx=0).stop 2023-09-23T03:28:18.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.955+0000 7f5c89dd7640 1 -- 172.21.15.154:0/166464540 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c8415cb20 msgr2=0x7f5c841f21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:18.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.955+0000 7f5c89dd7640 1 --2- 172.21.15.154:0/166464540 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c8415cb20 0x7f5c841f21c0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c00c7f0 tx=0x7f5c7c00ccb0 comp rx=0 tx=0).stop 2023-09-23T03:28:18.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.955+0000 7f5c89dd7640 1 -- 172.21.15.154:0/166464540 shutdown_connections 2023-09-23T03:28:18.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.955+0000 7f5c89dd7640 1 --2- 172.21.15.154:0/166464540 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5c6406af30 0x7f5c6406d3f0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:18.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.955+0000 7f5c89dd7640 1 --2- 172.21.15.154:0/166464540 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5c8415cb20 0x7f5c841f21c0 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:18.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.955+0000 7f5c89dd7640 1 --2- 172.21.15.154:0/166464540 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5c840cc200 0x7f5c841f1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:18.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.955+0000 7f5c89dd7640 1 --2- 172.21.15.154:0/166464540 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5c8406aff0 0x7f5c840c9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:18.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.955+0000 7f5c89dd7640 1 -- 172.21.15.154:0/166464540 >> 172.21.15.154:0/166464540 conn(0x7f5c840cd190 msgr2=0x7f5c840cb0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:18.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.955+0000 7f5c89dd7640 1 -- 172.21.15.154:0/166464540 shutdown_connections 2023-09-23T03:28:18.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:18.955+0000 7f5c89dd7640 1 -- 172.21.15.154:0/166464540 wait complete. 2023-09-23T03:28:18.974 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:28:19.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.183+0000 7fed4014c640 1 Processor -- start 2023-09-23T03:28:19.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.183+0000 7fed4014c640 1 -- start start 2023-09-23T03:28:19.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.185+0000 7fed4014c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fed380700e0 0x7fed38070560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:19.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.185+0000 7fed4014c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed3815ab20 con 0x7fed380b2770 2023-09-23T03:28:19.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.185+0000 7fed4014c640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed38159f50 con 0x7fed380ca6b0 2023-09-23T03:28:19.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.185+0000 7fed4014c640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed38070aa0 con 0x7fed380700e0 2023-09-23T03:28:19.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.185+0000 7fed3e6e9640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fed380b2770 0x7fed380b2ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:35276/0 (socket says 172.21.15.154:35276) 2023-09-23T03:28:19.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.185+0000 7fed3e6e9640 1 -- 172.21.15.154:0/2670872399 learned_addr learned my addr 172.21.15.154:0/2670872399 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:28:19.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.185+0000 7fed3d6e7640 1 --2- 172.21.15.154:0/2670872399 >> v2:172.21.15.190:3300/0 conn(0x7fed380700e0 0x7fed38070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:19.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.186+0000 7fed3cee6640 1 -- 172.21.15.154:0/2670872399 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2079636682 0 0) 0x7fed3815ab20 con 0x7fed380b2770 2023-09-23T03:28:19.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.186+0000 7fed3cee6640 1 -- 172.21.15.154:0/2670872399 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed24005730 con 0x7fed380b2770 2023-09-23T03:28:19.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.186+0000 7fed3cee6640 1 -- 172.21.15.154:0/2670872399 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1370281094 0 0) 0x7fed24005730 con 0x7fed380b2770 2023-09-23T03:28:19.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.186+0000 7fed3cee6640 1 -- 172.21.15.154:0/2670872399 >> v2:172.21.15.190:3300/0 conn(0x7fed380700e0 msgr2=0x7fed38070560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:19.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.186+0000 7fed3cee6640 1 --2- 172.21.15.154:0/2670872399 >> v2:172.21.15.190:3300/0 conn(0x7fed380700e0 0x7fed38070560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:19.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.186+0000 7fed3cee6640 1 -- 172.21.15.154:0/2670872399 >> v1:172.21.15.190:6789/0 conn(0x7fed380ca6b0 legacy=0x7fed3806d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:19.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.186+0000 7fed3cee6640 1 -- 172.21.15.154:0/2670872399 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed380caa90 con 0x7fed380b2770 2023-09-23T03:28:19.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.186+0000 7fed3cee6640 1 -- 172.21.15.154:0/2670872399 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fed300019f0 con 0x7fed380b2770 2023-09-23T03:28:19.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.186+0000 7fed3cee6640 1 -- 172.21.15.154:0/2670872399 >> v1:172.21.15.154:6789/0 conn(0x7fed380b2770 legacy=0x7fed380b2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:19.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.186+0000 7fed3cee6640 1 --2- 172.21.15.154:0/2670872399 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed24005b80 0x7fed24005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:19.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.186+0000 7fed3cee6640 1 --2- 172.21.15.154:0/2670872399 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed240069d0 0x7fed24006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:19.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.186+0000 7fed3cee6640 1 --2- 172.21.15.154:0/2670872399 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed24007390 0x7fed2400db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:19.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.186+0000 7fed3cee6640 1 -- 172.21.15.154:0/2670872399 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed38070aa0 con 0x7fed24007390 2023-09-23T03:28:19.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.186+0000 7fed3cee6640 1 -- 172.21.15.154:0/2670872399 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2400e230 con 0x7fed240069d0 2023-09-23T03:28:19.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.186+0000 7fed3cee6640 1 -- 172.21.15.154:0/2670872399 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed2400e3f0 con 0x7fed24005b80 2023-09-23T03:28:19.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.186+0000 7fed3e6e9640 1 --2- 172.21.15.154:0/2670872399 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed24007390 0x7fed2400db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:19.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.187+0000 7fed3d6e7640 1 --2- 172.21.15.154:0/2670872399 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed240069d0 0x7fed24006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:19.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.187+0000 7fed3e6e9640 1 -- 172.21.15.154:0/2670872399 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed24005b80 msgr2=0x7fed24005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:19.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.187+0000 7fed3dee8640 1 --2- 172.21.15.154:0/2670872399 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed24005b80 0x7fed24005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:19.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.187+0000 7fed3e6e9640 1 --2- 172.21.15.154:0/2670872399 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed24005b80 0x7fed24005fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:19.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.187+0000 7fed3e6e9640 1 -- 172.21.15.154:0/2670872399 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed240069d0 msgr2=0x7fed24006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:19.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.187+0000 7fed3e6e9640 1 --2- 172.21.15.154:0/2670872399 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed240069d0 0x7fed24006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:19.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.187+0000 7fed3e6e9640 1 -- 172.21.15.154:0/2670872399 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed24005730 con 0x7fed24007390 2023-09-23T03:28:19.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.187+0000 7fed3d6e7640 1 --2- 172.21.15.154:0/2670872399 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed240069d0 0x7fed24006e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:19.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.188+0000 7fed3e6e9640 1 -- 172.21.15.154:0/2670872399 reap_dead start 2023-09-23T03:28:19.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.188+0000 7fed3e6e9640 1 --2- 172.21.15.154:0/2670872399 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed24007390 0x7fed2400db90 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fed3000a830 tx=0x7fed24007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d56c82ee3a2c15 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:19.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.188+0000 7fed3cee6640 1 -- 172.21.15.154:0/2670872399 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed300072d0 con 0x7fed24007390 2023-09-23T03:28:19.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.188+0000 7fed3cee6640 1 -- 172.21.15.154:0/2670872399 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed3002f070 con 0x7fed24007390 2023-09-23T03:28:19.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.188+0000 7fed3cee6640 1 -- 172.21.15.154:0/2670872399 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed300076b0 con 0x7fed24007390 2023-09-23T03:28:19.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.189+0000 7fed4014c640 1 -- 172.21.15.154:0/2670872399 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed24007390 msgr2=0x7fed2400db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:19.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.189+0000 7fed4014c640 1 --2- 172.21.15.154:0/2670872399 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed24007390 0x7fed2400db90 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fed3000a830 tx=0x7fed24007a00 comp rx=0 tx=0).stop 2023-09-23T03:28:19.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.189+0000 7fed4014c640 1 -- 172.21.15.154:0/2670872399 shutdown_connections 2023-09-23T03:28:19.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.189+0000 7fed4014c640 1 --2- 172.21.15.154:0/2670872399 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed24007390 0x7fed2400db90 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:19.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.189+0000 7fed4014c640 1 -- 172.21.15.154:0/2670872399 >> 172.21.15.154:0/2670872399 conn(0x7fed3806a440 msgr2=0x7fed380b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:19.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.189+0000 7fed4014c640 1 -- 172.21.15.154:0/2670872399 shutdown_connections 2023-09-23T03:28:19.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.189+0000 7fed4014c640 1 -- 172.21.15.154:0/2670872399 wait complete. 2023-09-23T03:28:19.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.191+0000 7fed4014c640 1 Processor -- start 2023-09-23T03:28:19.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.191+0000 7fed4014c640 1 -- start start 2023-09-23T03:28:19.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.192+0000 7fed4014c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed24007390 0x7fed380c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:19.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.192+0000 7fed4014c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed38140b00 0x7fed38140f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:19.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.192+0000 7fed4014c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed38151560 0x7fed38151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:19.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.192+0000 7fed4014c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed381414c0 con 0x7fed38151560 2023-09-23T03:28:19.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.192+0000 7fed4014c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed38152320 con 0x7fed24007390 2023-09-23T03:28:19.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.192+0000 7fed4014c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed38152600 con 0x7fed38140b00 2023-09-23T03:28:19.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.192+0000 7fed3e6e9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed38151560 0x7fed38151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:19.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.192+0000 7fed3e6e9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed38151560 0x7fed38151a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59640/0 (socket says 172.21.15.154:59640) 2023-09-23T03:28:19.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.192+0000 7fed3e6e9640 1 -- 172.21.15.154:0/165368480 learned_addr learned my addr 172.21.15.154:0/165368480 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:19.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.193+0000 7fed3dee8640 1 --2- 172.21.15.154:0/165368480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed24007390 0x7fed380c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:19.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.193+0000 7fed3e6e9640 1 -- 172.21.15.154:0/165368480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed38140b00 msgr2=0x7fed38140f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:19.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.193+0000 7fed3e6e9640 1 --2- 172.21.15.154:0/165368480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed38140b00 0x7fed38140f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:19.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.193+0000 7fed3e6e9640 1 -- 172.21.15.154:0/165368480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed24007390 msgr2=0x7fed380c96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:19.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.193+0000 7fed3e6e9640 1 --2- 172.21.15.154:0/165368480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed24007390 0x7fed380c96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:19.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.193+0000 7fed3e6e9640 1 -- 172.21.15.154:0/165368480 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed3813eb50 con 0x7fed38151560 2023-09-23T03:28:19.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.193+0000 7fed3e6e9640 1 --2- 172.21.15.154:0/165368480 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed38151560 0x7fed38151a10 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fed3000a950 tx=0x7fed30001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:19.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.193+0000 7fed2e7fc640 1 -- 172.21.15.154:0/165368480 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed30044070 con 0x7fed38151560 2023-09-23T03:28:19.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.193+0000 7fed2e7fc640 1 -- 172.21.15.154:0/165368480 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed30001250 con 0x7fed38151560 2023-09-23T03:28:19.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.193+0000 7fed2e7fc640 1 -- 172.21.15.154:0/165368480 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed30007670 con 0x7fed38151560 2023-09-23T03:28:19.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.193+0000 7fed4014c640 1 -- 172.21.15.154:0/165368480 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed3813ee30 con 0x7fed38151560 2023-09-23T03:28:19.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.193+0000 7fed4014c640 1 -- 172.21.15.154:0/165368480 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed38151f50 con 0x7fed38151560 2023-09-23T03:28:19.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.195+0000 7fed4014c640 1 -- 172.21.15.154:0/165368480 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed381520e0 con 0x7fed38151560 2023-09-23T03:28:19.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.195+0000 7fed2e7fc640 1 -- 172.21.15.154:0/165368480 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fed30031030 con 0x7fed38151560 2023-09-23T03:28:19.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.197+0000 7fed2e7fc640 1 --2- 172.21.15.154:0/165368480 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fed1c06af30 0x7fed1c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:19.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.197+0000 7fed2e7fc640 1 -- 172.21.15.154:0/165368480 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6115+0+0 (secure 0 0 0) 0x7fed300aad90 con 0x7fed38151560 2023-09-23T03:28:19.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.197+0000 7fed3dee8640 1 --2- 172.21.15.154:0/165368480 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fed1c06af30 0x7fed1c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:19.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.198+0000 7fed3dee8640 1 --2- 172.21.15.154:0/165368480 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fed1c06af30 0x7fed1c06d3f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fed381f1380 tx=0x7fed34009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:19.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.201+0000 7fed2e7fc640 1 -- 172.21.15.154:0/165368480 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fed381520e0 con 0x7fed38151560 2023-09-23T03:28:19.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.391+0000 7fed4014c640 1 -- 172.21.15.154:0/165368480 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fed3813f350 con 0x7fed38151560 2023-09-23T03:28:19.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.392+0000 7fed2e7fc640 1 -- 172.21.15.154:0/165368480 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v127) v1 ==== 104+0+11 (secure 0 0 0) 0x7fed3813f350 con 0x7fed38151560 2023-09-23T03:28:19.392 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:28:19.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.394+0000 7fed4014c640 1 -- 172.21.15.154:0/165368480 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fed1c06af30 msgr2=0x7fed1c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:19.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.394+0000 7fed4014c640 1 --2- 172.21.15.154:0/165368480 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fed1c06af30 0x7fed1c06d3f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fed381f1380 tx=0x7fed34009040 comp rx=0 tx=0).stop 2023-09-23T03:28:19.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.394+0000 7fed4014c640 1 -- 172.21.15.154:0/165368480 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed38151560 msgr2=0x7fed38151a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:19.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.394+0000 7fed4014c640 1 --2- 172.21.15.154:0/165368480 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed38151560 0x7fed38151a10 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fed3000a950 tx=0x7fed30001d80 comp rx=0 tx=0).stop 2023-09-23T03:28:19.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.394+0000 7fed4014c640 1 -- 172.21.15.154:0/165368480 shutdown_connections 2023-09-23T03:28:19.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.394+0000 7fed4014c640 1 --2- 172.21.15.154:0/165368480 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fed1c06af30 0x7fed1c06d3f0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:19.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.395+0000 7fed4014c640 1 --2- 172.21.15.154:0/165368480 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed38151560 0x7fed38151a10 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:19.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.395+0000 7fed4014c640 1 --2- 172.21.15.154:0/165368480 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fed38140b00 0x7fed38140f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:19.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.395+0000 7fed4014c640 1 --2- 172.21.15.154:0/165368480 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fed24007390 0x7fed380c96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:19.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.395+0000 7fed4014c640 1 -- 172.21.15.154:0/165368480 >> 172.21.15.154:0/165368480 conn(0x7fed3806a440 msgr2=0x7fed38158bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:19.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.395+0000 7fed4014c640 1 -- 172.21.15.154:0/165368480 shutdown_connections 2023-09-23T03:28:19.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.395+0000 7fed4014c640 1 -- 172.21.15.154:0/165368480 wait complete. 2023-09-23T03:28:19.413 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:28:19.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.629+0000 7f10f5e94640 1 Processor -- start 2023-09-23T03:28:19.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.629+0000 7f10f5e94640 1 -- start start 2023-09-23T03:28:19.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.631+0000 7f10f5e94640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10f00ca6b0 0x7f10f006d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:19.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.631+0000 7f10f5e94640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f10f006dbd0 0x7f10f00700e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:19.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.631+0000 7f10f5e94640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10f015ab20 con 0x7f10f0070620 2023-09-23T03:28:19.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.631+0000 7f10f5e94640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10f0070ab0 con 0x7f10f006dbd0 2023-09-23T03:28:19.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.631+0000 7f10f5e94640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f0159f50 con 0x7f10f00ca6b0 2023-09-23T03:28:19.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.632+0000 7f10e7fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f10f006dbd0 0x7f10f00700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:19.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.632+0000 7f10ef7fe640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10f00ca6b0 0x7f10f006d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:19.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.632+0000 7f10effff640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7f10f0070620 0x7f10f006c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.154:59928/0 (socket says 172.21.15.154:59928) 2023-09-23T03:28:19.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.632+0000 7f10effff640 1 -- 172.21.15.154:0/761104197 learned_addr learned my addr 172.21.15.154:0/761104197 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:28:19.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.633+0000 7f10ef7fe640 1 -- 172.21.15.154:0/761104197 >> v2:172.21.15.190:3300/0 conn(0x7f10f006dbd0 msgr2=0x7f10f00700e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:19.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.633+0000 7f10ef7fe640 1 --2- 172.21.15.154:0/761104197 >> v2:172.21.15.190:3300/0 conn(0x7f10f006dbd0 0x7f10f00700e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:19.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.633+0000 7f10ef7fe640 1 -- 172.21.15.154:0/761104197 >> v1:172.21.15.190:6789/0 conn(0x7f10f0070620 legacy=0x7f10f006c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:19.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.633+0000 7f10ef7fe640 1 -- 172.21.15.154:0/761104197 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f00caa90 con 0x7f10f00ca6b0 2023-09-23T03:28:19.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.635+0000 7f10ef7fe640 1 --2- 172.21.15.154:0/761104197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10f00ca6b0 0x7f10f006d690 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f10e00097a0 tx=0x7f10e002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b34f39e55f89f1f5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:19.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.635+0000 7f10eeffd640 1 -- 172.21.15.154:0/761104197 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e002d040 con 0x7f10f00ca6b0 2023-09-23T03:28:19.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.635+0000 7f10eeffd640 1 -- 172.21.15.154:0/761104197 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e0031810 con 0x7f10f00ca6b0 2023-09-23T03:28:19.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.635+0000 7f10eeffd640 1 -- 172.21.15.154:0/761104197 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e0031ba0 con 0x7f10f00ca6b0 2023-09-23T03:28:19.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.636+0000 7f10f5e94640 1 -- 172.21.15.154:0/761104197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10f00ca6b0 msgr2=0x7f10f006d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:19.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.636+0000 7f10f5e94640 1 --2- 172.21.15.154:0/761104197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10f00ca6b0 0x7f10f006d690 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f10e00097a0 tx=0x7f10e002b8b0 comp rx=0 tx=0).stop 2023-09-23T03:28:19.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.636+0000 7f10f5e94640 1 -- 172.21.15.154:0/761104197 shutdown_connections 2023-09-23T03:28:19.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.636+0000 7f10f5e94640 1 --2- 172.21.15.154:0/761104197 >> v2:172.21.15.190:3300/0 conn(0x7f10f006dbd0 0x7f10f00700e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:19.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.636+0000 7f10f5e94640 1 --2- 172.21.15.154:0/761104197 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10f00ca6b0 0x7f10f006d690 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:19.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.636+0000 7f10f5e94640 1 -- 172.21.15.154:0/761104197 >> 172.21.15.154:0/761104197 conn(0x7f10f006a440 msgr2=0x7f10f00b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:19.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.636+0000 7f10f5e94640 1 -- 172.21.15.154:0/761104197 shutdown_connections 2023-09-23T03:28:19.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.636+0000 7f10f5e94640 1 -- 172.21.15.154:0/761104197 wait complete. 2023-09-23T03:28:19.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.638+0000 7f10f5e94640 1 Processor -- start 2023-09-23T03:28:19.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.638+0000 7f10f5e94640 1 -- start start 2023-09-23T03:28:19.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.639+0000 7f10f5e94640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10f006dbd0 0x7f10f01469e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:19.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.639+0000 7f10f5e94640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10f0070620 0x7f10f014fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:19.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.639+0000 7f10f5e94640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10f00ca6b0 0x7f10f0150200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:19.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.639+0000 7f10f5e94640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f00cbb80 con 0x7f10f006dbd0 2023-09-23T03:28:19.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.639+0000 7f10f5e94640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f00cacb0 con 0x7f10f0070620 2023-09-23T03:28:19.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.639+0000 7f10f5e94640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f00caf00 con 0x7f10f00ca6b0 2023-09-23T03:28:19.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.639+0000 7f10ef7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10f006dbd0 0x7f10f01469e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:19.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.639+0000 7f10ef7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10f006dbd0 0x7f10f01469e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59646/0 (socket says 172.21.15.154:59646) 2023-09-23T03:28:19.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.639+0000 7f10ef7fe640 1 -- 172.21.15.154:0/3193236028 learned_addr learned my addr 172.21.15.154:0/3193236028 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:19.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.640+0000 7f10e7fff640 1 --2- 172.21.15.154:0/3193236028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10f0070620 0x7f10f014fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:19.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.640+0000 7f10ef7fe640 1 -- 172.21.15.154:0/3193236028 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10f00ca6b0 msgr2=0x7f10f0150200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:19.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.640+0000 7f10ef7fe640 1 --2- 172.21.15.154:0/3193236028 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10f00ca6b0 0x7f10f0150200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:19.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.640+0000 7f10ef7fe640 1 -- 172.21.15.154:0/3193236028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10f0070620 msgr2=0x7f10f014fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:19.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.640+0000 7f10ef7fe640 1 --2- 172.21.15.154:0/3193236028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10f0070620 0x7f10f014fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:19.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.640+0000 7f10ef7fe640 1 -- 172.21.15.154:0/3193236028 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e0009400 con 0x7f10f006dbd0 2023-09-23T03:28:19.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.640+0000 7f10e7fff640 1 --2- 172.21.15.154:0/3193236028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10f0070620 0x7f10f014fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:19.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.640+0000 7f10ef7fe640 1 --2- 172.21.15.154:0/3193236028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10f006dbd0 0x7f10f01469e0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f10e0000c00 tx=0x7f10e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:19.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.640+0000 7f10ecff9640 1 -- 172.21.15.154:0/3193236028 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e0006e10 con 0x7f10f006dbd0 2023-09-23T03:28:19.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.641+0000 7f10ecff9640 1 -- 172.21.15.154:0/3193236028 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e003ad60 con 0x7f10f006dbd0 2023-09-23T03:28:19.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.641+0000 7f10f5e94640 1 -- 172.21.15.154:0/3193236028 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f00cb1e0 con 0x7f10f006dbd0 2023-09-23T03:28:19.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.641+0000 7f10ecff9640 1 -- 172.21.15.154:0/3193236028 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e0044630 con 0x7f10f006dbd0 2023-09-23T03:28:19.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.641+0000 7f10f5e94640 1 -- 172.21.15.154:0/3193236028 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f00cb7a0 con 0x7f10f006dbd0 2023-09-23T03:28:19.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.642+0000 7f10f5e94640 1 -- 172.21.15.154:0/3193236028 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f0073eb0 con 0x7f10f006dbd0 2023-09-23T03:28:19.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.643+0000 7f10ecff9640 1 -- 172.21.15.154:0/3193236028 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f10e004d450 con 0x7f10f006dbd0 2023-09-23T03:28:19.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.644+0000 7f10ecff9640 1 --2- 172.21.15.154:0/3193236028 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f10d006af30 0x7f10d006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:19.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.644+0000 7f10ecff9640 1 -- 172.21.15.154:0/3193236028 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f10e00abb20 con 0x7f10f006dbd0 2023-09-23T03:28:19.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.644+0000 7f10e7fff640 1 --2- 172.21.15.154:0/3193236028 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f10d006af30 0x7f10d006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:19.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.645+0000 7f10e7fff640 1 --2- 172.21.15.154:0/3193236028 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f10d006af30 0x7f10d006d3f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f10f01526e0 tx=0x7f10d800b3a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:19.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.648+0000 7f10ecff9640 1 -- 172.21.15.154:0/3193236028 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f10e0076920 con 0x7f10f006dbd0 2023-09-23T03:28:19.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.839+0000 7f10f5e94640 1 -- 172.21.15.154:0/3193236028 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f10f00b0bf0 con 0x7f10f006dbd0 2023-09-23T03:28:19.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.840+0000 7f10ecff9640 1 -- 172.21.15.154:0/3193236028 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v127) v1 ==== 100+0+11 (secure 0 0 0) 0x7f10e00350a0 con 0x7f10f006dbd0 2023-09-23T03:28:19.840 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:28:19.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.842+0000 7f10f5e94640 1 -- 172.21.15.154:0/3193236028 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f10d006af30 msgr2=0x7f10d006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:19.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.842+0000 7f10f5e94640 1 --2- 172.21.15.154:0/3193236028 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f10d006af30 0x7f10d006d3f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f10f01526e0 tx=0x7f10d800b3a0 comp rx=0 tx=0).stop 2023-09-23T03:28:19.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.842+0000 7f10f5e94640 1 -- 172.21.15.154:0/3193236028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10f006dbd0 msgr2=0x7f10f01469e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:19.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.842+0000 7f10f5e94640 1 --2- 172.21.15.154:0/3193236028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10f006dbd0 0x7f10f01469e0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f10e0000c00 tx=0x7f10e00046c0 comp rx=0 tx=0).stop 2023-09-23T03:28:19.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.843+0000 7f10f5e94640 1 -- 172.21.15.154:0/3193236028 shutdown_connections 2023-09-23T03:28:19.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.843+0000 7f10f5e94640 1 --2- 172.21.15.154:0/3193236028 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f10d006af30 0x7f10d006d3f0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:19.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.843+0000 7f10f5e94640 1 --2- 172.21.15.154:0/3193236028 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f10f00ca6b0 0x7f10f0150200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:19.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.843+0000 7f10f5e94640 1 --2- 172.21.15.154:0/3193236028 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f10f0070620 0x7f10f014fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:19.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.843+0000 7f10f5e94640 1 --2- 172.21.15.154:0/3193236028 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f10f006dbd0 0x7f10f01469e0 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:19.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.843+0000 7f10f5e94640 1 -- 172.21.15.154:0/3193236028 >> 172.21.15.154:0/3193236028 conn(0x7f10f006a440 msgr2=0x7f10f006c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:19.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.843+0000 7f10f5e94640 1 -- 172.21.15.154:0/3193236028 shutdown_connections 2023-09-23T03:28:19.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:19.843+0000 7f10f5e94640 1 -- 172.21.15.154:0/3193236028 wait complete. 2023-09-23T03:28:19.861 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-23T03:28:20.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.070+0000 7f355faa2640 1 Processor -- start 2023-09-23T03:28:20.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.070+0000 7f355faa2640 1 -- start start 2023-09-23T03:28:20.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.072+0000 7f355faa2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f355815aa20 0x7f35580cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:20.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.072+0000 7f355faa2640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35580cd140 con 0x7f355815a640 2023-09-23T03:28:20.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.072+0000 7f355faa2640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35580cb910 con 0x7f35580cb4f0 2023-09-23T03:28:20.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.072+0000 7f355faa2640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35580cc530 con 0x7f355815aa20 2023-09-23T03:28:20.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.072+0000 7f355d03d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f355815aa20 0x7f35580cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:20.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.072+0000 7f355d83e640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f355815a640 0x7f35580b3e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:35290/0 (socket says 172.21.15.154:35290) 2023-09-23T03:28:20.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.072+0000 7f355d83e640 1 -- 172.21.15.154:0/4221919949 learned_addr learned my addr 172.21.15.154:0/4221919949 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:28:20.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.073+0000 7f355d03d640 1 -- 172.21.15.154:0/4221919949 >> v1:172.21.15.190:6789/0 conn(0x7f35580cb4f0 legacy=0x7f35580b2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:20.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.073+0000 7f355d03d640 1 -- 172.21.15.154:0/4221919949 >> v1:172.21.15.154:6789/0 conn(0x7f355815a640 legacy=0x7f35580b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:20.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.073+0000 7f355d03d640 1 -- 172.21.15.154:0/4221919949 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35580cc810 con 0x7f355815aa20 2023-09-23T03:28:20.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.074+0000 7f355d03d640 1 --2- 172.21.15.154:0/4221919949 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f355815aa20 0x7f35580cafb0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f35480097a0 tx=0x7f354802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b46686db1ca7d28a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:20.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.074+0000 7f355c83c640 1 -- 172.21.15.154:0/4221919949 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354802e070 con 0x7f355815aa20 2023-09-23T03:28:20.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.074+0000 7f355c83c640 1 -- 172.21.15.154:0/4221919949 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35480315a0 con 0x7f355815aa20 2023-09-23T03:28:20.074 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.074+0000 7f355c83c640 1 -- 172.21.15.154:0/4221919949 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3548031930 con 0x7f355815aa20 2023-09-23T03:28:20.075 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.075+0000 7f355faa2640 1 -- 172.21.15.154:0/4221919949 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f355815aa20 msgr2=0x7f35580cafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:20.075 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.075+0000 7f355faa2640 1 --2- 172.21.15.154:0/4221919949 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f355815aa20 0x7f35580cafb0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f35480097a0 tx=0x7f354802b840 comp rx=0 tx=0).stop 2023-09-23T03:28:20.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.075+0000 7f355faa2640 1 -- 172.21.15.154:0/4221919949 shutdown_connections 2023-09-23T03:28:20.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.075+0000 7f355faa2640 1 --2- 172.21.15.154:0/4221919949 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f355815aa20 0x7f35580cafb0 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:20.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.075+0000 7f355faa2640 1 -- 172.21.15.154:0/4221919949 >> 172.21.15.154:0/4221919949 conn(0x7f355806c820 msgr2=0x7f3558069cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:20.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.075+0000 7f355faa2640 1 -- 172.21.15.154:0/4221919949 shutdown_connections 2023-09-23T03:28:20.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.075+0000 7f355faa2640 1 -- 172.21.15.154:0/4221919949 wait complete. 2023-09-23T03:28:20.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.077+0000 7f355faa2640 1 Processor -- start 2023-09-23T03:28:20.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.077+0000 7f355faa2640 1 -- start start 2023-09-23T03:28:20.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.078+0000 7f355faa2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35580cb4f0 0x7f355815ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:20.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.078+0000 7f355faa2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f355815a640 0x7f355815c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:20.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.078+0000 7f355faa2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f355815aa20 0x7f355814ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:20.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.078+0000 7f355faa2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35580c6b90 con 0x7f35580cb4f0 2023-09-23T03:28:20.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.078+0000 7f355faa2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35580c6de0 con 0x7f355815aa20 2023-09-23T03:28:20.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.078+0000 7f355faa2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35580c70c0 con 0x7f355815a640 2023-09-23T03:28:20.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.078+0000 7f355d83e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35580cb4f0 0x7f355815ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:20.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.078+0000 7f355d83e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35580cb4f0 0x7f355815ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59666/0 (socket says 172.21.15.154:59666) 2023-09-23T03:28:20.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.078+0000 7f355d83e640 1 -- 172.21.15.154:0/2793748827 learned_addr learned my addr 172.21.15.154:0/2793748827 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:20.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.079+0000 7f355d83e640 1 -- 172.21.15.154:0/2793748827 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f355815a640 msgr2=0x7f355815c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:20.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.079+0000 7f355d83e640 1 --2- 172.21.15.154:0/2793748827 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f355815a640 0x7f355815c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:20.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.079+0000 7f355d83e640 1 -- 172.21.15.154:0/2793748827 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f355815aa20 msgr2=0x7f355814ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:20.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.079+0000 7f355e03f640 1 --2- 172.21.15.154:0/2793748827 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f355815aa20 0x7f355814ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:20.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.079+0000 7f355d83e640 1 --2- 172.21.15.154:0/2793748827 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f355815aa20 0x7f355814ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:20.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.079+0000 7f355d83e640 1 -- 172.21.15.154:0/2793748827 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3548009400 con 0x7f35580cb4f0 2023-09-23T03:28:20.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.079+0000 7f355e03f640 1 --2- 172.21.15.154:0/2793748827 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f355815aa20 0x7f355814ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:20.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.079+0000 7f355d83e640 1 --2- 172.21.15.154:0/2793748827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35580cb4f0 0x7f355815ae30 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f355400c4a0 tx=0x7f355400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:20.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.079+0000 7f354e7fc640 1 -- 172.21.15.154:0/2793748827 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3554005ce0 con 0x7f35580cb4f0 2023-09-23T03:28:20.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.079+0000 7f354e7fc640 1 -- 172.21.15.154:0/2793748827 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3554005e40 con 0x7f35580cb4f0 2023-09-23T03:28:20.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.079+0000 7f354e7fc640 1 -- 172.21.15.154:0/2793748827 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3554007270 con 0x7f35580cb4f0 2023-09-23T03:28:20.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.079+0000 7f355faa2640 1 -- 172.21.15.154:0/2793748827 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35580c73a0 con 0x7f35580cb4f0 2023-09-23T03:28:20.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.079+0000 7f355faa2640 1 -- 172.21.15.154:0/2793748827 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35580c7960 con 0x7f35580cb4f0 2023-09-23T03:28:20.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.081+0000 7f355faa2640 1 -- 172.21.15.154:0/2793748827 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3558073eb0 con 0x7f35580cb4f0 2023-09-23T03:28:20.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.082+0000 7f354e7fc640 1 -- 172.21.15.154:0/2793748827 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f3554007490 con 0x7f35580cb4f0 2023-09-23T03:28:20.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.083+0000 7f354e7fc640 1 --2- 172.21.15.154:0/2793748827 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f353c06af30 0x7f353c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:20.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.083+0000 7f354e7fc640 1 -- 172.21.15.154:0/2793748827 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f3554088e70 con 0x7f35580cb4f0 2023-09-23T03:28:20.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.083+0000 7f355d03d640 1 --2- 172.21.15.154:0/2793748827 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f353c06af30 0x7f353c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:20.084 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.084+0000 7f355d03d640 1 --2- 172.21.15.154:0/2793748827 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f353c06af30 0x7f353c06d3f0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f3548036000 tx=0x7f3548030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:20.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.087+0000 7f354e7fc640 1 -- 172.21.15.154:0/2793748827 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f3554053d20 con 0x7f35580cb4f0 2023-09-23T03:28:20.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.288+0000 7f355faa2640 1 -- 172.21.15.154:0/2793748827 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f35580cd430 con 0x7f35580cb4f0 2023-09-23T03:28:20.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.289+0000 7f354e7fc640 1 -- 172.21.15.154:0/2793748827 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v127) v1 ==== 80+0+124 (secure 0 0 0) 0x7f3554017020 con 0x7f35580cb4f0 2023-09-23T03:28:20.289 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.190:6835/3558096060 2023-09-24T03:28:08.468628+0000 2023-09-23T03:28:20.289 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.190:6833/3558096060 2023-09-24T03:28:08.468628+0000 2023-09-23T03:28:20.289 INFO:teuthology.orchestra.run.smithi154.stderr:listed 2 entries 2023-09-23T03:28:20.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.291+0000 7f355faa2640 1 -- 172.21.15.154:0/2793748827 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f353c06af30 msgr2=0x7f353c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:20.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.291+0000 7f355faa2640 1 --2- 172.21.15.154:0/2793748827 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f353c06af30 0x7f353c06d3f0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f3548036000 tx=0x7f3548030000 comp rx=0 tx=0).stop 2023-09-23T03:28:20.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.291+0000 7f355faa2640 1 -- 172.21.15.154:0/2793748827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35580cb4f0 msgr2=0x7f355815ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:20.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.291+0000 7f355faa2640 1 --2- 172.21.15.154:0/2793748827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35580cb4f0 0x7f355815ae30 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f355400c4a0 tx=0x7f355400c960 comp rx=0 tx=0).stop 2023-09-23T03:28:20.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.291+0000 7f355faa2640 1 -- 172.21.15.154:0/2793748827 shutdown_connections 2023-09-23T03:28:20.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.291+0000 7f355faa2640 1 --2- 172.21.15.154:0/2793748827 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f353c06af30 0x7f353c06d3f0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:20.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.291+0000 7f355faa2640 1 --2- 172.21.15.154:0/2793748827 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f355815aa20 0x7f355814ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:20.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.291+0000 7f355faa2640 1 --2- 172.21.15.154:0/2793748827 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f355815a640 0x7f355815c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:20.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.291+0000 7f355faa2640 1 --2- 172.21.15.154:0/2793748827 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35580cb4f0 0x7f355815ae30 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:20.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.291+0000 7f355faa2640 1 -- 172.21.15.154:0/2793748827 >> 172.21.15.154:0/2793748827 conn(0x7f355806c820 msgr2=0x7f35580c3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:20.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.291+0000 7f355faa2640 1 -- 172.21.15.154:0/2793748827 shutdown_connections 2023-09-23T03:28:20.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.292+0000 7f355faa2640 1 -- 172.21.15.154:0/2793748827 wait complete. 2023-09-23T03:28:20.309 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:28:20.309 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:28:20.360 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:28:20.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.576+0000 7f708b2ea640 1 Processor -- start 2023-09-23T03:28:20.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.576+0000 7f708b2ea640 1 -- start start 2023-09-23T03:28:20.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.577+0000 7f708b2ea640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f70840ca3c0 0x7f7084070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:20.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.577+0000 7f708b2ea640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70840c9a40 con 0x7f7084159ba0 2023-09-23T03:28:20.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.578+0000 7f708b2ea640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f708415a5f0 con 0x7f70840cb320 2023-09-23T03:28:20.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.578+0000 7f708b2ea640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f708415abb0 con 0x7f70840ca3c0 2023-09-23T03:28:20.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.578+0000 7f7089ae7640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f7084159ba0 0x7f70840c9cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:35306/0 (socket says 172.21.15.154:35306) 2023-09-23T03:28:20.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.578+0000 7f7089ae7640 1 -- 172.21.15.154:0/3899312722 learned_addr learned my addr 172.21.15.154:0/3899312722 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:28:20.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.578+0000 7f708aae9640 1 --2- 172.21.15.154:0/3899312722 >> v2:172.21.15.190:3300/0 conn(0x7f70840ca3c0 0x7f7084070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:20.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.578+0000 7f70892e6640 1 -- 172.21.15.154:0/3899312722 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2406627529 0 0) 0x7f70840c9a40 con 0x7f7084159ba0 2023-09-23T03:28:20.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.578+0000 7f70892e6640 1 -- 172.21.15.154:0/3899312722 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7070003520 con 0x7f7084159ba0 2023-09-23T03:28:20.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.579+0000 7f70892e6640 1 -- 172.21.15.154:0/3899312722 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 295403518 0 0) 0x7f7070003520 con 0x7f7084159ba0 2023-09-23T03:28:20.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.579+0000 7f70892e6640 1 -- 172.21.15.154:0/3899312722 >> v2:172.21.15.190:3300/0 conn(0x7f70840ca3c0 msgr2=0x7f7084070ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:20.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.579+0000 7f70892e6640 1 --2- 172.21.15.154:0/3899312722 >> v2:172.21.15.190:3300/0 conn(0x7f70840ca3c0 0x7f7084070ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:20.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.579+0000 7f70892e6640 1 -- 172.21.15.154:0/3899312722 >> v1:172.21.15.190:6789/0 conn(0x7f70840cb320 legacy=0x7f70840c9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:20.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.579+0000 7f70892e6640 1 -- 172.21.15.154:0/3899312722 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f708415a8d0 con 0x7f7084159ba0 2023-09-23T03:28:20.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.579+0000 7f70892e6640 1 -- 172.21.15.154:0/3899312722 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f7074001670 con 0x7f7084159ba0 2023-09-23T03:28:20.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.579+0000 7f70892e6640 1 -- 172.21.15.154:0/3899312722 >> v1:172.21.15.154:6789/0 conn(0x7f7084159ba0 legacy=0x7f70840c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:20.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.579+0000 7f70892e6640 1 --2- 172.21.15.154:0/3899312722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7070003970 0x7f7070003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:20.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.579+0000 7f70892e6640 1 --2- 172.21.15.154:0/3899312722 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70700047c0 0x7f7070004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:20.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.579+0000 7f70892e6640 1 --2- 172.21.15.154:0/3899312722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7070005180 0x7f707000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:20.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.579+0000 7f70892e6640 1 -- 172.21.15.154:0/3899312722 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708415abb0 con 0x7f7070003970 2023-09-23T03:28:20.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.579+0000 7f70892e6640 1 -- 172.21.15.154:0/3899312722 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707000c0b0 con 0x7f7070005180 2023-09-23T03:28:20.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.579+0000 7f70892e6640 1 -- 172.21.15.154:0/3899312722 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f707000c270 con 0x7f70700047c0 2023-09-23T03:28:20.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.579+0000 7f708a2e8640 1 --2- 172.21.15.154:0/3899312722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7070003970 0x7f7070003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:20.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.580+0000 7f708a2e8640 1 -- 172.21.15.154:0/3899312722 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70700047c0 msgr2=0x7f7070004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:20.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.580+0000 7f708a2e8640 1 --2- 172.21.15.154:0/3899312722 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70700047c0 0x7f7070004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:20.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.580+0000 7f708a2e8640 1 -- 172.21.15.154:0/3899312722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7070005180 msgr2=0x7f707000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:20.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.580+0000 7f708a2e8640 1 --2- 172.21.15.154:0/3899312722 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7070005180 0x7f707000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:20.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.580+0000 7f708a2e8640 1 -- 172.21.15.154:0/3899312722 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7070003520 con 0x7f7070003970 2023-09-23T03:28:20.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.580+0000 7f708aae9640 1 -- 172.21.15.154:0/3899312722 reap_dead start 2023-09-23T03:28:20.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.581+0000 7f708a2e8640 1 --2- 172.21.15.154:0/3899312722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7070003970 0x7f7070003d70 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f7080003db0 tx=0x7f70700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e9de35d8525250 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:20.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.581+0000 7f70892e6640 1 -- 172.21.15.154:0/3899312722 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7080032600 con 0x7f7070003970 2023-09-23T03:28:20.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.581+0000 7f70892e6640 1 -- 172.21.15.154:0/3899312722 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f708002e050 con 0x7f7070003970 2023-09-23T03:28:20.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.581+0000 7f70892e6640 1 -- 172.21.15.154:0/3899312722 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70800329e0 con 0x7f7070003970 2023-09-23T03:28:20.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.581+0000 7f708b2ea640 1 -- 172.21.15.154:0/3899312722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7070003970 msgr2=0x7f7070003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:20.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.581+0000 7f708b2ea640 1 --2- 172.21.15.154:0/3899312722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7070003970 0x7f7070003d70 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f7080003db0 tx=0x7f70700057f0 comp rx=0 tx=0).stop 2023-09-23T03:28:20.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.582+0000 7f708b2ea640 1 -- 172.21.15.154:0/3899312722 shutdown_connections 2023-09-23T03:28:20.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.582+0000 7f708b2ea640 1 --2- 172.21.15.154:0/3899312722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7070003970 0x7f7070003d70 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:20.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.582+0000 7f708b2ea640 1 -- 172.21.15.154:0/3899312722 >> 172.21.15.154:0/3899312722 conn(0x7f708406a4c0 msgr2=0x7f70840afe50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:20.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.582+0000 7f708b2ea640 1 -- 172.21.15.154:0/3899312722 shutdown_connections 2023-09-23T03:28:20.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.582+0000 7f708b2ea640 1 -- 172.21.15.154:0/3899312722 wait complete. 2023-09-23T03:28:20.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.583+0000 7f708b2ea640 1 Processor -- start 2023-09-23T03:28:20.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.583+0000 7f708b2ea640 1 -- start start 2023-09-23T03:28:20.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.584+0000 7f708b2ea640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7070003970 0x7f70841531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:20.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.584+0000 7f708b2ea640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70840cc130 0x7f70840cc590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:20.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.584+0000 7f708b2ea640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70841e6210 0x7f70841e6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:20.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.584+0000 7f708b2ea640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70841e70d0 con 0x7f70840cc130 2023-09-23T03:28:20.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.584+0000 7f708b2ea640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70841e6bd0 con 0x7f7070003970 2023-09-23T03:28:20.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.584+0000 7f708b2ea640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70840ccad0 con 0x7f70841e6210 2023-09-23T03:28:20.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.585+0000 7f7089ae7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70840cc130 0x7f70840cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:20.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.585+0000 7f7089ae7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70840cc130 0x7f70840cc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59692/0 (socket says 172.21.15.154:59692) 2023-09-23T03:28:20.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.585+0000 7f7089ae7640 1 -- 172.21.15.154:0/1290980102 learned_addr learned my addr 172.21.15.154:0/1290980102 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:20.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.585+0000 7f708a2e8640 1 --2- 172.21.15.154:0/1290980102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7070003970 0x7f70841531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:20.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.585+0000 7f7089ae7640 1 -- 172.21.15.154:0/1290980102 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70841e6210 msgr2=0x7f70841e6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:20.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.585+0000 7f708aae9640 1 --2- 172.21.15.154:0/1290980102 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70841e6210 0x7f70841e6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:20.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.585+0000 7f7089ae7640 1 --2- 172.21.15.154:0/1290980102 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70841e6210 0x7f70841e6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:20.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.585+0000 7f7089ae7640 1 -- 172.21.15.154:0/1290980102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7070003970 msgr2=0x7f70841531b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:20.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.585+0000 7f7089ae7640 1 --2- 172.21.15.154:0/1290980102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7070003970 0x7f70841531b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:20.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.585+0000 7f7089ae7640 1 -- 172.21.15.154:0/1290980102 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70841e8e00 con 0x7f70840cc130 2023-09-23T03:28:20.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.586+0000 7f7089ae7640 1 --2- 172.21.15.154:0/1290980102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70840cc130 0x7f70840cc590 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f707400c770 tx=0x7f707400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:20.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.586+0000 7f707affd640 1 -- 172.21.15.154:0/1290980102 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70740074e0 con 0x7f70840cc130 2023-09-23T03:28:20.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.586+0000 7f707affd640 1 -- 172.21.15.154:0/1290980102 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7074007640 con 0x7f70840cc130 2023-09-23T03:28:20.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.586+0000 7f707affd640 1 -- 172.21.15.154:0/1290980102 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7074004a00 con 0x7f70840cc130 2023-09-23T03:28:20.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.586+0000 7f708b2ea640 1 -- 172.21.15.154:0/1290980102 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7084153a90 con 0x7f70840cc130 2023-09-23T03:28:20.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.586+0000 7f708b2ea640 1 -- 172.21.15.154:0/1290980102 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70841e87b0 con 0x7f70840cc130 2023-09-23T03:28:20.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.587+0000 7f708b2ea640 1 -- 172.21.15.154:0/1290980102 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7084066730 con 0x7f70840cc130 2023-09-23T03:28:20.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.588+0000 7f707affd640 1 -- 172.21.15.154:0/1290980102 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f7074004b60 con 0x7f70840cc130 2023-09-23T03:28:20.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.589+0000 7f707affd640 1 --2- 172.21.15.154:0/1290980102 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f706806af30 0x7f706806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:20.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.589+0000 7f707affd640 1 -- 172.21.15.154:0/1290980102 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f7074088dd0 con 0x7f70840cc130 2023-09-23T03:28:20.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.589+0000 7f708a2e8640 1 --2- 172.21.15.154:0/1290980102 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f706806af30 0x7f706806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:20.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.590+0000 7f708a2e8640 1 --2- 172.21.15.154:0/1290980102 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f706806af30 0x7f706806d3f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f7080034000 tx=0x7f7080009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:20.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.594+0000 7f707affd640 1 -- 172.21.15.154:0/1290980102 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f7074053b40 con 0x7f70840cc130 2023-09-23T03:28:20.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.783+0000 7f708b2ea640 1 -- 172.21.15.154:0/1290980102 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70840c8b00 con 0x7f70840cc130 2023-09-23T03:28:20.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.785+0000 7f707affd640 1 -- 172.21.15.154:0/1290980102 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+16951 (secure 0 0 0) 0x7f7074018070 con 0x7f70840cc130 2023-09-23T03:28:20.786 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:28:20.786 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":127,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:28:08.468660+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:27:42.207617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-09-23T03:27:43.473867+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.190:6835/3558096060":"2023-09-24T03:28:08.468628+0000","172.21.15.190:6833/3558096060":"2023-09-24T03:28:08.468628+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:28:20.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.788+0000 7f708b2ea640 1 -- 172.21.15.154:0/1290980102 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f706806af30 msgr2=0x7f706806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:20.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.788+0000 7f708b2ea640 1 --2- 172.21.15.154:0/1290980102 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f706806af30 0x7f706806d3f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f7080034000 tx=0x7f7080009030 comp rx=0 tx=0).stop 2023-09-23T03:28:20.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.788+0000 7f708b2ea640 1 -- 172.21.15.154:0/1290980102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70840cc130 msgr2=0x7f70840cc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:20.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.788+0000 7f708b2ea640 1 --2- 172.21.15.154:0/1290980102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70840cc130 0x7f70840cc590 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f707400c770 tx=0x7f707400cc30 comp rx=0 tx=0).stop 2023-09-23T03:28:20.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.788+0000 7f708b2ea640 1 -- 172.21.15.154:0/1290980102 shutdown_connections 2023-09-23T03:28:20.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.788+0000 7f708b2ea640 1 --2- 172.21.15.154:0/1290980102 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f706806af30 0x7f706806d3f0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:20.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.788+0000 7f708b2ea640 1 --2- 172.21.15.154:0/1290980102 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f70841e6210 0x7f70841e6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:20.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.788+0000 7f708b2ea640 1 --2- 172.21.15.154:0/1290980102 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f70840cc130 0x7f70840cc590 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:20.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.788+0000 7f708b2ea640 1 --2- 172.21.15.154:0/1290980102 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7070003970 0x7f70841531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:20.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.788+0000 7f708b2ea640 1 -- 172.21.15.154:0/1290980102 >> 172.21.15.154:0/1290980102 conn(0x7f708406a4c0 msgr2=0x7f70840726f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:20.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.788+0000 7f708b2ea640 1 -- 172.21.15.154:0/1290980102 shutdown_connections 2023-09-23T03:28:20.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:20.788+0000 7f708b2ea640 1 -- 172.21.15.154:0/1290980102 wait complete. 2023-09-23T03:28:20.807 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:27:42.207617+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:27:43.473867+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T03:28:20.808 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:28:21.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.009+0000 7efc4d378640 1 Processor -- start 2023-09-23T03:28:21.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.010+0000 7efc4d378640 1 -- start start 2023-09-23T03:28:21.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.011+0000 7efc4d378640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7efc4806c620 0x7efc480c9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:21.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.011+0000 7efc4d378640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efc4806dc60 0x7efc480c9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:21.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.011+0000 7efc4d378640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc4806cbc0 con 0x7efc4815ace0 2023-09-23T03:28:21.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.011+0000 7efc4d378640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc4815b0c0 con 0x7efc4806dc60 2023-09-23T03:28:21.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.011+0000 7efc4d378640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc4806cea0 con 0x7efc4806c620 2023-09-23T03:28:21.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.011+0000 7efc477fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efc4806dc60 0x7efc480c9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:21.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.011+0000 7efc477fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efc4806dc60 0x7efc480c9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59702/0 (socket says 172.21.15.154:59702) 2023-09-23T03:28:21.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.011+0000 7efc477fe640 1 -- 172.21.15.154:0/1866233572 learned_addr learned my addr 172.21.15.154:0/1866233572 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:21.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.011+0000 7efc3ffff640 1 --2- 172.21.15.154:0/1866233572 >> v2:172.21.15.190:3300/0 conn(0x7efc4806c620 0x7efc480c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:21.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.012+0000 7efc477fe640 1 -- 172.21.15.154:0/1866233572 >> v2:172.21.15.190:3300/0 conn(0x7efc4806c620 msgr2=0x7efc480c9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:21.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.012+0000 7efc477fe640 1 --2- 172.21.15.154:0/1866233572 >> v2:172.21.15.190:3300/0 conn(0x7efc4806c620 0x7efc480c9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:21.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.012+0000 7efc477fe640 1 -- 172.21.15.154:0/1866233572 >> v1:172.21.15.190:6789/0 conn(0x7efc4815ace0 legacy=0x7efc480b4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:21.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.012+0000 7efc477fe640 1 -- 172.21.15.154:0/1866233572 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc4806d180 con 0x7efc4806dc60 2023-09-23T03:28:21.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.013+0000 7efc477fe640 1 --2- 172.21.15.154:0/1866233572 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc4806dc60 0x7efc480c9870 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7efc480714f0 tx=0x7efc4002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f60d790984d8a7c5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:21.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.013+0000 7efc467fc640 1 -- 172.21.15.154:0/1866233572 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4002ca80 con 0x7efc4806dc60 2023-09-23T03:28:21.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.013+0000 7efc467fc640 1 -- 172.21.15.154:0/1866233572 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4002cbe0 con 0x7efc4806dc60 2023-09-23T03:28:21.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.013+0000 7efc467fc640 1 -- 172.21.15.154:0/1866233572 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc400343f0 con 0x7efc4806dc60 2023-09-23T03:28:21.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.014+0000 7efc4d378640 1 -- 172.21.15.154:0/1866233572 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc4806dc60 msgr2=0x7efc480c9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:21.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.014+0000 7efc4d378640 1 --2- 172.21.15.154:0/1866233572 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc4806dc60 0x7efc480c9870 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7efc480714f0 tx=0x7efc4002bee0 comp rx=0 tx=0).stop 2023-09-23T03:28:21.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.014+0000 7efc4d378640 1 -- 172.21.15.154:0/1866233572 shutdown_connections 2023-09-23T03:28:21.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.014+0000 7efc4d378640 1 --2- 172.21.15.154:0/1866233572 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc4806dc60 0x7efc480c9870 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:21.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.014+0000 7efc4d378640 1 --2- 172.21.15.154:0/1866233572 >> v2:172.21.15.190:3300/0 conn(0x7efc4806c620 0x7efc480c9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:21.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.014+0000 7efc4d378640 1 -- 172.21.15.154:0/1866233572 >> 172.21.15.154:0/1866233572 conn(0x7efc480cb070 msgr2=0x7efc480cb480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:21.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.014+0000 7efc4d378640 1 -- 172.21.15.154:0/1866233572 shutdown_connections 2023-09-23T03:28:21.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.014+0000 7efc4d378640 1 -- 172.21.15.154:0/1866233572 wait complete. 2023-09-23T03:28:21.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.016+0000 7efc4d378640 1 Processor -- start 2023-09-23T03:28:21.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.016+0000 7efc4d378640 1 -- start start 2023-09-23T03:28:21.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.017+0000 7efc4d378640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc4806c620 0x7efc481f2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:21.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.017+0000 7efc4d378640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc4806dc60 0x7efc4815dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:21.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.017+0000 7efc4d378640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc4815ace0 0x7efc48156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:21.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.017+0000 7efc4d378640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4815b310 con 0x7efc4815ace0 2023-09-23T03:28:21.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.017+0000 7efc4d378640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc48156880 con 0x7efc4806dc60 2023-09-23T03:28:21.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.017+0000 7efc4d378640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc48156b50 con 0x7efc4806c620 2023-09-23T03:28:21.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.017+0000 7efc477fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc4815ace0 0x7efc48156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:21.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.017+0000 7efc477fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc4815ace0 0x7efc48156340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59708/0 (socket says 172.21.15.154:59708) 2023-09-23T03:28:21.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.017+0000 7efc477fe640 1 -- 172.21.15.154:0/4127426834 learned_addr learned my addr 172.21.15.154:0/4127426834 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:21.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.018+0000 7efc477fe640 1 -- 172.21.15.154:0/4127426834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc4806c620 msgr2=0x7efc481f2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:21.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.018+0000 7efc477fe640 1 --2- 172.21.15.154:0/4127426834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc4806c620 0x7efc481f2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:21.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.018+0000 7efc477fe640 1 -- 172.21.15.154:0/4127426834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc4806dc60 msgr2=0x7efc4815dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:21.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.018+0000 7efc477fe640 1 --2- 172.21.15.154:0/4127426834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc4806dc60 0x7efc4815dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:21.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.018+0000 7efc477fe640 1 -- 172.21.15.154:0/4127426834 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc40009b30 con 0x7efc4815ace0 2023-09-23T03:28:21.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.018+0000 7efc477fe640 1 --2- 172.21.15.154:0/4127426834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc4815ace0 0x7efc48156340 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7efc40036040 tx=0x7efc40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:21.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.018+0000 7efc3f7fe640 1 -- 172.21.15.154:0/4127426834 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc40009510 con 0x7efc4815ace0 2023-09-23T03:28:21.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.018+0000 7efc3f7fe640 1 -- 172.21.15.154:0/4127426834 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4003e450 con 0x7efc4815ace0 2023-09-23T03:28:21.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.018+0000 7efc3f7fe640 1 -- 172.21.15.154:0/4127426834 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4003e7e0 con 0x7efc4815ace0 2023-09-23T03:28:21.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.018+0000 7efc4d378640 1 -- 172.21.15.154:0/4127426834 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc48156e30 con 0x7efc4815ace0 2023-09-23T03:28:21.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.018+0000 7efc4d378640 1 -- 172.21.15.154:0/4127426834 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc481573f0 con 0x7efc4815ace0 2023-09-23T03:28:21.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.020+0000 7efc4d378640 1 -- 172.21.15.154:0/4127426834 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc48067870 con 0x7efc4815ace0 2023-09-23T03:28:21.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.021+0000 7efc3f7fe640 1 -- 172.21.15.154:0/4127426834 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7efc4002cd50 con 0x7efc4815ace0 2023-09-23T03:28:21.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.021+0000 7efc3f7fe640 1 --2- 172.21.15.154:0/4127426834 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efc2806af30 0x7efc2806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:21.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.022+0000 7efc3f7fe640 1 -- 172.21.15.154:0/4127426834 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6115+0+0 (secure 0 0 0) 0x7efc400ac580 con 0x7efc4815ace0 2023-09-23T03:28:21.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.022+0000 7efc46ffd640 1 --2- 172.21.15.154:0/4127426834 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efc2806af30 0x7efc2806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:21.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.022+0000 7efc46ffd640 1 --2- 172.21.15.154:0/4127426834 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efc2806af30 0x7efc2806d3f0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7efc38009650 tx=0x7efc38006ba0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:21.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.026+0000 7efc3f7fe640 1 -- 172.21.15.154:0/4127426834 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7efc400772f0 con 0x7efc4815ace0 2023-09-23T03:28:21.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.210+0000 7efc4d378640 1 -- 172.21.15.154:0/4127426834 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efc48073eb0 con 0x7efc4815ace0 2023-09-23T03:28:21.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.211+0000 7efc3f7fe640 1 -- 172.21.15.154:0/4127426834 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) v1 ==== 93+0+10 (secure 0 0 0) 0x7efc40043020 con 0x7efc4815ace0 2023-09-23T03:28:21.211 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:28:21.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.213+0000 7efc4d378640 1 -- 172.21.15.154:0/4127426834 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efc2806af30 msgr2=0x7efc2806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:21.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.213+0000 7efc4d378640 1 --2- 172.21.15.154:0/4127426834 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efc2806af30 0x7efc2806d3f0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7efc38009650 tx=0x7efc38006ba0 comp rx=0 tx=0).stop 2023-09-23T03:28:21.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.213+0000 7efc4d378640 1 -- 172.21.15.154:0/4127426834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc4815ace0 msgr2=0x7efc48156340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:21.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.213+0000 7efc4d378640 1 --2- 172.21.15.154:0/4127426834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc4815ace0 0x7efc48156340 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7efc40036040 tx=0x7efc40004a40 comp rx=0 tx=0).stop 2023-09-23T03:28:21.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.214+0000 7efc4d378640 1 -- 172.21.15.154:0/4127426834 shutdown_connections 2023-09-23T03:28:21.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.214+0000 7efc4d378640 1 --2- 172.21.15.154:0/4127426834 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efc2806af30 0x7efc2806d3f0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:21.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.214+0000 7efc4d378640 1 --2- 172.21.15.154:0/4127426834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc4815ace0 0x7efc48156340 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:21.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.214+0000 7efc4d378640 1 --2- 172.21.15.154:0/4127426834 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efc4806dc60 0x7efc4815dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:21.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.214+0000 7efc4d378640 1 --2- 172.21.15.154:0/4127426834 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efc4806c620 0x7efc481f2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:21.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.214+0000 7efc4d378640 1 -- 172.21.15.154:0/4127426834 >> 172.21.15.154:0/4127426834 conn(0x7efc480cb070 msgr2=0x7efc480c9750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:21.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.214+0000 7efc4d378640 1 -- 172.21.15.154:0/4127426834 shutdown_connections 2023-09-23T03:28:21.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.214+0000 7efc4d378640 1 -- 172.21.15.154:0/4127426834 wait complete. 2023-09-23T03:28:21.231 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:28:21.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.443+0000 7f68d3839640 1 Processor -- start 2023-09-23T03:28:21.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.443+0000 7f68d3839640 1 -- start start 2023-09-23T03:28:21.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.445+0000 7f68d3839640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68cc0b3f50 0x7f68cc06db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:21.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.445+0000 7f68d3839640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f68cc0cb8a0 0x7f68cc0700e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:21.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.445+0000 7f68d3839640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f68cc070620 0x7f68cc06c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:21.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.445+0000 7f68d3839640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68cc070a80 con 0x7f68cc070620 2023-09-23T03:28:21.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.445+0000 7f68d3839640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68cc0cb410 con 0x7f68cc0cb8a0 2023-09-23T03:28:21.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.445+0000 7f68d3839640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc0b4330 con 0x7f68cc0b3f50 2023-09-23T03:28:21.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.445+0000 7f68d1dd6640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f68cc070620 0x7f68cc06c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:21.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.445+0000 7f68d1dd6640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f68cc070620 0x7f68cc06c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59718/0 (socket says 172.21.15.154:59718) 2023-09-23T03:28:21.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.445+0000 7f68d1dd6640 1 -- 172.21.15.154:0/3245033594 learned_addr learned my addr 172.21.15.154:0/3245033594 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:21.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.446+0000 7f68d0dd4640 1 --2- 172.21.15.154:0/3245033594 >> v2:172.21.15.190:3300/0 conn(0x7f68cc0cb8a0 0x7f68cc0700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:21.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.446+0000 7f68d15d5640 1 --2- 172.21.15.154:0/3245033594 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68cc0b3f50 0x7f68cc06db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:21.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.446+0000 7f68d1dd6640 1 -- 172.21.15.154:0/3245033594 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68cc0b3f50 msgr2=0x7f68cc06db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:21.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.446+0000 7f68d1dd6640 1 --2- 172.21.15.154:0/3245033594 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68cc0b3f50 0x7f68cc06db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:21.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.446+0000 7f68d1dd6640 1 -- 172.21.15.154:0/3245033594 >> v2:172.21.15.190:3300/0 conn(0x7f68cc0cb8a0 msgr2=0x7f68cc0700e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:21.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.446+0000 7f68d1dd6640 1 --2- 172.21.15.154:0/3245033594 >> v2:172.21.15.190:3300/0 conn(0x7f68cc0cb8a0 0x7f68cc0700e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:21.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.446+0000 7f68d1dd6640 1 -- 172.21.15.154:0/3245033594 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68cc0b0d30 con 0x7f68cc070620 2023-09-23T03:28:21.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.447+0000 7f68d1dd6640 1 --2- 172.21.15.154:0/3245033594 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68cc070620 0x7f68cc06c1b0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f68cc071580 tx=0x7f68c402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4acc720bbd762d25 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:21.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.447+0000 7f68c3fff640 1 -- 172.21.15.154:0/3245033594 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c402ca70 con 0x7f68cc070620 2023-09-23T03:28:21.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.447+0000 7f68c3fff640 1 -- 172.21.15.154:0/3245033594 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c402cbd0 con 0x7f68cc070620 2023-09-23T03:28:21.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.447+0000 7f68c3fff640 1 -- 172.21.15.154:0/3245033594 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c40343f0 con 0x7f68cc070620 2023-09-23T03:28:21.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.448+0000 7f68d3839640 1 -- 172.21.15.154:0/3245033594 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68cc070620 msgr2=0x7f68cc06c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:21.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.448+0000 7f68d3839640 1 --2- 172.21.15.154:0/3245033594 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68cc070620 0x7f68cc06c1b0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f68cc071580 tx=0x7f68c402bed0 comp rx=0 tx=0).stop 2023-09-23T03:28:21.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.448+0000 7f68d3839640 1 -- 172.21.15.154:0/3245033594 shutdown_connections 2023-09-23T03:28:21.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.448+0000 7f68d3839640 1 --2- 172.21.15.154:0/3245033594 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68cc070620 0x7f68cc06c1b0 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:21.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.448+0000 7f68d3839640 1 --2- 172.21.15.154:0/3245033594 >> v2:172.21.15.190:3300/0 conn(0x7f68cc0cb8a0 0x7f68cc0700e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:21.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.448+0000 7f68d3839640 1 --2- 172.21.15.154:0/3245033594 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68cc0b3f50 0x7f68cc06db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:21.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.448+0000 7f68d3839640 1 -- 172.21.15.154:0/3245033594 >> 172.21.15.154:0/3245033594 conn(0x7f68cc06a440 msgr2=0x7f68cc0b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:21.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.448+0000 7f68d3839640 1 -- 172.21.15.154:0/3245033594 shutdown_connections 2023-09-23T03:28:21.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.448+0000 7f68d3839640 1 -- 172.21.15.154:0/3245033594 wait complete. 2023-09-23T03:28:21.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.450+0000 7f68d3839640 1 Processor -- start 2023-09-23T03:28:21.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.450+0000 7f68d3839640 1 -- start start 2023-09-23T03:28:21.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.451+0000 7f68d3839640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68cc070620 0x7f68cc0c73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:21.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.451+0000 7f68d3839640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68cc0b3f50 0x7f68cc164ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:21.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.451+0000 7f68d3839640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68cc0cb8a0 0x7f68cc163770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:21.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.451+0000 7f68d3839640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc1379b0 con 0x7f68cc070620 2023-09-23T03:28:21.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.451+0000 7f68d3839640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc0c4500 con 0x7f68cc0cb8a0 2023-09-23T03:28:21.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.451+0000 7f68d3839640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc0c4750 con 0x7f68cc0b3f50 2023-09-23T03:28:21.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.451+0000 7f68d15d5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68cc070620 0x7f68cc0c73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:21.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.451+0000 7f68d15d5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68cc070620 0x7f68cc0c73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59722/0 (socket says 172.21.15.154:59722) 2023-09-23T03:28:21.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.451+0000 7f68d15d5640 1 -- 172.21.15.154:0/656389884 learned_addr learned my addr 172.21.15.154:0/656389884 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:21.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.451+0000 7f68d0dd4640 1 --2- 172.21.15.154:0/656389884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68cc0b3f50 0x7f68cc164ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:21.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.451+0000 7f68d15d5640 1 -- 172.21.15.154:0/656389884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68cc0b3f50 msgr2=0x7f68cc164ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:21.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.451+0000 7f68d15d5640 1 --2- 172.21.15.154:0/656389884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68cc0b3f50 0x7f68cc164ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:21.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.451+0000 7f68d15d5640 1 -- 172.21.15.154:0/656389884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68cc0cb8a0 msgr2=0x7f68cc163770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:21.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.451+0000 7f68d15d5640 1 --2- 172.21.15.154:0/656389884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68cc0cb8a0 0x7f68cc163770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:21.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.451+0000 7f68d15d5640 1 -- 172.21.15.154:0/656389884 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c4009b30 con 0x7f68cc070620 2023-09-23T03:28:21.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.452+0000 7f68d15d5640 1 --2- 172.21.15.154:0/656389884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68cc070620 0x7f68cc0c73a0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f68c800d800 tx=0x7f68c800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:21.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.452+0000 7f68c1ffb640 1 -- 172.21.15.154:0/656389884 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c80096a0 con 0x7f68cc070620 2023-09-23T03:28:21.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.452+0000 7f68c1ffb640 1 -- 172.21.15.154:0/656389884 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c8009800 con 0x7f68cc070620 2023-09-23T03:28:21.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.452+0000 7f68c1ffb640 1 -- 172.21.15.154:0/656389884 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c8009b90 con 0x7f68cc070620 2023-09-23T03:28:21.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.452+0000 7f68d3839640 1 -- 172.21.15.154:0/656389884 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68cc0c4a30 con 0x7f68cc070620 2023-09-23T03:28:21.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.452+0000 7f68d3839640 1 -- 172.21.15.154:0/656389884 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68cc0c4ff0 con 0x7f68cc070620 2023-09-23T03:28:21.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.453+0000 7f68d3839640 1 -- 172.21.15.154:0/656389884 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68cc0c8c60 con 0x7f68cc070620 2023-09-23T03:28:21.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.454+0000 7f68c1ffb640 1 -- 172.21.15.154:0/656389884 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f68c8009cf0 con 0x7f68cc070620 2023-09-23T03:28:21.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.456+0000 7f68c1ffb640 1 --2- 172.21.15.154:0/656389884 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f68b006af30 0x7f68b006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:21.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.456+0000 7f68c1ffb640 1 -- 172.21.15.154:0/656389884 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f68c8089d90 con 0x7f68cc070620 2023-09-23T03:28:21.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.456+0000 7f68d0dd4640 1 --2- 172.21.15.154:0/656389884 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f68b006af30 0x7f68b006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:21.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.456+0000 7f68d0dd4640 1 --2- 172.21.15.154:0/656389884 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f68b006af30 0x7f68b006d3f0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f68bc006fd0 tx=0x7f68bc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:21.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.460+0000 7f68c1ffb640 1 -- 172.21.15.154:0/656389884 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f68c8054c30 con 0x7f68cc070620 2023-09-23T03:28:21.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.651+0000 7f68d3839640 1 -- 172.21.15.154:0/656389884 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f68cc073eb0 con 0x7f68cc070620 2023-09-23T03:28:21.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.652+0000 7f68c1ffb640 1 -- 172.21.15.154:0/656389884 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v127) v1 ==== 104+0+11 (secure 0 0 0) 0x7f68c800b440 con 0x7f68cc070620 2023-09-23T03:28:21.652 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:28:21.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.654+0000 7f68d3839640 1 -- 172.21.15.154:0/656389884 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f68b006af30 msgr2=0x7f68b006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:21.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.654+0000 7f68d3839640 1 --2- 172.21.15.154:0/656389884 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f68b006af30 0x7f68b006d3f0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f68bc006fd0 tx=0x7f68bc008040 comp rx=0 tx=0).stop 2023-09-23T03:28:21.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.654+0000 7f68d3839640 1 -- 172.21.15.154:0/656389884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68cc070620 msgr2=0x7f68cc0c73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:21.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.654+0000 7f68d3839640 1 --2- 172.21.15.154:0/656389884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68cc070620 0x7f68cc0c73a0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f68c800d800 tx=0x7f68c800dcc0 comp rx=0 tx=0).stop 2023-09-23T03:28:21.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.654+0000 7f68d3839640 1 -- 172.21.15.154:0/656389884 shutdown_connections 2023-09-23T03:28:21.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.654+0000 7f68d3839640 1 --2- 172.21.15.154:0/656389884 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f68b006af30 0x7f68b006d3f0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:21.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.654+0000 7f68d3839640 1 --2- 172.21.15.154:0/656389884 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f68cc0cb8a0 0x7f68cc163770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:21.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.654+0000 7f68d3839640 1 --2- 172.21.15.154:0/656389884 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f68cc0b3f50 0x7f68cc164ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:21.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.654+0000 7f68d3839640 1 --2- 172.21.15.154:0/656389884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f68cc070620 0x7f68cc0c73a0 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:21.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.654+0000 7f68d3839640 1 -- 172.21.15.154:0/656389884 >> 172.21.15.154:0/656389884 conn(0x7f68cc06a440 msgr2=0x7f68cc1494c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:21.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.654+0000 7f68d3839640 1 -- 172.21.15.154:0/656389884 shutdown_connections 2023-09-23T03:28:21.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.654+0000 7f68d3839640 1 -- 172.21.15.154:0/656389884 wait complete. 2023-09-23T03:28:21.671 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:28:21.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.892+0000 7f3ace410640 1 Processor -- start 2023-09-23T03:28:21.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.892+0000 7f3ace410640 1 -- start start 2023-09-23T03:28:21.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.894+0000 7f3ace410640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3ac8144130 0x7f3ac80b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:21.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.894+0000 7f3ace410640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ac80caf80 0x7f3ac80cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:21.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.894+0000 7f3ace410640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ac80cb920 con 0x7f3ac815e520 2023-09-23T03:28:21.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.894+0000 7f3ace410640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ac8070fb0 con 0x7f3ac8144130 2023-09-23T03:28:21.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.894+0000 7f3ace410640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac80700e0 con 0x7f3ac80caf80 2023-09-23T03:28:21.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.894+0000 7f3acc9ad640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f3ac815e520 0x7f3ac815e9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:35312/0 (socket says 172.21.15.154:35312) 2023-09-23T03:28:21.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.894+0000 7f3acc9ad640 1 -- 172.21.15.154:0/3241996042 learned_addr learned my addr 172.21.15.154:0/3241996042 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:28:21.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.894+0000 7f3ac77fe640 1 --2- 172.21.15.154:0/3241996042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ac80caf80 0x7f3ac80cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:21.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.894+0000 7f3ac7fff640 1 --2- 172.21.15.154:0/3241996042 >> v2:172.21.15.190:3300/0 conn(0x7f3ac8144130 0x7f3ac80b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:21.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.895+0000 7f3ac6ffd640 1 -- 172.21.15.154:0/3241996042 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1346995174 0 0) 0x7f3ac80cb920 con 0x7f3ac815e520 2023-09-23T03:28:21.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.895+0000 7f3ac6ffd640 1 -- 172.21.15.154:0/3241996042 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ab0003520 con 0x7f3ac815e520 2023-09-23T03:28:21.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.895+0000 7f3ac6ffd640 1 -- 172.21.15.154:0/3241996042 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3068054574 0 0) 0x7f3ab0003520 con 0x7f3ac815e520 2023-09-23T03:28:21.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.895+0000 7f3ac6ffd640 1 -- 172.21.15.154:0/3241996042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ac80caf80 msgr2=0x7f3ac80cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:21.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.895+0000 7f3ac6ffd640 1 --2- 172.21.15.154:0/3241996042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ac80caf80 0x7f3ac80cb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:21.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.895+0000 7f3ac6ffd640 1 -- 172.21.15.154:0/3241996042 >> v2:172.21.15.190:3300/0 conn(0x7f3ac8144130 msgr2=0x7f3ac80b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:21.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.895+0000 7f3ac6ffd640 1 --2- 172.21.15.154:0/3241996042 >> v2:172.21.15.190:3300/0 conn(0x7f3ac8144130 0x7f3ac80b1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:21.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.895+0000 7f3ac6ffd640 1 -- 172.21.15.154:0/3241996042 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac8070330 con 0x7f3ac815e520 2023-09-23T03:28:21.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.895+0000 7f3ac6ffd640 1 -- 172.21.15.154:0/3241996042 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f3ac00019f0 con 0x7f3ac815e520 2023-09-23T03:28:21.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.895+0000 7f3ac6ffd640 1 -- 172.21.15.154:0/3241996042 >> v1:172.21.15.154:6789/0 conn(0x7f3ac815e520 legacy=0x7f3ac815e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:21.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.895+0000 7f3ac6ffd640 1 --2- 172.21.15.154:0/3241996042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ac80caf80 0x7f3ab0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:21.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.895+0000 7f3ac6ffd640 1 --2- 172.21.15.154:0/3241996042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ab0003f40 0x7f3ab00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:21.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.895+0000 7f3ac6ffd640 1 --2- 172.21.15.154:0/3241996042 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ab0004d00 0x7f3ab0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:21.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.895+0000 7f3ac6ffd640 1 -- 172.21.15.154:0/3241996042 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac8070fb0 con 0x7f3ab0004d00 2023-09-23T03:28:21.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.895+0000 7f3ac6ffd640 1 -- 172.21.15.154:0/3241996042 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac80700e0 con 0x7f3ab0003f40 2023-09-23T03:28:21.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.895+0000 7f3ac6ffd640 1 -- 172.21.15.154:0/3241996042 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab000b980 con 0x7f3ac80caf80 2023-09-23T03:28:21.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.896+0000 7f3acc9ad640 1 --2- 172.21.15.154:0/3241996042 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ab0004d00 0x7f3ab0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:21.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.896+0000 7f3ac77fe640 1 --2- 172.21.15.154:0/3241996042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ab0003f40 0x7f3ab00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:21.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.896+0000 7f3ac7fff640 1 --2- 172.21.15.154:0/3241996042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ac80caf80 0x7f3ab0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:21.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.896+0000 7f3acc9ad640 1 -- 172.21.15.154:0/3241996042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ac80caf80 msgr2=0x7f3ab0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:21.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.896+0000 7f3acc9ad640 1 --2- 172.21.15.154:0/3241996042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ac80caf80 0x7f3ab0003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:21.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.896+0000 7f3acc9ad640 1 -- 172.21.15.154:0/3241996042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ab0003f40 msgr2=0x7f3ab00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:21.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.896+0000 7f3acc9ad640 1 --2- 172.21.15.154:0/3241996042 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ab0003f40 0x7f3ab00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:21.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.896+0000 7f3acc9ad640 1 -- 172.21.15.154:0/3241996042 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab0003520 con 0x7f3ab0004d00 2023-09-23T03:28:21.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.896+0000 7f3ac7fff640 1 --2- 172.21.15.154:0/3241996042 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ac80caf80 0x7f3ab0003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:21.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.897+0000 7f3acc9ad640 1 --2- 172.21.15.154:0/3241996042 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ab0004d00 0x7f3ab0005180 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f3ac000a7b0 tx=0x7f3ab00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9829339f1190ed6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:21.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.898+0000 7f3ac6ffd640 1 -- 172.21.15.154:0/3241996042 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0034630 con 0x7f3ab0004d00 2023-09-23T03:28:21.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.898+0000 7f3ac6ffd640 1 -- 172.21.15.154:0/3241996042 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac002f070 con 0x7f3ab0004d00 2023-09-23T03:28:21.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.898+0000 7f3ac6ffd640 1 -- 172.21.15.154:0/3241996042 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0034a10 con 0x7f3ab0004d00 2023-09-23T03:28:21.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.898+0000 7f3ace410640 1 -- 172.21.15.154:0/3241996042 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ab0004d00 msgr2=0x7f3ab0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:21.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.898+0000 7f3ace410640 1 --2- 172.21.15.154:0/3241996042 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ab0004d00 0x7f3ab0005180 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f3ac000a7b0 tx=0x7f3ab00057f0 comp rx=0 tx=0).stop 2023-09-23T03:28:21.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.898+0000 7f3acc9ad640 1 -- 172.21.15.154:0/3241996042 reap_dead start 2023-09-23T03:28:21.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.898+0000 7f3ace410640 1 -- 172.21.15.154:0/3241996042 shutdown_connections 2023-09-23T03:28:21.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.898+0000 7f3ace410640 1 -- 172.21.15.154:0/3241996042 >> 172.21.15.154:0/3241996042 conn(0x7f3ac806a440 msgr2=0x7f3ac80b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:21.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.898+0000 7f3ace410640 1 -- 172.21.15.154:0/3241996042 shutdown_connections 2023-09-23T03:28:21.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.898+0000 7f3ace410640 1 -- 172.21.15.154:0/3241996042 wait complete. 2023-09-23T03:28:21.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.900+0000 7f3ace410640 1 Processor -- start 2023-09-23T03:28:21.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.900+0000 7f3ace410640 1 -- start start 2023-09-23T03:28:21.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.901+0000 7f3ace410640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ab0004d00 0x7f3ac80ca540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:21.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.901+0000 7f3ace410640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ac80a9890 0x7f3ac80c78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:21.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.901+0000 7f3ace410640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ac80a9d10 0x7f3ac80c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:21.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.901+0000 7f3ace410640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac80c96e0 con 0x7f3ac80a9890 2023-09-23T03:28:21.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.901+0000 7f3ace410640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac80c8ad0 con 0x7f3ac80a9d10 2023-09-23T03:28:21.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.901+0000 7f3ace410640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac80c8db0 con 0x7f3ab0004d00 2023-09-23T03:28:21.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.902+0000 7f3ac77fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ac80a9890 0x7f3ac80c78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:21.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.902+0000 7f3ac77fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ac80a9890 0x7f3ac80c78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59738/0 (socket says 172.21.15.154:59738) 2023-09-23T03:28:21.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.902+0000 7f3ac77fe640 1 -- 172.21.15.154:0/3824125759 learned_addr learned my addr 172.21.15.154:0/3824125759 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:21.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.902+0000 7f3ac77fe640 1 -- 172.21.15.154:0/3824125759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ab0004d00 msgr2=0x7f3ac80ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:21.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.902+0000 7f3ac77fe640 1 --2- 172.21.15.154:0/3824125759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ab0004d00 0x7f3ac80ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:21.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.902+0000 7f3acc9ad640 1 --2- 172.21.15.154:0/3824125759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ac80a9d10 0x7f3ac80c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:21.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.902+0000 7f3ac77fe640 1 -- 172.21.15.154:0/3824125759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ac80a9d10 msgr2=0x7f3ac80c7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:21.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.902+0000 7f3ac77fe640 1 --2- 172.21.15.154:0/3824125759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ac80a9d10 0x7f3ac80c7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:21.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.902+0000 7f3ac77fe640 1 -- 172.21.15.154:0/3824125759 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac80c9090 con 0x7f3ac80a9890 2023-09-23T03:28:21.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.902+0000 7f3ac77fe640 1 --2- 172.21.15.154:0/3824125759 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ac80a9890 0x7f3ac80c78c0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f3abc011600 tx=0x7f3abc011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:21.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.903+0000 7f3ac4ff9640 1 -- 172.21.15.154:0/3824125759 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc00f920 con 0x7f3ac80a9890 2023-09-23T03:28:21.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.903+0000 7f3ac4ff9640 1 -- 172.21.15.154:0/3824125759 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3abc00fa80 con 0x7f3ac80a9890 2023-09-23T03:28:21.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.903+0000 7f3ac4ff9640 1 -- 172.21.15.154:0/3824125759 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc00fdf0 con 0x7f3ac80a9890 2023-09-23T03:28:21.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.903+0000 7f3ace410640 1 -- 172.21.15.154:0/3824125759 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac80c9370 con 0x7f3ac80a9890 2023-09-23T03:28:21.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.903+0000 7f3ace410640 1 -- 172.21.15.154:0/3824125759 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac80caa80 con 0x7f3ac80a9890 2023-09-23T03:28:21.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.904+0000 7f3ace410640 1 -- 172.21.15.154:0/3824125759 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ac8073eb0 con 0x7f3ac80a9890 2023-09-23T03:28:21.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.905+0000 7f3ac4ff9640 1 -- 172.21.15.154:0/3824125759 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f3abc015460 con 0x7f3ac80a9890 2023-09-23T03:28:21.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.906+0000 7f3ac4ff9640 1 --2- 172.21.15.154:0/3824125759 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3ab406af30 0x7f3ab406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:21.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.906+0000 7f3ac4ff9640 1 -- 172.21.15.154:0/3824125759 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f3abc089d70 con 0x7f3ac80a9890 2023-09-23T03:28:21.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.906+0000 7f3ac7fff640 1 --2- 172.21.15.154:0/3824125759 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3ab406af30 0x7f3ab406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:21.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.907+0000 7f3ac7fff640 1 --2- 172.21.15.154:0/3824125759 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3ab406af30 0x7f3ab406d3f0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8009a60 tx=0x7f3ab8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:21.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:21.910+0000 7f3ac4ff9640 1 -- 172.21.15.154:0/3824125759 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f3abc054ca0 con 0x7f3ac80a9890 2023-09-23T03:28:22.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.100+0000 7f3ace410640 1 -- 172.21.15.154:0/3824125759 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3ac80cc290 con 0x7f3ac80a9890 2023-09-23T03:28:22.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.101+0000 7f3ac4ff9640 1 -- 172.21.15.154:0/3824125759 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v127) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3abc0220a0 con 0x7f3ac80a9890 2023-09-23T03:28:22.102 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:28:22.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.104+0000 7f3ace410640 1 -- 172.21.15.154:0/3824125759 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3ab406af30 msgr2=0x7f3ab406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:22.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.104+0000 7f3ace410640 1 --2- 172.21.15.154:0/3824125759 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3ab406af30 0x7f3ab406d3f0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8009a60 tx=0x7f3ab8008040 comp rx=0 tx=0).stop 2023-09-23T03:28:22.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.104+0000 7f3ace410640 1 -- 172.21.15.154:0/3824125759 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ac80a9890 msgr2=0x7f3ac80c78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:22.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.104+0000 7f3ace410640 1 --2- 172.21.15.154:0/3824125759 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ac80a9890 0x7f3ac80c78c0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f3abc011600 tx=0x7f3abc011ac0 comp rx=0 tx=0).stop 2023-09-23T03:28:22.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.104+0000 7f3ace410640 1 -- 172.21.15.154:0/3824125759 shutdown_connections 2023-09-23T03:28:22.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.104+0000 7f3ace410640 1 --2- 172.21.15.154:0/3824125759 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3ab406af30 0x7f3ab406d3f0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:22.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.104+0000 7f3ace410640 1 --2- 172.21.15.154:0/3824125759 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3ac80a9d10 0x7f3ac80c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:22.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.104+0000 7f3ace410640 1 --2- 172.21.15.154:0/3824125759 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ac80a9890 0x7f3ac80c78c0 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:22.106 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.104+0000 7f3ace410640 1 --2- 172.21.15.154:0/3824125759 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3ab0004d00 0x7f3ac80ca540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:22.106 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.104+0000 7f3ace410640 1 -- 172.21.15.154:0/3824125759 >> 172.21.15.154:0/3824125759 conn(0x7f3ac806a440 msgr2=0x7f3ac806c810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:22.106 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.104+0000 7f3ace410640 1 -- 172.21.15.154:0/3824125759 shutdown_connections 2023-09-23T03:28:22.106 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.105+0000 7f3ace410640 1 -- 172.21.15.154:0/3824125759 wait complete. 2023-09-23T03:28:22.124 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-23T03:28:22.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.341+0000 7f6ca359e640 1 Processor -- start 2023-09-23T03:28:22.342 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.342+0000 7f6ca359e640 1 -- start start 2023-09-23T03:28:22.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.343+0000 7f6ca359e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ca40b4a00 0x7f6ca40ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:22.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.343+0000 7f6ca359e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ca406db40 con 0x7f6ca415a680 2023-09-23T03:28:22.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.343+0000 7f6ca359e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ca406d2a0 con 0x7f6ca406b3f0 2023-09-23T03:28:22.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.343+0000 7f6ca359e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca406d580 con 0x7f6ca40b4a00 2023-09-23T03:28:22.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.343+0000 7f6ca259c640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f6ca415a680 0x7f6ca406ace0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:35320/0 (socket says 172.21.15.154:35320) 2023-09-23T03:28:22.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.343+0000 7f6ca259c640 1 -- 172.21.15.154:0/1878164938 learned_addr learned my addr 172.21.15.154:0/1878164938 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:28:22.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.344+0000 7f6ca159a640 1 -- 172.21.15.154:0/1878164938 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 928235604 0 0) 0x7f6ca406db40 con 0x7f6ca415a680 2023-09-23T03:28:22.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.344+0000 7f6ca1d9b640 1 --2- 172.21.15.154:0/1878164938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ca40b4a00 0x7f6ca40ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:22.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.344+0000 7f6ca159a640 1 -- 172.21.15.154:0/1878164938 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c90003520 con 0x7f6ca415a680 2023-09-23T03:28:22.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.344+0000 7f6ca159a640 1 -- 172.21.15.154:0/1878164938 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 350263085 0 0) 0x7f6c90003520 con 0x7f6ca415a680 2023-09-23T03:28:22.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.344+0000 7f6ca159a640 1 -- 172.21.15.154:0/1878164938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ca40b4a00 msgr2=0x7f6ca40ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:22.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.344+0000 7f6ca159a640 1 --2- 172.21.15.154:0/1878164938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ca40b4a00 0x7f6ca40ccec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:22.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.344+0000 7f6ca159a640 1 -- 172.21.15.154:0/1878164938 >> v1:172.21.15.190:6789/0 conn(0x7f6ca406b3f0 legacy=0x7f6ca40cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:22.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.344+0000 7f6ca159a640 1 -- 172.21.15.154:0/1878164938 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca406d860 con 0x7f6ca415a680 2023-09-23T03:28:22.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.344+0000 7f6ca159a640 1 -- 172.21.15.154:0/1878164938 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f6c94001670 con 0x7f6ca415a680 2023-09-23T03:28:22.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.345+0000 7f6ca159a640 1 -- 172.21.15.154:0/1878164938 >> v1:172.21.15.154:6789/0 conn(0x7f6ca415a680 legacy=0x7f6ca406ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:22.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.345+0000 7f6ca159a640 1 --2- 172.21.15.154:0/1878164938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c900039a0 0x7f6c90003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:22.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.345+0000 7f6ca159a640 1 --2- 172.21.15.154:0/1878164938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c900047c0 0x7f6c90004c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:22.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.345+0000 7f6ca159a640 1 --2- 172.21.15.154:0/1878164938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c90005140 0x7f6c9000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:22.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.345+0000 7f6ca159a640 1 -- 172.21.15.154:0/1878164938 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca406d580 con 0x7f6c90005140 2023-09-23T03:28:22.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.345+0000 7f6ca159a640 1 -- 172.21.15.154:0/1878164938 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9000bec0 con 0x7f6c900039a0 2023-09-23T03:28:22.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.345+0000 7f6ca159a640 1 -- 172.21.15.154:0/1878164938 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9000c030 con 0x7f6c900047c0 2023-09-23T03:28:22.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.345+0000 7f6ca2d9d640 1 --2- 172.21.15.154:0/1878164938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c90005140 0x7f6c9000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:22.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.345+0000 7f6ca259c640 1 --2- 172.21.15.154:0/1878164938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c900039a0 0x7f6c90003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:22.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.345+0000 7f6ca2d9d640 1 -- 172.21.15.154:0/1878164938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c900047c0 msgr2=0x7f6c90004c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:22.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.345+0000 7f6ca1d9b640 1 --2- 172.21.15.154:0/1878164938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c900047c0 0x7f6c90004c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:22.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.345+0000 7f6ca2d9d640 1 --2- 172.21.15.154:0/1878164938 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c900047c0 0x7f6c90004c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:22.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.346+0000 7f6ca2d9d640 1 -- 172.21.15.154:0/1878164938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c900039a0 msgr2=0x7f6c90003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:22.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.346+0000 7f6ca2d9d640 1 --2- 172.21.15.154:0/1878164938 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c900039a0 0x7f6c90003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:22.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.346+0000 7f6ca2d9d640 1 -- 172.21.15.154:0/1878164938 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c90003520 con 0x7f6c90005140 2023-09-23T03:28:22.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.347+0000 7f6ca2d9d640 1 --2- 172.21.15.154:0/1878164938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c90005140 0x7f6c9000b980 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c001e20 tx=0x7f6c900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a6ed4d4fb1a2e0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:22.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.347+0000 7f6ca159a640 1 -- 172.21.15.154:0/1878164938 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c003b70 con 0x7f6c90005140 2023-09-23T03:28:22.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.347+0000 7f6ca159a640 1 -- 172.21.15.154:0/1878164938 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c9c003cd0 con 0x7f6c90005140 2023-09-23T03:28:22.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.347+0000 7f6ca159a640 1 -- 172.21.15.154:0/1878164938 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c005310 con 0x7f6c90005140 2023-09-23T03:28:22.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.347+0000 7f6ca359e640 1 -- 172.21.15.154:0/1878164938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c90005140 msgr2=0x7f6c9000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:22.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.347+0000 7f6ca359e640 1 --2- 172.21.15.154:0/1878164938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c90005140 0x7f6c9000b980 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c001e20 tx=0x7f6c900057f0 comp rx=0 tx=0).stop 2023-09-23T03:28:22.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.347+0000 7f6ca2d9d640 1 -- 172.21.15.154:0/1878164938 reap_dead start 2023-09-23T03:28:22.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.348+0000 7f6ca359e640 1 -- 172.21.15.154:0/1878164938 shutdown_connections 2023-09-23T03:28:22.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.348+0000 7f6ca359e640 1 -- 172.21.15.154:0/1878164938 >> 172.21.15.154:0/1878164938 conn(0x7f6ca40706f0 msgr2=0x7f6ca4070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:22.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.348+0000 7f6ca359e640 1 -- 172.21.15.154:0/1878164938 shutdown_connections 2023-09-23T03:28:22.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.348+0000 7f6ca359e640 1 -- 172.21.15.154:0/1878164938 wait complete. 2023-09-23T03:28:22.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.350+0000 7f6ca359e640 1 Processor -- start 2023-09-23T03:28:22.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.350+0000 7f6ca359e640 1 -- start start 2023-09-23T03:28:22.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.351+0000 7f6ca359e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c90005140 0x7f6ca40c56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:22.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.351+0000 7f6ca359e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ca4156450 0x7f6ca4145750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:22.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.351+0000 7f6ca359e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ca41593b0 0x7f6ca4159840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:22.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.351+0000 7f6ca359e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca4159d80 con 0x7f6c90005140 2023-09-23T03:28:22.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.351+0000 7f6ca359e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca4145c90 con 0x7f6ca4156450 2023-09-23T03:28:22.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.351+0000 7f6ca359e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca41df8f0 con 0x7f6ca41593b0 2023-09-23T03:28:22.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.351+0000 7f6ca259c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c90005140 0x7f6ca40c56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:22.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.351+0000 7f6ca259c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c90005140 0x7f6ca40c56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:59754/0 (socket says 172.21.15.154:59754) 2023-09-23T03:28:22.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.351+0000 7f6ca259c640 1 -- 172.21.15.154:0/1672513473 learned_addr learned my addr 172.21.15.154:0/1672513473 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:22.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.351+0000 7f6ca259c640 1 -- 172.21.15.154:0/1672513473 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ca41593b0 msgr2=0x7f6ca4159840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:22.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.351+0000 7f6ca259c640 1 --2- 172.21.15.154:0/1672513473 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ca41593b0 0x7f6ca4159840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:22.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.351+0000 7f6ca259c640 1 -- 172.21.15.154:0/1672513473 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ca4156450 msgr2=0x7f6ca4145750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:22.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.351+0000 7f6ca1d9b640 1 --2- 172.21.15.154:0/1672513473 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ca4156450 0x7f6ca4145750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:22.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.351+0000 7f6ca259c640 1 --2- 172.21.15.154:0/1672513473 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ca4156450 0x7f6ca4145750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:22.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.351+0000 7f6ca259c640 1 -- 172.21.15.154:0/1672513473 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca41dfb90 con 0x7f6c90005140 2023-09-23T03:28:22.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.352+0000 7f6ca259c640 1 --2- 172.21.15.154:0/1672513473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c90005140 0x7f6ca40c56a0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400e780 tx=0x7f6c9400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:22.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.352+0000 7f6c9affd640 1 -- 172.21.15.154:0/1672513473 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9400ee90 con 0x7f6c90005140 2023-09-23T03:28:22.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.352+0000 7f6c9affd640 1 -- 172.21.15.154:0/1672513473 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c9400c870 con 0x7f6c90005140 2023-09-23T03:28:22.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.352+0000 7f6c9affd640 1 -- 172.21.15.154:0/1672513473 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9400cc20 con 0x7f6c90005140 2023-09-23T03:28:22.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.352+0000 7f6ca359e640 1 -- 172.21.15.154:0/1672513473 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca40c7f50 con 0x7f6c90005140 2023-09-23T03:28:22.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.352+0000 7f6ca359e640 1 -- 172.21.15.154:0/1672513473 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca41dfe70 con 0x7f6c90005140 2023-09-23T03:28:22.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.353+0000 7f6ca359e640 1 -- 172.21.15.154:0/1672513473 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ca406cde0 con 0x7f6c90005140 2023-09-23T03:28:22.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.354+0000 7f6c9affd640 1 -- 172.21.15.154:0/1672513473 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f6c9401d460 con 0x7f6c90005140 2023-09-23T03:28:22.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.355+0000 7f6c9affd640 1 --2- 172.21.15.154:0/1672513473 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6c8406af30 0x7f6c8406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:22.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.355+0000 7f6c9affd640 1 -- 172.21.15.154:0/1672513473 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6115+0+0 (secure 0 0 0) 0x7f6c94089950 con 0x7f6c90005140 2023-09-23T03:28:22.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.356+0000 7f6ca1d9b640 1 --2- 172.21.15.154:0/1672513473 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6c8406af30 0x7f6c8406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:22.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.356+0000 7f6ca1d9b640 1 --2- 172.21.15.154:0/1672513473 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6c8406af30 0x7f6c8406d3f0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6ca40c7b90 tx=0x7f6c8c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:22.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.360+0000 7f6c9affd640 1 -- 172.21.15.154:0/1672513473 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f6c940581e0 con 0x7f6c90005140 2023-09-23T03:28:22.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.561+0000 7f6ca359e640 1 -- 172.21.15.154:0/1672513473 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6ca404e920 con 0x7f6c90005140 2023-09-23T03:28:22.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.562+0000 7f6c9affd640 1 -- 172.21.15.154:0/1672513473 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v127) v1 ==== 80+0+124 (secure 0 0 0) 0x7f6c94001560 con 0x7f6c90005140 2023-09-23T03:28:22.563 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.190:6835/3558096060 2023-09-24T03:28:08.468628+0000 2023-09-23T03:28:22.563 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.190:6833/3558096060 2023-09-24T03:28:08.468628+0000 2023-09-23T03:28:22.563 INFO:teuthology.orchestra.run.smithi154.stderr:listed 2 entries 2023-09-23T03:28:22.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.565+0000 7f6ca359e640 1 -- 172.21.15.154:0/1672513473 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6c8406af30 msgr2=0x7f6c8406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:22.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.565+0000 7f6ca359e640 1 --2- 172.21.15.154:0/1672513473 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6c8406af30 0x7f6c8406d3f0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6ca40c7b90 tx=0x7f6c8c008040 comp rx=0 tx=0).stop 2023-09-23T03:28:22.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.565+0000 7f6ca359e640 1 -- 172.21.15.154:0/1672513473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c90005140 msgr2=0x7f6ca40c56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:22.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.565+0000 7f6ca359e640 1 --2- 172.21.15.154:0/1672513473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c90005140 0x7f6ca40c56a0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400e780 tx=0x7f6c9400ec40 comp rx=0 tx=0).stop 2023-09-23T03:28:22.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.565+0000 7f6ca359e640 1 -- 172.21.15.154:0/1672513473 shutdown_connections 2023-09-23T03:28:22.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.565+0000 7f6ca359e640 1 --2- 172.21.15.154:0/1672513473 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6c8406af30 0x7f6c8406d3f0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:22.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.565+0000 7f6ca359e640 1 --2- 172.21.15.154:0/1672513473 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6ca41593b0 0x7f6ca4159840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:22.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.565+0000 7f6ca359e640 1 --2- 172.21.15.154:0/1672513473 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6ca4156450 0x7f6ca4145750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:22.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.565+0000 7f6ca359e640 1 --2- 172.21.15.154:0/1672513473 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c90005140 0x7f6ca40c56a0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:22.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.565+0000 7f6ca359e640 1 -- 172.21.15.154:0/1672513473 >> 172.21.15.154:0/1672513473 conn(0x7f6ca40706f0 msgr2=0x7f6ca4158100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:22.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.565+0000 7f6ca359e640 1 -- 172.21.15.154:0/1672513473 shutdown_connections 2023-09-23T03:28:22.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:22.565+0000 7f6ca359e640 1 -- 172.21.15.154:0/1672513473 wait complete. 2023-09-23T03:28:22.584 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi154.front.sepia.ceph.com... 2023-09-23T03:28:22.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:22.584 DEBUG:teuthology.orchestra.run.smithi154:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-09-23T03:28:22.675 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:ceph-fuse[128588]: fuse finished with error 0 and tester_r 0 2023-09-23T03:28:22.678 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:28:28.780 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2023-09-23T03:28:28.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:28.780 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-09-23T03:28:28.840 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-09-23T03:28:29.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.054+0000 7feef5ab4640 1 Processor -- start 2023-09-23T03:28:29.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.055+0000 7feef5ab4640 1 -- start start 2023-09-23T03:28:29.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.056+0000 7feef5ab4640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7feef00ca3c0 0x7feef0070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:29.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.056+0000 7feef5ab4640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feef00c9a40 con 0x7feef0159ba0 2023-09-23T03:28:29.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.056+0000 7feef5ab4640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feef015a5f0 con 0x7feef00cb320 2023-09-23T03:28:29.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.056+0000 7feef5ab4640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feef015abb0 con 0x7feef00ca3c0 2023-09-23T03:28:29.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.056+0000 7feef52b3640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7feef00ca3c0 0x7feef0070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:29.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.056+0000 7feeeffff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7feef0159ba0 0x7feef00c9cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:49856/0 (socket says 172.21.15.154:49856) 2023-09-23T03:28:29.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.056+0000 7feeeffff640 1 -- 172.21.15.154:0/1556263831 learned_addr learned my addr 172.21.15.154:0/1556263831 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:28:29.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.057+0000 7feef52b3640 1 -- 172.21.15.154:0/1556263831 >> v1:172.21.15.190:6789/0 conn(0x7feef00cb320 legacy=0x7feef00c9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:29.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.057+0000 7feef52b3640 1 -- 172.21.15.154:0/1556263831 >> v1:172.21.15.154:6789/0 conn(0x7feef0159ba0 legacy=0x7feef00c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:29.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.057+0000 7feef52b3640 1 -- 172.21.15.154:0/1556263831 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef015a8d0 con 0x7feef00ca3c0 2023-09-23T03:28:29.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.057+0000 7feeef7fe640 1 -- 172.21.15.154:0/1556263831 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4091125180 0 0) 0x7feef00c9a40 con 0x7feef0159ba0 2023-09-23T03:28:29.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.058+0000 7feef52b3640 1 --2- 172.21.15.154:0/1556263831 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feef00ca3c0 0x7feef0070ee0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7feee80097a0 tx=0x7feee802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78cfe171cc76451f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:29.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.059+0000 7feeef7fe640 1 -- 172.21.15.154:0/1556263831 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee8031830 con 0x7feef00ca3c0 2023-09-23T03:28:29.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.059+0000 7feeef7fe640 1 -- 172.21.15.154:0/1556263831 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee8031990 con 0x7feef00ca3c0 2023-09-23T03:28:29.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.059+0000 7feeef7fe640 1 -- 172.21.15.154:0/1556263831 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee8031d20 con 0x7feef00ca3c0 2023-09-23T03:28:29.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.059+0000 7feef5ab4640 1 -- 172.21.15.154:0/1556263831 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feef00ca3c0 msgr2=0x7feef0070ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:29.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.059+0000 7feef5ab4640 1 --2- 172.21.15.154:0/1556263831 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feef00ca3c0 0x7feef0070ee0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7feee80097a0 tx=0x7feee802b8d0 comp rx=0 tx=0).stop 2023-09-23T03:28:29.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.059+0000 7feef5ab4640 1 -- 172.21.15.154:0/1556263831 shutdown_connections 2023-09-23T03:28:29.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.059+0000 7feef5ab4640 1 --2- 172.21.15.154:0/1556263831 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feef00ca3c0 0x7feef0070ee0 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:29.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.059+0000 7feef5ab4640 1 -- 172.21.15.154:0/1556263831 >> 172.21.15.154:0/1556263831 conn(0x7feef006a4c0 msgr2=0x7feef00afe50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:29.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.059+0000 7feef5ab4640 1 -- 172.21.15.154:0/1556263831 shutdown_connections 2023-09-23T03:28:29.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.059+0000 7feef5ab4640 1 -- 172.21.15.154:0/1556263831 wait complete. 2023-09-23T03:28:29.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.061+0000 7feef5ab4640 1 Processor -- start 2023-09-23T03:28:29.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.061+0000 7feef5ab4640 1 -- start start 2023-09-23T03:28:29.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.062+0000 7feef5ab4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feef00ca3c0 0x7feef00b35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:29.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.062+0000 7feef5ab4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feef00cb320 0x7feef014c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:29.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.062+0000 7feef5ab4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feef0159ba0 0x7feef00b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:29.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.062+0000 7feef5ab4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef00b3c80 con 0x7feef00cb320 2023-09-23T03:28:29.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.062+0000 7feef5ab4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef00c4800 con 0x7feef00ca3c0 2023-09-23T03:28:29.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.062+0000 7feef5ab4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feef014c940 con 0x7feef0159ba0 2023-09-23T03:28:29.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.062+0000 7feeeffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feef00cb320 0x7feef014c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:29.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.062+0000 7feeeffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feef00cb320 0x7feef014c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53724/0 (socket says 172.21.15.154:53724) 2023-09-23T03:28:29.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.062+0000 7feeeffff640 1 -- 172.21.15.154:0/282551287 learned_addr learned my addr 172.21.15.154:0/282551287 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:29.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.063+0000 7feef4ab2640 1 --2- 172.21.15.154:0/282551287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feef00ca3c0 0x7feef00b35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:29.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.063+0000 7feeeffff640 1 -- 172.21.15.154:0/282551287 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feef0159ba0 msgr2=0x7feef00b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:29.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.063+0000 7feeeffff640 1 --2- 172.21.15.154:0/282551287 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feef0159ba0 0x7feef00b0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:29.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.063+0000 7feeeffff640 1 -- 172.21.15.154:0/282551287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feef00ca3c0 msgr2=0x7feef00b35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:29.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.063+0000 7feeeffff640 1 --2- 172.21.15.154:0/282551287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feef00ca3c0 0x7feef00b35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:29.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.063+0000 7feeeffff640 1 -- 172.21.15.154:0/282551287 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee8009400 con 0x7feef00cb320 2023-09-23T03:28:29.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.063+0000 7feeeffff640 1 --2- 172.21.15.154:0/282551287 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feef00cb320 0x7feef014c400 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7feed800c3f0 tx=0x7feed800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:29.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.063+0000 7feeed7fa640 1 -- 172.21.15.154:0/282551287 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed80074e0 con 0x7feef00cb320 2023-09-23T03:28:29.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.063+0000 7feeed7fa640 1 -- 172.21.15.154:0/282551287 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed8007640 con 0x7feef00cb320 2023-09-23T03:28:29.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.063+0000 7feef5ab4640 1 -- 172.21.15.154:0/282551287 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feef00c6ea0 con 0x7feef00cb320 2023-09-23T03:28:29.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.064+0000 7feeed7fa640 1 -- 172.21.15.154:0/282551287 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed8004a00 con 0x7feef00cb320 2023-09-23T03:28:29.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.064+0000 7feef5ab4640 1 -- 172.21.15.154:0/282551287 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feef00c7460 con 0x7feef00cb320 2023-09-23T03:28:29.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.065+0000 7feef5ab4640 1 -- 172.21.15.154:0/282551287 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feef0066730 con 0x7feef00cb320 2023-09-23T03:28:29.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.066+0000 7feeed7fa640 1 -- 172.21.15.154:0/282551287 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7feed8004b60 con 0x7feef00cb320 2023-09-23T03:28:29.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.067+0000 7feeed7fa640 1 --2- 172.21.15.154:0/282551287 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7feed006af30 0x7feed006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:29.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.067+0000 7feeed7fa640 1 -- 172.21.15.154:0/282551287 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6115+0+0 (secure 0 0 0) 0x7feed8089810 con 0x7feef00cb320 2023-09-23T03:28:29.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.067+0000 7feef4ab2640 1 --2- 172.21.15.154:0/282551287 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7feed006af30 0x7feed006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:29.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.068+0000 7feef4ab2640 1 --2- 172.21.15.154:0/282551287 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7feed006af30 0x7feed006d3f0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7feee0007fd0 tx=0x7feee0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:29.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.072+0000 7feeed7fa640 1 -- 172.21.15.154:0/282551287 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7feed8054610 con 0x7feef00cb320 2023-09-23T03:28:29.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.294+0000 7feef5ab4640 1 -- 172.21.15.154:0/282551287 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7feef00c8de0 con 0x7feef00cb320 2023-09-23T03:28:29.295 INFO:tasks.ceph.mon.a.smithi154.stderr:2023-09-23T03:28:29.294+0000 7f3df35a6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-23T03:28:29.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.297+0000 7feeed7fa640 1 -- 172.21.15.154:0/282551287 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v115) v1 ==== 161+0+0 (secure 0 0 0) 0x7feed8057c30 con 0x7feef00cb320 2023-09-23T03:28:29.298 INFO:teuthology.orchestra.run.smithi154.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-09-23T03:28:29.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.300+0000 7feef5ab4640 1 -- 172.21.15.154:0/282551287 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7feed006af30 msgr2=0x7feed006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:29.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.300+0000 7feef5ab4640 1 --2- 172.21.15.154:0/282551287 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7feed006af30 0x7feed006d3f0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7feee0007fd0 tx=0x7feee0009040 comp rx=0 tx=0).stop 2023-09-23T03:28:29.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.300+0000 7feef5ab4640 1 -- 172.21.15.154:0/282551287 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feef00cb320 msgr2=0x7feef014c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:29.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.300+0000 7feef5ab4640 1 --2- 172.21.15.154:0/282551287 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feef00cb320 0x7feef014c400 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7feed800c3f0 tx=0x7feed800c8b0 comp rx=0 tx=0).stop 2023-09-23T03:28:29.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.301+0000 7feef5ab4640 1 -- 172.21.15.154:0/282551287 shutdown_connections 2023-09-23T03:28:29.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.301+0000 7feef5ab4640 1 --2- 172.21.15.154:0/282551287 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7feed006af30 0x7feed006d3f0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:29.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.301+0000 7feef5ab4640 1 --2- 172.21.15.154:0/282551287 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7feef0159ba0 0x7feef00b0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:29.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.301+0000 7feef5ab4640 1 --2- 172.21.15.154:0/282551287 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feef00cb320 0x7feef014c400 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:29.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.301+0000 7feef5ab4640 1 --2- 172.21.15.154:0/282551287 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7feef00ca3c0 0x7feef00b35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:29.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.301+0000 7feef5ab4640 1 -- 172.21.15.154:0/282551287 >> 172.21.15.154:0/282551287 conn(0x7feef006a4c0 msgr2=0x7feef00c9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:29.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.301+0000 7feef5ab4640 1 -- 172.21.15.154:0/282551287 shutdown_connections 2023-09-23T03:28:29.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.301+0000 7feef5ab4640 1 -- 172.21.15.154:0/282551287 wait complete. 2023-09-23T03:28:29.310 INFO:tasks.ceph.mds.c.smithi154.stderr: -3240> 2023-09-23T03:28:15.974+0000 7f391cc56640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-23T03:28:29.319 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents 2023-09-23T03:28:29.371 INFO:tasks.ceph.mds.c.smithi154.stdout:starting mds.c at 2023-09-23T03:28:29.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.407+0000 7f18314125c0 1 -- start start 2023-09-23T03:28:29.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.407+0000 7f18314125c0 1 Processor -- start 2023-09-23T03:28:29.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.408+0000 7f18314125c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d53b816c00 0x55d53b881b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:29.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.408+0000 7f18314125c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d53b817800 0x55d53b881600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:29.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.408+0000 7f18314125c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d53b817c00 0x55d53b881080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:29.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.408+0000 7f18314125c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d53b8e5180 con 0x55d53b817c00 2023-09-23T03:28:29.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.408+0000 7f18314125c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d53b8e4dc0 con 0x55d53b816c00 2023-09-23T03:28:29.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.408+0000 7f18314125c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d53b860a00 con 0x55d53b817800 2023-09-23T03:28:29.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.408+0000 7f182f8e1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d53b817c00 0x55d53b881080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:29.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.408+0000 7f182f8e1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d53b817c00 0x55d53b881080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53758/0 (socket says 172.21.15.154:53758) 2023-09-23T03:28:29.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.408+0000 7f182f8e1640 1 -- 172.21.15.154:0/2439774686 learned_addr learned my addr 172.21.15.154:0/2439774686 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:29.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.408+0000 7f182e8df640 1 --2- 172.21.15.154:0/2439774686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d53b817800 0x55d53b881600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:29.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.408+0000 7f182f0e0640 1 --2- 172.21.15.154:0/2439774686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d53b816c00 0x55d53b881b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:29.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.408+0000 7f182f8e1640 1 -- 172.21.15.154:0/2439774686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d53b817800 msgr2=0x55d53b881600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:29.410 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.408+0000 7f182f8e1640 1 --2- 172.21.15.154:0/2439774686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d53b817800 0x55d53b881600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:29.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.408+0000 7f182f8e1640 1 -- 172.21.15.154:0/2439774686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d53b816c00 msgr2=0x55d53b881b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:29.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.408+0000 7f182f8e1640 1 --2- 172.21.15.154:0/2439774686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d53b816c00 0x55d53b881b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:29.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.408+0000 7f182f8e1640 1 -- 172.21.15.154:0/2439774686 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d53b84ab60 con 0x55d53b817c00 2023-09-23T03:28:29.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.408+0000 7f182e8df640 1 --2- 172.21.15.154:0/2439774686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d53b817800 0x55d53b881600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:29.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.409+0000 7f182f8e1640 1 --2- 172.21.15.154:0/2439774686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d53b817c00 0x55d53b881080 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x55d53c61b3e0 tx=0x55d53bb00480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f331c771c576c88c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:29.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.409+0000 7f182c8db640 1 -- 172.21.15.154:0/2439774686 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d53b85f600 con 0x55d53b817c00 2023-09-23T03:28:29.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.409+0000 7f182c8db640 1 -- 172.21.15.154:0/2439774686 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d53bb02000 con 0x55d53b817c00 2023-09-23T03:28:29.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.409+0000 7f182c8db640 1 -- 172.21.15.154:0/2439774686 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d53b85f1e0 con 0x55d53b817c00 2023-09-23T03:28:29.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.409+0000 7f18314125c0 1 -- 172.21.15.154:0/2439774686 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d53b84a340 con 0x55d53b817c00 2023-09-23T03:28:29.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.409+0000 7f18314125c0 1 -- 172.21.15.154:0/2439774686 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55d53b84a000 con 0x55d53b817c00 2023-09-23T03:28:29.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.409+0000 7f18314125c0 4 waiting for MDS map... 2023-09-23T03:28:29.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.409+0000 7f182c8db640 1 -- 172.21.15.154:0/2439774686 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(128..128 src has 1..128) v4 ==== 6201+0+0 (secure 0 0 0) 0x55d53b8d6c40 con 0x55d53b817c00 2023-09-23T03:28:29.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.409+0000 7f182c8db640 1 -- 172.21.15.154:0/2439774686 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 115) v1 ==== 2723+0+0 (secure 0 0 0) 0x55d53b868f00 con 0x55d53b817c00 2023-09-23T03:28:29.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.409+0000 7f18314125c0 4 Got MDS map 115 2023-09-23T03:28:29.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.409+0000 7f18314125c0 4 datascan.main: Using metadata pool output 2023-09-23T03:28:29.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.409+0000 7f18314125c0 4 datascan.main: connecting to RADOS... 2023-09-23T03:28:29.415 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.410+0000 7f18314125c0 1 Processor -- start 2023-09-23T03:28:29.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.410+0000 7f18314125c0 1 -- start start 2023-09-23T03:28:29.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.411+0000 7f18314125c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d53bb0f000 0x55d53b880b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:29.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.411+0000 7f18314125c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d53bb0f400 0x55d53b880580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:29.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.411+0000 7f18314125c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d53bb0f800 0x55d53b882100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:29.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.411+0000 7f18314125c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d53b860b40 con 0x55d53bb0f400 2023-09-23T03:28:29.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.411+0000 7f18314125c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d53b8e4dc0 con 0x55d53bb0f800 2023-09-23T03:28:29.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.411+0000 7f18314125c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d53b8e4c80 con 0x55d53bb0f000 2023-09-23T03:28:29.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.411+0000 7f182f0e0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d53bb0f400 0x55d53b880580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:29.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.411+0000 7f182f0e0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d53bb0f400 0x55d53b880580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53764/0 (socket says 172.21.15.154:53764) 2023-09-23T03:28:29.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.411+0000 7f182f0e0640 1 -- 172.21.15.154:0/257357700 learned_addr learned my addr 172.21.15.154:0/257357700 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:29.416 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.412+0000 7f182f0e0640 1 -- 172.21.15.154:0/257357700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d53bb0f000 msgr2=0x55d53b880b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:29.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.412+0000 7f182f0e0640 1 --2- 172.21.15.154:0/257357700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d53bb0f000 0x55d53b880b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:29.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.412+0000 7f182f0e0640 1 -- 172.21.15.154:0/257357700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d53bb0f800 msgr2=0x55d53b882100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:29.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.412+0000 7f182f0e0640 1 --2- 172.21.15.154:0/257357700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d53bb0f800 0x55d53b882100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:29.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.412+0000 7f182f0e0640 1 -- 172.21.15.154:0/257357700 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d53b84aea0 con 0x55d53bb0f400 2023-09-23T03:28:29.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.412+0000 7f182f0e0640 1 --2- 172.21.15.154:0/257357700 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d53bb0f400 0x55d53b880580 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x55d53bb225a0 tx=0x55d53bb002a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1154cba566f045e server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:29.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.412+0000 7f182a8d7640 1 -- 172.21.15.154:0/257357700 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d53b85f080 con 0x55d53bb0f400 2023-09-23T03:28:29.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.412+0000 7f182a8d7640 1 -- 172.21.15.154:0/257357700 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d53bb02000 con 0x55d53bb0f400 2023-09-23T03:28:29.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.412+0000 7f182a8d7640 1 -- 172.21.15.154:0/257357700 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d53b85f600 con 0x55d53bb0f400 2023-09-23T03:28:29.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.412+0000 7f18314125c0 1 -- 172.21.15.154:0/257357700 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d53bb0f400 msgr2=0x55d53b880580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:29.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.412+0000 7f18314125c0 1 --2- 172.21.15.154:0/257357700 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d53bb0f400 0x55d53b880580 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x55d53bb225a0 tx=0x55d53bb002a0 comp rx=0 tx=0).stop 2023-09-23T03:28:29.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.413+0000 7f18314125c0 1 -- 172.21.15.154:0/257357700 shutdown_connections 2023-09-23T03:28:29.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.413+0000 7f18314125c0 1 --2- 172.21.15.154:0/257357700 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d53bb0f800 0x55d53b882100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:29.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.413+0000 7f18314125c0 1 --2- 172.21.15.154:0/257357700 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d53bb0f400 0x55d53b880580 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:29.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.413+0000 7f18314125c0 1 --2- 172.21.15.154:0/257357700 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d53bb0f000 0x55d53b880b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:29.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.413+0000 7f18314125c0 1 -- 172.21.15.154:0/257357700 >> 172.21.15.154:0/257357700 conn(0x55d53bb0ec00 msgr2=0x55d53c591000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:29.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.413+0000 7f18314125c0 1 -- 172.21.15.154:0/257357700 shutdown_connections 2023-09-23T03:28:29.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.413+0000 7f18314125c0 1 -- 172.21.15.154:0/257357700 wait complete. 2023-09-23T03:28:29.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.413+0000 7f18314125c0 1 Processor -- start 2023-09-23T03:28:29.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.413+0000 7f18314125c0 1 -- start start 2023-09-23T03:28:29.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.414+0000 7f18314125c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d53bb0f000 0x55d53b880b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:29.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.414+0000 7f18314125c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d53bb0f400 0x55d53b880580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:29.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.414+0000 7f18314125c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d53bb0f800 0x55d53b882100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:29.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.414+0000 7f18314125c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d53b8e4b40 con 0x55d53bb0f000 2023-09-23T03:28:29.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.414+0000 7f18314125c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d53b8e4a00 con 0x55d53bb0f800 2023-09-23T03:28:29.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.414+0000 7f18314125c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d53b8e48c0 con 0x55d53bb0f400 2023-09-23T03:28:29.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.414+0000 7f182e8df640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d53bb0f000 0x55d53b880b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:29.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.414+0000 7f182e8df640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d53bb0f000 0x55d53b880b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53780/0 (socket says 172.21.15.154:53780) 2023-09-23T03:28:29.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.414+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 learned_addr learned my addr 172.21.15.154:0/10744971 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:29.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.415+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d53bb0f400 msgr2=0x55d53b880580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:29.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.415+0000 7f182e8df640 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d53bb0f400 0x55d53b880580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:29.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.415+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d53bb0f800 msgr2=0x55d53b882100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:29.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.415+0000 7f182e8df640 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d53bb0f800 0x55d53b882100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:29.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.415+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d53b84b1e0 con 0x55d53bb0f000 2023-09-23T03:28:29.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.415+0000 7f182e8df640 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d53bb0f000 0x55d53b880b00 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x55d53bb23410 tx=0x55d53bb30cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:29.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.415+0000 7f18290d4640 1 -- 172.21.15.154:0/10744971 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d53b85f080 con 0x55d53bb0f000 2023-09-23T03:28:29.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.415+0000 7f18290d4640 1 -- 172.21.15.154:0/10744971 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d53b83d800 con 0x55d53bb0f000 2023-09-23T03:28:29.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.415+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d53b84aea0 con 0x55d53bb0f000 2023-09-23T03:28:29.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.415+0000 7f18290d4640 1 -- 172.21.15.154:0/10744971 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d53b85f600 con 0x55d53bb0f000 2023-09-23T03:28:29.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.415+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d53b84ab60 con 0x55d53bb0f000 2023-09-23T03:28:29.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.415+0000 7f18314125c0 4 datascan.init: resolving metadata pool 17 2023-09-23T03:28:29.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.417+0000 7f18290d4640 1 -- 172.21.15.154:0/10744971 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x55d53b848840 con 0x55d53bb0f000 2023-09-23T03:28:29.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.418+0000 7f18290d4640 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55d53bb9a800 0x55d53bc88000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:29.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.418+0000 7f18290d4640 1 -- 172.21.15.154:0/10744971 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6201+0+0 (secure 0 0 0) 0x55d53b8d6a80 con 0x55d53bb0f000 2023-09-23T03:28:29.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.418+0000 7f18314125c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-09-23T03:28:29.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.418+0000 7f18314125c0 4 datascan.main: data pool 'cephfs_data' has ID 18 2023-09-23T03:28:29.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.418+0000 7f18314125c0 4 datascan.main: opening data pool 'cephfs_data' 2023-09-23T03:28:29.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.418+0000 7f18314125c0 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55d53bb9ac00 0x55d53b882680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:29.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.418+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:1 18.0 18:00000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53b871880 con 0x55d53bb9ac00 2023-09-23T03:28:29.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.418+0000 7f182f0e0640 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55d53bb9a800 0x55d53bc88000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:29.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.418+0000 7f182e8df640 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55d53bb9ac00 0x55d53b882680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:29.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.418+0000 7f182f0e0640 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55d53bb9a800 0x55d53bc88000 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x55d53bb225a0 tx=0x55d53bb00180 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:29.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.419+0000 7f182e8df640 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55d53bb9ac00 0x55d53b882680 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:29.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.421+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.0 v2:172.21.15.154:6800/2920938138 1 ==== osd_op_reply(1 [pgnls start_epoch 128 out=81b] v126'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9ac00 2023-09-23T03:28:29.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.421+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:2 18.0 18:012cdbc7:::10000000206.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53b871c00 con 0x55d53bb9ac00 2023-09-23T03:28:29.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.421+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.0 v2:172.21.15.154:6800/2920938138 2 ==== osd_op_reply(2 10000000206.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9ac00 2023-09-23T03:28:29.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.421+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628294.0 2023-09-23T03:28:29.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.421+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:3 18.0 18:012cdbc7:::10000000206.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53b871880 con 0x55d53bb9ac00 2023-09-23T03:28:29.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.425+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.0 v2:172.21.15.154:6800/2920938138 3 ==== osd_op_reply(3 10000000206.00000000 [call] v128'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9ac00 2023-09-23T03:28:29.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.425+0000 7f18314125c0 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55d53bb9b000 0x55d53b882c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:29.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.425+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:4 18.20 18:04000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53b871c00 con 0x55d53bb9b000 2023-09-23T03:28:29.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.425+0000 7f182f8e1640 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55d53bb9b000 0x55d53b882c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:29.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.427+0000 7f182f8e1640 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55d53bb9b000 0x55d53b882c00 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:29.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.433+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 1 ==== osd_op_reply(4 [pgnls start_epoch 128 out=81b] v126'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.433+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:5 18.20 18:05552577:::1000000020b.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0000 con 0x55d53bb9b000 2023-09-23T03:28:29.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.433+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 2 ==== osd_op_reply(5 1000000020b.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.433+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628299.0 2023-09-23T03:28:29.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.433+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:6 18.20 18:05552577:::1000000020b.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0380 con 0x55d53bb9b000 2023-09-23T03:28:29.437 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.437+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 3 ==== osd_op_reply(6 1000000020b.00000000 [call] v128'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.437+0000 7f18314125c0 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55d53bb9b400 0x55d53b883180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:29.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.437+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:7 18.10 18:08000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0700 con 0x55d53bb9b400 2023-09-23T03:28:29.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.438+0000 7f182f0e0640 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55d53bb9b400 0x55d53b883180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:29.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.439+0000 7f182f0e0640 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55d53bb9b400 0x55d53b883180 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:29.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.440+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== osd_op_reply(7 [pgnls start_epoch 128 out=101b] v126'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9b400 2023-09-23T03:28:29.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.440+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:8 18.10 18:083d5826:::1000000022a.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0a80 con 0x55d53bb9b400 2023-09-23T03:28:29.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.441+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.6 v2:172.21.15.190:6816/1964294001 2 ==== osd_op_reply(8 1000000022a.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9b400 2023-09-23T03:28:29.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.441+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628330.0 2023-09-23T03:28:29.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.441+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:9 18.10 18:083d5826:::1000000022a.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0e00 con 0x55d53bb9b400 2023-09-23T03:28:29.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.443+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.6 v2:172.21.15.190:6816/1964294001 3 ==== osd_op_reply(9 1000000022a.00000000 [call] v128'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9b400 2023-09-23T03:28:29.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.443+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:10 18.10 18:08500202::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0700 con 0x55d53bb9b400 2023-09-23T03:28:29.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.444+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.6 v2:172.21.15.190:6816/1964294001 4 ==== osd_op_reply(10 [pgnls start_epoch 128 out=101b] v128'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9b400 2023-09-23T03:28:29.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.444+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:11 18.10 18:08500202:::1000000023e.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0a80 con 0x55d53bb9b400 2023-09-23T03:28:29.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.445+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.6 v2:172.21.15.190:6816/1964294001 5 ==== osd_op_reply(11 1000000023e.00000000 [stat out=16b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9b400 2023-09-23T03:28:29.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.445+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628350.0 2023-09-23T03:28:29.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.445+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:12 18.10 18:08500202:::1000000023e.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0000 con 0x55d53bb9b400 2023-09-23T03:28:29.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.448+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.6 v2:172.21.15.190:6816/1964294001 6 ==== osd_op_reply(12 1000000023e.00000000 [call] v128'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9b400 2023-09-23T03:28:29.448 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.448+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:13 18.10 18:08b99fea::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0380 con 0x55d53bb9b400 2023-09-23T03:28:29.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.449+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.6 v2:172.21.15.190:6816/1964294001 7 ==== osd_op_reply(13 [pgnls start_epoch 128 out=101b] v128'16 uv16 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9b400 2023-09-23T03:28:29.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.449+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:14 18.10 18:08b99fea:::100000001fd.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1180 con 0x55d53bb9b400 2023-09-23T03:28:29.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.450+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.6 v2:172.21.15.190:6816/1964294001 8 ==== osd_op_reply(14 100000001fd.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9b400 2023-09-23T03:28:29.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.450+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628285.0 2023-09-23T03:28:29.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.450+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:15 18.10 18:08b99fea:::100000001fd.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53b871c00 con 0x55d53bb9b400 2023-09-23T03:28:29.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.453+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.6 v2:172.21.15.190:6816/1964294001 9 ==== osd_op_reply(15 100000001fd.00000000 [call] v128'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9b400 2023-09-23T03:28:29.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.453+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:16 18.10 18:08e6f8ca::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53b871c00 con 0x55d53bb9b400 2023-09-23T03:28:29.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.454+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.6 v2:172.21.15.190:6816/1964294001 10 ==== osd_op_reply(16 [pgnls start_epoch 128 out=81b] v128'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9b400 2023-09-23T03:28:29.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.454+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:17 18.10 18:08e6f8ca:::10000000257.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0000 con 0x55d53bb9b400 2023-09-23T03:28:29.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.455+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.6 v2:172.21.15.190:6816/1964294001 11 ==== osd_op_reply(17 10000000257.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9b400 2023-09-23T03:28:29.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.455+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628375.0 2023-09-23T03:28:29.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.455+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:18 18.10 18:08e6f8ca:::10000000257.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0380 con 0x55d53bb9b400 2023-09-23T03:28:29.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.459+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.6 v2:172.21.15.190:6816/1964294001 12 ==== osd_op_reply(18 10000000257.00000000 [call] v128'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9b400 2023-09-23T03:28:29.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.459+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:19 18.30 18:0c000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0700 con 0x55d53bb9ac00 2023-09-23T03:28:29.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.459+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.0 v2:172.21.15.154:6800/2920938138 4 ==== osd_op_reply(19 [pgnls start_epoch 128 out=101b] v126'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9ac00 2023-09-23T03:28:29.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.459+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:20 18.30 18:0c00cb85:::1000000021d.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0a80 con 0x55d53bb9ac00 2023-09-23T03:28:29.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.460+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.0 v2:172.21.15.154:6800/2920938138 5 ==== osd_op_reply(20 1000000021d.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9ac00 2023-09-23T03:28:29.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.460+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628317.0 2023-09-23T03:28:29.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.460+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:21 18.30 18:0c00cb85:::1000000021d.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0e00 con 0x55d53bb9ac00 2023-09-23T03:28:29.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.462+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.0 v2:172.21.15.154:6800/2920938138 6 ==== osd_op_reply(21 1000000021d.00000000 [call] v128'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9ac00 2023-09-23T03:28:29.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.462+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:22 18.30 18:0c28cce1::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1180 con 0x55d53bb9ac00 2023-09-23T03:28:29.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.462+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.0 v2:172.21.15.154:6800/2920938138 7 ==== osd_op_reply(22 [pgnls start_epoch 128 out=101b] v128'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9ac00 2023-09-23T03:28:29.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.462+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:23 18.30 18:0c28cce1:::10000000244.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1500 con 0x55d53bb9ac00 2023-09-23T03:28:29.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.463+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.0 v2:172.21.15.154:6800/2920938138 8 ==== osd_op_reply(23 10000000244.00000000 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9ac00 2023-09-23T03:28:29.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.463+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628356.0 2023-09-23T03:28:29.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.463+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:24 18.30 18:0c28cce1:::10000000244.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1880 con 0x55d53bb9ac00 2023-09-23T03:28:29.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.465+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.0 v2:172.21.15.154:6800/2920938138 9 ==== osd_op_reply(24 10000000244.00000000 [call] v128'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9ac00 2023-09-23T03:28:29.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.465+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:25 18.30 18:0cc9e906::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0e00 con 0x55d53bb9ac00 2023-09-23T03:28:29.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.466+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.0 v2:172.21.15.154:6800/2920938138 10 ==== osd_op_reply(25 [pgnls start_epoch 128 out=101b] v128'19 uv19 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9ac00 2023-09-23T03:28:29.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.466+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:26 18.30 18:0cc9e906:::10000000227.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1180 con 0x55d53bb9ac00 2023-09-23T03:28:29.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.466+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.0 v2:172.21.15.154:6800/2920938138 11 ==== osd_op_reply(26 10000000227.00000000 [stat out=16b] v0'0 uv15 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9ac00 2023-09-23T03:28:29.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.466+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628327.0 2023-09-23T03:28:29.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.466+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:27 18.30 18:0cc9e906:::10000000227.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1500 con 0x55d53bb9ac00 2023-09-23T03:28:29.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.468+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.0 v2:172.21.15.154:6800/2920938138 12 ==== osd_op_reply(27 10000000227.00000000 [call] v128'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9ac00 2023-09-23T03:28:29.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.468+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:28 18.30 18:0e3d5a27::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1880 con 0x55d53bb9ac00 2023-09-23T03:28:29.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.469+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.0 v2:172.21.15.154:6800/2920938138 13 ==== osd_op_reply(28 [pgnls start_epoch 128 out=81b] v128'20 uv20 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9ac00 2023-09-23T03:28:29.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.469+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:29 18.30 18:0e3d5a27:::10000000250.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0700 con 0x55d53bb9ac00 2023-09-23T03:28:29.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.469+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.0 v2:172.21.15.154:6800/2920938138 14 ==== osd_op_reply(29 10000000250.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9ac00 2023-09-23T03:28:29.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.469+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628368.0 2023-09-23T03:28:29.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.469+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:30 18.30 18:0e3d5a27:::10000000250.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0a80 con 0x55d53bb9ac00 2023-09-23T03:28:29.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.471+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.0 v2:172.21.15.154:6800/2920938138 15 ==== osd_op_reply(30 10000000250.00000000 [call] v128'21 uv21 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9ac00 2023-09-23T03:28:29.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.471+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:31 18.8 18:10000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1c00 con 0x55d53bb9ac00 2023-09-23T03:28:29.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.471+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.0 v2:172.21.15.154:6800/2920938138 16 ==== osd_op_reply(31 [pgnls start_epoch 128 out=49b] v126'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9ac00 2023-09-23T03:28:29.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.471+0000 7f182a0d6640 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:32 18.28 18:14000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0e00 con 0x55d53bb9b000 2023-09-23T03:28:29.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.472+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 4 ==== osd_op_reply(32 [pgnls start_epoch 128 out=101b] v126'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.473+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:33 18.28 18:14ce3b86:::10000000233.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53b871880 con 0x55d53bb9b000 2023-09-23T03:28:29.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.473+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 5 ==== osd_op_reply(33 10000000233.00000000 [stat out=16b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.474+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628339.0 2023-09-23T03:28:29.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.474+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:34 18.28 18:14ce3b86:::10000000233.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1180 con 0x55d53bb9b000 2023-09-23T03:28:29.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.477+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 6 ==== osd_op_reply(34 10000000233.00000000 [call] v128'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.477+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:35 18.28 18:15246602::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1500 con 0x55d53bb9b000 2023-09-23T03:28:29.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.478+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 7 ==== osd_op_reply(35 [pgnls start_epoch 128 out=101b] v128'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.478+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:36 18.28 18:15246602:::10000000202.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1880 con 0x55d53bb9b000 2023-09-23T03:28:29.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.479+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 8 ==== osd_op_reply(36 10000000202.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.479+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628290.0 2023-09-23T03:28:29.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.480+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:37 18.28 18:15246602:::10000000202.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0700 con 0x55d53bb9b000 2023-09-23T03:28:29.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.483+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 9 ==== osd_op_reply(37 10000000202.00000000 [call] v128'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.483+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:38 18.28 18:1537fa38::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0e00 con 0x55d53bb9b000 2023-09-23T03:28:29.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.484+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 10 ==== osd_op_reply(38 [pgnls start_epoch 128 out=101b] v128'16 uv16 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.484+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:39 18.28 18:1537fa38:::10000000236.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1180 con 0x55d53bb9b000 2023-09-23T03:28:29.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.485+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 11 ==== osd_op_reply(39 10000000236.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.485 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.485+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628342.0 2023-09-23T03:28:29.486 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.485+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:40 18.28 18:1537fa38:::10000000236.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1500 con 0x55d53bb9b000 2023-09-23T03:28:29.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.489+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 12 ==== osd_op_reply(40 10000000236.00000000 [call] v128'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.489+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:41 18.28 18:177a762d::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53b871880 con 0x55d53bb9b000 2023-09-23T03:28:29.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.490+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 13 ==== osd_op_reply(41 [pgnls start_epoch 128 out=81b] v128'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.490+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:42 18.28 18:177a762d:::10000000201.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1880 con 0x55d53bb9b000 2023-09-23T03:28:29.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.491+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 14 ==== osd_op_reply(42 10000000201.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.491+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628289.0 2023-09-23T03:28:29.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.491+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:43 18.28 18:177a762d:::10000000201.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0700 con 0x55d53bb9b000 2023-09-23T03:28:29.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.495+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 15 ==== osd_op_reply(43 10000000201.00000000 [call] v128'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.495+0000 7f18314125c0 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55d53bb9b800 0x55d53b883700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:29.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.495+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:44 18.18 18:18000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0e00 con 0x55d53bb9b800 2023-09-23T03:28:29.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.495+0000 7f182e8df640 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55d53bb9b800 0x55d53b883700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:29.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.496+0000 7f182e8df640 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55d53bb9b800 0x55d53b883700 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:29.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.497+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.1 v2:172.21.15.154:6816/1267710005 1 ==== osd_op_reply(44 [pgnls start_epoch 128 out=49b] v126'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9b800 2023-09-23T03:28:29.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.497+0000 7f182a8d7640 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55d53bcb6000 0x55d53bc88580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:29.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.497+0000 7f182a8d7640 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:45 18.38 18:1c000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbc000 con 0x55d53bcb6000 2023-09-23T03:28:29.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.498+0000 7f182f8e1640 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55d53bcb6000 0x55d53bc88580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:29.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.499+0000 7f182f8e1640 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55d53bcb6000 0x55d53bc88580 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:29.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.499+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(45 [pgnls start_epoch 128 out=49b] v126'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.499+0000 7f182a0d6640 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:46 18.4 18:20000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbc380 con 0x55d53bb9ac00 2023-09-23T03:28:29.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.500+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.0 v2:172.21.15.154:6800/2920938138 17 ==== osd_op_reply(46 [pgnls start_epoch 128 out=81b] v126'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9ac00 2023-09-23T03:28:29.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.500+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:47 18.4 18:2231e764:::10000000220.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1180 con 0x55d53bb9ac00 2023-09-23T03:28:29.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.500+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.0 v2:172.21.15.154:6800/2920938138 18 ==== osd_op_reply(47 10000000220.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9ac00 2023-09-23T03:28:29.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.500+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628320.0 2023-09-23T03:28:29.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.501+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:48 18.4 18:2231e764:::10000000220.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1500 con 0x55d53bb9ac00 2023-09-23T03:28:29.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.503+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.0 v2:172.21.15.154:6800/2920938138 19 ==== osd_op_reply(48 10000000220.00000000 [call] v128'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9ac00 2023-09-23T03:28:29.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.503+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:49 18.24 18:24000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbc380 con 0x55d53bb9ac00 2023-09-23T03:28:29.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.504+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.0 v2:172.21.15.154:6800/2920938138 20 ==== osd_op_reply(49 [pgnls start_epoch 128 out=101b] v126'8 uv8 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9ac00 2023-09-23T03:28:29.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.504+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:50 18.24 18:242ba651:::10000000234.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbca80 con 0x55d53bb9ac00 2023-09-23T03:28:29.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.505+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.0 v2:172.21.15.154:6800/2920938138 21 ==== osd_op_reply(50 10000000234.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9ac00 2023-09-23T03:28:29.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.505+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628340.0 2023-09-23T03:28:29.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.505+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:51 18.24 18:242ba651:::10000000234.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbce00 con 0x55d53bb9ac00 2023-09-23T03:28:29.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.508+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.0 v2:172.21.15.154:6800/2920938138 22 ==== osd_op_reply(51 10000000234.00000000 [call] v128'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9ac00 2023-09-23T03:28:29.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.508+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:52 18.24 18:25b5fce8::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbd180 con 0x55d53bb9ac00 2023-09-23T03:28:29.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.509+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.0 v2:172.21.15.154:6800/2920938138 23 ==== osd_op_reply(52 [pgnls start_epoch 128 out=81b] v128'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9ac00 2023-09-23T03:28:29.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.509+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:53 18.24 18:25b5fce8:::10000000231.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbd500 con 0x55d53bb9ac00 2023-09-23T03:28:29.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.510+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.0 v2:172.21.15.154:6800/2920938138 24 ==== osd_op_reply(53 10000000231.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9ac00 2023-09-23T03:28:29.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.510+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628337.0 2023-09-23T03:28:29.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.510+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:54 18.24 18:25b5fce8:::10000000231.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbd880 con 0x55d53bb9ac00 2023-09-23T03:28:29.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.513+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.0 v2:172.21.15.154:6800/2920938138 25 ==== osd_op_reply(54 10000000231.00000000 [call] v128'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9ac00 2023-09-23T03:28:29.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.513+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:55 18.14 18:28000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbdc00 con 0x55d53bb9b400 2023-09-23T03:28:29.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.514+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.6 v2:172.21.15.190:6816/1964294001 13 ==== osd_op_reply(55 [pgnls start_epoch 128 out=101b] v126'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9b400 2023-09-23T03:28:29.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.515+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:56 18.14 18:29971f97:::1000000020e.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0a80 con 0x55d53bb9b400 2023-09-23T03:28:29.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.516+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.6 v2:172.21.15.190:6816/1964294001 14 ==== osd_op_reply(56 1000000020e.00000000 [stat out=16b] v0'0 uv15 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9b400 2023-09-23T03:28:29.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.516+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628302.0 2023-09-23T03:28:29.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.516+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:57 18.14 18:29971f97:::1000000020e.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1c00 con 0x55d53bb9b400 2023-09-23T03:28:29.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.520+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.6 v2:172.21.15.190:6816/1964294001 15 ==== osd_op_reply(57 1000000020e.00000000 [call] v128'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9b400 2023-09-23T03:28:29.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.520+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:58 18.14 18:2ab28be7::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0e00 con 0x55d53bb9b400 2023-09-23T03:28:29.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.523+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.6 v2:172.21.15.190:6816/1964294001 16 ==== osd_op_reply(58 [pgnls start_epoch 128 out=101b] v128'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9b400 2023-09-23T03:28:29.524 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.523+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:59 18.14 18:2ab28be7:::10000000211.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1180 con 0x55d53bb9b400 2023-09-23T03:28:29.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.525+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.6 v2:172.21.15.190:6816/1964294001 17 ==== osd_op_reply(59 10000000211.00000000 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9b400 2023-09-23T03:28:29.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.525+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628305.0 2023-09-23T03:28:29.525 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.525+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:60 18.14 18:2ab28be7:::10000000211.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1500 con 0x55d53bb9b400 2023-09-23T03:28:29.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.528+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.6 v2:172.21.15.190:6816/1964294001 18 ==== osd_op_reply(60 10000000211.00000000 [call] v128'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9b400 2023-09-23T03:28:29.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.529+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:61 18.14 18:2b89f3ba::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0000 con 0x55d53bb9b400 2023-09-23T03:28:29.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.530+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.6 v2:172.21.15.190:6816/1964294001 19 ==== osd_op_reply(61 [pgnls start_epoch 128 out=81b] v128'19 uv19 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9b400 2023-09-23T03:28:29.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.530+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:62 18.14 18:2b89f3ba:::10000000221.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0380 con 0x55d53bb9b400 2023-09-23T03:28:29.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.531+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.6 v2:172.21.15.190:6816/1964294001 20 ==== osd_op_reply(62 10000000221.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9b400 2023-09-23T03:28:29.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.531+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628321.0 2023-09-23T03:28:29.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.531+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:63 18.14 18:2b89f3ba:::10000000221.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0a80 con 0x55d53bb9b400 2023-09-23T03:28:29.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.535+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.6 v2:172.21.15.190:6816/1964294001 21 ==== osd_op_reply(63 10000000221.00000000 [call] v128'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9b400 2023-09-23T03:28:29.535 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.535+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:64 18.34 18:2c000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbdc00 con 0x55d53bb9ac00 2023-09-23T03:28:29.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.536+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.0 v2:172.21.15.154:6800/2920938138 26 ==== osd_op_reply(64 [pgnls start_epoch 128 out=49b] v126'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9ac00 2023-09-23T03:28:29.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.536+0000 7f182a8d7640 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:65 18.c 18:30000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbc380 con 0x55d53bb9b000 2023-09-23T03:28:29.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.537+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 16 ==== osd_op_reply(65 [pgnls start_epoch 128 out=81b] v126'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.537+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:66 18.c 18:30604137:::100000001fc.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53b871c00 con 0x55d53bb9b000 2023-09-23T03:28:29.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.538+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 17 ==== osd_op_reply(66 100000001fc.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.538+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628284.0 2023-09-23T03:28:29.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.538+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:67 18.c 18:30604137:::100000001fc.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1880 con 0x55d53bb9b000 2023-09-23T03:28:29.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.541+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 18 ==== osd_op_reply(67 100000001fc.00000000 [call] v128'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.542+0000 7f18314125c0 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55d53bb9bc00 0x55d53bc88b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:29.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.542+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:68 18.2c 18:34000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0700 con 0x55d53bb9bc00 2023-09-23T03:28:29.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.542+0000 7f182f0e0640 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55d53bb9bc00 0x55d53bc88b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:29.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.543+0000 7f182f0e0640 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55d53bb9bc00 0x55d53bc88b00 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:29.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.544+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.2 v2:172.21.15.154:6822/2884407243 1 ==== osd_op_reply(68 [pgnls start_epoch 128 out=101b] v126'12 uv12 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9bc00 2023-09-23T03:28:29.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.544+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:69 18.2c 18:35964134:::10000000223.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1c00 con 0x55d53bb9bc00 2023-09-23T03:28:29.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.545+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.2 v2:172.21.15.154:6822/2884407243 2 ==== osd_op_reply(69 10000000223.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9bc00 2023-09-23T03:28:29.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.545+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628323.0 2023-09-23T03:28:29.546 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.545+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:70 18.2c 18:35964134:::10000000223.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0e00 con 0x55d53bb9bc00 2023-09-23T03:28:29.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.549+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.2 v2:172.21.15.154:6822/2884407243 3 ==== osd_op_reply(70 10000000223.00000000 [call] v128'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9bc00 2023-09-23T03:28:29.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.549+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:71 18.2c 18:37389ade::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1180 con 0x55d53bb9bc00 2023-09-23T03:28:29.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.550+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.2 v2:172.21.15.154:6822/2884407243 4 ==== osd_op_reply(71 [pgnls start_epoch 128 out=81b] v128'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9bc00 2023-09-23T03:28:29.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.550+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:72 18.2c 18:37389ade:::10000000209.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1500 con 0x55d53bb9bc00 2023-09-23T03:28:29.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.551+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.2 v2:172.21.15.154:6822/2884407243 5 ==== osd_op_reply(72 10000000209.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9bc00 2023-09-23T03:28:29.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.551+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628297.0 2023-09-23T03:28:29.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.551+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:73 18.2c 18:37389ade:::10000000209.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbc000 con 0x55d53bb9bc00 2023-09-23T03:28:29.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.554+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.2 v2:172.21.15.154:6822/2884407243 6 ==== osd_op_reply(73 10000000209.00000000 [call] v128'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9bc00 2023-09-23T03:28:29.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.554+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:74 18.1c 18:38000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbc380 con 0x55d53bb9b800 2023-09-23T03:28:29.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.555+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.1 v2:172.21.15.154:6816/1267710005 2 ==== osd_op_reply(74 [pgnls start_epoch 128 out=49b] v126'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9b800 2023-09-23T03:28:29.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.555+0000 7f182a8d7640 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55d53bcb6400 0x55d53bc8b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:29.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.555+0000 7f182a8d7640 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:75 18.3c 18:3c000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbca80 con 0x55d53bcb6400 2023-09-23T03:28:29.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.556+0000 7f182e8df640 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55d53bcb6400 0x55d53bc8b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:29.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.557+0000 7f182e8df640 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55d53bcb6400 0x55d53bc8b180 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:29.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.558+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 1 ==== osd_op_reply(75 [pgnls start_epoch 128 out=49b] v126'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:29.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.558+0000 7f182a0d6640 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:76 18.2 18:40000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbc700 con 0x55d53bcb6400 2023-09-23T03:28:29.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.559+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 2 ==== osd_op_reply(76 [pgnls start_epoch 128 out=49b] v126'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:29.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.559+0000 7f182a8d7640 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:77 18.22 18:44000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0a80 con 0x55d53bcb6000 2023-09-23T03:28:29.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.560+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 2 ==== osd_op_reply(77 [pgnls start_epoch 128 out=101b] v126'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.560+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:78 18.22 18:450d4ca5:::1000000024c.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbce00 con 0x55d53bcb6000 2023-09-23T03:28:29.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.561+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 3 ==== osd_op_reply(78 1000000024c.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.561+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628364.0 2023-09-23T03:28:29.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.561+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:79 18.22 18:450d4ca5:::1000000024c.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbd180 con 0x55d53bcb6000 2023-09-23T03:28:29.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.564+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 4 ==== osd_op_reply(79 1000000024c.00000000 [call] v128'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.565+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:80 18.22 18:45b8cac8::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbd500 con 0x55d53bcb6000 2023-09-23T03:28:29.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.565+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 5 ==== osd_op_reply(80 [pgnls start_epoch 128 out=101b] v128'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.566+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:81 18.22 18:45b8cac8:::1000000023d.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbd880 con 0x55d53bcb6000 2023-09-23T03:28:29.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.566+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 6 ==== osd_op_reply(81 1000000023d.00000000 [stat out=16b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.566+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628349.0 2023-09-23T03:28:29.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.566+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:82 18.22 18:45b8cac8:::1000000023d.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbdc00 con 0x55d53bcb6000 2023-09-23T03:28:29.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.570+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 7 ==== osd_op_reply(82 1000000023d.00000000 [call] v128'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.570+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:83 18.22 18:47efb22f::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53b871880 con 0x55d53bcb6000 2023-09-23T03:28:29.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.571+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 8 ==== osd_op_reply(83 [pgnls start_epoch 128 out=81b] v128'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.571+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:84 18.22 18:47efb22f:::10000000235.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53b871c00 con 0x55d53bcb6000 2023-09-23T03:28:29.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.572+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 9 ==== osd_op_reply(84 10000000235.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.572+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628341.0 2023-09-23T03:28:29.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.572+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:85 18.22 18:47efb22f:::10000000235.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1880 con 0x55d53bcb6000 2023-09-23T03:28:29.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.575+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 10 ==== osd_op_reply(85 10000000235.00000000 [call] v128'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.576+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:86 18.12 18:48000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0a80 con 0x55d53bb9bc00 2023-09-23T03:28:29.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.576+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.2 v2:172.21.15.154:6822/2884407243 7 ==== osd_op_reply(86 [pgnls start_epoch 128 out=49b] v126'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9bc00 2023-09-23T03:28:29.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.577+0000 7f182a8d7640 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:87 18.32 18:4c000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0700 con 0x55d53bcb6000 2023-09-23T03:28:29.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.577+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 11 ==== osd_op_reply(87 [pgnls start_epoch 128 out=81b] v126'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.578+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:88 18.32 18:4e8c6251:::10000000252.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1c00 con 0x55d53bcb6000 2023-09-23T03:28:29.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.578+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 12 ==== osd_op_reply(88 10000000252.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.578+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628370.0 2023-09-23T03:28:29.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.578+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:89 18.32 18:4e8c6251:::10000000252.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0e00 con 0x55d53bcb6000 2023-09-23T03:28:29.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.582+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 13 ==== osd_op_reply(89 10000000252.00000000 [call] v128'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.582+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:90 18.a 18:50000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1180 con 0x55d53bcb6000 2023-09-23T03:28:29.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.583+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 14 ==== osd_op_reply(90 [pgnls start_epoch 128 out=101b] v126'9 uv9 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.583+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:91 18.a 18:52dcfedc:::1000000023f.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1500 con 0x55d53bcb6000 2023-09-23T03:28:29.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.584+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 15 ==== osd_op_reply(91 1000000023f.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.584+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628351.0 2023-09-23T03:28:29.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.584+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:92 18.a 18:52dcfedc:::1000000023f.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0000 con 0x55d53bcb6000 2023-09-23T03:28:29.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.587+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 16 ==== osd_op_reply(92 1000000023f.00000000 [call] v128'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.588+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:93 18.a 18:53d16f76::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0380 con 0x55d53bcb6000 2023-09-23T03:28:29.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.588+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 17 ==== osd_op_reply(93 [pgnls start_epoch 128 out=81b] v128'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.588+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:94 18.a 18:53d16f76:::1000000023b.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbce00 con 0x55d53bcb6000 2023-09-23T03:28:29.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.589+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 18 ==== osd_op_reply(94 1000000023b.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.589+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628347.0 2023-09-23T03:28:29.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.589+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:95 18.a 18:53d16f76:::1000000023b.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbd180 con 0x55d53bcb6000 2023-09-23T03:28:29.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.592+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 19 ==== osd_op_reply(95 1000000023b.00000000 [call] v128'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.593+0000 7f18314125c0 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55d53bcb6c00 0x55d53bc89080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:29.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.593+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:96 18.2a 18:54000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbd500 con 0x55d53bcb6c00 2023-09-23T03:28:29.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.593+0000 7f182f8e1640 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55d53bcb6c00 0x55d53bc89080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:29.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.594+0000 7f182f8e1640 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55d53bcb6c00 0x55d53bc89080 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:29.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.595+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 1 ==== osd_op_reply(96 [pgnls start_epoch 128 out=101b] v126'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:29.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.595+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:97 18.2a 18:55a6cd83:::1000000020a.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbd880 con 0x55d53bcb6c00 2023-09-23T03:28:29.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.596+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 2 ==== osd_op_reply(97 1000000020a.00000000 [stat out=16b] v0'0 uv15 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:29.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.596+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628298.0 2023-09-23T03:28:29.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.596+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:98 18.2a 18:55a6cd83:::1000000020a.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbdc00 con 0x55d53bcb6c00 2023-09-23T03:28:29.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.599+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 3 ==== osd_op_reply(98 1000000020a.00000000 [call] v128'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:29.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.600+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:99 18.2a 18:56889a45::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1880 con 0x55d53bcb6c00 2023-09-23T03:28:29.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.600+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 4 ==== osd_op_reply(99 [pgnls start_epoch 128 out=101b] v128'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:29.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.601+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:100 18.2a 18:56889a45:::10000000213.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0700 con 0x55d53bcb6c00 2023-09-23T03:28:29.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.601+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 5 ==== osd_op_reply(100 10000000213.00000000 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:29.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.602+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628307.0 2023-09-23T03:28:29.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.602+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:101 18.2a 18:56889a45:::10000000213.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1c00 con 0x55d53bcb6c00 2023-09-23T03:28:29.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.605+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 6 ==== osd_op_reply(101 10000000213.00000000 [call] v128'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:29.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.605+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:102 18.2a 18:576ad4f4::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0e00 con 0x55d53bcb6c00 2023-09-23T03:28:29.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.606+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 7 ==== osd_op_reply(102 [pgnls start_epoch 128 out=101b] v128'19 uv19 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:29.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.606+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:103 18.2a 18:576ad4f4:::10000000245.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1180 con 0x55d53bcb6c00 2023-09-23T03:28:29.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.607+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 8 ==== osd_op_reply(103 10000000245.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:29.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.607+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628357.0 2023-09-23T03:28:29.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.607+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:104 18.2a 18:576ad4f4:::10000000245.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1500 con 0x55d53bcb6c00 2023-09-23T03:28:29.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.610+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 9 ==== osd_op_reply(104 10000000245.00000000 [call] v128'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:29.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.610+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:105 18.2a 18:57fe7535::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53b871880 con 0x55d53bcb6c00 2023-09-23T03:28:29.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.611+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 10 ==== osd_op_reply(105 [pgnls start_epoch 128 out=81b] v128'20 uv20 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:29.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.611+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:106 18.2a 18:57fe7535:::100000001f6.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53b871c00 con 0x55d53bcb6c00 2023-09-23T03:28:29.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.612+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 11 ==== osd_op_reply(106 100000001f6.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:29.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.612+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628278.0 2023-09-23T03:28:29.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.612+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:107 18.2a 18:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0000 con 0x55d53bcb6c00 2023-09-23T03:28:29.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.616+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 12 ==== osd_op_reply(107 100000001f6.00000000 [call] v128'21 uv21 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:29.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.616+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:108 18.1a 18:58000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0380 con 0x55d53bcb6000 2023-09-23T03:28:29.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.617+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 20 ==== osd_op_reply(108 [pgnls start_epoch 128 out=101b] v126'11 uv11 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.617+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:109 18.1a 18:582f7b2d:::10000000226.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1880 con 0x55d53bcb6000 2023-09-23T03:28:29.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.617+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 21 ==== osd_op_reply(109 10000000226.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.618+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628326.0 2023-09-23T03:28:29.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.618+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:110 18.1a 18:582f7b2d:::10000000226.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0700 con 0x55d53bcb6000 2023-09-23T03:28:29.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.621+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 22 ==== osd_op_reply(110 10000000226.00000000 [call] v128'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.621+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:111 18.1a 18:5a2a1294::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbce00 con 0x55d53bcb6000 2023-09-23T03:28:29.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.622+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 23 ==== osd_op_reply(111 [pgnls start_epoch 128 out=81b] v128'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.622+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:112 18.1a 18:5a2a1294:::1000000023a.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbd180 con 0x55d53bcb6000 2023-09-23T03:28:29.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.623+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 24 ==== osd_op_reply(112 1000000023a.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.623+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628346.0 2023-09-23T03:28:29.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.623+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:113 18.1a 18:5a2a1294:::1000000023a.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbd500 con 0x55d53bcb6000 2023-09-23T03:28:29.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.627+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 25 ==== osd_op_reply(113 1000000023a.00000000 [call] v128'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.627+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:114 18.3a 18:5c000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbd880 con 0x55d53bb9ac00 2023-09-23T03:28:29.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.628+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.0 v2:172.21.15.154:6800/2920938138 27 ==== osd_op_reply(114 [pgnls start_epoch 128 out=49b] v126'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9ac00 2023-09-23T03:28:29.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.628+0000 7f182a8d7640 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:115 18.6 18:60000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbc000 con 0x55d53bcb6400 2023-09-23T03:28:29.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.629+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 3 ==== osd_op_reply(115 [pgnls start_epoch 128 out=81b] v126'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:29.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.629+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:116 18.6 18:614797eb:::10000000228.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbdc00 con 0x55d53bcb6400 2023-09-23T03:28:29.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.630+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 4 ==== osd_op_reply(116 10000000228.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:29.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.630+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628328.0 2023-09-23T03:28:29.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.630+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:117 18.6 18:614797eb:::10000000228.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1c00 con 0x55d53bcb6400 2023-09-23T03:28:29.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.634+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 5 ==== osd_op_reply(117 10000000228.00000000 [call] v128'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:29.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.634+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:118 18.26 18:64000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0e00 con 0x55d53bb9bc00 2023-09-23T03:28:29.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.635+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.2 v2:172.21.15.154:6822/2884407243 8 ==== osd_op_reply(118 [pgnls start_epoch 128 out=101b] v126'21 uv21 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9bc00 2023-09-23T03:28:29.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.635+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:119 18.26 18:658f2455:::100000001ff.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1180 con 0x55d53bb9bc00 2023-09-23T03:28:29.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.636+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.2 v2:172.21.15.154:6822/2884407243 9 ==== osd_op_reply(119 100000001ff.00000000 [stat out=16b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9bc00 2023-09-23T03:28:29.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.636+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628287.0 2023-09-23T03:28:29.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.636+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:120 18.26 18:658f2455:::100000001ff.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1500 con 0x55d53bb9bc00 2023-09-23T03:28:29.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.640+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.2 v2:172.21.15.154:6822/2884407243 10 ==== osd_op_reply(120 100000001ff.00000000 [call] v128'22 uv22 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9bc00 2023-09-23T03:28:29.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.640+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:121 18.26 18:65d05a5c::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0000 con 0x55d53bb9bc00 2023-09-23T03:28:29.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.641+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.2 v2:172.21.15.154:6822/2884407243 11 ==== osd_op_reply(121 [pgnls start_epoch 128 out=101b] v128'22 uv22 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9bc00 2023-09-23T03:28:29.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.641+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:122 18.26 18:65d05a5c:::10000000258.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0380 con 0x55d53bb9bc00 2023-09-23T03:28:29.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.642+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.2 v2:172.21.15.154:6822/2884407243 12 ==== osd_op_reply(122 10000000258.00000000 [stat out=16b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9bc00 2023-09-23T03:28:29.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.642+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628376.0 2023-09-23T03:28:29.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.642+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:123 18.26 18:65d05a5c:::10000000258.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1880 con 0x55d53bb9bc00 2023-09-23T03:28:29.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.646+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.2 v2:172.21.15.154:6822/2884407243 13 ==== osd_op_reply(123 10000000258.00000000 [call] v128'23 uv23 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9bc00 2023-09-23T03:28:29.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.646+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:124 18.26 18:65e03fee::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0700 con 0x55d53bb9bc00 2023-09-23T03:28:29.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.647+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.2 v2:172.21.15.154:6822/2884407243 14 ==== osd_op_reply(124 [pgnls start_epoch 128 out=81b] v128'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9bc00 2023-09-23T03:28:29.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.647+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:125 18.26 18:65e03fee:::10000000240.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53b871880 con 0x55d53bb9bc00 2023-09-23T03:28:29.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.647+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.2 v2:172.21.15.154:6822/2884407243 15 ==== osd_op_reply(125 10000000240.00000000 [stat out=16b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9bc00 2023-09-23T03:28:29.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.648+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628352.0 2023-09-23T03:28:29.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.648+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:126 18.26 18:65e03fee:::10000000240.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53b871c00 con 0x55d53bb9bc00 2023-09-23T03:28:29.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.651+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.2 v2:172.21.15.154:6822/2884407243 16 ==== osd_op_reply(126 10000000240.00000000 [call] v128'24 uv24 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9bc00 2023-09-23T03:28:29.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.651+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:127 18.16 18:68000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0a80 con 0x55d53bcb6000 2023-09-23T03:28:29.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.652+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 26 ==== osd_op_reply(127 [pgnls start_epoch 128 out=49b] v126'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.652+0000 7f182a0d6640 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:128 18.36 18:6c000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bce8000 con 0x55d53bb9b800 2023-09-23T03:28:29.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.653+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.1 v2:172.21.15.154:6816/1267710005 3 ==== osd_op_reply(128 [pgnls start_epoch 128 out=101b] v126'23 uv23 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9b800 2023-09-23T03:28:29.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.653+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:129 18.36 18:6cdfec98:::10000000255.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0e00 con 0x55d53bb9b800 2023-09-23T03:28:29.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.654+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.1 v2:172.21.15.154:6816/1267710005 4 ==== osd_op_reply(129 10000000255.00000000 [stat out=16b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9b800 2023-09-23T03:28:29.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.654+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628373.0 2023-09-23T03:28:29.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.654+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:130 18.36 18:6cdfec98:::10000000255.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1180 con 0x55d53bb9b800 2023-09-23T03:28:29.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.658+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.1 v2:172.21.15.154:6816/1267710005 5 ==== osd_op_reply(130 10000000255.00000000 [call] v128'24 uv24 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9b800 2023-09-23T03:28:29.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.658+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:131 18.36 18:6ee5e2c4::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1500 con 0x55d53bb9b800 2023-09-23T03:28:29.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.659+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.1 v2:172.21.15.154:6816/1267710005 6 ==== osd_op_reply(131 [pgnls start_epoch 128 out=101b] v128'24 uv24 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9b800 2023-09-23T03:28:29.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.659+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:132 18.36 18:6ee5e2c4:::100000001f9.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0000 con 0x55d53bb9b800 2023-09-23T03:28:29.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.660+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.1 v2:172.21.15.154:6816/1267710005 7 ==== osd_op_reply(132 100000001f9.00000000 [stat out=16b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9b800 2023-09-23T03:28:29.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.660+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628281.0 2023-09-23T03:28:29.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.660+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:133 18.36 18:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0380 con 0x55d53bb9b800 2023-09-23T03:28:29.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.664+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.1 v2:172.21.15.154:6816/1267710005 8 ==== osd_op_reply(133 100000001f9.00000000 [call] v128'25 uv25 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9b800 2023-09-23T03:28:29.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.664+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:134 18.36 18:6f746908::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1880 con 0x55d53bb9b800 2023-09-23T03:28:29.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.665+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.1 v2:172.21.15.154:6816/1267710005 9 ==== osd_op_reply(134 [pgnls start_epoch 128 out=101b] v128'25 uv25 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9b800 2023-09-23T03:28:29.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.665+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:135 18.36 18:6f746908:::1000000022c.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0700 con 0x55d53bb9b800 2023-09-23T03:28:29.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.666+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.1 v2:172.21.15.154:6816/1267710005 10 ==== osd_op_reply(135 1000000022c.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9b800 2023-09-23T03:28:29.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.666+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628332.0 2023-09-23T03:28:29.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.666+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:136 18.36 18:6f746908:::1000000022c.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bce8000 con 0x55d53bb9b800 2023-09-23T03:28:29.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.669+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.1 v2:172.21.15.154:6816/1267710005 11 ==== osd_op_reply(136 1000000022c.00000000 [call] v128'26 uv26 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9b800 2023-09-23T03:28:29.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.669+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:137 18.36 18:6fe38dc2::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bce8a80 con 0x55d53bb9b800 2023-09-23T03:28:29.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.670+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.1 v2:172.21.15.154:6816/1267710005 12 ==== osd_op_reply(137 [pgnls start_epoch 128 out=81b] v128'26 uv26 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9b800 2023-09-23T03:28:29.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.670+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:138 18.36 18:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bce8e00 con 0x55d53bb9b800 2023-09-23T03:28:29.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.671+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.1 v2:172.21.15.154:6816/1267710005 13 ==== osd_op_reply(138 100000001f7.00000000 [stat out=16b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9b800 2023-09-23T03:28:29.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.671+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628279.0 2023-09-23T03:28:29.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.671+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:139 18.36 18:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bce9180 con 0x55d53bb9b800 2023-09-23T03:28:29.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.675+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.1 v2:172.21.15.154:6816/1267710005 14 ==== osd_op_reply(139 100000001f7.00000000 [call] v128'27 uv27 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9b800 2023-09-23T03:28:29.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.675+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:140 18.e 18:70000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bce9500 con 0x55d53bb9ac00 2023-09-23T03:28:29.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.676+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.0 v2:172.21.15.154:6800/2920938138 28 ==== osd_op_reply(140 [pgnls start_epoch 128 out=81b] v126'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9ac00 2023-09-23T03:28:29.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.676+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:141 18.e 18:71c13145:::1000000022f.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bce9880 con 0x55d53bb9ac00 2023-09-23T03:28:29.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.677+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.0 v2:172.21.15.154:6800/2920938138 29 ==== osd_op_reply(141 1000000022f.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9ac00 2023-09-23T03:28:29.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.677+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628335.0 2023-09-23T03:28:29.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.677+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:142 18.e 18:71c13145:::1000000022f.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bce9c00 con 0x55d53bb9ac00 2023-09-23T03:28:29.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.681+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.0 v2:172.21.15.154:6800/2920938138 30 ==== osd_op_reply(142 1000000022f.00000000 [call] v128'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9ac00 2023-09-23T03:28:29.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.681+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:143 18.2e 18:74000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1c00 con 0x55d53bcb6000 2023-09-23T03:28:29.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.682+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 27 ==== osd_op_reply(143 [pgnls start_epoch 128 out=101b] v126'16 uv16 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.682+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:144 18.2e 18:74d13cfc:::1000000021c.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbc380 con 0x55d53bcb6000 2023-09-23T03:28:29.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.683+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 28 ==== osd_op_reply(144 1000000021c.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.683+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628316.0 2023-09-23T03:28:29.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.683+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:145 18.2e 18:74d13cfc:::1000000021c.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbca80 con 0x55d53bcb6000 2023-09-23T03:28:29.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.687+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 29 ==== osd_op_reply(145 1000000021c.00000000 [call] v128'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.687+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:146 18.2e 18:7528f44f::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbc700 con 0x55d53bcb6000 2023-09-23T03:28:29.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.688+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 30 ==== osd_op_reply(146 [pgnls start_epoch 128 out=101b] v128'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.688+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:147 18.2e 18:7528f44f:::10000000242.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbd880 con 0x55d53bcb6000 2023-09-23T03:28:29.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.689+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 31 ==== osd_op_reply(147 10000000242.00000000 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.689+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628354.0 2023-09-23T03:28:29.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.689+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:148 18.2e 18:7528f44f:::10000000242.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbc000 con 0x55d53bcb6000 2023-09-23T03:28:29.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.692+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 32 ==== osd_op_reply(148 10000000242.00000000 [call] v128'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.692+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:149 18.2e 18:7611a124::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbdc00 con 0x55d53bcb6000 2023-09-23T03:28:29.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.693+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 33 ==== osd_op_reply(149 [pgnls start_epoch 128 out=81b] v128'18 uv18 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.693+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:150 18.2e 18:7611a124:::1000000021e.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bce8000 con 0x55d53bcb6000 2023-09-23T03:28:29.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.694+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 34 ==== osd_op_reply(150 1000000021e.00000000 [stat out=16b] v0'0 uv15 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.694+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628318.0 2023-09-23T03:28:29.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.694+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:151 18.2e 18:7611a124:::1000000021e.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bce8a80 con 0x55d53bcb6000 2023-09-23T03:28:29.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.698+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 35 ==== osd_op_reply(151 1000000021e.00000000 [call] v128'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.698+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:152 18.1e 18:78000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbc000 con 0x55d53bb9b000 2023-09-23T03:28:29.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.700+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 19 ==== osd_op_reply(152 [pgnls start_epoch 128 out=101b] v126'11 uv11 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.700+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:153 18.1e 18:789b801b:::1000000020c.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbc380 con 0x55d53bb9b000 2023-09-23T03:28:29.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.701+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 20 ==== osd_op_reply(153 1000000020c.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.701+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628300.0 2023-09-23T03:28:29.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.701+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:154 18.1e 18:789b801b:::1000000020c.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbc700 con 0x55d53bb9b000 2023-09-23T03:28:29.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.705+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 21 ==== osd_op_reply(154 1000000020c.00000000 [call] v128'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.705+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:155 18.1e 18:7a9d9b47::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbca80 con 0x55d53bb9b000 2023-09-23T03:28:29.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.706+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 22 ==== osd_op_reply(155 [pgnls start_epoch 128 out=101b] v128'12 uv12 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.707+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:156 18.1e 18:7a9d9b47:::1000000022b.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbce00 con 0x55d53bb9b000 2023-09-23T03:28:29.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.708+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 23 ==== osd_op_reply(156 1000000022b.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.708+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628331.0 2023-09-23T03:28:29.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.708+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:157 18.1e 18:7a9d9b47:::1000000022b.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbd180 con 0x55d53bb9b000 2023-09-23T03:28:29.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.711+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 24 ==== osd_op_reply(157 1000000022b.00000000 [call] v128'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.711+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:158 18.1e 18:7ade2fd8::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbd500 con 0x55d53bb9b000 2023-09-23T03:28:29.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.712+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 25 ==== osd_op_reply(158 [pgnls start_epoch 128 out=81b] v128'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.712+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:159 18.1e 18:7ade2fd8:::1000000024e.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbd880 con 0x55d53bb9b000 2023-09-23T03:28:29.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.713+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 26 ==== osd_op_reply(159 1000000024e.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.713+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628366.0 2023-09-23T03:28:29.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.714+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:160 18.1e 18:7ade2fd8:::1000000024e.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbdc00 con 0x55d53bb9b000 2023-09-23T03:28:29.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.717+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 27 ==== osd_op_reply(160 1000000024e.00000000 [call] v128'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.717+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:161 18.3e 18:7c000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0000 con 0x55d53bcb6000 2023-09-23T03:28:29.718 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.718+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 36 ==== osd_op_reply(161 [pgnls start_epoch 128 out=81b] v126'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.718+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:162 18.3e 18:7f918703:::10000000229.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0380 con 0x55d53bcb6000 2023-09-23T03:28:29.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.719+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 37 ==== osd_op_reply(162 10000000229.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.719+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628329.0 2023-09-23T03:28:29.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.719+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:163 18.3e 18:7f918703:::10000000229.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0700 con 0x55d53bcb6000 2023-09-23T03:28:29.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.723+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 38 ==== osd_op_reply(163 10000000229.00000000 [call] v128'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.723+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:164 18.1 18:80000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0a80 con 0x55d53bcb6000 2023-09-23T03:28:29.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.724+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 39 ==== osd_op_reply(164 [pgnls start_epoch 128 out=81b] v126'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.724+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:165 18.1 18:805bc955:::10000000224.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0e00 con 0x55d53bcb6000 2023-09-23T03:28:29.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.725+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 40 ==== osd_op_reply(165 10000000224.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.725+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628324.0 2023-09-23T03:28:29.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.725+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:166 18.1 18:805bc955:::10000000224.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1180 con 0x55d53bcb6000 2023-09-23T03:28:29.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.729+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 41 ==== osd_op_reply(166 10000000224.00000000 [call] v128'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.729+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:167 18.21 18:84000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1500 con 0x55d53bb9b000 2023-09-23T03:28:29.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.730+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 28 ==== osd_op_reply(167 [pgnls start_epoch 128 out=49b] v126'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.730 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.730+0000 7f182a0d6640 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:168 18.11 18:88000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bce8380 con 0x55d53bb9b800 2023-09-23T03:28:29.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.731+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.1 v2:172.21.15.154:6816/1267710005 15 ==== osd_op_reply(168 [pgnls start_epoch 128 out=49b] v126'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9b800 2023-09-23T03:28:29.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.731+0000 7f182a8d7640 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:169 18.31 18:8c000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0000 con 0x55d53bcb6c00 2023-09-23T03:28:29.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.732+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 13 ==== osd_op_reply(169 [pgnls start_epoch 128 out=101b] v126'11 uv11 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:29.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.732+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:170 18.31 18:8c51d9b0:::10000000216.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1880 con 0x55d53bcb6c00 2023-09-23T03:28:29.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.733+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 14 ==== osd_op_reply(170 10000000216.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:29.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.733+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628310.0 2023-09-23T03:28:29.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.733+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:171 18.31 18:8c51d9b0:::10000000216.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1c00 con 0x55d53bcb6c00 2023-09-23T03:28:29.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.738+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 15 ==== osd_op_reply(171 10000000216.00000000 [call] v128'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:29.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.738+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:172 18.31 18:8fb7f7b7::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbc000 con 0x55d53bcb6c00 2023-09-23T03:28:29.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.739+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 16 ==== osd_op_reply(172 [pgnls start_epoch 128 out=81b] v128'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:29.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.739+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:173 18.31 18:8fb7f7b7:::10000000251.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbc380 con 0x55d53bcb6c00 2023-09-23T03:28:29.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.740+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 17 ==== osd_op_reply(173 10000000251.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:29.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.740+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628369.0 2023-09-23T03:28:29.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.740+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:174 18.31 18:8fb7f7b7:::10000000251.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbc700 con 0x55d53bcb6c00 2023-09-23T03:28:29.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.744+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 18 ==== osd_op_reply(174 10000000251.00000000 [call] v128'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:29.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.744+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:175 18.9 18:90000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbca80 con 0x55d53bcb6000 2023-09-23T03:28:29.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.745+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 42 ==== osd_op_reply(175 [pgnls start_epoch 128 out=101b] v126'6 uv6 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.745+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:176 18.9 18:915bff82:::1000000024d.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbce00 con 0x55d53bcb6000 2023-09-23T03:28:29.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.746+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 43 ==== osd_op_reply(176 1000000024d.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.746+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628365.0 2023-09-23T03:28:29.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.746+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:177 18.9 18:915bff82:::1000000024d.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbd180 con 0x55d53bcb6000 2023-09-23T03:28:29.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.750+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 44 ==== osd_op_reply(177 1000000024d.00000000 [call] v128'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.750+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:178 18.9 18:91e366d7::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbd500 con 0x55d53bcb6000 2023-09-23T03:28:29.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.751+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 45 ==== osd_op_reply(178 [pgnls start_epoch 128 out=81b] v128'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.751+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:179 18.9 18:91e366d7:::10000000230.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbd880 con 0x55d53bcb6000 2023-09-23T03:28:29.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.752+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 46 ==== osd_op_reply(179 10000000230.00000000 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.752+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628336.0 2023-09-23T03:28:29.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.752+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:180 18.9 18:91e366d7:::10000000230.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbdc00 con 0x55d53bcb6000 2023-09-23T03:28:29.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.756+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 47 ==== osd_op_reply(180 10000000230.00000000 [call] v128'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.756+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:181 18.29 18:94000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbc000 con 0x55d53bcb6400 2023-09-23T03:28:29.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.757+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 6 ==== osd_op_reply(181 [pgnls start_epoch 128 out=101b] v126'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:29.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.758+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:182 18.29 18:94c02207:::1000000024a.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbc380 con 0x55d53bcb6400 2023-09-23T03:28:29.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.759+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 7 ==== osd_op_reply(182 1000000024a.00000000 [stat out=16b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:29.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.759+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628362.0 2023-09-23T03:28:29.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.759+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:183 18.29 18:94c02207:::1000000024a.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbc700 con 0x55d53bcb6400 2023-09-23T03:28:29.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.763+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 8 ==== osd_op_reply(183 1000000024a.00000000 [call] v128'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:29.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.763+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:184 18.29 18:95e8e66d::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbca80 con 0x55d53bcb6400 2023-09-23T03:28:29.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.764+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 9 ==== osd_op_reply(184 [pgnls start_epoch 128 out=101b] v128'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:29.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.764+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:185 18.29 18:95e8e66d:::10000000256.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbc000 con 0x55d53bcb6400 2023-09-23T03:28:29.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.765+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 10 ==== osd_op_reply(185 10000000256.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:29.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.765+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628374.0 2023-09-23T03:28:29.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.765+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:186 18.29 18:95e8e66d:::10000000256.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbc380 con 0x55d53bcb6400 2023-09-23T03:28:29.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.769+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 11 ==== osd_op_reply(186 10000000256.00000000 [call] v128'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:29.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.770+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:187 18.29 18:97401066::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbc700 con 0x55d53bcb6400 2023-09-23T03:28:29.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.771+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 12 ==== osd_op_reply(187 [pgnls start_epoch 128 out=81b] v128'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:29.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.771+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:188 18.29 18:97401066:::10000000247.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbca80 con 0x55d53bcb6400 2023-09-23T03:28:29.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.772+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 13 ==== osd_op_reply(188 10000000247.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:29.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.772+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628359.0 2023-09-23T03:28:29.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.772+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:189 18.29 18:97401066:::10000000247.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bce8e00 con 0x55d53bcb6400 2023-09-23T03:28:29.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.775+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 14 ==== osd_op_reply(189 10000000247.00000000 [call] v128'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:29.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.776+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:190 18.19 18:98000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bce9180 con 0x55d53bcb6c00 2023-09-23T03:28:29.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.777+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 19 ==== osd_op_reply(190 [pgnls start_epoch 128 out=101b] v126'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:29.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.777+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:191 18.19 18:992a6246:::10000000200.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bce9500 con 0x55d53bcb6c00 2023-09-23T03:28:29.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.778+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 20 ==== osd_op_reply(191 10000000200.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:29.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.778+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628288.0 2023-09-23T03:28:29.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.778+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:192 18.19 18:992a6246:::10000000200.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bce9880 con 0x55d53bcb6c00 2023-09-23T03:28:29.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.783+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 21 ==== osd_op_reply(192 10000000200.00000000 [call] v128'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:29.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.783+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:193 18.19 18:997b7129::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bce9c00 con 0x55d53bcb6c00 2023-09-23T03:28:29.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.783+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 22 ==== osd_op_reply(193 [pgnls start_epoch 128 out=81b] v128'15 uv15 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:29.784 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.784+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:194 18.19 18:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bce8380 con 0x55d53bcb6c00 2023-09-23T03:28:29.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.785+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 23 ==== osd_op_reply(194 100000001f8.00000000 [stat out=16b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:29.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.785+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628280.0 2023-09-23T03:28:29.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.785+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:195 18.19 18:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0000 con 0x55d53bcb6c00 2023-09-23T03:28:29.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.788+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 24 ==== osd_op_reply(195 100000001f8.00000000 [call] v128'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:29.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.789+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:196 18.39 18:9c000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0380 con 0x55d53bb9ac00 2023-09-23T03:28:29.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.790+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.0 v2:172.21.15.154:6800/2920938138 31 ==== osd_op_reply(196 [pgnls start_epoch 128 out=81b] v126'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9ac00 2023-09-23T03:28:29.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.790+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:197 18.39 18:9f1351c3:::10000000222.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0700 con 0x55d53bb9ac00 2023-09-23T03:28:29.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.791+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.0 v2:172.21.15.154:6800/2920938138 32 ==== osd_op_reply(197 10000000222.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9ac00 2023-09-23T03:28:29.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.791+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628322.0 2023-09-23T03:28:29.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.791+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:198 18.39 18:9f1351c3:::10000000222.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0a80 con 0x55d53bb9ac00 2023-09-23T03:28:29.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.794+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.0 v2:172.21.15.154:6800/2920938138 33 ==== osd_op_reply(198 10000000222.00000000 [call] v128'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9ac00 2023-09-23T03:28:29.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.794+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:199 18.5 18:a0000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0e00 con 0x55d53bb9b800 2023-09-23T03:28:29.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.795+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.1 v2:172.21.15.154:6816/1267710005 16 ==== osd_op_reply(199 [pgnls start_epoch 128 out=101b] v126'9 uv9 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9b800 2023-09-23T03:28:29.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.795+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:200 18.5 18:a0beda95:::10000000204.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1180 con 0x55d53bb9b800 2023-09-23T03:28:29.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.796+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.1 v2:172.21.15.154:6816/1267710005 17 ==== osd_op_reply(200 10000000204.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9b800 2023-09-23T03:28:29.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.796+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628292.0 2023-09-23T03:28:29.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.796+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:201 18.5 18:a0beda95:::10000000204.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1500 con 0x55d53bb9b800 2023-09-23T03:28:29.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.800+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.1 v2:172.21.15.154:6816/1267710005 18 ==== osd_op_reply(201 10000000204.00000000 [call] v128'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9b800 2023-09-23T03:28:29.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.801+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:202 18.5 18:a3a50212::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1880 con 0x55d53bb9b800 2023-09-23T03:28:29.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.801+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.1 v2:172.21.15.154:6816/1267710005 19 ==== osd_op_reply(202 [pgnls start_epoch 128 out=81b] v128'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9b800 2023-09-23T03:28:29.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.802+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:203 18.5 18:a3a50212:::10000000254.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1c00 con 0x55d53bb9b800 2023-09-23T03:28:29.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.802+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.1 v2:172.21.15.154:6816/1267710005 20 ==== osd_op_reply(203 10000000254.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9b800 2023-09-23T03:28:29.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.803+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628372.0 2023-09-23T03:28:29.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.803+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:204 18.5 18:a3a50212:::10000000254.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0380 con 0x55d53bb9b800 2023-09-23T03:28:29.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.806+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.1 v2:172.21.15.154:6816/1267710005 21 ==== osd_op_reply(204 10000000254.00000000 [call] v128'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9b800 2023-09-23T03:28:29.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.806+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:205 18.25 18:a4000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0700 con 0x55d53bb9b400 2023-09-23T03:28:29.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.807+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.6 v2:172.21.15.190:6816/1964294001 22 ==== osd_op_reply(205 [pgnls start_epoch 128 out=101b] v126'10 uv10 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9b400 2023-09-23T03:28:29.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.807+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:206 18.25 18:a53878a9:::10000000237.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0a80 con 0x55d53bb9b400 2023-09-23T03:28:29.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.809+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.6 v2:172.21.15.190:6816/1964294001 23 ==== osd_op_reply(206 10000000237.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9b400 2023-09-23T03:28:29.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.809+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628343.0 2023-09-23T03:28:29.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.809+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:207 18.25 18:a53878a9:::10000000237.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0e00 con 0x55d53bb9b400 2023-09-23T03:28:29.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.813+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.6 v2:172.21.15.190:6816/1964294001 24 ==== osd_op_reply(207 10000000237.00000000 [call] v128'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9b400 2023-09-23T03:28:29.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.813+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:208 18.25 18:a7ae8a0f::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1180 con 0x55d53bb9b400 2023-09-23T03:28:29.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.814+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.6 v2:172.21.15.190:6816/1964294001 25 ==== osd_op_reply(208 [pgnls start_epoch 128 out=81b] v128'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9b400 2023-09-23T03:28:29.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.814+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:209 18.25 18:a7ae8a0f:::10000000243.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1500 con 0x55d53bb9b400 2023-09-23T03:28:29.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.815+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.6 v2:172.21.15.190:6816/1964294001 26 ==== osd_op_reply(209 10000000243.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9b400 2023-09-23T03:28:29.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.815+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628355.0 2023-09-23T03:28:29.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.815+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:210 18.25 18:a7ae8a0f:::10000000243.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bce8e00 con 0x55d53bb9b400 2023-09-23T03:28:29.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.819+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.6 v2:172.21.15.190:6816/1964294001 27 ==== osd_op_reply(210 10000000243.00000000 [call] v128'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9b400 2023-09-23T03:28:29.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.819+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:211 18.15 18:a8000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbc000 con 0x55d53bb9b000 2023-09-23T03:28:29.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.820+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 29 ==== osd_op_reply(211 [pgnls start_epoch 128 out=81b] v126'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.820+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:212 18.15 18:a8e69ea8:::1000000020d.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbc380 con 0x55d53bb9b000 2023-09-23T03:28:29.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.822+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 30 ==== osd_op_reply(212 1000000020d.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.822+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628301.0 2023-09-23T03:28:29.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.822+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:213 18.15 18:a8e69ea8:::1000000020d.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbc700 con 0x55d53bb9b000 2023-09-23T03:28:29.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.827+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 31 ==== osd_op_reply(213 1000000020d.00000000 [call] v128'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.827+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:214 18.35 18:ac000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbca80 con 0x55d53bcb6c00 2023-09-23T03:28:29.828 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.828+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 25 ==== osd_op_reply(214 [pgnls start_epoch 128 out=49b] v126'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:29.828 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.828+0000 7f182a8d7640 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:215 18.d 18:b0000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bce8000 con 0x55d53bcb6c00 2023-09-23T03:28:29.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.829+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 26 ==== osd_op_reply(215 [pgnls start_epoch 128 out=49b] v126'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:29.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.829+0000 7f182a0d6640 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:216 18.2d 18:b4000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bce8700 con 0x55d53bb9b400 2023-09-23T03:28:29.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.830+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.6 v2:172.21.15.190:6816/1964294001 28 ==== osd_op_reply(216 [pgnls start_epoch 128 out=81b] v126'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9b400 2023-09-23T03:28:29.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.830+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:217 18.2d 18:b79a54f6:::10000000218.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbce00 con 0x55d53bb9b400 2023-09-23T03:28:29.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.831+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.6 v2:172.21.15.190:6816/1964294001 29 ==== osd_op_reply(217 10000000218.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9b400 2023-09-23T03:28:29.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.831+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628312.0 2023-09-23T03:28:29.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.832+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:218 18.2d 18:b79a54f6:::10000000218.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbd180 con 0x55d53bb9b400 2023-09-23T03:28:29.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.835+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.6 v2:172.21.15.190:6816/1964294001 30 ==== osd_op_reply(218 10000000218.00000000 [call] v128'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9b400 2023-09-23T03:28:29.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.835+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:219 18.1d 18:b8000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbd500 con 0x55d53bb9ac00 2023-09-23T03:28:29.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.836+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.0 v2:172.21.15.154:6800/2920938138 34 ==== osd_op_reply(219 [pgnls start_epoch 128 out=81b] v126'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9ac00 2023-09-23T03:28:29.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.836+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:220 18.1d 18:bab97ecf:::10000000246.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbd880 con 0x55d53bb9ac00 2023-09-23T03:28:29.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.837+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.0 v2:172.21.15.154:6800/2920938138 35 ==== osd_op_reply(220 10000000246.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9ac00 2023-09-23T03:28:29.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.837+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628358.0 2023-09-23T03:28:29.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.837+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:221 18.1d 18:bab97ecf:::10000000246.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbdc00 con 0x55d53bb9ac00 2023-09-23T03:28:29.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.841+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.0 v2:172.21.15.154:6800/2920938138 36 ==== osd_op_reply(221 10000000246.00000000 [call] v128'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9ac00 2023-09-23T03:28:29.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.841+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:222 18.3d 18:bc000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbc000 con 0x55d53bcb6000 2023-09-23T03:28:29.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.842+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 48 ==== osd_op_reply(222 [pgnls start_epoch 128 out=101b] v126'11 uv11 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.842+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:223 18.3d 18:bc0c3701:::10000000219.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbc380 con 0x55d53bcb6000 2023-09-23T03:28:29.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.843+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 49 ==== osd_op_reply(223 10000000219.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.843+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628313.0 2023-09-23T03:28:29.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.843+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:224 18.3d 18:bc0c3701:::10000000219.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbc700 con 0x55d53bcb6000 2023-09-23T03:28:29.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.846+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 50 ==== osd_op_reply(224 10000000219.00000000 [call] v128'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.847+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:225 18.3d 18:bfcd3380::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0700 con 0x55d53bcb6000 2023-09-23T03:28:29.848 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.847+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 51 ==== osd_op_reply(225 [pgnls start_epoch 128 out=81b] v128'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.848 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.848+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:226 18.3d 18:bfcd3380:::10000000205.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0a80 con 0x55d53bcb6000 2023-09-23T03:28:29.848 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.848+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 52 ==== osd_op_reply(226 10000000205.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.848+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628293.0 2023-09-23T03:28:29.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.848+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:227 18.3d 18:bfcd3380:::10000000205.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0e00 con 0x55d53bcb6000 2023-09-23T03:28:29.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.852+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.3 v2:172.21.15.154:6808/1133688468 53 ==== osd_op_reply(227 10000000205.00000000 [call] v128'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6000 2023-09-23T03:28:29.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.852+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:228 18.3 18:c0000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1180 con 0x55d53bb9b800 2023-09-23T03:28:29.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.853+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.1 v2:172.21.15.154:6816/1267710005 22 ==== osd_op_reply(228 [pgnls start_epoch 128 out=49b] v126'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9b800 2023-09-23T03:28:29.853 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.853+0000 7f182a8d7640 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:229 18.23 18:c4000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bce8a80 con 0x55d53bb9bc00 2023-09-23T03:28:29.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.854+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.2 v2:172.21.15.154:6822/2884407243 17 ==== osd_op_reply(229 [pgnls start_epoch 128 out=49b] v126'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9bc00 2023-09-23T03:28:29.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.854+0000 7f182a0d6640 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:230 18.13 18:c8000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53c620000 con 0x55d53bcb6400 2023-09-23T03:28:29.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.855+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 15 ==== osd_op_reply(230 [pgnls start_epoch 128 out=101b] v126'25 uv25 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:29.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.855+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:231 18.13 18:c8893608:::1000000023c.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1500 con 0x55d53bcb6400 2023-09-23T03:28:29.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.856+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 16 ==== osd_op_reply(231 1000000023c.00000000 [stat out=16b] v0'0 uv24 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:29.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.857+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628348.0 2023-09-23T03:28:29.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.857+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:232 18.13 18:c8893608:::1000000023c.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0000 con 0x55d53bcb6400 2023-09-23T03:28:29.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.861+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 17 ==== osd_op_reply(232 1000000023c.00000000 [call] v128'26 uv26 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:29.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.861+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:233 18.13 18:c933d8fa::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bce8e00 con 0x55d53bcb6400 2023-09-23T03:28:29.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.862+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 18 ==== osd_op_reply(233 [pgnls start_epoch 128 out=101b] v128'26 uv26 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:29.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.862+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:234 18.13 18:c933d8fa:::10000000241.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bce8700 con 0x55d53bcb6400 2023-09-23T03:28:29.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.863+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 19 ==== osd_op_reply(234 10000000241.00000000 [stat out=16b] v0'0 uv25 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:29.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.863+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628353.0 2023-09-23T03:28:29.863 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.863+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:235 18.13 18:c933d8fa:::10000000241.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bce9180 con 0x55d53bcb6400 2023-09-23T03:28:29.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.866+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 20 ==== osd_op_reply(235 10000000241.00000000 [call] v128'27 uv27 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:29.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.866+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:236 18.13 18:c9855970::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bce9500 con 0x55d53bcb6400 2023-09-23T03:28:29.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.867+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 21 ==== osd_op_reply(236 [pgnls start_epoch 128 out=101b] v128'27 uv27 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:29.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.868+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:237 18.13 18:c9855970:::10000000239.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bce9880 con 0x55d53bcb6400 2023-09-23T03:28:29.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.868+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 22 ==== osd_op_reply(237 10000000239.00000000 [stat out=16b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:29.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.869+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628345.0 2023-09-23T03:28:29.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.869+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:238 18.13 18:c9855970:::10000000239.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bce9c00 con 0x55d53bcb6400 2023-09-23T03:28:29.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.872+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 23 ==== osd_op_reply(238 10000000239.00000000 [call] v128'28 uv28 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:29.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.872+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:239 18.13 18:ca8fdbba::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bce8380 con 0x55d53bcb6400 2023-09-23T03:28:29.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.874+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 24 ==== osd_op_reply(239 [pgnls start_epoch 128 out=101b] v128'28 uv28 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:29.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.874+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:240 18.13 18:ca8fdbba:::100000001fb.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53b871880 con 0x55d53bcb6400 2023-09-23T03:28:29.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.875+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 25 ==== osd_op_reply(240 100000001fb.00000000 [stat out=16b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:29.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.875+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628283.0 2023-09-23T03:28:29.875 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.875+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:241 18.13 18:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53b871c00 con 0x55d53bcb6400 2023-09-23T03:28:29.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.879+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 26 ==== osd_op_reply(241 100000001fb.00000000 [call] v128'29 uv29 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:29.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.879+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:242 18.13 18:cbcb6224::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53c620000 con 0x55d53bcb6400 2023-09-23T03:28:29.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.880+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 27 ==== osd_op_reply(242 [pgnls start_epoch 128 out=81b] v128'29 uv29 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:29.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.880+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:243 18.13 18:cbcb6224:::10000000212.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53c620e00 con 0x55d53bcb6400 2023-09-23T03:28:29.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.881+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 28 ==== osd_op_reply(243 10000000212.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:29.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.881+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628306.0 2023-09-23T03:28:29.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.881+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:244 18.13 18:cbcb6224:::10000000212.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53c621180 con 0x55d53bcb6400 2023-09-23T03:28:29.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.885+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 29 ==== osd_op_reply(244 10000000212.00000000 [call] v128'30 uv30 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:29.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.885+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:245 18.33 18:cc000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53c621500 con 0x55d53bb9b800 2023-09-23T03:28:29.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.886+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.1 v2:172.21.15.154:6816/1267710005 23 ==== osd_op_reply(245 [pgnls start_epoch 128 out=81b] v126'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9b800 2023-09-23T03:28:29.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.886+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:246 18.33 18:cf9e7a4f:::10000000248.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53c621880 con 0x55d53bb9b800 2023-09-23T03:28:29.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.887+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.1 v2:172.21.15.154:6816/1267710005 24 ==== osd_op_reply(246 10000000248.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9b800 2023-09-23T03:28:29.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.887+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628360.0 2023-09-23T03:28:29.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.887+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:247 18.33 18:cf9e7a4f:::10000000248.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53c621c00 con 0x55d53bb9b800 2023-09-23T03:28:29.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.891+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.1 v2:172.21.15.154:6816/1267710005 25 ==== osd_op_reply(247 10000000248.00000000 [call] v128'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9b800 2023-09-23T03:28:29.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.891+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:248 18.b 18:d0000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bce8e00 con 0x55d53bb9b000 2023-09-23T03:28:29.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.892+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 32 ==== osd_op_reply(248 [pgnls start_epoch 128 out=101b] v126'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.892+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:249 18.b 18:d0d26e50:::10000000225.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bce8700 con 0x55d53bb9b000 2023-09-23T03:28:29.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.893+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 33 ==== osd_op_reply(249 10000000225.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.893+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628325.0 2023-09-23T03:28:29.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.894+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:250 18.b 18:d0d26e50:::10000000225.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1880 con 0x55d53bb9b000 2023-09-23T03:28:29.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.897+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 34 ==== osd_op_reply(250 10000000225.00000000 [call] v128'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.897+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:251 18.b 18:d2dbd4ac::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1c00 con 0x55d53bb9b000 2023-09-23T03:28:29.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.898+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 35 ==== osd_op_reply(251 [pgnls start_epoch 128 out=101b] v128'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.898+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:252 18.b 18:d2dbd4ac:::10000000217.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0380 con 0x55d53bb9b000 2023-09-23T03:28:29.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.900+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 36 ==== osd_op_reply(252 10000000217.00000000 [stat out=16b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.900+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628311.0 2023-09-23T03:28:29.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.900+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:253 18.b 18:d2dbd4ac:::10000000217.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1180 con 0x55d53bb9b000 2023-09-23T03:28:29.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.904+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 37 ==== osd_op_reply(253 10000000217.00000000 [call] v128'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.904+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:254 18.b 18:d3eda40d::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1500 con 0x55d53bb9b000 2023-09-23T03:28:29.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.905+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 38 ==== osd_op_reply(254 [pgnls start_epoch 128 out=81b] v128'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.905+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:255 18.b 18:d3eda40d:::1000000020f.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0000 con 0x55d53bb9b000 2023-09-23T03:28:29.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.906+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 39 ==== osd_op_reply(255 1000000020f.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.906+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628303.0 2023-09-23T03:28:29.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.906+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:256 18.b 18:d3eda40d:::1000000020f.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbd500 con 0x55d53bb9b000 2023-09-23T03:28:29.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.910+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 40 ==== osd_op_reply(256 1000000020f.00000000 [call] v128'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.910+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:257 18.2b 18:d4000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbd880 con 0x55d53bcb6400 2023-09-23T03:28:29.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.911+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 30 ==== osd_op_reply(257 [pgnls start_epoch 128 out=101b] v126'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:29.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.911+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:258 18.2b 18:d4a58307:::10000000203.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbdc00 con 0x55d53bcb6400 2023-09-23T03:28:29.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.913+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 31 ==== osd_op_reply(258 10000000203.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:29.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.913+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628291.0 2023-09-23T03:28:29.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.913+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:259 18.2b 18:d4a58307:::10000000203.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53c620000 con 0x55d53bcb6400 2023-09-23T03:28:29.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.916+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 32 ==== osd_op_reply(259 10000000203.00000000 [call] v128'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:29.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.917+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:260 18.2b 18:d5ee56ee::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53c620e00 con 0x55d53bcb6400 2023-09-23T03:28:29.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.918+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 33 ==== osd_op_reply(260 [pgnls start_epoch 128 out=81b] v128'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:29.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.918+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:261 18.2b 18:d5ee56ee:::1000000022d.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53c621180 con 0x55d53bcb6400 2023-09-23T03:28:29.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.919+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 34 ==== osd_op_reply(261 1000000022d.00000000 [stat out=16b] v0'0 uv15 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:29.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.919+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628333.0 2023-09-23T03:28:29.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.919+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:262 18.2b 18:d5ee56ee:::1000000022d.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53c621500 con 0x55d53bcb6400 2023-09-23T03:28:29.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.923+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 35 ==== osd_op_reply(262 1000000022d.00000000 [call] v128'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:29.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.923+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:263 18.1b 18:d8000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53c621880 con 0x55d53bb9b800 2023-09-23T03:28:29.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.924+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.1 v2:172.21.15.154:6816/1267710005 26 ==== osd_op_reply(263 [pgnls start_epoch 128 out=49b] v126'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9b800 2023-09-23T03:28:29.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.924+0000 7f182a0d6640 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:264 18.3b 18:dc000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53c620380 con 0x55d53bcb6c00 2023-09-23T03:28:29.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.926+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 27 ==== osd_op_reply(264 [pgnls start_epoch 128 out=101b] v126'10 uv10 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:29.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.926+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:265 18.3b 18:dd50df43:::10000000249.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53c621c00 con 0x55d53bcb6c00 2023-09-23T03:28:29.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.927+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 28 ==== osd_op_reply(265 10000000249.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:29.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.927+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628361.0 2023-09-23T03:28:29.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.927+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:266 18.3b 18:dd50df43:::10000000249.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53b871880 con 0x55d53bcb6c00 2023-09-23T03:28:29.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.931+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 29 ==== osd_op_reply(266 10000000249.00000000 [call] v128'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:29.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.931+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:267 18.3b 18:deccbd37::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53b871c00 con 0x55d53bcb6c00 2023-09-23T03:28:29.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.932+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 30 ==== osd_op_reply(267 [pgnls start_epoch 128 out=101b] v128'11 uv11 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:29.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.932+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:268 18.3b 18:deccbd37:::100000001fe.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bce8000 con 0x55d53bcb6c00 2023-09-23T03:28:29.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.933+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 31 ==== osd_op_reply(268 100000001fe.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:29.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.933+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628286.0 2023-09-23T03:28:29.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.933+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:269 18.3b 18:deccbd37:::100000001fe.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bce8e00 con 0x55d53bcb6c00 2023-09-23T03:28:29.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.937+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 32 ==== osd_op_reply(269 100000001fe.00000000 [call] v128'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:29.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.937+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:270 18.3b 18:ded14b84::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bce8700 con 0x55d53bcb6c00 2023-09-23T03:28:29.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.938+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 33 ==== osd_op_reply(270 [pgnls start_epoch 128 out=81b] v128'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:29.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.938+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:271 18.3b 18:ded14b84:::1000000024b.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bce9180 con 0x55d53bcb6c00 2023-09-23T03:28:29.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.939+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 34 ==== osd_op_reply(271 1000000024b.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:29.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.939+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628363.0 2023-09-23T03:28:29.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.939+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:272 18.3b 18:ded14b84:::1000000024b.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bce9500 con 0x55d53bcb6c00 2023-09-23T03:28:29.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.943+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 35 ==== osd_op_reply(272 1000000024b.00000000 [call] v128'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:29.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.943+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:273 18.7 18:e0000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bce9880 con 0x55d53bb9bc00 2023-09-23T03:28:29.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.944+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.2 v2:172.21.15.154:6822/2884407243 18 ==== osd_op_reply(273 [pgnls start_epoch 128 out=101b] v126'12 uv12 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9bc00 2023-09-23T03:28:29.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.944+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:274 18.7 18:e15a914f:::10000000207.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bce9c00 con 0x55d53bb9bc00 2023-09-23T03:28:29.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.945+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.2 v2:172.21.15.154:6822/2884407243 19 ==== osd_op_reply(274 10000000207.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9bc00 2023-09-23T03:28:29.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.945+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628295.0 2023-09-23T03:28:29.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.945+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:275 18.7 18:e15a914f:::10000000207.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bce8380 con 0x55d53bb9bc00 2023-09-23T03:28:29.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.949+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.2 v2:172.21.15.154:6822/2884407243 20 ==== osd_op_reply(275 10000000207.00000000 [call] v128'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9bc00 2023-09-23T03:28:29.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.949+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:276 18.7 18:e2e07930::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0700 con 0x55d53bb9bc00 2023-09-23T03:28:29.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.950+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.2 v2:172.21.15.154:6822/2884407243 21 ==== osd_op_reply(276 [pgnls start_epoch 128 out=101b] v128'13 uv13 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9bc00 2023-09-23T03:28:29.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.951+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:277 18.7 18:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0a80 con 0x55d53bb9bc00 2023-09-23T03:28:29.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.951+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.2 v2:172.21.15.154:6822/2884407243 22 ==== osd_op_reply(277 10000000001.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9bc00 2023-09-23T03:28:29.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.951+0000 7f18314125c0 10 datascan.operator(): handling object 1099511627777.0 2023-09-23T03:28:29.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.951+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:278 18.7 18:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0e00 con 0x55d53bb9bc00 2023-09-23T03:28:29.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.956+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.2 v2:172.21.15.154:6822/2884407243 23 ==== osd_op_reply(278 10000000001.00000000 [call] v128'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9bc00 2023-09-23T03:28:29.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.956+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:279 18.7 18:e37752f2::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1880 con 0x55d53bb9bc00 2023-09-23T03:28:29.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.957+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.2 v2:172.21.15.154:6822/2884407243 24 ==== osd_op_reply(279 [pgnls start_epoch 128 out=81b] v128'14 uv14 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9bc00 2023-09-23T03:28:29.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.957+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:280 18.7 18:e37752f2:::1000000021f.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1c00 con 0x55d53bb9bc00 2023-09-23T03:28:29.958 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.958+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.2 v2:172.21.15.154:6822/2884407243 25 ==== osd_op_reply(280 1000000021f.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9bc00 2023-09-23T03:28:29.958 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.958+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628319.0 2023-09-23T03:28:29.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.958+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:281 18.7 18:e37752f2:::1000000021f.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0380 con 0x55d53bb9bc00 2023-09-23T03:28:29.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.963+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 <== osd.2 v2:172.21.15.154:6822/2884407243 26 ==== osd_op_reply(281 1000000021f.00000000 [call] v128'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2fc0 con 0x55d53bb9bc00 2023-09-23T03:28:29.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.963+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:282 18.27 18:e4000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1180 con 0x55d53bb9b000 2023-09-23T03:28:29.964 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.964+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 41 ==== osd_op_reply(282 [pgnls start_epoch 128 out=101b] v126'12 uv12 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.964+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:283 18.27 18:e40c9c52:::10000000210.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1500 con 0x55d53bb9b000 2023-09-23T03:28:29.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.966+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 42 ==== osd_op_reply(283 10000000210.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.966+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628304.0 2023-09-23T03:28:29.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.966+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:284 18.27 18:e40c9c52:::10000000210.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbca80 con 0x55d53bb9b000 2023-09-23T03:28:29.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.970+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 43 ==== osd_op_reply(284 10000000210.00000000 [call] v128'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.970+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:285 18.27 18:e5671758::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbc000 con 0x55d53bb9b000 2023-09-23T03:28:29.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.971+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 44 ==== osd_op_reply(285 [pgnls start_epoch 128 out=101b] v128'13 uv13 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.971+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:286 18.27 18:e5671758:::1000000024f.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbc380 con 0x55d53bb9b000 2023-09-23T03:28:29.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.972+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 45 ==== osd_op_reply(286 1000000024f.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.972+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628367.0 2023-09-23T03:28:29.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.972+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:287 18.27 18:e5671758:::1000000024f.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbc700 con 0x55d53bb9b000 2023-09-23T03:28:29.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.976+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 46 ==== osd_op_reply(287 1000000024f.00000000 [call] v128'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.977+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:288 18.27 18:e5993619::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0000 con 0x55d53bb9b000 2023-09-23T03:28:29.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.978+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 47 ==== osd_op_reply(288 [pgnls start_epoch 128 out=101b] v128'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.978+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:289 18.27 18:e5993619:::10000000208.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1180 con 0x55d53bb9b000 2023-09-23T03:28:29.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.979+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 48 ==== osd_op_reply(289 10000000208.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.979+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628296.0 2023-09-23T03:28:29.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.979+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:290 18.27 18:e5993619:::10000000208.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1500 con 0x55d53bb9b000 2023-09-23T03:28:29.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.984+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 49 ==== osd_op_reply(290 10000000208.00000000 [call] v128'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.984+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:291 18.27 18:e5a5749e::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0700 con 0x55d53bb9b000 2023-09-23T03:28:29.985 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.985+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 50 ==== osd_op_reply(291 [pgnls start_epoch 128 out=101b] v128'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.986 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.985+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:292 18.27 18:e5a5749e:::1000000021a.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0a80 con 0x55d53bb9b000 2023-09-23T03:28:29.986 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.986+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 51 ==== osd_op_reply(292 1000000021a.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.986+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628314.0 2023-09-23T03:28:29.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.986+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:293 18.27 18:e5a5749e:::1000000021a.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0e00 con 0x55d53bb9b000 2023-09-23T03:28:29.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.990+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 52 ==== osd_op_reply(293 1000000021a.00000000 [call] v128'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.990+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:294 18.27 18:e692fc64::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1880 con 0x55d53bb9b000 2023-09-23T03:28:29.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.991+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 53 ==== osd_op_reply(294 [pgnls start_epoch 128 out=81b] v128'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.991+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:295 18.27 18:e692fc64:::1000000021b.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbd500 con 0x55d53bb9b000 2023-09-23T03:28:29.992 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.992+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 54 ==== osd_op_reply(295 1000000021b.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.992 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.992+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628315.0 2023-09-23T03:28:29.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.992+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:296 18.27 18:e692fc64:::1000000021b.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbca80 con 0x55d53bb9b000 2023-09-23T03:28:29.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.996+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 55 ==== osd_op_reply(296 1000000021b.00000000 [call] v128'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.996+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:297 18.17 18:e8000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbc000 con 0x55d53bb9b000 2023-09-23T03:28:29.997 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.997+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 56 ==== osd_op_reply(297 [pgnls start_epoch 128 out=101b] v126'13 uv13 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.997+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:298 18.17 18:eace0b58:::10000000215.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbc380 con 0x55d53bb9b000 2023-09-23T03:28:29.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.998+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 57 ==== osd_op_reply(298 10000000215.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:29.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.998+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628309.0 2023-09-23T03:28:29.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:29.998+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:299 18.17 18:eace0b58:::10000000215.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbce00 con 0x55d53bb9b000 2023-09-23T03:28:30.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.002+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 58 ==== osd_op_reply(299 10000000215.00000000 [call] v128'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:30.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.002+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:300 18.17 18:eb1b33a4::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbd180 con 0x55d53bb9b000 2023-09-23T03:28:30.003 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.003+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 59 ==== osd_op_reply(300 [pgnls start_epoch 128 out=81b] v128'14 uv14 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:30.003 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.003+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:301 18.17 18:eb1b33a4:::10000000253.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53b871880 con 0x55d53bb9b000 2023-09-23T03:28:30.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.004+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 60 ==== osd_op_reply(301 10000000253.00000000 [stat out=16b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:30.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.004+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628371.0 2023-09-23T03:28:30.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.004+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:302 18.17 18:eb1b33a4:::10000000253.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53b871c00 con 0x55d53bb9b000 2023-09-23T03:28:30.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.009+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 61 ==== osd_op_reply(302 10000000253.00000000 [call] v128'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:30.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.009+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:303 18.37 18:ec000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53c620380 con 0x55d53bcb6400 2023-09-23T03:28:30.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.010+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 36 ==== osd_op_reply(303 [pgnls start_epoch 128 out=49b] v126'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:30.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.010+0000 7f182a0d6640 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:304 18.f 18:f0000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53c620700 con 0x55d53bb9b000 2023-09-23T03:28:30.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.011+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 62 ==== osd_op_reply(304 [pgnls start_epoch 128 out=81b] v126'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:30.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.012+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:305 18.f 18:f25dd5df:::1000000022e.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53c621c00 con 0x55d53bb9b000 2023-09-23T03:28:30.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.013+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 63 ==== osd_op_reply(305 1000000022e.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:30.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.013+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628334.0 2023-09-23T03:28:30.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.013+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:306 18.f 18:f25dd5df:::1000000022e.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53c620700 con 0x55d53bb9b000 2023-09-23T03:28:30.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.017+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.5 v2:172.21.15.190:6800/882536885 64 ==== osd_op_reply(306 1000000022e.00000000 [call] v128'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bb9b000 2023-09-23T03:28:30.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.017+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:307 18.2f 18:f4000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53b871880 con 0x55d53bb9b800 2023-09-23T03:28:30.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.018+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.1 v2:172.21.15.154:6816/1267710005 27 ==== osd_op_reply(307 [pgnls start_epoch 128 out=49b] v126'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bb9b800 2023-09-23T03:28:30.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.018+0000 7f182a0d6640 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:308 18.1f 18:f8000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53c620a80 con 0x55d53bcb6400 2023-09-23T03:28:30.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.019+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 37 ==== osd_op_reply(308 [pgnls start_epoch 128 out=101b] v126'10 uv10 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:30.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.019+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:309 18.1f 18:f87ea8f1:::10000000238.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53b871c00 con 0x55d53bcb6400 2023-09-23T03:28:30.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.020+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 38 ==== osd_op_reply(309 10000000238.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:30.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.020+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628344.0 2023-09-23T03:28:30.021 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.020+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:310 18.1f 18:f87ea8f1:::10000000238.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbc700 con 0x55d53bcb6400 2023-09-23T03:28:30.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.024+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 39 ==== osd_op_reply(310 10000000238.00000000 [call] v128'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:30.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.024+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:311 18.1f 18:f8e2d5cc::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbd500 con 0x55d53bcb6400 2023-09-23T03:28:30.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.025+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 40 ==== osd_op_reply(311 [pgnls start_epoch 128 out=81b] v128'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:30.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.025+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:312 18.1f 18:f8e2d5cc:::10000000214.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbca80 con 0x55d53bcb6400 2023-09-23T03:28:30.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.026+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 41 ==== osd_op_reply(312 10000000214.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:30.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.026+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628308.0 2023-09-23T03:28:30.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.026+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:313 18.1f 18:f8e2d5cc:::10000000214.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbc000 con 0x55d53bcb6400 2023-09-23T03:28:30.030 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.030+0000 7f182e8df640 1 -- 172.21.15.154:0/10744971 <== osd.7 v2:172.21.15.190:6824/3460136272 42 ==== osd_op_reply(313 10000000214.00000000 [call] v128'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2b40 con 0x55d53bcb6400 2023-09-23T03:28:30.030 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.030+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:314 18.3f 18:fc000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbc380 con 0x55d53bcb6c00 2023-09-23T03:28:30.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.031+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 36 ==== osd_op_reply(314 [pgnls start_epoch 128 out=101b] v126'12 uv12 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:30.032 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.031+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:315 18.3f 18:fd83cc12:::10000000232.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbce00 con 0x55d53bcb6c00 2023-09-23T03:28:30.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.032+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 37 ==== osd_op_reply(315 10000000232.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:30.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.032+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628338.0 2023-09-23T03:28:30.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.032+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:316 18.3f 18:fd83cc12:::10000000232.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bcbd180 con 0x55d53bcb6c00 2023-09-23T03:28:30.036 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.036+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 38 ==== osd_op_reply(316 10000000232.00000000 [call] v128'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:30.036 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.036+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:317 18.3f 18:feb1c803::::head [pgnls start_epoch 128 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca0000 con 0x55d53bcb6c00 2023-09-23T03:28:30.037 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.037+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 39 ==== osd_op_reply(317 [pgnls start_epoch 128 out=81b] v128'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:30.037 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.037+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:318 18.3f 18:feb1c803:::100000001fa.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1180 con 0x55d53bcb6c00 2023-09-23T03:28:30.038 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.038+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 40 ==== osd_op_reply(318 100000001fa.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:30.038 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.038+0000 7f18314125c0 10 datascan.operator(): handling object 1099511628282.0 2023-09-23T03:28:30.038 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.038+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:319 18.3f 18:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55d53bca1500 con 0x55d53bcb6c00 2023-09-23T03:28:30.042 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.041+0000 7f182f8e1640 1 -- 172.21.15.154:0/10744971 <== osd.4 v2:172.21.15.190:6808/717302641 41 ==== osd_op_reply(319 100000001fa.00000000 [call] v128'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d53baf2d80 con 0x55d53bcb6c00 2023-09-23T03:28:30.042 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.042+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55d53bb9ac00 msgr2=0x55d53b882680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:30.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.042+0000 7f18314125c0 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55d53bb9ac00 0x55d53b882680 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:30.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.042+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55d53bb9b800 msgr2=0x55d53b883700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:30.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.042+0000 7f18314125c0 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55d53bb9b800 0x55d53b883700 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:30.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.042+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55d53bb9bc00 msgr2=0x55d53bc88b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:30.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.042+0000 7f18314125c0 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55d53bb9bc00 0x55d53bc88b00 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:30.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.042+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55d53bcb6000 msgr2=0x55d53bc88580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:30.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.042+0000 7f18314125c0 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55d53bcb6000 0x55d53bc88580 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:30.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.042+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55d53bcb6c00 msgr2=0x55d53bc89080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:30.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.042+0000 7f18314125c0 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55d53bcb6c00 0x55d53bc89080 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:30.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.042+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55d53bb9b000 msgr2=0x55d53b882c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:30.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.042+0000 7f18314125c0 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55d53bb9b000 0x55d53b882c00 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:30.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.042+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 reap_dead start 2023-09-23T03:28:30.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.042+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55d53bb9b400 msgr2=0x55d53b883180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:30.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.042+0000 7f18314125c0 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55d53bb9b400 0x55d53b883180 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:30.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.042+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55d53bcb6400 msgr2=0x55d53bc8b180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:30.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.042+0000 7f18314125c0 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55d53bcb6400 0x55d53bc8b180 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:30.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.042+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55d53bb9a800 msgr2=0x55d53bc88000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:30.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.042+0000 7f18314125c0 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55d53bb9a800 0x55d53bc88000 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x55d53bb225a0 tx=0x55d53bb00180 comp rx=0 tx=0).stop 2023-09-23T03:28:30.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.042+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d53bb0f000 msgr2=0x55d53b880b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:30.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.042+0000 7f18314125c0 1 --2- 172.21.15.154:0/10744971 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d53bb0f000 0x55d53b880b00 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x55d53bb23410 tx=0x55d53bb30cc0 comp rx=0 tx=0).stop 2023-09-23T03:28:30.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.043+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 reap_dead start 2023-09-23T03:28:30.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.043+0000 7f182f0e0640 1 -- 172.21.15.154:0/10744971 reap_dead start 2023-09-23T03:28:30.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.043+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 shutdown_connections 2023-09-23T03:28:30.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.043+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 >> 172.21.15.154:0/10744971 conn(0x55d53bb0ec00 msgr2=0x55d53c591000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:30.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.043+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 shutdown_connections 2023-09-23T03:28:30.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.043+0000 7f18314125c0 1 -- 172.21.15.154:0/10744971 wait complete. 2023-09-23T03:28:30.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.044+0000 7f18314125c0 1 -- 172.21.15.154:0/2439774686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d53b817c00 msgr2=0x55d53b881080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:30.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.044+0000 7f18314125c0 1 --2- 172.21.15.154:0/2439774686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d53b817c00 0x55d53b881080 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x55d53c61b3e0 tx=0x55d53bb00480 comp rx=0 tx=0).stop 2023-09-23T03:28:30.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.045+0000 7f18314125c0 1 -- 172.21.15.154:0/2439774686 shutdown_connections 2023-09-23T03:28:30.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.045+0000 7f18314125c0 1 --2- 172.21.15.154:0/2439774686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d53b817c00 0x55d53b881080 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:30.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.045+0000 7f18314125c0 1 --2- 172.21.15.154:0/2439774686 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55d53b817800 0x55d53b881600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:30.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.045+0000 7f18314125c0 1 --2- 172.21.15.154:0/2439774686 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55d53b816c00 0x55d53b881b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:30.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.045+0000 7f18314125c0 1 -- 172.21.15.154:0/2439774686 >> 172.21.15.154:0/2439774686 conn(0x55d53b817000 msgr2=0x55d53c590800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:30.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.045+0000 7f18314125c0 1 -- 172.21.15.154:0/2439774686 shutdown_connections 2023-09-23T03:28:30.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.045+0000 7f18314125c0 1 -- 172.21.15.154:0/2439774686 wait complete. 2023-09-23T03:28:30.057 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents'] in time 0:00:00.737644, result: 2023-09-23T03:28:30.057 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes 2023-09-23T03:28:30.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.206+0000 7ff11803a5c0 1 -- start start 2023-09-23T03:28:30.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.207+0000 7ff11803a5c0 1 Processor -- start 2023-09-23T03:28:30.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.207+0000 7ff11803a5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56435fa58c00 0x56435fac2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:30.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.207+0000 7ff11803a5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56435fa59800 0x56435fac2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:30.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.207+0000 7ff11803a5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56435fa59c00 0x56435fac3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:30.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.208+0000 7ff11803a5c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x56435faa2000 con 0x56435fa58c00 2023-09-23T03:28:30.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.208+0000 7ff11803a5c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56435faa3180 con 0x56435fa59c00 2023-09-23T03:28:30.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.208+0000 7ff11803a5c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56435faa2140 con 0x56435fa59800 2023-09-23T03:28:30.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.208+0000 7ff116068640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56435fa58c00 0x56435fac2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:30.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.208+0000 7ff116068640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56435fa58c00 0x56435fac2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53788/0 (socket says 172.21.15.154:53788) 2023-09-23T03:28:30.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.208+0000 7ff116068640 1 -- 172.21.15.154:0/291424263 learned_addr learned my addr 172.21.15.154:0/291424263 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:30.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.208+0000 7ff116869640 1 --2- 172.21.15.154:0/291424263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56435fa59c00 0x56435fac3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:30.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.208+0000 7ff115867640 1 --2- 172.21.15.154:0/291424263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56435fa59800 0x56435fac2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:30.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.208+0000 7ff116068640 1 -- 172.21.15.154:0/291424263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56435fa59800 msgr2=0x56435fac2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:30.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.208+0000 7ff116068640 1 --2- 172.21.15.154:0/291424263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56435fa59800 0x56435fac2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:30.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.208+0000 7ff116068640 1 -- 172.21.15.154:0/291424263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56435fa59c00 msgr2=0x56435fac3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:30.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.208+0000 7ff116068640 1 --2- 172.21.15.154:0/291424263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56435fa59c00 0x56435fac3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:30.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.208+0000 7ff116068640 1 -- 172.21.15.154:0/291424263 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56435fa8cb60 con 0x56435fa58c00 2023-09-23T03:28:30.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.208+0000 7ff115867640 1 --2- 172.21.15.154:0/291424263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56435fa59800 0x56435fac2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:30.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.209+0000 7ff116068640 1 --2- 172.21.15.154:0/291424263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56435fa58c00 0x56435fac2580 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x564360867860 tx=0x56435fb38600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca269fd233123de server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:30.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.209+0000 7ff113863640 1 -- 172.21.15.154:0/291424263 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56435faa1600 con 0x56435fa58c00 2023-09-23T03:28:30.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.209+0000 7ff113863640 1 -- 172.21.15.154:0/291424263 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56435fa7f800 con 0x56435fa58c00 2023-09-23T03:28:30.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.209+0000 7ff113863640 1 -- 172.21.15.154:0/291424263 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56435faa11e0 con 0x56435fa58c00 2023-09-23T03:28:30.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.209+0000 7ff11803a5c0 1 -- 172.21.15.154:0/291424263 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56435fa8c000 con 0x56435fa58c00 2023-09-23T03:28:30.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.209+0000 7ff11803a5c0 1 -- 172.21.15.154:0/291424263 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x56435fa8cea0 con 0x56435fa58c00 2023-09-23T03:28:30.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.209+0000 7ff11803a5c0 4 waiting for MDS map... 2023-09-23T03:28:30.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.209+0000 7ff113863640 1 -- 172.21.15.154:0/291424263 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(128..128 src has 1..128) v4 ==== 6201+0+0 (secure 0 0 0) 0x56435fb18c40 con 0x56435fa58c00 2023-09-23T03:28:30.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.209+0000 7ff113863640 1 -- 172.21.15.154:0/291424263 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 115) v1 ==== 2723+0+0 (secure 0 0 0) 0x56435faab200 con 0x56435fa58c00 2023-09-23T03:28:30.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.209+0000 7ff11803a5c0 4 Got MDS map 115 2023-09-23T03:28:30.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.210+0000 7ff11803a5c0 4 datascan.main: Using metadata pool output 2023-09-23T03:28:30.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.210+0000 7ff11803a5c0 4 datascan.main: connecting to RADOS... 2023-09-23T03:28:30.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.210+0000 7ff11803a5c0 1 Processor -- start 2023-09-23T03:28:30.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.211+0000 7ff11803a5c0 1 -- start start 2023-09-23T03:28:30.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.211+0000 7ff11803a5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56435fd4f000 0x56435fac3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:30.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.211+0000 7ff11803a5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56435fd4f400 0x56435fac3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:30.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.211+0000 7ff11803a5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56435fd4f800 0x56435fac4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:30.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.211+0000 7ff11803a5c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x56435faa2a00 con 0x56435fd4f800 2023-09-23T03:28:30.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.211+0000 7ff11803a5c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56435fb27180 con 0x56435fd4f400 2023-09-23T03:28:30.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.211+0000 7ff11803a5c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56435fb26dc0 con 0x56435fd4f000 2023-09-23T03:28:30.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.212+0000 7ff115867640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56435fd4f800 0x56435fac4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:30.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.212+0000 7ff115867640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56435fd4f800 0x56435fac4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53790/0 (socket says 172.21.15.154:53790) 2023-09-23T03:28:30.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.212+0000 7ff115867640 1 -- 172.21.15.154:0/1301231575 learned_addr learned my addr 172.21.15.154:0/1301231575 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:30.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.212+0000 7ff116869640 1 --2- 172.21.15.154:0/1301231575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56435fd4f000 0x56435fac3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:30.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.212+0000 7ff116068640 1 --2- 172.21.15.154:0/1301231575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56435fd4f400 0x56435fac3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:30.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.212+0000 7ff115867640 1 -- 172.21.15.154:0/1301231575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56435fd4f000 msgr2=0x56435fac3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:30.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.212+0000 7ff115867640 1 --2- 172.21.15.154:0/1301231575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56435fd4f000 0x56435fac3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:30.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.212+0000 7ff115867640 1 -- 172.21.15.154:0/1301231575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56435fd4f400 msgr2=0x56435fac3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:30.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.212+0000 7ff115867640 1 --2- 172.21.15.154:0/1301231575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56435fd4f400 0x56435fac3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:30.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.212+0000 7ff115867640 1 -- 172.21.15.154:0/1301231575 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56435fa8d520 con 0x56435fd4f800 2023-09-23T03:28:30.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.212+0000 7ff116869640 1 --2- 172.21.15.154:0/1301231575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56435fd4f000 0x56435fac3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:30.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.212+0000 7ff116068640 1 --2- 172.21.15.154:0/1301231575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56435fd4f400 0x56435fac3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:30.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.212+0000 7ff115867640 1 --2- 172.21.15.154:0/1301231575 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56435fd4f800 0x56435fac4100 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x56435fd62e70 tx=0x56435fd6cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a40e335cb06ac31 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:30.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.213+0000 7ff11185f640 1 -- 172.21.15.154:0/1301231575 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56435faa1600 con 0x56435fd4f800 2023-09-23T03:28:30.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.213+0000 7ff11185f640 1 -- 172.21.15.154:0/1301231575 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56436083ac00 con 0x56435fd4f800 2023-09-23T03:28:30.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.213+0000 7ff11185f640 1 -- 172.21.15.154:0/1301231575 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56435faa11e0 con 0x56435fd4f800 2023-09-23T03:28:30.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.213+0000 7ff11803a5c0 1 -- 172.21.15.154:0/1301231575 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56435fd4f800 msgr2=0x56435fac4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:30.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.213+0000 7ff11803a5c0 1 --2- 172.21.15.154:0/1301231575 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56435fd4f800 0x56435fac4100 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x56435fd62e70 tx=0x56435fd6cc00 comp rx=0 tx=0).stop 2023-09-23T03:28:30.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.213+0000 7ff11803a5c0 1 -- 172.21.15.154:0/1301231575 shutdown_connections 2023-09-23T03:28:30.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.213+0000 7ff11803a5c0 1 --2- 172.21.15.154:0/1301231575 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56435fd4f800 0x56435fac4100 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:30.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.213+0000 7ff11803a5c0 1 --2- 172.21.15.154:0/1301231575 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56435fd4f400 0x56435fac3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:30.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.213+0000 7ff11803a5c0 1 --2- 172.21.15.154:0/1301231575 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56435fd4f000 0x56435fac3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:30.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.213+0000 7ff11803a5c0 1 -- 172.21.15.154:0/1301231575 >> 172.21.15.154:0/1301231575 conn(0x56435fd4ec00 msgr2=0x5643607d1800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:30.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.213+0000 7ff11803a5c0 1 -- 172.21.15.154:0/1301231575 shutdown_connections 2023-09-23T03:28:30.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.213+0000 7ff11803a5c0 1 -- 172.21.15.154:0/1301231575 wait complete. 2023-09-23T03:28:30.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.214+0000 7ff11803a5c0 1 Processor -- start 2023-09-23T03:28:30.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.214+0000 7ff11803a5c0 1 -- start start 2023-09-23T03:28:30.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.215+0000 7ff11803a5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56435fd4f000 0x56435fac5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:30.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.215+0000 7ff11803a5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56435fd4f400 0x56435fac4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:30.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.215+0000 7ff11803a5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56435fd4f800 0x56435fac4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:30.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.215+0000 7ff11803a5c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x56435faa3180 con 0x56435fd4f400 2023-09-23T03:28:30.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.215+0000 7ff11803a5c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x56435faa2000 con 0x56435fd4f800 2023-09-23T03:28:30.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.215+0000 7ff11803a5c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x56435faa2b40 con 0x56435fd4f000 2023-09-23T03:28:30.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.215+0000 7ff116068640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56435fd4f400 0x56435fac4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:30.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.215+0000 7ff116068640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56435fd4f400 0x56435fac4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53806/0 (socket says 172.21.15.154:53806) 2023-09-23T03:28:30.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.215+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 learned_addr learned my addr 172.21.15.154:0/405849260 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:30.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.216+0000 7ff115867640 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56435fd4f800 0x56435fac4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:30.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.216+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56435fd4f000 msgr2=0x56435fac5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:30.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.216+0000 7ff116068640 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56435fd4f000 0x56435fac5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:30.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.216+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56435fd4f800 msgr2=0x56435fac4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:30.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.216+0000 7ff116068640 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56435fd4f800 0x56435fac4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:30.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.216+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56435fa8c340 con 0x56435fd4f400 2023-09-23T03:28:30.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.216+0000 7ff116068640 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56435fd4f400 0x56435fac4c00 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x56435fd62cf0 tx=0x56435fb38060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:30.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.216+0000 7ff11005c640 1 -- 172.21.15.154:0/405849260 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56435faa1600 con 0x56435fd4f400 2023-09-23T03:28:30.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.216+0000 7ff11005c640 1 -- 172.21.15.154:0/405849260 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56436083aa80 con 0x56435fd4f400 2023-09-23T03:28:30.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.216+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56435fa8cb60 con 0x56435fd4f400 2023-09-23T03:28:30.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.216+0000 7ff11005c640 1 -- 172.21.15.154:0/405849260 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56435faa11e0 con 0x56435fd4f400 2023-09-23T03:28:30.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.216+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56435fa8c000 con 0x56435fd4f400 2023-09-23T03:28:30.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.216+0000 7ff11803a5c0 4 datascan.init: resolving metadata pool 17 2023-09-23T03:28:30.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.218+0000 7ff11005c640 1 -- 172.21.15.154:0/405849260 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x56435fa8b340 con 0x56435fd4f400 2023-09-23T03:28:30.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.219+0000 7ff11005c640 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x56435fdcb800 0x56435fac5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:30.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.219+0000 7ff11005c640 1 -- 172.21.15.154:0/405849260 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6201+0+0 (secure 0 0 0) 0x56435fb18a80 con 0x56435fd4f400 2023-09-23T03:28:30.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.219+0000 7ff11803a5c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-09-23T03:28:30.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.219+0000 7ff11803a5c0 4 datascan.main: data pool 'cephfs_data' has ID 18 2023-09-23T03:28:30.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.219+0000 7ff11803a5c0 4 datascan.main: opening data pool 'cephfs_data' 2023-09-23T03:28:30.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.219+0000 7ff11803a5c0 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x56435fdcbc00 0x56435fac4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:30.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.219+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:1 17.14 17:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fab3880 con 0x56435fdcbc00 2023-09-23T03:28:30.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.219+0000 7ff116869640 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x56435fdcb800 0x56435fac5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:30.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.219+0000 7ff115867640 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x56435fdcbc00 0x56435fac4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:30.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.220+0000 7ff116869640 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x56435fdcb800 0x56435fac5700 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x5643607d4420 tx=0x56435fb38420 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:30.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.220+0000 7ff115867640 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x56435fdcbc00 0x56435fac4100 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:30.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.220+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv19 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.220+0000 7ff11803a5c0 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x56435febc000 0x56435fac3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:30.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.220+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:2 17.1f 17:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fab3c00 con 0x56435febc000 2023-09-23T03:28:30.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.221+0000 7ff116869640 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x56435febc000 0x56435fac3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:30.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.222+0000 7ff116869640 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x56435febc000 0x56435fac3b80 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:30.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.223+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.223+0000 7ff11803a5c0 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x56435febc400 0x56435fac3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:30.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.223+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:3 18.0 18:00000000::::head [pgnls-filter start_epoch 128 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca000 con 0x56435febc400 2023-09-23T03:28:30.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.223+0000 7ff116068640 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x56435febc400 0x56435fac3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:30.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.224+0000 7ff116068640 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x56435febc400 0x56435fac3600 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:30.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.224+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.0 v2:172.21.15.154:6800/2920938138 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 128 out=81b] v128'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3efc0 con 0x56435febc400 2023-09-23T03:28:30.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.224+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:4 18.0 18:00000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca380 con 0x56435febc400 2023-09-23T03:28:30.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.224+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.0 v2:172.21.15.154:6800/2920938138 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 128 out=81b] v128'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3efc0 con 0x56435febc400 2023-09-23T03:28:30.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.224+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000206.00000000 2023-09-23T03:28:30.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.224+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000206.0 2023-09-23T03:28:30.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.224+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:5 18.0 18:012cdbc7:::10000000206.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca700 con 0x56435febc400 2023-09-23T03:28:30.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.225+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.0 v2:172.21.15.154:6800/2920938138 3 ==== osd_op_reply(5 10000000206.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3efc0 con 0x56435febc400 2023-09-23T03:28:30.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.225+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000206 2023-09-23T03:28:30.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.225+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/17 2023-09-23T03:28:30.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.225+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=17 2023-09-23T03:28:30.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.225+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:6 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca000 con 0x56435fdcbc00 2023-09-23T03:28:30.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.225+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 2 ==== osd_op_reply(6 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.225+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.225+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:7 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca380 con 0x56435febc000 2023-09-23T03:28:30.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.226+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 2 ==== osd_op_reply(7 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.226+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.226+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.226+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:30.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.226+0000 7ff11803a5c0 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x56435febcc00 0x56435feda000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:30.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.227+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:8 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecaa80 con 0x56435febcc00 2023-09-23T03:28:30.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.227+0000 7ff115867640 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x56435febcc00 0x56435feda000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:30.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.228+0000 7ff115867640 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x56435febcc00 0x56435feda000 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:30.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.229+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 1 ==== osd_op_reply(8 10000000000.01800000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.229+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-09-23T03:28:30.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.229+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:9 17.1b 17:dcc776e0:::10000000000.01800000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecae00 con 0x56435febcc00 2023-09-23T03:28:30.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.233+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 2 ==== osd_op_reply(9 10000000000.01800000 [create,omap-set-header] v128'6 uv6 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.233+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x10000000000 2023-09-23T03:28:30.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.233+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:10 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb180 con 0x56435febcc00 2023-09-23T03:28:30.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.234+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 3 ==== osd_op_reply(10 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.234+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=17 frag=1*, r=0 2023-09-23T03:28:30.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.234+0000 7ff11803a5c0 20 datascan.read_dentry: 17_head not found in result 2023-09-23T03:28:30.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.234+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000206 at 0x0x10000000000/17 with size=2 bytes 2023-09-23T03:28:30.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.234+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:11 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecaa80 con 0x56435febcc00 2023-09-23T03:28:30.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.237+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 4 ==== osd_op_reply(11 10000000000.01800000 [omap-set-vals] v128'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.237+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/17 pointing to 0x0x10000000206 2023-09-23T03:28:30.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.237+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x1/subdir 2023-09-23T03:28:30.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.237+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x1 target_dname=subdir 2023-09-23T03:28:30.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.237+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:12 17.1f 17:ff5b34d6:::1.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecae00 con 0x56435febc000 2023-09-23T03:28:30.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.238+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 3 ==== osd_op_reply(12 1.00000000 [getxattr (30) out=30b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+30 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.238+0000 7ff11803a5c0 4 datascan.get_frag_of: No backtrace on '1.00000000': cannot determine fragtree 2023-09-23T03:28:30.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.238+0000 7ff11803a5c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x1/subdir, will insert to root 2023-09-23T03:28:30.238 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.238+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:13 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb180 con 0x56435febc000 2023-09-23T03:28:30.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.239+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 4 ==== osd_op_reply(13 1.00000000 [omap-get-header out=274b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.239+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 6 2023-09-23T03:28:30.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.239+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x1 * 2023-09-23T03:28:30.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.239+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:14 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca380 con 0x56435febc000 2023-09-23T03:28:30.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.240+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 5 ==== osd_op_reply(14 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.240+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.240+0000 7ff11803a5c0 20 datascan.inject_with_backtrace: Dentry 0x0x1/subdir already exists and points to me 2023-09-23T03:28:30.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.240+0000 7ff11803a5c0 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x56435febd000 0x56435feda580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:30.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.240+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:15 18.20 18:04000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fab3c00 con 0x56435febd000 2023-09-23T03:28:30.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.241+0000 7ff116869640 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x56435febd000 0x56435feda580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:30.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.242+0000 7ff116869640 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x56435febd000 0x56435feda580 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:30.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.243+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 1 ==== osd_op_reply(15 [pgnls-filter start_epoch 128 out=81b] v128'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:30.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.243+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 1000000020b.00000000 2023-09-23T03:28:30.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.243+0000 7ff11803a5c0 10 datascan.operator(): handling object 1000000020b.0 2023-09-23T03:28:30.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.243+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:16 18.20 18:05552577:::1000000020b.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fab3880 con 0x56435febd000 2023-09-23T03:28:30.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.244+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 2 ==== osd_op_reply(16 1000000020b.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:30.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.244+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000020b 2023-09-23T03:28:30.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.244+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/22 2023-09-23T03:28:30.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.244+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=22 2023-09-23T03:28:30.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.244+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:17 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb180 con 0x56435fdcbc00 2023-09-23T03:28:30.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.245+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 3 ==== osd_op_reply(17 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.245+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.245+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:18 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca380 con 0x56435febc000 2023-09-23T03:28:30.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.246+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 6 ==== osd_op_reply(18 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.246+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.246+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.246+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:30.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.246+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:19 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca000 con 0x56435febcc00 2023-09-23T03:28:30.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.247+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 5 ==== osd_op_reply(19 10000000000.01800000 [omap-get-header out=274b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.247+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 7 2023-09-23T03:28:30.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.247+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:30.248 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.247+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:20 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb500 con 0x56435febcc00 2023-09-23T03:28:30.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.248+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 6 ==== osd_op_reply(20 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.249+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=22 frag=1*, r=0 2023-09-23T03:28:30.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.249+0000 7ff11803a5c0 20 datascan.read_dentry: 22_head not found in result 2023-09-23T03:28:30.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.249+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000020b at 0x0x10000000000/22 with size=2 bytes 2023-09-23T03:28:30.249 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.249+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:21 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb880 con 0x56435febcc00 2023-09-23T03:28:30.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.252+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 7 ==== osd_op_reply(21 10000000000.01800000 [omap-set-vals] v128'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.252+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/22 pointing to 0x0x1000000020b 2023-09-23T03:28:30.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.252+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:22 18.10 18:08000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecbc00 con 0x56435febc000 2023-09-23T03:28:30.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.253+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 7 ==== osd_op_reply(22 [pgnls-filter start_epoch 128 out=101b] v128'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.253+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 1000000022a.00000000 2023-09-23T03:28:30.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.253+0000 7ff11803a5c0 10 datascan.operator(): handling object 1000000022a.0 2023-09-23T03:28:30.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.253+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:23 18.10 18:083d5826:::1000000022a.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecaa80 con 0x56435febc000 2023-09-23T03:28:30.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.254+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 8 ==== osd_op_reply(23 1000000022a.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.254+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000022a 2023-09-23T03:28:30.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.254+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/53 2023-09-23T03:28:30.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.254+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=53 2023-09-23T03:28:30.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.254+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:24 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca000 con 0x56435fdcbc00 2023-09-23T03:28:30.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.255+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 4 ==== osd_op_reply(24 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.255+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.255+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:25 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb500 con 0x56435febc000 2023-09-23T03:28:30.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.256+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 9 ==== osd_op_reply(25 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.256+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.256+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.256+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:30.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.256+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:26 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecae00 con 0x56435febc000 2023-09-23T03:28:30.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.257+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 10 ==== osd_op_reply(26 10000000000.01000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.257+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-09-23T03:28:30.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.257+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:27 17.5 17:a40620de:::10000000000.01000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca380 con 0x56435febc000 2023-09-23T03:28:30.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.260+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 11 ==== osd_op_reply(27 10000000000.01000000 [create,omap-set-header] v128'5 uv5 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.260+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x10000000000 2023-09-23T03:28:30.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.260+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:28 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fab3c00 con 0x56435febc000 2023-09-23T03:28:30.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.261+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 12 ==== osd_op_reply(28 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.261+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=53 frag=0*, r=0 2023-09-23T03:28:30.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.261+0000 7ff11803a5c0 20 datascan.read_dentry: 53_head not found in result 2023-09-23T03:28:30.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.261+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000022a at 0x0x10000000000/53 with size=2 bytes 2023-09-23T03:28:30.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.261+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:29 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fab3880 con 0x56435febc000 2023-09-23T03:28:30.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.264+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 13 ==== osd_op_reply(29 10000000000.01000000 [omap-set-vals] v128'6 uv6 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.264+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/53 pointing to 0x0x1000000022a 2023-09-23T03:28:30.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.264+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x1/subdir 2023-09-23T03:28:30.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.264+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x1 target_dname=subdir 2023-09-23T03:28:30.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.264+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:30 17.1f 17:ff5b34d6:::1.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecbc00 con 0x56435febc000 2023-09-23T03:28:30.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.265+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 14 ==== osd_op_reply(30 1.00000000 [getxattr (30) out=30b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+30 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.265+0000 7ff11803a5c0 4 datascan.get_frag_of: No backtrace on '1.00000000': cannot determine fragtree 2023-09-23T03:28:30.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.265+0000 7ff11803a5c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x1/subdir, will insert to root 2023-09-23T03:28:30.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.265+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:31 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb180 con 0x56435febc000 2023-09-23T03:28:30.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.266+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 15 ==== osd_op_reply(31 1.00000000 [omap-get-header out=274b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.266+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 6 2023-09-23T03:28:30.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.266+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x1 * 2023-09-23T03:28:30.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.266+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:32 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecaa80 con 0x56435febc000 2023-09-23T03:28:30.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.267+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 16 ==== osd_op_reply(32 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.268+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.268+0000 7ff11803a5c0 20 datascan.inject_with_backtrace: Dentry 0x0x1/subdir already exists and points to me 2023-09-23T03:28:30.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.268+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:33 18.10 18:08500202::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb500 con 0x56435febc000 2023-09-23T03:28:30.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.269+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 17 ==== osd_op_reply(33 [pgnls-filter start_epoch 128 out=101b] v128'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.269+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 1000000023e.00000000 2023-09-23T03:28:30.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.269+0000 7ff11803a5c0 10 datascan.operator(): handling object 1000000023e.0 2023-09-23T03:28:30.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.269+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:34 18.10 18:08500202:::1000000023e.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecae00 con 0x56435febc000 2023-09-23T03:28:30.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.270+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 18 ==== osd_op_reply(34 1000000023e.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.270 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.270+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000023e 2023-09-23T03:28:30.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.270+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/73 2023-09-23T03:28:30.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.270+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=73 2023-09-23T03:28:30.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.270+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:35 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecbc00 con 0x56435fdcbc00 2023-09-23T03:28:30.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.271+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 5 ==== osd_op_reply(35 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.271+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.271 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.271+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:36 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb180 con 0x56435febc000 2023-09-23T03:28:30.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.272+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 19 ==== osd_op_reply(36 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.272+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.272+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.272+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:30.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.272+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:37 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecaa80 con 0x56435febc000 2023-09-23T03:28:30.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.273+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 20 ==== osd_op_reply(37 10000000000.01000000 [omap-get-header out=274b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.273+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 6 2023-09-23T03:28:30.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.273+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:30.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.273+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:38 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fab3c00 con 0x56435febc000 2023-09-23T03:28:30.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.275+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 21 ==== osd_op_reply(38 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.275+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=73 frag=0*, r=0 2023-09-23T03:28:30.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.275+0000 7ff11803a5c0 20 datascan.read_dentry: 73_head not found in result 2023-09-23T03:28:30.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.275+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000023e at 0x0x10000000000/73 with size=2 bytes 2023-09-23T03:28:30.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.275+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:39 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fab3880 con 0x56435febc000 2023-09-23T03:28:30.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.278+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 22 ==== osd_op_reply(39 10000000000.01000000 [omap-set-vals] v128'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.278+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/73 pointing to 0x0x1000000023e 2023-09-23T03:28:30.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.278+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:40 18.10 18:08b99fea::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fab3c00 con 0x56435febc000 2023-09-23T03:28:30.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.279+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 23 ==== osd_op_reply(40 [pgnls-filter start_epoch 128 out=101b] v128'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.279+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 100000001fd.00000000 2023-09-23T03:28:30.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.279+0000 7ff11803a5c0 10 datascan.operator(): handling object 100000001fd.0 2023-09-23T03:28:30.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.280+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:41 18.10 18:08b99fea:::100000001fd.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fab3880 con 0x56435febc000 2023-09-23T03:28:30.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.281+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 24 ==== osd_op_reply(41 100000001fd.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.281+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001fd 2023-09-23T03:28:30.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.281+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/8 2023-09-23T03:28:30.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.281+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=8 2023-09-23T03:28:30.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.281+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:42 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca380 con 0x56435fdcbc00 2023-09-23T03:28:30.282 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.281+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 6 ==== osd_op_reply(42 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.281+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.281+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:43 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecae00 con 0x56435febc000 2023-09-23T03:28:30.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.283+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 25 ==== osd_op_reply(43 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.283+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.283+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.283+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:30.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.283+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:44 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb180 con 0x56435febc000 2023-09-23T03:28:30.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.284+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 26 ==== osd_op_reply(44 10000000000.01000000 [omap-get-header out=274b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.284+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 7 2023-09-23T03:28:30.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.284+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:30.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.284+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:45 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecaa80 con 0x56435febc000 2023-09-23T03:28:30.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.285+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 27 ==== osd_op_reply(45 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.285+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=8 frag=0*, r=0 2023-09-23T03:28:30.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.285+0000 7ff11803a5c0 20 datascan.read_dentry: 8_head not found in result 2023-09-23T03:28:30.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.285+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fd at 0x0x10000000000/8 with size=1 bytes 2023-09-23T03:28:30.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.285+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:46 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086e000 con 0x56435febc000 2023-09-23T03:28:30.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.289+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 28 ==== osd_op_reply(46 10000000000.01000000 [omap-set-vals] v128'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.289+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/8 pointing to 0x0x100000001fd 2023-09-23T03:28:30.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.289+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:47 18.10 18:08e6f8ca::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086e380 con 0x56435febc000 2023-09-23T03:28:30.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.290+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 29 ==== osd_op_reply(47 [pgnls-filter start_epoch 128 out=81b] v128'18 uv18 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.290+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000257.00000000 2023-09-23T03:28:30.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.290+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000257.0 2023-09-23T03:28:30.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.290+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:48 18.10 18:08e6f8ca:::10000000257.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086e700 con 0x56435febc000 2023-09-23T03:28:30.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.292+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 30 ==== osd_op_reply(48 10000000257.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv18 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.292+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000257 2023-09-23T03:28:30.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.292+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/98 2023-09-23T03:28:30.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.292+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=98 2023-09-23T03:28:30.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.292+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:49 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086ee00 con 0x56435fdcbc00 2023-09-23T03:28:30.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.292+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 7 ==== osd_op_reply(49 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.292+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.292+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:50 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086f180 con 0x56435febc000 2023-09-23T03:28:30.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.294+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 31 ==== osd_op_reply(50 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.294+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.294+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.294+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:30.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.294+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:51 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086f500 con 0x56435febcc00 2023-09-23T03:28:30.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.295+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 8 ==== osd_op_reply(51 10000000000.01800000 [omap-get-header out=274b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.295+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 8 2023-09-23T03:28:30.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.295+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:30.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.295+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:52 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086f880 con 0x56435febcc00 2023-09-23T03:28:30.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.296+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 9 ==== osd_op_reply(52 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.297+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=98 frag=1*, r=0 2023-09-23T03:28:30.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.297+0000 7ff11803a5c0 20 datascan.read_dentry: 98_head not found in result 2023-09-23T03:28:30.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.297+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000257 at 0x0x10000000000/98 with size=2 bytes 2023-09-23T03:28:30.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.297+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:53 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086fc00 con 0x56435febcc00 2023-09-23T03:28:30.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.300+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 10 ==== osd_op_reply(53 10000000000.01800000 [omap-set-vals] v128'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.300 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.300+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/98 pointing to 0x0x10000000257 2023-09-23T03:28:30.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.300+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:54 18.30 18:0c000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086ee00 con 0x56435febc400 2023-09-23T03:28:30.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.301+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.0 v2:172.21.15.154:6800/2920938138 4 ==== osd_op_reply(54 [pgnls-filter start_epoch 128 out=101b] v128'21 uv21 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3efc0 con 0x56435febc400 2023-09-23T03:28:30.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.301+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 1000000021d.00000000 2023-09-23T03:28:30.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.301+0000 7ff11803a5c0 10 datascan.operator(): handling object 1000000021d.0 2023-09-23T03:28:30.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.301+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:55 18.30 18:0c00cb85:::1000000021d.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086e000 con 0x56435febc400 2023-09-23T03:28:30.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.302+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.0 v2:172.21.15.154:6800/2920938138 5 ==== osd_op_reply(55 1000000021d.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv18 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3efc0 con 0x56435febc400 2023-09-23T03:28:30.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.302+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000021d 2023-09-23T03:28:30.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.302+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/40 2023-09-23T03:28:30.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.302+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=40 2023-09-23T03:28:30.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.302+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:56 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb880 con 0x56435fdcbc00 2023-09-23T03:28:30.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.303+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 8 ==== osd_op_reply(56 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.304+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.304+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:57 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca000 con 0x56435febc000 2023-09-23T03:28:30.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.305+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 32 ==== osd_op_reply(57 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.305+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.305+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.305+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:30.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.305+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:58 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecbc00 con 0x56435febcc00 2023-09-23T03:28:30.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.306+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 11 ==== osd_op_reply(58 10000000000.01800000 [omap-get-header out=274b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.306+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 9 2023-09-23T03:28:30.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.306+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:30.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.306+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:59 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca380 con 0x56435febcc00 2023-09-23T03:28:30.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.307+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 12 ==== osd_op_reply(59 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.307+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=40 frag=1*, r=0 2023-09-23T03:28:30.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.307+0000 7ff11803a5c0 20 datascan.read_dentry: 40_head not found in result 2023-09-23T03:28:30.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.307+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000021d at 0x0x10000000000/40 with size=2 bytes 2023-09-23T03:28:30.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.307+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:60 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb500 con 0x56435febcc00 2023-09-23T03:28:30.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.311+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 13 ==== osd_op_reply(60 10000000000.01800000 [omap-set-vals] v128'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.311+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/40 pointing to 0x0x1000000021d 2023-09-23T03:28:30.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.311+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:61 18.30 18:0c28cce1::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecae00 con 0x56435febc400 2023-09-23T03:28:30.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.312+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.0 v2:172.21.15.154:6800/2920938138 6 ==== osd_op_reply(61 [pgnls-filter start_epoch 128 out=101b] v128'21 uv21 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3efc0 con 0x56435febc400 2023-09-23T03:28:30.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.312+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000244.00000000 2023-09-23T03:28:30.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.312+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000244.0 2023-09-23T03:28:30.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.312+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:62 18.30 18:0c28cce1:::10000000244.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb180 con 0x56435febc400 2023-09-23T03:28:30.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.314+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.0 v2:172.21.15.154:6800/2920938138 7 ==== osd_op_reply(62 10000000244.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv19 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3efc0 con 0x56435febc400 2023-09-23T03:28:30.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.314+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000244 2023-09-23T03:28:30.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.314+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/79 2023-09-23T03:28:30.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.314+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=79 2023-09-23T03:28:30.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.314+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:63 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fab3c00 con 0x56435fdcbc00 2023-09-23T03:28:30.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.314+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 9 ==== osd_op_reply(63 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.314+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.314+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:64 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fab3880 con 0x56435febc000 2023-09-23T03:28:30.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.316+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 33 ==== osd_op_reply(64 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.316+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.316+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.316+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:30.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.316+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:65 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca700 con 0x56435febcc00 2023-09-23T03:28:30.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.317+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 14 ==== osd_op_reply(65 10000000000.01800000 [omap-get-header out=274b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.317+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 10 2023-09-23T03:28:30.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.317+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:30.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.317+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:66 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecae00 con 0x56435febcc00 2023-09-23T03:28:30.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.318+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 15 ==== osd_op_reply(66 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.318+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=79 frag=1*, r=0 2023-09-23T03:28:30.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.318+0000 7ff11803a5c0 20 datascan.read_dentry: 79_head not found in result 2023-09-23T03:28:30.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.318+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000244 at 0x0x10000000000/79 with size=2 bytes 2023-09-23T03:28:30.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.318+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:67 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecaa80 con 0x56435febcc00 2023-09-23T03:28:30.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.320+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 16 ==== osd_op_reply(67 10000000000.01800000 [omap-set-vals] v128'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.320+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/79 pointing to 0x0x10000000244 2023-09-23T03:28:30.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.320+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:68 18.30 18:0cc9e906::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb880 con 0x56435febc400 2023-09-23T03:28:30.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.321+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.0 v2:172.21.15.154:6800/2920938138 8 ==== osd_op_reply(68 [pgnls-filter start_epoch 128 out=101b] v128'21 uv21 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3efc0 con 0x56435febc400 2023-09-23T03:28:30.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.321+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000227.00000000 2023-09-23T03:28:30.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.321+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000227.0 2023-09-23T03:28:30.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.321+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:69 18.30 18:0cc9e906:::10000000227.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecbc00 con 0x56435febc400 2023-09-23T03:28:30.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.322+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.0 v2:172.21.15.154:6800/2920938138 9 ==== osd_op_reply(69 10000000227.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv20 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3efc0 con 0x56435febc400 2023-09-23T03:28:30.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.322+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000227 2023-09-23T03:28:30.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.322+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/50 2023-09-23T03:28:30.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.322+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=50 2023-09-23T03:28:30.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.322+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:70 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086e000 con 0x56435fdcbc00 2023-09-23T03:28:30.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.323+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 10 ==== osd_op_reply(70 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.323+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.323+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:71 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086e380 con 0x56435febc000 2023-09-23T03:28:30.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.324+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 34 ==== osd_op_reply(71 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.324+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.324+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.324+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:30.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.324+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:72 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086ee00 con 0x56435febc000 2023-09-23T03:28:30.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.325+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 35 ==== osd_op_reply(72 10000000000.01000000 [omap-get-header out=274b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.325+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 8 2023-09-23T03:28:30.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.325+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:30.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.326+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:73 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086f500 con 0x56435febc000 2023-09-23T03:28:30.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.327+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 36 ==== osd_op_reply(73 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.327+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=50 frag=0*, r=0 2023-09-23T03:28:30.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.327+0000 7ff11803a5c0 20 datascan.read_dentry: 50_head not found in result 2023-09-23T03:28:30.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.327+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000227 at 0x0x10000000000/50 with size=2 bytes 2023-09-23T03:28:30.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.327+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:74 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086f880 con 0x56435febc000 2023-09-23T03:28:30.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.330+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 37 ==== osd_op_reply(74 10000000000.01000000 [omap-set-vals] v128'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.330+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/50 pointing to 0x0x10000000227 2023-09-23T03:28:30.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.330+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:75 18.30 18:0e3d5a27::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086fc00 con 0x56435febc400 2023-09-23T03:28:30.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.331+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.0 v2:172.21.15.154:6800/2920938138 10 ==== osd_op_reply(75 [pgnls-filter start_epoch 128 out=81b] v128'21 uv21 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3efc0 con 0x56435febc400 2023-09-23T03:28:30.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.331+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000250.00000000 2023-09-23T03:28:30.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.331+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000250.0 2023-09-23T03:28:30.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.331+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:76 18.30 18:0e3d5a27:::10000000250.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086ea80 con 0x56435febc400 2023-09-23T03:28:30.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.332+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.0 v2:172.21.15.154:6800/2920938138 11 ==== osd_op_reply(76 10000000250.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv21 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3efc0 con 0x56435febc400 2023-09-23T03:28:30.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.332+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000250 2023-09-23T03:28:30.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.332+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/91 2023-09-23T03:28:30.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.332+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=91 2023-09-23T03:28:30.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.332+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:77 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086f180 con 0x56435fdcbc00 2023-09-23T03:28:30.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.333+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 11 ==== osd_op_reply(77 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.333+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.333+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:78 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086e380 con 0x56435febc000 2023-09-23T03:28:30.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.334+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 38 ==== osd_op_reply(78 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.334+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.334+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.334+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:30.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.334+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:79 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086ee00 con 0x56435febcc00 2023-09-23T03:28:30.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.335+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 17 ==== osd_op_reply(79 10000000000.01800000 [omap-get-header out=274b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.335+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 11 2023-09-23T03:28:30.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.335+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:30.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.335+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:80 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086f500 con 0x56435febcc00 2023-09-23T03:28:30.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.336+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 18 ==== osd_op_reply(80 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.336+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=91 frag=1*, r=0 2023-09-23T03:28:30.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.336+0000 7ff11803a5c0 20 datascan.read_dentry: 91_head not found in result 2023-09-23T03:28:30.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.336+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000250 at 0x0x10000000000/91 with size=2 bytes 2023-09-23T03:28:30.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.336+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:81 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086f880 con 0x56435febcc00 2023-09-23T03:28:30.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.339+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 19 ==== osd_op_reply(81 10000000000.01800000 [omap-set-vals] v128'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.339+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/91 pointing to 0x0x10000000250 2023-09-23T03:28:30.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.339+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:82 18.8 18:10000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fab3880 con 0x56435febc400 2023-09-23T03:28:30.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.340+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.0 v2:172.21.15.154:6800/2920938138 12 ==== osd_op_reply(82 [pgnls-filter start_epoch 128 out=49b] v126'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56435fd3efc0 con 0x56435febc400 2023-09-23T03:28:30.341 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.340+0000 7ff11185f640 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:83 18.28 18:14000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fab3c00 con 0x56435febd000 2023-09-23T03:28:30.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.342+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 3 ==== osd_op_reply(83 [pgnls-filter start_epoch 128 out=101b] v128'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:30.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.342+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000233.00000000 2023-09-23T03:28:30.343 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.342+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000233.0 2023-09-23T03:28:30.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.342+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:84 18.28 18:14ce3b86:::10000000233.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca000 con 0x56435febd000 2023-09-23T03:28:30.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.343+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 4 ==== osd_op_reply(84 10000000233.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:30.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.343+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000233 2023-09-23T03:28:30.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.344+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/62 2023-09-23T03:28:30.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.344+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=62 2023-09-23T03:28:30.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.344+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:85 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086f180 con 0x56435fdcbc00 2023-09-23T03:28:30.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.344+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 12 ==== osd_op_reply(85 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.344+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.344+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:86 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb500 con 0x56435febc000 2023-09-23T03:28:30.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.345+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 39 ==== osd_op_reply(86 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.345+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.346+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.346+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:30.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.346+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:87 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca700 con 0x56435febcc00 2023-09-23T03:28:30.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.347+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 20 ==== osd_op_reply(87 10000000000.01800000 [omap-get-header out=274b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.347+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 12 2023-09-23T03:28:30.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.347+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:30.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.347+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:88 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecae00 con 0x56435febcc00 2023-09-23T03:28:30.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.348+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 21 ==== osd_op_reply(88 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.348+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=62 frag=1*, r=0 2023-09-23T03:28:30.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.348+0000 7ff11803a5c0 20 datascan.read_dentry: 62_head not found in result 2023-09-23T03:28:30.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.348+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000233 at 0x0x10000000000/62 with size=2 bytes 2023-09-23T03:28:30.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.348+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:89 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecaa80 con 0x56435febcc00 2023-09-23T03:28:30.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.352+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 22 ==== osd_op_reply(89 10000000000.01800000 [omap-set-vals] v128'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.352+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/62 pointing to 0x0x10000000233 2023-09-23T03:28:30.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.352+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:90 18.28 18:15246602::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe000 con 0x56435febd000 2023-09-23T03:28:30.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.353+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 5 ==== osd_op_reply(90 [pgnls-filter start_epoch 128 out=101b] v128'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:30.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.353+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000202.00000000 2023-09-23T03:28:30.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.353+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000202.0 2023-09-23T03:28:30.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.353+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:91 18.28 18:15246602:::10000000202.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe380 con 0x56435febd000 2023-09-23T03:28:30.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.354+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 6 ==== osd_op_reply(91 10000000202.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:30.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.355+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000202 2023-09-23T03:28:30.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.355+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/13 2023-09-23T03:28:30.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.355+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=13 2023-09-23T03:28:30.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.355+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:92 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefea80 con 0x56435fdcbc00 2023-09-23T03:28:30.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.355+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 13 ==== osd_op_reply(92 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.356+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.356+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:93 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefee00 con 0x56435febc000 2023-09-23T03:28:30.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.357+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 40 ==== osd_op_reply(93 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.357+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.357+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.357+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:30.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.357+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:94 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff180 con 0x56435febcc00 2023-09-23T03:28:30.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.358+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 23 ==== osd_op_reply(94 10000000000.01800000 [omap-get-header out=274b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.358+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 13 2023-09-23T03:28:30.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.358+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:30.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.358+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:95 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff500 con 0x56435febcc00 2023-09-23T03:28:30.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.359+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 24 ==== osd_op_reply(95 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.359+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=13 frag=1*, r=0 2023-09-23T03:28:30.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.359+0000 7ff11803a5c0 20 datascan.read_dentry: 13_head not found in result 2023-09-23T03:28:30.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.359+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000202 at 0x0x10000000000/13 with size=2 bytes 2023-09-23T03:28:30.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.359+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:96 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff880 con 0x56435febcc00 2023-09-23T03:28:30.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.363+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 25 ==== osd_op_reply(96 10000000000.01800000 [omap-set-vals] v128'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.363+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/13 pointing to 0x0x10000000202 2023-09-23T03:28:30.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.363+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:97 18.28 18:1537fa38::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feffc00 con 0x56435febd000 2023-09-23T03:28:30.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.364+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 7 ==== osd_op_reply(97 [pgnls-filter start_epoch 128 out=101b] v128'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:30.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.365+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000236.00000000 2023-09-23T03:28:30.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.365+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000236.0 2023-09-23T03:28:30.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.365+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:98 18.28 18:1537fa38:::10000000236.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe000 con 0x56435febd000 2023-09-23T03:28:30.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.366+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 8 ==== osd_op_reply(98 10000000236.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:30.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.367+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000236 2023-09-23T03:28:30.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.367+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/65 2023-09-23T03:28:30.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.367+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=65 2023-09-23T03:28:30.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.367+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:99 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe380 con 0x56435fdcbc00 2023-09-23T03:28:30.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.367+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 14 ==== osd_op_reply(99 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.368+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.368+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:100 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefee00 con 0x56435febc000 2023-09-23T03:28:30.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.369+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 41 ==== osd_op_reply(100 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.369+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.369+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.369+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:30.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.369+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:101 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feffc00 con 0x56435febc000 2023-09-23T03:28:30.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.370+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 42 ==== osd_op_reply(101 10000000000.01000000 [omap-get-header out=274b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.370+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 9 2023-09-23T03:28:30.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.370+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:30.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.371+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:102 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefea80 con 0x56435febc000 2023-09-23T03:28:30.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.372+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 43 ==== osd_op_reply(102 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.372+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=65 frag=0*, r=0 2023-09-23T03:28:30.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.372+0000 7ff11803a5c0 20 datascan.read_dentry: 65_head not found in result 2023-09-23T03:28:30.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.372+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000236 at 0x0x10000000000/65 with size=2 bytes 2023-09-23T03:28:30.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.372+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:103 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca000 con 0x56435febc000 2023-09-23T03:28:30.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.376+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 44 ==== osd_op_reply(103 10000000000.01000000 [omap-set-vals] v128'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.376+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/65 pointing to 0x0x10000000236 2023-09-23T03:28:30.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.376+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:104 18.28 18:177a762d::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb500 con 0x56435febd000 2023-09-23T03:28:30.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.378+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 9 ==== osd_op_reply(104 [pgnls-filter start_epoch 128 out=81b] v128'18 uv18 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:30.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.378+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000201.00000000 2023-09-23T03:28:30.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.378+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000201.0 2023-09-23T03:28:30.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.378+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:105 18.28 18:177a762d:::10000000201.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca700 con 0x56435febd000 2023-09-23T03:28:30.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.379+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 10 ==== osd_op_reply(105 10000000201.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv18 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:30.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.380+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000201 2023-09-23T03:28:30.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.380+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/12 2023-09-23T03:28:30.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.380+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=12 2023-09-23T03:28:30.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.380+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:106 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecaa80 con 0x56435fdcbc00 2023-09-23T03:28:30.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.380+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 15 ==== osd_op_reply(106 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.381+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.381+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:107 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086e380 con 0x56435febc000 2023-09-23T03:28:30.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.382+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 45 ==== osd_op_reply(107 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.382+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.382+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.382+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:30.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.382+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:108 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086e000 con 0x56435febc000 2023-09-23T03:28:30.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.383+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 46 ==== osd_op_reply(108 10000000000.01000000 [omap-get-header out=274b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.383+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 10 2023-09-23T03:28:30.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.383+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:30.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.383+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:109 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086ee00 con 0x56435febc000 2023-09-23T03:28:30.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.385+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 47 ==== osd_op_reply(109 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.385+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=12 frag=0*, r=0 2023-09-23T03:28:30.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.385+0000 7ff11803a5c0 20 datascan.read_dentry: 12_head not found in result 2023-09-23T03:28:30.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.385+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000201 at 0x0x10000000000/12 with size=2 bytes 2023-09-23T03:28:30.386 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.385+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:110 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086f500 con 0x56435febc000 2023-09-23T03:28:30.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.389+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 48 ==== osd_op_reply(110 10000000000.01000000 [omap-set-vals] v128'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.389+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/12 pointing to 0x0x10000000201 2023-09-23T03:28:30.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.389+0000 7ff11803a5c0 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x56435fdcb000 0x56435fedab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:30.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.389+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:111 18.18 18:18000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086f880 con 0x56435fdcb000 2023-09-23T03:28:30.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.389+0000 7ff116068640 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x56435fdcb000 0x56435fedab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:30.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.391+0000 7ff116068640 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x56435fdcb000 0x56435fedab00 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:30.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.392+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.1 v2:172.21.15.154:6816/1267710005 1 ==== osd_op_reply(111 [pgnls-filter start_epoch 128 out=49b] v126'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56435fd3efc0 con 0x56435fdcb000 2023-09-23T03:28:30.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.392+0000 7ff11105e640 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:112 18.38 18:1c000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fab3880 con 0x56435fdcbc00 2023-09-23T03:28:30.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.393+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 16 ==== osd_op_reply(112 [pgnls-filter start_epoch 128 out=49b] v126'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.393+0000 7ff11185f640 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:113 18.4 18:20000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe700 con 0x56435febc400 2023-09-23T03:28:30.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.394+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.0 v2:172.21.15.154:6800/2920938138 13 ==== osd_op_reply(113 [pgnls-filter start_epoch 128 out=81b] v128'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3efc0 con 0x56435febc400 2023-09-23T03:28:30.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.394+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000220.00000000 2023-09-23T03:28:30.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.394+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000220.0 2023-09-23T03:28:30.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.394+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:114 18.4 18:2231e764:::10000000220.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086f180 con 0x56435febc400 2023-09-23T03:28:30.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.395+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.0 v2:172.21.15.154:6800/2920938138 14 ==== osd_op_reply(114 10000000220.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3efc0 con 0x56435febc400 2023-09-23T03:28:30.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.395+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000220 2023-09-23T03:28:30.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.395+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/43 2023-09-23T03:28:30.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.395+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=43 2023-09-23T03:28:30.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.395+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:115 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086ea80 con 0x56435fdcbc00 2023-09-23T03:28:30.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.396+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 17 ==== osd_op_reply(115 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.396+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.396+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:116 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086e700 con 0x56435febc000 2023-09-23T03:28:30.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.397+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 49 ==== osd_op_reply(116 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.397+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.398+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.398+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:30.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.398+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:117 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086fc00 con 0x56435febc000 2023-09-23T03:28:30.400 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.399+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 50 ==== osd_op_reply(117 10000000000.01000000 [omap-get-header out=274b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.400 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.399+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 11 2023-09-23T03:28:30.400 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.399+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:30.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.399+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:118 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086f880 con 0x56435febc000 2023-09-23T03:28:30.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.400+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 51 ==== osd_op_reply(118 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.401+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=43 frag=0*, r=0 2023-09-23T03:28:30.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.401+0000 7ff11803a5c0 20 datascan.read_dentry: 43_head not found in result 2023-09-23T03:28:30.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.401+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000220 at 0x0x10000000000/43 with size=2 bytes 2023-09-23T03:28:30.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.401+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:119 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086e380 con 0x56435febc000 2023-09-23T03:28:30.406 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.404+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 52 ==== osd_op_reply(119 10000000000.01000000 [omap-set-vals] v128'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.406 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.404+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/43 pointing to 0x0x10000000220 2023-09-23T03:28:30.406 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.404+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:120 18.24 18:24000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086e000 con 0x56435febc400 2023-09-23T03:28:30.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.406+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.0 v2:172.21.15.154:6800/2920938138 15 ==== osd_op_reply(120 [pgnls-filter start_epoch 128 out=101b] v128'10 uv10 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3efc0 con 0x56435febc400 2023-09-23T03:28:30.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.406+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000234.00000000 2023-09-23T03:28:30.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.406+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000234.0 2023-09-23T03:28:30.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.406+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:121 18.24 18:242ba651:::10000000234.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecbc00 con 0x56435febc400 2023-09-23T03:28:30.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.407+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.0 v2:172.21.15.154:6800/2920938138 16 ==== osd_op_reply(121 10000000234.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv9 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3efc0 con 0x56435febc400 2023-09-23T03:28:30.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.407+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000234 2023-09-23T03:28:30.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.407+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/63 2023-09-23T03:28:30.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.407+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=63 2023-09-23T03:28:30.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.407+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:122 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb880 con 0x56435fdcbc00 2023-09-23T03:28:30.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.408+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 18 ==== osd_op_reply(122 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.408+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.408+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:123 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb180 con 0x56435febc000 2023-09-23T03:28:30.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.409+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 53 ==== osd_op_reply(123 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.409+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.409+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.409+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:30.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.409+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:124 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca000 con 0x56435febc000 2023-09-23T03:28:30.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.410+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 54 ==== osd_op_reply(124 10000000000.01000000 [omap-get-header out=274b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.410+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 12 2023-09-23T03:28:30.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.410+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:30.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.411+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:125 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb500 con 0x56435febc000 2023-09-23T03:28:30.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.412+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 55 ==== osd_op_reply(125 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.412+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=63 frag=0*, r=0 2023-09-23T03:28:30.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.412+0000 7ff11803a5c0 20 datascan.read_dentry: 63_head not found in result 2023-09-23T03:28:30.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.412+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000234 at 0x0x10000000000/63 with size=2 bytes 2023-09-23T03:28:30.413 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.412+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:126 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecae00 con 0x56435febc000 2023-09-23T03:28:30.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.415+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 56 ==== osd_op_reply(126 10000000000.01000000 [omap-set-vals] v128'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.416+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/63 pointing to 0x0x10000000234 2023-09-23T03:28:30.417 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.416+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:127 18.24 18:25b5fce8::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca700 con 0x56435febc400 2023-09-23T03:28:30.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.416+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.0 v2:172.21.15.154:6800/2920938138 17 ==== osd_op_reply(127 [pgnls-filter start_epoch 128 out=81b] v128'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3efc0 con 0x56435febc400 2023-09-23T03:28:30.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.417+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000231.00000000 2023-09-23T03:28:30.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.417+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000231.0 2023-09-23T03:28:30.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.417+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:128 18.24 18:25b5fce8:::10000000231.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefee00 con 0x56435febc400 2023-09-23T03:28:30.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.417+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.0 v2:172.21.15.154:6800/2920938138 18 ==== osd_op_reply(128 10000000231.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv10 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3efc0 con 0x56435febc400 2023-09-23T03:28:30.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.418+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000231 2023-09-23T03:28:30.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.418+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/60 2023-09-23T03:28:30.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.418+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=60 2023-09-23T03:28:30.419 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.418+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:129 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefea80 con 0x56435fdcbc00 2023-09-23T03:28:30.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.419+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 19 ==== osd_op_reply(129 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.419+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.420 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.419+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:130 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff180 con 0x56435febc000 2023-09-23T03:28:30.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.420+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 57 ==== osd_op_reply(130 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.420+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.420+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.420+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:30.421 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.420+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:131 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff500 con 0x56435febcc00 2023-09-23T03:28:30.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.421+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 26 ==== osd_op_reply(131 10000000000.01800000 [omap-get-header out=274b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.421+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 14 2023-09-23T03:28:30.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.421+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:30.423 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.421+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:132 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff880 con 0x56435febcc00 2023-09-23T03:28:30.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.422+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 27 ==== osd_op_reply(132 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.422+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=60 frag=1*, r=0 2023-09-23T03:28:30.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.422+0000 7ff11803a5c0 20 datascan.read_dentry: 60_head not found in result 2023-09-23T03:28:30.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.422+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000231 at 0x0x10000000000/60 with size=2 bytes 2023-09-23T03:28:30.424 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.422+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:133 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe380 con 0x56435febcc00 2023-09-23T03:28:30.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.426+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 28 ==== osd_op_reply(133 10000000000.01800000 [omap-set-vals] v128'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.426+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/60 pointing to 0x0x10000000231 2023-09-23T03:28:30.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.426+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:134 18.14 18:28000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefea80 con 0x56435febc000 2023-09-23T03:28:30.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.427+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 58 ==== osd_op_reply(134 [pgnls-filter start_epoch 128 out=101b] v128'20 uv20 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.428 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.427+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 1000000020e.00000000 2023-09-23T03:28:30.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.427+0000 7ff11803a5c0 10 datascan.operator(): handling object 1000000020e.0 2023-09-23T03:28:30.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.427+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:135 18.14 18:29971f97:::1000000020e.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecaa80 con 0x56435febc000 2023-09-23T03:28:30.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.428+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 59 ==== osd_op_reply(135 1000000020e.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv18 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.429+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000020e 2023-09-23T03:28:30.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.429+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/25 2023-09-23T03:28:30.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.429+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=25 2023-09-23T03:28:30.430 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.429+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:136 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086ea80 con 0x56435fdcbc00 2023-09-23T03:28:30.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.430+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 20 ==== osd_op_reply(136 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.430+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.430+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:137 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fab3880 con 0x56435febc000 2023-09-23T03:28:30.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.431+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 60 ==== osd_op_reply(137 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.431+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.431+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.431+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:30.432 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.431+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:138 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff180 con 0x56435febcc00 2023-09-23T03:28:30.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.432+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 29 ==== osd_op_reply(138 10000000000.01800000 [omap-get-header out=274b] v0'0 uv15 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.432+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 15 2023-09-23T03:28:30.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.432+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:30.433 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.432+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:139 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefea80 con 0x56435febcc00 2023-09-23T03:28:30.434 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.433+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 30 ==== osd_op_reply(139 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv15 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.433+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=25 frag=1*, r=0 2023-09-23T03:28:30.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.433+0000 7ff11803a5c0 20 datascan.read_dentry: 25_head not found in result 2023-09-23T03:28:30.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.433+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000020e at 0x0x10000000000/25 with size=2 bytes 2023-09-23T03:28:30.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.433+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:140 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb180 con 0x56435febcc00 2023-09-23T03:28:30.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.437+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 31 ==== osd_op_reply(140 10000000000.01800000 [omap-set-vals] v128'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.437+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/25 pointing to 0x0x1000000020e 2023-09-23T03:28:30.438 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.437+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:141 18.14 18:2ab28be7::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca000 con 0x56435febc000 2023-09-23T03:28:30.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.438+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 61 ==== osd_op_reply(141 [pgnls-filter start_epoch 128 out=101b] v128'20 uv20 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.439 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.438+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000211.00000000 2023-09-23T03:28:30.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.438+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000211.0 2023-09-23T03:28:30.440 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.438+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:142 18.14 18:2ab28be7:::10000000211.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb500 con 0x56435febc000 2023-09-23T03:28:30.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.440+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 62 ==== osd_op_reply(142 10000000211.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv19 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.440+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000211 2023-09-23T03:28:30.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.440+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/28 2023-09-23T03:28:30.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.440+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=28 2023-09-23T03:28:30.441 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.440+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:143 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086ee00 con 0x56435fdcbc00 2023-09-23T03:28:30.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.441+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 21 ==== osd_op_reply(143 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.441+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.442 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.441+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:144 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086f500 con 0x56435febc000 2023-09-23T03:28:30.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.442+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 63 ==== osd_op_reply(144 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.442+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.442+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.442+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:30.443 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.442+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:145 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086e700 con 0x56435febc000 2023-09-23T03:28:30.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.443+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 64 ==== osd_op_reply(145 10000000000.01000000 [omap-get-header out=274b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.443+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 13 2023-09-23T03:28:30.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.443+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:30.445 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.443+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:146 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086fc00 con 0x56435febc000 2023-09-23T03:28:30.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.445+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 65 ==== osd_op_reply(146 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.445+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=28 frag=0*, r=0 2023-09-23T03:28:30.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.445+0000 7ff11803a5c0 20 datascan.read_dentry: 28_head not found in result 2023-09-23T03:28:30.446 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.445+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000211 at 0x0x10000000000/28 with size=2 bytes 2023-09-23T03:28:30.447 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.445+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:147 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086f880 con 0x56435febc000 2023-09-23T03:28:30.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.449+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 66 ==== osd_op_reply(147 10000000000.01000000 [omap-set-vals] v128'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.449+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/28 pointing to 0x0x10000000211 2023-09-23T03:28:30.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.449+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:148 18.14 18:2b89f3ba::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56436086e380 con 0x56435febc000 2023-09-23T03:28:30.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.450+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 67 ==== osd_op_reply(148 [pgnls-filter start_epoch 128 out=81b] v128'20 uv20 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.450+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000221.00000000 2023-09-23T03:28:30.451 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.450+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000221.0 2023-09-23T03:28:30.452 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.450+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:149 18.14 18:2b89f3ba:::10000000221.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1c000 con 0x56435febc000 2023-09-23T03:28:30.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.451+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 68 ==== osd_op_reply(149 10000000221.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv20 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.451+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000221 2023-09-23T03:28:30.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.451+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/44 2023-09-23T03:28:30.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.451+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=44 2023-09-23T03:28:30.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.451+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:150 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1c700 con 0x56435fdcbc00 2023-09-23T03:28:30.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.453+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 22 ==== osd_op_reply(150 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.453+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.454 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.453+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:151 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1ca80 con 0x56435febc000 2023-09-23T03:28:30.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.454+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 69 ==== osd_op_reply(151 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.454+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.454+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.454+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:30.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.454+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:152 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1ce00 con 0x56435febcc00 2023-09-23T03:28:30.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.456+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 32 ==== osd_op_reply(152 10000000000.01800000 [omap-get-header out=274b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.456+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 16 2023-09-23T03:28:30.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.456+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:30.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.456+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:153 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d180 con 0x56435febcc00 2023-09-23T03:28:30.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.457+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 33 ==== osd_op_reply(153 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.457+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=44 frag=1*, r=0 2023-09-23T03:28:30.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.457+0000 7ff11803a5c0 20 datascan.read_dentry: 44_head not found in result 2023-09-23T03:28:30.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.457+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000221 at 0x0x10000000000/44 with size=2 bytes 2023-09-23T03:28:30.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.457+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:154 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d500 con 0x56435febcc00 2023-09-23T03:28:30.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.460+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 34 ==== osd_op_reply(154 10000000000.01800000 [omap-set-vals] v128'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.461+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/44 pointing to 0x0x10000000221 2023-09-23T03:28:30.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.461+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:155 18.34 18:2c000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d880 con 0x56435febc400 2023-09-23T03:28:30.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.462+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.0 v2:172.21.15.154:6800/2920938138 19 ==== osd_op_reply(155 [pgnls-filter start_epoch 128 out=49b] v126'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56435fd3efc0 con 0x56435febc400 2023-09-23T03:28:30.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.462+0000 7ff11105e640 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:156 18.c 18:30000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe700 con 0x56435febd000 2023-09-23T03:28:30.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.463+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 11 ==== osd_op_reply(156 [pgnls-filter start_epoch 128 out=81b] v128'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:30.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.463+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 100000001fc.00000000 2023-09-23T03:28:30.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.463+0000 7ff11803a5c0 10 datascan.operator(): handling object 100000001fc.0 2023-09-23T03:28:30.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.463+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:157 18.c 18:30604137:::100000001fc.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1dc00 con 0x56435febd000 2023-09-23T03:28:30.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.464+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 12 ==== osd_op_reply(157 100000001fc.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv9 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:30.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.464+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001fc 2023-09-23T03:28:30.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.464+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/7 2023-09-23T03:28:30.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.464+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=7 2023-09-23T03:28:30.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.464+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:158 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1c380 con 0x56435fdcbc00 2023-09-23T03:28:30.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.465+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 23 ==== osd_op_reply(158 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.466+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.466+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:159 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1ca80 con 0x56435febc000 2023-09-23T03:28:30.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.467+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 70 ==== osd_op_reply(159 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.467+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.467+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.467+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:30.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.467+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:160 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d880 con 0x56435febcc00 2023-09-23T03:28:30.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.468+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 35 ==== osd_op_reply(160 10000000000.01800000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.468+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 17 2023-09-23T03:28:30.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.468+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:30.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.468+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:161 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1c700 con 0x56435febcc00 2023-09-23T03:28:30.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.470+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 36 ==== osd_op_reply(161 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.470+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=7 frag=1*, r=0 2023-09-23T03:28:30.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.470+0000 7ff11803a5c0 20 datascan.read_dentry: 7_head not found in result 2023-09-23T03:28:30.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.470+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fc at 0x0x10000000000/7 with size=1 bytes 2023-09-23T03:28:30.471 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.470+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:162 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1ce00 con 0x56435febcc00 2023-09-23T03:28:30.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.473+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 37 ==== osd_op_reply(162 10000000000.01800000 [omap-set-vals] v128'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.473+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/7 pointing to 0x0x100000001fc 2023-09-23T03:28:30.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.473+0000 7ff11803a5c0 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x56435ff15800 0x56435fedb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:30.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.473+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:163 18.2c 18:34000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe700 con 0x56435ff15800 2023-09-23T03:28:30.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.474+0000 7ff115867640 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x56435ff15800 0x56435fedb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:30.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.475+0000 7ff115867640 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x56435ff15800 0x56435fedb080 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:30.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.476+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.2 v2:172.21.15.154:6822/2884407243 1 ==== osd_op_reply(163 [pgnls-filter start_epoch 128 out=101b] v128'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3eb40 con 0x56435ff15800 2023-09-23T03:28:30.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.476+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000223.00000000 2023-09-23T03:28:30.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.476+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000223.0 2023-09-23T03:28:30.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.476+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:164 18.2c 18:35964134:::10000000223.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feffc00 con 0x56435ff15800 2023-09-23T03:28:30.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.477+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.2 v2:172.21.15.154:6822/2884407243 2 ==== osd_op_reply(164 10000000223.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3eb40 con 0x56435ff15800 2023-09-23T03:28:30.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.477+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000223 2023-09-23T03:28:30.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.477+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/46 2023-09-23T03:28:30.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.477+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=46 2023-09-23T03:28:30.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.477+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:165 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff500 con 0x56435fdcbc00 2023-09-23T03:28:30.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.478+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 24 ==== osd_op_reply(165 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.478+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.479 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.478+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:166 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff880 con 0x56435febc000 2023-09-23T03:28:30.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.479+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 71 ==== osd_op_reply(166 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.479+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.479+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.479+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:30.481 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.479+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:167 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe380 con 0x56435febc000 2023-09-23T03:28:30.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.481+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 72 ==== osd_op_reply(167 10000000000.01000000 [omap-get-header out=274b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.481+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 14 2023-09-23T03:28:30.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.481+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:30.482 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.481+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:168 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff180 con 0x56435febc000 2023-09-23T03:28:30.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.482+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 73 ==== osd_op_reply(168 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.482+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=46 frag=0*, r=0 2023-09-23T03:28:30.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.482+0000 7ff11803a5c0 20 datascan.read_dentry: 46_head not found in result 2023-09-23T03:28:30.483 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.482+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000223 at 0x0x10000000000/46 with size=2 bytes 2023-09-23T03:28:30.484 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.482+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:169 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefea80 con 0x56435febc000 2023-09-23T03:28:30.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.486+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 74 ==== osd_op_reply(169 10000000000.01000000 [omap-set-vals] v128'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.486+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/46 pointing to 0x0x10000000223 2023-09-23T03:28:30.487 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.486+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:170 18.2c 18:37389ade::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fab3880 con 0x56435ff15800 2023-09-23T03:28:30.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.487+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.2 v2:172.21.15.154:6822/2884407243 3 ==== osd_op_reply(170 [pgnls-filter start_epoch 128 out=81b] v128'14 uv14 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3eb40 con 0x56435ff15800 2023-09-23T03:28:30.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.487+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000209.00000000 2023-09-23T03:28:30.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.487+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000209.0 2023-09-23T03:28:30.488 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.487+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:171 18.2c 18:37389ade:::10000000209.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fab3880 con 0x56435ff15800 2023-09-23T03:28:30.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.488+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.2 v2:172.21.15.154:6822/2884407243 4 ==== osd_op_reply(171 10000000209.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv14 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3eb40 con 0x56435ff15800 2023-09-23T03:28:30.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.488+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000209 2023-09-23T03:28:30.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.488+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/20 2023-09-23T03:28:30.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.488+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=20 2023-09-23T03:28:30.489 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.488+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:172 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefee00 con 0x56435fdcbc00 2023-09-23T03:28:30.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.489+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 25 ==== osd_op_reply(172 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.490 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.489+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.491 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.489+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:173 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feffc00 con 0x56435febc000 2023-09-23T03:28:30.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.490+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 75 ==== osd_op_reply(173 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.491+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.491+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.491+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:30.492 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.491+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:174 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff500 con 0x56435febc000 2023-09-23T03:28:30.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.492+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 76 ==== osd_op_reply(174 10000000000.01000000 [omap-get-header out=274b] v0'0 uv15 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.492+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 15 2023-09-23T03:28:30.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.492+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:30.493 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.492+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:175 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d180 con 0x56435febc000 2023-09-23T03:28:30.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.493+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 77 ==== osd_op_reply(175 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv15 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.493+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=20 frag=0*, r=0 2023-09-23T03:28:30.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.493+0000 7ff11803a5c0 20 datascan.read_dentry: 20_head not found in result 2023-09-23T03:28:30.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.493+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000209 at 0x0x10000000000/20 with size=2 bytes 2023-09-23T03:28:30.495 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.493+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:176 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d500 con 0x56435febc000 2023-09-23T03:28:30.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.497+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 78 ==== osd_op_reply(176 10000000000.01000000 [omap-set-vals] v128'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.497+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/20 pointing to 0x0x10000000209 2023-09-23T03:28:30.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.497+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:177 18.1c 18:38000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1c380 con 0x56435fdcb000 2023-09-23T03:28:30.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.498+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.1 v2:172.21.15.154:6816/1267710005 2 ==== osd_op_reply(177 [pgnls-filter start_epoch 128 out=49b] v126'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56435fd3efc0 con 0x56435fdcb000 2023-09-23T03:28:30.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.498+0000 7ff11105e640 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x56435fdca400 0x56435fedc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:30.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.499+0000 7ff11105e640 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:178 18.3c 18:3c000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fab3c00 con 0x56435fdca400 2023-09-23T03:28:30.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.500+0000 7ff116869640 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x56435fdca400 0x56435fedc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:30.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.501+0000 7ff116869640 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x56435fdca400 0x56435fedc680 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:30.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.502+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.7 v2:172.21.15.190:6824/3460136272 1 ==== osd_op_reply(178 [pgnls-filter start_epoch 128 out=49b] v126'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56435fd3ed80 con 0x56435fdca400 2023-09-23T03:28:30.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.502+0000 7ff11185f640 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:179 18.2 18:40000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe000 con 0x56435fdca400 2023-09-23T03:28:30.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.503+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.7 v2:172.21.15.190:6824/3460136272 2 ==== osd_op_reply(179 [pgnls-filter start_epoch 128 out=49b] v126'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56435fd3ed80 con 0x56435fdca400 2023-09-23T03:28:30.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.503+0000 7ff11105e640 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:180 18.22 18:44000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca000 con 0x56435fdcbc00 2023-09-23T03:28:30.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.504+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 26 ==== osd_op_reply(180 [pgnls-filter start_epoch 128 out=101b] v128'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.504+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 1000000024c.00000000 2023-09-23T03:28:30.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.504+0000 7ff11803a5c0 10 datascan.operator(): handling object 1000000024c.0 2023-09-23T03:28:30.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.504+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:181 18.22 18:450d4ca5:::1000000024c.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d880 con 0x56435fdcbc00 2023-09-23T03:28:30.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.505+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 27 ==== osd_op_reply(181 1000000024c.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.505+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000024c 2023-09-23T03:28:30.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.505+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/87 2023-09-23T03:28:30.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.505+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=87 2023-09-23T03:28:30.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.505+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:182 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1ce00 con 0x56435fdcbc00 2023-09-23T03:28:30.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.506+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 28 ==== osd_op_reply(182 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.506+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.506+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:183 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecaa80 con 0x56435febc000 2023-09-23T03:28:30.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.507+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 79 ==== osd_op_reply(183 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.507+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.507+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.507+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:30.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.507+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:184 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecae00 con 0x56435febc000 2023-09-23T03:28:30.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.508+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 80 ==== osd_op_reply(184 10000000000.01000000 [omap-get-header out=274b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.509+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 16 2023-09-23T03:28:30.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.509+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:30.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.509+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:185 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb180 con 0x56435febc000 2023-09-23T03:28:30.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.510+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 81 ==== osd_op_reply(185 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.510+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=87 frag=0*, r=0 2023-09-23T03:28:30.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.510+0000 7ff11803a5c0 20 datascan.read_dentry: 87_head not found in result 2023-09-23T03:28:30.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.510+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000024c at 0x0x10000000000/87 with size=2 bytes 2023-09-23T03:28:30.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.510+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:186 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb500 con 0x56435febc000 2023-09-23T03:28:30.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.514+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 82 ==== osd_op_reply(186 10000000000.01000000 [omap-set-vals] v128'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.514+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/87 pointing to 0x0x1000000024c 2023-09-23T03:28:30.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.514+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:187 18.22 18:45b8cac8::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb880 con 0x56435fdcbc00 2023-09-23T03:28:30.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.515+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 29 ==== osd_op_reply(187 [pgnls-filter start_epoch 128 out=101b] v128'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.515+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 1000000023d.00000000 2023-09-23T03:28:30.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.515+0000 7ff11803a5c0 10 datascan.operator(): handling object 1000000023d.0 2023-09-23T03:28:30.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.515+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:188 18.22 18:45b8cac8:::1000000023d.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecbc00 con 0x56435fdcbc00 2023-09-23T03:28:30.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.516+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 30 ==== osd_op_reply(188 1000000023d.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.516+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000023d 2023-09-23T03:28:30.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.516+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/72 2023-09-23T03:28:30.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.516+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=72 2023-09-23T03:28:30.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.516+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:189 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecae00 con 0x56435fdcbc00 2023-09-23T03:28:30.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.517+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 31 ==== osd_op_reply(189 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.517+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.517+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:190 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb180 con 0x56435febc000 2023-09-23T03:28:30.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.518+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 83 ==== osd_op_reply(190 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.518+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.518+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.518+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:30.520 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.519+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:191 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff880 con 0x56435febcc00 2023-09-23T03:28:30.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.520+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 38 ==== osd_op_reply(191 10000000000.01800000 [omap-get-header out=274b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.520+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 18 2023-09-23T03:28:30.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.520+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:30.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.520+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:192 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe380 con 0x56435febcc00 2023-09-23T03:28:30.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.521+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 39 ==== osd_op_reply(192 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.521+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=72 frag=1*, r=0 2023-09-23T03:28:30.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.521+0000 7ff11803a5c0 20 datascan.read_dentry: 72_head not found in result 2023-09-23T03:28:30.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.521+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000023d at 0x0x10000000000/72 with size=2 bytes 2023-09-23T03:28:30.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.521+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:193 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff180 con 0x56435febcc00 2023-09-23T03:28:30.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.524+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 40 ==== osd_op_reply(193 10000000000.01800000 [omap-set-vals] v128'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.525+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/72 pointing to 0x0x1000000023d 2023-09-23T03:28:30.526 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.525+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:194 18.22 18:47efb22f::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefea80 con 0x56435fdcbc00 2023-09-23T03:28:30.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.526+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 32 ==== osd_op_reply(194 [pgnls-filter start_epoch 128 out=81b] v128'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.526+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000235.00000000 2023-09-23T03:28:30.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.526+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000235.0 2023-09-23T03:28:30.527 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.526+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:195 18.22 18:47efb22f:::10000000235.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feffc00 con 0x56435fdcbc00 2023-09-23T03:28:30.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.527+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 33 ==== osd_op_reply(195 10000000235.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.527+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000235 2023-09-23T03:28:30.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.527+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/64 2023-09-23T03:28:30.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.527+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=64 2023-09-23T03:28:30.528 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.527+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:196 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe000 con 0x56435fdcbc00 2023-09-23T03:28:30.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.528+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 34 ==== osd_op_reply(196 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.528+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.529 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.528+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:197 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fab3c00 con 0x56435febc000 2023-09-23T03:28:30.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.529+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 84 ==== osd_op_reply(197 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.529+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.529+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.529+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:30.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.529+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:198 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1c000 con 0x56435febc000 2023-09-23T03:28:30.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.530+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 85 ==== osd_op_reply(198 10000000000.01000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.531 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.530+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 17 2023-09-23T03:28:30.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.530+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:30.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.530+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:199 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1dc00 con 0x56435febc000 2023-09-23T03:28:30.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.531+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 86 ==== osd_op_reply(199 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.531+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=64 frag=0*, r=0 2023-09-23T03:28:30.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.531+0000 7ff11803a5c0 20 datascan.read_dentry: 64_head not found in result 2023-09-23T03:28:30.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.532+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000235 at 0x0x10000000000/64 with size=2 bytes 2023-09-23T03:28:30.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.532+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:200 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1ca80 con 0x56435febc000 2023-09-23T03:28:30.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.535+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 87 ==== osd_op_reply(200 10000000000.01000000 [omap-set-vals] v128'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.535+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/64 pointing to 0x0x10000000235 2023-09-23T03:28:30.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.535+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:201 18.12 18:48000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d180 con 0x56435ff15800 2023-09-23T03:28:30.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.536+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.2 v2:172.21.15.154:6822/2884407243 5 ==== osd_op_reply(201 [pgnls-filter start_epoch 128 out=49b] v126'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56435fd3eb40 con 0x56435ff15800 2023-09-23T03:28:30.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.536+0000 7ff11105e640 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:202 18.32 18:4c000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca380 con 0x56435fdcbc00 2023-09-23T03:28:30.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.537+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 35 ==== osd_op_reply(202 [pgnls-filter start_epoch 128 out=81b] v128'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.537+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000252.00000000 2023-09-23T03:28:30.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.537+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000252.0 2023-09-23T03:28:30.539 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.537+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:203 18.32 18:4e8c6251:::10000000252.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d500 con 0x56435fdcbc00 2023-09-23T03:28:30.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.539+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 36 ==== osd_op_reply(203 10000000252.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.539+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000252 2023-09-23T03:28:30.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.539+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/93 2023-09-23T03:28:30.540 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.539+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=93 2023-09-23T03:28:30.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.539+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:204 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefee00 con 0x56435fdcbc00 2023-09-23T03:28:30.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.540+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 37 ==== osd_op_reply(204 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.540+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.540+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:205 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff880 con 0x56435febc000 2023-09-23T03:28:30.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.541+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 88 ==== osd_op_reply(205 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.541+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.541+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.541+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:30.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.541+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:206 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca000 con 0x56435febcc00 2023-09-23T03:28:30.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.542+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 41 ==== osd_op_reply(206 10000000000.01800000 [omap-get-header out=274b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.542+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 19 2023-09-23T03:28:30.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.542+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:30.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.543+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:207 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb880 con 0x56435febcc00 2023-09-23T03:28:30.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.544+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 42 ==== osd_op_reply(207 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.544+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=93 frag=1*, r=0 2023-09-23T03:28:30.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.544+0000 7ff11803a5c0 20 datascan.read_dentry: 93_head not found in result 2023-09-23T03:28:30.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.544+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000252 at 0x0x10000000000/93 with size=2 bytes 2023-09-23T03:28:30.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.544+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:208 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecaa80 con 0x56435febcc00 2023-09-23T03:28:30.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.547+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 43 ==== osd_op_reply(208 10000000000.01800000 [omap-set-vals] v128'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.548+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/93 pointing to 0x0x10000000252 2023-09-23T03:28:30.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.548+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:209 18.a 18:50000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecbc00 con 0x56435fdcbc00 2023-09-23T03:28:30.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.549+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 38 ==== osd_op_reply(209 [pgnls-filter start_epoch 128 out=101b] v128'11 uv11 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.549+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 1000000023f.00000000 2023-09-23T03:28:30.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.549+0000 7ff11803a5c0 10 datascan.operator(): handling object 1000000023f.0 2023-09-23T03:28:30.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.549+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:210 18.a 18:52dcfedc:::1000000023f.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecae00 con 0x56435fdcbc00 2023-09-23T03:28:30.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.550+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 39 ==== osd_op_reply(210 1000000023f.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv10 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.550+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000023f 2023-09-23T03:28:30.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.550+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/74 2023-09-23T03:28:30.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.550+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=74 2023-09-23T03:28:30.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.550+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:211 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d880 con 0x56435fdcbc00 2023-09-23T03:28:30.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.551+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 40 ==== osd_op_reply(211 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.551+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.551+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:212 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1ce00 con 0x56435febc000 2023-09-23T03:28:30.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.552+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 89 ==== osd_op_reply(212 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.552+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.552+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.552+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:30.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.552+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:213 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fab3880 con 0x56435febc000 2023-09-23T03:28:30.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.553+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 90 ==== osd_op_reply(213 10000000000.01000000 [omap-get-header out=274b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.553+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 18 2023-09-23T03:28:30.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.553+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:30.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.553+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:214 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca380 con 0x56435febc000 2023-09-23T03:28:30.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.555+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 91 ==== osd_op_reply(214 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.555+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=74 frag=0*, r=0 2023-09-23T03:28:30.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.555+0000 7ff11803a5c0 20 datascan.read_dentry: 74_head not found in result 2023-09-23T03:28:30.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.555+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000023f at 0x0x10000000000/74 with size=2 bytes 2023-09-23T03:28:30.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.555+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:215 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca000 con 0x56435febc000 2023-09-23T03:28:30.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.559+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 92 ==== osd_op_reply(215 10000000000.01000000 [omap-set-vals] v128'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.559+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/74 pointing to 0x0x1000000023f 2023-09-23T03:28:30.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.559+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:216 18.a 18:53d16f76::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb880 con 0x56435fdcbc00 2023-09-23T03:28:30.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.560+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 41 ==== osd_op_reply(216 [pgnls-filter start_epoch 128 out=81b] v128'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.560+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 1000000023b.00000000 2023-09-23T03:28:30.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.560+0000 7ff11803a5c0 10 datascan.operator(): handling object 1000000023b.0 2023-09-23T03:28:30.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.560+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:217 18.a 18:53d16f76:::1000000023b.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe380 con 0x56435fdcbc00 2023-09-23T03:28:30.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.562+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 42 ==== osd_op_reply(217 1000000023b.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.562+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000023b 2023-09-23T03:28:30.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.562+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/70 2023-09-23T03:28:30.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.562+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=70 2023-09-23T03:28:30.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.562+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:218 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefea80 con 0x56435fdcbc00 2023-09-23T03:28:30.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.563+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 43 ==== osd_op_reply(218 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.563+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.563+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:219 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff500 con 0x56435febc000 2023-09-23T03:28:30.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.564+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 93 ==== osd_op_reply(219 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.564+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.564+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.564+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:30.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.564+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:220 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feffc00 con 0x56435febc000 2023-09-23T03:28:30.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.565+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 94 ==== osd_op_reply(220 10000000000.01000000 [omap-get-header out=274b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.566+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 19 2023-09-23T03:28:30.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.566+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:30.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.566+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:221 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe000 con 0x56435febc000 2023-09-23T03:28:30.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.567+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 95 ==== osd_op_reply(221 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.567+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=70 frag=0*, r=0 2023-09-23T03:28:30.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.567+0000 7ff11803a5c0 20 datascan.read_dentry: 70_head not found in result 2023-09-23T03:28:30.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.567+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000023b at 0x0x10000000000/70 with size=2 bytes 2023-09-23T03:28:30.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.567+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:222 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe700 con 0x56435febc000 2023-09-23T03:28:30.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.570+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 96 ==== osd_op_reply(222 10000000000.01000000 [omap-set-vals] v128'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.571+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/70 pointing to 0x0x1000000023b 2023-09-23T03:28:30.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.571+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:223 18.2a 18:54000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefee00 con 0x56435febcc00 2023-09-23T03:28:30.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.572+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 44 ==== osd_op_reply(223 [pgnls-filter start_epoch 128 out=101b] v128'21 uv21 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.572+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 1000000020a.00000000 2023-09-23T03:28:30.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.572+0000 7ff11803a5c0 10 datascan.operator(): handling object 1000000020a.0 2023-09-23T03:28:30.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.572+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:224 18.2a 18:55a6cd83:::1000000020a.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d180 con 0x56435febcc00 2023-09-23T03:28:30.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.574+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 45 ==== osd_op_reply(224 1000000020a.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv18 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.574+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000020a 2023-09-23T03:28:30.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.574+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/21 2023-09-23T03:28:30.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.574+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=21 2023-09-23T03:28:30.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.574+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:225 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff34000 con 0x56435fdcbc00 2023-09-23T03:28:30.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.575+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 44 ==== osd_op_reply(225 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.575+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.575+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:226 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff34380 con 0x56435febc000 2023-09-23T03:28:30.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.576+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 97 ==== osd_op_reply(226 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.576+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.576+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.576+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:30.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.576+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:227 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff34700 con 0x56435febc000 2023-09-23T03:28:30.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.577+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 98 ==== osd_op_reply(227 10000000000.01000000 [omap-get-header out=274b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.578+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 20 2023-09-23T03:28:30.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.578+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:30.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.578+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:228 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff34a80 con 0x56435febc000 2023-09-23T03:28:30.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.579+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 99 ==== osd_op_reply(228 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.579+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=21 frag=0*, r=0 2023-09-23T03:28:30.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.579+0000 7ff11803a5c0 20 datascan.read_dentry: 21_head not found in result 2023-09-23T03:28:30.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.579+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000020a at 0x0x10000000000/21 with size=2 bytes 2023-09-23T03:28:30.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.579+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:229 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff34e00 con 0x56435febc000 2023-09-23T03:28:30.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.582+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 100 ==== osd_op_reply(229 10000000000.01000000 [omap-set-vals] v128'21 uv21 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.583+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/21 pointing to 0x0x1000000020a 2023-09-23T03:28:30.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.583+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:230 18.2a 18:56889a45::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff35180 con 0x56435febcc00 2023-09-23T03:28:30.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.584+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 46 ==== osd_op_reply(230 [pgnls-filter start_epoch 128 out=101b] v128'21 uv21 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.584+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000213.00000000 2023-09-23T03:28:30.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.584+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000213.0 2023-09-23T03:28:30.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.584+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:231 18.2a 18:56889a45:::10000000213.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff35500 con 0x56435febcc00 2023-09-23T03:28:30.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.585+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 47 ==== osd_op_reply(231 10000000213.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv19 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.585+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000213 2023-09-23T03:28:30.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.585+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/30 2023-09-23T03:28:30.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.585+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=30 2023-09-23T03:28:30.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.585+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:232 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff35c00 con 0x56435fdcbc00 2023-09-23T03:28:30.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.586+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 45 ==== osd_op_reply(232 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.586+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.586+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:233 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff34000 con 0x56435febc000 2023-09-23T03:28:30.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.587+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 101 ==== osd_op_reply(233 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.587+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.588+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.588+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:30.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.588+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:234 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff35180 con 0x56435febcc00 2023-09-23T03:28:30.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.589+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 48 ==== osd_op_reply(234 10000000000.01800000 [omap-get-header out=274b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.589+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 20 2023-09-23T03:28:30.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.589+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:30.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.589+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:235 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff34380 con 0x56435febcc00 2023-09-23T03:28:30.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.590+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 49 ==== osd_op_reply(235 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.590+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=30 frag=1*, r=0 2023-09-23T03:28:30.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.590+0000 7ff11803a5c0 20 datascan.read_dentry: 30_head not found in result 2023-09-23T03:28:30.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.590+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000213 at 0x0x10000000000/30 with size=2 bytes 2023-09-23T03:28:30.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.590+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:236 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff34700 con 0x56435febcc00 2023-09-23T03:28:30.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.593+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 50 ==== osd_op_reply(236 10000000000.01800000 [omap-set-vals] v128'21 uv21 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.594+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/30 pointing to 0x0x10000000213 2023-09-23T03:28:30.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.594+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:237 18.2a 18:576ad4f4::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecaa80 con 0x56435febcc00 2023-09-23T03:28:30.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.595+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 51 ==== osd_op_reply(237 [pgnls-filter start_epoch 128 out=101b] v128'21 uv21 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.595+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000245.00000000 2023-09-23T03:28:30.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.595+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000245.0 2023-09-23T03:28:30.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.595+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:238 18.2a 18:576ad4f4:::10000000245.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecbc00 con 0x56435febcc00 2023-09-23T03:28:30.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.597+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 52 ==== osd_op_reply(238 10000000245.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv20 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.597+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000245 2023-09-23T03:28:30.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.597+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/80 2023-09-23T03:28:30.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.597+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=80 2023-09-23T03:28:30.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.597+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:239 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb880 con 0x56435fdcbc00 2023-09-23T03:28:30.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.598+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 46 ==== osd_op_reply(239 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.598+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.598+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:240 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb500 con 0x56435febc000 2023-09-23T03:28:30.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.599+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 102 ==== osd_op_reply(240 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.599+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.599+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.599+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:30.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.599+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:241 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb180 con 0x56435febc000 2023-09-23T03:28:30.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.600+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 103 ==== osd_op_reply(241 10000000000.01000000 [omap-get-header out=274b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.600+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 21 2023-09-23T03:28:30.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.600+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:30.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.600+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:242 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca380 con 0x56435febc000 2023-09-23T03:28:30.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.602+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 104 ==== osd_op_reply(242 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.602+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=80 frag=0*, r=0 2023-09-23T03:28:30.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.602+0000 7ff11803a5c0 20 datascan.read_dentry: 80_head not found in result 2023-09-23T03:28:30.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.602+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000245 at 0x0x10000000000/80 with size=2 bytes 2023-09-23T03:28:30.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.602+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:243 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca000 con 0x56435febc000 2023-09-23T03:28:30.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.606+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 105 ==== osd_op_reply(243 10000000000.01000000 [omap-set-vals] v128'22 uv22 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.606+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/80 pointing to 0x0x10000000245 2023-09-23T03:28:30.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.606+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:244 18.2a 18:57fe7535::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fab3c00 con 0x56435febcc00 2023-09-23T03:28:30.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.607+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 53 ==== osd_op_reply(244 [pgnls-filter start_epoch 128 out=81b] v128'21 uv21 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.608+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 100000001f6.00000000 2023-09-23T03:28:30.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.608+0000 7ff11803a5c0 10 datascan.operator(): handling object 100000001f6.0 2023-09-23T03:28:30.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.608+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:245 18.2a 18:57fe7535:::100000001f6.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fab3880 con 0x56435febcc00 2023-09-23T03:28:30.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.609+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 54 ==== osd_op_reply(245 100000001f6.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv21 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.609+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f6 2023-09-23T03:28:30.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.609+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/1 2023-09-23T03:28:30.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.609+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=1 2023-09-23T03:28:30.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.609+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:246 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d880 con 0x56435fdcbc00 2023-09-23T03:28:30.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.610+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 47 ==== osd_op_reply(246 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.610+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.610+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:247 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d500 con 0x56435febc000 2023-09-23T03:28:30.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.611+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 106 ==== osd_op_reply(247 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.612+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.612+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.612+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:30.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.612+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:248 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d180 con 0x56435febc000 2023-09-23T03:28:30.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.613+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 107 ==== osd_op_reply(248 10000000000.01000000 [omap-get-header out=274b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.613+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 22 2023-09-23T03:28:30.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.613+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:30.615 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.613+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:249 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1c000 con 0x56435febc000 2023-09-23T03:28:30.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.614+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 108 ==== osd_op_reply(249 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.614+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=1 frag=0*, r=0 2023-09-23T03:28:30.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.614+0000 7ff11803a5c0 20 datascan.read_dentry: 1_head not found in result 2023-09-23T03:28:30.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.614+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f6 at 0x0x10000000000/1 with size=1 bytes 2023-09-23T03:28:30.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.615+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:250 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1dc00 con 0x56435febc000 2023-09-23T03:28:30.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.618+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 109 ==== osd_op_reply(250 10000000000.01000000 [omap-set-vals] v128'23 uv23 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.618+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/1 pointing to 0x0x100000001f6 2023-09-23T03:28:30.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.618+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:251 18.1a 18:58000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1ca80 con 0x56435fdcbc00 2023-09-23T03:28:30.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.619+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 48 ==== osd_op_reply(251 [pgnls-filter start_epoch 128 out=101b] v128'13 uv13 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.619+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000226.00000000 2023-09-23T03:28:30.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.619+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000226.0 2023-09-23T03:28:30.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.619+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:252 18.1a 18:582f7b2d:::10000000226.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1ce00 con 0x56435fdcbc00 2023-09-23T03:28:30.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.620+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 49 ==== osd_op_reply(252 10000000226.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.620+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000226 2023-09-23T03:28:30.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.620+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/49 2023-09-23T03:28:30.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.620+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=49 2023-09-23T03:28:30.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.620+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:253 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d880 con 0x56435fdcbc00 2023-09-23T03:28:30.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.621+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 50 ==== osd_op_reply(253 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.621+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.621+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:254 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1ca80 con 0x56435febc000 2023-09-23T03:28:30.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.622+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 110 ==== osd_op_reply(254 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.622+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.622+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.622+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:30.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.623+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:255 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fab3c00 con 0x56435febcc00 2023-09-23T03:28:30.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.624+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 55 ==== osd_op_reply(255 10000000000.01800000 [omap-get-header out=274b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.624+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 21 2023-09-23T03:28:30.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.624+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:30.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.624+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:256 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fab3880 con 0x56435febcc00 2023-09-23T03:28:30.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.625+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 56 ==== osd_op_reply(256 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.625+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=49 frag=1*, r=0 2023-09-23T03:28:30.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.625+0000 7ff11803a5c0 20 datascan.read_dentry: 49_head not found in result 2023-09-23T03:28:30.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.625+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000226 at 0x0x10000000000/49 with size=2 bytes 2023-09-23T03:28:30.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.625+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:257 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecaa80 con 0x56435febcc00 2023-09-23T03:28:30.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.629+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 57 ==== osd_op_reply(257 10000000000.01800000 [omap-set-vals] v128'22 uv22 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.629+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/49 pointing to 0x0x10000000226 2023-09-23T03:28:30.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.629+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:258 18.1a 18:5a2a1294::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecae00 con 0x56435fdcbc00 2023-09-23T03:28:30.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.630+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 51 ==== osd_op_reply(258 [pgnls-filter start_epoch 128 out=81b] v128'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.630+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 1000000023a.00000000 2023-09-23T03:28:30.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.630+0000 7ff11803a5c0 10 datascan.operator(): handling object 1000000023a.0 2023-09-23T03:28:30.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.630+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:259 18.1a 18:5a2a1294:::1000000023a.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecbc00 con 0x56435fdcbc00 2023-09-23T03:28:30.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.631+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 52 ==== osd_op_reply(259 1000000023a.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.631+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000023a 2023-09-23T03:28:30.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.631+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/69 2023-09-23T03:28:30.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.631+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=69 2023-09-23T03:28:30.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.631+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:260 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff35880 con 0x56435fdcbc00 2023-09-23T03:28:30.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.632+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 53 ==== osd_op_reply(260 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.632+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.632+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:261 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff35500 con 0x56435febc000 2023-09-23T03:28:30.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.633+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 111 ==== osd_op_reply(261 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.633+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.633+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.633+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:30.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.633+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:262 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff35c00 con 0x56435febc000 2023-09-23T03:28:30.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.635+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 112 ==== osd_op_reply(262 10000000000.01000000 [omap-get-header out=274b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.635+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 23 2023-09-23T03:28:30.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.635+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:30.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.635+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:263 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff35180 con 0x56435febc000 2023-09-23T03:28:30.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.636+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 113 ==== osd_op_reply(263 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.636+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=69 frag=0*, r=0 2023-09-23T03:28:30.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.636+0000 7ff11803a5c0 20 datascan.read_dentry: 69_head not found in result 2023-09-23T03:28:30.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.636+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000023a at 0x0x10000000000/69 with size=2 bytes 2023-09-23T03:28:30.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.636+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:264 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff34380 con 0x56435febc000 2023-09-23T03:28:30.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.640+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 114 ==== osd_op_reply(264 10000000000.01000000 [omap-set-vals] v128'24 uv24 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.640+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/69 pointing to 0x0x1000000023a 2023-09-23T03:28:30.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.640+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:265 18.3a 18:5c000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff34700 con 0x56435febc400 2023-09-23T03:28:30.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.641+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.0 v2:172.21.15.154:6800/2920938138 20 ==== osd_op_reply(265 [pgnls-filter start_epoch 128 out=49b] v126'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56435fd3efc0 con 0x56435febc400 2023-09-23T03:28:30.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.641+0000 7ff11105e640 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:266 18.6 18:60000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca700 con 0x56435fdca400 2023-09-23T03:28:30.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.642+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.7 v2:172.21.15.190:6824/3460136272 3 ==== osd_op_reply(266 [pgnls-filter start_epoch 128 out=81b] v128'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3ed80 con 0x56435fdca400 2023-09-23T03:28:30.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.642+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000228.00000000 2023-09-23T03:28:30.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.643+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000228.0 2023-09-23T03:28:30.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.643+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:267 18.6 18:614797eb:::10000000228.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe000 con 0x56435fdca400 2023-09-23T03:28:30.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.644+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.7 v2:172.21.15.190:6824/3460136272 4 ==== osd_op_reply(267 10000000228.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv10 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435fdca400 2023-09-23T03:28:30.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.644+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000228 2023-09-23T03:28:30.645 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.644+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/51 2023-09-23T03:28:30.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.644+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=51 2023-09-23T03:28:30.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.644+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:268 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe700 con 0x56435fdcbc00 2023-09-23T03:28:30.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.645+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 54 ==== osd_op_reply(268 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.645+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.646 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.645+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:269 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefea80 con 0x56435febc000 2023-09-23T03:28:30.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.646+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 115 ==== osd_op_reply(269 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.646+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.646+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.646+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:30.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.646+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:270 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefee00 con 0x56435febcc00 2023-09-23T03:28:30.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.647+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 58 ==== osd_op_reply(270 10000000000.01800000 [omap-get-header out=274b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.647+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 22 2023-09-23T03:28:30.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.647+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:30.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.647+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:271 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff180 con 0x56435febcc00 2023-09-23T03:28:30.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.648+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 59 ==== osd_op_reply(271 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.648+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=51 frag=1*, r=0 2023-09-23T03:28:30.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.648+0000 7ff11803a5c0 20 datascan.read_dentry: 51_head not found in result 2023-09-23T03:28:30.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.648+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000228 at 0x0x10000000000/51 with size=2 bytes 2023-09-23T03:28:30.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.648+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:272 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff500 con 0x56435febcc00 2023-09-23T03:28:30.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.652+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 60 ==== osd_op_reply(272 10000000000.01800000 [omap-set-vals] v128'23 uv23 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.652+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/51 pointing to 0x0x10000000228 2023-09-23T03:28:30.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.652+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:273 18.26 18:64000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff880 con 0x56435ff15800 2023-09-23T03:28:30.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.653+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.2 v2:172.21.15.154:6822/2884407243 6 ==== osd_op_reply(273 [pgnls-filter start_epoch 128 out=101b] v128'24 uv24 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3eb40 con 0x56435ff15800 2023-09-23T03:28:30.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.653+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 100000001ff.00000000 2023-09-23T03:28:30.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.653+0000 7ff11803a5c0 10 datascan.operator(): handling object 100000001ff.0 2023-09-23T03:28:30.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.653+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:274 18.26 18:658f2455:::100000001ff.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feffc00 con 0x56435ff15800 2023-09-23T03:28:30.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.655+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.2 v2:172.21.15.154:6822/2884407243 7 ==== osd_op_reply(274 100000001ff.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv22 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3eb40 con 0x56435ff15800 2023-09-23T03:28:30.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.655+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001ff 2023-09-23T03:28:30.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.655+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/10 2023-09-23T03:28:30.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.655+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=10 2023-09-23T03:28:30.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.655+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:275 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb500 con 0x56435fdcbc00 2023-09-23T03:28:30.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.656+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 55 ==== osd_op_reply(275 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.656+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.656+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:276 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb180 con 0x56435febc000 2023-09-23T03:28:30.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.657+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 116 ==== osd_op_reply(276 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.657+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.657+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.657+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:30.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.657+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:277 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca380 con 0x56435febc000 2023-09-23T03:28:30.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.658+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 117 ==== osd_op_reply(277 10000000000.01000000 [omap-get-header out=274b] v0'0 uv24 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.658+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 24 2023-09-23T03:28:30.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.658+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:30.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.658+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:278 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca000 con 0x56435febc000 2023-09-23T03:28:30.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.659+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 118 ==== osd_op_reply(278 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv24 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.660+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=10 frag=0*, r=0 2023-09-23T03:28:30.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.660+0000 7ff11803a5c0 20 datascan.read_dentry: 10_head not found in result 2023-09-23T03:28:30.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.660+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001ff at 0x0x10000000000/10 with size=2 bytes 2023-09-23T03:28:30.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.660+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:279 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca700 con 0x56435febc000 2023-09-23T03:28:30.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.664+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 119 ==== osd_op_reply(279 10000000000.01000000 [omap-set-vals] v128'25 uv25 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.664+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/10 pointing to 0x0x100000001ff 2023-09-23T03:28:30.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.664+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:280 18.26 18:65d05a5c::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff34a80 con 0x56435ff15800 2023-09-23T03:28:30.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.665+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.2 v2:172.21.15.154:6822/2884407243 8 ==== osd_op_reply(280 [pgnls-filter start_epoch 128 out=101b] v128'24 uv24 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3eb40 con 0x56435ff15800 2023-09-23T03:28:30.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.665+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000258.00000000 2023-09-23T03:28:30.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.665+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000258.0 2023-09-23T03:28:30.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.665+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:281 18.26 18:65d05a5c:::10000000258.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff34e00 con 0x56435ff15800 2023-09-23T03:28:30.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.666+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.2 v2:172.21.15.154:6822/2884407243 9 ==== osd_op_reply(281 10000000258.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv23 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3eb40 con 0x56435ff15800 2023-09-23T03:28:30.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.667+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000258 2023-09-23T03:28:30.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.667+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/99 2023-09-23T03:28:30.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.667+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=99 2023-09-23T03:28:30.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.667+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:282 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff35500 con 0x56435fdcbc00 2023-09-23T03:28:30.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.667+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 56 ==== osd_op_reply(282 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.667+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.667+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:283 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff35c00 con 0x56435febc000 2023-09-23T03:28:30.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.668+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 120 ==== osd_op_reply(283 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.669+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.669+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.669+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:30.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.669+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:284 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff35180 con 0x56435febcc00 2023-09-23T03:28:30.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.670+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 61 ==== osd_op_reply(284 10000000000.01800000 [omap-get-header out=274b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.670+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 23 2023-09-23T03:28:30.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.670+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:30.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.670+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:285 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff34380 con 0x56435febcc00 2023-09-23T03:28:30.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.671+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 62 ==== osd_op_reply(285 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.671+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=99 frag=1*, r=0 2023-09-23T03:28:30.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.671+0000 7ff11803a5c0 20 datascan.read_dentry: 99_head not found in result 2023-09-23T03:28:30.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.671+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000258 at 0x0x10000000000/99 with size=2 bytes 2023-09-23T03:28:30.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.671+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:286 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d500 con 0x56435febcc00 2023-09-23T03:28:30.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.674+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 63 ==== osd_op_reply(286 10000000000.01800000 [omap-set-vals] v128'24 uv24 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.674+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/99 pointing to 0x0x10000000258 2023-09-23T03:28:30.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.674+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:287 18.26 18:65e03fee::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d180 con 0x56435ff15800 2023-09-23T03:28:30.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.675+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.2 v2:172.21.15.154:6822/2884407243 10 ==== osd_op_reply(287 [pgnls-filter start_epoch 128 out=81b] v128'24 uv24 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3eb40 con 0x56435ff15800 2023-09-23T03:28:30.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.675+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000240.00000000 2023-09-23T03:28:30.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.675+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000240.0 2023-09-23T03:28:30.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.675+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:288 18.26 18:65e03fee:::10000000240.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1c000 con 0x56435ff15800 2023-09-23T03:28:30.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.676+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.2 v2:172.21.15.154:6822/2884407243 11 ==== osd_op_reply(288 10000000240.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv24 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3eb40 con 0x56435ff15800 2023-09-23T03:28:30.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.677+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000240 2023-09-23T03:28:30.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.677+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/75 2023-09-23T03:28:30.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.677+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=75 2023-09-23T03:28:30.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.677+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:289 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1ca80 con 0x56435fdcbc00 2023-09-23T03:28:30.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.678+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 57 ==== osd_op_reply(289 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.678+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.678+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:290 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe700 con 0x56435febc000 2023-09-23T03:28:30.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.679+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 121 ==== osd_op_reply(290 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.679+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.679+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.679+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:30.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.679+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:291 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefee00 con 0x56435febc000 2023-09-23T03:28:30.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.680+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 122 ==== osd_op_reply(291 10000000000.01000000 [omap-get-header out=274b] v0'0 uv25 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.680+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 25 2023-09-23T03:28:30.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.680+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:30.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.680+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:292 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff180 con 0x56435febc000 2023-09-23T03:28:30.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.681+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 123 ==== osd_op_reply(292 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv25 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.681+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=75 frag=0*, r=0 2023-09-23T03:28:30.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.681+0000 7ff11803a5c0 20 datascan.read_dentry: 75_head not found in result 2023-09-23T03:28:30.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.681+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000240 at 0x0x10000000000/75 with size=2 bytes 2023-09-23T03:28:30.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.681+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:293 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff500 con 0x56435febc000 2023-09-23T03:28:30.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.685+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 124 ==== osd_op_reply(293 10000000000.01000000 [omap-set-vals] v128'26 uv26 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.686+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/75 pointing to 0x0x10000000240 2023-09-23T03:28:30.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.686+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:294 18.16 18:68000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff880 con 0x56435fdcbc00 2023-09-23T03:28:30.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.686+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 58 ==== osd_op_reply(294 [pgnls-filter start_epoch 128 out=49b] v126'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.687+0000 7ff11185f640 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:295 18.36 18:6c000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5e000 con 0x56435fdcb000 2023-09-23T03:28:30.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.688+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.1 v2:172.21.15.154:6816/1267710005 3 ==== osd_op_reply(295 [pgnls-filter start_epoch 128 out=101b] v128'27 uv27 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3efc0 con 0x56435fdcb000 2023-09-23T03:28:30.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.688+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000255.00000000 2023-09-23T03:28:30.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.688+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000255.0 2023-09-23T03:28:30.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.688+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:296 18.36 18:6cdfec98:::10000000255.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe380 con 0x56435fdcb000 2023-09-23T03:28:30.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.689+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.1 v2:172.21.15.154:6816/1267710005 4 ==== osd_op_reply(296 10000000255.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv24 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3efc0 con 0x56435fdcb000 2023-09-23T03:28:30.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.689+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000255 2023-09-23T03:28:30.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.689+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/96 2023-09-23T03:28:30.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.689+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=96 2023-09-23T03:28:30.690 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.689+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:297 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecaa80 con 0x56435fdcbc00 2023-09-23T03:28:30.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.690+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 59 ==== osd_op_reply(297 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.690+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.691 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.690+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:298 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecae00 con 0x56435febc000 2023-09-23T03:28:30.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.691+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 125 ==== osd_op_reply(298 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.691+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.691+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.691+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:30.692 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.691+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:299 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb880 con 0x56435febc000 2023-09-23T03:28:30.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.692+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 126 ==== osd_op_reply(299 10000000000.01000000 [omap-get-header out=274b] v0'0 uv26 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.692+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 26 2023-09-23T03:28:30.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.692+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:30.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.692+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:300 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecbc00 con 0x56435febc000 2023-09-23T03:28:30.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.693+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 127 ==== osd_op_reply(300 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv26 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.693+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=96 frag=0*, r=0 2023-09-23T03:28:30.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.693+0000 7ff11803a5c0 20 datascan.read_dentry: 96_head not found in result 2023-09-23T03:28:30.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.693+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000255 at 0x0x10000000000/96 with size=2 bytes 2023-09-23T03:28:30.695 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.693+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:301 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fab3c00 con 0x56435febc000 2023-09-23T03:28:30.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.696+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 128 ==== osd_op_reply(301 10000000000.01000000 [omap-set-vals] v128'27 uv27 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.696+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/96 pointing to 0x0x10000000255 2023-09-23T03:28:30.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.697+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:302 18.36 18:6ee5e2c4::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fab3880 con 0x56435fdcb000 2023-09-23T03:28:30.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.698+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.1 v2:172.21.15.154:6816/1267710005 5 ==== osd_op_reply(302 [pgnls-filter start_epoch 128 out=101b] v128'27 uv27 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3efc0 con 0x56435fdcb000 2023-09-23T03:28:30.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.698+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 100000001f9.00000000 2023-09-23T03:28:30.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.698+0000 7ff11803a5c0 10 datascan.operator(): handling object 100000001f9.0 2023-09-23T03:28:30.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.698+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:303 18.36 18:6ee5e2c4:::100000001f9.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1c700 con 0x56435fdcb000 2023-09-23T03:28:30.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.699+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.1 v2:172.21.15.154:6816/1267710005 6 ==== osd_op_reply(303 100000001f9.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv25 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x56435fd3efc0 con 0x56435fdcb000 2023-09-23T03:28:30.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.699+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f9 2023-09-23T03:28:30.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.699+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/4 2023-09-23T03:28:30.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.699+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=4 2023-09-23T03:28:30.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.699+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:304 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d880 con 0x56435fdcbc00 2023-09-23T03:28:30.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.700+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 60 ==== osd_op_reply(304 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.700+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.700+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:305 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5ea80 con 0x56435febc000 2023-09-23T03:28:30.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.701+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 129 ==== osd_op_reply(305 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.701+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.701+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.701+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:30.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.701+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:306 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5ee00 con 0x56435febc000 2023-09-23T03:28:30.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.702+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 130 ==== osd_op_reply(306 10000000000.01000000 [omap-get-header out=274b] v0'0 uv27 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.702+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 27 2023-09-23T03:28:30.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.702+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:30.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.702+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:307 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5f180 con 0x56435febc000 2023-09-23T03:28:30.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.703+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 131 ==== osd_op_reply(307 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv27 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.703+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=4 frag=0*, r=0 2023-09-23T03:28:30.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.703+0000 7ff11803a5c0 20 datascan.read_dentry: 4_head not found in result 2023-09-23T03:28:30.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.703+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f9 at 0x0x10000000000/4 with size=1 bytes 2023-09-23T03:28:30.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.703+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:308 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5f500 con 0x56435febc000 2023-09-23T03:28:30.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.707+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 132 ==== osd_op_reply(308 10000000000.01000000 [omap-set-vals] v128'28 uv28 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.707+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/4 pointing to 0x0x100000001f9 2023-09-23T03:28:30.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.707+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:309 18.36 18:6f746908::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5f880 con 0x56435fdcb000 2023-09-23T03:28:30.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.708+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.1 v2:172.21.15.154:6816/1267710005 7 ==== osd_op_reply(309 [pgnls-filter start_epoch 128 out=101b] v128'27 uv27 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3efc0 con 0x56435fdcb000 2023-09-23T03:28:30.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.708+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 1000000022c.00000000 2023-09-23T03:28:30.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.708+0000 7ff11803a5c0 10 datascan.operator(): handling object 1000000022c.0 2023-09-23T03:28:30.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.708+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:310 18.36 18:6f746908:::1000000022c.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5fc00 con 0x56435fdcb000 2023-09-23T03:28:30.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.709+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.1 v2:172.21.15.154:6816/1267710005 8 ==== osd_op_reply(310 1000000022c.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv26 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3efc0 con 0x56435fdcb000 2023-09-23T03:28:30.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.709+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000022c 2023-09-23T03:28:30.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.709+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/55 2023-09-23T03:28:30.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.709+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=55 2023-09-23T03:28:30.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.709+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:311 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5ee00 con 0x56435fdcbc00 2023-09-23T03:28:30.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.710+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 61 ==== osd_op_reply(311 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.710+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.712 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.710+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:312 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5f180 con 0x56435febc000 2023-09-23T03:28:30.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.712+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 133 ==== osd_op_reply(312 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.712+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.712+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.712+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:30.713 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.712+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:313 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5f500 con 0x56435febc000 2023-09-23T03:28:30.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.713+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 134 ==== osd_op_reply(313 10000000000.01000000 [omap-get-header out=274b] v0'0 uv28 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.713+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 28 2023-09-23T03:28:30.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.713+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:30.714 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.713+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:314 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fab3c00 con 0x56435febc000 2023-09-23T03:28:30.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.714+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 135 ==== osd_op_reply(314 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv28 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.714+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=55 frag=0*, r=0 2023-09-23T03:28:30.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.714+0000 7ff11803a5c0 20 datascan.read_dentry: 55_head not found in result 2023-09-23T03:28:30.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.714+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000022c at 0x0x10000000000/55 with size=2 bytes 2023-09-23T03:28:30.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.714+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:315 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb180 con 0x56435febc000 2023-09-23T03:28:30.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.717+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 136 ==== osd_op_reply(315 10000000000.01000000 [omap-set-vals] v128'29 uv29 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.718+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/55 pointing to 0x0x1000000022c 2023-09-23T03:28:30.719 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.718+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:316 18.36 18:6fe38dc2::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca380 con 0x56435fdcb000 2023-09-23T03:28:30.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.719+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.1 v2:172.21.15.154:6816/1267710005 9 ==== osd_op_reply(316 [pgnls-filter start_epoch 128 out=81b] v128'27 uv27 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3efc0 con 0x56435fdcb000 2023-09-23T03:28:30.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.719+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2023-09-23T03:28:30.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.719+0000 7ff11803a5c0 10 datascan.operator(): handling object 100000001f7.0 2023-09-23T03:28:30.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.719+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:317 18.36 18:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca000 con 0x56435fdcb000 2023-09-23T03:28:30.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.720+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.1 v2:172.21.15.154:6816/1267710005 10 ==== osd_op_reply(317 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv27 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x56435fd3efc0 con 0x56435fdcb000 2023-09-23T03:28:30.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.720+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2023-09-23T03:28:30.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.720+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/2 2023-09-23T03:28:30.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.720+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=2 2023-09-23T03:28:30.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.720+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:318 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecae00 con 0x56435fdcbc00 2023-09-23T03:28:30.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.721+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 62 ==== osd_op_reply(318 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.721+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.721+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:319 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb880 con 0x56435febc000 2023-09-23T03:28:30.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.722+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 137 ==== osd_op_reply(319 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.722+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.722+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.722+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:30.723 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.722+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:320 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecbc00 con 0x56435febc000 2023-09-23T03:28:30.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.723+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 138 ==== osd_op_reply(320 10000000000.01000000 [omap-get-header out=274b] v0'0 uv29 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.723+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 29 2023-09-23T03:28:30.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.723+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:30.724 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.723+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:321 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe000 con 0x56435febc000 2023-09-23T03:28:30.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.724+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 139 ==== osd_op_reply(321 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv29 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.724+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=2 frag=0*, r=0 2023-09-23T03:28:30.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.724+0000 7ff11803a5c0 20 datascan.read_dentry: 2_head not found in result 2023-09-23T03:28:30.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.724+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000000/2 with size=1 bytes 2023-09-23T03:28:30.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.724+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:322 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefea80 con 0x56435febc000 2023-09-23T03:28:30.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.728+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 140 ==== osd_op_reply(322 10000000000.01000000 [omap-set-vals] v128'30 uv30 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.728+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/2 pointing to 0x0x100000001f7 2023-09-23T03:28:30.729 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.728+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:323 18.e 18:70000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe700 con 0x56435febc400 2023-09-23T03:28:30.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.729+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.0 v2:172.21.15.154:6800/2920938138 21 ==== osd_op_reply(323 [pgnls-filter start_epoch 128 out=81b] v128'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3efc0 con 0x56435febc400 2023-09-23T03:28:30.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.729+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 1000000022f.00000000 2023-09-23T03:28:30.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.729+0000 7ff11803a5c0 10 datascan.operator(): handling object 1000000022f.0 2023-09-23T03:28:30.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.729+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:324 18.e 18:71c13145:::1000000022f.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefee00 con 0x56435febc400 2023-09-23T03:28:30.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.731+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.0 v2:172.21.15.154:6800/2920938138 22 ==== osd_op_reply(324 1000000022f.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv8 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3efc0 con 0x56435febc400 2023-09-23T03:28:30.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.731+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000022f 2023-09-23T03:28:30.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.731+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/58 2023-09-23T03:28:30.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.731+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=58 2023-09-23T03:28:30.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.731+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:325 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff500 con 0x56435fdcbc00 2023-09-23T03:28:30.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.732+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 63 ==== osd_op_reply(325 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.732+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.732+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:326 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff35c00 con 0x56435febc000 2023-09-23T03:28:30.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.733+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 141 ==== osd_op_reply(326 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.733+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.733+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.733+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:30.735 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.733+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:327 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5fc00 con 0x56435febcc00 2023-09-23T03:28:30.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.734+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 64 ==== osd_op_reply(327 10000000000.01800000 [omap-get-header out=274b] v0'0 uv24 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.734+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 24 2023-09-23T03:28:30.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.734+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:30.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.734+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:328 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5ea80 con 0x56435febcc00 2023-09-23T03:28:30.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.735+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 65 ==== osd_op_reply(328 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv24 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.736+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=58 frag=1*, r=0 2023-09-23T03:28:30.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.736+0000 7ff11803a5c0 20 datascan.read_dentry: 58_head not found in result 2023-09-23T03:28:30.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.736+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000022f at 0x0x10000000000/58 with size=2 bytes 2023-09-23T03:28:30.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.736+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:329 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5f880 con 0x56435febcc00 2023-09-23T03:28:30.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.740+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 66 ==== osd_op_reply(329 10000000000.01800000 [omap-set-vals] v128'25 uv25 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.740+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/58 pointing to 0x0x1000000022f 2023-09-23T03:28:30.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.740+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:330 18.2e 18:74000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5e000 con 0x56435fdcbc00 2023-09-23T03:28:30.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.741+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 64 ==== osd_op_reply(330 [pgnls-filter start_epoch 128 out=101b] v128'19 uv19 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.741+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 1000000021c.00000000 2023-09-23T03:28:30.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.741+0000 7ff11803a5c0 10 datascan.operator(): handling object 1000000021c.0 2023-09-23T03:28:30.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.741+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:331 18.2e 18:74d13cfc:::1000000021c.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5ee00 con 0x56435fdcbc00 2023-09-23T03:28:30.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.742+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 65 ==== osd_op_reply(331 1000000021c.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.742+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000021c 2023-09-23T03:28:30.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.742+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/39 2023-09-23T03:28:30.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.742+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=39 2023-09-23T03:28:30.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.742+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:332 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca700 con 0x56435fdcbc00 2023-09-23T03:28:30.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.743+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 66 ==== osd_op_reply(332 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.743+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.743+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:333 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb500 con 0x56435febc000 2023-09-23T03:28:30.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.744+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 142 ==== osd_op_reply(333 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.744+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.745+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.745+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:30.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.745+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:334 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecaa80 con 0x56435febc000 2023-09-23T03:28:30.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.746+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 143 ==== osd_op_reply(334 10000000000.01000000 [omap-get-header out=274b] v0'0 uv30 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.746+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 30 2023-09-23T03:28:30.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.746+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:30.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.746+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:335 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe380 con 0x56435febc000 2023-09-23T03:28:30.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.747+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 144 ==== osd_op_reply(335 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv30 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.747+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=39 frag=0*, r=0 2023-09-23T03:28:30.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.747+0000 7ff11803a5c0 20 datascan.read_dentry: 39_head not found in result 2023-09-23T03:28:30.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.747+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000021c at 0x0x10000000000/39 with size=2 bytes 2023-09-23T03:28:30.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.747+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:336 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feffc00 con 0x56435febc000 2023-09-23T03:28:30.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.751+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 145 ==== osd_op_reply(336 10000000000.01000000 [omap-set-vals] v128'31 uv31 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.751+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/39 pointing to 0x0x1000000021c 2023-09-23T03:28:30.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.751+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:337 18.2e 18:7528f44f::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff880 con 0x56435fdcbc00 2023-09-23T03:28:30.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.752+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 67 ==== osd_op_reply(337 [pgnls-filter start_epoch 128 out=101b] v128'19 uv19 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.752+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000242.00000000 2023-09-23T03:28:30.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.752+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000242.0 2023-09-23T03:28:30.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.752+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:338 18.2e 18:7528f44f:::10000000242.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff35880 con 0x56435fdcbc00 2023-09-23T03:28:30.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.753+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 68 ==== osd_op_reply(338 10000000242.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv18 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.753+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000242 2023-09-23T03:28:30.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.753+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/77 2023-09-23T03:28:30.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.753+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=77 2023-09-23T03:28:30.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.753+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:339 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff34a80 con 0x56435fdcbc00 2023-09-23T03:28:30.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.754+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 69 ==== osd_op_reply(339 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.754+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.754+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:340 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff34000 con 0x56435febc000 2023-09-23T03:28:30.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.756+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 146 ==== osd_op_reply(340 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.756+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.756+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.756+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:30.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.756+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:341 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff34e00 con 0x56435febc000 2023-09-23T03:28:30.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.757+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 147 ==== osd_op_reply(341 10000000000.01000000 [omap-get-header out=274b] v0'0 uv31 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.757+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 31 2023-09-23T03:28:30.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.757+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:30.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.757+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:342 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff35500 con 0x56435febc000 2023-09-23T03:28:30.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.758+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 148 ==== osd_op_reply(342 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv31 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.758+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=77 frag=0*, r=0 2023-09-23T03:28:30.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.758+0000 7ff11803a5c0 20 datascan.read_dentry: 77_head not found in result 2023-09-23T03:28:30.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.758+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000242 at 0x0x10000000000/77 with size=2 bytes 2023-09-23T03:28:30.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.758+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:343 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff35180 con 0x56435febc000 2023-09-23T03:28:30.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.761+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 149 ==== osd_op_reply(343 10000000000.01000000 [omap-set-vals] v128'32 uv32 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.762+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/77 pointing to 0x0x10000000242 2023-09-23T03:28:30.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.762+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:344 18.2e 18:7611a124::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff34380 con 0x56435fdcbc00 2023-09-23T03:28:30.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.763+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 70 ==== osd_op_reply(344 [pgnls-filter start_epoch 128 out=81b] v128'19 uv19 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.763+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 1000000021e.00000000 2023-09-23T03:28:30.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.763+0000 7ff11803a5c0 10 datascan.operator(): handling object 1000000021e.0 2023-09-23T03:28:30.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.763+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:345 18.2e 18:7611a124:::1000000021e.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fab3880 con 0x56435fdcbc00 2023-09-23T03:28:30.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.764+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 71 ==== osd_op_reply(345 1000000021e.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv19 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.764+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000021e 2023-09-23T03:28:30.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.764+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/41 2023-09-23T03:28:30.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.764+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=41 2023-09-23T03:28:30.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.764+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:346 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1c380 con 0x56435fdcbc00 2023-09-23T03:28:30.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.765+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 72 ==== osd_op_reply(346 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.765+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.765+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:347 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1c700 con 0x56435febc000 2023-09-23T03:28:30.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.766+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 150 ==== osd_op_reply(347 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.767+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.767+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.767+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:30.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.767+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:348 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1ca80 con 0x56435febcc00 2023-09-23T03:28:30.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.768+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 67 ==== osd_op_reply(348 10000000000.01800000 [omap-get-header out=274b] v0'0 uv25 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.768+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 25 2023-09-23T03:28:30.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.768+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:30.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.768+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:349 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1ce00 con 0x56435febcc00 2023-09-23T03:28:30.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.769+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 68 ==== osd_op_reply(349 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv25 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.769+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=41 frag=1*, r=0 2023-09-23T03:28:30.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.769+0000 7ff11803a5c0 20 datascan.read_dentry: 41_head not found in result 2023-09-23T03:28:30.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.769+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000021e at 0x0x10000000000/41 with size=2 bytes 2023-09-23T03:28:30.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.769+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:350 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d180 con 0x56435febcc00 2023-09-23T03:28:30.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.773+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 69 ==== osd_op_reply(350 10000000000.01800000 [omap-set-vals] v128'26 uv26 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.773+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/41 pointing to 0x0x1000000021e 2023-09-23T03:28:30.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.773+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:351 18.1e 18:78000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d500 con 0x56435febd000 2023-09-23T03:28:30.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.774+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 13 ==== osd_op_reply(351 [pgnls-filter start_epoch 128 out=101b] v128'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:30.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.774+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 1000000020c.00000000 2023-09-23T03:28:30.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.774+0000 7ff11803a5c0 10 datascan.operator(): handling object 1000000020c.0 2023-09-23T03:28:30.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.774+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:352 18.1e 18:789b801b:::1000000020c.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d880 con 0x56435febd000 2023-09-23T03:28:30.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.775+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 14 ==== osd_op_reply(352 1000000020c.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:30.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.775+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000020c 2023-09-23T03:28:30.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.775+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/23 2023-09-23T03:28:30.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.775+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=23 2023-09-23T03:28:30.777 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.775+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:353 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1c700 con 0x56435fdcbc00 2023-09-23T03:28:30.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.777+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 73 ==== osd_op_reply(353 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.777+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.778 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.777+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:354 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d500 con 0x56435febc000 2023-09-23T03:28:30.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.778+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 151 ==== osd_op_reply(354 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.778+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.778+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.778+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:30.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.778+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:355 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1dc00 con 0x56435febcc00 2023-09-23T03:28:30.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.779+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 70 ==== osd_op_reply(355 10000000000.01800000 [omap-get-header out=274b] v0'0 uv26 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.779+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 26 2023-09-23T03:28:30.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.779+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:30.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.779+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:356 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1c000 con 0x56435febcc00 2023-09-23T03:28:30.782 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.780+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 71 ==== osd_op_reply(356 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv26 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.782 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.780+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=23 frag=1*, r=0 2023-09-23T03:28:30.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.780+0000 7ff11803a5c0 20 datascan.read_dentry: 23_head not found in result 2023-09-23T03:28:30.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.780+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000020c at 0x0x10000000000/23 with size=2 bytes 2023-09-23T03:28:30.783 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.781+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:357 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1c380 con 0x56435febcc00 2023-09-23T03:28:30.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.784+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 72 ==== osd_op_reply(357 10000000000.01800000 [omap-set-vals] v128'27 uv27 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.784+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/23 pointing to 0x0x1000000020c 2023-09-23T03:28:30.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.784+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:358 18.1e 18:7a9d9b47::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1ca80 con 0x56435febd000 2023-09-23T03:28:30.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.785+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 15 ==== osd_op_reply(358 [pgnls-filter start_epoch 128 out=101b] v128'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:30.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.785+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 1000000022b.00000000 2023-09-23T03:28:30.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.785+0000 7ff11803a5c0 10 datascan.operator(): handling object 1000000022b.0 2023-09-23T03:28:30.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.785+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:359 18.1e 18:7a9d9b47:::1000000022b.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fab3c00 con 0x56435febd000 2023-09-23T03:28:30.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.786+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 16 ==== osd_op_reply(359 1000000022b.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:30.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.787+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000022b 2023-09-23T03:28:30.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.787+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/54 2023-09-23T03:28:30.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.787+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=54 2023-09-23T03:28:30.788 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.787+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:360 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe000 con 0x56435fdcbc00 2023-09-23T03:28:30.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.788+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 74 ==== osd_op_reply(360 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.789 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.788+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.790 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.788+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:361 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefea80 con 0x56435febc000 2023-09-23T03:28:30.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.789+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 152 ==== osd_op_reply(361 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.789+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.789+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.789+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:30.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.789+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:362 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe380 con 0x56435febcc00 2023-09-23T03:28:30.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.790+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 73 ==== osd_op_reply(362 10000000000.01800000 [omap-get-header out=274b] v0'0 uv27 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.790+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 27 2023-09-23T03:28:30.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.790+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:30.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.790+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:363 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feffc00 con 0x56435febcc00 2023-09-23T03:28:30.793 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.791+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 74 ==== osd_op_reply(363 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv27 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.792+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=54 frag=1*, r=0 2023-09-23T03:28:30.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.792+0000 7ff11803a5c0 20 datascan.read_dentry: 54_head not found in result 2023-09-23T03:28:30.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.792+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000022b at 0x0x10000000000/54 with size=2 bytes 2023-09-23T03:28:30.794 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.792+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:364 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff500 con 0x56435febcc00 2023-09-23T03:28:30.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.796+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 75 ==== osd_op_reply(364 10000000000.01800000 [omap-set-vals] v128'28 uv28 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.796+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/54 pointing to 0x0x1000000022b 2023-09-23T03:28:30.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.796+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:365 18.1e 18:7ade2fd8::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff880 con 0x56435febd000 2023-09-23T03:28:30.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.797+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 17 ==== osd_op_reply(365 [pgnls-filter start_epoch 128 out=81b] v128'14 uv14 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:30.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.797+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 1000000024e.00000000 2023-09-23T03:28:30.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.797+0000 7ff11803a5c0 10 datascan.operator(): handling object 1000000024e.0 2023-09-23T03:28:30.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.797+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:366 18.1e 18:7ade2fd8:::1000000024e.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb180 con 0x56435febd000 2023-09-23T03:28:30.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.798+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 18 ==== osd_op_reply(366 1000000024e.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv14 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:30.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.799+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000024e 2023-09-23T03:28:30.800 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.799+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/89 2023-09-23T03:28:30.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.799+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=89 2023-09-23T03:28:30.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.799+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:367 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecbc00 con 0x56435fdcbc00 2023-09-23T03:28:30.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.799+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 75 ==== osd_op_reply(367 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.799+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.800+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:368 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb500 con 0x56435febc000 2023-09-23T03:28:30.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.801+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 153 ==== osd_op_reply(368 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.801+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.801+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.801+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:30.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.801+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:369 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecaa80 con 0x56435febcc00 2023-09-23T03:28:30.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.802+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 76 ==== osd_op_reply(369 10000000000.01800000 [omap-get-header out=274b] v0'0 uv28 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.802+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 28 2023-09-23T03:28:30.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.802+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:30.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.802+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:370 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecae00 con 0x56435febcc00 2023-09-23T03:28:30.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.803+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 77 ==== osd_op_reply(370 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv28 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.803+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=89 frag=1*, r=0 2023-09-23T03:28:30.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.803+0000 7ff11803a5c0 20 datascan.read_dentry: 89_head not found in result 2023-09-23T03:28:30.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.803+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000024e at 0x0x10000000000/89 with size=2 bytes 2023-09-23T03:28:30.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.803+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:371 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca380 con 0x56435febcc00 2023-09-23T03:28:30.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.807+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 78 ==== osd_op_reply(371 10000000000.01800000 [omap-set-vals] v128'29 uv29 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.807+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/89 pointing to 0x0x1000000024e 2023-09-23T03:28:30.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.807+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:372 18.3e 18:7c000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca700 con 0x56435fdcbc00 2023-09-23T03:28:30.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.808+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 76 ==== osd_op_reply(372 [pgnls-filter start_epoch 128 out=81b] v128'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.808+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000229.00000000 2023-09-23T03:28:30.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.808+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000229.0 2023-09-23T03:28:30.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.808+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:373 18.3e 18:7f918703:::10000000229.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5f180 con 0x56435fdcbc00 2023-09-23T03:28:30.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.809+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 77 ==== osd_op_reply(373 10000000229.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv7 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.809+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000229 2023-09-23T03:28:30.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.809+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/52 2023-09-23T03:28:30.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.809+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=52 2023-09-23T03:28:30.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.809+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:374 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5fc00 con 0x56435fdcbc00 2023-09-23T03:28:30.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.810+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 78 ==== osd_op_reply(374 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.810+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.810+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:375 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5ea80 con 0x56435febc000 2023-09-23T03:28:30.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.811+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 154 ==== osd_op_reply(375 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.811+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.811+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.811+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:30.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.811+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:376 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5f880 con 0x56435febc000 2023-09-23T03:28:30.814 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.812+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 155 ==== osd_op_reply(376 10000000000.01000000 [omap-get-header out=274b] v0'0 uv32 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.812+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 32 2023-09-23T03:28:30.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.812+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:30.815 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.812+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:377 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5e000 con 0x56435febc000 2023-09-23T03:28:30.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.813+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 156 ==== osd_op_reply(377 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv32 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.813+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=52 frag=0*, r=0 2023-09-23T03:28:30.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.813+0000 7ff11803a5c0 20 datascan.read_dentry: 52_head not found in result 2023-09-23T03:28:30.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.813+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000229 at 0x0x10000000000/52 with size=2 bytes 2023-09-23T03:28:30.816 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.813+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:378 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5ee00 con 0x56435febc000 2023-09-23T03:28:30.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.817+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 157 ==== osd_op_reply(378 10000000000.01000000 [omap-set-vals] v128'33 uv33 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.817+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/52 pointing to 0x0x10000000229 2023-09-23T03:28:30.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.817+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:379 18.1 18:80000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff34000 con 0x56435fdcbc00 2023-09-23T03:28:30.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.818+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 79 ==== osd_op_reply(379 [pgnls-filter start_epoch 128 out=81b] v128'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.819+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000224.00000000 2023-09-23T03:28:30.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.819+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000224.0 2023-09-23T03:28:30.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.819+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:380 18.1 18:805bc955:::10000000224.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff34380 con 0x56435fdcbc00 2023-09-23T03:28:30.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.820+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 80 ==== osd_op_reply(380 10000000224.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv8 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.820+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000224 2023-09-23T03:28:30.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.820+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/47 2023-09-23T03:28:30.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.820+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=47 2023-09-23T03:28:30.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.820+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:381 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff34a80 con 0x56435fdcbc00 2023-09-23T03:28:30.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.821+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 81 ==== osd_op_reply(381 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.821+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.821+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:382 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff34e00 con 0x56435febc000 2023-09-23T03:28:30.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.822+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 158 ==== osd_op_reply(382 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.822+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.822+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.822+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:30.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.822+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:383 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff35180 con 0x56435febcc00 2023-09-23T03:28:30.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.823+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 79 ==== osd_op_reply(383 10000000000.01800000 [omap-get-header out=274b] v0'0 uv29 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.823+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 29 2023-09-23T03:28:30.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.823+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:30.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.823+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:384 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff35500 con 0x56435febcc00 2023-09-23T03:28:30.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.824+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 80 ==== osd_op_reply(384 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv29 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.824+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=47 frag=1*, r=0 2023-09-23T03:28:30.828 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.824+0000 7ff11803a5c0 20 datascan.read_dentry: 47_head not found in result 2023-09-23T03:28:30.828 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.824+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000224 at 0x0x10000000000/47 with size=2 bytes 2023-09-23T03:28:30.828 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.824+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:385 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff35880 con 0x56435febcc00 2023-09-23T03:28:30.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.828+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 81 ==== osd_op_reply(385 10000000000.01800000 [omap-set-vals] v128'30 uv30 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.828+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/47 pointing to 0x0x10000000224 2023-09-23T03:28:30.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.828+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:386 18.21 18:84000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff35c00 con 0x56435febd000 2023-09-23T03:28:30.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.829+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 19 ==== osd_op_reply(386 [pgnls-filter start_epoch 128 out=49b] v126'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:30.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.829+0000 7ff11185f640 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:387 18.11 18:88000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5e380 con 0x56435fdcb000 2023-09-23T03:28:30.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.830+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.1 v2:172.21.15.154:6816/1267710005 11 ==== osd_op_reply(387 [pgnls-filter start_epoch 128 out=49b] v126'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56435fd3efc0 con 0x56435fdcb000 2023-09-23T03:28:30.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.831+0000 7ff11105e640 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:388 18.31 18:8c000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1ce00 con 0x56435febcc00 2023-09-23T03:28:30.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.832+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 82 ==== osd_op_reply(388 [pgnls-filter start_epoch 128 out=101b] v128'13 uv13 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.832+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000216.00000000 2023-09-23T03:28:30.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.832+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000216.0 2023-09-23T03:28:30.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.832+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:389 18.31 18:8c51d9b0:::10000000216.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff34000 con 0x56435febcc00 2023-09-23T03:28:30.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.833+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 83 ==== osd_op_reply(389 10000000216.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.833+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000216 2023-09-23T03:28:30.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.833+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/33 2023-09-23T03:28:30.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.833+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=33 2023-09-23T03:28:30.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.833+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:390 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5f180 con 0x56435fdcbc00 2023-09-23T03:28:30.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.834+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 82 ==== osd_op_reply(390 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.834+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.834+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:391 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5fc00 con 0x56435febc000 2023-09-23T03:28:30.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.836+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 159 ==== osd_op_reply(391 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.836+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.836+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.836+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:30.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.836+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:392 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecbc00 con 0x56435febcc00 2023-09-23T03:28:30.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.837+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 84 ==== osd_op_reply(392 10000000000.01800000 [omap-get-header out=274b] v0'0 uv30 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.837+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 30 2023-09-23T03:28:30.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.837+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:30.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.837+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:393 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecaa80 con 0x56435febcc00 2023-09-23T03:28:30.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.838+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 85 ==== osd_op_reply(393 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv30 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.838+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=33 frag=1*, r=0 2023-09-23T03:28:30.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.838+0000 7ff11803a5c0 20 datascan.read_dentry: 33_head not found in result 2023-09-23T03:28:30.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.838+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000216 at 0x0x10000000000/33 with size=2 bytes 2023-09-23T03:28:30.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.838+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:394 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecae00 con 0x56435febcc00 2023-09-23T03:28:30.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.842+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 86 ==== osd_op_reply(394 10000000000.01800000 [omap-set-vals] v128'31 uv31 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.842+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/33 pointing to 0x0x10000000216 2023-09-23T03:28:30.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.842+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:395 18.31 18:8fb7f7b7::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca380 con 0x56435febcc00 2023-09-23T03:28:30.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.843+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 87 ==== osd_op_reply(395 [pgnls-filter start_epoch 128 out=81b] v128'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.843+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000251.00000000 2023-09-23T03:28:30.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.843+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000251.0 2023-09-23T03:28:30.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.843+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:396 18.31 18:8fb7f7b7:::10000000251.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca700 con 0x56435febcc00 2023-09-23T03:28:30.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.844+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 88 ==== osd_op_reply(396 10000000251.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.844+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000251 2023-09-23T03:28:30.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.844+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/92 2023-09-23T03:28:30.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.844+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=92 2023-09-23T03:28:30.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.844+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:397 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe380 con 0x56435fdcbc00 2023-09-23T03:28:30.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.845+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 83 ==== osd_op_reply(397 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.845+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.845+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:398 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feffc00 con 0x56435febc000 2023-09-23T03:28:30.848 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.847+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 160 ==== osd_op_reply(398 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.848 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.847+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.848 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.847+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.848 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.847+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:30.848 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.847+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:399 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff500 con 0x56435febc000 2023-09-23T03:28:30.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.848+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 161 ==== osd_op_reply(399 10000000000.01000000 [omap-get-header out=274b] v0'0 uv33 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.848+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 33 2023-09-23T03:28:30.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.848+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:30.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.848+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:400 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1c000 con 0x56435febc000 2023-09-23T03:28:30.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.850+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 162 ==== osd_op_reply(400 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv33 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.850+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=92 frag=0*, r=0 2023-09-23T03:28:30.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.850+0000 7ff11803a5c0 20 datascan.read_dentry: 92_head not found in result 2023-09-23T03:28:30.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.850+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000251 at 0x0x10000000000/92 with size=2 bytes 2023-09-23T03:28:30.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.850+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:401 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1c380 con 0x56435febc000 2023-09-23T03:28:30.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.853+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 163 ==== osd_op_reply(401 10000000000.01000000 [omap-set-vals] v128'34 uv34 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.854 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.853+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/92 pointing to 0x0x10000000251 2023-09-23T03:28:30.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.853+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:402 18.9 18:90000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d880 con 0x56435fdcbc00 2023-09-23T03:28:30.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.854+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 84 ==== osd_op_reply(402 [pgnls-filter start_epoch 128 out=101b] v128'8 uv8 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.854+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 1000000024d.00000000 2023-09-23T03:28:30.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.854+0000 7ff11803a5c0 10 datascan.operator(): handling object 1000000024d.0 2023-09-23T03:28:30.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.854+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:403 18.9 18:915bff82:::1000000024d.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d500 con 0x56435fdcbc00 2023-09-23T03:28:30.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.855+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 85 ==== osd_op_reply(403 1000000024d.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv7 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.855+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000024d 2023-09-23T03:28:30.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.855+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/88 2023-09-23T03:28:30.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.855+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=88 2023-09-23T03:28:30.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.855+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:404 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1c000 con 0x56435fdcbc00 2023-09-23T03:28:30.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.856+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 86 ==== osd_op_reply(404 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.856+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.857+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:405 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1c380 con 0x56435febc000 2023-09-23T03:28:30.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.858+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 164 ==== osd_op_reply(405 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.858+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.858+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.858+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:30.859 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.858+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:406 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefea80 con 0x56435febc000 2023-09-23T03:28:30.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.859+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 165 ==== osd_op_reply(406 10000000000.01000000 [omap-get-header out=274b] v0'0 uv34 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.859+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 34 2023-09-23T03:28:30.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.859+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:30.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.859+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:407 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff880 con 0x56435febc000 2023-09-23T03:28:30.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.860+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 166 ==== osd_op_reply(407 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv34 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.860+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=88 frag=0*, r=0 2023-09-23T03:28:30.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.860+0000 7ff11803a5c0 20 datascan.read_dentry: 88_head not found in result 2023-09-23T03:28:30.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.860+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000024d at 0x0x10000000000/88 with size=2 bytes 2023-09-23T03:28:30.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.860+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:408 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feffc00 con 0x56435febc000 2023-09-23T03:28:30.865 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.864+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 167 ==== osd_op_reply(408 10000000000.01000000 [omap-set-vals] v128'35 uv35 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.864+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/88 pointing to 0x0x1000000024d 2023-09-23T03:28:30.866 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.864+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:409 18.9 18:91e366d7::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff500 con 0x56435fdcbc00 2023-09-23T03:28:30.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.865+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 87 ==== osd_op_reply(409 [pgnls-filter start_epoch 128 out=81b] v128'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.865+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000230.00000000 2023-09-23T03:28:30.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.865+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000230.0 2023-09-23T03:28:30.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.865+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:410 18.9 18:91e366d7:::10000000230.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5ea80 con 0x56435fdcbc00 2023-09-23T03:28:30.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.866+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 88 ==== osd_op_reply(410 10000000230.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv8 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.866+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000230 2023-09-23T03:28:30.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.866+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/59 2023-09-23T03:28:30.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.866+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=59 2023-09-23T03:28:30.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.867+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:411 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5e000 con 0x56435fdcbc00 2023-09-23T03:28:30.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.868+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 89 ==== osd_op_reply(411 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.868+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.869 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.868+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:412 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5ee00 con 0x56435febc000 2023-09-23T03:28:30.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.869+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 168 ==== osd_op_reply(412 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.869+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.869+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.869+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:30.870 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.869+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:413 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5fc00 con 0x56435febc000 2023-09-23T03:28:30.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.870+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 169 ==== osd_op_reply(413 10000000000.01000000 [omap-get-header out=274b] v0'0 uv35 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.870+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 35 2023-09-23T03:28:30.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.870+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:30.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.870+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:414 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff34e00 con 0x56435febc000 2023-09-23T03:28:30.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.872+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 170 ==== osd_op_reply(414 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv35 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.872+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=59 frag=0*, r=0 2023-09-23T03:28:30.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.872+0000 7ff11803a5c0 20 datascan.read_dentry: 59_head not found in result 2023-09-23T03:28:30.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.872+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000230 at 0x0x10000000000/59 with size=2 bytes 2023-09-23T03:28:30.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.872+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:415 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff35c00 con 0x56435febc000 2023-09-23T03:28:30.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.876+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 171 ==== osd_op_reply(415 10000000000.01000000 [omap-set-vals] v128'36 uv36 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.876+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/59 pointing to 0x0x10000000230 2023-09-23T03:28:30.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.876+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:416 18.29 18:94000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb880 con 0x56435fdca400 2023-09-23T03:28:30.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.877+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.7 v2:172.21.15.190:6824/3460136272 5 ==== osd_op_reply(416 [pgnls-filter start_epoch 128 out=101b] v128'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3ed80 con 0x56435fdca400 2023-09-23T03:28:30.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.878+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 1000000024a.00000000 2023-09-23T03:28:30.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.878+0000 7ff11803a5c0 10 datascan.operator(): handling object 1000000024a.0 2023-09-23T03:28:30.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.878+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:417 18.29 18:94c02207:::1000000024a.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb180 con 0x56435fdca400 2023-09-23T03:28:30.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.879+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.7 v2:172.21.15.190:6824/3460136272 6 ==== osd_op_reply(417 1000000024a.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435fdca400 2023-09-23T03:28:30.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.879+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000024a 2023-09-23T03:28:30.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.879+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/85 2023-09-23T03:28:30.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.879+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=85 2023-09-23T03:28:30.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.879+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:418 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fab3880 con 0x56435fdcbc00 2023-09-23T03:28:30.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.880+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 90 ==== osd_op_reply(418 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.881 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.880+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.880+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:419 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fab3c00 con 0x56435febc000 2023-09-23T03:28:30.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.882+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 172 ==== osd_op_reply(419 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.882+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.882+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.882+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:30.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.882+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:420 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe380 con 0x56435febc000 2023-09-23T03:28:30.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.883+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 173 ==== osd_op_reply(420 10000000000.01000000 [omap-get-header out=274b] v0'0 uv36 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.883+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 36 2023-09-23T03:28:30.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.883+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:30.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.883+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:421 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe000 con 0x56435febc000 2023-09-23T03:28:30.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.884+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 174 ==== osd_op_reply(421 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv36 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.884+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=85 frag=0*, r=0 2023-09-23T03:28:30.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.884+0000 7ff11803a5c0 20 datascan.read_dentry: 85_head not found in result 2023-09-23T03:28:30.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.884+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000024a at 0x0x10000000000/85 with size=2 bytes 2023-09-23T03:28:30.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.884+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:422 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff500 con 0x56435febc000 2023-09-23T03:28:30.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.887+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 175 ==== osd_op_reply(422 10000000000.01000000 [omap-set-vals] v128'37 uv37 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.887+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/85 pointing to 0x0x1000000024a 2023-09-23T03:28:30.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.887+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:423 18.29 18:95e8e66d::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d500 con 0x56435fdca400 2023-09-23T03:28:30.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.889+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.7 v2:172.21.15.190:6824/3460136272 7 ==== osd_op_reply(423 [pgnls-filter start_epoch 128 out=101b] v128'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3ed80 con 0x56435fdca400 2023-09-23T03:28:30.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.889+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000256.00000000 2023-09-23T03:28:30.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.889+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000256.0 2023-09-23T03:28:30.890 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.889+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:424 18.29 18:95e8e66d:::10000000256.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1ca80 con 0x56435fdca400 2023-09-23T03:28:30.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.890+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.7 v2:172.21.15.190:6824/3460136272 8 ==== osd_op_reply(424 10000000256.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435fdca400 2023-09-23T03:28:30.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.890+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000256 2023-09-23T03:28:30.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.890+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/97 2023-09-23T03:28:30.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.890+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=97 2023-09-23T03:28:30.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.890+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:425 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1ce00 con 0x56435fdcbc00 2023-09-23T03:28:30.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.891+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 91 ==== osd_op_reply(425 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.891+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.891+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:426 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1c000 con 0x56435febc000 2023-09-23T03:28:30.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.892+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 176 ==== osd_op_reply(426 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.892+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.892+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.892+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:30.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.892+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:427 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca700 con 0x56435febc000 2023-09-23T03:28:30.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.893+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 177 ==== osd_op_reply(427 10000000000.01000000 [omap-get-header out=274b] v0'0 uv37 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.893+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 37 2023-09-23T03:28:30.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.893+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:30.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.893+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:428 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca380 con 0x56435febc000 2023-09-23T03:28:30.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.894+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 178 ==== osd_op_reply(428 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv37 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.894+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=97 frag=0*, r=0 2023-09-23T03:28:30.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.894+0000 7ff11803a5c0 20 datascan.read_dentry: 97_head not found in result 2023-09-23T03:28:30.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.894+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000256 at 0x0x10000000000/97 with size=2 bytes 2023-09-23T03:28:30.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.894+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:429 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecae00 con 0x56435febc000 2023-09-23T03:28:30.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.898+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 179 ==== osd_op_reply(429 10000000000.01000000 [omap-set-vals] v128'38 uv38 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.898+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/97 pointing to 0x0x10000000256 2023-09-23T03:28:30.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.898+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:430 18.29 18:97401066::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecaa80 con 0x56435fdca400 2023-09-23T03:28:30.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.899+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.7 v2:172.21.15.190:6824/3460136272 9 ==== osd_op_reply(430 [pgnls-filter start_epoch 128 out=81b] v128'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3ed80 con 0x56435fdca400 2023-09-23T03:28:30.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.899+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000247.00000000 2023-09-23T03:28:30.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.899+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000247.0 2023-09-23T03:28:30.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.899+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:431 18.29 18:97401066:::10000000247.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecbc00 con 0x56435fdca400 2023-09-23T03:28:30.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.900+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.7 v2:172.21.15.190:6824/3460136272 10 ==== osd_op_reply(431 10000000247.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435fdca400 2023-09-23T03:28:30.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.900+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000247 2023-09-23T03:28:30.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.900+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/82 2023-09-23T03:28:30.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.900+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=82 2023-09-23T03:28:30.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.900+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:432 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5f500 con 0x56435fdcbc00 2023-09-23T03:28:30.902 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.901+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 92 ==== osd_op_reply(432 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.901+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.901+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:433 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff34000 con 0x56435febc000 2023-09-23T03:28:30.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.903+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 180 ==== osd_op_reply(433 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.903+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.903+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.903+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:30.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.903+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:434 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff35880 con 0x56435febcc00 2023-09-23T03:28:30.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.904+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 89 ==== osd_op_reply(434 10000000000.01800000 [omap-get-header out=274b] v0'0 uv31 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.904+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 31 2023-09-23T03:28:30.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.904+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:30.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.904+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:435 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff35500 con 0x56435febcc00 2023-09-23T03:28:30.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.905+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 90 ==== osd_op_reply(435 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv31 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.905+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=82 frag=1*, r=0 2023-09-23T03:28:30.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.905+0000 7ff11803a5c0 20 datascan.read_dentry: 82_head not found in result 2023-09-23T03:28:30.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.905+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000247 at 0x0x10000000000/82 with size=2 bytes 2023-09-23T03:28:30.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.905+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:436 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff35180 con 0x56435febcc00 2023-09-23T03:28:30.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.909+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 91 ==== osd_op_reply(436 10000000000.01800000 [omap-set-vals] v128'32 uv32 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.909+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/82 pointing to 0x0x10000000247 2023-09-23T03:28:30.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.909+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:437 18.19 18:98000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff34a80 con 0x56435febcc00 2023-09-23T03:28:30.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.910+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 92 ==== osd_op_reply(437 [pgnls-filter start_epoch 128 out=101b] v128'16 uv16 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.910+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000200.00000000 2023-09-23T03:28:30.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.910+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000200.0 2023-09-23T03:28:30.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.910+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:438 18.19 18:992a6246:::10000000200.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff34380 con 0x56435febcc00 2023-09-23T03:28:30.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.911+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 93 ==== osd_op_reply(438 10000000200.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.911+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000200 2023-09-23T03:28:30.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.911+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/11 2023-09-23T03:28:30.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.911+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=11 2023-09-23T03:28:30.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.911+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:439 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb880 con 0x56435fdcbc00 2023-09-23T03:28:30.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.912+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 93 ==== osd_op_reply(439 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.912+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.912+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:440 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb500 con 0x56435febc000 2023-09-23T03:28:30.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.914+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 181 ==== osd_op_reply(440 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.914+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.914+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.914+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:30.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.914+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:441 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb180 con 0x56435febc000 2023-09-23T03:28:30.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.915+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 182 ==== osd_op_reply(441 10000000000.01000000 [omap-get-header out=274b] v0'0 uv38 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.915+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 38 2023-09-23T03:28:30.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.915+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:30.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.915+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:442 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca700 con 0x56435febc000 2023-09-23T03:28:30.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.916+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 183 ==== osd_op_reply(442 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv38 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.916+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=11 frag=0*, r=0 2023-09-23T03:28:30.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.916+0000 7ff11803a5c0 20 datascan.read_dentry: 11_head not found in result 2023-09-23T03:28:30.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.916+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000200 at 0x0x10000000000/11 with size=2 bytes 2023-09-23T03:28:30.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.916+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:443 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca380 con 0x56435febc000 2023-09-23T03:28:30.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.920+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 184 ==== osd_op_reply(443 10000000000.01000000 [omap-set-vals] v128'39 uv39 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.920+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/11 pointing to 0x0x10000000200 2023-09-23T03:28:30.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.920+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:444 18.19 18:997b7129::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecae00 con 0x56435febcc00 2023-09-23T03:28:30.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.921+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 94 ==== osd_op_reply(444 [pgnls-filter start_epoch 128 out=81b] v128'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.921+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 100000001f8.00000000 2023-09-23T03:28:30.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.921+0000 7ff11803a5c0 10 datascan.operator(): handling object 100000001f8.0 2023-09-23T03:28:30.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.921+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:445 18.19 18:997b7129:::100000001f8.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1c380 con 0x56435febcc00 2023-09-23T03:28:30.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.923+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 95 ==== osd_op_reply(445 100000001f8.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.923+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f8 2023-09-23T03:28:30.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.923+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/3 2023-09-23T03:28:30.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.923+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=3 2023-09-23T03:28:30.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.923+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:446 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d880 con 0x56435fdcbc00 2023-09-23T03:28:30.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.924+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 94 ==== osd_op_reply(446 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.924+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.924+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:447 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1ca80 con 0x56435febc000 2023-09-23T03:28:30.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.925+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 185 ==== osd_op_reply(447 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.925+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.925+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.925+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:30.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.925+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:448 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1c000 con 0x56435febc000 2023-09-23T03:28:30.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.926+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 186 ==== osd_op_reply(448 10000000000.01000000 [omap-get-header out=274b] v0'0 uv39 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.927+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 39 2023-09-23T03:28:30.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.927+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:30.928 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.927+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:449 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefea80 con 0x56435febc000 2023-09-23T03:28:30.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.928+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 187 ==== osd_op_reply(449 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv39 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.928+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=3 frag=0*, r=0 2023-09-23T03:28:30.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.928+0000 7ff11803a5c0 20 datascan.read_dentry: 3_head not found in result 2023-09-23T03:28:30.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.928+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f8 at 0x0x10000000000/3 with size=1 bytes 2023-09-23T03:28:30.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.928+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:450 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff880 con 0x56435febc000 2023-09-23T03:28:30.932 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.931+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 188 ==== osd_op_reply(450 10000000000.01000000 [omap-set-vals] v128'40 uv40 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.931+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/3 pointing to 0x0x100000001f8 2023-09-23T03:28:30.933 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.931+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:451 18.39 18:9c000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feffc00 con 0x56435febc400 2023-09-23T03:28:30.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.932+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.0 v2:172.21.15.154:6800/2920938138 23 ==== osd_op_reply(451 [pgnls-filter start_epoch 128 out=81b] v128'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3efc0 con 0x56435febc400 2023-09-23T03:28:30.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.932+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000222.00000000 2023-09-23T03:28:30.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.932+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000222.0 2023-09-23T03:28:30.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.933+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:452 18.39 18:9f1351c3:::10000000222.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe380 con 0x56435febc400 2023-09-23T03:28:30.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.933+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.0 v2:172.21.15.154:6800/2920938138 24 ==== osd_op_reply(452 10000000222.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv8 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3efc0 con 0x56435febc400 2023-09-23T03:28:30.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.934+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000222 2023-09-23T03:28:30.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.934+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/45 2023-09-23T03:28:30.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.934+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=45 2023-09-23T03:28:30.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.934+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:453 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff500 con 0x56435fdcbc00 2023-09-23T03:28:30.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.935+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 95 ==== osd_op_reply(453 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.935+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.935+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:454 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fab3c00 con 0x56435febc000 2023-09-23T03:28:30.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.936+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 189 ==== osd_op_reply(454 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.936+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.936+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.936+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:30.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.936+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:455 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff34e00 con 0x56435febcc00 2023-09-23T03:28:30.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.937+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 96 ==== osd_op_reply(455 10000000000.01800000 [omap-get-header out=274b] v0'0 uv32 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.937+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 32 2023-09-23T03:28:30.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.937+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:30.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.937+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:456 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff35c00 con 0x56435febcc00 2023-09-23T03:28:30.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.938+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 97 ==== osd_op_reply(456 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv32 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.939+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=45 frag=1*, r=0 2023-09-23T03:28:30.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.939+0000 7ff11803a5c0 20 datascan.read_dentry: 45_head not found in result 2023-09-23T03:28:30.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.939+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000222 at 0x0x10000000000/45 with size=2 bytes 2023-09-23T03:28:30.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.939+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:457 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5ee00 con 0x56435febcc00 2023-09-23T03:28:30.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.942+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 98 ==== osd_op_reply(457 10000000000.01800000 [omap-set-vals] v128'33 uv33 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.942+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/45 pointing to 0x0x10000000222 2023-09-23T03:28:30.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.942+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:458 18.5 18:a0000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5fc00 con 0x56435fdcb000 2023-09-23T03:28:30.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.943+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.1 v2:172.21.15.154:6816/1267710005 12 ==== osd_op_reply(458 [pgnls-filter start_epoch 128 out=101b] v128'11 uv11 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3efc0 con 0x56435fdcb000 2023-09-23T03:28:30.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.943+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000204.00000000 2023-09-23T03:28:30.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.943+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000204.0 2023-09-23T03:28:30.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.943+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:459 18.5 18:a0beda95:::10000000204.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6c000 con 0x56435fdcb000 2023-09-23T03:28:30.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.944+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.1 v2:172.21.15.154:6816/1267710005 13 ==== osd_op_reply(459 10000000204.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv10 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3efc0 con 0x56435fdcb000 2023-09-23T03:28:30.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.945+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000204 2023-09-23T03:28:30.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.945+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/15 2023-09-23T03:28:30.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.945+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=15 2023-09-23T03:28:30.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.945+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:460 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6c700 con 0x56435fdcbc00 2023-09-23T03:28:30.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.945+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 96 ==== osd_op_reply(460 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.946+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.947 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.946+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:461 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6ca80 con 0x56435febc000 2023-09-23T03:28:30.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.947+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 190 ==== osd_op_reply(461 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.947+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.947+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.947+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:30.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.947+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:462 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6ce00 con 0x56435febc000 2023-09-23T03:28:30.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.948+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 191 ==== osd_op_reply(462 10000000000.01000000 [omap-get-header out=274b] v0'0 uv40 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.948+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 40 2023-09-23T03:28:30.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.948+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:30.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.948+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:463 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6d180 con 0x56435febc000 2023-09-23T03:28:30.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.949+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 192 ==== osd_op_reply(463 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv40 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.949+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=15 frag=0*, r=0 2023-09-23T03:28:30.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.949+0000 7ff11803a5c0 20 datascan.read_dentry: 15_head not found in result 2023-09-23T03:28:30.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.949+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000204 at 0x0x10000000000/15 with size=2 bytes 2023-09-23T03:28:30.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.949+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:464 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6d500 con 0x56435febc000 2023-09-23T03:28:30.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.953+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 193 ==== osd_op_reply(464 10000000000.01000000 [omap-set-vals] v128'41 uv41 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.953+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/15 pointing to 0x0x10000000204 2023-09-23T03:28:30.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.953+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:465 18.5 18:a3a50212::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6d880 con 0x56435fdcb000 2023-09-23T03:28:30.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.954+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.1 v2:172.21.15.154:6816/1267710005 14 ==== osd_op_reply(465 [pgnls-filter start_epoch 128 out=81b] v128'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3efc0 con 0x56435fdcb000 2023-09-23T03:28:30.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.954+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000254.00000000 2023-09-23T03:28:30.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.954+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000254.0 2023-09-23T03:28:30.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.954+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:466 18.5 18:a3a50212:::10000000254.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6dc00 con 0x56435fdcb000 2023-09-23T03:28:30.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.955+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.1 v2:172.21.15.154:6816/1267710005 15 ==== osd_op_reply(466 10000000254.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3efc0 con 0x56435fdcb000 2023-09-23T03:28:30.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.955+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000254 2023-09-23T03:28:30.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.955+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/95 2023-09-23T03:28:30.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.955+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=95 2023-09-23T03:28:30.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.955+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:467 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6e380 con 0x56435fdcbc00 2023-09-23T03:28:30.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.956+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 97 ==== osd_op_reply(467 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.956+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.956+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:468 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6e700 con 0x56435febc000 2023-09-23T03:28:30.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.958+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 194 ==== osd_op_reply(468 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.958+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.958+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.958+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:30.959 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.958+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:469 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6ea80 con 0x56435febc000 2023-09-23T03:28:30.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.959+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 195 ==== osd_op_reply(469 10000000000.01000000 [omap-get-header out=274b] v0'0 uv41 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.959+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 41 2023-09-23T03:28:30.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.959+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:30.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.959+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:470 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6ee00 con 0x56435febc000 2023-09-23T03:28:30.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.960+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 196 ==== osd_op_reply(470 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv41 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.960+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=95 frag=0*, r=0 2023-09-23T03:28:30.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.960+0000 7ff11803a5c0 20 datascan.read_dentry: 95_head not found in result 2023-09-23T03:28:30.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.960+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000254 at 0x0x10000000000/95 with size=2 bytes 2023-09-23T03:28:30.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.960+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:471 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6f180 con 0x56435febc000 2023-09-23T03:28:30.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.964+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 197 ==== osd_op_reply(471 10000000000.01000000 [omap-set-vals] v128'42 uv42 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.964+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/95 pointing to 0x0x10000000254 2023-09-23T03:28:30.965 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.964+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:472 18.25 18:a4000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6f500 con 0x56435febc000 2023-09-23T03:28:30.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.965+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 198 ==== osd_op_reply(472 [pgnls-filter start_epoch 128 out=101b] v128'12 uv12 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.965+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000237.00000000 2023-09-23T03:28:30.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.965+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000237.0 2023-09-23T03:28:30.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.965+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:473 18.25 18:a53878a9:::10000000237.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6f880 con 0x56435febc000 2023-09-23T03:28:30.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.966+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 199 ==== osd_op_reply(473 10000000237.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.966+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000237 2023-09-23T03:28:30.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.966+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/66 2023-09-23T03:28:30.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.966+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=66 2023-09-23T03:28:30.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.966+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:474 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6e700 con 0x56435fdcbc00 2023-09-23T03:28:30.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.967+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 98 ==== osd_op_reply(474 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.967+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.968+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:475 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6ea80 con 0x56435febc000 2023-09-23T03:28:30.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.969+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 200 ==== osd_op_reply(475 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.969+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.969+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.969+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:30.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.969+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:476 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6ca80 con 0x56435febc000 2023-09-23T03:28:30.971 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.970+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 201 ==== osd_op_reply(476 10000000000.01000000 [omap-get-header out=274b] v0'0 uv42 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.970+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 42 2023-09-23T03:28:30.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.970+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:30.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.970+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:477 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6ce00 con 0x56435febc000 2023-09-23T03:28:30.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.971+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 202 ==== osd_op_reply(477 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv42 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.972+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=66 frag=0*, r=0 2023-09-23T03:28:30.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.972+0000 7ff11803a5c0 20 datascan.read_dentry: 66_head not found in result 2023-09-23T03:28:30.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.972+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000237 at 0x0x10000000000/66 with size=2 bytes 2023-09-23T03:28:30.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.972+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:478 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6d180 con 0x56435febc000 2023-09-23T03:28:30.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.975+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 203 ==== osd_op_reply(478 10000000000.01000000 [omap-set-vals] v128'43 uv43 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.975+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/66 pointing to 0x0x10000000237 2023-09-23T03:28:30.976 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.975+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:479 18.25 18:a7ae8a0f::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6d500 con 0x56435febc000 2023-09-23T03:28:30.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.976+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 204 ==== osd_op_reply(479 [pgnls-filter start_epoch 128 out=81b] v128'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.976+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000243.00000000 2023-09-23T03:28:30.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.976+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000243.0 2023-09-23T03:28:30.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.976+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:480 18.25 18:a7ae8a0f:::10000000243.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6c700 con 0x56435febc000 2023-09-23T03:28:30.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.977+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 205 ==== osd_op_reply(480 10000000243.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.977+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000243 2023-09-23T03:28:30.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.977+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/78 2023-09-23T03:28:30.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.977+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=78 2023-09-23T03:28:30.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.977+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:481 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5f880 con 0x56435fdcbc00 2023-09-23T03:28:30.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.978+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 99 ==== osd_op_reply(481 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.978+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.980 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.978+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:482 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5ea80 con 0x56435febc000 2023-09-23T03:28:30.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.979+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 206 ==== osd_op_reply(482 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.979+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.979+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.979+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:30.981 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.980+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:483 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5e000 con 0x56435febcc00 2023-09-23T03:28:30.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.981+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 99 ==== osd_op_reply(483 10000000000.01800000 [omap-get-header out=274b] v0'0 uv33 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.981+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 33 2023-09-23T03:28:30.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.981+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:30.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.981+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:484 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5f500 con 0x56435febcc00 2023-09-23T03:28:30.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.982+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 100 ==== osd_op_reply(484 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv33 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.982+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=78 frag=1*, r=0 2023-09-23T03:28:30.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.982+0000 7ff11803a5c0 20 datascan.read_dentry: 78_head not found in result 2023-09-23T03:28:30.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.982+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000243 at 0x0x10000000000/78 with size=2 bytes 2023-09-23T03:28:30.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.982+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:485 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5ee00 con 0x56435febcc00 2023-09-23T03:28:30.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.986+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 101 ==== osd_op_reply(485 10000000000.01800000 [omap-set-vals] v128'34 uv34 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.986+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/78 pointing to 0x0x10000000243 2023-09-23T03:28:30.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.986+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:486 18.15 18:a8000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefea80 con 0x56435febd000 2023-09-23T03:28:30.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.987+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 20 ==== osd_op_reply(486 [pgnls-filter start_epoch 128 out=81b] v128'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:30.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.987+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 1000000020d.00000000 2023-09-23T03:28:30.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.987+0000 7ff11803a5c0 10 datascan.operator(): handling object 1000000020d.0 2023-09-23T03:28:30.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.987+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:487 18.15 18:a8e69ea8:::1000000020d.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff880 con 0x56435febd000 2023-09-23T03:28:30.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.989+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 21 ==== osd_op_reply(487 1000000020d.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv9 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:30.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.989+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000020d 2023-09-23T03:28:30.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.989+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/24 2023-09-23T03:28:30.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.989+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=24 2023-09-23T03:28:30.990 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.989+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:488 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1ca80 con 0x56435fdcbc00 2023-09-23T03:28:30.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.990+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 100 ==== osd_op_reply(488 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:30.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.990+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:30.991 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.990+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:489 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1c000 con 0x56435febc000 2023-09-23T03:28:30.992 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.991+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 207 ==== osd_op_reply(489 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:30.992 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.991+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:30.992 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.991+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:30.992 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.991+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:30.992 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.991+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:490 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1ce00 con 0x56435febcc00 2023-09-23T03:28:30.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.992+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 102 ==== osd_op_reply(490 10000000000.01800000 [omap-get-header out=274b] v0'0 uv34 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.992+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 34 2023-09-23T03:28:30.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.992+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:30.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.992+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:491 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d500 con 0x56435febcc00 2023-09-23T03:28:30.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.993+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 103 ==== osd_op_reply(491 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv34 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.993+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=24 frag=1*, r=0 2023-09-23T03:28:30.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.993+0000 7ff11803a5c0 20 datascan.read_dentry: 24_head not found in result 2023-09-23T03:28:30.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.993+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000020d at 0x0x10000000000/24 with size=2 bytes 2023-09-23T03:28:30.995 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.993+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:492 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1c380 con 0x56435febcc00 2023-09-23T03:28:30.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.997+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 104 ==== osd_op_reply(492 10000000000.01800000 [omap-set-vals] v128'35 uv35 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.997+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/24 pointing to 0x0x1000000020d 2023-09-23T03:28:30.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.997+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:493 18.35 18:ac000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d880 con 0x56435febcc00 2023-09-23T03:28:30.999 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.998+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 105 ==== osd_op_reply(493 [pgnls-filter start_epoch 128 out=49b] v126'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:30.999 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.998+0000 7ff11105e640 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:494 18.d 18:b0000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d180 con 0x56435febcc00 2023-09-23T03:28:31.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.999+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 106 ==== osd_op_reply(494 [pgnls-filter start_epoch 128 out=49b] v126'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:30.999+0000 7ff11185f640 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:495 18.2d 18:b4000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5e700 con 0x56435febc000 2023-09-23T03:28:31.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.000+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 208 ==== osd_op_reply(495 [pgnls-filter start_epoch 128 out=81b] v128'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.000+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000218.00000000 2023-09-23T03:28:31.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.000+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000218.0 2023-09-23T03:28:31.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.001+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:496 18.2d 18:b79a54f6:::10000000218.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecaa80 con 0x56435febc000 2023-09-23T03:28:31.003 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.002+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 209 ==== osd_op_reply(496 10000000218.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv10 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.003 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.002+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000218 2023-09-23T03:28:31.003 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.002+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/35 2023-09-23T03:28:31.003 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.002+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=35 2023-09-23T03:28:31.003 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.002+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:497 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb500 con 0x56435fdcbc00 2023-09-23T03:28:31.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.003+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 101 ==== osd_op_reply(497 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:31.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.003+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:31.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.003+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:498 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb180 con 0x56435febc000 2023-09-23T03:28:31.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.004+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 210 ==== osd_op_reply(498 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.004+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:31.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.004+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:31.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.004+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:31.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.004+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:499 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca700 con 0x56435febcc00 2023-09-23T03:28:31.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.006+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 107 ==== osd_op_reply(499 10000000000.01800000 [omap-get-header out=274b] v0'0 uv35 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.006+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 35 2023-09-23T03:28:31.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.006+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:31.007 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.006+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:500 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca380 con 0x56435febcc00 2023-09-23T03:28:31.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.007+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 108 ==== osd_op_reply(500 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv35 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.007+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=35 frag=1*, r=0 2023-09-23T03:28:31.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.007+0000 7ff11803a5c0 20 datascan.read_dentry: 35_head not found in result 2023-09-23T03:28:31.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.007+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000218 at 0x0x10000000000/35 with size=2 bytes 2023-09-23T03:28:31.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.007+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:501 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb880 con 0x56435febcc00 2023-09-23T03:28:31.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.010+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 109 ==== osd_op_reply(501 10000000000.01800000 [omap-set-vals] v128'36 uv36 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.010+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/35 pointing to 0x0x10000000218 2023-09-23T03:28:31.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.010+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:502 18.1d 18:b8000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecae00 con 0x56435febc400 2023-09-23T03:28:31.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.011+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.0 v2:172.21.15.154:6800/2920938138 25 ==== osd_op_reply(502 [pgnls-filter start_epoch 128 out=81b] v128'18 uv18 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3efc0 con 0x56435febc400 2023-09-23T03:28:31.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.011+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000246.00000000 2023-09-23T03:28:31.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.011+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000246.0 2023-09-23T03:28:31.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.011+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:503 18.1d 18:bab97ecf:::10000000246.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fab3c00 con 0x56435febc400 2023-09-23T03:28:31.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.012+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.0 v2:172.21.15.154:6800/2920938138 26 ==== osd_op_reply(503 10000000246.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv18 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3efc0 con 0x56435febc400 2023-09-23T03:28:31.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.013+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000246 2023-09-23T03:28:31.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.013+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/81 2023-09-23T03:28:31.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.013+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=81 2023-09-23T03:28:31.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.013+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:504 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6e000 con 0x56435fdcbc00 2023-09-23T03:28:31.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.013+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 102 ==== osd_op_reply(504 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:31.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.014+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:31.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.014+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:505 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6ee00 con 0x56435febc000 2023-09-23T03:28:31.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.015+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 211 ==== osd_op_reply(505 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.015+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:31.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.015+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:31.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.015+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:31.016 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.015+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:506 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6f180 con 0x56435febcc00 2023-09-23T03:28:31.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.016+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 110 ==== osd_op_reply(506 10000000000.01800000 [omap-get-header out=274b] v0'0 uv36 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.016+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 36 2023-09-23T03:28:31.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.016+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:31.017 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.016+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:507 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6f500 con 0x56435febcc00 2023-09-23T03:28:31.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.017+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 111 ==== osd_op_reply(507 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv36 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.017+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=81 frag=1*, r=0 2023-09-23T03:28:31.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.017+0000 7ff11803a5c0 20 datascan.read_dentry: 81_head not found in result 2023-09-23T03:28:31.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.017+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000246 at 0x0x10000000000/81 with size=2 bytes 2023-09-23T03:28:31.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.017+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:508 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6fc00 con 0x56435febcc00 2023-09-23T03:28:31.022 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.021+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 112 ==== osd_op_reply(508 10000000000.01800000 [omap-set-vals] v128'37 uv37 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.021+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/81 pointing to 0x0x10000000246 2023-09-23T03:28:31.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.021+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:509 18.3d 18:bc000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6e380 con 0x56435fdcbc00 2023-09-23T03:28:31.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.022+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 103 ==== osd_op_reply(509 [pgnls-filter start_epoch 128 out=101b] v128'13 uv13 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:31.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.022+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000219.00000000 2023-09-23T03:28:31.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.022+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000219.0 2023-09-23T03:28:31.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.022+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:510 18.3d 18:bc0c3701:::10000000219.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fab3880 con 0x56435fdcbc00 2023-09-23T03:28:31.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.023+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 104 ==== osd_op_reply(510 10000000219.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:31.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.024+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000219 2023-09-23T03:28:31.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.024+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/36 2023-09-23T03:28:31.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.024+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=36 2023-09-23T03:28:31.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.024+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:511 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecae00 con 0x56435fdcbc00 2023-09-23T03:28:31.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.024+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 105 ==== osd_op_reply(511 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:31.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.025+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:31.026 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.025+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:512 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6dc00 con 0x56435febc000 2023-09-23T03:28:31.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.026+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 212 ==== osd_op_reply(512 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.026+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:31.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.026+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:31.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.026+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:31.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.026+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:513 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6d880 con 0x56435febc000 2023-09-23T03:28:31.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.027+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 213 ==== osd_op_reply(513 10000000000.01000000 [omap-get-header out=274b] v0'0 uv43 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.029 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.027+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 43 2023-09-23T03:28:31.029 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.027+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:31.029 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.027+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:514 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6c000 con 0x56435febc000 2023-09-23T03:28:31.029 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.028+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 214 ==== osd_op_reply(514 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv43 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.029 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.028+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=36 frag=0*, r=0 2023-09-23T03:28:31.030 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.028+0000 7ff11803a5c0 20 datascan.read_dentry: 36_head not found in result 2023-09-23T03:28:31.030 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.028+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000219 at 0x0x10000000000/36 with size=2 bytes 2023-09-23T03:28:31.030 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.028+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:515 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6c380 con 0x56435febc000 2023-09-23T03:28:31.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.032+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 215 ==== osd_op_reply(515 10000000000.01000000 [omap-set-vals] v128'44 uv44 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.032+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/36 pointing to 0x0x10000000219 2023-09-23T03:28:31.033 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.032+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:516 18.3d 18:bfcd3380::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5fc00 con 0x56435fdcbc00 2023-09-23T03:28:31.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.033+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 106 ==== osd_op_reply(516 [pgnls-filter start_epoch 128 out=81b] v128'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:31.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.033+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000205.00000000 2023-09-23T03:28:31.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.033+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000205.0 2023-09-23T03:28:31.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.033+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:517 18.3d 18:bfcd3380:::10000000205.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5e380 con 0x56435fdcbc00 2023-09-23T03:28:31.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.034+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 107 ==== osd_op_reply(517 10000000205.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:31.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.034+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000205 2023-09-23T03:28:31.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.034+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/16 2023-09-23T03:28:31.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.034+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=16 2023-09-23T03:28:31.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.034+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:518 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe000 con 0x56435fdcbc00 2023-09-23T03:28:31.036 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.035+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 108 ==== osd_op_reply(518 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:31.036 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.035+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:31.036 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.035+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:519 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feffc00 con 0x56435febc000 2023-09-23T03:28:31.037 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.036+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 216 ==== osd_op_reply(519 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.037 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.036+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:31.037 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.036+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:31.037 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.036+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:31.037 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.036+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:520 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefee00 con 0x56435febcc00 2023-09-23T03:28:31.038 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.037+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 113 ==== osd_op_reply(520 10000000000.01800000 [omap-get-header out=274b] v0'0 uv37 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.038 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.037+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 37 2023-09-23T03:28:31.038 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.037+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:31.038 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.037+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:521 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff180 con 0x56435febcc00 2023-09-23T03:28:31.039 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.038+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 114 ==== osd_op_reply(521 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv37 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.039 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.038+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=16 frag=1*, r=0 2023-09-23T03:28:31.039 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.038+0000 7ff11803a5c0 20 datascan.read_dentry: 16_head not found in result 2023-09-23T03:28:31.039 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.038+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000205 at 0x0x10000000000/16 with size=2 bytes 2023-09-23T03:28:31.039 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.038+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:522 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe700 con 0x56435febcc00 2023-09-23T03:28:31.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.042+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 115 ==== osd_op_reply(522 10000000000.01800000 [omap-set-vals] v128'38 uv38 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.042+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/16 pointing to 0x0x10000000205 2023-09-23T03:28:31.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.042+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:523 18.3 18:c0000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5f880 con 0x56435fdcb000 2023-09-23T03:28:31.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.043+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.1 v2:172.21.15.154:6816/1267710005 16 ==== osd_op_reply(523 [pgnls-filter start_epoch 128 out=49b] v126'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56435fd3efc0 con 0x56435fdcb000 2023-09-23T03:28:31.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.043+0000 7ff11105e640 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:524 18.23 18:c4000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1c700 con 0x56435ff15800 2023-09-23T03:28:31.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.044+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.2 v2:172.21.15.154:6822/2884407243 12 ==== osd_op_reply(524 [pgnls-filter start_epoch 128 out=49b] v126'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56435fd3eb40 con 0x56435ff15800 2023-09-23T03:28:31.046 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.044+0000 7ff11185f640 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:525 18.13 18:c8000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff78000 con 0x56435fdca400 2023-09-23T03:28:31.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.045+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.7 v2:172.21.15.190:6824/3460136272 11 ==== osd_op_reply(525 [pgnls-filter start_epoch 128 out=101b] v128'30 uv30 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3ed80 con 0x56435fdca400 2023-09-23T03:28:31.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.046+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 1000000023c.00000000 2023-09-23T03:28:31.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.046+0000 7ff11803a5c0 10 datascan.operator(): handling object 1000000023c.0 2023-09-23T03:28:31.047 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.046+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:526 18.13 18:c8893608:::1000000023c.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5e000 con 0x56435fdca400 2023-09-23T03:28:31.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.047+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.7 v2:172.21.15.190:6824/3460136272 12 ==== osd_op_reply(526 1000000023c.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv26 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435fdca400 2023-09-23T03:28:31.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.047+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000023c 2023-09-23T03:28:31.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.047+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/71 2023-09-23T03:28:31.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.047+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=71 2023-09-23T03:28:31.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.047+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:527 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5ee00 con 0x56435fdcbc00 2023-09-23T03:28:31.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.048+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 109 ==== osd_op_reply(527 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:31.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.048+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:31.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.048+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:528 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5fc00 con 0x56435febc000 2023-09-23T03:28:31.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.049+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 217 ==== osd_op_reply(528 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.049+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:31.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.049+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:31.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.049+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:31.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.049+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:529 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb500 con 0x56435febcc00 2023-09-23T03:28:31.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.050+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 116 ==== osd_op_reply(529 10000000000.01800000 [omap-get-header out=274b] v0'0 uv38 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.050+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 38 2023-09-23T03:28:31.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.050+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:31.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.050+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:530 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca700 con 0x56435febcc00 2023-09-23T03:28:31.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.052+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 117 ==== osd_op_reply(530 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv38 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.052+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=71 frag=1*, r=0 2023-09-23T03:28:31.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.052+0000 7ff11803a5c0 20 datascan.read_dentry: 71_head not found in result 2023-09-23T03:28:31.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.052+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000023c at 0x0x10000000000/71 with size=2 bytes 2023-09-23T03:28:31.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.052+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:531 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca380 con 0x56435febcc00 2023-09-23T03:28:31.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.056+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 118 ==== osd_op_reply(531 10000000000.01800000 [omap-set-vals] v128'39 uv39 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.056+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/71 pointing to 0x0x1000000023c 2023-09-23T03:28:31.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.056+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:532 18.13 18:c933d8fa::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb880 con 0x56435fdca400 2023-09-23T03:28:31.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.057+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.7 v2:172.21.15.190:6824/3460136272 13 ==== osd_op_reply(532 [pgnls-filter start_epoch 128 out=101b] v128'30 uv30 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3ed80 con 0x56435fdca400 2023-09-23T03:28:31.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.057+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000241.00000000 2023-09-23T03:28:31.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.057+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000241.0 2023-09-23T03:28:31.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.057+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:533 18.13 18:c933d8fa:::10000000241.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca000 con 0x56435fdca400 2023-09-23T03:28:31.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.058+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.7 v2:172.21.15.190:6824/3460136272 14 ==== osd_op_reply(533 10000000241.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv27 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435fdca400 2023-09-23T03:28:31.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.058+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000241 2023-09-23T03:28:31.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.058+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/76 2023-09-23T03:28:31.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.058+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=76 2023-09-23T03:28:31.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.058+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:534 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fab3880 con 0x56435fdcbc00 2023-09-23T03:28:31.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.059+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 110 ==== osd_op_reply(534 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:31.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.059+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:31.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.059+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:535 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6e700 con 0x56435febc000 2023-09-23T03:28:31.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.060+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 218 ==== osd_op_reply(535 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.060+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:31.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.060+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:31.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.060+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:31.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.060+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:536 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6e000 con 0x56435febcc00 2023-09-23T03:28:31.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.062+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 119 ==== osd_op_reply(536 10000000000.01800000 [omap-get-header out=274b] v0'0 uv39 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.062+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 39 2023-09-23T03:28:31.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.062+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:31.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.062+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:537 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6f180 con 0x56435febcc00 2023-09-23T03:28:31.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.063+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 120 ==== osd_op_reply(537 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv39 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.063+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=76 frag=1*, r=0 2023-09-23T03:28:31.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.063+0000 7ff11803a5c0 20 datascan.read_dentry: 76_head not found in result 2023-09-23T03:28:31.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.063+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000241 at 0x0x10000000000/76 with size=2 bytes 2023-09-23T03:28:31.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.063+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:538 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6f500 con 0x56435febcc00 2023-09-23T03:28:31.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.067+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 121 ==== osd_op_reply(538 10000000000.01800000 [omap-set-vals] v128'40 uv40 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.068 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.067+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/76 pointing to 0x0x10000000241 2023-09-23T03:28:31.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.067+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:539 18.13 18:c9855970::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6fc00 con 0x56435fdca400 2023-09-23T03:28:31.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.068+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.7 v2:172.21.15.190:6824/3460136272 15 ==== osd_op_reply(539 [pgnls-filter start_epoch 128 out=101b] v128'30 uv30 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3ed80 con 0x56435fdca400 2023-09-23T03:28:31.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.068+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000239.00000000 2023-09-23T03:28:31.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.068+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000239.0 2023-09-23T03:28:31.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.069+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:540 18.13 18:c9855970:::10000000239.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6e380 con 0x56435fdca400 2023-09-23T03:28:31.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.070+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.7 v2:172.21.15.190:6824/3460136272 16 ==== osd_op_reply(540 10000000239.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv28 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435fdca400 2023-09-23T03:28:31.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.070+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000239 2023-09-23T03:28:31.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.070+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/68 2023-09-23T03:28:31.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.070+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=68 2023-09-23T03:28:31.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.070+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:541 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1ce00 con 0x56435fdcbc00 2023-09-23T03:28:31.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.071+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 111 ==== osd_op_reply(541 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:31.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.071+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:31.072 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.071+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:542 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d500 con 0x56435febc000 2023-09-23T03:28:31.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.072+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 219 ==== osd_op_reply(542 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.072+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:31.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.072+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:31.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.072+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:31.073 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.072+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:543 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1c380 con 0x56435febcc00 2023-09-23T03:28:31.075 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.073+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 122 ==== osd_op_reply(543 10000000000.01800000 [omap-get-header out=274b] v0'0 uv40 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.075 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.073+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 40 2023-09-23T03:28:31.075 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.073+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:31.075 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.074+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:544 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d880 con 0x56435febcc00 2023-09-23T03:28:31.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.075+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 123 ==== osd_op_reply(544 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv40 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.075+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=68 frag=1*, r=0 2023-09-23T03:28:31.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.075+0000 7ff11803a5c0 20 datascan.read_dentry: 68_head not found in result 2023-09-23T03:28:31.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.075+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000239 at 0x0x10000000000/68 with size=2 bytes 2023-09-23T03:28:31.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.075+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:545 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d180 con 0x56435febcc00 2023-09-23T03:28:31.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.078+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 124 ==== osd_op_reply(545 10000000000.01800000 [omap-set-vals] v128'41 uv41 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.078+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/68 pointing to 0x0x10000000239 2023-09-23T03:28:31.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.078+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:546 18.13 18:ca8fdbba::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff78000 con 0x56435fdca400 2023-09-23T03:28:31.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.079+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.7 v2:172.21.15.190:6824/3460136272 17 ==== osd_op_reply(546 [pgnls-filter start_epoch 128 out=101b] v128'30 uv30 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3ed80 con 0x56435fdca400 2023-09-23T03:28:31.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.079+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 100000001fb.00000000 2023-09-23T03:28:31.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.079+0000 7ff11803a5c0 10 datascan.operator(): handling object 100000001fb.0 2023-09-23T03:28:31.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.079+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:547 18.13 18:ca8fdbba:::100000001fb.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff78e00 con 0x56435fdca400 2023-09-23T03:28:31.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.080+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.7 v2:172.21.15.190:6824/3460136272 18 ==== osd_op_reply(547 100000001fb.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv29 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x56435fd3ed80 con 0x56435fdca400 2023-09-23T03:28:31.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.080+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001fb 2023-09-23T03:28:31.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.081+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/6 2023-09-23T03:28:31.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.081+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=6 2023-09-23T03:28:31.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.081+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:548 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff79500 con 0x56435fdcbc00 2023-09-23T03:28:31.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.081+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 112 ==== osd_op_reply(548 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:31.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.082+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:31.083 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.082+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:549 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff79880 con 0x56435febc000 2023-09-23T03:28:31.084 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.083+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 220 ==== osd_op_reply(549 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.084 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.083+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:31.084 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.083+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:31.084 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.083+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:31.084 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.083+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:550 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff79c00 con 0x56435febcc00 2023-09-23T03:28:31.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.084+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 125 ==== osd_op_reply(550 10000000000.01800000 [omap-get-header out=274b] v0'0 uv41 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.084+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 41 2023-09-23T03:28:31.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.084+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:31.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.084+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:551 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefea80 con 0x56435febcc00 2023-09-23T03:28:31.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.085+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 126 ==== osd_op_reply(551 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv41 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.085+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=6 frag=1*, r=0 2023-09-23T03:28:31.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.085+0000 7ff11803a5c0 20 datascan.read_dentry: 6_head not found in result 2023-09-23T03:28:31.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.085+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fb at 0x0x10000000000/6 with size=1 bytes 2023-09-23T03:28:31.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.085+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:552 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff500 con 0x56435febcc00 2023-09-23T03:28:31.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.089+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 127 ==== osd_op_reply(552 10000000000.01800000 [omap-set-vals] v128'42 uv42 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.089+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/6 pointing to 0x0x100000001fb 2023-09-23T03:28:31.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.089+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:553 18.13 18:cbcb6224::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff880 con 0x56435fdca400 2023-09-23T03:28:31.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.090+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.7 v2:172.21.15.190:6824/3460136272 19 ==== osd_op_reply(553 [pgnls-filter start_epoch 128 out=81b] v128'30 uv30 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3ed80 con 0x56435fdca400 2023-09-23T03:28:31.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.090+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000212.00000000 2023-09-23T03:28:31.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.090+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000212.0 2023-09-23T03:28:31.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.090+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:554 18.13 18:cbcb6224:::10000000212.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feffc00 con 0x56435fdca400 2023-09-23T03:28:31.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.092+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.7 v2:172.21.15.190:6824/3460136272 20 ==== osd_op_reply(554 10000000212.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv30 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435fdca400 2023-09-23T03:28:31.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.092+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000212 2023-09-23T03:28:31.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.092+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/29 2023-09-23T03:28:31.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.092+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=29 2023-09-23T03:28:31.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.092+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:555 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6ea80 con 0x56435fdcbc00 2023-09-23T03:28:31.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.092+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 113 ==== osd_op_reply(555 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:31.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.093+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:31.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.093+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:556 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6ee00 con 0x56435febc000 2023-09-23T03:28:31.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.094+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 221 ==== osd_op_reply(556 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.094+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:31.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.094+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:31.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.094+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:31.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.094+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:557 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecbc00 con 0x56435febcc00 2023-09-23T03:28:31.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.095+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 128 ==== osd_op_reply(557 10000000000.01800000 [omap-get-header out=274b] v0'0 uv42 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.095+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 42 2023-09-23T03:28:31.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.095+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:31.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.095+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:558 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecaa80 con 0x56435febcc00 2023-09-23T03:28:31.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.096+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 129 ==== osd_op_reply(558 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv42 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.096+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=29 frag=1*, r=0 2023-09-23T03:28:31.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.096+0000 7ff11803a5c0 20 datascan.read_dentry: 29_head not found in result 2023-09-23T03:28:31.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.096+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000212 at 0x0x10000000000/29 with size=2 bytes 2023-09-23T03:28:31.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.096+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:559 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb180 con 0x56435febcc00 2023-09-23T03:28:31.101 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.099+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 130 ==== osd_op_reply(559 10000000000.01800000 [omap-set-vals] v128'43 uv43 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.101 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.099+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/29 pointing to 0x0x10000000212 2023-09-23T03:28:31.101 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.100+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:560 18.33 18:cc000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5f180 con 0x56435fdcb000 2023-09-23T03:28:31.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.101+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.1 v2:172.21.15.154:6816/1267710005 17 ==== osd_op_reply(560 [pgnls-filter start_epoch 128 out=81b] v128'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3efc0 con 0x56435fdcb000 2023-09-23T03:28:31.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.101+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000248.00000000 2023-09-23T03:28:31.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.101+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000248.0 2023-09-23T03:28:31.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.101+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:561 18.33 18:cf9e7a4f:::10000000248.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5ea80 con 0x56435fdcb000 2023-09-23T03:28:31.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.102+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.1 v2:172.21.15.154:6816/1267710005 18 ==== osd_op_reply(561 10000000248.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3efc0 con 0x56435fdcb000 2023-09-23T03:28:31.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.102+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000248 2023-09-23T03:28:31.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.102+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/83 2023-09-23T03:28:31.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.102+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=83 2023-09-23T03:28:31.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.102+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:562 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1c000 con 0x56435fdcbc00 2023-09-23T03:28:31.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.103+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 114 ==== osd_op_reply(562 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:31.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.103+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:31.104 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.103+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:563 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb500 con 0x56435febc000 2023-09-23T03:28:31.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.104+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 222 ==== osd_op_reply(563 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.104+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:31.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.104+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:31.106 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.104+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:31.106 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.104+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:564 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca700 con 0x56435febc000 2023-09-23T03:28:31.106 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.105+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 223 ==== osd_op_reply(564 10000000000.01000000 [omap-get-header out=274b] v0'0 uv44 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.105+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 44 2023-09-23T03:28:31.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.105+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:31.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.105+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:565 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca380 con 0x56435febc000 2023-09-23T03:28:31.108 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.106+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 224 ==== osd_op_reply(565 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv44 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.108 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.106+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=83 frag=0*, r=0 2023-09-23T03:28:31.108 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.106+0000 7ff11803a5c0 20 datascan.read_dentry: 83_head not found in result 2023-09-23T03:28:31.108 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.107+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000248 at 0x0x10000000000/83 with size=2 bytes 2023-09-23T03:28:31.108 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.107+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:566 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecbc00 con 0x56435febc000 2023-09-23T03:28:31.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.110+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 225 ==== osd_op_reply(566 10000000000.01000000 [omap-set-vals] v128'45 uv45 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.110+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/83 pointing to 0x0x10000000248 2023-09-23T03:28:31.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.111+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:567 18.b 18:d0000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6e000 con 0x56435febd000 2023-09-23T03:28:31.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.112+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 22 ==== osd_op_reply(567 [pgnls-filter start_epoch 128 out=101b] v128'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:31.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.112+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000225.00000000 2023-09-23T03:28:31.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.112+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000225.0 2023-09-23T03:28:31.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.112+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:568 18.b 18:d0d26e50:::10000000225.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6f180 con 0x56435febd000 2023-09-23T03:28:31.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.113+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 23 ==== osd_op_reply(568 10000000225.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:31.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.113+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000225 2023-09-23T03:28:31.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.113+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/48 2023-09-23T03:28:31.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.113+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=48 2023-09-23T03:28:31.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.113+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:569 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6ea80 con 0x56435fdcbc00 2023-09-23T03:28:31.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.114+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 115 ==== osd_op_reply(569 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:31.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.114+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:31.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.114+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:570 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe380 con 0x56435febc000 2023-09-23T03:28:31.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.115+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 226 ==== osd_op_reply(570 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.115+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:31.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.115+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:31.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.115+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:31.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.116+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:571 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe000 con 0x56435febcc00 2023-09-23T03:28:31.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.117+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 131 ==== osd_op_reply(571 10000000000.01800000 [omap-get-header out=274b] v0'0 uv43 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.117+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 43 2023-09-23T03:28:31.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.117+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:31.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.117+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:572 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefee00 con 0x56435febcc00 2023-09-23T03:28:31.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.118+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 132 ==== osd_op_reply(572 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv43 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.118+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=48 frag=1*, r=0 2023-09-23T03:28:31.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.118+0000 7ff11803a5c0 20 datascan.read_dentry: 48_head not found in result 2023-09-23T03:28:31.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.118+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000225 at 0x0x10000000000/48 with size=2 bytes 2023-09-23T03:28:31.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.118+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:573 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff180 con 0x56435febcc00 2023-09-23T03:28:31.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.122+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 133 ==== osd_op_reply(573 10000000000.01800000 [omap-set-vals] v128'44 uv44 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.122+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/48 pointing to 0x0x10000000225 2023-09-23T03:28:31.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.122+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:574 18.b 18:d2dbd4ac::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe700 con 0x56435febd000 2023-09-23T03:28:31.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.123+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 24 ==== osd_op_reply(574 [pgnls-filter start_epoch 128 out=101b] v128'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:31.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.123+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000217.00000000 2023-09-23T03:28:31.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.123+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000217.0 2023-09-23T03:28:31.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.123+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:575 18.b 18:d2dbd4ac:::10000000217.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefea80 con 0x56435febd000 2023-09-23T03:28:31.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.124+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 25 ==== osd_op_reply(575 10000000217.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:31.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.125+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000217 2023-09-23T03:28:31.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.125+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/34 2023-09-23T03:28:31.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.125+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=34 2023-09-23T03:28:31.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.125+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:576 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff79500 con 0x56435fdcbc00 2023-09-23T03:28:31.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.126+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 116 ==== osd_op_reply(576 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:31.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.126+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:31.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.126+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:577 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff79c00 con 0x56435febc000 2023-09-23T03:28:31.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.127+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 227 ==== osd_op_reply(577 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.127+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:31.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.127+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:31.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.127+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:31.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.127+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:578 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1c700 con 0x56435febc000 2023-09-23T03:28:31.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.128+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 228 ==== osd_op_reply(578 10000000000.01000000 [omap-get-header out=274b] v0'0 uv45 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.128+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 45 2023-09-23T03:28:31.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.128+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:31.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.128+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:579 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1ce00 con 0x56435febc000 2023-09-23T03:28:31.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.129+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 229 ==== osd_op_reply(579 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv45 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.130+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=34 frag=0*, r=0 2023-09-23T03:28:31.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.130+0000 7ff11803a5c0 20 datascan.read_dentry: 34_head not found in result 2023-09-23T03:28:31.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.130+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000217 at 0x0x10000000000/34 with size=2 bytes 2023-09-23T03:28:31.131 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.130+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:580 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1c380 con 0x56435febc000 2023-09-23T03:28:31.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.133+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 230 ==== osd_op_reply(580 10000000000.01000000 [omap-set-vals] v128'46 uv46 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.133+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/34 pointing to 0x0x10000000217 2023-09-23T03:28:31.135 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.133+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:581 18.b 18:d3eda40d::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d880 con 0x56435febd000 2023-09-23T03:28:31.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.134+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 26 ==== osd_op_reply(581 [pgnls-filter start_epoch 128 out=81b] v128'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:31.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.135+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 1000000020f.00000000 2023-09-23T03:28:31.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.135+0000 7ff11803a5c0 10 datascan.operator(): handling object 1000000020f.0 2023-09-23T03:28:31.136 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.135+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:582 18.b 18:d3eda40d:::1000000020f.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d180 con 0x56435febd000 2023-09-23T03:28:31.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.136+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 27 ==== osd_op_reply(582 1000000020f.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:31.137 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.136+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000020f 2023-09-23T03:28:31.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.136+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/26 2023-09-23T03:28:31.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.136+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=26 2023-09-23T03:28:31.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.136+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:583 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5e380 con 0x56435fdcbc00 2023-09-23T03:28:31.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.137+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 117 ==== osd_op_reply(583 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:31.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.137+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:31.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.137+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:584 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5ee00 con 0x56435febc000 2023-09-23T03:28:31.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.138+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 231 ==== osd_op_reply(584 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.138+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:31.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.138+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:31.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.138+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:31.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.139+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:585 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6e700 con 0x56435febcc00 2023-09-23T03:28:31.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.140+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 134 ==== osd_op_reply(585 10000000000.01800000 [omap-get-header out=274b] v0'0 uv44 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.140+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 44 2023-09-23T03:28:31.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.140+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:31.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.140+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:586 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6fc00 con 0x56435febcc00 2023-09-23T03:28:31.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.141+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 135 ==== osd_op_reply(586 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv44 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.141+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=26 frag=1*, r=0 2023-09-23T03:28:31.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.141+0000 7ff11803a5c0 20 datascan.read_dentry: 26_head not found in result 2023-09-23T03:28:31.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.141+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000020f at 0x0x10000000000/26 with size=2 bytes 2023-09-23T03:28:31.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.141+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:587 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6e380 con 0x56435febcc00 2023-09-23T03:28:31.147 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.145+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 136 ==== osd_op_reply(587 10000000000.01800000 [omap-set-vals] v128'45 uv45 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.147 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.146+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/26 pointing to 0x0x1000000020f 2023-09-23T03:28:31.147 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.146+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:588 18.2b 18:d4000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6f880 con 0x56435fdca400 2023-09-23T03:28:31.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.147+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.7 v2:172.21.15.190:6824/3460136272 21 ==== osd_op_reply(588 [pgnls-filter start_epoch 128 out=101b] v128'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3ed80 con 0x56435fdca400 2023-09-23T03:28:31.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.147+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000203.00000000 2023-09-23T03:28:31.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.147+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000203.0 2023-09-23T03:28:31.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.147+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:589 18.2b 18:d4a58307:::10000000203.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6ee00 con 0x56435fdca400 2023-09-23T03:28:31.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.148+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.7 v2:172.21.15.190:6824/3460136272 22 ==== osd_op_reply(589 10000000203.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435fdca400 2023-09-23T03:28:31.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.148+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000203 2023-09-23T03:28:31.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.148+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/14 2023-09-23T03:28:31.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.148+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=14 2023-09-23T03:28:31.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.148+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:590 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff6f500 con 0x56435fdcbc00 2023-09-23T03:28:31.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.149+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 118 ==== osd_op_reply(590 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:31.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.149+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:31.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.149+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:591 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb880 con 0x56435febc000 2023-09-23T03:28:31.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.150+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 232 ==== osd_op_reply(591 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.151+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:31.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.151+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:31.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.151+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:31.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.151+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:592 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecae00 con 0x56435febcc00 2023-09-23T03:28:31.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.152+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 137 ==== osd_op_reply(592 10000000000.01800000 [omap-get-header out=274b] v0'0 uv45 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.152+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 45 2023-09-23T03:28:31.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.152+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:31.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.152+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:593 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca000 con 0x56435febcc00 2023-09-23T03:28:31.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.153+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 138 ==== osd_op_reply(593 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv45 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.153+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=14 frag=1*, r=0 2023-09-23T03:28:31.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.153+0000 7ff11803a5c0 20 datascan.read_dentry: 14_head not found in result 2023-09-23T03:28:31.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.153+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000203 at 0x0x10000000000/14 with size=2 bytes 2023-09-23T03:28:31.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.153+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:594 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb500 con 0x56435febcc00 2023-09-23T03:28:31.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.157+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 139 ==== osd_op_reply(594 10000000000.01800000 [omap-set-vals] v128'46 uv46 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.157+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/14 pointing to 0x0x10000000203 2023-09-23T03:28:31.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.157+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:595 18.2b 18:d5ee56ee::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca700 con 0x56435fdca400 2023-09-23T03:28:31.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.158+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.7 v2:172.21.15.190:6824/3460136272 23 ==== osd_op_reply(595 [pgnls-filter start_epoch 128 out=81b] v128'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3ed80 con 0x56435fdca400 2023-09-23T03:28:31.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.158+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 1000000022d.00000000 2023-09-23T03:28:31.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.158+0000 7ff11803a5c0 10 datascan.operator(): handling object 1000000022d.0 2023-09-23T03:28:31.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.158+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:596 18.2b 18:d5ee56ee:::1000000022d.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca380 con 0x56435fdca400 2023-09-23T03:28:31.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.160+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.7 v2:172.21.15.190:6824/3460136272 24 ==== osd_op_reply(596 1000000022d.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435fdca400 2023-09-23T03:28:31.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.160+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000022d 2023-09-23T03:28:31.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.160+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/56 2023-09-23T03:28:31.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.160+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=56 2023-09-23T03:28:31.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.160+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:597 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff880 con 0x56435fdcbc00 2023-09-23T03:28:31.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.161+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 119 ==== osd_op_reply(597 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:31.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.161+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:31.162 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.161+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:598 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feffc00 con 0x56435febc000 2023-09-23T03:28:31.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.162+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 233 ==== osd_op_reply(598 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.162+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:31.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.162+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:31.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.162+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:31.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.162+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:599 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff78000 con 0x56435febcc00 2023-09-23T03:28:31.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.163+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 140 ==== osd_op_reply(599 10000000000.01800000 [omap-get-header out=274b] v0'0 uv46 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.164+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 46 2023-09-23T03:28:31.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.164+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:31.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.164+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:600 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff79180 con 0x56435febcc00 2023-09-23T03:28:31.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.165+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 141 ==== osd_op_reply(600 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv46 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.165+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=56 frag=1*, r=0 2023-09-23T03:28:31.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.165+0000 7ff11803a5c0 20 datascan.read_dentry: 56_head not found in result 2023-09-23T03:28:31.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.165+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000022d at 0x0x10000000000/56 with size=2 bytes 2023-09-23T03:28:31.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.165+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:601 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff78e00 con 0x56435febcc00 2023-09-23T03:28:31.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.169+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 142 ==== osd_op_reply(601 10000000000.01800000 [omap-set-vals] v128'47 uv47 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.169+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/56 pointing to 0x0x1000000022d 2023-09-23T03:28:31.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.169+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:602 18.1b 18:d8000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff79880 con 0x56435fdcb000 2023-09-23T03:28:31.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.170+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.1 v2:172.21.15.154:6816/1267710005 19 ==== osd_op_reply(602 [pgnls-filter start_epoch 128 out=49b] v126'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56435fd3efc0 con 0x56435fdcb000 2023-09-23T03:28:31.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.170+0000 7ff11185f640 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:603 18.3b 18:dc000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff78380 con 0x56435febcc00 2023-09-23T03:28:31.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.171+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 143 ==== osd_op_reply(603 [pgnls-filter start_epoch 128 out=101b] v128'13 uv13 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.171+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000249.00000000 2023-09-23T03:28:31.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.171+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000249.0 2023-09-23T03:28:31.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.171+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:604 18.3b 18:dd50df43:::10000000249.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1ca80 con 0x56435febcc00 2023-09-23T03:28:31.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.173+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 144 ==== osd_op_reply(604 10000000249.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.173+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000249 2023-09-23T03:28:31.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.173+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/84 2023-09-23T03:28:31.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.173+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=84 2023-09-23T03:28:31.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.173+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:605 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5f500 con 0x56435fdcbc00 2023-09-23T03:28:31.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.174+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 120 ==== osd_op_reply(605 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:31.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.174+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:31.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.174+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:606 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5e000 con 0x56435febc000 2023-09-23T03:28:31.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.175+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 234 ==== osd_op_reply(606 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.175+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:31.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.175+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:31.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.175+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:31.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.175+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:607 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5fc00 con 0x56435febc000 2023-09-23T03:28:31.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.176+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 235 ==== osd_op_reply(607 10000000000.01000000 [omap-get-header out=274b] v0'0 uv46 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.176+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 46 2023-09-23T03:28:31.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.176+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:31.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.176+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:608 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff54000 con 0x56435febc000 2023-09-23T03:28:31.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.177+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 236 ==== osd_op_reply(608 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv46 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.177+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=84 frag=0*, r=0 2023-09-23T03:28:31.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.177+0000 7ff11803a5c0 20 datascan.read_dentry: 84_head not found in result 2023-09-23T03:28:31.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.177+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000249 at 0x0x10000000000/84 with size=2 bytes 2023-09-23T03:28:31.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.178+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:609 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff54380 con 0x56435febc000 2023-09-23T03:28:31.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.181+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 237 ==== osd_op_reply(609 10000000000.01000000 [omap-set-vals] v128'47 uv47 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.181+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/84 pointing to 0x0x10000000249 2023-09-23T03:28:31.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.181+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:610 18.3b 18:deccbd37::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff54700 con 0x56435febcc00 2023-09-23T03:28:31.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.182+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 145 ==== osd_op_reply(610 [pgnls-filter start_epoch 128 out=101b] v128'13 uv13 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.182+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 100000001fe.00000000 2023-09-23T03:28:31.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.182+0000 7ff11803a5c0 10 datascan.operator(): handling object 100000001fe.0 2023-09-23T03:28:31.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.183+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:611 18.3b 18:deccbd37:::100000001fe.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff54a80 con 0x56435febcc00 2023-09-23T03:28:31.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.184+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 146 ==== osd_op_reply(611 100000001fe.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.184+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001fe 2023-09-23T03:28:31.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.184+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/9 2023-09-23T03:28:31.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.184+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=9 2023-09-23T03:28:31.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.184+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:612 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff55180 con 0x56435fdcbc00 2023-09-23T03:28:31.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.185+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 121 ==== osd_op_reply(612 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:31.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.185+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:31.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.185+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:613 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff55500 con 0x56435febc000 2023-09-23T03:28:31.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.186+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 238 ==== osd_op_reply(613 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.186+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:31.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.186+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:31.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.186+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:31.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.186+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:614 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff55880 con 0x56435febc000 2023-09-23T03:28:31.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.187+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 239 ==== osd_op_reply(614 10000000000.01000000 [omap-get-header out=274b] v0'0 uv47 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.188+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 47 2023-09-23T03:28:31.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.188+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:31.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.188+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:615 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff55c00 con 0x56435febc000 2023-09-23T03:28:31.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.189+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 240 ==== osd_op_reply(615 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv47 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.189+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=9 frag=0*, r=0 2023-09-23T03:28:31.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.189+0000 7ff11803a5c0 20 datascan.read_dentry: 9_head not found in result 2023-09-23T03:28:31.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.189+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fe at 0x0x10000000000/9 with size=1 bytes 2023-09-23T03:28:31.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.189+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:616 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff54000 con 0x56435febc000 2023-09-23T03:28:31.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.192+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 241 ==== osd_op_reply(616 10000000000.01000000 [omap-set-vals] v128'48 uv48 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.193+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/9 pointing to 0x0x100000001fe 2023-09-23T03:28:31.194 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.193+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:617 18.3b 18:ded14b84::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff54380 con 0x56435febcc00 2023-09-23T03:28:31.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.194+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 147 ==== osd_op_reply(617 [pgnls-filter start_epoch 128 out=81b] v128'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.194+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 1000000024b.00000000 2023-09-23T03:28:31.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.194+0000 7ff11803a5c0 10 datascan.operator(): handling object 1000000024b.0 2023-09-23T03:28:31.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.194+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:618 18.3b 18:ded14b84:::1000000024b.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff55500 con 0x56435febcc00 2023-09-23T03:28:31.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.196+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 148 ==== osd_op_reply(618 1000000024b.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.196+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000024b 2023-09-23T03:28:31.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.196+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/86 2023-09-23T03:28:31.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.196+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=86 2023-09-23T03:28:31.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.196+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:619 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff54e00 con 0x56435fdcbc00 2023-09-23T03:28:31.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.197+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 122 ==== osd_op_reply(619 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:31.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.197+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:31.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.197+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:620 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5ee00 con 0x56435febc000 2023-09-23T03:28:31.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.198+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 242 ==== osd_op_reply(620 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.198+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:31.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.198+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:31.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.198+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:31.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.198+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:621 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5e000 con 0x56435febc000 2023-09-23T03:28:31.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.199+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 243 ==== osd_op_reply(621 10000000000.01000000 [omap-get-header out=274b] v0'0 uv48 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.199+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 48 2023-09-23T03:28:31.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.199+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:31.201 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.199+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:622 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5fc00 con 0x56435febc000 2023-09-23T03:28:31.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.200+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 244 ==== osd_op_reply(622 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv48 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.200+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=86 frag=0*, r=0 2023-09-23T03:28:31.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.200+0000 7ff11803a5c0 20 datascan.read_dentry: 86_head not found in result 2023-09-23T03:28:31.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.200+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000024b at 0x0x10000000000/86 with size=2 bytes 2023-09-23T03:28:31.202 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.200+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:623 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5e380 con 0x56435febc000 2023-09-23T03:28:31.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.204+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 245 ==== osd_op_reply(623 10000000000.01000000 [omap-set-vals] v128'49 uv49 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.204+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/86 pointing to 0x0x1000000024b 2023-09-23T03:28:31.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.204+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:624 18.7 18:e0000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5f500 con 0x56435ff15800 2023-09-23T03:28:31.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.205+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.2 v2:172.21.15.154:6822/2884407243 13 ==== osd_op_reply(624 [pgnls-filter start_epoch 128 out=101b] v128'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3eb40 con 0x56435ff15800 2023-09-23T03:28:31.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.205+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000207.00000000 2023-09-23T03:28:31.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.205+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000207.0 2023-09-23T03:28:31.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.205+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:625 18.7 18:e15a914f:::10000000207.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1c700 con 0x56435ff15800 2023-09-23T03:28:31.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.206+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.2 v2:172.21.15.154:6822/2884407243 14 ==== osd_op_reply(625 10000000207.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3eb40 con 0x56435ff15800 2023-09-23T03:28:31.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.206+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000207 2023-09-23T03:28:31.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.207+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/18 2023-09-23T03:28:31.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.207+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=18 2023-09-23T03:28:31.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.207+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:626 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1c380 con 0x56435fdcbc00 2023-09-23T03:28:31.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.207+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 123 ==== osd_op_reply(626 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:31.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.207+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:31.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.207+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:627 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d880 con 0x56435febc000 2023-09-23T03:28:31.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.209+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 246 ==== osd_op_reply(627 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.209+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:31.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.209+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:31.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.209+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:31.210 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.209+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:628 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d180 con 0x56435febcc00 2023-09-23T03:28:31.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.210+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 149 ==== osd_op_reply(628 10000000000.01800000 [omap-get-header out=274b] v0'0 uv47 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.210+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 47 2023-09-23T03:28:31.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.210+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:31.211 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.210+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:629 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1c000 con 0x56435febcc00 2023-09-23T03:28:31.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.211+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 150 ==== osd_op_reply(629 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv47 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.211+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=18 frag=1*, r=0 2023-09-23T03:28:31.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.211+0000 7ff11803a5c0 20 datascan.read_dentry: 18_head not found in result 2023-09-23T03:28:31.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.211+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000207 at 0x0x10000000000/18 with size=2 bytes 2023-09-23T03:28:31.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.211+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:630 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d500 con 0x56435febcc00 2023-09-23T03:28:31.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.216+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 151 ==== osd_op_reply(630 10000000000.01800000 [omap-set-vals] v128'48 uv48 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.216+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/18 pointing to 0x0x10000000207 2023-09-23T03:28:31.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.216+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:631 18.7 18:e2e07930::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1ca80 con 0x56435ff15800 2023-09-23T03:28:31.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.217+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.2 v2:172.21.15.154:6822/2884407243 15 ==== osd_op_reply(631 [pgnls-filter start_epoch 128 out=101b] v128'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3eb40 con 0x56435ff15800 2023-09-23T03:28:31.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.217+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2023-09-23T03:28:31.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.217+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000001.0 2023-09-23T03:28:31.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.217+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:632 18.7 18:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff79c00 con 0x56435ff15800 2023-09-23T03:28:31.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.218+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.2 v2:172.21.15.154:6822/2884407243 16 ==== osd_op_reply(632 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv14 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x56435fd3eb40 con 0x56435ff15800 2023-09-23T03:28:31.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.218+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2023-09-23T03:28:31.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.218+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/0 2023-09-23T03:28:31.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.218+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=0 2023-09-23T03:28:31.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.218+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:633 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff78000 con 0x56435fdcbc00 2023-09-23T03:28:31.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.219+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 124 ==== osd_op_reply(633 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:31.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.219+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:31.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.219+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:634 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff79180 con 0x56435febc000 2023-09-23T03:28:31.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.220+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 247 ==== osd_op_reply(634 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.220+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:31.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.220+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:31.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.220+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:31.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.220+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:635 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff78e00 con 0x56435febcc00 2023-09-23T03:28:31.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.221+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 152 ==== osd_op_reply(635 10000000000.01800000 [omap-get-header out=274b] v0'0 uv48 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.222+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 48 2023-09-23T03:28:31.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.222+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:31.223 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.222+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:636 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff78380 con 0x56435febcc00 2023-09-23T03:28:31.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.223+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 153 ==== osd_op_reply(636 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv48 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.223+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=0 frag=1*, r=0 2023-09-23T03:28:31.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.223+0000 7ff11803a5c0 20 datascan.read_dentry: 0_head not found in result 2023-09-23T03:28:31.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.223+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/0 with size=1 bytes 2023-09-23T03:28:31.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.223+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:637 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fab3880 con 0x56435febcc00 2023-09-23T03:28:31.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.227+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 154 ==== osd_op_reply(637 10000000000.01800000 [omap-set-vals] v128'49 uv49 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.227+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/0 pointing to 0x0x10000000001 2023-09-23T03:28:31.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.227+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:638 18.7 18:e37752f2::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe000 con 0x56435ff15800 2023-09-23T03:28:31.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.228+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.2 v2:172.21.15.154:6822/2884407243 17 ==== osd_op_reply(638 [pgnls-filter start_epoch 128 out=81b] v128'15 uv15 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3eb40 con 0x56435ff15800 2023-09-23T03:28:31.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.228+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 1000000021f.00000000 2023-09-23T03:28:31.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.228+0000 7ff11803a5c0 10 datascan.operator(): handling object 1000000021f.0 2023-09-23T03:28:31.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.228+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:639 18.7 18:e37752f2:::1000000021f.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe380 con 0x56435ff15800 2023-09-23T03:28:31.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.229+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.2 v2:172.21.15.154:6822/2884407243 18 ==== osd_op_reply(639 1000000021f.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3eb40 con 0x56435ff15800 2023-09-23T03:28:31.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.229+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000021f 2023-09-23T03:28:31.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.229+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/42 2023-09-23T03:28:31.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.229+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=42 2023-09-23T03:28:31.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.229+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:640 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefea80 con 0x56435fdcbc00 2023-09-23T03:28:31.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.230+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 125 ==== osd_op_reply(640 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:31.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.230+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:31.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.230+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:641 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefee00 con 0x56435febc000 2023-09-23T03:28:31.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.231+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 248 ==== osd_op_reply(641 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.231+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:31.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.231+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:31.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.231+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:31.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.231+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:642 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff180 con 0x56435febc000 2023-09-23T03:28:31.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.232+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 249 ==== osd_op_reply(642 10000000000.01000000 [omap-get-header out=274b] v0'0 uv49 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.232+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 49 2023-09-23T03:28:31.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.232+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:31.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.232+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:643 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff500 con 0x56435febc000 2023-09-23T03:28:31.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.233+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 250 ==== osd_op_reply(643 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv49 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.233+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=42 frag=0*, r=0 2023-09-23T03:28:31.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.233+0000 7ff11803a5c0 20 datascan.read_dentry: 42_head not found in result 2023-09-23T03:28:31.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.233+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000021f at 0x0x10000000000/42 with size=2 bytes 2023-09-23T03:28:31.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.233+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:644 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff880 con 0x56435febc000 2023-09-23T03:28:31.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.237+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 251 ==== osd_op_reply(644 10000000000.01000000 [omap-set-vals] v128'50 uv50 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.237+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/42 pointing to 0x0x1000000021f 2023-09-23T03:28:31.239 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.237+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:645 18.27 18:e4000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feffc00 con 0x56435febd000 2023-09-23T03:28:31.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.238+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 28 ==== osd_op_reply(645 [pgnls-filter start_epoch 128 out=101b] v128'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:31.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.239+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000210.00000000 2023-09-23T03:28:31.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.239+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000210.0 2023-09-23T03:28:31.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.239+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:646 18.27 18:e40c9c52:::10000000210.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca000 con 0x56435febd000 2023-09-23T03:28:31.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.240+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 29 ==== osd_op_reply(646 10000000210.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:31.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.240+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000210 2023-09-23T03:28:31.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.240+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/27 2023-09-23T03:28:31.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.240+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=27 2023-09-23T03:28:31.241 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.240+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:647 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca700 con 0x56435fdcbc00 2023-09-23T03:28:31.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.241+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 126 ==== osd_op_reply(647 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:31.242 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.241+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:31.243 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.241+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:648 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecaa80 con 0x56435febc000 2023-09-23T03:28:31.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.242+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 252 ==== osd_op_reply(648 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.243+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:31.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.243+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:31.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.243+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:31.244 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.243+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:649 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecae00 con 0x56435febc000 2023-09-23T03:28:31.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.244+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 253 ==== osd_op_reply(649 10000000000.01000000 [omap-get-header out=274b] v0'0 uv50 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.244+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 50 2023-09-23T03:28:31.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.244+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:31.245 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.244+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:650 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb180 con 0x56435febc000 2023-09-23T03:28:31.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.245+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 254 ==== osd_op_reply(650 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv50 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.245+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=27 frag=0*, r=0 2023-09-23T03:28:31.246 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.245+0000 7ff11803a5c0 20 datascan.read_dentry: 27_head not found in result 2023-09-23T03:28:31.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.245+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000210 at 0x0x10000000000/27 with size=2 bytes 2023-09-23T03:28:31.247 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.245+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:651 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb500 con 0x56435febc000 2023-09-23T03:28:31.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.249+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 255 ==== osd_op_reply(651 10000000000.01000000 [omap-set-vals] v128'51 uv51 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.249+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/27 pointing to 0x0x10000000210 2023-09-23T03:28:31.250 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.249+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:652 18.27 18:e5671758::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb880 con 0x56435febd000 2023-09-23T03:28:31.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.250+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 30 ==== osd_op_reply(652 [pgnls-filter start_epoch 128 out=101b] v128'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:31.251 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.250+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 1000000024f.00000000 2023-09-23T03:28:31.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.250+0000 7ff11803a5c0 10 datascan.operator(): handling object 1000000024f.0 2023-09-23T03:28:31.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.250+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:653 18.27 18:e5671758:::1000000024f.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecbc00 con 0x56435febd000 2023-09-23T03:28:31.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.251+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 31 ==== osd_op_reply(653 1000000024f.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv14 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:31.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.252+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000024f 2023-09-23T03:28:31.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.252+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/90 2023-09-23T03:28:31.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.252+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=90 2023-09-23T03:28:31.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.252+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:654 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff7c380 con 0x56435fdcbc00 2023-09-23T03:28:31.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.252+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 127 ==== osd_op_reply(654 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:31.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.253+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:31.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.253+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:655 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff7c700 con 0x56435febc000 2023-09-23T03:28:31.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.254+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 256 ==== osd_op_reply(655 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.254+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:31.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.254+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:31.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.254+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:31.255 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.254+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:656 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff7ca80 con 0x56435febc000 2023-09-23T03:28:31.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.255+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 257 ==== osd_op_reply(656 10000000000.01000000 [omap-get-header out=274b] v0'0 uv51 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.255+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 51 2023-09-23T03:28:31.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.255+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:31.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.255+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:657 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff7ce00 con 0x56435febc000 2023-09-23T03:28:31.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.256+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 258 ==== osd_op_reply(657 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv51 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.256+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=90 frag=0*, r=0 2023-09-23T03:28:31.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.256+0000 7ff11803a5c0 20 datascan.read_dentry: 90_head not found in result 2023-09-23T03:28:31.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.256+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000024f at 0x0x10000000000/90 with size=2 bytes 2023-09-23T03:28:31.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.256+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:658 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff7d180 con 0x56435febc000 2023-09-23T03:28:31.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.260+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 259 ==== osd_op_reply(658 10000000000.01000000 [omap-set-vals] v128'52 uv52 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.260+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/90 pointing to 0x0x1000000024f 2023-09-23T03:28:31.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.260+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:659 18.27 18:e5993619::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff7d500 con 0x56435febd000 2023-09-23T03:28:31.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.261+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 32 ==== osd_op_reply(659 [pgnls-filter start_epoch 128 out=101b] v128'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:31.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.261+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000208.00000000 2023-09-23T03:28:31.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.261+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000208.0 2023-09-23T03:28:31.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.261+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:660 18.27 18:e5993619:::10000000208.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff7d880 con 0x56435febd000 2023-09-23T03:28:31.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.263+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 33 ==== osd_op_reply(660 10000000208.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:31.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.263+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000208 2023-09-23T03:28:31.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.263+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/19 2023-09-23T03:28:31.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.263+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=19 2023-09-23T03:28:31.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.263+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:661 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff7c000 con 0x56435fdcbc00 2023-09-23T03:28:31.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.263+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 128 ==== osd_op_reply(661 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:31.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.264+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:31.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.264+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:662 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff7c700 con 0x56435febc000 2023-09-23T03:28:31.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.265+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 260 ==== osd_op_reply(662 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.265+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:31.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.265+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:31.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.265+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:31.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.265+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:663 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff7ca80 con 0x56435febc000 2023-09-23T03:28:31.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.266+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 261 ==== osd_op_reply(663 10000000000.01000000 [omap-get-header out=274b] v0'0 uv52 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.266+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 52 2023-09-23T03:28:31.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.266+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:31.267 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.266+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:664 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff7ce00 con 0x56435febc000 2023-09-23T03:28:31.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.267+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 262 ==== osd_op_reply(664 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv52 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.267+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=19 frag=0*, r=0 2023-09-23T03:28:31.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.267+0000 7ff11803a5c0 20 datascan.read_dentry: 19_head not found in result 2023-09-23T03:28:31.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.267+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000208 at 0x0x10000000000/19 with size=2 bytes 2023-09-23T03:28:31.268 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.267+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:665 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff7d180 con 0x56435febc000 2023-09-23T03:28:31.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.271+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 263 ==== osd_op_reply(665 10000000000.01000000 [omap-set-vals] v128'53 uv53 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.271+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/19 pointing to 0x0x10000000208 2023-09-23T03:28:31.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.271+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:666 18.27 18:e5a5749e::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca380 con 0x56435febd000 2023-09-23T03:28:31.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.272+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 34 ==== osd_op_reply(666 [pgnls-filter start_epoch 128 out=101b] v128'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:31.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.272+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 1000000021a.00000000 2023-09-23T03:28:31.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.272+0000 7ff11803a5c0 10 datascan.operator(): handling object 1000000021a.0 2023-09-23T03:28:31.274 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.272+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:667 18.27 18:e5a5749e:::1000000021a.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca000 con 0x56435febd000 2023-09-23T03:28:31.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.274+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 35 ==== osd_op_reply(667 1000000021a.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:31.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.274+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000021a 2023-09-23T03:28:31.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.274+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/37 2023-09-23T03:28:31.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.274+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=37 2023-09-23T03:28:31.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.274+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:668 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecae00 con 0x56435fdcbc00 2023-09-23T03:28:31.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.275+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 129 ==== osd_op_reply(668 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:31.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.275+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:31.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.275+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:669 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb180 con 0x56435febc000 2023-09-23T03:28:31.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.276+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 264 ==== osd_op_reply(669 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.276+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:31.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.276+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:31.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.276+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:31.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.276+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:670 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb500 con 0x56435febcc00 2023-09-23T03:28:31.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.278+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 155 ==== osd_op_reply(670 10000000000.01800000 [omap-get-header out=274b] v0'0 uv49 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.278+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 49 2023-09-23T03:28:31.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.278+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:31.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.278+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:671 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb880 con 0x56435febcc00 2023-09-23T03:28:31.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.279+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 156 ==== osd_op_reply(671 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv49 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.279+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=37 frag=1*, r=0 2023-09-23T03:28:31.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.279+0000 7ff11803a5c0 20 datascan.read_dentry: 37_head not found in result 2023-09-23T03:28:31.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.279+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000021a at 0x0x10000000000/37 with size=2 bytes 2023-09-23T03:28:31.281 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.279+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:672 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecbc00 con 0x56435febcc00 2023-09-23T03:28:31.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.283+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 157 ==== osd_op_reply(672 10000000000.01800000 [omap-set-vals] v128'50 uv50 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.283+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/37 pointing to 0x0x1000000021a 2023-09-23T03:28:31.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.283+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:673 18.27 18:e692fc64::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefee00 con 0x56435febd000 2023-09-23T03:28:31.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.284+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 36 ==== osd_op_reply(673 [pgnls-filter start_epoch 128 out=81b] v128'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:31.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.284+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 1000000021b.00000000 2023-09-23T03:28:31.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.284+0000 7ff11803a5c0 10 datascan.operator(): handling object 1000000021b.0 2023-09-23T03:28:31.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.284+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:674 18.27 18:e692fc64:::1000000021b.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff180 con 0x56435febd000 2023-09-23T03:28:31.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.285+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 37 ==== osd_op_reply(674 1000000021b.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:31.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.286+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000021b 2023-09-23T03:28:31.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.286+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/38 2023-09-23T03:28:31.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.286+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=38 2023-09-23T03:28:31.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.286+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:675 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff880 con 0x56435fdcbc00 2023-09-23T03:28:31.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.287+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 130 ==== osd_op_reply(675 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:31.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.287+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:31.288 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.287+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:676 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feffc00 con 0x56435febc000 2023-09-23T03:28:31.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.288+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 265 ==== osd_op_reply(676 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.288+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:31.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.288+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:31.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.288+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:31.289 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.288+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:677 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff79180 con 0x56435febc000 2023-09-23T03:28:31.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.289+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 266 ==== osd_op_reply(677 10000000000.01000000 [omap-get-header out=274b] v0'0 uv53 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.289+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 53 2023-09-23T03:28:31.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.289+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:31.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.289+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:678 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d880 con 0x56435febc000 2023-09-23T03:28:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.290+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 267 ==== osd_op_reply(678 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv53 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.290+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=38 frag=0*, r=0 2023-09-23T03:28:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.290+0000 7ff11803a5c0 20 datascan.read_dentry: 38_head not found in result 2023-09-23T03:28:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.290+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000021b at 0x0x10000000000/38 with size=2 bytes 2023-09-23T03:28:31.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.290+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:679 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5ee00 con 0x56435febc000 2023-09-23T03:28:31.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.294+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 268 ==== osd_op_reply(679 10000000000.01000000 [omap-set-vals] v128'54 uv54 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.294+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/38 pointing to 0x0x1000000021b 2023-09-23T03:28:31.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.294+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:680 18.17 18:e8000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5e000 con 0x56435febd000 2023-09-23T03:28:31.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.296+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 38 ==== osd_op_reply(680 [pgnls-filter start_epoch 128 out=101b] v128'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:31.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.296+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000215.00000000 2023-09-23T03:28:31.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.296+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000215.0 2023-09-23T03:28:31.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.296+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:681 18.17 18:eace0b58:::10000000215.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5fc00 con 0x56435febd000 2023-09-23T03:28:31.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.298+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 39 ==== osd_op_reply(681 10000000215.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv14 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:31.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.298+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000215 2023-09-23T03:28:31.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.298+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/32 2023-09-23T03:28:31.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.298+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=32 2023-09-23T03:28:31.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.298+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:682 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff55880 con 0x56435fdcbc00 2023-09-23T03:28:31.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.299+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 131 ==== osd_op_reply(682 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:31.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.299+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:31.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.299+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:683 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff55c00 con 0x56435febc000 2023-09-23T03:28:31.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.300+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 269 ==== osd_op_reply(683 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.300+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:31.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.300+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:31.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.300+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:31.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.300+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:684 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff54000 con 0x56435febcc00 2023-09-23T03:28:31.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.301+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 158 ==== osd_op_reply(684 10000000000.01800000 [omap-get-header out=274b] v0'0 uv50 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.301+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 50 2023-09-23T03:28:31.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.301+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:31.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.301+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:685 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff7c380 con 0x56435febcc00 2023-09-23T03:28:31.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.303+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 159 ==== osd_op_reply(685 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv50 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.303+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=32 frag=1*, r=0 2023-09-23T03:28:31.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.303+0000 7ff11803a5c0 20 datascan.read_dentry: 32_head not found in result 2023-09-23T03:28:31.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.303+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000215 at 0x0x10000000000/32 with size=2 bytes 2023-09-23T03:28:31.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.303+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:686 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff7c000 con 0x56435febcc00 2023-09-23T03:28:31.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.307+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 160 ==== osd_op_reply(686 10000000000.01800000 [omap-set-vals] v128'51 uv51 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.308 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.307+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/32 pointing to 0x0x10000000215 2023-09-23T03:28:31.309 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.307+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:687 18.17 18:eb1b33a4::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca700 con 0x56435febd000 2023-09-23T03:28:31.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.308+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 40 ==== osd_op_reply(687 [pgnls-filter start_epoch 128 out=81b] v128'15 uv15 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:31.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.308+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000253.00000000 2023-09-23T03:28:31.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.308+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000253.0 2023-09-23T03:28:31.310 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.308+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:688 18.17 18:eb1b33a4:::10000000253.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe000 con 0x56435febd000 2023-09-23T03:28:31.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.310+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 41 ==== osd_op_reply(688 10000000253.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:31.311 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.310+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000253 2023-09-23T03:28:31.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.310+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/94 2023-09-23T03:28:31.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.310+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=94 2023-09-23T03:28:31.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.310+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:689 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe380 con 0x56435fdcbc00 2023-09-23T03:28:31.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.311+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 132 ==== osd_op_reply(689 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:31.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.311+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:31.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.311+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:690 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefea80 con 0x56435febc000 2023-09-23T03:28:31.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.312+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 270 ==== osd_op_reply(690 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.312+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:31.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.312+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:31.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.312+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:31.314 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.312+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:691 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fab3880 con 0x56435febcc00 2023-09-23T03:28:31.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.313+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 161 ==== osd_op_reply(691 10000000000.01800000 [omap-get-header out=274b] v0'0 uv51 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.313+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 51 2023-09-23T03:28:31.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.313+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:31.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.313+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:692 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff79500 con 0x56435febcc00 2023-09-23T03:28:31.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.315+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 162 ==== osd_op_reply(692 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv51 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.315+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=94 frag=1*, r=0 2023-09-23T03:28:31.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.315+0000 7ff11803a5c0 20 datascan.read_dentry: 94_head not found in result 2023-09-23T03:28:31.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.315+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000253 at 0x0x10000000000/94 with size=2 bytes 2023-09-23T03:28:31.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.315+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:693 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff79c00 con 0x56435febcc00 2023-09-23T03:28:31.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.318+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 163 ==== osd_op_reply(693 10000000000.01800000 [omap-set-vals] v128'52 uv52 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.318+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/94 pointing to 0x0x10000000253 2023-09-23T03:28:31.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.318+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:694 18.37 18:ec000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff78000 con 0x56435fdca400 2023-09-23T03:28:31.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.319+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.7 v2:172.21.15.190:6824/3460136272 25 ==== osd_op_reply(694 [pgnls-filter start_epoch 128 out=49b] v126'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56435fd3ed80 con 0x56435fdca400 2023-09-23T03:28:31.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.319+0000 7ff11185f640 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:695 18.f 18:f0000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff78700 con 0x56435febd000 2023-09-23T03:28:31.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.320+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 42 ==== osd_op_reply(695 [pgnls-filter start_epoch 128 out=81b] v128'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:31.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.321+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 1000000022e.00000000 2023-09-23T03:28:31.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.321+0000 7ff11803a5c0 10 datascan.operator(): handling object 1000000022e.0 2023-09-23T03:28:31.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.321+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:696 18.f 18:f25dd5df:::1000000022e.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff78e00 con 0x56435febd000 2023-09-23T03:28:31.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.322+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.5 v2:172.21.15.190:6800/882536885 43 ==== osd_op_reply(696 1000000022e.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febd000 2023-09-23T03:28:31.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.322+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000022e 2023-09-23T03:28:31.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.322+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/57 2023-09-23T03:28:31.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.322+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=57 2023-09-23T03:28:31.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.322+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:697 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1ce00 con 0x56435fdcbc00 2023-09-23T03:28:31.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.323+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 133 ==== osd_op_reply(697 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:31.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.323+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:31.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.323+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:698 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1c700 con 0x56435febc000 2023-09-23T03:28:31.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.324+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 271 ==== osd_op_reply(698 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.324+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:31.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.324+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:31.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.324+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:31.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.324+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:699 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1c380 con 0x56435febcc00 2023-09-23T03:28:31.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.325+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 164 ==== osd_op_reply(699 10000000000.01800000 [omap-get-header out=274b] v0'0 uv52 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.325+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 52 2023-09-23T03:28:31.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.325+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:31.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.325+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:700 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d180 con 0x56435febcc00 2023-09-23T03:28:31.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.326+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 165 ==== osd_op_reply(700 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv52 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.327+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=57 frag=1*, r=0 2023-09-23T03:28:31.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.327+0000 7ff11803a5c0 20 datascan.read_dentry: 57_head not found in result 2023-09-23T03:28:31.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.327+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000022e at 0x0x10000000000/57 with size=2 bytes 2023-09-23T03:28:31.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.327+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:701 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1c000 con 0x56435febcc00 2023-09-23T03:28:31.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.330+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 166 ==== osd_op_reply(701 10000000000.01800000 [omap-set-vals] v128'53 uv53 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.330+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/57 pointing to 0x0x1000000022e 2023-09-23T03:28:31.331 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.330+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:702 18.2f 18:f4000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d500 con 0x56435fdcb000 2023-09-23T03:28:31.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.331+0000 7ff116068640 1 -- 172.21.15.154:0/405849260 <== osd.1 v2:172.21.15.154:6816/1267710005 20 ==== osd_op_reply(702 [pgnls-filter start_epoch 128 out=49b] v126'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56435fd3efc0 con 0x56435fdcb000 2023-09-23T03:28:31.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.331+0000 7ff11185f640 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:703 18.1f 18:f8000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff78a80 con 0x56435fdca400 2023-09-23T03:28:31.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.332+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.7 v2:172.21.15.190:6824/3460136272 26 ==== osd_op_reply(703 [pgnls-filter start_epoch 128 out=101b] v128'12 uv12 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3ed80 con 0x56435fdca400 2023-09-23T03:28:31.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.332+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000238.00000000 2023-09-23T03:28:31.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.332+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000238.0 2023-09-23T03:28:31.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.333+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:704 18.1f 18:f87ea8f1:::10000000238.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1ca80 con 0x56435fdca400 2023-09-23T03:28:31.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.334+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.7 v2:172.21.15.190:6824/3460136272 27 ==== osd_op_reply(704 10000000238.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435fdca400 2023-09-23T03:28:31.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.334+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000238 2023-09-23T03:28:31.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.334+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/67 2023-09-23T03:28:31.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.334+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=67 2023-09-23T03:28:31.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.334+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:705 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff54a80 con 0x56435fdcbc00 2023-09-23T03:28:31.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.335+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 134 ==== osd_op_reply(705 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:31.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.335+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:31.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.335+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:706 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff55180 con 0x56435febc000 2023-09-23T03:28:31.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.336+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 272 ==== osd_op_reply(706 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.336+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:31.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.336+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:31.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.336+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:31.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.336+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:707 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff54380 con 0x56435febcc00 2023-09-23T03:28:31.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.337+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 167 ==== osd_op_reply(707 10000000000.01800000 [omap-get-header out=274b] v0'0 uv53 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.338+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 53 2023-09-23T03:28:31.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.338+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:31.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.338+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:708 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff54700 con 0x56435febcc00 2023-09-23T03:28:31.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.338+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 168 ==== osd_op_reply(708 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv53 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.339+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=67 frag=1*, r=0 2023-09-23T03:28:31.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.339+0000 7ff11803a5c0 20 datascan.read_dentry: 67_head not found in result 2023-09-23T03:28:31.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.339+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000238 at 0x0x10000000000/67 with size=2 bytes 2023-09-23T03:28:31.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.339+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:709 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff55500 con 0x56435febcc00 2023-09-23T03:28:31.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.342+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 169 ==== osd_op_reply(709 10000000000.01800000 [omap-set-vals] v128'54 uv54 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.342+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/67 pointing to 0x0x10000000238 2023-09-23T03:28:31.344 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.342+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:710 18.1f 18:f8e2d5cc::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff54e00 con 0x56435fdca400 2023-09-23T03:28:31.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.344+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.7 v2:172.21.15.190:6824/3460136272 28 ==== osd_op_reply(710 [pgnls-filter start_epoch 128 out=81b] v128'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3ed80 con 0x56435fdca400 2023-09-23T03:28:31.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.344+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000214.00000000 2023-09-23T03:28:31.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.344+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000214.0 2023-09-23T03:28:31.345 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.344+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:711 18.1f 18:f8e2d5cc:::10000000214.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff79180 con 0x56435fdca400 2023-09-23T03:28:31.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.346+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.7 v2:172.21.15.190:6824/3460136272 29 ==== osd_op_reply(711 10000000214.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3ed80 con 0x56435fdca400 2023-09-23T03:28:31.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.346+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000214 2023-09-23T03:28:31.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.346+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/31 2023-09-23T03:28:31.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.346+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=31 2023-09-23T03:28:31.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.346+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:712 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff78700 con 0x56435fdcbc00 2023-09-23T03:28:31.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.347+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 135 ==== osd_op_reply(712 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:31.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.347+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:31.348 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.347+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:713 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefee00 con 0x56435febc000 2023-09-23T03:28:31.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.348+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 273 ==== osd_op_reply(713 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.348+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:31.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.348+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:31.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.348+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:31.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.348+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:714 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff500 con 0x56435febc000 2023-09-23T03:28:31.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.349+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 274 ==== osd_op_reply(714 10000000000.01000000 [omap-get-header out=274b] v0'0 uv54 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.349+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 54 2023-09-23T03:28:31.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.349+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:31.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.349+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:715 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feff180 con 0x56435febc000 2023-09-23T03:28:31.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.350+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 275 ==== osd_op_reply(715 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv54 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.350+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=31 frag=0*, r=0 2023-09-23T03:28:31.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.350+0000 7ff11803a5c0 20 datascan.read_dentry: 31_head not found in result 2023-09-23T03:28:31.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.350+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000214 at 0x0x10000000000/31 with size=2 bytes 2023-09-23T03:28:31.352 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.350+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:716 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feffc00 con 0x56435febc000 2023-09-23T03:28:31.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.354+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 276 ==== osd_op_reply(716 10000000000.01000000 [omap-set-vals] v128'55 uv55 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.354+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/31 pointing to 0x0x10000000214 2023-09-23T03:28:31.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.354+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:717 18.3f 18:fc000000::::head [pgnls-filter start_epoch 128 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe700 con 0x56435febcc00 2023-09-23T03:28:31.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.355+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 170 ==== osd_op_reply(717 [pgnls-filter start_epoch 128 out=101b] v128'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.355+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 10000000232.00000000 2023-09-23T03:28:31.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.355+0000 7ff11803a5c0 10 datascan.operator(): handling object 10000000232.0 2023-09-23T03:28:31.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.355+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:718 18.3f 18:fd83cc12:::10000000232.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fefe000 con 0x56435febcc00 2023-09-23T03:28:31.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.357+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 171 ==== osd_op_reply(718 10000000232.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.357+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000232 2023-09-23T03:28:31.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.357+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/61 2023-09-23T03:28:31.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.357+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=61 2023-09-23T03:28:31.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.357+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:719 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca380 con 0x56435fdcbc00 2023-09-23T03:28:31.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.358+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 136 ==== osd_op_reply(719 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:31.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.358+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:31.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.358+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:720 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecaa80 con 0x56435febc000 2023-09-23T03:28:31.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.359+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 277 ==== osd_op_reply(720 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.359+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:31.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.359+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:31.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.359+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 0* 2023-09-23T03:28:31.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.359+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:721 17.5 17:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca000 con 0x56435febc000 2023-09-23T03:28:31.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.360+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 278 ==== osd_op_reply(721 10000000000.01000000 [omap-get-header out=274b] v0'0 uv55 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.360+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 55 2023-09-23T03:28:31.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.360+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-09-23T03:28:31.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.360+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:722 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435fecb180 con 0x56435febc000 2023-09-23T03:28:31.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.361+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 279 ==== osd_op_reply(722 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv55 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.362+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=61 frag=0*, r=0 2023-09-23T03:28:31.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.362+0000 7ff11803a5c0 20 datascan.read_dentry: 61_head not found in result 2023-09-23T03:28:31.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.362+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000232 at 0x0x10000000000/61 with size=2 bytes 2023-09-23T03:28:31.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.362+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:723 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435feca700 con 0x56435febc000 2023-09-23T03:28:31.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.365+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 280 ==== osd_op_reply(723 10000000000.01000000 [omap-set-vals] v128'56 uv56 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.365+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/61 pointing to 0x0x10000000232 2023-09-23T03:28:31.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.365+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:724 18.3f 18:feb1c803::::head [pgnls-filter start_epoch 128 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff55c00 con 0x56435febcc00 2023-09-23T03:28:31.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.366+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 172 ==== osd_op_reply(724 [pgnls-filter start_epoch 128 out=81b] v128'14 uv14 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.367+0000 7ff11803a5c0 20 datascan.forall_objects: OSD matched oid 100000001fa.00000000 2023-09-23T03:28:31.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.367+0000 7ff11803a5c0 10 datascan.operator(): handling object 100000001fa.0 2023-09-23T03:28:31.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.367+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:725 18.3f 18:feb1c803:::100000001fa.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5ee00 con 0x56435febcc00 2023-09-23T03:28:31.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.368+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 173 ==== osd_op_reply(725 100000001fa.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv14 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.368+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001fa 2023-09-23T03:28:31.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.368+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/5 2023-09-23T03:28:31.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.368+0000 7ff11803a5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=5 2023-09-23T03:28:31.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.368+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:726 17.10 17:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5e380 con 0x56435fdcbc00 2023-09-23T03:28:31.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.369+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.3 v2:172.21.15.154:6808/1133688468 137 ==== osd_op_reply(726 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x56435fd3eb40 con 0x56435fdcbc00 2023-09-23T03:28:31.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.369+0000 7ff11803a5c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-09-23T03:28:31.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.369+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:727 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff5fc00 con 0x56435febc000 2023-09-23T03:28:31.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.370+0000 7ff116869640 1 -- 172.21.15.154:0/405849260 <== osd.6 v2:172.21.15.190:6816/1964294001 281 ==== osd_op_reply(727 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x56435fd3ed80 con 0x56435febc000 2023-09-23T03:28:31.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.370+0000 7ff11803a5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-09-23T03:28:31.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.370+0000 7ff11803a5c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-09-23T03:28:31.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.370+0000 7ff11803a5c0 20 datascan.get_frag_of: frag is 1* 2023-09-23T03:28:31.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.370+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:728 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff1d880 con 0x56435febcc00 2023-09-23T03:28:31.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.371+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 174 ==== osd_op_reply(728 10000000000.01800000 [omap-get-header out=274b] v0'0 uv54 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.371+0000 7ff11803a5c0 10 datascan.find_or_create_dirfrag: read_version = 54 2023-09-23T03:28:31.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.371+0000 7ff11803a5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-09-23T03:28:31.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.371+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:729 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff7d500 con 0x56435febcc00 2023-09-23T03:28:31.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.372+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 175 ==== osd_op_reply(729 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv54 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.373+0000 7ff11803a5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=5 frag=1*, r=0 2023-09-23T03:28:31.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.373+0000 7ff11803a5c0 20 datascan.read_dentry: 5_head not found in result 2023-09-23T03:28:31.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.373+0000 7ff11803a5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fa at 0x0x10000000000/5 with size=1 bytes 2023-09-23T03:28:31.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.373+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:730 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x56435ff7dc00 con 0x56435febcc00 2023-09-23T03:28:31.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.376+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 <== osd.4 v2:172.21.15.190:6808/717302641 176 ==== osd_op_reply(730 10000000000.01800000 [omap-set-vals] v128'55 uv55 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56435fd3eb40 con 0x56435febcc00 2023-09-23T03:28:31.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.376+0000 7ff11803a5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/5 pointing to 0x0x100000001fa 2023-09-23T03:28:31.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.377+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x56435febc400 msgr2=0x56435fac3600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:31.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.377+0000 7ff11803a5c0 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x56435febc400 0x56435fac3600 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:31.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.377+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x56435fdcb000 msgr2=0x56435fedab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:31.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.377+0000 7ff11803a5c0 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x56435fdcb000 0x56435fedab00 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:31.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.377+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x56435ff15800 msgr2=0x56435fedb080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:31.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.377+0000 7ff11803a5c0 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x56435ff15800 0x56435fedb080 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:31.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.377+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x56435fdcbc00 msgr2=0x56435fac4100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:31.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.377+0000 7ff11803a5c0 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x56435fdcbc00 0x56435fac4100 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:31.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.377+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x56435febcc00 msgr2=0x56435feda000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:31.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.377+0000 7ff11803a5c0 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x56435febcc00 0x56435feda000 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:31.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.377+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x56435febd000 msgr2=0x56435feda580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:31.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.377+0000 7ff11803a5c0 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x56435febd000 0x56435feda580 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:31.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.377+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 reap_dead start 2023-09-23T03:28:31.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.377+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x56435febc000 msgr2=0x56435fac3b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:31.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.377+0000 7ff11803a5c0 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x56435febc000 0x56435fac3b80 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:31.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.377+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x56435fdca400 msgr2=0x56435fedc680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:31.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.377+0000 7ff11803a5c0 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x56435fdca400 0x56435fedc680 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:31.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.377+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 reap_dead start 2023-09-23T03:28:31.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.377+0000 7ff115867640 1 -- 172.21.15.154:0/405849260 reap_dead start 2023-09-23T03:28:31.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.377+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x56435fdcb800 msgr2=0x56435fac5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:31.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.377+0000 7ff11803a5c0 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x56435fdcb800 0x56435fac5700 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x5643607d4420 tx=0x56435fb38420 comp rx=0 tx=0).stop 2023-09-23T03:28:31.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.377+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56435fd4f400 msgr2=0x56435fac4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:31.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.377+0000 7ff11803a5c0 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56435fd4f400 0x56435fac4c00 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x56435fd62cf0 tx=0x56435fb38060 comp rx=0 tx=0).stop 2023-09-23T03:28:31.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.378+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 shutdown_connections 2023-09-23T03:28:31.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.378+0000 7ff11803a5c0 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x56435fdcb800 0x56435fac5700 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:31.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.378+0000 7ff11803a5c0 1 --2- 172.21.15.154:0/405849260 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56435fd4f400 0x56435fac4c00 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:31.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.378+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 >> 172.21.15.154:0/405849260 conn(0x56435fd4ec00 msgr2=0x5643607d1800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:31.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.378+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 shutdown_connections 2023-09-23T03:28:31.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.378+0000 7ff11803a5c0 1 -- 172.21.15.154:0/405849260 wait complete. 2023-09-23T03:28:31.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.379+0000 7ff11803a5c0 1 -- 172.21.15.154:0/291424263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56435fa58c00 msgr2=0x56435fac2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:31.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.379+0000 7ff11803a5c0 1 --2- 172.21.15.154:0/291424263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56435fa58c00 0x56435fac2580 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x564360867860 tx=0x56435fb38600 comp rx=0 tx=0).stop 2023-09-23T03:28:31.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.379+0000 7ff11803a5c0 1 -- 172.21.15.154:0/291424263 shutdown_connections 2023-09-23T03:28:31.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.379+0000 7ff11803a5c0 1 --2- 172.21.15.154:0/291424263 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x56435fa59c00 0x56435fac3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:31.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.379+0000 7ff11803a5c0 1 --2- 172.21.15.154:0/291424263 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x56435fa59800 0x56435fac2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:31.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.379+0000 7ff11803a5c0 1 --2- 172.21.15.154:0/291424263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56435fa58c00 0x56435fac2580 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:31.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.379+0000 7ff11803a5c0 1 -- 172.21.15.154:0/291424263 >> 172.21.15.154:0/291424263 conn(0x56435fa59000 msgr2=0x5643607d1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:31.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.380+0000 7ff11803a5c0 1 -- 172.21.15.154:0/291424263 shutdown_connections 2023-09-23T03:28:31.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.380+0000 7ff11803a5c0 1 -- 172.21.15.154:0/291424263 wait complete. 2023-09-23T03:28:31.392 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes'] in time 0:00:01.334300, result: 2023-09-23T03:28:31.392 DEBUG:teuthology.orchestra.run.smithi154:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2023-09-23T03:28:31.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.532+0000 7f50a463f5c0 1 -- start start 2023-09-23T03:28:31.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.532+0000 7f50a463f5c0 1 Processor -- start 2023-09-23T03:28:31.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.533+0000 7f50a463f5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e8cf8c4c00 0x55e8cf92e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:31.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.533+0000 7f50a463f5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8cf8c5800 0x55e8cf92eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:31.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.533+0000 7f50a463f5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8cf8c5c00 0x55e8cf92f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:31.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.533+0000 7f50a463f5c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8cf90f180 con 0x55e8cf8c5c00 2023-09-23T03:28:31.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.533+0000 7f50a463f5c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8cf90e140 con 0x55e8cf8c5800 2023-09-23T03:28:31.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.533+0000 7f50a463f5c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e8cf90e000 con 0x55e8cf8c4c00 2023-09-23T03:28:31.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.533+0000 7f50a2e48640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8cf8c5c00 0x55e8cf92f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:31.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.533+0000 7f50a2e48640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8cf8c5c00 0x55e8cf92f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53822/0 (socket says 172.21.15.154:53822) 2023-09-23T03:28:31.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.533+0000 7f50a2e48640 1 -- 172.21.15.154:0/3946230653 learned_addr learned my addr 172.21.15.154:0/3946230653 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:31.536 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.533+0000 7f50a2647640 1 --2- 172.21.15.154:0/3946230653 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e8cf8c4c00 0x55e8cf92e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:31.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.533+0000 7f50a1e46640 1 --2- 172.21.15.154:0/3946230653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8cf8c5800 0x55e8cf92eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:31.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.533+0000 7f50a2e48640 1 -- 172.21.15.154:0/3946230653 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e8cf8c4c00 msgr2=0x55e8cf92e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:31.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.533+0000 7f50a2e48640 1 --2- 172.21.15.154:0/3946230653 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e8cf8c4c00 0x55e8cf92e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:31.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.533+0000 7f50a2e48640 1 -- 172.21.15.154:0/3946230653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8cf8c5800 msgr2=0x55e8cf92eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:31.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.533+0000 7f50a2e48640 1 --2- 172.21.15.154:0/3946230653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8cf8c5800 0x55e8cf92eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:31.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.533+0000 7f50a2e48640 1 -- 172.21.15.154:0/3946230653 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8cf8f8b60 con 0x55e8cf8c5c00 2023-09-23T03:28:31.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.534+0000 7f50a2e48640 1 --2- 172.21.15.154:0/3946230653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8cf8c5c00 0x55e8cf92f080 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x55e8d06cb3e0 tx=0x55e8cfba6420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5822b9fdbf381e6d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:31.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.534+0000 7f509fe42640 1 -- 172.21.15.154:0/3946230653 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e8cf90d600 con 0x55e8cf8c5c00 2023-09-23T03:28:31.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.534+0000 7f509fe42640 1 -- 172.21.15.154:0/3946230653 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e8cf8eb800 con 0x55e8cf8c5c00 2023-09-23T03:28:31.537 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.534+0000 7f509fe42640 1 -- 172.21.15.154:0/3946230653 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e8cf90d1e0 con 0x55e8cf8c5c00 2023-09-23T03:28:31.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.534+0000 7f50a463f5c0 1 -- 172.21.15.154:0/3946230653 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8cf8f8340 con 0x55e8cf8c5c00 2023-09-23T03:28:31.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.534+0000 7f50a463f5c0 1 -- 172.21.15.154:0/3946230653 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55e8cf8f8000 con 0x55e8cf8c5c00 2023-09-23T03:28:31.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.534+0000 7f50a463f5c0 4 waiting for MDS map... 2023-09-23T03:28:31.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.534+0000 7f509fe42640 1 -- 172.21.15.154:0/3946230653 <== mon.0 v2:172.21.15.154:3300/0 4 ==== osd_map(128..128 src has 1..128) v4 ==== 6201+0+0 (secure 0 0 0) 0x55e8cf984c40 con 0x55e8cf8c5c00 2023-09-23T03:28:31.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.534+0000 7f509fe42640 1 -- 172.21.15.154:0/3946230653 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 116) v1 ==== 3298+0+0 (secure 0 0 0) 0x55e8cf917200 con 0x55e8cf8c5c00 2023-09-23T03:28:31.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.534+0000 7f50a463f5c0 4 Got MDS map 116 2023-09-23T03:28:31.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.535+0000 7f50a463f5c0 4 datascan.main: Using metadata pool output 2023-09-23T03:28:31.538 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.535+0000 7f50a463f5c0 4 datascan.main: connecting to RADOS... 2023-09-23T03:28:31.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.535+0000 7f50a463f5c0 1 Processor -- start 2023-09-23T03:28:31.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.536+0000 7f50a463f5c0 1 -- start start 2023-09-23T03:28:31.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.536+0000 7f50a463f5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8cfbb7000 0x55e8cf92f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:31.541 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.536+0000 7f50a463f5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8cfbb7400 0x55e8cf92fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:31.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.536+0000 7f50a463f5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e8cfbb7800 0x55e8cf930100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:31.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.536+0000 7f50a463f5c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8cf90ea00 con 0x55e8cfbb7400 2023-09-23T03:28:31.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.536+0000 7f50a463f5c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8cf993180 con 0x55e8cfbb7000 2023-09-23T03:28:31.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.536+0000 7f50a463f5c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e8cf992dc0 con 0x55e8cfbb7800 2023-09-23T03:28:31.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.537+0000 7f50a2647640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8cfbb7400 0x55e8cf92fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:31.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.537+0000 7f50a2647640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8cfbb7400 0x55e8cf92fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53834/0 (socket says 172.21.15.154:53834) 2023-09-23T03:28:31.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.537+0000 7f50a2647640 1 -- 172.21.15.154:0/100363072 learned_addr learned my addr 172.21.15.154:0/100363072 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:31.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.537+0000 7f50a2647640 1 -- 172.21.15.154:0/100363072 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e8cfbb7800 msgr2=0x55e8cf930100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:31.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.537+0000 7f50a2647640 1 --2- 172.21.15.154:0/100363072 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e8cfbb7800 0x55e8cf930100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:31.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.537+0000 7f50a1e46640 1 --2- 172.21.15.154:0/100363072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8cfbb7000 0x55e8cf92f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:31.542 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.537+0000 7f50a2647640 1 -- 172.21.15.154:0/100363072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8cfbb7000 msgr2=0x55e8cf92f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:31.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.537+0000 7f50a2647640 1 --2- 172.21.15.154:0/100363072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8cfbb7000 0x55e8cf92f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:31.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.537+0000 7f50a2647640 1 -- 172.21.15.154:0/100363072 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8cf8f8ea0 con 0x55e8cfbb7400 2023-09-23T03:28:31.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.537+0000 7f50a1e46640 1 --2- 172.21.15.154:0/100363072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8cfbb7000 0x55e8cf92f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:31.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.537+0000 7f50a2647640 1 --2- 172.21.15.154:0/100363072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8cfbb7400 0x55e8cf92fb80 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x55e8d06caa80 tx=0x55e8cfba61e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f482948b8223d6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:31.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.537+0000 7f509de3e640 1 -- 172.21.15.154:0/100363072 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e8cf90d080 con 0x55e8cfbb7400 2023-09-23T03:28:31.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.538+0000 7f509de3e640 1 -- 172.21.15.154:0/100363072 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e8d0694900 con 0x55e8cfbb7400 2023-09-23T03:28:31.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.538+0000 7f509de3e640 1 -- 172.21.15.154:0/100363072 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e8cf90d600 con 0x55e8cfbb7400 2023-09-23T03:28:31.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.538+0000 7f50a463f5c0 1 -- 172.21.15.154:0/100363072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8cfbb7400 msgr2=0x55e8cf92fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:31.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.538+0000 7f50a463f5c0 1 --2- 172.21.15.154:0/100363072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8cfbb7400 0x55e8cf92fb80 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x55e8d06caa80 tx=0x55e8cfba61e0 comp rx=0 tx=0).stop 2023-09-23T03:28:31.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.538+0000 7f50a463f5c0 1 -- 172.21.15.154:0/100363072 shutdown_connections 2023-09-23T03:28:31.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.538+0000 7f50a463f5c0 1 --2- 172.21.15.154:0/100363072 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e8cfbb7800 0x55e8cf930100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:31.543 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.538+0000 7f50a463f5c0 1 --2- 172.21.15.154:0/100363072 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8cfbb7400 0x55e8cf92fb80 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:31.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.538+0000 7f50a463f5c0 1 --2- 172.21.15.154:0/100363072 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8cfbb7000 0x55e8cf92f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:31.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.538+0000 7f50a463f5c0 1 -- 172.21.15.154:0/100363072 >> 172.21.15.154:0/100363072 conn(0x55e8cfbb6c00 msgr2=0x55e8d063f000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:31.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.538+0000 7f50a463f5c0 1 -- 172.21.15.154:0/100363072 shutdown_connections 2023-09-23T03:28:31.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.538+0000 7f50a463f5c0 1 -- 172.21.15.154:0/100363072 wait complete. 2023-09-23T03:28:31.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.539+0000 7f50a463f5c0 1 Processor -- start 2023-09-23T03:28:31.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.539+0000 7f50a463f5c0 1 -- start start 2023-09-23T03:28:31.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.540+0000 7f50a463f5c0 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8cfbb7000 0x55e8cf931180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:31.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.540+0000 7f50a463f5c0 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e8cfbb7400 0x55e8cf930c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:31.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.540+0000 7f50a463f5c0 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8cfbb7800 0x55e8cf930680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:31.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.540+0000 7f50a463f5c0 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8cf992c80 con 0x55e8cfbb7000 2023-09-23T03:28:31.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.540+0000 7f50a463f5c0 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8cf90e140 con 0x55e8cfbb7800 2023-09-23T03:28:31.544 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.540+0000 7f50a463f5c0 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e8cf90eb40 con 0x55e8cfbb7400 2023-09-23T03:28:31.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.540+0000 7f50a1e46640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8cfbb7000 0x55e8cf931180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:31.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.540+0000 7f50a1e46640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8cfbb7000 0x55e8cf931180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53848/0 (socket says 172.21.15.154:53848) 2023-09-23T03:28:31.545 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.540+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 learned_addr learned my addr 172.21.15.154:0/26935237 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:31.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.541+0000 7f50a2647640 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e8cfbb7400 0x55e8cf930c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:31.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.541+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e8cfbb7400 msgr2=0x55e8cf930c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:31.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.541+0000 7f50a1e46640 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e8cfbb7400 0x55e8cf930c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:31.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.541+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8cfbb7800 msgr2=0x55e8cf930680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:31.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.541+0000 7f50a1e46640 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8cfbb7800 0x55e8cf930680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:31.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.541+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8cf8f91e0 con 0x55e8cfbb7000 2023-09-23T03:28:31.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.541+0000 7f50a1e46640 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8cfbb7000 0x55e8cf931180 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x55e8cfbc9440 tx=0x55e8cfbcccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:31.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.541+0000 7f509c63b640 1 -- 172.21.15.154:0/26935237 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e8cf90d080 con 0x55e8cfbb7000 2023-09-23T03:28:31.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.541+0000 7f509c63b640 1 -- 172.21.15.154:0/26935237 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e8cf8ebb00 con 0x55e8cfbb7000 2023-09-23T03:28:31.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.541+0000 7f509c63b640 1 -- 172.21.15.154:0/26935237 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e8cf90d600 con 0x55e8cfbb7000 2023-09-23T03:28:31.547 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.541+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e8cf8f8ea0 con 0x55e8cfbb7000 2023-09-23T03:28:31.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.541+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8cf8f8b60 con 0x55e8cfbb7000 2023-09-23T03:28:31.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.541+0000 7f50a463f5c0 4 datascan.init: resolving metadata pool 17 2023-09-23T03:28:31.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.543+0000 7f509c63b640 1 -- 172.21.15.154:0/26935237 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x55e8cf8f6840 con 0x55e8cfbb7000 2023-09-23T03:28:31.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.544+0000 7f509c63b640 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55e8cfc34800 0x55e8cf931700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:31.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.544+0000 7f509c63b640 1 -- 172.21.15.154:0/26935237 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6201+0+0 (secure 0 0 0) 0x55e8cf984a80 con 0x55e8cfbb7000 2023-09-23T03:28:31.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.544+0000 7f50a463f5c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-09-23T03:28:31.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.544+0000 7f50a463f5c0 4 datascan.main: resolving metadata pool 17 2023-09-23T03:28:31.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.544+0000 7f50a463f5c0 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55e8cfc34c00 0x55e8cf930100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:31.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.544+0000 7f50a2647640 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55e8cfc34800 0x55e8cf931700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:31.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.544+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:1 17.0 17:00000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8cfc34c00 2023-09-23T03:28:31.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.545+0000 7f50a2647640 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55e8cfc34800 0x55e8cf931700 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x55e8d06caa80 tx=0x55e8cfba60c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:31.548 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.545+0000 7f50a1e46640 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55e8cfc34c00 0x55e8cf930100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:31.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.546+0000 7f50a1e46640 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55e8cfc34c00 0x55e8cf930100 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:31.549 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.546+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 <== osd.7 v2:172.21.15.190:6824/3460136272 1 ==== osd_op_reply(1 [pgnls start_epoch 128 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d0766b40 con 0x55e8cfc34c00 2023-09-23T03:28:31.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.547+0000 7f50a1e46640 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55e8cfd1c000 0x55e8cfd1e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:31.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.547+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:2 17.10 17:08000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8cfd1c000 2023-09-23T03:28:31.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.547+0000 7f50a2e48640 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55e8cfd1c000 0x55e8cfd1e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:31.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.547+0000 7f50a2e48640 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55e8cfd1c000 0x55e8cfd1e000 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:31.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.548+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== osd_op_reply(2 [pgnls start_epoch 128 out=177b] v127'12 uv12 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1c000 2023-09-23T03:28:31.550 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.548+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:3 17.8 17:10000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8cfc34c00 2023-09-23T03:28:31.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.549+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 <== osd.7 v2:172.21.15.190:6824/3460136272 2 ==== osd_op_reply(3 [pgnls start_epoch 128 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d0766b40 con 0x55e8cfc34c00 2023-09-23T03:28:31.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.549+0000 7f50a1e46640 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55e8cfd1c400 0x55e8cfd1e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:31.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.549+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:4 17.18 17:18000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8cfd1c400 2023-09-23T03:28:31.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.549+0000 7f50a2647640 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55e8cfd1c400 0x55e8cfd1e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:31.551 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.550+0000 7f50a2647640 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55e8cfd1c400 0x55e8cfd1e580 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:31.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.550+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.0 v2:172.21.15.154:6800/2920938138 1 ==== osd_op_reply(4 [pgnls start_epoch 128 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8cfd1c400 2023-09-23T03:28:31.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.550+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:5 17.4 17:20000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8cfd1c400 2023-09-23T03:28:31.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.550+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.0 v2:172.21.15.154:6800/2920938138 2 ==== osd_op_reply(5 [pgnls start_epoch 128 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8cfd1c400 2023-09-23T03:28:31.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.550+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:6 17.14 17:28000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8cfd1c000 2023-09-23T03:28:31.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.551+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.3 v2:172.21.15.154:6808/1133688468 2 ==== osd_op_reply(6 [pgnls start_epoch 128 out=149b] v127'28 uv28 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1c000 2023-09-23T03:28:31.552 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.551+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:7 17.c 17:30000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8cfc34c00 2023-09-23T03:28:31.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.552+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 <== osd.7 v2:172.21.15.190:6824/3460136272 3 ==== osd_op_reply(7 [pgnls start_epoch 128 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d0766b40 con 0x55e8cfc34c00 2023-09-23T03:28:31.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.552+0000 7f50a1e46640 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55e8cfd1c800 0x55e8cfd1eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:31.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.552+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:8 17.1c 17:38000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8cfd1c800 2023-09-23T03:28:31.553 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.552+0000 7f50a1e46640 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55e8cfd1c800 0x55e8cfd1eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:31.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.552+0000 7f50a1e46640 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55e8cfd1c800 0x55e8cfd1eb00 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:31.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.553+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 <== osd.1 v2:172.21.15.154:6816/1267710005 1 ==== osd_op_reply(8 [pgnls start_epoch 128 out=73b] v125'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55e8d0766b40 con 0x55e8cfd1c800 2023-09-23T03:28:31.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.553+0000 7f50a1e46640 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55e8cfd1cc00 0x55e8cfd1f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:31.554 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.553+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:9 17.2 17:40000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8cfd1cc00 2023-09-23T03:28:31.555 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.554+0000 7f50a2e48640 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55e8cfd1cc00 0x55e8cfd1f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:31.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.554+0000 7f50a2e48640 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55e8cfd1cc00 0x55e8cfd1f080 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:31.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.555+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 1 ==== osd_op_reply(9 [pgnls start_epoch 128 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.555+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:10 17.12 17:48000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8cfd1c000 2023-09-23T03:28:31.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.556+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.3 v2:172.21.15.154:6808/1133688468 3 ==== osd_op_reply(10 [pgnls start_epoch 128 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1c000 2023-09-23T03:28:31.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.556+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:11 17.a 17:50000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8cfc34c00 2023-09-23T03:28:31.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.557+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 <== osd.7 v2:172.21.15.190:6824/3460136272 4 ==== osd_op_reply(11 [pgnls start_epoch 128 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d0766b40 con 0x55e8cfc34c00 2023-09-23T03:28:31.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.557+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:12 17.1a 17:58000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8cfd1c400 2023-09-23T03:28:31.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.557+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.0 v2:172.21.15.154:6800/2920938138 3 ==== osd_op_reply(12 [pgnls start_epoch 128 out=49b] v127'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8cfd1c400 2023-09-23T03:28:31.558 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.557+0000 7f50a2647640 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55e8d0697000 0x55e8cfd20100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:31.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.557+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:13 17.6 17:60000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8d0697000 2023-09-23T03:28:31.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.558+0000 7f50a2647640 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55e8d0697000 0x55e8cfd20100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:31.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.559+0000 7f50a2647640 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55e8d0697000 0x55e8cfd20100 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:31.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.560+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== osd_op_reply(13 [pgnls start_epoch 128 out=77b] v127'7 uv7 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.560+0000 7f50a2647640 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55e8d0697400 0x55e8cfd20680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:31.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.560+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:14 17.16 17:68000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8d0697400 2023-09-23T03:28:31.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.561+0000 7f50a1e46640 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55e8d0697400 0x55e8cfd20680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:31.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.562+0000 7f50a1e46640 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55e8d0697400 0x55e8cfd20680 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:31.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.562+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 <== osd.5 v2:172.21.15.190:6800/882536885 1 ==== osd_op_reply(14 [pgnls start_epoch 128 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d0766b40 con 0x55e8d0697400 2023-09-23T03:28:31.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.562+0000 7f50a1e46640 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55e8cfd1d000 0x55e8cfd1f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:31.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.562+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:15 17.e 17:70000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8cfd1d000 2023-09-23T03:28:31.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.563+0000 7f50a2e48640 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55e8cfd1d000 0x55e8cfd1f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:31.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.563+0000 7f50a2e48640 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55e8cfd1d000 0x55e8cfd1f600 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:31.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.564+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.2 v2:172.21.15.154:6822/2884407243 1 ==== osd_op_reply(15 [pgnls start_epoch 128 out=73b] v125'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1d000 2023-09-23T03:28:31.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.564+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:16 17.1e 17:78000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8cfd1c800 2023-09-23T03:28:31.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.564+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 <== osd.1 v2:172.21.15.154:6816/1267710005 2 ==== osd_op_reply(16 [pgnls start_epoch 128 out=49b] v127'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d0766b40 con 0x55e8cfd1c800 2023-09-23T03:28:31.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.564+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:17 17.1 17:80000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8cfd1c400 2023-09-23T03:28:31.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.565+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.0 v2:172.21.15.154:6800/2920938138 4 ==== osd_op_reply(17 [pgnls start_epoch 128 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8cfd1c400 2023-09-23T03:28:31.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.565+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:18 17.11 17:88000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8cfd1c000 2023-09-23T03:28:31.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.565+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.3 v2:172.21.15.154:6808/1133688468 4 ==== osd_op_reply(18 [pgnls start_epoch 128 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1c000 2023-09-23T03:28:31.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.565+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:19 17.9 17:90000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8cfd1c000 2023-09-23T03:28:31.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.565+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.3 v2:172.21.15.154:6808/1133688468 5 ==== osd_op_reply(19 [pgnls start_epoch 128 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1c000 2023-09-23T03:28:31.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.565+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:20 17.19 17:98000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8d0697400 2023-09-23T03:28:31.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.566+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 <== osd.5 v2:172.21.15.190:6800/882536885 2 ==== osd_op_reply(20 [pgnls start_epoch 128 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d0766b40 con 0x55e8d0697400 2023-09-23T03:28:31.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.566+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:21 17.5 17:a0000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8d0697000 2023-09-23T03:28:31.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.567+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 2 ==== osd_op_reply(21 [pgnls start_epoch 128 out=81b] v128'56 uv56 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.567+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:22 17.15 17:a8000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8d0697400 2023-09-23T03:28:31.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.568+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 <== osd.5 v2:172.21.15.190:6800/882536885 3 ==== osd_op_reply(22 [pgnls start_epoch 128 out=97b] v125'3 uv3 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x55e8d0766b40 con 0x55e8d0697400 2023-09-23T03:28:31.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.568+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:23 17.d 17:b0000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8cfc34c00 2023-09-23T03:28:31.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.569+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 <== osd.7 v2:172.21.15.190:6824/3460136272 5 ==== osd_op_reply(23 [pgnls start_epoch 128 out=74b] v125'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55e8d0766b40 con 0x55e8cfc34c00 2023-09-23T03:28:31.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.569+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:24 17.1d 17:b8000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8cfc34c00 2023-09-23T03:28:31.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.570+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 <== osd.7 v2:172.21.15.190:6824/3460136272 6 ==== osd_op_reply(24 [pgnls start_epoch 128 out=73b] v125'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55e8d0766b40 con 0x55e8cfc34c00 2023-09-23T03:28:31.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.570+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:25 17.3 17:c0000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8cfd1cc00 2023-09-23T03:28:31.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.571+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 2 ==== osd_op_reply(25 [pgnls start_epoch 128 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.571+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:26 17.13 17:c8000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8cfd1c000 2023-09-23T03:28:31.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.572+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.3 v2:172.21.15.154:6808/1133688468 6 ==== osd_op_reply(26 [pgnls start_epoch 128 out=98b] v126'3 uv3 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1c000 2023-09-23T03:28:31.573 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.572+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:27 17.b 17:d0000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8cfd1c400 2023-09-23T03:28:31.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.572+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.0 v2:172.21.15.154:6800/2920938138 5 ==== osd_op_reply(27 [pgnls start_epoch 128 out=76b] v127'4 uv3 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8cfd1c400 2023-09-23T03:28:31.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.572+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:28 17.1b 17:d8000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8cfd1cc00 2023-09-23T03:28:31.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.573+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 3 ==== osd_op_reply(28 [pgnls start_epoch 128 out=81b] v128'55 uv55 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.573+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:29 17.7 17:e0000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8cfd1c400 2023-09-23T03:28:31.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.574+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.0 v2:172.21.15.154:6800/2920938138 6 ==== osd_op_reply(29 [pgnls start_epoch 128 out=73b] v125'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8cfd1c400 2023-09-23T03:28:31.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.574+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:30 17.17 17:e8000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8d0697000 2023-09-23T03:28:31.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.575+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 3 ==== osd_op_reply(30 [pgnls start_epoch 128 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.575+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:31 17.f 17:f0000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8cfd1c800 2023-09-23T03:28:31.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.576+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 <== osd.1 v2:172.21.15.154:6816/1267710005 3 ==== osd_op_reply(31 [pgnls start_epoch 128 out=73b] v125'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55e8d0766b40 con 0x55e8cfd1c800 2023-09-23T03:28:31.577 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.576+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:32 17.1f 17:f8000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8d0697000 2023-09-23T03:28:31.578 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.577+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 4 ==== osd_op_reply(32 [pgnls start_epoch 128 out=101b] v127'7 uv6 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.577+0000 7f50a463f5c0 10 datascan.scan_links: step 1: handling object 601.00000000 2023-09-23T03:28:31.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.577+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:33 17.10 17:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91fc00 con 0x55e8cfd1c000 2023-09-23T03:28:31.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.577+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.3 v2:172.21.15.154:6808/1133688468 7 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1c000 2023-09-23T03:28:31.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.577+0000 7f50a463f5c0 10 datascan.scan_links: step 1: handling object 602.00000000 2023-09-23T03:28:31.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.577+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:34 17.10 17:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48000 con 0x55e8cfd1c000 2023-09-23T03:28:31.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.578+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.3 v2:172.21.15.154:6808/1133688468 8 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv6 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1c000 2023-09-23T03:28:31.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.578+0000 7f50a463f5c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2023-09-23T03:28:31.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.578+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:35 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48380 con 0x55e8cfd1c000 2023-09-23T03:28:31.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.579+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.3 v2:172.21.15.154:6808/1133688468 9 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=50980b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+50980 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1c000 2023-09-23T03:28:31.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.579+0000 7f50a463f5c0 10 datascan.scan_links: step 1: handling object 600.00000000 2023-09-23T03:28:31.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.579+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:36 17.10 17:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48700 con 0x55e8cfd1c000 2023-09-23T03:28:31.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.580+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.3 v2:172.21.15.154:6808/1133688468 10 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1c000 2023-09-23T03:28:31.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.580+0000 7f50a463f5c0 10 datascan.scan_links: step 1: handling object 603.00000000 2023-09-23T03:28:31.581 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.580+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:37 17.10 17:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48a80 con 0x55e8cfd1c000 2023-09-23T03:28:31.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.580+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.3 v2:172.21.15.154:6808/1133688468 11 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv8 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1c000 2023-09-23T03:28:31.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.580+0000 7f50a463f5c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2023-09-23T03:28:31.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.580+0000 7f50a463f5c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-09-23T03:28:31.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.580+0000 7f50a463f5c0 10 datascan.scan_links: step 1: handling object 200.00000000 2023-09-23T03:28:31.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.580+0000 7f50a463f5c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-09-23T03:28:31.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.580+0000 7f50a463f5c0 10 datascan.scan_links: step 1: handling object 200.00000001 2023-09-23T03:28:31.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.580+0000 7f50a463f5c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-09-23T03:28:31.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.580+0000 7f50a463f5c0 10 datascan.scan_links: step 1: handling object 606.00000000 2023-09-23T03:28:31.582 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.580+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:38 17.14 17:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48000 con 0x55e8cfd1c000 2023-09-23T03:28:31.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.581+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.3 v2:172.21.15.154:6808/1133688468 12 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1c000 2023-09-23T03:28:31.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.581+0000 7f50a463f5c0 10 datascan.scan_links: step 1: handling object 607.00000000 2023-09-23T03:28:31.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.581+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:39 17.1c 17:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48380 con 0x55e8cfd1c800 2023-09-23T03:28:31.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.581+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 <== osd.1 v2:172.21.15.154:6816/1267710005 4 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55e8d0766b40 con 0x55e8cfd1c800 2023-09-23T03:28:31.583 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.581+0000 7f50a463f5c0 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2023-09-23T03:28:31.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.581+0000 7f50a463f5c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2023-09-23T03:28:31.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.581+0000 7f50a463f5c0 10 datascan.scan_links: step 1: handling object 608.00000000 2023-09-23T03:28:31.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.581+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:40 17.e 17:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48e00 con 0x55e8cfd1d000 2023-09-23T03:28:31.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.581+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.2 v2:172.21.15.154:6822/2884407243 2 ==== osd_op_reply(40 608.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1d000 2023-09-23T03:28:31.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.581+0000 7f50a463f5c0 10 datascan.scan_links: step 1: handling object 10000000000.01000000 2023-09-23T03:28:31.584 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.581+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:41 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd49180 con 0x55e8d0697000 2023-09-23T03:28:31.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 5 ==== osd_op_reply(41 10000000000.01000000 [omap-get-vals out=25448b] v0'0 uv56 ondisk = 0) v8 ==== 164+0+25448 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.585 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.587 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 10 datascan.scan_links: step 1: handling object 604.00000000 2023-09-23T03:28:31.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.584+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:42 17.15 17:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91fc00 con 0x55e8d0697400 2023-09-23T03:28:31.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.585+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 <== osd.5 v2:172.21.15.190:6800/882536885 4 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55e8d0766b40 con 0x55e8d0697400 2023-09-23T03:28:31.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.585+0000 7f50a463f5c0 10 datascan.scan_links: step 1: handling object 500.00000000 2023-09-23T03:28:31.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.585+0000 7f50a463f5c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-09-23T03:28:31.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.585+0000 7f50a463f5c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2023-09-23T03:28:31.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.585+0000 7f50a463f5c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-09-23T03:28:31.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.585+0000 7f50a463f5c0 10 datascan.scan_links: step 1: handling object 605.00000000 2023-09-23T03:28:31.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.585+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:43 17.1d 17:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91fc00 con 0x55e8cfc34c00 2023-09-23T03:28:31.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.586+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 <== osd.7 v2:172.21.15.190:6824/3460136272 7 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55e8d0766b40 con 0x55e8cfc34c00 2023-09-23T03:28:31.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.586+0000 7f50a463f5c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2023-09-23T03:28:31.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.586+0000 7f50a463f5c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-09-23T03:28:31.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.586+0000 7f50a463f5c0 10 datascan.scan_links: step 1: handling object 100.00000000 2023-09-23T03:28:31.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.586+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:44 17.13 17:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8cfd1c000 2023-09-23T03:28:31.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.587+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.3 v2:172.21.15.154:6808/1133688468 13 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1c000 2023-09-23T03:28:31.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.587+0000 7f50a463f5c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2023-09-23T03:28:31.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.587+0000 7f50a463f5c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-09-23T03:28:31.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.587+0000 7f50a463f5c0 10 datascan.scan_links: step 1: handling object 10000000000.01800000 2023-09-23T03:28:31.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.587+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:45 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48000 con 0x55e8cfd1cc00 2023-09-23T03:28:31.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 4 ==== osd_op_reply(45 10000000000.01800000 [omap-get-vals out=24452b] v0'0 uv55 ondisk = 0) v8 ==== 164+0+24452 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.594 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.595 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 10 datascan.scan_links: step 1: handling object 609.00000000 2023-09-23T03:28:31.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.589+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:46 17.7 17:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48380 con 0x55e8cfd1c400 2023-09-23T03:28:31.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.590+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.0 v2:172.21.15.154:6800/2920938138 7 ==== osd_op_reply(46 609.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8cfd1c400 2023-09-23T03:28:31.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.590+0000 7f50a463f5c0 10 datascan.scan_links: step 1: handling object 400.00000000 2023-09-23T03:28:31.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.590+0000 7f50a463f5c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-09-23T03:28:31.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.590+0000 7f50a463f5c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2023-09-23T03:28:31.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.590+0000 7f50a463f5c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-09-23T03:28:31.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.590+0000 7f50a463f5c0 10 datascan.scan_links: step 1: handling object 1.00000000 2023-09-23T03:28:31.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.590+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:47 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48700 con 0x55e8d0697000 2023-09-23T03:28:31.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.591+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 6 ==== osd_op_reply(47 1.00000000 [omap-get-vals out=532b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+532 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.591+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:48 17.0 17:00000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48a80 con 0x55e8cfc34c00 2023-09-23T03:28:31.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.592+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 <== osd.7 v2:172.21.15.190:6824/3460136272 8 ==== osd_op_reply(48 [pgnls start_epoch 128 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d0766b40 con 0x55e8cfc34c00 2023-09-23T03:28:31.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.592+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:49 17.10 17:08000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48a80 con 0x55e8cfd1c000 2023-09-23T03:28:31.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.593+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.3 v2:172.21.15.154:6808/1133688468 14 ==== osd_op_reply(49 [pgnls start_epoch 128 out=177b] v127'12 uv12 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1c000 2023-09-23T03:28:31.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.593+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:50 17.8 17:10000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48a80 con 0x55e8cfc34c00 2023-09-23T03:28:31.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.594+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 <== osd.7 v2:172.21.15.190:6824/3460136272 9 ==== osd_op_reply(50 [pgnls start_epoch 128 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d0766b40 con 0x55e8cfc34c00 2023-09-23T03:28:31.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.594+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:51 17.18 17:18000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48a80 con 0x55e8cfd1c400 2023-09-23T03:28:31.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.594+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.0 v2:172.21.15.154:6800/2920938138 8 ==== osd_op_reply(51 [pgnls start_epoch 128 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8cfd1c400 2023-09-23T03:28:31.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.594+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:52 17.4 17:20000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48380 con 0x55e8cfd1c400 2023-09-23T03:28:31.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.594+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.0 v2:172.21.15.154:6800/2920938138 9 ==== osd_op_reply(52 [pgnls start_epoch 128 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8cfd1c400 2023-09-23T03:28:31.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.594+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:53 17.14 17:28000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48380 con 0x55e8cfd1c000 2023-09-23T03:28:31.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.595+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.3 v2:172.21.15.154:6808/1133688468 15 ==== osd_op_reply(53 [pgnls start_epoch 128 out=149b] v127'28 uv28 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1c000 2023-09-23T03:28:31.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.595+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:54 17.c 17:30000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48380 con 0x55e8cfc34c00 2023-09-23T03:28:31.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.596+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 <== osd.7 v2:172.21.15.190:6824/3460136272 10 ==== osd_op_reply(54 [pgnls start_epoch 128 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d0766b40 con 0x55e8cfc34c00 2023-09-23T03:28:31.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.596+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:55 17.1c 17:38000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48380 con 0x55e8cfd1c800 2023-09-23T03:28:31.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.596+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 <== osd.1 v2:172.21.15.154:6816/1267710005 5 ==== osd_op_reply(55 [pgnls start_epoch 128 out=73b] v125'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55e8d0766b40 con 0x55e8cfd1c800 2023-09-23T03:28:31.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.597+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:56 17.2 17:40000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48380 con 0x55e8cfd1cc00 2023-09-23T03:28:31.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.597+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 5 ==== osd_op_reply(56 [pgnls start_epoch 128 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.597+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:57 17.12 17:48000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48380 con 0x55e8cfd1c000 2023-09-23T03:28:31.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.598+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.3 v2:172.21.15.154:6808/1133688468 16 ==== osd_op_reply(57 [pgnls start_epoch 128 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1c000 2023-09-23T03:28:31.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.598+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:58 17.a 17:50000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48380 con 0x55e8cfc34c00 2023-09-23T03:28:31.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.599+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 <== osd.7 v2:172.21.15.190:6824/3460136272 11 ==== osd_op_reply(58 [pgnls start_epoch 128 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d0766b40 con 0x55e8cfc34c00 2023-09-23T03:28:31.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.599+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:59 17.1a 17:58000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48380 con 0x55e8cfd1c400 2023-09-23T03:28:31.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.600+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.0 v2:172.21.15.154:6800/2920938138 10 ==== osd_op_reply(59 [pgnls start_epoch 128 out=49b] v127'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8cfd1c400 2023-09-23T03:28:31.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.600+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:60 17.6 17:60000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48380 con 0x55e8d0697000 2023-09-23T03:28:31.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.601+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 7 ==== osd_op_reply(60 [pgnls start_epoch 128 out=77b] v127'7 uv7 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.601+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:61 17.16 17:68000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48380 con 0x55e8d0697400 2023-09-23T03:28:31.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.602+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 <== osd.5 v2:172.21.15.190:6800/882536885 5 ==== osd_op_reply(61 [pgnls start_epoch 128 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d0766b40 con 0x55e8d0697400 2023-09-23T03:28:31.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.602+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:62 17.e 17:70000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48380 con 0x55e8cfd1d000 2023-09-23T03:28:31.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.602+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.2 v2:172.21.15.154:6822/2884407243 3 ==== osd_op_reply(62 [pgnls start_epoch 128 out=73b] v125'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1d000 2023-09-23T03:28:31.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.602+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:63 17.1e 17:78000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48380 con 0x55e8cfd1c800 2023-09-23T03:28:31.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.603+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 <== osd.1 v2:172.21.15.154:6816/1267710005 6 ==== osd_op_reply(63 [pgnls start_epoch 128 out=49b] v127'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d0766b40 con 0x55e8cfd1c800 2023-09-23T03:28:31.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.603+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:64 17.1 17:80000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48380 con 0x55e8cfd1c400 2023-09-23T03:28:31.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.604+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.0 v2:172.21.15.154:6800/2920938138 11 ==== osd_op_reply(64 [pgnls start_epoch 128 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8cfd1c400 2023-09-23T03:28:31.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.604+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:65 17.11 17:88000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48380 con 0x55e8cfd1c000 2023-09-23T03:28:31.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.604+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.3 v2:172.21.15.154:6808/1133688468 17 ==== osd_op_reply(65 [pgnls start_epoch 128 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1c000 2023-09-23T03:28:31.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.604+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:66 17.9 17:90000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48380 con 0x55e8cfd1c000 2023-09-23T03:28:31.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.605+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.3 v2:172.21.15.154:6808/1133688468 18 ==== osd_op_reply(66 [pgnls start_epoch 128 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1c000 2023-09-23T03:28:31.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.605+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:67 17.19 17:98000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48380 con 0x55e8d0697400 2023-09-23T03:28:31.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.606+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 <== osd.5 v2:172.21.15.190:6800/882536885 6 ==== osd_op_reply(67 [pgnls start_epoch 128 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d0766b40 con 0x55e8d0697400 2023-09-23T03:28:31.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.606+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:68 17.5 17:a0000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48380 con 0x55e8d0697000 2023-09-23T03:28:31.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.607+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 8 ==== osd_op_reply(68 [pgnls start_epoch 128 out=81b] v128'56 uv56 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.607+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:69 17.15 17:a8000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48380 con 0x55e8d0697400 2023-09-23T03:28:31.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.609+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 <== osd.5 v2:172.21.15.190:6800/882536885 7 ==== osd_op_reply(69 [pgnls start_epoch 128 out=97b] v125'3 uv3 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x55e8d0766b40 con 0x55e8d0697400 2023-09-23T03:28:31.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.609+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:70 17.d 17:b0000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48380 con 0x55e8cfc34c00 2023-09-23T03:28:31.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.610+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 <== osd.7 v2:172.21.15.190:6824/3460136272 12 ==== osd_op_reply(70 [pgnls start_epoch 128 out=74b] v125'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55e8d0766b40 con 0x55e8cfc34c00 2023-09-23T03:28:31.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.610+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:71 17.1d 17:b8000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48380 con 0x55e8cfc34c00 2023-09-23T03:28:31.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.611+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 <== osd.7 v2:172.21.15.190:6824/3460136272 13 ==== osd_op_reply(71 [pgnls start_epoch 128 out=73b] v125'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55e8d0766b40 con 0x55e8cfc34c00 2023-09-23T03:28:31.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.611+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:72 17.3 17:c0000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48380 con 0x55e8cfd1cc00 2023-09-23T03:28:31.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.612+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 6 ==== osd_op_reply(72 [pgnls start_epoch 128 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.612+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:73 17.13 17:c8000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48380 con 0x55e8cfd1c000 2023-09-23T03:28:31.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.613+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.3 v2:172.21.15.154:6808/1133688468 19 ==== osd_op_reply(73 [pgnls start_epoch 128 out=98b] v126'3 uv3 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1c000 2023-09-23T03:28:31.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.613+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:74 17.b 17:d0000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48380 con 0x55e8cfd1c400 2023-09-23T03:28:31.615 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.613+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.0 v2:172.21.15.154:6800/2920938138 12 ==== osd_op_reply(74 [pgnls start_epoch 128 out=76b] v127'4 uv3 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8cfd1c400 2023-09-23T03:28:31.615 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.614+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:75 17.1b 17:d8000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48380 con 0x55e8cfd1cc00 2023-09-23T03:28:31.615 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.614+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 7 ==== osd_op_reply(75 [pgnls start_epoch 128 out=81b] v128'55 uv55 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.614+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:76 17.7 17:e0000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48380 con 0x55e8cfd1c400 2023-09-23T03:28:31.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.615+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.0 v2:172.21.15.154:6800/2920938138 13 ==== osd_op_reply(76 [pgnls start_epoch 128 out=73b] v125'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8cfd1c400 2023-09-23T03:28:31.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.615+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:77 17.17 17:e8000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48380 con 0x55e8d0697000 2023-09-23T03:28:31.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.616+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 9 ==== osd_op_reply(77 [pgnls start_epoch 128 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.616+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:78 17.f 17:f0000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48380 con 0x55e8cfd1c800 2023-09-23T03:28:31.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.617+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 <== osd.1 v2:172.21.15.154:6816/1267710005 7 ==== osd_op_reply(78 [pgnls start_epoch 128 out=73b] v125'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55e8d0766b40 con 0x55e8cfd1c800 2023-09-23T03:28:31.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.617+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:79 17.1f 17:f8000000::::head [pgnls start_epoch 128 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48380 con 0x55e8d0697000 2023-09-23T03:28:31.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.618+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 10 ==== osd_op_reply(79 [pgnls start_epoch 128 out=101b] v127'7 uv6 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.618+0000 7f50a463f5c0 10 datascan.scan_links: step 2: handling object 601.00000000 2023-09-23T03:28:31.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.618+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:80 17.10 17:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48e00 con 0x55e8cfd1c000 2023-09-23T03:28:31.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.619+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.3 v2:172.21.15.154:6808/1133688468 20 ==== osd_op_reply(80 601.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1c000 2023-09-23T03:28:31.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.619+0000 7f50a463f5c0 10 datascan.scan_links: step 2: handling object 602.00000000 2023-09-23T03:28:31.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.619+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:81 17.10 17:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd49180 con 0x55e8cfd1c000 2023-09-23T03:28:31.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.620+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.3 v2:172.21.15.154:6808/1133688468 21 ==== osd_op_reply(81 602.00000000 [omap-get-vals out=5b] v0'0 uv6 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1c000 2023-09-23T03:28:31.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.620+0000 7f50a463f5c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2023-09-23T03:28:31.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.620+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:82 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd49500 con 0x55e8cfd1c000 2023-09-23T03:28:31.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.622+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.3 v2:172.21.15.154:6808/1133688468 22 ==== osd_op_reply(82 10000000000.00000000 [omap-get-vals out=50980b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+50980 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1c000 2023-09-23T03:28:31.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.622+0000 7f50a463f5c0 10 datascan.scan_links: step 2: handling object 600.00000000 2023-09-23T03:28:31.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.622+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:83 17.10 17:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48000 con 0x55e8cfd1c000 2023-09-23T03:28:31.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.623+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.3 v2:172.21.15.154:6808/1133688468 23 ==== osd_op_reply(83 600.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1c000 2023-09-23T03:28:31.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.623+0000 7f50a463f5c0 10 datascan.scan_links: step 2: handling object 603.00000000 2023-09-23T03:28:31.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.623+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:84 17.10 17:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48e00 con 0x55e8cfd1c000 2023-09-23T03:28:31.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.624+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.3 v2:172.21.15.154:6808/1133688468 24 ==== osd_op_reply(84 603.00000000 [omap-get-vals out=5b] v0'0 uv8 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1c000 2023-09-23T03:28:31.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.624+0000 7f50a463f5c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2023-09-23T03:28:31.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.624+0000 7f50a463f5c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-09-23T03:28:31.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.624+0000 7f50a463f5c0 10 datascan.scan_links: step 2: handling object 200.00000000 2023-09-23T03:28:31.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.624+0000 7f50a463f5c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-09-23T03:28:31.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.624+0000 7f50a463f5c0 10 datascan.scan_links: step 2: handling object 200.00000001 2023-09-23T03:28:31.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.624+0000 7f50a463f5c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-09-23T03:28:31.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.624+0000 7f50a463f5c0 10 datascan.scan_links: step 2: handling object 606.00000000 2023-09-23T03:28:31.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.624+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:85 17.14 17:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd49180 con 0x55e8cfd1c000 2023-09-23T03:28:31.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.624+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.3 v2:172.21.15.154:6808/1133688468 25 ==== osd_op_reply(85 606.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1c000 2023-09-23T03:28:31.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.624+0000 7f50a463f5c0 10 datascan.scan_links: step 2: handling object 607.00000000 2023-09-23T03:28:31.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.625+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- osd_op(unknown.0.0:86 17.1c 17:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd49500 con 0x55e8cfd1c800 2023-09-23T03:28:31.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.625+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 <== osd.1 v2:172.21.15.154:6816/1267710005 8 ==== osd_op_reply(86 607.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55e8d0766b40 con 0x55e8cfd1c800 2023-09-23T03:28:31.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.625+0000 7f50a463f5c0 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2023-09-23T03:28:31.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.625+0000 7f50a463f5c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2023-09-23T03:28:31.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.625+0000 7f50a463f5c0 10 datascan.scan_links: step 2: handling object 608.00000000 2023-09-23T03:28:31.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.625+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- osd_op(unknown.0.0:87 17.e 17:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd49880 con 0x55e8cfd1d000 2023-09-23T03:28:31.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.626+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.2 v2:172.21.15.154:6822/2884407243 4 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1d000 2023-09-23T03:28:31.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.626+0000 7f50a463f5c0 10 datascan.scan_links: step 2: handling object 10000000000.01000000 2023-09-23T03:28:31.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.626+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:88 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd49c00 con 0x55e8d0697000 2023-09-23T03:28:31.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 11 ==== osd_op_reply(88 10000000000.01000000 [omap-get-vals out=25448b] v0'0 uv56 ondisk = 0) v8 ==== 164+0+25448 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x100000001ff for future processing to fix dnfirst 2023-09-23T03:28:31.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000200 for future processing to fix dnfirst 2023-09-23T03:28:31.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000201 for future processing to fix dnfirst 2023-09-23T03:28:31.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000204 for future processing to fix dnfirst 2023-09-23T03:28:31.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000208 for future processing to fix dnfirst 2023-09-23T03:28:31.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x100000001f6 for future processing to fix dnfirst 2023-09-23T03:28:31.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000209 for future processing to fix dnfirst 2023-09-23T03:28:31.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x1000000020a for future processing to fix dnfirst 2023-09-23T03:28:31.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000210 for future processing to fix dnfirst 2023-09-23T03:28:31.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000211 for future processing to fix dnfirst 2023-09-23T03:28:31.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2023-09-23T03:28:31.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000214 for future processing to fix dnfirst 2023-09-23T03:28:31.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000217 for future processing to fix dnfirst 2023-09-23T03:28:31.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000219 for future processing to fix dnfirst 2023-09-23T03:28:31.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x1000000021b for future processing to fix dnfirst 2023-09-23T03:28:31.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x1000000021c for future processing to fix dnfirst 2023-09-23T03:28:31.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x100000001f8 for future processing to fix dnfirst 2023-09-23T03:28:31.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x1000000021f for future processing to fix dnfirst 2023-09-23T03:28:31.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000220 for future processing to fix dnfirst 2023-09-23T03:28:31.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000223 for future processing to fix dnfirst 2023-09-23T03:28:31.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x100000001f9 for future processing to fix dnfirst 2023-09-23T03:28:31.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000227 for future processing to fix dnfirst 2023-09-23T03:28:31.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000229 for future processing to fix dnfirst 2023-09-23T03:28:31.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x1000000022a for future processing to fix dnfirst 2023-09-23T03:28:31.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x1000000022c for future processing to fix dnfirst 2023-09-23T03:28:31.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000230 for future processing to fix dnfirst 2023-09-23T03:28:31.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000232 for future processing to fix dnfirst 2023-09-23T03:28:31.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000234 for future processing to fix dnfirst 2023-09-23T03:28:31.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000235 for future processing to fix dnfirst 2023-09-23T03:28:31.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000236 for future processing to fix dnfirst 2023-09-23T03:28:31.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000237 for future processing to fix dnfirst 2023-09-23T03:28:31.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x1000000023a for future processing to fix dnfirst 2023-09-23T03:28:31.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x1000000023b for future processing to fix dnfirst 2023-09-23T03:28:31.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x1000000023e for future processing to fix dnfirst 2023-09-23T03:28:31.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x1000000023f for future processing to fix dnfirst 2023-09-23T03:28:31.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000240 for future processing to fix dnfirst 2023-09-23T03:28:31.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.628+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000242 for future processing to fix dnfirst 2023-09-23T03:28:31.639 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.629+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.629+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000245 for future processing to fix dnfirst 2023-09-23T03:28:31.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.629+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.629+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000248 for future processing to fix dnfirst 2023-09-23T03:28:31.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.629+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.629+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000249 for future processing to fix dnfirst 2023-09-23T03:28:31.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.629+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.629+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x1000000024a for future processing to fix dnfirst 2023-09-23T03:28:31.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.629+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.629+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x1000000024b for future processing to fix dnfirst 2023-09-23T03:28:31.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.629+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.629+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x1000000024c for future processing to fix dnfirst 2023-09-23T03:28:31.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.629+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.629+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x1000000024d for future processing to fix dnfirst 2023-09-23T03:28:31.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.629+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.629+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x100000001fd for future processing to fix dnfirst 2023-09-23T03:28:31.641 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.629+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.629+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x1000000024f for future processing to fix dnfirst 2023-09-23T03:28:31.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.629+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.629+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000251 for future processing to fix dnfirst 2023-09-23T03:28:31.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.629+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.629+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000254 for future processing to fix dnfirst 2023-09-23T03:28:31.642 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.629+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.629+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000255 for future processing to fix dnfirst 2023-09-23T03:28:31.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.629+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.629+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000256 for future processing to fix dnfirst 2023-09-23T03:28:31.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.629+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.629+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x100000001fe for future processing to fix dnfirst 2023-09-23T03:28:31.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.629+0000 7f50a463f5c0 10 datascan.scan_links: step 2: handling object 604.00000000 2023-09-23T03:28:31.643 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.629+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- osd_op(unknown.0.0:89 17.15 17:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8d0697400 2023-09-23T03:28:31.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.630+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 <== osd.5 v2:172.21.15.190:6800/882536885 8 ==== osd_op_reply(89 604.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55e8d0766b40 con 0x55e8d0697400 2023-09-23T03:28:31.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.630+0000 7f50a463f5c0 10 datascan.scan_links: step 2: handling object 500.00000000 2023-09-23T03:28:31.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.630+0000 7f50a463f5c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-09-23T03:28:31.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.630+0000 7f50a463f5c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2023-09-23T03:28:31.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.630+0000 7f50a463f5c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-09-23T03:28:31.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.630+0000 7f50a463f5c0 10 datascan.scan_links: step 2: handling object 605.00000000 2023-09-23T03:28:31.644 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.630+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:90 17.1d 17:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91fc00 con 0x55e8cfc34c00 2023-09-23T03:28:31.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.631+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 <== osd.7 v2:172.21.15.190:6824/3460136272 14 ==== osd_op_reply(90 605.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55e8d0766b40 con 0x55e8cfc34c00 2023-09-23T03:28:31.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.631+0000 7f50a463f5c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2023-09-23T03:28:31.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.631+0000 7f50a463f5c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-09-23T03:28:31.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.631+0000 7f50a463f5c0 10 datascan.scan_links: step 2: handling object 100.00000000 2023-09-23T03:28:31.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.631+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:91 17.13 17:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8cfd1c000 2023-09-23T03:28:31.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.632+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.3 v2:172.21.15.154:6808/1133688468 26 ==== osd_op_reply(91 100.00000000 [omap-get-vals out=5195b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1c000 2023-09-23T03:28:31.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.632+0000 7f50a463f5c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2023-09-23T03:28:31.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.632+0000 7f50a463f5c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-09-23T03:28:31.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.632+0000 7f50a463f5c0 10 datascan.scan_links: step 2: handling object 10000000000.01800000 2023-09-23T03:28:31.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.632+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:92 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd49500 con 0x55e8cfd1cc00 2023-09-23T03:28:31.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.634+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 8 ==== osd_op_reply(92 10000000000.01800000 [omap-get-vals out=24452b] v0'0 uv55 ondisk = 0) v8 ==== 164+0+24452 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.634+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.634+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2023-09-23T03:28:31.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.634+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000202 for future processing to fix dnfirst 2023-09-23T03:28:31.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000203 for future processing to fix dnfirst 2023-09-23T03:28:31.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000205 for future processing to fix dnfirst 2023-09-23T03:28:31.652 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000206 for future processing to fix dnfirst 2023-09-23T03:28:31.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000207 for future processing to fix dnfirst 2023-09-23T03:28:31.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x1000000020b for future processing to fix dnfirst 2023-09-23T03:28:31.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.653 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x1000000020c for future processing to fix dnfirst 2023-09-23T03:28:31.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x1000000020d for future processing to fix dnfirst 2023-09-23T03:28:31.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x1000000020e for future processing to fix dnfirst 2023-09-23T03:28:31.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x1000000020f for future processing to fix dnfirst 2023-09-23T03:28:31.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000212 for future processing to fix dnfirst 2023-09-23T03:28:31.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.654 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000213 for future processing to fix dnfirst 2023-09-23T03:28:31.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000215 for future processing to fix dnfirst 2023-09-23T03:28:31.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000216 for future processing to fix dnfirst 2023-09-23T03:28:31.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000218 for future processing to fix dnfirst 2023-09-23T03:28:31.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x1000000021a for future processing to fix dnfirst 2023-09-23T03:28:31.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x1000000021d for future processing to fix dnfirst 2023-09-23T03:28:31.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x1000000021e for future processing to fix dnfirst 2023-09-23T03:28:31.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000221 for future processing to fix dnfirst 2023-09-23T03:28:31.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000222 for future processing to fix dnfirst 2023-09-23T03:28:31.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000224 for future processing to fix dnfirst 2023-09-23T03:28:31.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000225 for future processing to fix dnfirst 2023-09-23T03:28:31.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000226 for future processing to fix dnfirst 2023-09-23T03:28:31.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000228 for future processing to fix dnfirst 2023-09-23T03:28:31.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x1000000022b for future processing to fix dnfirst 2023-09-23T03:28:31.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x1000000022d for future processing to fix dnfirst 2023-09-23T03:28:31.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x1000000022e for future processing to fix dnfirst 2023-09-23T03:28:31.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x1000000022f for future processing to fix dnfirst 2023-09-23T03:28:31.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x100000001fa for future processing to fix dnfirst 2023-09-23T03:28:31.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000231 for future processing to fix dnfirst 2023-09-23T03:28:31.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000233 for future processing to fix dnfirst 2023-09-23T03:28:31.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000238 for future processing to fix dnfirst 2023-09-23T03:28:31.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000239 for future processing to fix dnfirst 2023-09-23T03:28:31.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x100000001fb for future processing to fix dnfirst 2023-09-23T03:28:31.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x1000000023c for future processing to fix dnfirst 2023-09-23T03:28:31.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x1000000023d for future processing to fix dnfirst 2023-09-23T03:28:31.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000241 for future processing to fix dnfirst 2023-09-23T03:28:31.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000243 for future processing to fix dnfirst 2023-09-23T03:28:31.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000244 for future processing to fix dnfirst 2023-09-23T03:28:31.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x100000001fc for future processing to fix dnfirst 2023-09-23T03:28:31.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000246 for future processing to fix dnfirst 2023-09-23T03:28:31.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000247 for future processing to fix dnfirst 2023-09-23T03:28:31.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x1000000024e for future processing to fix dnfirst 2023-09-23T03:28:31.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000250 for future processing to fix dnfirst 2023-09-23T03:28:31.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000252 for future processing to fix dnfirst 2023-09-23T03:28:31.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000253 for future processing to fix dnfirst 2023-09-23T03:28:31.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000257 for future processing to fix dnfirst 2023-09-23T03:28:31.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: injected ino detected 2023-09-23T03:28:31.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 20 datascan.scan_links: adding 0x10000000258 for future processing to fix dnfirst 2023-09-23T03:28:31.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 10 datascan.scan_links: step 2: handling object 609.00000000 2023-09-23T03:28:31.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.635+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- osd_op(unknown.0.0:93 17.7 17:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48000 con 0x55e8cfd1c400 2023-09-23T03:28:31.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.636+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.0 v2:172.21.15.154:6800/2920938138 14 ==== osd_op_reply(93 609.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8cfd1c400 2023-09-23T03:28:31.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.636+0000 7f50a463f5c0 10 datascan.scan_links: step 2: handling object 400.00000000 2023-09-23T03:28:31.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.636+0000 7f50a463f5c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-09-23T03:28:31.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.636+0000 7f50a463f5c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2023-09-23T03:28:31.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.636+0000 7f50a463f5c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-09-23T03:28:31.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.636+0000 7f50a463f5c0 10 datascan.scan_links: step 2: handling object 1.00000000 2023-09-23T03:28:31.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.636+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:94 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48e00 con 0x55e8d0697000 2023-09-23T03:28:31.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 12 ==== osd_op_reply(94 1.00000000 [omap-get-vals out=532b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+532 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: processing 99 dup_primaries, 0 remote_links 2023-09-23T03:28:31.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000001 2023-09-23T03:28:31.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x100000001f6 2023-09-23T03:28:31.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x100000001f7 2023-09-23T03:28:31.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x100000001f8 2023-09-23T03:28:31.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x100000001f9 2023-09-23T03:28:31.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x100000001fa 2023-09-23T03:28:31.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x100000001fb 2023-09-23T03:28:31.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x100000001fc 2023-09-23T03:28:31.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x100000001fd 2023-09-23T03:28:31.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x100000001fe 2023-09-23T03:28:31.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x100000001ff 2023-09-23T03:28:31.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000200 2023-09-23T03:28:31.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000201 2023-09-23T03:28:31.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000202 2023-09-23T03:28:31.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000203 2023-09-23T03:28:31.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000204 2023-09-23T03:28:31.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000205 2023-09-23T03:28:31.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000206 2023-09-23T03:28:31.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000207 2023-09-23T03:28:31.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000208 2023-09-23T03:28:31.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000209 2023-09-23T03:28:31.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x1000000020b 2023-09-23T03:28:31.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x1000000020c 2023-09-23T03:28:31.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x1000000020d 2023-09-23T03:28:31.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x1000000020e 2023-09-23T03:28:31.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x1000000020f 2023-09-23T03:28:31.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000210 2023-09-23T03:28:31.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000211 2023-09-23T03:28:31.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000212 2023-09-23T03:28:31.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000213 2023-09-23T03:28:31.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000214 2023-09-23T03:28:31.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000215 2023-09-23T03:28:31.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000216 2023-09-23T03:28:31.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000217 2023-09-23T03:28:31.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000218 2023-09-23T03:28:31.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000219 2023-09-23T03:28:31.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x1000000021a 2023-09-23T03:28:31.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x1000000021b 2023-09-23T03:28:31.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x1000000021c 2023-09-23T03:28:31.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x1000000021d 2023-09-23T03:28:31.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.637+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x1000000021e 2023-09-23T03:28:31.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x1000000021f 2023-09-23T03:28:31.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000220 2023-09-23T03:28:31.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000221 2023-09-23T03:28:31.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000222 2023-09-23T03:28:31.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000223 2023-09-23T03:28:31.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000224 2023-09-23T03:28:31.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000225 2023-09-23T03:28:31.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000226 2023-09-23T03:28:31.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000227 2023-09-23T03:28:31.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000228 2023-09-23T03:28:31.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000229 2023-09-23T03:28:31.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x1000000022a 2023-09-23T03:28:31.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x1000000022b 2023-09-23T03:28:31.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x1000000022c 2023-09-23T03:28:31.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x1000000022d 2023-09-23T03:28:31.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x1000000022e 2023-09-23T03:28:31.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x1000000022f 2023-09-23T03:28:31.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000230 2023-09-23T03:28:31.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000231 2023-09-23T03:28:31.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000232 2023-09-23T03:28:31.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000233 2023-09-23T03:28:31.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000234 2023-09-23T03:28:31.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000235 2023-09-23T03:28:31.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000236 2023-09-23T03:28:31.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000237 2023-09-23T03:28:31.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000238 2023-09-23T03:28:31.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000239 2023-09-23T03:28:31.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x1000000023a 2023-09-23T03:28:31.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x1000000023b 2023-09-23T03:28:31.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x1000000023c 2023-09-23T03:28:31.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x1000000023d 2023-09-23T03:28:31.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x1000000023e 2023-09-23T03:28:31.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x1000000023f 2023-09-23T03:28:31.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000240 2023-09-23T03:28:31.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000241 2023-09-23T03:28:31.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000242 2023-09-23T03:28:31.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000243 2023-09-23T03:28:31.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000244 2023-09-23T03:28:31.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000245 2023-09-23T03:28:31.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000246 2023-09-23T03:28:31.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000247 2023-09-23T03:28:31.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000248 2023-09-23T03:28:31.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000249 2023-09-23T03:28:31.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x1000000024a 2023-09-23T03:28:31.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x1000000024b 2023-09-23T03:28:31.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x1000000024c 2023-09-23T03:28:31.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x1000000024d 2023-09-23T03:28:31.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x1000000024e 2023-09-23T03:28:31.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x1000000024f 2023-09-23T03:28:31.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000250 2023-09-23T03:28:31.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000251 2023-09-23T03:28:31.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000252 2023-09-23T03:28:31.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000253 2023-09-23T03:28:31.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000254 2023-09-23T03:28:31.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000255 2023-09-23T03:28:31.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000256 2023-09-23T03:28:31.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000257 2023-09-23T03:28:31.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling dup 0x10000000258 2023-09-23T03:28:31.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: removing dup dentries from 0 objects 2023-09-23T03:28:31.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: processing 0 bad_nlink_inos 2023-09-23T03:28:31.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: processing 100 injected_inos 2023-09-23T03:28:31.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000001 2023-09-23T03:28:31.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:0 2023-09-23T03:28:31.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.638+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:95 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd49180 con 0x55e8cfd1cc00 2023-09-23T03:28:31.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.639+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 9 ==== osd_op_reply(95 10000000000.01800000 [omap-get-vals-by-keys out=502b] v0'0 uv55 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.639+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=0 frag=1*, r=0 2023-09-23T03:28:31.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.639+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.640+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:96 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd49880 con 0x55e8cfd1cc00 2023-09-23T03:28:31.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.643+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 10 ==== osd_op_reply(96 10000000000.01800000 [omap-set-vals] v128'56 uv56 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.643+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/0 pointing to 0x0x10000000001 2023-09-23T03:28:31.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.643+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x100000001f6 2023-09-23T03:28:31.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.643+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:1 2023-09-23T03:28:31.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.643+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:97 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd49500 con 0x55e8d0697000 2023-09-23T03:28:31.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.645+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 13 ==== osd_op_reply(97 10000000000.01000000 [omap-get-vals-by-keys out=502b] v0'0 uv56 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.645+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=1 frag=0*, r=0 2023-09-23T03:28:31.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.645+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.645+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:98 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8d0697000 2023-09-23T03:28:31.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.649+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 14 ==== osd_op_reply(98 10000000000.01000000 [omap-set-vals] v128'57 uv57 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.649+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/1 pointing to 0x0x100000001f6 2023-09-23T03:28:31.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.649+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x100000001f7 2023-09-23T03:28:31.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.649+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:2 2023-09-23T03:28:31.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.649+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:99 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48a80 con 0x55e8d0697000 2023-09-23T03:28:31.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.650+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 15 ==== osd_op_reply(99 10000000000.01000000 [omap-get-vals-by-keys out=502b] v0'0 uv57 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.680 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.650+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=2 frag=0*, r=0 2023-09-23T03:28:31.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.650+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.650+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:100 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48700 con 0x55e8d0697000 2023-09-23T03:28:31.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.654+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 16 ==== osd_op_reply(100 10000000000.01000000 [omap-set-vals] v128'58 uv58 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.654+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/2 pointing to 0x0x100000001f7 2023-09-23T03:28:31.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.654+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x100000001f8 2023-09-23T03:28:31.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.654+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:3 2023-09-23T03:28:31.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.654+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:101 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48380 con 0x55e8d0697000 2023-09-23T03:28:31.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.655+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 17 ==== osd_op_reply(101 10000000000.01000000 [omap-get-vals-by-keys out=502b] v0'0 uv58 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.655+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=3 frag=0*, r=0 2023-09-23T03:28:31.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.655+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.655+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:102 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd49c00 con 0x55e8d0697000 2023-09-23T03:28:31.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.660+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 18 ==== osd_op_reply(102 10000000000.01000000 [omap-set-vals] v128'59 uv59 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.660+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/3 pointing to 0x0x100000001f8 2023-09-23T03:28:31.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.660+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x100000001f9 2023-09-23T03:28:31.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.660+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:4 2023-09-23T03:28:31.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.660+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:103 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48000 con 0x55e8d0697000 2023-09-23T03:28:31.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.661+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 19 ==== osd_op_reply(103 10000000000.01000000 [omap-get-vals-by-keys out=502b] v0'0 uv59 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.661+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=4 frag=0*, r=0 2023-09-23T03:28:31.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.661+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.661+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:104 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48e00 con 0x55e8d0697000 2023-09-23T03:28:31.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.665+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 20 ==== osd_op_reply(104 10000000000.01000000 [omap-set-vals] v128'60 uv60 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.665+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/4 pointing to 0x0x100000001f9 2023-09-23T03:28:31.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.665+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x100000001fa 2023-09-23T03:28:31.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.665+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:5 2023-09-23T03:28:31.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.665+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:105 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd49500 con 0x55e8cfd1cc00 2023-09-23T03:28:31.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.666+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 11 ==== osd_op_reply(105 10000000000.01800000 [omap-get-vals-by-keys out=502b] v0'0 uv56 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.666+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=5 frag=1*, r=0 2023-09-23T03:28:31.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.666+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.666+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:106 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48a80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.670+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 12 ==== osd_op_reply(106 10000000000.01800000 [omap-set-vals] v128'57 uv57 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.670+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/5 pointing to 0x0x100000001fa 2023-09-23T03:28:31.683 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.670+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x100000001fb 2023-09-23T03:28:31.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.670+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:6 2023-09-23T03:28:31.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.670+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:107 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48700 con 0x55e8cfd1cc00 2023-09-23T03:28:31.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.671+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 13 ==== osd_op_reply(107 10000000000.01800000 [omap-get-vals-by-keys out=502b] v0'0 uv57 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.671+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=6 frag=1*, r=0 2023-09-23T03:28:31.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.671+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.671+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:108 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48380 con 0x55e8cfd1cc00 2023-09-23T03:28:31.684 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.675+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 14 ==== osd_op_reply(108 10000000000.01800000 [omap-set-vals] v128'58 uv58 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.675+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/6 pointing to 0x0x100000001fb 2023-09-23T03:28:31.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.675+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x100000001fc 2023-09-23T03:28:31.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.675+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:7 2023-09-23T03:28:31.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.675+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:109 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8cfd1cc00 2023-09-23T03:28:31.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.676+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 15 ==== osd_op_reply(109 10000000000.01800000 [omap-get-vals-by-keys out=502b] v0'0 uv58 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.676+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=7 frag=1*, r=0 2023-09-23T03:28:31.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.676+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.685 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.676+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:110 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8cfd1cc00 2023-09-23T03:28:31.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.681+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 16 ==== osd_op_reply(110 10000000000.01800000 [omap-set-vals] v128'59 uv59 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.686 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.681+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/7 pointing to 0x0x100000001fc 2023-09-23T03:28:31.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.681+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x100000001fd 2023-09-23T03:28:31.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.681+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:8 2023-09-23T03:28:31.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.681+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:111 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd49180 con 0x55e8d0697000 2023-09-23T03:28:31.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.682+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 21 ==== osd_op_reply(111 10000000000.01000000 [omap-get-vals-by-keys out=502b] v0'0 uv60 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.682+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=8 frag=0*, r=0 2023-09-23T03:28:31.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.682+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.687 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.682+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:112 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd49880 con 0x55e8d0697000 2023-09-23T03:28:31.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.686+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 22 ==== osd_op_reply(112 10000000000.01000000 [omap-set-vals] v128'61 uv61 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.686+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/8 pointing to 0x0x100000001fd 2023-09-23T03:28:31.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.686+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x100000001fe 2023-09-23T03:28:31.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.686+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:9 2023-09-23T03:28:31.688 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.686+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:113 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd49500 con 0x55e8d0697000 2023-09-23T03:28:31.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.687+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 23 ==== osd_op_reply(113 10000000000.01000000 [omap-get-vals-by-keys out=502b] v0'0 uv61 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.687+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=9 frag=0*, r=0 2023-09-23T03:28:31.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.687+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.689 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.687+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:114 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48a80 con 0x55e8d0697000 2023-09-23T03:28:31.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.691+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 24 ==== osd_op_reply(114 10000000000.01000000 [omap-set-vals] v128'62 uv62 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.691+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/9 pointing to 0x0x100000001fe 2023-09-23T03:28:31.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.691+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x100000001ff 2023-09-23T03:28:31.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.691+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:10 2023-09-23T03:28:31.693 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.691+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:115 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48700 con 0x55e8d0697000 2023-09-23T03:28:31.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.692+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 25 ==== osd_op_reply(115 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv62 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.692+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=10 frag=0*, r=0 2023-09-23T03:28:31.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.693+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.694 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.693+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:116 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48380 con 0x55e8d0697000 2023-09-23T03:28:31.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.697+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 26 ==== osd_op_reply(116 10000000000.01000000 [omap-set-vals] v128'63 uv63 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.697+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/10 pointing to 0x0x100000001ff 2023-09-23T03:28:31.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.697+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000200 2023-09-23T03:28:31.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.697+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:11 2023-09-23T03:28:31.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.697+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:117 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd49c00 con 0x55e8d0697000 2023-09-23T03:28:31.699 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.698+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 27 ==== osd_op_reply(117 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv63 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.698+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=11 frag=0*, r=0 2023-09-23T03:28:31.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.698+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.698+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:118 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48000 con 0x55e8d0697000 2023-09-23T03:28:31.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.702+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 28 ==== osd_op_reply(118 10000000000.01000000 [omap-set-vals] v128'64 uv64 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.702+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/11 pointing to 0x0x10000000200 2023-09-23T03:28:31.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.702+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000201 2023-09-23T03:28:31.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.702+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:12 2023-09-23T03:28:31.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.702+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:119 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48e00 con 0x55e8d0697000 2023-09-23T03:28:31.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.703+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 29 ==== osd_op_reply(119 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv64 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.703+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=12 frag=0*, r=0 2023-09-23T03:28:31.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.703+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.704+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:120 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd49180 con 0x55e8d0697000 2023-09-23T03:28:31.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.708+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 30 ==== osd_op_reply(120 10000000000.01000000 [omap-set-vals] v128'65 uv65 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.708+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/12 pointing to 0x0x10000000201 2023-09-23T03:28:31.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.708+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000202 2023-09-23T03:28:31.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.708+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:13 2023-09-23T03:28:31.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.708+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:121 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd49880 con 0x55e8cfd1cc00 2023-09-23T03:28:31.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.710+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 17 ==== osd_op_reply(121 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv59 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.710+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=13 frag=1*, r=0 2023-09-23T03:28:31.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.710+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.710+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:122 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd49500 con 0x55e8cfd1cc00 2023-09-23T03:28:31.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.714+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 18 ==== osd_op_reply(122 10000000000.01800000 [omap-set-vals] v128'60 uv60 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.714+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/13 pointing to 0x0x10000000202 2023-09-23T03:28:31.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.714+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000203 2023-09-23T03:28:31.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.714+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:14 2023-09-23T03:28:31.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.714+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:123 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfd48a80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.716 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.715+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 19 ==== osd_op_reply(123 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv60 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.715+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=14 frag=1*, r=0 2023-09-23T03:28:31.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.715+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.717 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.715+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:124 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf8000 con 0x55e8cfd1cc00 2023-09-23T03:28:31.720 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.719+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 20 ==== osd_op_reply(124 10000000000.01800000 [omap-set-vals] v128'61 uv61 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.719+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/14 pointing to 0x0x10000000203 2023-09-23T03:28:31.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.719+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000204 2023-09-23T03:28:31.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.719+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:15 2023-09-23T03:28:31.721 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.719+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:125 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf8380 con 0x55e8d0697000 2023-09-23T03:28:31.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.721+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 31 ==== osd_op_reply(125 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv65 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.721+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=15 frag=0*, r=0 2023-09-23T03:28:31.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.721+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.722 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.721+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:126 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf8700 con 0x55e8d0697000 2023-09-23T03:28:31.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.724+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 32 ==== osd_op_reply(126 10000000000.01000000 [omap-set-vals] v128'66 uv66 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.725 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.724+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/15 pointing to 0x0x10000000204 2023-09-23T03:28:31.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.724+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000205 2023-09-23T03:28:31.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.724+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:16 2023-09-23T03:28:31.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.724+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:127 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf8a80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.726 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.725+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 21 ==== osd_op_reply(127 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv61 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.725+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=16 frag=1*, r=0 2023-09-23T03:28:31.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.725+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.727 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.725+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:128 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf8e00 con 0x55e8cfd1cc00 2023-09-23T03:28:31.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.730+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 22 ==== osd_op_reply(128 10000000000.01800000 [omap-set-vals] v128'62 uv62 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.731 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.730+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/16 pointing to 0x0x10000000205 2023-09-23T03:28:31.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.730+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000206 2023-09-23T03:28:31.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.730+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:17 2023-09-23T03:28:31.732 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.730+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:129 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf9180 con 0x55e8cfd1cc00 2023-09-23T03:28:31.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.731+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 23 ==== osd_op_reply(129 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv62 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.731+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=17 frag=1*, r=0 2023-09-23T03:28:31.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.732+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.733 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.732+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:130 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf9500 con 0x55e8cfd1cc00 2023-09-23T03:28:31.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.735+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 24 ==== osd_op_reply(130 10000000000.01800000 [omap-set-vals] v128'63 uv63 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.735+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/17 pointing to 0x0x10000000206 2023-09-23T03:28:31.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.735+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000207 2023-09-23T03:28:31.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.735+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:18 2023-09-23T03:28:31.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.735+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:131 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf9880 con 0x55e8cfd1cc00 2023-09-23T03:28:31.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.737+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 25 ==== osd_op_reply(131 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv63 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.737+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=18 frag=1*, r=0 2023-09-23T03:28:31.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.737+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.738 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.737+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:132 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf9c00 con 0x55e8cfd1cc00 2023-09-23T03:28:31.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.741+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 26 ==== osd_op_reply(132 10000000000.01800000 [omap-set-vals] v128'64 uv64 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.741+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/18 pointing to 0x0x10000000207 2023-09-23T03:28:31.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.741+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000208 2023-09-23T03:28:31.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.741+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:19 2023-09-23T03:28:31.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.741+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:133 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf8000 con 0x55e8d0697000 2023-09-23T03:28:31.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.742+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 33 ==== osd_op_reply(133 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv66 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.742+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=19 frag=0*, r=0 2023-09-23T03:28:31.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.742+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.744 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.742+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:134 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf8a80 con 0x55e8d0697000 2023-09-23T03:28:31.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.746+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 34 ==== osd_op_reply(134 10000000000.01000000 [omap-set-vals] v128'67 uv67 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.746+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/19 pointing to 0x0x10000000208 2023-09-23T03:28:31.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.746+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000209 2023-09-23T03:28:31.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.746+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:20 2023-09-23T03:28:31.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.746+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:135 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf8e00 con 0x55e8d0697000 2023-09-23T03:28:31.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.747+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 35 ==== osd_op_reply(135 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv67 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.747+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=20 frag=0*, r=0 2023-09-23T03:28:31.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.747+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.747+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:136 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf9180 con 0x55e8d0697000 2023-09-23T03:28:31.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.751+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 36 ==== osd_op_reply(136 10000000000.01000000 [omap-set-vals] v128'68 uv68 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.752+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/20 pointing to 0x0x10000000209 2023-09-23T03:28:31.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.752+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x1000000020a 2023-09-23T03:28:31.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.752+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:21 2023-09-23T03:28:31.753 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.752+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:137 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf8380 con 0x55e8d0697000 2023-09-23T03:28:31.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.753+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 37 ==== osd_op_reply(137 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv68 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.753+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=21 frag=0*, r=0 2023-09-23T03:28:31.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.753+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.753+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:138 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf8700 con 0x55e8d0697000 2023-09-23T03:28:31.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.757+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 38 ==== osd_op_reply(138 10000000000.01000000 [omap-set-vals] v128'69 uv69 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.757+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/21 pointing to 0x0x1000000020a 2023-09-23T03:28:31.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.757+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x1000000020b 2023-09-23T03:28:31.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.757+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:22 2023-09-23T03:28:31.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.757+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:139 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8cfd1cc00 2023-09-23T03:28:31.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.758+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 27 ==== osd_op_reply(139 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv64 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.758+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=22 frag=1*, r=0 2023-09-23T03:28:31.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.758+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.758+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:140 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfe000 con 0x55e8cfd1cc00 2023-09-23T03:28:31.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.762+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 28 ==== osd_op_reply(140 10000000000.01800000 [omap-set-vals] v128'65 uv65 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.762+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/22 pointing to 0x0x1000000020b 2023-09-23T03:28:31.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.762+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x1000000020c 2023-09-23T03:28:31.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.762+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:23 2023-09-23T03:28:31.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.762+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:141 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfe380 con 0x55e8cfd1cc00 2023-09-23T03:28:31.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.763+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 29 ==== osd_op_reply(141 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv65 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.763+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=23 frag=1*, r=0 2023-09-23T03:28:31.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.763+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.763+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:142 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfe700 con 0x55e8cfd1cc00 2023-09-23T03:28:31.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.767+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 30 ==== osd_op_reply(142 10000000000.01800000 [omap-set-vals] v128'66 uv66 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.767+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/23 pointing to 0x0x1000000020c 2023-09-23T03:28:31.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.767+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x1000000020d 2023-09-23T03:28:31.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.767+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:24 2023-09-23T03:28:31.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.767+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:143 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfea80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.768+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 31 ==== osd_op_reply(143 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv66 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.768+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=24 frag=1*, r=0 2023-09-23T03:28:31.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.769+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.769+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:144 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfee00 con 0x55e8cfd1cc00 2023-09-23T03:28:31.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.772+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 32 ==== osd_op_reply(144 10000000000.01800000 [omap-set-vals] v128'67 uv67 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.772+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/24 pointing to 0x0x1000000020d 2023-09-23T03:28:31.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.772+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x1000000020e 2023-09-23T03:28:31.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.772+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:25 2023-09-23T03:28:31.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.772+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:145 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdff180 con 0x55e8cfd1cc00 2023-09-23T03:28:31.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.774+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 33 ==== osd_op_reply(145 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv67 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.774+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=25 frag=1*, r=0 2023-09-23T03:28:31.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.774+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.774+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:146 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdff500 con 0x55e8cfd1cc00 2023-09-23T03:28:31.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.778+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 34 ==== osd_op_reply(146 10000000000.01800000 [omap-set-vals] v128'68 uv68 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.778+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/25 pointing to 0x0x1000000020e 2023-09-23T03:28:31.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.778+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x1000000020f 2023-09-23T03:28:31.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.778+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:26 2023-09-23T03:28:31.780 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.778+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:147 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdff880 con 0x55e8cfd1cc00 2023-09-23T03:28:31.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.780+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 35 ==== osd_op_reply(147 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv68 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.780+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=26 frag=1*, r=0 2023-09-23T03:28:31.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.780+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.781 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.780+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:148 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdffc00 con 0x55e8cfd1cc00 2023-09-23T03:28:31.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.784+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 36 ==== osd_op_reply(148 10000000000.01800000 [omap-set-vals] v128'69 uv69 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.784+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/26 pointing to 0x0x1000000020f 2023-09-23T03:28:31.785 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.784+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000210 2023-09-23T03:28:31.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.784+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:27 2023-09-23T03:28:31.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.784+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:149 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfe000 con 0x55e8d0697000 2023-09-23T03:28:31.786 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.785+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 39 ==== osd_op_reply(149 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv69 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.785+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=27 frag=0*, r=0 2023-09-23T03:28:31.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.785+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.787 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.785+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:150 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfe380 con 0x55e8d0697000 2023-09-23T03:28:31.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.789+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 40 ==== osd_op_reply(150 10000000000.01000000 [omap-set-vals] v128'70 uv70 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.790+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/27 pointing to 0x0x10000000210 2023-09-23T03:28:31.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.790+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000211 2023-09-23T03:28:31.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.790+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:28 2023-09-23T03:28:31.791 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.790+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:151 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfe700 con 0x55e8d0697000 2023-09-23T03:28:31.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.791+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 41 ==== osd_op_reply(151 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv70 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.791+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=28 frag=0*, r=0 2023-09-23T03:28:31.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.791+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.792 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.791+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:152 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfea80 con 0x55e8d0697000 2023-09-23T03:28:31.795 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.794+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 42 ==== osd_op_reply(152 10000000000.01000000 [omap-set-vals] v128'71 uv71 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.794+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/28 pointing to 0x0x10000000211 2023-09-23T03:28:31.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.794+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000212 2023-09-23T03:28:31.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.794+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:29 2023-09-23T03:28:31.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.794+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:153 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfee00 con 0x55e8cfd1cc00 2023-09-23T03:28:31.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.795+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 37 ==== osd_op_reply(153 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv69 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.795+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=29 frag=1*, r=0 2023-09-23T03:28:31.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.795+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.797 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.795+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:154 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8cfd1cc00 2023-09-23T03:28:31.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.799+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 38 ==== osd_op_reply(154 10000000000.01800000 [omap-set-vals] v128'70 uv70 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.800+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/29 pointing to 0x0x10000000212 2023-09-23T03:28:31.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.800+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000213 2023-09-23T03:28:31.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.800+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:30 2023-09-23T03:28:31.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.800+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:155 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf9500 con 0x55e8cfd1cc00 2023-09-23T03:28:31.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.801+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 39 ==== osd_op_reply(155 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv70 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.801+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=30 frag=1*, r=0 2023-09-23T03:28:31.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.801+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.801+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:156 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf9880 con 0x55e8cfd1cc00 2023-09-23T03:28:31.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.805+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 40 ==== osd_op_reply(156 10000000000.01800000 [omap-set-vals] v128'71 uv71 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.806+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/30 pointing to 0x0x10000000213 2023-09-23T03:28:31.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.806+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000214 2023-09-23T03:28:31.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.806+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:31 2023-09-23T03:28:31.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.806+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:157 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf9c00 con 0x55e8d0697000 2023-09-23T03:28:31.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.807+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 43 ==== osd_op_reply(157 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv71 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.807+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=31 frag=0*, r=0 2023-09-23T03:28:31.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.807+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.807+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:158 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfe000 con 0x55e8d0697000 2023-09-23T03:28:31.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.810+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 44 ==== osd_op_reply(158 10000000000.01000000 [omap-set-vals] v128'72 uv72 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.810+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/31 pointing to 0x0x10000000214 2023-09-23T03:28:31.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.810+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000215 2023-09-23T03:28:31.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.810+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:32 2023-09-23T03:28:31.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.810+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:159 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfe380 con 0x55e8cfd1cc00 2023-09-23T03:28:31.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.812+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 41 ==== osd_op_reply(159 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv71 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.812+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=32 frag=1*, r=0 2023-09-23T03:28:31.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.812+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.813 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.812+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:160 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfe700 con 0x55e8cfd1cc00 2023-09-23T03:28:31.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.816+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 42 ==== osd_op_reply(160 10000000000.01800000 [omap-set-vals] v128'72 uv72 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.816+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/32 pointing to 0x0x10000000215 2023-09-23T03:28:31.817 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.816+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000216 2023-09-23T03:28:31.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.816+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:33 2023-09-23T03:28:31.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.816+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:161 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf8000 con 0x55e8cfd1cc00 2023-09-23T03:28:31.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.817+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 43 ==== osd_op_reply(161 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv72 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.817+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=33 frag=1*, r=0 2023-09-23T03:28:31.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.817+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.818+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:162 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf8a80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.822+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 44 ==== osd_op_reply(162 10000000000.01800000 [omap-set-vals] v128'73 uv73 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.822+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/33 pointing to 0x0x10000000216 2023-09-23T03:28:31.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.822+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000217 2023-09-23T03:28:31.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.822+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:34 2023-09-23T03:28:31.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.822+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:163 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf8e00 con 0x55e8d0697000 2023-09-23T03:28:31.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.823+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 45 ==== osd_op_reply(163 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv72 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.823+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=34 frag=0*, r=0 2023-09-23T03:28:31.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.823+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.823+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:164 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf9180 con 0x55e8d0697000 2023-09-23T03:28:31.828 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.827+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 46 ==== osd_op_reply(164 10000000000.01000000 [omap-set-vals] v128'73 uv73 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.827+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/34 pointing to 0x0x10000000217 2023-09-23T03:28:31.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.827+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000218 2023-09-23T03:28:31.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.827+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:35 2023-09-23T03:28:31.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.827+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:165 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf8380 con 0x55e8cfd1cc00 2023-09-23T03:28:31.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.828+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 45 ==== osd_op_reply(165 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv73 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.828+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=35 frag=1*, r=0 2023-09-23T03:28:31.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.828+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.828+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:166 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf8700 con 0x55e8cfd1cc00 2023-09-23T03:28:31.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.833+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 46 ==== osd_op_reply(166 10000000000.01800000 [omap-set-vals] v128'74 uv74 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.833+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/35 pointing to 0x0x10000000218 2023-09-23T03:28:31.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.833+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000219 2023-09-23T03:28:31.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.833+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:36 2023-09-23T03:28:31.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.833+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:167 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdff180 con 0x55e8d0697000 2023-09-23T03:28:31.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.834+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 47 ==== osd_op_reply(167 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv73 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.834+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=36 frag=0*, r=0 2023-09-23T03:28:31.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.834+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.834+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:168 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdff500 con 0x55e8d0697000 2023-09-23T03:28:31.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.838+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 48 ==== osd_op_reply(168 10000000000.01000000 [omap-set-vals] v128'74 uv74 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.838+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/36 pointing to 0x0x10000000219 2023-09-23T03:28:31.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.838+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x1000000021a 2023-09-23T03:28:31.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.838+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:37 2023-09-23T03:28:31.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.838+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:169 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdff880 con 0x55e8cfd1cc00 2023-09-23T03:28:31.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.839+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 47 ==== osd_op_reply(169 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv74 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.840+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=37 frag=1*, r=0 2023-09-23T03:28:31.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.840+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.840+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:170 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdffc00 con 0x55e8cfd1cc00 2023-09-23T03:28:31.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.844+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 48 ==== osd_op_reply(170 10000000000.01800000 [omap-set-vals] v128'75 uv75 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.844+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/37 pointing to 0x0x1000000021a 2023-09-23T03:28:31.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.844+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x1000000021b 2023-09-23T03:28:31.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.844+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:38 2023-09-23T03:28:31.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.844+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:171 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfee00 con 0x55e8d0697000 2023-09-23T03:28:31.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.845+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 49 ==== osd_op_reply(171 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv74 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.845+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=38 frag=0*, r=0 2023-09-23T03:28:31.846 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.845+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.847 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.845+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:172 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfe380 con 0x55e8d0697000 2023-09-23T03:28:31.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.849+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 50 ==== osd_op_reply(172 10000000000.01000000 [omap-set-vals] v128'75 uv75 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.849+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/38 pointing to 0x0x1000000021b 2023-09-23T03:28:31.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.849+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x1000000021c 2023-09-23T03:28:31.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.849+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:39 2023-09-23T03:28:31.850 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.849+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:173 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfe700 con 0x55e8d0697000 2023-09-23T03:28:31.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.850+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 51 ==== osd_op_reply(173 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv75 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.850+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=39 frag=0*, r=0 2023-09-23T03:28:31.851 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.850+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.852 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.850+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:174 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf9500 con 0x55e8d0697000 2023-09-23T03:28:31.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.854+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 52 ==== osd_op_reply(174 10000000000.01000000 [omap-set-vals] v128'76 uv76 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.854+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/39 pointing to 0x0x1000000021c 2023-09-23T03:28:31.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.854+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x1000000021d 2023-09-23T03:28:31.855 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.854+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:40 2023-09-23T03:28:31.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.854+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:175 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf9880 con 0x55e8cfd1cc00 2023-09-23T03:28:31.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.855+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 49 ==== osd_op_reply(175 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv75 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.856 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.855+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=40 frag=1*, r=0 2023-09-23T03:28:31.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.855+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.855+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:176 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8cfd1cc00 2023-09-23T03:28:31.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.859+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 50 ==== osd_op_reply(176 10000000000.01800000 [omap-set-vals] v128'76 uv76 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.860 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.859+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/40 pointing to 0x0x1000000021d 2023-09-23T03:28:31.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.859+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x1000000021e 2023-09-23T03:28:31.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.859+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:41 2023-09-23T03:28:31.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.859+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:177 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfea80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.861+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 51 ==== osd_op_reply(177 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv76 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.861+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=41 frag=1*, r=0 2023-09-23T03:28:31.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.861+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.861+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:178 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfe000 con 0x55e8cfd1cc00 2023-09-23T03:28:31.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.865+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 52 ==== osd_op_reply(178 10000000000.01800000 [omap-set-vals] v128'77 uv77 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.865+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/41 pointing to 0x0x1000000021e 2023-09-23T03:28:31.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.865+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x1000000021f 2023-09-23T03:28:31.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.865+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:42 2023-09-23T03:28:31.867 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.865+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:179 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdff180 con 0x55e8d0697000 2023-09-23T03:28:31.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.867+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 53 ==== osd_op_reply(179 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv76 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.867+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=42 frag=0*, r=0 2023-09-23T03:28:31.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.867+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.868 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.867+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:180 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdff500 con 0x55e8d0697000 2023-09-23T03:28:31.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.871+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 54 ==== osd_op_reply(180 10000000000.01000000 [omap-set-vals] v128'77 uv77 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.871+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/42 pointing to 0x0x1000000021f 2023-09-23T03:28:31.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.871+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000220 2023-09-23T03:28:31.872 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.871+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:43 2023-09-23T03:28:31.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.871+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:181 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfee00 con 0x55e8d0697000 2023-09-23T03:28:31.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.872+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 55 ==== osd_op_reply(181 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv77 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.872+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=43 frag=0*, r=0 2023-09-23T03:28:31.873 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.872+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.872+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:182 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfe380 con 0x55e8d0697000 2023-09-23T03:28:31.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.876+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 56 ==== osd_op_reply(182 10000000000.01000000 [omap-set-vals] v128'78 uv78 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.877 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.876+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/43 pointing to 0x0x10000000220 2023-09-23T03:28:31.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.876+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000221 2023-09-23T03:28:31.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.876+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:44 2023-09-23T03:28:31.878 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.876+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:183 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfe700 con 0x55e8cfd1cc00 2023-09-23T03:28:31.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.877+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 53 ==== osd_op_reply(183 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv77 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.877+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=44 frag=1*, r=0 2023-09-23T03:28:31.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.877+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.877+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:184 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf9c00 con 0x55e8cfd1cc00 2023-09-23T03:28:31.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.881+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 54 ==== osd_op_reply(184 10000000000.01800000 [omap-set-vals] v128'78 uv78 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.881+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/44 pointing to 0x0x10000000221 2023-09-23T03:28:31.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.881+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000222 2023-09-23T03:28:31.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.881+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:45 2023-09-23T03:28:31.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.881+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:185 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf8e00 con 0x55e8cfd1cc00 2023-09-23T03:28:31.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.882+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 55 ==== osd_op_reply(185 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv78 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.882+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=45 frag=1*, r=0 2023-09-23T03:28:31.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.882+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.882+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:186 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf9180 con 0x55e8cfd1cc00 2023-09-23T03:28:31.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.886+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 56 ==== osd_op_reply(186 10000000000.01800000 [omap-set-vals] v128'79 uv79 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.886+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/45 pointing to 0x0x10000000222 2023-09-23T03:28:31.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.886+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000223 2023-09-23T03:28:31.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.886+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:46 2023-09-23T03:28:31.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.886+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:187 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdff880 con 0x55e8d0697000 2023-09-23T03:28:31.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.888+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 57 ==== osd_op_reply(187 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv78 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.888+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=46 frag=0*, r=0 2023-09-23T03:28:31.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.888+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.888+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:188 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdffc00 con 0x55e8d0697000 2023-09-23T03:28:31.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.892+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 58 ==== osd_op_reply(188 10000000000.01000000 [omap-set-vals] v128'79 uv79 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.892+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/46 pointing to 0x0x10000000223 2023-09-23T03:28:31.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.892+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000224 2023-09-23T03:28:31.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.892+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:47 2023-09-23T03:28:31.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.892+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:189 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfea80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.893+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 57 ==== osd_op_reply(189 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv79 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.893+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=47 frag=1*, r=0 2023-09-23T03:28:31.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.893+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.893+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:190 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfe000 con 0x55e8cfd1cc00 2023-09-23T03:28:31.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.897+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 58 ==== osd_op_reply(190 10000000000.01800000 [omap-set-vals] v128'80 uv80 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.897+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/47 pointing to 0x0x10000000224 2023-09-23T03:28:31.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.897+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000225 2023-09-23T03:28:31.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.897+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:48 2023-09-23T03:28:31.899 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.897+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:191 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfe700 con 0x55e8cfd1cc00 2023-09-23T03:28:31.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.898+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 59 ==== osd_op_reply(191 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv80 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.898+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=48 frag=1*, r=0 2023-09-23T03:28:31.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.898+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.900 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.898+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:192 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8cfd1cc00 2023-09-23T03:28:31.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.902+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 60 ==== osd_op_reply(192 10000000000.01800000 [omap-set-vals] v128'81 uv81 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.902+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/48 pointing to 0x0x10000000225 2023-09-23T03:28:31.903 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.902+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000226 2023-09-23T03:28:31.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.902+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:49 2023-09-23T03:28:31.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.902+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:193 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf8000 con 0x55e8cfd1cc00 2023-09-23T03:28:31.904 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.903+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 61 ==== osd_op_reply(193 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv81 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.903+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=49 frag=1*, r=0 2023-09-23T03:28:31.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.903+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.903+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:194 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf8a80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.908+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 62 ==== osd_op_reply(194 10000000000.01800000 [omap-set-vals] v128'82 uv82 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.908+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/49 pointing to 0x0x10000000226 2023-09-23T03:28:31.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.908+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000227 2023-09-23T03:28:31.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.908+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:50 2023-09-23T03:28:31.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.908+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:195 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf8380 con 0x55e8d0697000 2023-09-23T03:28:31.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.909+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 59 ==== osd_op_reply(195 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv79 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.909+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=50 frag=0*, r=0 2023-09-23T03:28:31.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.909+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.909+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:196 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf8700 con 0x55e8d0697000 2023-09-23T03:28:31.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.913+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 60 ==== osd_op_reply(196 10000000000.01000000 [omap-set-vals] v128'80 uv80 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.913+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/50 pointing to 0x0x10000000227 2023-09-23T03:28:31.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.913+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000228 2023-09-23T03:28:31.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.913+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:51 2023-09-23T03:28:31.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.913+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:197 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf9880 con 0x55e8cfd1cc00 2023-09-23T03:28:31.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.914+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 63 ==== osd_op_reply(197 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv82 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.914+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=51 frag=1*, r=0 2023-09-23T03:28:31.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.914+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.914+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:198 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfe20000 con 0x55e8cfd1cc00 2023-09-23T03:28:31.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.918+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 64 ==== osd_op_reply(198 10000000000.01800000 [omap-set-vals] v128'83 uv83 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.918+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/51 pointing to 0x0x10000000228 2023-09-23T03:28:31.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.918+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000229 2023-09-23T03:28:31.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.918+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:52 2023-09-23T03:28:31.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.918+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:199 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfe20380 con 0x55e8d0697000 2023-09-23T03:28:31.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.919+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 61 ==== osd_op_reply(199 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv80 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.919+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=52 frag=0*, r=0 2023-09-23T03:28:31.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.919+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.919+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:200 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfe20700 con 0x55e8d0697000 2023-09-23T03:28:31.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.923+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 62 ==== osd_op_reply(200 10000000000.01000000 [omap-set-vals] v128'81 uv81 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.923+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/52 pointing to 0x0x10000000229 2023-09-23T03:28:31.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.923+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x1000000022a 2023-09-23T03:28:31.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.923+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:53 2023-09-23T03:28:31.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.923+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:201 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfe20a80 con 0x55e8d0697000 2023-09-23T03:28:31.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.924+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 63 ==== osd_op_reply(201 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv81 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.924+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=53 frag=0*, r=0 2023-09-23T03:28:31.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.924+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.924+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:202 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfe20e00 con 0x55e8d0697000 2023-09-23T03:28:31.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.928+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 64 ==== osd_op_reply(202 10000000000.01000000 [omap-set-vals] v128'82 uv82 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.928+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/53 pointing to 0x0x1000000022a 2023-09-23T03:28:31.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.928+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x1000000022b 2023-09-23T03:28:31.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.928+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:54 2023-09-23T03:28:31.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.928+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:203 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfe21180 con 0x55e8cfd1cc00 2023-09-23T03:28:31.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.929+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 65 ==== osd_op_reply(203 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv83 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.930+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=54 frag=1*, r=0 2023-09-23T03:28:31.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.930+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.931 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.930+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:204 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfe21500 con 0x55e8cfd1cc00 2023-09-23T03:28:31.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.933+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 66 ==== osd_op_reply(204 10000000000.01800000 [omap-set-vals] v128'84 uv84 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.933+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/54 pointing to 0x0x1000000022b 2023-09-23T03:28:31.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.933+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x1000000022c 2023-09-23T03:28:31.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.933+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:55 2023-09-23T03:28:31.935 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.933+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:205 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfe21880 con 0x55e8d0697000 2023-09-23T03:28:31.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.934+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 65 ==== osd_op_reply(205 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv82 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.934+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=55 frag=0*, r=0 2023-09-23T03:28:31.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.934+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.936 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.935+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:206 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfe21c00 con 0x55e8d0697000 2023-09-23T03:28:31.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.938+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 66 ==== osd_op_reply(206 10000000000.01000000 [omap-set-vals] v128'83 uv83 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.938+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/55 pointing to 0x0x1000000022c 2023-09-23T03:28:31.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.938+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x1000000022d 2023-09-23T03:28:31.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.938+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:56 2023-09-23T03:28:31.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.938+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:207 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfe20000 con 0x55e8cfd1cc00 2023-09-23T03:28:31.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.939+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 67 ==== osd_op_reply(207 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv84 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.939+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=56 frag=1*, r=0 2023-09-23T03:28:31.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.939+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.939+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:208 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfe21180 con 0x55e8cfd1cc00 2023-09-23T03:28:31.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.943+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 68 ==== osd_op_reply(208 10000000000.01800000 [omap-set-vals] v128'85 uv85 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.944+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/56 pointing to 0x0x1000000022d 2023-09-23T03:28:31.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.944+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x1000000022e 2023-09-23T03:28:31.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.944+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:57 2023-09-23T03:28:31.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.944+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:209 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfe20380 con 0x55e8cfd1cc00 2023-09-23T03:28:31.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.945+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 69 ==== osd_op_reply(209 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv85 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.945+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=57 frag=1*, r=0 2023-09-23T03:28:31.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.945+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.946 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.945+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:210 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfe20700 con 0x55e8cfd1cc00 2023-09-23T03:28:31.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.949+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 70 ==== osd_op_reply(210 10000000000.01800000 [omap-set-vals] v128'86 uv86 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.949+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/57 pointing to 0x0x1000000022e 2023-09-23T03:28:31.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.949+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x1000000022f 2023-09-23T03:28:31.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.949+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:58 2023-09-23T03:28:31.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.949+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:211 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8cfd1cc00 2023-09-23T03:28:31.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.950+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 71 ==== osd_op_reply(211 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv86 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.950+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=58 frag=1*, r=0 2023-09-23T03:28:31.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.950+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.950+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:212 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf8000 con 0x55e8cfd1cc00 2023-09-23T03:28:31.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.954+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 72 ==== osd_op_reply(212 10000000000.01800000 [omap-set-vals] v128'87 uv87 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.954+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/58 pointing to 0x0x1000000022f 2023-09-23T03:28:31.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.954+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000230 2023-09-23T03:28:31.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.954+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:59 2023-09-23T03:28:31.956 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.954+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:213 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf8380 con 0x55e8d0697000 2023-09-23T03:28:31.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.956+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 67 ==== osd_op_reply(213 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv83 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.956+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=59 frag=0*, r=0 2023-09-23T03:28:31.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.956+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.957 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.956+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:214 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf8700 con 0x55e8d0697000 2023-09-23T03:28:31.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.959+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 68 ==== osd_op_reply(214 10000000000.01000000 [omap-set-vals] v128'84 uv84 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.960 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.959+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/59 pointing to 0x0x10000000230 2023-09-23T03:28:31.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.959+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000231 2023-09-23T03:28:31.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.959+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:60 2023-09-23T03:28:31.961 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.959+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:215 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf8a80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.960+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 73 ==== osd_op_reply(215 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv87 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.961+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=60 frag=1*, r=0 2023-09-23T03:28:31.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.961+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.962 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.961+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:216 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf8e00 con 0x55e8cfd1cc00 2023-09-23T03:28:31.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.965+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 74 ==== osd_op_reply(216 10000000000.01800000 [omap-set-vals] v128'88 uv88 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.965+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/60 pointing to 0x0x10000000231 2023-09-23T03:28:31.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.965+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000232 2023-09-23T03:28:31.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.965+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:61 2023-09-23T03:28:31.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.965+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:217 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf9180 con 0x55e8d0697000 2023-09-23T03:28:31.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.966+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 69 ==== osd_op_reply(217 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv84 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.966+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=61 frag=0*, r=0 2023-09-23T03:28:31.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.966+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.968 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.966+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:218 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf9500 con 0x55e8d0697000 2023-09-23T03:28:31.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.970+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 70 ==== osd_op_reply(218 10000000000.01000000 [omap-set-vals] v128'85 uv85 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.970+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/61 pointing to 0x0x10000000232 2023-09-23T03:28:31.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.970+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000233 2023-09-23T03:28:31.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.970+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:62 2023-09-23T03:28:31.972 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.970+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:219 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf9880 con 0x55e8cfd1cc00 2023-09-23T03:28:31.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.972+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 75 ==== osd_op_reply(219 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv88 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.972+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=62 frag=1*, r=0 2023-09-23T03:28:31.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.972+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.972+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:220 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf9c00 con 0x55e8cfd1cc00 2023-09-23T03:28:31.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.976+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 76 ==== osd_op_reply(220 10000000000.01800000 [omap-set-vals] v128'89 uv89 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.976+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/62 pointing to 0x0x10000000233 2023-09-23T03:28:31.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.976+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000234 2023-09-23T03:28:31.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.976+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:63 2023-09-23T03:28:31.977 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.976+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:221 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfe000 con 0x55e8d0697000 2023-09-23T03:28:31.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.977+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 71 ==== osd_op_reply(221 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv85 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.978 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.977+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=63 frag=0*, r=0 2023-09-23T03:28:31.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.977+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.979 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.977+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:222 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfe380 con 0x55e8d0697000 2023-09-23T03:28:31.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.981+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 72 ==== osd_op_reply(222 10000000000.01000000 [omap-set-vals] v128'86 uv86 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.981+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/63 pointing to 0x0x10000000234 2023-09-23T03:28:31.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.981+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000235 2023-09-23T03:28:31.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.981+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:64 2023-09-23T03:28:31.982 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.981+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:223 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfe700 con 0x55e8d0697000 2023-09-23T03:28:31.983 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.982+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 73 ==== osd_op_reply(223 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv86 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.982+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=64 frag=0*, r=0 2023-09-23T03:28:31.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.982+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.984 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.982+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:224 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfea80 con 0x55e8d0697000 2023-09-23T03:28:31.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.986+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 74 ==== osd_op_reply(224 10000000000.01000000 [omap-set-vals] v128'87 uv87 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.986+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/64 pointing to 0x0x10000000235 2023-09-23T03:28:31.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.986+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000236 2023-09-23T03:28:31.987 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.986+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:65 2023-09-23T03:28:31.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.986+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:225 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfee00 con 0x55e8d0697000 2023-09-23T03:28:31.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.988+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 75 ==== osd_op_reply(225 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv87 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.988 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.988+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=65 frag=0*, r=0 2023-09-23T03:28:31.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.988+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.989 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.988+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:226 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdff180 con 0x55e8d0697000 2023-09-23T03:28:31.992 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.991+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 76 ==== osd_op_reply(226 10000000000.01000000 [omap-set-vals] v128'88 uv88 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.991+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/65 pointing to 0x0x10000000236 2023-09-23T03:28:31.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.991+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000237 2023-09-23T03:28:31.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.991+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:66 2023-09-23T03:28:31.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.991+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:227 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdff500 con 0x55e8d0697000 2023-09-23T03:28:31.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.992+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 77 ==== osd_op_reply(227 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv88 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.992+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=66 frag=0*, r=0 2023-09-23T03:28:31.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.992+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.992+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:228 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdff880 con 0x55e8d0697000 2023-09-23T03:28:31.997 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.996+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 78 ==== osd_op_reply(228 10000000000.01000000 [omap-set-vals] v128'89 uv89 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:31.997 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.996+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/66 pointing to 0x0x10000000237 2023-09-23T03:28:31.997 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.996+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000238 2023-09-23T03:28:31.997 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.996+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:67 2023-09-23T03:28:31.997 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.996+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:229 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdffc00 con 0x55e8cfd1cc00 2023-09-23T03:28:31.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.997+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 77 ==== osd_op_reply(229 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv89 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:31.999 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.997+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=67 frag=1*, r=0 2023-09-23T03:28:31.999 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.997+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:31.999 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:31.997+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:230 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfe000 con 0x55e8cfd1cc00 2023-09-23T03:28:32.003 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.001+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 78 ==== osd_op_reply(230 10000000000.01800000 [omap-set-vals] v128'90 uv90 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:32.003 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.001+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/67 pointing to 0x0x10000000238 2023-09-23T03:28:32.003 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.001+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000239 2023-09-23T03:28:32.003 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.001+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:68 2023-09-23T03:28:32.003 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.002+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:231 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfe380 con 0x55e8cfd1cc00 2023-09-23T03:28:32.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.003+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 79 ==== osd_op_reply(231 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv90 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:32.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.003+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=68 frag=1*, r=0 2023-09-23T03:28:32.004 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.003+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:32.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.003+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:232 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfe700 con 0x55e8cfd1cc00 2023-09-23T03:28:32.008 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.007+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 80 ==== osd_op_reply(232 10000000000.01800000 [omap-set-vals] v128'91 uv91 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:32.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.007+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/68 pointing to 0x0x10000000239 2023-09-23T03:28:32.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.007+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x1000000023a 2023-09-23T03:28:32.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.007+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:69 2023-09-23T03:28:32.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.007+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:233 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfea80 con 0x55e8d0697000 2023-09-23T03:28:32.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.008+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 79 ==== osd_op_reply(233 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv89 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:32.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.008+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=69 frag=0*, r=0 2023-09-23T03:28:32.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.008+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:32.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.008+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:234 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf8380 con 0x55e8d0697000 2023-09-23T03:28:32.013 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.012+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 80 ==== osd_op_reply(234 10000000000.01000000 [omap-set-vals] v128'90 uv90 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:32.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.012+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/69 pointing to 0x0x1000000023a 2023-09-23T03:28:32.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.012+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x1000000023b 2023-09-23T03:28:32.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.012+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:70 2023-09-23T03:28:32.014 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.012+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:235 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf8700 con 0x55e8d0697000 2023-09-23T03:28:32.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.013+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 81 ==== osd_op_reply(235 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv90 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:32.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.013+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=70 frag=0*, r=0 2023-09-23T03:28:32.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.013+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:32.015 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.014+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:236 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf9180 con 0x55e8d0697000 2023-09-23T03:28:32.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.017+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 82 ==== osd_op_reply(236 10000000000.01000000 [omap-set-vals] v128'91 uv91 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:32.018 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.017+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/70 pointing to 0x0x1000000023b 2023-09-23T03:28:32.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.017+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x1000000023c 2023-09-23T03:28:32.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.017+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:71 2023-09-23T03:28:32.019 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.017+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:237 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf9500 con 0x55e8cfd1cc00 2023-09-23T03:28:32.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.018+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 81 ==== osd_op_reply(237 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv91 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:32.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.018+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=71 frag=1*, r=0 2023-09-23T03:28:32.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.018+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:32.020 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.018+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:238 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfe20a80 con 0x55e8cfd1cc00 2023-09-23T03:28:32.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.022+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 82 ==== osd_op_reply(238 10000000000.01800000 [omap-set-vals] v128'92 uv92 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:32.023 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.022+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/71 pointing to 0x0x1000000023c 2023-09-23T03:28:32.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.022+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x1000000023d 2023-09-23T03:28:32.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.022+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:72 2023-09-23T03:28:32.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.022+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:239 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfe20e00 con 0x55e8cfd1cc00 2023-09-23T03:28:32.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.024+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 83 ==== osd_op_reply(239 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv92 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:32.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.024+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=72 frag=1*, r=0 2023-09-23T03:28:32.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.024+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:32.025 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.024+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:240 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfe21880 con 0x55e8cfd1cc00 2023-09-23T03:28:32.029 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.027+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 84 ==== osd_op_reply(240 10000000000.01800000 [omap-set-vals] v128'93 uv93 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:32.029 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.028+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/72 pointing to 0x0x1000000023d 2023-09-23T03:28:32.029 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.028+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x1000000023e 2023-09-23T03:28:32.029 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.028+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:73 2023-09-23T03:28:32.029 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.028+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:241 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfe21c00 con 0x55e8d0697000 2023-09-23T03:28:32.030 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.029+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 83 ==== osd_op_reply(241 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv91 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:32.030 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.029+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=73 frag=0*, r=0 2023-09-23T03:28:32.030 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.029+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:32.030 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.029+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:242 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdffc00 con 0x55e8d0697000 2023-09-23T03:28:32.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.032+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 84 ==== osd_op_reply(242 10000000000.01000000 [omap-set-vals] v128'92 uv92 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:32.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.033+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/73 pointing to 0x0x1000000023e 2023-09-23T03:28:32.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.033+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x1000000023f 2023-09-23T03:28:32.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.033+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:74 2023-09-23T03:28:32.034 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.033+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:243 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfe000 con 0x55e8d0697000 2023-09-23T03:28:32.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.034+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 85 ==== osd_op_reply(243 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv92 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:32.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.034+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=74 frag=0*, r=0 2023-09-23T03:28:32.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.034+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:32.035 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.034+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:244 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf8000 con 0x55e8d0697000 2023-09-23T03:28:32.039 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.038+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 86 ==== osd_op_reply(244 10000000000.01000000 [omap-set-vals] v128'93 uv93 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:32.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.038+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/74 pointing to 0x0x1000000023f 2023-09-23T03:28:32.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.038+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000240 2023-09-23T03:28:32.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.038+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:75 2023-09-23T03:28:32.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.038+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:245 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf8a80 con 0x55e8d0697000 2023-09-23T03:28:32.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.039+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 87 ==== osd_op_reply(245 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv93 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:32.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.039+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=75 frag=0*, r=0 2023-09-23T03:28:32.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.039+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:32.041 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.039+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:246 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf8e00 con 0x55e8d0697000 2023-09-23T03:28:32.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.042+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 88 ==== osd_op_reply(246 10000000000.01000000 [omap-set-vals] v128'94 uv94 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:32.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.042+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/75 pointing to 0x0x10000000240 2023-09-23T03:28:32.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.042+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000241 2023-09-23T03:28:32.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.042+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:76 2023-09-23T03:28:32.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.042+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:247 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf9880 con 0x55e8cfd1cc00 2023-09-23T03:28:32.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.044+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 85 ==== osd_op_reply(247 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv93 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:32.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.044+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=76 frag=1*, r=0 2023-09-23T03:28:32.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.044+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:32.045 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.044+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:248 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf9c00 con 0x55e8cfd1cc00 2023-09-23T03:28:32.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.048+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 86 ==== osd_op_reply(248 10000000000.01800000 [omap-set-vals] v128'94 uv94 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:32.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.048+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/76 pointing to 0x0x10000000241 2023-09-23T03:28:32.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.048+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000242 2023-09-23T03:28:32.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.048+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:77 2023-09-23T03:28:32.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.048+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:249 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8d0697000 2023-09-23T03:28:32.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.049+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 89 ==== osd_op_reply(249 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv94 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:32.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.049+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=77 frag=0*, r=0 2023-09-23T03:28:32.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.049+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:32.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.049+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:250 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfe21500 con 0x55e8d0697000 2023-09-23T03:28:32.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.052+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 90 ==== osd_op_reply(250 10000000000.01000000 [omap-set-vals] v128'95 uv95 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:32.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.053+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/77 pointing to 0x0x10000000242 2023-09-23T03:28:32.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.053+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000243 2023-09-23T03:28:32.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.053+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:78 2023-09-23T03:28:32.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.053+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:251 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfe20000 con 0x55e8cfd1cc00 2023-09-23T03:28:32.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.054+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 87 ==== osd_op_reply(251 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv94 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:32.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.054+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=78 frag=1*, r=0 2023-09-23T03:28:32.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.054+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:32.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.054+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:252 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfe21180 con 0x55e8cfd1cc00 2023-09-23T03:28:32.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.058+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 88 ==== osd_op_reply(252 10000000000.01800000 [omap-set-vals] v128'95 uv95 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:32.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.058+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/78 pointing to 0x0x10000000243 2023-09-23T03:28:32.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.058+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000244 2023-09-23T03:28:32.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.058+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:79 2023-09-23T03:28:32.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.058+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:253 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfe20380 con 0x55e8cfd1cc00 2023-09-23T03:28:32.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.059+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 89 ==== osd_op_reply(253 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv95 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:32.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.059+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=79 frag=1*, r=0 2023-09-23T03:28:32.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.059+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:32.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.059+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:254 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfe20700 con 0x55e8cfd1cc00 2023-09-23T03:28:32.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.063+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 90 ==== osd_op_reply(254 10000000000.01800000 [omap-set-vals] v128'96 uv96 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:32.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.063+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/79 pointing to 0x0x10000000244 2023-09-23T03:28:32.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.063+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000245 2023-09-23T03:28:32.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.063+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:80 2023-09-23T03:28:32.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.063+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:255 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf8380 con 0x55e8d0697000 2023-09-23T03:28:32.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.064+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 91 ==== osd_op_reply(255 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv95 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:32.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.064+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=80 frag=0*, r=0 2023-09-23T03:28:32.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.064+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:32.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.064+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:256 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf8700 con 0x55e8d0697000 2023-09-23T03:28:32.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.068+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 92 ==== osd_op_reply(256 10000000000.01000000 [omap-set-vals] v128'96 uv96 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:32.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.068+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/80 pointing to 0x0x10000000245 2023-09-23T03:28:32.069 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.068+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000246 2023-09-23T03:28:32.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.068+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:81 2023-09-23T03:28:32.070 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.068+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:257 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf9180 con 0x55e8cfd1cc00 2023-09-23T03:28:32.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.069+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 91 ==== osd_op_reply(257 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv96 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:32.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.069+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=81 frag=1*, r=0 2023-09-23T03:28:32.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.070+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:32.071 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.070+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:258 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf8000 con 0x55e8cfd1cc00 2023-09-23T03:28:32.075 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.074+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 92 ==== osd_op_reply(258 10000000000.01800000 [omap-set-vals] v128'97 uv97 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:32.075 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.074+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/81 pointing to 0x0x10000000246 2023-09-23T03:28:32.075 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.074+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000247 2023-09-23T03:28:32.075 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.074+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:82 2023-09-23T03:28:32.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.074+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:259 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf8a80 con 0x55e8cfd1cc00 2023-09-23T03:28:32.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.075+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 93 ==== osd_op_reply(259 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv97 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:32.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.075+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=82 frag=1*, r=0 2023-09-23T03:28:32.076 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.075+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:32.077 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.075+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:260 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfee00 con 0x55e8cfd1cc00 2023-09-23T03:28:32.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.079+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 94 ==== osd_op_reply(260 10000000000.01800000 [omap-set-vals] v128'98 uv98 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:32.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.079+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/82 pointing to 0x0x10000000247 2023-09-23T03:28:32.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.079+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000248 2023-09-23T03:28:32.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.079+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:83 2023-09-23T03:28:32.080 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.079+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:261 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdff180 con 0x55e8d0697000 2023-09-23T03:28:32.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.080+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 93 ==== osd_op_reply(261 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv96 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:32.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.080+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=83 frag=0*, r=0 2023-09-23T03:28:32.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.080+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:32.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.080+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:262 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdff500 con 0x55e8d0697000 2023-09-23T03:28:32.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.084+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 94 ==== osd_op_reply(262 10000000000.01000000 [omap-set-vals] v128'97 uv97 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:32.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.084+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/83 pointing to 0x0x10000000248 2023-09-23T03:28:32.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.084+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000249 2023-09-23T03:28:32.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.084+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:84 2023-09-23T03:28:32.085 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.084+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:263 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdff880 con 0x55e8d0697000 2023-09-23T03:28:32.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.085+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 95 ==== osd_op_reply(263 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv97 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:32.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.085+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=84 frag=0*, r=0 2023-09-23T03:28:32.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.085+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:32.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.085+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:264 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfea80 con 0x55e8d0697000 2023-09-23T03:28:32.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.089+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 96 ==== osd_op_reply(264 10000000000.01000000 [omap-set-vals] v128'98 uv98 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:32.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.089+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/84 pointing to 0x0x10000000249 2023-09-23T03:28:32.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.089+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x1000000024a 2023-09-23T03:28:32.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.089+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:85 2023-09-23T03:28:32.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.089+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:265 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdffc00 con 0x55e8d0697000 2023-09-23T03:28:32.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.090+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 97 ==== osd_op_reply(265 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv98 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:32.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.090+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=85 frag=0*, r=0 2023-09-23T03:28:32.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.090+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:32.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.090+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:266 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfe000 con 0x55e8d0697000 2023-09-23T03:28:32.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.094+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 98 ==== osd_op_reply(266 10000000000.01000000 [omap-set-vals] v128'99 uv99 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:32.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.094+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/85 pointing to 0x0x1000000024a 2023-09-23T03:28:32.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.094+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x1000000024b 2023-09-23T03:28:32.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.094+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:86 2023-09-23T03:28:32.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.094+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:267 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfe21c00 con 0x55e8d0697000 2023-09-23T03:28:32.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.095+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 99 ==== osd_op_reply(267 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv99 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:32.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.095+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=86 frag=0*, r=0 2023-09-23T03:28:32.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.095+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:32.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.095+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:268 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf9500 con 0x55e8d0697000 2023-09-23T03:28:32.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.099+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 100 ==== osd_op_reply(268 10000000000.01000000 [omap-set-vals] v128'100 uv100 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:32.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.099+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/86 pointing to 0x0x1000000024b 2023-09-23T03:28:32.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.099+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x1000000024c 2023-09-23T03:28:32.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.099+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:87 2023-09-23T03:28:32.101 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.099+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:269 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf9880 con 0x55e8d0697000 2023-09-23T03:28:32.101 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.100+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 101 ==== osd_op_reply(269 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv100 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:32.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.100+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=87 frag=0*, r=0 2023-09-23T03:28:32.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.100+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:32.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.100+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:270 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf9c00 con 0x55e8d0697000 2023-09-23T03:28:32.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.104+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 102 ==== osd_op_reply(270 10000000000.01000000 [omap-set-vals] v128'101 uv101 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:32.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.104+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/87 pointing to 0x0x1000000024c 2023-09-23T03:28:32.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.104+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x1000000024d 2023-09-23T03:28:32.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.104+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:88 2023-09-23T03:28:32.105 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.104+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:271 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf9180 con 0x55e8d0697000 2023-09-23T03:28:32.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.105+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 103 ==== osd_op_reply(271 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv101 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:32.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.105+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=88 frag=0*, r=0 2023-09-23T03:28:32.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.105+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:32.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.105+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:272 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf8000 con 0x55e8d0697000 2023-09-23T03:28:32.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.109+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 104 ==== osd_op_reply(272 10000000000.01000000 [omap-set-vals] v128'102 uv102 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:32.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.109+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/88 pointing to 0x0x1000000024d 2023-09-23T03:28:32.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.109+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x1000000024e 2023-09-23T03:28:32.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.109+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:89 2023-09-23T03:28:32.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.109+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:273 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfe20a80 con 0x55e8cfd1cc00 2023-09-23T03:28:32.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.111+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 95 ==== osd_op_reply(273 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv98 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:32.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.111+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=89 frag=1*, r=0 2023-09-23T03:28:32.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.111+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:32.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.111+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:274 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfe20e00 con 0x55e8cfd1cc00 2023-09-23T03:28:32.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.115+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 96 ==== osd_op_reply(274 10000000000.01800000 [omap-set-vals] v128'99 uv99 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:32.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.115+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/89 pointing to 0x0x1000000024e 2023-09-23T03:28:32.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.115+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x1000000024f 2023-09-23T03:28:32.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.115+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:90 2023-09-23T03:28:32.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.115+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:275 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfe21880 con 0x55e8d0697000 2023-09-23T03:28:32.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.116+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 105 ==== osd_op_reply(275 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv102 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:32.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.116+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=90 frag=0*, r=0 2023-09-23T03:28:32.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.116+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:32.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.116+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:276 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfe20000 con 0x55e8d0697000 2023-09-23T03:28:32.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.120+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 106 ==== osd_op_reply(276 10000000000.01000000 [omap-set-vals] v128'103 uv103 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:32.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.120+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/90 pointing to 0x0x1000000024f 2023-09-23T03:28:32.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.120+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000250 2023-09-23T03:28:32.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.120+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:91 2023-09-23T03:28:32.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.120+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:277 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfe21180 con 0x55e8cfd1cc00 2023-09-23T03:28:32.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.121+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 97 ==== osd_op_reply(277 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv99 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:32.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.121+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=91 frag=1*, r=0 2023-09-23T03:28:32.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.121+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:32.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.121+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:278 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfe20380 con 0x55e8cfd1cc00 2023-09-23T03:28:32.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.126+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 98 ==== osd_op_reply(278 10000000000.01800000 [omap-set-vals] v128'100 uv100 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:32.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.126+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/91 pointing to 0x0x10000000250 2023-09-23T03:28:32.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.126+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000251 2023-09-23T03:28:32.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.126+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:92 2023-09-23T03:28:32.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.126+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:279 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfe20700 con 0x55e8d0697000 2023-09-23T03:28:32.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.127+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 107 ==== osd_op_reply(279 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv103 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:32.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.127+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=92 frag=0*, r=0 2023-09-23T03:28:32.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.127+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:32.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.127+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:280 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfe380 con 0x55e8d0697000 2023-09-23T03:28:32.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.131+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 108 ==== osd_op_reply(280 10000000000.01000000 [omap-set-vals] v128'104 uv104 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:32.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.131+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/92 pointing to 0x0x10000000251 2023-09-23T03:28:32.132 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.131+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000252 2023-09-23T03:28:32.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.131+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:93 2023-09-23T03:28:32.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.131+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:281 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfe700 con 0x55e8cfd1cc00 2023-09-23T03:28:32.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.132+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 99 ==== osd_op_reply(281 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv100 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:32.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.132+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=93 frag=1*, r=0 2023-09-23T03:28:32.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.132+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:32.134 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.132+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:282 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf8e00 con 0x55e8cfd1cc00 2023-09-23T03:28:32.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.136+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 100 ==== osd_op_reply(282 10000000000.01800000 [omap-set-vals] v128'101 uv101 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:32.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.137+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/93 pointing to 0x0x10000000252 2023-09-23T03:28:32.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.137+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000253 2023-09-23T03:28:32.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.137+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:94 2023-09-23T03:28:32.138 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.137+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:283 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf8380 con 0x55e8cfd1cc00 2023-09-23T03:28:32.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.138+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 101 ==== osd_op_reply(283 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv101 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:32.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.138+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=94 frag=1*, r=0 2023-09-23T03:28:32.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.138+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:32.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.138+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:284 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf8700 con 0x55e8cfd1cc00 2023-09-23T03:28:32.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.143+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 102 ==== osd_op_reply(284 10000000000.01800000 [omap-set-vals] v128'102 uv102 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:32.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.143+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/94 pointing to 0x0x10000000253 2023-09-23T03:28:32.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.143+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000254 2023-09-23T03:28:32.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.143+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:95 2023-09-23T03:28:32.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.143+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:285 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf9500 con 0x55e8d0697000 2023-09-23T03:28:32.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.144+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 109 ==== osd_op_reply(285 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv104 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:32.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.144+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=95 frag=0*, r=0 2023-09-23T03:28:32.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.144+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:32.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.144+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:286 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdf9880 con 0x55e8d0697000 2023-09-23T03:28:32.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.148+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 110 ==== osd_op_reply(286 10000000000.01000000 [omap-set-vals] v128'105 uv105 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:32.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.148+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/95 pointing to 0x0x10000000254 2023-09-23T03:28:32.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.148+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000255 2023-09-23T03:28:32.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.148+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:96 2023-09-23T03:28:32.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.148+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:287 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfe21500 con 0x55e8d0697000 2023-09-23T03:28:32.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.149+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 111 ==== osd_op_reply(287 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv105 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:32.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.149+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=96 frag=0*, r=0 2023-09-23T03:28:32.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.149+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:32.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.150+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:288 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfe21c00 con 0x55e8d0697000 2023-09-23T03:28:32.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.153+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 112 ==== osd_op_reply(288 10000000000.01000000 [omap-set-vals] v128'106 uv106 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:32.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.153+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/96 pointing to 0x0x10000000255 2023-09-23T03:28:32.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.153+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000256 2023-09-23T03:28:32.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.153+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:97 2023-09-23T03:28:32.155 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.153+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:289 17.5 17:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdff180 con 0x55e8d0697000 2023-09-23T03:28:32.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.155+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 113 ==== osd_op_reply(289 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv106 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:32.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.155+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=97 frag=0*, r=0 2023-09-23T03:28:32.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.155+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:32.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.155+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:290 17.5 17:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdff500 con 0x55e8d0697000 2023-09-23T03:28:32.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.158+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 <== osd.6 v2:172.21.15.190:6816/1964294001 114 ==== osd_op_reply(290 10000000000.01000000 [omap-set-vals] v128'107 uv107 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766fc0 con 0x55e8d0697000 2023-09-23T03:28:32.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.158+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/97 pointing to 0x0x10000000256 2023-09-23T03:28:32.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.158+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000257 2023-09-23T03:28:32.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.158+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:98 2023-09-23T03:28:32.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.158+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:291 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdff880 con 0x55e8cfd1cc00 2023-09-23T03:28:32.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.159+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 103 ==== osd_op_reply(291 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv102 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:32.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.160+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=98 frag=1*, r=0 2023-09-23T03:28:32.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.160+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:32.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.160+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:292 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfea80 con 0x55e8cfd1cc00 2023-09-23T03:28:32.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.164+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 104 ==== osd_op_reply(292 10000000000.01800000 [omap-set-vals] v128'103 uv103 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:32.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.164+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/98 pointing to 0x0x10000000257 2023-09-23T03:28:32.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.164+0000 7f50a463f5c0 10 datascan.scan_links: handling injected_ino 0x10000000258 2023-09-23T03:28:32.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.164+0000 7f50a463f5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:99 2023-09-23T03:28:32.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.164+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:293 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdffc00 con 0x55e8cfd1cc00 2023-09-23T03:28:32.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.165+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 105 ==== osd_op_reply(293 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv103 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:32.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.165+0000 7f50a463f5c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=99 frag=1*, r=0 2023-09-23T03:28:32.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.165+0000 7f50a463f5c0 20 datascan.scan_links: first is now 2 2023-09-23T03:28:32.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.165+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- osd_op(unknown.0.0:294 17.1b 17:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfe000 con 0x55e8cfd1cc00 2023-09-23T03:28:32.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.169+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.4 v2:172.21.15.190:6808/717302641 106 ==== osd_op_reply(294 10000000000.01800000 [omap-set-vals] v128'104 uv104 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1cc00 2023-09-23T03:28:32.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.169+0000 7f50a463f5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/99 pointing to 0x0x10000000258 2023-09-23T03:28:32.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.169+0000 7f50a463f5c0 10 datascan.scan_links: updating inotable 2023-09-23T03:28:32.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.169+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:295 17.13 17:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8cfd1c000 2023-09-23T03:28:32.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.170+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.3 v2:172.21.15.154:6808/1133688468 27 ==== osd_op_reply(295 mds0_inotable [read 0~50 out=50b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+50 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1c000 2023-09-23T03:28:32.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.170+0000 7f50a463f5c0 10 mds.0.inotable: skip_inos was [0x10000000002~0x1f4,0x10000000259~0xfffffffda7] 2023-09-23T03:28:32.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.170+0000 7f50a463f5c0 10 mds.0.inotable: skip_inos now [0x10000000259~0xfffffffda7] 2023-09-23T03:28:32.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.170+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- osd_op(unknown.0.0:296 17.13 17:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cf91f880 con 0x55e8cfd1c000 2023-09-23T03:28:32.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.174+0000 7f50a2e48640 1 -- 172.21.15.154:0/26935237 <== osd.3 v2:172.21.15.154:6808/1133688468 28 ==== osd_op_reply(296 mds0_inotable [writefull 0~34] v128'4 uv4 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x55e8d0766d80 con 0x55e8cfd1c000 2023-09-23T03:28:32.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.174+0000 7f50a463f5c0 10 datascan.scan_links: updating snaptable 2023-09-23T03:28:32.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.174+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- osd_op(unknown.0.0:297 17.d 17:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x55e8cfdfee00 con 0x55e8cfc34c00 2023-09-23T03:28:32.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.176+0000 7f50a1e46640 1 -- 172.21.15.154:0/26935237 <== osd.7 v2:172.21.15.190:6824/3460136272 15 ==== osd_op_reply(297 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x55e8d0766b40 con 0x55e8cfc34c00 2023-09-23T03:28:32.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.176+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55e8cfd1c400 msgr2=0x55e8cfd1e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:32.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.176+0000 7f50a463f5c0 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x55e8cfd1c400 0x55e8cfd1e580 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:32.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.176+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55e8cfd1c800 msgr2=0x55e8cfd1eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:32.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.176+0000 7f50a463f5c0 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x55e8cfd1c800 0x55e8cfd1eb00 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:32.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.176+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55e8cfd1d000 msgr2=0x55e8cfd1f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:32.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.176+0000 7f50a463f5c0 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x55e8cfd1d000 0x55e8cfd1f600 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:32.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.176+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55e8cfd1c000 msgr2=0x55e8cfd1e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:32.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.176+0000 7f50a463f5c0 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x55e8cfd1c000 0x55e8cfd1e000 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:32.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.176+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55e8cfd1cc00 msgr2=0x55e8cfd1f080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:32.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.176+0000 7f50a463f5c0 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x55e8cfd1cc00 0x55e8cfd1f080 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:32.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.176+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55e8d0697400 msgr2=0x55e8cfd20680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:32.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.176+0000 7f50a463f5c0 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x55e8d0697400 0x55e8cfd20680 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:32.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.176+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55e8d0697000 msgr2=0x55e8cfd20100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:32.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.176+0000 7f50a463f5c0 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x55e8d0697000 0x55e8cfd20100 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:32.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.176+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55e8cfc34c00 msgr2=0x55e8cf930100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:32.179 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.176+0000 7f50a463f5c0 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x55e8cfc34c00 0x55e8cf930100 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:32.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.176+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 reap_dead start 2023-09-23T03:28:32.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.177+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 reap_dead start 2023-09-23T03:28:32.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.177+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55e8cfc34800 msgr2=0x55e8cf931700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:32.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.177+0000 7f50a463f5c0 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x55e8cfc34800 0x55e8cf931700 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x55e8d06caa80 tx=0x55e8cfba60c0 comp rx=0 tx=0).stop 2023-09-23T03:28:32.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.177+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8cfbb7000 msgr2=0x55e8cf931180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:32.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.177+0000 7f50a463f5c0 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8cfbb7000 0x55e8cf931180 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x55e8cfbc9440 tx=0x55e8cfbcccc0 comp rx=0 tx=0).stop 2023-09-23T03:28:32.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.177+0000 7f50a2647640 1 -- 172.21.15.154:0/26935237 reap_dead start 2023-09-23T03:28:32.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.177+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 shutdown_connections 2023-09-23T03:28:32.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.177+0000 7f50a463f5c0 1 --2- 172.21.15.154:0/26935237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8cfbb7000 0x55e8cf931180 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:32.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.177+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 >> 172.21.15.154:0/26935237 conn(0x55e8cfbb6c00 msgr2=0x55e8d063f000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:32.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.177+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 shutdown_connections 2023-09-23T03:28:32.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.177+0000 7f50a463f5c0 1 -- 172.21.15.154:0/26935237 wait complete. 2023-09-23T03:28:32.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.178+0000 7f50a463f5c0 1 -- 172.21.15.154:0/3946230653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8cf8c5c00 msgr2=0x55e8cf92f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:32.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.178+0000 7f50a463f5c0 1 --2- 172.21.15.154:0/3946230653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8cf8c5c00 0x55e8cf92f080 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x55e8d06cb3e0 tx=0x55e8cfba6420 comp rx=0 tx=0).stop 2023-09-23T03:28:32.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.179+0000 7f50a463f5c0 1 -- 172.21.15.154:0/3946230653 shutdown_connections 2023-09-23T03:28:32.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.179+0000 7f50a463f5c0 1 --2- 172.21.15.154:0/3946230653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e8cf8c5c00 0x55e8cf92f080 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:32.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.179+0000 7f50a463f5c0 1 --2- 172.21.15.154:0/3946230653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x55e8cf8c5800 0x55e8cf92eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:32.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.179+0000 7f50a463f5c0 1 --2- 172.21.15.154:0/3946230653 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x55e8cf8c4c00 0x55e8cf92e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:32.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.179+0000 7f50a463f5c0 1 -- 172.21.15.154:0/3946230653 >> 172.21.15.154:0/3946230653 conn(0x55e8cf8c5000 msgr2=0x55e8d063e800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:32.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.179+0000 7f50a463f5c0 1 -- 172.21.15.154:0/3946230653 shutdown_connections 2023-09-23T03:28:32.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.179+0000 7f50a463f5c0 1 -- 172.21.15.154:0/3946230653 wait complete. 2023-09-23T03:28:32.189 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.797297, result: 2023-09-23T03:28:32.189 DEBUG:teuthology.orchestra.run.smithi154:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 10000000000.01000000 2023-09-23T03:28:32.346 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.345+0000 7f74b7036f40 1 Processor -- start 2023-09-23T03:28:32.347 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.345+0000 7f74b7036f40 1 -- start start 2023-09-23T03:28:32.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.346+0000 7f74b7036f40 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648b025e880 0x5648b0291c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:32.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.346+0000 7f74b7036f40 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5648b02ac6f0 0x5648b0330d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:32.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.346+0000 7f74b7036f40 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5648b0335360 0x5648b02ad360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:32.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.346+0000 7f74b7036f40 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648b020f170 con 0x5648b0335360 2023-09-23T03:28:32.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.346+0000 7f74b7036f40 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648b01aa800 con 0x5648b025e880 2023-09-23T03:28:32.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.346+0000 7f74b7036f40 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5648b01aa4a0 con 0x5648b02ac6f0 2023-09-23T03:28:32.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.346+0000 7f74b4b4b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5648b0335360 0x5648b02ad360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:32.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.346+0000 7f74b4b4b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5648b0335360 0x5648b02ad360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53866/0 (socket says 172.21.15.154:53866) 2023-09-23T03:28:32.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.346+0000 7f74b4b4b640 1 -- 172.21.15.154:0/2556931024 learned_addr learned my addr 172.21.15.154:0/2556931024 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:32.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.347+0000 7f74b434a640 1 --2- 172.21.15.154:0/2556931024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648b025e880 0x5648b0291c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:32.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.347+0000 7f74b4b4b640 1 -- 172.21.15.154:0/2556931024 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5648b02ac6f0 msgr2=0x5648b0330d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:32.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.347+0000 7f74b3b49640 1 --2- 172.21.15.154:0/2556931024 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5648b02ac6f0 0x5648b0330d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:32.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.347+0000 7f74b4b4b640 1 --2- 172.21.15.154:0/2556931024 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5648b02ac6f0 0x5648b0330d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:32.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.347+0000 7f74b4b4b640 1 -- 172.21.15.154:0/2556931024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648b025e880 msgr2=0x5648b0291c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:32.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.347+0000 7f74b4b4b640 1 --2- 172.21.15.154:0/2556931024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648b025e880 0x5648b0291c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:32.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.347+0000 7f74b4b4b640 1 -- 172.21.15.154:0/2556931024 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ac005d20 con 0x5648b0335360 2023-09-23T03:28:32.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.347+0000 7f74b3b49640 1 --2- 172.21.15.154:0/2556931024 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5648b02ac6f0 0x5648b0330d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:32.350 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.347+0000 7f74b434a640 1 --2- 172.21.15.154:0/2556931024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648b025e880 0x5648b0291c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:32.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.347+0000 7f74b4b4b640 1 --2- 172.21.15.154:0/2556931024 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5648b0335360 0x5648b02ad360 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f74ac055040 tx=0x7f74ac04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d92174c3626f3a39 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:32.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.348+0000 7f74b2b47640 1 -- 172.21.15.154:0/2556931024 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac0525b0 con 0x5648b0335360 2023-09-23T03:28:32.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.348+0000 7f74b2b47640 1 -- 172.21.15.154:0/2556931024 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74ac052710 con 0x5648b0335360 2023-09-23T03:28:32.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.348+0000 7f74b2b47640 1 -- 172.21.15.154:0/2556931024 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac052aa0 con 0x5648b0335360 2023-09-23T03:28:32.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.348+0000 7f74b7036f40 1 -- 172.21.15.154:0/2556931024 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5648b0335360 msgr2=0x5648b02ad360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:32.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.348+0000 7f74b7036f40 1 --2- 172.21.15.154:0/2556931024 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5648b0335360 0x5648b02ad360 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f74ac055040 tx=0x7f74ac04c000 comp rx=0 tx=0).stop 2023-09-23T03:28:32.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.348+0000 7f74b7036f40 1 -- 172.21.15.154:0/2556931024 shutdown_connections 2023-09-23T03:28:32.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.348+0000 7f74b7036f40 1 --2- 172.21.15.154:0/2556931024 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5648b0335360 0x5648b02ad360 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:32.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.348+0000 7f74b7036f40 1 --2- 172.21.15.154:0/2556931024 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5648b02ac6f0 0x5648b0330d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:32.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.348+0000 7f74b7036f40 1 --2- 172.21.15.154:0/2556931024 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648b025e880 0x5648b0291c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:32.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.348+0000 7f74b7036f40 1 -- 172.21.15.154:0/2556931024 >> 172.21.15.154:0/2556931024 conn(0x5648b028fc90 msgr2=0x5648b033a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:32.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.348+0000 7f74b7036f40 1 -- 172.21.15.154:0/2556931024 shutdown_connections 2023-09-23T03:28:32.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.348+0000 7f74b7036f40 1 -- 172.21.15.154:0/2556931024 wait complete. 2023-09-23T03:28:32.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.349+0000 7f74b7036f40 1 Processor -- start 2023-09-23T03:28:32.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.349+0000 7f74b7036f40 1 -- start start 2023-09-23T03:28:32.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.350+0000 7f74b7036f40 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648b025e880 0x5648b0331a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:32.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.350+0000 7f74b7036f40 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5648b02ac6f0 0x5648b03d2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:32.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.350+0000 7f74b7036f40 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5648b0335360 0x5648b03d3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:32.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.350+0000 7f74b7036f40 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648b019e4f0 con 0x5648b0335360 2023-09-23T03:28:32.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.350+0000 7f74b7036f40 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648b02ad720 con 0x5648b025e880 2023-09-23T03:28:32.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.350+0000 7f74b7036f40 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x5648b0291710 con 0x5648b02ac6f0 2023-09-23T03:28:32.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.351+0000 7f74b4b4b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5648b0335360 0x5648b03d3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:32.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.351+0000 7f74b4b4b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5648b0335360 0x5648b03d3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53870/0 (socket says 172.21.15.154:53870) 2023-09-23T03:28:32.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.351+0000 7f74b4b4b640 1 -- 172.21.15.154:0/3525141067 learned_addr learned my addr 172.21.15.154:0/3525141067 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:32.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.351+0000 7f74b3b49640 1 --2- 172.21.15.154:0/3525141067 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5648b02ac6f0 0x5648b03d2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:32.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.351+0000 7f74b434a640 1 --2- 172.21.15.154:0/3525141067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648b025e880 0x5648b0331a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:32.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.351+0000 7f74b4b4b640 1 -- 172.21.15.154:0/3525141067 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5648b02ac6f0 msgr2=0x5648b03d2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:32.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.351+0000 7f74b4b4b640 1 --2- 172.21.15.154:0/3525141067 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x5648b02ac6f0 0x5648b03d2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:32.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.351+0000 7f74b4b4b640 1 -- 172.21.15.154:0/3525141067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648b025e880 msgr2=0x5648b0331a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:32.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.351+0000 7f74b4b4b640 1 --2- 172.21.15.154:0/3525141067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648b025e880 0x5648b0331a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:32.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.351+0000 7f74b4b4b640 1 -- 172.21.15.154:0/3525141067 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648b01b6650 con 0x5648b0335360 2023-09-23T03:28:32.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.351+0000 7f74b434a640 1 --2- 172.21.15.154:0/3525141067 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x5648b025e880 0x5648b0331a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:32.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.351+0000 7f74b4b4b640 1 --2- 172.21.15.154:0/3525141067 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5648b0335360 0x5648b03d3470 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f74ac01d4c0 tx=0x7f74ac003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:32.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.352+0000 7f74b1b45640 1 -- 172.21.15.154:0/3525141067 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac05e650 con 0x5648b0335360 2023-09-23T03:28:32.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.352+0000 7f74b1b45640 1 -- 172.21.15.154:0/3525141067 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74ac05e7d0 con 0x5648b0335360 2023-09-23T03:28:32.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.352+0000 7f74b1b45640 1 -- 172.21.15.154:0/3525141067 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac05eb80 con 0x5648b0335360 2023-09-23T03:28:32.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.352+0000 7f74b7036f40 1 -- 172.21.15.154:0/3525141067 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5648b01a9800 con 0x5648b0335360 2023-09-23T03:28:32.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.352+0000 7f74b7036f40 1 -- 172.21.15.154:0/3525141067 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5648b01a9260 con 0x5648b0335360 2023-09-23T03:28:32.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.354+0000 7f74b1b45640 1 -- 172.21.15.154:0/3525141067 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f74ac05ece0 con 0x5648b0335360 2023-09-23T03:28:32.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.355+0000 7f74b1b45640 1 --2- 172.21.15.154:0/3525141067 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f74940698e0 0x7f749406bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:32.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.355+0000 7f74b1b45640 1 -- 172.21.15.154:0/3525141067 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f74ac0cb2c0 con 0x5648b0335360 2023-09-23T03:28:32.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.355+0000 7f74b434a640 1 --2- 172.21.15.154:0/3525141067 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f74940698e0 0x7f749406bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:32.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.355+0000 7f74b7036f40 1 --2- 172.21.15.154:0/3525141067 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x5648b0331f40 0x5648b03366f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:32.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.355+0000 7f74b7036f40 1 -- 172.21.15.154:0/3525141067 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- osd_op(unknown.0.0:1 17.5 17:a40620de:::10000000000.01000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x5648b025e220 con 0x5648b0331f40 2023-09-23T03:28:32.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.356+0000 7f74b434a640 1 --2- 172.21.15.154:0/3525141067 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f74940698e0 0x7f749406bda0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f74a4000f80 tx=0x7f74a4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:32.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.356+0000 7f74b3b49640 1 --2- 172.21.15.154:0/3525141067 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x5648b0331f40 0x5648b03366f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:32.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.357+0000 7f74b3b49640 1 --2- 172.21.15.154:0/3525141067 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x5648b0331f40 0x5648b03366f0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:32.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.358+0000 7f74b3b49640 1 -- 172.21.15.154:0/3525141067 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== osd_op_reply(1 10000000000.01000000 [omap-get-keys out=560b] v0'0 uv107 ondisk = 0) v8 ==== 164+0+560 (crc 0 0 0) 0x7f749c00e500 con 0x5648b0331f40 2023-09-23T03:28:32.360 INFO:teuthology.orchestra.run.smithi154.stdout:10_head 2023-09-23T03:28:32.360 INFO:teuthology.orchestra.run.smithi154.stdout:11_head 2023-09-23T03:28:32.360 INFO:teuthology.orchestra.run.smithi154.stdout:12_head 2023-09-23T03:28:32.360 INFO:teuthology.orchestra.run.smithi154.stdout:15_head 2023-09-23T03:28:32.360 INFO:teuthology.orchestra.run.smithi154.stdout:19_head 2023-09-23T03:28:32.360 INFO:teuthology.orchestra.run.smithi154.stdout:1_head 2023-09-23T03:28:32.361 INFO:teuthology.orchestra.run.smithi154.stdout:20_head 2023-09-23T03:28:32.361 INFO:teuthology.orchestra.run.smithi154.stdout:21_head 2023-09-23T03:28:32.361 INFO:teuthology.orchestra.run.smithi154.stdout:27_head 2023-09-23T03:28:32.361 INFO:teuthology.orchestra.run.smithi154.stdout:28_head 2023-09-23T03:28:32.361 INFO:teuthology.orchestra.run.smithi154.stdout:2_head 2023-09-23T03:28:32.361 INFO:teuthology.orchestra.run.smithi154.stdout:31_head 2023-09-23T03:28:32.361 INFO:teuthology.orchestra.run.smithi154.stdout:34_head 2023-09-23T03:28:32.361 INFO:teuthology.orchestra.run.smithi154.stdout:36_head 2023-09-23T03:28:32.361 INFO:teuthology.orchestra.run.smithi154.stdout:38_head 2023-09-23T03:28:32.361 INFO:teuthology.orchestra.run.smithi154.stdout:39_head 2023-09-23T03:28:32.361 INFO:teuthology.orchestra.run.smithi154.stdout:3_head 2023-09-23T03:28:32.361 INFO:teuthology.orchestra.run.smithi154.stdout:42_head 2023-09-23T03:28:32.361 INFO:teuthology.orchestra.run.smithi154.stdout:43_head 2023-09-23T03:28:32.362 INFO:teuthology.orchestra.run.smithi154.stdout:46_head 2023-09-23T03:28:32.362 INFO:teuthology.orchestra.run.smithi154.stdout:4_head 2023-09-23T03:28:32.362 INFO:teuthology.orchestra.run.smithi154.stdout:50_head 2023-09-23T03:28:32.362 INFO:teuthology.orchestra.run.smithi154.stdout:52_head 2023-09-23T03:28:32.362 INFO:teuthology.orchestra.run.smithi154.stdout:53_head 2023-09-23T03:28:32.362 INFO:teuthology.orchestra.run.smithi154.stdout:55_head 2023-09-23T03:28:32.362 INFO:teuthology.orchestra.run.smithi154.stdout:59_head 2023-09-23T03:28:32.362 INFO:teuthology.orchestra.run.smithi154.stdout:61_head 2023-09-23T03:28:32.362 INFO:teuthology.orchestra.run.smithi154.stdout:63_head 2023-09-23T03:28:32.362 INFO:teuthology.orchestra.run.smithi154.stdout:64_head 2023-09-23T03:28:32.362 INFO:teuthology.orchestra.run.smithi154.stdout:65_head 2023-09-23T03:28:32.362 INFO:teuthology.orchestra.run.smithi154.stdout:66_head 2023-09-23T03:28:32.363 INFO:teuthology.orchestra.run.smithi154.stdout:69_head 2023-09-23T03:28:32.363 INFO:teuthology.orchestra.run.smithi154.stdout:70_head 2023-09-23T03:28:32.363 INFO:teuthology.orchestra.run.smithi154.stdout:73_head 2023-09-23T03:28:32.363 INFO:teuthology.orchestra.run.smithi154.stdout:74_head 2023-09-23T03:28:32.363 INFO:teuthology.orchestra.run.smithi154.stdout:75_head 2023-09-23T03:28:32.363 INFO:teuthology.orchestra.run.smithi154.stdout:77_head 2023-09-23T03:28:32.363 INFO:teuthology.orchestra.run.smithi154.stdout:80_head 2023-09-23T03:28:32.363 INFO:teuthology.orchestra.run.smithi154.stdout:83_head 2023-09-23T03:28:32.363 INFO:teuthology.orchestra.run.smithi154.stdout:84_head 2023-09-23T03:28:32.363 INFO:teuthology.orchestra.run.smithi154.stdout:85_head 2023-09-23T03:28:32.363 INFO:teuthology.orchestra.run.smithi154.stdout:86_head 2023-09-23T03:28:32.364 INFO:teuthology.orchestra.run.smithi154.stdout:87_head 2023-09-23T03:28:32.364 INFO:teuthology.orchestra.run.smithi154.stdout:88_head 2023-09-23T03:28:32.364 INFO:teuthology.orchestra.run.smithi154.stdout:8_head 2023-09-23T03:28:32.364 INFO:teuthology.orchestra.run.smithi154.stdout:90_head 2023-09-23T03:28:32.364 INFO:teuthology.orchestra.run.smithi154.stdout:92_head 2023-09-23T03:28:32.364 INFO:teuthology.orchestra.run.smithi154.stdout:95_head 2023-09-23T03:28:32.364 INFO:teuthology.orchestra.run.smithi154.stdout:96_head 2023-09-23T03:28:32.364 INFO:teuthology.orchestra.run.smithi154.stdout:97_head 2023-09-23T03:28:32.364 INFO:teuthology.orchestra.run.smithi154.stdout:9_head 2023-09-23T03:28:32.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.358+0000 7f74b7036f40 1 -- 172.21.15.154:0/3525141067 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x5648b0331f40 msgr2=0x5648b03366f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:32.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.358+0000 7f74b7036f40 1 --2- 172.21.15.154:0/3525141067 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x5648b0331f40 0x5648b03366f0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:32.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.358+0000 7f74b7036f40 1 -- 172.21.15.154:0/3525141067 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f74940698e0 msgr2=0x7f749406bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:32.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.358+0000 7f74b7036f40 1 --2- 172.21.15.154:0/3525141067 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f74940698e0 0x7f749406bda0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f74a4000f80 tx=0x7f74a4008040 comp rx=0 tx=0).stop 2023-09-23T03:28:32.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.358+0000 7f74b7036f40 1 -- 172.21.15.154:0/3525141067 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5648b0335360 msgr2=0x5648b03d3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:32.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.358+0000 7f74b7036f40 1 --2- 172.21.15.154:0/3525141067 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5648b0335360 0x5648b03d3470 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f74ac01d4c0 tx=0x7f74ac003020 comp rx=0 tx=0).stop 2023-09-23T03:28:32.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.358+0000 7f74b4b4b640 1 -- 172.21.15.154:0/3525141067 reap_dead start 2023-09-23T03:28:32.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.358+0000 7f74b7036f40 1 -- 172.21.15.154:0/3525141067 shutdown_connections 2023-09-23T03:28:32.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.358+0000 7f74b7036f40 1 -- 172.21.15.154:0/3525141067 >> 172.21.15.154:0/3525141067 conn(0x5648b028fc90 msgr2=0x5648b0291c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:32.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.359+0000 7f74b7036f40 1 -- 172.21.15.154:0/3525141067 shutdown_connections 2023-09-23T03:28:32.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.359+0000 7f74b7036f40 1 -- 172.21.15.154:0/3525141067 wait complete. 2023-09-23T03:28:32.367 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-09-23T03:28:32.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.585+0000 7fd137834640 1 Processor -- start 2023-09-23T03:28:32.586 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.585+0000 7fd137834640 1 -- start start 2023-09-23T03:28:32.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.587+0000 7fd137834640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd1300707f0 0x7fd13006db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:32.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.587+0000 7fd137834640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1300cb410 con 0x7fd1300cb8a0 2023-09-23T03:28:32.588 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.587+0000 7fd137834640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1300b4330 con 0x7fd1300b3f50 2023-09-23T03:28:32.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.587+0000 7fd137834640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1300b0d30 con 0x7fd1300707f0 2023-09-23T03:28:32.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.587+0000 7fd134dcf640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd1300707f0 0x7fd13006db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:32.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.587+0000 7fd135dd1640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fd1300cb8a0 0x7fd13006c1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:49896/0 (socket says 172.21.15.154:49896) 2023-09-23T03:28:32.589 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.587+0000 7fd135dd1640 1 -- 172.21.15.154:0/428459636 learned_addr learned my addr 172.21.15.154:0/428459636 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:28:32.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.588+0000 7fd134dcf640 1 -- 172.21.15.154:0/428459636 >> v1:172.21.15.190:6789/0 conn(0x7fd1300b3f50 legacy=0x7fd1300700e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:32.590 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.588+0000 7fd134dcf640 1 -- 172.21.15.154:0/428459636 >> v1:172.21.15.154:6789/0 conn(0x7fd1300cb8a0 legacy=0x7fd13006c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:32.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.588+0000 7fd134dcf640 1 -- 172.21.15.154:0/428459636 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1300c8ca0 con 0x7fd1300707f0 2023-09-23T03:28:32.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.589+0000 7fd134dcf640 1 --2- 172.21.15.154:0/428459636 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1300707f0 0x7fd13006db50 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fd1200097a0 tx=0x7fd12002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73060c79582385da server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:32.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.589+0000 7fd127fff640 1 -- 172.21.15.154:0/428459636 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12002e070 con 0x7fd1300707f0 2023-09-23T03:28:32.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.589+0000 7fd127fff640 1 -- 172.21.15.154:0/428459636 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1200315a0 con 0x7fd1300707f0 2023-09-23T03:28:32.591 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.589+0000 7fd127fff640 1 -- 172.21.15.154:0/428459636 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd120031930 con 0x7fd1300707f0 2023-09-23T03:28:32.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.590+0000 7fd137834640 1 -- 172.21.15.154:0/428459636 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1300707f0 msgr2=0x7fd13006db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:32.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.590+0000 7fd137834640 1 --2- 172.21.15.154:0/428459636 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1300707f0 0x7fd13006db50 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fd1200097a0 tx=0x7fd12002b840 comp rx=0 tx=0).stop 2023-09-23T03:28:32.592 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.590+0000 7fd137834640 1 -- 172.21.15.154:0/428459636 shutdown_connections 2023-09-23T03:28:32.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.590+0000 7fd137834640 1 --2- 172.21.15.154:0/428459636 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1300707f0 0x7fd13006db50 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:32.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.590+0000 7fd137834640 1 -- 172.21.15.154:0/428459636 >> 172.21.15.154:0/428459636 conn(0x7fd13006a440 msgr2=0x7fd1300b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:32.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.590+0000 7fd137834640 1 -- 172.21.15.154:0/428459636 shutdown_connections 2023-09-23T03:28:32.593 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.590+0000 7fd137834640 1 -- 172.21.15.154:0/428459636 wait complete. 2023-09-23T03:28:32.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.591+0000 7fd137834640 1 Processor -- start 2023-09-23T03:28:32.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.592+0000 7fd137834640 1 -- start start 2023-09-23T03:28:32.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.593+0000 7fd137834640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1300707f0 0x7fd13013f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:32.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.593+0000 7fd137834640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1300b3f50 0x7fd13006c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:32.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.593+0000 7fd137834640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1300cb8a0 0x7fd13006c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:32.596 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.593+0000 7fd137834640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd130141120 con 0x7fd1300cb8a0 2023-09-23T03:28:32.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.593+0000 7fd137834640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd130141400 con 0x7fd1300b3f50 2023-09-23T03:28:32.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.593+0000 7fd137834640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1301416e0 con 0x7fd1300707f0 2023-09-23T03:28:32.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.593+0000 7fd135dd1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1300cb8a0 0x7fd13006c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:32.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.593+0000 7fd135dd1640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1300cb8a0 0x7fd13006c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53888/0 (socket says 172.21.15.154:53888) 2023-09-23T03:28:32.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.593+0000 7fd135dd1640 1 -- 172.21.15.154:0/3712724670 learned_addr learned my addr 172.21.15.154:0/3712724670 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:32.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.593+0000 7fd134dcf640 1 --2- 172.21.15.154:0/3712724670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1300b3f50 0x7fd13006c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:32.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.593+0000 7fd1355d0640 1 --2- 172.21.15.154:0/3712724670 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1300707f0 0x7fd13013f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:32.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.593+0000 7fd135dd1640 1 -- 172.21.15.154:0/3712724670 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1300707f0 msgr2=0x7fd13013f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:32.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.593+0000 7fd135dd1640 1 --2- 172.21.15.154:0/3712724670 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1300707f0 0x7fd13013f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:32.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.593+0000 7fd135dd1640 1 -- 172.21.15.154:0/3712724670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1300b3f50 msgr2=0x7fd13006c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:32.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.593+0000 7fd135dd1640 1 --2- 172.21.15.154:0/3712724670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1300b3f50 0x7fd13006c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:32.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.593+0000 7fd135dd1640 1 -- 172.21.15.154:0/3712724670 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd120009400 con 0x7fd1300cb8a0 2023-09-23T03:28:32.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.594+0000 7fd135dd1640 1 --2- 172.21.15.154:0/3712724670 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1300cb8a0 0x7fd13006c6f0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fd12800ca80 tx=0x7fd12800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:32.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.594+0000 7fd125ffb640 1 -- 172.21.15.154:0/3712724670 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12800ac30 con 0x7fd1300cb8a0 2023-09-23T03:28:32.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.594+0000 7fd125ffb640 1 -- 172.21.15.154:0/3712724670 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd12800ad90 con 0x7fd1300cb8a0 2023-09-23T03:28:32.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.594+0000 7fd137834640 1 -- 172.21.15.154:0/3712724670 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1301419c0 con 0x7fd1300cb8a0 2023-09-23T03:28:32.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.594+0000 7fd125ffb640 1 -- 172.21.15.154:0/3712724670 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd128012600 con 0x7fd1300cb8a0 2023-09-23T03:28:32.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.594+0000 7fd137834640 1 -- 172.21.15.154:0/3712724670 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1301403c0 con 0x7fd1300cb8a0 2023-09-23T03:28:32.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.595+0000 7fd137834640 1 -- 172.21.15.154:0/3712724670 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd130073eb0 con 0x7fd1300cb8a0 2023-09-23T03:28:32.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.596+0000 7fd125ffb640 1 -- 172.21.15.154:0/3712724670 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fd128007070 con 0x7fd1300cb8a0 2023-09-23T03:28:32.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.597+0000 7fd125ffb640 1 --2- 172.21.15.154:0/3712724670 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd11406af60 0x7fd11406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:32.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.597+0000 7fd125ffb640 1 -- 172.21.15.154:0/3712724670 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6201+0+0 (secure 0 0 0) 0x7fd1280896f0 con 0x7fd1300cb8a0 2023-09-23T03:28:32.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.597+0000 7fd1355d0640 1 --2- 172.21.15.154:0/3712724670 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd11406af60 0x7fd11406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:32.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.598+0000 7fd1355d0640 1 --2- 172.21.15.154:0/3712724670 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd11406af60 0x7fd11406d420 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fd12c003d90 tx=0x7fd12c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:32.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.602+0000 7fd125ffb640 1 -- 172.21.15.154:0/3712724670 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fd1280545d0 con 0x7fd1300cb8a0 2023-09-23T03:28:32.828 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:32.827+0000 7fd137834640 1 -- 172.21.15.154:0/3712724670 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fd1300cbda0 con 0x7fd1300cb8a0 2023-09-23T03:28:33.444 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.443+0000 7fd125ffb640 1 -- 172.21.15.154:0/3712724670 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v117) v1 ==== 162+0+0 (secure 0 0 0) 0x7fd128057bf0 con 0x7fd1300cb8a0 2023-09-23T03:28:33.445 INFO:teuthology.orchestra.run.smithi154.stderr:cephfs marked joinable; MDS may join as newly active. 2023-09-23T03:28:33.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.447+0000 7fd137834640 1 -- 172.21.15.154:0/3712724670 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd11406af60 msgr2=0x7fd11406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:33.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.447+0000 7fd137834640 1 --2- 172.21.15.154:0/3712724670 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd11406af60 0x7fd11406d420 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fd12c003d90 tx=0x7fd12c009040 comp rx=0 tx=0).stop 2023-09-23T03:28:33.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.447+0000 7fd137834640 1 -- 172.21.15.154:0/3712724670 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1300cb8a0 msgr2=0x7fd13006c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:33.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.447+0000 7fd137834640 1 --2- 172.21.15.154:0/3712724670 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1300cb8a0 0x7fd13006c6f0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fd12800ca80 tx=0x7fd12800cf40 comp rx=0 tx=0).stop 2023-09-23T03:28:33.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.447+0000 7fd137834640 1 -- 172.21.15.154:0/3712724670 shutdown_connections 2023-09-23T03:28:33.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.447+0000 7fd137834640 1 --2- 172.21.15.154:0/3712724670 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd11406af60 0x7fd11406d420 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:33.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.447+0000 7fd137834640 1 --2- 172.21.15.154:0/3712724670 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1300cb8a0 0x7fd13006c6f0 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:33.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.447+0000 7fd137834640 1 --2- 172.21.15.154:0/3712724670 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd1300b3f50 0x7fd13006c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:33.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.447+0000 7fd137834640 1 --2- 172.21.15.154:0/3712724670 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1300707f0 0x7fd13013f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:33.449 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.447+0000 7fd137834640 1 -- 172.21.15.154:0/3712724670 >> 172.21.15.154:0/3712724670 conn(0x7fd13006a440 msgr2=0x7fd130149550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:33.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.447+0000 7fd137834640 1 -- 172.21.15.154:0/3712724670 shutdown_connections 2023-09-23T03:28:33.450 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.447+0000 7fd137834640 1 -- 172.21.15.154:0/3712724670 wait complete. 2023-09-23T03:28:33.472 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:28:33.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.663+0000 7f48349e2640 1 Processor -- start 2023-09-23T03:28:33.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.663+0000 7f48349e2640 1 -- start start 2023-09-23T03:28:33.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.665+0000 7f48349e2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f48300b0780 0x7f483006cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:33.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.665+0000 7f48349e2640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f483006c8b0 con 0x7f48300cc1c0 2023-09-23T03:28:33.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.665+0000 7f48349e2640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4830165450 con 0x7f4830161540 2023-09-23T03:28:33.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.665+0000 7f48349e2640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4830165730 con 0x7f48300b0780 2023-09-23T03:28:33.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.665+0000 7f482dd9b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f48300b0780 0x7f483006cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:33.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.665+0000 7f482ed9d640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f48300cc1c0 0x7f483006d2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:49906/0 (socket says 172.21.15.154:49906) 2023-09-23T03:28:33.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.665+0000 7f482ed9d640 1 -- 172.21.15.154:0/83950436 learned_addr learned my addr 172.21.15.154:0/83950436 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:28:33.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.665+0000 7f482dd9b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f48300b0780 0x7f483006cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53898/0 (socket says 172.21.15.154:53898) 2023-09-23T03:28:33.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.666+0000 7f482dd9b640 1 -- 172.21.15.154:0/83950436 >> v1:172.21.15.190:6789/0 conn(0x7f4830161540 legacy=0x7f48300b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:33.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.666+0000 7f482dd9b640 1 -- 172.21.15.154:0/83950436 >> v1:172.21.15.154:6789/0 conn(0x7f48300cc1c0 legacy=0x7f483006d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:33.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.666+0000 7f482dd9b640 1 -- 172.21.15.154:0/83950436 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4830165a10 con 0x7f48300b0780 2023-09-23T03:28:33.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.667+0000 7f482dd9b640 1 --2- 172.21.15.154:0/83950436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48300b0780 0x7f483006cd60 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f48240097a0 tx=0x7f482402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=932b8ec9d4ab06e5 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:33.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.667+0000 7f482d59a640 1 -- 172.21.15.154:0/83950436 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4824031830 con 0x7f48300b0780 2023-09-23T03:28:33.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.667+0000 7f482d59a640 1 -- 172.21.15.154:0/83950436 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4824031990 con 0x7f48300b0780 2023-09-23T03:28:33.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.667+0000 7f482d59a640 1 -- 172.21.15.154:0/83950436 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4824031d20 con 0x7f48300b0780 2023-09-23T03:28:33.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.668+0000 7f48349e2640 1 -- 172.21.15.154:0/83950436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48300b0780 msgr2=0x7f483006cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:33.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.668+0000 7f48349e2640 1 --2- 172.21.15.154:0/83950436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48300b0780 0x7f483006cd60 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f48240097a0 tx=0x7f482402b8d0 comp rx=0 tx=0).stop 2023-09-23T03:28:33.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.668+0000 7f48349e2640 1 -- 172.21.15.154:0/83950436 shutdown_connections 2023-09-23T03:28:33.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.668+0000 7f48349e2640 1 --2- 172.21.15.154:0/83950436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48300b0780 0x7f483006cd60 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:33.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.668+0000 7f48349e2640 1 -- 172.21.15.154:0/83950436 >> 172.21.15.154:0/83950436 conn(0x7f48300cb070 msgr2=0x7f48300cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:33.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.668+0000 7f48349e2640 1 -- 172.21.15.154:0/83950436 shutdown_connections 2023-09-23T03:28:33.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.668+0000 7f48349e2640 1 -- 172.21.15.154:0/83950436 wait complete. 2023-09-23T03:28:33.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.670+0000 7f48349e2640 1 Processor -- start 2023-09-23T03:28:33.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.670+0000 7f48349e2640 1 -- start start 2023-09-23T03:28:33.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.671+0000 7f48349e2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48300b0780 0x7f483015cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:33.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.671+0000 7f48349e2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48300cc1c0 0x7f4830161c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:33.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.671+0000 7f48349e2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4830161540 0x7f48300c4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:33.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.671+0000 7f48349e2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48300b2450 con 0x7f4830161540 2023-09-23T03:28:33.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.671+0000 7f48349e2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483015d100 con 0x7f48300cc1c0 2023-09-23T03:28:33.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.671+0000 7f48349e2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48300c2be0 con 0x7f48300b0780 2023-09-23T03:28:33.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.671+0000 7f482ed9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4830161540 0x7f48300c4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:33.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.671+0000 7f482ed9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4830161540 0x7f48300c4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:53908/0 (socket says 172.21.15.154:53908) 2023-09-23T03:28:33.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.671+0000 7f482ed9d640 1 -- 172.21.15.154:0/948641490 learned_addr learned my addr 172.21.15.154:0/948641490 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:33.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.672+0000 7f482dd9b640 1 --2- 172.21.15.154:0/948641490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48300cc1c0 0x7f4830161c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:33.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.672+0000 7f482e59c640 1 --2- 172.21.15.154:0/948641490 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48300b0780 0x7f483015cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:33.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.672+0000 7f482ed9d640 1 -- 172.21.15.154:0/948641490 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48300b0780 msgr2=0x7f483015cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:33.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.672+0000 7f482ed9d640 1 --2- 172.21.15.154:0/948641490 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48300b0780 0x7f483015cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:33.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.672+0000 7f482ed9d640 1 -- 172.21.15.154:0/948641490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48300cc1c0 msgr2=0x7f4830161c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:33.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.672+0000 7f482ed9d640 1 --2- 172.21.15.154:0/948641490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48300cc1c0 0x7f4830161c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:33.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.672+0000 7f482ed9d640 1 -- 172.21.15.154:0/948641490 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4824009400 con 0x7f4830161540 2023-09-23T03:28:33.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.672+0000 7f482e59c640 1 --2- 172.21.15.154:0/948641490 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48300b0780 0x7f483015cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:33.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.672+0000 7f482ed9d640 1 --2- 172.21.15.154:0/948641490 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4830161540 0x7f48300c4560 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f482800cbc0 tx=0x7f482800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:33.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.672+0000 7f481effd640 1 -- 172.21.15.154:0/948641490 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482800ad40 con 0x7f4830161540 2023-09-23T03:28:33.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.672+0000 7f481effd640 1 -- 172.21.15.154:0/948641490 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4828012400 con 0x7f4830161540 2023-09-23T03:28:33.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.672+0000 7f481effd640 1 -- 172.21.15.154:0/948641490 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482800aea0 con 0x7f4830161540 2023-09-23T03:28:33.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.673+0000 7f48349e2640 1 -- 172.21.15.154:0/948641490 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48300c1fd0 con 0x7f4830161540 2023-09-23T03:28:33.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.673+0000 7f48349e2640 1 -- 172.21.15.154:0/948641490 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48300c2590 con 0x7f4830161540 2023-09-23T03:28:33.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.674+0000 7f48349e2640 1 -- 172.21.15.154:0/948641490 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4830067870 con 0x7f4830161540 2023-09-23T03:28:33.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.675+0000 7f481effd640 1 -- 172.21.15.154:0/948641490 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f4828007070 con 0x7f4830161540 2023-09-23T03:28:33.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.676+0000 7f481effd640 1 --2- 172.21.15.154:0/948641490 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f481006af30 0x7f481006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:33.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.676+0000 7f481effd640 1 -- 172.21.15.154:0/948641490 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f4828089890 con 0x7f4830161540 2023-09-23T03:28:33.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.676+0000 7f482e59c640 1 --2- 172.21.15.154:0/948641490 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f481006af30 0x7f481006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:33.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.677+0000 7f482e59c640 1 --2- 172.21.15.154:0/948641490 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f481006af30 0x7f481006d3f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f4820000f60 tx=0x7f4820009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:33.681 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.680+0000 7f481effd640 1 -- 172.21.15.154:0/948641490 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f48280546f0 con 0x7f4830161540 2023-09-23T03:28:33.908 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.907+0000 7f48349e2640 1 -- 172.21.15.154:0/948641490 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4830165cd0 con 0x7f4830161540 2023-09-23T03:28:33.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.908+0000 7f481effd640 1 -- 172.21.15.154:0/948641490 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+4935 (secure 0 0 0) 0x7f4830165cd0 con 0x7f4830161540 2023-09-23T03:28:33.910 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:28:33.910 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":118,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/2759187683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2759187683},{"type":"v1","addr":"172.21.15.154:6835","nonce":2759187683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":5924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3712825179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3712825179},{"type":"v1","addr":"172.21.15.190:6835","nonce":3712825179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":110},{"gid":6544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/4147378242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":4147378242},{"type":"v1","addr":"172.21.15.190:6834","nonce":4147378242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:27:44.470312+0000","modified":"2023-09-23T03:28:33.448618+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9306":{"gid":9306,"name":"c","rank":0,"incarnation":118,"state":"up:replay","state_seq":1,"addr":"172.21.15.154:6837/1250699891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1250699891},{"type":"v1","addr":"172.21.15.154:6837","nonce":1250699891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-09-23T03:28:33.910 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 118 2023-09-23T03:28:33.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.911+0000 7f48349e2640 1 -- 172.21.15.154:0/948641490 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f481006af30 msgr2=0x7f481006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:33.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.911+0000 7f48349e2640 1 --2- 172.21.15.154:0/948641490 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f481006af30 0x7f481006d3f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f4820000f60 tx=0x7f4820009040 comp rx=0 tx=0).stop 2023-09-23T03:28:33.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.911+0000 7f48349e2640 1 -- 172.21.15.154:0/948641490 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4830161540 msgr2=0x7f48300c4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:33.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.911+0000 7f48349e2640 1 --2- 172.21.15.154:0/948641490 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4830161540 0x7f48300c4560 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f482800cbc0 tx=0x7f482800a430 comp rx=0 tx=0).stop 2023-09-23T03:28:33.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.911+0000 7f48349e2640 1 -- 172.21.15.154:0/948641490 shutdown_connections 2023-09-23T03:28:33.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.911+0000 7f48349e2640 1 --2- 172.21.15.154:0/948641490 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f481006af30 0x7f481006d3f0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:33.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.911+0000 7f48349e2640 1 --2- 172.21.15.154:0/948641490 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4830161540 0x7f48300c4560 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:33.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.911+0000 7f48349e2640 1 --2- 172.21.15.154:0/948641490 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f48300cc1c0 0x7f4830161c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:33.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.911+0000 7f48349e2640 1 --2- 172.21.15.154:0/948641490 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f48300b0780 0x7f483015cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:33.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.911+0000 7f48349e2640 1 -- 172.21.15.154:0/948641490 >> 172.21.15.154:0/948641490 conn(0x7f48300cb070 msgr2=0x7f48300cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:33.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.911+0000 7f48349e2640 1 -- 172.21.15.154:0/948641490 shutdown_connections 2023-09-23T03:28:33.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:33.911+0000 7f48349e2640 1 -- 172.21.15.154:0/948641490 wait complete. 2023-09-23T03:28:34.332 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 118, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-23T03:27:44.470312+0000', 'modified': '2023-09-23T03:28:33.448618+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 128, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9306}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9306': {'gid': 9306, 'name': 'c', 'rank': 0, 'incarnation': 118, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.154:6837/1250699891', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6836', 'nonce': 1250699891}, {'type': 'v1', 'addr': '172.21.15.154:6837', 'nonce': 1250699891}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-23T03:28:34.332 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9306:up:replay 2023-09-23T03:28:35.333 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:28:35.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.557+0000 7fd12f94c640 1 Processor -- start 2023-09-23T03:28:35.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.557+0000 7fd12f94c640 1 -- start start 2023-09-23T03:28:35.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.559+0000 7fd12f94c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd1280b0780 0x7fd1280c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:35.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.559+0000 7fd12f94c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd128070b40 con 0x7fd1280ad9c0 2023-09-23T03:28:35.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.559+0000 7fd12f94c640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd128070e20 con 0x7fd12815ad70 2023-09-23T03:28:35.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.559+0000 7fd12f94c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd128071100 con 0x7fd1280b0780 2023-09-23T03:28:35.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.559+0000 7fd12cee7640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fd1280ad9c0 0x7fd12806c8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:41394/0 (socket says 172.21.15.154:41394) 2023-09-23T03:28:35.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.559+0000 7fd12cee7640 1 -- 172.21.15.154:0/2135658585 learned_addr learned my addr 172.21.15.154:0/2135658585 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:28:35.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.559+0000 7fd12dee9640 1 --2- 172.21.15.154:0/2135658585 >> v2:172.21.15.154:3300/0 conn(0x7fd1280b0780 0x7fd1280c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:35.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.560+0000 7fd11ffff640 1 -- 172.21.15.154:0/2135658585 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2515600770 0 0) 0x7fd128070b40 con 0x7fd1280ad9c0 2023-09-23T03:28:35.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.560+0000 7fd11ffff640 1 -- 172.21.15.154:0/2135658585 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd114003520 con 0x7fd1280ad9c0 2023-09-23T03:28:35.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.560+0000 7fd12dee9640 1 -- 172.21.15.154:0/2135658585 >> v1:172.21.15.190:6789/0 conn(0x7fd12815ad70 legacy=0x7fd1280b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:35.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.560+0000 7fd12dee9640 1 -- 172.21.15.154:0/2135658585 >> v1:172.21.15.154:6789/0 conn(0x7fd1280ad9c0 legacy=0x7fd12806c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:35.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.560+0000 7fd12dee9640 1 -- 172.21.15.154:0/2135658585 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1280c96d0 con 0x7fd1280b0780 2023-09-23T03:28:35.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.561+0000 7fd12dee9640 1 --2- 172.21.15.154:0/2135658585 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1280b0780 0x7fd1280c8e50 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fd120009dc0 tx=0x7fd12002be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b46206a44dd4282 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:35.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.562+0000 7fd11ffff640 1 -- 172.21.15.154:0/2135658585 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12002cc40 con 0x7fd1280b0780 2023-09-23T03:28:35.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.562+0000 7fd11ffff640 1 -- 172.21.15.154:0/2135658585 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd12002cda0 con 0x7fd1280b0780 2023-09-23T03:28:35.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.562+0000 7fd11ffff640 1 -- 172.21.15.154:0/2135658585 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd120034640 con 0x7fd1280b0780 2023-09-23T03:28:35.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.562+0000 7fd12f94c640 1 -- 172.21.15.154:0/2135658585 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1280b0780 msgr2=0x7fd1280c8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:35.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.562+0000 7fd12f94c640 1 --2- 172.21.15.154:0/2135658585 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1280b0780 0x7fd1280c8e50 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fd120009dc0 tx=0x7fd12002be90 comp rx=0 tx=0).stop 2023-09-23T03:28:35.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.562+0000 7fd12f94c640 1 -- 172.21.15.154:0/2135658585 shutdown_connections 2023-09-23T03:28:35.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.562+0000 7fd12f94c640 1 --2- 172.21.15.154:0/2135658585 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1280b0780 0x7fd1280c8e50 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:35.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.562+0000 7fd12f94c640 1 -- 172.21.15.154:0/2135658585 >> 172.21.15.154:0/2135658585 conn(0x7fd1280cb070 msgr2=0x7fd1280cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:35.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.562+0000 7fd12f94c640 1 -- 172.21.15.154:0/2135658585 shutdown_connections 2023-09-23T03:28:35.564 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.562+0000 7fd12f94c640 1 -- 172.21.15.154:0/2135658585 wait complete. 2023-09-23T03:28:35.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.564+0000 7fd12f94c640 1 Processor -- start 2023-09-23T03:28:35.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.564+0000 7fd12f94c640 1 -- start start 2023-09-23T03:28:35.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.565+0000 7fd12f94c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1280ad9c0 0x7fd1281edb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:35.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.565+0000 7fd12f94c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1280b0780 0x7fd1280c2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:35.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.565+0000 7fd12f94c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd12815ad70 0x7fd1280b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:35.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.565+0000 7fd12f94c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1280c32c0 con 0x7fd1280ad9c0 2023-09-23T03:28:35.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.565+0000 7fd12f94c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1280c2a20 con 0x7fd12815ad70 2023-09-23T03:28:35.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.565+0000 7fd12f94c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1280c2d00 con 0x7fd1280b0780 2023-09-23T03:28:35.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.566+0000 7fd12d6e8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1280ad9c0 0x7fd1281edb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:35.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.566+0000 7fd12d6e8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1280ad9c0 0x7fd1281edb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49020/0 (socket says 172.21.15.154:49020) 2023-09-23T03:28:35.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.566+0000 7fd12d6e8640 1 -- 172.21.15.154:0/3567959342 learned_addr learned my addr 172.21.15.154:0/3567959342 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:35.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.566+0000 7fd12dee9640 1 --2- 172.21.15.154:0/3567959342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd12815ad70 0x7fd1280b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:35.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.566+0000 7fd12cee7640 1 --2- 172.21.15.154:0/3567959342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1280b0780 0x7fd1280c2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:35.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.566+0000 7fd12d6e8640 1 -- 172.21.15.154:0/3567959342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1280b0780 msgr2=0x7fd1280c2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:35.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.566+0000 7fd12d6e8640 1 --2- 172.21.15.154:0/3567959342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1280b0780 0x7fd1280c2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:35.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.566+0000 7fd12d6e8640 1 -- 172.21.15.154:0/3567959342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd12815ad70 msgr2=0x7fd1280b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:35.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.566+0000 7fd12d6e8640 1 --2- 172.21.15.154:0/3567959342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd12815ad70 0x7fd1280b2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:35.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.566+0000 7fd12d6e8640 1 -- 172.21.15.154:0/3567959342 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd120009a70 con 0x7fd1280ad9c0 2023-09-23T03:28:35.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.566+0000 7fd12d6e8640 1 --2- 172.21.15.154:0/3567959342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1280ad9c0 0x7fd1281edb30 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fd12400b3f0 tx=0x7fd12400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:35.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.567+0000 7fd11dffb640 1 -- 172.21.15.154:0/3567959342 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd124011830 con 0x7fd1280ad9c0 2023-09-23T03:28:35.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.567+0000 7fd11dffb640 1 -- 172.21.15.154:0/3567959342 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd124011990 con 0x7fd1280ad9c0 2023-09-23T03:28:35.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.567+0000 7fd11dffb640 1 -- 172.21.15.154:0/3567959342 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd124011d40 con 0x7fd1280ad9c0 2023-09-23T03:28:35.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.567+0000 7fd12f94c640 1 -- 172.21.15.154:0/3567959342 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1280c2fe0 con 0x7fd1280ad9c0 2023-09-23T03:28:35.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.567+0000 7fd12f94c640 1 -- 172.21.15.154:0/3567959342 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1280c2660 con 0x7fd1280ad9c0 2023-09-23T03:28:35.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.568+0000 7fd12f94c640 1 -- 172.21.15.154:0/3567959342 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd128067870 con 0x7fd1280ad9c0 2023-09-23T03:28:35.570 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.569+0000 7fd11dffb640 1 -- 172.21.15.154:0/3567959342 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fd12401a400 con 0x7fd1280ad9c0 2023-09-23T03:28:35.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.570+0000 7fd11dffb640 1 --2- 172.21.15.154:0/3567959342 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd10c06af60 0x7fd10c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:35.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.570+0000 7fd11dffb640 1 -- 172.21.15.154:0/3567959342 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6201+0+0 (secure 0 0 0) 0x7fd1240897d0 con 0x7fd1280ad9c0 2023-09-23T03:28:35.571 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.570+0000 7fd12cee7640 1 --2- 172.21.15.154:0/3567959342 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd10c06af60 0x7fd10c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:35.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.571+0000 7fd12cee7640 1 --2- 172.21.15.154:0/3567959342 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd10c06af60 0x7fd10c06d420 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd128157710 tx=0x7fd118009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:35.575 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.574+0000 7fd11dffb640 1 -- 172.21.15.154:0/3567959342 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fd124054610 con 0x7fd1280ad9c0 2023-09-23T03:28:35.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.802+0000 7fd12f94c640 1 -- 172.21.15.154:0/3567959342 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1281f16a0 con 0x7fd1280ad9c0 2023-09-23T03:28:35.805 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.803+0000 7fd11dffb640 1 -- 172.21.15.154:0/3567959342 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+4935 (secure 0 0 0) 0x7fd1281f16a0 con 0x7fd1280ad9c0 2023-09-23T03:28:35.805 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:28:35.805 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":120,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/2759187683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2759187683},{"type":"v1","addr":"172.21.15.154:6835","nonce":2759187683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":5924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3712825179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3712825179},{"type":"v1","addr":"172.21.15.190:6835","nonce":3712825179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":110},{"gid":6544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/4147378242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":4147378242},{"type":"v1","addr":"172.21.15.190:6834","nonce":4147378242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:27:44.470312+0000","modified":"2023-09-23T03:28:34.470371+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9306":{"gid":9306,"name":"c","rank":0,"incarnation":118,"state":"up:rejoin","state_seq":4,"addr":"172.21.15.154:6837/1250699891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1250699891},{"type":"v1","addr":"172.21.15.154:6837","nonce":1250699891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-09-23T03:28:35.805 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 120 2023-09-23T03:28:35.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.806+0000 7fd12f94c640 1 -- 172.21.15.154:0/3567959342 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd10c06af60 msgr2=0x7fd10c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:35.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.806+0000 7fd12f94c640 1 --2- 172.21.15.154:0/3567959342 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd10c06af60 0x7fd10c06d420 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd128157710 tx=0x7fd118009040 comp rx=0 tx=0).stop 2023-09-23T03:28:35.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.807+0000 7fd12f94c640 1 -- 172.21.15.154:0/3567959342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1280ad9c0 msgr2=0x7fd1281edb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:35.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.807+0000 7fd12f94c640 1 --2- 172.21.15.154:0/3567959342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1280ad9c0 0x7fd1281edb30 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fd12400b3f0 tx=0x7fd12400b8b0 comp rx=0 tx=0).stop 2023-09-23T03:28:35.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.807+0000 7fd12f94c640 1 -- 172.21.15.154:0/3567959342 shutdown_connections 2023-09-23T03:28:35.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.807+0000 7fd12f94c640 1 --2- 172.21.15.154:0/3567959342 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd10c06af60 0x7fd10c06d420 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:35.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.807+0000 7fd12f94c640 1 --2- 172.21.15.154:0/3567959342 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd12815ad70 0x7fd1280b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:35.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.807+0000 7fd12f94c640 1 --2- 172.21.15.154:0/3567959342 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd1280b0780 0x7fd1280c2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:35.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.807+0000 7fd12f94c640 1 --2- 172.21.15.154:0/3567959342 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd1280ad9c0 0x7fd1281edb30 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:35.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.807+0000 7fd12f94c640 1 -- 172.21.15.154:0/3567959342 >> 172.21.15.154:0/3567959342 conn(0x7fd1280cb070 msgr2=0x7fd1280afdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:35.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.807+0000 7fd12f94c640 1 -- 172.21.15.154:0/3567959342 shutdown_connections 2023-09-23T03:28:35.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:35.807+0000 7fd12f94c640 1 -- 172.21.15.154:0/3567959342 wait complete. 2023-09-23T03:28:36.229 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 120, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-23T03:27:44.470312+0000', 'modified': '2023-09-23T03:28:34.470371+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 128, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9306}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9306': {'gid': 9306, 'name': 'c', 'rank': 0, 'incarnation': 118, 'state': 'up:rejoin', 'state_seq': 4, 'addr': '172.21.15.154:6837/1250699891', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6836', 'nonce': 1250699891}, {'type': 'v1', 'addr': '172.21.15.154:6837', 'nonce': 1250699891}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-23T03:28:36.229 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9306:up:rejoin 2023-09-23T03:28:37.230 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:28:37.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.451+0000 7f9972fd0640 1 Processor -- start 2023-09-23T03:28:37.453 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.452+0000 7f9972fd0640 1 -- start start 2023-09-23T03:28:37.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.453+0000 7f9972fd0640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f996c15ad70 0x7f996c0c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:37.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.453+0000 7f9972fd0640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f996c0ad9c0 0x7f996c0b0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:37.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.453+0000 7f9972fd0640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f996c070b40 con 0x7f996c0b05b0 2023-09-23T03:28:37.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.453+0000 7f9972fd0640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f996c071100 con 0x7f996c0ad9c0 2023-09-23T03:28:37.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.453+0000 7f9972fd0640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f996c0b0a60 con 0x7f996c15ad70 2023-09-23T03:28:37.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.454+0000 7f9963fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f996c0ad9c0 0x7f996c0b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:37.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.454+0000 7f9963fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f996c0ad9c0 0x7f996c0b0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49032/0 (socket says 172.21.15.154:49032) 2023-09-23T03:28:37.455 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.454+0000 7f9963fff640 1 -- 172.21.15.154:0/1944460112 learned_addr learned my addr 172.21.15.154:0/1944460112 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:37.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.454+0000 7f9970d6c640 1 --2- 172.21.15.154:0/1944460112 >> v2:172.21.15.190:3300/0 conn(0x7f996c15ad70 0x7f996c0c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:37.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.454+0000 7f9963fff640 1 -- 172.21.15.154:0/1944460112 >> v2:172.21.15.190:3300/0 conn(0x7f996c15ad70 msgr2=0x7f996c0c8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:37.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.454+0000 7f9963fff640 1 --2- 172.21.15.154:0/1944460112 >> v2:172.21.15.190:3300/0 conn(0x7f996c15ad70 0x7f996c0c8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:37.456 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.454+0000 7f9963fff640 1 -- 172.21.15.154:0/1944460112 >> v1:172.21.15.154:6789/0 conn(0x7f996c0b05b0 legacy=0x7f996c15e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:37.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.454+0000 7f9963fff640 1 -- 172.21.15.154:0/1944460112 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f996c070e20 con 0x7f996c0ad9c0 2023-09-23T03:28:37.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.454+0000 7f99637fe640 1 -- 172.21.15.154:0/1944460112 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1375995597 0 0) 0x7f996c070b40 con 0x7f996c0b05b0 2023-09-23T03:28:37.457 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.455+0000 7f9963fff640 1 --2- 172.21.15.154:0/1944460112 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f996c0ad9c0 0x7f996c0b0070 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f996c06ad70 tx=0x7f995c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6929e77cd793539a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:37.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.455+0000 7f99637fe640 1 -- 172.21.15.154:0/1944460112 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c030860 con 0x7f996c0ad9c0 2023-09-23T03:28:37.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.456+0000 7f99637fe640 1 -- 172.21.15.154:0/1944460112 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f995c0309c0 con 0x7f996c0ad9c0 2023-09-23T03:28:37.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.456+0000 7f99637fe640 1 -- 172.21.15.154:0/1944460112 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c030d50 con 0x7f996c0ad9c0 2023-09-23T03:28:37.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.456+0000 7f9972fd0640 1 -- 172.21.15.154:0/1944460112 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f996c0ad9c0 msgr2=0x7f996c0b0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:37.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.456+0000 7f9972fd0640 1 --2- 172.21.15.154:0/1944460112 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f996c0ad9c0 0x7f996c0b0070 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f996c06ad70 tx=0x7f995c02b9a0 comp rx=0 tx=0).stop 2023-09-23T03:28:37.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.456+0000 7f9972fd0640 1 -- 172.21.15.154:0/1944460112 shutdown_connections 2023-09-23T03:28:37.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.456+0000 7f9972fd0640 1 --2- 172.21.15.154:0/1944460112 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f996c0ad9c0 0x7f996c0b0070 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:37.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.456+0000 7f9972fd0640 1 --2- 172.21.15.154:0/1944460112 >> v2:172.21.15.190:3300/0 conn(0x7f996c15ad70 0x7f996c0c8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:37.458 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.456+0000 7f9972fd0640 1 -- 172.21.15.154:0/1944460112 >> 172.21.15.154:0/1944460112 conn(0x7f996c0cb070 msgr2=0x7f996c0cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:37.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.456+0000 7f9972fd0640 1 -- 172.21.15.154:0/1944460112 shutdown_connections 2023-09-23T03:28:37.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.456+0000 7f9972fd0640 1 -- 172.21.15.154:0/1944460112 wait complete. 2023-09-23T03:28:37.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.458+0000 7f9972fd0640 1 Processor -- start 2023-09-23T03:28:37.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.458+0000 7f9972fd0640 1 -- start start 2023-09-23T03:28:37.460 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.459+0000 7f9972fd0640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f996c0ad9c0 0x7f996c0c60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:37.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.459+0000 7f9972fd0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f996c0b05b0 0x7f996c0aee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:37.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.459+0000 7f9972fd0640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f996c15ad70 0x7f996c0c2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:37.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.459+0000 7f9972fd0640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996c0aea50 con 0x7f996c0b05b0 2023-09-23T03:28:37.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.459+0000 7f9972fd0640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996c0c5e00 con 0x7f996c15ad70 2023-09-23T03:28:37.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.459+0000 7f9972fd0640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f996c1f2330 con 0x7f996c0ad9c0 2023-09-23T03:28:37.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.459+0000 7f9963fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f996c0b05b0 0x7f996c0aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:37.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.459+0000 7f9963fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f996c0b05b0 0x7f996c0aee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49040/0 (socket says 172.21.15.154:49040) 2023-09-23T03:28:37.461 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.459+0000 7f9963fff640 1 -- 172.21.15.154:0/4009560653 learned_addr learned my addr 172.21.15.154:0/4009560653 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:37.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.460+0000 7f9970d6c640 1 --2- 172.21.15.154:0/4009560653 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f996c0ad9c0 0x7f996c0c60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:37.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.460+0000 7f997156d640 1 --2- 172.21.15.154:0/4009560653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f996c15ad70 0x7f996c0c2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:37.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.460+0000 7f9963fff640 1 -- 172.21.15.154:0/4009560653 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f996c0ad9c0 msgr2=0x7f996c0c60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:37.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.460+0000 7f9963fff640 1 --2- 172.21.15.154:0/4009560653 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f996c0ad9c0 0x7f996c0c60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:37.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.460+0000 7f9963fff640 1 -- 172.21.15.154:0/4009560653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f996c15ad70 msgr2=0x7f996c0c2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:37.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.460+0000 7f9963fff640 1 --2- 172.21.15.154:0/4009560653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f996c15ad70 0x7f996c0c2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:37.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.460+0000 7f9963fff640 1 -- 172.21.15.154:0/4009560653 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f995c0094d0 con 0x7f996c0b05b0 2023-09-23T03:28:37.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.460+0000 7f9970d6c640 1 --2- 172.21.15.154:0/4009560653 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f996c0ad9c0 0x7f996c0c60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:37.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.460+0000 7f9963fff640 1 --2- 172.21.15.154:0/4009560653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f996c0b05b0 0x7f996c0aee70 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f995c000c00 tx=0x7f995c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:37.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.461+0000 7f99617fa640 1 -- 172.21.15.154:0/4009560653 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c039d60 con 0x7f996c0b05b0 2023-09-23T03:28:37.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.461+0000 7f99617fa640 1 -- 172.21.15.154:0/4009560653 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f995c044480 con 0x7f996c0b05b0 2023-09-23T03:28:37.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.461+0000 7f99617fa640 1 -- 172.21.15.154:0/4009560653 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c044830 con 0x7f996c0b05b0 2023-09-23T03:28:37.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.461+0000 7f9972fd0640 1 -- 172.21.15.154:0/4009560653 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f996c0c67d0 con 0x7f996c0b05b0 2023-09-23T03:28:37.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.461+0000 7f9972fd0640 1 -- 172.21.15.154:0/4009560653 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f996c0c5840 con 0x7f996c0b05b0 2023-09-23T03:28:37.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.462+0000 7f9972fd0640 1 -- 172.21.15.154:0/4009560653 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f996c067870 con 0x7f996c0b05b0 2023-09-23T03:28:37.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.463+0000 7f99617fa640 1 -- 172.21.15.154:0/4009560653 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f995c04d3b0 con 0x7f996c0b05b0 2023-09-23T03:28:37.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.464+0000 7f99617fa640 1 --2- 172.21.15.154:0/4009560653 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f995806af60 0x7f995806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:37.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.464+0000 7f99617fa640 1 -- 172.21.15.154:0/4009560653 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f995c03e070 con 0x7f996c0b05b0 2023-09-23T03:28:37.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.464+0000 7f9970d6c640 1 --2- 172.21.15.154:0/4009560653 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f995806af60 0x7f995806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:37.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.465+0000 7f9970d6c640 1 --2- 172.21.15.154:0/4009560653 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f995806af60 0x7f995806d420 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f9968009f50 tx=0x7f996800b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:37.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.468+0000 7f99617fa640 1 -- 172.21.15.154:0/4009560653 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f995c077530 con 0x7f996c0b05b0 2023-09-23T03:28:37.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.701+0000 7f9972fd0640 1 -- 172.21.15.154:0/4009560653 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f996c0aeb90 con 0x7f996c0b05b0 2023-09-23T03:28:37.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.702+0000 7f99617fa640 1 -- 172.21.15.154:0/4009560653 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+4935 (secure 0 0 0) 0x7f996c0aeb90 con 0x7f996c0b05b0 2023-09-23T03:28:37.704 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:28:37.704 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":121,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/2759187683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2759187683},{"type":"v1","addr":"172.21.15.154:6835","nonce":2759187683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":5924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3712825179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3712825179},{"type":"v1","addr":"172.21.15.190:6835","nonce":3712825179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":110},{"gid":6544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/4147378242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":4147378242},{"type":"v1","addr":"172.21.15.190:6834","nonce":4147378242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:27:44.470312+0000","modified":"2023-09-23T03:28:35.478753+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9306":{"gid":9306,"name":"c","rank":0,"incarnation":118,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6837/1250699891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1250699891},{"type":"v1","addr":"172.21.15.154:6837","nonce":1250699891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-09-23T03:28:37.704 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 121 2023-09-23T03:28:37.706 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.705+0000 7f9972fd0640 1 -- 172.21.15.154:0/4009560653 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f995806af60 msgr2=0x7f995806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:37.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.705+0000 7f9972fd0640 1 --2- 172.21.15.154:0/4009560653 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f995806af60 0x7f995806d420 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f9968009f50 tx=0x7f996800b040 comp rx=0 tx=0).stop 2023-09-23T03:28:37.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.705+0000 7f9972fd0640 1 -- 172.21.15.154:0/4009560653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f996c0b05b0 msgr2=0x7f996c0aee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:37.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.705+0000 7f9972fd0640 1 --2- 172.21.15.154:0/4009560653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f996c0b05b0 0x7f996c0aee70 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f995c000c00 tx=0x7f995c004730 comp rx=0 tx=0).stop 2023-09-23T03:28:37.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.705+0000 7f9972fd0640 1 -- 172.21.15.154:0/4009560653 shutdown_connections 2023-09-23T03:28:37.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.705+0000 7f9972fd0640 1 --2- 172.21.15.154:0/4009560653 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f995806af60 0x7f995806d420 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:37.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.705+0000 7f9972fd0640 1 --2- 172.21.15.154:0/4009560653 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f996c15ad70 0x7f996c0c2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:37.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.705+0000 7f9972fd0640 1 --2- 172.21.15.154:0/4009560653 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f996c0b05b0 0x7f996c0aee70 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:37.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.705+0000 7f9972fd0640 1 --2- 172.21.15.154:0/4009560653 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f996c0ad9c0 0x7f996c0c60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:37.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.705+0000 7f9972fd0640 1 -- 172.21.15.154:0/4009560653 >> 172.21.15.154:0/4009560653 conn(0x7f996c0cb070 msgr2=0x7f996c159b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:37.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.705+0000 7f9972fd0640 1 -- 172.21.15.154:0/4009560653 shutdown_connections 2023-09-23T03:28:37.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:37.705+0000 7f9972fd0640 1 -- 172.21.15.154:0/4009560653 wait complete. 2023-09-23T03:28:38.128 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 121, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-23T03:27:44.470312+0000', 'modified': '2023-09-23T03:28:35.478753+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 128, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9306}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9306': {'gid': 9306, 'name': 'c', 'rank': 0, 'incarnation': 118, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.154:6837/1250699891', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6836', 'nonce': 1250699891}, {'type': 'v1', 'addr': '172.21.15.154:6837', 'nonce': 1250699891}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-23T03:28:38.128 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-23T03:28:38.129 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-09-23T03:28:38.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.357+0000 7f1ff4dc5640 1 Processor -- start 2023-09-23T03:28:38.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.357+0000 7f1ff4dc5640 1 -- start start 2023-09-23T03:28:38.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.359+0000 7f1ff4dc5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ff015aa20 0x7f1ff00cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:38.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.359+0000 7f1ff4dc5640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f1ff00cb4f0 0x7f1ff00b2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:38.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.359+0000 7f1ff4dc5640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ff00cd140 con 0x7f1ff015a640 2023-09-23T03:28:38.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.359+0000 7f1ff4dc5640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ff00cb9a0 con 0x7f1ff00cb4f0 2023-09-23T03:28:38.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.359+0000 7f1ff4dc5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff00cc530 con 0x7f1ff015aa20 2023-09-23T03:28:38.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.360+0000 7f1fee59c640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f1ff015a640 0x7f1ff00b3e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:41420/0 (socket says 172.21.15.154:41420) 2023-09-23T03:28:38.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.360+0000 7f1fee59c640 1 -- 172.21.15.154:0/3103565760 learned_addr learned my addr 172.21.15.154:0/3103565760 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:28:38.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.360+0000 7f1feed9d640 1 --2- 172.21.15.154:0/3103565760 >> v2:172.21.15.154:3300/0 conn(0x7f1ff00cb4f0 0x7f1ff00b2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:38.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.360+0000 7f1fe5d9b640 1 --2- 172.21.15.154:0/3103565760 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ff015aa20 0x7f1ff00cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:38.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.360+0000 7f1feed9d640 1 -- 172.21.15.154:0/3103565760 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ff015aa20 msgr2=0x7f1ff00cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:38.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.360+0000 7f1feed9d640 1 --2- 172.21.15.154:0/3103565760 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ff015aa20 0x7f1ff00cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.360+0000 7f1feed9d640 1 -- 172.21.15.154:0/3103565760 >> v1:172.21.15.154:6789/0 conn(0x7f1ff015a640 legacy=0x7f1ff00b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:38.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.360+0000 7f1feed9d640 1 -- 172.21.15.154:0/3103565760 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff00cc810 con 0x7f1ff00cb4f0 2023-09-23T03:28:38.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.361+0000 7f1feed9d640 1 --2- 172.21.15.154:0/3103565760 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ff00cb4f0 0x7f1ff00b2820 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f1fe80097a0 tx=0x7f1fe802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb4929487349cbf0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:38.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.361+0000 7f1fedd9b640 1 -- 172.21.15.154:0/3103565760 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe8031780 con 0x7f1ff00cb4f0 2023-09-23T03:28:38.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.362+0000 7f1fedd9b640 1 -- 172.21.15.154:0/3103565760 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe80318e0 con 0x7f1ff00cb4f0 2023-09-23T03:28:38.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.362+0000 7f1fedd9b640 1 -- 172.21.15.154:0/3103565760 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe8031c70 con 0x7f1ff00cb4f0 2023-09-23T03:28:38.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.362+0000 7f1ff4dc5640 1 -- 172.21.15.154:0/3103565760 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ff00cb4f0 msgr2=0x7f1ff00b2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:38.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.362+0000 7f1ff4dc5640 1 --2- 172.21.15.154:0/3103565760 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ff00cb4f0 0x7f1ff00b2820 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f1fe80097a0 tx=0x7f1fe802b820 comp rx=0 tx=0).stop 2023-09-23T03:28:38.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.362+0000 7f1ff4dc5640 1 -- 172.21.15.154:0/3103565760 shutdown_connections 2023-09-23T03:28:38.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.362+0000 7f1ff4dc5640 1 --2- 172.21.15.154:0/3103565760 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ff00cb4f0 0x7f1ff00b2820 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.362+0000 7f1ff4dc5640 1 --2- 172.21.15.154:0/3103565760 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ff015aa20 0x7f1ff00cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.362+0000 7f1ff4dc5640 1 -- 172.21.15.154:0/3103565760 >> 172.21.15.154:0/3103565760 conn(0x7f1ff006c820 msgr2=0x7f1ff0069cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:38.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.362+0000 7f1ff4dc5640 1 -- 172.21.15.154:0/3103565760 shutdown_connections 2023-09-23T03:28:38.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.362+0000 7f1ff4dc5640 1 -- 172.21.15.154:0/3103565760 wait complete. 2023-09-23T03:28:38.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.364+0000 7f1ff4dc5640 1 Processor -- start 2023-09-23T03:28:38.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.364+0000 7f1ff4dc5640 1 -- start start 2023-09-23T03:28:38.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.365+0000 7f1ff4dc5640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ff00cb4f0 0x7f1ff01550c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:38.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.365+0000 7f1ff4dc5640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ff015a640 0x7f1ff015c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:38.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.365+0000 7f1ff4dc5640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ff015aa20 0x7f1ff00b2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:38.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.365+0000 7f1ff4dc5640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff0156fd0 con 0x7f1ff015a640 2023-09-23T03:28:38.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.365+0000 7f1ff4dc5640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff0156a10 con 0x7f1ff00cb4f0 2023-09-23T03:28:38.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.365+0000 7f1ff4dc5640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff0156cf0 con 0x7f1ff015aa20 2023-09-23T03:28:38.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.366+0000 7f1fe5d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ff015a640 0x7f1ff015c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:38.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.366+0000 7f1fe5d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ff015a640 0x7f1ff015c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49058/0 (socket says 172.21.15.154:49058) 2023-09-23T03:28:38.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.366+0000 7f1fe5d9b640 1 -- 172.21.15.154:0/3279341800 learned_addr learned my addr 172.21.15.154:0/3279341800 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:38.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.366+0000 7f1fee59c640 1 --2- 172.21.15.154:0/3279341800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ff00cb4f0 0x7f1ff01550c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:38.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.366+0000 7f1feed9d640 1 --2- 172.21.15.154:0/3279341800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ff015aa20 0x7f1ff00b2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:38.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.366+0000 7f1fe5d9b640 1 -- 172.21.15.154:0/3279341800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ff015aa20 msgr2=0x7f1ff00b2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:38.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.366+0000 7f1fe5d9b640 1 --2- 172.21.15.154:0/3279341800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ff015aa20 0x7f1ff00b2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.366+0000 7f1fe5d9b640 1 -- 172.21.15.154:0/3279341800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ff00cb4f0 msgr2=0x7f1ff01550c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:38.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.366+0000 7f1fe5d9b640 1 --2- 172.21.15.154:0/3279341800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ff00cb4f0 0x7f1ff01550c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.366+0000 7f1fe5d9b640 1 -- 172.21.15.154:0/3279341800 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe8009400 con 0x7f1ff015a640 2023-09-23T03:28:38.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.366+0000 7f1fee59c640 1 --2- 172.21.15.154:0/3279341800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ff00cb4f0 0x7f1ff01550c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:38.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.366+0000 7f1fe5d9b640 1 --2- 172.21.15.154:0/3279341800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ff015a640 0x7f1ff015c470 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800d5c0 tx=0x7f1fd800da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:38.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.367+0000 7f1fe77fe640 1 -- 172.21.15.154:0/3279341800 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd80125f0 con 0x7f1ff015a640 2023-09-23T03:28:38.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.367+0000 7f1fe77fe640 1 -- 172.21.15.154:0/3279341800 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd8012750 con 0x7f1ff015a640 2023-09-23T03:28:38.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.367+0000 7f1fe77fe640 1 -- 172.21.15.154:0/3279341800 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8012b00 con 0x7f1ff015a640 2023-09-23T03:28:38.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.367+0000 7f1ff4dc5640 1 -- 172.21.15.154:0/3279341800 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ff0157370 con 0x7f1ff015a640 2023-09-23T03:28:38.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.367+0000 7f1ff4dc5640 1 -- 172.21.15.154:0/3279341800 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ff0157b70 con 0x7f1ff015a640 2023-09-23T03:28:38.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.369+0000 7f1fe77fe640 1 -- 172.21.15.154:0/3279341800 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f1fd8012c60 con 0x7f1ff015a640 2023-09-23T03:28:38.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.370+0000 7f1fe77fe640 1 --2- 172.21.15.154:0/3279341800 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1fd006af30 0x7f1fd006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:38.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.370+0000 7f1fe77fe640 1 -- 172.21.15.154:0/3279341800 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f1fd8089e30 con 0x7f1ff015a640 2023-09-23T03:28:38.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.370+0000 7f1fee59c640 1 --2- 172.21.15.154:0/3279341800 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1fd006af30 0x7f1fd006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:38.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.370+0000 7f1ff7a30740 1 Processor -- start 2023-09-23T03:28:38.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.371+0000 7f1ff7a30740 1 -- start start 2023-09-23T03:28:38.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.371+0000 7f1fee59c640 1 --2- 172.21.15.154:0/3279341800 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1fd006af30 0x7f1fd006d3f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0005c00 tx=0x7f1fe0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:38.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.371+0000 7f1ff7a30740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560c6ad57c00 0x560c6ad2ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:38.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.371+0000 7f1ff7a30740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c6ad57fe0 0x560c6ad5a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:38.373 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.371+0000 7f1ff7a30740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c6ad2b040 0x560c6ad5aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:38.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.371+0000 7f1ff7a30740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c6ac9e6d0 con 0x560c6ad57fe0 2023-09-23T03:28:38.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.371+0000 7f1ff7a30740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c6ac9bdc0 con 0x560c6ad2b040 2023-09-23T03:28:38.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.371+0000 7f1ff7a30740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c6ac976b0 con 0x560c6ad57c00 2023-09-23T03:28:38.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.372+0000 7f1fe5d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c6ad57fe0 0x560c6ad5a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:38.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.372+0000 7f1fe5d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c6ad57fe0 0x560c6ad5a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49068/0 (socket says 172.21.15.154:49068) 2023-09-23T03:28:38.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.372+0000 7f1fe5d9b640 1 -- 172.21.15.154:0/1963995026 learned_addr learned my addr 172.21.15.154:0/1963995026 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:38.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.372+0000 7f1feed9d640 1 --2- 172.21.15.154:0/1963995026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c6ad2b040 0x560c6ad5aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:38.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.372+0000 7f1fee59c640 1 --2- 172.21.15.154:0/1963995026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560c6ad57c00 0x560c6ad2ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:38.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.372+0000 7f1fe5d9b640 1 -- 172.21.15.154:0/1963995026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560c6ad57c00 msgr2=0x560c6ad2ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:38.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.372+0000 7f1fe5d9b640 1 --2- 172.21.15.154:0/1963995026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560c6ad57c00 0x560c6ad2ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.372+0000 7f1fe5d9b640 1 -- 172.21.15.154:0/1963995026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c6ad2b040 msgr2=0x560c6ad5aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:38.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.372+0000 7f1fe5d9b640 1 --2- 172.21.15.154:0/1963995026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c6ad2b040 0x560c6ad5aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.372+0000 7f1fe5d9b640 1 -- 172.21.15.154:0/1963995026 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c6ac5b8e0 con 0x560c6ad57fe0 2023-09-23T03:28:38.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.372+0000 7f1fee59c640 1 --2- 172.21.15.154:0/1963995026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560c6ad57c00 0x560c6ad2ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:38.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.372+0000 7f1fe5d9b640 1 --2- 172.21.15.154:0/1963995026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c6ad57fe0 0x560c6ad5a990 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8060700 tx=0x7f1fd8058350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a730593411a21db7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:38.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.372+0000 7f1fcffff640 1 -- 172.21.15.154:0/1963995026 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd805ca80 con 0x560c6ad57fe0 2023-09-23T03:28:38.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.373+0000 7f1fcffff640 1 -- 172.21.15.154:0/1963995026 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd8060c30 con 0x560c6ad57fe0 2023-09-23T03:28:38.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.373+0000 7f1fcffff640 1 -- 172.21.15.154:0/1963995026 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8056cd0 con 0x560c6ad57fe0 2023-09-23T03:28:38.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.373+0000 7f1ff7a30740 1 -- 172.21.15.154:0/1963995026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c6ad57fe0 msgr2=0x560c6ad5a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:38.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.373+0000 7f1ff7a30740 1 --2- 172.21.15.154:0/1963995026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c6ad57fe0 0x560c6ad5a990 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8060700 tx=0x7f1fd8058350 comp rx=0 tx=0).stop 2023-09-23T03:28:38.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.373+0000 7f1ff7a30740 1 -- 172.21.15.154:0/1963995026 shutdown_connections 2023-09-23T03:28:38.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.373+0000 7f1ff7a30740 1 --2- 172.21.15.154:0/1963995026 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c6ad2b040 0x560c6ad5aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.373+0000 7f1ff7a30740 1 --2- 172.21.15.154:0/1963995026 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c6ad57fe0 0x560c6ad5a990 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.373+0000 7f1ff7a30740 1 --2- 172.21.15.154:0/1963995026 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560c6ad57c00 0x560c6ad2ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.373+0000 7f1ff7a30740 1 -- 172.21.15.154:0/1963995026 >> 172.21.15.154:0/1963995026 conn(0x560c6ad2ce10 msgr2=0x560c6ad75990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:38.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.373+0000 7f1ff7a30740 1 -- 172.21.15.154:0/1963995026 shutdown_connections 2023-09-23T03:28:38.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.373+0000 7f1ff7a30740 1 -- 172.21.15.154:0/1963995026 wait complete. 2023-09-23T03:28:38.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.374+0000 7f1ff7a30740 1 -- start start 2023-09-23T03:28:38.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.374+0000 7f1ff7a30740 1 Processor -- start 2023-09-23T03:28:38.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.375+0000 7f1ff7a30740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560c6ad57c00 0x560c6ad53790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:38.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.375+0000 7f1ff7a30740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c6ad57fe0 0x560c6ad50a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:38.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.375+0000 7f1ff7a30740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c6ad4fb30 0x560c6ad50fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:38.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.375+0000 7f1ff7a30740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c6ac35940 con 0x560c6ad57fe0 2023-09-23T03:28:38.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.375+0000 7f1ff7a30740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c6ac355e0 con 0x560c6ad4fb30 2023-09-23T03:28:38.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.375+0000 7f1ff7a30740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c6a9fbf20 con 0x560c6ad57c00 2023-09-23T03:28:38.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.375+0000 7f1ff7a30740 10 client.0 _ll_register_callbacks cb 0x560c6ac735e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:28:38.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.375+0000 7f1fe5d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c6ad57fe0 0x560c6ad50a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:38.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.376+0000 7f1fe5d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c6ad57fe0 0x560c6ad50a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49074/0 (socket says 172.21.15.154:49074) 2023-09-23T03:28:38.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.376+0000 7f1fe5d9b640 1 -- 172.21.15.154:0/4029840528 learned_addr learned my addr 172.21.15.154:0/4029840528 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:38.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.376+0000 7f1fee59c640 1 --2- 172.21.15.154:0/4029840528 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560c6ad57c00 0x560c6ad53790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:38.380 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.376+0000 7f1fe5d9b640 1 -- 172.21.15.154:0/4029840528 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560c6ad57c00 msgr2=0x560c6ad53790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:38.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.376+0000 7f1feed9d640 1 --2- 172.21.15.154:0/4029840528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c6ad4fb30 0x560c6ad50fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:38.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.376+0000 7f1fe5d9b640 1 --2- 172.21.15.154:0/4029840528 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560c6ad57c00 0x560c6ad53790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.376+0000 7f1fe5d9b640 1 -- 172.21.15.154:0/4029840528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c6ad4fb30 msgr2=0x560c6ad50fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:38.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.376+0000 7f1fe5d9b640 1 --2- 172.21.15.154:0/4029840528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c6ad4fb30 0x560c6ad50fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.376+0000 7f1fe5d9b640 1 -- 172.21.15.154:0/4029840528 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ff0157b70 con 0x560c6ad57fe0 2023-09-23T03:28:38.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.376+0000 7f1fe5d9b640 1 --2- 172.21.15.154:0/4029840528 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c6ad57fe0 0x560c6ad50a80 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8054de0 tx=0x7f1fd8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25aaeb4b59d00e9d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:38.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.376+0000 7f1fcd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:28:38.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.376+0000 7f1fcd7fa640 1 -- 172.21.15.154:0/4029840528 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8077220 con 0x560c6ad57fe0 2023-09-23T03:28:38.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.376+0000 7f1fcd7fa640 1 -- 172.21.15.154:0/4029840528 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f1fd8055040 con 0x560c6ad57fe0 2023-09-23T03:28:38.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.376+0000 7f1ff7a30740 1 -- 172.21.15.154:0/4029840528 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c6ac90690 con 0x560c6ad57fe0 2023-09-23T03:28:38.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.377+0000 7f1fcd7fa640 1 -- 172.21.15.154:0/4029840528 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=121) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fd8077380 con 0x560c6ad57fe0 2023-09-23T03:28:38.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.377+0000 7f1ff7a30740 10 client.9408 fetch_fsmap learned FSMap version 121 2023-09-23T03:28:38.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.377+0000 7f1ff7a30740 1 -- 172.21.15.154:0/4029840528 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=121,monmap=2+}) v3 -- 0x560c6ac95d30 con 0x560c6ad57fe0 2023-09-23T03:28:38.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.377+0000 7f1fcd7fa640 1 -- 172.21.15.154:0/4029840528 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd807e070 con 0x560c6ad57fe0 2023-09-23T03:28:38.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.377+0000 7f1fcd7fa640 1 -- 172.21.15.154:0/4029840528 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 121) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1fd8071070 con 0x560c6ad57fe0 2023-09-23T03:28:38.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.377+0000 7f1ff7a30740 10 client.9408 fetch_fsmap finished waiting for FSMap version 121 2023-09-23T03:28:38.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.377+0000 7f1ff7a30740 10 client.9408 resolve_mds: resolved name 'c' to daemon mds.c 2023-09-23T03:28:38.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.377+0000 7f1ff7a30740 20 client.9408 populate_metadata read hostname 'smithi154' 2023-09-23T03:28:38.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.377+0000 7f1ff7a30740 1 --2- 172.21.15.154:0/4029840528 >> [v2:172.21.15.154:6836/1250699891,v1:172.21.15.154:6837/1250699891] conn(0x560c6ad89610 0x560c6ad899f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:38.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.377+0000 7f1ff7a30740 4 client.9408 mds_command: new command op to 9306 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-23T03:28:38.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.377+0000 7f1ff7a30740 1 -- 172.21.15.154:0/4029840528 --> [v2:172.21.15.154:6836/1250699891,v1:172.21.15.154:6837/1250699891] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c6ac903b0 con 0x560c6ad89610 2023-09-23T03:28:38.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.377+0000 7f1fee59c640 1 --2- 172.21.15.154:0/4029840528 >> [v2:172.21.15.154:6836/1250699891,v1:172.21.15.154:6837/1250699891] conn(0x560c6ad89610 0x560c6ad899f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:38.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.378+0000 7f1fee59c640 1 --2- 172.21.15.154:0/4029840528 >> [v2:172.21.15.154:6836/1250699891,v1:172.21.15.154:6837/1250699891] conn(0x560c6ad89610 0x560c6ad899f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46aba6f4c40eaf43 server_cookie=918bde3b2528cee4 in_seq=0 out_seq=0 2023-09-23T03:28:38.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.378+0000 7f1fcd7fa640 10 client.9408 ms_handle_connect on v2:172.21.15.154:6836/1250699891 2023-09-23T03:28:38.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.379+0000 7f1fcd7fa640 1 -- 172.21.15.154:0/4029840528 <== mds.0 v2:172.21.15.154:6836/1250699891 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560c6ac903b0 con 0x560c6ad89610 2023-09-23T03:28:38.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.379+0000 7f1fcd7fa640 10 client.9408 handle_command_reply: tid=0 2023-09-23T03:28:38.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.379+0000 7f1ff7a30740 1 client.9408 shutdown 2023-09-23T03:28:38.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.379+0000 7f1ff7a30740 1 -- 172.21.15.154:0/4029840528 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c6ad57fe0 msgr2=0x560c6ad50a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:38.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.379+0000 7f1ff7a30740 1 --2- 172.21.15.154:0/4029840528 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c6ad57fe0 0x560c6ad50a80 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8054de0 tx=0x7f1fd8000bc0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.379+0000 7f1ff7a30740 1 -- 172.21.15.154:0/4029840528 shutdown_connections 2023-09-23T03:28:38.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.379+0000 7f1ff7a30740 1 --2- 172.21.15.154:0/4029840528 >> [v2:172.21.15.154:6836/1250699891,v1:172.21.15.154:6837/1250699891] conn(0x560c6ad89610 0x560c6ad899f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.379+0000 7f1ff7a30740 1 --2- 172.21.15.154:0/4029840528 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c6ad4fb30 0x560c6ad50fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.379+0000 7f1ff7a30740 1 --2- 172.21.15.154:0/4029840528 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c6ad57fe0 0x560c6ad50a80 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.379+0000 7f1ff7a30740 1 --2- 172.21.15.154:0/4029840528 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560c6ad57c00 0x560c6ad53790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.379+0000 7f1ff7a30740 1 -- 172.21.15.154:0/4029840528 >> 172.21.15.154:0/4029840528 conn(0x560c6ad2b040 msgr2=0x560c6ad2e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:38.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.379+0000 7f1fcd7fa640 0 client.9408 ms_handle_reset on v2:172.21.15.154:6836/1250699891 2023-09-23T03:28:38.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.379+0000 7f1ff7a30740 1 -- 172.21.15.154:0/4029840528 shutdown_connections 2023-09-23T03:28:38.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.379+0000 7f1ff7a30740 1 -- 172.21.15.154:0/4029840528 wait complete. 2023-09-23T03:28:38.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.379+0000 7f1ff7a30740 20 client.9408 trim_cache size 0 max 16384 2023-09-23T03:28:38.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.395+0000 7f1ff7a30740 1 Processor -- start 2023-09-23T03:28:38.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.396+0000 7f1ff7a30740 1 -- start start 2023-09-23T03:28:38.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.396+0000 7f1ff7a30740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c6ad4fb30 0x560c6ad2e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:38.398 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.396+0000 7f1ff7a30740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560c6ad57c00 0x560c6ad78ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:38.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.396+0000 7f1ff7a30740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c6ad57fe0 0x560c6ad793e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:38.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.396+0000 7f1ff7a30740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c6ad59200 con 0x560c6ad4fb30 2023-09-23T03:28:38.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.396+0000 7f1ff7a30740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c6ad49730 con 0x560c6ad57fe0 2023-09-23T03:28:38.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.396+0000 7f1ff7a30740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c6ad4f940 con 0x560c6ad57c00 2023-09-23T03:28:38.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.397+0000 7f1fee59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c6ad4fb30 0x560c6ad2e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:38.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.397+0000 7f1fee59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c6ad4fb30 0x560c6ad2e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49080/0 (socket says 172.21.15.154:49080) 2023-09-23T03:28:38.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.397+0000 7f1fee59c640 1 -- 172.21.15.154:0/1141648554 learned_addr learned my addr 172.21.15.154:0/1141648554 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:38.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.397+0000 7f1fe5d9b640 1 --2- 172.21.15.154:0/1141648554 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560c6ad57c00 0x560c6ad78ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:38.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.397+0000 7f1feed9d640 1 --2- 172.21.15.154:0/1141648554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c6ad57fe0 0x560c6ad793e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:38.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.397+0000 7f1fee59c640 1 -- 172.21.15.154:0/1141648554 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560c6ad57c00 msgr2=0x560c6ad78ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:38.399 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.397+0000 7f1fee59c640 1 --2- 172.21.15.154:0/1141648554 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560c6ad57c00 0x560c6ad78ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.400 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.397+0000 7f1fee59c640 1 -- 172.21.15.154:0/1141648554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c6ad57fe0 msgr2=0x560c6ad793e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:38.400 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.397+0000 7f1fee59c640 1 --2- 172.21.15.154:0/1141648554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c6ad57fe0 0x560c6ad793e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.400 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.397+0000 7f1fee59c640 1 -- 172.21.15.154:0/1141648554 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c6ac34b50 con 0x560c6ad4fb30 2023-09-23T03:28:38.400 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.397+0000 7f1fe5d9b640 1 --2- 172.21.15.154:0/1141648554 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560c6ad57c00 0x560c6ad78ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:38.400 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.397+0000 7f1feed9d640 1 --2- 172.21.15.154:0/1141648554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c6ad57fe0 0x560c6ad793e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:38.400 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.397+0000 7f1fee59c640 1 --2- 172.21.15.154:0/1141648554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c6ad4fb30 0x560c6ad2e990 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x560c6ad507e0 tx=0x7f1fe0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d188aeea0307b50f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:38.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.398+0000 7f1fe4d99640 1 -- 172.21.15.154:0/1141648554 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0019040 con 0x560c6ad4fb30 2023-09-23T03:28:38.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.398+0000 7f1fe4d99640 1 -- 172.21.15.154:0/1141648554 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe001a040 con 0x560c6ad4fb30 2023-09-23T03:28:38.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.398+0000 7f1fe4d99640 1 -- 172.21.15.154:0/1141648554 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0022a00 con 0x560c6ad4fb30 2023-09-23T03:28:38.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.398+0000 7f1ff7a30740 1 -- 172.21.15.154:0/1141648554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c6ad4fb30 msgr2=0x560c6ad2e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:38.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.398+0000 7f1ff7a30740 1 --2- 172.21.15.154:0/1141648554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c6ad4fb30 0x560c6ad2e990 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x560c6ad507e0 tx=0x7f1fe0011000 comp rx=0 tx=0).stop 2023-09-23T03:28:38.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.398+0000 7f1ff7a30740 1 -- 172.21.15.154:0/1141648554 shutdown_connections 2023-09-23T03:28:38.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.398+0000 7f1ff7a30740 1 --2- 172.21.15.154:0/1141648554 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c6ad57fe0 0x560c6ad793e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.401 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.398+0000 7f1ff7a30740 1 --2- 172.21.15.154:0/1141648554 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560c6ad57c00 0x560c6ad78ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.398+0000 7f1ff7a30740 1 --2- 172.21.15.154:0/1141648554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c6ad4fb30 0x560c6ad2e990 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.398+0000 7f1ff7a30740 1 -- 172.21.15.154:0/1141648554 >> 172.21.15.154:0/1141648554 conn(0x560c6ad2b040 msgr2=0x560c6ad26340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:38.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.398+0000 7f1ff7a30740 1 -- 172.21.15.154:0/1141648554 shutdown_connections 2023-09-23T03:28:38.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.398+0000 7f1ff7a30740 1 -- 172.21.15.154:0/1141648554 wait complete. 2023-09-23T03:28:38.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.399+0000 7f1ff7a30740 1 -- start start 2023-09-23T03:28:38.402 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.399+0000 7f1ff7a30740 1 Processor -- start 2023-09-23T03:28:38.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.400+0000 7f1ff7a30740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c6ad4fb30 0x560c6ad59550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:38.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.400+0000 7f1ff7a30740 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560c6ad57c00 0x560c6ad2fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:38.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.400+0000 7f1ff7a30740 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c6ad57fe0 0x560c6ad879e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:38.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.400+0000 7f1ff7a30740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c6ad59fd0 con 0x560c6ad4fb30 2023-09-23T03:28:38.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.400+0000 7f1ff7a30740 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c6ad312d0 con 0x560c6ad57fe0 2023-09-23T03:28:38.405 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.400+0000 7f1ff7a30740 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c6ad75d20 con 0x560c6ad57c00 2023-09-23T03:28:38.406 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.400+0000 7f1ff7a30740 10 client.0 _ll_register_callbacks cb 0x7f1fd8012f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-23T03:28:38.406 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.400+0000 7f1fee59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c6ad4fb30 0x560c6ad59550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:38.406 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.400+0000 7f1fee59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c6ad4fb30 0x560c6ad59550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49082/0 (socket says 172.21.15.154:49082) 2023-09-23T03:28:38.406 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.401+0000 7f1fee59c640 1 -- 172.21.15.154:0/358641048 learned_addr learned my addr 172.21.15.154:0/358641048 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:38.406 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.401+0000 7f1feed9d640 1 --2- 172.21.15.154:0/358641048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c6ad57fe0 0x560c6ad879e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:38.406 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.401+0000 7f1fee59c640 1 -- 172.21.15.154:0/358641048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560c6ad57c00 msgr2=0x560c6ad2fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:38.406 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.401+0000 7f1fee59c640 1 --2- 172.21.15.154:0/358641048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560c6ad57c00 0x560c6ad2fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.406 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.401+0000 7f1fee59c640 1 -- 172.21.15.154:0/358641048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c6ad57fe0 msgr2=0x560c6ad879e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:38.406 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.401+0000 7f1fee59c640 1 --2- 172.21.15.154:0/358641048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c6ad57fe0 0x560c6ad879e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.406 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.401+0000 7f1fee59c640 1 -- 172.21.15.154:0/358641048 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ff00cd140 con 0x560c6ad4fb30 2023-09-23T03:28:38.406 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.401+0000 7f1fee59c640 1 --2- 172.21.15.154:0/358641048 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c6ad4fb30 0x560c6ad59550 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0029040 tx=0x7f1fe0001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33b4f88c8d81bec server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:38.406 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.401+0000 7f1fcd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2023-09-23T03:28:38.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.401+0000 7f1fcd7fa640 1 -- 172.21.15.154:0/358641048 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe002c640 con 0x560c6ad4fb30 2023-09-23T03:28:38.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.401+0000 7f1fcd7fa640 1 -- 172.21.15.154:0/358641048 <== mon.0 v2:172.21.15.154:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f1fe0019070 con 0x560c6ad4fb30 2023-09-23T03:28:38.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.401+0000 7f1ff7a30740 1 -- 172.21.15.154:0/358641048 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1fd8060c30 con 0x560c6ad4fb30 2023-09-23T03:28:38.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.402+0000 7f1fcd7fa640 1 -- 172.21.15.154:0/358641048 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=121) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fe00225b0 con 0x560c6ad4fb30 2023-09-23T03:28:38.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.402+0000 7f1ff7a30740 10 client.9414 fetch_fsmap learned FSMap version 121 2023-09-23T03:28:38.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.402+0000 7f1ff7a30740 1 -- 172.21.15.154:0/358641048 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=121,monmap=2+}) v3 -- 0x560c6ac5a140 con 0x560c6ad4fb30 2023-09-23T03:28:38.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.402+0000 7f1fcd7fa640 1 -- 172.21.15.154:0/358641048 <== mon.0 v2:172.21.15.154:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd8060c30 con 0x560c6ad4fb30 2023-09-23T03:28:38.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.402+0000 7f1fcd7fa640 1 -- 172.21.15.154:0/358641048 <== mon.0 v2:172.21.15.154:3300/0 5 ==== fsmap(e 121) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1fe001f040 con 0x560c6ad4fb30 2023-09-23T03:28:38.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.402+0000 7f1ff7a30740 10 client.9414 fetch_fsmap finished waiting for FSMap version 121 2023-09-23T03:28:38.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.402+0000 7f1ff7a30740 10 client.9414 resolve_mds: resolved name 'c' to daemon mds.c 2023-09-23T03:28:38.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.402+0000 7f1ff7a30740 20 client.9414 populate_metadata read hostname 'smithi154' 2023-09-23T03:28:38.407 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.402+0000 7f1ff7a30740 1 --2- 172.21.15.154:0/358641048 >> [v2:172.21.15.154:6836/1250699891,v1:172.21.15.154:6837/1250699891] conn(0x560c6adc6ff0 0x560c6ad87f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:38.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.402+0000 7f1ff7a30740 4 client.9414 mds_command: new command op to 9306 tid=0[{"prefix": "status"}] 2023-09-23T03:28:38.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.402+0000 7f1ff7a30740 1 -- 172.21.15.154:0/358641048 --> [v2:172.21.15.154:6836/1250699891,v1:172.21.15.154:6837/1250699891] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560c6a9996c0 con 0x560c6adc6ff0 2023-09-23T03:28:38.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.402+0000 7f1fe5d9b640 1 --2- 172.21.15.154:0/358641048 >> [v2:172.21.15.154:6836/1250699891,v1:172.21.15.154:6837/1250699891] conn(0x560c6adc6ff0 0x560c6ad87f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:38.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.403+0000 7f1fe5d9b640 1 --2- 172.21.15.154:0/358641048 >> [v2:172.21.15.154:6836/1250699891,v1:172.21.15.154:6837/1250699891] conn(0x560c6adc6ff0 0x560c6ad87f20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93c2c11a5f1ec445 server_cookie=1c9aa272e5d42d3c in_seq=0 out_seq=0 2023-09-23T03:28:38.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.403+0000 7f1fcd7fa640 10 client.9414 ms_handle_connect on v2:172.21.15.154:6836/1250699891 2023-09-23T03:28:38.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.403+0000 7f1fcd7fa640 1 -- 172.21.15.154:0/358641048 <== mds.0 v2:172.21.15.154:6836/1250699891 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x560c6a9996c0 con 0x560c6adc6ff0 2023-09-23T03:28:38.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.403+0000 7f1fcd7fa640 10 client.9414 handle_command_reply: tid=0 2023-09-23T03:28:38.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.403+0000 7f1ff7a30740 1 client.9414 shutdown 2023-09-23T03:28:38.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.403+0000 7f1ff7a30740 1 -- 172.21.15.154:0/358641048 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c6ad4fb30 msgr2=0x560c6ad59550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:38.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.403+0000 7f1ff7a30740 1 --2- 172.21.15.154:0/358641048 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c6ad4fb30 0x560c6ad59550 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0029040 tx=0x7f1fe0001250 comp rx=0 tx=0).stop 2023-09-23T03:28:38.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.403+0000 7f1ff7a30740 1 -- 172.21.15.154:0/358641048 shutdown_connections 2023-09-23T03:28:38.408 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.403+0000 7f1ff7a30740 1 --2- 172.21.15.154:0/358641048 >> [v2:172.21.15.154:6836/1250699891,v1:172.21.15.154:6837/1250699891] conn(0x560c6adc6ff0 0x560c6ad87f20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.403+0000 7f1ff7a30740 1 --2- 172.21.15.154:0/358641048 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x560c6ad57fe0 0x560c6ad879e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.403+0000 7f1ff7a30740 1 --2- 172.21.15.154:0/358641048 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x560c6ad57c00 0x560c6ad2fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.403+0000 7f1ff7a30740 1 --2- 172.21.15.154:0/358641048 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c6ad4fb30 0x560c6ad59550 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.403+0000 7f1fcd7fa640 0 client.9414 ms_handle_reset on v2:172.21.15.154:6836/1250699891 2023-09-23T03:28:38.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.403+0000 7f1ff7a30740 1 -- 172.21.15.154:0/358641048 >> 172.21.15.154:0/358641048 conn(0x560c6ad2b040 msgr2=0x560c6ad4ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:38.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.403+0000 7f1ff7a30740 1 -- 172.21.15.154:0/358641048 shutdown_connections 2023-09-23T03:28:38.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.403+0000 7f1ff7a30740 1 -- 172.21.15.154:0/358641048 wait complete. 2023-09-23T03:28:38.409 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.404+0000 7f1ff7a30740 20 client.9414 trim_cache size 0 max 16384 2023-09-23T03:28:38.409 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:28:38.410 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_fsid": "8c8039d3-6c84-43db-8808-85e0dbdd47c8", 2023-09-23T03:28:38.410 INFO:teuthology.orchestra.run.smithi154.stdout: "whoami": 0, 2023-09-23T03:28:38.410 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 9306, 2023-09-23T03:28:38.410 INFO:teuthology.orchestra.run.smithi154.stdout: "want_state": "up:active", 2023-09-23T03:28:38.410 INFO:teuthology.orchestra.run.smithi154.stdout: "state": "up:active", 2023-09-23T03:28:38.410 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs", 2023-09-23T03:28:38.410 INFO:teuthology.orchestra.run.smithi154.stdout: "rank_uptime": 4.9528327150000004, 2023-09-23T03:28:38.410 INFO:teuthology.orchestra.run.smithi154.stdout: "mdsmap_epoch": 121, 2023-09-23T03:28:38.410 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_epoch": 128, 2023-09-23T03:28:38.410 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_epoch_barrier": 128, 2023-09-23T03:28:38.410 INFO:teuthology.orchestra.run.smithi154.stdout: "uptime": 9.0291772259999998 2023-09-23T03:28:38.410 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:28:38.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.404+0000 7f1ff4dc5640 1 -- 172.21.15.154:0/3279341800 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1fd006af30 msgr2=0x7f1fd006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:38.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.404+0000 7f1ff4dc5640 1 --2- 172.21.15.154:0/3279341800 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1fd006af30 0x7f1fd006d3f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0005c00 tx=0x7f1fe0009040 comp rx=0 tx=0).stop 2023-09-23T03:28:38.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.404+0000 7f1ff4dc5640 1 -- 172.21.15.154:0/3279341800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ff015a640 msgr2=0x7f1ff015c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:38.411 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.404+0000 7f1ff4dc5640 1 --2- 172.21.15.154:0/3279341800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ff015a640 0x7f1ff015c470 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800d5c0 tx=0x7f1fd800da80 comp rx=0 tx=0).stop 2023-09-23T03:28:38.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.404+0000 7f1ff4dc5640 1 -- 172.21.15.154:0/3279341800 shutdown_connections 2023-09-23T03:28:38.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.404+0000 7f1ff4dc5640 1 --2- 172.21.15.154:0/3279341800 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f1fd006af30 0x7f1fd006d3f0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.404+0000 7f1ff4dc5640 1 --2- 172.21.15.154:0/3279341800 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f1ff015aa20 0x7f1ff00b2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.404+0000 7f1ff4dc5640 1 --2- 172.21.15.154:0/3279341800 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1ff015a640 0x7f1ff015c470 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.404+0000 7f1ff4dc5640 1 --2- 172.21.15.154:0/3279341800 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f1ff00cb4f0 0x7f1ff01550c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.404+0000 7f1ff4dc5640 1 -- 172.21.15.154:0/3279341800 >> 172.21.15.154:0/3279341800 conn(0x7f1ff006c820 msgr2=0x7f1ff00ca2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:38.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.404+0000 7f1ff4dc5640 1 -- 172.21.15.154:0/3279341800 shutdown_connections 2023-09-23T03:28:38.412 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.404+0000 7f1ff4dc5640 1 -- 172.21.15.154:0/3279341800 wait complete. 2023-09-23T03:28:38.421 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-23T03:28:38.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.613+0000 7fe31c9d7640 1 Processor -- start 2023-09-23T03:28:38.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.613+0000 7fe31c9d7640 1 -- start start 2023-09-23T03:28:38.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.615+0000 7fe31c9d7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3180b3ff0 0x7fe3180ca910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:38.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.615+0000 7fe31c9d7640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe3181441e0 0x7fe3180cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:38.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.615+0000 7fe31c9d7640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3180cc200 con 0x7fe318143e00 2023-09-23T03:28:38.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.615+0000 7fe31c9d7640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3180cb680 con 0x7fe3181441e0 2023-09-23T03:28:38.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.615+0000 7fe31c9d7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3180cb960 con 0x7fe3180b3ff0 2023-09-23T03:28:38.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.615+0000 7fe30ffff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3180b3ff0 0x7fe3180ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:38.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.615+0000 7fe316d9d640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fe3181441e0 0x7fe3180cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:38.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.615+0000 7fe30ffff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3180b3ff0 0x7fe3180ca910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:41794/0 (socket says 172.21.15.154:41794) 2023-09-23T03:28:38.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.615+0000 7fe30ffff640 1 -- 172.21.15.154:0/2471334835 learned_addr learned my addr 172.21.15.154:0/2471334835 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:38.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.616+0000 7fe30ffff640 1 -- 172.21.15.154:0/2471334835 >> v2:172.21.15.190:3300/0 conn(0x7fe3181441e0 msgr2=0x7fe3180cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:38.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.616+0000 7fe30ffff640 1 --2- 172.21.15.154:0/2471334835 >> v2:172.21.15.190:3300/0 conn(0x7fe3181441e0 0x7fe3180cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.616+0000 7fe30ffff640 1 -- 172.21.15.154:0/2471334835 >> v1:172.21.15.190:6789/0 conn(0x7fe318143e00 legacy=0x7fe3180b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:38.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.616+0000 7fe30ffff640 1 -- 172.21.15.154:0/2471334835 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3180cbc40 con 0x7fe3180b3ff0 2023-09-23T03:28:38.618 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.617+0000 7fe30ffff640 1 --2- 172.21.15.154:0/2471334835 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3180b3ff0 0x7fe3180ca910 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe300009870 tx=0x7fe30002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=614a5644a9a3a957 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:38.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.618+0000 7fe315d9b640 1 -- 172.21.15.154:0/2471334835 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe300030840 con 0x7fe3180b3ff0 2023-09-23T03:28:38.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.618+0000 7fe315d9b640 1 -- 172.21.15.154:0/2471334835 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3000309a0 con 0x7fe3180b3ff0 2023-09-23T03:28:38.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.618+0000 7fe315d9b640 1 -- 172.21.15.154:0/2471334835 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe300030d30 con 0x7fe3180b3ff0 2023-09-23T03:28:38.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.619+0000 7fe31c9d7640 1 -- 172.21.15.154:0/2471334835 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3180b3ff0 msgr2=0x7fe3180ca910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:38.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.619+0000 7fe31c9d7640 1 --2- 172.21.15.154:0/2471334835 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3180b3ff0 0x7fe3180ca910 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe300009870 tx=0x7fe30002b980 comp rx=0 tx=0).stop 2023-09-23T03:28:38.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.619+0000 7fe31c9d7640 1 -- 172.21.15.154:0/2471334835 shutdown_connections 2023-09-23T03:28:38.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.619+0000 7fe31c9d7640 1 --2- 172.21.15.154:0/2471334835 >> v2:172.21.15.190:3300/0 conn(0x7fe3181441e0 0x7fe3180cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.619+0000 7fe31c9d7640 1 --2- 172.21.15.154:0/2471334835 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3180b3ff0 0x7fe3180ca910 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.619+0000 7fe31c9d7640 1 -- 172.21.15.154:0/2471334835 >> 172.21.15.154:0/2471334835 conn(0x7fe318069cd0 msgr2=0x7fe31806a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:38.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.619+0000 7fe31c9d7640 1 -- 172.21.15.154:0/2471334835 shutdown_connections 2023-09-23T03:28:38.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.619+0000 7fe31c9d7640 1 -- 172.21.15.154:0/2471334835 wait complete. 2023-09-23T03:28:38.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.621+0000 7fe31c9d7640 1 Processor -- start 2023-09-23T03:28:38.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.621+0000 7fe31c9d7640 1 -- start start 2023-09-23T03:28:38.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.623+0000 7fe31c9d7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3180b3ff0 0x7fe31815b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:38.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.623+0000 7fe31c9d7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe318143e00 0x7fe31815d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:38.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.623+0000 7fe31c9d7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3181441e0 0x7fe3180abac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:38.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.623+0000 7fe31c9d7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31815c110 con 0x7fe318143e00 2023-09-23T03:28:38.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.623+0000 7fe31c9d7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3180ac9e0 con 0x7fe3180b3ff0 2023-09-23T03:28:38.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.623+0000 7fe31c9d7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe31815dd50 con 0x7fe3181441e0 2023-09-23T03:28:38.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.623+0000 7fe30ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe318143e00 0x7fe31815d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:38.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.623+0000 7fe30ffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe318143e00 0x7fe31815d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49094/0 (socket says 172.21.15.154:49094) 2023-09-23T03:28:38.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.623+0000 7fe30ffff640 1 -- 172.21.15.154:0/1542649916 learned_addr learned my addr 172.21.15.154:0/1542649916 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:38.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.623+0000 7fe31659c640 1 --2- 172.21.15.154:0/1542649916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3180b3ff0 0x7fe31815b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:38.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.623+0000 7fe316d9d640 1 --2- 172.21.15.154:0/1542649916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3181441e0 0x7fe3180abac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:38.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.623+0000 7fe30ffff640 1 -- 172.21.15.154:0/1542649916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3181441e0 msgr2=0x7fe3180abac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:38.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.623+0000 7fe30ffff640 1 --2- 172.21.15.154:0/1542649916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3181441e0 0x7fe3180abac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.623+0000 7fe30ffff640 1 -- 172.21.15.154:0/1542649916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3180b3ff0 msgr2=0x7fe31815b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:38.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.624+0000 7fe30ffff640 1 --2- 172.21.15.154:0/1542649916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3180b3ff0 0x7fe31815b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.624+0000 7fe30ffff640 1 -- 172.21.15.154:0/1542649916 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3000094d0 con 0x7fe318143e00 2023-09-23T03:28:38.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.624+0000 7fe30ffff640 1 --2- 172.21.15.154:0/1542649916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe318143e00 0x7fe31815d810 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fe300000c00 tx=0x7fe300004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:38.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.624+0000 7fe30effd640 1 -- 172.21.15.154:0/1542649916 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe300039c90 con 0x7fe318143e00 2023-09-23T03:28:38.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.624+0000 7fe30effd640 1 -- 172.21.15.154:0/1542649916 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe300039df0 con 0x7fe318143e00 2023-09-23T03:28:38.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.624+0000 7fe31c9d7640 1 -- 172.21.15.154:0/1542649916 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe31815be30 con 0x7fe318143e00 2023-09-23T03:28:38.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.624+0000 7fe30effd640 1 -- 172.21.15.154:0/1542649916 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe300043640 con 0x7fe318143e00 2023-09-23T03:28:38.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.624+0000 7fe31c9d7640 1 -- 172.21.15.154:0/1542649916 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3180acf80 con 0x7fe318143e00 2023-09-23T03:28:38.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.625+0000 7fe31c9d7640 1 -- 172.21.15.154:0/1542649916 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe318073eb0 con 0x7fe318143e00 2023-09-23T03:28:38.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.626+0000 7fe30effd640 1 -- 172.21.15.154:0/1542649916 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fe3000437a0 con 0x7fe318143e00 2023-09-23T03:28:38.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.627+0000 7fe30effd640 1 --2- 172.21.15.154:0/1542649916 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe2f806af30 0x7fe2f806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:38.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.628+0000 7fe30effd640 1 -- 172.21.15.154:0/1542649916 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6201+0+0 (secure 0 0 0) 0x7fe30003e070 con 0x7fe318143e00 2023-09-23T03:28:38.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.628+0000 7fe31659c640 1 --2- 172.21.15.154:0/1542649916 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe2f806af30 0x7fe2f806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:38.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.629+0000 7fe31659c640 1 --2- 172.21.15.154:0/1542649916 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe2f806af30 0x7fe2f806d3f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe308000f60 tx=0x7fe308009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:38.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.632+0000 7fe30effd640 1 -- 172.21.15.154:0/1542649916 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fe3000777d0 con 0x7fe318143e00 2023-09-23T03:28:38.857 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.856+0000 7fe31c9d7640 1 -- 172.21.15.154:0/1542649916 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe3180cd740 con 0x7fe318143e00 2023-09-23T03:28:38.858 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.857+0000 7fe30effd640 1 -- 172.21.15.154:0/1542649916 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v121) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe30007adf0 con 0x7fe318143e00 2023-09-23T03:28:38.858 INFO:teuthology.orchestra.run.smithi154.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-09-23T03:28:38.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.859+0000 7fe31c9d7640 1 -- 172.21.15.154:0/1542649916 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe2f806af30 msgr2=0x7fe2f806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:38.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.859+0000 7fe31c9d7640 1 --2- 172.21.15.154:0/1542649916 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe2f806af30 0x7fe2f806d3f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe308000f60 tx=0x7fe308009040 comp rx=0 tx=0).stop 2023-09-23T03:28:38.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.859+0000 7fe31c9d7640 1 -- 172.21.15.154:0/1542649916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe318143e00 msgr2=0x7fe31815d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:38.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.859+0000 7fe31c9d7640 1 --2- 172.21.15.154:0/1542649916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe318143e00 0x7fe31815d810 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fe300000c00 tx=0x7fe300004730 comp rx=0 tx=0).stop 2023-09-23T03:28:38.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.859+0000 7fe31c9d7640 1 -- 172.21.15.154:0/1542649916 shutdown_connections 2023-09-23T03:28:38.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.859+0000 7fe31c9d7640 1 --2- 172.21.15.154:0/1542649916 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fe2f806af30 0x7fe2f806d3f0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.859+0000 7fe31c9d7640 1 --2- 172.21.15.154:0/1542649916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fe3181441e0 0x7fe3180abac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.861 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.859+0000 7fe31c9d7640 1 --2- 172.21.15.154:0/1542649916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe318143e00 0x7fe31815d810 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.859+0000 7fe31c9d7640 1 --2- 172.21.15.154:0/1542649916 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fe3180b3ff0 0x7fe31815b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:38.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.859+0000 7fe31c9d7640 1 -- 172.21.15.154:0/1542649916 >> 172.21.15.154:0/1542649916 conn(0x7fe318069cd0 msgr2=0x7fe3180ca7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:38.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.859+0000 7fe31c9d7640 1 -- 172.21.15.154:0/1542649916 shutdown_connections 2023-09-23T03:28:38.862 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:38.860+0000 7fe31c9d7640 1 -- 172.21.15.154:0/1542649916 wait complete. 2023-09-23T03:28:38.878 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-23T03:28:38.878 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-23T03:28:38.878 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi154.front.sepia.ceph.com 2023-09-23T03:28:38.878 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-09-23T03:28:38.878 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-23T03:28:38.878 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-23T03:28:38.878 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-23T03:28:38.878 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-23T03:28:38.878 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-23T03:28:38.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:38.879 DEBUG:teuthology.orchestra.run.smithi154:> ip addr 2023-09-23T03:28:38.931 INFO:teuthology.orchestra.run.smithi154.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-23T03:28:38.931 INFO:teuthology.orchestra.run.smithi154.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-23T03:28:38.931 INFO:teuthology.orchestra.run.smithi154.stdout: inet 127.0.0.1/8 scope host lo 2023-09-23T03:28:38.931 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:28:38.931 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 ::1/128 scope host 2023-09-23T03:28:38.931 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:28:38.932 INFO:teuthology.orchestra.run.smithi154.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:28:38.932 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:54 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:28:38.932 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f0 2023-09-23T03:28:38.932 INFO:teuthology.orchestra.run.smithi154.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:28:38.932 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:d9:1a:55 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:28:38.932 INFO:teuthology.orchestra.run.smithi154.stdout: altname enp7s0f1 2023-09-23T03:28:38.932 INFO:teuthology.orchestra.run.smithi154.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-23T03:28:38.932 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:60 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:28:38.932 INFO:teuthology.orchestra.run.smithi154.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-23T03:28:38.932 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 0c:c4:7a:88:77:61 brd ff:ff:ff:ff:ff:ff 2023-09-23T03:28:38.933 INFO:teuthology.orchestra.run.smithi154.stdout: inet 172.21.15.154/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-09-23T03:28:38.933 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft 41606sec preferred_lft 41606sec 2023-09-23T03:28:38.933 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::ec4:7aff:fe88:7761/64 scope link 2023-09-23T03:28:38.933 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:28:38.933 INFO:teuthology.orchestra.run.smithi154.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-09-23T03:28:38.933 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 5a:c6:b2:96:fb:dc brd ff:ff:ff:ff:ff:ff 2023-09-23T03:28:38.933 INFO:teuthology.orchestra.run.smithi154.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-09-23T03:28:38.933 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:28:38.933 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::a801:22ff:febf:2833/64 scope link 2023-09-23T03:28:38.933 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:28:38.934 INFO:teuthology.orchestra.run.smithi154.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-23T03:28:38.934 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether 5a:c6:b2:96:fb:dc brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:28:38.934 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::58c6:b2ff:fe96:fbdc/64 scope link 2023-09-23T03:28:38.934 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:28:38.934 INFO:teuthology.orchestra.run.smithi154.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-09-23T03:28:38.934 INFO:teuthology.orchestra.run.smithi154.stdout: link/ether b2:23:b0:e4:c2:ab brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:28:38.934 INFO:teuthology.orchestra.run.smithi154.stdout: inet6 fe80::b023:b0ff:fee4:c2ab/64 scope link 2023-09-23T03:28:38.934 INFO:teuthology.orchestra.run.smithi154.stdout: valid_lft forever preferred_lft forever 2023-09-23T03:28:38.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:38.935 DEBUG:teuthology.orchestra.run.smithi154:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-23T03:28:39.023 INFO:teuthology.orchestra.run.smithi154.stdout:1 2023-09-23T03:28:39.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:39.028 DEBUG:teuthology.orchestra.run.smithi154:> route 2023-09-23T03:28:39.081 INFO:teuthology.orchestra.run.smithi154.stdout:Kernel IP routing table 2023-09-23T03:28:39.081 INFO:teuthology.orchestra.run.smithi154.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-23T03:28:39.081 INFO:teuthology.orchestra.run.smithi154.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-09-23T03:28:39.081 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-09-23T03:28:39.081 INFO:teuthology.orchestra.run.smithi154.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-23T03:28:39.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:39.082 DEBUG:teuthology.orchestra.run.smithi154:> (cd / && exec bash -c ' 2023-09-23T03:28:39.082 DEBUG:teuthology.orchestra.run.smithi154:> set -e 2023-09-23T03:28:39.082 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-09-23T03:28:39.082 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-09-23T03:28:39.082 DEBUG:teuthology.orchestra.run.smithi154:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-09-23T03:28:39.082 DEBUG:teuthology.orchestra.run.smithi154:> ') 2023-09-23T03:28:39.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:39.265 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2023-09-23T03:28:39.315 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-09-23T03:28:39.315 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-09-23T03:28:39.316 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-23T03:28:39.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:39.316 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link set brx.0 up 2023-09-23T03:28:39.416 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-09-23T03:28:39.417 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-23T03:28:39.417 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-09-23T03:28:39.470 INFO:teuthology.orchestra.run.smithi154.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-09-23T03:28:39.470 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-23T03:28:39.470 DEBUG:teuthology.orchestra.run.smithi154:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-09-23T03:28:39.518 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-09-23T03:28:39.617 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2023-09-23T03:28:39.676 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2023-09-23T03:28:39.676 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2023-09-23T03:28:39.676 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2023-09-23T03:28:39.677 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2023-09-23T03:28:39.677 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2023-09-23T03:28:39.677 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2023-09-23T03:28:39.677 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2023-09-23T03:28:39.677 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2023-09-23T03:28:39.677 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2023-09-23T03:28:39.677 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2023-09-23T03:28:39.677 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2023-09-23T03:28:39.677 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T03:28:39.677 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2023-09-23T03:28:39.677 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2023-09-23T03:28:39.678 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2023-09-23T03:28:39.678 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2023-09-23T03:28:39.678 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2023-09-23T03:28:39.678 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2023-09-23T03:28:39.678 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2023-09-23T03:28:39.678 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-09-23T03:28:39.678 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-09-23T03:28:39.678 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2023-09-23T03:28:39.678 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2023-09-23T03:28:39.678 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2023-09-23T03:28:39.678 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-0 2023-09-23T03:28:39.679 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-1 2023-09-23T03:28:39.679 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-2 2023-09-23T03:28:39.679 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-3 2023-09-23T03:28:39.679 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2023-09-23T03:28:39.679 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:28:39.679 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:28:39.679 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:28:39.679 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:28:39.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:39.680 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-09-23T03:28:39.732 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-23T03:28:39.732 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-09-23T03:28:39.738 DEBUG:teuthology.orchestra.run.smithi154:> sudo modprobe fuse 2023-09-23T03:28:39.818 DEBUG:teuthology.orchestra.run.smithi154:> cat /proc/self/mounts | awk '{print $2}' 2023-09-23T03:28:39.861 INFO:teuthology.orchestra.run.smithi154.stdout:/proc 2023-09-23T03:28:39.861 INFO:teuthology.orchestra.run.smithi154.stdout:/sys 2023-09-23T03:28:39.861 INFO:teuthology.orchestra.run.smithi154.stdout:/dev 2023-09-23T03:28:39.861 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/security 2023-09-23T03:28:39.861 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/shm 2023-09-23T03:28:39.861 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/pts 2023-09-23T03:28:39.861 INFO:teuthology.orchestra.run.smithi154.stdout:/run 2023-09-23T03:28:39.862 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/cgroup 2023-09-23T03:28:39.862 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/pstore 2023-09-23T03:28:39.862 INFO:teuthology.orchestra.run.smithi154.stdout:/ 2023-09-23T03:28:39.862 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/selinux 2023-09-23T03:28:39.862 INFO:teuthology.orchestra.run.smithi154.stdout:/proc/sys/fs/binfmt_misc 2023-09-23T03:28:39.862 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/mqueue 2023-09-23T03:28:39.862 INFO:teuthology.orchestra.run.smithi154.stdout:/dev/hugepages 2023-09-23T03:28:39.862 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/debug 2023-09-23T03:28:39.862 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/tracing 2023-09-23T03:28:39.862 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/fs/fuse/connections 2023-09-23T03:28:39.863 INFO:teuthology.orchestra.run.smithi154.stdout:/sys/kernel/config 2023-09-23T03:28:39.863 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-sysctl.service 2023-09-23T03:28:39.863 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-09-23T03:28:39.863 INFO:teuthology.orchestra.run.smithi154.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-09-23T03:28:39.863 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/nfs/rpc_pipefs 2023-09-23T03:28:39.863 INFO:teuthology.orchestra.run.smithi154.stdout:/run/user/1000 2023-09-23T03:28:39.863 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph 2023-09-23T03:28:39.863 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-0 2023-09-23T03:28:39.863 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-1 2023-09-23T03:28:39.864 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-2 2023-09-23T03:28:39.864 INFO:teuthology.orchestra.run.smithi154.stdout:/var/lib/ceph/osd/ceph-3 2023-09-23T03:28:39.864 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns 2023-09-23T03:28:39.864 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:28:39.864 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:28:39.864 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:28:39.864 INFO:teuthology.orchestra.run.smithi154.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:28:39.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:39.865 DEBUG:teuthology.orchestra.run.smithi154:> ls /sys/fs/fuse/connections 2023-09-23T03:28:39.932 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:2023-09-23T03:28:39.931+0000 7f912f756480 -1 init, newargv = 0x55c0118b5b40 newargc=15 2023-09-23T03:28:39.933 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:ceph-fuse[130024]: starting ceph client 2023-09-23T03:28:39.942 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:ceph-fuse[130024]: starting fuse 2023-09-23T03:28:39.954 INFO:teuthology.orchestra.run.smithi154.stdout:39 2023-09-23T03:28:39.954 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-09-23T03:28:39.955 DEBUG:teuthology.orchestra.run.smithi154:> sudo stdin-killer -- python3 -c ' 2023-09-23T03:28:39.955 DEBUG:teuthology.orchestra.run.smithi154:> import glob 2023-09-23T03:28:39.955 DEBUG:teuthology.orchestra.run.smithi154:> import re 2023-09-23T03:28:39.955 DEBUG:teuthology.orchestra.run.smithi154:> import os 2023-09-23T03:28:39.955 DEBUG:teuthology.orchestra.run.smithi154:> import subprocess 2023-09-23T03:28:39.955 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:28:39.955 DEBUG:teuthology.orchestra.run.smithi154:> def _find_admin_socket(client_name): 2023-09-23T03:28:39.955 DEBUG:teuthology.orchestra.run.smithi154:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-09-23T03:28:39.955 DEBUG:teuthology.orchestra.run.smithi154:> files = glob.glob(asok_path) 2023-09-23T03:28:39.955 DEBUG:teuthology.orchestra.run.smithi154:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-09-23T03:28:39.956 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:28:39.956 DEBUG:teuthology.orchestra.run.smithi154:> # Given a non-glob path, it better be there 2023-09-23T03:28:39.956 DEBUG:teuthology.orchestra.run.smithi154:> if "*" not in asok_path: 2023-09-23T03:28:39.956 DEBUG:teuthology.orchestra.run.smithi154:> assert(len(files) == 1) 2023-09-23T03:28:39.956 DEBUG:teuthology.orchestra.run.smithi154:> return files[0] 2023-09-23T03:28:39.956 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:28:39.956 DEBUG:teuthology.orchestra.run.smithi154:> for f in files: 2023-09-23T03:28:39.956 DEBUG:teuthology.orchestra.run.smithi154:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-23T03:28:39.956 DEBUG:teuthology.orchestra.run.smithi154:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-23T03:28:39.956 DEBUG:teuthology.orchestra.run.smithi154:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-23T03:28:39.956 DEBUG:teuthology.orchestra.run.smithi154:> contents = proc_f.read() 2023-09-23T03:28:39.956 DEBUG:teuthology.orchestra.run.smithi154:> if mountpoint in contents: 2023-09-23T03:28:39.956 DEBUG:teuthology.orchestra.run.smithi154:> return f 2023-09-23T03:28:39.957 DEBUG:teuthology.orchestra.run.smithi154:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-23T03:28:39.957 DEBUG:teuthology.orchestra.run.smithi154:> 2023-09-23T03:28:39.957 DEBUG:teuthology.orchestra.run.smithi154:> print(_find_admin_socket("client.0")) 2023-09-23T03:28:39.957 DEBUG:teuthology.orchestra.run.smithi154:> ' 2023-09-23T03:28:40.155 INFO:teuthology.orchestra.run.smithi154.stdout:/var/run/ceph/ceph-client.0.130024.asok 2023-09-23T03:28:40.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-23T03:28:40.171 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.130024.asok 2023-09-23T03:28:40.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:40.172 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.130024.asok status 2023-09-23T03:28:40.401 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:28:40.401 INFO:teuthology.orchestra.run.smithi154.stdout: "metadata": { 2023-09-23T03:28:40.402 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_sha1": "862663913a74a99e1c52d50e6a7e59dd6705749a", 2023-09-23T03:28:40.402 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_version": "ceph version 18.0.0-6336-g86266391 (862663913a74a99e1c52d50e6a7e59dd6705749a) reef (dev)", 2023-09-23T03:28:40.402 INFO:teuthology.orchestra.run.smithi154.stdout: "entity_id": "0", 2023-09-23T03:28:40.402 INFO:teuthology.orchestra.run.smithi154.stdout: "hostname": "smithi154", 2023-09-23T03:28:40.402 INFO:teuthology.orchestra.run.smithi154.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-09-23T03:28:40.402 INFO:teuthology.orchestra.run.smithi154.stdout: "pid": "130024", 2023-09-23T03:28:40.403 INFO:teuthology.orchestra.run.smithi154.stdout: "root": "/" 2023-09-23T03:28:40.403 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:28:40.403 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_count": 0, 2023-09-23T03:28:40.403 INFO:teuthology.orchestra.run.smithi154.stdout: "dentry_pinned_count": 0, 2023-09-23T03:28:40.403 INFO:teuthology.orchestra.run.smithi154.stdout: "id": 9423, 2023-09-23T03:28:40.403 INFO:teuthology.orchestra.run.smithi154.stdout: "inst": { 2023-09-23T03:28:40.404 INFO:teuthology.orchestra.run.smithi154.stdout: "name": { 2023-09-23T03:28:40.404 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "client", 2023-09-23T03:28:40.404 INFO:teuthology.orchestra.run.smithi154.stdout: "num": 9423 2023-09-23T03:28:40.404 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:28:40.404 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2023-09-23T03:28:40.404 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2023-09-23T03:28:40.405 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.1:0", 2023-09-23T03:28:40.405 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 242719024 2023-09-23T03:28:40.405 INFO:teuthology.orchestra.run.smithi154.stdout: } 2023-09-23T03:28:40.405 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:28:40.405 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": { 2023-09-23T03:28:40.405 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2023-09-23T03:28:40.406 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "192.168.0.1:0", 2023-09-23T03:28:40.406 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 242719024 2023-09-23T03:28:40.406 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2023-09-23T03:28:40.406 INFO:teuthology.orchestra.run.smithi154.stdout: "inst_str": "client.9423 192.168.0.1:0/242719024", 2023-09-23T03:28:40.406 INFO:teuthology.orchestra.run.smithi154.stdout: "addr_str": "192.168.0.1:0/242719024", 2023-09-23T03:28:40.406 INFO:teuthology.orchestra.run.smithi154.stdout: "inode_count": 1, 2023-09-23T03:28:40.407 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_epoch": 121, 2023-09-23T03:28:40.407 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch": 128, 2023-09-23T03:28:40.407 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_epoch_barrier": 0, 2023-09-23T03:28:40.407 INFO:teuthology.orchestra.run.smithi154.stdout: "blocklisted": false, 2023-09-23T03:28:40.407 INFO:teuthology.orchestra.run.smithi154.stdout: "fs_name": "cephfs" 2023-09-23T03:28:40.407 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:28:40.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:40.416 DEBUG:teuthology.orchestra.run.smithi154:> stat --file-system '--printf=%T 2023-09-23T03:28:40.416 DEBUG:teuthology.orchestra.run.smithi154:> ' -- /home/ubuntu/cephtest/mnt.0 2023-09-23T03:28:40.470 INFO:teuthology.orchestra.run.smithi154.stdout:fuseblk 2023-09-23T03:28:40.470 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-09-23T03:28:40.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:40.471 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-09-23T03:28:40.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:40.567 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -l subdir/) 2023-09-23T03:28:40.755 INFO:teuthology.orchestra.run.smithi154.stdout:total 50 2023-09-23T03:28:40.756 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 1 Sep 23 03:27 0 2023-09-23T03:28:40.756 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 1 Sep 23 03:27 1 2023-09-23T03:28:40.756 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 10 2023-09-23T03:28:40.756 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 11 2023-09-23T03:28:40.756 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 12 2023-09-23T03:28:40.756 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 13 2023-09-23T03:28:40.756 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 14 2023-09-23T03:28:40.756 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 15 2023-09-23T03:28:40.756 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 16 2023-09-23T03:28:40.756 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 17 2023-09-23T03:28:40.756 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 18 2023-09-23T03:28:40.756 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 19 2023-09-23T03:28:40.757 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 1 Sep 23 03:27 2 2023-09-23T03:28:40.757 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 20 2023-09-23T03:28:40.757 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 22 2023-09-23T03:28:40.757 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 23 2023-09-23T03:28:40.757 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 24 2023-09-23T03:28:40.757 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 25 2023-09-23T03:28:40.757 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 26 2023-09-23T03:28:40.757 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 27 2023-09-23T03:28:40.757 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 28 2023-09-23T03:28:40.757 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 29 2023-09-23T03:28:40.757 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 1 Sep 23 03:27 3 2023-09-23T03:28:40.757 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 30 2023-09-23T03:28:40.758 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 31 2023-09-23T03:28:40.758 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 32 2023-09-23T03:28:40.758 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 33 2023-09-23T03:28:40.758 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 34 2023-09-23T03:28:40.758 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 35 2023-09-23T03:28:40.758 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 36 2023-09-23T03:28:40.758 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 37 2023-09-23T03:28:40.758 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 38 2023-09-23T03:28:40.758 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 39 2023-09-23T03:28:40.758 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 1 Sep 23 03:27 4 2023-09-23T03:28:40.758 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 40 2023-09-23T03:28:40.758 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 41 2023-09-23T03:28:40.759 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 42 2023-09-23T03:28:40.759 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 43 2023-09-23T03:28:40.759 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 44 2023-09-23T03:28:40.759 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 45 2023-09-23T03:28:40.759 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 46 2023-09-23T03:28:40.759 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 47 2023-09-23T03:28:40.759 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 48 2023-09-23T03:28:40.759 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 49 2023-09-23T03:28:40.759 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 1 Sep 23 03:27 5 2023-09-23T03:28:40.759 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 50 2023-09-23T03:28:40.759 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 51 2023-09-23T03:28:40.759 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 52 2023-09-23T03:28:40.760 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 53 2023-09-23T03:28:40.760 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 54 2023-09-23T03:28:40.760 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 55 2023-09-23T03:28:40.760 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 56 2023-09-23T03:28:40.760 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 57 2023-09-23T03:28:40.760 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 58 2023-09-23T03:28:40.760 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 59 2023-09-23T03:28:40.760 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 1 Sep 23 03:27 6 2023-09-23T03:28:40.760 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 60 2023-09-23T03:28:40.760 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 61 2023-09-23T03:28:40.760 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 62 2023-09-23T03:28:40.760 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 63 2023-09-23T03:28:40.760 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 64 2023-09-23T03:28:40.761 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 65 2023-09-23T03:28:40.761 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 66 2023-09-23T03:28:40.761 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 67 2023-09-23T03:28:40.761 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 68 2023-09-23T03:28:40.761 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 69 2023-09-23T03:28:40.761 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 1 Sep 23 03:27 7 2023-09-23T03:28:40.761 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 70 2023-09-23T03:28:40.761 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 71 2023-09-23T03:28:40.761 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 72 2023-09-23T03:28:40.761 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 73 2023-09-23T03:28:40.761 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 74 2023-09-23T03:28:40.761 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 75 2023-09-23T03:28:40.762 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 76 2023-09-23T03:28:40.762 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 77 2023-09-23T03:28:40.762 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 78 2023-09-23T03:28:40.762 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 79 2023-09-23T03:28:40.762 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 1 Sep 23 03:27 8 2023-09-23T03:28:40.762 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 80 2023-09-23T03:28:40.762 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 81 2023-09-23T03:28:40.762 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 82 2023-09-23T03:28:40.762 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 83 2023-09-23T03:28:40.762 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 84 2023-09-23T03:28:40.762 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 85 2023-09-23T03:28:40.762 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 86 2023-09-23T03:28:40.763 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 87 2023-09-23T03:28:40.763 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 88 2023-09-23T03:28:40.763 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 89 2023-09-23T03:28:40.763 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 1 Sep 23 03:27 9 2023-09-23T03:28:40.763 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 90 2023-09-23T03:28:40.763 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 91 2023-09-23T03:28:40.763 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 92 2023-09-23T03:28:40.763 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 93 2023-09-23T03:28:40.763 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 94 2023-09-23T03:28:40.763 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 95 2023-09-23T03:28:40.763 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 96 2023-09-23T03:28:40.763 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 97 2023-09-23T03:28:40.764 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 98 2023-09-23T03:28:40.764 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Sep 23 03:27 99 2023-09-23T03:28:40.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:40.765 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo cat subdir/21') 2023-09-23T03:28:40.863 INFO:teuthology.orchestra.run.smithi154.stderr:cat: subdir/21: No such file or directory 2023-09-23T03:28:40.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T03:28:40.867 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-09-23T03:28:40.868 DEBUG:teuthology.orchestra.run.smithi154:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-23T03:28:40.978 DEBUG:teuthology.orchestra.run.smithi190:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-09-23T03:28:41.104 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-09-23T03:28:41.104 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:28:41.104 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:28:41.160 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:28:41.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.364+0000 7f3790eaf640 1 Processor -- start 2023-09-23T03:28:41.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.364+0000 7f3790eaf640 1 -- start start 2023-09-23T03:28:41.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.366+0000 7f3790eaf640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f378c0ad9c0 0x7f378c0c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:41.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.366+0000 7f3790eaf640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f378c070b40 con 0x7f378c15ad70 2023-09-23T03:28:41.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.366+0000 7f3790eaf640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f378c070e20 con 0x7f378c0b0780 2023-09-23T03:28:41.367 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.366+0000 7f3790eaf640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f378c071100 con 0x7f378c0ad9c0 2023-09-23T03:28:41.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.366+0000 7f3783fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f378c0ad9c0 0x7f378c0c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:41.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.366+0000 7f378a59c640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f378c15ad70 0x7f378c0b0070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:41428/0 (socket says 172.21.15.154:41428) 2023-09-23T03:28:41.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.366+0000 7f378a59c640 1 -- 172.21.15.154:0/3862458964 learned_addr learned my addr 172.21.15.154:0/3862458964 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:28:41.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.366+0000 7f3783fff640 1 -- 172.21.15.154:0/3862458964 >> v1:172.21.15.190:6789/0 conn(0x7f378c0b0780 legacy=0x7f378c06c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:41.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.366+0000 7f3783fff640 1 -- 172.21.15.154:0/3862458964 >> v1:172.21.15.154:6789/0 conn(0x7f378c15ad70 legacy=0x7f378c0b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:41.368 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.367+0000 7f3783fff640 1 -- 172.21.15.154:0/3862458964 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f378c0c96d0 con 0x7f378c0ad9c0 2023-09-23T03:28:41.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.366+0000 7f3789d9b640 1 -- 172.21.15.154:0/3862458964 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1152205985 0 0) 0x7f378c070b40 con 0x7f378c15ad70 2023-09-23T03:28:41.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.368+0000 7f3783fff640 1 --2- 172.21.15.154:0/3862458964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f378c0ad9c0 0x7f378c0c8e50 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f37740097d0 tx=0x7f377402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4816b66c7623ba7 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:41.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.368+0000 7f3789d9b640 1 -- 172.21.15.154:0/3862458964 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3774031830 con 0x7f378c0ad9c0 2023-09-23T03:28:41.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.368+0000 7f3789d9b640 1 -- 172.21.15.154:0/3862458964 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3774031990 con 0x7f378c0ad9c0 2023-09-23T03:28:41.369 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.368+0000 7f3789d9b640 1 -- 172.21.15.154:0/3862458964 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3774031d20 con 0x7f378c0ad9c0 2023-09-23T03:28:41.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.369+0000 7f3790eaf640 1 -- 172.21.15.154:0/3862458964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f378c0ad9c0 msgr2=0x7f378c0c8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:41.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.369+0000 7f3790eaf640 1 --2- 172.21.15.154:0/3862458964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f378c0ad9c0 0x7f378c0c8e50 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f37740097d0 tx=0x7f377402b8d0 comp rx=0 tx=0).stop 2023-09-23T03:28:41.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.369+0000 7f3790eaf640 1 -- 172.21.15.154:0/3862458964 shutdown_connections 2023-09-23T03:28:41.370 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.369+0000 7f3790eaf640 1 --2- 172.21.15.154:0/3862458964 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f378c0ad9c0 0x7f378c0c8e50 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:41.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.369+0000 7f3790eaf640 1 -- 172.21.15.154:0/3862458964 >> 172.21.15.154:0/3862458964 conn(0x7f378c0cb070 msgr2=0x7f378c0cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:41.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.369+0000 7f3790eaf640 1 -- 172.21.15.154:0/3862458964 shutdown_connections 2023-09-23T03:28:41.371 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.369+0000 7f3790eaf640 1 -- 172.21.15.154:0/3862458964 wait complete. 2023-09-23T03:28:41.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.371+0000 7f3790eaf640 1 Processor -- start 2023-09-23T03:28:41.372 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.371+0000 7f3790eaf640 1 -- start start 2023-09-23T03:28:41.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.372+0000 7f3790eaf640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f378c0ad9c0 0x7f378c0af590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:41.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.372+0000 7f3790eaf640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f378c0b0780 0x7f378c15db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:41.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.372+0000 7f3790eaf640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f378c15ad70 0x7f378c1ef970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:41.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.372+0000 7f3790eaf640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f378c15c780 con 0x7f378c15ad70 2023-09-23T03:28:41.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.372+0000 7f3790eaf640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f378c1f39b0 con 0x7f378c0ad9c0 2023-09-23T03:28:41.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.372+0000 7f3790eaf640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f378c1f2e30 con 0x7f378c0b0780 2023-09-23T03:28:41.374 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.372+0000 7f378ad9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f378c15ad70 0x7f378c1ef970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:41.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.372+0000 7f378ad9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f378c15ad70 0x7f378c1ef970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49114/0 (socket says 172.21.15.154:49114) 2023-09-23T03:28:41.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.372+0000 7f378ad9d640 1 -- 172.21.15.154:0/3308148732 learned_addr learned my addr 172.21.15.154:0/3308148732 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:41.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.372+0000 7f3783fff640 1 --2- 172.21.15.154:0/3308148732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f378c0b0780 0x7f378c15db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:41.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.373+0000 7f378a59c640 1 --2- 172.21.15.154:0/3308148732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f378c0ad9c0 0x7f378c0af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:41.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.373+0000 7f378ad9d640 1 -- 172.21.15.154:0/3308148732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f378c0b0780 msgr2=0x7f378c15db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:41.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.373+0000 7f378ad9d640 1 --2- 172.21.15.154:0/3308148732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f378c0b0780 0x7f378c15db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:41.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.373+0000 7f378ad9d640 1 -- 172.21.15.154:0/3308148732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f378c0ad9c0 msgr2=0x7f378c0af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:41.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.373+0000 7f378ad9d640 1 --2- 172.21.15.154:0/3308148732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f378c0ad9c0 0x7f378c0af590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:41.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.373+0000 7f378ad9d640 1 -- 172.21.15.154:0/3308148732 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3774009400 con 0x7f378c15ad70 2023-09-23T03:28:41.375 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.373+0000 7f378ad9d640 1 --2- 172.21.15.154:0/3308148732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f378c15ad70 0x7f378c1ef970 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f378400c780 tx=0x7f378400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:41.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.373+0000 7f3782ffd640 1 -- 172.21.15.154:0/3308148732 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3784003d80 con 0x7f378c15ad70 2023-09-23T03:28:41.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.373+0000 7f3782ffd640 1 -- 172.21.15.154:0/3308148732 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378400ce30 con 0x7f378c15ad70 2023-09-23T03:28:41.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.373+0000 7f3782ffd640 1 -- 172.21.15.154:0/3308148732 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3784005290 con 0x7f378c15ad70 2023-09-23T03:28:41.376 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.373+0000 7f3790eaf640 1 -- 172.21.15.154:0/3308148732 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f378c1f3110 con 0x7f378c15ad70 2023-09-23T03:28:41.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.374+0000 7f3790eaf640 1 -- 172.21.15.154:0/3308148732 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f378c0c37a0 con 0x7f378c15ad70 2023-09-23T03:28:41.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.375+0000 7f3790eaf640 1 -- 172.21.15.154:0/3308148732 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f378c067870 con 0x7f378c15ad70 2023-09-23T03:28:41.377 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.376+0000 7f3782ffd640 1 -- 172.21.15.154:0/3308148732 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f37840053f0 con 0x7f378c15ad70 2023-09-23T03:28:41.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.376+0000 7f3782ffd640 1 --2- 172.21.15.154:0/3308148732 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f376c06af30 0x7f376c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:41.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.377+0000 7f3782ffd640 1 -- 172.21.15.154:0/3308148732 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f3784088e40 con 0x7f378c15ad70 2023-09-23T03:28:41.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.377+0000 7f378a59c640 1 --2- 172.21.15.154:0/3308148732 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f376c06af30 0x7f376c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:41.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.377+0000 7f378a59c640 1 --2- 172.21.15.154:0/3308148732 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f376c06af30 0x7f376c06d3f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f377c000f60 tx=0x7f377c007c00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:41.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.381+0000 7f3782ffd640 1 -- 172.21.15.154:0/3308148732 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f3784053bf0 con 0x7f378c15ad70 2023-09-23T03:28:41.574 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.572+0000 7f3790eaf640 1 -- 172.21.15.154:0/3308148732 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f378c073eb0 con 0x7f378c15ad70 2023-09-23T03:28:41.576 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.575+0000 7f3782ffd640 1 -- 172.21.15.154:0/3308148732 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+17083 (secure 0 0 0) 0x7f3784017020 con 0x7f378c15ad70 2023-09-23T03:28:41.576 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:28:41.577 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":128,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:28:29.295164+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:27:42.207617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-09-23T03:27:43.473867+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6837/4006549651":"2023-09-24T03:28:29.295137+0000","172.21.15.154:6836/4006549651":"2023-09-24T03:28:29.295137+0000","172.21.15.190:6835/3558096060":"2023-09-24T03:28:08.468628+0000","172.21.15.190:6833/3558096060":"2023-09-24T03:28:08.468628+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:28:41.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.577+0000 7f3790eaf640 1 -- 172.21.15.154:0/3308148732 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f376c06af30 msgr2=0x7f376c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:41.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.577+0000 7f3790eaf640 1 --2- 172.21.15.154:0/3308148732 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f376c06af30 0x7f376c06d3f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f377c000f60 tx=0x7f377c007c00 comp rx=0 tx=0).stop 2023-09-23T03:28:41.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.577+0000 7f3790eaf640 1 -- 172.21.15.154:0/3308148732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f378c15ad70 msgr2=0x7f378c1ef970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:41.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.577+0000 7f3790eaf640 1 --2- 172.21.15.154:0/3308148732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f378c15ad70 0x7f378c1ef970 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f378400c780 tx=0x7f378400cc40 comp rx=0 tx=0).stop 2023-09-23T03:28:41.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.577+0000 7f3790eaf640 1 -- 172.21.15.154:0/3308148732 shutdown_connections 2023-09-23T03:28:41.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.577+0000 7f3790eaf640 1 --2- 172.21.15.154:0/3308148732 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f376c06af30 0x7f376c06d3f0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:41.579 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.577+0000 7f3790eaf640 1 --2- 172.21.15.154:0/3308148732 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f378c15ad70 0x7f378c1ef970 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:41.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.577+0000 7f3790eaf640 1 --2- 172.21.15.154:0/3308148732 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f378c0b0780 0x7f378c15db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:41.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.577+0000 7f3790eaf640 1 --2- 172.21.15.154:0/3308148732 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f378c0ad9c0 0x7f378c0af590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:41.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.577+0000 7f3790eaf640 1 -- 172.21.15.154:0/3308148732 >> 172.21.15.154:0/3308148732 conn(0x7f378c0cb070 msgr2=0x7f378c0aca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:41.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.578+0000 7f3790eaf640 1 -- 172.21.15.154:0/3308148732 shutdown_connections 2023-09-23T03:28:41.580 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.578+0000 7f3790eaf640 1 -- 172.21.15.154:0/3308148732 wait complete. 2023-09-23T03:28:41.598 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:27:42.207617+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:27:43.473867+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T03:28:41.598 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:28:41.818 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.817+0000 7f7c21a32640 1 Processor -- start 2023-09-23T03:28:41.819 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.817+0000 7f7c21a32640 1 -- start start 2023-09-23T03:28:41.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.819+0000 7f7c21a32640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7c1c159d60 0x7f7c1c0cd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:41.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.819+0000 7f7c21a32640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c1c0adc40 con 0x7f7c1c0b0780 2023-09-23T03:28:41.820 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.819+0000 7f7c21a32640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c1c0cc340 con 0x7f7c1c159980 2023-09-23T03:28:41.821 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.819+0000 7f7c21a32640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c1c0cc620 con 0x7f7c1c159d60 2023-09-23T03:28:41.822 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.820+0000 7f7c127fc640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f7c1c0b0780 0x7f7c1c0cce20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:41444/0 (socket says 172.21.15.154:41444) 2023-09-23T03:28:41.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.820+0000 7f7c127fc640 1 -- 172.21.15.154:0/308030452 learned_addr learned my addr 172.21.15.154:0/308030452 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:28:41.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.820+0000 7f7c1b7fe640 1 --2- 172.21.15.154:0/308030452 >> v2:172.21.15.190:3300/0 conn(0x7f7c1c159d60 0x7f7c1c0cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:41.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.820+0000 7f7c1a7fc640 1 -- 172.21.15.154:0/308030452 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3219936438 0 0) 0x7f7c1c0adc40 con 0x7f7c1c0b0780 2023-09-23T03:28:41.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.820+0000 7f7c1a7fc640 1 -- 172.21.15.154:0/308030452 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c08003520 con 0x7f7c1c0b0780 2023-09-23T03:28:41.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.820+0000 7f7c1a7fc640 1 -- 172.21.15.154:0/308030452 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 176858987 0 0) 0x7f7c08003520 con 0x7f7c1c0b0780 2023-09-23T03:28:41.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.820+0000 7f7c1a7fc640 1 -- 172.21.15.154:0/308030452 >> v2:172.21.15.190:3300/0 conn(0x7f7c1c159d60 msgr2=0x7f7c1c0cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:41.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.820+0000 7f7c1a7fc640 1 --2- 172.21.15.154:0/308030452 >> v2:172.21.15.190:3300/0 conn(0x7f7c1c159d60 0x7f7c1c0cd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:41.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.820+0000 7f7c1a7fc640 1 -- 172.21.15.154:0/308030452 >> v1:172.21.15.190:6789/0 conn(0x7f7c1c159980 legacy=0x7f7c1c0b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:41.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.820+0000 7f7c1a7fc640 1 -- 172.21.15.154:0/308030452 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c1c0cc900 con 0x7f7c1c0b0780 2023-09-23T03:28:41.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.820+0000 7f7c1a7fc640 1 -- 172.21.15.154:0/308030452 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f7c040019f0 con 0x7f7c1c0b0780 2023-09-23T03:28:41.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.820+0000 7f7c1a7fc640 1 -- 172.21.15.154:0/308030452 >> v1:172.21.15.154:6789/0 conn(0x7f7c1c0b0780 legacy=0x7f7c1c0cce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:41.823 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.820+0000 7f7c1a7fc640 1 --2- 172.21.15.154:0/308030452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c080039c0 0x7f7c08003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:41.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.821+0000 7f7c1a7fc640 1 --2- 172.21.15.154:0/308030452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c080047c0 0x7f7c08004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:41.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.821+0000 7f7c1a7fc640 1 --2- 172.21.15.154:0/308030452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c08005180 0x7f7c0800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:41.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.821+0000 7f7c1a7fc640 1 -- 172.21.15.154:0/308030452 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1c0cc620 con 0x7f7c080047c0 2023-09-23T03:28:41.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.821+0000 7f7c1a7fc640 1 -- 172.21.15.154:0/308030452 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0800c080 con 0x7f7c080039c0 2023-09-23T03:28:41.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.821+0000 7f7c1a7fc640 1 -- 172.21.15.154:0/308030452 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0800c240 con 0x7f7c08005180 2023-09-23T03:28:41.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.821+0000 7f7c127fc640 1 --2- 172.21.15.154:0/308030452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c080047c0 0x7f7c08004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:41.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.821+0000 7f7c1affd640 1 --2- 172.21.15.154:0/308030452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c080039c0 0x7f7c08003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:41.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.821+0000 7f7c1b7fe640 1 --2- 172.21.15.154:0/308030452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c08005180 0x7f7c0800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:41.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.821+0000 7f7c127fc640 1 -- 172.21.15.154:0/308030452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c08005180 msgr2=0x7f7c0800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:41.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.821+0000 7f7c127fc640 1 --2- 172.21.15.154:0/308030452 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c08005180 0x7f7c0800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:41.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.821+0000 7f7c127fc640 1 -- 172.21.15.154:0/308030452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c080039c0 msgr2=0x7f7c08003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:41.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.821+0000 7f7c127fc640 1 --2- 172.21.15.154:0/308030452 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c080039c0 0x7f7c08003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:41.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.821+0000 7f7c127fc640 1 -- 172.21.15.154:0/308030452 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c08003520 con 0x7f7c080047c0 2023-09-23T03:28:41.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.821+0000 7f7c1b7fe640 1 -- 172.21.15.154:0/308030452 reap_dead start 2023-09-23T03:28:41.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.822+0000 7f7c127fc640 1 --2- 172.21.15.154:0/308030452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c080047c0 0x7f7c08004c40 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f7c0400a760 tx=0x7f7c080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b5ecfab6db03ba server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:41.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.823+0000 7f7c1a7fc640 1 -- 172.21.15.154:0/308030452 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c04034630 con 0x7f7c080047c0 2023-09-23T03:28:41.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.823+0000 7f7c1a7fc640 1 -- 172.21.15.154:0/308030452 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c0402f070 con 0x7f7c080047c0 2023-09-23T03:28:41.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.823+0000 7f7c1a7fc640 1 -- 172.21.15.154:0/308030452 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c04034a50 con 0x7f7c080047c0 2023-09-23T03:28:41.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.823+0000 7f7c21a32640 1 -- 172.21.15.154:0/308030452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c080047c0 msgr2=0x7f7c08004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:41.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.823+0000 7f7c21a32640 1 --2- 172.21.15.154:0/308030452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c080047c0 0x7f7c08004c40 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f7c0400a760 tx=0x7f7c080057f0 comp rx=0 tx=0).stop 2023-09-23T03:28:41.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.823+0000 7f7c21a32640 1 -- 172.21.15.154:0/308030452 shutdown_connections 2023-09-23T03:28:41.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.823+0000 7f7c21a32640 1 --2- 172.21.15.154:0/308030452 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c080047c0 0x7f7c08004c40 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:41.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.823+0000 7f7c21a32640 1 -- 172.21.15.154:0/308030452 >> 172.21.15.154:0/308030452 conn(0x7f7c1c0cb070 msgr2=0x7f7c1c0cb480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:41.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.823+0000 7f7c21a32640 1 -- 172.21.15.154:0/308030452 shutdown_connections 2023-09-23T03:28:41.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.824+0000 7f7c21a32640 1 -- 172.21.15.154:0/308030452 wait complete. 2023-09-23T03:28:41.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.825+0000 7f7c21a32640 1 Processor -- start 2023-09-23T03:28:41.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.825+0000 7f7c21a32640 1 -- start start 2023-09-23T03:28:41.828 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.826+0000 7f7c21a32640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c080047c0 0x7f7c1c0b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:41.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.826+0000 7f7c21a32640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c1c13f730 0x7f7c1c1f2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:41.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.827+0000 7f7c21a32640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c1c0b3d70 0x7f7c1c070030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:41.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.827+0000 7f7c21a32640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1c0b38e0 con 0x7f7c080047c0 2023-09-23T03:28:41.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.827+0000 7f7c21a32640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1c0b2fb0 con 0x7f7c1c13f730 2023-09-23T03:28:41.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.827+0000 7f7c21a32640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1c0b3290 con 0x7f7c1c0b3d70 2023-09-23T03:28:41.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.827+0000 7f7c1affd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c080047c0 0x7f7c1c0b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:41.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.827+0000 7f7c1affd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c080047c0 0x7f7c1c0b0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49138/0 (socket says 172.21.15.154:49138) 2023-09-23T03:28:41.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.827+0000 7f7c1affd640 1 -- 172.21.15.154:0/2341663823 learned_addr learned my addr 172.21.15.154:0/2341663823 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:41.829 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.827+0000 7f7c1affd640 1 -- 172.21.15.154:0/2341663823 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c1c0b3d70 msgr2=0x7f7c1c070030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:41.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.827+0000 7f7c1affd640 1 --2- 172.21.15.154:0/2341663823 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c1c0b3d70 0x7f7c1c070030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:41.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.827+0000 7f7c1affd640 1 -- 172.21.15.154:0/2341663823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c1c13f730 msgr2=0x7f7c1c1f2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:41.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.827+0000 7f7c127fc640 1 --2- 172.21.15.154:0/2341663823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c1c13f730 0x7f7c1c1f2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:41.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.827+0000 7f7c1affd640 1 --2- 172.21.15.154:0/2341663823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c1c13f730 0x7f7c1c1f2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:41.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.827+0000 7f7c1affd640 1 -- 172.21.15.154:0/2341663823 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c1c0b3570 con 0x7f7c080047c0 2023-09-23T03:28:41.830 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.827+0000 7f7c1affd640 1 --2- 172.21.15.154:0/2341663823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c080047c0 0x7f7c1c0b0f80 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c00e400 tx=0x7f7c0c00e8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:41.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.828+0000 7f7c18ff9640 1 -- 172.21.15.154:0/2341663823 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c0052e0 con 0x7f7c080047c0 2023-09-23T03:28:41.831 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.828+0000 7f7c18ff9640 1 -- 172.21.15.154:0/2341663823 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c0c005460 con 0x7f7c080047c0 2023-09-23T03:28:41.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.828+0000 7f7c18ff9640 1 -- 172.21.15.154:0/2341663823 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c003830 con 0x7f7c080047c0 2023-09-23T03:28:41.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.828+0000 7f7c21a32640 1 -- 172.21.15.154:0/2341663823 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c1c0b2c60 con 0x7f7c080047c0 2023-09-23T03:28:41.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.828+0000 7f7c21a32640 1 -- 172.21.15.154:0/2341663823 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c1c1f2580 con 0x7f7c080047c0 2023-09-23T03:28:41.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.829+0000 7f7c21a32640 1 -- 172.21.15.154:0/2341663823 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c1c067870 con 0x7f7c080047c0 2023-09-23T03:28:41.832 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.830+0000 7f7c18ff9640 1 -- 172.21.15.154:0/2341663823 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f7c0c003990 con 0x7f7c080047c0 2023-09-23T03:28:41.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.831+0000 7f7c18ff9640 1 --2- 172.21.15.154:0/2341663823 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7bfc06af60 0x7f7bfc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:41.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.831+0000 7f7c18ff9640 1 -- 172.21.15.154:0/2341663823 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f7c0c0890b0 con 0x7f7c080047c0 2023-09-23T03:28:41.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.832+0000 7f7c127fc640 1 --2- 172.21.15.154:0/2341663823 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7bfc06af60 0x7f7bfc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:41.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.832+0000 7f7c127fc640 1 --2- 172.21.15.154:0/2341663823 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7bfc06af60 0x7f7bfc06d420 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f7c04034000 tx=0x7f7c0402f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:41.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:41.836+0000 7f7c18ff9640 1 -- 172.21.15.154:0/2341663823 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f7c0c054040 con 0x7f7c080047c0 2023-09-23T03:28:42.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.022+0000 7f7c21a32640 1 -- 172.21.15.154:0/2341663823 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7c1c0b3700 con 0x7f7c080047c0 2023-09-23T03:28:42.024 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.023+0000 7f7c18ff9640 1 -- 172.21.15.154:0/2341663823 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v128) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7c0c0190a0 con 0x7f7c080047c0 2023-09-23T03:28:42.024 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:28:42.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.026+0000 7f7c21a32640 1 -- 172.21.15.154:0/2341663823 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7bfc06af60 msgr2=0x7f7bfc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:42.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.026+0000 7f7c21a32640 1 --2- 172.21.15.154:0/2341663823 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7bfc06af60 0x7f7bfc06d420 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f7c04034000 tx=0x7f7c0402f000 comp rx=0 tx=0).stop 2023-09-23T03:28:42.027 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.026+0000 7f7c21a32640 1 -- 172.21.15.154:0/2341663823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c080047c0 msgr2=0x7f7c1c0b0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:42.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.026+0000 7f7c21a32640 1 --2- 172.21.15.154:0/2341663823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c080047c0 0x7f7c1c0b0f80 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c00e400 tx=0x7f7c0c00e8c0 comp rx=0 tx=0).stop 2023-09-23T03:28:42.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.026+0000 7f7c21a32640 1 -- 172.21.15.154:0/2341663823 shutdown_connections 2023-09-23T03:28:42.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.026+0000 7f7c21a32640 1 --2- 172.21.15.154:0/2341663823 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7bfc06af60 0x7f7bfc06d420 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:42.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.026+0000 7f7c21a32640 1 --2- 172.21.15.154:0/2341663823 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7c1c0b3d70 0x7f7c1c070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:42.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.026+0000 7f7c21a32640 1 --2- 172.21.15.154:0/2341663823 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7c1c13f730 0x7f7c1c1f2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:42.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.026+0000 7f7c21a32640 1 --2- 172.21.15.154:0/2341663823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7c080047c0 0x7f7c1c0b0f80 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:42.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.026+0000 7f7c21a32640 1 -- 172.21.15.154:0/2341663823 >> 172.21.15.154:0/2341663823 conn(0x7f7c1c0cb070 msgr2=0x7f7c1c0affd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:42.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.026+0000 7f7c21a32640 1 -- 172.21.15.154:0/2341663823 shutdown_connections 2023-09-23T03:28:42.028 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.026+0000 7f7c21a32640 1 -- 172.21.15.154:0/2341663823 wait complete. 2023-09-23T03:28:42.046 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:28:42.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.251+0000 7efe6e8da640 1 Processor -- start 2023-09-23T03:28:42.252 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.251+0000 7efe6e8da640 1 -- start start 2023-09-23T03:28:42.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.252+0000 7efe6e8da640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe6815cb70 0x7efe6806b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:42.253 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.252+0000 7efe6e8da640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe680ce780 con 0x7efe680d25a0 2023-09-23T03:28:42.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.252+0000 7efe6e8da640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe680ce1c0 con 0x7efe680d0d60 2023-09-23T03:28:42.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.252+0000 7efe6e8da640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe680ce4a0 con 0x7efe6815cb70 2023-09-23T03:28:42.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.252+0000 7efe6d0d7640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7efe680d25a0 0x7efe680d2a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:41452/0 (socket says 172.21.15.154:41452) 2023-09-23T03:28:42.254 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.252+0000 7efe6d0d7640 1 -- 172.21.15.154:0/2219225534 learned_addr learned my addr 172.21.15.154:0/2219225534 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:28:42.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.253+0000 7efe6d8d8640 1 --2- 172.21.15.154:0/2219225534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe6815cb70 0x7efe6806b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:42.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.253+0000 7efe6c8d6640 1 -- 172.21.15.154:0/2219225534 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2988261049 0 0) 0x7efe680ce780 con 0x7efe680d25a0 2023-09-23T03:28:42.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.253+0000 7efe6c8d6640 1 -- 172.21.15.154:0/2219225534 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe54003520 con 0x7efe680d25a0 2023-09-23T03:28:42.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.253+0000 7efe6c8d6640 1 -- 172.21.15.154:0/2219225534 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1025180221 0 0) 0x7efe54003520 con 0x7efe680d25a0 2023-09-23T03:28:42.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.253+0000 7efe6c8d6640 1 -- 172.21.15.154:0/2219225534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe6815cb70 msgr2=0x7efe6806b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:42.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.253+0000 7efe6c8d6640 1 --2- 172.21.15.154:0/2219225534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe6815cb70 0x7efe6806b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:42.256 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.253+0000 7efe6c8d6640 1 -- 172.21.15.154:0/2219225534 >> v1:172.21.15.190:6789/0 conn(0x7efe680d0d60 legacy=0x7efe680d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:42.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.253+0000 7efe6c8d6640 1 -- 172.21.15.154:0/2219225534 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe680d0560 con 0x7efe680d25a0 2023-09-23T03:28:42.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.253+0000 7efe6c8d6640 1 -- 172.21.15.154:0/2219225534 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7efe58001670 con 0x7efe680d25a0 2023-09-23T03:28:42.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.253+0000 7efe6c8d6640 1 -- 172.21.15.154:0/2219225534 >> v1:172.21.15.154:6789/0 conn(0x7efe680d25a0 legacy=0x7efe680d2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:42.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.253+0000 7efe6c8d6640 1 --2- 172.21.15.154:0/2219225534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe540039a0 0x7efe54003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:42.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.253+0000 7efe6c8d6640 1 --2- 172.21.15.154:0/2219225534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe6815cb70 0x7efe540047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:42.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.253+0000 7efe6c8d6640 1 --2- 172.21.15.154:0/2219225534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe54004d00 0x7efe54005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:42.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.253+0000 7efe6c8d6640 1 -- 172.21.15.154:0/2219225534 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe680ce4a0 con 0x7efe540039a0 2023-09-23T03:28:42.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.253+0000 7efe6c8d6640 1 -- 172.21.15.154:0/2219225534 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5400ba30 con 0x7efe54004d00 2023-09-23T03:28:42.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.253+0000 7efe6c8d6640 1 -- 172.21.15.154:0/2219225534 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe5400bbf0 con 0x7efe6815cb70 2023-09-23T03:28:42.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.254+0000 7efe6d8d8640 1 --2- 172.21.15.154:0/2219225534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe540039a0 0x7efe54003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:42.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.254+0000 7efe6d8d8640 1 -- 172.21.15.154:0/2219225534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe6815cb70 msgr2=0x7efe540047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:42.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.254+0000 7efe6d8d8640 1 --2- 172.21.15.154:0/2219225534 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe6815cb70 0x7efe540047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:42.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.254+0000 7efe6d8d8640 1 -- 172.21.15.154:0/2219225534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe54004d00 msgr2=0x7efe54005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:42.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.254+0000 7efe6e0d9640 1 --2- 172.21.15.154:0/2219225534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe54004d00 0x7efe54005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:42.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.254+0000 7efe6d8d8640 1 --2- 172.21.15.154:0/2219225534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe54004d00 0x7efe54005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:42.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.254+0000 7efe6d8d8640 1 -- 172.21.15.154:0/2219225534 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe54003520 con 0x7efe540039a0 2023-09-23T03:28:42.258 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.254+0000 7efe6e0d9640 1 --2- 172.21.15.154:0/2219225534 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe54004d00 0x7efe54005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:42.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.255+0000 7efe6d8d8640 1 --2- 172.21.15.154:0/2219225534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe540039a0 0x7efe54003dc0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7efe6400e140 tx=0x7efe540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb48179e86f0c334 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:42.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.255+0000 7efe6c8d6640 1 -- 172.21.15.154:0/2219225534 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6402cc60 con 0x7efe540039a0 2023-09-23T03:28:42.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.255+0000 7efe6c8d6640 1 -- 172.21.15.154:0/2219225534 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe6400f040 con 0x7efe540039a0 2023-09-23T03:28:42.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.256+0000 7efe6c8d6640 1 -- 172.21.15.154:0/2219225534 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe64034440 con 0x7efe540039a0 2023-09-23T03:28:42.259 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.256+0000 7efe6e8da640 1 -- 172.21.15.154:0/2219225534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe540039a0 msgr2=0x7efe54003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:42.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.256+0000 7efe6e8da640 1 --2- 172.21.15.154:0/2219225534 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe540039a0 0x7efe54003dc0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7efe6400e140 tx=0x7efe540057f0 comp rx=0 tx=0).stop 2023-09-23T03:28:42.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.256+0000 7efe6e0d9640 1 -- 172.21.15.154:0/2219225534 reap_dead start 2023-09-23T03:28:42.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.256+0000 7efe6e8da640 1 -- 172.21.15.154:0/2219225534 shutdown_connections 2023-09-23T03:28:42.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.256+0000 7efe6e8da640 1 -- 172.21.15.154:0/2219225534 >> 172.21.15.154:0/2219225534 conn(0x7efe6809bb90 msgr2=0x7efe68089f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:42.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.256+0000 7efe6e8da640 1 -- 172.21.15.154:0/2219225534 shutdown_connections 2023-09-23T03:28:42.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.256+0000 7efe6e8da640 1 -- 172.21.15.154:0/2219225534 wait complete. 2023-09-23T03:28:42.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.258+0000 7efe6e8da640 1 Processor -- start 2023-09-23T03:28:42.260 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.258+0000 7efe6e8da640 1 -- start start 2023-09-23T03:28:42.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.259+0000 7efe6e8da640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe540039a0 0x7efe68151df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:42.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.259+0000 7efe6e8da640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe681f6240 0x7efe680b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:42.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.259+0000 7efe6e8da640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe681f66c0 0x7efe680b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:42.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.259+0000 7efe6e8da640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe680cc400 con 0x7efe681f66c0 2023-09-23T03:28:42.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.259+0000 7efe6e8da640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe680cb360 con 0x7efe681f6240 2023-09-23T03:28:42.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.259+0000 7efe6e8da640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe680cb640 con 0x7efe540039a0 2023-09-23T03:28:42.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.259+0000 7efe6e0d9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe681f66c0 0x7efe680b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:42.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.259+0000 7efe6e0d9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe681f66c0 0x7efe680b4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49154/0 (socket says 172.21.15.154:49154) 2023-09-23T03:28:42.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.260+0000 7efe6e0d9640 1 -- 172.21.15.154:0/756509980 learned_addr learned my addr 172.21.15.154:0/756509980 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:42.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.260+0000 7efe6d8d8640 1 --2- 172.21.15.154:0/756509980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe540039a0 0x7efe68151df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:42.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.260+0000 7efe6d0d7640 1 --2- 172.21.15.154:0/756509980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe681f6240 0x7efe680b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:42.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.260+0000 7efe6e0d9640 1 -- 172.21.15.154:0/756509980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe540039a0 msgr2=0x7efe68151df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:42.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.260+0000 7efe6e0d9640 1 --2- 172.21.15.154:0/756509980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe540039a0 0x7efe68151df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:42.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.260+0000 7efe6e0d9640 1 -- 172.21.15.154:0/756509980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe681f6240 msgr2=0x7efe680b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:42.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.260+0000 7efe6e0d9640 1 --2- 172.21.15.154:0/756509980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe681f6240 0x7efe680b48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:42.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.260+0000 7efe6e0d9640 1 -- 172.21.15.154:0/756509980 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe680cb920 con 0x7efe681f66c0 2023-09-23T03:28:42.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.260+0000 7efe6d0d7640 1 --2- 172.21.15.154:0/756509980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe681f6240 0x7efe680b48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:42.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.260+0000 7efe6e0d9640 1 --2- 172.21.15.154:0/756509980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe681f66c0 0x7efe680b4e10 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7efe680b5450 tx=0x7efe6000d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:42.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.260+0000 7efe5e7fc640 1 -- 172.21.15.154:0/756509980 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6000b070 con 0x7efe681f66c0 2023-09-23T03:28:42.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.261+0000 7efe5e7fc640 1 -- 172.21.15.154:0/756509980 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe600054a0 con 0x7efe681f66c0 2023-09-23T03:28:42.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.261+0000 7efe5e7fc640 1 -- 172.21.15.154:0/756509980 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe60007270 con 0x7efe681f66c0 2023-09-23T03:28:42.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.261+0000 7efe6e8da640 1 -- 172.21.15.154:0/756509980 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe680cbc00 con 0x7efe681f66c0 2023-09-23T03:28:42.264 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.261+0000 7efe6e8da640 1 -- 172.21.15.154:0/756509980 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe68152330 con 0x7efe681f66c0 2023-09-23T03:28:42.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.262+0000 7efe6e8da640 1 -- 172.21.15.154:0/756509980 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe680cc120 con 0x7efe681f66c0 2023-09-23T03:28:42.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.263+0000 7efe5e7fc640 1 -- 172.21.15.154:0/756509980 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7efe600073d0 con 0x7efe681f66c0 2023-09-23T03:28:42.265 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.264+0000 7efe5e7fc640 1 --2- 172.21.15.154:0/756509980 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efe4c06af60 0x7efe4c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:42.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.264+0000 7efe5e7fc640 1 -- 172.21.15.154:0/756509980 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6201+0+0 (secure 0 0 0) 0x7efe60089460 con 0x7efe681f66c0 2023-09-23T03:28:42.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.264+0000 7efe6d8d8640 1 --2- 172.21.15.154:0/756509980 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efe4c06af60 0x7efe4c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:42.266 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.265+0000 7efe6d8d8640 1 --2- 172.21.15.154:0/756509980 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efe4c06af60 0x7efe4c06d420 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7efe64036000 tx=0x7efe6402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:42.269 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.268+0000 7efe5e7fc640 1 -- 172.21.15.154:0/756509980 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7efe680cc120 con 0x7efe681f66c0 2023-09-23T03:28:42.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.457+0000 7efe6e8da640 1 -- 172.21.15.154:0/756509980 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efe680cc120 con 0x7efe681f66c0 2023-09-23T03:28:42.459 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.458+0000 7efe5e7fc640 1 -- 172.21.15.154:0/756509980 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v128) v1 ==== 104+0+11 (secure 0 0 0) 0x7efe680cc120 con 0x7efe681f66c0 2023-09-23T03:28:42.460 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:28:42.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.462+0000 7efe6e8da640 1 -- 172.21.15.154:0/756509980 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efe4c06af60 msgr2=0x7efe4c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:42.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.462+0000 7efe6e8da640 1 --2- 172.21.15.154:0/756509980 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efe4c06af60 0x7efe4c06d420 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7efe64036000 tx=0x7efe6402e000 comp rx=0 tx=0).stop 2023-09-23T03:28:42.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.462+0000 7efe6e8da640 1 -- 172.21.15.154:0/756509980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe681f66c0 msgr2=0x7efe680b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:42.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.462+0000 7efe6e8da640 1 --2- 172.21.15.154:0/756509980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe681f66c0 0x7efe680b4e10 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7efe680b5450 tx=0x7efe6000d890 comp rx=0 tx=0).stop 2023-09-23T03:28:42.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.462+0000 7efe6e8da640 1 -- 172.21.15.154:0/756509980 shutdown_connections 2023-09-23T03:28:42.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.462+0000 7efe6e8da640 1 --2- 172.21.15.154:0/756509980 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efe4c06af60 0x7efe4c06d420 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:42.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.462+0000 7efe6e8da640 1 --2- 172.21.15.154:0/756509980 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe681f66c0 0x7efe680b4e10 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:42.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.462+0000 7efe6e8da640 1 --2- 172.21.15.154:0/756509980 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efe681f6240 0x7efe680b48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:42.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.462+0000 7efe6e8da640 1 --2- 172.21.15.154:0/756509980 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efe540039a0 0x7efe68151df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:42.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.462+0000 7efe6e8da640 1 -- 172.21.15.154:0/756509980 >> 172.21.15.154:0/756509980 conn(0x7efe6809bb90 msgr2=0x7efe680cd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:42.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.462+0000 7efe6e8da640 1 -- 172.21.15.154:0/756509980 shutdown_connections 2023-09-23T03:28:42.464 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.462+0000 7efe6e8da640 1 -- 172.21.15.154:0/756509980 wait complete. 2023-09-23T03:28:42.480 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:28:42.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.696+0000 7f92320a9640 1 Processor -- start 2023-09-23T03:28:42.698 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.697+0000 7f92320a9640 1 -- start start 2023-09-23T03:28:42.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.698+0000 7f92320a9640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f922c0d23b0 0x7f922c0d05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:42.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.698+0000 7f92320a9640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f922c0d0b20 0x7f922c0b5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:42.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.698+0000 7f92320a9640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f922c15d070 con 0x7f922c0cf370 2023-09-23T03:28:42.700 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.698+0000 7f92320a9640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f922c0d2820 con 0x7f922c0d23b0 2023-09-23T03:28:42.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.698+0000 7f92320a9640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f922c0d1f20 con 0x7f922c0d0b20 2023-09-23T03:28:42.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.699+0000 7f922affd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f922c0d23b0 0x7f922c0d05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:42.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.699+0000 7f922affd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f922c0d23b0 0x7f922c0d05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49156/0 (socket says 172.21.15.154:49156) 2023-09-23T03:28:42.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.699+0000 7f922affd640 1 -- 172.21.15.154:0/4190343064 learned_addr learned my addr 172.21.15.154:0/4190343064 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:42.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.699+0000 7f922bfff640 1 --2- 172.21.15.154:0/4190343064 >> v2:172.21.15.190:3300/0 conn(0x7f922c0d0b20 0x7f922c0b5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:42.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.699+0000 7f922affd640 1 -- 172.21.15.154:0/4190343064 >> v2:172.21.15.190:3300/0 conn(0x7f922c0d0b20 msgr2=0x7f922c0b5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:42.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.699+0000 7f922affd640 1 --2- 172.21.15.154:0/4190343064 >> v2:172.21.15.190:3300/0 conn(0x7f922c0d0b20 0x7f922c0b5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:42.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.699+0000 7f922affd640 1 -- 172.21.15.154:0/4190343064 >> v1:172.21.15.190:6789/0 conn(0x7f922c0cf370 legacy=0x7f922c0cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:42.701 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.699+0000 7f922affd640 1 -- 172.21.15.154:0/4190343064 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f922c0d1310 con 0x7f922c0d23b0 2023-09-23T03:28:42.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.700+0000 7f922affd640 1 --2- 172.21.15.154:0/4190343064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f922c0d23b0 0x7f922c0d05e0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f922c06bc30 tx=0x7f921402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f37383ec7c0ee12 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:42.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.701+0000 7f922a7fc640 1 -- 172.21.15.154:0/4190343064 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9214030860 con 0x7f922c0d23b0 2023-09-23T03:28:42.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.701+0000 7f922a7fc640 1 -- 172.21.15.154:0/4190343064 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92140309c0 con 0x7f922c0d23b0 2023-09-23T03:28:42.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.701+0000 7f922a7fc640 1 -- 172.21.15.154:0/4190343064 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9214030d50 con 0x7f922c0d23b0 2023-09-23T03:28:42.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.701+0000 7f92320a9640 1 -- 172.21.15.154:0/4190343064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f922c0d23b0 msgr2=0x7f922c0d05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:42.702 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.701+0000 7f92320a9640 1 --2- 172.21.15.154:0/4190343064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f922c0d23b0 0x7f922c0d05e0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f922c06bc30 tx=0x7f921402b9a0 comp rx=0 tx=0).stop 2023-09-23T03:28:42.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.701+0000 7f92320a9640 1 -- 172.21.15.154:0/4190343064 shutdown_connections 2023-09-23T03:28:42.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.701+0000 7f92320a9640 1 --2- 172.21.15.154:0/4190343064 >> v2:172.21.15.190:3300/0 conn(0x7f922c0d0b20 0x7f922c0b5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:42.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.701+0000 7f92320a9640 1 --2- 172.21.15.154:0/4190343064 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f922c0d23b0 0x7f922c0d05e0 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:42.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.701+0000 7f92320a9640 1 -- 172.21.15.154:0/4190343064 >> 172.21.15.154:0/4190343064 conn(0x7f922c06aeb0 msgr2=0x7f922c06b290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:42.703 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.701+0000 7f92320a9640 1 -- 172.21.15.154:0/4190343064 shutdown_connections 2023-09-23T03:28:42.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.701+0000 7f92320a9640 1 -- 172.21.15.154:0/4190343064 wait complete. 2023-09-23T03:28:42.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.703+0000 7f92320a9640 1 Processor -- start 2023-09-23T03:28:42.704 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.703+0000 7f92320a9640 1 -- start start 2023-09-23T03:28:42.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.704+0000 7f92320a9640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f922c0cf370 0x7f922c152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:42.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.704+0000 7f92320a9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f922c0d0b20 0x7f922c0c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:42.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.704+0000 7f92320a9640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f922c0d23b0 0x7f922c0c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:42.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.704+0000 7f92320a9640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922c0c81c0 con 0x7f922c0d0b20 2023-09-23T03:28:42.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.704+0000 7f92320a9640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922c1f2770 con 0x7f922c0d23b0 2023-09-23T03:28:42.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.704+0000 7f92320a9640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f922c0c7040 con 0x7f922c0cf370 2023-09-23T03:28:42.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.705+0000 7f922affd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f922c0d0b20 0x7f922c0c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:42.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.705+0000 7f922affd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f922c0d0b20 0x7f922c0c64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49166/0 (socket says 172.21.15.154:49166) 2023-09-23T03:28:42.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.705+0000 7f922affd640 1 -- 172.21.15.154:0/1390201268 learned_addr learned my addr 172.21.15.154:0/1390201268 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:42.707 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.705+0000 7f922bfff640 1 --2- 172.21.15.154:0/1390201268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f922c0d23b0 0x7f922c0c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:42.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.705+0000 7f922b7fe640 1 --2- 172.21.15.154:0/1390201268 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f922c0cf370 0x7f922c152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:42.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.705+0000 7f922affd640 1 -- 172.21.15.154:0/1390201268 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f922c0cf370 msgr2=0x7f922c152fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:42.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.705+0000 7f922affd640 1 --2- 172.21.15.154:0/1390201268 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f922c0cf370 0x7f922c152fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:42.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.705+0000 7f922affd640 1 -- 172.21.15.154:0/1390201268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f922c0d23b0 msgr2=0x7f922c0c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:42.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.705+0000 7f922affd640 1 --2- 172.21.15.154:0/1390201268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f922c0d23b0 0x7f922c0c6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:42.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.705+0000 7f922affd640 1 -- 172.21.15.154:0/1390201268 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92140094d0 con 0x7f922c0d0b20 2023-09-23T03:28:42.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.705+0000 7f922bfff640 1 --2- 172.21.15.154:0/1390201268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f922c0d23b0 0x7f922c0c6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:42.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.705+0000 7f922b7fe640 1 --2- 172.21.15.154:0/1390201268 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f922c0cf370 0x7f922c152fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:42.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.706+0000 7f922affd640 1 --2- 172.21.15.154:0/1390201268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f922c0d0b20 0x7f922c0c64d0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f921402b9e0 tx=0x7f92140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:42.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.706+0000 7f9223fff640 1 -- 172.21.15.154:0/1390201268 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921403e070 con 0x7f922c0d0b20 2023-09-23T03:28:42.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.706+0000 7f9223fff640 1 -- 172.21.15.154:0/1390201268 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9214039a60 con 0x7f922c0d0b20 2023-09-23T03:28:42.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.706+0000 7f9223fff640 1 -- 172.21.15.154:0/1390201268 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9214039df0 con 0x7f922c0d0b20 2023-09-23T03:28:42.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.706+0000 7f92320a9640 1 -- 172.21.15.154:0/1390201268 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f922c0c7320 con 0x7f922c0d0b20 2023-09-23T03:28:42.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.706+0000 7f92320a9640 1 -- 172.21.15.154:0/1390201268 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f922c0c7de0 con 0x7f922c0d0b20 2023-09-23T03:28:42.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.707+0000 7f92320a9640 1 -- 172.21.15.154:0/1390201268 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f922c0cee60 con 0x7f922c0d0b20 2023-09-23T03:28:42.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.708+0000 7f9223fff640 1 -- 172.21.15.154:0/1390201268 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f921404b400 con 0x7f922c0d0b20 2023-09-23T03:28:42.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.709+0000 7f9223fff640 1 --2- 172.21.15.154:0/1390201268 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f920c06af30 0x7f920c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:42.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.709+0000 7f9223fff640 1 -- 172.21.15.154:0/1390201268 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f92140abea0 con 0x7f922c0d0b20 2023-09-23T03:28:42.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.709+0000 7f922b7fe640 1 --2- 172.21.15.154:0/1390201268 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f920c06af30 0x7f920c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:42.711 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.710+0000 7f922b7fe640 1 --2- 172.21.15.154:0/1390201268 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f920c06af30 0x7f920c06d3f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f921c004520 tx=0x7f921c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:42.715 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.714+0000 7f9223fff640 1 -- 172.21.15.154:0/1390201268 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f9214076e30 con 0x7f922c0d0b20 2023-09-23T03:28:42.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.904+0000 7f92320a9640 1 -- 172.21.15.154:0/1390201268 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f922c0c8300 con 0x7f922c0d0b20 2023-09-23T03:28:42.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.905+0000 7f9223fff640 1 -- 172.21.15.154:0/1390201268 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v128) v1 ==== 100+0+11 (secure 0 0 0) 0x7f922c0c8300 con 0x7f922c0d0b20 2023-09-23T03:28:42.906 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:28:42.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.908+0000 7f92320a9640 1 -- 172.21.15.154:0/1390201268 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f920c06af30 msgr2=0x7f920c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:42.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.908+0000 7f92320a9640 1 --2- 172.21.15.154:0/1390201268 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f920c06af30 0x7f920c06d3f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f921c004520 tx=0x7f921c009040 comp rx=0 tx=0).stop 2023-09-23T03:28:42.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.908+0000 7f92320a9640 1 -- 172.21.15.154:0/1390201268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f922c0d0b20 msgr2=0x7f922c0c64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:42.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.908+0000 7f92320a9640 1 --2- 172.21.15.154:0/1390201268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f922c0d0b20 0x7f922c0c64d0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f921402b9e0 tx=0x7f92140046c0 comp rx=0 tx=0).stop 2023-09-23T03:28:42.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.908+0000 7f92320a9640 1 -- 172.21.15.154:0/1390201268 shutdown_connections 2023-09-23T03:28:42.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.908+0000 7f92320a9640 1 --2- 172.21.15.154:0/1390201268 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f920c06af30 0x7f920c06d3f0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:42.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.908+0000 7f92320a9640 1 --2- 172.21.15.154:0/1390201268 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f922c0d23b0 0x7f922c0c6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:42.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.908+0000 7f92320a9640 1 --2- 172.21.15.154:0/1390201268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f922c0d0b20 0x7f922c0c64d0 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:42.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.908+0000 7f92320a9640 1 --2- 172.21.15.154:0/1390201268 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f922c0cf370 0x7f922c152fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:42.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.908+0000 7f92320a9640 1 -- 172.21.15.154:0/1390201268 >> 172.21.15.154:0/1390201268 conn(0x7f922c06aeb0 msgr2=0x7f922c15b200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:42.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.908+0000 7f92320a9640 1 -- 172.21.15.154:0/1390201268 shutdown_connections 2023-09-23T03:28:42.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:42.908+0000 7f92320a9640 1 -- 172.21.15.154:0/1390201268 wait complete. 2023-09-23T03:28:42.928 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-23T03:28:43.139 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.138+0000 7ff3a237c640 1 Processor -- start 2023-09-23T03:28:43.140 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.138+0000 7ff3a237c640 1 -- start start 2023-09-23T03:28:43.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.140+0000 7ff3a237c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff39c15d770 0x7ff39c15ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:43.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.140+0000 7ff3a237c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff39c15d3b0 con 0x7ff39c15cf50 2023-09-23T03:28:43.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.140+0000 7ff3a237c640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff39c0711e0 con 0x7ff39c1534f0 2023-09-23T03:28:43.141 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.140+0000 7ff3a237c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff39c1538d0 con 0x7ff39c15d770 2023-09-23T03:28:43.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.140+0000 7ff3a0919640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7ff39c15cf50 0x7ff39c0ca6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:41456/0 (socket says 172.21.15.154:41456) 2023-09-23T03:28:43.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.140+0000 7ff3a0919640 1 -- 172.21.15.154:0/3109924521 learned_addr learned my addr 172.21.15.154:0/3109924521 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:28:43.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.140+0000 7ff39b7fe640 1 --2- 172.21.15.154:0/3109924521 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff39c15d770 0x7ff39c15ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:43.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.141+0000 7ff39affd640 1 -- 172.21.15.154:0/3109924521 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3398261038 0 0) 0x7ff39c15d3b0 con 0x7ff39c15cf50 2023-09-23T03:28:43.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.141+0000 7ff39affd640 1 -- 172.21.15.154:0/3109924521 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff384003440 con 0x7ff39c15cf50 2023-09-23T03:28:43.142 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.141+0000 7ff39b7fe640 1 -- 172.21.15.154:0/3109924521 >> v1:172.21.15.190:6789/0 conn(0x7ff39c1534f0 legacy=0x7ff39c0b0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:43.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.141+0000 7ff39b7fe640 1 -- 172.21.15.154:0/3109924521 >> v1:172.21.15.154:6789/0 conn(0x7ff39c15cf50 legacy=0x7ff39c0ca6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:43.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.141+0000 7ff39b7fe640 1 -- 172.21.15.154:0/3109924521 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff39c070c20 con 0x7ff39c15d770 2023-09-23T03:28:43.143 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.142+0000 7ff39b7fe640 1 --2- 172.21.15.154:0/3109924521 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff39c15d770 0x7ff39c15ca10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff3900097d0 tx=0x7ff39002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cdf6837eae989ed server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:43.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.143+0000 7ff39affd640 1 -- 172.21.15.154:0/3109924521 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39002d040 con 0x7ff39c15d770 2023-09-23T03:28:43.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.143+0000 7ff39affd640 1 -- 172.21.15.154:0/3109924521 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff390031810 con 0x7ff39c15d770 2023-09-23T03:28:43.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.143+0000 7ff39affd640 1 -- 172.21.15.154:0/3109924521 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff390031ba0 con 0x7ff39c15d770 2023-09-23T03:28:43.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.144+0000 7ff3a237c640 1 -- 172.21.15.154:0/3109924521 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff39c15d770 msgr2=0x7ff39c15ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:43.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.144+0000 7ff3a237c640 1 --2- 172.21.15.154:0/3109924521 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff39c15d770 0x7ff39c15ca10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff3900097d0 tx=0x7ff39002b8b0 comp rx=0 tx=0).stop 2023-09-23T03:28:43.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.144+0000 7ff3a237c640 1 -- 172.21.15.154:0/3109924521 shutdown_connections 2023-09-23T03:28:43.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.144+0000 7ff3a237c640 1 --2- 172.21.15.154:0/3109924521 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff39c15d770 0x7ff39c15ca10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:43.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.144+0000 7ff3a237c640 1 -- 172.21.15.154:0/3109924521 >> 172.21.15.154:0/3109924521 conn(0x7ff39c06c820 msgr2=0x7ff39c069cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:43.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.144+0000 7ff3a237c640 1 -- 172.21.15.154:0/3109924521 shutdown_connections 2023-09-23T03:28:43.146 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.144+0000 7ff3a237c640 1 -- 172.21.15.154:0/3109924521 wait complete. 2023-09-23T03:28:43.147 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.146+0000 7ff3a237c640 1 Processor -- start 2023-09-23T03:28:43.147 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.146+0000 7ff3a237c640 1 -- start start 2023-09-23T03:28:43.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.147+0000 7ff3a237c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff39c1534f0 0x7ff39c1e67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:43.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.147+0000 7ff3a237c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff39c15cf50 0x7ff39c1e4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:43.148 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.147+0000 7ff3a237c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff39c0c5a10 0x7ff39c0c5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:43.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.147+0000 7ff3a237c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39c0c6400 con 0x7ff39c15cf50 2023-09-23T03:28:43.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.147+0000 7ff3a237c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39c144f10 con 0x7ff39c1534f0 2023-09-23T03:28:43.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.147+0000 7ff3a237c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff39c1e6cf0 con 0x7ff39c0c5a10 2023-09-23T03:28:43.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.147+0000 7ff39b7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff39c15cf50 0x7ff39c1e4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:43.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.147+0000 7ff39b7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff39c15cf50 0x7ff39c1e4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:49182/0 (socket says 172.21.15.154:49182) 2023-09-23T03:28:43.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.147+0000 7ff39b7fe640 1 -- 172.21.15.154:0/2536669391 learned_addr learned my addr 172.21.15.154:0/2536669391 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:43.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.148+0000 7ff39b7fe640 1 -- 172.21.15.154:0/2536669391 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff39c0c5a10 msgr2=0x7ff39c0c5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:43.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.148+0000 7ff39b7fe640 1 --2- 172.21.15.154:0/2536669391 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff39c0c5a10 0x7ff39c0c5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:43.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.148+0000 7ff39b7fe640 1 -- 172.21.15.154:0/2536669391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff39c1534f0 msgr2=0x7ff39c1e67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:43.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.148+0000 7ff39b7fe640 1 --2- 172.21.15.154:0/2536669391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff39c1534f0 0x7ff39c1e67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:43.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.148+0000 7ff39b7fe640 1 -- 172.21.15.154:0/2536669391 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff390009400 con 0x7ff39c15cf50 2023-09-23T03:28:43.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.148+0000 7ff39b7fe640 1 --2- 172.21.15.154:0/2536669391 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff39c15cf50 0x7ff39c1e4900 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7ff3900097d0 tx=0x7ff3900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:43.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.148+0000 7ff398ff9640 1 -- 172.21.15.154:0/2536669391 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff390006e10 con 0x7ff39c15cf50 2023-09-23T03:28:43.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.148+0000 7ff398ff9640 1 -- 172.21.15.154:0/2536669391 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff39003ad60 con 0x7ff39c15cf50 2023-09-23T03:28:43.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.148+0000 7ff398ff9640 1 -- 172.21.15.154:0/2536669391 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff390044610 con 0x7ff39c15cf50 2023-09-23T03:28:43.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.148+0000 7ff3a237c640 1 -- 172.21.15.154:0/2536669391 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff39c1e4e40 con 0x7ff39c15cf50 2023-09-23T03:28:43.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.148+0000 7ff3a237c640 1 -- 172.21.15.154:0/2536669391 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff39c145440 con 0x7ff39c15cf50 2023-09-23T03:28:43.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.150+0000 7ff3a237c640 1 -- 172.21.15.154:0/2536669391 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff39c073eb0 con 0x7ff39c15cf50 2023-09-23T03:28:43.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.150+0000 7ff398ff9640 1 -- 172.21.15.154:0/2536669391 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7ff39004d450 con 0x7ff39c15cf50 2023-09-23T03:28:43.152 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.151+0000 7ff398ff9640 1 --2- 172.21.15.154:0/2536669391 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff38806af60 0x7ff38806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:43.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.152+0000 7ff398ff9640 1 -- 172.21.15.154:0/2536669391 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6201+0+0 (secure 0 0 0) 0x7ff3900abae0 con 0x7ff39c15cf50 2023-09-23T03:28:43.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.152+0000 7ff39bfff640 1 --2- 172.21.15.154:0/2536669391 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff38806af60 0x7ff38806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:43.153 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.152+0000 7ff39bfff640 1 --2- 172.21.15.154:0/2536669391 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff38806af60 0x7ff38806d420 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7ff38c000f60 tx=0x7ff38c0079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:43.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.156+0000 7ff398ff9640 1 -- 172.21.15.154:0/2536669391 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7ff390076930 con 0x7ff39c15cf50 2023-09-23T03:28:43.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.356+0000 7ff3a237c640 1 -- 172.21.15.154:0/2536669391 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff39c15a1e0 con 0x7ff39c15cf50 2023-09-23T03:28:43.358 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.357+0000 7ff398ff9640 1 -- 172.21.15.154:0/2536669391 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v128) v1 ==== 80+0+248 (secure 0 0 0) 0x7ff390035110 con 0x7ff39c15cf50 2023-09-23T03:28:43.358 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.154:6837/4006549651 2023-09-24T03:28:29.295137+0000 2023-09-23T03:28:43.358 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.154:6836/4006549651 2023-09-24T03:28:29.295137+0000 2023-09-23T03:28:43.359 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.190:6835/3558096060 2023-09-24T03:28:08.468628+0000 2023-09-23T03:28:43.359 INFO:teuthology.orchestra.run.smithi154.stdout:172.21.15.190:6833/3558096060 2023-09-24T03:28:08.468628+0000 2023-09-23T03:28:43.359 INFO:teuthology.orchestra.run.smithi154.stderr:listed 4 entries 2023-09-23T03:28:43.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.359+0000 7ff3a237c640 1 -- 172.21.15.154:0/2536669391 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff38806af60 msgr2=0x7ff38806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:43.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.359+0000 7ff3a237c640 1 --2- 172.21.15.154:0/2536669391 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff38806af60 0x7ff38806d420 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7ff38c000f60 tx=0x7ff38c0079b0 comp rx=0 tx=0).stop 2023-09-23T03:28:43.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.359+0000 7ff3a237c640 1 -- 172.21.15.154:0/2536669391 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff39c15cf50 msgr2=0x7ff39c1e4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:43.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.359+0000 7ff3a237c640 1 --2- 172.21.15.154:0/2536669391 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff39c15cf50 0x7ff39c1e4900 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7ff3900097d0 tx=0x7ff3900046c0 comp rx=0 tx=0).stop 2023-09-23T03:28:43.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.360+0000 7ff3a237c640 1 -- 172.21.15.154:0/2536669391 shutdown_connections 2023-09-23T03:28:43.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.360+0000 7ff3a237c640 1 --2- 172.21.15.154:0/2536669391 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff38806af60 0x7ff38806d420 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:43.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.360+0000 7ff3a237c640 1 --2- 172.21.15.154:0/2536669391 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff39c0c5a10 0x7ff39c0c5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:43.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.360+0000 7ff3a237c640 1 --2- 172.21.15.154:0/2536669391 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff39c15cf50 0x7ff39c1e4900 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:43.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.360+0000 7ff3a237c640 1 --2- 172.21.15.154:0/2536669391 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff39c1534f0 0x7ff39c1e67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:43.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.360+0000 7ff3a237c640 1 -- 172.21.15.154:0/2536669391 >> 172.21.15.154:0/2536669391 conn(0x7ff39c06c820 msgr2=0x7ff39c148330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:43.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.360+0000 7ff3a237c640 1 -- 172.21.15.154:0/2536669391 shutdown_connections 2023-09-23T03:28:43.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:43.360+0000 7ff3a237c640 1 -- 172.21.15.154:0/2536669391 wait complete. 2023-09-23T03:28:43.377 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi154.front.sepia.ceph.com... 2023-09-23T03:28:43.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:43.378 DEBUG:teuthology.orchestra.run.smithi154:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-09-23T03:28:43.459 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi154.stderr:ceph-fuse[130024]: fuse finished with error 0 and tester_r 0 2023-09-23T03:28:43.463 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2023-09-23T03:28:43.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:43.463 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-09-23T03:28:43.516 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:28:43.516 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:28:43.576 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053603,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248804k,nr_inodes=812201,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-09-23T03:28:43.576 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi190.front.sepia.ceph.com 2023-09-23T03:28:43.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:43.577 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-09-23T03:28:43.631 INFO:teuthology.orchestra.run.smithi190.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-09-23T03:28:43.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T03:28:43.633 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:28:43.633 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:28:43.691 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053600,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-09-23T03:28:43.691 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi154.front.sepia.ceph.com 2023-09-23T03:28:43.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:43.692 DEBUG:teuthology.orchestra.run.smithi154:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-09-23T03:28:43.745 INFO:teuthology.orchestra.run.smithi154.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-09-23T03:28:43.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T03:28:43.747 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:28:43.747 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:28:43.807 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053603,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248804k,nr_inodes=812201,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-09-23T03:28:43.807 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi190.front.sepia.ceph.com 2023-09-23T03:28:43.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:43.808 DEBUG:teuthology.orchestra.run.smithi190:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-09-23T03:28:43.862 INFO:teuthology.orchestra.run.smithi190.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-09-23T03:28:43.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T03:28:43.863 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:28:44.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.085+0000 7f228d87c640 1 Processor -- start 2023-09-23T03:28:44.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.085+0000 7f228d87c640 1 -- start start 2023-09-23T03:28:44.088 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.087+0000 7f228d87c640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2288159dd0 0x7f228806d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:44.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.087+0000 7f228d87c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f22880b4010 0x7f22880cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:44.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.087+0000 7f228d87c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22880b0c10 con 0x7f22880cddf0 2023-09-23T03:28:44.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.087+0000 7f228d87c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22880ce260 con 0x7f22880b4010 2023-09-23T03:28:44.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.087+0000 7f228d87c640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22880ad760 con 0x7f2288159dd0 2023-09-23T03:28:44.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.087+0000 7f22877fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f22880cddf0 0x7f228806c150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:34026/0 (socket says 172.21.15.154:34026) 2023-09-23T03:28:44.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.087+0000 7f22877fe640 1 -- 172.21.15.154:0/614985894 learned_addr learned my addr 172.21.15.154:0/614985894 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:28:44.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.087+0000 7f227ffff640 1 --2- 172.21.15.154:0/614985894 >> v2:172.21.15.154:3300/0 conn(0x7f22880b4010 0x7f22880cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:44.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.087+0000 7f2286ffd640 1 --2- 172.21.15.154:0/614985894 >> v2:172.21.15.190:3300/0 conn(0x7f2288159dd0 0x7f228806d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:44.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.087+0000 7f22867fc640 1 -- 172.21.15.154:0/614985894 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3697208379 0 0) 0x7f22880b0c10 con 0x7f22880cddf0 2023-09-23T03:28:44.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.087+0000 7f227ffff640 1 -- 172.21.15.154:0/614985894 >> v2:172.21.15.190:3300/0 conn(0x7f2288159dd0 msgr2=0x7f228806d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:44.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.087+0000 7f227ffff640 1 --2- 172.21.15.154:0/614985894 >> v2:172.21.15.190:3300/0 conn(0x7f2288159dd0 0x7f228806d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:44.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.087+0000 7f227ffff640 1 -- 172.21.15.154:0/614985894 >> v1:172.21.15.154:6789/0 conn(0x7f22880cddf0 legacy=0x7f228806c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:44.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.088+0000 7f227ffff640 1 -- 172.21.15.154:0/614985894 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22880ada40 con 0x7f22880b4010 2023-09-23T03:28:44.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.089+0000 7f227ffff640 1 --2- 172.21.15.154:0/614985894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22880b4010 0x7f22880cd8b0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f2288071a90 tx=0x7f227002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f756e23cf4ffcfb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:44.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.089+0000 7f22867fc640 1 -- 172.21.15.154:0/614985894 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2270031830 con 0x7f22880b4010 2023-09-23T03:28:44.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.089+0000 7f22867fc640 1 -- 172.21.15.154:0/614985894 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2270031990 con 0x7f22880b4010 2023-09-23T03:28:44.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.089+0000 7f22867fc640 1 -- 172.21.15.154:0/614985894 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2270031d20 con 0x7f22880b4010 2023-09-23T03:28:44.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.089+0000 7f228d87c640 1 -- 172.21.15.154:0/614985894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22880b4010 msgr2=0x7f22880cd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:44.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.089+0000 7f228d87c640 1 --2- 172.21.15.154:0/614985894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22880b4010 0x7f22880cd8b0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f2288071a90 tx=0x7f227002b8d0 comp rx=0 tx=0).stop 2023-09-23T03:28:44.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.090+0000 7f228d87c640 1 -- 172.21.15.154:0/614985894 shutdown_connections 2023-09-23T03:28:44.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.090+0000 7f228d87c640 1 --2- 172.21.15.154:0/614985894 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22880b4010 0x7f22880cd8b0 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:44.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.090+0000 7f228d87c640 1 --2- 172.21.15.154:0/614985894 >> v2:172.21.15.190:3300/0 conn(0x7f2288159dd0 0x7f228806d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:44.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.090+0000 7f228d87c640 1 -- 172.21.15.154:0/614985894 >> 172.21.15.154:0/614985894 conn(0x7f22880cb070 msgr2=0x7f22880cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:44.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.090+0000 7f228d87c640 1 -- 172.21.15.154:0/614985894 shutdown_connections 2023-09-23T03:28:44.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.090+0000 7f228d87c640 1 -- 172.21.15.154:0/614985894 wait complete. 2023-09-23T03:28:44.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.091+0000 7f228d87c640 1 Processor -- start 2023-09-23T03:28:44.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.091+0000 7f228d87c640 1 -- start start 2023-09-23T03:28:44.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.092+0000 7f228d87c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22880b4010 0x7f22881efc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:44.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.092+0000 7f228d87c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22880cddf0 0x7f228815a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:44.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.092+0000 7f228d87c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2288159dd0 0x7f22880ac870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:44.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.092+0000 7f228d87c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22880ab860 con 0x7f22880b4010 2023-09-23T03:28:44.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.092+0000 7f228d87c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22880abab0 con 0x7f22880cddf0 2023-09-23T03:28:44.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.092+0000 7f228d87c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22880abd90 con 0x7f2288159dd0 2023-09-23T03:28:44.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.093+0000 7f2286ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22880b4010 0x7f22881efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:44.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.093+0000 7f2286ffd640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22880b4010 0x7f22881efc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56942/0 (socket says 172.21.15.154:56942) 2023-09-23T03:28:44.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.093+0000 7f2286ffd640 1 -- 172.21.15.154:0/2201637461 learned_addr learned my addr 172.21.15.154:0/2201637461 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:44.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.093+0000 7f2286ffd640 1 -- 172.21.15.154:0/2201637461 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2288159dd0 msgr2=0x7f22880ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:44.095 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.093+0000 7f2286ffd640 1 --2- 172.21.15.154:0/2201637461 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2288159dd0 0x7f22880ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:44.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.093+0000 7f2286ffd640 1 -- 172.21.15.154:0/2201637461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22880cddf0 msgr2=0x7f228815a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:44.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.093+0000 7f2286ffd640 1 --2- 172.21.15.154:0/2201637461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22880cddf0 0x7f228815a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:44.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.093+0000 7f2286ffd640 1 -- 172.21.15.154:0/2201637461 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2270009400 con 0x7f22880b4010 2023-09-23T03:28:44.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.093+0000 7f2286ffd640 1 --2- 172.21.15.154:0/2201637461 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22880b4010 0x7f22881efc60 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f227800d5f0 tx=0x7f227800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:44.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.094+0000 7f2284ff9640 1 -- 172.21.15.154:0/2201637461 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22780125f0 con 0x7f22880b4010 2023-09-23T03:28:44.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.094+0000 7f2284ff9640 1 -- 172.21.15.154:0/2201637461 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2278012750 con 0x7f22880b4010 2023-09-23T03:28:44.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.094+0000 7f228d87c640 1 -- 172.21.15.154:0/2201637461 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22880ac070 con 0x7f22880b4010 2023-09-23T03:28:44.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.094+0000 7f2284ff9640 1 -- 172.21.15.154:0/2201637461 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2278012b00 con 0x7f22880b4010 2023-09-23T03:28:44.096 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.094+0000 7f228d87c640 1 -- 172.21.15.154:0/2201637461 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22881f01a0 con 0x7f22880b4010 2023-09-23T03:28:44.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.095+0000 7f228d87c640 1 -- 172.21.15.154:0/2201637461 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2288067870 con 0x7f22880b4010 2023-09-23T03:28:44.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.096+0000 7f2284ff9640 1 -- 172.21.15.154:0/2201637461 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f2278012c60 con 0x7f22880b4010 2023-09-23T03:28:44.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.097+0000 7f2284ff9640 1 --2- 172.21.15.154:0/2201637461 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f226806af30 0x7f226806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:44.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.097+0000 7f2284ff9640 1 -- 172.21.15.154:0/2201637461 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f2278089f10 con 0x7f22880b4010 2023-09-23T03:28:44.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.097+0000 7f227ffff640 1 --2- 172.21.15.154:0/2201637461 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f226806af30 0x7f226806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:44.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.098+0000 7f227ffff640 1 --2- 172.21.15.154:0/2201637461 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f226806af30 0x7f226806d3f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f2270036000 tx=0x7f2270030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:44.101 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.100+0000 7f2284ff9640 1 -- 172.21.15.154:0/2201637461 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f2278054cc0 con 0x7f22880b4010 2023-09-23T03:28:44.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.326+0000 7f228d87c640 1 -- 172.21.15.154:0/2201637461 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2288073eb0 con 0x7f22880b4010 2023-09-23T03:28:44.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.327+0000 7f2284ff9640 1 -- 172.21.15.154:0/2201637461 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+4935 (secure 0 0 0) 0x7f22780582e0 con 0x7f22880b4010 2023-09-23T03:28:44.329 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:28:44.329 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":121,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/2759187683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2759187683},{"type":"v1","addr":"172.21.15.154:6835","nonce":2759187683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":5924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3712825179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3712825179},{"type":"v1","addr":"172.21.15.190:6835","nonce":3712825179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":110},{"gid":6544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/4147378242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":4147378242},{"type":"v1","addr":"172.21.15.190:6834","nonce":4147378242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:27:44.470312+0000","modified":"2023-09-23T03:28:35.478753+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9306":{"gid":9306,"name":"c","rank":0,"incarnation":118,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6837/1250699891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1250699891},{"type":"v1","addr":"172.21.15.154:6837","nonce":1250699891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-09-23T03:28:44.329 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 121 2023-09-23T03:28:44.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.330+0000 7f228d87c640 1 -- 172.21.15.154:0/2201637461 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f226806af30 msgr2=0x7f226806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:44.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.330+0000 7f228d87c640 1 --2- 172.21.15.154:0/2201637461 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f226806af30 0x7f226806d3f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f2270036000 tx=0x7f2270030000 comp rx=0 tx=0).stop 2023-09-23T03:28:44.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.330+0000 7f228d87c640 1 -- 172.21.15.154:0/2201637461 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22880b4010 msgr2=0x7f22881efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:44.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.330+0000 7f228d87c640 1 --2- 172.21.15.154:0/2201637461 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22880b4010 0x7f22881efc60 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f227800d5f0 tx=0x7f227800dab0 comp rx=0 tx=0).stop 2023-09-23T03:28:44.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.330+0000 7f228d87c640 1 -- 172.21.15.154:0/2201637461 shutdown_connections 2023-09-23T03:28:44.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.330+0000 7f228d87c640 1 --2- 172.21.15.154:0/2201637461 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f226806af30 0x7f226806d3f0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:44.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.330+0000 7f228d87c640 1 --2- 172.21.15.154:0/2201637461 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2288159dd0 0x7f22880ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:44.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.330+0000 7f228d87c640 1 --2- 172.21.15.154:0/2201637461 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f22880cddf0 0x7f228815a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:44.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.330+0000 7f228d87c640 1 --2- 172.21.15.154:0/2201637461 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22880b4010 0x7f22881efc60 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:44.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.330+0000 7f228d87c640 1 -- 172.21.15.154:0/2201637461 >> 172.21.15.154:0/2201637461 conn(0x7f22880cb070 msgr2=0x7f22880cc1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:44.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.330+0000 7f228d87c640 1 -- 172.21.15.154:0/2201637461 shutdown_connections 2023-09-23T03:28:44.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.331+0000 7f228d87c640 1 -- 172.21.15.154:0/2201637461 wait complete. 2023-09-23T03:28:44.351 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:28:44.556 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.555+0000 7fdaba256640 1 Processor -- start 2023-09-23T03:28:44.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.555+0000 7fdaba256640 1 -- start start 2023-09-23T03:28:44.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.556+0000 7fdaba256640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdab4159dd0 0x7fdab406d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:44.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.556+0000 7fdaba256640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdab40b4010 0x7fdab40cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:44.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.556+0000 7fdaba256640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fdab40cddf0 0x7fdab406c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:44.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.556+0000 7fdaba256640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdab40ce280 con 0x7fdab40b4010 2023-09-23T03:28:44.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.556+0000 7fdaba256640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdab40b0c10 con 0x7fdab40cddf0 2023-09-23T03:28:44.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.556+0000 7fdaba256640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdab40ad760 con 0x7fdab4159dd0 2023-09-23T03:28:44.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.557+0000 7fdab2ffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdab40b4010 0x7fdab40cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:44.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.557+0000 7fdab2ffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdab40b4010 0x7fdab40cd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56952/0 (socket says 172.21.15.154:56952) 2023-09-23T03:28:44.559 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.557+0000 7fdab2ffd640 1 -- 172.21.15.154:0/2464899916 learned_addr learned my addr 172.21.15.154:0/2464899916 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:44.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.557+0000 7fdab37fe640 1 --2- 172.21.15.154:0/2464899916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdab4159dd0 0x7fdab406d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:44.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.557+0000 7fdab3fff640 1 --2- 172.21.15.154:0/2464899916 >> v2:172.21.15.190:3300/0 conn(0x7fdab40cddf0 0x7fdab406c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:44.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.557+0000 7fdab2ffd640 1 -- 172.21.15.154:0/2464899916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdab4159dd0 msgr2=0x7fdab406d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:44.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.557+0000 7fdab2ffd640 1 --2- 172.21.15.154:0/2464899916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdab4159dd0 0x7fdab406d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:44.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.557+0000 7fdab2ffd640 1 -- 172.21.15.154:0/2464899916 >> v2:172.21.15.190:3300/0 conn(0x7fdab40cddf0 msgr2=0x7fdab406c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:44.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.557+0000 7fdab2ffd640 1 --2- 172.21.15.154:0/2464899916 >> v2:172.21.15.190:3300/0 conn(0x7fdab40cddf0 0x7fdab406c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:44.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.557+0000 7fdab2ffd640 1 -- 172.21.15.154:0/2464899916 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab40ada40 con 0x7fdab40b4010 2023-09-23T03:28:44.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.557+0000 7fdab37fe640 1 --2- 172.21.15.154:0/2464899916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdab4159dd0 0x7fdab406d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:44.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.557+0000 7fdab3fff640 1 --2- 172.21.15.154:0/2464899916 >> v2:172.21.15.190:3300/0 conn(0x7fdab40cddf0 0x7fdab406c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:44.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.558+0000 7fdab2ffd640 1 --2- 172.21.15.154:0/2464899916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdab40b4010 0x7fdab40cd8b0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7fdab40ae020 tx=0x7fdaa802b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7fb48e6c4f411af server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:44.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.559+0000 7fdab27fc640 1 -- 172.21.15.154:0/2464899916 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa8031880 con 0x7fdab40b4010 2023-09-23T03:28:44.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.559+0000 7fdab27fc640 1 -- 172.21.15.154:0/2464899916 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa80319e0 con 0x7fdab40b4010 2023-09-23T03:28:44.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.559+0000 7fdab27fc640 1 -- 172.21.15.154:0/2464899916 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa8031d70 con 0x7fdab40b4010 2023-09-23T03:28:44.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.559+0000 7fdaba256640 1 -- 172.21.15.154:0/2464899916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdab40b4010 msgr2=0x7fdab40cd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:44.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.559+0000 7fdaba256640 1 --2- 172.21.15.154:0/2464899916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdab40b4010 0x7fdab40cd8b0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7fdab40ae020 tx=0x7fdaa802b920 comp rx=0 tx=0).stop 2023-09-23T03:28:44.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.559+0000 7fdaba256640 1 -- 172.21.15.154:0/2464899916 shutdown_connections 2023-09-23T03:28:44.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.559+0000 7fdaba256640 1 --2- 172.21.15.154:0/2464899916 >> v2:172.21.15.190:3300/0 conn(0x7fdab40cddf0 0x7fdab406c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:44.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.559+0000 7fdaba256640 1 --2- 172.21.15.154:0/2464899916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdab40b4010 0x7fdab40cd8b0 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:44.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.559+0000 7fdaba256640 1 --2- 172.21.15.154:0/2464899916 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdab4159dd0 0x7fdab406d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:44.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.559+0000 7fdaba256640 1 -- 172.21.15.154:0/2464899916 >> 172.21.15.154:0/2464899916 conn(0x7fdab40cb070 msgr2=0x7fdab40cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:44.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.559+0000 7fdaba256640 1 -- 172.21.15.154:0/2464899916 shutdown_connections 2023-09-23T03:28:44.563 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.560+0000 7fdaba256640 1 -- 172.21.15.154:0/2464899916 wait complete. 2023-09-23T03:28:44.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.561+0000 7fdaba256640 1 Processor -- start 2023-09-23T03:28:44.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.561+0000 7fdaba256640 1 -- start start 2023-09-23T03:28:44.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.562+0000 7fdaba256640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdab40b4010 0x7fdab414d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:44.565 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.562+0000 7fdaba256640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdab40cddf0 0x7fdab40b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:44.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.562+0000 7fdaba256640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdab4159dd0 0x7fdab41570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:44.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.562+0000 7fdaba256640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab41580f0 con 0x7fdab4159dd0 2023-09-23T03:28:44.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.562+0000 7fdaba256640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab40afd60 con 0x7fdab40cddf0 2023-09-23T03:28:44.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.562+0000 7fdaba256640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdab4157b30 con 0x7fdab40b4010 2023-09-23T03:28:44.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.562+0000 7fdab3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdab4159dd0 0x7fdab41570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:44.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.563+0000 7fdab3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdab4159dd0 0x7fdab41570d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56966/0 (socket says 172.21.15.154:56966) 2023-09-23T03:28:44.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.563+0000 7fdab3fff640 1 -- 172.21.15.154:0/2632872464 learned_addr learned my addr 172.21.15.154:0/2632872464 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:44.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.563+0000 7fdab37fe640 1 --2- 172.21.15.154:0/2632872464 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdab40b4010 0x7fdab414d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:44.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.563+0000 7fdab2ffd640 1 --2- 172.21.15.154:0/2632872464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdab40cddf0 0x7fdab40b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:44.566 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.563+0000 7fdab3fff640 1 -- 172.21.15.154:0/2632872464 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdab40b4010 msgr2=0x7fdab414d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:44.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.563+0000 7fdab3fff640 1 --2- 172.21.15.154:0/2632872464 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdab40b4010 0x7fdab414d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:44.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.563+0000 7fdab3fff640 1 -- 172.21.15.154:0/2632872464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdab40cddf0 msgr2=0x7fdab40b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:44.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.563+0000 7fdab3fff640 1 --2- 172.21.15.154:0/2632872464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdab40cddf0 0x7fdab40b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:44.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.563+0000 7fdab3fff640 1 -- 172.21.15.154:0/2632872464 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa80094d0 con 0x7fdab4159dd0 2023-09-23T03:28:44.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.563+0000 7fdab2ffd640 1 --2- 172.21.15.154:0/2632872464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdab40cddf0 0x7fdab40b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:44.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.563+0000 7fdab37fe640 1 --2- 172.21.15.154:0/2632872464 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdab40b4010 0x7fdab414d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:44.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.563+0000 7fdab3fff640 1 --2- 172.21.15.154:0/2632872464 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdab4159dd0 0x7fdab41570d0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fdaac00cd10 tx=0x7fdaac00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:44.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.564+0000 7fda9bfff640 1 -- 172.21.15.154:0/2632872464 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaac00f680 con 0x7fdab4159dd0 2023-09-23T03:28:44.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.564+0000 7fda9bfff640 1 -- 172.21.15.154:0/2632872464 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaac00f800 con 0x7fdab4159dd0 2023-09-23T03:28:44.567 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.564+0000 7fda9bfff640 1 -- 172.21.15.154:0/2632872464 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaac00fbb0 con 0x7fdab4159dd0 2023-09-23T03:28:44.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.564+0000 7fdaba256640 1 -- 172.21.15.154:0/2632872464 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdab4157e10 con 0x7fdab4159dd0 2023-09-23T03:28:44.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.564+0000 7fdaba256640 1 -- 172.21.15.154:0/2632872464 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdab415b280 con 0x7fdab4159dd0 2023-09-23T03:28:44.568 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.565+0000 7fdaba256640 1 -- 172.21.15.154:0/2632872464 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdab4067870 con 0x7fdab4159dd0 2023-09-23T03:28:44.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.566+0000 7fda9bfff640 1 -- 172.21.15.154:0/2632872464 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fdaac00b040 con 0x7fdab4159dd0 2023-09-23T03:28:44.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.567+0000 7fda9bfff640 1 --2- 172.21.15.154:0/2632872464 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdaa006af30 0x7fdaa006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:44.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.567+0000 7fda9bfff640 1 -- 172.21.15.154:0/2632872464 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6201+0+0 (secure 0 0 0) 0x7fdaac08a320 con 0x7fdab4159dd0 2023-09-23T03:28:44.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.567+0000 7fdab37fe640 1 --2- 172.21.15.154:0/2632872464 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdaa006af30 0x7fdaa006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:44.569 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.568+0000 7fdab37fe640 1 --2- 172.21.15.154:0/2632872464 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdaa006af30 0x7fdaa006d3f0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4006fd0 tx=0x7fdaa4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:44.572 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.571+0000 7fda9bfff640 1 -- 172.21.15.154:0/2632872464 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fdaac0551a0 con 0x7fdab4159dd0 2023-09-23T03:28:44.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.761+0000 7fdaba256640 1 -- 172.21.15.154:0/2632872464 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdab415afa0 con 0x7fdab4159dd0 2023-09-23T03:28:44.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.764+0000 7fda9bfff640 1 -- 172.21.15.154:0/2632872464 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+17083 (secure 0 0 0) 0x7fdab415afa0 con 0x7fdab4159dd0 2023-09-23T03:28:44.766 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:28:44.766 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":128,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:28:29.295164+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:27:42.207617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-09-23T03:27:43.473867+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6837/4006549651":"2023-09-24T03:28:29.295137+0000","172.21.15.154:6836/4006549651":"2023-09-24T03:28:29.295137+0000","172.21.15.190:6835/3558096060":"2023-09-24T03:28:08.468628+0000","172.21.15.190:6833/3558096060":"2023-09-24T03:28:08.468628+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:28:44.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.767+0000 7fdaba256640 1 -- 172.21.15.154:0/2632872464 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdaa006af30 msgr2=0x7fdaa006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:44.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.767+0000 7fdaba256640 1 --2- 172.21.15.154:0/2632872464 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdaa006af30 0x7fdaa006d3f0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4006fd0 tx=0x7fdaa4008040 comp rx=0 tx=0).stop 2023-09-23T03:28:44.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.767+0000 7fdaba256640 1 -- 172.21.15.154:0/2632872464 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdab4159dd0 msgr2=0x7fdab41570d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:44.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.767+0000 7fdaba256640 1 --2- 172.21.15.154:0/2632872464 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdab4159dd0 0x7fdab41570d0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fdaac00cd10 tx=0x7fdaac00a650 comp rx=0 tx=0).stop 2023-09-23T03:28:44.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.767+0000 7fdaba256640 1 -- 172.21.15.154:0/2632872464 shutdown_connections 2023-09-23T03:28:44.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.767+0000 7fdaba256640 1 --2- 172.21.15.154:0/2632872464 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdaa006af30 0x7fdaa006d3f0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:44.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.767+0000 7fdaba256640 1 --2- 172.21.15.154:0/2632872464 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdab4159dd0 0x7fdab41570d0 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:44.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.767+0000 7fdaba256640 1 --2- 172.21.15.154:0/2632872464 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdab40cddf0 0x7fdab40b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:44.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.767+0000 7fdaba256640 1 --2- 172.21.15.154:0/2632872464 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdab40b4010 0x7fdab414d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:44.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.767+0000 7fdaba256640 1 -- 172.21.15.154:0/2632872464 >> 172.21.15.154:0/2632872464 conn(0x7fdab40cb070 msgr2=0x7fdab40cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:44.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.767+0000 7fdaba256640 1 -- 172.21.15.154:0/2632872464 shutdown_connections 2023-09-23T03:28:44.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.767+0000 7fdaba256640 1 -- 172.21.15.154:0/2632872464 wait complete. 2023-09-23T03:28:44.786 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-23T03:15:09.236980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-23T03:27:42.207617+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-09-23T03:27:43.473867+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-23T03:28:44.787 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-23T03:28:44.813 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:28:44.840 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:28:44.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.994+0000 7f58d359e640 1 Processor -- start 2023-09-23T03:28:44.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.995+0000 7f58d359e640 1 -- start start 2023-09-23T03:28:44.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.996+0000 7f58d359e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58d415ca40 0x7f58d406b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:44.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.996+0000 7f58d359e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f58d40d2590 0x7f58d40d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:44.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.996+0000 7f58d359e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58d40d09c0 con 0x7f58d40cd730 2023-09-23T03:28:44.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.996+0000 7f58d359e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58d40ce440 con 0x7f58d40d2590 2023-09-23T03:28:44.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.996+0000 7f58d359e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d40ce6e0 con 0x7f58d415ca40 2023-09-23T03:28:44.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.997+0000 7f58d1d9b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f58d40d2590 0x7f58d40d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:44.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.997+0000 7f58d2d9d640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f58d40cd730 0x7f58d40cdbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:34032/0 (socket says 172.21.15.154:34032) 2023-09-23T03:28:44.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.997+0000 7f58d2d9d640 1 -- 172.21.15.154:0/1721110585 learned_addr learned my addr 172.21.15.154:0/1721110585 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:28:44.999 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.997+0000 7f58d259c640 1 --2- 172.21.15.154:0/1721110585 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58d415ca40 0x7f58d406b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:44.999 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.997+0000 7f58d1d9b640 1 -- 172.21.15.154:0/1721110585 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58d415ca40 msgr2=0x7f58d406b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:44.999 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.997+0000 7f58d1d9b640 1 --2- 172.21.15.154:0/1721110585 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58d415ca40 0x7f58d406b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:45.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.997+0000 7f58d1d9b640 1 -- 172.21.15.154:0/1721110585 >> v1:172.21.15.154:6789/0 conn(0x7f58d40cd730 legacy=0x7f58d40cdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:45.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.997+0000 7f58d1d9b640 1 -- 172.21.15.154:0/1721110585 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d40d0370 con 0x7f58d40d2590 2023-09-23T03:28:45.000 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.998+0000 7f58d1d9b640 1 --2- 172.21.15.154:0/1721110585 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58d40d2590 0x7f58d40d29b0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f58c80097a0 tx=0x7f58c802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2249563670513413 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:45.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.999+0000 7f58d159a640 1 -- 172.21.15.154:0/1721110585 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c802d040 con 0x7f58d40d2590 2023-09-23T03:28:45.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.999+0000 7f58d159a640 1 -- 172.21.15.154:0/1721110585 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c8031810 con 0x7f58d40d2590 2023-09-23T03:28:45.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.999+0000 7f58d159a640 1 -- 172.21.15.154:0/1721110585 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8031ba0 con 0x7f58d40d2590 2023-09-23T03:28:45.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.999+0000 7f58d359e640 1 -- 172.21.15.154:0/1721110585 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58d40d2590 msgr2=0x7f58d40d29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:45.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.999+0000 7f58d359e640 1 --2- 172.21.15.154:0/1721110585 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58d40d2590 0x7f58d40d29b0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f58c80097a0 tx=0x7f58c802b8b0 comp rx=0 tx=0).stop 2023-09-23T03:28:45.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.999+0000 7f58d359e640 1 -- 172.21.15.154:0/1721110585 shutdown_connections 2023-09-23T03:28:45.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.999+0000 7f58d359e640 1 --2- 172.21.15.154:0/1721110585 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58d40d2590 0x7f58d40d29b0 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:45.001 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.999+0000 7f58d359e640 1 --2- 172.21.15.154:0/1721110585 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58d415ca40 0x7f58d406b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:45.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.999+0000 7f58d359e640 1 -- 172.21.15.154:0/1721110585 >> 172.21.15.154:0/1721110585 conn(0x7f58d408c1f0 msgr2=0x7f58d409bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:45.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.999+0000 7f58d359e640 1 -- 172.21.15.154:0/1721110585 shutdown_connections 2023-09-23T03:28:45.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:44.999+0000 7f58d359e640 1 -- 172.21.15.154:0/1721110585 wait complete. 2023-09-23T03:28:45.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.001+0000 7f58d359e640 1 Processor -- start 2023-09-23T03:28:45.002 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.001+0000 7f58d359e640 1 -- start start 2023-09-23T03:28:45.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.002+0000 7f58d359e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58d40cd730 0x7f58d40c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:45.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.002+0000 7f58d359e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58d40d2590 0x7f58d41f5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:45.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.002+0000 7f58d359e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58d415ca40 0x7f58d415d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:45.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.002+0000 7f58d359e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d41f1ec0 con 0x7f58d40d2590 2023-09-23T03:28:45.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.002+0000 7f58d359e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d41f64e0 con 0x7f58d415ca40 2023-09-23T03:28:45.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.002+0000 7f58d359e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d41f5f20 con 0x7f58d40cd730 2023-09-23T03:28:45.005 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.002+0000 7f58d1d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58d40d2590 0x7f58d41f5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:45.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.002+0000 7f58d1d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58d40d2590 0x7f58d41f5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56984/0 (socket says 172.21.15.154:56984) 2023-09-23T03:28:45.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.002+0000 7f58d1d9b640 1 -- 172.21.15.154:0/3818499291 learned_addr learned my addr 172.21.15.154:0/3818499291 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:45.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.003+0000 7f58d2d9d640 1 --2- 172.21.15.154:0/3818499291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58d415ca40 0x7f58d415d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:45.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.003+0000 7f58d1d9b640 1 -- 172.21.15.154:0/3818499291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58d40cd730 msgr2=0x7f58d40c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:45.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.003+0000 7f58d1d9b640 1 --2- 172.21.15.154:0/3818499291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58d40cd730 0x7f58d40c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:45.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.003+0000 7f58d1d9b640 1 -- 172.21.15.154:0/3818499291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58d415ca40 msgr2=0x7f58d415d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:45.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.003+0000 7f58d1d9b640 1 --2- 172.21.15.154:0/3818499291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58d415ca40 0x7f58d415d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:45.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.003+0000 7f58d1d9b640 1 -- 172.21.15.154:0/3818499291 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c8009400 con 0x7f58d40d2590 2023-09-23T03:28:45.006 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.003+0000 7f58d2d9d640 1 --2- 172.21.15.154:0/3818499291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58d415ca40 0x7f58d415d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:45.009 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.003+0000 7f58d1d9b640 1 --2- 172.21.15.154:0/3818499291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58d40d2590 0x7f58d41f5560 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f58c8000c00 tx=0x7f58c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:45.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.003+0000 7f58c2ffd640 1 -- 172.21.15.154:0/3818499291 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c803ad60 con 0x7f58d40d2590 2023-09-23T03:28:45.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.003+0000 7f58c2ffd640 1 -- 172.21.15.154:0/3818499291 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c80443f0 con 0x7f58d40d2590 2023-09-23T03:28:45.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.003+0000 7f58c2ffd640 1 -- 172.21.15.154:0/3818499291 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c80447a0 con 0x7f58d40d2590 2023-09-23T03:28:45.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.004+0000 7f58d359e640 1 -- 172.21.15.154:0/3818499291 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58d41f6200 con 0x7f58d40d2590 2023-09-23T03:28:45.010 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.004+0000 7f58d359e640 1 -- 172.21.15.154:0/3818499291 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58d40c7730 con 0x7f58d40d2590 2023-09-23T03:28:45.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.005+0000 7f58d359e640 1 -- 172.21.15.154:0/3818499291 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58d404e920 con 0x7f58d40d2590 2023-09-23T03:28:45.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.006+0000 7f58c2ffd640 1 -- 172.21.15.154:0/3818499291 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f58c804d450 con 0x7f58d40d2590 2023-09-23T03:28:45.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.006+0000 7f58c2ffd640 1 --2- 172.21.15.154:0/3818499291 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f58b406af30 0x7f58b406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:45.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.007+0000 7f58c2ffd640 1 -- 172.21.15.154:0/3818499291 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f58c80abd50 con 0x7f58d40d2590 2023-09-23T03:28:45.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.007+0000 7f58d259c640 1 --2- 172.21.15.154:0/3818499291 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f58b406af30 0x7f58b406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:45.011 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.007+0000 7f58d259c640 1 --2- 172.21.15.154:0/3818499291 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f58b406af30 0x7f58b406d3f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f58c4006e20 tx=0x7f58c4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:45.012 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.011+0000 7f58c2ffd640 1 -- 172.21.15.154:0/3818499291 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f58c8076d30 con 0x7f58d40d2590 2023-09-23T03:28:45.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.203+0000 7f58d359e640 1 -- 172.21.15.154:0/3818499291 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f58d406afb0 con 0x7f58d40d2590 2023-09-23T03:28:45.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.204+0000 7f58c2ffd640 1 -- 172.21.15.154:0/3818499291 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v128) v1 ==== 93+0+10 (secure 0 0 0) 0x7f58c807a350 con 0x7f58d40d2590 2023-09-23T03:28:45.205 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2023-09-23T03:28:45.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.207+0000 7f58d359e640 1 -- 172.21.15.154:0/3818499291 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f58b406af30 msgr2=0x7f58b406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:45.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.207+0000 7f58d359e640 1 --2- 172.21.15.154:0/3818499291 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f58b406af30 0x7f58b406d3f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f58c4006e20 tx=0x7f58c4008040 comp rx=0 tx=0).stop 2023-09-23T03:28:45.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.207+0000 7f58d359e640 1 -- 172.21.15.154:0/3818499291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58d40d2590 msgr2=0x7f58d41f5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:45.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.207+0000 7f58d359e640 1 --2- 172.21.15.154:0/3818499291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58d40d2590 0x7f58d41f5560 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f58c8000c00 tx=0x7f58c80046c0 comp rx=0 tx=0).stop 2023-09-23T03:28:45.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.207+0000 7f58d359e640 1 -- 172.21.15.154:0/3818499291 shutdown_connections 2023-09-23T03:28:45.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.207+0000 7f58d359e640 1 --2- 172.21.15.154:0/3818499291 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f58b406af30 0x7f58b406d3f0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:45.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.207+0000 7f58d359e640 1 --2- 172.21.15.154:0/3818499291 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f58d415ca40 0x7f58d415d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:45.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.207+0000 7f58d359e640 1 --2- 172.21.15.154:0/3818499291 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58d40d2590 0x7f58d41f5560 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:45.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.207+0000 7f58d359e640 1 --2- 172.21.15.154:0/3818499291 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f58d40cd730 0x7f58d40c5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:45.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.207+0000 7f58d359e640 1 -- 172.21.15.154:0/3818499291 >> 172.21.15.154:0/3818499291 conn(0x7f58d408c1f0 msgr2=0x7f58d40b38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:45.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.207+0000 7f58d359e640 1 -- 172.21.15.154:0/3818499291 shutdown_connections 2023-09-23T03:28:45.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.207+0000 7f58d359e640 1 -- 172.21.15.154:0/3818499291 wait complete. 2023-09-23T03:28:45.228 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-23T03:28:45.496 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.495+0000 7fd90c12c640 1 Processor -- start 2023-09-23T03:28:45.497 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.495+0000 7fd90c12c640 1 -- start start 2023-09-23T03:28:45.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.497+0000 7fd90c12c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd9040708e0 0x7fd904070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:45.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.497+0000 7fd90c12c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9040c9b10 0x7fd90415d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:45.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.497+0000 7fd90c12c640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd904071200 con 0x7fd90415d5f0 2023-09-23T03:28:45.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.497+0000 7fd90c12c640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd90406c1b0 con 0x7fd9040708e0 2023-09-23T03:28:45.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.497+0000 7fd90c12c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd90415dde0 con 0x7fd9040c9b10 2023-09-23T03:28:45.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.497+0000 7fd909ec8640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd9040708e0 0x7fd904070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:45.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.497+0000 7fd909ec8640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd9040708e0 0x7fd904070cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:56992/0 (socket says 172.21.15.154:56992) 2023-09-23T03:28:45.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.497+0000 7fd909ec8640 1 -- 172.21.15.154:0/2939209902 learned_addr learned my addr 172.21.15.154:0/2939209902 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:45.499 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.498+0000 7fd9096c7640 1 --2- 172.21.15.154:0/2939209902 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9040c9b10 0x7fd90415d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:45.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.498+0000 7fd909ec8640 1 -- 172.21.15.154:0/2939209902 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9040c9b10 msgr2=0x7fd90415d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:45.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.498+0000 7fd909ec8640 1 --2- 172.21.15.154:0/2939209902 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9040c9b10 0x7fd90415d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:45.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.498+0000 7fd909ec8640 1 -- 172.21.15.154:0/2939209902 >> v1:172.21.15.190:6789/0 conn(0x7fd90415d5f0 legacy=0x7fd90406cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:45.500 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.498+0000 7fd909ec8640 1 -- 172.21.15.154:0/2939209902 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd90406c3c0 con 0x7fd9040708e0 2023-09-23T03:28:45.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.500+0000 7fd909ec8640 1 --2- 172.21.15.154:0/2939209902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9040708e0 0x7fd904070cc0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fd900009870 tx=0x7fd90002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9923780eeab2c9f6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:45.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.500+0000 7fd908ec6640 1 -- 172.21.15.154:0/2939209902 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd900030840 con 0x7fd9040708e0 2023-09-23T03:28:45.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.500+0000 7fd908ec6640 1 -- 172.21.15.154:0/2939209902 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9000309a0 con 0x7fd9040708e0 2023-09-23T03:28:45.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.500+0000 7fd908ec6640 1 -- 172.21.15.154:0/2939209902 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd900030d30 con 0x7fd9040708e0 2023-09-23T03:28:45.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.501+0000 7fd90c12c640 1 -- 172.21.15.154:0/2939209902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9040708e0 msgr2=0x7fd904070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:45.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.501+0000 7fd90c12c640 1 --2- 172.21.15.154:0/2939209902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9040708e0 0x7fd904070cc0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fd900009870 tx=0x7fd90002b980 comp rx=0 tx=0).stop 2023-09-23T03:28:45.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.501+0000 7fd90c12c640 1 -- 172.21.15.154:0/2939209902 shutdown_connections 2023-09-23T03:28:45.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.501+0000 7fd90c12c640 1 --2- 172.21.15.154:0/2939209902 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9040c9b10 0x7fd90415d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:45.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.501+0000 7fd90c12c640 1 --2- 172.21.15.154:0/2939209902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9040708e0 0x7fd904070cc0 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:45.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.501+0000 7fd90c12c640 1 -- 172.21.15.154:0/2939209902 >> 172.21.15.154:0/2939209902 conn(0x7fd90406a440 msgr2=0x7fd9040b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:45.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.501+0000 7fd90c12c640 1 -- 172.21.15.154:0/2939209902 shutdown_connections 2023-09-23T03:28:45.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.501+0000 7fd90c12c640 1 -- 172.21.15.154:0/2939209902 wait complete. 2023-09-23T03:28:45.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.503+0000 7fd90c12c640 1 Processor -- start 2023-09-23T03:28:45.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.503+0000 7fd90c12c640 1 -- start start 2023-09-23T03:28:45.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.504+0000 7fd90c12c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9040708e0 0x7fd904145520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:45.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.504+0000 7fd90c12c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9040c9b10 0x7fd904149c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:45.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.504+0000 7fd90c12c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd90415d5f0 0x7fd9040cc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:45.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.504+0000 7fd90c12c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90414d910 con 0x7fd9040708e0 2023-09-23T03:28:45.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.504+0000 7fd90c12c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90414c870 con 0x7fd9040c9b10 2023-09-23T03:28:45.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.504+0000 7fd90c12c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd90414cb50 con 0x7fd90415d5f0 2023-09-23T03:28:45.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.504+0000 7fd909ec8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9040708e0 0x7fd904145520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:45.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.504+0000 7fd909ec8640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9040708e0 0x7fd904145520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57000/0 (socket says 172.21.15.154:57000) 2023-09-23T03:28:45.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.504+0000 7fd909ec8640 1 -- 172.21.15.154:0/4101386424 learned_addr learned my addr 172.21.15.154:0/4101386424 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:45.506 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.505+0000 7fd909ec8640 1 -- 172.21.15.154:0/4101386424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd90415d5f0 msgr2=0x7fd9040cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:45.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.505+0000 7fd909ec8640 1 --2- 172.21.15.154:0/4101386424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd90415d5f0 0x7fd9040cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:45.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.505+0000 7fd909ec8640 1 -- 172.21.15.154:0/4101386424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9040c9b10 msgr2=0x7fd904149c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:45.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.505+0000 7fd909ec8640 1 --2- 172.21.15.154:0/4101386424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9040c9b10 0x7fd904149c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:45.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.505+0000 7fd909ec8640 1 -- 172.21.15.154:0/4101386424 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9000094d0 con 0x7fd9040708e0 2023-09-23T03:28:45.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.505+0000 7fd909ec8640 1 --2- 172.21.15.154:0/4101386424 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9040708e0 0x7fd904145520 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7fd900000c00 tx=0x7fd900004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:45.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.505+0000 7fd8fa7fc640 1 -- 172.21.15.154:0/4101386424 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd900039c90 con 0x7fd9040708e0 2023-09-23T03:28:45.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.505+0000 7fd8fa7fc640 1 -- 172.21.15.154:0/4101386424 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd900039df0 con 0x7fd9040708e0 2023-09-23T03:28:45.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.505+0000 7fd8fa7fc640 1 -- 172.21.15.154:0/4101386424 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd900043640 con 0x7fd9040708e0 2023-09-23T03:28:45.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.505+0000 7fd90c12c640 1 -- 172.21.15.154:0/4101386424 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd90414ce30 con 0x7fd9040708e0 2023-09-23T03:28:45.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.505+0000 7fd90c12c640 1 -- 172.21.15.154:0/4101386424 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd90414d3f0 con 0x7fd9040708e0 2023-09-23T03:28:45.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.507+0000 7fd90c12c640 1 -- 172.21.15.154:0/4101386424 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd904073eb0 con 0x7fd9040708e0 2023-09-23T03:28:45.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.507+0000 7fd8fa7fc640 1 -- 172.21.15.154:0/4101386424 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fd9000437a0 con 0x7fd9040708e0 2023-09-23T03:28:45.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.509+0000 7fd8fa7fc640 1 --2- 172.21.15.154:0/4101386424 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd8e806af30 0x7fd8e806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:45.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.509+0000 7fd8fa7fc640 1 -- 172.21.15.154:0/4101386424 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6201+0+0 (secure 0 0 0) 0x7fd90003e070 con 0x7fd9040708e0 2023-09-23T03:28:45.510 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.509+0000 7fd9096c7640 1 --2- 172.21.15.154:0/4101386424 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd8e806af30 0x7fd8e806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:45.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.510+0000 7fd9096c7640 1 --2- 172.21.15.154:0/4101386424 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd8e806af30 0x7fd8e806d3f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fd9040ccff0 tx=0x7fd8f40095a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:45.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.513+0000 7fd8fa7fc640 1 -- 172.21.15.154:0/4101386424 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fd900077a20 con 0x7fd9040708e0 2023-09-23T03:28:45.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.703+0000 7fd90c12c640 1 -- 172.21.15.154:0/4101386424 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd9040b0d60 con 0x7fd9040708e0 2023-09-23T03:28:45.705 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.704+0000 7fd8fa7fc640 1 -- 172.21.15.154:0/4101386424 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v128) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd9000340a0 con 0x7fd9040708e0 2023-09-23T03:28:45.706 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 32 2023-09-23T03:28:45.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.706+0000 7fd90c12c640 1 -- 172.21.15.154:0/4101386424 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd8e806af30 msgr2=0x7fd8e806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:45.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.706+0000 7fd90c12c640 1 --2- 172.21.15.154:0/4101386424 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd8e806af30 0x7fd8e806d3f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fd9040ccff0 tx=0x7fd8f40095a0 comp rx=0 tx=0).stop 2023-09-23T03:28:45.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.706+0000 7fd90c12c640 1 -- 172.21.15.154:0/4101386424 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9040708e0 msgr2=0x7fd904145520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:45.708 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.706+0000 7fd90c12c640 1 --2- 172.21.15.154:0/4101386424 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9040708e0 0x7fd904145520 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7fd900000c00 tx=0x7fd900004730 comp rx=0 tx=0).stop 2023-09-23T03:28:45.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.706+0000 7fd90c12c640 1 -- 172.21.15.154:0/4101386424 shutdown_connections 2023-09-23T03:28:45.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.706+0000 7fd90c12c640 1 --2- 172.21.15.154:0/4101386424 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd8e806af30 0x7fd8e806d3f0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:45.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.706+0000 7fd90c12c640 1 --2- 172.21.15.154:0/4101386424 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd90415d5f0 0x7fd9040cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:45.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.706+0000 7fd90c12c640 1 --2- 172.21.15.154:0/4101386424 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd9040c9b10 0x7fd904149c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:45.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.706+0000 7fd90c12c640 1 --2- 172.21.15.154:0/4101386424 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9040708e0 0x7fd904145520 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:45.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.707+0000 7fd90c12c640 1 -- 172.21.15.154:0/4101386424 >> 172.21.15.154:0/4101386424 conn(0x7fd90406a440 msgr2=0x7fd9041554f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:45.709 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.707+0000 7fd90c12c640 1 -- 172.21.15.154:0/4101386424 shutdown_connections 2023-09-23T03:28:45.710 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.707+0000 7fd90c12c640 1 -- 172.21.15.154:0/4101386424 wait complete. 2023-09-23T03:28:45.725 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-23T03:28:45.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.936+0000 7fd8b059e640 1 Processor -- start 2023-09-23T03:28:45.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.937+0000 7fd8b059e640 1 -- start start 2023-09-23T03:28:45.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.938+0000 7fd8b059e640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd8a80ce640 0x7fd8a80cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:45.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.938+0000 7fd8b059e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8a80ce110 0x7fd8a8159f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:45.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.938+0000 7fd8b059e640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8a80cef60 con 0x7fd8a815a4c0 2023-09-23T03:28:45.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.938+0000 7fd8b059e640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8a815acf0 con 0x7fd8a80ce640 2023-09-23T03:28:45.940 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.938+0000 7fd8b059e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a80cd350 con 0x7fd8a80ce110 2023-09-23T03:28:45.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.939+0000 7fd8aeb3b640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fd8a815a4c0 0x7fd8a80b1160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:34034/0 (socket says 172.21.15.154:34034) 2023-09-23T03:28:45.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.939+0000 7fd8aeb3b640 1 -- 172.21.15.154:0/4280074979 learned_addr learned my addr 172.21.15.154:0/4280074979 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:28:45.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.939+0000 7fd8ae33a640 1 --2- 172.21.15.154:0/4280074979 >> v2:172.21.15.190:3300/0 conn(0x7fd8a80ce640 0x7fd8a80cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:45.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.939+0000 7fd8adb39640 1 --2- 172.21.15.154:0/4280074979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8a80ce110 0x7fd8a8159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:45.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.939+0000 7fd8ad338640 1 -- 172.21.15.154:0/4280074979 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3310244002 0 0) 0x7fd8a80cef60 con 0x7fd8a815a4c0 2023-09-23T03:28:45.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.939+0000 7fd8ad338640 1 -- 172.21.15.154:0/4280074979 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd890003520 con 0x7fd8a815a4c0 2023-09-23T03:28:45.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.939+0000 7fd8ad338640 1 -- 172.21.15.154:0/4280074979 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2981340981 0 0) 0x7fd890003520 con 0x7fd8a815a4c0 2023-09-23T03:28:45.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.939+0000 7fd8ad338640 1 -- 172.21.15.154:0/4280074979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8a80ce110 msgr2=0x7fd8a8159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:45.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.939+0000 7fd8ad338640 1 --2- 172.21.15.154:0/4280074979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8a80ce110 0x7fd8a8159f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:45.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.939+0000 7fd8ad338640 1 -- 172.21.15.154:0/4280074979 >> v2:172.21.15.190:3300/0 conn(0x7fd8a80ce640 msgr2=0x7fd8a80cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:45.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.939+0000 7fd8ad338640 1 --2- 172.21.15.154:0/4280074979 >> v2:172.21.15.190:3300/0 conn(0x7fd8a80ce640 0x7fd8a80cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:45.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.939+0000 7fd8ad338640 1 -- 172.21.15.154:0/4280074979 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a80cd630 con 0x7fd8a815a4c0 2023-09-23T03:28:45.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.940+0000 7fd8ad338640 1 -- 172.21.15.154:0/4280074979 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fd8a00019f0 con 0x7fd8a815a4c0 2023-09-23T03:28:45.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.940+0000 7fd8ad338640 1 -- 172.21.15.154:0/4280074979 >> v1:172.21.15.154:6789/0 conn(0x7fd8a815a4c0 legacy=0x7fd8a80b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:45.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.940+0000 7fd8ad338640 1 --2- 172.21.15.154:0/4280074979 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd890003970 0x7fd890003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:45.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.940+0000 7fd8ad338640 1 --2- 172.21.15.154:0/4280074979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8a80ce110 0x7fd8900047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:45.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.940+0000 7fd8ad338640 1 --2- 172.21.15.154:0/4280074979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd890004d00 0x7fd890005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:45.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.940+0000 7fd8ad338640 1 -- 172.21.15.154:0/4280074979 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a815acf0 con 0x7fd890003970 2023-09-23T03:28:45.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.940+0000 7fd8ad338640 1 -- 172.21.15.154:0/4280074979 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a80cd350 con 0x7fd890004d00 2023-09-23T03:28:45.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.940+0000 7fd8ad338640 1 -- 172.21.15.154:0/4280074979 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd89000fd90 con 0x7fd8a80ce110 2023-09-23T03:28:45.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.940+0000 7fd8ae33a640 1 --2- 172.21.15.154:0/4280074979 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd890003970 0x7fd890003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:45.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.940+0000 7fd8adb39640 1 --2- 172.21.15.154:0/4280074979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8a80ce110 0x7fd8900047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:45.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.940+0000 7fd8ae33a640 1 -- 172.21.15.154:0/4280074979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8a80ce110 msgr2=0x7fd8900047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:45.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.940+0000 7fd8ae33a640 1 --2- 172.21.15.154:0/4280074979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8a80ce110 0x7fd8900047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:45.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.940+0000 7fd8ae33a640 1 -- 172.21.15.154:0/4280074979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd890004d00 msgr2=0x7fd890005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:45.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.940+0000 7fd8ae33a640 1 --2- 172.21.15.154:0/4280074979 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd890004d00 0x7fd890005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:45.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.940+0000 7fd8ae33a640 1 -- 172.21.15.154:0/4280074979 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd890003520 con 0x7fd890003970 2023-09-23T03:28:45.945 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.940+0000 7fd8adb39640 1 --2- 172.21.15.154:0/4280074979 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8a80ce110 0x7fd8900047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:45.948 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.941+0000 7fd8ae33a640 1 --2- 172.21.15.154:0/4280074979 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd890003970 0x7fd890003d90 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7fd898009ac0 tx=0x7fd8900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de1a984ccfa2e192 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:45.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.942+0000 7fd8ad338640 1 -- 172.21.15.154:0/4280074979 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89802c710 con 0x7fd890003970 2023-09-23T03:28:45.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.942+0000 7fd8ad338640 1 -- 172.21.15.154:0/4280074979 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd89802c870 con 0x7fd890003970 2023-09-23T03:28:45.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.942+0000 7fd8ad338640 1 -- 172.21.15.154:0/4280074979 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89802cc00 con 0x7fd890003970 2023-09-23T03:28:45.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.942+0000 7fd8b059e640 1 -- 172.21.15.154:0/4280074979 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd890003970 msgr2=0x7fd890003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:45.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.942+0000 7fd8b059e640 1 --2- 172.21.15.154:0/4280074979 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd890003970 0x7fd890003d90 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7fd898009ac0 tx=0x7fd8900057f0 comp rx=0 tx=0).stop 2023-09-23T03:28:45.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.942+0000 7fd8aeb3b640 1 -- 172.21.15.154:0/4280074979 reap_dead start 2023-09-23T03:28:45.949 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.942+0000 7fd8b059e640 1 -- 172.21.15.154:0/4280074979 shutdown_connections 2023-09-23T03:28:45.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.942+0000 7fd8b059e640 1 -- 172.21.15.154:0/4280074979 >> 172.21.15.154:0/4280074979 conn(0x7fd8a806ec40 msgr2=0x7fd8a806f020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:45.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.942+0000 7fd8b059e640 1 -- 172.21.15.154:0/4280074979 shutdown_connections 2023-09-23T03:28:45.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.942+0000 7fd8b059e640 1 -- 172.21.15.154:0/4280074979 wait complete. 2023-09-23T03:28:45.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.944+0000 7fd8b059e640 1 Processor -- start 2023-09-23T03:28:45.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.944+0000 7fd8b059e640 1 -- start start 2023-09-23T03:28:45.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.945+0000 7fd8b059e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd890003970 0x7fd8a814fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:45.950 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.945+0000 7fd8b059e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8a815b3f0 0x7fd8a80b4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:45.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.945+0000 7fd8b059e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8a815b850 0x7fd8a80b4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:45.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.945+0000 7fd8b059e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a80c6220 con 0x7fd8a815b850 2023-09-23T03:28:45.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.945+0000 7fd8b059e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a80c5f40 con 0x7fd8a815b3f0 2023-09-23T03:28:45.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.945+0000 7fd8b059e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a80af1e0 con 0x7fd890003970 2023-09-23T03:28:45.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.945+0000 7fd8aeb3b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8a815b850 0x7fd8a80b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:45.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.946+0000 7fd8aeb3b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8a815b850 0x7fd8a80b4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57016/0 (socket says 172.21.15.154:57016) 2023-09-23T03:28:45.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.946+0000 7fd8aeb3b640 1 -- 172.21.15.154:0/2656346698 learned_addr learned my addr 172.21.15.154:0/2656346698 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:45.951 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.946+0000 7fd8aeb3b640 1 -- 172.21.15.154:0/2656346698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd890003970 msgr2=0x7fd8a814fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:45.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.946+0000 7fd8ae33a640 1 --2- 172.21.15.154:0/2656346698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd890003970 0x7fd8a814fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:45.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.946+0000 7fd8adb39640 1 --2- 172.21.15.154:0/2656346698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8a815b3f0 0x7fd8a80b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:45.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.946+0000 7fd8aeb3b640 1 --2- 172.21.15.154:0/2656346698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd890003970 0x7fd8a814fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:45.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.946+0000 7fd8aeb3b640 1 -- 172.21.15.154:0/2656346698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8a815b3f0 msgr2=0x7fd8a80b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:45.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.946+0000 7fd8aeb3b640 1 --2- 172.21.15.154:0/2656346698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8a815b3f0 0x7fd8a80b4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:45.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.946+0000 7fd8aeb3b640 1 -- 172.21.15.154:0/2656346698 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a80af4c0 con 0x7fd8a815b850 2023-09-23T03:28:45.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.946+0000 7fd8aeb3b640 1 --2- 172.21.15.154:0/2656346698 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8a815b850 0x7fd8a80b4970 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0001350 tx=0x7fd8a000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:45.952 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.946+0000 7fd8a6ffd640 1 -- 172.21.15.154:0/2656346698 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a000ce50 con 0x7fd8a815b850 2023-09-23T03:28:45.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.946+0000 7fd8a6ffd640 1 -- 172.21.15.154:0/2656346698 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a000a610 con 0x7fd8a815b850 2023-09-23T03:28:45.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.946+0000 7fd8a6ffd640 1 -- 172.21.15.154:0/2656346698 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a000a9c0 con 0x7fd8a815b850 2023-09-23T03:28:45.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.946+0000 7fd8b059e640 1 -- 172.21.15.154:0/2656346698 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a80af7a0 con 0x7fd8a815b850 2023-09-23T03:28:45.953 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.947+0000 7fd8b059e640 1 -- 172.21.15.154:0/2656346698 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8a80c4a90 con 0x7fd8a815b850 2023-09-23T03:28:45.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.948+0000 7fd8b059e640 1 -- 172.21.15.154:0/2656346698 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8a8073eb0 con 0x7fd8a815b850 2023-09-23T03:28:45.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.949+0000 7fd8a6ffd640 1 -- 172.21.15.154:0/2656346698 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fd8a000ab20 con 0x7fd8a815b850 2023-09-23T03:28:45.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.949+0000 7fd8a6ffd640 1 --2- 172.21.15.154:0/2656346698 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd89406af60 0x7fd89406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:45.954 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.950+0000 7fd8a6ffd640 1 -- 172.21.15.154:0/2656346698 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6201+0+0 (secure 0 0 0) 0x7fd8a00890d0 con 0x7fd8a815b850 2023-09-23T03:28:45.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.950+0000 7fd8ae33a640 1 --2- 172.21.15.154:0/2656346698 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd89406af60 0x7fd89406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:45.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.950+0000 7fd8ae33a640 1 --2- 172.21.15.154:0/2656346698 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd89406af60 0x7fd89406d420 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fd898036000 tx=0x7fd898030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:45.955 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:45.954+0000 7fd8a6ffd640 1 -- 172.21.15.154:0/2656346698 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fd8a0053e80 con 0x7fd8a815b850 2023-09-23T03:28:46.144 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.143+0000 7fd8b059e640 1 -- 172.21.15.154:0/2656346698 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd8a815a280 con 0x7fd8a815b850 2023-09-23T03:28:46.145 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.144+0000 7fd8a6ffd640 1 -- 172.21.15.154:0/2656346698 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v128) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd8a001a020 con 0x7fd8a815b850 2023-09-23T03:28:46.145 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 64 2023-09-23T03:28:46.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.147+0000 7fd8b059e640 1 -- 172.21.15.154:0/2656346698 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd89406af60 msgr2=0x7fd89406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:46.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.147+0000 7fd8b059e640 1 --2- 172.21.15.154:0/2656346698 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd89406af60 0x7fd89406d420 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fd898036000 tx=0x7fd898030000 comp rx=0 tx=0).stop 2023-09-23T03:28:46.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.147+0000 7fd8b059e640 1 -- 172.21.15.154:0/2656346698 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8a815b850 msgr2=0x7fd8a80b4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:46.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.147+0000 7fd8b059e640 1 --2- 172.21.15.154:0/2656346698 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8a815b850 0x7fd8a80b4970 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0001350 tx=0x7fd8a000cc00 comp rx=0 tx=0).stop 2023-09-23T03:28:46.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.147+0000 7fd8b059e640 1 -- 172.21.15.154:0/2656346698 shutdown_connections 2023-09-23T03:28:46.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.147+0000 7fd8b059e640 1 --2- 172.21.15.154:0/2656346698 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd89406af60 0x7fd89406d420 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:46.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.147+0000 7fd8b059e640 1 --2- 172.21.15.154:0/2656346698 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8a815b850 0x7fd8a80b4970 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:46.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.147+0000 7fd8b059e640 1 --2- 172.21.15.154:0/2656346698 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8a815b3f0 0x7fd8a80b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:46.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.147+0000 7fd8b059e640 1 --2- 172.21.15.154:0/2656346698 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd890003970 0x7fd8a814fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:46.149 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.147+0000 7fd8b059e640 1 -- 172.21.15.154:0/2656346698 >> 172.21.15.154:0/2656346698 conn(0x7fd8a806ec40 msgr2=0x7fd8a80cae30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:46.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.148+0000 7fd8b059e640 1 -- 172.21.15.154:0/2656346698 shutdown_connections 2023-09-23T03:28:46.150 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.148+0000 7fd8b059e640 1 -- 172.21.15.154:0/2656346698 wait complete. 2023-09-23T03:28:46.167 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:28:46.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.377+0000 7fd52ef65640 1 Processor -- start 2023-09-23T03:28:46.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.378+0000 7fd52ef65640 1 -- start start 2023-09-23T03:28:46.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.379+0000 7fd52ef65640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd5280b4010 0x7fd52806d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:46.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.379+0000 7fd52ef65640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5280b0c10 con 0x7fd528159dd0 2023-09-23T03:28:46.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.379+0000 7fd52ef65640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5280ad760 con 0x7fd5280cdfc0 2023-09-23T03:28:46.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.379+0000 7fd52ef65640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5280ada40 con 0x7fd5280b4010 2023-09-23T03:28:46.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.380+0000 7fd52cd01640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fd528159dd0 0x7fd5280cd8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:34038/0 (socket says 172.21.15.154:34038) 2023-09-23T03:28:46.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.380+0000 7fd52cd01640 1 -- 172.21.15.154:0/3596054110 learned_addr learned my addr 172.21.15.154:0/3596054110 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:28:46.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.380+0000 7fd52d502640 1 --2- 172.21.15.154:0/3596054110 >> v2:172.21.15.190:3300/0 conn(0x7fd5280b4010 0x7fd52806d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:46.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.380+0000 7fd5277fe640 1 -- 172.21.15.154:0/3596054110 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3592191360 0 0) 0x7fd5280b0c10 con 0x7fd528159dd0 2023-09-23T03:28:46.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.380+0000 7fd5277fe640 1 -- 172.21.15.154:0/3596054110 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd510003520 con 0x7fd528159dd0 2023-09-23T03:28:46.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.380+0000 7fd5277fe640 1 -- 172.21.15.154:0/3596054110 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2477770960 0 0) 0x7fd510003520 con 0x7fd528159dd0 2023-09-23T03:28:46.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.380+0000 7fd5277fe640 1 -- 172.21.15.154:0/3596054110 >> v2:172.21.15.190:3300/0 conn(0x7fd5280b4010 msgr2=0x7fd52806d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:46.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.380+0000 7fd5277fe640 1 --2- 172.21.15.154:0/3596054110 >> v2:172.21.15.190:3300/0 conn(0x7fd5280b4010 0x7fd52806d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:46.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.380+0000 7fd5277fe640 1 -- 172.21.15.154:0/3596054110 >> v1:172.21.15.190:6789/0 conn(0x7fd5280cdfc0 legacy=0x7fd5280cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:46.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.380+0000 7fd5277fe640 1 -- 172.21.15.154:0/3596054110 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5280ca470 con 0x7fd528159dd0 2023-09-23T03:28:46.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.380+0000 7fd5277fe640 1 -- 172.21.15.154:0/3596054110 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fd51c001670 con 0x7fd528159dd0 2023-09-23T03:28:46.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.380+0000 7fd5277fe640 1 -- 172.21.15.154:0/3596054110 >> v1:172.21.15.154:6789/0 conn(0x7fd528159dd0 legacy=0x7fd5280cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:46.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.381+0000 7fd5277fe640 1 --2- 172.21.15.154:0/3596054110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd510003970 0x7fd510003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:46.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.381+0000 7fd5277fe640 1 --2- 172.21.15.154:0/3596054110 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5100047c0 0x7fd510004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:46.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.381+0000 7fd5277fe640 1 --2- 172.21.15.154:0/3596054110 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd510005180 0x7fd51000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:46.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.381+0000 7fd5277fe640 1 -- 172.21.15.154:0/3596054110 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5280ada40 con 0x7fd5100047c0 2023-09-23T03:28:46.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.381+0000 7fd5277fe640 1 -- 172.21.15.154:0/3596054110 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51000c120 con 0x7fd510003970 2023-09-23T03:28:46.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.381+0000 7fd5277fe640 1 -- 172.21.15.154:0/3596054110 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd51000c2e0 con 0x7fd510005180 2023-09-23T03:28:46.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.381+0000 7fd527fff640 1 --2- 172.21.15.154:0/3596054110 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5100047c0 0x7fd510004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:46.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.381+0000 7fd52cd01640 1 --2- 172.21.15.154:0/3596054110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd510003970 0x7fd510003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:46.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.381+0000 7fd52d502640 1 --2- 172.21.15.154:0/3596054110 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd510005180 0x7fd51000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:46.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.381+0000 7fd527fff640 1 -- 172.21.15.154:0/3596054110 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd510005180 msgr2=0x7fd51000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:46.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.381+0000 7fd527fff640 1 --2- 172.21.15.154:0/3596054110 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd510005180 0x7fd51000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:46.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.381+0000 7fd527fff640 1 -- 172.21.15.154:0/3596054110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd510003970 msgr2=0x7fd510003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:46.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.381+0000 7fd527fff640 1 --2- 172.21.15.154:0/3596054110 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd510003970 0x7fd510003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:46.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.381+0000 7fd527fff640 1 -- 172.21.15.154:0/3596054110 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd510003520 con 0x7fd5100047c0 2023-09-23T03:28:46.385 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.381+0000 7fd52d502640 1 -- 172.21.15.154:0/3596054110 reap_dead start 2023-09-23T03:28:46.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.383+0000 7fd527fff640 1 --2- 172.21.15.154:0/3596054110 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5100047c0 0x7fd510004c40 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fd518001750 tx=0x7fd5100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac046329f80c0d6 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:46.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.383+0000 7fd5277fe640 1 -- 172.21.15.154:0/3596054110 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5180052d0 con 0x7fd5100047c0 2023-09-23T03:28:46.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.383+0000 7fd5277fe640 1 -- 172.21.15.154:0/3596054110 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd51802e050 con 0x7fd5100047c0 2023-09-23T03:28:46.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.383+0000 7fd5277fe640 1 -- 172.21.15.154:0/3596054110 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd518003830 con 0x7fd5100047c0 2023-09-23T03:28:46.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.383+0000 7fd52ef65640 1 -- 172.21.15.154:0/3596054110 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5100047c0 msgr2=0x7fd510004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:46.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.383+0000 7fd52ef65640 1 --2- 172.21.15.154:0/3596054110 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5100047c0 0x7fd510004c40 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fd518001750 tx=0x7fd5100057f0 comp rx=0 tx=0).stop 2023-09-23T03:28:46.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.384+0000 7fd52ef65640 1 -- 172.21.15.154:0/3596054110 shutdown_connections 2023-09-23T03:28:46.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.384+0000 7fd52ef65640 1 --2- 172.21.15.154:0/3596054110 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5100047c0 0x7fd510004c40 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:46.387 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.384+0000 7fd52ef65640 1 -- 172.21.15.154:0/3596054110 >> 172.21.15.154:0/3596054110 conn(0x7fd5280cb070 msgr2=0x7fd5280cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:46.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.384+0000 7fd52ef65640 1 -- 172.21.15.154:0/3596054110 shutdown_connections 2023-09-23T03:28:46.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.384+0000 7fd52ef65640 1 -- 172.21.15.154:0/3596054110 wait complete. 2023-09-23T03:28:46.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.385+0000 7fd52ef65640 1 Processor -- start 2023-09-23T03:28:46.388 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.386+0000 7fd52ef65640 1 -- start start 2023-09-23T03:28:46.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.387+0000 7fd52ef65640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5100047c0 0x7fd5281e4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:46.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.387+0000 7fd52ef65640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5281f3d40 0x7fd528157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:46.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.387+0000 7fd52ef65640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5281f41c0 0x7fd5281579b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:46.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.387+0000 7fd52ef65640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5281f4ce0 con 0x7fd5281f3d40 2023-09-23T03:28:46.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.387+0000 7fd52ef65640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5281e24b0 con 0x7fd5281f41c0 2023-09-23T03:28:46.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.387+0000 7fd52ef65640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5281f4720 con 0x7fd5100047c0 2023-09-23T03:28:46.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.387+0000 7fd527fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5281f3d40 0x7fd528157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:46.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.387+0000 7fd527fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5281f3d40 0x7fd528157470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57028/0 (socket says 172.21.15.154:57028) 2023-09-23T03:28:46.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.387+0000 7fd527fff640 1 -- 172.21.15.154:0/226913689 learned_addr learned my addr 172.21.15.154:0/226913689 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:46.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.387+0000 7fd52cd01640 1 --2- 172.21.15.154:0/226913689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5100047c0 0x7fd5281e4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:46.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.387+0000 7fd52d502640 1 --2- 172.21.15.154:0/226913689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5281f41c0 0x7fd5281579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:46.390 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.387+0000 7fd527fff640 1 -- 172.21.15.154:0/226913689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5100047c0 msgr2=0x7fd5281e4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:46.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.387+0000 7fd527fff640 1 --2- 172.21.15.154:0/226913689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5100047c0 0x7fd5281e4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:46.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.387+0000 7fd527fff640 1 -- 172.21.15.154:0/226913689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5281f41c0 msgr2=0x7fd5281579b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:46.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.387+0000 7fd527fff640 1 --2- 172.21.15.154:0/226913689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5281f41c0 0x7fd5281579b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:46.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.387+0000 7fd527fff640 1 -- 172.21.15.154:0/226913689 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5281f4a00 con 0x7fd5281f3d40 2023-09-23T03:28:46.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.388+0000 7fd527fff640 1 --2- 172.21.15.154:0/226913689 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5281f3d40 0x7fd528157470 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7fd518005040 tx=0x7fd518001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:46.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.388+0000 7fd5257fa640 1 -- 172.21.15.154:0/226913689 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd518007a30 con 0x7fd5281f3d40 2023-09-23T03:28:46.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.388+0000 7fd5257fa640 1 -- 172.21.15.154:0/226913689 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5180069e0 con 0x7fd5281f3d40 2023-09-23T03:28:46.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.388+0000 7fd52ef65640 1 -- 172.21.15.154:0/226913689 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5280c3500 con 0x7fd5281f3d40 2023-09-23T03:28:46.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.388+0000 7fd5257fa640 1 -- 172.21.15.154:0/226913689 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd518006d70 con 0x7fd5281f3d40 2023-09-23T03:28:46.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.388+0000 7fd52ef65640 1 -- 172.21.15.154:0/226913689 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd52815cd20 con 0x7fd5281f3d40 2023-09-23T03:28:46.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.389+0000 7fd52ef65640 1 -- 172.21.15.154:0/226913689 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd528067870 con 0x7fd5281f3d40 2023-09-23T03:28:46.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.390+0000 7fd5257fa640 1 -- 172.21.15.154:0/226913689 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fd5180035f0 con 0x7fd5281f3d40 2023-09-23T03:28:46.392 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.392+0000 7fd5257fa640 1 --2- 172.21.15.154:0/226913689 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd51406af30 0x7fd51406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:46.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.392+0000 7fd5257fa640 1 -- 172.21.15.154:0/226913689 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6201+0+0 (secure 0 0 0) 0x7fd5180ab9d0 con 0x7fd5281f3d40 2023-09-23T03:28:46.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.392+0000 7fd52cd01640 1 --2- 172.21.15.154:0/226913689 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd51406af30 0x7fd51406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:46.393 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.393+0000 7fd52cd01640 1 --2- 172.21.15.154:0/226913689 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd51406af30 0x7fd51406d3f0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fd51c001c90 tx=0x7fd51c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:46.397 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.396+0000 7fd5257fa640 1 -- 172.21.15.154:0/226913689 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fd518076850 con 0x7fd5281f3d40 2023-09-23T03:28:46.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.609+0000 7fd52ef65640 1 -- 172.21.15.154:0/226913689 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5280c3360 con 0x7fd5281f3d40 2023-09-23T03:28:46.612 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.611+0000 7fd5257fa640 1 -- 172.21.15.154:0/226913689 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+4935 (secure 0 0 0) 0x7fd5280c3360 con 0x7fd5281f3d40 2023-09-23T03:28:46.612 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:28:46.613 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":121,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/2759187683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2759187683},{"type":"v1","addr":"172.21.15.154:6835","nonce":2759187683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":5924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3712825179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3712825179},{"type":"v1","addr":"172.21.15.190:6835","nonce":3712825179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":110},{"gid":6544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/4147378242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":4147378242},{"type":"v1","addr":"172.21.15.190:6834","nonce":4147378242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:27:44.470312+0000","modified":"2023-09-23T03:28:35.478753+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9306":{"gid":9306,"name":"c","rank":0,"incarnation":118,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6837/1250699891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1250699891},{"type":"v1","addr":"172.21.15.154:6837","nonce":1250699891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-09-23T03:28:46.613 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 121 2023-09-23T03:28:46.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.613+0000 7fd52ef65640 1 -- 172.21.15.154:0/226913689 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd51406af30 msgr2=0x7fd51406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:46.614 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.613+0000 7fd52ef65640 1 --2- 172.21.15.154:0/226913689 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd51406af30 0x7fd51406d3f0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fd51c001c90 tx=0x7fd51c009040 comp rx=0 tx=0).stop 2023-09-23T03:28:46.615 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.613+0000 7fd52ef65640 1 -- 172.21.15.154:0/226913689 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5281f3d40 msgr2=0x7fd528157470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:46.615 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.613+0000 7fd52ef65640 1 --2- 172.21.15.154:0/226913689 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5281f3d40 0x7fd528157470 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7fd518005040 tx=0x7fd518001120 comp rx=0 tx=0).stop 2023-09-23T03:28:46.615 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.613+0000 7fd52ef65640 1 -- 172.21.15.154:0/226913689 shutdown_connections 2023-09-23T03:28:46.615 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.613+0000 7fd52ef65640 1 --2- 172.21.15.154:0/226913689 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd51406af30 0x7fd51406d3f0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:46.615 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.613+0000 7fd52ef65640 1 --2- 172.21.15.154:0/226913689 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd5281f41c0 0x7fd5281579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:46.615 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.613+0000 7fd52ef65640 1 --2- 172.21.15.154:0/226913689 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5281f3d40 0x7fd528157470 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:46.615 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.613+0000 7fd52ef65640 1 --2- 172.21.15.154:0/226913689 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd5100047c0 0x7fd5281e4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:46.615 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.613+0000 7fd52ef65640 1 -- 172.21.15.154:0/226913689 >> 172.21.15.154:0/226913689 conn(0x7fd5280cb070 msgr2=0x7fd5280ccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:46.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.614+0000 7fd52ef65640 1 -- 172.21.15.154:0/226913689 shutdown_connections 2023-09-23T03:28:46.616 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.614+0000 7fd52ef65640 1 -- 172.21.15.154:0/226913689 wait complete. 2023-09-23T03:28:46.632 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:28:46.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.831+0000 7f35d5d1d640 1 Processor -- start 2023-09-23T03:28:46.833 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.832+0000 7f35d5d1d640 1 -- start start 2023-09-23T03:28:46.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.833+0000 7f35d5d1d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f35d00b0780 0x7f35d00c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:46.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.833+0000 7f35d5d1d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35d0070b40 con 0x7f35d00ad9c0 2023-09-23T03:28:46.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.833+0000 7f35d5d1d640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35d0070e20 con 0x7f35d015ad70 2023-09-23T03:28:46.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.833+0000 7f35d5d1d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35d0071100 con 0x7f35d00b0780 2023-09-23T03:28:46.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.834+0000 7f35cffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f35d00b0780 0x7f35d00c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:46.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.834+0000 7f35cffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f35d00b0780 0x7f35d00c8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57038/0 (socket says 172.21.15.154:57038) 2023-09-23T03:28:46.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.834+0000 7f35cffff640 1 -- 172.21.15.154:0/3718907919 learned_addr learned my addr 172.21.15.154:0/3718907919 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:46.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.834+0000 7f35c7fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f35d00ad9c0 0x7f35d006c8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:34054/0 (socket says 172.21.15.154:34054) 2023-09-23T03:28:46.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.834+0000 7f35cffff640 1 -- 172.21.15.154:0/3718907919 >> v1:172.21.15.190:6789/0 conn(0x7f35d015ad70 legacy=0x7f35d00b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:46.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.834+0000 7f35cffff640 1 -- 172.21.15.154:0/3718907919 >> v1:172.21.15.154:6789/0 conn(0x7f35d00ad9c0 legacy=0x7f35d006c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:46.836 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.834+0000 7f35cffff640 1 -- 172.21.15.154:0/3718907919 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d00c96d0 con 0x7f35d00b0780 2023-09-23T03:28:46.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.835+0000 7f35cffff640 1 --2- 172.21.15.154:0/3718907919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35d00b0780 0x7f35d00c8e50 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f35c8009e80 tx=0x7f35c802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=490cf1803d53a087 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:46.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.835+0000 7f35ceffd640 1 -- 172.21.15.154:0/3718907919 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c802caf0 con 0x7f35d00b0780 2023-09-23T03:28:46.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.835+0000 7f35ceffd640 1 -- 172.21.15.154:0/3718907919 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c802cc50 con 0x7f35d00b0780 2023-09-23T03:28:46.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.835+0000 7f35ceffd640 1 -- 172.21.15.154:0/3718907919 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c80343f0 con 0x7f35d00b0780 2023-09-23T03:28:46.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.836+0000 7f35d5d1d640 1 -- 172.21.15.154:0/3718907919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35d00b0780 msgr2=0x7f35d00c8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:46.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.836+0000 7f35d5d1d640 1 --2- 172.21.15.154:0/3718907919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35d00b0780 0x7f35d00c8e50 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f35c8009e80 tx=0x7f35c802bf50 comp rx=0 tx=0).stop 2023-09-23T03:28:46.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.836+0000 7f35d5d1d640 1 -- 172.21.15.154:0/3718907919 shutdown_connections 2023-09-23T03:28:46.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.836+0000 7f35d5d1d640 1 --2- 172.21.15.154:0/3718907919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35d00b0780 0x7f35d00c8e50 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:46.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.836+0000 7f35d5d1d640 1 -- 172.21.15.154:0/3718907919 >> 172.21.15.154:0/3718907919 conn(0x7f35d00cb070 msgr2=0x7f35d00cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:46.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.836+0000 7f35d5d1d640 1 -- 172.21.15.154:0/3718907919 shutdown_connections 2023-09-23T03:28:46.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.836+0000 7f35d5d1d640 1 -- 172.21.15.154:0/3718907919 wait complete. 2023-09-23T03:28:46.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.838+0000 7f35d5d1d640 1 Processor -- start 2023-09-23T03:28:46.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.838+0000 7f35d5d1d640 1 -- start start 2023-09-23T03:28:46.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.839+0000 7f35d5d1d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35d00ad9c0 0x7f35d0154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:46.840 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.839+0000 7f35d5d1d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35d00b0780 0x7f35d0153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:46.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.839+0000 7f35d5d1d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35d015ad70 0x7f35d00c6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:46.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.839+0000 7f35d5d1d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d01544c0 con 0x7f35d00ad9c0 2023-09-23T03:28:46.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.839+0000 7f35d5d1d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d00addc0 con 0x7f35d015ad70 2023-09-23T03:28:46.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.839+0000 7f35d5d1d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d00ae0a0 con 0x7f35d00b0780 2023-09-23T03:28:46.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.839+0000 7f35cf7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35d00ad9c0 0x7f35d0154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:46.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.839+0000 7f35cf7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35d00ad9c0 0x7f35d0154ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57042/0 (socket says 172.21.15.154:57042) 2023-09-23T03:28:46.841 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.839+0000 7f35cf7fe640 1 -- 172.21.15.154:0/129747260 learned_addr learned my addr 172.21.15.154:0/129747260 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:46.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.840+0000 7f35cf7fe640 1 -- 172.21.15.154:0/129747260 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35d00b0780 msgr2=0x7f35d0153f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:46.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.840+0000 7f35cf7fe640 1 --2- 172.21.15.154:0/129747260 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35d00b0780 0x7f35d0153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:46.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.840+0000 7f35cf7fe640 1 -- 172.21.15.154:0/129747260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35d015ad70 msgr2=0x7f35d00c6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:46.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.840+0000 7f35cf7fe640 1 --2- 172.21.15.154:0/129747260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35d015ad70 0x7f35d00c6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:46.842 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.840+0000 7f35cf7fe640 1 -- 172.21.15.154:0/129747260 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c8009b30 con 0x7f35d00ad9c0 2023-09-23T03:28:46.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.840+0000 7f35cf7fe640 1 --2- 172.21.15.154:0/129747260 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35d00ad9c0 0x7f35d0154ba0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f35c000b3f0 tx=0x7f35c000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:46.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.840+0000 7f35ccff9640 1 -- 172.21.15.154:0/129747260 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c0011830 con 0x7f35d00ad9c0 2023-09-23T03:28:46.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.840+0000 7f35ccff9640 1 -- 172.21.15.154:0/129747260 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c0011990 con 0x7f35d00ad9c0 2023-09-23T03:28:46.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.840+0000 7f35d5d1d640 1 -- 172.21.15.154:0/129747260 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d00ae380 con 0x7f35d00ad9c0 2023-09-23T03:28:46.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.840+0000 7f35ccff9640 1 -- 172.21.15.154:0/129747260 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c0011d40 con 0x7f35d00ad9c0 2023-09-23T03:28:46.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.840+0000 7f35d5d1d640 1 -- 172.21.15.154:0/129747260 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d00ae940 con 0x7f35d00ad9c0 2023-09-23T03:28:46.843 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.841+0000 7f35d5d1d640 1 -- 172.21.15.154:0/129747260 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35d0067870 con 0x7f35d00ad9c0 2023-09-23T03:28:46.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.842+0000 7f35ccff9640 1 -- 172.21.15.154:0/129747260 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f35c001a400 con 0x7f35d00ad9c0 2023-09-23T03:28:46.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.843+0000 7f35ccff9640 1 --2- 172.21.15.154:0/129747260 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f35b006af30 0x7f35b006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:46.844 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.843+0000 7f35ccff9640 1 -- 172.21.15.154:0/129747260 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f35c0089790 con 0x7f35d00ad9c0 2023-09-23T03:28:46.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.844+0000 7f35c7fff640 1 --2- 172.21.15.154:0/129747260 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f35b006af30 0x7f35b006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:46.845 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.844+0000 7f35c7fff640 1 --2- 172.21.15.154:0/129747260 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f35b006af30 0x7f35b006d3f0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f35b8000f60 tx=0x7f35b8007a00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:46.849 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:46.848+0000 7f35ccff9640 1 -- 172.21.15.154:0/129747260 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f35c00545d0 con 0x7f35d00ad9c0 2023-09-23T03:28:47.038 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.036+0000 7f35d5d1d640 1 -- 172.21.15.154:0/129747260 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35d0073eb0 con 0x7f35d00ad9c0 2023-09-23T03:28:47.040 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.039+0000 7f35ccff9640 1 -- 172.21.15.154:0/129747260 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+17083 (secure 0 0 0) 0x7f35c0057bf0 con 0x7f35d00ad9c0 2023-09-23T03:28:47.041 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:28:47.041 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":128,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:28:29.295164+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:27:42.207617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-09-23T03:27:43.473867+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6837/4006549651":"2023-09-24T03:28:29.295137+0000","172.21.15.154:6836/4006549651":"2023-09-24T03:28:29.295137+0000","172.21.15.190:6835/3558096060":"2023-09-24T03:28:08.468628+0000","172.21.15.190:6833/3558096060":"2023-09-24T03:28:08.468628+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:28:47.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.042+0000 7f35d5d1d640 1 -- 172.21.15.154:0/129747260 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f35b006af30 msgr2=0x7f35b006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:47.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.042+0000 7f35d5d1d640 1 --2- 172.21.15.154:0/129747260 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f35b006af30 0x7f35b006d3f0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f35b8000f60 tx=0x7f35b8007a00 comp rx=0 tx=0).stop 2023-09-23T03:28:47.043 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.042+0000 7f35d5d1d640 1 -- 172.21.15.154:0/129747260 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35d00ad9c0 msgr2=0x7f35d0154ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:47.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.042+0000 7f35d5d1d640 1 --2- 172.21.15.154:0/129747260 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35d00ad9c0 0x7f35d0154ba0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f35c000b3f0 tx=0x7f35c000b8b0 comp rx=0 tx=0).stop 2023-09-23T03:28:47.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.042+0000 7f35d5d1d640 1 -- 172.21.15.154:0/129747260 shutdown_connections 2023-09-23T03:28:47.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.042+0000 7f35d5d1d640 1 --2- 172.21.15.154:0/129747260 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f35b006af30 0x7f35b006d3f0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:47.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.042+0000 7f35d5d1d640 1 --2- 172.21.15.154:0/129747260 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f35d015ad70 0x7f35d00c6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:47.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.042+0000 7f35d5d1d640 1 --2- 172.21.15.154:0/129747260 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f35d00b0780 0x7f35d0153f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:47.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.042+0000 7f35d5d1d640 1 --2- 172.21.15.154:0/129747260 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35d00ad9c0 0x7f35d0154ba0 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:47.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.042+0000 7f35d5d1d640 1 -- 172.21.15.154:0/129747260 >> 172.21.15.154:0/129747260 conn(0x7f35d00cb070 msgr2=0x7f35d00aca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:47.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.042+0000 7f35d5d1d640 1 -- 172.21.15.154:0/129747260 shutdown_connections 2023-09-23T03:28:47.044 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.042+0000 7f35d5d1d640 1 -- 172.21.15.154:0/129747260 wait complete. 2023-09-23T03:28:47.061 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-09-23T03:28:47.061 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:28:47.272 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.271+0000 7f85ee634640 1 Processor -- start 2023-09-23T03:28:47.273 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.271+0000 7f85ee634640 1 -- start start 2023-09-23T03:28:47.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.273+0000 7f85ee634640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f85e8160280 0x7f85e806d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:47.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.273+0000 7f85ee634640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85e8160970 0x7f85e80b0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:47.275 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.273+0000 7f85ee634640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f85e8160df0 0x7f85e80b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:47.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.273+0000 7f85ee634640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85e806c810 con 0x7f85e8160df0 2023-09-23T03:28:47.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.273+0000 7f85ee634640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85e81613c0 con 0x7f85e8160280 2023-09-23T03:28:47.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.273+0000 7f85ee634640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e81616a0 con 0x7f85e8160970 2023-09-23T03:28:47.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.273+0000 7f85ecbd1640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f85e8160df0 0x7f85e80b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:47.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.273+0000 7f85ecbd1640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f85e8160df0 0x7f85e80b05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57050/0 (socket says 172.21.15.154:57050) 2023-09-23T03:28:47.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.273+0000 7f85ecbd1640 1 -- 172.21.15.154:0/3364872744 learned_addr learned my addr 172.21.15.154:0/3364872744 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:47.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.274+0000 7f85e77fe640 1 --2- 172.21.15.154:0/3364872744 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85e8160970 0x7f85e80b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:47.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.274+0000 7f85e7fff640 1 --2- 172.21.15.154:0/3364872744 >> v2:172.21.15.190:3300/0 conn(0x7f85e8160280 0x7f85e806d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:47.276 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.274+0000 7f85ecbd1640 1 -- 172.21.15.154:0/3364872744 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85e8160970 msgr2=0x7f85e80b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:47.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.274+0000 7f85ecbd1640 1 --2- 172.21.15.154:0/3364872744 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85e8160970 0x7f85e80b0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:47.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.274+0000 7f85ecbd1640 1 -- 172.21.15.154:0/3364872744 >> v2:172.21.15.190:3300/0 conn(0x7f85e8160280 msgr2=0x7f85e806d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:47.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.274+0000 7f85ecbd1640 1 --2- 172.21.15.154:0/3364872744 >> v2:172.21.15.190:3300/0 conn(0x7f85e8160280 0x7f85e806d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:47.277 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.274+0000 7f85ecbd1640 1 -- 172.21.15.154:0/3364872744 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e806ca20 con 0x7f85e8160df0 2023-09-23T03:28:47.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.275+0000 7f85ecbd1640 1 --2- 172.21.15.154:0/3364872744 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85e8160df0 0x7f85e80b05b0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f85e80cbfc0 tx=0x7f85e002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=237db7af97568eef server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:47.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.276+0000 7f85e6ffd640 1 -- 172.21.15.154:0/3364872744 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e002ca80 con 0x7f85e8160df0 2023-09-23T03:28:47.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.276+0000 7f85e6ffd640 1 -- 172.21.15.154:0/3364872744 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e002cbe0 con 0x7f85e8160df0 2023-09-23T03:28:47.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.276+0000 7f85e6ffd640 1 -- 172.21.15.154:0/3364872744 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e00343f0 con 0x7f85e8160df0 2023-09-23T03:28:47.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.276+0000 7f85ee634640 1 -- 172.21.15.154:0/3364872744 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85e8160df0 msgr2=0x7f85e80b05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:47.278 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.276+0000 7f85ee634640 1 --2- 172.21.15.154:0/3364872744 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85e8160df0 0x7f85e80b05b0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f85e80cbfc0 tx=0x7f85e002bee0 comp rx=0 tx=0).stop 2023-09-23T03:28:47.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.276+0000 7f85ee634640 1 -- 172.21.15.154:0/3364872744 shutdown_connections 2023-09-23T03:28:47.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.276+0000 7f85ee634640 1 --2- 172.21.15.154:0/3364872744 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85e8160df0 0x7f85e80b05b0 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:47.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.276+0000 7f85ee634640 1 --2- 172.21.15.154:0/3364872744 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85e8160970 0x7f85e80b0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:47.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.276+0000 7f85ee634640 1 --2- 172.21.15.154:0/3364872744 >> v2:172.21.15.190:3300/0 conn(0x7f85e8160280 0x7f85e806d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:47.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.276+0000 7f85ee634640 1 -- 172.21.15.154:0/3364872744 >> 172.21.15.154:0/3364872744 conn(0x7f85e80cb070 msgr2=0x7f85e80cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:47.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.276+0000 7f85ee634640 1 -- 172.21.15.154:0/3364872744 shutdown_connections 2023-09-23T03:28:47.279 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.277+0000 7f85ee634640 1 -- 172.21.15.154:0/3364872744 wait complete. 2023-09-23T03:28:47.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.279+0000 7f85ee634640 1 Processor -- start 2023-09-23T03:28:47.280 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.279+0000 7f85ee634640 1 -- start start 2023-09-23T03:28:47.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.280+0000 7f85ee634640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85e8160280 0x7f85e80b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:47.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.280+0000 7f85ee634640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85e8160970 0x7f85e8158a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:47.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.280+0000 7f85ee634640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85e8160df0 0x7f85e80c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:47.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.280+0000 7f85ee634640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e80b1c20 con 0x7f85e8160280 2023-09-23T03:28:47.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.280+0000 7f85ee634640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e80b1f00 con 0x7f85e8160df0 2023-09-23T03:28:47.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.280+0000 7f85ee634640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e80b21e0 con 0x7f85e8160970 2023-09-23T03:28:47.283 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.280+0000 7f85e7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85e8160280 0x7f85e80b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:47.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.280+0000 7f85e7fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85e8160280 0x7f85e80b2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57058/0 (socket says 172.21.15.154:57058) 2023-09-23T03:28:47.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.280+0000 7f85e7fff640 1 -- 172.21.15.154:0/3307359982 learned_addr learned my addr 172.21.15.154:0/3307359982 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:47.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.281+0000 7f85e77fe640 1 --2- 172.21.15.154:0/3307359982 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85e8160970 0x7f85e8158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:47.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.281+0000 7f85e7fff640 1 -- 172.21.15.154:0/3307359982 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85e8160970 msgr2=0x7f85e8158a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:47.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.281+0000 7f85e7fff640 1 --2- 172.21.15.154:0/3307359982 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85e8160970 0x7f85e8158a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:47.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.281+0000 7f85e7fff640 1 -- 172.21.15.154:0/3307359982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85e8160df0 msgr2=0x7f85e80c77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:47.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.281+0000 7f85ecbd1640 1 --2- 172.21.15.154:0/3307359982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85e8160df0 0x7f85e80c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:47.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.281+0000 7f85e7fff640 1 --2- 172.21.15.154:0/3307359982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85e8160df0 0x7f85e80c77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:47.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.281+0000 7f85e7fff640 1 -- 172.21.15.154:0/3307359982 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e0009b30 con 0x7f85e8160280 2023-09-23T03:28:47.284 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.281+0000 7f85e77fe640 1 --2- 172.21.15.154:0/3307359982 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85e8160970 0x7f85e8158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:47.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.281+0000 7f85e7fff640 1 --2- 172.21.15.154:0/3307359982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85e8160280 0x7f85e80b2fa0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f85d800d800 tx=0x7f85d800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:47.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.281+0000 7f85e4ff9640 1 -- 172.21.15.154:0/3307359982 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d80096a0 con 0x7f85e8160280 2023-09-23T03:28:47.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.282+0000 7f85e4ff9640 1 -- 172.21.15.154:0/3307359982 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d8009800 con 0x7f85e8160280 2023-09-23T03:28:47.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.282+0000 7f85e4ff9640 1 -- 172.21.15.154:0/3307359982 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d8009b90 con 0x7f85e8160280 2023-09-23T03:28:47.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.282+0000 7f85ee634640 1 -- 172.21.15.154:0/3307359982 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85e80b24c0 con 0x7f85e8160280 2023-09-23T03:28:47.285 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.282+0000 7f85ee634640 1 -- 172.21.15.154:0/3307359982 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85e80b2a80 con 0x7f85e8160280 2023-09-23T03:28:47.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.283+0000 7f85ee634640 1 -- 172.21.15.154:0/3307359982 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85e8067870 con 0x7f85e8160280 2023-09-23T03:28:47.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.284+0000 7f85e4ff9640 1 -- 172.21.15.154:0/3307359982 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f85d8009cf0 con 0x7f85e8160280 2023-09-23T03:28:47.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.285+0000 7f85e4ff9640 1 --2- 172.21.15.154:0/3307359982 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f85d406af60 0x7f85d406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:47.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.285+0000 7f85e4ff9640 1 -- 172.21.15.154:0/3307359982 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f85d8089d20 con 0x7f85e8160280 2023-09-23T03:28:47.286 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.285+0000 7f85e77fe640 1 --2- 172.21.15.154:0/3307359982 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f85d406af60 0x7f85d406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:47.287 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.286+0000 7f85e77fe640 1 --2- 172.21.15.154:0/3307359982 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f85d406af60 0x7f85d406d420 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f85e80b34e0 tx=0x7f85dc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:47.290 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.289+0000 7f85e4ff9640 1 -- 172.21.15.154:0/3307359982 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f85d8054b70 con 0x7f85e8160280 2023-09-23T03:28:47.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.516+0000 7f85ee634640 1 -- 172.21.15.154:0/3307359982 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85e8073eb0 con 0x7f85e8160280 2023-09-23T03:28:47.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.517+0000 7f85e4ff9640 1 -- 172.21.15.154:0/3307359982 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+4935 (secure 0 0 0) 0x7f85d800b440 con 0x7f85e8160280 2023-09-23T03:28:47.519 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:28:47.519 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":121,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/2759187683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2759187683},{"type":"v1","addr":"172.21.15.154:6835","nonce":2759187683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":5924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3712825179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3712825179},{"type":"v1","addr":"172.21.15.190:6835","nonce":3712825179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":110},{"gid":6544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/4147378242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":4147378242},{"type":"v1","addr":"172.21.15.190:6834","nonce":4147378242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:27:44.470312+0000","modified":"2023-09-23T03:28:35.478753+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9306":{"gid":9306,"name":"c","rank":0,"incarnation":118,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6837/1250699891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1250699891},{"type":"v1","addr":"172.21.15.154:6837","nonce":1250699891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-09-23T03:28:47.519 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 121 2023-09-23T03:28:47.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.520+0000 7f85ee634640 1 -- 172.21.15.154:0/3307359982 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f85d406af60 msgr2=0x7f85d406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:47.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.520+0000 7f85ee634640 1 --2- 172.21.15.154:0/3307359982 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f85d406af60 0x7f85d406d420 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f85e80b34e0 tx=0x7f85dc008040 comp rx=0 tx=0).stop 2023-09-23T03:28:47.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.520+0000 7f85ee634640 1 -- 172.21.15.154:0/3307359982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85e8160280 msgr2=0x7f85e80b2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:47.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.520+0000 7f85ee634640 1 --2- 172.21.15.154:0/3307359982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85e8160280 0x7f85e80b2fa0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f85d800d800 tx=0x7f85d800dcc0 comp rx=0 tx=0).stop 2023-09-23T03:28:47.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.520+0000 7f85ee634640 1 -- 172.21.15.154:0/3307359982 shutdown_connections 2023-09-23T03:28:47.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.520+0000 7f85ee634640 1 --2- 172.21.15.154:0/3307359982 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f85d406af60 0x7f85d406d420 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:47.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.520+0000 7f85ee634640 1 --2- 172.21.15.154:0/3307359982 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85e8160df0 0x7f85e80c77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:47.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.520+0000 7f85ee634640 1 --2- 172.21.15.154:0/3307359982 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85e8160970 0x7f85e8158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:47.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.520+0000 7f85ee634640 1 --2- 172.21.15.154:0/3307359982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85e8160280 0x7f85e80b2fa0 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:47.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.520+0000 7f85ee634640 1 -- 172.21.15.154:0/3307359982 >> 172.21.15.154:0/3307359982 conn(0x7f85e80cb070 msgr2=0x7f85e80ca2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:47.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.520+0000 7f85ee634640 1 -- 172.21.15.154:0/3307359982 shutdown_connections 2023-09-23T03:28:47.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.520+0000 7f85ee634640 1 -- 172.21.15.154:0/3307359982 wait complete. 2023-09-23T03:28:47.541 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:28:47.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.749+0000 7f4a3bfff640 1 Processor -- start 2023-09-23T03:28:47.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.749+0000 7f4a3bfff640 1 -- start start 2023-09-23T03:28:47.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.750+0000 7f4a3bfff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a3c0ce230 0x7f4a3c0b5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:47.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.750+0000 7f4a3bfff640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a3c0b3d20 con 0x7f4a3c0d10e0 2023-09-23T03:28:47.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.750+0000 7f4a3bfff640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a3c0b3a40 con 0x7f4a3c1600b0 2023-09-23T03:28:47.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.750+0000 7f4a3bfff640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c0ce950 con 0x7f4a3c0ce230 2023-09-23T03:28:47.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.751+0000 7f4a3b7fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f4a3c0d10e0 0x7f4a3c0b5d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:34066/0 (socket says 172.21.15.154:34066) 2023-09-23T03:28:47.752 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.751+0000 7f4a3b7fe640 1 -- 172.21.15.154:0/3127530998 learned_addr learned my addr 172.21.15.154:0/3127530998 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:28:47.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.751+0000 7f4a3a7fc640 1 --2- 172.21.15.154:0/3127530998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a3c0ce230 0x7f4a3c0b5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:47.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.751+0000 7f4a39ffb640 1 -- 172.21.15.154:0/3127530998 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 436100523 0 0) 0x7f4a3c0b3d20 con 0x7f4a3c0d10e0 2023-09-23T03:28:47.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.751+0000 7f4a39ffb640 1 -- 172.21.15.154:0/3127530998 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a24003520 con 0x7f4a3c0d10e0 2023-09-23T03:28:47.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.752+0000 7f4a39ffb640 1 -- 172.21.15.154:0/3127530998 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2900712525 0 0) 0x7f4a24003520 con 0x7f4a3c0d10e0 2023-09-23T03:28:47.754 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.752+0000 7f4a39ffb640 1 -- 172.21.15.154:0/3127530998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a3c0ce230 msgr2=0x7f4a3c0b5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:47.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.752+0000 7f4a39ffb640 1 --2- 172.21.15.154:0/3127530998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a3c0ce230 0x7f4a3c0b5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:47.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.752+0000 7f4a39ffb640 1 -- 172.21.15.154:0/3127530998 >> v1:172.21.15.190:6789/0 conn(0x7f4a3c1600b0 legacy=0x7f4a3c0d09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:47.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.752+0000 7f4a39ffb640 1 -- 172.21.15.154:0/3127530998 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a3c0cec30 con 0x7f4a3c0d10e0 2023-09-23T03:28:47.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.752+0000 7f4a39ffb640 1 -- 172.21.15.154:0/3127530998 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f4a340019f0 con 0x7f4a3c0d10e0 2023-09-23T03:28:47.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.752+0000 7f4a39ffb640 1 -- 172.21.15.154:0/3127530998 >> v1:172.21.15.154:6789/0 conn(0x7f4a3c0d10e0 legacy=0x7f4a3c0b5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:47.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.752+0000 7f4a39ffb640 1 --2- 172.21.15.154:0/3127530998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a240039a0 0x7f4a24003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:47.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.752+0000 7f4a39ffb640 1 --2- 172.21.15.154:0/3127530998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a240047c0 0x7f4a24004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:47.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.752+0000 7f4a39ffb640 1 --2- 172.21.15.154:0/3127530998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a3c0ce230 0x7f4a240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:47.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.752+0000 7f4a39ffb640 1 -- 172.21.15.154:0/3127530998 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c0ce950 con 0x7f4a240047c0 2023-09-23T03:28:47.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.752+0000 7f4a39ffb640 1 -- 172.21.15.154:0/3127530998 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2400bab0 con 0x7f4a240039a0 2023-09-23T03:28:47.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.752+0000 7f4a39ffb640 1 -- 172.21.15.154:0/3127530998 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2400bc70 con 0x7f4a3c0ce230 2023-09-23T03:28:47.755 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.752+0000 7f4a3a7fc640 1 --2- 172.21.15.154:0/3127530998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a240047c0 0x7f4a24004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:47.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.752+0000 7f4a3affd640 1 --2- 172.21.15.154:0/3127530998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a240039a0 0x7f4a24003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:47.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.752+0000 7f4a3b7fe640 1 --2- 172.21.15.154:0/3127530998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a3c0ce230 0x7f4a240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:47.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.752+0000 7f4a3a7fc640 1 -- 172.21.15.154:0/3127530998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a3c0ce230 msgr2=0x7f4a240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:47.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.752+0000 7f4a3a7fc640 1 --2- 172.21.15.154:0/3127530998 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a3c0ce230 0x7f4a240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:47.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.752+0000 7f4a3a7fc640 1 -- 172.21.15.154:0/3127530998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a240039a0 msgr2=0x7f4a24003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:47.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.752+0000 7f4a3a7fc640 1 --2- 172.21.15.154:0/3127530998 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a240039a0 0x7f4a24003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:47.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.753+0000 7f4a3a7fc640 1 -- 172.21.15.154:0/3127530998 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a24003520 con 0x7f4a240047c0 2023-09-23T03:28:47.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.754+0000 7f4a3a7fc640 1 --2- 172.21.15.154:0/3127530998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a240047c0 0x7f4a24004c40 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f4a30009ac0 tx=0x7f4a240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=558686bf0fd80e89 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:47.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.754+0000 7f4a39ffb640 1 -- 172.21.15.154:0/3127530998 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3002c6b0 con 0x7f4a240047c0 2023-09-23T03:28:47.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.754+0000 7f4a39ffb640 1 -- 172.21.15.154:0/3127530998 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3002c810 con 0x7f4a240047c0 2023-09-23T03:28:47.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.754+0000 7f4a39ffb640 1 -- 172.21.15.154:0/3127530998 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3002cba0 con 0x7f4a240047c0 2023-09-23T03:28:47.757 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.754+0000 7f4a3bfff640 1 -- 172.21.15.154:0/3127530998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a240047c0 msgr2=0x7f4a24004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:47.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.754+0000 7f4a3bfff640 1 --2- 172.21.15.154:0/3127530998 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a240047c0 0x7f4a24004c40 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f4a30009ac0 tx=0x7f4a240057f0 comp rx=0 tx=0).stop 2023-09-23T03:28:47.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.755+0000 7f4a3b7fe640 1 -- 172.21.15.154:0/3127530998 reap_dead start 2023-09-23T03:28:47.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.755+0000 7f4a3bfff640 1 -- 172.21.15.154:0/3127530998 shutdown_connections 2023-09-23T03:28:47.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.755+0000 7f4a3bfff640 1 -- 172.21.15.154:0/3127530998 >> 172.21.15.154:0/3127530998 conn(0x7f4a3c09bb30 msgr2=0x7f4a3c0d27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:47.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.755+0000 7f4a3bfff640 1 -- 172.21.15.154:0/3127530998 shutdown_connections 2023-09-23T03:28:47.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.755+0000 7f4a3bfff640 1 -- 172.21.15.154:0/3127530998 wait complete. 2023-09-23T03:28:47.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.756+0000 7f4a3bfff640 1 Processor -- start 2023-09-23T03:28:47.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.757+0000 7f4a3bfff640 1 -- start start 2023-09-23T03:28:47.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.757+0000 7f4a3bfff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a240047c0 0x7f4a3c154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:47.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.757+0000 7f4a3bfff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a3c0cbdf0 0x7f4a3c0cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:47.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.757+0000 7f4a3bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a3c0b57e0 0x7f4a3c0b5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:47.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.757+0000 7f4a3bfff640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c0b61a0 con 0x7f4a3c0b57e0 2023-09-23T03:28:47.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.757+0000 7f4a3bfff640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c0cc900 con 0x7f4a3c0cbdf0 2023-09-23T03:28:47.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.757+0000 7f4a3bfff640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c1f2440 con 0x7f4a240047c0 2023-09-23T03:28:47.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.758+0000 7f4a3b7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a3c0b57e0 0x7f4a3c0b5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:47.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.758+0000 7f4a3b7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a3c0b57e0 0x7f4a3c0b5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57076/0 (socket says 172.21.15.154:57076) 2023-09-23T03:28:47.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.758+0000 7f4a3b7fe640 1 -- 172.21.15.154:0/2473746238 learned_addr learned my addr 172.21.15.154:0/2473746238 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:47.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.758+0000 7f4a3a7fc640 1 --2- 172.21.15.154:0/2473746238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a3c0cbdf0 0x7f4a3c0cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:47.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.758+0000 7f4a3affd640 1 --2- 172.21.15.154:0/2473746238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a240047c0 0x7f4a3c154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:47.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.758+0000 7f4a3b7fe640 1 -- 172.21.15.154:0/2473746238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a240047c0 msgr2=0x7f4a3c154de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:47.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.758+0000 7f4a3b7fe640 1 --2- 172.21.15.154:0/2473746238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a240047c0 0x7f4a3c154de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:47.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.758+0000 7f4a3b7fe640 1 -- 172.21.15.154:0/2473746238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a3c0cbdf0 msgr2=0x7f4a3c0cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:47.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.758+0000 7f4a3b7fe640 1 --2- 172.21.15.154:0/2473746238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a3c0cbdf0 0x7f4a3c0cc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:47.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.758+0000 7f4a3b7fe640 1 -- 172.21.15.154:0/2473746238 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a3c1f2720 con 0x7f4a3c0b57e0 2023-09-23T03:28:47.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.758+0000 7f4a3affd640 1 --2- 172.21.15.154:0/2473746238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a240047c0 0x7f4a3c154de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:47.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.758+0000 7f4a3b7fe640 1 --2- 172.21.15.154:0/2473746238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a3c0b57e0 0x7f4a3c0b5c60 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c08a550 tx=0x7f4a34007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:47.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.759+0000 7f4a23fff640 1 -- 172.21.15.154:0/2473746238 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3400eb00 con 0x7f4a3c0b57e0 2023-09-23T03:28:47.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.759+0000 7f4a23fff640 1 -- 172.21.15.154:0/2473746238 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3400ec60 con 0x7f4a3c0b57e0 2023-09-23T03:28:47.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.759+0000 7f4a23fff640 1 -- 172.21.15.154:0/2473746238 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a34013420 con 0x7f4a3c0b57e0 2023-09-23T03:28:47.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.759+0000 7f4a3bfff640 1 -- 172.21.15.154:0/2473746238 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a3c1f2a00 con 0x7f4a3c0b57e0 2023-09-23T03:28:47.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.759+0000 7f4a3bfff640 1 -- 172.21.15.154:0/2473746238 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a3c1f2130 con 0x7f4a3c0b57e0 2023-09-23T03:28:47.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.760+0000 7f4a3bfff640 1 -- 172.21.15.154:0/2473746238 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a3c1f2bf0 con 0x7f4a3c0b57e0 2023-09-23T03:28:47.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.761+0000 7f4a23fff640 1 -- 172.21.15.154:0/2473746238 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f4a34013580 con 0x7f4a3c0b57e0 2023-09-23T03:28:47.763 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.762+0000 7f4a23fff640 1 --2- 172.21.15.154:0/2473746238 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4a2806af30 0x7f4a2806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:47.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.762+0000 7f4a23fff640 1 -- 172.21.15.154:0/2473746238 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f4a34089cf0 con 0x7f4a3c0b57e0 2023-09-23T03:28:47.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.762+0000 7f4a3affd640 1 --2- 172.21.15.154:0/2473746238 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4a2806af30 0x7f4a2806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:47.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.763+0000 7f4a3affd640 1 --2- 172.21.15.154:0/2473746238 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4a2806af30 0x7f4a2806d3f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c0c8920 tx=0x7f4a2c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:47.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.766+0000 7f4a23fff640 1 -- 172.21.15.154:0/2473746238 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f4a3408e070 con 0x7f4a3c0b57e0 2023-09-23T03:28:47.993 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.992+0000 7f4a3bfff640 1 -- 172.21.15.154:0/2473746238 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a3c04e920 con 0x7f4a3c0b57e0 2023-09-23T03:28:47.994 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.994+0000 7f4a23fff640 1 -- 172.21.15.154:0/2473746238 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+4935 (secure 0 0 0) 0x7f4a34054c90 con 0x7f4a3c0b57e0 2023-09-23T03:28:47.994 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:28:47.994 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":121,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/2759187683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2759187683},{"type":"v1","addr":"172.21.15.154:6835","nonce":2759187683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":5924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3712825179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3712825179},{"type":"v1","addr":"172.21.15.190:6835","nonce":3712825179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":110},{"gid":6544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/4147378242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":4147378242},{"type":"v1","addr":"172.21.15.190:6834","nonce":4147378242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-23T03:27:44.470312+0000","modified":"2023-09-23T03:28:35.478753+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9306},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9306":{"gid":9306,"name":"c","rank":0,"incarnation":118,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6837/1250699891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":1250699891},{"type":"v1","addr":"172.21.15.154:6837","nonce":1250699891}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-09-23T03:28:47.995 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 121 2023-09-23T03:28:47.996 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.995+0000 7f4a3bfff640 1 -- 172.21.15.154:0/2473746238 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4a2806af30 msgr2=0x7f4a2806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:47.997 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.995+0000 7f4a3bfff640 1 --2- 172.21.15.154:0/2473746238 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4a2806af30 0x7f4a2806d3f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c0c8920 tx=0x7f4a2c009040 comp rx=0 tx=0).stop 2023-09-23T03:28:47.997 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.995+0000 7f4a3bfff640 1 -- 172.21.15.154:0/2473746238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a3c0b57e0 msgr2=0x7f4a3c0b5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:47.997 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.995+0000 7f4a3bfff640 1 --2- 172.21.15.154:0/2473746238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a3c0b57e0 0x7f4a3c0b5c60 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c08a550 tx=0x7f4a34007bc0 comp rx=0 tx=0).stop 2023-09-23T03:28:47.997 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.995+0000 7f4a3bfff640 1 -- 172.21.15.154:0/2473746238 shutdown_connections 2023-09-23T03:28:47.997 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.995+0000 7f4a3bfff640 1 --2- 172.21.15.154:0/2473746238 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4a2806af30 0x7f4a2806d3f0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:47.997 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.995+0000 7f4a3bfff640 1 --2- 172.21.15.154:0/2473746238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a3c0b57e0 0x7f4a3c0b5c60 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:47.997 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.995+0000 7f4a3bfff640 1 --2- 172.21.15.154:0/2473746238 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4a3c0cbdf0 0x7f4a3c0cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:47.997 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.995+0000 7f4a3bfff640 1 --2- 172.21.15.154:0/2473746238 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4a240047c0 0x7f4a3c154de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:47.997 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.995+0000 7f4a3bfff640 1 -- 172.21.15.154:0/2473746238 >> 172.21.15.154:0/2473746238 conn(0x7f4a3c09bb30 msgr2=0x7f4a3c0b70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:47.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.995+0000 7f4a3bfff640 1 -- 172.21.15.154:0/2473746238 shutdown_connections 2023-09-23T03:28:47.998 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:47.995+0000 7f4a3bfff640 1 -- 172.21.15.154:0/2473746238 wait complete. 2023-09-23T03:28:48.015 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:28:48.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.224+0000 7f1553fff640 1 Processor -- start 2023-09-23T03:28:48.224 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.224+0000 7f1553fff640 1 -- start start 2023-09-23T03:28:48.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.224+0000 7f1553fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f15540cb320 0x7f15540c9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:48.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.224+0000 7f1553fff640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f15540ca3c0 0x7f1554070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:48.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.224+0000 7f1553fff640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f155415a5f0 con 0x7f1554159ba0 2023-09-23T03:28:48.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.224+0000 7f1553fff640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f155415abb0 con 0x7f15540cb320 2023-09-23T03:28:48.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.224+0000 7f1553fff640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f155415a8d0 con 0x7f15540ca3c0 2023-09-23T03:28:48.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.225+0000 7f1552ffd640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f15540cb320 0x7f15540c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:48.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.225+0000 7f15527fc640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f1554159ba0 0x7f15540c9cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:34074/0 (socket says 172.21.15.154:34074) 2023-09-23T03:28:48.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.225+0000 7f15527fc640 1 -- 172.21.15.154:0/3025589620 learned_addr learned my addr 172.21.15.154:0/3025589620 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:28:48.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.225+0000 7f15537fe640 1 --2- 172.21.15.154:0/3025589620 >> v2:172.21.15.190:3300/0 conn(0x7f15540ca3c0 0x7f1554070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:48.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.225+0000 7f1552ffd640 1 -- 172.21.15.154:0/3025589620 >> v2:172.21.15.190:3300/0 conn(0x7f15540ca3c0 msgr2=0x7f1554070ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:48.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.225+0000 7f1552ffd640 1 --2- 172.21.15.154:0/3025589620 >> v2:172.21.15.190:3300/0 conn(0x7f15540ca3c0 0x7f1554070ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:48.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.225+0000 7f1552ffd640 1 -- 172.21.15.154:0/3025589620 >> v1:172.21.15.154:6789/0 conn(0x7f1554159ba0 legacy=0x7f15540c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:48.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.225+0000 7f1552ffd640 1 -- 172.21.15.154:0/3025589620 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15540c8b30 con 0x7f15540cb320 2023-09-23T03:28:48.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.225+0000 7f1551ffb640 1 -- 172.21.15.154:0/3025589620 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1784458955 0 0) 0x7f155415a5f0 con 0x7f1554159ba0 2023-09-23T03:28:48.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.226+0000 7f1552ffd640 1 --2- 172.21.15.154:0/3025589620 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15540cb320 0x7f15540c9330 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f15540b0f00 tx=0x7f154402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c1c73c88fae74d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:48.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.227+0000 7f1551ffb640 1 -- 172.21.15.154:0/3025589620 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154402e070 con 0x7f15540cb320 2023-09-23T03:28:48.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.227+0000 7f1551ffb640 1 -- 172.21.15.154:0/3025589620 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15440315a0 con 0x7f15540cb320 2023-09-23T03:28:48.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.227+0000 7f1551ffb640 1 -- 172.21.15.154:0/3025589620 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1544031930 con 0x7f15540cb320 2023-09-23T03:28:48.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.227+0000 7f1553fff640 1 -- 172.21.15.154:0/3025589620 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15540cb320 msgr2=0x7f15540c9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:48.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.227+0000 7f1553fff640 1 --2- 172.21.15.154:0/3025589620 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15540cb320 0x7f15540c9330 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f15540b0f00 tx=0x7f154402b840 comp rx=0 tx=0).stop 2023-09-23T03:28:48.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.227+0000 7f1553fff640 1 -- 172.21.15.154:0/3025589620 shutdown_connections 2023-09-23T03:28:48.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.227+0000 7f1553fff640 1 --2- 172.21.15.154:0/3025589620 >> v2:172.21.15.190:3300/0 conn(0x7f15540ca3c0 0x7f1554070ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:48.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.227+0000 7f1553fff640 1 --2- 172.21.15.154:0/3025589620 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f15540cb320 0x7f15540c9330 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:48.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.227+0000 7f1553fff640 1 -- 172.21.15.154:0/3025589620 >> 172.21.15.154:0/3025589620 conn(0x7f155406a4c0 msgr2=0x7f15540afe50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:48.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.227+0000 7f1553fff640 1 -- 172.21.15.154:0/3025589620 shutdown_connections 2023-09-23T03:28:48.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.227+0000 7f1553fff640 1 -- 172.21.15.154:0/3025589620 wait complete. 2023-09-23T03:28:48.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.229+0000 7f1553fff640 1 Processor -- start 2023-09-23T03:28:48.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.229+0000 7f1553fff640 1 -- start start 2023-09-23T03:28:48.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.230+0000 7f1553fff640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15540ca3c0 0x7f1554159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:48.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.230+0000 7f1553fff640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15540cb320 0x7f1554145570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:48.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.230+0000 7f1553fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1554159ba0 0x7f1554145ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:48.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.230+0000 7f1553fff640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155414a140 con 0x7f1554159ba0 2023-09-23T03:28:48.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.230+0000 7f1553fff640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155414a390 con 0x7f15540cb320 2023-09-23T03:28:48.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.230+0000 7f1553fff640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f155414a670 con 0x7f15540ca3c0 2023-09-23T03:28:48.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.230+0000 7f15537fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1554159ba0 0x7f1554145ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:48.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.231+0000 7f15537fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1554159ba0 0x7f1554145ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57090/0 (socket says 172.21.15.154:57090) 2023-09-23T03:28:48.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.231+0000 7f15537fe640 1 -- 172.21.15.154:0/678536799 learned_addr learned my addr 172.21.15.154:0/678536799 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:48.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.231+0000 7f1552ffd640 1 --2- 172.21.15.154:0/678536799 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15540ca3c0 0x7f1554159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:48.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.231+0000 7f15527fc640 1 --2- 172.21.15.154:0/678536799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15540cb320 0x7f1554145570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:48.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.231+0000 7f15537fe640 1 -- 172.21.15.154:0/678536799 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15540ca3c0 msgr2=0x7f1554159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:48.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.231+0000 7f15537fe640 1 --2- 172.21.15.154:0/678536799 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15540ca3c0 0x7f1554159020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:48.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.231+0000 7f15537fe640 1 -- 172.21.15.154:0/678536799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15540cb320 msgr2=0x7f1554145570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:48.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.231+0000 7f15537fe640 1 --2- 172.21.15.154:0/678536799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15540cb320 0x7f1554145570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:48.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.231+0000 7f15537fe640 1 -- 172.21.15.154:0/678536799 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1544009400 con 0x7f1554159ba0 2023-09-23T03:28:48.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.231+0000 7f15527fc640 1 --2- 172.21.15.154:0/678536799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15540cb320 0x7f1554145570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:48.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.231+0000 7f15537fe640 1 --2- 172.21.15.154:0/678536799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1554159ba0 0x7f1554145ab0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f154c00cd70 tx=0x7f154c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:48.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.232+0000 7f153b7fe640 1 -- 172.21.15.154:0/678536799 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c003b90 con 0x7f1554159ba0 2023-09-23T03:28:48.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.232+0000 7f153b7fe640 1 -- 172.21.15.154:0/678536799 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f154c003cf0 con 0x7f1554159ba0 2023-09-23T03:28:48.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.232+0000 7f153b7fe640 1 -- 172.21.15.154:0/678536799 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c004240 con 0x7f1554159ba0 2023-09-23T03:28:48.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.232+0000 7f1553fff640 1 -- 172.21.15.154:0/678536799 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f155414a950 con 0x7f1554159ba0 2023-09-23T03:28:48.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.232+0000 7f1553fff640 1 -- 172.21.15.154:0/678536799 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f155414b150 con 0x7f1554159ba0 2023-09-23T03:28:48.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.233+0000 7f1553fff640 1 -- 172.21.15.154:0/678536799 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1554066730 con 0x7f1554159ba0 2023-09-23T03:28:48.235 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.234+0000 7f153b7fe640 1 -- 172.21.15.154:0/678536799 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f154c0043a0 con 0x7f1554159ba0 2023-09-23T03:28:48.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.235+0000 7f153b7fe640 1 --2- 172.21.15.154:0/678536799 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f154006af60 0x7f154006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:48.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.235+0000 7f153b7fe640 1 -- 172.21.15.154:0/678536799 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f154c012070 con 0x7f1554159ba0 2023-09-23T03:28:48.236 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.235+0000 7f1552ffd640 1 --2- 172.21.15.154:0/678536799 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f154006af60 0x7f154006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:48.237 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.236+0000 7f1552ffd640 1 --2- 172.21.15.154:0/678536799 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f154006af60 0x7f154006d420 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f1544036000 tx=0x7f154403d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:48.240 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.239+0000 7f153b7fe640 1 -- 172.21.15.154:0/678536799 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f154c055100 con 0x7f1554159ba0 2023-09-23T03:28:48.429 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.428+0000 7f1553fff640 1 -- 172.21.15.154:0/678536799 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f155414ae70 con 0x7f1554159ba0 2023-09-23T03:28:48.431 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.430+0000 7f153b7fe640 1 -- 172.21.15.154:0/678536799 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+17083 (secure 0 0 0) 0x7f155414ae70 con 0x7f1554159ba0 2023-09-23T03:28:48.432 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:28:48.432 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":128,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:28:29.295164+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-09-23T03:27:42.207617+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-09-23T03:27:43.473867+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6837/4006549651":"2023-09-24T03:28:29.295137+0000","172.21.15.154:6836/4006549651":"2023-09-24T03:28:29.295137+0000","172.21.15.190:6835/3558096060":"2023-09-24T03:28:08.468628+0000","172.21.15.190:6833/3558096060":"2023-09-24T03:28:08.468628+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:28:48.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.433+0000 7f1553fff640 1 -- 172.21.15.154:0/678536799 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f154006af60 msgr2=0x7f154006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:48.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.433+0000 7f1553fff640 1 --2- 172.21.15.154:0/678536799 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f154006af60 0x7f154006d420 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f1544036000 tx=0x7f154403d040 comp rx=0 tx=0).stop 2023-09-23T03:28:48.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.433+0000 7f1553fff640 1 -- 172.21.15.154:0/678536799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1554159ba0 msgr2=0x7f1554145ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:48.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.433+0000 7f1553fff640 1 --2- 172.21.15.154:0/678536799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1554159ba0 0x7f1554145ab0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f154c00cd70 tx=0x7f154c00a660 comp rx=0 tx=0).stop 2023-09-23T03:28:48.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.433+0000 7f1553fff640 1 -- 172.21.15.154:0/678536799 shutdown_connections 2023-09-23T03:28:48.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.433+0000 7f1553fff640 1 --2- 172.21.15.154:0/678536799 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f154006af60 0x7f154006d420 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:48.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.433+0000 7f1553fff640 1 --2- 172.21.15.154:0/678536799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1554159ba0 0x7f1554145ab0 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:48.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.433+0000 7f1553fff640 1 --2- 172.21.15.154:0/678536799 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f15540cb320 0x7f1554145570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:48.435 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.433+0000 7f1553fff640 1 --2- 172.21.15.154:0/678536799 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f15540ca3c0 0x7f1554159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:48.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.433+0000 7f1553fff640 1 -- 172.21.15.154:0/678536799 >> 172.21.15.154:0/678536799 conn(0x7f155406a4c0 msgr2=0x7f15540c9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:48.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.433+0000 7f1553fff640 1 -- 172.21.15.154:0/678536799 shutdown_connections 2023-09-23T03:28:48.436 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.433+0000 7f1553fff640 1 -- 172.21.15.154:0/678536799 wait complete. 2023-09-23T03:28:48.453 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-09-23T03:28:48.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.653+0000 7f5eb0cfc640 1 Processor -- start 2023-09-23T03:28:48.655 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.654+0000 7f5eb0cfc640 1 -- start start 2023-09-23T03:28:48.656 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.656+0000 7f5eb0cfc640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5eac15ca80 0x7f5eac161140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:48.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.656+0000 7f5eb0cfc640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eac0cd870 con 0x7f5eac06a3e0 2023-09-23T03:28:48.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.656+0000 7f5eb0cfc640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eac161680 con 0x7f5eac071250 2023-09-23T03:28:48.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.656+0000 7f5eb0cfc640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eac0716e0 con 0x7f5eac15ca80 2023-09-23T03:28:48.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.656+0000 7f5eaa59c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5eac15ca80 0x7f5eac161140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:48.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.656+0000 7f5eaa59c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f5eac15ca80 0x7f5eac161140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57104/0 (socket says 172.21.15.154:57104) 2023-09-23T03:28:48.657 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.656+0000 7f5eaa59c640 1 -- 172.21.15.154:0/1490596112 learned_addr learned my addr 172.21.15.154:0/1490596112 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:48.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.656+0000 7f5eaa59c640 1 -- 172.21.15.154:0/1490596112 >> v1:172.21.15.190:6789/0 conn(0x7f5eac071250 legacy=0x7f5eac069cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:48.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.656+0000 7f5eaa59c640 1 -- 172.21.15.154:0/1490596112 >> v1:172.21.15.154:6789/0 conn(0x7f5eac06a3e0 legacy=0x7f5eac06ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:48.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.656+0000 7f5eaa59c640 1 -- 172.21.15.154:0/1490596112 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eac0719c0 con 0x7f5eac15ca80 2023-09-23T03:28:48.658 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.656+0000 7f5ea959a640 1 -- 172.21.15.154:0/1490596112 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1197352844 0 0) 0x7f5eac0cd870 con 0x7f5eac06a3e0 2023-09-23T03:28:48.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.657+0000 7f5eaa59c640 1 --2- 172.21.15.154:0/1490596112 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5eac15ca80 0x7f5eac161140 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c009870 tx=0x7f5e9c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890b2890530010a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:48.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.657+0000 7f5ea959a640 1 -- 172.21.15.154:0/1490596112 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c030860 con 0x7f5eac15ca80 2023-09-23T03:28:48.659 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.657+0000 7f5ea959a640 1 -- 172.21.15.154:0/1490596112 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e9c0309c0 con 0x7f5eac15ca80 2023-09-23T03:28:48.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.657+0000 7f5ea959a640 1 -- 172.21.15.154:0/1490596112 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c030d50 con 0x7f5eac15ca80 2023-09-23T03:28:48.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.658+0000 7f5eb0cfc640 1 -- 172.21.15.154:0/1490596112 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5eac15ca80 msgr2=0x7f5eac161140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:48.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.658+0000 7f5eb0cfc640 1 --2- 172.21.15.154:0/1490596112 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5eac15ca80 0x7f5eac161140 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c009870 tx=0x7f5e9c02b9a0 comp rx=0 tx=0).stop 2023-09-23T03:28:48.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.658+0000 7f5eb0cfc640 1 -- 172.21.15.154:0/1490596112 shutdown_connections 2023-09-23T03:28:48.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.658+0000 7f5eb0cfc640 1 --2- 172.21.15.154:0/1490596112 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5eac15ca80 0x7f5eac161140 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:48.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.658+0000 7f5eb0cfc640 1 -- 172.21.15.154:0/1490596112 >> 172.21.15.154:0/1490596112 conn(0x7f5eac0cc130 msgr2=0x7f5eac0cc540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:48.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.658+0000 7f5eb0cfc640 1 -- 172.21.15.154:0/1490596112 shutdown_connections 2023-09-23T03:28:48.660 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.658+0000 7f5eb0cfc640 1 -- 172.21.15.154:0/1490596112 wait complete. 2023-09-23T03:28:48.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.661+0000 7f5eb0cfc640 1 Processor -- start 2023-09-23T03:28:48.661 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.661+0000 7f5eb0cfc640 1 -- start start 2023-09-23T03:28:48.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.662+0000 7f5eb0cfc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5eac06a3e0 0x7f5eac150ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:48.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.662+0000 7f5eb0cfc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5eac15ca80 0x7f5eac0b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:48.662 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.662+0000 7f5eb0cfc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5eac0b4f70 0x7f5eac0b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:48.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.662+0000 7f5eb0cfc640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac15ed20 con 0x7f5eac06a3e0 2023-09-23T03:28:48.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.662+0000 7f5eb0cfc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac15f000 con 0x7f5eac0b4f70 2023-09-23T03:28:48.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.662+0000 7f5eb0cfc640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac15f2e0 con 0x7f5eac15ca80 2023-09-23T03:28:48.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.662+0000 7f5eaa59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5eac06a3e0 0x7f5eac150ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:48.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.662+0000 7f5eaa59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5eac06a3e0 0x7f5eac150ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57118/0 (socket says 172.21.15.154:57118) 2023-09-23T03:28:48.663 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.662+0000 7f5eaa59c640 1 -- 172.21.15.154:0/3682544723 learned_addr learned my addr 172.21.15.154:0/3682544723 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:48.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.662+0000 7f5eaa59c640 1 -- 172.21.15.154:0/3682544723 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5eac15ca80 msgr2=0x7f5eac0b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:48.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.662+0000 7f5eaa59c640 1 --2- 172.21.15.154:0/3682544723 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5eac15ca80 0x7f5eac0b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:48.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.662+0000 7f5eaa59c640 1 -- 172.21.15.154:0/3682544723 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5eac0b4f70 msgr2=0x7f5eac0b0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:48.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.662+0000 7f5eaa59c640 1 --2- 172.21.15.154:0/3682544723 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5eac0b4f70 0x7f5eac0b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:48.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.662+0000 7f5eaa59c640 1 -- 172.21.15.154:0/3682544723 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e9c0094d0 con 0x7f5eac06a3e0 2023-09-23T03:28:48.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.662+0000 7f5eaa59c640 1 --2- 172.21.15.154:0/3682544723 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5eac06a3e0 0x7f5eac150ad0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c009870 tx=0x7f5e9c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:48.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.662+0000 7f5ea2ffd640 1 -- 172.21.15.154:0/3682544723 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c039b60 con 0x7f5eac06a3e0 2023-09-23T03:28:48.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.662+0000 7f5ea2ffd640 1 -- 172.21.15.154:0/3682544723 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e9c039cc0 con 0x7f5eac06a3e0 2023-09-23T03:28:48.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.662+0000 7f5ea2ffd640 1 -- 172.21.15.154:0/3682544723 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c043630 con 0x7f5eac06a3e0 2023-09-23T03:28:48.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.662+0000 7f5eb0cfc640 1 -- 172.21.15.154:0/3682544723 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eac0aece0 con 0x7f5eac06a3e0 2023-09-23T03:28:48.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.663+0000 7f5eb0cfc640 1 -- 172.21.15.154:0/3682544723 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eac151010 con 0x7f5eac06a3e0 2023-09-23T03:28:48.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.664+0000 7f5eb0cfc640 1 -- 172.21.15.154:0/3682544723 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eac0af200 con 0x7f5eac06a3e0 2023-09-23T03:28:48.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.665+0000 7f5ea2ffd640 1 -- 172.21.15.154:0/3682544723 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f5e9c02f3b0 con 0x7f5eac06a3e0 2023-09-23T03:28:48.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.666+0000 7f5ea2ffd640 1 --2- 172.21.15.154:0/3682544723 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5e8c06af30 0x7f5e8c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:48.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.666+0000 7f5ea2ffd640 1 -- 172.21.15.154:0/3682544723 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6201+0+0 (secure 0 0 0) 0x7f5e9c0abd20 con 0x7f5eac06a3e0 2023-09-23T03:28:48.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.666+0000 7f5ea9d9b640 1 --2- 172.21.15.154:0/3682544723 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5e8c06af30 0x7f5e8c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:48.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.667+0000 7f5ea9d9b640 1 --2- 172.21.15.154:0/3682544723 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5e8c06af30 0x7f5e8c06d3f0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f5eac1f6d30 tx=0x7f5e940079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:48.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.671+0000 7f5ea2ffd640 1 -- 172.21.15.154:0/3682544723 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f5e9c0b0070 con 0x7f5eac06a3e0 2023-09-23T03:28:48.896 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.895+0000 7f5eb0cfc640 1 -- 172.21.15.154:0/3682544723 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5eac073eb0 con 0x7f5eac06a3e0 2023-09-23T03:28:48.897 INFO:tasks.ceph.mon.a.smithi154.stderr:2023-09-23T03:28:48.897+0000 7f3df35a6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-23T03:28:48.901 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.900+0000 7f5ea2ffd640 1 -- 172.21.15.154:0/3682544723 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v122) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5e9c034090 con 0x7f5eac06a3e0 2023-09-23T03:28:48.901 INFO:teuthology.orchestra.run.smithi154.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-09-23T03:28:48.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.905+0000 7f5eb0cfc640 1 -- 172.21.15.154:0/3682544723 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5e8c06af30 msgr2=0x7f5e8c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:48.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.905+0000 7f5eb0cfc640 1 --2- 172.21.15.154:0/3682544723 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5e8c06af30 0x7f5e8c06d3f0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f5eac1f6d30 tx=0x7f5e940079b0 comp rx=0 tx=0).stop 2023-09-23T03:28:48.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.905+0000 7f5eb0cfc640 1 -- 172.21.15.154:0/3682544723 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5eac06a3e0 msgr2=0x7f5eac150ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:48.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.905+0000 7f5eb0cfc640 1 --2- 172.21.15.154:0/3682544723 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5eac06a3e0 0x7f5eac150ad0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c009870 tx=0x7f5e9c004730 comp rx=0 tx=0).stop 2023-09-23T03:28:48.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.905+0000 7f5eb0cfc640 1 -- 172.21.15.154:0/3682544723 shutdown_connections 2023-09-23T03:28:48.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.905+0000 7f5eb0cfc640 1 --2- 172.21.15.154:0/3682544723 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f5e8c06af30 0x7f5e8c06d3f0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:48.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.905+0000 7f5eb0cfc640 1 --2- 172.21.15.154:0/3682544723 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f5eac0b4f70 0x7f5eac0b0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:48.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.905+0000 7f5eb0cfc640 1 --2- 172.21.15.154:0/3682544723 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5eac15ca80 0x7f5eac0b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:48.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.905+0000 7f5eb0cfc640 1 --2- 172.21.15.154:0/3682544723 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5eac06a3e0 0x7f5eac150ad0 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:48.906 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.905+0000 7f5eb0cfc640 1 -- 172.21.15.154:0/3682544723 >> 172.21.15.154:0/3682544723 conn(0x7f5eac0cc130 msgr2=0x7f5eac0cc510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:48.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.906+0000 7f5eb0cfc640 1 -- 172.21.15.154:0/3682544723 shutdown_connections 2023-09-23T03:28:48.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:48.906+0000 7f5eb0cfc640 1 -- 172.21.15.154:0/3682544723 wait complete. 2023-09-23T03:28:48.924 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-09-23T03:28:48.987 INFO:tasks.ceph.mds.c.smithi154.stdout:starting mds.c at 2023-09-23T03:28:49.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.117+0000 7efd1b1c7640 1 Processor -- start 2023-09-23T03:28:49.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.118+0000 7efd1b1c7640 1 -- start start 2023-09-23T03:28:49.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.119+0000 7efd1b1c7640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7efd14070560 0x7efd140b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:49.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.119+0000 7efd1b1c7640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd1415ab20 con 0x7efd140ca6b0 2023-09-23T03:28:49.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.119+0000 7efd1b1c7640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd14159f50 con 0x7efd140700e0 2023-09-23T03:28:49.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.119+0000 7efd1b1c7640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd140caa90 con 0x7efd14070560 2023-09-23T03:28:49.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.119+0000 7efd18f63640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7efd140ca6b0 0x7efd1406d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:34094/0 (socket says 172.21.15.154:34094) 2023-09-23T03:28:49.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.119+0000 7efd18f63640 1 -- 172.21.15.154:0/2464751761 learned_addr learned my addr 172.21.15.154:0/2464751761 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:28:49.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.119+0000 7efd19764640 1 --2- 172.21.15.154:0/2464751761 >> v2:172.21.15.154:3300/0 conn(0x7efd14070560 0x7efd140b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:49.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.119+0000 7efd19764640 1 -- 172.21.15.154:0/2464751761 >> v1:172.21.15.190:6789/0 conn(0x7efd140700e0 legacy=0x7efd140b2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:49.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.120+0000 7efd19764640 1 -- 172.21.15.154:0/2464751761 >> v1:172.21.15.154:6789/0 conn(0x7efd140ca6b0 legacy=0x7efd1406d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:49.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.120+0000 7efd19764640 1 -- 172.21.15.154:0/2464751761 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd1415a1f0 con 0x7efd14070560 2023-09-23T03:28:49.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.121+0000 7efd19764640 1 --2- 172.21.15.154:0/2464751761 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd14070560 0x7efd140b2e80 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7efd0c0097a0 tx=0x7efd0c02dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a03aaaac565e74d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:49.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.121+0000 7efd0b7fe640 1 -- 172.21.15.154:0/2464751761 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0c02f8a0 con 0x7efd14070560 2023-09-23T03:28:49.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.121+0000 7efd0b7fe640 1 -- 172.21.15.154:0/2464751761 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd0c02fa00 con 0x7efd14070560 2023-09-23T03:28:49.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.121+0000 7efd0b7fe640 1 -- 172.21.15.154:0/2464751761 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0c02fd90 con 0x7efd14070560 2023-09-23T03:28:49.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.121+0000 7efd1b1c7640 1 -- 172.21.15.154:0/2464751761 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd14070560 msgr2=0x7efd140b2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:49.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.121+0000 7efd1b1c7640 1 --2- 172.21.15.154:0/2464751761 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd14070560 0x7efd140b2e80 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7efd0c0097a0 tx=0x7efd0c02dae0 comp rx=0 tx=0).stop 2023-09-23T03:28:49.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.122+0000 7efd1b1c7640 1 -- 172.21.15.154:0/2464751761 shutdown_connections 2023-09-23T03:28:49.122 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.122+0000 7efd1b1c7640 1 --2- 172.21.15.154:0/2464751761 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd14070560 0x7efd140b2e80 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:49.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.122+0000 7efd1b1c7640 1 -- 172.21.15.154:0/2464751761 >> 172.21.15.154:0/2464751761 conn(0x7efd1406a440 msgr2=0x7efd140b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:49.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.122+0000 7efd1b1c7640 1 -- 172.21.15.154:0/2464751761 shutdown_connections 2023-09-23T03:28:49.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.122+0000 7efd1b1c7640 1 -- 172.21.15.154:0/2464751761 wait complete. 2023-09-23T03:28:49.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.123+0000 7efd1b1c7640 1 Processor -- start 2023-09-23T03:28:49.124 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.124+0000 7efd1b1c7640 1 -- start start 2023-09-23T03:28:49.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.124+0000 7efd1b1c7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd140700e0 0x7efd14152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:49.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.124+0000 7efd1b1c7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd14070560 0x7efd141f1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:49.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.124+0000 7efd1b1c7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd140ca6b0 0x7efd141548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:49.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.124+0000 7efd1b1c7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd141544b0 con 0x7efd14070560 2023-09-23T03:28:49.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.124+0000 7efd1b1c7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd141f2950 con 0x7efd140700e0 2023-09-23T03:28:49.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.124+0000 7efd1b1c7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd141f2c30 con 0x7efd140ca6b0 2023-09-23T03:28:49.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.125+0000 7efd0bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd14070560 0x7efd141f1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:49.125 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.125+0000 7efd0bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd14070560 0x7efd141f1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57150/0 (socket says 172.21.15.154:57150) 2023-09-23T03:28:49.126 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.125+0000 7efd0bfff640 1 -- 172.21.15.154:0/2506329673 learned_addr learned my addr 172.21.15.154:0/2506329673 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:49.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.125+0000 7efd18f63640 1 --2- 172.21.15.154:0/2506329673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd140700e0 0x7efd14152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:49.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.125+0000 7efd19764640 1 --2- 172.21.15.154:0/2506329673 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd140ca6b0 0x7efd141548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:49.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.125+0000 7efd0bfff640 1 -- 172.21.15.154:0/2506329673 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd140ca6b0 msgr2=0x7efd141548d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:49.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.125+0000 7efd0bfff640 1 --2- 172.21.15.154:0/2506329673 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd140ca6b0 0x7efd141548d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:49.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.125+0000 7efd0bfff640 1 -- 172.21.15.154:0/2506329673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd140700e0 msgr2=0x7efd14152fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:49.127 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.125+0000 7efd0bfff640 1 --2- 172.21.15.154:0/2506329673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd140700e0 0x7efd14152fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:49.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.125+0000 7efd0bfff640 1 -- 172.21.15.154:0/2506329673 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd0c009400 con 0x7efd14070560 2023-09-23T03:28:49.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.125+0000 7efd19764640 1 --2- 172.21.15.154:0/2506329673 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd140ca6b0 0x7efd141548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:49.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.125+0000 7efd18f63640 1 --2- 172.21.15.154:0/2506329673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd140700e0 0x7efd14152fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:49.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.125+0000 7efd0bfff640 1 --2- 172.21.15.154:0/2506329673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd14070560 0x7efd141f1f60 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7efd0400b3e0 tx=0x7efd0400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:49.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.126+0000 7efd097fa640 1 -- 172.21.15.154:0/2506329673 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd04011830 con 0x7efd14070560 2023-09-23T03:28:49.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.126+0000 7efd097fa640 1 -- 172.21.15.154:0/2506329673 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd040119b0 con 0x7efd14070560 2023-09-23T03:28:49.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.126+0000 7efd1b1c7640 1 -- 172.21.15.154:0/2506329673 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd141f2f10 con 0x7efd14070560 2023-09-23T03:28:49.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.126+0000 7efd097fa640 1 -- 172.21.15.154:0/2506329673 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd04011d60 con 0x7efd14070560 2023-09-23T03:28:49.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.126+0000 7efd1b1c7640 1 -- 172.21.15.154:0/2506329673 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd14153510 con 0x7efd14070560 2023-09-23T03:28:49.128 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.127+0000 7efd1b1c7640 1 -- 172.21.15.154:0/2506329673 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd141545f0 con 0x7efd14070560 2023-09-23T03:28:49.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.128+0000 7efd097fa640 1 -- 172.21.15.154:0/2506329673 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7efd0401a400 con 0x7efd14070560 2023-09-23T03:28:49.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.129+0000 7efd097fa640 1 --2- 172.21.15.154:0/2506329673 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efd0006af30 0x7efd0006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:49.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.129+0000 7efd097fa640 1 -- 172.21.15.154:0/2506329673 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6287+0+0 (secure 0 0 0) 0x7efd04089bb0 con 0x7efd14070560 2023-09-23T03:28:49.129 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.129+0000 7efd18f63640 1 --2- 172.21.15.154:0/2506329673 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efd0006af30 0x7efd0006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:49.130 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.130+0000 7efd18f63640 1 --2- 172.21.15.154:0/2506329673 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efd0006af30 0x7efd0006d3f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7efd10005c00 tx=0x7efd10009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:49.133 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.133+0000 7efd097fa640 1 -- 172.21.15.154:0/2506329673 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7efd04054910 con 0x7efd14070560 2023-09-23T03:28:49.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.360+0000 7efd1b1c7640 1 -- 172.21.15.154:0/2506329673 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7efd14153700 con 0x7efd14070560 2023-09-23T03:28:49.905 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.904+0000 7efd097fa640 1 -- 172.21.15.154:0/2506329673 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v123) v1 ==== 104+0+0 (secure 0 0 0) 0x7efd040150a0 con 0x7efd14070560 2023-09-23T03:28:49.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.908+0000 7efd1b1c7640 1 -- 172.21.15.154:0/2506329673 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efd0006af30 msgr2=0x7efd0006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:49.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.908+0000 7efd1b1c7640 1 --2- 172.21.15.154:0/2506329673 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efd0006af30 0x7efd0006d3f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7efd10005c00 tx=0x7efd10009040 comp rx=0 tx=0).stop 2023-09-23T03:28:49.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.908+0000 7efd1b1c7640 1 -- 172.21.15.154:0/2506329673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd14070560 msgr2=0x7efd141f1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:49.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.908+0000 7efd1b1c7640 1 --2- 172.21.15.154:0/2506329673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd14070560 0x7efd141f1f60 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7efd0400b3e0 tx=0x7efd0400b8a0 comp rx=0 tx=0).stop 2023-09-23T03:28:49.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.908+0000 7efd1b1c7640 1 -- 172.21.15.154:0/2506329673 shutdown_connections 2023-09-23T03:28:49.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.908+0000 7efd1b1c7640 1 --2- 172.21.15.154:0/2506329673 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7efd0006af30 0x7efd0006d3f0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:49.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.908+0000 7efd1b1c7640 1 --2- 172.21.15.154:0/2506329673 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7efd140ca6b0 0x7efd141548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:49.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.908+0000 7efd1b1c7640 1 --2- 172.21.15.154:0/2506329673 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efd14070560 0x7efd141f1f60 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:49.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.908+0000 7efd1b1c7640 1 --2- 172.21.15.154:0/2506329673 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7efd140700e0 0x7efd14152fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:49.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.908+0000 7efd1b1c7640 1 -- 172.21.15.154:0/2506329673 >> 172.21.15.154:0/2506329673 conn(0x7efd1406a440 msgr2=0x7efd1415ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:49.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.908+0000 7efd1b1c7640 1 -- 172.21.15.154:0/2506329673 shutdown_connections 2023-09-23T03:28:49.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:49.909+0000 7efd1b1c7640 1 -- 172.21.15.154:0/2506329673 wait complete. 2023-09-23T03:28:49.929 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-09-23T03:28:50.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.151+0000 7f6189783640 1 Processor -- start 2023-09-23T03:28:50.151 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.151+0000 7f6189783640 1 -- start start 2023-09-23T03:28:50.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.153+0000 7f6189783640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6184070560 0x7f61840b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:50.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.153+0000 7f6189783640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f618415ab20 con 0x7f61840700e0 2023-09-23T03:28:50.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.153+0000 7f6189783640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6184159f50 con 0x7f61840ca6b0 2023-09-23T03:28:50.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.153+0000 7f6189783640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61840caa90 con 0x7f6184070560 2023-09-23T03:28:50.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.153+0000 7f617bfff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f61840700e0 0x7f61840b2770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:34102/0 (socket says 172.21.15.154:34102) 2023-09-23T03:28:50.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.153+0000 7f617bfff640 1 -- 172.21.15.154:0/2227454590 learned_addr learned my addr 172.21.15.154:0/2227454590 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:28:50.154 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.154+0000 7f61837fe640 1 --2- 172.21.15.154:0/2227454590 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6184070560 0x7f61840b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:50.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.154+0000 7f61827fc640 1 -- 172.21.15.154:0/2227454590 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1617869012 0 0) 0x7f618415ab20 con 0x7f61840700e0 2023-09-23T03:28:50.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.154+0000 7f61827fc640 1 -- 172.21.15.154:0/2227454590 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6170005730 con 0x7f61840700e0 2023-09-23T03:28:50.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.154+0000 7f61827fc640 1 -- 172.21.15.154:0/2227454590 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 668206259 0 0) 0x7f6170005730 con 0x7f61840700e0 2023-09-23T03:28:50.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.154+0000 7f61827fc640 1 -- 172.21.15.154:0/2227454590 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6184070560 msgr2=0x7f61840b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:50.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.154+0000 7f61827fc640 1 --2- 172.21.15.154:0/2227454590 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6184070560 0x7f61840b2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:50.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.154+0000 7f61827fc640 1 -- 172.21.15.154:0/2227454590 >> v1:172.21.15.190:6789/0 conn(0x7f61840ca6b0 legacy=0x7f618406d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:50.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.154+0000 7f61827fc640 1 -- 172.21.15.154:0/2227454590 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618415a1f0 con 0x7f61840700e0 2023-09-23T03:28:50.156 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.154+0000 7f61827fc640 1 -- 172.21.15.154:0/2227454590 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f616c001670 con 0x7f61840700e0 2023-09-23T03:28:50.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.154+0000 7f61827fc640 1 -- 172.21.15.154:0/2227454590 >> v1:172.21.15.154:6789/0 conn(0x7f61840700e0 legacy=0x7f61840b2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:50.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.154+0000 7f61827fc640 1 --2- 172.21.15.154:0/2227454590 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6184070560 0x7f6170005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:50.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.154+0000 7f61827fc640 1 --2- 172.21.15.154:0/2227454590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6170006130 0x7f61700069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:50.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.155+0000 7f61827fc640 1 --2- 172.21.15.154:0/2227454590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6170006f10 0x7f6170007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:50.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.155+0000 7f61827fc640 1 -- 172.21.15.154:0/2227454590 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61840caa90 con 0x7f6170006130 2023-09-23T03:28:50.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.155+0000 7f61827fc640 1 -- 172.21.15.154:0/2227454590 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617000dc70 con 0x7f6170006f10 2023-09-23T03:28:50.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.155+0000 7f61827fc640 1 -- 172.21.15.154:0/2227454590 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f617000de30 con 0x7f6184070560 2023-09-23T03:28:50.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.155+0000 7f617bfff640 1 --2- 172.21.15.154:0/2227454590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6170006130 0x7f61700069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:50.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.155+0000 7f617bfff640 1 -- 172.21.15.154:0/2227454590 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6184070560 msgr2=0x7f6170005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:50.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.155+0000 7f617bfff640 1 --2- 172.21.15.154:0/2227454590 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6184070560 0x7f6170005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:50.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.155+0000 7f61837fe640 1 --2- 172.21.15.154:0/2227454590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6170006f10 0x7f6170007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:50.157 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.155+0000 7f617bfff640 1 -- 172.21.15.154:0/2227454590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6170006f10 msgr2=0x7f6170007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:50.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.155+0000 7f617bfff640 1 --2- 172.21.15.154:0/2227454590 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6170006f10 0x7f6170007390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:50.158 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.155+0000 7f617bfff640 1 -- 172.21.15.154:0/2227454590 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6170005730 con 0x7f6170006130 2023-09-23T03:28:50.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.156+0000 7f617bfff640 1 --2- 172.21.15.154:0/2227454590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6170006130 0x7f61700069d0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f616c00c240 tx=0x7f6170007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=127d6da6075df89a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:50.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.156+0000 7f61827fc640 1 -- 172.21.15.154:0/2227454590 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616c005ad0 con 0x7f6170006130 2023-09-23T03:28:50.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.156+0000 7f61827fc640 1 -- 172.21.15.154:0/2227454590 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f616c005c30 con 0x7f6170006130 2023-09-23T03:28:50.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.157+0000 7f61827fc640 1 -- 172.21.15.154:0/2227454590 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616c02caf0 con 0x7f6170006130 2023-09-23T03:28:50.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.157+0000 7f6189783640 1 -- 172.21.15.154:0/2227454590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6170006130 msgr2=0x7f61700069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:50.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.157+0000 7f6189783640 1 --2- 172.21.15.154:0/2227454590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6170006130 0x7f61700069d0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f616c00c240 tx=0x7f6170007a00 comp rx=0 tx=0).stop 2023-09-23T03:28:50.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.157+0000 7f61837fe640 1 -- 172.21.15.154:0/2227454590 reap_dead start 2023-09-23T03:28:50.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.157+0000 7f6189783640 1 -- 172.21.15.154:0/2227454590 shutdown_connections 2023-09-23T03:28:50.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.157+0000 7f6189783640 1 -- 172.21.15.154:0/2227454590 >> 172.21.15.154:0/2227454590 conn(0x7f618406a440 msgr2=0x7f61840b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:50.159 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.157+0000 7f6189783640 1 -- 172.21.15.154:0/2227454590 shutdown_connections 2023-09-23T03:28:50.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.157+0000 7f6189783640 1 -- 172.21.15.154:0/2227454590 wait complete. 2023-09-23T03:28:50.160 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.159+0000 7f6189783640 1 Processor -- start 2023-09-23T03:28:50.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.159+0000 7f6189783640 1 -- start start 2023-09-23T03:28:50.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.160+0000 7f6189783640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6170006130 0x7f6184147060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:50.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.160+0000 7f6189783640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61840c6c70 0x7f61841f13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:50.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.160+0000 7f6189783640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61840c70f0 0x7f618415c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:50.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.160+0000 7f6189783640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61840c3b70 con 0x7f61840c6c70 2023-09-23T03:28:50.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.160+0000 7f6189783640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61841475a0 con 0x7f61840c70f0 2023-09-23T03:28:50.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.160+0000 7f6189783640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61841f1e30 con 0x7f6170006130 2023-09-23T03:28:50.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.160+0000 7f617bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61840c6c70 0x7f61841f13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:50.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.161+0000 7f617bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61840c6c70 0x7f61841f13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57170/0 (socket says 172.21.15.154:57170) 2023-09-23T03:28:50.161 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.161+0000 7f617bfff640 1 -- 172.21.15.154:0/1804094772 learned_addr learned my addr 172.21.15.154:0/1804094772 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:50.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.161+0000 7f61837fe640 1 --2- 172.21.15.154:0/1804094772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61840c70f0 0x7f618415c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:50.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.161+0000 7f6182ffd640 1 --2- 172.21.15.154:0/1804094772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6170006130 0x7f6184147060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:50.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.161+0000 7f617bfff640 1 -- 172.21.15.154:0/1804094772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6170006130 msgr2=0x7f6184147060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:50.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.161+0000 7f617bfff640 1 --2- 172.21.15.154:0/1804094772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6170006130 0x7f6184147060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:50.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.161+0000 7f617bfff640 1 -- 172.21.15.154:0/1804094772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61840c70f0 msgr2=0x7f618415c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:50.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.161+0000 7f617bfff640 1 --2- 172.21.15.154:0/1804094772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61840c70f0 0x7f618415c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:50.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.161+0000 7f617bfff640 1 -- 172.21.15.154:0/1804094772 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61841f2110 con 0x7f61840c6c70 2023-09-23T03:28:50.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.161+0000 7f617bfff640 1 --2- 172.21.15.154:0/1804094772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61840c6c70 0x7f61841f13b0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f616c00c240 tx=0x7f616c001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:50.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.162+0000 7f617b7fe640 1 -- 172.21.15.154:0/1804094772 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616c03c040 con 0x7f61840c6c70 2023-09-23T03:28:50.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.162+0000 7f617b7fe640 1 -- 172.21.15.154:0/1804094772 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f616c02ccc0 con 0x7f61840c6c70 2023-09-23T03:28:50.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.162+0000 7f6189783640 1 -- 172.21.15.154:0/1804094772 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61840c34b0 con 0x7f61840c6c70 2023-09-23T03:28:50.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.162+0000 7f617b7fe640 1 -- 172.21.15.154:0/1804094772 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616c007290 con 0x7f61840c6c70 2023-09-23T03:28:50.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.162+0000 7f6189783640 1 -- 172.21.15.154:0/1804094772 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61840ccdf0 con 0x7f61840c6c70 2023-09-23T03:28:50.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.163+0000 7f6189783640 1 -- 172.21.15.154:0/1804094772 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61840c3cb0 con 0x7f61840c6c70 2023-09-23T03:28:50.164 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.164+0000 7f617b7fe640 1 -- 172.21.15.154:0/1804094772 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f616c032070 con 0x7f61840c6c70 2023-09-23T03:28:50.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.165+0000 7f617b7fe640 1 --2- 172.21.15.154:0/1804094772 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f616406af60 0x7f616406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:50.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.165+0000 7f617b7fe640 1 -- 172.21.15.154:0/1804094772 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6287+0+0 (secure 0 0 0) 0x7f616c0aade0 con 0x7f61840c6c70 2023-09-23T03:28:50.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.166+0000 7f6182ffd640 1 --2- 172.21.15.154:0/1804094772 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f616406af60 0x7f616406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:50.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.166+0000 7f6182ffd640 1 --2- 172.21.15.154:0/1804094772 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f616406af60 0x7f616406d420 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f61740016f0 tx=0x7f6174009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:50.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.170+0000 7f617b7fe640 1 -- 172.21.15.154:0/1804094772 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f61840c3cb0 con 0x7f61840c6c70 2023-09-23T03:28:50.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.358+0000 7f6189783640 1 -- 172.21.15.154:0/1804094772 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f61840c3cb0 con 0x7f61840c6c70 2023-09-23T03:28:50.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.917+0000 7f617b7fe640 1 -- 172.21.15.154:0/1804094772 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v130) v1 ==== 181+0+0 (secure 0 0 0) 0x7f61840c3cb0 con 0x7f61840c6c70 2023-09-23T03:28:50.917 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_metadata' removed 2023-09-23T03:28:50.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.923+0000 7f6189783640 1 -- 172.21.15.154:0/1804094772 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f616406af60 msgr2=0x7f616406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:50.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.923+0000 7f6189783640 1 --2- 172.21.15.154:0/1804094772 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f616406af60 0x7f616406d420 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f61740016f0 tx=0x7f6174009040 comp rx=0 tx=0).stop 2023-09-23T03:28:50.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.923+0000 7f6189783640 1 -- 172.21.15.154:0/1804094772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61840c6c70 msgr2=0x7f61841f13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:50.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.923+0000 7f6189783640 1 --2- 172.21.15.154:0/1804094772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61840c6c70 0x7f61841f13b0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f616c00c240 tx=0x7f616c001a00 comp rx=0 tx=0).stop 2023-09-23T03:28:50.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.923+0000 7f6189783640 1 -- 172.21.15.154:0/1804094772 shutdown_connections 2023-09-23T03:28:50.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.923+0000 7f6189783640 1 --2- 172.21.15.154:0/1804094772 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f616406af60 0x7f616406d420 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:50.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.923+0000 7f6189783640 1 --2- 172.21.15.154:0/1804094772 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f61840c70f0 0x7f618415c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:50.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.923+0000 7f6189783640 1 --2- 172.21.15.154:0/1804094772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f61840c6c70 0x7f61841f13b0 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:50.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.923+0000 7f6189783640 1 --2- 172.21.15.154:0/1804094772 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6170006130 0x7f6184147060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:50.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.923+0000 7f6189783640 1 -- 172.21.15.154:0/1804094772 >> 172.21.15.154:0/1804094772 conn(0x7f618406a440 msgr2=0x7f6184159120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:50.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.924+0000 7f6189783640 1 -- 172.21.15.154:0/1804094772 shutdown_connections 2023-09-23T03:28:50.925 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:50.924+0000 7f6189783640 1 -- 172.21.15.154:0/1804094772 wait complete. 2023-09-23T03:28:50.945 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-09-23T03:28:51.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.169+0000 7f7ebb1e2640 1 Processor -- start 2023-09-23T03:28:51.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.169+0000 7f7ebb1e2640 1 -- start start 2023-09-23T03:28:51.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.170+0000 7f7ebb1e2640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f7eb40cb3e0 0x7f7eb406ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:51.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.170+0000 7f7ebb1e2640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7eb40700e0 con 0x7f7eb40caf80 2023-09-23T03:28:51.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.170+0000 7f7ebb1e2640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7eb4070330 con 0x7f7eb4144130 2023-09-23T03:28:51.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.170+0000 7f7ebb1e2640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7eb4070fb0 con 0x7f7eb40cb3e0 2023-09-23T03:28:51.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.171+0000 7f7eb3fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f7eb40caf80 0x7f7eb406c320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:34114/0 (socket says 172.21.15.154:34114) 2023-09-23T03:28:51.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.171+0000 7f7eb3fff640 1 -- 172.21.15.154:0/2136125050 learned_addr learned my addr 172.21.15.154:0/2136125050 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:28:51.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.171+0000 7f7eb977f640 1 --2- 172.21.15.154:0/2136125050 >> v2:172.21.15.190:3300/0 conn(0x7f7eb40cb3e0 0x7f7eb406ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:51.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.171+0000 7f7eb37fe640 1 -- 172.21.15.154:0/2136125050 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3234063749 0 0) 0x7f7eb40700e0 con 0x7f7eb40caf80 2023-09-23T03:28:51.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.171+0000 7f7eb37fe640 1 -- 172.21.15.154:0/2136125050 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e9c003520 con 0x7f7eb40caf80 2023-09-23T03:28:51.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.171+0000 7f7eb37fe640 1 -- 172.21.15.154:0/2136125050 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 142825965 0 0) 0x7f7e9c003520 con 0x7f7eb40caf80 2023-09-23T03:28:51.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.171+0000 7f7eb37fe640 1 -- 172.21.15.154:0/2136125050 >> v2:172.21.15.190:3300/0 conn(0x7f7eb40cb3e0 msgr2=0x7f7eb406ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:51.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.171+0000 7f7eb37fe640 1 --2- 172.21.15.154:0/2136125050 >> v2:172.21.15.190:3300/0 conn(0x7f7eb40cb3e0 0x7f7eb406ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:51.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.171+0000 7f7eb37fe640 1 -- 172.21.15.154:0/2136125050 >> v1:172.21.15.190:6789/0 conn(0x7f7eb4144130 legacy=0x7f7eb40b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:51.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.172+0000 7f7eb37fe640 1 -- 172.21.15.154:0/2136125050 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb4070610 con 0x7f7eb40caf80 2023-09-23T03:28:51.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.172+0000 7f7eb37fe640 1 -- 172.21.15.154:0/2136125050 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f7ea80019f0 con 0x7f7eb40caf80 2023-09-23T03:28:51.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.172+0000 7f7eb37fe640 1 -- 172.21.15.154:0/2136125050 >> v1:172.21.15.154:6789/0 conn(0x7f7eb40caf80 legacy=0x7f7eb406c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:51.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.172+0000 7f7eb37fe640 1 --2- 172.21.15.154:0/2136125050 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e9c003970 0x7f7e9c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:51.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.172+0000 7f7eb37fe640 1 --2- 172.21.15.154:0/2136125050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e9c0047c0 0x7f7e9c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:51.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.172+0000 7f7eb37fe640 1 --2- 172.21.15.154:0/2136125050 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e9c005180 0x7f7e9c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:51.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.172+0000 7f7eb37fe640 1 -- 172.21.15.154:0/2136125050 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb4070fb0 con 0x7f7e9c0047c0 2023-09-23T03:28:51.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.172+0000 7f7eb37fe640 1 -- 172.21.15.154:0/2136125050 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c00c020 con 0x7f7e9c005180 2023-09-23T03:28:51.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.172+0000 7f7eb37fe640 1 -- 172.21.15.154:0/2136125050 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c00c1e0 con 0x7f7e9c003970 2023-09-23T03:28:51.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.172+0000 7f7eb3fff640 1 --2- 172.21.15.154:0/2136125050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e9c0047c0 0x7f7e9c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:51.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.172+0000 7f7eb977f640 1 --2- 172.21.15.154:0/2136125050 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e9c005180 0x7f7e9c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:51.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.172+0000 7f7eb3fff640 1 -- 172.21.15.154:0/2136125050 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e9c003970 msgr2=0x7f7e9c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:51.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.172+0000 7f7eb3fff640 1 --2- 172.21.15.154:0/2136125050 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7e9c003970 0x7f7e9c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:51.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.172+0000 7f7eb3fff640 1 -- 172.21.15.154:0/2136125050 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e9c005180 msgr2=0x7f7e9c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:51.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.172+0000 7f7eb3fff640 1 --2- 172.21.15.154:0/2136125050 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e9c005180 0x7f7e9c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:51.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.172+0000 7f7eb3fff640 1 -- 172.21.15.154:0/2136125050 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e9c003520 con 0x7f7e9c0047c0 2023-09-23T03:28:51.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.172+0000 7f7eb977f640 1 --2- 172.21.15.154:0/2136125050 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7e9c005180 0x7f7e9c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:51.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.172+0000 7f7eb977f640 1 -- 172.21.15.154:0/2136125050 reap_dead start 2023-09-23T03:28:51.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.173+0000 7f7eb3fff640 1 --2- 172.21.15.154:0/2136125050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e9c0047c0 0x7f7e9c004c40 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f7ea800a830 tx=0x7f7e9c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf2b6ac4fa327d3a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:51.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.174+0000 7f7eb37fe640 1 -- 172.21.15.154:0/2136125050 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea80072d0 con 0x7f7e9c0047c0 2023-09-23T03:28:51.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.174+0000 7f7eb37fe640 1 -- 172.21.15.154:0/2136125050 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea802f070 con 0x7f7e9c0047c0 2023-09-23T03:28:51.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.174+0000 7f7eb37fe640 1 -- 172.21.15.154:0/2136125050 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea80076f0 con 0x7f7e9c0047c0 2023-09-23T03:28:51.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.174+0000 7f7ebb1e2640 1 -- 172.21.15.154:0/2136125050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e9c0047c0 msgr2=0x7f7e9c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:51.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.174+0000 7f7ebb1e2640 1 --2- 172.21.15.154:0/2136125050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e9c0047c0 0x7f7e9c004c40 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f7ea800a830 tx=0x7f7e9c0057f0 comp rx=0 tx=0).stop 2023-09-23T03:28:51.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.174+0000 7f7ebb1e2640 1 -- 172.21.15.154:0/2136125050 shutdown_connections 2023-09-23T03:28:51.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.174+0000 7f7ebb1e2640 1 --2- 172.21.15.154:0/2136125050 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e9c0047c0 0x7f7e9c004c40 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:51.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.174+0000 7f7ebb1e2640 1 -- 172.21.15.154:0/2136125050 >> 172.21.15.154:0/2136125050 conn(0x7f7eb406a440 msgr2=0x7f7eb40b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:51.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.174+0000 7f7ebb1e2640 1 -- 172.21.15.154:0/2136125050 shutdown_connections 2023-09-23T03:28:51.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.175+0000 7f7ebb1e2640 1 -- 172.21.15.154:0/2136125050 wait complete. 2023-09-23T03:28:51.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.176+0000 7f7ebb1e2640 1 Processor -- start 2023-09-23T03:28:51.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.176+0000 7f7ebb1e2640 1 -- start start 2023-09-23T03:28:51.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.177+0000 7f7ebb1e2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e9c0047c0 0x7f7eb40a7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:51.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.177+0000 7f7ebb1e2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7eb41333b0 0x7f7eb4133810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:51.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.177+0000 7f7ebb1e2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7eb4134190 0x7f7eb4134610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:51.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.177+0000 7f7ebb1e2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb40b2280 con 0x7f7e9c0047c0 2023-09-23T03:28:51.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.177+0000 7f7ebb1e2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb4132100 con 0x7f7eb41333b0 2023-09-23T03:28:51.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.177+0000 7f7ebb1e2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb4132f20 con 0x7f7eb4134190 2023-09-23T03:28:51.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.178+0000 7f7eb8f7e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e9c0047c0 0x7f7eb40a7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:51.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.178+0000 7f7eb8f7e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e9c0047c0 0x7f7eb40a7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57182/0 (socket says 172.21.15.154:57182) 2023-09-23T03:28:51.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.178+0000 7f7eb8f7e640 1 -- 172.21.15.154:0/735455001 learned_addr learned my addr 172.21.15.154:0/735455001 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:51.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.178+0000 7f7eb3fff640 1 --2- 172.21.15.154:0/735455001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7eb41333b0 0x7f7eb4133810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:51.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.178+0000 7f7eb977f640 1 --2- 172.21.15.154:0/735455001 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7eb4134190 0x7f7eb4134610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:51.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.178+0000 7f7eb8f7e640 1 -- 172.21.15.154:0/735455001 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7eb4134190 msgr2=0x7f7eb4134610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:51.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.178+0000 7f7eb8f7e640 1 --2- 172.21.15.154:0/735455001 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7eb4134190 0x7f7eb4134610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:51.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.178+0000 7f7eb8f7e640 1 -- 172.21.15.154:0/735455001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7eb41333b0 msgr2=0x7f7eb4133810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:51.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.178+0000 7f7eb8f7e640 1 --2- 172.21.15.154:0/735455001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7eb41333b0 0x7f7eb4133810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:51.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.178+0000 7f7eb8f7e640 1 -- 172.21.15.154:0/735455001 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb40a7980 con 0x7f7e9c0047c0 2023-09-23T03:28:51.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.178+0000 7f7eb3fff640 1 --2- 172.21.15.154:0/735455001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7eb41333b0 0x7f7eb4133810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:51.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.179+0000 7f7eb8f7e640 1 --2- 172.21.15.154:0/735455001 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e9c0047c0 0x7f7eb40a7440 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00c3c0 tx=0x7f7eac00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:51.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.179+0000 7f7eb17fa640 1 -- 172.21.15.154:0/735455001 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac0052d0 con 0x7f7e9c0047c0 2023-09-23T03:28:51.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.179+0000 7f7eb17fa640 1 -- 172.21.15.154:0/735455001 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eac005450 con 0x7f7e9c0047c0 2023-09-23T03:28:51.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.179+0000 7f7eb17fa640 1 -- 172.21.15.154:0/735455001 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac007260 con 0x7f7e9c0047c0 2023-09-23T03:28:51.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.179+0000 7f7ebb1e2640 1 -- 172.21.15.154:0/735455001 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eb4132310 con 0x7f7e9c0047c0 2023-09-23T03:28:51.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.179+0000 7f7ebb1e2640 1 -- 172.21.15.154:0/735455001 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eb41328d0 con 0x7f7e9c0047c0 2023-09-23T03:28:51.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.180+0000 7f7ebb1e2640 1 -- 172.21.15.154:0/735455001 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eb4073eb0 con 0x7f7e9c0047c0 2023-09-23T03:28:51.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.181+0000 7f7eb17fa640 1 -- 172.21.15.154:0/735455001 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f7eac0073c0 con 0x7f7e9c0047c0 2023-09-23T03:28:51.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.182+0000 7f7eb17fa640 1 --2- 172.21.15.154:0/735455001 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7ea006af30 0x7f7ea006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:51.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.182+0000 7f7eb17fa640 1 -- 172.21.15.154:0/735455001 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5849+0+0 (secure 0 0 0) 0x7f7eac088bf0 con 0x7f7e9c0047c0 2023-09-23T03:28:51.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.182+0000 7f7eb3fff640 1 --2- 172.21.15.154:0/735455001 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7ea006af30 0x7f7ea006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:51.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.183+0000 7f7eb3fff640 1 --2- 172.21.15.154:0/735455001 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7ea006af30 0x7f7ea006d3f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8034000 tx=0x7f7ea8007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:51.186 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.186+0000 7f7eb17fa640 1 -- 172.21.15.154:0/735455001 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f7eac053b00 con 0x7f7e9c0047c0 2023-09-23T03:28:51.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.381+0000 7f7ebb1e2640 1 -- 172.21.15.154:0/735455001 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7eb40cf150 con 0x7f7e9c0047c0 2023-09-23T03:28:51.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.920+0000 7f7eb17fa640 1 -- 172.21.15.154:0/735455001 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v131) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7eac057120 con 0x7f7e9c0047c0 2023-09-23T03:28:51.921 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'cephfs_data' removed 2023-09-23T03:28:51.926 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.924+0000 7f7ebb1e2640 1 -- 172.21.15.154:0/735455001 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7ea006af30 msgr2=0x7f7ea006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:51.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.924+0000 7f7ebb1e2640 1 --2- 172.21.15.154:0/735455001 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7ea006af30 0x7f7ea006d3f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8034000 tx=0x7f7ea8007000 comp rx=0 tx=0).stop 2023-09-23T03:28:51.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.924+0000 7f7ebb1e2640 1 -- 172.21.15.154:0/735455001 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e9c0047c0 msgr2=0x7f7eb40a7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:51.927 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.924+0000 7f7ebb1e2640 1 --2- 172.21.15.154:0/735455001 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e9c0047c0 0x7f7eb40a7440 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00c3c0 tx=0x7f7eac00c880 comp rx=0 tx=0).stop 2023-09-23T03:28:51.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.928+0000 7f7ebb1e2640 1 -- 172.21.15.154:0/735455001 shutdown_connections 2023-09-23T03:28:51.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.928+0000 7f7ebb1e2640 1 --2- 172.21.15.154:0/735455001 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f7ea006af30 0x7f7ea006d3f0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:51.929 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.928+0000 7f7ebb1e2640 1 --2- 172.21.15.154:0/735455001 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f7eb4134190 0x7f7eb4134610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:51.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.928+0000 7f7ebb1e2640 1 --2- 172.21.15.154:0/735455001 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f7eb41333b0 0x7f7eb4133810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:51.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.928+0000 7f7ebb1e2640 1 --2- 172.21.15.154:0/735455001 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e9c0047c0 0x7f7eb40a7440 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:51.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.928+0000 7f7ebb1e2640 1 -- 172.21.15.154:0/735455001 >> 172.21.15.154:0/735455001 conn(0x7f7eb406a440 msgr2=0x7f7eb40cc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:51.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.929+0000 7f7ebb1e2640 1 -- 172.21.15.154:0/735455001 shutdown_connections 2023-09-23T03:28:51.930 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:51.929+0000 7f7ebb1e2640 1 -- 172.21.15.154:0/735455001 wait complete. 2023-09-23T03:28:51.954 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-09-23T03:28:52.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.163+0000 7fbc0ec34640 1 Processor -- start 2023-09-23T03:28:52.163 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.163+0000 7fbc0ec34640 1 -- start start 2023-09-23T03:28:52.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.164+0000 7fbc0ec34640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc080caf80 0x7fbc080cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:52.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.165+0000 7fbc0ec34640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbc0806c320 0x7fbc0806c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:52.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.165+0000 7fbc0ec34640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc080cb920 con 0x7fbc08144130 2023-09-23T03:28:52.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.165+0000 7fbc0ec34640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc08070fb0 con 0x7fbc0806c320 2023-09-23T03:28:52.165 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.165+0000 7fbc0ec34640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc080b2280 con 0x7fbc080caf80 2023-09-23T03:28:52.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.165+0000 7fbc0d1d1640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fbc0806c320 0x7fbc0806c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:52.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.165+0000 7fbc0c9d0640 1 --1- >> v1:172.21.15.190:6789/0 conn(0x7fbc08144130 0x7fbc080b1b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.154:33508/0 (socket says 172.21.15.154:33508) 2023-09-23T03:28:52.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.165+0000 7fbc0c9d0640 1 -- 172.21.15.154:0/3926332837 learned_addr learned my addr 172.21.15.154:0/3926332837 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:28:52.166 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.165+0000 7fbc07fff640 1 --2- 172.21.15.154:0/3926332837 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc080caf80 0x7fbc080cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:52.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.166+0000 7fbc0d1d1640 1 -- 172.21.15.154:0/3926332837 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc080caf80 msgr2=0x7fbc080cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:52.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.166+0000 7fbc0d1d1640 1 --2- 172.21.15.154:0/3926332837 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc080caf80 0x7fbc080cb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:52.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.166+0000 7fbc0d1d1640 1 -- 172.21.15.154:0/3926332837 >> v1:172.21.15.190:6789/0 conn(0x7fbc08144130 legacy=0x7fbc080b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:52.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.166+0000 7fbc0d1d1640 1 -- 172.21.15.154:0/3926332837 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc080700e0 con 0x7fbc0806c320 2023-09-23T03:28:52.167 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.167+0000 7fbc0d1d1640 1 --2- 172.21.15.154:0/3926332837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc0806c320 0x7fbc0806c7b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fbc00009d90 tx=0x7fbc0002bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fa92396141b63ab server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:52.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.168+0000 7fbc077fe640 1 -- 172.21.15.154:0/3926332837 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0002cc50 con 0x7fbc0806c320 2023-09-23T03:28:52.168 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.168+0000 7fbc077fe640 1 -- 172.21.15.154:0/3926332837 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc0002cdb0 con 0x7fbc0806c320 2023-09-23T03:28:52.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.168+0000 7fbc077fe640 1 -- 172.21.15.154:0/3926332837 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc00034660 con 0x7fbc0806c320 2023-09-23T03:28:52.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.169+0000 7fbc0ec34640 1 -- 172.21.15.154:0/3926332837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc0806c320 msgr2=0x7fbc0806c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:52.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.169+0000 7fbc0ec34640 1 --2- 172.21.15.154:0/3926332837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc0806c320 0x7fbc0806c7b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fbc00009d90 tx=0x7fbc0002bea0 comp rx=0 tx=0).stop 2023-09-23T03:28:52.169 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.169+0000 7fbc0ec34640 1 -- 172.21.15.154:0/3926332837 shutdown_connections 2023-09-23T03:28:52.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.169+0000 7fbc0ec34640 1 --2- 172.21.15.154:0/3926332837 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc0806c320 0x7fbc0806c7b0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:52.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.169+0000 7fbc0ec34640 1 --2- 172.21.15.154:0/3926332837 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc080caf80 0x7fbc080cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:52.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.169+0000 7fbc0ec34640 1 -- 172.21.15.154:0/3926332837 >> 172.21.15.154:0/3926332837 conn(0x7fbc0806a440 msgr2=0x7fbc080b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:52.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.169+0000 7fbc0ec34640 1 -- 172.21.15.154:0/3926332837 shutdown_connections 2023-09-23T03:28:52.170 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.169+0000 7fbc0ec34640 1 -- 172.21.15.154:0/3926332837 wait complete. 2023-09-23T03:28:52.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.171+0000 7fbc0ec34640 1 Processor -- start 2023-09-23T03:28:52.171 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.171+0000 7fbc0ec34640 1 -- start start 2023-09-23T03:28:52.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.172+0000 7fbc0ec34640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc0806c320 0x7fbc080a8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:52.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.172+0000 7fbc0ec34640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc080caf80 0x7fbc081f3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:52.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.172+0000 7fbc0ec34640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc08144130 0x7fbc081f35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:52.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.172+0000 7fbc0ec34640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc08137130 con 0x7fbc0806c320 2023-09-23T03:28:52.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.172+0000 7fbc0ec34640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc080c6380 con 0x7fbc080caf80 2023-09-23T03:28:52.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.172+0000 7fbc0ec34640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc080a87a0 con 0x7fbc08144130 2023-09-23T03:28:52.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.172+0000 7fbc0c9d0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc0806c320 0x7fbc080a8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:52.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.172+0000 7fbc0c9d0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc0806c320 0x7fbc080a8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57190/0 (socket says 172.21.15.154:57190) 2023-09-23T03:28:52.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.172+0000 7fbc0c9d0640 1 -- 172.21.15.154:0/2843059168 learned_addr learned my addr 172.21.15.154:0/2843059168 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:52.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.172+0000 7fbc0c9d0640 1 -- 172.21.15.154:0/2843059168 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc08144130 msgr2=0x7fbc081f35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:52.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.172+0000 7fbc0c9d0640 1 --2- 172.21.15.154:0/2843059168 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc08144130 0x7fbc081f35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:52.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.172+0000 7fbc0c9d0640 1 -- 172.21.15.154:0/2843059168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc080caf80 msgr2=0x7fbc081f3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:52.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.172+0000 7fbc07fff640 1 --2- 172.21.15.154:0/2843059168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc080caf80 0x7fbc081f3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:52.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.173+0000 7fbc0c9d0640 1 --2- 172.21.15.154:0/2843059168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc080caf80 0x7fbc081f3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:52.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.173+0000 7fbc0c9d0640 1 -- 172.21.15.154:0/2843059168 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc00009a70 con 0x7fbc0806c320 2023-09-23T03:28:52.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.173+0000 7fbc0c9d0640 1 --2- 172.21.15.154:0/2843059168 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc0806c320 0x7fbc080a8260 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc00c4b0 tx=0x7fbbfc00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:52.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.173+0000 7fbc057fa640 1 -- 172.21.15.154:0/2843059168 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc005ce0 con 0x7fbc0806c320 2023-09-23T03:28:52.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.173+0000 7fbc057fa640 1 -- 172.21.15.154:0/2843059168 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbfc005e40 con 0x7fbc0806c320 2023-09-23T03:28:52.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.173+0000 7fbc057fa640 1 -- 172.21.15.154:0/2843059168 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc007270 con 0x7fbc0806c320 2023-09-23T03:28:52.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.173+0000 7fbc0ec34640 1 -- 172.21.15.154:0/2843059168 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc080c59b0 con 0x7fbc0806c320 2023-09-23T03:28:52.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.173+0000 7fbc0ec34640 1 -- 172.21.15.154:0/2843059168 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc080c5f70 con 0x7fbc0806c320 2023-09-23T03:28:52.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.174+0000 7fbc0ec34640 1 -- 172.21.15.154:0/2843059168 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc08073eb0 con 0x7fbc0806c320 2023-09-23T03:28:52.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.175+0000 7fbc057fa640 1 -- 172.21.15.154:0/2843059168 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fbbfc0073d0 con 0x7fbc0806c320 2023-09-23T03:28:52.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.176+0000 7fbc057fa640 1 --2- 172.21.15.154:0/2843059168 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbbec06af30 0x7fbbec06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:52.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.176+0000 7fbc057fa640 1 -- 172.21.15.154:0/2843059168 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fbbfc088aa0 con 0x7fbc0806c320 2023-09-23T03:28:52.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.177+0000 7fbc07fff640 1 --2- 172.21.15.154:0/2843059168 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbbec06af30 0x7fbbec06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:52.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.177+0000 7fbc07fff640 1 --2- 172.21.15.154:0/2843059168 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbbec06af30 0x7fbbec06d3f0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fbc08141760 tx=0x7fbbf4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:52.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.181+0000 7fbc057fa640 1 -- 172.21.15.154:0/2843059168 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fbbfc053c90 con 0x7fbc0806c320 2023-09-23T03:28:52.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.375+0000 7fbc0ec34640 1 -- 172.21.15.154:0/2843059168 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fbc080b38e0 con 0x7fbc0806c320 2023-09-23T03:28:52.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.388+0000 7fbc057fa640 1 -- 172.21.15.154:0/2843059168 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbbfc0170a0 con 0x7fbc0806c320 2023-09-23T03:28:52.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.393+0000 7fbc0ec34640 1 -- 172.21.15.154:0/2843059168 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbbec06af30 msgr2=0x7fbbec06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:52.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.393+0000 7fbc0ec34640 1 --2- 172.21.15.154:0/2843059168 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbbec06af30 0x7fbbec06d3f0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fbc08141760 tx=0x7fbbf4008040 comp rx=0 tx=0).stop 2023-09-23T03:28:52.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.394+0000 7fbc0ec34640 1 -- 172.21.15.154:0/2843059168 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc0806c320 msgr2=0x7fbc080a8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:52.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.394+0000 7fbc0ec34640 1 --2- 172.21.15.154:0/2843059168 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc0806c320 0x7fbc080a8260 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc00c4b0 tx=0x7fbbfc00c970 comp rx=0 tx=0).stop 2023-09-23T03:28:52.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.394+0000 7fbc0ec34640 1 -- 172.21.15.154:0/2843059168 shutdown_connections 2023-09-23T03:28:52.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.394+0000 7fbc0ec34640 1 --2- 172.21.15.154:0/2843059168 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fbbec06af30 0x7fbbec06d3f0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:52.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.394+0000 7fbc0ec34640 1 --2- 172.21.15.154:0/2843059168 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fbc08144130 0x7fbc081f35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:52.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.394+0000 7fbc0ec34640 1 --2- 172.21.15.154:0/2843059168 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fbc080caf80 0x7fbc081f3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:52.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.394+0000 7fbc0ec34640 1 --2- 172.21.15.154:0/2843059168 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbc0806c320 0x7fbc080a8260 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:52.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.394+0000 7fbc0ec34640 1 -- 172.21.15.154:0/2843059168 >> 172.21.15.154:0/2843059168 conn(0x7fbc0806a440 msgr2=0x7fbc0806c700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:52.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.394+0000 7fbc0ec34640 1 -- 172.21.15.154:0/2843059168 shutdown_connections 2023-09-23T03:28:52.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.394+0000 7fbc0ec34640 1 -- 172.21.15.154:0/2843059168 wait complete. 2023-09-23T03:28:52.413 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_data_scan.TestDataScan.test_fragmented_injection' 2023-09-23T03:28:52.597 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.597+0000 7fc402826640 1 Processor -- start 2023-09-23T03:28:52.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.597+0000 7fc402826640 1 -- start start 2023-09-23T03:28:52.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.599+0000 7fc402826640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc3fc160970 0x7fc3fc0b0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:52.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.599+0000 7fc402826640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3fc160df0 0x7fc3fc0b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:52.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.599+0000 7fc402826640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3fc06d930 con 0x7fc3fc160280 2023-09-23T03:28:52.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.599+0000 7fc402826640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3fc06c810 con 0x7fc3fc160970 2023-09-23T03:28:52.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.599+0000 7fc402826640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3fc1613c0 con 0x7fc3fc160df0 2023-09-23T03:28:52.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.599+0000 7fc3fb7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc3fc160970 0x7fc3fc0b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:52.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.599+0000 7fc3fb7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc3fc160970 0x7fc3fc0b0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57194/0 (socket says 172.21.15.154:57194) 2023-09-23T03:28:52.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.599+0000 7fc3fbfff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fc3fc160280 0x7fc3fc06d220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:34120/0 (socket says 172.21.15.154:34120) 2023-09-23T03:28:52.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.599+0000 7fc3fb7fe640 1 -- 172.21.15.154:0/577163477 learned_addr learned my addr 172.21.15.154:0/577163477 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:52.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.600+0000 7fc3fb7fe640 1 -- 172.21.15.154:0/577163477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3fc160df0 msgr2=0x7fc3fc0b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:52.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.600+0000 7fc400dc3640 1 --2- 172.21.15.154:0/577163477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3fc160df0 0x7fc3fc0b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:52.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.600+0000 7fc3fb7fe640 1 --2- 172.21.15.154:0/577163477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3fc160df0 0x7fc3fc0b05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:52.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.600+0000 7fc3fb7fe640 1 -- 172.21.15.154:0/577163477 >> v1:172.21.15.154:6789/0 conn(0x7fc3fc160280 legacy=0x7fc3fc06d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:52.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.600+0000 7fc3fb7fe640 1 -- 172.21.15.154:0/577163477 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3fc1616a0 con 0x7fc3fc160970 2023-09-23T03:28:52.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.600+0000 7fc3faffd640 1 -- 172.21.15.154:0/577163477 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3608247855 0 0) 0x7fc3fc06d930 con 0x7fc3fc160280 2023-09-23T03:28:52.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.600+0000 7fc400dc3640 1 --2- 172.21.15.154:0/577163477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3fc160df0 0x7fc3fc0b05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:52.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.601+0000 7fc3fb7fe640 1 --2- 172.21.15.154:0/577163477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3fc160970 0x7fc3fc0b0070 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0009870 tx=0x7fc3f002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=157f2dd64b38de9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:52.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.601+0000 7fc3faffd640 1 -- 172.21.15.154:0/577163477 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0030840 con 0x7fc3fc160970 2023-09-23T03:28:52.602 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.601+0000 7fc3faffd640 1 -- 172.21.15.154:0/577163477 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3f00309a0 con 0x7fc3fc160970 2023-09-23T03:28:52.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.601+0000 7fc3faffd640 1 -- 172.21.15.154:0/577163477 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0030d30 con 0x7fc3fc160970 2023-09-23T03:28:52.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.602+0000 7fc402826640 1 -- 172.21.15.154:0/577163477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3fc160970 msgr2=0x7fc3fc0b0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:52.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.602+0000 7fc402826640 1 --2- 172.21.15.154:0/577163477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3fc160970 0x7fc3fc0b0070 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0009870 tx=0x7fc3f002b980 comp rx=0 tx=0).stop 2023-09-23T03:28:52.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.602+0000 7fc402826640 1 -- 172.21.15.154:0/577163477 shutdown_connections 2023-09-23T03:28:52.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.602+0000 7fc402826640 1 --2- 172.21.15.154:0/577163477 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3fc160df0 0x7fc3fc0b05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:52.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.602+0000 7fc402826640 1 --2- 172.21.15.154:0/577163477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3fc160970 0x7fc3fc0b0070 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:52.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.602+0000 7fc402826640 1 -- 172.21.15.154:0/577163477 >> 172.21.15.154:0/577163477 conn(0x7fc3fc0cb070 msgr2=0x7fc3fc0cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:52.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.602+0000 7fc402826640 1 -- 172.21.15.154:0/577163477 shutdown_connections 2023-09-23T03:28:52.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.602+0000 7fc402826640 1 -- 172.21.15.154:0/577163477 wait complete. 2023-09-23T03:28:52.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.603+0000 7fc402826640 1 Processor -- start 2023-09-23T03:28:52.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.604+0000 7fc402826640 1 -- start start 2023-09-23T03:28:52.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.604+0000 7fc402826640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3fc160280 0x7fc3fc0b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:52.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.605+0000 7fc402826640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3fc160970 0x7fc3fc0b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:52.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.605+0000 7fc402826640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3fc160df0 0x7fc3fc15f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:52.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.605+0000 7fc402826640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3fc0b2160 con 0x7fc3fc160df0 2023-09-23T03:28:52.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.605+0000 7fc402826640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3fc14e660 con 0x7fc3fc160280 2023-09-23T03:28:52.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.605+0000 7fc402826640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3fc0b2370 con 0x7fc3fc160970 2023-09-23T03:28:52.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.605+0000 7fc400dc3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3fc160df0 0x7fc3fc15f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:52.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.605+0000 7fc400dc3640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3fc160df0 0x7fc3fc15f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:57196/0 (socket says 172.21.15.154:57196) 2023-09-23T03:28:52.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.605+0000 7fc400dc3640 1 -- 172.21.15.154:0/3096107556 learned_addr learned my addr 172.21.15.154:0/3096107556 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:52.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.605+0000 7fc3fb7fe640 1 --2- 172.21.15.154:0/3096107556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3fc160970 0x7fc3fc0b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:52.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.605+0000 7fc3fbfff640 1 --2- 172.21.15.154:0/3096107556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3fc160280 0x7fc3fc0b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:52.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.605+0000 7fc400dc3640 1 -- 172.21.15.154:0/3096107556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3fc160970 msgr2=0x7fc3fc0b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:52.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.605+0000 7fc400dc3640 1 --2- 172.21.15.154:0/3096107556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3fc160970 0x7fc3fc0b1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:52.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.605+0000 7fc400dc3640 1 -- 172.21.15.154:0/3096107556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3fc160280 msgr2=0x7fc3fc0b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:52.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.605+0000 7fc400dc3640 1 --2- 172.21.15.154:0/3096107556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3fc160280 0x7fc3fc0b3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:52.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.605+0000 7fc400dc3640 1 -- 172.21.15.154:0/3096107556 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f00094d0 con 0x7fc3fc160df0 2023-09-23T03:28:52.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.606+0000 7fc400dc3640 1 --2- 172.21.15.154:0/3096107556 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3fc160df0 0x7fc3fc15f000 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fc3f400cd50 tx=0x7fc3f400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:52.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.606+0000 7fc3f8ff9640 1 -- 172.21.15.154:0/3096107556 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f400f680 con 0x7fc3fc160df0 2023-09-23T03:28:52.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.606+0000 7fc3f8ff9640 1 -- 172.21.15.154:0/3096107556 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3f400f800 con 0x7fc3fc160df0 2023-09-23T03:28:52.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.606+0000 7fc3f8ff9640 1 -- 172.21.15.154:0/3096107556 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f400fbb0 con 0x7fc3fc160df0 2023-09-23T03:28:52.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.606+0000 7fc402826640 1 -- 172.21.15.154:0/3096107556 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3fc0c5760 con 0x7fc3fc160df0 2023-09-23T03:28:52.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.606+0000 7fc402826640 1 -- 172.21.15.154:0/3096107556 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3fc0c5d20 con 0x7fc3fc160df0 2023-09-23T03:28:52.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.607+0000 7fc402826640 1 -- 172.21.15.154:0/3096107556 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3fc067870 con 0x7fc3fc160df0 2023-09-23T03:28:52.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.608+0000 7fc3f8ff9640 1 -- 172.21.15.154:0/3096107556 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fc3f400b040 con 0x7fc3fc160df0 2023-09-23T03:28:52.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.609+0000 7fc3f8ff9640 1 --2- 172.21.15.154:0/3096107556 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc3e806af30 0x7fc3e806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:52.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.609+0000 7fc3f8ff9640 1 -- 172.21.15.154:0/3096107556 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fc3f4089fe0 con 0x7fc3fc160df0 2023-09-23T03:28:52.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.609+0000 7fc3fbfff640 1 --2- 172.21.15.154:0/3096107556 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc3e806af30 0x7fc3e806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:52.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.610+0000 7fc3fbfff640 1 --2- 172.21.15.154:0/3096107556 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc3e806af30 0x7fc3e806d3f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec007fd0 tx=0x7fc3ec009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:52.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.613+0000 7fc3f8ff9640 1 -- 172.21.15.154:0/3096107556 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fc3f40551c0 con 0x7fc3fc160df0 2023-09-23T03:28:52.874 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.873+0000 7fc402826640 1 -- 172.21.15.154:0/3096107556 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_data_scan.TestDataScan.test_fragmented_injection"]} v 0) v1 -- 0x7fc3fc073eb0 con 0x7fc3fc160df0 2023-09-23T03:28:52.934 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.933+0000 7fc3f8ff9640 1 -- 172.21.15.154:0/3096107556 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_data_scan.TestDataScan.test_fragmented_injection"]}]=0 v609) v1 ==== 145+0+0 (secure 0 0 0) 0x7fc3f40587e0 con 0x7fc3fc160df0 2023-09-23T03:28:52.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.936+0000 7fc402826640 1 -- 172.21.15.154:0/3096107556 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc3e806af30 msgr2=0x7fc3e806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:52.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.936+0000 7fc402826640 1 --2- 172.21.15.154:0/3096107556 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc3e806af30 0x7fc3e806d3f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec007fd0 tx=0x7fc3ec009040 comp rx=0 tx=0).stop 2023-09-23T03:28:52.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.937+0000 7fc402826640 1 -- 172.21.15.154:0/3096107556 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3fc160df0 msgr2=0x7fc3fc15f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:52.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.937+0000 7fc402826640 1 --2- 172.21.15.154:0/3096107556 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3fc160df0 0x7fc3fc15f000 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fc3f400cd50 tx=0x7fc3f400a650 comp rx=0 tx=0).stop 2023-09-23T03:28:52.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.937+0000 7fc402826640 1 -- 172.21.15.154:0/3096107556 shutdown_connections 2023-09-23T03:28:52.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.937+0000 7fc402826640 1 --2- 172.21.15.154:0/3096107556 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc3e806af30 0x7fc3e806d3f0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:52.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.937+0000 7fc402826640 1 --2- 172.21.15.154:0/3096107556 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc3fc160df0 0x7fc3fc15f000 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:52.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.937+0000 7fc402826640 1 --2- 172.21.15.154:0/3096107556 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc3fc160970 0x7fc3fc0b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:52.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.937+0000 7fc402826640 1 --2- 172.21.15.154:0/3096107556 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc3fc160280 0x7fc3fc0b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:52.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.937+0000 7fc402826640 1 -- 172.21.15.154:0/3096107556 >> 172.21.15.154:0/3096107556 conn(0x7fc3fc0cb070 msgr2=0x7fc3fc15d870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:52.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.937+0000 7fc402826640 1 -- 172.21.15.154:0/3096107556 shutdown_connections 2023-09-23T03:28:52.938 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:52.937+0000 7fc402826640 1 -- 172.21.15.154:0/3096107556 wait complete. 2023-09-23T03:28:52.957 INFO:tasks.cephfs_test_runner:That when injecting a dentry into a fragmented directory, we put it in the right fragment. ... ERROR 2023-09-23T03:28:52.957 INFO:tasks.cephfs_test_runner: 2023-09-23T03:28:52.958 INFO:tasks.cephfs_test_runner:====================================================================== 2023-09-23T03:28:52.958 INFO:tasks.cephfs_test_runner:ERROR: test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan) 2023-09-23T03:28:52.958 INFO:tasks.cephfs_test_runner:That when injecting a dentry into a fragmented directory, we put it in the right fragment. 2023-09-23T03:28:52.958 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-09-23T03:28:52.958 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-09-23T03:28:52.958 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_862663913a74a99e1c52d50e6a7e59dd6705749a/qa/tasks/cephfs/test_data_scan.py", line 560, in test_fragmented_injection 2023-09-23T03:28:52.958 INFO:tasks.cephfs_test_runner: out = self.mount_a.run_shell_payload(f"sudo cat subdir/{victim_dentry}", omit_sudo=False).stdout.getvalue().strip() 2023-09-23T03:28:52.958 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_862663913a74a99e1c52d50e6a7e59dd6705749a/qa/tasks/cephfs/mount.py", line 793, in run_shell_payload 2023-09-23T03:28:52.958 INFO:tasks.cephfs_test_runner: return self.run_shell(**kwargs) 2023-09-23T03:28:52.958 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_862663913a74a99e1c52d50e6a7e59dd6705749a/qa/tasks/cephfs/mount.py", line 784, in run_shell 2023-09-23T03:28:52.958 INFO:tasks.cephfs_test_runner: return self.client_remote.run(args=args, cwd=cwd, timeout=timeout, 2023-09-23T03:28:52.959 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run 2023-09-23T03:28:52.959 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-09-23T03:28:52.959 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run 2023-09-23T03:28:52.959 INFO:tasks.cephfs_test_runner: r.wait() 2023-09-23T03:28:52.959 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait 2023-09-23T03:28:52.959 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-09-23T03:28:52.959 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-09-23T03:28:52.959 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-09-23T03:28:52.959 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi154 with status 1: "(cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo cat subdir/21')" 2023-09-23T03:28:52.959 INFO:tasks.cephfs_test_runner: 2023-09-23T03:28:52.959 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-09-23T03:28:52.959 INFO:tasks.cephfs_test_runner:Ran 5 tests in 746.811s 2023-09-23T03:28:52.959 INFO:tasks.cephfs_test_runner: 2023-09-23T03:28:52.960 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-09-23T03:28:52.960 INFO:tasks.cephfs_test_runner: 2023-09-23T03:28:52.960 INFO:tasks.cephfs_test_runner:====================================================================== 2023-09-23T03:28:52.960 INFO:tasks.cephfs_test_runner:ERROR: test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan) 2023-09-23T03:28:52.960 INFO:tasks.cephfs_test_runner:That when injecting a dentry into a fragmented directory, we put it in the right fragment. 2023-09-23T03:28:52.960 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-09-23T03:28:52.960 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-09-23T03:28:52.960 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_862663913a74a99e1c52d50e6a7e59dd6705749a/qa/tasks/cephfs/test_data_scan.py", line 560, in test_fragmented_injection 2023-09-23T03:28:52.960 INFO:tasks.cephfs_test_runner: out = self.mount_a.run_shell_payload(f"sudo cat subdir/{victim_dentry}", omit_sudo=False).stdout.getvalue().strip() 2023-09-23T03:28:52.960 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_862663913a74a99e1c52d50e6a7e59dd6705749a/qa/tasks/cephfs/mount.py", line 793, in run_shell_payload 2023-09-23T03:28:52.960 INFO:tasks.cephfs_test_runner: return self.run_shell(**kwargs) 2023-09-23T03:28:52.960 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_862663913a74a99e1c52d50e6a7e59dd6705749a/qa/tasks/cephfs/mount.py", line 784, in run_shell 2023-09-23T03:28:52.960 INFO:tasks.cephfs_test_runner: return self.client_remote.run(args=args, cwd=cwd, timeout=timeout, 2023-09-23T03:28:52.961 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run 2023-09-23T03:28:52.961 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-09-23T03:28:52.961 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run 2023-09-23T03:28:52.961 INFO:tasks.cephfs_test_runner: r.wait() 2023-09-23T03:28:52.961 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait 2023-09-23T03:28:52.961 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-09-23T03:28:52.961 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-09-23T03:28:52.961 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-09-23T03:28:52.961 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi154 with status 1: "(cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo cat subdir/21')" 2023-09-23T03:28:52.961 INFO:tasks.cephfs_test_runner: 2023-09-23T03:28:52.961 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_862663913a74a99e1c52d50e6a7e59dd6705749a/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan) 2023-09-23T03:28:53.049 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=def5a513a6ad4cbf831d15971c7dc327 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_862663913a74a99e1c52d50e6a7e59dd6705749a/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan) 2023-09-23T03:28:53.050 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-09-23T03:28:53.059 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-09-23T03:28:53.083 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-09-23T03:28:53.083 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:28:53.083 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:28:53.154 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053600,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-09-23T03:28:53.154 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2023-09-23T03:28:53.154 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:28:53.205 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053600,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-09-23T03:28:53.205 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:28:53.206 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:28:53.265 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053603,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248804k,nr_inodes=812201,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-09-23T03:28:53.265 DEBUG:teuthology.orchestra.run.smithi190:> set -ex 2023-09-23T03:28:53.266 DEBUG:teuthology.orchestra.run.smithi190:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-23T03:28:53.319 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053603,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248804k,nr_inodes=812201,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497612k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-09-23T03:28:53.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:53.320 DEBUG:teuthology.orchestra.run.smithi154:> ip netns list 2023-09-23T03:28:53.378 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-09-23T03:28:53.379 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-09-23T03:28:53.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:53.380 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-09-23T03:28:53.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:53.479 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-23T03:28:53.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:53.569 DEBUG:teuthology.orchestra.run.smithi154:> sudo ip link delete ceph-brx 2023-09-23T03:28:53.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:53.686 DEBUG:teuthology.orchestra.run.smithi190:> ip netns list 2023-09-23T03:28:53.747 INFO:teuthology.orchestra.run.smithi190.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-09-23T03:28:53.747 INFO:teuthology.orchestra.run.smithi190.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-09-23T03:28:53.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:53.748 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-09-23T03:28:53.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:53.848 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-09-23T03:28:53.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-23T03:28:53.935 DEBUG:teuthology.orchestra.run.smithi190:> sudo ip link delete ceph-brx 2023-09-23T03:28:54.051 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-09-23T03:28:54.060 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-09-23T03:28:54.061 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-23T03:28:54.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.290+0000 7f51e74ea640 1 Processor -- start 2023-09-23T03:28:54.291 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.290+0000 7f51e74ea640 1 -- start start 2023-09-23T03:28:54.292 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.292+0000 7f51e74ea640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f51e0071c90 0x7f51e0165500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:54.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.292+0000 7f51e74ea640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51e00cac40 0x7f51e00cb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:54.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.292+0000 7f51e74ea640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51e00c98b0 con 0x7f51e00c8ba0 2023-09-23T03:28:54.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.292+0000 7f51e74ea640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51e0161370 con 0x7f51e0071c90 2023-09-23T03:28:54.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.292+0000 7f51e74ea640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e00cb5d0 con 0x7f51e00cac40 2023-09-23T03:28:54.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.292+0000 7f51e64e8640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f51e0071c90 0x7f51e0165500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:54.293 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.292+0000 7f51e5ce7640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f51e00c8ba0 0x7f51e006fe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:49448/0 (socket says 172.21.15.154:49448) 2023-09-23T03:28:54.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.292+0000 7f51e5ce7640 1 -- 172.21.15.154:0/4082056 learned_addr learned my addr 172.21.15.154:0/4082056 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:28:54.294 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.292+0000 7f51e64e8640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f51e0071c90 0x7f51e0165500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44712/0 (socket says 172.21.15.154:44712) 2023-09-23T03:28:54.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.293+0000 7f51e6ce9640 1 --2- 172.21.15.154:0/4082056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51e00cac40 0x7f51e00cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:54.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.293+0000 7f51e64e8640 1 -- 172.21.15.154:0/4082056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51e00cac40 msgr2=0x7f51e00cb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:54.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.293+0000 7f51e64e8640 1 --2- 172.21.15.154:0/4082056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51e00cac40 0x7f51e00cb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:54.295 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.293+0000 7f51e64e8640 1 -- 172.21.15.154:0/4082056 >> v1:172.21.15.154:6789/0 conn(0x7f51e00c8ba0 legacy=0x7f51e006fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:54.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.293+0000 7f51e54e6640 1 -- 172.21.15.154:0/4082056 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2191900065 0 0) 0x7f51e00c98b0 con 0x7f51e00c8ba0 2023-09-23T03:28:54.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.293+0000 7f51e64e8640 1 -- 172.21.15.154:0/4082056 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e00c92f0 con 0x7f51e0071c90 2023-09-23T03:28:54.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.294+0000 7f51e64e8640 1 --2- 172.21.15.154:0/4082056 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51e0071c90 0x7f51e0165500 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f51d4009d90 tx=0x7f51d402bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38114c4d0f34334 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:54.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.295+0000 7f51e54e6640 1 -- 172.21.15.154:0/4082056 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d402cce0 con 0x7f51e0071c90 2023-09-23T03:28:54.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.295+0000 7f51e54e6640 1 -- 172.21.15.154:0/4082056 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d402ce40 con 0x7f51e0071c90 2023-09-23T03:28:54.296 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.295+0000 7f51e54e6640 1 -- 172.21.15.154:0/4082056 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d4034660 con 0x7f51e0071c90 2023-09-23T03:28:54.297 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.295+0000 7f51e74ea640 1 -- 172.21.15.154:0/4082056 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51e0071c90 msgr2=0x7f51e0165500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:54.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.295+0000 7f51e74ea640 1 --2- 172.21.15.154:0/4082056 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51e0071c90 0x7f51e0165500 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f51d4009d90 tx=0x7f51d402bf30 comp rx=0 tx=0).stop 2023-09-23T03:28:54.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.295+0000 7f51e74ea640 1 -- 172.21.15.154:0/4082056 shutdown_connections 2023-09-23T03:28:54.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.295+0000 7f51e74ea640 1 --2- 172.21.15.154:0/4082056 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51e00cac40 0x7f51e00cb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:54.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.295+0000 7f51e74ea640 1 --2- 172.21.15.154:0/4082056 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51e0071c90 0x7f51e0165500 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:54.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.295+0000 7f51e74ea640 1 -- 172.21.15.154:0/4082056 >> 172.21.15.154:0/4082056 conn(0x7f51e006a4c0 msgr2=0x7f51e00afe50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:54.298 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.295+0000 7f51e74ea640 1 -- 172.21.15.154:0/4082056 shutdown_connections 2023-09-23T03:28:54.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.295+0000 7f51e74ea640 1 -- 172.21.15.154:0/4082056 wait complete. 2023-09-23T03:28:54.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.297+0000 7f51e74ea640 1 Processor -- start 2023-09-23T03:28:54.299 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.297+0000 7f51e74ea640 1 -- start start 2023-09-23T03:28:54.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.298+0000 7f51e74ea640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51e0071c90 0x7f51e00c4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:54.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.298+0000 7f51e74ea640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51e00c8ba0 0x7f51e015cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:54.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.298+0000 7f51e74ea640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51e00cac40 0x7f51e00c2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:54.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.299+0000 7f51e74ea640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e00c6190 con 0x7f51e00cac40 2023-09-23T03:28:54.301 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.299+0000 7f51e74ea640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e00c24e0 con 0x7f51e00c8ba0 2023-09-23T03:28:54.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.299+0000 7f51e74ea640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e00c27c0 con 0x7f51e0071c90 2023-09-23T03:28:54.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.299+0000 7f51e6ce9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51e00cac40 0x7f51e00c2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:54.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.299+0000 7f51e6ce9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51e00cac40 0x7f51e00c2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44722/0 (socket says 172.21.15.154:44722) 2023-09-23T03:28:54.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.299+0000 7f51e6ce9640 1 -- 172.21.15.154:0/3089477937 learned_addr learned my addr 172.21.15.154:0/3089477937 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:54.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.299+0000 7f51e5ce7640 1 --2- 172.21.15.154:0/3089477937 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51e00c8ba0 0x7f51e015cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:54.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.299+0000 7f51e64e8640 1 --2- 172.21.15.154:0/3089477937 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51e0071c90 0x7f51e00c4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:54.302 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.299+0000 7f51e6ce9640 1 -- 172.21.15.154:0/3089477937 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51e0071c90 msgr2=0x7f51e00c4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:54.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.299+0000 7f51e6ce9640 1 --2- 172.21.15.154:0/3089477937 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51e0071c90 0x7f51e00c4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:54.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.299+0000 7f51e6ce9640 1 -- 172.21.15.154:0/3089477937 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51e00c8ba0 msgr2=0x7f51e015cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:54.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.299+0000 7f51e6ce9640 1 --2- 172.21.15.154:0/3089477937 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51e00c8ba0 0x7f51e015cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:54.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.299+0000 7f51e6ce9640 1 -- 172.21.15.154:0/3089477937 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d4009a70 con 0x7f51e00cac40 2023-09-23T03:28:54.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.299+0000 7f51e64e8640 1 --2- 172.21.15.154:0/3089477937 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51e0071c90 0x7f51e00c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:54.303 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.300+0000 7f51e6ce9640 1 --2- 172.21.15.154:0/3089477937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51e00cac40 0x7f51e00c2e00 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f51d800d800 tx=0x7f51d800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:54.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.300+0000 7f51deffd640 1 -- 172.21.15.154:0/3089477937 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d800b6c0 con 0x7f51e00cac40 2023-09-23T03:28:54.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.300+0000 7f51deffd640 1 -- 172.21.15.154:0/3089477937 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d800b840 con 0x7f51e00cac40 2023-09-23T03:28:54.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.300+0000 7f51deffd640 1 -- 172.21.15.154:0/3089477937 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d800bbf0 con 0x7f51e00cac40 2023-09-23T03:28:54.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.300+0000 7f51e74ea640 1 -- 172.21.15.154:0/3089477937 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51e00c2aa0 con 0x7f51e00cac40 2023-09-23T03:28:54.304 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.300+0000 7f51e74ea640 1 -- 172.21.15.154:0/3089477937 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e015d520 con 0x7f51e00cac40 2023-09-23T03:28:54.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.301+0000 7f51e74ea640 1 -- 172.21.15.154:0/3089477937 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51e0066730 con 0x7f51e00cac40 2023-09-23T03:28:54.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.302+0000 7f51deffd640 1 -- 172.21.15.154:0/3089477937 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f51d800bd50 con 0x7f51e00cac40 2023-09-23T03:28:54.305 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.303+0000 7f51deffd640 1 --2- 172.21.15.154:0/3089477937 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f51c406af30 0x7f51c406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:54.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.303+0000 7f51deffd640 1 -- 172.21.15.154:0/3089477937 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f51d8089cc0 con 0x7f51e00cac40 2023-09-23T03:28:54.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.303+0000 7f51e64e8640 1 --2- 172.21.15.154:0/3089477937 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f51c406af30 0x7f51c406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:54.306 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.304+0000 7f51e64e8640 1 --2- 172.21.15.154:0/3089477937 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f51c406af30 0x7f51c406d3f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f51d4036000 tx=0x7f51d4030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:54.307 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.307+0000 7f51deffd640 1 -- 172.21.15.154:0/3089477937 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f51d8090080 con 0x7f51e00cac40 2023-09-23T03:28:54.494 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.494+0000 7f51e74ea640 1 -- 172.21.15.154:0/3089477937 --> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f51e00cc3b0 con 0x7f51c406af30 2023-09-23T03:28:54.498 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.498+0000 7f51deffd640 1 -- 172.21.15.154:0/3089477937 <== mgr.4110 v2:172.21.15.154:6832/2422652104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48311 (secure 0 0 0) 0x7f51d4045090 con 0x7f51c406af30 2023-09-23T03:28:54.499 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:28:54.499 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2023-09-23T03:28:54.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.500+0000 7f51e74ea640 1 -- 172.21.15.154:0/3089477937 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f51c406af30 msgr2=0x7f51c406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:54.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.500+0000 7f51e74ea640 1 --2- 172.21.15.154:0/3089477937 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f51c406af30 0x7f51c406d3f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f51d4036000 tx=0x7f51d4030000 comp rx=0 tx=0).stop 2023-09-23T03:28:54.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.500+0000 7f51e74ea640 1 -- 172.21.15.154:0/3089477937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51e00cac40 msgr2=0x7f51e00c2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:54.501 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.500+0000 7f51e74ea640 1 --2- 172.21.15.154:0/3089477937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51e00cac40 0x7f51e00c2e00 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f51d800d800 tx=0x7f51d800dcc0 comp rx=0 tx=0).stop 2023-09-23T03:28:54.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.501+0000 7f51e74ea640 1 -- 172.21.15.154:0/3089477937 shutdown_connections 2023-09-23T03:28:54.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.501+0000 7f51e74ea640 1 --2- 172.21.15.154:0/3089477937 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f51c406af30 0x7f51c406d3f0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:54.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.501+0000 7f51e74ea640 1 --2- 172.21.15.154:0/3089477937 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f51e00cac40 0x7f51e00c2e00 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:54.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.501+0000 7f51e74ea640 1 --2- 172.21.15.154:0/3089477937 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f51e00c8ba0 0x7f51e015cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:54.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.501+0000 7f51e74ea640 1 --2- 172.21.15.154:0/3089477937 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f51e0071c90 0x7f51e00c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:54.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.501+0000 7f51e74ea640 1 -- 172.21.15.154:0/3089477937 >> 172.21.15.154:0/3089477937 conn(0x7f51e006a4c0 msgr2=0x7f51e0070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:54.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.501+0000 7f51e74ea640 1 -- 172.21.15.154:0/3089477937 shutdown_connections 2023-09-23T03:28:54.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.501+0000 7f51e74ea640 1 -- 172.21.15.154:0/3089477937 wait complete. 2023-09-23T03:28:54.519 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":536,"stamp":"2023-09-23T03:28:53.454653+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264680,"kb_used_data":15264,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754710040,"statfs":{"total":773094113280,"available":772823080960,"internally_reserved":0,"allocated":15630336,"data_stored":8230418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":255315514},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-404392,"num_objects":-123,"num_object_clones":0,"num_object_copies":-246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-123,"num_whiteouts":0,"num_read":-1808,"num_read_kb":-1513,"num_write":-913,"num_write_kb":-892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001923"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":260,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-23T03:28:29.522907+0000","last_change":"2023-09-23T03:15:11.042447+0000","last_active":"2023-09-23T03:28:29.522907+0000","last_peered":"2023-09-23T03:28:29.522907+0000","last_clean":"2023-09-23T03:28:29.522907+0000","last_became_active":"2023-09-23T03:15:11.040322+0000","last_became_peered":"2023-09-23T03:15:11.040322+0000","last_unstale":"2023-09-23T03:28:29.522907+0000","last_undegraded":"2023-09-23T03:28:29.522907+0000","last_fullsized":"2023-09-23T03:28:29.522907+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_clean_scrub_stamp":"2023-09-23T03:15:09.474842+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:18:36.331645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575014,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33436,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338404,"statfs":{"total":96636764160,"available":96602525696,"internally_reserved":0,"allocated":2314240,"data_stored":1370614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0649999999999999,"15min":1.026},"min":{"1min":0.67600000000000005,"5min":0.58199999999999996,"15min":0.442},"max":{"1min":1.5880000000000001,"5min":2.0299999999999998,"15min":2.149},"last":1.044},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0840000000000001,"15min":1.0640000000000001},"min":{"1min":0.77700000000000002,"5min":0.63100000000000001,"15min":0.52000000000000002},"max":{"1min":1.5009999999999999,"5min":1.9910000000000001,"15min":2.1480000000000001},"last":1.073}]},{"osd":1,"last update":"Sat Sep 23 03:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.2390000000000001,"15min":1.218},"min":{"1min":0.80800000000000005,"5min":0.69599999999999995,"15min":0.504},"max":{"1min":1.663,"5min":2.6400000000000001,"15min":3.1859999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":1.264,"5min":1.2769999999999999,"15min":1.284},"min":{"1min":0.80600000000000005,"5min":0.71299999999999997,"15min":0.67900000000000005},"max":{"1min":1.6120000000000001,"5min":2.3149999999999999,"15min":2.613},"last":0.66700000000000004}]},{"osd":2,"last update":"Sat Sep 23 03:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.143,"15min":1.125},"min":{"1min":0.67500000000000004,"5min":0.61199999999999999,"15min":0.42399999999999999},"max":{"1min":1.619,"5min":1.6439999999999999,"15min":2.331},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.206,"5min":1.262,"15min":1.2330000000000001},"min":{"1min":0.89400000000000002,"5min":0.79600000000000004,"15min":0.55200000000000005},"max":{"1min":1.538,"5min":2.2719999999999998,"15min":3.093},"last":0.749}]},{"osd":3,"last update":"Sat Sep 23 03:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.236,"15min":1.236},"min":{"1min":0.78700000000000003,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.7030000000000001,"5min":2.0840000000000001,"15min":2.532},"last":1.427},{"interface":"front","average":{"1min":1.343,"5min":1.381,"15min":1.361},"min":{"1min":0.99299999999999999,"5min":0.71899999999999997,"15min":0.69799999999999995},"max":{"1min":1.681,"5min":2.4289999999999998,"15min":3.2349999999999999},"last":1.3819999999999999}]},{"osd":4,"last update":"Sat Sep 23 03:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2869999999999999,"15min":1.349},"min":{"1min":0.85799999999999998,"5min":0.57999999999999996,"15min":0.45400000000000001},"max":{"1min":1.6259999999999999,"5min":2.3260000000000001,"15min":2.7210000000000001},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.446,"15min":1.4630000000000001},"min":{"1min":0.93500000000000005,"5min":0.73599999999999999,"15min":0.48099999999999998},"max":{"1min":1.8200000000000001,"5min":2.2240000000000002,"15min":3.7719999999999998},"last":1.5069999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.4079999999999999,"15min":1.405},"min":{"1min":0.86699999999999999,"5min":0.86499999999999999,"15min":0.47999999999999998},"max":{"1min":1.6180000000000001,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.407,"15min":1.429},"min":{"1min":1.002,"5min":0.77000000000000002,"15min":0.497},"max":{"1min":1.748,"5min":2.5070000000000001,"15min":2.5070000000000001},"last":1.472}]},{"osd":6,"last update":"Sat Sep 23 03:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.498,"15min":1.528},"min":{"1min":1.0449999999999999,"5min":0.85499999999999998,"15min":0.53800000000000003},"max":{"1min":1.8580000000000001,"5min":2.649,"15min":2.8420000000000001},"last":1.343},{"interface":"front","average":{"1min":1.401,"5min":1.4279999999999999,"15min":1.4670000000000001},"min":{"1min":1.1319999999999999,"5min":0.78900000000000003,"15min":0.45900000000000002},"max":{"1min":1.726,"5min":2.4500000000000002,"15min":3.6739999999999999},"last":1.5600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575014,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32844,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338996,"statfs":{"total":96636764160,"available":96603131904,"internally_reserved":0,"allocated":1708032,"data_stored":780052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:28:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.145,"15min":1.095},"min":{"1min":0.57199999999999995,"5min":0.53600000000000003,"15min":0.38},"max":{"1min":1.9830000000000001,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.081,"15min":1.0700000000000001},"min":{"1min":0.68600000000000005,"5min":0.60999999999999999,"15min":0.46100000000000002},"max":{"1min":1.4510000000000001,"5min":3.7629999999999999,"15min":3.7629999999999999},"last":1.6799999999999999}]},{"osd":1,"last update":"Sat Sep 23 03:28:26 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.157,"15min":1.125},"min":{"1min":0.66000000000000003,"5min":0.63800000000000001,"15min":0.438},"max":{"1min":1.5589999999999999,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.2090000000000001,"15min":1.194},"min":{"1min":0.96099999999999997,"5min":0.56599999999999995,"15min":0.54400000000000004},"max":{"1min":1.776,"5min":2.972,"15min":2.972},"last":0.69099999999999995}]},{"osd":2,"last update":"Sat Sep 23 03:28:26 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.1990000000000001,"15min":1.141},"min":{"1min":0.998,"5min":0.56499999999999995,"15min":0.51300000000000001},"max":{"1min":1.605,"5min":3.4390000000000001,"15min":3.4390000000000001},"last":1.569},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2869999999999999,"15min":1.2370000000000001},"min":{"1min":0.94499999999999995,"5min":0.68000000000000005,"15min":0.64300000000000002},"max":{"1min":1.625,"5min":2.177,"15min":2.177},"last":1.1359999999999999}]},{"osd":3,"last update":"Sat Sep 23 03:28:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.381,"15min":1.339},"min":{"1min":0.96099999999999997,"5min":0.78500000000000003,"15min":0.54600000000000004},"max":{"1min":1.6890000000000001,"5min":2.129,"15min":2.6509999999999998},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.339,"15min":1.365},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.74299999999999999},"max":{"1min":1.8899999999999999,"5min":3.0910000000000002,"15min":3.0910000000000002},"last":1.2829999999999999}]},{"osd":4,"last update":"Sat Sep 23 03:28:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.3360000000000001,"15min":1.365},"min":{"1min":0.91700000000000004,"5min":0.73299999999999998,"15min":0.498},"max":{"1min":2.0960000000000001,"5min":3.1789999999999998,"15min":3.1789999999999998},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3360000000000001,"15min":1.3009999999999999},"min":{"1min":0.92900000000000005,"5min":0.72399999999999998,"15min":0.56000000000000005},"max":{"1min":1.5760000000000001,"5min":3.6480000000000001,"15min":3.6480000000000001},"last":1.208}]},{"osd":5,"last update":"Sat Sep 23 03:28:26 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.4330000000000001,"15min":1.411},"min":{"1min":1.01,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.7150000000000001,"5min":4.093,"15min":4.093},"last":1.462},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.464,"15min":1.4419999999999999},"min":{"1min":1.0780000000000001,"5min":0.69899999999999995,"15min":0.59899999999999998},"max":{"1min":1.837,"5min":2.0459999999999998,"15min":4.758},"last":1.913}]},{"osd":7,"last update":"Sat Sep 23 03:28:26 2023","interfaces":[{"interface":"back","average":{"1min":1.633,"5min":1.55,"15min":1.524},"min":{"1min":1.423,"5min":0.76500000000000001,"15min":0.63500000000000001},"max":{"1min":2.1859999999999999,"5min":3.536,"15min":3.536},"last":1.371},{"interface":"front","average":{"1min":1.532,"5min":1.5569999999999999,"15min":1.5329999999999999},"min":{"1min":1.081,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":2.0190000000000001,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":1.76}]}]},{"osd":5,"up_from":13,"seq":55834575016,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32880,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338960,"statfs":{"total":96636764160,"available":96603095040,"internally_reserved":0,"allocated":1740800,"data_stored":786211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:28:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.087,"15min":1.048},"min":{"1min":0.64100000000000001,"5min":0.61199999999999999,"15min":0.57199999999999995},"max":{"1min":1.639,"5min":2.9740000000000002,"15min":2.9740000000000002},"last":1.421},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.1120000000000001,"15min":1.083},"min":{"1min":0.69599999999999995,"5min":0.624,"15min":0.61399999999999999},"max":{"1min":1.603,"5min":1.962,"15min":2.2320000000000002},"last":1.236}]},{"osd":1,"last update":"Sat Sep 23 03:28:27 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2050000000000001,"15min":1.218},"min":{"1min":0.73999999999999999,"5min":0.71899999999999997,"15min":0.62},"max":{"1min":1.657,"5min":2.4369999999999998,"15min":2.4369999999999998},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.268,"5min":1.294,"15min":1.2829999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67200000000000004},"max":{"1min":1.6120000000000001,"5min":3.5859999999999999,"15min":3.5859999999999999},"last":1.52}]},{"osd":2,"last update":"Sat Sep 23 03:28:27 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.2050000000000001,"15min":1.2010000000000001},"min":{"1min":0.875,"5min":0.65300000000000002,"15min":0.64300000000000002},"max":{"1min":1.76,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.1399999999999999,"15min":1.2050000000000001},"min":{"1min":0.71699999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.4370000000000001,"5min":2.8500000000000001,"15min":2.8500000000000001},"last":0.85299999999999998}]},{"osd":3,"last update":"Sat Sep 23 03:28:27 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2669999999999999,"15min":1.254},"min":{"1min":0.92800000000000005,"5min":0.67200000000000004,"15min":0.66400000000000003},"max":{"1min":1.6379999999999999,"5min":3.4359999999999999,"15min":3.4359999999999999},"last":1.329},{"interface":"front","average":{"1min":1.224,"5min":1.2969999999999999,"15min":1.339},"min":{"1min":0.91000000000000003,"5min":0.85999999999999999,"15min":0.69799999999999995},"max":{"1min":1.6719999999999999,"5min":1.9159999999999999,"15min":3.6110000000000002},"last":1.2909999999999999}]},{"osd":4,"last update":"Sat Sep 23 03:28:27 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.393,"15min":1.3899999999999999},"min":{"1min":0.92000000000000004,"5min":0.68600000000000005,"15min":0.54100000000000004},"max":{"1min":1.843,"5min":4.2750000000000004,"15min":4.2750000000000004},"last":1.585},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.3009999999999999,"15min":1.341},"min":{"1min":1.0429999999999999,"5min":0.54100000000000004,"15min":0.48899999999999999},"max":{"1min":1.4990000000000001,"5min":3.3090000000000002,"15min":3.3090000000000002},"last":1.1519999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:28:27 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.504,"15min":1.5049999999999999},"min":{"1min":1.1180000000000001,"5min":1.0329999999999999,"15min":0.59199999999999997},"max":{"1min":1.77,"5min":4.6040000000000001,"15min":4.6040000000000001},"last":1.484},{"interface":"front","average":{"1min":1.389,"5min":1.4159999999999999,"15min":1.5069999999999999},"min":{"1min":1.206,"5min":0.81200000000000006,"15min":0.55100000000000005},"max":{"1min":1.677,"5min":3.169,"15min":9.6560000000000006},"last":1.6419999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:28:27 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.4159999999999999,"15min":1.429},"min":{"1min":1.091,"5min":0.77100000000000002,"15min":0.626},"max":{"1min":1.708,"5min":2.077,"15min":2.077},"last":1.6890000000000001},{"interface":"front","average":{"1min":1.488,"5min":1.5009999999999999,"15min":1.5389999999999999},"min":{"1min":1.25,"5min":0.92200000000000004,"15min":0.60799999999999998},"max":{"1min":1.849,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":1.542}]}]},{"osd":4,"up_from":13,"seq":55834575015,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33228,"kb_used_data":2052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338612,"statfs":{"total":96636764160,"available":96602738688,"internally_reserved":0,"allocated":2101248,"data_stored":1182864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:28:24 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.091,"15min":1.1000000000000001},"min":{"1min":0.63800000000000001,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.5740000000000001,"5min":1.6699999999999999,"15min":1.9770000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.079,"5min":1.1599999999999999,"15min":1.167},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.7350000000000001,"5min":2.004,"15min":2.004},"last":1.506}]},{"osd":1,"last update":"Sat Sep 23 03:28:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.224,"15min":1.214},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.7490000000000001,"5min":1.8200000000000001,"15min":2.1110000000000002},"last":1.631},{"interface":"front","average":{"1min":1.022,"5min":1.0649999999999999,"15min":1.123},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60299999999999998},"max":{"1min":1.4890000000000001,"5min":1.718,"15min":3.2189999999999999},"last":1.1990000000000001}]},{"osd":2,"last update":"Sat Sep 23 03:28:24 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.1559999999999999,"15min":1.1599999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.512,"5min":1.694,"15min":2.5920000000000001},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.252,"5min":1.278,"15min":1.25},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68300000000000005},"max":{"1min":1.8839999999999999,"5min":1.99,"15min":1.99},"last":0.76800000000000002}]},{"osd":3,"last update":"Sat Sep 23 03:28:24 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.1699999999999999,"15min":1.1659999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4379999999999999,"5min":1.702,"15min":1.9450000000000001},"last":1.071},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2589999999999999,"15min":1.286},"min":{"1min":0.75900000000000001,"5min":0.69799999999999995,"15min":0.67400000000000004},"max":{"1min":1.611,"5min":1.833,"15min":2.4969999999999999},"last":1.4510000000000001}]},{"osd":5,"last update":"Sat Sep 23 03:28:24 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.3540000000000001,"15min":1.363},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.70199999999999996},"max":{"1min":1.879,"5min":2.0409999999999999,"15min":2.8050000000000002},"last":1.7330000000000001},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.2969999999999999,"15min":1.3240000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.619,"5min":1.8939999999999999,"15min":2.887},"last":1.2569999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:28:24 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3660000000000001,"15min":1.369},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.9530000000000001,"5min":1.9530000000000001,"15min":2.4249999999999998},"last":1.696},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.478,"15min":1.4930000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.9350000000000001,"5min":2.0939999999999999,"15min":2.1779999999999999},"last":1.8}]},{"osd":7,"last update":"Sat Sep 23 03:28:24 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.419,"15min":1.45},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.734,"5min":2.1389999999999998,"15min":2.5680000000000001},"last":1.377},{"interface":"front","average":{"1min":1.464,"5min":1.534,"15min":1.544},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.4769999999999999},"last":1.5740000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575015,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33260,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338580,"statfs":{"total":96636764160,"available":96602705920,"internally_reserved":0,"allocated":2134016,"data_stored":1182914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.81000000000000005,"15min":0.85099999999999998},"min":{"1min":0.52600000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.22,"5min":1.5009999999999999,"15min":2.952},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.91500000000000004,"15min":0.89500000000000002},"min":{"1min":0.41799999999999998,"5min":0.377,"15min":0.29299999999999998},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":0.84199999999999997}]},{"osd":1,"last update":"Sat Sep 23 03:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.81899999999999995,"15min":0.86699999999999999},"min":{"1min":0.40000000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.48,"5min":1.48,"15min":1.6839999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.98499999999999999,"15min":0.98399999999999999},"min":{"1min":0.48299999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.482,"5min":1.7010000000000001,"15min":2.903},"last":1.079}]},{"osd":2,"last update":"Sat Sep 23 03:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.004,"15min":1.0409999999999999},"min":{"1min":0.54000000000000004,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.524,"5min":1.669,"15min":2.2240000000000002},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.119,"15min":1.1080000000000001},"min":{"1min":0.68799999999999994,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.8,"5min":1.8,"15min":3.129},"last":0.80300000000000005}]},{"osd":4,"last update":"Sat Sep 23 03:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1439999999999999,"15min":1.2070000000000001},"min":{"1min":0.88200000000000001,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.7010000000000001,"5min":1.7609999999999999,"15min":2.698},"last":1.6160000000000001},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.264,"15min":1.327},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.752},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":2.419},"last":1.2709999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.282,"15min":1.3240000000000001},"min":{"1min":0.82499999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.9950000000000001},"last":1.772},{"interface":"front","average":{"1min":1.212,"5min":1.2090000000000001,"15min":1.266},"min":{"1min":0.80200000000000005,"5min":0.624,"15min":0.624},"max":{"1min":1.6799999999999999,"5min":1.9830000000000001,"15min":2.0070000000000001},"last":1.7989999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.284,"15min":1.337},"min":{"1min":0.77900000000000003,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.827,"5min":1.9530000000000001,"15min":2.6440000000000001},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.363,"5min":1.3720000000000001,"15min":1.419},"min":{"1min":0.86399999999999999,"5min":0.73299999999999998,"15min":0.67800000000000005},"max":{"1min":2.0880000000000001,"5min":2.177,"15min":2.9750000000000001},"last":1.5860000000000001}]},{"osd":7,"last update":"Sat Sep 23 03:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.286,"15min":1.357},"min":{"1min":0.81299999999999994,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.6279999999999999,"5min":2.0859999999999999,"15min":2.7879999999999998},"last":1.7170000000000001},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3520000000000001,"15min":1.4079999999999999},"min":{"1min":0.90300000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.7290000000000001,"5min":2.0270000000000001,"15min":2.5379999999999998},"last":1.454}]}]},{"osd":2,"up_from":13,"seq":55834575016,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32840,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339000,"statfs":{"total":96636764160,"available":96603136000,"internally_reserved":0,"allocated":1699840,"data_stored":779468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.81299999999999994,"15min":0.82899999999999996},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.30299999999999999},"max":{"1min":0.92400000000000004,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.91200000000000003,"15min":0.88200000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.375,"5min":2.613,"15min":2.613},"last":0.63100000000000001}]},{"osd":1,"last update":"Sat Sep 23 03:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.85399999999999998,"15min":0.85499999999999998},"min":{"1min":0.46600000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.391,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.006,"5min":1.032,"15min":0.98299999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.38},"max":{"1min":1.526,"5min":2.9809999999999999,"15min":2.9809999999999999},"last":1.3}]},{"osd":3,"last update":"Sat Sep 23 03:28:28 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.0740000000000001,"15min":1.006},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.40699999999999997},"max":{"1min":1.6279999999999999,"5min":1.698,"15min":1.698},"last":1.157},{"interface":"front","average":{"1min":0.96999999999999997,"5min":1.044,"15min":1.01},"min":{"1min":0.46800000000000003,"5min":0.441,"15min":0.34999999999999998},"max":{"1min":1.482,"5min":1.827,"15min":1.827},"last":0.94899999999999995}]},{"osd":4,"last update":"Sat Sep 23 03:28:28 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.3240000000000001,"15min":1.286},"min":{"1min":0.82899999999999996,"5min":0.79400000000000004,"15min":0.55800000000000005},"max":{"1min":1.5309999999999999,"5min":5.6669999999999998,"15min":5.6669999999999998},"last":1.121},{"interface":"front","average":{"1min":1.28,"5min":1.302,"15min":1.2949999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.7190000000000001,"5min":2.6909999999999998,"15min":2.6909999999999998},"last":1.3500000000000001}]},{"osd":5,"last update":"Sat Sep 23 03:28:28 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.3260000000000001,"15min":1.335},"min":{"1min":0.83399999999999996,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.6879999999999999,"5min":2.8969999999999998,"15min":3.9409999999999998},"last":1.405},{"interface":"front","average":{"1min":1.234,"5min":1.379,"15min":1.3959999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.56399999999999995},"max":{"1min":1.8109999999999999,"5min":3.415,"15min":4.9199999999999999},"last":1.464}]},{"osd":6,"last update":"Sat Sep 23 03:28:28 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.357,"15min":1.3580000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.755},"max":{"1min":1.7090000000000001,"5min":2.7839999999999998,"15min":3.0009999999999999},"last":1.518},{"interface":"front","average":{"1min":1.321,"5min":1.4570000000000001,"15min":1.4219999999999999},"min":{"1min":0.91800000000000004,"5min":0.76600000000000001,"15min":0.65100000000000002},"max":{"1min":1.851,"5min":7.8300000000000001,"15min":7.8300000000000001},"last":1.1990000000000001}]},{"osd":7,"last update":"Sat Sep 23 03:28:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.423,"15min":1.395},"min":{"1min":0.83099999999999996,"5min":0.82899999999999996,"15min":0.61399999999999999},"max":{"1min":1.954,"5min":4.0330000000000004,"15min":4.0330000000000004},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.4199999999999999,"15min":1.4239999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.74399999999999999},"max":{"1min":2.3140000000000001,"5min":4.2199999999999998,"15min":4.2199999999999998},"last":1.4430000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575017,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32796,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339044,"statfs":{"total":96636764160,"available":96603181056,"internally_reserved":0,"allocated":1658880,"data_stored":779467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:28:40 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.81899999999999995,"15min":0.80200000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.32300000000000001},"max":{"1min":1.8660000000000001,"5min":1.8660000000000001,"15min":2.8279999999999998},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88300000000000001,"15min":0.84899999999999998},"min":{"1min":0.40899999999999997,"5min":0.40600000000000003,"15min":0.312},"max":{"1min":2.2490000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":0.71999999999999997}]},{"osd":2,"last update":"Sat Sep 23 03:28:40 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.86499999999999999,"15min":0.86899999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.36699999999999999},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.3340000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":1.069,"5min":1.0580000000000001,"15min":1.0329999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.909,"5min":1.909,"15min":1.9570000000000001},"last":0.65500000000000003}]},{"osd":3,"last update":"Sat Sep 23 03:28:40 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.046,"15min":1.0329999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.47299999999999998},"max":{"1min":1.992,"5min":1.992,"15min":2.2370000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.141,"15min":1.109},"min":{"1min":0.495,"5min":0.495,"15min":0.45900000000000002},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":3.2130000000000001},"last":0.80100000000000005}]},{"osd":4,"last update":"Sat Sep 23 03:28:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.2789999999999999,"15min":1.2490000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.798,"5min":1.885,"15min":3.2629999999999999},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.266,"5min":1.244,"15min":1.2529999999999999},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66500000000000004},"max":{"1min":2.2690000000000001,"5min":2.2690000000000001,"15min":2.4689999999999999},"last":0.92300000000000004}]},{"osd":5,"last update":"Sat Sep 23 03:28:40 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.2929999999999999,"15min":1.2789999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.64600000000000002},"max":{"1min":2.024,"5min":2.024,"15min":3.161},"last":1.528},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3280000000000001,"15min":1.3260000000000001},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.72399999999999998},"max":{"1min":3.0830000000000002,"5min":3.0830000000000002,"15min":3.0830000000000002},"last":0.88600000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:28:40 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3640000000000001,"15min":1.329},"min":{"1min":0.81299999999999994,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":3.0179999999999998},"last":1.482},{"interface":"front","average":{"1min":1.296,"5min":1.363,"15min":1.363},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":2.1309999999999998,"5min":2.1309999999999998,"15min":3.3580000000000001},"last":0.96299999999999997}]},{"osd":7,"last update":"Sat Sep 23 03:28:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.337,"15min":1.3600000000000001},"min":{"1min":0.77600000000000002,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.946,"5min":1.946,"15min":3.1099999999999999},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4039999999999999,"15min":1.419},"min":{"1min":0.86699999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":3.1179999999999999,"5min":3.1179999999999999,"15min":3.306},"last":1.3879999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575017,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33396,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338444,"statfs":{"total":96636764160,"available":96602566656,"internally_reserved":0,"allocated":2273280,"data_stored":1368828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 23 03:28:30 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80900000000000005,"15min":0.83499999999999996},"min":{"1min":0.44700000000000001,"5min":0.38,"15min":0.30299999999999999},"max":{"1min":1.3799999999999999,"5min":1.383,"15min":1.919},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.88900000000000001,"15min":0.88800000000000001},"min":{"1min":0.36699999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.351,"5min":1.4470000000000001,"15min":2.1360000000000001},"last":0.96699999999999997}]},{"osd":2,"last update":"Sat Sep 23 03:28:30 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.873,"15min":0.85399999999999998},"min":{"1min":0.46899999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.335,"5min":1.5249999999999999,"15min":3.089},"last":0.749},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.97299999999999998,"15min":0.999},"min":{"1min":0.47699999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.4970000000000001,"5min":1.5760000000000001,"15min":2.633},"last":1.008}]},{"osd":3,"last update":"Sat Sep 23 03:28:30 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.98799999999999999,"15min":0.98099999999999998},"min":{"1min":0.5,"5min":0.433,"15min":0.433},"max":{"1min":1.417,"5min":1.615,"15min":2.8660000000000001},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.03,"15min":1.018},"min":{"1min":0.505,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.446,"5min":1.802,"15min":2.012},"last":0.70499999999999996}]},{"osd":4,"last update":"Sat Sep 23 03:28:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.252,"15min":1.2609999999999999},"min":{"1min":0.73599999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.621,"5min":1.885,"15min":2.4140000000000001},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2270000000000001,"15min":1.2609999999999999},"min":{"1min":0.83199999999999996,"5min":0.73999999999999999,"15min":0.65300000000000002},"max":{"1min":1.5309999999999999,"5min":1.845,"15min":3.0230000000000001},"last":1.506}]},{"osd":5,"last update":"Sat Sep 23 03:28:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.296,"15min":1.274},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.65600000000000003},"max":{"1min":1.9530000000000001,"5min":2.1240000000000001,"15min":2.5489999999999999},"last":1.6890000000000001},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3320000000000001,"15min":1.3380000000000001},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.7210000000000001,"5min":1.9159999999999999,"15min":2.1539999999999999},"last":1.2729999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:28:30 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3149999999999999,"15min":1.325},"min":{"1min":0.78900000000000003,"5min":0.77700000000000002,"15min":0.75600000000000001},"max":{"1min":1.8600000000000001,"5min":2.0870000000000002,"15min":2.9489999999999998},"last":1.5660000000000001},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.3839999999999999,"15min":1.3859999999999999},"min":{"1min":0.71199999999999997,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.8100000000000001,"5min":1.9670000000000001,"15min":2.3239999999999998},"last":1.151}]},{"osd":7,"last update":"Sat Sep 23 03:28:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.3420000000000001,"15min":1.3640000000000001},"min":{"1min":0.93200000000000005,"5min":0.752,"15min":0.747},"max":{"1min":1.722,"5min":2.052,"15min":2.2330000000000001},"last":1.8580000000000001},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.405,"15min":1.4359999999999999},"min":{"1min":1.0960000000000001,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.7889999999999999,"5min":2.1419999999999999,"15min":2.3330000000000002},"last":1.798}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-23T03:28:54.521 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-23T03:28:54.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.734+0000 7f3da5114640 1 Processor -- start 2023-09-23T03:28:54.734 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.734+0000 7f3da5114640 1 -- start start 2023-09-23T03:28:54.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.735+0000 7f3da5114640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f3da00cb320 0x7f3da00c9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:54.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.735+0000 7f3da5114640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3da00ca3c0 0x7f3da0070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:54.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.735+0000 7f3da5114640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3da015a5f0 con 0x7f3da0159ba0 2023-09-23T03:28:54.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.735+0000 7f3da5114640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3da015abb0 con 0x7f3da00cb320 2023-09-23T03:28:54.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.735+0000 7f3da5114640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3da015a8d0 con 0x7f3da00ca3c0 2023-09-23T03:28:54.736 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.736+0000 7f3d97fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f3da0159ba0 0x7f3da00c9cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:49460/0 (socket says 172.21.15.154:49460) 2023-09-23T03:28:54.737 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.736+0000 7f3d97fff640 1 -- 172.21.15.154:0/2966912099 learned_addr learned my addr 172.21.15.154:0/2966912099 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:28:54.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.736+0000 7f3d9ffff640 1 --2- 172.21.15.154:0/2966912099 >> v2:172.21.15.190:3300/0 conn(0x7f3da00cb320 0x7f3da00c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:54.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.736+0000 7f3d9f7fe640 1 -- 172.21.15.154:0/2966912099 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1504498261 0 0) 0x7f3da015a5f0 con 0x7f3da0159ba0 2023-09-23T03:28:54.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.736+0000 7f3d9f7fe640 1 -- 172.21.15.154:0/2966912099 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d8c005730 con 0x7f3da0159ba0 2023-09-23T03:28:54.739 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.736+0000 7f3da4913640 1 --2- 172.21.15.154:0/2966912099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3da00ca3c0 0x7f3da0070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:54.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.736+0000 7f3d9f7fe640 1 -- 172.21.15.154:0/2966912099 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1558724222 0 0) 0x7f3d8c005730 con 0x7f3da0159ba0 2023-09-23T03:28:54.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.736+0000 7f3d9f7fe640 1 -- 172.21.15.154:0/2966912099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3da00ca3c0 msgr2=0x7f3da0070ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:54.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.736+0000 7f3d9f7fe640 1 --2- 172.21.15.154:0/2966912099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3da00ca3c0 0x7f3da0070ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:54.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.736+0000 7f3d9f7fe640 1 -- 172.21.15.154:0/2966912099 >> v2:172.21.15.190:3300/0 conn(0x7f3da00cb320 msgr2=0x7f3da00c9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:54.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.736+0000 7f3d9f7fe640 1 --2- 172.21.15.154:0/2966912099 >> v2:172.21.15.190:3300/0 conn(0x7f3da00cb320 0x7f3da00c9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:54.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.736+0000 7f3d9f7fe640 1 -- 172.21.15.154:0/2966912099 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da00c8b30 con 0x7f3da0159ba0 2023-09-23T03:28:54.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.737+0000 7f3d9f7fe640 1 -- 172.21.15.154:0/2966912099 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f3d88001670 con 0x7f3da0159ba0 2023-09-23T03:28:54.740 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.737+0000 7f3d9f7fe640 1 -- 172.21.15.154:0/2966912099 >> v1:172.21.15.154:6789/0 conn(0x7f3da0159ba0 legacy=0x7f3da00c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:54.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.737+0000 7f3d9f7fe640 1 --2- 172.21.15.154:0/2966912099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d8c005b80 0x7f3d8c005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:54.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.737+0000 7f3d9f7fe640 1 --2- 172.21.15.154:0/2966912099 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d8c0069d0 0x7f3d8c006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:54.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.737+0000 7f3d9f7fe640 1 --2- 172.21.15.154:0/2966912099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3da00ca3c0 0x7f3d8c007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:54.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.737+0000 7f3d9f7fe640 1 -- 172.21.15.154:0/2966912099 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da015abb0 con 0x7f3d8c0069d0 2023-09-23T03:28:54.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.737+0000 7f3d9f7fe640 1 -- 172.21.15.154:0/2966912099 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da015a8d0 con 0x7f3d8c005b80 2023-09-23T03:28:54.741 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.737+0000 7f3d9f7fe640 1 -- 172.21.15.154:0/2966912099 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8c00dd20 con 0x7f3da00ca3c0 2023-09-23T03:28:54.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.737+0000 7f3d97fff640 1 --2- 172.21.15.154:0/2966912099 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d8c0069d0 0x7f3d8c006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:54.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.737+0000 7f3d9ffff640 1 --2- 172.21.15.154:0/2966912099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d8c005b80 0x7f3d8c005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:54.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.737+0000 7f3da4913640 1 --2- 172.21.15.154:0/2966912099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3da00ca3c0 0x7f3d8c007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:54.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.737+0000 7f3d97fff640 1 -- 172.21.15.154:0/2966912099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3da00ca3c0 msgr2=0x7f3d8c007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:54.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.737+0000 7f3d97fff640 1 --2- 172.21.15.154:0/2966912099 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3da00ca3c0 0x7f3d8c007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:54.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.737+0000 7f3d97fff640 1 -- 172.21.15.154:0/2966912099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d8c005b80 msgr2=0x7f3d8c005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:54.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.737+0000 7f3d97fff640 1 --2- 172.21.15.154:0/2966912099 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3d8c005b80 0x7f3d8c005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:54.742 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.737+0000 7f3d97fff640 1 -- 172.21.15.154:0/2966912099 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d8c005730 con 0x7f3d8c0069d0 2023-09-23T03:28:54.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.739+0000 7f3d97fff640 1 --2- 172.21.15.154:0/2966912099 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d8c0069d0 0x7f3d8c006e50 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f3d8800c210 tx=0x7f3d8c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=929d98f806d7819f server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:54.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.739+0000 7f3d9f7fe640 1 -- 172.21.15.154:0/2966912099 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d88005ad0 con 0x7f3d8c0069d0 2023-09-23T03:28:54.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.739+0000 7f3d9f7fe640 1 -- 172.21.15.154:0/2966912099 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d88005c30 con 0x7f3d8c0069d0 2023-09-23T03:28:54.743 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.739+0000 7f3d9f7fe640 1 -- 172.21.15.154:0/2966912099 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8802c8a0 con 0x7f3d8c0069d0 2023-09-23T03:28:54.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.739+0000 7f3da5114640 1 -- 172.21.15.154:0/2966912099 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d8c0069d0 msgr2=0x7f3d8c006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:54.745 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.739+0000 7f3da5114640 1 --2- 172.21.15.154:0/2966912099 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d8c0069d0 0x7f3d8c006e50 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f3d8800c210 tx=0x7f3d8c007a00 comp rx=0 tx=0).stop 2023-09-23T03:28:54.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.739+0000 7f3da4913640 1 -- 172.21.15.154:0/2966912099 reap_dead start 2023-09-23T03:28:54.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.740+0000 7f3da5114640 1 -- 172.21.15.154:0/2966912099 shutdown_connections 2023-09-23T03:28:54.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.740+0000 7f3da5114640 1 -- 172.21.15.154:0/2966912099 >> 172.21.15.154:0/2966912099 conn(0x7f3da006a4c0 msgr2=0x7f3da00afe50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:54.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.740+0000 7f3da5114640 1 -- 172.21.15.154:0/2966912099 shutdown_connections 2023-09-23T03:28:54.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.740+0000 7f3da5114640 1 -- 172.21.15.154:0/2966912099 wait complete. 2023-09-23T03:28:54.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.741+0000 7f3da5114640 1 Processor -- start 2023-09-23T03:28:54.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.742+0000 7f3da5114640 1 -- start start 2023-09-23T03:28:54.746 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.743+0000 7f3da5114640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d8c0069d0 0x7f3da00c6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:54.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.743+0000 7f3da5114640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3da00b29c0 0x7f3da00c7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:54.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.743+0000 7f3da5114640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3da00c69f0 0x7f3da0154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:54.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.743+0000 7f3da5114640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da00b39d0 con 0x7f3da00b29c0 2023-09-23T03:28:54.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.743+0000 7f3da5114640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da01ed8d0 con 0x7f3da00c69f0 2023-09-23T03:28:54.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.743+0000 7f3da5114640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3da00b2e50 con 0x7f3d8c0069d0 2023-09-23T03:28:54.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.743+0000 7f3d97fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3da00b29c0 0x7f3da00c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:54.747 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.743+0000 7f3d97fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3da00b29c0 0x7f3da00c7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44740/0 (socket says 172.21.15.154:44740) 2023-09-23T03:28:54.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.743+0000 7f3d97fff640 1 -- 172.21.15.154:0/2100296293 learned_addr learned my addr 172.21.15.154:0/2100296293 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:54.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.743+0000 7f3da4913640 1 --2- 172.21.15.154:0/2100296293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3da00c69f0 0x7f3da0154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:54.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.743+0000 7f3d9ffff640 1 --2- 172.21.15.154:0/2100296293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d8c0069d0 0x7f3da00c6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:54.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.743+0000 7f3d97fff640 1 -- 172.21.15.154:0/2100296293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d8c0069d0 msgr2=0x7f3da00c6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:54.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.743+0000 7f3d97fff640 1 --2- 172.21.15.154:0/2100296293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d8c0069d0 0x7f3da00c6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:54.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.743+0000 7f3d97fff640 1 -- 172.21.15.154:0/2100296293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3da00c69f0 msgr2=0x7f3da0154f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:54.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.743+0000 7f3d97fff640 1 --2- 172.21.15.154:0/2100296293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3da00c69f0 0x7f3da0154f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:54.748 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.743+0000 7f3d97fff640 1 -- 172.21.15.154:0/2100296293 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da00b3130 con 0x7f3da00b29c0 2023-09-23T03:28:54.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.743+0000 7f3da4913640 1 --2- 172.21.15.154:0/2100296293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3da00c69f0 0x7f3da0154f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:54.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.744+0000 7f3d97fff640 1 --2- 172.21.15.154:0/2100296293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3da00b29c0 0x7f3da00c7380 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f3d8802c440 tx=0x7f3d88001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:54.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.744+0000 7f3d9d7fa640 1 -- 172.21.15.154:0/2100296293 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d88039070 con 0x7f3da00b29c0 2023-09-23T03:28:54.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.744+0000 7f3d9d7fa640 1 -- 172.21.15.154:0/2100296293 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d8802ccb0 con 0x7f3da00b29c0 2023-09-23T03:28:54.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.744+0000 7f3d9d7fa640 1 -- 172.21.15.154:0/2100296293 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8803c040 con 0x7f3da00b29c0 2023-09-23T03:28:54.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.744+0000 7f3da5114640 1 -- 172.21.15.154:0/2100296293 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3da00b3410 con 0x7f3da00b29c0 2023-09-23T03:28:54.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.744+0000 7f3da5114640 1 -- 172.21.15.154:0/2100296293 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3da01f0bb0 con 0x7f3da00b29c0 2023-09-23T03:28:54.749 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.745+0000 7f3da5114640 1 -- 172.21.15.154:0/2100296293 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3da0066730 con 0x7f3da00b29c0 2023-09-23T03:28:54.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.746+0000 7f3d9d7fa640 1 -- 172.21.15.154:0/2100296293 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f3d88032070 con 0x7f3da00b29c0 2023-09-23T03:28:54.750 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.747+0000 7f3d9d7fa640 1 --2- 172.21.15.154:0/2100296293 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3d8006af30 0x7f3d8006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:54.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.747+0000 7f3d9d7fa640 1 -- 172.21.15.154:0/2100296293 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f3d880aaa70 con 0x7f3da00b29c0 2023-09-23T03:28:54.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.747+0000 7f3d9ffff640 1 --2- 172.21.15.154:0/2100296293 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3d8006af30 0x7f3d8006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:54.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.748+0000 7f3d9ffff640 1 --2- 172.21.15.154:0/2100296293 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3d8006af30 0x7f3d8006d3f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f3d90006fd0 tx=0x7f3d90008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:54.751 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.751+0000 7f3d9d7fa640 1 -- 172.21.15.154:0/2100296293 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f3d88075bb0 con 0x7f3da00b29c0 2023-09-23T03:28:54.937 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.937+0000 7f3da5114640 1 -- 172.21.15.154:0/2100296293 --> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3da00c8e10 con 0x7f3d8006af30 2023-09-23T03:28:54.939 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.939+0000 7f3d9d7fa640 1 -- 172.21.15.154:0/2100296293 <== mgr.4110 v2:172.21.15.154:6832/2422652104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48311 (secure 0 0 0) 0x7f3d9000b560 con 0x7f3d8006af30 2023-09-23T03:28:54.940 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:28:54.941 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2023-09-23T03:28:54.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.942+0000 7f3da5114640 1 -- 172.21.15.154:0/2100296293 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3d8006af30 msgr2=0x7f3d8006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:54.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.942+0000 7f3da5114640 1 --2- 172.21.15.154:0/2100296293 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3d8006af30 0x7f3d8006d3f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f3d90006fd0 tx=0x7f3d90008040 comp rx=0 tx=0).stop 2023-09-23T03:28:54.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.942+0000 7f3da5114640 1 -- 172.21.15.154:0/2100296293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3da00b29c0 msgr2=0x7f3da00c7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:54.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.942+0000 7f3da5114640 1 --2- 172.21.15.154:0/2100296293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3da00b29c0 0x7f3da00c7380 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f3d8802c440 tx=0x7f3d88001a00 comp rx=0 tx=0).stop 2023-09-23T03:28:54.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.942+0000 7f3da5114640 1 -- 172.21.15.154:0/2100296293 shutdown_connections 2023-09-23T03:28:54.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.942+0000 7f3da5114640 1 --2- 172.21.15.154:0/2100296293 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f3d8006af30 0x7f3d8006d3f0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:54.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.942+0000 7f3da5114640 1 --2- 172.21.15.154:0/2100296293 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f3da00c69f0 0x7f3da0154f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:54.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.942+0000 7f3da5114640 1 --2- 172.21.15.154:0/2100296293 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3da00b29c0 0x7f3da00c7380 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:54.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.942+0000 7f3da5114640 1 --2- 172.21.15.154:0/2100296293 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f3d8c0069d0 0x7f3da00c6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:54.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.942+0000 7f3da5114640 1 -- 172.21.15.154:0/2100296293 >> 172.21.15.154:0/2100296293 conn(0x7f3da006a4c0 msgr2=0x7f3da00ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:54.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.942+0000 7f3da5114640 1 -- 172.21.15.154:0/2100296293 shutdown_connections 2023-09-23T03:28:54.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:54.942+0000 7f3da5114640 1 -- 172.21.15.154:0/2100296293 wait complete. 2023-09-23T03:28:54.962 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":536,"stamp":"2023-09-23T03:28:53.454653+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264680,"kb_used_data":15264,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754710040,"statfs":{"total":773094113280,"available":772823080960,"internally_reserved":0,"allocated":15630336,"data_stored":8230418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":255315514},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-404392,"num_objects":-123,"num_object_clones":0,"num_object_copies":-246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-123,"num_whiteouts":0,"num_read":-1808,"num_read_kb":-1513,"num_write":-913,"num_write_kb":-892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001923"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":260,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-23T03:28:29.522907+0000","last_change":"2023-09-23T03:15:11.042447+0000","last_active":"2023-09-23T03:28:29.522907+0000","last_peered":"2023-09-23T03:28:29.522907+0000","last_clean":"2023-09-23T03:28:29.522907+0000","last_became_active":"2023-09-23T03:15:11.040322+0000","last_became_peered":"2023-09-23T03:15:11.040322+0000","last_unstale":"2023-09-23T03:28:29.522907+0000","last_undegraded":"2023-09-23T03:28:29.522907+0000","last_fullsized":"2023-09-23T03:28:29.522907+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_clean_scrub_stamp":"2023-09-23T03:15:09.474842+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:18:36.331645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575014,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33436,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338404,"statfs":{"total":96636764160,"available":96602525696,"internally_reserved":0,"allocated":2314240,"data_stored":1370614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0649999999999999,"15min":1.026},"min":{"1min":0.67600000000000005,"5min":0.58199999999999996,"15min":0.442},"max":{"1min":1.5880000000000001,"5min":2.0299999999999998,"15min":2.149},"last":1.044},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0840000000000001,"15min":1.0640000000000001},"min":{"1min":0.77700000000000002,"5min":0.63100000000000001,"15min":0.52000000000000002},"max":{"1min":1.5009999999999999,"5min":1.9910000000000001,"15min":2.1480000000000001},"last":1.073}]},{"osd":1,"last update":"Sat Sep 23 03:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.2390000000000001,"15min":1.218},"min":{"1min":0.80800000000000005,"5min":0.69599999999999995,"15min":0.504},"max":{"1min":1.663,"5min":2.6400000000000001,"15min":3.1859999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":1.264,"5min":1.2769999999999999,"15min":1.284},"min":{"1min":0.80600000000000005,"5min":0.71299999999999997,"15min":0.67900000000000005},"max":{"1min":1.6120000000000001,"5min":2.3149999999999999,"15min":2.613},"last":0.66700000000000004}]},{"osd":2,"last update":"Sat Sep 23 03:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.143,"15min":1.125},"min":{"1min":0.67500000000000004,"5min":0.61199999999999999,"15min":0.42399999999999999},"max":{"1min":1.619,"5min":1.6439999999999999,"15min":2.331},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.206,"5min":1.262,"15min":1.2330000000000001},"min":{"1min":0.89400000000000002,"5min":0.79600000000000004,"15min":0.55200000000000005},"max":{"1min":1.538,"5min":2.2719999999999998,"15min":3.093},"last":0.749}]},{"osd":3,"last update":"Sat Sep 23 03:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.236,"15min":1.236},"min":{"1min":0.78700000000000003,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.7030000000000001,"5min":2.0840000000000001,"15min":2.532},"last":1.427},{"interface":"front","average":{"1min":1.343,"5min":1.381,"15min":1.361},"min":{"1min":0.99299999999999999,"5min":0.71899999999999997,"15min":0.69799999999999995},"max":{"1min":1.681,"5min":2.4289999999999998,"15min":3.2349999999999999},"last":1.3819999999999999}]},{"osd":4,"last update":"Sat Sep 23 03:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2869999999999999,"15min":1.349},"min":{"1min":0.85799999999999998,"5min":0.57999999999999996,"15min":0.45400000000000001},"max":{"1min":1.6259999999999999,"5min":2.3260000000000001,"15min":2.7210000000000001},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.446,"15min":1.4630000000000001},"min":{"1min":0.93500000000000005,"5min":0.73599999999999999,"15min":0.48099999999999998},"max":{"1min":1.8200000000000001,"5min":2.2240000000000002,"15min":3.7719999999999998},"last":1.5069999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.4079999999999999,"15min":1.405},"min":{"1min":0.86699999999999999,"5min":0.86499999999999999,"15min":0.47999999999999998},"max":{"1min":1.6180000000000001,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.407,"15min":1.429},"min":{"1min":1.002,"5min":0.77000000000000002,"15min":0.497},"max":{"1min":1.748,"5min":2.5070000000000001,"15min":2.5070000000000001},"last":1.472}]},{"osd":6,"last update":"Sat Sep 23 03:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.498,"15min":1.528},"min":{"1min":1.0449999999999999,"5min":0.85499999999999998,"15min":0.53800000000000003},"max":{"1min":1.8580000000000001,"5min":2.649,"15min":2.8420000000000001},"last":1.343},{"interface":"front","average":{"1min":1.401,"5min":1.4279999999999999,"15min":1.4670000000000001},"min":{"1min":1.1319999999999999,"5min":0.78900000000000003,"15min":0.45900000000000002},"max":{"1min":1.726,"5min":2.4500000000000002,"15min":3.6739999999999999},"last":1.5600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575014,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32844,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338996,"statfs":{"total":96636764160,"available":96603131904,"internally_reserved":0,"allocated":1708032,"data_stored":780052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:28:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.145,"15min":1.095},"min":{"1min":0.57199999999999995,"5min":0.53600000000000003,"15min":0.38},"max":{"1min":1.9830000000000001,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.081,"15min":1.0700000000000001},"min":{"1min":0.68600000000000005,"5min":0.60999999999999999,"15min":0.46100000000000002},"max":{"1min":1.4510000000000001,"5min":3.7629999999999999,"15min":3.7629999999999999},"last":1.6799999999999999}]},{"osd":1,"last update":"Sat Sep 23 03:28:26 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.157,"15min":1.125},"min":{"1min":0.66000000000000003,"5min":0.63800000000000001,"15min":0.438},"max":{"1min":1.5589999999999999,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.2090000000000001,"15min":1.194},"min":{"1min":0.96099999999999997,"5min":0.56599999999999995,"15min":0.54400000000000004},"max":{"1min":1.776,"5min":2.972,"15min":2.972},"last":0.69099999999999995}]},{"osd":2,"last update":"Sat Sep 23 03:28:26 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.1990000000000001,"15min":1.141},"min":{"1min":0.998,"5min":0.56499999999999995,"15min":0.51300000000000001},"max":{"1min":1.605,"5min":3.4390000000000001,"15min":3.4390000000000001},"last":1.569},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2869999999999999,"15min":1.2370000000000001},"min":{"1min":0.94499999999999995,"5min":0.68000000000000005,"15min":0.64300000000000002},"max":{"1min":1.625,"5min":2.177,"15min":2.177},"last":1.1359999999999999}]},{"osd":3,"last update":"Sat Sep 23 03:28:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.381,"15min":1.339},"min":{"1min":0.96099999999999997,"5min":0.78500000000000003,"15min":0.54600000000000004},"max":{"1min":1.6890000000000001,"5min":2.129,"15min":2.6509999999999998},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.339,"15min":1.365},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.74299999999999999},"max":{"1min":1.8899999999999999,"5min":3.0910000000000002,"15min":3.0910000000000002},"last":1.2829999999999999}]},{"osd":4,"last update":"Sat Sep 23 03:28:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.3360000000000001,"15min":1.365},"min":{"1min":0.91700000000000004,"5min":0.73299999999999998,"15min":0.498},"max":{"1min":2.0960000000000001,"5min":3.1789999999999998,"15min":3.1789999999999998},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3360000000000001,"15min":1.3009999999999999},"min":{"1min":0.92900000000000005,"5min":0.72399999999999998,"15min":0.56000000000000005},"max":{"1min":1.5760000000000001,"5min":3.6480000000000001,"15min":3.6480000000000001},"last":1.208}]},{"osd":5,"last update":"Sat Sep 23 03:28:26 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.4330000000000001,"15min":1.411},"min":{"1min":1.01,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.7150000000000001,"5min":4.093,"15min":4.093},"last":1.462},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.464,"15min":1.4419999999999999},"min":{"1min":1.0780000000000001,"5min":0.69899999999999995,"15min":0.59899999999999998},"max":{"1min":1.837,"5min":2.0459999999999998,"15min":4.758},"last":1.913}]},{"osd":7,"last update":"Sat Sep 23 03:28:26 2023","interfaces":[{"interface":"back","average":{"1min":1.633,"5min":1.55,"15min":1.524},"min":{"1min":1.423,"5min":0.76500000000000001,"15min":0.63500000000000001},"max":{"1min":2.1859999999999999,"5min":3.536,"15min":3.536},"last":1.371},{"interface":"front","average":{"1min":1.532,"5min":1.5569999999999999,"15min":1.5329999999999999},"min":{"1min":1.081,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":2.0190000000000001,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":1.76}]}]},{"osd":5,"up_from":13,"seq":55834575016,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32880,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338960,"statfs":{"total":96636764160,"available":96603095040,"internally_reserved":0,"allocated":1740800,"data_stored":786211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:28:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.087,"15min":1.048},"min":{"1min":0.64100000000000001,"5min":0.61199999999999999,"15min":0.57199999999999995},"max":{"1min":1.639,"5min":2.9740000000000002,"15min":2.9740000000000002},"last":1.421},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.1120000000000001,"15min":1.083},"min":{"1min":0.69599999999999995,"5min":0.624,"15min":0.61399999999999999},"max":{"1min":1.603,"5min":1.962,"15min":2.2320000000000002},"last":1.236}]},{"osd":1,"last update":"Sat Sep 23 03:28:27 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2050000000000001,"15min":1.218},"min":{"1min":0.73999999999999999,"5min":0.71899999999999997,"15min":0.62},"max":{"1min":1.657,"5min":2.4369999999999998,"15min":2.4369999999999998},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.268,"5min":1.294,"15min":1.2829999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67200000000000004},"max":{"1min":1.6120000000000001,"5min":3.5859999999999999,"15min":3.5859999999999999},"last":1.52}]},{"osd":2,"last update":"Sat Sep 23 03:28:27 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.2050000000000001,"15min":1.2010000000000001},"min":{"1min":0.875,"5min":0.65300000000000002,"15min":0.64300000000000002},"max":{"1min":1.76,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.1399999999999999,"15min":1.2050000000000001},"min":{"1min":0.71699999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.4370000000000001,"5min":2.8500000000000001,"15min":2.8500000000000001},"last":0.85299999999999998}]},{"osd":3,"last update":"Sat Sep 23 03:28:27 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2669999999999999,"15min":1.254},"min":{"1min":0.92800000000000005,"5min":0.67200000000000004,"15min":0.66400000000000003},"max":{"1min":1.6379999999999999,"5min":3.4359999999999999,"15min":3.4359999999999999},"last":1.329},{"interface":"front","average":{"1min":1.224,"5min":1.2969999999999999,"15min":1.339},"min":{"1min":0.91000000000000003,"5min":0.85999999999999999,"15min":0.69799999999999995},"max":{"1min":1.6719999999999999,"5min":1.9159999999999999,"15min":3.6110000000000002},"last":1.2909999999999999}]},{"osd":4,"last update":"Sat Sep 23 03:28:27 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.393,"15min":1.3899999999999999},"min":{"1min":0.92000000000000004,"5min":0.68600000000000005,"15min":0.54100000000000004},"max":{"1min":1.843,"5min":4.2750000000000004,"15min":4.2750000000000004},"last":1.585},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.3009999999999999,"15min":1.341},"min":{"1min":1.0429999999999999,"5min":0.54100000000000004,"15min":0.48899999999999999},"max":{"1min":1.4990000000000001,"5min":3.3090000000000002,"15min":3.3090000000000002},"last":1.1519999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:28:27 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.504,"15min":1.5049999999999999},"min":{"1min":1.1180000000000001,"5min":1.0329999999999999,"15min":0.59199999999999997},"max":{"1min":1.77,"5min":4.6040000000000001,"15min":4.6040000000000001},"last":1.484},{"interface":"front","average":{"1min":1.389,"5min":1.4159999999999999,"15min":1.5069999999999999},"min":{"1min":1.206,"5min":0.81200000000000006,"15min":0.55100000000000005},"max":{"1min":1.677,"5min":3.169,"15min":9.6560000000000006},"last":1.6419999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:28:27 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.4159999999999999,"15min":1.429},"min":{"1min":1.091,"5min":0.77100000000000002,"15min":0.626},"max":{"1min":1.708,"5min":2.077,"15min":2.077},"last":1.6890000000000001},{"interface":"front","average":{"1min":1.488,"5min":1.5009999999999999,"15min":1.5389999999999999},"min":{"1min":1.25,"5min":0.92200000000000004,"15min":0.60799999999999998},"max":{"1min":1.849,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":1.542}]}]},{"osd":4,"up_from":13,"seq":55834575015,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33228,"kb_used_data":2052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338612,"statfs":{"total":96636764160,"available":96602738688,"internally_reserved":0,"allocated":2101248,"data_stored":1182864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:28:24 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.091,"15min":1.1000000000000001},"min":{"1min":0.63800000000000001,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.5740000000000001,"5min":1.6699999999999999,"15min":1.9770000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.079,"5min":1.1599999999999999,"15min":1.167},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.7350000000000001,"5min":2.004,"15min":2.004},"last":1.506}]},{"osd":1,"last update":"Sat Sep 23 03:28:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.224,"15min":1.214},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.7490000000000001,"5min":1.8200000000000001,"15min":2.1110000000000002},"last":1.631},{"interface":"front","average":{"1min":1.022,"5min":1.0649999999999999,"15min":1.123},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60299999999999998},"max":{"1min":1.4890000000000001,"5min":1.718,"15min":3.2189999999999999},"last":1.1990000000000001}]},{"osd":2,"last update":"Sat Sep 23 03:28:24 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.1559999999999999,"15min":1.1599999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.512,"5min":1.694,"15min":2.5920000000000001},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.252,"5min":1.278,"15min":1.25},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68300000000000005},"max":{"1min":1.8839999999999999,"5min":1.99,"15min":1.99},"last":0.76800000000000002}]},{"osd":3,"last update":"Sat Sep 23 03:28:24 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.1699999999999999,"15min":1.1659999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4379999999999999,"5min":1.702,"15min":1.9450000000000001},"last":1.071},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2589999999999999,"15min":1.286},"min":{"1min":0.75900000000000001,"5min":0.69799999999999995,"15min":0.67400000000000004},"max":{"1min":1.611,"5min":1.833,"15min":2.4969999999999999},"last":1.4510000000000001}]},{"osd":5,"last update":"Sat Sep 23 03:28:24 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.3540000000000001,"15min":1.363},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.70199999999999996},"max":{"1min":1.879,"5min":2.0409999999999999,"15min":2.8050000000000002},"last":1.7330000000000001},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.2969999999999999,"15min":1.3240000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.619,"5min":1.8939999999999999,"15min":2.887},"last":1.2569999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:28:24 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3660000000000001,"15min":1.369},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.9530000000000001,"5min":1.9530000000000001,"15min":2.4249999999999998},"last":1.696},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.478,"15min":1.4930000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.9350000000000001,"5min":2.0939999999999999,"15min":2.1779999999999999},"last":1.8}]},{"osd":7,"last update":"Sat Sep 23 03:28:24 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.419,"15min":1.45},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.734,"5min":2.1389999999999998,"15min":2.5680000000000001},"last":1.377},{"interface":"front","average":{"1min":1.464,"5min":1.534,"15min":1.544},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.4769999999999999},"last":1.5740000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575015,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33260,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338580,"statfs":{"total":96636764160,"available":96602705920,"internally_reserved":0,"allocated":2134016,"data_stored":1182914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.81000000000000005,"15min":0.85099999999999998},"min":{"1min":0.52600000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.22,"5min":1.5009999999999999,"15min":2.952},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.91500000000000004,"15min":0.89500000000000002},"min":{"1min":0.41799999999999998,"5min":0.377,"15min":0.29299999999999998},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":0.84199999999999997}]},{"osd":1,"last update":"Sat Sep 23 03:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.81899999999999995,"15min":0.86699999999999999},"min":{"1min":0.40000000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.48,"5min":1.48,"15min":1.6839999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.98499999999999999,"15min":0.98399999999999999},"min":{"1min":0.48299999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.482,"5min":1.7010000000000001,"15min":2.903},"last":1.079}]},{"osd":2,"last update":"Sat Sep 23 03:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.004,"15min":1.0409999999999999},"min":{"1min":0.54000000000000004,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.524,"5min":1.669,"15min":2.2240000000000002},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.119,"15min":1.1080000000000001},"min":{"1min":0.68799999999999994,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.8,"5min":1.8,"15min":3.129},"last":0.80300000000000005}]},{"osd":4,"last update":"Sat Sep 23 03:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1439999999999999,"15min":1.2070000000000001},"min":{"1min":0.88200000000000001,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.7010000000000001,"5min":1.7609999999999999,"15min":2.698},"last":1.6160000000000001},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.264,"15min":1.327},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.752},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":2.419},"last":1.2709999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.282,"15min":1.3240000000000001},"min":{"1min":0.82499999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.9950000000000001},"last":1.772},{"interface":"front","average":{"1min":1.212,"5min":1.2090000000000001,"15min":1.266},"min":{"1min":0.80200000000000005,"5min":0.624,"15min":0.624},"max":{"1min":1.6799999999999999,"5min":1.9830000000000001,"15min":2.0070000000000001},"last":1.7989999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.284,"15min":1.337},"min":{"1min":0.77900000000000003,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.827,"5min":1.9530000000000001,"15min":2.6440000000000001},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.363,"5min":1.3720000000000001,"15min":1.419},"min":{"1min":0.86399999999999999,"5min":0.73299999999999998,"15min":0.67800000000000005},"max":{"1min":2.0880000000000001,"5min":2.177,"15min":2.9750000000000001},"last":1.5860000000000001}]},{"osd":7,"last update":"Sat Sep 23 03:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.286,"15min":1.357},"min":{"1min":0.81299999999999994,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.6279999999999999,"5min":2.0859999999999999,"15min":2.7879999999999998},"last":1.7170000000000001},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3520000000000001,"15min":1.4079999999999999},"min":{"1min":0.90300000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.7290000000000001,"5min":2.0270000000000001,"15min":2.5379999999999998},"last":1.454}]}]},{"osd":2,"up_from":13,"seq":55834575016,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32840,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339000,"statfs":{"total":96636764160,"available":96603136000,"internally_reserved":0,"allocated":1699840,"data_stored":779468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.81299999999999994,"15min":0.82899999999999996},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.30299999999999999},"max":{"1min":0.92400000000000004,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.91200000000000003,"15min":0.88200000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.375,"5min":2.613,"15min":2.613},"last":0.63100000000000001}]},{"osd":1,"last update":"Sat Sep 23 03:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.85399999999999998,"15min":0.85499999999999998},"min":{"1min":0.46600000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.391,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.006,"5min":1.032,"15min":0.98299999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.38},"max":{"1min":1.526,"5min":2.9809999999999999,"15min":2.9809999999999999},"last":1.3}]},{"osd":3,"last update":"Sat Sep 23 03:28:28 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.0740000000000001,"15min":1.006},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.40699999999999997},"max":{"1min":1.6279999999999999,"5min":1.698,"15min":1.698},"last":1.157},{"interface":"front","average":{"1min":0.96999999999999997,"5min":1.044,"15min":1.01},"min":{"1min":0.46800000000000003,"5min":0.441,"15min":0.34999999999999998},"max":{"1min":1.482,"5min":1.827,"15min":1.827},"last":0.94899999999999995}]},{"osd":4,"last update":"Sat Sep 23 03:28:28 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.3240000000000001,"15min":1.286},"min":{"1min":0.82899999999999996,"5min":0.79400000000000004,"15min":0.55800000000000005},"max":{"1min":1.5309999999999999,"5min":5.6669999999999998,"15min":5.6669999999999998},"last":1.121},{"interface":"front","average":{"1min":1.28,"5min":1.302,"15min":1.2949999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.7190000000000001,"5min":2.6909999999999998,"15min":2.6909999999999998},"last":1.3500000000000001}]},{"osd":5,"last update":"Sat Sep 23 03:28:28 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.3260000000000001,"15min":1.335},"min":{"1min":0.83399999999999996,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.6879999999999999,"5min":2.8969999999999998,"15min":3.9409999999999998},"last":1.405},{"interface":"front","average":{"1min":1.234,"5min":1.379,"15min":1.3959999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.56399999999999995},"max":{"1min":1.8109999999999999,"5min":3.415,"15min":4.9199999999999999},"last":1.464}]},{"osd":6,"last update":"Sat Sep 23 03:28:28 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.357,"15min":1.3580000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.755},"max":{"1min":1.7090000000000001,"5min":2.7839999999999998,"15min":3.0009999999999999},"last":1.518},{"interface":"front","average":{"1min":1.321,"5min":1.4570000000000001,"15min":1.4219999999999999},"min":{"1min":0.91800000000000004,"5min":0.76600000000000001,"15min":0.65100000000000002},"max":{"1min":1.851,"5min":7.8300000000000001,"15min":7.8300000000000001},"last":1.1990000000000001}]},{"osd":7,"last update":"Sat Sep 23 03:28:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.423,"15min":1.395},"min":{"1min":0.83099999999999996,"5min":0.82899999999999996,"15min":0.61399999999999999},"max":{"1min":1.954,"5min":4.0330000000000004,"15min":4.0330000000000004},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.4199999999999999,"15min":1.4239999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.74399999999999999},"max":{"1min":2.3140000000000001,"5min":4.2199999999999998,"15min":4.2199999999999998},"last":1.4430000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575017,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32796,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339044,"statfs":{"total":96636764160,"available":96603181056,"internally_reserved":0,"allocated":1658880,"data_stored":779467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:28:40 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.81899999999999995,"15min":0.80200000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.32300000000000001},"max":{"1min":1.8660000000000001,"5min":1.8660000000000001,"15min":2.8279999999999998},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88300000000000001,"15min":0.84899999999999998},"min":{"1min":0.40899999999999997,"5min":0.40600000000000003,"15min":0.312},"max":{"1min":2.2490000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":0.71999999999999997}]},{"osd":2,"last update":"Sat Sep 23 03:28:40 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.86499999999999999,"15min":0.86899999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.36699999999999999},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.3340000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":1.069,"5min":1.0580000000000001,"15min":1.0329999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.909,"5min":1.909,"15min":1.9570000000000001},"last":0.65500000000000003}]},{"osd":3,"last update":"Sat Sep 23 03:28:40 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.046,"15min":1.0329999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.47299999999999998},"max":{"1min":1.992,"5min":1.992,"15min":2.2370000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.141,"15min":1.109},"min":{"1min":0.495,"5min":0.495,"15min":0.45900000000000002},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":3.2130000000000001},"last":0.80100000000000005}]},{"osd":4,"last update":"Sat Sep 23 03:28:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.2789999999999999,"15min":1.2490000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.798,"5min":1.885,"15min":3.2629999999999999},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.266,"5min":1.244,"15min":1.2529999999999999},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66500000000000004},"max":{"1min":2.2690000000000001,"5min":2.2690000000000001,"15min":2.4689999999999999},"last":0.92300000000000004}]},{"osd":5,"last update":"Sat Sep 23 03:28:40 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.2929999999999999,"15min":1.2789999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.64600000000000002},"max":{"1min":2.024,"5min":2.024,"15min":3.161},"last":1.528},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3280000000000001,"15min":1.3260000000000001},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.72399999999999998},"max":{"1min":3.0830000000000002,"5min":3.0830000000000002,"15min":3.0830000000000002},"last":0.88600000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:28:40 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3640000000000001,"15min":1.329},"min":{"1min":0.81299999999999994,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":3.0179999999999998},"last":1.482},{"interface":"front","average":{"1min":1.296,"5min":1.363,"15min":1.363},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":2.1309999999999998,"5min":2.1309999999999998,"15min":3.3580000000000001},"last":0.96299999999999997}]},{"osd":7,"last update":"Sat Sep 23 03:28:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.337,"15min":1.3600000000000001},"min":{"1min":0.77600000000000002,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.946,"5min":1.946,"15min":3.1099999999999999},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4039999999999999,"15min":1.419},"min":{"1min":0.86699999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":3.1179999999999999,"5min":3.1179999999999999,"15min":3.306},"last":1.3879999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575017,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33396,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338444,"statfs":{"total":96636764160,"available":96602566656,"internally_reserved":0,"allocated":2273280,"data_stored":1368828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 23 03:28:30 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80900000000000005,"15min":0.83499999999999996},"min":{"1min":0.44700000000000001,"5min":0.38,"15min":0.30299999999999999},"max":{"1min":1.3799999999999999,"5min":1.383,"15min":1.919},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.88900000000000001,"15min":0.88800000000000001},"min":{"1min":0.36699999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.351,"5min":1.4470000000000001,"15min":2.1360000000000001},"last":0.96699999999999997}]},{"osd":2,"last update":"Sat Sep 23 03:28:30 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.873,"15min":0.85399999999999998},"min":{"1min":0.46899999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.335,"5min":1.5249999999999999,"15min":3.089},"last":0.749},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.97299999999999998,"15min":0.999},"min":{"1min":0.47699999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.4970000000000001,"5min":1.5760000000000001,"15min":2.633},"last":1.008}]},{"osd":3,"last update":"Sat Sep 23 03:28:30 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.98799999999999999,"15min":0.98099999999999998},"min":{"1min":0.5,"5min":0.433,"15min":0.433},"max":{"1min":1.417,"5min":1.615,"15min":2.8660000000000001},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.03,"15min":1.018},"min":{"1min":0.505,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.446,"5min":1.802,"15min":2.012},"last":0.70499999999999996}]},{"osd":4,"last update":"Sat Sep 23 03:28:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.252,"15min":1.2609999999999999},"min":{"1min":0.73599999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.621,"5min":1.885,"15min":2.4140000000000001},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2270000000000001,"15min":1.2609999999999999},"min":{"1min":0.83199999999999996,"5min":0.73999999999999999,"15min":0.65300000000000002},"max":{"1min":1.5309999999999999,"5min":1.845,"15min":3.0230000000000001},"last":1.506}]},{"osd":5,"last update":"Sat Sep 23 03:28:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.296,"15min":1.274},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.65600000000000003},"max":{"1min":1.9530000000000001,"5min":2.1240000000000001,"15min":2.5489999999999999},"last":1.6890000000000001},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3320000000000001,"15min":1.3380000000000001},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.7210000000000001,"5min":1.9159999999999999,"15min":2.1539999999999999},"last":1.2729999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:28:30 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3149999999999999,"15min":1.325},"min":{"1min":0.78900000000000003,"5min":0.77700000000000002,"15min":0.75600000000000001},"max":{"1min":1.8600000000000001,"5min":2.0870000000000002,"15min":2.9489999999999998},"last":1.5660000000000001},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.3839999999999999,"15min":1.3859999999999999},"min":{"1min":0.71199999999999997,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.8100000000000001,"5min":1.9670000000000001,"15min":2.3239999999999998},"last":1.151}]},{"osd":7,"last update":"Sat Sep 23 03:28:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.3420000000000001,"15min":1.3640000000000001},"min":{"1min":0.93200000000000005,"5min":0.752,"15min":0.747},"max":{"1min":1.722,"5min":2.052,"15min":2.2330000000000001},"last":1.8580000000000001},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.405,"15min":1.4359999999999999},"min":{"1min":1.0960000000000001,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.7889999999999999,"5min":2.1419999999999999,"15min":2.3330000000000002},"last":1.798}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-23T03:28:54.963 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-09-23T03:28:54.964 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-23T03:28:55.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.185+0000 7ff9e838a640 1 Processor -- start 2023-09-23T03:28:55.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.185+0000 7ff9e838a640 1 -- start start 2023-09-23T03:28:55.187 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.187+0000 7ff9e838a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff9e015ad70 0x7ff9e00c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:55.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.187+0000 7ff9e838a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9e00b0780 0x7ff9e006c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:55.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.187+0000 7ff9e838a640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9e0070b40 con 0x7ff9e00ad9c0 2023-09-23T03:28:55.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.187+0000 7ff9e838a640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9e0071100 con 0x7ff9e015ad70 2023-09-23T03:28:55.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.187+0000 7ff9e838a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e00c9390 con 0x7ff9e00b0780 2023-09-23T03:28:55.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.187+0000 7ff9e6126640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff9e015ad70 0x7ff9e00c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:55.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.187+0000 7ff9e6927640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9e00b0780 0x7ff9e006c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:55.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.187+0000 7ff9e6126640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7ff9e015ad70 0x7ff9e00c8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.154:58840/0 (socket says 172.21.15.154:58840) 2023-09-23T03:28:55.189 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.187+0000 7ff9e6126640 1 -- 172.21.15.154:0/1693015377 learned_addr learned my addr 172.21.15.154:0/1693015377 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:55.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.188+0000 7ff9e5124640 1 -- 172.21.15.154:0/1693015377 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 302827047 0 0) 0x7ff9e0070b40 con 0x7ff9e00ad9c0 2023-09-23T03:28:55.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.188+0000 7ff9e5124640 1 -- 172.21.15.154:0/1693015377 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9cc003520 con 0x7ff9e00ad9c0 2023-09-23T03:28:55.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.188+0000 7ff9e6927640 1 -- 172.21.15.154:0/1693015377 >> v2:172.21.15.190:3300/0 conn(0x7ff9e015ad70 msgr2=0x7ff9e00c8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:55.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.188+0000 7ff9e6927640 1 --2- 172.21.15.154:0/1693015377 >> v2:172.21.15.190:3300/0 conn(0x7ff9e015ad70 0x7ff9e00c8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:55.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.188+0000 7ff9e6927640 1 -- 172.21.15.154:0/1693015377 >> v1:172.21.15.190:6789/0 conn(0x7ff9e00ad9c0 legacy=0x7ff9e00b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:55.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.188+0000 7ff9e6927640 1 -- 172.21.15.154:0/1693015377 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e0070e20 con 0x7ff9e00b0780 2023-09-23T03:28:55.190 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.189+0000 7ff9e6927640 1 --2- 172.21.15.154:0/1693015377 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9e00b0780 0x7ff9e006c8c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8009d90 tx=0x7ff9d802bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a033ad8740beef8a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:55.191 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.190+0000 7ff9e5124640 1 -- 172.21.15.154:0/1693015377 <== mon.2 v2:172.21.15.190:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d802cc50 con 0x7ff9e00b0780 2023-09-23T03:28:55.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.190+0000 7ff9e5124640 1 -- 172.21.15.154:0/1693015377 <== mon.2 v2:172.21.15.190:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d802cdb0 con 0x7ff9e00b0780 2023-09-23T03:28:55.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.190+0000 7ff9e5124640 1 -- 172.21.15.154:0/1693015377 <== mon.2 v2:172.21.15.190:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8034660 con 0x7ff9e00b0780 2023-09-23T03:28:55.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.191+0000 7ff9e838a640 1 -- 172.21.15.154:0/1693015377 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9e00b0780 msgr2=0x7ff9e006c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:55.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.191+0000 7ff9e838a640 1 --2- 172.21.15.154:0/1693015377 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9e00b0780 0x7ff9e006c8c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8009d90 tx=0x7ff9d802bea0 comp rx=0 tx=0).stop 2023-09-23T03:28:55.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.191+0000 7ff9e838a640 1 -- 172.21.15.154:0/1693015377 shutdown_connections 2023-09-23T03:28:55.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.191+0000 7ff9e838a640 1 --2- 172.21.15.154:0/1693015377 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9e00b0780 0x7ff9e006c8c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:55.192 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.191+0000 7ff9e838a640 1 --2- 172.21.15.154:0/1693015377 >> v2:172.21.15.190:3300/0 conn(0x7ff9e015ad70 0x7ff9e00c8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:55.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.191+0000 7ff9e838a640 1 -- 172.21.15.154:0/1693015377 >> 172.21.15.154:0/1693015377 conn(0x7ff9e00cb070 msgr2=0x7ff9e00cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:55.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.191+0000 7ff9e838a640 1 -- 172.21.15.154:0/1693015377 shutdown_connections 2023-09-23T03:28:55.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.191+0000 7ff9e838a640 1 -- 172.21.15.154:0/1693015377 wait complete. 2023-09-23T03:28:55.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.193+0000 7ff9e838a640 1 Processor -- start 2023-09-23T03:28:55.193 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.193+0000 7ff9e838a640 1 -- start start 2023-09-23T03:28:55.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.194+0000 7ff9e838a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9e00ad9c0 0x7ff9e01f2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:55.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.194+0000 7ff9e838a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9e00b0780 0x7ff9e00cc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:55.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.194+0000 7ff9e838a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9e015ad70 0x7ff9e01595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:55.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.194+0000 7ff9e838a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e00ac3a0 con 0x7ff9e015ad70 2023-09-23T03:28:55.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.194+0000 7ff9e838a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e00ac680 con 0x7ff9e00b0780 2023-09-23T03:28:55.195 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.194+0000 7ff9e838a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e00ac960 con 0x7ff9e00ad9c0 2023-09-23T03:28:55.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.194+0000 7ff9e6927640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9e015ad70 0x7ff9e01595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:55.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.194+0000 7ff9e6927640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9e015ad70 0x7ff9e01595c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44752/0 (socket says 172.21.15.154:44752) 2023-09-23T03:28:55.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.194+0000 7ff9e6927640 1 -- 172.21.15.154:0/2378492992 learned_addr learned my addr 172.21.15.154:0/2378492992 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:55.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.195+0000 7ff9e5925640 1 --2- 172.21.15.154:0/2378492992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9e00b0780 0x7ff9e00cc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:55.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.195+0000 7ff9e6126640 1 --2- 172.21.15.154:0/2378492992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9e00ad9c0 0x7ff9e01f2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:55.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.195+0000 7ff9e6927640 1 -- 172.21.15.154:0/2378492992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9e00ad9c0 msgr2=0x7ff9e01f2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:55.196 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.195+0000 7ff9e6927640 1 --2- 172.21.15.154:0/2378492992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9e00ad9c0 0x7ff9e01f2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:55.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.195+0000 7ff9e6927640 1 -- 172.21.15.154:0/2378492992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9e00b0780 msgr2=0x7ff9e00cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:55.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.195+0000 7ff9e6927640 1 --2- 172.21.15.154:0/2378492992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9e00b0780 0x7ff9e00cc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:55.197 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.195+0000 7ff9e6927640 1 -- 172.21.15.154:0/2378492992 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d8009a70 con 0x7ff9e015ad70 2023-09-23T03:28:55.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.195+0000 7ff9e6927640 1 --2- 172.21.15.154:0/2378492992 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9e015ad70 0x7ff9e01595c0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8009eb0 tx=0x7ff9d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:55.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.195+0000 7ff9d6ffd640 1 -- 172.21.15.154:0/2378492992 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d803e610 con 0x7ff9e015ad70 2023-09-23T03:28:55.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.195+0000 7ff9d6ffd640 1 -- 172.21.15.154:0/2378492992 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d803e770 con 0x7ff9e015ad70 2023-09-23T03:28:55.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.195+0000 7ff9d6ffd640 1 -- 172.21.15.154:0/2378492992 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d803eb20 con 0x7ff9e015ad70 2023-09-23T03:28:55.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.195+0000 7ff9e838a640 1 -- 172.21.15.154:0/2378492992 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9e00acc40 con 0x7ff9e015ad70 2023-09-23T03:28:55.198 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.196+0000 7ff9e838a640 1 -- 172.21.15.154:0/2378492992 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9e00ce080 con 0x7ff9e015ad70 2023-09-23T03:28:55.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.197+0000 7ff9e838a640 1 -- 172.21.15.154:0/2378492992 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9e0067870 con 0x7ff9e015ad70 2023-09-23T03:28:55.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.198+0000 7ff9d6ffd640 1 -- 172.21.15.154:0/2378492992 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7ff9d803b050 con 0x7ff9e015ad70 2023-09-23T03:28:55.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.199+0000 7ff9d6ffd640 1 --2- 172.21.15.154:0/2378492992 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff9c406af30 0x7ff9c406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:55.199 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.199+0000 7ff9d6ffd640 1 -- 172.21.15.154:0/2378492992 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5435+0+0 (secure 0 0 0) 0x7ff9d80abd80 con 0x7ff9e015ad70 2023-09-23T03:28:55.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.199+0000 7ff9e6126640 1 --2- 172.21.15.154:0/2378492992 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff9c406af30 0x7ff9c406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:55.200 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.200+0000 7ff9e6126640 1 --2- 172.21.15.154:0/2378492992 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff9c406af30 0x7ff9c406d3f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc009aa0 tx=0x7ff9dc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:55.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.203+0000 7ff9d6ffd640 1 -- 172.21.15.154:0/2378492992 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7ff9d8077080 con 0x7ff9e015ad70 2023-09-23T03:28:55.389 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.389+0000 7ff9e838a640 1 -- 172.21.15.154:0/2378492992 --> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff9e00c99b0 con 0x7ff9c406af30 2023-09-23T03:28:55.391 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.391+0000 7ff9d6ffd640 1 -- 172.21.15.154:0/2378492992 <== mgr.4110 v2:172.21.15.154:6832/2422652104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48311 (secure 0 0 0) 0x7ff9dc010930 con 0x7ff9c406af30 2023-09-23T03:28:55.392 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:28:55.393 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2023-09-23T03:28:55.394 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.394+0000 7ff9e838a640 1 -- 172.21.15.154:0/2378492992 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff9c406af30 msgr2=0x7ff9c406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:55.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.394+0000 7ff9e838a640 1 --2- 172.21.15.154:0/2378492992 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff9c406af30 0x7ff9c406d3f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc009aa0 tx=0x7ff9dc008040 comp rx=0 tx=0).stop 2023-09-23T03:28:55.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.394+0000 7ff9e838a640 1 -- 172.21.15.154:0/2378492992 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9e015ad70 msgr2=0x7ff9e01595c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:55.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.394+0000 7ff9e838a640 1 --2- 172.21.15.154:0/2378492992 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9e015ad70 0x7ff9e01595c0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8009eb0 tx=0x7ff9d8004a40 comp rx=0 tx=0).stop 2023-09-23T03:28:55.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.394+0000 7ff9e838a640 1 -- 172.21.15.154:0/2378492992 shutdown_connections 2023-09-23T03:28:55.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.394+0000 7ff9e838a640 1 --2- 172.21.15.154:0/2378492992 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7ff9c406af30 0x7ff9c406d3f0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:55.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.394+0000 7ff9e838a640 1 --2- 172.21.15.154:0/2378492992 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff9e015ad70 0x7ff9e01595c0 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:55.395 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.394+0000 7ff9e838a640 1 --2- 172.21.15.154:0/2378492992 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7ff9e00b0780 0x7ff9e00cc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:55.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.394+0000 7ff9e838a640 1 --2- 172.21.15.154:0/2378492992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7ff9e00ad9c0 0x7ff9e01f2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:55.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.394+0000 7ff9e838a640 1 -- 172.21.15.154:0/2378492992 >> 172.21.15.154:0/2378492992 conn(0x7ff9e00cb070 msgr2=0x7ff9e00afdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:55.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.394+0000 7ff9e838a640 1 -- 172.21.15.154:0/2378492992 shutdown_connections 2023-09-23T03:28:55.396 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.394+0000 7ff9e838a640 1 -- 172.21.15.154:0/2378492992 wait complete. 2023-09-23T03:28:55.414 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":536,"stamp":"2023-09-23T03:28:53.454653+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264680,"kb_used_data":15264,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754710040,"statfs":{"total":773094113280,"available":772823080960,"internally_reserved":0,"allocated":15630336,"data_stored":8230418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":255315514},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-404392,"num_objects":-123,"num_object_clones":0,"num_object_copies":-246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-123,"num_whiteouts":0,"num_read":-1808,"num_read_kb":-1513,"num_write":-913,"num_write_kb":-892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001923"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":260,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-09-23T03:28:29.522907+0000","last_change":"2023-09-23T03:15:11.042447+0000","last_active":"2023-09-23T03:28:29.522907+0000","last_peered":"2023-09-23T03:28:29.522907+0000","last_clean":"2023-09-23T03:28:29.522907+0000","last_became_active":"2023-09-23T03:15:11.040322+0000","last_became_peered":"2023-09-23T03:15:11.040322+0000","last_unstale":"2023-09-23T03:28:29.522907+0000","last_undegraded":"2023-09-23T03:28:29.522907+0000","last_fullsized":"2023-09-23T03:28:29.522907+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_clean_scrub_stamp":"2023-09-23T03:15:09.474842+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:18:36.331645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575014,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33436,"kb_used_data":2260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338404,"statfs":{"total":96636764160,"available":96602525696,"internally_reserved":0,"allocated":2314240,"data_stored":1370614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0649999999999999,"15min":1.026},"min":{"1min":0.67600000000000005,"5min":0.58199999999999996,"15min":0.442},"max":{"1min":1.5880000000000001,"5min":2.0299999999999998,"15min":2.149},"last":1.044},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0840000000000001,"15min":1.0640000000000001},"min":{"1min":0.77700000000000002,"5min":0.63100000000000001,"15min":0.52000000000000002},"max":{"1min":1.5009999999999999,"5min":1.9910000000000001,"15min":2.1480000000000001},"last":1.073}]},{"osd":1,"last update":"Sat Sep 23 03:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.2390000000000001,"15min":1.218},"min":{"1min":0.80800000000000005,"5min":0.69599999999999995,"15min":0.504},"max":{"1min":1.663,"5min":2.6400000000000001,"15min":3.1859999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":1.264,"5min":1.2769999999999999,"15min":1.284},"min":{"1min":0.80600000000000005,"5min":0.71299999999999997,"15min":0.67900000000000005},"max":{"1min":1.6120000000000001,"5min":2.3149999999999999,"15min":2.613},"last":0.66700000000000004}]},{"osd":2,"last update":"Sat Sep 23 03:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.143,"15min":1.125},"min":{"1min":0.67500000000000004,"5min":0.61199999999999999,"15min":0.42399999999999999},"max":{"1min":1.619,"5min":1.6439999999999999,"15min":2.331},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.206,"5min":1.262,"15min":1.2330000000000001},"min":{"1min":0.89400000000000002,"5min":0.79600000000000004,"15min":0.55200000000000005},"max":{"1min":1.538,"5min":2.2719999999999998,"15min":3.093},"last":0.749}]},{"osd":3,"last update":"Sat Sep 23 03:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.236,"15min":1.236},"min":{"1min":0.78700000000000003,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.7030000000000001,"5min":2.0840000000000001,"15min":2.532},"last":1.427},{"interface":"front","average":{"1min":1.343,"5min":1.381,"15min":1.361},"min":{"1min":0.99299999999999999,"5min":0.71899999999999997,"15min":0.69799999999999995},"max":{"1min":1.681,"5min":2.4289999999999998,"15min":3.2349999999999999},"last":1.3819999999999999}]},{"osd":4,"last update":"Sat Sep 23 03:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2869999999999999,"15min":1.349},"min":{"1min":0.85799999999999998,"5min":0.57999999999999996,"15min":0.45400000000000001},"max":{"1min":1.6259999999999999,"5min":2.3260000000000001,"15min":2.7210000000000001},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.446,"15min":1.4630000000000001},"min":{"1min":0.93500000000000005,"5min":0.73599999999999999,"15min":0.48099999999999998},"max":{"1min":1.8200000000000001,"5min":2.2240000000000002,"15min":3.7719999999999998},"last":1.5069999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.4079999999999999,"15min":1.405},"min":{"1min":0.86699999999999999,"5min":0.86499999999999999,"15min":0.47999999999999998},"max":{"1min":1.6180000000000001,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.407,"15min":1.429},"min":{"1min":1.002,"5min":0.77000000000000002,"15min":0.497},"max":{"1min":1.748,"5min":2.5070000000000001,"15min":2.5070000000000001},"last":1.472}]},{"osd":6,"last update":"Sat Sep 23 03:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.498,"15min":1.528},"min":{"1min":1.0449999999999999,"5min":0.85499999999999998,"15min":0.53800000000000003},"max":{"1min":1.8580000000000001,"5min":2.649,"15min":2.8420000000000001},"last":1.343},{"interface":"front","average":{"1min":1.401,"5min":1.4279999999999999,"15min":1.4670000000000001},"min":{"1min":1.1319999999999999,"5min":0.78900000000000003,"15min":0.45900000000000002},"max":{"1min":1.726,"5min":2.4500000000000002,"15min":3.6739999999999999},"last":1.5600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575014,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32844,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338996,"statfs":{"total":96636764160,"available":96603131904,"internally_reserved":0,"allocated":1708032,"data_stored":780052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:28:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.145,"15min":1.095},"min":{"1min":0.57199999999999995,"5min":0.53600000000000003,"15min":0.38},"max":{"1min":1.9830000000000001,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.081,"15min":1.0700000000000001},"min":{"1min":0.68600000000000005,"5min":0.60999999999999999,"15min":0.46100000000000002},"max":{"1min":1.4510000000000001,"5min":3.7629999999999999,"15min":3.7629999999999999},"last":1.6799999999999999}]},{"osd":1,"last update":"Sat Sep 23 03:28:26 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.157,"15min":1.125},"min":{"1min":0.66000000000000003,"5min":0.63800000000000001,"15min":0.438},"max":{"1min":1.5589999999999999,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.2090000000000001,"15min":1.194},"min":{"1min":0.96099999999999997,"5min":0.56599999999999995,"15min":0.54400000000000004},"max":{"1min":1.776,"5min":2.972,"15min":2.972},"last":0.69099999999999995}]},{"osd":2,"last update":"Sat Sep 23 03:28:26 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.1990000000000001,"15min":1.141},"min":{"1min":0.998,"5min":0.56499999999999995,"15min":0.51300000000000001},"max":{"1min":1.605,"5min":3.4390000000000001,"15min":3.4390000000000001},"last":1.569},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2869999999999999,"15min":1.2370000000000001},"min":{"1min":0.94499999999999995,"5min":0.68000000000000005,"15min":0.64300000000000002},"max":{"1min":1.625,"5min":2.177,"15min":2.177},"last":1.1359999999999999}]},{"osd":3,"last update":"Sat Sep 23 03:28:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.381,"15min":1.339},"min":{"1min":0.96099999999999997,"5min":0.78500000000000003,"15min":0.54600000000000004},"max":{"1min":1.6890000000000001,"5min":2.129,"15min":2.6509999999999998},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.339,"15min":1.365},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.74299999999999999},"max":{"1min":1.8899999999999999,"5min":3.0910000000000002,"15min":3.0910000000000002},"last":1.2829999999999999}]},{"osd":4,"last update":"Sat Sep 23 03:28:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.3360000000000001,"15min":1.365},"min":{"1min":0.91700000000000004,"5min":0.73299999999999998,"15min":0.498},"max":{"1min":2.0960000000000001,"5min":3.1789999999999998,"15min":3.1789999999999998},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3360000000000001,"15min":1.3009999999999999},"min":{"1min":0.92900000000000005,"5min":0.72399999999999998,"15min":0.56000000000000005},"max":{"1min":1.5760000000000001,"5min":3.6480000000000001,"15min":3.6480000000000001},"last":1.208}]},{"osd":5,"last update":"Sat Sep 23 03:28:26 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.4330000000000001,"15min":1.411},"min":{"1min":1.01,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.7150000000000001,"5min":4.093,"15min":4.093},"last":1.462},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.464,"15min":1.4419999999999999},"min":{"1min":1.0780000000000001,"5min":0.69899999999999995,"15min":0.59899999999999998},"max":{"1min":1.837,"5min":2.0459999999999998,"15min":4.758},"last":1.913}]},{"osd":7,"last update":"Sat Sep 23 03:28:26 2023","interfaces":[{"interface":"back","average":{"1min":1.633,"5min":1.55,"15min":1.524},"min":{"1min":1.423,"5min":0.76500000000000001,"15min":0.63500000000000001},"max":{"1min":2.1859999999999999,"5min":3.536,"15min":3.536},"last":1.371},{"interface":"front","average":{"1min":1.532,"5min":1.5569999999999999,"15min":1.5329999999999999},"min":{"1min":1.081,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":2.0190000000000001,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":1.76}]}]},{"osd":5,"up_from":13,"seq":55834575016,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32880,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338960,"statfs":{"total":96636764160,"available":96603095040,"internally_reserved":0,"allocated":1740800,"data_stored":786211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:28:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.087,"15min":1.048},"min":{"1min":0.64100000000000001,"5min":0.61199999999999999,"15min":0.57199999999999995},"max":{"1min":1.639,"5min":2.9740000000000002,"15min":2.9740000000000002},"last":1.421},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.1120000000000001,"15min":1.083},"min":{"1min":0.69599999999999995,"5min":0.624,"15min":0.61399999999999999},"max":{"1min":1.603,"5min":1.962,"15min":2.2320000000000002},"last":1.236}]},{"osd":1,"last update":"Sat Sep 23 03:28:27 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2050000000000001,"15min":1.218},"min":{"1min":0.73999999999999999,"5min":0.71899999999999997,"15min":0.62},"max":{"1min":1.657,"5min":2.4369999999999998,"15min":2.4369999999999998},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.268,"5min":1.294,"15min":1.2829999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67200000000000004},"max":{"1min":1.6120000000000001,"5min":3.5859999999999999,"15min":3.5859999999999999},"last":1.52}]},{"osd":2,"last update":"Sat Sep 23 03:28:27 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.2050000000000001,"15min":1.2010000000000001},"min":{"1min":0.875,"5min":0.65300000000000002,"15min":0.64300000000000002},"max":{"1min":1.76,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.1399999999999999,"15min":1.2050000000000001},"min":{"1min":0.71699999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.4370000000000001,"5min":2.8500000000000001,"15min":2.8500000000000001},"last":0.85299999999999998}]},{"osd":3,"last update":"Sat Sep 23 03:28:27 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2669999999999999,"15min":1.254},"min":{"1min":0.92800000000000005,"5min":0.67200000000000004,"15min":0.66400000000000003},"max":{"1min":1.6379999999999999,"5min":3.4359999999999999,"15min":3.4359999999999999},"last":1.329},{"interface":"front","average":{"1min":1.224,"5min":1.2969999999999999,"15min":1.339},"min":{"1min":0.91000000000000003,"5min":0.85999999999999999,"15min":0.69799999999999995},"max":{"1min":1.6719999999999999,"5min":1.9159999999999999,"15min":3.6110000000000002},"last":1.2909999999999999}]},{"osd":4,"last update":"Sat Sep 23 03:28:27 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.393,"15min":1.3899999999999999},"min":{"1min":0.92000000000000004,"5min":0.68600000000000005,"15min":0.54100000000000004},"max":{"1min":1.843,"5min":4.2750000000000004,"15min":4.2750000000000004},"last":1.585},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.3009999999999999,"15min":1.341},"min":{"1min":1.0429999999999999,"5min":0.54100000000000004,"15min":0.48899999999999999},"max":{"1min":1.4990000000000001,"5min":3.3090000000000002,"15min":3.3090000000000002},"last":1.1519999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:28:27 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.504,"15min":1.5049999999999999},"min":{"1min":1.1180000000000001,"5min":1.0329999999999999,"15min":0.59199999999999997},"max":{"1min":1.77,"5min":4.6040000000000001,"15min":4.6040000000000001},"last":1.484},{"interface":"front","average":{"1min":1.389,"5min":1.4159999999999999,"15min":1.5069999999999999},"min":{"1min":1.206,"5min":0.81200000000000006,"15min":0.55100000000000005},"max":{"1min":1.677,"5min":3.169,"15min":9.6560000000000006},"last":1.6419999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:28:27 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.4159999999999999,"15min":1.429},"min":{"1min":1.091,"5min":0.77100000000000002,"15min":0.626},"max":{"1min":1.708,"5min":2.077,"15min":2.077},"last":1.6890000000000001},{"interface":"front","average":{"1min":1.488,"5min":1.5009999999999999,"15min":1.5389999999999999},"min":{"1min":1.25,"5min":0.92200000000000004,"15min":0.60799999999999998},"max":{"1min":1.849,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":1.542}]}]},{"osd":4,"up_from":13,"seq":55834575015,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33228,"kb_used_data":2052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338612,"statfs":{"total":96636764160,"available":96602738688,"internally_reserved":0,"allocated":2101248,"data_stored":1182864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:28:24 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.091,"15min":1.1000000000000001},"min":{"1min":0.63800000000000001,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.5740000000000001,"5min":1.6699999999999999,"15min":1.9770000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.079,"5min":1.1599999999999999,"15min":1.167},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.7350000000000001,"5min":2.004,"15min":2.004},"last":1.506}]},{"osd":1,"last update":"Sat Sep 23 03:28:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.224,"15min":1.214},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.7490000000000001,"5min":1.8200000000000001,"15min":2.1110000000000002},"last":1.631},{"interface":"front","average":{"1min":1.022,"5min":1.0649999999999999,"15min":1.123},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60299999999999998},"max":{"1min":1.4890000000000001,"5min":1.718,"15min":3.2189999999999999},"last":1.1990000000000001}]},{"osd":2,"last update":"Sat Sep 23 03:28:24 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.1559999999999999,"15min":1.1599999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.512,"5min":1.694,"15min":2.5920000000000001},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.252,"5min":1.278,"15min":1.25},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68300000000000005},"max":{"1min":1.8839999999999999,"5min":1.99,"15min":1.99},"last":0.76800000000000002}]},{"osd":3,"last update":"Sat Sep 23 03:28:24 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.1699999999999999,"15min":1.1659999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4379999999999999,"5min":1.702,"15min":1.9450000000000001},"last":1.071},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2589999999999999,"15min":1.286},"min":{"1min":0.75900000000000001,"5min":0.69799999999999995,"15min":0.67400000000000004},"max":{"1min":1.611,"5min":1.833,"15min":2.4969999999999999},"last":1.4510000000000001}]},{"osd":5,"last update":"Sat Sep 23 03:28:24 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.3540000000000001,"15min":1.363},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.70199999999999996},"max":{"1min":1.879,"5min":2.0409999999999999,"15min":2.8050000000000002},"last":1.7330000000000001},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.2969999999999999,"15min":1.3240000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.619,"5min":1.8939999999999999,"15min":2.887},"last":1.2569999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:28:24 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3660000000000001,"15min":1.369},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.9530000000000001,"5min":1.9530000000000001,"15min":2.4249999999999998},"last":1.696},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.478,"15min":1.4930000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.9350000000000001,"5min":2.0939999999999999,"15min":2.1779999999999999},"last":1.8}]},{"osd":7,"last update":"Sat Sep 23 03:28:24 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.419,"15min":1.45},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.734,"5min":2.1389999999999998,"15min":2.5680000000000001},"last":1.377},{"interface":"front","average":{"1min":1.464,"5min":1.534,"15min":1.544},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.4769999999999999},"last":1.5740000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575015,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33260,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338580,"statfs":{"total":96636764160,"available":96602705920,"internally_reserved":0,"allocated":2134016,"data_stored":1182914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.81000000000000005,"15min":0.85099999999999998},"min":{"1min":0.52600000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.22,"5min":1.5009999999999999,"15min":2.952},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.91500000000000004,"15min":0.89500000000000002},"min":{"1min":0.41799999999999998,"5min":0.377,"15min":0.29299999999999998},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":0.84199999999999997}]},{"osd":1,"last update":"Sat Sep 23 03:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.81899999999999995,"15min":0.86699999999999999},"min":{"1min":0.40000000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.48,"5min":1.48,"15min":1.6839999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.98499999999999999,"15min":0.98399999999999999},"min":{"1min":0.48299999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.482,"5min":1.7010000000000001,"15min":2.903},"last":1.079}]},{"osd":2,"last update":"Sat Sep 23 03:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.004,"15min":1.0409999999999999},"min":{"1min":0.54000000000000004,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.524,"5min":1.669,"15min":2.2240000000000002},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.119,"15min":1.1080000000000001},"min":{"1min":0.68799999999999994,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.8,"5min":1.8,"15min":3.129},"last":0.80300000000000005}]},{"osd":4,"last update":"Sat Sep 23 03:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1439999999999999,"15min":1.2070000000000001},"min":{"1min":0.88200000000000001,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.7010000000000001,"5min":1.7609999999999999,"15min":2.698},"last":1.6160000000000001},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.264,"15min":1.327},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.752},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":2.419},"last":1.2709999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.282,"15min":1.3240000000000001},"min":{"1min":0.82499999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.9950000000000001},"last":1.772},{"interface":"front","average":{"1min":1.212,"5min":1.2090000000000001,"15min":1.266},"min":{"1min":0.80200000000000005,"5min":0.624,"15min":0.624},"max":{"1min":1.6799999999999999,"5min":1.9830000000000001,"15min":2.0070000000000001},"last":1.7989999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.284,"15min":1.337},"min":{"1min":0.77900000000000003,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.827,"5min":1.9530000000000001,"15min":2.6440000000000001},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.363,"5min":1.3720000000000001,"15min":1.419},"min":{"1min":0.86399999999999999,"5min":0.73299999999999998,"15min":0.67800000000000005},"max":{"1min":2.0880000000000001,"5min":2.177,"15min":2.9750000000000001},"last":1.5860000000000001}]},{"osd":7,"last update":"Sat Sep 23 03:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.286,"15min":1.357},"min":{"1min":0.81299999999999994,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.6279999999999999,"5min":2.0859999999999999,"15min":2.7879999999999998},"last":1.7170000000000001},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3520000000000001,"15min":1.4079999999999999},"min":{"1min":0.90300000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.7290000000000001,"5min":2.0270000000000001,"15min":2.5379999999999998},"last":1.454}]}]},{"osd":2,"up_from":13,"seq":55834575016,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32840,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339000,"statfs":{"total":96636764160,"available":96603136000,"internally_reserved":0,"allocated":1699840,"data_stored":779468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.81299999999999994,"15min":0.82899999999999996},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.30299999999999999},"max":{"1min":0.92400000000000004,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.91200000000000003,"15min":0.88200000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.375,"5min":2.613,"15min":2.613},"last":0.63100000000000001}]},{"osd":1,"last update":"Sat Sep 23 03:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.85399999999999998,"15min":0.85499999999999998},"min":{"1min":0.46600000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.391,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.006,"5min":1.032,"15min":0.98299999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.38},"max":{"1min":1.526,"5min":2.9809999999999999,"15min":2.9809999999999999},"last":1.3}]},{"osd":3,"last update":"Sat Sep 23 03:28:28 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.0740000000000001,"15min":1.006},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.40699999999999997},"max":{"1min":1.6279999999999999,"5min":1.698,"15min":1.698},"last":1.157},{"interface":"front","average":{"1min":0.96999999999999997,"5min":1.044,"15min":1.01},"min":{"1min":0.46800000000000003,"5min":0.441,"15min":0.34999999999999998},"max":{"1min":1.482,"5min":1.827,"15min":1.827},"last":0.94899999999999995}]},{"osd":4,"last update":"Sat Sep 23 03:28:28 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.3240000000000001,"15min":1.286},"min":{"1min":0.82899999999999996,"5min":0.79400000000000004,"15min":0.55800000000000005},"max":{"1min":1.5309999999999999,"5min":5.6669999999999998,"15min":5.6669999999999998},"last":1.121},{"interface":"front","average":{"1min":1.28,"5min":1.302,"15min":1.2949999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.7190000000000001,"5min":2.6909999999999998,"15min":2.6909999999999998},"last":1.3500000000000001}]},{"osd":5,"last update":"Sat Sep 23 03:28:28 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.3260000000000001,"15min":1.335},"min":{"1min":0.83399999999999996,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.6879999999999999,"5min":2.8969999999999998,"15min":3.9409999999999998},"last":1.405},{"interface":"front","average":{"1min":1.234,"5min":1.379,"15min":1.3959999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.56399999999999995},"max":{"1min":1.8109999999999999,"5min":3.415,"15min":4.9199999999999999},"last":1.464}]},{"osd":6,"last update":"Sat Sep 23 03:28:28 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.357,"15min":1.3580000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.755},"max":{"1min":1.7090000000000001,"5min":2.7839999999999998,"15min":3.0009999999999999},"last":1.518},{"interface":"front","average":{"1min":1.321,"5min":1.4570000000000001,"15min":1.4219999999999999},"min":{"1min":0.91800000000000004,"5min":0.76600000000000001,"15min":0.65100000000000002},"max":{"1min":1.851,"5min":7.8300000000000001,"15min":7.8300000000000001},"last":1.1990000000000001}]},{"osd":7,"last update":"Sat Sep 23 03:28:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.423,"15min":1.395},"min":{"1min":0.83099999999999996,"5min":0.82899999999999996,"15min":0.61399999999999999},"max":{"1min":1.954,"5min":4.0330000000000004,"15min":4.0330000000000004},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.4199999999999999,"15min":1.4239999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.74399999999999999},"max":{"1min":2.3140000000000001,"5min":4.2199999999999998,"15min":4.2199999999999998},"last":1.4430000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575017,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32796,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339044,"statfs":{"total":96636764160,"available":96603181056,"internally_reserved":0,"allocated":1658880,"data_stored":779467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:28:40 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.81899999999999995,"15min":0.80200000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.32300000000000001},"max":{"1min":1.8660000000000001,"5min":1.8660000000000001,"15min":2.8279999999999998},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88300000000000001,"15min":0.84899999999999998},"min":{"1min":0.40899999999999997,"5min":0.40600000000000003,"15min":0.312},"max":{"1min":2.2490000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":0.71999999999999997}]},{"osd":2,"last update":"Sat Sep 23 03:28:40 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.86499999999999999,"15min":0.86899999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.36699999999999999},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.3340000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":1.069,"5min":1.0580000000000001,"15min":1.0329999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.909,"5min":1.909,"15min":1.9570000000000001},"last":0.65500000000000003}]},{"osd":3,"last update":"Sat Sep 23 03:28:40 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.046,"15min":1.0329999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.47299999999999998},"max":{"1min":1.992,"5min":1.992,"15min":2.2370000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.141,"15min":1.109},"min":{"1min":0.495,"5min":0.495,"15min":0.45900000000000002},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":3.2130000000000001},"last":0.80100000000000005}]},{"osd":4,"last update":"Sat Sep 23 03:28:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.2789999999999999,"15min":1.2490000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.798,"5min":1.885,"15min":3.2629999999999999},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.266,"5min":1.244,"15min":1.2529999999999999},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66500000000000004},"max":{"1min":2.2690000000000001,"5min":2.2690000000000001,"15min":2.4689999999999999},"last":0.92300000000000004}]},{"osd":5,"last update":"Sat Sep 23 03:28:40 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.2929999999999999,"15min":1.2789999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.64600000000000002},"max":{"1min":2.024,"5min":2.024,"15min":3.161},"last":1.528},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3280000000000001,"15min":1.3260000000000001},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.72399999999999998},"max":{"1min":3.0830000000000002,"5min":3.0830000000000002,"15min":3.0830000000000002},"last":0.88600000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:28:40 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3640000000000001,"15min":1.329},"min":{"1min":0.81299999999999994,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":3.0179999999999998},"last":1.482},{"interface":"front","average":{"1min":1.296,"5min":1.363,"15min":1.363},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":2.1309999999999998,"5min":2.1309999999999998,"15min":3.3580000000000001},"last":0.96299999999999997}]},{"osd":7,"last update":"Sat Sep 23 03:28:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.337,"15min":1.3600000000000001},"min":{"1min":0.77600000000000002,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.946,"5min":1.946,"15min":3.1099999999999999},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4039999999999999,"15min":1.419},"min":{"1min":0.86699999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":3.1179999999999999,"5min":3.1179999999999999,"15min":3.306},"last":1.3879999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575017,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33396,"kb_used_data":2220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338444,"statfs":{"total":96636764160,"available":96602566656,"internally_reserved":0,"allocated":2273280,"data_stored":1368828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 23 03:28:30 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80900000000000005,"15min":0.83499999999999996},"min":{"1min":0.44700000000000001,"5min":0.38,"15min":0.30299999999999999},"max":{"1min":1.3799999999999999,"5min":1.383,"15min":1.919},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.88900000000000001,"15min":0.88800000000000001},"min":{"1min":0.36699999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.351,"5min":1.4470000000000001,"15min":2.1360000000000001},"last":0.96699999999999997}]},{"osd":2,"last update":"Sat Sep 23 03:28:30 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.873,"15min":0.85399999999999998},"min":{"1min":0.46899999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.335,"5min":1.5249999999999999,"15min":3.089},"last":0.749},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.97299999999999998,"15min":0.999},"min":{"1min":0.47699999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.4970000000000001,"5min":1.5760000000000001,"15min":2.633},"last":1.008}]},{"osd":3,"last update":"Sat Sep 23 03:28:30 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.98799999999999999,"15min":0.98099999999999998},"min":{"1min":0.5,"5min":0.433,"15min":0.433},"max":{"1min":1.417,"5min":1.615,"15min":2.8660000000000001},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.03,"15min":1.018},"min":{"1min":0.505,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.446,"5min":1.802,"15min":2.012},"last":0.70499999999999996}]},{"osd":4,"last update":"Sat Sep 23 03:28:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.252,"15min":1.2609999999999999},"min":{"1min":0.73599999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.621,"5min":1.885,"15min":2.4140000000000001},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2270000000000001,"15min":1.2609999999999999},"min":{"1min":0.83199999999999996,"5min":0.73999999999999999,"15min":0.65300000000000002},"max":{"1min":1.5309999999999999,"5min":1.845,"15min":3.0230000000000001},"last":1.506}]},{"osd":5,"last update":"Sat Sep 23 03:28:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.296,"15min":1.274},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.65600000000000003},"max":{"1min":1.9530000000000001,"5min":2.1240000000000001,"15min":2.5489999999999999},"last":1.6890000000000001},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3320000000000001,"15min":1.3380000000000001},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.7210000000000001,"5min":1.9159999999999999,"15min":2.1539999999999999},"last":1.2729999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:28:30 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3149999999999999,"15min":1.325},"min":{"1min":0.78900000000000003,"5min":0.77700000000000002,"15min":0.75600000000000001},"max":{"1min":1.8600000000000001,"5min":2.0870000000000002,"15min":2.9489999999999998},"last":1.5660000000000001},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.3839999999999999,"15min":1.3859999999999999},"min":{"1min":0.71199999999999997,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.8100000000000001,"5min":1.9670000000000001,"15min":2.3239999999999998},"last":1.151}]},{"osd":7,"last update":"Sat Sep 23 03:28:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.3420000000000001,"15min":1.3640000000000001},"min":{"1min":0.93200000000000005,"5min":0.752,"15min":0.747},"max":{"1min":1.722,"5min":2.052,"15min":2.2330000000000001},"last":1.8580000000000001},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.405,"15min":1.4359999999999999},"min":{"1min":1.0960000000000001,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.7889999999999999,"5min":2.1419999999999999,"15min":2.3330000000000002},"last":1.798}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-23T03:28:55.416 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-23T03:28:55.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.623+0000 7fd95f044640 1 Processor -- start 2023-09-23T03:28:55.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.623+0000 7fd95f044640 1 -- start start 2023-09-23T03:28:55.625 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.625+0000 7fd95f044640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fd9580cc6c0 0x7fd9580ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:55.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.625+0000 7fd95f044640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9580cb090 con 0x7fd9580b28a0 2023-09-23T03:28:55.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.625+0000 7fd95f044640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9580cad60 con 0x7fd95815cc00 2023-09-23T03:28:55.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.625+0000 7fd95f044640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9581615a0 con 0x7fd9580cc6c0 2023-09-23T03:28:55.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.625+0000 7fd957fff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fd9580b28a0 0x7fd9580cb520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:49468/0 (socket says 172.21.15.154:49468) 2023-09-23T03:28:55.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.625+0000 7fd957fff640 1 -- 172.21.15.154:0/693593790 learned_addr learned my addr 172.21.15.154:0/693593790 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:28:55.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.625+0000 7fd95d5e1640 1 --2- 172.21.15.154:0/693593790 >> v2:172.21.15.154:3300/0 conn(0x7fd9580cc6c0 0x7fd9580ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:55.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.626+0000 7fd9577fe640 1 -- 172.21.15.154:0/693593790 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2203386417 0 0) 0x7fd9580cb090 con 0x7fd9580b28a0 2023-09-23T03:28:55.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.626+0000 7fd9577fe640 1 -- 172.21.15.154:0/693593790 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd948003520 con 0x7fd9580b28a0 2023-09-23T03:28:55.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.626+0000 7fd95d5e1640 1 -- 172.21.15.154:0/693593790 >> v1:172.21.15.190:6789/0 conn(0x7fd95815cc00 legacy=0x7fd9580b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:55.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.626+0000 7fd95d5e1640 1 -- 172.21.15.154:0/693593790 >> v1:172.21.15.154:6789/0 conn(0x7fd9580b28a0 legacy=0x7fd9580cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:55.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.626+0000 7fd95d5e1640 1 -- 172.21.15.154:0/693593790 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd958160ff0 con 0x7fd9580cc6c0 2023-09-23T03:28:55.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.627+0000 7fd95d5e1640 1 --2- 172.21.15.154:0/693593790 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9580cc6c0 0x7fd9580ce4f0 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7fd950009d90 tx=0x7fd95002bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36245ea4c2f28588 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:55.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.627+0000 7fd9577fe640 1 -- 172.21.15.154:0/693593790 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95002ccf0 con 0x7fd9580cc6c0 2023-09-23T03:28:55.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.627+0000 7fd9577fe640 1 -- 172.21.15.154:0/693593790 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd95002ce50 con 0x7fd9580cc6c0 2023-09-23T03:28:55.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.627+0000 7fd9577fe640 1 -- 172.21.15.154:0/693593790 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd950034660 con 0x7fd9580cc6c0 2023-09-23T03:28:55.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.628+0000 7fd95f044640 1 -- 172.21.15.154:0/693593790 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9580cc6c0 msgr2=0x7fd9580ce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:55.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.628+0000 7fd95f044640 1 --2- 172.21.15.154:0/693593790 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9580cc6c0 0x7fd9580ce4f0 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7fd950009d90 tx=0x7fd95002bf40 comp rx=0 tx=0).stop 2023-09-23T03:28:55.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.628+0000 7fd95f044640 1 -- 172.21.15.154:0/693593790 shutdown_connections 2023-09-23T03:28:55.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.628+0000 7fd95f044640 1 --2- 172.21.15.154:0/693593790 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9580cc6c0 0x7fd9580ce4f0 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:55.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.628+0000 7fd95f044640 1 -- 172.21.15.154:0/693593790 >> 172.21.15.154:0/693593790 conn(0x7fd9580cd190 msgr2=0x7fd9580cd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:55.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.628+0000 7fd95f044640 1 -- 172.21.15.154:0/693593790 shutdown_connections 2023-09-23T03:28:55.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.628+0000 7fd95f044640 1 -- 172.21.15.154:0/693593790 wait complete. 2023-09-23T03:28:55.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.630+0000 7fd95f044640 1 Processor -- start 2023-09-23T03:28:55.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.630+0000 7fd95f044640 1 -- start start 2023-09-23T03:28:55.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.631+0000 7fd95f044640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9580b28a0 0x7fd9581f5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:55.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.631+0000 7fd95f044640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9580cc6c0 0x7fd958157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:55.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.631+0000 7fd95f044640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd95815cc00 0x7fd958157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:55.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.631+0000 7fd95f044640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9580ca310 con 0x7fd9580b28a0 2023-09-23T03:28:55.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.631+0000 7fd95f044640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd958157db0 con 0x7fd95815cc00 2023-09-23T03:28:55.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.631+0000 7fd95f044640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9580c9690 con 0x7fd9580cc6c0 2023-09-23T03:28:55.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.631+0000 7fd95cde0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9580b28a0 0x7fd9581f5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:55.632 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.631+0000 7fd95cde0640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9580b28a0 0x7fd9581f5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44784/0 (socket says 172.21.15.154:44784) 2023-09-23T03:28:55.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.631+0000 7fd95cde0640 1 -- 172.21.15.154:0/1356763603 learned_addr learned my addr 172.21.15.154:0/1356763603 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:55.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.632+0000 7fd95d5e1640 1 --2- 172.21.15.154:0/1356763603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd95815cc00 0x7fd958157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:55.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.632+0000 7fd957fff640 1 --2- 172.21.15.154:0/1356763603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9580cc6c0 0x7fd958157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:55.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.632+0000 7fd95cde0640 1 -- 172.21.15.154:0/1356763603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9580cc6c0 msgr2=0x7fd958157330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:55.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.632+0000 7fd95cde0640 1 --2- 172.21.15.154:0/1356763603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9580cc6c0 0x7fd958157330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:55.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.632+0000 7fd95cde0640 1 -- 172.21.15.154:0/1356763603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd95815cc00 msgr2=0x7fd958157870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:55.634 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.632+0000 7fd95cde0640 1 --2- 172.21.15.154:0/1356763603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd95815cc00 0x7fd958157870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:55.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.632+0000 7fd95cde0640 1 -- 172.21.15.154:0/1356763603 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd950009a70 con 0x7fd9580b28a0 2023-09-23T03:28:55.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.632+0000 7fd957fff640 1 --2- 172.21.15.154:0/1356763603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9580cc6c0 0x7fd958157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:55.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.632+0000 7fd95cde0640 1 --2- 172.21.15.154:0/1356763603 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9580b28a0 0x7fd9581f5bf0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7fd94c00c400 tx=0x7fd94c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:55.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.632+0000 7fd9557fa640 1 -- 172.21.15.154:0/1356763603 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94c0052d0 con 0x7fd9580b28a0 2023-09-23T03:28:55.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.632+0000 7fd9557fa640 1 -- 172.21.15.154:0/1356763603 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd94c005430 con 0x7fd9580b28a0 2023-09-23T03:28:55.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.632+0000 7fd9557fa640 1 -- 172.21.15.154:0/1356763603 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94c003830 con 0x7fd9580b28a0 2023-09-23T03:28:55.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.632+0000 7fd95f044640 1 -- 172.21.15.154:0/1356763603 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9580c9970 con 0x7fd9580b28a0 2023-09-23T03:28:55.635 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.633+0000 7fd95f044640 1 -- 172.21.15.154:0/1356763603 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9580c9d00 con 0x7fd9580b28a0 2023-09-23T03:28:55.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.634+0000 7fd95f044640 1 -- 172.21.15.154:0/1356763603 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd958073eb0 con 0x7fd9580b28a0 2023-09-23T03:28:55.636 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.636+0000 7fd9557fa640 1 -- 172.21.15.154:0/1356763603 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fd94c003990 con 0x7fd9580b28a0 2023-09-23T03:28:55.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.637+0000 7fd9557fa640 1 --2- 172.21.15.154:0/1356763603 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd93c06af30 0x7fd93c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:55.637 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.637+0000 7fd9557fa640 1 -- 172.21.15.154:0/1356763603 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fd94c088f20 con 0x7fd9580b28a0 2023-09-23T03:28:55.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.637+0000 7fd957fff640 1 --2- 172.21.15.154:0/1356763603 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd93c06af30 0x7fd93c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:55.638 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.638+0000 7fd957fff640 1 --2- 172.21.15.154:0/1356763603 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd93c06af30 0x7fd93c06d3f0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd9580afef0 tx=0x7fd944009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:55.640 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.640+0000 7fd9557fa640 1 -- 172.21.15.154:0/1356763603 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fd94c0541e0 con 0x7fd9580b28a0 2023-09-23T03:28:55.834 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.833+0000 7fd95f044640 1 -- 172.21.15.154:0/1356763603 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9580b6690 con 0x7fd9580b28a0 2023-09-23T03:28:55.835 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.834+0000 7fd9557fa640 1 -- 172.21.15.154:0/1356763603 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+13503 (secure 0 0 0) 0x7fd94c057800 con 0x7fd9580b28a0 2023-09-23T03:28:55.835 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:28:55.835 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":131,"fsid":"8c8039d3-6c84-43db-8808-85e0dbdd47c8","created":"2023-09-23T03:14:56.081056+0000","modified":"2023-09-23T03:28:51.917195+0000","last_up_change":"2023-09-23T03:15:08.448598+0000","last_in_change":"2023-09-23T03:14:59.536492+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-23T03:15:09.236980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"995eaf65-9eef-4aa4-93e6-39576cf4b85f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6801","nonce":2920938138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6803","nonce":2920938138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6807","nonce":2920938138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2920938138},{"type":"v1","addr":"172.21.15.154:6805","nonce":2920938138}]},"public_addr":"172.21.15.154:6801/2920938138","cluster_addr":"172.21.15.154:6803/2920938138","heartbeat_back_addr":"172.21.15.154:6807/2920938138","heartbeat_front_addr":"172.21.15.154:6805/2920938138","state":["exists","up"]},{"osd":1,"uuid":"409c115c-d865-492f-85b2-d9bc90f1a296","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6817","nonce":1267710005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6819","nonce":1267710005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6823","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6824","nonce":1267710005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1267710005},{"type":"v1","addr":"172.21.15.154:6821","nonce":1267710005}]},"public_addr":"172.21.15.154:6817/1267710005","cluster_addr":"172.21.15.154:6819/1267710005","heartbeat_back_addr":"172.21.15.154:6824/1267710005","heartbeat_front_addr":"172.21.15.154:6821/1267710005","state":["exists","up"]},{"osd":2,"uuid":"9e99c1ee-098c-40d9-b017-530bc1140196","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6825","nonce":2884407243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6827","nonce":2884407243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6831","nonce":2884407243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":2884407243},{"type":"v1","addr":"172.21.15.154:6829","nonce":2884407243}]},"public_addr":"172.21.15.154:6825/2884407243","cluster_addr":"172.21.15.154:6827/2884407243","heartbeat_back_addr":"172.21.15.154:6831/2884407243","heartbeat_front_addr":"172.21.15.154:6829/2884407243","state":["exists","up"]},{"osd":3,"uuid":"ac0e7a15-dbce-4197-af2e-396aabbb8de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6809","nonce":1133688468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6811","nonce":1133688468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6815","nonce":1133688468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1133688468},{"type":"v1","addr":"172.21.15.154:6813","nonce":1133688468}]},"public_addr":"172.21.15.154:6809/1133688468","cluster_addr":"172.21.15.154:6811/1133688468","heartbeat_back_addr":"172.21.15.154:6815/1133688468","heartbeat_front_addr":"172.21.15.154:6813/1133688468","state":["exists","up"]},{"osd":4,"uuid":"d183a34d-8694-44b7-a523-785255ba1869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6808","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6809","nonce":717302641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6810","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6811","nonce":717302641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6814","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6815","nonce":717302641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6812","nonce":717302641},{"type":"v1","addr":"172.21.15.190:6813","nonce":717302641}]},"public_addr":"172.21.15.190:6809/717302641","cluster_addr":"172.21.15.190:6811/717302641","heartbeat_back_addr":"172.21.15.190:6815/717302641","heartbeat_front_addr":"172.21.15.190:6813/717302641","state":["exists","up"]},{"osd":5,"uuid":"461ae909-2278-4b66-b35c-6a32ea06fbc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6800","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6801","nonce":882536885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6802","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6803","nonce":882536885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6806","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6807","nonce":882536885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6804","nonce":882536885},{"type":"v1","addr":"172.21.15.190:6805","nonce":882536885}]},"public_addr":"172.21.15.190:6801/882536885","cluster_addr":"172.21.15.190:6803/882536885","heartbeat_back_addr":"172.21.15.190:6807/882536885","heartbeat_front_addr":"172.21.15.190:6805/882536885","state":["exists","up"]},{"osd":6,"uuid":"579b92c0-3741-4058-9173-eb780c598577","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6816","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6817","nonce":1964294001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6818","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6819","nonce":1964294001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6822","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6823","nonce":1964294001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6820","nonce":1964294001},{"type":"v1","addr":"172.21.15.190:6821","nonce":1964294001}]},"public_addr":"172.21.15.190:6817/1964294001","cluster_addr":"172.21.15.190:6819/1964294001","heartbeat_back_addr":"172.21.15.190:6823/1964294001","heartbeat_front_addr":"172.21.15.190:6821/1964294001","state":["exists","up"]},{"osd":7,"uuid":"14ad9957-5ab0-43b2-8a1d-be664cbb9756","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6824","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6825","nonce":3460136272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6826","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6827","nonce":3460136272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6830","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6831","nonce":3460136272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6828","nonce":3460136272},{"type":"v1","addr":"172.21.15.190:6829","nonce":3460136272}]},"public_addr":"172.21.15.190:6825/3460136272","cluster_addr":"172.21.15.190:6827/3460136272","heartbeat_back_addr":"172.21.15.190:6831/3460136272","heartbeat_front_addr":"172.21.15.190:6829/3460136272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:05.049838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.625128+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.926231+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.885431+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.851103+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.845290+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.888643+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-23T03:15:04.582683+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.154:6837/1250699891":"2023-09-24T03:28:48.897069+0000","172.21.15.154:6836/1250699891":"2023-09-24T03:28:48.897069+0000","172.21.15.154:6837/4006549651":"2023-09-24T03:28:29.295137+0000","172.21.15.154:6836/4006549651":"2023-09-24T03:28:29.295137+0000","172.21.15.190:6835/3558096060":"2023-09-24T03:28:08.468628+0000","172.21.15.190:6833/3558096060":"2023-09-24T03:28:08.468628+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-23T03:28:55.837 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.837+0000 7fd95f044640 1 -- 172.21.15.154:0/1356763603 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd93c06af30 msgr2=0x7fd93c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:55.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.837+0000 7fd95f044640 1 --2- 172.21.15.154:0/1356763603 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd93c06af30 0x7fd93c06d3f0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd9580afef0 tx=0x7fd944009040 comp rx=0 tx=0).stop 2023-09-23T03:28:55.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.837+0000 7fd95f044640 1 -- 172.21.15.154:0/1356763603 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9580b28a0 msgr2=0x7fd9581f5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:55.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.837+0000 7fd95f044640 1 --2- 172.21.15.154:0/1356763603 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9580b28a0 0x7fd9581f5bf0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7fd94c00c400 tx=0x7fd94c00c8c0 comp rx=0 tx=0).stop 2023-09-23T03:28:55.838 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.837+0000 7fd95f044640 1 -- 172.21.15.154:0/1356763603 shutdown_connections 2023-09-23T03:28:55.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.837+0000 7fd95f044640 1 --2- 172.21.15.154:0/1356763603 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd93c06af30 0x7fd93c06d3f0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:55.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.837+0000 7fd95f044640 1 --2- 172.21.15.154:0/1356763603 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd95815cc00 0x7fd958157870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:55.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.838+0000 7fd95f044640 1 --2- 172.21.15.154:0/1356763603 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd9580cc6c0 0x7fd958157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:55.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.838+0000 7fd95f044640 1 --2- 172.21.15.154:0/1356763603 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd9580b28a0 0x7fd9581f5bf0 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:55.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.838+0000 7fd95f044640 1 -- 172.21.15.154:0/1356763603 >> 172.21.15.154:0/1356763603 conn(0x7fd9580cd190 msgr2=0x7fd9580b0110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:55.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.838+0000 7fd95f044640 1 -- 172.21.15.154:0/1356763603 shutdown_connections 2023-09-23T03:28:55.839 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:55.838+0000 7fd95f044640 1 -- 172.21.15.154:0/1356763603 wait complete. 2023-09-23T03:28:56.856 INFO:tasks.ceph:Scrubbing osd.0 2023-09-23T03:28:56.856 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-09-23T03:28:57.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.086+0000 7f9a3dbe7640 1 Processor -- start 2023-09-23T03:28:57.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.087+0000 7f9a3dbe7640 1 -- start start 2023-09-23T03:28:57.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.088+0000 7f9a3dbe7640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9a380ca6b0 0x7f9a3806d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:57.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.088+0000 7f9a3dbe7640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a3815ab20 con 0x7f9a3806dbd0 2023-09-23T03:28:57.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.088+0000 7f9a3dbe7640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a38159f50 con 0x7f9a380707f0 2023-09-23T03:28:57.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.088+0000 7f9a3dbe7640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a380caa90 con 0x7f9a380ca6b0 2023-09-23T03:28:57.089 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.089+0000 7f9a377fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f9a380ca6b0 0x7f9a3806d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:57.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.089+0000 7f9a2ffff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f9a3806dbd0 0x7f9a380700e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:49478/0 (socket says 172.21.15.154:49478) 2023-09-23T03:28:57.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.089+0000 7f9a2ffff640 1 -- 172.21.15.154:0/584788601 learned_addr learned my addr 172.21.15.154:0/584788601 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:28:57.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.089+0000 7f9a36ffd640 1 -- 172.21.15.154:0/584788601 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2389389236 0 0) 0x7f9a3815ab20 con 0x7f9a3806dbd0 2023-09-23T03:28:57.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.089+0000 7f9a36ffd640 1 -- 172.21.15.154:0/584788601 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a24003520 con 0x7f9a3806dbd0 2023-09-23T03:28:57.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.089+0000 7f9a377fe640 1 -- 172.21.15.154:0/584788601 >> v1:172.21.15.190:6789/0 conn(0x7f9a380707f0 legacy=0x7f9a380b2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:57.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.089+0000 7f9a377fe640 1 -- 172.21.15.154:0/584788601 >> v1:172.21.15.154:6789/0 conn(0x7f9a3806dbd0 legacy=0x7f9a380700e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:57.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.089+0000 7f9a377fe640 1 -- 172.21.15.154:0/584788601 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a3815a1f0 con 0x7f9a380ca6b0 2023-09-23T03:28:57.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.090+0000 7f9a377fe640 1 --2- 172.21.15.154:0/584788601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a380ca6b0 0x7f9a3806d690 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f9a280097a0 tx=0x7f9a2802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f66937061195c8a0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:57.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.091+0000 7f9a36ffd640 1 -- 172.21.15.154:0/584788601 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a28031830 con 0x7f9a380ca6b0 2023-09-23T03:28:57.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.091+0000 7f9a36ffd640 1 -- 172.21.15.154:0/584788601 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a28031990 con 0x7f9a380ca6b0 2023-09-23T03:28:57.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.091+0000 7f9a36ffd640 1 -- 172.21.15.154:0/584788601 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a28031d20 con 0x7f9a380ca6b0 2023-09-23T03:28:57.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.091+0000 7f9a3dbe7640 1 -- 172.21.15.154:0/584788601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a380ca6b0 msgr2=0x7f9a3806d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:57.093 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.091+0000 7f9a3dbe7640 1 --2- 172.21.15.154:0/584788601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a380ca6b0 0x7f9a3806d690 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f9a280097a0 tx=0x7f9a2802b8d0 comp rx=0 tx=0).stop 2023-09-23T03:28:57.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.091+0000 7f9a3dbe7640 1 -- 172.21.15.154:0/584788601 shutdown_connections 2023-09-23T03:28:57.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.091+0000 7f9a3dbe7640 1 --2- 172.21.15.154:0/584788601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a380ca6b0 0x7f9a3806d690 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:57.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.091+0000 7f9a3dbe7640 1 -- 172.21.15.154:0/584788601 >> 172.21.15.154:0/584788601 conn(0x7f9a3806a440 msgr2=0x7f9a380b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:57.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.091+0000 7f9a3dbe7640 1 -- 172.21.15.154:0/584788601 shutdown_connections 2023-09-23T03:28:57.094 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.091+0000 7f9a3dbe7640 1 -- 172.21.15.154:0/584788601 wait complete. 2023-09-23T03:28:57.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.093+0000 7f9a3dbe7640 1 Processor -- start 2023-09-23T03:28:57.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.093+0000 7f9a3dbe7640 1 -- start start 2023-09-23T03:28:57.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.094+0000 7f9a3dbe7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a3806dbd0 0x7f9a380cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:57.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.094+0000 7f9a3dbe7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a380707f0 0x7f9a38158930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:57.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.094+0000 7f9a3dbe7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a380ca6b0 0x7f9a3814eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:57.097 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.094+0000 7f9a3dbe7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3814ff00 con 0x7f9a3806dbd0 2023-09-23T03:28:57.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.094+0000 7f9a3dbe7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3814f940 con 0x7f9a380707f0 2023-09-23T03:28:57.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.094+0000 7f9a3dbe7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3814fc20 con 0x7f9a380ca6b0 2023-09-23T03:28:57.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.095+0000 7f9a377fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a3806dbd0 0x7f9a380cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:57.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.095+0000 7f9a377fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a3806dbd0 0x7f9a380cbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44800/0 (socket says 172.21.15.154:44800) 2023-09-23T03:28:57.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.095+0000 7f9a377fe640 1 -- 172.21.15.154:0/3738674107 learned_addr learned my addr 172.21.15.154:0/3738674107 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:57.098 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.095+0000 7f9a2ffff640 1 --2- 172.21.15.154:0/3738674107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a380707f0 0x7f9a38158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:57.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.095+0000 7f9a37fff640 1 --2- 172.21.15.154:0/3738674107 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a380ca6b0 0x7f9a3814eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:57.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.095+0000 7f9a377fe640 1 -- 172.21.15.154:0/3738674107 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a380ca6b0 msgr2=0x7f9a3814eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:57.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.095+0000 7f9a377fe640 1 --2- 172.21.15.154:0/3738674107 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a380ca6b0 0x7f9a3814eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:57.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.095+0000 7f9a377fe640 1 -- 172.21.15.154:0/3738674107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a380707f0 msgr2=0x7f9a38158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:57.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.095+0000 7f9a377fe640 1 --2- 172.21.15.154:0/3738674107 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f9a380707f0 0x7f9a38158930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:57.099 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.095+0000 7f9a377fe640 1 -- 172.21.15.154:0/3738674107 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a28009400 con 0x7f9a3806dbd0 2023-09-23T03:28:57.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.095+0000 7f9a37fff640 1 --2- 172.21.15.154:0/3738674107 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f9a380ca6b0 0x7f9a3814eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:57.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.096+0000 7f9a377fe640 1 --2- 172.21.15.154:0/3738674107 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a3806dbd0 0x7f9a380cbce0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f9a280098c0 tx=0x7f9a280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:57.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.096+0000 7f9a34ff9640 1 -- 172.21.15.154:0/3738674107 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2803a6f0 con 0x7f9a3806dbd0 2023-09-23T03:28:57.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.096+0000 7f9a34ff9640 1 -- 172.21.15.154:0/3738674107 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a2803a870 con 0x7f9a3806dbd0 2023-09-23T03:28:57.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.096+0000 7f9a3dbe7640 1 -- 172.21.15.154:0/3738674107 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a38151500 con 0x7f9a3806dbd0 2023-09-23T03:28:57.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.096+0000 7f9a34ff9640 1 -- 172.21.15.154:0/3738674107 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2803ac20 con 0x7f9a3806dbd0 2023-09-23T03:28:57.100 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.096+0000 7f9a3dbe7640 1 -- 172.21.15.154:0/3738674107 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a380cc220 con 0x7f9a3806dbd0 2023-09-23T03:28:57.101 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.097+0000 7f9a3dbe7640 1 -- 172.21.15.154:0/3738674107 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9a3815a7b0 con 0x7f9a3806dbd0 2023-09-23T03:28:57.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.098+0000 7f9a34ff9640 1 -- 172.21.15.154:0/3738674107 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f9a2804c3b0 con 0x7f9a3806dbd0 2023-09-23T03:28:57.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.099+0000 7f9a34ff9640 1 --2- 172.21.15.154:0/3738674107 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9a1806af30 0x7f9a1806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:57.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.099+0000 7f9a34ff9640 1 -- 172.21.15.154:0/3738674107 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f9a280ab8b0 con 0x7f9a3806dbd0 2023-09-23T03:28:57.102 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.099+0000 7f9a2ffff640 1 --2- 172.21.15.154:0/3738674107 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9a1806af30 0x7f9a1806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:57.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.099+0000 7f9a34ff9640 1 --2- 172.21.15.154:0/3738674107 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x7f9a18071370 0x7f9a180737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:57.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.099+0000 7f9a34ff9640 1 -- 172.21.15.154:0/3738674107 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9a18073e80 con 0x7f9a18071370 2023-09-23T03:28:57.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.099+0000 7f9a34ff9640 1 -- 172.21.15.154:0/3738674107 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=131) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a280abc20 con 0x7f9a3806dbd0 2023-09-23T03:28:57.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.100+0000 7f9a37fff640 1 --2- 172.21.15.154:0/3738674107 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x7f9a18071370 0x7f9a180737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:57.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.100+0000 7f9a2ffff640 1 --2- 172.21.15.154:0/3738674107 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9a1806af30 0x7f9a1806d3f0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f9a200011e0 tx=0x7f9a20009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:57.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.100+0000 7f9a37fff640 1 --2- 172.21.15.154:0/3738674107 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x7f9a18071370 0x7f9a180737b0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:57.103 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.101+0000 7f9a34ff9640 1 -- 172.21.15.154:0/3738674107 <== osd.0 v2:172.21.15.154:6800/2920938138 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f9a18073e80 con 0x7f9a18071370 2023-09-23T03:28:57.115 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.115+0000 7f9a3dbe7640 1 -- 172.21.15.154:0/3738674107 --> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9a3815a4d0 con 0x7f9a18071370 2023-09-23T03:28:57.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.116+0000 7f9a34ff9640 1 -- 172.21.15.154:0/3738674107 <== osd.0 v2:172.21.15.154:6800/2920938138 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f9a30004130 con 0x7f9a18071370 2023-09-23T03:28:57.116 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:28:57.116 INFO:teuthology.orchestra.run.smithi154.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-23T03:28:57.117 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:28:57.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.116+0000 7f9a3dbe7640 1 -- 172.21.15.154:0/3738674107 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x7f9a18071370 msgr2=0x7f9a180737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:57.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.116+0000 7f9a3dbe7640 1 --2- 172.21.15.154:0/3738674107 >> [v2:172.21.15.154:6800/2920938138,v1:172.21.15.154:6801/2920938138] conn(0x7f9a18071370 0x7f9a180737b0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:57.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.116+0000 7f9a3dbe7640 1 -- 172.21.15.154:0/3738674107 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9a1806af30 msgr2=0x7f9a1806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:57.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.116+0000 7f9a3dbe7640 1 --2- 172.21.15.154:0/3738674107 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f9a1806af30 0x7f9a1806d3f0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f9a200011e0 tx=0x7f9a20009040 comp rx=0 tx=0).stop 2023-09-23T03:28:57.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.116+0000 7f9a3dbe7640 1 -- 172.21.15.154:0/3738674107 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a3806dbd0 msgr2=0x7f9a380cbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:57.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.116+0000 7f9a3dbe7640 1 --2- 172.21.15.154:0/3738674107 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a3806dbd0 0x7f9a380cbce0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f9a280098c0 tx=0x7f9a280046c0 comp rx=0 tx=0).stop 2023-09-23T03:28:57.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.116+0000 7f9a37fff640 1 -- 172.21.15.154:0/3738674107 reap_dead start 2023-09-23T03:28:57.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.116+0000 7f9a3dbe7640 1 -- 172.21.15.154:0/3738674107 shutdown_connections 2023-09-23T03:28:57.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.116+0000 7f9a3dbe7640 1 -- 172.21.15.154:0/3738674107 >> 172.21.15.154:0/3738674107 conn(0x7f9a3806a440 msgr2=0x7f9a38157e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:57.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.117+0000 7f9a3dbe7640 1 -- 172.21.15.154:0/3738674107 shutdown_connections 2023-09-23T03:28:57.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.117+0000 7f9a3dbe7640 1 -- 172.21.15.154:0/3738674107 wait complete. 2023-09-23T03:28:57.134 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-09-23T03:28:57.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.348+0000 7f00d2d2c640 1 Processor -- start 2023-09-23T03:28:57.349 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.349+0000 7f00d2d2c640 1 -- start start 2023-09-23T03:28:57.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.350+0000 7f00d2d2c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00cc15a640 0x7f00cc0b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:57.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.350+0000 7f00d2d2c640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00cc0cd140 con 0x7f00cc0b43a0 2023-09-23T03:28:57.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.350+0000 7f00d2d2c640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00cc0cc530 con 0x7f00cc0cb6c0 2023-09-23T03:28:57.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.350+0000 7f00d2d2c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc0cc810 con 0x7f00cc15a640 2023-09-23T03:28:57.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.351+0000 7f00cbfff640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f00cc0b43a0 0x7f00cc0cafb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:49484/0 (socket says 172.21.15.154:49484) 2023-09-23T03:28:57.351 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.351+0000 7f00cbfff640 1 -- 172.21.15.154:0/3330369442 learned_addr learned my addr 172.21.15.154:0/3330369442 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:28:57.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.351+0000 7f00d0ac8640 1 --2- 172.21.15.154:0/3330369442 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00cc15a640 0x7f00cc0b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:57.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.351+0000 7f00cb7fe640 1 -- 172.21.15.154:0/3330369442 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3200556652 0 0) 0x7f00cc0cd140 con 0x7f00cc0b43a0 2023-09-23T03:28:57.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.351+0000 7f00cb7fe640 1 -- 172.21.15.154:0/3330369442 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00b4003520 con 0x7f00cc0b43a0 2023-09-23T03:28:57.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.352+0000 7f00cb7fe640 1 -- 172.21.15.154:0/3330369442 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 409256820 0 0) 0x7f00b4003520 con 0x7f00cc0b43a0 2023-09-23T03:28:57.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.352+0000 7f00cb7fe640 1 -- 172.21.15.154:0/3330369442 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00cc15a640 msgr2=0x7f00cc0b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:57.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.352+0000 7f00cb7fe640 1 --2- 172.21.15.154:0/3330369442 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00cc15a640 0x7f00cc0b3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:57.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.352+0000 7f00cb7fe640 1 -- 172.21.15.154:0/3330369442 >> v1:172.21.15.190:6789/0 conn(0x7f00cc0cb6c0 legacy=0x7f00cc0b2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:57.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.352+0000 7f00cb7fe640 1 -- 172.21.15.154:0/3330369442 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00cc0ccaf0 con 0x7f00cc0b43a0 2023-09-23T03:28:57.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.352+0000 7f00cb7fe640 1 -- 172.21.15.154:0/3330369442 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f00c00019f0 con 0x7f00cc0b43a0 2023-09-23T03:28:57.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.352+0000 7f00cb7fe640 1 -- 172.21.15.154:0/3330369442 >> v1:172.21.15.154:6789/0 conn(0x7f00cc0b43a0 legacy=0x7f00cc0cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:57.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.352+0000 7f00cb7fe640 1 --2- 172.21.15.154:0/3330369442 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00cc15a640 0x7f00b4003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:57.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.352+0000 7f00cb7fe640 1 --2- 172.21.15.154:0/3330369442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00b4003f90 0x7f00b40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:57.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.352+0000 7f00cb7fe640 1 --2- 172.21.15.154:0/3330369442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00b4004d00 0x7f00b4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:57.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.352+0000 7f00cb7fe640 1 -- 172.21.15.154:0/3330369442 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc0cc810 con 0x7f00b4003f90 2023-09-23T03:28:57.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.352+0000 7f00cb7fe640 1 -- 172.21.15.154:0/3330369442 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b400bb70 con 0x7f00b4004d00 2023-09-23T03:28:57.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.352+0000 7f00cb7fe640 1 -- 172.21.15.154:0/3330369442 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00b400bd30 con 0x7f00cc15a640 2023-09-23T03:28:57.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.352+0000 7f00cbfff640 1 --2- 172.21.15.154:0/3330369442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00b4003f90 0x7f00b40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:57.355 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.352+0000 7f00d0ac8640 1 --2- 172.21.15.154:0/3330369442 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00cc15a640 0x7f00b4003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:57.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.353+0000 7f00cbfff640 1 -- 172.21.15.154:0/3330369442 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00cc15a640 msgr2=0x7f00b4003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:57.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.352+0000 7f00d12c9640 1 --2- 172.21.15.154:0/3330369442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00b4004d00 0x7f00b4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:57.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.353+0000 7f00cbfff640 1 --2- 172.21.15.154:0/3330369442 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00cc15a640 0x7f00b4003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:57.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.353+0000 7f00cbfff640 1 -- 172.21.15.154:0/3330369442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00b4004d00 msgr2=0x7f00b4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:57.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.353+0000 7f00cbfff640 1 --2- 172.21.15.154:0/3330369442 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00b4004d00 0x7f00b4005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:57.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.353+0000 7f00cbfff640 1 -- 172.21.15.154:0/3330369442 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b4003520 con 0x7f00b4003f90 2023-09-23T03:28:57.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.354+0000 7f00cbfff640 1 --2- 172.21.15.154:0/3330369442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00b4003f90 0x7f00b40047c0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f00c000a830 tx=0x7f00b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6bed7f05bd11c1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:57.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.354+0000 7f00cb7fe640 1 -- 172.21.15.154:0/3330369442 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c00072d0 con 0x7f00b4003f90 2023-09-23T03:28:57.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.354+0000 7f00cb7fe640 1 -- 172.21.15.154:0/3330369442 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c002f070 con 0x7f00b4003f90 2023-09-23T03:28:57.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.354+0000 7f00cb7fe640 1 -- 172.21.15.154:0/3330369442 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c00076f0 con 0x7f00b4003f90 2023-09-23T03:28:57.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.354+0000 7f00d2d2c640 1 -- 172.21.15.154:0/3330369442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00b4003f90 msgr2=0x7f00b40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:57.359 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.354+0000 7f00d2d2c640 1 --2- 172.21.15.154:0/3330369442 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00b4003f90 0x7f00b40047c0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f00c000a830 tx=0x7f00b40057f0 comp rx=0 tx=0).stop 2023-09-23T03:28:57.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.354+0000 7f00d12c9640 1 -- 172.21.15.154:0/3330369442 reap_dead start 2023-09-23T03:28:57.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.354+0000 7f00d2d2c640 1 -- 172.21.15.154:0/3330369442 shutdown_connections 2023-09-23T03:28:57.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.354+0000 7f00d2d2c640 1 -- 172.21.15.154:0/3330369442 >> 172.21.15.154:0/3330369442 conn(0x7f00cc06c820 msgr2=0x7f00cc069cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:57.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.355+0000 7f00d2d2c640 1 -- 172.21.15.154:0/3330369442 shutdown_connections 2023-09-23T03:28:57.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.355+0000 7f00d2d2c640 1 -- 172.21.15.154:0/3330369442 wait complete. 2023-09-23T03:28:57.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.356+0000 7f00d2d2c640 1 Processor -- start 2023-09-23T03:28:57.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.356+0000 7f00d2d2c640 1 -- start start 2023-09-23T03:28:57.360 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.357+0000 7f00d2d2c640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00b4003f90 0x7f00cc1f2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:57.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.357+0000 7f00d2d2c640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00cc0c8270 0x7f00cc154820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:57.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.357+0000 7f00d2d2c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00cc0c86f0 0x7f00cc070650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:57.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.357+0000 7f00d2d2c640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc1eca90 con 0x7f00cc0c86f0 2023-09-23T03:28:57.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.357+0000 7f00d2d2c640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc1f2c10 con 0x7f00b4003f90 2023-09-23T03:28:57.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.357+0000 7f00d2d2c640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc1f2ef0 con 0x7f00cc0c8270 2023-09-23T03:28:57.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.358+0000 7f00d12c9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00cc0c86f0 0x7f00cc070650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:57.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.358+0000 7f00d12c9640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00cc0c86f0 0x7f00cc070650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44826/0 (socket says 172.21.15.154:44826) 2023-09-23T03:28:57.361 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.358+0000 7f00d12c9640 1 -- 172.21.15.154:0/2541845750 learned_addr learned my addr 172.21.15.154:0/2541845750 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:57.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.358+0000 7f00d0ac8640 1 --2- 172.21.15.154:0/2541845750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00b4003f90 0x7f00cc1f2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:57.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.358+0000 7f00d12c9640 1 -- 172.21.15.154:0/2541845750 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00cc0c8270 msgr2=0x7f00cc154820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:57.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.358+0000 7f00d12c9640 1 --2- 172.21.15.154:0/2541845750 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00cc0c8270 0x7f00cc154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:57.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.358+0000 7f00d12c9640 1 -- 172.21.15.154:0/2541845750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00b4003f90 msgr2=0x7f00cc1f2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:57.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.358+0000 7f00d12c9640 1 --2- 172.21.15.154:0/2541845750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00b4003f90 0x7f00cc1f2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:57.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.358+0000 7f00d12c9640 1 -- 172.21.15.154:0/2541845750 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00cc1f31d0 con 0x7f00cc0c86f0 2023-09-23T03:28:57.362 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.358+0000 7f00d12c9640 1 --2- 172.21.15.154:0/2541845750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00cc0c86f0 0x7f00cc070650 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f00cc06e3b0 tx=0x7f00c400d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:57.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.359+0000 7f00c97fa640 1 -- 172.21.15.154:0/2541845750 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c400b070 con 0x7f00cc0c86f0 2023-09-23T03:28:57.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.359+0000 7f00c97fa640 1 -- 172.21.15.154:0/2541845750 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c40054e0 con 0x7f00cc0c86f0 2023-09-23T03:28:57.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.359+0000 7f00c97fa640 1 -- 172.21.15.154:0/2541845750 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c4003810 con 0x7f00cc0c86f0 2023-09-23T03:28:57.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.359+0000 7f00d2d2c640 1 -- 172.21.15.154:0/2541845750 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00cc1f34b0 con 0x7f00cc0c86f0 2023-09-23T03:28:57.363 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.359+0000 7f00d2d2c640 1 -- 172.21.15.154:0/2541845750 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00cc154d60 con 0x7f00cc0c86f0 2023-09-23T03:28:57.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.360+0000 7f00d2d2c640 1 -- 172.21.15.154:0/2541845750 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00cc070370 con 0x7f00cc0c86f0 2023-09-23T03:28:57.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.361+0000 7f00c97fa640 1 -- 172.21.15.154:0/2541845750 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f00c4003970 con 0x7f00cc0c86f0 2023-09-23T03:28:57.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.362+0000 7f00c97fa640 1 --2- 172.21.15.154:0/2541845750 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f00b806af30 0x7f00b806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:57.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.362+0000 7f00c97fa640 1 -- 172.21.15.154:0/2541845750 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f00c4088d90 con 0x7f00cc0c86f0 2023-09-23T03:28:57.364 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.362+0000 7f00d0ac8640 1 --2- 172.21.15.154:0/2541845750 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f00b806af30 0x7f00b806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:57.365 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.362+0000 7f00d0ac8640 1 --2- 172.21.15.154:0/2541845750 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f00b806af30 0x7f00b806d3f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f00bc009950 tx=0x7f00bc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:57.366 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.366+0000 7f00c97fa640 1 -- 172.21.15.154:0/2541845750 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f00cc070370 con 0x7f00cc0c86f0 2023-09-23T03:28:57.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.556+0000 7f00d2d2c640 1 -- 172.21.15.154:0/2541845750 --> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f00cc0ccdd0 con 0x7f00b806af30 2023-09-23T03:28:57.557 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.557+0000 7f00c97fa640 1 -- 172.21.15.154:0/2541845750 <== mgr.4110 v2:172.21.15.154:6832/2422652104 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f00bc0095c0 con 0x7f00b806af30 2023-09-23T03:28:57.557 INFO:teuthology.orchestra.run.smithi154.stderr:instructed osd(s) 0 to deep-scrub 2023-09-23T03:28:57.560 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.560+0000 7f00d2d2c640 1 -- 172.21.15.154:0/2541845750 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f00b806af30 msgr2=0x7f00b806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:57.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.560+0000 7f00d2d2c640 1 --2- 172.21.15.154:0/2541845750 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f00b806af30 0x7f00b806d3f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f00bc009950 tx=0x7f00bc008040 comp rx=0 tx=0).stop 2023-09-23T03:28:57.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.560+0000 7f00d2d2c640 1 -- 172.21.15.154:0/2541845750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00cc0c86f0 msgr2=0x7f00cc070650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:57.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.560+0000 7f00d2d2c640 1 --2- 172.21.15.154:0/2541845750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00cc0c86f0 0x7f00cc070650 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f00cc06e3b0 tx=0x7f00c400d890 comp rx=0 tx=0).stop 2023-09-23T03:28:57.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.560+0000 7f00d2d2c640 1 -- 172.21.15.154:0/2541845750 shutdown_connections 2023-09-23T03:28:57.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.560+0000 7f00d2d2c640 1 --2- 172.21.15.154:0/2541845750 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f00b806af30 0x7f00b806d3f0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:57.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.560+0000 7f00d2d2c640 1 --2- 172.21.15.154:0/2541845750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f00cc0c86f0 0x7f00cc070650 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:57.561 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.560+0000 7f00d2d2c640 1 --2- 172.21.15.154:0/2541845750 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f00cc0c8270 0x7f00cc154820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:57.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.560+0000 7f00d2d2c640 1 --2- 172.21.15.154:0/2541845750 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f00b4003f90 0x7f00cc1f2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:57.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.560+0000 7f00d2d2c640 1 -- 172.21.15.154:0/2541845750 >> 172.21.15.154:0/2541845750 conn(0x7f00cc06c820 msgr2=0x7f00cc0c3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:57.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.560+0000 7f00d2d2c640 1 -- 172.21.15.154:0/2541845750 shutdown_connections 2023-09-23T03:28:57.562 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.560+0000 7f00d2d2c640 1 -- 172.21.15.154:0/2541845750 wait complete. 2023-09-23T03:28:57.579 INFO:tasks.ceph:Scrubbing osd.1 2023-09-23T03:28:57.579 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-09-23T03:28:57.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.796+0000 7fb9ef887640 1 Processor -- start 2023-09-23T03:28:57.796 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.796+0000 7fb9ef887640 1 -- start start 2023-09-23T03:28:57.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.798+0000 7fb9ef887640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb9e8144130 0x7fb9e80b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:57.798 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.798+0000 7fb9ef887640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fb9e80caf80 0x7fb9e80cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:57.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.798+0000 7fb9ef887640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9e80cb920 con 0x7fb9e815e520 2023-09-23T03:28:57.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.798+0000 7fb9ef887640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9e8070fb0 con 0x7fb9e8144130 2023-09-23T03:28:57.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.798+0000 7fb9ef887640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9e80700e0 con 0x7fb9e80caf80 2023-09-23T03:28:57.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.798+0000 7fb9ed623640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fb9e8144130 0x7fb9e80b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:57.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.798+0000 7fb9ede24640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fb9e815e520 0x7fb9e815e990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:49496/0 (socket says 172.21.15.154:49496) 2023-09-23T03:28:57.799 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.798+0000 7fb9ede24640 1 -- 172.21.15.154:0/2073445555 learned_addr learned my addr 172.21.15.154:0/2073445555 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:28:57.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.798+0000 7fb9ece22640 1 --2- 172.21.15.154:0/2073445555 >> v2:172.21.15.190:3300/0 conn(0x7fb9e80caf80 0x7fb9e80cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:57.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.799+0000 7fb9dffff640 1 -- 172.21.15.154:0/2073445555 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3325269146 0 0) 0x7fb9e80cb920 con 0x7fb9e815e520 2023-09-23T03:28:57.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.799+0000 7fb9dffff640 1 -- 172.21.15.154:0/2073445555 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9d4003520 con 0x7fb9e815e520 2023-09-23T03:28:57.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.799+0000 7fb9ed623640 1 -- 172.21.15.154:0/2073445555 >> v2:172.21.15.190:3300/0 conn(0x7fb9e80caf80 msgr2=0x7fb9e80cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:57.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.799+0000 7fb9ed623640 1 --2- 172.21.15.154:0/2073445555 >> v2:172.21.15.190:3300/0 conn(0x7fb9e80caf80 0x7fb9e80cb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:57.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.799+0000 7fb9ed623640 1 -- 172.21.15.154:0/2073445555 >> v1:172.21.15.154:6789/0 conn(0x7fb9e815e520 legacy=0x7fb9e815e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:57.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.799+0000 7fb9ed623640 1 -- 172.21.15.154:0/2073445555 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e8070330 con 0x7fb9e8144130 2023-09-23T03:28:57.801 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.800+0000 7fb9ed623640 1 --2- 172.21.15.154:0/2073445555 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9e8144130 0x7fb9e80b1b70 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fb9e80cae10 tx=0x7fb9e402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13fba1180a2c2f16 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:57.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.800+0000 7fb9dffff640 1 -- 172.21.15.154:0/2073445555 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e4031830 con 0x7fb9e8144130 2023-09-23T03:28:57.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.800+0000 7fb9dffff640 1 -- 172.21.15.154:0/2073445555 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9e4031990 con 0x7fb9e8144130 2023-09-23T03:28:57.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.800+0000 7fb9dffff640 1 -- 172.21.15.154:0/2073445555 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e4031d20 con 0x7fb9e8144130 2023-09-23T03:28:57.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.801+0000 7fb9ef887640 1 -- 172.21.15.154:0/2073445555 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9e8144130 msgr2=0x7fb9e80b1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:57.802 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.801+0000 7fb9ef887640 1 --2- 172.21.15.154:0/2073445555 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9e8144130 0x7fb9e80b1b70 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fb9e80cae10 tx=0x7fb9e402b8d0 comp rx=0 tx=0).stop 2023-09-23T03:28:57.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.801+0000 7fb9ef887640 1 -- 172.21.15.154:0/2073445555 shutdown_connections 2023-09-23T03:28:57.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.801+0000 7fb9ef887640 1 --2- 172.21.15.154:0/2073445555 >> v2:172.21.15.190:3300/0 conn(0x7fb9e80caf80 0x7fb9e80cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:57.803 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.801+0000 7fb9ef887640 1 --2- 172.21.15.154:0/2073445555 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9e8144130 0x7fb9e80b1b70 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:57.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.801+0000 7fb9ef887640 1 -- 172.21.15.154:0/2073445555 >> 172.21.15.154:0/2073445555 conn(0x7fb9e806a440 msgr2=0x7fb9e80b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:57.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.801+0000 7fb9ef887640 1 -- 172.21.15.154:0/2073445555 shutdown_connections 2023-09-23T03:28:57.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.801+0000 7fb9ef887640 1 -- 172.21.15.154:0/2073445555 wait complete. 2023-09-23T03:28:57.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.803+0000 7fb9ef887640 1 Processor -- start 2023-09-23T03:28:57.804 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.803+0000 7fb9ef887640 1 -- start start 2023-09-23T03:28:57.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.804+0000 7fb9ef887640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9e80caf80 0x7fb9e81e7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:57.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.804+0000 7fb9ef887640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9e8144130 0x7fb9e815a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:57.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.804+0000 7fb9ef887640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9e815e520 0x7fb9e80a7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:57.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.804+0000 7fb9ef887640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e815a840 con 0x7fb9e8144130 2023-09-23T03:28:57.806 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.804+0000 7fb9ef887640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e80a8340 con 0x7fb9e815e520 2023-09-23T03:28:57.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.804+0000 7fb9ef887640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e81427b0 con 0x7fb9e80caf80 2023-09-23T03:28:57.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.804+0000 7fb9ece22640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9e8144130 0x7fb9e815a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:57.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.804+0000 7fb9ece22640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9e8144130 0x7fb9e815a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44846/0 (socket says 172.21.15.154:44846) 2023-09-23T03:28:57.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.804+0000 7fb9ece22640 1 -- 172.21.15.154:0/2899955977 learned_addr learned my addr 172.21.15.154:0/2899955977 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:57.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.804+0000 7fb9ede24640 1 --2- 172.21.15.154:0/2899955977 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9e815e520 0x7fb9e80a7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:57.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.804+0000 7fb9ece22640 1 -- 172.21.15.154:0/2899955977 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9e80caf80 msgr2=0x7fb9e81e7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:57.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.804+0000 7fb9ed623640 1 --2- 172.21.15.154:0/2899955977 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9e80caf80 0x7fb9e81e7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:57.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.804+0000 7fb9ece22640 1 --2- 172.21.15.154:0/2899955977 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fb9e80caf80 0x7fb9e81e7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:57.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.804+0000 7fb9ece22640 1 -- 172.21.15.154:0/2899955977 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9e815e520 msgr2=0x7fb9e80a7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:57.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.804+0000 7fb9ece22640 1 --2- 172.21.15.154:0/2899955977 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9e815e520 0x7fb9e80a7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:57.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.805+0000 7fb9ece22640 1 -- 172.21.15.154:0/2899955977 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e4009400 con 0x7fb9e8144130 2023-09-23T03:28:57.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.805+0000 7fb9ede24640 1 --2- 172.21.15.154:0/2899955977 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fb9e815e520 0x7fb9e80a7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:57.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.805+0000 7fb9ece22640 1 --2- 172.21.15.154:0/2899955977 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9e8144130 0x7fb9e815a300 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7fb9d800d830 tx=0x7fb9d800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:57.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.805+0000 7fb9ddffb640 1 -- 172.21.15.154:0/2899955977 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d800b660 con 0x7fb9e8144130 2023-09-23T03:28:57.808 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.805+0000 7fb9ddffb640 1 -- 172.21.15.154:0/2899955977 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d800b7c0 con 0x7fb9e8144130 2023-09-23T03:28:57.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.805+0000 7fb9ddffb640 1 -- 172.21.15.154:0/2899955977 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d800bb50 con 0x7fb9e8144130 2023-09-23T03:28:57.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.805+0000 7fb9ef887640 1 -- 172.21.15.154:0/2899955977 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9e81e60e0 con 0x7fb9e8144130 2023-09-23T03:28:57.809 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.805+0000 7fb9ef887640 1 -- 172.21.15.154:0/2899955977 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e81e66a0 con 0x7fb9e8144130 2023-09-23T03:28:57.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.806+0000 7fb9ef887640 1 -- 172.21.15.154:0/2899955977 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb9e80708f0 con 0x7fb9e8144130 2023-09-23T03:28:57.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.807+0000 7fb9ddffb640 1 -- 172.21.15.154:0/2899955977 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fb9d801e460 con 0x7fb9e8144130 2023-09-23T03:28:57.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.808+0000 7fb9ddffb640 1 --2- 172.21.15.154:0/2899955977 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb9cc06af60 0x7fb9cc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:57.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.808+0000 7fb9ddffb640 1 -- 172.21.15.154:0/2899955977 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fb9d8089d40 con 0x7fb9e8144130 2023-09-23T03:28:57.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.809+0000 7fb9ed623640 1 --2- 172.21.15.154:0/2899955977 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb9cc06af60 0x7fb9cc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:57.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.809+0000 7fb9ddffb640 1 --2- 172.21.15.154:0/2899955977 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x7fb9cc0713a0 0x7fb9cc0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:57.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.809+0000 7fb9ddffb640 1 -- 172.21.15.154:0/2899955977 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb9cc073eb0 con 0x7fb9cc0713a0 2023-09-23T03:28:57.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.809+0000 7fb9ddffb640 1 -- 172.21.15.154:0/2899955977 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=131) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9d808c400 con 0x7fb9e8144130 2023-09-23T03:28:57.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.809+0000 7fb9ede24640 1 --2- 172.21.15.154:0/2899955977 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x7fb9cc0713a0 0x7fb9cc0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:57.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.809+0000 7fb9ed623640 1 --2- 172.21.15.154:0/2899955977 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb9cc06af60 0x7fb9cc06d420 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4036000 tx=0x7fb9e4008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:57.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.809+0000 7fb9ede24640 1 --2- 172.21.15.154:0/2899955977 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x7fb9cc0713a0 0x7fb9cc0737e0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:57.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.810+0000 7fb9ddffb640 1 -- 172.21.15.154:0/2899955977 <== osd.1 v2:172.21.15.154:6816/1267710005 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fb9cc073eb0 con 0x7fb9cc0713a0 2023-09-23T03:28:57.824 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.824+0000 7fb9ef887640 1 -- 172.21.15.154:0/2899955977 --> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb9e8070610 con 0x7fb9cc0713a0 2023-09-23T03:28:57.825 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.824+0000 7fb9ddffb640 1 -- 172.21.15.154:0/2899955977 <== osd.1 v2:172.21.15.154:6816/1267710005 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fb9e00070f0 con 0x7fb9cc0713a0 2023-09-23T03:28:57.825 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:28:57.825 INFO:teuthology.orchestra.run.smithi154.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-23T03:28:57.825 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:28:57.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.825+0000 7fb9ef887640 1 -- 172.21.15.154:0/2899955977 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x7fb9cc0713a0 msgr2=0x7fb9cc0737e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:57.826 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.825+0000 7fb9ef887640 1 --2- 172.21.15.154:0/2899955977 >> [v2:172.21.15.154:6816/1267710005,v1:172.21.15.154:6817/1267710005] conn(0x7fb9cc0713a0 0x7fb9cc0737e0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:57.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.825+0000 7fb9ef887640 1 -- 172.21.15.154:0/2899955977 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb9cc06af60 msgr2=0x7fb9cc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:57.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.825+0000 7fb9ef887640 1 --2- 172.21.15.154:0/2899955977 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fb9cc06af60 0x7fb9cc06d420 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4036000 tx=0x7fb9e4008030 comp rx=0 tx=0).stop 2023-09-23T03:28:57.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.825+0000 7fb9ef887640 1 -- 172.21.15.154:0/2899955977 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9e8144130 msgr2=0x7fb9e815a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:57.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.825+0000 7fb9ef887640 1 --2- 172.21.15.154:0/2899955977 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb9e8144130 0x7fb9e815a300 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7fb9d800d830 tx=0x7fb9d800dcf0 comp rx=0 tx=0).stop 2023-09-23T03:28:57.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.825+0000 7fb9ede24640 1 -- 172.21.15.154:0/2899955977 reap_dead start 2023-09-23T03:28:57.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.825+0000 7fb9ef887640 1 -- 172.21.15.154:0/2899955977 shutdown_connections 2023-09-23T03:28:57.827 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.825+0000 7fb9ef887640 1 -- 172.21.15.154:0/2899955977 >> 172.21.15.154:0/2899955977 conn(0x7fb9e806a440 msgr2=0x7fb9e8142ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:57.828 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.825+0000 7fb9ef887640 1 -- 172.21.15.154:0/2899955977 shutdown_connections 2023-09-23T03:28:57.828 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:57.825+0000 7fb9ef887640 1 -- 172.21.15.154:0/2899955977 wait complete. 2023-09-23T03:28:57.842 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-09-23T03:28:58.049 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.049+0000 7f4ff21e7640 1 Processor -- start 2023-09-23T03:28:58.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.049+0000 7f4ff21e7640 1 -- start start 2023-09-23T03:28:58.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.051+0000 7f4ff21e7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fec15aa20 0x7f4fec0cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:58.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.051+0000 7f4ff21e7640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fec0cd140 con 0x7f4fec15a640 2023-09-23T03:28:58.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.051+0000 7f4ff21e7640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fec0cc530 con 0x7f4fec0cb4f0 2023-09-23T03:28:58.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.051+0000 7f4ff21e7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec0cb9a0 con 0x7f4fec15aa20 2023-09-23T03:28:58.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.051+0000 7f4feb7fe640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f4fec15a640 0x7f4fec0b3e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:49512/0 (socket says 172.21.15.154:49512) 2023-09-23T03:28:58.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.051+0000 7f4feb7fe640 1 -- 172.21.15.154:0/4029587842 learned_addr learned my addr 172.21.15.154:0/4029587842 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:28:58.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.051+0000 7f4fe3fff640 1 --2- 172.21.15.154:0/4029587842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fec15aa20 0x7f4fec0cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:58.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.051+0000 7f4feaffd640 1 -- 172.21.15.154:0/4029587842 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3243035112 0 0) 0x7f4fec0cd140 con 0x7f4fec15a640 2023-09-23T03:28:58.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.052+0000 7f4feaffd640 1 -- 172.21.15.154:0/4029587842 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fd8003520 con 0x7f4fec15a640 2023-09-23T03:28:58.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.052+0000 7f4feaffd640 1 -- 172.21.15.154:0/4029587842 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3003583762 0 0) 0x7f4fd8003520 con 0x7f4fec15a640 2023-09-23T03:28:58.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.052+0000 7f4feaffd640 1 -- 172.21.15.154:0/4029587842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fec15aa20 msgr2=0x7f4fec0cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:58.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.052+0000 7f4feaffd640 1 --2- 172.21.15.154:0/4029587842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fec15aa20 0x7f4fec0cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:58.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.052+0000 7f4feaffd640 1 -- 172.21.15.154:0/4029587842 >> v1:172.21.15.190:6789/0 conn(0x7f4fec0cb4f0 legacy=0x7f4fec0b2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:58.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.052+0000 7f4feaffd640 1 -- 172.21.15.154:0/4029587842 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fec0cc810 con 0x7f4fec15a640 2023-09-23T03:28:58.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.052+0000 7f4feaffd640 1 -- 172.21.15.154:0/4029587842 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f4fdc001670 con 0x7f4fec15a640 2023-09-23T03:28:58.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.052+0000 7f4fe3fff640 1 --2- 172.21.15.154:0/4029587842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fec15aa20 0x7f4fec0cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-23T03:28:58.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.052+0000 7f4feaffd640 1 -- 172.21.15.154:0/4029587842 >> v1:172.21.15.154:6789/0 conn(0x7f4fec15a640 legacy=0x7f4fec0b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:58.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.052+0000 7f4feaffd640 1 --2- 172.21.15.154:0/4029587842 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fd8003a10 0x7f4fd8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:58.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.052+0000 7f4feaffd640 1 --2- 172.21.15.154:0/4029587842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fec15aa20 0x7f4fd80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:58.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.052+0000 7f4feaffd640 1 --2- 172.21.15.154:0/4029587842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fd8004d50 0x7f4fd80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:58.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.052+0000 7f4feaffd640 1 -- 172.21.15.154:0/4029587842 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec0cb9a0 con 0x7f4fd8003a10 2023-09-23T03:28:58.056 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.052+0000 7f4feaffd640 1 -- 172.21.15.154:0/4029587842 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd800bbe0 con 0x7f4fd8004d50 2023-09-23T03:28:58.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.052+0000 7f4feaffd640 1 -- 172.21.15.154:0/4029587842 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd800bda0 con 0x7f4fec15aa20 2023-09-23T03:28:58.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.052+0000 7f4feb7fe640 1 --2- 172.21.15.154:0/4029587842 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fd8003a10 0x7f4fd8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:58.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.053+0000 7f4feb7fe640 1 -- 172.21.15.154:0/4029587842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fec15aa20 msgr2=0x7f4fd80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:58.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.053+0000 7f4feb7fe640 1 --2- 172.21.15.154:0/4029587842 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fec15aa20 0x7f4fd80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:58.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.053+0000 7f4feb7fe640 1 -- 172.21.15.154:0/4029587842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fd8004d50 msgr2=0x7f4fd80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:58.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.053+0000 7f4febfff640 1 --2- 172.21.15.154:0/4029587842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fd8004d50 0x7f4fd80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:58.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.053+0000 7f4feb7fe640 1 --2- 172.21.15.154:0/4029587842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fd8004d50 0x7f4fd80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:58.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.053+0000 7f4feb7fe640 1 -- 172.21.15.154:0/4029587842 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd8003520 con 0x7f4fd8003a10 2023-09-23T03:28:58.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.053+0000 7f4febfff640 1 --2- 172.21.15.154:0/4029587842 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fd8004d50 0x7f4fd80051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:58.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.054+0000 7f4feb7fe640 1 --2- 172.21.15.154:0/4029587842 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fd8003a10 0x7f4fd8003e30 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc007fb0 tx=0x7f4fd80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac2e34576a26a599 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:58.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.054+0000 7f4feaffd640 1 -- 172.21.15.154:0/4029587842 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc005ad0 con 0x7f4fd8003a10 2023-09-23T03:28:58.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.054+0000 7f4feaffd640 1 -- 172.21.15.154:0/4029587842 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fdc005c30 con 0x7f4fd8003a10 2023-09-23T03:28:58.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.054+0000 7f4feaffd640 1 -- 172.21.15.154:0/4029587842 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc02c7c0 con 0x7f4fd8003a10 2023-09-23T03:28:58.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.055+0000 7f4ff21e7640 1 -- 172.21.15.154:0/4029587842 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fd8003a10 msgr2=0x7f4fd8003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:58.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.055+0000 7f4ff21e7640 1 --2- 172.21.15.154:0/4029587842 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fd8003a10 0x7f4fd8003e30 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc007fb0 tx=0x7f4fd80057f0 comp rx=0 tx=0).stop 2023-09-23T03:28:58.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.055+0000 7f4febfff640 1 -- 172.21.15.154:0/4029587842 reap_dead start 2023-09-23T03:28:58.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.055+0000 7f4ff21e7640 1 -- 172.21.15.154:0/4029587842 shutdown_connections 2023-09-23T03:28:58.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.055+0000 7f4ff21e7640 1 -- 172.21.15.154:0/4029587842 >> 172.21.15.154:0/4029587842 conn(0x7f4fec06c820 msgr2=0x7f4fec069cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:58.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.055+0000 7f4ff21e7640 1 -- 172.21.15.154:0/4029587842 shutdown_connections 2023-09-23T03:28:58.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.055+0000 7f4ff21e7640 1 -- 172.21.15.154:0/4029587842 wait complete. 2023-09-23T03:28:58.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.057+0000 7f4ff21e7640 1 Processor -- start 2023-09-23T03:28:58.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.057+0000 7f4ff21e7640 1 -- start start 2023-09-23T03:28:58.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.058+0000 7f4ff21e7640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fd8003a10 0x7f4fec14c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:58.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.058+0000 7f4ff21e7640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fec0c5670 0x7f4fec0afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:58.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.058+0000 7f4ff21e7640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fec0affe0 0x7f4fec157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:58.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.058+0000 7f4ff21e7640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec0c5af0 con 0x7f4fd8003a10 2023-09-23T03:28:58.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.058+0000 7f4ff21e7640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec0b0460 con 0x7f4fec0affe0 2023-09-23T03:28:58.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.058+0000 7f4ff21e7640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec0c7f40 con 0x7f4fec0c5670 2023-09-23T03:28:58.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.058+0000 7f4feb7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fd8003a10 0x7f4fec14c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:58.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.058+0000 7f4feb7fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fd8003a10 0x7f4fec14c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44864/0 (socket says 172.21.15.154:44864) 2023-09-23T03:28:58.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.059+0000 7f4feb7fe640 1 -- 172.21.15.154:0/2911859209 learned_addr learned my addr 172.21.15.154:0/2911859209 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:58.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.059+0000 7f4feb7fe640 1 -- 172.21.15.154:0/2911859209 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fec0c5670 msgr2=0x7f4fec0afaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:58.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.059+0000 7f4feb7fe640 1 --2- 172.21.15.154:0/2911859209 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fec0c5670 0x7f4fec0afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:58.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.059+0000 7f4febfff640 1 --2- 172.21.15.154:0/2911859209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fec0affe0 0x7f4fec157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:58.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.059+0000 7f4feb7fe640 1 -- 172.21.15.154:0/2911859209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fec0affe0 msgr2=0x7f4fec157e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:58.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.059+0000 7f4feb7fe640 1 --2- 172.21.15.154:0/2911859209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fec0affe0 0x7f4fec157e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:58.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.059+0000 7f4feb7fe640 1 -- 172.21.15.154:0/2911859209 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fec0c8220 con 0x7f4fd8003a10 2023-09-23T03:28:58.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.059+0000 7f4feb7fe640 1 --2- 172.21.15.154:0/2911859209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fd8003a10 0x7f4fec14c810 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc007fb0 tx=0x7f4fec0cd0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:58.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.060+0000 7f4fe8ff9640 1 -- 172.21.15.154:0/2911859209 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc02cbd0 con 0x7f4fd8003a10 2023-09-23T03:28:58.064 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.060+0000 7f4fe8ff9640 1 -- 172.21.15.154:0/2911859209 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fdc02cd30 con 0x7f4fd8003a10 2023-09-23T03:28:58.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.060+0000 7f4fe8ff9640 1 -- 172.21.15.154:0/2911859209 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc02cea0 con 0x7f4fd8003a10 2023-09-23T03:28:58.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.060+0000 7f4ff21e7640 1 -- 172.21.15.154:0/2911859209 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fec157360 con 0x7f4fd8003a10 2023-09-23T03:28:58.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.060+0000 7f4ff21e7640 1 -- 172.21.15.154:0/2911859209 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fec157920 con 0x7f4fd8003a10 2023-09-23T03:28:58.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.061+0000 7f4ff21e7640 1 -- 172.21.15.154:0/2911859209 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fec157ae0 con 0x7f4fd8003a10 2023-09-23T03:28:58.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.062+0000 7f4fe8ff9640 1 -- 172.21.15.154:0/2911859209 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f4fdc032070 con 0x7f4fd8003a10 2023-09-23T03:28:58.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.063+0000 7f4fe8ff9640 1 --2- 172.21.15.154:0/2911859209 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4fcc06af30 0x7f4fcc06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:58.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.063+0000 7f4fe8ff9640 1 -- 172.21.15.154:0/2911859209 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f4fdc0aa580 con 0x7f4fd8003a10 2023-09-23T03:28:58.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.063+0000 7f4fe3fff640 1 --2- 172.21.15.154:0/2911859209 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4fcc06af30 0x7f4fcc06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:58.066 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.064+0000 7f4fe3fff640 1 --2- 172.21.15.154:0/2911859209 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4fcc06af30 0x7f4fcc06d3f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4006ce0 tx=0x7f4fd4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:58.067 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.067+0000 7f4fe8ff9640 1 -- 172.21.15.154:0/2911859209 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f4fdc075770 con 0x7f4fd8003a10 2023-09-23T03:28:58.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.256+0000 7f4ff21e7640 1 -- 172.21.15.154:0/2911859209 --> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f4fec0ccdd0 con 0x7f4fcc06af30 2023-09-23T03:28:58.257 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.257+0000 7f4fe8ff9640 1 -- 172.21.15.154:0/2911859209 <== mgr.4110 v2:172.21.15.154:6832/2422652104 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4fd4015440 con 0x7f4fcc06af30 2023-09-23T03:28:58.258 INFO:teuthology.orchestra.run.smithi154.stderr:instructed osd(s) 1 to deep-scrub 2023-09-23T03:28:58.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.260+0000 7f4ff21e7640 1 -- 172.21.15.154:0/2911859209 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4fcc06af30 msgr2=0x7f4fcc06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:58.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.260+0000 7f4ff21e7640 1 --2- 172.21.15.154:0/2911859209 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4fcc06af30 0x7f4fcc06d3f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4006ce0 tx=0x7f4fd4008040 comp rx=0 tx=0).stop 2023-09-23T03:28:58.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.260+0000 7f4ff21e7640 1 -- 172.21.15.154:0/2911859209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fd8003a10 msgr2=0x7f4fec14c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:58.261 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.260+0000 7f4ff21e7640 1 --2- 172.21.15.154:0/2911859209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fd8003a10 0x7f4fec14c810 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc007fb0 tx=0x7f4fec0cd0b0 comp rx=0 tx=0).stop 2023-09-23T03:28:58.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.260+0000 7f4ff21e7640 1 -- 172.21.15.154:0/2911859209 shutdown_connections 2023-09-23T03:28:58.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.260+0000 7f4ff21e7640 1 --2- 172.21.15.154:0/2911859209 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f4fcc06af30 0x7f4fcc06d3f0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:58.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.260+0000 7f4ff21e7640 1 --2- 172.21.15.154:0/2911859209 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4fec0affe0 0x7f4fec157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:58.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.260+0000 7f4ff21e7640 1 --2- 172.21.15.154:0/2911859209 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4fec0c5670 0x7f4fec0afaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:58.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.260+0000 7f4ff21e7640 1 --2- 172.21.15.154:0/2911859209 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4fd8003a10 0x7f4fec14c810 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:58.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.260+0000 7f4ff21e7640 1 -- 172.21.15.154:0/2911859209 >> 172.21.15.154:0/2911859209 conn(0x7f4fec06c820 msgr2=0x7f4fec0c8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:58.262 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.260+0000 7f4ff21e7640 1 -- 172.21.15.154:0/2911859209 shutdown_connections 2023-09-23T03:28:58.263 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.260+0000 7f4ff21e7640 1 -- 172.21.15.154:0/2911859209 wait complete. 2023-09-23T03:28:58.278 INFO:tasks.ceph:Scrubbing osd.2 2023-09-23T03:28:58.278 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-09-23T03:28:58.502 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.502+0000 7fc13dcaf640 1 Processor -- start 2023-09-23T03:28:58.503 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.502+0000 7fc13dcaf640 1 -- start start 2023-09-23T03:28:58.504 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.504+0000 7fc13dcaf640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc138159580 0x7fc13806c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:58.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.504+0000 7fc13dcaf640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fc1380cafc0 0x7fc1380cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:58.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.504+0000 7fc13dcaf640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1380cb930 con 0x7fc138153520 2023-09-23T03:28:58.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.504+0000 7fc13dcaf640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1380aff60 con 0x7fc138159580 2023-09-23T03:28:58.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.504+0000 7fc13dcaf640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc138159b70 con 0x7fc1380cafc0 2023-09-23T03:28:58.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.504+0000 7fc12ffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc138159580 0x7fc13806c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:58.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.504+0000 7fc12ffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc138159580 0x7fc13806c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44874/0 (socket says 172.21.15.154:44874) 2023-09-23T03:28:58.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.504+0000 7fc12ffff640 1 -- 172.21.15.154:0/1330120781 learned_addr learned my addr 172.21.15.154:0/1330120781 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:58.505 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.504+0000 7fc137fff640 1 --2- 172.21.15.154:0/1330120781 >> v2:172.21.15.190:3300/0 conn(0x7fc1380cafc0 0x7fc1380cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:58.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.505+0000 7fc12ffff640 1 -- 172.21.15.154:0/1330120781 >> v2:172.21.15.190:3300/0 conn(0x7fc1380cafc0 msgr2=0x7fc1380cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:58.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.505+0000 7fc12ffff640 1 --2- 172.21.15.154:0/1330120781 >> v2:172.21.15.190:3300/0 conn(0x7fc1380cafc0 0x7fc1380cb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:58.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.505+0000 7fc12ffff640 1 -- 172.21.15.154:0/1330120781 >> v1:172.21.15.190:6789/0 conn(0x7fc138153520 legacy=0x7fc1380c8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:58.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.505+0000 7fc12ffff640 1 -- 172.21.15.154:0/1330120781 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc138159e50 con 0x7fc138159580 2023-09-23T03:28:58.507 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.506+0000 7fc12ffff640 1 --2- 172.21.15.154:0/1330120781 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc138159580 0x7fc13806c6d0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7fc138071310 tx=0x7fc12002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d36bd5b4cb8acead server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:58.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.506+0000 7fc136ffd640 1 -- 172.21.15.154:0/1330120781 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc120031870 con 0x7fc138159580 2023-09-23T03:28:58.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.506+0000 7fc136ffd640 1 -- 172.21.15.154:0/1330120781 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1200319d0 con 0x7fc138159580 2023-09-23T03:28:58.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.506+0000 7fc136ffd640 1 -- 172.21.15.154:0/1330120781 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc120031d60 con 0x7fc138159580 2023-09-23T03:28:58.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.507+0000 7fc13dcaf640 1 -- 172.21.15.154:0/1330120781 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc138159580 msgr2=0x7fc13806c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:58.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.507+0000 7fc13dcaf640 1 --2- 172.21.15.154:0/1330120781 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc138159580 0x7fc13806c6d0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7fc138071310 tx=0x7fc12002b910 comp rx=0 tx=0).stop 2023-09-23T03:28:58.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.507+0000 7fc13dcaf640 1 -- 172.21.15.154:0/1330120781 shutdown_connections 2023-09-23T03:28:58.508 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.507+0000 7fc13dcaf640 1 --2- 172.21.15.154:0/1330120781 >> v2:172.21.15.190:3300/0 conn(0x7fc1380cafc0 0x7fc1380cb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:58.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.507+0000 7fc13dcaf640 1 --2- 172.21.15.154:0/1330120781 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc138159580 0x7fc13806c6d0 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:58.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.507+0000 7fc13dcaf640 1 -- 172.21.15.154:0/1330120781 >> 172.21.15.154:0/1330120781 conn(0x7fc13806a440 msgr2=0x7fc1380b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:58.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.507+0000 7fc13dcaf640 1 -- 172.21.15.154:0/1330120781 shutdown_connections 2023-09-23T03:28:58.509 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.507+0000 7fc13dcaf640 1 -- 172.21.15.154:0/1330120781 wait complete. 2023-09-23T03:28:58.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.508+0000 7fc13dcaf640 1 Processor -- start 2023-09-23T03:28:58.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.509+0000 7fc13dcaf640 1 -- start start 2023-09-23T03:28:58.511 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.509+0000 7fc13dcaf640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1380cafc0 0x7fc1381529e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:58.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.509+0000 7fc13dcaf640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc138153520 0x7fc138150b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:58.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.509+0000 7fc13dcaf640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc138159580 0x7fc138148150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:58.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.510+0000 7fc13dcaf640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1380b33c0 con 0x7fc1380cafc0 2023-09-23T03:28:58.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.510+0000 7fc13dcaf640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13814b230 con 0x7fc138159580 2023-09-23T03:28:58.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.510+0000 7fc13dcaf640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc138151210 con 0x7fc138153520 2023-09-23T03:28:58.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.510+0000 7fc1377fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1380cafc0 0x7fc1381529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:58.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.510+0000 7fc1377fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1380cafc0 0x7fc1381529e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44886/0 (socket says 172.21.15.154:44886) 2023-09-23T03:28:58.512 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.510+0000 7fc1377fe640 1 -- 172.21.15.154:0/3666484142 learned_addr learned my addr 172.21.15.154:0/3666484142 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:58.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.510+0000 7fc1377fe640 1 -- 172.21.15.154:0/3666484142 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc138153520 msgr2=0x7fc138150b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:58.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.510+0000 7fc1377fe640 1 --2- 172.21.15.154:0/3666484142 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc138153520 0x7fc138150b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:58.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.510+0000 7fc1377fe640 1 -- 172.21.15.154:0/3666484142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc138159580 msgr2=0x7fc138148150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:58.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.510+0000 7fc1377fe640 1 --2- 172.21.15.154:0/3666484142 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc138159580 0x7fc138148150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:58.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.510+0000 7fc1377fe640 1 -- 172.21.15.154:0/3666484142 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1200094d0 con 0x7fc1380cafc0 2023-09-23T03:28:58.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.510+0000 7fc1377fe640 1 --2- 172.21.15.154:0/3666484142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1380cafc0 0x7fc1381529e0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fc12800c430 tx=0x7fc12800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:58.513 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.511+0000 7fc134ff9640 1 -- 172.21.15.154:0/3666484142 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc128011790 con 0x7fc1380cafc0 2023-09-23T03:28:58.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.511+0000 7fc134ff9640 1 -- 172.21.15.154:0/3666484142 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1280118f0 con 0x7fc1380cafc0 2023-09-23T03:28:58.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.511+0000 7fc13dcaf640 1 -- 172.21.15.154:0/3666484142 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1381514f0 con 0x7fc1380cafc0 2023-09-23T03:28:58.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.511+0000 7fc134ff9640 1 -- 172.21.15.154:0/3666484142 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc128011ce0 con 0x7fc1380cafc0 2023-09-23T03:28:58.514 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.511+0000 7fc13dcaf640 1 -- 172.21.15.154:0/3666484142 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1380b3610 con 0x7fc1380cafc0 2023-09-23T03:28:58.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.512+0000 7fc13dcaf640 1 -- 172.21.15.154:0/3666484142 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc13815a410 con 0x7fc1380cafc0 2023-09-23T03:28:58.515 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.513+0000 7fc134ff9640 1 -- 172.21.15.154:0/3666484142 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fc12801c460 con 0x7fc1380cafc0 2023-09-23T03:28:58.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.514+0000 7fc134ff9640 1 --2- 172.21.15.154:0/3666484142 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc11806af30 0x7fc11806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:58.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.514+0000 7fc134ff9640 1 -- 172.21.15.154:0/3666484142 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fc128089630 con 0x7fc1380cafc0 2023-09-23T03:28:58.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.514+0000 7fc12ffff640 1 --2- 172.21.15.154:0/3666484142 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc11806af30 0x7fc11806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:58.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.514+0000 7fc134ff9640 1 --2- 172.21.15.154:0/3666484142 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x7fc118071370 0x7fc1180737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:58.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.514+0000 7fc134ff9640 1 -- 172.21.15.154:0/3666484142 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc118073e80 con 0x7fc118071370 2023-09-23T03:28:58.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.514+0000 7fc134ff9640 1 -- 172.21.15.154:0/3666484142 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=131) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc12808bc10 con 0x7fc1380cafc0 2023-09-23T03:28:58.516 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.514+0000 7fc137fff640 1 --2- 172.21.15.154:0/3666484142 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x7fc118071370 0x7fc1180737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:58.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.515+0000 7fc12ffff640 1 --2- 172.21.15.154:0/3666484142 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc11806af30 0x7fc11806d3f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fc120036000 tx=0x7fc120030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:58.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.515+0000 7fc137fff640 1 --2- 172.21.15.154:0/3666484142 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x7fc118071370 0x7fc1180737b0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:58.517 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.516+0000 7fc134ff9640 1 -- 172.21.15.154:0/3666484142 <== osd.2 v2:172.21.15.154:6822/2884407243 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fc118073e80 con 0x7fc118071370 2023-09-23T03:28:58.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.530+0000 7fc13dcaf640 1 -- 172.21.15.154:0/3666484142 --> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc13815a130 con 0x7fc118071370 2023-09-23T03:28:58.530 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.530+0000 7fc134ff9640 1 -- 172.21.15.154:0/3666484142 <== osd.2 v2:172.21.15.154:6822/2884407243 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fc130009a30 con 0x7fc118071370 2023-09-23T03:28:58.531 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:28:58.531 INFO:teuthology.orchestra.run.smithi154.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-23T03:28:58.531 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:28:58.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.530+0000 7fc13dcaf640 1 -- 172.21.15.154:0/3666484142 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x7fc118071370 msgr2=0x7fc1180737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:58.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.530+0000 7fc13dcaf640 1 --2- 172.21.15.154:0/3666484142 >> [v2:172.21.15.154:6822/2884407243,v1:172.21.15.154:6825/2884407243] conn(0x7fc118071370 0x7fc1180737b0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:58.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.531+0000 7fc13dcaf640 1 -- 172.21.15.154:0/3666484142 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc11806af30 msgr2=0x7fc11806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:58.532 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.531+0000 7fc13dcaf640 1 --2- 172.21.15.154:0/3666484142 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc11806af30 0x7fc11806d3f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fc120036000 tx=0x7fc120030000 comp rx=0 tx=0).stop 2023-09-23T03:28:58.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.531+0000 7fc13dcaf640 1 -- 172.21.15.154:0/3666484142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1380cafc0 msgr2=0x7fc1381529e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:58.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.531+0000 7fc13dcaf640 1 --2- 172.21.15.154:0/3666484142 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1380cafc0 0x7fc1381529e0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fc12800c430 tx=0x7fc12800c8f0 comp rx=0 tx=0).stop 2023-09-23T03:28:58.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.531+0000 7fc137fff640 1 -- 172.21.15.154:0/3666484142 reap_dead start 2023-09-23T03:28:58.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.531+0000 7fc13dcaf640 1 -- 172.21.15.154:0/3666484142 shutdown_connections 2023-09-23T03:28:58.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.531+0000 7fc13dcaf640 1 -- 172.21.15.154:0/3666484142 >> 172.21.15.154:0/3666484142 conn(0x7fc13806a440 msgr2=0x7fc1381503b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:58.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.531+0000 7fc13dcaf640 1 -- 172.21.15.154:0/3666484142 shutdown_connections 2023-09-23T03:28:58.533 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.531+0000 7fc13dcaf640 1 -- 172.21.15.154:0/3666484142 wait complete. 2023-09-23T03:28:58.549 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-09-23T03:28:58.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.761+0000 7fcaf71fb640 1 Processor -- start 2023-09-23T03:28:58.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.762+0000 7fcaf71fb640 1 -- start start 2023-09-23T03:28:58.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.763+0000 7fcaf71fb640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fcaf015a8c0 0x7fcaf015e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:58.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.763+0000 7fcaf71fb640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcaf00cad90 0x7fcaf00b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:58.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.763+0000 7fcaf71fb640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf015ea60 con 0x7fcaf00b2700 2023-09-23T03:28:58.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.763+0000 7fcaf71fb640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaf00b2b90 con 0x7fcaf00cad90 2023-09-23T03:28:58.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.763+0000 7fcaf71fb640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaf015ed40 con 0x7fcaf015a8c0 2023-09-23T03:28:58.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.764+0000 7fcaeffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcaf00cad90 0x7fcaf00b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:58.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.764+0000 7fcaeffff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fcaf00cad90 0x7fcaf00b21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44896/0 (socket says 172.21.15.154:44896) 2023-09-23T03:28:58.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.764+0000 7fcaeffff640 1 -- 172.21.15.154:0/3512560153 learned_addr learned my addr 172.21.15.154:0/3512560153 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:58.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.764+0000 7fcaf5798640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fcaf00b2700 0x7fcaf006fcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:49514/0 (socket says 172.21.15.154:49514) 2023-09-23T03:28:58.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.764+0000 7fcaf4f97640 1 --2- 172.21.15.154:0/3512560153 >> v2:172.21.15.190:3300/0 conn(0x7fcaf015a8c0 0x7fcaf015e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:58.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.764+0000 7fcaeffff640 1 -- 172.21.15.154:0/3512560153 >> v2:172.21.15.190:3300/0 conn(0x7fcaf015a8c0 msgr2=0x7fcaf015e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:58.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.764+0000 7fcaeffff640 1 --2- 172.21.15.154:0/3512560153 >> v2:172.21.15.190:3300/0 conn(0x7fcaf015a8c0 0x7fcaf015e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:58.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.764+0000 7fcaeffff640 1 -- 172.21.15.154:0/3512560153 >> v1:172.21.15.154:6789/0 conn(0x7fcaf00b2700 legacy=0x7fcaf006fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:58.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.764+0000 7fcaeffff640 1 -- 172.21.15.154:0/3512560153 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf015f020 con 0x7fcaf00cad90 2023-09-23T03:28:58.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.765+0000 7fcaeffff640 1 --2- 172.21.15.154:0/3512560153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaf00cad90 0x7fcaf00b21c0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fcaf00737d0 tx=0x7fcad802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb01934b1d0dc35 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:58.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.766+0000 7fcaef7fe640 1 -- 172.21.15.154:0/3512560153 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad8030860 con 0x7fcaf00cad90 2023-09-23T03:28:58.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.766+0000 7fcaef7fe640 1 -- 172.21.15.154:0/3512560153 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad80309c0 con 0x7fcaf00cad90 2023-09-23T03:28:58.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.766+0000 7fcaef7fe640 1 -- 172.21.15.154:0/3512560153 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad8030d50 con 0x7fcaf00cad90 2023-09-23T03:28:58.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.766+0000 7fcaf71fb640 1 -- 172.21.15.154:0/3512560153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaf00cad90 msgr2=0x7fcaf00b21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:58.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.766+0000 7fcaf71fb640 1 --2- 172.21.15.154:0/3512560153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaf00cad90 0x7fcaf00b21c0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fcaf00737d0 tx=0x7fcad802b9a0 comp rx=0 tx=0).stop 2023-09-23T03:28:58.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.766+0000 7fcaf71fb640 1 -- 172.21.15.154:0/3512560153 shutdown_connections 2023-09-23T03:28:58.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.766+0000 7fcaf71fb640 1 --2- 172.21.15.154:0/3512560153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaf00cad90 0x7fcaf00b21c0 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:58.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.766+0000 7fcaf71fb640 1 --2- 172.21.15.154:0/3512560153 >> v2:172.21.15.190:3300/0 conn(0x7fcaf015a8c0 0x7fcaf015e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:58.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.766+0000 7fcaf71fb640 1 -- 172.21.15.154:0/3512560153 >> 172.21.15.154:0/3512560153 conn(0x7fcaf00cd1c0 msgr2=0x7fcaf00cd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:58.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.767+0000 7fcaf71fb640 1 -- 172.21.15.154:0/3512560153 shutdown_connections 2023-09-23T03:28:58.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.767+0000 7fcaf71fb640 1 -- 172.21.15.154:0/3512560153 wait complete. 2023-09-23T03:28:58.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.769+0000 7fcaf71fb640 1 Processor -- start 2023-09-23T03:28:58.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.769+0000 7fcaf71fb640 1 -- start start 2023-09-23T03:28:58.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.770+0000 7fcaf71fb640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcaf00b2700 0x7fcaf015cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:58.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.770+0000 7fcaf71fb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaf00cad90 0x7fcaf014e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:58.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.770+0000 7fcaf71fb640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaf015a8c0 0x7fcaf00c6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:58.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.770+0000 7fcaf71fb640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf014e8f0 con 0x7fcaf00cad90 2023-09-23T03:28:58.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.770+0000 7fcaf71fb640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf015ddd0 con 0x7fcaf00b2700 2023-09-23T03:28:58.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.770+0000 7fcaf71fb640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf015d250 con 0x7fcaf015a8c0 2023-09-23T03:28:58.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.770+0000 7fcaeffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaf00cad90 0x7fcaf014e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:58.771 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.770+0000 7fcaeffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaf00cad90 0x7fcaf014e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44902/0 (socket says 172.21.15.154:44902) 2023-09-23T03:28:58.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.770+0000 7fcaeffff640 1 -- 172.21.15.154:0/2044028062 learned_addr learned my addr 172.21.15.154:0/2044028062 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:58.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.771+0000 7fcaf4f97640 1 --2- 172.21.15.154:0/2044028062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcaf00b2700 0x7fcaf015cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:58.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.771+0000 7fcaeffff640 1 -- 172.21.15.154:0/2044028062 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaf015a8c0 msgr2=0x7fcaf00c6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:58.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.771+0000 7fcaf5798640 1 --2- 172.21.15.154:0/2044028062 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaf015a8c0 0x7fcaf00c6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:58.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.771+0000 7fcaeffff640 1 --2- 172.21.15.154:0/2044028062 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaf015a8c0 0x7fcaf00c6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:58.773 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.771+0000 7fcaeffff640 1 -- 172.21.15.154:0/2044028062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcaf00b2700 msgr2=0x7fcaf015cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:58.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.771+0000 7fcaeffff640 1 --2- 172.21.15.154:0/2044028062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcaf00b2700 0x7fcaf015cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:58.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.771+0000 7fcaeffff640 1 -- 172.21.15.154:0/2044028062 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad80094d0 con 0x7fcaf00cad90 2023-09-23T03:28:58.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.771+0000 7fcaf4f97640 1 --2- 172.21.15.154:0/2044028062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcaf00b2700 0x7fcaf015cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:58.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.771+0000 7fcaeffff640 1 --2- 172.21.15.154:0/2044028062 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaf00cad90 0x7fcaf014e3b0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7fcad802b9e0 tx=0x7fcad8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:58.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.772+0000 7fcaed7fa640 1 -- 172.21.15.154:0/2044028062 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad8039d60 con 0x7fcaf00cad90 2023-09-23T03:28:58.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.772+0000 7fcaed7fa640 1 -- 172.21.15.154:0/2044028062 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad8043400 con 0x7fcaf00cad90 2023-09-23T03:28:58.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.772+0000 7fcaed7fa640 1 -- 172.21.15.154:0/2044028062 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad80437b0 con 0x7fcaf00cad90 2023-09-23T03:28:58.774 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.772+0000 7fcaf71fb640 1 -- 172.21.15.154:0/2044028062 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf015d530 con 0x7fcaf00cad90 2023-09-23T03:28:58.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.772+0000 7fcaf71fb640 1 -- 172.21.15.154:0/2044028062 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf015daf0 con 0x7fcaf00cad90 2023-09-23T03:28:58.775 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.773+0000 7fcaf71fb640 1 -- 172.21.15.154:0/2044028062 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaf0073eb0 con 0x7fcaf00cad90 2023-09-23T03:28:58.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.774+0000 7fcaed7fa640 1 -- 172.21.15.154:0/2044028062 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fcad8043910 con 0x7fcaf00cad90 2023-09-23T03:28:58.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.775+0000 7fcaed7fa640 1 --2- 172.21.15.154:0/2044028062 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcad006af60 0x7fcad006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:58.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.775+0000 7fcaed7fa640 1 -- 172.21.15.154:0/2044028062 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fcad803e070 con 0x7fcaf00cad90 2023-09-23T03:28:58.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.775+0000 7fcaf4f97640 1 --2- 172.21.15.154:0/2044028062 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcad006af60 0x7fcad006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:58.776 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.776+0000 7fcaf4f97640 1 --2- 172.21.15.154:0/2044028062 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcad006af60 0x7fcad006d420 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fcae0009f50 tx=0x7fcae000b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:58.779 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.779+0000 7fcaed7fa640 1 -- 172.21.15.154:0/2044028062 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fcad8077910 con 0x7fcaf00cad90 2023-09-23T03:28:58.969 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.969+0000 7fcaf71fb640 1 -- 172.21.15.154:0/2044028062 --> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fcaf00ce3b0 con 0x7fcad006af60 2023-09-23T03:28:58.970 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.970+0000 7fcaed7fa640 1 -- 172.21.15.154:0/2044028062 <== mgr.4110 v2:172.21.15.154:6832/2422652104 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fcae0009b60 con 0x7fcad006af60 2023-09-23T03:28:58.970 INFO:teuthology.orchestra.run.smithi154.stderr:instructed osd(s) 2 to deep-scrub 2023-09-23T03:28:58.973 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.973+0000 7fcaf71fb640 1 -- 172.21.15.154:0/2044028062 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcad006af60 msgr2=0x7fcad006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:58.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.973+0000 7fcaf71fb640 1 --2- 172.21.15.154:0/2044028062 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcad006af60 0x7fcad006d420 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fcae0009f50 tx=0x7fcae000b040 comp rx=0 tx=0).stop 2023-09-23T03:28:58.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.973+0000 7fcaf71fb640 1 -- 172.21.15.154:0/2044028062 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaf00cad90 msgr2=0x7fcaf014e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:58.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.973+0000 7fcaf71fb640 1 --2- 172.21.15.154:0/2044028062 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaf00cad90 0x7fcaf014e3b0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7fcad802b9e0 tx=0x7fcad8004730 comp rx=0 tx=0).stop 2023-09-23T03:28:58.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.973+0000 7fcaf71fb640 1 -- 172.21.15.154:0/2044028062 shutdown_connections 2023-09-23T03:28:58.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.973+0000 7fcaf71fb640 1 --2- 172.21.15.154:0/2044028062 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fcad006af60 0x7fcad006d420 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:58.974 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.973+0000 7fcaf71fb640 1 --2- 172.21.15.154:0/2044028062 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcaf015a8c0 0x7fcaf00c6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:58.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.973+0000 7fcaf71fb640 1 --2- 172.21.15.154:0/2044028062 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcaf00cad90 0x7fcaf014e3b0 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:58.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.973+0000 7fcaf71fb640 1 --2- 172.21.15.154:0/2044028062 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcaf00b2700 0x7fcaf015cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:58.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.973+0000 7fcaf71fb640 1 -- 172.21.15.154:0/2044028062 >> 172.21.15.154:0/2044028062 conn(0x7fcaf00cd1c0 msgr2=0x7fcaf00b1450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:58.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.973+0000 7fcaf71fb640 1 -- 172.21.15.154:0/2044028062 shutdown_connections 2023-09-23T03:28:58.975 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:58.973+0000 7fcaf71fb640 1 -- 172.21.15.154:0/2044028062 wait complete. 2023-09-23T03:28:58.990 INFO:tasks.ceph:Scrubbing osd.3 2023-09-23T03:28:58.990 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-09-23T03:28:59.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.202+0000 7fa6d60c2640 1 Processor -- start 2023-09-23T03:28:59.203 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.203+0000 7fa6d60c2640 1 -- start start 2023-09-23T03:28:59.204 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.204+0000 7fa6d60c2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa6d0144130 0x7fa6d00b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:59.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.204+0000 7fa6d60c2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6d00caf80 0x7fa6d00cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:59.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.204+0000 7fa6d60c2640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6d00cb920 con 0x7fa6d015e520 2023-09-23T03:28:59.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.204+0000 7fa6d60c2640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6d0070fb0 con 0x7fa6d0144130 2023-09-23T03:28:59.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.204+0000 7fa6d60c2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d00700e0 con 0x7fa6d00caf80 2023-09-23T03:28:59.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.204+0000 7fa6cf7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa6d0144130 0x7fa6d00b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:59.205 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.204+0000 7fa6cf7fe640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fa6d0144130 0x7fa6d00b1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44912/0 (socket says 172.21.15.154:44912) 2023-09-23T03:28:59.206 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.204+0000 7fa6cf7fe640 1 -- 172.21.15.154:0/834841539 learned_addr learned my addr 172.21.15.154:0/834841539 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:59.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.205+0000 7fa6cf7fe640 1 -- 172.21.15.154:0/834841539 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6d00caf80 msgr2=0x7fa6d00cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:59.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.205+0000 7fa6cf7fe640 1 --2- 172.21.15.154:0/834841539 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6d00caf80 0x7fa6d00cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:59.207 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.205+0000 7fa6cf7fe640 1 -- 172.21.15.154:0/834841539 >> v1:172.21.15.190:6789/0 conn(0x7fa6d015e520 legacy=0x7fa6d015e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:59.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.205+0000 7fa6cf7fe640 1 -- 172.21.15.154:0/834841539 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d0070330 con 0x7fa6d0144130 2023-09-23T03:28:59.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.206+0000 7fa6cf7fe640 1 --2- 172.21.15.154:0/834841539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6d0144130 0x7fa6d00b1b70 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0009870 tx=0x7fa6c002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3f2ff533ce7fc5a server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:59.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.206+0000 7fa6ceffd640 1 -- 172.21.15.154:0/834841539 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0031640 con 0x7fa6d0144130 2023-09-23T03:28:59.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.206+0000 7fa6ceffd640 1 -- 172.21.15.154:0/834841539 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c00317a0 con 0x7fa6d0144130 2023-09-23T03:28:59.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.206+0000 7fa6ceffd640 1 -- 172.21.15.154:0/834841539 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c0031b30 con 0x7fa6d0144130 2023-09-23T03:28:59.208 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.207+0000 7fa6d60c2640 1 -- 172.21.15.154:0/834841539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6d0144130 msgr2=0x7fa6d00b1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:59.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.207+0000 7fa6d60c2640 1 --2- 172.21.15.154:0/834841539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6d0144130 0x7fa6d00b1b70 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0009870 tx=0x7fa6c002b8f0 comp rx=0 tx=0).stop 2023-09-23T03:28:59.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.207+0000 7fa6d60c2640 1 -- 172.21.15.154:0/834841539 shutdown_connections 2023-09-23T03:28:59.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.207+0000 7fa6d60c2640 1 --2- 172.21.15.154:0/834841539 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6d00caf80 0x7fa6d00cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:59.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.207+0000 7fa6d60c2640 1 --2- 172.21.15.154:0/834841539 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6d0144130 0x7fa6d00b1b70 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:59.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.207+0000 7fa6d60c2640 1 -- 172.21.15.154:0/834841539 >> 172.21.15.154:0/834841539 conn(0x7fa6d006a440 msgr2=0x7fa6d00b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:59.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.207+0000 7fa6d60c2640 1 -- 172.21.15.154:0/834841539 shutdown_connections 2023-09-23T03:28:59.209 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.207+0000 7fa6d60c2640 1 -- 172.21.15.154:0/834841539 wait complete. 2023-09-23T03:28:59.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.209+0000 7fa6d60c2640 1 Processor -- start 2023-09-23T03:28:59.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.209+0000 7fa6d60c2640 1 -- start start 2023-09-23T03:28:59.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.210+0000 7fa6d60c2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6d00caf80 0x7fa6d013e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:59.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.210+0000 7fa6d60c2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6d0144130 0x7fa6d015a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:59.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.210+0000 7fa6d60c2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6d015e520 0x7fa6d013c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:59.212 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.210+0000 7fa6d60c2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d0140310 con 0x7fa6d015e520 2023-09-23T03:28:59.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.210+0000 7fa6d60c2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d015af10 con 0x7fa6d0144130 2023-09-23T03:28:59.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.210+0000 7fa6d60c2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d013ce00 con 0x7fa6d00caf80 2023-09-23T03:28:59.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.210+0000 7fa6cffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6d015e520 0x7fa6d013c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:59.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.210+0000 7fa6cffff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6d015e520 0x7fa6d013c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44922/0 (socket says 172.21.15.154:44922) 2023-09-23T03:28:59.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.210+0000 7fa6cffff640 1 -- 172.21.15.154:0/1125840784 learned_addr learned my addr 172.21.15.154:0/1125840784 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:59.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.210+0000 7fa6c7fff640 1 --2- 172.21.15.154:0/1125840784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6d0144130 0x7fa6d015a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:59.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.210+0000 7fa6cffff640 1 -- 172.21.15.154:0/1125840784 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6d00caf80 msgr2=0x7fa6d013e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:59.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.210+0000 7fa6cffff640 1 --2- 172.21.15.154:0/1125840784 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fa6d00caf80 0x7fa6d013e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:59.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.210+0000 7fa6cffff640 1 -- 172.21.15.154:0/1125840784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6d0144130 msgr2=0x7fa6d015a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:59.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.210+0000 7fa6cffff640 1 --2- 172.21.15.154:0/1125840784 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fa6d0144130 0x7fa6d015a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:59.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.210+0000 7fa6cffff640 1 -- 172.21.15.154:0/1125840784 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c00094d0 con 0x7fa6d015e520 2023-09-23T03:28:59.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.211+0000 7fa6cffff640 1 --2- 172.21.15.154:0/1125840784 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6d015e520 0x7fa6d013c8c0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fa6c800b9c0 tx=0x7fa6c800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:59.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.211+0000 7fa6ccff9640 1 -- 172.21.15.154:0/1125840784 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c800cba0 con 0x7fa6d015e520 2023-09-23T03:28:59.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.211+0000 7fa6ccff9640 1 -- 172.21.15.154:0/1125840784 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c800cd00 con 0x7fa6d015e520 2023-09-23T03:28:59.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.211+0000 7fa6d60c2640 1 -- 172.21.15.154:0/1125840784 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6d013d0a0 con 0x7fa6d015e520 2023-09-23T03:28:59.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.211+0000 7fa6ccff9640 1 -- 172.21.15.154:0/1125840784 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c8014680 con 0x7fa6d015e520 2023-09-23T03:28:59.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.211+0000 7fa6d60c2640 1 -- 172.21.15.154:0/1125840784 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6d013d660 con 0x7fa6d015e520 2023-09-23T03:28:59.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.212+0000 7fa6d60c2640 1 -- 172.21.15.154:0/1125840784 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa6d00708f0 con 0x7fa6d015e520 2023-09-23T03:28:59.216 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.213+0000 7fa6ccff9640 1 -- 172.21.15.154:0/1125840784 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fa6c801d3b0 con 0x7fa6d015e520 2023-09-23T03:28:59.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.214+0000 7fa6ccff9640 1 --2- 172.21.15.154:0/1125840784 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa6b006af30 0x7fa6b006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:59.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.214+0000 7fa6ccff9640 1 -- 172.21.15.154:0/1125840784 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fa6c8089d20 con 0x7fa6d015e520 2023-09-23T03:28:59.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.214+0000 7fa6cf7fe640 1 --2- 172.21.15.154:0/1125840784 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa6b006af30 0x7fa6b006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:59.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.214+0000 7fa6ccff9640 1 --2- 172.21.15.154:0/1125840784 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x7fa6b0071370 0x7fa6b00737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:59.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.214+0000 7fa6ccff9640 1 -- 172.21.15.154:0/1125840784 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa6b0073e80 con 0x7fa6b0071370 2023-09-23T03:28:59.217 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.215+0000 7fa6ccff9640 1 -- 172.21.15.154:0/1125840784 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=131) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6c808a090 con 0x7fa6d015e520 2023-09-23T03:28:59.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.215+0000 7fa6c7fff640 1 --2- 172.21.15.154:0/1125840784 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x7fa6b0071370 0x7fa6b00737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:59.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.215+0000 7fa6cf7fe640 1 --2- 172.21.15.154:0/1125840784 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa6b006af30 0x7fa6b006d3f0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0036000 tx=0x7fa6c0030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:59.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.215+0000 7fa6c7fff640 1 --2- 172.21.15.154:0/1125840784 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x7fa6b0071370 0x7fa6b00737b0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:59.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.216+0000 7fa6ccff9640 1 -- 172.21.15.154:0/1125840784 <== osd.3 v2:172.21.15.154:6808/1133688468 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fa6b0073e80 con 0x7fa6b0071370 2023-09-23T03:28:59.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.230+0000 7fa6d60c2640 1 -- 172.21.15.154:0/1125840784 --> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa6d0070610 con 0x7fa6b0071370 2023-09-23T03:28:59.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.230+0000 7fa6ccff9640 1 -- 172.21.15.154:0/1125840784 <== osd.3 v2:172.21.15.154:6808/1133688468 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fa6b800a940 con 0x7fa6b0071370 2023-09-23T03:28:59.231 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:28:59.232 INFO:teuthology.orchestra.run.smithi154.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-23T03:28:59.232 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:28:59.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.231+0000 7fa6d60c2640 1 -- 172.21.15.154:0/1125840784 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x7fa6b0071370 msgr2=0x7fa6b00737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:59.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.231+0000 7fa6d60c2640 1 --2- 172.21.15.154:0/1125840784 >> [v2:172.21.15.154:6808/1133688468,v1:172.21.15.154:6809/1133688468] conn(0x7fa6b0071370 0x7fa6b00737b0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:59.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.231+0000 7fa6d60c2640 1 -- 172.21.15.154:0/1125840784 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa6b006af30 msgr2=0x7fa6b006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:59.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.231+0000 7fa6d60c2640 1 --2- 172.21.15.154:0/1125840784 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fa6b006af30 0x7fa6b006d3f0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0036000 tx=0x7fa6c0030000 comp rx=0 tx=0).stop 2023-09-23T03:28:59.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.231+0000 7fa6d60c2640 1 -- 172.21.15.154:0/1125840784 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6d015e520 msgr2=0x7fa6d013c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:59.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.231+0000 7fa6d60c2640 1 --2- 172.21.15.154:0/1125840784 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa6d015e520 0x7fa6d013c8c0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fa6c800b9c0 tx=0x7fa6c800be80 comp rx=0 tx=0).stop 2023-09-23T03:28:59.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.231+0000 7fa6cffff640 1 -- 172.21.15.154:0/1125840784 reap_dead start 2023-09-23T03:28:59.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.231+0000 7fa6d60c2640 1 -- 172.21.15.154:0/1125840784 shutdown_connections 2023-09-23T03:28:59.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.231+0000 7fa6d60c2640 1 -- 172.21.15.154:0/1125840784 >> 172.21.15.154:0/1125840784 conn(0x7fa6d006a440 msgr2=0x7fa6d0142370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:59.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.232+0000 7fa6d60c2640 1 -- 172.21.15.154:0/1125840784 shutdown_connections 2023-09-23T03:28:59.234 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.232+0000 7fa6d60c2640 1 -- 172.21.15.154:0/1125840784 wait complete. 2023-09-23T03:28:59.250 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-09-23T03:28:59.462 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.462+0000 7f2c98790640 1 Processor -- start 2023-09-23T03:28:59.463 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.462+0000 7f2c98790640 1 -- start start 2023-09-23T03:28:59.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.464+0000 7f2c98790640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c9015a640 0x7f2c900b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:59.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.464+0000 7f2c98790640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2c900b43a0 0x7f2c900cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:59.465 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.464+0000 7f2c98790640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c900cd140 con 0x7f2c9015aa20 2023-09-23T03:28:59.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.464+0000 7f2c98790640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c900cc530 con 0x7f2c900b43a0 2023-09-23T03:28:59.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.464+0000 7f2c98790640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c900cc810 con 0x7f2c9015a640 2023-09-23T03:28:59.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.464+0000 7f2c95d2b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2c900b43a0 0x7f2c900cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:59.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.464+0000 7f2c95d2b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2c900b43a0 0x7f2c900cafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44928/0 (socket says 172.21.15.154:44928) 2023-09-23T03:28:59.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.464+0000 7f2c95d2b640 1 -- 172.21.15.154:0/3640282186 learned_addr learned my addr 172.21.15.154:0/3640282186 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:59.466 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.465+0000 7f2c9652c640 1 --2- 172.21.15.154:0/3640282186 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c9015a640 0x7f2c900b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:59.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.465+0000 7f2c95d2b640 1 -- 172.21.15.154:0/3640282186 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c9015a640 msgr2=0x7f2c900b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:59.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.465+0000 7f2c95d2b640 1 --2- 172.21.15.154:0/3640282186 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c9015a640 0x7f2c900b3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:59.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.465+0000 7f2c95d2b640 1 -- 172.21.15.154:0/3640282186 >> v1:172.21.15.190:6789/0 conn(0x7f2c9015aa20 legacy=0x7f2c9015e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:59.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.465+0000 7f2c95d2b640 1 -- 172.21.15.154:0/3640282186 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c900ccaf0 con 0x7f2c900b43a0 2023-09-23T03:28:59.467 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.465+0000 7f2c9652c640 1 --2- 172.21.15.154:0/3640282186 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c9015a640 0x7f2c900b3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:28:59.468 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.466+0000 7f2c95d2b640 1 --2- 172.21.15.154:0/3640282186 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c900b43a0 0x7f2c900cafb0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f2c80009870 tx=0x7f2c8002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=578354c38057eb60 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:59.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.466+0000 7f2c9552a640 1 -- 172.21.15.154:0/3640282186 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c800317c0 con 0x7f2c900b43a0 2023-09-23T03:28:59.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.466+0000 7f2c9552a640 1 -- 172.21.15.154:0/3640282186 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c80031920 con 0x7f2c900b43a0 2023-09-23T03:28:59.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.466+0000 7f2c9552a640 1 -- 172.21.15.154:0/3640282186 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c80031cb0 con 0x7f2c900b43a0 2023-09-23T03:28:59.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.467+0000 7f2c98790640 1 -- 172.21.15.154:0/3640282186 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c900b43a0 msgr2=0x7f2c900cafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:59.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.467+0000 7f2c98790640 1 --2- 172.21.15.154:0/3640282186 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c900b43a0 0x7f2c900cafb0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f2c80009870 tx=0x7f2c8002b860 comp rx=0 tx=0).stop 2023-09-23T03:28:59.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.467+0000 7f2c98790640 1 -- 172.21.15.154:0/3640282186 shutdown_connections 2023-09-23T03:28:59.469 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.467+0000 7f2c98790640 1 --2- 172.21.15.154:0/3640282186 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c900b43a0 0x7f2c900cafb0 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:59.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.467+0000 7f2c98790640 1 --2- 172.21.15.154:0/3640282186 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c9015a640 0x7f2c900b3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:59.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.467+0000 7f2c98790640 1 -- 172.21.15.154:0/3640282186 >> 172.21.15.154:0/3640282186 conn(0x7f2c9006c820 msgr2=0x7f2c90069cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:59.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.467+0000 7f2c98790640 1 -- 172.21.15.154:0/3640282186 shutdown_connections 2023-09-23T03:28:59.470 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.467+0000 7f2c98790640 1 -- 172.21.15.154:0/3640282186 wait complete. 2023-09-23T03:28:59.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.469+0000 7f2c98790640 1 Processor -- start 2023-09-23T03:28:59.472 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.469+0000 7f2c98790640 1 -- start start 2023-09-23T03:28:59.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.470+0000 7f2c98790640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c900b43a0 0x7f2c900c6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:59.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.470+0000 7f2c98790640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c9015a640 0x7f2c9014df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:59.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.470+0000 7f2c98790640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c9015aa20 0x7f2c90071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:59.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.470+0000 7f2c98790640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c900c6560 con 0x7f2c9015aa20 2023-09-23T03:28:59.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.470+0000 7f2c98790640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9015dec0 con 0x7f2c900b43a0 2023-09-23T03:28:59.473 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.470+0000 7f2c98790640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9015dbe0 con 0x7f2c9015a640 2023-09-23T03:28:59.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.471+0000 7f2c96d2d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c9015aa20 0x7f2c90071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:59.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.471+0000 7f2c96d2d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c9015aa20 0x7f2c90071210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44940/0 (socket says 172.21.15.154:44940) 2023-09-23T03:28:59.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.471+0000 7f2c96d2d640 1 -- 172.21.15.154:0/3649197584 learned_addr learned my addr 172.21.15.154:0/3649197584 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:59.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.471+0000 7f2c9652c640 1 --2- 172.21.15.154:0/3649197584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c900b43a0 0x7f2c900c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:59.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.471+0000 7f2c95d2b640 1 --2- 172.21.15.154:0/3649197584 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c9015a640 0x7f2c9014df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:59.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.471+0000 7f2c96d2d640 1 -- 172.21.15.154:0/3649197584 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c9015a640 msgr2=0x7f2c9014df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:59.474 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.471+0000 7f2c96d2d640 1 --2- 172.21.15.154:0/3649197584 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c9015a640 0x7f2c9014df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:59.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.471+0000 7f2c96d2d640 1 -- 172.21.15.154:0/3649197584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c900b43a0 msgr2=0x7f2c900c6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:59.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.471+0000 7f2c96d2d640 1 --2- 172.21.15.154:0/3649197584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c900b43a0 0x7f2c900c6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:59.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.471+0000 7f2c96d2d640 1 -- 172.21.15.154:0/3649197584 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c800094d0 con 0x7f2c9015aa20 2023-09-23T03:28:59.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.471+0000 7f2c96d2d640 1 --2- 172.21.15.154:0/3649197584 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c9015aa20 0x7f2c90071210 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f2c8800b9e0 tx=0x7f2c8800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:59.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.472+0000 7f2c86ffd640 1 -- 172.21.15.154:0/3649197584 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8800cb20 con 0x7f2c9015aa20 2023-09-23T03:28:59.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.472+0000 7f2c86ffd640 1 -- 172.21.15.154:0/3649197584 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c8800cc80 con 0x7f2c9015aa20 2023-09-23T03:28:59.475 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.472+0000 7f2c86ffd640 1 -- 172.21.15.154:0/3649197584 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c88014680 con 0x7f2c9015aa20 2023-09-23T03:28:59.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.472+0000 7f2c98790640 1 -- 172.21.15.154:0/3649197584 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c901f5630 con 0x7f2c9015aa20 2023-09-23T03:28:59.476 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.472+0000 7f2c98790640 1 -- 172.21.15.154:0/3649197584 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c901f5bf0 con 0x7f2c9015aa20 2023-09-23T03:28:59.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.473+0000 7f2c98790640 1 -- 172.21.15.154:0/3649197584 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c90073eb0 con 0x7f2c9015aa20 2023-09-23T03:28:59.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.474+0000 7f2c86ffd640 1 -- 172.21.15.154:0/3649197584 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f2c8801d3b0 con 0x7f2c9015aa20 2023-09-23T03:28:59.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.475+0000 7f2c86ffd640 1 --2- 172.21.15.154:0/3649197584 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2c7406af30 0x7f2c7406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:59.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.475+0000 7f2c86ffd640 1 -- 172.21.15.154:0/3649197584 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f2c88089590 con 0x7f2c9015aa20 2023-09-23T03:28:59.477 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.476+0000 7f2c9652c640 1 --2- 172.21.15.154:0/3649197584 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2c7406af30 0x7f2c7406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:59.478 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.476+0000 7f2c9652c640 1 --2- 172.21.15.154:0/3649197584 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2c7406af30 0x7f2c7406d3f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00a990 tx=0x7f2c8c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:59.480 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.480+0000 7f2c86ffd640 1 -- 172.21.15.154:0/3649197584 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f2c880546d0 con 0x7f2c9015aa20 2023-09-23T03:28:59.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.669+0000 7f2c98790640 1 -- 172.21.15.154:0/3649197584 --> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c900ccdd0 con 0x7f2c7406af30 2023-09-23T03:28:59.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.669+0000 7f2c86ffd640 1 -- 172.21.15.154:0/3649197584 <== mgr.4110 v2:172.21.15.154:6832/2422652104 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2c8c00a710 con 0x7f2c7406af30 2023-09-23T03:28:59.670 INFO:teuthology.orchestra.run.smithi154.stderr:instructed osd(s) 3 to deep-scrub 2023-09-23T03:28:59.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.672+0000 7f2c98790640 1 -- 172.21.15.154:0/3649197584 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2c7406af30 msgr2=0x7f2c7406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:59.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.672+0000 7f2c98790640 1 --2- 172.21.15.154:0/3649197584 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2c7406af30 0x7f2c7406d3f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00a990 tx=0x7f2c8c008040 comp rx=0 tx=0).stop 2023-09-23T03:28:59.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.672+0000 7f2c98790640 1 -- 172.21.15.154:0/3649197584 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c9015aa20 msgr2=0x7f2c90071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:59.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.672+0000 7f2c98790640 1 --2- 172.21.15.154:0/3649197584 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c9015aa20 0x7f2c90071210 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f2c8800b9e0 tx=0x7f2c8800bea0 comp rx=0 tx=0).stop 2023-09-23T03:28:59.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.672+0000 7f2c98790640 1 -- 172.21.15.154:0/3649197584 shutdown_connections 2023-09-23T03:28:59.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.672+0000 7f2c98790640 1 --2- 172.21.15.154:0/3649197584 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2c7406af30 0x7f2c7406d3f0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:59.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.672+0000 7f2c98790640 1 --2- 172.21.15.154:0/3649197584 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c9015aa20 0x7f2c90071210 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:59.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.672+0000 7f2c98790640 1 --2- 172.21.15.154:0/3649197584 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2c9015a640 0x7f2c9014df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:59.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.672+0000 7f2c98790640 1 --2- 172.21.15.154:0/3649197584 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2c900b43a0 0x7f2c900c6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:59.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.672+0000 7f2c98790640 1 -- 172.21.15.154:0/3649197584 >> 172.21.15.154:0/3649197584 conn(0x7f2c9006c820 msgr2=0x7f2c900b2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:59.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.673+0000 7f2c98790640 1 -- 172.21.15.154:0/3649197584 shutdown_connections 2023-09-23T03:28:59.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.673+0000 7f2c98790640 1 -- 172.21.15.154:0/3649197584 wait complete. 2023-09-23T03:28:59.691 INFO:tasks.ceph:Scrubbing osd.4 2023-09-23T03:28:59.691 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-09-23T03:28:59.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.907+0000 7fd84981a640 1 Processor -- start 2023-09-23T03:28:59.907 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.907+0000 7fd84981a640 1 -- start start 2023-09-23T03:28:59.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.908+0000 7fd84981a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd844161100 0x7fd844164ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:59.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.908+0000 7fd84981a640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd8440c2280 0x7fd84406ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:59.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.909+0000 7fd84981a640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd84406d200 con 0x7fd844160d20 2023-09-23T03:28:59.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.909+0000 7fd84981a640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd844165a60 con 0x7fd8440c2280 2023-09-23T03:28:59.909 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.909+0000 7fd84981a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8440c2710 con 0x7fd844161100 2023-09-23T03:28:59.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.909+0000 7fd8437fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd8440c2280 0x7fd84406ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:59.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.909+0000 7fd8427fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd844161100 0x7fd844164ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:59.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.909+0000 7fd8427fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd844161100 0x7fd844164ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3301/0 says I am v2:172.21.15.154:51024/0 (socket says 172.21.15.154:51024) 2023-09-23T03:28:59.910 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.909+0000 7fd8437fe640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7fd8440c2280 0x7fd84406ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.190:3300/0 says I am v2:172.21.15.154:58998/0 (socket says 172.21.15.154:58998) 2023-09-23T03:28:59.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.909+0000 7fd8427fc640 1 -- 172.21.15.154:0/1160066499 learned_addr learned my addr 172.21.15.154:0/1160066499 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:59.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.909+0000 7fd842ffd640 1 --1- 172.21.15.154:0/1160066499 >> v1:172.21.15.190:6789/0 conn(0x7fd844160d20 0x7fd8440c9430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.190:6789/0 says I am v1:172.21.15.154:46978/0 (socket says 172.21.15.154:46978) 2023-09-23T03:28:59.911 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.910+0000 7fd8437fe640 1 -- 172.21.15.154:0/1160066499 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd844161100 msgr2=0x7fd844164ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:28:59.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.910+0000 7fd8437fe640 1 --2- 172.21.15.154:0/1160066499 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd844161100 0x7fd844164ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:59.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.910+0000 7fd8437fe640 1 -- 172.21.15.154:0/1160066499 >> v1:172.21.15.190:6789/0 conn(0x7fd844160d20 legacy=0x7fd8440c9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:59.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.910+0000 7fd8437fe640 1 -- 172.21.15.154:0/1160066499 --> v2:172.21.15.190:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8441651c0 con 0x7fd8440c2280 2023-09-23T03:28:59.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.910+0000 7fd8427fc640 1 --2- 172.21.15.154:0/1160066499 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd844161100 0x7fd844164ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-23T03:28:59.912 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.911+0000 7fd8437fe640 1 --2- 172.21.15.154:0/1160066499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8440c2280 0x7fd84406ace0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd83c0097a0 tx=0x7fd83c02b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1779b5a191207eea server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:59.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.912+0000 7fd841ffb640 1 -- 172.21.15.154:0/1160066499 <== mon.1 v2:172.21.15.190:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c030800 con 0x7fd8440c2280 2023-09-23T03:28:59.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.912+0000 7fd841ffb640 1 -- 172.21.15.154:0/1160066499 <== mon.1 v2:172.21.15.190:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd83c030960 con 0x7fd8440c2280 2023-09-23T03:28:59.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.912+0000 7fd841ffb640 1 -- 172.21.15.154:0/1160066499 <== mon.1 v2:172.21.15.190:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c030cf0 con 0x7fd8440c2280 2023-09-23T03:28:59.913 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.912+0000 7fd84981a640 1 -- 172.21.15.154:0/1160066499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8440c2280 msgr2=0x7fd84406ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:59.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.912+0000 7fd84981a640 1 --2- 172.21.15.154:0/1160066499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8440c2280 0x7fd84406ace0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd83c0097a0 tx=0x7fd83c02b940 comp rx=0 tx=0).stop 2023-09-23T03:28:59.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.913+0000 7fd84981a640 1 -- 172.21.15.154:0/1160066499 shutdown_connections 2023-09-23T03:28:59.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.913+0000 7fd84981a640 1 --2- 172.21.15.154:0/1160066499 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd8440c2280 0x7fd84406ace0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:59.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.913+0000 7fd84981a640 1 --2- 172.21.15.154:0/1160066499 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd844161100 0x7fd844164ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:59.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.913+0000 7fd84981a640 1 -- 172.21.15.154:0/1160066499 >> 172.21.15.154:0/1160066499 conn(0x7fd8440708f0 msgr2=0x7fd844070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:59.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.913+0000 7fd84981a640 1 -- 172.21.15.154:0/1160066499 shutdown_connections 2023-09-23T03:28:59.914 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.913+0000 7fd84981a640 1 -- 172.21.15.154:0/1160066499 wait complete. 2023-09-23T03:28:59.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.914+0000 7fd84981a640 1 Processor -- start 2023-09-23T03:28:59.915 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.914+0000 7fd84981a640 1 -- start start 2023-09-23T03:28:59.916 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.915+0000 7fd84981a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8440c2280 0x7fd84415a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:59.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.915+0000 7fd84981a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd844160d20 0x7fd84414d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:59.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.915+0000 7fd84981a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd844161100 0x7fd8440c4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:59.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.915+0000 7fd84981a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8441ec230 con 0x7fd844161100 2023-09-23T03:28:59.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.915+0000 7fd84981a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8441eb470 con 0x7fd844160d20 2023-09-23T03:28:59.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.915+0000 7fd84981a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8441eb750 con 0x7fd8440c2280 2023-09-23T03:28:59.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.916+0000 7fd8437fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd844161100 0x7fd8440c4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:59.917 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.916+0000 7fd8437fe640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd844161100 0x7fd8440c4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44948/0 (socket says 172.21.15.154:44948) 2023-09-23T03:28:59.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.916+0000 7fd8437fe640 1 -- 172.21.15.154:0/4174947221 learned_addr learned my addr 172.21.15.154:0/4174947221 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:28:59.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.916+0000 7fd842ffd640 1 --2- 172.21.15.154:0/4174947221 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8440c2280 0x7fd84415a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:59.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.916+0000 7fd8427fc640 1 --2- 172.21.15.154:0/4174947221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd844160d20 0x7fd84414d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:59.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.916+0000 7fd8437fe640 1 -- 172.21.15.154:0/4174947221 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8440c2280 msgr2=0x7fd84415a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:59.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.916+0000 7fd8437fe640 1 --2- 172.21.15.154:0/4174947221 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fd8440c2280 0x7fd84415a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:59.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.916+0000 7fd8437fe640 1 -- 172.21.15.154:0/4174947221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd844160d20 msgr2=0x7fd84414d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:59.918 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.916+0000 7fd8437fe640 1 --2- 172.21.15.154:0/4174947221 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fd844160d20 0x7fd84414d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:59.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.916+0000 7fd8437fe640 1 -- 172.21.15.154:0/4174947221 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd82c0094d0 con 0x7fd844161100 2023-09-23T03:28:59.919 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.917+0000 7fd8437fe640 1 --2- 172.21.15.154:0/4174947221 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd844161100 0x7fd8440c4560 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fd83c0098c0 tx=0x7fd83c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:59.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.917+0000 7fd83bfff640 1 -- 172.21.15.154:0/4174947221 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c03e070 con 0x7fd844161100 2023-09-23T03:28:59.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.917+0000 7fd83bfff640 1 -- 172.21.15.154:0/4174947221 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd83c039990 con 0x7fd844161100 2023-09-23T03:28:59.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.917+0000 7fd84981a640 1 -- 172.21.15.154:0/4174947221 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd83c009400 con 0x7fd844161100 2023-09-23T03:28:59.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.917+0000 7fd83bfff640 1 -- 172.21.15.154:0/4174947221 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c039d40 con 0x7fd844161100 2023-09-23T03:28:59.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.917+0000 7fd84981a640 1 -- 172.21.15.154:0/4174947221 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8441ebd10 con 0x7fd844161100 2023-09-23T03:28:59.920 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.918+0000 7fd84981a640 1 -- 172.21.15.154:0/4174947221 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd844165780 con 0x7fd844161100 2023-09-23T03:28:59.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.919+0000 7fd83bfff640 1 -- 172.21.15.154:0/4174947221 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fd83c0423f0 con 0x7fd844161100 2023-09-23T03:28:59.921 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.920+0000 7fd83bfff640 1 --2- 172.21.15.154:0/4174947221 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd82406af30 0x7fd82406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:59.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.920+0000 7fd83bfff640 1 -- 172.21.15.154:0/4174947221 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fd83c0abb10 con 0x7fd844161100 2023-09-23T03:28:59.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.921+0000 7fd842ffd640 1 --2- 172.21.15.154:0/4174947221 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd82406af30 0x7fd82406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:59.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.921+0000 7fd83bfff640 1 --2- 172.21.15.154:0/4174947221 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x7fd824071370 0x7fd8240737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:28:59.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.921+0000 7fd83bfff640 1 -- 172.21.15.154:0/4174947221 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd824073e80 con 0x7fd824071370 2023-09-23T03:28:59.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.921+0000 7fd83bfff640 1 -- 172.21.15.154:0/4174947221 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=131) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd83c0abe80 con 0x7fd844161100 2023-09-23T03:28:59.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.921+0000 7fd8427fc640 1 --2- 172.21.15.154:0/4174947221 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x7fd824071370 0x7fd8240737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:28:59.922 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.921+0000 7fd842ffd640 1 --2- 172.21.15.154:0/4174947221 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd82406af30 0x7fd82406d3f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fd8340012e0 tx=0x7fd834005b90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:59.923 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.922+0000 7fd8427fc640 1 --2- 172.21.15.154:0/4174947221 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x7fd824071370 0x7fd8240737b0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:28:59.924 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.924+0000 7fd83bfff640 1 -- 172.21.15.154:0/4174947221 <== osd.4 v2:172.21.15.190:6808/717302641 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fd824073e80 con 0x7fd824071370 2023-09-23T03:28:59.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.940+0000 7fd84981a640 1 -- 172.21.15.154:0/4174947221 --> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd8441654a0 con 0x7fd824071370 2023-09-23T03:28:59.941 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.941+0000 7fd83bfff640 1 -- 172.21.15.154:0/4174947221 <== osd.4 v2:172.21.15.190:6808/717302641 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fd82c009d40 con 0x7fd824071370 2023-09-23T03:28:59.941 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:28:59.941 INFO:teuthology.orchestra.run.smithi154.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-23T03:28:59.942 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:28:59.942 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.941+0000 7fd84981a640 1 -- 172.21.15.154:0/4174947221 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x7fd824071370 msgr2=0x7fd8240737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:59.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.941+0000 7fd84981a640 1 --2- 172.21.15.154:0/4174947221 >> [v2:172.21.15.190:6808/717302641,v1:172.21.15.190:6809/717302641] conn(0x7fd824071370 0x7fd8240737b0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:28:59.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.941+0000 7fd84981a640 1 -- 172.21.15.154:0/4174947221 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd82406af30 msgr2=0x7fd82406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:59.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.941+0000 7fd84981a640 1 --2- 172.21.15.154:0/4174947221 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fd82406af30 0x7fd82406d3f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fd8340012e0 tx=0x7fd834005b90 comp rx=0 tx=0).stop 2023-09-23T03:28:59.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.941+0000 7fd84981a640 1 -- 172.21.15.154:0/4174947221 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd844161100 msgr2=0x7fd8440c4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:28:59.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.941+0000 7fd84981a640 1 --2- 172.21.15.154:0/4174947221 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd844161100 0x7fd8440c4560 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fd83c0098c0 tx=0x7fd83c0046c0 comp rx=0 tx=0).stop 2023-09-23T03:28:59.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.941+0000 7fd8437fe640 1 -- 172.21.15.154:0/4174947221 reap_dead start 2023-09-23T03:28:59.943 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.942+0000 7fd84981a640 1 -- 172.21.15.154:0/4174947221 shutdown_connections 2023-09-23T03:28:59.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.942+0000 7fd84981a640 1 -- 172.21.15.154:0/4174947221 >> 172.21.15.154:0/4174947221 conn(0x7fd8440708f0 msgr2=0x7fd84406a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:28:59.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.942+0000 7fd84981a640 1 -- 172.21.15.154:0/4174947221 shutdown_connections 2023-09-23T03:28:59.944 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:28:59.942+0000 7fd84981a640 1 -- 172.21.15.154:0/4174947221 wait complete. 2023-09-23T03:28:59.959 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-09-23T03:29:00.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.171+0000 7f85b40d4640 1 Processor -- start 2023-09-23T03:29:00.172 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.172+0000 7f85b40d4640 1 -- start start 2023-09-23T03:29:00.173 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.173+0000 7f85b40d4640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f85ac0c9300 0x7f85ac0cc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:00.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.173+0000 7f85b40d4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85ac070090 0x7f85ac0cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:00.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.173+0000 7f85b40d4640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85ac070600 con 0x7f85ac143210 2023-09-23T03:29:00.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.173+0000 7f85b40d4640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85ac1435f0 con 0x7f85ac0c9300 2023-09-23T03:29:00.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.173+0000 7f85b40d4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac0708e0 con 0x7f85ac070090 2023-09-23T03:29:00.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.173+0000 7f85b166f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f85ac0c9300 0x7f85ac0cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:00.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.173+0000 7f85b166f640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f85ac0c9300 0x7f85ac0cc170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44960/0 (socket says 172.21.15.154:44960) 2023-09-23T03:29:00.174 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.173+0000 7f85b166f640 1 -- 172.21.15.154:0/617305437 learned_addr learned my addr 172.21.15.154:0/617305437 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:29:00.175 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.174+0000 7f85b2671640 1 --2- 172.21.15.154:0/617305437 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85ac070090 0x7f85ac0cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:00.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.174+0000 7f85b166f640 1 -- 172.21.15.154:0/617305437 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85ac070090 msgr2=0x7f85ac0cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:29:00.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.174+0000 7f85b166f640 1 --2- 172.21.15.154:0/617305437 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85ac070090 0x7f85ac0cc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:00.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.174+0000 7f85b166f640 1 -- 172.21.15.154:0/617305437 >> v1:172.21.15.190:6789/0 conn(0x7f85ac143210 legacy=0x7f85ac0c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:29:00.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.174+0000 7f85b166f640 1 -- 172.21.15.154:0/617305437 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85ac070bc0 con 0x7f85ac0c9300 2023-09-23T03:29:00.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.175+0000 7f85b166f640 1 --2- 172.21.15.154:0/617305437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85ac0c9300 0x7f85ac0cc170 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009870 tx=0x7f85a002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c7fdc82cace3cc2 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:29:00.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.175+0000 7f85b0e6e640 1 -- 172.21.15.154:0/617305437 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a0031640 con 0x7f85ac0c9300 2023-09-23T03:29:00.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.175+0000 7f85b0e6e640 1 -- 172.21.15.154:0/617305437 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a00317a0 con 0x7f85ac0c9300 2023-09-23T03:29:00.176 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.175+0000 7f85b0e6e640 1 -- 172.21.15.154:0/617305437 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a0031b30 con 0x7f85ac0c9300 2023-09-23T03:29:00.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.176+0000 7f85b40d4640 1 -- 172.21.15.154:0/617305437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85ac0c9300 msgr2=0x7f85ac0cc170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:00.177 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.176+0000 7f85b40d4640 1 --2- 172.21.15.154:0/617305437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85ac0c9300 0x7f85ac0cc170 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009870 tx=0x7f85a002b8f0 comp rx=0 tx=0).stop 2023-09-23T03:29:00.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.176+0000 7f85b40d4640 1 -- 172.21.15.154:0/617305437 shutdown_connections 2023-09-23T03:29:00.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.176+0000 7f85b40d4640 1 --2- 172.21.15.154:0/617305437 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85ac070090 0x7f85ac0cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:00.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.176+0000 7f85b40d4640 1 --2- 172.21.15.154:0/617305437 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85ac0c9300 0x7f85ac0cc170 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:00.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.176+0000 7f85b40d4640 1 -- 172.21.15.154:0/617305437 >> 172.21.15.154:0/617305437 conn(0x7f85ac06c820 msgr2=0x7f85ac069cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:29:00.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.176+0000 7f85b40d4640 1 -- 172.21.15.154:0/617305437 shutdown_connections 2023-09-23T03:29:00.178 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.176+0000 7f85b40d4640 1 -- 172.21.15.154:0/617305437 wait complete. 2023-09-23T03:29:00.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.178+0000 7f85b40d4640 1 Processor -- start 2023-09-23T03:29:00.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.178+0000 7f85b40d4640 1 -- start start 2023-09-23T03:29:00.180 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.179+0000 7f85b40d4640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85ac070090 0x7f85ac13a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:00.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.179+0000 7f85b40d4640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85ac0c9300 0x7f85ac13b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:00.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.179+0000 7f85b40d4640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85ac143210 0x7f85ac15ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:00.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.179+0000 7f85b40d4640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac13df00 con 0x7f85ac070090 2023-09-23T03:29:00.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.179+0000 7f85b40d4640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac13bf20 con 0x7f85ac0c9300 2023-09-23T03:29:00.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.179+0000 7f85b40d4640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac15d360 con 0x7f85ac143210 2023-09-23T03:29:00.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.179+0000 7f85b1e70640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85ac070090 0x7f85ac13a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:00.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.179+0000 7f85b1e70640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85ac070090 0x7f85ac13a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44968/0 (socket says 172.21.15.154:44968) 2023-09-23T03:29:00.181 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.179+0000 7f85b1e70640 1 -- 172.21.15.154:0/3853147446 learned_addr learned my addr 172.21.15.154:0/3853147446 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:29:00.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.180+0000 7f85b1e70640 1 -- 172.21.15.154:0/3853147446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85ac143210 msgr2=0x7f85ac15ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:00.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.180+0000 7f85b2671640 1 --2- 172.21.15.154:0/3853147446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85ac143210 0x7f85ac15ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:00.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.180+0000 7f85b1e70640 1 --2- 172.21.15.154:0/3853147446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85ac143210 0x7f85ac15ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:00.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.180+0000 7f85b1e70640 1 -- 172.21.15.154:0/3853147446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85ac0c9300 msgr2=0x7f85ac13b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:00.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.180+0000 7f85b166f640 1 --2- 172.21.15.154:0/3853147446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85ac0c9300 0x7f85ac13b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:00.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.180+0000 7f85b1e70640 1 --2- 172.21.15.154:0/3853147446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85ac0c9300 0x7f85ac13b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:00.182 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.180+0000 7f85b1e70640 1 -- 172.21.15.154:0/3853147446 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a00094d0 con 0x7f85ac070090 2023-09-23T03:29:00.183 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.180+0000 7f85b1e70640 1 --2- 172.21.15.154:0/3853147446 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85ac070090 0x7f85ac13a9f0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f859c00b3f0 tx=0x7f859c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:29:00.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.180+0000 7f85aa7fc640 1 -- 172.21.15.154:0/3853147446 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859c011830 con 0x7f85ac070090 2023-09-23T03:29:00.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.180+0000 7f85b40d4640 1 -- 172.21.15.154:0/3853147446 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85ac13c180 con 0x7f85ac070090 2023-09-23T03:29:00.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.180+0000 7f85aa7fc640 1 -- 172.21.15.154:0/3853147446 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f859c011990 con 0x7f85ac070090 2023-09-23T03:29:00.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.180+0000 7f85aa7fc640 1 -- 172.21.15.154:0/3853147446 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859c011d40 con 0x7f85ac070090 2023-09-23T03:29:00.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.181+0000 7f85b40d4640 1 -- 172.21.15.154:0/3853147446 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85ac13c740 con 0x7f85ac070090 2023-09-23T03:29:00.184 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.182+0000 7f85b40d4640 1 -- 172.21.15.154:0/3853147446 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85ac13e040 con 0x7f85ac070090 2023-09-23T03:29:00.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.183+0000 7f85aa7fc640 1 -- 172.21.15.154:0/3853147446 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f859c01a400 con 0x7f85ac070090 2023-09-23T03:29:00.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.183+0000 7f85aa7fc640 1 --2- 172.21.15.154:0/3853147446 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f859806af30 0x7f859806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:00.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.184+0000 7f85aa7fc640 1 -- 172.21.15.154:0/3853147446 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f859c0894f0 con 0x7f85ac070090 2023-09-23T03:29:00.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.184+0000 7f85b166f640 1 --2- 172.21.15.154:0/3853147446 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f859806af30 0x7f859806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:00.185 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.184+0000 7f85b166f640 1 --2- 172.21.15.154:0/3853147446 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f859806af30 0x7f859806d3f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f85a0036000 tx=0x7f85a003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:29:00.188 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.188+0000 7f85aa7fc640 1 -- 172.21.15.154:0/3853147446 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f85ac13e040 con 0x7f85ac070090 2023-09-23T03:29:00.378 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.377+0000 7f85b40d4640 1 -- 172.21.15.154:0/3853147446 --> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f85ac071180 con 0x7f859806af30 2023-09-23T03:29:00.379 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.379+0000 7f85aa7fc640 1 -- 172.21.15.154:0/3853147446 <== mgr.4110 v2:172.21.15.154:6832/2422652104 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f85a0031a90 con 0x7f859806af30 2023-09-23T03:29:00.379 INFO:teuthology.orchestra.run.smithi154.stderr:instructed osd(s) 4 to deep-scrub 2023-09-23T03:29:00.381 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.381+0000 7f85b40d4640 1 -- 172.21.15.154:0/3853147446 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f859806af30 msgr2=0x7f859806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:00.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.381+0000 7f85b40d4640 1 --2- 172.21.15.154:0/3853147446 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f859806af30 0x7f859806d3f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f85a0036000 tx=0x7f85a003c040 comp rx=0 tx=0).stop 2023-09-23T03:29:00.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.381+0000 7f85b40d4640 1 -- 172.21.15.154:0/3853147446 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85ac070090 msgr2=0x7f85ac13a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:00.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.381+0000 7f85b40d4640 1 --2- 172.21.15.154:0/3853147446 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85ac070090 0x7f85ac13a9f0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f859c00b3f0 tx=0x7f859c00b8b0 comp rx=0 tx=0).stop 2023-09-23T03:29:00.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.381+0000 7f85b40d4640 1 -- 172.21.15.154:0/3853147446 shutdown_connections 2023-09-23T03:29:00.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.381+0000 7f85b40d4640 1 --2- 172.21.15.154:0/3853147446 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f859806af30 0x7f859806d3f0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:00.382 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.381+0000 7f85b40d4640 1 --2- 172.21.15.154:0/3853147446 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f85ac143210 0x7f85ac15ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:00.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.381+0000 7f85b40d4640 1 --2- 172.21.15.154:0/3853147446 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f85ac0c9300 0x7f85ac13b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:00.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.381+0000 7f85b40d4640 1 --2- 172.21.15.154:0/3853147446 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f85ac070090 0x7f85ac13a9f0 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:00.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.381+0000 7f85b40d4640 1 -- 172.21.15.154:0/3853147446 >> 172.21.15.154:0/3853147446 conn(0x7f85ac06c820 msgr2=0x7f85ac1506b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:29:00.383 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.382+0000 7f85b40d4640 1 -- 172.21.15.154:0/3853147446 shutdown_connections 2023-09-23T03:29:00.384 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.382+0000 7f85b40d4640 1 -- 172.21.15.154:0/3853147446 wait complete. 2023-09-23T03:29:00.400 INFO:tasks.ceph:Scrubbing osd.5 2023-09-23T03:29:00.400 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-09-23T03:29:00.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.616+0000 7f27bb05d640 1 Processor -- start 2023-09-23T03:29:00.617 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.617+0000 7f27bb05d640 1 -- start start 2023-09-23T03:29:00.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.618+0000 7f27bb05d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f27b40cd030 0x7f27b406aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:00.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.618+0000 7f27bb05d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27b40b08e0 0x7f27b40b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:00.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.618+0000 7f27bb05d640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27b40af9f0 con 0x7f27b406b7f0 2023-09-23T03:29:00.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.618+0000 7f27bb05d640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27b40b0490 con 0x7f27b40cd030 2023-09-23T03:29:00.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.618+0000 7f27bb05d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b40cd5f0 con 0x7f27b40b08e0 2023-09-23T03:29:00.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.618+0000 7f27b3fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f27b40cd030 0x7f27b406aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:00.619 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.619+0000 7f27b3fff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f27b40cd030 0x7f27b406aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44970/0 (socket says 172.21.15.154:44970) 2023-09-23T03:29:00.620 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.619+0000 7f27b3fff640 1 -- 172.21.15.154:0/478470084 learned_addr learned my addr 172.21.15.154:0/478470084 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:29:00.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.619+0000 7f27b95fa640 1 --2- 172.21.15.154:0/478470084 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27b40b08e0 0x7f27b40b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:00.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.619+0000 7f27b3fff640 1 -- 172.21.15.154:0/478470084 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27b40b08e0 msgr2=0x7f27b40b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:29:00.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.619+0000 7f27b3fff640 1 --2- 172.21.15.154:0/478470084 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27b40b08e0 0x7f27b40b0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:00.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.619+0000 7f27b3fff640 1 -- 172.21.15.154:0/478470084 >> v1:172.21.15.154:6789/0 conn(0x7f27b406b7f0 legacy=0x7f27b406bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:00.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.619+0000 7f27b3fff640 1 -- 172.21.15.154:0/478470084 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b40cd8d0 con 0x7f27b40cd030 2023-09-23T03:29:00.621 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.619+0000 7f27b37fe640 1 -- 172.21.15.154:0/478470084 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4265655608 0 0) 0x7f27b40af9f0 con 0x7f27b406b7f0 2023-09-23T03:29:00.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.620+0000 7f27b3fff640 1 --2- 172.21.15.154:0/478470084 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27b40cd030 0x7f27b406aff0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f27a8009870 tx=0x7f27a802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263ecbd8b077609c server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:29:00.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.620+0000 7f27b37fe640 1 -- 172.21.15.154:0/478470084 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a8030840 con 0x7f27b40cd030 2023-09-23T03:29:00.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.620+0000 7f27b37fe640 1 -- 172.21.15.154:0/478470084 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a80309a0 con 0x7f27b40cd030 2023-09-23T03:29:00.622 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.620+0000 7f27b37fe640 1 -- 172.21.15.154:0/478470084 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a8030d30 con 0x7f27b40cd030 2023-09-23T03:29:00.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.621+0000 7f27bb05d640 1 -- 172.21.15.154:0/478470084 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27b40cd030 msgr2=0x7f27b406aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:00.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.621+0000 7f27bb05d640 1 --2- 172.21.15.154:0/478470084 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27b40cd030 0x7f27b406aff0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f27a8009870 tx=0x7f27a802b980 comp rx=0 tx=0).stop 2023-09-23T03:29:00.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.621+0000 7f27bb05d640 1 -- 172.21.15.154:0/478470084 shutdown_connections 2023-09-23T03:29:00.623 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.621+0000 7f27bb05d640 1 --2- 172.21.15.154:0/478470084 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27b40b08e0 0x7f27b40b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:00.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.621+0000 7f27bb05d640 1 --2- 172.21.15.154:0/478470084 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27b40cd030 0x7f27b406aff0 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:00.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.621+0000 7f27bb05d640 1 -- 172.21.15.154:0/478470084 >> 172.21.15.154:0/478470084 conn(0x7f27b4099b20 msgr2=0x7f27b4089f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:29:00.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.621+0000 7f27bb05d640 1 -- 172.21.15.154:0/478470084 shutdown_connections 2023-09-23T03:29:00.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.621+0000 7f27bb05d640 1 -- 172.21.15.154:0/478470084 wait complete. 2023-09-23T03:29:00.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.623+0000 7f27bb05d640 1 Processor -- start 2023-09-23T03:29:00.624 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.623+0000 7f27bb05d640 1 -- start start 2023-09-23T03:29:00.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.624+0000 7f27bb05d640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27b406b7f0 0x7f27b415d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:00.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.624+0000 7f27bb05d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27b40b08e0 0x7f27b415f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:00.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.624+0000 7f27bb05d640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27b40cd030 0x7f27b40c8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:00.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.624+0000 7f27bb05d640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b4158240 con 0x7f27b40b08e0 2023-09-23T03:29:00.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.624+0000 7f27bb05d640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b4158aa0 con 0x7f27b40cd030 2023-09-23T03:29:00.626 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.624+0000 7f27bb05d640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b415ffe0 con 0x7f27b406b7f0 2023-09-23T03:29:00.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.624+0000 7f27b3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27b40b08e0 0x7f27b415f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:00.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.625+0000 7f27b3fff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27b40b08e0 0x7f27b415f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44978/0 (socket says 172.21.15.154:44978) 2023-09-23T03:29:00.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.625+0000 7f27b3fff640 1 -- 172.21.15.154:0/1009948608 learned_addr learned my addr 172.21.15.154:0/1009948608 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:29:00.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.625+0000 7f27b95fa640 1 --2- 172.21.15.154:0/1009948608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27b40cd030 0x7f27b40c8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:00.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.625+0000 7f27b8df9640 1 --2- 172.21.15.154:0/1009948608 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27b406b7f0 0x7f27b415d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:00.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.625+0000 7f27b3fff640 1 -- 172.21.15.154:0/1009948608 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27b406b7f0 msgr2=0x7f27b415d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:00.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.625+0000 7f27b3fff640 1 --2- 172.21.15.154:0/1009948608 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27b406b7f0 0x7f27b415d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:00.627 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.625+0000 7f27b3fff640 1 -- 172.21.15.154:0/1009948608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27b40cd030 msgr2=0x7f27b40c8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:00.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.625+0000 7f27b3fff640 1 --2- 172.21.15.154:0/1009948608 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f27b40cd030 0x7f27b40c8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:00.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.625+0000 7f27b3fff640 1 -- 172.21.15.154:0/1009948608 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a80094d0 con 0x7f27b40b08e0 2023-09-23T03:29:00.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.625+0000 7f27b8df9640 1 --2- 172.21.15.154:0/1009948608 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f27b406b7f0 0x7f27b415d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:29:00.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.625+0000 7f27b3fff640 1 --2- 172.21.15.154:0/1009948608 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27b40b08e0 0x7f27b415f990 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f27a8000c00 tx=0x7f27a8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:29:00.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.626+0000 7f27b17fa640 1 -- 172.21.15.154:0/1009948608 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a8047070 con 0x7f27b40b08e0 2023-09-23T03:29:00.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.626+0000 7f27b17fa640 1 -- 172.21.15.154:0/1009948608 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a8039a90 con 0x7f27b40b08e0 2023-09-23T03:29:00.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.626+0000 7f27b17fa640 1 -- 172.21.15.154:0/1009948608 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a8039e20 con 0x7f27b40b08e0 2023-09-23T03:29:00.628 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.626+0000 7f27bb05d640 1 -- 172.21.15.154:0/1009948608 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27b4158450 con 0x7f27b40b08e0 2023-09-23T03:29:00.629 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.626+0000 7f27bb05d640 1 -- 172.21.15.154:0/1009948608 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27b40c90d0 con 0x7f27b40b08e0 2023-09-23T03:29:00.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.627+0000 7f27bb05d640 1 -- 172.21.15.154:0/1009948608 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f27b40cde90 con 0x7f27b40b08e0 2023-09-23T03:29:00.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.628+0000 7f27b17fa640 1 -- 172.21.15.154:0/1009948608 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f27a804c410 con 0x7f27b40b08e0 2023-09-23T03:29:00.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.629+0000 7f27b17fa640 1 --2- 172.21.15.154:0/1009948608 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f27a006af30 0x7f27a006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:00.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.629+0000 7f27b17fa640 1 -- 172.21.15.154:0/1009948608 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f27a803e070 con 0x7f27b40b08e0 2023-09-23T03:29:00.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.629+0000 7f27b8df9640 1 --2- 172.21.15.154:0/1009948608 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f27a006af30 0x7f27a006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:00.630 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.629+0000 7f27b17fa640 1 --2- 172.21.15.154:0/1009948608 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x7f27a0071370 0x7f27a00737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:00.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.629+0000 7f27b17fa640 1 -- 172.21.15.154:0/1009948608 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f27a0073e80 con 0x7f27a0071370 2023-09-23T03:29:00.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.629+0000 7f27b17fa640 1 -- 172.21.15.154:0/1009948608 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=131) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27a80b02a0 con 0x7f27b40b08e0 2023-09-23T03:29:00.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.630+0000 7f27b8df9640 1 --2- 172.21.15.154:0/1009948608 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f27a006af30 0x7f27a006d3f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f27a4001db0 tx=0x7f27a4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:29:00.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.630+0000 7f27b95fa640 1 --2- 172.21.15.154:0/1009948608 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x7f27a0071370 0x7f27a00737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:00.631 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.631+0000 7f27b95fa640 1 --2- 172.21.15.154:0/1009948608 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x7f27a0071370 0x7f27a00737b0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:29:00.633 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.633+0000 7f27b17fa640 1 -- 172.21.15.154:0/1009948608 <== osd.5 v2:172.21.15.190:6800/882536885 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f27a0073e80 con 0x7f27a0071370 2023-09-23T03:29:00.647 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.647+0000 7f27bb05d640 1 -- 172.21.15.154:0/1009948608 --> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f27b40cdbb0 con 0x7f27a0071370 2023-09-23T03:29:00.648 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.648+0000 7f27b17fa640 1 -- 172.21.15.154:0/1009948608 <== osd.5 v2:172.21.15.190:6800/882536885 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f27ac00d580 con 0x7f27a0071370 2023-09-23T03:29:00.648 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:29:00.648 INFO:teuthology.orchestra.run.smithi154.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-23T03:29:00.648 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:29:00.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.648+0000 7f27bb05d640 1 -- 172.21.15.154:0/1009948608 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x7f27a0071370 msgr2=0x7f27a00737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:00.649 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.648+0000 7f27bb05d640 1 --2- 172.21.15.154:0/1009948608 >> [v2:172.21.15.190:6800/882536885,v1:172.21.15.190:6801/882536885] conn(0x7f27a0071370 0x7f27a00737b0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:00.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.648+0000 7f27bb05d640 1 -- 172.21.15.154:0/1009948608 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f27a006af30 msgr2=0x7f27a006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:00.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.648+0000 7f27bb05d640 1 --2- 172.21.15.154:0/1009948608 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f27a006af30 0x7f27a006d3f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f27a4001db0 tx=0x7f27a4009040 comp rx=0 tx=0).stop 2023-09-23T03:29:00.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.648+0000 7f27bb05d640 1 -- 172.21.15.154:0/1009948608 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27b40b08e0 msgr2=0x7f27b415f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:00.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.648+0000 7f27bb05d640 1 --2- 172.21.15.154:0/1009948608 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27b40b08e0 0x7f27b415f990 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f27a8000c00 tx=0x7f27a8004730 comp rx=0 tx=0).stop 2023-09-23T03:29:00.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.648+0000 7f27b95fa640 1 -- 172.21.15.154:0/1009948608 reap_dead start 2023-09-23T03:29:00.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.648+0000 7f27bb05d640 1 -- 172.21.15.154:0/1009948608 shutdown_connections 2023-09-23T03:29:00.650 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.648+0000 7f27bb05d640 1 -- 172.21.15.154:0/1009948608 >> 172.21.15.154:0/1009948608 conn(0x7f27b4099b20 msgr2=0x7f27b406bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:29:00.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.648+0000 7f27bb05d640 1 -- 172.21.15.154:0/1009948608 shutdown_connections 2023-09-23T03:29:00.651 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.648+0000 7f27bb05d640 1 -- 172.21.15.154:0/1009948608 wait complete. 2023-09-23T03:29:00.666 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-09-23T03:29:00.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.881+0000 7faa7a6fc640 1 Processor -- start 2023-09-23T03:29:00.882 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.881+0000 7faa7a6fc640 1 -- start start 2023-09-23T03:29:00.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.883+0000 7faa7a6fc640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7faa740c9300 0x7faa740cc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:00.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.883+0000 7faa7a6fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa74070090 0x7faa740cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:00.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.883+0000 7faa7a6fc640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa74070600 con 0x7faa74143210 2023-09-23T03:29:00.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.883+0000 7faa7a6fc640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa741435f0 con 0x7faa740c9300 2023-09-23T03:29:00.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.883+0000 7faa7a6fc640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa740708e0 con 0x7faa74070090 2023-09-23T03:29:00.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.883+0000 7faa6bfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7faa740c9300 0x7faa740cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:00.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.883+0000 7faa6bfff640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7faa740c9300 0x7faa740cc170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44982/0 (socket says 172.21.15.154:44982) 2023-09-23T03:29:00.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.883+0000 7faa6bfff640 1 -- 172.21.15.154:0/3581999669 learned_addr learned my addr 172.21.15.154:0/3581999669 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:29:00.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.884+0000 7faa6bfff640 1 -- 172.21.15.154:0/3581999669 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa74070090 msgr2=0x7faa740cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:29:00.885 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.884+0000 7faa6bfff640 1 --2- 172.21.15.154:0/3581999669 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa74070090 0x7faa740cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:00.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.884+0000 7faa6bfff640 1 -- 172.21.15.154:0/3581999669 >> v1:172.21.15.190:6789/0 conn(0x7faa74143210 legacy=0x7faa740c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:29:00.886 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.884+0000 7faa6bfff640 1 -- 172.21.15.154:0/3581999669 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa74070bc0 con 0x7faa740c9300 2023-09-23T03:29:00.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.885+0000 7faa6bfff640 1 --2- 172.21.15.154:0/3581999669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa740c9300 0x7faa740cc170 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7faa5c009870 tx=0x7faa5c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15ab354adffdda1 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:29:00.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.885+0000 7faa737fe640 1 -- 172.21.15.154:0/3581999669 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5c031640 con 0x7faa740c9300 2023-09-23T03:29:00.887 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.885+0000 7faa737fe640 1 -- 172.21.15.154:0/3581999669 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa5c0317a0 con 0x7faa740c9300 2023-09-23T03:29:00.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.885+0000 7faa737fe640 1 -- 172.21.15.154:0/3581999669 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5c031b30 con 0x7faa740c9300 2023-09-23T03:29:00.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.886+0000 7faa7a6fc640 1 -- 172.21.15.154:0/3581999669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa740c9300 msgr2=0x7faa740cc170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:00.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.886+0000 7faa7a6fc640 1 --2- 172.21.15.154:0/3581999669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa740c9300 0x7faa740cc170 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7faa5c009870 tx=0x7faa5c02b8f0 comp rx=0 tx=0).stop 2023-09-23T03:29:00.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.886+0000 7faa7a6fc640 1 -- 172.21.15.154:0/3581999669 shutdown_connections 2023-09-23T03:29:00.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.886+0000 7faa7a6fc640 1 --2- 172.21.15.154:0/3581999669 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa74070090 0x7faa740cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:00.888 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.886+0000 7faa7a6fc640 1 --2- 172.21.15.154:0/3581999669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa740c9300 0x7faa740cc170 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:00.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.886+0000 7faa7a6fc640 1 -- 172.21.15.154:0/3581999669 >> 172.21.15.154:0/3581999669 conn(0x7faa7406c820 msgr2=0x7faa74069cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:29:00.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.886+0000 7faa7a6fc640 1 -- 172.21.15.154:0/3581999669 shutdown_connections 2023-09-23T03:29:00.889 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.886+0000 7faa7a6fc640 1 -- 172.21.15.154:0/3581999669 wait complete. 2023-09-23T03:29:00.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.888+0000 7faa7a6fc640 1 Processor -- start 2023-09-23T03:29:00.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.888+0000 7faa7a6fc640 1 -- start start 2023-09-23T03:29:00.891 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.889+0000 7faa7a6fc640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa74070090 0x7faa740ce960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:00.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.889+0000 7faa7a6fc640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa740c9300 0x7faa740c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:00.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.889+0000 7faa7a6fc640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa74143210 0x7faa74159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:00.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.889+0000 7faa7a6fc640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa740c7a70 con 0x7faa740c9300 2023-09-23T03:29:00.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.889+0000 7faa7a6fc640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa740c74b0 con 0x7faa74070090 2023-09-23T03:29:00.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.889+0000 7faa7a6fc640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa740c7790 con 0x7faa74143210 2023-09-23T03:29:00.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.889+0000 7faa6bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa740c9300 0x7faa740c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:00.892 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.889+0000 7faa6bfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa740c9300 0x7faa740c6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44988/0 (socket says 172.21.15.154:44988) 2023-09-23T03:29:00.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.889+0000 7faa6bfff640 1 -- 172.21.15.154:0/156607413 learned_addr learned my addr 172.21.15.154:0/156607413 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:29:00.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.889+0000 7faa73fff640 1 --2- 172.21.15.154:0/156607413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa74070090 0x7faa740ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:00.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.890+0000 7faa6bfff640 1 -- 172.21.15.154:0/156607413 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa74143210 msgr2=0x7faa74159fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:00.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.890+0000 7faa78c99640 1 --2- 172.21.15.154:0/156607413 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa74143210 0x7faa74159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:00.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.890+0000 7faa6bfff640 1 --2- 172.21.15.154:0/156607413 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa74143210 0x7faa74159fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:00.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.890+0000 7faa6bfff640 1 -- 172.21.15.154:0/156607413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa74070090 msgr2=0x7faa740ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:00.893 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.890+0000 7faa6bfff640 1 --2- 172.21.15.154:0/156607413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa74070090 0x7faa740ce960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:00.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.890+0000 7faa6bfff640 1 -- 172.21.15.154:0/156607413 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa5c0094d0 con 0x7faa740c9300 2023-09-23T03:29:00.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.890+0000 7faa73fff640 1 --2- 172.21.15.154:0/156607413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa74070090 0x7faa740ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:29:00.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.890+0000 7faa6bfff640 1 --2- 172.21.15.154:0/156607413 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa740c9300 0x7faa740c6b40 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7faa5c009870 tx=0x7faa5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:29:00.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.890+0000 7faa717fa640 1 -- 172.21.15.154:0/156607413 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5c03ac60 con 0x7faa740c9300 2023-09-23T03:29:00.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.890+0000 7faa717fa640 1 -- 172.21.15.154:0/156607413 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa5c03adc0 con 0x7faa740c9300 2023-09-23T03:29:00.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.890+0000 7faa7a6fc640 1 -- 172.21.15.154:0/156607413 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa741597c0 con 0x7faa740c9300 2023-09-23T03:29:00.894 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.890+0000 7faa717fa640 1 -- 172.21.15.154:0/156607413 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5c044630 con 0x7faa740c9300 2023-09-23T03:29:00.895 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.891+0000 7faa7a6fc640 1 -- 172.21.15.154:0/156607413 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa740ceea0 con 0x7faa740c9300 2023-09-23T03:29:00.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.892+0000 7faa7a6fc640 1 -- 172.21.15.154:0/156607413 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa74073eb0 con 0x7faa740c9300 2023-09-23T03:29:00.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.893+0000 7faa717fa640 1 -- 172.21.15.154:0/156607413 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7faa5c02f050 con 0x7faa740c9300 2023-09-23T03:29:00.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.893+0000 7faa717fa640 1 --2- 172.21.15.154:0/156607413 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7faa5406af30 0x7faa5406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:00.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.894+0000 7faa717fa640 1 -- 172.21.15.154:0/156607413 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5435+0+0 (secure 0 0 0) 0x7faa5c0ab6d0 con 0x7faa740c9300 2023-09-23T03:29:00.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.894+0000 7faa73fff640 1 --2- 172.21.15.154:0/156607413 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7faa5406af30 0x7faa5406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:00.897 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.894+0000 7faa73fff640 1 --2- 172.21.15.154:0/156607413 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7faa5406af30 0x7faa5406d3f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7faa6400a830 tx=0x7faa64008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:29:00.898 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:00.898+0000 7faa717fa640 1 -- 172.21.15.154:0/156607413 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7faa5c076820 con 0x7faa740c9300 2023-09-23T03:29:01.086 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.086+0000 7faa7a6fc640 1 -- 172.21.15.154:0/156607413 --> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7faa74071180 con 0x7faa5406af30 2023-09-23T03:29:01.087 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.087+0000 7faa717fa640 1 -- 172.21.15.154:0/156607413 <== mgr.4110 v2:172.21.15.154:6832/2422652104 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7faa6400adb0 con 0x7faa5406af30 2023-09-23T03:29:01.087 INFO:teuthology.orchestra.run.smithi154.stderr:instructed osd(s) 5 to deep-scrub 2023-09-23T03:29:01.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.089+0000 7faa7a6fc640 1 -- 172.21.15.154:0/156607413 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7faa5406af30 msgr2=0x7faa5406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:01.090 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.089+0000 7faa7a6fc640 1 --2- 172.21.15.154:0/156607413 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7faa5406af30 0x7faa5406d3f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7faa6400a830 tx=0x7faa64008040 comp rx=0 tx=0).stop 2023-09-23T03:29:01.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.089+0000 7faa7a6fc640 1 -- 172.21.15.154:0/156607413 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa740c9300 msgr2=0x7faa740c6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:01.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.089+0000 7faa7a6fc640 1 --2- 172.21.15.154:0/156607413 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa740c9300 0x7faa740c6b40 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7faa5c009870 tx=0x7faa5c0046c0 comp rx=0 tx=0).stop 2023-09-23T03:29:01.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.090+0000 7faa7a6fc640 1 -- 172.21.15.154:0/156607413 shutdown_connections 2023-09-23T03:29:01.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.090+0000 7faa7a6fc640 1 --2- 172.21.15.154:0/156607413 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7faa5406af30 0x7faa5406d3f0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:01.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.090+0000 7faa7a6fc640 1 --2- 172.21.15.154:0/156607413 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7faa74143210 0x7faa74159fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:01.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.090+0000 7faa7a6fc640 1 --2- 172.21.15.154:0/156607413 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7faa740c9300 0x7faa740c6b40 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:01.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.090+0000 7faa7a6fc640 1 --2- 172.21.15.154:0/156607413 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7faa74070090 0x7faa740ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:01.091 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.090+0000 7faa7a6fc640 1 -- 172.21.15.154:0/156607413 >> 172.21.15.154:0/156607413 conn(0x7faa7406c820 msgr2=0x7faa74157500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:29:01.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.090+0000 7faa7a6fc640 1 -- 172.21.15.154:0/156607413 shutdown_connections 2023-09-23T03:29:01.092 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.090+0000 7faa7a6fc640 1 -- 172.21.15.154:0/156607413 wait complete. 2023-09-23T03:29:01.107 INFO:tasks.ceph:Scrubbing osd.6 2023-09-23T03:29:01.108 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-09-23T03:29:01.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.321+0000 7fccd2739640 1 Processor -- start 2023-09-23T03:29:01.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.322+0000 7fccd2739640 1 -- start start 2023-09-23T03:29:01.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.323+0000 7fccd2739640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcccc0caf80 0x7fcccc0cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:01.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.323+0000 7fccd2739640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcccc0cb920 con 0x7fcccc06c320 2023-09-23T03:29:01.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.323+0000 7fccd2739640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcccc0700e0 con 0x7fcccc144130 2023-09-23T03:29:01.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.323+0000 7fccd2739640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc070fb0 con 0x7fcccc0caf80 2023-09-23T03:29:01.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.324+0000 7fccd0cd6640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7fcccc06c320 0x7fcccc06c7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:49534/0 (socket says 172.21.15.154:49534) 2023-09-23T03:29:01.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.324+0000 7fccd0cd6640 1 -- 172.21.15.154:0/53344051 learned_addr learned my addr 172.21.15.154:0/53344051 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:29:01.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.324+0000 7fcccb7fe640 1 --2- 172.21.15.154:0/53344051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcccc0caf80 0x7fcccc0cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:01.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.324+0000 7fcccaffd640 1 -- 172.21.15.154:0/53344051 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3353719778 0 0) 0x7fcccc0cb920 con 0x7fcccc06c320 2023-09-23T03:29:01.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.324+0000 7fcccaffd640 1 -- 172.21.15.154:0/53344051 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccb8003520 con 0x7fcccc06c320 2023-09-23T03:29:01.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.325+0000 7fcccaffd640 1 -- 172.21.15.154:0/53344051 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1568150493 0 0) 0x7fccb8003520 con 0x7fcccc06c320 2023-09-23T03:29:01.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.325+0000 7fcccaffd640 1 -- 172.21.15.154:0/53344051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcccc0caf80 msgr2=0x7fcccc0cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:29:01.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.325+0000 7fcccaffd640 1 --2- 172.21.15.154:0/53344051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcccc0caf80 0x7fcccc0cb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:01.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.325+0000 7fcccaffd640 1 -- 172.21.15.154:0/53344051 >> v1:172.21.15.190:6789/0 conn(0x7fcccc144130 legacy=0x7fcccc0b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:01.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.325+0000 7fcccaffd640 1 -- 172.21.15.154:0/53344051 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcccc070330 con 0x7fcccc06c320 2023-09-23T03:29:01.327 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.325+0000 7fcccaffd640 1 -- 172.21.15.154:0/53344051 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7fccc4001670 con 0x7fcccc06c320 2023-09-23T03:29:01.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.325+0000 7fcccaffd640 1 -- 172.21.15.154:0/53344051 >> v1:172.21.15.154:6789/0 conn(0x7fcccc06c320 legacy=0x7fcccc06c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:01.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.325+0000 7fcccaffd640 1 --2- 172.21.15.154:0/53344051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccb80039a0 0x7fccb8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:01.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.325+0000 7fcccaffd640 1 --2- 172.21.15.154:0/53344051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccb80047c0 0x7fccb8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:01.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.325+0000 7fcccaffd640 1 --2- 172.21.15.154:0/53344051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcccc0caf80 0x7fccb8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:01.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.325+0000 7fcccaffd640 1 -- 172.21.15.154:0/53344051 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc070fb0 con 0x7fccb80047c0 2023-09-23T03:29:01.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.325+0000 7fcccaffd640 1 -- 172.21.15.154:0/53344051 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb800ba30 con 0x7fccb80039a0 2023-09-23T03:29:01.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.325+0000 7fcccaffd640 1 -- 172.21.15.154:0/53344051 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb800bbf0 con 0x7fcccc0caf80 2023-09-23T03:29:01.328 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.325+0000 7fcccb7fe640 1 --2- 172.21.15.154:0/53344051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccb80047c0 0x7fccb8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:01.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.325+0000 7fcccbfff640 1 --2- 172.21.15.154:0/53344051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccb80039a0 0x7fccb8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:01.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.325+0000 7fcccb7fe640 1 -- 172.21.15.154:0/53344051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcccc0caf80 msgr2=0x7fccb8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:29:01.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.325+0000 7fccd0cd6640 1 --2- 172.21.15.154:0/53344051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcccc0caf80 0x7fccb8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:01.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.326+0000 7fcccb7fe640 1 --2- 172.21.15.154:0/53344051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcccc0caf80 0x7fccb8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:01.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.326+0000 7fcccb7fe640 1 -- 172.21.15.154:0/53344051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccb80039a0 msgr2=0x7fccb8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:29:01.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.326+0000 7fcccb7fe640 1 --2- 172.21.15.154:0/53344051 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fccb80039a0 0x7fccb8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:01.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.326+0000 7fcccb7fe640 1 -- 172.21.15.154:0/53344051 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb8003520 con 0x7fccb80047c0 2023-09-23T03:29:01.330 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.326+0000 7fccd0cd6640 1 --2- 172.21.15.154:0/53344051 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcccc0caf80 0x7fccb8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:29:01.332 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.327+0000 7fcccb7fe640 1 --2- 172.21.15.154:0/53344051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccb80047c0 0x7fccb8004c40 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7fccb4009a90 tx=0x7fccb80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=824bb15e65a6b05d server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:29:01.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.327+0000 7fcccaffd640 1 -- 172.21.15.154:0/53344051 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb402c660 con 0x7fccb80047c0 2023-09-23T03:29:01.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.327+0000 7fcccaffd640 1 -- 172.21.15.154:0/53344051 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb402c7c0 con 0x7fccb80047c0 2023-09-23T03:29:01.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.327+0000 7fcccaffd640 1 -- 172.21.15.154:0/53344051 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb402cb50 con 0x7fccb80047c0 2023-09-23T03:29:01.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.327+0000 7fccd2739640 1 -- 172.21.15.154:0/53344051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccb80047c0 msgr2=0x7fccb8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:01.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.327+0000 7fccd2739640 1 --2- 172.21.15.154:0/53344051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccb80047c0 0x7fccb8004c40 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7fccb4009a90 tx=0x7fccb80057f0 comp rx=0 tx=0).stop 2023-09-23T03:29:01.333 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.327+0000 7fccd0cd6640 1 -- 172.21.15.154:0/53344051 reap_dead start 2023-09-23T03:29:01.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.328+0000 7fccd2739640 1 -- 172.21.15.154:0/53344051 shutdown_connections 2023-09-23T03:29:01.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.328+0000 7fccd2739640 1 -- 172.21.15.154:0/53344051 >> 172.21.15.154:0/53344051 conn(0x7fcccc06a440 msgr2=0x7fcccc0b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:29:01.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.328+0000 7fccd2739640 1 -- 172.21.15.154:0/53344051 shutdown_connections 2023-09-23T03:29:01.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.328+0000 7fccd2739640 1 -- 172.21.15.154:0/53344051 wait complete. 2023-09-23T03:29:01.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.329+0000 7fccd2739640 1 Processor -- start 2023-09-23T03:29:01.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.330+0000 7fccd2739640 1 -- start start 2023-09-23T03:29:01.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.330+0000 7fccd2739640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccb80047c0 0x7fcccc0b0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:01.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.331+0000 7fccd2739640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcccc13e470 0x7fcccc0b0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:01.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.331+0000 7fccd2739640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcccc0c7070 0x7fcccc0c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:01.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.331+0000 7fccd2739640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc0b2280 con 0x7fccb80047c0 2023-09-23T03:29:01.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.331+0000 7fccd2739640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc0c50b0 con 0x7fcccc13e470 2023-09-23T03:29:01.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.331+0000 7fccd2739640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc0c6130 con 0x7fcccc0c7070 2023-09-23T03:29:01.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.331+0000 7fcccbfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccb80047c0 0x7fcccc0b0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:01.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.331+0000 7fcccbfff640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccb80047c0 0x7fcccc0b0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45008/0 (socket says 172.21.15.154:45008) 2023-09-23T03:29:01.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.331+0000 7fcccbfff640 1 -- 172.21.15.154:0/2763790128 learned_addr learned my addr 172.21.15.154:0/2763790128 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:29:01.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.331+0000 7fcccb7fe640 1 --2- 172.21.15.154:0/2763790128 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcccc13e470 0x7fcccc0b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:01.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.331+0000 7fccd0cd6640 1 --2- 172.21.15.154:0/2763790128 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcccc0c7070 0x7fcccc0c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:01.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.331+0000 7fcccbfff640 1 -- 172.21.15.154:0/2763790128 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcccc0c7070 msgr2=0x7fcccc0c74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:01.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.331+0000 7fcccbfff640 1 --2- 172.21.15.154:0/2763790128 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fcccc0c7070 0x7fcccc0c74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:01.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.331+0000 7fcccbfff640 1 -- 172.21.15.154:0/2763790128 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcccc13e470 msgr2=0x7fcccc0b0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:01.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.331+0000 7fcccbfff640 1 --2- 172.21.15.154:0/2763790128 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fcccc13e470 0x7fcccc0b0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:01.336 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.331+0000 7fcccbfff640 1 -- 172.21.15.154:0/2763790128 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcccc0c5370 con 0x7fccb80047c0 2023-09-23T03:29:01.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.332+0000 7fcccbfff640 1 --2- 172.21.15.154:0/2763790128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccb80047c0 0x7fcccc0b0240 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fccbc00e8c0 tx=0x7fccbc00ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:29:01.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.332+0000 7fccc8ff9640 1 -- 172.21.15.154:0/2763790128 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc003b80 con 0x7fccb80047c0 2023-09-23T03:29:01.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.332+0000 7fccc8ff9640 1 -- 172.21.15.154:0/2763790128 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccbc003ce0 con 0x7fccb80047c0 2023-09-23T03:29:01.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.332+0000 7fccc8ff9640 1 -- 172.21.15.154:0/2763790128 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc0052f0 con 0x7fccb80047c0 2023-09-23T03:29:01.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.332+0000 7fccd2739640 1 -- 172.21.15.154:0/2763790128 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcccc0c5650 con 0x7fccb80047c0 2023-09-23T03:29:01.337 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.332+0000 7fccd2739640 1 -- 172.21.15.154:0/2763790128 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcccc0c5c10 con 0x7fccb80047c0 2023-09-23T03:29:01.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.333+0000 7fccd2739640 1 -- 172.21.15.154:0/2763790128 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcccc0708f0 con 0x7fccb80047c0 2023-09-23T03:29:01.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.334+0000 7fccc8ff9640 1 -- 172.21.15.154:0/2763790128 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fccbc005450 con 0x7fccb80047c0 2023-09-23T03:29:01.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.335+0000 7fccc8ff9640 1 --2- 172.21.15.154:0/2763790128 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fccac06af60 0x7fccac06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:01.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.335+0000 7fccc8ff9640 1 -- 172.21.15.154:0/2763790128 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fccbc0897d0 con 0x7fccb80047c0 2023-09-23T03:29:01.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.335+0000 7fcccb7fe640 1 --2- 172.21.15.154:0/2763790128 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fccac06af60 0x7fccac06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:01.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.335+0000 7fccc8ff9640 1 --2- 172.21.15.154:0/2763790128 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x7fccac0713a0 0x7fccac0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:01.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.335+0000 7fccc8ff9640 1 -- 172.21.15.154:0/2763790128 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fccac073eb0 con 0x7fccac0713a0 2023-09-23T03:29:01.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.335+0000 7fccc8ff9640 1 -- 172.21.15.154:0/2763790128 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=131) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccbc08bd30 con 0x7fccb80047c0 2023-09-23T03:29:01.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.336+0000 7fccd0cd6640 1 --2- 172.21.15.154:0/2763790128 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x7fccac0713a0 0x7fccac0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:01.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.336+0000 7fcccb7fe640 1 --2- 172.21.15.154:0/2763790128 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fccac06af60 0x7fccac06d420 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fccb4036000 tx=0x7fccb4030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:29:01.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.337+0000 7fccd0cd6640 1 --2- 172.21.15.154:0/2763790128 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x7fccac0713a0 0x7fccac0737e0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:29:01.340 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.339+0000 7fccc8ff9640 1 -- 172.21.15.154:0/2763790128 <== osd.6 v2:172.21.15.190:6816/1964294001 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fccac073eb0 con 0x7fccac0713a0 2023-09-23T03:29:01.353 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.353+0000 7fccd2739640 1 -- 172.21.15.154:0/2763790128 --> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fcccc070610 con 0x7fccac0713a0 2023-09-23T03:29:01.354 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.354+0000 7fccc8ff9640 1 -- 172.21.15.154:0/2763790128 <== osd.6 v2:172.21.15.190:6816/1964294001 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fccc400d040 con 0x7fccac0713a0 2023-09-23T03:29:01.355 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:29:01.355 INFO:teuthology.orchestra.run.smithi154.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-23T03:29:01.355 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:29:01.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.355+0000 7fccd2739640 1 -- 172.21.15.154:0/2763790128 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x7fccac0713a0 msgr2=0x7fccac0737e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:01.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.355+0000 7fccd2739640 1 --2- 172.21.15.154:0/2763790128 >> [v2:172.21.15.190:6816/1964294001,v1:172.21.15.190:6817/1964294001] conn(0x7fccac0713a0 0x7fccac0737e0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:01.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.355+0000 7fccd2739640 1 -- 172.21.15.154:0/2763790128 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fccac06af60 msgr2=0x7fccac06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:01.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.355+0000 7fccd2739640 1 --2- 172.21.15.154:0/2763790128 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fccac06af60 0x7fccac06d420 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fccb4036000 tx=0x7fccb4030000 comp rx=0 tx=0).stop 2023-09-23T03:29:01.356 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.355+0000 7fccd2739640 1 -- 172.21.15.154:0/2763790128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccb80047c0 msgr2=0x7fcccc0b0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:01.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.355+0000 7fccd2739640 1 --2- 172.21.15.154:0/2763790128 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccb80047c0 0x7fcccc0b0240 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fccbc00e8c0 tx=0x7fccbc00ed80 comp rx=0 tx=0).stop 2023-09-23T03:29:01.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.355+0000 7fccd0cd6640 1 -- 172.21.15.154:0/2763790128 reap_dead start 2023-09-23T03:29:01.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.355+0000 7fccd2739640 1 -- 172.21.15.154:0/2763790128 shutdown_connections 2023-09-23T03:29:01.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.355+0000 7fccd2739640 1 -- 172.21.15.154:0/2763790128 >> 172.21.15.154:0/2763790128 conn(0x7fcccc06a440 msgr2=0x7fcccc15b180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:29:01.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.355+0000 7fccd2739640 1 -- 172.21.15.154:0/2763790128 shutdown_connections 2023-09-23T03:29:01.357 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.355+0000 7fccd2739640 1 -- 172.21.15.154:0/2763790128 wait complete. 2023-09-23T03:29:01.372 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-09-23T03:29:01.598 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.590+0000 7f598d0af640 1 Processor -- start 2023-09-23T03:29:01.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.590+0000 7f598d0af640 1 -- start start 2023-09-23T03:29:01.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.592+0000 7f598d0af640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f59880b3e50 0x7f598816dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:01.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.592+0000 7f598d0af640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f598806e3b0 0x7f59880cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:01.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.592+0000 7f598d0af640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f598816d230 con 0x7f5988070ba0 2023-09-23T03:29:01.599 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.592+0000 7f598d0af640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5988169bd0 con 0x7f59880b3e50 2023-09-23T03:29:01.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.592+0000 7f598d0af640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59880b4230 con 0x7f598806e3b0 2023-09-23T03:29:01.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.592+0000 7f5986d9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f59880b3e50 0x7f598816dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:01.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.592+0000 7f5986d9d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f59880b3e50 0x7f598816dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45010/0 (socket says 172.21.15.154:45010) 2023-09-23T03:29:01.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.592+0000 7f5986d9d640 1 -- 172.21.15.154:0/4181435811 learned_addr learned my addr 172.21.15.154:0/4181435811 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:29:01.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.592+0000 7f598659c640 1 --2- 172.21.15.154:0/4181435811 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f598806e3b0 0x7f59880cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:01.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.593+0000 7f5985d9b640 1 -- 172.21.15.154:0/4181435811 <== mon.1 v1:172.21.15.190:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4280292036 0 0) 0x7f598816d230 con 0x7f5988070ba0 2023-09-23T03:29:01.600 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.597+0000 7f5985d9b640 1 -- 172.21.15.154:0/4181435811 --> v1:172.21.15.190:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5970007930 con 0x7f5988070ba0 2023-09-23T03:29:01.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.597+0000 7f5986d9d640 1 -- 172.21.15.154:0/4181435811 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f598806e3b0 msgr2=0x7f59880cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:29:01.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.597+0000 7f5986d9d640 1 --2- 172.21.15.154:0/4181435811 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f598806e3b0 0x7f59880cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:01.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.597+0000 7f5986d9d640 1 -- 172.21.15.154:0/4181435811 >> v1:172.21.15.190:6789/0 conn(0x7f5988070ba0 legacy=0x7f59880cc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:01.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.597+0000 7f5986d9d640 1 -- 172.21.15.154:0/4181435811 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59880cdc50 con 0x7f59880b3e50 2023-09-23T03:29:01.601 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.599+0000 7f5986d9d640 1 --2- 172.21.15.154:0/4181435811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59880b3e50 0x7f598816dc80 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f597800a910 tx=0x7f5978034620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec52f390c2a11dd0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:29:01.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.599+0000 7f5985d9b640 1 -- 172.21.15.154:0/4181435811 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5978034ea0 con 0x7f59880b3e50 2023-09-23T03:29:01.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.599+0000 7f5985d9b640 1 -- 172.21.15.154:0/4181435811 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5978037030 con 0x7f59880b3e50 2023-09-23T03:29:01.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.599+0000 7f5985d9b640 1 -- 172.21.15.154:0/4181435811 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597803a680 con 0x7f59880b3e50 2023-09-23T03:29:01.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.599+0000 7f598d0af640 1 -- 172.21.15.154:0/4181435811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59880b3e50 msgr2=0x7f598816dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:01.603 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.599+0000 7f598d0af640 1 --2- 172.21.15.154:0/4181435811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59880b3e50 0x7f598816dc80 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f597800a910 tx=0x7f5978034620 comp rx=0 tx=0).stop 2023-09-23T03:29:01.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.600+0000 7f598d0af640 1 -- 172.21.15.154:0/4181435811 shutdown_connections 2023-09-23T03:29:01.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.600+0000 7f598d0af640 1 --2- 172.21.15.154:0/4181435811 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f598806e3b0 0x7f59880cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:01.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.600+0000 7f598d0af640 1 --2- 172.21.15.154:0/4181435811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f59880b3e50 0x7f598816dc80 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:01.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.600+0000 7f598d0af640 1 -- 172.21.15.154:0/4181435811 >> 172.21.15.154:0/4181435811 conn(0x7f5988069cd0 msgr2=0x7f598806a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:29:01.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.600+0000 7f598d0af640 1 -- 172.21.15.154:0/4181435811 shutdown_connections 2023-09-23T03:29:01.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.600+0000 7f598d0af640 1 -- 172.21.15.154:0/4181435811 wait complete. 2023-09-23T03:29:01.604 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.602+0000 7f598d0af640 1 Processor -- start 2023-09-23T03:29:01.605 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.602+0000 7f598d0af640 1 -- start start 2023-09-23T03:29:01.606 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.603+0000 7f598d0af640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f598806e3b0 0x7f5988167590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:01.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.603+0000 7f598d0af640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5988070ba0 0x7f598816a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:01.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.603+0000 7f598d0af640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59880b3e50 0x7f598816a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:01.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.603+0000 7f598d0af640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598816af00 con 0x7f598806e3b0 2023-09-23T03:29:01.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.603+0000 7f598d0af640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5988167ad0 con 0x7f59880b3e50 2023-09-23T03:29:01.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.603+0000 7f598d0af640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f598816b710 con 0x7f5988070ba0 2023-09-23T03:29:01.607 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.604+0000 7f5986d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f598806e3b0 0x7f5988167590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:01.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.604+0000 7f5986d9d640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f598806e3b0 0x7f5988167590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45012/0 (socket says 172.21.15.154:45012) 2023-09-23T03:29:01.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.604+0000 7f5986d9d640 1 -- 172.21.15.154:0/3679525727 learned_addr learned my addr 172.21.15.154:0/3679525727 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:29:01.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.604+0000 7f598659c640 1 --2- 172.21.15.154:0/3679525727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5988070ba0 0x7f598816a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:01.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.604+0000 7f598759e640 1 --2- 172.21.15.154:0/3679525727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59880b3e50 0x7f598816a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:01.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.604+0000 7f5986d9d640 1 -- 172.21.15.154:0/3679525727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5988070ba0 msgr2=0x7f598816a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:01.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.604+0000 7f5986d9d640 1 --2- 172.21.15.154:0/3679525727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5988070ba0 0x7f598816a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:01.608 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.604+0000 7f5986d9d640 1 -- 172.21.15.154:0/3679525727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59880b3e50 msgr2=0x7f598816a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:01.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.604+0000 7f5986d9d640 1 --2- 172.21.15.154:0/3679525727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59880b3e50 0x7f598816a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:01.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.604+0000 7f5986d9d640 1 -- 172.21.15.154:0/3679525727 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f597800a570 con 0x7f598806e3b0 2023-09-23T03:29:01.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.604+0000 7f5986d9d640 1 --2- 172.21.15.154:0/3679525727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f598806e3b0 0x7f5988167590 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f5978000c00 tx=0x7f59780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:29:01.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.605+0000 7f596f7fe640 1 -- 172.21.15.154:0/3679525727 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5978034ea0 con 0x7f598806e3b0 2023-09-23T03:29:01.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.605+0000 7f596f7fe640 1 -- 172.21.15.154:0/3679525727 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5978003c50 con 0x7f598806e3b0 2023-09-23T03:29:01.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.605+0000 7f596f7fe640 1 -- 172.21.15.154:0/3679525727 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597803a480 con 0x7f598806e3b0 2023-09-23T03:29:01.609 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.605+0000 7f598d0af640 1 -- 172.21.15.154:0/3679525727 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f598816c970 con 0x7f598806e3b0 2023-09-23T03:29:01.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.605+0000 7f598d0af640 1 -- 172.21.15.154:0/3679525727 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f598816b430 con 0x7f598806e3b0 2023-09-23T03:29:01.610 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.606+0000 7f598d0af640 1 -- 172.21.15.154:0/3679525727 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59881feb20 con 0x7f598806e3b0 2023-09-23T03:29:01.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.607+0000 7f596f7fe640 1 -- 172.21.15.154:0/3679525727 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f597803a5e0 con 0x7f598806e3b0 2023-09-23T03:29:01.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.608+0000 7f596f7fe640 1 --2- 172.21.15.154:0/3679525727 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f597406af30 0x7f597406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:01.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.608+0000 7f596f7fe640 1 -- 172.21.15.154:0/3679525727 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f59780b4200 con 0x7f598806e3b0 2023-09-23T03:29:01.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.608+0000 7f598659c640 1 --2- 172.21.15.154:0/3679525727 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f597406af30 0x7f597406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:01.611 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.609+0000 7f598659c640 1 --2- 172.21.15.154:0/3679525727 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f597406af30 0x7f597406d3f0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f59880ca2a0 tx=0x7f597c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:29:01.613 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.613+0000 7f596f7fe640 1 -- 172.21.15.154:0/3679525727 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f597807f460 con 0x7f598806e3b0 2023-09-23T03:29:01.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.806+0000 7f598d0af640 1 -- 172.21.15.154:0/3679525727 --> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f598816d480 con 0x7f597406af30 2023-09-23T03:29:01.807 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.807+0000 7f596f7fe640 1 -- 172.21.15.154:0/3679525727 <== mgr.4110 v2:172.21.15.154:6832/2422652104 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f597c010910 con 0x7f597406af30 2023-09-23T03:29:01.808 INFO:teuthology.orchestra.run.smithi154.stderr:instructed osd(s) 6 to deep-scrub 2023-09-23T03:29:01.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.809+0000 7f598d0af640 1 -- 172.21.15.154:0/3679525727 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f597406af30 msgr2=0x7f597406d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:01.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.810+0000 7f598d0af640 1 --2- 172.21.15.154:0/3679525727 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f597406af30 0x7f597406d3f0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f59880ca2a0 tx=0x7f597c008040 comp rx=0 tx=0).stop 2023-09-23T03:29:01.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.810+0000 7f598d0af640 1 -- 172.21.15.154:0/3679525727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f598806e3b0 msgr2=0x7f5988167590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:01.810 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.810+0000 7f598d0af640 1 --2- 172.21.15.154:0/3679525727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f598806e3b0 0x7f5988167590 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f5978000c00 tx=0x7f59780046c0 comp rx=0 tx=0).stop 2023-09-23T03:29:01.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.810+0000 7f598d0af640 1 -- 172.21.15.154:0/3679525727 shutdown_connections 2023-09-23T03:29:01.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.810+0000 7f598d0af640 1 --2- 172.21.15.154:0/3679525727 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f597406af30 0x7f597406d3f0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:01.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.810+0000 7f598d0af640 1 --2- 172.21.15.154:0/3679525727 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f59880b3e50 0x7f598816a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:01.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.810+0000 7f598d0af640 1 --2- 172.21.15.154:0/3679525727 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f5988070ba0 0x7f598816a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:01.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.810+0000 7f598d0af640 1 --2- 172.21.15.154:0/3679525727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f598806e3b0 0x7f5988167590 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:01.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.810+0000 7f598d0af640 1 -- 172.21.15.154:0/3679525727 >> 172.21.15.154:0/3679525727 conn(0x7f5988069cd0 msgr2=0x7f598806d880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:29:01.811 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.810+0000 7f598d0af640 1 -- 172.21.15.154:0/3679525727 shutdown_connections 2023-09-23T03:29:01.812 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:01.810+0000 7f598d0af640 1 -- 172.21.15.154:0/3679525727 wait complete. 2023-09-23T03:29:01.827 INFO:tasks.ceph:Scrubbing osd.7 2023-09-23T03:29:01.828 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-09-23T03:29:02.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.047+0000 7f2f54a00640 1 Processor -- start 2023-09-23T03:29:02.048 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.048+0000 7f2f54a00640 1 -- start start 2023-09-23T03:29:02.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.049+0000 7f2f54a00640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2f500ca6b0 0x7f2f5006d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:02.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.049+0000 7f2f54a00640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f2f5006dbd0 0x7f2f500700e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:02.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.049+0000 7f2f54a00640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f5015ab20 con 0x7f2f50070620 2023-09-23T03:29:02.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.049+0000 7f2f54a00640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f50070a50 con 0x7f2f500ca6b0 2023-09-23T03:29:02.050 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.049+0000 7f2f54a00640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f50159f50 con 0x7f2f5006dbd0 2023-09-23T03:29:02.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.050+0000 7f2f4e59c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2f500ca6b0 0x7f2f5006d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:02.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.050+0000 7f2f4e59c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f2f500ca6b0 0x7f2f5006d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45024/0 (socket says 172.21.15.154:45024) 2023-09-23T03:29:02.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.050+0000 7f2f4e59c640 1 -- 172.21.15.154:0/2065394097 learned_addr learned my addr 172.21.15.154:0/2065394097 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:29:02.051 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.050+0000 7f2f4dd9b640 1 --2- 172.21.15.154:0/2065394097 >> v2:172.21.15.190:3300/0 conn(0x7f2f5006dbd0 0x7f2f500700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:02.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.050+0000 7f2f4e59c640 1 -- 172.21.15.154:0/2065394097 >> v2:172.21.15.190:3300/0 conn(0x7f2f5006dbd0 msgr2=0x7f2f500700e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:29:02.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.050+0000 7f2f4e59c640 1 --2- 172.21.15.154:0/2065394097 >> v2:172.21.15.190:3300/0 conn(0x7f2f5006dbd0 0x7f2f500700e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:02.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.050+0000 7f2f4e59c640 1 -- 172.21.15.154:0/2065394097 >> v1:172.21.15.190:6789/0 conn(0x7f2f50070620 legacy=0x7f2f5006c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:29:02.052 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.050+0000 7f2f4e59c640 1 -- 172.21.15.154:0/2065394097 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f500caa90 con 0x7f2f500ca6b0 2023-09-23T03:29:02.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.051+0000 7f2f4e59c640 1 --2- 172.21.15.154:0/2065394097 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f500ca6b0 0x7f2f5006d690 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f2f500b0210 tx=0x7f2f4002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc27f89df7ca1993 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:29:02.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.052+0000 7f2f4d59a640 1 -- 172.21.15.154:0/2065394097 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4002ca80 con 0x7f2f500ca6b0 2023-09-23T03:29:02.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.052+0000 7f2f4d59a640 1 -- 172.21.15.154:0/2065394097 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f4002cbe0 con 0x7f2f500ca6b0 2023-09-23T03:29:02.053 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.052+0000 7f2f4d59a640 1 -- 172.21.15.154:0/2065394097 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f400343f0 con 0x7f2f500ca6b0 2023-09-23T03:29:02.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.052+0000 7f2f54a00640 1 -- 172.21.15.154:0/2065394097 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f500ca6b0 msgr2=0x7f2f5006d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:02.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.052+0000 7f2f54a00640 1 --2- 172.21.15.154:0/2065394097 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f500ca6b0 0x7f2f5006d690 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f2f500b0210 tx=0x7f2f4002bee0 comp rx=0 tx=0).stop 2023-09-23T03:29:02.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.052+0000 7f2f54a00640 1 -- 172.21.15.154:0/2065394097 shutdown_connections 2023-09-23T03:29:02.054 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.052+0000 7f2f54a00640 1 --2- 172.21.15.154:0/2065394097 >> v2:172.21.15.190:3300/0 conn(0x7f2f5006dbd0 0x7f2f500700e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:02.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.052+0000 7f2f54a00640 1 --2- 172.21.15.154:0/2065394097 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f500ca6b0 0x7f2f5006d690 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:02.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.052+0000 7f2f54a00640 1 -- 172.21.15.154:0/2065394097 >> 172.21.15.154:0/2065394097 conn(0x7f2f5006a440 msgr2=0x7f2f500b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:29:02.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.053+0000 7f2f54a00640 1 -- 172.21.15.154:0/2065394097 shutdown_connections 2023-09-23T03:29:02.055 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.053+0000 7f2f54a00640 1 -- 172.21.15.154:0/2065394097 wait complete. 2023-09-23T03:29:02.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.054+0000 7f2f54a00640 1 Processor -- start 2023-09-23T03:29:02.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.055+0000 7f2f54a00640 1 -- start start 2023-09-23T03:29:02.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.055+0000 7f2f54a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f5006dbd0 0x7f2f500cd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:02.057 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.055+0000 7f2f54a00640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f50070620 0x7f2f5013bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:02.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.056+0000 7f2f54a00640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f500ca6b0 0x7f2f5013e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:02.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.056+0000 7f2f54a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5013c0a0 con 0x7f2f5006dbd0 2023-09-23T03:29:02.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.056+0000 7f2f54a00640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f500cc190 con 0x7f2f50070620 2023-09-23T03:29:02.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.056+0000 7f2f54a00640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f500cc470 con 0x7f2f500ca6b0 2023-09-23T03:29:02.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.056+0000 7f2f4e59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f5006dbd0 0x7f2f500cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:02.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.056+0000 7f2f4e59c640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f5006dbd0 0x7f2f500cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45040/0 (socket says 172.21.15.154:45040) 2023-09-23T03:29:02.058 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.056+0000 7f2f4e59c640 1 -- 172.21.15.154:0/1066700672 learned_addr learned my addr 172.21.15.154:0/1066700672 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:29:02.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.056+0000 7f2f4dd9b640 1 --2- 172.21.15.154:0/1066700672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f50070620 0x7f2f5013bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:02.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.056+0000 7f2f4e59c640 1 -- 172.21.15.154:0/1066700672 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f500ca6b0 msgr2=0x7f2f5013e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:02.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.056+0000 7f2f4e59c640 1 --2- 172.21.15.154:0/1066700672 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f2f500ca6b0 0x7f2f5013e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:02.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.056+0000 7f2f4e59c640 1 -- 172.21.15.154:0/1066700672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f50070620 msgr2=0x7f2f5013bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:02.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.056+0000 7f2f4e59c640 1 --2- 172.21.15.154:0/1066700672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f50070620 0x7f2f5013bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:02.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.056+0000 7f2f4e59c640 1 -- 172.21.15.154:0/1066700672 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f40009b30 con 0x7f2f5006dbd0 2023-09-23T03:29:02.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.056+0000 7f2f4dd9b640 1 --2- 172.21.15.154:0/1066700672 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f2f50070620 0x7f2f5013bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:29:02.059 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.057+0000 7f2f4e59c640 1 --2- 172.21.15.154:0/1066700672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f5006dbd0 0x7f2f500cd230 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f2f40000f80 tx=0x7f2f40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:29:02.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.057+0000 7f2f46ffd640 1 -- 172.21.15.154:0/1066700672 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f40009510 con 0x7f2f5006dbd0 2023-09-23T03:29:02.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.057+0000 7f2f46ffd640 1 -- 172.21.15.154:0/1066700672 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f4003e450 con 0x7f2f5006dbd0 2023-09-23T03:29:02.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.057+0000 7f2f46ffd640 1 -- 172.21.15.154:0/1066700672 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4003e800 con 0x7f2f5006dbd0 2023-09-23T03:29:02.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.057+0000 7f2f54a00640 1 -- 172.21.15.154:0/1066700672 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f500cc750 con 0x7f2f5006dbd0 2023-09-23T03:29:02.060 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.057+0000 7f2f54a00640 1 -- 172.21.15.154:0/1066700672 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f500ccd10 con 0x7f2f5006dbd0 2023-09-23T03:29:02.061 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.058+0000 7f2f54a00640 1 -- 172.21.15.154:0/1066700672 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2f5015a4d0 con 0x7f2f5006dbd0 2023-09-23T03:29:02.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.059+0000 7f2f46ffd640 1 -- 172.21.15.154:0/1066700672 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f2f4002cd50 con 0x7f2f5006dbd0 2023-09-23T03:29:02.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.060+0000 7f2f46ffd640 1 --2- 172.21.15.154:0/1066700672 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2f3006af30 0x7f2f3006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:02.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.060+0000 7f2f46ffd640 1 -- 172.21.15.154:0/1066700672 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f2f400ac220 con 0x7f2f5006dbd0 2023-09-23T03:29:02.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.061+0000 7f2f4dd9b640 1 --2- 172.21.15.154:0/1066700672 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2f3006af30 0x7f2f3006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:02.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.061+0000 7f2f46ffd640 1 --2- 172.21.15.154:0/1066700672 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x7f2f30071370 0x7f2f300737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:02.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.061+0000 7f2f46ffd640 1 -- 172.21.15.154:0/1066700672 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2f30073e80 con 0x7f2f30071370 2023-09-23T03:29:02.062 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.061+0000 7f2f46ffd640 1 -- 172.21.15.154:0/1066700672 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_get_version_reply(handle=1 version=131) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f400ae800 con 0x7f2f5006dbd0 2023-09-23T03:29:02.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.061+0000 7f2f4dd9b640 1 --2- 172.21.15.154:0/1066700672 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2f3006af30 0x7f2f3006d3f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f2f38006fd0 tx=0x7f2f38008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:29:02.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.061+0000 7f2f4ed9d640 1 --2- 172.21.15.154:0/1066700672 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x7f2f30071370 0x7f2f300737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:02.063 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.062+0000 7f2f4ed9d640 1 --2- 172.21.15.154:0/1066700672 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x7f2f30071370 0x7f2f300737b0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:29:02.065 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.064+0000 7f2f46ffd640 1 -- 172.21.15.154:0/1066700672 <== osd.7 v2:172.21.15.190:6824/3460136272 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f2f30073e80 con 0x7f2f30071370 2023-09-23T03:29:02.078 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.078+0000 7f2f54a00640 1 -- 172.21.15.154:0/1066700672 --> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2f5015a1f0 con 0x7f2f30071370 2023-09-23T03:29:02.079 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.079+0000 7f2f46ffd640 1 -- 172.21.15.154:0/1066700672 <== osd.7 v2:172.21.15.190:6824/3460136272 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f2f48009720 con 0x7f2f30071370 2023-09-23T03:29:02.080 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2023-09-23T03:29:02.080 INFO:teuthology.orchestra.run.smithi154.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-09-23T03:29:02.080 INFO:teuthology.orchestra.run.smithi154.stdout:} 2023-09-23T03:29:02.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.080+0000 7f2f54a00640 1 -- 172.21.15.154:0/1066700672 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x7f2f30071370 msgr2=0x7f2f300737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:02.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.080+0000 7f2f54a00640 1 --2- 172.21.15.154:0/1066700672 >> [v2:172.21.15.190:6824/3460136272,v1:172.21.15.190:6825/3460136272] conn(0x7f2f30071370 0x7f2f300737b0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:02.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.080+0000 7f2f54a00640 1 -- 172.21.15.154:0/1066700672 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2f3006af30 msgr2=0x7f2f3006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:02.081 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.080+0000 7f2f54a00640 1 --2- 172.21.15.154:0/1066700672 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f2f3006af30 0x7f2f3006d3f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f2f38006fd0 tx=0x7f2f38008040 comp rx=0 tx=0).stop 2023-09-23T03:29:02.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.080+0000 7f2f54a00640 1 -- 172.21.15.154:0/1066700672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f5006dbd0 msgr2=0x7f2f500cd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:02.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.080+0000 7f2f54a00640 1 --2- 172.21.15.154:0/1066700672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f5006dbd0 0x7f2f500cd230 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f2f40000f80 tx=0x7f2f40004a40 comp rx=0 tx=0).stop 2023-09-23T03:29:02.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.080+0000 7f2f4ed9d640 1 -- 172.21.15.154:0/1066700672 reap_dead start 2023-09-23T03:29:02.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.080+0000 7f2f54a00640 1 -- 172.21.15.154:0/1066700672 shutdown_connections 2023-09-23T03:29:02.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.080+0000 7f2f54a00640 1 -- 172.21.15.154:0/1066700672 >> 172.21.15.154:0/1066700672 conn(0x7f2f5006a440 msgr2=0x7f2f5015d330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:29:02.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.080+0000 7f2f54a00640 1 -- 172.21.15.154:0/1066700672 shutdown_connections 2023-09-23T03:29:02.082 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.080+0000 7f2f54a00640 1 -- 172.21.15.154:0/1066700672 wait complete. 2023-09-23T03:29:02.097 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-09-23T03:29:02.312 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.312+0000 7f6c1b6f2640 1 Processor -- start 2023-09-23T03:29:02.313 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.312+0000 7f6c1b6f2640 1 -- start start 2023-09-23T03:29:02.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.314+0000 7f6c1b6f2640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6c140c9300 0x7f6c140cc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:02.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.314+0000 7f6c1b6f2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c14070090 0x7f6c140cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:02.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.314+0000 7f6c1b6f2640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c14070600 con 0x7f6c14143210 2023-09-23T03:29:02.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.314+0000 7f6c1b6f2640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c141435f0 con 0x7f6c140c9300 2023-09-23T03:29:02.315 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.314+0000 7f6c1b6f2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c140708e0 con 0x7f6c14070090 2023-09-23T03:29:02.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.314+0000 7f6c18c8d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6c140c9300 0x7f6c140cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:02.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.314+0000 7f6c18c8d640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f6c140c9300 0x7f6c140cc170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45050/0 (socket says 172.21.15.154:45050) 2023-09-23T03:29:02.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.314+0000 7f6c18c8d640 1 -- 172.21.15.154:0/1552698992 learned_addr learned my addr 172.21.15.154:0/1552698992 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:29:02.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.315+0000 7f6c19c8f640 1 --2- 172.21.15.154:0/1552698992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c14070090 0x7f6c140cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:02.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.315+0000 7f6c18c8d640 1 -- 172.21.15.154:0/1552698992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c14070090 msgr2=0x7f6c140cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:29:02.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.315+0000 7f6c18c8d640 1 --2- 172.21.15.154:0/1552698992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c14070090 0x7f6c140cc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:02.316 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.315+0000 7f6c18c8d640 1 -- 172.21.15.154:0/1552698992 >> v1:172.21.15.190:6789/0 conn(0x7f6c14143210 legacy=0x7f6c140c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:29:02.317 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.315+0000 7f6c18c8d640 1 -- 172.21.15.154:0/1552698992 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c14070bc0 con 0x7f6c140c9300 2023-09-23T03:29:02.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.316+0000 7f6c18c8d640 1 --2- 172.21.15.154:0/1552698992 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c140c9300 0x7f6c140cc170 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f6c04009870 tx=0x7f6c0402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e2dea7cb8667cb server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:29:02.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.316+0000 7f6c0bfff640 1 -- 172.21.15.154:0/1552698992 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c04031640 con 0x7f6c140c9300 2023-09-23T03:29:02.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.316+0000 7f6c0bfff640 1 -- 172.21.15.154:0/1552698992 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c040317a0 con 0x7f6c140c9300 2023-09-23T03:29:02.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.316+0000 7f6c0bfff640 1 -- 172.21.15.154:0/1552698992 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c04031b30 con 0x7f6c140c9300 2023-09-23T03:29:02.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.317+0000 7f6c1b6f2640 1 -- 172.21.15.154:0/1552698992 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c140c9300 msgr2=0x7f6c140cc170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:02.318 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.317+0000 7f6c1b6f2640 1 --2- 172.21.15.154:0/1552698992 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c140c9300 0x7f6c140cc170 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f6c04009870 tx=0x7f6c0402b8f0 comp rx=0 tx=0).stop 2023-09-23T03:29:02.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.317+0000 7f6c1b6f2640 1 -- 172.21.15.154:0/1552698992 shutdown_connections 2023-09-23T03:29:02.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.317+0000 7f6c1b6f2640 1 --2- 172.21.15.154:0/1552698992 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c14070090 0x7f6c140cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:02.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.317+0000 7f6c1b6f2640 1 --2- 172.21.15.154:0/1552698992 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c140c9300 0x7f6c140cc170 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:02.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.317+0000 7f6c1b6f2640 1 -- 172.21.15.154:0/1552698992 >> 172.21.15.154:0/1552698992 conn(0x7f6c1406c820 msgr2=0x7f6c14069cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:29:02.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.317+0000 7f6c1b6f2640 1 -- 172.21.15.154:0/1552698992 shutdown_connections 2023-09-23T03:29:02.319 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.317+0000 7f6c1b6f2640 1 -- 172.21.15.154:0/1552698992 wait complete. 2023-09-23T03:29:02.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.319+0000 7f6c1b6f2640 1 Processor -- start 2023-09-23T03:29:02.320 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.319+0000 7f6c1b6f2640 1 -- start start 2023-09-23T03:29:02.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.320+0000 7f6c1b6f2640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c14070090 0x7f6c141552c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:02.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.320+0000 7f6c1b6f2640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c140c9300 0x7f6c1413a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:02.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.320+0000 7f6c1b6f2640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c14143210 0x7f6c14138950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:02.321 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.320+0000 7f6c1b6f2640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c14155800 con 0x7f6c14070090 2023-09-23T03:29:02.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.320+0000 7f6c1b6f2640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c140cdb90 con 0x7f6c14143210 2023-09-23T03:29:02.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.320+0000 7f6c1b6f2640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c140cde70 con 0x7f6c140c9300 2023-09-23T03:29:02.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.320+0000 7f6c1948e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c14070090 0x7f6c141552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:02.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.320+0000 7f6c1948e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c14070090 0x7f6c141552c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45060/0 (socket says 172.21.15.154:45060) 2023-09-23T03:29:02.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.320+0000 7f6c1948e640 1 -- 172.21.15.154:0/934081005 learned_addr learned my addr 172.21.15.154:0/934081005 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:29:02.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.321+0000 7f6c1948e640 1 -- 172.21.15.154:0/934081005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c140c9300 msgr2=0x7f6c1413a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:02.322 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.321+0000 7f6c1948e640 1 --2- 172.21.15.154:0/934081005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c140c9300 0x7f6c1413a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:02.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.321+0000 7f6c1948e640 1 -- 172.21.15.154:0/934081005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c14143210 msgr2=0x7f6c14138950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:02.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.321+0000 7f6c1948e640 1 --2- 172.21.15.154:0/934081005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c14143210 0x7f6c14138950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:02.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.321+0000 7f6c1948e640 1 -- 172.21.15.154:0/934081005 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c040094d0 con 0x7f6c14070090 2023-09-23T03:29:02.323 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.321+0000 7f6c1948e640 1 --2- 172.21.15.154:0/934081005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c14070090 0x7f6c141552c0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f6c1000b3f0 tx=0x7f6c1000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:29:02.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.321+0000 7f6c09ffb640 1 -- 172.21.15.154:0/934081005 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c10011830 con 0x7f6c14070090 2023-09-23T03:29:02.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.321+0000 7f6c09ffb640 1 -- 172.21.15.154:0/934081005 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c10011990 con 0x7f6c14070090 2023-09-23T03:29:02.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.321+0000 7f6c1b6f2640 1 -- 172.21.15.154:0/934081005 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c14139da0 con 0x7f6c14070090 2023-09-23T03:29:02.324 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.321+0000 7f6c09ffb640 1 -- 172.21.15.154:0/934081005 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c10011d40 con 0x7f6c14070090 2023-09-23T03:29:02.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.322+0000 7f6c1b6f2640 1 -- 172.21.15.154:0/934081005 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c1413a360 con 0x7f6c14070090 2023-09-23T03:29:02.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.323+0000 7f6c1b6f2640 1 -- 172.21.15.154:0/934081005 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c1413a580 con 0x7f6c14070090 2023-09-23T03:29:02.325 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.324+0000 7f6c09ffb640 1 -- 172.21.15.154:0/934081005 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f6c1001a400 con 0x7f6c14070090 2023-09-23T03:29:02.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.325+0000 7f6c09ffb640 1 --2- 172.21.15.154:0/934081005 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6bf806af30 0x7f6bf806d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:02.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.325+0000 7f6c09ffb640 1 -- 172.21.15.154:0/934081005 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f6c10089520 con 0x7f6c14070090 2023-09-23T03:29:02.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.325+0000 7f6c18c8d640 1 --2- 172.21.15.154:0/934081005 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6bf806af30 0x7f6bf806d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:02.326 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.326+0000 7f6c18c8d640 1 --2- 172.21.15.154:0/934081005 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6bf806af30 0x7f6bf806d3f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f6c04036000 tx=0x7f6c04030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:29:02.329 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.329+0000 7f6c09ffb640 1 -- 172.21.15.154:0/934081005 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f6c100545d0 con 0x7f6c14070090 2023-09-23T03:29:02.518 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.518+0000 7f6c1b6f2640 1 -- 172.21.15.154:0/934081005 --> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c14071180 con 0x7f6bf806af30 2023-09-23T03:29:02.519 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.518+0000 7f6c09ffb640 1 -- 172.21.15.154:0/934081005 <== mgr.4110 v2:172.21.15.154:6832/2422652104 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6c04031a10 con 0x7f6bf806af30 2023-09-23T03:29:02.519 INFO:teuthology.orchestra.run.smithi154.stderr:instructed osd(s) 7 to deep-scrub 2023-09-23T03:29:02.521 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.521+0000 7f6c1b6f2640 1 -- 172.21.15.154:0/934081005 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6bf806af30 msgr2=0x7f6bf806d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:02.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.521+0000 7f6c1b6f2640 1 --2- 172.21.15.154:0/934081005 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6bf806af30 0x7f6bf806d3f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f6c04036000 tx=0x7f6c04030000 comp rx=0 tx=0).stop 2023-09-23T03:29:02.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.521+0000 7f6c1b6f2640 1 -- 172.21.15.154:0/934081005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c14070090 msgr2=0x7f6c141552c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:02.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.521+0000 7f6c1b6f2640 1 --2- 172.21.15.154:0/934081005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c14070090 0x7f6c141552c0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f6c1000b3f0 tx=0x7f6c1000b8b0 comp rx=0 tx=0).stop 2023-09-23T03:29:02.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.521+0000 7f6c1b6f2640 1 -- 172.21.15.154:0/934081005 shutdown_connections 2023-09-23T03:29:02.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.521+0000 7f6c1b6f2640 1 --2- 172.21.15.154:0/934081005 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f6bf806af30 0x7f6bf806d3f0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:02.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.521+0000 7f6c1b6f2640 1 --2- 172.21.15.154:0/934081005 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f6c14143210 0x7f6c14138950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:02.522 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.521+0000 7f6c1b6f2640 1 --2- 172.21.15.154:0/934081005 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f6c140c9300 0x7f6c1413a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:02.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.521+0000 7f6c1b6f2640 1 --2- 172.21.15.154:0/934081005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6c14070090 0x7f6c141552c0 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:02.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.521+0000 7f6c1b6f2640 1 -- 172.21.15.154:0/934081005 >> 172.21.15.154:0/934081005 conn(0x7f6c1406c820 msgr2=0x7f6c14141320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:29:02.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.521+0000 7f6c1b6f2640 1 -- 172.21.15.154:0/934081005 shutdown_connections 2023-09-23T03:29:02.523 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.521+0000 7f6c1b6f2640 1 -- 172.21.15.154:0/934081005 wait complete. 2023-09-23T03:29:02.541 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-23T03:29:02.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.756+0000 7f63b2f9e640 1 Processor -- start 2023-09-23T03:29:02.756 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.756+0000 7f63b2f9e640 1 -- start start 2023-09-23T03:29:02.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.757+0000 7f63b2f9e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f63ac158200 0x7f63ac15e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:02.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.757+0000 7f63b2f9e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63ac06c820 0x7f63ac0c8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:02.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.757+0000 7f63b2f9e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63ac0b4270 con 0x7f63ac0c8ed0 2023-09-23T03:29:02.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.757+0000 7f63b2f9e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63ac15f150 con 0x7f63ac158200 2023-09-23T03:29:02.758 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.757+0000 7f63b2f9e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac06cc60 con 0x7f63ac06c820 2023-09-23T03:29:02.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.758+0000 7f63b1f9c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f63ac158200 0x7f63ac15e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:02.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.758+0000 7f63b1f9c640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7f63ac158200 0x7f63ac15e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45068/0 (socket says 172.21.15.154:45068) 2023-09-23T03:29:02.759 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.758+0000 7f63b1f9c640 1 -- 172.21.15.154:0/3468692143 learned_addr learned my addr 172.21.15.154:0/3468692143 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:29:02.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.758+0000 7f63b1f9c640 1 -- 172.21.15.154:0/3468692143 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63ac06c820 msgr2=0x7f63ac0c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:29:02.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.758+0000 7f63b1f9c640 1 --2- 172.21.15.154:0/3468692143 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63ac06c820 0x7f63ac0c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:02.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.758+0000 7f63b1f9c640 1 -- 172.21.15.154:0/3468692143 >> v1:172.21.15.190:6789/0 conn(0x7f63ac0c8ed0 legacy=0x7f63ac15b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:29:02.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.758+0000 7f63b1f9c640 1 -- 172.21.15.154:0/3468692143 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ac15e8b0 con 0x7f63ac158200 2023-09-23T03:29:02.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.759+0000 7f63b1f9c640 1 --2- 172.21.15.154:0/3468692143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63ac158200 0x7f63ac15e2d0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f63a8009870 tx=0x7f63a802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da3952d369f79bf8 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:29:02.760 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.760+0000 7f63b0f9a640 1 -- 172.21.15.154:0/3468692143 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a80317c0 con 0x7f63ac158200 2023-09-23T03:29:02.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.760+0000 7f63b0f9a640 1 -- 172.21.15.154:0/3468692143 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a8031920 con 0x7f63ac158200 2023-09-23T03:29:02.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.760+0000 7f63b0f9a640 1 -- 172.21.15.154:0/3468692143 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a8031cb0 con 0x7f63ac158200 2023-09-23T03:29:02.761 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.760+0000 7f63b2f9e640 1 -- 172.21.15.154:0/3468692143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63ac158200 msgr2=0x7f63ac15e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:02.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.760+0000 7f63b2f9e640 1 --2- 172.21.15.154:0/3468692143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63ac158200 0x7f63ac15e2d0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f63a8009870 tx=0x7f63a802b860 comp rx=0 tx=0).stop 2023-09-23T03:29:02.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.760+0000 7f63b2f9e640 1 -- 172.21.15.154:0/3468692143 shutdown_connections 2023-09-23T03:29:02.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.760+0000 7f63b2f9e640 1 --2- 172.21.15.154:0/3468692143 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63ac06c820 0x7f63ac0c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:02.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.760+0000 7f63b2f9e640 1 --2- 172.21.15.154:0/3468692143 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63ac158200 0x7f63ac15e2d0 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:02.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.760+0000 7f63b2f9e640 1 -- 172.21.15.154:0/3468692143 >> 172.21.15.154:0/3468692143 conn(0x7f63ac06a4c0 msgr2=0x7f63ac0afe50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:29:02.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.760+0000 7f63b2f9e640 1 -- 172.21.15.154:0/3468692143 shutdown_connections 2023-09-23T03:29:02.762 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.761+0000 7f63b2f9e640 1 -- 172.21.15.154:0/3468692143 wait complete. 2023-09-23T03:29:02.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.762+0000 7f63b2f9e640 1 Processor -- start 2023-09-23T03:29:02.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.762+0000 7f63b2f9e640 1 -- start start 2023-09-23T03:29:02.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.763+0000 7f63b2f9e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63ac06c820 0x7f63ac1577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:02.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.763+0000 7f63b2f9e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63ac0c8ed0 0x7f63ac0c5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:02.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.763+0000 7f63b2f9e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63ac158200 0x7f63ac0ca400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:02.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.763+0000 7f63b2f9e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac0c1e70 con 0x7f63ac0c8ed0 2023-09-23T03:29:02.764 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.763+0000 7f63b2f9e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac0c2150 con 0x7f63ac158200 2023-09-23T03:29:02.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.763+0000 7f63b2f9e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac0c2430 con 0x7f63ac06c820 2023-09-23T03:29:02.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.763+0000 7f63b179b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63ac0c8ed0 0x7f63ac0c5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:02.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.764+0000 7f63b179b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63ac0c8ed0 0x7f63ac0c5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:45070/0 (socket says 172.21.15.154:45070) 2023-09-23T03:29:02.765 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.764+0000 7f63b179b640 1 -- 172.21.15.154:0/2185403952 learned_addr learned my addr 172.21.15.154:0/2185403952 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:29:02.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.764+0000 7f63b1f9c640 1 --2- 172.21.15.154:0/2185403952 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63ac06c820 0x7f63ac1577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:02.766 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.764+0000 7f63b279d640 1 --2- 172.21.15.154:0/2185403952 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63ac158200 0x7f63ac0ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:02.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.764+0000 7f63b179b640 1 -- 172.21.15.154:0/2185403952 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63ac06c820 msgr2=0x7f63ac1577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:02.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.764+0000 7f63b179b640 1 --2- 172.21.15.154:0/2185403952 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63ac06c820 0x7f63ac1577c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:02.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.764+0000 7f63b179b640 1 -- 172.21.15.154:0/2185403952 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63ac158200 msgr2=0x7f63ac0ca400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:02.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.764+0000 7f63b179b640 1 --2- 172.21.15.154:0/2185403952 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63ac158200 0x7f63ac0ca400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:02.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.764+0000 7f63b179b640 1 -- 172.21.15.154:0/2185403952 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a80094d0 con 0x7f63ac0c8ed0 2023-09-23T03:29:02.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.764+0000 7f63b1f9c640 1 --2- 172.21.15.154:0/2185403952 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63ac06c820 0x7f63ac1577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:29:02.767 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.764+0000 7f63b279d640 1 --2- 172.21.15.154:0/2185403952 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63ac158200 0x7f63ac0ca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:29:02.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.764+0000 7f63b179b640 1 --2- 172.21.15.154:0/2185403952 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63ac0c8ed0 0x7f63ac0c5c90 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f639c00b4a0 tx=0x7f639c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:29:02.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.764+0000 7f63a27fc640 1 -- 172.21.15.154:0/2185403952 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639c010610 con 0x7f63ac0c8ed0 2023-09-23T03:29:02.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.765+0000 7f63a27fc640 1 -- 172.21.15.154:0/2185403952 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f639c010770 con 0x7f63ac0c8ed0 2023-09-23T03:29:02.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.765+0000 7f63a27fc640 1 -- 172.21.15.154:0/2185403952 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639c010b20 con 0x7f63ac0c8ed0 2023-09-23T03:29:02.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.765+0000 7f63b2f9e640 1 -- 172.21.15.154:0/2185403952 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63ac0c2710 con 0x7f63ac0c8ed0 2023-09-23T03:29:02.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.765+0000 7f63b2f9e640 1 -- 172.21.15.154:0/2185403952 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63ac0c2cd0 con 0x7f63ac0c8ed0 2023-09-23T03:29:02.768 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.766+0000 7f63b2f9e640 1 -- 172.21.15.154:0/2185403952 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63ac066730 con 0x7f63ac0c8ed0 2023-09-23T03:29:02.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.767+0000 7f63a27fc640 1 -- 172.21.15.154:0/2185403952 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f639c010c80 con 0x7f63ac0c8ed0 2023-09-23T03:29:02.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.768+0000 7f63a27fc640 1 --2- 172.21.15.154:0/2185403952 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f639006af30 0x7f639006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:02.769 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.768+0000 7f63a27fc640 1 -- 172.21.15.154:0/2185403952 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f639c0892b0 con 0x7f63ac0c8ed0 2023-09-23T03:29:02.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.768+0000 7f63b1f9c640 1 --2- 172.21.15.154:0/2185403952 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f639006af30 0x7f639006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:02.770 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.769+0000 7f63b1f9c640 1 --2- 172.21.15.154:0/2185403952 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f639006af30 0x7f639006d3f0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f63a8036000 tx=0x7f63a803d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:29:02.772 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.772+0000 7f63a27fc640 1 -- 172.21.15.154:0/2185403952 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f639c054400 con 0x7f63ac0c8ed0 2023-09-23T03:29:02.958 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.958+0000 7f63b2f9e640 1 -- 172.21.15.154:0/2185403952 --> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f63ac15ee70 con 0x7f639006af30 2023-09-23T03:29:02.963 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.962+0000 7f63a27fc640 1 -- 172.21.15.154:0/2185403952 <== mgr.4110 v2:172.21.15.154:6832/2422652104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48357 (secure 0 0 0) 0x7f63a8031c10 con 0x7f639006af30 2023-09-23T03:29:02.963 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:29:02.964 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2023-09-23T03:29:02.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.965+0000 7f63b2f9e640 1 -- 172.21.15.154:0/2185403952 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f639006af30 msgr2=0x7f639006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:02.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.965+0000 7f63b2f9e640 1 --2- 172.21.15.154:0/2185403952 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f639006af30 0x7f639006d3f0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f63a8036000 tx=0x7f63a803d040 comp rx=0 tx=0).stop 2023-09-23T03:29:02.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.965+0000 7f63b2f9e640 1 -- 172.21.15.154:0/2185403952 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63ac0c8ed0 msgr2=0x7f63ac0c5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:02.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.965+0000 7f63b2f9e640 1 --2- 172.21.15.154:0/2185403952 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63ac0c8ed0 0x7f63ac0c5c90 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f639c00b4a0 tx=0x7f639c00b960 comp rx=0 tx=0).stop 2023-09-23T03:29:02.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.965+0000 7f63b2f9e640 1 -- 172.21.15.154:0/2185403952 shutdown_connections 2023-09-23T03:29:02.966 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.965+0000 7f63b2f9e640 1 --2- 172.21.15.154:0/2185403952 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f639006af30 0x7f639006d3f0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:02.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.965+0000 7f63b2f9e640 1 --2- 172.21.15.154:0/2185403952 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f63ac158200 0x7f63ac0ca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:02.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.965+0000 7f63b2f9e640 1 --2- 172.21.15.154:0/2185403952 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f63ac0c8ed0 0x7f63ac0c5c90 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:02.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.965+0000 7f63b2f9e640 1 --2- 172.21.15.154:0/2185403952 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f63ac06c820 0x7f63ac1577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:02.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.965+0000 7f63b2f9e640 1 -- 172.21.15.154:0/2185403952 >> 172.21.15.154:0/2185403952 conn(0x7f63ac06a4c0 msgr2=0x7f63ac15be10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:29:02.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.965+0000 7f63b2f9e640 1 -- 172.21.15.154:0/2185403952 shutdown_connections 2023-09-23T03:29:02.967 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:02.965+0000 7f63b2f9e640 1 -- 172.21.15.154:0/2185403952 wait complete. 2023-09-23T03:29:02.982 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":540,"stamp":"2023-09-23T03:29:01.456599+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":263428,"kb_used_data":13956,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711292,"statfs":{"total":773094113280,"available":772824363008,"internally_reserved":0,"allocated":14290944,"data_stored":7568061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":255315514},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-190,"num_objects":-100,"num_object_clones":0,"num_object_copies":-200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-100,"num_whiteouts":0,"num_read":-1100,"num_read_kb":-500,"num_write":-600,"num_write_kb":-100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002437"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":262,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-09-23T03:28:50.946536+0000","last_change":"2023-09-23T03:15:11.042447+0000","last_active":"2023-09-23T03:28:50.946536+0000","last_peered":"2023-09-23T03:28:50.946536+0000","last_clean":"2023-09-23T03:28:50.946536+0000","last_became_active":"2023-09-23T03:15:11.040322+0000","last_became_peered":"2023-09-23T03:15:11.040322+0000","last_unstale":"2023-09-23T03:28:50.946536+0000","last_undegraded":"2023-09-23T03:28:50.946536+0000","last_fullsized":"2023-09-23T03:28:50.946536+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-23T03:15:09.474842+0000","last_clean_scrub_stamp":"2023-09-23T03:15:09.474842+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T04:18:36.331645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575016,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33364,"kb_used_data":2180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338476,"statfs":{"total":96636764160,"available":96602599424,"internally_reserved":0,"allocated":2232320,"data_stored":1388911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0649999999999999,"15min":1.026},"min":{"1min":0.67600000000000005,"5min":0.58199999999999996,"15min":0.442},"max":{"1min":1.5880000000000001,"5min":2.0299999999999998,"15min":2.149},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0840000000000001,"15min":1.0640000000000001},"min":{"1min":0.77700000000000002,"5min":0.63100000000000001,"15min":0.52000000000000002},"max":{"1min":1.5009999999999999,"5min":1.9910000000000001,"15min":2.1480000000000001},"last":0.73499999999999999}]},{"osd":1,"last update":"Sat Sep 23 03:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.2390000000000001,"15min":1.218},"min":{"1min":0.80800000000000005,"5min":0.69599999999999995,"15min":0.504},"max":{"1min":1.663,"5min":2.6400000000000001,"15min":3.1859999999999999},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.264,"5min":1.2769999999999999,"15min":1.284},"min":{"1min":0.80600000000000005,"5min":0.71299999999999997,"15min":0.67900000000000005},"max":{"1min":1.6120000000000001,"5min":2.3149999999999999,"15min":2.613},"last":0.96499999999999997}]},{"osd":2,"last update":"Sat Sep 23 03:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.143,"15min":1.125},"min":{"1min":0.67500000000000004,"5min":0.61199999999999999,"15min":0.42399999999999999},"max":{"1min":1.619,"5min":1.6439999999999999,"15min":2.331},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.206,"5min":1.262,"15min":1.2330000000000001},"min":{"1min":0.89400000000000002,"5min":0.79600000000000004,"15min":0.55200000000000005},"max":{"1min":1.538,"5min":2.2719999999999998,"15min":3.093},"last":0.64400000000000002}]},{"osd":3,"last update":"Sat Sep 23 03:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.236,"15min":1.236},"min":{"1min":0.78700000000000003,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.7030000000000001,"5min":2.0840000000000001,"15min":2.532},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.343,"5min":1.381,"15min":1.361},"min":{"1min":0.99299999999999999,"5min":0.71899999999999997,"15min":0.69799999999999995},"max":{"1min":1.681,"5min":2.4289999999999998,"15min":3.2349999999999999},"last":1.0940000000000001}]},{"osd":4,"last update":"Sat Sep 23 03:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2869999999999999,"15min":1.349},"min":{"1min":0.85799999999999998,"5min":0.57999999999999996,"15min":0.45400000000000001},"max":{"1min":1.6259999999999999,"5min":2.3260000000000001,"15min":2.7210000000000001},"last":1.048},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.446,"15min":1.4630000000000001},"min":{"1min":0.93500000000000005,"5min":0.73599999999999999,"15min":0.48099999999999998},"max":{"1min":1.8200000000000001,"5min":2.2240000000000002,"15min":3.7719999999999998},"last":1.238}]},{"osd":5,"last update":"Sat Sep 23 03:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.4079999999999999,"15min":1.405},"min":{"1min":0.86699999999999999,"5min":0.86499999999999999,"15min":0.47999999999999998},"max":{"1min":1.6180000000000001,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.407,"15min":1.429},"min":{"1min":1.002,"5min":0.77000000000000002,"15min":0.497},"max":{"1min":1.748,"5min":2.5070000000000001,"15min":2.5070000000000001},"last":1.284}]},{"osd":6,"last update":"Sat Sep 23 03:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.498,"15min":1.528},"min":{"1min":1.0449999999999999,"5min":0.85499999999999998,"15min":0.53800000000000003},"max":{"1min":1.8580000000000001,"5min":2.649,"15min":2.8420000000000001},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.401,"5min":1.4279999999999999,"15min":1.4670000000000001},"min":{"1min":1.1319999999999999,"5min":0.78900000000000003,"15min":0.45900000000000002},"max":{"1min":1.726,"5min":2.4500000000000002,"15min":3.6739999999999999},"last":1.1339999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575016,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32784,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339056,"statfs":{"total":96636764160,"available":96603193344,"internally_reserved":0,"allocated":1638400,"data_stored":798543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:28:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.145,"15min":1.095},"min":{"1min":0.57199999999999995,"5min":0.53600000000000003,"15min":0.38},"max":{"1min":1.9830000000000001,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":0.75800000000000001},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.081,"15min":1.0700000000000001},"min":{"1min":0.68600000000000005,"5min":0.60999999999999999,"15min":0.46100000000000002},"max":{"1min":1.4510000000000001,"5min":3.7629999999999999,"15min":3.7629999999999999},"last":1.1479999999999999}]},{"osd":1,"last update":"Sat Sep 23 03:28:26 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.157,"15min":1.125},"min":{"1min":0.66000000000000003,"5min":0.63800000000000001,"15min":0.438},"max":{"1min":1.5589999999999999,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.2090000000000001,"15min":1.194},"min":{"1min":0.96099999999999997,"5min":0.56599999999999995,"15min":0.54400000000000004},"max":{"1min":1.776,"5min":2.972,"15min":2.972},"last":0.87}]},{"osd":2,"last update":"Sat Sep 23 03:28:26 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.1990000000000001,"15min":1.141},"min":{"1min":0.998,"5min":0.56499999999999995,"15min":0.51300000000000001},"max":{"1min":1.605,"5min":3.4390000000000001,"15min":3.4390000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2869999999999999,"15min":1.2370000000000001},"min":{"1min":0.94499999999999995,"5min":0.68000000000000005,"15min":0.64300000000000002},"max":{"1min":1.625,"5min":2.177,"15min":2.177},"last":1.3819999999999999}]},{"osd":3,"last update":"Sat Sep 23 03:28:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.381,"15min":1.339},"min":{"1min":0.96099999999999997,"5min":0.78500000000000003,"15min":0.54600000000000004},"max":{"1min":1.6890000000000001,"5min":2.129,"15min":2.6509999999999998},"last":1.54},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.339,"15min":1.365},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.74299999999999999},"max":{"1min":1.8899999999999999,"5min":3.0910000000000002,"15min":3.0910000000000002},"last":0.96999999999999997}]},{"osd":4,"last update":"Sat Sep 23 03:28:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.3360000000000001,"15min":1.365},"min":{"1min":0.91700000000000004,"5min":0.73299999999999998,"15min":0.498},"max":{"1min":2.0960000000000001,"5min":3.1789999999999998,"15min":3.1789999999999998},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3360000000000001,"15min":1.3009999999999999},"min":{"1min":0.92900000000000005,"5min":0.72399999999999998,"15min":0.56000000000000005},"max":{"1min":1.5760000000000001,"5min":3.6480000000000001,"15min":3.6480000000000001},"last":1.3069999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:28:26 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.4330000000000001,"15min":1.411},"min":{"1min":1.01,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.7150000000000001,"5min":4.093,"15min":4.093},"last":1.417},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.464,"15min":1.4419999999999999},"min":{"1min":1.0780000000000001,"5min":0.69899999999999995,"15min":0.59899999999999998},"max":{"1min":1.837,"5min":2.0459999999999998,"15min":4.758},"last":1.4830000000000001}]},{"osd":7,"last update":"Sat Sep 23 03:28:26 2023","interfaces":[{"interface":"back","average":{"1min":1.633,"5min":1.55,"15min":1.524},"min":{"1min":1.423,"5min":0.76500000000000001,"15min":0.63500000000000001},"max":{"1min":2.1859999999999999,"5min":3.536,"15min":3.536},"last":1.071},{"interface":"front","average":{"1min":1.532,"5min":1.5569999999999999,"15min":1.5329999999999999},"min":{"1min":1.081,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":2.0190000000000001,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":1.5880000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575018,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32784,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339056,"statfs":{"total":96636764160,"available":96603193344,"internally_reserved":0,"allocated":1638400,"data_stored":798543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:28:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.087,"15min":1.048},"min":{"1min":0.64100000000000001,"5min":0.61199999999999999,"15min":0.57199999999999995},"max":{"1min":1.639,"5min":2.9740000000000002,"15min":2.9740000000000002},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.1120000000000001,"15min":1.083},"min":{"1min":0.69599999999999995,"5min":0.624,"15min":0.61399999999999999},"max":{"1min":1.603,"5min":1.962,"15min":2.2320000000000002},"last":0.875}]},{"osd":1,"last update":"Sat Sep 23 03:28:27 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2050000000000001,"15min":1.218},"min":{"1min":0.73999999999999999,"5min":0.71899999999999997,"15min":0.62},"max":{"1min":1.657,"5min":2.4369999999999998,"15min":2.4369999999999998},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.268,"5min":1.294,"15min":1.2829999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67200000000000004},"max":{"1min":1.6120000000000001,"5min":3.5859999999999999,"15min":3.5859999999999999},"last":1.2090000000000001}]},{"osd":2,"last update":"Sat Sep 23 03:28:27 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.2050000000000001,"15min":1.2010000000000001},"min":{"1min":0.875,"5min":0.65300000000000002,"15min":0.64300000000000002},"max":{"1min":1.76,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.1399999999999999,"15min":1.2050000000000001},"min":{"1min":0.71699999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.4370000000000001,"5min":2.8500000000000001,"15min":2.8500000000000001},"last":1.266}]},{"osd":3,"last update":"Sat Sep 23 03:28:27 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2669999999999999,"15min":1.254},"min":{"1min":0.92800000000000005,"5min":0.67200000000000004,"15min":0.66400000000000003},"max":{"1min":1.6379999999999999,"5min":3.4359999999999999,"15min":3.4359999999999999},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.224,"5min":1.2969999999999999,"15min":1.339},"min":{"1min":0.91000000000000003,"5min":0.85999999999999999,"15min":0.69799999999999995},"max":{"1min":1.6719999999999999,"5min":1.9159999999999999,"15min":3.6110000000000002},"last":1.097}]},{"osd":4,"last update":"Sat Sep 23 03:28:27 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.393,"15min":1.3899999999999999},"min":{"1min":0.92000000000000004,"5min":0.68600000000000005,"15min":0.54100000000000004},"max":{"1min":1.843,"5min":4.2750000000000004,"15min":4.2750000000000004},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.3009999999999999,"15min":1.341},"min":{"1min":1.0429999999999999,"5min":0.54100000000000004,"15min":0.48899999999999999},"max":{"1min":1.4990000000000001,"5min":3.3090000000000002,"15min":3.3090000000000002},"last":1.6120000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:28:27 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.504,"15min":1.5049999999999999},"min":{"1min":1.1180000000000001,"5min":1.0329999999999999,"15min":0.59199999999999997},"max":{"1min":1.77,"5min":4.6040000000000001,"15min":4.6040000000000001},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.389,"5min":1.4159999999999999,"15min":1.5069999999999999},"min":{"1min":1.206,"5min":0.81200000000000006,"15min":0.55100000000000005},"max":{"1min":1.677,"5min":3.169,"15min":9.6560000000000006},"last":1.7230000000000001}]},{"osd":7,"last update":"Sat Sep 23 03:28:27 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.4159999999999999,"15min":1.429},"min":{"1min":1.091,"5min":0.77100000000000002,"15min":0.626},"max":{"1min":1.708,"5min":2.077,"15min":2.077},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.488,"5min":1.5009999999999999,"15min":1.5389999999999999},"min":{"1min":1.25,"5min":0.92200000000000004,"15min":0.60799999999999998},"max":{"1min":1.849,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":1.905}]}]},{"osd":4,"up_from":13,"seq":55834575017,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32784,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339056,"statfs":{"total":96636764160,"available":96603193344,"internally_reserved":0,"allocated":1638400,"data_stored":798543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:28:24 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.091,"15min":1.1000000000000001},"min":{"1min":0.63800000000000001,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.5740000000000001,"5min":1.6699999999999999,"15min":1.9770000000000001},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.079,"5min":1.1599999999999999,"15min":1.167},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.7350000000000001,"5min":2.004,"15min":2.004},"last":0.99299999999999999}]},{"osd":1,"last update":"Sat Sep 23 03:28:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.224,"15min":1.214},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.7490000000000001,"5min":1.8200000000000001,"15min":2.1110000000000002},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.022,"5min":1.0649999999999999,"15min":1.123},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60299999999999998},"max":{"1min":1.4890000000000001,"5min":1.718,"15min":3.2189999999999999},"last":0.94899999999999995}]},{"osd":2,"last update":"Sat Sep 23 03:28:24 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.1559999999999999,"15min":1.1599999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.512,"5min":1.694,"15min":2.5920000000000001},"last":1.46},{"interface":"front","average":{"1min":1.252,"5min":1.278,"15min":1.25},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68300000000000005},"max":{"1min":1.8839999999999999,"5min":1.99,"15min":1.99},"last":1.2929999999999999}]},{"osd":3,"last update":"Sat Sep 23 03:28:24 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.1699999999999999,"15min":1.1659999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4379999999999999,"5min":1.702,"15min":1.9450000000000001},"last":1.141},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2589999999999999,"15min":1.286},"min":{"1min":0.75900000000000001,"5min":0.69799999999999995,"15min":0.67400000000000004},"max":{"1min":1.611,"5min":1.833,"15min":2.4969999999999999},"last":1.0620000000000001}]},{"osd":5,"last update":"Sat Sep 23 03:28:24 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.3540000000000001,"15min":1.363},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.70199999999999996},"max":{"1min":1.879,"5min":2.0409999999999999,"15min":2.8050000000000002},"last":1.1000000000000001},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.2969999999999999,"15min":1.3240000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.619,"5min":1.8939999999999999,"15min":2.887},"last":1.3240000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:28:24 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3660000000000001,"15min":1.369},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.9530000000000001,"5min":1.9530000000000001,"15min":2.4249999999999998},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.478,"15min":1.4930000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.9350000000000001,"5min":2.0939999999999999,"15min":2.1779999999999999},"last":1.3939999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:28:24 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.419,"15min":1.45},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.734,"5min":2.1389999999999998,"15min":2.5680000000000001},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.464,"5min":1.534,"15min":1.544},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.4769999999999999},"last":1.3540000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575017,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32784,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339056,"statfs":{"total":96636764160,"available":96603193344,"internally_reserved":0,"allocated":1638400,"data_stored":798543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.81000000000000005,"15min":0.85099999999999998},"min":{"1min":0.52600000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.22,"5min":1.5009999999999999,"15min":2.952},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.91500000000000004,"15min":0.89500000000000002},"min":{"1min":0.41799999999999998,"5min":0.377,"15min":0.29299999999999998},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":0.60899999999999999}]},{"osd":1,"last update":"Sat Sep 23 03:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.81899999999999995,"15min":0.86699999999999999},"min":{"1min":0.40000000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.48,"5min":1.48,"15min":1.6839999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.98499999999999999,"15min":0.98399999999999999},"min":{"1min":0.48299999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.482,"5min":1.7010000000000001,"15min":2.903},"last":0.90800000000000003}]},{"osd":2,"last update":"Sat Sep 23 03:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.004,"15min":1.0409999999999999},"min":{"1min":0.54000000000000004,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.524,"5min":1.669,"15min":2.2240000000000002},"last":0.71099999999999997},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.119,"15min":1.1080000000000001},"min":{"1min":0.68799999999999994,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.8,"5min":1.8,"15min":3.129},"last":0.81000000000000005}]},{"osd":4,"last update":"Sat Sep 23 03:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1439999999999999,"15min":1.2070000000000001},"min":{"1min":0.88200000000000001,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.7010000000000001,"5min":1.7609999999999999,"15min":2.698},"last":0.78700000000000003},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.264,"15min":1.327},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.752},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":2.419},"last":0.97999999999999998}]},{"osd":5,"last update":"Sat Sep 23 03:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.282,"15min":1.3240000000000001},"min":{"1min":0.82499999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.9950000000000001},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.212,"5min":1.2090000000000001,"15min":1.266},"min":{"1min":0.80200000000000005,"5min":0.624,"15min":0.624},"max":{"1min":1.6799999999999999,"5min":1.9830000000000001,"15min":2.0070000000000001},"last":1.1970000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.284,"15min":1.337},"min":{"1min":0.77900000000000003,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.827,"5min":1.9530000000000001,"15min":2.6440000000000001},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.363,"5min":1.3720000000000001,"15min":1.419},"min":{"1min":0.86399999999999999,"5min":0.73299999999999998,"15min":0.67800000000000005},"max":{"1min":2.0880000000000001,"5min":2.177,"15min":2.9750000000000001},"last":1.53}]},{"osd":7,"last update":"Sat Sep 23 03:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.286,"15min":1.357},"min":{"1min":0.81299999999999994,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.6279999999999999,"5min":2.0859999999999999,"15min":2.7879999999999998},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3520000000000001,"15min":1.4079999999999999},"min":{"1min":0.90300000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.7290000000000001,"5min":2.0270000000000001,"15min":2.5379999999999998},"last":1.357}]}]},{"osd":2,"up_from":13,"seq":55834575018,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32784,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339056,"statfs":{"total":96636764160,"available":96603193344,"internally_reserved":0,"allocated":1638400,"data_stored":798543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.81299999999999994,"15min":0.82899999999999996},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.30299999999999999},"max":{"1min":0.92400000000000004,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.91200000000000003,"15min":0.88200000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.375,"5min":2.613,"15min":2.613},"last":0.72099999999999997}]},{"osd":1,"last update":"Sat Sep 23 03:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.85399999999999998,"15min":0.85499999999999998},"min":{"1min":0.46600000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.391,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.006,"5min":1.032,"15min":0.98299999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.38},"max":{"1min":1.526,"5min":2.9809999999999999,"15min":2.9809999999999999},"last":0.85199999999999998}]},{"osd":3,"last update":"Sat Sep 23 03:28:28 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.0740000000000001,"15min":1.006},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.40699999999999997},"max":{"1min":1.6279999999999999,"5min":1.698,"15min":1.698},"last":1.034},{"interface":"front","average":{"1min":0.96999999999999997,"5min":1.044,"15min":1.01},"min":{"1min":0.46800000000000003,"5min":0.441,"15min":0.34999999999999998},"max":{"1min":1.482,"5min":1.827,"15min":1.827},"last":1.2729999999999999}]},{"osd":4,"last update":"Sat Sep 23 03:28:28 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.3240000000000001,"15min":1.286},"min":{"1min":0.82899999999999996,"5min":0.79400000000000004,"15min":0.55800000000000005},"max":{"1min":1.5309999999999999,"5min":5.6669999999999998,"15min":5.6669999999999998},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.28,"5min":1.302,"15min":1.2949999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.7190000000000001,"5min":2.6909999999999998,"15min":2.6909999999999998},"last":1.236}]},{"osd":5,"last update":"Sat Sep 23 03:28:28 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.3260000000000001,"15min":1.335},"min":{"1min":0.83399999999999996,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.6879999999999999,"5min":2.8969999999999998,"15min":3.9409999999999998},"last":1.012},{"interface":"front","average":{"1min":1.234,"5min":1.379,"15min":1.3959999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.56399999999999995},"max":{"1min":1.8109999999999999,"5min":3.415,"15min":4.9199999999999999},"last":1.3009999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:28:28 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.357,"15min":1.3580000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.755},"max":{"1min":1.7090000000000001,"5min":2.7839999999999998,"15min":3.0009999999999999},"last":1.331},{"interface":"front","average":{"1min":1.321,"5min":1.4570000000000001,"15min":1.4219999999999999},"min":{"1min":0.91800000000000004,"5min":0.76600000000000001,"15min":0.65100000000000002},"max":{"1min":1.851,"5min":7.8300000000000001,"15min":7.8300000000000001},"last":1.1519999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:28:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.423,"15min":1.395},"min":{"1min":0.83099999999999996,"5min":0.82899999999999996,"15min":0.61399999999999999},"max":{"1min":1.954,"5min":4.0330000000000004,"15min":4.0330000000000004},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.4199999999999999,"15min":1.4239999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.74399999999999999},"max":{"1min":2.3140000000000001,"5min":4.2199999999999998,"15min":4.2199999999999998},"last":1.123}]}]},{"osd":1,"up_from":13,"seq":55834575019,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32784,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339056,"statfs":{"total":96636764160,"available":96603193344,"internally_reserved":0,"allocated":1638400,"data_stored":798543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:28:40 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.81899999999999995,"15min":0.80200000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.32300000000000001},"max":{"1min":1.8660000000000001,"5min":1.8660000000000001,"15min":2.8279999999999998},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88300000000000001,"15min":0.84899999999999998},"min":{"1min":0.40899999999999997,"5min":0.40600000000000003,"15min":0.312},"max":{"1min":2.2490000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":0.64700000000000002}]},{"osd":2,"last update":"Sat Sep 23 03:28:40 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.86499999999999999,"15min":0.86899999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.36699999999999999},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.3340000000000001},"last":1.024},{"interface":"front","average":{"1min":1.069,"5min":1.0580000000000001,"15min":1.0329999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.909,"5min":1.909,"15min":1.9570000000000001},"last":1.411}]},{"osd":3,"last update":"Sat Sep 23 03:28:40 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.046,"15min":1.0329999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.47299999999999998},"max":{"1min":1.992,"5min":1.992,"15min":2.2370000000000001},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.141,"15min":1.109},"min":{"1min":0.495,"5min":0.495,"15min":0.45900000000000002},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":3.2130000000000001},"last":1.1120000000000001}]},{"osd":4,"last update":"Sat Sep 23 03:28:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.2789999999999999,"15min":1.2490000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.798,"5min":1.885,"15min":3.2629999999999999},"last":1.607},{"interface":"front","average":{"1min":1.266,"5min":1.244,"15min":1.2529999999999999},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66500000000000004},"max":{"1min":2.2690000000000001,"5min":2.2690000000000001,"15min":2.4689999999999999},"last":1.361}]},{"osd":5,"last update":"Sat Sep 23 03:28:40 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.2929999999999999,"15min":1.2789999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.64600000000000002},"max":{"1min":2.024,"5min":2.024,"15min":3.161},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3280000000000001,"15min":1.3260000000000001},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.72399999999999998},"max":{"1min":3.0830000000000002,"5min":3.0830000000000002,"15min":3.0830000000000002},"last":1.5620000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:28:40 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3640000000000001,"15min":1.329},"min":{"1min":0.81299999999999994,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":3.0179999999999998},"last":1.516},{"interface":"front","average":{"1min":1.296,"5min":1.363,"15min":1.363},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":2.1309999999999998,"5min":2.1309999999999998,"15min":3.3580000000000001},"last":1.45}]},{"osd":7,"last update":"Sat Sep 23 03:28:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.337,"15min":1.3600000000000001},"min":{"1min":0.77600000000000002,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.946,"5min":1.946,"15min":3.1099999999999999},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4039999999999999,"15min":1.419},"min":{"1min":0.86699999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":3.1179999999999999,"5min":3.1179999999999999,"15min":3.306},"last":1.4850000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575019,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33360,"kb_used_data":2176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338480,"statfs":{"total":96636764160,"available":96602603520,"internally_reserved":0,"allocated":2228224,"data_stored":1387892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 23 03:28:30 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80900000000000005,"15min":0.83499999999999996},"min":{"1min":0.44700000000000001,"5min":0.38,"15min":0.30299999999999999},"max":{"1min":1.3799999999999999,"5min":1.383,"15min":1.919},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.88900000000000001,"15min":0.88800000000000001},"min":{"1min":0.36699999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.351,"5min":1.4470000000000001,"15min":2.1360000000000001},"last":1.163}]},{"osd":2,"last update":"Sat Sep 23 03:28:30 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.873,"15min":0.85399999999999998},"min":{"1min":0.46899999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.335,"5min":1.5249999999999999,"15min":3.089},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.97299999999999998,"15min":0.999},"min":{"1min":0.47699999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.4970000000000001,"5min":1.5760000000000001,"15min":2.633},"last":1.3049999999999999}]},{"osd":3,"last update":"Sat Sep 23 03:28:30 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.98799999999999999,"15min":0.98099999999999998},"min":{"1min":0.5,"5min":0.433,"15min":0.433},"max":{"1min":1.417,"5min":1.615,"15min":2.8660000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.03,"15min":1.018},"min":{"1min":0.505,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.446,"5min":1.802,"15min":2.012},"last":0.78200000000000003}]},{"osd":4,"last update":"Sat Sep 23 03:28:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.252,"15min":1.2609999999999999},"min":{"1min":0.73599999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.621,"5min":1.885,"15min":2.4140000000000001},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2270000000000001,"15min":1.2609999999999999},"min":{"1min":0.83199999999999996,"5min":0.73999999999999999,"15min":0.65300000000000002},"max":{"1min":1.5309999999999999,"5min":1.845,"15min":3.0230000000000001},"last":1.036}]},{"osd":5,"last update":"Sat Sep 23 03:28:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.296,"15min":1.274},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.65600000000000003},"max":{"1min":1.9530000000000001,"5min":2.1240000000000001,"15min":2.5489999999999999},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3320000000000001,"15min":1.3380000000000001},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.7210000000000001,"5min":1.9159999999999999,"15min":2.1539999999999999},"last":1.5289999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:28:30 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3149999999999999,"15min":1.325},"min":{"1min":0.78900000000000003,"5min":0.77700000000000002,"15min":0.75600000000000001},"max":{"1min":1.8600000000000001,"5min":2.0870000000000002,"15min":2.9489999999999998},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.3839999999999999,"15min":1.3859999999999999},"min":{"1min":0.71199999999999997,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.8100000000000001,"5min":1.9670000000000001,"15min":2.3239999999999998},"last":1.6220000000000001}]},{"osd":7,"last update":"Sat Sep 23 03:28:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.3420000000000001,"15min":1.3640000000000001},"min":{"1min":0.93200000000000005,"5min":0.752,"15min":0.747},"max":{"1min":1.722,"5min":2.052,"15min":2.2330000000000001},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.405,"15min":1.4359999999999999},"min":{"1min":1.0960000000000001,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.7889999999999999,"5min":2.1419999999999999,"15min":2.3330000000000002},"last":1.224}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-23T03:29:02.986 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-09-23T03:15:09.474842+0000 time.struct_time(tm_year=2023, tm_mon=9, tm_mday=23, tm_hour=3, tm_min=15, tm_sec=9, tm_wday=5, tm_yday=266, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=9, tm_mday=23, tm_hour=3, tm_min=28, tm_sec=55, tm_wday=5, tm_yday=266, tm_isdst=0) 2023-09-23T03:29:02.986 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-09-23T03:29:14.977 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:29:14.987 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:29:22.988 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-23T03:29:23.213 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.212+0000 7f47643fb640 1 Processor -- start 2023-09-23T03:29:23.214 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.212+0000 7f47643fb640 1 -- start start 2023-09-23T03:29:23.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.213+0000 7f47643fb640 1 --2- >> v2:172.21.15.190:3300/0 conn(0x7f475c15ad70 0x7f475c0c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:23.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.214+0000 7f47643fb640 1 -- --> v1:172.21.15.154:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f475c070b40 con 0x7f475c0b0780 2023-09-23T03:29:23.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.214+0000 7f47643fb640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f475c070e20 con 0x7f475c0ad9c0 2023-09-23T03:29:23.215 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.214+0000 7f47643fb640 1 -- --> v2:172.21.15.190:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f475c071100 con 0x7f475c15ad70 2023-09-23T03:29:23.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.214+0000 7f4762998640 1 --1- >> v1:172.21.15.154:6789/0 conn(0x7f475c0b0780 0x7f475c06c8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.154:6789/0 says I am v1:172.21.15.154:50918/0 (socket says 172.21.15.154:50918) 2023-09-23T03:29:23.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.214+0000 7f4762998640 1 -- 172.21.15.154:0/1961836267 learned_addr learned my addr 172.21.15.154:0/1961836267 (peer_addr_for_me v1:172.21.15.154:0/0) 2023-09-23T03:29:23.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.214+0000 7f4762197640 1 --2- 172.21.15.154:0/1961836267 >> v2:172.21.15.190:3300/0 conn(0x7f475c15ad70 0x7f475c0c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:23.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.215+0000 7f4761195640 1 -- 172.21.15.154:0/1961836267 <== mon.0 v1:172.21.15.154:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2842771503 0 0) 0x7f475c070b40 con 0x7f475c0b0780 2023-09-23T03:29:23.218 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.215+0000 7f4761195640 1 -- 172.21.15.154:0/1961836267 --> v1:172.21.15.154:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4748003520 con 0x7f475c0b0780 2023-09-23T03:29:23.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.215+0000 7f4761195640 1 -- 172.21.15.154:0/1961836267 <== mon.0 v1:172.21.15.154:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 525119581 0 0) 0x7f4748003520 con 0x7f475c0b0780 2023-09-23T03:29:23.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.215+0000 7f4761195640 1 -- 172.21.15.154:0/1961836267 >> v2:172.21.15.190:3300/0 conn(0x7f475c15ad70 msgr2=0x7f475c0c8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:29:23.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.215+0000 7f4761195640 1 --2- 172.21.15.154:0/1961836267 >> v2:172.21.15.190:3300/0 conn(0x7f475c15ad70 0x7f475c0c8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:23.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.215+0000 7f4761195640 1 -- 172.21.15.154:0/1961836267 >> v1:172.21.15.190:6789/0 conn(0x7f475c0ad9c0 legacy=0x7f475c0b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:29:23.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.215+0000 7f4761195640 1 -- 172.21.15.154:0/1961836267 --> v1:172.21.15.154:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f475c0c96d0 con 0x7f475c0b0780 2023-09-23T03:29:23.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.215+0000 7f4761195640 1 -- 172.21.15.154:0/1961836267 <== mon.0 v1:172.21.15.154:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1847274302 0 0) 0x7f47540019f0 con 0x7f475c0b0780 2023-09-23T03:29:23.219 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.215+0000 7f4761195640 1 -- 172.21.15.154:0/1961836267 >> v1:172.21.15.154:6789/0 conn(0x7f475c0b0780 legacy=0x7f475c06c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:23.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.215+0000 7f4761195640 1 --2- 172.21.15.154:0/1961836267 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4748003970 0x7f4748003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:23.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.215+0000 7f4761195640 1 --2- 172.21.15.154:0/1961836267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47480047c0 0x7f4748004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:23.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.215+0000 7f4761195640 1 --2- 172.21.15.154:0/1961836267 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4748005180 0x7f474800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:23.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.215+0000 7f4761195640 1 -- 172.21.15.154:0/1961836267 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475c071100 con 0x7f4748005180 2023-09-23T03:29:23.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.215+0000 7f4761195640 1 -- 172.21.15.154:0/1961836267 --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f474800c000 con 0x7f47480047c0 2023-09-23T03:29:23.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.215+0000 7f4761195640 1 -- 172.21.15.154:0/1961836267 --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f474800c1c0 con 0x7f4748003970 2023-09-23T03:29:23.220 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.215+0000 7f4762998640 1 --2- 172.21.15.154:0/1961836267 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4748005180 0x7f474800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:23.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.216+0000 7f4761996640 1 --2- 172.21.15.154:0/1961836267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47480047c0 0x7f4748004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:23.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.216+0000 7f4762197640 1 --2- 172.21.15.154:0/1961836267 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4748003970 0x7f4748003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:23.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.216+0000 7f4762998640 1 -- 172.21.15.154:0/1961836267 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4748003970 msgr2=0x7f4748003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:29:23.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.216+0000 7f4762998640 1 --2- 172.21.15.154:0/1961836267 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4748003970 0x7f4748003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:23.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.216+0000 7f4762998640 1 -- 172.21.15.154:0/1961836267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47480047c0 msgr2=0x7f4748004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:29:23.221 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.216+0000 7f4762998640 1 --2- 172.21.15.154:0/1961836267 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f47480047c0 0x7f4748004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:23.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.216+0000 7f4762998640 1 -- 172.21.15.154:0/1961836267 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4748003520 con 0x7f4748005180 2023-09-23T03:29:23.222 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.216+0000 7f4762197640 1 --2- 172.21.15.154:0/1961836267 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f4748003970 0x7f4748003d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:29:23.225 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.217+0000 7f4762998640 1 -- 172.21.15.154:0/1961836267 reap_dead start 2023-09-23T03:29:23.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.217+0000 7f4762998640 1 --2- 172.21.15.154:0/1961836267 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4748005180 0x7f474800b980 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f475c0700e0 tx=0x7f47480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c102488ff5cdbe72 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:29:23.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.218+0000 7f4761195640 1 -- 172.21.15.154:0/1961836267 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4754034630 con 0x7f4748005180 2023-09-23T03:29:23.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.218+0000 7f4761195640 1 -- 172.21.15.154:0/1961836267 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f475402f070 con 0x7f4748005180 2023-09-23T03:29:23.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.218+0000 7f4761195640 1 -- 172.21.15.154:0/1961836267 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47540349d0 con 0x7f4748005180 2023-09-23T03:29:23.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.218+0000 7f47643fb640 1 -- 172.21.15.154:0/1961836267 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4748005180 msgr2=0x7f474800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:23.226 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.218+0000 7f47643fb640 1 --2- 172.21.15.154:0/1961836267 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4748005180 0x7f474800b980 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f475c0700e0 tx=0x7f47480057f0 comp rx=0 tx=0).stop 2023-09-23T03:29:23.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.218+0000 7f47643fb640 1 -- 172.21.15.154:0/1961836267 shutdown_connections 2023-09-23T03:29:23.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.218+0000 7f47643fb640 1 --2- 172.21.15.154:0/1961836267 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4748005180 0x7f474800b980 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:23.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.218+0000 7f47643fb640 1 -- 172.21.15.154:0/1961836267 >> 172.21.15.154:0/1961836267 conn(0x7f475c0cb070 msgr2=0x7f475c0cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:29:23.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.219+0000 7f47643fb640 1 -- 172.21.15.154:0/1961836267 shutdown_connections 2023-09-23T03:29:23.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.219+0000 7f47643fb640 1 -- 172.21.15.154:0/1961836267 wait complete. 2023-09-23T03:29:23.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.220+0000 7f47643fb640 1 Processor -- start 2023-09-23T03:29:23.227 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.221+0000 7f47643fb640 1 -- start start 2023-09-23T03:29:23.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.221+0000 7f47643fb640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4748005180 0x7f475c155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:23.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.222+0000 7f47643fb640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f475c1e4f40 0x7f475c1e3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:23.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.222+0000 7f47643fb640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f475c1594f0 0x7f475c159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:23.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.222+0000 7f47643fb640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475c159ec0 con 0x7f475c1594f0 2023-09-23T03:29:23.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.222+0000 7f47643fb640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475c0c7ba0 con 0x7f4748005180 2023-09-23T03:29:23.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.222+0000 7f47643fb640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f475c0c6f90 con 0x7f475c1e4f40 2023-09-23T03:29:23.228 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.222+0000 7f4762998640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f475c1594f0 0x7f475c159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:23.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.222+0000 7f4762998640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f475c1594f0 0x7f475c159980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43772/0 (socket says 172.21.15.154:43772) 2023-09-23T03:29:23.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.222+0000 7f4762998640 1 -- 172.21.15.154:0/759393749 learned_addr learned my addr 172.21.15.154:0/759393749 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:29:23.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.222+0000 7f4762197640 1 --2- 172.21.15.154:0/759393749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4748005180 0x7f475c155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:23.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.222+0000 7f4761996640 1 --2- 172.21.15.154:0/759393749 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f475c1e4f40 0x7f475c1e3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:23.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.222+0000 7f4762998640 1 -- 172.21.15.154:0/759393749 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f475c1e4f40 msgr2=0x7f475c1e3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:23.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.222+0000 7f4762998640 1 --2- 172.21.15.154:0/759393749 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f475c1e4f40 0x7f475c1e3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:23.229 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.222+0000 7f4762998640 1 -- 172.21.15.154:0/759393749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4748005180 msgr2=0x7f475c155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:23.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.222+0000 7f4762998640 1 --2- 172.21.15.154:0/759393749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4748005180 0x7f475c155d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:23.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.222+0000 7f4762998640 1 -- 172.21.15.154:0/759393749 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f475c0c7270 con 0x7f475c1594f0 2023-09-23T03:29:23.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.223+0000 7f4762998640 1 --2- 172.21.15.154:0/759393749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f475c1594f0 0x7f475c159980 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f475c0b4760 tx=0x7f4754001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:29:23.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.223+0000 7f4752ffd640 1 -- 172.21.15.154:0/759393749 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4754001930 con 0x7f475c1594f0 2023-09-23T03:29:23.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.223+0000 7f4752ffd640 1 -- 172.21.15.154:0/759393749 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4754001a90 con 0x7f475c1594f0 2023-09-23T03:29:23.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.223+0000 7f4752ffd640 1 -- 172.21.15.154:0/759393749 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4754034650 con 0x7f475c1594f0 2023-09-23T03:29:23.230 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.223+0000 7f47643fb640 1 -- 172.21.15.154:0/759393749 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f475c0c7550 con 0x7f475c1594f0 2023-09-23T03:29:23.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.223+0000 7f47643fb640 1 -- 172.21.15.154:0/759393749 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f475c1e44b0 con 0x7f475c1594f0 2023-09-23T03:29:23.231 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.224+0000 7f47643fb640 1 -- 172.21.15.154:0/759393749 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f475c067870 con 0x7f475c1594f0 2023-09-23T03:29:23.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.225+0000 7f4752ffd640 1 -- 172.21.15.154:0/759393749 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7f4754031030 con 0x7f475c1594f0 2023-09-23T03:29:23.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.226+0000 7f4752ffd640 1 --2- 172.21.15.154:0/759393749 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f474006af30 0x7f474006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:23.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.226+0000 7f4752ffd640 1 -- 172.21.15.154:0/759393749 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5435+0+0 (secure 0 0 0) 0x7f47540ab3c0 con 0x7f475c1594f0 2023-09-23T03:29:23.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.226+0000 7f4762197640 1 --2- 172.21.15.154:0/759393749 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f474006af30 0x7f474006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:23.232 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.227+0000 7f4762197640 1 --2- 172.21.15.154:0/759393749 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f474006af30 0x7f474006d3f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f4758009aa0 tx=0x7f4758008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:29:23.233 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.231+0000 7f4752ffd640 1 -- 172.21.15.154:0/759393749 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7f4754076590 con 0x7f475c1594f0 2023-09-23T03:29:23.418 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.417+0000 7f47643fb640 1 -- 172.21.15.154:0/759393749 --> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f475c15b150 con 0x7f474006af30 2023-09-23T03:29:23.422 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.421+0000 7f4752ffd640 1 -- 172.21.15.154:0/759393749 <== mgr.4110 v2:172.21.15.154:6832/2422652104 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48157 (secure 0 0 0) 0x7f47580129b0 con 0x7f474006af30 2023-09-23T03:29:23.423 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:29:23.423 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2023-09-23T03:29:23.425 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.424+0000 7f47643fb640 1 -- 172.21.15.154:0/759393749 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f474006af30 msgr2=0x7f474006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:23.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.424+0000 7f47643fb640 1 --2- 172.21.15.154:0/759393749 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f474006af30 0x7f474006d3f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f4758009aa0 tx=0x7f4758008040 comp rx=0 tx=0).stop 2023-09-23T03:29:23.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.424+0000 7f47643fb640 1 -- 172.21.15.154:0/759393749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f475c1594f0 msgr2=0x7f475c159980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:23.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.424+0000 7f47643fb640 1 --2- 172.21.15.154:0/759393749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f475c1594f0 0x7f475c159980 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f475c0b4760 tx=0x7f4754001d80 comp rx=0 tx=0).stop 2023-09-23T03:29:23.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.424+0000 7f47643fb640 1 -- 172.21.15.154:0/759393749 shutdown_connections 2023-09-23T03:29:23.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.424+0000 7f47643fb640 1 --2- 172.21.15.154:0/759393749 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7f474006af30 0x7f474006d3f0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:23.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.424+0000 7f47643fb640 1 --2- 172.21.15.154:0/759393749 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f475c1594f0 0x7f475c159980 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:23.426 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.424+0000 7f47643fb640 1 --2- 172.21.15.154:0/759393749 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7f475c1e4f40 0x7f475c1e3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:23.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.424+0000 7f47643fb640 1 --2- 172.21.15.154:0/759393749 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7f4748005180 0x7f475c155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:23.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.424+0000 7f47643fb640 1 -- 172.21.15.154:0/759393749 >> 172.21.15.154:0/759393749 conn(0x7f475c0cb070 msgr2=0x7f475c158ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:29:23.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.424+0000 7f47643fb640 1 -- 172.21.15.154:0/759393749 shutdown_connections 2023-09-23T03:29:23.427 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.424+0000 7f47643fb640 1 -- 172.21.15.154:0/759393749 wait complete. 2023-09-23T03:29:23.445 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":550,"stamp":"2023-09-23T03:29:21.461962+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":263412,"kb_used_data":13956,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711308,"statfs":{"total":773094113280,"available":772824379392,"internally_reserved":0,"allocated":14290944,"data_stored":7568061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12742,"internal_metadata":255315514},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003261"},"pg_stats":[{"pgid":"1.0","version":"131'119","reported_seq":274,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-09-23T03:29:02.630411+0000","last_change":"2023-09-23T03:29:02.630411+0000","last_active":"2023-09-23T03:29:02.630411+0000","last_peered":"2023-09-23T03:29:02.630411+0000","last_clean":"2023-09-23T03:29:02.630411+0000","last_became_active":"2023-09-23T03:15:11.040322+0000","last_became_peered":"2023-09-23T03:15:11.040322+0000","last_unstale":"2023-09-23T03:29:02.630411+0000","last_undegraded":"2023-09-23T03:29:02.630411+0000","last_fullsized":"2023-09-23T03:29:02.630411+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"131'119","last_scrub_stamp":"2023-09-23T03:29:02.630145+0000","last_deep_scrub":"131'119","last_deep_scrub_stamp":"2023-09-23T03:29:02.630145+0000","last_clean_scrub_stamp":"2023-09-23T03:29:02.630145+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-09-24T11:55:05.321595+0000","scrub_duration":0.0095918410000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575020,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33356,"kb_used_data":2180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338484,"statfs":{"total":96636764160,"available":96602607616,"internally_reserved":0,"allocated":2232320,"data_stored":1388911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0649999999999999,"15min":1.026},"min":{"1min":0.67600000000000005,"5min":0.58199999999999996,"15min":0.442},"max":{"1min":1.5880000000000001,"5min":2.0299999999999998,"15min":2.149},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0840000000000001,"15min":1.0640000000000001},"min":{"1min":0.77700000000000002,"5min":0.63100000000000001,"15min":0.52000000000000002},"max":{"1min":1.5009999999999999,"5min":1.9910000000000001,"15min":2.1480000000000001},"last":0.98999999999999999}]},{"osd":1,"last update":"Sat Sep 23 03:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.2390000000000001,"15min":1.218},"min":{"1min":0.80800000000000005,"5min":0.69599999999999995,"15min":0.504},"max":{"1min":1.663,"5min":2.6400000000000001,"15min":3.1859999999999999},"last":1.034},{"interface":"front","average":{"1min":1.264,"5min":1.2769999999999999,"15min":1.284},"min":{"1min":0.80600000000000005,"5min":0.71299999999999997,"15min":0.67900000000000005},"max":{"1min":1.6120000000000001,"5min":2.3149999999999999,"15min":2.613},"last":1.3260000000000001}]},{"osd":2,"last update":"Sat Sep 23 03:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.143,"15min":1.125},"min":{"1min":0.67500000000000004,"5min":0.61199999999999999,"15min":0.42399999999999999},"max":{"1min":1.619,"5min":1.6439999999999999,"15min":2.331},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.206,"5min":1.262,"15min":1.2330000000000001},"min":{"1min":0.89400000000000002,"5min":0.79600000000000004,"15min":0.55200000000000005},"max":{"1min":1.538,"5min":2.2719999999999998,"15min":3.093},"last":0.92300000000000004}]},{"osd":3,"last update":"Sat Sep 23 03:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.236,"15min":1.236},"min":{"1min":0.78700000000000003,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.7030000000000001,"5min":2.0840000000000001,"15min":2.532},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.343,"5min":1.381,"15min":1.361},"min":{"1min":0.99299999999999999,"5min":0.71899999999999997,"15min":0.69799999999999995},"max":{"1min":1.681,"5min":2.4289999999999998,"15min":3.2349999999999999},"last":1.2729999999999999}]},{"osd":4,"last update":"Sat Sep 23 03:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2869999999999999,"15min":1.349},"min":{"1min":0.85799999999999998,"5min":0.57999999999999996,"15min":0.45400000000000001},"max":{"1min":1.6259999999999999,"5min":2.3260000000000001,"15min":2.7210000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.446,"15min":1.4630000000000001},"min":{"1min":0.93500000000000005,"5min":0.73599999999999999,"15min":0.48099999999999998},"max":{"1min":1.8200000000000001,"5min":2.2240000000000002,"15min":3.7719999999999998},"last":1.085}]},{"osd":5,"last update":"Sat Sep 23 03:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.4079999999999999,"15min":1.405},"min":{"1min":0.86699999999999999,"5min":0.86499999999999999,"15min":0.47999999999999998},"max":{"1min":1.6180000000000001,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.754},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.407,"15min":1.429},"min":{"1min":1.002,"5min":0.77000000000000002,"15min":0.497},"max":{"1min":1.748,"5min":2.5070000000000001,"15min":2.5070000000000001},"last":1.1870000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:28:35 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.498,"15min":1.528},"min":{"1min":1.0449999999999999,"5min":0.85499999999999998,"15min":0.53800000000000003},"max":{"1min":1.8580000000000001,"5min":2.649,"15min":2.8420000000000001},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.401,"5min":1.4279999999999999,"15min":1.4670000000000001},"min":{"1min":1.1319999999999999,"5min":0.78900000000000003,"15min":0.45900000000000002},"max":{"1min":1.726,"5min":2.4500000000000002,"15min":3.6739999999999999},"last":1.2969999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575020,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32784,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339056,"statfs":{"total":96636764160,"available":96603193344,"internally_reserved":0,"allocated":1638400,"data_stored":798543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:28:26 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.145,"15min":1.095},"min":{"1min":0.57199999999999995,"5min":0.53600000000000003,"15min":0.38},"max":{"1min":1.9830000000000001,"5min":2.8050000000000002,"15min":2.8050000000000002},"last":1.48},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.081,"15min":1.0700000000000001},"min":{"1min":0.68600000000000005,"5min":0.60999999999999999,"15min":0.46100000000000002},"max":{"1min":1.4510000000000001,"5min":3.7629999999999999,"15min":3.7629999999999999},"last":1.4490000000000001}]},{"osd":1,"last update":"Sat Sep 23 03:28:26 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.157,"15min":1.125},"min":{"1min":0.66000000000000003,"5min":0.63800000000000001,"15min":0.438},"max":{"1min":1.5589999999999999,"5min":2.2400000000000002,"15min":2.2400000000000002},"last":1.149},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.2090000000000001,"15min":1.194},"min":{"1min":0.96099999999999997,"5min":0.56599999999999995,"15min":0.54400000000000004},"max":{"1min":1.776,"5min":2.972,"15min":2.972},"last":1.3180000000000001}]},{"osd":2,"last update":"Sat Sep 23 03:28:26 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.1990000000000001,"15min":1.141},"min":{"1min":0.998,"5min":0.56499999999999995,"15min":0.51300000000000001},"max":{"1min":1.605,"5min":3.4390000000000001,"15min":3.4390000000000001},"last":1.272},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2869999999999999,"15min":1.2370000000000001},"min":{"1min":0.94499999999999995,"5min":0.68000000000000005,"15min":0.64300000000000002},"max":{"1min":1.625,"5min":2.177,"15min":2.177},"last":1.1180000000000001}]},{"osd":3,"last update":"Sat Sep 23 03:28:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.381,"15min":1.339},"min":{"1min":0.96099999999999997,"5min":0.78500000000000003,"15min":0.54600000000000004},"max":{"1min":1.6890000000000001,"5min":2.129,"15min":2.6509999999999998},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.339,"15min":1.365},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.74299999999999999},"max":{"1min":1.8899999999999999,"5min":3.0910000000000002,"15min":3.0910000000000002},"last":1.4199999999999999}]},{"osd":4,"last update":"Sat Sep 23 03:28:26 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.3360000000000001,"15min":1.365},"min":{"1min":0.91700000000000004,"5min":0.73299999999999998,"15min":0.498},"max":{"1min":2.0960000000000001,"5min":3.1789999999999998,"15min":3.1789999999999998},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3360000000000001,"15min":1.3009999999999999},"min":{"1min":0.92900000000000005,"5min":0.72399999999999998,"15min":0.56000000000000005},"max":{"1min":1.5760000000000001,"5min":3.6480000000000001,"15min":3.6480000000000001},"last":1.504}]},{"osd":5,"last update":"Sat Sep 23 03:28:26 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.4330000000000001,"15min":1.411},"min":{"1min":1.01,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.7150000000000001,"5min":4.093,"15min":4.093},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.464,"15min":1.4419999999999999},"min":{"1min":1.0780000000000001,"5min":0.69899999999999995,"15min":0.59899999999999998},"max":{"1min":1.837,"5min":2.0459999999999998,"15min":4.758},"last":1.0720000000000001}]},{"osd":7,"last update":"Sat Sep 23 03:28:26 2023","interfaces":[{"interface":"back","average":{"1min":1.633,"5min":1.55,"15min":1.524},"min":{"1min":1.423,"5min":0.76500000000000001,"15min":0.63500000000000001},"max":{"1min":2.1859999999999999,"5min":3.536,"15min":3.536},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.532,"5min":1.5569999999999999,"15min":1.5329999999999999},"min":{"1min":1.081,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":2.0190000000000001,"5min":2.5169999999999999,"15min":2.5169999999999999},"last":1.347}]}]},{"osd":5,"up_from":13,"seq":55834575022,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32784,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339056,"statfs":{"total":96636764160,"available":96603193344,"internally_reserved":0,"allocated":1638400,"data_stored":798543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:28:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.087,"15min":1.048},"min":{"1min":0.64100000000000001,"5min":0.61199999999999999,"15min":0.57199999999999995},"max":{"1min":1.639,"5min":2.9740000000000002,"15min":2.9740000000000002},"last":1.147},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.1120000000000001,"15min":1.083},"min":{"1min":0.69599999999999995,"5min":0.624,"15min":0.61399999999999999},"max":{"1min":1.603,"5min":1.962,"15min":2.2320000000000002},"last":0.81599999999999995}]},{"osd":1,"last update":"Sat Sep 23 03:28:27 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2050000000000001,"15min":1.218},"min":{"1min":0.73999999999999999,"5min":0.71899999999999997,"15min":0.62},"max":{"1min":1.657,"5min":2.4369999999999998,"15min":2.4369999999999998},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.268,"5min":1.294,"15min":1.2829999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67200000000000004},"max":{"1min":1.6120000000000001,"5min":3.5859999999999999,"15min":3.5859999999999999},"last":0.72899999999999998}]},{"osd":2,"last update":"Sat Sep 23 03:28:27 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.2050000000000001,"15min":1.2010000000000001},"min":{"1min":0.875,"5min":0.65300000000000002,"15min":0.64300000000000002},"max":{"1min":1.76,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.1399999999999999,"15min":1.2050000000000001},"min":{"1min":0.71699999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.4370000000000001,"5min":2.8500000000000001,"15min":2.8500000000000001},"last":0.88800000000000001}]},{"osd":3,"last update":"Sat Sep 23 03:28:27 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.2669999999999999,"15min":1.254},"min":{"1min":0.92800000000000005,"5min":0.67200000000000004,"15min":0.66400000000000003},"max":{"1min":1.6379999999999999,"5min":3.4359999999999999,"15min":3.4359999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.224,"5min":1.2969999999999999,"15min":1.339},"min":{"1min":0.91000000000000003,"5min":0.85999999999999999,"15min":0.69799999999999995},"max":{"1min":1.6719999999999999,"5min":1.9159999999999999,"15min":3.6110000000000002},"last":0.70099999999999996}]},{"osd":4,"last update":"Sat Sep 23 03:28:27 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.393,"15min":1.3899999999999999},"min":{"1min":0.92000000000000004,"5min":0.68600000000000005,"15min":0.54100000000000004},"max":{"1min":1.843,"5min":4.2750000000000004,"15min":4.2750000000000004},"last":0.50800000000000001},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.3009999999999999,"15min":1.341},"min":{"1min":1.0429999999999999,"5min":0.54100000000000004,"15min":0.48899999999999999},"max":{"1min":1.4990000000000001,"5min":3.3090000000000002,"15min":3.3090000000000002},"last":0.76200000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:28:27 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.504,"15min":1.5049999999999999},"min":{"1min":1.1180000000000001,"5min":1.0329999999999999,"15min":0.59199999999999997},"max":{"1min":1.77,"5min":4.6040000000000001,"15min":4.6040000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":1.389,"5min":1.4159999999999999,"15min":1.5069999999999999},"min":{"1min":1.206,"5min":0.81200000000000006,"15min":0.55100000000000005},"max":{"1min":1.677,"5min":3.169,"15min":9.6560000000000006},"last":0.98499999999999999}]},{"osd":7,"last update":"Sat Sep 23 03:28:27 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.4159999999999999,"15min":1.429},"min":{"1min":1.091,"5min":0.77100000000000002,"15min":0.626},"max":{"1min":1.708,"5min":2.077,"15min":2.077},"last":0.53900000000000003},{"interface":"front","average":{"1min":1.488,"5min":1.5009999999999999,"15min":1.5389999999999999},"min":{"1min":1.25,"5min":0.92200000000000004,"15min":0.60799999999999998},"max":{"1min":1.849,"5min":3.0739999999999998,"15min":3.0739999999999998},"last":1.052}]}]},{"osd":4,"up_from":13,"seq":55834575021,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32784,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339056,"statfs":{"total":96636764160,"available":96603193344,"internally_reserved":0,"allocated":1638400,"data_stored":798543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:28:24 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.091,"15min":1.1000000000000001},"min":{"1min":0.63800000000000001,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.5740000000000001,"5min":1.6699999999999999,"15min":1.9770000000000001},"last":1.554},{"interface":"front","average":{"1min":1.079,"5min":1.1599999999999999,"15min":1.167},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.7350000000000001,"5min":2.004,"15min":2.004},"last":1.325}]},{"osd":1,"last update":"Sat Sep 23 03:28:24 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.224,"15min":1.214},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.7490000000000001,"5min":1.8200000000000001,"15min":2.1110000000000002},"last":1.159},{"interface":"front","average":{"1min":1.022,"5min":1.0649999999999999,"15min":1.123},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60299999999999998},"max":{"1min":1.4890000000000001,"5min":1.718,"15min":3.2189999999999999},"last":1.111}]},{"osd":2,"last update":"Sat Sep 23 03:28:24 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":1.1559999999999999,"15min":1.1599999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.512,"5min":1.694,"15min":2.5920000000000001},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.252,"5min":1.278,"15min":1.25},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68300000000000005},"max":{"1min":1.8839999999999999,"5min":1.99,"15min":1.99},"last":1.657}]},{"osd":3,"last update":"Sat Sep 23 03:28:24 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.1699999999999999,"15min":1.1659999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4379999999999999,"5min":1.702,"15min":1.9450000000000001},"last":1.216},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2589999999999999,"15min":1.286},"min":{"1min":0.75900000000000001,"5min":0.69799999999999995,"15min":0.67400000000000004},"max":{"1min":1.611,"5min":1.833,"15min":2.4969999999999999},"last":1.262}]},{"osd":5,"last update":"Sat Sep 23 03:28:24 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.3540000000000001,"15min":1.363},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.70199999999999996},"max":{"1min":1.879,"5min":2.0409999999999999,"15min":2.8050000000000002},"last":1.591},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.2969999999999999,"15min":1.3240000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.619,"5min":1.8939999999999999,"15min":2.887},"last":0.93600000000000005}]},{"osd":6,"last update":"Sat Sep 23 03:28:24 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3660000000000001,"15min":1.369},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.9530000000000001,"5min":1.9530000000000001,"15min":2.4249999999999998},"last":1.393},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.478,"15min":1.4930000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.9350000000000001,"5min":2.0939999999999999,"15min":2.1779999999999999},"last":1.4370000000000001}]},{"osd":7,"last update":"Sat Sep 23 03:28:24 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.419,"15min":1.45},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.734,"5min":2.1389999999999998,"15min":2.5680000000000001},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.464,"5min":1.534,"15min":1.544},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.4769999999999999},"last":1.6220000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575021,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32784,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339056,"statfs":{"total":96636764160,"available":96603193344,"internally_reserved":0,"allocated":1638400,"data_stored":798543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.81000000000000005,"15min":0.85099999999999998},"min":{"1min":0.52600000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.22,"5min":1.5009999999999999,"15min":2.952},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.91500000000000004,"15min":0.89500000000000002},"min":{"1min":0.41799999999999998,"5min":0.377,"15min":0.29299999999999998},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":0.95599999999999996}]},{"osd":1,"last update":"Sat Sep 23 03:28:37 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.81899999999999995,"15min":0.86699999999999999},"min":{"1min":0.40000000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.48,"5min":1.48,"15min":1.6839999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.98499999999999999,"15min":0.98399999999999999},"min":{"1min":0.48299999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.482,"5min":1.7010000000000001,"15min":2.903},"last":1.101}]},{"osd":2,"last update":"Sat Sep 23 03:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.004,"15min":1.0409999999999999},"min":{"1min":0.54000000000000004,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.524,"5min":1.669,"15min":2.2240000000000002},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.119,"15min":1.1080000000000001},"min":{"1min":0.68799999999999994,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.8,"5min":1.8,"15min":3.129},"last":1.4039999999999999}]},{"osd":4,"last update":"Sat Sep 23 03:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1439999999999999,"15min":1.2070000000000001},"min":{"1min":0.88200000000000001,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.7010000000000001,"5min":1.7609999999999999,"15min":2.698},"last":1.198},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.264,"15min":1.327},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.752},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":2.419},"last":1.679}]},{"osd":5,"last update":"Sat Sep 23 03:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.282,"15min":1.3240000000000001},"min":{"1min":0.82499999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.9950000000000001},"last":1.246},{"interface":"front","average":{"1min":1.212,"5min":1.2090000000000001,"15min":1.266},"min":{"1min":0.80200000000000005,"5min":0.624,"15min":0.624},"max":{"1min":1.6799999999999999,"5min":1.9830000000000001,"15min":2.0070000000000001},"last":1.2749999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.284,"15min":1.337},"min":{"1min":0.77900000000000003,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.827,"5min":1.9530000000000001,"15min":2.6440000000000001},"last":1.7669999999999999},{"interface":"front","average":{"1min":1.363,"5min":1.3720000000000001,"15min":1.419},"min":{"1min":0.86399999999999999,"5min":0.73299999999999998,"15min":0.67800000000000005},"max":{"1min":2.0880000000000001,"5min":2.177,"15min":2.9750000000000001},"last":1.353}]},{"osd":7,"last update":"Sat Sep 23 03:28:37 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.286,"15min":1.357},"min":{"1min":0.81299999999999994,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.6279999999999999,"5min":2.0859999999999999,"15min":2.7879999999999998},"last":1.504},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3520000000000001,"15min":1.4079999999999999},"min":{"1min":0.90300000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.7290000000000001,"5min":2.0270000000000001,"15min":2.5379999999999998},"last":1.585}]}]},{"osd":2,"up_from":13,"seq":55834575022,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32784,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339056,"statfs":{"total":96636764160,"available":96603193344,"internally_reserved":0,"allocated":1638400,"data_stored":798543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.81299999999999994,"15min":0.82899999999999996},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.30299999999999999},"max":{"1min":0.92400000000000004,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.121},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.91200000000000003,"15min":0.88200000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.375,"5min":2.613,"15min":2.613},"last":0.92600000000000005}]},{"osd":1,"last update":"Sat Sep 23 03:28:28 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.85399999999999998,"15min":0.85499999999999998},"min":{"1min":0.46600000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.391,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":1.048},{"interface":"front","average":{"1min":1.006,"5min":1.032,"15min":0.98299999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.38},"max":{"1min":1.526,"5min":2.9809999999999999,"15min":2.9809999999999999},"last":1.0129999999999999}]},{"osd":3,"last update":"Sat Sep 23 03:28:28 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.0740000000000001,"15min":1.006},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.40699999999999997},"max":{"1min":1.6279999999999999,"5min":1.698,"15min":1.698},"last":1.159},{"interface":"front","average":{"1min":0.96999999999999997,"5min":1.044,"15min":1.01},"min":{"1min":0.46800000000000003,"5min":0.441,"15min":0.34999999999999998},"max":{"1min":1.482,"5min":1.827,"15min":1.827},"last":0.871}]},{"osd":4,"last update":"Sat Sep 23 03:28:28 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.3240000000000001,"15min":1.286},"min":{"1min":0.82899999999999996,"5min":0.79400000000000004,"15min":0.55800000000000005},"max":{"1min":1.5309999999999999,"5min":5.6669999999999998,"15min":5.6669999999999998},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.28,"5min":1.302,"15min":1.2949999999999999},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.7190000000000001,"5min":2.6909999999999998,"15min":2.6909999999999998},"last":1.7030000000000001}]},{"osd":5,"last update":"Sat Sep 23 03:28:28 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.3260000000000001,"15min":1.335},"min":{"1min":0.83399999999999996,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.6879999999999999,"5min":2.8969999999999998,"15min":3.9409999999999998},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.234,"5min":1.379,"15min":1.3959999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.56399999999999995},"max":{"1min":1.8109999999999999,"5min":3.415,"15min":4.9199999999999999},"last":1.3440000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:28:28 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.357,"15min":1.3580000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.755},"max":{"1min":1.7090000000000001,"5min":2.7839999999999998,"15min":3.0009999999999999},"last":1.794},{"interface":"front","average":{"1min":1.321,"5min":1.4570000000000001,"15min":1.4219999999999999},"min":{"1min":0.91800000000000004,"5min":0.76600000000000001,"15min":0.65100000000000002},"max":{"1min":1.851,"5min":7.8300000000000001,"15min":7.8300000000000001},"last":1.286}]},{"osd":7,"last update":"Sat Sep 23 03:28:28 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.423,"15min":1.395},"min":{"1min":0.83099999999999996,"5min":0.82899999999999996,"15min":0.61399999999999999},"max":{"1min":1.954,"5min":4.0330000000000004,"15min":4.0330000000000004},"last":1.25},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.4199999999999999,"15min":1.4239999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.74399999999999999},"max":{"1min":2.3140000000000001,"5min":4.2199999999999998,"15min":4.2199999999999998},"last":1.8520000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32784,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339056,"statfs":{"total":96636764160,"available":96603193344,"internally_reserved":0,"allocated":1638400,"data_stored":798543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 23 03:28:40 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.81899999999999995,"15min":0.80200000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.32300000000000001},"max":{"1min":1.8660000000000001,"5min":1.8660000000000001,"15min":2.8279999999999998},"last":1.1739999999999999},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.88300000000000001,"15min":0.84899999999999998},"min":{"1min":0.40899999999999997,"5min":0.40600000000000003,"15min":0.312},"max":{"1min":2.2490000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":0.96599999999999997}]},{"osd":2,"last update":"Sat Sep 23 03:28:40 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.86499999999999999,"15min":0.86899999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.36699999999999999},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":2.3340000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.069,"5min":1.0580000000000001,"15min":1.0329999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.909,"5min":1.909,"15min":1.9570000000000001},"last":1.3440000000000001}]},{"osd":3,"last update":"Sat Sep 23 03:28:40 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.046,"15min":1.0329999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.47299999999999998},"max":{"1min":1.992,"5min":1.992,"15min":2.2370000000000001},"last":1.127},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.141,"15min":1.109},"min":{"1min":0.495,"5min":0.495,"15min":0.45900000000000002},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":3.2130000000000001},"last":1.264}]},{"osd":4,"last update":"Sat Sep 23 03:28:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.2789999999999999,"15min":1.2490000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.798,"5min":1.885,"15min":3.2629999999999999},"last":1.6439999999999999},{"interface":"front","average":{"1min":1.266,"5min":1.244,"15min":1.2529999999999999},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66500000000000004},"max":{"1min":2.2690000000000001,"5min":2.2690000000000001,"15min":2.4689999999999999},"last":1.399}]},{"osd":5,"last update":"Sat Sep 23 03:28:40 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.2929999999999999,"15min":1.2789999999999999},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.64600000000000002},"max":{"1min":2.024,"5min":2.024,"15min":3.161},"last":1.367},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3280000000000001,"15min":1.3260000000000001},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.72399999999999998},"max":{"1min":3.0830000000000002,"5min":3.0830000000000002,"15min":3.0830000000000002},"last":1.6220000000000001}]},{"osd":6,"last update":"Sat Sep 23 03:28:40 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3640000000000001,"15min":1.329},"min":{"1min":0.81299999999999994,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":3.0179999999999998},"last":1.466},{"interface":"front","average":{"1min":1.296,"5min":1.363,"15min":1.363},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":2.1309999999999998,"5min":2.1309999999999998,"15min":3.3580000000000001},"last":1.6890000000000001}]},{"osd":7,"last update":"Sat Sep 23 03:28:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.337,"15min":1.3600000000000001},"min":{"1min":0.77600000000000002,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.946,"5min":1.946,"15min":3.1099999999999999},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4039999999999999,"15min":1.419},"min":{"1min":0.86699999999999999,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":3.1179999999999999,"5min":3.1179999999999999,"15min":3.306},"last":1.5109999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575023,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33352,"kb_used_data":2176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338488,"statfs":{"total":96636764160,"available":96602611712,"internally_reserved":0,"allocated":2228224,"data_stored":1387892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 23 03:28:30 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80900000000000005,"15min":0.83499999999999996},"min":{"1min":0.44700000000000001,"5min":0.38,"15min":0.30299999999999999},"max":{"1min":1.3799999999999999,"5min":1.383,"15min":1.919},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.91400000000000003,"5min":0.88900000000000001,"15min":0.88800000000000001},"min":{"1min":0.36699999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.351,"5min":1.4470000000000001,"15min":2.1360000000000001},"last":1.0169999999999999}]},{"osd":2,"last update":"Sat Sep 23 03:28:30 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.873,"15min":0.85399999999999998},"min":{"1min":0.46899999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.335,"5min":1.5249999999999999,"15min":3.089},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.97299999999999998,"15min":0.999},"min":{"1min":0.47699999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.4970000000000001,"5min":1.5760000000000001,"15min":2.633},"last":1.3720000000000001}]},{"osd":3,"last update":"Sat Sep 23 03:28:30 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.98799999999999999,"15min":0.98099999999999998},"min":{"1min":0.5,"5min":0.433,"15min":0.433},"max":{"1min":1.417,"5min":1.615,"15min":2.8660000000000001},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.03,"15min":1.018},"min":{"1min":0.505,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.446,"5min":1.802,"15min":2.012},"last":0.97499999999999998}]},{"osd":4,"last update":"Sat Sep 23 03:28:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.252,"15min":1.2609999999999999},"min":{"1min":0.73599999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.621,"5min":1.885,"15min":2.4140000000000001},"last":1.72},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2270000000000001,"15min":1.2609999999999999},"min":{"1min":0.83199999999999996,"5min":0.73999999999999999,"15min":0.65300000000000002},"max":{"1min":1.5309999999999999,"5min":1.845,"15min":3.0230000000000001},"last":1.5209999999999999}]},{"osd":5,"last update":"Sat Sep 23 03:28:30 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.296,"15min":1.274},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.65600000000000003},"max":{"1min":1.9530000000000001,"5min":2.1240000000000001,"15min":2.5489999999999999},"last":1.548},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3320000000000001,"15min":1.3380000000000001},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.7210000000000001,"5min":1.9159999999999999,"15min":2.1539999999999999},"last":1.7969999999999999}]},{"osd":6,"last update":"Sat Sep 23 03:28:30 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3149999999999999,"15min":1.325},"min":{"1min":0.78900000000000003,"5min":0.77700000000000002,"15min":0.75600000000000001},"max":{"1min":1.8600000000000001,"5min":2.0870000000000002,"15min":2.9489999999999998},"last":1.675},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.3839999999999999,"15min":1.3859999999999999},"min":{"1min":0.71199999999999997,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.8100000000000001,"5min":1.9670000000000001,"15min":2.3239999999999998},"last":1.863}]},{"osd":7,"last update":"Sat Sep 23 03:28:30 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.3420000000000001,"15min":1.3640000000000001},"min":{"1min":0.93200000000000005,"5min":0.752,"15min":0.747},"max":{"1min":1.722,"5min":2.052,"15min":2.2330000000000001},"last":1.454},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.405,"15min":1.4359999999999999},"min":{"1min":1.0960000000000001,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.7889999999999999,"5min":2.1419999999999999,"15min":2.3330000000000002},"last":1.585}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-23T03:29:23.447 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-09-23T03:29:23.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.662+0000 7fc0ce57a640 1 Processor -- start 2023-09-23T03:29:23.664 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.663+0000 7fc0ce57a640 1 -- start start 2023-09-23T03:29:23.665 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.664+0000 7fc0ce57a640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc0c81585e0 0x7fc0c80c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:23.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.664+0000 7fc0ce57a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0c80c9820 0x7fc0c8070750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:23.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.664+0000 7fc0ce57a640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0c80c9f80 con 0x7fc0c815f170 2023-09-23T03:29:23.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.664+0000 7fc0ce57a640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0c815b6a0 con 0x7fc0c81585e0 2023-09-23T03:29:23.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.664+0000 7fc0ce57a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c815adf0 con 0x7fc0c80c9820 2023-09-23T03:29:23.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.664+0000 7fc0cd578640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc0c81585e0 0x7fc0c80c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:23.666 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.665+0000 7fc0cd578640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fc0c81585e0 0x7fc0c80c92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43774/0 (socket says 172.21.15.154:43774) 2023-09-23T03:29:23.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.665+0000 7fc0cd578640 1 -- 172.21.15.154:0/1772571605 learned_addr learned my addr 172.21.15.154:0/1772571605 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:29:23.667 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.665+0000 7fc0ccd77640 1 --2- 172.21.15.154:0/1772571605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0c80c9820 0x7fc0c8070750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:23.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.665+0000 7fc0cd578640 1 -- 172.21.15.154:0/1772571605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0c80c9820 msgr2=0x7fc0c8070750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:29:23.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.665+0000 7fc0cd578640 1 --2- 172.21.15.154:0/1772571605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0c80c9820 0x7fc0c8070750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:23.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.665+0000 7fc0cd578640 1 -- 172.21.15.154:0/1772571605 >> v1:172.21.15.190:6789/0 conn(0x7fc0c815f170 legacy=0x7fc0c8069cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:29:23.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.665+0000 7fc0cd578640 1 -- 172.21.15.154:0/1772571605 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c80c9ca0 con 0x7fc0c81585e0 2023-09-23T03:29:23.668 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.666+0000 7fc0cd578640 1 --2- 172.21.15.154:0/1772571605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0c81585e0 0x7fc0c80c92e0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4009870 tx=0x7fc0c402b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=376ec527b4e931b9 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:29:23.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.667+0000 7fc0bbfff640 1 -- 172.21.15.154:0/1772571605 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4031740 con 0x7fc0c81585e0 2023-09-23T03:29:23.669 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.667+0000 7fc0bbfff640 1 -- 172.21.15.154:0/1772571605 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c40318a0 con 0x7fc0c81585e0 2023-09-23T03:29:23.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.667+0000 7fc0bbfff640 1 -- 172.21.15.154:0/1772571605 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4031c30 con 0x7fc0c81585e0 2023-09-23T03:29:23.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.667+0000 7fc0ce57a640 1 -- 172.21.15.154:0/1772571605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0c81585e0 msgr2=0x7fc0c80c92e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:23.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.667+0000 7fc0ce57a640 1 --2- 172.21.15.154:0/1772571605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0c81585e0 0x7fc0c80c92e0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4009870 tx=0x7fc0c402b7e0 comp rx=0 tx=0).stop 2023-09-23T03:29:23.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.668+0000 7fc0ce57a640 1 -- 172.21.15.154:0/1772571605 shutdown_connections 2023-09-23T03:29:23.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.668+0000 7fc0ce57a640 1 --2- 172.21.15.154:0/1772571605 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0c80c9820 0x7fc0c8070750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:23.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.668+0000 7fc0ce57a640 1 --2- 172.21.15.154:0/1772571605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0c81585e0 0x7fc0c80c92e0 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:23.670 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.668+0000 7fc0ce57a640 1 -- 172.21.15.154:0/1772571605 >> 172.21.15.154:0/1772571605 conn(0x7fc0c80cb710 msgr2=0x7fc0c80cbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:29:23.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.668+0000 7fc0ce57a640 1 -- 172.21.15.154:0/1772571605 shutdown_connections 2023-09-23T03:29:23.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.668+0000 7fc0ce57a640 1 -- 172.21.15.154:0/1772571605 wait complete. 2023-09-23T03:29:23.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.670+0000 7fc0ce57a640 1 Processor -- start 2023-09-23T03:29:23.671 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.670+0000 7fc0ce57a640 1 -- start start 2023-09-23T03:29:23.672 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.671+0000 7fc0ce57a640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0c80c9820 0x7fc0c80b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:23.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.671+0000 7fc0ce57a640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0c81585e0 0x7fc0c81552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:23.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.671+0000 7fc0ce57a640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0c80b2110 0x7fc0c8156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:23.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.671+0000 7fc0ce57a640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c81566f0 con 0x7fc0c80b2110 2023-09-23T03:29:23.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.671+0000 7fc0ce57a640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c81557f0 con 0x7fc0c81585e0 2023-09-23T03:29:23.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.671+0000 7fc0ce57a640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c8155a70 con 0x7fc0c80c9820 2023-09-23T03:29:23.673 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.671+0000 7fc0cdd79640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0c80b2110 0x7fc0c8156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:23.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.671+0000 7fc0cdd79640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0c80b2110 0x7fc0c8156b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:43786/0 (socket says 172.21.15.154:43786) 2023-09-23T03:29:23.674 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.671+0000 7fc0cdd79640 1 -- 172.21.15.154:0/903134817 learned_addr learned my addr 172.21.15.154:0/903134817 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:29:23.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.672+0000 7fc0cdd79640 1 -- 172.21.15.154:0/903134817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0c80c9820 msgr2=0x7fc0c80b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:23.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.672+0000 7fc0cd578640 1 --2- 172.21.15.154:0/903134817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0c80c9820 0x7fc0c80b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:23.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.672+0000 7fc0ccd77640 1 --2- 172.21.15.154:0/903134817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0c81585e0 0x7fc0c81552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:23.675 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.672+0000 7fc0cdd79640 1 --2- 172.21.15.154:0/903134817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0c80c9820 0x7fc0c80b43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:23.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.672+0000 7fc0cdd79640 1 -- 172.21.15.154:0/903134817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0c81585e0 msgr2=0x7fc0c81552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:23.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.672+0000 7fc0cdd79640 1 --2- 172.21.15.154:0/903134817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0c81585e0 0x7fc0c81552b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:23.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.672+0000 7fc0cdd79640 1 -- 172.21.15.154:0/903134817 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c40094d0 con 0x7fc0c80b2110 2023-09-23T03:29:23.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.672+0000 7fc0cd578640 1 --2- 172.21.15.154:0/903134817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0c80c9820 0x7fc0c80b43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:29:23.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.672+0000 7fc0cdd79640 1 --2- 172.21.15.154:0/903134817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0c80b2110 0x7fc0c8156b10 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000b9e0 tx=0x7fc0c000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:29:23.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.672+0000 7fc0b9ffb640 1 -- 172.21.15.154:0/903134817 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0003b70 con 0x7fc0c80b2110 2023-09-23T03:29:23.676 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.672+0000 7fc0b9ffb640 1 -- 172.21.15.154:0/903134817 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c0003cd0 con 0x7fc0c80b2110 2023-09-23T03:29:23.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.672+0000 7fc0b9ffb640 1 -- 172.21.15.154:0/903134817 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c000cb60 con 0x7fc0c80b2110 2023-09-23T03:29:23.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.672+0000 7fc0ce57a640 1 -- 172.21.15.154:0/903134817 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0c8155d50 con 0x7fc0c80b2110 2023-09-23T03:29:23.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.673+0000 7fc0ce57a640 1 -- 172.21.15.154:0/903134817 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0c8156310 con 0x7fc0c80b2110 2023-09-23T03:29:23.677 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.674+0000 7fc0ce57a640 1 -- 172.21.15.154:0/903134817 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0c8156830 con 0x7fc0c80b2110 2023-09-23T03:29:23.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.675+0000 7fc0b9ffb640 1 -- 172.21.15.154:0/903134817 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fc0c000ccc0 con 0x7fc0c80b2110 2023-09-23T03:29:23.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.676+0000 7fc0b9ffb640 1 --2- 172.21.15.154:0/903134817 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc0ac06af30 0x7fc0ac06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:23.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.676+0000 7fc0b9ffb640 1 -- 172.21.15.154:0/903134817 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fc0c0089330 con 0x7fc0c80b2110 2023-09-23T03:29:23.678 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.677+0000 7fc0cd578640 1 --2- 172.21.15.154:0/903134817 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc0ac06af30 0x7fc0ac06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:23.679 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.677+0000 7fc0cd578640 1 --2- 172.21.15.154:0/903134817 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc0ac06af30 0x7fc0ac06d3f0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4035000 tx=0x7fc0c4008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:29:23.682 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.681+0000 7fc0b9ffb640 1 -- 172.21.15.154:0/903134817 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fc0c8156830 con 0x7fc0c80b2110 2023-09-23T03:29:23.871 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.870+0000 7fc0ce57a640 1 -- 172.21.15.154:0/903134817 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fc0c8156830 con 0x7fc0c80b2110 2023-09-23T03:29:23.879 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.877+0000 7fc0b9ffb640 1 -- 172.21.15.154:0/903134817 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v20)=0 v20) v1 ==== 123+0+0 (secure 0 0 0) 0x7fc0c8156830 con 0x7fc0c80b2110 2023-09-23T03:29:23.880 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.879+0000 7fc0b9ffb640 1 -- 172.21.15.154:0/903134817 <== mon.0 v2:172.21.15.154:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fc0c0009880 con 0x7fc0c80b2110 2023-09-23T03:29:23.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.881+0000 7fc0ce57a640 1 -- 172.21.15.154:0/903134817 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc0ac06af30 msgr2=0x7fc0ac06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:23.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.881+0000 7fc0ce57a640 1 --2- 172.21.15.154:0/903134817 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc0ac06af30 0x7fc0ac06d3f0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4035000 tx=0x7fc0c4008030 comp rx=0 tx=0).stop 2023-09-23T03:29:23.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.881+0000 7fc0ce57a640 1 -- 172.21.15.154:0/903134817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0c80b2110 msgr2=0x7fc0c8156b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:23.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.881+0000 7fc0ce57a640 1 --2- 172.21.15.154:0/903134817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0c80b2110 0x7fc0c8156b10 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000b9e0 tx=0x7fc0c000bea0 comp rx=0 tx=0).stop 2023-09-23T03:29:23.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.881+0000 7fc0ce57a640 1 -- 172.21.15.154:0/903134817 shutdown_connections 2023-09-23T03:29:23.883 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.881+0000 7fc0ce57a640 1 --2- 172.21.15.154:0/903134817 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fc0ac06af30 0x7fc0ac06d3f0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:23.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.881+0000 7fc0ce57a640 1 --2- 172.21.15.154:0/903134817 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0c80b2110 0x7fc0c8156b10 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:23.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.881+0000 7fc0ce57a640 1 --2- 172.21.15.154:0/903134817 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fc0c81585e0 0x7fc0c81552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:23.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.881+0000 7fc0ce57a640 1 --2- 172.21.15.154:0/903134817 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fc0c80c9820 0x7fc0c80b43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:23.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.881+0000 7fc0ce57a640 1 -- 172.21.15.154:0/903134817 >> 172.21.15.154:0/903134817 conn(0x7fc0c80cb710 msgr2=0x7fc0c8154180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:29:23.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.882+0000 7fc0ce57a640 1 -- 172.21.15.154:0/903134817 shutdown_connections 2023-09-23T03:29:23.884 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:23.882+0000 7fc0ce57a640 1 -- 172.21.15.154:0/903134817 wait complete. 2023-09-23T03:29:23.900 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-09-23T03:29:23.901 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-23T03:29:24.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.106+0000 7fdd0f59e640 1 Processor -- start 2023-09-23T03:29:24.107 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.106+0000 7fdd0f59e640 1 -- start start 2023-09-23T03:29:24.109 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.107+0000 7fdd0f59e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd100cb320 0x7fdd100c9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:24.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.107+0000 7fdd0f59e640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdd10159ba0 0x7fdd100c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:24.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.107+0000 7fdd0f59e640 1 -- --> v1:172.21.15.190:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd1015a5f0 con 0x7fdd1015a050 2023-09-23T03:29:24.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.107+0000 7fdd0f59e640 1 -- --> v2:172.21.15.154:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd1015abb0 con 0x7fdd10159ba0 2023-09-23T03:29:24.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.107+0000 7fdd0f59e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1015a8d0 con 0x7fdd100cb320 2023-09-23T03:29:24.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.108+0000 7fdd05d9b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdd10159ba0 0x7fdd100c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:24.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.108+0000 7fdd05d9b640 1 --2- >> v2:172.21.15.154:3300/0 conn(0x7fdd10159ba0 0x7fdd100c9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:41472/0 (socket says 172.21.15.154:41472) 2023-09-23T03:29:24.110 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.108+0000 7fdd05d9b640 1 -- 172.21.15.154:0/811187122 learned_addr learned my addr 172.21.15.154:0/811187122 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:29:24.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.108+0000 7fdd0e59c640 1 --2- 172.21.15.154:0/811187122 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd100cb320 0x7fdd100c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:24.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.108+0000 7fdd05d9b640 1 -- 172.21.15.154:0/811187122 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd100cb320 msgr2=0x7fdd100c9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:29:24.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.108+0000 7fdd05d9b640 1 --2- 172.21.15.154:0/811187122 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd100cb320 0x7fdd100c9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:24.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.108+0000 7fdd05d9b640 1 -- 172.21.15.154:0/811187122 >> v1:172.21.15.190:6789/0 conn(0x7fdd1015a050 legacy=0x7fdd10070ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-23T03:29:24.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.108+0000 7fdd05d9b640 1 -- 172.21.15.154:0/811187122 --> v2:172.21.15.154:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd100c8b30 con 0x7fdd10159ba0 2023-09-23T03:29:24.111 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.108+0000 7fdd0e59c640 1 --2- 172.21.15.154:0/811187122 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd100cb320 0x7fdd100c9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:29:24.112 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.109+0000 7fdd05d9b640 1 --2- 172.21.15.154:0/811187122 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd10159ba0 0x7fdd100c9cb0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8009870 tx=0x7fdcf802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3469d954d0d00aa2 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:29:24.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.110+0000 7fdd0dd9b640 1 -- 172.21.15.154:0/811187122 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf80317c0 con 0x7fdd10159ba0 2023-09-23T03:29:24.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.110+0000 7fdd0dd9b640 1 -- 172.21.15.154:0/811187122 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fdcf8031920 con 0x7fdd10159ba0 2023-09-23T03:29:24.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.110+0000 7fdd0dd9b640 1 -- 172.21.15.154:0/811187122 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8031cd0 con 0x7fdd10159ba0 2023-09-23T03:29:24.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.110+0000 7fdd0f59e640 1 -- 172.21.15.154:0/811187122 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd10159ba0 msgr2=0x7fdd100c9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:24.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.110+0000 7fdd0f59e640 1 --2- 172.21.15.154:0/811187122 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd10159ba0 0x7fdd100c9cb0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8009870 tx=0x7fdcf802b860 comp rx=0 tx=0).stop 2023-09-23T03:29:24.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.110+0000 7fdd0f59e640 1 -- 172.21.15.154:0/811187122 shutdown_connections 2023-09-23T03:29:24.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.110+0000 7fdd0f59e640 1 --2- 172.21.15.154:0/811187122 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd10159ba0 0x7fdd100c9cb0 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:24.113 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.110+0000 7fdd0f59e640 1 --2- 172.21.15.154:0/811187122 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd100cb320 0x7fdd100c9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:24.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.110+0000 7fdd0f59e640 1 -- 172.21.15.154:0/811187122 >> 172.21.15.154:0/811187122 conn(0x7fdd1006a4c0 msgr2=0x7fdd100afe50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:29:24.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.110+0000 7fdd0f59e640 1 -- 172.21.15.154:0/811187122 shutdown_connections 2023-09-23T03:29:24.114 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.110+0000 7fdd0f59e640 1 -- 172.21.15.154:0/811187122 wait complete. 2023-09-23T03:29:24.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.112+0000 7fdd0f59e640 1 Processor -- start 2023-09-23T03:29:24.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.112+0000 7fdd0f59e640 1 -- start start 2023-09-23T03:29:24.116 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.113+0000 7fdd0f59e640 1 --2- >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd100cb320 0x7fdd10152f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:24.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.113+0000 7fdd0f59e640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd10159ba0 0x7fdd101f25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:24.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.113+0000 7fdd0f59e640 1 --2- >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd1015a050 0x7fdd101f3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:24.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.113+0000 7fdd0f59e640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd100c67f0 con 0x7fdd10159ba0 2023-09-23T03:29:24.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.113+0000 7fdd0f59e640 1 -- --> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd101f2ae0 con 0x7fdd100cb320 2023-09-23T03:29:24.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.113+0000 7fdd0f59e640 1 -- --> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd100c6a20 con 0x7fdd1015a050 2023-09-23T03:29:24.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.113+0000 7fdd05d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd10159ba0 0x7fdd101f25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:24.117 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.113+0000 7fdd05d9b640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd10159ba0 0x7fdd101f25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:41486/0 (socket says 172.21.15.154:41486) 2023-09-23T03:29:24.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.113+0000 7fdd05d9b640 1 -- 172.21.15.154:0/3899110596 learned_addr learned my addr 172.21.15.154:0/3899110596 (peer_addr_for_me v2:172.21.15.154:0/0) 2023-09-23T03:29:24.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.114+0000 7fdd0e59c640 1 --2- 172.21.15.154:0/3899110596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd100cb320 0x7fdd10152f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:24.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.114+0000 7fdd0ed9d640 1 --2- 172.21.15.154:0/3899110596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd1015a050 0x7fdd101f3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:24.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.114+0000 7fdd05d9b640 1 -- 172.21.15.154:0/3899110596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd1015a050 msgr2=0x7fdd101f3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:24.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.114+0000 7fdd05d9b640 1 --2- 172.21.15.154:0/3899110596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd1015a050 0x7fdd101f3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:24.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.114+0000 7fdd05d9b640 1 -- 172.21.15.154:0/3899110596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd100cb320 msgr2=0x7fdd10152f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:24.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.114+0000 7fdd05d9b640 1 --2- 172.21.15.154:0/3899110596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd100cb320 0x7fdd10152f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:24.118 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.114+0000 7fdd05d9b640 1 -- 172.21.15.154:0/3899110596 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf80094d0 con 0x7fdd10159ba0 2023-09-23T03:29:24.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.114+0000 7fdd0e59c640 1 --2- 172.21.15.154:0/3899110596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd100cb320 0x7fdd10152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:29:24.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.114+0000 7fdd0ed9d640 1 --2- 172.21.15.154:0/3899110596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd1015a050 0x7fdd101f3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-23T03:29:24.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.114+0000 7fdd05d9b640 1 --2- 172.21.15.154:0/3899110596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd10159ba0 0x7fdd101f25a0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8000c00 tx=0x7fdcf80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:29:24.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.114+0000 7fdd077fe640 1 -- 172.21.15.154:0/3899110596 <== mon.0 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8032a20 con 0x7fdd10159ba0 2023-09-23T03:29:24.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.115+0000 7fdd077fe640 1 -- 172.21.15.154:0/3899110596 <== mon.0 v2:172.21.15.154:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fdcf8032ba0 con 0x7fdd10159ba0 2023-09-23T03:29:24.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.115+0000 7fdd0f59e640 1 -- 172.21.15.154:0/3899110596 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd100c6cc0 con 0x7fdd10159ba0 2023-09-23T03:29:24.119 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.115+0000 7fdd077fe640 1 -- 172.21.15.154:0/3899110596 <== mon.0 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf80453f0 con 0x7fdd10159ba0 2023-09-23T03:29:24.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.115+0000 7fdd0f59e640 1 -- 172.21.15.154:0/3899110596 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd100c7280 con 0x7fdd10159ba0 2023-09-23T03:29:24.120 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.116+0000 7fdd0f59e640 1 -- 172.21.15.154:0/3899110596 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd10066730 con 0x7fdd10159ba0 2023-09-23T03:29:24.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.117+0000 7fdd077fe640 1 -- 172.21.15.154:0/3899110596 <== mon.0 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 5) v1 ==== 89355+0+0 (secure 0 0 0) 0x7fdcf804d3f0 con 0x7fdd10159ba0 2023-09-23T03:29:24.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.117+0000 7fdd077fe640 1 --2- 172.21.15.154:0/3899110596 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdcf006af60 0x7fdcf006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-23T03:29:24.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.118+0000 7fdd077fe640 1 -- 172.21.15.154:0/3899110596 <== mon.0 v2:172.21.15.154:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5435+0+0 (secure 0 0 0) 0x7fdcf80ab730 con 0x7fdd10159ba0 2023-09-23T03:29:24.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.118+0000 7fdd0e59c640 1 --2- 172.21.15.154:0/3899110596 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdcf006af60 0x7fdcf006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-23T03:29:24.121 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.118+0000 7fdd0e59c640 1 --2- 172.21.15.154:0/3899110596 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdcf006af60 0x7fdcf006d420 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fdd00006fd0 tx=0x7fdd00008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-23T03:29:24.123 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.122+0000 7fdd077fe640 1 -- 172.21.15.154:0/3899110596 <== mon.0 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178749 (secure 0 0 0) 0x7fdcf80767e0 con 0x7fdd10159ba0 2023-09-23T03:29:24.334 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.333+0000 7fdd0f59e640 1 -- 172.21.15.154:0/3899110596 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd1004e920 con 0x7fdd10159ba0 2023-09-23T03:29:24.335 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.333+0000 7fdd077fe640 1 -- 172.21.15.154:0/3899110596 <== mon.0 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) v1 ==== 95+0+3716 (secure 0 0 0) 0x7fdcf8079e00 con 0x7fdd10159ba0 2023-09-23T03:29:24.335 INFO:teuthology.orchestra.run.smithi154.stdout: 2023-09-23T03:29:24.335 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":123,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5840,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/2759187683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":2759187683},{"type":"v1","addr":"172.21.15.154:6835","nonce":2759187683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102},{"gid":5924,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6835/3712825179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6833","nonce":3712825179},{"type":"v1","addr":"172.21.15.190:6835","nonce":3712825179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":110},{"gid":6544,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.190:6834/4147378242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.190:6832","nonce":4147378242},{"type":"v1","addr":"172.21.15.190:6834","nonce":4147378242}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103},{"gid":9543,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/3970611329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3970611329},{"type":"v1","addr":"172.21.15.154:6837","nonce":3970611329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123}],"filesystems":[]} 2023-09-23T03:29:24.336 INFO:teuthology.orchestra.run.smithi154.stderr:dumped fsmap epoch 123 2023-09-23T03:29:24.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.336+0000 7fdd0f59e640 1 -- 172.21.15.154:0/3899110596 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdcf006af60 msgr2=0x7fdcf006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:24.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.336+0000 7fdd0f59e640 1 --2- 172.21.15.154:0/3899110596 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdcf006af60 0x7fdcf006d420 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fdd00006fd0 tx=0x7fdd00008040 comp rx=0 tx=0).stop 2023-09-23T03:29:24.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.336+0000 7fdd0f59e640 1 -- 172.21.15.154:0/3899110596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd10159ba0 msgr2=0x7fdd101f25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-23T03:29:24.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.336+0000 7fdd0f59e640 1 --2- 172.21.15.154:0/3899110596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd10159ba0 0x7fdd101f25a0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8000c00 tx=0x7fdcf80046c0 comp rx=0 tx=0).stop 2023-09-23T03:29:24.338 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.336+0000 7fdd0f59e640 1 -- 172.21.15.154:0/3899110596 shutdown_connections 2023-09-23T03:29:24.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.336+0000 7fdd0f59e640 1 --2- 172.21.15.154:0/3899110596 >> [v2:172.21.15.154:6832/2422652104,v1:172.21.15.154:6833/2422652104] conn(0x7fdcf006af60 0x7fdcf006d420 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:24.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.336+0000 7fdd0f59e640 1 --2- 172.21.15.154:0/3899110596 >> [v2:172.21.15.190:3301/0,v1:172.21.15.190:6790/0] conn(0x7fdd1015a050 0x7fdd101f3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:24.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.336+0000 7fdd0f59e640 1 --2- 172.21.15.154:0/3899110596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd10159ba0 0x7fdd101f25a0 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:24.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.336+0000 7fdd0f59e640 1 --2- 172.21.15.154:0/3899110596 >> [v2:172.21.15.190:3300/0,v1:172.21.15.190:6789/0] conn(0x7fdd100cb320 0x7fdd10152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-23T03:29:24.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.336+0000 7fdd0f59e640 1 -- 172.21.15.154:0/3899110596 >> 172.21.15.154:0/3899110596 conn(0x7fdd1006a4c0 msgr2=0x7fdd100cc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-23T03:29:24.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.337+0000 7fdd0f59e640 1 -- 172.21.15.154:0/3899110596 shutdown_connections 2023-09-23T03:29:24.339 INFO:teuthology.orchestra.run.smithi154.stderr:2023-09-23T03:29:24.337+0000 7fdd0f59e640 1 -- 172.21.15.154:0/3899110596 wait complete. 2023-09-23T03:29:24.355 DEBUG:tasks.cephfs.filesystem:already dead... 2023-09-23T03:29:24.355 INFO:teuthology.misc:Shutting down mds daemons... 2023-09-23T03:29:24.355 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-09-23T03:29:24.355 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:29:24.372 INFO:tasks.ceph.mds.a:Stopped 2023-09-23T03:29:24.373 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-09-23T03:29:24.373 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:29:24.446 INFO:tasks.ceph.mds.c:Stopped 2023-09-23T03:29:24.447 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-09-23T03:29:24.447 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:29:24.468 INFO:tasks.ceph.mds.b:Stopped 2023-09-23T03:29:24.469 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-09-23T03:29:24.469 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:29:24.532 INFO:tasks.ceph.mds.d:Stopped 2023-09-23T03:29:24.532 INFO:teuthology.misc:Shutting down osd daemons... 2023-09-23T03:29:24.533 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-09-23T03:29:24.533 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:29:24.585 INFO:tasks.ceph.osd.0:Stopped 2023-09-23T03:29:24.586 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-09-23T03:29:24.586 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:29:24.662 INFO:tasks.ceph.osd.1:Stopped 2023-09-23T03:29:24.662 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-09-23T03:29:24.662 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:29:24.725 INFO:tasks.ceph.osd.2:Stopped 2023-09-23T03:29:24.726 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-09-23T03:29:24.726 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:29:24.733 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2023-09-23T03:29:24.733 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-09-23T03:29:24.734 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-09-23T03:29:24.797 INFO:tasks.ceph.osd.3:Stopped 2023-09-23T03:29:24.798 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-09-23T03:29:24.798 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:29:24.877 INFO:tasks.ceph.osd.4:Stopped 2023-09-23T03:29:24.877 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-09-23T03:29:24.877 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:29:24.954 INFO:tasks.ceph.osd.5:Stopped 2023-09-23T03:29:24.955 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-09-23T03:29:24.955 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:29:25.017 INFO:tasks.ceph.osd.6:Stopped 2023-09-23T03:29:25.018 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-09-23T03:29:25.018 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:29:25.088 INFO:tasks.ceph.osd.7:Stopped 2023-09-23T03:29:25.088 INFO:teuthology.misc:Shutting down mgr daemons... 2023-09-23T03:29:25.088 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-09-23T03:29:25.088 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:29:25.166 INFO:tasks.ceph.mgr.y:Stopped 2023-09-23T03:29:25.167 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-09-23T03:29:25.167 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:29:30.036 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2023-09-23T03:29:30.037 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2023-09-23T03:29:30.037 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-09-23T03:29:30.037 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-09-23T03:29:31.268 INFO:tasks.ceph.mgr.x:Stopped 2023-09-23T03:29:31.268 INFO:teuthology.misc:Shutting down mon daemons... 2023-09-23T03:29:31.269 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-09-23T03:29:31.269 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:29:31.301 INFO:tasks.ceph.mon.a:Stopped 2023-09-23T03:29:31.301 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-09-23T03:29:31.302 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:29:31.326 INFO:tasks.ceph.mon.b:Stopped 2023-09-23T03:29:31.326 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-09-23T03:29:31.326 INFO:teuthology.orchestra.run:waiting for 300 2023-09-23T03:29:31.405 INFO:tasks.ceph.mon.c:Stopped 2023-09-23T03:29:31.405 INFO:tasks.ceph:Checking cluster log for badness... 2023-09-23T03:29:31.406 DEBUG:teuthology.orchestra.run.smithi154:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'bad backtrace' | egrep -v 'object missing on disk' | egrep -v 'error reading table object' | egrep -v 'error reading sessionmap' | egrep -v 'unmatched fragstat' | egrep -v 'unmatched rstat' | egrep -v 'was unreadable, recreating it now' | egrep -v 'Scrub error on inode' | egrep -v 'Metadata damage detected' | egrep -v 'inconsistent rstat on inode' | egrep -v 'Error recovering journal' | head -n 1 2023-09-23T03:29:31.524 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi154.front.sepia.ceph.com 2023-09-23T03:29:31.524 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-09-23T03:29:31.874 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi154.front.sepia.ceph.com 2023-09-23T03:29:31.874 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-09-23T03:29:32.147 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi154.front.sepia.ceph.com 2023-09-23T03:29:32.148 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-09-23T03:29:32.433 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi154.front.sepia.ceph.com 2023-09-23T03:29:32.434 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-09-23T03:29:32.719 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi190.front.sepia.ceph.com 2023-09-23T03:29:33.009 DEBUG:teuthology.orchestra.run.smithi190:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-09-23T03:29:33.365 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi190.front.sepia.ceph.com 2023-09-23T03:29:33.366 DEBUG:teuthology.orchestra.run.smithi190:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-09-23T03:29:33.640 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi190.front.sepia.ceph.com 2023-09-23T03:29:33.641 DEBUG:teuthology.orchestra.run.smithi190:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-09-23T03:29:33.914 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi190.front.sepia.ceph.com 2023-09-23T03:29:33.914 DEBUG:teuthology.orchestra.run.smithi190:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-09-23T03:29:34.176 INFO:tasks.ceph:Archiving mon data... 2023-09-23T03:29:34.178 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401456/data/mon.a.tgz 2023-09-23T03:29:34.178 DEBUG:teuthology.orchestra.run.smithi154:> mktemp 2023-09-23T03:29:34.238 INFO:teuthology.orchestra.run.smithi154.stdout:/tmp/tmp.nKAvYBpPaU 2023-09-23T03:29:34.238 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.nKAvYBpPaU 2023-09-23T03:29:34.481 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0666 /tmp/tmp.nKAvYBpPaU 2023-09-23T03:29:34.640 DEBUG:teuthology.orchestra.remote:smithi154:/tmp/tmp.nKAvYBpPaU is 467KB 2023-09-23T03:29:34.749 DEBUG:teuthology.orchestra.run.smithi154:> rm -fr /tmp/tmp.nKAvYBpPaU 2023-09-23T03:29:34.808 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401456/data/mon.b.tgz 2023-09-23T03:29:34.808 DEBUG:teuthology.orchestra.run.smithi190:> mktemp 2023-09-23T03:29:34.867 INFO:teuthology.orchestra.run.smithi190.stdout:/tmp/tmp.CxrfF5zl1Y 2023-09-23T03:29:34.867 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.CxrfF5zl1Y 2023-09-23T03:29:35.038 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x has been restored 2023-09-23T03:29:35.082 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0666 /tmp/tmp.CxrfF5zl1Y 2023-09-23T03:29:35.239 DEBUG:teuthology.orchestra.remote:smithi190:/tmp/tmp.CxrfF5zl1Y is 482KB 2023-09-23T03:29:35.348 DEBUG:teuthology.orchestra.run.smithi190:> rm -fr /tmp/tmp.CxrfF5zl1Y 2023-09-23T03:29:35.407 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401456/data/mon.c.tgz 2023-09-23T03:29:35.408 DEBUG:teuthology.orchestra.run.smithi190:> mktemp 2023-09-23T03:29:35.463 INFO:teuthology.orchestra.run.smithi190.stdout:/tmp/tmp.2dLg7MhSiX 2023-09-23T03:29:35.464 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.2dLg7MhSiX 2023-09-23T03:29:35.667 DEBUG:teuthology.orchestra.run.smithi190:> sudo chmod 0666 /tmp/tmp.2dLg7MhSiX 2023-09-23T03:29:35.821 DEBUG:teuthology.orchestra.remote:smithi190:/tmp/tmp.2dLg7MhSiX is 482KB 2023-09-23T03:29:35.930 DEBUG:teuthology.orchestra.run.smithi190:> rm -fr /tmp/tmp.2dLg7MhSiX 2023-09-23T03:29:35.990 INFO:tasks.ceph:Cleaning ceph cluster... 2023-09-23T03:29:35.990 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-09-23T03:29:35.999 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-09-23T03:29:36.099 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-09-23T03:29:36.100 DEBUG:teuthology.orchestra.run.smithi154:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-09-23T03:29:36.108 DEBUG:teuthology.orchestra.run.smithi190:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-09-23T03:29:36.200 INFO:teuthology.orchestra.run.smithi154.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-09-23T03:29:36.214 INFO:teuthology.orchestra.run.smithi190.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-09-23T03:29:36.226 INFO:tasks.ceph:Archiving crash dumps... 2023-09-23T03:29:36.228 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401456/remote/smithi154/crash 2023-09-23T03:29:36.228 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-09-23T03:29:36.322 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401456/remote/smithi190/crash 2023-09-23T03:29:36.323 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-09-23T03:29:36.419 INFO:tasks.ceph:Shutting down logrotate 2023-09-23T03:29:36.419 DEBUG:teuthology.orchestra.run.smithi154:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:29:36.429 DEBUG:teuthology.orchestra.run.smithi190:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-23T03:29:36.559 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-09-23T03:29:36.660 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-09-23T03:29:36.763 INFO:tasks.ceph:Compressing logs... 2023-09-23T03:29:36.763 DEBUG:teuthology.orchestra.run.smithi154:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-09-23T03:29:36.772 DEBUG:teuthology.orchestra.run.smithi190:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-09-23T03:29:36.861 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131603.log 2023-09-23T03:29:36.863 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103720.log 2023-09-23T03:29:36.864 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.131603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102747.log 2023-09-23T03:29:36.865 INFO:teuthology.orchestra.run.smithi154.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.131603.log.gz 2023-09-23T03:29:36.866 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.103720.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103720.log.gz 2023-09-23T03:29:36.866 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122399.log 2023-09-23T03:29:36.866 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-09-23T03:29:36.867 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.102747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97498.log 2023-09-23T03:29:36.867 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86970.log 2023-09-23T03:29:36.868 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.122399.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.102747.log.gz 2023-09-23T03:29:36.868 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87090.log 2023-09-23T03:29:36.869 INFO:teuthology.orchestra.run.smithi154.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.122399.log.gz 2023-09-23T03:29:36.869 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93760.log 2023-09-23T03:29:36.870 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.97498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93902.log 2023-09-23T03:29:36.870 INFO:teuthology.orchestra.run.smithi154.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97498.log.gz 2023-09-23T03:29:36.870 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.86970.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-09-23T03:29:36.871 INFO:teuthology.orchestra.run.smithi190.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.86970.log.gz 2023-09-23T03:29:36.872 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.87090.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-09-23T03:29:36.872 INFO:teuthology.orchestra.run.smithi190.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.87090.log.gz 2023-09-23T03:29:36.872 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.93760.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.93760.log.gzgzip 2023-09-23T03:29:36.872 INFO:teuthology.orchestra.run.smithi154.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.102369.log 2023-09-23T03:29:36.873 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.92902.log 2023-09-23T03:29:36.873 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.93902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111298.log 2023-09-23T03:29:36.873 INFO:teuthology.orchestra.run.smithi154.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.93902.log.gz 2023-09-23T03:29:36.874 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-09-23T03:29:36.874 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.102369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127620.log 2023-09-23T03:29:36.874 INFO:teuthology.orchestra.run.smithi154.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.102369.log.gz 2023-09-23T03:29:36.875 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.3.92902.log: 92.3% -- replaced with /var/log/ceph/ceph-client.3.92902.log.gz 2023-09-23T03:29:36.875 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.92450.log 2023-09-23T03:29:36.876 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.111298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92575.log 2023-09-23T03:29:36.876 INFO:teuthology.orchestra.run.smithi154.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.111298.log.gz 2023-09-23T03:29:36.877 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.127620.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.127620.log.gz 2023-09-23T03:29:36.877 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109111.log 2023-09-23T03:29:36.878 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-osd.4.log: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-09-23T03:29:36.878 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87010.log 2023-09-23T03:29:36.878 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.1.92450.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-09-23T03:29:36.878 INFO:teuthology.orchestra.run.smithi190.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.1.92450.log.gz 2023-09-23T03:29:36.879 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.87010.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-09-23T03:29:36.879 INFO:teuthology.orchestra.run.smithi190.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.87010.log.gz 2023-09-23T03:29:36.880 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.92575.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.92575.log.gz 2023-09-23T03:29:36.880 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104523.log 2023-09-23T03:29:36.880 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.109111.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.109111.log.gz 2023-09-23T03:29:36.880 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-09-23T03:29:36.882 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87050.log 2023-09-23T03:29:36.882 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86890.log 2023-09-23T03:29:36.882 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.104523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100259.log 2023-09-23T03:29:36.882 INFO:teuthology.orchestra.run.smithi154.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.104523.log.gz 2023-09-23T03:29:36.882 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107953.log 2023-09-23T03:29:36.883 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.100259.log: gzip -5 --verbose -- 88.5% /var/log/ceph/ceph-client.admin.106470.log 2023-09-23T03:29:36.883 INFO:teuthology.orchestra.run.smithi154.stderr: -- replaced with /var/log/ceph/ceph-client.admin.100259.log.gz 2023-09-23T03:29:36.884 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.87050.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.87050.log.gz 2023-09-23T03:29:36.884 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-09-23T03:29:36.885 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.107953.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.107953.log.gz 2023-09-23T03:29:36.885 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121876.log 2023-09-23T03:29:36.886 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.106470.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.106470.log.gz 2023-09-23T03:29:36.886 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124927.log 2023-09-23T03:29:36.887 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.86890.log: gzip -5 --verbose 88.0% -- replaced with /var/log/ceph/ceph-client.admin.86890.log.gz 2023-09-23T03:29:36.887 INFO:teuthology.orchestra.run.smithi190.stderr: -- /var/log/ceph/ceph-client.admin.86850.log 2023-09-23T03:29:36.887 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86930.log 2023-09-23T03:29:36.888 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.121876.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.121876.log.gz 2023-09-23T03:29:36.889 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86970.log 2023-09-23T03:29:36.889 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.124927.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.124927.log.gz 2023-09-23T03:29:36.889 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128178.log 2023-09-23T03:29:36.890 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.86970.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.86970.log.gz 2023-09-23T03:29:36.890 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102411.log 2023-09-23T03:29:36.891 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.86850.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.86850.log.gz 2023-09-23T03:29:36.891 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87130.log 2023-09-23T03:29:36.891 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.86930.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-09-23T03:29:36.892 INFO:teuthology.orchestra.run.smithi190.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.86930.log.gz 2023-09-23T03:29:36.892 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.128178.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.128178.log.gz 2023-09-23T03:29:36.893 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116384.log 2023-09-23T03:29:36.893 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.102411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98070.log 2023-09-23T03:29:36.893 INFO:teuthology.orchestra.run.smithi154.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.102411.log.gz 2023-09-23T03:29:36.894 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.116384.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.116384.log.gz 2023-09-23T03:29:36.894 INFO:teuthology.orchestra.run.smithi154.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.127947.log 2023-09-23T03:29:36.895 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-client.admin.87130.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-09-23T03:29:36.895 INFO:teuthology.orchestra.run.smithi190.stderr: 87.7%/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-09-23T03:29:36.896 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.98070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115296.log 2023-09-23T03:29:36.896 INFO:teuthology.orchestra.run.smithi154.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.98070.log.gz 2023-09-23T03:29:36.896 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.127947.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.127947.log.gz 2023-09-23T03:29:36.896 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108039.log 2023-09-23T03:29:36.897 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.115296.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.115296.log.gz 2023-09-23T03:29:36.897 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101261.log 2023-09-23T03:29:36.898 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.108039.log: 87.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.108039.log.gz 2023-09-23T03:29:36.898 INFO:teuthology.orchestra.run.smithi154.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.111288.log 2023-09-23T03:29:36.899 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-mon.b.log: 94.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-09-23T03:29:36.900 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.101261.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.101261.log.gz 2023-09-23T03:29:36.900 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110013.log 2023-09-23T03:29:36.901 INFO:teuthology.orchestra.run.smithi190.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.87130.log.gz 2023-09-23T03:29:36.901 INFO:teuthology.orchestra.run.smithi190.stderr:gzip -- replaced with /var/log/ceph/ceph.log.gz 2023-09-23T03:29:36.901 INFO:teuthology.orchestra.run.smithi190.stderr: -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-09-23T03:29:36.902 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.111288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119818.log 2023-09-23T03:29:36.903 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.110013.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.111288.log.gz 2023-09-23T03:29:36.903 INFO:teuthology.orchestra.run.smithi154.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110013.log.gz 2023-09-23T03:29:36.903 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125347.log 2023-09-23T03:29:36.903 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.119818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110321.log 2023-09-23T03:29:36.904 INFO:teuthology.orchestra.run.smithi154.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.119818.log.gz 2023-09-23T03:29:36.905 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.125347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95972.log 2023-09-23T03:29:36.905 INFO:teuthology.orchestra.run.smithi154.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.125347.log.gz 2023-09-23T03:29:36.906 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.110321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129300.log 2023-09-23T03:29:36.906 INFO:teuthology.orchestra.run.smithi154.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.110321.log.gz 2023-09-23T03:29:36.907 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.95972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117842.log 2023-09-23T03:29:36.907 INFO:teuthology.orchestra.run.smithi154.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.95972.log.gz 2023-09-23T03:29:36.908 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.129300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117507.log 2023-09-23T03:29:36.909 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.117842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125534.log 2023-09-23T03:29:36.910 INFO:teuthology.orchestra.run.smithi154.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.117842.log.gz 2023-09-23T03:29:36.910 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.117507.log: 93.1% -- replaced with /var/log/ceph/ceph-client.admin.129300.log.gz 2023-09-23T03:29:36.911 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125702.log 2023-09-23T03:29:36.912 INFO:teuthology.orchestra.run.smithi154.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.117507.log.gz 2023-09-23T03:29:36.912 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.125534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131813.log 2023-09-23T03:29:36.913 INFO:teuthology.orchestra.run.smithi154.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.125534.log.gz 2023-09-23T03:29:36.913 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.125702.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.125702.log.gz 2023-09-23T03:29:36.913 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95365.log 2023-09-23T03:29:36.915 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.131813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92025.log 2023-09-23T03:29:36.915 INFO:teuthology.orchestra.run.smithi154.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.131813.log.gz 2023-09-23T03:29:36.916 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.95365.log: gzip -5 --verbose 88.2% -- /var/log/ceph/ceph-client.admin.116342.log -- replaced with /var/log/ceph/ceph-client.admin.95365.log.gz 2023-09-23T03:29:36.916 INFO:teuthology.orchestra.run.smithi154.stderr: 2023-09-23T03:29:36.916 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.92025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113365.log 2023-09-23T03:29:36.917 INFO:teuthology.orchestra.run.smithi154.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.92025.log.gz 2023-09-23T03:29:36.918 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.116342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101177.log 2023-09-23T03:29:36.918 INFO:teuthology.orchestra.run.smithi154.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.116342.log.gz 2023-09-23T03:29:36.919 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.113365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123084.log 2023-09-23T03:29:36.919 INFO:teuthology.orchestra.run.smithi154.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.113365.log.gz 2023-09-23T03:29:36.921 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.101177.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.101177.log.gz 2023-09-23T03:29:36.921 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87083.log 2023-09-23T03:29:36.922 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.123084.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.123084.log.gz 2023-09-23T03:29:36.922 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109787.log 2023-09-23T03:29:36.923 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.87083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124289.log 2023-09-23T03:29:36.923 INFO:teuthology.orchestra.run.smithi154.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.87083.log.gz 2023-09-23T03:29:36.924 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.109787.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.109787.log.gz 2023-09-23T03:29:36.924 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.112314.log 2023-09-23T03:29:36.925 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.124289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129199.log 2023-09-23T03:29:36.926 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.0.112314.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.124289.log.gz 2023-09-23T03:29:36.926 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95049.log 2023-09-23T03:29:36.928 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.129199.log: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.129199.log.gz -5 2023-09-23T03:29:36.928 INFO:teuthology.orchestra.run.smithi154.stderr: --verbose -- /var/log/ceph/ceph-client.admin.114461.log 2023-09-23T03:29:36.929 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.95049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94721.log 2023-09-23T03:29:36.929 INFO:teuthology.orchestra.run.smithi154.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95049.log.gz 2023-09-23T03:29:36.931 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.114461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98658.log 89.2% 2023-09-23T03:29:36.931 INFO:teuthology.orchestra.run.smithi154.stderr: -- replaced with /var/log/ceph/ceph-client.admin.114461.log.gz 2023-09-23T03:29:36.932 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.94721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109027.log 2023-09-23T03:29:36.932 INFO:teuthology.orchestra.run.smithi154.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.94721.log.gz 2023-09-23T03:29:36.933 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.98658.log: gzip -5 --verbose 88.5% -- -- replaced with /var/log/ceph/ceph-client.admin.98658.log.gz 2023-09-23T03:29:36.934 INFO:teuthology.orchestra.run.smithi154.stderr: /var/log/ceph/ceph-client.admin.98921.log 2023-09-23T03:29:36.935 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.109027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110820.log 2023-09-23T03:29:36.937 INFO:teuthology.orchestra.run.smithi154.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.109027.log.gz 2023-09-23T03:29:36.938 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.98921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91819.log 2023-09-23T03:29:36.939 INFO:teuthology.orchestra.run.smithi154.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.98921.log.gz 2023-09-23T03:29:36.939 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.110820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98364.log 2023-09-23T03:29:36.939 INFO:teuthology.orchestra.run.smithi154.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.110820.log.gz 2023-09-23T03:29:36.939 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.91819.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.91819.log.gz 2023-09-23T03:29:36.939 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116426.log 2023-09-23T03:29:36.940 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.98364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130953.log 2023-09-23T03:29:36.940 INFO:teuthology.orchestra.run.smithi154.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.98364.log.gz 2023-09-23T03:29:36.942 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.116426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119230.log 2023-09-23T03:29:36.942 INFO:teuthology.orchestra.run.smithi154.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.116426.log.gz 2023-09-23T03:29:36.943 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.130953.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.130953.log.gz 2023-09-23T03:29:36.943 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120686.log 2023-09-23T03:29:36.944 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.119230.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.119230.log.gz 2023-09-23T03:29:36.945 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125786.log 2023-09-23T03:29:36.946 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.120686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114335.log 2023-09-23T03:29:36.946 INFO:teuthology.orchestra.run.smithi154.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.120686.log.gz 2023-09-23T03:29:36.947 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.125786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106764.log 2023-09-23T03:29:36.947 INFO:teuthology.orchestra.run.smithi154.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.125786.log.gz 2023-09-23T03:29:36.948 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.114335.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.114335.log.gz 2023-09-23T03:29:36.949 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86715.log 2023-09-23T03:29:36.950 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.106764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112519.log 2023-09-23T03:29:36.950 INFO:teuthology.orchestra.run.smithi154.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.106764.log.gz 2023-09-23T03:29:36.951 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.86715.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.86715.log.gz 2023-09-23T03:29:36.951 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108165.log 2023-09-23T03:29:36.953 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.112519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130659.log 2023-09-23T03:29:36.953 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.108165.log: 94.0% -- replaced with /var/log/ceph/ceph-client.0.112314.log.gz 2023-09-23T03:29:36.954 INFO:teuthology.orchestra.run.smithi154.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.112519.log.gz 2023-09-23T03:29:36.954 INFO:teuthology.orchestra.run.smithi154.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.108165.log.gz 2023-09-23T03:29:36.954 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105282.log 2023-09-23T03:29:36.955 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.130659.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.130659.log.gz 2023-09-23T03:29:36.956 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106275.log 2023-09-23T03:29:36.956 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.105282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121004.log 2023-09-23T03:29:36.957 INFO:teuthology.orchestra.run.smithi154.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.105282.log.gz 2023-09-23T03:29:36.958 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.106275.log: gzip -5 --verbose -- 87.8% /var/log/ceph/ceph-client.admin.126084.log 2023-09-23T03:29:36.958 INFO:teuthology.orchestra.run.smithi154.stderr: -- replaced with /var/log/ceph/ceph-client.admin.106275.log.gz 2023-09-23T03:29:36.959 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.121004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119902.log 2023-09-23T03:29:36.959 INFO:teuthology.orchestra.run.smithi154.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.121004.log.gz 2023-09-23T03:29:36.960 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.126084.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.126084.log.gz 2023-09-23T03:29:36.960 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123216.log 2023-09-23T03:29:36.961 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.119902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106191.log 2023-09-23T03:29:36.961 INFO:teuthology.orchestra.run.smithi154.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.119902.log.gz 2023-09-23T03:29:36.962 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.123216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111303.log 2023-09-23T03:29:36.963 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.106191.log: gzip -5 --verbose -- 88.9% /var/log/ceph/ceph-client.admin.117884.log -- replaced with /var/log/ceph/ceph-client.admin.106191.log.gz 2023-09-23T03:29:36.963 INFO:teuthology.orchestra.run.smithi154.stderr: 2023-09-23T03:29:36.964 INFO:teuthology.orchestra.run.smithi154.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.123216.log.gz 2023-09-23T03:29:36.965 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.111303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129501.log 2023-09-23T03:29:36.966 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.117884.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.111303.log.gz 2023-09-23T03:29:36.966 INFO:teuthology.orchestra.run.smithi154.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.117884.log.gz 2023-09-23T03:29:36.966 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107305.log 2023-09-23T03:29:36.967 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.129501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100217.log 2023-09-23T03:29:36.967 INFO:teuthology.orchestra.run.smithi154.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.129501.log.gz 2023-09-23T03:29:36.968 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.107305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130869.log 2023-09-23T03:29:36.968 INFO:teuthology.orchestra.run.smithi154.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.107305.log.gz 2023-09-23T03:29:36.969 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.100217.log: gzip -5 --verbose 88.7% -- -- replaced with /var/log/ceph/ceph-client.admin.100217.log.gz /var/log/ceph/ceph-client.admin.127352.log 2023-09-23T03:29:36.969 INFO:teuthology.orchestra.run.smithi154.stderr: 2023-09-23T03:29:36.971 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.130869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95048.log 2023-09-23T03:29:36.971 INFO:teuthology.orchestra.run.smithi154.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.130869.log.gz 2023-09-23T03:29:36.972 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.127352.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.127352.log.gz 2023-09-23T03:29:36.972 INFO:teuthology.orchestra.run.smithi154.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.118741.log 2023-09-23T03:29:36.973 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.95048.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95048.log.gz 2023-09-23T03:29:36.973 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126569.log 2023-09-23T03:29:36.974 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.118741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101513.log 2023-09-23T03:29:36.974 INFO:teuthology.orchestra.run.smithi154.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.118741.log.gz 2023-09-23T03:29:36.975 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.126569.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.126569.log.gz 2023-09-23T03:29:36.975 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129073.log 2023-09-23T03:29:36.976 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.101513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129595.log 2023-09-23T03:29:36.977 INFO:teuthology.orchestra.run.smithi154.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.101513.log.gz 2023-09-23T03:29:36.977 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.129073.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.129073.log.gz 2023-09-23T03:29:36.978 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122441.log 2023-09-23T03:29:36.979 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.129595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93944.log 2023-09-23T03:29:36.979 INFO:teuthology.orchestra.run.smithi154.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.129595.log.gz 2023-09-23T03:29:36.980 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.122441.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.122441.log.gz 2023-09-23T03:29:36.980 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116931.log 2023-09-23T03:29:36.981 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.93944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102223.log 2023-09-23T03:29:36.981 INFO:teuthology.orchestra.run.smithi154.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.93944.log.gz 2023-09-23T03:29:36.982 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.116931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103636.log 2023-09-23T03:29:36.982 INFO:teuthology.orchestra.run.smithi154.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.116931.log.gz 2023-09-23T03:29:36.983 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.102223.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.102223.log.gz 2023-09-23T03:29:36.984 INFO:teuthology.orchestra.run.smithi154.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.105740.log 2023-09-23T03:29:36.985 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.103636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120201.log 2023-09-23T03:29:36.985 INFO:teuthology.orchestra.run.smithi154.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.103636.log.gz 2023-09-23T03:29:36.986 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.105740.log: gzip 91.1% -- replaced with /var/log/ceph/ceph-client.admin.105740.log.gz -5 2023-09-23T03:29:36.987 INFO:teuthology.orchestra.run.smithi154.stderr: --verbose -- /var/log/ceph/ceph-client.admin.124648.log 2023-09-23T03:29:36.987 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.120201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115212.log 2023-09-23T03:29:36.987 INFO:teuthology.orchestra.run.smithi154.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.120201.log.gz 2023-09-23T03:29:36.988 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.124648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97860.log 2023-09-23T03:29:36.989 INFO:teuthology.orchestra.run.smithi154.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.124648.log.gz 2023-09-23T03:29:36.989 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.115212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107263.log 2023-09-23T03:29:36.990 INFO:teuthology.orchestra.run.smithi154.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.115212.log.gz 2023-09-23T03:29:36.991 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.97860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119860.log 2023-09-23T03:29:36.991 INFO:teuthology.orchestra.run.smithi154.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.97860.log.gz 2023-09-23T03:29:36.992 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.107263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.103090.log 2023-09-23T03:29:36.992 INFO:teuthology.orchestra.run.smithi154.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.107263.log.gz 2023-09-23T03:29:36.993 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.119860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96056.log 2023-09-23T03:29:36.993 INFO:teuthology.orchestra.run.smithi154.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.119860.log.gz 2023-09-23T03:29:36.994 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.0.103090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100033.log 2023-09-23T03:29:36.995 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.96056.log: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.96056.log.gz 2023-09-23T03:29:36.995 INFO:teuthology.orchestra.run.smithi154.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.118034.log 2023-09-23T03:29:36.996 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.100033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97776.log 2023-09-23T03:29:36.996 INFO:teuthology.orchestra.run.smithi154.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100033.log.gz 2023-09-23T03:29:36.997 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.118034.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.118034.log.gz 2023-09-23T03:29:36.998 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100547.log 2023-09-23T03:29:36.999 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.97776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114419.log 2023-09-23T03:29:36.999 INFO:teuthology.orchestra.run.smithi154.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.97776.log.gz 2023-09-23T03:29:37.000 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.100547.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.100547.log.gz 2023-09-23T03:29:37.000 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102179.log 2023-09-23T03:29:37.002 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.114419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94292.log 2023-09-23T03:29:37.002 INFO:teuthology.orchestra.run.smithi154.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.114419.log.gz 2023-09-23T03:29:37.003 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.102179.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.102179.log.gz 2023-09-23T03:29:37.003 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95042.log 2023-09-23T03:29:37.004 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.94292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110713.log 2023-09-23T03:29:37.004 INFO:teuthology.orchestra.run.smithi154.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94292.log.gz 2023-09-23T03:29:37.006 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.95042.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.95042.log.gz 2023-09-23T03:29:37.006 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122483.log 2023-09-23T03:29:37.007 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.110713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100117.log 2023-09-23T03:29:37.007 INFO:teuthology.orchestra.run.smithi154.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.110713.log.gz 2023-09-23T03:29:37.008 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.122483.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.122483.log.gz 2023-09-23T03:29:37.008 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100673.log 2023-09-23T03:29:37.009 INFO:teuthology.orchestra.run.smithi154.stderr: 94.5% -- replaced with /var/log/ceph/ceph-client.0.103090.log.gz 2023-09-23T03:29:37.010 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.100117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122273.log 2023-09-23T03:29:37.010 INFO:teuthology.orchestra.run.smithi154.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100117.log.gz 2023-09-23T03:29:37.011 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.100673.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100673.log.gz 2023-09-23T03:29:37.011 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114886.log 2023-09-23T03:29:37.013 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.122273.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.122273.log.gz 2023-09-23T03:29:37.013 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119314.log 2023-09-23T03:29:37.014 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.114886.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.114886.log.gz 2023-09-23T03:29:37.014 INFO:teuthology.orchestra.run.smithi154.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.103820.log 2023-09-23T03:29:37.015 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.119314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106806.log 2023-09-23T03:29:37.015 INFO:teuthology.orchestra.run.smithi154.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.119314.log.gz 2023-09-23T03:29:37.016 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.103820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106554.log 2023-09-23T03:29:37.016 INFO:teuthology.orchestra.run.smithi154.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.103820.log.gz 2023-09-23T03:29:37.017 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.106806.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.106806.log.gz 2023-09-23T03:29:37.017 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96182.log 2023-09-23T03:29:37.018 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.106554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97818.log 2023-09-23T03:29:37.018 INFO:teuthology.orchestra.run.smithi154.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106554.log.gz 2023-09-23T03:29:37.019 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.96182.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.96182.log.gz 2023-09-23T03:29:37.019 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109913.log 2023-09-23T03:29:37.020 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.97818.log: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.97818.log.gz 2023-09-23T03:29:37.021 INFO:teuthology.orchestra.run.smithi154.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.107442.log 2023-09-23T03:29:37.022 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.109913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110755.log 2023-09-23T03:29:37.022 INFO:teuthology.orchestra.run.smithi154.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.109913.log.gz 2023-09-23T03:29:37.023 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.107442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104229.log 2023-09-23T03:29:37.023 INFO:teuthology.orchestra.run.smithi154.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.107442.log.gz 2023-09-23T03:29:37.024 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.110755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111825.log 2023-09-23T03:29:37.024 INFO:teuthology.orchestra.run.smithi154.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110755.log.gz 2023-09-23T03:29:37.025 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.104229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114556.log 2023-09-23T03:29:37.025 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.111825.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.104229.log.gz 2023-09-23T03:29:37.026 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95397.log 2023-09-23T03:29:37.026 INFO:teuthology.orchestra.run.smithi154.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.111825.log.gz 2023-09-23T03:29:37.028 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.114556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116987.log 2023-09-23T03:29:37.028 INFO:teuthology.orchestra.run.smithi154.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.114556.log.gz 2023-09-23T03:29:37.028 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.95397.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.95397.log.gz 2023-09-23T03:29:37.029 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98280.log 2023-09-23T03:29:37.030 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.116987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129360.log 2023-09-23T03:29:37.030 INFO:teuthology.orchestra.run.smithi154.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.116987.log.gz 2023-09-23T03:29:37.031 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.98280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131687.log 2023-09-23T03:29:37.031 INFO:teuthology.orchestra.run.smithi154.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98280.log.gz 2023-09-23T03:29:37.032 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.129360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124774.log 2023-09-23T03:29:37.033 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.131687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130557.log 2023-09-23T03:29:37.034 INFO:teuthology.orchestra.run.smithi154.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.131687.log.gz 2023-09-23T03:29:37.035 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.124774.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.124774.log.gz 2023-09-23T03:29:37.035 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99603.log 2023-09-23T03:29:37.036 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.130557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119440.log 2023-09-23T03:29:37.036 INFO:teuthology.orchestra.run.smithi154.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.130557.log.gz 2023-09-23T03:29:37.038 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.99603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95930.log 2023-09-23T03:29:37.038 INFO:teuthology.orchestra.run.smithi154.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.99603.log.gz 2023-09-23T03:29:37.039 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.119440.log: 89.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.119440.log.gz 2023-09-23T03:29:37.039 INFO:teuthology.orchestra.run.smithi154.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.100757.log 2023-09-23T03:29:37.040 INFO:teuthology.orchestra.run.smithi154.stderr: 93.4% -- replaced with /var/log/ceph/ceph-client.admin.129360.log.gz 2023-09-23T03:29:37.040 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.95930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114844.log 2023-09-23T03:29:37.040 INFO:teuthology.orchestra.run.smithi154.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.95930.log.gz 2023-09-23T03:29:37.041 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.100757.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.100757.log.gz 2023-09-23T03:29:37.041 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114717.log 2023-09-23T03:29:37.042 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.114844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102621.log 2023-09-23T03:29:37.043 INFO:teuthology.orchestra.run.smithi154.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.114844.log.gz 2023-09-23T03:29:37.044 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.114717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115152.log 2023-09-23T03:29:37.044 INFO:teuthology.orchestra.run.smithi154.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.114717.log.gz 2023-09-23T03:29:37.045 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.102621.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102621.log.gz 2023-09-23T03:29:37.045 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113560.log 2023-09-23T03:29:37.046 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.115152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103315.log 2023-09-23T03:29:37.046 INFO:teuthology.orchestra.run.smithi154.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.115152.log.gz 2023-09-23T03:29:37.047 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.113560.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.113560.log.gz 2023-09-23T03:29:37.047 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101387.log 2023-09-23T03:29:37.048 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.103315.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.103315.log.gz 2023-09-23T03:29:37.048 INFO:teuthology.orchestra.run.smithi154.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.106233.log 2023-09-23T03:29:37.049 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.101387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108459.log 2023-09-23T03:29:37.049 INFO:teuthology.orchestra.run.smithi154.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.101387.log.gz 2023-09-23T03:29:37.050 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.106233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126485.log 2023-09-23T03:29:37.050 INFO:teuthology.orchestra.run.smithi154.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.106233.log.gz 2023-09-23T03:29:37.052 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.108459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101135.log 2023-09-23T03:29:37.052 INFO:teuthology.orchestra.run.smithi154.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.108459.log.gz 2023-09-23T03:29:37.052 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.126485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117375.log 2023-09-23T03:29:37.053 INFO:teuthology.orchestra.run.smithi154.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.126485.log.gz 2023-09-23T03:29:37.054 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.101135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125407.log 2023-09-23T03:29:37.054 INFO:teuthology.orchestra.run.smithi154.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101135.log.gz 2023-09-23T03:29:37.055 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.117375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126845.log 2023-09-23T03:29:37.055 INFO:teuthology.orchestra.run.smithi154.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.117375.log.gz 2023-09-23T03:29:37.056 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.125407.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.125407.log.gz 2023-09-23T03:29:37.056 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92785.log 2023-09-23T03:29:37.057 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.126845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101597.log 2023-09-23T03:29:37.057 INFO:teuthology.orchestra.run.smithi154.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.126845.log.gz 2023-09-23T03:29:37.058 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.92785.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.92785.log.gz 2023-09-23T03:29:37.058 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100301.log 2023-09-23T03:29:37.059 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.101597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107773.log 2023-09-23T03:29:37.059 INFO:teuthology.orchestra.run.smithi154.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.101597.log.gz 2023-09-23T03:29:37.060 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.100301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119062.log 2023-09-23T03:29:37.060 INFO:teuthology.orchestra.run.smithi154.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.100301.log.gz 2023-09-23T03:29:37.061 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.107773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107137.log 2023-09-23T03:29:37.062 INFO:teuthology.orchestra.run.smithi154.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107773.log.gz 2023-09-23T03:29:37.062 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.119062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125961.log 2023-09-23T03:29:37.063 INFO:teuthology.orchestra.run.smithi154.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.119062.log.gz 2023-09-23T03:29:37.064 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.107137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113602.log 2023-09-23T03:29:37.064 INFO:teuthology.orchestra.run.smithi154.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107137.log.gz 2023-09-23T03:29:37.064 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.125961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98196.log 2023-09-23T03:29:37.065 INFO:teuthology.orchestra.run.smithi154.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.125961.log.gz 2023-09-23T03:29:37.066 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.113602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123126.log 2023-09-23T03:29:37.066 INFO:teuthology.orchestra.run.smithi154.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.113602.log.gz 2023-09-23T03:29:37.067 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.98196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117291.log 2023-09-23T03:29:37.067 INFO:teuthology.orchestra.run.smithi154.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.98196.log.gz 2023-09-23T03:29:37.068 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.123126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101927.log 2023-09-23T03:29:37.069 INFO:teuthology.orchestra.run.smithi154.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.123126.log.gz 2023-09-23T03:29:37.070 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.117291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107221.log 2023-09-23T03:29:37.070 INFO:teuthology.orchestra.run.smithi154.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.117291.log.gz 2023-09-23T03:29:37.071 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.101927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94208.log 2023-09-23T03:29:37.071 INFO:teuthology.orchestra.run.smithi154.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101927.log.gz 2023-09-23T03:29:37.072 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.107221.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.107221.log.gz 2023-09-23T03:29:37.072 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99189.log 2023-09-23T03:29:37.073 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.94208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94726.log 2023-09-23T03:29:37.073 INFO:teuthology.orchestra.run.smithi154.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.94208.log.gz 2023-09-23T03:29:37.074 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.99189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116610.log 2023-09-23T03:29:37.075 INFO:teuthology.orchestra.run.smithi154.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99189.log.gz 2023-09-23T03:29:37.075 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.94726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104762.log 2023-09-23T03:29:37.076 INFO:teuthology.orchestra.run.smithi154.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94726.log.gz 2023-09-23T03:29:37.076 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.116610.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.116610.log.gz 2023-09-23T03:29:37.077 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93818.log 2023-09-23T03:29:37.078 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.104762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113323.log 2023-09-23T03:29:37.078 INFO:teuthology.orchestra.run.smithi154.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.104762.log.gz 2023-09-23T03:29:37.079 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.93818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126359.log 2023-09-23T03:29:37.079 INFO:teuthology.orchestra.run.smithi154.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.93818.log.gz 2023-09-23T03:29:37.081 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.113323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118867.log 2023-09-23T03:29:37.081 INFO:teuthology.orchestra.run.smithi154.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.113323.log.gz 2023-09-23T03:29:37.081 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.126359.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.126359.log.gz 2023-09-23T03:29:37.081 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113449.log 2023-09-23T03:29:37.083 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.118867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108417.log 2023-09-23T03:29:37.083 INFO:teuthology.orchestra.run.smithi154.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.118867.log.gz 2023-09-23T03:29:37.084 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.113449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131424.log 2023-09-23T03:29:37.084 INFO:teuthology.orchestra.run.smithi154.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.113449.log.gz 2023-09-23T03:29:37.085 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.108417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125263.log 2023-09-23T03:29:37.085 INFO:teuthology.orchestra.run.smithi154.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108417.log.gz 2023-09-23T03:29:37.086 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.131424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120602.log 2023-09-23T03:29:37.086 INFO:teuthology.orchestra.run.smithi154.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.131424.log.gz 2023-09-23T03:29:37.087 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.125263.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.125263.log.gz 2023-09-23T03:29:37.087 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103678.log 2023-09-23T03:29:37.089 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.120602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101219.log 2023-09-23T03:29:37.089 INFO:teuthology.orchestra.run.smithi154.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.120602.log.gz 2023-09-23T03:29:37.090 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.103678.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.103678.log.gz 2023-09-23T03:29:37.090 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92447.log 2023-09-23T03:29:37.091 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.101219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117652.log 2023-09-23T03:29:37.091 INFO:teuthology.orchestra.run.smithi154.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101219.log.gz 2023-09-23T03:29:37.092 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.92447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110418.log 2023-09-23T03:29:37.092 INFO:teuthology.orchestra.run.smithi154.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.92447.log.gz 2023-09-23T03:29:37.093 INFO:teuthology.orchestra.run.smithi190.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mon.c.log: 93.6% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-09-23T03:29:37.093 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.117652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120387.log 2023-09-23T03:29:37.093 INFO:teuthology.orchestra.run.smithi154.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.117652.log.gz 2023-09-23T03:29:37.094 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.110418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120938.log 2023-09-23T03:29:37.095 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.120387.log: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.110418.log.gz 2023-09-23T03:29:37.095 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124564.log 2023-09-23T03:29:37.095 INFO:teuthology.orchestra.run.smithi154.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.120387.log.gz 2023-09-23T03:29:37.097 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.120938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122045.log 2023-09-23T03:29:37.097 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.124564.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.120938.log.gz 2023-09-23T03:29:37.098 INFO:teuthology.orchestra.run.smithi154.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.124564.log.gz 2023-09-23T03:29:37.098 INFO:teuthology.orchestra.run.smithi190.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-09-23T03:29:37.098 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93860.log 2023-09-23T03:29:37.099 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.122045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97650.log 2023-09-23T03:29:37.099 INFO:teuthology.orchestra.run.smithi154.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.122045.log.gz 2023-09-23T03:29:37.100 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.93860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130515.log 2023-09-23T03:29:37.100 INFO:teuthology.orchestra.run.smithi154.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.93860.log.gz 2023-09-23T03:29:37.101 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.97650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94124.log 2023-09-23T03:29:37.101 INFO:teuthology.orchestra.run.smithi154.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.97650.log.gz 2023-09-23T03:29:37.102 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.130515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104804.log 2023-09-23T03:29:37.102 INFO:teuthology.orchestra.run.smithi154.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.130515.log.gz 2023-09-23T03:29:37.103 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.94124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128092.log 2023-09-23T03:29:37.103 INFO:teuthology.orchestra.run.smithi154.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.94124.log.gz 2023-09-23T03:29:37.105 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.104804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100589.log 2023-09-23T03:29:37.105 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.128092.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.128092.log.gz 2023-09-23T03:29:37.106 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111290.log 2023-09-23T03:29:37.106 INFO:teuthology.orchestra.run.smithi154.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.104804.log.gz 2023-09-23T03:29:37.107 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.100589.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100589.log.gz 2023-09-23T03:29:37.107 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107731.log 2023-09-23T03:29:37.108 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.111290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92361.log 2023-09-23T03:29:37.109 INFO:teuthology.orchestra.run.smithi154.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.111290.log.gz 2023-09-23T03:29:37.109 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.107731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120770.log 2023-09-23T03:29:37.109 INFO:teuthology.orchestra.run.smithi154.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.107731.log.gz 2023-09-23T03:29:37.111 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.92361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98616.log 2023-09-23T03:29:37.111 INFO:teuthology.orchestra.run.smithi154.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.92361.log.gz 2023-09-23T03:29:37.112 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.120770.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.120770.log.gz 2023-09-23T03:29:37.112 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115110.log 2023-09-23T03:29:37.113 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.98616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125137.log 2023-09-23T03:29:37.113 INFO:teuthology.orchestra.run.smithi154.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.98616.log.gz 2023-09-23T03:29:37.114 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.115110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97438.log 2023-09-23T03:29:37.114 INFO:teuthology.orchestra.run.smithi154.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.115110.log.gz 2023-09-23T03:29:37.115 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.125137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127924.log 2023-09-23T03:29:37.115 INFO:teuthology.orchestra.run.smithi154.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.125137.log.gz 2023-09-23T03:29:37.116 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.97438.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.97438.log.gz 2023-09-23T03:29:37.117 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100967.log 2023-09-23T03:29:37.118 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.127924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131519.log 2023-09-23T03:29:37.118 INFO:teuthology.orchestra.run.smithi154.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.127924.log.gz 2023-09-23T03:29:37.119 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.100967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107035.log 2023-09-23T03:29:37.119 INFO:teuthology.orchestra.run.smithi154.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100967.log.gz 2023-09-23T03:29:37.120 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.131519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98837.log 2023-09-23T03:29:37.120 INFO:teuthology.orchestra.run.smithi154.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.131519.log.gz 2023-09-23T03:29:37.121 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.107035.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.107035.log.gz 2023-09-23T03:29:37.121 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114759.log 2023-09-23T03:29:37.122 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.98837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116834.log 2023-09-23T03:29:37.122 INFO:teuthology.orchestra.run.smithi154.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98837.log.gz 2023-09-23T03:29:37.123 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.114759.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.114759.log.gz 2023-09-23T03:29:37.123 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.108844.log 2023-09-23T03:29:37.124 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.116834.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.116834.log.gz 2023-09-23T03:29:37.124 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132023.log 2023-09-23T03:29:37.125 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.0.108844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96224.log 2023-09-23T03:29:37.126 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.132023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108207.log 2023-09-23T03:29:37.127 INFO:teuthology.orchestra.run.smithi154.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.132023.log.gz 2023-09-23T03:29:37.128 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.96224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129115.log 2023-09-23T03:29:37.128 INFO:teuthology.orchestra.run.smithi154.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.96224.log.gz 2023-09-23T03:29:37.129 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.108207.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.108207.log.gz 2023-09-23T03:29:37.129 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124690.log 2023-09-23T03:29:37.130 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.129115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103229.log 2023-09-23T03:29:37.131 INFO:teuthology.orchestra.run.smithi154.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.129115.log.gz 2023-09-23T03:29:37.131 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.124690.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.124690.log.gz 2023-09-23T03:29:37.132 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117142.log 2023-09-23T03:29:37.133 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.103229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104030.log 2023-09-23T03:29:37.133 INFO:teuthology.orchestra.run.smithi154.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103229.log.gz 2023-09-23T03:29:37.134 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.117142.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.117142.log.gz 2023-09-23T03:29:37.134 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102663.log 2023-09-23T03:29:37.135 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.104030.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.104030.log.gz 2023-09-23T03:29:37.136 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92743.log 2023-09-23T03:29:37.136 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.102663.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.102663.log.gz 2023-09-23T03:29:37.137 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122851.log 2023-09-23T03:29:37.138 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.92743.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.92743.log.gz 2023-09-23T03:29:37.138 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125179.log 2023-09-23T03:29:37.139 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.122851.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.122851.log.gz 2023-09-23T03:29:37.139 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119482.log 2023-09-23T03:29:37.141 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.125179.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.125179.log.gz 2023-09-23T03:29:37.141 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96098.log 2023-09-23T03:29:37.142 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.119482.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.119482.log.gz 2023-09-23T03:29:37.142 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105174.log 2023-09-23T03:29:37.143 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.96098.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96098.log.gz -5 2023-09-23T03:29:37.143 INFO:teuthology.orchestra.run.smithi154.stderr: --verbose -- /var/log/ceph/ceph-client.admin.127762.log 2023-09-23T03:29:37.144 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.105174.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.105174.log.gz 2023-09-23T03:29:37.144 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125744.log 2023-09-23T03:29:37.146 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.127762.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.127762.log.gz 2023-09-23T03:29:37.146 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120644.log 2023-09-23T03:29:37.147 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.125744.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.125744.log.gz 2023-09-23T03:29:37.147 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110952.log 2023-09-23T03:29:37.148 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.120644.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.120644.log.gz 2023-09-23T03:29:37.149 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114167.log 2023-09-23T03:29:37.150 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.110952.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.110952.log.gz 2023-09-23T03:29:37.150 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123174.log 2023-09-23T03:29:37.151 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.114167.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.114167.log.gz 2023-09-23T03:29:37.151 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109871.log 2023-09-23T03:29:37.152 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.123174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110994.log 2023-09-23T03:29:37.152 INFO:teuthology.orchestra.run.smithi154.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.123174.log.gz 2023-09-23T03:29:37.153 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.109871.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.109871.log.gz 2023-09-23T03:29:37.154 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125492.log 2023-09-23T03:29:37.155 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.110994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123545.log 2023-09-23T03:29:37.155 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.125492.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.110994.log.gz 2023-09-23T03:29:37.156 INFO:teuthology.orchestra.run.smithi154.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.125492.log.gz 2023-09-23T03:29:37.156 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108333.log 2023-09-23T03:29:37.157 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.123545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129639.log 2023-09-23T03:29:37.158 INFO:teuthology.orchestra.run.smithi154.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.123545.log.gz 2023-09-23T03:29:37.158 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.108333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102267.log 2023-09-23T03:29:37.159 INFO:teuthology.orchestra.run.smithi154.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.108333.log.gz 2023-09-23T03:29:37.159 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.129639.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.129639.log.gz 2023-09-23T03:29:37.160 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131897.log 2023-09-23T03:29:37.161 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.102267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104354.log 2023-09-23T03:29:37.161 INFO:teuthology.orchestra.run.smithi154.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.102267.log.gz 2023-09-23T03:29:37.162 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.131897.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.131897.log.gz 2023-09-23T03:29:37.162 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-09-23T03:29:37.163 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.104354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108291.log 2023-09-23T03:29:37.163 INFO:teuthology.orchestra.run.smithi154.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.104354.log.gz 2023-09-23T03:29:37.164 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104481.log 2023-09-23T03:29:37.165 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.108291.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.108291.log.gz 2023-09-23T03:29:37.166 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104397.log 2023-09-23T03:29:37.167 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132107.log 2023-09-23T03:29:37.167 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.104481.log: 89.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-09-23T03:29:37.168 INFO:teuthology.orchestra.run.smithi154.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.104481.log.gz 2023-09-23T03:29:37.168 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.104397.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.104397.log.gz 2023-09-23T03:29:37.168 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106950.log 2023-09-23T03:29:37.169 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.132107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129031.log 2023-09-23T03:29:37.169 INFO:teuthology.orchestra.run.smithi154.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.132107.log.gz 2023-09-23T03:29:37.170 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.106950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117992.log 2023-09-23T03:29:37.171 INFO:teuthology.orchestra.run.smithi154.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.106950.log.gz 2023-09-23T03:29:37.172 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.129031.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.129031.log.gz 2023-09-23T03:29:37.172 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130278.log 2023-09-23T03:29:37.173 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.117992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103273.log 2023-09-23T03:29:37.173 INFO:teuthology.orchestra.run.smithi154.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.117992.log.gz 2023-09-23T03:29:37.174 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.130278.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.130278.log.gz 2023-09-23T03:29:37.174 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92659.log 2023-09-23T03:29:37.175 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.103273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115066.log 2023-09-23T03:29:37.175 INFO:teuthology.orchestra.run.smithi154.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.103273.log.gz 2023-09-23T03:29:37.176 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.92659.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.92659.log.gz 2023-09-23T03:29:37.176 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98154.log 2023-09-23T03:29:37.177 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.115066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98490.log 2023-09-23T03:29:37.178 INFO:teuthology.orchestra.run.smithi154.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.115066.log.gz 2023-09-23T03:29:37.182 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.98154.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.98154.log.gz 2023-09-23T03:29:37.183 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110778.log 2023-09-23T03:29:37.183 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.98490.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.98490.log.gz 2023-09-23T03:29:37.183 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132149.log 2023-09-23T03:29:37.183 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.110778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128889.log 2023-09-23T03:29:37.183 INFO:teuthology.orchestra.run.smithi154.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.110778.log.gz 2023-09-23T03:29:37.184 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.132149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99814.log 2023-09-23T03:29:37.184 INFO:teuthology.orchestra.run.smithi154.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.132149.log.gz 2023-09-23T03:29:37.185 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.128889.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.128889.log.gz 2023-09-23T03:29:37.185 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107522.log 2023-09-23T03:29:37.185 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.99814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126611.log 2023-09-23T03:29:37.185 INFO:teuthology.orchestra.run.smithi154.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.99814.log.gz 2023-09-23T03:29:37.186 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.107522.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.107522.log.gz 2023-09-23T03:29:37.186 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123042.log 2023-09-23T03:29:37.187 INFO:teuthology.orchestra.run.smithi154.stderr: 94.7% -- replaced with /var/log/ceph/ceph-client.0.108844.log.gz 2023-09-23T03:29:37.187 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.126611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114082.log 2023-09-23T03:29:37.187 INFO:teuthology.orchestra.run.smithi154.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.126611.log.gz 2023-09-23T03:29:37.188 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.123042.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.123042.log.gz 2023-09-23T03:29:37.188 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103552.log 2023-09-23T03:29:37.190 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.114082.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.114082.log.gz 2023-09-23T03:29:37.190 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127846.log 2023-09-23T03:29:37.191 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.103552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128040.log 2023-09-23T03:29:37.191 INFO:teuthology.orchestra.run.smithi154.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103552.log.gz 2023-09-23T03:29:37.192 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.127846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98963.log 2023-09-23T03:29:37.192 INFO:teuthology.orchestra.run.smithi154.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.127846.log.gz 2023-09-23T03:29:37.193 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.128040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102813.log 2023-09-23T03:29:37.193 INFO:teuthology.orchestra.run.smithi154.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.128040.log.gz 2023-09-23T03:29:37.193 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.98963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123361.log 2023-09-23T03:29:37.194 INFO:teuthology.orchestra.run.smithi154.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.98963.log.gz 2023-09-23T03:29:37.194 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.102813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126401.log 2023-09-23T03:29:37.194 INFO:teuthology.orchestra.run.smithi154.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.102813.log.gz 2023-09-23T03:29:37.195 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.123361.log: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.123361.log.gz -5 2023-09-23T03:29:37.195 INFO:teuthology.orchestra.run.smithi154.stderr: --verbose -- /var/log/ceph/ceph-client.admin.110139.log 2023-09-23T03:29:37.196 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.126401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117760.log 2023-09-23T03:29:37.197 INFO:teuthology.orchestra.run.smithi154.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.126401.log.gz 2023-09-23T03:29:37.197 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.110139.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.110139.log.gz 2023-09-23T03:29:37.197 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95058.log 2023-09-23T03:29:37.198 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.117760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95846.log 2023-09-23T03:29:37.198 INFO:teuthology.orchestra.run.smithi154.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.117760.log.gz 2023-09-23T03:29:37.199 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.95058.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.95058.log.gz 2023-09-23T03:29:37.199 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130701.log 2023-09-23T03:29:37.200 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.95846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98700.log 88.4% 2023-09-23T03:29:37.200 INFO:teuthology.orchestra.run.smithi154.stderr: -- replaced with /var/log/ceph/ceph-client.admin.95846.log.gz 2023-09-23T03:29:37.201 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.130701.log: gzip -5 89.1% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.130701.log.gz 2023-09-23T03:29:37.201 INFO:teuthology.orchestra.run.smithi154.stderr: /var/log/ceph/ceph-client.admin.127984.log 2023-09-23T03:29:37.202 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.98700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111021.log 2023-09-23T03:29:37.202 INFO:teuthology.orchestra.run.smithi154.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.98700.log.gz 2023-09-23T03:29:37.203 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.127984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131981.log 2023-09-23T03:29:37.203 INFO:teuthology.orchestra.run.smithi154.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.127984.log.gz 2023-09-23T03:29:37.204 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.111021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108567.log 2023-09-23T03:29:37.205 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.131981.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.111021.log.gz 2023-09-23T03:29:37.205 INFO:teuthology.orchestra.run.smithi154.stderr:gzip 88.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.131981.log.gz --verbose 2023-09-23T03:29:37.206 INFO:teuthology.orchestra.run.smithi154.stderr: -- /var/log/ceph/ceph-client.admin.104565.log 2023-09-23T03:29:37.207 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.108567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122697.log 2023-09-23T03:29:37.207 INFO:teuthology.orchestra.run.smithi154.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.108567.log.gz 2023-09-23T03:29:37.207 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.104565.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.104565.log.gz 2023-09-23T03:29:37.208 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122977.log 2023-09-23T03:29:37.209 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.122697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98028.log 2023-09-23T03:29:37.209 INFO:teuthology.orchestra.run.smithi154.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.122697.log.gz 2023-09-23T03:29:37.210 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.122977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118951.log 2023-09-23T03:29:37.210 INFO:teuthology.orchestra.run.smithi154.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.122977.log.gz 2023-09-23T03:29:37.211 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.98028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105038.log 2023-09-23T03:29:37.211 INFO:teuthology.orchestra.run.smithi154.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.98028.log.gz 2023-09-23T03:29:37.212 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.118951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132293.log 2023-09-23T03:29:37.212 INFO:teuthology.orchestra.run.smithi154.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.118951.log.gz 2023-09-23T03:29:37.213 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.105038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106596.log 2023-09-23T03:29:37.213 INFO:teuthology.orchestra.run.smithi154.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.105038.log.gz 2023-09-23T03:29:37.214 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.132293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100463.log 2023-09-23T03:29:37.214 INFO:teuthology.orchestra.run.smithi154.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.132293.log.gz 2023-09-23T03:29:37.214 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.106596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92893.log 2023-09-23T03:29:37.215 INFO:teuthology.orchestra.run.smithi154.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.106596.log.gz 2023-09-23T03:29:37.216 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.100463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110544.log 2023-09-23T03:29:37.216 INFO:teuthology.orchestra.run.smithi154.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.100463.log.gz 2023-09-23T03:29:37.217 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.92893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131140.log 2023-09-23T03:29:37.217 INFO:teuthology.orchestra.run.smithi154.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.92893.log.gz 2023-09-23T03:29:37.218 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.110544.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110544.log.gz 2023-09-23T03:29:37.218 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.118311.log 2023-09-23T03:29:37.219 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.131140.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.131140.log.gz 2023-09-23T03:29:37.219 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131339.log 2023-09-23T03:29:37.220 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.0.118311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99147.log 2023-09-23T03:29:37.221 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.131339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121987.log 2023-09-23T03:29:37.221 INFO:teuthology.orchestra.run.smithi154.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.131339.log.gz 2023-09-23T03:29:37.222 INFO:teuthology.orchestra.run.smithi154.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.0.118311.log.gz 2023-09-23T03:29:37.222 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.99147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95059.log 2023-09-23T03:29:37.222 INFO:teuthology.orchestra.run.smithi154.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.99147.log.gz 2023-09-23T03:29:37.223 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.121987.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.121987.log.gz 2023-09-23T03:29:37.223 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107995.log 2023-09-23T03:29:37.225 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.95059.log: gzip -5 --verbose -- 88.5% /var/log/ceph/ceph-client.admin.109745.log 2023-09-23T03:29:37.225 INFO:teuthology.orchestra.run.smithi154.stderr: -- replaced with /var/log/ceph/ceph-client.admin.95059.log.gz 2023-09-23T03:29:37.225 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.107995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120854.log 2023-09-23T03:29:37.225 INFO:teuthology.orchestra.run.smithi154.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107995.log.gz 2023-09-23T03:29:37.226 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.109745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103988.log 2023-09-23T03:29:37.226 INFO:teuthology.orchestra.run.smithi154.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.109745.log.gz 2023-09-23T03:29:37.227 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.120854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129445.log 88.5% 2023-09-23T03:29:37.227 INFO:teuthology.orchestra.run.smithi154.stderr: -- replaced with /var/log/ceph/ceph-client.admin.120854.log.gz 2023-09-23T03:29:37.228 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.103988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127578.log 2023-09-23T03:29:37.228 INFO:teuthology.orchestra.run.smithi154.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.103988.log.gz 2023-09-23T03:29:37.230 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.129445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105890.log 2023-09-23T03:29:37.230 INFO:teuthology.orchestra.run.smithi154.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.129445.log.gz 2023-09-23T03:29:37.230 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.127578.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.127578.log.gz 2023-09-23T03:29:37.231 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122173.log 2023-09-23T03:29:37.232 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.105890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116694.log 2023-09-23T03:29:37.233 INFO:teuthology.orchestra.run.smithi154.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.105890.log.gz 2023-09-23T03:29:37.234 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.122173.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.122173.log.gz 2023-09-23T03:29:37.234 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131645.log 2023-09-23T03:29:37.235 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.116694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98532.log 2023-09-23T03:29:37.235 INFO:teuthology.orchestra.run.smithi154.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.116694.log.gz 2023-09-23T03:29:37.236 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.131645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126255.log 2023-09-23T03:29:37.236 INFO:teuthology.orchestra.run.smithi154.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.131645.log.gz 2023-09-23T03:29:37.237 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.98532.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98532.log.gz 2023-09-23T03:29:37.237 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94334.log 2023-09-23T03:29:37.238 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.126255.log: gzip -5 --verbose 87.8% -- /var/log/ceph/ceph-client.admin.95051.log -- replaced with /var/log/ceph/ceph-client.admin.126255.log.gz 2023-09-23T03:29:37.238 INFO:teuthology.orchestra.run.smithi154.stderr: 2023-09-23T03:29:37.239 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.94334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104588.log 2023-09-23T03:29:37.239 INFO:teuthology.orchestra.run.smithi154.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94334.log.gz 2023-09-23T03:29:37.240 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.95051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96266.log 2023-09-23T03:29:37.240 INFO:teuthology.orchestra.run.smithi154.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.95051.log.gz 2023-09-23T03:29:37.241 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.104588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110474.log 2023-09-23T03:29:37.241 INFO:teuthology.orchestra.run.smithi154.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.104588.log.gz 2023-09-23T03:29:37.242 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.96266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104108.log 2023-09-23T03:29:37.242 INFO:teuthology.orchestra.run.smithi154.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.96266.log.gz 2023-09-23T03:29:37.243 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.110474.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110474.log.gz 2023-09-23T03:29:37.243 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.121281.log 2023-09-23T03:29:37.244 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.104108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131561.log 2023-09-23T03:29:37.244 INFO:teuthology.orchestra.run.smithi154.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.104108.log.gz 2023-09-23T03:29:37.245 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.0.121281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110629.log 2023-09-23T03:29:37.246 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.131561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119104.log 2023-09-23T03:29:37.246 INFO:teuthology.orchestra.run.smithi154.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.131561.log.gz 2023-09-23T03:29:37.247 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.110629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121506.log 2023-09-23T03:29:37.247 INFO:teuthology.orchestra.run.smithi154.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.110629.log.gz 2023-09-23T03:29:37.248 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.119104.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.119104.log.gz 2023-09-23T03:29:37.249 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96944.log 2023-09-23T03:29:37.250 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.121506.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.121506.log.gz 2023-09-23T03:29:37.250 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107647.log 2023-09-23T03:29:37.251 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.96944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119734.log 2023-09-23T03:29:37.251 INFO:teuthology.orchestra.run.smithi154.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.96944.log.gz 2023-09-23T03:29:37.252 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.107647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92405.log 2023-09-23T03:29:37.252 INFO:teuthology.orchestra.run.smithi154.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.107647.log.gz 2023-09-23T03:29:37.253 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.119734.log: gzip 87.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.119734.log.gz 2023-09-23T03:29:37.253 INFO:teuthology.orchestra.run.smithi154.stderr: --verbose -- /var/log/ceph/ceph-client.admin.121420.log 2023-09-23T03:29:37.255 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.92405.log: gzip -5 --verbose 88.1% -- -- replaced with /var/log/ceph/ceph-client.admin.92405.log.gz /var/log/ceph/ceph-client.admin.115254.log 2023-09-23T03:29:37.255 INFO:teuthology.orchestra.run.smithi154.stderr: 2023-09-23T03:29:37.256 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.121420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116652.log 2023-09-23T03:29:37.256 INFO:teuthology.orchestra.run.smithi154.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.121420.log.gz 2023-09-23T03:29:37.257 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.115254.log: gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.115254.log.gz -5 2023-09-23T03:29:37.257 INFO:teuthology.orchestra.run.smithi154.stderr: --verbose -- /var/log/ceph/ceph-client.admin.114209.log 2023-09-23T03:29:37.258 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.116652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98112.log 2023-09-23T03:29:37.259 INFO:teuthology.orchestra.run.smithi154.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.116652.log.gz 2023-09-23T03:29:37.259 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.114209.log: 93.4% -- replaced with /var/log/ceph/ceph-client.0.121281.log.gz 2023-09-23T03:29:37.259 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115422.log 2023-09-23T03:29:37.260 INFO:teuthology.orchestra.run.smithi154.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.114209.log.gz 2023-09-23T03:29:37.261 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.98112.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.98112.log.gz 2023-09-23T03:29:37.261 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107179.log 2023-09-23T03:29:37.262 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.115422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126169.log 2023-09-23T03:29:37.262 INFO:teuthology.orchestra.run.smithi154.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.115422.log.gz 2023-09-23T03:29:37.263 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.107179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117549.log 2023-09-23T03:29:37.263 INFO:teuthology.orchestra.run.smithi154.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107179.log.gz 2023-09-23T03:29:37.264 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.126169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115990.log 2023-09-23T03:29:37.264 INFO:teuthology.orchestra.run.smithi154.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.126169.log.gz 2023-09-23T03:29:37.265 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.117549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119272.log 2023-09-23T03:29:37.266 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.115990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104672.log 2023-09-23T03:29:37.266 INFO:teuthology.orchestra.run.smithi154.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.115990.log.gz 2023-09-23T03:29:37.266 INFO:teuthology.orchestra.run.smithi154.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.117549.log.gz 2023-09-23T03:29:37.267 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.119272.log: gzip 88.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.119272.log.gz --verbose 2023-09-23T03:29:37.267 INFO:teuthology.orchestra.run.smithi154.stderr: -- /var/log/ceph/ceph-osd.1.log 2023-09-23T03:29:37.268 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.104672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95804.log 2023-09-23T03:29:37.268 INFO:teuthology.orchestra.run.smithi154.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.104672.log.gz 2023-09-23T03:29:37.269 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131771.log 2023-09-23T03:29:37.270 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.95804.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.95804.log.gz 2023-09-23T03:29:37.270 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114251.log 2023-09-23T03:29:37.271 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.131771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113938.log 2023-09-23T03:29:37.272 INFO:teuthology.orchestra.run.smithi154.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.131771.log.gz 2023-09-23T03:29:37.273 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.114251.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.114251.log.gz 2023-09-23T03:29:37.273 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119146.log 2023-09-23T03:29:37.274 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.113938.log: gzip -5 88.6% --verbose -- replaced with /var/log/ceph/ceph-client.admin.113938.log.gz 2023-09-23T03:29:37.274 INFO:teuthology.orchestra.run.smithi154.stderr: -- /var/log/ceph/ceph-client.admin.100505.log 2023-09-23T03:29:37.275 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.119146.log: gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.119146.log.gz -5 2023-09-23T03:29:37.275 INFO:teuthology.orchestra.run.smithi154.stderr: --verbose -- /var/log/ceph/ceph-client.admin.94724.log 2023-09-23T03:29:37.276 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.100505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106107.log 2023-09-23T03:29:37.277 INFO:teuthology.orchestra.run.smithi154.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.100505.log.gz 2023-09-23T03:29:37.278 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.94724.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.94724.log.gz 2023-09-23T03:29:37.278 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102053.log 2023-09-23T03:29:37.279 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.106107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125095.log 2023-09-23T03:29:37.279 INFO:teuthology.orchestra.run.smithi154.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106107.log.gz 2023-09-23T03:29:37.280 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.102053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122357.log 2023-09-23T03:29:37.280 INFO:teuthology.orchestra.run.smithi154.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.102053.log.gz 2023-09-23T03:29:37.281 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.125095.log: gzip -5 --verbose -- 88.6% /var/log/ceph/ceph-client.admin.100799.log -- replaced with /var/log/ceph/ceph-client.admin.125095.log.gz 2023-09-23T03:29:37.281 INFO:teuthology.orchestra.run.smithi154.stderr: 2023-09-23T03:29:37.282 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.122357.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.122357.log.gzgzip 2023-09-23T03:29:37.282 INFO:teuthology.orchestra.run.smithi154.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.93718.log 2023-09-23T03:29:37.284 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.100799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114636.log 2023-09-23T03:29:37.284 INFO:teuthology.orchestra.run.smithi154.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.100799.log.gz 2023-09-23T03:29:37.285 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.93718.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93718.log.gz 2023-09-23T03:29:37.285 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94716.log 2023-09-23T03:29:37.286 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.114636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127804.log 2023-09-23T03:29:37.286 INFO:teuthology.orchestra.run.smithi154.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.114636.log.gz 2023-09-23T03:29:37.287 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.94716.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.94716.log.gz 2023-09-23T03:29:37.287 INFO:teuthology.orchestra.run.smithi154.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.124139.log 2023-09-23T03:29:37.288 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.127804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117333.log 2023-09-23T03:29:37.288 INFO:teuthology.orchestra.run.smithi154.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.127804.log.gz 2023-09-23T03:29:37.290 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.124139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95062.log 2023-09-23T03:29:37.290 INFO:teuthology.orchestra.run.smithi154.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.124139.log.gz 2023-09-23T03:29:37.291 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.117333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92106.log 2023-09-23T03:29:37.291 INFO:teuthology.orchestra.run.smithi154.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.117333.log.gz 2023-09-23T03:29:37.292 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.95062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130236.log 2023-09-23T03:29:37.293 INFO:teuthology.orchestra.run.smithi154.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95062.log.gz 2023-09-23T03:29:37.294 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.92106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102495.log 2023-09-23T03:29:37.294 INFO:teuthology.orchestra.run.smithi154.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.92106.log.gz 2023-09-23T03:29:37.295 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.130236.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.130236.log.gz 2023-09-23T03:29:37.295 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115464.log 2023-09-23T03:29:37.296 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.102495.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.102495.log.gz 2023-09-23T03:29:37.296 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124885.log 2023-09-23T03:29:37.297 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.115464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125053.log 2023-09-23T03:29:37.298 INFO:teuthology.orchestra.run.smithi154.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.115464.log.gz 2023-09-23T03:29:37.299 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.124885.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.124885.log.gz 2023-09-23T03:29:37.299 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106722.log 2023-09-23T03:29:37.300 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.125053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104714.log 2023-09-23T03:29:37.300 INFO:teuthology.orchestra.run.smithi154.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.125053.log.gz 2023-09-23T03:29:37.301 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.106722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99435.log 2023-09-23T03:29:37.301 INFO:teuthology.orchestra.run.smithi154.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.106722.log.gz 2023-09-23T03:29:37.303 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.104714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131182.log 2023-09-23T03:29:37.303 INFO:teuthology.orchestra.run.smithi154.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.104714.log.gz 2023-09-23T03:29:37.304 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.99435.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99435.log.gz 2023-09-23T03:29:37.304 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120560.log 2023-09-23T03:29:37.305 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.131182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112037.log 2023-09-23T03:29:37.306 INFO:teuthology.orchestra.run.smithi154.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.131182.log.gz 2023-09-23T03:29:37.306 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.120560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99907.log 2023-09-23T03:29:37.307 INFO:teuthology.orchestra.run.smithi154.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.120560.log.gz 2023-09-23T03:29:37.308 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.112037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110181.log 2023-09-23T03:29:37.308 INFO:teuthology.orchestra.run.smithi154.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.112037.log.gz 2023-09-23T03:29:37.309 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.99907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113239.log 2023-09-23T03:29:37.309 INFO:teuthology.orchestra.run.smithi154.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.99907.log.gz 2023-09-23T03:29:37.310 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.110181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131056.log 2023-09-23T03:29:37.310 INFO:teuthology.orchestra.run.smithi154.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.110181.log.gz 2023-09-23T03:29:37.311 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.113239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98574.log 2023-09-23T03:29:37.311 INFO:teuthology.orchestra.run.smithi154.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.113239.log.gz 2023-09-23T03:29:37.313 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.131056.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.131056.log.gz 2023-09-23T03:29:37.313 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125011.log 2023-09-23T03:29:37.314 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.98574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94544.log 2023-09-23T03:29:37.314 INFO:teuthology.orchestra.run.smithi154.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.98574.log.gz 2023-09-23T03:29:37.315 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.125011.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.125011.log.gz 2023-09-23T03:29:37.315 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114125.log 2023-09-23T03:29:37.316 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.94544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99477.log 2023-09-23T03:29:37.316 INFO:teuthology.orchestra.run.smithi154.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.94544.log.gz 2023-09-23T03:29:37.317 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.114125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110587.log 2023-09-23T03:29:37.318 INFO:teuthology.orchestra.run.smithi154.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.114125.log.gz 2023-09-23T03:29:37.318 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.99477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111911.log 2023-09-23T03:29:37.319 INFO:teuthology.orchestra.run.smithi154.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.99477.log.gz 2023-09-23T03:29:37.320 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.110587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102327.log 2023-09-23T03:29:37.320 INFO:teuthology.orchestra.run.smithi154.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.110587.log.gz 2023-09-23T03:29:37.321 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.111911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99273.log 2023-09-23T03:29:37.321 INFO:teuthology.orchestra.run.smithi154.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.111911.log.gz 2023-09-23T03:29:37.322 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.102327.log: gzip -5 88.3% --verbose -- replaced with /var/log/ceph/ceph-client.admin.102327.log.gz -- 2023-09-23T03:29:37.322 INFO:teuthology.orchestra.run.smithi154.stderr: /var/log/ceph/ceph-client.admin.100631.log 2023-09-23T03:29:37.324 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.99273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118783.log 2023-09-23T03:29:37.324 INFO:teuthology.orchestra.run.smithi154.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.99273.log.gz 2023-09-23T03:29:37.325 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.100631.log: gzip -5 --verbose 88.8% -- -- replaced with /var/log/ceph/ceph-client.admin.100631.log.gz /var/log/ceph/ceph-client.admin.128931.log 2023-09-23T03:29:37.325 INFO:teuthology.orchestra.run.smithi154.stderr: 2023-09-23T03:29:37.326 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.118783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94418.log 2023-09-23T03:29:37.326 INFO:teuthology.orchestra.run.smithi154.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.118783.log.gz 2023-09-23T03:29:37.327 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.128931.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.128931.log.gz 2023-09-23T03:29:37.327 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126299.log 2023-09-23T03:29:37.328 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.94418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132335.log 2023-09-23T03:29:37.328 INFO:teuthology.orchestra.run.smithi154.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94418.log.gz 2023-09-23T03:29:37.330 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.126299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95395.log 2023-09-23T03:29:37.330 INFO:teuthology.orchestra.run.smithi154.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.126299.log.gz 2023-09-23T03:29:37.331 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.132335.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.132335.log.gz 2023-09-23T03:29:37.331 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95061.log 2023-09-23T03:29:37.332 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.95395.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.95395.log.gz 2023-09-23T03:29:37.333 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119188.log 2023-09-23T03:29:37.334 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.95061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97986.log 2023-09-23T03:29:37.334 INFO:teuthology.orchestra.run.smithi154.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.95061.log.gz 2023-09-23T03:29:37.335 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.119188.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.119188.log.gz 2023-09-23T03:29:37.335 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131729.log 2023-09-23T03:29:37.335 INFO:teuthology.orchestra.run.smithi190.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-09-23T03:29:37.336 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.97986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101051.log 2023-09-23T03:29:37.336 INFO:teuthology.orchestra.run.smithi154.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.97986.log.gz 2023-09-23T03:29:37.337 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.131729.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.131729.log.gz 2023-09-23T03:29:37.337 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111007.log 2023-09-23T03:29:37.338 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.101051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125660.log 2023-09-23T03:29:37.339 INFO:teuthology.orchestra.run.smithi154.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.101051.log.gz 2023-09-23T03:29:37.340 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.111007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92533.log 2023-09-23T03:29:37.340 INFO:teuthology.orchestra.run.smithi154.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.111007.log.gz 2023-09-23T03:29:37.342 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.125660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119776.log 2023-09-23T03:29:37.342 INFO:teuthology.orchestra.run.smithi154.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.125660.log.gz 2023-09-23T03:29:37.343 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.92533.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.92533.log.gz 2023-09-23T03:29:37.343 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102011.log 2023-09-23T03:29:37.345 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.119776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95379.log 2023-09-23T03:29:37.345 INFO:teuthology.orchestra.run.smithi154.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.119776.log.gz 2023-09-23T03:29:37.346 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.102011.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102011.log.gz 2023-09-23T03:29:37.346 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96014.log 2023-09-23T03:29:37.346 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.95379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117057.log 2023-09-23T03:29:37.346 INFO:teuthology.orchestra.run.smithi154.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.95379.log.gz 2023-09-23T03:29:37.348 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.96014.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96014.log.gz -5 2023-09-23T03:29:37.348 INFO:teuthology.orchestra.run.smithi154.stderr: --verbose -- /var/log/ceph/ceph-client.admin.122215.log 2023-09-23T03:29:37.349 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.117057.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.80700.log 2023-09-23T03:29:37.349 INFO:teuthology.orchestra.run.smithi154.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.117057.log.gz 2023-09-23T03:29:37.350 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.122215.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.122215.log.gz 2023-09-23T03:29:37.351 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101429.log 2023-09-23T03:29:37.351 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph.tmp-client.admin.80700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130320.log 2023-09-23T03:29:37.352 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.80700.log.gz 2023-09-23T03:29:37.352 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.101429.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.101429.log.gz 2023-09-23T03:29:37.353 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111977.log 2023-09-23T03:29:37.354 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.130320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116568.log 2023-09-23T03:29:37.354 INFO:teuthology.orchestra.run.smithi154.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.130320.log.gz 2023-09-23T03:29:37.355 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.111977.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.111977.log.gz 2023-09-23T03:29:37.355 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128805.log 2023-09-23T03:29:37.356 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.116568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106512.log 2023-09-23T03:29:37.356 INFO:teuthology.orchestra.run.smithi154.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.116568.log.gz 2023-09-23T03:29:37.357 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.128805.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.128805.log.gz 2023-09-23T03:29:37.358 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105216.log 2023-09-23T03:29:37.358 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.106512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127494.log 2023-09-23T03:29:37.359 INFO:teuthology.orchestra.run.smithi154.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.106512.log.gz 2023-09-23T03:29:37.360 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.105216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122641.log 2023-09-23T03:29:37.360 INFO:teuthology.orchestra.run.smithi154.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.105216.log.gz 2023-09-23T03:29:37.361 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.127494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130911.log 2023-09-23T03:29:37.362 INFO:teuthology.orchestra.run.smithi154.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.127494.log.gz 2023-09-23T03:29:37.363 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.122641.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-09-23T03:29:37.363 INFO:teuthology.orchestra.run.smithi154.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.122641.log.gz 2023-09-23T03:29:37.365 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.130911.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.130911.log.gz 2023-09-23T03:29:37.365 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129747.log 2023-09-23T03:29:37.366 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93592.log 2023-09-23T03:29:37.368 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.129747.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.129747.log.gz 2023-09-23T03:29:37.368 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113770.log 2023-09-23T03:29:37.369 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.93592.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.93592.log.gz 2023-09-23T03:29:37.369 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126695.log 2023-09-23T03:29:37.371 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.113770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120243.log 2023-09-23T03:29:37.371 INFO:teuthology.orchestra.run.smithi154.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.113770.log.gz 2023-09-23T03:29:37.372 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.126695.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.126695.log.gz 2023-09-23T03:29:37.373 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106680.log 2023-09-23T03:29:37.374 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.120243.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.120243.log.gz 2023-09-23T03:29:37.374 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113407.log 2023-09-23T03:29:37.375 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.106680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119650.log 2023-09-23T03:29:37.376 INFO:teuthology.orchestra.run.smithi154.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.106680.log.gz 2023-09-23T03:29:37.377 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.113407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103862.log 2023-09-23T03:29:37.377 INFO:teuthology.orchestra.run.smithi154.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.113407.log.gz 2023-09-23T03:29:37.379 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.119650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130362.log 2023-09-23T03:29:37.379 INFO:teuthology.orchestra.run.smithi154.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.119650.log.gz 2023-09-23T03:29:37.380 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.103862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110998.log 2023-09-23T03:29:37.380 INFO:teuthology.orchestra.run.smithi154.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103862.log.gz 2023-09-23T03:29:37.382 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.130362.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.130362.log.gz 2023-09-23T03:29:37.382 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103357.log 2023-09-23T03:29:37.384 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.110998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101303.log 2023-09-23T03:29:37.384 INFO:teuthology.orchestra.run.smithi154.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.110998.log.gz 2023-09-23T03:29:37.385 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.103357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101800.log 2023-09-23T03:29:37.385 INFO:teuthology.orchestra.run.smithi154.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.103357.log.gz 2023-09-23T03:29:37.386 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.101303.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.101303.log.gz 2023-09-23T03:29:37.387 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128847.log 2023-09-23T03:29:37.388 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.101800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109829.log 2023-09-23T03:29:37.388 INFO:teuthology.orchestra.run.smithi154.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101800.log.gz 2023-09-23T03:29:37.390 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.128847.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.128847.log.gz 2023-09-23T03:29:37.390 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131381.log 2023-09-23T03:29:37.392 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.109829.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.109829.log.gz 2023-09-23T03:29:37.392 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117100.log 2023-09-23T03:29:37.393 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.131381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123503.log 2023-09-23T03:29:37.393 INFO:teuthology.orchestra.run.smithi154.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.131381.log.gz 2023-09-23T03:29:37.395 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.117100.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.117100.log.gz 2023-09-23T03:29:37.395 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92317.log 2023-09-23T03:29:37.396 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.123503.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.123503.log.gz 2023-09-23T03:29:37.397 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116484.log 2023-09-23T03:29:37.398 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.92317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100925.log 2023-09-23T03:29:37.398 INFO:teuthology.orchestra.run.smithi154.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.92317.log.gz 2023-09-23T03:29:37.400 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.116484.log: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.116484.log.gz 2023-09-23T03:29:37.400 INFO:teuthology.orchestra.run.smithi154.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.116074.log 2023-09-23T03:29:37.401 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.100925.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100925.log.gz 2023-09-23T03:29:37.402 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110097.log 2023-09-23T03:29:37.403 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.116074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100385.log 2023-09-23T03:29:37.403 INFO:teuthology.orchestra.run.smithi154.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.116074.log.gz 2023-09-23T03:29:37.405 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.110097.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.110097.log.gz 2023-09-23T03:29:37.405 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125618.log 2023-09-23T03:29:37.406 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.100385.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100385.log.gz 2023-09-23T03:29:37.407 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117926.log 2023-09-23T03:29:37.408 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.125618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.115807.log 2023-09-23T03:29:37.408 INFO:teuthology.orchestra.run.smithi154.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.125618.log.gz 2023-09-23T03:29:37.409 INFO:teuthology.orchestra.run.smithi190.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-09-23T03:29:37.410 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.117926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119398.log 2023-09-23T03:29:37.410 INFO:teuthology.orchestra.run.smithi154.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.117926.log.gz 2023-09-23T03:29:37.411 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.0.115807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105806.log 2023-09-23T03:29:37.413 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.119398.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.119398.log.gz 2023-09-23T03:29:37.413 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98406.log 2023-09-23T03:29:37.414 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.105806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105130.log 2023-09-23T03:29:37.416 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.98406.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.105806.log.gz 2023-09-23T03:29:37.416 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111013.log 2023-09-23T03:29:37.416 INFO:teuthology.orchestra.run.smithi154.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.98406.log.gz 2023-09-23T03:29:37.417 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.105130.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.105130.log.gz 2023-09-23T03:29:37.418 INFO:teuthology.orchestra.run.smithi154.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.0.115807.log.gz 2023-09-23T03:29:37.418 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98238.log 2023-09-23T03:29:37.419 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.111013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95393.log 2023-09-23T03:29:37.420 INFO:teuthology.orchestra.run.smithi154.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.111013.log.gz 2023-09-23T03:29:37.420 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.98238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111323.log 2023-09-23T03:29:37.421 INFO:teuthology.orchestra.run.smithi154.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.98238.log.gz 2023-09-23T03:29:37.422 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.95393.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95393.log.gz 2023-09-23T03:29:37.422 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99561.log 2023-09-23T03:29:37.424 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.111323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102095.log 2023-09-23T03:29:37.424 INFO:teuthology.orchestra.run.smithi154.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.111323.log.gz 2023-09-23T03:29:37.425 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.99561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120285.log 2023-09-23T03:29:37.425 INFO:teuthology.orchestra.run.smithi154.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.99561.log.gz 2023-09-23T03:29:37.426 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.102095.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102095.log.gz 2023-09-23T03:29:37.427 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.127122.log 2023-09-23T03:29:37.428 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.120285.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.120285.log.gz 2023-09-23T03:29:37.428 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93986.log 2023-09-23T03:29:37.429 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.0.127122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92233.log 2023-09-23T03:29:37.431 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.93986.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.93986.log.gz 2023-09-23T03:29:37.431 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100343.log 2023-09-23T03:29:37.433 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.92233.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.92233.log.gz 2023-09-23T03:29:37.433 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127394.log 2023-09-23T03:29:37.435 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.100343.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100343.log.gz 2023-09-23T03:29:37.435 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94717.log 2023-09-23T03:29:37.436 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.127394.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.127394.log.gz 2023-09-23T03:29:37.437 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94715.log 2023-09-23T03:29:37.438 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.94717.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94717.log.gz 2023-09-23T03:29:37.439 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129277.log 2023-09-23T03:29:37.440 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.94715.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.94715.log.gz 2023-09-23T03:29:37.440 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122089.log 2023-09-23T03:29:37.441 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.129277.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.129277.log.gz 2023-09-23T03:29:37.442 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114974.log 2023-09-23T03:29:37.443 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.122089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111312.log 2023-09-23T03:29:37.443 INFO:teuthology.orchestra.run.smithi154.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.122089.log.gz 2023-09-23T03:29:37.445 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.114974.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.114974.log.gz 2023-09-23T03:29:37.445 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97734.log 2023-09-23T03:29:37.446 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.111312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102453.log 2023-09-23T03:29:37.447 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.97734.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.111312.log.gz 2023-09-23T03:29:37.447 INFO:teuthology.orchestra.run.smithi154.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.97734.log.gz 2023-09-23T03:29:37.448 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94250.log 2023-09-23T03:29:37.449 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.102453.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.102453.log.gz 2023-09-23T03:29:37.449 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113812.log 2023-09-23T03:29:37.450 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.94250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132065.log 2023-09-23T03:29:37.451 INFO:teuthology.orchestra.run.smithi154.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.94250.log.gz 2023-09-23T03:29:37.452 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.113812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107817.log 2023-09-23T03:29:37.452 INFO:teuthology.orchestra.run.smithi154.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.113812.log.gz 2023-09-23T03:29:37.454 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.132065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123653.log 2023-09-23T03:29:37.454 INFO:teuthology.orchestra.run.smithi154.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.132065.log.gz 2023-09-23T03:29:37.456 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.107817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.96840.log 2023-09-23T03:29:37.456 INFO:teuthology.orchestra.run.smithi154.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.107817.log.gz 2023-09-23T03:29:37.457 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.123653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103946.log 2023-09-23T03:29:37.458 INFO:teuthology.orchestra.run.smithi154.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.123653.log.gz 2023-09-23T03:29:37.459 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.0.96840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113728.log 2023-09-23T03:29:37.460 INFO:teuthology.orchestra.run.smithi154.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.96840.log.gz 2023-09-23T03:29:37.460 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.103946.log: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126443.log 2023-09-23T03:29:37.461 INFO:teuthology.orchestra.run.smithi154.stderr: -- replaced with /var/log/ceph/ceph-client.admin.103946.log.gz 2023-09-23T03:29:37.462 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.113728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100841.log 2023-09-23T03:29:37.462 INFO:teuthology.orchestra.run.smithi154.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.113728.log.gz 2023-09-23T03:29:37.463 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.126443.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.126443.log.gz 2023-09-23T03:29:37.463 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106890.log 2023-09-23T03:29:37.465 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.100841.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.100841.log.gz 2023-09-23T03:29:37.465 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98322.log 2023-09-23T03:29:37.466 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.106890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114022.log 2023-09-23T03:29:37.467 INFO:teuthology.orchestra.run.smithi154.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.106890.log.gz 2023-09-23T03:29:37.468 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.98322.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98322.log.gz 2023-09-23T03:29:37.468 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110904.log 2023-09-23T03:29:37.469 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.114022.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.114022.log.gz 2023-09-23T03:29:37.470 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99063.log 2023-09-23T03:29:37.471 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.110904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108375.log 2023-09-23T03:29:37.472 INFO:teuthology.orchestra.run.smithi154.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.110904.log.gz 2023-09-23T03:29:37.472 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.99063.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.99063.log.gz 2023-09-23T03:29:37.473 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129157.log 2023-09-23T03:29:37.475 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.108375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95678.log 2023-09-23T03:29:37.475 INFO:teuthology.orchestra.run.smithi154.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.108375.log.gz 2023-09-23T03:29:37.475 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.129157.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.129157.log.gz 2023-09-23T03:29:37.476 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113644.log 2023-09-23T03:29:37.477 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.95678.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.95678.log.gz 2023-09-23T03:29:37.477 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101843.log 2023-09-23T03:29:37.479 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.113644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122935.log 2023-09-23T03:29:37.479 INFO:teuthology.orchestra.run.smithi154.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.113644.log.gz 2023-09-23T03:29:37.480 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.101843.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.101843.log.gz 2023-09-23T03:29:37.480 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122893.log 2023-09-23T03:29:37.482 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.122935.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.122935.log.gz 2023-09-23T03:29:37.482 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124606.log 2023-09-23T03:29:37.483 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.122893.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.122893.log.gz 2023-09-23T03:29:37.483 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106149.log 2023-09-23T03:29:37.485 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.124606.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.124606.log.gz 2023-09-23T03:29:37.485 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95762.log 2023-09-23T03:29:37.486 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.106149.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.106149.log.gz 2023-09-23T03:29:37.487 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108249.log 2023-09-23T03:29:37.488 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.95762.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.95762.log.gz 2023-09-23T03:29:37.488 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-09-23T03:29:37.489 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.108249.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.108249.log.gz 2023-09-23T03:29:37.490 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122766.log 2023-09-23T03:29:37.491 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96436.log 2023-09-23T03:29:37.493 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.122766.log: gzip -5 88.5% -- replaced with /var/log/ceph/ceph-client.admin.122766.log.gz 2023-09-23T03:29:37.493 INFO:teuthology.orchestra.run.smithi154.stderr: --verbose -- /var/log/ceph/ceph-client.admin.115530.log 2023-09-23T03:29:37.494 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.96436.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.96436.log.gz 2023-09-23T03:29:37.495 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124205.log 2023-09-23T03:29:37.496 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.115530.log: gzip -5 88.8% -- replaced with /var/log/ceph/ceph-client.admin.115530.log.gz 2023-09-23T03:29:37.497 INFO:teuthology.orchestra.run.smithi154.stderr: --verbose -- /var/log/ceph/ceph-client.admin.125305.log 2023-09-23T03:29:37.498 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.124205.log: gzip -5 --verbose -- 91.0% -- replaced with /var/log/ceph/ceph-client.admin.124205.log.gz 2023-09-23T03:29:37.499 INFO:teuthology.orchestra.run.smithi154.stderr: /var/log/ceph/ceph-client.admin.130785.log 2023-09-23T03:29:37.500 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.125305.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.125305.log.gz 2023-09-23T03:29:37.500 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124969.log 2023-09-23T03:29:37.502 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.130785.log: 93.2% -- replaced with /var/log/ceph/ceph-client.0.127122.log.gz 2023-09-23T03:29:37.502 INFO:teuthology.orchestra.run.smithi154.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.130785.log.gz 2023-09-23T03:29:37.502 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117591.log 2023-09-23T03:29:37.503 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.124969.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.124969.log.gz 2023-09-23T03:29:37.503 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104949.log 2023-09-23T03:29:37.505 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.117591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128134.log 2023-09-23T03:29:37.506 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.104949.log: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.117591.log.gz 2023-09-23T03:29:37.507 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose 89.0% -- replaced with /var/log/ceph/ceph-client.admin.104949.log.gz 2023-09-23T03:29:37.507 INFO:teuthology.orchestra.run.smithi154.stderr: -- /var/log/ceph/ceph-client.admin.105324.log 2023-09-23T03:29:37.508 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.128134.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.128134.log.gz 2023-09-23T03:29:37.509 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115024.log 2023-09-23T03:29:37.510 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.105324.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.105324.log.gz 2023-09-23T03:29:37.511 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99771.log 2023-09-23T03:29:37.512 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.115024.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.115024.log.gz 2023-09-23T03:29:37.512 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98879.log 2023-09-23T03:29:37.513 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.99771.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.99771.log.gz 2023-09-23T03:29:37.514 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119608.log 2023-09-23T03:29:37.515 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.98879.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98879.log.gz 2023-09-23T03:29:37.516 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111582.log 2023-09-23T03:29:37.518 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.119608.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.119608.log.gz 2023-09-23T03:29:37.518 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94166.log 2023-09-23T03:29:37.519 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.111582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121464.log 2023-09-23T03:29:37.521 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.94166.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94166.log.gz 2023-09-23T03:29:37.521 INFO:teuthology.orchestra.run.smithi154.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.111582.log.gz 2023-09-23T03:29:37.521 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111867.log 2023-09-23T03:29:37.522 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.121464.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.121464.log.gz 2023-09-23T03:29:37.523 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108983.log 2023-09-23T03:29:37.525 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.111867.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111867.log.gz 2023-09-23T03:29:37.525 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101555.log 2023-09-23T03:29:37.526 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.108983.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.108983.log.gz 2023-09-23T03:29:37.526 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103594.log 2023-09-23T03:29:37.528 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.101555.log: gzip 88.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.96140.log 2023-09-23T03:29:37.528 INFO:teuthology.orchestra.run.smithi154.stderr: -- replaced with /var/log/ceph/ceph-client.admin.101555.log.gz 2023-09-23T03:29:37.529 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.103594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99519.log 2023-09-23T03:29:37.529 INFO:teuthology.orchestra.run.smithi154.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.103594.log.gz 2023-09-23T03:29:37.531 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.96140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126127.log 2023-09-23T03:29:37.531 INFO:teuthology.orchestra.run.smithi154.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.96140.log.gz 2023-09-23T03:29:37.533 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.99519.log: gzip -5 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99519.log.gz 2023-09-23T03:29:37.533 INFO:teuthology.orchestra.run.smithi154.stderr: --verbose -- /var/log/ceph/ceph-client.admin.125450.log 2023-09-23T03:29:37.534 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.126127.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.126127.log.gz 2023-09-23T03:29:37.534 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104846.log 2023-09-23T03:29:37.535 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.125450.log: gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.125450.log.gz 2023-09-23T03:29:37.536 INFO:teuthology.orchestra.run.smithi154.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.106428.log 2023-09-23T03:29:37.537 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.104846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120119.log 2023-09-23T03:29:37.539 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.106428.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.106428.log.gz 2023-09-23T03:29:37.540 INFO:teuthology.orchestra.run.smithi154.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.104846.log.gz 2023-09-23T03:29:37.540 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114377.log 2023-09-23T03:29:37.540 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.120119.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.120119.log.gz 2023-09-23T03:29:37.541 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126527.log 2023-09-23T03:29:37.542 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.114377.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.114377.log.gz 2023-09-23T03:29:37.543 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94725.log 2023-09-23T03:29:37.545 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.126527.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.126527.log.gz 2023-09-23T03:29:37.545 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118600.log 2023-09-23T03:29:37.546 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.94725.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94725.log.gz 2023-09-23T03:29:37.547 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126737.log 2023-09-23T03:29:37.548 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.118600.log: gzip -5 --verbose -- 91.2% -- replaced with /var/log/ceph/ceph-client.admin.118600.log.gz 2023-09-23T03:29:37.549 INFO:teuthology.orchestra.run.smithi154.stderr: /var/log/ceph/ceph-client.admin.117226.log 2023-09-23T03:29:37.549 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.126737.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.126737.log.gz 2023-09-23T03:29:37.550 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97692.log 2023-09-23T03:29:37.551 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.117226.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.117226.log.gz 2023-09-23T03:29:37.554 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101471.log 2023-09-23T03:29:37.555 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.97692.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.97692.log.gz 2023-09-23T03:29:37.555 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94714.log 2023-09-23T03:29:37.555 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.101471.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101471.log.gz 2023-09-23T03:29:37.556 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120329.log 2023-09-23T03:29:37.557 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.94714.log: gzip 87.6% -5 --verbose -- /var/log/ceph/ceph-client.admin.119356.log 2023-09-23T03:29:37.557 INFO:teuthology.orchestra.run.smithi154.stderr: -- replaced with /var/log/ceph/ceph-client.admin.94714.log.gz 2023-09-23T03:29:37.558 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.120329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116216.log 2023-09-23T03:29:37.559 INFO:teuthology.orchestra.run.smithi154.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.120329.log.gz 2023-09-23T03:29:37.560 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.119356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94028.log 2023-09-23T03:29:37.560 INFO:teuthology.orchestra.run.smithi154.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.119356.log.gz 2023-09-23T03:29:37.562 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.116216.log: gzip -5 88.4% -- replaced with /var/log/ceph/ceph-client.admin.116216.log.gz 2023-09-23T03:29:37.562 INFO:teuthology.orchestra.run.smithi154.stderr: --verbose -- /var/log/ceph/ceph-client.admin.106638.log 2023-09-23T03:29:37.563 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.94028.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.94028.log.gz 2023-09-23T03:29:37.563 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104888.log 2023-09-23T03:29:37.565 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.106638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108501.log 2023-09-23T03:29:37.565 INFO:teuthology.orchestra.run.smithi154.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.106638.log.gz 2023-09-23T03:29:37.567 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.104888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97608.log 2023-09-23T03:29:37.568 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.108501.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.104888.log.gz 2023-09-23T03:29:37.569 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109971.log 91.2% -- replaced with /var/log/ceph/ceph-client.admin.108501.log.gz 2023-09-23T03:29:37.569 INFO:teuthology.orchestra.run.smithi154.stderr: 2023-09-23T03:29:37.569 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.97608.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.97608.log.gz 2023-09-23T03:29:37.570 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122809.log 2023-09-23T03:29:37.572 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.109971.log: gzip -5 89.6% -- replaced with /var/log/ceph/ceph-client.admin.109971.log.gz 2023-09-23T03:29:37.572 INFO:teuthology.orchestra.run.smithi154.stderr: --verbose -- /var/log/ceph/ceph-client.admin.109703.log 2023-09-23T03:29:37.573 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.122809.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.122809.log.gz 2023-09-23T03:29:37.574 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.130024.log 2023-09-23T03:29:37.575 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.109703.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.109703.log.gz 2023-09-23T03:29:37.576 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95636.log 2023-09-23T03:29:37.577 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.0.130024.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-09-23T03:29:37.578 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.95636.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95636.log.gz 2023-09-23T03:29:37.579 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114293.log 2023-09-23T03:29:37.581 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111294.log 2023-09-23T03:29:37.582 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.114293.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92827.log 2023-09-23T03:29:37.583 INFO:teuthology.orchestra.run.smithi154.stderr: -- replaced with /var/log/ceph/ceph-client.admin.114293.log.gz 2023-09-23T03:29:37.584 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.111294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117417.log 2023-09-23T03:29:37.585 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.92827.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.111294.log.gz 2023-09-23T03:29:37.587 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93550.log 2023-09-23T03:29:37.588 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.117417.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.92827.log.gz 2023-09-23T03:29:37.590 INFO:teuthology.orchestra.run.smithi154.stderr: 91.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110862.log 2023-09-23T03:29:37.590 INFO:teuthology.orchestra.run.smithi154.stderr: -- replaced with /var/log/ceph/ceph-client.admin.117417.log.gz 2023-09-23T03:29:37.591 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103904.log 2023-09-23T03:29:37.592 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.93550.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.93550.log.gz 2023-09-23T03:29:37.593 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.110862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103762.log 2023-09-23T03:29:37.594 INFO:teuthology.orchestra.run.smithi154.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.110862.log.gz 2023-09-23T03:29:37.595 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.103904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129553.log 2023-09-23T03:29:37.596 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.103762.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103904.log.gz 2023-09-23T03:29:37.597 INFO:teuthology.orchestra.run.smithi154.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.103762.log.gz 2023-09-23T03:29:37.598 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99991.log 2023-09-23T03:29:37.600 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.129553.log: 88.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.129553.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.125881.log 2023-09-23T03:29:37.600 INFO:teuthology.orchestra.run.smithi154.stderr: 2023-09-23T03:29:37.601 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99949.log 2023-09-23T03:29:37.602 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.99991.log: /var/log/ceph/ceph-client.admin.125881.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99991.log.gz 2023-09-23T03:29:37.603 INFO:teuthology.orchestra.run.smithi154.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.125881.log.gz 2023-09-23T03:29:37.604 INFO:teuthology.orchestra.run.smithi154.stderr: 95.0% -- replaced with /var/log/ceph/ceph-client.0.130024.log.gz 2023-09-23T03:29:37.604 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.128588.log 2023-09-23T03:29:37.606 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.99949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120039.log 2023-09-23T03:29:37.607 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.0.128588.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.99949.log.gz 2023-09-23T03:29:37.608 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94502.log 2023-09-23T03:29:37.609 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120475.log 2023-09-23T03:29:37.611 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.120039.log: /var/log/ceph/ceph-client.admin.94502.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.120039.log.gz 2023-09-23T03:29:37.611 INFO:teuthology.orchestra.run.smithi154.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.94502.log.gz 2023-09-23T03:29:37.612 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93634.log 2023-09-23T03:29:37.614 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120728.log 2023-09-23T03:29:37.615 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.120475.log: /var/log/ceph/ceph-client.admin.93634.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.120475.log.gz 2023-09-23T03:29:37.616 INFO:teuthology.orchestra.run.smithi154.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.93634.log.gz 2023-09-23T03:29:37.616 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94460.log 2023-09-23T03:29:37.618 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107095.log 2023-09-23T03:29:37.619 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.120728.log: /var/log/ceph/ceph-client.admin.94460.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.120728.log.gz 2023-09-23T03:29:37.620 INFO:teuthology.orchestra.run.smithi154.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.94460.log.gz 2023-09-23T03:29:37.620 INFO:teuthology.orchestra.run.smithi154.stderr: 94.3% -- replaced with /var/log/ceph/ceph-client.0.128588.log.gz 2023-09-23T03:29:37.620 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118516.log 2023-09-23T03:29:37.623 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96308.log 2023-09-23T03:29:37.623 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.107095.log: /var/log/ceph/ceph-client.admin.118516.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107095.log.gz 2023-09-23T03:29:37.624 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 90.9% -- replaced with /var/log/ceph/ceph-client.admin.118516.log.gz 2023-09-23T03:29:37.624 INFO:teuthology.orchestra.run.smithi154.stderr: --verbose -- /var/log/ceph/ceph-client.admin.99105.log 2023-09-23T03:29:37.626 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.96308.log: gzip -5 --verbose -- 91.2% /var/log/ceph/ceph-client.admin.99351.log 2023-09-23T03:29:37.626 INFO:teuthology.orchestra.run.smithi154.stderr: -- replaced with /var/log/ceph/ceph-client.admin.96308.log.gz 2023-09-23T03:29:37.628 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.99105.log: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92489.log 2023-09-23T03:29:37.628 INFO:teuthology.orchestra.run.smithi154.stderr: -- replaced with /var/log/ceph/ceph-client.admin.99105.log.gz 2023-09-23T03:29:37.629 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.99351.log: gzip -5 --verbose 89.0% -- replaced with /var/log/ceph/ceph-client.admin.99351.log.gz 2023-09-23T03:29:37.629 INFO:teuthology.orchestra.run.smithi154.stderr: -- /var/log/ceph/ceph-client.admin.126042.log 2023-09-23T03:29:37.630 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.92489.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.92489.log.gz 2023-09-23T03:29:37.631 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92275.log 2023-09-23T03:29:37.632 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.126042.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.126042.log.gz 2023-09-23T03:29:37.633 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92701.log 2023-09-23T03:29:37.635 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.92275.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.92275.log.gz 2023-09-23T03:29:37.635 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131098.log 2023-09-23T03:29:37.637 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.92701.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.92701.log.gz 2023-09-23T03:29:37.637 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130404.log 2023-09-23T03:29:37.638 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.131098.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.131098.log.gz 2023-09-23T03:29:37.638 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98795.log 2023-09-23T03:29:37.639 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.130404.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.130404.log.gz 2023-09-23T03:29:37.640 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127536.log 2023-09-23T03:29:37.642 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.98795.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.98795.log.gz 2023-09-23T03:29:37.642 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111643.log 2023-09-23T03:29:37.643 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.127536.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.127536.log.gz 2023-09-23T03:29:37.644 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107605.log 2023-09-23T03:29:37.646 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.111643.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111643.log.gz 2023-09-23T03:29:37.646 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95720.log 2023-09-23T03:29:37.647 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.107605.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107605.log.gz 2023-09-23T03:29:37.647 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123611.log 2023-09-23T03:29:37.649 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.95720.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.95720.log.gz 2023-09-23T03:29:37.649 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97902.log 2023-09-23T03:29:37.650 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.123611.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.123611.log.gz 2023-09-23T03:29:37.651 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99687.log 2023-09-23T03:29:37.653 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.97902.log: gzip 88.0% -- replaced with /var/log/ceph/ceph-client.admin.97902.log.gz 2023-09-23T03:29:37.653 INFO:teuthology.orchestra.run.smithi154.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.112603.log 2023-09-23T03:29:37.654 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.99687.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.99687.log.gz 2023-09-23T03:29:37.654 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.105601.log 2023-09-23T03:29:37.655 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.112603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102579.log 2023-09-23T03:29:37.657 INFO:teuthology.orchestra.run.smithi154.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.112603.log.gz 2023-09-23T03:29:37.657 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120812.log 2023-09-23T03:29:37.659 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.0.105601.log: /var/log/ceph/ceph-client.admin.102579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131939.log 2023-09-23T03:29:37.659 INFO:teuthology.orchestra.run.smithi154.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.102579.log.gz 2023-09-23T03:29:37.660 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.120812.log: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93331.log 2023-09-23T03:29:37.661 INFO:teuthology.orchestra.run.smithi154.stderr: -- replaced with /var/log/ceph/ceph-client.admin.120812.log.gz 2023-09-23T03:29:37.662 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.131939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116258.log 2023-09-23T03:29:37.662 INFO:teuthology.orchestra.run.smithi154.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.131939.log.gz 2023-09-23T03:29:37.664 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.93331.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-09-23T03:29:37.665 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.116258.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.93331.log.gz 2023-09-23T03:29:37.665 INFO:teuthology.orchestra.run.smithi154.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.116258.log.gz 2023-09-23T03:29:37.666 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116526.log 2023-09-23T03:29:37.667 INFO:teuthology.orchestra.run.smithi154.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.0.105601.log.gz 2023-09-23T03:29:37.669 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95398.log 2023-09-23T03:29:37.669 INFO:teuthology.orchestra.run.smithi154.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-09-23T03:29:37.670 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.116526.log: gzip -5 89.4% -- replaced with /var/log/ceph/ceph-client.admin.116526.log.gz 2023-09-23T03:29:37.670 INFO:teuthology.orchestra.run.smithi154.stderr: --verbose -- /var/log/ceph/ceph-client.admin.125221.log 2023-09-23T03:29:37.671 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.95398.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.95398.log.gz 2023-09-23T03:29:37.671 INFO:teuthology.orchestra.run.smithi154.stderr: 87.0% -- replaced with /var/log/ceph/ceph.log.gz 2023-09-23T03:29:37.672 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107861.log 2023-09-23T03:29:37.672 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.125221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116300.log 2023-09-23T03:29:37.673 INFO:teuthology.orchestra.run.smithi154.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.125221.log.gz/var/log/ceph/ceph-client.admin.107861.log: 2023-09-23T03:29:37.674 INFO:teuthology.orchestra.run.smithi154.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107861.log.gz 2023-09-23T03:29:37.674 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124732.log 2023-09-23T03:29:37.676 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.116300.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.116300.log.gz 2023-09-23T03:29:37.676 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99393.log 2023-09-23T03:29:37.677 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.124732.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.124732.log.gz 2023-09-23T03:29:37.677 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-09-23T03:29:37.678 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.99393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130743.log 2023-09-23T03:29:37.679 INFO:teuthology.orchestra.run.smithi154.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.99393.log.gz 2023-09-23T03:29:37.680 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95888.log 2023-09-23T03:29:37.682 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.130743.log: gzip -5 --verbose 88.9% -- replaced with /var/log/ceph/ceph-client.admin.130743.log.gz 2023-09-23T03:29:37.682 INFO:teuthology.orchestra.run.smithi154.stderr: -- /var/log/ceph/ceph-client.admin.118825.log 2023-09-23T03:29:37.683 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.95888.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.95888.log.gz 2023-09-23T03:29:37.683 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92191.log 2023-09-23T03:29:37.685 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.118825.log: gzip 88.0% -- replaced with /var/log/ceph/ceph-client.admin.118825.log.gz 2023-09-23T03:29:37.685 INFO:teuthology.orchestra.run.smithi154.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.108081.log 2023-09-23T03:29:37.686 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.92191.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.92191.log.gz 2023-09-23T03:29:37.687 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104285.log 2023-09-23T03:29:37.688 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.108081.log: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113686.log 2023-09-23T03:29:37.689 INFO:teuthology.orchestra.run.smithi154.stderr: -- replaced with /var/log/ceph/ceph-client.admin.108081.log.gz 2023-09-23T03:29:37.690 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.104285.log: gzip -5 89.0% -- replaced with /var/log/ceph/ceph-client.admin.104285.log.gz 2023-09-23T03:29:37.690 INFO:teuthology.orchestra.run.smithi154.stderr: --verbose -- /var/log/ceph/ceph-client.admin.113281.log 2023-09-23T03:29:37.691 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.113686.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.113686.log.gz 2023-09-23T03:29:37.692 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109069.log 2023-09-23T03:29:37.693 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.113281.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.113281.log.gz 2023-09-23T03:29:37.694 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-09-23T03:29:37.695 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.109069.log: gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.109069.log.gz 2023-09-23T03:29:37.696 INFO:teuthology.orchestra.run.smithi154.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.126653.log 2023-09-23T03:29:37.697 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116772.log 2023-09-23T03:29:37.699 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.126653.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.126653.log.gz 2023-09-23T03:29:37.699 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115338.log 2023-09-23T03:29:37.701 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.116772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104150.log 2023-09-23T03:29:37.702 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.115338.log: 88.9% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.116772.log.gz 2023-09-23T03:29:37.702 INFO:teuthology.orchestra.run.smithi154.stderr: -- replaced with /var/log/ceph/ceph-client.admin.115338.log.gz 2023-09-23T03:29:37.704 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114930.log 2023-09-23T03:29:37.705 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.104150.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.104150.log.gz 2023-09-23T03:29:37.705 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107689.log 2023-09-23T03:29:37.707 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106993.log 2023-09-23T03:29:37.708 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.114930.log: /var/log/ceph/ceph-client.admin.107689.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.114930.log.gz 2023-09-23T03:29:37.708 INFO:teuthology.orchestra.run.smithi154.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.107689.log.gz 2023-09-23T03:29:37.710 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110055.log 2023-09-23T03:29:37.712 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.106993.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.106993.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110259.log 2023-09-23T03:29:37.712 INFO:teuthology.orchestra.run.smithi154.stderr: 2023-09-23T03:29:37.713 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.110055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91929.log 2023-09-23T03:29:37.714 INFO:teuthology.orchestra.run.smithi154.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.110055.log.gz 2023-09-23T03:29:37.715 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.110259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110995.log 2023-09-23T03:29:37.715 INFO:teuthology.orchestra.run.smithi154.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.110259.log.gz 2023-09-23T03:29:37.717 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.91929.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.91929.log.gz 2023-09-23T03:29:37.719 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95594.log 2023-09-23T03:29:37.720 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.110995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92617.log 2023-09-23T03:29:37.721 INFO:teuthology.orchestra.run.smithi154.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.110995.log.gz 2023-09-23T03:29:37.723 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.95594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108123.log 2023-09-23T03:29:37.724 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.92617.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.95594.log.gz 2023-09-23T03:29:37.724 INFO:teuthology.orchestra.run.smithi154.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.92617.log.gz 2023-09-23T03:29:37.726 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99005.log 2023-09-23T03:29:37.727 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.108123.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.108123.log.gz 2023-09-23T03:29:37.727 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101093.log 2023-09-23T03:29:37.729 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123258.log 2023-09-23T03:29:37.730 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.99005.log: /var/log/ceph/ceph-client.admin.101093.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99005.log.gz 2023-09-23T03:29:37.730 INFO:teuthology.orchestra.run.smithi154.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101093.log.gz 2023-09-23T03:29:37.733 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97944.log 2023-09-23T03:29:37.734 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.123258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131467.log 2023-09-23T03:29:37.737 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.97944.log: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.123258.log.gz 2023-09-23T03:29:37.737 INFO:teuthology.orchestra.run.smithi154.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.97944.log.gz 2023-09-23T03:29:37.738 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118909.log 2023-09-23T03:29:37.739 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129403.log 2023-09-23T03:29:37.740 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.131467.log: /var/log/ceph/ceph-client.admin.118909.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.131467.log.gz 2023-09-23T03:29:37.741 INFO:teuthology.orchestra.run.smithi154.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.118909.log.gz 2023-09-23T03:29:37.742 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102137.log 2023-09-23T03:29:37.743 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125576.log 2023-09-23T03:29:37.745 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.129403.log: /var/log/ceph/ceph-client.admin.102137.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.102137.log.gz 2023-09-23T03:29:37.746 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128973.log 2023-09-23T03:29:37.748 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.125576.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.125576.log.gz 2023-09-23T03:29:37.749 INFO:teuthology.orchestra.run.smithi154.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.admin.129403.log.gz 2023-09-23T03:29:37.750 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100175.log 2023-09-23T03:29:37.751 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.128973.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.128973.log.gz 2023-09-23T03:29:37.751 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101345.log 2023-09-23T03:29:37.753 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.97334.log 2023-09-23T03:29:37.754 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.100175.log: /var/log/ceph/ceph-client.admin.101345.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.100175.log.gz 2023-09-23T03:29:37.755 INFO:teuthology.orchestra.run.smithi154.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101345.log.gz 2023-09-23T03:29:37.756 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129681.log 2023-09-23T03:29:37.757 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.2.97334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117184.log 2023-09-23T03:29:37.758 INFO:teuthology.orchestra.run.smithi154.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.2.97334.log.gz 2023-09-23T03:29:37.760 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.129681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118450.log 2023-09-23T03:29:37.760 INFO:teuthology.orchestra.run.smithi154.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.129681.log.gz 2023-09-23T03:29:37.762 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.117184.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.117184.log.gz 2023-09-23T03:29:37.764 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113980.log 2023-09-23T03:29:37.765 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.118450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106848.log 2023-09-23T03:29:37.766 INFO:teuthology.orchestra.run.smithi154.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.118450.log.gz 2023-09-23T03:29:37.767 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.113980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100883.log 2023-09-23T03:29:37.768 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.106848.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.113980.log.gz 87.9% -- replaced with /var/log/ceph/ceph-client.admin.106848.log.gz 2023-09-23T03:29:37.769 INFO:teuthology.orchestra.run.smithi154.stderr: 2023-09-23T03:29:37.770 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126779.log 2023-09-23T03:29:37.771 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.100883.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100883.log.gz 2023-09-23T03:29:37.771 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130615.log 2023-09-23T03:29:37.773 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101009.log 2023-09-23T03:29:37.775 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.126779.log: /var/log/ceph/ceph-client.admin.130615.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.130615.log.gz 2023-09-23T03:29:37.775 INFO:teuthology.orchestra.run.smithi154.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.126779.log.gz 2023-09-23T03:29:37.776 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102537.log 2023-09-23T03:29:37.778 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.101009.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101009.log.gz 2023-09-23T03:29:37.778 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116032.log 2023-09-23T03:29:37.779 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128286.log 2023-09-23T03:29:37.780 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.102537.log: /var/log/ceph/ceph-client.admin.116032.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.102537.log.gz 2023-09-23T03:29:37.781 INFO:teuthology.orchestra.run.smithi154.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.116032.log.gz 2023-09-23T03:29:37.782 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120896.log 2023-09-23T03:29:37.783 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93676.log 2023-09-23T03:29:37.784 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.128286.log: /var/log/ceph/ceph-client.admin.120896.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.128286.log.gz 2023-09-23T03:29:37.785 INFO:teuthology.orchestra.run.smithi154.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.120896.log.gz 2023-09-23T03:29:37.786 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113854.log 2023-09-23T03:29:37.787 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132253.log 2023-09-23T03:29:37.788 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.93676.log: /var/log/ceph/ceph-client.admin.113854.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.93676.log.gz 2023-09-23T03:29:37.790 INFO:teuthology.orchestra.run.smithi154.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.113854.log.gz 2023-09-23T03:29:37.790 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126211.log 2023-09-23T03:29:37.791 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102705.log 2023-09-23T03:29:37.792 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.132253.log: /var/log/ceph/ceph-client.admin.126211.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.132253.log.gz 2023-09-23T03:29:37.793 INFO:teuthology.orchestra.run.smithi154.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.126211.log.gz 2023-09-23T03:29:37.794 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99645.log 2023-09-23T03:29:37.794 INFO:teuthology.orchestra.run.smithi154.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-09-23T03:29:37.796 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.102705.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.102705.log.gz 2023-09-23T03:29:37.796 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99729.log 2023-09-23T03:29:37.797 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.99645.log: gzip 88.8% -5 --verbose -- /var/log/ceph/ceph-client.admin.113896.log 2023-09-23T03:29:37.797 INFO:teuthology.orchestra.run.smithi154.stderr: -- replaced with /var/log/ceph/ceph-client.admin.99645.log.gz 2023-09-23T03:29:37.798 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.99729.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99729.log.gz 2023-09-23T03:29:37.799 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95387.log 2023-09-23T03:29:37.800 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.113896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127720.log 2023-09-23T03:29:37.800 INFO:teuthology.orchestra.run.smithi154.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.113896.log.gz 2023-09-23T03:29:37.801 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.95387.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.95387.log.gz 2023-09-23T03:29:37.802 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122562.log 2023-09-23T03:29:37.803 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.127720.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.127720.log.gz 2023-09-23T03:29:37.803 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119524.log 2023-09-23T03:29:37.805 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.122562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104630.log 2023-09-23T03:29:37.805 INFO:teuthology.orchestra.run.smithi154.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.122562.log.gz 2023-09-23T03:29:37.805 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.119524.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.119524.log.gz 2023-09-23T03:29:37.806 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130827.log 2023-09-23T03:29:37.807 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.104630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128220.log 2023-09-23T03:29:37.808 INFO:teuthology.orchestra.run.smithi154.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.104630.log.gz 2023-09-23T03:29:37.809 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.130827.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.130827.log.gz 2023-09-23T03:29:37.809 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119944.log 2023-09-23T03:29:37.811 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.128220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120517.log 2023-09-23T03:29:37.811 INFO:teuthology.orchestra.run.smithi154.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.128220.log.gz 2023-09-23T03:29:37.812 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.119944.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.119944.log.gz 2023-09-23T03:29:37.813 INFO:teuthology.orchestra.run.smithi154.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.123300.log 2023-09-23T03:29:37.813 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.120517.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.120517.log.gz 2023-09-23T03:29:37.814 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101885.log 2023-09-23T03:29:37.815 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.123300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111733.log 2023-09-23T03:29:37.817 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.101885.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.123940.log 2023-09-23T03:29:37.817 INFO:teuthology.orchestra.run.smithi154.stderr: -- replaced with /var/log/ceph/ceph-client.admin.101885.log.gz 92.3% -- replaced with /var/log/ceph/ceph-client.admin.123300.log.gz 2023-09-23T03:29:37.817 INFO:teuthology.orchestra.run.smithi154.stderr: 2023-09-23T03:29:37.819 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.111733.log: gzip 89.3% -- replaced with /var/log/ceph/ceph-client.admin.111733.log.gz 2023-09-23T03:29:37.819 INFO:teuthology.orchestra.run.smithi154.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.120431.log 2023-09-23T03:29:37.820 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.0.123940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94376.log 2023-09-23T03:29:37.821 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.120431.log: gzip 89.3% -5 --verbose -- /var/log/ceph/ceph-client.admin.100715.log 2023-09-23T03:29:37.822 INFO:teuthology.orchestra.run.smithi154.stderr: -- replaced with /var/log/ceph/ceph-client.admin.120431.log.gz 2023-09-23T03:29:37.823 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.94376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117268.log 2023-09-23T03:29:37.824 INFO:teuthology.orchestra.run.smithi154.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94376.log.gz 2023-09-23T03:29:37.825 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.100715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117465.log 2023-09-23T03:29:37.826 INFO:teuthology.orchestra.run.smithi154.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.100715.log.gz 2023-09-23T03:29:37.827 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.117268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132191.log 2023-09-23T03:29:37.828 INFO:teuthology.orchestra.run.smithi154.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.117268.log.gz 2023-09-23T03:29:37.829 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.117465.log: gzip -5 --verbose 92.3% -- replaced with /var/log/ceph/ceph-client.0.123940.log.gz 2023-09-23T03:29:37.829 INFO:teuthology.orchestra.run.smithi154.stderr: -- /var/log/ceph/ceph-client.admin.107911.log 2023-09-23T03:29:37.831 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.132191.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.117465.log.gz 88.5% -- replaced with /var/log/ceph/ceph-client.admin.132191.log.gz 2023-09-23T03:29:37.831 INFO:teuthology.orchestra.run.smithi154.stderr: 2023-09-23T03:29:37.831 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106317.log 2023-09-23T03:29:37.832 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119566.log 2023-09-23T03:29:37.834 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.106317.log: /var/log/ceph/ceph-client.admin.107911.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122131.log 2023-09-23T03:29:37.834 INFO:teuthology.orchestra.run.smithi154.stderr: -- replaced with /var/log/ceph/ceph-client.admin.106317.log.gz 2023-09-23T03:29:37.835 INFO:teuthology.orchestra.run.smithi154.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107911.log.gz 2023-09-23T03:29:37.835 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.119566.log: gzip -5 88.6% -- replaced with /var/log/ceph/ceph-client.admin.119566.log.gz 2023-09-23T03:29:37.836 INFO:teuthology.orchestra.run.smithi154.stderr: --verbose -- /var/log/ceph/ceph-client.admin.115946.log 2023-09-23T03:29:37.837 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.122131.log: gzip -5 88.2% -- replaced with /var/log/ceph/ceph-client.admin.122131.log.gz 2023-09-23T03:29:37.837 INFO:teuthology.orchestra.run.smithi154.stderr: --verbose -- /var/log/ceph/ceph-client.admin.119692.log 2023-09-23T03:29:37.837 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.115946.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.115946.log.gz 2023-09-23T03:29:37.838 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112453.log 2023-09-23T03:29:37.840 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.119692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104439.log 2023-09-23T03:29:37.840 INFO:teuthology.orchestra.run.smithi154.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.119692.log.gz 2023-09-23T03:29:37.842 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.112453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92148.log 2023-09-23T03:29:37.842 INFO:teuthology.orchestra.run.smithi154.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.112453.log.gz 2023-09-23T03:29:37.842 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.104439.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.104439.log.gz 2023-09-23T03:29:37.843 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107347.log 2023-09-23T03:29:37.844 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.92148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115380.log 2023-09-23T03:29:37.844 INFO:teuthology.orchestra.run.smithi154.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.92148.log.gz 2023-09-23T03:29:37.846 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.107347.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.107347.log.gz 2023-09-23T03:29:37.846 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110671.log 2023-09-23T03:29:37.847 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.115380.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.115380.log.gz 2023-09-23T03:29:37.847 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99231.log 2023-09-23T03:29:37.849 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.110671.log: gzip 89.3% -- replaced with /var/log/ceph/ceph-client.admin.110671.log.gz 2023-09-23T03:29:37.850 INFO:teuthology.orchestra.run.smithi154.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.123000.log 2023-09-23T03:29:37.850 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.99231.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.99231.log.gz 2023-09-23T03:29:37.850 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127678.log 2023-09-23T03:29:37.853 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.123000.log: gzip 90.7% -- replaced with /var/log/ceph/ceph-client.admin.123000.log.gz 2023-09-23T03:29:37.853 INFO:teuthology.orchestra.run.smithi154.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.101677.log 2023-09-23T03:29:37.853 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.127678.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.127678.log.gz 2023-09-23T03:29:37.854 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-09-23T03:29:37.855 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.101677.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101677.log.gz 2023-09-23T03:29:37.856 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101969.log 2023-09-23T03:29:37.858 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122315.log 2023-09-23T03:29:37.860 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.101969.log: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127452.log 2023-09-23T03:29:37.860 INFO:teuthology.orchestra.run.smithi154.stderr: -- replaced with /var/log/ceph/ceph-client.admin.101969.log.gz 2023-09-23T03:29:37.861 INFO:teuthology.orchestra.run.smithi154.stderr:gzip/var/log/ceph/ceph-client.admin.122315.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.100075.log 2023-09-23T03:29:37.862 INFO:teuthology.orchestra.run.smithi154.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.122315.log.gz 2023-09-23T03:29:37.863 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.127452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95386.log 2023-09-23T03:29:37.864 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.100075.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.127452.log.gz 2023-09-23T03:29:37.864 INFO:teuthology.orchestra.run.smithi154.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100075.log.gz 2023-09-23T03:29:37.866 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114802.log 2023-09-23T03:29:37.867 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111000.log 2023-09-23T03:29:37.868 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.95386.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.95386.log.gz 2023-09-23T03:29:37.870 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.114802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131013.log 2023-09-23T03:29:37.871 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.111000.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.114802.log.gz 2023-09-23T03:29:37.872 INFO:teuthology.orchestra.run.smithi154.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.111000.log.gz 2023-09-23T03:29:37.874 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98448.log 2023-09-23T03:29:37.875 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.131013.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.131013.log.gz 2023-09-23T03:29:37.875 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101758.log 2023-09-23T03:29:37.877 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131855.log 2023-09-23T03:29:37.878 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.98448.log: /var/log/ceph/ceph-client.admin.101758.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98448.log.gz 2023-09-23T03:29:37.878 INFO:teuthology.orchestra.run.smithi154.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.101758.log.gz 2023-09-23T03:29:37.879 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121548.log 2023-09-23T03:29:37.881 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.131855.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.131855.log.gz/var/log/ceph/ceph-client.admin.121548.log: 2023-09-23T03:29:37.882 INFO:teuthology.orchestra.run.smithi154.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.121548.log.gz 2023-09-23T03:29:37.897 INFO:teuthology.orchestra.run.smithi154.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-09-23T03:29:37.995 INFO:teuthology.orchestra.run.smithi154.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-09-23T03:29:38.444 INFO:teuthology.orchestra.run.smithi190.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-09-23T03:29:38.472 INFO:teuthology.orchestra.run.smithi190.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-09-23T03:29:38.518 INFO:teuthology.orchestra.run.smithi190.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-09-23T03:29:38.582 INFO:teuthology.orchestra.run.smithi190.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-09-23T03:29:38.587 INFO:teuthology.orchestra.run.smithi190.stderr: 2023-09-23T03:29:38.587 INFO:teuthology.orchestra.run.smithi190.stderr:real 0m1.761s 2023-09-23T03:29:38.587 INFO:teuthology.orchestra.run.smithi190.stderr:user 0m7.337s 2023-09-23T03:29:38.587 INFO:teuthology.orchestra.run.smithi190.stderr:sys 0m0.703s 2023-09-23T03:29:38.904 INFO:teuthology.orchestra.run.smithi154.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-09-23T03:29:39.081 INFO:teuthology.orchestra.run.smithi154.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-09-23T03:29:39.105 INFO:teuthology.orchestra.run.smithi154.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-09-23T03:29:39.336 INFO:teuthology.orchestra.run.smithi154.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-09-23T03:29:39.340 INFO:teuthology.orchestra.run.smithi154.stderr: 2023-09-23T03:29:39.341 INFO:teuthology.orchestra.run.smithi154.stderr:real 0m2.521s 2023-09-23T03:29:39.341 INFO:teuthology.orchestra.run.smithi154.stderr:user 0m7.878s 2023-09-23T03:29:39.341 INFO:teuthology.orchestra.run.smithi154.stderr:sys 0m2.598s 2023-09-23T03:29:39.342 INFO:tasks.ceph:Archiving logs... 2023-09-23T03:29:39.342 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401456/remote/smithi154/log 2023-09-23T03:29:39.343 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-23T03:29:40.918 DEBUG:teuthology.misc:Transferring archived files from smithi190:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401456/remote/smithi190/log 2023-09-23T03:29:40.919 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-23T03:29:41.988 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-09-23T03:29:41.998 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-09-23T03:29:41.998 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-23T03:29:42.007 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-23T03:29:42.168 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-09-23T03:29:42.177 INFO:teuthology.task.clock:Checking final clock skew... 2023-09-23T03:29:42.177 DEBUG:teuthology.orchestra.run.smithi154:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-23T03:29:42.187 DEBUG:teuthology.orchestra.run.smithi190:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-23T03:29:42.237 INFO:teuthology.orchestra.run.smithi154.stderr:bash: line 1: ntpq: command not found 2023-09-23T03:29:42.245 INFO:teuthology.orchestra.run.smithi190.stderr:bash: line 1: ntpq: command not found 2023-09-23T03:29:42.248 INFO:teuthology.orchestra.run.smithi154.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-23T03:29:42.248 INFO:teuthology.orchestra.run.smithi154.stdout:=============================================================================== 2023-09-23T03:29:42.248 INFO:teuthology.orchestra.run.smithi154.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 13 +34us[ +34us] +/- 95ms 2023-09-23T03:29:42.248 INFO:teuthology.orchestra.run.smithi154.stdout:^* hv02.front.sepia.ceph.com 3 6 377 14 -1144us[-1132us] +/- 46ms 2023-09-23T03:29:42.248 INFO:teuthology.orchestra.run.smithi154.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 +1579us[+1579us] +/- 72ms 2023-09-23T03:29:42.248 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T03:29:42.255 INFO:teuthology.orchestra.run.smithi190.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-23T03:29:42.255 INFO:teuthology.orchestra.run.smithi190.stdout:=============================================================================== 2023-09-23T03:29:42.256 INFO:teuthology.orchestra.run.smithi190.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 12 +47us[ +47us] +/- 95ms 2023-09-23T03:29:42.256 INFO:teuthology.orchestra.run.smithi190.stdout:^* hv02.front.sepia.ceph.com 3 6 377 15 -1118us[-1110us] +/- 46ms 2023-09-23T03:29:42.256 INFO:teuthology.orchestra.run.smithi190.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 9 +1594us[+1594us] +/- 72ms 2023-09-23T03:29:42.256 INFO:teuthology.orchestra.run.smithi190.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-09-23T03:29:42.257 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-09-23T03:29:42.267 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-09-23T03:29:42.267 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-09-23T03:29:42.277 DEBUG:teuthology.orchestra.run.smithi154:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-09-23T03:29:42.569 DEBUG:teuthology.orchestra.run.smithi190:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-09-23T03:29:42.836 DEBUG:teuthology.orchestra.run.smithi154:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-23T03:29:42.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T03:29:42.947 DEBUG:teuthology.orchestra.run.smithi190:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-23T03:29:43.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T03:29:43.057 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-09-23T03:29:43.066 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-09-23T03:29:43.077 INFO:teuthology.task.internal:Duration was 1642.748999 seconds 2023-09-23T03:29:43.077 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-09-23T03:29:43.086 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-09-23T03:29:43.086 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-23T03:29:43.096 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-23T03:29:43.253 INFO:teuthology.orchestra.run.smithi154.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-23T03:29:43.269 INFO:teuthology.orchestra.run.smithi190.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-23T03:29:43.557 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-09-23T03:29:43.558 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi154.front.sepia.ceph.com 2023-09-23T03:29:43.558 DEBUG:teuthology.orchestra.run.smithi154:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-23T03:29:43.622 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi190.front.sepia.ceph.com 2023-09-23T03:29:43.623 DEBUG:teuthology.orchestra.run.smithi190:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-23T03:29:43.696 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-09-23T03:29:43.696 DEBUG:teuthology.orchestra.run.smithi154:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-23T03:29:43.705 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-23T03:29:43.801 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-09-23T03:29:43.801 DEBUG:teuthology.orchestra.run.smithi154:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-23T03:29:43.807 DEBUG:teuthology.orchestra.run.smithi190:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-23T03:29:44.015 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-09-23T03:29:44.025 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-09-23T03:29:44.026 DEBUG:teuthology.orchestra.run.smithi154:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-23T03:29:44.128 DEBUG:teuthology.orchestra.run.smithi190:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-23T03:29:44.230 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-09-23T03:29:44.241 DEBUG:teuthology.orchestra.run.smithi154:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-23T03:29:44.250 DEBUG:teuthology.orchestra.run.smithi190:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-23T03:29:44.351 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern = core 2023-09-23T03:29:44.360 INFO:teuthology.orchestra.run.smithi190.stdout:kernel.core_pattern = core 2023-09-23T03:29:44.413 DEBUG:teuthology.orchestra.run.smithi154:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-23T03:29:44.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T03:29:44.463 DEBUG:teuthology.orchestra.run.smithi190:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-23T03:29:44.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-23T03:29:44.521 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-09-23T03:29:44.531 INFO:teuthology.task.internal:Transferring archived files... 2023-09-23T03:29:44.531 DEBUG:teuthology.misc:Transferring archived files from smithi154:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401456/remote/smithi154 2023-09-23T03:29:44.532 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-23T03:29:44.646 DEBUG:teuthology.misc:Transferring archived files from smithi190:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401456/remote/smithi190 2023-09-23T03:29:44.646 DEBUG:teuthology.orchestra.run.smithi190:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-23T03:29:44.752 INFO:teuthology.task.internal:Removing archive directory... 2023-09-23T03:29:44.753 DEBUG:teuthology.orchestra.run.smithi154:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-23T03:29:44.762 DEBUG:teuthology.orchestra.run.smithi190:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-23T03:29:44.820 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-09-23T03:29:44.831 INFO:teuthology.task.internal:Not uploading archives. 2023-09-23T03:29:44.832 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-09-23T03:29:44.841 INFO:teuthology.task.internal:Tidying up after the test... 2023-09-23T03:29:44.841 DEBUG:teuthology.orchestra.run.smithi154:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-23T03:29:44.850 DEBUG:teuthology.orchestra.run.smithi190:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-23T03:29:44.901 INFO:teuthology.orchestra.run.smithi154.stdout: 393864 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 23 03:29 /home/ubuntu/cephtest 2023-09-23T03:29:44.910 INFO:teuthology.orchestra.run.smithi190.stdout: 393249 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 23 03:29 /home/ubuntu/cephtest 2023-09-23T03:29:44.913 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-09-23T03:29:44.923 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-09-23T03:29:44.943 INFO:teuthology.nuke:Checking targets against current locks 2023-09-23T03:29:44.969 DEBUG:teuthology.nuke:shortname: smithi154 2023-09-23T03:29:44.969 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-23T03:29:44.990 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi154.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401456', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-23 02:51:39.834710', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF4iXVXCH7ifGpuKynAenSBvP8YukEyWTkkNqHusRK804qngv2s+hXqAV9tTcZI3im5o9ajvhnPaepqhn47Uq8o='} 2023-09-23T03:29:44.995 DEBUG:teuthology.nuke:shortname: smithi190 2023-09-23T03:29:44.996 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-23T03:29:45.017 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi190.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-09-23_02:50:30-fs-wip-vshankar-testing-20230922.144532-testing-default-smithi/7401456', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-23 02:51:39.833090', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHJ5AsjgG46Z7O7yPa20pbgWscDSfsOFTIUe566npy1eyPjD925g8/ay8Tkm9pywa2BcrKkpubGa3HYMD0SDJXE='} 2023-09-23T03:29:45.034 INFO:teuthology.orchestra.console.smithi154:Power off 2023-09-23T03:29:45.034 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-23T03:29:45.055 DEBUG:teuthology.orchestra.console.smithi154:power off output: Chassis Power Control: Down/Off 2023-09-23T03:29:45.056 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-23T03:29:45.077 INFO:teuthology.orchestra.console.smithi190:Power off 2023-09-23T03:29:45.077 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-23T03:29:45.086 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is on 2023-09-23T03:29:45.098 DEBUG:teuthology.orchestra.console.smithi190:power off output: Chassis Power Control: Down/Off 2023-09-23T03:29:45.098 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-23T03:29:45.121 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on 2023-09-23T03:29:49.087 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-23T03:29:49.122 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-23T03:29:49.198 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is on 2023-09-23T03:29:49.231 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on 2023-09-23T03:29:53.202 DEBUG:teuthology.orchestra.console.smithi154:pexpect command: ipmitool -H smithi154.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-23T03:29:53.232 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-23T03:29:53.313 DEBUG:teuthology.orchestra.console.smithi154:check power output: Chassis Power is off 2023-09-23T03:29:53.344 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is on 2023-09-23T03:29:53.414 INFO:teuthology.orchestra.console.smithi154:Power off completed 2023-09-23T03:29:57.347 DEBUG:teuthology.orchestra.console.smithi190:pexpect command: ipmitool -H smithi190.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-23T03:29:57.456 DEBUG:teuthology.orchestra.console.smithi190:check power output: Chassis Power is off 2023-09-23T03:29:57.557 INFO:teuthology.orchestra.console.smithi190:Power off completed 2023-09-23T03:29:57.662 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/data-scan} duration: 1642.7489988803864 failure_reason: 'Test failure: test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan)' flavor: default mon.a-kernel-sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c mon.b-kernel-sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c owner: scheduled_vshankar@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=def5a513a6ad4cbf831d15971c7dc327 status: fail success: false 2023-09-23T03:29:57.662 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-23T03:29:57.746 INFO:teuthology.run:FAIL